2023-10-06T03:13:57.189 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T03:13:57.189 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --archive /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199 --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199/orig.config.yaml 2023-10-06T03:13:57.235 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T03:13:57.320 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199 branch: wip-yuri-testing-2023-10-05-0730-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: yweinste@redhat.com first_in_suite: false job_id: '7412199' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri-testing-2023-10-05-0730-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-10-05-0730-quincy sha1: f1e315a9732d36c05ec316678e2397dc58561c7f owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7519 sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-10-05-0730-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f1e315a9732d36c05ec316678e2397dc58561c7f targets: smithi055.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGqd+edrJM33HqoVOXuByo5UCZMCd9lwtqSdRynCQEwqmfYkHm6qMqUcBj6r1uCSm4Q/dDGu1v0wzAQCwCu2NcM= smithi163.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbdyRKYMgXgX8qIaeWtbeZtloToSZebR7zRlsyDZRCmJ9QOXny++FCvXp1BImVGFJHlIh2orVhQWKxCENrLDWI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:52:50 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T03:13:57.320 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa; will attempt to use it 2023-10-06T03:13:57.321 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks 2023-10-06T03:13:57.321 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T03:13:57.323 INFO:teuthology.task.internal:Checking packages... 2023-10-06T03:13:57.348 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2' 2023-10-06T03:13:57.349 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T03:13:57.349 INFO:teuthology.packaging:ref: None 2023-10-06T03:13:57.349 INFO:teuthology.packaging:tag: None 2023-10-06T03:13:57.349 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-10-05-0730-quincy 2023-10-06T03:13:57.349 INFO:teuthology.packaging:sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T03:13:57.349 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-10-05-0730-quincy 2023-10-06T03:13:57.563 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1296.gf1e315a9 2023-10-06T03:13:57.563 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T03:13:57.571 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T03:13:57.571 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T03:13:57.581 INFO:teuthology.task.internal:Saving configuration 2023-10-06T03:13:57.599 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T03:13:57.606 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T03:13:57.623 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 03:08:09.668313', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGqd+edrJM33HqoVOXuByo5UCZMCd9lwtqSdRynCQEwqmfYkHm6qMqUcBj6r1uCSm4Q/dDGu1v0wzAQCwCu2NcM='} 2023-10-06T03:13:57.647 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 03:08:09.666339', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbdyRKYMgXgX8qIaeWtbeZtloToSZebR7zRlsyDZRCmJ9QOXny++FCvXp1BImVGFJHlIh2orVhQWKxCENrLDWI='} 2023-10-06T03:13:57.647 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T03:13:57.653 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-06T03:13:57.653 INFO:teuthology.task.internal:roles: ubuntu@smithi163.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-06T03:13:57.653 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T03:13:57.722 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f501ed33b80>, signals=[15]) 2023-10-06T03:13:57.723 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T03:13:57.730 INFO:teuthology.task.internal:Opening connections... 2023-10-06T03:13:57.730 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T03:13:57.733 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:13:57.802 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi163.front.sepia.ceph.com 2023-10-06T03:13:57.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:13:57.869 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T03:13:57.876 DEBUG:teuthology.orchestra.run.smithi055:> uname -m 2023-10-06T03:13:57.895 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2023-10-06T03:13:57.895 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2023-10-06T03:13:57.952 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T03:13:57.952 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T03:13:57.952 INFO:teuthology.orchestra.run.smithi055.stdout:ID="rhel" 2023-10-06T03:13:57.952 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="fedora" 2023-10-06T03:13:57.952 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="8.4" 2023-10-06T03:13:57.952 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el8" 2023-10-06T03:13:57.952 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T03:13:57.952 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2023-10-06T03:13:57.952 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T03:13:57.952 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T03:13:57.953 INFO:teuthology.orchestra.run.smithi055.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T03:13:57.953 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T03:13:57.953 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:13:57.953 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T03:13:57.953 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T03:13:57.953 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T03:13:57.953 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T03:13:57.954 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2023-10-06T03:13:57.978 DEBUG:teuthology.orchestra.run.smithi163:> uname -m 2023-10-06T03:13:57.995 INFO:teuthology.orchestra.run.smithi163.stdout:x86_64 2023-10-06T03:13:57.995 DEBUG:teuthology.orchestra.run.smithi163:> cat /etc/os-release 2023-10-06T03:13:58.051 INFO:teuthology.orchestra.run.smithi163.stdout:NAME="Red Hat Enterprise Linux" 2023-10-06T03:13:58.051 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION="8.4 (Ootpa)" 2023-10-06T03:13:58.051 INFO:teuthology.orchestra.run.smithi163.stdout:ID="rhel" 2023-10-06T03:13:58.051 INFO:teuthology.orchestra.run.smithi163.stdout:ID_LIKE="fedora" 2023-10-06T03:13:58.052 INFO:teuthology.orchestra.run.smithi163.stdout:VERSION_ID="8.4" 2023-10-06T03:13:58.052 INFO:teuthology.orchestra.run.smithi163.stdout:PLATFORM_ID="platform:el8" 2023-10-06T03:13:58.052 INFO:teuthology.orchestra.run.smithi163.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-06T03:13:58.052 INFO:teuthology.orchestra.run.smithi163.stdout:ANSI_COLOR="0;31" 2023-10-06T03:13:58.052 INFO:teuthology.orchestra.run.smithi163.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-06T03:13:58.052 INFO:teuthology.orchestra.run.smithi163.stdout:HOME_URL="https://www.redhat.com/" 2023-10-06T03:13:58.052 INFO:teuthology.orchestra.run.smithi163.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-06T03:13:58.052 INFO:teuthology.orchestra.run.smithi163.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T03:13:58.052 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:13:58.052 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T03:13:58.053 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-06T03:13:58.053 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-06T03:13:58.053 INFO:teuthology.orchestra.run.smithi163.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-06T03:13:58.053 INFO:teuthology.lock.ops:Updating smithi163.front.sepia.ceph.com on lock server 2023-10-06T03:13:58.077 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T03:13:58.087 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T03:13:58.092 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T03:13:58.093 DEBUG:teuthology.orchestra.run.smithi055:> test '!' -e /home/ubuntu/cephtest 2023-10-06T03:13:58.095 DEBUG:teuthology.orchestra.run.smithi163:> test '!' -e /home/ubuntu/cephtest 2023-10-06T03:13:58.111 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T03:13:58.117 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T03:13:58.117 DEBUG:teuthology.orchestra.run.smithi055:> test -z $(ls -A /var/lib/ceph) 2023-10-06T03:13:58.152 DEBUG:teuthology.orchestra.run.smithi163:> test -z $(ls -A /var/lib/ceph) 2023-10-06T03:13:58.168 INFO:teuthology.orchestra.run.smithi055.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T03:13:58.169 INFO:teuthology.orchestra.run.smithi163.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-06T03:13:58.169 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T03:13:58.220 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T03:13:58.292 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-06T03:13:58.292 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-10-06T03:13:58.293 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-10-06T03:13:58.293 DEBUG:teuthology.orchestra.run.smithi055:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T03:13:58.294 DEBUG:teuthology.orchestra.run.smithi163:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T03:13:58.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:13:58.310 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-06T03:13:58.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:13:58.314 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-06T03:13:58.470 DEBUG:teuthology.task.kernel:sha1 for client.0 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T03:13:58.470 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T03:13:58.470 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2023-10-06T03:13:58.489 INFO:teuthology.orchestra.run.smithi055.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T03:13:58.490 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-305.25.1.el8_4.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T03:13:58.490 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T03:13:58.490 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum upgrade -y linux-firmware 2023-10-06T03:13:58.552 DEBUG:teuthology.task.kernel:sha1 for client.1 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T03:13:58.552 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-06T03:13:58.553 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2023-10-06T03:13:58.569 INFO:teuthology.orchestra.run.smithi163.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-06T03:13:58.569 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-305.25.1.el8_4.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T03:13:58.570 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-06T03:13:58.570 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum upgrade -y linux-firmware 2023-10-06T03:13:59.992 INFO:teuthology.orchestra.run.smithi055.stdout:Updating Subscription Management repositories. 2023-10-06T03:13:59.992 INFO:teuthology.orchestra.run.smithi055.stdout:Unable to read consumer identity 2023-10-06T03:14:00.028 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2023-10-06T03:14:00.029 INFO:teuthology.orchestra.run.smithi163.stdout:Unable to read consumer identity 2023-10-06T03:14:01.353 INFO:teuthology.orchestra.run.smithi163.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-10-06T03:14:01.408 INFO:teuthology.orchestra.run.smithi055.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-10-06T03:14:01.859 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 34 MB/s | 16 MB 00:00 2023-10-06T03:14:02.366 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-10-06T03:14:05.209 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 4.7 MB/s | 24 kB 00:00 2023-10-06T03:14:05.887 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 4.4 MB/s | 24 kB 00:00 2023-10-06T03:14:06.137 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2023-10-06T03:14:06.138 INFO:teuthology.orchestra.run.smithi163.stdout:Nothing to do. 2023-10-06T03:14:06.138 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2023-10-06T03:14:06.245 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.1... 2023-10-06T03:14:06.246 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T03:14:06.399 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-06T03:14:06.400 DEBUG:teuthology.orchestra.run.smithi163:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-06T03:14:06.402 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi163.front.sepia.ceph.com to complete... 2023-10-06T03:14:06.796 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-10-06T03:14:06.797 INFO:teuthology.orchestra.run.smithi055.stdout:Nothing to do. 2023-10-06T03:14:06.797 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-10-06T03:14:06.903 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-06T03:14:06.903 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-06T03:14:07.051 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-06T03:14:07.051 DEBUG:teuthology.orchestra.run.smithi055:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-06T03:14:07.053 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi055.front.sepia.ceph.com to complete... 2023-10-06T03:14:25.105 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06 03:14:25 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-06T03:14:25.106 INFO:teuthology.orchestra.run.smithi163.stderr:FINISHED --2023-10-06 03:14:25-- 2023-10-06T03:14:25.106 INFO:teuthology.orchestra.run.smithi163.stderr:Total wall clock time: 19s 2023-10-06T03:14:25.106 INFO:teuthology.orchestra.run.smithi163.stderr:Downloaded: 1 files, 363M in 18s (19.8 MB/s) 2023-10-06T03:14:25.108 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.1... 2023-10-06T03:14:25.108 DEBUG:teuthology.orchestra.run.smithi163:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-06T03:14:26.826 INFO:teuthology.orchestra.run.smithi163.stdout:Verifying... ######################################## 2023-10-06T03:14:27.208 INFO:teuthology.orchestra.run.smithi163.stdout:Preparing... ######################################## 2023-10-06T03:14:27.346 INFO:teuthology.orchestra.run.smithi163.stdout:Updating / installing... 2023-10-06T03:14:32.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06 03:14:32 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-06T03:14:32.674 INFO:teuthology.orchestra.run.smithi055.stderr:FINISHED --2023-10-06 03:14:32-- 2023-10-06T03:14:32.674 INFO:teuthology.orchestra.run.smithi055.stderr:Total wall clock time: 26s 2023-10-06T03:14:32.674 INFO:teuthology.orchestra.run.smithi055.stderr:Downloaded: 1 files, 363M in 25s (14.4 MB/s) 2023-10-06T03:14:32.675 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-06T03:14:32.675 DEBUG:teuthology.orchestra.run.smithi055:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-06T03:14:34.470 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying... ######################################## 2023-10-06T03:14:34.827 INFO:teuthology.orchestra.run.smithi055.stdout:Preparing... ######################################## 2023-10-06T03:14:34.970 INFO:teuthology.orchestra.run.smithi055.stdout:Updating / installing... 2023-10-06T03:15:05.488 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-06T03:15:15.609 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-06T03:15:30.725 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi163.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-06T03:15:30.725 DEBUG:teuthology.orchestra.run.smithi163:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-06T03:15:30.779 INFO:teuthology.orchestra.run.smithi163.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:30.780 INFO:teuthology.orchestra.run.smithi163.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:30.780 INFO:teuthology.orchestra.run.smithi163.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:30.780 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:30.780 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-06T03:15:30.780 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-06T03:15:30.780 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-06T03:15:30.780 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-06T03:15:30.780 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T03:15:30.780 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T03:15:30.780 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T03:15:30.780 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T03:15:30.781 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T03:15:30.781 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T03:15:30.781 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T03:15:30.781 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-06T03:15:30.781 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T03:15:30.781 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T03:15:30.781 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T03:15:30.781 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T03:15:30.781 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T03:15:30.781 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T03:15:30.781 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T03:15:30.781 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T03:15:30.782 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T03:15:30.782 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T03:15:30.782 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T03:15:30.782 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T03:15:30.782 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-06T03:15:30.782 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-06T03:15:30.782 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-06T03:15:30.782 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T03:15:30.782 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-06T03:15:30.782 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T03:15:30.782 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T03:15:30.783 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T03:15:30.783 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-06T03:15:30.783 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T03:15:30.783 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T03:15:30.783 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T03:15:30.783 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-06T03:15:30.783 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T03:15:30.783 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T03:15:30.783 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T03:15:30.783 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T03:15:30.783 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T03:15:30.783 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T03:15:30.784 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T03:15:30.784 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T03:15:30.784 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T03:15:30.784 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T03:15:30.784 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T03:15:30.784 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T03:15:30.784 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T03:15:30.784 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T03:15:30.784 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T03:15:30.784 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T03:15:30.784 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T03:15:30.784 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T03:15:30.785 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T03:15:30.785 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T03:15:30.785 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T03:15:30.785 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T03:15:30.785 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T03:15:30.785 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T03:15:30.785 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T03:15:30.785 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T03:15:30.785 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T03:15:30.785 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T03:15:30.785 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T03:15:30.786 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T03:15:30.786 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T03:15:30.786 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T03:15:30.786 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T03:15:30.786 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T03:15:30.786 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T03:15:30.786 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T03:15:30.786 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T03:15:30.786 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T03:15:30.786 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T03:15:30.786 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T03:15:30.786 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T03:15:30.787 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T03:15:30.787 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T03:15:30.787 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T03:15:30.787 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T03:15:30.787 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-06T03:15:30.787 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-06T03:15:30.787 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T03:15:30.787 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T03:15:30.787 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-06T03:15:30.787 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T03:15:30.787 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T03:15:30.787 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T03:15:30.788 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T03:15:30.788 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T03:15:30.788 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-06T03:15:30.788 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T03:15:30.788 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T03:15:30.788 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-06T03:15:30.788 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T03:15:30.788 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T03:15:30.788 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T03:15:30.788 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T03:15:30.788 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T03:15:30.788 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T03:15:30.789 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T03:15:30.789 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T03:15:30.789 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T03:15:30.789 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T03:15:30.789 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T03:15:30.789 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T03:15:30.789 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T03:15:30.789 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T03:15:30.789 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T03:15:30.789 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T03:15:30.789 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T03:15:30.789 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T03:15:30.790 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T03:15:30.790 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T03:15:30.790 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T03:15:30.790 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T03:15:30.790 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T03:15:30.790 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T03:15:30.790 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T03:15:30.790 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T03:15:30.790 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T03:15:30.790 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T03:15:30.790 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T03:15:30.791 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T03:15:30.791 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T03:15:30.791 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T03:15:30.791 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T03:15:30.791 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T03:15:30.791 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T03:15:30.791 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-06T03:15:30.791 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T03:15:30.791 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T03:15:30.791 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T03:15:30.791 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T03:15:30.791 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T03:15:30.792 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T03:15:30.792 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T03:15:30.792 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T03:15:30.792 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T03:15:30.792 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T03:15:30.792 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T03:15:30.792 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T03:15:30.792 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T03:15:30.792 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T03:15:30.792 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T03:15:30.792 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T03:15:30.792 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T03:15:30.792 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-06T03:15:30.793 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T03:15:30.793 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-06T03:15:30.793 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-06T03:15:30.793 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-06T03:15:30.793 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-06T03:15:30.793 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-06T03:15:30.793 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T03:15:30.793 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-06T03:15:30.793 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-06T03:15:30.793 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T03:15:30.793 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-06T03:15:30.793 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-06T03:15:30.794 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T03:15:30.794 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T03:15:30.794 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-06T03:15:30.794 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-06T03:15:30.794 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T03:15:30.794 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T03:15:30.794 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-06T03:15:30.794 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T03:15:30.794 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T03:15:30.794 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T03:15:30.794 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-06T03:15:30.794 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-06T03:15:30.795 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T03:15:30.795 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-06T03:15:30.795 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-06T03:15:30.795 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T03:15:30.795 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-06T03:15:30.795 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T03:15:30.795 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-06T03:15:30.795 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T03:15:30.795 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T03:15:30.795 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T03:15:30.795 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T03:15:30.795 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T03:15:30.796 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-06T03:15:30.796 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T03:15:30.796 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T03:15:30.796 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T03:15:30.796 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T03:15:30.796 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T03:15:30.796 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T03:15:30.796 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T03:15:30.796 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-06T03:15:30.796 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T03:15:30.796 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T03:15:30.797 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T03:15:30.797 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-06T03:15:30.797 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-06T03:15:30.797 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T03:15:30.797 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T03:15:30.797 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T03:15:30.797 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-06T03:15:30.797 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-06T03:15:30.797 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-06T03:15:30.797 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-06T03:15:30.797 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-06T03:15:30.797 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-06T03:15:30.798 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T03:15:30.798 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T03:15:30.798 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-06T03:15:30.798 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-06T03:15:30.798 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T03:15:30.798 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-06T03:15:30.798 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-06T03:15:30.798 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T03:15:30.798 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T03:15:30.798 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-06T03:15:30.798 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-06T03:15:30.798 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-06T03:15:30.799 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T03:15:30.799 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-06T03:15:30.799 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T03:15:30.799 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T03:15:30.799 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T03:15:30.799 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-06T03:15:30.799 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T03:15:30.799 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-06T03:15:30.799 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T03:15:30.799 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-06T03:15:30.799 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-06T03:15:30.799 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-06T03:15:30.800 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-06T03:15:30.800 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T03:15:30.800 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-06T03:15:30.800 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T03:15:30.800 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-06T03:15:30.800 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T03:15:30.800 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T03:15:30.800 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T03:15:30.800 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T03:15:30.800 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T03:15:30.800 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T03:15:30.800 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T03:15:30.801 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T03:15:30.801 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T03:15:30.801 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T03:15:30.801 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T03:15:30.801 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T03:15:30.801 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T03:15:30.801 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-06T03:15:30.801 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-06T03:15:30.801 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T03:15:30.801 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-06T03:15:30.801 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T03:15:30.801 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T03:15:30.802 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-06T03:15:30.802 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T03:15:30.802 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T03:15:30.802 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T03:15:30.802 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T03:15:30.802 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T03:15:30.802 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T03:15:30.802 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T03:15:30.802 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T03:15:30.802 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T03:15:30.802 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T03:15:30.803 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T03:15:30.803 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T03:15:30.803 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T03:15:30.803 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T03:15:30.803 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T03:15:30.803 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T03:15:30.803 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T03:15:30.803 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T03:15:30.803 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T03:15:30.803 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T03:15:30.803 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T03:15:30.803 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T03:15:30.803 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T03:15:30.804 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T03:15:30.804 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T03:15:30.804 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T03:15:30.804 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T03:15:30.804 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T03:15:30.804 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T03:15:30.804 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T03:15:30.804 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T03:15:30.804 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T03:15:30.804 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T03:15:30.804 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T03:15:30.805 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T03:15:30.805 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T03:15:30.805 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T03:15:30.805 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T03:15:30.805 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T03:15:30.805 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T03:15:30.805 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T03:15:30.805 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T03:15:30.805 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T03:15:30.805 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T03:15:30.805 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T03:15:30.806 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T03:15:30.806 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T03:15:30.806 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T03:15:30.806 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T03:15:30.806 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T03:15:30.806 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T03:15:30.806 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T03:15:30.806 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T03:15:30.806 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T03:15:30.806 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T03:15:30.806 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T03:15:30.806 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T03:15:30.807 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T03:15:30.807 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T03:15:30.807 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T03:15:30.807 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T03:15:30.807 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T03:15:30.807 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T03:15:30.807 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T03:15:30.807 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-06T03:15:30.807 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T03:15:30.807 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T03:15:30.807 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T03:15:30.807 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-06T03:15:30.808 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T03:15:30.808 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T03:15:30.808 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T03:15:30.808 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T03:15:30.808 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T03:15:30.808 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T03:15:30.808 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T03:15:30.808 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T03:15:30.808 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T03:15:30.808 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T03:15:30.808 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T03:15:30.809 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T03:15:30.809 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T03:15:30.809 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T03:15:30.809 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T03:15:30.809 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T03:15:30.809 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T03:15:30.809 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T03:15:30.809 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T03:15:30.809 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T03:15:30.809 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T03:15:30.810 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T03:15:30.810 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T03:15:30.810 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T03:15:30.810 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T03:15:30.810 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T03:15:30.810 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T03:15:30.810 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T03:15:30.810 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T03:15:30.810 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T03:15:30.810 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T03:15:30.811 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T03:15:30.811 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T03:15:30.811 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T03:15:30.811 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T03:15:30.811 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T03:15:30.811 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T03:15:30.811 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T03:15:30.811 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T03:15:30.811 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T03:15:30.811 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T03:15:30.811 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T03:15:30.811 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T03:15:30.812 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T03:15:30.812 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T03:15:30.812 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T03:15:30.812 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T03:15:30.812 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T03:15:30.812 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T03:15:30.812 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T03:15:30.812 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T03:15:30.812 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T03:15:30.812 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T03:15:30.812 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T03:15:30.813 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T03:15:30.813 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T03:15:30.813 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T03:15:30.813 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T03:15:30.813 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T03:15:30.813 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T03:15:30.813 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T03:15:30.813 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T03:15:30.813 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T03:15:30.813 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T03:15:30.814 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T03:15:30.814 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T03:15:30.814 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T03:15:30.814 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T03:15:30.814 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T03:15:30.814 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T03:15:30.814 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T03:15:30.814 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T03:15:30.814 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T03:15:30.814 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T03:15:30.814 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T03:15:30.815 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T03:15:30.815 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T03:15:30.815 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T03:15:30.815 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T03:15:30.815 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T03:15:30.815 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T03:15:30.815 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T03:15:30.815 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T03:15:30.815 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T03:15:30.815 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T03:15:30.815 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T03:15:30.815 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T03:15:30.816 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T03:15:30.816 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-06T03:15:30.816 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T03:15:30.816 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T03:15:30.816 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T03:15:30.816 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T03:15:30.816 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T03:15:30.816 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T03:15:30.816 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T03:15:30.816 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T03:15:30.816 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T03:15:30.816 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T03:15:30.816 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T03:15:30.817 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T03:15:30.817 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T03:15:30.817 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T03:15:30.817 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T03:15:30.817 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T03:15:30.817 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T03:15:30.817 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T03:15:30.817 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T03:15:30.817 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T03:15:30.817 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T03:15:30.817 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T03:15:30.817 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T03:15:30.818 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T03:15:30.818 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T03:15:30.818 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T03:15:30.818 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T03:15:30.818 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T03:15:30.818 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T03:15:30.818 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T03:15:30.818 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T03:15:30.818 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T03:15:30.818 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T03:15:30.818 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T03:15:30.818 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T03:15:30.819 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T03:15:30.819 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T03:15:30.819 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T03:15:30.819 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T03:15:30.819 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T03:15:30.819 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T03:15:30.819 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-06T03:15:30.819 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-06T03:15:30.819 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T03:15:30.819 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T03:15:30.819 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-06T03:15:30.819 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T03:15:30.819 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T03:15:30.820 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T03:15:30.820 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T03:15:30.820 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T03:15:30.820 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T03:15:30.820 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T03:15:30.820 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T03:15:30.820 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T03:15:30.820 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T03:15:30.820 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T03:15:30.820 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T03:15:30.820 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T03:15:30.820 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T03:15:30.821 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T03:15:30.821 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T03:15:30.821 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T03:15:30.821 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T03:15:30.821 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T03:15:30.821 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T03:15:30.821 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T03:15:30.821 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T03:15:30.821 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T03:15:30.821 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-06T03:15:30.821 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T03:15:30.821 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-06T03:15:30.822 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T03:15:30.822 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T03:15:30.822 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-06T03:15:30.822 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-06T03:15:30.822 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T03:15:30.822 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T03:15:30.822 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T03:15:30.822 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T03:15:30.822 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T03:15:30.822 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T03:15:30.822 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T03:15:30.822 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-06T03:15:30.823 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-06T03:15:30.823 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T03:15:30.823 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-06T03:15:30.823 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T03:15:30.823 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-06T03:15:30.823 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T03:15:30.823 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-06T03:15:30.823 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T03:15:30.823 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-06T03:15:30.823 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-06T03:15:30.823 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T03:15:30.823 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-06T03:15:30.824 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T03:15:30.824 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-06T03:15:30.824 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T03:15:30.824 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-06T03:15:30.824 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T03:15:30.824 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-06T03:15:30.824 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T03:15:30.824 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-06T03:15:30.824 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-06T03:15:30.824 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T03:15:30.824 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T03:15:30.825 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T03:15:30.825 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T03:15:30.825 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T03:15:30.825 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T03:15:30.825 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-06T03:15:30.825 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T03:15:30.825 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-06T03:15:30.825 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T03:15:30.825 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T03:15:30.825 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T03:15:30.825 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T03:15:30.825 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T03:15:30.826 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T03:15:30.826 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T03:15:30.826 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T03:15:30.826 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-06T03:15:30.826 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T03:15:30.826 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-06T03:15:30.826 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T03:15:30.826 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T03:15:30.826 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T03:15:30.826 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T03:15:30.826 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T03:15:30.827 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-06T03:15:30.827 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-06T03:15:30.827 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T03:15:30.827 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-06T03:15:30.827 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T03:15:30.827 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T03:15:30.827 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T03:15:30.827 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T03:15:30.827 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-06T03:15:30.827 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T03:15:30.827 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T03:15:30.827 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T03:15:30.828 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T03:15:30.828 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T03:15:30.828 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T03:15:30.828 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T03:15:30.828 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T03:15:30.828 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-06T03:15:30.828 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-06T03:15:30.828 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T03:15:30.828 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T03:15:30.828 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T03:15:30.828 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T03:15:30.828 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T03:15:30.828 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T03:15:30.829 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T03:15:30.829 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T03:15:30.829 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T03:15:30.829 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T03:15:30.829 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T03:15:30.829 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T03:15:30.829 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T03:15:30.829 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T03:15:30.829 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T03:15:30.829 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T03:15:30.830 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T03:15:30.830 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T03:15:30.830 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T03:15:30.830 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-06T03:15:30.830 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T03:15:30.830 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T03:15:30.830 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T03:15:30.830 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T03:15:30.830 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T03:15:30.830 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-06T03:15:30.830 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-06T03:15:30.831 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T03:15:30.831 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T03:15:30.831 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T03:15:30.831 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T03:15:30.831 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T03:15:30.831 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T03:15:30.831 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T03:15:30.831 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-06T03:15:30.831 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T03:15:30.831 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T03:15:30.831 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-06T03:15:30.831 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T03:15:30.832 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T03:15:30.832 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T03:15:30.832 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T03:15:30.832 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-06T03:15:30.832 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-06T03:15:30.832 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T03:15:30.832 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T03:15:30.832 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T03:15:30.832 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T03:15:30.832 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-06T03:15:30.832 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T03:15:30.832 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-06T03:15:30.833 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T03:15:30.833 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T03:15:30.833 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T03:15:30.833 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T03:15:30.833 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T03:15:30.833 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T03:15:30.833 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T03:15:30.833 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T03:15:30.833 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-06T03:15:30.833 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T03:15:30.833 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T03:15:30.833 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-06T03:15:30.834 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T03:15:30.834 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T03:15:30.834 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T03:15:30.834 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-06T03:15:30.834 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-06T03:15:30.834 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-06T03:15:30.834 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-06T03:15:30.834 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-06T03:15:30.834 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-06T03:15:30.834 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T03:15:30.834 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T03:15:30.835 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T03:15:30.835 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T03:15:30.835 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-06T03:15:30.835 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T03:15:30.835 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-06T03:15:30.835 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-06T03:15:30.835 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T03:15:30.835 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-06T03:15:30.835 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-06T03:15:30.835 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T03:15:30.835 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T03:15:30.835 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T03:15:30.836 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-06T03:15:30.836 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T03:15:30.836 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T03:15:30.836 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T03:15:30.836 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-06T03:15:30.836 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-06T03:15:30.836 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T03:15:30.836 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T03:15:30.836 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T03:15:30.836 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T03:15:30.836 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-06T03:15:30.836 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T03:15:30.837 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T03:15:30.837 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T03:15:30.837 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T03:15:30.837 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-06T03:15:30.837 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T03:15:30.837 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-06T03:15:30.837 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T03:15:30.837 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-06T03:15:30.837 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T03:15:30.837 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-06T03:15:30.837 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T03:15:30.837 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-06T03:15:30.838 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T03:15:30.838 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T03:15:30.838 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T03:15:30.838 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-06T03:15:30.838 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-06T03:15:30.838 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T03:15:30.838 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-06T03:15:30.838 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T03:15:30.838 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-06T03:15:30.838 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T03:15:30.838 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-06T03:15:30.839 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T03:15:30.839 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T03:15:30.839 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-06T03:15:30.839 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T03:15:30.839 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T03:15:30.839 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-06T03:15:30.839 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T03:15:30.839 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T03:15:30.839 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-06T03:15:30.839 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T03:15:30.839 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T03:15:30.840 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-06T03:15:30.840 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-06T03:15:30.840 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T03:15:30.840 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-06T03:15:30.840 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-06T03:15:30.840 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T03:15:30.840 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-06T03:15:30.840 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T03:15:30.840 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-06T03:15:30.840 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T03:15:30.840 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-06T03:15:30.840 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T03:15:30.841 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-06T03:15:30.841 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T03:15:30.841 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-06T03:15:30.841 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-06T03:15:30.841 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T03:15:30.841 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-06T03:15:30.841 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-06T03:15:30.841 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T03:15:30.841 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T03:15:30.841 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T03:15:30.841 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T03:15:30.842 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T03:15:30.842 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T03:15:30.842 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-06T03:15:30.842 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T03:15:30.842 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T03:15:30.842 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T03:15:30.842 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-06T03:15:30.842 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-06T03:15:30.842 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T03:15:30.842 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T03:15:30.842 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T03:15:30.842 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-06T03:15:30.842 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T03:15:30.843 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-06T03:15:30.843 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T03:15:30.843 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-06T03:15:30.843 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T03:15:30.843 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-06T03:15:30.843 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T03:15:30.843 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-06T03:15:30.843 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T03:15:30.843 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-06T03:15:30.843 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T03:15:30.843 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-06T03:15:30.844 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T03:15:30.844 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-06T03:15:30.844 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T03:15:30.844 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-06T03:15:30.844 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T03:15:30.844 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-06T03:15:30.844 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T03:15:30.844 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T03:15:30.844 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-06T03:15:30.844 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T03:15:30.844 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T03:15:30.844 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T03:15:30.845 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-06T03:15:30.845 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-06T03:15:30.845 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T03:15:30.845 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T03:15:30.845 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-06T03:15:30.845 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-06T03:15:30.845 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T03:15:30.845 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-06T03:15:30.845 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T03:15:30.845 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-06T03:15:30.846 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-06T03:15:30.846 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T03:15:30.846 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-06T03:15:30.846 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T03:15:30.846 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T03:15:30.846 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-06T03:15:30.846 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T03:15:30.846 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-06T03:15:30.846 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T03:15:30.846 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-06T03:15:30.846 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T03:15:30.847 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T03:15:30.847 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-06T03:15:30.847 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-06T03:15:30.847 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T03:15:30.847 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T03:15:30.847 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-06T03:15:30.847 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T03:15:30.847 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-06T03:15:30.847 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T03:15:30.848 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-06T03:15:30.848 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T03:15:30.848 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T03:15:30.848 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T03:15:30.848 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T03:15:30.848 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-06T03:15:30.848 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-06T03:15:30.848 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-06T03:15:30.848 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T03:15:30.848 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-06T03:15:30.848 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T03:15:30.849 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-06T03:15:30.849 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T03:15:30.849 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T03:15:30.849 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-06T03:15:30.849 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T03:15:30.849 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T03:15:30.849 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T03:15:30.849 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T03:15:30.849 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-06T03:15:30.849 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-06T03:15:30.849 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T03:15:30.850 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T03:15:30.850 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-06T03:15:30.850 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T03:15:30.850 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T03:15:30.850 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T03:15:30.850 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T03:15:30.850 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-06T03:15:30.850 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T03:15:30.850 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-06T03:15:30.850 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T03:15:30.850 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-06T03:15:30.850 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T03:15:30.851 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T03:15:30.851 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-06T03:15:30.851 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T03:15:30.851 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T03:15:30.851 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-06T03:15:30.851 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T03:15:30.851 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T03:15:30.851 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-06T03:15:30.851 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T03:15:30.851 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-06T03:15:30.851 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T03:15:30.852 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T03:15:30.852 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-06T03:15:30.852 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T03:15:30.852 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T03:15:30.852 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T03:15:30.852 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-06T03:15:30.852 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T03:15:30.852 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T03:15:30.852 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T03:15:30.852 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T03:15:30.852 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T03:15:30.853 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-06T03:15:30.853 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T03:15:30.853 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T03:15:30.853 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-06T03:15:30.853 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T03:15:30.853 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T03:15:30.853 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-06T03:15:30.853 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T03:15:30.853 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T03:15:30.853 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-06T03:15:30.853 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T03:15:30.853 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T03:15:30.854 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T03:15:30.854 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T03:15:30.854 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T03:15:30.854 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-06T03:15:30.854 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T03:15:30.854 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T03:15:30.854 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T03:15:30.858 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T03:15:30.858 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T03:15:30.858 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T03:15:30.858 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T03:15:30.858 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T03:15:30.858 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T03:15:30.858 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T03:15:30.859 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T03:15:30.859 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T03:15:30.859 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-06T03:15:30.859 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T03:15:30.859 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T03:15:30.859 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T03:15:30.859 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-06T03:15:30.859 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T03:15:30.859 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T03:15:30.859 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T03:15:30.859 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T03:15:30.860 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T03:15:30.860 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-06T03:15:30.860 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-06T03:15:30.860 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T03:15:30.860 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T03:15:30.860 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T03:15:30.860 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-06T03:15:30.860 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T03:15:30.860 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T03:15:30.860 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T03:15:30.860 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T03:15:30.861 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T03:15:30.861 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T03:15:30.861 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-06T03:15:30.861 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T03:15:30.861 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-06T03:15:30.861 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-06T03:15:30.861 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T03:15:30.861 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-06T03:15:30.861 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T03:15:30.861 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-06T03:15:30.861 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T03:15:30.861 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-06T03:15:30.861 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-06T03:15:30.862 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T03:15:30.862 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-06T03:15:30.862 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-06T03:15:30.862 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T03:15:30.862 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T03:15:30.862 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-06T03:15:30.862 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T03:15:30.862 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T03:15:30.862 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T03:15:30.862 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T03:15:30.862 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-06T03:15:30.863 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-06T03:15:30.863 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T03:15:30.863 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-06T03:15:30.863 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-06T03:15:30.863 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T03:15:30.863 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-06T03:15:30.863 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T03:15:30.863 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T03:15:30.863 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T03:15:30.863 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-06T03:15:30.863 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-06T03:15:30.863 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T03:15:30.864 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-06T03:15:30.864 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-06T03:15:30.864 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T03:15:30.864 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T03:15:30.864 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T03:15:30.864 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T03:15:30.864 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T03:15:30.864 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T03:15:30.864 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T03:15:30.864 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T03:15:30.864 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T03:15:30.864 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T03:15:30.865 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T03:15:30.865 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T03:15:30.865 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-06T03:15:30.865 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T03:15:30.865 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T03:15:30.865 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T03:15:30.865 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T03:15:30.865 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T03:15:30.865 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T03:15:30.865 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T03:15:30.865 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T03:15:30.866 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T03:15:30.866 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T03:15:30.866 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T03:15:30.866 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T03:15:30.866 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-06T03:15:30.866 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T03:15:30.866 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T03:15:30.866 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-06T03:15:30.866 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-06T03:15:30.866 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T03:15:30.866 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T03:15:30.867 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T03:15:30.867 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T03:15:30.867 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-06T03:15:30.867 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T03:15:30.867 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T03:15:30.867 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-06T03:15:30.867 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T03:15:30.867 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T03:15:30.867 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T03:15:30.867 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T03:15:30.867 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T03:15:30.867 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T03:15:30.868 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T03:15:30.868 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T03:15:30.868 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T03:15:30.868 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T03:15:30.868 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T03:15:30.868 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T03:15:30.868 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T03:15:30.868 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T03:15:30.868 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T03:15:30.868 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T03:15:30.868 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T03:15:30.868 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T03:15:30.869 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T03:15:30.869 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T03:15:30.869 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T03:15:30.869 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T03:15:30.869 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T03:15:30.869 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T03:15:30.869 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T03:15:30.869 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T03:15:30.869 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T03:15:30.869 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T03:15:30.869 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T03:15:30.869 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T03:15:30.870 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T03:15:30.870 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T03:15:30.870 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-06T03:15:30.870 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T03:15:30.870 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T03:15:30.870 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T03:15:30.870 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T03:15:30.870 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T03:15:30.870 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-06T03:15:30.870 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T03:15:30.870 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T03:15:30.871 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-06T03:15:30.871 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T03:15:30.871 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T03:15:30.871 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T03:15:30.871 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-06T03:15:30.871 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T03:15:30.871 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T03:15:30.871 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-06T03:15:30.871 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T03:15:30.871 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-06T03:15:30.871 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T03:15:30.872 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-06T03:15:30.872 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T03:15:30.872 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-06T03:15:30.872 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T03:15:30.872 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T03:15:30.872 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-06T03:15:30.872 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T03:15:30.872 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-06T03:15:30.872 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-06T03:15:30.872 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T03:15:30.872 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-06T03:15:30.872 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T03:15:30.873 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T03:15:30.873 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T03:15:30.873 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-06T03:15:30.873 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-06T03:15:30.873 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-06T03:15:30.873 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-06T03:15:30.873 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-06T03:15:30.873 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T03:15:30.873 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-06T03:15:30.873 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T03:15:30.873 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T03:15:30.874 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-06T03:15:30.874 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T03:15:30.874 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T03:15:30.874 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-06T03:15:30.874 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T03:15:30.874 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T03:15:30.874 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T03:15:30.874 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T03:15:30.874 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T03:15:30.874 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T03:15:30.874 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T03:15:30.874 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-06T03:15:30.875 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T03:15:30.875 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T03:15:30.875 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T03:15:30.875 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-06T03:15:30.875 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T03:15:30.875 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T03:15:30.875 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T03:15:30.875 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-06T03:15:30.875 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T03:15:30.875 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-06T03:15:30.875 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T03:15:30.875 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-06T03:15:30.876 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T03:15:30.876 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T03:15:30.876 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T03:15:30.876 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T03:15:30.876 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-06T03:15:30.876 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T03:15:30.876 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-06T03:15:30.876 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T03:15:30.876 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T03:15:30.876 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-06T03:15:30.876 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T03:15:30.876 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T03:15:30.876 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T03:15:30.877 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-06T03:15:30.877 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T03:15:30.877 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T03:15:30.877 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-06T03:15:30.877 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T03:15:30.877 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T03:15:30.877 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T03:15:30.877 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T03:15:30.877 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T03:15:30.877 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T03:15:30.877 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T03:15:30.878 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T03:15:30.878 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T03:15:30.878 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T03:15:30.878 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T03:15:30.878 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-06T03:15:30.878 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T03:15:30.878 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T03:15:30.878 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T03:15:30.878 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T03:15:30.878 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-06T03:15:30.878 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T03:15:30.878 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-06T03:15:30.879 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-06T03:15:30.879 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T03:15:30.879 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-06T03:15:30.879 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T03:15:30.879 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T03:15:30.879 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T03:15:30.879 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T03:15:30.879 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T03:15:30.879 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T03:15:30.879 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-06T03:15:30.879 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T03:15:30.879 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-06T03:15:30.880 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-06T03:15:30.880 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-06T03:15:30.880 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T03:15:30.880 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T03:15:30.880 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T03:15:30.880 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T03:15:30.880 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T03:15:30.880 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T03:15:30.880 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T03:15:30.880 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T03:15:30.880 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T03:15:30.880 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T03:15:30.880 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T03:15:30.881 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T03:15:30.881 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T03:15:30.881 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T03:15:30.881 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T03:15:30.881 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-06T03:15:30.881 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-06T03:15:30.881 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T03:15:30.881 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T03:15:30.881 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T03:15:30.881 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T03:15:30.881 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-06T03:15:30.882 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-06T03:15:30.882 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T03:15:30.882 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T03:15:30.882 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-06T03:15:30.882 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T03:15:30.882 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T03:15:30.882 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-06T03:15:30.882 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-06T03:15:30.882 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T03:15:30.882 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-06T03:15:30.882 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T03:15:30.882 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-06T03:15:30.883 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T03:15:30.883 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T03:15:30.883 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T03:15:30.883 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T03:15:30.883 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T03:15:30.883 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-06T03:15:30.883 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-06T03:15:30.883 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T03:15:30.883 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T03:15:30.883 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T03:15:30.883 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T03:15:30.884 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-06T03:15:30.884 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T03:15:30.884 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-06T03:15:30.884 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T03:15:30.884 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T03:15:30.884 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-06T03:15:30.884 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T03:15:30.884 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T03:15:30.884 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T03:15:30.884 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T03:15:30.884 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T03:15:30.885 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T03:15:30.886 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T03:15:30.886 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-06T03:15:30.886 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T03:15:30.886 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-06T03:15:30.886 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T03:15:30.886 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T03:15:30.886 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T03:15:30.886 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T03:15:30.886 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T03:15:30.886 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T03:15:30.886 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T03:15:30.886 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T03:15:30.887 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T03:15:30.887 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T03:15:30.887 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T03:15:30.887 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T03:15:30.887 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T03:15:30.887 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T03:15:30.887 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T03:15:30.887 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T03:15:30.887 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T03:15:30.887 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T03:15:30.887 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T03:15:30.887 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T03:15:30.888 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T03:15:30.888 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T03:15:30.888 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T03:15:30.888 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T03:15:30.888 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T03:15:30.888 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T03:15:30.888 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T03:15:30.888 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T03:15:30.888 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T03:15:30.888 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T03:15:30.888 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T03:15:30.888 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T03:15:30.889 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T03:15:30.889 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T03:15:30.889 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T03:15:30.889 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T03:15:30.889 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T03:15:30.889 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T03:15:30.889 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T03:15:30.889 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T03:15:30.889 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T03:15:30.889 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T03:15:30.889 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T03:15:30.889 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T03:15:30.890 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T03:15:30.890 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-06T03:15:30.890 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T03:15:30.890 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T03:15:30.890 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T03:15:30.890 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T03:15:30.890 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T03:15:30.890 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T03:15:30.890 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T03:15:30.890 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T03:15:30.890 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T03:15:30.891 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T03:15:30.891 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T03:15:30.891 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T03:15:30.891 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T03:15:30.891 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T03:15:30.891 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T03:15:30.891 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-06T03:15:30.891 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T03:15:30.891 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T03:15:30.891 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T03:15:30.891 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-06T03:15:30.892 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-06T03:15:30.892 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T03:15:30.892 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T03:15:30.892 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T03:15:30.892 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T03:15:30.892 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-06T03:15:30.892 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T03:15:30.892 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T03:15:30.892 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T03:15:30.892 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T03:15:30.892 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-06T03:15:30.892 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-06T03:15:30.893 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T03:15:30.893 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T03:15:30.893 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T03:15:30.893 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-06T03:15:30.893 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-06T03:15:30.893 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-06T03:15:30.893 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-06T03:15:30.893 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-06T03:15:30.893 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-06T03:15:30.893 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-06T03:15:30.893 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-06T03:15:30.893 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-06T03:15:30.893 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-06T03:15:30.894 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T03:15:30.894 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T03:15:30.894 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-06T03:15:30.894 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T03:15:30.894 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T03:15:30.894 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T03:15:30.894 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T03:15:30.894 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T03:15:30.894 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T03:15:30.894 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T03:15:30.894 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T03:15:30.895 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T03:15:30.895 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T03:15:30.895 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T03:15:30.895 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T03:15:30.895 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-06T03:15:30.895 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T03:15:30.895 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T03:15:30.895 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T03:15:30.895 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T03:15:30.895 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T03:15:30.895 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T03:15:30.895 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T03:15:30.895 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T03:15:30.896 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T03:15:30.896 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T03:15:30.896 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T03:15:30.896 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T03:15:30.896 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T03:15:30.896 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T03:15:30.896 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T03:15:30.896 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T03:15:30.896 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T03:15:30.896 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T03:15:30.896 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T03:15:30.896 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T03:15:30.897 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T03:15:30.897 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T03:15:30.897 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T03:15:30.897 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T03:15:30.897 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T03:15:30.897 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T03:15:30.897 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-06T03:15:30.897 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-06T03:15:30.897 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T03:15:30.897 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T03:15:30.897 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T03:15:30.897 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-06T03:15:30.897 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T03:15:30.898 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T03:15:30.898 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-06T03:15:30.898 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T03:15:30.898 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-06T03:15:30.898 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-06T03:15:30.898 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T03:15:30.898 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-06T03:15:30.898 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T03:15:30.898 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-06T03:15:30.898 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T03:15:30.898 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T03:15:30.898 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-06T03:15:30.899 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T03:15:30.899 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-06T03:15:30.899 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T03:15:30.899 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-06T03:15:30.899 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T03:15:30.899 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-06T03:15:30.899 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T03:15:30.899 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-06T03:15:30.899 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T03:15:30.899 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-06T03:15:30.899 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T03:15:30.899 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-06T03:15:30.900 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-06T03:15:30.900 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-06T03:15:30.900 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T03:15:30.900 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-06T03:15:30.900 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-06T03:15:30.900 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T03:15:30.900 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-06T03:15:30.900 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-06T03:15:30.900 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T03:15:30.900 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-06T03:15:30.900 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T03:15:30.900 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-06T03:15:30.901 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-06T03:15:30.901 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T03:15:30.901 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-06T03:15:30.901 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T03:15:30.901 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-06T03:15:30.901 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T03:15:30.901 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-06T03:15:30.901 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T03:15:30.901 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-06T03:15:30.901 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T03:15:30.901 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-06T03:15:30.901 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T03:15:30.902 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-06T03:15:30.902 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T03:15:30.902 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-06T03:15:30.902 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T03:15:30.902 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-06T03:15:30.902 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-06T03:15:30.902 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T03:15:30.902 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-06T03:15:30.902 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T03:15:30.902 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-06T03:15:30.902 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T03:15:30.902 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T03:15:30.903 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T03:15:30.903 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T03:15:30.903 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-06T03:15:30.903 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T03:15:30.903 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T03:15:30.903 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-06T03:15:30.903 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T03:15:30.903 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-06T03:15:30.903 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T03:15:30.903 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-06T03:15:30.903 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T03:15:30.903 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T03:15:30.904 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T03:15:30.904 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T03:15:30.904 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T03:15:30.904 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T03:15:30.904 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T03:15:30.904 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T03:15:30.904 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T03:15:30.904 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T03:15:30.904 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T03:15:30.904 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T03:15:30.904 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T03:15:30.904 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T03:15:30.905 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T03:15:30.905 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T03:15:30.905 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T03:15:30.905 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T03:15:30.905 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T03:15:30.905 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T03:15:30.905 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T03:15:30.905 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T03:15:30.905 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T03:15:30.905 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T03:15:30.905 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T03:15:30.905 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T03:15:30.905 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T03:15:30.906 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T03:15:30.906 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T03:15:30.906 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T03:15:30.906 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T03:15:30.906 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T03:15:30.906 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T03:15:30.906 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T03:15:30.906 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T03:15:30.906 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T03:15:30.906 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T03:15:30.906 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T03:15:30.907 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T03:15:30.907 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T03:15:30.907 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T03:15:30.907 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T03:15:30.907 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T03:15:30.907 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T03:15:30.907 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T03:15:30.907 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T03:15:30.907 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T03:15:30.907 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T03:15:30.907 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T03:15:30.907 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-06T03:15:30.908 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-06T03:15:30.908 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T03:15:30.908 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-06T03:15:30.908 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T03:15:30.908 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-06T03:15:30.908 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T03:15:30.908 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T03:15:30.908 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T03:15:30.908 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T03:15:30.908 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T03:15:30.908 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-06T03:15:30.908 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T03:15:30.908 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-06T03:15:30.909 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T03:15:30.909 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-06T03:15:30.909 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T03:15:30.909 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-06T03:15:30.909 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T03:15:30.909 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-06T03:15:30.909 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-06T03:15:30.909 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T03:15:30.909 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-06T03:15:30.909 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T03:15:30.909 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T03:15:30.909 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-06T03:15:30.910 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T03:15:30.910 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-06T03:15:30.910 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T03:15:30.910 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-06T03:15:30.910 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T03:15:30.910 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-06T03:15:30.910 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T03:15:30.910 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-06T03:15:30.910 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T03:15:30.910 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-06T03:15:30.910 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-06T03:15:30.910 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-06T03:15:30.911 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T03:15:30.911 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-06T03:15:30.911 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-06T03:15:30.911 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-06T03:15:30.911 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-06T03:15:30.911 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T03:15:30.911 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T03:15:30.911 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-06T03:15:30.911 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T03:15:30.911 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T03:15:30.911 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T03:15:30.912 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T03:15:30.912 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-06T03:15:30.912 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T03:15:30.912 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T03:15:30.912 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-06T03:15:30.912 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T03:15:30.912 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-06T03:15:30.912 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T03:15:30.912 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-06T03:15:30.912 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T03:15:30.912 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T03:15:30.912 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T03:15:30.912 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T03:15:30.913 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-06T03:15:30.913 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-06T03:15:30.913 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-06T03:15:30.913 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T03:15:30.913 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-06T03:15:30.913 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-06T03:15:30.913 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T03:15:30.913 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T03:15:30.913 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T03:15:30.913 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T03:15:30.913 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T03:15:30.913 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T03:15:30.914 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T03:15:30.914 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T03:15:30.914 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-06T03:15:30.914 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T03:15:30.914 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T03:15:30.914 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T03:15:30.914 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T03:15:30.914 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T03:15:30.914 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-06T03:15:30.914 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T03:15:30.914 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-06T03:15:30.914 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T03:15:30.915 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T03:15:30.915 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T03:15:30.915 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T03:15:30.915 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-06T03:15:30.915 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T03:15:30.915 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-06T03:15:30.915 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T03:15:30.915 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T03:15:30.915 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-06T03:15:30.915 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T03:15:30.915 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T03:15:30.916 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T03:15:30.916 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T03:15:30.916 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T03:15:30.916 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T03:15:30.916 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T03:15:30.916 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T03:15:30.916 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T03:15:30.916 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T03:15:30.916 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T03:15:30.916 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T03:15:30.916 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T03:15:30.916 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T03:15:30.916 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T03:15:30.917 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T03:15:30.917 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T03:15:30.917 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T03:15:30.917 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T03:15:30.917 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T03:15:30.917 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T03:15:30.917 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-06T03:15:30.917 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T03:15:30.917 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T03:15:30.917 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-06T03:15:30.917 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T03:15:30.917 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-06T03:15:30.918 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-06T03:15:30.918 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T03:15:30.918 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-06T03:15:30.918 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T03:15:30.918 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T03:15:30.918 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T03:15:30.918 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T03:15:30.918 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-06T03:15:30.918 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-06T03:15:30.918 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-06T03:15:30.918 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T03:15:30.919 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T03:15:30.919 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T03:15:30.919 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T03:15:30.919 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T03:15:30.919 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-06T03:15:30.919 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-06T03:15:30.919 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T03:15:30.919 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T03:15:30.919 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T03:15:30.919 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-06T03:15:30.919 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T03:15:30.919 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T03:15:30.920 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T03:15:30.920 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T03:15:30.920 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T03:15:30.920 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T03:15:30.920 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T03:15:30.920 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T03:15:30.920 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T03:15:30.920 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T03:15:30.920 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-06T03:15:30.920 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T03:15:30.920 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T03:15:30.920 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T03:15:30.920 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T03:15:30.921 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T03:15:30.921 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T03:15:30.921 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T03:15:30.921 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T03:15:30.921 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T03:15:30.921 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T03:15:30.921 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T03:15:30.921 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T03:15:30.921 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T03:15:30.921 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T03:15:30.921 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T03:15:30.922 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T03:15:30.922 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T03:15:30.922 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T03:15:30.922 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T03:15:30.922 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T03:15:30.922 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T03:15:30.922 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T03:15:30.922 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T03:15:30.922 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T03:15:30.922 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T03:15:30.922 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T03:15:30.922 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T03:15:30.923 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T03:15:30.923 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T03:15:30.923 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T03:15:30.923 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T03:15:30.923 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T03:15:30.923 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T03:15:30.923 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T03:15:30.923 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T03:15:30.923 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T03:15:30.923 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T03:15:30.923 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T03:15:30.923 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-06T03:15:30.923 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T03:15:30.923 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T03:15:30.924 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T03:15:30.924 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T03:15:30.924 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T03:15:30.924 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T03:15:30.924 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T03:15:30.924 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-06T03:15:30.924 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T03:15:30.924 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T03:15:30.924 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T03:15:30.924 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T03:15:30.924 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T03:15:30.924 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T03:15:30.924 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T03:15:30.925 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T03:15:30.925 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T03:15:30.925 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T03:15:30.925 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T03:15:30.925 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T03:15:30.925 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T03:15:30.925 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T03:15:30.925 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T03:15:30.925 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T03:15:30.925 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T03:15:30.925 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T03:15:30.925 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T03:15:30.925 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T03:15:30.926 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T03:15:30.926 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T03:15:30.926 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T03:15:30.926 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T03:15:30.926 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T03:15:30.926 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T03:15:30.926 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T03:15:30.926 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-06T03:15:30.926 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T03:15:30.926 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-06T03:15:30.926 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T03:15:30.927 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T03:15:30.927 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T03:15:30.927 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T03:15:30.927 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T03:15:30.927 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T03:15:30.927 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T03:15:30.927 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-06T03:15:30.927 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T03:15:30.927 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-06T03:15:30.927 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T03:15:30.927 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-06T03:15:30.927 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T03:15:30.928 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-06T03:15:30.928 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-06T03:15:30.928 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T03:15:30.928 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T03:15:30.928 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T03:15:30.928 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T03:15:30.928 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T03:15:30.928 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T03:15:30.928 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T03:15:30.928 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T03:15:30.929 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T03:15:30.929 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T03:15:30.929 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T03:15:30.929 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T03:15:30.929 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T03:15:30.929 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T03:15:30.929 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T03:15:30.929 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T03:15:30.929 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-06T03:15:30.929 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T03:15:30.929 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T03:15:30.929 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T03:15:30.930 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T03:15:30.930 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T03:15:30.930 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T03:15:30.930 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T03:15:30.930 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T03:15:30.930 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T03:15:30.930 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T03:15:30.930 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T03:15:30.930 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T03:15:30.930 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T03:15:30.930 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T03:15:30.930 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T03:15:30.931 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T03:15:30.931 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T03:15:30.931 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T03:15:30.931 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T03:15:30.931 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T03:15:30.931 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T03:15:30.931 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T03:15:30.931 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T03:15:30.931 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T03:15:30.931 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T03:15:30.931 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T03:15:30.931 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T03:15:30.932 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T03:15:30.932 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T03:15:30.932 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T03:15:30.932 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T03:15:30.932 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T03:15:30.932 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T03:15:30.932 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T03:15:30.932 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T03:15:30.932 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T03:15:30.932 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T03:15:30.932 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T03:15:30.932 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T03:15:30.933 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T03:15:30.933 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T03:15:30.933 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T03:15:30.933 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T03:15:30.933 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T03:15:30.933 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T03:15:30.933 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T03:15:30.933 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T03:15:30.933 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T03:15:30.933 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T03:15:30.934 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T03:15:30.934 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T03:15:30.934 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T03:15:30.934 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T03:15:30.934 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T03:15:30.934 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T03:15:30.934 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T03:15:30.934 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T03:15:30.934 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T03:15:30.934 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-06T03:15:30.934 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T03:15:30.935 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T03:15:30.935 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T03:15:30.935 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T03:15:30.935 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-06T03:15:30.935 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T03:15:30.935 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T03:15:30.935 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T03:15:30.935 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T03:15:30.935 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T03:15:30.935 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T03:15:30.935 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T03:15:30.935 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T03:15:30.936 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T03:15:30.936 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T03:15:30.936 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T03:15:30.936 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T03:15:30.936 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T03:15:30.936 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T03:15:30.936 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T03:15:30.936 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T03:15:30.936 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T03:15:30.936 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T03:15:30.936 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T03:15:30.936 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T03:15:30.936 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T03:15:30.937 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T03:15:30.937 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T03:15:30.937 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T03:15:30.937 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T03:15:30.937 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T03:15:30.937 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T03:15:30.937 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T03:15:30.937 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T03:15:30.937 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T03:15:30.937 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T03:15:30.937 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T03:15:30.937 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T03:15:30.938 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T03:15:30.938 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T03:15:30.938 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T03:15:30.938 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T03:15:30.938 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T03:15:30.938 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T03:15:30.938 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T03:15:30.938 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T03:15:30.938 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T03:15:30.938 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T03:15:30.938 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T03:15:30.939 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T03:15:30.939 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T03:15:30.939 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T03:15:30.939 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T03:15:30.939 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T03:15:30.939 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T03:15:30.939 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T03:15:30.939 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T03:15:30.939 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T03:15:30.939 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T03:15:30.939 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T03:15:30.940 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T03:15:30.940 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T03:15:30.940 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T03:15:30.940 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T03:15:30.940 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T03:15:30.940 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T03:15:30.940 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T03:15:30.940 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T03:15:30.940 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T03:15:30.940 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T03:15:30.940 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T03:15:30.940 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T03:15:30.940 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T03:15:30.941 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T03:15:30.941 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T03:15:30.941 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T03:15:30.941 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-06T03:15:30.941 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T03:15:30.941 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-06T03:15:30.941 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T03:15:30.941 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-06T03:15:30.941 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T03:15:30.941 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T03:15:30.941 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T03:15:30.941 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T03:15:30.942 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-06T03:15:30.942 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T03:15:30.942 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-06T03:15:30.942 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T03:15:30.942 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T03:15:30.942 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T03:15:30.942 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-06T03:15:30.942 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T03:15:30.942 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T03:15:30.942 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T03:15:30.943 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T03:15:30.943 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T03:15:30.943 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T03:15:30.943 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T03:15:30.943 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T03:15:30.943 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T03:15:30.943 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T03:15:30.943 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T03:15:30.943 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T03:15:30.943 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T03:15:30.943 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T03:15:30.944 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T03:15:30.944 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T03:15:30.944 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T03:15:30.944 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T03:15:30.944 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T03:15:30.944 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T03:15:30.944 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T03:15:30.944 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T03:15:30.944 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T03:15:30.944 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T03:15:30.944 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T03:15:30.945 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T03:15:30.945 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T03:15:30.945 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T03:15:30.945 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T03:15:30.945 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T03:15:30.945 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T03:15:30.945 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T03:15:30.945 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T03:15:30.945 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T03:15:30.945 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T03:15:30.945 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T03:15:30.945 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T03:15:30.946 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T03:15:30.946 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T03:15:30.946 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T03:15:30.946 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T03:15:30.946 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T03:15:30.946 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T03:15:30.946 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T03:15:30.946 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-06T03:15:30.946 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T03:15:30.946 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T03:15:30.946 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-06T03:15:30.946 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-06T03:15:30.947 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T03:15:30.947 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T03:15:30.947 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T03:15:30.947 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-06T03:15:30.947 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T03:15:30.947 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-06T03:15:30.947 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T03:15:30.947 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T03:15:30.947 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-06T03:15:30.947 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T03:15:30.947 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-06T03:15:30.947 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T03:15:30.948 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T03:15:30.948 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T03:15:30.948 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T03:15:30.948 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T03:15:30.948 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-06T03:15:30.948 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T03:15:30.948 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-06T03:15:30.948 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-06T03:15:30.948 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-06T03:15:30.948 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-06T03:15:30.948 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-06T03:15:30.949 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-06T03:15:30.949 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-06T03:15:30.949 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-06T03:15:30.949 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T03:15:30.949 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T03:15:30.949 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T03:15:30.949 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T03:15:30.949 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T03:15:30.949 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T03:15:30.949 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T03:15:30.950 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T03:15:30.950 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T03:15:30.950 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T03:15:30.950 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T03:15:30.950 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T03:15:30.950 INFO:teuthology.orchestra.run.smithi163.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T03:15:30.951 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:30.951 DEBUG:teuthology.orchestra.run.smithi163:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-06T03:15:30.975 INFO:teuthology.orchestra.run.smithi163.stdout:postinstall scriptlet (using /bin/sh): 2023-10-06T03:15:30.975 INFO:teuthology.orchestra.run.smithi163.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-06T03:15:30.975 INFO:teuthology.orchestra.run.smithi163.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T03:15:30.975 INFO:teuthology.orchestra.run.smithi163.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T03:15:30.975 INFO:teuthology.orchestra.run.smithi163.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:30.975 INFO:teuthology.orchestra.run.smithi163.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T03:15:30.975 INFO:teuthology.orchestra.run.smithi163.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T03:15:30.975 INFO:teuthology.orchestra.run.smithi163.stdout:fi 2023-10-06T03:15:30.976 INFO:teuthology.orchestra.run.smithi163.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-06T03:15:30.976 INFO:teuthology.orchestra.run.smithi163.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-06T03:15:30.976 INFO:teuthology.orchestra.run.smithi163.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-06T03:15:30.976 INFO:teuthology.orchestra.run.smithi163.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-06T03:15:30.976 INFO:teuthology.orchestra.run.smithi163.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:30.976 INFO:teuthology.orchestra.run.smithi163.stdout:fi 2023-10-06T03:15:30.976 INFO:teuthology.orchestra.run.smithi163.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-06T03:15:30.976 INFO:teuthology.orchestra.run.smithi163.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-06T03:15:30.976 INFO:teuthology.orchestra.run.smithi163.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:30.976 INFO:teuthology.orchestra.run.smithi163.stdout:fi 2023-10-06T03:15:30.977 DEBUG:teuthology.orchestra.run.smithi163:> sudo rpm -qi grub2-tools 2023-10-06T03:15:31.054 INFO:teuthology.orchestra.run.smithi163.stdout:Name : grub2-tools 2023-10-06T03:15:31.054 INFO:teuthology.orchestra.run.smithi163.stdout:Epoch : 1 2023-10-06T03:15:31.054 INFO:teuthology.orchestra.run.smithi163.stdout:Version : 2.02 2023-10-06T03:15:31.054 INFO:teuthology.orchestra.run.smithi163.stdout:Release : 99.el8_4.1 2023-10-06T03:15:31.054 INFO:teuthology.orchestra.run.smithi163.stdout:Architecture: x86_64 2023-10-06T03:15:31.054 INFO:teuthology.orchestra.run.smithi163.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T03:15:31.055 INFO:teuthology.orchestra.run.smithi163.stdout:Group : System Environment/Base 2023-10-06T03:15:31.055 INFO:teuthology.orchestra.run.smithi163.stdout:Size : 9264631 2023-10-06T03:15:31.055 INFO:teuthology.orchestra.run.smithi163.stdout:License : GPLv3+ 2023-10-06T03:15:31.055 INFO:teuthology.orchestra.run.smithi163.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T03:15:31.055 INFO:teuthology.orchestra.run.smithi163.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T03:15:31.055 INFO:teuthology.orchestra.run.smithi163.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T03:15:31.055 INFO:teuthology.orchestra.run.smithi163.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T03:15:31.055 INFO:teuthology.orchestra.run.smithi163.stdout:Relocations : (not relocatable) 2023-10-06T03:15:31.055 INFO:teuthology.orchestra.run.smithi163.stdout:Packager : Red Hat, Inc. 2023-10-06T03:15:31.055 INFO:teuthology.orchestra.run.smithi163.stdout:Vendor : Red Hat, Inc. 2023-10-06T03:15:31.055 INFO:teuthology.orchestra.run.smithi163.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T03:15:31.055 INFO:teuthology.orchestra.run.smithi163.stdout:Summary : Support tools for GRUB. 2023-10-06T03:15:31.056 INFO:teuthology.orchestra.run.smithi163.stdout:Description : 2023-10-06T03:15:31.056 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:15:31.056 INFO:teuthology.orchestra.run.smithi163.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T03:15:31.056 INFO:teuthology.orchestra.run.smithi163.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T03:15:31.056 INFO:teuthology.orchestra.run.smithi163.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T03:15:31.056 INFO:teuthology.orchestra.run.smithi163.stdout:hardware devices. 2023-10-06T03:15:31.056 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:15:31.056 INFO:teuthology.orchestra.run.smithi163.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T03:15:31.057 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T03:15:31.057 INFO:teuthology.task.kernel:Updating grub on smithi163 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:31.057 DEBUG:teuthology.orchestra.run.smithi163:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T03:15:34.820 INFO:teuthology.orchestra.run.smithi163.stderr:Generating grub configuration file ... 2023-10-06T03:15:37.237 INFO:teuthology.orchestra.run.smithi163.stderr:done 2023-10-06T03:15:37.239 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2023-10-06T03:15:37.274 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.agDEtNUHIX 2023-10-06T03:15:37.274 DEBUG:teuthology.orchestra.run.smithi163:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.agDEtNUHIX 2023-10-06T03:15:37.351 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.agDEtNUHIX 2023-10-06T03:15:37.514 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.agDEtNUHIX is 6KB 2023-10-06T03:15:37.562 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.agDEtNUHIX 2023-10-06T03:15:37.576 DEBUG:teuthology.orchestra.run.smithi163:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T03:15:37.647 INFO:teuthology.orchestra.run.smithi163.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T03:15:37.647 INFO:teuthology.orchestra.run.smithi163.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T03:15:37.647 INFO:teuthology.orchestra.run.smithi163.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T03:15:37.647 INFO:teuthology.orchestra.run.smithi163.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T03:15:37.647 INFO:teuthology.orchestra.run.smithi163.stdout:298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b.conf 2023-10-06T03:15:37.648 DEBUG:teuthology.orchestra.run.smithi163:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:37.809 DEBUG:teuthology.orchestra.run.smithi163:> sudo shutdown -r now 2023-10-06T03:15:41.405 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi055.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-06T03:15:41.405 DEBUG:teuthology.orchestra.run.smithi055:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-06T03:15:41.466 INFO:teuthology.orchestra.run.smithi055.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:41.466 INFO:teuthology.orchestra.run.smithi055.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:41.466 INFO:teuthology.orchestra.run.smithi055.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:41.466 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:41.466 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-06T03:15:41.466 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-06T03:15:41.466 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-06T03:15:41.466 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-06T03:15:41.467 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-06T03:15:41.467 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-06T03:15:41.467 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-06T03:15:41.467 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-06T03:15:41.467 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-06T03:15:41.467 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-06T03:15:41.467 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-06T03:15:41.467 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-06T03:15:41.467 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-06T03:15:41.468 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-06T03:15:41.468 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-06T03:15:41.468 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-06T03:15:41.468 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-06T03:15:41.468 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-06T03:15:41.468 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-06T03:15:41.468 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-06T03:15:41.468 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-06T03:15:41.468 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-06T03:15:41.468 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-06T03:15:41.469 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-06T03:15:41.469 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-06T03:15:41.469 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-06T03:15:41.469 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-06T03:15:41.469 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-06T03:15:41.469 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-06T03:15:41.469 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-06T03:15:41.469 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-06T03:15:41.469 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-06T03:15:41.470 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-06T03:15:41.470 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-06T03:15:41.470 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-06T03:15:41.470 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-06T03:15:41.470 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-06T03:15:41.470 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-06T03:15:41.470 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-06T03:15:41.470 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-06T03:15:41.470 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-06T03:15:41.470 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-06T03:15:41.471 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-06T03:15:41.471 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-06T03:15:41.471 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-06T03:15:41.471 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-06T03:15:41.471 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-06T03:15:41.471 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-06T03:15:41.471 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-06T03:15:41.471 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-06T03:15:41.471 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-06T03:15:41.471 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-06T03:15:41.472 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-06T03:15:41.472 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-06T03:15:41.472 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-06T03:15:41.472 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-06T03:15:41.472 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-06T03:15:41.472 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-06T03:15:41.472 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-06T03:15:41.472 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-06T03:15:41.472 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-06T03:15:41.472 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-06T03:15:41.473 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-06T03:15:41.473 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-06T03:15:41.473 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-06T03:15:41.473 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-06T03:15:41.473 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-06T03:15:41.473 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-06T03:15:41.473 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-06T03:15:41.473 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-06T03:15:41.473 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-06T03:15:41.473 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-06T03:15:41.474 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-06T03:15:41.474 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-06T03:15:41.474 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-06T03:15:41.474 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-06T03:15:41.474 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-06T03:15:41.474 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-06T03:15:41.474 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-06T03:15:41.474 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-06T03:15:41.474 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-06T03:15:41.475 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-06T03:15:41.475 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-06T03:15:41.475 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-06T03:15:41.475 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-06T03:15:41.475 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-06T03:15:41.475 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-06T03:15:41.475 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-06T03:15:41.475 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-06T03:15:41.475 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-06T03:15:41.475 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-06T03:15:41.476 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-06T03:15:41.476 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-06T03:15:41.476 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-06T03:15:41.476 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-06T03:15:41.476 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-06T03:15:41.476 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-06T03:15:41.476 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-06T03:15:41.476 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-06T03:15:41.476 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-06T03:15:41.476 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-06T03:15:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-06T03:15:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-06T03:15:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-06T03:15:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-06T03:15:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-06T03:15:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-06T03:15:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-06T03:15:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-06T03:15:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-06T03:15:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-06T03:15:41.478 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-06T03:15:41.478 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-06T03:15:41.478 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-06T03:15:41.478 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-06T03:15:41.478 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-06T03:15:41.478 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-06T03:15:41.478 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-06T03:15:41.478 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-06T03:15:41.478 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-06T03:15:41.479 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-06T03:15:41.479 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-06T03:15:41.479 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-06T03:15:41.479 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-06T03:15:41.479 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-06T03:15:41.479 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-06T03:15:41.479 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-06T03:15:41.479 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-06T03:15:41.479 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-06T03:15:41.479 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-06T03:15:41.480 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-06T03:15:41.480 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-06T03:15:41.480 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-06T03:15:41.480 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-06T03:15:41.480 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-06T03:15:41.480 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-06T03:15:41.480 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-06T03:15:41.480 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-06T03:15:41.480 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-06T03:15:41.480 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-06T03:15:41.481 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-06T03:15:41.481 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-06T03:15:41.481 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-06T03:15:41.481 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-06T03:15:41.481 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-06T03:15:41.481 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-06T03:15:41.481 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-06T03:15:41.481 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-06T03:15:41.481 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-06T03:15:41.481 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-06T03:15:41.482 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-06T03:15:41.482 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-06T03:15:41.482 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-06T03:15:41.482 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-06T03:15:41.482 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-06T03:15:41.482 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-06T03:15:41.482 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-06T03:15:41.482 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-06T03:15:41.482 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-06T03:15:41.482 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-06T03:15:41.483 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-06T03:15:41.483 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-06T03:15:41.483 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-06T03:15:41.483 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-06T03:15:41.483 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-06T03:15:41.483 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-06T03:15:41.483 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-06T03:15:41.483 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-06T03:15:41.483 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-06T03:15:41.483 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-06T03:15:41.484 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-06T03:15:41.484 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-06T03:15:41.484 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-06T03:15:41.484 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-06T03:15:41.484 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-06T03:15:41.484 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-06T03:15:41.484 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-06T03:15:41.484 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-06T03:15:41.484 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-06T03:15:41.484 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-06T03:15:41.484 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-06T03:15:41.485 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-06T03:15:41.485 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-06T03:15:41.485 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-06T03:15:41.485 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-06T03:15:41.485 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-06T03:15:41.485 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-06T03:15:41.485 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-06T03:15:41.485 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-06T03:15:41.485 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-06T03:15:41.485 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-06T03:15:41.486 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-06T03:15:41.486 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-06T03:15:41.486 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-06T03:15:41.486 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-06T03:15:41.486 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-06T03:15:41.486 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-06T03:15:41.486 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-06T03:15:41.486 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-06T03:15:41.486 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-06T03:15:41.486 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-06T03:15:41.487 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-06T03:15:41.487 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-06T03:15:41.487 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-06T03:15:41.487 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-06T03:15:41.487 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-06T03:15:41.487 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-06T03:15:41.487 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-06T03:15:41.487 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-06T03:15:41.487 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-06T03:15:41.487 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-06T03:15:41.488 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-06T03:15:41.488 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-06T03:15:41.488 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-06T03:15:41.488 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-06T03:15:41.488 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-06T03:15:41.488 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-06T03:15:41.488 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-06T03:15:41.488 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-06T03:15:41.488 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-06T03:15:41.488 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-06T03:15:41.489 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-06T03:15:41.489 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-06T03:15:41.489 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-06T03:15:41.489 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-06T03:15:41.489 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-06T03:15:41.489 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-06T03:15:41.489 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-06T03:15:41.489 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-06T03:15:41.489 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-06T03:15:41.489 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-06T03:15:41.489 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-06T03:15:41.490 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-06T03:15:41.490 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-06T03:15:41.490 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-06T03:15:41.490 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-06T03:15:41.490 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-06T03:15:41.490 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-06T03:15:41.490 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-06T03:15:41.490 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-06T03:15:41.490 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-06T03:15:41.491 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-06T03:15:41.491 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-06T03:15:41.491 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-06T03:15:41.491 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-06T03:15:41.491 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-06T03:15:41.491 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-06T03:15:41.491 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-06T03:15:41.491 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-06T03:15:41.491 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-06T03:15:41.492 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-06T03:15:41.492 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-06T03:15:41.492 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-06T03:15:41.492 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-06T03:15:41.492 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-06T03:15:41.492 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-06T03:15:41.492 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-06T03:15:41.492 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-06T03:15:41.492 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-06T03:15:41.493 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-06T03:15:41.493 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-06T03:15:41.493 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-06T03:15:41.493 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-06T03:15:41.493 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-06T03:15:41.493 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-06T03:15:41.493 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-06T03:15:41.493 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-06T03:15:41.493 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-06T03:15:41.493 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-06T03:15:41.493 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-06T03:15:41.494 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-06T03:15:41.494 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-06T03:15:41.494 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-06T03:15:41.494 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-06T03:15:41.494 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-06T03:15:41.494 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-06T03:15:41.494 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-06T03:15:41.494 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-06T03:15:41.494 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-06T03:15:41.494 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-06T03:15:41.495 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-06T03:15:41.495 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-06T03:15:41.495 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-06T03:15:41.495 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-06T03:15:41.495 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-06T03:15:41.495 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-06T03:15:41.495 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-06T03:15:41.495 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-06T03:15:41.495 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-06T03:15:41.495 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-06T03:15:41.495 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-06T03:15:41.496 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-06T03:15:41.496 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-06T03:15:41.496 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-06T03:15:41.496 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-06T03:15:41.496 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-06T03:15:41.496 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-06T03:15:41.496 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-06T03:15:41.496 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-06T03:15:41.496 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-06T03:15:41.496 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-06T03:15:41.496 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-06T03:15:41.497 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-06T03:15:41.497 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-06T03:15:41.497 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-06T03:15:41.497 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-06T03:15:41.497 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-06T03:15:41.497 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-06T03:15:41.497 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-06T03:15:41.497 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-06T03:15:41.497 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-06T03:15:41.497 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-06T03:15:41.497 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-06T03:15:41.498 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-06T03:15:41.498 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-06T03:15:41.498 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-06T03:15:41.498 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-06T03:15:41.498 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-06T03:15:41.498 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-06T03:15:41.498 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-06T03:15:41.498 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-06T03:15:41.498 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-06T03:15:41.499 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-06T03:15:41.499 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-06T03:15:41.499 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-06T03:15:41.499 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-06T03:15:41.499 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-06T03:15:41.499 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-06T03:15:41.499 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-06T03:15:41.499 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-06T03:15:41.499 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-06T03:15:41.499 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-06T03:15:41.499 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-06T03:15:41.500 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-06T03:15:41.500 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-06T03:15:41.500 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-06T03:15:41.500 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-06T03:15:41.500 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-06T03:15:41.500 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-06T03:15:41.500 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-06T03:15:41.500 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-06T03:15:41.500 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-06T03:15:41.500 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-06T03:15:41.501 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-06T03:15:41.501 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-06T03:15:41.501 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-06T03:15:41.501 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-06T03:15:41.501 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-06T03:15:41.501 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-06T03:15:41.501 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-06T03:15:41.501 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-06T03:15:41.501 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-06T03:15:41.501 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-06T03:15:41.501 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-06T03:15:41.502 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-06T03:15:41.502 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-06T03:15:41.502 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-06T03:15:41.502 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-06T03:15:41.502 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-06T03:15:41.502 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-06T03:15:41.502 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-06T03:15:41.502 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-06T03:15:41.502 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-06T03:15:41.503 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-06T03:15:41.503 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-06T03:15:41.503 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-06T03:15:41.503 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-06T03:15:41.503 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-06T03:15:41.503 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-06T03:15:41.503 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-06T03:15:41.503 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-06T03:15:41.504 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-06T03:15:41.504 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-06T03:15:41.504 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-06T03:15:41.504 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-06T03:15:41.504 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-06T03:15:41.504 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-06T03:15:41.504 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-06T03:15:41.504 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-06T03:15:41.504 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-06T03:15:41.504 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-06T03:15:41.505 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-06T03:15:41.505 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-06T03:15:41.505 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-06T03:15:41.505 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-06T03:15:41.505 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-06T03:15:41.505 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-06T03:15:41.505 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-06T03:15:41.505 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-06T03:15:41.505 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-06T03:15:41.505 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-06T03:15:41.506 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-06T03:15:41.506 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-06T03:15:41.506 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-06T03:15:41.506 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-06T03:15:41.506 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-06T03:15:41.506 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-06T03:15:41.506 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-06T03:15:41.506 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-06T03:15:41.507 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-06T03:15:41.507 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-06T03:15:41.507 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-06T03:15:41.507 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-06T03:15:41.507 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-06T03:15:41.507 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-06T03:15:41.507 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-06T03:15:41.507 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-06T03:15:41.507 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-06T03:15:41.507 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-06T03:15:41.507 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-06T03:15:41.508 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-06T03:15:41.508 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-06T03:15:41.508 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-06T03:15:41.508 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-06T03:15:41.508 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-06T03:15:41.508 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-06T03:15:41.508 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-06T03:15:41.508 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-06T03:15:41.508 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-06T03:15:41.508 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-06T03:15:41.509 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-06T03:15:41.509 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-06T03:15:41.509 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-06T03:15:41.509 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-06T03:15:41.509 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-06T03:15:41.509 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-06T03:15:41.509 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-06T03:15:41.509 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-06T03:15:41.509 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-06T03:15:41.510 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-06T03:15:41.510 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-06T03:15:41.510 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-06T03:15:41.510 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-06T03:15:41.510 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-06T03:15:41.510 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-06T03:15:41.510 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-06T03:15:41.510 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-06T03:15:41.511 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-06T03:15:41.511 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-06T03:15:41.511 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-06T03:15:41.511 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-06T03:15:41.511 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-06T03:15:41.511 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-06T03:15:41.511 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-06T03:15:41.511 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-06T03:15:41.511 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-06T03:15:41.512 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-06T03:15:41.512 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-06T03:15:41.512 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-06T03:15:41.512 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-06T03:15:41.512 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-06T03:15:41.512 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-06T03:15:41.512 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-06T03:15:41.512 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-06T03:15:41.512 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-06T03:15:41.512 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-06T03:15:41.513 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-06T03:15:41.513 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-06T03:15:41.513 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-06T03:15:41.513 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-06T03:15:41.513 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-06T03:15:41.513 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-06T03:15:41.513 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-06T03:15:41.513 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-06T03:15:41.513 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-06T03:15:41.513 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-06T03:15:41.513 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-06T03:15:41.514 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-06T03:15:41.514 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-06T03:15:41.514 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-06T03:15:41.514 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-06T03:15:41.514 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-06T03:15:41.514 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-06T03:15:41.514 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-06T03:15:41.514 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-06T03:15:41.514 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-06T03:15:41.515 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-06T03:15:41.515 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-06T03:15:41.515 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-06T03:15:41.515 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-06T03:15:41.515 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-06T03:15:41.515 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-06T03:15:41.515 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-06T03:15:41.515 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-06T03:15:41.515 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-06T03:15:41.515 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-06T03:15:41.516 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-06T03:15:41.516 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-06T03:15:41.516 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-06T03:15:41.516 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-06T03:15:41.516 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-06T03:15:41.516 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-06T03:15:41.516 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-06T03:15:41.516 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-06T03:15:41.516 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-06T03:15:41.516 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-06T03:15:41.517 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-06T03:15:41.517 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-06T03:15:41.517 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-06T03:15:41.517 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-06T03:15:41.517 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-06T03:15:41.517 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-06T03:15:41.517 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-06T03:15:41.517 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-06T03:15:41.517 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-06T03:15:41.518 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-06T03:15:41.518 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-06T03:15:41.518 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-06T03:15:41.518 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-06T03:15:41.518 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-06T03:15:41.518 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-06T03:15:41.518 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-06T03:15:41.518 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-06T03:15:41.518 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-06T03:15:41.519 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-06T03:15:41.519 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-06T03:15:41.519 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-06T03:15:41.519 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-06T03:15:41.519 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-06T03:15:41.519 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-06T03:15:41.519 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-06T03:15:41.519 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-06T03:15:41.519 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-06T03:15:41.519 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-06T03:15:41.520 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-06T03:15:41.520 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-06T03:15:41.520 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-06T03:15:41.520 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-06T03:15:41.520 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-06T03:15:41.520 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-06T03:15:41.520 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-06T03:15:41.520 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-06T03:15:41.520 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-06T03:15:41.520 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-06T03:15:41.521 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-06T03:15:41.521 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-06T03:15:41.521 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-06T03:15:41.521 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-06T03:15:41.521 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-06T03:15:41.521 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-06T03:15:41.521 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-06T03:15:41.521 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-06T03:15:41.522 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-06T03:15:41.522 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-06T03:15:41.522 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-06T03:15:41.522 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-06T03:15:41.522 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-06T03:15:41.522 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-06T03:15:41.522 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-06T03:15:41.522 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-06T03:15:41.522 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-06T03:15:41.523 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-06T03:15:41.523 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-06T03:15:41.523 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-06T03:15:41.523 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-06T03:15:41.523 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-06T03:15:41.523 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-06T03:15:41.523 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-06T03:15:41.523 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-06T03:15:41.523 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-06T03:15:41.523 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-06T03:15:41.523 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-06T03:15:41.524 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-06T03:15:41.524 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-06T03:15:41.524 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-06T03:15:41.524 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-06T03:15:41.524 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-06T03:15:41.524 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-06T03:15:41.524 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-06T03:15:41.524 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-06T03:15:41.525 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-06T03:15:41.525 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-06T03:15:41.525 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-06T03:15:41.525 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-06T03:15:41.525 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-06T03:15:41.525 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-06T03:15:41.525 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-06T03:15:41.525 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-06T03:15:41.525 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-06T03:15:41.525 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-06T03:15:41.526 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-06T03:15:41.526 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-06T03:15:41.526 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-06T03:15:41.526 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-06T03:15:41.526 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-06T03:15:41.526 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-06T03:15:41.526 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-06T03:15:41.526 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-06T03:15:41.526 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-06T03:15:41.526 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-06T03:15:41.527 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-06T03:15:41.529 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-06T03:15:41.529 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-06T03:15:41.529 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-06T03:15:41.529 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-06T03:15:41.529 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-06T03:15:41.529 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-06T03:15:41.530 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-06T03:15:41.530 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-06T03:15:41.530 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-06T03:15:41.530 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-06T03:15:41.530 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-06T03:15:41.530 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-06T03:15:41.530 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-06T03:15:41.530 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-06T03:15:41.530 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-06T03:15:41.530 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-06T03:15:41.531 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-06T03:15:41.531 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-06T03:15:41.531 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-06T03:15:41.531 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-06T03:15:41.531 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-06T03:15:41.531 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-06T03:15:41.531 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-06T03:15:41.531 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-06T03:15:41.531 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-06T03:15:41.531 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-06T03:15:41.532 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-06T03:15:41.532 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-06T03:15:41.532 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-06T03:15:41.532 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-06T03:15:41.532 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-06T03:15:41.532 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-06T03:15:41.532 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-06T03:15:41.532 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-06T03:15:41.532 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-06T03:15:41.532 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-06T03:15:41.533 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-06T03:15:41.533 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-06T03:15:41.533 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-06T03:15:41.533 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-06T03:15:41.533 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-06T03:15:41.533 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-06T03:15:41.533 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-06T03:15:41.533 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-06T03:15:41.533 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-06T03:15:41.533 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-06T03:15:41.534 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-06T03:15:41.534 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-06T03:15:41.534 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-06T03:15:41.534 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-06T03:15:41.534 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-06T03:15:41.534 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-06T03:15:41.534 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-06T03:15:41.534 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-06T03:15:41.534 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-06T03:15:41.534 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-06T03:15:41.535 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-06T03:15:41.535 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-06T03:15:41.535 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-06T03:15:41.535 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-06T03:15:41.535 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-06T03:15:41.535 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-06T03:15:41.535 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-06T03:15:41.535 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-06T03:15:41.535 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-06T03:15:41.535 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-06T03:15:41.536 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-06T03:15:41.537 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-06T03:15:41.537 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-06T03:15:41.537 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-06T03:15:41.537 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-06T03:15:41.537 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-06T03:15:41.537 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-06T03:15:41.537 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-06T03:15:41.537 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-06T03:15:41.537 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-06T03:15:41.537 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-06T03:15:41.538 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-06T03:15:41.539 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-06T03:15:41.539 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-06T03:15:41.539 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-06T03:15:41.539 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-06T03:15:41.539 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-06T03:15:41.539 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-06T03:15:41.539 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-06T03:15:41.539 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-06T03:15:41.540 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-06T03:15:41.541 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-06T03:15:41.541 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-06T03:15:41.541 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-06T03:15:41.541 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-06T03:15:41.541 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-06T03:15:41.541 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-06T03:15:41.541 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-06T03:15:41.541 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-06T03:15:41.541 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-06T03:15:41.541 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-06T03:15:41.542 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-06T03:15:41.542 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-06T03:15:41.542 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-06T03:15:41.542 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-06T03:15:41.542 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-06T03:15:41.542 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-06T03:15:41.542 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-06T03:15:41.542 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-06T03:15:41.542 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-06T03:15:41.543 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-06T03:15:41.543 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-06T03:15:41.543 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-06T03:15:41.543 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-06T03:15:41.543 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-06T03:15:41.543 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-06T03:15:41.543 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-06T03:15:41.543 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-06T03:15:41.543 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-06T03:15:41.543 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-06T03:15:41.544 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-06T03:15:41.544 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-06T03:15:41.544 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-06T03:15:41.544 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-06T03:15:41.544 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-06T03:15:41.544 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-06T03:15:41.544 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-06T03:15:41.544 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-06T03:15:41.544 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-06T03:15:41.544 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-06T03:15:41.545 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-06T03:15:41.545 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-06T03:15:41.545 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-06T03:15:41.545 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-06T03:15:41.545 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-06T03:15:41.545 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-06T03:15:41.545 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-06T03:15:41.545 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-06T03:15:41.545 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-06T03:15:41.545 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-06T03:15:41.546 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-06T03:15:41.546 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-06T03:15:41.546 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-06T03:15:41.546 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-06T03:15:41.546 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-06T03:15:41.546 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-06T03:15:41.546 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-06T03:15:41.546 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-06T03:15:41.546 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-06T03:15:41.547 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-06T03:15:41.547 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-06T03:15:41.547 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-06T03:15:41.547 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-06T03:15:41.547 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-06T03:15:41.547 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-06T03:15:41.547 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-06T03:15:41.547 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-06T03:15:41.547 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-06T03:15:41.547 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-06T03:15:41.548 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-06T03:15:41.548 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-06T03:15:41.548 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-06T03:15:41.548 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-06T03:15:41.548 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-06T03:15:41.548 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-06T03:15:41.548 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-06T03:15:41.548 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-06T03:15:41.548 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-06T03:15:41.548 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-06T03:15:41.549 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-06T03:15:41.549 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-06T03:15:41.549 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-06T03:15:41.549 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-06T03:15:41.549 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-06T03:15:41.549 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-06T03:15:41.549 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-06T03:15:41.549 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-06T03:15:41.549 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-06T03:15:41.549 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-06T03:15:41.550 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-06T03:15:41.550 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-06T03:15:41.550 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-06T03:15:41.550 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-06T03:15:41.550 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-06T03:15:41.550 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-06T03:15:41.550 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-06T03:15:41.550 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-06T03:15:41.550 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-06T03:15:41.551 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-06T03:15:41.551 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-06T03:15:41.551 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-06T03:15:41.551 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-06T03:15:41.551 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-06T03:15:41.551 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-06T03:15:41.551 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-06T03:15:41.551 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-06T03:15:41.551 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-06T03:15:41.552 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-06T03:15:41.552 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-06T03:15:41.552 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-06T03:15:41.552 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-06T03:15:41.552 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-06T03:15:41.552 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-06T03:15:41.552 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-06T03:15:41.552 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-06T03:15:41.552 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-06T03:15:41.553 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-06T03:15:41.553 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-06T03:15:41.553 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-06T03:15:41.553 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-06T03:15:41.553 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-06T03:15:41.553 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-06T03:15:41.553 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-06T03:15:41.553 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-06T03:15:41.553 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-06T03:15:41.553 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-06T03:15:41.554 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-06T03:15:41.554 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-06T03:15:41.554 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-06T03:15:41.554 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-06T03:15:41.554 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-06T03:15:41.554 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-06T03:15:41.554 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-06T03:15:41.554 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-06T03:15:41.554 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-06T03:15:41.554 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-06T03:15:41.555 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-06T03:15:41.555 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-06T03:15:41.555 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-06T03:15:41.555 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-06T03:15:41.555 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-06T03:15:41.555 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-06T03:15:41.555 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-06T03:15:41.555 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-06T03:15:41.555 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-06T03:15:41.555 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-06T03:15:41.556 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-06T03:15:41.556 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-06T03:15:41.556 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-06T03:15:41.556 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-06T03:15:41.556 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-06T03:15:41.556 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-06T03:15:41.556 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-06T03:15:41.556 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-06T03:15:41.556 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-06T03:15:41.556 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-06T03:15:41.557 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-06T03:15:41.557 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-06T03:15:41.557 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-06T03:15:41.557 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-06T03:15:41.557 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-06T03:15:41.557 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-06T03:15:41.557 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-06T03:15:41.557 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-06T03:15:41.557 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-06T03:15:41.557 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-06T03:15:41.558 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-06T03:15:41.558 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-06T03:15:41.558 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-06T03:15:41.558 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-06T03:15:41.558 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-06T03:15:41.558 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-06T03:15:41.558 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-06T03:15:41.558 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-06T03:15:41.558 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-06T03:15:41.559 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-06T03:15:41.559 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-06T03:15:41.559 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-06T03:15:41.559 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-06T03:15:41.559 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-06T03:15:41.559 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-06T03:15:41.559 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-06T03:15:41.559 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-06T03:15:41.559 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-06T03:15:41.559 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-06T03:15:41.560 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-06T03:15:41.560 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-06T03:15:41.560 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-06T03:15:41.560 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-06T03:15:41.560 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-06T03:15:41.560 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-06T03:15:41.560 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-06T03:15:41.560 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-06T03:15:41.560 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-06T03:15:41.561 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-06T03:15:41.561 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-06T03:15:41.561 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-06T03:15:41.561 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-06T03:15:41.561 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-06T03:15:41.561 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-06T03:15:41.561 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-06T03:15:41.561 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-06T03:15:41.561 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-06T03:15:41.562 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-06T03:15:41.562 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-06T03:15:41.562 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-06T03:15:41.562 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-06T03:15:41.562 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-06T03:15:41.562 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-06T03:15:41.562 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-06T03:15:41.562 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-06T03:15:41.562 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-06T03:15:41.563 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-06T03:15:41.563 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-06T03:15:41.563 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-06T03:15:41.563 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-06T03:15:41.563 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-06T03:15:41.563 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-06T03:15:41.563 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-06T03:15:41.563 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-06T03:15:41.563 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-06T03:15:41.563 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-06T03:15:41.564 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-06T03:15:41.564 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-06T03:15:41.564 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-06T03:15:41.564 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-06T03:15:41.564 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-06T03:15:41.564 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-06T03:15:41.564 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-06T03:15:41.564 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-06T03:15:41.564 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-06T03:15:41.565 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-06T03:15:41.565 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-06T03:15:41.565 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-06T03:15:41.565 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-06T03:15:41.565 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-06T03:15:41.565 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-06T03:15:41.565 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-06T03:15:41.565 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-06T03:15:41.565 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-06T03:15:41.565 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-06T03:15:41.566 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-06T03:15:41.566 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-06T03:15:41.566 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-06T03:15:41.566 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-06T03:15:41.566 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-06T03:15:41.566 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-06T03:15:41.566 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-06T03:15:41.566 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-06T03:15:41.566 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-06T03:15:41.567 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-06T03:15:41.567 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-06T03:15:41.567 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-06T03:15:41.567 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-06T03:15:41.567 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-06T03:15:41.567 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-06T03:15:41.567 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-06T03:15:41.567 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-06T03:15:41.567 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-06T03:15:41.567 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-06T03:15:41.568 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-06T03:15:41.568 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-06T03:15:41.568 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-06T03:15:41.568 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-06T03:15:41.568 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-06T03:15:41.568 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-06T03:15:41.568 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-06T03:15:41.568 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-06T03:15:41.568 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-06T03:15:41.569 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-06T03:15:41.569 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-06T03:15:41.569 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-06T03:15:41.569 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-06T03:15:41.569 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-06T03:15:41.569 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-06T03:15:41.569 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-06T03:15:41.569 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-06T03:15:41.569 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-06T03:15:41.569 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-06T03:15:41.570 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-06T03:15:41.570 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-06T03:15:41.570 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-06T03:15:41.570 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-06T03:15:41.570 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-06T03:15:41.570 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-06T03:15:41.570 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-06T03:15:41.570 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-06T03:15:41.570 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-06T03:15:41.571 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-06T03:15:41.571 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-06T03:15:41.571 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-06T03:15:41.571 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-06T03:15:41.571 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-06T03:15:41.571 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-06T03:15:41.571 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-06T03:15:41.571 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-06T03:15:41.571 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-06T03:15:41.571 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-06T03:15:41.572 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-06T03:15:41.572 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-06T03:15:41.572 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-06T03:15:41.572 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-06T03:15:41.572 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-06T03:15:41.572 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-06T03:15:41.572 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-06T03:15:41.572 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-06T03:15:41.572 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-06T03:15:41.573 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-06T03:15:41.573 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-06T03:15:41.573 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-06T03:15:41.573 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-06T03:15:41.573 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-06T03:15:41.573 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-06T03:15:41.573 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-06T03:15:41.573 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-06T03:15:41.574 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-06T03:15:41.574 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-06T03:15:41.574 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-06T03:15:41.574 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-06T03:15:41.574 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-06T03:15:41.574 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-06T03:15:41.574 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-06T03:15:41.574 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-06T03:15:41.574 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-06T03:15:41.574 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-06T03:15:41.575 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-06T03:15:41.575 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-06T03:15:41.575 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-06T03:15:41.575 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-06T03:15:41.575 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-06T03:15:41.575 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-06T03:15:41.575 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-06T03:15:41.575 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-06T03:15:41.575 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-06T03:15:41.576 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-06T03:15:41.576 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-06T03:15:41.576 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-06T03:15:41.576 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-06T03:15:41.576 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-06T03:15:41.576 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-06T03:15:41.576 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-06T03:15:41.576 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-06T03:15:41.576 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-06T03:15:41.577 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-06T03:15:41.577 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-06T03:15:41.577 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-06T03:15:41.577 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-06T03:15:41.577 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-06T03:15:41.577 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-06T03:15:41.577 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-06T03:15:41.577 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-06T03:15:41.577 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-06T03:15:41.578 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-06T03:15:41.578 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-06T03:15:41.578 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-06T03:15:41.578 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-06T03:15:41.578 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-06T03:15:41.578 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-06T03:15:41.578 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-06T03:15:41.578 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-06T03:15:41.578 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-06T03:15:41.579 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-06T03:15:41.579 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-06T03:15:41.579 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-06T03:15:41.579 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-06T03:15:41.579 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-06T03:15:41.579 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-06T03:15:41.579 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-06T03:15:41.579 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-06T03:15:41.579 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-06T03:15:41.579 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-06T03:15:41.580 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-06T03:15:41.580 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-06T03:15:41.580 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-06T03:15:41.580 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-06T03:15:41.580 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-06T03:15:41.580 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-06T03:15:41.580 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-06T03:15:41.580 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-06T03:15:41.580 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-06T03:15:41.580 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-06T03:15:41.581 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-06T03:15:41.581 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-06T03:15:41.581 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-06T03:15:41.581 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-06T03:15:41.581 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-06T03:15:41.581 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-06T03:15:41.581 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-06T03:15:41.581 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-06T03:15:41.581 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-06T03:15:41.582 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-06T03:15:41.582 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-06T03:15:41.582 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-06T03:15:41.582 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-06T03:15:41.582 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-06T03:15:41.582 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-06T03:15:41.582 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-06T03:15:41.582 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-06T03:15:41.582 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-06T03:15:41.583 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-06T03:15:41.583 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-06T03:15:41.583 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-06T03:15:41.583 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-06T03:15:41.583 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-06T03:15:41.583 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-06T03:15:41.583 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-06T03:15:41.583 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-06T03:15:41.583 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-06T03:15:41.584 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-06T03:15:41.584 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-06T03:15:41.584 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-06T03:15:41.584 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-06T03:15:41.584 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-06T03:15:41.584 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-06T03:15:41.584 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-06T03:15:41.584 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-06T03:15:41.584 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-06T03:15:41.584 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-06T03:15:41.585 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-06T03:15:41.585 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-06T03:15:41.585 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-06T03:15:41.585 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-06T03:15:41.585 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-06T03:15:41.585 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-06T03:15:41.585 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-06T03:15:41.585 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-06T03:15:41.585 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-06T03:15:41.585 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-06T03:15:41.586 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-06T03:15:41.586 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-06T03:15:41.586 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-06T03:15:41.586 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-06T03:15:41.586 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-06T03:15:41.586 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-06T03:15:41.586 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-06T03:15:41.586 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-06T03:15:41.586 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-06T03:15:41.587 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-06T03:15:41.587 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-06T03:15:41.587 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-06T03:15:41.587 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-06T03:15:41.587 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-06T03:15:41.587 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-06T03:15:41.587 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-06T03:15:41.587 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-06T03:15:41.587 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-06T03:15:41.587 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-06T03:15:41.588 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-06T03:15:41.588 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-06T03:15:41.588 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-06T03:15:41.588 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-06T03:15:41.588 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-06T03:15:41.588 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-06T03:15:41.588 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-06T03:15:41.588 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-06T03:15:41.588 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-06T03:15:41.588 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-06T03:15:41.589 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-06T03:15:41.589 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-06T03:15:41.589 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-06T03:15:41.589 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-06T03:15:41.589 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-06T03:15:41.589 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-06T03:15:41.589 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-06T03:15:41.589 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-06T03:15:41.589 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-06T03:15:41.589 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-06T03:15:41.590 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-06T03:15:41.590 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-06T03:15:41.590 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-06T03:15:41.590 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-06T03:15:41.590 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-06T03:15:41.590 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-06T03:15:41.590 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-06T03:15:41.590 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-06T03:15:41.590 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-06T03:15:41.591 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-06T03:15:41.591 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-06T03:15:41.591 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-06T03:15:41.591 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-06T03:15:41.591 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-06T03:15:41.591 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-06T03:15:41.591 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-06T03:15:41.591 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-06T03:15:41.591 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-06T03:15:41.591 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-06T03:15:41.592 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-06T03:15:41.592 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-06T03:15:41.592 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-06T03:15:41.592 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-06T03:15:41.592 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-06T03:15:41.592 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-06T03:15:41.592 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-06T03:15:41.592 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-06T03:15:41.592 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-06T03:15:41.593 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-06T03:15:41.593 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-06T03:15:41.593 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-06T03:15:41.593 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-06T03:15:41.593 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-06T03:15:41.593 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-06T03:15:41.593 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-06T03:15:41.593 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-06T03:15:41.593 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-06T03:15:41.593 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-06T03:15:41.594 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-06T03:15:41.594 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-06T03:15:41.594 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-06T03:15:41.594 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-06T03:15:41.594 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-06T03:15:41.594 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-06T03:15:41.594 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-06T03:15:41.594 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-06T03:15:41.594 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-06T03:15:41.595 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-06T03:15:41.595 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-06T03:15:41.595 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-06T03:15:41.595 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-06T03:15:41.595 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-06T03:15:41.595 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-06T03:15:41.595 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-06T03:15:41.595 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-06T03:15:41.595 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-06T03:15:41.595 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-06T03:15:41.596 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-06T03:15:41.596 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-06T03:15:41.596 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-06T03:15:41.596 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-06T03:15:41.596 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-06T03:15:41.596 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-06T03:15:41.596 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-06T03:15:41.596 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-06T03:15:41.596 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-06T03:15:41.596 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-06T03:15:41.597 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-06T03:15:41.597 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-06T03:15:41.597 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-06T03:15:41.597 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-06T03:15:41.597 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-06T03:15:41.597 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-06T03:15:41.597 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-06T03:15:41.597 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-06T03:15:41.597 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-06T03:15:41.597 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-06T03:15:41.598 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-06T03:15:41.598 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-06T03:15:41.598 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-06T03:15:41.598 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-06T03:15:41.598 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-06T03:15:41.598 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-06T03:15:41.598 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-06T03:15:41.598 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-06T03:15:41.598 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-06T03:15:41.599 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-06T03:15:41.599 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-06T03:15:41.599 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-06T03:15:41.599 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-06T03:15:41.599 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-06T03:15:41.599 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-06T03:15:41.599 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-06T03:15:41.599 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-06T03:15:41.599 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-06T03:15:41.599 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-06T03:15:41.600 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-06T03:15:41.600 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-06T03:15:41.600 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-06T03:15:41.600 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-06T03:15:41.600 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-06T03:15:41.600 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-06T03:15:41.600 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-06T03:15:41.600 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-06T03:15:41.600 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-06T03:15:41.600 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-06T03:15:41.601 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-06T03:15:41.601 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-06T03:15:41.601 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-06T03:15:41.601 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-06T03:15:41.601 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-06T03:15:41.601 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-06T03:15:41.601 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-06T03:15:41.601 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-06T03:15:41.601 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-06T03:15:41.601 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-06T03:15:41.602 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-06T03:15:41.602 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-06T03:15:41.602 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-06T03:15:41.602 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-06T03:15:41.602 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-06T03:15:41.602 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-06T03:15:41.602 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-06T03:15:41.602 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-06T03:15:41.602 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-06T03:15:41.602 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-06T03:15:41.603 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-06T03:15:41.603 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-06T03:15:41.603 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-06T03:15:41.603 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-06T03:15:41.603 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-06T03:15:41.603 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-06T03:15:41.603 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-06T03:15:41.603 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-06T03:15:41.603 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-06T03:15:41.603 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-06T03:15:41.604 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-06T03:15:41.604 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-06T03:15:41.604 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-06T03:15:41.604 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-06T03:15:41.604 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-06T03:15:41.604 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-06T03:15:41.604 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-06T03:15:41.604 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-06T03:15:41.604 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-06T03:15:41.604 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-06T03:15:41.605 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-06T03:15:41.605 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-06T03:15:41.605 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-06T03:15:41.605 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-06T03:15:41.605 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-06T03:15:41.605 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-06T03:15:41.605 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-06T03:15:41.605 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-06T03:15:41.605 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-06T03:15:41.605 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-06T03:15:41.605 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-06T03:15:41.606 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-06T03:15:41.606 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-06T03:15:41.606 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-06T03:15:41.606 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-06T03:15:41.606 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-06T03:15:41.606 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-06T03:15:41.606 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-06T03:15:41.606 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-06T03:15:41.606 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-06T03:15:41.607 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-06T03:15:41.607 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-06T03:15:41.607 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-06T03:15:41.607 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-06T03:15:41.607 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-06T03:15:41.607 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-06T03:15:41.607 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-06T03:15:41.607 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-06T03:15:41.607 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-06T03:15:41.607 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-06T03:15:41.608 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-06T03:15:41.608 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-06T03:15:41.608 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-06T03:15:41.608 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-06T03:15:41.608 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-06T03:15:41.608 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-06T03:15:41.608 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-06T03:15:41.608 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-06T03:15:41.608 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-06T03:15:41.609 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-06T03:15:41.609 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-06T03:15:41.609 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-06T03:15:41.609 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-06T03:15:41.609 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-06T03:15:41.609 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-06T03:15:41.609 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-06T03:15:41.609 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-06T03:15:41.609 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-06T03:15:41.609 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-06T03:15:41.610 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-06T03:15:41.610 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-06T03:15:41.610 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-06T03:15:41.610 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-06T03:15:41.610 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-06T03:15:41.610 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-06T03:15:41.610 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-06T03:15:41.610 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-06T03:15:41.610 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-06T03:15:41.610 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-06T03:15:41.611 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-06T03:15:41.611 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-06T03:15:41.611 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-06T03:15:41.611 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-06T03:15:41.611 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-06T03:15:41.611 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-06T03:15:41.611 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-06T03:15:41.611 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-06T03:15:41.611 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-06T03:15:41.612 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-06T03:15:41.612 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-06T03:15:41.612 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-06T03:15:41.612 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-06T03:15:41.612 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-06T03:15:41.612 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-06T03:15:41.612 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-06T03:15:41.612 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-06T03:15:41.612 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-06T03:15:41.612 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-06T03:15:41.613 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-06T03:15:41.613 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-06T03:15:41.613 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-06T03:15:41.613 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-06T03:15:41.613 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-06T03:15:41.613 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-06T03:15:41.613 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-06T03:15:41.613 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-06T03:15:41.613 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-06T03:15:41.614 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-06T03:15:41.614 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-06T03:15:41.614 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-06T03:15:41.614 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-06T03:15:41.614 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-06T03:15:41.614 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-06T03:15:41.614 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-06T03:15:41.614 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-06T03:15:41.614 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-06T03:15:41.614 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-06T03:15:41.615 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-06T03:15:41.615 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-06T03:15:41.615 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-06T03:15:41.615 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-06T03:15:41.615 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-06T03:15:41.615 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-06T03:15:41.615 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-06T03:15:41.615 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-06T03:15:41.615 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-06T03:15:41.615 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-06T03:15:41.616 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-06T03:15:41.616 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-06T03:15:41.616 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-06T03:15:41.616 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-06T03:15:41.616 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-06T03:15:41.616 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-06T03:15:41.616 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-06T03:15:41.616 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-06T03:15:41.616 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-06T03:15:41.617 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-06T03:15:41.617 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-06T03:15:41.617 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-06T03:15:41.617 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-06T03:15:41.617 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-06T03:15:41.617 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-06T03:15:41.617 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-06T03:15:41.617 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-06T03:15:41.617 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-06T03:15:41.617 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-06T03:15:41.618 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-06T03:15:41.618 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-06T03:15:41.618 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-06T03:15:41.618 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-06T03:15:41.618 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-06T03:15:41.618 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-06T03:15:41.618 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-06T03:15:41.618 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-06T03:15:41.618 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-06T03:15:41.619 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-06T03:15:41.619 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-06T03:15:41.619 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-06T03:15:41.619 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-06T03:15:41.619 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-06T03:15:41.619 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-06T03:15:41.619 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-06T03:15:41.619 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-06T03:15:41.619 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-06T03:15:41.619 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-06T03:15:41.620 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-06T03:15:41.620 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-06T03:15:41.620 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-06T03:15:41.620 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-06T03:15:41.620 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-06T03:15:41.620 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-06T03:15:41.620 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-06T03:15:41.620 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-06T03:15:41.620 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-06T03:15:41.621 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-06T03:15:41.621 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-06T03:15:41.621 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-06T03:15:41.621 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-06T03:15:41.621 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-06T03:15:41.621 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-06T03:15:41.621 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-06T03:15:41.621 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-06T03:15:41.621 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-06T03:15:41.622 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-06T03:15:41.622 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-06T03:15:41.622 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-06T03:15:41.622 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-06T03:15:41.622 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-06T03:15:41.622 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-06T03:15:41.622 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-06T03:15:41.622 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-06T03:15:41.622 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-06T03:15:41.623 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-06T03:15:41.623 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-06T03:15:41.623 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-06T03:15:41.623 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-06T03:15:41.623 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-06T03:15:41.623 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-06T03:15:41.623 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-06T03:15:41.623 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-06T03:15:41.623 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-06T03:15:41.623 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-06T03:15:41.624 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-06T03:15:41.624 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-06T03:15:41.624 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-06T03:15:41.624 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-06T03:15:41.624 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-06T03:15:41.624 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-06T03:15:41.624 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-06T03:15:41.624 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-06T03:15:41.624 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-06T03:15:41.625 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-06T03:15:41.625 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-06T03:15:41.625 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-06T03:15:41.625 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-06T03:15:41.625 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-06T03:15:41.625 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-06T03:15:41.625 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-06T03:15:41.625 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-06T03:15:41.625 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-06T03:15:41.625 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-06T03:15:41.626 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-06T03:15:41.626 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-06T03:15:41.626 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-06T03:15:41.626 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-06T03:15:41.626 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-06T03:15:41.626 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-06T03:15:41.626 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-06T03:15:41.626 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-06T03:15:41.626 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-06T03:15:41.627 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-06T03:15:41.627 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-06T03:15:41.627 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-06T03:15:41.627 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-06T03:15:41.627 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-06T03:15:41.627 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-06T03:15:41.627 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-06T03:15:41.627 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-06T03:15:41.627 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-06T03:15:41.628 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-06T03:15:41.628 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-06T03:15:41.628 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-06T03:15:41.628 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-06T03:15:41.628 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-06T03:15:41.628 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-06T03:15:41.628 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-06T03:15:41.628 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-06T03:15:41.628 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-06T03:15:41.628 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-06T03:15:41.629 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-06T03:15:41.629 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-06T03:15:41.629 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-06T03:15:41.629 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-06T03:15:41.629 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-06T03:15:41.629 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-06T03:15:41.629 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-06T03:15:41.629 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-06T03:15:41.629 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-06T03:15:41.629 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-06T03:15:41.630 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-06T03:15:41.630 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-06T03:15:41.630 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-06T03:15:41.630 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-06T03:15:41.630 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-06T03:15:41.630 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-06T03:15:41.630 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-06T03:15:41.630 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-06T03:15:41.630 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-06T03:15:41.631 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-06T03:15:41.631 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-06T03:15:41.631 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-06T03:15:41.631 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-06T03:15:41.631 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-06T03:15:41.631 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-06T03:15:41.631 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-06T03:15:41.631 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-06T03:15:41.631 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-06T03:15:41.631 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-06T03:15:41.632 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-06T03:15:41.632 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-06T03:15:41.632 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-06T03:15:41.632 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-06T03:15:41.632 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-06T03:15:41.632 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-06T03:15:41.632 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-06T03:15:41.632 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-06T03:15:41.632 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-06T03:15:41.633 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-06T03:15:41.633 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-06T03:15:41.633 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-06T03:15:41.633 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-06T03:15:41.633 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-06T03:15:41.633 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-06T03:15:41.633 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-06T03:15:41.633 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-06T03:15:41.633 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-06T03:15:41.633 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-06T03:15:41.634 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-06T03:15:41.634 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-06T03:15:41.634 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-06T03:15:41.634 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-06T03:15:41.634 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-06T03:15:41.634 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-06T03:15:41.634 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-06T03:15:41.634 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-06T03:15:41.634 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-06T03:15:41.634 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-06T03:15:41.635 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-06T03:15:41.635 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-06T03:15:41.635 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-06T03:15:41.635 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-06T03:15:41.635 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-06T03:15:41.635 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-06T03:15:41.635 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-06T03:15:41.635 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-06T03:15:41.635 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-06T03:15:41.636 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-06T03:15:41.636 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-06T03:15:41.636 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-06T03:15:41.636 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-06T03:15:41.636 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-06T03:15:41.636 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-06T03:15:41.636 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-06T03:15:41.636 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-06T03:15:41.636 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-06T03:15:41.636 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-06T03:15:41.637 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-06T03:15:41.637 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-06T03:15:41.637 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-06T03:15:41.637 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-06T03:15:41.637 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-06T03:15:41.637 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-06T03:15:41.637 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-06T03:15:41.637 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-06T03:15:41.637 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-06T03:15:41.637 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-06T03:15:41.638 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-06T03:15:41.638 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-06T03:15:41.638 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-06T03:15:41.638 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-06T03:15:41.638 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-06T03:15:41.638 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-06T03:15:41.638 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-06T03:15:41.638 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-06T03:15:41.638 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-06T03:15:41.639 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-06T03:15:41.639 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-06T03:15:41.639 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-06T03:15:41.639 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-06T03:15:41.639 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-06T03:15:41.639 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-06T03:15:41.639 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-06T03:15:41.639 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-06T03:15:41.639 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-06T03:15:41.639 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-06T03:15:41.640 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-06T03:15:41.640 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-06T03:15:41.640 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-06T03:15:41.640 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-06T03:15:41.640 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-06T03:15:41.640 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-06T03:15:41.640 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-06T03:15:41.640 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-06T03:15:41.640 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-06T03:15:41.641 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-06T03:15:41.641 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-06T03:15:41.641 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-06T03:15:41.641 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-06T03:15:41.641 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-06T03:15:41.641 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-06T03:15:41.641 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-06T03:15:41.641 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-06T03:15:41.641 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-06T03:15:41.641 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-06T03:15:41.642 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-06T03:15:41.642 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-06T03:15:41.642 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-06T03:15:41.642 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-06T03:15:41.642 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-06T03:15:41.642 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-06T03:15:41.642 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-06T03:15:41.642 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-06T03:15:41.642 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-06T03:15:41.642 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-06T03:15:41.643 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-06T03:15:41.643 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-06T03:15:41.643 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-06T03:15:41.643 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-06T03:15:41.643 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-06T03:15:41.643 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-06T03:15:41.643 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-06T03:15:41.643 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-06T03:15:41.643 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-06T03:15:41.644 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-06T03:15:41.644 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-06T03:15:41.644 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-06T03:15:41.644 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-06T03:15:41.644 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-06T03:15:41.644 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-06T03:15:41.644 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-06T03:15:41.644 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-06T03:15:41.644 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-06T03:15:41.644 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-06T03:15:41.645 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-06T03:15:41.645 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-06T03:15:41.645 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-06T03:15:41.645 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-06T03:15:41.645 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-06T03:15:41.645 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-06T03:15:41.645 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-06T03:15:41.645 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-06T03:15:41.645 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-06T03:15:41.645 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-06T03:15:41.646 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-06T03:15:41.646 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-06T03:15:41.646 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-06T03:15:41.646 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-06T03:15:41.646 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-06T03:15:41.646 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-06T03:15:41.646 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-06T03:15:41.646 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-06T03:15:41.646 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-06T03:15:41.647 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-06T03:15:41.647 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-06T03:15:41.647 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-06T03:15:41.647 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-06T03:15:41.647 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-06T03:15:41.647 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-06T03:15:41.647 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-06T03:15:41.647 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-06T03:15:41.647 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-06T03:15:41.648 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-06T03:15:41.648 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-06T03:15:41.648 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-06T03:15:41.648 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-06T03:15:41.648 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-06T03:15:41.648 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-06T03:15:41.648 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-06T03:15:41.648 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-06T03:15:41.648 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-06T03:15:41.648 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-06T03:15:41.649 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-06T03:15:41.649 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-06T03:15:41.649 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-06T03:15:41.649 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-06T03:15:41.649 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-06T03:15:41.649 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-06T03:15:41.649 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-06T03:15:41.649 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-06T03:15:41.649 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-06T03:15:41.650 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-06T03:15:41.650 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-06T03:15:41.650 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-06T03:15:41.650 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-06T03:15:41.650 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-06T03:15:41.650 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-06T03:15:41.650 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-06T03:15:41.650 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-06T03:15:41.650 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-06T03:15:41.651 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-06T03:15:41.651 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-06T03:15:41.651 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-06T03:15:41.651 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-06T03:15:41.651 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-06T03:15:41.651 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-06T03:15:41.651 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-06T03:15:41.651 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-06T03:15:41.651 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-06T03:15:41.652 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-06T03:15:41.652 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-06T03:15:41.652 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-06T03:15:41.652 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-06T03:15:41.652 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-06T03:15:41.652 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-06T03:15:41.652 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-06T03:15:41.652 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-06T03:15:41.652 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-06T03:15:41.653 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-06T03:15:41.653 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-06T03:15:41.653 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-06T03:15:41.653 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-06T03:15:41.653 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-06T03:15:41.653 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-06T03:15:41.653 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-06T03:15:41.653 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-06T03:15:41.653 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-06T03:15:41.654 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-06T03:15:41.654 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-06T03:15:41.654 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-06T03:15:41.654 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-06T03:15:41.654 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-06T03:15:41.654 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-06T03:15:41.654 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-06T03:15:41.654 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-06T03:15:41.654 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-06T03:15:41.655 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-06T03:15:41.655 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-06T03:15:41.655 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-06T03:15:41.655 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-06T03:15:41.655 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-06T03:15:41.655 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-06T03:15:41.655 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-06T03:15:41.655 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-06T03:15:41.655 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-06T03:15:41.655 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-06T03:15:41.656 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-06T03:15:41.656 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-06T03:15:41.656 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-06T03:15:41.656 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-06T03:15:41.656 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-06T03:15:41.656 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-06T03:15:41.656 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-06T03:15:41.656 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-06T03:15:41.656 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-06T03:15:41.656 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-06T03:15:41.657 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-06T03:15:41.657 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-06T03:15:41.657 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-06T03:15:41.657 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-06T03:15:41.657 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-06T03:15:41.657 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-06T03:15:41.657 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-06T03:15:41.657 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-06T03:15:41.657 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-06T03:15:41.657 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-06T03:15:41.658 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-06T03:15:41.658 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-06T03:15:41.658 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-06T03:15:41.658 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-06T03:15:41.658 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-06T03:15:41.658 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-06T03:15:41.658 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-06T03:15:41.658 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-06T03:15:41.658 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-06T03:15:41.659 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-06T03:15:41.659 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-06T03:15:41.659 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-06T03:15:41.659 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-06T03:15:41.659 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-06T03:15:41.659 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-06T03:15:41.659 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-06T03:15:41.659 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-06T03:15:41.660 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-06T03:15:41.660 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-06T03:15:41.660 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-06T03:15:41.660 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-06T03:15:41.660 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-06T03:15:41.660 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-06T03:15:41.660 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-06T03:15:41.660 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-06T03:15:41.660 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-06T03:15:41.661 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-06T03:15:41.661 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-06T03:15:41.661 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-06T03:15:41.661 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-06T03:15:41.661 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-06T03:15:41.661 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-06T03:15:41.661 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-06T03:15:41.661 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-06T03:15:41.661 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-06T03:15:41.661 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-06T03:15:41.662 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-06T03:15:41.662 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-06T03:15:41.662 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-06T03:15:41.662 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-06T03:15:41.662 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-06T03:15:41.662 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-06T03:15:41.662 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-06T03:15:41.662 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-06T03:15:41.662 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-06T03:15:41.663 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-06T03:15:41.663 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-06T03:15:41.663 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-06T03:15:41.663 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-06T03:15:41.663 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-06T03:15:41.663 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-06T03:15:41.663 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-06T03:15:41.663 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-06T03:15:41.663 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-06T03:15:41.663 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-06T03:15:41.664 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-06T03:15:41.664 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-06T03:15:41.664 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-06T03:15:41.664 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-06T03:15:41.664 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-06T03:15:41.664 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-06T03:15:41.664 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-06T03:15:41.664 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-06T03:15:41.664 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-06T03:15:41.664 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-06T03:15:41.665 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-06T03:15:41.665 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-06T03:15:41.665 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-06T03:15:41.665 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-06T03:15:41.665 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-06T03:15:41.665 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-06T03:15:41.665 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-06T03:15:41.665 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-06T03:15:41.665 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-06T03:15:41.665 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-06T03:15:41.666 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-06T03:15:41.666 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-06T03:15:41.666 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-06T03:15:41.666 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-06T03:15:41.666 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-06T03:15:41.666 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-06T03:15:41.666 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-06T03:15:41.666 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-06T03:15:41.666 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-06T03:15:41.667 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-06T03:15:41.667 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-06T03:15:41.667 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-06T03:15:41.667 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-06T03:15:41.667 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-06T03:15:41.667 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-06T03:15:41.667 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-06T03:15:41.667 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-06T03:15:41.667 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-06T03:15:41.668 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-06T03:15:41.668 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-06T03:15:41.668 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-06T03:15:41.668 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-06T03:15:41.668 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-06T03:15:41.668 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-06T03:15:41.668 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-06T03:15:41.668 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-06T03:15:41.668 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-06T03:15:41.668 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-06T03:15:41.669 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-06T03:15:41.669 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-06T03:15:41.669 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-06T03:15:41.669 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-06T03:15:41.669 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-06T03:15:41.669 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-06T03:15:41.669 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-06T03:15:41.669 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-06T03:15:41.669 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-06T03:15:41.669 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-06T03:15:41.670 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-06T03:15:41.670 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-06T03:15:41.670 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-06T03:15:41.670 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-06T03:15:41.670 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-06T03:15:41.670 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-06T03:15:41.670 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-06T03:15:41.670 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-06T03:15:41.671 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-06T03:15:41.671 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-06T03:15:41.671 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-06T03:15:41.671 INFO:teuthology.orchestra.run.smithi055.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-06T03:15:41.671 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:41.672 DEBUG:teuthology.orchestra.run.smithi055:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-06T03:15:41.696 INFO:teuthology.orchestra.run.smithi055.stdout:postinstall scriptlet (using /bin/sh): 2023-10-06T03:15:41.697 INFO:teuthology.orchestra.run.smithi055.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-06T03:15:41.697 INFO:teuthology.orchestra.run.smithi055.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T03:15:41.697 INFO:teuthology.orchestra.run.smithi055.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T03:15:41.697 INFO:teuthology.orchestra.run.smithi055.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:41.697 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T03:15:41.697 INFO:teuthology.orchestra.run.smithi055.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-06T03:15:41.698 INFO:teuthology.orchestra.run.smithi055.stdout:fi 2023-10-06T03:15:41.698 INFO:teuthology.orchestra.run.smithi055.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-06T03:15:41.698 INFO:teuthology.orchestra.run.smithi055.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-06T03:15:41.698 INFO:teuthology.orchestra.run.smithi055.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-06T03:15:41.698 INFO:teuthology.orchestra.run.smithi055.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-06T03:15:41.698 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:41.698 INFO:teuthology.orchestra.run.smithi055.stdout:fi 2023-10-06T03:15:41.698 INFO:teuthology.orchestra.run.smithi055.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-06T03:15:41.698 INFO:teuthology.orchestra.run.smithi055.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-06T03:15:41.698 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:41.699 INFO:teuthology.orchestra.run.smithi055.stdout:fi 2023-10-06T03:15:41.700 DEBUG:teuthology.orchestra.run.smithi055:> sudo rpm -qi grub2-tools 2023-10-06T03:15:41.776 INFO:teuthology.orchestra.run.smithi055.stdout:Name : grub2-tools 2023-10-06T03:15:41.776 INFO:teuthology.orchestra.run.smithi055.stdout:Epoch : 1 2023-10-06T03:15:41.776 INFO:teuthology.orchestra.run.smithi055.stdout:Version : 2.02 2023-10-06T03:15:41.776 INFO:teuthology.orchestra.run.smithi055.stdout:Release : 99.el8_4.1 2023-10-06T03:15:41.776 INFO:teuthology.orchestra.run.smithi055.stdout:Architecture: x86_64 2023-10-06T03:15:41.776 INFO:teuthology.orchestra.run.smithi055.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-06T03:15:41.776 INFO:teuthology.orchestra.run.smithi055.stdout:Group : System Environment/Base 2023-10-06T03:15:41.776 INFO:teuthology.orchestra.run.smithi055.stdout:Size : 9264631 2023-10-06T03:15:41.777 INFO:teuthology.orchestra.run.smithi055.stdout:License : GPLv3+ 2023-10-06T03:15:41.777 INFO:teuthology.orchestra.run.smithi055.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-06T03:15:41.777 INFO:teuthology.orchestra.run.smithi055.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-06T03:15:41.777 INFO:teuthology.orchestra.run.smithi055.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-06T03:15:41.777 INFO:teuthology.orchestra.run.smithi055.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-06T03:15:41.777 INFO:teuthology.orchestra.run.smithi055.stdout:Relocations : (not relocatable) 2023-10-06T03:15:41.777 INFO:teuthology.orchestra.run.smithi055.stdout:Packager : Red Hat, Inc. 2023-10-06T03:15:41.778 INFO:teuthology.orchestra.run.smithi055.stdout:Vendor : Red Hat, Inc. 2023-10-06T03:15:41.778 INFO:teuthology.orchestra.run.smithi055.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T03:15:41.778 INFO:teuthology.orchestra.run.smithi055.stdout:Summary : Support tools for GRUB. 2023-10-06T03:15:41.778 INFO:teuthology.orchestra.run.smithi055.stdout:Description : 2023-10-06T03:15:41.778 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:15:41.778 INFO:teuthology.orchestra.run.smithi055.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T03:15:41.778 INFO:teuthology.orchestra.run.smithi055.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T03:15:41.778 INFO:teuthology.orchestra.run.smithi055.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T03:15:41.778 INFO:teuthology.orchestra.run.smithi055.stdout:hardware devices. 2023-10-06T03:15:41.779 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:15:41.779 INFO:teuthology.orchestra.run.smithi055.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T03:15:41.780 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T03:15:41.780 INFO:teuthology.task.kernel:Updating grub on smithi055 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:41.780 DEBUG:teuthology.orchestra.run.smithi055:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T03:15:44.464 INFO:teuthology.orchestra.run.smithi055.stderr:Generating grub configuration file ... 2023-10-06T03:15:46.822 INFO:teuthology.orchestra.run.smithi055.stderr:done 2023-10-06T03:15:46.824 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2023-10-06T03:15:46.839 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.Pfh1kTXR13 2023-10-06T03:15:46.839 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Pfh1kTXR13 2023-10-06T03:15:46.907 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.Pfh1kTXR13 2023-10-06T03:15:47.068 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.Pfh1kTXR13 is 6KB 2023-10-06T03:15:47.116 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.Pfh1kTXR13 2023-10-06T03:15:47.130 DEBUG:teuthology.orchestra.run.smithi055:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T03:15:47.197 INFO:teuthology.orchestra.run.smithi055.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-06T03:15:47.198 INFO:teuthology.orchestra.run.smithi055.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-06T03:15:47.198 INFO:teuthology.orchestra.run.smithi055.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-06T03:15:47.198 INFO:teuthology.orchestra.run.smithi055.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-06T03:15:47.198 INFO:teuthology.orchestra.run.smithi055.stdout:298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b.conf 2023-10-06T03:15:47.199 DEBUG:teuthology.orchestra.run.smithi055:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b 2023-10-06T03:15:47.384 DEBUG:teuthology.orchestra.run.smithi055:> sudo shutdown -r now 2023-10-06T03:16:07.831 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-06T03:16:07.831 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-10-06T03:16:07.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:16:17.394 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-06T03:16:17.395 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-10-06T03:16:17.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:16:42.490 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2023-10-06T03:16:51.498 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-10-06T03:16:51.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:16:54.554 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.163 2023-10-06T03:17:06.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-10-06T03:17:06.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:17:17.406 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T03:17:26.414 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-10-06T03:17:26.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:17:26.740 DEBUG:teuthology.orchestra.run.smithi055:> true 2023-10-06T03:17:27.093 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-10-06T03:17:27.093 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T03:17:27.093 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2023-10-06T03:17:27.172 INFO:teuthology.orchestra.run.smithi055.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T03:17:27.173 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T03:17:27.173 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T03:17:27.173 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T03:17:28.174 DEBUG:teuthology.parallel:result is None 2023-10-06T03:18:06.569 DEBUG:teuthology.orchestra.remote:timed out 2023-10-06T03:18:21.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-10-06T03:18:21.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:18:21.957 DEBUG:teuthology.orchestra.run.smithi163:> true 2023-10-06T03:18:22.353 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-10-06T03:18:22.353 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-06T03:18:22.353 DEBUG:teuthology.orchestra.run.smithi163:> uname -r 2023-10-06T03:18:22.389 INFO:teuthology.orchestra.run.smithi163.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-06T03:18:22.390 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-06T03:18:22.390 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T03:18:22.390 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-06T03:18:23.392 DEBUG:teuthology.parallel:result is None 2023-10-06T03:18:23.392 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T03:18:23.399 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T03:18:23.399 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T03:18:23.405 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T03:18:23.444 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T03:18:23.450 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T03:18:23.457 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T03:18:23.457 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T03:18:23.487 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T03:18:23.541 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T03:18:23.548 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T03:18:23.548 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T03:18:23.574 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T03:18:23.708 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T03:18:23.718 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T03:18:23.743 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T03:18:23.753 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T03:18:23.757 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T03:18:23.764 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T03:18:23.764 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T03:18:23.770 DEBUG:teuthology.orchestra.run.smithi163:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T03:18:23.831 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T03:18:23.839 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T03:18:23.840 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T03:18:23.844 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T03:18:23.879 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T03:18:23.985 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T03:18:24.072 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T03:18:24.137 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T03:18:24.214 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:18:24.214 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T03:18:24.288 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T03:18:24.355 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T03:18:24.453 DEBUG:teuthology.orchestra.run.smithi163:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T03:18:24.518 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T03:18:24.592 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:18:24.592 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T03:18:24.654 DEBUG:teuthology.orchestra.run.smithi055:> sudo service rsyslog restart 2023-10-06T03:18:24.659 DEBUG:teuthology.orchestra.run.smithi163:> sudo service rsyslog restart 2023-10-06T03:18:24.781 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T03:18:24.809 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T03:18:25.175 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T03:18:25.246 INFO:teuthology.task.internal:Starting timer... 2023-10-06T03:18:25.246 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T03:18:25.255 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T03:18:25.264 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T03:18:25.265 DEBUG:teuthology.orchestra.run.smithi055:> sudo service auditd rotate 2023-10-06T03:18:25.370 INFO:teuthology.orchestra.run.smithi055.stdout:Rotating logs: [ OK ] 2023-10-06T03:18:25.374 DEBUG:teuthology.orchestra.run.smithi163:> sudo service auditd rotate 2023-10-06T03:18:25.488 INFO:teuthology.orchestra.run.smithi163.stdout:Rotating logs: [ OK ] 2023-10-06T03:18:25.492 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T03:18:25.492 DEBUG:teuthology.orchestra.run.smithi055:> /usr/sbin/getenforce 2023-10-06T03:18:25.536 INFO:teuthology.orchestra.run.smithi055.stdout:Permissive 2023-10-06T03:18:25.536 DEBUG:teuthology.orchestra.run.smithi163:> /usr/sbin/getenforce 2023-10-06T03:18:25.580 INFO:teuthology.orchestra.run.smithi163.stdout:Permissive 2023-10-06T03:18:25.580 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi055.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi163.front.sepia.ceph.com': 'permissive'} 2023-10-06T03:18:25.580 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T03:18:25.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:18:25.646 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T03:18:25.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:18:25.722 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T03:18:25.722 DEBUG:teuthology.orchestra.run.smithi055:> sudo /usr/sbin/setenforce permissive 2023-10-06T03:18:25.786 DEBUG:teuthology.orchestra.run.smithi163:> sudo /usr/sbin/setenforce permissive 2023-10-06T03:18:25.850 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T03:18:25.859 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-10-06T03:18:25.864 INFO:teuthology.repo_utils:Fetching main from origin 2023-10-06T03:18:25.976 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T03:18:25.994 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T03:18:25.995 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi055.front.sepia.ceph.com,smithi163.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T03:29:46.646 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi055.front.sepia.ceph.com'), Remote(name='ubuntu@smithi163.front.sepia.ceph.com')] 2023-10-06T03:29:46.647 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-10-06T03:29:46.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:29:46.736 DEBUG:teuthology.orchestra.run.smithi055:> true 2023-10-06T03:29:46.850 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2023-10-06T03:29:46.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-10-06T03:29:46.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi163.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T03:29:46.941 DEBUG:teuthology.orchestra.run.smithi163:> true 2023-10-06T03:29:47.047 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi163.front.sepia.ceph.com' 2023-10-06T03:29:47.047 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T03:29:47.058 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T03:29:47.058 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T03:29:47.058 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T03:29:47.062 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T03:29:47.063 DEBUG:teuthology.orchestra.run.smithi163:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T03:29:47.128 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T03:29:47.156 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T03:29:47.163 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T03:29:47.190 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T03:29:47.225 INFO:teuthology.orchestra.run.smithi055.stderr:sudo: ntpd: command not found 2023-10-06T03:29:47.251 INFO:teuthology.orchestra.run.smithi163.stderr:sudo: ntpd: command not found 2023-10-06T03:29:47.252 INFO:teuthology.orchestra.run.smithi055.stdout:506 Cannot talk to daemon 2023-10-06T03:29:47.278 INFO:teuthology.orchestra.run.smithi163.stdout:506 Cannot talk to daemon 2023-10-06T03:29:47.287 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T03:29:47.311 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T03:29:47.321 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T03:29:47.345 INFO:teuthology.orchestra.run.smithi163.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T03:29:47.407 INFO:teuthology.orchestra.run.smithi055.stderr:bash: ntpq: command not found 2023-10-06T03:29:47.412 INFO:teuthology.orchestra.run.smithi055.stdout:210 Number of sources = 4 2023-10-06T03:29:47.412 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T03:29:47.412 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2023-10-06T03:29:47.412 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:29:47.412 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:29:47.412 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:29:47.412 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:29:47.428 INFO:teuthology.orchestra.run.smithi163.stderr:bash: ntpq: command not found 2023-10-06T03:29:47.433 INFO:teuthology.orchestra.run.smithi163.stdout:210 Number of sources = 4 2023-10-06T03:29:47.433 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T03:29:47.433 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2023-10-06T03:29:47.433 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:29:47.434 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:29:47.434 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:29:47.434 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T03:29:47.435 INFO:teuthology.run_tasks:Running task install... 2023-10-06T03:29:47.443 DEBUG:teuthology.task.install:project ceph 2023-10-06T03:29:47.443 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}} 2023-10-06T03:29:47.443 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'} 2023-10-06T03:29:47.443 INFO:teuthology.task.install:Using flavor: default 2023-10-06T03:29:47.449 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T03:29:47.449 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T03:29:47.449 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'tag': None, 'wait_for_package': False} 2023-10-06T03:29:47.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T03:29:47.451 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'tag': None, 'wait_for_package': False} 2023-10-06T03:29:47.452 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T03:29:47.594 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/ 2023-10-06T03:29:47.594 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1296.g736ac5ba 2023-10-06T03:29:47.693 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/ 2023-10-06T03:29:47.693 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1296.g736ac5ba 2023-10-06T03:29:47.792 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T03:29:47.792 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:29:47.792 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T03:29:47.801 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T03:29:47.801 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:29:47.801 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T03:29:47.855 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T03:29:47.856 DEBUG:teuthology.orchestra.run.smithi163:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T03:29:47.865 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T03:29:47.865 DEBUG:teuthology.orchestra.run.smithi055:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T03:29:47.959 DEBUG:teuthology.orchestra.run.smithi163:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T03:29:47.968 DEBUG:teuthology.orchestra.run.smithi055:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T03:29:48.048 DEBUG:teuthology.orchestra.run.smithi163:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T03:29:48.058 DEBUG:teuthology.orchestra.run.smithi055:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T03:29:48.109 INFO:teuthology.orchestra.run.smithi163.stdout:check_obsoletes = 1 2023-10-06T03:29:48.112 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum clean all 2023-10-06T03:29:48.119 INFO:teuthology.orchestra.run.smithi055.stdout:check_obsoletes = 1 2023-10-06T03:29:48.123 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum clean all 2023-10-06T03:29:48.574 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2023-10-06T03:29:48.618 INFO:teuthology.orchestra.run.smithi055.stdout:Updating Subscription Management repositories. 2023-10-06T03:29:48.942 INFO:teuthology.orchestra.run.smithi163.stdout:44 files removed 2023-10-06T03:29:49.007 DEBUG:teuthology.orchestra.run.smithi163:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T03:29:49.030 INFO:teuthology.orchestra.run.smithi055.stdout:44 files removed 2023-10-06T03:29:49.089 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T03:29:49.475 INFO:teuthology.orchestra.run.smithi163.stdout:Updating Subscription Management repositories. 2023-10-06T03:29:49.584 INFO:teuthology.orchestra.run.smithi055.stdout:Updating Subscription Management repositories. 2023-10-06T03:29:50.328 INFO:teuthology.orchestra.run.smithi163.stdout:ceph packages for x86_64 248 kB/s | 78 kB 00:00 2023-10-06T03:29:50.496 INFO:teuthology.orchestra.run.smithi055.stdout:ceph packages for x86_64 230 kB/s | 78 kB 00:00 2023-10-06T03:29:50.676 INFO:teuthology.orchestra.run.smithi163.stdout:ceph noarch packages 30 kB/s | 9.2 kB 00:00 2023-10-06T03:29:50.850 INFO:teuthology.orchestra.run.smithi055.stdout:ceph noarch packages 30 kB/s | 9.2 kB 00:00 2023-10-06T03:29:51.010 INFO:teuthology.orchestra.run.smithi163.stdout:ceph source packages 4.8 kB/s | 1.4 kB 00:00 2023-10-06T03:29:51.123 INFO:teuthology.orchestra.run.smithi163.stdout:Copr repo for python3-asyncssh owned by ceph 42 kB/s | 3.5 kB 00:00 2023-10-06T03:29:51.127 INFO:teuthology.orchestra.run.smithi055.stdout:ceph source packages 5.9 kB/s | 1.4 kB 00:00 2023-10-06T03:29:51.245 INFO:teuthology.orchestra.run.smithi055.stdout:Copr repo for python3-asyncssh owned by ceph 41 kB/s | 3.5 kB 00:00 2023-10-06T03:29:52.202 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-10-06T03:29:52.921 INFO:teuthology.orchestra.run.smithi163.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 16 MB 00:01 2023-10-06T03:29:55.939 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 3.8 MB/s | 24 kB 00:00 2023-10-06T03:29:56.394 INFO:teuthology.orchestra.run.smithi163.stdout:lab-extras 3.9 MB/s | 24 kB 00:00 2023-10-06T03:29:56.497 INFO:teuthology.orchestra.run.smithi055.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 70 MB/s | 33 MB 00:00 2023-10-06T03:29:56.919 INFO:teuthology.orchestra.run.smithi163.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 86 MB/s | 38 MB 00:00 2023-10-06T03:30:02.534 INFO:teuthology.orchestra.run.smithi055.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 72 MB/s | 38 MB 00:00 2023-10-06T03:30:02.768 INFO:teuthology.orchestra.run.smithi163.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 84 MB/s | 33 MB 00:00 2023-10-06T03:30:11.121 INFO:teuthology.orchestra.run.smithi163.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T03:30:11.121 INFO:teuthology.orchestra.run.smithi163.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T03:30:11.132 INFO:teuthology.orchestra.run.smithi163.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T03:30:11.133 INFO:teuthology.orchestra.run.smithi163.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T03:30:11.134 INFO:teuthology.orchestra.run.smithi163.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T03:30:11.135 INFO:teuthology.orchestra.run.smithi163.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T03:30:11.137 INFO:teuthology.orchestra.run.smithi163.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T03:30:11.139 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T03:30:11.140 INFO:teuthology.orchestra.run.smithi163.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T03:30:11.598 INFO:teuthology.orchestra.run.smithi055.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T03:30:11.599 INFO:teuthology.orchestra.run.smithi055.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T03:30:11.610 INFO:teuthology.orchestra.run.smithi055.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-06T03:30:11.611 INFO:teuthology.orchestra.run.smithi055.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-06T03:30:11.612 INFO:teuthology.orchestra.run.smithi055.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-06T03:30:11.613 INFO:teuthology.orchestra.run.smithi055.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T03:30:11.615 INFO:teuthology.orchestra.run.smithi055.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T03:30:11.617 INFO:teuthology.orchestra.run.smithi055.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-06T03:30:11.618 INFO:teuthology.orchestra.run.smithi055.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-06T03:30:11.660 INFO:teuthology.orchestra.run.smithi163.stdout:Dependencies resolved. 2023-10-06T03:30:11.672 INFO:teuthology.orchestra.run.smithi163.stdout:====================================================================================================================== 2023-10-06T03:30:11.672 INFO:teuthology.orchestra.run.smithi163.stdout: Package Arch Version Repository Size 2023-10-06T03:30:11.672 INFO:teuthology.orchestra.run.smithi163.stdout:====================================================================================================================== 2023-10-06T03:30:11.672 INFO:teuthology.orchestra.run.smithi163.stdout:Installing: 2023-10-06T03:30:11.672 INFO:teuthology.orchestra.run.smithi163.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T03:30:11.672 INFO:teuthology.orchestra.run.smithi163.stdout: ceph x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 6.4 k 2023-10-06T03:30:11.672 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.2 M 2023-10-06T03:30:11.672 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 876 k 2023-10-06T03:30:11.672 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 162 k 2023-10-06T03:30:11.673 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 1.6 M 2023-10-06T03:30:11.673 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 121 k 2023-10-06T03:30:11.673 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 1.7 M 2023-10-06T03:30:11.673 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 7.4 M 2023-10-06T03:30:11.673 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 47 k 2023-10-06T03:30:11.673 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T03:30:11.673 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 52 M 2023-10-06T03:30:11.673 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 267 k 2023-10-06T03:30:11.674 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 80 k 2023-10-06T03:30:11.674 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 255 k 2023-10-06T03:30:11.674 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 18 k 2023-10-06T03:30:11.674 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T03:30:11.674 INFO:teuthology.orchestra.run.smithi163.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T03:30:11.674 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T03:30:11.674 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 25 k 2023-10-06T03:30:11.674 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 797 k 2023-10-06T03:30:11.675 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 130 k 2023-10-06T03:30:11.675 INFO:teuthology.orchestra.run.smithi163.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T03:30:11.675 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T03:30:11.675 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T03:30:11.675 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 190 k 2023-10-06T03:30:11.675 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 371 k 2023-10-06T03:30:11.675 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 358 k 2023-10-06T03:30:11.675 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 107 k 2023-10-06T03:30:11.676 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 90 k 2023-10-06T03:30:11.676 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T03:30:11.676 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 173 k 2023-10-06T03:30:11.676 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T03:30:11.676 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrading: 2023-10-06T03:30:11.677 INFO:teuthology.orchestra.run.smithi163.stdout: librados2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T03:30:11.677 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.6 M 2023-10-06T03:30:11.677 INFO:teuthology.orchestra.run.smithi163.stdout:Installing dependencies: 2023-10-06T03:30:11.677 INFO:teuthology.orchestra.run.smithi163.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T03:30:11.677 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 23 M 2023-10-06T03:30:11.677 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 23 k 2023-10-06T03:30:11.677 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 2.3 M 2023-10-06T03:30:11.678 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 238 k 2023-10-06T03:30:11.678 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.9 M 2023-10-06T03:30:11.678 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T03:30:11.678 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 14 k 2023-10-06T03:30:11.678 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 24 k 2023-10-06T03:30:11.678 INFO:teuthology.orchestra.run.smithi163.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T03:30:11.679 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T03:30:11.679 INFO:teuthology.orchestra.run.smithi163.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T03:30:11.679 INFO:teuthology.orchestra.run.smithi163.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T03:30:11.679 INFO:teuthology.orchestra.run.smithi163.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T03:30:11.679 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 174 k 2023-10-06T03:30:11.679 INFO:teuthology.orchestra.run.smithi163.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T03:30:11.679 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T03:30:11.679 INFO:teuthology.orchestra.run.smithi163.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T03:30:11.680 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T03:30:11.680 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T03:30:11.680 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 507 k 2023-10-06T03:30:11.680 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T03:30:11.680 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.6 M 2023-10-06T03:30:11.680 INFO:teuthology.orchestra.run.smithi163.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T03:30:11.680 INFO:teuthology.orchestra.run.smithi163.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T03:30:11.680 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T03:30:11.680 INFO:teuthology.orchestra.run.smithi163.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T03:30:11.681 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T03:30:11.681 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T03:30:11.681 INFO:teuthology.orchestra.run.smithi163.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T03:30:11.681 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T03:30:11.681 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T03:30:11.681 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T03:30:11.681 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T03:30:11.681 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T03:30:11.681 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T03:30:11.681 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T03:30:11.682 INFO:teuthology.orchestra.run.smithi163.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T03:30:11.682 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T03:30:11.682 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 45 k 2023-10-06T03:30:11.682 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 115 k 2023-10-06T03:30:11.682 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T03:30:11.682 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T03:30:11.682 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T03:30:11.682 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T03:30:11.683 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T03:30:11.683 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T03:30:11.683 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T03:30:11.683 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T03:30:11.683 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T03:30:11.683 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T03:30:11.683 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T03:30:11.683 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T03:30:11.684 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T03:30:11.684 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T03:30:11.684 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T03:30:11.684 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T03:30:11.684 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T03:30:11.684 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T03:30:11.684 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T03:30:11.684 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T03:30:11.684 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T03:30:11.685 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T03:30:11.685 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T03:30:11.685 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T03:30:11.685 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T03:30:11.685 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T03:30:11.685 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T03:30:11.685 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T03:30:11.686 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T03:30:11.686 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T03:30:11.686 INFO:teuthology.orchestra.run.smithi163.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T03:30:11.686 INFO:teuthology.orchestra.run.smithi163.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T03:30:11.686 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T03:30:11.686 INFO:teuthology.orchestra.run.smithi163.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T03:30:11.686 INFO:teuthology.orchestra.run.smithi163.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T03:30:11.687 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T03:30:11.687 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T03:30:11.687 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T03:30:11.687 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T03:30:11.687 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T03:30:11.687 INFO:teuthology.orchestra.run.smithi163.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T03:30:11.687 INFO:teuthology.orchestra.run.smithi163.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T03:30:11.687 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T03:30:11.688 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T03:30:11.688 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:30:11.688 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction Summary 2023-10-06T03:30:11.688 INFO:teuthology.orchestra.run.smithi163.stdout:====================================================================================================================== 2023-10-06T03:30:11.688 INFO:teuthology.orchestra.run.smithi163.stdout:Install 118 Packages 2023-10-06T03:30:11.688 INFO:teuthology.orchestra.run.smithi163.stdout:Upgrade 2 Packages 2023-10-06T03:30:11.688 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:30:11.689 INFO:teuthology.orchestra.run.smithi163.stdout:Total download size: 201 M 2023-10-06T03:30:11.689 INFO:teuthology.orchestra.run.smithi163.stdout:Downloading Packages: 2023-10-06T03:30:12.168 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2023-10-06T03:30:12.181 INFO:teuthology.orchestra.run.smithi055.stdout:====================================================================================================================== 2023-10-06T03:30:12.181 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repository Size 2023-10-06T03:30:12.181 INFO:teuthology.orchestra.run.smithi055.stdout:====================================================================================================================== 2023-10-06T03:30:12.181 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2023-10-06T03:30:12.182 INFO:teuthology.orchestra.run.smithi055.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-06T03:30:12.182 INFO:teuthology.orchestra.run.smithi055.stdout: ceph x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 6.4 k 2023-10-06T03:30:12.182 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.2 M 2023-10-06T03:30:12.182 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 876 k 2023-10-06T03:30:12.182 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 162 k 2023-10-06T03:30:12.183 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 1.6 M 2023-10-06T03:30:12.185 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 121 k 2023-10-06T03:30:12.185 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 1.7 M 2023-10-06T03:30:12.185 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 7.4 M 2023-10-06T03:30:12.185 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-rook noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 47 k 2023-10-06T03:30:12.185 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-radosgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T03:30:12.185 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 52 M 2023-10-06T03:30:12.185 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-volume noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 267 k 2023-10-06T03:30:12.185 INFO:teuthology.orchestra.run.smithi055.stdout: cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 80 k 2023-10-06T03:30:12.186 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 255 k 2023-10-06T03:30:12.186 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-top noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 18 k 2023-10-06T03:30:12.186 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-06T03:30:12.186 INFO:teuthology.orchestra.run.smithi055.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-06T03:30:12.186 INFO:teuthology.orchestra.run.smithi055.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-06T03:30:12.186 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 25 k 2023-10-06T03:30:12.186 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 797 k 2023-10-06T03:30:12.186 INFO:teuthology.orchestra.run.smithi055.stdout: librados-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 130 k 2023-10-06T03:30:12.186 INFO:teuthology.orchestra.run.smithi055.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-06T03:30:12.186 INFO:teuthology.orchestra.run.smithi055.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-06T03:30:12.187 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-06T03:30:12.187 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cephfs x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 190 k 2023-10-06T03:30:12.187 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 371 k 2023-10-06T03:30:12.187 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 358 k 2023-10-06T03:30:12.187 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 107 k 2023-10-06T03:30:12.187 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 90 k 2023-10-06T03:30:12.187 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T03:30:12.187 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-nbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 173 k 2023-10-06T03:30:12.188 INFO:teuthology.orchestra.run.smithi055.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-06T03:30:12.188 INFO:teuthology.orchestra.run.smithi055.stdout:Upgrading: 2023-10-06T03:30:12.188 INFO:teuthology.orchestra.run.smithi055.stdout: librados2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T03:30:12.188 INFO:teuthology.orchestra.run.smithi055.stdout: librbd1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.6 M 2023-10-06T03:30:12.188 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2023-10-06T03:30:12.188 INFO:teuthology.orchestra.run.smithi055.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-06T03:30:12.188 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 23 M 2023-10-06T03:30:12.188 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 23 k 2023-10-06T03:30:12.188 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mds x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 2.3 M 2023-10-06T03:30:12.188 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 238 k 2023-10-06T03:30:12.189 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.9 M 2023-10-06T03:30:12.189 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-osd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T03:30:12.189 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 14 k 2023-10-06T03:30:12.189 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-selinux x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 24 k 2023-10-06T03:30:12.189 INFO:teuthology.orchestra.run.smithi055.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-06T03:30:12.191 INFO:teuthology.orchestra.run.smithi055.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T03:30:12.191 INFO:teuthology.orchestra.run.smithi055.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-06T03:30:12.191 INFO:teuthology.orchestra.run.smithi055.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-06T03:30:12.191 INFO:teuthology.orchestra.run.smithi055.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-06T03:30:12.191 INFO:teuthology.orchestra.run.smithi055.stdout: libcephsqlite x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 174 k 2023-10-06T03:30:12.192 INFO:teuthology.orchestra.run.smithi055.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-06T03:30:12.192 INFO:teuthology.orchestra.run.smithi055.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-06T03:30:12.192 INFO:teuthology.orchestra.run.smithi055.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T03:30:12.192 INFO:teuthology.orchestra.run.smithi055.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-06T03:30:12.192 INFO:teuthology.orchestra.run.smithi055.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-06T03:30:12.192 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 507 k 2023-10-06T03:30:12.192 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-06T03:30:12.192 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.6 M 2023-10-06T03:30:12.192 INFO:teuthology.orchestra.run.smithi055.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-06T03:30:12.192 INFO:teuthology.orchestra.run.smithi055.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-06T03:30:12.193 INFO:teuthology.orchestra.run.smithi055.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T03:30:12.193 INFO:teuthology.orchestra.run.smithi055.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-06T03:30:12.193 INFO:teuthology.orchestra.run.smithi055.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-06T03:30:12.193 INFO:teuthology.orchestra.run.smithi055.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-06T03:30:12.193 INFO:teuthology.orchestra.run.smithi055.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-06T03:30:12.193 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-06T03:30:12.193 INFO:teuthology.orchestra.run.smithi055.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-06T03:30:12.193 INFO:teuthology.orchestra.run.smithi055.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-06T03:30:12.194 INFO:teuthology.orchestra.run.smithi055.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-06T03:30:12.194 INFO:teuthology.orchestra.run.smithi055.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T03:30:12.194 INFO:teuthology.orchestra.run.smithi055.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-06T03:30:12.194 INFO:teuthology.orchestra.run.smithi055.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T03:30:12.194 INFO:teuthology.orchestra.run.smithi055.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T03:30:12.194 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T03:30:12.195 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 45 k 2023-10-06T03:30:12.195 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 115 k 2023-10-06T03:30:12.195 INFO:teuthology.orchestra.run.smithi055.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T03:30:12.195 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T03:30:12.195 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T03:30:12.195 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T03:30:12.195 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T03:30:12.195 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T03:30:12.195 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-06T03:30:12.195 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-06T03:30:12.196 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-06T03:30:12.196 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T03:30:12.196 INFO:teuthology.orchestra.run.smithi055.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T03:30:12.196 INFO:teuthology.orchestra.run.smithi055.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T03:30:12.196 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-06T03:30:12.196 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-06T03:30:12.196 INFO:teuthology.orchestra.run.smithi055.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T03:30:12.196 INFO:teuthology.orchestra.run.smithi055.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T03:30:12.196 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-06T03:30:12.196 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-06T03:30:12.197 INFO:teuthology.orchestra.run.smithi055.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-06T03:30:12.197 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T03:30:12.197 INFO:teuthology.orchestra.run.smithi055.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T03:30:12.197 INFO:teuthology.orchestra.run.smithi055.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-06T03:30:12.197 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-06T03:30:12.197 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-06T03:30:12.197 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-06T03:30:12.197 INFO:teuthology.orchestra.run.smithi055.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T03:30:12.197 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-06T03:30:12.197 INFO:teuthology.orchestra.run.smithi055.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T03:30:12.198 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T03:30:12.198 INFO:teuthology.orchestra.run.smithi055.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-06T03:30:12.198 INFO:teuthology.orchestra.run.smithi055.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T03:30:12.198 INFO:teuthology.orchestra.run.smithi055.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T03:30:12.198 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T03:30:12.198 INFO:teuthology.orchestra.run.smithi055.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T03:30:12.198 INFO:teuthology.orchestra.run.smithi055.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T03:30:12.198 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T03:30:12.198 INFO:teuthology.orchestra.run.smithi055.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T03:30:12.199 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T03:30:12.199 INFO:teuthology.orchestra.run.smithi055.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-06T03:30:12.199 INFO:teuthology.orchestra.run.smithi055.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T03:30:12.199 INFO:teuthology.orchestra.run.smithi055.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-06T03:30:12.199 INFO:teuthology.orchestra.run.smithi055.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T03:30:12.199 INFO:teuthology.orchestra.run.smithi055.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-06T03:30:12.199 INFO:teuthology.orchestra.run.smithi055.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-06T03:30:12.200 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:30:12.200 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2023-10-06T03:30:12.200 INFO:teuthology.orchestra.run.smithi055.stdout:====================================================================================================================== 2023-10-06T03:30:12.200 INFO:teuthology.orchestra.run.smithi055.stdout:Install 118 Packages 2023-10-06T03:30:12.200 INFO:teuthology.orchestra.run.smithi055.stdout:Upgrade 2 Packages 2023-10-06T03:30:12.200 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:30:12.201 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 201 M 2023-10-06T03:30:12.201 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2023-10-06T03:30:12.326 INFO:teuthology.orchestra.run.smithi163.stdout:(1/120): ceph-17.2.6-1296.g736ac5ba.el8.x86_64. 22 kB/s | 6.4 kB 00:00 2023-10-06T03:30:12.515 INFO:teuthology.orchestra.run.smithi055.stdout:(1/120): ceph-17.2.6-1296.g736ac5ba.el8.x86_64. 26 kB/s | 6.4 kB 00:00 2023-10-06T03:30:13.148 INFO:teuthology.orchestra.run.smithi163.stdout:(2/120): ceph-fuse-17.2.6-1296.g736ac5ba.el8.x8 1.0 MB/s | 876 kB 00:00 2023-10-06T03:30:13.322 INFO:teuthology.orchestra.run.smithi163.stdout:(3/120): ceph-immutable-object-cache-17.2.6-129 929 kB/s | 162 kB 00:00 2023-10-06T03:30:13.397 INFO:teuthology.orchestra.run.smithi163.stdout:(4/120): ceph-base-17.2.6-1296.g736ac5ba.el8.x8 3.8 MB/s | 5.2 MB 00:01 2023-10-06T03:30:13.463 INFO:teuthology.orchestra.run.smithi055.stdout:(2/120): ceph-fuse-17.2.6-1296.g736ac5ba.el8.x8 925 kB/s | 876 kB 00:00 2023-10-06T03:30:13.667 INFO:teuthology.orchestra.run.smithi163.stdout:(5/120): ceph-mgr-17.2.6-1296.g736ac5ba.el8.x86 5.8 MB/s | 1.6 MB 00:00 2023-10-06T03:30:13.675 INFO:teuthology.orchestra.run.smithi055.stdout:(3/120): ceph-immutable-object-cache-17.2.6-129 765 kB/s | 162 kB 00:00 2023-10-06T03:30:14.249 INFO:teuthology.orchestra.run.smithi163.stdout:(6/120): ceph-mon-17.2.6-1296.g736ac5ba.el8.x86 6.8 MB/s | 3.9 MB 00:00 2023-10-06T03:30:15.031 INFO:teuthology.orchestra.run.smithi055.stdout:(4/120): ceph-base-17.2.6-1296.g736ac5ba.el8.x8 1.9 MB/s | 5.2 MB 00:02 2023-10-06T03:30:15.047 INFO:teuthology.orchestra.run.smithi055.stdout:(5/120): ceph-mds-17.2.6-1296.g736ac5ba.el8.x86 1.7 MB/s | 2.3 MB 00:01 2023-10-06T03:30:15.058 INFO:teuthology.orchestra.run.smithi163.stdout:(7/120): ceph-mds-17.2.6-1296.g736ac5ba.el8.x86 1.3 MB/s | 2.3 MB 00:01 2023-10-06T03:30:15.599 INFO:teuthology.orchestra.run.smithi055.stdout:(6/120): ceph-mgr-17.2.6-1296.g736ac5ba.el8.x86 2.8 MB/s | 1.6 MB 00:00 2023-10-06T03:30:16.642 INFO:teuthology.orchestra.run.smithi163.stdout:(8/120): ceph-common-17.2.6-1296.g736ac5ba.el8. 5.1 MB/s | 23 MB 00:04 2023-10-06T03:30:16.694 INFO:teuthology.orchestra.run.smithi163.stdout:(9/120): ceph-selinux-17.2.6-1296.g736ac5ba.el8 475 kB/s | 24 kB 00:00 2023-10-06T03:30:16.793 INFO:teuthology.orchestra.run.smithi055.stdout:(7/120): ceph-mon-17.2.6-1296.g736ac5ba.el8.x86 2.3 MB/s | 3.9 MB 00:01 2023-10-06T03:30:16.817 INFO:teuthology.orchestra.run.smithi163.stdout:(10/120): ceph-osd-17.2.6-1296.g736ac5ba.el8.x8 6.5 MB/s | 17 MB 00:02 2023-10-06T03:30:16.881 INFO:teuthology.orchestra.run.smithi163.stdout:(11/120): cephfs-mirror-17.2.6-1296.g736ac5ba.e 4.0 MB/s | 255 kB 00:00 2023-10-06T03:30:16.936 INFO:teuthology.orchestra.run.smithi163.stdout:(12/120): libcephfs-devel-17.2.6-1296.g736ac5ba 456 kB/s | 25 kB 00:00 2023-10-06T03:30:17.061 INFO:teuthology.orchestra.run.smithi163.stdout:(13/120): libcephfs2-17.2.6-1296.g736ac5ba.el8. 6.2 MB/s | 797 kB 00:00 2023-10-06T03:30:17.119 INFO:teuthology.orchestra.run.smithi163.stdout:(14/120): libcephsqlite-17.2.6-1296.g736ac5ba.e 2.9 MB/s | 174 kB 00:00 2023-10-06T03:30:17.178 INFO:teuthology.orchestra.run.smithi163.stdout:(15/120): librados-devel-17.2.6-1296.g736ac5ba. 2.3 MB/s | 130 kB 00:00 2023-10-06T03:30:17.286 INFO:teuthology.orchestra.run.smithi163.stdout:(16/120): libradosstriper1-17.2.6-1296.g736ac5b 4.5 MB/s | 507 kB 00:00 2023-10-06T03:30:18.174 INFO:teuthology.orchestra.run.smithi163.stdout:(17/120): librgw2-17.2.6-1296.g736ac5ba.el8.x86 6.3 MB/s | 5.6 MB 00:00 2023-10-06T03:30:18.233 INFO:teuthology.orchestra.run.smithi163.stdout:(18/120): python3-ceph-argparse-17.2.6-1296.g73 762 kB/s | 45 kB 00:00 2023-10-06T03:30:18.298 INFO:teuthology.orchestra.run.smithi163.stdout:(19/120): python3-ceph-common-17.2.6-1296.g736a 1.8 MB/s | 115 kB 00:00 2023-10-06T03:30:18.358 INFO:teuthology.orchestra.run.smithi163.stdout:(20/120): python3-cephfs-17.2.6-1296.g736ac5ba. 3.1 MB/s | 190 kB 00:00 2023-10-06T03:30:18.426 INFO:teuthology.orchestra.run.smithi163.stdout:(21/120): python3-rados-17.2.6-1296.g736ac5ba.e 5.4 MB/s | 371 kB 00:00 2023-10-06T03:30:18.497 INFO:teuthology.orchestra.run.smithi163.stdout:(22/120): python3-rbd-17.2.6-1296.g736ac5ba.el8 4.9 MB/s | 358 kB 00:00 2023-10-06T03:30:18.566 INFO:teuthology.orchestra.run.smithi163.stdout:(23/120): python3-rgw-17.2.6-1296.g736ac5ba.el8 1.5 MB/s | 107 kB 00:00 2023-10-06T03:30:18.630 INFO:teuthology.orchestra.run.smithi163.stdout:(24/120): rbd-fuse-17.2.6-1296.g736ac5ba.el8.x8 1.4 MB/s | 90 kB 00:00 2023-10-06T03:30:19.304 INFO:teuthology.orchestra.run.smithi163.stdout:(25/120): rbd-mirror-17.2.6-1296.g736ac5ba.el8. 5.1 MB/s | 3.5 MB 00:00 2023-10-06T03:30:19.361 INFO:teuthology.orchestra.run.smithi163.stdout:(26/120): rbd-nbd-17.2.6-1296.g736ac5ba.el8.x86 3.1 MB/s | 173 kB 00:00 2023-10-06T03:30:19.410 INFO:teuthology.orchestra.run.smithi163.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1296.g 461 kB/s | 23 kB 00:00 2023-10-06T03:30:19.457 INFO:teuthology.orchestra.run.smithi163.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1296.g736ac5b 2.5 MB/s | 121 kB 00:00 2023-10-06T03:30:19.819 INFO:teuthology.orchestra.run.smithi163.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1296.g736ac 4.7 MB/s | 1.7 MB 00:00 2023-10-06T03:30:21.074 INFO:teuthology.orchestra.run.smithi163.stdout:(30/120): ceph-mgr-diskprediction-local-17.2.6- 5.9 MB/s | 7.4 MB 00:01 2023-10-06T03:30:21.079 INFO:teuthology.orchestra.run.smithi055.stdout:(8/120): ceph-osd-17.2.6-1296.g736ac5ba.el8.x86 3.1 MB/s | 17 MB 00:05 2023-10-06T03:30:21.112 INFO:teuthology.orchestra.run.smithi163.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1296.g73 6.1 MB/s | 238 kB 00:00 2023-10-06T03:30:21.114 INFO:teuthology.orchestra.run.smithi055.stdout:(9/120): ceph-selinux-17.2.6-1296.g736ac5ba.el8 717 kB/s | 24 kB 00:00 2023-10-06T03:30:21.141 INFO:teuthology.orchestra.run.smithi163.stdout:(32/120): ceph-mgr-rook-17.2.6-1296.g736ac5ba.e 1.6 MB/s | 47 kB 00:00 2023-10-06T03:30:21.179 INFO:teuthology.orchestra.run.smithi163.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1296.g7 383 kB/s | 14 kB 00:00 2023-10-06T03:30:21.228 INFO:teuthology.orchestra.run.smithi163.stdout:(34/120): ceph-volume-17.2.6-1296.g736ac5ba.el8 5.5 MB/s | 267 kB 00:00 2023-10-06T03:30:21.279 INFO:teuthology.orchestra.run.smithi163.stdout:(35/120): cephadm-17.2.6-1296.g736ac5ba.el8.noa 1.5 MB/s | 80 kB 00:00 2023-10-06T03:30:21.322 INFO:teuthology.orchestra.run.smithi163.stdout:(36/120): cephfs-top-17.2.6-1296.g736ac5ba.el8. 414 kB/s | 18 kB 00:00 2023-10-06T03:30:21.535 INFO:teuthology.orchestra.run.smithi163.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2023-10-06T03:30:21.570 INFO:teuthology.orchestra.run.smithi163.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 1.6 MB/s | 59 kB 00:00 2023-10-06T03:30:21.606 INFO:teuthology.orchestra.run.smithi163.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 2.0 MB/s | 75 kB 00:00 2023-10-06T03:30:21.679 INFO:teuthology.orchestra.run.smithi163.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 6.4 MB/s | 476 kB 00:00 2023-10-06T03:30:21.710 INFO:teuthology.orchestra.run.smithi163.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.3 MB/s | 44 kB 00:00 2023-10-06T03:30:21.747 INFO:teuthology.orchestra.run.smithi163.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 5.0 MB/s | 185 kB 00:00 2023-10-06T03:30:21.779 INFO:teuthology.orchestra.run.smithi163.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 1.0 MB/s | 33 kB 00:00 2023-10-06T03:30:21.810 INFO:teuthology.orchestra.run.smithi163.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 527 kB/s | 16 kB 00:00 2023-10-06T03:30:21.846 INFO:teuthology.orchestra.run.smithi163.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 4.7 MB/s | 173 kB 00:00 2023-10-06T03:30:21.890 INFO:teuthology.orchestra.run.smithi163.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 8.7 MB/s | 384 kB 00:00 2023-10-06T03:30:21.922 INFO:teuthology.orchestra.run.smithi163.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 2.4 MB/s | 81 kB 00:00 2023-10-06T03:30:21.954 INFO:teuthology.orchestra.run.smithi163.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 360 kB/s | 11 kB 00:00 2023-10-06T03:30:21.985 INFO:teuthology.orchestra.run.smithi163.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 566 kB/s | 18 kB 00:00 2023-10-06T03:30:22.058 INFO:teuthology.orchestra.run.smithi163.stdout:(50/120): python3-kubernetes-11.0.0-6.el8.noarc 16 MB/s | 1.2 MB 00:00 2023-10-06T03:30:22.089 INFO:teuthology.orchestra.run.smithi163.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 1.5 MB/s | 49 kB 00:00 2023-10-06T03:30:22.121 INFO:teuthology.orchestra.run.smithi163.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 1.8 MB/s | 59 kB 00:00 2023-10-06T03:30:22.153 INFO:teuthology.orchestra.run.smithi163.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.9 MB/s | 60 kB 00:00 2023-10-06T03:30:22.188 INFO:teuthology.orchestra.run.smithi163.stdout:(54/120): python3-pecan-1.3.2-9.el8.noarch.rpm 8.1 MB/s | 283 kB 00:00 2023-10-06T03:30:22.219 INFO:teuthology.orchestra.run.smithi163.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 505 kB/s | 16 kB 00:00 2023-10-06T03:30:22.251 INFO:teuthology.orchestra.run.smithi163.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2023-10-06T03:30:22.284 INFO:teuthology.orchestra.run.smithi163.stdout:(57/120): python3-routes-2.4.1-12.el8.noarch.rp 5.8 MB/s | 196 kB 00:00 2023-10-06T03:30:22.316 INFO:teuthology.orchestra.run.smithi163.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 2.0 MB/s | 64 kB 00:00 2023-10-06T03:30:22.347 INFO:teuthology.orchestra.run.smithi163.stdout:(59/120): python3-simplegeneric-0.8.1-17.el8.no 603 kB/s | 19 kB 00:00 2023-10-06T03:30:22.379 INFO:teuthology.orchestra.run.smithi163.stdout:(60/120): python3-singledispatch-3.4.0.3-18.el8 769 kB/s | 24 kB 00:00 2023-10-06T03:30:22.410 INFO:teuthology.orchestra.run.smithi163.stdout:(61/120): python3-tempora-1.14.1-5.el8.noarch.r 941 kB/s | 29 kB 00:00 2023-10-06T03:30:22.441 INFO:teuthology.orchestra.run.smithi163.stdout:(62/120): python3-trustme-0.6.0-4.el8.noarch.rp 885 kB/s | 27 kB 00:00 2023-10-06T03:30:22.476 INFO:teuthology.orchestra.run.smithi163.stdout:(63/120): python3-waitress-1.4.3-1.el8.noarch.r 7.4 MB/s | 255 kB 00:00 2023-10-06T03:30:22.510 INFO:teuthology.orchestra.run.smithi163.stdout:(64/120): python3-webob-1.8.5-1.el8.1.noarch.rp 7.2 MB/s | 251 kB 00:00 2023-10-06T03:30:22.543 INFO:teuthology.orchestra.run.smithi163.stdout:(65/120): python3-websocket-client-0.56.0-5.el8 1.8 MB/s | 61 kB 00:00 2023-10-06T03:30:22.575 INFO:teuthology.orchestra.run.smithi163.stdout:(66/120): python3-webtest-2.0.33-1.el8.noarch.r 2.6 MB/s | 85 kB 00:00 2023-10-06T03:30:22.607 INFO:teuthology.orchestra.run.smithi163.stdout:(67/120): python3-zc-lockfile-2.0-2.el8.noarch. 741 kB/s | 23 kB 00:00 2023-10-06T03:30:22.680 INFO:teuthology.orchestra.run.smithi163.stdout:(68/120): thrift-0.13.0-2.el8.x86_64.rpm 24 MB/s | 1.7 MB 00:00 2023-10-06T03:30:22.684 INFO:teuthology.orchestra.run.smithi163.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 19 MB/s | 70 kB 00:00 2023-10-06T03:30:22.716 INFO:teuthology.orchestra.run.smithi163.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 604 kB/s | 19 kB 00:00 2023-10-06T03:30:22.731 INFO:teuthology.orchestra.run.smithi163.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.8 MB/s | 43 kB 00:00 2023-10-06T03:30:22.748 INFO:teuthology.orchestra.run.smithi163.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 9.2 MB/s | 155 kB 00:00 2023-10-06T03:30:22.765 INFO:teuthology.orchestra.run.smithi163.stdout:(73/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 13 MB/s | 220 kB 00:00 2023-10-06T03:30:22.779 INFO:teuthology.orchestra.run.smithi163.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.3 MB/s | 48 kB 00:00 2023-10-06T03:30:22.796 INFO:teuthology.orchestra.run.smithi163.stdout:(75/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 14 MB/s | 229 kB 00:00 2023-10-06T03:30:22.809 INFO:teuthology.orchestra.run.smithi163.stdout:(76/120): python3-requests-oauthlib-1.0.0-1.el8 3.3 MB/s | 43 kB 00:00 2023-10-06T03:30:22.858 INFO:teuthology.orchestra.run.smithi163.stdout:(77/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 48 MB/s | 2.3 MB 00:00 2023-10-06T03:30:22.871 INFO:teuthology.orchestra.run.smithi163.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.3 MB/s | 18 kB 00:00 2023-10-06T03:30:22.893 INFO:teuthology.orchestra.run.smithi163.stdout:(79/120): pcre2-devel-10.32-2.el8.x86_64.rpm 27 MB/s | 605 kB 00:00 2023-10-06T03:30:22.907 INFO:teuthology.orchestra.run.smithi163.stdout:(80/120): fuse-2.9.7-12.el8.x86_64.rpm 5.9 MB/s | 83 kB 00:00 2023-10-06T03:30:22.920 INFO:teuthology.orchestra.run.smithi163.stdout:(81/120): elfutils-libelf-devel-0.182-3.el8.x86 4.6 MB/s | 59 kB 00:00 2023-10-06T03:30:22.936 INFO:teuthology.orchestra.run.smithi163.stdout:(82/120): libquadmath-8.4.1-1.el8.x86_64.rpm 11 MB/s | 169 kB 00:00 2023-10-06T03:30:22.949 INFO:teuthology.orchestra.run.smithi163.stdout:(83/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.9 MB/s | 38 kB 00:00 2023-10-06T03:30:22.961 INFO:teuthology.orchestra.run.smithi163.stdout:(84/120): libsepol-devel-2.9-2.el8.x86_64.rpm 6.9 MB/s | 86 kB 00:00 2023-10-06T03:30:22.973 INFO:teuthology.orchestra.run.smithi163.stdout:(85/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.7 MB/s | 58 kB 00:00 2023-10-06T03:30:22.985 INFO:teuthology.orchestra.run.smithi163.stdout:(86/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 4.0 MB/s | 47 kB 00:00 2023-10-06T03:30:23.005 INFO:teuthology.orchestra.run.smithi163.stdout:(87/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 28 MB/s | 559 kB 00:00 2023-10-06T03:30:23.027 INFO:teuthology.orchestra.run.smithi163.stdout:(88/120): libgfortran-8.4.1-1.el8.x86_64.rpm 30 MB/s | 639 kB 00:00 2023-10-06T03:30:23.041 INFO:teuthology.orchestra.run.smithi163.stdout:(89/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 5.1 MB/s | 70 kB 00:00 2023-10-06T03:30:23.056 INFO:teuthology.orchestra.run.smithi163.stdout:(90/120): libselinux-devel-2.9-5.el8.x86_64.rpm 13 MB/s | 200 kB 00:00 2023-10-06T03:30:23.069 INFO:teuthology.orchestra.run.smithi163.stdout:(91/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 7.3 MB/s | 97 kB 00:00 2023-10-06T03:30:23.134 INFO:teuthology.orchestra.run.smithi055.stdout:(10/120): ceph-radosgw-17.2.6-1296.g736ac5ba.el 2.6 MB/s | 17 MB 00:06 2023-10-06T03:30:23.226 INFO:teuthology.orchestra.run.smithi055.stdout:(11/120): cephfs-mirror-17.2.6-1296.g736ac5ba.e 2.7 MB/s | 255 kB 00:00 2023-10-06T03:30:23.264 INFO:teuthology.orchestra.run.smithi055.stdout:(12/120): libcephfs-devel-17.2.6-1296.g736ac5ba 658 kB/s | 25 kB 00:00 2023-10-06T03:30:23.268 INFO:teuthology.orchestra.run.smithi163.stdout:(92/120): ceph-radosgw-17.2.6-1296.g736ac5ba.el 2.0 MB/s | 17 MB 00:08 2023-10-06T03:30:23.296 INFO:teuthology.orchestra.run.smithi163.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 1.6 MB/s | 44 kB 00:00 2023-10-06T03:30:23.308 INFO:teuthology.orchestra.run.smithi163.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 2.1 MB/s | 27 kB 00:00 2023-10-06T03:30:23.323 INFO:teuthology.orchestra.run.smithi163.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 11 MB/s | 161 kB 00:00 2023-10-06T03:30:23.341 INFO:teuthology.orchestra.run.smithi163.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 18 MB/s | 320 kB 00:00 2023-10-06T03:30:23.389 INFO:teuthology.orchestra.run.smithi163.stdout:(97/120): python3-scipy-1.0.0-20.module+el8.1.0 45 MB/s | 15 MB 00:00 2023-10-06T03:30:23.397 INFO:teuthology.orchestra.run.smithi163.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 8.0 MB/s | 457 kB 00:00 2023-10-06T03:30:23.411 INFO:teuthology.orchestra.run.smithi163.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.8 MB/s | 39 kB 00:00 2023-10-06T03:30:23.438 INFO:teuthology.orchestra.run.smithi163.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 25 MB/s | 688 kB 00:00 2023-10-06T03:30:23.445 INFO:teuthology.orchestra.run.smithi055.stdout:(13/120): libcephfs2-17.2.6-1296.g736ac5ba.el8. 4.4 MB/s | 797 kB 00:00 2023-10-06T03:30:23.457 INFO:teuthology.orchestra.run.smithi163.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 15 MB/s | 259 kB 00:00 2023-10-06T03:30:23.472 INFO:teuthology.orchestra.run.smithi163.stdout:(102/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 14 MB/s | 225 kB 00:00 2023-10-06T03:30:23.490 INFO:teuthology.orchestra.run.smithi055.stdout:(14/120): libcephsqlite-17.2.6-1296.g736ac5ba.e 3.6 MB/s | 174 kB 00:00 2023-10-06T03:30:23.494 INFO:teuthology.orchestra.run.smithi163.stdout:(103/120): python3-babel-2.5.1-5.el8.noarch.rpm 46 MB/s | 4.8 MB 00:00 2023-10-06T03:30:23.497 INFO:teuthology.orchestra.run.smithi163.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 5.0 MB/s | 126 kB 00:00 2023-10-06T03:30:23.518 INFO:teuthology.orchestra.run.smithi163.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 8.7 MB/s | 187 kB 00:00 2023-10-06T03:30:23.533 INFO:teuthology.orchestra.run.smithi055.stdout:(15/120): librados-devel-17.2.6-1296.g736ac5ba. 3.0 MB/s | 130 kB 00:00 2023-10-06T03:30:23.542 INFO:teuthology.orchestra.run.smithi163.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 23 MB/s | 538 kB 00:00 2023-10-06T03:30:23.559 INFO:teuthology.orchestra.run.smithi163.stdout:(107/120): socat-1.7.3.3-2.el8.x86_64.rpm 17 MB/s | 302 kB 00:00 2023-10-06T03:30:23.581 INFO:teuthology.orchestra.run.smithi163.stdout:(108/120): python3-numpy-1.14.3-9.el8.x86_64.rp 43 MB/s | 3.7 MB 00:00 2023-10-06T03:30:23.585 INFO:teuthology.orchestra.run.smithi163.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 6.0 MB/s | 157 kB 00:00 2023-10-06T03:30:23.596 INFO:teuthology.orchestra.run.smithi163.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 7.3 MB/s | 110 kB 00:00 2023-10-06T03:30:23.602 INFO:teuthology.orchestra.run.smithi163.stdout:(111/120): librdkafka-0.11.4-1.el8.x86_64.rpm 20 MB/s | 353 kB 00:00 2023-10-06T03:30:23.609 INFO:teuthology.orchestra.run.smithi163.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.5 MB/s | 20 kB 00:00 2023-10-06T03:30:23.623 INFO:teuthology.orchestra.run.smithi163.stdout:(113/120): python3-pytz-2017.2-9.el8.noarch.rpm 4.1 MB/s | 54 kB 00:00 2023-10-06T03:30:23.628 INFO:teuthology.orchestra.run.smithi163.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 27 MB/s | 709 kB 00:00 2023-10-06T03:30:23.656 INFO:teuthology.orchestra.run.smithi055.stdout:(16/120): libradosstriper1-17.2.6-1296.g736ac5b 4.1 MB/s | 507 kB 00:00 2023-10-06T03:30:23.738 INFO:teuthology.orchestra.run.smithi163.stdout:(115/120): openblas-threads-0.3.12-1.el8.x86_64 41 MB/s | 4.7 MB 00:00 2023-10-06T03:30:23.757 INFO:teuthology.orchestra.run.smithi163.stdout:(116/120): openblas-0.3.12-1.el8.x86_64.rpm 36 MB/s | 4.6 MB 00:00 2023-10-06T03:30:23.771 INFO:teuthology.orchestra.run.smithi163.stdout:(117/120): automake-1.16.1-7.el8.noarch.rpm 22 MB/s | 713 kB 00:00 2023-10-06T03:30:24.502 INFO:teuthology.orchestra.run.smithi163.stdout:(118/120): ceph-test-17.2.6-1296.g736ac5ba.el8. 6.6 MB/s | 52 MB 00:07 2023-10-06T03:30:24.515 INFO:teuthology.orchestra.run.smithi163.stdout:(119/120): librbd1-17.2.6-1296.g736ac5ba.el8.x8 4.8 MB/s | 3.6 MB 00:00 2023-10-06T03:30:24.873 INFO:teuthology.orchestra.run.smithi055.stdout:(17/120): librgw2-17.2.6-1296.g736ac5ba.el8.x86 4.6 MB/s | 5.6 MB 00:01 2023-10-06T03:30:24.911 INFO:teuthology.orchestra.run.smithi055.stdout:(18/120): python3-ceph-argparse-17.2.6-1296.g73 1.2 MB/s | 45 kB 00:00 2023-10-06T03:30:24.928 INFO:teuthology.orchestra.run.smithi163.stdout:(120/120): librados2-17.2.6-1296.g736ac5ba.el8. 3.0 MB/s | 3.5 MB 00:01 2023-10-06T03:30:24.932 INFO:teuthology.orchestra.run.smithi163.stdout:-------------------------------------------------------------------------------- 2023-10-06T03:30:24.933 INFO:teuthology.orchestra.run.smithi163.stdout:Total 15 MB/s | 201 MB 00:13 2023-10-06T03:30:24.948 INFO:teuthology.orchestra.run.smithi055.stdout:(19/120): python3-ceph-common-17.2.6-1296.g736a 3.0 MB/s | 115 kB 00:00 2023-10-06T03:30:25.006 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction check 2023-10-06T03:30:25.039 INFO:teuthology.orchestra.run.smithi055.stdout:(20/120): ceph-common-17.2.6-1296.g736ac5ba.el8 1.8 MB/s | 23 MB 00:12 2023-10-06T03:30:25.042 INFO:teuthology.orchestra.run.smithi055.stdout:(21/120): python3-cephfs-17.2.6-1296.g736ac5ba. 2.0 MB/s | 190 kB 00:00 2023-10-06T03:30:25.121 INFO:teuthology.orchestra.run.smithi055.stdout:(22/120): python3-rbd-17.2.6-1296.g736ac5ba.el8 4.4 MB/s | 358 kB 00:00 2023-10-06T03:30:25.131 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction check succeeded. 2023-10-06T03:30:25.132 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction test 2023-10-06T03:30:25.148 INFO:teuthology.orchestra.run.smithi055.stdout:(23/120): python3-rados-17.2.6-1296.g736ac5ba.e 3.3 MB/s | 371 kB 00:00 2023-10-06T03:30:25.160 INFO:teuthology.orchestra.run.smithi055.stdout:(24/120): python3-rgw-17.2.6-1296.g736ac5ba.el8 2.7 MB/s | 107 kB 00:00 2023-10-06T03:30:25.186 INFO:teuthology.orchestra.run.smithi055.stdout:(25/120): rbd-fuse-17.2.6-1296.g736ac5ba.el8.x8 2.4 MB/s | 90 kB 00:00 2023-10-06T03:30:25.258 INFO:teuthology.orchestra.run.smithi055.stdout:(26/120): rbd-nbd-17.2.6-1296.g736ac5ba.el8.x86 2.3 MB/s | 173 kB 00:00 2023-10-06T03:30:25.295 INFO:teuthology.orchestra.run.smithi055.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1296.g 631 kB/s | 23 kB 00:00 2023-10-06T03:30:25.333 INFO:teuthology.orchestra.run.smithi055.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1296.g736ac5b 3.1 MB/s | 121 kB 00:00 2023-10-06T03:30:25.783 INFO:teuthology.orchestra.run.smithi055.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1296.g736ac 3.8 MB/s | 1.7 MB 00:00 2023-10-06T03:30:25.817 INFO:teuthology.orchestra.run.smithi055.stdout:(30/120): rbd-mirror-17.2.6-1296.g736ac5ba.el8. 5.3 MB/s | 3.5 MB 00:00 2023-10-06T03:30:25.904 INFO:teuthology.orchestra.run.smithi055.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1296.g73 2.7 MB/s | 238 kB 00:00 2023-10-06T03:30:25.941 INFO:teuthology.orchestra.run.smithi055.stdout:(32/120): ceph-mgr-rook-17.2.6-1296.g736ac5ba.e 1.3 MB/s | 47 kB 00:00 2023-10-06T03:30:25.978 INFO:teuthology.orchestra.run.smithi055.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1296.g7 399 kB/s | 14 kB 00:00 2023-10-06T03:30:26.051 INFO:teuthology.orchestra.run.smithi055.stdout:(34/120): ceph-volume-17.2.6-1296.g736ac5ba.el8 3.6 MB/s | 267 kB 00:00 2023-10-06T03:30:26.088 INFO:teuthology.orchestra.run.smithi055.stdout:(35/120): cephadm-17.2.6-1296.g736ac5ba.el8.noa 2.1 MB/s | 80 kB 00:00 2023-10-06T03:30:26.124 INFO:teuthology.orchestra.run.smithi055.stdout:(36/120): cephfs-top-17.2.6-1296.g736ac5ba.el8. 498 kB/s | 18 kB 00:00 2023-10-06T03:30:26.359 INFO:teuthology.orchestra.run.smithi055.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2023-10-06T03:30:26.397 INFO:teuthology.orchestra.run.smithi055.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 1.5 MB/s | 59 kB 00:00 2023-10-06T03:30:26.437 INFO:teuthology.orchestra.run.smithi055.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 1.9 MB/s | 75 kB 00:00 2023-10-06T03:30:26.510 INFO:teuthology.orchestra.run.smithi055.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 6.4 MB/s | 476 kB 00:00 2023-10-06T03:30:26.545 INFO:teuthology.orchestra.run.smithi055.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.2 MB/s | 44 kB 00:00 2023-10-06T03:30:26.584 INFO:teuthology.orchestra.run.smithi055.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 4.7 MB/s | 185 kB 00:00 2023-10-06T03:30:26.618 INFO:teuthology.orchestra.run.smithi055.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 956 kB/s | 33 kB 00:00 2023-10-06T03:30:26.653 INFO:teuthology.orchestra.run.smithi055.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 472 kB/s | 16 kB 00:00 2023-10-06T03:30:26.692 INFO:teuthology.orchestra.run.smithi055.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 4.4 MB/s | 173 kB 00:00 2023-10-06T03:30:26.738 INFO:teuthology.orchestra.run.smithi055.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 8.3 MB/s | 384 kB 00:00 2023-10-06T03:30:26.774 INFO:teuthology.orchestra.run.smithi055.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 2.2 MB/s | 81 kB 00:00 2023-10-06T03:30:26.809 INFO:teuthology.orchestra.run.smithi055.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 325 kB/s | 11 kB 00:00 2023-10-06T03:30:26.843 INFO:teuthology.orchestra.run.smithi055.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 511 kB/s | 18 kB 00:00 2023-10-06T03:30:26.888 INFO:teuthology.orchestra.run.smithi163.stdout:Transaction test succeeded. 2023-10-06T03:30:26.893 INFO:teuthology.orchestra.run.smithi163.stdout:Running transaction 2023-10-06T03:30:26.914 INFO:teuthology.orchestra.run.smithi055.stdout:(50/120): python3-kubernetes-11.0.0-6.el8.noarc 17 MB/s | 1.2 MB 00:00 2023-10-06T03:30:26.949 INFO:teuthology.orchestra.run.smithi055.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2023-10-06T03:30:26.984 INFO:teuthology.orchestra.run.smithi055.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 1.6 MB/s | 59 kB 00:00 2023-10-06T03:30:27.019 INFO:teuthology.orchestra.run.smithi055.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 1.7 MB/s | 60 kB 00:00 2023-10-06T03:30:27.059 INFO:teuthology.orchestra.run.smithi055.stdout:(54/120): python3-pecan-1.3.2-9.el8.noarch.rpm 7.0 MB/s | 283 kB 00:00 2023-10-06T03:30:27.094 INFO:teuthology.orchestra.run.smithi055.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 456 kB/s | 16 kB 00:00 2023-10-06T03:30:27.129 INFO:teuthology.orchestra.run.smithi055.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 977 kB/s | 34 kB 00:00 2023-10-06T03:30:27.167 INFO:teuthology.orchestra.run.smithi055.stdout:(57/120): python3-routes-2.4.1-12.el8.noarch.rp 5.1 MB/s | 196 kB 00:00 2023-10-06T03:30:27.202 INFO:teuthology.orchestra.run.smithi055.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 1.8 MB/s | 64 kB 00:00 2023-10-06T03:30:27.237 INFO:teuthology.orchestra.run.smithi055.stdout:(59/120): python3-simplegeneric-0.8.1-17.el8.no 541 kB/s | 19 kB 00:00 2023-10-06T03:30:27.269 INFO:teuthology.orchestra.run.smithi055.stdout:(60/120): ceph-mgr-diskprediction-local-17.2.6- 5.0 MB/s | 7.4 MB 00:01 2023-10-06T03:30:27.444 INFO:teuthology.orchestra.run.smithi055.stdout:(61/120): ceph-test-17.2.6-1296.g736ac5ba.el8.x 8.2 MB/s | 52 MB 00:06 2023-10-06T03:30:27.444 INFO:teuthology.orchestra.run.smithi055.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 115 kB/s | 24 kB 00:00 2023-10-06T03:30:27.484 INFO:teuthology.orchestra.run.smithi055.stdout:(63/120): python3-waitress-1.4.3-1.el8.noarch.r 6.4 MB/s | 255 kB 00:00 2023-10-06T03:30:27.523 INFO:teuthology.orchestra.run.smithi055.stdout:(64/120): python3-webob-1.8.5-1.el8.1.noarch.rp 6.3 MB/s | 251 kB 00:00 2023-10-06T03:30:27.524 INFO:teuthology.orchestra.run.smithi055.stdout:(65/120): python3-tempora-1.14.1-5.el8.noarch.r 115 kB/s | 29 kB 00:00 2023-10-06T03:30:27.558 INFO:teuthology.orchestra.run.smithi055.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 1.7 MB/s | 61 kB 00:00 2023-10-06T03:30:27.559 INFO:teuthology.orchestra.run.smithi055.stdout:(67/120): python3-trustme-0.6.0-4.el8.noarch.rp 240 kB/s | 27 kB 00:00 2023-10-06T03:30:27.593 INFO:teuthology.orchestra.run.smithi055.stdout:(68/120): python3-zc-lockfile-2.0-2.el8.noarch. 669 kB/s | 23 kB 00:00 2023-10-06T03:30:27.597 INFO:teuthology.orchestra.run.smithi055.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 16 MB/s | 70 kB 00:00 2023-10-06T03:30:27.621 INFO:teuthology.orchestra.run.smithi055.stdout:(70/120): python3-webtest-2.0.33-1.el8.noarch.r 876 kB/s | 85 kB 00:00 2023-10-06T03:30:27.654 INFO:teuthology.orchestra.run.smithi055.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-10-06T03:30:27.669 INFO:teuthology.orchestra.run.smithi055.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.8 MB/s | 27 kB 00:00 2023-10-06T03:30:27.687 INFO:teuthology.orchestra.run.smithi055.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 8.8 MB/s | 161 kB 00:00 2023-10-06T03:30:27.707 INFO:teuthology.orchestra.run.smithi055.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 16 MB/s | 320 kB 00:00 2023-10-06T03:30:27.732 INFO:teuthology.orchestra.run.smithi055.stdout:(75/120): python3-werkzeug-0.12.2-4.el8.noarch. 18 MB/s | 457 kB 00:00 2023-10-06T03:30:27.803 INFO:teuthology.orchestra.run.smithi055.stdout:(76/120): thrift-0.13.0-2.el8.x86_64.rpm 7.1 MB/s | 1.7 MB 00:00 2023-10-06T03:30:27.834 INFO:teuthology.orchestra.run.smithi055.stdout:(77/120): python3-markupsafe-0.23-19.el8.x86_64 1.2 MB/s | 39 kB 00:00 2023-10-06T03:30:27.867 INFO:teuthology.orchestra.run.smithi055.stdout:(78/120): bison-3.0.4-10.el8.x86_64.rpm 21 MB/s | 688 kB 00:00 2023-10-06T03:30:27.884 INFO:teuthology.orchestra.run.smithi055.stdout:(79/120): python3-babel-2.5.1-5.el8.noarch.rpm 32 MB/s | 4.8 MB 00:00 2023-10-06T03:30:27.891 INFO:teuthology.orchestra.run.smithi055.stdout:(80/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 10 MB/s | 259 kB 00:00 2023-10-06T03:30:27.903 INFO:teuthology.orchestra.run.smithi055.stdout:(81/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 12 MB/s | 225 kB 00:00 2023-10-06T03:30:27.909 INFO:teuthology.orchestra.run.smithi055.stdout:(82/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 7.0 MB/s | 126 kB 00:00 2023-10-06T03:30:27.929 INFO:teuthology.orchestra.run.smithi055.stdout:(83/120): oniguruma-6.8.2-2.el8.x86_64.rpm 9.4 MB/s | 187 kB 00:00 2023-10-06T03:30:28.001 INFO:teuthology.orchestra.run.smithi055.stdout:(84/120): python3-scipy-1.0.0-20.module+el8.1.0 36 MB/s | 15 MB 00:00 2023-10-06T03:30:28.009 INFO:teuthology.orchestra.run.smithi055.stdout:(85/120): python3-jinja2-2.10.1-2.el8_0.noarch. 6.6 MB/s | 538 kB 00:00 2023-10-06T03:30:28.022 INFO:teuthology.orchestra.run.smithi055.stdout:(86/120): socat-1.7.3.3-2.el8.x86_64.rpm 14 MB/s | 302 kB 00:00 2023-10-06T03:30:28.027 INFO:teuthology.orchestra.run.smithi055.stdout:(87/120): python3-mako-1.0.6-13.el8.noarch.rpm 8.7 MB/s | 157 kB 00:00 2023-10-06T03:30:28.038 INFO:teuthology.orchestra.run.smithi055.stdout:(88/120): python3-pyasn1-modules-0.3.7-6.el8.no 6.9 MB/s | 110 kB 00:00 2023-10-06T03:30:28.055 INFO:teuthology.orchestra.run.smithi055.stdout:(89/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 25 MB/s | 3.7 MB 00:00 2023-10-06T03:30:28.056 INFO:teuthology.orchestra.run.smithi055.stdout:(90/120): python3-jsonpointer-1.10-11.el8.noarc 1.2 MB/s | 20 kB 00:00 2023-10-06T03:30:28.061 INFO:teuthology.orchestra.run.smithi055.stdout:(91/120): librdkafka-0.11.4-1.el8.x86_64.rpm 10 MB/s | 353 kB 00:00 2023-10-06T03:30:28.071 INFO:teuthology.orchestra.run.smithi055.stdout:(92/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.4 MB/s | 54 kB 00:00 2023-10-06T03:30:28.083 INFO:teuthology.orchestra.run.smithi055.stdout:(93/120): libtool-2.4.6-25.el8.x86_64.rpm 25 MB/s | 709 kB 00:00 2023-10-06T03:30:28.118 INFO:teuthology.orchestra.run.smithi055.stdout:(94/120): automake-1.16.1-7.el8.noarch.rpm 20 MB/s | 713 kB 00:00 2023-10-06T03:30:28.133 INFO:teuthology.orchestra.run.smithi055.stdout:(95/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.3 MB/s | 19 kB 00:00 2023-10-06T03:30:28.147 INFO:teuthology.orchestra.run.smithi055.stdout:(96/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.0 MB/s | 43 kB 00:00 2023-10-06T03:30:28.165 INFO:teuthology.orchestra.run.smithi055.stdout:(97/120): python3-oauthlib-2.1.0-1.el8.noarch.r 8.7 MB/s | 155 kB 00:00 2023-10-06T03:30:28.198 INFO:teuthology.orchestra.run.smithi055.stdout:(98/120): openblas-threads-0.3.12-1.el8.x86_64. 34 MB/s | 4.7 MB 00:00 2023-10-06T03:30:28.200 INFO:teuthology.orchestra.run.smithi055.stdout:(99/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 6.2 MB/s | 220 kB 00:00 2023-10-06T03:30:28.219 INFO:teuthology.orchestra.run.smithi055.stdout:(100/120): openblas-0.3.12-1.el8.x86_64.rpm 31 MB/s | 4.6 MB 00:00 2023-10-06T03:30:28.220 INFO:teuthology.orchestra.run.smithi055.stdout:(101/120): keyutils-libs-devel-1.5.10-6.el8.x86 2.2 MB/s | 48 kB 00:00 2023-10-06T03:30:28.223 INFO:teuthology.orchestra.run.smithi055.stdout:(102/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 9.6 MB/s | 229 kB 00:00 2023-10-06T03:30:28.239 INFO:teuthology.orchestra.run.smithi055.stdout:(103/120): python3-requests-oauthlib-1.0.0-1.el 2.0 MB/s | 43 kB 00:00 2023-10-06T03:30:28.245 INFO:teuthology.orchestra.run.smithi055.stdout:(104/120): libverto-devel-0.3.0-5.el8.x86_64.rp 829 kB/s | 18 kB 00:00 2023-10-06T03:30:28.262 INFO:teuthology.orchestra.run.smithi055.stdout:(105/120): fuse-2.9.7-12.el8.x86_64.rpm 5.0 MB/s | 83 kB 00:00 2023-10-06T03:30:28.270 INFO:teuthology.orchestra.run.smithi055.stdout:(106/120): pcre2-devel-10.32-2.el8.x86_64.rpm 19 MB/s | 605 kB 00:00 2023-10-06T03:30:28.283 INFO:teuthology.orchestra.run.smithi055.stdout:(107/120): openssl-devel-1.1.1g-15.el8_3.x86_64 37 MB/s | 2.3 MB 00:00 2023-10-06T03:30:28.285 INFO:teuthology.orchestra.run.smithi055.stdout:(108/120): elfutils-libelf-devel-0.182-3.el8.x8 2.5 MB/s | 59 kB 00:00 2023-10-06T03:30:28.291 INFO:teuthology.orchestra.run.smithi055.stdout:(109/120): libquadmath-8.4.1-1.el8.x86_64.rpm 8.2 MB/s | 169 kB 00:00 2023-10-06T03:30:28.300 INFO:teuthology.orchestra.run.smithi055.stdout:(110/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.3 MB/s | 38 kB 00:00 2023-10-06T03:30:28.302 INFO:teuthology.orchestra.run.smithi055.stdout:(111/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.0 MB/s | 86 kB 00:00 2023-10-06T03:30:28.308 INFO:teuthology.orchestra.run.smithi055.stdout:(112/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.4 MB/s | 58 kB 00:00 2023-10-06T03:30:28.316 INFO:teuthology.orchestra.run.smithi055.stdout:(113/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.8 MB/s | 47 kB 00:00 2023-10-06T03:30:28.328 INFO:teuthology.orchestra.run.smithi055.stdout:(114/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 21 MB/s | 559 kB 00:00 2023-10-06T03:30:28.333 INFO:teuthology.orchestra.run.smithi055.stdout:(115/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 4.1 MB/s | 70 kB 00:00 2023-10-06T03:30:28.340 INFO:teuthology.orchestra.run.smithi055.stdout:(116/120): libgfortran-8.4.1-1.el8.x86_64.rpm 20 MB/s | 639 kB 00:00 2023-10-06T03:30:28.346 INFO:teuthology.orchestra.run.smithi055.stdout:(117/120): libselinux-devel-2.9-5.el8.x86_64.rp 11 MB/s | 200 kB 00:00 2023-10-06T03:30:28.349 INFO:teuthology.orchestra.run.smithi055.stdout:(118/120): libuuid-devel-2.32.1-27.el8.x86_64.r 6.1 MB/s | 97 kB 00:00 2023-10-06T03:30:28.669 INFO:teuthology.orchestra.run.smithi055.stdout:(119/120): librbd1-17.2.6-1296.g736ac5ba.el8.x8 11 MB/s | 3.6 MB 00:00 2023-10-06T03:30:29.024 INFO:teuthology.orchestra.run.smithi055.stdout:(120/120): librados2-17.2.6-1296.g736ac5ba.el8. 5.1 MB/s | 3.5 MB 00:00 2023-10-06T03:30:29.028 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2023-10-06T03:30:29.028 INFO:teuthology.orchestra.run.smithi055.stdout:Total 12 MB/s | 201 MB 00:16 2023-10-06T03:30:29.103 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2023-10-06T03:30:29.238 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2023-10-06T03:30:29.238 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2023-10-06T03:30:31.045 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2023-10-06T03:30:31.050 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2023-10-06T03:30:31.449 INFO:teuthology.orchestra.run.smithi163.stdout: Preparing : 1/1 2023-10-06T03:30:31.654 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T03:30:31.893 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T03:30:32.045 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-06T03:30:32.251 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T03:30:32.685 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-06T03:30:32.901 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T03:30:33.379 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T03:30:33.601 INFO:teuthology.orchestra.run.smithi163.stdout: Upgrading : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T03:30:33.757 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T03:30:33.934 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T03:30:34.058 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T03:30:34.281 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T03:30:34.619 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-06T03:30:35.354 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T03:30:35.526 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T03:30:35.687 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T03:30:36.488 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2023-10-06T03:30:36.586 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T03:30:36.679 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-06T03:30:36.779 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T03:30:36.906 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-06T03:30:36.992 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T03:30:37.085 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T03:30:37.129 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T03:30:37.269 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-06T03:30:37.308 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T03:30:37.416 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T03:30:37.452 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T03:30:37.721 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-06T03:30:37.913 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-06T03:30:38.028 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-06T03:30:38.268 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-06T03:30:38.310 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T03:30:38.436 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-06T03:30:38.617 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T03:30:38.781 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 5/122 2023-10-06T03:30:38.782 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-06T03:30:38.941 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T03:30:39.018 INFO:teuthology.orchestra.run.smithi055.stdout: Upgrading : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T03:30:39.180 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T03:30:39.377 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T03:30:39.504 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T03:30:39.553 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-06T03:30:39.691 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T03:30:39.870 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/122 2023-10-06T03:30:40.302 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-06T03:30:40.483 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T03:30:40.602 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-06T03:30:40.760 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T03:30:40.906 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T03:30:41.184 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-06T03:30:41.360 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-06T03:30:41.388 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-06T03:30:41.593 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T03:30:41.676 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-06T03:30:41.722 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T03:30:41.934 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T03:30:41.949 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-06T03:30:42.028 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 14/122 2023-10-06T03:30:42.211 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-06T03:30:42.240 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T03:30:42.376 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T03:30:42.424 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 26/122 2023-10-06T03:30:42.548 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-06T03:30:42.579 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 27/122 2023-10-06T03:30:42.743 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T03:30:42.885 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 28/122 2023-10-06T03:30:43.138 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-06T03:30:43.397 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-06T03:30:43.477 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-06T03:30:43.584 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T03:30:43.707 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T03:30:44.217 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-06T03:30:44.401 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T03:30:45.317 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-06T03:30:45.587 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-06T03:30:45.788 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-06T03:30:45.934 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-06T03:30:46.174 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-06T03:30:46.223 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-06T03:30:46.555 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-06T03:30:46.564 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-06T03:30:46.785 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-06T03:30:47.000 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 26/122 2023-10-06T03:30:47.140 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 27/122 2023-10-06T03:30:47.453 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 28/122 2023-10-06T03:30:47.719 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-06T03:30:48.123 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-06T03:30:48.369 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-06T03:30:48.607 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-06T03:30:48.648 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T03:30:48.854 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T03:30:48.958 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-06T03:30:49.087 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 38/122 2023-10-06T03:30:49.249 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-06T03:30:49.273 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 39/122 2023-10-06T03:30:49.556 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 40/122 2023-10-06T03:30:49.556 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-06T03:30:49.807 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-06T03:30:49.856 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-06T03:30:50.008 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-06T03:30:50.067 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T03:30:50.235 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-06T03:30:50.446 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-06T03:30:50.606 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T03:30:50.709 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-06T03:30:50.891 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-06T03:30:51.076 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-06T03:30:51.268 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-06T03:30:51.478 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-06T03:30:51.669 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-06T03:30:51.909 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-06T03:30:52.089 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-06T03:30:52.302 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-06T03:30:52.504 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-06T03:30:52.641 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-06T03:30:52.850 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-06T03:30:52.989 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-06T03:30:53.133 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-06T03:30:53.268 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-06T03:30:53.433 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-06T03:30:53.448 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-06T03:30:53.699 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 38/122 2023-10-06T03:30:53.743 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-06T03:30:53.878 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 39/122 2023-10-06T03:30:53.925 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T03:30:54.095 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 40/122 2023-10-06T03:30:54.116 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T03:30:54.267 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-06T03:30:54.445 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T03:30:54.456 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-06T03:30:54.639 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-06T03:30:54.675 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T03:30:54.853 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-06T03:30:54.914 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T03:30:54.996 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-06T03:30:55.070 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T03:30:55.190 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-06T03:30:55.253 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T03:30:55.399 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-06T03:30:55.515 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T03:30:55.735 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-06T03:30:55.748 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T03:30:55.972 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T03:30:56.006 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-06T03:30:56.218 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T03:30:56.289 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-06T03:30:56.405 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T03:30:56.670 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T03:30:56.700 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-06T03:30:56.912 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T03:30:56.925 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-06T03:30:57.126 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-06T03:30:57.179 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T03:30:57.321 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-06T03:30:57.501 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T03:30:57.548 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-06T03:30:57.699 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T03:30:57.797 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-06T03:30:57.823 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T03:30:58.019 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-06T03:30:58.033 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T03:30:58.162 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-06T03:30:58.283 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T03:30:58.370 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-06T03:30:58.598 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-06T03:30:58.893 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-06T03:30:59.144 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-06T03:30:59.351 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-06T03:30:59.556 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-06T03:30:59.713 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T03:30:59.716 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-06T03:30:59.917 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-06T03:31:00.004 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T03:31:00.132 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-06T03:31:00.280 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T03:31:00.373 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-06T03:31:00.546 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-06T03:31:00.626 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T03:31:00.709 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-06T03:31:00.885 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T03:31:00.973 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-06T03:31:01.250 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-06T03:31:01.270 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T03:31:01.450 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-06T03:31:01.512 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T03:31:01.709 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T03:31:01.795 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-06T03:31:01.955 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T03:31:02.085 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-06T03:31:02.290 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-06T03:31:02.569 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-06T03:31:04.038 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-06T03:31:04.368 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-06T03:31:04.634 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-06T03:31:04.961 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-06T03:31:05.198 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-06T03:31:05.580 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-06T03:31:05.827 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-06T03:31:06.014 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-06T03:31:06.249 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-06T03:31:09.843 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T03:31:10.112 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T03:31:10.794 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T03:31:11.077 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T03:31:11.269 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T03:31:11.411 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T03:31:13.918 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T03:31:14.173 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T03:31:15.276 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 87/122 2023-10-06T03:31:15.526 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T03:31:15.694 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 88/122 2023-10-06T03:31:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T03:31:30.965 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T03:31:30.965 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /sys 2023-10-06T03:31:30.965 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /proc 2023-10-06T03:31:30.966 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /dev 2023-10-06T03:31:30.966 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /run 2023-10-06T03:31:30.966 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /mnt 2023-10-06T03:31:30.966 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /var/tmp 2023-10-06T03:31:30.966 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /home 2023-10-06T03:31:30.966 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /tmp 2023-10-06T03:31:30.966 INFO:teuthology.orchestra.run.smithi163.stdout:skipping the directory /dev 2023-10-06T03:31:30.966 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:31:31.408 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T03:31:33.041 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T03:31:33.264 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T03:31:33.537 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T03:31:33.725 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T03:31:34.190 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T03:31:34.391 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T03:31:34.431 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T03:31:36.106 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 89/122 2023-10-06T03:31:36.106 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /sys 2023-10-06T03:31:36.106 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /proc 2023-10-06T03:31:36.106 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /dev 2023-10-06T03:31:36.106 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /run 2023-10-06T03:31:36.106 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /mnt 2023-10-06T03:31:36.107 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /var/tmp 2023-10-06T03:31:36.107 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /home 2023-10-06T03:31:36.107 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /tmp 2023-10-06T03:31:36.107 INFO:teuthology.orchestra.run.smithi055.stdout:skipping the directory /dev 2023-10-06T03:31:36.107 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:31:36.545 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T03:31:38.280 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 90/122 2023-10-06T03:31:38.448 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T03:31:38.736 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 91/122 2023-10-06T03:31:38.886 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T03:31:38.938 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T03:31:39.125 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T03:31:39.315 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 95/122 2023-10-06T03:31:39.384 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 92/122 2023-10-06T03:31:39.543 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 96/122 2023-10-06T03:31:39.570 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T03:31:39.611 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 93/122 2023-10-06T03:31:39.774 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T03:31:40.041 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T03:31:40.255 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T03:31:41.370 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T03:31:41.539 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T03:31:41.680 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T03:31:42.131 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 100/122 2023-10-06T03:31:42.355 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T03:31:42.425 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T03:31:42.589 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T03:31:42.616 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T03:31:44.178 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T03:31:44.359 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 103/122 2023-10-06T03:31:44.367 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 94/122 2023-10-06T03:31:44.569 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 95/122 2023-10-06T03:31:44.638 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T03:31:44.740 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 96/122 2023-10-06T03:31:44.762 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T03:31:44.923 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T03:31:44.966 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T03:31:44.996 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T03:31:45.156 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T03:31:45.245 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 97/122 2023-10-06T03:31:45.463 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T03:31:45.567 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T03:31:45.739 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T03:31:46.655 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 98/122 2023-10-06T03:31:46.801 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T03:31:46.947 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 99/122 2023-10-06T03:31:47.376 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 100/122 2023-10-06T03:31:47.581 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T03:31:47.652 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/122 2023-10-06T03:31:47.784 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T03:31:47.812 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 102/122 2023-10-06T03:31:49.665 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 103/122 2023-10-06T03:31:49.930 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T03:31:50.057 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 104/122 2023-10-06T03:31:50.201 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T03:31:50.277 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1296.g736ac 105/122 2023-10-06T03:31:50.451 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T03:31:50.890 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 106/122 2023-10-06T03:31:51.060 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T03:31:51.112 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T03:31:52.436 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 108/122 2023-10-06T03:31:52.851 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-06T03:31:53.060 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 110/122 2023-10-06T03:31:53.306 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-06T03:31:53.504 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T03:31:53.531 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-06T03:31:53.734 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 113/122 2023-10-06T03:31:53.926 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T03:31:54.096 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 115/122 2023-10-06T03:31:54.280 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 116/122 2023-10-06T03:31:54.584 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 117/122 2023-10-06T03:31:54.762 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T03:31:54.845 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-06T03:31:55.054 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T03:31:55.077 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-06T03:31:55.224 INFO:teuthology.orchestra.run.smithi163.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-06T03:31:55.224 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T03:31:55.468 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T03:31:55.468 INFO:teuthology.orchestra.run.smithi163.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:31:55.884 INFO:teuthology.orchestra.run.smithi163.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:31:55.884 INFO:teuthology.orchestra.run.smithi163.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-06T03:31:55.884 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:31:56.783 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 1/122 2023-10-06T03:31:56.783 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2/122 2023-10-06T03:31:56.783 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 3/122 2023-10-06T03:31:56.783 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T03:31:56.783 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 5/122 2023-10-06T03:31:56.783 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T03:31:56.783 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T03:31:56.783 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/122 2023-10-06T03:31:56.783 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 9/122 2023-10-06T03:31:56.784 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T03:31:56.784 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 11/122 2023-10-06T03:31:56.784 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 12/122 2023-10-06T03:31:56.784 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T03:31:56.784 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 14/122 2023-10-06T03:31:56.784 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 15/122 2023-10-06T03:31:56.784 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 16/122 2023-10-06T03:31:56.784 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 17/122 2023-10-06T03:31:56.784 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 18/122 2023-10-06T03:31:56.784 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 19/122 2023-10-06T03:31:56.784 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 20/122 2023-10-06T03:31:56.785 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 21/122 2023-10-06T03:31:56.785 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 22/122 2023-10-06T03:31:56.785 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 23/122 2023-10-06T03:31:56.785 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 24/122 2023-10-06T03:31:56.785 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 25/122 2023-10-06T03:31:56.785 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 26/122 2023-10-06T03:31:56.785 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 27/122 2023-10-06T03:31:56.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 28/122 2023-10-06T03:31:56.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 29/122 2023-10-06T03:31:56.793 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 30/122 2023-10-06T03:31:56.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 31/122 2023-10-06T03:31:56.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 32/122 2023-10-06T03:31:56.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 33/122 2023-10-06T03:31:56.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 34/122 2023-10-06T03:31:56.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 35/122 2023-10-06T03:31:56.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 36/122 2023-10-06T03:31:56.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 37/122 2023-10-06T03:31:56.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 38/122 2023-10-06T03:31:56.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T03:31:56.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T03:31:56.794 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T03:31:56.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T03:31:56.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T03:31:56.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T03:31:56.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T03:31:56.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T03:31:56.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T03:31:56.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T03:31:56.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T03:31:56.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T03:31:56.795 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T03:31:56.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T03:31:56.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T03:31:56.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T03:31:56.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T03:31:56.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T03:31:56.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T03:31:56.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T03:31:56.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T03:31:56.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T03:31:56.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T03:31:56.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T03:31:56.796 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T03:31:56.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T03:31:56.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T03:31:56.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T03:31:56.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T03:31:56.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T03:31:56.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T03:31:56.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T03:31:56.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T03:31:56.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-06T03:31:56.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-06T03:31:56.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-06T03:31:56.797 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-06T03:31:56.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-06T03:31:56.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-06T03:31:56.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-06T03:31:56.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-06T03:31:56.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-06T03:31:56.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-06T03:31:56.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-06T03:31:56.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-06T03:31:56.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-06T03:31:56.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-06T03:31:56.798 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-06T03:31:56.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-06T03:31:56.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-06T03:31:56.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-06T03:31:56.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-06T03:31:56.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-06T03:31:56.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-06T03:31:56.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-06T03:31:56.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-06T03:31:56.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-06T03:31:56.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-06T03:31:56.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-06T03:31:56.799 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-06T03:31:56.800 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-06T03:31:56.800 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-06T03:31:56.800 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-06T03:31:56.800 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-06T03:31:56.800 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-06T03:31:56.800 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-06T03:31:56.800 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-06T03:31:56.800 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-06T03:31:56.800 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-06T03:31:56.800 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-06T03:31:56.800 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-06T03:31:56.801 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-06T03:31:56.801 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-06T03:31:56.801 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-06T03:31:56.801 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-06T03:31:56.801 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-06T03:31:56.801 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-06T03:31:56.801 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-06T03:31:56.801 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-06T03:31:56.801 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-06T03:31:56.801 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 119/122 2023-10-06T03:31:56.801 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T03:31:56.801 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 121/122 2023-10-06T03:31:56.802 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 107/122 2023-10-06T03:31:57.153 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 108/122 2023-10-06T03:31:57.291 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T03:31:58.442 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-06T03:31:58.838 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-06T03:31:59.024 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 111/122 2023-10-06T03:31:59.139 INFO:teuthology.orchestra.run.smithi163.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:31:59.139 INFO:teuthology.orchestra.run.smithi163.stdout:Last metadata expiration check: 0:01:57 ago on Fri 06 Oct 2023 03:30:02 AM UTC. 2023-10-06T03:31:59.221 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-06T03:31:59.347 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 113/122 2023-10-06T03:31:59.473 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-06T03:31:59.660 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 115/122 2023-10-06T03:31:59.860 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 116/122 2023-10-06T03:32:00.035 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 117/122 2023-10-06T03:32:00.266 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-06T03:32:00.474 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T03:32:00.559 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-06T03:32:00.725 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T03:32:00.750 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-06T03:32:00.750 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T03:32:00.998 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-06T03:32:00.999 INFO:teuthology.orchestra.run.smithi055.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:32:01.354 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:32:01.354 INFO:teuthology.orchestra.run.smithi055.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-06T03:32:01.354 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:02.253 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 1/122 2023-10-06T03:32:02.253 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2/122 2023-10-06T03:32:02.253 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 3/122 2023-10-06T03:32:02.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/122 2023-10-06T03:32:02.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 5/122 2023-10-06T03:32:02.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/122 2023-10-06T03:32:02.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/122 2023-10-06T03:32:02.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/122 2023-10-06T03:32:02.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 9/122 2023-10-06T03:32:02.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/122 2023-10-06T03:32:02.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 11/122 2023-10-06T03:32:02.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 12/122 2023-10-06T03:32:02.254 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/122 2023-10-06T03:32:02.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 14/122 2023-10-06T03:32:02.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 15/122 2023-10-06T03:32:02.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 16/122 2023-10-06T03:32:02.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 17/122 2023-10-06T03:32:02.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 18/122 2023-10-06T03:32:02.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 19/122 2023-10-06T03:32:02.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 20/122 2023-10-06T03:32:02.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 21/122 2023-10-06T03:32:02.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 22/122 2023-10-06T03:32:02.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 23/122 2023-10-06T03:32:02.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 24/122 2023-10-06T03:32:02.255 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 25/122 2023-10-06T03:32:02.256 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 26/122 2023-10-06T03:32:02.256 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 27/122 2023-10-06T03:32:02.256 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 28/122 2023-10-06T03:32:02.256 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 29/122 2023-10-06T03:32:02.256 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 30/122 2023-10-06T03:32:02.264 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 31/122 2023-10-06T03:32:02.264 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 32/122 2023-10-06T03:32:02.265 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 33/122 2023-10-06T03:32:02.265 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 34/122 2023-10-06T03:32:02.265 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 35/122 2023-10-06T03:32:02.265 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 36/122 2023-10-06T03:32:02.265 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 37/122 2023-10-06T03:32:02.265 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 38/122 2023-10-06T03:32:02.265 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-06T03:32:02.265 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-06T03:32:02.265 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-06T03:32:02.265 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-06T03:32:02.266 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-06T03:32:02.266 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-06T03:32:02.266 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-06T03:32:02.266 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-06T03:32:02.266 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-06T03:32:02.266 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-06T03:32:02.266 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-06T03:32:02.266 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-06T03:32:02.266 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-06T03:32:02.266 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-06T03:32:02.266 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-06T03:32:02.267 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-06T03:32:02.267 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-06T03:32:02.267 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-06T03:32:02.267 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-06T03:32:02.267 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-06T03:32:02.267 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-06T03:32:02.267 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-06T03:32:02.267 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-06T03:32:02.267 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-06T03:32:02.267 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-06T03:32:02.267 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-06T03:32:02.267 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-06T03:32:02.268 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-06T03:32:02.268 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-06T03:32:02.268 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-06T03:32:02.268 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-06T03:32:02.268 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-06T03:32:02.268 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-06T03:32:02.268 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-06T03:32:02.268 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-06T03:32:02.268 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-06T03:32:02.268 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-06T03:32:02.268 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-06T03:32:02.269 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-06T03:32:02.269 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-06T03:32:02.269 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-06T03:32:02.269 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-06T03:32:02.269 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-06T03:32:02.269 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-06T03:32:02.269 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-06T03:32:02.269 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-06T03:32:02.269 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-06T03:32:02.269 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-06T03:32:02.269 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-06T03:32:02.270 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-06T03:32:02.270 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-06T03:32:02.270 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-06T03:32:02.270 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-06T03:32:02.270 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-06T03:32:02.270 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-06T03:32:02.270 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-06T03:32:02.270 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-06T03:32:02.270 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-06T03:32:02.270 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-06T03:32:02.270 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-06T03:32:02.271 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-06T03:32:02.271 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-06T03:32:02.271 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-06T03:32:02.271 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-06T03:32:02.271 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-06T03:32:02.271 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-06T03:32:02.271 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-06T03:32:02.271 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-06T03:32:02.271 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-06T03:32:02.271 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-06T03:32:02.271 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-06T03:32:02.271 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-06T03:32:02.272 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-06T03:32:02.272 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-06T03:32:02.272 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-06T03:32:02.272 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-06T03:32:02.272 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-06T03:32:02.272 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-06T03:32:02.272 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-06T03:32:02.272 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-06T03:32:02.272 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 119/122 2023-10-06T03:32:02.272 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-06T03:32:02.272 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 121/122 2023-10-06T03:32:03.561 INFO:teuthology.orchestra.run.smithi163.stdout:Installed products updated. 2023-10-06T03:32:03.733 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:03.733 INFO:teuthology.orchestra.run.smithi163.stdout:Upgraded: 2023-10-06T03:32:03.733 INFO:teuthology.orchestra.run.smithi163.stdout: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.734 INFO:teuthology.orchestra.run.smithi163.stdout: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.734 INFO:teuthology.orchestra.run.smithi163.stdout:Installed: 2023-10-06T03:32:03.734 INFO:teuthology.orchestra.run.smithi163.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T03:32:03.734 INFO:teuthology.orchestra.run.smithi163.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T03:32:03.734 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.734 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.734 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.734 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.734 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:03.734 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-immutable-object-cache-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.735 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.735 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.735 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:03.735 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:03.735 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:03.735 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:03.735 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:03.735 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.735 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.735 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:03.735 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.736 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.736 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.736 INFO:teuthology.orchestra.run.smithi163.stdout: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:03.736 INFO:teuthology.orchestra.run.smithi163.stdout: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:03.736 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.736 INFO:teuthology.orchestra.run.smithi163.stdout: cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:03.736 INFO:teuthology.orchestra.run.smithi163.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T03:32:03.736 INFO:teuthology.orchestra.run.smithi163.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T03:32:03.736 INFO:teuthology.orchestra.run.smithi163.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T03:32:03.736 INFO:teuthology.orchestra.run.smithi163.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T03:32:03.736 INFO:teuthology.orchestra.run.smithi163.stdout: jq-1.5-12.el8.x86_64 2023-10-06T03:32:03.736 INFO:teuthology.orchestra.run.smithi163.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T03:32:03.737 INFO:teuthology.orchestra.run.smithi163.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T03:32:03.737 INFO:teuthology.orchestra.run.smithi163.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T03:32:03.737 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.737 INFO:teuthology.orchestra.run.smithi163.stdout: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.737 INFO:teuthology.orchestra.run.smithi163.stdout: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.737 INFO:teuthology.orchestra.run.smithi163.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T03:32:03.737 INFO:teuthology.orchestra.run.smithi163.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T03:32:03.737 INFO:teuthology.orchestra.run.smithi163.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T03:32:03.737 INFO:teuthology.orchestra.run.smithi163.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T03:32:03.737 INFO:teuthology.orchestra.run.smithi163.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T03:32:03.737 INFO:teuthology.orchestra.run.smithi163.stdout: librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.738 INFO:teuthology.orchestra.run.smithi163.stdout: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.738 INFO:teuthology.orchestra.run.smithi163.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T03:32:03.738 INFO:teuthology.orchestra.run.smithi163.stdout: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.738 INFO:teuthology.orchestra.run.smithi163.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T03:32:03.738 INFO:teuthology.orchestra.run.smithi163.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T03:32:03.738 INFO:teuthology.orchestra.run.smithi163.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T03:32:03.738 INFO:teuthology.orchestra.run.smithi163.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T03:32:03.738 INFO:teuthology.orchestra.run.smithi163.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T03:32:03.738 INFO:teuthology.orchestra.run.smithi163.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T03:32:03.738 INFO:teuthology.orchestra.run.smithi163.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T03:32:03.738 INFO:teuthology.orchestra.run.smithi163.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T03:32:03.739 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T03:32:03.739 INFO:teuthology.orchestra.run.smithi163.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T03:32:03.739 INFO:teuthology.orchestra.run.smithi163.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T03:32:03.739 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T03:32:03.739 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T03:32:03.739 INFO:teuthology.orchestra.run.smithi163.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T03:32:03.739 INFO:teuthology.orchestra.run.smithi163.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T03:32:03.739 INFO:teuthology.orchestra.run.smithi163.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T03:32:03.739 INFO:teuthology.orchestra.run.smithi163.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T03:32:03.739 INFO:teuthology.orchestra.run.smithi163.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T03:32:03.739 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T03:32:03.740 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.740 INFO:teuthology.orchestra.run.smithi163.stdout: python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.740 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.740 INFO:teuthology.orchestra.run.smithi163.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T03:32:03.740 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T03:32:03.740 INFO:teuthology.orchestra.run.smithi163.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T03:32:03.740 INFO:teuthology.orchestra.run.smithi163.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T03:32:03.740 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T03:32:03.740 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T03:32:03.740 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T03:32:03.740 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T03:32:03.740 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T03:32:03.741 INFO:teuthology.orchestra.run.smithi163.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T03:32:03.741 INFO:teuthology.orchestra.run.smithi163.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T03:32:03.741 INFO:teuthology.orchestra.run.smithi163.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T03:32:03.741 INFO:teuthology.orchestra.run.smithi163.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T03:32:03.741 INFO:teuthology.orchestra.run.smithi163.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T03:32:03.741 INFO:teuthology.orchestra.run.smithi163.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T03:32:03.741 INFO:teuthology.orchestra.run.smithi163.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T03:32:03.741 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T03:32:03.741 INFO:teuthology.orchestra.run.smithi163.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T03:32:03.741 INFO:teuthology.orchestra.run.smithi163.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T03:32:03.742 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T03:32:03.742 INFO:teuthology.orchestra.run.smithi163.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T03:32:03.742 INFO:teuthology.orchestra.run.smithi163.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T03:32:03.742 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T03:32:03.742 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T03:32:03.742 INFO:teuthology.orchestra.run.smithi163.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T03:32:03.742 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.742 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.742 INFO:teuthology.orchestra.run.smithi163.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T03:32:03.742 INFO:teuthology.orchestra.run.smithi163.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T03:32:03.742 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.742 INFO:teuthology.orchestra.run.smithi163.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T03:32:03.743 INFO:teuthology.orchestra.run.smithi163.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T03:32:03.743 INFO:teuthology.orchestra.run.smithi163.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T03:32:03.743 INFO:teuthology.orchestra.run.smithi163.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T03:32:03.743 INFO:teuthology.orchestra.run.smithi163.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T03:32:03.743 INFO:teuthology.orchestra.run.smithi163.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T03:32:03.743 INFO:teuthology.orchestra.run.smithi163.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T03:32:03.743 INFO:teuthology.orchestra.run.smithi163.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T03:32:03.743 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T03:32:03.744 INFO:teuthology.orchestra.run.smithi163.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T03:32:03.744 INFO:teuthology.orchestra.run.smithi163.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T03:32:03.744 INFO:teuthology.orchestra.run.smithi163.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T03:32:03.744 INFO:teuthology.orchestra.run.smithi163.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T03:32:03.744 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.744 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.744 INFO:teuthology.orchestra.run.smithi163.stdout: rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:03.744 INFO:teuthology.orchestra.run.smithi163.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T03:32:03.744 INFO:teuthology.orchestra.run.smithi163.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T03:32:03.744 INFO:teuthology.orchestra.run.smithi163.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T03:32:03.744 INFO:teuthology.orchestra.run.smithi163.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T03:32:03.745 INFO:teuthology.orchestra.run.smithi163.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T03:32:03.745 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:03.745 INFO:teuthology.orchestra.run.smithi163.stdout:Complete! 2023-10-06T03:32:04.117 DEBUG:teuthology.parallel:result is None 2023-10-06T03:32:04.760 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-06T03:32:04.761 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:02:02 ago on Fri 06 Oct 2023 03:30:02 AM UTC. 2023-10-06T03:32:08.722 INFO:teuthology.orchestra.run.smithi055.stdout:Installed products updated. 2023-10-06T03:32:08.893 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:08.893 INFO:teuthology.orchestra.run.smithi055.stdout:Upgraded: 2023-10-06T03:32:08.893 INFO:teuthology.orchestra.run.smithi055.stdout: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.893 INFO:teuthology.orchestra.run.smithi055.stdout: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.893 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2023-10-06T03:32:08.893 INFO:teuthology.orchestra.run.smithi055.stdout: automake-1.16.1-7.el8.noarch 2023-10-06T03:32:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T03:32:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-immutable-object-cache-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:08.894 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:08.895 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T03:32:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-06T03:32:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T03:32:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-06T03:32:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T03:32:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: jq-1.5-12.el8.x86_64 2023-10-06T03:32:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-06T03:32:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-06T03:32:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T03:32:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.896 INFO:teuthology.orchestra.run.smithi055.stdout: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-06T03:32:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-06T03:32:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T03:32:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-06T03:32:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-06T03:32:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-06T03:32:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-06T03:32:08.897 INFO:teuthology.orchestra.run.smithi055.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-06T03:32:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T03:32:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T03:32:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-06T03:32:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-06T03:32:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T03:32:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T03:32:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-06T03:32:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-06T03:32:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-06T03:32:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-06T03:32:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-06T03:32:08.898 INFO:teuthology.orchestra.run.smithi055.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-06T03:32:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T03:32:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-06T03:32:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T03:32:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T03:32:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T03:32:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T03:32:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T03:32:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T03:32:08.899 INFO:teuthology.orchestra.run.smithi055.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T03:32:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T03:32:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T03:32:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-06T03:32:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T03:32:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T03:32:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T03:32:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T03:32:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T03:32:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-06T03:32:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T03:32:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T03:32:08.900 INFO:teuthology.orchestra.run.smithi055.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T03:32:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-06T03:32:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-06T03:32:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T03:32:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T03:32:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T03:32:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T03:32:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T03:32:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T03:32:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-06T03:32:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.901 INFO:teuthology.orchestra.run.smithi055.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T03:32:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T03:32:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T03:32:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T03:32:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-06T03:32:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T03:32:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T03:32:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T03:32:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T03:32:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T03:32:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T03:32:08.902 INFO:teuthology.orchestra.run.smithi055.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T03:32:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T03:32:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T03:32:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T03:32:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T03:32:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-06T03:32:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T03:32:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-06T03:32:08.903 INFO:teuthology.orchestra.run.smithi055.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T03:32:08.904 INFO:teuthology.orchestra.run.smithi055.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-06T03:32:08.904 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:08.904 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2023-10-06T03:32:09.191 DEBUG:teuthology.parallel:result is None 2023-10-06T03:32:09.191 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T03:32:09.339 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T03:32:09.387 INFO:teuthology.orchestra.run.smithi055.stdout:17.2.6-1296.g736ac5ba.el8 2023-10-06T03:32:09.388 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1296.g736ac5ba.el8 2023-10-06T03:32:09.388 INFO:teuthology.task.install:The correct ceph version 17.2.6-1296.g736ac5ba is installed. 2023-10-06T03:32:09.389 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T03:32:09.536 DEBUG:teuthology.orchestra.run.smithi163:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T03:32:09.588 INFO:teuthology.orchestra.run.smithi163.stdout:17.2.6-1296.g736ac5ba.el8 2023-10-06T03:32:09.589 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1296.g736ac5ba.el8 2023-10-06T03:32:09.589 INFO:teuthology.task.install:The correct ceph version 17.2.6-1296.g736ac5ba is installed. 2023-10-06T03:32:09.589 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T03:32:09.589 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:09.590 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T03:32:09.674 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:09.675 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T03:32:09.760 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T03:32:09.794 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:09.794 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T03:32:09.860 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T03:32:09.963 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:09.963 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T03:32:10.027 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T03:32:10.129 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T03:32:10.129 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:10.129 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T03:32:10.192 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T03:32:10.298 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:10.298 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T03:32:10.359 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T03:32:10.460 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T03:32:10.460 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:10.461 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T03:32:10.526 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T03:32:10.629 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:10.629 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T03:32:10.690 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T03:32:10.791 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T03:32:10.848 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T03:32:10.849 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 777 /var/log/ceph 2023-10-06T03:32:10.853 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 777 /var/log/ceph 2023-10-06T03:32:10.913 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T03:32:10.913 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T03:32:10.918 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T03:32:10.979 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T03:32:10.979 DEBUG:teuthology.orchestra.run.smithi055:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T03:32:10.983 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T03:32:11.050 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T03:32:11.051 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T03:32:11.051 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T03:32:11.051 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:11.051 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T03:32:11.088 DEBUG:teuthology.orchestra.run.smithi055:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T03:32:11.296 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T03:32:11.360 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:11.360 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T03:32:11.396 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T03:32:11.593 DEBUG:teuthology.orchestra.run.smithi163:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T03:32:11.660 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T03:32:11.660 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T03:32:11.661 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199', 'branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412199', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-10-05-0730-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'sha1': 'f1e315a9732d36c05ec316678e2397dc58561c7f'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7519, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f1e315a9732d36c05ec316678e2397dc58561c7f', 'targets': {'smithi055.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGqd+edrJM33HqoVOXuByo5UCZMCd9lwtqSdRynCQEwqmfYkHm6qMqUcBj6r1uCSm4Q/dDGu1v0wzAQCwCu2NcM=', 'smithi163.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbdyRKYMgXgX8qIaeWtbeZtloToSZebR7zRlsyDZRCmJ9QOXny++FCvXp1BImVGFJHlIh2orVhQWKxCENrLDWI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:52:50', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T03:32:11.661 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T03:32:11.666 DEBUG:teuthology.orchestra.run.smithi163:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T03:32:11.701 DEBUG:teuthology.orchestra.run.smithi055:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T03:32:11.746 DEBUG:teuthology.orchestra.run.smithi163:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T03:32:11.807 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:11.808 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T03:32:11.844 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T03:32:11.844 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_1 2023-10-06T03:32:11.923 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T03:32:11.923 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:32:11.923 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-10-06T03:32:11.923 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:32:11.923 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:32:11.923 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2023-10-06 03:32:01.682779545 +0000 2023-10-06T03:32:11.923 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2023-10-06 03:29:05.766605615 +0000 2023-10-06T03:32:11.923 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2023-10-06 03:29:05.766605615 +0000 2023-10-06T03:32:11.923 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2023-10-06T03:32:11.924 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T03:32:12.023 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2023-10-06T03:32:12.024 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2023-10-06T03:32:12.024 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000520032 s, 985 kB/s 2023-10-06T03:32:12.027 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T03:32:12.068 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_2 2023-10-06T03:32:12.146 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T03:32:12.146 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:32:12.146 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-10-06T03:32:12.146 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:32:12.146 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:32:12.147 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2023-10-06 03:32:01.682779545 +0000 2023-10-06T03:32:12.147 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2023-10-06 03:29:06.567589853 +0000 2023-10-06T03:32:12.147 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2023-10-06 03:29:06.567589853 +0000 2023-10-06T03:32:12.147 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2023-10-06T03:32:12.147 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T03:32:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2023-10-06T03:32:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2023-10-06T03:32:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000402669 s, 1.3 MB/s 2023-10-06T03:32:12.250 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T03:32:12.289 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_3 2023-10-06T03:32:12.369 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T03:32:12.370 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:32:12.370 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-10-06T03:32:12.370 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:32:12.370 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:32:12.370 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2023-10-06 03:32:01.682779545 +0000 2023-10-06T03:32:12.370 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2023-10-06 03:29:07.385573756 +0000 2023-10-06T03:32:12.370 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2023-10-06 03:29:07.385573756 +0000 2023-10-06T03:32:12.370 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2023-10-06T03:32:12.371 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T03:32:12.471 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2023-10-06T03:32:12.471 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2023-10-06T03:32:12.471 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000347328 s, 1.5 MB/s 2023-10-06T03:32:12.474 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T03:32:12.514 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_4 2023-10-06T03:32:12.594 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T03:32:12.594 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:32:12.594 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-10-06T03:32:12.594 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:32:12.594 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:32:12.594 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2023-10-06 03:32:01.682779545 +0000 2023-10-06T03:32:12.595 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2023-10-06 03:29:08.210557522 +0000 2023-10-06T03:32:12.595 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2023-10-06 03:29:08.210557522 +0000 2023-10-06T03:32:12.595 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: - 2023-10-06T03:32:12.595 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T03:32:12.695 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2023-10-06T03:32:12.696 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2023-10-06T03:32:12.696 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000374833 s, 1.4 MB/s 2023-10-06T03:32:12.699 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T03:32:12.739 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:32:12.739 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:12.739 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T03:32:12.779 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T03:32:12.779 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_1 2023-10-06T03:32:12.860 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T03:32:12.860 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:32:12.860 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-10-06T03:32:12.860 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:32:12.860 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:32:12.860 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2023-10-06 03:31:56.218864017 +0000 2023-10-06T03:32:12.860 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2023-10-06 03:28:52.688470220 +0000 2023-10-06T03:32:12.860 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2023-10-06 03:28:52.688470220 +0000 2023-10-06T03:32:12.861 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2023-10-06T03:32:12.861 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T03:32:12.960 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2023-10-06T03:32:12.960 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2023-10-06T03:32:12.961 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.00054516 s, 939 kB/s 2023-10-06T03:32:12.963 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T03:32:13.001 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_2 2023-10-06T03:32:13.079 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T03:32:13.079 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:32:13.079 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-10-06T03:32:13.079 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:32:13.079 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:32:13.079 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2023-10-06 03:31:56.218864017 +0000 2023-10-06T03:32:13.079 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2023-10-06 03:28:53.479452211 +0000 2023-10-06T03:32:13.079 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2023-10-06 03:28:53.479452211 +0000 2023-10-06T03:32:13.079 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2023-10-06T03:32:13.080 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T03:32:13.176 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2023-10-06T03:32:13.176 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2023-10-06T03:32:13.176 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000305624 s, 1.7 MB/s 2023-10-06T03:32:13.179 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T03:32:13.217 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_3 2023-10-06T03:32:13.293 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T03:32:13.293 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:32:13.294 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-10-06T03:32:13.294 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:32:13.294 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:32:13.294 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2023-10-06 03:31:56.218864017 +0000 2023-10-06T03:32:13.294 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2023-10-06 03:28:54.258434475 +0000 2023-10-06T03:32:13.294 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2023-10-06 03:28:54.258434475 +0000 2023-10-06T03:32:13.294 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2023-10-06T03:32:13.294 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T03:32:13.392 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2023-10-06T03:32:13.392 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2023-10-06T03:32:13.392 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000321479 s, 1.6 MB/s 2023-10-06T03:32:13.395 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T03:32:13.432 DEBUG:teuthology.orchestra.run.smithi163:> stat /dev/vg_nvme/lv_4 2023-10-06T03:32:13.509 INFO:teuthology.orchestra.run.smithi163.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T03:32:13.509 INFO:teuthology.orchestra.run.smithi163.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T03:32:13.510 INFO:teuthology.orchestra.run.smithi163.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-10-06T03:32:13.510 INFO:teuthology.orchestra.run.smithi163.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T03:32:13.510 INFO:teuthology.orchestra.run.smithi163.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T03:32:13.510 INFO:teuthology.orchestra.run.smithi163.stdout:Access: 2023-10-06 03:31:56.219863994 +0000 2023-10-06T03:32:13.510 INFO:teuthology.orchestra.run.smithi163.stdout:Modify: 2023-10-06 03:28:55.067416056 +0000 2023-10-06T03:32:13.510 INFO:teuthology.orchestra.run.smithi163.stdout:Change: 2023-10-06 03:28:55.067416056 +0000 2023-10-06T03:32:13.510 INFO:teuthology.orchestra.run.smithi163.stdout: Birth: - 2023-10-06T03:32:13.510 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T03:32:13.607 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records in 2023-10-06T03:32:13.607 INFO:teuthology.orchestra.run.smithi163.stderr:1+0 records out 2023-10-06T03:32:13.607 INFO:teuthology.orchestra.run.smithi163.stderr:512 bytes copied, 0.000503713 s, 1.0 MB/s 2023-10-06T03:32:13.610 DEBUG:teuthology.orchestra.run.smithi163:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T03:32:13.648 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:32:13.649 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi055.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi163.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T03:32:13.649 INFO:tasks.ceph:Generating config... 2023-10-06T03:32:13.650 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T03:32:13.650 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T03:32:13.650 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T03:32:13.650 INFO:tasks.ceph:[client] pid file = 2023-10-06T03:32:13.650 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T03:32:13.651 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T03:32:13.651 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-06T03:32:13.651 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T03:32:13.651 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T03:32:13.651 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T03:32:13.651 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T03:32:13.651 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T03:32:13.651 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T03:32:13.651 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T03:32:13.651 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T03:32:13.651 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T03:32:13.652 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-06T03:32:13.652 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-06T03:32:13.652 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T03:32:13.652 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T03:32:13.652 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T03:32:13.652 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T03:32:13.652 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T03:32:13.652 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T03:32:13.652 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-06T03:32:13.652 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-06T03:32:13.652 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-06T03:32:13.653 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T03:32:13.653 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T03:32:13.653 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-06T03:32:13.653 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-06T03:32:13.653 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T03:32:13.653 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T03:32:13.653 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-06T03:32:13.653 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T03:32:13.653 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T03:32:13.653 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T03:32:13.654 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T03:32:13.654 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T03:32:13.654 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T03:32:13.654 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T03:32:13.654 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T03:32:13.749 INFO:teuthology.orchestra.run.smithi055.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T03:32:13.754 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T03:32:13.845 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T03:32:13.908 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.55'), ('mon.b', '172.21.15.163'), ('mon.c', '[v2:172.21.15.163:3301,v1:172.21.15.163:6790]')] 2023-10-06T03:32:13.908 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.55,172.21.15.163,[v2:172.21.15.163:3301,v1:172.21.15.163:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T03:32:13.909 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:13.909 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T03:32:13.947 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.55 --add b 172.21.15.163 --addv c '[v2:172.21.15.163:3301,v1:172.21.15.163:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:32:14.062 INFO:teuthology.orchestra.run.smithi055.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:32:14.062 INFO:teuthology.orchestra.run.smithi055.stdout:monmaptool: generated fsid 8132c8ba-ee1c-41a2-b781-913925034ab6 2023-10-06T03:32:14.062 INFO:teuthology.orchestra.run.smithi055.stdout:setting min_mon_release = octopus 2023-10-06T03:32:14.062 INFO:teuthology.orchestra.run.smithi055.stdout:epoch 0 2023-10-06T03:32:14.062 INFO:teuthology.orchestra.run.smithi055.stdout:fsid 8132c8ba-ee1c-41a2-b781-913925034ab6 2023-10-06T03:32:14.062 INFO:teuthology.orchestra.run.smithi055.stdout:last_changed 2023-10-06T03:32:14.061903+0000 2023-10-06T03:32:14.063 INFO:teuthology.orchestra.run.smithi055.stdout:created 2023-10-06T03:32:14.061903+0000 2023-10-06T03:32:14.063 INFO:teuthology.orchestra.run.smithi055.stdout:min_mon_release 15 (octopus) 2023-10-06T03:32:14.063 INFO:teuthology.orchestra.run.smithi055.stdout:election_strategy: 1 2023-10-06T03:32:14.063 INFO:teuthology.orchestra.run.smithi055.stdout:0: [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon.a 2023-10-06T03:32:14.063 INFO:teuthology.orchestra.run.smithi055.stdout:1: [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] mon.b 2023-10-06T03:32:14.063 INFO:teuthology.orchestra.run.smithi055.stdout:2: [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] mon.c 2023-10-06T03:32:14.063 INFO:teuthology.orchestra.run.smithi055.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T03:32:14.065 DEBUG:teuthology.orchestra.run.smithi055:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T03:32:14.101 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8132c8ba-ee1c-41a2-b781-913925034ab6... 2023-10-06T03:32:14.102 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T03:32:14.146 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T03:32:14.258 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2023-10-06T03:32:14.258 INFO:teuthology.orchestra.run.smithi055.stdout: chdir = "" 2023-10-06T03:32:14.258 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T03:32:14.258 INFO:teuthology.orchestra.run.smithi055.stdout: auth supported = cephx 2023-10-06T03:32:14.258 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stdout: filestore xattr use omap = true 2023-10-06T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stdout: mon clock drift allowed = 1.000 2023-10-06T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stdout: osd crush chooseleaf type = 0 2023-10-06T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stdout: auth debug = true 2023-10-06T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on old message = true 2023-10-06T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on bug = true 2023-10-06T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.259 INFO:teuthology.orchestra.run.smithi055.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T03:32:14.260 INFO:teuthology.orchestra.run.smithi055.stdout: mon pg warn max object skew = 0 2023-10-06T03:32:14.260 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.260 INFO:teuthology.orchestra.run.smithi055.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T03:32:14.260 INFO:teuthology.orchestra.run.smithi055.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T03:32:14.260 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.260 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default size = 2 2023-10-06T03:32:14.260 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.260 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow primary affinity = true 2023-10-06T03:32:14.260 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd allow pg remap = true 2023-10-06T03:32:14.260 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on legacy crush tunables = false 2023-10-06T03:32:14.260 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on crush straw calc version zero = false 2023-10-06T03:32:14.260 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on no sortbitwise = false 2023-10-06T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on osd down out interval zero = false 2023-10-06T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stdout: mon warn on too few osds = false 2023-10-06T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stdout: mon_allow_pool_size_one = true 2023-10-06T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stdout: osd default data pool replay window = 5 2023-10-06T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stdout: mon allow pool delete = true 2023-10-06T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.261 INFO:teuthology.orchestra.run.smithi055.stdout: mon cluster log file level = debug 2023-10-06T03:32:14.262 INFO:teuthology.orchestra.run.smithi055.stdout: debug asserts on shutdown = true 2023-10-06T03:32:14.262 INFO:teuthology.orchestra.run.smithi055.stdout: mon health detail to clog = false 2023-10-06T03:32:14.262 INFO:teuthology.orchestra.run.smithi055.stdout: mon host = "172.21.15.55,172.21.15.163,[v2:172.21.15.163:3301,v1:172.21.15.163:6790]" 2023-10-06T03:32:14.262 INFO:teuthology.orchestra.run.smithi055.stdout: ms die on skipped message = False 2023-10-06T03:32:14.262 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = 8132c8ba-ee1c-41a2-b781-913925034ab6 2023-10-06T03:32:14.262 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.262 INFO:teuthology.orchestra.run.smithi055.stdout:[osd] 2023-10-06T03:32:14.262 INFO:teuthology.orchestra.run.smithi055.stdout: osd journal size = 100 2023-10-06T03:32:14.262 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.262 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub load threshold = 5.0 2023-10-06T03:32:14.262 INFO:teuthology.orchestra.run.smithi055.stdout: osd scrub max interval = 600 2023-10-06T03:32:14.262 INFO:teuthology.orchestra.run.smithi055.stdout: osd mclock profile = high_recovery_ops 2023-10-06T03:32:14.263 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.263 INFO:teuthology.orchestra.run.smithi055.stdout: osd recover clone overlap = true 2023-10-06T03:32:14.263 INFO:teuthology.orchestra.run.smithi055.stdout: osd recovery max chunk = 1048576 2023-10-06T03:32:14.263 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.263 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug shutdown = true 2023-10-06T03:32:14.263 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug op order = true 2023-10-06T03:32:14.263 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug verify stray on activate = true 2023-10-06T03:32:14.263 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.263 INFO:teuthology.orchestra.run.smithi055.stdout: osd open classes on start = true 2023-10-06T03:32:14.263 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug pg log writeout = true 2023-10-06T03:32:14.263 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.263 INFO:teuthology.orchestra.run.smithi055.stdout: osd deep scrub update digest min age = 30 2023-10-06T03:32:14.264 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.264 INFO:teuthology.orchestra.run.smithi055.stdout: osd map max advance = 10 2023-10-06T03:32:14.264 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.264 INFO:teuthology.orchestra.run.smithi055.stdout: journal zero on create = true 2023-10-06T03:32:14.264 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.264 INFO:teuthology.orchestra.run.smithi055.stdout: filestore ondisk finisher threads = 3 2023-10-06T03:32:14.264 INFO:teuthology.orchestra.run.smithi055.stdout: filestore apply finisher threads = 3 2023-10-06T03:32:14.264 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.264 INFO:teuthology.orchestra.run.smithi055.stdout: bdev debug aio = true 2023-10-06T03:32:14.264 INFO:teuthology.orchestra.run.smithi055.stdout: osd debug misdirected ops = true 2023-10-06T03:32:14.264 INFO:teuthology.orchestra.run.smithi055.stdout: bdev async discard = True 2023-10-06T03:32:14.264 INFO:teuthology.orchestra.run.smithi055.stdout: bdev enable discard = True 2023-10-06T03:32:14.265 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore allocator = bitmap 2023-10-06T03:32:14.265 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore block size = 96636764160 2023-10-06T03:32:14.265 INFO:teuthology.orchestra.run.smithi055.stdout: bluestore fsck on mount = True 2023-10-06T03:32:14.265 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluefs = 1/20 2023-10-06T03:32:14.265 INFO:teuthology.orchestra.run.smithi055.stdout: debug bluestore = 1/20 2023-10-06T03:32:14.265 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2023-10-06T03:32:14.265 INFO:teuthology.orchestra.run.smithi055.stdout: debug osd = 20 2023-10-06T03:32:14.265 INFO:teuthology.orchestra.run.smithi055.stdout: debug rocksdb = 4/10 2023-10-06T03:32:14.265 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T03:32:14.265 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd full ratio = 0.9 2023-10-06T03:32:14.265 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd nearfull ratio = 0.8 2023-10-06T03:32:14.265 INFO:teuthology.orchestra.run.smithi055.stdout: osd failsafe full ratio = 0.95 2023-10-06T03:32:14.265 INFO:teuthology.orchestra.run.smithi055.stdout: osd objectstore = bluestore 2023-10-06T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2023-10-06T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr] 2023-10-06T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2023-10-06T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stdout: debug mgr = 20 2023-10-06T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2023-10-06T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2023-10-06T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2023-10-06T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2023-10-06T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stdout: mgr/telemetry/nag = false 2023-10-06T03:32:14.266 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2023-10-06T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stdout: debug finisher = 20 2023-10-06T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stdout:[mon] 2023-10-06T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2023-10-06T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stdout: debug mon = 20 2023-10-06T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stdout: debug paxos = 20 2023-10-06T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stdout: debug auth = 20 2023-10-06T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stdout: mon data avail warn = 5 2023-10-06T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stdout: mon mgr mkfs grace = 240 2023-10-06T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min pgs per osd = 4 2023-10-06T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd reporter subtree level = osd 2023-10-06T03:32:14.267 INFO:teuthology.orchestra.run.smithi055.stdout: mon osd prime pg temp = true 2023-10-06T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stdout: mon reweight min bytes per osd = 10 2023-10-06T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stdout: # 1m isn't quite enough 2023-10-06T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stdout: mon_down_mkfs_grace = 2m 2023-10-06T03:32:14.268 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.269 INFO:teuthology.orchestra.run.smithi055.stdout: mon_warn_on_filestore_osds = false 2023-10-06T03:32:14.269 INFO:teuthology.orchestra.run.smithi055.stdout: mon op complaint time = 120 2023-10-06T03:32:14.269 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:14.269 INFO:teuthology.orchestra.run.smithi055.stdout:[client] 2023-10-06T03:32:14.269 INFO:teuthology.orchestra.run.smithi055.stdout: rgw cache enabled = true 2023-10-06T03:32:14.269 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable ops log = true 2023-10-06T03:32:14.269 INFO:teuthology.orchestra.run.smithi055.stdout: rgw enable usage log = true 2023-10-06T03:32:14.269 INFO:teuthology.orchestra.run.smithi055.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T03:32:14.269 INFO:teuthology.orchestra.run.smithi055.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T03:32:14.269 INFO:teuthology.orchestra.run.smithi055.stdout: client mount timeout = 600 2023-10-06T03:32:14.269 INFO:teuthology.orchestra.run.smithi055.stdout: debug client = 20 2023-10-06T03:32:14.269 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2023-10-06T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stdout: pid file = "" 2023-10-06T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2023-10-06T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2023-10-06T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.a] 2023-10-06T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.b] 2023-10-06T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stdout:[mon.c] 2023-10-06T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2023-10-06T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2023-10-06T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2023-10-06T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d] 2023-10-06T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stdout:[mds] 2023-10-06T03:32:14.270 INFO:teuthology.orchestra.run.smithi055.stdout: debug mds = 20 2023-10-06T03:32:14.271 INFO:teuthology.orchestra.run.smithi055.stdout: debug ms = 1 2023-10-06T03:32:14.271 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug frag = True 2023-10-06T03:32:14.271 INFO:teuthology.orchestra.run.smithi055.stdout: mds debug scatterstat = True 2023-10-06T03:32:14.271 INFO:teuthology.orchestra.run.smithi055.stdout: mds op complaint time = 180 2023-10-06T03:32:14.271 INFO:teuthology.orchestra.run.smithi055.stdout: mds verify scatter = True 2023-10-06T03:32:14.271 INFO:teuthology.orchestra.run.smithi055.stdout: osd op complaint time = 180 2023-10-06T03:32:14.271 INFO:teuthology.orchestra.run.smithi055.stdout: rados mon op timeout = 900 2023-10-06T03:32:14.271 INFO:teuthology.orchestra.run.smithi055.stdout: rados osd op timeout = 900 2023-10-06T03:32:14.271 INFO:teuthology.orchestra.run.smithi163.stdout:[global] 2023-10-06T03:32:14.272 INFO:teuthology.orchestra.run.smithi163.stdout: chdir = "" 2023-10-06T03:32:14.272 INFO:teuthology.orchestra.run.smithi163.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T03:32:14.272 INFO:teuthology.orchestra.run.smithi163.stdout: auth supported = cephx 2023-10-06T03:32:14.272 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.272 INFO:teuthology.orchestra.run.smithi163.stdout: filestore xattr use omap = true 2023-10-06T03:32:14.272 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.272 INFO:teuthology.orchestra.run.smithi163.stdout: mon clock drift allowed = 1.000 2023-10-06T03:32:14.272 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.272 INFO:teuthology.orchestra.run.smithi163.stdout: osd crush chooseleaf type = 0 2023-10-06T03:32:14.272 INFO:teuthology.orchestra.run.smithi163.stdout: auth debug = true 2023-10-06T03:32:14.273 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.273 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on old message = true 2023-10-06T03:32:14.273 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on bug = true 2023-10-06T03:32:14.273 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.273 INFO:teuthology.orchestra.run.smithi163.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T03:32:14.273 INFO:teuthology.orchestra.run.smithi163.stdout: mon pg warn max object skew = 0 2023-10-06T03:32:14.273 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.273 INFO:teuthology.orchestra.run.smithi163.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T03:32:14.273 INFO:teuthology.orchestra.run.smithi163.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T03:32:14.273 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.273 INFO:teuthology.orchestra.run.smithi163.stdout: osd pool default size = 2 2023-10-06T03:32:14.273 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd allow primary affinity = true 2023-10-06T03:32:14.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd allow pg remap = true 2023-10-06T03:32:14.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on legacy crush tunables = false 2023-10-06T03:32:14.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on crush straw calc version zero = false 2023-10-06T03:32:14.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on no sortbitwise = false 2023-10-06T03:32:14.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on osd down out interval zero = false 2023-10-06T03:32:14.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon warn on too few osds = false 2023-10-06T03:32:14.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T03:32:14.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T03:32:14.274 INFO:teuthology.orchestra.run.smithi163.stdout: mon_allow_pool_size_one = true 2023-10-06T03:32:14.274 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.274 INFO:teuthology.orchestra.run.smithi163.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T03:32:14.275 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.275 INFO:teuthology.orchestra.run.smithi163.stdout: osd default data pool replay window = 5 2023-10-06T03:32:14.275 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.275 INFO:teuthology.orchestra.run.smithi163.stdout: mon allow pool delete = true 2023-10-06T03:32:14.275 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.275 INFO:teuthology.orchestra.run.smithi163.stdout: mon cluster log file level = debug 2023-10-06T03:32:14.275 INFO:teuthology.orchestra.run.smithi163.stdout: debug asserts on shutdown = true 2023-10-06T03:32:14.275 INFO:teuthology.orchestra.run.smithi163.stdout: mon health detail to clog = false 2023-10-06T03:32:14.275 INFO:teuthology.orchestra.run.smithi163.stdout: mon host = "172.21.15.55,172.21.15.163,[v2:172.21.15.163:3301,v1:172.21.15.163:6790]" 2023-10-06T03:32:14.275 INFO:teuthology.orchestra.run.smithi163.stdout: ms die on skipped message = False 2023-10-06T03:32:14.275 INFO:teuthology.orchestra.run.smithi163.stdout: fsid = 8132c8ba-ee1c-41a2-b781-913925034ab6 2023-10-06T03:32:14.275 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.276 INFO:teuthology.orchestra.run.smithi163.stdout:[osd] 2023-10-06T03:32:14.276 INFO:teuthology.orchestra.run.smithi163.stdout: osd journal size = 100 2023-10-06T03:32:14.276 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.276 INFO:teuthology.orchestra.run.smithi163.stdout: osd scrub load threshold = 5.0 2023-10-06T03:32:14.276 INFO:teuthology.orchestra.run.smithi163.stdout: osd scrub max interval = 600 2023-10-06T03:32:14.276 INFO:teuthology.orchestra.run.smithi163.stdout: osd mclock profile = high_recovery_ops 2023-10-06T03:32:14.276 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.276 INFO:teuthology.orchestra.run.smithi163.stdout: osd recover clone overlap = true 2023-10-06T03:32:14.276 INFO:teuthology.orchestra.run.smithi163.stdout: osd recovery max chunk = 1048576 2023-10-06T03:32:14.276 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.276 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug shutdown = true 2023-10-06T03:32:14.276 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug op order = true 2023-10-06T03:32:14.276 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug verify stray on activate = true 2023-10-06T03:32:14.277 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.277 INFO:teuthology.orchestra.run.smithi163.stdout: osd open classes on start = true 2023-10-06T03:32:14.277 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug pg log writeout = true 2023-10-06T03:32:14.277 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.277 INFO:teuthology.orchestra.run.smithi163.stdout: osd deep scrub update digest min age = 30 2023-10-06T03:32:14.277 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.277 INFO:teuthology.orchestra.run.smithi163.stdout: osd map max advance = 10 2023-10-06T03:32:14.277 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.277 INFO:teuthology.orchestra.run.smithi163.stdout: journal zero on create = true 2023-10-06T03:32:14.277 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.277 INFO:teuthology.orchestra.run.smithi163.stdout: filestore ondisk finisher threads = 3 2023-10-06T03:32:14.277 INFO:teuthology.orchestra.run.smithi163.stdout: filestore apply finisher threads = 3 2023-10-06T03:32:14.278 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.278 INFO:teuthology.orchestra.run.smithi163.stdout: bdev debug aio = true 2023-10-06T03:32:14.278 INFO:teuthology.orchestra.run.smithi163.stdout: osd debug misdirected ops = true 2023-10-06T03:32:14.278 INFO:teuthology.orchestra.run.smithi163.stdout: bdev async discard = True 2023-10-06T03:32:14.278 INFO:teuthology.orchestra.run.smithi163.stdout: bdev enable discard = True 2023-10-06T03:32:14.278 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore allocator = bitmap 2023-10-06T03:32:14.278 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore block size = 96636764160 2023-10-06T03:32:14.278 INFO:teuthology.orchestra.run.smithi163.stdout: bluestore fsck on mount = True 2023-10-06T03:32:14.278 INFO:teuthology.orchestra.run.smithi163.stdout: debug bluefs = 1/20 2023-10-06T03:32:14.278 INFO:teuthology.orchestra.run.smithi163.stdout: debug bluestore = 1/20 2023-10-06T03:32:14.278 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2023-10-06T03:32:14.278 INFO:teuthology.orchestra.run.smithi163.stdout: debug osd = 20 2023-10-06T03:32:14.279 INFO:teuthology.orchestra.run.smithi163.stdout: debug rocksdb = 4/10 2023-10-06T03:32:14.279 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T03:32:14.279 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd full ratio = 0.9 2023-10-06T03:32:14.279 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd nearfull ratio = 0.8 2023-10-06T03:32:14.279 INFO:teuthology.orchestra.run.smithi163.stdout: osd failsafe full ratio = 0.95 2023-10-06T03:32:14.279 INFO:teuthology.orchestra.run.smithi163.stdout: osd objectstore = bluestore 2023-10-06T03:32:14.279 INFO:teuthology.orchestra.run.smithi163.stdout: osd op complaint time = 180 2023-10-06T03:32:14.279 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.279 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr] 2023-10-06T03:32:14.279 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2023-10-06T03:32:14.279 INFO:teuthology.orchestra.run.smithi163.stdout: debug mgr = 20 2023-10-06T03:32:14.279 INFO:teuthology.orchestra.run.smithi163.stdout: debug mon = 20 2023-10-06T03:32:14.280 INFO:teuthology.orchestra.run.smithi163.stdout: debug auth = 20 2023-10-06T03:32:14.280 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min pgs per osd = 4 2023-10-06T03:32:14.280 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min bytes per osd = 10 2023-10-06T03:32:14.280 INFO:teuthology.orchestra.run.smithi163.stdout: mgr/telemetry/nag = false 2023-10-06T03:32:14.280 INFO:teuthology.orchestra.run.smithi163.stdout: debug client = 20 2023-10-06T03:32:14.280 INFO:teuthology.orchestra.run.smithi163.stdout: debug finisher = 20 2023-10-06T03:32:14.280 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.280 INFO:teuthology.orchestra.run.smithi163.stdout:[mon] 2023-10-06T03:32:14.280 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2023-10-06T03:32:14.280 INFO:teuthology.orchestra.run.smithi163.stdout: debug mon = 20 2023-10-06T03:32:14.280 INFO:teuthology.orchestra.run.smithi163.stdout: debug paxos = 20 2023-10-06T03:32:14.280 INFO:teuthology.orchestra.run.smithi163.stdout: debug auth = 20 2023-10-06T03:32:14.281 INFO:teuthology.orchestra.run.smithi163.stdout: mon data avail warn = 5 2023-10-06T03:32:14.281 INFO:teuthology.orchestra.run.smithi163.stdout: mon mgr mkfs grace = 240 2023-10-06T03:32:14.281 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min pgs per osd = 4 2023-10-06T03:32:14.281 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd reporter subtree level = osd 2023-10-06T03:32:14.281 INFO:teuthology.orchestra.run.smithi163.stdout: mon osd prime pg temp = true 2023-10-06T03:32:14.281 INFO:teuthology.orchestra.run.smithi163.stdout: mon reweight min bytes per osd = 10 2023-10-06T03:32:14.281 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.281 INFO:teuthology.orchestra.run.smithi163.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T03:32:14.281 INFO:teuthology.orchestra.run.smithi163.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T03:32:14.281 INFO:teuthology.orchestra.run.smithi163.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T03:32:14.281 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.281 INFO:teuthology.orchestra.run.smithi163.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T03:32:14.282 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T03:32:14.282 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T03:32:14.282 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.282 INFO:teuthology.orchestra.run.smithi163.stdout: # 1m isn't quite enough 2023-10-06T03:32:14.282 INFO:teuthology.orchestra.run.smithi163.stdout: mon_down_mkfs_grace = 2m 2023-10-06T03:32:14.282 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.282 INFO:teuthology.orchestra.run.smithi163.stdout: mon_warn_on_filestore_osds = false 2023-10-06T03:32:14.282 INFO:teuthology.orchestra.run.smithi163.stdout: mon op complaint time = 120 2023-10-06T03:32:14.282 INFO:teuthology.orchestra.run.smithi163.stdout: 2023-10-06T03:32:14.282 INFO:teuthology.orchestra.run.smithi163.stdout:[client] 2023-10-06T03:32:14.282 INFO:teuthology.orchestra.run.smithi163.stdout: rgw cache enabled = true 2023-10-06T03:32:14.283 INFO:teuthology.orchestra.run.smithi163.stdout: rgw enable ops log = true 2023-10-06T03:32:14.283 INFO:teuthology.orchestra.run.smithi163.stdout: rgw enable usage log = true 2023-10-06T03:32:14.283 INFO:teuthology.orchestra.run.smithi163.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T03:32:14.283 INFO:teuthology.orchestra.run.smithi163.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T03:32:14.283 INFO:teuthology.orchestra.run.smithi163.stdout: client mount timeout = 600 2023-10-06T03:32:14.283 INFO:teuthology.orchestra.run.smithi163.stdout: debug client = 20 2023-10-06T03:32:14.283 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2023-10-06T03:32:14.283 INFO:teuthology.orchestra.run.smithi163.stdout: pid file = "" 2023-10-06T03:32:14.283 INFO:teuthology.orchestra.run.smithi163.stdout: rados mon op timeout = 900 2023-10-06T03:32:14.283 INFO:teuthology.orchestra.run.smithi163.stdout: rados osd op timeout = 900 2023-10-06T03:32:14.283 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.a] 2023-10-06T03:32:14.283 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.b] 2023-10-06T03:32:14.283 INFO:teuthology.orchestra.run.smithi163.stdout:[mon.c] 2023-10-06T03:32:14.284 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.a] 2023-10-06T03:32:14.284 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.c] 2023-10-06T03:32:14.284 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.b] 2023-10-06T03:32:14.284 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.d] 2023-10-06T03:32:14.284 INFO:teuthology.orchestra.run.smithi163.stdout:[mds] 2023-10-06T03:32:14.284 INFO:teuthology.orchestra.run.smithi163.stdout: debug mds = 20 2023-10-06T03:32:14.284 INFO:teuthology.orchestra.run.smithi163.stdout: debug ms = 1 2023-10-06T03:32:14.284 INFO:teuthology.orchestra.run.smithi163.stdout: mds debug frag = True 2023-10-06T03:32:14.284 INFO:teuthology.orchestra.run.smithi163.stdout: mds debug scatterstat = True 2023-10-06T03:32:14.284 INFO:teuthology.orchestra.run.smithi163.stdout: mds op complaint time = 180 2023-10-06T03:32:14.284 INFO:teuthology.orchestra.run.smithi163.stdout: mds verify scatter = True 2023-10-06T03:32:14.284 INFO:teuthology.orchestra.run.smithi163.stdout: osd op complaint time = 180 2023-10-06T03:32:14.285 INFO:teuthology.orchestra.run.smithi163.stdout: rados mon op timeout = 900 2023-10-06T03:32:14.285 INFO:teuthology.orchestra.run.smithi163.stdout: rados osd op timeout = 900 2023-10-06T03:32:14.292 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T03:32:14.293 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T03:32:14.399 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T03:32:14.399 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:14.399 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T03:32:14.435 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:14.435 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T03:32:14.511 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T03:32:14.511 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:14.512 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T03:32:14.512 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T03:32:14.645 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:14.645 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:32:14.726 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi163.front.sepia.ceph.com 2023-10-06T03:32:14.726 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:14.726 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T03:32:14.726 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T03:32:14.814 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:14.814 DEBUG:teuthology.orchestra.run.smithi163:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:32:14.889 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T03:32:14.890 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T03:32:14.890 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T03:32:15.003 INFO:teuthology.orchestra.run.smithi055.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T03:32:15.009 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T03:32:15.303 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T03:32:15.309 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T03:32:15.309 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T03:32:15.425 INFO:teuthology.orchestra.run.smithi055.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T03:32:15.431 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T03:32:15.495 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T03:32:15.614 INFO:teuthology.orchestra.run.smithi055.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T03:32:15.619 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T03:32:15.684 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T03:32:15.801 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T03:32:15.805 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T03:32:15.867 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T03:32:15.977 INFO:teuthology.orchestra.run.smithi163.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T03:32:15.982 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T03:32:16.043 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T03:32:16.043 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T03:32:16.130 INFO:teuthology.orchestra.run.smithi055.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T03:32:16.164 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-06T03:32:16.251 INFO:teuthology.orchestra.run.smithi055.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-06T03:32:16.285 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T03:32:16.372 INFO:teuthology.orchestra.run.smithi163.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T03:32:16.403 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-06T03:32:16.485 INFO:teuthology.orchestra.run.smithi163.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-06T03:32:16.516 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T03:32:16.517 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi055.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi163.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T03:32:16.517 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T03:32:16.581 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:32:16.581 INFO:tasks.ceph:role: osd.0 2023-10-06T03:32:16.581 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T03:32:16.581 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T03:32:16.672 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:32:16.672 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:32:16.672 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:32:16.672 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 2023-10-06T03:32:16.672 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:32:16.672 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:32:16.672 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:32:16.672 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:32:16.672 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:32:16.672 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:32:17.959 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2023-10-06T03:32:17.963 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2023-10-06T03:32:17.964 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T03:32:18.069 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T03:32:18.141 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T03:32:18.208 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:32:18.208 INFO:tasks.ceph:role: osd.1 2023-10-06T03:32:18.208 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T03:32:18.208 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T03:32:18.299 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:32:18.299 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:32:18.299 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:32:18.299 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 2023-10-06T03:32:18.300 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:32:18.300 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:32:18.300 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:32:18.300 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:32:18.300 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:32:18.300 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:32:19.596 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2023-10-06T03:32:19.601 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2023-10-06T03:32:19.601 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T03:32:19.703 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T03:32:19.776 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T03:32:19.839 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:32:19.840 INFO:tasks.ceph:role: osd.2 2023-10-06T03:32:19.840 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T03:32:19.840 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T03:32:19.929 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:32:19.930 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:32:19.930 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:32:19.930 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 2023-10-06T03:32:19.930 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:32:19.930 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:32:19.930 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:32:19.930 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:32:19.930 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:32:19.930 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:32:21.220 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2023-10-06T03:32:21.225 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2023-10-06T03:32:21.226 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T03:32:21.332 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T03:32:21.401 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T03:32:21.469 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T03:32:21.469 INFO:tasks.ceph:role: osd.3 2023-10-06T03:32:21.469 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T03:32:21.469 DEBUG:teuthology.orchestra.run.smithi055:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T03:32:21.562 INFO:teuthology.orchestra.run.smithi055.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:32:21.562 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:32:21.562 INFO:teuthology.orchestra.run.smithi055.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:32:21.562 INFO:teuthology.orchestra.run.smithi055.stdout: = reflink=1 2023-10-06T03:32:21.562 INFO:teuthology.orchestra.run.smithi055.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:32:21.562 INFO:teuthology.orchestra.run.smithi055.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:32:21.563 INFO:teuthology.orchestra.run.smithi055.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:32:21.563 INFO:teuthology.orchestra.run.smithi055.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:32:21.563 INFO:teuthology.orchestra.run.smithi055.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:32:21.563 INFO:teuthology.orchestra.run.smithi055.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:32:22.856 INFO:teuthology.orchestra.run.smithi055.stdout:Discarding blocks...Done. 2023-10-06T03:32:22.860 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi055.front.sepia.ceph.com -o noatime 2023-10-06T03:32:22.860 DEBUG:teuthology.orchestra.run.smithi055:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T03:32:22.964 DEBUG:teuthology.orchestra.run.smithi055:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T03:32:23.035 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:32:23.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:23.227+0000 7f94bc157540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T03:32:23.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:23.227+0000 7f94bc157540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T03:32:23.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:23.227+0000 7f94bc157540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T03:32:23.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:23.227+0000 7f94bc157540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T03:32:23.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:23.227+0000 7f94bc157540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T03:32:23.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:23.230+0000 7f94bc157540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T03:32:23.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:23.778+0000 7f94bc157540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:24.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:24.550+0000 7f94bc157540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:25.328+0000 7f94bc157540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:26.152 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T03:32:26.214 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:32:26.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:26.306+0000 7f9184470540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T03:32:26.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:26.307+0000 7f9184470540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T03:32:26.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:26.307+0000 7f9184470540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T03:32:26.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:26.307+0000 7f9184470540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T03:32:26.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:26.307+0000 7f9184470540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T03:32:26.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:26.309+0000 7f9184470540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T03:32:26.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:26.850+0000 7f9184470540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:27.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:27.629+0000 7f9184470540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:28.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:28.403+0000 7f9184470540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:29.257 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T03:32:29.323 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:32:29.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:29.422+0000 7f14cc0d0540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T03:32:29.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:29.423+0000 7f14cc0d0540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T03:32:29.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:29.423+0000 7f14cc0d0540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T03:32:29.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:29.423+0000 7f14cc0d0540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T03:32:29.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:29.423+0000 7f14cc0d0540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T03:32:29.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:29.424+0000 7f14cc0d0540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T03:32:29.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:29.974+0000 7f14cc0d0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:30.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:30.744+0000 7f14cc0d0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:31.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:31.531+0000 7f14cc0d0540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:32.374 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T03:32:32.439 DEBUG:teuthology.orchestra.run.smithi055:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:32:32.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:32.531+0000 7fa227abb540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T03:32:32.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:32.532+0000 7fa227abb540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T03:32:32.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:32.532+0000 7fa227abb540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T03:32:32.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:32.532+0000 7fa227abb540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T03:32:32.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:32.532+0000 7fa227abb540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T03:32:32.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:32.534+0000 7fa227abb540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T03:32:33.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:33.085+0000 7fa227abb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:33.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:33.860+0000 7fa227abb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:34.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:34.643+0000 7fa227abb540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:35.487 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T03:32:35.549 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T03:32:35.621 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:32:35.622 INFO:tasks.ceph:role: osd.4 2023-10-06T03:32:35.622 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-06T03:32:35.622 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T03:32:35.693 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:32:35.693 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:32:35.693 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:32:35.693 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 2023-10-06T03:32:35.693 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:32:35.693 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:32:35.693 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:32:35.693 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:32:35.693 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:32:35.694 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:32:36.124 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2023-10-06T03:32:36.128 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2023-10-06T03:32:36.128 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T03:32:36.318 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T03:32:36.388 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T03:32:36.450 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:32:36.450 INFO:tasks.ceph:role: osd.5 2023-10-06T03:32:36.450 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-06T03:32:36.450 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T03:32:36.518 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:32:36.518 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:32:36.518 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:32:36.518 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 2023-10-06T03:32:36.519 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:32:36.519 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:32:36.519 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:32:36.519 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:32:36.519 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:32:36.519 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:32:36.930 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2023-10-06T03:32:36.933 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2023-10-06T03:32:36.934 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T03:32:37.061 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T03:32:37.130 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T03:32:37.191 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:32:37.191 INFO:tasks.ceph:role: osd.6 2023-10-06T03:32:37.191 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-06T03:32:37.191 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T03:32:37.260 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:32:37.260 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:32:37.260 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:32:37.260 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 2023-10-06T03:32:37.260 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:32:37.260 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:32:37.260 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:32:37.261 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:32:37.261 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:32:37.261 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:32:37.700 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2023-10-06T03:32:37.704 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2023-10-06T03:32:37.704 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T03:32:37.881 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T03:32:37.952 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T03:32:38.013 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T03:32:38.014 INFO:tasks.ceph:role: osd.7 2023-10-06T03:32:38.014 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-06T03:32:38.014 DEBUG:teuthology.orchestra.run.smithi163:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T03:32:38.082 INFO:teuthology.orchestra.run.smithi163.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T03:32:38.082 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T03:32:38.082 INFO:teuthology.orchestra.run.smithi163.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T03:32:38.082 INFO:teuthology.orchestra.run.smithi163.stdout: = reflink=1 2023-10-06T03:32:38.082 INFO:teuthology.orchestra.run.smithi163.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T03:32:38.082 INFO:teuthology.orchestra.run.smithi163.stdout: = sunit=0 swidth=0 blks 2023-10-06T03:32:38.082 INFO:teuthology.orchestra.run.smithi163.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T03:32:38.082 INFO:teuthology.orchestra.run.smithi163.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T03:32:38.082 INFO:teuthology.orchestra.run.smithi163.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T03:32:38.083 INFO:teuthology.orchestra.run.smithi163.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T03:32:38.516 INFO:teuthology.orchestra.run.smithi163.stdout:Discarding blocks...Done. 2023-10-06T03:32:38.521 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi163.front.sepia.ceph.com -o noatime 2023-10-06T03:32:38.521 DEBUG:teuthology.orchestra.run.smithi163:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T03:32:38.686 DEBUG:teuthology.orchestra.run.smithi163:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T03:32:38.754 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:32:38.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:38.861+0000 7fef62b91540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T03:32:38.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:38.861+0000 7fef62b91540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T03:32:38.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:38.861+0000 7fef62b91540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T03:32:38.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:38.861+0000 7fef62b91540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T03:32:38.864 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:38.861+0000 7fef62b91540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T03:32:38.865 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:38.863+0000 7fef62b91540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T03:32:39.404 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:39.402+0000 7fef62b91540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:40.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:40.161+0000 7fef62b91540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:40.930 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:40.928+0000 7fef62b91540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:41.662 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:32:41.667 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:32:41.757 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T03:32:41.820 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:32:41.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:41.949+0000 7f1a9882a540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T03:32:41.951 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:41.949+0000 7f1a9882a540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T03:32:41.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:41.949+0000 7f1a9882a540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T03:32:41.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:41.949+0000 7f1a9882a540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T03:32:41.952 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:41.949+0000 7f1a9882a540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T03:32:41.953 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:41.951+0000 7f1a9882a540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T03:32:42.490 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:42.488+0000 7f1a9882a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:43.250 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:43.248+0000 7f1a9882a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:44.014 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:44.012+0000 7f1a9882a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:44.826 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T03:32:44.891 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:32:44.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:44.981+0000 7f16b2744540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T03:32:44.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:44.981+0000 7f16b2744540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T03:32:44.983 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:44.981+0000 7f16b2744540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T03:32:44.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:44.981+0000 7f16b2744540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T03:32:44.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:44.981+0000 7f16b2744540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T03:32:44.984 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:44.983+0000 7f16b2744540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T03:32:45.522 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:45.521+0000 7f16b2744540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:46.286 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:46.284+0000 7f16b2744540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:47.048 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:47.046+0000 7f16b2744540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:47.860 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T03:32:47.924 DEBUG:teuthology.orchestra.run.smithi163:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:32:48.015 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:48.013+0000 7f5ebda48540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T03:32:48.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:48.013+0000 7f5ebda48540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T03:32:48.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:48.013+0000 7f5ebda48540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T03:32:48.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:48.013+0000 7f5ebda48540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T03:32:48.016 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:48.014+0000 7f5ebda48540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T03:32:48.017 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:48.015+0000 7f5ebda48540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T03:32:48.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:48.554+0000 7f5ebda48540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:49.318 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:49.316+0000 7f5ebda48540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:50.081 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:50.079+0000 7f5ebda48540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T03:32:50.892 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T03:32:50.955 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T03:32:50.956 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:50.956 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T03:32:51.019 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:51.019 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T03:32:51.123 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:51.123 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T03:32:51.226 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:51.226 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T03:32:51.327 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:51.328 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T03:32:51.430 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:51.430 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T03:32:51.534 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:51.534 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T03:32:51.641 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:51.641 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T03:32:51.705 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:51.705 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T03:32:51.806 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:51.806 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T03:32:51.907 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:51.907 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T03:32:52.007 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:52.008 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T03:32:52.108 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:52.108 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T03:32:52.208 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:52.208 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T03:32:52.308 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:52.308 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T03:32:52.345 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:52.345 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-06T03:32:52.421 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:52.421 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T03:32:52.457 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:52.457 DEBUG:teuthology.orchestra.run.smithi163:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-06T03:32:52.532 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T03:32:52.533 DEBUG:teuthology.orchestra.run.smithi055:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T03:32:52.537 DEBUG:teuthology.orchestra.run.smithi163:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T03:32:52.599 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr.y] 2023-10-06T03:32:52.599 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC/fx9lCwU2ABAAMijxXd8hwgk3XzcdP4NZyg== 2023-10-06T03:32:52.599 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.a] 2023-10-06T03:32:52.599 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC/fx9li2RaGRAAgRjtbw3h40vylEPjolzlpw== 2023-10-06T03:32:52.600 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.c] 2023-10-06T03:32:52.600 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC/fx9lS22fJBAAZi6CfyLyJAFgIkaarad1mA== 2023-10-06T03:32:52.600 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.0] 2023-10-06T03:32:52.600 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDHfx9lmTeaDRAAuycc9i7fPj8xrDkuZCAGVg== 2023-10-06T03:32:52.600 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.1] 2023-10-06T03:32:52.600 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDKfx9lxOdXEhAA9S8tdOvhnmBeEbutqnUsZg== 2023-10-06T03:32:52.600 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.2] 2023-10-06T03:32:52.600 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDNfx9l76I+GRAAjaj0gM7An9BCyGmfQgaDFw== 2023-10-06T03:32:52.600 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.3] 2023-10-06T03:32:52.601 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDQfx9lsPK+HxAAzQCV0EuBenqOkUpiQW9uog== 2023-10-06T03:32:52.601 INFO:teuthology.orchestra.run.smithi055.stdout:[mgr.x] 2023-10-06T03:32:52.601 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC/fx9lQAANEhAAsNiWPqCNEyUS3IELXSC1Cw== 2023-10-06T03:32:52.601 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.b] 2023-10-06T03:32:52.601 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC/fx9lXZu7LxAAu3QAtdDxtD/AuuqGG12sDw== 2023-10-06T03:32:52.601 INFO:teuthology.orchestra.run.smithi055.stdout:[mds.d] 2023-10-06T03:32:52.601 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQC/fx9ll0RIOhAAGtGdPv4Y2bge7xIv3eu2NQ== 2023-10-06T03:32:52.601 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.4] 2023-10-06T03:32:52.601 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDWfx9lSHNrMxAAIn7awDmljO2cRtKyDalWnw== 2023-10-06T03:32:52.601 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.5] 2023-10-06T03:32:52.602 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDZfx9lFoWuOBAAhzQOt3YChvuv0MhrspOjPw== 2023-10-06T03:32:52.602 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.6] 2023-10-06T03:32:52.602 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDcfx9lB8iSOhAAwamSgGIXZ8owFE4FdbOOBw== 2023-10-06T03:32:52.602 INFO:teuthology.orchestra.run.smithi055.stdout:[osd.7] 2023-10-06T03:32:52.602 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDgfx9lUb3mABAAJ75UO2UQg2r+l+tDYVn0Dg== 2023-10-06T03:32:52.602 INFO:teuthology.orchestra.run.smithi055.stdout:[client.0] 2023-10-06T03:32:52.602 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDAfx9lfNvCBxAA1ZJq4XsWTZHVY0u0NdOqIw== 2023-10-06T03:32:52.602 INFO:teuthology.orchestra.run.smithi055.stdout:[client.2] 2023-10-06T03:32:52.602 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDAfx9l+9b4DhAAc5XHdikWbXsEblSE3FGScQ== 2023-10-06T03:32:52.602 INFO:teuthology.orchestra.run.smithi055.stdout:[client.1] 2023-10-06T03:32:52.603 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDAfx9lnvkoFhAAafxsOmItxmwUr1GJrGYJ6g== 2023-10-06T03:32:52.603 INFO:teuthology.orchestra.run.smithi055.stdout:[client.3] 2023-10-06T03:32:52.603 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQDAfx9luSXrHBAAg6Ufa1Ep/EOgsQtuOPQgYg== 2023-10-06T03:32:52.630 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr.y] 2023-10-06T03:32:52.630 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC/fx9lCwU2ABAAMijxXd8hwgk3XzcdP4NZyg== 2023-10-06T03:32:52.630 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.a] 2023-10-06T03:32:52.631 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC/fx9li2RaGRAAgRjtbw3h40vylEPjolzlpw== 2023-10-06T03:32:52.631 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.c] 2023-10-06T03:32:52.631 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC/fx9lS22fJBAAZi6CfyLyJAFgIkaarad1mA== 2023-10-06T03:32:52.631 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.0] 2023-10-06T03:32:52.631 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDHfx9lmTeaDRAAuycc9i7fPj8xrDkuZCAGVg== 2023-10-06T03:32:52.631 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.1] 2023-10-06T03:32:52.631 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDKfx9lxOdXEhAA9S8tdOvhnmBeEbutqnUsZg== 2023-10-06T03:32:52.631 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.2] 2023-10-06T03:32:52.631 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDNfx9l76I+GRAAjaj0gM7An9BCyGmfQgaDFw== 2023-10-06T03:32:52.631 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.3] 2023-10-06T03:32:52.631 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDQfx9lsPK+HxAAzQCV0EuBenqOkUpiQW9uog== 2023-10-06T03:32:52.632 INFO:teuthology.orchestra.run.smithi163.stdout:[mgr.x] 2023-10-06T03:32:52.632 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC/fx9lQAANEhAAsNiWPqCNEyUS3IELXSC1Cw== 2023-10-06T03:32:52.632 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.b] 2023-10-06T03:32:52.632 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC/fx9lXZu7LxAAu3QAtdDxtD/AuuqGG12sDw== 2023-10-06T03:32:52.632 INFO:teuthology.orchestra.run.smithi163.stdout:[mds.d] 2023-10-06T03:32:52.632 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQC/fx9ll0RIOhAAGtGdPv4Y2bge7xIv3eu2NQ== 2023-10-06T03:32:52.632 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.4] 2023-10-06T03:32:52.632 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDWfx9lSHNrMxAAIn7awDmljO2cRtKyDalWnw== 2023-10-06T03:32:52.632 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.5] 2023-10-06T03:32:52.632 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDZfx9lFoWuOBAAhzQOt3YChvuv0MhrspOjPw== 2023-10-06T03:32:52.632 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.6] 2023-10-06T03:32:52.632 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDcfx9lB8iSOhAAwamSgGIXZ8owFE4FdbOOBw== 2023-10-06T03:32:52.633 INFO:teuthology.orchestra.run.smithi163.stdout:[osd.7] 2023-10-06T03:32:52.633 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDgfx9lUb3mABAAJ75UO2UQg2r+l+tDYVn0Dg== 2023-10-06T03:32:52.633 INFO:teuthology.orchestra.run.smithi163.stdout:[client.0] 2023-10-06T03:32:52.633 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDAfx9lfNvCBxAA1ZJq4XsWTZHVY0u0NdOqIw== 2023-10-06T03:32:52.633 INFO:teuthology.orchestra.run.smithi163.stdout:[client.2] 2023-10-06T03:32:52.633 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDAfx9l+9b4DhAAc5XHdikWbXsEblSE3FGScQ== 2023-10-06T03:32:52.633 INFO:teuthology.orchestra.run.smithi163.stdout:[client.1] 2023-10-06T03:32:52.633 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDAfx9lnvkoFhAAafxsOmItxmwUr1GJrGYJ6g== 2023-10-06T03:32:52.633 INFO:teuthology.orchestra.run.smithi163.stdout:[client.3] 2023-10-06T03:32:52.633 INFO:teuthology.orchestra.run.smithi163.stdout: key = AQDAfx9luSXrHBAAg6Ufa1Ep/EOgsQtuOPQgYg== 2023-10-06T03:32:52.634 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T03:32:52.649 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T03:32:52.769 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:32:52.774 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:32:52.872 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:32:52.876 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:32:52.979 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:52.983 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:53.083 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:53.088 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:53.183 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:53.187 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:53.283 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:53.287 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:53.382 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T03:32:53.388 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T03:32:53.483 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:32:53.487 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:32:53.585 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:32:53.590 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T03:32:53.687 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:53.691 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:53.787 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:53.791 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:53.892 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:53.896 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:53.992 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:53.996 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T03:32:54.091 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:32:54.095 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:32:54.194 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:32:54.200 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:32:54.297 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:32:54.301 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:32:54.400 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:32:54.404 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T03:32:54.502 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T03:32:54.503 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T03:32:54.566 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T03:32:54.687 DEBUG:teuthology.orchestra.run.smithi055:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T03:32:54.751 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T03:32:54.813 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T03:32:54.921 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T03:32:54.982 DEBUG:teuthology.orchestra.run.smithi163:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T03:32:55.043 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T03:32:55.151 DEBUG:teuthology.orchestra.run.smithi163:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T03:32:55.214 DEBUG:teuthology.orchestra.run.smithi055:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:32:55.218 DEBUG:teuthology.orchestra.run.smithi163:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T03:32:55.254 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T03:32:55.254 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T03:32:55.255 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T03:32:55.295 INFO:tasks.ceph.mon.a:Started 2023-10-06T03:32:55.295 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T03:32:55.296 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T03:32:55.300 INFO:tasks.ceph.mon.b:Started 2023-10-06T03:32:55.300 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T03:32:55.300 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T03:32:55.304 INFO:tasks.ceph.mon.c:Started 2023-10-06T03:32:55.304 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T03:32:55.305 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T03:32:55.305 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T03:32:55.310 INFO:tasks.ceph.mgr.y:Started 2023-10-06T03:32:55.310 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T03:32:55.311 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T03:32:55.315 INFO:tasks.ceph.mgr.x:Started 2023-10-06T03:32:55.315 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T03:32:55.315 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T03:32:55.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.512+0000 7f29e2d9d700 1 Processor -- start 2023-10-06T03:32:55.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.512+0000 7f29e2d9d700 1 -- start start 2023-10-06T03:32:55.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.514+0000 7f29e2d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f29dc0b1b40 0x7f29dc0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:55.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.514+0000 7f29e2d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29dc0cfb00 con 0x7f29dc0b16d0 2023-10-06T03:32:55.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.514+0000 7f29e2d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29dc0b5c40 con 0x7f29dc0b4e70 2023-10-06T03:32:55.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.514+0000 7f29e2d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29dc0b5f20 con 0x7f29dc0b1b40 2023-10-06T03:32:55.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.514+0000 7f29e159a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f29dc0b16d0 0x7f29dc0b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52740/0 (socket says 172.21.15.55:52740) 2023-10-06T03:32:55.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.514+0000 7f29e159a700 1 -- 172.21.15.55:0/422879434 learned_addr learned my addr 172.21.15.55:0/422879434 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:32:55.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.514+0000 7f29e259c700 1 --2- 172.21.15.55:0/422879434 >> v2:172.21.15.163:3300/0 conn(0x7f29dc0b1b40 0x7f29dc0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:55.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.515+0000 7f29e0d99700 1 -- 172.21.15.55:0/422879434 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130071586 0 0) 0x7f29dc0cfb00 con 0x7f29dc0b16d0 2023-10-06T03:32:55.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.515+0000 7f29e0d99700 1 -- 172.21.15.55:0/422879434 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c8003570 con 0x7f29dc0b16d0 2023-10-06T03:32:55.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.515+0000 7f29e0d99700 1 -- 172.21.15.55:0/422879434 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473077339 0 0) 0x7f29dc0b5c40 con 0x7f29dc0b4e70 2023-10-06T03:32:55.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.515+0000 7f29e0d99700 1 -- 172.21.15.55:0/422879434 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29dc0cfb00 con 0x7f29dc0b4e70 2023-10-06T03:32:55.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.515+0000 7f29e0d99700 1 -- 172.21.15.55:0/422879434 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1756796833 0 0) 0x7f29c8003570 con 0x7f29dc0b16d0 2023-10-06T03:32:55.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.515+0000 7f29e259c700 1 -- 172.21.15.55:0/422879434 >> v1:172.21.15.163:6789/0 conn(0x7f29dc0b4e70 legacy=0x7f29dc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:55.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.515+0000 7f29e259c700 1 -- 172.21.15.55:0/422879434 >> v1:172.21.15.55:6789/0 conn(0x7f29dc0b16d0 legacy=0x7f29dc0b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:55.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.515+0000 7f29e259c700 1 -- 172.21.15.55:0/422879434 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc0b6200 con 0x7f29dc0b1b40 2023-10-06T03:32:55.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.515+0000 7f29e259c700 1 --2- 172.21.15.55:0/422879434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29dc0b1b40 0x7f29dc0b3ac0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009e60 tx=0x7f29d4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be731db3757c40ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:55.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.515+0000 7f29e0d99700 1 -- 172.21.15.55:0/422879434 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d400e070 con 0x7f29dc0b1b40 2023-10-06T03:32:55.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.516+0000 7f29e0d99700 1 -- 172.21.15.55:0/422879434 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d400b7a0 con 0x7f29dc0b1b40 2023-10-06T03:32:55.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.516+0000 7f29e0d99700 1 -- 172.21.15.55:0/422879434 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc001470 con 0x7f29dc0b1b40 2023-10-06T03:32:55.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.516+0000 7f29e2d9d700 1 -- 172.21.15.55:0/422879434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29dc0b1b40 msgr2=0x7f29dc0b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:55.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.516+0000 7f29e2d9d700 1 --2- 172.21.15.55:0/422879434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29dc0b1b40 0x7f29dc0b3ac0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009e60 tx=0x7f29d4009780 comp rx=0 tx=0).stop 2023-10-06T03:32:55.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.516+0000 7f29e2d9d700 1 -- 172.21.15.55:0/422879434 shutdown_connections 2023-10-06T03:32:55.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.516+0000 7f29e2d9d700 1 --2- 172.21.15.55:0/422879434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29dc0b1b40 0x7f29dc0b3ac0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:55.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.516+0000 7f29e2d9d700 1 -- 172.21.15.55:0/422879434 >> 172.21.15.55:0/422879434 conn(0x7f29dc0d10c0 msgr2=0x7f29dc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:55.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.516+0000 7f29e2d9d700 1 -- 172.21.15.55:0/422879434 shutdown_connections 2023-10-06T03:32:55.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.516+0000 7f29e2d9d700 1 -- 172.21.15.55:0/422879434 wait complete. 2023-10-06T03:32:55.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.518+0000 7f29e2d9d700 1 Processor -- start 2023-10-06T03:32:55.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.519+0000 7f29e2d9d700 1 -- start start 2023-10-06T03:32:55.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.520+0000 7f29e2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29dc0b16d0 0x7f29dc169c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:55.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.520+0000 7f29e2d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29dc0b1b40 0x7f29dc15b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:55.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.520+0000 7f29e2d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29dc0b4e70 0x7f29dc1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:55.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.520+0000 7f29e2d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc16a170 con 0x7f29dc0b1b40 2023-10-06T03:32:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.520+0000 7f29e2d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc15bd10 con 0x7f29dc0b16d0 2023-10-06T03:32:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.520+0000 7f29e2d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc0c5770 con 0x7f29dc0b4e70 2023-10-06T03:32:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.520+0000 7f29e159a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29dc0b1b40 0x7f29dc15b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.520+0000 7f29e159a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29dc0b1b40 0x7f29dc15b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50772/0 (socket says 172.21.15.55:50772) 2023-10-06T03:32:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.520+0000 7f29e159a700 1 -- 172.21.15.55:0/2188080733 learned_addr learned my addr 172.21.15.55:0/2188080733 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:32:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.520+0000 7f29e1d9b700 1 --2- 172.21.15.55:0/2188080733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29dc0b16d0 0x7f29dc169c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.520+0000 7f29e259c700 1 --2- 172.21.15.55:0/2188080733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29dc0b4e70 0x7f29dc1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.521+0000 7f29e1d9b700 1 -- 172.21.15.55:0/2188080733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29dc0b4e70 msgr2=0x7f29dc1561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.521+0000 7f29e1d9b700 1 --2- 172.21.15.55:0/2188080733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29dc0b4e70 0x7f29dc1561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.521+0000 7f29e1d9b700 1 -- 172.21.15.55:0/2188080733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29dc0b1b40 msgr2=0x7f29dc15b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.521+0000 7f29e1d9b700 1 --2- 172.21.15.55:0/2188080733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29dc0b1b40 0x7f29dc15b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.521+0000 7f29e1d9b700 1 -- 172.21.15.55:0/2188080733 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d4009a20 con 0x7f29dc0b16d0 2023-10-06T03:32:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.521+0000 7f29e1d9b700 1 --2- 172.21.15.55:0/2188080733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29dc0b16d0 0x7f29dc169c30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c4b0 tx=0x7f29d8007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.521+0000 7f29d1ffb700 1 -- 172.21.15.55:0/2188080733 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d800ce60 con 0x7f29dc0b16d0 2023-10-06T03:32:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.521+0000 7f29d1ffb700 1 -- 172.21.15.55:0/2188080733 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d8011590 con 0x7f29dc0b16d0 2023-10-06T03:32:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.521+0000 7f29d1ffb700 1 -- 172.21.15.55:0/2188080733 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8011950 con 0x7f29dc0b16d0 2023-10-06T03:32:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.521+0000 7f29e2d9d700 1 -- 172.21.15.55:0/2188080733 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29dc0c5a50 con 0x7f29dc0b16d0 2023-10-06T03:32:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.522+0000 7f29e2d9d700 1 -- 172.21.15.55:0/2188080733 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc0c6010 con 0x7f29dc0b16d0 2023-10-06T03:32:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.522+0000 7f29d1ffb700 1 -- 172.21.15.55:0/2188080733 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f29d801c460 con 0x7f29dc0b16d0 2023-10-06T03:32:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.522+0000 7f29d1ffb700 1 -- 172.21.15.55:0/2188080733 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f29d801ca60 con 0x7f29dc0b16d0 2023-10-06T03:32:55.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.523+0000 7f29e2d9d700 1 -- 172.21.15.55:0/2188080733 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29dc088990 con 0x7f29dc0b16d0 2023-10-06T03:32:55.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.526+0000 7f29d1ffb700 1 -- 172.21.15.55:0/2188080733 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f29d8026430 con 0x7f29dc0b16d0 2023-10-06T03:32:55.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.615+0000 7f29e2d9d700 1 -- 172.21.15.55:0/2188080733 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f29dc04bb80 con 0x7f29dc0b16d0 2023-10-06T03:32:55.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.623+0000 7f29d1ffb700 1 -- 172.21.15.55:0/2188080733 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f29d80265d0 con 0x7f29dc0b16d0 2023-10-06T03:32:55.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.624+0000 7f29e2d9d700 1 -- 172.21.15.55:0/2188080733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29dc0b16d0 msgr2=0x7f29dc169c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:55.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.624+0000 7f29e2d9d700 1 --2- 172.21.15.55:0/2188080733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29dc0b16d0 0x7f29dc169c30 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c4b0 tx=0x7f29d8007d50 comp rx=0 tx=0).stop 2023-10-06T03:32:55.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.624+0000 7f29e2d9d700 1 -- 172.21.15.55:0/2188080733 shutdown_connections 2023-10-06T03:32:55.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.624+0000 7f29e2d9d700 1 --2- 172.21.15.55:0/2188080733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29dc0b4e70 0x7f29dc1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:55.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.624+0000 7f29e2d9d700 1 --2- 172.21.15.55:0/2188080733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29dc0b16d0 0x7f29dc169c30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:55.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.624+0000 7f29e2d9d700 1 --2- 172.21.15.55:0/2188080733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29dc0b1b40 0x7f29dc15b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:55.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.624+0000 7f29e2d9d700 1 -- 172.21.15.55:0/2188080733 >> 172.21.15.55:0/2188080733 conn(0x7f29dc0d10c0 msgr2=0x7f29dc1590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:55.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.624+0000 7f29e2d9d700 1 -- 172.21.15.55:0/2188080733 shutdown_connections 2023-10-06T03:32:55.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:55.624+0000 7f29e2d9d700 1 -- 172.21.15.55:0/2188080733 wait complete. 2023-10-06T03:32:55.627 INFO:teuthology.orchestra.run.smithi055.stderr:adjusted tunables profile to default 2023-10-06T03:32:55.643 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T03:32:55.643 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:55.643 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T03:32:55.708 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:55.708 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T03:32:55.817 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:55.859 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T03:32:55.923 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2023-10-06T03:32:55.923 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T03:32:56.029 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:56.029 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T03:32:56.094 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:56.095 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T03:32:56.195 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:56.195 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T03:32:56.299 DEBUG:teuthology.orchestra.run.smithi163:> set -ex 2023-10-06T03:32:56.299 DEBUG:teuthology.orchestra.run.smithi163:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T03:32:56.410 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new a60fada0-fedf-4541-8e31-aaa108953efe 0 2023-10-06T03:32:56.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.634+0000 7f77c2dff700 1 Processor -- start 2023-10-06T03:32:56.635 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.634+0000 7f77c2dff700 1 -- start start 2023-10-06T03:32:56.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.635+0000 7f77c2dff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f77bc0caa90 0x7f77bc0c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:56.636 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.635+0000 7f77c2dff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77bc15d7e0 con 0x7f77bc06bcd0 2023-10-06T03:32:56.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.635+0000 7f77c2dff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77bc15dac0 con 0x7f77bc06c0a0 2023-10-06T03:32:56.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.635+0000 7f77c2dff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77bc0caef0 con 0x7f77bc0caa90 2023-10-06T03:32:56.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.635+0000 7f77c139c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f77bc0caa90 0x7f77bc0c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:56.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.636+0000 7f77c139c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f77bc0caa90 0x7f77bc0c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39476/0 (socket says 172.21.15.163:39476) 2023-10-06T03:32:56.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.636+0000 7f77c139c700 1 -- 172.21.15.163:0/4032991981 learned_addr learned my addr 172.21.15.163:0/4032991981 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-06T03:32:56.637 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.636+0000 7f77bbfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f77bc06c0a0 0x7f77bc15df50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:40046/0 (socket says 172.21.15.163:40046) 2023-10-06T03:32:56.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.636+0000 7f77c139c700 1 -- 172.21.15.163:0/4032991981 >> v1:172.21.15.163:6789/0 conn(0x7f77bc06c0a0 legacy=0x7f77bc15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:56.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.636+0000 7f77c139c700 1 -- 172.21.15.163:0/4032991981 >> v1:172.21.15.55:6789/0 conn(0x7f77bc06bcd0 legacy=0x7f77bc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:56.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.636+0000 7f77c139c700 1 -- 172.21.15.163:0/4032991981 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc15aa30 con 0x7f77bc0caa90 2023-10-06T03:32:56.638 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.636+0000 7f77c139c700 1 --2- 172.21.15.163:0/4032991981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77bc0caa90 0x7f77bc0c8c40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009f20 tx=0x7f77b4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a56dd6d306c7dd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:56.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.636+0000 7f77bb7fe700 1 -- 172.21.15.163:0/4032991981 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b400e070 con 0x7f77bc0caa90 2023-10-06T03:32:56.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.637+0000 7f77bb7fe700 1 -- 172.21.15.163:0/4032991981 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b400ba80 con 0x7f77bc0caa90 2023-10-06T03:32:56.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.637+0000 7f77bb7fe700 1 -- 172.21.15.163:0/4032991981 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40092e0 con 0x7f77bc0caa90 2023-10-06T03:32:56.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.637+0000 7f77c2dff700 1 -- 172.21.15.163:0/4032991981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77bc0caa90 msgr2=0x7f77bc0c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:56.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.637+0000 7f77c2dff700 1 --2- 172.21.15.163:0/4032991981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77bc0caa90 0x7f77bc0c8c40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009f20 tx=0x7f77b4009840 comp rx=0 tx=0).stop 2023-10-06T03:32:56.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.637+0000 7f77c2dff700 1 -- 172.21.15.163:0/4032991981 shutdown_connections 2023-10-06T03:32:56.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.637+0000 7f77c2dff700 1 --2- 172.21.15.163:0/4032991981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77bc0caa90 0x7f77bc0c8c40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:56.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.637+0000 7f77c2dff700 1 -- 172.21.15.163:0/4032991981 >> 172.21.15.163:0/4032991981 conn(0x7f77bc0ce600 msgr2=0x7f77bc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:56.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.637+0000 7f77c2dff700 1 -- 172.21.15.163:0/4032991981 shutdown_connections 2023-10-06T03:32:56.639 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.637+0000 7f77c2dff700 1 -- 172.21.15.163:0/4032991981 wait complete. 2023-10-06T03:32:56.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.639+0000 7f77c2dff700 1 Processor -- start 2023-10-06T03:32:56.640 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.639+0000 7f77c2dff700 1 -- start start 2023-10-06T03:32:56.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c2dff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77bc06bcd0 0x7f77bc159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:56.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c2dff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77bc06c0a0 0x7f77bc15f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:56.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c2dff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77bc0caa90 0x7f77bc146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:56.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c2dff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc158ee0 con 0x7f77bc06bcd0 2023-10-06T03:32:56.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c2dff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc1582d0 con 0x7f77bc06c0a0 2023-10-06T03:32:56.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c2dff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc1585b0 con 0x7f77bc0caa90 2023-10-06T03:32:56.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77bbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77bc06c0a0 0x7f77bc15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:56.642 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c139c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77bc0caa90 0x7f77bc146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c139c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77bc0caa90 0x7f77bc146110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:35968/0 (socket says 172.21.15.163:35968) 2023-10-06T03:32:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c139c700 1 -- 172.21.15.163:0/410317641 learned_addr learned my addr 172.21.15.163:0/410317641 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-06T03:32:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c0b9b700 1 --2- 172.21.15.163:0/410317641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77bc06bcd0 0x7f77bc159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c139c700 1 -- 172.21.15.163:0/410317641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77bc06c0a0 msgr2=0x7f77bc15f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c139c700 1 --2- 172.21.15.163:0/410317641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77bc06c0a0 0x7f77bc15f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c139c700 1 -- 172.21.15.163:0/410317641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77bc06bcd0 msgr2=0x7f77bc159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c139c700 1 --2- 172.21.15.163:0/410317641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77bc06bcd0 0x7f77bc159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77c139c700 1 -- 172.21.15.163:0/410317641 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b4009ae0 con 0x7f77bc0caa90 2023-10-06T03:32:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.640+0000 7f77bbfff700 1 --2- 172.21.15.163:0/410317641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77bc06c0a0 0x7f77bc15f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:32:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.641+0000 7f77c139c700 1 --2- 172.21.15.163:0/410317641 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77bc0caa90 0x7f77bc146110 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f77b400b6c0 tx=0x7f77b4018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.641+0000 7f77b8ff9700 1 -- 172.21.15.163:0/410317641 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40070a0 con 0x7f77bc0caa90 2023-10-06T03:32:56.643 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.641+0000 7f77b8ff9700 1 -- 172.21.15.163:0/410317641 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b4007d60 con 0x7f77bc0caa90 2023-10-06T03:32:56.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.641+0000 7f77c2dff700 1 -- 172.21.15.163:0/410317641 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77bc158890 con 0x7f77bc0caa90 2023-10-06T03:32:56.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.641+0000 7f77b8ff9700 1 -- 172.21.15.163:0/410317641 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b401c640 con 0x7f77bc0caa90 2023-10-06T03:32:56.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.641+0000 7f77c2dff700 1 -- 172.21.15.163:0/410317641 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc15f570 con 0x7f77bc0caa90 2023-10-06T03:32:56.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.642+0000 7f77b8ff9700 1 -- 172.21.15.163:0/410317641 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f77b4003050 con 0x7f77bc0caa90 2023-10-06T03:32:56.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.642+0000 7f77b8ff9700 1 -- 172.21.15.163:0/410317641 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f77b401cd40 con 0x7f77bc0caa90 2023-10-06T03:32:56.644 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.642+0000 7f77c2dff700 1 -- 172.21.15.163:0/410317641 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77bc159050 con 0x7f77bc0caa90 2023-10-06T03:32:56.645 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.645+0000 7f77b8ff9700 1 -- 172.21.15.163:0/410317641 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f77b4029020 con 0x7f77bc0caa90 2023-10-06T03:32:56.731 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.730+0000 7f77c2dff700 1 -- 172.21.15.163:0/410317641 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a60fada0-fedf-4541-8e31-aaa108953efe", "id": 0} v 0) v1 -- 0x7f77bc04c3c0 con 0x7f77bc0caa90 2023-10-06T03:32:56.736 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.735+0000 7f77b8ff9700 1 -- 172.21.15.163:0/410317641 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a60fada0-fedf-4541-8e31-aaa108953efe", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f77b4024400 con 0x7f77bc0caa90 2023-10-06T03:32:56.736 INFO:teuthology.orchestra.run.smithi163.stdout:0 2023-10-06T03:32:56.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.737+0000 7f77c2dff700 1 -- 172.21.15.163:0/410317641 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77bc0caa90 msgr2=0x7f77bc146110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:56.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.737+0000 7f77c2dff700 1 --2- 172.21.15.163:0/410317641 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77bc0caa90 0x7f77bc146110 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f77b400b6c0 tx=0x7f77b4018000 comp rx=0 tx=0).stop 2023-10-06T03:32:56.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.737+0000 7f77c2dff700 1 -- 172.21.15.163:0/410317641 shutdown_connections 2023-10-06T03:32:56.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.737+0000 7f77c2dff700 1 --2- 172.21.15.163:0/410317641 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f77bc0caa90 0x7f77bc146110 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:56.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.737+0000 7f77c2dff700 1 --2- 172.21.15.163:0/410317641 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77bc06bcd0 0x7f77bc159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:56.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.737+0000 7f77c2dff700 1 --2- 172.21.15.163:0/410317641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77bc06c0a0 0x7f77bc15f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:56.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.737+0000 7f77c2dff700 1 -- 172.21.15.163:0/410317641 >> 172.21.15.163:0/410317641 conn(0x7f77bc0ce600 msgr2=0x7f77bc156360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:56.738 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.737+0000 7f77c2dff700 1 -- 172.21.15.163:0/410317641 shutdown_connections 2023-10-06T03:32:56.739 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.737+0000 7f77c2dff700 1 -- 172.21.15.163:0/410317641 wait complete. 2023-10-06T03:32:56.754 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 880c7d1c-2d15-48fb-9065-15bc117b6ebd 1 2023-10-06T03:32:56.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.932+0000 7f1f8279d700 1 Processor -- start 2023-10-06T03:32:56.933 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.932+0000 7f1f8279d700 1 -- start start 2023-10-06T03:32:56.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.933+0000 7f1f8279d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1f7c0b4e70 0x7f1f7c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:56.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.933+0000 7f1f8279d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1f7c0b1b40 0x7f1f7c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:56.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.933+0000 7f1f8279d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c0cfb00 con 0x7f1f7c0b16d0 2023-10-06T03:32:56.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.933+0000 7f1f8279d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7c0b5c40 con 0x7f1f7c0b4e70 2023-10-06T03:32:56.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.933+0000 7f1f8279d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7c0b5f20 con 0x7f1f7c0b1b40 2023-10-06T03:32:56.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.933+0000 7f1f81f9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1f7c0b1b40 0x7f1f7c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:56.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.933+0000 7f1f80f9a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1f7c0b16d0 0x7f1f7c15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:40050/0 (socket says 172.21.15.163:40050) 2023-10-06T03:32:56.935 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.933+0000 7f1f80f9a700 1 -- 172.21.15.163:0/3865732044 learned_addr learned my addr 172.21.15.163:0/3865732044 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-06T03:32:56.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.934+0000 7f1f8179b700 1 --2- 172.21.15.163:0/3865732044 >> v2:172.21.15.55:3300/0 conn(0x7f1f7c0b4e70 0x7f1f7c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:56.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.934+0000 7f1f73fff700 1 -- 172.21.15.163:0/3865732044 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129041004 0 0) 0x7f1f7c0cfb00 con 0x7f1f7c0b16d0 2023-10-06T03:32:56.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.934+0000 7f1f73fff700 1 -- 172.21.15.163:0/3865732044 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f68003570 con 0x7f1f7c0b16d0 2023-10-06T03:32:56.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.934+0000 7f1f81f9c700 1 -- 172.21.15.163:0/3865732044 >> v2:172.21.15.55:3300/0 conn(0x7f1f7c0b4e70 msgr2=0x7f1f7c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:32:56.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.934+0000 7f1f81f9c700 1 --2- 172.21.15.163:0/3865732044 >> v2:172.21.15.55:3300/0 conn(0x7f1f7c0b4e70 0x7f1f7c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:56.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.934+0000 7f1f81f9c700 1 -- 172.21.15.163:0/3865732044 >> v1:172.21.15.163:6789/0 conn(0x7f1f7c0b16d0 legacy=0x7f1f7c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:56.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.934+0000 7f1f81f9c700 1 -- 172.21.15.163:0/3865732044 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c0b6200 con 0x7f1f7c0b1b40 2023-10-06T03:32:56.936 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.934+0000 7f1f81f9c700 1 --2- 172.21.15.163:0/3865732044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f7c0b1b40 0x7f1f7c15d950 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c071380 tx=0x7f1f74009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dc99bc6e6ff9562 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:56.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.934+0000 7f1f73fff700 1 -- 172.21.15.163:0/3865732044 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7400e070 con 0x7f1f7c0b1b40 2023-10-06T03:32:56.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.935+0000 7f1f73fff700 1 -- 172.21.15.163:0/3865732044 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7400bab0 con 0x7f1f7c0b1b40 2023-10-06T03:32:56.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.935+0000 7f1f73fff700 1 -- 172.21.15.163:0/3865732044 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7400be70 con 0x7f1f7c0b1b40 2023-10-06T03:32:56.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.935+0000 7f1f8279d700 1 -- 172.21.15.163:0/3865732044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f7c0b1b40 msgr2=0x7f1f7c15d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:56.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.935+0000 7f1f8279d700 1 --2- 172.21.15.163:0/3865732044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f7c0b1b40 0x7f1f7c15d950 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c071380 tx=0x7f1f74009780 comp rx=0 tx=0).stop 2023-10-06T03:32:56.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.935+0000 7f1f8279d700 1 -- 172.21.15.163:0/3865732044 shutdown_connections 2023-10-06T03:32:56.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.935+0000 7f1f8279d700 1 --2- 172.21.15.163:0/3865732044 >> v2:172.21.15.55:3300/0 conn(0x7f1f7c0b4e70 0x7f1f7c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:56.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.935+0000 7f1f8279d700 1 --2- 172.21.15.163:0/3865732044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f7c0b1b40 0x7f1f7c15d950 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:56.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.935+0000 7f1f8279d700 1 -- 172.21.15.163:0/3865732044 >> 172.21.15.163:0/3865732044 conn(0x7f1f7c0d10c0 msgr2=0x7f1f7c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:56.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.935+0000 7f1f8279d700 1 -- 172.21.15.163:0/3865732044 shutdown_connections 2023-10-06T03:32:56.938 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.935+0000 7f1f8279d700 1 -- 172.21.15.163:0/3865732044 wait complete. 2023-10-06T03:32:56.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.937+0000 7f1f8279d700 1 Processor -- start 2023-10-06T03:32:56.939 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.937+0000 7f1f8279d700 1 -- start start 2023-10-06T03:32:56.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f8279d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f7c0b16d0 0x7f1f7c15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:56.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f8279d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f7c0b1b40 0x7f1f7c0c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:56.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f8279d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f7c0b4e70 0x7f1f7c0cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:56.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f8279d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c15e630 con 0x7f1f7c0b1b40 2023-10-06T03:32:56.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f8279d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c15da20 con 0x7f1f7c0b4e70 2023-10-06T03:32:56.941 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f8279d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c15dd00 con 0x7f1f7c0b16d0 2023-10-06T03:32:56.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f8179b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f7c0b16d0 0x7f1f7c15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:56.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f8179b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f7c0b16d0 0x7f1f7c15eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:35982/0 (socket says 172.21.15.163:35982) 2023-10-06T03:32:56.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f8179b700 1 -- 172.21.15.163:0/627504462 learned_addr learned my addr 172.21.15.163:0/627504462 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-06T03:32:56.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f80f9a700 1 --2- 172.21.15.163:0/627504462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f7c0b1b40 0x7f1f7c0c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:56.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f8179b700 1 -- 172.21.15.163:0/627504462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f7c0b4e70 msgr2=0x7f1f7c0cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:56.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f81f9c700 1 --2- 172.21.15.163:0/627504462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f7c0b4e70 0x7f1f7c0cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:56.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f8179b700 1 --2- 172.21.15.163:0/627504462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f7c0b4e70 0x7f1f7c0cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:56.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f8179b700 1 -- 172.21.15.163:0/627504462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f7c0b1b40 msgr2=0x7f1f7c0c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:56.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f8179b700 1 --2- 172.21.15.163:0/627504462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f7c0b1b40 0x7f1f7c0c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:56.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f8179b700 1 -- 172.21.15.163:0/627504462 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f74009a20 con 0x7f1f7c0b16d0 2023-10-06T03:32:56.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.938+0000 7f1f81f9c700 1 --2- 172.21.15.163:0/627504462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f7c0b4e70 0x7f1f7c0cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:32:56.942 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.939+0000 7f1f8179b700 1 --2- 172.21.15.163:0/627504462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f7c0b16d0 0x7f1f7c15eac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800e930 tx=0x7f1f780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:56.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.939+0000 7f1f717fa700 1 -- 172.21.15.163:0/627504462 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78013680 con 0x7f1f7c0b16d0 2023-10-06T03:32:56.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.939+0000 7f1f717fa700 1 -- 172.21.15.163:0/627504462 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f780137e0 con 0x7f1f7c0b16d0 2023-10-06T03:32:56.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.939+0000 7f1f8279d700 1 -- 172.21.15.163:0/627504462 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c15dfe0 con 0x7f1f7c0b16d0 2023-10-06T03:32:56.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.939+0000 7f1f717fa700 1 -- 172.21.15.163:0/627504462 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78013b80 con 0x7f1f7c0b16d0 2023-10-06T03:32:56.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.939+0000 7f1f8279d700 1 -- 172.21.15.163:0/627504462 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c15f000 con 0x7f1f7c0b16d0 2023-10-06T03:32:56.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.939+0000 7f1f717fa700 1 -- 172.21.15.163:0/627504462 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f1f78005c80 con 0x7f1f7c0b16d0 2023-10-06T03:32:56.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.939+0000 7f1f717fa700 1 -- 172.21.15.163:0/627504462 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f1f7800c460 con 0x7f1f7c0b16d0 2023-10-06T03:32:56.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.940+0000 7f1f8279d700 1 -- 172.21.15.163:0/627504462 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7c088990 con 0x7f1f7c0b16d0 2023-10-06T03:32:56.943 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:56.942+0000 7f1f717fa700 1 -- 172.21.15.163:0/627504462 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1f78021060 con 0x7f1f7c0b16d0 2023-10-06T03:32:57.029 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.028+0000 7f1f8279d700 1 -- 172.21.15.163:0/627504462 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd new", "uuid": "880c7d1c-2d15-48fb-9065-15bc117b6ebd", "id": 1} v 0) v1 -- 0x7f1f7c04bb80 con 0x7f1f7c0b16d0 2023-10-06T03:32:57.037 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.036+0000 7f1f717fa700 1 -- 172.21.15.163:0/627504462 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "880c7d1c-2d15-48fb-9065-15bc117b6ebd", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1f78017070 con 0x7f1f7c0b16d0 2023-10-06T03:32:57.037 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-06T03:32:57.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.037+0000 7f1f8279d700 1 -- 172.21.15.163:0/627504462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f7c0b16d0 msgr2=0x7f1f7c15eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.037+0000 7f1f8279d700 1 --2- 172.21.15.163:0/627504462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f7c0b16d0 0x7f1f7c15eac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800e930 tx=0x7f1f780091a0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.037+0000 7f1f8279d700 1 -- 172.21.15.163:0/627504462 shutdown_connections 2023-10-06T03:32:57.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.037+0000 7f1f8279d700 1 --2- 172.21.15.163:0/627504462 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f7c0b4e70 0x7f1f7c0cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.037+0000 7f1f8279d700 1 --2- 172.21.15.163:0/627504462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f7c0b16d0 0x7f1f7c15eac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.037+0000 7f1f8279d700 1 --2- 172.21.15.163:0/627504462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f7c0b1b40 0x7f1f7c0c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.037+0000 7f1f8279d700 1 -- 172.21.15.163:0/627504462 >> 172.21.15.163:0/627504462 conn(0x7f1f7c0d10c0 msgr2=0x7f1f7c159050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:57.039 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.037+0000 7f1f8279d700 1 -- 172.21.15.163:0/627504462 shutdown_connections 2023-10-06T03:32:57.040 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.037+0000 7f1f8279d700 1 -- 172.21.15.163:0/627504462 wait complete. 2023-10-06T03:32:57.056 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new e36c0a33-7bf6-4b80-b9fc-e9512ae922cd 2 2023-10-06T03:32:57.094 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:32:57.093+0000 7fead2280180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:32:57.106 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:32:57.105+0000 7f07ce0ee180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:32:57.234 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.233+0000 7fa11619d700 1 Processor -- start 2023-10-06T03:32:57.235 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.233+0000 7fa11619d700 1 -- start start 2023-10-06T03:32:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.235+0000 7fa11619d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1100b4e70 0x7fa1100ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.235+0000 7fa11619d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa1100b16d0 0x7fa1100b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.235+0000 7fa11619d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa11015d240 0x7fa11015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.235+0000 7fa11619d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1100b2080 con 0x7fa1100b16d0 2023-10-06T03:32:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.235+0000 7fa11619d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1100cfb00 con 0x7fa11015d240 2023-10-06T03:32:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.235+0000 7fa11619d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1100b5c40 con 0x7fa1100b4e70 2023-10-06T03:32:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.235+0000 7fa11519b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1100b4e70 0x7fa1100ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.237 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.235+0000 7fa11599c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa11015d240 0x7fa11015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.235+0000 7fa11599c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa11015d240 0x7fa11015d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39526/0 (socket says 172.21.15.163:39526) 2023-10-06T03:32:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.235+0000 7fa11519b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1100b4e70 0x7fa1100ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:35992/0 (socket says 172.21.15.163:35992) 2023-10-06T03:32:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.235+0000 7fa11599c700 1 -- 172.21.15.163:0/2888636603 learned_addr learned my addr 172.21.15.163:0/2888636603 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-06T03:32:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.235+0000 7fa11499a700 1 --2- 172.21.15.163:0/2888636603 >> v2:172.21.15.55:3300/0 conn(0x7fa1100b16d0 0x7fa1100b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.236+0000 7fa11519b700 1 -- 172.21.15.163:0/2888636603 >> v2:172.21.15.163:3300/0 conn(0x7fa11015d240 msgr2=0x7fa11015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:32:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.236+0000 7fa11519b700 1 --2- 172.21.15.163:0/2888636603 >> v2:172.21.15.163:3300/0 conn(0x7fa11015d240 0x7fa11015d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.236+0000 7fa11519b700 1 -- 172.21.15.163:0/2888636603 >> v2:172.21.15.55:3300/0 conn(0x7fa1100b16d0 msgr2=0x7fa1100b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:32:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.236+0000 7fa11519b700 1 --2- 172.21.15.163:0/2888636603 >> v2:172.21.15.55:3300/0 conn(0x7fa1100b16d0 0x7fa1100b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.236+0000 7fa11519b700 1 -- 172.21.15.163:0/2888636603 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1100b5f20 con 0x7fa1100b4e70 2023-10-06T03:32:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.236+0000 7fa11599c700 1 --2- 172.21.15.163:0/2888636603 >> v2:172.21.15.163:3300/0 conn(0x7fa11015d240 0x7fa11015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:32:57.238 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.236+0000 7fa11519b700 1 --2- 172.21.15.163:0/2888636603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1100b4e70 0x7fa1100ccff0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa11015d180 tx=0x7fa10c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=555c51e3ea98b610 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:57.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.236+0000 7fa107fff700 1 -- 172.21.15.163:0/2888636603 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c011630 con 0x7fa1100b4e70 2023-10-06T03:32:57.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.236+0000 7fa107fff700 1 -- 172.21.15.163:0/2888636603 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c0117b0 con 0x7fa1100b4e70 2023-10-06T03:32:57.239 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.236+0000 7fa107fff700 1 -- 172.21.15.163:0/2888636603 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c011b70 con 0x7fa1100b4e70 2023-10-06T03:32:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.237+0000 7fa11619d700 1 -- 172.21.15.163:0/2888636603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1100b4e70 msgr2=0x7fa1100ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.237+0000 7fa11619d700 1 --2- 172.21.15.163:0/2888636603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1100b4e70 0x7fa1100ccff0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa11015d180 tx=0x7fa10c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.237+0000 7fa11619d700 1 -- 172.21.15.163:0/2888636603 shutdown_connections 2023-10-06T03:32:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.237+0000 7fa11619d700 1 --2- 172.21.15.163:0/2888636603 >> v2:172.21.15.163:3300/0 conn(0x7fa11015d240 0x7fa11015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.237+0000 7fa11619d700 1 --2- 172.21.15.163:0/2888636603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1100b4e70 0x7fa1100ccff0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.242 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.237+0000 7fa11619d700 1 --2- 172.21.15.163:0/2888636603 >> v2:172.21.15.55:3300/0 conn(0x7fa1100b16d0 0x7fa1100b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.237+0000 7fa11619d700 1 -- 172.21.15.163:0/2888636603 >> 172.21.15.163:0/2888636603 conn(0x7fa1100d10c0 msgr2=0x7fa1100affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.237+0000 7fa11619d700 1 -- 172.21.15.163:0/2888636603 shutdown_connections 2023-10-06T03:32:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.237+0000 7fa11619d700 1 -- 172.21.15.163:0/2888636603 wait complete. 2023-10-06T03:32:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.238+0000 7fa11619d700 1 Processor -- start 2023-10-06T03:32:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.238+0000 7fa11619d700 1 -- start start 2023-10-06T03:32:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.239+0000 7fa11619d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1100b16d0 0x7fa1101456e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.239+0000 7fa11619d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1100b4e70 0x7fa1100c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.239+0000 7fa11619d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa11015d240 0x7fa11015c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.239+0000 7fa11619d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa110169d20 con 0x7fa11015d240 2023-10-06T03:32:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.239+0000 7fa11619d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11016a580 con 0x7fa1100b4e70 2023-10-06T03:32:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.239+0000 7fa11619d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa110145c20 con 0x7fa1100b16d0 2023-10-06T03:32:57.243 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.240+0000 7fa11499a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1100b4e70 0x7fa1100c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.240+0000 7fa11519b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1100b16d0 0x7fa1101456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.240+0000 7fa11519b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1100b16d0 0x7fa1101456e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:35996/0 (socket says 172.21.15.163:35996) 2023-10-06T03:32:57.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.240+0000 7fa11519b700 1 -- 172.21.15.163:0/3822738764 learned_addr learned my addr 172.21.15.163:0/3822738764 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-06T03:32:57.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.240+0000 7fa11599c700 1 --2- 172.21.15.163:0/3822738764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa11015d240 0x7fa11015c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.240+0000 7fa11499a700 1 -- 172.21.15.163:0/3822738764 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1100b16d0 msgr2=0x7fa1101456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.240+0000 7fa11499a700 1 --2- 172.21.15.163:0/3822738764 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1100b16d0 0x7fa1101456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.240+0000 7fa11499a700 1 -- 172.21.15.163:0/3822738764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa11015d240 msgr2=0x7fa11015c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.240+0000 7fa11499a700 1 --2- 172.21.15.163:0/3822738764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa11015d240 0x7fa11015c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.240+0000 7fa11499a700 1 -- 172.21.15.163:0/3822738764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa108009ae0 con 0x7fa1100b4e70 2023-10-06T03:32:57.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.240+0000 7fa11519b700 1 --2- 172.21.15.163:0/3822738764 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1100b16d0 0x7fa1101456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:32:57.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.240+0000 7fa11499a700 1 --2- 172.21.15.163:0/3822738764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1100b4e70 0x7fa1100c98d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa10000e900 tx=0x7fa1000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:57.244 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.241+0000 7fa1057fa700 1 -- 172.21.15.163:0/3822738764 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100013680 con 0x7fa1100b4e70 2023-10-06T03:32:57.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.241+0000 7fa1057fa700 1 -- 172.21.15.163:0/3822738764 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1000137e0 con 0x7fa1100b4e70 2023-10-06T03:32:57.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.241+0000 7fa11619d700 1 -- 172.21.15.163:0/3822738764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10c009420 con 0x7fa1100b4e70 2023-10-06T03:32:57.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.241+0000 7fa1057fa700 1 -- 172.21.15.163:0/3822738764 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100013ba0 con 0x7fa1100b4e70 2023-10-06T03:32:57.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.241+0000 7fa11619d700 1 -- 172.21.15.163:0/3822738764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1100ca1d0 con 0x7fa1100b4e70 2023-10-06T03:32:57.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.241+0000 7fa1057fa700 1 -- 172.21.15.163:0/3822738764 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa100005c80 con 0x7fa1100b4e70 2023-10-06T03:32:57.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.241+0000 7fa1057fa700 1 -- 172.21.15.163:0/3822738764 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fa100006020 con 0x7fa1100b4e70 2023-10-06T03:32:57.245 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.242+0000 7fa11619d700 1 -- 172.21.15.163:0/3822738764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11016a6f0 con 0x7fa1100b4e70 2023-10-06T03:32:57.246 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.244+0000 7fa1057fa700 1 -- 172.21.15.163:0/3822738764 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fa100021060 con 0x7fa1100b4e70 2023-10-06T03:32:57.267 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:32:57.266+0000 7fead2280180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:32:57.278 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:32:57.277+0000 7f07ce0ee180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:32:57.330 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.329+0000 7fa11619d700 1 -- 172.21.15.163:0/3822738764 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e36c0a33-7bf6-4b80-b9fc-e9512ae922cd", "id": 2} v 0) v1 -- 0x7fa11004bb80 con 0x7fa1100b4e70 2023-10-06T03:32:57.338 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.337+0000 7fa1057fa700 1 -- 172.21.15.163:0/3822738764 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e36c0a33-7bf6-4b80-b9fc-e9512ae922cd", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa100029020 con 0x7fa1100b4e70 2023-10-06T03:32:57.338 INFO:teuthology.orchestra.run.smithi163.stdout:2 2023-10-06T03:32:57.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.339+0000 7fa11619d700 1 -- 172.21.15.163:0/3822738764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1100b4e70 msgr2=0x7fa1100c98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.339+0000 7fa11619d700 1 --2- 172.21.15.163:0/3822738764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1100b4e70 0x7fa1100c98d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa10000e900 tx=0x7fa1000091a0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.340 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.339+0000 7fa11619d700 1 -- 172.21.15.163:0/3822738764 shutdown_connections 2023-10-06T03:32:57.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.339+0000 7fa11619d700 1 --2- 172.21.15.163:0/3822738764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa11015d240 0x7fa11015c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.339+0000 7fa11619d700 1 --2- 172.21.15.163:0/3822738764 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1100b16d0 0x7fa1101456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.339+0000 7fa11619d700 1 --2- 172.21.15.163:0/3822738764 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1100b4e70 0x7fa1100c98d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.339+0000 7fa11619d700 1 -- 172.21.15.163:0/3822738764 >> 172.21.15.163:0/3822738764 conn(0x7fa1100d10c0 msgr2=0x7fa110069bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:57.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.339+0000 7fa11619d700 1 -- 172.21.15.163:0/3822738764 shutdown_connections 2023-10-06T03:32:57.341 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.339+0000 7fa11619d700 1 -- 172.21.15.163:0/3822738764 wait complete. 2023-10-06T03:32:57.364 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 3311eaac-bc3b-443c-8513-53016de7c85a 3 2023-10-06T03:32:57.374 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:32:57.373+0000 7fead2280180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:32:57.378 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:32:57.376+0000 7f07ce0ee180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:32:57.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.552+0000 7fea5359e700 1 Processor -- start 2023-10-06T03:32:57.553 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.552+0000 7fea5359e700 1 -- start start 2023-10-06T03:32:57.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.553+0000 7fea5359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea540b16d0 0x7fea540b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.555 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.553+0000 7fea5359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea540b2080 con 0x7fea540b4e70 2023-10-06T03:32:57.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.553+0000 7fea5359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea540cfb00 con 0x7fea540b33b0 2023-10-06T03:32:57.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.553+0000 7fea5359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea540b5c40 con 0x7fea540b16d0 2023-10-06T03:32:57.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.553+0000 7fea51d9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea540b16d0 0x7fea540b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.556 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.553+0000 7fea52d9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fea540b33b0 0x7fea540b3830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.163:40066/0 (socket says 172.21.15.163:40066) 2023-10-06T03:32:57.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.553+0000 7fea52d9d700 1 -- 172.21.15.163:0/3456756141 learned_addr learned my addr 172.21.15.163:0/3456756141 (peer_addr_for_me v1:172.21.15.163:0/0) 2023-10-06T03:32:57.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.553+0000 7fea5259c700 1 --1- 172.21.15.163:0/3456756141 >> v1:172.21.15.55:6789/0 conn(0x7fea540b4e70 0x7fea540ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.163:59208/0 (socket says 172.21.15.163:59208) 2023-10-06T03:32:57.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.554+0000 7fea5159a700 1 -- 172.21.15.163:0/3456756141 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009134881 0 0) 0x7fea540cfb00 con 0x7fea540b33b0 2023-10-06T03:32:57.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.554+0000 7fea5159a700 1 -- 172.21.15.163:0/3456756141 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea3c003570 con 0x7fea540b33b0 2023-10-06T03:32:57.557 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.554+0000 7fea5159a700 1 -- 172.21.15.163:0/3456756141 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141028589 0 0) 0x7fea540b2080 con 0x7fea540b4e70 2023-10-06T03:32:57.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.554+0000 7fea51d9b700 1 -- 172.21.15.163:0/3456756141 >> v1:172.21.15.163:6789/0 conn(0x7fea540b33b0 legacy=0x7fea540b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.554+0000 7fea51d9b700 1 -- 172.21.15.163:0/3456756141 >> v1:172.21.15.55:6789/0 conn(0x7fea540b4e70 legacy=0x7fea540ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.554+0000 7fea51d9b700 1 -- 172.21.15.163:0/3456756141 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea540b5f20 con 0x7fea540b16d0 2023-10-06T03:32:57.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.554+0000 7fea51d9b700 1 --2- 172.21.15.163:0/3456756141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea540b16d0 0x7fea540b1b40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea48009910 tx=0x7fea480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc172dfcbb3f96da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:57.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.554+0000 7fea5159a700 1 -- 172.21.15.163:0/3456756141 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4800b680 con 0x7fea540b16d0 2023-10-06T03:32:57.558 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.554+0000 7fea5159a700 1 -- 172.21.15.163:0/3456756141 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4800b7e0 con 0x7fea540b16d0 2023-10-06T03:32:57.559 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.555+0000 7fea5159a700 1 -- 172.21.15.163:0/3456756141 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4800bba0 con 0x7fea540b16d0 2023-10-06T03:32:57.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.555+0000 7fea5359e700 1 -- 172.21.15.163:0/3456756141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea540b16d0 msgr2=0x7fea540b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.555+0000 7fea5359e700 1 --2- 172.21.15.163:0/3456756141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea540b16d0 0x7fea540b1b40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea48009910 tx=0x7fea480091a0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.555+0000 7fea5359e700 1 -- 172.21.15.163:0/3456756141 shutdown_connections 2023-10-06T03:32:57.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.555+0000 7fea5359e700 1 --2- 172.21.15.163:0/3456756141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea540b16d0 0x7fea540b1b40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.562 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.555+0000 7fea5359e700 1 -- 172.21.15.163:0/3456756141 >> 172.21.15.163:0/3456756141 conn(0x7fea540d10c0 msgr2=0x7fea540affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:57.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.555+0000 7fea5359e700 1 -- 172.21.15.163:0/3456756141 shutdown_connections 2023-10-06T03:32:57.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.555+0000 7fea5359e700 1 -- 172.21.15.163:0/3456756141 wait complete. 2023-10-06T03:32:57.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.557+0000 7fea5359e700 1 Processor -- start 2023-10-06T03:32:57.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.557+0000 7fea5359e700 1 -- start start 2023-10-06T03:32:57.563 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea5359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea540b16d0 0x7fea541563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea5359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea540b33b0 0x7fea5415eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea5359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea540b4e70 0x7fea54158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea5359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea540c7960 con 0x7fea540b33b0 2023-10-06T03:32:57.564 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea5359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea540c6d50 con 0x7fea540b4e70 2023-10-06T03:32:57.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea5359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea540c7030 con 0x7fea540b16d0 2023-10-06T03:32:57.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea5259c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea540b16d0 0x7fea541563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea52d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea540b4e70 0x7fea54158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea52d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea540b4e70 0x7fea54158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39538/0 (socket says 172.21.15.163:39538) 2023-10-06T03:32:57.565 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea5259c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea540b16d0 0x7fea541563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:36026/0 (socket says 172.21.15.163:36026) 2023-10-06T03:32:57.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea51d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea540b33b0 0x7fea5415eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea52d9d700 1 -- 172.21.15.163:0/2392106714 learned_addr learned my addr 172.21.15.163:0/2392106714 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-06T03:32:57.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea52d9d700 1 -- 172.21.15.163:0/2392106714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea540b16d0 msgr2=0x7fea541563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea52d9d700 1 --2- 172.21.15.163:0/2392106714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea540b16d0 0x7fea541563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.566 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea52d9d700 1 -- 172.21.15.163:0/2392106714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea540b33b0 msgr2=0x7fea5415eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea52d9d700 1 --2- 172.21.15.163:0/2392106714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea540b33b0 0x7fea5415eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.558+0000 7fea52d9d700 1 -- 172.21.15.163:0/2392106714 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea48009420 con 0x7fea540b4e70 2023-10-06T03:32:57.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.559+0000 7fea52d9d700 1 --2- 172.21.15.163:0/2392106714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea540b4e70 0x7fea54158130 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fea4c001460 tx=0x7fea4c001ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:57.567 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.559+0000 7fea427fc700 1 -- 172.21.15.163:0/2392106714 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c00a890 con 0x7fea540b4e70 2023-10-06T03:32:57.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.559+0000 7fea427fc700 1 -- 172.21.15.163:0/2392106714 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4c00a9f0 con 0x7fea540b4e70 2023-10-06T03:32:57.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.559+0000 7fea427fc700 1 -- 172.21.15.163:0/2392106714 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c00ad70 con 0x7fea540b4e70 2023-10-06T03:32:57.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.559+0000 7fea5359e700 1 -- 172.21.15.163:0/2392106714 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea540c7310 con 0x7fea540b4e70 2023-10-06T03:32:57.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.559+0000 7fea5359e700 1 -- 172.21.15.163:0/2392106714 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5415f000 con 0x7fea540b4e70 2023-10-06T03:32:57.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.559+0000 7fea427fc700 1 -- 172.21.15.163:0/2392106714 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fea4c007300 con 0x7fea540b4e70 2023-10-06T03:32:57.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.559+0000 7fea427fc700 1 -- 172.21.15.163:0/2392106714 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7fea4c012400 con 0x7fea540b4e70 2023-10-06T03:32:57.568 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.560+0000 7fea5359e700 1 -- 172.21.15.163:0/2392106714 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea540c7ad0 con 0x7fea540b4e70 2023-10-06T03:32:57.569 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.562+0000 7fea427fc700 1 -- 172.21.15.163:0/2392106714 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fea4c0075b0 con 0x7fea540b4e70 2023-10-06T03:32:57.607 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:32:57.606+0000 7fead2280180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:32:57.609 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:32:57.607+0000 7f07ce0ee180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:32:57.648 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.647+0000 7fea5359e700 1 -- 172.21.15.163:0/2392106714 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3311eaac-bc3b-443c-8513-53016de7c85a", "id": 3} v 0) v1 -- 0x7fea5404bb80 con 0x7fea540b4e70 2023-10-06T03:32:57.656 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.655+0000 7fea427fc700 1 -- 172.21.15.163:0/2392106714 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3311eaac-bc3b-443c-8513-53016de7c85a", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fea4c0077f0 con 0x7fea540b4e70 2023-10-06T03:32:57.656 INFO:teuthology.orchestra.run.smithi163.stdout:3 2023-10-06T03:32:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.656+0000 7fea5359e700 1 -- 172.21.15.163:0/2392106714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea540b4e70 msgr2=0x7fea54158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.656+0000 7fea5359e700 1 --2- 172.21.15.163:0/2392106714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea540b4e70 0x7fea54158130 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fea4c001460 tx=0x7fea4c001ae0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.656+0000 7fea5359e700 1 -- 172.21.15.163:0/2392106714 shutdown_connections 2023-10-06T03:32:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.656+0000 7fea5359e700 1 --2- 172.21.15.163:0/2392106714 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea540b4e70 0x7fea54158130 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.656+0000 7fea5359e700 1 --2- 172.21.15.163:0/2392106714 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea540b16d0 0x7fea541563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.657 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.656+0000 7fea5359e700 1 --2- 172.21.15.163:0/2392106714 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea540b33b0 0x7fea5415eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.656+0000 7fea5359e700 1 -- 172.21.15.163:0/2392106714 >> 172.21.15.163:0/2392106714 conn(0x7fea540d10c0 msgr2=0x7fea541590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:57.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.656+0000 7fea5359e700 1 -- 172.21.15.163:0/2392106714 shutdown_connections 2023-10-06T03:32:57.658 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.656+0000 7fea5359e700 1 -- 172.21.15.163:0/2392106714 wait complete. 2023-10-06T03:32:57.673 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 1c3239d0-0585-40df-bdb0-2ffc191022d8 4 2023-10-06T03:32:57.831 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:32:57.830+0000 7fead2280180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:32:57.832 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:32:57.830+0000 7f07ce0ee180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:32:57.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.852+0000 7f22be19d700 1 Processor -- start 2023-10-06T03:32:57.853 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.852+0000 7f22be19d700 1 -- start start 2023-10-06T03:32:57.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.853+0000 7f22be19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22b80b4e70 0x7f22b80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.853+0000 7f22be19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22b80b1b40 0x7f22b815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.853+0000 7f22be19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b80cfb00 con 0x7f22b80b16d0 2023-10-06T03:32:57.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.853+0000 7f22be19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b80b5c40 con 0x7f22b80b1b40 2023-10-06T03:32:57.854 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.853+0000 7f22be19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80b5f20 con 0x7f22b80b4e70 2023-10-06T03:32:57.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.853+0000 7f22bd19b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22b80b4e70 0x7f22b80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.856 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.854+0000 7f22bd19b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22b80b4e70 0x7f22b80ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:36040/0 (socket says 172.21.15.163:36040) 2023-10-06T03:32:57.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.854+0000 7f22bd99c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22b80b1b40 0x7f22b815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.854+0000 7f22bd19b700 1 -- 172.21.15.163:0/2610202385 learned_addr learned my addr 172.21.15.163:0/2610202385 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-06T03:32:57.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.854+0000 7f22bd99c700 1 -- 172.21.15.163:0/2610202385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22b80b4e70 msgr2=0x7f22b80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:32:57.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.854+0000 7f22bd99c700 1 --2- 172.21.15.163:0/2610202385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22b80b4e70 0x7f22b80ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.854+0000 7f22bd99c700 1 -- 172.21.15.163:0/2610202385 >> v1:172.21.15.55:6789/0 conn(0x7f22b80b16d0 legacy=0x7f22b815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.854+0000 7f22bd99c700 1 -- 172.21.15.163:0/2610202385 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b80b6200 con 0x7f22b80b1b40 2023-10-06T03:32:57.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.854+0000 7f22bd19b700 1 --2- 172.21.15.163:0/2610202385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22b80b4e70 0x7f22b80ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:32:57.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.854+0000 7f22affff700 1 -- 172.21.15.163:0/2610202385 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914208927 0 0) 0x7f22b80cfb00 con 0x7f22b80b16d0 2023-10-06T03:32:57.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.854+0000 7f22bd99c700 1 --2- 172.21.15.163:0/2610202385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22b80b1b40 0x7f22b815d950 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009e60 tx=0x7f22b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2915d8fd0e5add93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:57.857 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.854+0000 7f22affff700 1 -- 172.21.15.163:0/2610202385 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b000e070 con 0x7f22b80b1b40 2023-10-06T03:32:57.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.855+0000 7f22affff700 1 -- 172.21.15.163:0/2610202385 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b000bb60 con 0x7f22b80b1b40 2023-10-06T03:32:57.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.855+0000 7f22affff700 1 -- 172.21.15.163:0/2610202385 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0012430 con 0x7f22b80b1b40 2023-10-06T03:32:57.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.855+0000 7f22be19d700 1 -- 172.21.15.163:0/2610202385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22b80b1b40 msgr2=0x7f22b815d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.855+0000 7f22be19d700 1 --2- 172.21.15.163:0/2610202385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22b80b1b40 0x7f22b815d950 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009e60 tx=0x7f22b0009780 comp rx=0 tx=0).stop 2023-10-06T03:32:57.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.855+0000 7f22be19d700 1 -- 172.21.15.163:0/2610202385 shutdown_connections 2023-10-06T03:32:57.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.855+0000 7f22be19d700 1 --2- 172.21.15.163:0/2610202385 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22b80b1b40 0x7f22b815d950 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.855+0000 7f22be19d700 1 --2- 172.21.15.163:0/2610202385 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22b80b4e70 0x7f22b80ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.855+0000 7f22be19d700 1 -- 172.21.15.163:0/2610202385 >> 172.21.15.163:0/2610202385 conn(0x7f22b80d10c0 msgr2=0x7f22b80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:57.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.855+0000 7f22be19d700 1 -- 172.21.15.163:0/2610202385 shutdown_connections 2023-10-06T03:32:57.858 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.855+0000 7f22be19d700 1 -- 172.21.15.163:0/2610202385 wait complete. 2023-10-06T03:32:57.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.857+0000 7f22be19d700 1 Processor -- start 2023-10-06T03:32:57.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.857+0000 7f22be19d700 1 -- start start 2023-10-06T03:32:57.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.857+0000 7f22be19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22b80b16d0 0x7f22b80af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22be19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22b80b1b40 0x7f22b80c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22be19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22b80b4e70 0x7f22b80ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:57.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22be19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80af5a0 con 0x7f22b80b16d0 2023-10-06T03:32:57.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22be19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80c9e10 con 0x7f22b80b4e70 2023-10-06T03:32:57.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22be19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80ca0f0 con 0x7f22b80b1b40 2023-10-06T03:32:57.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22bd99c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22b80b4e70 0x7f22b80ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.861 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22bd19b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22b80b16d0 0x7f22b80af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22bc99a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22b80b1b40 0x7f22b80c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:57.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22bd99c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22b80b4e70 0x7f22b80ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39552/0 (socket says 172.21.15.163:39552) 2023-10-06T03:32:57.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22bd99c700 1 -- 172.21.15.163:0/3389882905 learned_addr learned my addr 172.21.15.163:0/3389882905 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-06T03:32:57.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22bd19b700 1 -- 172.21.15.163:0/3389882905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22b80b1b40 msgr2=0x7f22b80c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22bd19b700 1 --2- 172.21.15.163:0/3389882905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22b80b1b40 0x7f22b80c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22bd19b700 1 -- 172.21.15.163:0/3389882905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22b80b4e70 msgr2=0x7f22b80ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22bd19b700 1 --2- 172.21.15.163:0/3389882905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22b80b4e70 0x7f22b80ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.858+0000 7f22bd19b700 1 -- 172.21.15.163:0/3389882905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b40094f0 con 0x7f22b80b16d0 2023-10-06T03:32:57.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.859+0000 7f22bd19b700 1 --2- 172.21.15.163:0/3389882905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22b80b16d0 0x7f22b80af060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f22b4006fd0 tx=0x7f22b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:57.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.859+0000 7f22ad7fa700 1 -- 172.21.15.163:0/3389882905 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b400c6a0 con 0x7f22b80b16d0 2023-10-06T03:32:57.862 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.859+0000 7f22ad7fa700 1 -- 172.21.15.163:0/3389882905 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b400c800 con 0x7f22b80b16d0 2023-10-06T03:32:57.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.859+0000 7f22be19d700 1 -- 172.21.15.163:0/3389882905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b0009a20 con 0x7f22b80b16d0 2023-10-06T03:32:57.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.859+0000 7f22ad7fa700 1 -- 172.21.15.163:0/3389882905 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b400cb80 con 0x7f22b80b16d0 2023-10-06T03:32:57.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.859+0000 7f22be19d700 1 -- 172.21.15.163:0/3389882905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b80c81d0 con 0x7f22b80b16d0 2023-10-06T03:32:57.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.859+0000 7f22ad7fa700 1 -- 172.21.15.163:0/3389882905 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f22b4005c80 con 0x7f22b80b16d0 2023-10-06T03:32:57.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.859+0000 7f22ad7fa700 1 -- 172.21.15.163:0/3389882905 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f22b4006020 con 0x7f22b80b16d0 2023-10-06T03:32:57.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.860+0000 7f22be19d700 1 -- 172.21.15.163:0/3389882905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b80b1510 con 0x7f22b80b16d0 2023-10-06T03:32:57.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.863+0000 7f22ad7fa700 1 -- 172.21.15.163:0/3389882905 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f22b4020060 con 0x7f22b80b16d0 2023-10-06T03:32:57.930 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:32:57.929+0000 7fead2280180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:32:57.931 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:32:57.930+0000 7f07ce0ee180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:32:57.948 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.947+0000 7f22be19d700 1 -- 172.21.15.163:0/3389882905 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1c3239d0-0585-40df-bdb0-2ffc191022d8", "id": 4} v 0) v1 -- 0x7f22b80c83c0 con 0x7f22b80b16d0 2023-10-06T03:32:57.955 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.954+0000 7f22ad7fa700 1 -- 172.21.15.163:0/3389882905 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1c3239d0-0585-40df-bdb0-2ffc191022d8", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f22b4028020 con 0x7f22b80b16d0 2023-10-06T03:32:57.955 INFO:teuthology.orchestra.run.smithi163.stdout:4 2023-10-06T03:32:57.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.955+0000 7f22be19d700 1 -- 172.21.15.163:0/3389882905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22b80b16d0 msgr2=0x7f22b80af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:57.956 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.955+0000 7f22be19d700 1 --2- 172.21.15.163:0/3389882905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22b80b16d0 0x7f22b80af060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f22b4006fd0 tx=0x7f22b4009250 comp rx=0 tx=0).stop 2023-10-06T03:32:57.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.955+0000 7f22be19d700 1 -- 172.21.15.163:0/3389882905 shutdown_connections 2023-10-06T03:32:57.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.955+0000 7f22be19d700 1 --2- 172.21.15.163:0/3389882905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22b80b4e70 0x7f22b80ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.955+0000 7f22be19d700 1 --2- 172.21.15.163:0/3389882905 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22b80b16d0 0x7f22b80af060 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.955+0000 7f22be19d700 1 --2- 172.21.15.163:0/3389882905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22b80b1b40 0x7f22b80c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:57.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.955+0000 7f22be19d700 1 -- 172.21.15.163:0/3389882905 >> 172.21.15.163:0/3389882905 conn(0x7f22b80d10c0 msgr2=0x7f22b80b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:57.957 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.956+0000 7f22be19d700 1 -- 172.21.15.163:0/3389882905 shutdown_connections 2023-10-06T03:32:57.958 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:57.956+0000 7f22be19d700 1 -- 172.21.15.163:0/3389882905 wait complete. 2023-10-06T03:32:57.983 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new f49dfabb-4d9c-46bd-811b-984484ad87a1 5 2023-10-06T03:32:58.099 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:32:58.098+0000 7f07ce0ee180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:32:58.109 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:32:58.108+0000 7fead2280180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:32:58.161 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.160+0000 7f20eb59e700 1 Processor -- start 2023-10-06T03:32:58.162 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.160+0000 7f20eb59e700 1 -- start start 2023-10-06T03:32:58.163 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.162+0000 7f20eb59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f20ec0b4e70 0x7f20ec0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.162+0000 7f20eb59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f20ec0b1b40 0x7f20ec15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.162+0000 7f20eb59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ec0cfb00 con 0x7f20ec0b16d0 2023-10-06T03:32:58.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.162+0000 7f20eb59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ec0b5c40 con 0x7f20ec0b1b40 2023-10-06T03:32:58.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.162+0000 7f20eb59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ec0b5f20 con 0x7f20ec0b4e70 2023-10-06T03:32:58.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.162+0000 7f20ea59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f20ec0b4e70 0x7f20ec0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.162+0000 7f20ea59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f20ec0b4e70 0x7f20ec0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39568/0 (socket says 172.21.15.163:39568) 2023-10-06T03:32:58.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.162+0000 7f20ea59c700 1 -- 172.21.15.163:0/323713337 learned_addr learned my addr 172.21.15.163:0/323713337 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-06T03:32:58.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.162+0000 7f20ead9d700 1 --2- 172.21.15.163:0/323713337 >> v2:172.21.15.55:3300/0 conn(0x7f20ec0b1b40 0x7f20ec15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.164 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.162+0000 7f20ea59c700 1 -- 172.21.15.163:0/323713337 >> v2:172.21.15.55:3300/0 conn(0x7f20ec0b1b40 msgr2=0x7f20ec15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:32:58.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.162+0000 7f20ea59c700 1 --2- 172.21.15.163:0/323713337 >> v2:172.21.15.55:3300/0 conn(0x7f20ec0b1b40 0x7f20ec15d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.162+0000 7f20ea59c700 1 -- 172.21.15.163:0/323713337 >> v1:172.21.15.55:6789/0 conn(0x7f20ec0b16d0 legacy=0x7f20ec15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.162+0000 7f20ea59c700 1 -- 172.21.15.163:0/323713337 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec0b6200 con 0x7f20ec0b4e70 2023-10-06T03:32:58.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.163+0000 7f20ea59c700 1 --2- 172.21.15.163:0/323713337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20ec0b4e70 0x7f20ec0ccff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f20ec064910 tx=0x7f20dc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99bc28db394e8b30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:58.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.163+0000 7f20e959a700 1 -- 172.21.15.163:0/323713337 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc009e70 con 0x7f20ec0b4e70 2023-10-06T03:32:58.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.163+0000 7f20e959a700 1 -- 172.21.15.163:0/323713337 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20dc00b860 con 0x7f20ec0b4e70 2023-10-06T03:32:58.165 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.163+0000 7f20e959a700 1 -- 172.21.15.163:0/323713337 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00bc00 con 0x7f20ec0b4e70 2023-10-06T03:32:58.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.163+0000 7f20eb59e700 1 -- 172.21.15.163:0/323713337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20ec0b4e70 msgr2=0x7f20ec0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.163+0000 7f20eb59e700 1 --2- 172.21.15.163:0/323713337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20ec0b4e70 0x7f20ec0ccff0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f20ec064910 tx=0x7f20dc009250 comp rx=0 tx=0).stop 2023-10-06T03:32:58.166 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.164+0000 7f20eb59e700 1 -- 172.21.15.163:0/323713337 shutdown_connections 2023-10-06T03:32:58.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.164+0000 7f20eb59e700 1 --2- 172.21.15.163:0/323713337 >> v2:172.21.15.55:3300/0 conn(0x7f20ec0b1b40 0x7f20ec15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.164+0000 7f20eb59e700 1 --2- 172.21.15.163:0/323713337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20ec0b4e70 0x7f20ec0ccff0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.164+0000 7f20eb59e700 1 -- 172.21.15.163:0/323713337 >> 172.21.15.163:0/323713337 conn(0x7f20ec0d10c0 msgr2=0x7f20ec0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:58.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.164+0000 7f20eb59e700 1 -- 172.21.15.163:0/323713337 shutdown_connections 2023-10-06T03:32:58.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.164+0000 7f20eb59e700 1 -- 172.21.15.163:0/323713337 wait complete. 2023-10-06T03:32:58.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.165+0000 7f20eb59e700 1 Processor -- start 2023-10-06T03:32:58.167 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.165+0000 7f20eb59e700 1 -- start start 2023-10-06T03:32:58.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.166+0000 7f20eb59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20ec0b16d0 0x7f20ec1585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.169 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.166+0000 7f20eb59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20ec0b1b40 0x7f20ec15c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.166+0000 7f20eb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20ec0b4e70 0x7f20ec0cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.166+0000 7f20eb59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec15ac30 con 0x7f20ec0b4e70 2023-10-06T03:32:58.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.166+0000 7f20eb59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec15c3c0 con 0x7f20ec0b1b40 2023-10-06T03:32:58.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.166+0000 7f20eb59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec158b00 con 0x7f20ec0b16d0 2023-10-06T03:32:58.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.166+0000 7f20e9d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20ec0b1b40 0x7f20ec15c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.166+0000 7f20ea59c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20ec0b16d0 0x7f20ec1585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.166+0000 7f20ea59c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20ec0b16d0 0x7f20ec1585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:36044/0 (socket says 172.21.15.163:36044) 2023-10-06T03:32:58.170 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.166+0000 7f20ea59c700 1 -- 172.21.15.163:0/1699741316 learned_addr learned my addr 172.21.15.163:0/1699741316 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-06T03:32:58.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.166+0000 7f20ead9d700 1 --2- 172.21.15.163:0/1699741316 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20ec0b4e70 0x7f20ec0cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.167+0000 7f20e9d9b700 1 -- 172.21.15.163:0/1699741316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20ec0b16d0 msgr2=0x7f20ec1585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.167+0000 7f20e9d9b700 1 --2- 172.21.15.163:0/1699741316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20ec0b16d0 0x7f20ec1585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.167+0000 7f20e9d9b700 1 -- 172.21.15.163:0/1699741316 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20ec0b4e70 msgr2=0x7f20ec0cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.167+0000 7f20e9d9b700 1 --2- 172.21.15.163:0/1699741316 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20ec0b4e70 0x7f20ec0cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.167+0000 7f20e9d9b700 1 -- 172.21.15.163:0/1699741316 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20dc0094f0 con 0x7f20ec0b1b40 2023-10-06T03:32:58.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.167+0000 7f20ea59c700 1 --2- 172.21.15.163:0/1699741316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20ec0b16d0 0x7f20ec1585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:32:58.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.167+0000 7f20e9d9b700 1 --2- 172.21.15.163:0/1699741316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20ec0b1b40 0x7f20ec15c850 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f20e000c590 tx=0x7f20e0007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:58.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.167+0000 7f20da7fc700 1 -- 172.21.15.163:0/1699741316 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0011590 con 0x7f20ec0b1b40 2023-10-06T03:32:58.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.167+0000 7f20da7fc700 1 -- 172.21.15.163:0/1699741316 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e0011710 con 0x7f20ec0b1b40 2023-10-06T03:32:58.171 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.167+0000 7f20eb59e700 1 -- 172.21.15.163:0/1699741316 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec15a950 con 0x7f20ec0b1b40 2023-10-06T03:32:58.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.167+0000 7f20da7fc700 1 -- 172.21.15.163:0/1699741316 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0011ad0 con 0x7f20ec0b1b40 2023-10-06T03:32:58.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.167+0000 7f20eb59e700 1 -- 172.21.15.163:0/1699741316 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec15c0d0 con 0x7f20ec0b1b40 2023-10-06T03:32:58.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.168+0000 7f20da7fc700 1 -- 172.21.15.163:0/1699741316 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f20e001c6c0 con 0x7f20ec0b1b40 2023-10-06T03:32:58.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.168+0000 7f20da7fc700 1 -- 172.21.15.163:0/1699741316 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f20e0012420 con 0x7f20ec0b1b40 2023-10-06T03:32:58.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.168+0000 7f20eb59e700 1 -- 172.21.15.163:0/1699741316 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ec088990 con 0x7f20ec0b1b40 2023-10-06T03:32:58.172 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.171+0000 7f20da7fc700 1 -- 172.21.15.163:0/1699741316 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f20e0017070 con 0x7f20ec0b1b40 2023-10-06T03:32:58.223 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:32:58.222+0000 7f07ce0ee180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:32:58.232 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:32:58.231+0000 7fead2280180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:32:58.255 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.254+0000 7f20eb59e700 1 -- 172.21.15.163:0/1699741316 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f49dfabb-4d9c-46bd-811b-984484ad87a1", "id": 5} v 0) v1 -- 0x7f20ec1596b0 con 0x7f20ec0b1b40 2023-10-06T03:32:58.263 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.262+0000 7f20da7fc700 1 -- 172.21.15.163:0/1699741316 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f49dfabb-4d9c-46bd-811b-984484ad87a1", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f20e001c970 con 0x7f20ec0b1b40 2023-10-06T03:32:58.263 INFO:teuthology.orchestra.run.smithi163.stdout:5 2023-10-06T03:32:58.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.264+0000 7f20eb59e700 1 -- 172.21.15.163:0/1699741316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20ec0b1b40 msgr2=0x7f20ec15c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.264+0000 7f20eb59e700 1 --2- 172.21.15.163:0/1699741316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20ec0b1b40 0x7f20ec15c850 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f20e000c590 tx=0x7f20e0007ec0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.264+0000 7f20eb59e700 1 -- 172.21.15.163:0/1699741316 shutdown_connections 2023-10-06T03:32:58.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.264+0000 7f20eb59e700 1 --2- 172.21.15.163:0/1699741316 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20ec0b4e70 0x7f20ec0cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.264+0000 7f20eb59e700 1 --2- 172.21.15.163:0/1699741316 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20ec0b16d0 0x7f20ec1585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.264+0000 7f20eb59e700 1 --2- 172.21.15.163:0/1699741316 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20ec0b1b40 0x7f20ec15c850 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.264+0000 7f20eb59e700 1 -- 172.21.15.163:0/1699741316 >> 172.21.15.163:0/1699741316 conn(0x7f20ec0d10c0 msgr2=0x7f20ec0c55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:58.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.264+0000 7f20eb59e700 1 -- 172.21.15.163:0/1699741316 shutdown_connections 2023-10-06T03:32:58.265 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.264+0000 7f20eb59e700 1 -- 172.21.15.163:0/1699741316 wait complete. 2023-10-06T03:32:58.285 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new d78418d1-8ff5-4678-a71d-1d18d19ffbfc 6 2023-10-06T03:32:58.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.463+0000 7fb06ef9d700 1 Processor -- start 2023-10-06T03:32:58.465 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.464+0000 7fb06ef9d700 1 -- start start 2023-10-06T03:32:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.465+0000 7fb06ef9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680b4e70 0x7fb0680ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.465+0000 7fb06ef9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb0680b16d0 0x7fb0680b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.465+0000 7fb06ef9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0680b2080 con 0x7fb06815d240 2023-10-06T03:32:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.465+0000 7fb06ef9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0680cfb00 con 0x7fb0680b16d0 2023-10-06T03:32:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.465+0000 7fb06ef9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680b5c40 con 0x7fb0680b4e70 2023-10-06T03:32:58.467 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.465+0000 7fb06d79a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb0680b16d0 0x7fb0680b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.465+0000 7fb06df9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680b4e70 0x7fb0680ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.465+0000 7fb06d79a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb0680b16d0 0x7fb0680b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39592/0 (socket says 172.21.15.163:39592) 2023-10-06T03:32:58.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.465+0000 7fb06d79a700 1 -- 172.21.15.163:0/3286498872 learned_addr learned my addr 172.21.15.163:0/3286498872 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-06T03:32:58.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.466+0000 7fb06d79a700 1 -- 172.21.15.163:0/3286498872 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680b4e70 msgr2=0x7fb0680ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:32:58.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.466+0000 7fb06d79a700 1 --2- 172.21.15.163:0/3286498872 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680b4e70 0x7fb0680ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.466+0000 7fb06d79a700 1 -- 172.21.15.163:0/3286498872 >> v1:172.21.15.55:6789/0 conn(0x7fb06815d240 legacy=0x7fb06815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.466+0000 7fb06d79a700 1 -- 172.21.15.163:0/3286498872 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0680b5f20 con 0x7fb0680b16d0 2023-10-06T03:32:58.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.466+0000 7fb06df9b700 1 --2- 172.21.15.163:0/3286498872 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680b4e70 0x7fb0680ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:32:58.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.466+0000 7fb06d79a700 1 --2- 172.21.15.163:0/3286498872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0680b16d0 0x7fb0680b1b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb0580099b0 tx=0x7fb058009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=769b6be351245c2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:58.468 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.466+0000 7fb06cf99700 1 -- 172.21.15.163:0/3286498872 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058009ea0 con 0x7fb0680b16d0 2023-10-06T03:32:58.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.466+0000 7fb06cf99700 1 -- 172.21.15.163:0/3286498872 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05800b860 con 0x7fb0680b16d0 2023-10-06T03:32:58.469 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.466+0000 7fb06cf99700 1 -- 172.21.15.163:0/3286498872 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05800bc00 con 0x7fb0680b16d0 2023-10-06T03:32:58.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.467+0000 7fb06ef9d700 1 -- 172.21.15.163:0/3286498872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0680b16d0 msgr2=0x7fb0680b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.467+0000 7fb06ef9d700 1 --2- 172.21.15.163:0/3286498872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0680b16d0 0x7fb0680b1b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb0580099b0 tx=0x7fb058009250 comp rx=0 tx=0).stop 2023-10-06T03:32:58.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.467+0000 7fb06ef9d700 1 -- 172.21.15.163:0/3286498872 shutdown_connections 2023-10-06T03:32:58.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.467+0000 7fb06ef9d700 1 --2- 172.21.15.163:0/3286498872 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680b4e70 0x7fb0680ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.467+0000 7fb06ef9d700 1 --2- 172.21.15.163:0/3286498872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0680b16d0 0x7fb0680b1b40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.467+0000 7fb06ef9d700 1 -- 172.21.15.163:0/3286498872 >> 172.21.15.163:0/3286498872 conn(0x7fb0680d10c0 msgr2=0x7fb0680affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:58.472 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.467+0000 7fb06ef9d700 1 -- 172.21.15.163:0/3286498872 shutdown_connections 2023-10-06T03:32:58.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.467+0000 7fb06ef9d700 1 -- 172.21.15.163:0/3286498872 wait complete. 2023-10-06T03:32:58.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.468+0000 7fb06ef9d700 1 Processor -- start 2023-10-06T03:32:58.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.468+0000 7fb06ef9d700 1 -- start start 2023-10-06T03:32:58.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.469+0000 7fb06ef9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0680b16d0 0x7fb06814c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.469+0000 7fb06ef9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680b4e70 0x7fb0680c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.469+0000 7fb06ef9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb06815d240 0x7fb0680ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.469+0000 7fb06ef9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680c6db0 con 0x7fb06815d240 2023-10-06T03:32:58.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.469+0000 7fb06ef9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680c7cc0 con 0x7fb0680b16d0 2023-10-06T03:32:58.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.469+0000 7fb06ef9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06814c5f0 con 0x7fb0680b4e70 2023-10-06T03:32:58.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.469+0000 7fb06df9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0680b16d0 0x7fb06814c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.469+0000 7fb06df9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0680b16d0 0x7fb06814c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.163:39608/0 (socket says 172.21.15.163:39608) 2023-10-06T03:32:58.473 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.470+0000 7fb06d79a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680b4e70 0x7fb0680c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.470+0000 7fb06df9b700 1 -- 172.21.15.163:0/419508002 learned_addr learned my addr 172.21.15.163:0/419508002 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-06T03:32:58.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.470+0000 7fb06e79c700 1 --2- 172.21.15.163:0/419508002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb06815d240 0x7fb0680ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.470+0000 7fb06df9b700 1 -- 172.21.15.163:0/419508002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680b4e70 msgr2=0x7fb0680c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.470+0000 7fb06df9b700 1 --2- 172.21.15.163:0/419508002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680b4e70 0x7fb0680c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.470+0000 7fb06df9b700 1 -- 172.21.15.163:0/419508002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb06815d240 msgr2=0x7fb0680ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.470+0000 7fb06df9b700 1 --2- 172.21.15.163:0/419508002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb06815d240 0x7fb0680ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.470+0000 7fb06df9b700 1 -- 172.21.15.163:0/419508002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb064009420 con 0x7fb0680b16d0 2023-10-06T03:32:58.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.470+0000 7fb06e79c700 1 --2- 172.21.15.163:0/419508002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb06815d240 0x7fb0680ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:32:58.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.470+0000 7fb06df9b700 1 --2- 172.21.15.163:0/419508002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0680b16d0 0x7fb06814c0b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb06400ec80 tx=0x7fb0640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:58.474 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.470+0000 7fb05dffb700 1 -- 172.21.15.163:0/419508002 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06400c8f0 con 0x7fb0680b16d0 2023-10-06T03:32:58.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.470+0000 7fb05dffb700 1 -- 172.21.15.163:0/419508002 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06400ca50 con 0x7fb0680b16d0 2023-10-06T03:32:58.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.470+0000 7fb05dffb700 1 -- 172.21.15.163:0/419508002 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06400cdd0 con 0x7fb0680b16d0 2023-10-06T03:32:58.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.470+0000 7fb06ef9d700 1 -- 172.21.15.163:0/419508002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0580094f0 con 0x7fb0680b16d0 2023-10-06T03:32:58.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.471+0000 7fb06ef9d700 1 -- 172.21.15.163:0/419508002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0680c6ad0 con 0x7fb0680b16d0 2023-10-06T03:32:58.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.471+0000 7fb05dffb700 1 -- 172.21.15.163:0/419508002 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fb064005c80 con 0x7fb0680b16d0 2023-10-06T03:32:58.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.471+0000 7fb05dffb700 1 -- 172.21.15.163:0/419508002 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7fb064013680 con 0x7fb0680b16d0 2023-10-06T03:32:58.475 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.472+0000 7fb06ef9d700 1 -- 172.21.15.163:0/419508002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb068088990 con 0x7fb0680b16d0 2023-10-06T03:32:58.476 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.474+0000 7fb05dffb700 1 -- 172.21.15.163:0/419508002 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fb064021060 con 0x7fb0680b16d0 2023-10-06T03:32:58.561 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.560+0000 7fb06ef9d700 1 -- 172.21.15.163:0/419508002 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d78418d1-8ff5-4678-a71d-1d18d19ffbfc", "id": 6} v 0) v1 -- 0x7fb06804bb80 con 0x7fb0680b16d0 2023-10-06T03:32:58.570 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.569+0000 7fb05dffb700 1 -- 172.21.15.163:0/419508002 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d78418d1-8ff5-4678-a71d-1d18d19ffbfc", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb064029020 con 0x7fb0680b16d0 2023-10-06T03:32:58.570 INFO:teuthology.orchestra.run.smithi163.stdout:6 2023-10-06T03:32:58.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.570+0000 7fb06ef9d700 1 -- 172.21.15.163:0/419508002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0680b16d0 msgr2=0x7fb06814c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.570+0000 7fb06ef9d700 1 --2- 172.21.15.163:0/419508002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0680b16d0 0x7fb06814c0b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb06400ec80 tx=0x7fb0640091a0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.571 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.570+0000 7fb06ef9d700 1 -- 172.21.15.163:0/419508002 shutdown_connections 2023-10-06T03:32:58.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.570+0000 7fb06ef9d700 1 --2- 172.21.15.163:0/419508002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb06815d240 0x7fb0680ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.570+0000 7fb06ef9d700 1 --2- 172.21.15.163:0/419508002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0680b16d0 0x7fb06814c0b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.570+0000 7fb06ef9d700 1 --2- 172.21.15.163:0/419508002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0680b4e70 0x7fb0680c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.570+0000 7fb06ef9d700 1 -- 172.21.15.163:0/419508002 >> 172.21.15.163:0/419508002 conn(0x7fb0680d10c0 msgr2=0x7fb06815c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:58.572 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.570+0000 7fb06ef9d700 1 -- 172.21.15.163:0/419508002 shutdown_connections 2023-10-06T03:32:58.573 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.570+0000 7fb06ef9d700 1 -- 172.21.15.163:0/419508002 wait complete. 2023-10-06T03:32:58.589 DEBUG:teuthology.orchestra.run.smithi163:> sudo ceph --cluster ceph osd new 7b768a49-37d1-4cb3-a670-ab9027d1248f 7 2023-10-06T03:32:58.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.768+0000 7fee3499d700 1 Processor -- start 2023-10-06T03:32:58.769 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.768+0000 7fee3499d700 1 -- start start 2023-10-06T03:32:58.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.769+0000 7fee3499d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fee300b4e70 0x7fee300ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.769+0000 7fee3499d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee300b1b40 0x7fee3015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.771 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.769+0000 7fee3499d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee300cfb00 con 0x7fee300b16d0 2023-10-06T03:32:58.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.769+0000 7fee3499d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee300b5c40 con 0x7fee300b4e70 2023-10-06T03:32:58.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.769+0000 7fee3499d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee300b5f20 con 0x7fee300b1b40 2023-10-06T03:32:58.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.769+0000 7fee2ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee300b1b40 0x7fee3015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.769+0000 7fee2f7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fee300b4e70 0x7fee300ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.769+0000 7fee2ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee300b1b40 0x7fee3015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:36076/0 (socket says 172.21.15.163:36076) 2023-10-06T03:32:58.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.770+0000 7fee2effd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fee300b16d0 0x7fee3015d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.163:59252/0 (socket says 172.21.15.163:59252) 2023-10-06T03:32:58.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.770+0000 7fee2ffff700 1 -- 172.21.15.163:0/1199722323 learned_addr learned my addr 172.21.15.163:0/1199722323 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-06T03:32:58.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.770+0000 7fee2f7fe700 1 -- 172.21.15.163:0/1199722323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee300b1b40 msgr2=0x7fee3015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:32:58.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.770+0000 7fee2f7fe700 1 --2- 172.21.15.163:0/1199722323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee300b1b40 0x7fee3015d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.770+0000 7fee2f7fe700 1 -- 172.21.15.163:0/1199722323 >> v1:172.21.15.55:6789/0 conn(0x7fee300b16d0 legacy=0x7fee3015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.770+0000 7fee2f7fe700 1 -- 172.21.15.163:0/1199722323 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee300b6200 con 0x7fee300b4e70 2023-10-06T03:32:58.772 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.770+0000 7fee2e7fc700 1 -- 172.21.15.163:0/1199722323 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362519518 0 0) 0x7fee300cfb00 con 0x7fee300b16d0 2023-10-06T03:32:58.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.770+0000 7fee2ffff700 1 --2- 172.21.15.163:0/1199722323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee300b1b40 0x7fee3015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:32:58.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.770+0000 7fee2f7fe700 1 --2- 172.21.15.163:0/1199722323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee300b4e70 0x7fee300ccff0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fee200098e0 tx=0x7fee200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19623cfa9b9a6d3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:58.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.770+0000 7fee2e7fc700 1 -- 172.21.15.163:0/1199722323 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2000b640 con 0x7fee300b4e70 2023-10-06T03:32:58.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.771+0000 7fee2e7fc700 1 -- 172.21.15.163:0/1199722323 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2000b7a0 con 0x7fee300b4e70 2023-10-06T03:32:58.773 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.771+0000 7fee2e7fc700 1 -- 172.21.15.163:0/1199722323 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2000bb40 con 0x7fee300b4e70 2023-10-06T03:32:58.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.771+0000 7fee3499d700 1 -- 172.21.15.163:0/1199722323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee300b4e70 msgr2=0x7fee300ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.771+0000 7fee3499d700 1 --2- 172.21.15.163:0/1199722323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee300b4e70 0x7fee300ccff0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fee200098e0 tx=0x7fee200091a0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.771+0000 7fee3499d700 1 -- 172.21.15.163:0/1199722323 shutdown_connections 2023-10-06T03:32:58.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.771+0000 7fee3499d700 1 --2- 172.21.15.163:0/1199722323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee300b1b40 0x7fee3015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.774 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.771+0000 7fee3499d700 1 --2- 172.21.15.163:0/1199722323 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee300b4e70 0x7fee300ccff0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.771+0000 7fee3499d700 1 -- 172.21.15.163:0/1199722323 >> 172.21.15.163:0/1199722323 conn(0x7fee300d10c0 msgr2=0x7fee300affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:58.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.771+0000 7fee3499d700 1 -- 172.21.15.163:0/1199722323 shutdown_connections 2023-10-06T03:32:58.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.771+0000 7fee3499d700 1 -- 172.21.15.163:0/1199722323 wait complete. 2023-10-06T03:32:58.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.773+0000 7fee3499d700 1 Processor -- start 2023-10-06T03:32:58.775 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.773+0000 7fee3499d700 1 -- start start 2023-10-06T03:32:58.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee3499d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee300b16d0 0x7fee30156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee3499d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee300b1b40 0x7fee300c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee3499d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee300b4e70 0x7fee300c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:58.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee3499d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee301567b0 con 0x7fee300b1b40 2023-10-06T03:32:58.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee3499d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee300c6770 con 0x7fee300b4e70 2023-10-06T03:32:58.777 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee3499d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee300c9020 con 0x7fee300b16d0 2023-10-06T03:32:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee2f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee300b16d0 0x7fee30156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee2f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee300b16d0 0x7fee30156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.163:36086/0 (socket says 172.21.15.163:36086) 2023-10-06T03:32:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee2f7fe700 1 -- 172.21.15.163:0/4040182946 learned_addr learned my addr 172.21.15.163:0/4040182946 (peer_addr_for_me v2:172.21.15.163:0/0) 2023-10-06T03:32:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee2effd700 1 --2- 172.21.15.163:0/4040182946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee300b1b40 0x7fee300c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee2ffff700 1 --2- 172.21.15.163:0/4040182946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee300b4e70 0x7fee300c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee2f7fe700 1 -- 172.21.15.163:0/4040182946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee300b4e70 msgr2=0x7fee300c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee2f7fe700 1 --2- 172.21.15.163:0/4040182946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee300b4e70 0x7fee300c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.778 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee2f7fe700 1 -- 172.21.15.163:0/4040182946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee300b1b40 msgr2=0x7fee300c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.774+0000 7fee2f7fe700 1 --2- 172.21.15.163:0/4040182946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee300b1b40 0x7fee300c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.775+0000 7fee2f7fe700 1 -- 172.21.15.163:0/4040182946 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee28009ae0 con 0x7fee300b16d0 2023-10-06T03:32:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.775+0000 7fee2effd700 1 --2- 172.21.15.163:0/4040182946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee300b1b40 0x7fee300c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:32:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.775+0000 7fee2f7fe700 1 --2- 172.21.15.163:0/4040182946 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee300b16d0 0x7fee30156270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee30070dd0 tx=0x7fee20017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.775+0000 7fee177fe700 1 -- 172.21.15.163:0/4040182946 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2000bb20 con 0x7fee300b16d0 2023-10-06T03:32:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.775+0000 7fee177fe700 1 -- 172.21.15.163:0/4040182946 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee20000c90 con 0x7fee300b16d0 2023-10-06T03:32:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.775+0000 7fee177fe700 1 -- 172.21.15.163:0/4040182946 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee20011630 con 0x7fee300b16d0 2023-10-06T03:32:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.775+0000 7fee3499d700 1 -- 172.21.15.163:0/4040182946 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee20009420 con 0x7fee300b16d0 2023-10-06T03:32:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.775+0000 7fee3499d700 1 -- 172.21.15.163:0/4040182946 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee300c8d40 con 0x7fee300b16d0 2023-10-06T03:32:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.775+0000 7fee177fe700 1 -- 172.21.15.163:0/4040182946 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fee200119a0 con 0x7fee300b16d0 2023-10-06T03:32:58.779 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.776+0000 7fee177fe700 1 -- 172.21.15.163:0/4040182946 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7fee20011c50 con 0x7fee300b16d0 2023-10-06T03:32:58.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.776+0000 7fee3499d700 1 -- 172.21.15.163:0/4040182946 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee30088990 con 0x7fee300b16d0 2023-10-06T03:32:58.780 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.779+0000 7fee177fe700 1 -- 172.21.15.163:0/4040182946 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fee20031060 con 0x7fee300b16d0 2023-10-06T03:32:58.849 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:32:58.848+0000 7fead2280180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:32:58.851 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:32:58.850+0000 7f07ce0ee180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:32:58.863 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.862+0000 7fee3499d700 1 -- 172.21.15.163:0/4040182946 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7b768a49-37d1-4cb3-a670-ab9027d1248f", "id": 7} v 0) v1 -- 0x7fee3004bb80 con 0x7fee300b16d0 2023-10-06T03:32:58.874 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.873+0000 7fee177fe700 1 -- 172.21.15.163:0/4040182946 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7b768a49-37d1-4cb3-a670-ab9027d1248f", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fee20015020 con 0x7fee300b16d0 2023-10-06T03:32:58.874 INFO:teuthology.orchestra.run.smithi163.stdout:7 2023-10-06T03:32:58.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.874+0000 7fee3499d700 1 -- 172.21.15.163:0/4040182946 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee300b16d0 msgr2=0x7fee30156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:58.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.874+0000 7fee3499d700 1 --2- 172.21.15.163:0/4040182946 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee300b16d0 0x7fee30156270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fee30070dd0 tx=0x7fee20017000 comp rx=0 tx=0).stop 2023-10-06T03:32:58.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.874+0000 7fee3499d700 1 -- 172.21.15.163:0/4040182946 shutdown_connections 2023-10-06T03:32:58.875 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.874+0000 7fee3499d700 1 --2- 172.21.15.163:0/4040182946 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee300b4e70 0x7fee300c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.874+0000 7fee3499d700 1 --2- 172.21.15.163:0/4040182946 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee300b16d0 0x7fee30156270 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.874+0000 7fee3499d700 1 --2- 172.21.15.163:0/4040182946 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee300b1b40 0x7fee300c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:58.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.874+0000 7fee3499d700 1 -- 172.21.15.163:0/4040182946 >> 172.21.15.163:0/4040182946 conn(0x7fee300d10c0 msgr2=0x7fee301590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:58.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.874+0000 7fee3499d700 1 -- 172.21.15.163:0/4040182946 shutdown_connections 2023-10-06T03:32:58.876 INFO:teuthology.orchestra.run.smithi163.stderr:2023-10-06T03:32:58.874+0000 7fee3499d700 1 -- 172.21.15.163:0/4040182946 wait complete. 2023-10-06T03:32:58.892 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T03:32:58.892 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T03:32:58.896 INFO:tasks.ceph.osd.0:Started 2023-10-06T03:32:58.896 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T03:32:58.896 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T03:32:58.901 INFO:tasks.ceph.osd.1:Started 2023-10-06T03:32:58.902 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T03:32:58.902 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T03:32:58.908 INFO:tasks.ceph.osd.2:Started 2023-10-06T03:32:58.908 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T03:32:58.908 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T03:32:58.914 INFO:tasks.ceph.osd.3:Started 2023-10-06T03:32:58.914 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T03:32:58.914 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T03:32:58.919 INFO:tasks.ceph.osd.4:Started 2023-10-06T03:32:58.919 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T03:32:58.919 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T03:32:58.923 INFO:tasks.ceph.osd.5:Started 2023-10-06T03:32:58.923 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T03:32:58.923 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T03:32:58.927 INFO:tasks.ceph.osd.6:Started 2023-10-06T03:32:58.928 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T03:32:58.928 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T03:32:58.933 INFO:tasks.ceph.osd.7:Started 2023-10-06T03:32:58.934 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:32:58.954 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:32:58.953+0000 7fead2280180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:32:58.971 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:32:58.970+0000 7f07ce0ee180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:32:59.067 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:32:59.066+0000 7fead2280180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:32:59.110 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:32:59.109+0000 7f07ce0ee180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:32:59.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.178+0000 7f26aaf9d700 1 Processor -- start 2023-10-06T03:32:59.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.178+0000 7f26aaf9d700 1 -- start start 2023-10-06T03:32:59.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.179+0000 7f26aaf9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a4159420 0x7f26a40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:59.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.179+0000 7f26aaf9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26a41597f0 0x7f26a40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:59.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.179+0000 7f26aaf9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f26a415d230 0x7f26a415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:59.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.179+0000 7f26aaf9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a415dbd0 con 0x7f26a41597f0 2023-10-06T03:32:59.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.179+0000 7f26aaf9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a40b11d0 con 0x7f26a415d230 2023-10-06T03:32:59.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.179+0000 7f26aaf9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40b14b0 con 0x7f26a4159420 2023-10-06T03:32:59.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.180+0000 7f26a979a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26a41597f0 0x7f26a40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:59.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.180+0000 7f26a979a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26a41597f0 0x7f26a40b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50814/0 (socket says 172.21.15.55:50814) 2023-10-06T03:32:59.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.180+0000 7f26a979a700 1 -- 172.21.15.55:0/2549187831 learned_addr learned my addr 172.21.15.55:0/2549187831 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:32:59.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.180+0000 7f26a9f9b700 1 --2- 172.21.15.55:0/2549187831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a4159420 0x7f26a40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:59.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.180+0000 7f26aa79c700 1 --2- 172.21.15.55:0/2549187831 >> v2:172.21.15.163:3300/0 conn(0x7f26a415d230 0x7f26a415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.180+0000 7f26a979a700 1 -- 172.21.15.55:0/2549187831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a4159420 msgr2=0x7f26a40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:32:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.180+0000 7f26a979a700 1 --2- 172.21.15.55:0/2549187831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a4159420 0x7f26a40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.180+0000 7f26a979a700 1 -- 172.21.15.55:0/2549187831 >> v2:172.21.15.163:3300/0 conn(0x7f26a415d230 msgr2=0x7f26a415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:32:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.180+0000 7f26a979a700 1 --2- 172.21.15.55:0/2549187831 >> v2:172.21.15.163:3300/0 conn(0x7f26a415d230 0x7f26a415d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.180+0000 7f26a979a700 1 -- 172.21.15.55:0/2549187831 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a40b1790 con 0x7f26a41597f0 2023-10-06T03:32:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.180+0000 7f26a979a700 1 --2- 172.21.15.55:0/2549187831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a41597f0 0x7f26a40b2270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f26a40d1720 tx=0x7f2694009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b947280af85323 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:59.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.181+0000 7f26a8f99700 1 -- 172.21.15.55:0/2549187831 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269400b640 con 0x7f26a41597f0 2023-10-06T03:32:59.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.181+0000 7f26a8f99700 1 -- 172.21.15.55:0/2549187831 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269400b7a0 con 0x7f26a41597f0 2023-10-06T03:32:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.181+0000 7f26a8f99700 1 -- 172.21.15.55:0/2549187831 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269400bb60 con 0x7f26a41597f0 2023-10-06T03:32:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.181+0000 7f26aaf9d700 1 -- 172.21.15.55:0/2549187831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a41597f0 msgr2=0x7f26a40b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.181+0000 7f26aaf9d700 1 --2- 172.21.15.55:0/2549187831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a41597f0 0x7f26a40b2270 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f26a40d1720 tx=0x7f2694009250 comp rx=0 tx=0).stop 2023-10-06T03:32:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.181+0000 7f26aaf9d700 1 -- 172.21.15.55:0/2549187831 shutdown_connections 2023-10-06T03:32:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.181+0000 7f26aaf9d700 1 --2- 172.21.15.55:0/2549187831 >> v2:172.21.15.163:3300/0 conn(0x7f26a415d230 0x7f26a415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.181+0000 7f26aaf9d700 1 --2- 172.21.15.55:0/2549187831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a4159420 0x7f26a40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.181+0000 7f26aaf9d700 1 --2- 172.21.15.55:0/2549187831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a41597f0 0x7f26a40b2270 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.181+0000 7f26aaf9d700 1 -- 172.21.15.55:0/2549187831 >> 172.21.15.55:0/2549187831 conn(0x7f26a40d0660 msgr2=0x7f26a40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:59.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.181+0000 7f26aaf9d700 1 -- 172.21.15.55:0/2549187831 shutdown_connections 2023-10-06T03:32:59.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.181+0000 7f26aaf9d700 1 -- 172.21.15.55:0/2549187831 wait complete. 2023-10-06T03:32:59.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.183+0000 7f26aaf9d700 1 Processor -- start 2023-10-06T03:32:59.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.183+0000 7f26aaf9d700 1 -- start start 2023-10-06T03:32:59.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.184+0000 7f26aaf9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a4159420 0x7f26a40b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:59.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.184+0000 7f26aaf9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a41597f0 0x7f26a40c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:59.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.184+0000 7f26aaf9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a415d230 0x7f26a40c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:32:59.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.184+0000 7f26aaf9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40ca820 con 0x7f26a4159420 2023-10-06T03:32:59.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.184+0000 7f26aaf9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40c9720 con 0x7f26a41597f0 2023-10-06T03:32:59.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.184+0000 7f26aaf9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a416dde0 con 0x7f26a415d230 2023-10-06T03:32:59.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.184+0000 7f26a9f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a4159420 0x7f26a40b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:59.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.184+0000 7f26a9f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a4159420 0x7f26a40b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50826/0 (socket says 172.21.15.55:50826) 2023-10-06T03:32:59.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.184+0000 7f26a9f9b700 1 -- 172.21.15.55:0/2371792209 learned_addr learned my addr 172.21.15.55:0/2371792209 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:32:59.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.184+0000 7f26a979a700 1 --2- 172.21.15.55:0/2371792209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a41597f0 0x7f26a40c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:59.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.184+0000 7f26aa79c700 1 --2- 172.21.15.55:0/2371792209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a415d230 0x7f26a40c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:32:59.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.185+0000 7f26a9f9b700 1 -- 172.21.15.55:0/2371792209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a415d230 msgr2=0x7f26a40c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:59.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.185+0000 7f26a9f9b700 1 --2- 172.21.15.55:0/2371792209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a415d230 0x7f26a40c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:59.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.185+0000 7f26a9f9b700 1 -- 172.21.15.55:0/2371792209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a41597f0 msgr2=0x7f26a40c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:59.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.185+0000 7f26a9f9b700 1 --2- 172.21.15.55:0/2371792209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a41597f0 0x7f26a40c8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:59.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.185+0000 7f26a9f9b700 1 -- 172.21.15.55:0/2371792209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26940094f0 con 0x7f26a4159420 2023-10-06T03:32:59.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.185+0000 7f26a9f9b700 1 --2- 172.21.15.55:0/2371792209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a4159420 0x7f26a40b40a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f26a000ec80 tx=0x7f26a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:32:59.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.185+0000 7f2699ffb700 1 -- 172.21.15.55:0/2371792209 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a000c850 con 0x7f26a4159420 2023-10-06T03:32:59.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.185+0000 7f2699ffb700 1 -- 172.21.15.55:0/2371792209 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a000c9b0 con 0x7f26a4159420 2023-10-06T03:32:59.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.185+0000 7f2699ffb700 1 -- 172.21.15.55:0/2371792209 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a000cd50 con 0x7f26a4159420 2023-10-06T03:32:59.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.185+0000 7f26aaf9d700 1 -- 172.21.15.55:0/2371792209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a4169d70 con 0x7f26a4159420 2023-10-06T03:32:59.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.185+0000 7f26aaf9d700 1 -- 172.21.15.55:0/2371792209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a416a330 con 0x7f26a4159420 2023-10-06T03:32:59.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.185+0000 7f2699ffb700 1 -- 172.21.15.55:0/2371792209 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f26a0005c80 con 0x7f26a4159420 2023-10-06T03:32:59.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.185+0000 7f2699ffb700 1 -- 172.21.15.55:0/2371792209 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f26a0013680 con 0x7f26a4159420 2023-10-06T03:32:59.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.186+0000 7f26aaf9d700 1 -- 172.21.15.55:0/2371792209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a416a6a0 con 0x7f26a4159420 2023-10-06T03:32:59.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.191+0000 7f2699ffb700 1 -- 172.21.15.55:0/2371792209 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f26a0021060 con 0x7f26a4159420 2023-10-06T03:32:59.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.263+0000 7f26aaf9d700 1 -- 172.21.15.55:0/2371792209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26a404c3c0 con 0x7f26a4159420 2023-10-06T03:32:59.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.264+0000 7f2699ffb700 1 -- 172.21.15.55:0/2371792209 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f26a0029020 con 0x7f26a4159420 2023-10-06T03:32:59.266 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:32:59.266 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":10,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:32:58.864136+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:32:59.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.265+0000 7f26aaf9d700 1 -- 172.21.15.55:0/2371792209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a4159420 msgr2=0x7f26a40b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:32:59.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.265+0000 7f26aaf9d700 1 --2- 172.21.15.55:0/2371792209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a4159420 0x7f26a40b40a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f26a000ec80 tx=0x7f26a00091a0 comp rx=0 tx=0).stop 2023-10-06T03:32:59.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.265+0000 7f26aaf9d700 1 -- 172.21.15.55:0/2371792209 shutdown_connections 2023-10-06T03:32:59.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.265+0000 7f26aaf9d700 1 --2- 172.21.15.55:0/2371792209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a415d230 0x7f26a40c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:59.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.265+0000 7f26aaf9d700 1 --2- 172.21.15.55:0/2371792209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a4159420 0x7f26a40b40a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:59.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.265+0000 7f26aaf9d700 1 --2- 172.21.15.55:0/2371792209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a41597f0 0x7f26a40c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:32:59.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.265+0000 7f26aaf9d700 1 -- 172.21.15.55:0/2371792209 >> 172.21.15.55:0/2371792209 conn(0x7f26a40d0660 msgr2=0x7f26a40b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:32:59.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.266+0000 7f26aaf9d700 1 -- 172.21.15.55:0/2371792209 shutdown_connections 2023-10-06T03:32:59.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:32:59.266+0000 7f26aaf9d700 1 -- 172.21.15.55:0/2371792209 wait complete. 2023-10-06T03:32:59.285 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T03:32:59.285 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T03:32:59.577 INFO:tasks.ceph.osd.0.smithi055.stderr:2023-10-06T03:32:59.576+0000 7f92faddb540 -1 Falling back to public interface 2023-10-06T03:32:59.598 INFO:tasks.ceph.osd.6.smithi163.stderr:2023-10-06T03:32:59.597+0000 7f59796b7540 -1 Falling back to public interface 2023-10-06T03:32:59.610 INFO:tasks.ceph.osd.1.smithi055.stderr:2023-10-06T03:32:59.609+0000 7f157532d540 -1 Falling back to public interface 2023-10-06T03:32:59.611 INFO:tasks.ceph.osd.7.smithi163.stderr:2023-10-06T03:32:59.610+0000 7fd3157d6540 -1 Falling back to public interface 2023-10-06T03:32:59.620 INFO:tasks.ceph.osd.2.smithi055.stderr:2023-10-06T03:32:59.619+0000 7fad6097a540 -1 Falling back to public interface 2023-10-06T03:32:59.627 INFO:tasks.ceph.osd.5.smithi163.stderr:2023-10-06T03:32:59.627+0000 7f49494d2540 -1 Falling back to public interface 2023-10-06T03:32:59.629 INFO:tasks.ceph.osd.3.smithi055.stderr:2023-10-06T03:32:59.628+0000 7f8431bc2540 -1 Falling back to public interface 2023-10-06T03:32:59.637 INFO:tasks.ceph.osd.4.smithi163.stderr:2023-10-06T03:32:59.636+0000 7fea277b8540 -1 Falling back to public interface 2023-10-06T03:32:59.810 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:32:59.809+0000 7fead2280180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:32:59.903 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:32:59.901+0000 7f07ce0ee180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:32:59.981 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:32:59.980+0000 7fead2280180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:33:00.081 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:33:00.080+0000 7f07ce0ee180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:33:00.090 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:00.094 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:33:00.093+0000 7fead2280180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:33:00.198 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:33:00.197+0000 7f07ce0ee180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:33:00.255 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.253+0000 7fb38dd9d700 1 Processor -- start 2023-10-06T03:33:00.255 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.253+0000 7fb38dd9d700 1 -- start start 2023-10-06T03:33:00.256 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.255+0000 7fb38dd9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb3880b2a00 0x7fb3880c6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:00.256 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.255+0000 7fb38dd9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb3880c5400 0x7fb3880cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:00.256 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.255+0000 7fb38dd9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3880cf5e0 con 0x7fb3880cf180 2023-10-06T03:33:00.256 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.255+0000 7fb38dd9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb388159f10 con 0x7fb3880b2a00 2023-10-06T03:33:00.257 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.255+0000 7fb38dd9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3880c6b70 con 0x7fb3880c5400 2023-10-06T03:33:00.257 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.255+0000 7fb38cd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb3880b2a00 0x7fb3880c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:00.257 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.255+0000 7fb38cd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb3880b2a00 0x7fb3880c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50830/0 (socket says 172.21.15.55:50830) 2023-10-06T03:33:00.257 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.255+0000 7fb38cd9b700 1 -- 172.21.15.55:0/2066359692 learned_addr learned my addr 172.21.15.55:0/2066359692 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:00.257 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.255+0000 7fb387fff700 1 --2- 172.21.15.55:0/2066359692 >> v2:172.21.15.163:3300/0 conn(0x7fb3880c5400 0x7fb3880cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:00.258 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.255+0000 7fb38cd9b700 1 -- 172.21.15.55:0/2066359692 >> v2:172.21.15.163:3300/0 conn(0x7fb3880c5400 msgr2=0x7fb3880cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:00.258 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.255+0000 7fb38cd9b700 1 --2- 172.21.15.55:0/2066359692 >> v2:172.21.15.163:3300/0 conn(0x7fb3880c5400 0x7fb3880cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:00.259 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.255+0000 7fb38cd9b700 1 -- 172.21.15.55:0/2066359692 >> v1:172.21.15.55:6789/0 conn(0x7fb3880cf180 legacy=0x7fb38815d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:00.259 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.255+0000 7fb38cd9b700 1 -- 172.21.15.55:0/2066359692 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3880c6e50 con 0x7fb3880b2a00 2023-10-06T03:33:00.259 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.256+0000 7fb38cd9b700 1 --2- 172.21.15.55:0/2066359692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3880b2a00 0x7fb3880c6320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0099b0 tx=0x7fb37c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a570ef08a0c345 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:00.259 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.256+0000 7fb3877fe700 1 -- 172.21.15.55:0/2066359692 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c009ea0 con 0x7fb3880b2a00 2023-10-06T03:33:00.259 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.256+0000 7fb3877fe700 1 -- 172.21.15.55:0/2066359692 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c00b820 con 0x7fb3880b2a00 2023-10-06T03:33:00.259 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.256+0000 7fb3877fe700 1 -- 172.21.15.55:0/2066359692 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c00bbc0 con 0x7fb3880b2a00 2023-10-06T03:33:00.259 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.257+0000 7fb38dd9d700 1 -- 172.21.15.55:0/2066359692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3880b2a00 msgr2=0x7fb3880c6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:00.259 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.257+0000 7fb38dd9d700 1 --2- 172.21.15.55:0/2066359692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3880b2a00 0x7fb3880c6320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0099b0 tx=0x7fb37c009250 comp rx=0 tx=0).stop 2023-10-06T03:33:00.259 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.257+0000 7fb38dd9d700 1 -- 172.21.15.55:0/2066359692 shutdown_connections 2023-10-06T03:33:00.259 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.257+0000 7fb38dd9d700 1 --2- 172.21.15.55:0/2066359692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3880b2a00 0x7fb3880c6320 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:00.259 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.257+0000 7fb38dd9d700 1 --2- 172.21.15.55:0/2066359692 >> v2:172.21.15.163:3300/0 conn(0x7fb3880c5400 0x7fb3880cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:00.260 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.257+0000 7fb38dd9d700 1 -- 172.21.15.55:0/2066359692 >> 172.21.15.55:0/2066359692 conn(0x7fb388087ff0 msgr2=0x7fb38806d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:00.260 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.257+0000 7fb38dd9d700 1 -- 172.21.15.55:0/2066359692 shutdown_connections 2023-10-06T03:33:00.260 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.257+0000 7fb38dd9d700 1 -- 172.21.15.55:0/2066359692 wait complete. 2023-10-06T03:33:00.261 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.258+0000 7fb38dd9d700 1 Processor -- start 2023-10-06T03:33:00.261 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.258+0000 7fb38dd9d700 1 -- start start 2023-10-06T03:33:00.261 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.259+0000 7fb38dd9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3880b2a00 0x7fb388157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:00.261 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.259+0000 7fb38dd9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3880c5400 0x7fb38809cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:00.261 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.259+0000 7fb38dd9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3880cf180 0x7fb38806e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:00.261 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.259+0000 7fb38dd9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38815b5c0 con 0x7fb3880cf180 2023-10-06T03:33:00.261 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.259+0000 7fb38dd9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38809d7f0 con 0x7fb3880b2a00 2023-10-06T03:33:00.261 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.259+0000 7fb38dd9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38809dad0 con 0x7fb3880c5400 2023-10-06T03:33:00.261 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.260+0000 7fb38d59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3880cf180 0x7fb38806e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:00.262 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.260+0000 7fb38cd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3880b2a00 0x7fb388157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:00.262 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.260+0000 7fb387fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3880c5400 0x7fb38809cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:00.262 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.260+0000 7fb38cd9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3880b2a00 0x7fb388157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:36588/0 (socket says 172.21.15.55:36588) 2023-10-06T03:33:00.262 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.260+0000 7fb38cd9b700 1 -- 172.21.15.55:0/1192395390 learned_addr learned my addr 172.21.15.55:0/1192395390 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:00.263 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.260+0000 7fb38d59c700 1 -- 172.21.15.55:0/1192395390 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3880c5400 msgr2=0x7fb38809cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:00.263 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.260+0000 7fb38d59c700 1 --2- 172.21.15.55:0/1192395390 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3880c5400 0x7fb38809cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:00.263 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.260+0000 7fb38d59c700 1 -- 172.21.15.55:0/1192395390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3880b2a00 msgr2=0x7fb388157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:00.263 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.260+0000 7fb38d59c700 1 --2- 172.21.15.55:0/1192395390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3880b2a00 0x7fb388157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:00.264 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.260+0000 7fb38d59c700 1 -- 172.21.15.55:0/1192395390 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c0094f0 con 0x7fb3880cf180 2023-10-06T03:33:00.264 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.260+0000 7fb38d59c700 1 --2- 172.21.15.55:0/1192395390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3880cf180 0x7fb38806e4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb38000c790 tx=0x7fb3800015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:00.264 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.261+0000 7fb384ff9700 1 -- 172.21.15.55:0/1192395390 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380012630 con 0x7fb3880cf180 2023-10-06T03:33:00.264 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.261+0000 7fb384ff9700 1 -- 172.21.15.55:0/1192395390 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3800127b0 con 0x7fb3880cf180 2023-10-06T03:33:00.264 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.261+0000 7fb384ff9700 1 -- 172.21.15.55:0/1192395390 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380012b70 con 0x7fb3880cf180 2023-10-06T03:33:00.264 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.261+0000 7fb38dd9d700 1 -- 172.21.15.55:0/1192395390 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38809ddb0 con 0x7fb3880cf180 2023-10-06T03:33:00.264 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.261+0000 7fb38dd9d700 1 -- 172.21.15.55:0/1192395390 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38809d4e0 con 0x7fb3880cf180 2023-10-06T03:33:00.264 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.261+0000 7fb384ff9700 1 -- 172.21.15.55:0/1192395390 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fb380003030 con 0x7fb3880cf180 2023-10-06T03:33:00.264 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.261+0000 7fb384ff9700 1 -- 172.21.15.55:0/1192395390 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fb38001b400 con 0x7fb3880cf180 2023-10-06T03:33:00.264 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.262+0000 7fb38dd9d700 1 -- 172.21.15.55:0/1192395390 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38815b730 con 0x7fb3880cf180 2023-10-06T03:33:00.265 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.264+0000 7fb384ff9700 1 -- 172.21.15.55:0/1192395390 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fb3880cf5e0 con 0x7fb3880cf180 2023-10-06T03:33:00.340 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.339+0000 7fb38dd9d700 1 -- 172.21.15.55:0/1192395390 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb388064b50 con 0x7fb3880cf180 2023-10-06T03:33:00.341 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.339+0000 7fb384ff9700 1 -- 172.21.15.55:0/1192395390 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fb38001b5c0 con 0x7fb3880cf180 2023-10-06T03:33:00.341 INFO:teuthology.misc.health.smithi055.stdout: 2023-10-06T03:33:00.341 INFO:teuthology.misc.health.smithi055.stdout:{"epoch":10,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:32:58.864136+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:00.342 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.340+0000 7fb38dd9d700 1 -- 172.21.15.55:0/1192395390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3880cf180 msgr2=0x7fb38806e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:00.342 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.340+0000 7fb38dd9d700 1 --2- 172.21.15.55:0/1192395390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3880cf180 0x7fb38806e4e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb38000c790 tx=0x7fb3800015f0 comp rx=0 tx=0).stop 2023-10-06T03:33:00.342 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.341+0000 7fb38dd9d700 1 -- 172.21.15.55:0/1192395390 shutdown_connections 2023-10-06T03:33:00.342 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.341+0000 7fb38dd9d700 1 --2- 172.21.15.55:0/1192395390 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3880cf180 0x7fb38806e4e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:00.342 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.341+0000 7fb38dd9d700 1 --2- 172.21.15.55:0/1192395390 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3880b2a00 0x7fb388157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:00.342 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.341+0000 7fb38dd9d700 1 --2- 172.21.15.55:0/1192395390 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3880c5400 0x7fb38809cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:00.343 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.341+0000 7fb38dd9d700 1 -- 172.21.15.55:0/1192395390 >> 172.21.15.55:0/1192395390 conn(0x7fb388087ff0 msgr2=0x7fb3880cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:00.343 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.341+0000 7fb38dd9d700 1 -- 172.21.15.55:0/1192395390 shutdown_connections 2023-10-06T03:33:00.343 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:00.341+0000 7fb38dd9d700 1 -- 172.21.15.55:0/1192395390 wait complete. 2023-10-06T03:33:00.357 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T03:33:00.397 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:33:00.396+0000 7fead2280180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:33:00.494 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:33:00.493+0000 7fead2280180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:33:00.508 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:33:00.506+0000 7f07ce0ee180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:33:00.608 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:33:00.607+0000 7f07ce0ee180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:33:01.495 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:33:01.494+0000 7fead2280180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:33:01.589 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:33:01.589+0000 7fead2280180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:33:01.648 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:33:01.647+0000 7f07ce0ee180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:33:01.683 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:33:01.682+0000 7fead2280180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:33:01.746 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:33:01.745+0000 7f07ce0ee180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:33:01.843 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:33:01.842+0000 7f07ce0ee180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:33:01.851 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:33:01.850+0000 7fead2280180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:33:01.957 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:33:01.956+0000 7fead2280180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:33:02.012 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:33:02.011+0000 7f07ce0ee180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:33:02.126 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:33:02.125+0000 7f07ce0ee180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:33:02.159 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:33:02.158+0000 7fead2280180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:33:02.266 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:33:02.265+0000 7fead2280180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:33:02.327 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:33:02.326+0000 7f07ce0ee180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:33:02.367 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:33:02.366+0000 7fead2280180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:33:02.435 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:33:02.434+0000 7f07ce0ee180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:33:02.470 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:33:02.469+0000 7fead2280180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:33:02.551 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:33:02.550+0000 7f07ce0ee180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:33:02.667 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:33:02.666+0000 7f07ce0ee180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:33:02.876 INFO:tasks.ceph.osd.6.smithi163.stderr:2023-10-06T03:33:02.876+0000 7f59796b7540 -1 osd.6 0 log_to_monitors true 2023-10-06T03:33:02.931 INFO:tasks.ceph.osd.7.smithi163.stderr:2023-10-06T03:33:02.930+0000 7fd3157d6540 -1 osd.7 0 log_to_monitors true 2023-10-06T03:33:03.107 INFO:tasks.ceph.osd.0.smithi055.stderr:2023-10-06T03:33:03.106+0000 7f92faddb540 -1 osd.0 0 log_to_monitors true 2023-10-06T03:33:03.176 INFO:tasks.ceph.osd.1.smithi055.stderr:2023-10-06T03:33:03.174+0000 7f157532d540 -1 osd.1 0 log_to_monitors true 2023-10-06T03:33:03.197 INFO:tasks.ceph.osd.5.smithi163.stderr:2023-10-06T03:33:03.197+0000 7f49494d2540 -1 osd.5 0 log_to_monitors true 2023-10-06T03:33:03.221 INFO:tasks.ceph.osd.4.smithi163.stderr:2023-10-06T03:33:03.221+0000 7fea277b8540 -1 osd.4 0 log_to_monitors true 2023-10-06T03:33:03.407 INFO:tasks.ceph.osd.2.smithi055.stderr:2023-10-06T03:33:03.406+0000 7fad6097a540 -1 osd.2 0 log_to_monitors true 2023-10-06T03:33:03.460 INFO:tasks.ceph.osd.3.smithi055.stderr:2023-10-06T03:33:03.459+0000 7f8431bc2540 -1 osd.3 0 log_to_monitors true 2023-10-06T03:33:03.493 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:33:03.492+0000 7fea8a3fa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T03:33:03.493 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:33:03.493+0000 7fea8a3fa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T03:33:06.440 INFO:tasks.ceph.osd.7.smithi163.stderr:2023-10-06T03:33:06.438+0000 7fd3091f6700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T03:33:06.448 INFO:tasks.ceph.osd.7.smithi163.stderr:2023-10-06T03:33:06.447+0000 7fd3021e8700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:33:06.825 INFO:tasks.ceph.osd.6.smithi163.stderr:2023-10-06T03:33:06.825+0000 7f596d1f6700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T03:33:06.838 INFO:tasks.ceph.osd.6.smithi163.stderr:2023-10-06T03:33:06.838+0000 7f59661e8700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:33:06.952 INFO:tasks.ceph.osd.3.smithi055.stderr:2023-10-06T03:33:06.951+0000 7f84255f6700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T03:33:06.974 INFO:tasks.ceph.osd.3.smithi055.stderr:2023-10-06T03:33:06.973+0000 7f841e5e8700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:33:06.993 INFO:tasks.ceph.osd.2.smithi055.stderr:2023-10-06T03:33:06.991+0000 7fad543f6700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T03:33:07.004 INFO:tasks.ceph.osd.2.smithi055.stderr:2023-10-06T03:33:07.003+0000 7fad4d3e8700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:33:07.005 INFO:tasks.ceph.osd.0.smithi055.stderr:2023-10-06T03:33:07.004+0000 7f92ee7f6700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T03:33:07.014 INFO:tasks.ceph.osd.0.smithi055.stderr:2023-10-06T03:33:07.013+0000 7f92e77e8700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:33:07.024 INFO:tasks.ceph.osd.1.smithi055.stderr:2023-10-06T03:33:07.023+0000 7f1568df6700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T03:33:07.034 INFO:tasks.ceph.osd.1.smithi055.stderr:2023-10-06T03:33:07.033+0000 7f1561de8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:33:07.082 INFO:tasks.ceph.osd.5.smithi163.stderr:2023-10-06T03:33:07.081+0000 7f493cff6700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T03:33:07.089 INFO:tasks.ceph.osd.5.smithi163.stderr:2023-10-06T03:33:07.088+0000 7f4935fe8700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:33:07.092 INFO:tasks.ceph.osd.4.smithi163.stderr:2023-10-06T03:33:07.091+0000 7fea1b1f6700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T03:33:07.098 INFO:tasks.ceph.osd.4.smithi163.stderr:2023-10-06T03:33:07.098+0000 7fea141e8700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T03:33:07.163 DEBUG:teuthology.orchestra.run.smithi055:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:07.325 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.323+0000 7fe23899d700 1 Processor -- start 2023-10-06T03:33:07.325 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.323+0000 7fe23899d700 1 -- start start 2023-10-06T03:33:07.328 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe23899d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe234157dd0 0x7fe2340aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:07.328 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe23899d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe234154310 0x7fe2340af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:07.328 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe23899d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe234157a20 con 0x7fe234153f40 2023-10-06T03:33:07.328 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe23899d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe234156c00 con 0x7fe234154310 2023-10-06T03:33:07.328 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe23899d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2340b23e0 con 0x7fe234157dd0 2023-10-06T03:33:07.328 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe233fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe234154310 0x7fe2340af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:07.328 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe2337fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe234153f40 0x7fe2340b1cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53468/0 (socket says 172.21.15.55:53468) 2023-10-06T03:33:07.329 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe2337fe700 1 -- 172.21.15.55:0/162147444 learned_addr learned my addr 172.21.15.55:0/162147444 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:07.329 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe233fff700 1 --2- 172.21.15.55:0/162147444 >> v2:172.21.15.55:3300/0 conn(0x7fe234154310 0x7fe2340af320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33930/0 (socket says 172.21.15.55:33930) 2023-10-06T03:33:07.329 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe233fff700 1 -- 172.21.15.55:0/162147444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe234157dd0 msgr2=0x7fe2340aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:07.329 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe232ffd700 1 --2- 172.21.15.55:0/162147444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe234157dd0 0x7fe2340aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:07.329 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe233fff700 1 --2- 172.21.15.55:0/162147444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe234157dd0 0x7fe2340aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:07.329 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe233fff700 1 -- 172.21.15.55:0/162147444 >> v1:172.21.15.55:6789/0 conn(0x7fe234153f40 legacy=0x7fe2340b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:07.329 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe233fff700 1 -- 172.21.15.55:0/162147444 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe234156e10 con 0x7fe234154310 2023-10-06T03:33:07.329 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe2327fc700 1 -- 172.21.15.55:0/162147444 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881003088 0 0) 0x7fe234157a20 con 0x7fe234153f40 2023-10-06T03:33:07.329 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.325+0000 7fe232ffd700 1 --2- 172.21.15.55:0/162147444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe234157dd0 0x7fe2340aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:07.329 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.326+0000 7fe233fff700 1 --2- 172.21.15.55:0/162147444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe234154310 0x7fe2340af320 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009e60 tx=0x7fe22c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75e0fd3a34ddf28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:07.329 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.326+0000 7fe2327fc700 1 -- 172.21.15.55:0/162147444 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c00bc30 con 0x7fe234154310 2023-10-06T03:33:07.330 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.326+0000 7fe2327fc700 1 -- 172.21.15.55:0/162147444 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22c00bd90 con 0x7fe234154310 2023-10-06T03:33:07.330 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.326+0000 7fe2327fc700 1 -- 172.21.15.55:0/162147444 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22c0116b0 con 0x7fe234154310 2023-10-06T03:33:07.330 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.326+0000 7fe23899d700 1 -- 172.21.15.55:0/162147444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe234154310 msgr2=0x7fe2340af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:07.330 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.326+0000 7fe23899d700 1 --2- 172.21.15.55:0/162147444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe234154310 0x7fe2340af320 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009e60 tx=0x7fe22c009780 comp rx=0 tx=0).stop 2023-10-06T03:33:07.330 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.327+0000 7fe23899d700 1 -- 172.21.15.55:0/162147444 shutdown_connections 2023-10-06T03:33:07.330 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.327+0000 7fe23899d700 1 --2- 172.21.15.55:0/162147444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe234154310 0x7fe2340af320 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:07.330 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.327+0000 7fe23899d700 1 --2- 172.21.15.55:0/162147444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe234157dd0 0x7fe2340aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:07.330 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.327+0000 7fe23899d700 1 -- 172.21.15.55:0/162147444 >> 172.21.15.55:0/162147444 conn(0x7fe23406d990 msgr2=0x7fe23406dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:07.330 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.327+0000 7fe23899d700 1 -- 172.21.15.55:0/162147444 shutdown_connections 2023-10-06T03:33:07.330 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.327+0000 7fe23899d700 1 -- 172.21.15.55:0/162147444 wait complete. 2023-10-06T03:33:07.333 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.328+0000 7fe23899d700 1 Processor -- start 2023-10-06T03:33:07.333 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.328+0000 7fe23899d700 1 -- start start 2023-10-06T03:33:07.333 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.329+0000 7fe23899d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe234153f40 0x7fe234167280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:07.333 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.329+0000 7fe23899d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe234154310 0x7fe23416a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:07.333 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.329+0000 7fe23899d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe234157dd0 0x7fe2340cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:07.333 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.329+0000 7fe23899d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2341677c0 con 0x7fe234153f40 2023-10-06T03:33:07.333 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.329+0000 7fe23899d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23416a5b0 con 0x7fe234154310 2023-10-06T03:33:07.333 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.329+0000 7fe23899d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23416ad10 con 0x7fe234157dd0 2023-10-06T03:33:07.333 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.329+0000 7fe2337fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe234153f40 0x7fe234167280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:07.334 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.329+0000 7fe2337fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe234153f40 0x7fe234167280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33944/0 (socket says 172.21.15.55:33944) 2023-10-06T03:33:07.334 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.329+0000 7fe2337fe700 1 -- 172.21.15.55:0/189508125 learned_addr learned my addr 172.21.15.55:0/189508125 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:07.334 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.330+0000 7fe232ffd700 1 --2- 172.21.15.55:0/189508125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe234154310 0x7fe23416a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:07.334 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.330+0000 7fe233fff700 1 --2- 172.21.15.55:0/189508125 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe234157dd0 0x7fe2340cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:07.334 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.330+0000 7fe2337fe700 1 -- 172.21.15.55:0/189508125 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe234157dd0 msgr2=0x7fe2340cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:07.334 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.330+0000 7fe2337fe700 1 --2- 172.21.15.55:0/189508125 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe234157dd0 0x7fe2340cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:07.334 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.330+0000 7fe2337fe700 1 -- 172.21.15.55:0/189508125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe234154310 msgr2=0x7fe23416a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:07.334 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.330+0000 7fe2337fe700 1 --2- 172.21.15.55:0/189508125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe234154310 0x7fe23416a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:07.335 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.330+0000 7fe2337fe700 1 -- 172.21.15.55:0/189508125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c009a20 con 0x7fe234153f40 2023-10-06T03:33:07.335 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.330+0000 7fe2337fe700 1 --2- 172.21.15.55:0/189508125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe234153f40 0x7fe234167280 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe22400c7e0 tx=0x7fe224007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:07.335 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.330+0000 7fe21b7fe700 1 -- 172.21.15.55:0/189508125 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224005b30 con 0x7fe234153f40 2023-10-06T03:33:07.335 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.330+0000 7fe21b7fe700 1 -- 172.21.15.55:0/189508125 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe224005c90 con 0x7fe234153f40 2023-10-06T03:33:07.335 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.330+0000 7fe21b7fe700 1 -- 172.21.15.55:0/189508125 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224007300 con 0x7fe234153f40 2023-10-06T03:33:07.335 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.330+0000 7fe23899d700 1 -- 172.21.15.55:0/189508125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23416aff0 con 0x7fe234153f40 2023-10-06T03:33:07.335 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.330+0000 7fe23899d700 1 -- 172.21.15.55:0/189508125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23416b5b0 con 0x7fe234153f40 2023-10-06T03:33:07.335 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.332+0000 7fe23899d700 1 -- 172.21.15.55:0/189508125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe234064b50 con 0x7fe234153f40 2023-10-06T03:33:07.336 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.332+0000 7fe21b7fe700 1 -- 172.21.15.55:0/189508125 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fe22401b400 con 0x7fe234153f40 2023-10-06T03:33:07.336 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.333+0000 7fe21b7fe700 1 --2- 172.21.15.55:0/189508125 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe220066ea0 0x7fe220069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:07.336 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.333+0000 7fe21b7fe700 1 -- 172.21.15.55:0/189508125 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7fe224083180 con 0x7fe234153f40 2023-10-06T03:33:07.337 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.333+0000 7fe232ffd700 1 --2- 172.21.15.55:0/189508125 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe220066ea0 0x7fe220069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:07.337 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.334+0000 7fe232ffd700 1 --2- 172.21.15.55:0/189508125 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe220066ea0 0x7fe220069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe234150390 tx=0x7fe228008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:07.339 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.338+0000 7fe21b7fe700 1 -- 172.21.15.55:0/189508125 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2240512e0 con 0x7fe234153f40 2023-10-06T03:33:07.551 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.549+0000 7fe23899d700 1 -- 172.21.15.55:0/189508125 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe23404bb80 con 0x7fe234153f40 2023-10-06T03:33:07.551 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.550+0000 7fe21b7fe700 1 -- 172.21.15.55:0/189508125 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10444 (secure 0 0 0) 0x7fe224054900 con 0x7fe234153f40 2023-10-06T03:33:07.552 INFO:teuthology.misc.health.smithi055.stdout: 2023-10-06T03:33:07.552 INFO:teuthology.misc.health.smithi055.stdout:{"epoch":13,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:07.443920+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:07.554 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.553+0000 7fe23899d700 1 -- 172.21.15.55:0/189508125 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe220066ea0 msgr2=0x7fe220069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:07.554 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.553+0000 7fe23899d700 1 --2- 172.21.15.55:0/189508125 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe220066ea0 0x7fe220069350 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe234150390 tx=0x7fe228008040 comp rx=0 tx=0).stop 2023-10-06T03:33:07.555 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.553+0000 7fe23899d700 1 -- 172.21.15.55:0/189508125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe234153f40 msgr2=0x7fe234167280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:07.555 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.553+0000 7fe23899d700 1 --2- 172.21.15.55:0/189508125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe234153f40 0x7fe234167280 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe22400c7e0 tx=0x7fe224007df0 comp rx=0 tx=0).stop 2023-10-06T03:33:07.555 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.553+0000 7fe23899d700 1 -- 172.21.15.55:0/189508125 shutdown_connections 2023-10-06T03:33:07.555 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.553+0000 7fe23899d700 1 --2- 172.21.15.55:0/189508125 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe220066ea0 0x7fe220069350 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:07.555 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.553+0000 7fe23899d700 1 --2- 172.21.15.55:0/189508125 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe234157dd0 0x7fe2340cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:07.555 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.553+0000 7fe23899d700 1 --2- 172.21.15.55:0/189508125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe234153f40 0x7fe234167280 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:07.555 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.553+0000 7fe23899d700 1 --2- 172.21.15.55:0/189508125 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe234154310 0x7fe23416a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:07.555 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.553+0000 7fe23899d700 1 -- 172.21.15.55:0/189508125 >> 172.21.15.55:0/189508125 conn(0x7fe23406d990 msgr2=0x7fe234159910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:07.555 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.553+0000 7fe23899d700 1 -- 172.21.15.55:0/189508125 shutdown_connections 2023-10-06T03:33:07.555 INFO:teuthology.misc.health.smithi055.stderr:2023-10-06T03:33:07.553+0000 7fe23899d700 1 -- 172.21.15.55:0/189508125 wait complete. 2023-10-06T03:33:07.571 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T03:33:07.571 INFO:tasks.ceph:Creating RBD pool 2023-10-06T03:33:07.571 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T03:33:07.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.752+0000 7f891359e700 1 Processor -- start 2023-10-06T03:33:07.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.752+0000 7f891359e700 1 -- start start 2023-10-06T03:33:07.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.753+0000 7f891359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f89141407e0 0x7f891406a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:07.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.753+0000 7f891359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89140c6820 con 0x7f891406dc00 2023-10-06T03:33:07.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.753+0000 7f891359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89140c6de0 con 0x7f8914140bb0 2023-10-06T03:33:07.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.753+0000 7f891359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8914141270 con 0x7f89141407e0 2023-10-06T03:33:07.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.753+0000 7f891259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f89141407e0 0x7f891406a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:07.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.753+0000 7f8912d9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f891406dc00 0x7f89140c7270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53480/0 (socket says 172.21.15.55:53480) 2023-10-06T03:33:07.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.753+0000 7f8912d9d700 1 -- 172.21.15.55:0/2764659866 learned_addr learned my addr 172.21.15.55:0/2764659866 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:07.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.754+0000 7f891259c700 1 -- 172.21.15.55:0/2764659866 >> v1:172.21.15.163:6789/0 conn(0x7f8914140bb0 legacy=0x7f891406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:07.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.754+0000 7f891259c700 1 -- 172.21.15.55:0/2764659866 >> v1:172.21.15.55:6789/0 conn(0x7f891406dc00 legacy=0x7f89140c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:07.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.754+0000 7f891259c700 1 -- 172.21.15.55:0/2764659866 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891406acf0 con 0x7f89141407e0 2023-10-06T03:33:07.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.754+0000 7f891159a700 1 -- 172.21.15.55:0/2764659866 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778824283 0 0) 0x7f89140c6820 con 0x7f891406dc00 2023-10-06T03:33:07.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.754+0000 7f891259c700 1 --2- 172.21.15.55:0/2764659866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89141407e0 0x7f891406a180 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f89040098e0 tx=0x7f89040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad881d50d05bab9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:07.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.754+0000 7f891159a700 1 -- 172.21.15.55:0/2764659866 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8904009e80 con 0x7f89141407e0 2023-10-06T03:33:07.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.754+0000 7f891159a700 1 -- 172.21.15.55:0/2764659866 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890400b820 con 0x7f89141407e0 2023-10-06T03:33:07.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.754+0000 7f891159a700 1 -- 172.21.15.55:0/2764659866 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890400bba0 con 0x7f89141407e0 2023-10-06T03:33:07.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.755+0000 7f891359e700 1 -- 172.21.15.55:0/2764659866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89141407e0 msgr2=0x7f891406a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:07.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.755+0000 7f891359e700 1 --2- 172.21.15.55:0/2764659866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89141407e0 0x7f891406a180 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f89040098e0 tx=0x7f89040091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:07.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.755+0000 7f891359e700 1 -- 172.21.15.55:0/2764659866 shutdown_connections 2023-10-06T03:33:07.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.755+0000 7f891359e700 1 --2- 172.21.15.55:0/2764659866 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89141407e0 0x7f891406a180 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:07.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.755+0000 7f891359e700 1 -- 172.21.15.55:0/2764659866 >> 172.21.15.55:0/2764659866 conn(0x7f89140cc1e0 msgr2=0x7f89140cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:07.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.755+0000 7f891359e700 1 -- 172.21.15.55:0/2764659866 shutdown_connections 2023-10-06T03:33:07.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.755+0000 7f891359e700 1 -- 172.21.15.55:0/2764659866 wait complete. 2023-10-06T03:33:07.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.756+0000 7f891359e700 1 Processor -- start 2023-10-06T03:33:07.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.757+0000 7f891359e700 1 -- start start 2023-10-06T03:33:07.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.757+0000 7f891359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f891406dc00 0x7f891413bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:07.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.757+0000 7f891359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89141407e0 0x7f891416a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:07.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f891359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8914140bb0 0x7f89140a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:07.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f891359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89140aa7d0 con 0x7f89141407e0 2023-10-06T03:33:07.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f891359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89140c5020 con 0x7f8914140bb0 2023-10-06T03:33:07.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f891359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891413c1b0 con 0x7f891406dc00 2023-10-06T03:33:07.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f8911d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89141407e0 0x7f891416a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:07.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f8911d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89141407e0 0x7f891416a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33962/0 (socket says 172.21.15.55:33962) 2023-10-06T03:33:07.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f8911d9b700 1 -- 172.21.15.55:0/1067189522 learned_addr learned my addr 172.21.15.55:0/1067189522 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:07.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f891259c700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f891406dc00 0x7f891413bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:07.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f8912d9d700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8914140bb0 0x7f89140a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:07.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f8911d9b700 1 -- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f891406dc00 msgr2=0x7f891413bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:07.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f8911d9b700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f891406dc00 0x7f891413bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:07.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f8911d9b700 1 -- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8914140bb0 msgr2=0x7f89140a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:07.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f8911d9b700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8914140bb0 0x7f89140a98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:07.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f8911d9b700 1 -- 172.21.15.55:0/1067189522 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8904009420 con 0x7f89141407e0 2023-10-06T03:33:07.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f891259c700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f891406dc00 0x7f891413bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:07.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f8912d9d700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8914140bb0 0x7f89140a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:07.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.758+0000 7f8911d9b700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89141407e0 0x7f891416a9d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f890800b610 tx=0x7f8908009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:07.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.759+0000 7f89027fc700 1 -- 172.21.15.55:0/1067189522 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890800f510 con 0x7f89141407e0 2023-10-06T03:33:07.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.759+0000 7f89027fc700 1 -- 172.21.15.55:0/1067189522 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890800f670 con 0x7f89141407e0 2023-10-06T03:33:07.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.759+0000 7f891359e700 1 -- 172.21.15.55:0/1067189522 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89140c4410 con 0x7f89141407e0 2023-10-06T03:33:07.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.759+0000 7f89027fc700 1 -- 172.21.15.55:0/1067189522 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890800fa30 con 0x7f89141407e0 2023-10-06T03:33:07.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.759+0000 7f891359e700 1 -- 172.21.15.55:0/1067189522 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89140c49d0 con 0x7f89141407e0 2023-10-06T03:33:07.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.760+0000 7f891359e700 1 -- 172.21.15.55:0/1067189522 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89140c51d0 con 0x7f89141407e0 2023-10-06T03:33:07.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.761+0000 7f89027fc700 1 -- 172.21.15.55:0/1067189522 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f890801a400 con 0x7f89141407e0 2023-10-06T03:33:07.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.762+0000 7f89027fc700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f88f4066ea0 0x7f88f4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:07.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.762+0000 7f89027fc700 1 -- 172.21.15.55:0/1067189522 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f8908083ae0 con 0x7f89141407e0 2023-10-06T03:33:07.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.762+0000 7f891259c700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f88f4066ea0 0x7f88f4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:07.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.763+0000 7f891259c700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f88f4066ea0 0x7f88f4069350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f89140cb900 tx=0x7f8904016000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:07.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.766+0000 7f89027fc700 1 -- 172.21.15.55:0/1067189522 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89140c51d0 con 0x7f89141407e0 2023-10-06T03:33:07.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:07.968+0000 7f891359e700 1 -- 172.21.15.55:0/1067189522 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f89140c51d0 con 0x7f89141407e0 2023-10-06T03:33:08.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.454+0000 7f89027fc700 1 -- 172.21.15.55:0/1067189522 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f89140c51d0 con 0x7f89141407e0 2023-10-06T03:33:08.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.458+0000 7f891359e700 1 -- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f88f4066ea0 msgr2=0x7f88f4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:08.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.458+0000 7f891359e700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f88f4066ea0 0x7f88f4069350 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f89140cb900 tx=0x7f8904016000 comp rx=0 tx=0).stop 2023-10-06T03:33:08.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.458+0000 7f891359e700 1 -- 172.21.15.55:0/1067189522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89141407e0 msgr2=0x7f891416a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:08.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.458+0000 7f891359e700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89141407e0 0x7f891416a9d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f890800b610 tx=0x7f8908009120 comp rx=0 tx=0).stop 2023-10-06T03:33:08.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.458+0000 7f891359e700 1 -- 172.21.15.55:0/1067189522 shutdown_connections 2023-10-06T03:33:08.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.458+0000 7f891359e700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8914140bb0 0x7f89140a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:08.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.458+0000 7f891359e700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f88f4066ea0 0x7f88f4069350 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:08.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.458+0000 7f891359e700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f891406dc00 0x7f891413bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:08.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.458+0000 7f891359e700 1 --2- 172.21.15.55:0/1067189522 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f89141407e0 0x7f891416a9d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:08.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.458+0000 7f891359e700 1 -- 172.21.15.55:0/1067189522 >> 172.21.15.55:0/1067189522 conn(0x7f89140cc1e0 msgr2=0x7f89140c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:08.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.458+0000 7f891359e700 1 -- 172.21.15.55:0/1067189522 shutdown_connections 2023-10-06T03:33:08.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.458+0000 7f891359e700 1 -- 172.21.15.55:0/1067189522 wait complete. 2023-10-06T03:33:08.461 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'rbd' created 2023-10-06T03:33:08.479 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T03:33:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.662+0000 7f16eb59d700 1 Processor -- start 2023-10-06T03:33:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.662+0000 7f16eb59d700 1 -- start start 2023-10-06T03:33:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.663+0000 7f16eb59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f16e40cb560 0x7f16e40cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.663+0000 7f16eb59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16e4069ec0 0x7f16e415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.663+0000 7f16eb59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e4159db0 con 0x7f16e40cc970 2023-10-06T03:33:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.663+0000 7f16eb59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e406aa30 con 0x7f16e40cb560 2023-10-06T03:33:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.663+0000 7f16eb59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e40cce10 con 0x7f16e4069ec0 2023-10-06T03:33:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.663+0000 7f16ea59b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f16e40cb560 0x7f16e40cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.663+0000 7f16ead9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16e4069ec0 0x7f16e415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.663+0000 7f16ea59b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f16e40cb560 0x7f16e40cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:41486/0 (socket says 172.21.15.55:41486) 2023-10-06T03:33:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.663+0000 7f16ea59b700 1 -- 172.21.15.55:0/542526744 learned_addr learned my addr 172.21.15.55:0/542526744 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.663+0000 7f16e9d9a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f16e40cc970 0x7f16e40697b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:56178/0 (socket says 172.21.15.55:56178) 2023-10-06T03:33:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.664+0000 7f16ea59b700 1 -- 172.21.15.55:0/542526744 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16e4069ec0 msgr2=0x7f16e415ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.664+0000 7f16ea59b700 1 --2- 172.21.15.55:0/542526744 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16e4069ec0 0x7f16e415ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.664+0000 7f16ea59b700 1 -- 172.21.15.55:0/542526744 >> v1:172.21.15.163:6789/0 conn(0x7f16e40cc970 legacy=0x7f16e40697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.664+0000 7f16ea59b700 1 -- 172.21.15.55:0/542526744 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e406a470 con 0x7f16e40cb560 2023-10-06T03:33:08.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.664+0000 7f16e9599700 1 -- 172.21.15.55:0/542526744 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462840480 0 0) 0x7f16e4159db0 con 0x7f16e40cc970 2023-10-06T03:33:08.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.664+0000 7f16ea59b700 1 --2- 172.21.15.55:0/542526744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16e40cb560 0x7f16e40cc430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f16e00099e0 tx=0x7f16e0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23f3b0c497e64e7a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.664+0000 7f16e9599700 1 -- 172.21.15.55:0/542526744 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e000b820 con 0x7f16e40cb560 2023-10-06T03:33:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.665+0000 7f16e9599700 1 -- 172.21.15.55:0/542526744 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e000b980 con 0x7f16e40cb560 2023-10-06T03:33:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.665+0000 7f16e9599700 1 -- 172.21.15.55:0/542526744 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e000bd20 con 0x7f16e40cb560 2023-10-06T03:33:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.665+0000 7f16eb59d700 1 -- 172.21.15.55:0/542526744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16e40cb560 msgr2=0x7f16e40cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.665+0000 7f16eb59d700 1 --2- 172.21.15.55:0/542526744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16e40cb560 0x7f16e40cc430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f16e00099e0 tx=0x7f16e0009250 comp rx=0 tx=0).stop 2023-10-06T03:33:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.665+0000 7f16eb59d700 1 -- 172.21.15.55:0/542526744 shutdown_connections 2023-10-06T03:33:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.665+0000 7f16eb59d700 1 --2- 172.21.15.55:0/542526744 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16e4069ec0 0x7f16e415ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.665+0000 7f16eb59d700 1 --2- 172.21.15.55:0/542526744 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16e40cb560 0x7f16e40cc430 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.665+0000 7f16eb59d700 1 -- 172.21.15.55:0/542526744 >> 172.21.15.55:0/542526744 conn(0x7f16e406baf0 msgr2=0x7f16e406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.665+0000 7f16eb59d700 1 -- 172.21.15.55:0/542526744 shutdown_connections 2023-10-06T03:33:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.665+0000 7f16eb59d700 1 -- 172.21.15.55:0/542526744 wait complete. 2023-10-06T03:33:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.667+0000 7f16eb59d700 1 Processor -- start 2023-10-06T03:33:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.667+0000 7f16eb59d700 1 -- start start 2023-10-06T03:33:08.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.668+0000 7f16eb59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16e4069ec0 0x7f16e4150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:08.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.668+0000 7f16eb59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16e40cb560 0x7f16e414ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:08.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.668+0000 7f16eb59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16e40cc970 0x7f16e4153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:08.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.668+0000 7f16eb59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e41577a0 con 0x7f16e40cc970 2023-10-06T03:33:08.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.668+0000 7f16eb59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e40b2b60 con 0x7f16e40cb560 2023-10-06T03:33:08.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.668+0000 7f16eb59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e4151400 con 0x7f16e4069ec0 2023-10-06T03:33:08.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.668+0000 7f16ead9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16e40cc970 0x7f16e4153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:08.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.668+0000 7f16ea59b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16e4069ec0 0x7f16e4150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:08.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.668+0000 7f16ead9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16e40cc970 0x7f16e4153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:33964/0 (socket says 172.21.15.55:33964) 2023-10-06T03:33:08.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.668+0000 7f16ead9c700 1 -- 172.21.15.55:0/1478180998 learned_addr learned my addr 172.21.15.55:0/1478180998 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:08.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.668+0000 7f16e9d9a700 1 --2- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16e40cb560 0x7f16e414ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.669+0000 7f16ea59b700 1 -- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16e40cb560 msgr2=0x7f16e414ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.669+0000 7f16ea59b700 1 --2- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16e40cb560 0x7f16e414ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.669+0000 7f16ea59b700 1 -- 172.21.15.55:0/1478180998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16e40cc970 msgr2=0x7f16e4153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.669+0000 7f16ea59b700 1 --2- 172.21.15.55:0/1478180998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16e40cc970 0x7f16e4153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.669+0000 7f16ea59b700 1 -- 172.21.15.55:0/1478180998 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e00094f0 con 0x7f16e4069ec0 2023-10-06T03:33:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.669+0000 7f16e9d9a700 1 --2- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16e40cb560 0x7f16e414ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.669+0000 7f16ea59b700 1 --2- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16e4069ec0 0x7f16e4150ec0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f16e00099b0 tx=0x7f16e0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.669+0000 7f16da7fc700 1 -- 172.21.15.55:0/1478180998 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e0011d70 con 0x7f16e4069ec0 2023-10-06T03:33:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.669+0000 7f16da7fc700 1 -- 172.21.15.55:0/1478180998 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e0024440 con 0x7f16e4069ec0 2023-10-06T03:33:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.669+0000 7f16da7fc700 1 -- 172.21.15.55:0/1478180998 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e00247e0 con 0x7f16e4069ec0 2023-10-06T03:33:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.669+0000 7f16eb59d700 1 -- 172.21.15.55:0/1478180998 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e40b2810 con 0x7f16e4069ec0 2023-10-06T03:33:08.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.670+0000 7f16eb59d700 1 -- 172.21.15.55:0/1478180998 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e40cedd0 con 0x7f16e4069ec0 2023-10-06T03:33:08.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.670+0000 7f16eb59d700 1 -- 172.21.15.55:0/1478180998 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e404c3c0 con 0x7f16e4069ec0 2023-10-06T03:33:08.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.671+0000 7f16da7fc700 1 -- 172.21.15.55:0/1478180998 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f16e002d440 con 0x7f16e4069ec0 2023-10-06T03:33:08.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.672+0000 7f16da7fc700 1 --2- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f16c8066ea0 0x7f16c8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:08.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.672+0000 7f16da7fc700 1 -- 172.21.15.55:0/1478180998 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4809+0+0 (secure 0 0 0) 0x7f16e0085ab0 con 0x7f16e4069ec0 2023-10-06T03:33:08.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.673+0000 7f16e9d9a700 1 --2- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f16c8066ea0 0x7f16c8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:08.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.673+0000 7f16e9d9a700 1 --2- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f16c8066ea0 0x7f16c8069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f16e406b060 tx=0x7f16d4007ef0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:08.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.678+0000 7f16da7fc700 1 -- 172.21.15.55:0/1478180998 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16e00530e0 con 0x7f16e4069ec0 2023-10-06T03:33:08.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:08.881+0000 7f16eb59d700 1 -- 172.21.15.55:0/1478180998 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f16e406a750 con 0x7f16e4069ec0 2023-10-06T03:33:09.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.465+0000 7f16da7fc700 1 -- 172.21.15.55:0/1478180998 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f16e0056700 con 0x7f16e4069ec0 2023-10-06T03:33:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.468+0000 7f16eb59d700 1 -- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f16c8066ea0 msgr2=0x7f16c8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.468+0000 7f16eb59d700 1 --2- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f16c8066ea0 0x7f16c8069350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f16e406b060 tx=0x7f16d4007ef0 comp rx=0 tx=0).stop 2023-10-06T03:33:09.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.468+0000 7f16eb59d700 1 -- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16e4069ec0 msgr2=0x7f16e4150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:09.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.468+0000 7f16eb59d700 1 --2- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16e4069ec0 0x7f16e4150ec0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f16e00099b0 tx=0x7f16e0017000 comp rx=0 tx=0).stop 2023-10-06T03:33:09.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.469+0000 7f16eb59d700 1 -- 172.21.15.55:0/1478180998 shutdown_connections 2023-10-06T03:33:09.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.469+0000 7f16eb59d700 1 --2- 172.21.15.55:0/1478180998 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16e40cc970 0x7f16e4153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:09.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.469+0000 7f16eb59d700 1 --2- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f16c8066ea0 0x7f16c8069350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:09.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.469+0000 7f16eb59d700 1 --2- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16e4069ec0 0x7f16e4150ec0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:09.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.469+0000 7f16eb59d700 1 --2- 172.21.15.55:0/1478180998 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16e40cb560 0x7f16e414ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:09.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.469+0000 7f16eb59d700 1 -- 172.21.15.55:0/1478180998 >> 172.21.15.55:0/1478180998 conn(0x7f16e406baf0 msgr2=0x7f16e41579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:09.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.469+0000 7f16eb59d700 1 -- 172.21.15.55:0/1478180998 shutdown_connections 2023-10-06T03:33:09.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.469+0000 7f16eb59d700 1 -- 172.21.15.55:0/1478180998 wait complete. 2023-10-06T03:33:09.472 INFO:teuthology.orchestra.run.smithi055.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T03:33:09.491 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T03:33:09.492 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:33:09.492 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:33:09.496 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:33:09.496 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:33:09.496 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:33:09.501 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:33:09.501 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:33:09.501 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:33:09.506 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:33:09.507 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:33:09.507 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:33:09.511 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:33:09.511 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T03:33:09.511 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:09.642 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-10-06T03:33:09.643 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T03:33:09.650 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2023-10-06T03:33:09.651 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T03:33:09.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.714+0000 7faeb159d700 1 Processor -- start 2023-10-06T03:33:09.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.714+0000 7faeb159d700 1 -- start start 2023-10-06T03:33:09.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.716+0000 7faeb159d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeac1597f0 0x7faeac0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:09.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.716+0000 7faeb159d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faeac15d230 0x7faeac15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:09.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.716+0000 7faeb159d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeac15dbd0 con 0x7faeac159420 2023-10-06T03:33:09.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.716+0000 7faeb159d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeac0b11d0 con 0x7faeac15d230 2023-10-06T03:33:09.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.716+0000 7faeb159d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0b14b0 con 0x7faeac1597f0 2023-10-06T03:33:09.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.716+0000 7faeb0d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faeac15d230 0x7faeac15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:09.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.716+0000 7faeb0d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faeac15d230 0x7faeac15d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34002/0 (socket says 172.21.15.55:34002) 2023-10-06T03:33:09.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.716+0000 7faeb0d9c700 1 -- 172.21.15.55:0/4210283533 learned_addr learned my addr 172.21.15.55:0/4210283533 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:09.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.716+0000 7faeab7fe700 1 --2- 172.21.15.55:0/4210283533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeac1597f0 0x7faeac0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:09.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.716+0000 7faeabfff700 1 --1- 172.21.15.55:0/4210283533 >> v1:172.21.15.163:6789/0 conn(0x7faeac159420 0x7faeac0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:56196/0 (socket says 172.21.15.55:56196) 2023-10-06T03:33:09.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.716+0000 7faeb0d9c700 1 -- 172.21.15.55:0/4210283533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeac1597f0 msgr2=0x7faeac0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:09.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.716+0000 7faeb0d9c700 1 --2- 172.21.15.55:0/4210283533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeac1597f0 0x7faeac0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:09.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.716+0000 7faeb0d9c700 1 -- 172.21.15.55:0/4210283533 >> v1:172.21.15.163:6789/0 conn(0x7faeac159420 legacy=0x7faeac0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:09.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.716+0000 7faeb0d9c700 1 -- 172.21.15.55:0/4210283533 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac0b1790 con 0x7faeac15d230 2023-10-06T03:33:09.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.717+0000 7faeb0d9c700 1 --2- 172.21.15.55:0/4210283533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeac15d230 0x7faeac15d690 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faea4009f20 tx=0x7faea4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e1c4c9b0f9bba8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:09.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.717+0000 7faeaaffd700 1 -- 172.21.15.55:0/4210283533 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea400e070 con 0x7faeac15d230 2023-10-06T03:33:09.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.717+0000 7faeaaffd700 1 -- 172.21.15.55:0/4210283533 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea400b9d0 con 0x7faeac15d230 2023-10-06T03:33:09.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.717+0000 7faeaaffd700 1 -- 172.21.15.55:0/4210283533 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea40092e0 con 0x7faeac15d230 2023-10-06T03:33:09.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.717+0000 7faeb159d700 1 -- 172.21.15.55:0/4210283533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeac15d230 msgr2=0x7faeac15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:09.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.717+0000 7faeb159d700 1 --2- 172.21.15.55:0/4210283533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeac15d230 0x7faeac15d690 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faea4009f20 tx=0x7faea4009840 comp rx=0 tx=0).stop 2023-10-06T03:33:09.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.717+0000 7faeb159d700 1 -- 172.21.15.55:0/4210283533 shutdown_connections 2023-10-06T03:33:09.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.717+0000 7faeb159d700 1 --2- 172.21.15.55:0/4210283533 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeac15d230 0x7faeac15d690 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:09.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.717+0000 7faeb159d700 1 --2- 172.21.15.55:0/4210283533 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeac1597f0 0x7faeac0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:09.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.717+0000 7faeb159d700 1 -- 172.21.15.55:0/4210283533 >> 172.21.15.55:0/4210283533 conn(0x7faeac0d0660 msgr2=0x7faeac0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:09.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.717+0000 7faeb159d700 1 -- 172.21.15.55:0/4210283533 shutdown_connections 2023-10-06T03:33:09.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.717+0000 7faeb159d700 1 -- 172.21.15.55:0/4210283533 wait complete. 2023-10-06T03:33:09.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.719+0000 7faeb159d700 1 Processor -- start 2023-10-06T03:33:09.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.719+0000 7faeb159d700 1 -- start start 2023-10-06T03:33:09.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeb159d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeac159420 0x7faeac0cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:09.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeb159d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeac1597f0 0x7faeac0c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:09.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeb159d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeac15d230 0x7faeac157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:09.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeb159d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0cbc40 con 0x7faeac15d230 2023-10-06T03:33:09.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeb159d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0cc380 con 0x7faeac1597f0 2023-10-06T03:33:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeb159d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeac0c8a00 con 0x7faeac159420 2023-10-06T03:33:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeb0d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeac15d230 0x7faeac157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeb0d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeac15d230 0x7faeac157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34006/0 (socket says 172.21.15.55:34006) 2023-10-06T03:33:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeb0d9c700 1 -- 172.21.15.55:0/2615832330 learned_addr learned my addr 172.21.15.55:0/2615832330 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeabfff700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeac159420 0x7faeac0cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeab7fe700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeac1597f0 0x7faeac0c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeb0d9c700 1 -- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeac159420 msgr2=0x7faeac0cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeb0d9c700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeac159420 0x7faeac0cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeb0d9c700 1 -- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeac1597f0 msgr2=0x7faeac0c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeb0d9c700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeac1597f0 0x7faeac0c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeb0d9c700 1 -- 172.21.15.55:0/2615832330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea4009ae0 con 0x7faeac15d230 2023-10-06T03:33:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.720+0000 7faeab7fe700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeac1597f0 0x7faeac0c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:09.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.721+0000 7faeabfff700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeac159420 0x7faeac0cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:09.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.721+0000 7faeb0d9c700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeac15d230 0x7faeac157430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faea400b610 tx=0x7faea4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:09.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.721+0000 7fae93fff700 1 -- 172.21.15.55:0/2615832330 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4008c30 con 0x7faeac15d230 2023-10-06T03:33:09.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.721+0000 7fae93fff700 1 -- 172.21.15.55:0/2615832330 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea4008d90 con 0x7faeac15d230 2023-10-06T03:33:09.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.721+0000 7faeb159d700 1 -- 172.21.15.55:0/2615832330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeac0c8030 con 0x7faeac15d230 2023-10-06T03:33:09.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.721+0000 7fae93fff700 1 -- 172.21.15.55:0/2615832330 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4025690 con 0x7faeac15d230 2023-10-06T03:33:09.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.721+0000 7faeb159d700 1 -- 172.21.15.55:0/2615832330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeac0c79e0 con 0x7faeac15d230 2023-10-06T03:33:09.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.722+0000 7faeb159d700 1 -- 172.21.15.55:0/2615832330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeac0c81c0 con 0x7faeac15d230 2023-10-06T03:33:09.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.723+0000 7fae93fff700 1 -- 172.21.15.55:0/2615832330 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7faea4003040 con 0x7faeac15d230 2023-10-06T03:33:09.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.724+0000 7fae93fff700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fae98066ea0 0x7fae98069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:09.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.725+0000 7fae93fff700 1 -- 172.21.15.55:0/2615832330 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7faea4085f20 con 0x7faeac15d230 2023-10-06T03:33:09.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.725+0000 7faeabfff700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fae98066ea0 0x7fae98069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:09.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.725+0000 7faeabfff700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fae98066ea0 0x7fae98069350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fae9c007fd0 tx=0x7fae9c007e10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:09.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.730+0000 7fae93fff700 1 -- 172.21.15.55:0/2615832330 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faea40533d0 con 0x7faeac15d230 2023-10-06T03:33:09.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.934+0000 7faeb159d700 1 -- 172.21.15.55:0/2615832330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faeac0b1a70 con 0x7faeac15d230 2023-10-06T03:33:09.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.935+0000 7fae93fff700 1 -- 172.21.15.55:0/2615832330 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+13714 (secure 0 0 0) 0x7faea40569f0 con 0x7faeac15d230 2023-10-06T03:33:09.937 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:09.937 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":15,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:09.448931+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:09.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.937+0000 7faeb159d700 1 -- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fae98066ea0 msgr2=0x7fae98069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:09.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.937+0000 7faeb159d700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fae98066ea0 0x7fae98069350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fae9c007fd0 tx=0x7fae9c007e10 comp rx=0 tx=0).stop 2023-10-06T03:33:09.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.937+0000 7faeb159d700 1 -- 172.21.15.55:0/2615832330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeac15d230 msgr2=0x7faeac157430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:09.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.937+0000 7faeb159d700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeac15d230 0x7faeac157430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faea400b610 tx=0x7faea4018000 comp rx=0 tx=0).stop 2023-10-06T03:33:09.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.937+0000 7faeb159d700 1 -- 172.21.15.55:0/2615832330 shutdown_connections 2023-10-06T03:33:09.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.937+0000 7faeb159d700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faeac15d230 0x7faeac157430 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:09.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.937+0000 7faeb159d700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fae98066ea0 0x7fae98069350 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:09.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.937+0000 7faeb159d700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faeac159420 0x7faeac0cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:09.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.937+0000 7faeb159d700 1 --2- 172.21.15.55:0/2615832330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faeac1597f0 0x7faeac0c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:09.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.937+0000 7faeb159d700 1 -- 172.21.15.55:0/2615832330 >> 172.21.15.55:0/2615832330 conn(0x7faeac0d0660 msgr2=0x7faeac158360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:09.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.938+0000 7faeb159d700 1 -- 172.21.15.55:0/2615832330 shutdown_connections 2023-10-06T03:33:09.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:09.938+0000 7faeb159d700 1 -- 172.21.15.55:0/2615832330 wait complete. 2023-10-06T03:33:09.956 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}}] 2023-10-06T03:33:09.957 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:33:10.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.146+0000 7f5c56b9d700 1 Processor -- start 2023-10-06T03:33:10.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.146+0000 7f5c56b9d700 1 -- start start 2023-10-06T03:33:10.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.147+0000 7f5c56b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c500ccc00 0x7f5c50069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.147+0000 7f5c56b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5c5006a020 0x7f5c500cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.147+0000 7f5c56b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c500c9f30 con 0x7f5c50159a60 2023-10-06T03:33:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.147+0000 7f5c56b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c5006dfc0 con 0x7f5c5006a020 2023-10-06T03:33:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.147+0000 7f5c56b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c500ca470 con 0x7f5c500ccc00 2023-10-06T03:33:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.147+0000 7f5c55b9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5c50159a60 0x7f5c500c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53484/0 (socket says 172.21.15.55:53484) 2023-10-06T03:33:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.147+0000 7f5c55b9b700 1 -- 172.21.15.55:0/1793351745 learned_addr learned my addr 172.21.15.55:0/1793351745 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.147+0000 7f5c5639c700 1 --2- 172.21.15.55:0/1793351745 >> v2:172.21.15.55:3300/0 conn(0x7f5c5006a020 0x7f5c500cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.148+0000 7f5c5539a700 1 --2- 172.21.15.55:0/1793351745 >> v2:172.21.15.163:3300/0 conn(0x7f5c500ccc00 0x7f5c50069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.148+0000 7f5c54b99700 1 -- 172.21.15.55:0/1793351745 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974242101 0 0) 0x7f5c500c9f30 con 0x7f5c50159a60 2023-10-06T03:33:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.148+0000 7f5c54b99700 1 -- 172.21.15.55:0/1793351745 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c3c003570 con 0x7f5c50159a60 2023-10-06T03:33:10.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.148+0000 7f5c5639c700 1 -- 172.21.15.55:0/1793351745 >> v2:172.21.15.163:3300/0 conn(0x7f5c500ccc00 msgr2=0x7f5c50069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:10.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.148+0000 7f5c5639c700 1 --2- 172.21.15.55:0/1793351745 >> v2:172.21.15.163:3300/0 conn(0x7f5c500ccc00 0x7f5c50069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.148+0000 7f5c5639c700 1 -- 172.21.15.55:0/1793351745 >> v1:172.21.15.55:6789/0 conn(0x7f5c50159a60 legacy=0x7f5c500c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:10.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.148+0000 7f5c5639c700 1 -- 172.21.15.55:0/1793351745 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c500ca750 con 0x7f5c5006a020 2023-10-06T03:33:10.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.148+0000 7f5c5639c700 1 --2- 172.21.15.55:0/1793351745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c5006a020 0x7f5c500cb230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5c500cf6a0 tx=0x7f5c48009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537257c521f88a23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:10.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.148+0000 7f5c54b99700 1 -- 172.21.15.55:0/1793351745 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4800e070 con 0x7f5c5006a020 2023-10-06T03:33:10.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.148+0000 7f5c54b99700 1 -- 172.21.15.55:0/1793351745 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4800bb40 con 0x7f5c5006a020 2023-10-06T03:33:10.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.148+0000 7f5c54b99700 1 -- 172.21.15.55:0/1793351745 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c48012430 con 0x7f5c5006a020 2023-10-06T03:33:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.149+0000 7f5c56b9d700 1 -- 172.21.15.55:0/1793351745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c5006a020 msgr2=0x7f5c500cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.149+0000 7f5c56b9d700 1 --2- 172.21.15.55:0/1793351745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c5006a020 0x7f5c500cb230 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5c500cf6a0 tx=0x7f5c48009780 comp rx=0 tx=0).stop 2023-10-06T03:33:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.149+0000 7f5c56b9d700 1 -- 172.21.15.55:0/1793351745 shutdown_connections 2023-10-06T03:33:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.149+0000 7f5c56b9d700 1 --2- 172.21.15.55:0/1793351745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c5006a020 0x7f5c500cb230 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.149+0000 7f5c56b9d700 1 --2- 172.21.15.55:0/1793351745 >> v2:172.21.15.163:3300/0 conn(0x7f5c500ccc00 0x7f5c50069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.149+0000 7f5c56b9d700 1 -- 172.21.15.55:0/1793351745 >> 172.21.15.55:0/1793351745 conn(0x7f5c5006baa0 msgr2=0x7f5c5006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.149+0000 7f5c56b9d700 1 -- 172.21.15.55:0/1793351745 shutdown_connections 2023-10-06T03:33:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.149+0000 7f5c56b9d700 1 -- 172.21.15.55:0/1793351745 wait complete. 2023-10-06T03:33:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.150+0000 7f5c56b9d700 1 Processor -- start 2023-10-06T03:33:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.151+0000 7f5c56b9d700 1 -- start start 2023-10-06T03:33:10.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c56b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c5006a020 0x7f5c501462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:10.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c56b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c500ccc00 0x7f5c501570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:10.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c56b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c50159a60 0x7f5c5016a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:10.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c56b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5016ac50 con 0x7f5c5006a020 2023-10-06T03:33:10.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c56b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c501577e0 con 0x7f5c500ccc00 2023-10-06T03:33:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c56b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c500b2f40 con 0x7f5c50159a60 2023-10-06T03:33:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c55b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c5006a020 0x7f5c501462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c55b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c5006a020 0x7f5c501462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34032/0 (socket says 172.21.15.55:34032) 2023-10-06T03:33:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c55b9b700 1 -- 172.21.15.55:0/3293588903 learned_addr learned my addr 172.21.15.55:0/3293588903 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c5539a700 1 --2- 172.21.15.55:0/3293588903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c500ccc00 0x7f5c501570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c5639c700 1 --2- 172.21.15.55:0/3293588903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c50159a60 0x7f5c5016a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c55b9b700 1 -- 172.21.15.55:0/3293588903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c50159a60 msgr2=0x7f5c5016a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c55b9b700 1 --2- 172.21.15.55:0/3293588903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c50159a60 0x7f5c5016a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c55b9b700 1 -- 172.21.15.55:0/3293588903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c500ccc00 msgr2=0x7f5c501570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c55b9b700 1 --2- 172.21.15.55:0/3293588903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c500ccc00 0x7f5c501570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.152+0000 7f5c55b9b700 1 -- 172.21.15.55:0/3293588903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c48009a20 con 0x7f5c5006a020 2023-10-06T03:33:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.153+0000 7f5c55b9b700 1 --2- 172.21.15.55:0/3293588903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c5006a020 0x7f5c501462d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00c590 tx=0x7f5c4c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.153+0000 7f5c45ffb700 1 -- 172.21.15.55:0/3293588903 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c005b30 con 0x7f5c5006a020 2023-10-06T03:33:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.153+0000 7f5c45ffb700 1 -- 172.21.15.55:0/3293588903 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4c005cb0 con 0x7f5c5006a020 2023-10-06T03:33:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.153+0000 7f5c45ffb700 1 -- 172.21.15.55:0/3293588903 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c007320 con 0x7f5c5006a020 2023-10-06T03:33:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.153+0000 7f5c56b9d700 1 -- 172.21.15.55:0/3293588903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c500b3220 con 0x7f5c5006a020 2023-10-06T03:33:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.153+0000 7f5c56b9d700 1 -- 172.21.15.55:0/3293588903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c500b2bf0 con 0x7f5c5006a020 2023-10-06T03:33:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.154+0000 7f5c56b9d700 1 -- 172.21.15.55:0/3293588903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c500b3740 con 0x7f5c5006a020 2023-10-06T03:33:10.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.155+0000 7f5c45ffb700 1 -- 172.21.15.55:0/3293588903 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f5c4c01b3b0 con 0x7f5c5006a020 2023-10-06T03:33:10.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.156+0000 7f5c45ffb700 1 --2- 172.21.15.55:0/3293588903 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c34066ea0 0x7f5c34069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:10.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.156+0000 7f5c45ffb700 1 -- 172.21.15.55:0/3293588903 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f5c4c083730 con 0x7f5c5006a020 2023-10-06T03:33:10.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.156+0000 7f5c5539a700 1 --2- 172.21.15.55:0/3293588903 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c34066ea0 0x7f5c34069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:10.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.157+0000 7f5c5539a700 1 --2- 172.21.15.55:0/3293588903 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c34066ea0 0x7f5c34069350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c500c8740 tx=0x7f5c400091a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:10.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.160+0000 7f5c45ffb700 1 -- 172.21.15.55:0/3293588903 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c500b3740 con 0x7f5c5006a020 2023-10-06T03:33:10.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.365+0000 7f5c56b9d700 1 -- 172.21.15.55:0/3293588903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5c500b3740 con 0x7f5c5006a020 2023-10-06T03:33:10.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.365+0000 7f5c45ffb700 1 -- 172.21.15.55:0/3293588903 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5c4c017020 con 0x7f5c5006a020 2023-10-06T03:33:10.367 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:33:10.369 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.368+0000 7f5c56b9d700 1 -- 172.21.15.55:0/3293588903 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c34066ea0 msgr2=0x7f5c34069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:10.369 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.368+0000 7f5c56b9d700 1 --2- 172.21.15.55:0/3293588903 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c34066ea0 0x7f5c34069350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c500c8740 tx=0x7f5c400091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.369 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.368+0000 7f5c56b9d700 1 -- 172.21.15.55:0/3293588903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c5006a020 msgr2=0x7f5c501462d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:10.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.368+0000 7f5c56b9d700 1 --2- 172.21.15.55:0/3293588903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c5006a020 0x7f5c501462d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00c590 tx=0x7f5c4c007e00 comp rx=0 tx=0).stop 2023-10-06T03:33:10.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.368+0000 7f5c56b9d700 1 -- 172.21.15.55:0/3293588903 shutdown_connections 2023-10-06T03:33:10.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.368+0000 7f5c56b9d700 1 --2- 172.21.15.55:0/3293588903 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c34066ea0 0x7f5c34069350 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.368+0000 7f5c56b9d700 1 --2- 172.21.15.55:0/3293588903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c50159a60 0x7f5c5016a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.368+0000 7f5c56b9d700 1 --2- 172.21.15.55:0/3293588903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c5006a020 0x7f5c501462d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.368+0000 7f5c56b9d700 1 --2- 172.21.15.55:0/3293588903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c500ccc00 0x7f5c501570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.368+0000 7f5c56b9d700 1 -- 172.21.15.55:0/3293588903 >> 172.21.15.55:0/3293588903 conn(0x7f5c5006baa0 msgr2=0x7f5c501540c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:10.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.368+0000 7f5c56b9d700 1 -- 172.21.15.55:0/3293588903 shutdown_connections 2023-10-06T03:33:10.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.368+0000 7f5c56b9d700 1 -- 172.21.15.55:0/3293588903 wait complete. 2023-10-06T03:33:10.387 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:33:10.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.589+0000 7f23da59d700 1 Processor -- start 2023-10-06T03:33:10.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.590+0000 7f23da59d700 1 -- start start 2023-10-06T03:33:10.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.591+0000 7f23da59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23d4157a50 0x7f23d40c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:10.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.591+0000 7f23da59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23d40cb590 0x7f23d40c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:10.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.591+0000 7f23da59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d40cd240 con 0x7f23d4157680 2023-10-06T03:33:10.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.591+0000 7f23da59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d40cc700 con 0x7f23d4157a50 2023-10-06T03:33:10.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.591+0000 7f23da59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40c9a20 con 0x7f23d40cb590 2023-10-06T03:33:10.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.591+0000 7f23d959b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f23d4157680 0x7f23d40cae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53488/0 (socket says 172.21.15.55:53488) 2023-10-06T03:33:10.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.591+0000 7f23d959b700 1 -- 172.21.15.55:0/3181409594 learned_addr learned my addr 172.21.15.55:0/3181409594 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:10.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.591+0000 7f23d8d9a700 1 --2- 172.21.15.55:0/3181409594 >> v2:172.21.15.163:3300/0 conn(0x7f23d4157a50 0x7f23d40c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:10.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.591+0000 7f23d9d9c700 1 --2- 172.21.15.55:0/3181409594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23d40cb590 0x7f23d40c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:10.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.591+0000 7f23cbfff700 1 -- 172.21.15.55:0/3181409594 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1113605770 0 0) 0x7f23d40cd240 con 0x7f23d4157680 2023-10-06T03:33:10.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.592+0000 7f23cbfff700 1 -- 172.21.15.55:0/3181409594 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23c0003570 con 0x7f23d4157680 2023-10-06T03:33:10.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.592+0000 7f23d8d9a700 1 -- 172.21.15.55:0/3181409594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23d40cb590 msgr2=0x7f23d40c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:10.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.592+0000 7f23d8d9a700 1 --2- 172.21.15.55:0/3181409594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23d40cb590 0x7f23d40c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.592+0000 7f23d8d9a700 1 -- 172.21.15.55:0/3181409594 >> v1:172.21.15.55:6789/0 conn(0x7f23d4157680 legacy=0x7f23d40cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:10.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.592+0000 7f23d8d9a700 1 -- 172.21.15.55:0/3181409594 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d40c9750 con 0x7f23d4157a50 2023-10-06T03:33:10.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.592+0000 7f23d8d9a700 1 --2- 172.21.15.55:0/3181409594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d4157a50 0x7f23d40c8cd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f23c40098e0 tx=0x7f23c40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=236faf576502dbcb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:10.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.592+0000 7f23cbfff700 1 -- 172.21.15.55:0/3181409594 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4011630 con 0x7f23d4157a50 2023-10-06T03:33:10.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.592+0000 7f23cbfff700 1 -- 172.21.15.55:0/3181409594 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c4011790 con 0x7f23d4157a50 2023-10-06T03:33:10.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.592+0000 7f23cbfff700 1 -- 172.21.15.55:0/3181409594 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4011b10 con 0x7f23d4157a50 2023-10-06T03:33:10.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.593+0000 7f23da59d700 1 -- 172.21.15.55:0/3181409594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d4157a50 msgr2=0x7f23d40c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:10.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.593+0000 7f23da59d700 1 --2- 172.21.15.55:0/3181409594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d4157a50 0x7f23d40c8cd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f23c40098e0 tx=0x7f23c40091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.593+0000 7f23da59d700 1 -- 172.21.15.55:0/3181409594 shutdown_connections 2023-10-06T03:33:10.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.593+0000 7f23da59d700 1 --2- 172.21.15.55:0/3181409594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23d40cb590 0x7f23d40c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.593+0000 7f23da59d700 1 --2- 172.21.15.55:0/3181409594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d4157a50 0x7f23d40c8cd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.593+0000 7f23da59d700 1 -- 172.21.15.55:0/3181409594 >> 172.21.15.55:0/3181409594 conn(0x7f23d406baa0 msgr2=0x7f23d406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:10.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.593+0000 7f23da59d700 1 -- 172.21.15.55:0/3181409594 shutdown_connections 2023-10-06T03:33:10.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.593+0000 7f23da59d700 1 -- 172.21.15.55:0/3181409594 wait complete. 2023-10-06T03:33:10.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.595+0000 7f23da59d700 1 Processor -- start 2023-10-06T03:33:10.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.595+0000 7f23da59d700 1 -- start start 2023-10-06T03:33:10.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23da59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40cb590 0x7f23d40c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:10.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23da59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d4157680 0x7f23d4152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:10.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23da59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23d4157a50 0x7f23d415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:10.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23da59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d4169f50 con 0x7f23d40cb590 2023-10-06T03:33:10.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23da59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d4159690 con 0x7f23d4157680 2023-10-06T03:33:10.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23da59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d4158a80 con 0x7f23d4157a50 2023-10-06T03:33:10.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23d959b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40cb590 0x7f23d40c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:10.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23d959b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40cb590 0x7f23d40c6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34036/0 (socket says 172.21.15.55:34036) 2023-10-06T03:33:10.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23d959b700 1 -- 172.21.15.55:0/2093066446 learned_addr learned my addr 172.21.15.55:0/2093066446 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:10.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23d9d9c700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23d4157a50 0x7f23d415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:10.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23d8d9a700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d4157680 0x7f23d4152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:10.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23d959b700 1 -- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23d4157a50 msgr2=0x7f23d415ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:10.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23d959b700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23d4157a50 0x7f23d415ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23d959b700 1 -- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d4157680 msgr2=0x7f23d4152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:10.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23d959b700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d4157680 0x7f23d4152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23d959b700 1 -- 172.21.15.55:0/2093066446 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c4009420 con 0x7f23d40cb590 2023-10-06T03:33:10.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23d9d9c700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23d4157a50 0x7f23d415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:10.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23d8d9a700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d4157680 0x7f23d4152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:10.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.596+0000 7f23d959b700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40cb590 0x7f23d40c6480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f23d00011e0 tx=0x7f23d0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:10.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.597+0000 7f23c97fa700 1 -- 172.21.15.55:0/2093066446 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0005b30 con 0x7f23d40cb590 2023-10-06T03:33:10.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.597+0000 7f23c97fa700 1 -- 172.21.15.55:0/2093066446 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d0005c90 con 0x7f23d40cb590 2023-10-06T03:33:10.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.597+0000 7f23c97fa700 1 -- 172.21.15.55:0/2093066446 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d00072e0 con 0x7f23d40cb590 2023-10-06T03:33:10.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.597+0000 7f23da59d700 1 -- 172.21.15.55:0/2093066446 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d4158dc0 con 0x7f23d40cb590 2023-10-06T03:33:10.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.597+0000 7f23da59d700 1 -- 172.21.15.55:0/2093066446 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d4152b60 con 0x7f23d40cb590 2023-10-06T03:33:10.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.598+0000 7f23da59d700 1 -- 172.21.15.55:0/2093066446 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d406eba0 con 0x7f23d40cb590 2023-10-06T03:33:10.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.599+0000 7f23c97fa700 1 -- 172.21.15.55:0/2093066446 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f23d0007650 con 0x7f23d40cb590 2023-10-06T03:33:10.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.600+0000 7f23c97fa700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f23b8066ea0 0x7f23b8069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:10.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.600+0000 7f23c97fa700 1 -- 172.21.15.55:0/2093066446 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f23d0083740 con 0x7f23d40cb590 2023-10-06T03:33:10.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.600+0000 7f23d8d9a700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f23b8066ea0 0x7f23b8069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:10.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.601+0000 7f23d8d9a700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f23b8066ea0 0x7f23b8069350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f23c40098e0 tx=0x7f23c4017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:10.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.604+0000 7f23c97fa700 1 -- 172.21.15.55:0/2093066446 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23d004b5a0 con 0x7f23d40cb590 2023-10-06T03:33:10.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.813+0000 7f23da59d700 1 -- 172.21.15.55:0/2093066446 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23d404bb80 con 0x7f23d40cb590 2023-10-06T03:33:10.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.813+0000 7f23c97fa700 1 -- 172.21.15.55:0/2093066446 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23d0017020 con 0x7f23d40cb590 2023-10-06T03:33:10.815 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:33:10.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.816+0000 7f23da59d700 1 -- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f23b8066ea0 msgr2=0x7f23b8069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:10.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.816+0000 7f23da59d700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f23b8066ea0 0x7f23b8069350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f23c40098e0 tx=0x7f23c4017000 comp rx=0 tx=0).stop 2023-10-06T03:33:10.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.816+0000 7f23da59d700 1 -- 172.21.15.55:0/2093066446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40cb590 msgr2=0x7f23d40c6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:10.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.816+0000 7f23da59d700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40cb590 0x7f23d40c6480 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f23d00011e0 tx=0x7f23d0007df0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.816+0000 7f23da59d700 1 -- 172.21.15.55:0/2093066446 shutdown_connections 2023-10-06T03:33:10.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.816+0000 7f23da59d700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f23b8066ea0 0x7f23b8069350 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.816+0000 7f23da59d700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23d4157a50 0x7f23d415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.816+0000 7f23da59d700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23d40cb590 0x7f23d40c6480 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.816+0000 7f23da59d700 1 --2- 172.21.15.55:0/2093066446 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23d4157680 0x7f23d4152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:10.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.816+0000 7f23da59d700 1 -- 172.21.15.55:0/2093066446 >> 172.21.15.55:0/2093066446 conn(0x7f23d406baa0 msgr2=0x7f23d40c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:10.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.816+0000 7f23da59d700 1 -- 172.21.15.55:0/2093066446 shutdown_connections 2023-10-06T03:33:10.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:10.816+0000 7f23da59d700 1 -- 172.21.15.55:0/2093066446 wait complete. 2023-10-06T03:33:10.837 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:33:11.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.027+0000 7f78357ff700 1 Processor -- start 2023-10-06T03:33:11.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.027+0000 7f78357ff700 1 -- start start 2023-10-06T03:33:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.028+0000 7f78357ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7830069560 0x7f78300699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.028+0000 7f78357ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7830069ee0 con 0x7f7830159890 2023-10-06T03:33:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.028+0000 7f78357ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783015a160 con 0x7f7830159c60 2023-10-06T03:33:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.028+0000 7f78357ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78300d13a0 con 0x7f7830069560 2023-10-06T03:33:11.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.029+0000 7f782effd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7830159890 0x7f783015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53494/0 (socket says 172.21.15.55:53494) 2023-10-06T03:33:11.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.029+0000 7f782effd700 1 -- 172.21.15.55:0/1168511562 learned_addr learned my addr 172.21.15.55:0/1168511562 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:11.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.029+0000 7f782f7fe700 1 --2- 172.21.15.55:0/1168511562 >> v2:172.21.15.163:3300/0 conn(0x7f7830069560 0x7f78300699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.029+0000 7f782dffb700 1 -- 172.21.15.55:0/1168511562 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2336451083 0 0) 0x7f7830069ee0 con 0x7f7830159890 2023-10-06T03:33:11.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.029+0000 7f782dffb700 1 -- 172.21.15.55:0/1168511562 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7818003570 con 0x7f7830159890 2023-10-06T03:33:11.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.029+0000 7f782dffb700 1 -- 172.21.15.55:0/1168511562 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244711968 0 0) 0x7f783015a160 con 0x7f7830159c60 2023-10-06T03:33:11.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.029+0000 7f782dffb700 1 -- 172.21.15.55:0/1168511562 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7830069ee0 con 0x7f7830159c60 2023-10-06T03:33:11.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.029+0000 7f782f7fe700 1 -- 172.21.15.55:0/1168511562 >> v1:172.21.15.163:6789/0 conn(0x7f7830159c60 legacy=0x7f783015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.029+0000 7f782f7fe700 1 -- 172.21.15.55:0/1168511562 >> v1:172.21.15.55:6789/0 conn(0x7f7830159890 legacy=0x7f783015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.029+0000 7f782f7fe700 1 -- 172.21.15.55:0/1168511562 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783015e140 con 0x7f7830069560 2023-10-06T03:33:11.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.030+0000 7f782f7fe700 1 --2- 172.21.15.55:0/1168511562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7830069560 0x7f78300699a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7828009e60 tx=0x7f7828009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eff3933100d4b6e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:11.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.030+0000 7f782dffb700 1 -- 172.21.15.55:0/1168511562 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782800e070 con 0x7f7830069560 2023-10-06T03:33:11.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.030+0000 7f782dffb700 1 -- 172.21.15.55:0/1168511562 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782800bab0 con 0x7f7830069560 2023-10-06T03:33:11.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.030+0000 7f782dffb700 1 -- 172.21.15.55:0/1168511562 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782800be70 con 0x7f7830069560 2023-10-06T03:33:11.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.031+0000 7f78357ff700 1 -- 172.21.15.55:0/1168511562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7830069560 msgr2=0x7f78300699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.031+0000 7f78357ff700 1 --2- 172.21.15.55:0/1168511562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7830069560 0x7f78300699a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7828009e60 tx=0x7f7828009780 comp rx=0 tx=0).stop 2023-10-06T03:33:11.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.031+0000 7f78357ff700 1 -- 172.21.15.55:0/1168511562 shutdown_connections 2023-10-06T03:33:11.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.031+0000 7f78357ff700 1 --2- 172.21.15.55:0/1168511562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7830069560 0x7f78300699a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.031+0000 7f78357ff700 1 -- 172.21.15.55:0/1168511562 >> 172.21.15.55:0/1168511562 conn(0x7f7830068e20 msgr2=0x7f78300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:11.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.031+0000 7f78357ff700 1 -- 172.21.15.55:0/1168511562 shutdown_connections 2023-10-06T03:33:11.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.031+0000 7f78357ff700 1 -- 172.21.15.55:0/1168511562 wait complete. 2023-10-06T03:33:11.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.032+0000 7f78357ff700 1 Processor -- start 2023-10-06T03:33:11.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.032+0000 7f78357ff700 1 -- start start 2023-10-06T03:33:11.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.033+0000 7f78357ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7830069560 0x7f78300b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.033+0000 7f78357ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7830159890 0x7f78300c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.033+0000 7f78357ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7830159c60 0x7f7830157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.033+0000 7f78357ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78300b6060 con 0x7f7830069560 2023-10-06T03:33:11.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.033+0000 7f78357ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78300b5450 con 0x7f7830159c60 2023-10-06T03:33:11.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.033+0000 7f78357ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78300b5730 con 0x7f7830159890 2023-10-06T03:33:11.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.033+0000 7f782effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7830069560 0x7f78300b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.033+0000 7f782effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7830069560 0x7f78300b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34038/0 (socket says 172.21.15.55:34038) 2023-10-06T03:33:11.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.034+0000 7f782effd700 1 -- 172.21.15.55:0/3158438112 learned_addr learned my addr 172.21.15.55:0/3158438112 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:11.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.034+0000 7f782f7fe700 1 --2- 172.21.15.55:0/3158438112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7830159c60 0x7f7830157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.034+0000 7f782e7fc700 1 --2- 172.21.15.55:0/3158438112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7830159890 0x7f78300c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.034+0000 7f782effd700 1 -- 172.21.15.55:0/3158438112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7830159890 msgr2=0x7f78300c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.034+0000 7f782effd700 1 --2- 172.21.15.55:0/3158438112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7830159890 0x7f78300c9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.034+0000 7f782effd700 1 -- 172.21.15.55:0/3158438112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7830159c60 msgr2=0x7f7830157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.034+0000 7f782effd700 1 --2- 172.21.15.55:0/3158438112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7830159c60 0x7f7830157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.034+0000 7f782effd700 1 -- 172.21.15.55:0/3158438112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7828009a20 con 0x7f7830069560 2023-10-06T03:33:11.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.034+0000 7f782effd700 1 --2- 172.21.15.55:0/3158438112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7830069560 0x7f78300b4b70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f782000c790 tx=0x7f7820007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:11.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.034+0000 7f7816ffd700 1 -- 172.21.15.55:0/3158438112 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820005b30 con 0x7f7830069560 2023-10-06T03:33:11.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.034+0000 7f7816ffd700 1 -- 172.21.15.55:0/3158438112 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7820005cd0 con 0x7f7830069560 2023-10-06T03:33:11.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.034+0000 7f7816ffd700 1 -- 172.21.15.55:0/3158438112 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7820007320 con 0x7f7830069560 2023-10-06T03:33:11.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.034+0000 7f78357ff700 1 -- 172.21.15.55:0/3158438112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78300b5a10 con 0x7f7830069560 2023-10-06T03:33:11.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.035+0000 7f78357ff700 1 -- 172.21.15.55:0/3158438112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78300b50b0 con 0x7f7830069560 2023-10-06T03:33:11.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.036+0000 7f78357ff700 1 -- 172.21.15.55:0/3158438112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783004c3c0 con 0x7f7830069560 2023-10-06T03:33:11.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.036+0000 7f7816ffd700 1 -- 172.21.15.55:0/3158438112 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f7820012400 con 0x7f7830069560 2023-10-06T03:33:11.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.037+0000 7f7816ffd700 1 --2- 172.21.15.55:0/3158438112 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f781c066ea0 0x7f781c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.038+0000 7f7816ffd700 1 -- 172.21.15.55:0/3158438112 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7820083680 con 0x7f7830069560 2023-10-06T03:33:11.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.038+0000 7f782e7fc700 1 --2- 172.21.15.55:0/3158438112 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f781c066ea0 0x7f781c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.039+0000 7f782e7fc700 1 --2- 172.21.15.55:0/3158438112 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f781c066ea0 0x7f781c069350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f78300ca8a0 tx=0x7f7824009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:11.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.042+0000 7f7816ffd700 1 -- 172.21.15.55:0/3158438112 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f782004b5a0 con 0x7f7830069560 2023-10-06T03:33:11.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.300+0000 7f78357ff700 1 -- 172.21.15.55:0/3158438112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f783006a5b0 con 0x7f7830069560 2023-10-06T03:33:11.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.300+0000 7f7816ffd700 1 -- 172.21.15.55:0/3158438112 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f7820017090 con 0x7f7830069560 2023-10-06T03:33:11.302 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:11.302 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2677235076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":2677235076},{"type":"v1","addr":"172.21.15.55:6835","nonce":2677235076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1030747394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1030747394},{"type":"v1","addr":"172.21.15.55:6834","nonce":1030747394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2040975046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2040975046},{"type":"v1","addr":"172.21.15.163:6836","nonce":2040975046}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6837/1910887369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1910887369},{"type":"v1","addr":"172.21.15.163:6837","nonce":1910887369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-06T03:33:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.303+0000 7f78357ff700 1 -- 172.21.15.55:0/3158438112 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f781c066ea0 msgr2=0x7f781c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.303+0000 7f78357ff700 1 --2- 172.21.15.55:0/3158438112 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f781c066ea0 0x7f781c069350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f78300ca8a0 tx=0x7f7824009040 comp rx=0 tx=0).stop 2023-10-06T03:33:11.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.303+0000 7f78357ff700 1 -- 172.21.15.55:0/3158438112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7830069560 msgr2=0x7f78300b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.303+0000 7f78357ff700 1 --2- 172.21.15.55:0/3158438112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7830069560 0x7f78300b4b70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f782000c790 tx=0x7f7820007e80 comp rx=0 tx=0).stop 2023-10-06T03:33:11.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.303+0000 7f78357ff700 1 -- 172.21.15.55:0/3158438112 shutdown_connections 2023-10-06T03:33:11.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.303+0000 7f78357ff700 1 --2- 172.21.15.55:0/3158438112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7830159c60 0x7f7830157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.303+0000 7f78357ff700 1 --2- 172.21.15.55:0/3158438112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7830069560 0x7f78300b4b70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.303+0000 7f78357ff700 1 --2- 172.21.15.55:0/3158438112 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f781c066ea0 0x7f781c069350 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.303+0000 7f78357ff700 1 --2- 172.21.15.55:0/3158438112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7830159890 0x7f78300c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.303+0000 7f78357ff700 1 -- 172.21.15.55:0/3158438112 >> 172.21.15.55:0/3158438112 conn(0x7f7830068e20 msgr2=0x7f783015d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:11.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.303+0000 7f78357ff700 1 -- 172.21.15.55:0/3158438112 shutdown_connections 2023-10-06T03:33:11.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.303+0000 7f78357ff700 1 -- 172.21.15.55:0/3158438112 wait complete. 2023-10-06T03:33:11.306 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 3 2023-10-06T03:33:11.322 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:11.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.512+0000 7f021bfff700 1 Processor -- start 2023-10-06T03:33:11.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.512+0000 7f021bfff700 1 -- start start 2023-10-06T03:33:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.514+0000 7f021bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f021c0ae710 0x7f021c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.514+0000 7f021bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021c0ca860 con 0x7f021c1566e0 2023-10-06T03:33:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.514+0000 7f021bfff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f021c0cab40 con 0x7f021c156310 2023-10-06T03:33:11.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.514+0000 7f021bfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f021c0cae20 con 0x7f021c0ae710 2023-10-06T03:33:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.514+0000 7f021b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f021c0ae710 0x7f021c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.514+0000 7f021b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f021c0ae710 0x7f021c0c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34048/0 (socket says 172.21.15.55:34048) 2023-10-06T03:33:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.514+0000 7f021b7fe700 1 -- 172.21.15.55:0/1163008321 learned_addr learned my addr 172.21.15.55:0/1163008321 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.514+0000 7f021a7fc700 1 --1- 172.21.15.55:0/1163008321 >> v1:172.21.15.55:6789/0 conn(0x7f021c1566e0 0x7f021c0ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53504/0 (socket says 172.21.15.55:53504) 2023-10-06T03:33:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.514+0000 7f021affd700 1 --1- 172.21.15.55:0/1163008321 >> v1:172.21.15.163:6789/0 conn(0x7f021c156310 0x7f021c0c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:56224/0 (socket says 172.21.15.55:56224) 2023-10-06T03:33:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.514+0000 7f0219ffb700 1 -- 172.21.15.55:0/1163008321 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147647584 0 0) 0x7f021c0cab40 con 0x7f021c156310 2023-10-06T03:33:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.515+0000 7f021b7fe700 1 -- 172.21.15.55:0/1163008321 >> v1:172.21.15.163:6789/0 conn(0x7f021c156310 legacy=0x7f021c0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.515+0000 7f021b7fe700 1 -- 172.21.15.55:0/1163008321 >> v1:172.21.15.55:6789/0 conn(0x7f021c1566e0 legacy=0x7f021c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.515+0000 7f021b7fe700 1 -- 172.21.15.55:0/1163008321 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c0cb100 con 0x7f021c0ae710 2023-10-06T03:33:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.515+0000 7f021b7fe700 1 --2- 172.21.15.55:0/1163008321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f021c0ae710 0x7f021c0c6830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f021400d1a0 tx=0x7f021400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7186844726944e37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:11.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.515+0000 7f0219ffb700 1 -- 172.21.15.55:0/1163008321 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214010070 con 0x7f021c0ae710 2023-10-06T03:33:11.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.515+0000 7f0219ffb700 1 -- 172.21.15.55:0/1163008321 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021400b0d0 con 0x7f021c0ae710 2023-10-06T03:33:11.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.515+0000 7f0219ffb700 1 -- 172.21.15.55:0/1163008321 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021400b490 con 0x7f021c0ae710 2023-10-06T03:33:11.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.515+0000 7f021bfff700 1 -- 172.21.15.55:0/1163008321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f021c0ae710 msgr2=0x7f021c0c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.516+0000 7f021bfff700 1 --2- 172.21.15.55:0/1163008321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f021c0ae710 0x7f021c0c6830 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f021400d1a0 tx=0x7f021400ba50 comp rx=0 tx=0).stop 2023-10-06T03:33:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.516+0000 7f021bfff700 1 -- 172.21.15.55:0/1163008321 shutdown_connections 2023-10-06T03:33:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.516+0000 7f021bfff700 1 --2- 172.21.15.55:0/1163008321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f021c0ae710 0x7f021c0c6830 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.516+0000 7f021bfff700 1 -- 172.21.15.55:0/1163008321 >> 172.21.15.55:0/1163008321 conn(0x7f021c0cc1f0 msgr2=0x7f021c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.516+0000 7f021bfff700 1 -- 172.21.15.55:0/1163008321 shutdown_connections 2023-10-06T03:33:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.516+0000 7f021bfff700 1 -- 172.21.15.55:0/1163008321 wait complete. 2023-10-06T03:33:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.517+0000 7f021bfff700 1 Processor -- start 2023-10-06T03:33:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.517+0000 7f021bfff700 1 -- start start 2023-10-06T03:33:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.518+0000 7f021bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f021c0ae710 0x7f021c0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.518+0000 7f021bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f021c156310 0x7f021c13ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.518+0000 7f021bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f021c1566e0 0x7f021c169670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.518+0000 7f021bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c0aef90 con 0x7f021c0ae710 2023-10-06T03:33:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.518+0000 7f021bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c16a200 con 0x7f021c1566e0 2023-10-06T03:33:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.518+0000 7f021bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021c16a4e0 con 0x7f021c156310 2023-10-06T03:33:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.518+0000 7f021affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f021c0ae710 0x7f021c0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.518+0000 7f021affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f021c0ae710 0x7f021c0afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34058/0 (socket says 172.21.15.55:34058) 2023-10-06T03:33:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.518+0000 7f021affd700 1 -- 172.21.15.55:0/3277564962 learned_addr learned my addr 172.21.15.55:0/3277564962 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.519+0000 7f021b7fe700 1 --2- 172.21.15.55:0/3277564962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f021c1566e0 0x7f021c169670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.519+0000 7f021a7fc700 1 --2- 172.21.15.55:0/3277564962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f021c156310 0x7f021c13ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.519+0000 7f021affd700 1 -- 172.21.15.55:0/3277564962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f021c156310 msgr2=0x7f021c13ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.519+0000 7f021affd700 1 --2- 172.21.15.55:0/3277564962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f021c156310 0x7f021c13ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.519+0000 7f021affd700 1 -- 172.21.15.55:0/3277564962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f021c1566e0 msgr2=0x7f021c169670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.519+0000 7f021affd700 1 --2- 172.21.15.55:0/3277564962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f021c1566e0 0x7f021c169670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.519+0000 7f021affd700 1 -- 172.21.15.55:0/3277564962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021400bcf0 con 0x7f021c0ae710 2023-10-06T03:33:11.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.519+0000 7f021affd700 1 --2- 172.21.15.55:0/3277564962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f021c0ae710 0x7f021c0afa50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f020c0011e0 tx=0x7f020c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:11.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.519+0000 7f0202ffd700 1 -- 172.21.15.55:0/3277564962 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c005a70 con 0x7f021c0ae710 2023-10-06T03:33:11.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.519+0000 7f0202ffd700 1 -- 172.21.15.55:0/3277564962 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020c005bd0 con 0x7f021c0ae710 2023-10-06T03:33:11.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.519+0000 7f021bfff700 1 -- 172.21.15.55:0/3277564962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021c0af250 con 0x7f021c0ae710 2023-10-06T03:33:11.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.519+0000 7f0202ffd700 1 -- 172.21.15.55:0/3277564962 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c0070d0 con 0x7f021c0ae710 2023-10-06T03:33:11.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.519+0000 7f021bfff700 1 -- 172.21.15.55:0/3277564962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021c13f0f0 con 0x7f021c0ae710 2023-10-06T03:33:11.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.520+0000 7f021bfff700 1 -- 172.21.15.55:0/3277564962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021c04c3c0 con 0x7f021c0ae710 2023-10-06T03:33:11.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.521+0000 7f0202ffd700 1 -- 172.21.15.55:0/3277564962 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f020c007480 con 0x7f021c0ae710 2023-10-06T03:33:11.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.522+0000 7f0202ffd700 1 --2- 172.21.15.55:0/3277564962 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0208066ea0 0x7f0208069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.522+0000 7f0202ffd700 1 -- 172.21.15.55:0/3277564962 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f020c083770 con 0x7f021c0ae710 2023-10-06T03:33:11.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.522+0000 7f021a7fc700 1 --2- 172.21.15.55:0/3277564962 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0208066ea0 0x7f0208069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.523+0000 7f021a7fc700 1 --2- 172.21.15.55:0/3277564962 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0208066ea0 0x7f0208069350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f021c0c3140 tx=0x7f0210007eb0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:11.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.526+0000 7f0202ffd700 1 -- 172.21.15.55:0/3277564962 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f020c050c10 con 0x7f021c0ae710 2023-10-06T03:33:11.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.728+0000 7f021bfff700 1 -- 172.21.15.55:0/3277564962 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f021c0cd0b0 con 0x7f021c0ae710 2023-10-06T03:33:11.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.729+0000 7f0202ffd700 1 -- 172.21.15.55:0/3277564962 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13696 (secure 0 0 0) 0x7f020c017070 con 0x7f021c0ae710 2023-10-06T03:33:11.730 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:11.731 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":17,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:11.470728+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.731+0000 7f021bfff700 1 -- 172.21.15.55:0/3277564962 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0208066ea0 msgr2=0x7f0208069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.731+0000 7f021bfff700 1 --2- 172.21.15.55:0/3277564962 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0208066ea0 0x7f0208069350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f021c0c3140 tx=0x7f0210007eb0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.731+0000 7f021bfff700 1 -- 172.21.15.55:0/3277564962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f021c0ae710 msgr2=0x7f021c0afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.731+0000 7f021bfff700 1 --2- 172.21.15.55:0/3277564962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f021c0ae710 0x7f021c0afa50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f020c0011e0 tx=0x7f020c007e10 comp rx=0 tx=0).stop 2023-10-06T03:33:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.731+0000 7f021bfff700 1 -- 172.21.15.55:0/3277564962 shutdown_connections 2023-10-06T03:33:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.731+0000 7f021bfff700 1 --2- 172.21.15.55:0/3277564962 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f021c1566e0 0x7f021c169670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.731+0000 7f021bfff700 1 --2- 172.21.15.55:0/3277564962 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f021c0ae710 0x7f021c0afa50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.731+0000 7f021bfff700 1 --2- 172.21.15.55:0/3277564962 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0208066ea0 0x7f0208069350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.731+0000 7f021bfff700 1 --2- 172.21.15.55:0/3277564962 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f021c156310 0x7f021c13ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.731+0000 7f021bfff700 1 -- 172.21.15.55:0/3277564962 >> 172.21.15.55:0/3277564962 conn(0x7f021c0cc1f0 msgr2=0x7f021c069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.731+0000 7f021bfff700 1 -- 172.21.15.55:0/3277564962 shutdown_connections 2023-10-06T03:33:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.731+0000 7f021bfff700 1 -- 172.21.15.55:0/3277564962 wait complete. 2023-10-06T03:33:11.750 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T03:33:11.750 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:33:11.814 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:33:11.818 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:33:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.947+0000 7f992ed9d700 1 Processor -- start 2023-10-06T03:33:11.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.947+0000 7f992ed9d700 1 -- start start 2023-10-06T03:33:11.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.948+0000 7f992ed9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f99280c9db0 0x7f992806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.948+0000 7f992ed9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9928157130 0x7f992806a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.948+0000 7f992ed9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992806e150 con 0x7f9928153900 2023-10-06T03:33:11.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.948+0000 7f992ed9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99280c7610 con 0x7f99280c9db0 2023-10-06T03:33:11.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.948+0000 7f992ed9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99280c7800 con 0x7f9928157130 2023-10-06T03:33:11.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.948+0000 7f992dd9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9928153900 0x7f99280c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53512/0 (socket says 172.21.15.55:53512) 2023-10-06T03:33:11.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.949+0000 7f992dd9b700 1 -- 172.21.15.55:0/3999316057 learned_addr learned my addr 172.21.15.55:0/3999316057 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:11.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.949+0000 7f992d59a700 1 --2- 172.21.15.55:0/3999316057 >> v2:172.21.15.163:3300/0 conn(0x7f99280c9db0 0x7f992806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.949+0000 7f992e59c700 1 --2- 172.21.15.55:0/3999316057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9928157130 0x7f992806a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.949+0000 7f992cd99700 1 -- 172.21.15.55:0/3999316057 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785779514 0 0) 0x7f992806e150 con 0x7f9928153900 2023-10-06T03:33:11.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.949+0000 7f992cd99700 1 -- 172.21.15.55:0/3999316057 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9914003570 con 0x7f9928153900 2023-10-06T03:33:11.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.949+0000 7f992d59a700 1 -- 172.21.15.55:0/3999316057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9928157130 msgr2=0x7f992806a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:11.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.949+0000 7f992d59a700 1 --2- 172.21.15.55:0/3999316057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9928157130 0x7f992806a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.949+0000 7f992d59a700 1 -- 172.21.15.55:0/3999316057 >> v1:172.21.15.55:6789/0 conn(0x7f9928153900 legacy=0x7f99280c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.949+0000 7f992d59a700 1 -- 172.21.15.55:0/3999316057 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99280c7ae0 con 0x7f99280c9db0 2023-10-06T03:33:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.949+0000 7f992e59c700 1 --2- 172.21.15.55:0/3999316057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9928157130 0x7f992806a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.949+0000 7f992d59a700 1 --2- 172.21.15.55:0/3999316057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99280c9db0 0x7f992806a000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f99180098e0 tx=0x7f99180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63074c915c4e51a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.950+0000 7f992cd99700 1 -- 172.21.15.55:0/3999316057 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918011630 con 0x7f99280c9db0 2023-10-06T03:33:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.950+0000 7f992cd99700 1 -- 172.21.15.55:0/3999316057 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9918011790 con 0x7f99280c9db0 2023-10-06T03:33:11.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.950+0000 7f992cd99700 1 -- 172.21.15.55:0/3999316057 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918011b30 con 0x7f99280c9db0 2023-10-06T03:33:11.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.950+0000 7f992ed9d700 1 -- 172.21.15.55:0/3999316057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99280c9db0 msgr2=0x7f992806a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.950+0000 7f992ed9d700 1 --2- 172.21.15.55:0/3999316057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99280c9db0 0x7f992806a000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f99180098e0 tx=0x7f99180091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.950+0000 7f992ed9d700 1 -- 172.21.15.55:0/3999316057 shutdown_connections 2023-10-06T03:33:11.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.950+0000 7f992ed9d700 1 --2- 172.21.15.55:0/3999316057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9928157130 0x7f992806a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.950+0000 7f992ed9d700 1 --2- 172.21.15.55:0/3999316057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99280c9db0 0x7f992806a000 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.950+0000 7f992ed9d700 1 -- 172.21.15.55:0/3999316057 >> 172.21.15.55:0/3999316057 conn(0x7f99280cc1e0 msgr2=0x7f99280cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:11.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.950+0000 7f992ed9d700 1 -- 172.21.15.55:0/3999316057 shutdown_connections 2023-10-06T03:33:11.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.950+0000 7f992ed9d700 1 -- 172.21.15.55:0/3999316057 wait complete. 2023-10-06T03:33:11.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.952+0000 7f992ed9d700 1 Processor -- start 2023-10-06T03:33:11.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.952+0000 7f992ed9d700 1 -- start start 2023-10-06T03:33:11.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99280c9db0 0x7f9928149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9928153900 0x7f99280b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992ed9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9928157130 0x7f99280af450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992ed9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9928168af0 con 0x7f99280c9db0 2023-10-06T03:33:11.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992ed9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99280ae990 con 0x7f9928153900 2023-10-06T03:33:11.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992ed9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9928168cf0 con 0x7f9928157130 2023-10-06T03:33:11.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992dd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99280c9db0 0x7f9928149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992dd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99280c9db0 0x7f9928149860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34068/0 (socket says 172.21.15.55:34068) 2023-10-06T03:33:11.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992dd9b700 1 -- 172.21.15.55:0/3167914282 learned_addr learned my addr 172.21.15.55:0/3167914282 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:11.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992d59a700 1 --2- 172.21.15.55:0/3167914282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9928153900 0x7f99280b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992e59c700 1 --2- 172.21.15.55:0/3167914282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9928157130 0x7f99280af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992dd9b700 1 -- 172.21.15.55:0/3167914282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9928157130 msgr2=0x7f99280af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992dd9b700 1 --2- 172.21.15.55:0/3167914282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9928157130 0x7f99280af450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992dd9b700 1 -- 172.21.15.55:0/3167914282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9928153900 msgr2=0x7f99280b06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:11.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992dd9b700 1 --2- 172.21.15.55:0/3167914282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9928153900 0x7f99280b06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:11.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.953+0000 7f992dd9b700 1 -- 172.21.15.55:0/3167914282 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9920009a20 con 0x7f99280c9db0 2023-10-06T03:33:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.954+0000 7f992dd9b700 1 --2- 172.21.15.55:0/3167914282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99280c9db0 0x7f9928149860 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f992400c7c0 tx=0x7f9924007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.954+0000 7f991dffb700 1 -- 172.21.15.55:0/3167914282 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924005b30 con 0x7f99280c9db0 2023-10-06T03:33:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.954+0000 7f991dffb700 1 -- 172.21.15.55:0/3167914282 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9924005cb0 con 0x7f99280c9db0 2023-10-06T03:33:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.954+0000 7f991dffb700 1 -- 172.21.15.55:0/3167914282 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924007320 con 0x7f99280c9db0 2023-10-06T03:33:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.954+0000 7f992ed9d700 1 -- 172.21.15.55:0/3167914282 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9918009420 con 0x7f99280c9db0 2023-10-06T03:33:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.954+0000 7f992ed9d700 1 -- 172.21.15.55:0/3167914282 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99280ae5b0 con 0x7f99280c9db0 2023-10-06T03:33:11.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.955+0000 7f992ed9d700 1 -- 172.21.15.55:0/3167914282 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99281690a0 con 0x7f99280c9db0 2023-10-06T03:33:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.956+0000 7f991dffb700 1 -- 172.21.15.55:0/3167914282 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f992401b3b0 con 0x7f99280c9db0 2023-10-06T03:33:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.957+0000 7f991dffb700 1 --2- 172.21.15.55:0/3167914282 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f990c066ea0 0x7f990c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:11.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.957+0000 7f991dffb700 1 -- 172.21.15.55:0/3167914282 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f99240837b0 con 0x7f99280c9db0 2023-10-06T03:33:11.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.957+0000 7f992d59a700 1 --2- 172.21.15.55:0/3167914282 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f990c066ea0 0x7f990c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:11.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.958+0000 7f992d59a700 1 --2- 172.21.15.55:0/3167914282 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f990c066ea0 0x7f990c069350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99280ca1d0 tx=0x7f9918017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:11.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:11.961+0000 7f991dffb700 1 -- 172.21.15.55:0/3167914282 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9924050c50 con 0x7f99280c9db0 2023-10-06T03:33:12.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.167+0000 7f992ed9d700 1 -- 172.21.15.55:0/3167914282 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f99280c7dc0 con 0x7f99280c9db0 2023-10-06T03:33:12.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.167+0000 7f991dffb700 1 -- 172.21.15.55:0/3167914282 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9924017020 con 0x7f99280c9db0 2023-10-06T03:33:12.169 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:33:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.170+0000 7f992ed9d700 1 -- 172.21.15.55:0/3167914282 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f990c066ea0 msgr2=0x7f990c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.170+0000 7f992ed9d700 1 --2- 172.21.15.55:0/3167914282 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f990c066ea0 0x7f990c069350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99280ca1d0 tx=0x7f9918017000 comp rx=0 tx=0).stop 2023-10-06T03:33:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.170+0000 7f992ed9d700 1 -- 172.21.15.55:0/3167914282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99280c9db0 msgr2=0x7f9928149860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.170+0000 7f992ed9d700 1 --2- 172.21.15.55:0/3167914282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99280c9db0 0x7f9928149860 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f992400c7c0 tx=0x7f9924007e00 comp rx=0 tx=0).stop 2023-10-06T03:33:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.170+0000 7f992ed9d700 1 -- 172.21.15.55:0/3167914282 shutdown_connections 2023-10-06T03:33:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.170+0000 7f992ed9d700 1 --2- 172.21.15.55:0/3167914282 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f990c066ea0 0x7f990c069350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.170+0000 7f992ed9d700 1 --2- 172.21.15.55:0/3167914282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9928157130 0x7f99280af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.170+0000 7f992ed9d700 1 --2- 172.21.15.55:0/3167914282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99280c9db0 0x7f9928149860 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.170+0000 7f992ed9d700 1 --2- 172.21.15.55:0/3167914282 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9928153900 0x7f99280b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.170+0000 7f992ed9d700 1 -- 172.21.15.55:0/3167914282 >> 172.21.15.55:0/3167914282 conn(0x7f99280cc1e0 msgr2=0x7f99281578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:12.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.170+0000 7f992ed9d700 1 -- 172.21.15.55:0/3167914282 shutdown_connections 2023-10-06T03:33:12.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.170+0000 7f992ed9d700 1 -- 172.21.15.55:0/3167914282 wait complete. 2023-10-06T03:33:12.189 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:33:12.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.419+0000 7f867d19d700 1 Processor -- start 2023-10-06T03:33:12.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.419+0000 7f867d19d700 1 -- start start 2023-10-06T03:33:12.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.421+0000 7f867d19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8678159a70 0x7f86780c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:12.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.421+0000 7f867d19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8678068aa0 0x7f86780c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:12.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.421+0000 7f867d19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86780ca920 con 0x7f867806bfa0 2023-10-06T03:33:12.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.421+0000 7f867d19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867806af00 con 0x7f8678068aa0 2023-10-06T03:33:12.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.421+0000 7f867d19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867806b1e0 con 0x7f8678159a70 2023-10-06T03:33:12.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.421+0000 7f867c99c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8678068aa0 0x7f86780c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:12.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.421+0000 7f867c99c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8678068aa0 0x7f86780c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34076/0 (socket says 172.21.15.55:34076) 2023-10-06T03:33:12.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.421+0000 7f867c99c700 1 -- 172.21.15.55:0/719056516 learned_addr learned my addr 172.21.15.55:0/719056516 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:12.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.421+0000 7f8677fff700 1 --2- 172.21.15.55:0/719056516 >> v2:172.21.15.163:3300/0 conn(0x7f8678159a70 0x7f86780c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:12.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.421+0000 7f867c99c700 1 -- 172.21.15.55:0/719056516 >> v2:172.21.15.163:3300/0 conn(0x7f8678159a70 msgr2=0x7f86780c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:12.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.421+0000 7f867c99c700 1 --2- 172.21.15.55:0/719056516 >> v2:172.21.15.163:3300/0 conn(0x7f8678159a70 0x7f86780c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.421+0000 7f867c99c700 1 -- 172.21.15.55:0/719056516 >> v1:172.21.15.55:6789/0 conn(0x7f867806bfa0 legacy=0x7f8678068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:12.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.421+0000 7f867c99c700 1 -- 172.21.15.55:0/719056516 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867806b4c0 con 0x7f8678068aa0 2023-10-06T03:33:12.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.421+0000 7f8676ffd700 1 -- 172.21.15.55:0/719056516 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580746667 0 0) 0x7f86780ca920 con 0x7f867806bfa0 2023-10-06T03:33:12.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.422+0000 7f867c99c700 1 --2- 172.21.15.55:0/719056516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8678068aa0 0x7f86780c8830 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f86780640b0 tx=0x7f8670009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c01d0986679773 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:12.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.422+0000 7f8676ffd700 1 -- 172.21.15.55:0/719056516 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867000e070 con 0x7f8678068aa0 2023-10-06T03:33:12.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.422+0000 7f8676ffd700 1 -- 172.21.15.55:0/719056516 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867000ba20 con 0x7f8678068aa0 2023-10-06T03:33:12.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.422+0000 7f8676ffd700 1 -- 172.21.15.55:0/719056516 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86700092e0 con 0x7f8678068aa0 2023-10-06T03:33:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.423+0000 7f867d19d700 1 -- 172.21.15.55:0/719056516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8678068aa0 msgr2=0x7f86780c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.423+0000 7f867d19d700 1 --2- 172.21.15.55:0/719056516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8678068aa0 0x7f86780c8830 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f86780640b0 tx=0x7f8670009840 comp rx=0 tx=0).stop 2023-10-06T03:33:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.423+0000 7f867d19d700 1 -- 172.21.15.55:0/719056516 shutdown_connections 2023-10-06T03:33:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.423+0000 7f867d19d700 1 --2- 172.21.15.55:0/719056516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8678068aa0 0x7f86780c8830 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.423+0000 7f867d19d700 1 --2- 172.21.15.55:0/719056516 >> v2:172.21.15.163:3300/0 conn(0x7f8678159a70 0x7f86780c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.423+0000 7f867d19d700 1 -- 172.21.15.55:0/719056516 >> 172.21.15.55:0/719056516 conn(0x7f86780cddc0 msgr2=0x7f86780c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.423+0000 7f867d19d700 1 -- 172.21.15.55:0/719056516 shutdown_connections 2023-10-06T03:33:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.423+0000 7f867d19d700 1 -- 172.21.15.55:0/719056516 wait complete. 2023-10-06T03:33:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.424+0000 7f867d19d700 1 Processor -- start 2023-10-06T03:33:12.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.424+0000 7f867d19d700 1 -- start start 2023-10-06T03:33:12.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.425+0000 7f867d19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8678068aa0 0x7f867816a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:12.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.425+0000 7f867d19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f867806bfa0 0x7f86780ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:12.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.425+0000 7f867d19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8678159a70 0x7f867814f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.425+0000 7f867d19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867816de50 con 0x7f867806bfa0 2023-10-06T03:33:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.425+0000 7f867d19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867816d240 con 0x7f8678159a70 2023-10-06T03:33:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.425+0000 7f867d19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867816d520 con 0x7f8678068aa0 2023-10-06T03:33:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.426+0000 7f86777fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f867806bfa0 0x7f86780ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.426+0000 7f86777fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f867806bfa0 0x7f86780ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34084/0 (socket says 172.21.15.55:34084) 2023-10-06T03:33:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.426+0000 7f86777fe700 1 -- 172.21.15.55:0/1343785490 learned_addr learned my addr 172.21.15.55:0/1343785490 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.426+0000 7f8677fff700 1 --2- 172.21.15.55:0/1343785490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8678068aa0 0x7f867816a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.426+0000 7f867c99c700 1 --2- 172.21.15.55:0/1343785490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8678159a70 0x7f867814f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.426+0000 7f86777fe700 1 -- 172.21.15.55:0/1343785490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8678068aa0 msgr2=0x7f867816a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.426+0000 7f86777fe700 1 --2- 172.21.15.55:0/1343785490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8678068aa0 0x7f867816a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.426+0000 7f86777fe700 1 -- 172.21.15.55:0/1343785490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8678159a70 msgr2=0x7f867814f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.426+0000 7f86777fe700 1 --2- 172.21.15.55:0/1343785490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8678159a70 0x7f867814f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.426+0000 7f86777fe700 1 -- 172.21.15.55:0/1343785490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670009ae0 con 0x7f867806bfa0 2023-10-06T03:33:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.426+0000 7f86777fe700 1 --2- 172.21.15.55:0/1343785490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f867806bfa0 0x7f86780ccca0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f866c001eb0 tx=0x7f866c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.426+0000 7f865ffff700 1 -- 172.21.15.55:0/1343785490 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c00f470 con 0x7f867806bfa0 2023-10-06T03:33:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.426+0000 7f865ffff700 1 -- 172.21.15.55:0/1343785490 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c00f5d0 con 0x7f867806bfa0 2023-10-06T03:33:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.427+0000 7f865ffff700 1 -- 172.21.15.55:0/1343785490 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c00f990 con 0x7f867806bfa0 2023-10-06T03:33:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.427+0000 7f867d19d700 1 -- 172.21.15.55:0/1343785490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867816d800 con 0x7f867806bfa0 2023-10-06T03:33:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.427+0000 7f867d19d700 1 -- 172.21.15.55:0/1343785490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867814fc80 con 0x7f867806bfa0 2023-10-06T03:33:12.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.428+0000 7f867d19d700 1 -- 172.21.15.55:0/1343785490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867806d560 con 0x7f867806bfa0 2023-10-06T03:33:12.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.429+0000 7f865ffff700 1 -- 172.21.15.55:0/1343785490 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f866c01b3b0 con 0x7f867806bfa0 2023-10-06T03:33:12.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.430+0000 7f865ffff700 1 --2- 172.21.15.55:0/1343785490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8664066ea0 0x7f8664069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:12.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.430+0000 7f865ffff700 1 -- 172.21.15.55:0/1343785490 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f866c0837a0 con 0x7f867806bfa0 2023-10-06T03:33:12.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.430+0000 7f8677fff700 1 --2- 172.21.15.55:0/1343785490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8664066ea0 0x7f8664069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:12.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.431+0000 7f8677fff700 1 --2- 172.21.15.55:0/1343785490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8664066ea0 0x7f8664069350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8668009d10 tx=0x7f86680091a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:12.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.434+0000 7f865ffff700 1 -- 172.21.15.55:0/1343785490 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f866c050c40 con 0x7f867806bfa0 2023-10-06T03:33:12.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.634+0000 7f867d19d700 1 -- 172.21.15.55:0/1343785490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f867806b7a0 con 0x7f867806bfa0 2023-10-06T03:33:12.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.635+0000 7f865ffff700 1 -- 172.21.15.55:0/1343785490 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f866c017070 con 0x7f867806bfa0 2023-10-06T03:33:12.639 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:33:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.638+0000 7f867d19d700 1 -- 172.21.15.55:0/1343785490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8664066ea0 msgr2=0x7f8664069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.638+0000 7f867d19d700 1 --2- 172.21.15.55:0/1343785490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8664066ea0 0x7f8664069350 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8668009d10 tx=0x7f86680091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.638+0000 7f867d19d700 1 -- 172.21.15.55:0/1343785490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f867806bfa0 msgr2=0x7f86780ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.638+0000 7f867d19d700 1 --2- 172.21.15.55:0/1343785490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f867806bfa0 0x7f86780ccca0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f866c001eb0 tx=0x7f866c007f60 comp rx=0 tx=0).stop 2023-10-06T03:33:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.638+0000 7f867d19d700 1 -- 172.21.15.55:0/1343785490 shutdown_connections 2023-10-06T03:33:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.638+0000 7f867d19d700 1 --2- 172.21.15.55:0/1343785490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8678159a70 0x7f867814f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.638+0000 7f867d19d700 1 --2- 172.21.15.55:0/1343785490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8664066ea0 0x7f8664069350 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.638+0000 7f867d19d700 1 --2- 172.21.15.55:0/1343785490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8678068aa0 0x7f867816a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.638+0000 7f867d19d700 1 --2- 172.21.15.55:0/1343785490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f867806bfa0 0x7f86780ccca0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.638+0000 7f867d19d700 1 -- 172.21.15.55:0/1343785490 >> 172.21.15.55:0/1343785490 conn(0x7f86780cddc0 msgr2=0x7f8678150b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.638+0000 7f867d19d700 1 -- 172.21.15.55:0/1343785490 shutdown_connections 2023-10-06T03:33:12.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.638+0000 7f867d19d700 1 -- 172.21.15.55:0/1343785490 wait complete. 2023-10-06T03:33:12.659 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:33:12.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.846+0000 7f4a6139d700 1 Processor -- start 2023-10-06T03:33:12.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.847+0000 7f4a6139d700 1 -- start start 2023-10-06T03:33:12.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.848+0000 7f4a6139d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a5c0b02a0 0x7f4a5c15bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:12.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.848+0000 7f4a6139d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c06ae70 con 0x7f4a5c06d950 2023-10-06T03:33:12.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.848+0000 7f4a6139d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c0c8a60 con 0x7f4a5c06a0a0 2023-10-06T03:33:12.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.848+0000 7f4a6139d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c06a540 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.848+0000 7f4a5bfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4a5c06d950 0x7f4a5c0c8350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:53530/0 (socket says 172.21.15.55:53530) 2023-10-06T03:33:12.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.848+0000 7f4a5bfff700 1 -- 172.21.15.55:0/1243163913 learned_addr learned my addr 172.21.15.55:0/1243163913 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:12.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.848+0000 7f4a60b9c700 1 --2- 172.21.15.55:0/1243163913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a5c0b02a0 0x7f4a5c15bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:12.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.849+0000 7f4a5affd700 1 -- 172.21.15.55:0/1243163913 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386049538 0 0) 0x7f4a5c06ae70 con 0x7f4a5c06d950 2023-10-06T03:33:12.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.849+0000 7f4a5affd700 1 -- 172.21.15.55:0/1243163913 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a44003570 con 0x7f4a5c06d950 2023-10-06T03:33:12.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.849+0000 7f4a60b9c700 1 -- 172.21.15.55:0/1243163913 >> v1:172.21.15.163:6789/0 conn(0x7f4a5c06a0a0 legacy=0x7f4a5c15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:12.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.849+0000 7f4a60b9c700 1 -- 172.21.15.55:0/1243163913 >> v1:172.21.15.55:6789/0 conn(0x7f4a5c06d950 legacy=0x7f4a5c0c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:12.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.849+0000 7f4a60b9c700 1 -- 172.21.15.55:0/1243163913 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c06a820 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.849+0000 7f4a60b9c700 1 --2- 172.21.15.55:0/1243163913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a5c0b02a0 0x7f4a5c15bb20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009e90 tx=0x7f4a54009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11a208c903810f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:12.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.858+0000 7f4a5affd700 1 -- 172.21.15.55:0/1243163913 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400e070 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.858+0000 7f4a5affd700 1 -- 172.21.15.55:0/1243163913 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5400bc30 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.858+0000 7f4a5affd700 1 -- 172.21.15.55:0/1243163913 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54012430 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.858+0000 7f4a6139d700 1 -- 172.21.15.55:0/1243163913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a5c0b02a0 msgr2=0x7f4a5c15bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:12.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.858+0000 7f4a6139d700 1 --2- 172.21.15.55:0/1243163913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a5c0b02a0 0x7f4a5c15bb20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009e90 tx=0x7f4a54009780 comp rx=0 tx=0).stop 2023-10-06T03:33:12.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.859+0000 7f4a6139d700 1 -- 172.21.15.55:0/1243163913 shutdown_connections 2023-10-06T03:33:12.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.859+0000 7f4a6139d700 1 --2- 172.21.15.55:0/1243163913 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a5c0b02a0 0x7f4a5c15bb20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.859+0000 7f4a6139d700 1 -- 172.21.15.55:0/1243163913 >> 172.21.15.55:0/1243163913 conn(0x7f4a5c0cc1f0 msgr2=0x7f4a5c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:12.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.859+0000 7f4a6139d700 1 -- 172.21.15.55:0/1243163913 shutdown_connections 2023-10-06T03:33:12.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.859+0000 7f4a6139d700 1 -- 172.21.15.55:0/1243163913 wait complete. 2023-10-06T03:33:12.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.860+0000 7f4a6139d700 1 Processor -- start 2023-10-06T03:33:12.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.860+0000 7f4a6139d700 1 -- start start 2023-10-06T03:33:12.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.861+0000 7f4a6139d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a5c06a0a0 0x7f4a5c150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:12.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.861+0000 7f4a6139d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a5c06d950 0x7f4a5c0c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:12.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.861+0000 7f4a6139d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a5c0b02a0 0x7f4a5c1521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:12.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.861+0000 7f4a6139d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c0c3d20 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.861+0000 7f4a6139d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c0c4b40 con 0x7f4a5c06a0a0 2023-10-06T03:33:12.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.861+0000 7f4a6139d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c150cc0 con 0x7f4a5c06d950 2023-10-06T03:33:12.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.862+0000 7f4a60b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a5c0b02a0 0x7f4a5c1521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:12.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.862+0000 7f4a60b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a5c0b02a0 0x7f4a5c1521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34088/0 (socket says 172.21.15.55:34088) 2023-10-06T03:33:12.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.862+0000 7f4a60b9c700 1 -- 172.21.15.55:0/2674855490 learned_addr learned my addr 172.21.15.55:0/2674855490 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.862+0000 7f4a5b7fe700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a5c06d950 0x7f4a5c0c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.862+0000 7f4a5bfff700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a5c06a0a0 0x7f4a5c150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.862+0000 7f4a60b9c700 1 -- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a5c06d950 msgr2=0x7f4a5c0c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.862+0000 7f4a60b9c700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a5c06d950 0x7f4a5c0c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.862+0000 7f4a60b9c700 1 -- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a5c06a0a0 msgr2=0x7f4a5c150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.862+0000 7f4a60b9c700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a5c06a0a0 0x7f4a5c150780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.862+0000 7f4a60b9c700 1 -- 172.21.15.55:0/2674855490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a54009a20 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.862+0000 7f4a5b7fe700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a5c06d950 0x7f4a5c0c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.862+0000 7f4a5bfff700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a5c06a0a0 0x7f4a5c150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.862+0000 7f4a60b9c700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a5c0b02a0 0x7f4a5c1521f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a54013000 tx=0x7f4a54019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.862+0000 7f4a43fff700 1 -- 172.21.15.55:0/2674855490 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54012ea0 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.863+0000 7f4a43fff700 1 -- 172.21.15.55:0/2674855490 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5401d630 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.863+0000 7f4a43fff700 1 -- 172.21.15.55:0/2674855490 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5401d9d0 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.863+0000 7f4a6139d700 1 -- 172.21.15.55:0/2674855490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5c0c3f30 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.863+0000 7f4a6139d700 1 -- 172.21.15.55:0/2674855490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5c0c44f0 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.864+0000 7f4a6139d700 1 -- 172.21.15.55:0/2674855490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5c0c4cb0 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.865+0000 7f4a43fff700 1 -- 172.21.15.55:0/2674855490 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a54024070 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.865+0000 7f4a43fff700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4a48066f30 0x7f4a480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:12.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.865+0000 7f4a43fff700 1 -- 172.21.15.55:0/2674855490 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4a54086d60 con 0x7f4a5c0b02a0 2023-10-06T03:33:12.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.866+0000 7f4a5bfff700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4a48066f30 0x7f4a480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:12.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.866+0000 7f4a5bfff700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4a48066f30 0x7f4a480693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c007fd0 tx=0x7f4a4c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:12.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:12.870+0000 7f4a43fff700 1 -- 172.21.15.55:0/2674855490 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a54054120 con 0x7f4a5c0b02a0 2023-10-06T03:33:13.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.074+0000 7f4a6139d700 1 -- 172.21.15.55:0/2674855490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4a5c04c3c0 con 0x7f4a5c0b02a0 2023-10-06T03:33:13.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.074+0000 7f4a43fff700 1 -- 172.21.15.55:0/2674855490 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7f4a5402c090 con 0x7f4a5c0b02a0 2023-10-06T03:33:13.076 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:13.076 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:33:13.076 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:33:13.076 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2023-10-06T03:33:13.076 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": "rbd" 2023-10-06T03:33:13.076 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:33:13.076 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:33:13.077 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 2, 2023-10-06T03:33:13.077 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2023-10-06T03:33:13.077 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:33:13.077 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:33:13.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.077+0000 7f4a6139d700 1 -- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4a48066f30 msgr2=0x7f4a480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.077+0000 7f4a6139d700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4a48066f30 0x7f4a480693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c007fd0 tx=0x7f4a4c009040 comp rx=0 tx=0).stop 2023-10-06T03:33:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.077+0000 7f4a6139d700 1 -- 172.21.15.55:0/2674855490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a5c0b02a0 msgr2=0x7f4a5c1521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.077+0000 7f4a6139d700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a5c0b02a0 0x7f4a5c1521f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a54013000 tx=0x7f4a54019000 comp rx=0 tx=0).stop 2023-10-06T03:33:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.077+0000 7f4a6139d700 1 -- 172.21.15.55:0/2674855490 shutdown_connections 2023-10-06T03:33:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.077+0000 7f4a6139d700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a5c0b02a0 0x7f4a5c1521f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.077+0000 7f4a6139d700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a5c06a0a0 0x7f4a5c150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.077+0000 7f4a6139d700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4a48066f30 0x7f4a480693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.077+0000 7f4a6139d700 1 --2- 172.21.15.55:0/2674855490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a5c06d950 0x7f4a5c0c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.077+0000 7f4a6139d700 1 -- 172.21.15.55:0/2674855490 >> 172.21.15.55:0/2674855490 conn(0x7f4a5c0cc1f0 msgr2=0x7f4a5c0c7070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.077+0000 7f4a6139d700 1 -- 172.21.15.55:0/2674855490 shutdown_connections 2023-10-06T03:33:13.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.077+0000 7f4a6139d700 1 -- 172.21.15.55:0/2674855490 wait complete. 2023-10-06T03:33:13.096 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:33:13.096 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:33:13.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.286+0000 7fc4e759e700 1 Processor -- start 2023-10-06T03:33:13.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.286+0000 7fc4e759e700 1 -- start start 2023-10-06T03:33:13.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.288+0000 7fc4e759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e815b050 0x7fc4e80ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:13.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.288+0000 7fc4e759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e8157a50 con 0x7fc4e815bb30 2023-10-06T03:33:13.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.288+0000 7fc4e759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e8158f10 con 0x7fc4e80cc8d0 2023-10-06T03:33:13.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.288+0000 7fc4e759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e81591b0 con 0x7fc4e815b050 2023-10-06T03:33:13.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.288+0000 7fc4e6d9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc4e815bb30 0x7fc4e80ada30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56188/0 (socket says 172.21.15.55:56188) 2023-10-06T03:33:13.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.288+0000 7fc4e6d9d700 1 -- 172.21.15.55:0/2778338131 learned_addr learned my addr 172.21.15.55:0/2778338131 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:13.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.288+0000 7fc4e659c700 1 --2- 172.21.15.55:0/2778338131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e815b050 0x7fc4e80ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:13.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.288+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585095082 0 0) 0x7fc4e8157a50 con 0x7fc4e815bb30 2023-10-06T03:33:13.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.288+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4d0003570 con 0x7fc4e815bb30 2023-10-06T03:33:13.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955868574 0 0) 0x7fc4e8158f10 con 0x7fc4e80cc8d0 2023-10-06T03:33:13.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4e8157a50 con 0x7fc4e80cc8d0 2023-10-06T03:33:13.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 850661520 0 0) 0x7fc4d0003570 con 0x7fc4e815bb30 2023-10-06T03:33:13.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e815b050 msgr2=0x7fc4e80ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:13.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 --2- 172.21.15.55:0/2778338131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e815b050 0x7fc4e80ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:13.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 >> v1:172.21.15.163:6789/0 conn(0x7fc4e80cc8d0 legacy=0x7fc4e815b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:13.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e80ca9e0 con 0x7fc4e815bb30 2023-10-06T03:33:13.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fc4e00017f0 con 0x7fc4e815bb30 2023-10-06T03:33:13.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e659c700 1 --2- 172.21.15.55:0/2778338131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e815b050 0x7fc4e80ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:13.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 >> v1:172.21.15.55:6789/0 conn(0x7fc4e815bb30 legacy=0x7fc4e80ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:13.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 --2- 172.21.15.55:0/2778338131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e815b050 0x7fc4d0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:13.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 --2- 172.21.15.55:0/2778338131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4d00041b0 0x7fc4d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:13.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 --2- 172.21.15.55:0/2778338131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4d0004d50 0x7fc4d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:13.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e81591b0 con 0x7fc4d00041b0 2023-10-06T03:33:13.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d000bbf0 con 0x7fc4d0004d50 2023-10-06T03:33:13.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d000bdb0 con 0x7fc4e815b050 2023-10-06T03:33:13.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e659c700 1 --2- 172.21.15.55:0/2778338131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e815b050 0x7fc4d0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:13.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e5d9b700 1 --2- 172.21.15.55:0/2778338131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4d00041b0 0x7fc4d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:13.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.289+0000 7fc4e6d9d700 1 --2- 172.21.15.55:0/2778338131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4d0004d50 0x7fc4d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:13.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.290+0000 7fc4e659c700 1 -- 172.21.15.55:0/2778338131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4d0004d50 msgr2=0x7fc4d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:13.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.290+0000 7fc4e659c700 1 --2- 172.21.15.55:0/2778338131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4d0004d50 0x7fc4d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:13.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.290+0000 7fc4e659c700 1 -- 172.21.15.55:0/2778338131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4d00041b0 msgr2=0x7fc4d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:13.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.290+0000 7fc4e659c700 1 --2- 172.21.15.55:0/2778338131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4d00041b0 0x7fc4d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:13.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.290+0000 7fc4e659c700 1 -- 172.21.15.55:0/2778338131 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d8009420 con 0x7fc4e815b050 2023-10-06T03:33:13.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.290+0000 7fc4e5d9b700 1 --2- 172.21.15.55:0/2778338131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4d00041b0 0x7fc4d0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:13.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.290+0000 7fc4e659c700 1 --2- 172.21.15.55:0/2778338131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e815b050 0x7fc4d0003c20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009db0 tx=0x7fc4d80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a762164593c2b979 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:13.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.290+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8009740 con 0x7fc4e815b050 2023-10-06T03:33:13.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.290+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d800e920 con 0x7fc4e815b050 2023-10-06T03:33:13.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.290+0000 7fc4e559a700 1 -- 172.21.15.55:0/2778338131 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800ece0 con 0x7fc4e815b050 2023-10-06T03:33:13.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.291+0000 7fc4e759e700 1 -- 172.21.15.55:0/2778338131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e815b050 msgr2=0x7fc4d0003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:13.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.291+0000 7fc4e759e700 1 --2- 172.21.15.55:0/2778338131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e815b050 0x7fc4d0003c20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009db0 tx=0x7fc4d80091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:13.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.291+0000 7fc4e6d9d700 1 -- 172.21.15.55:0/2778338131 reap_dead start 2023-10-06T03:33:13.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.291+0000 7fc4e759e700 1 -- 172.21.15.55:0/2778338131 shutdown_connections 2023-10-06T03:33:13.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.291+0000 7fc4e759e700 1 -- 172.21.15.55:0/2778338131 >> 172.21.15.55:0/2778338131 conn(0x7fc4e8068e20 msgr2=0x7fc4e80cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:13.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.291+0000 7fc4e759e700 1 -- 172.21.15.55:0/2778338131 shutdown_connections 2023-10-06T03:33:13.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.291+0000 7fc4e759e700 1 -- 172.21.15.55:0/2778338131 wait complete. 2023-10-06T03:33:13.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.292+0000 7fc4e759e700 1 Processor -- start 2023-10-06T03:33:13.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.292+0000 7fc4e759e700 1 -- start start 2023-10-06T03:33:13.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.293+0000 7fc4e759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4e815b050 0x7fc4e813cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:13.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.293+0000 7fc4e759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4e814e150 0x7fc4e814fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:13.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.293+0000 7fc4e759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e814e5c0 0x7fc4e814bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:13.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.293+0000 7fc4e759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e814f740 con 0x7fc4e814e150 2023-10-06T03:33:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.293+0000 7fc4e759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e814f0f0 con 0x7fc4e815b050 2023-10-06T03:33:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.293+0000 7fc4e759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e814eb30 con 0x7fc4e814e5c0 2023-10-06T03:33:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.293+0000 7fc4e5d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4e814e150 0x7fc4e814fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.293+0000 7fc4e5d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4e814e150 0x7fc4e814fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40856/0 (socket says 172.21.15.55:40856) 2023-10-06T03:33:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.293+0000 7fc4e5d9b700 1 -- 172.21.15.55:0/2854742124 learned_addr learned my addr 172.21.15.55:0/2854742124 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4e6d9d700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e814e5c0 0x7fc4e814bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4e659c700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4e815b050 0x7fc4e813cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4e5d9b700 1 -- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e814e5c0 msgr2=0x7fc4e814bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4e5d9b700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e814e5c0 0x7fc4e814bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4e5d9b700 1 -- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4e815b050 msgr2=0x7fc4e813cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:13.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4e5d9b700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4e815b050 0x7fc4e813cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4e5d9b700 1 -- 172.21.15.55:0/2854742124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e814ee10 con 0x7fc4e814e150 2023-10-06T03:33:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4e6d9d700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e814e5c0 0x7fc4e814bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4e659c700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4e815b050 0x7fc4e813cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4e5d9b700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4e814e150 0x7fc4e814fbd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00fc00 tx=0x7fc4dc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4d67fc700 1 -- 172.21.15.55:0/2854742124 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc00d920 con 0x7fc4e814e150 2023-10-06T03:33:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4d67fc700 1 -- 172.21.15.55:0/2854742124 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc00da80 con 0x7fc4e814e150 2023-10-06T03:33:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4d67fc700 1 -- 172.21.15.55:0/2854742124 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc00de00 con 0x7fc4e814e150 2023-10-06T03:33:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4e759e700 1 -- 172.21.15.55:0/2854742124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e814f3d0 con 0x7fc4e814e150 2023-10-06T03:33:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.294+0000 7fc4e759e700 1 -- 172.21.15.55:0/2854742124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e8167cc0 con 0x7fc4e814e150 2023-10-06T03:33:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.295+0000 7fc4e759e700 1 -- 172.21.15.55:0/2854742124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e804c3c0 con 0x7fc4e814e150 2023-10-06T03:33:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.296+0000 7fc4d67fc700 1 -- 172.21.15.55:0/2854742124 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc4dc006a30 con 0x7fc4e814e150 2023-10-06T03:33:13.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.298+0000 7fc4d67fc700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc4c8066f30 0x7fc4c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:13.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.298+0000 7fc4d67fc700 1 -- 172.21.15.55:0/2854742124 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc4dc084490 con 0x7fc4e814e150 2023-10-06T03:33:13.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.298+0000 7fc4e659c700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc4c8066f30 0x7fc4c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:13.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.299+0000 7fc4e659c700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc4c8066f30 0x7fc4c80693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8006ed0 tx=0x7fc4d8019000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:13.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.302+0000 7fc4d67fc700 1 -- 172.21.15.55:0/2854742124 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4dc051850 con 0x7fc4e814e150 2023-10-06T03:33:13.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.504+0000 7fc4e759e700 1 -- 172.21.15.55:0/2854742124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc4e80cacc0 con 0x7fc4e814e150 2023-10-06T03:33:13.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.863+0000 7fc4d67fc700 1 -- 172.21.15.55:0/2854742124 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc4dc054e70 con 0x7fc4e814e150 2023-10-06T03:33:13.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.866+0000 7fc4e759e700 1 -- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc4c8066f30 msgr2=0x7fc4c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:13.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.866+0000 7fc4e759e700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc4c8066f30 0x7fc4c80693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8006ed0 tx=0x7fc4d8019000 comp rx=0 tx=0).stop 2023-10-06T03:33:13.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.866+0000 7fc4e759e700 1 -- 172.21.15.55:0/2854742124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4e814e150 msgr2=0x7fc4e814fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:13.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.866+0000 7fc4e759e700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4e814e150 0x7fc4e814fbd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00fc00 tx=0x7fc4dc007d30 comp rx=0 tx=0).stop 2023-10-06T03:33:13.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.866+0000 7fc4e759e700 1 -- 172.21.15.55:0/2854742124 shutdown_connections 2023-10-06T03:33:13.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.866+0000 7fc4e759e700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc4c8066f30 0x7fc4c80693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:13.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.866+0000 7fc4e759e700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4e814e5c0 0x7fc4e814bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:13.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.866+0000 7fc4e759e700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4e815b050 0x7fc4e813cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:13.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.866+0000 7fc4e759e700 1 --2- 172.21.15.55:0/2854742124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4e814e150 0x7fc4e814fbd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:13.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.866+0000 7fc4e759e700 1 -- 172.21.15.55:0/2854742124 >> 172.21.15.55:0/2854742124 conn(0x7fc4e8068e20 msgr2=0x7fc4e815b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:13.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.866+0000 7fc4e759e700 1 -- 172.21.15.55:0/2854742124 shutdown_connections 2023-10-06T03:33:13.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:13.866+0000 7fc4e759e700 1 -- 172.21.15.55:0/2854742124 wait complete. 2023-10-06T03:33:13.870 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2023-10-06T03:33:13.891 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:33:14.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.085+0000 7f480cd9d700 1 Processor -- start 2023-10-06T03:33:14.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.085+0000 7f480cd9d700 1 -- start start 2023-10-06T03:33:14.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.086+0000 7f480cd9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f480806d840 0x7f48080c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:14.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.086+0000 7f480cd9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f480806b360 0x7f48080ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:14.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.086+0000 7f480cd9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48080c9860 con 0x7f480806af90 2023-10-06T03:33:14.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.086+0000 7f480cd9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48080c8cd0 con 0x7f480806d840 2023-10-06T03:33:14.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.086+0000 7f480cd9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48080cd200 con 0x7f480806b360 2023-10-06T03:33:14.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.087+0000 7f4806ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f480806d840 0x7f48080c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:14.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.087+0000 7f4807fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f480806b360 0x7f48080ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:14.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.087+0000 7f4806ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f480806d840 0x7f48080c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:47946/0 (socket says 172.21.15.55:47946) 2023-10-06T03:33:14.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.087+0000 7f4806ffd700 1 -- 172.21.15.55:0/2959882130 learned_addr learned my addr 172.21.15.55:0/2959882130 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:14.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.087+0000 7f4807fff700 1 -- 172.21.15.55:0/2959882130 >> v2:172.21.15.163:3300/0 conn(0x7f480806d840 msgr2=0x7f48080c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:14.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.087+0000 7f4807fff700 1 --2- 172.21.15.55:0/2959882130 >> v2:172.21.15.163:3300/0 conn(0x7f480806d840 0x7f48080c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:14.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.087+0000 7f4807fff700 1 -- 172.21.15.55:0/2959882130 >> v1:172.21.15.163:6789/0 conn(0x7f480806af90 legacy=0x7f480806d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:14.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.087+0000 7f4807fff700 1 -- 172.21.15.55:0/2959882130 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48080c8f30 con 0x7f480806b360 2023-10-06T03:33:14.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.087+0000 7f4806ffd700 1 --2- 172.21.15.55:0/2959882130 >> v2:172.21.15.163:3300/0 conn(0x7f480806d840 0x7f48080c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:14.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.087+0000 7f4807fff700 1 --2- 172.21.15.55:0/2959882130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f480806b360 0x7f48080ccb00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4800009e60 tx=0x7f4800009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6feab04d721059f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:14.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.088+0000 7f48067fc700 1 -- 172.21.15.55:0/2959882130 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480000e070 con 0x7f480806b360 2023-10-06T03:33:14.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.088+0000 7f48067fc700 1 -- 172.21.15.55:0/2959882130 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480000bbf0 con 0x7f480806b360 2023-10-06T03:33:14.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.088+0000 7f48067fc700 1 -- 172.21.15.55:0/2959882130 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800012430 con 0x7f480806b360 2023-10-06T03:33:14.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.088+0000 7f480cd9d700 1 -- 172.21.15.55:0/2959882130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f480806b360 msgr2=0x7f48080ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:14.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.088+0000 7f480cd9d700 1 --2- 172.21.15.55:0/2959882130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f480806b360 0x7f48080ccb00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4800009e60 tx=0x7f4800009780 comp rx=0 tx=0).stop 2023-10-06T03:33:14.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.089+0000 7f480cd9d700 1 -- 172.21.15.55:0/2959882130 shutdown_connections 2023-10-06T03:33:14.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.089+0000 7f480cd9d700 1 --2- 172.21.15.55:0/2959882130 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f480806b360 0x7f48080ccb00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:14.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.089+0000 7f480cd9d700 1 --2- 172.21.15.55:0/2959882130 >> v2:172.21.15.163:3300/0 conn(0x7f480806d840 0x7f48080c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:14.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.089+0000 7f480cd9d700 1 -- 172.21.15.55:0/2959882130 >> 172.21.15.55:0/2959882130 conn(0x7f480806baa0 msgr2=0x7f480806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:14.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.089+0000 7f480cd9d700 1 -- 172.21.15.55:0/2959882130 shutdown_connections 2023-10-06T03:33:14.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.089+0000 7f480cd9d700 1 -- 172.21.15.55:0/2959882130 wait complete. 2023-10-06T03:33:14.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.090+0000 7f480cd9d700 1 Processor -- start 2023-10-06T03:33:14.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.090+0000 7f480cd9d700 1 -- start start 2023-10-06T03:33:14.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.091+0000 7f480cd9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f480806af90 0x7f48080b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:14.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.091+0000 7f480cd9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f480806b360 0x7f48080c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:14.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.091+0000 7f480cd9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f480806d840 0x7f48080c72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:14.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.091+0000 7f480cd9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48080b1e10 con 0x7f480806b360 2023-10-06T03:33:14.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.091+0000 7f480cd9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48080b20f0 con 0x7f480806af90 2023-10-06T03:33:14.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.091+0000 7f480cd9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48080b23d0 con 0x7f480806d840 2023-10-06T03:33:14.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.091+0000 7f4806ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f480806b360 0x7f48080c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:14.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.091+0000 7f4806ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f480806b360 0x7f48080c6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40872/0 (socket says 172.21.15.55:40872) 2023-10-06T03:33:14.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.091+0000 7f4806ffd700 1 -- 172.21.15.55:0/4170237156 learned_addr learned my addr 172.21.15.55:0/4170237156 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:14.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.092+0000 7f4807fff700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f480806d840 0x7f48080c72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:14.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.091+0000 7f48077fe700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f480806af90 0x7f48080b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:14.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.092+0000 7f4806ffd700 1 -- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f480806d840 msgr2=0x7f48080c72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:14.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.092+0000 7f4806ffd700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f480806d840 0x7f48080c72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:14.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.092+0000 7f4806ffd700 1 -- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f480806af90 msgr2=0x7f48080b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:14.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.092+0000 7f4806ffd700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f480806af90 0x7f48080b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:14.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.092+0000 7f4806ffd700 1 -- 172.21.15.55:0/4170237156 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc0094f0 con 0x7f480806b360 2023-10-06T03:33:14.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.092+0000 7f4807fff700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f480806d840 0x7f48080c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:14.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.092+0000 7f4806ffd700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f480806b360 0x7f48080c6d90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00ec50 tx=0x7f47fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:14.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.092+0000 7f47ef7fe700 1 -- 172.21.15.55:0/4170237156 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc00c9d0 con 0x7f480806b360 2023-10-06T03:33:14.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.092+0000 7f480cd9d700 1 -- 172.21.15.55:0/4170237156 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4800009a20 con 0x7f480806b360 2023-10-06T03:33:14.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.092+0000 7f47ef7fe700 1 -- 172.21.15.55:0/4170237156 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc00cb30 con 0x7f480806b360 2023-10-06T03:33:14.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.092+0000 7f47ef7fe700 1 -- 172.21.15.55:0/4170237156 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc013470 con 0x7f480806b360 2023-10-06T03:33:14.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.092+0000 7f480cd9d700 1 -- 172.21.15.55:0/4170237156 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48080b2990 con 0x7f480806b360 2023-10-06T03:33:14.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.093+0000 7f480cd9d700 1 -- 172.21.15.55:0/4170237156 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480804c3c0 con 0x7f480806b360 2023-10-06T03:33:14.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.095+0000 7f47ef7fe700 1 -- 172.21.15.55:0/4170237156 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f47fc013860 con 0x7f480806b360 2023-10-06T03:33:14.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.095+0000 7f47ef7fe700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f47f4066f30 0x7f47f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:14.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.096+0000 7f47ef7fe700 1 -- 172.21.15.55:0/4170237156 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f47fc085150 con 0x7f480806b360 2023-10-06T03:33:14.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.096+0000 7f48077fe700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f47f4066f30 0x7f47f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:14.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.097+0000 7f48077fe700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f47f4066f30 0x7f47f40693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47f8003860 tx=0x7f47f8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:14.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.099+0000 7f47ef7fe700 1 -- 172.21.15.55:0/4170237156 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47fc0523a0 con 0x7f480806b360 2023-10-06T03:33:14.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.302+0000 7f480cd9d700 1 -- 172.21.15.55:0/4170237156 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4808065910 con 0x7f480806b360 2023-10-06T03:33:14.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.874+0000 7f47ef7fe700 1 -- 172.21.15.55:0/4170237156 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f47fc0170d0 con 0x7f480806b360 2023-10-06T03:33:14.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.894+0000 7f480cd9d700 1 -- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f47f4066f30 msgr2=0x7f47f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:14.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.894+0000 7f480cd9d700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f47f4066f30 0x7f47f40693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47f8003860 tx=0x7f47f8009040 comp rx=0 tx=0).stop 2023-10-06T03:33:14.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.894+0000 7f480cd9d700 1 -- 172.21.15.55:0/4170237156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f480806b360 msgr2=0x7f48080c6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:14.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.894+0000 7f480cd9d700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f480806b360 0x7f48080c6d90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00ec50 tx=0x7f47fc009250 comp rx=0 tx=0).stop 2023-10-06T03:33:14.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.895+0000 7f480cd9d700 1 -- 172.21.15.55:0/4170237156 shutdown_connections 2023-10-06T03:33:14.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.895+0000 7f480cd9d700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f47f4066f30 0x7f47f40693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:14.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.895+0000 7f480cd9d700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f480806d840 0x7f48080c72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:14.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.895+0000 7f480cd9d700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f480806af90 0x7f48080b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:14.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.895+0000 7f480cd9d700 1 --2- 172.21.15.55:0/4170237156 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f480806b360 0x7f48080c6d90 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00ec50 tx=0x7f47fc009250 comp rx=0 tx=0).stop 2023-10-06T03:33:14.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.895+0000 7f480cd9d700 1 -- 172.21.15.55:0/4170237156 >> 172.21.15.55:0/4170237156 conn(0x7f480806baa0 msgr2=0x7f48080c3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:14.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.898+0000 7f480cd9d700 1 -- 172.21.15.55:0/4170237156 shutdown_connections 2023-10-06T03:33:14.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:14.900+0000 7f480cd9d700 1 -- 172.21.15.55:0/4170237156 wait complete. 2023-10-06T03:33:14.904 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2023-10-06T03:33:14.923 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:33:15.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.115+0000 7fe36959d700 1 Processor -- start 2023-10-06T03:33:15.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.115+0000 7fe36959d700 1 -- start start 2023-10-06T03:33:15.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.117+0000 7fe36959d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe3641407e0 0x7fe36406a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:15.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.117+0000 7fe36959d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe36406dc00 0x7fe36406a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:15.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.117+0000 7fe36959d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3640c6820 con 0x7fe364140bb0 2023-10-06T03:33:15.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.117+0000 7fe36959d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3640c6de0 con 0x7fe36406dc00 2023-10-06T03:33:15.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.117+0000 7fe36959d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe364141270 con 0x7fe3641407e0 2023-10-06T03:33:15.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.117+0000 7fe368d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe36406dc00 0x7fe36406a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:15.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.117+0000 7fe368d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe36406dc00 0x7fe36406a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40888/0 (socket says 172.21.15.55:40888) 2023-10-06T03:33:15.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.117+0000 7fe368d9c700 1 -- 172.21.15.55:0/3221125839 learned_addr learned my addr 172.21.15.55:0/3221125839 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:15.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.117+0000 7fe363fff700 1 --2- 172.21.15.55:0/3221125839 >> v2:172.21.15.163:3300/0 conn(0x7fe3641407e0 0x7fe36406a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:15.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.117+0000 7fe368d9c700 1 -- 172.21.15.55:0/3221125839 >> v2:172.21.15.163:3300/0 conn(0x7fe3641407e0 msgr2=0x7fe36406a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:15.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.117+0000 7fe368d9c700 1 --2- 172.21.15.55:0/3221125839 >> v2:172.21.15.163:3300/0 conn(0x7fe3641407e0 0x7fe36406a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:15.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.117+0000 7fe368d9c700 1 -- 172.21.15.55:0/3221125839 >> v1:172.21.15.55:6789/0 conn(0x7fe364140bb0 legacy=0x7fe36406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:15.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.117+0000 7fe368d9c700 1 -- 172.21.15.55:0/3221125839 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36406acf0 con 0x7fe36406dc00 2023-10-06T03:33:15.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.117+0000 7fe362ffd700 1 -- 172.21.15.55:0/3221125839 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621931416 0 0) 0x7fe3640c6820 con 0x7fe364140bb0 2023-10-06T03:33:15.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.118+0000 7fe363fff700 1 --2- 172.21.15.55:0/3221125839 >> v2:172.21.15.163:3300/0 conn(0x7fe3641407e0 0x7fe36406a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:15.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.118+0000 7fe368d9c700 1 --2- 172.21.15.55:0/3221125839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe36406dc00 0x7fe36406a6c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe36406f9a0 tx=0x7fe35c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0768d766cabee13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:15.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.118+0000 7fe362ffd700 1 -- 172.21.15.55:0/3221125839 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00e070 con 0x7fe36406dc00 2023-10-06T03:33:15.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.118+0000 7fe362ffd700 1 -- 172.21.15.55:0/3221125839 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c00ba50 con 0x7fe36406dc00 2023-10-06T03:33:15.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.118+0000 7fe362ffd700 1 -- 172.21.15.55:0/3221125839 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c0092e0 con 0x7fe36406dc00 2023-10-06T03:33:15.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.118+0000 7fe36959d700 1 -- 172.21.15.55:0/3221125839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe36406dc00 msgr2=0x7fe36406a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:15.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.118+0000 7fe36959d700 1 --2- 172.21.15.55:0/3221125839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe36406dc00 0x7fe36406a6c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe36406f9a0 tx=0x7fe35c009840 comp rx=0 tx=0).stop 2023-10-06T03:33:15.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.119+0000 7fe36959d700 1 -- 172.21.15.55:0/3221125839 shutdown_connections 2023-10-06T03:33:15.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.119+0000 7fe36959d700 1 --2- 172.21.15.55:0/3221125839 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe36406dc00 0x7fe36406a6c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:15.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.119+0000 7fe36959d700 1 --2- 172.21.15.55:0/3221125839 >> v2:172.21.15.163:3300/0 conn(0x7fe3641407e0 0x7fe36406a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.119+0000 7fe36959d700 1 -- 172.21.15.55:0/3221125839 >> 172.21.15.55:0/3221125839 conn(0x7fe3640cc1e0 msgr2=0x7fe3640cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.119+0000 7fe36959d700 1 -- 172.21.15.55:0/3221125839 shutdown_connections 2023-10-06T03:33:15.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.119+0000 7fe36959d700 1 -- 172.21.15.55:0/3221125839 wait complete. 2023-10-06T03:33:15.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.120+0000 7fe36959d700 1 Processor -- start 2023-10-06T03:33:15.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.120+0000 7fe36959d700 1 -- start start 2023-10-06T03:33:15.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.121+0000 7fe36959d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe36406dc00 0x7fe36415ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:15.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.121+0000 7fe36959d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3641407e0 0x7fe364157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:15.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.121+0000 7fe36959d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe364140bb0 0x7fe3640c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:15.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.121+0000 7fe36959d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3640b1610 con 0x7fe364140bb0 2023-10-06T03:33:15.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.121+0000 7fe36959d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3640b18f0 con 0x7fe36406dc00 2023-10-06T03:33:15.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.121+0000 7fe36959d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3640b1bd0 con 0x7fe3641407e0 2023-10-06T03:33:15.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.121+0000 7fe368d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe364140bb0 0x7fe3640c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:15.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.122+0000 7fe368d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe364140bb0 0x7fe3640c46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40900/0 (socket says 172.21.15.55:40900) 2023-10-06T03:33:15.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.122+0000 7fe368d9c700 1 -- 172.21.15.55:0/2451407906 learned_addr learned my addr 172.21.15.55:0/2451407906 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:15.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.122+0000 7fe363fff700 1 --2- 172.21.15.55:0/2451407906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe36406dc00 0x7fe36415ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:15.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.122+0000 7fe3637fe700 1 --2- 172.21.15.55:0/2451407906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3641407e0 0x7fe364157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:15.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.122+0000 7fe368d9c700 1 -- 172.21.15.55:0/2451407906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3641407e0 msgr2=0x7fe364157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:15.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.122+0000 7fe368d9c700 1 --2- 172.21.15.55:0/2451407906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3641407e0 0x7fe364157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:15.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.122+0000 7fe368d9c700 1 -- 172.21.15.55:0/2451407906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe36406dc00 msgr2=0x7fe36415ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:15.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.122+0000 7fe368d9c700 1 --2- 172.21.15.55:0/2451407906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe36406dc00 0x7fe36415ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:15.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.122+0000 7fe368d9c700 1 -- 172.21.15.55:0/2451407906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c009ae0 con 0x7fe364140bb0 2023-10-06T03:33:15.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.122+0000 7fe368d9c700 1 --2- 172.21.15.55:0/2451407906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe364140bb0 0x7fe3640c46e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe35c012000 tx=0x7fe35c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:15.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.122+0000 7fe34bfff700 1 -- 172.21.15.55:0/2451407906 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c008e10 con 0x7fe364140bb0 2023-10-06T03:33:15.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.122+0000 7fe34bfff700 1 -- 172.21.15.55:0/2451407906 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c026460 con 0x7fe364140bb0 2023-10-06T03:33:15.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.122+0000 7fe36959d700 1 -- 172.21.15.55:0/2451407906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36415c150 con 0x7fe364140bb0 2023-10-06T03:33:15.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.122+0000 7fe34bfff700 1 -- 172.21.15.55:0/2451407906 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c026820 con 0x7fe364140bb0 2023-10-06T03:33:15.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.123+0000 7fe36959d700 1 -- 172.21.15.55:0/2451407906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe364157550 con 0x7fe364140bb0 2023-10-06T03:33:15.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.123+0000 7fe36959d700 1 -- 172.21.15.55:0/2451407906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36415c670 con 0x7fe364140bb0 2023-10-06T03:33:15.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.124+0000 7fe34bfff700 1 -- 172.21.15.55:0/2451407906 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe35c003040 con 0x7fe364140bb0 2023-10-06T03:33:15.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.125+0000 7fe34bfff700 1 --2- 172.21.15.55:0/2451407906 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe350066f30 0x7fe3500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:15.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.125+0000 7fe34bfff700 1 -- 172.21.15.55:0/2451407906 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fe35c086700 con 0x7fe364140bb0 2023-10-06T03:33:15.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.126+0000 7fe363fff700 1 --2- 172.21.15.55:0/2451407906 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe350066f30 0x7fe3500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:15.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.126+0000 7fe363fff700 1 --2- 172.21.15.55:0/2451407906 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe350066f30 0x7fe3500693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe35400adc0 tx=0x7fe354006f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:15.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.130+0000 7fe34bfff700 1 -- 172.21.15.55:0/2451407906 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe36415c670 con 0x7fe364140bb0 2023-10-06T03:33:15.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.381+0000 7fe36959d700 1 -- 172.21.15.55:0/2451407906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe36415c670 con 0x7fe364140bb0 2023-10-06T03:33:15.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.870+0000 7fe34bfff700 1 -- 172.21.15.55:0/2451407906 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe36415c670 con 0x7fe364140bb0 2023-10-06T03:33:15.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.872+0000 7fe36959d700 1 -- 172.21.15.55:0/2451407906 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe350066f30 msgr2=0x7fe3500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:15.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.872+0000 7fe36959d700 1 --2- 172.21.15.55:0/2451407906 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe350066f30 0x7fe3500693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe35400adc0 tx=0x7fe354006f90 comp rx=0 tx=0).stop 2023-10-06T03:33:15.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.872+0000 7fe36959d700 1 -- 172.21.15.55:0/2451407906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe364140bb0 msgr2=0x7fe3640c46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:15.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.872+0000 7fe36959d700 1 --2- 172.21.15.55:0/2451407906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe364140bb0 0x7fe3640c46e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe35c012000 tx=0x7fe35c018000 comp rx=0 tx=0).stop 2023-10-06T03:33:15.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.872+0000 7fe36959d700 1 -- 172.21.15.55:0/2451407906 shutdown_connections 2023-10-06T03:33:15.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.872+0000 7fe36959d700 1 --2- 172.21.15.55:0/2451407906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe364140bb0 0x7fe3640c46e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:15.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.872+0000 7fe36959d700 1 --2- 172.21.15.55:0/2451407906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe36406dc00 0x7fe36415ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:15.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.872+0000 7fe36959d700 1 --2- 172.21.15.55:0/2451407906 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe350066f30 0x7fe3500693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:15.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.872+0000 7fe36959d700 1 --2- 172.21.15.55:0/2451407906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe3641407e0 0x7fe364157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:15.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.872+0000 7fe36959d700 1 -- 172.21.15.55:0/2451407906 >> 172.21.15.55:0/2451407906 conn(0x7fe3640cc1e0 msgr2=0x7fe36406a420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:15.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.872+0000 7fe36959d700 1 -- 172.21.15.55:0/2451407906 shutdown_connections 2023-10-06T03:33:15.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:15.873+0000 7fe36959d700 1 -- 172.21.15.55:0/2451407906 wait complete. 2023-10-06T03:33:15.876 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T03:33:15.897 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:16.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.083+0000 7fa7f6d9d700 1 Processor -- start 2023-10-06T03:33:16.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.083+0000 7fa7f6d9d700 1 -- start start 2023-10-06T03:33:16.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f6d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa7f0156310 0x7fa7f00c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f6d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7f00ae710 0x7fa7f00c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f6d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f00ca860 con 0x7fa7f01566e0 2023-10-06T03:33:16.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f6d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f00cab40 con 0x7fa7f0156310 2023-10-06T03:33:16.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f6d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f00cae20 con 0x7fa7f00ae710 2023-10-06T03:33:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f5d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa7f0156310 0x7fa7f00c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f5d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa7f0156310 0x7fa7f00c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40914/0 (socket says 172.21.15.55:40914) 2023-10-06T03:33:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f5d9b700 1 -- 172.21.15.55:0/877348637 learned_addr learned my addr 172.21.15.55:0/877348637 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f559a700 1 --1- 172.21.15.55:0/877348637 >> v1:172.21.15.55:6789/0 conn(0x7fa7f01566e0 0x7fa7f00ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56204/0 (socket says 172.21.15.55:56204) 2023-10-06T03:33:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f659c700 1 --2- 172.21.15.55:0/877348637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7f00ae710 0x7fa7f00c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f5d9b700 1 -- 172.21.15.55:0/877348637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7f00ae710 msgr2=0x7fa7f00c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f5d9b700 1 --2- 172.21.15.55:0/877348637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7f00ae710 0x7fa7f00c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f5d9b700 1 -- 172.21.15.55:0/877348637 >> v1:172.21.15.55:6789/0 conn(0x7fa7f01566e0 legacy=0x7fa7f00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f4d99700 1 -- 172.21.15.55:0/877348637 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948151386 0 0) 0x7fa7f00ca860 con 0x7fa7f01566e0 2023-10-06T03:33:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.085+0000 7fa7f5d9b700 1 -- 172.21.15.55:0/877348637 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f00cb100 con 0x7fa7f0156310 2023-10-06T03:33:16.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.086+0000 7fa7f5d9b700 1 --2- 172.21.15.55:0/877348637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7f0156310 0x7fa7f00c9dc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0099b0 tx=0x7fa7ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732ea48e29b1f970 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.086+0000 7fa7f4d99700 1 -- 172.21.15.55:0/877348637 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00b860 con 0x7fa7f0156310 2023-10-06T03:33:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.086+0000 7fa7f4d99700 1 -- 172.21.15.55:0/877348637 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec00b9c0 con 0x7fa7f0156310 2023-10-06T03:33:16.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.086+0000 7fa7f4d99700 1 -- 172.21.15.55:0/877348637 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec00bd60 con 0x7fa7f0156310 2023-10-06T03:33:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.086+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/877348637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7f0156310 msgr2=0x7fa7f00c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.086+0000 7fa7f6d9d700 1 --2- 172.21.15.55:0/877348637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7f0156310 0x7fa7f00c9dc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0099b0 tx=0x7fa7ec009250 comp rx=0 tx=0).stop 2023-10-06T03:33:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.086+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/877348637 shutdown_connections 2023-10-06T03:33:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.087+0000 7fa7f6d9d700 1 --2- 172.21.15.55:0/877348637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7f00ae710 0x7fa7f00c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.087+0000 7fa7f6d9d700 1 --2- 172.21.15.55:0/877348637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7f0156310 0x7fa7f00c9dc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.087+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/877348637 >> 172.21.15.55:0/877348637 conn(0x7fa7f00cc1f0 msgr2=0x7fa7f00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.087+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/877348637 shutdown_connections 2023-10-06T03:33:16.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.087+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/877348637 wait complete. 2023-10-06T03:33:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.088+0000 7fa7f6d9d700 1 Processor -- start 2023-10-06T03:33:16.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.088+0000 7fa7f6d9d700 1 -- start start 2023-10-06T03:33:16.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.089+0000 7fa7f6d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7f00ae710 0x7fa7f015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.089+0000 7fa7f6d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7f0156310 0x7fa7f0141be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.089+0000 7fa7f6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7f01566e0 0x7fa7f015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.089+0000 7fa7f6d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f015e780 con 0x7fa7f01566e0 2023-10-06T03:33:16.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.089+0000 7fa7f6d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f015ea60 con 0x7fa7f0156310 2023-10-06T03:33:16.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.089+0000 7fa7f6d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f015ed40 con 0x7fa7f00ae710 2023-10-06T03:33:16.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.089+0000 7fa7f659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7f01566e0 0x7fa7f015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.089+0000 7fa7f659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7f01566e0 0x7fa7f015f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40920/0 (socket says 172.21.15.55:40920) 2023-10-06T03:33:16.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.089+0000 7fa7f659c700 1 -- 172.21.15.55:0/2681178434 learned_addr learned my addr 172.21.15.55:0/2681178434 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:16.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.090+0000 7fa7f5d9b700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7f00ae710 0x7fa7f015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.090+0000 7fa7f559a700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7f0156310 0x7fa7f0141be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.090+0000 7fa7f659c700 1 -- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7f00ae710 msgr2=0x7fa7f015bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.090+0000 7fa7f659c700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7f00ae710 0x7fa7f015bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.090+0000 7fa7f659c700 1 -- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7f0156310 msgr2=0x7fa7f0141be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.090+0000 7fa7f659c700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7f0156310 0x7fa7f0141be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.090+0000 7fa7f659c700 1 -- 172.21.15.55:0/2681178434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec0094f0 con 0x7fa7f01566e0 2023-10-06T03:33:16.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.090+0000 7fa7f559a700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7f0156310 0x7fa7f0141be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:16.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.090+0000 7fa7f659c700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7f01566e0 0x7fa7f015f4b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8007f30 tx=0x7fa7e80098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.090+0000 7fa7e5ffb700 1 -- 172.21.15.55:0/2681178434 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e800ec40 con 0x7fa7f01566e0 2023-10-06T03:33:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.090+0000 7fa7e5ffb700 1 -- 172.21.15.55:0/2681178434 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e800eda0 con 0x7fa7f01566e0 2023-10-06T03:33:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.090+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/2681178434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f015f020 con 0x7fa7f01566e0 2023-10-06T03:33:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.090+0000 7fa7e5ffb700 1 -- 172.21.15.55:0/2681178434 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8014670 con 0x7fa7f01566e0 2023-10-06T03:33:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.091+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/2681178434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f0142120 con 0x7fa7f01566e0 2023-10-06T03:33:16.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.091+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/2681178434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f004c3c0 con 0x7fa7f01566e0 2023-10-06T03:33:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.092+0000 7fa7e5ffb700 1 -- 172.21.15.55:0/2681178434 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7e8014a20 con 0x7fa7f01566e0 2023-10-06T03:33:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.093+0000 7fa7e5ffb700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa7d4066f30 0x7fa7d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.093+0000 7fa7e5ffb700 1 -- 172.21.15.55:0/2681178434 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa7e8084e20 con 0x7fa7f01566e0 2023-10-06T03:33:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.093+0000 7fa7f5d9b700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa7d4066f30 0x7fa7d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.094+0000 7fa7f5d9b700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa7d4066f30 0x7fa7d40693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0099b0 tx=0x7fa7ec017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:16.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.097+0000 7fa7e5ffb700 1 -- 172.21.15.55:0/2681178434 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7e8051e90 con 0x7fa7f01566e0 2023-10-06T03:33:16.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.301+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/2681178434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7f0152540 con 0x7fa7f01566e0 2023-10-06T03:33:16.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.302+0000 7fa7e5ffb700 1 -- 172.21.15.55:0/2681178434 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16927 (secure 0 0 0) 0x7fa7e8071230 con 0x7fa7f01566e0 2023-10-06T03:33:16.342 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:16.342 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":20,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:15.382791+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.304+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa7d4066f30 msgr2=0x7fa7d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.305+0000 7fa7f6d9d700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa7d4066f30 0x7fa7d40693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0099b0 tx=0x7fa7ec017000 comp rx=0 tx=0).stop 2023-10-06T03:33:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.305+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/2681178434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7f01566e0 msgr2=0x7fa7f015f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.305+0000 7fa7f6d9d700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7f01566e0 0x7fa7f015f4b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8007f30 tx=0x7fa7e80098e0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.305+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/2681178434 shutdown_connections 2023-10-06T03:33:16.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.305+0000 7fa7f6d9d700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7f01566e0 0x7fa7f015f4b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.305+0000 7fa7f6d9d700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa7d4066f30 0x7fa7d40693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.305+0000 7fa7f6d9d700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7f00ae710 0x7fa7f015bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.305+0000 7fa7f6d9d700 1 --2- 172.21.15.55:0/2681178434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7f0156310 0x7fa7f0141be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.305+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/2681178434 >> 172.21.15.55:0/2681178434 conn(0x7fa7f00cc1f0 msgr2=0x7fa7f00b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:16.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.305+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/2681178434 shutdown_connections 2023-10-06T03:33:16.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.305+0000 7fa7f6d9d700 1 -- 172.21.15.55:0/2681178434 wait complete. 2023-10-06T03:33:16.348 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:16.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.535+0000 7f348959d700 1 Processor -- start 2023-10-06T03:33:16.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.536+0000 7f348959d700 1 -- start start 2023-10-06T03:33:16.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.537+0000 7f348959d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34840ae710 0x7f34840c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.537+0000 7f348959d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34840ca860 con 0x7f3484156310 2023-10-06T03:33:16.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.537+0000 7f348959d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34840cab40 con 0x7f34841566e0 2023-10-06T03:33:16.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.537+0000 7f348959d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34840cae20 con 0x7f34840ae710 2023-10-06T03:33:16.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.537+0000 7f3483fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3484156310 0x7f34840c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56212/0 (socket says 172.21.15.55:56212) 2023-10-06T03:33:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.537+0000 7f3483fff700 1 -- 172.21.15.55:0/1222010624 learned_addr learned my addr 172.21.15.55:0/1222010624 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.537+0000 7f3488d9c700 1 --2- 172.21.15.55:0/1222010624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34840ae710 0x7f34840c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481407928 0 0) 0x7f34840ca860 con 0x7f3484156310 2023-10-06T03:33:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f346c003570 con 0x7f3484156310 2023-10-06T03:33:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471122284 0 0) 0x7f34840cab40 con 0x7f34841566e0 2023-10-06T03:33:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34840ca860 con 0x7f34841566e0 2023-10-06T03:33:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1896106331 0 0) 0x7f346c003570 con 0x7f3484156310 2023-10-06T03:33:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34840ae710 msgr2=0x7f34840c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 --2- 172.21.15.55:0/1222010624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34840ae710 0x7f34840c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 >> v1:172.21.15.163:6789/0 conn(0x7f34841566e0 legacy=0x7f34840ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34840cb100 con 0x7f3484156310 2023-10-06T03:33:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f3474001470 con 0x7f3484156310 2023-10-06T03:33:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 >> v1:172.21.15.55:6789/0 conn(0x7f3484156310 legacy=0x7f34840c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 --2- 172.21.15.55:0/1222010624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f346c003c20 0x7f346c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 --2- 172.21.15.55:0/1222010624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f346c004810 0x7f346c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 --2- 172.21.15.55:0/1222010624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34840ae710 0x7f346c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34840cae20 con 0x7f346c003c20 2023-10-06T03:33:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c00de00 con 0x7f346c004810 2023-10-06T03:33:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c00dfc0 con 0x7f34840ae710 2023-10-06T03:33:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3483fff700 1 --2- 172.21.15.55:0/1222010624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f346c003c20 0x7f346c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f3488d9c700 1 --2- 172.21.15.55:0/1222010624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34840ae710 0x7f346c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.538+0000 7f34837fe700 1 --2- 172.21.15.55:0/1222010624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f346c004810 0x7f346c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.539+0000 7f3483fff700 1 -- 172.21.15.55:0/1222010624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34840ae710 msgr2=0x7f346c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.539+0000 7f3483fff700 1 --2- 172.21.15.55:0/1222010624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34840ae710 0x7f346c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.539+0000 7f3483fff700 1 -- 172.21.15.55:0/1222010624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f346c004810 msgr2=0x7f346c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.539+0000 7f3483fff700 1 --2- 172.21.15.55:0/1222010624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f346c004810 0x7f346c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.539+0000 7f3483fff700 1 -- 172.21.15.55:0/1222010624 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c003570 con 0x7f346c003c20 2023-10-06T03:33:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.539+0000 7f3483fff700 1 --2- 172.21.15.55:0/1222010624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f346c003c20 0x7f346c004010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f347400c210 tx=0x7f3474001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d6e70afe5bce65 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.539+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347400cc10 con 0x7f346c003c20 2023-10-06T03:33:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.539+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347400cd70 con 0x7f346c003c20 2023-10-06T03:33:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.539+0000 7f3482ffd700 1 -- 172.21.15.55:0/1222010624 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474005b70 con 0x7f346c003c20 2023-10-06T03:33:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.540+0000 7f348959d700 1 -- 172.21.15.55:0/1222010624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f346c003c20 msgr2=0x7f346c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.540+0000 7f348959d700 1 --2- 172.21.15.55:0/1222010624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f346c003c20 0x7f346c004010 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f347400c210 tx=0x7f3474001d40 comp rx=0 tx=0).stop 2023-10-06T03:33:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.540+0000 7f3488d9c700 1 -- 172.21.15.55:0/1222010624 reap_dead start 2023-10-06T03:33:16.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.540+0000 7f348959d700 1 -- 172.21.15.55:0/1222010624 shutdown_connections 2023-10-06T03:33:16.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.540+0000 7f348959d700 1 -- 172.21.15.55:0/1222010624 >> 172.21.15.55:0/1222010624 conn(0x7f34840cc1f0 msgr2=0x7f34840cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:16.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.540+0000 7f348959d700 1 -- 172.21.15.55:0/1222010624 shutdown_connections 2023-10-06T03:33:16.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.540+0000 7f348959d700 1 -- 172.21.15.55:0/1222010624 wait complete. 2023-10-06T03:33:16.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.541+0000 7f348959d700 1 Processor -- start 2023-10-06T03:33:16.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.542+0000 7f348959d700 1 -- start start 2023-10-06T03:33:16.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.542+0000 7f348959d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f346c003c20 0x7f34840c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.542+0000 7f348959d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348415a440 0x7f3484143c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.542+0000 7f348959d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3484157b50 0x7f34840c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.542+0000 7f348959d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34840c38f0 con 0x7f348415a440 2023-10-06T03:33:16.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.542+0000 7f348959d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348415ac20 con 0x7f3484157b50 2023-10-06T03:33:16.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.542+0000 7f348959d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348415af00 con 0x7f346c003c20 2023-10-06T03:33:16.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.543+0000 7f34837fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348415a440 0x7f3484143c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.543+0000 7f34837fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348415a440 0x7f3484143c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40932/0 (socket says 172.21.15.55:40932) 2023-10-06T03:33:16.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.543+0000 7f34837fe700 1 -- 172.21.15.55:0/2016903270 learned_addr learned my addr 172.21.15.55:0/2016903270 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:16.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.543+0000 7f3483fff700 1 --2- 172.21.15.55:0/2016903270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f346c003c20 0x7f34840c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.543+0000 7f3488d9c700 1 --2- 172.21.15.55:0/2016903270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3484157b50 0x7f34840c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.543+0000 7f34837fe700 1 -- 172.21.15.55:0/2016903270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f346c003c20 msgr2=0x7f34840c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.543+0000 7f34837fe700 1 --2- 172.21.15.55:0/2016903270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f346c003c20 0x7f34840c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.543+0000 7f34837fe700 1 -- 172.21.15.55:0/2016903270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3484157b50 msgr2=0x7f34840c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.543+0000 7f34837fe700 1 --2- 172.21.15.55:0/2016903270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3484157b50 0x7f34840c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.543+0000 7f34837fe700 1 -- 172.21.15.55:0/2016903270 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34840c32a0 con 0x7f348415a440 2023-10-06T03:33:16.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.543+0000 7f34837fe700 1 --2- 172.21.15.55:0/2016903270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348415a440 0x7f3484143c00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3478002bd0 tx=0x7f3478007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:16.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.543+0000 7f346bfff700 1 -- 172.21.15.55:0/2016903270 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347800f920 con 0x7f348415a440 2023-10-06T03:33:16.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.544+0000 7f346bfff700 1 -- 172.21.15.55:0/2016903270 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347800faa0 con 0x7f348415a440 2023-10-06T03:33:16.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.544+0000 7f348959d700 1 -- 172.21.15.55:0/2016903270 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34840c3580 con 0x7f348415a440 2023-10-06T03:33:16.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.544+0000 7f346bfff700 1 -- 172.21.15.55:0/2016903270 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347800fe60 con 0x7f348415a440 2023-10-06T03:33:16.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.544+0000 7f348959d700 1 -- 172.21.15.55:0/2016903270 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348415a8b0 con 0x7f348415a440 2023-10-06T03:33:16.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.545+0000 7f348959d700 1 -- 172.21.15.55:0/2016903270 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34840c3aa0 con 0x7f348415a440 2023-10-06T03:33:16.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.546+0000 7f346bfff700 1 -- 172.21.15.55:0/2016903270 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3478013600 con 0x7f348415a440 2023-10-06T03:33:16.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.546+0000 7f346bfff700 1 --2- 172.21.15.55:0/2016903270 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3470066f30 0x7f34700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.546+0000 7f346bfff700 1 -- 172.21.15.55:0/2016903270 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3478084d60 con 0x7f348415a440 2023-10-06T03:33:16.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.547+0000 7f3483fff700 1 --2- 172.21.15.55:0/2016903270 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3470066f30 0x7f34700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.547+0000 7f3483fff700 1 --2- 172.21.15.55:0/2016903270 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3470066f30 0x7f34700693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3484159670 tx=0x7f3474015000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:16.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.551+0000 7f346bfff700 1 -- 172.21.15.55:0/2016903270 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34840c3aa0 con 0x7f348415a440 2023-10-06T03:33:16.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.756+0000 7f348959d700 1 -- 172.21.15.55:0/2016903270 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f348404c3c0 con 0x7f348415a440 2023-10-06T03:33:16.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.757+0000 7f346bfff700 1 -- 172.21.15.55:0/2016903270 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f3478017090 con 0x7f348415a440 2023-10-06T03:33:16.759 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:16.759 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:16.470817+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:16.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.759+0000 7f348959d700 1 -- 172.21.15.55:0/2016903270 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3470066f30 msgr2=0x7f34700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.759+0000 7f348959d700 1 --2- 172.21.15.55:0/2016903270 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3470066f30 0x7f34700693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3484159670 tx=0x7f3474015000 comp rx=0 tx=0).stop 2023-10-06T03:33:16.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.759+0000 7f348959d700 1 -- 172.21.15.55:0/2016903270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348415a440 msgr2=0x7f3484143c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.759+0000 7f348959d700 1 --2- 172.21.15.55:0/2016903270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348415a440 0x7f3484143c00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3478002bd0 tx=0x7f3478007d30 comp rx=0 tx=0).stop 2023-10-06T03:33:16.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.759+0000 7f348959d700 1 -- 172.21.15.55:0/2016903270 shutdown_connections 2023-10-06T03:33:16.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.759+0000 7f348959d700 1 --2- 172.21.15.55:0/2016903270 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3484157b50 0x7f34840c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.759+0000 7f348959d700 1 --2- 172.21.15.55:0/2016903270 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3470066f30 0x7f34700693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.759+0000 7f348959d700 1 --2- 172.21.15.55:0/2016903270 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f346c003c20 0x7f34840c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.759+0000 7f348959d700 1 --2- 172.21.15.55:0/2016903270 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348415a440 0x7f3484143c00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.759+0000 7f348959d700 1 -- 172.21.15.55:0/2016903270 >> 172.21.15.55:0/2016903270 conn(0x7f34840cc1f0 msgr2=0x7f3484155120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:16.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.759+0000 7f348959d700 1 -- 172.21.15.55:0/2016903270 shutdown_connections 2023-10-06T03:33:16.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.759+0000 7f348959d700 1 -- 172.21.15.55:0/2016903270 wait complete. 2023-10-06T03:33:16.778 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:33:16.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.969+0000 7f8b9559d700 1 Processor -- start 2023-10-06T03:33:16.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.970+0000 7f8b9559d700 1 -- start start 2023-10-06T03:33:16.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.971+0000 7f8b9559d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b900cb1a0 0x7f8b9015a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.971+0000 7f8b9559d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b9015a8a0 0x7f8b9006f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.971+0000 7f8b9559d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b9006ebd0 con 0x7f8b9006f5a0 2023-10-06T03:33:16.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.971+0000 7f8b9559d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b9015b2e0 con 0x7f8b9015a8a0 2023-10-06T03:33:16.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.971+0000 7f8b9559d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9015ad20 con 0x7f8b900cb1a0 2023-10-06T03:33:16.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.971+0000 7f8b8f7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b9015a8a0 0x7f8b9006f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.971+0000 7f8b8f7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b9015a8a0 0x7f8b9006f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40940/0 (socket says 172.21.15.55:40940) 2023-10-06T03:33:16.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.971+0000 7f8b8f7fe700 1 -- 172.21.15.55:0/3894948742 learned_addr learned my addr 172.21.15.55:0/3894948742 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:16.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.971+0000 7f8b8ffff700 1 --2- 172.21.15.55:0/3894948742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b900cb1a0 0x7f8b9015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.971+0000 7f8b8f7fe700 1 -- 172.21.15.55:0/3894948742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b900cb1a0 msgr2=0x7f8b9015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.971+0000 7f8b8f7fe700 1 --2- 172.21.15.55:0/3894948742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b900cb1a0 0x7f8b9015a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.972+0000 7f8b8f7fe700 1 -- 172.21.15.55:0/3894948742 >> v1:172.21.15.163:6789/0 conn(0x7f8b9006f5a0 legacy=0x7f8b900ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.972+0000 7f8b8f7fe700 1 -- 172.21.15.55:0/3894948742 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9015b000 con 0x7f8b9015a8a0 2023-10-06T03:33:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.972+0000 7f8b8f7fe700 1 --2- 172.21.15.55:0/3894948742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b9015a8a0 0x7f8b9006f060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8b840099b0 tx=0x7f8b84009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a000d7cfaeb041cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.972+0000 7f8b8effd700 1 -- 172.21.15.55:0/3894948742 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b84009ea0 con 0x7f8b9015a8a0 2023-10-06T03:33:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.972+0000 7f8b8effd700 1 -- 172.21.15.55:0/3894948742 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8400b8f0 con 0x7f8b9015a8a0 2023-10-06T03:33:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.972+0000 7f8b8effd700 1 -- 172.21.15.55:0/3894948742 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400bc90 con 0x7f8b9015a8a0 2023-10-06T03:33:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.972+0000 7f8b9559d700 1 -- 172.21.15.55:0/3894948742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b9015a8a0 msgr2=0x7f8b9006f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.972+0000 7f8b9559d700 1 --2- 172.21.15.55:0/3894948742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b9015a8a0 0x7f8b9006f060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8b840099b0 tx=0x7f8b84009250 comp rx=0 tx=0).stop 2023-10-06T03:33:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.973+0000 7f8b9559d700 1 -- 172.21.15.55:0/3894948742 shutdown_connections 2023-10-06T03:33:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.973+0000 7f8b9559d700 1 --2- 172.21.15.55:0/3894948742 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b900cb1a0 0x7f8b9015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.973+0000 7f8b9559d700 1 --2- 172.21.15.55:0/3894948742 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b9015a8a0 0x7f8b9006f060 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.973+0000 7f8b9559d700 1 -- 172.21.15.55:0/3894948742 >> 172.21.15.55:0/3894948742 conn(0x7f8b900cc1e0 msgr2=0x7f8b900cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:16.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.973+0000 7f8b9559d700 1 -- 172.21.15.55:0/3894948742 shutdown_connections 2023-10-06T03:33:16.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.973+0000 7f8b9559d700 1 -- 172.21.15.55:0/3894948742 wait complete. 2023-10-06T03:33:16.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.974+0000 7f8b9559d700 1 Processor -- start 2023-10-06T03:33:16.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.974+0000 7f8b9559d700 1 -- start start 2023-10-06T03:33:16.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.975+0000 7f8b9559d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b9006f5a0 0x7f8b901508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.975+0000 7f8b9559d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b900cb1a0 0x7f8b90159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.975+0000 7f8b9559d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b901598d0 0x7f8b900b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.975+0000 7f8b9559d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b90150e20 con 0x7f8b9006f5a0 2023-10-06T03:33:16.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.975+0000 7f8b9559d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b900c35e0 con 0x7f8b900cb1a0 2023-10-06T03:33:16.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.975+0000 7f8b9559d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b900c38c0 con 0x7f8b901598d0 2023-10-06T03:33:16.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.975+0000 7f8b8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b9006f5a0 0x7f8b901508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.975+0000 7f8b8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b9006f5a0 0x7f8b901508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40942/0 (socket says 172.21.15.55:40942) 2023-10-06T03:33:16.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.975+0000 7f8b8ffff700 1 -- 172.21.15.55:0/4026974102 learned_addr learned my addr 172.21.15.55:0/4026974102 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:16.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.976+0000 7f8b94d9c700 1 --2- 172.21.15.55:0/4026974102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b901598d0 0x7f8b900b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.976+0000 7f8b8f7fe700 1 --2- 172.21.15.55:0/4026974102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b900cb1a0 0x7f8b90159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.976+0000 7f8b8ffff700 1 -- 172.21.15.55:0/4026974102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b901598d0 msgr2=0x7f8b900b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.976+0000 7f8b8ffff700 1 --2- 172.21.15.55:0/4026974102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b901598d0 0x7f8b900b0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.976+0000 7f8b8ffff700 1 -- 172.21.15.55:0/4026974102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b900cb1a0 msgr2=0x7f8b90159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:16.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.976+0000 7f8b8ffff700 1 --2- 172.21.15.55:0/4026974102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b900cb1a0 0x7f8b90159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:16.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.976+0000 7f8b8ffff700 1 -- 172.21.15.55:0/4026974102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b840094f0 con 0x7f8b9006f5a0 2023-10-06T03:33:16.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.976+0000 7f8b8ffff700 1 --2- 172.21.15.55:0/4026974102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b9006f5a0 0x7f8b901508e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000ec80 tx=0x7f8b800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:16.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.976+0000 7f8b77fff700 1 -- 172.21.15.55:0/4026974102 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000c9d0 con 0x7f8b9006f5a0 2023-10-06T03:33:16.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.976+0000 7f8b77fff700 1 -- 172.21.15.55:0/4026974102 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8000cb30 con 0x7f8b9006f5a0 2023-10-06T03:33:16.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.976+0000 7f8b9559d700 1 -- 172.21.15.55:0/4026974102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b900c3ba0 con 0x7f8b9006f5a0 2023-10-06T03:33:16.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.976+0000 7f8b77fff700 1 -- 172.21.15.55:0/4026974102 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80013470 con 0x7f8b9006f5a0 2023-10-06T03:33:16.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.976+0000 7f8b9559d700 1 -- 172.21.15.55:0/4026974102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b900c55e0 con 0x7f8b9006f5a0 2023-10-06T03:33:16.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.977+0000 7f8b9559d700 1 -- 172.21.15.55:0/4026974102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b900c5440 con 0x7f8b9006f5a0 2023-10-06T03:33:16.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.978+0000 7f8b77fff700 1 -- 172.21.15.55:0/4026974102 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b80013820 con 0x7f8b9006f5a0 2023-10-06T03:33:16.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.979+0000 7f8b77fff700 1 --2- 172.21.15.55:0/4026974102 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b7c066f30 0x7f8b7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:16.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.979+0000 7f8b77fff700 1 -- 172.21.15.55:0/4026974102 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8b80084ac0 con 0x7f8b9006f5a0 2023-10-06T03:33:16.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.979+0000 7f8b8f7fe700 1 --2- 172.21.15.55:0/4026974102 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b7c066f30 0x7f8b7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:16.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.980+0000 7f8b8f7fe700 1 --2- 172.21.15.55:0/4026974102 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b7c066f30 0x7f8b7c0693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8b840099b0 tx=0x7f8b84017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:16.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:16.983+0000 7f8b77fff700 1 -- 172.21.15.55:0/4026974102 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b900c5440 con 0x7f8b9006f5a0 2023-10-06T03:33:17.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:17.234+0000 7f8b9559d700 1 -- 172.21.15.55:0/4026974102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8b900c5440 con 0x7f8b9006f5a0 2023-10-06T03:33:17.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:17.909+0000 7f8b77fff700 1 -- 172.21.15.55:0/4026974102 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8b900c5440 con 0x7f8b9006f5a0 2023-10-06T03:33:17.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:17.912+0000 7f8b9559d700 1 -- 172.21.15.55:0/4026974102 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b7c066f30 msgr2=0x7f8b7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:17.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:17.912+0000 7f8b9559d700 1 --2- 172.21.15.55:0/4026974102 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b7c066f30 0x7f8b7c0693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8b840099b0 tx=0x7f8b84017000 comp rx=0 tx=0).stop 2023-10-06T03:33:17.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:17.912+0000 7f8b9559d700 1 -- 172.21.15.55:0/4026974102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b9006f5a0 msgr2=0x7f8b901508e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:17.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:17.912+0000 7f8b9559d700 1 --2- 172.21.15.55:0/4026974102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b9006f5a0 0x7f8b901508e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000ec80 tx=0x7f8b800091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:17.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:17.912+0000 7f8b9559d700 1 -- 172.21.15.55:0/4026974102 shutdown_connections 2023-10-06T03:33:17.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:17.912+0000 7f8b9559d700 1 --2- 172.21.15.55:0/4026974102 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b7c066f30 0x7f8b7c0693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:17.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:17.912+0000 7f8b9559d700 1 --2- 172.21.15.55:0/4026974102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b901598d0 0x7f8b900b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:17.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:17.912+0000 7f8b9559d700 1 --2- 172.21.15.55:0/4026974102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b9006f5a0 0x7f8b901508e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:17.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:17.912+0000 7f8b9559d700 1 --2- 172.21.15.55:0/4026974102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b900cb1a0 0x7f8b90159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:17.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:17.912+0000 7f8b9559d700 1 -- 172.21.15.55:0/4026974102 >> 172.21.15.55:0/4026974102 conn(0x7f8b900cc1e0 msgr2=0x7f8b900af0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:17.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:17.912+0000 7f8b9559d700 1 -- 172.21.15.55:0/4026974102 shutdown_connections 2023-10-06T03:33:17.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:17.912+0000 7f8b9559d700 1 -- 172.21.15.55:0/4026974102 wait complete. 2023-10-06T03:33:17.934 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:33:18.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.127+0000 7fe1c959d700 1 Processor -- start 2023-10-06T03:33:18.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.127+0000 7fe1c959d700 1 -- start start 2023-10-06T03:33:18.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.129+0000 7fe1c959d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c415a3b0 0x7fe1c415a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:18.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.129+0000 7fe1c959d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1c406e470 0x7fe1c406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:18.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.129+0000 7fe1c959d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c40aea70 con 0x7fe1c40aef00 2023-10-06T03:33:18.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.129+0000 7fe1c959d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c406ee30 con 0x7fe1c406e470 2023-10-06T03:33:18.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.129+0000 7fe1c959d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c415dad0 con 0x7fe1c415a3b0 2023-10-06T03:33:18.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.129+0000 7fe1c8d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1c406e470 0x7fe1c406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:18.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.129+0000 7fe1c8d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1c406e470 0x7fe1c406e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40958/0 (socket says 172.21.15.55:40958) 2023-10-06T03:33:18.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.129+0000 7fe1c8d9c700 1 -- 172.21.15.55:0/2288556713 learned_addr learned my addr 172.21.15.55:0/2288556713 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:18.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.129+0000 7fe1c3fff700 1 --2- 172.21.15.55:0/2288556713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c415a3b0 0x7fe1c415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:18.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.130+0000 7fe1c8d9c700 1 -- 172.21.15.55:0/2288556713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c415a3b0 msgr2=0x7fe1c415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:18.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.130+0000 7fe1c8d9c700 1 --2- 172.21.15.55:0/2288556713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c415a3b0 0x7fe1c415a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:18.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.130+0000 7fe1c8d9c700 1 -- 172.21.15.55:0/2288556713 >> v1:172.21.15.163:6789/0 conn(0x7fe1c40aef00 legacy=0x7fe1c415d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:18.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.130+0000 7fe1c8d9c700 1 -- 172.21.15.55:0/2288556713 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c40ae480 con 0x7fe1c406e470 2023-10-06T03:33:18.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.130+0000 7fe1c8d9c700 1 --2- 172.21.15.55:0/2288556713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c406e470 0x7fe1c406e8f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009f20 tx=0x7fe1bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c082cb0a970e17 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:18.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.130+0000 7fe1c2ffd700 1 -- 172.21.15.55:0/2288556713 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00e070 con 0x7fe1c406e470 2023-10-06T03:33:18.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.130+0000 7fe1c2ffd700 1 -- 172.21.15.55:0/2288556713 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc00b9d0 con 0x7fe1c406e470 2023-10-06T03:33:18.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.130+0000 7fe1c2ffd700 1 -- 172.21.15.55:0/2288556713 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc0092e0 con 0x7fe1c406e470 2023-10-06T03:33:18.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.131+0000 7fe1c959d700 1 -- 172.21.15.55:0/2288556713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c406e470 msgr2=0x7fe1c406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.131+0000 7fe1c959d700 1 --2- 172.21.15.55:0/2288556713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c406e470 0x7fe1c406e8f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009f20 tx=0x7fe1bc009840 comp rx=0 tx=0).stop 2023-10-06T03:33:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.131+0000 7fe1c959d700 1 -- 172.21.15.55:0/2288556713 shutdown_connections 2023-10-06T03:33:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.131+0000 7fe1c959d700 1 --2- 172.21.15.55:0/2288556713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c406e470 0x7fe1c406e8f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.131+0000 7fe1c959d700 1 --2- 172.21.15.55:0/2288556713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c415a3b0 0x7fe1c415a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.131+0000 7fe1c959d700 1 -- 172.21.15.55:0/2288556713 >> 172.21.15.55:0/2288556713 conn(0x7fe1c40c9f70 msgr2=0x7fe1c406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.131+0000 7fe1c959d700 1 -- 172.21.15.55:0/2288556713 shutdown_connections 2023-10-06T03:33:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.131+0000 7fe1c959d700 1 -- 172.21.15.55:0/2288556713 wait complete. 2023-10-06T03:33:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.132+0000 7fe1c959d700 1 Processor -- start 2023-10-06T03:33:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.132+0000 7fe1c959d700 1 -- start start 2023-10-06T03:33:18.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.133+0000 7fe1c959d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c406e470 0x7fe1c40c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:18.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.133+0000 7fe1c959d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c40aef00 0x7fe1c4157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:18.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.133+0000 7fe1c959d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c415a3b0 0x7fe1c4151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:18.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.133+0000 7fe1c959d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c4155090 con 0x7fe1c40aef00 2023-10-06T03:33:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.133+0000 7fe1c959d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c40b13b0 con 0x7fe1c406e470 2023-10-06T03:33:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.133+0000 7fe1c959d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c40b1970 con 0x7fe1c415a3b0 2023-10-06T03:33:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.133+0000 7fe1c37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c40aef00 0x7fe1c4157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.133+0000 7fe1c37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c40aef00 0x7fe1c4157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40968/0 (socket says 172.21.15.55:40968) 2023-10-06T03:33:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.133+0000 7fe1c37fe700 1 -- 172.21.15.55:0/2097577307 learned_addr learned my addr 172.21.15.55:0/2097577307 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.134+0000 7fe1c3fff700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c406e470 0x7fe1c40c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.134+0000 7fe1c8d9c700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c415a3b0 0x7fe1c4151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.134+0000 7fe1c37fe700 1 -- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c415a3b0 msgr2=0x7fe1c4151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.134+0000 7fe1c37fe700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c415a3b0 0x7fe1c4151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.134+0000 7fe1c37fe700 1 -- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c406e470 msgr2=0x7fe1c40c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.134+0000 7fe1c37fe700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c406e470 0x7fe1c40c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.134+0000 7fe1c37fe700 1 -- 172.21.15.55:0/2097577307 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc009ae0 con 0x7fe1c40aef00 2023-10-06T03:33:18.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.134+0000 7fe1c8d9c700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c415a3b0 0x7fe1c4151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:18.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.134+0000 7fe1c37fe700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c40aef00 0x7fe1c4157200 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800c590 tx=0x7fe1b800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:18.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.134+0000 7fe1abfff700 1 -- 172.21.15.55:0/2097577307 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8011590 con 0x7fe1c40aef00 2023-10-06T03:33:18.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.134+0000 7fe1abfff700 1 -- 172.21.15.55:0/2097577307 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b8011710 con 0x7fe1c40aef00 2023-10-06T03:33:18.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.134+0000 7fe1abfff700 1 -- 172.21.15.55:0/2097577307 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8011ad0 con 0x7fe1c40aef00 2023-10-06T03:33:18.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.134+0000 7fe1c959d700 1 -- 172.21.15.55:0/2097577307 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c40b0480 con 0x7fe1c40aef00 2023-10-06T03:33:18.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.134+0000 7fe1c959d700 1 -- 172.21.15.55:0/2097577307 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c40b0810 con 0x7fe1c40aef00 2023-10-06T03:33:18.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.136+0000 7fe1c959d700 1 -- 172.21.15.55:0/2097577307 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c40ad200 con 0x7fe1c40aef00 2023-10-06T03:33:18.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.137+0000 7fe1abfff700 1 -- 172.21.15.55:0/2097577307 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1b801b3b0 con 0x7fe1c40aef00 2023-10-06T03:33:18.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.138+0000 7fe1abfff700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1b0066f30 0x7fe1b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:18.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.138+0000 7fe1abfff700 1 -- 172.21.15.55:0/2097577307 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe1b8083c60 con 0x7fe1c40aef00 2023-10-06T03:33:18.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.138+0000 7fe1c3fff700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1b0066f30 0x7fe1b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:18.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.139+0000 7fe1c3fff700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1b0066f30 0x7fe1b00693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009d10 tx=0x7fe1b40091a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:18.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.142+0000 7fe1abfff700 1 -- 172.21.15.55:0/2097577307 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1b8050cd0 con 0x7fe1c40aef00 2023-10-06T03:33:18.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.391+0000 7fe1c959d700 1 -- 172.21.15.55:0/2097577307 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe1c40b1ae0 con 0x7fe1c40aef00 2023-10-06T03:33:18.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.918+0000 7fe1abfff700 1 -- 172.21.15.55:0/2097577307 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe1b8017070 con 0x7fe1c40aef00 2023-10-06T03:33:18.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.921+0000 7fe1c959d700 1 -- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1b0066f30 msgr2=0x7fe1b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:18.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.921+0000 7fe1c959d700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1b0066f30 0x7fe1b00693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe1b4009d10 tx=0x7fe1b40091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:18.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.921+0000 7fe1c959d700 1 -- 172.21.15.55:0/2097577307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c40aef00 msgr2=0x7fe1c4157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:18.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.921+0000 7fe1c959d700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c40aef00 0x7fe1c4157200 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800c590 tx=0x7fe1b800c8a0 comp rx=0 tx=0).stop 2023-10-06T03:33:18.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.921+0000 7fe1c959d700 1 -- 172.21.15.55:0/2097577307 shutdown_connections 2023-10-06T03:33:18.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.921+0000 7fe1c959d700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1b0066f30 0x7fe1b00693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:18.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.921+0000 7fe1c959d700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c415a3b0 0x7fe1c4151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:18.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.921+0000 7fe1c959d700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c406e470 0x7fe1c40c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:18.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.921+0000 7fe1c959d700 1 --2- 172.21.15.55:0/2097577307 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c40aef00 0x7fe1c4157200 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:18.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.921+0000 7fe1c959d700 1 -- 172.21.15.55:0/2097577307 >> 172.21.15.55:0/2097577307 conn(0x7fe1c40c9f70 msgr2=0x7fe1c415be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:18.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.922+0000 7fe1c959d700 1 -- 172.21.15.55:0/2097577307 shutdown_connections 2023-10-06T03:33:18.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:18.922+0000 7fe1c959d700 1 -- 172.21.15.55:0/2097577307 wait complete. 2023-10-06T03:33:18.943 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:33:19.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.133+0000 7fa0f1f9d700 1 Processor -- start 2023-10-06T03:33:19.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.133+0000 7fa0f1f9d700 1 -- start start 2023-10-06T03:33:19.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.134+0000 7fa0f1f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0ec159420 0x7fa0ec0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:19.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.134+0000 7fa0f1f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0ec1597f0 0x7fa0ec0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:19.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.134+0000 7fa0f1f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec0b11d0 con 0x7fa0ec15d230 2023-10-06T03:33:19.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.134+0000 7fa0f1f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ec0b14b0 con 0x7fa0ec1597f0 2023-10-06T03:33:19.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.134+0000 7fa0f1f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0b1790 con 0x7fa0ec159420 2023-10-06T03:33:19.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.135+0000 7fa0ebfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0ec1597f0 0x7fa0ec0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:19.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.135+0000 7fa0f0f9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0ec159420 0x7fa0ec0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:19.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.135+0000 7fa0ebfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0ec1597f0 0x7fa0ec0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:48030/0 (socket says 172.21.15.55:48030) 2023-10-06T03:33:19.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.135+0000 7fa0ebfff700 1 -- 172.21.15.55:0/1704489757 learned_addr learned my addr 172.21.15.55:0/1704489757 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:19.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.135+0000 7fa0f0f9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0ec159420 0x7fa0ec0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:56966/0 (socket says 172.21.15.55:56966) 2023-10-06T03:33:19.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.135+0000 7fa0ebfff700 1 -- 172.21.15.55:0/1704489757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0ec159420 msgr2=0x7fa0ec0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:19.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.135+0000 7fa0ebfff700 1 --2- 172.21.15.55:0/1704489757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0ec159420 0x7fa0ec0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.135+0000 7fa0ebfff700 1 -- 172.21.15.55:0/1704489757 >> v1:172.21.15.163:6789/0 conn(0x7fa0ec15d230 legacy=0x7fa0ec0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:19.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.135+0000 7fa0ebfff700 1 -- 172.21.15.55:0/1704489757 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec0b1a70 con 0x7fa0ec1597f0 2023-10-06T03:33:19.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.135+0000 7fa0f0f9b700 1 --2- 172.21.15.55:0/1704489757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0ec159420 0x7fa0ec0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:19.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.136+0000 7fa0ebfff700 1 --2- 172.21.15.55:0/1704489757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ec1597f0 0x7fa0ec0b2270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80099b0 tx=0x7fa0d8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e41cf2c6409d3b5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:19.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.136+0000 7fa0eb7fe700 1 -- 172.21.15.55:0/1704489757 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8009ea0 con 0x7fa0ec1597f0 2023-10-06T03:33:19.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.136+0000 7fa0eb7fe700 1 -- 172.21.15.55:0/1704489757 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d800b860 con 0x7fa0ec1597f0 2023-10-06T03:33:19.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.136+0000 7fa0eb7fe700 1 -- 172.21.15.55:0/1704489757 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d800bc00 con 0x7fa0ec1597f0 2023-10-06T03:33:19.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.136+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/1704489757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ec1597f0 msgr2=0x7fa0ec0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:19.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.136+0000 7fa0f1f9d700 1 --2- 172.21.15.55:0/1704489757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ec1597f0 0x7fa0ec0b2270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80099b0 tx=0x7fa0d8009250 comp rx=0 tx=0).stop 2023-10-06T03:33:19.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.136+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/1704489757 shutdown_connections 2023-10-06T03:33:19.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.136+0000 7fa0f1f9d700 1 --2- 172.21.15.55:0/1704489757 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0ec159420 0x7fa0ec0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.136+0000 7fa0f1f9d700 1 --2- 172.21.15.55:0/1704489757 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ec1597f0 0x7fa0ec0b2270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.136+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/1704489757 >> 172.21.15.55:0/1704489757 conn(0x7fa0ec0d0660 msgr2=0x7fa0ec0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:19.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.137+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/1704489757 shutdown_connections 2023-10-06T03:33:19.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.137+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/1704489757 wait complete. 2023-10-06T03:33:19.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.138+0000 7fa0f1f9d700 1 Processor -- start 2023-10-06T03:33:19.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.138+0000 7fa0f1f9d700 1 -- start start 2023-10-06T03:33:19.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.139+0000 7fa0f1f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0ec159420 0x7fa0ec156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:19.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.139+0000 7fa0f1f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0ec1597f0 0x7fa0ec069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:19.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.139+0000 7fa0f1f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ec15d230 0x7fa0ec069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:19.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.139+0000 7fa0f1f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0c9660 con 0x7fa0ec1597f0 2023-10-06T03:33:19.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.139+0000 7fa0f1f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec1570a0 con 0x7fa0ec15d230 2023-10-06T03:33:19.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.139+0000 7fa0f1f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0c9330 con 0x7fa0ec159420 2023-10-06T03:33:19.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.139+0000 7fa0ebfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0ec1597f0 0x7fa0ec069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:19.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.139+0000 7fa0ebfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0ec1597f0 0x7fa0ec069510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40984/0 (socket says 172.21.15.55:40984) 2023-10-06T03:33:19.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.139+0000 7fa0ebfff700 1 -- 172.21.15.55:0/3087317805 learned_addr learned my addr 172.21.15.55:0/3087317805 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:19.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.139+0000 7fa0f0f9b700 1 --2- 172.21.15.55:0/3087317805 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0ec159420 0x7fa0ec156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:19.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.139+0000 7fa0f179c700 1 --2- 172.21.15.55:0/3087317805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ec15d230 0x7fa0ec069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:19.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.140+0000 7fa0ebfff700 1 -- 172.21.15.55:0/3087317805 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0ec159420 msgr2=0x7fa0ec156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:19.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.140+0000 7fa0ebfff700 1 --2- 172.21.15.55:0/3087317805 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0ec159420 0x7fa0ec156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.140+0000 7fa0ebfff700 1 -- 172.21.15.55:0/3087317805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ec15d230 msgr2=0x7fa0ec069a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:19.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.140+0000 7fa0ebfff700 1 --2- 172.21.15.55:0/3087317805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ec15d230 0x7fa0ec069a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.140+0000 7fa0ebfff700 1 -- 172.21.15.55:0/3087317805 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e0009420 con 0x7fa0ec1597f0 2023-10-06T03:33:19.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.140+0000 7fa0ebfff700 1 --2- 172.21.15.55:0/3087317805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0ec1597f0 0x7fa0ec069510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8006fd0 tx=0x7fa0d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:19.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.140+0000 7fa0e8ff9700 1 -- 172.21.15.55:0/3087317805 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d800bbe0 con 0x7fa0ec1597f0 2023-10-06T03:33:19.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.140+0000 7fa0e8ff9700 1 -- 172.21.15.55:0/3087317805 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d8000c90 con 0x7fa0ec1597f0 2023-10-06T03:33:19.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.140+0000 7fa0e8ff9700 1 -- 172.21.15.55:0/3087317805 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8011650 con 0x7fa0ec1597f0 2023-10-06T03:33:19.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.140+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/3087317805 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d80094f0 con 0x7fa0ec1597f0 2023-10-06T03:33:19.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.141+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/3087317805 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ec0c7930 con 0x7fa0ec1597f0 2023-10-06T03:33:19.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.142+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/3087317805 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ec0c9190 con 0x7fa0ec1597f0 2023-10-06T03:33:19.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.142+0000 7fa0e8ff9700 1 -- 172.21.15.55:0/3087317805 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa0d8011a40 con 0x7fa0ec1597f0 2023-10-06T03:33:19.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.143+0000 7fa0e8ff9700 1 --2- 172.21.15.55:0/3087317805 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa0d0066f30 0x7fa0d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:19.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.143+0000 7fa0e8ff9700 1 -- 172.21.15.55:0/3087317805 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa0d8085ec0 con 0x7fa0ec1597f0 2023-10-06T03:33:19.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.144+0000 7fa0f0f9b700 1 --2- 172.21.15.55:0/3087317805 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa0d0066f30 0x7fa0d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:19.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.144+0000 7fa0f0f9b700 1 --2- 172.21.15.55:0/3087317805 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa0d0066f30 0x7fa0d00693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009f40 tx=0x7fa0e00091a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:19.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.147+0000 7fa0e8ff9700 1 -- 172.21.15.55:0/3087317805 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0ec0c9190 con 0x7fa0ec1597f0 2023-10-06T03:33:19.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.395+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/3087317805 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0ec0c9190 con 0x7fa0ec1597f0 2023-10-06T03:33:19.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.396+0000 7fa0e8ff9700 1 -- 172.21.15.55:0/3087317805 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7fa0ec0c9190 con 0x7fa0ec1597f0 2023-10-06T03:33:19.398 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:19.399 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2677235076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":2677235076},{"type":"v1","addr":"172.21.15.55:6835","nonce":2677235076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1030747394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1030747394},{"type":"v1","addr":"172.21.15.55:6834","nonce":1030747394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2040975046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2040975046},{"type":"v1","addr":"172.21.15.163:6836","nonce":2040975046}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:33:15.383124+0000","modified":"2023-10-06T03:33:18.908835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/1910887369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1910887369},{"type":"v1","addr":"172.21.15.163:6837","nonce":1910887369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:33:19.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.398+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/3087317805 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa0d0066f30 msgr2=0x7fa0d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:19.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.398+0000 7fa0f1f9d700 1 --2- 172.21.15.55:0/3087317805 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa0d0066f30 0x7fa0d00693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009f40 tx=0x7fa0e00091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.398+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/3087317805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0ec1597f0 msgr2=0x7fa0ec069510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:19.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.398+0000 7fa0f1f9d700 1 --2- 172.21.15.55:0/3087317805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0ec1597f0 0x7fa0ec069510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8006fd0 tx=0x7fa0d8017000 comp rx=0 tx=0).stop 2023-10-06T03:33:19.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.398+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/3087317805 shutdown_connections 2023-10-06T03:33:19.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.398+0000 7fa0f1f9d700 1 --2- 172.21.15.55:0/3087317805 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0ec15d230 0x7fa0ec069a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.398+0000 7fa0f1f9d700 1 --2- 172.21.15.55:0/3087317805 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa0d0066f30 0x7fa0d00693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.398+0000 7fa0f1f9d700 1 --2- 172.21.15.55:0/3087317805 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0ec159420 0x7fa0ec156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.398+0000 7fa0f1f9d700 1 --2- 172.21.15.55:0/3087317805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0ec1597f0 0x7fa0ec069510 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.398+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/3087317805 >> 172.21.15.55:0/3087317805 conn(0x7fa0ec0d0660 msgr2=0x7fa0ec15bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:19.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.398+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/3087317805 shutdown_connections 2023-10-06T03:33:19.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.398+0000 7fa0f1f9d700 1 -- 172.21.15.55:0/3087317805 wait complete. 2023-10-06T03:33:19.402 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 8 2023-10-06T03:33:19.418 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:19.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.603+0000 7f774f19d700 1 Processor -- start 2023-10-06T03:33:19.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.603+0000 7f774f19d700 1 -- start start 2023-10-06T03:33:19.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774f19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f774806a0a0 0x7f774815b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:19.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774f19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f774806dd20 0x7f774815b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:19.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774f19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f774806ae70 con 0x7f774806d950 2023-10-06T03:33:19.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774f19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77480c8a60 con 0x7f774806a0a0 2023-10-06T03:33:19.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774f19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774806a540 con 0x7f774806dd20 2023-10-06T03:33:19.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774d99a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f774806a0a0 0x7f774815b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:19.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774e19b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f774806d950 0x7f77480c8350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56222/0 (socket says 172.21.15.55:56222) 2023-10-06T03:33:19.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774e19b700 1 -- 172.21.15.55:0/3449728606 learned_addr learned my addr 172.21.15.55:0/3449728606 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:19.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774d99a700 1 --2- 172.21.15.55:0/3449728606 >> v2:172.21.15.55:3300/0 conn(0x7f774806a0a0 0x7f774815b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40994/0 (socket says 172.21.15.55:40994) 2023-10-06T03:33:19.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774e99c700 1 --2- 172.21.15.55:0/3449728606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f774806dd20 0x7f774815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:19.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774d99a700 1 -- 172.21.15.55:0/3449728606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f774806dd20 msgr2=0x7f774815b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:19.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774d99a700 1 --2- 172.21.15.55:0/3449728606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f774806dd20 0x7f774815b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774d99a700 1 -- 172.21.15.55:0/3449728606 >> v1:172.21.15.55:6789/0 conn(0x7f774806d950 legacy=0x7f77480c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:19.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774d199700 1 -- 172.21.15.55:0/3449728606 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800119922 0 0) 0x7f774806ae70 con 0x7f774806d950 2023-10-06T03:33:19.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774d99a700 1 -- 172.21.15.55:0/3449728606 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774806a820 con 0x7f774806a0a0 2023-10-06T03:33:19.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.605+0000 7f774e99c700 1 --2- 172.21.15.55:0/3449728606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f774806dd20 0x7f774815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:19.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.606+0000 7f774d99a700 1 --2- 172.21.15.55:0/3449728606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f774806a0a0 0x7f774815b410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f77380098e0 tx=0x7f77380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0b67375b1ffb3c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:19.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.606+0000 7f774d199700 1 -- 172.21.15.55:0/3449728606 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738009e80 con 0x7f774806a0a0 2023-10-06T03:33:19.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.606+0000 7f774d199700 1 -- 172.21.15.55:0/3449728606 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773800b860 con 0x7f774806a0a0 2023-10-06T03:33:19.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.606+0000 7f774d199700 1 -- 172.21.15.55:0/3449728606 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773800bbe0 con 0x7f774806a0a0 2023-10-06T03:33:19.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.606+0000 7f774f19d700 1 -- 172.21.15.55:0/3449728606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f774806a0a0 msgr2=0x7f774815b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:19.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.606+0000 7f774f19d700 1 --2- 172.21.15.55:0/3449728606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f774806a0a0 0x7f774815b410 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f77380098e0 tx=0x7f77380091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.607+0000 7f774f19d700 1 -- 172.21.15.55:0/3449728606 shutdown_connections 2023-10-06T03:33:19.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.607+0000 7f774f19d700 1 --2- 172.21.15.55:0/3449728606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f774806dd20 0x7f774815b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.607+0000 7f774f19d700 1 --2- 172.21.15.55:0/3449728606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f774806a0a0 0x7f774815b410 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.607+0000 7f774f19d700 1 -- 172.21.15.55:0/3449728606 >> 172.21.15.55:0/3449728606 conn(0x7f77480cc1f0 msgr2=0x7f77480cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:19.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.607+0000 7f774f19d700 1 -- 172.21.15.55:0/3449728606 shutdown_connections 2023-10-06T03:33:19.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.607+0000 7f774f19d700 1 -- 172.21.15.55:0/3449728606 wait complete. 2023-10-06T03:33:19.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.608+0000 7f774f19d700 1 Processor -- start 2023-10-06T03:33:19.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.608+0000 7f774f19d700 1 -- start start 2023-10-06T03:33:19.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.609+0000 7f774f19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f774806a0a0 0x7f7748146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:19.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.609+0000 7f774f19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f774806d950 0x7f774816a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:19.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.609+0000 7f774f19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f774806dd20 0x7f77480b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:19.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.609+0000 7f774f19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77480b1010 con 0x7f774806dd20 2023-10-06T03:33:19.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.609+0000 7f774f19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7748147320 con 0x7f774806d950 2023-10-06T03:33:19.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.609+0000 7f774f19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77480b0400 con 0x7f774806a0a0 2023-10-06T03:33:19.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.609+0000 7f774e99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f774806dd20 0x7f77480b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:19.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.609+0000 7f774e99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f774806dd20 0x7f77480b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41006/0 (socket says 172.21.15.55:41006) 2023-10-06T03:33:19.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.609+0000 7f774e99c700 1 -- 172.21.15.55:0/534390600 learned_addr learned my addr 172.21.15.55:0/534390600 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:19.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.609+0000 7f774e19b700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f774806a0a0 0x7f7748146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:19.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.610+0000 7f774d99a700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f774806d950 0x7f774816a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:19.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.610+0000 7f774e99c700 1 -- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f774806a0a0 msgr2=0x7f7748146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:19.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.610+0000 7f774e99c700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f774806a0a0 0x7f7748146de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.610+0000 7f774e99c700 1 -- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f774806d950 msgr2=0x7f774816a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:19.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.610+0000 7f774e99c700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f774806d950 0x7f774816a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.610+0000 7f774e99c700 1 -- 172.21.15.55:0/534390600 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7738009420 con 0x7f774806dd20 2023-10-06T03:33:19.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.610+0000 7f774e19b700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f774806a0a0 0x7f7748146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:19.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.610+0000 7f774e99c700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f774806dd20 0x7f77480b14a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f774000d1b0 tx=0x7f7740009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:19.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.610+0000 7f773e7fc700 1 -- 172.21.15.55:0/534390600 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774000dda0 con 0x7f774806dd20 2023-10-06T03:33:19.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.610+0000 7f773e7fc700 1 -- 172.21.15.55:0/534390600 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774000f040 con 0x7f774806dd20 2023-10-06T03:33:19.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.610+0000 7f774f19d700 1 -- 172.21.15.55:0/534390600 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77480b06e0 con 0x7f774806dd20 2023-10-06T03:33:19.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.610+0000 7f773e7fc700 1 -- 172.21.15.55:0/534390600 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7740015690 con 0x7f774806dd20 2023-10-06T03:33:19.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.610+0000 7f774f19d700 1 -- 172.21.15.55:0/534390600 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77480b0ca0 con 0x7f774806dd20 2023-10-06T03:33:19.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.611+0000 7f774f19d700 1 -- 172.21.15.55:0/534390600 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77480b11c0 con 0x7f774806dd20 2023-10-06T03:33:19.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.612+0000 7f773e7fc700 1 -- 172.21.15.55:0/534390600 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f774001e3b0 con 0x7f774806dd20 2023-10-06T03:33:19.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.613+0000 7f773e7fc700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f772c066f30 0x7f772c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:19.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.613+0000 7f773e7fc700 1 -- 172.21.15.55:0/534390600 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7740084c90 con 0x7f774806dd20 2023-10-06T03:33:19.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.614+0000 7f774e19b700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f772c066f30 0x7f772c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:19.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.614+0000 7f774e19b700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f772c066f30 0x7f772c0693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f77440011e0 tx=0x7f7744009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:19.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.617+0000 7f773e7fc700 1 -- 172.21.15.55:0/534390600 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77480b11c0 con 0x7f774806dd20 2023-10-06T03:33:19.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.820+0000 7f774f19d700 1 -- 172.21.15.55:0/534390600 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77480b11c0 con 0x7f774806dd20 2023-10-06T03:33:19.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.821+0000 7f773e7fc700 1 -- 172.21.15.55:0/534390600 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f77480b11c0 con 0x7f774806dd20 2023-10-06T03:33:19.823 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:19.824 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:16.470817+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:19.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.824+0000 7f774f19d700 1 -- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f772c066f30 msgr2=0x7f772c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:19.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.824+0000 7f774f19d700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f772c066f30 0x7f772c0693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f77440011e0 tx=0x7f7744009040 comp rx=0 tx=0).stop 2023-10-06T03:33:19.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.824+0000 7f774f19d700 1 -- 172.21.15.55:0/534390600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f774806dd20 msgr2=0x7f77480b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:19.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.824+0000 7f774f19d700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f774806dd20 0x7f77480b14a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f774000d1b0 tx=0x7f7740009a50 comp rx=0 tx=0).stop 2023-10-06T03:33:19.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.824+0000 7f774f19d700 1 -- 172.21.15.55:0/534390600 shutdown_connections 2023-10-06T03:33:19.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.824+0000 7f774f19d700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f774806dd20 0x7f77480b14a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.824+0000 7f774f19d700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f772c066f30 0x7f772c0693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.824+0000 7f774f19d700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f774806a0a0 0x7f7748146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.824+0000 7f774f19d700 1 --2- 172.21.15.55:0/534390600 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f774806d950 0x7f774816a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:19.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.824+0000 7f774f19d700 1 -- 172.21.15.55:0/534390600 >> 172.21.15.55:0/534390600 conn(0x7f77480cc1f0 msgr2=0x7f77480c9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:19.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.824+0000 7f774f19d700 1 -- 172.21.15.55:0/534390600 shutdown_connections 2023-10-06T03:33:19.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:19.824+0000 7f774f19d700 1 -- 172.21.15.55:0/534390600 wait complete. 2023-10-06T03:33:19.846 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:33:19.847 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:33:20.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.033+0000 7ff60279d700 1 Processor -- start 2023-10-06T03:33:20.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.033+0000 7ff60279d700 1 -- start start 2023-10-06T03:33:20.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff60279d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff5fc150180 0x7ff5fc157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff60279d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5fc0c7410 0x7ff5fc069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff60279d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5fc06e2a0 con 0x7ff5fc150550 2023-10-06T03:33:20.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff60279d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5fc06b2e0 con 0x7ff5fc150180 2023-10-06T03:33:20.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff60279d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0c78e0 con 0x7ff5fc0c7410 2023-10-06T03:33:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff60179b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff5fc150180 0x7ff5fc157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff60179b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff5fc150180 0x7ff5fc157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41008/0 (socket says 172.21.15.55:41008) 2023-10-06T03:33:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff60179b700 1 -- 172.21.15.55:0/627178220 learned_addr learned my addr 172.21.15.55:0/627178220 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff600f9a700 1 --1- 172.21.15.55:0/627178220 >> v1:172.21.15.55:6789/0 conn(0x7ff5fc150550 0x7ff5fc0c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56236/0 (socket says 172.21.15.55:56236) 2023-10-06T03:33:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff60179b700 1 -- 172.21.15.55:0/627178220 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5fc0c7410 msgr2=0x7ff5fc069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff601f9c700 1 --2- 172.21.15.55:0/627178220 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5fc0c7410 0x7ff5fc069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff60179b700 1 --2- 172.21.15.55:0/627178220 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5fc0c7410 0x7ff5fc069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff60179b700 1 -- 172.21.15.55:0/627178220 >> v1:172.21.15.55:6789/0 conn(0x7ff5fc150550 legacy=0x7ff5fc0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff60179b700 1 -- 172.21.15.55:0/627178220 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc0c7bc0 con 0x7ff5fc150180 2023-10-06T03:33:20.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff601f9c700 1 --2- 172.21.15.55:0/627178220 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5fc0c7410 0x7ff5fc069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:20.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.035+0000 7ff5f3fff700 1 -- 172.21.15.55:0/627178220 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367768200 0 0) 0x7ff5fc06e2a0 con 0x7ff5fc150550 2023-10-06T03:33:20.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.036+0000 7ff60179b700 1 --2- 172.21.15.55:0/627178220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5fc150180 0x7ff5fc157770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800bc40 tx=0x7ff5f800b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f631dbdfadf0fd6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:20.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.036+0000 7ff5f3fff700 1 -- 172.21.15.55:0/627178220 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800db40 con 0x7ff5fc150180 2023-10-06T03:33:20.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.036+0000 7ff5f3fff700 1 -- 172.21.15.55:0/627178220 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f800dca0 con 0x7ff5fc150180 2023-10-06T03:33:20.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.036+0000 7ff5f3fff700 1 -- 172.21.15.55:0/627178220 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8017640 con 0x7ff5fc150180 2023-10-06T03:33:20.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.036+0000 7ff60279d700 1 -- 172.21.15.55:0/627178220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5fc150180 msgr2=0x7ff5fc157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.036+0000 7ff60279d700 1 --2- 172.21.15.55:0/627178220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5fc150180 0x7ff5fc157770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800bc40 tx=0x7ff5f800b4e0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.036+0000 7ff60279d700 1 -- 172.21.15.55:0/627178220 shutdown_connections 2023-10-06T03:33:20.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.036+0000 7ff60279d700 1 --2- 172.21.15.55:0/627178220 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5fc0c7410 0x7ff5fc069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.036+0000 7ff60279d700 1 --2- 172.21.15.55:0/627178220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5fc150180 0x7ff5fc157770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.036+0000 7ff60279d700 1 -- 172.21.15.55:0/627178220 >> 172.21.15.55:0/627178220 conn(0x7ff5fc0cc1f0 msgr2=0x7ff5fc0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.036+0000 7ff60279d700 1 -- 172.21.15.55:0/627178220 shutdown_connections 2023-10-06T03:33:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.037+0000 7ff60279d700 1 -- 172.21.15.55:0/627178220 wait complete. 2023-10-06T03:33:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.038+0000 7ff60279d700 1 Processor -- start 2023-10-06T03:33:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.038+0000 7ff60279d700 1 -- start start 2023-10-06T03:33:20.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.039+0000 7ff60279d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5fc0c7410 0x7ff5fc14a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.039+0000 7ff60279d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5fc150180 0x7ff5fc0b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.039+0000 7ff60279d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5fc150550 0x7ff5fc0ad720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.039+0000 7ff60279d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0b05b0 con 0x7ff5fc150550 2023-10-06T03:33:20.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.039+0000 7ff60279d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0afc80 con 0x7ff5fc0c7410 2023-10-06T03:33:20.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.039+0000 7ff60279d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc0aff60 con 0x7ff5fc150180 2023-10-06T03:33:20.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.039+0000 7ff601f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5fc150550 0x7ff5fc0ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.039+0000 7ff601f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5fc150550 0x7ff5fc0ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41018/0 (socket says 172.21.15.55:41018) 2023-10-06T03:33:20.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.039+0000 7ff601f9c700 1 -- 172.21.15.55:0/946512474 learned_addr learned my addr 172.21.15.55:0/946512474 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:20.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.040+0000 7ff600f9a700 1 --2- 172.21.15.55:0/946512474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5fc150180 0x7ff5fc0b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.040+0000 7ff60179b700 1 --2- 172.21.15.55:0/946512474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5fc0c7410 0x7ff5fc14a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.040+0000 7ff601f9c700 1 -- 172.21.15.55:0/946512474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5fc150180 msgr2=0x7ff5fc0b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.040+0000 7ff601f9c700 1 --2- 172.21.15.55:0/946512474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5fc150180 0x7ff5fc0b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.040+0000 7ff601f9c700 1 -- 172.21.15.55:0/946512474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5fc0c7410 msgr2=0x7ff5fc14a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.040+0000 7ff601f9c700 1 --2- 172.21.15.55:0/946512474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5fc0c7410 0x7ff5fc14a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.040+0000 7ff601f9c700 1 -- 172.21.15.55:0/946512474 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f800b780 con 0x7ff5fc150550 2023-10-06T03:33:20.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.040+0000 7ff601f9c700 1 --2- 172.21.15.55:0/946512474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5fc150550 0x7ff5fc0ad720 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400cf90 tx=0x7ff5f40078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.040+0000 7ff5f17fa700 1 -- 172.21.15.55:0/946512474 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f400ae60 con 0x7ff5fc150550 2023-10-06T03:33:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.040+0000 7ff5f17fa700 1 -- 172.21.15.55:0/946512474 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f4015440 con 0x7ff5fc150550 2023-10-06T03:33:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.040+0000 7ff5f17fa700 1 -- 172.21.15.55:0/946512474 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4015800 con 0x7ff5fc150550 2023-10-06T03:33:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.040+0000 7ff60279d700 1 -- 172.21.15.55:0/946512474 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5fc0b0240 con 0x7ff5fc150550 2023-10-06T03:33:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.041+0000 7ff60279d700 1 -- 172.21.15.55:0/946512474 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5fc0af950 con 0x7ff5fc150550 2023-10-06T03:33:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.042+0000 7ff60279d700 1 -- 172.21.15.55:0/946512474 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5fc0b03d0 con 0x7ff5fc150550 2023-10-06T03:33:20.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.043+0000 7ff5f17fa700 1 -- 172.21.15.55:0/946512474 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5f400d440 con 0x7ff5fc150550 2023-10-06T03:33:20.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.043+0000 7ff5f17fa700 1 --2- 172.21.15.55:0/946512474 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff5e0066f30 0x7ff5e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.044+0000 7ff5f17fa700 1 -- 172.21.15.55:0/946512474 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff5f4085320 con 0x7ff5fc150550 2023-10-06T03:33:20.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.044+0000 7ff60179b700 1 --2- 172.21.15.55:0/946512474 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff5e0066f30 0x7ff5e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.044+0000 7ff60179b700 1 --2- 172.21.15.55:0/946512474 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff5e0066f30 0x7ff5e00693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800b520 tx=0x7ff5f8019000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:20.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.048+0000 7ff5f17fa700 1 -- 172.21.15.55:0/946512474 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5f4052430 con 0x7ff5fc150550 2023-10-06T03:33:20.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.246+0000 7ff60279d700 1 -- 172.21.15.55:0/946512474 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5fc0afdf0 con 0x7ff5e0066f30 2023-10-06T03:33:20.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.258+0000 7ff5f17fa700 1 -- 172.21.15.55:0/946512474 <== mgr.4114 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286935 (secure 0 0 0) 0x7ff5fc0afdf0 con 0x7ff5e0066f30 2023-10-06T03:33:20.264 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:20.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.262+0000 7ff60279d700 1 -- 172.21.15.55:0/946512474 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff5e0066f30 msgr2=0x7ff5e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.262+0000 7ff60279d700 1 --2- 172.21.15.55:0/946512474 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff5e0066f30 0x7ff5e00693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800b520 tx=0x7ff5f8019000 comp rx=0 tx=0).stop 2023-10-06T03:33:20.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.262+0000 7ff60279d700 1 -- 172.21.15.55:0/946512474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5fc150550 msgr2=0x7ff5fc0ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.262+0000 7ff60279d700 1 --2- 172.21.15.55:0/946512474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5fc150550 0x7ff5fc0ad720 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff5f400cf90 tx=0x7ff5f40078a0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.262+0000 7ff60279d700 1 -- 172.21.15.55:0/946512474 shutdown_connections 2023-10-06T03:33:20.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.262+0000 7ff60279d700 1 --2- 172.21.15.55:0/946512474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5fc150550 0x7ff5fc0ad720 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.262+0000 7ff60279d700 1 --2- 172.21.15.55:0/946512474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5fc0c7410 0x7ff5fc14a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.262+0000 7ff60279d700 1 --2- 172.21.15.55:0/946512474 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff5e0066f30 0x7ff5e00693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.262+0000 7ff60279d700 1 --2- 172.21.15.55:0/946512474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5fc150180 0x7ff5fc0b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.262+0000 7ff60279d700 1 -- 172.21.15.55:0/946512474 >> 172.21.15.55:0/946512474 conn(0x7ff5fc0cc1f0 msgr2=0x7ff5fc157ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:20.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.262+0000 7ff60279d700 1 -- 172.21.15.55:0/946512474 shutdown_connections 2023-10-06T03:33:20.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.262+0000 7ff60279d700 1 -- 172.21.15.55:0/946512474 wait complete. 2023-10-06T03:33:20.268 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:33:20.281 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-06T03:33:18.489355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397580,"kb_used_data":2892,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752577140,"statfs":{"total":773094113280,"available":770638991360,"internally_reserved":0,"allocated":2961408,"data_stored":1808427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.626782"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.489781+0000","last_change":"2023-10-06T03:33:15.485195+0000","last_active":"2023-10-06T03:33:15.489781+0000","last_peered":"2023-10-06T03:33:15.489781+0000","last_clean":"2023-10-06T03:33:15.489781+0000","last_became_active":"2023-10-06T03:33:15.484848+0000","last_became_peered":"2023-10-06T03:33:15.484848+0000","last_unstale":"2023-10-06T03:33:15.489781+0000","last_undegraded":"2023-10-06T03:33:15.489781+0000","last_fullsized":"2023-10-06T03:33:15.489781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:08.862091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502702+0000","last_change":"2023-10-06T03:33:15.498977+0000","last_active":"2023-10-06T03:33:15.502702+0000","last_peered":"2023-10-06T03:33:15.502702+0000","last_clean":"2023-10-06T03:33:15.502702+0000","last_became_active":"2023-10-06T03:33:15.498671+0000","last_became_peered":"2023-10-06T03:33:15.498671+0000","last_unstale":"2023-10-06T03:33:15.502702+0000","last_undegraded":"2023-10-06T03:33:15.502702+0000","last_fullsized":"2023-10-06T03:33:15.502702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500827+0000","last_change":"2023-10-06T03:33:15.494700+0000","last_active":"2023-10-06T03:33:15.500827+0000","last_peered":"2023-10-06T03:33:15.500827+0000","last_clean":"2023-10-06T03:33:15.500827+0000","last_became_active":"2023-10-06T03:33:15.494361+0000","last_became_peered":"2023-10-06T03:33:15.494361+0000","last_unstale":"2023-10-06T03:33:15.500827+0000","last_undegraded":"2023-10-06T03:33:15.500827+0000","last_fullsized":"2023-10-06T03:33:15.500827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:14.663102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.488889+0000","last_change":"2023-10-06T03:33:15.485455+0000","last_active":"2023-10-06T03:33:15.488889+0000","last_peered":"2023-10-06T03:33:15.488889+0000","last_clean":"2023-10-06T03:33:15.488889+0000","last_became_active":"2023-10-06T03:33:15.485159+0000","last_became_peered":"2023-10-06T03:33:15.485159+0000","last_unstale":"2023-10-06T03:33:15.488889+0000","last_undegraded":"2023-10-06T03:33:15.488889+0000","last_fullsized":"2023-10-06T03:33:15.488889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:02.594885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500387+0000","last_change":"2023-10-06T03:33:15.498785+0000","last_active":"2023-10-06T03:33:15.500387+0000","last_peered":"2023-10-06T03:33:15.500387+0000","last_clean":"2023-10-06T03:33:15.500387+0000","last_became_active":"2023-10-06T03:33:15.498503+0000","last_became_peered":"2023-10-06T03:33:15.498503+0000","last_unstale":"2023-10-06T03:33:15.500387+0000","last_undegraded":"2023-10-06T03:33:15.500387+0000","last_fullsized":"2023-10-06T03:33:15.500387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:59.545999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494228+0000","last_change":"2023-10-06T03:33:15.484222+0000","last_active":"2023-10-06T03:33:15.494228+0000","last_peered":"2023-10-06T03:33:15.494228+0000","last_clean":"2023-10-06T03:33:15.494228+0000","last_became_active":"2023-10-06T03:33:15.483910+0000","last_became_peered":"2023-10-06T03:33:15.483910+0000","last_unstale":"2023-10-06T03:33:15.494228+0000","last_undegraded":"2023-10-06T03:33:15.494228+0000","last_fullsized":"2023-10-06T03:33:15.494228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:41.309883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501590+0000","last_change":"2023-10-06T03:33:15.497903+0000","last_active":"2023-10-06T03:33:15.501590+0000","last_peered":"2023-10-06T03:33:15.501590+0000","last_clean":"2023-10-06T03:33:15.501590+0000","last_became_active":"2023-10-06T03:33:15.496979+0000","last_became_peered":"2023-10-06T03:33:15.496979+0000","last_unstale":"2023-10-06T03:33:15.501590+0000","last_undegraded":"2023-10-06T03:33:15.501590+0000","last_fullsized":"2023-10-06T03:33:15.501590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502728+0000","last_change":"2023-10-06T03:33:15.501214+0000","last_active":"2023-10-06T03:33:15.502728+0000","last_peered":"2023-10-06T03:33:15.502728+0000","last_clean":"2023-10-06T03:33:15.502728+0000","last_became_active":"2023-10-06T03:33:15.500739+0000","last_became_peered":"2023-10-06T03:33:15.500739+0000","last_unstale":"2023-10-06T03:33:15.502728+0000","last_undegraded":"2023-10-06T03:33:15.502728+0000","last_fullsized":"2023-10-06T03:33:15.502728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.964350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.499625+0000","last_change":"2023-10-06T03:33:15.498383+0000","last_active":"2023-10-06T03:33:15.499625+0000","last_peered":"2023-10-06T03:33:15.499625+0000","last_clean":"2023-10-06T03:33:15.499625+0000","last_became_active":"2023-10-06T03:33:15.498162+0000","last_became_peered":"2023-10-06T03:33:15.498162+0000","last_unstale":"2023-10-06T03:33:15.499625+0000","last_undegraded":"2023-10-06T03:33:15.499625+0000","last_fullsized":"2023-10-06T03:33:15.499625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:02.889905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501799+0000","last_change":"2023-10-06T03:33:15.496957+0000","last_active":"2023-10-06T03:33:15.501799+0000","last_peered":"2023-10-06T03:33:15.501799+0000","last_clean":"2023-10-06T03:33:15.501799+0000","last_became_active":"2023-10-06T03:33:15.496673+0000","last_became_peered":"2023-10-06T03:33:15.496673+0000","last_unstale":"2023-10-06T03:33:15.501799+0000","last_undegraded":"2023-10-06T03:33:15.501799+0000","last_fullsized":"2023-10-06T03:33:15.501799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500848+0000","last_change":"2023-10-06T03:33:15.495118+0000","last_active":"2023-10-06T03:33:15.500848+0000","last_peered":"2023-10-06T03:33:15.500848+0000","last_clean":"2023-10-06T03:33:15.500848+0000","last_became_active":"2023-10-06T03:33:15.494732+0000","last_became_peered":"2023-10-06T03:33:15.494732+0000","last_unstale":"2023-10-06T03:33:15.500848+0000","last_undegraded":"2023-10-06T03:33:15.500848+0000","last_fullsized":"2023-10-06T03:33:15.500848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:02.889905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494010+0000","last_change":"2023-10-06T03:33:15.491772+0000","last_active":"2023-10-06T03:33:15.494010+0000","last_peered":"2023-10-06T03:33:15.494010+0000","last_clean":"2023-10-06T03:33:15.494010+0000","last_became_active":"2023-10-06T03:33:15.491530+0000","last_became_peered":"2023-10-06T03:33:15.491530+0000","last_unstale":"2023-10-06T03:33:15.494010+0000","last_undegraded":"2023-10-06T03:33:15.494010+0000","last_fullsized":"2023-10-06T03:33:15.494010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:57.097910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.489816+0000","last_change":"2023-10-06T03:33:15.485334+0000","last_active":"2023-10-06T03:33:15.489816+0000","last_peered":"2023-10-06T03:33:15.489816+0000","last_clean":"2023-10-06T03:33:15.489816+0000","last_became_active":"2023-10-06T03:33:15.485051+0000","last_became_peered":"2023-10-06T03:33:15.485051+0000","last_unstale":"2023-10-06T03:33:15.489816+0000","last_undegraded":"2023-10-06T03:33:15.489816+0000","last_fullsized":"2023-10-06T03:33:15.489816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:42.519384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498233+0000","last_change":"2023-10-06T03:33:15.495208+0000","last_active":"2023-10-06T03:33:15.498233+0000","last_peered":"2023-10-06T03:33:15.498233+0000","last_clean":"2023-10-06T03:33:15.498233+0000","last_became_active":"2023-10-06T03:33:15.494897+0000","last_became_peered":"2023-10-06T03:33:15.494897+0000","last_unstale":"2023-10-06T03:33:15.498233+0000","last_undegraded":"2023-10-06T03:33:15.498233+0000","last_fullsized":"2023-10-06T03:33:15.498233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:12.635649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501608+0000","last_change":"2023-10-06T03:33:15.498114+0000","last_active":"2023-10-06T03:33:15.501608+0000","last_peered":"2023-10-06T03:33:15.501608+0000","last_clean":"2023-10-06T03:33:15.501608+0000","last_became_active":"2023-10-06T03:33:15.497401+0000","last_became_peered":"2023-10-06T03:33:15.497401+0000","last_unstale":"2023-10-06T03:33:15.501608+0000","last_undegraded":"2023-10-06T03:33:15.501608+0000","last_fullsized":"2023-10-06T03:33:15.501608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494198+0000","last_change":"2023-10-06T03:33:15.484297+0000","last_active":"2023-10-06T03:33:15.494198+0000","last_peered":"2023-10-06T03:33:15.494198+0000","last_clean":"2023-10-06T03:33:15.494198+0000","last_became_active":"2023-10-06T03:33:15.483921+0000","last_became_peered":"2023-10-06T03:33:15.483921+0000","last_unstale":"2023-10-06T03:33:15.494198+0000","last_undegraded":"2023-10-06T03:33:15.494198+0000","last_fullsized":"2023-10-06T03:33:15.494198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:03.085548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.495074+0000","last_change":"2023-10-06T03:33:15.492170+0000","last_active":"2023-10-06T03:33:15.495074+0000","last_peered":"2023-10-06T03:33:15.495074+0000","last_clean":"2023-10-06T03:33:15.495074+0000","last_became_active":"2023-10-06T03:33:15.491847+0000","last_became_peered":"2023-10-06T03:33:15.491847+0000","last_unstale":"2023-10-06T03:33:15.495074+0000","last_undegraded":"2023-10-06T03:33:15.495074+0000","last_fullsized":"2023-10-06T03:33:15.495074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:30.660724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496120+0000","last_change":"2023-10-06T03:33:15.492652+0000","last_active":"2023-10-06T03:33:15.496120+0000","last_peered":"2023-10-06T03:33:15.496120+0000","last_clean":"2023-10-06T03:33:15.496120+0000","last_became_active":"2023-10-06T03:33:15.491815+0000","last_became_peered":"2023-10-06T03:33:15.491815+0000","last_unstale":"2023-10-06T03:33:15.496120+0000","last_undegraded":"2023-10-06T03:33:15.496120+0000","last_fullsized":"2023-10-06T03:33:15.496120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:20.661516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.489846+0000","last_change":"2023-10-06T03:33:15.486081+0000","last_active":"2023-10-06T03:33:15.489846+0000","last_peered":"2023-10-06T03:33:15.489846+0000","last_clean":"2023-10-06T03:33:15.489846+0000","last_became_active":"2023-10-06T03:33:15.485842+0000","last_became_peered":"2023-10-06T03:33:15.485842+0000","last_unstale":"2023-10-06T03:33:15.489846+0000","last_undegraded":"2023-10-06T03:33:15.489846+0000","last_fullsized":"2023-10-06T03:33:15.489846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:16.260090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496154+0000","last_change":"2023-10-06T03:33:15.490482+0000","last_active":"2023-10-06T03:33:15.496154+0000","last_peered":"2023-10-06T03:33:15.496154+0000","last_clean":"2023-10-06T03:33:15.496154+0000","last_became_active":"2023-10-06T03:33:15.490185+0000","last_became_peered":"2023-10-06T03:33:15.490185+0000","last_unstale":"2023-10-06T03:33:15.496154+0000","last_undegraded":"2023-10-06T03:33:15.496154+0000","last_fullsized":"2023-10-06T03:33:15.496154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:05.828284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502137+0000","last_change":"2023-10-06T03:33:15.499609+0000","last_active":"2023-10-06T03:33:15.502137+0000","last_peered":"2023-10-06T03:33:15.502137+0000","last_clean":"2023-10-06T03:33:15.502137+0000","last_became_active":"2023-10-06T03:33:15.499317+0000","last_became_peered":"2023-10-06T03:33:15.499317+0000","last_unstale":"2023-10-06T03:33:15.502137+0000","last_undegraded":"2023-10-06T03:33:15.502137+0000","last_fullsized":"2023-10-06T03:33:15.502137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494425+0000","last_change":"2023-10-06T03:33:15.491198+0000","last_active":"2023-10-06T03:33:15.494425+0000","last_peered":"2023-10-06T03:33:15.494425+0000","last_clean":"2023-10-06T03:33:15.494425+0000","last_became_active":"2023-10-06T03:33:15.490933+0000","last_became_peered":"2023-10-06T03:33:15.490933+0000","last_unstale":"2023-10-06T03:33:15.494425+0000","last_undegraded":"2023-10-06T03:33:15.494425+0000","last_fullsized":"2023-10-06T03:33:15.494425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500604+0000","last_change":"2023-10-06T03:33:15.492259+0000","last_active":"2023-10-06T03:33:15.500604+0000","last_peered":"2023-10-06T03:33:15.500604+0000","last_clean":"2023-10-06T03:33:15.500604+0000","last_became_active":"2023-10-06T03:33:15.492018+0000","last_became_peered":"2023-10-06T03:33:15.492018+0000","last_unstale":"2023-10-06T03:33:15.500604+0000","last_undegraded":"2023-10-06T03:33:15.500604+0000","last_fullsized":"2023-10-06T03:33:15.500604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:31:34.027062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501954+0000","last_change":"2023-10-06T03:33:15.500133+0000","last_active":"2023-10-06T03:33:15.501954+0000","last_peered":"2023-10-06T03:33:15.501954+0000","last_clean":"2023-10-06T03:33:15.501954+0000","last_became_active":"2023-10-06T03:33:15.499092+0000","last_became_peered":"2023-10-06T03:33:15.499092+0000","last_unstale":"2023-10-06T03:33:15.501954+0000","last_undegraded":"2023-10-06T03:33:15.501954+0000","last_fullsized":"2023-10-06T03:33:15.501954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493234+0000","last_change":"2023-10-06T03:33:15.487133+0000","last_active":"2023-10-06T03:33:15.493234+0000","last_peered":"2023-10-06T03:33:15.493234+0000","last_clean":"2023-10-06T03:33:15.493234+0000","last_became_active":"2023-10-06T03:33:15.486582+0000","last_became_peered":"2023-10-06T03:33:15.486582+0000","last_unstale":"2023-10-06T03:33:15.493234+0000","last_undegraded":"2023-10-06T03:33:15.493234+0000","last_fullsized":"2023-10-06T03:33:15.493234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:37.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498958+0000","last_change":"2023-10-06T03:33:15.489677+0000","last_active":"2023-10-06T03:33:15.498958+0000","last_peered":"2023-10-06T03:33:15.498958+0000","last_clean":"2023-10-06T03:33:15.498958+0000","last_became_active":"2023-10-06T03:33:15.489431+0000","last_became_peered":"2023-10-06T03:33:15.489431+0000","last_unstale":"2023-10-06T03:33:15.498958+0000","last_undegraded":"2023-10-06T03:33:15.498958+0000","last_fullsized":"2023-10-06T03:33:15.498958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502491+0000","last_change":"2023-10-06T03:33:15.498305+0000","last_active":"2023-10-06T03:33:15.502491+0000","last_peered":"2023-10-06T03:33:15.502491+0000","last_clean":"2023-10-06T03:33:15.502491+0000","last_became_active":"2023-10-06T03:33:15.497920+0000","last_became_peered":"2023-10-06T03:33:15.497920+0000","last_unstale":"2023-10-06T03:33:15.502491+0000","last_undegraded":"2023-10-06T03:33:15.502491+0000","last_fullsized":"2023-10-06T03:33:15.502491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.880457+0000","last_change":"2023-10-06T03:33:14.894963+0000","last_active":"2023-10-06T03:33:15.880457+0000","last_peered":"2023-10-06T03:33:15.880457+0000","last_clean":"2023-10-06T03:33:15.880457+0000","last_became_active":"2023-10-06T03:33:14.886952+0000","last_became_peered":"2023-10-06T03:33:14.886952+0000","last_unstale":"2023-10-06T03:33:15.880457+0000","last_undegraded":"2023-10-06T03:33:15.880457+0000","last_fullsized":"2023-10-06T03:33:15.880457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501009+0000","last_change":"2023-10-06T03:33:15.498264+0000","last_active":"2023-10-06T03:33:15.501009+0000","last_peered":"2023-10-06T03:33:15.501009+0000","last_clean":"2023-10-06T03:33:15.501009+0000","last_became_active":"2023-10-06T03:33:15.498030+0000","last_became_peered":"2023-10-06T03:33:15.498030+0000","last_unstale":"2023-10-06T03:33:15.501009+0000","last_undegraded":"2023-10-06T03:33:15.501009+0000","last_fullsized":"2023-10-06T03:33:15.501009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:05.828284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.473505+0000","last_change":"2023-10-06T03:33:14.895404+0000","last_active":"2023-10-06T03:33:15.473505+0000","last_peered":"2023-10-06T03:33:15.473505+0000","last_clean":"2023-10-06T03:33:15.473505+0000","last_became_active":"2023-10-06T03:33:14.894742+0000","last_became_peered":"2023-10-06T03:33:14.894742+0000","last_unstale":"2023-10-06T03:33:15.473505+0000","last_undegraded":"2023-10-06T03:33:15.473505+0000","last_fullsized":"2023-10-06T03:33:15.473505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490157+0000","last_change":"2023-10-06T03:33:15.485976+0000","last_active":"2023-10-06T03:33:15.490157+0000","last_peered":"2023-10-06T03:33:15.490157+0000","last_clean":"2023-10-06T03:33:15.490157+0000","last_became_active":"2023-10-06T03:33:15.485658+0000","last_became_peered":"2023-10-06T03:33:15.485658+0000","last_unstale":"2023-10-06T03:33:15.490157+0000","last_undegraded":"2023-10-06T03:33:15.490157+0000","last_fullsized":"2023-10-06T03:33:15.490157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482088+0000","last_change":"2023-10-06T03:33:14.903103+0000","last_active":"2023-10-06T03:33:15.482088+0000","last_peered":"2023-10-06T03:33:15.482088+0000","last_clean":"2023-10-06T03:33:15.482088+0000","last_became_active":"2023-10-06T03:33:14.902707+0000","last_became_peered":"2023-10-06T03:33:14.902707+0000","last_unstale":"2023-10-06T03:33:15.482088+0000","last_undegraded":"2023-10-06T03:33:15.482088+0000","last_fullsized":"2023-10-06T03:33:15.482088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501406+0000","last_change":"2023-10-06T03:33:15.496437+0000","last_active":"2023-10-06T03:33:15.501406+0000","last_peered":"2023-10-06T03:33:15.501406+0000","last_clean":"2023-10-06T03:33:15.501406+0000","last_became_active":"2023-10-06T03:33:15.496138+0000","last_became_peered":"2023-10-06T03:33:15.496138+0000","last_unstale":"2023-10-06T03:33:15.501406+0000","last_undegraded":"2023-10-06T03:33:15.501406+0000","last_fullsized":"2023-10-06T03:33:15.501406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:09:05.289396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.478575+0000","last_change":"2023-10-06T03:33:14.897019+0000","last_active":"2023-10-06T03:33:15.478575+0000","last_peered":"2023-10-06T03:33:15.478575+0000","last_clean":"2023-10-06T03:33:15.478575+0000","last_became_active":"2023-10-06T03:33:14.896740+0000","last_became_peered":"2023-10-06T03:33:14.896740+0000","last_unstale":"2023-10-06T03:33:15.478575+0000","last_undegraded":"2023-10-06T03:33:15.478575+0000","last_fullsized":"2023-10-06T03:33:15.478575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500886+0000","last_change":"2023-10-06T03:33:15.496977+0000","last_active":"2023-10-06T03:33:15.500886+0000","last_peered":"2023-10-06T03:33:15.500886+0000","last_clean":"2023-10-06T03:33:15.500886+0000","last_became_active":"2023-10-06T03:33:15.496726+0000","last_became_peered":"2023-10-06T03:33:15.496726+0000","last_unstale":"2023-10-06T03:33:15.500886+0000","last_undegraded":"2023-10-06T03:33:15.500886+0000","last_fullsized":"2023-10-06T03:33:15.500886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:02.889905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.491600+0000","last_change":"2023-10-06T03:33:14.896854+0000","last_active":"2023-10-06T03:33:15.491600+0000","last_peered":"2023-10-06T03:33:15.491600+0000","last_clean":"2023-10-06T03:33:15.491600+0000","last_became_active":"2023-10-06T03:33:14.896606+0000","last_became_peered":"2023-10-06T03:33:14.896606+0000","last_unstale":"2023-10-06T03:33:15.491600+0000","last_undegraded":"2023-10-06T03:33:15.491600+0000","last_fullsized":"2023-10-06T03:33:15.491600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500655+0000","last_change":"2023-10-06T03:33:15.494940+0000","last_active":"2023-10-06T03:33:15.500655+0000","last_peered":"2023-10-06T03:33:15.500655+0000","last_clean":"2023-10-06T03:33:15.500655+0000","last_became_active":"2023-10-06T03:33:15.494251+0000","last_became_peered":"2023-10-06T03:33:15.494251+0000","last_unstale":"2023-10-06T03:33:15.500655+0000","last_undegraded":"2023-10-06T03:33:15.500655+0000","last_fullsized":"2023-10-06T03:33:15.500655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.885266+0000","last_change":"2023-10-06T03:33:14.895082+0000","last_active":"2023-10-06T03:33:15.885266+0000","last_peered":"2023-10-06T03:33:15.885266+0000","last_clean":"2023-10-06T03:33:15.885266+0000","last_became_active":"2023-10-06T03:33:14.894804+0000","last_became_peered":"2023-10-06T03:33:14.894804+0000","last_unstale":"2023-10-06T03:33:15.885266+0000","last_undegraded":"2023-10-06T03:33:15.885266+0000","last_fullsized":"2023-10-06T03:33:15.885266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:01.585501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493308+0000","last_change":"2023-10-06T03:33:15.487049+0000","last_active":"2023-10-06T03:33:15.493308+0000","last_peered":"2023-10-06T03:33:15.493308+0000","last_clean":"2023-10-06T03:33:15.493308+0000","last_became_active":"2023-10-06T03:33:15.486777+0000","last_became_peered":"2023-10-06T03:33:15.486777+0000","last_unstale":"2023-10-06T03:33:15.493308+0000","last_undegraded":"2023-10-06T03:33:15.493308+0000","last_fullsized":"2023-10-06T03:33:15.493308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:12.635649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.888581+0000","last_change":"2023-10-06T03:33:14.898475+0000","last_active":"2023-10-06T03:33:15.888581+0000","last_peered":"2023-10-06T03:33:15.888581+0000","last_clean":"2023-10-06T03:33:15.888581+0000","last_became_active":"2023-10-06T03:33:14.898144+0000","last_became_peered":"2023-10-06T03:33:14.898144+0000","last_unstale":"2023-10-06T03:33:15.888581+0000","last_undegraded":"2023-10-06T03:33:15.888581+0000","last_fullsized":"2023-10-06T03:33:15.888581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494412+0000","last_change":"2023-10-06T03:33:15.487549+0000","last_active":"2023-10-06T03:33:15.494412+0000","last_peered":"2023-10-06T03:33:15.494412+0000","last_clean":"2023-10-06T03:33:15.494412+0000","last_became_active":"2023-10-06T03:33:15.487277+0000","last_became_peered":"2023-10-06T03:33:15.487277+0000","last_unstale":"2023-10-06T03:33:15.494412+0000","last_undegraded":"2023-10-06T03:33:15.494412+0000","last_fullsized":"2023-10-06T03:33:15.494412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:07.613983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.885740+0000","last_change":"2023-10-06T03:33:14.896271+0000","last_active":"2023-10-06T03:33:15.885740+0000","last_peered":"2023-10-06T03:33:15.885740+0000","last_clean":"2023-10-06T03:33:15.885740+0000","last_became_active":"2023-10-06T03:33:14.896004+0000","last_became_peered":"2023-10-06T03:33:14.896004+0000","last_unstale":"2023-10-06T03:33:15.885740+0000","last_undegraded":"2023-10-06T03:33:15.885740+0000","last_fullsized":"2023-10-06T03:33:15.885740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:15.567270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500942+0000","last_change":"2023-10-06T03:33:15.497569+0000","last_active":"2023-10-06T03:33:15.500942+0000","last_peered":"2023-10-06T03:33:15.500942+0000","last_clean":"2023-10-06T03:33:15.500942+0000","last_became_active":"2023-10-06T03:33:15.497287+0000","last_became_peered":"2023-10-06T03:33:15.497287+0000","last_unstale":"2023-10-06T03:33:15.500942+0000","last_undegraded":"2023-10-06T03:33:15.500942+0000","last_fullsized":"2023-10-06T03:33:15.500942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:47.107849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.476556+0000","last_change":"2023-10-06T03:33:14.897199+0000","last_active":"2023-10-06T03:33:15.476556+0000","last_peered":"2023-10-06T03:33:15.476556+0000","last_clean":"2023-10-06T03:33:15.476556+0000","last_became_active":"2023-10-06T03:33:14.896925+0000","last_became_peered":"2023-10-06T03:33:14.896925+0000","last_unstale":"2023-10-06T03:33:15.476556+0000","last_undegraded":"2023-10-06T03:33:15.476556+0000","last_fullsized":"2023-10-06T03:33:15.476556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:41.510001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498210+0000","last_change":"2023-10-06T03:33:15.495381+0000","last_active":"2023-10-06T03:33:15.498210+0000","last_peered":"2023-10-06T03:33:15.498210+0000","last_clean":"2023-10-06T03:33:15.498210+0000","last_became_active":"2023-10-06T03:33:15.494972+0000","last_became_peered":"2023-10-06T03:33:15.494972+0000","last_unstale":"2023-10-06T03:33:15.498210+0000","last_undegraded":"2023-10-06T03:33:15.498210+0000","last_fullsized":"2023-10-06T03:33:15.498210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.483344+0000","last_change":"2023-10-06T03:33:14.901135+0000","last_active":"2023-10-06T03:33:15.483344+0000","last_peered":"2023-10-06T03:33:15.483344+0000","last_clean":"2023-10-06T03:33:15.483344+0000","last_became_active":"2023-10-06T03:33:14.900871+0000","last_became_peered":"2023-10-06T03:33:14.900871+0000","last_unstale":"2023-10-06T03:33:15.483344+0000","last_undegraded":"2023-10-06T03:33:15.483344+0000","last_fullsized":"2023-10-06T03:33:15.483344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494496+0000","last_change":"2023-10-06T03:33:15.491822+0000","last_active":"2023-10-06T03:33:15.494496+0000","last_peered":"2023-10-06T03:33:15.494496+0000","last_clean":"2023-10-06T03:33:15.494496+0000","last_became_active":"2023-10-06T03:33:15.491560+0000","last_became_peered":"2023-10-06T03:33:15.491560+0000","last_unstale":"2023-10-06T03:33:15.494496+0000","last_undegraded":"2023-10-06T03:33:15.494496+0000","last_fullsized":"2023-10-06T03:33:15.494496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.964350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886225+0000","last_change":"2023-10-06T03:33:14.898845+0000","last_active":"2023-10-06T03:33:15.886225+0000","last_peered":"2023-10-06T03:33:15.886225+0000","last_clean":"2023-10-06T03:33:15.886225+0000","last_became_active":"2023-10-06T03:33:14.898491+0000","last_became_peered":"2023-10-06T03:33:14.898491+0000","last_unstale":"2023-10-06T03:33:15.886225+0000","last_undegraded":"2023-10-06T03:33:15.886225+0000","last_fullsized":"2023-10-06T03:33:15.886225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500404+0000","last_change":"2023-10-06T03:33:15.487774+0000","last_active":"2023-10-06T03:33:15.500404+0000","last_peered":"2023-10-06T03:33:15.500404+0000","last_clean":"2023-10-06T03:33:15.500404+0000","last_became_active":"2023-10-06T03:33:15.487515+0000","last_became_peered":"2023-10-06T03:33:15.487515+0000","last_unstale":"2023-10-06T03:33:15.500404+0000","last_undegraded":"2023-10-06T03:33:15.500404+0000","last_fullsized":"2023-10-06T03:33:15.500404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:30.660724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.884947+0000","last_change":"2023-10-06T03:33:14.896171+0000","last_active":"2023-10-06T03:33:15.884947+0000","last_peered":"2023-10-06T03:33:15.884947+0000","last_clean":"2023-10-06T03:33:15.884947+0000","last_became_active":"2023-10-06T03:33:14.895903+0000","last_became_peered":"2023-10-06T03:33:14.895903+0000","last_unstale":"2023-10-06T03:33:15.884947+0000","last_undegraded":"2023-10-06T03:33:15.884947+0000","last_fullsized":"2023-10-06T03:33:15.884947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:44.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500464+0000","last_change":"2023-10-06T03:33:15.496285+0000","last_active":"2023-10-06T03:33:15.500464+0000","last_peered":"2023-10-06T03:33:15.500464+0000","last_clean":"2023-10-06T03:33:15.500464+0000","last_became_active":"2023-10-06T03:33:15.495868+0000","last_became_peered":"2023-10-06T03:33:15.495868+0000","last_unstale":"2023-10-06T03:33:15.500464+0000","last_undegraded":"2023-10-06T03:33:15.500464+0000","last_fullsized":"2023-10-06T03:33:15.500464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.885984+0000","last_change":"2023-10-06T03:33:14.896322+0000","last_active":"2023-10-06T03:33:15.885984+0000","last_peered":"2023-10-06T03:33:15.885984+0000","last_clean":"2023-10-06T03:33:15.885984+0000","last_became_active":"2023-10-06T03:33:14.896022+0000","last_became_peered":"2023-10-06T03:33:14.896022+0000","last_unstale":"2023-10-06T03:33:15.885984+0000","last_undegraded":"2023-10-06T03:33:15.885984+0000","last_fullsized":"2023-10-06T03:33:15.885984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:21.680218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500853+0000","last_change":"2023-10-06T03:33:15.494878+0000","last_active":"2023-10-06T03:33:15.500853+0000","last_peered":"2023-10-06T03:33:15.500853+0000","last_clean":"2023-10-06T03:33:15.500853+0000","last_became_active":"2023-10-06T03:33:15.494597+0000","last_became_peered":"2023-10-06T03:33:15.494597+0000","last_unstale":"2023-10-06T03:33:15.500853+0000","last_undegraded":"2023-10-06T03:33:15.500853+0000","last_fullsized":"2023-10-06T03:33:15.500853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:30.660724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.887336+0000","last_change":"2023-10-06T03:33:14.897659+0000","last_active":"2023-10-06T03:33:15.887336+0000","last_peered":"2023-10-06T03:33:15.887336+0000","last_clean":"2023-10-06T03:33:15.887336+0000","last_became_active":"2023-10-06T03:33:14.897368+0000","last_became_peered":"2023-10-06T03:33:14.897368+0000","last_unstale":"2023-10-06T03:33:15.887336+0000","last_undegraded":"2023-10-06T03:33:15.887336+0000","last_fullsized":"2023-10-06T03:33:15.887336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:46:12.361089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493933+0000","last_change":"2023-10-06T03:33:15.492136+0000","last_active":"2023-10-06T03:33:15.493933+0000","last_peered":"2023-10-06T03:33:15.493933+0000","last_clean":"2023-10-06T03:33:15.493933+0000","last_became_active":"2023-10-06T03:33:15.491858+0000","last_became_peered":"2023-10-06T03:33:15.491858+0000","last_unstale":"2023-10-06T03:33:15.493933+0000","last_undegraded":"2023-10-06T03:33:15.493933+0000","last_fullsized":"2023-10-06T03:33:15.493933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:16.576653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490953+0000","last_change":"2023-10-06T03:33:14.890596+0000","last_active":"2023-10-06T03:33:15.490953+0000","last_peered":"2023-10-06T03:33:15.490953+0000","last_clean":"2023-10-06T03:33:15.490953+0000","last_became_active":"2023-10-06T03:33:14.889486+0000","last_became_peered":"2023-10-06T03:33:14.889486+0000","last_unstale":"2023-10-06T03:33:15.490953+0000","last_undegraded":"2023-10-06T03:33:15.490953+0000","last_fullsized":"2023-10-06T03:33:15.490953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:41.510001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.495179+0000","last_change":"2023-10-06T03:33:15.493860+0000","last_active":"2023-10-06T03:33:15.495179+0000","last_peered":"2023-10-06T03:33:15.495179+0000","last_clean":"2023-10-06T03:33:15.495179+0000","last_became_active":"2023-10-06T03:33:15.493610+0000","last_became_peered":"2023-10-06T03:33:15.493610+0000","last_unstale":"2023-10-06T03:33:15.495179+0000","last_undegraded":"2023-10-06T03:33:15.495179+0000","last_fullsized":"2023-10-06T03:33:15.495179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.881042+0000","last_change":"2023-10-06T03:33:14.903094+0000","last_active":"2023-10-06T03:33:15.881042+0000","last_peered":"2023-10-06T03:33:15.881042+0000","last_clean":"2023-10-06T03:33:15.881042+0000","last_became_active":"2023-10-06T03:33:14.902686+0000","last_became_peered":"2023-10-06T03:33:14.902686+0000","last_unstale":"2023-10-06T03:33:15.881042+0000","last_undegraded":"2023-10-06T03:33:15.881042+0000","last_fullsized":"2023-10-06T03:33:15.881042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496134+0000","last_change":"2023-10-06T03:33:15.489915+0000","last_active":"2023-10-06T03:33:15.496134+0000","last_peered":"2023-10-06T03:33:15.496134+0000","last_clean":"2023-10-06T03:33:15.496134+0000","last_became_active":"2023-10-06T03:33:15.488544+0000","last_became_peered":"2023-10-06T03:33:15.488544+0000","last_unstale":"2023-10-06T03:33:15.496134+0000","last_undegraded":"2023-10-06T03:33:15.496134+0000","last_fullsized":"2023-10-06T03:33:15.496134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.478249+0000","last_change":"2023-10-06T03:33:14.886562+0000","last_active":"2023-10-06T03:33:15.478249+0000","last_peered":"2023-10-06T03:33:15.478249+0000","last_clean":"2023-10-06T03:33:15.478249+0000","last_became_active":"2023-10-06T03:33:14.886157+0000","last_became_peered":"2023-10-06T03:33:14.886157+0000","last_unstale":"2023-10-06T03:33:15.478249+0000","last_undegraded":"2023-10-06T03:33:15.478249+0000","last_fullsized":"2023-10-06T03:33:15.478249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496415+0000","last_change":"2023-10-06T03:33:15.491844+0000","last_active":"2023-10-06T03:33:15.496415+0000","last_peered":"2023-10-06T03:33:15.496415+0000","last_clean":"2023-10-06T03:33:15.496415+0000","last_became_active":"2023-10-06T03:33:15.491574+0000","last_became_peered":"2023-10-06T03:33:15.491574+0000","last_unstale":"2023-10-06T03:33:15.496415+0000","last_undegraded":"2023-10-06T03:33:15.496415+0000","last_fullsized":"2023-10-06T03:33:15.496415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:35.426668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482572+0000","last_change":"2023-10-06T03:33:14.895658+0000","last_active":"2023-10-06T03:33:15.482572+0000","last_peered":"2023-10-06T03:33:15.482572+0000","last_clean":"2023-10-06T03:33:15.482572+0000","last_became_active":"2023-10-06T03:33:14.895332+0000","last_became_peered":"2023-10-06T03:33:14.895332+0000","last_unstale":"2023-10-06T03:33:15.482572+0000","last_undegraded":"2023-10-06T03:33:15.482572+0000","last_fullsized":"2023-10-06T03:33:15.482572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:28.684332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496089+0000","last_change":"2023-10-06T03:33:15.490478+0000","last_active":"2023-10-06T03:33:15.496089+0000","last_peered":"2023-10-06T03:33:15.496089+0000","last_clean":"2023-10-06T03:33:15.496089+0000","last_became_active":"2023-10-06T03:33:15.485974+0000","last_became_peered":"2023-10-06T03:33:15.485974+0000","last_unstale":"2023-10-06T03:33:15.496089+0000","last_undegraded":"2023-10-06T03:33:15.496089+0000","last_fullsized":"2023-10-06T03:33:15.496089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.481946+0000","last_change":"2023-10-06T03:33:14.891288+0000","last_active":"2023-10-06T03:33:15.481946+0000","last_peered":"2023-10-06T03:33:15.481946+0000","last_clean":"2023-10-06T03:33:15.481946+0000","last_became_active":"2023-10-06T03:33:14.890186+0000","last_became_peered":"2023-10-06T03:33:14.890186+0000","last_unstale":"2023-10-06T03:33:15.481946+0000","last_undegraded":"2023-10-06T03:33:15.481946+0000","last_fullsized":"2023-10-06T03:33:15.481946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:21.680218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493182+0000","last_change":"2023-10-06T03:33:15.486956+0000","last_active":"2023-10-06T03:33:15.493182+0000","last_peered":"2023-10-06T03:33:15.493182+0000","last_clean":"2023-10-06T03:33:15.493182+0000","last_became_active":"2023-10-06T03:33:15.486188+0000","last_became_peered":"2023-10-06T03:33:15.486188+0000","last_unstale":"2023-10-06T03:33:15.493182+0000","last_undegraded":"2023-10-06T03:33:15.493182+0000","last_fullsized":"2023-10-06T03:33:15.493182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.887149+0000","last_change":"2023-10-06T03:33:14.895426+0000","last_active":"2023-10-06T03:33:15.887149+0000","last_peered":"2023-10-06T03:33:15.887149+0000","last_clean":"2023-10-06T03:33:15.887149+0000","last_became_active":"2023-10-06T03:33:14.893425+0000","last_became_peered":"2023-10-06T03:33:14.893425+0000","last_unstale":"2023-10-06T03:33:15.887149+0000","last_undegraded":"2023-10-06T03:33:15.887149+0000","last_fullsized":"2023-10-06T03:33:15.887149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:44.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.484841+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:33:15.484841+0000","last_peered":"2023-10-06T03:33:15.484841+0000","last_clean":"2023-10-06T03:33:15.484841+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:33:15.484841+0000","last_undegraded":"2023-10-06T03:33:15.484841+0000","last_fullsized":"2023-10-06T03:33:15.484841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500598+0000","last_change":"2023-10-06T03:33:15.492324+0000","last_active":"2023-10-06T03:33:15.500598+0000","last_peered":"2023-10-06T03:33:15.500598+0000","last_clean":"2023-10-06T03:33:15.500598+0000","last_became_active":"2023-10-06T03:33:15.492037+0000","last_became_peered":"2023-10-06T03:33:15.492037+0000","last_unstale":"2023-10-06T03:33:15.500598+0000","last_undegraded":"2023-10-06T03:33:15.500598+0000","last_fullsized":"2023-10-06T03:33:15.500598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.484831+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:33:15.484831+0000","last_peered":"2023-10-06T03:33:15.484831+0000","last_clean":"2023-10-06T03:33:15.484831+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:33:15.484831+0000","last_undegraded":"2023-10-06T03:33:15.484831+0000","last_fullsized":"2023-10-06T03:33:15.484831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.477783+0000","last_change":"2023-10-06T03:33:14.896368+0000","last_active":"2023-10-06T03:33:15.477783+0000","last_peered":"2023-10-06T03:33:15.477783+0000","last_clean":"2023-10-06T03:33:15.477783+0000","last_became_active":"2023-10-06T03:33:14.892459+0000","last_became_peered":"2023-10-06T03:33:14.892459+0000","last_unstale":"2023-10-06T03:33:15.477783+0000","last_undegraded":"2023-10-06T03:33:15.477783+0000","last_fullsized":"2023-10-06T03:33:15.477783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501686+0000","last_change":"2023-10-06T03:33:15.499324+0000","last_active":"2023-10-06T03:33:15.501686+0000","last_peered":"2023-10-06T03:33:15.501686+0000","last_clean":"2023-10-06T03:33:15.501686+0000","last_became_active":"2023-10-06T03:33:15.499039+0000","last_became_peered":"2023-10-06T03:33:15.499039+0000","last_unstale":"2023-10-06T03:33:15.501686+0000","last_undegraded":"2023-10-06T03:33:15.501686+0000","last_fullsized":"2023-10-06T03:33:15.501686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:43.269972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.473587+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:33:15.473587+0000","last_peered":"2023-10-06T03:33:15.473587+0000","last_clean":"2023-10-06T03:33:15.473587+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:33:15.473587+0000","last_undegraded":"2023-10-06T03:33:15.473587+0000","last_fullsized":"2023-10-06T03:33:15.473587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.483628+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:33:15.483628+0000","last_peered":"2023-10-06T03:33:15.483628+0000","last_clean":"2023-10-06T03:33:15.483628+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:33:15.483628+0000","last_undegraded":"2023-10-06T03:33:15.483628+0000","last_fullsized":"2023-10-06T03:33:15.483628+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.476509+0000","last_change":"2023-10-06T03:33:14.891206+0000","last_active":"2023-10-06T03:33:15.476509+0000","last_peered":"2023-10-06T03:33:15.476509+0000","last_clean":"2023-10-06T03:33:15.476509+0000","last_became_active":"2023-10-06T03:33:14.890823+0000","last_became_peered":"2023-10-06T03:33:14.890823+0000","last_unstale":"2023-10-06T03:33:15.476509+0000","last_undegraded":"2023-10-06T03:33:15.476509+0000","last_fullsized":"2023-10-06T03:33:15.476509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:28.684332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490137+0000","last_change":"2023-10-06T03:33:15.483582+0000","last_active":"2023-10-06T03:33:15.490137+0000","last_peered":"2023-10-06T03:33:15.490137+0000","last_clean":"2023-10-06T03:33:15.490137+0000","last_became_active":"2023-10-06T03:33:15.483324+0000","last_became_peered":"2023-10-06T03:33:15.483324+0000","last_unstale":"2023-10-06T03:33:15.490137+0000","last_undegraded":"2023-10-06T03:33:15.490137+0000","last_fullsized":"2023-10-06T03:33:15.490137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:12.635649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482905+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:33:15.482905+0000","last_peered":"2023-10-06T03:33:15.482905+0000","last_clean":"2023-10-06T03:33:15.482905+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:33:15.482905+0000","last_undegraded":"2023-10-06T03:33:15.482905+0000","last_fullsized":"2023-10-06T03:33:15.482905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.474679+0000","last_change":"2023-10-06T03:33:14.902096+0000","last_active":"2023-10-06T03:33:15.474679+0000","last_peered":"2023-10-06T03:33:15.474679+0000","last_clean":"2023-10-06T03:33:15.474679+0000","last_became_active":"2023-10-06T03:33:14.896599+0000","last_became_peered":"2023-10-06T03:33:14.896599+0000","last_unstale":"2023-10-06T03:33:15.474679+0000","last_undegraded":"2023-10-06T03:33:15.474679+0000","last_fullsized":"2023-10-06T03:33:15.474679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502084+0000","last_change":"2023-10-06T03:33:15.500022+0000","last_active":"2023-10-06T03:33:15.502084+0000","last_peered":"2023-10-06T03:33:15.502084+0000","last_clean":"2023-10-06T03:33:15.502084+0000","last_became_active":"2023-10-06T03:33:15.499610+0000","last_became_peered":"2023-10-06T03:33:15.499610+0000","last_unstale":"2023-10-06T03:33:15.502084+0000","last_undegraded":"2023-10-06T03:33:15.502084+0000","last_fullsized":"2023-10-06T03:33:15.502084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:58.092449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.483163+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:33:15.483163+0000","last_peered":"2023-10-06T03:33:15.483163+0000","last_clean":"2023-10-06T03:33:15.483163+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:33:15.483163+0000","last_undegraded":"2023-10-06T03:33:15.483163+0000","last_fullsized":"2023-10-06T03:33:15.483163+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482546+0000","last_change":"2023-10-06T03:33:14.895562+0000","last_active":"2023-10-06T03:33:15.482546+0000","last_peered":"2023-10-06T03:33:15.482546+0000","last_clean":"2023-10-06T03:33:15.482546+0000","last_became_active":"2023-10-06T03:33:14.893552+0000","last_became_peered":"2023-10-06T03:33:14.893552+0000","last_unstale":"2023-10-06T03:33:15.482546+0000","last_undegraded":"2023-10-06T03:33:15.482546+0000","last_fullsized":"2023-10-06T03:33:15.482546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:41.510001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498193+0000","last_change":"2023-10-06T03:33:15.495951+0000","last_active":"2023-10-06T03:33:15.498193+0000","last_peered":"2023-10-06T03:33:15.498193+0000","last_clean":"2023-10-06T03:33:15.498193+0000","last_became_active":"2023-10-06T03:33:15.495675+0000","last_became_peered":"2023-10-06T03:33:15.495675+0000","last_unstale":"2023-10-06T03:33:15.498193+0000","last_undegraded":"2023-10-06T03:33:15.498193+0000","last_fullsized":"2023-10-06T03:33:15.498193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:46:13.370472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490686+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:33:15.490686+0000","last_peered":"2023-10-06T03:33:15.490686+0000","last_clean":"2023-10-06T03:33:15.490686+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:33:15.490686+0000","last_undegraded":"2023-10-06T03:33:15.490686+0000","last_fullsized":"2023-10-06T03:33:15.490686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.480469+0000","last_change":"2023-10-06T03:33:14.901103+0000","last_active":"2023-10-06T03:33:15.480469+0000","last_peered":"2023-10-06T03:33:15.480469+0000","last_clean":"2023-10-06T03:33:15.480469+0000","last_became_active":"2023-10-06T03:33:14.900784+0000","last_became_peered":"2023-10-06T03:33:14.900784+0000","last_unstale":"2023-10-06T03:33:15.480469+0000","last_undegraded":"2023-10-06T03:33:15.480469+0000","last_fullsized":"2023-10-06T03:33:15.480469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493280+0000","last_change":"2023-10-06T03:33:15.488395+0000","last_active":"2023-10-06T03:33:15.493280+0000","last_peered":"2023-10-06T03:33:15.493280+0000","last_clean":"2023-10-06T03:33:15.493280+0000","last_became_active":"2023-10-06T03:33:15.483269+0000","last_became_peered":"2023-10-06T03:33:15.483269+0000","last_unstale":"2023-10-06T03:33:15.493280+0000","last_undegraded":"2023-10-06T03:33:15.493280+0000","last_fullsized":"2023-10-06T03:33:15.493280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.477353+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:33:15.477353+0000","last_peered":"2023-10-06T03:33:15.477353+0000","last_clean":"2023-10-06T03:33:15.477353+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:33:15.477353+0000","last_undegraded":"2023-10-06T03:33:15.477353+0000","last_fullsized":"2023-10-06T03:33:15.477353+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.474703+0000","last_change":"2023-10-06T03:33:14.896667+0000","last_active":"2023-10-06T03:33:15.474703+0000","last_peered":"2023-10-06T03:33:15.474703+0000","last_clean":"2023-10-06T03:33:15.474703+0000","last_became_active":"2023-10-06T03:33:14.895890+0000","last_became_peered":"2023-10-06T03:33:14.895890+0000","last_unstale":"2023-10-06T03:33:15.474703+0000","last_undegraded":"2023-10-06T03:33:15.474703+0000","last_fullsized":"2023-10-06T03:33:15.474703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:28.684332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498919+0000","last_change":"2023-10-06T03:33:15.496955+0000","last_active":"2023-10-06T03:33:15.498919+0000","last_peered":"2023-10-06T03:33:15.498919+0000","last_clean":"2023-10-06T03:33:15.498919+0000","last_became_active":"2023-10-06T03:33:15.496666+0000","last_became_peered":"2023-10-06T03:33:15.496666+0000","last_unstale":"2023-10-06T03:33:15.498919+0000","last_undegraded":"2023-10-06T03:33:15.498919+0000","last_fullsized":"2023-10-06T03:33:15.498919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.478419+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:33:15.478419+0000","last_peered":"2023-10-06T03:33:15.478419+0000","last_clean":"2023-10-06T03:33:15.478419+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:33:15.478419+0000","last_undegraded":"2023-10-06T03:33:15.478419+0000","last_fullsized":"2023-10-06T03:33:15.478419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.491835+0000","last_change":"2023-10-06T03:33:14.893120+0000","last_active":"2023-10-06T03:33:15.491835+0000","last_peered":"2023-10-06T03:33:15.491835+0000","last_clean":"2023-10-06T03:33:15.491835+0000","last_became_active":"2023-10-06T03:33:14.892839+0000","last_became_peered":"2023-10-06T03:33:14.892839+0000","last_unstale":"2023-10-06T03:33:15.491835+0000","last_undegraded":"2023-10-06T03:33:15.491835+0000","last_fullsized":"2023-10-06T03:33:15.491835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:56.088527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501655+0000","last_change":"2023-10-06T03:33:15.496296+0000","last_active":"2023-10-06T03:33:15.501655+0000","last_peered":"2023-10-06T03:33:15.501655+0000","last_clean":"2023-10-06T03:33:15.501655+0000","last_became_active":"2023-10-06T03:33:15.496028+0000","last_became_peered":"2023-10-06T03:33:15.496028+0000","last_unstale":"2023-10-06T03:33:15.501655+0000","last_undegraded":"2023-10-06T03:33:15.501655+0000","last_fullsized":"2023-10-06T03:33:15.501655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.884176+0000","last_change":"2023-10-06T03:33:14.897368+0000","last_active":"2023-10-06T03:33:15.884176+0000","last_peered":"2023-10-06T03:33:15.884176+0000","last_clean":"2023-10-06T03:33:15.884176+0000","last_became_active":"2023-10-06T03:33:14.892728+0000","last_became_peered":"2023-10-06T03:33:14.892728+0000","last_unstale":"2023-10-06T03:33:15.884176+0000","last_undegraded":"2023-10-06T03:33:15.884176+0000","last_fullsized":"2023-10-06T03:33:15.884176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494671+0000","last_change":"2023-10-06T03:33:15.493442+0000","last_active":"2023-10-06T03:33:15.494671+0000","last_peered":"2023-10-06T03:33:15.494671+0000","last_clean":"2023-10-06T03:33:15.494671+0000","last_became_active":"2023-10-06T03:33:15.493169+0000","last_became_peered":"2023-10-06T03:33:15.493169+0000","last_unstale":"2023-10-06T03:33:15.494671+0000","last_undegraded":"2023-10-06T03:33:15.494671+0000","last_fullsized":"2023-10-06T03:33:15.494671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:12.900415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886080+0000","last_change":"2023-10-06T03:33:14.899177+0000","last_active":"2023-10-06T03:33:15.886080+0000","last_peered":"2023-10-06T03:33:15.886080+0000","last_clean":"2023-10-06T03:33:15.886080+0000","last_became_active":"2023-10-06T03:33:14.898884+0000","last_became_peered":"2023-10-06T03:33:14.898884+0000","last_unstale":"2023-10-06T03:33:15.886080+0000","last_undegraded":"2023-10-06T03:33:15.886080+0000","last_fullsized":"2023-10-06T03:33:15.886080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501741+0000","last_change":"2023-10-06T03:33:15.499965+0000","last_active":"2023-10-06T03:33:15.501741+0000","last_peered":"2023-10-06T03:33:15.501741+0000","last_clean":"2023-10-06T03:33:15.501741+0000","last_became_active":"2023-10-06T03:33:15.499644+0000","last_became_peered":"2023-10-06T03:33:15.499644+0000","last_unstale":"2023-10-06T03:33:15.501741+0000","last_undegraded":"2023-10-06T03:33:15.501741+0000","last_fullsized":"2023-10-06T03:33:15.501741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:47.107849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886042+0000","last_change":"2023-10-06T03:33:14.898472+0000","last_active":"2023-10-06T03:33:15.886042+0000","last_peered":"2023-10-06T03:33:15.886042+0000","last_clean":"2023-10-06T03:33:15.886042+0000","last_became_active":"2023-10-06T03:33:14.898230+0000","last_became_peered":"2023-10-06T03:33:14.898230+0000","last_unstale":"2023-10-06T03:33:15.886042+0000","last_undegraded":"2023-10-06T03:33:15.886042+0000","last_fullsized":"2023-10-06T03:33:15.886042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494448+0000","last_change":"2023-10-06T03:33:15.488792+0000","last_active":"2023-10-06T03:33:15.494448+0000","last_peered":"2023-10-06T03:33:15.494448+0000","last_clean":"2023-10-06T03:33:15.494448+0000","last_became_active":"2023-10-06T03:33:15.488504+0000","last_became_peered":"2023-10-06T03:33:15.488504+0000","last_unstale":"2023-10-06T03:33:15.494448+0000","last_undegraded":"2023-10-06T03:33:15.494448+0000","last_fullsized":"2023-10-06T03:33:15.494448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:59.545999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.884688+0000","last_change":"2023-10-06T03:33:14.882297+0000","last_active":"2023-10-06T03:33:15.884688+0000","last_peered":"2023-10-06T03:33:15.884688+0000","last_clean":"2023-10-06T03:33:15.884688+0000","last_became_active":"2023-10-06T03:33:14.881990+0000","last_became_peered":"2023-10-06T03:33:14.881990+0000","last_unstale":"2023-10-06T03:33:15.884688+0000","last_undegraded":"2023-10-06T03:33:15.884688+0000","last_fullsized":"2023-10-06T03:33:15.884688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:56.088527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500582+0000","last_change":"2023-10-06T03:33:15.491458+0000","last_active":"2023-10-06T03:33:15.500582+0000","last_peered":"2023-10-06T03:33:15.500582+0000","last_clean":"2023-10-06T03:33:15.500582+0000","last_became_active":"2023-10-06T03:33:15.491167+0000","last_became_peered":"2023-10-06T03:33:15.491167+0000","last_unstale":"2023-10-06T03:33:15.500582+0000","last_undegraded":"2023-10-06T03:33:15.500582+0000","last_fullsized":"2023-10-06T03:33:15.500582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.964350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886971+0000","last_change":"2023-10-06T03:33:14.896256+0000","last_active":"2023-10-06T03:33:15.886971+0000","last_peered":"2023-10-06T03:33:15.886971+0000","last_clean":"2023-10-06T03:33:15.886971+0000","last_became_active":"2023-10-06T03:33:14.893978+0000","last_became_peered":"2023-10-06T03:33:14.893978+0000","last_unstale":"2023-10-06T03:33:15.886971+0000","last_undegraded":"2023-10-06T03:33:15.886971+0000","last_fullsized":"2023-10-06T03:33:15.886971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:56.088527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494409+0000","last_change":"2023-10-06T03:33:15.485216+0000","last_active":"2023-10-06T03:33:15.494409+0000","last_peered":"2023-10-06T03:33:15.494409+0000","last_clean":"2023-10-06T03:33:15.494409+0000","last_became_active":"2023-10-06T03:33:15.484942+0000","last_became_peered":"2023-10-06T03:33:15.484942+0000","last_unstale":"2023-10-06T03:33:15.494409+0000","last_undegraded":"2023-10-06T03:33:15.494409+0000","last_fullsized":"2023-10-06T03:33:15.494409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:50.279026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.492719+0000","last_change":"2023-10-06T03:33:15.489726+0000","last_active":"2023-10-06T03:33:15.492719+0000","last_peered":"2023-10-06T03:33:15.492719+0000","last_clean":"2023-10-06T03:33:15.492719+0000","last_became_active":"2023-10-06T03:33:15.489424+0000","last_became_peered":"2023-10-06T03:33:15.489424+0000","last_unstale":"2023-10-06T03:33:15.492719+0000","last_undegraded":"2023-10-06T03:33:15.492719+0000","last_fullsized":"2023-10-06T03:33:15.492719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:37.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500986+0000","last_change":"2023-10-06T03:33:15.497582+0000","last_active":"2023-10-06T03:33:15.500986+0000","last_peered":"2023-10-06T03:33:15.500986+0000","last_clean":"2023-10-06T03:33:15.500986+0000","last_became_active":"2023-10-06T03:33:15.497328+0000","last_became_peered":"2023-10-06T03:33:15.497328+0000","last_unstale":"2023-10-06T03:33:15.500986+0000","last_undegraded":"2023-10-06T03:33:15.500986+0000","last_fullsized":"2023-10-06T03:33:15.500986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:50.279026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496144+0000","last_change":"2023-10-06T03:33:15.489875+0000","last_active":"2023-10-06T03:33:15.496144+0000","last_peered":"2023-10-06T03:33:15.496144+0000","last_clean":"2023-10-06T03:33:15.496144+0000","last_became_active":"2023-10-06T03:33:15.489634+0000","last_became_peered":"2023-10-06T03:33:15.489634+0000","last_unstale":"2023-10-06T03:33:15.496144+0000","last_undegraded":"2023-10-06T03:33:15.496144+0000","last_fullsized":"2023-10-06T03:33:15.496144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:31:34.027062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500770+0000","last_change":"2023-10-06T03:33:15.493502+0000","last_active":"2023-10-06T03:33:15.500770+0000","last_peered":"2023-10-06T03:33:15.500770+0000","last_clean":"2023-10-06T03:33:15.500770+0000","last_became_active":"2023-10-06T03:33:15.492769+0000","last_became_peered":"2023-10-06T03:33:15.492769+0000","last_unstale":"2023-10-06T03:33:15.500770+0000","last_undegraded":"2023-10-06T03:33:15.500770+0000","last_fullsized":"2023-10-06T03:33:15.500770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:59.545999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493258+0000","last_change":"2023-10-06T03:33:15.487849+0000","last_active":"2023-10-06T03:33:15.493258+0000","last_peered":"2023-10-06T03:33:15.493258+0000","last_clean":"2023-10-06T03:33:15.493258+0000","last_became_active":"2023-10-06T03:33:15.487604+0000","last_became_peered":"2023-10-06T03:33:15.487604+0000","last_unstale":"2023-10-06T03:33:15.493258+0000","last_undegraded":"2023-10-06T03:33:15.493258+0000","last_fullsized":"2023-10-06T03:33:15.493258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:47.342414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298104,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073736,"statfs":{"total":96636764160,"available":96331505664,"internally_reserved":0,"allocated":835584,"data_stored":670346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":79204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.619}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302176,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069664,"statfs":{"total":96636764160,"available":96327335936,"internally_reserved":0,"allocated":811008,"data_stored":669572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301596,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070244,"statfs":{"total":96636764160,"available":96327929856,"internally_reserved":0,"allocated":217088,"data_stored":78780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301592,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070248,"statfs":{"total":96636764160,"available":96327933952,"internally_reserved":0,"allocated":212992,"data_stored":78226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301588,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070252,"statfs":{"total":96636764160,"available":96327938048,"internally_reserved":0,"allocated":208896,"data_stored":72653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":80796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":78850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:33:20.292 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:33:20.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.506+0000 7f471359e700 1 Processor -- start 2023-10-06T03:33:20.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.506+0000 7f471359e700 1 -- start start 2023-10-06T03:33:20.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.508+0000 7f471359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4714156310 0x7f47140c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.508+0000 7f471359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f47141566e0 0x7f47140ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.508+0000 7f471359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47140ca860 con 0x7f47140ca300 2023-10-06T03:33:20.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.508+0000 7f471359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47140cab40 con 0x7f47141566e0 2023-10-06T03:33:20.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.508+0000 7f471359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47140cae20 con 0x7f4714156310 2023-10-06T03:33:20.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.508+0000 7f4711d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f47141566e0 0x7f47140ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.508+0000 7f4711d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f47141566e0 0x7f47140ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41024/0 (socket says 172.21.15.55:41024) 2023-10-06T03:33:20.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.508+0000 7f4711d9b700 1 -- 172.21.15.55:0/1179243501 learned_addr learned my addr 172.21.15.55:0/1179243501 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:20.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.508+0000 7f4712d9d700 1 --1- 172.21.15.55:0/1179243501 >> v1:172.21.15.55:6789/0 conn(0x7f47140ca300 0x7f47140c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56242/0 (socket says 172.21.15.55:56242) 2023-10-06T03:33:20.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.508+0000 7f471259c700 1 --2- 172.21.15.55:0/1179243501 >> v2:172.21.15.163:3300/0 conn(0x7f4714156310 0x7f47140c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.508+0000 7f471159a700 1 -- 172.21.15.55:0/1179243501 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109450409 0 0) 0x7f47140ca860 con 0x7f47140ca300 2023-10-06T03:33:20.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.508+0000 7f4711d9b700 1 -- 172.21.15.55:0/1179243501 >> v2:172.21.15.163:3300/0 conn(0x7f4714156310 msgr2=0x7f47140c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:20.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.508+0000 7f4711d9b700 1 --2- 172.21.15.55:0/1179243501 >> v2:172.21.15.163:3300/0 conn(0x7f4714156310 0x7f47140c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.509+0000 7f4711d9b700 1 -- 172.21.15.55:0/1179243501 >> v1:172.21.15.55:6789/0 conn(0x7f47140ca300 legacy=0x7f47140c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.509+0000 7f4711d9b700 1 -- 172.21.15.55:0/1179243501 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47140cb100 con 0x7f47141566e0 2023-10-06T03:33:20.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.509+0000 7f4711d9b700 1 --2- 172.21.15.55:0/1179243501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47141566e0 0x7f47140ae000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f47140cce20 tx=0x7f4708009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f4c4f7cf8c2ed6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:20.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.509+0000 7f471159a700 1 -- 172.21.15.55:0/1179243501 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708009e70 con 0x7f47141566e0 2023-10-06T03:33:20.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.509+0000 7f471159a700 1 -- 172.21.15.55:0/1179243501 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470800b860 con 0x7f47141566e0 2023-10-06T03:33:20.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.509+0000 7f471159a700 1 -- 172.21.15.55:0/1179243501 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470800bc00 con 0x7f47141566e0 2023-10-06T03:33:20.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.510+0000 7f471359e700 1 -- 172.21.15.55:0/1179243501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47141566e0 msgr2=0x7f47140ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.510+0000 7f471359e700 1 --2- 172.21.15.55:0/1179243501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47141566e0 0x7f47140ae000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f47140cce20 tx=0x7f4708009250 comp rx=0 tx=0).stop 2023-10-06T03:33:20.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.510+0000 7f471359e700 1 -- 172.21.15.55:0/1179243501 shutdown_connections 2023-10-06T03:33:20.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.510+0000 7f471359e700 1 --2- 172.21.15.55:0/1179243501 >> v2:172.21.15.163:3300/0 conn(0x7f4714156310 0x7f47140c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.510+0000 7f471359e700 1 --2- 172.21.15.55:0/1179243501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47141566e0 0x7f47140ae000 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.510+0000 7f471359e700 1 -- 172.21.15.55:0/1179243501 >> 172.21.15.55:0/1179243501 conn(0x7f47140cc1f0 msgr2=0x7f47140cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:20.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.510+0000 7f471359e700 1 -- 172.21.15.55:0/1179243501 shutdown_connections 2023-10-06T03:33:20.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.510+0000 7f471359e700 1 -- 172.21.15.55:0/1179243501 wait complete. 2023-10-06T03:33:20.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.511+0000 7f471359e700 1 Processor -- start 2023-10-06T03:33:20.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.511+0000 7f471359e700 1 -- start start 2023-10-06T03:33:20.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.512+0000 7f471359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47140ca300 0x7f471414f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.512+0000 7f471359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4714156310 0x7f4714069870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.512+0000 7f471359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47141566e0 0x7f4714069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.512+0000 7f471359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47141406f0 con 0x7f47141566e0 2023-10-06T03:33:20.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.512+0000 7f471359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471413f650 con 0x7f4714156310 2023-10-06T03:33:20.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.512+0000 7f471359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471413f930 con 0x7f47140ca300 2023-10-06T03:33:20.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.512+0000 7f4712d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47141566e0 0x7f4714069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.513+0000 7f4712d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47141566e0 0x7f4714069db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41040/0 (socket says 172.21.15.55:41040) 2023-10-06T03:33:20.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.513+0000 7f4712d9d700 1 -- 172.21.15.55:0/2068994774 learned_addr learned my addr 172.21.15.55:0/2068994774 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:20.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.513+0000 7f4711d9b700 1 --2- 172.21.15.55:0/2068994774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4714156310 0x7f4714069870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.513+0000 7f471259c700 1 --2- 172.21.15.55:0/2068994774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47140ca300 0x7f471414f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.513+0000 7f4712d9d700 1 -- 172.21.15.55:0/2068994774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47140ca300 msgr2=0x7f471414f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.513+0000 7f4712d9d700 1 --2- 172.21.15.55:0/2068994774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47140ca300 0x7f471414f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.513+0000 7f4712d9d700 1 -- 172.21.15.55:0/2068994774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4714156310 msgr2=0x7f4714069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.513+0000 7f4712d9d700 1 --2- 172.21.15.55:0/2068994774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4714156310 0x7f4714069870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.513+0000 7f4712d9d700 1 -- 172.21.15.55:0/2068994774 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47080094f0 con 0x7f47141566e0 2023-10-06T03:33:20.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.513+0000 7f4712d9d700 1 --2- 172.21.15.55:0/2068994774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47141566e0 0x7f4714069db0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f470c00cd50 tx=0x7f470c001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:20.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.513+0000 7f47027fc700 1 -- 172.21.15.55:0/2068994774 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c005df0 con 0x7f47141566e0 2023-10-06T03:33:20.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.513+0000 7f47027fc700 1 -- 172.21.15.55:0/2068994774 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c00a8b0 con 0x7f47141566e0 2023-10-06T03:33:20.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.513+0000 7f471359e700 1 -- 172.21.15.55:0/2068994774 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471413fc10 con 0x7f47141566e0 2023-10-06T03:33:20.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.513+0000 7f47027fc700 1 -- 172.21.15.55:0/2068994774 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c00ac70 con 0x7f47141566e0 2023-10-06T03:33:20.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.514+0000 7f471359e700 1 -- 172.21.15.55:0/2068994774 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47141401d0 con 0x7f47141566e0 2023-10-06T03:33:20.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.514+0000 7f471359e700 1 -- 172.21.15.55:0/2068994774 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47141403f0 con 0x7f47141566e0 2023-10-06T03:33:20.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.515+0000 7f47027fc700 1 -- 172.21.15.55:0/2068994774 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f470c007300 con 0x7f47141566e0 2023-10-06T03:33:20.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.517+0000 7f47027fc700 1 --2- 172.21.15.55:0/2068994774 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f46f4066f30 0x7f46f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.517+0000 7f47027fc700 1 -- 172.21.15.55:0/2068994774 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f470c084810 con 0x7f47141566e0 2023-10-06T03:33:20.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.517+0000 7f471259c700 1 --2- 172.21.15.55:0/2068994774 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f46f4066f30 0x7f46f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.518+0000 7f471259c700 1 --2- 172.21.15.55:0/2068994774 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f46f4066f30 0x7f46f40693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4704009db0 tx=0x7f47040091a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:20.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.521+0000 7f47027fc700 1 -- 172.21.15.55:0/2068994774 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f470c051900 con 0x7f47141566e0 2023-10-06T03:33:20.662 INFO:tasks.ceph.mds.b.smithi163.stderr:2023-10-06T03:33:20.660+0000 7fb0dedf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:33:20.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.720+0000 7f471359e700 1 -- 172.21.15.55:0/2068994774 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f471404c3c0 con 0x7f46f4066f30 2023-10-06T03:33:20.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.729+0000 7f47027fc700 1 -- 172.21.15.55:0/2068994774 <== mgr.4114 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286935 (secure 0 0 0) 0x7f471404c3c0 con 0x7f46f4066f30 2023-10-06T03:33:20.735 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.734+0000 7f471359e700 1 -- 172.21.15.55:0/2068994774 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f46f4066f30 msgr2=0x7f46f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.734+0000 7f471359e700 1 --2- 172.21.15.55:0/2068994774 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f46f4066f30 0x7f46f40693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4704009db0 tx=0x7f47040091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.734+0000 7f471359e700 1 -- 172.21.15.55:0/2068994774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47141566e0 msgr2=0x7f4714069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.734+0000 7f471359e700 1 --2- 172.21.15.55:0/2068994774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47141566e0 0x7f4714069db0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f470c00cd50 tx=0x7f470c001b10 comp rx=0 tx=0).stop 2023-10-06T03:33:20.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.734+0000 7f471359e700 1 -- 172.21.15.55:0/2068994774 shutdown_connections 2023-10-06T03:33:20.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.734+0000 7f471359e700 1 --2- 172.21.15.55:0/2068994774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47141566e0 0x7f4714069db0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.734+0000 7f471359e700 1 --2- 172.21.15.55:0/2068994774 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f46f4066f30 0x7f46f40693e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.734+0000 7f471359e700 1 --2- 172.21.15.55:0/2068994774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47140ca300 0x7f471414f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.734+0000 7f471359e700 1 --2- 172.21.15.55:0/2068994774 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4714156310 0x7f4714069870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.734+0000 7f471359e700 1 -- 172.21.15.55:0/2068994774 >> 172.21.15.55:0/2068994774 conn(0x7f47140cc1f0 msgr2=0x7f47140c18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:20.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.734+0000 7f471359e700 1 -- 172.21.15.55:0/2068994774 shutdown_connections 2023-10-06T03:33:20.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.734+0000 7f471359e700 1 -- 172.21.15.55:0/2068994774 wait complete. 2023-10-06T03:33:20.738 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:33:20.753 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-06T03:33:20.489953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397580,"kb_used_data":2892,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752577140,"statfs":{"total":773094113280,"available":770638991360,"internally_reserved":0,"allocated":2961408,"data_stored":1808427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001362"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.489781+0000","last_change":"2023-10-06T03:33:15.485195+0000","last_active":"2023-10-06T03:33:15.489781+0000","last_peered":"2023-10-06T03:33:15.489781+0000","last_clean":"2023-10-06T03:33:15.489781+0000","last_became_active":"2023-10-06T03:33:15.484848+0000","last_became_peered":"2023-10-06T03:33:15.484848+0000","last_unstale":"2023-10-06T03:33:15.489781+0000","last_undegraded":"2023-10-06T03:33:15.489781+0000","last_fullsized":"2023-10-06T03:33:15.489781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:08.862091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502702+0000","last_change":"2023-10-06T03:33:15.498977+0000","last_active":"2023-10-06T03:33:15.502702+0000","last_peered":"2023-10-06T03:33:15.502702+0000","last_clean":"2023-10-06T03:33:15.502702+0000","last_became_active":"2023-10-06T03:33:15.498671+0000","last_became_peered":"2023-10-06T03:33:15.498671+0000","last_unstale":"2023-10-06T03:33:15.502702+0000","last_undegraded":"2023-10-06T03:33:15.502702+0000","last_fullsized":"2023-10-06T03:33:15.502702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500827+0000","last_change":"2023-10-06T03:33:15.494700+0000","last_active":"2023-10-06T03:33:15.500827+0000","last_peered":"2023-10-06T03:33:15.500827+0000","last_clean":"2023-10-06T03:33:15.500827+0000","last_became_active":"2023-10-06T03:33:15.494361+0000","last_became_peered":"2023-10-06T03:33:15.494361+0000","last_unstale":"2023-10-06T03:33:15.500827+0000","last_undegraded":"2023-10-06T03:33:15.500827+0000","last_fullsized":"2023-10-06T03:33:15.500827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:14.663102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.488889+0000","last_change":"2023-10-06T03:33:15.485455+0000","last_active":"2023-10-06T03:33:15.488889+0000","last_peered":"2023-10-06T03:33:15.488889+0000","last_clean":"2023-10-06T03:33:15.488889+0000","last_became_active":"2023-10-06T03:33:15.485159+0000","last_became_peered":"2023-10-06T03:33:15.485159+0000","last_unstale":"2023-10-06T03:33:15.488889+0000","last_undegraded":"2023-10-06T03:33:15.488889+0000","last_fullsized":"2023-10-06T03:33:15.488889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:02.594885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500387+0000","last_change":"2023-10-06T03:33:15.498785+0000","last_active":"2023-10-06T03:33:15.500387+0000","last_peered":"2023-10-06T03:33:15.500387+0000","last_clean":"2023-10-06T03:33:15.500387+0000","last_became_active":"2023-10-06T03:33:15.498503+0000","last_became_peered":"2023-10-06T03:33:15.498503+0000","last_unstale":"2023-10-06T03:33:15.500387+0000","last_undegraded":"2023-10-06T03:33:15.500387+0000","last_fullsized":"2023-10-06T03:33:15.500387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:59.545999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494228+0000","last_change":"2023-10-06T03:33:15.484222+0000","last_active":"2023-10-06T03:33:15.494228+0000","last_peered":"2023-10-06T03:33:15.494228+0000","last_clean":"2023-10-06T03:33:15.494228+0000","last_became_active":"2023-10-06T03:33:15.483910+0000","last_became_peered":"2023-10-06T03:33:15.483910+0000","last_unstale":"2023-10-06T03:33:15.494228+0000","last_undegraded":"2023-10-06T03:33:15.494228+0000","last_fullsized":"2023-10-06T03:33:15.494228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:41.309883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501590+0000","last_change":"2023-10-06T03:33:15.497903+0000","last_active":"2023-10-06T03:33:15.501590+0000","last_peered":"2023-10-06T03:33:15.501590+0000","last_clean":"2023-10-06T03:33:15.501590+0000","last_became_active":"2023-10-06T03:33:15.496979+0000","last_became_peered":"2023-10-06T03:33:15.496979+0000","last_unstale":"2023-10-06T03:33:15.501590+0000","last_undegraded":"2023-10-06T03:33:15.501590+0000","last_fullsized":"2023-10-06T03:33:15.501590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502728+0000","last_change":"2023-10-06T03:33:15.501214+0000","last_active":"2023-10-06T03:33:15.502728+0000","last_peered":"2023-10-06T03:33:15.502728+0000","last_clean":"2023-10-06T03:33:15.502728+0000","last_became_active":"2023-10-06T03:33:15.500739+0000","last_became_peered":"2023-10-06T03:33:15.500739+0000","last_unstale":"2023-10-06T03:33:15.502728+0000","last_undegraded":"2023-10-06T03:33:15.502728+0000","last_fullsized":"2023-10-06T03:33:15.502728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.964350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.499625+0000","last_change":"2023-10-06T03:33:15.498383+0000","last_active":"2023-10-06T03:33:15.499625+0000","last_peered":"2023-10-06T03:33:15.499625+0000","last_clean":"2023-10-06T03:33:15.499625+0000","last_became_active":"2023-10-06T03:33:15.498162+0000","last_became_peered":"2023-10-06T03:33:15.498162+0000","last_unstale":"2023-10-06T03:33:15.499625+0000","last_undegraded":"2023-10-06T03:33:15.499625+0000","last_fullsized":"2023-10-06T03:33:15.499625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:02.889905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501799+0000","last_change":"2023-10-06T03:33:15.496957+0000","last_active":"2023-10-06T03:33:15.501799+0000","last_peered":"2023-10-06T03:33:15.501799+0000","last_clean":"2023-10-06T03:33:15.501799+0000","last_became_active":"2023-10-06T03:33:15.496673+0000","last_became_peered":"2023-10-06T03:33:15.496673+0000","last_unstale":"2023-10-06T03:33:15.501799+0000","last_undegraded":"2023-10-06T03:33:15.501799+0000","last_fullsized":"2023-10-06T03:33:15.501799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500848+0000","last_change":"2023-10-06T03:33:15.495118+0000","last_active":"2023-10-06T03:33:15.500848+0000","last_peered":"2023-10-06T03:33:15.500848+0000","last_clean":"2023-10-06T03:33:15.500848+0000","last_became_active":"2023-10-06T03:33:15.494732+0000","last_became_peered":"2023-10-06T03:33:15.494732+0000","last_unstale":"2023-10-06T03:33:15.500848+0000","last_undegraded":"2023-10-06T03:33:15.500848+0000","last_fullsized":"2023-10-06T03:33:15.500848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:02.889905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494010+0000","last_change":"2023-10-06T03:33:15.491772+0000","last_active":"2023-10-06T03:33:15.494010+0000","last_peered":"2023-10-06T03:33:15.494010+0000","last_clean":"2023-10-06T03:33:15.494010+0000","last_became_active":"2023-10-06T03:33:15.491530+0000","last_became_peered":"2023-10-06T03:33:15.491530+0000","last_unstale":"2023-10-06T03:33:15.494010+0000","last_undegraded":"2023-10-06T03:33:15.494010+0000","last_fullsized":"2023-10-06T03:33:15.494010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:57.097910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.489816+0000","last_change":"2023-10-06T03:33:15.485334+0000","last_active":"2023-10-06T03:33:15.489816+0000","last_peered":"2023-10-06T03:33:15.489816+0000","last_clean":"2023-10-06T03:33:15.489816+0000","last_became_active":"2023-10-06T03:33:15.485051+0000","last_became_peered":"2023-10-06T03:33:15.485051+0000","last_unstale":"2023-10-06T03:33:15.489816+0000","last_undegraded":"2023-10-06T03:33:15.489816+0000","last_fullsized":"2023-10-06T03:33:15.489816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:42.519384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498233+0000","last_change":"2023-10-06T03:33:15.495208+0000","last_active":"2023-10-06T03:33:15.498233+0000","last_peered":"2023-10-06T03:33:15.498233+0000","last_clean":"2023-10-06T03:33:15.498233+0000","last_became_active":"2023-10-06T03:33:15.494897+0000","last_became_peered":"2023-10-06T03:33:15.494897+0000","last_unstale":"2023-10-06T03:33:15.498233+0000","last_undegraded":"2023-10-06T03:33:15.498233+0000","last_fullsized":"2023-10-06T03:33:15.498233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:12.635649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501608+0000","last_change":"2023-10-06T03:33:15.498114+0000","last_active":"2023-10-06T03:33:15.501608+0000","last_peered":"2023-10-06T03:33:15.501608+0000","last_clean":"2023-10-06T03:33:15.501608+0000","last_became_active":"2023-10-06T03:33:15.497401+0000","last_became_peered":"2023-10-06T03:33:15.497401+0000","last_unstale":"2023-10-06T03:33:15.501608+0000","last_undegraded":"2023-10-06T03:33:15.501608+0000","last_fullsized":"2023-10-06T03:33:15.501608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494198+0000","last_change":"2023-10-06T03:33:15.484297+0000","last_active":"2023-10-06T03:33:15.494198+0000","last_peered":"2023-10-06T03:33:15.494198+0000","last_clean":"2023-10-06T03:33:15.494198+0000","last_became_active":"2023-10-06T03:33:15.483921+0000","last_became_peered":"2023-10-06T03:33:15.483921+0000","last_unstale":"2023-10-06T03:33:15.494198+0000","last_undegraded":"2023-10-06T03:33:15.494198+0000","last_fullsized":"2023-10-06T03:33:15.494198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:03.085548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.495074+0000","last_change":"2023-10-06T03:33:15.492170+0000","last_active":"2023-10-06T03:33:15.495074+0000","last_peered":"2023-10-06T03:33:15.495074+0000","last_clean":"2023-10-06T03:33:15.495074+0000","last_became_active":"2023-10-06T03:33:15.491847+0000","last_became_peered":"2023-10-06T03:33:15.491847+0000","last_unstale":"2023-10-06T03:33:15.495074+0000","last_undegraded":"2023-10-06T03:33:15.495074+0000","last_fullsized":"2023-10-06T03:33:15.495074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:30.660724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496120+0000","last_change":"2023-10-06T03:33:15.492652+0000","last_active":"2023-10-06T03:33:15.496120+0000","last_peered":"2023-10-06T03:33:15.496120+0000","last_clean":"2023-10-06T03:33:15.496120+0000","last_became_active":"2023-10-06T03:33:15.491815+0000","last_became_peered":"2023-10-06T03:33:15.491815+0000","last_unstale":"2023-10-06T03:33:15.496120+0000","last_undegraded":"2023-10-06T03:33:15.496120+0000","last_fullsized":"2023-10-06T03:33:15.496120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:20.661516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.489846+0000","last_change":"2023-10-06T03:33:15.486081+0000","last_active":"2023-10-06T03:33:15.489846+0000","last_peered":"2023-10-06T03:33:15.489846+0000","last_clean":"2023-10-06T03:33:15.489846+0000","last_became_active":"2023-10-06T03:33:15.485842+0000","last_became_peered":"2023-10-06T03:33:15.485842+0000","last_unstale":"2023-10-06T03:33:15.489846+0000","last_undegraded":"2023-10-06T03:33:15.489846+0000","last_fullsized":"2023-10-06T03:33:15.489846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:16.260090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496154+0000","last_change":"2023-10-06T03:33:15.490482+0000","last_active":"2023-10-06T03:33:15.496154+0000","last_peered":"2023-10-06T03:33:15.496154+0000","last_clean":"2023-10-06T03:33:15.496154+0000","last_became_active":"2023-10-06T03:33:15.490185+0000","last_became_peered":"2023-10-06T03:33:15.490185+0000","last_unstale":"2023-10-06T03:33:15.496154+0000","last_undegraded":"2023-10-06T03:33:15.496154+0000","last_fullsized":"2023-10-06T03:33:15.496154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:05.828284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502137+0000","last_change":"2023-10-06T03:33:15.499609+0000","last_active":"2023-10-06T03:33:15.502137+0000","last_peered":"2023-10-06T03:33:15.502137+0000","last_clean":"2023-10-06T03:33:15.502137+0000","last_became_active":"2023-10-06T03:33:15.499317+0000","last_became_peered":"2023-10-06T03:33:15.499317+0000","last_unstale":"2023-10-06T03:33:15.502137+0000","last_undegraded":"2023-10-06T03:33:15.502137+0000","last_fullsized":"2023-10-06T03:33:15.502137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494425+0000","last_change":"2023-10-06T03:33:15.491198+0000","last_active":"2023-10-06T03:33:15.494425+0000","last_peered":"2023-10-06T03:33:15.494425+0000","last_clean":"2023-10-06T03:33:15.494425+0000","last_became_active":"2023-10-06T03:33:15.490933+0000","last_became_peered":"2023-10-06T03:33:15.490933+0000","last_unstale":"2023-10-06T03:33:15.494425+0000","last_undegraded":"2023-10-06T03:33:15.494425+0000","last_fullsized":"2023-10-06T03:33:15.494425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500604+0000","last_change":"2023-10-06T03:33:15.492259+0000","last_active":"2023-10-06T03:33:15.500604+0000","last_peered":"2023-10-06T03:33:15.500604+0000","last_clean":"2023-10-06T03:33:15.500604+0000","last_became_active":"2023-10-06T03:33:15.492018+0000","last_became_peered":"2023-10-06T03:33:15.492018+0000","last_unstale":"2023-10-06T03:33:15.500604+0000","last_undegraded":"2023-10-06T03:33:15.500604+0000","last_fullsized":"2023-10-06T03:33:15.500604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:31:34.027062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501954+0000","last_change":"2023-10-06T03:33:15.500133+0000","last_active":"2023-10-06T03:33:15.501954+0000","last_peered":"2023-10-06T03:33:15.501954+0000","last_clean":"2023-10-06T03:33:15.501954+0000","last_became_active":"2023-10-06T03:33:15.499092+0000","last_became_peered":"2023-10-06T03:33:15.499092+0000","last_unstale":"2023-10-06T03:33:15.501954+0000","last_undegraded":"2023-10-06T03:33:15.501954+0000","last_fullsized":"2023-10-06T03:33:15.501954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493234+0000","last_change":"2023-10-06T03:33:15.487133+0000","last_active":"2023-10-06T03:33:15.493234+0000","last_peered":"2023-10-06T03:33:15.493234+0000","last_clean":"2023-10-06T03:33:15.493234+0000","last_became_active":"2023-10-06T03:33:15.486582+0000","last_became_peered":"2023-10-06T03:33:15.486582+0000","last_unstale":"2023-10-06T03:33:15.493234+0000","last_undegraded":"2023-10-06T03:33:15.493234+0000","last_fullsized":"2023-10-06T03:33:15.493234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:37.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498958+0000","last_change":"2023-10-06T03:33:15.489677+0000","last_active":"2023-10-06T03:33:15.498958+0000","last_peered":"2023-10-06T03:33:15.498958+0000","last_clean":"2023-10-06T03:33:15.498958+0000","last_became_active":"2023-10-06T03:33:15.489431+0000","last_became_peered":"2023-10-06T03:33:15.489431+0000","last_unstale":"2023-10-06T03:33:15.498958+0000","last_undegraded":"2023-10-06T03:33:15.498958+0000","last_fullsized":"2023-10-06T03:33:15.498958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502491+0000","last_change":"2023-10-06T03:33:15.498305+0000","last_active":"2023-10-06T03:33:15.502491+0000","last_peered":"2023-10-06T03:33:15.502491+0000","last_clean":"2023-10-06T03:33:15.502491+0000","last_became_active":"2023-10-06T03:33:15.497920+0000","last_became_peered":"2023-10-06T03:33:15.497920+0000","last_unstale":"2023-10-06T03:33:15.502491+0000","last_undegraded":"2023-10-06T03:33:15.502491+0000","last_fullsized":"2023-10-06T03:33:15.502491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.880457+0000","last_change":"2023-10-06T03:33:14.894963+0000","last_active":"2023-10-06T03:33:15.880457+0000","last_peered":"2023-10-06T03:33:15.880457+0000","last_clean":"2023-10-06T03:33:15.880457+0000","last_became_active":"2023-10-06T03:33:14.886952+0000","last_became_peered":"2023-10-06T03:33:14.886952+0000","last_unstale":"2023-10-06T03:33:15.880457+0000","last_undegraded":"2023-10-06T03:33:15.880457+0000","last_fullsized":"2023-10-06T03:33:15.880457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501009+0000","last_change":"2023-10-06T03:33:15.498264+0000","last_active":"2023-10-06T03:33:15.501009+0000","last_peered":"2023-10-06T03:33:15.501009+0000","last_clean":"2023-10-06T03:33:15.501009+0000","last_became_active":"2023-10-06T03:33:15.498030+0000","last_became_peered":"2023-10-06T03:33:15.498030+0000","last_unstale":"2023-10-06T03:33:15.501009+0000","last_undegraded":"2023-10-06T03:33:15.501009+0000","last_fullsized":"2023-10-06T03:33:15.501009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:05.828284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.473505+0000","last_change":"2023-10-06T03:33:14.895404+0000","last_active":"2023-10-06T03:33:15.473505+0000","last_peered":"2023-10-06T03:33:15.473505+0000","last_clean":"2023-10-06T03:33:15.473505+0000","last_became_active":"2023-10-06T03:33:14.894742+0000","last_became_peered":"2023-10-06T03:33:14.894742+0000","last_unstale":"2023-10-06T03:33:15.473505+0000","last_undegraded":"2023-10-06T03:33:15.473505+0000","last_fullsized":"2023-10-06T03:33:15.473505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490157+0000","last_change":"2023-10-06T03:33:15.485976+0000","last_active":"2023-10-06T03:33:15.490157+0000","last_peered":"2023-10-06T03:33:15.490157+0000","last_clean":"2023-10-06T03:33:15.490157+0000","last_became_active":"2023-10-06T03:33:15.485658+0000","last_became_peered":"2023-10-06T03:33:15.485658+0000","last_unstale":"2023-10-06T03:33:15.490157+0000","last_undegraded":"2023-10-06T03:33:15.490157+0000","last_fullsized":"2023-10-06T03:33:15.490157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482088+0000","last_change":"2023-10-06T03:33:14.903103+0000","last_active":"2023-10-06T03:33:15.482088+0000","last_peered":"2023-10-06T03:33:15.482088+0000","last_clean":"2023-10-06T03:33:15.482088+0000","last_became_active":"2023-10-06T03:33:14.902707+0000","last_became_peered":"2023-10-06T03:33:14.902707+0000","last_unstale":"2023-10-06T03:33:15.482088+0000","last_undegraded":"2023-10-06T03:33:15.482088+0000","last_fullsized":"2023-10-06T03:33:15.482088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501406+0000","last_change":"2023-10-06T03:33:15.496437+0000","last_active":"2023-10-06T03:33:15.501406+0000","last_peered":"2023-10-06T03:33:15.501406+0000","last_clean":"2023-10-06T03:33:15.501406+0000","last_became_active":"2023-10-06T03:33:15.496138+0000","last_became_peered":"2023-10-06T03:33:15.496138+0000","last_unstale":"2023-10-06T03:33:15.501406+0000","last_undegraded":"2023-10-06T03:33:15.501406+0000","last_fullsized":"2023-10-06T03:33:15.501406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:09:05.289396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.478575+0000","last_change":"2023-10-06T03:33:14.897019+0000","last_active":"2023-10-06T03:33:15.478575+0000","last_peered":"2023-10-06T03:33:15.478575+0000","last_clean":"2023-10-06T03:33:15.478575+0000","last_became_active":"2023-10-06T03:33:14.896740+0000","last_became_peered":"2023-10-06T03:33:14.896740+0000","last_unstale":"2023-10-06T03:33:15.478575+0000","last_undegraded":"2023-10-06T03:33:15.478575+0000","last_fullsized":"2023-10-06T03:33:15.478575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500886+0000","last_change":"2023-10-06T03:33:15.496977+0000","last_active":"2023-10-06T03:33:15.500886+0000","last_peered":"2023-10-06T03:33:15.500886+0000","last_clean":"2023-10-06T03:33:15.500886+0000","last_became_active":"2023-10-06T03:33:15.496726+0000","last_became_peered":"2023-10-06T03:33:15.496726+0000","last_unstale":"2023-10-06T03:33:15.500886+0000","last_undegraded":"2023-10-06T03:33:15.500886+0000","last_fullsized":"2023-10-06T03:33:15.500886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:02.889905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.491600+0000","last_change":"2023-10-06T03:33:14.896854+0000","last_active":"2023-10-06T03:33:15.491600+0000","last_peered":"2023-10-06T03:33:15.491600+0000","last_clean":"2023-10-06T03:33:15.491600+0000","last_became_active":"2023-10-06T03:33:14.896606+0000","last_became_peered":"2023-10-06T03:33:14.896606+0000","last_unstale":"2023-10-06T03:33:15.491600+0000","last_undegraded":"2023-10-06T03:33:15.491600+0000","last_fullsized":"2023-10-06T03:33:15.491600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500655+0000","last_change":"2023-10-06T03:33:15.494940+0000","last_active":"2023-10-06T03:33:15.500655+0000","last_peered":"2023-10-06T03:33:15.500655+0000","last_clean":"2023-10-06T03:33:15.500655+0000","last_became_active":"2023-10-06T03:33:15.494251+0000","last_became_peered":"2023-10-06T03:33:15.494251+0000","last_unstale":"2023-10-06T03:33:15.500655+0000","last_undegraded":"2023-10-06T03:33:15.500655+0000","last_fullsized":"2023-10-06T03:33:15.500655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.885266+0000","last_change":"2023-10-06T03:33:14.895082+0000","last_active":"2023-10-06T03:33:15.885266+0000","last_peered":"2023-10-06T03:33:15.885266+0000","last_clean":"2023-10-06T03:33:15.885266+0000","last_became_active":"2023-10-06T03:33:14.894804+0000","last_became_peered":"2023-10-06T03:33:14.894804+0000","last_unstale":"2023-10-06T03:33:15.885266+0000","last_undegraded":"2023-10-06T03:33:15.885266+0000","last_fullsized":"2023-10-06T03:33:15.885266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:01.585501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493308+0000","last_change":"2023-10-06T03:33:15.487049+0000","last_active":"2023-10-06T03:33:15.493308+0000","last_peered":"2023-10-06T03:33:15.493308+0000","last_clean":"2023-10-06T03:33:15.493308+0000","last_became_active":"2023-10-06T03:33:15.486777+0000","last_became_peered":"2023-10-06T03:33:15.486777+0000","last_unstale":"2023-10-06T03:33:15.493308+0000","last_undegraded":"2023-10-06T03:33:15.493308+0000","last_fullsized":"2023-10-06T03:33:15.493308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:12.635649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.888581+0000","last_change":"2023-10-06T03:33:14.898475+0000","last_active":"2023-10-06T03:33:15.888581+0000","last_peered":"2023-10-06T03:33:15.888581+0000","last_clean":"2023-10-06T03:33:15.888581+0000","last_became_active":"2023-10-06T03:33:14.898144+0000","last_became_peered":"2023-10-06T03:33:14.898144+0000","last_unstale":"2023-10-06T03:33:15.888581+0000","last_undegraded":"2023-10-06T03:33:15.888581+0000","last_fullsized":"2023-10-06T03:33:15.888581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494412+0000","last_change":"2023-10-06T03:33:15.487549+0000","last_active":"2023-10-06T03:33:15.494412+0000","last_peered":"2023-10-06T03:33:15.494412+0000","last_clean":"2023-10-06T03:33:15.494412+0000","last_became_active":"2023-10-06T03:33:15.487277+0000","last_became_peered":"2023-10-06T03:33:15.487277+0000","last_unstale":"2023-10-06T03:33:15.494412+0000","last_undegraded":"2023-10-06T03:33:15.494412+0000","last_fullsized":"2023-10-06T03:33:15.494412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:07.613983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.885740+0000","last_change":"2023-10-06T03:33:14.896271+0000","last_active":"2023-10-06T03:33:15.885740+0000","last_peered":"2023-10-06T03:33:15.885740+0000","last_clean":"2023-10-06T03:33:15.885740+0000","last_became_active":"2023-10-06T03:33:14.896004+0000","last_became_peered":"2023-10-06T03:33:14.896004+0000","last_unstale":"2023-10-06T03:33:15.885740+0000","last_undegraded":"2023-10-06T03:33:15.885740+0000","last_fullsized":"2023-10-06T03:33:15.885740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:15.567270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500942+0000","last_change":"2023-10-06T03:33:15.497569+0000","last_active":"2023-10-06T03:33:15.500942+0000","last_peered":"2023-10-06T03:33:15.500942+0000","last_clean":"2023-10-06T03:33:15.500942+0000","last_became_active":"2023-10-06T03:33:15.497287+0000","last_became_peered":"2023-10-06T03:33:15.497287+0000","last_unstale":"2023-10-06T03:33:15.500942+0000","last_undegraded":"2023-10-06T03:33:15.500942+0000","last_fullsized":"2023-10-06T03:33:15.500942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:47.107849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.476556+0000","last_change":"2023-10-06T03:33:14.897199+0000","last_active":"2023-10-06T03:33:15.476556+0000","last_peered":"2023-10-06T03:33:15.476556+0000","last_clean":"2023-10-06T03:33:15.476556+0000","last_became_active":"2023-10-06T03:33:14.896925+0000","last_became_peered":"2023-10-06T03:33:14.896925+0000","last_unstale":"2023-10-06T03:33:15.476556+0000","last_undegraded":"2023-10-06T03:33:15.476556+0000","last_fullsized":"2023-10-06T03:33:15.476556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:41.510001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498210+0000","last_change":"2023-10-06T03:33:15.495381+0000","last_active":"2023-10-06T03:33:15.498210+0000","last_peered":"2023-10-06T03:33:15.498210+0000","last_clean":"2023-10-06T03:33:15.498210+0000","last_became_active":"2023-10-06T03:33:15.494972+0000","last_became_peered":"2023-10-06T03:33:15.494972+0000","last_unstale":"2023-10-06T03:33:15.498210+0000","last_undegraded":"2023-10-06T03:33:15.498210+0000","last_fullsized":"2023-10-06T03:33:15.498210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.483344+0000","last_change":"2023-10-06T03:33:14.901135+0000","last_active":"2023-10-06T03:33:15.483344+0000","last_peered":"2023-10-06T03:33:15.483344+0000","last_clean":"2023-10-06T03:33:15.483344+0000","last_became_active":"2023-10-06T03:33:14.900871+0000","last_became_peered":"2023-10-06T03:33:14.900871+0000","last_unstale":"2023-10-06T03:33:15.483344+0000","last_undegraded":"2023-10-06T03:33:15.483344+0000","last_fullsized":"2023-10-06T03:33:15.483344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494496+0000","last_change":"2023-10-06T03:33:15.491822+0000","last_active":"2023-10-06T03:33:15.494496+0000","last_peered":"2023-10-06T03:33:15.494496+0000","last_clean":"2023-10-06T03:33:15.494496+0000","last_became_active":"2023-10-06T03:33:15.491560+0000","last_became_peered":"2023-10-06T03:33:15.491560+0000","last_unstale":"2023-10-06T03:33:15.494496+0000","last_undegraded":"2023-10-06T03:33:15.494496+0000","last_fullsized":"2023-10-06T03:33:15.494496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.964350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886225+0000","last_change":"2023-10-06T03:33:14.898845+0000","last_active":"2023-10-06T03:33:15.886225+0000","last_peered":"2023-10-06T03:33:15.886225+0000","last_clean":"2023-10-06T03:33:15.886225+0000","last_became_active":"2023-10-06T03:33:14.898491+0000","last_became_peered":"2023-10-06T03:33:14.898491+0000","last_unstale":"2023-10-06T03:33:15.886225+0000","last_undegraded":"2023-10-06T03:33:15.886225+0000","last_fullsized":"2023-10-06T03:33:15.886225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500404+0000","last_change":"2023-10-06T03:33:15.487774+0000","last_active":"2023-10-06T03:33:15.500404+0000","last_peered":"2023-10-06T03:33:15.500404+0000","last_clean":"2023-10-06T03:33:15.500404+0000","last_became_active":"2023-10-06T03:33:15.487515+0000","last_became_peered":"2023-10-06T03:33:15.487515+0000","last_unstale":"2023-10-06T03:33:15.500404+0000","last_undegraded":"2023-10-06T03:33:15.500404+0000","last_fullsized":"2023-10-06T03:33:15.500404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:30.660724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.884947+0000","last_change":"2023-10-06T03:33:14.896171+0000","last_active":"2023-10-06T03:33:15.884947+0000","last_peered":"2023-10-06T03:33:15.884947+0000","last_clean":"2023-10-06T03:33:15.884947+0000","last_became_active":"2023-10-06T03:33:14.895903+0000","last_became_peered":"2023-10-06T03:33:14.895903+0000","last_unstale":"2023-10-06T03:33:15.884947+0000","last_undegraded":"2023-10-06T03:33:15.884947+0000","last_fullsized":"2023-10-06T03:33:15.884947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:44.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500464+0000","last_change":"2023-10-06T03:33:15.496285+0000","last_active":"2023-10-06T03:33:15.500464+0000","last_peered":"2023-10-06T03:33:15.500464+0000","last_clean":"2023-10-06T03:33:15.500464+0000","last_became_active":"2023-10-06T03:33:15.495868+0000","last_became_peered":"2023-10-06T03:33:15.495868+0000","last_unstale":"2023-10-06T03:33:15.500464+0000","last_undegraded":"2023-10-06T03:33:15.500464+0000","last_fullsized":"2023-10-06T03:33:15.500464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.885984+0000","last_change":"2023-10-06T03:33:14.896322+0000","last_active":"2023-10-06T03:33:15.885984+0000","last_peered":"2023-10-06T03:33:15.885984+0000","last_clean":"2023-10-06T03:33:15.885984+0000","last_became_active":"2023-10-06T03:33:14.896022+0000","last_became_peered":"2023-10-06T03:33:14.896022+0000","last_unstale":"2023-10-06T03:33:15.885984+0000","last_undegraded":"2023-10-06T03:33:15.885984+0000","last_fullsized":"2023-10-06T03:33:15.885984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:21.680218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500853+0000","last_change":"2023-10-06T03:33:15.494878+0000","last_active":"2023-10-06T03:33:15.500853+0000","last_peered":"2023-10-06T03:33:15.500853+0000","last_clean":"2023-10-06T03:33:15.500853+0000","last_became_active":"2023-10-06T03:33:15.494597+0000","last_became_peered":"2023-10-06T03:33:15.494597+0000","last_unstale":"2023-10-06T03:33:15.500853+0000","last_undegraded":"2023-10-06T03:33:15.500853+0000","last_fullsized":"2023-10-06T03:33:15.500853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:30.660724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.887336+0000","last_change":"2023-10-06T03:33:14.897659+0000","last_active":"2023-10-06T03:33:15.887336+0000","last_peered":"2023-10-06T03:33:15.887336+0000","last_clean":"2023-10-06T03:33:15.887336+0000","last_became_active":"2023-10-06T03:33:14.897368+0000","last_became_peered":"2023-10-06T03:33:14.897368+0000","last_unstale":"2023-10-06T03:33:15.887336+0000","last_undegraded":"2023-10-06T03:33:15.887336+0000","last_fullsized":"2023-10-06T03:33:15.887336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:46:12.361089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493933+0000","last_change":"2023-10-06T03:33:15.492136+0000","last_active":"2023-10-06T03:33:15.493933+0000","last_peered":"2023-10-06T03:33:15.493933+0000","last_clean":"2023-10-06T03:33:15.493933+0000","last_became_active":"2023-10-06T03:33:15.491858+0000","last_became_peered":"2023-10-06T03:33:15.491858+0000","last_unstale":"2023-10-06T03:33:15.493933+0000","last_undegraded":"2023-10-06T03:33:15.493933+0000","last_fullsized":"2023-10-06T03:33:15.493933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:16.576653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490953+0000","last_change":"2023-10-06T03:33:14.890596+0000","last_active":"2023-10-06T03:33:15.490953+0000","last_peered":"2023-10-06T03:33:15.490953+0000","last_clean":"2023-10-06T03:33:15.490953+0000","last_became_active":"2023-10-06T03:33:14.889486+0000","last_became_peered":"2023-10-06T03:33:14.889486+0000","last_unstale":"2023-10-06T03:33:15.490953+0000","last_undegraded":"2023-10-06T03:33:15.490953+0000","last_fullsized":"2023-10-06T03:33:15.490953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:41.510001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.495179+0000","last_change":"2023-10-06T03:33:15.493860+0000","last_active":"2023-10-06T03:33:15.495179+0000","last_peered":"2023-10-06T03:33:15.495179+0000","last_clean":"2023-10-06T03:33:15.495179+0000","last_became_active":"2023-10-06T03:33:15.493610+0000","last_became_peered":"2023-10-06T03:33:15.493610+0000","last_unstale":"2023-10-06T03:33:15.495179+0000","last_undegraded":"2023-10-06T03:33:15.495179+0000","last_fullsized":"2023-10-06T03:33:15.495179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.881042+0000","last_change":"2023-10-06T03:33:14.903094+0000","last_active":"2023-10-06T03:33:15.881042+0000","last_peered":"2023-10-06T03:33:15.881042+0000","last_clean":"2023-10-06T03:33:15.881042+0000","last_became_active":"2023-10-06T03:33:14.902686+0000","last_became_peered":"2023-10-06T03:33:14.902686+0000","last_unstale":"2023-10-06T03:33:15.881042+0000","last_undegraded":"2023-10-06T03:33:15.881042+0000","last_fullsized":"2023-10-06T03:33:15.881042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496134+0000","last_change":"2023-10-06T03:33:15.489915+0000","last_active":"2023-10-06T03:33:15.496134+0000","last_peered":"2023-10-06T03:33:15.496134+0000","last_clean":"2023-10-06T03:33:15.496134+0000","last_became_active":"2023-10-06T03:33:15.488544+0000","last_became_peered":"2023-10-06T03:33:15.488544+0000","last_unstale":"2023-10-06T03:33:15.496134+0000","last_undegraded":"2023-10-06T03:33:15.496134+0000","last_fullsized":"2023-10-06T03:33:15.496134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.478249+0000","last_change":"2023-10-06T03:33:14.886562+0000","last_active":"2023-10-06T03:33:15.478249+0000","last_peered":"2023-10-06T03:33:15.478249+0000","last_clean":"2023-10-06T03:33:15.478249+0000","last_became_active":"2023-10-06T03:33:14.886157+0000","last_became_peered":"2023-10-06T03:33:14.886157+0000","last_unstale":"2023-10-06T03:33:15.478249+0000","last_undegraded":"2023-10-06T03:33:15.478249+0000","last_fullsized":"2023-10-06T03:33:15.478249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496415+0000","last_change":"2023-10-06T03:33:15.491844+0000","last_active":"2023-10-06T03:33:15.496415+0000","last_peered":"2023-10-06T03:33:15.496415+0000","last_clean":"2023-10-06T03:33:15.496415+0000","last_became_active":"2023-10-06T03:33:15.491574+0000","last_became_peered":"2023-10-06T03:33:15.491574+0000","last_unstale":"2023-10-06T03:33:15.496415+0000","last_undegraded":"2023-10-06T03:33:15.496415+0000","last_fullsized":"2023-10-06T03:33:15.496415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:35.426668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482572+0000","last_change":"2023-10-06T03:33:14.895658+0000","last_active":"2023-10-06T03:33:15.482572+0000","last_peered":"2023-10-06T03:33:15.482572+0000","last_clean":"2023-10-06T03:33:15.482572+0000","last_became_active":"2023-10-06T03:33:14.895332+0000","last_became_peered":"2023-10-06T03:33:14.895332+0000","last_unstale":"2023-10-06T03:33:15.482572+0000","last_undegraded":"2023-10-06T03:33:15.482572+0000","last_fullsized":"2023-10-06T03:33:15.482572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:28.684332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496089+0000","last_change":"2023-10-06T03:33:15.490478+0000","last_active":"2023-10-06T03:33:15.496089+0000","last_peered":"2023-10-06T03:33:15.496089+0000","last_clean":"2023-10-06T03:33:15.496089+0000","last_became_active":"2023-10-06T03:33:15.485974+0000","last_became_peered":"2023-10-06T03:33:15.485974+0000","last_unstale":"2023-10-06T03:33:15.496089+0000","last_undegraded":"2023-10-06T03:33:15.496089+0000","last_fullsized":"2023-10-06T03:33:15.496089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.481946+0000","last_change":"2023-10-06T03:33:14.891288+0000","last_active":"2023-10-06T03:33:15.481946+0000","last_peered":"2023-10-06T03:33:15.481946+0000","last_clean":"2023-10-06T03:33:15.481946+0000","last_became_active":"2023-10-06T03:33:14.890186+0000","last_became_peered":"2023-10-06T03:33:14.890186+0000","last_unstale":"2023-10-06T03:33:15.481946+0000","last_undegraded":"2023-10-06T03:33:15.481946+0000","last_fullsized":"2023-10-06T03:33:15.481946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:21.680218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493182+0000","last_change":"2023-10-06T03:33:15.486956+0000","last_active":"2023-10-06T03:33:15.493182+0000","last_peered":"2023-10-06T03:33:15.493182+0000","last_clean":"2023-10-06T03:33:15.493182+0000","last_became_active":"2023-10-06T03:33:15.486188+0000","last_became_peered":"2023-10-06T03:33:15.486188+0000","last_unstale":"2023-10-06T03:33:15.493182+0000","last_undegraded":"2023-10-06T03:33:15.493182+0000","last_fullsized":"2023-10-06T03:33:15.493182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.887149+0000","last_change":"2023-10-06T03:33:14.895426+0000","last_active":"2023-10-06T03:33:15.887149+0000","last_peered":"2023-10-06T03:33:15.887149+0000","last_clean":"2023-10-06T03:33:15.887149+0000","last_became_active":"2023-10-06T03:33:14.893425+0000","last_became_peered":"2023-10-06T03:33:14.893425+0000","last_unstale":"2023-10-06T03:33:15.887149+0000","last_undegraded":"2023-10-06T03:33:15.887149+0000","last_fullsized":"2023-10-06T03:33:15.887149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:44.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.484841+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:33:15.484841+0000","last_peered":"2023-10-06T03:33:15.484841+0000","last_clean":"2023-10-06T03:33:15.484841+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:33:15.484841+0000","last_undegraded":"2023-10-06T03:33:15.484841+0000","last_fullsized":"2023-10-06T03:33:15.484841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500598+0000","last_change":"2023-10-06T03:33:15.492324+0000","last_active":"2023-10-06T03:33:15.500598+0000","last_peered":"2023-10-06T03:33:15.500598+0000","last_clean":"2023-10-06T03:33:15.500598+0000","last_became_active":"2023-10-06T03:33:15.492037+0000","last_became_peered":"2023-10-06T03:33:15.492037+0000","last_unstale":"2023-10-06T03:33:15.500598+0000","last_undegraded":"2023-10-06T03:33:15.500598+0000","last_fullsized":"2023-10-06T03:33:15.500598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.484831+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:33:15.484831+0000","last_peered":"2023-10-06T03:33:15.484831+0000","last_clean":"2023-10-06T03:33:15.484831+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:33:15.484831+0000","last_undegraded":"2023-10-06T03:33:15.484831+0000","last_fullsized":"2023-10-06T03:33:15.484831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.477783+0000","last_change":"2023-10-06T03:33:14.896368+0000","last_active":"2023-10-06T03:33:15.477783+0000","last_peered":"2023-10-06T03:33:15.477783+0000","last_clean":"2023-10-06T03:33:15.477783+0000","last_became_active":"2023-10-06T03:33:14.892459+0000","last_became_peered":"2023-10-06T03:33:14.892459+0000","last_unstale":"2023-10-06T03:33:15.477783+0000","last_undegraded":"2023-10-06T03:33:15.477783+0000","last_fullsized":"2023-10-06T03:33:15.477783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501686+0000","last_change":"2023-10-06T03:33:15.499324+0000","last_active":"2023-10-06T03:33:15.501686+0000","last_peered":"2023-10-06T03:33:15.501686+0000","last_clean":"2023-10-06T03:33:15.501686+0000","last_became_active":"2023-10-06T03:33:15.499039+0000","last_became_peered":"2023-10-06T03:33:15.499039+0000","last_unstale":"2023-10-06T03:33:15.501686+0000","last_undegraded":"2023-10-06T03:33:15.501686+0000","last_fullsized":"2023-10-06T03:33:15.501686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:43.269972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.473587+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:33:15.473587+0000","last_peered":"2023-10-06T03:33:15.473587+0000","last_clean":"2023-10-06T03:33:15.473587+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:33:15.473587+0000","last_undegraded":"2023-10-06T03:33:15.473587+0000","last_fullsized":"2023-10-06T03:33:15.473587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.483628+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:33:15.483628+0000","last_peered":"2023-10-06T03:33:15.483628+0000","last_clean":"2023-10-06T03:33:15.483628+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:33:15.483628+0000","last_undegraded":"2023-10-06T03:33:15.483628+0000","last_fullsized":"2023-10-06T03:33:15.483628+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.476509+0000","last_change":"2023-10-06T03:33:14.891206+0000","last_active":"2023-10-06T03:33:15.476509+0000","last_peered":"2023-10-06T03:33:15.476509+0000","last_clean":"2023-10-06T03:33:15.476509+0000","last_became_active":"2023-10-06T03:33:14.890823+0000","last_became_peered":"2023-10-06T03:33:14.890823+0000","last_unstale":"2023-10-06T03:33:15.476509+0000","last_undegraded":"2023-10-06T03:33:15.476509+0000","last_fullsized":"2023-10-06T03:33:15.476509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:28.684332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490137+0000","last_change":"2023-10-06T03:33:15.483582+0000","last_active":"2023-10-06T03:33:15.490137+0000","last_peered":"2023-10-06T03:33:15.490137+0000","last_clean":"2023-10-06T03:33:15.490137+0000","last_became_active":"2023-10-06T03:33:15.483324+0000","last_became_peered":"2023-10-06T03:33:15.483324+0000","last_unstale":"2023-10-06T03:33:15.490137+0000","last_undegraded":"2023-10-06T03:33:15.490137+0000","last_fullsized":"2023-10-06T03:33:15.490137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:12.635649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482905+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:33:15.482905+0000","last_peered":"2023-10-06T03:33:15.482905+0000","last_clean":"2023-10-06T03:33:15.482905+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:33:15.482905+0000","last_undegraded":"2023-10-06T03:33:15.482905+0000","last_fullsized":"2023-10-06T03:33:15.482905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.474679+0000","last_change":"2023-10-06T03:33:14.902096+0000","last_active":"2023-10-06T03:33:15.474679+0000","last_peered":"2023-10-06T03:33:15.474679+0000","last_clean":"2023-10-06T03:33:15.474679+0000","last_became_active":"2023-10-06T03:33:14.896599+0000","last_became_peered":"2023-10-06T03:33:14.896599+0000","last_unstale":"2023-10-06T03:33:15.474679+0000","last_undegraded":"2023-10-06T03:33:15.474679+0000","last_fullsized":"2023-10-06T03:33:15.474679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502084+0000","last_change":"2023-10-06T03:33:15.500022+0000","last_active":"2023-10-06T03:33:15.502084+0000","last_peered":"2023-10-06T03:33:15.502084+0000","last_clean":"2023-10-06T03:33:15.502084+0000","last_became_active":"2023-10-06T03:33:15.499610+0000","last_became_peered":"2023-10-06T03:33:15.499610+0000","last_unstale":"2023-10-06T03:33:15.502084+0000","last_undegraded":"2023-10-06T03:33:15.502084+0000","last_fullsized":"2023-10-06T03:33:15.502084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:58.092449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.483163+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:33:15.483163+0000","last_peered":"2023-10-06T03:33:15.483163+0000","last_clean":"2023-10-06T03:33:15.483163+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:33:15.483163+0000","last_undegraded":"2023-10-06T03:33:15.483163+0000","last_fullsized":"2023-10-06T03:33:15.483163+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482546+0000","last_change":"2023-10-06T03:33:14.895562+0000","last_active":"2023-10-06T03:33:15.482546+0000","last_peered":"2023-10-06T03:33:15.482546+0000","last_clean":"2023-10-06T03:33:15.482546+0000","last_became_active":"2023-10-06T03:33:14.893552+0000","last_became_peered":"2023-10-06T03:33:14.893552+0000","last_unstale":"2023-10-06T03:33:15.482546+0000","last_undegraded":"2023-10-06T03:33:15.482546+0000","last_fullsized":"2023-10-06T03:33:15.482546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:41.510001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498193+0000","last_change":"2023-10-06T03:33:15.495951+0000","last_active":"2023-10-06T03:33:15.498193+0000","last_peered":"2023-10-06T03:33:15.498193+0000","last_clean":"2023-10-06T03:33:15.498193+0000","last_became_active":"2023-10-06T03:33:15.495675+0000","last_became_peered":"2023-10-06T03:33:15.495675+0000","last_unstale":"2023-10-06T03:33:15.498193+0000","last_undegraded":"2023-10-06T03:33:15.498193+0000","last_fullsized":"2023-10-06T03:33:15.498193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:46:13.370472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490686+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:33:15.490686+0000","last_peered":"2023-10-06T03:33:15.490686+0000","last_clean":"2023-10-06T03:33:15.490686+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:33:15.490686+0000","last_undegraded":"2023-10-06T03:33:15.490686+0000","last_fullsized":"2023-10-06T03:33:15.490686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.480469+0000","last_change":"2023-10-06T03:33:14.901103+0000","last_active":"2023-10-06T03:33:15.480469+0000","last_peered":"2023-10-06T03:33:15.480469+0000","last_clean":"2023-10-06T03:33:15.480469+0000","last_became_active":"2023-10-06T03:33:14.900784+0000","last_became_peered":"2023-10-06T03:33:14.900784+0000","last_unstale":"2023-10-06T03:33:15.480469+0000","last_undegraded":"2023-10-06T03:33:15.480469+0000","last_fullsized":"2023-10-06T03:33:15.480469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493280+0000","last_change":"2023-10-06T03:33:15.488395+0000","last_active":"2023-10-06T03:33:15.493280+0000","last_peered":"2023-10-06T03:33:15.493280+0000","last_clean":"2023-10-06T03:33:15.493280+0000","last_became_active":"2023-10-06T03:33:15.483269+0000","last_became_peered":"2023-10-06T03:33:15.483269+0000","last_unstale":"2023-10-06T03:33:15.493280+0000","last_undegraded":"2023-10-06T03:33:15.493280+0000","last_fullsized":"2023-10-06T03:33:15.493280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.477353+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:33:15.477353+0000","last_peered":"2023-10-06T03:33:15.477353+0000","last_clean":"2023-10-06T03:33:15.477353+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:33:15.477353+0000","last_undegraded":"2023-10-06T03:33:15.477353+0000","last_fullsized":"2023-10-06T03:33:15.477353+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.474703+0000","last_change":"2023-10-06T03:33:14.896667+0000","last_active":"2023-10-06T03:33:15.474703+0000","last_peered":"2023-10-06T03:33:15.474703+0000","last_clean":"2023-10-06T03:33:15.474703+0000","last_became_active":"2023-10-06T03:33:14.895890+0000","last_became_peered":"2023-10-06T03:33:14.895890+0000","last_unstale":"2023-10-06T03:33:15.474703+0000","last_undegraded":"2023-10-06T03:33:15.474703+0000","last_fullsized":"2023-10-06T03:33:15.474703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:28.684332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498919+0000","last_change":"2023-10-06T03:33:15.496955+0000","last_active":"2023-10-06T03:33:15.498919+0000","last_peered":"2023-10-06T03:33:15.498919+0000","last_clean":"2023-10-06T03:33:15.498919+0000","last_became_active":"2023-10-06T03:33:15.496666+0000","last_became_peered":"2023-10-06T03:33:15.496666+0000","last_unstale":"2023-10-06T03:33:15.498919+0000","last_undegraded":"2023-10-06T03:33:15.498919+0000","last_fullsized":"2023-10-06T03:33:15.498919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.478419+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:33:15.478419+0000","last_peered":"2023-10-06T03:33:15.478419+0000","last_clean":"2023-10-06T03:33:15.478419+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:33:15.478419+0000","last_undegraded":"2023-10-06T03:33:15.478419+0000","last_fullsized":"2023-10-06T03:33:15.478419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.491835+0000","last_change":"2023-10-06T03:33:14.893120+0000","last_active":"2023-10-06T03:33:15.491835+0000","last_peered":"2023-10-06T03:33:15.491835+0000","last_clean":"2023-10-06T03:33:15.491835+0000","last_became_active":"2023-10-06T03:33:14.892839+0000","last_became_peered":"2023-10-06T03:33:14.892839+0000","last_unstale":"2023-10-06T03:33:15.491835+0000","last_undegraded":"2023-10-06T03:33:15.491835+0000","last_fullsized":"2023-10-06T03:33:15.491835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:56.088527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501655+0000","last_change":"2023-10-06T03:33:15.496296+0000","last_active":"2023-10-06T03:33:15.501655+0000","last_peered":"2023-10-06T03:33:15.501655+0000","last_clean":"2023-10-06T03:33:15.501655+0000","last_became_active":"2023-10-06T03:33:15.496028+0000","last_became_peered":"2023-10-06T03:33:15.496028+0000","last_unstale":"2023-10-06T03:33:15.501655+0000","last_undegraded":"2023-10-06T03:33:15.501655+0000","last_fullsized":"2023-10-06T03:33:15.501655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.884176+0000","last_change":"2023-10-06T03:33:14.897368+0000","last_active":"2023-10-06T03:33:15.884176+0000","last_peered":"2023-10-06T03:33:15.884176+0000","last_clean":"2023-10-06T03:33:15.884176+0000","last_became_active":"2023-10-06T03:33:14.892728+0000","last_became_peered":"2023-10-06T03:33:14.892728+0000","last_unstale":"2023-10-06T03:33:15.884176+0000","last_undegraded":"2023-10-06T03:33:15.884176+0000","last_fullsized":"2023-10-06T03:33:15.884176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494671+0000","last_change":"2023-10-06T03:33:15.493442+0000","last_active":"2023-10-06T03:33:15.494671+0000","last_peered":"2023-10-06T03:33:15.494671+0000","last_clean":"2023-10-06T03:33:15.494671+0000","last_became_active":"2023-10-06T03:33:15.493169+0000","last_became_peered":"2023-10-06T03:33:15.493169+0000","last_unstale":"2023-10-06T03:33:15.494671+0000","last_undegraded":"2023-10-06T03:33:15.494671+0000","last_fullsized":"2023-10-06T03:33:15.494671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:12.900415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886080+0000","last_change":"2023-10-06T03:33:14.899177+0000","last_active":"2023-10-06T03:33:15.886080+0000","last_peered":"2023-10-06T03:33:15.886080+0000","last_clean":"2023-10-06T03:33:15.886080+0000","last_became_active":"2023-10-06T03:33:14.898884+0000","last_became_peered":"2023-10-06T03:33:14.898884+0000","last_unstale":"2023-10-06T03:33:15.886080+0000","last_undegraded":"2023-10-06T03:33:15.886080+0000","last_fullsized":"2023-10-06T03:33:15.886080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501741+0000","last_change":"2023-10-06T03:33:15.499965+0000","last_active":"2023-10-06T03:33:15.501741+0000","last_peered":"2023-10-06T03:33:15.501741+0000","last_clean":"2023-10-06T03:33:15.501741+0000","last_became_active":"2023-10-06T03:33:15.499644+0000","last_became_peered":"2023-10-06T03:33:15.499644+0000","last_unstale":"2023-10-06T03:33:15.501741+0000","last_undegraded":"2023-10-06T03:33:15.501741+0000","last_fullsized":"2023-10-06T03:33:15.501741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:47.107849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886042+0000","last_change":"2023-10-06T03:33:14.898472+0000","last_active":"2023-10-06T03:33:15.886042+0000","last_peered":"2023-10-06T03:33:15.886042+0000","last_clean":"2023-10-06T03:33:15.886042+0000","last_became_active":"2023-10-06T03:33:14.898230+0000","last_became_peered":"2023-10-06T03:33:14.898230+0000","last_unstale":"2023-10-06T03:33:15.886042+0000","last_undegraded":"2023-10-06T03:33:15.886042+0000","last_fullsized":"2023-10-06T03:33:15.886042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494448+0000","last_change":"2023-10-06T03:33:15.488792+0000","last_active":"2023-10-06T03:33:15.494448+0000","last_peered":"2023-10-06T03:33:15.494448+0000","last_clean":"2023-10-06T03:33:15.494448+0000","last_became_active":"2023-10-06T03:33:15.488504+0000","last_became_peered":"2023-10-06T03:33:15.488504+0000","last_unstale":"2023-10-06T03:33:15.494448+0000","last_undegraded":"2023-10-06T03:33:15.494448+0000","last_fullsized":"2023-10-06T03:33:15.494448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:59.545999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.884688+0000","last_change":"2023-10-06T03:33:14.882297+0000","last_active":"2023-10-06T03:33:15.884688+0000","last_peered":"2023-10-06T03:33:15.884688+0000","last_clean":"2023-10-06T03:33:15.884688+0000","last_became_active":"2023-10-06T03:33:14.881990+0000","last_became_peered":"2023-10-06T03:33:14.881990+0000","last_unstale":"2023-10-06T03:33:15.884688+0000","last_undegraded":"2023-10-06T03:33:15.884688+0000","last_fullsized":"2023-10-06T03:33:15.884688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:56.088527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500582+0000","last_change":"2023-10-06T03:33:15.491458+0000","last_active":"2023-10-06T03:33:15.500582+0000","last_peered":"2023-10-06T03:33:15.500582+0000","last_clean":"2023-10-06T03:33:15.500582+0000","last_became_active":"2023-10-06T03:33:15.491167+0000","last_became_peered":"2023-10-06T03:33:15.491167+0000","last_unstale":"2023-10-06T03:33:15.500582+0000","last_undegraded":"2023-10-06T03:33:15.500582+0000","last_fullsized":"2023-10-06T03:33:15.500582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.964350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886971+0000","last_change":"2023-10-06T03:33:14.896256+0000","last_active":"2023-10-06T03:33:15.886971+0000","last_peered":"2023-10-06T03:33:15.886971+0000","last_clean":"2023-10-06T03:33:15.886971+0000","last_became_active":"2023-10-06T03:33:14.893978+0000","last_became_peered":"2023-10-06T03:33:14.893978+0000","last_unstale":"2023-10-06T03:33:15.886971+0000","last_undegraded":"2023-10-06T03:33:15.886971+0000","last_fullsized":"2023-10-06T03:33:15.886971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:56.088527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494409+0000","last_change":"2023-10-06T03:33:15.485216+0000","last_active":"2023-10-06T03:33:15.494409+0000","last_peered":"2023-10-06T03:33:15.494409+0000","last_clean":"2023-10-06T03:33:15.494409+0000","last_became_active":"2023-10-06T03:33:15.484942+0000","last_became_peered":"2023-10-06T03:33:15.484942+0000","last_unstale":"2023-10-06T03:33:15.494409+0000","last_undegraded":"2023-10-06T03:33:15.494409+0000","last_fullsized":"2023-10-06T03:33:15.494409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:50.279026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.492719+0000","last_change":"2023-10-06T03:33:15.489726+0000","last_active":"2023-10-06T03:33:15.492719+0000","last_peered":"2023-10-06T03:33:15.492719+0000","last_clean":"2023-10-06T03:33:15.492719+0000","last_became_active":"2023-10-06T03:33:15.489424+0000","last_became_peered":"2023-10-06T03:33:15.489424+0000","last_unstale":"2023-10-06T03:33:15.492719+0000","last_undegraded":"2023-10-06T03:33:15.492719+0000","last_fullsized":"2023-10-06T03:33:15.492719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:37.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500986+0000","last_change":"2023-10-06T03:33:15.497582+0000","last_active":"2023-10-06T03:33:15.500986+0000","last_peered":"2023-10-06T03:33:15.500986+0000","last_clean":"2023-10-06T03:33:15.500986+0000","last_became_active":"2023-10-06T03:33:15.497328+0000","last_became_peered":"2023-10-06T03:33:15.497328+0000","last_unstale":"2023-10-06T03:33:15.500986+0000","last_undegraded":"2023-10-06T03:33:15.500986+0000","last_fullsized":"2023-10-06T03:33:15.500986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:50.279026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496144+0000","last_change":"2023-10-06T03:33:15.489875+0000","last_active":"2023-10-06T03:33:15.496144+0000","last_peered":"2023-10-06T03:33:15.496144+0000","last_clean":"2023-10-06T03:33:15.496144+0000","last_became_active":"2023-10-06T03:33:15.489634+0000","last_became_peered":"2023-10-06T03:33:15.489634+0000","last_unstale":"2023-10-06T03:33:15.496144+0000","last_undegraded":"2023-10-06T03:33:15.496144+0000","last_fullsized":"2023-10-06T03:33:15.496144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:31:34.027062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500770+0000","last_change":"2023-10-06T03:33:15.493502+0000","last_active":"2023-10-06T03:33:15.500770+0000","last_peered":"2023-10-06T03:33:15.500770+0000","last_clean":"2023-10-06T03:33:15.500770+0000","last_became_active":"2023-10-06T03:33:15.492769+0000","last_became_peered":"2023-10-06T03:33:15.492769+0000","last_unstale":"2023-10-06T03:33:15.500770+0000","last_undegraded":"2023-10-06T03:33:15.500770+0000","last_fullsized":"2023-10-06T03:33:15.500770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:59.545999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493258+0000","last_change":"2023-10-06T03:33:15.487849+0000","last_active":"2023-10-06T03:33:15.493258+0000","last_peered":"2023-10-06T03:33:15.493258+0000","last_clean":"2023-10-06T03:33:15.493258+0000","last_became_active":"2023-10-06T03:33:15.487604+0000","last_became_peered":"2023-10-06T03:33:15.487604+0000","last_unstale":"2023-10-06T03:33:15.493258+0000","last_undegraded":"2023-10-06T03:33:15.493258+0000","last_fullsized":"2023-10-06T03:33:15.493258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:47.342414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298104,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073736,"statfs":{"total":96636764160,"available":96331505664,"internally_reserved":0,"allocated":835584,"data_stored":670346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":79204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0680000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.619}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302176,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069664,"statfs":{"total":96636764160,"available":96327335936,"internally_reserved":0,"allocated":811008,"data_stored":669572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301596,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070244,"statfs":{"total":96636764160,"available":96327929856,"internally_reserved":0,"allocated":217088,"data_stored":78780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301592,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070248,"statfs":{"total":96636764160,"available":96327933952,"internally_reserved":0,"allocated":212992,"data_stored":78226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301588,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070252,"statfs":{"total":96636764160,"available":96327938048,"internally_reserved":0,"allocated":208896,"data_stored":72653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":80796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":78850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:33:20.759 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:33:20.759 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T03:33:20.759 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T03:33:20.759 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T03:33:20.760 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T03:33:20.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.978+0000 7f48ae59d700 1 Processor -- start 2023-10-06T03:33:20.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.978+0000 7f48ae59d700 1 -- start start 2023-10-06T03:33:20.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.979+0000 7f48ae59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48a806d950 0x7f48a80c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.979+0000 7f48ae59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f48a806a0a0 0x7f48a80c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.979+0000 7f48ae59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a806ae70 con 0x7f48a806dd20 2023-10-06T03:33:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.979+0000 7f48ae59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a806a540 con 0x7f48a806a0a0 2023-10-06T03:33:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.979+0000 7f48ae59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a806a820 con 0x7f48a806d950 2023-10-06T03:33:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.979+0000 7f48acd9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f48a806a0a0 0x7f48a80c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.980+0000 7f48acd9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f48a806a0a0 0x7f48a80c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41046/0 (socket says 172.21.15.55:41046) 2023-10-06T03:33:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.980+0000 7f48acd9a700 1 -- 172.21.15.55:0/964585202 learned_addr learned my addr 172.21.15.55:0/964585202 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.980+0000 7f48add9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f48a806dd20 0x7f48a80c7150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56254/0 (socket says 172.21.15.55:56254) 2023-10-06T03:33:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.980+0000 7f48ad59b700 1 --2- 172.21.15.55:0/964585202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48a806d950 0x7f48a80c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.980+0000 7f48acd9a700 1 -- 172.21.15.55:0/964585202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48a806d950 msgr2=0x7f48a80c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.980+0000 7f48acd9a700 1 --2- 172.21.15.55:0/964585202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48a806d950 0x7f48a80c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.980+0000 7f48acd9a700 1 -- 172.21.15.55:0/964585202 >> v1:172.21.15.55:6789/0 conn(0x7f48a806dd20 legacy=0x7f48a80c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.980+0000 7f48acd9a700 1 -- 172.21.15.55:0/964585202 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a806ab00 con 0x7f48a806a0a0 2023-10-06T03:33:20.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.980+0000 7f489ffff700 1 -- 172.21.15.55:0/964585202 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913144836 0 0) 0x7f48a806ae70 con 0x7f48a806dd20 2023-10-06T03:33:20.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.980+0000 7f48acd9a700 1 --2- 172.21.15.55:0/964585202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48a806a0a0 0x7f48a80c6c10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f48980099b0 tx=0x7f4898009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501ac3d959f64223 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:20.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.980+0000 7f489ffff700 1 -- 172.21.15.55:0/964585202 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489800b860 con 0x7f48a806a0a0 2023-10-06T03:33:20.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.981+0000 7f489ffff700 1 -- 172.21.15.55:0/964585202 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489800b9c0 con 0x7f48a806a0a0 2023-10-06T03:33:20.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.981+0000 7f489ffff700 1 -- 172.21.15.55:0/964585202 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489800bd60 con 0x7f48a806a0a0 2023-10-06T03:33:20.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.981+0000 7f48ae59d700 1 -- 172.21.15.55:0/964585202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48a806a0a0 msgr2=0x7f48a80c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.981+0000 7f48ae59d700 1 --2- 172.21.15.55:0/964585202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48a806a0a0 0x7f48a80c6c10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f48980099b0 tx=0x7f4898009250 comp rx=0 tx=0).stop 2023-10-06T03:33:20.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.981+0000 7f48ae59d700 1 -- 172.21.15.55:0/964585202 shutdown_connections 2023-10-06T03:33:20.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.981+0000 7f48ae59d700 1 --2- 172.21.15.55:0/964585202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48a806d950 0x7f48a80c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.981+0000 7f48ae59d700 1 --2- 172.21.15.55:0/964585202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48a806a0a0 0x7f48a80c6c10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.981+0000 7f48ae59d700 1 -- 172.21.15.55:0/964585202 >> 172.21.15.55:0/964585202 conn(0x7f48a80cc1f0 msgr2=0x7f48a80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:20.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.981+0000 7f48ae59d700 1 -- 172.21.15.55:0/964585202 shutdown_connections 2023-10-06T03:33:20.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.981+0000 7f48ae59d700 1 -- 172.21.15.55:0/964585202 wait complete. 2023-10-06T03:33:20.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.983+0000 7f48ae59d700 1 Processor -- start 2023-10-06T03:33:20.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.983+0000 7f48ae59d700 1 -- start start 2023-10-06T03:33:20.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.983+0000 7f48ae59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48a806a0a0 0x7f48a8151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48ae59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48a806d950 0x7f48a80c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48ae59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48a806dd20 0x7f48a8157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48ae59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a80c32a0 con 0x7f48a806dd20 2023-10-06T03:33:20.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48ae59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a80c3580 con 0x7f48a806d950 2023-10-06T03:33:20.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48ae59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a81526e0 con 0x7f48a806a0a0 2023-10-06T03:33:20.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48add9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48a806dd20 0x7f48a8157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48add9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48a806dd20 0x7f48a8157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41052/0 (socket says 172.21.15.55:41052) 2023-10-06T03:33:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48add9c700 1 -- 172.21.15.55:0/3882949529 learned_addr learned my addr 172.21.15.55:0/3882949529 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48ad59b700 1 --2- 172.21.15.55:0/3882949529 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48a806a0a0 0x7f48a8151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48acd9a700 1 --2- 172.21.15.55:0/3882949529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48a806d950 0x7f48a80c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48add9c700 1 -- 172.21.15.55:0/3882949529 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48a806a0a0 msgr2=0x7f48a8151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48add9c700 1 --2- 172.21.15.55:0/3882949529 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48a806a0a0 0x7f48a8151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48add9c700 1 -- 172.21.15.55:0/3882949529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48a806d950 msgr2=0x7f48a80c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:20.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48add9c700 1 --2- 172.21.15.55:0/3882949529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48a806d950 0x7f48a80c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:20.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.984+0000 7f48add9c700 1 -- 172.21.15.55:0/3882949529 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48980094f0 con 0x7f48a806dd20 2023-10-06T03:33:20.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.985+0000 7f48add9c700 1 --2- 172.21.15.55:0/3882949529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48a806dd20 0x7f48a8157de0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f48a000ccc0 tx=0x7f48a0001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:20.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.985+0000 7f489d7fa700 1 -- 172.21.15.55:0/3882949529 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0005df0 con 0x7f48a806dd20 2023-10-06T03:33:20.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.985+0000 7f489d7fa700 1 -- 172.21.15.55:0/3882949529 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a000a600 con 0x7f48a806dd20 2023-10-06T03:33:20.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.985+0000 7f48ae59d700 1 -- 172.21.15.55:0/3882949529 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a815a190 con 0x7f48a806dd20 2023-10-06T03:33:20.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.985+0000 7f489d7fa700 1 -- 172.21.15.55:0/3882949529 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a000a9c0 con 0x7f48a806dd20 2023-10-06T03:33:20.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.985+0000 7f48ae59d700 1 -- 172.21.15.55:0/3882949529 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a815a750 con 0x7f48a806dd20 2023-10-06T03:33:20.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.986+0000 7f48ae59d700 1 -- 172.21.15.55:0/3882949529 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a80c36f0 con 0x7f48a806dd20 2023-10-06T03:33:20.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.987+0000 7f489d7fa700 1 -- 172.21.15.55:0/3882949529 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48a00070f0 con 0x7f48a806dd20 2023-10-06T03:33:20.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.988+0000 7f489d7fa700 1 --2- 172.21.15.55:0/3882949529 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f488c066f30 0x7f488c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:20.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.988+0000 7f489d7fa700 1 -- 172.21.15.55:0/3882949529 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f48a0084760 con 0x7f48a806dd20 2023-10-06T03:33:20.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.988+0000 7f48ad59b700 1 --2- 172.21.15.55:0/3882949529 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f488c066f30 0x7f488c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:20.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.989+0000 7f48ad59b700 1 --2- 172.21.15.55:0/3882949529 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f488c066f30 0x7f488c0693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f48a4006fd0 tx=0x7f48a4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:20.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:20.992+0000 7f489d7fa700 1 -- 172.21.15.55:0/3882949529 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48a0051900 con 0x7f48a806dd20 2023-10-06T03:33:21.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.248+0000 7f48ae59d700 1 -- 172.21.15.55:0/3882949529 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f48a804c3c0 con 0x7f48a806dd20 2023-10-06T03:33:21.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.252+0000 7f489d7fa700 1 -- 172.21.15.55:0/3882949529 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163704 (secure 0 0 0) 0x7f48a0054f20 con 0x7f48a806dd20 2023-10-06T03:33:21.256 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:21.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.255+0000 7f48ae59d700 1 -- 172.21.15.55:0/3882949529 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f488c066f30 msgr2=0x7f488c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:21.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.255+0000 7f48ae59d700 1 --2- 172.21.15.55:0/3882949529 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f488c066f30 0x7f488c0693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f48a4006fd0 tx=0x7f48a4008040 comp rx=0 tx=0).stop 2023-10-06T03:33:21.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.255+0000 7f48ae59d700 1 -- 172.21.15.55:0/3882949529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48a806dd20 msgr2=0x7f48a8157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.255+0000 7f48ae59d700 1 --2- 172.21.15.55:0/3882949529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48a806dd20 0x7f48a8157de0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f48a000ccc0 tx=0x7f48a0001a70 comp rx=0 tx=0).stop 2023-10-06T03:33:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.255+0000 7f48ae59d700 1 -- 172.21.15.55:0/3882949529 shutdown_connections 2023-10-06T03:33:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.255+0000 7f48ae59d700 1 --2- 172.21.15.55:0/3882949529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48a806dd20 0x7f48a8157de0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.255+0000 7f48ae59d700 1 --2- 172.21.15.55:0/3882949529 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f488c066f30 0x7f488c0693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.255+0000 7f48ae59d700 1 --2- 172.21.15.55:0/3882949529 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48a806a0a0 0x7f48a8151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.255+0000 7f48ae59d700 1 --2- 172.21.15.55:0/3882949529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48a806d950 0x7f48a80c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.255+0000 7f48ae59d700 1 -- 172.21.15.55:0/3882949529 >> 172.21.15.55:0/3882949529 conn(0x7f48a80cc1f0 msgr2=0x7f48a80c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.255+0000 7f48ae59d700 1 -- 172.21.15.55:0/3882949529 shutdown_connections 2023-10-06T03:33:21.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.255+0000 7f48ae59d700 1 -- 172.21.15.55:0/3882949529 wait complete. 2023-10-06T03:33:21.274 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":5,"active_gid":4114,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":85191},{"type":"v1","addr":"172.21.15.163:6833","nonce":85191}]},"active_addr":"172.21.15.163:6833/85191","active_change":"2023-10-06T03:33:02.473032+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":2882606989}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":65446995}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":2869540037}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":2697678523}]}]} 2023-10-06T03:33:21.278 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T03:33:21.278 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T03:33:21.278 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:21.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.464+0000 7f858f59e700 1 Processor -- start 2023-10-06T03:33:21.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.464+0000 7f858f59e700 1 -- start start 2023-10-06T03:33:21.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.466+0000 7f858f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f859006a660 0x7f8590157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:21.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.466+0000 7f858f59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f859006e2a0 con 0x7f8590150180 2023-10-06T03:33:21.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.466+0000 7f858f59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f859006b2e0 con 0x7f8590150550 2023-10-06T03:33:21.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.466+0000 7f858f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85900c78e0 con 0x7f859006a660 2023-10-06T03:33:21.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.466+0000 7f858e59c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8590150180 0x7f85900c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56266/0 (socket says 172.21.15.55:56266) 2023-10-06T03:33:21.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.466+0000 7f858e59c700 1 -- 172.21.15.55:0/2597675122 learned_addr learned my addr 172.21.15.55:0/2597675122 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:21.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.466+0000 7f858ed9d700 1 --2- 172.21.15.55:0/2597675122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f859006a660 0x7f8590157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:21.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.466+0000 7f858d59a700 1 -- 172.21.15.55:0/2597675122 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069673470 0 0) 0x7f859006e2a0 con 0x7f8590150180 2023-10-06T03:33:21.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.466+0000 7f858d59a700 1 -- 172.21.15.55:0/2597675122 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8578004680 con 0x7f8590150180 2023-10-06T03:33:21.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.466+0000 7f858d59a700 1 -- 172.21.15.55:0/2597675122 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155456249 0 0) 0x7f859006b2e0 con 0x7f8590150550 2023-10-06T03:33:21.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.467+0000 7f858d59a700 1 -- 172.21.15.55:0/2597675122 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f859006e2a0 con 0x7f8590150550 2023-10-06T03:33:21.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.467+0000 7f858ed9d700 1 -- 172.21.15.55:0/2597675122 >> v1:172.21.15.163:6789/0 conn(0x7f8590150550 legacy=0x7f8590069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:21.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.467+0000 7f858ed9d700 1 -- 172.21.15.55:0/2597675122 >> v1:172.21.15.55:6789/0 conn(0x7f8590150180 legacy=0x7f85900c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:21.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.467+0000 7f858ed9d700 1 -- 172.21.15.55:0/2597675122 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85900c7bc0 con 0x7f859006a660 2023-10-06T03:33:21.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.467+0000 7f858ed9d700 1 --2- 172.21.15.55:0/2597675122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f859006a660 0x7f8590157770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8588009e90 tx=0x7f8588009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ffc0c607932c0eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:21.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.467+0000 7f858d59a700 1 -- 172.21.15.55:0/2597675122 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858800e070 con 0x7f859006a660 2023-10-06T03:33:21.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.467+0000 7f858d59a700 1 -- 172.21.15.55:0/2597675122 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858800bb80 con 0x7f859006a660 2023-10-06T03:33:21.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.467+0000 7f858d59a700 1 -- 172.21.15.55:0/2597675122 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588012430 con 0x7f859006a660 2023-10-06T03:33:21.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.468+0000 7f858f59e700 1 -- 172.21.15.55:0/2597675122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f859006a660 msgr2=0x7f8590157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:21.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.468+0000 7f858f59e700 1 --2- 172.21.15.55:0/2597675122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f859006a660 0x7f8590157770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8588009e90 tx=0x7f8588009780 comp rx=0 tx=0).stop 2023-10-06T03:33:21.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.468+0000 7f858f59e700 1 -- 172.21.15.55:0/2597675122 shutdown_connections 2023-10-06T03:33:21.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.468+0000 7f858f59e700 1 --2- 172.21.15.55:0/2597675122 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f859006a660 0x7f8590157770 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.468+0000 7f858f59e700 1 -- 172.21.15.55:0/2597675122 >> 172.21.15.55:0/2597675122 conn(0x7f85900cc1f0 msgr2=0x7f85900cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:21.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.468+0000 7f858f59e700 1 -- 172.21.15.55:0/2597675122 shutdown_connections 2023-10-06T03:33:21.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.468+0000 7f858f59e700 1 -- 172.21.15.55:0/2597675122 wait complete. 2023-10-06T03:33:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.469+0000 7f858f59e700 1 Processor -- start 2023-10-06T03:33:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.469+0000 7f858f59e700 1 -- start start 2023-10-06T03:33:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.470+0000 7f858f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f859006a660 0x7f8590167140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.470+0000 7f858f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8590150180 0x7f85900b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.470+0000 7f858f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8590150550 0x7f859016d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:21.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.470+0000 7f858f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859016e050 con 0x7f8590150550 2023-10-06T03:33:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.470+0000 7f858f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85900a9ee0 con 0x7f8590150180 2023-10-06T03:33:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.470+0000 7f858f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85900a98d0 con 0x7f859006a660 2023-10-06T03:33:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.471+0000 7f858ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8590150550 0x7f859016d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.471+0000 7f858ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8590150550 0x7f859016d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41054/0 (socket says 172.21.15.55:41054) 2023-10-06T03:33:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.471+0000 7f858ed9d700 1 -- 172.21.15.55:0/2539941473 learned_addr learned my addr 172.21.15.55:0/2539941473 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.471+0000 7f858dd9b700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8590150180 0x7f85900b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.471+0000 7f858e59c700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f859006a660 0x7f8590167140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.471+0000 7f858ed9d700 1 -- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f859006a660 msgr2=0x7f8590167140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:21.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.471+0000 7f858ed9d700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f859006a660 0x7f8590167140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.471+0000 7f858ed9d700 1 -- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8590150180 msgr2=0x7f85900b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.471+0000 7f858ed9d700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8590150180 0x7f85900b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.471+0000 7f858ed9d700 1 -- 172.21.15.55:0/2539941473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8588009a20 con 0x7f8590150550 2023-10-06T03:33:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.471+0000 7f858dd9b700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8590150180 0x7f85900b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:21.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.471+0000 7f858ed9d700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8590150550 0x7f859016d1e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8588013000 tx=0x7f8588019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.471+0000 7f857e7fc700 1 -- 172.21.15.55:0/2539941473 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588012e00 con 0x7f8590150550 2023-10-06T03:33:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.472+0000 7f857e7fc700 1 -- 172.21.15.55:0/2539941473 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858801b630 con 0x7f8590150550 2023-10-06T03:33:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.472+0000 7f857e7fc700 1 -- 172.21.15.55:0/2539941473 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858801b9f0 con 0x7f8590150550 2023-10-06T03:33:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.472+0000 7f858f59e700 1 -- 172.21.15.55:0/2539941473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85900afd80 con 0x7f8590150550 2023-10-06T03:33:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.472+0000 7f858f59e700 1 -- 172.21.15.55:0/2539941473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85900a9b70 con 0x7f8590150550 2023-10-06T03:33:21.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.473+0000 7f858f59e700 1 -- 172.21.15.55:0/2539941473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85900aa090 con 0x7f8590150550 2023-10-06T03:33:21.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.474+0000 7f857e7fc700 1 -- 172.21.15.55:0/2539941473 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f858801c400 con 0x7f8590150550 2023-10-06T03:33:21.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.475+0000 7f857e7fc700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8570066f30 0x7f85700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:21.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.475+0000 7f857e7fc700 1 -- 172.21.15.55:0/2539941473 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f85880860b0 con 0x7f8590150550 2023-10-06T03:33:21.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.475+0000 7f858e59c700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8570066f30 0x7f85700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:21.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.476+0000 7f858e59c700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8570066f30 0x7f85700693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85800058d0 tx=0x7f8580009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:21.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.481+0000 7f857e7fc700 1 -- 172.21.15.55:0/2539941473 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85900aa090 con 0x7f8590150550 2023-10-06T03:33:21.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.684+0000 7f858f59e700 1 -- 172.21.15.55:0/2539941473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85900aa050 con 0x7f8590150550 2023-10-06T03:33:21.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.685+0000 7f857e7fc700 1 -- 172.21.15.55:0/2539941473 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f8588056d00 con 0x7f8590150550 2023-10-06T03:33:21.687 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:21.687 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:16.470817+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:21.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.687+0000 7f858f59e700 1 -- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8570066f30 msgr2=0x7f85700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:21.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.687+0000 7f858f59e700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8570066f30 0x7f85700693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85800058d0 tx=0x7f8580009040 comp rx=0 tx=0).stop 2023-10-06T03:33:21.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.687+0000 7f858f59e700 1 -- 172.21.15.55:0/2539941473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8590150550 msgr2=0x7f859016d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:21.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.687+0000 7f858f59e700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8590150550 0x7f859016d1e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8588013000 tx=0x7f8588019000 comp rx=0 tx=0).stop 2023-10-06T03:33:21.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.687+0000 7f858f59e700 1 -- 172.21.15.55:0/2539941473 shutdown_connections 2023-10-06T03:33:21.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.687+0000 7f858f59e700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8590150550 0x7f859016d1e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.687+0000 7f858f59e700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8570066f30 0x7f85700693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.687+0000 7f858f59e700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f859006a660 0x7f8590167140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.687+0000 7f858f59e700 1 --2- 172.21.15.55:0/2539941473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8590150180 0x7f85900b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.687+0000 7f858f59e700 1 -- 172.21.15.55:0/2539941473 >> 172.21.15.55:0/2539941473 conn(0x7f85900cc1f0 msgr2=0x7f85900c8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:21.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.687+0000 7f858f59e700 1 -- 172.21.15.55:0/2539941473 shutdown_connections 2023-10-06T03:33:21.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.687+0000 7f858f59e700 1 -- 172.21.15.55:0/2539941473 wait complete. 2023-10-06T03:33:21.706 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T03:33:21.707 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:21.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.891+0000 7f8d4e79d700 1 Processor -- start 2023-10-06T03:33:21.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.891+0000 7f8d4e79d700 1 -- start start 2023-10-06T03:33:21.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.892+0000 7f8d4e79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d480ad870 0x7f8d4806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:21.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.892+0000 7f8d4e79d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d4806bf20 0x7f8d4806a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:21.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.892+0000 7f8d4e79d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d4806c350 con 0x7f8d48142ea0 2023-10-06T03:33:21.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.892+0000 7f8d4e79d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d480b40f0 con 0x7f8d4806bf20 2023-10-06T03:33:21.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.892+0000 7f8d4e79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48069640 con 0x7f8d480ad870 2023-10-06T03:33:21.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.892+0000 7f8d4df9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d4806bf20 0x7f8d4806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:21.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.892+0000 7f8d4df9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d4806bf20 0x7f8d4806a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41056/0 (socket says 172.21.15.55:41056) 2023-10-06T03:33:21.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.892+0000 7f8d4df9c700 1 -- 172.21.15.55:0/2069251055 learned_addr learned my addr 172.21.15.55:0/2069251055 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:21.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.892+0000 7f8d4cf9a700 1 --2- 172.21.15.55:0/2069251055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d480ad870 0x7f8d4806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:21.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.893+0000 7f8d4df9c700 1 -- 172.21.15.55:0/2069251055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d480ad870 msgr2=0x7f8d4806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:21.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.893+0000 7f8d4df9c700 1 --2- 172.21.15.55:0/2069251055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d480ad870 0x7f8d4806b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.893+0000 7f8d4df9c700 1 -- 172.21.15.55:0/2069251055 >> v1:172.21.15.163:6789/0 conn(0x7f8d48142ea0 legacy=0x7f8d480b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:21.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.893+0000 7f8d4df9c700 1 -- 172.21.15.55:0/2069251055 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48069920 con 0x7f8d4806bf20 2023-10-06T03:33:21.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.893+0000 7f8d4df9c700 1 --2- 172.21.15.55:0/2069251055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d4806bf20 0x7f8d4806a120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009f20 tx=0x7f8d40009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62224531e77c101 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:21.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.893+0000 7f8d3ffff700 1 -- 172.21.15.55:0/2069251055 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4000e070 con 0x7f8d4806bf20 2023-10-06T03:33:21.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.893+0000 7f8d3ffff700 1 -- 172.21.15.55:0/2069251055 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4000b9d0 con 0x7f8d4806bf20 2023-10-06T03:33:21.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.893+0000 7f8d3ffff700 1 -- 172.21.15.55:0/2069251055 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d400092e0 con 0x7f8d4806bf20 2023-10-06T03:33:21.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.894+0000 7f8d4e79d700 1 -- 172.21.15.55:0/2069251055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d4806bf20 msgr2=0x7f8d4806a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:21.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.894+0000 7f8d4e79d700 1 --2- 172.21.15.55:0/2069251055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d4806bf20 0x7f8d4806a120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009f20 tx=0x7f8d40009840 comp rx=0 tx=0).stop 2023-10-06T03:33:21.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.894+0000 7f8d4e79d700 1 -- 172.21.15.55:0/2069251055 shutdown_connections 2023-10-06T03:33:21.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.894+0000 7f8d4e79d700 1 --2- 172.21.15.55:0/2069251055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d4806bf20 0x7f8d4806a120 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.894+0000 7f8d4e79d700 1 --2- 172.21.15.55:0/2069251055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d480ad870 0x7f8d4806b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.894+0000 7f8d4e79d700 1 -- 172.21.15.55:0/2069251055 >> 172.21.15.55:0/2069251055 conn(0x7f8d480ce430 msgr2=0x7f8d480ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:21.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.894+0000 7f8d4e79d700 1 -- 172.21.15.55:0/2069251055 shutdown_connections 2023-10-06T03:33:21.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.894+0000 7f8d4e79d700 1 -- 172.21.15.55:0/2069251055 wait complete. 2023-10-06T03:33:21.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.895+0000 7f8d4e79d700 1 Processor -- start 2023-10-06T03:33:21.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.896+0000 7f8d4e79d700 1 -- start start 2023-10-06T03:33:21.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.896+0000 7f8d4e79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d4806bf20 0x7f8d4813da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:21.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.896+0000 7f8d4e79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d48142ea0 0x7f8d4813acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:21.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.896+0000 7f8d4e79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d4813b210 0x7f8d480af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:21.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.896+0000 7f8d4e79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4813b690 con 0x7f8d48142ea0 2023-10-06T03:33:21.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.896+0000 7f8d4e79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480b0ac0 con 0x7f8d4813b210 2023-10-06T03:33:21.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.896+0000 7f8d4e79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480afeb0 con 0x7f8d4806bf20 2023-10-06T03:33:21.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.897+0000 7f8d4cf9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d48142ea0 0x7f8d4813acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:21.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.897+0000 7f8d4cf9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d48142ea0 0x7f8d4813acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41058/0 (socket says 172.21.15.55:41058) 2023-10-06T03:33:21.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.897+0000 7f8d4cf9a700 1 -- 172.21.15.55:0/3301139602 learned_addr learned my addr 172.21.15.55:0/3301139602 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:21.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.897+0000 7f8d4df9c700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d4813b210 0x7f8d480af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:21.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.897+0000 7f8d4d79b700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d4806bf20 0x7f8d4813da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:21.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.897+0000 7f8d4cf9a700 1 -- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d4806bf20 msgr2=0x7f8d4813da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:21.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.897+0000 7f8d4cf9a700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d4806bf20 0x7f8d4813da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.897+0000 7f8d4cf9a700 1 -- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d4813b210 msgr2=0x7f8d480af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:21.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.897+0000 7f8d4cf9a700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d4813b210 0x7f8d480af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:21.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.897+0000 7f8d4cf9a700 1 -- 172.21.15.55:0/3301139602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d40009ae0 con 0x7f8d48142ea0 2023-10-06T03:33:21.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.897+0000 7f8d4df9c700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d4813b210 0x7f8d480af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:21.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.897+0000 7f8d4cf9a700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d48142ea0 0x7f8d4813acd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800ec80 tx=0x7f8d380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:21.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.897+0000 7f8d3d7fa700 1 -- 172.21.15.55:0/3301139602 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800c9d0 con 0x7f8d48142ea0 2023-10-06T03:33:21.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.898+0000 7f8d3d7fa700 1 -- 172.21.15.55:0/3301139602 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3800cb30 con 0x7f8d48142ea0 2023-10-06T03:33:21.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.898+0000 7f8d4e79d700 1 -- 172.21.15.55:0/3301139602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d480b0190 con 0x7f8d48142ea0 2023-10-06T03:33:21.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.898+0000 7f8d3d7fa700 1 -- 172.21.15.55:0/3301139602 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38013470 con 0x7f8d48142ea0 2023-10-06T03:33:21.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.898+0000 7f8d4e79d700 1 -- 172.21.15.55:0/3301139602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d480b0750 con 0x7f8d48142ea0 2023-10-06T03:33:21.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.899+0000 7f8d4e79d700 1 -- 172.21.15.55:0/3301139602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4804c3c0 con 0x7f8d48142ea0 2023-10-06T03:33:21.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.899+0000 7f8d3d7fa700 1 -- 172.21.15.55:0/3301139602 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d380137e0 con 0x7f8d48142ea0 2023-10-06T03:33:21.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.901+0000 7f8d3d7fa700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8d2c066f30 0x7f8d2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:21.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.901+0000 7f8d3d7fa700 1 -- 172.21.15.55:0/3301139602 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8d38084a60 con 0x7f8d48142ea0 2023-10-06T03:33:21.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.901+0000 7f8d4d79b700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8d2c066f30 0x7f8d2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:21.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.902+0000 7f8d4d79b700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8d2c066f30 0x7f8d2c0693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d440037c0 tx=0x7f8d44009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:21.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:21.905+0000 7f8d3d7fa700 1 -- 172.21.15.55:0/3301139602 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d38051ad0 con 0x7f8d48142ea0 2023-10-06T03:33:22.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.107+0000 7f8d4e79d700 1 -- 172.21.15.55:0/3301139602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d48069c00 con 0x7f8d48142ea0 2023-10-06T03:33:22.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.108+0000 7f8d3d7fa700 1 -- 172.21.15.55:0/3301139602 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f8d380550f0 con 0x7f8d48142ea0 2023-10-06T03:33:22.110 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:22.110 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:16.470817+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:22.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.110+0000 7f8d4e79d700 1 -- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8d2c066f30 msgr2=0x7f8d2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.110+0000 7f8d4e79d700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8d2c066f30 0x7f8d2c0693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8d440037c0 tx=0x7f8d44009040 comp rx=0 tx=0).stop 2023-10-06T03:33:22.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.110+0000 7f8d4e79d700 1 -- 172.21.15.55:0/3301139602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d48142ea0 msgr2=0x7f8d4813acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.110+0000 7f8d4e79d700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d48142ea0 0x7f8d4813acd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800ec80 tx=0x7f8d380091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.110+0000 7f8d4e79d700 1 -- 172.21.15.55:0/3301139602 shutdown_connections 2023-10-06T03:33:22.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.110+0000 7f8d4e79d700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d4813b210 0x7f8d480af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.110+0000 7f8d4e79d700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8d2c066f30 0x7f8d2c0693e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.110+0000 7f8d4e79d700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d4806bf20 0x7f8d4813da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.110+0000 7f8d4e79d700 1 --2- 172.21.15.55:0/3301139602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d48142ea0 0x7f8d4813acd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.110+0000 7f8d4e79d700 1 -- 172.21.15.55:0/3301139602 >> 172.21.15.55:0/3301139602 conn(0x7f8d480ce430 msgr2=0x7f8d48141d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.111+0000 7f8d4e79d700 1 -- 172.21.15.55:0/3301139602 shutdown_connections 2023-10-06T03:33:22.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.111+0000 7f8d4e79d700 1 -- 172.21.15.55:0/3301139602 wait complete. 2023-10-06T03:33:22.132 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T03:33:22.132 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T03:33:22.132 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T03:33:22.133 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T03:33:22.133 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T03:33:22.134 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T03:33:22.134 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T03:33:22.134 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T03:33:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.468+0000 7f30041ff700 1 Processor -- start 2023-10-06T03:33:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.469+0000 7f30041ff700 1 -- start start 2023-10-06T03:33:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.470+0000 7fe368b9d700 1 Processor -- start 2023-10-06T03:33:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.469+0000 7fc092d9d700 1 Processor -- start 2023-10-06T03:33:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.470+0000 7fc092d9d700 1 -- start start 2023-10-06T03:33:22.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.470+0000 7fe368b9d700 1 -- start start 2023-10-06T03:33:22.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7f3001f9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2ff40e1ef0 0x7f2ff40e22c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:54106/0 (socket says 172.21.15.55:54106) 2023-10-06T03:33:22.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7f300179a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2ff4053f00 0x7f2ff4038c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56270/0 (socket says 172.21.15.55:56270) 2023-10-06T03:33:22.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7fc092d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc0840e9a50 0x7fc084006300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7fc092d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc0840e9e20 0x7fc084038a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7fc092d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc08403b840 0x7fc084038f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7fc09159a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc0840e9e20 0x7fc084038a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7fe368b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe35c052320 0x7fe35c033b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.473+0000 7fe3637fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fe35c0de0e0 0x7fe35c053450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:54122/0 (socket says 172.21.15.55:54122) 2023-10-06T03:33:22.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7fc092d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc084007c80 con 0x7fc0840e9e20 2023-10-06T03:33:22.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7fc092d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc084006be0 con 0x7fc0840e9a50 2023-10-06T03:33:22.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7fc092d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc084006ec0 con 0x7fc08403b840 2023-10-06T03:33:22.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7fc091d9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc0840e9a50 0x7fc084006300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7fc09159a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc0840e9e20 0x7fc084038a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41068/0 (socket says 172.21.15.55:41068) 2023-10-06T03:33:22.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7fc091d9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc0840e9a50 0x7fc084006300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:48100/0 (socket says 172.21.15.55:48100) 2023-10-06T03:33:22.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7f3001f9b700 1 -- 172.21.15.55:0/40754759 learned_addr learned my addr 172.21.15.55:0/40754759 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:22.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.473+0000 7f30041ff700 1 --2- 172.21.15.55:0/40754759 >> v2:172.21.15.55:3300/0 conn(0x7f2ff4039360 0x7f2ff4054350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.472+0000 7fc09159a700 1 -- 172.21.15.55:0/387593258 learned_addr learned my addr 172.21.15.55:0/387593258 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:22.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.473+0000 7fc09259c700 1 --2- 172.21.15.55:0/387593258 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc08403b840 0x7fc084038f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.474+0000 7f30041ff700 1 -- 172.21.15.55:0/40754759 --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff4039ae0 con 0x7f2ff4053f00 2023-10-06T03:33:22.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.474+0000 7f30041ff700 1 -- 172.21.15.55:0/40754759 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff4054890 con 0x7f2ff40e1ef0 2023-10-06T03:33:22.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.474+0000 7f30041ff700 1 -- 172.21.15.55:0/40754759 --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff40556d0 con 0x7f2ff4039360 2023-10-06T03:33:22.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.474+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683903389 0 0) 0x7f2ff4039ae0 con 0x7f2ff4053f00 2023-10-06T03:33:22.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.473+0000 7fc09259c700 1 -- 172.21.15.55:0/387593258 >> v2:172.21.15.163:3300/0 conn(0x7fc0840e9a50 msgr2=0x7fc084006300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:22.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.473+0000 7fc09259c700 1 --2- 172.21.15.55:0/387593258 >> v2:172.21.15.163:3300/0 conn(0x7fc0840e9a50 0x7fc084006300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.473+0000 7fc09259c700 1 -- 172.21.15.55:0/387593258 >> v2:172.21.15.55:3300/0 conn(0x7fc0840e9e20 msgr2=0x7fc084038a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:22.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.473+0000 7fc09259c700 1 --2- 172.21.15.55:0/387593258 >> v2:172.21.15.55:3300/0 conn(0x7fc0840e9e20 0x7fc084038a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.473+0000 7fc09259c700 1 -- 172.21.15.55:0/387593258 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0840071a0 con 0x7fc08403b840 2023-10-06T03:33:22.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.473+0000 7fc09159a700 1 --2- 172.21.15.55:0/387593258 >> v2:172.21.15.55:3300/0 conn(0x7fc0840e9e20 0x7fc084038a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:22.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.473+0000 7fc09259c700 1 --2- 172.21.15.55:0/387593258 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc08403b840 0x7fc084038f90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc08c071b60 tx=0x7fc08c066260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4fcdcd3b61c1eb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.474+0000 7fc090d99700 1 -- 172.21.15.55:0/387593258 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c04c490 con 0x7fc08403b840 2023-10-06T03:33:22.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.474+0000 7fc090d99700 1 -- 172.21.15.55:0/387593258 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c067610 con 0x7fc08403b840 2023-10-06T03:33:22.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.474+0000 7fc090d99700 1 -- 172.21.15.55:0/387593258 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c064550 con 0x7fc08403b840 2023-10-06T03:33:22.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.474+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe8003570 con 0x7f2ff4053f00 2023-10-06T03:33:22.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758751712 0 0) 0x7f2ff4054890 con 0x7f2ff40e1ef0 2023-10-06T03:33:22.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ff4039ae0 con 0x7f2ff40e1ef0 2023-10-06T03:33:22.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 185711945 0 0) 0x7f2fe8003570 con 0x7f2ff4053f00 2023-10-06T03:33:22.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 >> v2:172.21.15.55:3300/0 conn(0x7f2ff4039360 msgr2=0x7f2ff4054350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:33:22.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 --2- 172.21.15.55:0/40754759 >> v2:172.21.15.55:3300/0 conn(0x7f2ff4039360 0x7f2ff4054350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 >> v1:172.21.15.163:6789/0 conn(0x7f2ff40e1ef0 legacy=0x7f2ff40e22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff4054ac0 con 0x7f2ff4053f00 2023-10-06T03:33:22.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f2fec001470 con 0x7f2ff4053f00 2023-10-06T03:33:22.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 >> v1:172.21.15.55:6789/0 conn(0x7f2ff4053f00 legacy=0x7f2ff4038c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 --2- 172.21.15.55:0/40754759 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fe8004c50 0x7f2fe8005040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7fe363fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe35c04f450 0x7fe35c0e92f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56274/0 (socket says 172.21.15.55:56274) 2023-10-06T03:33:22.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7fe3637fe700 1 -- 172.21.15.55:0/3207572200 learned_addr learned my addr 172.21.15.55:0/3207572200 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:22.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 --2- 172.21.15.55:0/40754759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe80037b0 0x7f2fe8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 --2- 172.21.15.55:0/40754759 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe80042b0 0x7f2fe800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff40556d0 con 0x7f2fe8004c50 2023-10-06T03:33:22.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe800c230 con 0x7f2fe80037b0 2023-10-06T03:33:22.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.475+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe800c3f0 con 0x7f2fe80042b0 2023-10-06T03:33:22.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.476+0000 7fe368b9d700 1 -- 172.21.15.55:0/3207572200 --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c006030 con 0x7fe35c04f450 2023-10-06T03:33:22.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.476+0000 7fe368b9d700 1 -- 172.21.15.55:0/3207572200 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c053cc0 con 0x7fe35c0de0e0 2023-10-06T03:33:22.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.476+0000 7fe368b9d700 1 -- 172.21.15.55:0/3207572200 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35c053fa0 con 0x7fe35c052320 2023-10-06T03:33:22.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7fc092d9d700 1 -- 172.21.15.55:0/387593258 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc08403b840 msgr2=0x7fc084038f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7fc092d9d700 1 --2- 172.21.15.55:0/387593258 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc08403b840 0x7fc084038f90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc08c071b60 tx=0x7fc08c066260 comp rx=0 tx=0).stop 2023-10-06T03:33:22.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.476+0000 7f300179a700 1 --2- 172.21.15.55:0/40754759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe80037b0 0x7f2fe8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.476+0000 7f300179a700 1 -- 172.21.15.55:0/40754759 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe80042b0 msgr2=0x7f2fe800ba60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:33:22.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.476+0000 7f300179a700 1 --2- 172.21.15.55:0/40754759 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fe80042b0 0x7f2fe800ba60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7f300179a700 1 -- 172.21.15.55:0/40754759 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fe8004c50 msgr2=0x7f2fe8005040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:22.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7f300179a700 1 --2- 172.21.15.55:0/40754759 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fe8004c50 0x7f2fe8005040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7f300179a700 1 -- 172.21.15.55:0/40754759 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff4054ac0 con 0x7f2fe80037b0 2023-10-06T03:33:22.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7f300279c700 1 -- 172.21.15.55:0/40754759 reap_dead start 2023-10-06T03:33:22.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.476+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068722852 0 0) 0x7fe35c006030 con 0x7fe35c04f450 2023-10-06T03:33:22.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.476+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe354003570 con 0x7fe35c04f450 2023-10-06T03:33:22.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.476+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608652227 0 0) 0x7fe35c053cc0 con 0x7fe35c0de0e0 2023-10-06T03:33:22.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.476+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe35c006030 con 0x7fe35c0de0e0 2023-10-06T03:33:22.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7fe362ffd700 1 --2- 172.21.15.55:0/3207572200 >> v2:172.21.15.163:3300/0 conn(0x7fe35c052320 0x7fe35c033b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7f300179a700 1 --2- 172.21.15.55:0/40754759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe80037b0 0x7f2fe8003c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00c2f0 tx=0x7f2fec001e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a6f343acbe10fe3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 <== mon.1 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2066266270 0 0) 0x7fe35c006030 con 0x7fe35c0de0e0 2023-10-06T03:33:22.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 >> v2:172.21.15.163:3300/0 conn(0x7fe35c052320 msgr2=0x7fe35c033b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:22.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7fe3627fc700 1 --2- 172.21.15.55:0/3207572200 >> v2:172.21.15.163:3300/0 conn(0x7fe35c052320 0x7fe35c033b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 >> v1:172.21.15.55:6789/0 conn(0x7fe35c04f450 legacy=0x7fe35c0e92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c054280 con 0x7fe35c0de0e0 2023-10-06T03:33:22.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 <== mon.1 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fe3500013f0 con 0x7fe35c0de0e0 2023-10-06T03:33:22.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.477+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 >> v1:172.21.15.163:6789/0 conn(0x7fe35c0de0e0 legacy=0x7fe35c053450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.478+0000 7fe3627fc700 1 --2- 172.21.15.55:0/3207572200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe354003bf0 0x7fe354003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.478+0000 7fe3627fc700 1 --2- 172.21.15.55:0/3207572200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe354004810 0x7fe354004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.478+0000 7fe3627fc700 1 --2- 172.21.15.55:0/3207572200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3540051c0 0x7fe35400dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.478+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe364071690 con 0x7fe3540051c0 2023-10-06T03:33:22.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.478+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c053fa0 con 0x7fe354004810 2023-10-06T03:33:22.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.478+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35400e1e0 con 0x7fe354003bf0 2023-10-06T03:33:22.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.478+0000 7fe3637fe700 1 --2- 172.21.15.55:0/3207572200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe354003bf0 0x7fe354003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.478+0000 7fe362ffd700 1 --2- 172.21.15.55:0/3207572200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe354004810 0x7fe354004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.479+0000 7fe363fff700 1 --2- 172.21.15.55:0/3207572200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3540051c0 0x7fe35400dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.479+0000 7fe363fff700 1 -- 172.21.15.55:0/3207572200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe354003bf0 msgr2=0x7fe354003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:22.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.479+0000 7fe363fff700 1 --2- 172.21.15.55:0/3207572200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe354003bf0 0x7fe354003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.479+0000 7fe363fff700 1 -- 172.21.15.55:0/3207572200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe354004810 msgr2=0x7fe354004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:22.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.479+0000 7fe363fff700 1 --2- 172.21.15.55:0/3207572200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe354004810 0x7fe354004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.479+0000 7fe363fff700 1 -- 172.21.15.55:0/3207572200 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c054280 con 0x7fe3540051c0 2023-10-06T03:33:22.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.479+0000 7fe3637fe700 1 --2- 172.21.15.55:0/3207572200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe354003bf0 0x7fe354003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:22.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.479+0000 7fe362ffd700 1 --2- 172.21.15.55:0/3207572200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe354004810 0x7fe354004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:22.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.479+0000 7fe363fff700 1 -- 172.21.15.55:0/3207572200 reap_dead start 2023-10-06T03:33:22.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.480+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec00cb40 con 0x7f2fe80037b0 2023-10-06T03:33:22.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.480+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fec00ccc0 con 0x7f2fe80037b0 2023-10-06T03:33:22.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.480+0000 7f3000f99700 1 -- 172.21.15.55:0/40754759 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec005ab0 con 0x7f2fe80037b0 2023-10-06T03:33:22.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.480+0000 7fe363fff700 1 --2- 172.21.15.55:0/3207572200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3540051c0 0x7fe35400dc70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe36404cc40 tx=0x7fe364067410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6022bab00f93b1bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.480+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364064620 con 0x7fe3540051c0 2023-10-06T03:33:22.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.480+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3640647a0 con 0x7fe3540051c0 2023-10-06T03:33:22.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.480+0000 7fe3627fc700 1 -- 172.21.15.55:0/3207572200 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364067d80 con 0x7fe3540051c0 2023-10-06T03:33:22.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.481+0000 7f16fb59d700 1 Processor -- start 2023-10-06T03:33:22.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.481+0000 7f16fb59d700 1 -- start start 2023-10-06T03:33:22.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.481+0000 7fc092d9d700 1 -- 172.21.15.55:0/387593258 shutdown_connections 2023-10-06T03:33:22.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.481+0000 7fc092d9d700 1 --2- 172.21.15.55:0/387593258 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc08403b840 0x7fc084038f90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.481+0000 7fc092d9d700 1 --2- 172.21.15.55:0/387593258 >> v2:172.21.15.163:3300/0 conn(0x7fc0840e9a50 0x7fc084006300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.481+0000 7fc092d9d700 1 --2- 172.21.15.55:0/387593258 >> v2:172.21.15.55:3300/0 conn(0x7fc0840e9e20 0x7fc084038a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.481+0000 7fc092d9d700 1 -- 172.21.15.55:0/387593258 >> 172.21.15.55:0/387593258 conn(0x7fc08400f1c0 msgr2=0x7fc084059180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.482+0000 7fc092d9d700 1 -- 172.21.15.55:0/387593258 shutdown_connections 2023-10-06T03:33:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.481+0000 7f30041ff700 1 -- 172.21.15.55:0/40754759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe80037b0 msgr2=0x7f2fe8003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.481+0000 7f30041ff700 1 --2- 172.21.15.55:0/40754759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe80037b0 0x7f2fe8003c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00c2f0 tx=0x7f2fec001e20 comp rx=0 tx=0).stop 2023-10-06T03:33:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.481+0000 7f30041ff700 1 -- 172.21.15.55:0/40754759 shutdown_connections 2023-10-06T03:33:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.481+0000 7f30041ff700 1 --2- 172.21.15.55:0/40754759 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fe80037b0 0x7f2fe8003c20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.481+0000 7f30041ff700 1 -- 172.21.15.55:0/40754759 >> 172.21.15.55:0/40754759 conn(0x7f2ff400f0b0 msgr2=0x7f2ff4059070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.482+0000 7f30041ff700 1 -- 172.21.15.55:0/40754759 shutdown_connections 2023-10-06T03:33:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.482+0000 7f30041ff700 1 -- 172.21.15.55:0/40754759 wait complete. 2023-10-06T03:33:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.482+0000 7fe368b9d700 1 -- 172.21.15.55:0/3207572200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3540051c0 msgr2=0x7fe35400dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.482+0000 7fe368b9d700 1 --2- 172.21.15.55:0/3207572200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3540051c0 0x7fe35400dc70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe36404cc40 tx=0x7fe364067410 comp rx=0 tx=0).stop 2023-10-06T03:33:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.483+0000 7f16fb59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16ec0dffc0 0x7f16ec007100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.483+0000 7fc092d9d700 1 -- 172.21.15.55:0/387593258 wait complete. 2023-10-06T03:33:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.483+0000 7f16fb59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f16ec008120 0x7f16ec04fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.483+0000 7f16fb59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ec037fa0 con 0x7f16ec0e0390 2023-10-06T03:33:22.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.483+0000 7f16fb59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ec006c70 con 0x7f16ec008120 2023-10-06T03:33:22.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.483+0000 7f16fb59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec037cc0 con 0x7f16ec0dffc0 2023-10-06T03:33:22.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.483+0000 7fa1fbdff700 1 Processor -- start 2023-10-06T03:33:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.484+0000 7f16fa59b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16ec0dffc0 0x7f16ec007100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.484+0000 7f16fa59b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16ec0dffc0 0x7f16ec007100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:57108/0 (socket says 172.21.15.55:57108) 2023-10-06T03:33:22.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.484+0000 7f16fa59b700 1 -- 172.21.15.55:0/4193862797 learned_addr learned my addr 172.21.15.55:0/4193862797 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.483+0000 7fa1fbdff700 1 -- start start 2023-10-06T03:33:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.484+0000 7f30041ff700 1 Processor -- start 2023-10-06T03:33:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.484+0000 7f16fad9c700 1 --2- 172.21.15.55:0/4193862797 >> v2:172.21.15.163:3300/0 conn(0x7f16ec008120 0x7f16ec04fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.484+0000 7f16fa59b700 1 -- 172.21.15.55:0/4193862797 >> v2:172.21.15.163:3300/0 conn(0x7f16ec008120 msgr2=0x7f16ec04fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.485+0000 7f30041ff700 1 -- start start 2023-10-06T03:33:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.484+0000 7f16fa59b700 1 --2- 172.21.15.55:0/4193862797 >> v2:172.21.15.163:3300/0 conn(0x7f16ec008120 0x7f16ec04fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.484+0000 7f16fa59b700 1 -- 172.21.15.55:0/4193862797 >> v1:172.21.15.55:6789/0 conn(0x7f16ec0e0390 legacy=0x7f16ec007a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:33:22.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.484+0000 7f16fa59b700 1 -- 172.21.15.55:0/4193862797 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ec006620 con 0x7f16ec0dffc0 2023-10-06T03:33:22.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.485+0000 7fc092d9d700 1 Processor -- start 2023-10-06T03:33:22.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.485+0000 7f16fad9c700 1 --2- 172.21.15.55:0/4193862797 >> v2:172.21.15.163:3300/0 conn(0x7f16ec008120 0x7f16ec04fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:22.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.485+0000 7f16fa59b700 1 --2- 172.21.15.55:0/4193862797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16ec0dffc0 0x7f16ec007100 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f16f00099b0 tx=0x7f16f0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=759a6fa305c66efe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.485+0000 7fe368b9d700 1 -- 172.21.15.55:0/3207572200 shutdown_connections 2023-10-06T03:33:22.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.485+0000 7fe368b9d700 1 --2- 172.21.15.55:0/3207572200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe3540051c0 0x7fe35400dc70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.485+0000 7fe368b9d700 1 -- 172.21.15.55:0/3207572200 >> 172.21.15.55:0/3207572200 conn(0x7fe35c051810 msgr2=0x7fe35c051c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.485+0000 7fc092d9d700 1 -- start start 2023-10-06T03:33:22.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.486+0000 7fa1fbdff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1f41618c0 0x7fa1f40b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.486+0000 7f30041ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fe80037b0 0x7f2ff40e1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.486+0000 7fa1f9b9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1f41618c0 0x7fa1f40b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.486+0000 7fa1f9b9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1f41618c0 0x7fa1f40b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:48156/0 (socket says 172.21.15.55:48156) 2023-10-06T03:33:22.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.486+0000 7fc092d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc08403b840 0x7fc08404da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.486+0000 7f30041ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ff40e76f0 0x7f2ff4006470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.486+0000 7fa1f939a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa1f40b1500 0x7fa1f4146ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:54124/0 (socket says 172.21.15.55:54124) 2023-10-06T03:33:22.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.486+0000 7fa1f9b9b700 1 -- 172.21.15.55:0/4102659686 learned_addr learned my addr 172.21.15.55:0/4102659686 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:22.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.486+0000 7fc092d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0840e9a50 0x7fc0840d64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7fc092d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0840e9e20 0x7fc0840d69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.486+0000 7f3001f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fe80037b0 0x7f2ff40e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.486+0000 7f3001f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fe80037b0 0x7f2ff40e1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41104/0 (socket says 172.21.15.55:41104) 2023-10-06T03:33:22.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7f3001f9b700 1 -- 172.21.15.55:0/3276249837 learned_addr learned my addr 172.21.15.55:0/3276249837 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:22.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7f30041ff700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ff40e7b60 0x7f2ff40069b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7fa1fbdff700 1 -- 172.21.15.55:0/4102659686 --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f40b2600 con 0x7fa1f41473b0 2023-10-06T03:33:22.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7fa1fbdff700 1 -- 172.21.15.55:0/4102659686 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f40b19f0 con 0x7fa1f40b1500 2023-10-06T03:33:22.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7fc091d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc08403b840 0x7fc08404da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7f300179a700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ff40e76f0 0x7f2ff4006470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7f30041ff700 1 -- 172.21.15.55:0/3276249837 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff404cc00 con 0x7f2fe80037b0 2023-10-06T03:33:22.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7f30041ff700 1 -- 172.21.15.55:0/3276249837 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff40e8060 con 0x7f2ff40e76f0 2023-10-06T03:33:22.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7f30041ff700 1 -- 172.21.15.55:0/3276249837 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff40e8340 con 0x7f2ff40e7b60 2023-10-06T03:33:22.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7fa1fbdff700 1 -- 172.21.15.55:0/4102659686 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f40b1cd0 con 0x7fa1f41618c0 2023-10-06T03:33:22.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7fc09159a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0840e9a50 0x7fc0840d64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7fc09159a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0840e9a50 0x7fc0840d64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:57112/0 (socket says 172.21.15.55:57112) 2023-10-06T03:33:22.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7fc09159a700 1 -- 172.21.15.55:0/2797044604 learned_addr learned my addr 172.21.15.55:0/2797044604 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:22.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7fc092d9d700 1 -- 172.21.15.55:0/2797044604 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc084059620 con 0x7fc0840e9e20 2023-10-06T03:33:22.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7fe368b9d700 1 -- 172.21.15.55:0/3207572200 shutdown_connections 2023-10-06T03:33:22.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7fc092d9d700 1 -- 172.21.15.55:0/2797044604 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840d8370 con 0x7fc08403b840 2023-10-06T03:33:22.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7fe368b9d700 1 -- 172.21.15.55:0/3207572200 wait complete. 2023-10-06T03:33:22.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7fc092d9d700 1 -- 172.21.15.55:0/2797044604 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840d8650 con 0x7fc0840e9a50 2023-10-06T03:33:22.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.487+0000 7f300279c700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ff40e7b60 0x7f2ff40069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7f16f9599700 1 -- 172.21.15.55:0/4193862797 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f000b820 con 0x7f16ec0dffc0 2023-10-06T03:33:22.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7fa1f9b9b700 1 -- 172.21.15.55:0/4102659686 >> v1:172.21.15.163:6789/0 conn(0x7fa1f40b1500 legacy=0x7fa1f4146ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7fa1f9b9b700 1 -- 172.21.15.55:0/4102659686 >> v1:172.21.15.55:6789/0 conn(0x7fa1f41473b0 legacy=0x7fa1f40b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:22.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7f16f9599700 1 -- 172.21.15.55:0/4193862797 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f000b980 con 0x7f16ec0dffc0 2023-10-06T03:33:22.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7fa1f9b9b700 1 -- 172.21.15.55:0/4102659686 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f40b1fb0 con 0x7fa1f41618c0 2023-10-06T03:33:22.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7f300279c700 1 -- 172.21.15.55:0/3276249837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ff40e76f0 msgr2=0x7f2ff4006470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7f300279c700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ff40e76f0 0x7f2ff4006470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7f300279c700 1 -- 172.21.15.55:0/3276249837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fe80037b0 msgr2=0x7f2ff40e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7f300279c700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fe80037b0 0x7f2ff40e1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7f300279c700 1 -- 172.21.15.55:0/3276249837 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff40e8620 con 0x7f2ff40e7b60 2023-10-06T03:33:22.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7f3001f9b700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fe80037b0 0x7f2ff40e1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:22.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.489+0000 7f300279c700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ff40e7b60 0x7f2ff40069b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0701a0 tx=0x7f2ffc066260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.489+0000 7f16fb59d700 1 -- 172.21.15.55:0/4193862797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16ec0dffc0 msgr2=0x7f16ec007100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.489+0000 7f16fb59d700 1 --2- 172.21.15.55:0/4193862797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16ec0dffc0 0x7f16ec007100 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f16f00099b0 tx=0x7f16f0009250 comp rx=0 tx=0).stop 2023-10-06T03:33:22.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7fa1f9b9b700 1 --2- 172.21.15.55:0/4102659686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f41618c0 0x7fa1f40b0fc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009f20 tx=0x7fa1ec009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9da7d5f4f502a514 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7fd63259d700 1 Processor -- start 2023-10-06T03:33:22.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7fc09259c700 1 --2- 172.21.15.55:0/2797044604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0840e9e20 0x7fc0840d69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.489+0000 7fc09159a700 1 -- 172.21.15.55:0/2797044604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc08403b840 msgr2=0x7fc08404da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.489+0000 7fc09159a700 1 --2- 172.21.15.55:0/2797044604 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc08403b840 0x7fc08404da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.489+0000 7fc09159a700 1 -- 172.21.15.55:0/2797044604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0840e9e20 msgr2=0x7fc0840d69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.488+0000 7fd63259d700 1 -- start start 2023-10-06T03:33:22.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.489+0000 7fc09159a700 1 --2- 172.21.15.55:0/2797044604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0840e9e20 0x7fc0840d69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.489+0000 7fc09159a700 1 -- 172.21.15.55:0/2797044604 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c0094f0 con 0x7fc0840e9a50 2023-10-06T03:33:22.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7fc09259c700 1 --2- 172.21.15.55:0/2797044604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0840e9e20 0x7fc0840d69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:22.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.489+0000 7f2ff1ffb700 1 -- 172.21.15.55:0/3276249837 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc067d40 con 0x7f2ff40e7b60 2023-10-06T03:33:22.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7f2ff1ffb700 1 -- 172.21.15.55:0/3276249837 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc067ee0 con 0x7f2ff40e7b60 2023-10-06T03:33:22.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7fe368b9d700 1 Processor -- start 2023-10-06T03:33:22.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7f2ff1ffb700 1 -- 172.21.15.55:0/3276249837 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc06a660 con 0x7f2ff40e7b60 2023-10-06T03:33:22.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7f16fb59d700 1 -- 172.21.15.55:0/4193862797 shutdown_connections 2023-10-06T03:33:22.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7f16fb59d700 1 --2- 172.21.15.55:0/4193862797 >> v2:172.21.15.163:3300/0 conn(0x7f16ec008120 0x7f16ec04fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7f16fb59d700 1 --2- 172.21.15.55:0/4193862797 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16ec0dffc0 0x7f16ec007100 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7f16fb59d700 1 -- 172.21.15.55:0/4193862797 >> 172.21.15.55:0/4193862797 conn(0x7f16ec0520a0 msgr2=0x7f16ec0524a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7fc09159a700 1 --2- 172.21.15.55:0/2797044604 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0840e9a50 0x7fc0840d64b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00ec50 tx=0x7fc07c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7f16fb59d700 1 -- 172.21.15.55:0/4193862797 shutdown_connections 2023-10-06T03:33:22.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7f16fb59d700 1 -- 172.21.15.55:0/4193862797 wait complete. 2023-10-06T03:33:22.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7fc081ffb700 1 -- 172.21.15.55:0/2797044604 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00c850 con 0x7fc0840e9a50 2023-10-06T03:33:22.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7fc081ffb700 1 -- 172.21.15.55:0/2797044604 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07c00c9b0 con 0x7fc0840e9a50 2023-10-06T03:33:22.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7fc081ffb700 1 -- 172.21.15.55:0/2797044604 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00cd50 con 0x7fc0840e9a50 2023-10-06T03:33:22.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.491+0000 7fc092d9d700 1 -- 172.21.15.55:0/2797044604 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c04c9a0 con 0x7fc0840e9a50 2023-10-06T03:33:22.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.491+0000 7f30041ff700 1 -- 172.21.15.55:0/3276249837 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff40e8900 con 0x7f2ff40e7b60 2023-10-06T03:33:22.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.491+0000 7fc092d9d700 1 -- 172.21.15.55:0/2797044604 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0840d8c10 con 0x7fc0840e9a50 2023-10-06T03:33:22.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.491+0000 7fd63259d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd62c0af570 0x7fd62c0b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.491+0000 7f30041ff700 1 -- 172.21.15.55:0/3276249837 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff4052d10 con 0x7f2ff40e7b60 2023-10-06T03:33:22.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.491+0000 7fd63259d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd62c15dbc0 0x7fd62c0acda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.491+0000 7fd63259d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62c0ac850 con 0x7fd62c0ab6d0 2023-10-06T03:33:22.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.490+0000 7fe368b9d700 1 -- start start 2023-10-06T03:33:22.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.492+0000 7fe368b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3540051c0 0x7fe35c0d4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.491+0000 7fd63259d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62c0b20f0 con 0x7fd62c0af570 2023-10-06T03:33:22.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.491+0000 7fd63259d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0abc40 con 0x7fd62c15dbc0 2023-10-06T03:33:22.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.492+0000 7fe368b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe35c04ccb0 0x7fe35c035c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.492+0000 7fd631d9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd62c15dbc0 0x7fd62c0acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.492+0000 7fd63159b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd62c0af570 0x7fd62c0b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.492+0000 7fd631d9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd62c15dbc0 0x7fd62c0acda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:57128/0 (socket says 172.21.15.55:57128) 2023-10-06T03:33:22.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.492+0000 7fd631d9c700 1 -- 172.21.15.55:0/3558831882 learned_addr learned my addr 172.21.15.55:0/3558831882 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:22.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.492+0000 7fe3637fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3540051c0 0x7fe35c0d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.492+0000 7fe3637fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3540051c0 0x7fe35c0d4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:48194/0 (socket says 172.21.15.55:48194) 2023-10-06T03:33:22.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fd631d9c700 1 -- 172.21.15.55:0/3558831882 >> v2:172.21.15.163:3300/0 conn(0x7fd62c0af570 msgr2=0x7fd62c0b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:22.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fe3637fe700 1 -- 172.21.15.55:0/2338133678 learned_addr learned my addr 172.21.15.55:0/2338133678 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:22.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fc092d9d700 1 -- 172.21.15.55:0/2797044604 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc084005930 con 0x7fc0840e9a50 2023-10-06T03:33:22.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fe362ffd700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe35c04ccb0 0x7fe35c035c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fe368b9d700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe35c0361a0 0x7fe35c039bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fe368b9d700 1 -- 172.21.15.55:0/2338133678 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c03a920 con 0x7fe35c04ccb0 2023-10-06T03:33:22.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fd631d9c700 1 --2- 172.21.15.55:0/3558831882 >> v2:172.21.15.163:3300/0 conn(0x7fd62c0af570 0x7fd62c0b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fd631d9c700 1 -- 172.21.15.55:0/3558831882 >> v1:172.21.15.55:6789/0 conn(0x7fd62c0ab6d0 legacy=0x7fd62c15d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fd631d9c700 1 -- 172.21.15.55:0/3558831882 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c0abf20 con 0x7fd62c15dbc0 2023-10-06T03:33:22.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fe368b9d700 1 -- 172.21.15.55:0/2338133678 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c04c820 con 0x7fe3540051c0 2023-10-06T03:33:22.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fd631d9c700 1 --2- 172.21.15.55:0/3558831882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd62c15dbc0 0x7fd62c0acda0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd6240099b0 tx=0x7fd624009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f8c6ca7fa32c92b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fe368b9d700 1 -- 172.21.15.55:0/2338133678 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0365f0 con 0x7fe35c0361a0 2023-10-06T03:33:22.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.494+0000 7f16fb59d700 1 Processor -- start 2023-10-06T03:33:22.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.494+0000 7fd62bfff700 1 -- 172.21.15.55:0/3558831882 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62400b760 con 0x7fd62c15dbc0 2023-10-06T03:33:22.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.494+0000 7fd62bfff700 1 -- 172.21.15.55:0/3558831882 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62400b8c0 con 0x7fd62c15dbc0 2023-10-06T03:33:22.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.494+0000 7fd62bfff700 1 -- 172.21.15.55:0/3558831882 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62400bc60 con 0x7fd62c15dbc0 2023-10-06T03:33:22.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.494+0000 7f16fb59d700 1 -- start start 2023-10-06T03:33:22.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fe363fff700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe35c0361a0 0x7fe35c039bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.494+0000 7fe363fff700 1 -- 172.21.15.55:0/2338133678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3540051c0 msgr2=0x7fe35c0d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fa1f8b99700 1 -- 172.21.15.55:0/4102659686 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec00e070 con 0x7fa1f41618c0 2023-10-06T03:33:22.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fa1f8b99700 1 -- 172.21.15.55:0/4102659686 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec00ba40 con 0x7fa1f41618c0 2023-10-06T03:33:22.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.493+0000 7fa1f8b99700 1 -- 172.21.15.55:0/4102659686 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0092e0 con 0x7fa1f41618c0 2023-10-06T03:33:22.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.494+0000 7fa1fbdff700 1 -- 172.21.15.55:0/4102659686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f41618c0 msgr2=0x7fa1f40b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.494+0000 7fa1fbdff700 1 --2- 172.21.15.55:0/4102659686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f41618c0 0x7fa1f40b0fc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009f20 tx=0x7fa1ec009840 comp rx=0 tx=0).stop 2023-10-06T03:33:22.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.495+0000 7fa1fbdff700 1 -- 172.21.15.55:0/4102659686 shutdown_connections 2023-10-06T03:33:22.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.495+0000 7fa1fbdff700 1 --2- 172.21.15.55:0/4102659686 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f41618c0 0x7fa1f40b0fc0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009f20 tx=0x7fa1ec009840 comp rx=0 tx=0).stop 2023-10-06T03:33:22.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.495+0000 7fa1fbdff700 1 -- 172.21.15.55:0/4102659686 >> 172.21.15.55:0/4102659686 conn(0x7fa1f40d05f0 msgr2=0x7fa1f40d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.495+0000 7fd63259d700 1 -- 172.21.15.55:0/3558831882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd62c15dbc0 msgr2=0x7fd62c0acda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.495+0000 7fd63259d700 1 --2- 172.21.15.55:0/3558831882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd62c15dbc0 0x7fd62c0acda0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd6240099b0 tx=0x7fd624009250 comp rx=0 tx=0).stop 2023-10-06T03:33:22.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.494+0000 7fe363fff700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe3540051c0 0x7fe35c0d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.494+0000 7fe363fff700 1 -- 172.21.15.55:0/2338133678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe35c04ccb0 msgr2=0x7fe35c035c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.495+0000 7fd63259d700 1 -- 172.21.15.55:0/3558831882 shutdown_connections 2023-10-06T03:33:22.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.495+0000 7fd63259d700 1 --2- 172.21.15.55:0/3558831882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd62c15dbc0 0x7fd62c0acda0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.495+0000 7fd63259d700 1 --2- 172.21.15.55:0/3558831882 >> v2:172.21.15.163:3300/0 conn(0x7fd62c0af570 0x7fd62c0b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.495+0000 7fd63259d700 1 -- 172.21.15.55:0/3558831882 >> 172.21.15.55:0/3558831882 conn(0x7fd62c0cc1a0 msgr2=0x7fd62c0cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.494+0000 7fe363fff700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe35c04ccb0 0x7fe35c035c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.494+0000 7fe363fff700 1 -- 172.21.15.55:0/2338133678 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c04bef0 con 0x7fe35c0361a0 2023-10-06T03:33:22.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.495+0000 7fe363fff700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe35c0361a0 0x7fe35c039bf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe364071b60 tx=0x7fe36406c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.495+0000 7fc081ffb700 1 -- 172.21.15.55:0/2797044604 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc07c013680 con 0x7fc0840e9a50 2023-10-06T03:33:22.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7fd63259d700 1 -- 172.21.15.55:0/3558831882 shutdown_connections 2023-10-06T03:33:22.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7f16fb59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16ec008120 0x7f16ec0d8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7fe34f7fe700 1 -- 172.21.15.55:0/2338133678 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36404c9a0 con 0x7fe35c0361a0 2023-10-06T03:33:22.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7fe34f7fe700 1 -- 172.21.15.55:0/2338133678 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe364064910 con 0x7fe35c0361a0 2023-10-06T03:33:22.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7fe34f7fe700 1 -- 172.21.15.55:0/2338133678 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36406a870 con 0x7fe35c0361a0 2023-10-06T03:33:22.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7f16fb59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16ec0dffc0 0x7f16ec0da240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7fe368b9d700 1 -- 172.21.15.55:0/2338133678 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c04c1d0 con 0x7fe35c0361a0 2023-10-06T03:33:22.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7fe368b9d700 1 -- 172.21.15.55:0/2338133678 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c0d4c80 con 0x7fe35c0361a0 2023-10-06T03:33:22.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.495+0000 7f2ff1ffb700 1 -- 172.21.15.55:0/3276249837 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ffc06a9d0 con 0x7f2ff40e7b60 2023-10-06T03:33:22.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7f16fa59b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16ec008120 0x7f16ec0d8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.497+0000 7f16fa59b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16ec008120 0x7f16ec0d8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:57146/0 (socket says 172.21.15.55:57146) 2023-10-06T03:33:22.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.497+0000 7f16fa59b700 1 -- 172.21.15.55:0/3655301202 learned_addr learned my addr 172.21.15.55:0/3655301202 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:22.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.497+0000 7f16f9d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16ec0dffc0 0x7f16ec0da240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.497+0000 7f16fb59d700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16ec0e0390 0x7f16ec04d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.497+0000 7f16fb59d700 1 -- 172.21.15.55:0/3655301202 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec0dabb0 con 0x7f16ec0dffc0 2023-10-06T03:33:22.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.497+0000 7f16fb59d700 1 -- 172.21.15.55:0/3655301202 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec0dae90 con 0x7f16ec0e0390 2023-10-06T03:33:22.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.498+0000 7f16fb59d700 1 -- 172.21.15.55:0/3655301202 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ec0db170 con 0x7f16ec008120 2023-10-06T03:33:22.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.497+0000 7fa1fbdff700 1 -- 172.21.15.55:0/4102659686 shutdown_connections 2023-10-06T03:33:22.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.497+0000 7fa1fbdff700 1 -- 172.21.15.55:0/4102659686 wait complete. 2023-10-06T03:33:22.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7fd63259d700 1 -- 172.21.15.55:0/3558831882 wait complete. 2023-10-06T03:33:22.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.498+0000 7fd63259d700 1 Processor -- start 2023-10-06T03:33:22.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7f14c09fe700 1 Processor -- start 2023-10-06T03:33:22.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.497+0000 7f14c09fe700 1 -- start start 2023-10-06T03:33:22.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.498+0000 7f16fad9c700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16ec0e0390 0x7f16ec04d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.498+0000 7f16fa59b700 1 -- 172.21.15.55:0/3655301202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16ec0e0390 msgr2=0x7f16ec04d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.498+0000 7f16fa59b700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16ec0e0390 0x7f16ec04d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.498+0000 7f16fa59b700 1 -- 172.21.15.55:0/3655301202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16ec0dffc0 msgr2=0x7f16ec0da240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.498+0000 7f16fa59b700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16ec0dffc0 0x7f16ec0da240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.498+0000 7f16fa59b700 1 -- 172.21.15.55:0/3655301202 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f00094f0 con 0x7f16ec008120 2023-10-06T03:33:22.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.498+0000 7f16fad9c700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f16ec0e0390 0x7f16ec04d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:22.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7f16f9d9a700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16ec0dffc0 0x7f16ec0da240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:22.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.495+0000 7f30041ff700 1 -- 172.21.15.55:0/3276249837 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ff4052f60 con 0x7f2ff40e7b60 2023-10-06T03:33:22.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7f2ff1ffb700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2fe0067080 0x7f2fe0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7f2ff1ffb700 1 -- 172.21.15.55:0/3276249837 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2ffc0fb830 con 0x7f2ff40e7b60 2023-10-06T03:33:22.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7f2ff1ffb700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.55:6808/115446,v1:172.21.15.55:6809/115446] conn(0x7f2fe006f8b0 0x7f2fe0071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.496+0000 7f2ff1ffb700 1 -- 172.21.15.55:0/3276249837 --> [v2:172.21.15.55:6808/115446,v1:172.21.15.55:6809/115446] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2fe0072370 con 0x7f2fe006f8b0 2023-10-06T03:33:22.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7f16fa59b700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16ec008120 0x7f16ec0d8160 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009290 tx=0x7f16f0016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.497+0000 7fc081ffb700 1 --2- 172.21.15.55:0/2797044604 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc070067080 0x7fc070069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.497+0000 7fc081ffb700 1 -- 172.21.15.55:0/2797044604 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc07c084710 con 0x7fc0840e9a50 2023-10-06T03:33:22.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.497+0000 7fc081ffb700 1 --2- 172.21.15.55:0/2797044604 >> [v2:172.21.15.55:6800/115443,v1:172.21.15.55:6801/115443] conn(0x7fc07006f930 0x7fc070071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.497+0000 7fc081ffb700 1 -- 172.21.15.55:0/2797044604 --> [v2:172.21.15.55:6800/115443,v1:172.21.15.55:6801/115443] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0700723f0 con 0x7fc07006f930 2023-10-06T03:33:22.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.497+0000 7fc081ffb700 1 -- 172.21.15.55:0/2797044604 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc07c086bf0 con 0x7fc0840e9a50 2023-10-06T03:33:22.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7fa1fbdff700 1 Processor -- start 2023-10-06T03:33:22.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7fa1fbdff700 1 -- start start 2023-10-06T03:33:22.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.498+0000 7fc09259c700 1 --2- 172.21.15.55:0/2797044604 >> [v2:172.21.15.55:6800/115443,v1:172.21.15.55:6801/115443] conn(0x7fc07006f930 0x7fc070071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7fc091d9b700 1 --2- 172.21.15.55:0/2797044604 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc070067080 0x7fc070069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.498+0000 7fd63259d700 1 -- start start 2023-10-06T03:33:22.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7fd63259d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd62c0ab6d0 0x7fd62c0a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7f14c09fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc1471f0 0x7f14bc161630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7f14c09fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14bc0cec00 0x7f14bc161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7f14c09fe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14bc162410 con 0x7f14bc0d01e0 2023-10-06T03:33:22.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7f14c09fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14bc0cf140 con 0x7f14bc0cec00 2023-10-06T03:33:22.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7f14c09fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc0cf420 con 0x7f14bc1471f0 2023-10-06T03:33:22.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7f14bb59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14bc0cec00 0x7f14bc161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7f14bb59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14bc0cec00 0x7f14bc161b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:48202/0 (socket says 172.21.15.55:48202) 2023-10-06T03:33:22.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7f14bb59e700 1 -- 172.21.15.55:0/129705222 learned_addr learned my addr 172.21.15.55:0/129705222 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:22.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7f14bb59e700 1 -- 172.21.15.55:0/129705222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc1471f0 msgr2=0x7f14bc161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:22.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7fd63259d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd62c0af570 0x7fd62c16e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7f14bb59e700 1 --2- 172.21.15.55:0/129705222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc1471f0 0x7f14bc161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7f14bb59e700 1 -- 172.21.15.55:0/129705222 >> v1:172.21.15.163:6789/0 conn(0x7f14bc0d01e0 legacy=0x7f14bc0ce4f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:33:22.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7f14bb59e700 1 -- 172.21.15.55:0/129705222 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc0cf700 con 0x7f14bc0cec00 2023-10-06T03:33:22.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7f16ea7fc700 1 -- 172.21.15.55:0/3655301202 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f001b840 con 0x7f16ec008120 2023-10-06T03:33:22.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7fc09259c700 1 --2- 172.21.15.55:0/2797044604 >> [v2:172.21.15.55:6800/115443,v1:172.21.15.55:6801/115443] conn(0x7fc07006f930 0x7fc070071d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7f2f131ff700 1 Processor -- start 2023-10-06T03:33:22.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7fd63159b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd62c0ab6d0 0x7fd62c0a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7fd63159b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd62c0ab6d0 0x7fd62c0a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:57164/0 (socket says 172.21.15.55:57164) 2023-10-06T03:33:22.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7fd63159b700 1 -- 172.21.15.55:0/2568352730 learned_addr learned my addr 172.21.15.55:0/2568352730 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:22.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7fa1fbdff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1f40b1500 0x7fa1f416d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7fc091d9b700 1 --2- 172.21.15.55:0/2797044604 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc070067080 0x7fc070069530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc088009db0 tx=0x7fc0880091a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7fd630d9a700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd62c0af570 0x7fd62c16e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7fd63259d700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd62c15dbc0 0x7fd62c16e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7fa1fbdff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f41473b0 0x7fa1f40cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7fa1fbdff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1f40cfa60 0x7fa1f406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7fa1fbdff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40cff00 con 0x7fa1f40b1500 2023-10-06T03:33:22.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7fa1fbdff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f415f0b0 con 0x7fa1f41473b0 2023-10-06T03:33:22.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7fa1fbdff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f415fed0 con 0x7fa1f40cfa60 2023-10-06T03:33:22.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7fd63259d700 1 -- 172.21.15.55:0/2568352730 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c160680 con 0x7fd62c0af570 2023-10-06T03:33:22.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7fd63259d700 1 -- 172.21.15.55:0/2568352730 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c161850 con 0x7fd62c15dbc0 2023-10-06T03:33:22.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7fd631d9c700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd62c15dbc0 0x7fd62c16e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7f16ea7fc700 1 -- 172.21.15.55:0/3655301202 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f001b9a0 con 0x7f16ec008120 2023-10-06T03:33:22.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7fd63259d700 1 -- 172.21.15.55:0/2568352730 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c161b30 con 0x7fd62c0ab6d0 2023-10-06T03:33:22.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7f16fb59d700 1 -- 172.21.15.55:0/3655301202 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ec0db450 con 0x7f16ec008120 2023-10-06T03:33:22.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7f16fb59d700 1 -- 172.21.15.55:0/3655301202 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ec0dbc50 con 0x7f16ec008120 2023-10-06T03:33:22.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7fc081ffb700 1 -- 172.21.15.55:0/2797044604 <== osd.0 v2:172.21.15.55:6800/115443 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fc0700723f0 con 0x7fc07006f930 2023-10-06T03:33:22.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.498+0000 7f300179a700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.55:6808/115446,v1:172.21.15.55:6809/115446] conn(0x7f2fe006f8b0 0x7f2fe0071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.498+0000 7f3001f9b700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2fe0067080 0x7f2fe0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7f2ff1ffb700 1 -- 172.21.15.55:0/3276249837 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ffc068050 con 0x7f2ff40e7b60 2023-10-06T03:33:22.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.499+0000 7f300179a700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.55:6808/115446,v1:172.21.15.55:6809/115446] conn(0x7f2fe006f8b0 0x7f2fe0071cc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7f2ff1ffb700 1 -- 172.21.15.55:0/3276249837 <== osd.1 v2:172.21.15.55:6808/115446 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2fe0072370 con 0x7f2fe006f8b0 2023-10-06T03:33:22.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7f3001f9b700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2fe0067080 0x7f2fe0069530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8007fd0 tx=0x7f2ff8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7f16ea7fc700 1 -- 172.21.15.55:0/3655301202 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f001bd20 con 0x7f16ec008120 2023-10-06T03:33:22.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7fa1f939a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f41473b0 0x7fa1f40cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7fa1f939a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f41473b0 0x7fa1f40cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:48210/0 (socket says 172.21.15.55:48210) 2023-10-06T03:33:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7fa1f939a700 1 -- 172.21.15.55:0/3976627068 learned_addr learned my addr 172.21.15.55:0/3976627068 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7fa1fa39c700 1 --2- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1f40cfa60 0x7fa1f406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7fa1f939a700 1 -- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1f40cfa60 msgr2=0x7fa1f406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7fa1f939a700 1 --2- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1f40cfa60 0x7fa1f406d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7fa1f939a700 1 -- 172.21.15.55:0/3976627068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1f40b1500 msgr2=0x7fa1f416d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7fa1f939a700 1 --2- 172.21.15.55:0/3976627068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1f40b1500 0x7fa1f416d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7fa1f939a700 1 -- 172.21.15.55:0/3976627068 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec009ae0 con 0x7fa1f41473b0 2023-10-06T03:33:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7f14bb59e700 1 --2- 172.21.15.55:0/129705222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc0cec00 0x7f14bc161b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009f20 tx=0x7f14b4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c10f652d2f3c9c87 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7f14b9d9b700 1 -- 172.21.15.55:0/129705222 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b40090d0 con 0x7f14bc0cec00 2023-10-06T03:33:22.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7f14b9d9b700 1 -- 172.21.15.55:0/129705222 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b401f070 con 0x7f14bc0cec00 2023-10-06T03:33:22.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7f14c09fe700 1 -- 172.21.15.55:0/129705222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc0cec00 msgr2=0x7f14bc161b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7f14c09fe700 1 --2- 172.21.15.55:0/129705222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc0cec00 0x7f14bc161b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009f20 tx=0x7f14b4009840 comp rx=0 tx=0).stop 2023-10-06T03:33:22.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7f16dffff700 1 -- 172.21.15.55:0/3655301202 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f16ec0054e0 con 0x7f16ec008120 2023-10-06T03:33:22.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7f2f131ff700 1 -- start start 2023-10-06T03:33:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7fa1f939a700 1 --2- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f41473b0 0x7fa1f40cf520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400c790 tx=0x7fa1e4007f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7f14c09fe700 1 -- 172.21.15.55:0/129705222 shutdown_connections 2023-10-06T03:33:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7f14c09fe700 1 --2- 172.21.15.55:0/129705222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc1471f0 0x7f14bc161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7f14c09fe700 1 --2- 172.21.15.55:0/129705222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc0cec00 0x7f14bc161b70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7f14c09fe700 1 -- 172.21.15.55:0/129705222 >> 172.21.15.55:0/129705222 conn(0x7f14bc0d0660 msgr2=0x7f14bc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7f14c09fe700 1 -- 172.21.15.55:0/129705222 shutdown_connections 2023-10-06T03:33:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7f14c09fe700 1 -- 172.21.15.55:0/129705222 wait complete. 2023-10-06T03:33:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7f2f131ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2f0c0aefa0 0x7f2f0c06a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7f2f131ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f0c06d9a0 con 0x7f2f0c0cb4d0 2023-10-06T03:33:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7f2f131ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f0c06aef0 con 0x7f2f0c155b50 2023-10-06T03:33:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.502+0000 7f2f131ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f0c06d700 con 0x7f2f0c0aefa0 2023-10-06T03:33:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7f2f0bfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2f0c0cb4d0 0x7f2f0c06a150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56304/0 (socket says 172.21.15.55:56304) 2023-10-06T03:33:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7f2f0bfff700 1 -- 172.21.15.55:0/263872883 learned_addr learned my addr 172.21.15.55:0/263872883 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:22.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7fd63159b700 1 -- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd62c15dbc0 msgr2=0x7fd62c16e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7fd63159b700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd62c15dbc0 0x7fd62c16e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7fd63159b700 1 -- 172.21.15.55:0/2568352730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd62c0af570 msgr2=0x7fd62c16e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7fd63159b700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd62c0af570 0x7fd62c16e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7fd63159b700 1 -- 172.21.15.55:0/2568352730 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6240094f0 con 0x7fd62c0ab6d0 2023-10-06T03:33:22.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7fd63159b700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd62c0ab6d0 0x7fd62c0a5eb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd620009fa0 tx=0x7fd620009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.504+0000 7fd6297fa700 1 -- 172.21.15.55:0/2568352730 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62000ec70 con 0x7fd62c0ab6d0 2023-10-06T03:33:22.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.504+0000 7fd6297fa700 1 -- 172.21.15.55:0/2568352730 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62000edd0 con 0x7fd62c0ab6d0 2023-10-06T03:33:22.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.504+0000 7fd6297fa700 1 -- 172.21.15.55:0/2568352730 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd620014650 con 0x7fd62c0ab6d0 2023-10-06T03:33:22.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7f2f1179c700 1 --2- 172.21.15.55:0/263872883 >> v2:172.21.15.55:3300/0 conn(0x7f2f0c0aefa0 0x7f2f0c06a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.504+0000 7fd63259d700 1 -- 172.21.15.55:0/2568352730 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62c161e10 con 0x7fd62c0ab6d0 2023-10-06T03:33:22.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.504+0000 7f2f1179c700 1 -- 172.21.15.55:0/263872883 >> v1:172.21.15.163:6789/0 conn(0x7f2f0c155b50 legacy=0x7f2f0c0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:22.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.504+0000 7f2f1179c700 1 -- 172.21.15.55:0/263872883 >> v1:172.21.15.55:6789/0 conn(0x7f2f0c0cb4d0 legacy=0x7f2f0c06a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.504+0000 7f2f1179c700 1 -- 172.21.15.55:0/263872883 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c06b1d0 con 0x7f2f0c0aefa0 2023-10-06T03:33:22.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7fa1e9ffb700 1 -- 172.21.15.55:0/3976627068 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4005a70 con 0x7fa1f41473b0 2023-10-06T03:33:22.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.503+0000 7fa1e9ffb700 1 -- 172.21.15.55:0/3976627068 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e4005bd0 con 0x7fa1f41473b0 2023-10-06T03:33:22.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.504+0000 7fd63259d700 1 -- 172.21.15.55:0/2568352730 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62c161420 con 0x7fd62c0ab6d0 2023-10-06T03:33:22.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.504+0000 7fa1e9ffb700 1 -- 172.21.15.55:0/3976627068 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40070d0 con 0x7fa1f41473b0 2023-10-06T03:33:22.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.504+0000 7f2f1179c700 1 --2- 172.21.15.55:0/263872883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f0c0aefa0 0x7f2f0c06a860 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400d300 tx=0x7f2f04007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb068b926953e1e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.505+0000 7fa1fbdff700 1 -- 172.21.15.55:0/3976627068 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f415f2c0 con 0x7fa1f41473b0 2023-10-06T03:33:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.500+0000 7fe34f7fe700 1 -- 172.21.15.55:0/2338133678 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe364092070 con 0x7fe35c0361a0 2023-10-06T03:33:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.505+0000 7fa1fbdff700 1 -- 172.21.15.55:0/3976627068 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f415f880 con 0x7fa1f41473b0 2023-10-06T03:33:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.501+0000 7fe368b9d700 1 -- 172.21.15.55:0/2338133678 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe35c0054e0 con 0x7fe35c0361a0 2023-10-06T03:33:22.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.505+0000 7f14c09fe700 1 Processor -- start 2023-10-06T03:33:22.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7fe34f7fe700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe348067080 0x7fe348069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.505+0000 7f14c09fe700 1 -- start start 2023-10-06T03:33:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.506+0000 7f14c09fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc0cec00 0x7f14bc06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.506+0000 7f14c09fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14bc0d01e0 0x7f14bc0b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.506+0000 7f14c09fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc1471f0 0x7f14bc0b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.506+0000 7f14c09fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc13e5a0 con 0x7f14bc0d01e0 2023-10-06T03:33:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.506+0000 7f14c09fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc06ede0 con 0x7f14bc1471f0 2023-10-06T03:33:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.506+0000 7f14c09fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14bc162a50 con 0x7f14bc0cec00 2023-10-06T03:33:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7f14bb59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc1471f0 0x7f14bc0b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7f14bad9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc0cec00 0x7f14bc06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7f14bb59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc1471f0 0x7f14bc0b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:48226/0 (socket says 172.21.15.55:48226) 2023-10-06T03:33:22.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7f14bb59e700 1 -- 172.21.15.55:0/580045808 learned_addr learned my addr 172.21.15.55:0/580045808 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:22.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7f14ba59c700 1 --2- 172.21.15.55:0/580045808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14bc0d01e0 0x7f14bc0b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7fe3637fe700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe348067080 0x7fe348069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7fe34f7fe700 1 -- 172.21.15.55:0/2338133678 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe3640fb9d0 con 0x7fe35c0361a0 2023-10-06T03:33:22.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.508+0000 7fe3637fe700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe348067080 0x7fe348069530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe350005cc0 tx=0x7fe350009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7f16ea7fc700 1 -- 172.21.15.55:0/3655301202 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f16f001c660 con 0x7f16ec008120 2023-10-06T03:33:22.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7fa1fbdff700 1 -- 172.21.15.55:0/3976627068 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa1f404c3c0 con 0x7fa1f41473b0 2023-10-06T03:33:22.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7fd6297fa700 1 -- 172.21.15.55:0/2568352730 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd620014a00 con 0x7fd62c0ab6d0 2023-10-06T03:33:22.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7f14bb59e700 1 -- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc0cec00 msgr2=0x7f14bc06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7f14bb59e700 1 --2- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14bc0cec00 0x7f14bc06e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7f14bb59e700 1 -- 172.21.15.55:0/580045808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14bc0d01e0 msgr2=0x7f14bc0b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7f14bb59e700 1 --2- 172.21.15.55:0/580045808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14bc0d01e0 0x7f14bc0b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7f14bb59e700 1 -- 172.21.15.55:0/580045808 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b4009ae0 con 0x7f14bc1471f0 2023-10-06T03:33:22.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.508+0000 7f14ba59c700 1 --2- 172.21.15.55:0/580045808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14bc0d01e0 0x7f14bc0b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:22.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.508+0000 7f14bb59e700 1 --2- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc1471f0 0x7f14bc0b13c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f14b4003000 tx=0x7f14b4028000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.508+0000 7fe34f7fe700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.55:6816/115462,v1:172.21.15.55:6817/115462] conn(0x7fe34806f930 0x7fe348071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.509+0000 7fd63259d700 1 -- 172.21.15.55:0/2568352730 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd62c06c3e0 con 0x7fd62c0ab6d0 2023-10-06T03:33:22.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.509+0000 7fd6297fa700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd610066fb0 0x7fd610069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.509+0000 7f16ea7fc700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f16d8067080 0x7f16d8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.509+0000 7fd6297fa700 1 -- 172.21.15.55:0/2568352730 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd620084e10 con 0x7fd62c0ab6d0 2023-10-06T03:33:22.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.509+0000 7fe362ffd700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.55:6816/115462,v1:172.21.15.55:6817/115462] conn(0x7fe34806f930 0x7fe348071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.509+0000 7f16ea7fc700 1 -- 172.21.15.55:0/3655301202 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f16f0086a40 con 0x7f16ec008120 2023-10-06T03:33:22.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.509+0000 7f16f9d9a700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f16d8067080 0x7f16d8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.507+0000 7f2f0b7fe700 1 -- 172.21.15.55:0/263872883 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04008bc0 con 0x7f2f0c0aefa0 2023-10-06T03:33:22.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.508+0000 7f2f0b7fe700 1 -- 172.21.15.55:0/263872883 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f04008d20 con 0x7f2f0c0aefa0 2023-10-06T03:33:22.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.508+0000 7f2f0b7fe700 1 -- 172.21.15.55:0/263872883 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04012720 con 0x7f2f0c0aefa0 2023-10-06T03:33:22.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.509+0000 7f2f131ff700 1 -- 172.21.15.55:0/263872883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f0c0aefa0 msgr2=0x7f2f0c06a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.509+0000 7f2f131ff700 1 --2- 172.21.15.55:0/263872883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f0c0aefa0 0x7f2f0c06a860 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400d300 tx=0x7f2f04007620 comp rx=0 tx=0).stop 2023-10-06T03:33:22.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.509+0000 7fd6297fa700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:6816/85786,v1:172.21.15.163:6817/85786] conn(0x7fd61006f7e0 0x7fd610071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.510+0000 7f2f131ff700 1 -- 172.21.15.55:0/263872883 shutdown_connections 2023-10-06T03:33:22.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.510+0000 7f2f131ff700 1 --2- 172.21.15.55:0/263872883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f0c0aefa0 0x7f2f0c06a860 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.510+0000 7f2f131ff700 1 -- 172.21.15.55:0/263872883 >> 172.21.15.55:0/263872883 conn(0x7f2f0c0cccb0 msgr2=0x7f2f0c0c8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.510+0000 7fd6297fa700 1 -- 172.21.15.55:0/2568352730 --> [v2:172.21.15.163:6816/85786,v1:172.21.15.163:6817/85786] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd6100722a0 con 0x7fd61006f7e0 2023-10-06T03:33:22.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.510+0000 7fd6297fa700 1 -- 172.21.15.55:0/2568352730 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd620051e80 con 0x7fd62c0ab6d0 2023-10-06T03:33:22.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.509+0000 7fa1e9ffb700 1 -- 172.21.15.55:0/3976627068 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa1e40074c0 con 0x7fa1f41473b0 2023-10-06T03:33:22.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.510+0000 7fe362ffd700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.55:6816/115462,v1:172.21.15.55:6817/115462] conn(0x7fe34806f930 0x7fe348071d40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.510+0000 7fe34f7fe700 1 -- 172.21.15.55:0/2338133678 --> [v2:172.21.15.55:6816/115462,v1:172.21.15.55:6817/115462] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3480723f0 con 0x7fe34806f930 2023-10-06T03:33:22.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.510+0000 7f16f9d9a700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f16d8067080 0x7f16d8069530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0e7220 tx=0x7f16e4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.510+0000 7f16ea7fc700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.55:6824/115464,v1:172.21.15.55:6825/115464] conn(0x7f16d806f930 0x7f16d8071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.510+0000 7f16ea7fc700 1 -- 172.21.15.55:0/3655301202 --> [v2:172.21.15.55:6824/115464,v1:172.21.15.55:6825/115464] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16d80723f0 con 0x7f16d806f930 2023-10-06T03:33:22.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.510+0000 7fa1e9ffb700 1 --2- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa1d8066f80 0x7fa1d8069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.510+0000 7f16ea7fc700 1 -- 172.21.15.55:0/3655301202 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16f0086d70 con 0x7f16ec008120 2023-10-06T03:33:22.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.511+0000 7fa1e9ffb700 1 -- 172.21.15.55:0/3976627068 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa1e4083b80 con 0x7fa1f41473b0 2023-10-06T03:33:22.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.511+0000 7f16fad9c700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.55:6824/115464,v1:172.21.15.55:6825/115464] conn(0x7f16d806f930 0x7f16d8071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.511+0000 7fa1e9ffb700 1 --2- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:6824/85783,v1:172.21.15.163:6825/85783] conn(0x7fa1d806f6c0 0x7fa1d8071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.511+0000 7fa1e9ffb700 1 -- 172.21.15.55:0/3976627068 --> [v2:172.21.15.163:6824/85783,v1:172.21.15.163:6825/85783] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1d8072180 con 0x7fa1d806f6c0 2023-10-06T03:33:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.511+0000 7fa1e9ffb700 1 -- 172.21.15.55:0/3976627068 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1e4088220 con 0x7fa1f41473b0 2023-10-06T03:33:22.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.510+0000 7f2f131ff700 1 -- 172.21.15.55:0/263872883 shutdown_connections 2023-10-06T03:33:22.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.510+0000 7f2f131ff700 1 -- 172.21.15.55:0/263872883 wait complete. 2023-10-06T03:33:22.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.511+0000 7f14a2ffd700 1 -- 172.21.15.55:0/580045808 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b40092e0 con 0x7f14bc1471f0 2023-10-06T03:33:22.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.511+0000 7fe34f7fe700 1 -- 172.21.15.55:0/2338133678 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe364100200 con 0x7fe35c0361a0 2023-10-06T03:33:22.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.511+0000 7f16fad9c700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.55:6824/115464,v1:172.21.15.55:6825/115464] conn(0x7f16d806f930 0x7f16d8071d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.512+0000 7fa1fa39c700 1 --2- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:6824/85783,v1:172.21.15.163:6825/85783] conn(0x7fa1d806f6c0 0x7fa1d8071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.512+0000 7fa1f9b9b700 1 --2- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa1d8066f80 0x7fa1d8069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.512+0000 7fe34f7fe700 1 -- 172.21.15.55:0/2338133678 <== osd.2 v2:172.21.15.55:6816/115462 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe3480723f0 con 0x7fe34806f930 2023-10-06T03:33:22.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.511+0000 7f14c09fe700 1 -- 172.21.15.55:0/580045808 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14bc162720 con 0x7f14bc1471f0 2023-10-06T03:33:22.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.512+0000 7f14c09fe700 1 -- 172.21.15.55:0/580045808 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14bc06f2d0 con 0x7f14bc1471f0 2023-10-06T03:33:22.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.512+0000 7fd630d9a700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd610066fb0 0x7fd610069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.512+0000 7fd631d9c700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:6816/85786,v1:172.21.15.163:6817/85786] conn(0x7fd61006f7e0 0x7fd610071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.512+0000 7f14a2ffd700 1 -- 172.21.15.55:0/580045808 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b402e420 con 0x7f14bc1471f0 2023-10-06T03:33:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.513+0000 7f14a2ffd700 1 -- 172.21.15.55:0/580045808 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b402e7e0 con 0x7f14bc1471f0 2023-10-06T03:33:22.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.512+0000 7fa1fa39c700 1 --2- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:6824/85783,v1:172.21.15.163:6825/85783] conn(0x7fa1d806f6c0 0x7fa1d8071ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.513+0000 7fd631d9c700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:6816/85786,v1:172.21.15.163:6817/85786] conn(0x7fd61006f7e0 0x7fd610071bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.513+0000 7f16ea7fc700 1 -- 172.21.15.55:0/3655301202 <== osd.3 v2:172.21.15.55:6824/115464 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f16d80723f0 con 0x7f16d806f930 2023-10-06T03:33:22.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.512+0000 7f2f131ff700 1 Processor -- start 2023-10-06T03:33:22.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.513+0000 7f2f131ff700 1 -- start start 2023-10-06T03:33:22.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.513+0000 7fd630d9a700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd610066fb0 0x7fd610069460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd62c160a10 tx=0x7fd618009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.513+0000 7fa1e9ffb700 1 -- 172.21.15.55:0/3976627068 <== osd.4 v2:172.21.15.163:6824/85783 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa1d8072180 con 0x7fa1d806f6c0 2023-10-06T03:33:22.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.513+0000 7f14c09fe700 1 -- 172.21.15.55:0/580045808 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f149c000fd0 con 0x7f14bc1471f0 2023-10-06T03:33:22.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.514+0000 7fa1f9b9b700 1 --2- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa1d8066f80 0x7fa1d8069430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4088e30 tx=0x7fa1ec017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.514+0000 7f2f131ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f0c0aefa0 0x7f2f0c0c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.514+0000 7f2f131ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f0c0cb4d0 0x7f2f0c0b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.515+0000 7f2f131ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f0c155b50 0x7f2f0c152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.515+0000 7f2f10f9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f0c0aefa0 0x7f2f0c0c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.515+0000 7f2f10f9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f0c0aefa0 0x7f2f0c0c3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:48236/0 (socket says 172.21.15.55:48236) 2023-10-06T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.515+0000 7f2f0bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f0c0cb4d0 0x7f2f0c0b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.515+0000 7f2f0bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f0c0cb4d0 0x7f2f0c0b10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:57180/0 (socket says 172.21.15.55:57180) 2023-10-06T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.515+0000 7f2f10f9b700 1 -- 172.21.15.55:0/47549562 learned_addr learned my addr 172.21.15.55:0/47549562 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.515+0000 7f2f131ff700 1 -- 172.21.15.55:0/47549562 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c146780 con 0x7f2f0c155b50 2023-10-06T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.515+0000 7f2f131ff700 1 -- 172.21.15.55:0/47549562 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c0c5360 con 0x7f2f0c0aefa0 2023-10-06T03:33:22.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.515+0000 7f2f131ff700 1 -- 172.21.15.55:0/47549562 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0c0c3e30 con 0x7f2f0c0cb4d0 2023-10-06T03:33:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.516+0000 7fd6297fa700 1 -- 172.21.15.55:0/2568352730 <== osd.5 v2:172.21.15.163:6816/85786 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd6100722a0 con 0x7fd61006f7e0 2023-10-06T03:33:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.516+0000 7f14a2ffd700 1 -- 172.21.15.55:0/580045808 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14b402eb90 con 0x7f14bc1471f0 2023-10-06T03:33:22.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.517+0000 7f14a2ffd700 1 --2- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f14a8067000 0x7f14a80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.517+0000 7f14a2ffd700 1 -- 172.21.15.55:0/580045808 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f14b4097240 con 0x7f14bc1471f0 2023-10-06T03:33:22.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.517+0000 7f2f0bfff700 1 -- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f0c0aefa0 msgr2=0x7f2f0c0c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.517+0000 7f2f0bfff700 1 --2- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f0c0aefa0 0x7f2f0c0c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.517+0000 7f2f0bfff700 1 -- 172.21.15.55:0/47549562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f0c155b50 msgr2=0x7f2f0c152840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:33:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.517+0000 7f14a2ffd700 1 --2- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:6800/85781,v1:172.21.15.163:6801/85781] conn(0x7f14a806f740 0x7f14a8071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.517+0000 7f14a2ffd700 1 -- 172.21.15.55:0/580045808 --> [v2:172.21.15.163:6800/85781,v1:172.21.15.163:6801/85781] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f14a8072200 con 0x7f14a806f740 2023-10-06T03:33:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.517+0000 7f14a2ffd700 1 -- 172.21.15.55:0/580045808 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14b40997e0 con 0x7f14bc1471f0 2023-10-06T03:33:22.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.519+0000 7f2f1179c700 1 --2- 172.21.15.55:0/47549562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f0c155b50 0x7f2f0c152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.519+0000 7f2f0bfff700 1 --2- 172.21.15.55:0/47549562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f0c155b50 0x7f2f0c152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.519+0000 7f2f0bfff700 1 -- 172.21.15.55:0/47549562 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f040078c0 con 0x7f2f0c0cb4d0 2023-10-06T03:33:22.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.519+0000 7f2f1179c700 1 --2- 172.21.15.55:0/47549562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f0c155b50 0x7f2f0c152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:22.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.519+0000 7f2f0bfff700 1 --2- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f0c0cb4d0 0x7f2f0c0b10e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000c7c0 tx=0x7f2f00007eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.519+0000 7f14bad9d700 1 --2- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f14a8067000 0x7f14a80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.519+0000 7f2f08ff9700 1 -- 172.21.15.55:0/47549562 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00005b30 con 0x7f2f0c0cb4d0 2023-10-06T03:33:22.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.520+0000 7f2f08ff9700 1 -- 172.21.15.55:0/47549562 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f00005cb0 con 0x7f2f0c0cb4d0 2023-10-06T03:33:22.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.521+0000 7f2f08ff9700 1 -- 172.21.15.55:0/47549562 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00004a80 con 0x7f2f0c0cb4d0 2023-10-06T03:33:22.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.521+0000 7f2f131ff700 1 -- 172.21.15.55:0/47549562 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0c0c5900 con 0x7f2f0c0cb4d0 2023-10-06T03:33:22.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.521+0000 7f2f131ff700 1 -- 172.21.15.55:0/47549562 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0c0c3870 con 0x7f2f0c0cb4d0 2023-10-06T03:33:22.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.522+0000 7f2ef27fc700 1 -- 172.21.15.55:0/47549562 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2ee4000fd0 con 0x7f2f0c0cb4d0 2023-10-06T03:33:22.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.522+0000 7f14ba59c700 1 --2- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:6800/85781,v1:172.21.15.163:6801/85781] conn(0x7f14a806f740 0x7f14a8071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.525+0000 7f2f08ff9700 1 -- 172.21.15.55:0/47549562 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2f00005030 con 0x7f2f0c0cb4d0 2023-10-06T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.526+0000 7f2f08ff9700 1 --2- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2ef8067000 0x7f2ef80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.526+0000 7f2f08ff9700 1 -- 172.21.15.55:0/47549562 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2f00083f40 con 0x7f2f0c0cb4d0 2023-10-06T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.526+0000 7f2f08ff9700 1 --2- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:6808/85782,v1:172.21.15.163:6809/85782] conn(0x7f2ef806f8b0 0x7f2ef8071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.526+0000 7f2f08ff9700 1 -- 172.21.15.55:0/47549562 --> [v2:172.21.15.163:6808/85782,v1:172.21.15.163:6809/85782] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2ef8072370 con 0x7f2ef806f8b0 2023-10-06T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.526+0000 7f2f08ff9700 1 -- 172.21.15.55:0/47549562 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f000864a0 con 0x7f2f0c0cb4d0 2023-10-06T03:33:22.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.527+0000 7f2f1179c700 1 --2- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:6808/85782,v1:172.21.15.163:6809/85782] conn(0x7f2ef806f8b0 0x7f2ef8071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.528+0000 7f14bad9d700 1 --2- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f14a8067000 0x7f14a80694b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0099d0 tx=0x7f14ac008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.528+0000 7f14ba59c700 1 --2- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:6800/85781,v1:172.21.15.163:6801/85781] conn(0x7f14a806f740 0x7f14a8071b50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.528+0000 7f2f10f9b700 1 --2- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2ef8067000 0x7f2ef80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:22.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.529+0000 7f2f1179c700 1 --2- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:6808/85782,v1:172.21.15.163:6809/85782] conn(0x7f2ef806f8b0 0x7f2ef8071cc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.529+0000 7f14a2ffd700 1 -- 172.21.15.55:0/580045808 <== osd.6 v2:172.21.15.163:6800/85781 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f14a8072200 con 0x7f14a806f740 2023-10-06T03:33:22.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.530+0000 7f2f08ff9700 1 -- 172.21.15.55:0/47549562 <== osd.7 v2:172.21.15.163:6808/85782 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2ef8072370 con 0x7f2ef806f8b0 2023-10-06T03:33:22.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.530+0000 7fc092d9d700 1 -- 172.21.15.55:0/2797044604 --> [v2:172.21.15.55:6800/115443,v1:172.21.15.55:6801/115443] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc0840d8790 con 0x7fc07006f930 2023-10-06T03:33:22.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.530+0000 7fc081ffb700 1 -- 172.21.15.55:0/2797044604 <== osd.0 v2:172.21.15.55:6800/115443 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc0840d8790 con 0x7fc07006f930 2023-10-06T03:33:22.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.532+0000 7fc092d9d700 1 -- 172.21.15.55:0/2797044604 >> [v2:172.21.15.55:6800/115443,v1:172.21.15.55:6801/115443] conn(0x7fc07006f930 msgr2=0x7fc070071d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.532+0000 7fc092d9d700 1 --2- 172.21.15.55:0/2797044604 >> [v2:172.21.15.55:6800/115443,v1:172.21.15.55:6801/115443] conn(0x7fc07006f930 0x7fc070071d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.533+0000 7f2f10f9b700 1 --2- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2ef8067000 0x7f2ef80694b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009bb0 tx=0x7f2efc009120 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:22.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.535+0000 7fc092d9d700 1 -- 172.21.15.55:0/2797044604 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc070067080 msgr2=0x7fc070069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.535+0000 7fc092d9d700 1 --2- 172.21.15.55:0/2797044604 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc070067080 0x7fc070069530 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc088009db0 tx=0x7fc0880091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.535+0000 7fc092d9d700 1 -- 172.21.15.55:0/2797044604 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0840e9a50 msgr2=0x7fc0840d64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.535+0000 7fc092d9d700 1 --2- 172.21.15.55:0/2797044604 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0840e9a50 0x7fc0840d64b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00ec50 tx=0x7fc07c009250 comp rx=0 tx=0).stop 2023-10-06T03:33:22.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.535+0000 7fc09259c700 1 -- 172.21.15.55:0/2797044604 reap_dead start 2023-10-06T03:33:22.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.534+0000 7f30041ff700 1 -- 172.21.15.55:0/3276249837 --> [v2:172.21.15.55:6808/115446,v1:172.21.15.55:6809/115446] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2ff40057e0 con 0x7f2fe006f8b0 2023-10-06T03:33:22.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.535+0000 7f2ff1ffb700 1 -- 172.21.15.55:0/3276249837 <== osd.1 v2:172.21.15.55:6808/115446 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2ff40057e0 con 0x7f2fe006f8b0 2023-10-06T03:33:22.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.536+0000 7fc092d9d700 1 -- 172.21.15.55:0/2797044604 shutdown_connections 2023-10-06T03:33:22.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.536+0000 7fc092d9d700 1 -- 172.21.15.55:0/2797044604 >> 172.21.15.55:0/2797044604 conn(0x7fc08400f1c0 msgr2=0x7fc0840e8750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.539+0000 7fc092d9d700 1 -- 172.21.15.55:0/2797044604 shutdown_connections 2023-10-06T03:33:22.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.539+0000 7fc092d9d700 1 -- 172.21.15.55:0/2797044604 wait complete. 2023-10-06T03:33:22.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.539+0000 7f30041ff700 1 -- 172.21.15.55:0/3276249837 >> [v2:172.21.15.55:6808/115446,v1:172.21.15.55:6809/115446] conn(0x7f2fe006f8b0 msgr2=0x7f2fe0071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.539+0000 7f30041ff700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.55:6808/115446,v1:172.21.15.55:6809/115446] conn(0x7f2fe006f8b0 0x7f2fe0071cc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.540+0000 7f30041ff700 1 -- 172.21.15.55:0/3276249837 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2fe0067080 msgr2=0x7f2fe0069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.540+0000 7f30041ff700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2fe0067080 0x7f2fe0069530 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8007fd0 tx=0x7f2ff8009040 comp rx=0 tx=0).stop 2023-10-06T03:33:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.540+0000 7f30041ff700 1 -- 172.21.15.55:0/3276249837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ff40e7b60 msgr2=0x7f2ff40069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.540+0000 7f30041ff700 1 --2- 172.21.15.55:0/3276249837 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ff40e7b60 0x7f2ff40069b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc0701a0 tx=0x7f2ffc066260 comp rx=0 tx=0).stop 2023-10-06T03:33:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.540+0000 7f300279c700 1 -- 172.21.15.55:0/3276249837 reap_dead start 2023-10-06T03:33:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.540+0000 7f30041ff700 1 -- 172.21.15.55:0/3276249837 shutdown_connections 2023-10-06T03:33:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.540+0000 7f30041ff700 1 -- 172.21.15.55:0/3276249837 >> 172.21.15.55:0/3276249837 conn(0x7f2ff400f0b0 msgr2=0x7f2ff403cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.541+0000 7f30041ff700 1 -- 172.21.15.55:0/3276249837 shutdown_connections 2023-10-06T03:33:22.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.541+0000 7f30041ff700 1 -- 172.21.15.55:0/3276249837 wait complete. 2023-10-06T03:33:22.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.542+0000 7fe368b9d700 1 -- 172.21.15.55:0/2338133678 --> [v2:172.21.15.55:6816/115462,v1:172.21.15.55:6817/115462] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe35c054560 con 0x7fe34806f930 2023-10-06T03:33:22.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.543+0000 7fe34f7fe700 1 -- 172.21.15.55:0/2338133678 <== osd.2 v2:172.21.15.55:6816/115462 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe35c054560 con 0x7fe34806f930 2023-10-06T03:33:22.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.544+0000 7fe34d7fa700 1 -- 172.21.15.55:0/2338133678 >> [v2:172.21.15.55:6816/115462,v1:172.21.15.55:6817/115462] conn(0x7fe34806f930 msgr2=0x7fe348071d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.544+0000 7fe34d7fa700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.55:6816/115462,v1:172.21.15.55:6817/115462] conn(0x7fe34806f930 0x7fe348071d40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.545+0000 7fe34d7fa700 1 -- 172.21.15.55:0/2338133678 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe348067080 msgr2=0x7fe348069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.545+0000 7fe34d7fa700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe348067080 0x7fe348069530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe350005cc0 tx=0x7fe350009040 comp rx=0 tx=0).stop 2023-10-06T03:33:22.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.545+0000 7fe34d7fa700 1 -- 172.21.15.55:0/2338133678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe35c0361a0 msgr2=0x7fe35c039bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.546+0000 7fe34d7fa700 1 --2- 172.21.15.55:0/2338133678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe35c0361a0 0x7fe35c039bf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe364071b60 tx=0x7fe36406c000 comp rx=0 tx=0).stop 2023-10-06T03:33:22.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.546+0000 7fe363fff700 1 -- 172.21.15.55:0/2338133678 reap_dead start 2023-10-06T03:33:22.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.546+0000 7fe34d7fa700 1 -- 172.21.15.55:0/2338133678 shutdown_connections 2023-10-06T03:33:22.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.546+0000 7fe34d7fa700 1 -- 172.21.15.55:0/2338133678 >> 172.21.15.55:0/2338133678 conn(0x7fe35c051810 msgr2=0x7fe35c0532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.547+0000 7fe34d7fa700 1 -- 172.21.15.55:0/2338133678 shutdown_connections 2023-10-06T03:33:22.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.547+0000 7fe34d7fa700 1 -- 172.21.15.55:0/2338133678 wait complete. 2023-10-06T03:33:22.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.547+0000 7fa1fbdff700 1 -- 172.21.15.55:0/3976627068 --> [v2:172.21.15.163:6824/85783,v1:172.21.15.163:6825/85783] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa1f406f8a0 con 0x7fa1d806f6c0 2023-10-06T03:33:22.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.547+0000 7fa1e9ffb700 1 -- 172.21.15.55:0/3976627068 <== osd.4 v2:172.21.15.163:6824/85783 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa1f406f8a0 con 0x7fa1d806f6c0 2023-10-06T03:33:22.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.548+0000 7fa1d77fe700 1 -- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:6824/85783,v1:172.21.15.163:6825/85783] conn(0x7fa1d806f6c0 msgr2=0x7fa1d8071ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.548+0000 7fa1d77fe700 1 --2- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:6824/85783,v1:172.21.15.163:6825/85783] conn(0x7fa1d806f6c0 0x7fa1d8071ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.549+0000 7fa1d77fe700 1 -- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa1d8066f80 msgr2=0x7fa1d8069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.549+0000 7fa1d77fe700 1 --2- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa1d8066f80 0x7fa1d8069430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4088e30 tx=0x7fa1ec017000 comp rx=0 tx=0).stop 2023-10-06T03:33:22.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.549+0000 7fa1d77fe700 1 -- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f41473b0 msgr2=0x7fa1f40cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.549+0000 7fa1d77fe700 1 --2- 172.21.15.55:0/3976627068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1f41473b0 0x7fa1f40cf520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400c790 tx=0x7fa1e4007f40 comp rx=0 tx=0).stop 2023-10-06T03:33:22.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.549+0000 7fa1fa39c700 1 -- 172.21.15.55:0/3976627068 reap_dead start 2023-10-06T03:33:22.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.550+0000 7fa1d77fe700 1 -- 172.21.15.55:0/3976627068 shutdown_connections 2023-10-06T03:33:22.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.550+0000 7fa1d77fe700 1 -- 172.21.15.55:0/3976627068 >> 172.21.15.55:0/3976627068 conn(0x7fa1f40d05f0 msgr2=0x7fa1f4068880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.551+0000 7f16dffff700 1 -- 172.21.15.55:0/3655301202 --> [v2:172.21.15.55:6824/115464,v1:172.21.15.55:6825/115464] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f16ec006900 con 0x7f16d806f930 2023-10-06T03:33:22.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.551+0000 7fa1d77fe700 1 -- 172.21.15.55:0/3976627068 shutdown_connections 2023-10-06T03:33:22.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.551+0000 7fa1d77fe700 1 -- 172.21.15.55:0/3976627068 wait complete. 2023-10-06T03:33:22.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.551+0000 7f16ea7fc700 1 -- 172.21.15.55:0/3655301202 <== osd.3 v2:172.21.15.55:6824/115464 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f16ec006900 con 0x7f16d806f930 2023-10-06T03:33:22.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.552+0000 7fd63259d700 1 -- 172.21.15.55:0/2568352730 --> [v2:172.21.15.163:6816/85786,v1:172.21.15.163:6817/85786] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd62c06c3e0 con 0x7fd61006f7e0 2023-10-06T03:33:22.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.552+0000 7f16dffff700 1 -- 172.21.15.55:0/3655301202 >> [v2:172.21.15.55:6824/115464,v1:172.21.15.55:6825/115464] conn(0x7f16d806f930 msgr2=0x7f16d8071d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.553+0000 7f16dffff700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.55:6824/115464,v1:172.21.15.55:6825/115464] conn(0x7f16d806f930 0x7f16d8071d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.553+0000 7fd6297fa700 1 -- 172.21.15.55:0/2568352730 <== osd.5 v2:172.21.15.163:6816/85786 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd62c06c3e0 con 0x7fd61006f7e0 2023-10-06T03:33:22.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.553+0000 7f16dffff700 1 -- 172.21.15.55:0/3655301202 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f16d8067080 msgr2=0x7f16d8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.553+0000 7f16dffff700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f16d8067080 0x7f16d8069530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f16ec0e7220 tx=0x7f16e4008040 comp rx=0 tx=0).stop 2023-10-06T03:33:22.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.553+0000 7f16dffff700 1 -- 172.21.15.55:0/3655301202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16ec008120 msgr2=0x7f16ec0d8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.554+0000 7fd63259d700 1 -- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:6816/85786,v1:172.21.15.163:6817/85786] conn(0x7fd61006f7e0 msgr2=0x7fd610071bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.554+0000 7fd63259d700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:6816/85786,v1:172.21.15.163:6817/85786] conn(0x7fd61006f7e0 0x7fd610071bf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.554+0000 7f16dffff700 1 --2- 172.21.15.55:0/3655301202 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f16ec008120 0x7f16ec0d8160 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009290 tx=0x7f16f0016000 comp rx=0 tx=0).stop 2023-10-06T03:33:22.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.554+0000 7f16fad9c700 1 -- 172.21.15.55:0/3655301202 reap_dead start 2023-10-06T03:33:22.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.554+0000 7fd63259d700 1 -- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd610066fb0 msgr2=0x7fd610069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.554+0000 7fd63259d700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd610066fb0 0x7fd610069460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd62c160a10 tx=0x7fd618009040 comp rx=0 tx=0).stop 2023-10-06T03:33:22.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.554+0000 7fd63259d700 1 -- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd62c0ab6d0 msgr2=0x7fd62c0a5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.554+0000 7fd63259d700 1 --2- 172.21.15.55:0/2568352730 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd62c0ab6d0 0x7fd62c0a5eb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd620009fa0 tx=0x7fd620009880 comp rx=0 tx=0).stop 2023-10-06T03:33:22.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.554+0000 7fd631d9c700 1 -- 172.21.15.55:0/2568352730 reap_dead start 2023-10-06T03:33:22.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.555+0000 7f16dffff700 1 -- 172.21.15.55:0/3655301202 shutdown_connections 2023-10-06T03:33:22.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.555+0000 7f16dffff700 1 -- 172.21.15.55:0/3655301202 >> 172.21.15.55:0/3655301202 conn(0x7f16ec0520a0 msgr2=0x7f16ec00c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.556+0000 7fd63259d700 1 -- 172.21.15.55:0/2568352730 shutdown_connections 2023-10-06T03:33:22.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.556+0000 7fd63259d700 1 -- 172.21.15.55:0/2568352730 >> 172.21.15.55:0/2568352730 conn(0x7fd62c0cc1a0 msgr2=0x7fd62c06e830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.556+0000 7f16dffff700 1 -- 172.21.15.55:0/3655301202 shutdown_connections 2023-10-06T03:33:22.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.556+0000 7fd63259d700 1 -- 172.21.15.55:0/2568352730 shutdown_connections 2023-10-06T03:33:22.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.556+0000 7fd63259d700 1 -- 172.21.15.55:0/2568352730 wait complete. 2023-10-06T03:33:22.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.556+0000 7f16dffff700 1 -- 172.21.15.55:0/3655301202 wait complete. 2023-10-06T03:33:22.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.559+0000 7f14c09fe700 1 -- 172.21.15.55:0/580045808 --> [v2:172.21.15.163:6800/85781,v1:172.21.15.163:6801/85781] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f149c002d30 con 0x7f14a806f740 2023-10-06T03:33:22.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.560+0000 7f14a2ffd700 1 -- 172.21.15.55:0/580045808 <== osd.6 v2:172.21.15.163:6800/85781 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f149c002d30 con 0x7f14a806f740 2023-10-06T03:33:22.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.560+0000 7f14c09fe700 1 -- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:6800/85781,v1:172.21.15.163:6801/85781] conn(0x7f14a806f740 msgr2=0x7f14a8071b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.561+0000 7f14c09fe700 1 --2- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:6800/85781,v1:172.21.15.163:6801/85781] conn(0x7f14a806f740 0x7f14a8071b50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.563+0000 7f2ef27fc700 1 -- 172.21.15.55:0/47549562 --> [v2:172.21.15.163:6808/85782,v1:172.21.15.163:6809/85782] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2ee4002d30 con 0x7f2ef806f8b0 2023-10-06T03:33:22.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.563+0000 7f14c09fe700 1 -- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f14a8067000 msgr2=0x7f14a80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.563+0000 7f14c09fe700 1 --2- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f14a8067000 0x7f14a80694b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0099d0 tx=0x7f14ac008040 comp rx=0 tx=0).stop 2023-10-06T03:33:22.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.563+0000 7f14c09fe700 1 -- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc1471f0 msgr2=0x7f14bc0b13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.563+0000 7f14c09fe700 1 --2- 172.21.15.55:0/580045808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14bc1471f0 0x7f14bc0b13c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f14b4003000 tx=0x7f14b4028000 comp rx=0 tx=0).stop 2023-10-06T03:33:22.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.563+0000 7f14bb59e700 1 -- 172.21.15.55:0/580045808 reap_dead start 2023-10-06T03:33:22.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.564+0000 7f14c09fe700 1 -- 172.21.15.55:0/580045808 shutdown_connections 2023-10-06T03:33:22.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.564+0000 7f14c09fe700 1 -- 172.21.15.55:0/580045808 >> 172.21.15.55:0/580045808 conn(0x7f14bc0d0660 msgr2=0x7f14bc069ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.564+0000 7f2f08ff9700 1 -- 172.21.15.55:0/47549562 <== osd.7 v2:172.21.15.163:6808/85782 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2ee4002d30 con 0x7f2ef806f8b0 2023-10-06T03:33:22.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.565+0000 7f14c09fe700 1 -- 172.21.15.55:0/580045808 shutdown_connections 2023-10-06T03:33:22.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.565+0000 7f14c09fe700 1 -- 172.21.15.55:0/580045808 wait complete. 2023-10-06T03:33:22.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.566+0000 7f2f131ff700 1 -- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:6808/85782,v1:172.21.15.163:6809/85782] conn(0x7f2ef806f8b0 msgr2=0x7f2ef8071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.566+0000 7f2f131ff700 1 --2- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:6808/85782,v1:172.21.15.163:6809/85782] conn(0x7f2ef806f8b0 0x7f2ef8071cc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.566+0000 7f2f131ff700 1 -- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2ef8067000 msgr2=0x7f2ef80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.566+0000 7f2f131ff700 1 --2- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2ef8067000 0x7f2ef80694b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009bb0 tx=0x7f2efc009120 comp rx=0 tx=0).stop 2023-10-06T03:33:22.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.567+0000 7f2f131ff700 1 -- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f0c0cb4d0 msgr2=0x7f2f0c0b10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:22.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.567+0000 7f2f131ff700 1 --2- 172.21.15.55:0/47549562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f0c0cb4d0 0x7f2f0c0b10e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000c7c0 tx=0x7f2f00007eb0 comp rx=0 tx=0).stop 2023-10-06T03:33:22.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.567+0000 7f2f1179c700 1 -- 172.21.15.55:0/47549562 reap_dead start 2023-10-06T03:33:22.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.567+0000 7f2f131ff700 1 -- 172.21.15.55:0/47549562 shutdown_connections 2023-10-06T03:33:22.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.567+0000 7f2f131ff700 1 -- 172.21.15.55:0/47549562 >> 172.21.15.55:0/47549562 conn(0x7f2f0c0cccb0 msgr2=0x7f2f0c154f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:22.631 INFO:teuthology.orchestra.run.smithi055.stdout:55834574852 2023-10-06T03:33:22.632 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T03:33:22.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.570+0000 7f2f131ff700 1 -- 172.21.15.55:0/47549562 shutdown_connections 2023-10-06T03:33:22.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:22.570+0000 7f2f131ff700 1 -- 172.21.15.55:0/47549562 wait complete. 2023-10-06T03:33:22.633 INFO:teuthology.orchestra.run.smithi055.stdout:55834574854 2023-10-06T03:33:22.633 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T03:33:22.634 INFO:teuthology.orchestra.run.smithi055.stdout:55834574853 2023-10-06T03:33:22.635 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T03:33:22.635 INFO:teuthology.orchestra.run.smithi055.stdout:55834574853 2023-10-06T03:33:22.636 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T03:33:22.636 INFO:teuthology.orchestra.run.smithi055.stdout:55834574853 2023-10-06T03:33:22.637 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T03:33:22.637 INFO:teuthology.orchestra.run.smithi055.stdout:55834574853 2023-10-06T03:33:22.638 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T03:33:22.639 INFO:teuthology.orchestra.run.smithi055.stdout:55834574853 2023-10-06T03:33:22.639 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T03:33:22.640 INFO:teuthology.orchestra.run.smithi055.stdout:55834574853 2023-10-06T03:33:22.640 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T03:33:23.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.007+0000 7f14bf5ff700 1 Processor -- start 2023-10-06T03:33:23.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.007+0000 7f14bf5ff700 1 -- start start 2023-10-06T03:33:23.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.009+0000 7f10f51ff700 1 Processor -- start 2023-10-06T03:33:23.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.009+0000 7f10f51ff700 1 -- start start 2023-10-06T03:33:23.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.010+0000 7f14bf5ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f14b00d5240 0x7f14b00060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.010+0000 7f14bf5ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f14b00340e0 0x7f14b0050270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.010+0000 7f14bf5ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b0053b20 con 0x7f14b004fe20 2023-10-06T03:33:23.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.010+0000 7f14bf5ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b0050b60 con 0x7f14b00340e0 2023-10-06T03:33:23.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.010+0000 7f14bf5ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b0050880 con 0x7f14b00d5240 2023-10-06T03:33:23.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.011+0000 7f14bcb9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f14b004fe20 0x7f14b00339d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56314/0 (socket says 172.21.15.55:56314) 2023-10-06T03:33:23.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.011+0000 7f14bcb9a700 1 -- 172.21.15.55:0/1386408881 learned_addr learned my addr 172.21.15.55:0/1386408881 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:23.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.011+0000 7f14bdb9c700 1 --2- 172.21.15.55:0/1386408881 >> v2:172.21.15.55:3300/0 conn(0x7f14b00340e0 0x7f14b0050270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.012+0000 7f10f51ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f10f0159720 0x7f10f0159af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.012+0000 7f14bd39b700 1 --2- 172.21.15.55:0/1386408881 >> v2:172.21.15.163:3300/0 conn(0x7f14b00d5240 0x7f14b00060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.012+0000 7f10f51ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f10f00ad250 0x7f10f00ad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.012+0000 7f10eed9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f10f0159720 0x7f10f0159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.012+0000 7f10ee59c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f10f00ab4e0 0x7f10f00a98d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56322/0 (socket says 172.21.15.55:56322) 2023-10-06T03:33:23.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.012+0000 7f10ee59c700 1 -- 172.21.15.55:0/1182520132 learned_addr learned my addr 172.21.15.55:0/1182520132 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:23.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.013+0000 7f10f51ff700 1 -- 172.21.15.55:0/1182520132 --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f00adbb0 con 0x7f10f00ab4e0 2023-10-06T03:33:23.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.013+0000 7f10f51ff700 1 -- 172.21.15.55:0/1182520132 --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f00ab980 con 0x7f10f00ad250 2023-10-06T03:33:23.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.013+0000 7f10f51ff700 1 -- 172.21.15.55:0/1182520132 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f00ac020 con 0x7f10f0159720 2023-10-06T03:33:23.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.013+0000 7f10ef59e700 1 --2- 172.21.15.55:0/1182520132 >> v2:172.21.15.55:3300/0 conn(0x7f10f00ad250 0x7f10f00ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.012+0000 7f14bdb9c700 1 -- 172.21.15.55:0/1386408881 >> v2:172.21.15.163:3300/0 conn(0x7f14b00d5240 msgr2=0x7f14b00060c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:23.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.013+0000 7f14affff700 1 -- 172.21.15.55:0/1386408881 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628439916 0 0) 0x7f14b0053b20 con 0x7f14b004fe20 2023-10-06T03:33:23.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.013+0000 7f14bdb9c700 1 --2- 172.21.15.55:0/1386408881 >> v2:172.21.15.163:3300/0 conn(0x7f14b00d5240 0x7f14b00060c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.013+0000 7f14bdb9c700 1 -- 172.21.15.55:0/1386408881 >> v1:172.21.15.55:6789/0 conn(0x7f14b004fe20 legacy=0x7f14b00339d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.013+0000 7f14bdb9c700 1 -- 172.21.15.55:0/1386408881 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b00534d0 con 0x7f14b00340e0 2023-10-06T03:33:23.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.013+0000 7f14bd39b700 1 --2- 172.21.15.55:0/1386408881 >> v2:172.21.15.163:3300/0 conn(0x7f14b00d5240 0x7f14b00060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:23.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.014+0000 7f14bdb9c700 1 --2- 172.21.15.55:0/1386408881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b00340e0 0x7f14b0050270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f14b000d8c0 tx=0x7f14b40080e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7409b31081abeeb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.014+0000 7f14affff700 1 -- 172.21.15.55:0/1386408881 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b400b750 con 0x7f14b00340e0 2023-10-06T03:33:23.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.014+0000 7f14affff700 1 -- 172.21.15.55:0/1386408881 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b400b8b0 con 0x7f14b00340e0 2023-10-06T03:33:23.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.014+0000 7f14affff700 1 -- 172.21.15.55:0/1386408881 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b400bc50 con 0x7f14b00340e0 2023-10-06T03:33:23.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.015+0000 7f10ef59e700 1 -- 172.21.15.55:0/1182520132 >> v2:172.21.15.163:3300/0 conn(0x7f10f0159720 msgr2=0x7f10f0159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:23.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.015+0000 7f10ef59e700 1 --2- 172.21.15.55:0/1182520132 >> v2:172.21.15.163:3300/0 conn(0x7f10f0159720 0x7f10f0159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.015+0000 7f10ef59e700 1 -- 172.21.15.55:0/1182520132 >> v1:172.21.15.55:6789/0 conn(0x7f10f00ab4e0 legacy=0x7f10f00a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.015+0000 7f10ef59e700 1 -- 172.21.15.55:0/1182520132 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f00aa440 con 0x7f10f00ad250 2023-10-06T03:33:23.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.015+0000 7f10ef59e700 1 --2- 172.21.15.55:0/1182520132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f00ad250 0x7f10f00ad670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f10f0063d40 tx=0x7f10e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18dff047c8eceb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.016+0000 7f10edd9b700 1 -- 172.21.15.55:0/1182520132 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4011630 con 0x7f10f00ad250 2023-10-06T03:33:23.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.016+0000 7f10edd9b700 1 -- 172.21.15.55:0/1182520132 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e4011790 con 0x7f10f00ad250 2023-10-06T03:33:23.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.016+0000 7f10edd9b700 1 -- 172.21.15.55:0/1182520132 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4011b50 con 0x7f10f00ad250 2023-10-06T03:33:23.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.017+0000 7f10f51ff700 1 -- 172.21.15.55:0/1182520132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f00ad250 msgr2=0x7f10f00ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.017+0000 7f10f51ff700 1 --2- 172.21.15.55:0/1182520132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f00ad250 0x7f10f00ad670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f10f0063d40 tx=0x7f10e40091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.017+0000 7f10f51ff700 1 -- 172.21.15.55:0/1182520132 shutdown_connections 2023-10-06T03:33:23.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.017+0000 7f10f51ff700 1 --2- 172.21.15.55:0/1182520132 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f00ad250 0x7f10f00ad670 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.017+0000 7f10f51ff700 1 --2- 172.21.15.55:0/1182520132 >> v2:172.21.15.163:3300/0 conn(0x7f10f0159720 0x7f10f0159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.017+0000 7f10f51ff700 1 -- 172.21.15.55:0/1182520132 >> 172.21.15.55:0/1182520132 conn(0x7f10f006a320 msgr2=0x7f10f00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.020+0000 7f14bf5ff700 1 -- 172.21.15.55:0/1386408881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b00340e0 msgr2=0x7f14b0050270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.020+0000 7f14bf5ff700 1 --2- 172.21.15.55:0/1386408881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b00340e0 0x7f14b0050270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f14b000d8c0 tx=0x7f14b40080e0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.020+0000 7f10f51ff700 1 -- 172.21.15.55:0/1182520132 shutdown_connections 2023-10-06T03:33:23.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.020+0000 7f10f51ff700 1 -- 172.21.15.55:0/1182520132 wait complete. 2023-10-06T03:33:23.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.020+0000 7f14bf5ff700 1 -- 172.21.15.55:0/1386408881 shutdown_connections 2023-10-06T03:33:23.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.020+0000 7f14bf5ff700 1 --2- 172.21.15.55:0/1386408881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b00340e0 0x7f14b0050270 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.020+0000 7f14bf5ff700 1 --2- 172.21.15.55:0/1386408881 >> v2:172.21.15.163:3300/0 conn(0x7f14b00d5240 0x7f14b00060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.020+0000 7f14bf5ff700 1 -- 172.21.15.55:0/1386408881 >> 172.21.15.55:0/1386408881 conn(0x7f14b00518a0 msgr2=0x7f14b0051ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.021+0000 7f14bf5ff700 1 -- 172.21.15.55:0/1386408881 shutdown_connections 2023-10-06T03:33:23.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.021+0000 7f14bf5ff700 1 -- 172.21.15.55:0/1386408881 wait complete. 2023-10-06T03:33:23.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.022+0000 7f2a1279d700 1 Processor -- start 2023-10-06T03:33:23.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.023+0000 7f2a1279d700 1 -- start start 2023-10-06T03:33:23.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.024+0000 7f10f51ff700 1 Processor -- start 2023-10-06T03:33:23.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.024+0000 7f10f51ff700 1 -- start start 2023-10-06T03:33:23.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.024+0000 7f14bf5ff700 1 Processor -- start 2023-10-06T03:33:23.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.024+0000 7f14bf5ff700 1 -- start start 2023-10-06T03:33:23.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.024+0000 7f90a7fff700 1 Processor -- start 2023-10-06T03:33:23.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.025+0000 7f90a7fff700 1 -- start start 2023-10-06T03:33:23.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.025+0000 7f2a1279d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a0c0cf6c0 0x7f2a0c0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f14bf5ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b00340e0 0x7f14b00d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f14bf5ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b004fe20 0x7f14b004ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.025+0000 7f2a1279d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2a0c15a090 0x7f2a0c15a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f2a1179b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f2a0c159cc0 0x7f2a0c0b2920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:54138/0 (socket says 172.21.15.55:54138) 2023-10-06T03:33:23.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f14bf5ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14b00d5240 0x7f14b004d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f2a10f9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a0c0cf6c0 0x7f2a0c0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f2a10f9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a0c0cf6c0 0x7f2a0c0b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:57182/0 (socket says 172.21.15.55:57182) 2023-10-06T03:33:23.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f14bf5ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b004e2c0 con 0x7f14b004fe20 2023-10-06T03:33:23.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f14bdb9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14b00d5240 0x7f14b004d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f2a10f9a700 1 -- 172.21.15.55:0/2874369127 learned_addr learned my addr 172.21.15.55:0/2874369127 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:23.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f14bdb9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14b00d5240 0x7f14b004d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:57196/0 (socket says 172.21.15.55:57196) 2023-10-06T03:33:23.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f14bdb9c700 1 -- 172.21.15.55:0/2003222636 learned_addr learned my addr 172.21.15.55:0/2003222636 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:23.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f2a11f9c700 1 --2- 172.21.15.55:0/2874369127 >> v2:172.21.15.55:3300/0 conn(0x7f2a0c15a090 0x7f2a0c15a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f10f51ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10f00ab4e0 0x7f10f016a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f14bf5ff700 1 -- 172.21.15.55:0/2003222636 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b00d24b0 con 0x7f14b00340e0 2023-10-06T03:33:23.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f14bf5ff700 1 -- 172.21.15.55:0/2003222636 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b004da20 con 0x7f14b00d5240 2023-10-06T03:33:23.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f10f51ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10f00ad250 0x7f10f014f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.027+0000 7f10f51ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0159720 0x7f10f0150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.027+0000 7f14bd39b700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b00340e0 0x7f14b00d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.027+0000 7f14bcb9a700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b004fe20 0x7f14b004ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.027+0000 7f10ef59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0159720 0x7f10f0150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.027+0000 7f10ef59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0159720 0x7f10f0150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41170/0 (socket says 172.21.15.55:41170) 2023-10-06T03:33:23.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.027+0000 7f10ef59e700 1 -- 172.21.15.55:0/1211066135 learned_addr learned my addr 172.21.15.55:0/1211066135 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:23.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f10f51ff700 1 -- 172.21.15.55:0/1211066135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f0152e40 con 0x7f10f0159720 2023-10-06T03:33:23.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f10f51ff700 1 -- 172.21.15.55:0/1211066135 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00c17e0 con 0x7f10f00ad250 2023-10-06T03:33:23.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f10f51ff700 1 -- 172.21.15.55:0/1211066135 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f01528c0 con 0x7f10f00ab4e0 2023-10-06T03:33:23.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f14bdb9c700 1 -- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b00340e0 msgr2=0x7f14b00d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f14bdb9c700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b00340e0 0x7f14b00d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f14bdb9c700 1 -- 172.21.15.55:0/2003222636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b004fe20 msgr2=0x7f14b004ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f14bdb9c700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b004fe20 0x7f14b004ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f14bdb9c700 1 -- 172.21.15.55:0/2003222636 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b400a4d0 con 0x7f14b00d5240 2023-10-06T03:33:23.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.027+0000 7f90a7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90a815da80 0x7f90a806e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f14bd39b700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b00340e0 0x7f14b00d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:23.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f1db519d700 1 Processor -- start 2023-10-06T03:33:23.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f1db519d700 1 -- start start 2023-10-06T03:33:23.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f14bdb9c700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14b00d5240 0x7f14b004d3c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f14b4013000 tx=0x7f14b4019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f10ef59e700 1 -- 172.21.15.55:0/1211066135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10f00ab4e0 msgr2=0x7f10f016a3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:33:23.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f10ef59e700 1 --2- 172.21.15.55:0/1211066135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10f00ab4e0 0x7f10f016a3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f10ef59e700 1 -- 172.21.15.55:0/1211066135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10f00ad250 msgr2=0x7f10f014f320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:33:23.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f10ef59e700 1 --2- 172.21.15.55:0/1211066135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10f00ad250 0x7f10f014f320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f14bcb9a700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b004fe20 0x7f14b004ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:23.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f10ef59e700 1 -- 172.21.15.55:0/1211066135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4009420 con 0x7f10f0159720 2023-10-06T03:33:23.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f14ad7fa700 1 -- 172.21.15.55:0/2003222636 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b401f070 con 0x7f14b00d5240 2023-10-06T03:33:23.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.026+0000 7f2a1279d700 1 -- 172.21.15.55:0/2874369127 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0c0b5b40 con 0x7f2a0c159cc0 2023-10-06T03:33:23.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f2a1279d700 1 -- 172.21.15.55:0/2874369127 --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0c0ce260 con 0x7f2a0c15a090 2023-10-06T03:33:23.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f2a1279d700 1 -- 172.21.15.55:0/2874369127 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0b5e20 con 0x7f2a0c0cf6c0 2023-10-06T03:33:23.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f14ad7fa700 1 -- 172.21.15.55:0/2003222636 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b400bbf0 con 0x7f14b00d5240 2023-10-06T03:33:23.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f10ef59e700 1 --2- 172.21.15.55:0/1211066135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0159720 0x7f10f0150200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f10e40093f0 tx=0x7f10e4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f14ad7fa700 1 -- 172.21.15.55:0/2003222636 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b401f070 con 0x7f14b00d5240 2023-10-06T03:33:23.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.030+0000 7f14bf5ff700 1 -- 172.21.15.55:0/2003222636 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b004dd00 con 0x7f14b00d5240 2023-10-06T03:33:23.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.030+0000 7f14bf5ff700 1 -- 172.21.15.55:0/2003222636 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b00d0d20 con 0x7f14b00d5240 2023-10-06T03:33:23.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.030+0000 7f10d6ffd700 1 -- 172.21.15.55:0/1211066135 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e401a450 con 0x7f10f0159720 2023-10-06T03:33:23.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.030+0000 7f10f51ff700 1 -- 172.21.15.55:0/1211066135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f0152b60 con 0x7f10f0159720 2023-10-06T03:33:23.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.030+0000 7f10f51ff700 1 -- 172.21.15.55:0/1211066135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f016a8f0 con 0x7f10f0159720 2023-10-06T03:33:23.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.031+0000 7f10d6ffd700 1 -- 172.21.15.55:0/1211066135 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e401a5b0 con 0x7f10f0159720 2023-10-06T03:33:23.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.031+0000 7f10d6ffd700 1 -- 172.21.15.55:0/1211066135 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e401a950 con 0x7f10f0159720 2023-10-06T03:33:23.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.031+0000 7f1db519d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1db01597f0 0x7f1db00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.027+0000 7f90a7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f90a815de50 0x7f90a80c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f90a7fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a80ca410 con 0x7f90a80696c0 2023-10-06T03:33:23.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f90a7fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a80c6f80 con 0x7f90a815de50 2023-10-06T03:33:23.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f90a7fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a80c7220 con 0x7f90a815da80 2023-10-06T03:33:23.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f90a67fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f90a815de50 0x7f90a80c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f90a67fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f90a815de50 0x7f90a80c6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41188/0 (socket says 172.21.15.55:41188) 2023-10-06T03:33:23.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.028+0000 7f90a67fc700 1 -- 172.21.15.55:0/2734943656 learned_addr learned my addr 172.21.15.55:0/2734943656 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:23.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f90a67fc700 1 -- 172.21.15.55:0/2734943656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90a815da80 msgr2=0x7f90a806e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:23.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f90a67fc700 1 --2- 172.21.15.55:0/2734943656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90a815da80 0x7f90a806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f90a67fc700 1 -- 172.21.15.55:0/2734943656 >> v1:172.21.15.55:6789/0 conn(0x7f90a80696c0 legacy=0x7f90a806ec60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:33:23.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f90a67fc700 1 -- 172.21.15.55:0/2734943656 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a80c7500 con 0x7f90a815de50 2023-10-06T03:33:23.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.029+0000 7f90a67fc700 1 --2- 172.21.15.55:0/2734943656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90a815de50 0x7f90a80c6a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009f20 tx=0x7f90a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1cd99e01b48c4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.031+0000 7f90a5ffb700 1 -- 172.21.15.55:0/2734943656 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a000e070 con 0x7f90a815de50 2023-10-06T03:33:23.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.031+0000 7f90a5ffb700 1 -- 172.21.15.55:0/2734943656 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a000ba40 con 0x7f90a815de50 2023-10-06T03:33:23.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.031+0000 7f90a5ffb700 1 -- 172.21.15.55:0/2734943656 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a00092e0 con 0x7f90a815de50 2023-10-06T03:33:23.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.031+0000 7f1db519d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db00b11d0 con 0x7f1db015d230 2023-10-06T03:33:23.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.031+0000 7f1db519d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db00b14b0 con 0x7f1db0159420 2023-10-06T03:33:23.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.031+0000 7f1db519d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db00b1790 con 0x7f1db01597f0 2023-10-06T03:33:23.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.031+0000 7f78865ff700 1 Processor -- start 2023-10-06T03:33:23.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.031+0000 7f78865ff700 1 -- start start 2023-10-06T03:33:23.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.031+0000 7f6f6cfff700 1 Processor -- start 2023-10-06T03:33:23.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.030+0000 7f2a11f9c700 1 -- 172.21.15.55:0/2874369127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a0c0cf6c0 msgr2=0x7f2a0c0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:23.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.030+0000 7f2a11f9c700 1 --2- 172.21.15.55:0/2874369127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a0c0cf6c0 0x7f2a0c0b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.030+0000 7f2a11f9c700 1 -- 172.21.15.55:0/2874369127 >> v1:172.21.15.163:6789/0 conn(0x7f2a0c159cc0 legacy=0x7f2a0c0b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.030+0000 7f2a11f9c700 1 -- 172.21.15.55:0/2874369127 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c0b6100 con 0x7f2a0c15a090 2023-10-06T03:33:23.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.032+0000 7f6f6cfff700 1 -- start start 2023-10-06T03:33:23.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.032+0000 7f90a7fff700 1 -- 172.21.15.55:0/2734943656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90a815de50 msgr2=0x7f90a80c6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.032+0000 7f2a11f9c700 1 --2- 172.21.15.55:0/2874369127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a0c15a090 0x7f2a0c15a510 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009e60 tx=0x7f2a04009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f60b16dacc00ca4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.032+0000 7f90a7fff700 1 --2- 172.21.15.55:0/2734943656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90a815de50 0x7f90a80c6a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009f20 tx=0x7f90a0009840 comp rx=0 tx=0).stop 2023-10-06T03:33:23.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.033+0000 7f1db499c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1db015d230 0x7f1db015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56330/0 (socket says 172.21.15.55:56330) 2023-10-06T03:33:23.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.032+0000 7f14bf5ff700 1 -- 172.21.15.55:0/2003222636 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b00d2280 con 0x7f14b00d5240 2023-10-06T03:33:23.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.033+0000 7f1db499c700 1 -- 172.21.15.55:0/3599227754 learned_addr learned my addr 172.21.15.55:0/3599227754 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:23.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.033+0000 7f1daf7fe700 1 --2- 172.21.15.55:0/3599227754 >> v2:172.21.15.163:3300/0 conn(0x7f1db01597f0 0x7f1db00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.033+0000 7f2a03fff700 1 -- 172.21.15.55:0/2874369127 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0400e070 con 0x7f2a0c15a090 2023-10-06T03:33:23.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.033+0000 7f14ad7fa700 1 -- 172.21.15.55:0/2003222636 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f14b4019660 con 0x7f14b00d5240 2023-10-06T03:33:23.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.033+0000 7f90a7fff700 1 -- 172.21.15.55:0/2734943656 shutdown_connections 2023-10-06T03:33:23.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.033+0000 7f90a7fff700 1 --2- 172.21.15.55:0/2734943656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90a815da80 0x7f90a806e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.033+0000 7f90a7fff700 1 --2- 172.21.15.55:0/2734943656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90a815de50 0x7f90a80c6a40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.033+0000 7f90a7fff700 1 -- 172.21.15.55:0/2734943656 >> 172.21.15.55:0/2734943656 conn(0x7f90a80cc1f0 msgr2=0x7f90a80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.033+0000 7f2a03fff700 1 -- 172.21.15.55:0/2874369127 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0400bb20 con 0x7f2a0c15a090 2023-10-06T03:33:23.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.033+0000 7f2a03fff700 1 -- 172.21.15.55:0/2874369127 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04012430 con 0x7f2a0c15a090 2023-10-06T03:33:23.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f10f51ff700 1 -- 172.21.15.55:0/1211066135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d00052b0 con 0x7f10f0159720 2023-10-06T03:33:23.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f78865ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f78780380a0 0x7f78780e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f1daeffd700 1 -- 172.21.15.55:0/3599227754 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077428393 0 0) 0x7f1db00b14b0 con 0x7f1db0159420 2023-10-06T03:33:23.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f1daeffd700 1 -- 172.21.15.55:0/3599227754 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d98003570 con 0x7f1db0159420 2023-10-06T03:33:23.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f78865ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7878008360 0x7f78780e3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f78865ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7878035100 con 0x7f7878038470 2023-10-06T03:33:23.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f78865ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7878009550 con 0x7f78780380a0 2023-10-06T03:33:23.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f78865ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7878008940 con 0x7f7878008360 2023-10-06T03:33:23.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f1daeffd700 1 -- 172.21.15.55:0/3599227754 <== mon.1 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3577760264 0 0) 0x7f1d98003570 con 0x7f1db0159420 2023-10-06T03:33:23.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f1daeffd700 1 -- 172.21.15.55:0/3599227754 >> v2:172.21.15.163:3300/0 conn(0x7f1db01597f0 msgr2=0x7f1db00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:23.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f1daeffd700 1 --2- 172.21.15.55:0/3599227754 >> v2:172.21.15.163:3300/0 conn(0x7f1db01597f0 0x7f1db00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f1daeffd700 1 -- 172.21.15.55:0/3599227754 >> v1:172.21.15.55:6789/0 conn(0x7f1db015d230 legacy=0x7f1db015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:23.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f1daeffd700 1 -- 172.21.15.55:0/3599227754 --> v1:172.21.15.163:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db00b1a70 con 0x7f1db0159420 2023-10-06T03:33:23.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f6f6659c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6f68159470 0x7f6f6815b100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:54150/0 (socket says 172.21.15.55:54150) 2023-10-06T03:33:23.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f787f7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f7878038470 0x7f7878007c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:54144/0 (socket says 172.21.15.55:54144) 2023-10-06T03:33:23.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f1daeffd700 1 -- 172.21.15.55:0/3599227754 <== mon.1 v1:172.21.15.163:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f1da0001450 con 0x7f1db0159420 2023-10-06T03:33:23.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f1daeffd700 1 -- 172.21.15.55:0/3599227754 >> v1:172.21.15.163:6789/0 conn(0x7f1db0159420 legacy=0x7f1db00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f6f6659c700 1 -- 172.21.15.55:0/3654872120 learned_addr learned my addr 172.21.15.55:0/3654872120 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:23.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f14ad7fa700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f149c066ff0 0x7f149c0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f2a1279d700 1 -- 172.21.15.55:0/2874369127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a0c15a090 msgr2=0x7f2a0c15a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f14ad7fa700 1 -- 172.21.15.55:0/2003222636 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f14b4085d40 con 0x7f14b00d5240 2023-10-06T03:33:23.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f2a1279d700 1 --2- 172.21.15.55:0/2874369127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a0c15a090 0x7f2a0c15a510 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009e60 tx=0x7f2a04009780 comp rx=0 tx=0).stop 2023-10-06T03:33:23.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f1daeffd700 1 --2- 172.21.15.55:0/3599227754 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d98003aa0 0x7f1d98003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f1daeffd700 1 --2- 172.21.15.55:0/3599227754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d98004810 0x7f1d98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f1daeffd700 1 --2- 172.21.15.55:0/3599227754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d980052e0 0x7f1d9800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f787f7fe700 1 -- 172.21.15.55:0/2409264902 learned_addr learned my addr 172.21.15.55:0/2409264902 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:23.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f1daeffd700 1 -- 172.21.15.55:0/3599227754 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00b1790 con 0x7f1d98003aa0 2023-10-06T03:33:23.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f1daeffd700 1 -- 172.21.15.55:0/3599227754 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9800c190 con 0x7f1d98004810 2023-10-06T03:33:23.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f6f6cfff700 1 --2- 172.21.15.55:0/3654872120 >> v2:172.21.15.163:3300/0 conn(0x7f6f68159840 0x7f6f680b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f1daeffd700 1 -- 172.21.15.55:0/3599227754 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9800c350 con 0x7f1d980052e0 2023-10-06T03:33:23.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f6f6cfff700 1 --2- 172.21.15.55:0/3654872120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f680cd8b0 0x7f6f680cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7fd1e75ff700 1 Processor -- start 2023-10-06T03:33:23.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f787ffff700 1 --2- 172.21.15.55:0/2409264902 >> v2:172.21.15.55:3300/0 conn(0x7f78780380a0 0x7f78780e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7fd1e75ff700 1 -- start start 2023-10-06T03:33:23.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f6f6cfff700 1 -- 172.21.15.55:0/3654872120 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f6815a680 con 0x7f6f68159470 2023-10-06T03:33:23.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f1daffff700 1 --2- 172.21.15.55:0/3599227754 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d98003aa0 0x7f1d98003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f6f66d9d700 1 --2- 172.21.15.55:0/3654872120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f680cd8b0 0x7f6f680cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f6f65d9b700 1 --2- 172.21.15.55:0/3654872120 >> v2:172.21.15.163:3300/0 conn(0x7f6f68159840 0x7f6f680b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f6f6cfff700 1 -- 172.21.15.55:0/3654872120 --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6815ae90 con 0x7f6f68159840 2023-10-06T03:33:23.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.034+0000 7f90a7fff700 1 -- 172.21.15.55:0/2734943656 shutdown_connections 2023-10-06T03:33:23.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f6f6cfff700 1 -- 172.21.15.55:0/3654872120 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680b5070 con 0x7f6f680cd8b0 2023-10-06T03:33:23.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f14bd39b700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f149c066ff0 0x7f149c0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f6f6559a700 1 -- 172.21.15.55:0/3654872120 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850693589 0 0) 0x7f6f6815a680 con 0x7f6f68159470 2023-10-06T03:33:23.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f6f6559a700 1 -- 172.21.15.55:0/3654872120 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f50003570 con 0x7f6f68159470 2023-10-06T03:33:23.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f787effd700 1 -- 172.21.15.55:0/2409264902 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590795181 0 0) 0x7f7878035100 con 0x7f7878038470 2023-10-06T03:33:23.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.035+0000 7f787effd700 1 -- 172.21.15.55:0/2409264902 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7870005780 con 0x7f7878038470 2023-10-06T03:33:23.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f787effd700 1 -- 172.21.15.55:0/2409264902 <== mon.1 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3302195756 0 0) 0x7f7870005780 con 0x7f7878038470 2023-10-06T03:33:23.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f787ffff700 1 -- 172.21.15.55:0/2409264902 >> v2:172.21.15.163:3300/0 conn(0x7f7878008360 msgr2=0x7f78780e3000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:33:23.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f787ffff700 1 --2- 172.21.15.55:0/2409264902 >> v2:172.21.15.163:3300/0 conn(0x7f7878008360 0x7f78780e3000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f787ffff700 1 -- 172.21.15.55:0/2409264902 >> v1:172.21.15.163:6789/0 conn(0x7f7878038470 legacy=0x7f7878007c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f787ffff700 1 -- 172.21.15.55:0/2409264902 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7878008c20 con 0x7f78780380a0 2023-10-06T03:33:23.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f1daf7fe700 1 --2- 172.21.15.55:0/3599227754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d98004810 0x7f1d98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f787ffff700 1 --2- 172.21.15.55:0/2409264902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78780380a0 0x7f78780e2ac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7880000eb0 tx=0x7f7880066260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe8b956901ddd01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f6f66d9d700 1 -- 172.21.15.55:0/3654872120 >> v2:172.21.15.163:3300/0 conn(0x7f6f68159840 msgr2=0x7f6f680b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:23.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f6f66d9d700 1 --2- 172.21.15.55:0/3654872120 >> v2:172.21.15.163:3300/0 conn(0x7f6f68159840 0x7f6f680b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.036+0000 7f6f66d9d700 1 -- 172.21.15.55:0/3654872120 >> v1:172.21.15.163:6789/0 conn(0x7f6f68159470 legacy=0x7f6f6815b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f1db499c700 1 --2- 172.21.15.55:0/3599227754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d980052e0 0x7f1d9800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f1daffff700 1 -- 172.21.15.55:0/3599227754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d980052e0 msgr2=0x7f1d9800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:23.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f6f66d9d700 1 -- 172.21.15.55:0/3654872120 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f680b4d40 con 0x7f6f680cd8b0 2023-10-06T03:33:23.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f1daffff700 1 --2- 172.21.15.55:0/3599227754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d980052e0 0x7f1d9800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f1daffff700 1 -- 172.21.15.55:0/3599227754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d98004810 msgr2=0x7f1d98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:23.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f1daffff700 1 --2- 172.21.15.55:0/3599227754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d98004810 0x7f1d98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f1daffff700 1 -- 172.21.15.55:0/3599227754 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db00b1a70 con 0x7f1d98003aa0 2023-10-06T03:33:23.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f1daf7fe700 1 --2- 172.21.15.55:0/3599227754 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d98004810 0x7f1d98004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:23.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f2a1279d700 1 -- 172.21.15.55:0/2874369127 shutdown_connections 2023-10-06T03:33:23.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f2a1279d700 1 --2- 172.21.15.55:0/2874369127 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a0c15a090 0x7f2a0c15a510 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f2a1279d700 1 --2- 172.21.15.55:0/2874369127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a0c0cf6c0 0x7f2a0c0b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f2a1279d700 1 -- 172.21.15.55:0/2874369127 >> 172.21.15.55:0/2874369127 conn(0x7f2a0c0d0560 msgr2=0x7f2a0c0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f14bd39b700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f149c066ff0 0x7f149c0694a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f14a8008ac0 tx=0x7f14a8008050 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7fd1e75ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e0155740 0x7fd1e0155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f10d6ffd700 1 -- 172.21.15.55:0/1211066135 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f10e402d3b0 con 0x7f10f0159720 2023-10-06T03:33:23.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f1db499c700 1 --2- 172.21.15.55:0/3599227754 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d980052e0 0x7f1d9800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:23.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f90a7fff700 1 -- 172.21.15.55:0/2734943656 wait complete. 2023-10-06T03:33:23.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7fd1e75ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e00aeec0 con 0x7fd1e00b1f40 2023-10-06T03:33:23.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f6f66d9d700 1 --2- 172.21.15.55:0/3654872120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f680cd8b0 0x7f6f680cf100 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009e80 tx=0x7f6f60009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f4ff09c753c0482 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7fd1e75ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e01567e0 con 0x7fd1e00b0480 2023-10-06T03:33:23.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7fd1e75ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e00b23b0 con 0x7fd1e0155740 2023-10-06T03:33:23.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.037+0000 7f1db499c700 1 -- 172.21.15.55:0/3599227754 reap_dead start 2023-10-06T03:33:23.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.038+0000 7fd1e5b9c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd1e00b0480 0x7fd1e0156c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:54158/0 (socket says 172.21.15.55:54158) 2023-10-06T03:33:23.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.038+0000 7fd1e5b9c700 1 -- 172.21.15.55:0/3924539964 learned_addr learned my addr 172.21.15.55:0/3924539964 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:23.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.038+0000 7fd1e539b700 1 --2- 172.21.15.55:0/3924539964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e0155740 0x7fd1e0155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.038+0000 7f1daffff700 1 --2- 172.21.15.55:0/3599227754 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d98003aa0 0x7f1d98003ed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1da000c2f0 tx=0x7f1da0002ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf486a9ed981005c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.038+0000 7fd1d7fff700 1 -- 172.21.15.55:0/3924539964 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 192185709 0 0) 0x7fd1e01567e0 con 0x7fd1e00b0480 2023-10-06T03:33:23.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.038+0000 7fd1d7fff700 1 -- 172.21.15.55:0/3924539964 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1cc003570 con 0x7fd1e00b0480 2023-10-06T03:33:23.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.038+0000 7f2a1279d700 1 -- 172.21.15.55:0/2874369127 shutdown_connections 2023-10-06T03:33:23.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.038+0000 7f2a1279d700 1 -- 172.21.15.55:0/2874369127 wait complete. 2023-10-06T03:33:23.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.038+0000 7fd1d7fff700 1 -- 172.21.15.55:0/3924539964 <== mon.1 v1:172.21.15.163:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3271660932 0 0) 0x7fd1cc003570 con 0x7fd1e00b0480 2023-10-06T03:33:23.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.038+0000 7fd1e539b700 1 -- 172.21.15.55:0/3924539964 >> v1:172.21.15.163:6789/0 conn(0x7fd1e00b0480 legacy=0x7fd1e0156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.038+0000 7fd1e539b700 1 -- 172.21.15.55:0/3924539964 >> v1:172.21.15.55:6789/0 conn(0x7fd1e00b1f40 legacy=0x7fd1e00afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.038+0000 7fd1e539b700 1 -- 172.21.15.55:0/3924539964 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e00cc2a0 con 0x7fd1e0155740 2023-10-06T03:33:23.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.039+0000 7fd1e539b700 1 --2- 172.21.15.55:0/3924539964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e0155740 0x7fd1e0155b10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009890 tx=0x7fd1dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae6b03ceade4854a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.039+0000 7f90a7fff700 1 Processor -- start 2023-10-06T03:33:23.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.039+0000 7f90a7fff700 1 -- start start 2023-10-06T03:33:23.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.039+0000 7f10d6ffd700 1 --2- 172.21.15.55:0/1211066135 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f10dc067080 0x7f10dc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.040+0000 7f10eed9d700 1 --2- 172.21.15.55:0/1211066135 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f10dc067080 0x7f10dc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.040+0000 7f90a7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90a80696c0 0x7f90a80c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.040+0000 7f10d6ffd700 1 -- 172.21.15.55:0/1211066135 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f10e4086450 con 0x7f10f0159720 2023-10-06T03:33:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.040+0000 7f90a7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90a815da80 0x7f90a80c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.040+0000 7f90a7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a815de50 0x7f90a80b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.040+0000 7f90a7fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a80c5a90 con 0x7f90a815da80 2023-10-06T03:33:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.040+0000 7f90a7fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a80c41f0 con 0x7f90a815de50 2023-10-06T03:33:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.040+0000 7f90a7fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a80c4e80 con 0x7f90a80696c0 2023-10-06T03:33:23.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.040+0000 7f90a67fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90a815da80 0x7f90a80c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.040+0000 7f90a67fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90a815da80 0x7f90a80c3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41212/0 (socket says 172.21.15.55:41212) 2023-10-06T03:33:23.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.040+0000 7f90a67fc700 1 -- 172.21.15.55:0/3773957045 learned_addr learned my addr 172.21.15.55:0/3773957045 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:23.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.040+0000 7f90a6ffd700 1 --2- 172.21.15.55:0/3773957045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90a80696c0 0x7f90a80c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.040+0000 7f90a77fe700 1 --2- 172.21.15.55:0/3773957045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a815de50 0x7f90a80b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.041+0000 7f2a1279d700 1 Processor -- start 2023-10-06T03:33:23.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.041+0000 7f90a67fc700 1 -- 172.21.15.55:0/3773957045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90a80696c0 msgr2=0x7f90a80c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.041+0000 7f90a67fc700 1 --2- 172.21.15.55:0/3773957045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90a80696c0 0x7f90a80c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.041+0000 7f90a67fc700 1 -- 172.21.15.55:0/3773957045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a815de50 msgr2=0x7f90a80b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.041+0000 7f90a67fc700 1 --2- 172.21.15.55:0/3773957045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a815de50 0x7f90a80b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.041+0000 7f90a67fc700 1 -- 172.21.15.55:0/3773957045 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a0009ae0 con 0x7f90a815da80 2023-10-06T03:33:23.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.041+0000 7f2a1279d700 1 -- start start 2023-10-06T03:33:23.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.041+0000 7f90a67fc700 1 --2- 172.21.15.55:0/3773957045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90a815da80 0x7f90a80c3cb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f90a806fe40 tx=0x7f90a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.041+0000 7fd1d7fff700 1 -- 172.21.15.55:0/3924539964 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0001470 con 0x7fd1e0155740 2023-10-06T03:33:23.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.041+0000 7f6f6559a700 1 -- 172.21.15.55:0/3654872120 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6000be60 con 0x7f6f680cd8b0 2023-10-06T03:33:23.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.041+0000 7f6f6559a700 1 -- 172.21.15.55:0/3654872120 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f60013410 con 0x7f6f680cd8b0 2023-10-06T03:33:23.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.041+0000 7f6f6559a700 1 -- 172.21.15.55:0/3654872120 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f600137d0 con 0x7f6f680cd8b0 2023-10-06T03:33:23.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.042+0000 7f14ad7fa700 1 -- 172.21.15.55:0/2003222636 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14b00d2280 con 0x7f14b00d5240 2023-10-06T03:33:23.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.042+0000 7fd1d7fff700 1 -- 172.21.15.55:0/3924539964 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc011420 con 0x7fd1e0155740 2023-10-06T03:33:23.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.042+0000 7fd1d7fff700 1 -- 172.21.15.55:0/3924539964 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0117a0 con 0x7fd1e0155740 2023-10-06T03:33:23.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.042+0000 7f6f6cfff700 1 -- 172.21.15.55:0/3654872120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f680cd8b0 msgr2=0x7f6f680cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.042+0000 7f6f6cfff700 1 --2- 172.21.15.55:0/3654872120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f680cd8b0 0x7f6f680cf100 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6f60009e80 tx=0x7f6f60009720 comp rx=0 tx=0).stop 2023-10-06T03:33:23.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.042+0000 7f6f6cfff700 1 -- 172.21.15.55:0/3654872120 shutdown_connections 2023-10-06T03:33:23.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.042+0000 7f6f6cfff700 1 --2- 172.21.15.55:0/3654872120 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f680cd8b0 0x7f6f680cf100 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.042+0000 7f6f6cfff700 1 --2- 172.21.15.55:0/3654872120 >> v2:172.21.15.163:3300/0 conn(0x7f6f68159840 0x7f6f680b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.042+0000 7f6f6cfff700 1 -- 172.21.15.55:0/3654872120 >> 172.21.15.55:0/3654872120 conn(0x7f6f680d0ff0 msgr2=0x7f6f680ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.042+0000 7f6f6cfff700 1 -- 172.21.15.55:0/3654872120 shutdown_connections 2023-10-06T03:33:23.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.042+0000 7f10eed9d700 1 --2- 172.21.15.55:0/1211066135 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f10dc067080 0x7f10dc069530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f10f0151dd0 tx=0x7f10e0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.042+0000 7f6f6cfff700 1 -- 172.21.15.55:0/3654872120 wait complete. 2023-10-06T03:33:23.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.043+0000 7f2a1279d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a0c0cf6c0 0x7f2a0c149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.043+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3924539964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e0155740 msgr2=0x7fd1e0155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.043+0000 7fd1e75ff700 1 --2- 172.21.15.55:0/3924539964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e0155740 0x7fd1e0155b10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009890 tx=0x7fd1dc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.043+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3924539964 shutdown_connections 2023-10-06T03:33:23.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.043+0000 7fd1e75ff700 1 --2- 172.21.15.55:0/3924539964 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e0155740 0x7fd1e0155b10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.043+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3924539964 >> 172.21.15.55:0/3924539964 conn(0x7fd1e006a320 msgr2=0x7fd1e00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.043+0000 7f2a1279d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a0c159cc0 0x7f2a0c148460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.043+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3924539964 shutdown_connections 2023-10-06T03:33:23.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.043+0000 7f2a1179b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a0c0cf6c0 0x7f2a0c149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.043+0000 7f2a1179b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a0c0cf6c0 0x7f2a0c149e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:48300/0 (socket says 172.21.15.55:48300) 2023-10-06T03:33:23.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.043+0000 7f2a1179b700 1 -- 172.21.15.55:0/2153137624 learned_addr learned my addr 172.21.15.55:0/2153137624 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:23.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.044+0000 7f2a10f9a700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a0c159cc0 0x7f2a0c148460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.044+0000 7f2a1279d700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a0c15a090 0x7f2a0c159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.044+0000 7f2a1279d700 1 -- 172.21.15.55:0/2153137624 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c149150 con 0x7f2a0c15a090 2023-10-06T03:33:23.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.044+0000 7f2a1279d700 1 -- 172.21.15.55:0/2153137624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c1542e0 con 0x7f2a0c0cf6c0 2023-10-06T03:33:23.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.044+0000 7f10d6ffd700 1 -- 172.21.15.55:0/1211066135 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10e4053540 con 0x7f10f0159720 2023-10-06T03:33:23.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.045+0000 7f2a1279d700 1 -- 172.21.15.55:0/2153137624 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c154530 con 0x7f2a0c159cc0 2023-10-06T03:33:23.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.045+0000 7f787effd700 1 -- 172.21.15.55:0/2409264902 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788008d070 con 0x7f78780380a0 2023-10-06T03:33:23.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.045+0000 7f2a11f9c700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a0c15a090 0x7f2a0c159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.045+0000 7f1daeffd700 1 -- 172.21.15.55:0/3599227754 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0007b60 con 0x7f1d98003aa0 2023-10-06T03:33:23.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.045+0000 7f787effd700 1 -- 172.21.15.55:0/2409264902 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788004c3c0 con 0x7f78780380a0 2023-10-06T03:33:23.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.045+0000 7f787effd700 1 -- 172.21.15.55:0/2409264902 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874001470 con 0x7f78780380a0 2023-10-06T03:33:23.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.045+0000 7f1daeffd700 1 -- 172.21.15.55:0/3599227754 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da000cc20 con 0x7f1d98003aa0 2023-10-06T03:33:23.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.045+0000 7f1daeffd700 1 -- 172.21.15.55:0/3599227754 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da0007b60 con 0x7f1d98003aa0 2023-10-06T03:33:23.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.045+0000 7f6f6cfff700 1 Processor -- start 2023-10-06T03:33:23.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.043+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3924539964 wait complete. 2023-10-06T03:33:23.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.045+0000 7fd1e75ff700 1 Processor -- start 2023-10-06T03:33:23.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.045+0000 7f6f6cfff700 1 -- start start 2023-10-06T03:33:23.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f78865ff700 1 -- 172.21.15.55:0/2409264902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78780380a0 msgr2=0x7f78780e2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f78865ff700 1 --2- 172.21.15.55:0/2409264902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78780380a0 0x7f78780e2ac0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7880000eb0 tx=0x7f7880066260 comp rx=0 tx=0).stop 2023-10-06T03:33:23.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f78865ff700 1 -- 172.21.15.55:0/2409264902 shutdown_connections 2023-10-06T03:33:23.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f78865ff700 1 --2- 172.21.15.55:0/2409264902 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78780380a0 0x7f78780e2ac0 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7880000eb0 tx=0x7f7880066260 comp rx=0 tx=0).stop 2023-10-06T03:33:23.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f78865ff700 1 --2- 172.21.15.55:0/2409264902 >> v2:172.21.15.163:3300/0 conn(0x7f7878008360 0x7f78780e3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f78865ff700 1 -- 172.21.15.55:0/2409264902 >> 172.21.15.55:0/2409264902 conn(0x7f7878052810 msgr2=0x7f7878052c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.045+0000 7f908effd700 1 -- 172.21.15.55:0/3773957045 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a00128d0 con 0x7f90a815da80 2023-10-06T03:33:23.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.045+0000 7f908effd700 1 -- 172.21.15.55:0/3773957045 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a0012a30 con 0x7f90a815da80 2023-10-06T03:33:23.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f908effd700 1 -- 172.21.15.55:0/3773957045 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0012df0 con 0x7f90a815da80 2023-10-06T03:33:23.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f2a1179b700 1 -- 172.21.15.55:0/2153137624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a0c159cc0 msgr2=0x7f2a0c148460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f2a1179b700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a0c159cc0 0x7f2a0c148460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f2a1179b700 1 -- 172.21.15.55:0/2153137624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a0c15a090 msgr2=0x7f2a0c159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f2a1179b700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a0c15a090 0x7f2a0c159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f2a1179b700 1 -- 172.21.15.55:0/2153137624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a04009a20 con 0x7f2a0c0cf6c0 2023-10-06T03:33:23.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f2a11f9c700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a0c15a090 0x7f2a0c159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:23.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f78865ff700 1 -- 172.21.15.55:0/2409264902 shutdown_connections 2023-10-06T03:33:23.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7f1db519d700 1 -- 172.21.15.55:0/3599227754 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d98003aa0 msgr2=0x7f1d98003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.047+0000 7f1db519d700 1 --2- 172.21.15.55:0/3599227754 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d98003aa0 0x7f1d98003ed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1da000c2f0 tx=0x7f1da0002ea0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.047+0000 7f78865ff700 1 -- 172.21.15.55:0/2409264902 wait complete. 2023-10-06T03:33:23.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.047+0000 7f1db519d700 1 -- 172.21.15.55:0/3599227754 shutdown_connections 2023-10-06T03:33:23.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.047+0000 7f1db519d700 1 --2- 172.21.15.55:0/3599227754 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d98003aa0 0x7f1d98003ed0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.047+0000 7f1db519d700 1 -- 172.21.15.55:0/3599227754 >> 172.21.15.55:0/3599227754 conn(0x7f1db00d0660 msgr2=0x7f1db00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.047+0000 7f2a1179b700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a0c0cf6c0 0x7f2a0c149e80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800c5e0 tx=0x7f2a08007f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.047+0000 7f1db519d700 1 -- 172.21.15.55:0/3599227754 shutdown_connections 2023-10-06T03:33:23.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.047+0000 7f90a7fff700 1 -- 172.21.15.55:0/3773957045 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a80c5160 con 0x7f90a815da80 2023-10-06T03:33:23.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.047+0000 7f1db519d700 1 -- 172.21.15.55:0/3599227754 wait complete. 2023-10-06T03:33:23.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.047+0000 7f2a017fa700 1 -- 172.21.15.55:0/2153137624 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08005a70 con 0x7f2a0c0cf6c0 2023-10-06T03:33:23.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.047+0000 7f2a017fa700 1 -- 172.21.15.55:0/2153137624 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a08005bd0 con 0x7f2a0c0cf6c0 2023-10-06T03:33:23.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.046+0000 7fd1e75ff700 1 -- start start 2023-10-06T03:33:23.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.047+0000 7fd1e75ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e00b0480 0x7fd1e00a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.047+0000 7f90a7fff700 1 -- 172.21.15.55:0/3773957045 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a80c5720 con 0x7f90a815da80 2023-10-06T03:33:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.049+0000 7f78865ff700 1 Processor -- start 2023-10-06T03:33:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.049+0000 7f78865ff700 1 -- start start 2023-10-06T03:33:23.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.050+0000 7f6f6cfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f680cd8b0 0x7f6f680c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.048+0000 7f2a017fa700 1 -- 172.21.15.55:0/2153137624 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a080070d0 con 0x7f2a0c0cf6c0 2023-10-06T03:33:23.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.048+0000 7f2a1279d700 1 -- 172.21.15.55:0/2153137624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0c154810 con 0x7f2a0c0cf6c0 2023-10-06T03:33:23.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.049+0000 7f2a1279d700 1 -- 172.21.15.55:0/2153137624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0c154dd0 con 0x7f2a0c0cf6c0 2023-10-06T03:33:23.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.050+0000 7f908cff9700 1 -- 172.21.15.55:0/3773957045 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a80c5c40 con 0x7f90a815da80 2023-10-06T03:33:23.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.050+0000 7f78865ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7878008360 0x7f787804d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.050+0000 7f6f6659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f680cd8b0 0x7f6f680c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.050+0000 7f6f6659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f680cd8b0 0x7f6f680c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41230/0 (socket says 172.21.15.55:41230) 2023-10-06T03:33:23.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.050+0000 7f6f6659c700 1 -- 172.21.15.55:0/1031145017 learned_addr learned my addr 172.21.15.55:0/1031145017 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:23.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.050+0000 7f78865ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7878038470 0x7f78780e2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.050+0000 7f6f6cfff700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f68159470 0x7f6f68142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f78865ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f787804f9d0 0x7f787800cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f6f6cfff700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f68159840 0x7f6f6815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f78865ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780e1c20 con 0x7f7878008360 2023-10-06T03:33:23.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f78865ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78780e1ec0 con 0x7f787804f9d0 2023-10-06T03:33:23.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f6f6cfff700 1 -- 172.21.15.55:0/1031145017 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f68158970 con 0x7f6f680cd8b0 2023-10-06T03:33:23.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f6f65d9b700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f68159470 0x7f6f68142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f6f6cfff700 1 -- 172.21.15.55:0/1031145017 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680c5a40 con 0x7f6f68159470 2023-10-06T03:33:23.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f787f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7878038470 0x7f78780e2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f787f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7878038470 0x7f78780e2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:57256/0 (socket says 172.21.15.55:57256) 2023-10-06T03:33:23.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f787f7fe700 1 -- 172.21.15.55:0/562810781 learned_addr learned my addr 172.21.15.55:0/562810781 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:23.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f6f6cfff700 1 -- 172.21.15.55:0/1031145017 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680c6410 con 0x7f6f68159840 2023-10-06T03:33:23.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f7884b9c700 1 --2- 172.21.15.55:0/562810781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f787804f9d0 0x7f787800cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f78865ff700 1 -- 172.21.15.55:0/562810781 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78780e21a0 con 0x7f7878038470 2023-10-06T03:33:23.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f6f66d9d700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f68159840 0x7f6f6815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.052+0000 7f2a017fa700 1 -- 172.21.15.55:0/2153137624 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a08007440 con 0x7f2a0c0cf6c0 2023-10-06T03:33:23.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.052+0000 7f6f66d9d700 1 -- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f68159470 msgr2=0x7f6f68142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.052+0000 7f6f66d9d700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f68159470 0x7f6f68142e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.052+0000 7f787f7fe700 1 -- 172.21.15.55:0/562810781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f787804f9d0 msgr2=0x7f787800cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.052+0000 7f6f66d9d700 1 -- 172.21.15.55:0/1031145017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f680cd8b0 msgr2=0x7f6f680c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.052+0000 7f6f66d9d700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f680cd8b0 0x7f6f680c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.052+0000 7f6f66d9d700 1 -- 172.21.15.55:0/1031145017 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f600099c0 con 0x7f6f68159840 2023-10-06T03:33:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.052+0000 7f6f66d9d700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f68159840 0x7f6f6815bbb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6f68065b70 tx=0x7f6f60017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7fd1e539b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e00b0480 0x7fd1e00a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.052+0000 7f787f7fe700 1 --2- 172.21.15.55:0/562810781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f787804f9d0 0x7f787800cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7fd1e75ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1e00b1f40 0x7fd1e00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7fd1e75ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1e0155740 0x7fd1e00a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7f6f567fc700 1 -- 172.21.15.55:0/1031145017 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60012a70 con 0x7f6f68159840 2023-10-06T03:33:23.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7f6f567fc700 1 -- 172.21.15.55:0/1031145017 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f60012c10 con 0x7f6f68159840 2023-10-06T03:33:23.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7f6f567fc700 1 -- 172.21.15.55:0/1031145017 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6001d440 con 0x7f6f68159840 2023-10-06T03:33:23.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7f2a1279d700 1 -- 172.21.15.55:0/2153137624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a0c154fe0 con 0x7f2a0c0cf6c0 2023-10-06T03:33:23.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7f6f6cfff700 1 -- 172.21.15.55:0/1031145017 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f68158690 con 0x7f6f68159840 2023-10-06T03:33:23.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.052+0000 7f787f7fe700 1 -- 172.21.15.55:0/562810781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7878008360 msgr2=0x7f787804d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.052+0000 7f787f7fe700 1 --2- 172.21.15.55:0/562810781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7878008360 0x7f787804d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.052+0000 7f787f7fe700 1 -- 172.21.15.55:0/562810781 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788004c9a0 con 0x7f7878038470 2023-10-06T03:33:23.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.054+0000 7f6f6cfff700 1 -- 172.21.15.55:0/1031145017 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f680c56d0 con 0x7f6f68159840 2023-10-06T03:33:23.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.054+0000 7f2a017fa700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f29f0066f80 0x7f29f0069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.054+0000 7f2a017fa700 1 -- 172.21.15.55:0/2153137624 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2a08083a90 con 0x7f2a0c0cf6c0 2023-10-06T03:33:23.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.054+0000 7f787f7fe700 1 --2- 172.21.15.55:0/562810781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7878038470 0x7f78780e2c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f787400c7f0 tx=0x7f78740019e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.054+0000 7f2a10f9a700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f29f0066f80 0x7f29f0069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7f908effd700 1 -- 172.21.15.55:0/3773957045 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90a0019030 con 0x7f90a815da80 2023-10-06T03:33:23.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.054+0000 7f908effd700 1 --2- 172.21.15.55:0/3773957045 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9094067000 0x7f90940694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.049+0000 7f1db519d700 1 Processor -- start 2023-10-06T03:33:23.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.049+0000 7f1db519d700 1 -- start start 2023-10-06T03:33:23.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f1db519d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1db0087ac0 0x7f1db00b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.051+0000 7f1db519d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1db0159ca0 0x7f1db017a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.055+0000 7f1daf7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1db0159ca0 0x7f1db017a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.055+0000 7f1daf7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1db0159ca0 0x7f1db017a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:57286/0 (socket says 172.21.15.55:57286) 2023-10-06T03:33:23.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.055+0000 7f1daf7fe700 1 -- 172.21.15.55:0/913192813 learned_addr learned my addr 172.21.15.55:0/913192813 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:23.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.055+0000 7f1db519d700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1db017ac10 0x7f1db0069360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.054+0000 7f786bfff700 1 -- 172.21.15.55:0/562810781 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874007360 con 0x7f7878038470 2023-10-06T03:33:23.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.055+0000 7f1daffff700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1db0087ac0 0x7f1db00b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.055+0000 7f786bfff700 1 -- 172.21.15.55:0/562810781 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78740074c0 con 0x7f7878038470 2023-10-06T03:33:23.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.055+0000 7f1db519d700 1 -- 172.21.15.55:0/913192813 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db015ad50 con 0x7f1db017ac10 2023-10-06T03:33:23.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.055+0000 7f1db519d700 1 -- 172.21.15.55:0/913192813 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db0087e90 con 0x7f1db0087ac0 2023-10-06T03:33:23.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.055+0000 7f1db519d700 1 -- 172.21.15.55:0/913192813 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db00b4080 con 0x7f1db0159ca0 2023-10-06T03:33:23.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.055+0000 7f6f6cfff700 1 -- 172.21.15.55:0/1031145017 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f68088be0 con 0x7f6f68159840 2023-10-06T03:33:23.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.055+0000 7f90a6ffd700 1 --2- 172.21.15.55:0/3773957045 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9094067000 0x7f90940694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.055+0000 7f786bfff700 1 -- 172.21.15.55:0/562810781 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874004830 con 0x7f7878038470 2023-10-06T03:33:23.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.055+0000 7f1db499c700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1db017ac10 0x7f1db0069360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.055+0000 7f78865ff700 1 -- 172.21.15.55:0/562810781 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78780e2480 con 0x7f7878038470 2023-10-06T03:33:23.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.056+0000 7f78865ff700 1 -- 172.21.15.55:0/562810781 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787804dcd0 con 0x7f7878038470 2023-10-06T03:33:23.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.056+0000 7f908effd700 1 -- 172.21.15.55:0/3773957045 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f90a0086120 con 0x7f90a815da80 2023-10-06T03:33:23.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.056+0000 7f1db499c700 1 -- 172.21.15.55:0/913192813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1db0159ca0 msgr2=0x7f1db017a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.056+0000 7f1db499c700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1db0159ca0 0x7f1db017a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.056+0000 7f1db499c700 1 -- 172.21.15.55:0/913192813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1db0087ac0 msgr2=0x7f1db00b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.056+0000 7f1db499c700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1db0087ac0 0x7f1db00b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.056+0000 7f1db499c700 1 -- 172.21.15.55:0/913192813 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db015a4b0 con 0x7f1db017ac10 2023-10-06T03:33:23.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.056+0000 7f90a6ffd700 1 --2- 172.21.15.55:0/3773957045 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9094067000 0x7f90940694b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9098009bf0 tx=0x7f90980091a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.057+0000 7f1db499c700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1db017ac10 0x7f1db0069360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1db015d600 tx=0x7f1da8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7fd1e75ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e00a5a00 con 0x7fd1e0155740 2023-10-06T03:33:23.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7fd1e75ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e00a67c0 con 0x7fd1e00b1f40 2023-10-06T03:33:23.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7fd1e75ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e016e8f0 con 0x7fd1e00b0480 2023-10-06T03:33:23.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7fd1e4b9a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1e00b1f40 0x7fd1e00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7fd1e4b9a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1e00b1f40 0x7fd1e00a6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:33410/0 (socket says 172.21.15.55:33410) 2023-10-06T03:33:23.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7fd1e4b9a700 1 -- 172.21.15.55:0/3551606306 learned_addr learned my addr 172.21.15.55:0/3551606306 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:23.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.053+0000 7fd1e5b9c700 1 --2- 172.21.15.55:0/3551606306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1e0155740 0x7fd1e00a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.054+0000 7fd1e4b9a700 1 -- 172.21.15.55:0/3551606306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e00b0480 msgr2=0x7fd1e00a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.054+0000 7fd1e4b9a700 1 --2- 172.21.15.55:0/3551606306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e00b0480 0x7fd1e00a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.054+0000 7fd1e4b9a700 1 -- 172.21.15.55:0/3551606306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1e0155740 msgr2=0x7fd1e00a6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.057+0000 7fd1e4b9a700 1 --2- 172.21.15.55:0/3551606306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1e0155740 0x7fd1e00a6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.057+0000 7fd1e4b9a700 1 -- 172.21.15.55:0/3551606306 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc009420 con 0x7fd1e00b1f40 2023-10-06T03:33:23.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.057+0000 7fd1e5b9c700 1 --2- 172.21.15.55:0/3551606306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1e0155740 0x7fd1e00a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:23.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.057+0000 7f6f567fc700 1 -- 172.21.15.55:0/1031145017 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f6001d870 con 0x7f6f68159840 2023-10-06T03:33:23.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.057+0000 7f2a10f9a700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f29f0066f80 0x7f29f0069430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f29fc006fd0 tx=0x7f29fc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.057+0000 7fd1e4b9a700 1 --2- 172.21.15.55:0/3551606306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1e00b1f40 0x7fd1e00a6280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800cb10 tx=0x7fd1d80017b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.058+0000 7f7869ffb700 1 -- 172.21.15.55:0/562810781 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7878055be0 con 0x7f7878038470 2023-10-06T03:33:23.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.059+0000 7f6f567fc700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6f48067000 0x7f6f480694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.059+0000 7f6f567fc700 1 -- 172.21.15.55:0/1031145017 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6f600869a0 con 0x7f6f68159840 2023-10-06T03:33:23.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.059+0000 7f6f6659c700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6f48067000 0x7f6f480694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.059+0000 7f786bfff700 1 -- 172.21.15.55:0/562810781 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7874004be0 con 0x7f7878038470 2023-10-06T03:33:23.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.060+0000 7fd1d57fa700 1 -- 172.21.15.55:0/3551606306 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d800a760 con 0x7fd1e00b1f40 2023-10-06T03:33:23.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.060+0000 7fd1d57fa700 1 -- 172.21.15.55:0/3551606306 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d800a8e0 con 0x7fd1e00b1f40 2023-10-06T03:33:23.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.060+0000 7fd1d57fa700 1 -- 172.21.15.55:0/3551606306 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d800aca0 con 0x7fd1e00b1f40 2023-10-06T03:33:23.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.060+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3551606306 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e016ebd0 con 0x7fd1e00b1f40 2023-10-06T03:33:23.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.060+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3551606306 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e016f190 con 0x7fd1e00b1f40 2023-10-06T03:33:23.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.060+0000 7f6f6659c700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6f48067000 0x7f6f480694b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6f58007fd0 tx=0x7f6f58007df0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.061+0000 7f2a017fa700 1 -- 172.21.15.55:0/2153137624 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a08050c20 con 0x7f2a0c0cf6c0 2023-10-06T03:33:23.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.061+0000 7f786bfff700 1 --2- 172.21.15.55:0/562810781 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7864067000 0x7f78640694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.061+0000 7f786bfff700 1 -- 172.21.15.55:0/562810781 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7874083d10 con 0x7f7878038470 2023-10-06T03:33:23.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.062+0000 7f787ffff700 1 --2- 172.21.15.55:0/562810781 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7864067000 0x7f78640694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.063+0000 7f908effd700 1 -- 172.21.15.55:0/3773957045 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90a80c5c40 con 0x7f90a815da80 2023-10-06T03:33:23.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.063+0000 7f787ffff700 1 --2- 172.21.15.55:0/562810781 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7864067000 0x7f78640694b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f787800ec40 tx=0x7f788008f000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.062+0000 7f1d97fff700 1 -- 172.21.15.55:0/913192813 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da800b740 con 0x7f1db017ac10 2023-10-06T03:33:23.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.063+0000 7f1d97fff700 1 -- 172.21.15.55:0/913192813 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da800b8a0 con 0x7f1db017ac10 2023-10-06T03:33:23.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.063+0000 7f1db519d700 1 -- 172.21.15.55:0/913192813 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db015a790 con 0x7f1db017ac10 2023-10-06T03:33:23.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.063+0000 7f1db519d700 1 -- 172.21.15.55:0/913192813 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db015a110 con 0x7f1db017ac10 2023-10-06T03:33:23.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.064+0000 7f1d97fff700 1 -- 172.21.15.55:0/913192813 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da800b740 con 0x7f1db017ac10 2023-10-06T03:33:23.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.064+0000 7f6f567fc700 1 -- 172.21.15.55:0/1031145017 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f60053a90 con 0x7f6f68159840 2023-10-06T03:33:23.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.062+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3551606306 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e004c3c0 con 0x7fd1e00b1f40 2023-10-06T03:33:23.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.065+0000 7f1db519d700 1 -- 172.21.15.55:0/913192813 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db004c3c0 con 0x7f1db017ac10 2023-10-06T03:33:23.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.064+0000 7fd1d57fa700 1 -- 172.21.15.55:0/3551606306 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd1d8003030 con 0x7fd1e00b1f40 2023-10-06T03:33:23.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.066+0000 7fd1d57fa700 1 --2- 172.21.15.55:0/3551606306 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd1c4067080 0x7fd1c4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.066+0000 7fd1d57fa700 1 -- 172.21.15.55:0/3551606306 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd1d80847c0 con 0x7fd1e00b1f40 2023-10-06T03:33:23.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.066+0000 7fd1e539b700 1 --2- 172.21.15.55:0/3551606306 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd1c4067080 0x7fd1c4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.067+0000 7fd1e539b700 1 --2- 172.21.15.55:0/3551606306 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd1c4067080 0x7fd1c4069530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009860 tx=0x7fd1dc016000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.067+0000 7f1d97fff700 1 -- 172.21.15.55:0/913192813 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1da800bc60 con 0x7f1db017ac10 2023-10-06T03:33:23.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.068+0000 7f1d97fff700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1d9c066f90 0x7f1d9c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:23.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.068+0000 7f1d97fff700 1 -- 172.21.15.55:0/913192813 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1da8084550 con 0x7f1db017ac10 2023-10-06T03:33:23.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.068+0000 7f786bfff700 1 -- 172.21.15.55:0/562810781 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f787404c5a0 con 0x7f7878038470 2023-10-06T03:33:23.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.068+0000 7f1daffff700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1d9c066f90 0x7f1d9c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:23.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.069+0000 7f1daffff700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1d9c066f90 0x7f1d9c069440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1db00896d0 tx=0x7f1da0017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:23.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.070+0000 7fd1d57fa700 1 -- 172.21.15.55:0/3551606306 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1d80518b0 con 0x7fd1e00b1f40 2023-10-06T03:33:23.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.075+0000 7f1d97fff700 1 -- 172.21.15.55:0/913192813 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1da80515c0 con 0x7f1db017ac10 2023-10-06T03:33:23.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.493+0000 7f14bf5ff700 1 -- 172.21.15.55:0/2003222636 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f14b00d2280 con 0x7f14b00d5240 2023-10-06T03:33:23.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.496+0000 7f14ad7fa700 1 -- 172.21.15.55:0/2003222636 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14b4014020 con 0x7f14b00d5240 2023-10-06T03:33:23.498 INFO:teuthology.orchestra.run.smithi055.stdout:55834574851 2023-10-06T03:33:23.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.502+0000 7f14a2ffd700 1 -- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f149c066ff0 msgr2=0x7f149c0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.502+0000 7f14a2ffd700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f149c066ff0 0x7f149c0694a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f14a8008ac0 tx=0x7f14a8008050 comp rx=0 tx=0).stop 2023-10-06T03:33:23.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.502+0000 7f14a2ffd700 1 -- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14b00d5240 msgr2=0x7f14b004d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.503+0000 7f14a2ffd700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14b00d5240 0x7f14b004d3c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f14b4013000 tx=0x7f14b4019000 comp rx=0 tx=0).stop 2023-10-06T03:33:23.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.503+0000 7f14a2ffd700 1 -- 172.21.15.55:0/2003222636 shutdown_connections 2023-10-06T03:33:23.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.503+0000 7f14a2ffd700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f149c066ff0 0x7f149c0694a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.503+0000 7f14a2ffd700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14b00d5240 0x7f14b004d3c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.503+0000 7f14a2ffd700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14b00340e0 0x7f14b00d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.503+0000 7f14a2ffd700 1 --2- 172.21.15.55:0/2003222636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14b004fe20 0x7f14b004ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.504+0000 7f14a2ffd700 1 -- 172.21.15.55:0/2003222636 >> 172.21.15.55:0/2003222636 conn(0x7f14b00518a0 msgr2=0x7f14b0053fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.504+0000 7f14a2ffd700 1 -- 172.21.15.55:0/2003222636 shutdown_connections 2023-10-06T03:33:23.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.504+0000 7f14a2ffd700 1 -- 172.21.15.55:0/2003222636 wait complete. 2023-10-06T03:33:23.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.509+0000 7f10f51ff700 1 -- 172.21.15.55:0/1211066135 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f10d0005560 con 0x7f10f0159720 2023-10-06T03:33:23.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.510+0000 7f10d6ffd700 1 -- 172.21.15.55:0/1211066135 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f10e4056b60 con 0x7f10f0159720 2023-10-06T03:33:23.511 INFO:teuthology.orchestra.run.smithi055.stdout:55834574853 2023-10-06T03:33:23.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.517+0000 7f2a1279d700 1 -- 172.21.15.55:0/2153137624 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2a0c04c3c0 con 0x7f2a0c0cf6c0 2023-10-06T03:33:23.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.517+0000 7f10d4ff9700 1 -- 172.21.15.55:0/1211066135 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f10dc067080 msgr2=0x7f10dc069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.518+0000 7f10d4ff9700 1 --2- 172.21.15.55:0/1211066135 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f10dc067080 0x7f10dc069530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f10f0151dd0 tx=0x7f10e0008040 comp rx=0 tx=0).stop 2023-10-06T03:33:23.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.518+0000 7f10d4ff9700 1 -- 172.21.15.55:0/1211066135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0159720 msgr2=0x7f10f0150200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.518+0000 7f10d4ff9700 1 --2- 172.21.15.55:0/1211066135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0159720 0x7f10f0150200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f10e40093f0 tx=0x7f10e4016000 comp rx=0 tx=0).stop 2023-10-06T03:33:23.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.518+0000 7f908cff9700 1 -- 172.21.15.55:0/3773957045 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f90a804c3c0 con 0x7f90a815da80 2023-10-06T03:33:23.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.518+0000 7f2a017fa700 1 -- 172.21.15.55:0/2153137624 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2a08017020 con 0x7f2a0c0cf6c0 2023-10-06T03:33:23.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.518+0000 7f10d4ff9700 1 -- 172.21.15.55:0/1211066135 shutdown_connections 2023-10-06T03:33:23.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.518+0000 7f10d4ff9700 1 --2- 172.21.15.55:0/1211066135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10f0159720 0x7f10f0150200 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.518+0000 7f10d4ff9700 1 --2- 172.21.15.55:0/1211066135 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f10dc067080 0x7f10dc069530 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.518+0000 7f10d4ff9700 1 --2- 172.21.15.55:0/1211066135 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10f00ab4e0 0x7f10f016a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.519+0000 7f10d4ff9700 1 --2- 172.21.15.55:0/1211066135 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10f00ad250 0x7f10f014f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.519+0000 7f10d4ff9700 1 -- 172.21.15.55:0/1211066135 >> 172.21.15.55:0/1211066135 conn(0x7f10f006a320 msgr2=0x7f10f01573f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.519+0000 7f908effd700 1 -- 172.21.15.55:0/3773957045 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f90a005ae60 con 0x7f90a815da80 2023-10-06T03:33:23.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.519+0000 7f10d4ff9700 1 -- 172.21.15.55:0/1211066135 shutdown_connections 2023-10-06T03:33:23.522 INFO:teuthology.orchestra.run.smithi055.stdout:55834574852 2023-10-06T03:33:23.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.519+0000 7f10d4ff9700 1 -- 172.21.15.55:0/1211066135 wait complete. 2023-10-06T03:33:23.523 INFO:teuthology.orchestra.run.smithi055.stdout:55834574852 2023-10-06T03:33:23.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.521+0000 7f6f6cfff700 1 -- 172.21.15.55:0/1031145017 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6f6804bb80 con 0x7f6f68159840 2023-10-06T03:33:23.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.522+0000 7f6f567fc700 1 -- 172.21.15.55:0/1031145017 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6f60021020 con 0x7f6f68159840 2023-10-06T03:33:23.523 INFO:teuthology.orchestra.run.smithi055.stdout:55834574852 2023-10-06T03:33:23.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.522+0000 7f908cff9700 1 -- 172.21.15.55:0/3773957045 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9094067000 msgr2=0x7f90940694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.523+0000 7f908cff9700 1 --2- 172.21.15.55:0/3773957045 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9094067000 0x7f90940694b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9098009bf0 tx=0x7f90980091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.523+0000 7f908cff9700 1 -- 172.21.15.55:0/3773957045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90a815da80 msgr2=0x7f90a80c3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.523+0000 7f908cff9700 1 --2- 172.21.15.55:0/3773957045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90a815da80 0x7f90a80c3cb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f90a806fe40 tx=0x7f90a0017000 comp rx=0 tx=0).stop 2023-10-06T03:33:23.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.523+0000 7f2a1279d700 1 -- 172.21.15.55:0/2153137624 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f29f0066f80 msgr2=0x7f29f0069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.523+0000 7f2a1279d700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f29f0066f80 0x7f29f0069430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f29fc006fd0 tx=0x7f29fc008040 comp rx=0 tx=0).stop 2023-10-06T03:33:23.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.524+0000 7f2a1279d700 1 -- 172.21.15.55:0/2153137624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a0c0cf6c0 msgr2=0x7f2a0c149e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.524+0000 7f908cff9700 1 -- 172.21.15.55:0/3773957045 shutdown_connections 2023-10-06T03:33:23.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.524+0000 7f908cff9700 1 --2- 172.21.15.55:0/3773957045 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90a815de50 0x7f90a80b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.524+0000 7f908cff9700 1 --2- 172.21.15.55:0/3773957045 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9094067000 0x7f90940694b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.524+0000 7f908cff9700 1 --2- 172.21.15.55:0/3773957045 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90a80696c0 0x7f90a80c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.524+0000 7f908cff9700 1 --2- 172.21.15.55:0/3773957045 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90a815da80 0x7f90a80c3cb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.524+0000 7f908cff9700 1 -- 172.21.15.55:0/3773957045 >> 172.21.15.55:0/3773957045 conn(0x7f90a80cc1f0 msgr2=0x7f90a80c8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.524+0000 7f908cff9700 1 -- 172.21.15.55:0/3773957045 shutdown_connections 2023-10-06T03:33:23.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.524+0000 7f908cff9700 1 -- 172.21.15.55:0/3773957045 wait complete. 2023-10-06T03:33:23.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.524+0000 7f2a1279d700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a0c0cf6c0 0x7f2a0c149e80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a0800c5e0 tx=0x7f2a08007f70 comp rx=0 tx=0).stop 2023-10-06T03:33:23.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.525+0000 7f2a1279d700 1 -- 172.21.15.55:0/2153137624 shutdown_connections 2023-10-06T03:33:23.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.525+0000 7f2a1279d700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a0c15a090 0x7f2a0c159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.525+0000 7f2a1279d700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a0c0cf6c0 0x7f2a0c149e80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.525+0000 7f2a1279d700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f29f0066f80 0x7f29f0069430 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.525+0000 7f2a1279d700 1 --2- 172.21.15.55:0/2153137624 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a0c159cc0 0x7f2a0c148460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.525+0000 7f2a1279d700 1 -- 172.21.15.55:0/2153137624 >> 172.21.15.55:0/2153137624 conn(0x7f2a0c0d0560 msgr2=0x7f2a0c0b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.526+0000 7f2a1279d700 1 -- 172.21.15.55:0/2153137624 shutdown_connections 2023-10-06T03:33:23.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.526+0000 7f2a1279d700 1 -- 172.21.15.55:0/2153137624 wait complete. 2023-10-06T03:33:23.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.527+0000 7f6f47fff700 1 -- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6f48067000 msgr2=0x7f6f480694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.527+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3551606306 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd1e00aebe0 con 0x7fd1e00b1f40 2023-10-06T03:33:23.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.527+0000 7f6f47fff700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6f48067000 0x7f6f480694b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6f58007fd0 tx=0x7f6f58007df0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.527+0000 7f6f47fff700 1 -- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f68159840 msgr2=0x7f6f6815bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.527+0000 7f6f47fff700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f68159840 0x7f6f6815bbb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6f68065b70 tx=0x7f6f60017000 comp rx=0 tx=0).stop 2023-10-06T03:33:23.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.528+0000 7fd1d57fa700 1 -- 172.21.15.55:0/3551606306 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd1d8054ed0 con 0x7fd1e00b1f40 2023-10-06T03:33:23.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.528+0000 7f6f47fff700 1 -- 172.21.15.55:0/1031145017 shutdown_connections 2023-10-06T03:33:23.531 INFO:teuthology.orchestra.run.smithi055.stdout:55834574852 2023-10-06T03:33:23.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.528+0000 7f6f47fff700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6f48067000 0x7f6f480694b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.528+0000 7f6f47fff700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f68159840 0x7f6f6815bbb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.528+0000 7f6f47fff700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f680cd8b0 0x7f6f680c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.528+0000 7f6f47fff700 1 --2- 172.21.15.55:0/1031145017 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f68159470 0x7f6f68142e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.528+0000 7f6f47fff700 1 -- 172.21.15.55:0/1031145017 >> 172.21.15.55:0/1031145017 conn(0x7f6f680d0ff0 msgr2=0x7f6f680b5710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.528+0000 7f6f47fff700 1 -- 172.21.15.55:0/1031145017 shutdown_connections 2023-10-06T03:33:23.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.529+0000 7f6f47fff700 1 -- 172.21.15.55:0/1031145017 wait complete. 2023-10-06T03:33:23.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.533+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3551606306 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd1c4067080 msgr2=0x7fd1c4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.533+0000 7fd1e75ff700 1 --2- 172.21.15.55:0/3551606306 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd1c4067080 0x7fd1c4069530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc009860 tx=0x7fd1dc016000 comp rx=0 tx=0).stop 2023-10-06T03:33:23.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.533+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3551606306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1e00b1f40 msgr2=0x7fd1e00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.533+0000 7fd1e75ff700 1 --2- 172.21.15.55:0/3551606306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1e00b1f40 0x7fd1e00a6280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800cb10 tx=0x7fd1d80017b0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.534+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3551606306 shutdown_connections 2023-10-06T03:33:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.534+0000 7fd1e75ff700 1 --2- 172.21.15.55:0/3551606306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1e0155740 0x7fd1e00a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.534+0000 7fd1e75ff700 1 --2- 172.21.15.55:0/3551606306 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd1c4067080 0x7fd1c4069530 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.534+0000 7fd1e75ff700 1 --2- 172.21.15.55:0/3551606306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1e00b0480 0x7fd1e00a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.534+0000 7fd1e75ff700 1 --2- 172.21.15.55:0/3551606306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1e00b1f40 0x7fd1e00a6280 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.534+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3551606306 >> 172.21.15.55:0/3551606306 conn(0x7fd1e006a320 msgr2=0x7fd1e0157cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.534+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3551606306 shutdown_connections 2023-10-06T03:33:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.535+0000 7fd1e75ff700 1 -- 172.21.15.55:0/3551606306 wait complete. 2023-10-06T03:33:23.537 INFO:teuthology.orchestra.run.smithi055.stdout:55834574851 2023-10-06T03:33:23.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2023-10-06T03:33:23.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.534+0000 7f7869ffb700 1 -- 172.21.15.55:0/562810781 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f78780057e0 con 0x7f7878038470 2023-10-06T03:33:23.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.535+0000 7f786bfff700 1 -- 172.21.15.55:0/562810781 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7874018090 con 0x7f7878038470 2023-10-06T03:33:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.539+0000 7f7869ffb700 1 -- 172.21.15.55:0/562810781 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7864067000 msgr2=0x7f78640694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.539+0000 7f7869ffb700 1 --2- 172.21.15.55:0/562810781 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7864067000 0x7f78640694b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f787800ec40 tx=0x7f788008f000 comp rx=0 tx=0).stop 2023-10-06T03:33:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.540+0000 7f7869ffb700 1 -- 172.21.15.55:0/562810781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7878038470 msgr2=0x7f78780e2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.540+0000 7f7869ffb700 1 --2- 172.21.15.55:0/562810781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7878038470 0x7f78780e2c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f787400c7f0 tx=0x7f78740019e0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.541+0000 7f1db519d700 1 -- 172.21.15.55:0/913192813 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1db00b3e50 con 0x7f1db017ac10 2023-10-06T03:33:23.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.541+0000 7f7869ffb700 1 -- 172.21.15.55:0/562810781 shutdown_connections 2023-10-06T03:33:23.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.541+0000 7f7869ffb700 1 --2- 172.21.15.55:0/562810781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f787804f9d0 0x7f787800cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.541+0000 7f7869ffb700 1 --2- 172.21.15.55:0/562810781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7878008360 0x7f787804d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.541+0000 7f7869ffb700 1 --2- 172.21.15.55:0/562810781 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7864067000 0x7f78640694b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.541+0000 7f7869ffb700 1 --2- 172.21.15.55:0/562810781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7878038470 0x7f78780e2c60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.541+0000 7f7869ffb700 1 -- 172.21.15.55:0/562810781 >> 172.21.15.55:0/562810781 conn(0x7f7878052810 msgr2=0x7f787804bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.542+0000 7f1d97fff700 1 -- 172.21.15.55:0/913192813 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1da80515c0 con 0x7f1db017ac10 2023-10-06T03:33:23.544 INFO:teuthology.orchestra.run.smithi055.stdout:55834574852 2023-10-06T03:33:23.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.543+0000 7f7869ffb700 1 -- 172.21.15.55:0/562810781 shutdown_connections 2023-10-06T03:33:23.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.543+0000 7f7869ffb700 1 -- 172.21.15.55:0/562810781 wait complete. 2023-10-06T03:33:23.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.547+0000 7f1db519d700 1 -- 172.21.15.55:0/913192813 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1d9c066f90 msgr2=0x7f1d9c069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.547+0000 7f1db519d700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1d9c066f90 0x7f1d9c069440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1db00896d0 tx=0x7f1da0017000 comp rx=0 tx=0).stop 2023-10-06T03:33:23.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.547+0000 7f1db519d700 1 -- 172.21.15.55:0/913192813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1db017ac10 msgr2=0x7f1db0069360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:23.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.547+0000 7f1db519d700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1db017ac10 0x7f1db0069360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1db015d600 tx=0x7f1da8009040 comp rx=0 tx=0).stop 2023-10-06T03:33:23.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.547+0000 7f1db519d700 1 -- 172.21.15.55:0/913192813 shutdown_connections 2023-10-06T03:33:23.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.547+0000 7f1db519d700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1db017ac10 0x7f1db0069360 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.547+0000 7f1db519d700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1db0087ac0 0x7f1db00b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.547+0000 7f1db519d700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1d9c066f90 0x7f1d9c069440 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.547+0000 7f1db519d700 1 --2- 172.21.15.55:0/913192813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1db0159ca0 0x7f1db017a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:23.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.547+0000 7f1db519d700 1 -- 172.21.15.55:0/913192813 >> 172.21.15.55:0/913192813 conn(0x7f1db00d0660 msgr2=0x7f1db00cc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:23.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.547+0000 7f1db519d700 1 -- 172.21.15.55:0/913192813 shutdown_connections 2023-10-06T03:33:23.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:23.548+0000 7f1db519d700 1 -- 172.21.15.55:0/913192813 wait complete. 2023-10-06T03:33:23.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-10-06T03:33:23.557 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2023-10-06T03:33:23.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2023-10-06T03:33:23.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2023-10-06T03:33:23.568 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2023-10-06T03:33:23.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574851 for osd.7 2023-10-06T03:33:23.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2023-10-06T03:33:24.538 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T03:33:24.556 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T03:33:24.557 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T03:33:24.564 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T03:33:24.565 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T03:33:24.569 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T03:33:24.574 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T03:33:24.575 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T03:33:24.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.845+0000 7fecc759e700 1 Processor -- start 2023-10-06T03:33:24.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.845+0000 7fecc759e700 1 -- start start 2023-10-06T03:33:24.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.847+0000 7fecc759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fecc80c77e0 0x7fecc81607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.847+0000 7fecc759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc806dd20 con 0x7fecc8156af0 2023-10-06T03:33:24.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.847+0000 7fecc759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc806e000 con 0x7fecc8155720 2023-10-06T03:33:24.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.847+0000 7fecc759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc806e2e0 con 0x7fecc80c77e0 2023-10-06T03:33:24.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.848+0000 7fecc6d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fecc80c77e0 0x7fecc81607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.849+0000 7fecc6d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fecc80c77e0 0x7fecc81607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:33438/0 (socket says 172.21.15.55:33438) 2023-10-06T03:33:24.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.849+0000 7fecc6d9d700 1 -- 172.21.15.55:0/2765297732 learned_addr learned my addr 172.21.15.55:0/2765297732 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:24.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.850+0000 7fecc6d9d700 1 -- 172.21.15.55:0/2765297732 >> v1:172.21.15.163:6789/0 conn(0x7fecc8155720 legacy=0x7fecc806f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:24.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.850+0000 7fecc6d9d700 1 -- 172.21.15.55:0/2765297732 >> v1:172.21.15.55:6789/0 conn(0x7fecc8156af0 legacy=0x7fecc80c70d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T03:33:24.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.850+0000 7fecc6d9d700 1 -- 172.21.15.55:0/2765297732 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc806e5c0 con 0x7fecc80c77e0 2023-10-06T03:33:24.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.851+0000 7fecc6d9d700 1 --2- 172.21.15.55:0/2765297732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc80c77e0 0x7fecc81607e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fecb80089b0 tx=0x7fecb80082d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcc6ea219b5286c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:24.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.851+0000 7fecc559a700 1 -- 172.21.15.55:0/2765297732 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8010070 con 0x7fecc80c77e0 2023-10-06T03:33:24.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.852+0000 7fecc559a700 1 -- 172.21.15.55:0/2765297732 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb800dab0 con 0x7fecc80c77e0 2023-10-06T03:33:24.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.853+0000 7fecc759e700 1 -- 172.21.15.55:0/2765297732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc80c77e0 msgr2=0x7fecc81607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.853+0000 7fecc759e700 1 --2- 172.21.15.55:0/2765297732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc80c77e0 0x7fecc81607e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fecb80089b0 tx=0x7fecb80082d0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.853+0000 7fecc759e700 1 -- 172.21.15.55:0/2765297732 shutdown_connections 2023-10-06T03:33:24.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.853+0000 7fecc759e700 1 --2- 172.21.15.55:0/2765297732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc80c77e0 0x7fecc81607e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.853+0000 7fecc759e700 1 -- 172.21.15.55:0/2765297732 >> 172.21.15.55:0/2765297732 conn(0x7fecc80cc1f0 msgr2=0x7fecc80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:24.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.854+0000 7fecc759e700 1 -- 172.21.15.55:0/2765297732 shutdown_connections 2023-10-06T03:33:24.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.854+0000 7fecc759e700 1 -- 172.21.15.55:0/2765297732 wait complete. 2023-10-06T03:33:24.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.857+0000 7fecc759e700 1 Processor -- start 2023-10-06T03:33:24.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.857+0000 7fecc759e700 1 -- start start 2023-10-06T03:33:24.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.859+0000 7fecc759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecc8155720 0x7fecc8159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.860+0000 7fecc659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecc8155720 0x7fecc8159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.860+0000 7fecc659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecc8155720 0x7fecc8159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59146/0 (socket says 172.21.15.55:59146) 2023-10-06T03:33:24.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.860+0000 7fecc659c700 1 -- 172.21.15.55:0/2255465032 learned_addr learned my addr 172.21.15.55:0/2255465032 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:24.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.860+0000 7fecc759e700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecc8156af0 0x7fecc80c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.861+0000 7fecc759e700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc8154a80 0x7fecc80c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.861+0000 7fecc5d9b700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecc8156af0 0x7fecc80c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.861+0000 7fecc759e700 1 -- 172.21.15.55:0/2255465032 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc80c4670 con 0x7fecc8155720 2023-10-06T03:33:24.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.862+0000 7fecc6d9d700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc8154a80 0x7fecc80c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.862+0000 7fecc759e700 1 -- 172.21.15.55:0/2255465032 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc80c37e0 con 0x7fecc8154a80 2023-10-06T03:33:24.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.862+0000 7fecc759e700 1 -- 172.21.15.55:0/2255465032 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc80c3a60 con 0x7fecc8156af0 2023-10-06T03:33:24.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.863+0000 7fecc659c700 1 -- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecc8156af0 msgr2=0x7fecc80c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.863+0000 7fecc659c700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecc8156af0 0x7fecc80c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.863+0000 7fecc659c700 1 -- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc8154a80 msgr2=0x7fecc80c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.863+0000 7fecc659c700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc8154a80 0x7fecc80c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.863+0000 7fecc659c700 1 -- 172.21.15.55:0/2255465032 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb8008570 con 0x7fecc8155720 2023-10-06T03:33:24.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.863+0000 7fecc659c700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecc8155720 0x7fecc8159540 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c890 tx=0x7fecbc007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:24.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.865+0000 7fecc5d9b700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecc8156af0 0x7fecc80c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:24.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.865+0000 7fecc6d9d700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc8154a80 0x7fecc80c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:24.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.865+0000 7fecb67fc700 1 -- 172.21.15.55:0/2255465032 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc012420 con 0x7fecc8155720 2023-10-06T03:33:24.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.866+0000 7fecc759e700 1 -- 172.21.15.55:0/2255465032 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc80c3d40 con 0x7fecc8155720 2023-10-06T03:33:24.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.866+0000 7fecc759e700 1 -- 172.21.15.55:0/2255465032 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc80c4300 con 0x7fecc8155720 2023-10-06T03:33:24.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.870+0000 7fecb67fc700 1 -- 172.21.15.55:0/2255465032 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc012580 con 0x7fecc8155720 2023-10-06T03:33:24.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.870+0000 7fecb67fc700 1 -- 172.21.15.55:0/2255465032 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc012940 con 0x7fecc8155720 2023-10-06T03:33:24.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.872+0000 7fecb67fc700 1 -- 172.21.15.55:0/2255465032 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fecbc012d30 con 0x7fecc8155720 2023-10-06T03:33:24.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.873+0000 7fecb67fc700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7feca8067100 0x7feca80695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.874+0000 7fecb67fc700 1 -- 172.21.15.55:0/2255465032 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fecbc084850 con 0x7fecc8155720 2023-10-06T03:33:24.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.874+0000 7fecc5d9b700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7feca8067100 0x7feca80695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.874+0000 7fecc759e700 1 -- 172.21.15.55:0/2255465032 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecac0052b0 con 0x7fecc8155720 2023-10-06T03:33:24.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.885+0000 7fecc5d9b700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7feca8067100 0x7feca80695b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fecc817a8b0 tx=0x7fecc00074d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:24.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.885+0000 7fecb67fc700 1 -- 172.21.15.55:0/2255465032 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecbc016070 con 0x7fecc8155720 2023-10-06T03:33:24.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.922+0000 7f8275d9d700 1 Processor -- start 2023-10-06T03:33:24.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.922+0000 7f8275d9d700 1 -- start start 2023-10-06T03:33:24.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.925+0000 7f8275d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f82680e3f80 0x7f82680e4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.925+0000 7f8275d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8268006fb0 con 0x7f8268006a40 2023-10-06T03:33:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.925+0000 7f8275d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8268007290 con 0x7f82680ea2b0 2023-10-06T03:33:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.925+0000 7f8275d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8268007570 con 0x7f82680e3f80 2023-10-06T03:33:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.926+0000 7f826ffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8268006a40 0x7f826804ff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43936/0 (socket says 172.21.15.55:43936) 2023-10-06T03:33:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.926+0000 7f826ffff700 1 -- 172.21.15.55:0/1363320526 learned_addr learned my addr 172.21.15.55:0/1363320526 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.927+0000 7f8274d9b700 1 --2- 172.21.15.55:0/1363320526 >> v2:172.21.15.163:3300/0 conn(0x7f82680e3f80 0x7f82680e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.924+0000 7f48c1b9d700 1 Processor -- start 2023-10-06T03:33:24.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.925+0000 7f48c1b9d700 1 -- start start 2023-10-06T03:33:24.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.927+0000 7f48c1b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f48bc0af220 0x7f48bc0af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.927+0000 7f826f7fe700 1 -- 172.21.15.55:0/1363320526 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093028769 0 0) 0x7f8268007290 con 0x7f82680ea2b0 2023-10-06T03:33:24.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.927+0000 7f826f7fe700 1 -- 172.21.15.55:0/1363320526 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8260003570 con 0x7f82680ea2b0 2023-10-06T03:33:24.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.927+0000 7f48c1b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48bc0afeb0 con 0x7f48bc087ad0 2023-10-06T03:33:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.927+0000 7f48c1b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48bc0ad010 con 0x7f48bc087660 2023-10-06T03:33:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.927+0000 7f48c1b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48bc0af5f0 con 0x7f48bc0af220 2023-10-06T03:33:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.928+0000 7f48bbfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f48bc087660 0x7f48bc0b15d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:59492/0 (socket says 172.21.15.55:59492) 2023-10-06T03:33:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.928+0000 7f48bbfff700 1 -- 172.21.15.55:0/4267788832 learned_addr learned my addr 172.21.15.55:0/4267788832 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.928+0000 7f48c0b9b700 1 --2- 172.21.15.55:0/4267788832 >> v2:172.21.15.163:3300/0 conn(0x7f48bc0af220 0x7f48bc0af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.928+0000 7f826f7fe700 1 -- 172.21.15.55:0/1363320526 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184845548 0 0) 0x7f8268006fb0 con 0x7f8268006a40 2023-10-06T03:33:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.928+0000 7f826f7fe700 1 -- 172.21.15.55:0/1363320526 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8268007290 con 0x7f8268006a40 2023-10-06T03:33:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.929+0000 7f8274d9b700 1 -- 172.21.15.55:0/1363320526 >> v1:172.21.15.163:6789/0 conn(0x7f82680ea2b0 legacy=0x7f8268053630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.929+0000 7f8274d9b700 1 -- 172.21.15.55:0/1363320526 >> v1:172.21.15.55:6789/0 conn(0x7f8268006a40 legacy=0x7f826804ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.929+0000 7f8274d9b700 1 -- 172.21.15.55:0/1363320526 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8268007850 con 0x7f82680e3f80 2023-10-06T03:33:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.929+0000 7f8274d9b700 1 --2- 172.21.15.55:0/1363320526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82680e3f80 0x7f82680e4350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82640098e0 tx=0x7f82640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43421df2c6710b2f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.930+0000 7f826f7fe700 1 -- 172.21.15.55:0/1363320526 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8264011630 con 0x7f82680e3f80 2023-10-06T03:33:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.930+0000 7f826f7fe700 1 -- 172.21.15.55:0/1363320526 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8264011790 con 0x7f82680e3f80 2023-10-06T03:33:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.930+0000 7f826f7fe700 1 -- 172.21.15.55:0/1363320526 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8264011b30 con 0x7f82680e3f80 2023-10-06T03:33:24.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.930+0000 7f09d37ff700 1 Processor -- start 2023-10-06T03:33:24.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.929+0000 7f48c0b9b700 1 -- 172.21.15.55:0/4267788832 >> v1:172.21.15.163:6789/0 conn(0x7f48bc087660 legacy=0x7f48bc0b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.929+0000 7f48c0b9b700 1 -- 172.21.15.55:0/4267788832 >> v1:172.21.15.55:6789/0 conn(0x7f48bc087ad0 legacy=0x7f48bc0ad4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:24.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.929+0000 7f48c0b9b700 1 -- 172.21.15.55:0/4267788832 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc0ac9c0 con 0x7f48bc0af220 2023-10-06T03:33:24.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.929+0000 7f48c0b9b700 1 --2- 172.21.15.55:0/4267788832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48bc0af220 0x7f48bc0af970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009850 tx=0x7f48ac009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ff9470c0d6bbaa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:24.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.930+0000 7f48bb7fe700 1 -- 172.21.15.55:0/4267788832 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac011630 con 0x7f48bc0af220 2023-10-06T03:33:24.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.930+0000 7f48bb7fe700 1 -- 172.21.15.55:0/4267788832 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac011790 con 0x7f48bc0af220 2023-10-06T03:33:24.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.930+0000 7f48bb7fe700 1 -- 172.21.15.55:0/4267788832 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac011b30 con 0x7f48bc0af220 2023-10-06T03:33:24.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.931+0000 7f8275d9d700 1 -- 172.21.15.55:0/1363320526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82680e3f80 msgr2=0x7f82680e4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.931+0000 7f8275d9d700 1 --2- 172.21.15.55:0/1363320526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82680e3f80 0x7f82680e4350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f82640098e0 tx=0x7f82640091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.931+0000 7f48c1b9d700 1 -- 172.21.15.55:0/4267788832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48bc0af220 msgr2=0x7f48bc0af970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.931+0000 7f48c1b9d700 1 --2- 172.21.15.55:0/4267788832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48bc0af220 0x7f48bc0af970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009850 tx=0x7f48ac009120 comp rx=0 tx=0).stop 2023-10-06T03:33:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.931+0000 7f48c1b9d700 1 -- 172.21.15.55:0/4267788832 shutdown_connections 2023-10-06T03:33:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.931+0000 7f48c1b9d700 1 --2- 172.21.15.55:0/4267788832 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48bc0af220 0x7f48bc0af970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.931+0000 7f48c1b9d700 1 -- 172.21.15.55:0/4267788832 >> 172.21.15.55:0/4267788832 conn(0x7f48bc0ccc80 msgr2=0x7f48bc068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.931+0000 7f48c1b9d700 1 -- 172.21.15.55:0/4267788832 shutdown_connections 2023-10-06T03:33:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.931+0000 7f48c1b9d700 1 -- 172.21.15.55:0/4267788832 wait complete. 2023-10-06T03:33:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.931+0000 7f8275d9d700 1 -- 172.21.15.55:0/1363320526 shutdown_connections 2023-10-06T03:33:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.931+0000 7f8275d9d700 1 --2- 172.21.15.55:0/1363320526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82680e3f80 0x7f82680e4350 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.931+0000 7f09d37ff700 1 -- start start 2023-10-06T03:33:24.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.932+0000 7f8275d9d700 1 -- 172.21.15.55:0/1363320526 >> 172.21.15.55:0/1363320526 conn(0x7f8268054da0 msgr2=0x7f82680551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:24.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.932+0000 7f8275d9d700 1 -- 172.21.15.55:0/1363320526 shutdown_connections 2023-10-06T03:33:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.933+0000 7f8275d9d700 1 -- 172.21.15.55:0/1363320526 wait complete. 2023-10-06T03:33:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.933+0000 7f48c1b9d700 1 Processor -- start 2023-10-06T03:33:24.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.933+0000 7f09d37ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f09cc1599e0 0x7f09cc069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.933+0000 7f09d37ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f09cc159db0 0x7f09cc069080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.933+0000 7f09d37ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09cc06a2b0 0x7f09cc06a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.933+0000 7f09d37ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09cc0c96a0 con 0x7f09cc1599e0 2023-10-06T03:33:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.933+0000 7f09d37ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09cc0cb9e0 con 0x7f09cc159db0 2023-10-06T03:33:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.933+0000 7f09d37ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0cab60 con 0x7f09cc06a2b0 2023-10-06T03:33:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.934+0000 7f09d0d9a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f09cc159db0 0x7f09cc069080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.934+0000 7f09d0d9a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f09cc159db0 0x7f09cc069080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:33464/0 (socket says 172.21.15.55:33464) 2023-10-06T03:33:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.934+0000 7f09d0d9a700 1 -- 172.21.15.55:0/545502222 learned_addr learned my addr 172.21.15.55:0/545502222 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:24.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.934+0000 7f09d159b700 1 --2- 172.21.15.55:0/545502222 >> v2:172.21.15.55:3300/0 conn(0x7f09cc1599e0 0x7f09cc069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.935+0000 7f48c1b9d700 1 -- start start 2023-10-06T03:33:24.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.935+0000 7f09d159b700 1 -- 172.21.15.55:0/545502222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09cc06a2b0 msgr2=0x7f09cc06a6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T03:33:24.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.935+0000 7f09d159b700 1 --2- 172.21.15.55:0/545502222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09cc06a2b0 0x7f09cc06a6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.935+0000 7f09d159b700 1 -- 172.21.15.55:0/545502222 >> v2:172.21.15.163:3300/0 conn(0x7f09cc159db0 msgr2=0x7f09cc069080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:24.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.935+0000 7f09d159b700 1 --2- 172.21.15.55:0/545502222 >> v2:172.21.15.163:3300/0 conn(0x7f09cc159db0 0x7f09cc069080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.935+0000 7f09d159b700 1 -- 172.21.15.55:0/545502222 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc0cb420 con 0x7f09cc1599e0 2023-10-06T03:33:24.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.936+0000 7f09d159b700 1 --2- 172.21.15.55:0/545502222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09cc1599e0 0x7f09cc069ac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0c73c0 tx=0x7f09c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82be40b326497ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:24.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.937+0000 7f09cbfff700 1 -- 172.21.15.55:0/545502222 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000b650 con 0x7f09cc1599e0 2023-10-06T03:33:24.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.937+0000 7f09cbfff700 1 -- 172.21.15.55:0/545502222 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c000b7b0 con 0x7f09cc1599e0 2023-10-06T03:33:24.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.938+0000 7f09d37ff700 1 -- 172.21.15.55:0/545502222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09cc1599e0 msgr2=0x7f09cc069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.938+0000 7f09cbfff700 1 -- 172.21.15.55:0/545502222 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000b650 con 0x7f09cc1599e0 2023-10-06T03:33:24.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.938+0000 7f09d37ff700 1 --2- 172.21.15.55:0/545502222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09cc1599e0 0x7f09cc069ac0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f09cc0c73c0 tx=0x7f09c00091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.935+0000 7f8275d9d700 1 Processor -- start 2023-10-06T03:33:24.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.936+0000 7f8275d9d700 1 -- start start 2023-10-06T03:33:24.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.937+0000 7f8275d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8268006a40 0x7f8268103980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.938+0000 7f8274d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8268006a40 0x7f8268103980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.938+0000 7f8274d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8268006a40 0x7f8268103980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59158/0 (socket says 172.21.15.55:59158) 2023-10-06T03:33:24.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.938+0000 7f8274d9b700 1 -- 172.21.15.55:0/3275568763 learned_addr learned my addr 172.21.15.55:0/3275568763 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:24.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.938+0000 7f8275d9d700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82680e3f80 0x7f82680ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.938+0000 7f8275d9d700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82680ea2b0 0x7f82680fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.938+0000 7f8275d9d700 1 -- 172.21.15.55:0/3275568763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8268103ec0 con 0x7f8268006a40 2023-10-06T03:33:24.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.938+0000 7f8275d9d700 1 -- 172.21.15.55:0/3275568763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82680deef0 con 0x7f82680e3f80 2023-10-06T03:33:24.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.938+0000 7f8275d9d700 1 -- 172.21.15.55:0/3275568763 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82680ffeb0 con 0x7f82680ea2b0 2023-10-06T03:33:24.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.937+0000 7f48c1b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48bc087660 0x7f48bc0cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.937+0000 7f48c1b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48bc087ad0 0x7f48bc166710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.937+0000 7f48c1b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48bc0af220 0x7f48bc0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.937+0000 7f48c1b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48bc0cb2d0 con 0x7f48bc087ad0 2023-10-06T03:33:24.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.937+0000 7f48c1b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48bc167c20 con 0x7f48bc0af220 2023-10-06T03:33:24.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.937+0000 7f48c1b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48bc167010 con 0x7f48bc087660 2023-10-06T03:33:24.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f827559c700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82680ea2b0 0x7f82680fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f826ffff700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82680e3f80 0x7f82680ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f48c0b9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48bc087660 0x7f48bc0cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f09d37ff700 1 -- 172.21.15.55:0/545502222 shutdown_connections 2023-10-06T03:33:24.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f09d37ff700 1 --2- 172.21.15.55:0/545502222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09cc06a2b0 0x7f09cc06a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f09d37ff700 1 --2- 172.21.15.55:0/545502222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09cc1599e0 0x7f09cc069ac0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f09d37ff700 1 --2- 172.21.15.55:0/545502222 >> v2:172.21.15.163:3300/0 conn(0x7f09cc159db0 0x7f09cc069080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f09d37ff700 1 -- 172.21.15.55:0/545502222 >> 172.21.15.55:0/545502222 conn(0x7f09cc0c9860 msgr2=0x7f09cc0c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:24.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7fca9159d700 1 Processor -- start 2023-10-06T03:33:24.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f48c0b9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48bc087660 0x7f48bc0cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:40244/0 (socket says 172.21.15.55:40244) 2023-10-06T03:33:24.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7fca9159d700 1 -- start start 2023-10-06T03:33:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f48c0b9b700 1 -- 172.21.15.55:0/851392808 learned_addr learned my addr 172.21.15.55:0/851392808 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f48c139c700 1 --2- 172.21.15.55:0/851392808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48bc0af220 0x7f48bc0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f48bbfff700 1 --2- 172.21.15.55:0/851392808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48bc087ad0 0x7f48bc166710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f8274d9b700 1 -- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82680ea2b0 msgr2=0x7f82680fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f8274d9b700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82680ea2b0 0x7f82680fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f8274d9b700 1 -- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82680e3f80 msgr2=0x7f82680ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f8274d9b700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82680e3f80 0x7f82680ff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.939+0000 7f8274d9b700 1 -- 172.21.15.55:0/3275568763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8264009420 con 0x7f8268006a40 2023-10-06T03:33:24.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.940+0000 7f09d37ff700 1 -- 172.21.15.55:0/545502222 shutdown_connections 2023-10-06T03:33:24.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.940+0000 7f8274d9b700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8268006a40 0x7f8268103980 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f826800daf0 tx=0x7f8264017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:24.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.940+0000 7f48c0b9b700 1 -- 172.21.15.55:0/851392808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48bc0af220 msgr2=0x7f48bc0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.940+0000 7f48c0b9b700 1 --2- 172.21.15.55:0/851392808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48bc0af220 0x7f48bc0c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.940+0000 7f48c0b9b700 1 -- 172.21.15.55:0/851392808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48bc087ad0 msgr2=0x7f48bc166710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.940+0000 7f48c0b9b700 1 --2- 172.21.15.55:0/851392808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48bc087ad0 0x7f48bc166710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.940+0000 7f48c0b9b700 1 -- 172.21.15.55:0/851392808 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac0093c0 con 0x7f48bc087660 2023-10-06T03:33:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.940+0000 7f09d37ff700 1 -- 172.21.15.55:0/545502222 wait complete. 2023-10-06T03:33:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.941+0000 7f0e82dff700 1 Processor -- start 2023-10-06T03:33:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.941+0000 7f826cff9700 1 -- 172.21.15.55:0/3275568763 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8264041070 con 0x7f8268006a40 2023-10-06T03:33:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.941+0000 7fca9159d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fca8c0adbd0 0x7fca8c0adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.941+0000 7f0e82dff700 1 -- start start 2023-10-06T03:33:24.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.941+0000 7fca9159d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca8c158930 0x7fca8c06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.941+0000 7fca9159d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fca8c06da30 0x7fca8c0c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.941+0000 7fca9159d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c06e1e0 con 0x7fca8c0adbd0 2023-10-06T03:33:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.941+0000 7fca9159d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8c157c40 con 0x7fca8c06da30 2023-10-06T03:33:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.941+0000 7fca9159d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c1584a0 con 0x7fca8c158930 2023-10-06T03:33:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.942+0000 7fca90d9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fca8c06da30 0x7fca8c0c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.942+0000 7fca90d9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fca8c06da30 0x7fca8c0c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:33498/0 (socket says 172.21.15.55:33498) 2023-10-06T03:33:24.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.942+0000 7fca90d9c700 1 -- 172.21.15.55:0/1132896733 learned_addr learned my addr 172.21.15.55:0/1132896733 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:24.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.942+0000 7fca8bfff700 1 --2- 172.21.15.55:0/1132896733 >> v2:172.21.15.55:3300/0 conn(0x7fca8c0adbd0 0x7fca8c0adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.942+0000 7fca90d9c700 1 -- 172.21.15.55:0/1132896733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca8c158930 msgr2=0x7fca8c06d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:24.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.942+0000 7fca90d9c700 1 --2- 172.21.15.55:0/1132896733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca8c158930 0x7fca8c06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.942+0000 7fca90d9c700 1 -- 172.21.15.55:0/1132896733 >> v2:172.21.15.55:3300/0 conn(0x7fca8c0adbd0 msgr2=0x7fca8c0adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:24.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.942+0000 7f826cff9700 1 -- 172.21.15.55:0/3275568763 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8264011b10 con 0x7f8268006a40 2023-10-06T03:33:24.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.942+0000 7fca90d9c700 1 --2- 172.21.15.55:0/1132896733 >> v2:172.21.15.55:3300/0 conn(0x7fca8c0adbd0 0x7fca8c0adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.942+0000 7f826cff9700 1 -- 172.21.15.55:0/3275568763 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8264045640 con 0x7f8268006a40 2023-10-06T03:33:24.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.942+0000 7fca90d9c700 1 -- 172.21.15.55:0/1132896733 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c06df00 con 0x7fca8c06da30 2023-10-06T03:33:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.943+0000 7fca90d9c700 1 --2- 172.21.15.55:0/1132896733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca8c06da30 0x7fca8c0c7f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0ae680 tx=0x7fca84009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c8d2034f864093 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.943+0000 7f8275d9d700 1 -- 172.21.15.55:0/3275568763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8268100960 con 0x7f8268006a40 2023-10-06T03:33:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.943+0000 7f8275d9d700 1 -- 172.21.15.55:0/3275568763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826804d7b0 con 0x7f8268006a40 2023-10-06T03:33:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.944+0000 7f09d37ff700 1 Processor -- start 2023-10-06T03:33:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.944+0000 7f09d37ff700 1 -- start start 2023-10-06T03:33:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.943+0000 7fca8affd700 1 -- 172.21.15.55:0/1132896733 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8400f350 con 0x7fca8c06da30 2023-10-06T03:33:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.943+0000 7fca8affd700 1 -- 172.21.15.55:0/1132896733 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca84009a10 con 0x7fca8c06da30 2023-10-06T03:33:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.943+0000 7fca8affd700 1 -- 172.21.15.55:0/1132896733 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8400f4b0 con 0x7fca8c06da30 2023-10-06T03:33:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.944+0000 7fca9159d700 1 -- 172.21.15.55:0/1132896733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca8c06da30 msgr2=0x7fca8c0c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.944+0000 7fca9159d700 1 --2- 172.21.15.55:0/1132896733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca8c06da30 0x7fca8c0c7f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0ae680 tx=0x7fca84009250 comp rx=0 tx=0).stop 2023-10-06T03:33:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.944+0000 7fca9159d700 1 -- 172.21.15.55:0/1132896733 shutdown_connections 2023-10-06T03:33:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.944+0000 7fca9159d700 1 --2- 172.21.15.55:0/1132896733 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca8c06da30 0x7fca8c0c7f60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.944+0000 7fca9159d700 1 --2- 172.21.15.55:0/1132896733 >> v2:172.21.15.55:3300/0 conn(0x7fca8c0adbd0 0x7fca8c0adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.944+0000 7fca9159d700 1 --2- 172.21.15.55:0/1132896733 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca8c158930 0x7fca8c06d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.944+0000 7fca9159d700 1 -- 172.21.15.55:0/1132896733 >> 172.21.15.55:0/1132896733 conn(0x7fca8c0cc1f0 msgr2=0x7fca8c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.944+0000 7f0e82dff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e7c155740 0x7f0e7c155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.945+0000 7f0e8139c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0e7c0b0480 0x7f0e7c156c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43962/0 (socket says 172.21.15.55:43962) 2023-10-06T03:33:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.945+0000 7f0e8139c700 1 -- 172.21.15.55:0/3957946340 learned_addr learned my addr 172.21.15.55:0/3957946340 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.945+0000 7f0e82dff700 1 -- 172.21.15.55:0/3957946340 --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7c0aeec0 con 0x7f0e7c0b0480 2023-10-06T03:33:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.943+0000 7f46a2f9d700 1 Processor -- start 2023-10-06T03:33:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.944+0000 7f46a2f9d700 1 -- start start 2023-10-06T03:33:24.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f0e82dff700 1 -- 172.21.15.55:0/3957946340 --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7c1567e0 con 0x7f0e7c0b1f40 2023-10-06T03:33:24.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f0e82dff700 1 -- 172.21.15.55:0/3957946340 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0b23b0 con 0x7f0e7c155740 2023-10-06T03:33:24.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f46a2f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f469c15c9a0 0x7f469c0cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f46a2f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469c0ce900 0x7f469c0cb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f46a2f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469c0cfa60 con 0x7f469c15cd70 2023-10-06T03:33:24.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f46a2f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469c0cee50 con 0x7f469c15c9a0 2023-10-06T03:33:24.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f46a2f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c0cf130 con 0x7f469c0ce900 2023-10-06T03:33:24.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f09d37ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09cc06a2b0 0x7f09cc141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.942+0000 7f48c0b9b700 1 --2- 172.21.15.55:0/851392808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48bc087660 0x7f48bc0cad90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009160 tx=0x7f48ac016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:24.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.943+0000 7f48b8ff9700 1 -- 172.21.15.55:0/851392808 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac000d10 con 0x7f48bc087660 2023-10-06T03:33:24.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f09d37ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09cc1599e0 0x7f09cc14d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.943+0000 7f48b8ff9700 1 -- 172.21.15.55:0/851392808 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac020070 con 0x7f48bc087660 2023-10-06T03:33:24.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.943+0000 7f48b8ff9700 1 -- 172.21.15.55:0/851392808 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac0276d0 con 0x7f48bc087660 2023-10-06T03:33:24.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f46a179a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469c0ce900 0x7f469c0cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f0e80b9b700 1 --2- 172.21.15.55:0/3957946340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e7c155740 0x7f0e7c155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f46a1f9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f469c15c9a0 0x7f469c0cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f48c1b9d700 1 -- 172.21.15.55:0/851392808 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48bc1672f0 con 0x7f48bc087660 2023-10-06T03:33:24.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f48c1b9d700 1 -- 172.21.15.55:0/851392808 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48bc1678b0 con 0x7f48bc087660 2023-10-06T03:33:24.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f46a1f9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f469c15c9a0 0x7f469c0cfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:33502/0 (socket says 172.21.15.55:33502) 2023-10-06T03:33:24.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.946+0000 7f0e7b7fe700 1 -- 172.21.15.55:0/3957946340 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460977234 0 0) 0x7f0e7c0aeec0 con 0x7f0e7c0b0480 2023-10-06T03:33:24.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.945+0000 7f825a7fc700 1 -- 172.21.15.55:0/3275568763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82500052b0 con 0x7f8268006a40 2023-10-06T03:33:24.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f46a1f9b700 1 -- 172.21.15.55:0/632799212 learned_addr learned my addr 172.21.15.55:0/632799212 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:24.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f0e7b7fe700 1 -- 172.21.15.55:0/3957946340 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e6c003570 con 0x7f0e7c0b0480 2023-10-06T03:33:24.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f09d159b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09cc06a2b0 0x7f09cc141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7fca9159d700 1 -- 172.21.15.55:0/1132896733 shutdown_connections 2023-10-06T03:33:24.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f09d37ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09cc159db0 0x7f09cc171730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f09d37ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc141f40 con 0x7f09cc159db0 2023-10-06T03:33:24.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f46a1f9b700 1 -- 172.21.15.55:0/632799212 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469c0ce900 msgr2=0x7f469c0cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:24.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f46a1f9b700 1 --2- 172.21.15.55:0/632799212 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469c0ce900 0x7f469c0cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f46a1f9b700 1 -- 172.21.15.55:0/632799212 >> v1:172.21.15.163:6789/0 conn(0x7f469c15cd70 legacy=0x7f469c15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f09d37ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc14e1d0 con 0x7f09cc06a2b0 2023-10-06T03:33:24.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.948+0000 7f46a1f9b700 1 -- 172.21.15.55:0/632799212 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c0cf410 con 0x7f469c15c9a0 2023-10-06T03:33:24.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7fca9159d700 1 -- 172.21.15.55:0/1132896733 wait complete. 2023-10-06T03:33:24.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f09d37ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc14e4b0 con 0x7f09cc1599e0 2023-10-06T03:33:24.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.948+0000 7f48c1b9d700 1 -- 172.21.15.55:0/851392808 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48bc06c450 con 0x7f48bc087660 2023-10-06T03:33:24.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.948+0000 7f46a1f9b700 1 --2- 172.21.15.55:0/632799212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f469c15c9a0 0x7f469c0cfef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f468c0099b0 tx=0x7f468c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf77addcd6f22e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:24.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f0e80b9b700 1 -- 172.21.15.55:0/3957946340 >> v1:172.21.15.163:6789/0 conn(0x7f0e7c0b1f40 legacy=0x7f0e7c0afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f0e80b9b700 1 -- 172.21.15.55:0/3957946340 >> v1:172.21.15.55:6789/0 conn(0x7f0e7c0b0480 legacy=0x7f0e7c156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f0e80b9b700 1 -- 172.21.15.55:0/3957946340 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c0cc2a0 con 0x7f0e7c155740 2023-10-06T03:33:24.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f0e7b7fe700 1 -- 172.21.15.55:0/3957946340 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71085724 0 0) 0x7f0e7c1567e0 con 0x7f0e7c0b1f40 2023-10-06T03:33:24.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.948+0000 7f0e80b9b700 1 --2- 172.21.15.55:0/3957946340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e7c155740 0x7f0e7c155b10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0e70008920 tx=0x7f0e70008210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2351b47d84fb0c4e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:24.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.948+0000 7f0e7b7fe700 1 -- 172.21.15.55:0/3957946340 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70010070 con 0x7f0e7c155740 2023-10-06T03:33:24.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.948+0000 7f0e7b7fe700 1 -- 172.21.15.55:0/3957946340 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7000da20 con 0x7f0e7c155740 2023-10-06T03:33:24.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.948+0000 7f0e7b7fe700 1 -- 172.21.15.55:0/3957946340 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7000dda0 con 0x7f0e7c155740 2023-10-06T03:33:24.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.949+0000 7f46a0f99700 1 -- 172.21.15.55:0/632799212 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c009ea0 con 0x7f469c15c9a0 2023-10-06T03:33:24.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.949+0000 7f46a0f99700 1 -- 172.21.15.55:0/632799212 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c00b860 con 0x7f469c15c9a0 2023-10-06T03:33:24.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.949+0000 7f46a0f99700 1 -- 172.21.15.55:0/632799212 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c00bc00 con 0x7f469c15c9a0 2023-10-06T03:33:24.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.947+0000 7f09d0d9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09cc1599e0 0x7f09cc14d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.948+0000 7f09d1d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09cc159db0 0x7f09cc171730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.948+0000 7f09d1d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09cc159db0 0x7f09cc171730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59182/0 (socket says 172.21.15.55:59182) 2023-10-06T03:33:24.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.948+0000 7f09d1d9c700 1 -- 172.21.15.55:0/1355265752 learned_addr learned my addr 172.21.15.55:0/1355265752 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:24.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.949+0000 7f09d1d9c700 1 -- 172.21.15.55:0/1355265752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09cc1599e0 msgr2=0x7f09cc14d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.949+0000 7f09d1d9c700 1 --2- 172.21.15.55:0/1355265752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09cc1599e0 0x7f09cc14d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.949+0000 7f09d1d9c700 1 -- 172.21.15.55:0/1355265752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09cc06a2b0 msgr2=0x7f09cc141a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.949+0000 7f09d1d9c700 1 --2- 172.21.15.55:0/1355265752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09cc06a2b0 0x7f09cc141a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.949+0000 7f09d1d9c700 1 -- 172.21.15.55:0/1355265752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c0009420 con 0x7f09cc159db0 2023-10-06T03:33:24.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.949+0000 7f09d1d9c700 1 --2- 172.21.15.55:0/1355265752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09cc159db0 0x7f09cc171730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00b5e0 tx=0x7f09bc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:24.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.949+0000 7f826cff9700 1 -- 172.21.15.55:0/3275568763 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8264045a30 con 0x7f8268006a40 2023-10-06T03:33:24.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.950+0000 7fa1b25ff700 1 Processor -- start 2023-10-06T03:33:24.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.950+0000 7f48b8ff9700 1 -- 172.21.15.55:0/851392808 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48ac027a80 con 0x7f48bc087660 2023-10-06T03:33:24.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.950+0000 7f46a2f9d700 1 -- 172.21.15.55:0/632799212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f469c15c9a0 msgr2=0x7f469c0cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.950+0000 7f46a2f9d700 1 --2- 172.21.15.55:0/632799212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f469c15c9a0 0x7f469c0cfef0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f468c0099b0 tx=0x7f468c009250 comp rx=0 tx=0).stop 2023-10-06T03:33:24.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.950+0000 7fca9159d700 1 Processor -- start 2023-10-06T03:33:24.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.950+0000 7f0e82dff700 1 -- 172.21.15.55:0/3957946340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e7c155740 msgr2=0x7f0e7c155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:24.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.950+0000 7fa1b25ff700 1 -- start start 2023-10-06T03:33:24.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.950+0000 7f0e82dff700 1 --2- 172.21.15.55:0/3957946340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e7c155740 0x7f0e7c155b10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0e70008920 tx=0x7f0e70008210 comp rx=0 tx=0).stop 2023-10-06T03:33:24.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.951+0000 7fca9159d700 1 -- start start 2023-10-06T03:33:24.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.951+0000 7f46a2f9d700 1 -- 172.21.15.55:0/632799212 shutdown_connections 2023-10-06T03:33:24.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.951+0000 7f46a2f9d700 1 --2- 172.21.15.55:0/632799212 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f469c15c9a0 0x7f469c0cfef0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.951+0000 7f46a2f9d700 1 --2- 172.21.15.55:0/632799212 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469c0ce900 0x7f469c0cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.951+0000 7f46a2f9d700 1 -- 172.21.15.55:0/632799212 >> 172.21.15.55:0/632799212 conn(0x7f469c0d05c0 msgr2=0x7f469c0d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:24.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.951+0000 7f46a2f9d700 1 -- 172.21.15.55:0/632799212 shutdown_connections 2023-10-06T03:33:24.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.951+0000 7f46a2f9d700 1 -- 172.21.15.55:0/632799212 wait complete. 2023-10-06T03:33:24.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.951+0000 7f0e82dff700 1 -- 172.21.15.55:0/3957946340 shutdown_connections 2023-10-06T03:33:24.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.951+0000 7f0e82dff700 1 --2- 172.21.15.55:0/3957946340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e7c155740 0x7f0e7c155b10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:24.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.951+0000 7f0e82dff700 1 -- 172.21.15.55:0/3957946340 >> 172.21.15.55:0/3957946340 conn(0x7f0e7c06a320 msgr2=0x7f0e7c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:24.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.951+0000 7f826cff9700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8254066fb0 0x7f8254069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.951+0000 7f826cff9700 1 -- 172.21.15.55:0/3275568763 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f82640a79c0 con 0x7f8268006a40 2023-10-06T03:33:24.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.952+0000 7f0e82dff700 1 -- 172.21.15.55:0/3957946340 shutdown_connections 2023-10-06T03:33:24.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.952+0000 7f48b8ff9700 1 --2- 172.21.15.55:0/851392808 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f48a8067080 0x7f48a8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.953+0000 7f48b8ff9700 1 -- 172.21.15.55:0/851392808 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f48ac022070 con 0x7f48bc087660 2023-10-06T03:33:24.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.953+0000 7fa1b25ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1ac0b2060 0x7fa1ac159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.953+0000 7fca9159d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca8c06da30 0x7fca8c158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.953+0000 7fa1b25ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ac0d01f0 con 0x7fa1ac0cd0f0 2023-10-06T03:33:24.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.953+0000 7fa1b25ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ac0cd540 con 0x7fa1ac1594d0 2023-10-06T03:33:24.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.953+0000 7fa1b25ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1ac0cf5e0 con 0x7fa1ac0b2060 2023-10-06T03:33:24.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.953+0000 7f48bbfff700 1 --2- 172.21.15.55:0/851392808 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f48a8067080 0x7f48a8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.953+0000 7fca9159d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca8c0adbd0 0x7fca8c148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.953+0000 7fa1abfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa1ac1594d0 0x7fa1ac0b5fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:59516/0 (socket says 172.21.15.55:59516) 2023-10-06T03:33:24.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.953+0000 7fa1abfff700 1 -- 172.21.15.55:0/1162244493 learned_addr learned my addr 172.21.15.55:0/1162244493 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:24.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.953+0000 7f0e82dff700 1 -- 172.21.15.55:0/3957946340 wait complete. 2023-10-06T03:33:24.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.953+0000 7fa1b0b9c700 1 --2- 172.21.15.55:0/1162244493 >> v2:172.21.15.163:3300/0 conn(0x7fa1ac0b2060 0x7fa1ac159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.953+0000 7fca9159d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8c158930 0x7fca8c166420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:24.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.953+0000 7fca8bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca8c06da30 0x7fca8c158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7fca8bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca8c06da30 0x7fca8c158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:33534/0 (socket says 172.21.15.55:33534) 2023-10-06T03:33:24.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7fca8b7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca8c0adbd0 0x7fca8c148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:24.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7fca8b7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca8c0adbd0 0x7fca8c148650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:40302/0 (socket says 172.21.15.55:40302) 2023-10-06T03:33:24.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7f826ffff700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8254066fb0 0x7f8254069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7f48bbfff700 1 --2- 172.21.15.55:0/851392808 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f48a8067080 0x7f48a8069530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f48bc16f400 tx=0x7f48b4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7fca90d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8c158930 0x7fca8c166420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7fca90d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8c158930 0x7fca8c166420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59192/0 (socket says 172.21.15.55:59192) 2023-10-06T03:33:25.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7fca8bfff700 1 -- 172.21.15.55:0/2569593498 learned_addr learned my addr 172.21.15.55:0/2569593498 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:25.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7fca9159d700 1 -- 172.21.15.55:0/2569593498 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c0c1af0 con 0x7fca8c158930 2023-10-06T03:33:25.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7fca9159d700 1 -- 172.21.15.55:0/2569593498 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c148b90 con 0x7fca8c06da30 2023-10-06T03:33:25.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7fca9159d700 1 -- 172.21.15.55:0/2569593498 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c151120 con 0x7fca8c0adbd0 2023-10-06T03:33:25.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7fa1b0b9c700 1 -- 172.21.15.55:0/1162244493 >> v1:172.21.15.163:6789/0 conn(0x7fa1ac1594d0 legacy=0x7fa1ac0b5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7fa1b0b9c700 1 -- 172.21.15.55:0/1162244493 >> v1:172.21.15.55:6789/0 conn(0x7fa1ac0cd0f0 legacy=0x7fa1ac1598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7fa1b0b9c700 1 -- 172.21.15.55:0/1162244493 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ac0cf8c0 con 0x7fa1ac0b2060 2023-10-06T03:33:25.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.955+0000 7f826ffff700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8254066fb0 0x7f8254069460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f82680369b0 tx=0x7f825c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.955+0000 7fa1b0b9c700 1 --2- 172.21.15.55:0/1162244493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0b2060 0x7fa1ac159fb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009de0 tx=0x7fa1a4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f77f6ff6c1259268 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.954+0000 7f46a2f9d700 1 Processor -- start 2023-10-06T03:33:25.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.955+0000 7f46a2f9d700 1 -- start start 2023-10-06T03:33:25.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.955+0000 7fa1aaffd700 1 -- 172.21.15.55:0/1162244493 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a400e070 con 0x7fa1ac0b2060 2023-10-06T03:33:25.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.955+0000 7fa1aaffd700 1 -- 172.21.15.55:0/1162244493 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a400bbb0 con 0x7fa1ac0b2060 2023-10-06T03:33:25.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.955+0000 7fa1aaffd700 1 -- 172.21.15.55:0/1162244493 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4012430 con 0x7fa1ac0b2060 2023-10-06T03:33:25.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.955+0000 7fca90d9c700 1 -- 172.21.15.55:0/2569593498 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca8c0adbd0 msgr2=0x7fca8c148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.955+0000 7fca90d9c700 1 --2- 172.21.15.55:0/2569593498 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca8c0adbd0 0x7fca8c148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.955+0000 7fca90d9c700 1 -- 172.21.15.55:0/2569593498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca8c06da30 msgr2=0x7fca8c158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.955+0000 7fca90d9c700 1 --2- 172.21.15.55:0/2569593498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca8c06da30 0x7fca8c158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.955+0000 7fca90d9c700 1 -- 172.21.15.55:0/2569593498 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8400f610 con 0x7fca8c158930 2023-10-06T03:33:25.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.955+0000 7fca90d9c700 1 --2- 172.21.15.55:0/2569593498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8c158930 0x7fca8c166420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0640c0 tx=0x7fca84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.956+0000 7f46a2f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f469c0ce900 0x7f469c1769e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.956+0000 7f46a2f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469c15c9a0 0x7f469c170940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.956+0000 7f46a2f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c15cd70 0x7f469c15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.956+0000 7f46a2f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c0b4d30 con 0x7f469c15cd70 2023-10-06T03:33:25.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.956+0000 7f46a1f9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f469c0ce900 0x7f469c1769e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.956+0000 7fa1b25ff700 1 -- 172.21.15.55:0/1162244493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0b2060 msgr2=0x7fa1ac159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.956+0000 7fa1b25ff700 1 --2- 172.21.15.55:0/1162244493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0b2060 0x7fa1ac159fb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009de0 tx=0x7fa1a4009780 comp rx=0 tx=0).stop 2023-10-06T03:33:25.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.956+0000 7f46a1f9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f469c0ce900 0x7f469c1769e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:33542/0 (socket says 172.21.15.55:33542) 2023-10-06T03:33:25.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.957+0000 7f46a1f9b700 1 -- 172.21.15.55:0/1958440485 learned_addr learned my addr 172.21.15.55:0/1958440485 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:25.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.957+0000 7f46a2f9d700 1 -- 172.21.15.55:0/1958440485 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c068670 con 0x7f469c0ce900 2023-10-06T03:33:25.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.957+0000 7f46a2f9d700 1 -- 172.21.15.55:0/1958440485 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c177000 con 0x7f469c15c9a0 2023-10-06T03:33:25.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.957+0000 7fa1b25ff700 1 -- 172.21.15.55:0/1162244493 shutdown_connections 2023-10-06T03:33:25.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.957+0000 7fa1b25ff700 1 --2- 172.21.15.55:0/1162244493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0b2060 0x7fa1ac159fb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.957+0000 7fa1b25ff700 1 -- 172.21.15.55:0/1162244493 >> 172.21.15.55:0/1162244493 conn(0x7fa1ac0d05f0 msgr2=0x7fa1ac0d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:25.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.957+0000 7f0e82dff700 1 Processor -- start 2023-10-06T03:33:25.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.957+0000 7f0e82dff700 1 -- start start 2023-10-06T03:33:25.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.957+0000 7f48b8ff9700 1 -- 172.21.15.55:0/851392808 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48ac053800 con 0x7f48bc087660 2023-10-06T03:33:25.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.957+0000 7fa1b25ff700 1 -- 172.21.15.55:0/1162244493 shutdown_connections 2023-10-06T03:33:25.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.957+0000 7f46a179a700 1 --2- 172.21.15.55:0/1958440485 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469c15c9a0 0x7f469c170940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.957+0000 7f46a279c700 1 --2- 172.21.15.55:0/1958440485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c15cd70 0x7f469c15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7fa1b25ff700 1 -- 172.21.15.55:0/1162244493 wait complete. 2023-10-06T03:33:25.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f826cff9700 1 -- 172.21.15.55:0/3275568763 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8264074b60 con 0x7f8268006a40 2023-10-06T03:33:25.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f09c97fa700 1 -- 172.21.15.55:0/1355265752 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00f510 con 0x7f09cc159db0 2023-10-06T03:33:25.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f09c97fa700 1 -- 172.21.15.55:0/1355265752 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc00f690 con 0x7f09cc159db0 2023-10-06T03:33:25.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f09c97fa700 1 -- 172.21.15.55:0/1355265752 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00fa50 con 0x7f09cc159db0 2023-10-06T03:33:25.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f09d37ff700 1 -- 172.21.15.55:0/1355265752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09cc14e790 con 0x7f09cc159db0 2023-10-06T03:33:25.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f09d37ff700 1 -- 172.21.15.55:0/1355265752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc14ed50 con 0x7f09cc159db0 2023-10-06T03:33:25.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.959+0000 7fca83fff700 1 -- 172.21.15.55:0/2569593498 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8400f420 con 0x7fca8c158930 2023-10-06T03:33:25.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.959+0000 7fca83fff700 1 -- 172.21.15.55:0/2569593498 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca84022660 con 0x7fca8c158930 2023-10-06T03:33:25.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.959+0000 7fca9159d700 1 -- 172.21.15.55:0/2569593498 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8c0c14a0 con 0x7fca8c158930 2023-10-06T03:33:25.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f0e82dff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e7c0b0480 0x7f0e7c146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f0e82dff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e7c0b1f40 0x7f0e7c0c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f0e82dff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e7c155740 0x7f0e7c0c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f0e82dff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0c1370 con 0x7f0e7c155740 2023-10-06T03:33:25.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f0e82dff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0ab8c0 con 0x7f0e7c0b0480 2023-10-06T03:33:25.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f0e82dff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0c1610 con 0x7f0e7c0b1f40 2023-10-06T03:33:25.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.959+0000 7fca9159d700 1 -- 172.21.15.55:0/2569593498 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c166960 con 0x7fca8c158930 2023-10-06T03:33:25.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f46a179a700 1 -- 172.21.15.55:0/1958440485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f469c0ce900 msgr2=0x7f469c1769e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f46a179a700 1 --2- 172.21.15.55:0/1958440485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f469c0ce900 0x7f469c1769e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f46a179a700 1 -- 172.21.15.55:0/1958440485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c15cd70 msgr2=0x7f469c15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f46a179a700 1 --2- 172.21.15.55:0/1958440485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c15cd70 0x7f469c15aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.958+0000 7f46a179a700 1 -- 172.21.15.55:0/1958440485 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c0094f0 con 0x7f469c15c9a0 2023-10-06T03:33:25.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.960+0000 7f0e7bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e7c0b1f40 0x7f0e7c0c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.960+0000 7f0e7bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e7c0b1f40 0x7f0e7c0c36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:40328/0 (socket says 172.21.15.55:40328) 2023-10-06T03:33:25.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.960+0000 7f0e7bfff700 1 -- 172.21.15.55:0/1853558510 learned_addr learned my addr 172.21.15.55:0/1853558510 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:25.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.960+0000 7f46a179a700 1 --2- 172.21.15.55:0/1958440485 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469c15c9a0 0x7f469c170940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f469800ea50 tx=0x7f46980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.961+0000 7fa1b25ff700 1 Processor -- start 2023-10-06T03:33:25.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.960+0000 7f0e7bfff700 1 -- 172.21.15.55:0/1853558510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e7c0b0480 msgr2=0x7f0e7c146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.960+0000 7f0e7bfff700 1 --2- 172.21.15.55:0/1853558510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e7c0b0480 0x7f0e7c146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.961+0000 7f0e7bfff700 1 -- 172.21.15.55:0/1853558510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e7c155740 msgr2=0x7f0e7c0c3be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:33:25.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.961+0000 7f0e7bfff700 1 --2- 172.21.15.55:0/1853558510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e7c155740 0x7f0e7c0c3be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.961+0000 7f0e7bfff700 1 -- 172.21.15.55:0/1853558510 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e700084b0 con 0x7f0e7c0b1f40 2023-10-06T03:33:25.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.961+0000 7fa1b25ff700 1 -- start start 2023-10-06T03:33:25.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.961+0000 7f0e7bfff700 1 --2- 172.21.15.55:0/1853558510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e7c0b1f40 0x7f0e7c0c36a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800c7e0 tx=0x7f0e68007df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.961+0000 7f0e78ff9700 1 -- 172.21.15.55:0/1853558510 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68012700 con 0x7f0e7c0b1f40 2023-10-06T03:33:25.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.961+0000 7f0e78ff9700 1 -- 172.21.15.55:0/1853558510 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e68012860 con 0x7f0e7c0b1f40 2023-10-06T03:33:25.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.961+0000 7f0e78ff9700 1 -- 172.21.15.55:0/1853558510 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68012be0 con 0x7f0e7c0b1f40 2023-10-06T03:33:25.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.962+0000 7f0e82dff700 1 -- 172.21.15.55:0/1853558510 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c0c18f0 con 0x7f0e7c0b1f40 2023-10-06T03:33:25.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.962+0000 7f0e82dff700 1 -- 172.21.15.55:0/1853558510 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c147410 con 0x7f0e7c0b1f40 2023-10-06T03:33:25.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.963+0000 7f4691ffb700 1 -- 172.21.15.55:0/1958440485 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469800c7f0 con 0x7f469c15c9a0 2023-10-06T03:33:25.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.963+0000 7f46a2f9d700 1 -- 172.21.15.55:0/1958440485 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469c1772e0 con 0x7f469c15c9a0 2023-10-06T03:33:25.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.963+0000 7f09c97fa700 1 -- 172.21.15.55:0/1355265752 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f09bc01a400 con 0x7f09cc159db0 2023-10-06T03:33:25.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.963+0000 7fa1b25ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0b2060 0x7fa1ac0c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.963+0000 7fa1b25ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1ac0cd0f0 0x7fa1ac158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.964+0000 7fa1b25ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1ac1594d0 0x7fa1ac140990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.963+0000 7f46a2f9d700 1 -- 172.21.15.55:0/1958440485 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469c1778a0 con 0x7f469c15c9a0 2023-10-06T03:33:25.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.964+0000 7fa1abfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0b2060 0x7fa1ac0c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.964+0000 7fa1abfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0b2060 0x7fa1ac0c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:33556/0 (socket says 172.21.15.55:33556) 2023-10-06T03:33:25.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.964+0000 7fa1ab7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1ac0cd0f0 0x7fa1ac158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.964+0000 7fa1ab7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1ac0cd0f0 0x7fa1ac158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59214/0 (socket says 172.21.15.55:59214) 2023-10-06T03:33:25.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.964+0000 7f4691ffb700 1 -- 172.21.15.55:0/1958440485 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469800c970 con 0x7f469c15c9a0 2023-10-06T03:33:25.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.964+0000 7fa1abfff700 1 -- 172.21.15.55:0/3822088944 learned_addr learned my addr 172.21.15.55:0/3822088944 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:25.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.964+0000 7fa1b25ff700 1 -- 172.21.15.55:0/3822088944 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac157360 con 0x7fa1ac0cd0f0 2023-10-06T03:33:25.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.964+0000 7fa1b25ff700 1 -- 172.21.15.55:0/3822088944 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac157640 con 0x7fa1ac0b2060 2023-10-06T03:33:25.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.964+0000 7fa1b25ff700 1 -- 172.21.15.55:0/3822088944 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac157920 con 0x7fa1ac1594d0 2023-10-06T03:33:25.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.964+0000 7fa1b0b9c700 1 --2- 172.21.15.55:0/3822088944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1ac1594d0 0x7fa1ac140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.965+0000 7f4691ffb700 1 -- 172.21.15.55:0/1958440485 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469800cd30 con 0x7f469c15c9a0 2023-10-06T03:33:25.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.965+0000 7f0e78ff9700 1 -- 172.21.15.55:0/1853558510 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e6801b400 con 0x7f0e7c0b1f40 2023-10-06T03:33:25.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.966+0000 7f46877fe700 1 -- 172.21.15.55:0/1958440485 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c0052b0 con 0x7f469c15c9a0 2023-10-06T03:33:25.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.966+0000 7f0e82dff700 1 -- 172.21.15.55:0/1853558510 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7c04c3c0 con 0x7f0e7c0b1f40 2023-10-06T03:33:25.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.965+0000 7fa1abfff700 1 -- 172.21.15.55:0/3822088944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1ac1594d0 msgr2=0x7fa1ac140990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.965+0000 7fa1abfff700 1 --2- 172.21.15.55:0/3822088944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1ac1594d0 0x7fa1ac140990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.965+0000 7fa1abfff700 1 -- 172.21.15.55:0/3822088944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1ac0cd0f0 msgr2=0x7fa1ac158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.965+0000 7fa1abfff700 1 --2- 172.21.15.55:0/3822088944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1ac0cd0f0 0x7fa1ac158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.965+0000 7fa1abfff700 1 -- 172.21.15.55:0/3822088944 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a4009a20 con 0x7fa1ac0b2060 2023-10-06T03:33:25.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.965+0000 7fa1ab7fe700 1 --2- 172.21.15.55:0/3822088944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1ac0cd0f0 0x7fa1ac158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:25.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.966+0000 7f0e78ff9700 1 --2- 172.21.15.55:0/1853558510 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0e60066fb0 0x7f0e60069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.967+0000 7f0e78ff9700 1 -- 172.21.15.55:0/1853558510 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0e68084330 con 0x7f0e7c0b1f40 2023-10-06T03:33:25.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.967+0000 7f0e80b9b700 1 --2- 172.21.15.55:0/1853558510 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0e60066fb0 0x7f0e60069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.968+0000 7f0e80b9b700 1 --2- 172.21.15.55:0/1853558510 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0e60066fb0 0x7f0e60069460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0e70008870 tx=0x7f0e7001c000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.968+0000 7f4691ffb700 1 -- 172.21.15.55:0/1958440485 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4698013680 con 0x7f469c15c9a0 2023-10-06T03:33:25.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.965+0000 7fca83fff700 1 -- 172.21.15.55:0/2569593498 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca840229e0 con 0x7fca8c158930 2023-10-06T03:33:25.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.965+0000 7fca9159d700 1 -- 172.21.15.55:0/2569593498 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca700052b0 con 0x7fca8c158930 2023-10-06T03:33:25.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.966+0000 7fca83fff700 1 -- 172.21.15.55:0/2569593498 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca8401e070 con 0x7fca8c158930 2023-10-06T03:33:25.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.968+0000 7fca83fff700 1 --2- 172.21.15.55:0/2569593498 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fca6c067080 0x7fca6c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.968+0000 7fca83fff700 1 -- 172.21.15.55:0/2569593498 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fca84089f50 con 0x7fca8c158930 2023-10-06T03:33:25.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.968+0000 7fca8bfff700 1 --2- 172.21.15.55:0/2569593498 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fca6c067080 0x7fca6c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.969+0000 7fca8bfff700 1 --2- 172.21.15.55:0/2569593498 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fca6c067080 0x7fca6c069530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fca74006fd0 tx=0x7fca74008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.966+0000 7f09d37ff700 1 -- 172.21.15.55:0/1355265752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09cc14ef70 con 0x7f09cc159db0 2023-10-06T03:33:25.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.968+0000 7f09c97fa700 1 --2- 172.21.15.55:0/1355265752 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f09b8067000 0x7f09b80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.968+0000 7f09c97fa700 1 -- 172.21.15.55:0/1355265752 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f09bc083fc0 con 0x7f09cc159db0 2023-10-06T03:33:25.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.969+0000 7f09d159b700 1 --2- 172.21.15.55:0/1355265752 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f09b8067000 0x7f09b80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.969+0000 7f4691ffb700 1 --2- 172.21.15.55:0/1958440485 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4680067000 0x7f46800694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.970+0000 7f09d159b700 1 --2- 172.21.15.55:0/1355265752 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f09b8067000 0x7f09b80694b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f09c00093f0 tx=0x7f09c0017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.970+0000 7f4691ffb700 1 -- 172.21.15.55:0/1958440485 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f46980848f0 con 0x7f469c15c9a0 2023-10-06T03:33:25.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.970+0000 7f46a1f9b700 1 --2- 172.21.15.55:0/1958440485 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4680067000 0x7f46800694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.971+0000 7f46a1f9b700 1 --2- 172.21.15.55:0/1958440485 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4680067000 0x7f46800694b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f468c009fd0 tx=0x7f468c017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.971+0000 7fa1abfff700 1 --2- 172.21.15.55:0/3822088944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0b2060 0x7fa1ac0c5bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00c810 tx=0x7fa19c007ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.971+0000 7fa193fff700 1 -- 172.21.15.55:0/3822088944 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c005b30 con 0x7fa1ac0b2060 2023-10-06T03:33:25.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.971+0000 7fa193fff700 1 -- 172.21.15.55:0/3822088944 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19c005c90 con 0x7fa1ac0b2060 2023-10-06T03:33:25.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.972+0000 7fa193fff700 1 -- 172.21.15.55:0/3822088944 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0072e0 con 0x7fa1ac0b2060 2023-10-06T03:33:25.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.973+0000 7f0e78ff9700 1 -- 172.21.15.55:0/1853558510 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e680513a0 con 0x7f0e7c0b1f40 2023-10-06T03:33:25.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.973+0000 7fa1b25ff700 1 -- 172.21.15.55:0/3822088944 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ac157c00 con 0x7fa1ac0b2060 2023-10-06T03:33:25.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.973+0000 7fa1b25ff700 1 -- 172.21.15.55:0/3822088944 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ac156a70 con 0x7fa1ac0b2060 2023-10-06T03:33:25.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.974+0000 7fa1b25ff700 1 -- 172.21.15.55:0/3822088944 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ac04c3c0 con 0x7fa1ac0b2060 2023-10-06T03:33:25.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.975+0000 7fca83fff700 1 -- 172.21.15.55:0/2569593498 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca840570c0 con 0x7fca8c158930 2023-10-06T03:33:25.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.976+0000 7fa193fff700 1 -- 172.21.15.55:0/3822088944 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa19c01c440 con 0x7fa1ac0b2060 2023-10-06T03:33:25.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.978+0000 7fa193fff700 1 --2- 172.21.15.55:0/3822088944 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa198066f30 0x7fa1980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.978+0000 7fa193fff700 1 -- 172.21.15.55:0/3822088944 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa19c083f50 con 0x7fa1ac0b2060 2023-10-06T03:33:25.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.978+0000 7fa1ab7fe700 1 --2- 172.21.15.55:0/3822088944 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa198066f30 0x7fa1980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.979+0000 7fa1ab7fe700 1 --2- 172.21.15.55:0/3822088944 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa198066f30 0x7fa1980693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0c71b0 tx=0x7fa1a0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.979+0000 7f4691ffb700 1 -- 172.21.15.55:0/1958440485 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46980519e0 con 0x7f469c15c9a0 2023-10-06T03:33:25.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.983+0000 7fa193fff700 1 -- 172.21.15.55:0/3822088944 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa19c0510f0 con 0x7fa1ac0b2060 2023-10-06T03:33:25.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:24.985+0000 7f09c97fa700 1 -- 172.21.15.55:0/1355265752 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09bc051230 con 0x7f09cc159db0 2023-10-06T03:33:25.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.330+0000 7fecc759e700 1 -- 172.21.15.55:0/2255465032 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fecac005560 con 0x7fecc8155720 2023-10-06T03:33:25.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.331+0000 7fecb67fc700 1 -- 172.21.15.55:0/2255465032 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fecbc0519c0 con 0x7fecc8155720 2023-10-06T03:33:25.332 INFO:teuthology.orchestra.run.smithi055.stdout:55834574853 2023-10-06T03:33:25.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.335+0000 7feca7fff700 1 -- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7feca8067100 msgr2=0x7feca80695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.335+0000 7feca7fff700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7feca8067100 0x7feca80695b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fecc817a8b0 tx=0x7fecc00074d0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.335+0000 7feca7fff700 1 -- 172.21.15.55:0/2255465032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecc8155720 msgr2=0x7fecc8159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.335+0000 7feca7fff700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecc8155720 0x7fecc8159540 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c890 tx=0x7fecbc007f60 comp rx=0 tx=0).stop 2023-10-06T03:33:25.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.336+0000 7feca7fff700 1 -- 172.21.15.55:0/2255465032 shutdown_connections 2023-10-06T03:33:25.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.336+0000 7feca7fff700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fecc8154a80 0x7fecc80c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.336+0000 7feca7fff700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fecc8155720 0x7fecc8159540 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.336+0000 7feca7fff700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7feca8067100 0x7feca80695b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.337+0000 7feca7fff700 1 --2- 172.21.15.55:0/2255465032 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fecc8156af0 0x7fecc80c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.337+0000 7feca7fff700 1 -- 172.21.15.55:0/2255465032 >> 172.21.15.55:0/2255465032 conn(0x7fecc80cc1f0 msgr2=0x7fecc80c2750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:25.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.337+0000 7feca7fff700 1 -- 172.21.15.55:0/2255465032 shutdown_connections 2023-10-06T03:33:25.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.337+0000 7feca7fff700 1 -- 172.21.15.55:0/2255465032 wait complete. 2023-10-06T03:33:25.368 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.1 2023-10-06T03:33:25.369 DEBUG:teuthology.parallel:result is None 2023-10-06T03:33:25.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.395+0000 7f09d37ff700 1 -- 172.21.15.55:0/1355265752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f09cc04c3c0 con 0x7f09cc159db0 2023-10-06T03:33:25.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.396+0000 7f09c97fa700 1 -- 172.21.15.55:0/1355265752 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f09bc054850 con 0x7f09cc159db0 2023-10-06T03:33:25.397 INFO:teuthology.orchestra.run.smithi055.stdout:55834574854 2023-10-06T03:33:25.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.398+0000 7f09d37ff700 1 -- 172.21.15.55:0/1355265752 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f09b8067000 msgr2=0x7f09b80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.398+0000 7f09d37ff700 1 --2- 172.21.15.55:0/1355265752 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f09b8067000 0x7f09b80694b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f09c00093f0 tx=0x7f09c0017000 comp rx=0 tx=0).stop 2023-10-06T03:33:25.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.398+0000 7f09d37ff700 1 -- 172.21.15.55:0/1355265752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09cc159db0 msgr2=0x7f09cc171730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.398+0000 7f09d37ff700 1 --2- 172.21.15.55:0/1355265752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09cc159db0 0x7f09cc171730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00b5e0 tx=0x7f09bc009120 comp rx=0 tx=0).stop 2023-10-06T03:33:25.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.399+0000 7f09d37ff700 1 -- 172.21.15.55:0/1355265752 shutdown_connections 2023-10-06T03:33:25.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.399+0000 7f09d37ff700 1 --2- 172.21.15.55:0/1355265752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09cc159db0 0x7f09cc171730 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.399+0000 7f09d37ff700 1 --2- 172.21.15.55:0/1355265752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09cc06a2b0 0x7f09cc141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.399+0000 7f09d37ff700 1 --2- 172.21.15.55:0/1355265752 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f09b8067000 0x7f09b80694b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.399+0000 7f09d37ff700 1 --2- 172.21.15.55:0/1355265752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09cc1599e0 0x7f09cc14d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.399+0000 7f09d37ff700 1 -- 172.21.15.55:0/1355265752 >> 172.21.15.55:0/1355265752 conn(0x7f09cc0c9860 msgr2=0x7f09cc0c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:25.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.399+0000 7f09d37ff700 1 -- 172.21.15.55:0/1355265752 shutdown_connections 2023-10-06T03:33:25.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.399+0000 7f09d37ff700 1 -- 172.21.15.55:0/1355265752 wait complete. 2023-10-06T03:33:25.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.403+0000 7f825a7fc700 1 -- 172.21.15.55:0/3275568763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f8250005560 con 0x7f8268006a40 2023-10-06T03:33:25.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.404+0000 7f826cff9700 1 -- 172.21.15.55:0/3275568763 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f826404a090 con 0x7f8268006a40 2023-10-06T03:33:25.405 INFO:teuthology.orchestra.run.smithi055.stdout:55834574854 2023-10-06T03:33:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.412+0000 7f48c1b9d700 1 -- 172.21.15.55:0/851392808 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f48bc06c450 con 0x7f48bc087660 2023-10-06T03:33:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.408+0000 7f825a7fc700 1 -- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8254066fb0 msgr2=0x7f8254069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.408+0000 7f825a7fc700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8254066fb0 0x7f8254069460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f82680369b0 tx=0x7f825c009040 comp rx=0 tx=0).stop 2023-10-06T03:33:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.408+0000 7f825a7fc700 1 -- 172.21.15.55:0/3275568763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8268006a40 msgr2=0x7f8268103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.408+0000 7f825a7fc700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8268006a40 0x7f8268103980 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f826800daf0 tx=0x7f8264017000 comp rx=0 tx=0).stop 2023-10-06T03:33:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.412+0000 7f825a7fc700 1 -- 172.21.15.55:0/3275568763 shutdown_connections 2023-10-06T03:33:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.412+0000 7f825a7fc700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8254066fb0 0x7f8254069460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.412+0000 7f825a7fc700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82680ea2b0 0x7f82680fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.412+0000 7f825a7fc700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8268006a40 0x7f8268103980 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.412+0000 7f825a7fc700 1 --2- 172.21.15.55:0/3275568763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82680e3f80 0x7f82680ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.412+0000 7f825a7fc700 1 -- 172.21.15.55:0/3275568763 >> 172.21.15.55:0/3275568763 conn(0x7f8268054da0 msgr2=0x7f82680e7590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:25.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.412+0000 7f825a7fc700 1 -- 172.21.15.55:0/3275568763 shutdown_connections 2023-10-06T03:33:25.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.412+0000 7f825a7fc700 1 -- 172.21.15.55:0/3275568763 wait complete. 2023-10-06T03:33:25.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.413+0000 7f48b8ff9700 1 -- 172.21.15.55:0/851392808 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f48ac056e20 con 0x7f48bc087660 2023-10-06T03:33:25.415 INFO:teuthology.orchestra.run.smithi055.stdout:55834574855 2023-10-06T03:33:25.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.416+0000 7f48c1b9d700 1 -- 172.21.15.55:0/851392808 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f48a8067080 msgr2=0x7f48a8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.416+0000 7f48c1b9d700 1 --2- 172.21.15.55:0/851392808 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f48a8067080 0x7f48a8069530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f48bc16f400 tx=0x7f48b4009040 comp rx=0 tx=0).stop 2023-10-06T03:33:25.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.416+0000 7f48c1b9d700 1 -- 172.21.15.55:0/851392808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48bc087660 msgr2=0x7f48bc0cad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.416+0000 7f48c1b9d700 1 --2- 172.21.15.55:0/851392808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48bc087660 0x7f48bc0cad90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009160 tx=0x7f48ac016000 comp rx=0 tx=0).stop 2023-10-06T03:33:25.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.416+0000 7f48c1b9d700 1 -- 172.21.15.55:0/851392808 shutdown_connections 2023-10-06T03:33:25.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.416+0000 7f48c1b9d700 1 --2- 172.21.15.55:0/851392808 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48bc0af220 0x7f48bc0c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.416+0000 7f48c1b9d700 1 --2- 172.21.15.55:0/851392808 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f48a8067080 0x7f48a8069530 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.417+0000 7f48c1b9d700 1 --2- 172.21.15.55:0/851392808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48bc087660 0x7f48bc0cad90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.417+0000 7f48c1b9d700 1 --2- 172.21.15.55:0/851392808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48bc087ad0 0x7f48bc166710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.417+0000 7f48c1b9d700 1 -- 172.21.15.55:0/851392808 >> 172.21.15.55:0/851392808 conn(0x7f48bc0ccc80 msgr2=0x7f48bc0a98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:25.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.417+0000 7f48c1b9d700 1 -- 172.21.15.55:0/851392808 shutdown_connections 2023-10-06T03:33:25.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.417+0000 7f48c1b9d700 1 -- 172.21.15.55:0/851392808 wait complete. 2023-10-06T03:33:25.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.424+0000 7f0e82dff700 1 -- 172.21.15.55:0/1853558510 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0e7c0aebe0 con 0x7f0e7c0b1f40 2023-10-06T03:33:25.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.425+0000 7f0e78ff9700 1 -- 172.21.15.55:0/1853558510 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0e680549c0 con 0x7f0e7c0b1f40 2023-10-06T03:33:25.427 INFO:teuthology.orchestra.run.smithi055.stdout:55834574854 2023-10-06T03:33:25.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.4 2023-10-06T03:33:25.428 DEBUG:teuthology.parallel:result is None 2023-10-06T03:33:25.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.428+0000 7fca9159d700 1 -- 172.21.15.55:0/2569593498 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fca70005560 con 0x7fca8c158930 2023-10-06T03:33:25.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.428+0000 7f0e82dff700 1 -- 172.21.15.55:0/1853558510 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0e60066fb0 msgr2=0x7f0e60069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.428+0000 7f0e82dff700 1 --2- 172.21.15.55:0/1853558510 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0e60066fb0 0x7f0e60069460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0e70008870 tx=0x7f0e7001c000 comp rx=0 tx=0).stop 2023-10-06T03:33:25.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.428+0000 7fca83fff700 1 -- 172.21.15.55:0/2569593498 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fca8405a6e0 con 0x7fca8c158930 2023-10-06T03:33:25.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.428+0000 7f0e82dff700 1 -- 172.21.15.55:0/1853558510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e7c0b1f40 msgr2=0x7f0e7c0c36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.428+0000 7f0e82dff700 1 --2- 172.21.15.55:0/1853558510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e7c0b1f40 0x7f0e7c0c36a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800c7e0 tx=0x7f0e68007df0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.428+0000 7f0e82dff700 1 -- 172.21.15.55:0/1853558510 shutdown_connections 2023-10-06T03:33:25.431 INFO:teuthology.orchestra.run.smithi055.stdout:55834574854 2023-10-06T03:33:25.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.428+0000 7f0e82dff700 1 --2- 172.21.15.55:0/1853558510 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e7c155740 0x7f0e7c0c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.429+0000 7f0e82dff700 1 --2- 172.21.15.55:0/1853558510 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e7c0b0480 0x7f0e7c146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.429+0000 7f0e82dff700 1 --2- 172.21.15.55:0/1853558510 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0e60066fb0 0x7f0e60069460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.429+0000 7f0e82dff700 1 --2- 172.21.15.55:0/1853558510 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e7c0b1f40 0x7f0e7c0c36a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.429+0000 7f0e82dff700 1 -- 172.21.15.55:0/1853558510 >> 172.21.15.55:0/1853558510 conn(0x7f0e7c06a320 msgr2=0x7f0e7c06f610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.429+0000 7f0e82dff700 1 -- 172.21.15.55:0/1853558510 shutdown_connections 2023-10-06T03:33:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.429+0000 7f0e82dff700 1 -- 172.21.15.55:0/1853558510 wait complete. 2023-10-06T03:33:25.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.432+0000 7fca9159d700 1 -- 172.21.15.55:0/2569593498 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fca6c067080 msgr2=0x7fca6c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.432+0000 7fca9159d700 1 --2- 172.21.15.55:0/2569593498 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fca6c067080 0x7fca6c069530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fca74006fd0 tx=0x7fca74008040 comp rx=0 tx=0).stop 2023-10-06T03:33:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.432+0000 7fca9159d700 1 -- 172.21.15.55:0/2569593498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8c158930 msgr2=0x7fca8c166420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.432+0000 7fca9159d700 1 --2- 172.21.15.55:0/2569593498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8c158930 0x7fca8c166420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0640c0 tx=0x7fca84017000 comp rx=0 tx=0).stop 2023-10-06T03:33:25.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.433+0000 7fca9159d700 1 -- 172.21.15.55:0/2569593498 shutdown_connections 2023-10-06T03:33:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.433+0000 7fca9159d700 1 --2- 172.21.15.55:0/2569593498 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca8c158930 0x7fca8c166420 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.433+0000 7fca9159d700 1 --2- 172.21.15.55:0/2569593498 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca8c06da30 0x7fca8c158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.433+0000 7fca9159d700 1 --2- 172.21.15.55:0/2569593498 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fca6c067080 0x7fca6c069530 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.433+0000 7fca9159d700 1 --2- 172.21.15.55:0/2569593498 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca8c0adbd0 0x7fca8c148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.433+0000 7fca9159d700 1 -- 172.21.15.55:0/2569593498 >> 172.21.15.55:0/2569593498 conn(0x7fca8c0cc1f0 msgr2=0x7fca8c154ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.433+0000 7fca9159d700 1 -- 172.21.15.55:0/2569593498 shutdown_connections 2023-10-06T03:33:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.433+0000 7fca9159d700 1 -- 172.21.15.55:0/2569593498 wait complete. 2023-10-06T03:33:25.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.436+0000 7fa1b25ff700 1 -- 172.21.15.55:0/3822088944 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa1ac0cfba0 con 0x7fa1ac0b2060 2023-10-06T03:33:25.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.437+0000 7fa193fff700 1 -- 172.21.15.55:0/3822088944 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa19c017090 con 0x7fa1ac0b2060 2023-10-06T03:33:25.439 INFO:teuthology.orchestra.run.smithi055.stdout:55834574854 2023-10-06T03:33:25.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.0 2023-10-06T03:33:25.442 DEBUG:teuthology.parallel:result is None 2023-10-06T03:33:25.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.440+0000 7fa1b25ff700 1 -- 172.21.15.55:0/3822088944 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa198066f30 msgr2=0x7fa1980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.440+0000 7fa1b25ff700 1 --2- 172.21.15.55:0/3822088944 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa198066f30 0x7fa1980693e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0c71b0 tx=0x7fa1a0009040 comp rx=0 tx=0).stop 2023-10-06T03:33:25.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.441+0000 7fa1b25ff700 1 -- 172.21.15.55:0/3822088944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0b2060 msgr2=0x7fa1ac0c5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.441+0000 7fa1b25ff700 1 --2- 172.21.15.55:0/3822088944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0b2060 0x7fa1ac0c5bf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00c810 tx=0x7fa19c007ef0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.441+0000 7fa1b25ff700 1 -- 172.21.15.55:0/3822088944 shutdown_connections 2023-10-06T03:33:25.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.441+0000 7fa1b25ff700 1 --2- 172.21.15.55:0/3822088944 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa198066f30 0x7fa1980693e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.441+0000 7fa1b25ff700 1 --2- 172.21.15.55:0/3822088944 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1ac1594d0 0x7fa1ac140990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.441+0000 7fa1b25ff700 1 --2- 172.21.15.55:0/3822088944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1ac0b2060 0x7fa1ac0c5bf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.441+0000 7fa1b25ff700 1 --2- 172.21.15.55:0/3822088944 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1ac0cd0f0 0x7fa1ac158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.441+0000 7f46877fe700 1 -- 172.21.15.55:0/1958440485 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f467c005560 con 0x7f469c15c9a0 2023-10-06T03:33:25.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.441+0000 7fa1b25ff700 1 -- 172.21.15.55:0/3822088944 >> 172.21.15.55:0/3822088944 conn(0x7fa1ac0d05f0 msgr2=0x7fa1ac153d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:25.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.441+0000 7fa1b25ff700 1 -- 172.21.15.55:0/3822088944 shutdown_connections 2023-10-06T03:33:25.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.441+0000 7fa1b25ff700 1 -- 172.21.15.55:0/3822088944 wait complete. 2023-10-06T03:33:25.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.441+0000 7f4691ffb700 1 -- 172.21.15.55:0/1958440485 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4698055000 con 0x7f469c15c9a0 2023-10-06T03:33:25.445 INFO:teuthology.orchestra.run.smithi055.stdout:55834574853 2023-10-06T03:33:25.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.444+0000 7f46877fe700 1 -- 172.21.15.55:0/1958440485 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4680067000 msgr2=0x7f46800694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.445+0000 7f46877fe700 1 --2- 172.21.15.55:0/1958440485 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4680067000 0x7f46800694b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f468c009fd0 tx=0x7f468c017000 comp rx=0 tx=0).stop 2023-10-06T03:33:25.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.445+0000 7f46877fe700 1 -- 172.21.15.55:0/1958440485 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469c15c9a0 msgr2=0x7f469c170940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.445+0000 7f46877fe700 1 --2- 172.21.15.55:0/1958440485 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469c15c9a0 0x7f469c170940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f469800ea50 tx=0x7f46980091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.445+0000 7f46877fe700 1 -- 172.21.15.55:0/1958440485 shutdown_connections 2023-10-06T03:33:25.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.445+0000 7f46877fe700 1 --2- 172.21.15.55:0/1958440485 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f469c15cd70 0x7f469c15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.445+0000 7f46877fe700 1 --2- 172.21.15.55:0/1958440485 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f469c0ce900 0x7f469c1769e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.445+0000 7f46877fe700 1 --2- 172.21.15.55:0/1958440485 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4680067000 0x7f46800694b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.445+0000 7f46877fe700 1 --2- 172.21.15.55:0/1958440485 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f469c15c9a0 0x7f469c170940 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f469800ea50 tx=0x7f46980091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.445+0000 7f46877fe700 1 -- 172.21.15.55:0/1958440485 >> 172.21.15.55:0/1958440485 conn(0x7f469c0d05c0 msgr2=0x7f469c0af660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:25.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.6 2023-10-06T03:33:25.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.446+0000 7f46877fe700 1 -- 172.21.15.55:0/1958440485 shutdown_connections 2023-10-06T03:33:25.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.446+0000 7f46877fe700 1 -- 172.21.15.55:0/1958440485 wait complete. 2023-10-06T03:33:25.449 DEBUG:teuthology.parallel:result is None 2023-10-06T03:33:25.453 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.3 2023-10-06T03:33:25.453 DEBUG:teuthology.parallel:result is None 2023-10-06T03:33:25.463 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.5 2023-10-06T03:33:25.463 DEBUG:teuthology.parallel:result is None 2023-10-06T03:33:25.468 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.2 2023-10-06T03:33:25.469 DEBUG:teuthology.parallel:result is None 2023-10-06T03:33:25.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2023-10-06T03:33:25.470 DEBUG:teuthology.parallel:result is None 2023-10-06T03:33:25.470 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T03:33:25.471 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:33:25.698 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.696+0000 7f60fd19d700 1 Processor -- start 2023-10-06T03:33:25.698 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.696+0000 7f60fd19d700 1 -- start start 2023-10-06T03:33:25.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.698+0000 7f60fd19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60f8156310 0x7f60f80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.698+0000 7f60fd19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f60f81566e0 0x7f60f80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.698+0000 7f60fd19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f60f80ca300 0x7f60f80ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.698+0000 7f60fd19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f80ca860 con 0x7f60f81566e0 2023-10-06T03:33:25.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.698+0000 7f60fd19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f80cab40 con 0x7f60f80ca300 2023-10-06T03:33:25.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.698+0000 7f60fd19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f80cae20 con 0x7f60f8156310 2023-10-06T03:33:25.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.698+0000 7f60f77fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f60f81566e0 0x7f60f80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.698+0000 7f60f77fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f60f81566e0 0x7f60f80ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59218/0 (socket says 172.21.15.55:59218) 2023-10-06T03:33:25.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.698+0000 7f60f77fe700 1 -- 172.21.15.55:0/1507549215 learned_addr learned my addr 172.21.15.55:0/1507549215 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:25.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.698+0000 7f60fc99c700 1 --2- 172.21.15.55:0/1507549215 >> v2:172.21.15.163:3300/0 conn(0x7f60f80ca300 0x7f60f80ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.698+0000 7f60f7fff700 1 --2- 172.21.15.55:0/1507549215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60f8156310 0x7f60f80c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.698+0000 7f60f77fe700 1 -- 172.21.15.55:0/1507549215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60f8156310 msgr2=0x7f60f80c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:25.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.699+0000 7f60f77fe700 1 --2- 172.21.15.55:0/1507549215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60f8156310 0x7f60f80c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.699+0000 7f60f77fe700 1 -- 172.21.15.55:0/1507549215 >> v2:172.21.15.163:3300/0 conn(0x7f60f80ca300 msgr2=0x7f60f80ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:25.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.699+0000 7f60f77fe700 1 --2- 172.21.15.55:0/1507549215 >> v2:172.21.15.163:3300/0 conn(0x7f60f80ca300 0x7f60f80ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.699+0000 7f60f77fe700 1 -- 172.21.15.55:0/1507549215 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f80cb100 con 0x7f60f81566e0 2023-10-06T03:33:25.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.699+0000 7f60f7fff700 1 --2- 172.21.15.55:0/1507549215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60f8156310 0x7f60f80c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:25.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.699+0000 7f60f77fe700 1 --2- 172.21.15.55:0/1507549215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f81566e0 0x7f60f80ae000 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f60f806f970 tx=0x7f60ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97261999301f5975 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.699+0000 7f60f6ffd700 1 -- 172.21.15.55:0/1507549215 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00b860 con 0x7f60f81566e0 2023-10-06T03:33:25.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.699+0000 7f60f6ffd700 1 -- 172.21.15.55:0/1507549215 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec00b9c0 con 0x7f60f81566e0 2023-10-06T03:33:25.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.699+0000 7f60f6ffd700 1 -- 172.21.15.55:0/1507549215 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00bd60 con 0x7f60f81566e0 2023-10-06T03:33:25.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.700+0000 7f60fd19d700 1 -- 172.21.15.55:0/1507549215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f81566e0 msgr2=0x7f60f80ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.700+0000 7f60fd19d700 1 --2- 172.21.15.55:0/1507549215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f81566e0 0x7f60f80ae000 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f60f806f970 tx=0x7f60ec009250 comp rx=0 tx=0).stop 2023-10-06T03:33:25.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.700+0000 7f60fd19d700 1 -- 172.21.15.55:0/1507549215 shutdown_connections 2023-10-06T03:33:25.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.700+0000 7f60fd19d700 1 --2- 172.21.15.55:0/1507549215 >> v2:172.21.15.163:3300/0 conn(0x7f60f80ca300 0x7f60f80ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.700+0000 7f60fd19d700 1 --2- 172.21.15.55:0/1507549215 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60f8156310 0x7f60f80c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.700+0000 7f60fd19d700 1 --2- 172.21.15.55:0/1507549215 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f81566e0 0x7f60f80ae000 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.700+0000 7f60fd19d700 1 -- 172.21.15.55:0/1507549215 >> 172.21.15.55:0/1507549215 conn(0x7f60f80cc1f0 msgr2=0x7f60f80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:25.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.700+0000 7f60fd19d700 1 -- 172.21.15.55:0/1507549215 shutdown_connections 2023-10-06T03:33:25.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.700+0000 7f60fd19d700 1 -- 172.21.15.55:0/1507549215 wait complete. 2023-10-06T03:33:25.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.701+0000 7f60fd19d700 1 Processor -- start 2023-10-06T03:33:25.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.701+0000 7f60fd19d700 1 -- start start 2023-10-06T03:33:25.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.702+0000 7f60fd19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f80ca300 0x7f60f8141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.702+0000 7f60fd19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60f8156310 0x7f60f815c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.702+0000 7f60fd19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60f81566e0 0x7f60f816dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.702+0000 7f60fd19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f816e1a0 con 0x7f60f80ca300 2023-10-06T03:33:25.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.702+0000 7f60fd19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f815e730 con 0x7f60f8156310 2023-10-06T03:33:25.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.702+0000 7f60fd19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f815c700 con 0x7f60f81566e0 2023-10-06T03:33:25.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.702+0000 7f60f7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f80ca300 0x7f60f8141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.703+0000 7f60f7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f80ca300 0x7f60f8141d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59224/0 (socket says 172.21.15.55:59224) 2023-10-06T03:33:25.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.703+0000 7f60f7fff700 1 -- 172.21.15.55:0/3273831346 learned_addr learned my addr 172.21.15.55:0/3273831346 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:25.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.703+0000 7f60fc99c700 1 --2- 172.21.15.55:0/3273831346 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60f81566e0 0x7f60f816dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.703+0000 7f60f77fe700 1 --2- 172.21.15.55:0/3273831346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60f8156310 0x7f60f815c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.703+0000 7f60f7fff700 1 -- 172.21.15.55:0/3273831346 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60f81566e0 msgr2=0x7f60f816dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.703+0000 7f60f7fff700 1 --2- 172.21.15.55:0/3273831346 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60f81566e0 0x7f60f816dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.703+0000 7f60f7fff700 1 -- 172.21.15.55:0/3273831346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60f8156310 msgr2=0x7f60f815c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.703+0000 7f60f7fff700 1 --2- 172.21.15.55:0/3273831346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60f8156310 0x7f60f815c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.703+0000 7f60f7fff700 1 -- 172.21.15.55:0/3273831346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec0094f0 con 0x7f60f80ca300 2023-10-06T03:33:25.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.703+0000 7f60f7fff700 1 --2- 172.21.15.55:0/3273831346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f80ca300 0x7f60f8141d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f60e800c830 tx=0x7f60e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.703+0000 7f60dffff700 1 -- 172.21.15.55:0/3273831346 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e8011600 con 0x7f60f80ca300 2023-10-06T03:33:25.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.703+0000 7f60dffff700 1 -- 172.21.15.55:0/3273831346 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e8011760 con 0x7f60f80ca300 2023-10-06T03:33:25.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.703+0000 7f60dffff700 1 -- 172.21.15.55:0/3273831346 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e8011b20 con 0x7f60f80ca300 2023-10-06T03:33:25.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.703+0000 7f60fd19d700 1 -- 172.21.15.55:0/3273831346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f815c9e0 con 0x7f60f80ca300 2023-10-06T03:33:25.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.704+0000 7f60fd19d700 1 -- 172.21.15.55:0/3273831346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f815e070 con 0x7f60f80ca300 2023-10-06T03:33:25.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.704+0000 7f60fd19d700 1 -- 172.21.15.55:0/3273831346 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f815cc20 con 0x7f60f80ca300 2023-10-06T03:33:25.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.705+0000 7f60dffff700 1 -- 172.21.15.55:0/3273831346 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f60e8018360 con 0x7f60f80ca300 2023-10-06T03:33:25.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.706+0000 7f60dffff700 1 --2- 172.21.15.55:0/3273831346 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f60e4066f30 0x7f60e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:25.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.706+0000 7f60dffff700 1 -- 172.21.15.55:0/3273831346 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f60e80846c0 con 0x7f60f80ca300 2023-10-06T03:33:25.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.707+0000 7f60f77fe700 1 --2- 172.21.15.55:0/3273831346 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f60e4066f30 0x7f60e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:25.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.707+0000 7f60f77fe700 1 --2- 172.21.15.55:0/3273831346 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f60e4066f30 0x7f60e40693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f60f8156b50 tx=0x7f60ec017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:25.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.711+0000 7f60dffff700 1 -- 172.21.15.55:0/3273831346 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60e8051730 con 0x7f60f80ca300 2023-10-06T03:33:25.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.906+0000 7f60fd19d700 1 -- 172.21.15.55:0/3273831346 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60f804c3c0 con 0x7f60e4066f30 2023-10-06T03:33:25.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.915+0000 7f60dffff700 1 -- 172.21.15.55:0/3273831346 <== mgr.4114 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286756 (secure 0 0 0) 0x7f60f804c3c0 con 0x7f60e4066f30 2023-10-06T03:33:25.921 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:25.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.919+0000 7f60fd19d700 1 -- 172.21.15.55:0/3273831346 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f60e4066f30 msgr2=0x7f60e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.919+0000 7f60fd19d700 1 --2- 172.21.15.55:0/3273831346 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f60e4066f30 0x7f60e40693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f60f8156b50 tx=0x7f60ec017000 comp rx=0 tx=0).stop 2023-10-06T03:33:25.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.919+0000 7f60fd19d700 1 -- 172.21.15.55:0/3273831346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f80ca300 msgr2=0x7f60f8141d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:25.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.919+0000 7f60fd19d700 1 --2- 172.21.15.55:0/3273831346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f80ca300 0x7f60f8141d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f60e800c830 tx=0x7f60e8008040 comp rx=0 tx=0).stop 2023-10-06T03:33:25.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.919+0000 7f60fd19d700 1 -- 172.21.15.55:0/3273831346 shutdown_connections 2023-10-06T03:33:25.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.919+0000 7f60fd19d700 1 --2- 172.21.15.55:0/3273831346 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f60e4066f30 0x7f60e40693e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.919+0000 7f60fd19d700 1 --2- 172.21.15.55:0/3273831346 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60f81566e0 0x7f60f816dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.919+0000 7f60fd19d700 1 --2- 172.21.15.55:0/3273831346 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60f80ca300 0x7f60f8141d90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.919+0000 7f60fd19d700 1 --2- 172.21.15.55:0/3273831346 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60f8156310 0x7f60f815c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:25.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.919+0000 7f60fd19d700 1 -- 172.21.15.55:0/3273831346 >> 172.21.15.55:0/3273831346 conn(0x7f60f80cc1f0 msgr2=0x7f60f80c27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:25.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.920+0000 7f60fd19d700 1 -- 172.21.15.55:0/3273831346 shutdown_connections 2023-10-06T03:33:25.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:25.920+0000 7f60fd19d700 1 -- 172.21.15.55:0/3273831346 wait complete. 2023-10-06T03:33:25.925 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:33:25.938 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T03:33:24.490873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397592,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752577128,"statfs":{"total":773094113280,"available":770638979072,"internally_reserved":0,"allocated":2973696,"data_stored":1815068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.015987"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.489781+0000","last_change":"2023-10-06T03:33:15.485195+0000","last_active":"2023-10-06T03:33:15.489781+0000","last_peered":"2023-10-06T03:33:15.489781+0000","last_clean":"2023-10-06T03:33:15.489781+0000","last_became_active":"2023-10-06T03:33:15.484848+0000","last_became_peered":"2023-10-06T03:33:15.484848+0000","last_unstale":"2023-10-06T03:33:15.489781+0000","last_undegraded":"2023-10-06T03:33:15.489781+0000","last_fullsized":"2023-10-06T03:33:15.489781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:08.862091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502702+0000","last_change":"2023-10-06T03:33:15.498977+0000","last_active":"2023-10-06T03:33:15.502702+0000","last_peered":"2023-10-06T03:33:15.502702+0000","last_clean":"2023-10-06T03:33:15.502702+0000","last_became_active":"2023-10-06T03:33:15.498671+0000","last_became_peered":"2023-10-06T03:33:15.498671+0000","last_unstale":"2023-10-06T03:33:15.502702+0000","last_undegraded":"2023-10-06T03:33:15.502702+0000","last_fullsized":"2023-10-06T03:33:15.502702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500827+0000","last_change":"2023-10-06T03:33:15.494700+0000","last_active":"2023-10-06T03:33:15.500827+0000","last_peered":"2023-10-06T03:33:15.500827+0000","last_clean":"2023-10-06T03:33:15.500827+0000","last_became_active":"2023-10-06T03:33:15.494361+0000","last_became_peered":"2023-10-06T03:33:15.494361+0000","last_unstale":"2023-10-06T03:33:15.500827+0000","last_undegraded":"2023-10-06T03:33:15.500827+0000","last_fullsized":"2023-10-06T03:33:15.500827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:14.663102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.488889+0000","last_change":"2023-10-06T03:33:15.485455+0000","last_active":"2023-10-06T03:33:15.488889+0000","last_peered":"2023-10-06T03:33:15.488889+0000","last_clean":"2023-10-06T03:33:15.488889+0000","last_became_active":"2023-10-06T03:33:15.485159+0000","last_became_peered":"2023-10-06T03:33:15.485159+0000","last_unstale":"2023-10-06T03:33:15.488889+0000","last_undegraded":"2023-10-06T03:33:15.488889+0000","last_fullsized":"2023-10-06T03:33:15.488889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:02.594885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500387+0000","last_change":"2023-10-06T03:33:15.498785+0000","last_active":"2023-10-06T03:33:15.500387+0000","last_peered":"2023-10-06T03:33:15.500387+0000","last_clean":"2023-10-06T03:33:15.500387+0000","last_became_active":"2023-10-06T03:33:15.498503+0000","last_became_peered":"2023-10-06T03:33:15.498503+0000","last_unstale":"2023-10-06T03:33:15.500387+0000","last_undegraded":"2023-10-06T03:33:15.500387+0000","last_fullsized":"2023-10-06T03:33:15.500387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:59.545999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494228+0000","last_change":"2023-10-06T03:33:15.484222+0000","last_active":"2023-10-06T03:33:15.494228+0000","last_peered":"2023-10-06T03:33:15.494228+0000","last_clean":"2023-10-06T03:33:15.494228+0000","last_became_active":"2023-10-06T03:33:15.483910+0000","last_became_peered":"2023-10-06T03:33:15.483910+0000","last_unstale":"2023-10-06T03:33:15.494228+0000","last_undegraded":"2023-10-06T03:33:15.494228+0000","last_fullsized":"2023-10-06T03:33:15.494228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:41.309883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501590+0000","last_change":"2023-10-06T03:33:15.497903+0000","last_active":"2023-10-06T03:33:15.501590+0000","last_peered":"2023-10-06T03:33:15.501590+0000","last_clean":"2023-10-06T03:33:15.501590+0000","last_became_active":"2023-10-06T03:33:15.496979+0000","last_became_peered":"2023-10-06T03:33:15.496979+0000","last_unstale":"2023-10-06T03:33:15.501590+0000","last_undegraded":"2023-10-06T03:33:15.501590+0000","last_fullsized":"2023-10-06T03:33:15.501590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502728+0000","last_change":"2023-10-06T03:33:15.501214+0000","last_active":"2023-10-06T03:33:15.502728+0000","last_peered":"2023-10-06T03:33:15.502728+0000","last_clean":"2023-10-06T03:33:15.502728+0000","last_became_active":"2023-10-06T03:33:15.500739+0000","last_became_peered":"2023-10-06T03:33:15.500739+0000","last_unstale":"2023-10-06T03:33:15.502728+0000","last_undegraded":"2023-10-06T03:33:15.502728+0000","last_fullsized":"2023-10-06T03:33:15.502728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.964350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.499625+0000","last_change":"2023-10-06T03:33:15.498383+0000","last_active":"2023-10-06T03:33:15.499625+0000","last_peered":"2023-10-06T03:33:15.499625+0000","last_clean":"2023-10-06T03:33:15.499625+0000","last_became_active":"2023-10-06T03:33:15.498162+0000","last_became_peered":"2023-10-06T03:33:15.498162+0000","last_unstale":"2023-10-06T03:33:15.499625+0000","last_undegraded":"2023-10-06T03:33:15.499625+0000","last_fullsized":"2023-10-06T03:33:15.499625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:02.889905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501799+0000","last_change":"2023-10-06T03:33:15.496957+0000","last_active":"2023-10-06T03:33:15.501799+0000","last_peered":"2023-10-06T03:33:15.501799+0000","last_clean":"2023-10-06T03:33:15.501799+0000","last_became_active":"2023-10-06T03:33:15.496673+0000","last_became_peered":"2023-10-06T03:33:15.496673+0000","last_unstale":"2023-10-06T03:33:15.501799+0000","last_undegraded":"2023-10-06T03:33:15.501799+0000","last_fullsized":"2023-10-06T03:33:15.501799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500848+0000","last_change":"2023-10-06T03:33:15.495118+0000","last_active":"2023-10-06T03:33:15.500848+0000","last_peered":"2023-10-06T03:33:15.500848+0000","last_clean":"2023-10-06T03:33:15.500848+0000","last_became_active":"2023-10-06T03:33:15.494732+0000","last_became_peered":"2023-10-06T03:33:15.494732+0000","last_unstale":"2023-10-06T03:33:15.500848+0000","last_undegraded":"2023-10-06T03:33:15.500848+0000","last_fullsized":"2023-10-06T03:33:15.500848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:02.889905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494010+0000","last_change":"2023-10-06T03:33:15.491772+0000","last_active":"2023-10-06T03:33:15.494010+0000","last_peered":"2023-10-06T03:33:15.494010+0000","last_clean":"2023-10-06T03:33:15.494010+0000","last_became_active":"2023-10-06T03:33:15.491530+0000","last_became_peered":"2023-10-06T03:33:15.491530+0000","last_unstale":"2023-10-06T03:33:15.494010+0000","last_undegraded":"2023-10-06T03:33:15.494010+0000","last_fullsized":"2023-10-06T03:33:15.494010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:57.097910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.489816+0000","last_change":"2023-10-06T03:33:15.485334+0000","last_active":"2023-10-06T03:33:15.489816+0000","last_peered":"2023-10-06T03:33:15.489816+0000","last_clean":"2023-10-06T03:33:15.489816+0000","last_became_active":"2023-10-06T03:33:15.485051+0000","last_became_peered":"2023-10-06T03:33:15.485051+0000","last_unstale":"2023-10-06T03:33:15.489816+0000","last_undegraded":"2023-10-06T03:33:15.489816+0000","last_fullsized":"2023-10-06T03:33:15.489816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:42.519384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498233+0000","last_change":"2023-10-06T03:33:15.495208+0000","last_active":"2023-10-06T03:33:15.498233+0000","last_peered":"2023-10-06T03:33:15.498233+0000","last_clean":"2023-10-06T03:33:15.498233+0000","last_became_active":"2023-10-06T03:33:15.494897+0000","last_became_peered":"2023-10-06T03:33:15.494897+0000","last_unstale":"2023-10-06T03:33:15.498233+0000","last_undegraded":"2023-10-06T03:33:15.498233+0000","last_fullsized":"2023-10-06T03:33:15.498233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:12.635649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501608+0000","last_change":"2023-10-06T03:33:15.498114+0000","last_active":"2023-10-06T03:33:15.501608+0000","last_peered":"2023-10-06T03:33:15.501608+0000","last_clean":"2023-10-06T03:33:15.501608+0000","last_became_active":"2023-10-06T03:33:15.497401+0000","last_became_peered":"2023-10-06T03:33:15.497401+0000","last_unstale":"2023-10-06T03:33:15.501608+0000","last_undegraded":"2023-10-06T03:33:15.501608+0000","last_fullsized":"2023-10-06T03:33:15.501608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494198+0000","last_change":"2023-10-06T03:33:15.484297+0000","last_active":"2023-10-06T03:33:15.494198+0000","last_peered":"2023-10-06T03:33:15.494198+0000","last_clean":"2023-10-06T03:33:15.494198+0000","last_became_active":"2023-10-06T03:33:15.483921+0000","last_became_peered":"2023-10-06T03:33:15.483921+0000","last_unstale":"2023-10-06T03:33:15.494198+0000","last_undegraded":"2023-10-06T03:33:15.494198+0000","last_fullsized":"2023-10-06T03:33:15.494198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:03.085548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.495074+0000","last_change":"2023-10-06T03:33:15.492170+0000","last_active":"2023-10-06T03:33:15.495074+0000","last_peered":"2023-10-06T03:33:15.495074+0000","last_clean":"2023-10-06T03:33:15.495074+0000","last_became_active":"2023-10-06T03:33:15.491847+0000","last_became_peered":"2023-10-06T03:33:15.491847+0000","last_unstale":"2023-10-06T03:33:15.495074+0000","last_undegraded":"2023-10-06T03:33:15.495074+0000","last_fullsized":"2023-10-06T03:33:15.495074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:30.660724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496120+0000","last_change":"2023-10-06T03:33:15.492652+0000","last_active":"2023-10-06T03:33:15.496120+0000","last_peered":"2023-10-06T03:33:15.496120+0000","last_clean":"2023-10-06T03:33:15.496120+0000","last_became_active":"2023-10-06T03:33:15.491815+0000","last_became_peered":"2023-10-06T03:33:15.491815+0000","last_unstale":"2023-10-06T03:33:15.496120+0000","last_undegraded":"2023-10-06T03:33:15.496120+0000","last_fullsized":"2023-10-06T03:33:15.496120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:20.661516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.489846+0000","last_change":"2023-10-06T03:33:15.486081+0000","last_active":"2023-10-06T03:33:15.489846+0000","last_peered":"2023-10-06T03:33:15.489846+0000","last_clean":"2023-10-06T03:33:15.489846+0000","last_became_active":"2023-10-06T03:33:15.485842+0000","last_became_peered":"2023-10-06T03:33:15.485842+0000","last_unstale":"2023-10-06T03:33:15.489846+0000","last_undegraded":"2023-10-06T03:33:15.489846+0000","last_fullsized":"2023-10-06T03:33:15.489846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:16.260090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496154+0000","last_change":"2023-10-06T03:33:15.490482+0000","last_active":"2023-10-06T03:33:15.496154+0000","last_peered":"2023-10-06T03:33:15.496154+0000","last_clean":"2023-10-06T03:33:15.496154+0000","last_became_active":"2023-10-06T03:33:15.490185+0000","last_became_peered":"2023-10-06T03:33:15.490185+0000","last_unstale":"2023-10-06T03:33:15.496154+0000","last_undegraded":"2023-10-06T03:33:15.496154+0000","last_fullsized":"2023-10-06T03:33:15.496154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:05.828284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502137+0000","last_change":"2023-10-06T03:33:15.499609+0000","last_active":"2023-10-06T03:33:15.502137+0000","last_peered":"2023-10-06T03:33:15.502137+0000","last_clean":"2023-10-06T03:33:15.502137+0000","last_became_active":"2023-10-06T03:33:15.499317+0000","last_became_peered":"2023-10-06T03:33:15.499317+0000","last_unstale":"2023-10-06T03:33:15.502137+0000","last_undegraded":"2023-10-06T03:33:15.502137+0000","last_fullsized":"2023-10-06T03:33:15.502137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494425+0000","last_change":"2023-10-06T03:33:15.491198+0000","last_active":"2023-10-06T03:33:15.494425+0000","last_peered":"2023-10-06T03:33:15.494425+0000","last_clean":"2023-10-06T03:33:15.494425+0000","last_became_active":"2023-10-06T03:33:15.490933+0000","last_became_peered":"2023-10-06T03:33:15.490933+0000","last_unstale":"2023-10-06T03:33:15.494425+0000","last_undegraded":"2023-10-06T03:33:15.494425+0000","last_fullsized":"2023-10-06T03:33:15.494425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500604+0000","last_change":"2023-10-06T03:33:15.492259+0000","last_active":"2023-10-06T03:33:15.500604+0000","last_peered":"2023-10-06T03:33:15.500604+0000","last_clean":"2023-10-06T03:33:15.500604+0000","last_became_active":"2023-10-06T03:33:15.492018+0000","last_became_peered":"2023-10-06T03:33:15.492018+0000","last_unstale":"2023-10-06T03:33:15.500604+0000","last_undegraded":"2023-10-06T03:33:15.500604+0000","last_fullsized":"2023-10-06T03:33:15.500604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:31:34.027062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501954+0000","last_change":"2023-10-06T03:33:15.500133+0000","last_active":"2023-10-06T03:33:15.501954+0000","last_peered":"2023-10-06T03:33:15.501954+0000","last_clean":"2023-10-06T03:33:15.501954+0000","last_became_active":"2023-10-06T03:33:15.499092+0000","last_became_peered":"2023-10-06T03:33:15.499092+0000","last_unstale":"2023-10-06T03:33:15.501954+0000","last_undegraded":"2023-10-06T03:33:15.501954+0000","last_fullsized":"2023-10-06T03:33:15.501954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493234+0000","last_change":"2023-10-06T03:33:15.487133+0000","last_active":"2023-10-06T03:33:15.493234+0000","last_peered":"2023-10-06T03:33:15.493234+0000","last_clean":"2023-10-06T03:33:15.493234+0000","last_became_active":"2023-10-06T03:33:15.486582+0000","last_became_peered":"2023-10-06T03:33:15.486582+0000","last_unstale":"2023-10-06T03:33:15.493234+0000","last_undegraded":"2023-10-06T03:33:15.493234+0000","last_fullsized":"2023-10-06T03:33:15.493234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:37.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498958+0000","last_change":"2023-10-06T03:33:15.489677+0000","last_active":"2023-10-06T03:33:15.498958+0000","last_peered":"2023-10-06T03:33:15.498958+0000","last_clean":"2023-10-06T03:33:15.498958+0000","last_became_active":"2023-10-06T03:33:15.489431+0000","last_became_peered":"2023-10-06T03:33:15.489431+0000","last_unstale":"2023-10-06T03:33:15.498958+0000","last_undegraded":"2023-10-06T03:33:15.498958+0000","last_fullsized":"2023-10-06T03:33:15.498958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502491+0000","last_change":"2023-10-06T03:33:15.498305+0000","last_active":"2023-10-06T03:33:15.502491+0000","last_peered":"2023-10-06T03:33:15.502491+0000","last_clean":"2023-10-06T03:33:15.502491+0000","last_became_active":"2023-10-06T03:33:15.497920+0000","last_became_peered":"2023-10-06T03:33:15.497920+0000","last_unstale":"2023-10-06T03:33:15.502491+0000","last_undegraded":"2023-10-06T03:33:15.502491+0000","last_fullsized":"2023-10-06T03:33:15.502491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.880457+0000","last_change":"2023-10-06T03:33:14.894963+0000","last_active":"2023-10-06T03:33:15.880457+0000","last_peered":"2023-10-06T03:33:15.880457+0000","last_clean":"2023-10-06T03:33:15.880457+0000","last_became_active":"2023-10-06T03:33:14.886952+0000","last_became_peered":"2023-10-06T03:33:14.886952+0000","last_unstale":"2023-10-06T03:33:15.880457+0000","last_undegraded":"2023-10-06T03:33:15.880457+0000","last_fullsized":"2023-10-06T03:33:15.880457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501009+0000","last_change":"2023-10-06T03:33:15.498264+0000","last_active":"2023-10-06T03:33:15.501009+0000","last_peered":"2023-10-06T03:33:15.501009+0000","last_clean":"2023-10-06T03:33:15.501009+0000","last_became_active":"2023-10-06T03:33:15.498030+0000","last_became_peered":"2023-10-06T03:33:15.498030+0000","last_unstale":"2023-10-06T03:33:15.501009+0000","last_undegraded":"2023-10-06T03:33:15.501009+0000","last_fullsized":"2023-10-06T03:33:15.501009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:05.828284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.473505+0000","last_change":"2023-10-06T03:33:14.895404+0000","last_active":"2023-10-06T03:33:15.473505+0000","last_peered":"2023-10-06T03:33:15.473505+0000","last_clean":"2023-10-06T03:33:15.473505+0000","last_became_active":"2023-10-06T03:33:14.894742+0000","last_became_peered":"2023-10-06T03:33:14.894742+0000","last_unstale":"2023-10-06T03:33:15.473505+0000","last_undegraded":"2023-10-06T03:33:15.473505+0000","last_fullsized":"2023-10-06T03:33:15.473505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490157+0000","last_change":"2023-10-06T03:33:15.485976+0000","last_active":"2023-10-06T03:33:15.490157+0000","last_peered":"2023-10-06T03:33:15.490157+0000","last_clean":"2023-10-06T03:33:15.490157+0000","last_became_active":"2023-10-06T03:33:15.485658+0000","last_became_peered":"2023-10-06T03:33:15.485658+0000","last_unstale":"2023-10-06T03:33:15.490157+0000","last_undegraded":"2023-10-06T03:33:15.490157+0000","last_fullsized":"2023-10-06T03:33:15.490157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482088+0000","last_change":"2023-10-06T03:33:14.903103+0000","last_active":"2023-10-06T03:33:15.482088+0000","last_peered":"2023-10-06T03:33:15.482088+0000","last_clean":"2023-10-06T03:33:15.482088+0000","last_became_active":"2023-10-06T03:33:14.902707+0000","last_became_peered":"2023-10-06T03:33:14.902707+0000","last_unstale":"2023-10-06T03:33:15.482088+0000","last_undegraded":"2023-10-06T03:33:15.482088+0000","last_fullsized":"2023-10-06T03:33:15.482088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501406+0000","last_change":"2023-10-06T03:33:15.496437+0000","last_active":"2023-10-06T03:33:15.501406+0000","last_peered":"2023-10-06T03:33:15.501406+0000","last_clean":"2023-10-06T03:33:15.501406+0000","last_became_active":"2023-10-06T03:33:15.496138+0000","last_became_peered":"2023-10-06T03:33:15.496138+0000","last_unstale":"2023-10-06T03:33:15.501406+0000","last_undegraded":"2023-10-06T03:33:15.501406+0000","last_fullsized":"2023-10-06T03:33:15.501406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:09:05.289396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.478575+0000","last_change":"2023-10-06T03:33:14.897019+0000","last_active":"2023-10-06T03:33:15.478575+0000","last_peered":"2023-10-06T03:33:15.478575+0000","last_clean":"2023-10-06T03:33:15.478575+0000","last_became_active":"2023-10-06T03:33:14.896740+0000","last_became_peered":"2023-10-06T03:33:14.896740+0000","last_unstale":"2023-10-06T03:33:15.478575+0000","last_undegraded":"2023-10-06T03:33:15.478575+0000","last_fullsized":"2023-10-06T03:33:15.478575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500886+0000","last_change":"2023-10-06T03:33:15.496977+0000","last_active":"2023-10-06T03:33:15.500886+0000","last_peered":"2023-10-06T03:33:15.500886+0000","last_clean":"2023-10-06T03:33:15.500886+0000","last_became_active":"2023-10-06T03:33:15.496726+0000","last_became_peered":"2023-10-06T03:33:15.496726+0000","last_unstale":"2023-10-06T03:33:15.500886+0000","last_undegraded":"2023-10-06T03:33:15.500886+0000","last_fullsized":"2023-10-06T03:33:15.500886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:02.889905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.491600+0000","last_change":"2023-10-06T03:33:14.896854+0000","last_active":"2023-10-06T03:33:15.491600+0000","last_peered":"2023-10-06T03:33:15.491600+0000","last_clean":"2023-10-06T03:33:15.491600+0000","last_became_active":"2023-10-06T03:33:14.896606+0000","last_became_peered":"2023-10-06T03:33:14.896606+0000","last_unstale":"2023-10-06T03:33:15.491600+0000","last_undegraded":"2023-10-06T03:33:15.491600+0000","last_fullsized":"2023-10-06T03:33:15.491600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500655+0000","last_change":"2023-10-06T03:33:15.494940+0000","last_active":"2023-10-06T03:33:15.500655+0000","last_peered":"2023-10-06T03:33:15.500655+0000","last_clean":"2023-10-06T03:33:15.500655+0000","last_became_active":"2023-10-06T03:33:15.494251+0000","last_became_peered":"2023-10-06T03:33:15.494251+0000","last_unstale":"2023-10-06T03:33:15.500655+0000","last_undegraded":"2023-10-06T03:33:15.500655+0000","last_fullsized":"2023-10-06T03:33:15.500655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.885266+0000","last_change":"2023-10-06T03:33:14.895082+0000","last_active":"2023-10-06T03:33:15.885266+0000","last_peered":"2023-10-06T03:33:15.885266+0000","last_clean":"2023-10-06T03:33:15.885266+0000","last_became_active":"2023-10-06T03:33:14.894804+0000","last_became_peered":"2023-10-06T03:33:14.894804+0000","last_unstale":"2023-10-06T03:33:15.885266+0000","last_undegraded":"2023-10-06T03:33:15.885266+0000","last_fullsized":"2023-10-06T03:33:15.885266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:01.585501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493308+0000","last_change":"2023-10-06T03:33:15.487049+0000","last_active":"2023-10-06T03:33:15.493308+0000","last_peered":"2023-10-06T03:33:15.493308+0000","last_clean":"2023-10-06T03:33:15.493308+0000","last_became_active":"2023-10-06T03:33:15.486777+0000","last_became_peered":"2023-10-06T03:33:15.486777+0000","last_unstale":"2023-10-06T03:33:15.493308+0000","last_undegraded":"2023-10-06T03:33:15.493308+0000","last_fullsized":"2023-10-06T03:33:15.493308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:12.635649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.888581+0000","last_change":"2023-10-06T03:33:14.898475+0000","last_active":"2023-10-06T03:33:15.888581+0000","last_peered":"2023-10-06T03:33:15.888581+0000","last_clean":"2023-10-06T03:33:15.888581+0000","last_became_active":"2023-10-06T03:33:14.898144+0000","last_became_peered":"2023-10-06T03:33:14.898144+0000","last_unstale":"2023-10-06T03:33:15.888581+0000","last_undegraded":"2023-10-06T03:33:15.888581+0000","last_fullsized":"2023-10-06T03:33:15.888581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494412+0000","last_change":"2023-10-06T03:33:15.487549+0000","last_active":"2023-10-06T03:33:15.494412+0000","last_peered":"2023-10-06T03:33:15.494412+0000","last_clean":"2023-10-06T03:33:15.494412+0000","last_became_active":"2023-10-06T03:33:15.487277+0000","last_became_peered":"2023-10-06T03:33:15.487277+0000","last_unstale":"2023-10-06T03:33:15.494412+0000","last_undegraded":"2023-10-06T03:33:15.494412+0000","last_fullsized":"2023-10-06T03:33:15.494412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:07.613983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.885740+0000","last_change":"2023-10-06T03:33:14.896271+0000","last_active":"2023-10-06T03:33:15.885740+0000","last_peered":"2023-10-06T03:33:15.885740+0000","last_clean":"2023-10-06T03:33:15.885740+0000","last_became_active":"2023-10-06T03:33:14.896004+0000","last_became_peered":"2023-10-06T03:33:14.896004+0000","last_unstale":"2023-10-06T03:33:15.885740+0000","last_undegraded":"2023-10-06T03:33:15.885740+0000","last_fullsized":"2023-10-06T03:33:15.885740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:15.567270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500942+0000","last_change":"2023-10-06T03:33:15.497569+0000","last_active":"2023-10-06T03:33:15.500942+0000","last_peered":"2023-10-06T03:33:15.500942+0000","last_clean":"2023-10-06T03:33:15.500942+0000","last_became_active":"2023-10-06T03:33:15.497287+0000","last_became_peered":"2023-10-06T03:33:15.497287+0000","last_unstale":"2023-10-06T03:33:15.500942+0000","last_undegraded":"2023-10-06T03:33:15.500942+0000","last_fullsized":"2023-10-06T03:33:15.500942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:47.107849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.476556+0000","last_change":"2023-10-06T03:33:14.897199+0000","last_active":"2023-10-06T03:33:15.476556+0000","last_peered":"2023-10-06T03:33:15.476556+0000","last_clean":"2023-10-06T03:33:15.476556+0000","last_became_active":"2023-10-06T03:33:14.896925+0000","last_became_peered":"2023-10-06T03:33:14.896925+0000","last_unstale":"2023-10-06T03:33:15.476556+0000","last_undegraded":"2023-10-06T03:33:15.476556+0000","last_fullsized":"2023-10-06T03:33:15.476556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:41.510001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498210+0000","last_change":"2023-10-06T03:33:15.495381+0000","last_active":"2023-10-06T03:33:15.498210+0000","last_peered":"2023-10-06T03:33:15.498210+0000","last_clean":"2023-10-06T03:33:15.498210+0000","last_became_active":"2023-10-06T03:33:15.494972+0000","last_became_peered":"2023-10-06T03:33:15.494972+0000","last_unstale":"2023-10-06T03:33:15.498210+0000","last_undegraded":"2023-10-06T03:33:15.498210+0000","last_fullsized":"2023-10-06T03:33:15.498210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.483344+0000","last_change":"2023-10-06T03:33:14.901135+0000","last_active":"2023-10-06T03:33:15.483344+0000","last_peered":"2023-10-06T03:33:15.483344+0000","last_clean":"2023-10-06T03:33:15.483344+0000","last_became_active":"2023-10-06T03:33:14.900871+0000","last_became_peered":"2023-10-06T03:33:14.900871+0000","last_unstale":"2023-10-06T03:33:15.483344+0000","last_undegraded":"2023-10-06T03:33:15.483344+0000","last_fullsized":"2023-10-06T03:33:15.483344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494496+0000","last_change":"2023-10-06T03:33:15.491822+0000","last_active":"2023-10-06T03:33:15.494496+0000","last_peered":"2023-10-06T03:33:15.494496+0000","last_clean":"2023-10-06T03:33:15.494496+0000","last_became_active":"2023-10-06T03:33:15.491560+0000","last_became_peered":"2023-10-06T03:33:15.491560+0000","last_unstale":"2023-10-06T03:33:15.494496+0000","last_undegraded":"2023-10-06T03:33:15.494496+0000","last_fullsized":"2023-10-06T03:33:15.494496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.964350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886225+0000","last_change":"2023-10-06T03:33:14.898845+0000","last_active":"2023-10-06T03:33:15.886225+0000","last_peered":"2023-10-06T03:33:15.886225+0000","last_clean":"2023-10-06T03:33:15.886225+0000","last_became_active":"2023-10-06T03:33:14.898491+0000","last_became_peered":"2023-10-06T03:33:14.898491+0000","last_unstale":"2023-10-06T03:33:15.886225+0000","last_undegraded":"2023-10-06T03:33:15.886225+0000","last_fullsized":"2023-10-06T03:33:15.886225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500404+0000","last_change":"2023-10-06T03:33:15.487774+0000","last_active":"2023-10-06T03:33:15.500404+0000","last_peered":"2023-10-06T03:33:15.500404+0000","last_clean":"2023-10-06T03:33:15.500404+0000","last_became_active":"2023-10-06T03:33:15.487515+0000","last_became_peered":"2023-10-06T03:33:15.487515+0000","last_unstale":"2023-10-06T03:33:15.500404+0000","last_undegraded":"2023-10-06T03:33:15.500404+0000","last_fullsized":"2023-10-06T03:33:15.500404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:30.660724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.884947+0000","last_change":"2023-10-06T03:33:14.896171+0000","last_active":"2023-10-06T03:33:15.884947+0000","last_peered":"2023-10-06T03:33:15.884947+0000","last_clean":"2023-10-06T03:33:15.884947+0000","last_became_active":"2023-10-06T03:33:14.895903+0000","last_became_peered":"2023-10-06T03:33:14.895903+0000","last_unstale":"2023-10-06T03:33:15.884947+0000","last_undegraded":"2023-10-06T03:33:15.884947+0000","last_fullsized":"2023-10-06T03:33:15.884947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:44.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500464+0000","last_change":"2023-10-06T03:33:15.496285+0000","last_active":"2023-10-06T03:33:15.500464+0000","last_peered":"2023-10-06T03:33:15.500464+0000","last_clean":"2023-10-06T03:33:15.500464+0000","last_became_active":"2023-10-06T03:33:15.495868+0000","last_became_peered":"2023-10-06T03:33:15.495868+0000","last_unstale":"2023-10-06T03:33:15.500464+0000","last_undegraded":"2023-10-06T03:33:15.500464+0000","last_fullsized":"2023-10-06T03:33:15.500464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.885984+0000","last_change":"2023-10-06T03:33:14.896322+0000","last_active":"2023-10-06T03:33:15.885984+0000","last_peered":"2023-10-06T03:33:15.885984+0000","last_clean":"2023-10-06T03:33:15.885984+0000","last_became_active":"2023-10-06T03:33:14.896022+0000","last_became_peered":"2023-10-06T03:33:14.896022+0000","last_unstale":"2023-10-06T03:33:15.885984+0000","last_undegraded":"2023-10-06T03:33:15.885984+0000","last_fullsized":"2023-10-06T03:33:15.885984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:21.680218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500853+0000","last_change":"2023-10-06T03:33:15.494878+0000","last_active":"2023-10-06T03:33:15.500853+0000","last_peered":"2023-10-06T03:33:15.500853+0000","last_clean":"2023-10-06T03:33:15.500853+0000","last_became_active":"2023-10-06T03:33:15.494597+0000","last_became_peered":"2023-10-06T03:33:15.494597+0000","last_unstale":"2023-10-06T03:33:15.500853+0000","last_undegraded":"2023-10-06T03:33:15.500853+0000","last_fullsized":"2023-10-06T03:33:15.500853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:30.660724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.887336+0000","last_change":"2023-10-06T03:33:14.897659+0000","last_active":"2023-10-06T03:33:15.887336+0000","last_peered":"2023-10-06T03:33:15.887336+0000","last_clean":"2023-10-06T03:33:15.887336+0000","last_became_active":"2023-10-06T03:33:14.897368+0000","last_became_peered":"2023-10-06T03:33:14.897368+0000","last_unstale":"2023-10-06T03:33:15.887336+0000","last_undegraded":"2023-10-06T03:33:15.887336+0000","last_fullsized":"2023-10-06T03:33:15.887336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:46:12.361089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493933+0000","last_change":"2023-10-06T03:33:15.492136+0000","last_active":"2023-10-06T03:33:15.493933+0000","last_peered":"2023-10-06T03:33:15.493933+0000","last_clean":"2023-10-06T03:33:15.493933+0000","last_became_active":"2023-10-06T03:33:15.491858+0000","last_became_peered":"2023-10-06T03:33:15.491858+0000","last_unstale":"2023-10-06T03:33:15.493933+0000","last_undegraded":"2023-10-06T03:33:15.493933+0000","last_fullsized":"2023-10-06T03:33:15.493933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:16.576653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490953+0000","last_change":"2023-10-06T03:33:14.890596+0000","last_active":"2023-10-06T03:33:15.490953+0000","last_peered":"2023-10-06T03:33:15.490953+0000","last_clean":"2023-10-06T03:33:15.490953+0000","last_became_active":"2023-10-06T03:33:14.889486+0000","last_became_peered":"2023-10-06T03:33:14.889486+0000","last_unstale":"2023-10-06T03:33:15.490953+0000","last_undegraded":"2023-10-06T03:33:15.490953+0000","last_fullsized":"2023-10-06T03:33:15.490953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:41.510001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.495179+0000","last_change":"2023-10-06T03:33:15.493860+0000","last_active":"2023-10-06T03:33:15.495179+0000","last_peered":"2023-10-06T03:33:15.495179+0000","last_clean":"2023-10-06T03:33:15.495179+0000","last_became_active":"2023-10-06T03:33:15.493610+0000","last_became_peered":"2023-10-06T03:33:15.493610+0000","last_unstale":"2023-10-06T03:33:15.495179+0000","last_undegraded":"2023-10-06T03:33:15.495179+0000","last_fullsized":"2023-10-06T03:33:15.495179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.881042+0000","last_change":"2023-10-06T03:33:14.903094+0000","last_active":"2023-10-06T03:33:15.881042+0000","last_peered":"2023-10-06T03:33:15.881042+0000","last_clean":"2023-10-06T03:33:15.881042+0000","last_became_active":"2023-10-06T03:33:14.902686+0000","last_became_peered":"2023-10-06T03:33:14.902686+0000","last_unstale":"2023-10-06T03:33:15.881042+0000","last_undegraded":"2023-10-06T03:33:15.881042+0000","last_fullsized":"2023-10-06T03:33:15.881042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496134+0000","last_change":"2023-10-06T03:33:15.489915+0000","last_active":"2023-10-06T03:33:15.496134+0000","last_peered":"2023-10-06T03:33:15.496134+0000","last_clean":"2023-10-06T03:33:15.496134+0000","last_became_active":"2023-10-06T03:33:15.488544+0000","last_became_peered":"2023-10-06T03:33:15.488544+0000","last_unstale":"2023-10-06T03:33:15.496134+0000","last_undegraded":"2023-10-06T03:33:15.496134+0000","last_fullsized":"2023-10-06T03:33:15.496134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.478249+0000","last_change":"2023-10-06T03:33:14.886562+0000","last_active":"2023-10-06T03:33:15.478249+0000","last_peered":"2023-10-06T03:33:15.478249+0000","last_clean":"2023-10-06T03:33:15.478249+0000","last_became_active":"2023-10-06T03:33:14.886157+0000","last_became_peered":"2023-10-06T03:33:14.886157+0000","last_unstale":"2023-10-06T03:33:15.478249+0000","last_undegraded":"2023-10-06T03:33:15.478249+0000","last_fullsized":"2023-10-06T03:33:15.478249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496415+0000","last_change":"2023-10-06T03:33:15.491844+0000","last_active":"2023-10-06T03:33:15.496415+0000","last_peered":"2023-10-06T03:33:15.496415+0000","last_clean":"2023-10-06T03:33:15.496415+0000","last_became_active":"2023-10-06T03:33:15.491574+0000","last_became_peered":"2023-10-06T03:33:15.491574+0000","last_unstale":"2023-10-06T03:33:15.496415+0000","last_undegraded":"2023-10-06T03:33:15.496415+0000","last_fullsized":"2023-10-06T03:33:15.496415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:35.426668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482572+0000","last_change":"2023-10-06T03:33:14.895658+0000","last_active":"2023-10-06T03:33:15.482572+0000","last_peered":"2023-10-06T03:33:15.482572+0000","last_clean":"2023-10-06T03:33:15.482572+0000","last_became_active":"2023-10-06T03:33:14.895332+0000","last_became_peered":"2023-10-06T03:33:14.895332+0000","last_unstale":"2023-10-06T03:33:15.482572+0000","last_undegraded":"2023-10-06T03:33:15.482572+0000","last_fullsized":"2023-10-06T03:33:15.482572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:28.684332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496089+0000","last_change":"2023-10-06T03:33:15.490478+0000","last_active":"2023-10-06T03:33:15.496089+0000","last_peered":"2023-10-06T03:33:15.496089+0000","last_clean":"2023-10-06T03:33:15.496089+0000","last_became_active":"2023-10-06T03:33:15.485974+0000","last_became_peered":"2023-10-06T03:33:15.485974+0000","last_unstale":"2023-10-06T03:33:15.496089+0000","last_undegraded":"2023-10-06T03:33:15.496089+0000","last_fullsized":"2023-10-06T03:33:15.496089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.481946+0000","last_change":"2023-10-06T03:33:14.891288+0000","last_active":"2023-10-06T03:33:15.481946+0000","last_peered":"2023-10-06T03:33:15.481946+0000","last_clean":"2023-10-06T03:33:15.481946+0000","last_became_active":"2023-10-06T03:33:14.890186+0000","last_became_peered":"2023-10-06T03:33:14.890186+0000","last_unstale":"2023-10-06T03:33:15.481946+0000","last_undegraded":"2023-10-06T03:33:15.481946+0000","last_fullsized":"2023-10-06T03:33:15.481946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:21.680218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493182+0000","last_change":"2023-10-06T03:33:15.486956+0000","last_active":"2023-10-06T03:33:15.493182+0000","last_peered":"2023-10-06T03:33:15.493182+0000","last_clean":"2023-10-06T03:33:15.493182+0000","last_became_active":"2023-10-06T03:33:15.486188+0000","last_became_peered":"2023-10-06T03:33:15.486188+0000","last_unstale":"2023-10-06T03:33:15.493182+0000","last_undegraded":"2023-10-06T03:33:15.493182+0000","last_fullsized":"2023-10-06T03:33:15.493182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.887149+0000","last_change":"2023-10-06T03:33:14.895426+0000","last_active":"2023-10-06T03:33:15.887149+0000","last_peered":"2023-10-06T03:33:15.887149+0000","last_clean":"2023-10-06T03:33:15.887149+0000","last_became_active":"2023-10-06T03:33:14.893425+0000","last_became_peered":"2023-10-06T03:33:14.893425+0000","last_unstale":"2023-10-06T03:33:15.887149+0000","last_undegraded":"2023-10-06T03:33:15.887149+0000","last_fullsized":"2023-10-06T03:33:15.887149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:44.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.484841+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:33:15.484841+0000","last_peered":"2023-10-06T03:33:15.484841+0000","last_clean":"2023-10-06T03:33:15.484841+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:33:15.484841+0000","last_undegraded":"2023-10-06T03:33:15.484841+0000","last_fullsized":"2023-10-06T03:33:15.484841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500598+0000","last_change":"2023-10-06T03:33:15.492324+0000","last_active":"2023-10-06T03:33:15.500598+0000","last_peered":"2023-10-06T03:33:15.500598+0000","last_clean":"2023-10-06T03:33:15.500598+0000","last_became_active":"2023-10-06T03:33:15.492037+0000","last_became_peered":"2023-10-06T03:33:15.492037+0000","last_unstale":"2023-10-06T03:33:15.500598+0000","last_undegraded":"2023-10-06T03:33:15.500598+0000","last_fullsized":"2023-10-06T03:33:15.500598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.484831+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:33:15.484831+0000","last_peered":"2023-10-06T03:33:15.484831+0000","last_clean":"2023-10-06T03:33:15.484831+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:33:15.484831+0000","last_undegraded":"2023-10-06T03:33:15.484831+0000","last_fullsized":"2023-10-06T03:33:15.484831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:33:16.884845+0000","last_change":"2023-10-06T03:33:14.896368+0000","last_active":"2023-10-06T03:33:16.884845+0000","last_peered":"2023-10-06T03:33:16.884845+0000","last_clean":"2023-10-06T03:33:16.884845+0000","last_became_active":"2023-10-06T03:33:14.892459+0000","last_became_peered":"2023-10-06T03:33:14.892459+0000","last_unstale":"2023-10-06T03:33:16.884845+0000","last_undegraded":"2023-10-06T03:33:16.884845+0000","last_fullsized":"2023-10-06T03:33:16.884845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501686+0000","last_change":"2023-10-06T03:33:15.499324+0000","last_active":"2023-10-06T03:33:15.501686+0000","last_peered":"2023-10-06T03:33:15.501686+0000","last_clean":"2023-10-06T03:33:15.501686+0000","last_became_active":"2023-10-06T03:33:15.499039+0000","last_became_peered":"2023-10-06T03:33:15.499039+0000","last_unstale":"2023-10-06T03:33:15.501686+0000","last_undegraded":"2023-10-06T03:33:15.501686+0000","last_fullsized":"2023-10-06T03:33:15.501686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:43.269972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.473587+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:33:15.473587+0000","last_peered":"2023-10-06T03:33:15.473587+0000","last_clean":"2023-10-06T03:33:15.473587+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:33:15.473587+0000","last_undegraded":"2023-10-06T03:33:15.473587+0000","last_fullsized":"2023-10-06T03:33:15.473587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.483628+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:33:15.483628+0000","last_peered":"2023-10-06T03:33:15.483628+0000","last_clean":"2023-10-06T03:33:15.483628+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:33:15.483628+0000","last_undegraded":"2023-10-06T03:33:15.483628+0000","last_fullsized":"2023-10-06T03:33:15.483628+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.476509+0000","last_change":"2023-10-06T03:33:14.891206+0000","last_active":"2023-10-06T03:33:15.476509+0000","last_peered":"2023-10-06T03:33:15.476509+0000","last_clean":"2023-10-06T03:33:15.476509+0000","last_became_active":"2023-10-06T03:33:14.890823+0000","last_became_peered":"2023-10-06T03:33:14.890823+0000","last_unstale":"2023-10-06T03:33:15.476509+0000","last_undegraded":"2023-10-06T03:33:15.476509+0000","last_fullsized":"2023-10-06T03:33:15.476509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:28.684332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490137+0000","last_change":"2023-10-06T03:33:15.483582+0000","last_active":"2023-10-06T03:33:15.490137+0000","last_peered":"2023-10-06T03:33:15.490137+0000","last_clean":"2023-10-06T03:33:15.490137+0000","last_became_active":"2023-10-06T03:33:15.483324+0000","last_became_peered":"2023-10-06T03:33:15.483324+0000","last_unstale":"2023-10-06T03:33:15.490137+0000","last_undegraded":"2023-10-06T03:33:15.490137+0000","last_fullsized":"2023-10-06T03:33:15.490137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:12.635649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482905+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:33:15.482905+0000","last_peered":"2023-10-06T03:33:15.482905+0000","last_clean":"2023-10-06T03:33:15.482905+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:33:15.482905+0000","last_undegraded":"2023-10-06T03:33:15.482905+0000","last_fullsized":"2023-10-06T03:33:15.482905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.474679+0000","last_change":"2023-10-06T03:33:14.902096+0000","last_active":"2023-10-06T03:33:15.474679+0000","last_peered":"2023-10-06T03:33:15.474679+0000","last_clean":"2023-10-06T03:33:15.474679+0000","last_became_active":"2023-10-06T03:33:14.896599+0000","last_became_peered":"2023-10-06T03:33:14.896599+0000","last_unstale":"2023-10-06T03:33:15.474679+0000","last_undegraded":"2023-10-06T03:33:15.474679+0000","last_fullsized":"2023-10-06T03:33:15.474679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502084+0000","last_change":"2023-10-06T03:33:15.500022+0000","last_active":"2023-10-06T03:33:15.502084+0000","last_peered":"2023-10-06T03:33:15.502084+0000","last_clean":"2023-10-06T03:33:15.502084+0000","last_became_active":"2023-10-06T03:33:15.499610+0000","last_became_peered":"2023-10-06T03:33:15.499610+0000","last_unstale":"2023-10-06T03:33:15.502084+0000","last_undegraded":"2023-10-06T03:33:15.502084+0000","last_fullsized":"2023-10-06T03:33:15.502084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:58.092449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.483163+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:33:15.483163+0000","last_peered":"2023-10-06T03:33:15.483163+0000","last_clean":"2023-10-06T03:33:15.483163+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:33:15.483163+0000","last_undegraded":"2023-10-06T03:33:15.483163+0000","last_fullsized":"2023-10-06T03:33:15.483163+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482546+0000","last_change":"2023-10-06T03:33:14.895562+0000","last_active":"2023-10-06T03:33:15.482546+0000","last_peered":"2023-10-06T03:33:15.482546+0000","last_clean":"2023-10-06T03:33:15.482546+0000","last_became_active":"2023-10-06T03:33:14.893552+0000","last_became_peered":"2023-10-06T03:33:14.893552+0000","last_unstale":"2023-10-06T03:33:15.482546+0000","last_undegraded":"2023-10-06T03:33:15.482546+0000","last_fullsized":"2023-10-06T03:33:15.482546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:41.510001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498193+0000","last_change":"2023-10-06T03:33:15.495951+0000","last_active":"2023-10-06T03:33:15.498193+0000","last_peered":"2023-10-06T03:33:15.498193+0000","last_clean":"2023-10-06T03:33:15.498193+0000","last_became_active":"2023-10-06T03:33:15.495675+0000","last_became_peered":"2023-10-06T03:33:15.495675+0000","last_unstale":"2023-10-06T03:33:15.498193+0000","last_undegraded":"2023-10-06T03:33:15.498193+0000","last_fullsized":"2023-10-06T03:33:15.498193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:46:13.370472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490686+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:33:15.490686+0000","last_peered":"2023-10-06T03:33:15.490686+0000","last_clean":"2023-10-06T03:33:15.490686+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:33:15.490686+0000","last_undegraded":"2023-10-06T03:33:15.490686+0000","last_fullsized":"2023-10-06T03:33:15.490686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.480469+0000","last_change":"2023-10-06T03:33:14.901103+0000","last_active":"2023-10-06T03:33:15.480469+0000","last_peered":"2023-10-06T03:33:15.480469+0000","last_clean":"2023-10-06T03:33:15.480469+0000","last_became_active":"2023-10-06T03:33:14.900784+0000","last_became_peered":"2023-10-06T03:33:14.900784+0000","last_unstale":"2023-10-06T03:33:15.480469+0000","last_undegraded":"2023-10-06T03:33:15.480469+0000","last_fullsized":"2023-10-06T03:33:15.480469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493280+0000","last_change":"2023-10-06T03:33:15.488395+0000","last_active":"2023-10-06T03:33:15.493280+0000","last_peered":"2023-10-06T03:33:15.493280+0000","last_clean":"2023-10-06T03:33:15.493280+0000","last_became_active":"2023-10-06T03:33:15.483269+0000","last_became_peered":"2023-10-06T03:33:15.483269+0000","last_unstale":"2023-10-06T03:33:15.493280+0000","last_undegraded":"2023-10-06T03:33:15.493280+0000","last_fullsized":"2023-10-06T03:33:15.493280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.477353+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:33:15.477353+0000","last_peered":"2023-10-06T03:33:15.477353+0000","last_clean":"2023-10-06T03:33:15.477353+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:33:15.477353+0000","last_undegraded":"2023-10-06T03:33:15.477353+0000","last_fullsized":"2023-10-06T03:33:15.477353+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.474703+0000","last_change":"2023-10-06T03:33:14.896667+0000","last_active":"2023-10-06T03:33:15.474703+0000","last_peered":"2023-10-06T03:33:15.474703+0000","last_clean":"2023-10-06T03:33:15.474703+0000","last_became_active":"2023-10-06T03:33:14.895890+0000","last_became_peered":"2023-10-06T03:33:14.895890+0000","last_unstale":"2023-10-06T03:33:15.474703+0000","last_undegraded":"2023-10-06T03:33:15.474703+0000","last_fullsized":"2023-10-06T03:33:15.474703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:28.684332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498919+0000","last_change":"2023-10-06T03:33:15.496955+0000","last_active":"2023-10-06T03:33:15.498919+0000","last_peered":"2023-10-06T03:33:15.498919+0000","last_clean":"2023-10-06T03:33:15.498919+0000","last_became_active":"2023-10-06T03:33:15.496666+0000","last_became_peered":"2023-10-06T03:33:15.496666+0000","last_unstale":"2023-10-06T03:33:15.498919+0000","last_undegraded":"2023-10-06T03:33:15.498919+0000","last_fullsized":"2023-10-06T03:33:15.498919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.478419+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:33:15.478419+0000","last_peered":"2023-10-06T03:33:15.478419+0000","last_clean":"2023-10-06T03:33:15.478419+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:33:15.478419+0000","last_undegraded":"2023-10-06T03:33:15.478419+0000","last_fullsized":"2023-10-06T03:33:15.478419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.491835+0000","last_change":"2023-10-06T03:33:14.893120+0000","last_active":"2023-10-06T03:33:15.491835+0000","last_peered":"2023-10-06T03:33:15.491835+0000","last_clean":"2023-10-06T03:33:15.491835+0000","last_became_active":"2023-10-06T03:33:14.892839+0000","last_became_peered":"2023-10-06T03:33:14.892839+0000","last_unstale":"2023-10-06T03:33:15.491835+0000","last_undegraded":"2023-10-06T03:33:15.491835+0000","last_fullsized":"2023-10-06T03:33:15.491835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:56.088527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501655+0000","last_change":"2023-10-06T03:33:15.496296+0000","last_active":"2023-10-06T03:33:15.501655+0000","last_peered":"2023-10-06T03:33:15.501655+0000","last_clean":"2023-10-06T03:33:15.501655+0000","last_became_active":"2023-10-06T03:33:15.496028+0000","last_became_peered":"2023-10-06T03:33:15.496028+0000","last_unstale":"2023-10-06T03:33:15.501655+0000","last_undegraded":"2023-10-06T03:33:15.501655+0000","last_fullsized":"2023-10-06T03:33:15.501655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.884176+0000","last_change":"2023-10-06T03:33:14.897368+0000","last_active":"2023-10-06T03:33:15.884176+0000","last_peered":"2023-10-06T03:33:15.884176+0000","last_clean":"2023-10-06T03:33:15.884176+0000","last_became_active":"2023-10-06T03:33:14.892728+0000","last_became_peered":"2023-10-06T03:33:14.892728+0000","last_unstale":"2023-10-06T03:33:15.884176+0000","last_undegraded":"2023-10-06T03:33:15.884176+0000","last_fullsized":"2023-10-06T03:33:15.884176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494671+0000","last_change":"2023-10-06T03:33:15.493442+0000","last_active":"2023-10-06T03:33:15.494671+0000","last_peered":"2023-10-06T03:33:15.494671+0000","last_clean":"2023-10-06T03:33:15.494671+0000","last_became_active":"2023-10-06T03:33:15.493169+0000","last_became_peered":"2023-10-06T03:33:15.493169+0000","last_unstale":"2023-10-06T03:33:15.494671+0000","last_undegraded":"2023-10-06T03:33:15.494671+0000","last_fullsized":"2023-10-06T03:33:15.494671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:12.900415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886080+0000","last_change":"2023-10-06T03:33:14.899177+0000","last_active":"2023-10-06T03:33:15.886080+0000","last_peered":"2023-10-06T03:33:15.886080+0000","last_clean":"2023-10-06T03:33:15.886080+0000","last_became_active":"2023-10-06T03:33:14.898884+0000","last_became_peered":"2023-10-06T03:33:14.898884+0000","last_unstale":"2023-10-06T03:33:15.886080+0000","last_undegraded":"2023-10-06T03:33:15.886080+0000","last_fullsized":"2023-10-06T03:33:15.886080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501741+0000","last_change":"2023-10-06T03:33:15.499965+0000","last_active":"2023-10-06T03:33:15.501741+0000","last_peered":"2023-10-06T03:33:15.501741+0000","last_clean":"2023-10-06T03:33:15.501741+0000","last_became_active":"2023-10-06T03:33:15.499644+0000","last_became_peered":"2023-10-06T03:33:15.499644+0000","last_unstale":"2023-10-06T03:33:15.501741+0000","last_undegraded":"2023-10-06T03:33:15.501741+0000","last_fullsized":"2023-10-06T03:33:15.501741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:47.107849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886042+0000","last_change":"2023-10-06T03:33:14.898472+0000","last_active":"2023-10-06T03:33:15.886042+0000","last_peered":"2023-10-06T03:33:15.886042+0000","last_clean":"2023-10-06T03:33:15.886042+0000","last_became_active":"2023-10-06T03:33:14.898230+0000","last_became_peered":"2023-10-06T03:33:14.898230+0000","last_unstale":"2023-10-06T03:33:15.886042+0000","last_undegraded":"2023-10-06T03:33:15.886042+0000","last_fullsized":"2023-10-06T03:33:15.886042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494448+0000","last_change":"2023-10-06T03:33:15.488792+0000","last_active":"2023-10-06T03:33:15.494448+0000","last_peered":"2023-10-06T03:33:15.494448+0000","last_clean":"2023-10-06T03:33:15.494448+0000","last_became_active":"2023-10-06T03:33:15.488504+0000","last_became_peered":"2023-10-06T03:33:15.488504+0000","last_unstale":"2023-10-06T03:33:15.494448+0000","last_undegraded":"2023-10-06T03:33:15.494448+0000","last_fullsized":"2023-10-06T03:33:15.494448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:59.545999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.884688+0000","last_change":"2023-10-06T03:33:14.882297+0000","last_active":"2023-10-06T03:33:15.884688+0000","last_peered":"2023-10-06T03:33:15.884688+0000","last_clean":"2023-10-06T03:33:15.884688+0000","last_became_active":"2023-10-06T03:33:14.881990+0000","last_became_peered":"2023-10-06T03:33:14.881990+0000","last_unstale":"2023-10-06T03:33:15.884688+0000","last_undegraded":"2023-10-06T03:33:15.884688+0000","last_fullsized":"2023-10-06T03:33:15.884688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:56.088527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500582+0000","last_change":"2023-10-06T03:33:15.491458+0000","last_active":"2023-10-06T03:33:15.500582+0000","last_peered":"2023-10-06T03:33:15.500582+0000","last_clean":"2023-10-06T03:33:15.500582+0000","last_became_active":"2023-10-06T03:33:15.491167+0000","last_became_peered":"2023-10-06T03:33:15.491167+0000","last_unstale":"2023-10-06T03:33:15.500582+0000","last_undegraded":"2023-10-06T03:33:15.500582+0000","last_fullsized":"2023-10-06T03:33:15.500582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.964350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886971+0000","last_change":"2023-10-06T03:33:14.896256+0000","last_active":"2023-10-06T03:33:15.886971+0000","last_peered":"2023-10-06T03:33:15.886971+0000","last_clean":"2023-10-06T03:33:15.886971+0000","last_became_active":"2023-10-06T03:33:14.893978+0000","last_became_peered":"2023-10-06T03:33:14.893978+0000","last_unstale":"2023-10-06T03:33:15.886971+0000","last_undegraded":"2023-10-06T03:33:15.886971+0000","last_fullsized":"2023-10-06T03:33:15.886971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:56.088527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494409+0000","last_change":"2023-10-06T03:33:15.485216+0000","last_active":"2023-10-06T03:33:15.494409+0000","last_peered":"2023-10-06T03:33:15.494409+0000","last_clean":"2023-10-06T03:33:15.494409+0000","last_became_active":"2023-10-06T03:33:15.484942+0000","last_became_peered":"2023-10-06T03:33:15.484942+0000","last_unstale":"2023-10-06T03:33:15.494409+0000","last_undegraded":"2023-10-06T03:33:15.494409+0000","last_fullsized":"2023-10-06T03:33:15.494409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:50.279026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.492719+0000","last_change":"2023-10-06T03:33:15.489726+0000","last_active":"2023-10-06T03:33:15.492719+0000","last_peered":"2023-10-06T03:33:15.492719+0000","last_clean":"2023-10-06T03:33:15.492719+0000","last_became_active":"2023-10-06T03:33:15.489424+0000","last_became_peered":"2023-10-06T03:33:15.489424+0000","last_unstale":"2023-10-06T03:33:15.492719+0000","last_undegraded":"2023-10-06T03:33:15.492719+0000","last_fullsized":"2023-10-06T03:33:15.492719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:37.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500986+0000","last_change":"2023-10-06T03:33:15.497582+0000","last_active":"2023-10-06T03:33:15.500986+0000","last_peered":"2023-10-06T03:33:15.500986+0000","last_clean":"2023-10-06T03:33:15.500986+0000","last_became_active":"2023-10-06T03:33:15.497328+0000","last_became_peered":"2023-10-06T03:33:15.497328+0000","last_unstale":"2023-10-06T03:33:15.500986+0000","last_undegraded":"2023-10-06T03:33:15.500986+0000","last_fullsized":"2023-10-06T03:33:15.500986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:50.279026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496144+0000","last_change":"2023-10-06T03:33:15.489875+0000","last_active":"2023-10-06T03:33:15.496144+0000","last_peered":"2023-10-06T03:33:15.496144+0000","last_clean":"2023-10-06T03:33:15.496144+0000","last_became_active":"2023-10-06T03:33:15.489634+0000","last_became_peered":"2023-10-06T03:33:15.489634+0000","last_unstale":"2023-10-06T03:33:15.496144+0000","last_undegraded":"2023-10-06T03:33:15.496144+0000","last_fullsized":"2023-10-06T03:33:15.496144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:31:34.027062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500770+0000","last_change":"2023-10-06T03:33:15.493502+0000","last_active":"2023-10-06T03:33:15.500770+0000","last_peered":"2023-10-06T03:33:15.500770+0000","last_clean":"2023-10-06T03:33:15.500770+0000","last_became_active":"2023-10-06T03:33:15.492769+0000","last_became_peered":"2023-10-06T03:33:15.492769+0000","last_unstale":"2023-10-06T03:33:15.500770+0000","last_undegraded":"2023-10-06T03:33:15.500770+0000","last_fullsized":"2023-10-06T03:33:15.500770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:59.545999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493258+0000","last_change":"2023-10-06T03:33:15.487849+0000","last_active":"2023-10-06T03:33:15.493258+0000","last_peered":"2023-10-06T03:33:15.493258+0000","last_clean":"2023-10-06T03:33:15.493258+0000","last_became_active":"2023-10-06T03:33:15.487604+0000","last_became_peered":"2023-10-06T03:33:15.487604+0000","last_unstale":"2023-10-06T03:33:15.493258+0000","last_undegraded":"2023-10-06T03:33:15.493258+0000","last_fullsized":"2023-10-06T03:33:15.493258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:47.342414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298104,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073736,"statfs":{"total":96636764160,"available":96331505664,"internally_reserved":0,"allocated":835584,"data_stored":670346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":79204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302176,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069664,"statfs":{"total":96636764160,"available":96327335936,"internally_reserved":0,"allocated":811008,"data_stored":669572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301596,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070244,"statfs":{"total":96636764160,"available":96327929856,"internally_reserved":0,"allocated":217088,"data_stored":78780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301592,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070248,"statfs":{"total":96636764160,"available":96327933952,"internally_reserved":0,"allocated":212992,"data_stored":78226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301600,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070240,"statfs":{"total":96636764160,"available":96327925760,"internally_reserved":0,"allocated":221184,"data_stored":79294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":80796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":78850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:33:25.945 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:33:26.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.164+0000 7f04c0b9d700 1 Processor -- start 2023-10-06T03:33:26.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.164+0000 7f04c0b9d700 1 -- start start 2023-10-06T03:33:26.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.165+0000 7f04c0b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f04bc159420 0x7f04bc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.165+0000 7f04c0b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04bc1597f0 0x7f04bc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.165+0000 7f04c0b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04bc0b11d0 con 0x7f04bc15d230 2023-10-06T03:33:26.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.165+0000 7f04c0b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04bc0b14b0 con 0x7f04bc159420 2023-10-06T03:33:26.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.165+0000 7f04c0b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc0b1790 con 0x7f04bc1597f0 2023-10-06T03:33:26.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.165+0000 7f04bb7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f04bc159420 0x7f04bc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.166+0000 7f04bb7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f04bc159420 0x7f04bc0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59230/0 (socket says 172.21.15.55:59230) 2023-10-06T03:33:26.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.166+0000 7f04bb7fe700 1 -- 172.21.15.55:0/3030304086 learned_addr learned my addr 172.21.15.55:0/3030304086 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:26.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.166+0000 7f04bbfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f04bc15d230 0x7f04bc0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43984/0 (socket says 172.21.15.55:43984) 2023-10-06T03:33:26.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.166+0000 7f04baffd700 1 --2- 172.21.15.55:0/3030304086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04bc1597f0 0x7f04bc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.166+0000 7f04bb7fe700 1 -- 172.21.15.55:0/3030304086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04bc1597f0 msgr2=0x7f04bc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:26.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.166+0000 7f04bb7fe700 1 --2- 172.21.15.55:0/3030304086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04bc1597f0 0x7f04bc0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.166+0000 7f04bb7fe700 1 -- 172.21.15.55:0/3030304086 >> v1:172.21.15.55:6789/0 conn(0x7f04bc15d230 legacy=0x7f04bc0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:26.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.166+0000 7f04bb7fe700 1 -- 172.21.15.55:0/3030304086 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc0b1a70 con 0x7f04bc159420 2023-10-06T03:33:26.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.166+0000 7f04bb7fe700 1 --2- 172.21.15.55:0/3030304086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04bc159420 0x7f04bc0b5f40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0099b0 tx=0x7f04ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5b036b8510c760 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:26.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.166+0000 7f04ba7fc700 1 -- 172.21.15.55:0/3030304086 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac00b860 con 0x7f04bc159420 2023-10-06T03:33:26.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.166+0000 7f04ba7fc700 1 -- 172.21.15.55:0/3030304086 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ac00b9c0 con 0x7f04bc159420 2023-10-06T03:33:26.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.166+0000 7f04ba7fc700 1 -- 172.21.15.55:0/3030304086 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac00bd60 con 0x7f04bc159420 2023-10-06T03:33:26.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.167+0000 7f04c0b9d700 1 -- 172.21.15.55:0/3030304086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04bc159420 msgr2=0x7f04bc0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:26.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.167+0000 7f04c0b9d700 1 --2- 172.21.15.55:0/3030304086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04bc159420 0x7f04bc0b5f40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0099b0 tx=0x7f04ac009250 comp rx=0 tx=0).stop 2023-10-06T03:33:26.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.167+0000 7f04c0b9d700 1 -- 172.21.15.55:0/3030304086 shutdown_connections 2023-10-06T03:33:26.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.167+0000 7f04c0b9d700 1 --2- 172.21.15.55:0/3030304086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04bc159420 0x7f04bc0b5f40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.167+0000 7f04c0b9d700 1 --2- 172.21.15.55:0/3030304086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04bc1597f0 0x7f04bc0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.167+0000 7f04c0b9d700 1 -- 172.21.15.55:0/3030304086 >> 172.21.15.55:0/3030304086 conn(0x7f04bc0d0660 msgr2=0x7f04bc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:26.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.167+0000 7f04c0b9d700 1 -- 172.21.15.55:0/3030304086 shutdown_connections 2023-10-06T03:33:26.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.167+0000 7f04c0b9d700 1 -- 172.21.15.55:0/3030304086 wait complete. 2023-10-06T03:33:26.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.168+0000 7f04c0b9d700 1 Processor -- start 2023-10-06T03:33:26.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.169+0000 7f04c0b9d700 1 -- start start 2023-10-06T03:33:26.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.169+0000 7f04c0b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04bc159420 0x7f04bc0cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.169+0000 7f04c0b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04bc1597f0 0x7f04bc0cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.169+0000 7f04c0b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04bc15d230 0x7f04bc0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.169+0000 7f04c0b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc153fa0 con 0x7f04bc1597f0 2023-10-06T03:33:26.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.169+0000 7f04c0b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc0cff00 con 0x7f04bc159420 2023-10-06T03:33:26.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.169+0000 7f04c0b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc0d01e0 con 0x7f04bc15d230 2023-10-06T03:33:26.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.170+0000 7f04baffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04bc1597f0 0x7f04bc0cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.170+0000 7f04baffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04bc1597f0 0x7f04bc0cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59236/0 (socket says 172.21.15.55:59236) 2023-10-06T03:33:26.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.170+0000 7f04baffd700 1 -- 172.21.15.55:0/4164608884 learned_addr learned my addr 172.21.15.55:0/4164608884 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:26.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.170+0000 7f04bb7fe700 1 --2- 172.21.15.55:0/4164608884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04bc159420 0x7f04bc0cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.170+0000 7f04bbfff700 1 --2- 172.21.15.55:0/4164608884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04bc15d230 0x7f04bc0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.170+0000 7f04baffd700 1 -- 172.21.15.55:0/4164608884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04bc15d230 msgr2=0x7f04bc0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:26.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.170+0000 7f04baffd700 1 --2- 172.21.15.55:0/4164608884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04bc15d230 0x7f04bc0cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.170+0000 7f04baffd700 1 -- 172.21.15.55:0/4164608884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04bc159420 msgr2=0x7f04bc0cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:26.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.170+0000 7f04baffd700 1 --2- 172.21.15.55:0/4164608884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04bc159420 0x7f04bc0cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.170+0000 7f04baffd700 1 -- 172.21.15.55:0/4164608884 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac0094f0 con 0x7f04bc1597f0 2023-10-06T03:33:26.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.170+0000 7f04baffd700 1 --2- 172.21.15.55:0/4164608884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04bc1597f0 0x7f04bc0cf390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f04b000d890 tx=0x7f04b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:26.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.171+0000 7f04a37fe700 1 -- 172.21.15.55:0/4164608884 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b000b600 con 0x7f04bc1597f0 2023-10-06T03:33:26.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.171+0000 7f04a37fe700 1 -- 172.21.15.55:0/4164608884 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b000b760 con 0x7f04bc1597f0 2023-10-06T03:33:26.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.171+0000 7f04c0b9d700 1 -- 172.21.15.55:0/4164608884 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04bc15a620 con 0x7f04bc1597f0 2023-10-06T03:33:26.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.171+0000 7f04a37fe700 1 -- 172.21.15.55:0/4164608884 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b000bae0 con 0x7f04bc1597f0 2023-10-06T03:33:26.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.171+0000 7f04c0b9d700 1 -- 172.21.15.55:0/4164608884 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04bc15abe0 con 0x7f04bc1597f0 2023-10-06T03:33:26.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.172+0000 7f04c0b9d700 1 -- 172.21.15.55:0/4164608884 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04bc04c3c0 con 0x7f04bc1597f0 2023-10-06T03:33:26.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.173+0000 7f04a37fe700 1 -- 172.21.15.55:0/4164608884 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f04b001e460 con 0x7f04bc1597f0 2023-10-06T03:33:26.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.173+0000 7f04a37fe700 1 --2- 172.21.15.55:0/4164608884 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f04a8066f30 0x7f04a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.173+0000 7f04a37fe700 1 -- 172.21.15.55:0/4164608884 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f04b0084de0 con 0x7f04bc1597f0 2023-10-06T03:33:26.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.174+0000 7f04bb7fe700 1 --2- 172.21.15.55:0/4164608884 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f04a8066f30 0x7f04a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.175+0000 7f04bb7fe700 1 --2- 172.21.15.55:0/4164608884 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f04a8066f30 0x7f04a80693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0099b0 tx=0x7f04ac017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:26.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.177+0000 7f04a37fe700 1 -- 172.21.15.55:0/4164608884 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04b0051e50 con 0x7f04bc1597f0 2023-10-06T03:33:26.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.376+0000 7f04c0b9d700 1 -- 172.21.15.55:0/4164608884 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04bc0b1d50 con 0x7f04a8066f30 2023-10-06T03:33:26.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.384+0000 7f04a37fe700 1 -- 172.21.15.55:0/4164608884 <== mgr.4114 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286756 (secure 0 0 0) 0x7f04bc0b1d50 con 0x7f04a8066f30 2023-10-06T03:33:26.389 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:26.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.388+0000 7f04c0b9d700 1 -- 172.21.15.55:0/4164608884 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f04a8066f30 msgr2=0x7f04a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:26.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.388+0000 7f04c0b9d700 1 --2- 172.21.15.55:0/4164608884 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f04a8066f30 0x7f04a80693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f04ac0099b0 tx=0x7f04ac017000 comp rx=0 tx=0).stop 2023-10-06T03:33:26.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.388+0000 7f04c0b9d700 1 -- 172.21.15.55:0/4164608884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04bc1597f0 msgr2=0x7f04bc0cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:26.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.388+0000 7f04c0b9d700 1 --2- 172.21.15.55:0/4164608884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04bc1597f0 0x7f04bc0cf390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f04b000d890 tx=0x7f04b0008040 comp rx=0 tx=0).stop 2023-10-06T03:33:26.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.388+0000 7f04c0b9d700 1 -- 172.21.15.55:0/4164608884 shutdown_connections 2023-10-06T03:33:26.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.388+0000 7f04c0b9d700 1 --2- 172.21.15.55:0/4164608884 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f04a8066f30 0x7f04a80693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.388+0000 7f04c0b9d700 1 --2- 172.21.15.55:0/4164608884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04bc15d230 0x7f04bc0cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.388+0000 7f04c0b9d700 1 --2- 172.21.15.55:0/4164608884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04bc159420 0x7f04bc0cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.388+0000 7f04c0b9d700 1 --2- 172.21.15.55:0/4164608884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04bc1597f0 0x7f04bc0cf390 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.388+0000 7f04c0b9d700 1 -- 172.21.15.55:0/4164608884 >> 172.21.15.55:0/4164608884 conn(0x7f04bc0d0660 msgr2=0x7f04bc0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:26.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.388+0000 7f04c0b9d700 1 -- 172.21.15.55:0/4164608884 shutdown_connections 2023-10-06T03:33:26.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.388+0000 7f04c0b9d700 1 -- 172.21.15.55:0/4164608884 wait complete. 2023-10-06T03:33:26.393 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:33:26.407 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T03:33:24.490873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2397592,"kb_used_data":2904,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752577128,"statfs":{"total":773094113280,"available":770638979072,"internally_reserved":0,"allocated":2973696,"data_stored":1815068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.015987"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.489781+0000","last_change":"2023-10-06T03:33:15.485195+0000","last_active":"2023-10-06T03:33:15.489781+0000","last_peered":"2023-10-06T03:33:15.489781+0000","last_clean":"2023-10-06T03:33:15.489781+0000","last_became_active":"2023-10-06T03:33:15.484848+0000","last_became_peered":"2023-10-06T03:33:15.484848+0000","last_unstale":"2023-10-06T03:33:15.489781+0000","last_undegraded":"2023-10-06T03:33:15.489781+0000","last_fullsized":"2023-10-06T03:33:15.489781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:08.862091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502702+0000","last_change":"2023-10-06T03:33:15.498977+0000","last_active":"2023-10-06T03:33:15.502702+0000","last_peered":"2023-10-06T03:33:15.502702+0000","last_clean":"2023-10-06T03:33:15.502702+0000","last_became_active":"2023-10-06T03:33:15.498671+0000","last_became_peered":"2023-10-06T03:33:15.498671+0000","last_unstale":"2023-10-06T03:33:15.502702+0000","last_undegraded":"2023-10-06T03:33:15.502702+0000","last_fullsized":"2023-10-06T03:33:15.502702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500827+0000","last_change":"2023-10-06T03:33:15.494700+0000","last_active":"2023-10-06T03:33:15.500827+0000","last_peered":"2023-10-06T03:33:15.500827+0000","last_clean":"2023-10-06T03:33:15.500827+0000","last_became_active":"2023-10-06T03:33:15.494361+0000","last_became_peered":"2023-10-06T03:33:15.494361+0000","last_unstale":"2023-10-06T03:33:15.500827+0000","last_undegraded":"2023-10-06T03:33:15.500827+0000","last_fullsized":"2023-10-06T03:33:15.500827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:14.663102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.488889+0000","last_change":"2023-10-06T03:33:15.485455+0000","last_active":"2023-10-06T03:33:15.488889+0000","last_peered":"2023-10-06T03:33:15.488889+0000","last_clean":"2023-10-06T03:33:15.488889+0000","last_became_active":"2023-10-06T03:33:15.485159+0000","last_became_peered":"2023-10-06T03:33:15.485159+0000","last_unstale":"2023-10-06T03:33:15.488889+0000","last_undegraded":"2023-10-06T03:33:15.488889+0000","last_fullsized":"2023-10-06T03:33:15.488889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:02.594885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500387+0000","last_change":"2023-10-06T03:33:15.498785+0000","last_active":"2023-10-06T03:33:15.500387+0000","last_peered":"2023-10-06T03:33:15.500387+0000","last_clean":"2023-10-06T03:33:15.500387+0000","last_became_active":"2023-10-06T03:33:15.498503+0000","last_became_peered":"2023-10-06T03:33:15.498503+0000","last_unstale":"2023-10-06T03:33:15.500387+0000","last_undegraded":"2023-10-06T03:33:15.500387+0000","last_fullsized":"2023-10-06T03:33:15.500387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:59.545999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494228+0000","last_change":"2023-10-06T03:33:15.484222+0000","last_active":"2023-10-06T03:33:15.494228+0000","last_peered":"2023-10-06T03:33:15.494228+0000","last_clean":"2023-10-06T03:33:15.494228+0000","last_became_active":"2023-10-06T03:33:15.483910+0000","last_became_peered":"2023-10-06T03:33:15.483910+0000","last_unstale":"2023-10-06T03:33:15.494228+0000","last_undegraded":"2023-10-06T03:33:15.494228+0000","last_fullsized":"2023-10-06T03:33:15.494228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:59:41.309883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501590+0000","last_change":"2023-10-06T03:33:15.497903+0000","last_active":"2023-10-06T03:33:15.501590+0000","last_peered":"2023-10-06T03:33:15.501590+0000","last_clean":"2023-10-06T03:33:15.501590+0000","last_became_active":"2023-10-06T03:33:15.496979+0000","last_became_peered":"2023-10-06T03:33:15.496979+0000","last_unstale":"2023-10-06T03:33:15.501590+0000","last_undegraded":"2023-10-06T03:33:15.501590+0000","last_fullsized":"2023-10-06T03:33:15.501590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502728+0000","last_change":"2023-10-06T03:33:15.501214+0000","last_active":"2023-10-06T03:33:15.502728+0000","last_peered":"2023-10-06T03:33:15.502728+0000","last_clean":"2023-10-06T03:33:15.502728+0000","last_became_active":"2023-10-06T03:33:15.500739+0000","last_became_peered":"2023-10-06T03:33:15.500739+0000","last_unstale":"2023-10-06T03:33:15.502728+0000","last_undegraded":"2023-10-06T03:33:15.502728+0000","last_fullsized":"2023-10-06T03:33:15.502728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.964350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.499625+0000","last_change":"2023-10-06T03:33:15.498383+0000","last_active":"2023-10-06T03:33:15.499625+0000","last_peered":"2023-10-06T03:33:15.499625+0000","last_clean":"2023-10-06T03:33:15.499625+0000","last_became_active":"2023-10-06T03:33:15.498162+0000","last_became_peered":"2023-10-06T03:33:15.498162+0000","last_unstale":"2023-10-06T03:33:15.499625+0000","last_undegraded":"2023-10-06T03:33:15.499625+0000","last_fullsized":"2023-10-06T03:33:15.499625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:02.889905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501799+0000","last_change":"2023-10-06T03:33:15.496957+0000","last_active":"2023-10-06T03:33:15.501799+0000","last_peered":"2023-10-06T03:33:15.501799+0000","last_clean":"2023-10-06T03:33:15.501799+0000","last_became_active":"2023-10-06T03:33:15.496673+0000","last_became_peered":"2023-10-06T03:33:15.496673+0000","last_unstale":"2023-10-06T03:33:15.501799+0000","last_undegraded":"2023-10-06T03:33:15.501799+0000","last_fullsized":"2023-10-06T03:33:15.501799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500848+0000","last_change":"2023-10-06T03:33:15.495118+0000","last_active":"2023-10-06T03:33:15.500848+0000","last_peered":"2023-10-06T03:33:15.500848+0000","last_clean":"2023-10-06T03:33:15.500848+0000","last_became_active":"2023-10-06T03:33:15.494732+0000","last_became_peered":"2023-10-06T03:33:15.494732+0000","last_unstale":"2023-10-06T03:33:15.500848+0000","last_undegraded":"2023-10-06T03:33:15.500848+0000","last_fullsized":"2023-10-06T03:33:15.500848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:02.889905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494010+0000","last_change":"2023-10-06T03:33:15.491772+0000","last_active":"2023-10-06T03:33:15.494010+0000","last_peered":"2023-10-06T03:33:15.494010+0000","last_clean":"2023-10-06T03:33:15.494010+0000","last_became_active":"2023-10-06T03:33:15.491530+0000","last_became_peered":"2023-10-06T03:33:15.491530+0000","last_unstale":"2023-10-06T03:33:15.494010+0000","last_undegraded":"2023-10-06T03:33:15.494010+0000","last_fullsized":"2023-10-06T03:33:15.494010+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:57.097910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.489816+0000","last_change":"2023-10-06T03:33:15.485334+0000","last_active":"2023-10-06T03:33:15.489816+0000","last_peered":"2023-10-06T03:33:15.489816+0000","last_clean":"2023-10-06T03:33:15.489816+0000","last_became_active":"2023-10-06T03:33:15.485051+0000","last_became_peered":"2023-10-06T03:33:15.485051+0000","last_unstale":"2023-10-06T03:33:15.489816+0000","last_undegraded":"2023-10-06T03:33:15.489816+0000","last_fullsized":"2023-10-06T03:33:15.489816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:42.519384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498233+0000","last_change":"2023-10-06T03:33:15.495208+0000","last_active":"2023-10-06T03:33:15.498233+0000","last_peered":"2023-10-06T03:33:15.498233+0000","last_clean":"2023-10-06T03:33:15.498233+0000","last_became_active":"2023-10-06T03:33:15.494897+0000","last_became_peered":"2023-10-06T03:33:15.494897+0000","last_unstale":"2023-10-06T03:33:15.498233+0000","last_undegraded":"2023-10-06T03:33:15.498233+0000","last_fullsized":"2023-10-06T03:33:15.498233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:12.635649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501608+0000","last_change":"2023-10-06T03:33:15.498114+0000","last_active":"2023-10-06T03:33:15.501608+0000","last_peered":"2023-10-06T03:33:15.501608+0000","last_clean":"2023-10-06T03:33:15.501608+0000","last_became_active":"2023-10-06T03:33:15.497401+0000","last_became_peered":"2023-10-06T03:33:15.497401+0000","last_unstale":"2023-10-06T03:33:15.501608+0000","last_undegraded":"2023-10-06T03:33:15.501608+0000","last_fullsized":"2023-10-06T03:33:15.501608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494198+0000","last_change":"2023-10-06T03:33:15.484297+0000","last_active":"2023-10-06T03:33:15.494198+0000","last_peered":"2023-10-06T03:33:15.494198+0000","last_clean":"2023-10-06T03:33:15.494198+0000","last_became_active":"2023-10-06T03:33:15.483921+0000","last_became_peered":"2023-10-06T03:33:15.483921+0000","last_unstale":"2023-10-06T03:33:15.494198+0000","last_undegraded":"2023-10-06T03:33:15.494198+0000","last_fullsized":"2023-10-06T03:33:15.494198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:03.085548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.495074+0000","last_change":"2023-10-06T03:33:15.492170+0000","last_active":"2023-10-06T03:33:15.495074+0000","last_peered":"2023-10-06T03:33:15.495074+0000","last_clean":"2023-10-06T03:33:15.495074+0000","last_became_active":"2023-10-06T03:33:15.491847+0000","last_became_peered":"2023-10-06T03:33:15.491847+0000","last_unstale":"2023-10-06T03:33:15.495074+0000","last_undegraded":"2023-10-06T03:33:15.495074+0000","last_fullsized":"2023-10-06T03:33:15.495074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:30.660724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496120+0000","last_change":"2023-10-06T03:33:15.492652+0000","last_active":"2023-10-06T03:33:15.496120+0000","last_peered":"2023-10-06T03:33:15.496120+0000","last_clean":"2023-10-06T03:33:15.496120+0000","last_became_active":"2023-10-06T03:33:15.491815+0000","last_became_peered":"2023-10-06T03:33:15.491815+0000","last_unstale":"2023-10-06T03:33:15.496120+0000","last_undegraded":"2023-10-06T03:33:15.496120+0000","last_fullsized":"2023-10-06T03:33:15.496120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:20.661516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.489846+0000","last_change":"2023-10-06T03:33:15.486081+0000","last_active":"2023-10-06T03:33:15.489846+0000","last_peered":"2023-10-06T03:33:15.489846+0000","last_clean":"2023-10-06T03:33:15.489846+0000","last_became_active":"2023-10-06T03:33:15.485842+0000","last_became_peered":"2023-10-06T03:33:15.485842+0000","last_unstale":"2023-10-06T03:33:15.489846+0000","last_undegraded":"2023-10-06T03:33:15.489846+0000","last_fullsized":"2023-10-06T03:33:15.489846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:16.260090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496154+0000","last_change":"2023-10-06T03:33:15.490482+0000","last_active":"2023-10-06T03:33:15.496154+0000","last_peered":"2023-10-06T03:33:15.496154+0000","last_clean":"2023-10-06T03:33:15.496154+0000","last_became_active":"2023-10-06T03:33:15.490185+0000","last_became_peered":"2023-10-06T03:33:15.490185+0000","last_unstale":"2023-10-06T03:33:15.496154+0000","last_undegraded":"2023-10-06T03:33:15.496154+0000","last_fullsized":"2023-10-06T03:33:15.496154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:05.828284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502137+0000","last_change":"2023-10-06T03:33:15.499609+0000","last_active":"2023-10-06T03:33:15.502137+0000","last_peered":"2023-10-06T03:33:15.502137+0000","last_clean":"2023-10-06T03:33:15.502137+0000","last_became_active":"2023-10-06T03:33:15.499317+0000","last_became_peered":"2023-10-06T03:33:15.499317+0000","last_unstale":"2023-10-06T03:33:15.502137+0000","last_undegraded":"2023-10-06T03:33:15.502137+0000","last_fullsized":"2023-10-06T03:33:15.502137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494425+0000","last_change":"2023-10-06T03:33:15.491198+0000","last_active":"2023-10-06T03:33:15.494425+0000","last_peered":"2023-10-06T03:33:15.494425+0000","last_clean":"2023-10-06T03:33:15.494425+0000","last_became_active":"2023-10-06T03:33:15.490933+0000","last_became_peered":"2023-10-06T03:33:15.490933+0000","last_unstale":"2023-10-06T03:33:15.494425+0000","last_undegraded":"2023-10-06T03:33:15.494425+0000","last_fullsized":"2023-10-06T03:33:15.494425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500604+0000","last_change":"2023-10-06T03:33:15.492259+0000","last_active":"2023-10-06T03:33:15.500604+0000","last_peered":"2023-10-06T03:33:15.500604+0000","last_clean":"2023-10-06T03:33:15.500604+0000","last_became_active":"2023-10-06T03:33:15.492018+0000","last_became_peered":"2023-10-06T03:33:15.492018+0000","last_unstale":"2023-10-06T03:33:15.500604+0000","last_undegraded":"2023-10-06T03:33:15.500604+0000","last_fullsized":"2023-10-06T03:33:15.500604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:31:34.027062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501954+0000","last_change":"2023-10-06T03:33:15.500133+0000","last_active":"2023-10-06T03:33:15.501954+0000","last_peered":"2023-10-06T03:33:15.501954+0000","last_clean":"2023-10-06T03:33:15.501954+0000","last_became_active":"2023-10-06T03:33:15.499092+0000","last_became_peered":"2023-10-06T03:33:15.499092+0000","last_unstale":"2023-10-06T03:33:15.501954+0000","last_undegraded":"2023-10-06T03:33:15.501954+0000","last_fullsized":"2023-10-06T03:33:15.501954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493234+0000","last_change":"2023-10-06T03:33:15.487133+0000","last_active":"2023-10-06T03:33:15.493234+0000","last_peered":"2023-10-06T03:33:15.493234+0000","last_clean":"2023-10-06T03:33:15.493234+0000","last_became_active":"2023-10-06T03:33:15.486582+0000","last_became_peered":"2023-10-06T03:33:15.486582+0000","last_unstale":"2023-10-06T03:33:15.493234+0000","last_undegraded":"2023-10-06T03:33:15.493234+0000","last_fullsized":"2023-10-06T03:33:15.493234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:37.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498958+0000","last_change":"2023-10-06T03:33:15.489677+0000","last_active":"2023-10-06T03:33:15.498958+0000","last_peered":"2023-10-06T03:33:15.498958+0000","last_clean":"2023-10-06T03:33:15.498958+0000","last_became_active":"2023-10-06T03:33:15.489431+0000","last_became_peered":"2023-10-06T03:33:15.489431+0000","last_unstale":"2023-10-06T03:33:15.498958+0000","last_undegraded":"2023-10-06T03:33:15.498958+0000","last_fullsized":"2023-10-06T03:33:15.498958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502491+0000","last_change":"2023-10-06T03:33:15.498305+0000","last_active":"2023-10-06T03:33:15.502491+0000","last_peered":"2023-10-06T03:33:15.502491+0000","last_clean":"2023-10-06T03:33:15.502491+0000","last_became_active":"2023-10-06T03:33:15.497920+0000","last_became_peered":"2023-10-06T03:33:15.497920+0000","last_unstale":"2023-10-06T03:33:15.502491+0000","last_undegraded":"2023-10-06T03:33:15.502491+0000","last_fullsized":"2023-10-06T03:33:15.502491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.880457+0000","last_change":"2023-10-06T03:33:14.894963+0000","last_active":"2023-10-06T03:33:15.880457+0000","last_peered":"2023-10-06T03:33:15.880457+0000","last_clean":"2023-10-06T03:33:15.880457+0000","last_became_active":"2023-10-06T03:33:14.886952+0000","last_became_peered":"2023-10-06T03:33:14.886952+0000","last_unstale":"2023-10-06T03:33:15.880457+0000","last_undegraded":"2023-10-06T03:33:15.880457+0000","last_fullsized":"2023-10-06T03:33:15.880457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501009+0000","last_change":"2023-10-06T03:33:15.498264+0000","last_active":"2023-10-06T03:33:15.501009+0000","last_peered":"2023-10-06T03:33:15.501009+0000","last_clean":"2023-10-06T03:33:15.501009+0000","last_became_active":"2023-10-06T03:33:15.498030+0000","last_became_peered":"2023-10-06T03:33:15.498030+0000","last_unstale":"2023-10-06T03:33:15.501009+0000","last_undegraded":"2023-10-06T03:33:15.501009+0000","last_fullsized":"2023-10-06T03:33:15.501009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:16:05.828284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.473505+0000","last_change":"2023-10-06T03:33:14.895404+0000","last_active":"2023-10-06T03:33:15.473505+0000","last_peered":"2023-10-06T03:33:15.473505+0000","last_clean":"2023-10-06T03:33:15.473505+0000","last_became_active":"2023-10-06T03:33:14.894742+0000","last_became_peered":"2023-10-06T03:33:14.894742+0000","last_unstale":"2023-10-06T03:33:15.473505+0000","last_undegraded":"2023-10-06T03:33:15.473505+0000","last_fullsized":"2023-10-06T03:33:15.473505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490157+0000","last_change":"2023-10-06T03:33:15.485976+0000","last_active":"2023-10-06T03:33:15.490157+0000","last_peered":"2023-10-06T03:33:15.490157+0000","last_clean":"2023-10-06T03:33:15.490157+0000","last_became_active":"2023-10-06T03:33:15.485658+0000","last_became_peered":"2023-10-06T03:33:15.485658+0000","last_unstale":"2023-10-06T03:33:15.490157+0000","last_undegraded":"2023-10-06T03:33:15.490157+0000","last_fullsized":"2023-10-06T03:33:15.490157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482088+0000","last_change":"2023-10-06T03:33:14.903103+0000","last_active":"2023-10-06T03:33:15.482088+0000","last_peered":"2023-10-06T03:33:15.482088+0000","last_clean":"2023-10-06T03:33:15.482088+0000","last_became_active":"2023-10-06T03:33:14.902707+0000","last_became_peered":"2023-10-06T03:33:14.902707+0000","last_unstale":"2023-10-06T03:33:15.482088+0000","last_undegraded":"2023-10-06T03:33:15.482088+0000","last_fullsized":"2023-10-06T03:33:15.482088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501406+0000","last_change":"2023-10-06T03:33:15.496437+0000","last_active":"2023-10-06T03:33:15.501406+0000","last_peered":"2023-10-06T03:33:15.501406+0000","last_clean":"2023-10-06T03:33:15.501406+0000","last_became_active":"2023-10-06T03:33:15.496138+0000","last_became_peered":"2023-10-06T03:33:15.496138+0000","last_unstale":"2023-10-06T03:33:15.501406+0000","last_undegraded":"2023-10-06T03:33:15.501406+0000","last_fullsized":"2023-10-06T03:33:15.501406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:09:05.289396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.478575+0000","last_change":"2023-10-06T03:33:14.897019+0000","last_active":"2023-10-06T03:33:15.478575+0000","last_peered":"2023-10-06T03:33:15.478575+0000","last_clean":"2023-10-06T03:33:15.478575+0000","last_became_active":"2023-10-06T03:33:14.896740+0000","last_became_peered":"2023-10-06T03:33:14.896740+0000","last_unstale":"2023-10-06T03:33:15.478575+0000","last_undegraded":"2023-10-06T03:33:15.478575+0000","last_fullsized":"2023-10-06T03:33:15.478575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500886+0000","last_change":"2023-10-06T03:33:15.496977+0000","last_active":"2023-10-06T03:33:15.500886+0000","last_peered":"2023-10-06T03:33:15.500886+0000","last_clean":"2023-10-06T03:33:15.500886+0000","last_became_active":"2023-10-06T03:33:15.496726+0000","last_became_peered":"2023-10-06T03:33:15.496726+0000","last_unstale":"2023-10-06T03:33:15.500886+0000","last_undegraded":"2023-10-06T03:33:15.500886+0000","last_fullsized":"2023-10-06T03:33:15.500886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:44:02.889905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.491600+0000","last_change":"2023-10-06T03:33:14.896854+0000","last_active":"2023-10-06T03:33:15.491600+0000","last_peered":"2023-10-06T03:33:15.491600+0000","last_clean":"2023-10-06T03:33:15.491600+0000","last_became_active":"2023-10-06T03:33:14.896606+0000","last_became_peered":"2023-10-06T03:33:14.896606+0000","last_unstale":"2023-10-06T03:33:15.491600+0000","last_undegraded":"2023-10-06T03:33:15.491600+0000","last_fullsized":"2023-10-06T03:33:15.491600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500655+0000","last_change":"2023-10-06T03:33:15.494940+0000","last_active":"2023-10-06T03:33:15.500655+0000","last_peered":"2023-10-06T03:33:15.500655+0000","last_clean":"2023-10-06T03:33:15.500655+0000","last_became_active":"2023-10-06T03:33:15.494251+0000","last_became_peered":"2023-10-06T03:33:15.494251+0000","last_unstale":"2023-10-06T03:33:15.500655+0000","last_undegraded":"2023-10-06T03:33:15.500655+0000","last_fullsized":"2023-10-06T03:33:15.500655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.885266+0000","last_change":"2023-10-06T03:33:14.895082+0000","last_active":"2023-10-06T03:33:15.885266+0000","last_peered":"2023-10-06T03:33:15.885266+0000","last_clean":"2023-10-06T03:33:15.885266+0000","last_became_active":"2023-10-06T03:33:14.894804+0000","last_became_peered":"2023-10-06T03:33:14.894804+0000","last_unstale":"2023-10-06T03:33:15.885266+0000","last_undegraded":"2023-10-06T03:33:15.885266+0000","last_fullsized":"2023-10-06T03:33:15.885266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:01.585501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493308+0000","last_change":"2023-10-06T03:33:15.487049+0000","last_active":"2023-10-06T03:33:15.493308+0000","last_peered":"2023-10-06T03:33:15.493308+0000","last_clean":"2023-10-06T03:33:15.493308+0000","last_became_active":"2023-10-06T03:33:15.486777+0000","last_became_peered":"2023-10-06T03:33:15.486777+0000","last_unstale":"2023-10-06T03:33:15.493308+0000","last_undegraded":"2023-10-06T03:33:15.493308+0000","last_fullsized":"2023-10-06T03:33:15.493308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:12.635649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.888581+0000","last_change":"2023-10-06T03:33:14.898475+0000","last_active":"2023-10-06T03:33:15.888581+0000","last_peered":"2023-10-06T03:33:15.888581+0000","last_clean":"2023-10-06T03:33:15.888581+0000","last_became_active":"2023-10-06T03:33:14.898144+0000","last_became_peered":"2023-10-06T03:33:14.898144+0000","last_unstale":"2023-10-06T03:33:15.888581+0000","last_undegraded":"2023-10-06T03:33:15.888581+0000","last_fullsized":"2023-10-06T03:33:15.888581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494412+0000","last_change":"2023-10-06T03:33:15.487549+0000","last_active":"2023-10-06T03:33:15.494412+0000","last_peered":"2023-10-06T03:33:15.494412+0000","last_clean":"2023-10-06T03:33:15.494412+0000","last_became_active":"2023-10-06T03:33:15.487277+0000","last_became_peered":"2023-10-06T03:33:15.487277+0000","last_unstale":"2023-10-06T03:33:15.494412+0000","last_undegraded":"2023-10-06T03:33:15.494412+0000","last_fullsized":"2023-10-06T03:33:15.494412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:07.613983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.885740+0000","last_change":"2023-10-06T03:33:14.896271+0000","last_active":"2023-10-06T03:33:15.885740+0000","last_peered":"2023-10-06T03:33:15.885740+0000","last_clean":"2023-10-06T03:33:15.885740+0000","last_became_active":"2023-10-06T03:33:14.896004+0000","last_became_peered":"2023-10-06T03:33:14.896004+0000","last_unstale":"2023-10-06T03:33:15.885740+0000","last_undegraded":"2023-10-06T03:33:15.885740+0000","last_fullsized":"2023-10-06T03:33:15.885740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:15.567270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500942+0000","last_change":"2023-10-06T03:33:15.497569+0000","last_active":"2023-10-06T03:33:15.500942+0000","last_peered":"2023-10-06T03:33:15.500942+0000","last_clean":"2023-10-06T03:33:15.500942+0000","last_became_active":"2023-10-06T03:33:15.497287+0000","last_became_peered":"2023-10-06T03:33:15.497287+0000","last_unstale":"2023-10-06T03:33:15.500942+0000","last_undegraded":"2023-10-06T03:33:15.500942+0000","last_fullsized":"2023-10-06T03:33:15.500942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:47.107849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.476556+0000","last_change":"2023-10-06T03:33:14.897199+0000","last_active":"2023-10-06T03:33:15.476556+0000","last_peered":"2023-10-06T03:33:15.476556+0000","last_clean":"2023-10-06T03:33:15.476556+0000","last_became_active":"2023-10-06T03:33:14.896925+0000","last_became_peered":"2023-10-06T03:33:14.896925+0000","last_unstale":"2023-10-06T03:33:15.476556+0000","last_undegraded":"2023-10-06T03:33:15.476556+0000","last_fullsized":"2023-10-06T03:33:15.476556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:41.510001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498210+0000","last_change":"2023-10-06T03:33:15.495381+0000","last_active":"2023-10-06T03:33:15.498210+0000","last_peered":"2023-10-06T03:33:15.498210+0000","last_clean":"2023-10-06T03:33:15.498210+0000","last_became_active":"2023-10-06T03:33:15.494972+0000","last_became_peered":"2023-10-06T03:33:15.494972+0000","last_unstale":"2023-10-06T03:33:15.498210+0000","last_undegraded":"2023-10-06T03:33:15.498210+0000","last_fullsized":"2023-10-06T03:33:15.498210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.483344+0000","last_change":"2023-10-06T03:33:14.901135+0000","last_active":"2023-10-06T03:33:15.483344+0000","last_peered":"2023-10-06T03:33:15.483344+0000","last_clean":"2023-10-06T03:33:15.483344+0000","last_became_active":"2023-10-06T03:33:14.900871+0000","last_became_peered":"2023-10-06T03:33:14.900871+0000","last_unstale":"2023-10-06T03:33:15.483344+0000","last_undegraded":"2023-10-06T03:33:15.483344+0000","last_fullsized":"2023-10-06T03:33:15.483344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494496+0000","last_change":"2023-10-06T03:33:15.491822+0000","last_active":"2023-10-06T03:33:15.494496+0000","last_peered":"2023-10-06T03:33:15.494496+0000","last_clean":"2023-10-06T03:33:15.494496+0000","last_became_active":"2023-10-06T03:33:15.491560+0000","last_became_peered":"2023-10-06T03:33:15.491560+0000","last_unstale":"2023-10-06T03:33:15.494496+0000","last_undegraded":"2023-10-06T03:33:15.494496+0000","last_fullsized":"2023-10-06T03:33:15.494496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.964350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886225+0000","last_change":"2023-10-06T03:33:14.898845+0000","last_active":"2023-10-06T03:33:15.886225+0000","last_peered":"2023-10-06T03:33:15.886225+0000","last_clean":"2023-10-06T03:33:15.886225+0000","last_became_active":"2023-10-06T03:33:14.898491+0000","last_became_peered":"2023-10-06T03:33:14.898491+0000","last_unstale":"2023-10-06T03:33:15.886225+0000","last_undegraded":"2023-10-06T03:33:15.886225+0000","last_fullsized":"2023-10-06T03:33:15.886225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:25:49.507090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500404+0000","last_change":"2023-10-06T03:33:15.487774+0000","last_active":"2023-10-06T03:33:15.500404+0000","last_peered":"2023-10-06T03:33:15.500404+0000","last_clean":"2023-10-06T03:33:15.500404+0000","last_became_active":"2023-10-06T03:33:15.487515+0000","last_became_peered":"2023-10-06T03:33:15.487515+0000","last_unstale":"2023-10-06T03:33:15.500404+0000","last_undegraded":"2023-10-06T03:33:15.500404+0000","last_fullsized":"2023-10-06T03:33:15.500404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:30.660724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.884947+0000","last_change":"2023-10-06T03:33:14.896171+0000","last_active":"2023-10-06T03:33:15.884947+0000","last_peered":"2023-10-06T03:33:15.884947+0000","last_clean":"2023-10-06T03:33:15.884947+0000","last_became_active":"2023-10-06T03:33:14.895903+0000","last_became_peered":"2023-10-06T03:33:14.895903+0000","last_unstale":"2023-10-06T03:33:15.884947+0000","last_undegraded":"2023-10-06T03:33:15.884947+0000","last_fullsized":"2023-10-06T03:33:15.884947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:44.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500464+0000","last_change":"2023-10-06T03:33:15.496285+0000","last_active":"2023-10-06T03:33:15.500464+0000","last_peered":"2023-10-06T03:33:15.500464+0000","last_clean":"2023-10-06T03:33:15.500464+0000","last_became_active":"2023-10-06T03:33:15.495868+0000","last_became_peered":"2023-10-06T03:33:15.495868+0000","last_unstale":"2023-10-06T03:33:15.500464+0000","last_undegraded":"2023-10-06T03:33:15.500464+0000","last_fullsized":"2023-10-06T03:33:15.500464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.885984+0000","last_change":"2023-10-06T03:33:14.896322+0000","last_active":"2023-10-06T03:33:15.885984+0000","last_peered":"2023-10-06T03:33:15.885984+0000","last_clean":"2023-10-06T03:33:15.885984+0000","last_became_active":"2023-10-06T03:33:14.896022+0000","last_became_peered":"2023-10-06T03:33:14.896022+0000","last_unstale":"2023-10-06T03:33:15.885984+0000","last_undegraded":"2023-10-06T03:33:15.885984+0000","last_fullsized":"2023-10-06T03:33:15.885984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:21.680218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500853+0000","last_change":"2023-10-06T03:33:15.494878+0000","last_active":"2023-10-06T03:33:15.500853+0000","last_peered":"2023-10-06T03:33:15.500853+0000","last_clean":"2023-10-06T03:33:15.500853+0000","last_became_active":"2023-10-06T03:33:15.494597+0000","last_became_peered":"2023-10-06T03:33:15.494597+0000","last_unstale":"2023-10-06T03:33:15.500853+0000","last_undegraded":"2023-10-06T03:33:15.500853+0000","last_fullsized":"2023-10-06T03:33:15.500853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:30.660724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.887336+0000","last_change":"2023-10-06T03:33:14.897659+0000","last_active":"2023-10-06T03:33:15.887336+0000","last_peered":"2023-10-06T03:33:15.887336+0000","last_clean":"2023-10-06T03:33:15.887336+0000","last_became_active":"2023-10-06T03:33:14.897368+0000","last_became_peered":"2023-10-06T03:33:14.897368+0000","last_unstale":"2023-10-06T03:33:15.887336+0000","last_undegraded":"2023-10-06T03:33:15.887336+0000","last_fullsized":"2023-10-06T03:33:15.887336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:46:12.361089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493933+0000","last_change":"2023-10-06T03:33:15.492136+0000","last_active":"2023-10-06T03:33:15.493933+0000","last_peered":"2023-10-06T03:33:15.493933+0000","last_clean":"2023-10-06T03:33:15.493933+0000","last_became_active":"2023-10-06T03:33:15.491858+0000","last_became_peered":"2023-10-06T03:33:15.491858+0000","last_unstale":"2023-10-06T03:33:15.493933+0000","last_undegraded":"2023-10-06T03:33:15.493933+0000","last_fullsized":"2023-10-06T03:33:15.493933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:16.576653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490953+0000","last_change":"2023-10-06T03:33:14.890596+0000","last_active":"2023-10-06T03:33:15.490953+0000","last_peered":"2023-10-06T03:33:15.490953+0000","last_clean":"2023-10-06T03:33:15.490953+0000","last_became_active":"2023-10-06T03:33:14.889486+0000","last_became_peered":"2023-10-06T03:33:14.889486+0000","last_unstale":"2023-10-06T03:33:15.490953+0000","last_undegraded":"2023-10-06T03:33:15.490953+0000","last_fullsized":"2023-10-06T03:33:15.490953+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:41.510001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.495179+0000","last_change":"2023-10-06T03:33:15.493860+0000","last_active":"2023-10-06T03:33:15.495179+0000","last_peered":"2023-10-06T03:33:15.495179+0000","last_clean":"2023-10-06T03:33:15.495179+0000","last_became_active":"2023-10-06T03:33:15.493610+0000","last_became_peered":"2023-10-06T03:33:15.493610+0000","last_unstale":"2023-10-06T03:33:15.495179+0000","last_undegraded":"2023-10-06T03:33:15.495179+0000","last_fullsized":"2023-10-06T03:33:15.495179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:18.285701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.881042+0000","last_change":"2023-10-06T03:33:14.903094+0000","last_active":"2023-10-06T03:33:15.881042+0000","last_peered":"2023-10-06T03:33:15.881042+0000","last_clean":"2023-10-06T03:33:15.881042+0000","last_became_active":"2023-10-06T03:33:14.902686+0000","last_became_peered":"2023-10-06T03:33:14.902686+0000","last_unstale":"2023-10-06T03:33:15.881042+0000","last_undegraded":"2023-10-06T03:33:15.881042+0000","last_fullsized":"2023-10-06T03:33:15.881042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496134+0000","last_change":"2023-10-06T03:33:15.489915+0000","last_active":"2023-10-06T03:33:15.496134+0000","last_peered":"2023-10-06T03:33:15.496134+0000","last_clean":"2023-10-06T03:33:15.496134+0000","last_became_active":"2023-10-06T03:33:15.488544+0000","last_became_peered":"2023-10-06T03:33:15.488544+0000","last_unstale":"2023-10-06T03:33:15.496134+0000","last_undegraded":"2023-10-06T03:33:15.496134+0000","last_fullsized":"2023-10-06T03:33:15.496134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.478249+0000","last_change":"2023-10-06T03:33:14.886562+0000","last_active":"2023-10-06T03:33:15.478249+0000","last_peered":"2023-10-06T03:33:15.478249+0000","last_clean":"2023-10-06T03:33:15.478249+0000","last_became_active":"2023-10-06T03:33:14.886157+0000","last_became_peered":"2023-10-06T03:33:14.886157+0000","last_unstale":"2023-10-06T03:33:15.478249+0000","last_undegraded":"2023-10-06T03:33:15.478249+0000","last_fullsized":"2023-10-06T03:33:15.478249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496415+0000","last_change":"2023-10-06T03:33:15.491844+0000","last_active":"2023-10-06T03:33:15.496415+0000","last_peered":"2023-10-06T03:33:15.496415+0000","last_clean":"2023-10-06T03:33:15.496415+0000","last_became_active":"2023-10-06T03:33:15.491574+0000","last_became_peered":"2023-10-06T03:33:15.491574+0000","last_unstale":"2023-10-06T03:33:15.496415+0000","last_undegraded":"2023-10-06T03:33:15.496415+0000","last_fullsized":"2023-10-06T03:33:15.496415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:01:35.426668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482572+0000","last_change":"2023-10-06T03:33:14.895658+0000","last_active":"2023-10-06T03:33:15.482572+0000","last_peered":"2023-10-06T03:33:15.482572+0000","last_clean":"2023-10-06T03:33:15.482572+0000","last_became_active":"2023-10-06T03:33:14.895332+0000","last_became_peered":"2023-10-06T03:33:14.895332+0000","last_unstale":"2023-10-06T03:33:15.482572+0000","last_undegraded":"2023-10-06T03:33:15.482572+0000","last_fullsized":"2023-10-06T03:33:15.482572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:28.684332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496089+0000","last_change":"2023-10-06T03:33:15.490478+0000","last_active":"2023-10-06T03:33:15.496089+0000","last_peered":"2023-10-06T03:33:15.496089+0000","last_clean":"2023-10-06T03:33:15.496089+0000","last_became_active":"2023-10-06T03:33:15.485974+0000","last_became_peered":"2023-10-06T03:33:15.485974+0000","last_unstale":"2023-10-06T03:33:15.496089+0000","last_undegraded":"2023-10-06T03:33:15.496089+0000","last_fullsized":"2023-10-06T03:33:15.496089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.481946+0000","last_change":"2023-10-06T03:33:14.891288+0000","last_active":"2023-10-06T03:33:15.481946+0000","last_peered":"2023-10-06T03:33:15.481946+0000","last_clean":"2023-10-06T03:33:15.481946+0000","last_became_active":"2023-10-06T03:33:14.890186+0000","last_became_peered":"2023-10-06T03:33:14.890186+0000","last_unstale":"2023-10-06T03:33:15.481946+0000","last_undegraded":"2023-10-06T03:33:15.481946+0000","last_fullsized":"2023-10-06T03:33:15.481946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:39:21.680218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493182+0000","last_change":"2023-10-06T03:33:15.486956+0000","last_active":"2023-10-06T03:33:15.493182+0000","last_peered":"2023-10-06T03:33:15.493182+0000","last_clean":"2023-10-06T03:33:15.493182+0000","last_became_active":"2023-10-06T03:33:15.486188+0000","last_became_peered":"2023-10-06T03:33:15.486188+0000","last_unstale":"2023-10-06T03:33:15.493182+0000","last_undegraded":"2023-10-06T03:33:15.493182+0000","last_fullsized":"2023-10-06T03:33:15.493182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:37:37.061068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.887149+0000","last_change":"2023-10-06T03:33:14.895426+0000","last_active":"2023-10-06T03:33:15.887149+0000","last_peered":"2023-10-06T03:33:15.887149+0000","last_clean":"2023-10-06T03:33:15.887149+0000","last_became_active":"2023-10-06T03:33:14.893425+0000","last_became_peered":"2023-10-06T03:33:14.893425+0000","last_unstale":"2023-10-06T03:33:15.887149+0000","last_undegraded":"2023-10-06T03:33:15.887149+0000","last_fullsized":"2023-10-06T03:33:15.887149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:44.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.484841+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:33:15.484841+0000","last_peered":"2023-10-06T03:33:15.484841+0000","last_clean":"2023-10-06T03:33:15.484841+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:33:15.484841+0000","last_undegraded":"2023-10-06T03:33:15.484841+0000","last_fullsized":"2023-10-06T03:33:15.484841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500598+0000","last_change":"2023-10-06T03:33:15.492324+0000","last_active":"2023-10-06T03:33:15.500598+0000","last_peered":"2023-10-06T03:33:15.500598+0000","last_clean":"2023-10-06T03:33:15.500598+0000","last_became_active":"2023-10-06T03:33:15.492037+0000","last_became_peered":"2023-10-06T03:33:15.492037+0000","last_unstale":"2023-10-06T03:33:15.500598+0000","last_undegraded":"2023-10-06T03:33:15.500598+0000","last_fullsized":"2023-10-06T03:33:15.500598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.484831+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:33:15.484831+0000","last_peered":"2023-10-06T03:33:15.484831+0000","last_clean":"2023-10-06T03:33:15.484831+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:33:15.484831+0000","last_undegraded":"2023-10-06T03:33:15.484831+0000","last_fullsized":"2023-10-06T03:33:15.484831+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T03:33:16.884845+0000","last_change":"2023-10-06T03:33:14.896368+0000","last_active":"2023-10-06T03:33:16.884845+0000","last_peered":"2023-10-06T03:33:16.884845+0000","last_clean":"2023-10-06T03:33:16.884845+0000","last_became_active":"2023-10-06T03:33:14.892459+0000","last_became_peered":"2023-10-06T03:33:14.892459+0000","last_unstale":"2023-10-06T03:33:16.884845+0000","last_undegraded":"2023-10-06T03:33:16.884845+0000","last_fullsized":"2023-10-06T03:33:16.884845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501686+0000","last_change":"2023-10-06T03:33:15.499324+0000","last_active":"2023-10-06T03:33:15.501686+0000","last_peered":"2023-10-06T03:33:15.501686+0000","last_clean":"2023-10-06T03:33:15.501686+0000","last_became_active":"2023-10-06T03:33:15.499039+0000","last_became_peered":"2023-10-06T03:33:15.499039+0000","last_unstale":"2023-10-06T03:33:15.501686+0000","last_undegraded":"2023-10-06T03:33:15.501686+0000","last_fullsized":"2023-10-06T03:33:15.501686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:43.269972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.473587+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:33:15.473587+0000","last_peered":"2023-10-06T03:33:15.473587+0000","last_clean":"2023-10-06T03:33:15.473587+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:33:15.473587+0000","last_undegraded":"2023-10-06T03:33:15.473587+0000","last_fullsized":"2023-10-06T03:33:15.473587+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":163,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.483628+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:33:15.483628+0000","last_peered":"2023-10-06T03:33:15.483628+0000","last_clean":"2023-10-06T03:33:15.483628+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:33:15.483628+0000","last_undegraded":"2023-10-06T03:33:15.483628+0000","last_fullsized":"2023-10-06T03:33:15.483628+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.476509+0000","last_change":"2023-10-06T03:33:14.891206+0000","last_active":"2023-10-06T03:33:15.476509+0000","last_peered":"2023-10-06T03:33:15.476509+0000","last_clean":"2023-10-06T03:33:15.476509+0000","last_became_active":"2023-10-06T03:33:14.890823+0000","last_became_peered":"2023-10-06T03:33:14.890823+0000","last_unstale":"2023-10-06T03:33:15.476509+0000","last_undegraded":"2023-10-06T03:33:15.476509+0000","last_fullsized":"2023-10-06T03:33:15.476509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:28.684332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490137+0000","last_change":"2023-10-06T03:33:15.483582+0000","last_active":"2023-10-06T03:33:15.490137+0000","last_peered":"2023-10-06T03:33:15.490137+0000","last_clean":"2023-10-06T03:33:15.490137+0000","last_became_active":"2023-10-06T03:33:15.483324+0000","last_became_peered":"2023-10-06T03:33:15.483324+0000","last_unstale":"2023-10-06T03:33:15.490137+0000","last_undegraded":"2023-10-06T03:33:15.490137+0000","last_fullsized":"2023-10-06T03:33:15.490137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:14:12.635649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482905+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:33:15.482905+0000","last_peered":"2023-10-06T03:33:15.482905+0000","last_clean":"2023-10-06T03:33:15.482905+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:33:15.482905+0000","last_undegraded":"2023-10-06T03:33:15.482905+0000","last_fullsized":"2023-10-06T03:33:15.482905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.474679+0000","last_change":"2023-10-06T03:33:14.902096+0000","last_active":"2023-10-06T03:33:15.474679+0000","last_peered":"2023-10-06T03:33:15.474679+0000","last_clean":"2023-10-06T03:33:15.474679+0000","last_became_active":"2023-10-06T03:33:14.896599+0000","last_became_peered":"2023-10-06T03:33:14.896599+0000","last_unstale":"2023-10-06T03:33:15.474679+0000","last_undegraded":"2023-10-06T03:33:15.474679+0000","last_fullsized":"2023-10-06T03:33:15.474679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.502084+0000","last_change":"2023-10-06T03:33:15.500022+0000","last_active":"2023-10-06T03:33:15.502084+0000","last_peered":"2023-10-06T03:33:15.502084+0000","last_clean":"2023-10-06T03:33:15.502084+0000","last_became_active":"2023-10-06T03:33:15.499610+0000","last_became_peered":"2023-10-06T03:33:15.499610+0000","last_unstale":"2023-10-06T03:33:15.502084+0000","last_undegraded":"2023-10-06T03:33:15.502084+0000","last_fullsized":"2023-10-06T03:33:15.502084+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:42:58.092449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.483163+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:33:15.483163+0000","last_peered":"2023-10-06T03:33:15.483163+0000","last_clean":"2023-10-06T03:33:15.483163+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:33:15.483163+0000","last_undegraded":"2023-10-06T03:33:15.483163+0000","last_fullsized":"2023-10-06T03:33:15.483163+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.482546+0000","last_change":"2023-10-06T03:33:14.895562+0000","last_active":"2023-10-06T03:33:15.482546+0000","last_peered":"2023-10-06T03:33:15.482546+0000","last_clean":"2023-10-06T03:33:15.482546+0000","last_became_active":"2023-10-06T03:33:14.893552+0000","last_became_peered":"2023-10-06T03:33:14.893552+0000","last_unstale":"2023-10-06T03:33:15.482546+0000","last_undegraded":"2023-10-06T03:33:15.482546+0000","last_fullsized":"2023-10-06T03:33:15.482546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:29:41.510001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498193+0000","last_change":"2023-10-06T03:33:15.495951+0000","last_active":"2023-10-06T03:33:15.498193+0000","last_peered":"2023-10-06T03:33:15.498193+0000","last_clean":"2023-10-06T03:33:15.498193+0000","last_became_active":"2023-10-06T03:33:15.495675+0000","last_became_peered":"2023-10-06T03:33:15.495675+0000","last_unstale":"2023-10-06T03:33:15.498193+0000","last_undegraded":"2023-10-06T03:33:15.498193+0000","last_fullsized":"2023-10-06T03:33:15.498193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:46:13.370472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.490686+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:33:15.490686+0000","last_peered":"2023-10-06T03:33:15.490686+0000","last_clean":"2023-10-06T03:33:15.490686+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:33:15.490686+0000","last_undegraded":"2023-10-06T03:33:15.490686+0000","last_fullsized":"2023-10-06T03:33:15.490686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.480469+0000","last_change":"2023-10-06T03:33:14.901103+0000","last_active":"2023-10-06T03:33:15.480469+0000","last_peered":"2023-10-06T03:33:15.480469+0000","last_clean":"2023-10-06T03:33:15.480469+0000","last_became_active":"2023-10-06T03:33:14.900784+0000","last_became_peered":"2023-10-06T03:33:14.900784+0000","last_unstale":"2023-10-06T03:33:15.480469+0000","last_undegraded":"2023-10-06T03:33:15.480469+0000","last_fullsized":"2023-10-06T03:33:15.480469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493280+0000","last_change":"2023-10-06T03:33:15.488395+0000","last_active":"2023-10-06T03:33:15.493280+0000","last_peered":"2023-10-06T03:33:15.493280+0000","last_clean":"2023-10-06T03:33:15.493280+0000","last_became_active":"2023-10-06T03:33:15.483269+0000","last_became_peered":"2023-10-06T03:33:15.483269+0000","last_unstale":"2023-10-06T03:33:15.493280+0000","last_undegraded":"2023-10-06T03:33:15.493280+0000","last_fullsized":"2023-10-06T03:33:15.493280+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:40.421942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.477353+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:33:15.477353+0000","last_peered":"2023-10-06T03:33:15.477353+0000","last_clean":"2023-10-06T03:33:15.477353+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:33:15.477353+0000","last_undegraded":"2023-10-06T03:33:15.477353+0000","last_fullsized":"2023-10-06T03:33:15.477353+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.474703+0000","last_change":"2023-10-06T03:33:14.896667+0000","last_active":"2023-10-06T03:33:15.474703+0000","last_peered":"2023-10-06T03:33:15.474703+0000","last_clean":"2023-10-06T03:33:15.474703+0000","last_became_active":"2023-10-06T03:33:14.895890+0000","last_became_peered":"2023-10-06T03:33:14.895890+0000","last_unstale":"2023-10-06T03:33:15.474703+0000","last_undegraded":"2023-10-06T03:33:15.474703+0000","last_fullsized":"2023-10-06T03:33:15.474703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:28.684332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.498919+0000","last_change":"2023-10-06T03:33:15.496955+0000","last_active":"2023-10-06T03:33:15.498919+0000","last_peered":"2023-10-06T03:33:15.498919+0000","last_clean":"2023-10-06T03:33:15.498919+0000","last_became_active":"2023-10-06T03:33:15.496666+0000","last_became_peered":"2023-10-06T03:33:15.496666+0000","last_unstale":"2023-10-06T03:33:15.498919+0000","last_undegraded":"2023-10-06T03:33:15.498919+0000","last_fullsized":"2023-10-06T03:33:15.498919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:22.858918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.478419+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:33:15.478419+0000","last_peered":"2023-10-06T03:33:15.478419+0000","last_clean":"2023-10-06T03:33:15.478419+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:33:15.478419+0000","last_undegraded":"2023-10-06T03:33:15.478419+0000","last_fullsized":"2023-10-06T03:33:15.478419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.491835+0000","last_change":"2023-10-06T03:33:14.893120+0000","last_active":"2023-10-06T03:33:15.491835+0000","last_peered":"2023-10-06T03:33:15.491835+0000","last_clean":"2023-10-06T03:33:15.491835+0000","last_became_active":"2023-10-06T03:33:14.892839+0000","last_became_peered":"2023-10-06T03:33:14.892839+0000","last_unstale":"2023-10-06T03:33:15.491835+0000","last_undegraded":"2023-10-06T03:33:15.491835+0000","last_fullsized":"2023-10-06T03:33:15.491835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:56.088527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501655+0000","last_change":"2023-10-06T03:33:15.496296+0000","last_active":"2023-10-06T03:33:15.501655+0000","last_peered":"2023-10-06T03:33:15.501655+0000","last_clean":"2023-10-06T03:33:15.501655+0000","last_became_active":"2023-10-06T03:33:15.496028+0000","last_became_peered":"2023-10-06T03:33:15.496028+0000","last_unstale":"2023-10-06T03:33:15.501655+0000","last_undegraded":"2023-10-06T03:33:15.501655+0000","last_fullsized":"2023-10-06T03:33:15.501655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:16.946250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.884176+0000","last_change":"2023-10-06T03:33:14.897368+0000","last_active":"2023-10-06T03:33:15.884176+0000","last_peered":"2023-10-06T03:33:15.884176+0000","last_clean":"2023-10-06T03:33:15.884176+0000","last_became_active":"2023-10-06T03:33:14.892728+0000","last_became_peered":"2023-10-06T03:33:14.892728+0000","last_unstale":"2023-10-06T03:33:15.884176+0000","last_undegraded":"2023-10-06T03:33:15.884176+0000","last_fullsized":"2023-10-06T03:33:15.884176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:15.250707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494671+0000","last_change":"2023-10-06T03:33:15.493442+0000","last_active":"2023-10-06T03:33:15.494671+0000","last_peered":"2023-10-06T03:33:15.494671+0000","last_clean":"2023-10-06T03:33:15.494671+0000","last_became_active":"2023-10-06T03:33:15.493169+0000","last_became_peered":"2023-10-06T03:33:15.493169+0000","last_unstale":"2023-10-06T03:33:15.494671+0000","last_undegraded":"2023-10-06T03:33:15.494671+0000","last_fullsized":"2023-10-06T03:33:15.494671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:12.900415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886080+0000","last_change":"2023-10-06T03:33:14.899177+0000","last_active":"2023-10-06T03:33:15.886080+0000","last_peered":"2023-10-06T03:33:15.886080+0000","last_clean":"2023-10-06T03:33:15.886080+0000","last_became_active":"2023-10-06T03:33:14.898884+0000","last_became_peered":"2023-10-06T03:33:14.898884+0000","last_unstale":"2023-10-06T03:33:15.886080+0000","last_undegraded":"2023-10-06T03:33:15.886080+0000","last_fullsized":"2023-10-06T03:33:15.886080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:11.891032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.501741+0000","last_change":"2023-10-06T03:33:15.499965+0000","last_active":"2023-10-06T03:33:15.501741+0000","last_peered":"2023-10-06T03:33:15.501741+0000","last_clean":"2023-10-06T03:33:15.501741+0000","last_became_active":"2023-10-06T03:33:15.499644+0000","last_became_peered":"2023-10-06T03:33:15.499644+0000","last_unstale":"2023-10-06T03:33:15.501741+0000","last_undegraded":"2023-10-06T03:33:15.501741+0000","last_fullsized":"2023-10-06T03:33:15.501741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:47.107849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886042+0000","last_change":"2023-10-06T03:33:14.898472+0000","last_active":"2023-10-06T03:33:15.886042+0000","last_peered":"2023-10-06T03:33:15.886042+0000","last_clean":"2023-10-06T03:33:15.886042+0000","last_became_active":"2023-10-06T03:33:14.898230+0000","last_became_peered":"2023-10-06T03:33:14.898230+0000","last_unstale":"2023-10-06T03:33:15.886042+0000","last_undegraded":"2023-10-06T03:33:15.886042+0000","last_fullsized":"2023-10-06T03:33:15.886042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:07.852708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494448+0000","last_change":"2023-10-06T03:33:15.488792+0000","last_active":"2023-10-06T03:33:15.494448+0000","last_peered":"2023-10-06T03:33:15.494448+0000","last_clean":"2023-10-06T03:33:15.494448+0000","last_became_active":"2023-10-06T03:33:15.488504+0000","last_became_peered":"2023-10-06T03:33:15.488504+0000","last_unstale":"2023-10-06T03:33:15.494448+0000","last_undegraded":"2023-10-06T03:33:15.494448+0000","last_fullsized":"2023-10-06T03:33:15.494448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:59.545999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.884688+0000","last_change":"2023-10-06T03:33:14.882297+0000","last_active":"2023-10-06T03:33:15.884688+0000","last_peered":"2023-10-06T03:33:15.884688+0000","last_clean":"2023-10-06T03:33:15.884688+0000","last_became_active":"2023-10-06T03:33:14.881990+0000","last_became_peered":"2023-10-06T03:33:14.881990+0000","last_unstale":"2023-10-06T03:33:15.884688+0000","last_undegraded":"2023-10-06T03:33:15.884688+0000","last_fullsized":"2023-10-06T03:33:15.884688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:56.088527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500582+0000","last_change":"2023-10-06T03:33:15.491458+0000","last_active":"2023-10-06T03:33:15.500582+0000","last_peered":"2023-10-06T03:33:15.500582+0000","last_clean":"2023-10-06T03:33:15.500582+0000","last_became_active":"2023-10-06T03:33:15.491167+0000","last_became_peered":"2023-10-06T03:33:15.491167+0000","last_unstale":"2023-10-06T03:33:15.500582+0000","last_undegraded":"2023-10-06T03:33:15.500582+0000","last_fullsized":"2023-10-06T03:33:15.500582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:18:14.964350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.886971+0000","last_change":"2023-10-06T03:33:14.896256+0000","last_active":"2023-10-06T03:33:15.886971+0000","last_peered":"2023-10-06T03:33:15.886971+0000","last_clean":"2023-10-06T03:33:15.886971+0000","last_became_active":"2023-10-06T03:33:14.893978+0000","last_became_peered":"2023-10-06T03:33:14.893978+0000","last_unstale":"2023-10-06T03:33:15.886971+0000","last_undegraded":"2023-10-06T03:33:15.886971+0000","last_fullsized":"2023-10-06T03:33:15.886971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:13.851121+0000","last_clean_scrub_stamp":"2023-10-06T03:33:13.851121+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:39:56.088527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.494409+0000","last_change":"2023-10-06T03:33:15.485216+0000","last_active":"2023-10-06T03:33:15.494409+0000","last_peered":"2023-10-06T03:33:15.494409+0000","last_clean":"2023-10-06T03:33:15.494409+0000","last_became_active":"2023-10-06T03:33:15.484942+0000","last_became_peered":"2023-10-06T03:33:15.484942+0000","last_unstale":"2023-10-06T03:33:15.494409+0000","last_undegraded":"2023-10-06T03:33:15.494409+0000","last_fullsized":"2023-10-06T03:33:15.494409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:50.279026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.492719+0000","last_change":"2023-10-06T03:33:15.489726+0000","last_active":"2023-10-06T03:33:15.492719+0000","last_peered":"2023-10-06T03:33:15.492719+0000","last_clean":"2023-10-06T03:33:15.492719+0000","last_became_active":"2023-10-06T03:33:15.489424+0000","last_became_peered":"2023-10-06T03:33:15.489424+0000","last_unstale":"2023-10-06T03:33:15.492719+0000","last_undegraded":"2023-10-06T03:33:15.492719+0000","last_fullsized":"2023-10-06T03:33:15.492719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:29:37.682025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500986+0000","last_change":"2023-10-06T03:33:15.497582+0000","last_active":"2023-10-06T03:33:15.500986+0000","last_peered":"2023-10-06T03:33:15.500986+0000","last_clean":"2023-10-06T03:33:15.500986+0000","last_became_active":"2023-10-06T03:33:15.497328+0000","last_became_peered":"2023-10-06T03:33:15.497328+0000","last_unstale":"2023-10-06T03:33:15.500986+0000","last_undegraded":"2023-10-06T03:33:15.500986+0000","last_fullsized":"2023-10-06T03:33:15.500986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:50.279026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.496144+0000","last_change":"2023-10-06T03:33:15.489875+0000","last_active":"2023-10-06T03:33:15.496144+0000","last_peered":"2023-10-06T03:33:15.496144+0000","last_clean":"2023-10-06T03:33:15.496144+0000","last_became_active":"2023-10-06T03:33:15.489634+0000","last_became_peered":"2023-10-06T03:33:15.489634+0000","last_unstale":"2023-10-06T03:33:15.496144+0000","last_undegraded":"2023-10-06T03:33:15.496144+0000","last_fullsized":"2023-10-06T03:33:15.496144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:31:34.027062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.500770+0000","last_change":"2023-10-06T03:33:15.493502+0000","last_active":"2023-10-06T03:33:15.500770+0000","last_peered":"2023-10-06T03:33:15.500770+0000","last_clean":"2023-10-06T03:33:15.500770+0000","last_became_active":"2023-10-06T03:33:15.492769+0000","last_became_peered":"2023-10-06T03:33:15.492769+0000","last_unstale":"2023-10-06T03:33:15.500770+0000","last_undegraded":"2023-10-06T03:33:15.500770+0000","last_fullsized":"2023-10-06T03:33:15.500770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:59.545999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-06T03:33:15.493258+0000","last_change":"2023-10-06T03:33:15.487849+0000","last_active":"2023-10-06T03:33:15.493258+0000","last_peered":"2023-10-06T03:33:15.493258+0000","last_clean":"2023-10-06T03:33:15.493258+0000","last_became_active":"2023-10-06T03:33:15.487604+0000","last_became_peered":"2023-10-06T03:33:15.487604+0000","last_unstale":"2023-10-06T03:33:15.493258+0000","last_undegraded":"2023-10-06T03:33:15.493258+0000","last_fullsized":"2023-10-06T03:33:15.493258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:14.860504+0000","last_clean_scrub_stamp":"2023-10-06T03:33:14.860504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:47.342414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298104,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073736,"statfs":{"total":96636764160,"available":96331505664,"internally_reserved":0,"allocated":835584,"data_stored":670346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":79204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302176,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069664,"statfs":{"total":96636764160,"available":96327335936,"internally_reserved":0,"allocated":811008,"data_stored":669572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301596,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070244,"statfs":{"total":96636764160,"available":96327929856,"internally_reserved":0,"allocated":217088,"data_stored":78780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301592,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070248,"statfs":{"total":96636764160,"available":96327933952,"internally_reserved":0,"allocated":212992,"data_stored":78226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301600,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070240,"statfs":{"total":96636764160,"available":96327925760,"internally_reserved":0,"allocated":221184,"data_stored":79294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":237568,"data_stored":80796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1950000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297504,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074336,"statfs":{"total":96636764160,"available":96332120064,"internally_reserved":0,"allocated":221184,"data_stored":78850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:33:26.414 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T03:33:26.414 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T03:33:26.414 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T03:33:26.415 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T03:33:26.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.631+0000 7f6dd359d700 1 Processor -- start 2023-10-06T03:33:26.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.631+0000 7f6dd359d700 1 -- start start 2023-10-06T03:33:26.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.632+0000 7f6dd359d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6dcc0ce900 0x7f6dcc0cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.632+0000 7f6dd359d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dcc15cd70 0x7f6dcc15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.632+0000 7f6dd359d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dcc0cfa60 con 0x7f6dcc15c9a0 2023-10-06T03:33:26.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.632+0000 7f6dd359d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dcc0cee50 con 0x7f6dcc0ce900 2023-10-06T03:33:26.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.632+0000 7f6dd359d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0cf130 con 0x7f6dcc15cd70 2023-10-06T03:33:26.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd259b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6dcc15c9a0 0x7f6dcc0cb750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44000/0 (socket says 172.21.15.55:44000) 2023-10-06T03:33:26.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd259b700 1 -- 172.21.15.55:0/158892454 learned_addr learned my addr 172.21.15.55:0/158892454 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:26.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd1599700 1 -- 172.21.15.55:0/158892454 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930512534 0 0) 0x7f6dcc0cfa60 con 0x7f6dcc15c9a0 2023-10-06T03:33:26.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd2d9c700 1 --2- 172.21.15.55:0/158892454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dcc15cd70 0x7f6dcc15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd1599700 1 -- 172.21.15.55:0/158892454 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6db8003570 con 0x7f6dcc15c9a0 2023-10-06T03:33:26.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd1d9a700 1 --2- 172.21.15.55:0/158892454 >> v2:172.21.15.163:3300/0 conn(0x7f6dcc0ce900 0x7f6dcc0cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd1599700 1 -- 172.21.15.55:0/158892454 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 367723605 0 0) 0x7f6db8003570 con 0x7f6dcc15c9a0 2023-10-06T03:33:26.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd1599700 1 -- 172.21.15.55:0/158892454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dcc15cd70 msgr2=0x7f6dcc15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:26.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd1599700 1 --2- 172.21.15.55:0/158892454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dcc15cd70 0x7f6dcc15d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd1599700 1 -- 172.21.15.55:0/158892454 >> v2:172.21.15.163:3300/0 conn(0x7f6dcc0ce900 msgr2=0x7f6dcc0cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:26.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd1599700 1 --2- 172.21.15.55:0/158892454 >> v2:172.21.15.163:3300/0 conn(0x7f6dcc0ce900 0x7f6dcc0cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd1599700 1 -- 172.21.15.55:0/158892454 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc0cf410 con 0x7f6dcc15c9a0 2023-10-06T03:33:26.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd1599700 1 -- 172.21.15.55:0/158892454 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f6dc8001470 con 0x7f6dcc15c9a0 2023-10-06T03:33:26.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd1d9a700 1 --2- 172.21.15.55:0/158892454 >> v2:172.21.15.163:3300/0 conn(0x7f6dcc0ce900 0x7f6dcc0cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:26.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd1599700 1 -- 172.21.15.55:0/158892454 >> v1:172.21.15.55:6789/0 conn(0x7f6dcc15c9a0 legacy=0x7f6dcc0cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:26.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd1599700 1 --2- 172.21.15.55:0/158892454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db8003a60 0x7f6db8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.633+0000 7f6dd1599700 1 --2- 172.21.15.55:0/158892454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dcc15cd70 0x7f6db8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.634+0000 7f6dd1599700 1 --2- 172.21.15.55:0/158892454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db8004d50 0x7f6db80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.634+0000 7f6dd1599700 1 -- 172.21.15.55:0/158892454 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0cee50 con 0x7f6db8004d50 2023-10-06T03:33:26.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.634+0000 7f6dd1599700 1 -- 172.21.15.55:0/158892454 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0cf130 con 0x7f6db8003a60 2023-10-06T03:33:26.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.634+0000 7f6dd1599700 1 -- 172.21.15.55:0/158892454 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db800bae0 con 0x7f6dcc15cd70 2023-10-06T03:33:26.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.634+0000 7f6dd2d9c700 1 --2- 172.21.15.55:0/158892454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db8004d50 0x7f6db80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.634+0000 7f6dd259b700 1 --2- 172.21.15.55:0/158892454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db8003a60 0x7f6db8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.634+0000 7f6dd1d9a700 1 --2- 172.21.15.55:0/158892454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dcc15cd70 0x7f6db8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.634+0000 7f6dd2d9c700 1 -- 172.21.15.55:0/158892454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dcc15cd70 msgr2=0x7f6db8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:26.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.634+0000 7f6dd2d9c700 1 --2- 172.21.15.55:0/158892454 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6dcc15cd70 0x7f6db8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.634+0000 7f6dd2d9c700 1 -- 172.21.15.55:0/158892454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db8003a60 msgr2=0x7f6db8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:26.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.634+0000 7f6dd2d9c700 1 --2- 172.21.15.55:0/158892454 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db8003a60 0x7f6db8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.634+0000 7f6dd2d9c700 1 -- 172.21.15.55:0/158892454 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db8003570 con 0x7f6db8004d50 2023-10-06T03:33:26.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.634+0000 7f6dd2d9c700 1 --2- 172.21.15.55:0/158892454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db8004d50 0x7f6db80051c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400e210 tx=0x7f6dc400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392b5630b3a2bb1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:26.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.635+0000 7f6dd1599700 1 -- 172.21.15.55:0/158892454 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400eac0 con 0x7f6db8004d50 2023-10-06T03:33:26.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.635+0000 7f6dd1599700 1 -- 172.21.15.55:0/158892454 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc400f040 con 0x7f6db8004d50 2023-10-06T03:33:26.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.635+0000 7f6dd1599700 1 -- 172.21.15.55:0/158892454 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400ee70 con 0x7f6db8004d50 2023-10-06T03:33:26.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.635+0000 7f6dd359d700 1 -- 172.21.15.55:0/158892454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db8004d50 msgr2=0x7f6db80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:26.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.635+0000 7f6dd359d700 1 --2- 172.21.15.55:0/158892454 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db8004d50 0x7f6db80051c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400e210 tx=0x7f6dc400b040 comp rx=0 tx=0).stop 2023-10-06T03:33:26.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.635+0000 7f6dd2d9c700 1 -- 172.21.15.55:0/158892454 reap_dead start 2023-10-06T03:33:26.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.635+0000 7f6dd359d700 1 -- 172.21.15.55:0/158892454 shutdown_connections 2023-10-06T03:33:26.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.635+0000 7f6dd359d700 1 -- 172.21.15.55:0/158892454 >> 172.21.15.55:0/158892454 conn(0x7f6dcc0d05c0 msgr2=0x7f6dcc0d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:26.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.635+0000 7f6dd359d700 1 -- 172.21.15.55:0/158892454 shutdown_connections 2023-10-06T03:33:26.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.635+0000 7f6dd359d700 1 -- 172.21.15.55:0/158892454 wait complete. 2023-10-06T03:33:26.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.637+0000 7f6dd359d700 1 Processor -- start 2023-10-06T03:33:26.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.637+0000 7f6dd359d700 1 -- start start 2023-10-06T03:33:26.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd359d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8004d50 0x7f6dcc16aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd359d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6dcc0b3360 0x7f6dcc16c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd359d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dcc0b37d0 0x7f6dcc16c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd359d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc16bec0 con 0x7f6dcc0b3360 2023-10-06T03:33:26.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd359d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc16b010 con 0x7f6dcc0b37d0 2023-10-06T03:33:26.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd359d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc16b2b0 con 0x7f6db8004d50 2023-10-06T03:33:26.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd1d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6dcc0b3360 0x7f6dcc16c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd1d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6dcc0b3360 0x7f6dcc16c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59264/0 (socket says 172.21.15.55:59264) 2023-10-06T03:33:26.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd1d9a700 1 -- 172.21.15.55:0/2835870251 learned_addr learned my addr 172.21.15.55:0/2835870251 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:26.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd259b700 1 --2- 172.21.15.55:0/2835870251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8004d50 0x7f6dcc16aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd2d9c700 1 --2- 172.21.15.55:0/2835870251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dcc0b37d0 0x7f6dcc16c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd1d9a700 1 -- 172.21.15.55:0/2835870251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8004d50 msgr2=0x7f6dcc16aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:26.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd1d9a700 1 --2- 172.21.15.55:0/2835870251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8004d50 0x7f6dcc16aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd1d9a700 1 -- 172.21.15.55:0/2835870251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dcc0b37d0 msgr2=0x7f6dcc16c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:26.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd1d9a700 1 --2- 172.21.15.55:0/2835870251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dcc0b37d0 0x7f6dcc16c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.638+0000 7f6dd1d9a700 1 -- 172.21.15.55:0/2835870251 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc16b590 con 0x7f6dcc0b3360 2023-10-06T03:33:26.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.639+0000 7f6dd1d9a700 1 --2- 172.21.15.55:0/2835870251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6dcc0b3360 0x7f6dcc16c350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc011990 tx=0x7f6dbc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:26.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.639+0000 7f6dc27fc700 1 -- 172.21.15.55:0/2835870251 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc015670 con 0x7f6dcc0b3360 2023-10-06T03:33:26.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.639+0000 7f6dc27fc700 1 -- 172.21.15.55:0/2835870251 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc0157d0 con 0x7f6dcc0b3360 2023-10-06T03:33:26.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.639+0000 7f6dd359d700 1 -- 172.21.15.55:0/2835870251 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc16b870 con 0x7f6dcc0b3360 2023-10-06T03:33:26.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.639+0000 7f6dc27fc700 1 -- 172.21.15.55:0/2835870251 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc015b50 con 0x7f6dcc0b3360 2023-10-06T03:33:26.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.639+0000 7f6dd359d700 1 -- 172.21.15.55:0/2835870251 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc0c61e0 con 0x7f6dcc0b3360 2023-10-06T03:33:26.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.640+0000 7f6dd359d700 1 -- 172.21.15.55:0/2835870251 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dcc16c030 con 0x7f6dcc0b3360 2023-10-06T03:33:26.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.641+0000 7f6dc27fc700 1 -- 172.21.15.55:0/2835870251 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6dbc013410 con 0x7f6dcc0b3360 2023-10-06T03:33:26.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.642+0000 7f6dc27fc700 1 --2- 172.21.15.55:0/2835870251 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6db0066f30 0x7f6db00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:26.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.642+0000 7f6dc27fc700 1 -- 172.21.15.55:0/2835870251 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6dbc084660 con 0x7f6dcc0b3360 2023-10-06T03:33:26.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.642+0000 7f6dd259b700 1 --2- 172.21.15.55:0/2835870251 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6db0066f30 0x7f6db00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:26.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.643+0000 7f6dd259b700 1 --2- 172.21.15.55:0/2835870251 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6db0066f30 0x7f6db00693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc16aa80 tx=0x7f6dc8001e90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:26.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.646+0000 7f6dc27fc700 1 -- 172.21.15.55:0/2835870251 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6dbc0516d0 con 0x7f6dcc0b3360 2023-10-06T03:33:26.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.890+0000 7f6dd359d700 1 -- 172.21.15.55:0/2835870251 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6dcc0cf6f0 con 0x7f6dcc0b3360 2023-10-06T03:33:26.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.890+0000 7f6dc27fc700 1 -- 172.21.15.55:0/2835870251 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6dbc054cf0 con 0x7f6dcc0b3360 2023-10-06T03:33:26.892 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:26.892 INFO:teuthology.orchestra.run.smithi055.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T03:33:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.893+0000 7f6dd359d700 1 -- 172.21.15.55:0/2835870251 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6db0066f30 msgr2=0x7f6db00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:26.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.893+0000 7f6dd359d700 1 --2- 172.21.15.55:0/2835870251 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6db0066f30 0x7f6db00693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc16aa80 tx=0x7f6dc8001e90 comp rx=0 tx=0).stop 2023-10-06T03:33:26.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.893+0000 7f6dd359d700 1 -- 172.21.15.55:0/2835870251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6dcc0b3360 msgr2=0x7f6dcc16c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:26.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.893+0000 7f6dd359d700 1 --2- 172.21.15.55:0/2835870251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6dcc0b3360 0x7f6dcc16c350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc011990 tx=0x7f6dbc008040 comp rx=0 tx=0).stop 2023-10-06T03:33:26.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.893+0000 7f6dd359d700 1 -- 172.21.15.55:0/2835870251 shutdown_connections 2023-10-06T03:33:26.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.893+0000 7f6dd359d700 1 --2- 172.21.15.55:0/2835870251 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6dcc0b37d0 0x7f6dcc16c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.893+0000 7f6dd359d700 1 --2- 172.21.15.55:0/2835870251 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6db0066f30 0x7f6db00693e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.893+0000 7f6dd359d700 1 --2- 172.21.15.55:0/2835870251 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8004d50 0x7f6dcc16aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.893+0000 7f6dd359d700 1 --2- 172.21.15.55:0/2835870251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6dcc0b3360 0x7f6dcc16c350 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:26.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.893+0000 7f6dd359d700 1 -- 172.21.15.55:0/2835870251 >> 172.21.15.55:0/2835870251 conn(0x7f6dcc0d05c0 msgr2=0x7f6dcc0d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:26.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.893+0000 7f6dd359d700 1 -- 172.21.15.55:0/2835870251 shutdown_connections 2023-10-06T03:33:26.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:26.893+0000 7f6dd359d700 1 -- 172.21.15.55:0/2835870251 wait complete. 2023-10-06T03:33:26.912 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T03:33:26.913 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:27.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.100+0000 7f84ef59e700 1 Processor -- start 2023-10-06T03:33:27.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.100+0000 7f84ef59e700 1 -- start start 2023-10-06T03:33:27.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.101+0000 7f84ef59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f84f00ad870 0x7f84f006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.101+0000 7f84ef59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f006bf20 0x7f84f006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.101+0000 7f84ef59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f00b40f0 con 0x7f84f0142ea0 2023-10-06T03:33:27.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.101+0000 7f84ef59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f006c3a0 con 0x7f84f00ad870 2023-10-06T03:33:27.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.101+0000 7f84ef59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f0069640 con 0x7f84f006bf20 2023-10-06T03:33:27.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.101+0000 7f84edd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f84f00ad870 0x7f84f006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.101+0000 7f84edd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f84f00ad870 0x7f84f006b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59268/0 (socket says 172.21.15.55:59268) 2023-10-06T03:33:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.101+0000 7f84edd9b700 1 -- 172.21.15.55:0/4279320246 learned_addr learned my addr 172.21.15.55:0/4279320246 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.101+0000 7f84eed9d700 1 --2- 172.21.15.55:0/4279320246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f006bf20 0x7f84f006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.102+0000 7f84edd9b700 1 -- 172.21.15.55:0/4279320246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f006bf20 msgr2=0x7f84f006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.102+0000 7f84edd9b700 1 --2- 172.21.15.55:0/4279320246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f006bf20 0x7f84f006a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.102+0000 7f84edd9b700 1 -- 172.21.15.55:0/4279320246 >> v1:172.21.15.163:6789/0 conn(0x7f84f0142ea0 legacy=0x7f84f00b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.102+0000 7f84edd9b700 1 -- 172.21.15.55:0/4279320246 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f0069920 con 0x7f84f00ad870 2023-10-06T03:33:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.102+0000 7f84eed9d700 1 --2- 172.21.15.55:0/4279320246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f006bf20 0x7f84f006a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.102+0000 7f84edd9b700 1 --2- 172.21.15.55:0/4279320246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84f00ad870 0x7f84f006b9e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f84e40099b0 tx=0x7f84e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e005a38ba45560c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:27.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.102+0000 7f84ed59a700 1 -- 172.21.15.55:0/4279320246 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e400b860 con 0x7f84f00ad870 2023-10-06T03:33:27.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.102+0000 7f84ed59a700 1 -- 172.21.15.55:0/4279320246 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e400b9c0 con 0x7f84f00ad870 2023-10-06T03:33:27.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.102+0000 7f84ed59a700 1 -- 172.21.15.55:0/4279320246 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e400bd60 con 0x7f84f00ad870 2023-10-06T03:33:27.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.103+0000 7f84ef59e700 1 -- 172.21.15.55:0/4279320246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84f00ad870 msgr2=0x7f84f006b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.103+0000 7f84ef59e700 1 --2- 172.21.15.55:0/4279320246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84f00ad870 0x7f84f006b9e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f84e40099b0 tx=0x7f84e4009250 comp rx=0 tx=0).stop 2023-10-06T03:33:27.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.103+0000 7f84ef59e700 1 -- 172.21.15.55:0/4279320246 shutdown_connections 2023-10-06T03:33:27.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.103+0000 7f84ef59e700 1 --2- 172.21.15.55:0/4279320246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f006bf20 0x7f84f006a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.103+0000 7f84ef59e700 1 --2- 172.21.15.55:0/4279320246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84f00ad870 0x7f84f006b9e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.103+0000 7f84ef59e700 1 -- 172.21.15.55:0/4279320246 >> 172.21.15.55:0/4279320246 conn(0x7f84f00ce430 msgr2=0x7f84f00ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:27.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.103+0000 7f84ef59e700 1 -- 172.21.15.55:0/4279320246 shutdown_connections 2023-10-06T03:33:27.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.103+0000 7f84ef59e700 1 -- 172.21.15.55:0/4279320246 wait complete. 2023-10-06T03:33:27.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.105+0000 7f84ef59e700 1 Processor -- start 2023-10-06T03:33:27.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.105+0000 7f84ef59e700 1 -- start start 2023-10-06T03:33:27.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84ef59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f006bf20 0x7f84f00c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84ef59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84f0142ea0 0x7f84f00c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84ef59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84f013fd20 0x7f84f015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84ef59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f00ca280 con 0x7f84f0142ea0 2023-10-06T03:33:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84ef59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f015c8e0 con 0x7f84f013fd20 2023-10-06T03:33:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84ef59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f015bfb0 con 0x7f84f006bf20 2023-10-06T03:33:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84edd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84f0142ea0 0x7f84f00c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84edd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84f0142ea0 0x7f84f00c9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59274/0 (socket says 172.21.15.55:59274) 2023-10-06T03:33:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84edd9b700 1 -- 172.21.15.55:0/2024862993 learned_addr learned my addr 172.21.15.55:0/2024862993 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:27.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84ee59c700 1 --2- 172.21.15.55:0/2024862993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f006bf20 0x7f84f00c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84eed9d700 1 --2- 172.21.15.55:0/2024862993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84f013fd20 0x7f84f015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84edd9b700 1 -- 172.21.15.55:0/2024862993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f006bf20 msgr2=0x7f84f00c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84edd9b700 1 --2- 172.21.15.55:0/2024862993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f006bf20 0x7f84f00c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84edd9b700 1 -- 172.21.15.55:0/2024862993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84f013fd20 msgr2=0x7f84f015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84edd9b700 1 --2- 172.21.15.55:0/2024862993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84f013fd20 0x7f84f015cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.106+0000 7f84edd9b700 1 -- 172.21.15.55:0/2024862993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e40094f0 con 0x7f84f0142ea0 2023-10-06T03:33:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.107+0000 7f84edd9b700 1 --2- 172.21.15.55:0/2024862993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84f0142ea0 0x7f84f00c9c40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f84e40090b0 tx=0x7f84e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.107+0000 7f84de7fc700 1 -- 172.21.15.55:0/2024862993 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4026480 con 0x7f84f0142ea0 2023-10-06T03:33:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.107+0000 7f84de7fc700 1 -- 172.21.15.55:0/2024862993 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e401f070 con 0x7f84f0142ea0 2023-10-06T03:33:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.107+0000 7f84ef59e700 1 -- 172.21.15.55:0/2024862993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f015c290 con 0x7f84f0142ea0 2023-10-06T03:33:27.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.107+0000 7f84de7fc700 1 -- 172.21.15.55:0/2024862993 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4026830 con 0x7f84f0142ea0 2023-10-06T03:33:27.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.107+0000 7f84ef59e700 1 -- 172.21.15.55:0/2024862993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f00c8840 con 0x7f84f0142ea0 2023-10-06T03:33:27.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.108+0000 7f84ef59e700 1 -- 172.21.15.55:0/2024862993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f015ca50 con 0x7f84f0142ea0 2023-10-06T03:33:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.109+0000 7f84de7fc700 1 -- 172.21.15.55:0/2024862993 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f84e4026ba0 con 0x7f84f0142ea0 2023-10-06T03:33:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.110+0000 7f84de7fc700 1 --2- 172.21.15.55:0/2024862993 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f84d0066f30 0x7f84d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.110+0000 7f84de7fc700 1 -- 172.21.15.55:0/2024862993 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f84e4021070 con 0x7f84f0142ea0 2023-10-06T03:33:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.111+0000 7f84ee59c700 1 --2- 172.21.15.55:0/2024862993 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f84d0066f30 0x7f84d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.111+0000 7f84ee59c700 1 --2- 172.21.15.55:0/2024862993 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f84d0066f30 0x7f84d00693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f84e00037c0 tx=0x7f84e0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.114+0000 7f84de7fc700 1 -- 172.21.15.55:0/2024862993 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84e40536c0 con 0x7f84f0142ea0 2023-10-06T03:33:27.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.316+0000 7f84ef59e700 1 -- 172.21.15.55:0/2024862993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84f004c3c0 con 0x7f84f0142ea0 2023-10-06T03:33:27.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.317+0000 7f84de7fc700 1 -- 172.21.15.55:0/2024862993 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f84e4056ce0 con 0x7f84f0142ea0 2023-10-06T03:33:27.319 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:27.319 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:16.470817+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:27.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.319+0000 7f84ef59e700 1 -- 172.21.15.55:0/2024862993 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f84d0066f30 msgr2=0x7f84d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.319+0000 7f84ef59e700 1 --2- 172.21.15.55:0/2024862993 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f84d0066f30 0x7f84d00693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f84e00037c0 tx=0x7f84e0009040 comp rx=0 tx=0).stop 2023-10-06T03:33:27.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.319+0000 7f84ef59e700 1 -- 172.21.15.55:0/2024862993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84f0142ea0 msgr2=0x7f84f00c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.319+0000 7f84ef59e700 1 --2- 172.21.15.55:0/2024862993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84f0142ea0 0x7f84f00c9c40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f84e40090b0 tx=0x7f84e4017000 comp rx=0 tx=0).stop 2023-10-06T03:33:27.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.319+0000 7f84ef59e700 1 -- 172.21.15.55:0/2024862993 shutdown_connections 2023-10-06T03:33:27.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.319+0000 7f84ef59e700 1 --2- 172.21.15.55:0/2024862993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84f013fd20 0x7f84f015cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.319+0000 7f84ef59e700 1 --2- 172.21.15.55:0/2024862993 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f84d0066f30 0x7f84d00693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.319+0000 7f84ef59e700 1 --2- 172.21.15.55:0/2024862993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84f006bf20 0x7f84f00c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.319+0000 7f84ef59e700 1 --2- 172.21.15.55:0/2024862993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84f0142ea0 0x7f84f00c9c40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.319+0000 7f84ef59e700 1 -- 172.21.15.55:0/2024862993 >> 172.21.15.55:0/2024862993 conn(0x7f84f00ce430 msgr2=0x7f84f00cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:27.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.320+0000 7f84ef59e700 1 -- 172.21.15.55:0/2024862993 shutdown_connections 2023-10-06T03:33:27.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.320+0000 7f84ef59e700 1 -- 172.21.15.55:0/2024862993 wait complete. 2023-10-06T03:33:27.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:33:13.543361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:33:14.341572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:33:27.339 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:33:27.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.527+0000 7f0e0f59e700 1 Processor -- start 2023-10-06T03:33:27.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.527+0000 7f0e0f59e700 1 -- start start 2023-10-06T03:33:27.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.528+0000 7f0e0f59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0e10159a60 0x7f0e100c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.529+0000 7f0e0f59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0e100cb230 0x7f0e100cb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.529+0000 7f0e0f59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e100cbb90 con 0x7f0e100ccc00 2023-10-06T03:33:27.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.529+0000 7f0e0f59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1006dfc0 con 0x7f0e10159a60 2023-10-06T03:33:27.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.529+0000 7f0e0f59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e100c9cb0 con 0x7f0e100cb230 2023-10-06T03:33:27.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.529+0000 7f0e0e59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0e10159a60 0x7f0e100c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.529+0000 7f0e0dd9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0e100ccc00 0x7f0e10069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44010/0 (socket says 172.21.15.55:44010) 2023-10-06T03:33:27.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.529+0000 7f0e0dd9b700 1 -- 172.21.15.55:0/2635466170 learned_addr learned my addr 172.21.15.55:0/2635466170 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:27.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.529+0000 7f0e0e59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0e10159a60 0x7f0e100c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59278/0 (socket says 172.21.15.55:59278) 2023-10-06T03:33:27.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.529+0000 7f0e0ed9d700 1 --2- 172.21.15.55:0/2635466170 >> v2:172.21.15.163:3300/0 conn(0x7f0e100cb230 0x7f0e100cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.529+0000 7f0e0e59c700 1 -- 172.21.15.55:0/2635466170 >> v2:172.21.15.163:3300/0 conn(0x7f0e100cb230 msgr2=0x7f0e100cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:27.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.529+0000 7f0e0e59c700 1 --2- 172.21.15.55:0/2635466170 >> v2:172.21.15.163:3300/0 conn(0x7f0e100cb230 0x7f0e100cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.529+0000 7f0e0e59c700 1 -- 172.21.15.55:0/2635466170 >> v1:172.21.15.55:6789/0 conn(0x7f0e100ccc00 legacy=0x7f0e10069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.529+0000 7f0e0e59c700 1 -- 172.21.15.55:0/2635466170 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e100c9f30 con 0x7f0e10159a60 2023-10-06T03:33:27.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.529+0000 7f0e0d59a700 1 -- 172.21.15.55:0/2635466170 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810204783 0 0) 0x7f0e100cbb90 con 0x7f0e100ccc00 2023-10-06T03:33:27.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.530+0000 7f0e0e59c700 1 --2- 172.21.15.55:0/2635466170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e10159a60 0x7f0e100c9770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0e100cc410 tx=0x7f0e000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab838891e2eaed53 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:27.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.530+0000 7f0e0d59a700 1 -- 172.21.15.55:0/2635466170 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000b690 con 0x7f0e10159a60 2023-10-06T03:33:27.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.530+0000 7f0e0d59a700 1 -- 172.21.15.55:0/2635466170 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0000b7f0 con 0x7f0e10159a60 2023-10-06T03:33:27.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.530+0000 7f0e0d59a700 1 -- 172.21.15.55:0/2635466170 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0000bb90 con 0x7f0e10159a60 2023-10-06T03:33:27.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.530+0000 7f0e0f59e700 1 -- 172.21.15.55:0/2635466170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e10159a60 msgr2=0x7f0e100c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.530+0000 7f0e0f59e700 1 --2- 172.21.15.55:0/2635466170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e10159a60 0x7f0e100c9770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0e100cc410 tx=0x7f0e000091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.530+0000 7f0e0f59e700 1 -- 172.21.15.55:0/2635466170 shutdown_connections 2023-10-06T03:33:27.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.530+0000 7f0e0f59e700 1 --2- 172.21.15.55:0/2635466170 >> v2:172.21.15.163:3300/0 conn(0x7f0e100cb230 0x7f0e100cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.530+0000 7f0e0f59e700 1 --2- 172.21.15.55:0/2635466170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e10159a60 0x7f0e100c9770 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.530+0000 7f0e0f59e700 1 -- 172.21.15.55:0/2635466170 >> 172.21.15.55:0/2635466170 conn(0x7f0e1006baa0 msgr2=0x7f0e1006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:27.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.531+0000 7f0e0f59e700 1 -- 172.21.15.55:0/2635466170 shutdown_connections 2023-10-06T03:33:27.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.531+0000 7f0e0f59e700 1 -- 172.21.15.55:0/2635466170 wait complete. 2023-10-06T03:33:27.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.532+0000 7f0e0f59e700 1 Processor -- start 2023-10-06T03:33:27.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.532+0000 7f0e0f59e700 1 -- start start 2023-10-06T03:33:27.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.533+0000 7f0e0f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e100cb230 0x7f0e10144cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.533+0000 7f0e0f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e100ccc00 0x7f0e100c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.533+0000 7f0e0f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e10159a60 0x7f0e100c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.533+0000 7f0e0f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100a9960 con 0x7f0e100ccc00 2023-10-06T03:33:27.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.533+0000 7f0e0f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100c7de0 con 0x7f0e100cb230 2023-10-06T03:33:27.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.533+0000 7f0e0f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100c5a00 con 0x7f0e10159a60 2023-10-06T03:33:27.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.533+0000 7f0e0dd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e100ccc00 0x7f0e100c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.533+0000 7f0e0dd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e100ccc00 0x7f0e100c6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59292/0 (socket says 172.21.15.55:59292) 2023-10-06T03:33:27.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.533+0000 7f0e0dd9b700 1 -- 172.21.15.55:0/537180691 learned_addr learned my addr 172.21.15.55:0/537180691 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:27.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0e0ed9d700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e10159a60 0x7f0e100c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0e0e59c700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e100cb230 0x7f0e10144cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0e0dd9b700 1 -- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e10159a60 msgr2=0x7f0e100c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0e0dd9b700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e10159a60 0x7f0e100c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0e0dd9b700 1 -- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e100cb230 msgr2=0x7f0e10144cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0e0dd9b700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e100cb230 0x7f0e10144cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0e0dd9b700 1 -- 172.21.15.55:0/537180691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e00009420 con 0x7f0e100ccc00 2023-10-06T03:33:27.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0e0ed9d700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e10159a60 0x7f0e100c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:27.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0e0e59c700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e100cb230 0x7f0e10144cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:27.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0e0dd9b700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e100ccc00 0x7f0e100c6630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c7e0 tx=0x7f0e04007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:27.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0dfe7fc700 1 -- 172.21.15.55:0/537180691 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e04005b30 con 0x7f0e100ccc00 2023-10-06T03:33:27.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0dfe7fc700 1 -- 172.21.15.55:0/537180691 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e04005c90 con 0x7f0e100ccc00 2023-10-06T03:33:27.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0dfe7fc700 1 -- 172.21.15.55:0/537180691 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e04007300 con 0x7f0e100ccc00 2023-10-06T03:33:27.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0e0f59e700 1 -- 172.21.15.55:0/537180691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e10145200 con 0x7f0e100ccc00 2023-10-06T03:33:27.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.534+0000 7f0e0f59e700 1 -- 172.21.15.55:0/537180691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e100c74b0 con 0x7f0e100ccc00 2023-10-06T03:33:27.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.535+0000 7f0e0f59e700 1 -- 172.21.15.55:0/537180691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1006eba0 con 0x7f0e100ccc00 2023-10-06T03:33:27.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.536+0000 7f0dfe7fc700 1 -- 172.21.15.55:0/537180691 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e0401b400 con 0x7f0e100ccc00 2023-10-06T03:33:27.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.537+0000 7f0dfe7fc700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0df0066f30 0x7f0df00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.537+0000 7f0dfe7fc700 1 -- 172.21.15.55:0/537180691 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0e04084270 con 0x7f0e100ccc00 2023-10-06T03:33:27.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.537+0000 7f0e0e59c700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0df0066f30 0x7f0df00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.538+0000 7f0e0e59c700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0df0066f30 0x7f0df00693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e000093f0 tx=0x7f0e00017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.541+0000 7f0dfe7fc700 1 -- 172.21.15.55:0/537180691 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e04051360 con 0x7f0e100ccc00 2023-10-06T03:33:27.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.743+0000 7f0e0f59e700 1 -- 172.21.15.55:0/537180691 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0e1004bb80 con 0x7f0e100ccc00 2023-10-06T03:33:27.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.744+0000 7f0dfe7fc700 1 -- 172.21.15.55:0/537180691 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0e04054980 con 0x7f0e100ccc00 2023-10-06T03:33:27.745 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:33:27.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.746+0000 7f0e0f59e700 1 -- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0df0066f30 msgr2=0x7f0df00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.746+0000 7f0e0f59e700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0df0066f30 0x7f0df00693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e000093f0 tx=0x7f0e00017000 comp rx=0 tx=0).stop 2023-10-06T03:33:27.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.746+0000 7f0e0f59e700 1 -- 172.21.15.55:0/537180691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e100ccc00 msgr2=0x7f0e100c6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.746+0000 7f0e0f59e700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e100ccc00 0x7f0e100c6630 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c7e0 tx=0x7f0e04007df0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.746+0000 7f0e0f59e700 1 -- 172.21.15.55:0/537180691 shutdown_connections 2023-10-06T03:33:27.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.746+0000 7f0e0f59e700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0df0066f30 0x7f0df00693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.746+0000 7f0e0f59e700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e10159a60 0x7f0e100c6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.746+0000 7f0e0f59e700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e100cb230 0x7f0e10144cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.746+0000 7f0e0f59e700 1 --2- 172.21.15.55:0/537180691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e100ccc00 0x7f0e100c6630 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.746+0000 7f0e0f59e700 1 -- 172.21.15.55:0/537180691 >> 172.21.15.55:0/537180691 conn(0x7f0e1006baa0 msgr2=0x7f0e1015a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:27.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.746+0000 7f0e0f59e700 1 -- 172.21.15.55:0/537180691 shutdown_connections 2023-10-06T03:33:27.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.746+0000 7f0e0f59e700 1 -- 172.21.15.55:0/537180691 wait complete. 2023-10-06T03:33:27.765 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:33:27.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.953+0000 7fad7719d700 1 Processor -- start 2023-10-06T03:33:27.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.953+0000 7fad7719d700 1 -- start start 2023-10-06T03:33:27.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.954+0000 7fad7719d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fad700c9db0 0x7fad7006a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.954+0000 7fad7719d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7006e150 con 0x7fad70157130 2023-10-06T03:33:27.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.954+0000 7fad7719d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad700c7800 con 0x7fad70153900 2023-10-06T03:33:27.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.954+0000 7fad7719d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad700c7610 con 0x7fad700c9db0 2023-10-06T03:33:27.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.955+0000 7fad7699c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fad70157130 0x7fad7006e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44012/0 (socket says 172.21.15.55:44012) 2023-10-06T03:33:27.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.955+0000 7fad7699c700 1 -- 172.21.15.55:0/2687324002 learned_addr learned my addr 172.21.15.55:0/2687324002 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:27.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.955+0000 7fad7599a700 1 --2- 172.21.15.55:0/2687324002 >> v2:172.21.15.163:3300/0 conn(0x7fad700c9db0 0x7fad7006a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.955+0000 7fad75199700 1 -- 172.21.15.55:0/2687324002 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785420156 0 0) 0x7fad7006e150 con 0x7fad70157130 2023-10-06T03:33:27.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.955+0000 7fad75199700 1 -- 172.21.15.55:0/2687324002 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad5c003570 con 0x7fad70157130 2023-10-06T03:33:27.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.955+0000 7fad75199700 1 -- 172.21.15.55:0/2687324002 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688624387 0 0) 0x7fad700c7800 con 0x7fad70153900 2023-10-06T03:33:27.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.955+0000 7fad75199700 1 -- 172.21.15.55:0/2687324002 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad7006e150 con 0x7fad70153900 2023-10-06T03:33:27.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.955+0000 7fad7599a700 1 -- 172.21.15.55:0/2687324002 >> v1:172.21.15.163:6789/0 conn(0x7fad70153900 legacy=0x7fad700c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.955+0000 7fad7599a700 1 -- 172.21.15.55:0/2687324002 >> v1:172.21.15.55:6789/0 conn(0x7fad70157130 legacy=0x7fad7006e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.955+0000 7fad7599a700 1 -- 172.21.15.55:0/2687324002 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad700c7ae0 con 0x7fad700c9db0 2023-10-06T03:33:27.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.955+0000 7fad7599a700 1 --2- 172.21.15.55:0/2687324002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad700c9db0 0x7fad7006a000 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fad6000aa90 tx=0x7fad6000a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=130d0e82b0334e6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:27.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.956+0000 7fad75199700 1 -- 172.21.15.55:0/2687324002 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6000c970 con 0x7fad700c9db0 2023-10-06T03:33:27.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.956+0000 7fad75199700 1 -- 172.21.15.55:0/2687324002 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6000cad0 con 0x7fad700c9db0 2023-10-06T03:33:27.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.956+0000 7fad75199700 1 -- 172.21.15.55:0/2687324002 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6000ce70 con 0x7fad700c9db0 2023-10-06T03:33:27.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.956+0000 7fad7719d700 1 -- 172.21.15.55:0/2687324002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad700c9db0 msgr2=0x7fad7006a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.956+0000 7fad7719d700 1 --2- 172.21.15.55:0/2687324002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad700c9db0 0x7fad7006a000 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fad6000aa90 tx=0x7fad6000a330 comp rx=0 tx=0).stop 2023-10-06T03:33:27.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.956+0000 7fad7719d700 1 -- 172.21.15.55:0/2687324002 shutdown_connections 2023-10-06T03:33:27.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.956+0000 7fad7719d700 1 --2- 172.21.15.55:0/2687324002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad700c9db0 0x7fad7006a000 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.956+0000 7fad7719d700 1 -- 172.21.15.55:0/2687324002 >> 172.21.15.55:0/2687324002 conn(0x7fad700cc1e0 msgr2=0x7fad700cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:27.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.956+0000 7fad7719d700 1 -- 172.21.15.55:0/2687324002 shutdown_connections 2023-10-06T03:33:27.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.956+0000 7fad7719d700 1 -- 172.21.15.55:0/2687324002 wait complete. 2023-10-06T03:33:27.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.958+0000 7fad7719d700 1 Processor -- start 2023-10-06T03:33:27.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.958+0000 7fad7719d700 1 -- start start 2023-10-06T03:33:27.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.959+0000 7fad7719d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad700c9db0 0x7fad700afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.959+0000 7fad7719d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad70153900 0x7fad7014afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.959+0000 7fad7719d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad70157130 0x7fad700b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.959+0000 7fad7719d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad700c1870 con 0x7fad70157130 2023-10-06T03:33:27.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.959+0000 7fad7719d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7014b520 con 0x7fad700c9db0 2023-10-06T03:33:27.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.959+0000 7fad7719d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad700c0f80 con 0x7fad70153900 2023-10-06T03:33:27.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.959+0000 7fad7699c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad70157130 0x7fad700b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.959+0000 7fad7699c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad70157130 0x7fad700b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59298/0 (socket says 172.21.15.55:59298) 2023-10-06T03:33:27.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.959+0000 7fad7699c700 1 -- 172.21.15.55:0/3404719688 learned_addr learned my addr 172.21.15.55:0/3404719688 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:27.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.959+0000 7fad7619b700 1 --2- 172.21.15.55:0/3404719688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad700c9db0 0x7fad700afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.959+0000 7fad7599a700 1 --2- 172.21.15.55:0/3404719688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad70153900 0x7fad7014afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.960+0000 7fad7699c700 1 -- 172.21.15.55:0/3404719688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad70153900 msgr2=0x7fad7014afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.960+0000 7fad7699c700 1 --2- 172.21.15.55:0/3404719688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad70153900 0x7fad7014afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.960+0000 7fad7699c700 1 -- 172.21.15.55:0/3404719688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad700c9db0 msgr2=0x7fad700afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:27.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.960+0000 7fad7699c700 1 --2- 172.21.15.55:0/3404719688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad700c9db0 0x7fad700afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:27.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.960+0000 7fad7699c700 1 -- 172.21.15.55:0/3404719688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6000a5d0 con 0x7fad70157130 2023-10-06T03:33:27.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.960+0000 7fad7699c700 1 --2- 172.21.15.55:0/3404719688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad70157130 0x7fad700b1590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fad6800cc20 tx=0x7fad68001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:27.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.960+0000 7fad667fc700 1 -- 172.21.15.55:0/3404719688 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6800a960 con 0x7fad70157130 2023-10-06T03:33:27.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.960+0000 7fad667fc700 1 -- 172.21.15.55:0/3404719688 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6800aac0 con 0x7fad70157130 2023-10-06T03:33:27.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.960+0000 7fad667fc700 1 -- 172.21.15.55:0/3404719688 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6800ae80 con 0x7fad70157130 2023-10-06T03:33:27.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.960+0000 7fad7719d700 1 -- 172.21.15.55:0/3404719688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad700b1ad0 con 0x7fad70157130 2023-10-06T03:33:27.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.960+0000 7fad7719d700 1 -- 172.21.15.55:0/3404719688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad700c1220 con 0x7fad70157130 2023-10-06T03:33:27.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.961+0000 7fad7719d700 1 -- 172.21.15.55:0/3404719688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad700c19e0 con 0x7fad70157130 2023-10-06T03:33:27.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.962+0000 7fad667fc700 1 -- 172.21.15.55:0/3404719688 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad68007300 con 0x7fad70157130 2023-10-06T03:33:27.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.963+0000 7fad667fc700 1 --2- 172.21.15.55:0/3404719688 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fad54066f30 0x7fad540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:27.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.963+0000 7fad667fc700 1 -- 172.21.15.55:0/3404719688 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fad68084b00 con 0x7fad70157130 2023-10-06T03:33:27.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.963+0000 7fad7619b700 1 --2- 172.21.15.55:0/3404719688 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fad54066f30 0x7fad540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:27.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.964+0000 7fad7619b700 1 --2- 172.21.15.55:0/3404719688 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fad54066f30 0x7fad540693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fad6c003870 tx=0x7fad6c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:27.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:27.967+0000 7fad667fc700 1 -- 172.21.15.55:0/3404719688 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad68086e60 con 0x7fad70157130 2023-10-06T03:33:28.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.169+0000 7fad7719d700 1 -- 172.21.15.55:0/3404719688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad700c7dc0 con 0x7fad70157130 2023-10-06T03:33:28.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.169+0000 7fad667fc700 1 -- 172.21.15.55:0/3404719688 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad68051bf0 con 0x7fad70157130 2023-10-06T03:33:28.171 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:33:28.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.171+0000 7fad7719d700 1 -- 172.21.15.55:0/3404719688 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fad54066f30 msgr2=0x7fad540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:28.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.171+0000 7fad7719d700 1 --2- 172.21.15.55:0/3404719688 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fad54066f30 0x7fad540693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fad6c003870 tx=0x7fad6c009040 comp rx=0 tx=0).stop 2023-10-06T03:33:28.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.171+0000 7fad7719d700 1 -- 172.21.15.55:0/3404719688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad70157130 msgr2=0x7fad700b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:28.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.171+0000 7fad7719d700 1 --2- 172.21.15.55:0/3404719688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad70157130 0x7fad700b1590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fad6800cc20 tx=0x7fad68001a40 comp rx=0 tx=0).stop 2023-10-06T03:33:28.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.172+0000 7fad7719d700 1 -- 172.21.15.55:0/3404719688 shutdown_connections 2023-10-06T03:33:28.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.172+0000 7fad7719d700 1 --2- 172.21.15.55:0/3404719688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad70157130 0x7fad700b1590 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.172+0000 7fad7719d700 1 --2- 172.21.15.55:0/3404719688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad700c9db0 0x7fad700afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.172+0000 7fad7719d700 1 --2- 172.21.15.55:0/3404719688 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fad54066f30 0x7fad540693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.172+0000 7fad7719d700 1 --2- 172.21.15.55:0/3404719688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad70153900 0x7fad7014afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.172+0000 7fad7719d700 1 -- 172.21.15.55:0/3404719688 >> 172.21.15.55:0/3404719688 conn(0x7fad700cc1e0 msgr2=0x7fad70151b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:28.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.172+0000 7fad7719d700 1 -- 172.21.15.55:0/3404719688 shutdown_connections 2023-10-06T03:33:28.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.172+0000 7fad7719d700 1 -- 172.21.15.55:0/3404719688 wait complete. 2023-10-06T03:33:28.191 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:33:28.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.380+0000 7f2d7259d700 1 Processor -- start 2023-10-06T03:33:28.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.380+0000 7f2d7259d700 1 -- start start 2023-10-06T03:33:28.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.381+0000 7f2d7259d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c0cb1a0 0x7f2d6c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.381+0000 7f2d7259d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c06ebd0 con 0x7f2d6c15a8a0 2023-10-06T03:33:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.381+0000 7f2d7259d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c15ad20 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.381+0000 7f2d7259d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c15b2e0 con 0x7f2d6c0cb1a0 2023-10-06T03:33:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.381+0000 7f2d70d9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2d6c15a8a0 0x7f2d6c06f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44014/0 (socket says 172.21.15.55:44014) 2023-10-06T03:33:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.381+0000 7f2d70d9a700 1 -- 172.21.15.55:0/1859732432 learned_addr learned my addr 172.21.15.55:0/1859732432 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.382+0000 7f2d7159b700 1 --2- 172.21.15.55:0/1859732432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c0cb1a0 0x7f2d6c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.382+0000 7f2d63fff700 1 -- 172.21.15.55:0/1859732432 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598952693 0 0) 0x7f2d6c06ebd0 con 0x7f2d6c15a8a0 2023-10-06T03:33:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.382+0000 7f2d63fff700 1 -- 172.21.15.55:0/1859732432 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d58003570 con 0x7f2d6c15a8a0 2023-10-06T03:33:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.382+0000 7f2d63fff700 1 -- 172.21.15.55:0/1859732432 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113757104 0 0) 0x7f2d6c15ad20 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.382+0000 7f2d63fff700 1 -- 172.21.15.55:0/1859732432 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d6c06ebd0 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.382+0000 7f2d7159b700 1 -- 172.21.15.55:0/1859732432 >> v1:172.21.15.163:6789/0 conn(0x7f2d6c0ae0d0 legacy=0x7f2d6c0c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.382+0000 7f2d7159b700 1 -- 172.21.15.55:0/1859732432 >> v1:172.21.15.55:6789/0 conn(0x7f2d6c15a8a0 legacy=0x7f2d6c06f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.382+0000 7f2d7159b700 1 -- 172.21.15.55:0/1859732432 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c15b000 con 0x7f2d6c0cb1a0 2023-10-06T03:33:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.382+0000 7f2d7159b700 1 --2- 172.21.15.55:0/1859732432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c0cb1a0 0x7f2d6c15a360 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009910 tx=0x7f2d680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dab5f993d754519d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.383+0000 7f2d63fff700 1 -- 172.21.15.55:0/1859732432 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800b690 con 0x7f2d6c0cb1a0 2023-10-06T03:33:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.383+0000 7f2d63fff700 1 -- 172.21.15.55:0/1859732432 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6800b7f0 con 0x7f2d6c0cb1a0 2023-10-06T03:33:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.383+0000 7f2d63fff700 1 -- 172.21.15.55:0/1859732432 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800bb90 con 0x7f2d6c0cb1a0 2023-10-06T03:33:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.383+0000 7f2d7259d700 1 -- 172.21.15.55:0/1859732432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c0cb1a0 msgr2=0x7f2d6c15a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.383+0000 7f2d7259d700 1 --2- 172.21.15.55:0/1859732432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c0cb1a0 0x7f2d6c15a360 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009910 tx=0x7f2d680091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.383+0000 7f2d7259d700 1 -- 172.21.15.55:0/1859732432 shutdown_connections 2023-10-06T03:33:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.383+0000 7f2d7259d700 1 --2- 172.21.15.55:0/1859732432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c0cb1a0 0x7f2d6c15a360 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.383+0000 7f2d7259d700 1 -- 172.21.15.55:0/1859732432 >> 172.21.15.55:0/1859732432 conn(0x7f2d6c0cc1e0 msgr2=0x7f2d6c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.383+0000 7f2d7259d700 1 -- 172.21.15.55:0/1859732432 shutdown_connections 2023-10-06T03:33:28.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.383+0000 7f2d7259d700 1 -- 172.21.15.55:0/1859732432 wait complete. 2023-10-06T03:33:28.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.385+0000 7f2d7259d700 1 Processor -- start 2023-10-06T03:33:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.385+0000 7f2d7259d700 1 -- start start 2023-10-06T03:33:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.386+0000 7f2d7259d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ae0d0 0x7f2d6c14ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.386+0000 7f2d7259d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c0cb1a0 0x7f2d6c1514a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.386+0000 7f2d7259d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d6c0b0850 0x7f2d6c0b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.386+0000 7f2d7259d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c152be0 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.386+0000 7f2d7259d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0b1230 con 0x7f2d6c0b0850 2023-10-06T03:33:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.386+0000 7f2d7259d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0aff70 con 0x7f2d6c0cb1a0 2023-10-06T03:33:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.386+0000 7f2d7159b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ae0d0 0x7f2d6c14ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.386+0000 7f2d7159b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ae0d0 0x7f2d6c14ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59306/0 (socket says 172.21.15.55:59306) 2023-10-06T03:33:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.386+0000 7f2d7159b700 1 -- 172.21.15.55:0/1979678208 learned_addr learned my addr 172.21.15.55:0/1979678208 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:28.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.386+0000 7f2d70d9a700 1 --2- 172.21.15.55:0/1979678208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c0cb1a0 0x7f2d6c1514a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:28.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.386+0000 7f2d71d9c700 1 --2- 172.21.15.55:0/1979678208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d6c0b0850 0x7f2d6c0b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:28.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.386+0000 7f2d7159b700 1 -- 172.21.15.55:0/1979678208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c0cb1a0 msgr2=0x7f2d6c1514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:28.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.386+0000 7f2d7159b700 1 --2- 172.21.15.55:0/1979678208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c0cb1a0 0x7f2d6c1514a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.387+0000 7f2d7159b700 1 -- 172.21.15.55:0/1979678208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d6c0b0850 msgr2=0x7f2d6c0b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:28.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.387+0000 7f2d7159b700 1 --2- 172.21.15.55:0/1979678208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d6c0b0850 0x7f2d6c0b0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.387+0000 7f2d7159b700 1 -- 172.21.15.55:0/1979678208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d68009420 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.387+0000 7f2d7159b700 1 --2- 172.21.15.55:0/1979678208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ae0d0 0x7f2d6c14ee10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009f00 tx=0x7f2d68017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:28.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.387+0000 7f2d617fa700 1 -- 172.21.15.55:0/1979678208 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6801f070 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.387+0000 7f2d617fa700 1 -- 172.21.15.55:0/1979678208 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d68002f70 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.387+0000 7f2d7259d700 1 -- 172.21.15.55:0/1979678208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c0af360 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.387+0000 7f2d617fa700 1 -- 172.21.15.55:0/1979678208 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6800b960 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.387+0000 7f2d7259d700 1 -- 172.21.15.55:0/1979678208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c0af920 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.388+0000 7f2d7259d700 1 -- 172.21.15.55:0/1979678208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c06c450 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.389+0000 7f2d617fa700 1 -- 172.21.15.55:0/1979678208 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d68023070 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.390+0000 7f2d617fa700 1 --2- 172.21.15.55:0/1979678208 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2d50066f30 0x7f2d500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:28.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.390+0000 7f2d617fa700 1 -- 172.21.15.55:0/1979678208 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2d68085a70 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.390+0000 7f2d70d9a700 1 --2- 172.21.15.55:0/1979678208 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2d50066f30 0x7f2d500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:28.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.391+0000 7f2d70d9a700 1 --2- 172.21.15.55:0/1979678208 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2d50066f30 0x7f2d500693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0adf70 tx=0x7f2d5c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:28.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.394+0000 7f2d617fa700 1 -- 172.21.15.55:0/1979678208 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d68052ae0 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.597+0000 7f2d7259d700 1 -- 172.21.15.55:0/1979678208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d6c06e860 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.598+0000 7f2d617fa700 1 -- 172.21.15.55:0/1979678208 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d68056100 con 0x7f2d6c0ae0d0 2023-10-06T03:33:28.599 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T03:33:28.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.600+0000 7f2d7259d700 1 -- 172.21.15.55:0/1979678208 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2d50066f30 msgr2=0x7f2d500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:28.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.600+0000 7f2d7259d700 1 --2- 172.21.15.55:0/1979678208 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2d50066f30 0x7f2d500693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0adf70 tx=0x7f2d5c009040 comp rx=0 tx=0).stop 2023-10-06T03:33:28.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.600+0000 7f2d7259d700 1 -- 172.21.15.55:0/1979678208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ae0d0 msgr2=0x7f2d6c14ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:28.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.600+0000 7f2d7259d700 1 --2- 172.21.15.55:0/1979678208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ae0d0 0x7f2d6c14ee10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009f00 tx=0x7f2d68017000 comp rx=0 tx=0).stop 2023-10-06T03:33:28.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.600+0000 7f2d7259d700 1 -- 172.21.15.55:0/1979678208 shutdown_connections 2023-10-06T03:33:28.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.600+0000 7f2d7259d700 1 --2- 172.21.15.55:0/1979678208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2d6c0b0850 0x7f2d6c0b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.600+0000 7f2d7259d700 1 --2- 172.21.15.55:0/1979678208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2d6c0ae0d0 0x7f2d6c14ee10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.600+0000 7f2d7259d700 1 --2- 172.21.15.55:0/1979678208 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2d50066f30 0x7f2d500693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.600+0000 7f2d7259d700 1 --2- 172.21.15.55:0/1979678208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2d6c0cb1a0 0x7f2d6c1514a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.600+0000 7f2d7259d700 1 -- 172.21.15.55:0/1979678208 >> 172.21.15.55:0/1979678208 conn(0x7f2d6c0cc1e0 msgr2=0x7f2d6c0c79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:28.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.600+0000 7f2d7259d700 1 -- 172.21.15.55:0/1979678208 shutdown_connections 2023-10-06T03:33:28.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.600+0000 7f2d7259d700 1 -- 172.21.15.55:0/1979678208 wait complete. 2023-10-06T03:33:28.619 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:33:28.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.806+0000 7fcbec99d700 1 Processor -- start 2023-10-06T03:33:28.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.807+0000 7fcbec99d700 1 -- start start 2023-10-06T03:33:28.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.808+0000 7fcbec99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe80c9db0 0x7fcbe806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:28.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.808+0000 7fcbec99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcbe8157130 0x7fcbe806a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:28.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.808+0000 7fcbec99d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe806e150 con 0x7fcbe8153900 2023-10-06T03:33:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.808+0000 7fcbec99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbe80c7610 con 0x7fcbe8157130 2023-10-06T03:33:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.808+0000 7fcbec99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe80c7800 con 0x7fcbe80c9db0 2023-10-06T03:33:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.808+0000 7fcbe7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcbe8157130 0x7fcbe806a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.808+0000 7fcbe7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcbe8157130 0x7fcbe806a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59314/0 (socket says 172.21.15.55:59314) 2023-10-06T03:33:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.808+0000 7fcbe7fff700 1 -- 172.21.15.55:0/395218001 learned_addr learned my addr 172.21.15.55:0/395218001 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.808+0000 7fcbe6ffd700 1 --2- 172.21.15.55:0/395218001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe80c9db0 0x7fcbe806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.809+0000 7fcbe7fff700 1 -- 172.21.15.55:0/395218001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe80c9db0 msgr2=0x7fcbe806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.809+0000 7fcbe7fff700 1 --2- 172.21.15.55:0/395218001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe80c9db0 0x7fcbe806a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.809+0000 7fcbe7fff700 1 -- 172.21.15.55:0/395218001 >> v1:172.21.15.163:6789/0 conn(0x7fcbe8153900 legacy=0x7fcbe80c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.809+0000 7fcbe7fff700 1 -- 172.21.15.55:0/395218001 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe80c7ae0 con 0x7fcbe8157130 2023-10-06T03:33:28.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.809+0000 7fcbe7fff700 1 --2- 172.21.15.55:0/395218001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe8157130 0x7fcbe806a540 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009f20 tx=0x7fcbe0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932f5ce8c83e78df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:28.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.809+0000 7fcbe67fc700 1 -- 172.21.15.55:0/395218001 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00092e0 con 0x7fcbe8157130 2023-10-06T03:33:28.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.809+0000 7fcbe67fc700 1 -- 172.21.15.55:0/395218001 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe000ba80 con 0x7fcbe8157130 2023-10-06T03:33:28.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.809+0000 7fcbe67fc700 1 -- 172.21.15.55:0/395218001 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe000be40 con 0x7fcbe8157130 2023-10-06T03:33:28.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.810+0000 7fcbec99d700 1 -- 172.21.15.55:0/395218001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe8157130 msgr2=0x7fcbe806a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:28.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.810+0000 7fcbec99d700 1 --2- 172.21.15.55:0/395218001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe8157130 0x7fcbe806a540 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009f20 tx=0x7fcbe0009840 comp rx=0 tx=0).stop 2023-10-06T03:33:28.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.810+0000 7fcbec99d700 1 -- 172.21.15.55:0/395218001 shutdown_connections 2023-10-06T03:33:28.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.810+0000 7fcbec99d700 1 --2- 172.21.15.55:0/395218001 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe8157130 0x7fcbe806a540 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.810+0000 7fcbec99d700 1 --2- 172.21.15.55:0/395218001 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe80c9db0 0x7fcbe806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.810+0000 7fcbec99d700 1 -- 172.21.15.55:0/395218001 >> 172.21.15.55:0/395218001 conn(0x7fcbe80cc1e0 msgr2=0x7fcbe80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.810+0000 7fcbec99d700 1 -- 172.21.15.55:0/395218001 shutdown_connections 2023-10-06T03:33:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.810+0000 7fcbec99d700 1 -- 172.21.15.55:0/395218001 wait complete. 2023-10-06T03:33:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.811+0000 7fcbec99d700 1 Processor -- start 2023-10-06T03:33:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.811+0000 7fcbec99d700 1 -- start start 2023-10-06T03:33:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.812+0000 7fcbec99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe80c9db0 0x7fcbe80c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.812+0000 7fcbec99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe8153900 0x7fcbe80ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.812+0000 7fcbec99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbe8157130 0x7fcbe80ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.812+0000 7fcbec99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe80b1ee0 con 0x7fcbe8153900 2023-10-06T03:33:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.812+0000 7fcbec99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe80c59d0 con 0x7fcbe8157130 2023-10-06T03:33:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.812+0000 7fcbec99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe80c5cb0 con 0x7fcbe80c9db0 2023-10-06T03:33:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.812+0000 7fcbe6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe8153900 0x7fcbe80ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:28.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.812+0000 7fcbe6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe8153900 0x7fcbe80ae460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59330/0 (socket says 172.21.15.55:59330) 2023-10-06T03:33:28.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.812+0000 7fcbe6ffd700 1 -- 172.21.15.55:0/391774175 learned_addr learned my addr 172.21.15.55:0/391774175 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:28.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.813+0000 7fcbe7fff700 1 --2- 172.21.15.55:0/391774175 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbe8157130 0x7fcbe80ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:28.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.813+0000 7fcbe77fe700 1 --2- 172.21.15.55:0/391774175 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe80c9db0 0x7fcbe80c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:28.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.813+0000 7fcbe6ffd700 1 -- 172.21.15.55:0/391774175 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe80c9db0 msgr2=0x7fcbe80c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:28.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.813+0000 7fcbe6ffd700 1 --2- 172.21.15.55:0/391774175 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe80c9db0 0x7fcbe80c5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.813+0000 7fcbe6ffd700 1 -- 172.21.15.55:0/391774175 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbe8157130 msgr2=0x7fcbe80ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:28.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.813+0000 7fcbe6ffd700 1 --2- 172.21.15.55:0/391774175 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbe8157130 0x7fcbe80ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:28.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.813+0000 7fcbe6ffd700 1 -- 172.21.15.55:0/391774175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe0009ae0 con 0x7fcbe8153900 2023-10-06T03:33:28.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.813+0000 7fcbe6ffd700 1 --2- 172.21.15.55:0/391774175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe8153900 0x7fcbe80ae460 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00d890 tx=0x7fcbdc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:28.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.813+0000 7fcbcf7fe700 1 -- 172.21.15.55:0/391774175 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0125f0 con 0x7fcbe8153900 2023-10-06T03:33:28.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.813+0000 7fcbcf7fe700 1 -- 172.21.15.55:0/391774175 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbdc012750 con 0x7fcbe8153900 2023-10-06T03:33:28.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.813+0000 7fcbcf7fe700 1 -- 172.21.15.55:0/391774175 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc012af0 con 0x7fcbe8153900 2023-10-06T03:33:28.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.813+0000 7fcbec99d700 1 -- 172.21.15.55:0/391774175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe80c5f90 con 0x7fcbe8153900 2023-10-06T03:33:28.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.814+0000 7fcbec99d700 1 -- 172.21.15.55:0/391774175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe80adca0 con 0x7fcbe8153900 2023-10-06T03:33:28.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.815+0000 7fcbec99d700 1 -- 172.21.15.55:0/391774175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe80adec0 con 0x7fcbe8153900 2023-10-06T03:33:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.815+0000 7fcbcf7fe700 1 -- 172.21.15.55:0/391774175 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcbdc01e460 con 0x7fcbe8153900 2023-10-06T03:33:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.816+0000 7fcbcf7fe700 1 --2- 172.21.15.55:0/391774175 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fcbd4066f30 0x7fcbd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.816+0000 7fcbcf7fe700 1 -- 172.21.15.55:0/391774175 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcbdc084c20 con 0x7fcbe8153900 2023-10-06T03:33:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.817+0000 7fcbe77fe700 1 --2- 172.21.15.55:0/391774175 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fcbd4066f30 0x7fcbd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.817+0000 7fcbe77fe700 1 --2- 172.21.15.55:0/391774175 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fcbd4066f30 0x7fcbd40693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8003a80 tx=0x7fcbd8007ec0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:28.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:28.820+0000 7fcbcf7fe700 1 -- 172.21.15.55:0/391774175 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbdc051c90 con 0x7fcbe8153900 2023-10-06T03:33:29.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.019+0000 7fcbec99d700 1 -- 172.21.15.55:0/391774175 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcbe80c65e0 con 0x7fcbe8153900 2023-10-06T03:33:29.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.019+0000 7fcbcf7fe700 1 -- 172.21.15.55:0/391774175 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcbe80c65e0 con 0x7fcbe8153900 2023-10-06T03:33:29.021 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T03:33:29.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.021+0000 7fcbec99d700 1 -- 172.21.15.55:0/391774175 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fcbd4066f30 msgr2=0x7fcbd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.021+0000 7fcbec99d700 1 --2- 172.21.15.55:0/391774175 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fcbd4066f30 0x7fcbd40693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcbd8003a80 tx=0x7fcbd8007ec0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.021+0000 7fcbec99d700 1 -- 172.21.15.55:0/391774175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe8153900 msgr2=0x7fcbe80ae460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.021+0000 7fcbec99d700 1 --2- 172.21.15.55:0/391774175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe8153900 0x7fcbe80ae460 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00d890 tx=0x7fcbdc008040 comp rx=0 tx=0).stop 2023-10-06T03:33:29.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.021+0000 7fcbec99d700 1 -- 172.21.15.55:0/391774175 shutdown_connections 2023-10-06T03:33:29.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.021+0000 7fcbec99d700 1 --2- 172.21.15.55:0/391774175 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcbe8157130 0x7fcbe80ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.021+0000 7fcbec99d700 1 --2- 172.21.15.55:0/391774175 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fcbd4066f30 0x7fcbd40693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.021+0000 7fcbec99d700 1 --2- 172.21.15.55:0/391774175 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcbe80c9db0 0x7fcbe80c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.021+0000 7fcbec99d700 1 --2- 172.21.15.55:0/391774175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcbe8153900 0x7fcbe80ae460 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.021+0000 7fcbec99d700 1 -- 172.21.15.55:0/391774175 >> 172.21.15.55:0/391774175 conn(0x7fcbe80cc1e0 msgr2=0x7fcbe8158540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:29.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.021+0000 7fcbec99d700 1 -- 172.21.15.55:0/391774175 shutdown_connections 2023-10-06T03:33:29.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.022+0000 7fcbec99d700 1 -- 172.21.15.55:0/391774175 wait complete. 2023-10-06T03:33:29.040 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:33:29.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.226+0000 7fb345d9d700 1 Processor -- start 2023-10-06T03:33:29.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.226+0000 7fb345d9d700 1 -- start start 2023-10-06T03:33:29.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.227+0000 7fb345d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3401597f0 0x7fb3400b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:29.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.227+0000 7fb345d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3400b11d0 con 0x7fb340159420 2023-10-06T03:33:29.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.227+0000 7fb345d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3400b14b0 con 0x7fb34015d230 2023-10-06T03:33:29.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.227+0000 7fb345d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400b1790 con 0x7fb3401597f0 2023-10-06T03:33:29.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.227+0000 7fb344d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb340159420 0x7fb3400b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44026/0 (socket says 172.21.15.55:44026) 2023-10-06T03:33:29.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.227+0000 7fb344d9b700 1 -- 172.21.15.55:0/3102703830 learned_addr learned my addr 172.21.15.55:0/3102703830 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:29.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.228+0000 7fb33ffff700 1 --2- 172.21.15.55:0/3102703830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3401597f0 0x7fb3400b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:29.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.228+0000 7fb33f7fe700 1 -- 172.21.15.55:0/3102703830 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111304711 0 0) 0x7fb3400b11d0 con 0x7fb340159420 2023-10-06T03:33:29.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.228+0000 7fb33f7fe700 1 -- 172.21.15.55:0/3102703830 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb330003570 con 0x7fb340159420 2023-10-06T03:33:29.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.228+0000 7fb33f7fe700 1 -- 172.21.15.55:0/3102703830 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858165746 0 0) 0x7fb3400b14b0 con 0x7fb34015d230 2023-10-06T03:33:29.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.228+0000 7fb33f7fe700 1 -- 172.21.15.55:0/3102703830 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3400b11d0 con 0x7fb34015d230 2023-10-06T03:33:29.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.228+0000 7fb33ffff700 1 -- 172.21.15.55:0/3102703830 >> v1:172.21.15.163:6789/0 conn(0x7fb34015d230 legacy=0x7fb3400b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.228+0000 7fb33ffff700 1 -- 172.21.15.55:0/3102703830 >> v1:172.21.15.55:6789/0 conn(0x7fb340159420 legacy=0x7fb3400b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.228+0000 7fb33ffff700 1 -- 172.21.15.55:0/3102703830 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400b1a70 con 0x7fb3401597f0 2023-10-06T03:33:29.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.228+0000 7fb33ffff700 1 --2- 172.21.15.55:0/3102703830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3401597f0 0x7fb3400b2270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009910 tx=0x7fb32c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51c3746a27133f1b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:29.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.229+0000 7fb33f7fe700 1 -- 172.21.15.55:0/3102703830 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c011630 con 0x7fb3401597f0 2023-10-06T03:33:29.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.229+0000 7fb33f7fe700 1 -- 172.21.15.55:0/3102703830 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c011790 con 0x7fb3401597f0 2023-10-06T03:33:29.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.229+0000 7fb33f7fe700 1 -- 172.21.15.55:0/3102703830 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c011b30 con 0x7fb3401597f0 2023-10-06T03:33:29.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.229+0000 7fb345d9d700 1 -- 172.21.15.55:0/3102703830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3401597f0 msgr2=0x7fb3400b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.229+0000 7fb345d9d700 1 --2- 172.21.15.55:0/3102703830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3401597f0 0x7fb3400b2270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009910 tx=0x7fb32c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.229+0000 7fb345d9d700 1 -- 172.21.15.55:0/3102703830 shutdown_connections 2023-10-06T03:33:29.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.229+0000 7fb345d9d700 1 --2- 172.21.15.55:0/3102703830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3401597f0 0x7fb3400b2270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.229+0000 7fb345d9d700 1 -- 172.21.15.55:0/3102703830 >> 172.21.15.55:0/3102703830 conn(0x7fb3400d0660 msgr2=0x7fb3400d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:29.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.229+0000 7fb345d9d700 1 -- 172.21.15.55:0/3102703830 shutdown_connections 2023-10-06T03:33:29.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.229+0000 7fb345d9d700 1 -- 172.21.15.55:0/3102703830 wait complete. 2023-10-06T03:33:29.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.231+0000 7fb345d9d700 1 Processor -- start 2023-10-06T03:33:29.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.231+0000 7fb345d9d700 1 -- start start 2023-10-06T03:33:29.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb345d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb340159420 0x7fb34015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:29.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb345d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3401597f0 0x7fb340169940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:29.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb345d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb34015d230 0x7fb340169e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:29.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb345d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34016aee0 con 0x7fb340159420 2023-10-06T03:33:29.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb345d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34016abb0 con 0x7fb34015d230 2023-10-06T03:33:29.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb345d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34015b0d0 con 0x7fb3401597f0 2023-10-06T03:33:29.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb344d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb340159420 0x7fb34015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:29.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb344d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb340159420 0x7fb34015a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59334/0 (socket says 172.21.15.55:59334) 2023-10-06T03:33:29.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb344d9b700 1 -- 172.21.15.55:0/1785106652 learned_addr learned my addr 172.21.15.55:0/1785106652 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:29.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb34559c700 1 --2- 172.21.15.55:0/1785106652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb34015d230 0x7fb340169e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:29.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb33ffff700 1 --2- 172.21.15.55:0/1785106652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3401597f0 0x7fb340169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:29.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb344d9b700 1 -- 172.21.15.55:0/1785106652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3401597f0 msgr2=0x7fb340169940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb344d9b700 1 --2- 172.21.15.55:0/1785106652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3401597f0 0x7fb340169940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb344d9b700 1 -- 172.21.15.55:0/1785106652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb34015d230 msgr2=0x7fb340169e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb344d9b700 1 --2- 172.21.15.55:0/1785106652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb34015d230 0x7fb340169e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.232+0000 7fb344d9b700 1 -- 172.21.15.55:0/1785106652 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c009420 con 0x7fb340159420 2023-10-06T03:33:29.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.233+0000 7fb344d9b700 1 --2- 172.21.15.55:0/1785106652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb340159420 0x7fb34015a610 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb33400c790 tx=0x7fb334007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:29.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.233+0000 7fb33cff9700 1 -- 172.21.15.55:0/1785106652 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334005b30 con 0x7fb340159420 2023-10-06T03:33:29.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.233+0000 7fb33cff9700 1 -- 172.21.15.55:0/1785106652 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb334005c90 con 0x7fb340159420 2023-10-06T03:33:29.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.233+0000 7fb33cff9700 1 -- 172.21.15.55:0/1785106652 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334007300 con 0x7fb340159420 2023-10-06T03:33:29.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.233+0000 7fb345d9d700 1 -- 172.21.15.55:0/1785106652 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34015b3b0 con 0x7fb340159420 2023-10-06T03:33:29.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.233+0000 7fb345d9d700 1 -- 172.21.15.55:0/1785106652 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34015b690 con 0x7fb340159420 2023-10-06T03:33:29.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.234+0000 7fb345d9d700 1 -- 172.21.15.55:0/1785106652 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34004c3c0 con 0x7fb340159420 2023-10-06T03:33:29.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.235+0000 7fb33cff9700 1 -- 172.21.15.55:0/1785106652 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb33401b400 con 0x7fb340159420 2023-10-06T03:33:29.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.235+0000 7fb33cff9700 1 --2- 172.21.15.55:0/1785106652 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb324066f30 0x7fb3240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:29.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.236+0000 7fb33cff9700 1 -- 172.21.15.55:0/1785106652 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb334084260 con 0x7fb340159420 2023-10-06T03:33:29.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.236+0000 7fb33ffff700 1 --2- 172.21.15.55:0/1785106652 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb324066f30 0x7fb3240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:29.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.237+0000 7fb33ffff700 1 --2- 172.21.15.55:0/1785106652 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb324066f30 0x7fb3240693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009860 tx=0x7fb32c017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:29.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.240+0000 7fb33cff9700 1 -- 172.21.15.55:0/1785106652 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb334054e90 con 0x7fb340159420 2023-10-06T03:33:29.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.485+0000 7fb345d9d700 1 -- 172.21.15.55:0/1785106652 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3400b1d50 con 0x7fb340159420 2023-10-06T03:33:29.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.486+0000 7fb33cff9700 1 -- 172.21.15.55:0/1785106652 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7fb3340076f0 con 0x7fb340159420 2023-10-06T03:33:29.487 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:29.487 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2677235076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":2677235076},{"type":"v1","addr":"172.21.15.55:6835","nonce":2677235076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1030747394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1030747394},{"type":"v1","addr":"172.21.15.55:6834","nonce":1030747394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2040975046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2040975046},{"type":"v1","addr":"172.21.15.163:6836","nonce":2040975046}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:33:15.383124+0000","modified":"2023-10-06T03:33:18.908835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/1910887369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1910887369},{"type":"v1","addr":"172.21.15.163:6837","nonce":1910887369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:33:29.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.488+0000 7fb345d9d700 1 -- 172.21.15.55:0/1785106652 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb324066f30 msgr2=0x7fb3240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.488+0000 7fb345d9d700 1 --2- 172.21.15.55:0/1785106652 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb324066f30 0x7fb3240693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009860 tx=0x7fb32c017000 comp rx=0 tx=0).stop 2023-10-06T03:33:29.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.488+0000 7fb345d9d700 1 -- 172.21.15.55:0/1785106652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb340159420 msgr2=0x7fb34015a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.488+0000 7fb345d9d700 1 --2- 172.21.15.55:0/1785106652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb340159420 0x7fb34015a610 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb33400c790 tx=0x7fb334007ea0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.488+0000 7fb345d9d700 1 -- 172.21.15.55:0/1785106652 shutdown_connections 2023-10-06T03:33:29.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.488+0000 7fb345d9d700 1 --2- 172.21.15.55:0/1785106652 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb34015d230 0x7fb340169e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.488+0000 7fb345d9d700 1 --2- 172.21.15.55:0/1785106652 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb340159420 0x7fb34015a610 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.488+0000 7fb345d9d700 1 --2- 172.21.15.55:0/1785106652 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb324066f30 0x7fb3240693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.488+0000 7fb345d9d700 1 --2- 172.21.15.55:0/1785106652 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3401597f0 0x7fb340169940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.488+0000 7fb345d9d700 1 -- 172.21.15.55:0/1785106652 >> 172.21.15.55:0/1785106652 conn(0x7fb3400d0660 msgr2=0x7fb340155290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:29.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.488+0000 7fb345d9d700 1 -- 172.21.15.55:0/1785106652 shutdown_connections 2023-10-06T03:33:29.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.488+0000 7fb345d9d700 1 -- 172.21.15.55:0/1785106652 wait complete. 2023-10-06T03:33:29.491 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 8 2023-10-06T03:33:29.507 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:29.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.693+0000 7f960139d700 1 Processor -- start 2023-10-06T03:33:29.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.693+0000 7f960139d700 1 -- start start 2023-10-06T03:33:29.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.694+0000 7f960139d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95fc1597f0 0x7f95fc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:29.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.694+0000 7f960139d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f95fc15d230 0x7f95fc15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:29.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.694+0000 7f960139d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95fc15dbd0 con 0x7f95fc159420 2023-10-06T03:33:29.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.694+0000 7f960139d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc0b11d0 con 0x7f95fc15d230 2023-10-06T03:33:29.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.694+0000 7f960139d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0b14b0 con 0x7f95fc1597f0 2023-10-06T03:33:29.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.695+0000 7f9600b9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f95fc15d230 0x7f95fc15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:29.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.695+0000 7f9600b9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f95fc15d230 0x7f95fc15d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59346/0 (socket says 172.21.15.55:59346) 2023-10-06T03:33:29.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.695+0000 7f9600b9c700 1 -- 172.21.15.55:0/927573583 learned_addr learned my addr 172.21.15.55:0/927573583 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:29.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.695+0000 7f95fb7fe700 1 --2- 172.21.15.55:0/927573583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95fc1597f0 0x7f95fc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:29.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.695+0000 7f9600b9c700 1 -- 172.21.15.55:0/927573583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95fc1597f0 msgr2=0x7f95fc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:29.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.695+0000 7f9600b9c700 1 --2- 172.21.15.55:0/927573583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95fc1597f0 0x7f95fc0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.698 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.695+0000 7f9600b9c700 1 -- 172.21.15.55:0/927573583 >> v1:172.21.15.163:6789/0 conn(0x7f95fc159420 legacy=0x7f95fc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.698 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.695+0000 7f9600b9c700 1 -- 172.21.15.55:0/927573583 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc0b1790 con 0x7f95fc15d230 2023-10-06T03:33:29.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.695+0000 7f9600b9c700 1 --2- 172.21.15.55:0/927573583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95fc15d230 0x7f95fc15d690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009f20 tx=0x7f95f4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2feabd0b78983c46 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:29.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.696+0000 7f95faffd700 1 -- 172.21.15.55:0/927573583 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f400e070 con 0x7f95fc15d230 2023-10-06T03:33:29.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.696+0000 7f95faffd700 1 -- 172.21.15.55:0/927573583 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f400b9d0 con 0x7f95fc15d230 2023-10-06T03:33:29.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.696+0000 7f95faffd700 1 -- 172.21.15.55:0/927573583 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f40092e0 con 0x7f95fc15d230 2023-10-06T03:33:29.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.696+0000 7f960139d700 1 -- 172.21.15.55:0/927573583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95fc15d230 msgr2=0x7f95fc15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.696+0000 7f960139d700 1 --2- 172.21.15.55:0/927573583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95fc15d230 0x7f95fc15d690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009f20 tx=0x7f95f4009840 comp rx=0 tx=0).stop 2023-10-06T03:33:29.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.696+0000 7f960139d700 1 -- 172.21.15.55:0/927573583 shutdown_connections 2023-10-06T03:33:29.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.696+0000 7f960139d700 1 --2- 172.21.15.55:0/927573583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95fc15d230 0x7f95fc15d690 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.696+0000 7f960139d700 1 --2- 172.21.15.55:0/927573583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95fc1597f0 0x7f95fc0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.696+0000 7f960139d700 1 -- 172.21.15.55:0/927573583 >> 172.21.15.55:0/927573583 conn(0x7f95fc0d0660 msgr2=0x7f95fc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:29.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.696+0000 7f960139d700 1 -- 172.21.15.55:0/927573583 shutdown_connections 2023-10-06T03:33:29.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.697+0000 7f960139d700 1 -- 172.21.15.55:0/927573583 wait complete. 2023-10-06T03:33:29.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.698+0000 7f960139d700 1 Processor -- start 2023-10-06T03:33:29.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.698+0000 7f960139d700 1 -- start start 2023-10-06T03:33:29.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.699+0000 7f960139d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95fc159420 0x7f95fc168ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:29.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.699+0000 7f960139d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95fc1597f0 0x7f95fc155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:29.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.699+0000 7f960139d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95fc15d230 0x7f95fc156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:29.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.699+0000 7f960139d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0b58e0 con 0x7f95fc1597f0 2023-10-06T03:33:29.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.699+0000 7f960139d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0b5bc0 con 0x7f95fc159420 2023-10-06T03:33:29.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.699+0000 7f960139d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0b5ea0 con 0x7f95fc15d230 2023-10-06T03:33:29.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.699+0000 7f95fb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95fc1597f0 0x7f95fc155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:29.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.699+0000 7f95fb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95fc1597f0 0x7f95fc155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59348/0 (socket says 172.21.15.55:59348) 2023-10-06T03:33:29.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.699+0000 7f95fb7fe700 1 -- 172.21.15.55:0/3191725863 learned_addr learned my addr 172.21.15.55:0/3191725863 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:29.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f9600b9c700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95fc15d230 0x7f95fc156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:29.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f95fb7fe700 1 -- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95fc15d230 msgr2=0x7f95fc156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f95fb7fe700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95fc15d230 0x7f95fc156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f95fb7fe700 1 -- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95fc159420 msgr2=0x7f95fc168ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f95fbfff700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95fc159420 0x7f95fc168ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:29.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f95fb7fe700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95fc159420 0x7f95fc168ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f95fb7fe700 1 -- 172.21.15.55:0/3191725863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f4009ae0 con 0x7f95fc1597f0 2023-10-06T03:33:29.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f9600b9c700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95fc15d230 0x7f95fc156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:29.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f95fbfff700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95fc159420 0x7f95fc168ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:29.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f95fb7fe700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95fc1597f0 0x7f95fc155cf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f95f000ec80 tx=0x7f95f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:29.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f95e3fff700 1 -- 172.21.15.55:0/3191725863 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f000c9d0 con 0x7f95fc1597f0 2023-10-06T03:33:29.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f95e3fff700 1 -- 172.21.15.55:0/3191725863 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f000cb30 con 0x7f95fc1597f0 2023-10-06T03:33:29.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f95e3fff700 1 -- 172.21.15.55:0/3191725863 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0013470 con 0x7f95fc1597f0 2023-10-06T03:33:29.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f960139d700 1 -- 172.21.15.55:0/3191725863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc0b6180 con 0x7f95fc1597f0 2023-10-06T03:33:29.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.700+0000 7f960139d700 1 -- 172.21.15.55:0/3191725863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc169410 con 0x7f95fc1597f0 2023-10-06T03:33:29.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.701+0000 7f960139d700 1 -- 172.21.15.55:0/3191725863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95fc0d1180 con 0x7f95fc1597f0 2023-10-06T03:33:29.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.703+0000 7f95e3fff700 1 -- 172.21.15.55:0/3191725863 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f95f0013820 con 0x7f95fc1597f0 2023-10-06T03:33:29.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.704+0000 7f95e3fff700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f95e8066f30 0x7f95e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:29.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.704+0000 7f95e3fff700 1 -- 172.21.15.55:0/3191725863 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f95f0084ac0 con 0x7f95fc1597f0 2023-10-06T03:33:29.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.704+0000 7f95fbfff700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f95e8066f30 0x7f95e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:29.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.705+0000 7f95fbfff700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f95e8066f30 0x7f95e80693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f95ec001840 tx=0x7f95ec009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:29.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.708+0000 7f95e3fff700 1 -- 172.21.15.55:0/3191725863 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95f0051b30 con 0x7f95fc1597f0 2023-10-06T03:33:29.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.910+0000 7f960139d700 1 -- 172.21.15.55:0/3191725863 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95fc04c3c0 con 0x7f95fc1597f0 2023-10-06T03:33:29.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.911+0000 7f95e3fff700 1 -- 172.21.15.55:0/3191725863 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f95f0055150 con 0x7f95fc1597f0 2023-10-06T03:33:29.913 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:29.913 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:16.470817+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:29.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.913+0000 7f960139d700 1 -- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f95e8066f30 msgr2=0x7f95e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.913+0000 7f960139d700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f95e8066f30 0x7f95e80693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f95ec001840 tx=0x7f95ec009040 comp rx=0 tx=0).stop 2023-10-06T03:33:29.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.913+0000 7f960139d700 1 -- 172.21.15.55:0/3191725863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95fc1597f0 msgr2=0x7f95fc155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:29.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.913+0000 7f960139d700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95fc1597f0 0x7f95fc155cf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f95f000ec80 tx=0x7f95f00091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.913+0000 7f960139d700 1 -- 172.21.15.55:0/3191725863 shutdown_connections 2023-10-06T03:33:29.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.913+0000 7f960139d700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f95e8066f30 0x7f95e80693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.913+0000 7f960139d700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95fc15d230 0x7f95fc156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.913+0000 7f960139d700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95fc159420 0x7f95fc168ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.913+0000 7f960139d700 1 --2- 172.21.15.55:0/3191725863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95fc1597f0 0x7f95fc155cf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:29.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.913+0000 7f960139d700 1 -- 172.21.15.55:0/3191725863 >> 172.21.15.55:0/3191725863 conn(0x7f95fc0d0660 msgr2=0x7f95fc157220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:29.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.913+0000 7f960139d700 1 -- 172.21.15.55:0/3191725863 shutdown_connections 2023-10-06T03:33:29.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:29.913+0000 7f960139d700 1 -- 172.21.15.55:0/3191725863 wait complete. 2023-10-06T03:33:29.932 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:33:13.543361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:33:14.341572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:33:29.933 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:33:30.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.120+0000 7f25d719d700 1 Processor -- start 2023-10-06T03:33:30.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.120+0000 7f25d719d700 1 -- start start 2023-10-06T03:33:30.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.121+0000 7f25d719d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f25d00b4e70 0x7f25d00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.121+0000 7f25d719d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d00cfb00 con 0x7f25d00b1b40 2023-10-06T03:33:30.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.121+0000 7f25d719d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d015e060 con 0x7f25d00b16d0 2023-10-06T03:33:30.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.121+0000 7f25d719d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d00b5c40 con 0x7f25d00b4e70 2023-10-06T03:33:30.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.121+0000 7f25d619b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f25d00b4e70 0x7f25d00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.121+0000 7f25d619b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f25d00b4e70 0x7f25d00ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59360/0 (socket says 172.21.15.55:59360) 2023-10-06T03:33:30.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.121+0000 7f25d619b700 1 -- 172.21.15.55:0/3109070987 learned_addr learned my addr 172.21.15.55:0/3109070987 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:30.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.121+0000 7f25d699c700 1 --1- 172.21.15.55:0/3109070987 >> v1:172.21.15.55:6789/0 conn(0x7f25d00b1b40 0x7f25d015d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44028/0 (socket says 172.21.15.55:44028) 2023-10-06T03:33:30.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.122+0000 7f25d619b700 1 -- 172.21.15.55:0/3109070987 >> v1:172.21.15.163:6789/0 conn(0x7f25d00b16d0 legacy=0x7f25d015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.122+0000 7f25d619b700 1 -- 172.21.15.55:0/3109070987 >> v1:172.21.15.55:6789/0 conn(0x7f25d00b1b40 legacy=0x7f25d015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.122+0000 7f25d619b700 1 -- 172.21.15.55:0/3109070987 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d00b5f20 con 0x7f25d00b4e70 2023-10-06T03:33:30.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.122+0000 7f25d619b700 1 --2- 172.21.15.55:0/3109070987 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00b4e70 0x7f25d00ccff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0099b0 tx=0x7f25cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f201adced1b3e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:30.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.122+0000 7f25d5199700 1 -- 172.21.15.55:0/3109070987 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc00b860 con 0x7f25d00b4e70 2023-10-06T03:33:30.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.122+0000 7f25d5199700 1 -- 172.21.15.55:0/3109070987 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc00b9c0 con 0x7f25d00b4e70 2023-10-06T03:33:30.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.122+0000 7f25d5199700 1 -- 172.21.15.55:0/3109070987 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc00bd60 con 0x7f25d00b4e70 2023-10-06T03:33:30.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.123+0000 7f25d719d700 1 -- 172.21.15.55:0/3109070987 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00b4e70 msgr2=0x7f25d00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.123+0000 7f25d719d700 1 --2- 172.21.15.55:0/3109070987 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00b4e70 0x7f25d00ccff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0099b0 tx=0x7f25cc009250 comp rx=0 tx=0).stop 2023-10-06T03:33:30.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.123+0000 7f25d719d700 1 -- 172.21.15.55:0/3109070987 shutdown_connections 2023-10-06T03:33:30.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.123+0000 7f25d719d700 1 --2- 172.21.15.55:0/3109070987 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00b4e70 0x7f25d00ccff0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.123+0000 7f25d719d700 1 -- 172.21.15.55:0/3109070987 >> 172.21.15.55:0/3109070987 conn(0x7f25d00d10c0 msgr2=0x7f25d00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:30.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.123+0000 7f25d719d700 1 -- 172.21.15.55:0/3109070987 shutdown_connections 2023-10-06T03:33:30.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.123+0000 7f25d719d700 1 -- 172.21.15.55:0/3109070987 wait complete. 2023-10-06T03:33:30.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.124+0000 7f25d719d700 1 Processor -- start 2023-10-06T03:33:30.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.125+0000 7f25d719d700 1 -- start start 2023-10-06T03:33:30.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.125+0000 7f25d719d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d00b16d0 0x7f25d00c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.125+0000 7f25d719d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00b1b40 0x7f25d016cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d719d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25d00b4e70 0x7f25d0169dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d719d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00c8ed0 con 0x7f25d00b1b40 2023-10-06T03:33:30.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d719d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00c5530 con 0x7f25d00b16d0 2023-10-06T03:33:30.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d719d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00c5810 con 0x7f25d00b4e70 2023-10-06T03:33:30.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d599a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00b1b40 0x7f25d016cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d599a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00b1b40 0x7f25d016cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59362/0 (socket says 172.21.15.55:59362) 2023-10-06T03:33:30.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d599a700 1 -- 172.21.15.55:0/3371823589 learned_addr learned my addr 172.21.15.55:0/3371823589 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:30.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d619b700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d00b16d0 0x7f25d00c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d699c700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25d00b4e70 0x7f25d0169dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d599a700 1 -- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25d00b4e70 msgr2=0x7f25d0169dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d599a700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25d00b4e70 0x7f25d0169dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d599a700 1 -- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d00b16d0 msgr2=0x7f25d00c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d599a700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d00b16d0 0x7f25d00c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d599a700 1 -- 172.21.15.55:0/3371823589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc0094f0 con 0x7f25d00b1b40 2023-10-06T03:33:30.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d619b700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d00b16d0 0x7f25d00c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:30.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.126+0000 7f25d699c700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25d00b4e70 0x7f25d0169dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:30.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.127+0000 7f25d599a700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00b1b40 0x7f25d016cac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f25c000c590 tx=0x7f25c000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:30.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.127+0000 7f25c67fc700 1 -- 172.21.15.55:0/3371823589 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c0011590 con 0x7f25d00b1b40 2023-10-06T03:33:30.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.127+0000 7f25c67fc700 1 -- 172.21.15.55:0/3371823589 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c00116f0 con 0x7f25d00b1b40 2023-10-06T03:33:30.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.127+0000 7f25c67fc700 1 -- 172.21.15.55:0/3371823589 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c0011ab0 con 0x7f25d00b1b40 2023-10-06T03:33:30.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.127+0000 7f25d719d700 1 -- 172.21.15.55:0/3371823589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d00c5af0 con 0x7f25d00b1b40 2023-10-06T03:33:30.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.127+0000 7f25d719d700 1 -- 172.21.15.55:0/3371823589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d016d000 con 0x7f25d00b1b40 2023-10-06T03:33:30.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.128+0000 7f25d719d700 1 -- 172.21.15.55:0/3371823589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d00c5f90 con 0x7f25d00b1b40 2023-10-06T03:33:30.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.129+0000 7f25c67fc700 1 -- 172.21.15.55:0/3371823589 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25c001b3b0 con 0x7f25d00b1b40 2023-10-06T03:33:30.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.130+0000 7f25c67fc700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f25b4066f30 0x7f25b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.130+0000 7f25c67fc700 1 -- 172.21.15.55:0/3371823589 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f25c0083ce0 con 0x7f25d00b1b40 2023-10-06T03:33:30.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.130+0000 7f25d619b700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f25b4066f30 0x7f25b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.131+0000 7f25d619b700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f25b4066f30 0x7f25b40693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0099b0 tx=0x7f25cc017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:30.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.134+0000 7f25c67fc700 1 -- 172.21.15.55:0/3371823589 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25c0050d50 con 0x7f25d00b1b40 2023-10-06T03:33:30.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.335+0000 7f25d719d700 1 -- 172.21.15.55:0/3371823589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f25d004bb80 con 0x7f25d00b1b40 2023-10-06T03:33:30.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.335+0000 7f25c67fc700 1 -- 172.21.15.55:0/3371823589 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f25c0017020 con 0x7f25d00b1b40 2023-10-06T03:33:30.337 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:33:30.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.337+0000 7f25d719d700 1 -- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f25b4066f30 msgr2=0x7f25b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.337+0000 7f25d719d700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f25b4066f30 0x7f25b40693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0099b0 tx=0x7f25cc017000 comp rx=0 tx=0).stop 2023-10-06T03:33:30.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.337+0000 7f25d719d700 1 -- 172.21.15.55:0/3371823589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00b1b40 msgr2=0x7f25d016cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.337+0000 7f25d719d700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00b1b40 0x7f25d016cac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f25c000c590 tx=0x7f25c000c950 comp rx=0 tx=0).stop 2023-10-06T03:33:30.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.337+0000 7f25d719d700 1 -- 172.21.15.55:0/3371823589 shutdown_connections 2023-10-06T03:33:30.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.337+0000 7f25d719d700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f25b4066f30 0x7f25b40693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.337+0000 7f25d719d700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25d00b4e70 0x7f25d0169dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.337+0000 7f25d719d700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d00b16d0 0x7f25d00c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.337+0000 7f25d719d700 1 --2- 172.21.15.55:0/3371823589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00b1b40 0x7f25d016cac0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.337+0000 7f25d719d700 1 -- 172.21.15.55:0/3371823589 >> 172.21.15.55:0/3371823589 conn(0x7f25d00d10c0 msgr2=0x7f25d0159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:30.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.338+0000 7f25d719d700 1 -- 172.21.15.55:0/3371823589 shutdown_connections 2023-10-06T03:33:30.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.338+0000 7f25d719d700 1 -- 172.21.15.55:0/3371823589 wait complete. 2023-10-06T03:33:30.357 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:33:30.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.546+0000 7fa6c619d700 1 Processor -- start 2023-10-06T03:33:30.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.546+0000 7fa6c619d700 1 -- start start 2023-10-06T03:33:30.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.547+0000 7fa6c619d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6c01407e0 0x7fa6c006a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.547+0000 7fa6c619d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6c0140bb0 0x7fa6c006a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.547+0000 7fa6c619d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c00c6820 con 0x7fa6c006d4f0 2023-10-06T03:33:30.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.547+0000 7fa6c619d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c00c6de0 con 0x7fa6c01407e0 2023-10-06T03:33:30.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.547+0000 7fa6c619d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c0141270 con 0x7fa6c0140bb0 2023-10-06T03:33:30.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.548+0000 7fa6c519b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6c01407e0 0x7fa6c006a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.548+0000 7fa6c519b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6c01407e0 0x7fa6c006a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59372/0 (socket says 172.21.15.55:59372) 2023-10-06T03:33:30.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.548+0000 7fa6c599c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa6c006d4f0 0x7fa6c015b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44040/0 (socket says 172.21.15.55:44040) 2023-10-06T03:33:30.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.548+0000 7fa6c519b700 1 -- 172.21.15.55:0/2241179082 learned_addr learned my addr 172.21.15.55:0/2241179082 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:30.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.548+0000 7fa6c499a700 1 --2- 172.21.15.55:0/2241179082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6c0140bb0 0x7fa6c006a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.548+0000 7fa6c519b700 1 -- 172.21.15.55:0/2241179082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6c0140bb0 msgr2=0x7fa6c006a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:30.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.548+0000 7fa6b7fff700 1 -- 172.21.15.55:0/2241179082 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300867033 0 0) 0x7fa6c00c6820 con 0x7fa6c006d4f0 2023-10-06T03:33:30.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.548+0000 7fa6c519b700 1 --2- 172.21.15.55:0/2241179082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6c0140bb0 0x7fa6c006a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.548+0000 7fa6c519b700 1 -- 172.21.15.55:0/2241179082 >> v1:172.21.15.55:6789/0 conn(0x7fa6c006d4f0 legacy=0x7fa6c015b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.548+0000 7fa6c519b700 1 -- 172.21.15.55:0/2241179082 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c006acf0 con 0x7fa6c01407e0 2023-10-06T03:33:30.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.548+0000 7fa6c499a700 1 --2- 172.21.15.55:0/2241179082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6c0140bb0 0x7fa6c006a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:30.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.548+0000 7fa6c519b700 1 --2- 172.21.15.55:0/2241179082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c01407e0 0x7fa6c006a180 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0099b0 tx=0x7fa6bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf28b73b17909da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:30.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.549+0000 7fa6b7fff700 1 -- 172.21.15.55:0/2241179082 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00b860 con 0x7fa6c01407e0 2023-10-06T03:33:30.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.549+0000 7fa6b7fff700 1 -- 172.21.15.55:0/2241179082 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc00b9c0 con 0x7fa6c01407e0 2023-10-06T03:33:30.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.549+0000 7fa6b7fff700 1 -- 172.21.15.55:0/2241179082 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00bd60 con 0x7fa6c01407e0 2023-10-06T03:33:30.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.549+0000 7fa6c619d700 1 -- 172.21.15.55:0/2241179082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c01407e0 msgr2=0x7fa6c006a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.549+0000 7fa6c619d700 1 --2- 172.21.15.55:0/2241179082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c01407e0 0x7fa6c006a180 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0099b0 tx=0x7fa6bc009250 comp rx=0 tx=0).stop 2023-10-06T03:33:30.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.549+0000 7fa6c619d700 1 -- 172.21.15.55:0/2241179082 shutdown_connections 2023-10-06T03:33:30.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.549+0000 7fa6c619d700 1 --2- 172.21.15.55:0/2241179082 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c01407e0 0x7fa6c006a180 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.549+0000 7fa6c619d700 1 --2- 172.21.15.55:0/2241179082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6c0140bb0 0x7fa6c006a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.549+0000 7fa6c619d700 1 -- 172.21.15.55:0/2241179082 >> 172.21.15.55:0/2241179082 conn(0x7fa6c00cc1e0 msgr2=0x7fa6c00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:30.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.549+0000 7fa6c619d700 1 -- 172.21.15.55:0/2241179082 shutdown_connections 2023-10-06T03:33:30.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.549+0000 7fa6c619d700 1 -- 172.21.15.55:0/2241179082 wait complete. 2023-10-06T03:33:30.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.551+0000 7fa6c619d700 1 Processor -- start 2023-10-06T03:33:30.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.551+0000 7fa6c619d700 1 -- start start 2023-10-06T03:33:30.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c619d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c006d4f0 0x7fa6c0156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c619d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c01407e0 0x7fa6c00c23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c619d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6c0140bb0 0x7fa6c00c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c619d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c016aa60 con 0x7fa6c006d4f0 2023-10-06T03:33:30.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c619d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c016a130 con 0x7fa6c01407e0 2023-10-06T03:33:30.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c619d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c016a410 con 0x7fa6c0140bb0 2023-10-06T03:33:30.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c519b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c006d4f0 0x7fa6c0156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c519b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c006d4f0 0x7fa6c0156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59386/0 (socket says 172.21.15.55:59386) 2023-10-06T03:33:30.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c519b700 1 -- 172.21.15.55:0/2280931590 learned_addr learned my addr 172.21.15.55:0/2280931590 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:30.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c599c700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6c0140bb0 0x7fa6c00c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c499a700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c01407e0 0x7fa6c00c23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c519b700 1 -- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6c0140bb0 msgr2=0x7fa6c00c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c519b700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6c0140bb0 0x7fa6c00c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c519b700 1 -- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c01407e0 msgr2=0x7fa6c00c23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c519b700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c01407e0 0x7fa6c00c23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c519b700 1 -- 172.21.15.55:0/2280931590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc0094f0 con 0x7fa6c006d4f0 2023-10-06T03:33:30.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.552+0000 7fa6c599c700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6c0140bb0 0x7fa6c00c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:30.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.553+0000 7fa6c519b700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c006d4f0 0x7fa6c0156ec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0090b0 tx=0x7fa6bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:30.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.553+0000 7fa6b57fa700 1 -- 172.21.15.55:0/2280931590 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc01a450 con 0x7fa6c006d4f0 2023-10-06T03:33:30.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.553+0000 7fa6b57fa700 1 -- 172.21.15.55:0/2280931590 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc01a5b0 con 0x7fa6c006d4f0 2023-10-06T03:33:30.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.553+0000 7fa6b57fa700 1 -- 172.21.15.55:0/2280931590 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc01a970 con 0x7fa6c006d4f0 2023-10-06T03:33:30.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.553+0000 7fa6c619d700 1 -- 172.21.15.55:0/2280931590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c016a6f0 con 0x7fa6c006d4f0 2023-10-06T03:33:30.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.553+0000 7fa6c619d700 1 -- 172.21.15.55:0/2280931590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c015c170 con 0x7fa6c006d4f0 2023-10-06T03:33:30.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.554+0000 7fa6c619d700 1 -- 172.21.15.55:0/2280931590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c016ad40 con 0x7fa6c006d4f0 2023-10-06T03:33:30.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.555+0000 7fa6b57fa700 1 -- 172.21.15.55:0/2280931590 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa6bc01b430 con 0x7fa6c006d4f0 2023-10-06T03:33:30.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.556+0000 7fa6b57fa700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa6a4066f30 0x7fa6a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.556+0000 7fa6b57fa700 1 -- 172.21.15.55:0/2280931590 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa6bc085e70 con 0x7fa6c006d4f0 2023-10-06T03:33:30.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.556+0000 7fa6c499a700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa6a4066f30 0x7fa6a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.557+0000 7fa6c499a700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa6a4066f30 0x7fa6a40693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0006f00 tx=0x7fa6b0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:30.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.560+0000 7fa6b57fa700 1 -- 172.21.15.55:0/2280931590 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6bc052ee0 con 0x7fa6c006d4f0 2023-10-06T03:33:30.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.764+0000 7fa6c619d700 1 -- 172.21.15.55:0/2280931590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa6c006afd0 con 0x7fa6c006d4f0 2023-10-06T03:33:30.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.765+0000 7fa6b57fa700 1 -- 172.21.15.55:0/2280931590 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa6bc0150a0 con 0x7fa6c006d4f0 2023-10-06T03:33:30.766 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:33:30.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.767+0000 7fa6c619d700 1 -- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa6a4066f30 msgr2=0x7fa6a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.767+0000 7fa6c619d700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa6a4066f30 0x7fa6a40693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0006f00 tx=0x7fa6b0008040 comp rx=0 tx=0).stop 2023-10-06T03:33:30.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.767+0000 7fa6c619d700 1 -- 172.21.15.55:0/2280931590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c006d4f0 msgr2=0x7fa6c0156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.767+0000 7fa6c619d700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c006d4f0 0x7fa6c0156ec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc0090b0 tx=0x7fa6bc017000 comp rx=0 tx=0).stop 2023-10-06T03:33:30.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.767+0000 7fa6c619d700 1 -- 172.21.15.55:0/2280931590 shutdown_connections 2023-10-06T03:33:30.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.767+0000 7fa6c619d700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa6a4066f30 0x7fa6a40693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.767+0000 7fa6c619d700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6c0140bb0 0x7fa6c00c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.767+0000 7fa6c619d700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6c006d4f0 0x7fa6c0156ec0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.767+0000 7fa6c619d700 1 --2- 172.21.15.55:0/2280931590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6c01407e0 0x7fa6c00c23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.767+0000 7fa6c619d700 1 -- 172.21.15.55:0/2280931590 >> 172.21.15.55:0/2280931590 conn(0x7fa6c00cc1e0 msgr2=0x7fa6c00c12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:30.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.767+0000 7fa6c619d700 1 -- 172.21.15.55:0/2280931590 shutdown_connections 2023-10-06T03:33:30.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.767+0000 7fa6c619d700 1 -- 172.21.15.55:0/2280931590 wait complete. 2023-10-06T03:33:30.786 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:33:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.976+0000 7f36e499d700 1 Processor -- start 2023-10-06T03:33:30.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.976+0000 7f36e499d700 1 -- start start 2023-10-06T03:33:30.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.977+0000 7f36e499d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e00ae7e0 0x7f36e00c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.977+0000 7f36e499d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e006ebd0 con 0x7f36e006f060 2023-10-06T03:33:30.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.977+0000 7f36e499d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e015aa70 con 0x7f36e00cb1a0 2023-10-06T03:33:30.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.977+0000 7f36e499d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e015b2e0 con 0x7f36e00ae7e0 2023-10-06T03:33:30.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.978+0000 7f36deffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f36e006f060 0x7f36e00ae0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44048/0 (socket says 172.21.15.55:44048) 2023-10-06T03:33:30.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.978+0000 7f36deffd700 1 -- 172.21.15.55:0/3568919670 learned_addr learned my addr 172.21.15.55:0/3568919670 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:30.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.978+0000 7f36dffff700 1 --2- 172.21.15.55:0/3568919670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e00ae7e0 0x7f36e00c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.978+0000 7f36de7fc700 1 -- 172.21.15.55:0/3568919670 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 781893771 0 0) 0x7f36e006ebd0 con 0x7f36e006f060 2023-10-06T03:33:30.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.978+0000 7f36de7fc700 1 -- 172.21.15.55:0/3568919670 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36c8003570 con 0x7f36e006f060 2023-10-06T03:33:30.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.978+0000 7f36de7fc700 1 -- 172.21.15.55:0/3568919670 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 200980166 0 0) 0x7f36c8003570 con 0x7f36e006f060 2023-10-06T03:33:30.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.978+0000 7f36de7fc700 1 -- 172.21.15.55:0/3568919670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e00ae7e0 msgr2=0x7f36e00c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:30.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.978+0000 7f36de7fc700 1 --2- 172.21.15.55:0/3568919670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e00ae7e0 0x7f36e00c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.978+0000 7f36de7fc700 1 -- 172.21.15.55:0/3568919670 >> v1:172.21.15.163:6789/0 conn(0x7f36e00cb1a0 legacy=0x7f36e015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.978+0000 7f36de7fc700 1 -- 172.21.15.55:0/3568919670 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e015ad20 con 0x7f36e006f060 2023-10-06T03:33:30.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.978+0000 7f36de7fc700 1 -- 172.21.15.55:0/3568919670 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f36d4001470 con 0x7f36e006f060 2023-10-06T03:33:30.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36de7fc700 1 -- 172.21.15.55:0/3568919670 >> v1:172.21.15.55:6789/0 conn(0x7f36e006f060 legacy=0x7f36e00ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36de7fc700 1 --2- 172.21.15.55:0/3568919670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e00ae7e0 0x7f36c8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36de7fc700 1 --2- 172.21.15.55:0/3568919670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c8003fd0 0x7f36c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36de7fc700 1 --2- 172.21.15.55:0/3568919670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36c8004d50 0x7f36c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36de7fc700 1 -- 172.21.15.55:0/3568919670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e015b2e0 con 0x7f36c8004d50 2023-10-06T03:33:30.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36de7fc700 1 -- 172.21.15.55:0/3568919670 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c800bb40 con 0x7f36c8003fd0 2023-10-06T03:33:30.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36de7fc700 1 -- 172.21.15.55:0/3568919670 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c800df10 con 0x7f36e00ae7e0 2023-10-06T03:33:30.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36dffff700 1 --2- 172.21.15.55:0/3568919670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36c8004d50 0x7f36c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36deffd700 1 --2- 172.21.15.55:0/3568919670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c8003fd0 0x7f36c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36df7fe700 1 --2- 172.21.15.55:0/3568919670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e00ae7e0 0x7f36c8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36dffff700 1 -- 172.21.15.55:0/3568919670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e00ae7e0 msgr2=0x7f36c8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:30.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36dffff700 1 --2- 172.21.15.55:0/3568919670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e00ae7e0 0x7f36c8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36dffff700 1 -- 172.21.15.55:0/3568919670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c8003fd0 msgr2=0x7f36c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:30.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36dffff700 1 --2- 172.21.15.55:0/3568919670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36c8003fd0 0x7f36c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.979+0000 7f36dffff700 1 -- 172.21.15.55:0/3568919670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c8003570 con 0x7f36c8004d50 2023-10-06T03:33:30.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.980+0000 7f36dffff700 1 --2- 172.21.15.55:0/3568919670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36c8004d50 0x7f36c80051c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f36e006d5a0 tx=0x7f36d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745fed673a385609 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:30.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.980+0000 7f36de7fc700 1 -- 172.21.15.55:0/3568919670 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d800ea60 con 0x7f36c8004d50 2023-10-06T03:33:30.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.980+0000 7f36de7fc700 1 -- 172.21.15.55:0/3568919670 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d800f040 con 0x7f36c8004d50 2023-10-06T03:33:30.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.980+0000 7f36de7fc700 1 -- 172.21.15.55:0/3568919670 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d800ee10 con 0x7f36c8004d50 2023-10-06T03:33:30.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.980+0000 7f36e499d700 1 -- 172.21.15.55:0/3568919670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36c8004d50 msgr2=0x7f36c80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.980+0000 7f36e499d700 1 --2- 172.21.15.55:0/3568919670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36c8004d50 0x7f36c80051c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f36e006d5a0 tx=0x7f36d800b040 comp rx=0 tx=0).stop 2023-10-06T03:33:30.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.980+0000 7f36dffff700 1 -- 172.21.15.55:0/3568919670 reap_dead start 2023-10-06T03:33:30.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.980+0000 7f36e499d700 1 -- 172.21.15.55:0/3568919670 shutdown_connections 2023-10-06T03:33:30.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.980+0000 7f36e499d700 1 -- 172.21.15.55:0/3568919670 >> 172.21.15.55:0/3568919670 conn(0x7f36e00cc1e0 msgr2=0x7f36e00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:30.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.980+0000 7f36e499d700 1 -- 172.21.15.55:0/3568919670 shutdown_connections 2023-10-06T03:33:30.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.980+0000 7f36e499d700 1 -- 172.21.15.55:0/3568919670 wait complete. 2023-10-06T03:33:30.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.982+0000 7f36e499d700 1 Processor -- start 2023-10-06T03:33:30.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.982+0000 7f36e499d700 1 -- start start 2023-10-06T03:33:30.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.983+0000 7f36e499d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36c8004d50 0x7f36e0157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.983+0000 7f36e499d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e016a8a0 0x7f36e00c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.983+0000 7f36e499d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e0156d90 0x7f36e00c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.983+0000 7f36e499d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e01587e0 con 0x7f36c8004d50 2023-10-06T03:33:30.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.983+0000 7f36e499d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e00c5200 con 0x7f36e0156d90 2023-10-06T03:33:30.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.983+0000 7f36e499d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e016caa0 con 0x7f36e016a8a0 2023-10-06T03:33:30.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.983+0000 7f36df7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36c8004d50 0x7f36e0157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.983+0000 7f36df7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36c8004d50 0x7f36e0157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59404/0 (socket says 172.21.15.55:59404) 2023-10-06T03:33:30.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.983+0000 7f36df7fe700 1 -- 172.21.15.55:0/2300551547 learned_addr learned my addr 172.21.15.55:0/2300551547 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:30.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.983+0000 7f36deffd700 1 --2- 172.21.15.55:0/2300551547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e016a8a0 0x7f36e00c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.983+0000 7f36dffff700 1 --2- 172.21.15.55:0/2300551547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e0156d90 0x7f36e00c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.984+0000 7f36df7fe700 1 -- 172.21.15.55:0/2300551547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e016a8a0 msgr2=0x7f36e00c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.984+0000 7f36df7fe700 1 --2- 172.21.15.55:0/2300551547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e016a8a0 0x7f36e00c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.984+0000 7f36df7fe700 1 -- 172.21.15.55:0/2300551547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e0156d90 msgr2=0x7f36e00c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:30.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.984+0000 7f36df7fe700 1 --2- 172.21.15.55:0/2300551547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e0156d90 0x7f36e00c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:30.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.984+0000 7f36df7fe700 1 -- 172.21.15.55:0/2300551547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e00c4950 con 0x7f36c8004d50 2023-10-06T03:33:30.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.984+0000 7f36df7fe700 1 --2- 172.21.15.55:0/2300551547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36c8004d50 0x7f36e0157c50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f36d000e7b0 tx=0x7f36d0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:30.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.984+0000 7f36c77fe700 1 -- 172.21.15.55:0/2300551547 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0001630 con 0x7f36c8004d50 2023-10-06T03:33:30.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.984+0000 7f36c77fe700 1 -- 172.21.15.55:0/2300551547 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d0010040 con 0x7f36c8004d50 2023-10-06T03:33:30.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.984+0000 7f36c77fe700 1 -- 172.21.15.55:0/2300551547 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d0001a00 con 0x7f36c8004d50 2023-10-06T03:33:30.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.984+0000 7f36e499d700 1 -- 172.21.15.55:0/2300551547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e00c4bd0 con 0x7f36c8004d50 2023-10-06T03:33:30.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.984+0000 7f36e499d700 1 -- 172.21.15.55:0/2300551547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e00c4e90 con 0x7f36c8004d50 2023-10-06T03:33:30.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.985+0000 7f36e499d700 1 -- 172.21.15.55:0/2300551547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e006c450 con 0x7f36c8004d50 2023-10-06T03:33:30.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.986+0000 7f36c77fe700 1 -- 172.21.15.55:0/2300551547 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f36d001d400 con 0x7f36c8004d50 2023-10-06T03:33:30.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.987+0000 7f36c77fe700 1 --2- 172.21.15.55:0/2300551547 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f36cc066f30 0x7f36cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:30.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.987+0000 7f36c77fe700 1 -- 172.21.15.55:0/2300551547 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f36d0084510 con 0x7f36c8004d50 2023-10-06T03:33:30.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.987+0000 7f36deffd700 1 --2- 172.21.15.55:0/2300551547 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f36cc066f30 0x7f36cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:30.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.988+0000 7f36deffd700 1 --2- 172.21.15.55:0/2300551547 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f36cc066f30 0x7f36cc0693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f36e0159ab0 tx=0x7f36d4001df0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:30.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:30.991+0000 7f36c77fe700 1 -- 172.21.15.55:0/2300551547 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36d0051630 con 0x7f36c8004d50 2023-10-06T03:33:31.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.192+0000 7f36e499d700 1 -- 172.21.15.55:0/2300551547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36e015b000 con 0x7f36c8004d50 2023-10-06T03:33:31.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.193+0000 7f36c77fe700 1 -- 172.21.15.55:0/2300551547 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f36d0001db0 con 0x7f36c8004d50 2023-10-06T03:33:31.194 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T03:33:31.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.195+0000 7f36e499d700 1 -- 172.21.15.55:0/2300551547 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f36cc066f30 msgr2=0x7f36cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:31.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.195+0000 7f36e499d700 1 --2- 172.21.15.55:0/2300551547 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f36cc066f30 0x7f36cc0693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f36e0159ab0 tx=0x7f36d4001df0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.195+0000 7f36e499d700 1 -- 172.21.15.55:0/2300551547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36c8004d50 msgr2=0x7f36e0157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:31.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.195+0000 7f36e499d700 1 --2- 172.21.15.55:0/2300551547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36c8004d50 0x7f36e0157c50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f36d000e7b0 tx=0x7f36d0009040 comp rx=0 tx=0).stop 2023-10-06T03:33:31.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.195+0000 7f36e499d700 1 -- 172.21.15.55:0/2300551547 shutdown_connections 2023-10-06T03:33:31.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.195+0000 7f36e499d700 1 --2- 172.21.15.55:0/2300551547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36e0156d90 0x7f36e00c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.195+0000 7f36e499d700 1 --2- 172.21.15.55:0/2300551547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36c8004d50 0x7f36e0157c50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.195+0000 7f36e499d700 1 --2- 172.21.15.55:0/2300551547 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f36cc066f30 0x7f36cc0693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.195+0000 7f36e499d700 1 --2- 172.21.15.55:0/2300551547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36e016a8a0 0x7f36e00c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.195+0000 7f36e499d700 1 -- 172.21.15.55:0/2300551547 >> 172.21.15.55:0/2300551547 conn(0x7f36e00cc1e0 msgr2=0x7f36e00c1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:31.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.195+0000 7f36e499d700 1 -- 172.21.15.55:0/2300551547 shutdown_connections 2023-10-06T03:33:31.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.195+0000 7f36e499d700 1 -- 172.21.15.55:0/2300551547 wait complete. 2023-10-06T03:33:31.214 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:33:31.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.401+0000 7fbf4c9fe700 1 Processor -- start 2023-10-06T03:33:31.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.401+0000 7fbf4c9fe700 1 -- start start 2023-10-06T03:33:31.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.402+0000 7fbf4c9fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbf48159a60 0x7fbf480c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:31.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.402+0000 7fbf4c9fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbf480cb230 0x7fbf480cb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:31.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.402+0000 7fbf4c9fe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf480cbb90 con 0x7fbf480ccc00 2023-10-06T03:33:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.403+0000 7fbf4c9fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4806dfc0 con 0x7fbf48159a60 2023-10-06T03:33:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.403+0000 7fbf4c9fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf480c9cb0 con 0x7fbf480cb230 2023-10-06T03:33:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.403+0000 7fbf46d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbf48159a60 0x7fbf480c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.403+0000 7fbf46d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbf48159a60 0x7fbf480c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59410/0 (socket says 172.21.15.55:59410) 2023-10-06T03:33:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.403+0000 7fbf46d9d700 1 -- 172.21.15.55:0/1495379187 learned_addr learned my addr 172.21.15.55:0/1495379187 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.403+0000 7fbf4759e700 1 --2- 172.21.15.55:0/1495379187 >> v2:172.21.15.163:3300/0 conn(0x7fbf480cb230 0x7fbf480cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.403+0000 7fbf46d9d700 1 -- 172.21.15.55:0/1495379187 >> v2:172.21.15.163:3300/0 conn(0x7fbf480cb230 msgr2=0x7fbf480cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.403+0000 7fbf46d9d700 1 --2- 172.21.15.55:0/1495379187 >> v2:172.21.15.163:3300/0 conn(0x7fbf480cb230 0x7fbf480cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.403+0000 7fbf46d9d700 1 -- 172.21.15.55:0/1495379187 >> v1:172.21.15.163:6789/0 conn(0x7fbf480ccc00 legacy=0x7fbf48069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.403+0000 7fbf46d9d700 1 -- 172.21.15.55:0/1495379187 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf480c9f30 con 0x7fbf48159a60 2023-10-06T03:33:31.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.403+0000 7fbf4759e700 1 --2- 172.21.15.55:0/1495379187 >> v2:172.21.15.163:3300/0 conn(0x7fbf480cb230 0x7fbf480cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.404+0000 7fbf46d9d700 1 --2- 172.21.15.55:0/1495379187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf48159a60 0x7fbf480c9770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf480cc410 tx=0x7fbf38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab82490b3f18472 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.404+0000 7fbf45d9b700 1 -- 172.21.15.55:0/1495379187 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800b8f0 con 0x7fbf48159a60 2023-10-06T03:33:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.404+0000 7fbf45d9b700 1 -- 172.21.15.55:0/1495379187 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3800ba50 con 0x7fbf48159a60 2023-10-06T03:33:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.404+0000 7fbf45d9b700 1 -- 172.21.15.55:0/1495379187 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3800bdf0 con 0x7fbf48159a60 2023-10-06T03:33:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.404+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1495379187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf48159a60 msgr2=0x7fbf480c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.404+0000 7fbf4c9fe700 1 --2- 172.21.15.55:0/1495379187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf48159a60 0x7fbf480c9770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf480cc410 tx=0x7fbf38009250 comp rx=0 tx=0).stop 2023-10-06T03:33:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.404+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1495379187 shutdown_connections 2023-10-06T03:33:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.404+0000 7fbf4c9fe700 1 --2- 172.21.15.55:0/1495379187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf48159a60 0x7fbf480c9770 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.404+0000 7fbf4c9fe700 1 --2- 172.21.15.55:0/1495379187 >> v2:172.21.15.163:3300/0 conn(0x7fbf480cb230 0x7fbf480cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.405+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1495379187 >> 172.21.15.55:0/1495379187 conn(0x7fbf4806baa0 msgr2=0x7fbf4806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:31.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.405+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1495379187 shutdown_connections 2023-10-06T03:33:31.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.405+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1495379187 wait complete. 2023-10-06T03:33:31.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.406+0000 7fbf4c9fe700 1 Processor -- start 2023-10-06T03:33:31.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.406+0000 7fbf4c9fe700 1 -- start start 2023-10-06T03:33:31.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.407+0000 7fbf4c9fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf480cb230 0x7fbf4816a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:31.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.407+0000 7fbf4c9fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf480ccc00 0x7fbf4815b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:31.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.407+0000 7fbf4c9fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf48159a60 0x7fbf48154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:31.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.407+0000 7fbf4c9fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf480b29a0 con 0x7fbf48159a60 2023-10-06T03:33:31.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.407+0000 7fbf4c9fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf480b2c80 con 0x7fbf480ccc00 2023-10-06T03:33:31.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.407+0000 7fbf4c9fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf480b2f60 con 0x7fbf480cb230 2023-10-06T03:33:31.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.407+0000 7fbf4759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf48159a60 0x7fbf48154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:31.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.407+0000 7fbf4759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf48159a60 0x7fbf48154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59412/0 (socket says 172.21.15.55:59412) 2023-10-06T03:33:31.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.407+0000 7fbf4759e700 1 -- 172.21.15.55:0/1757516368 learned_addr learned my addr 172.21.15.55:0/1757516368 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:31.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.408+0000 7fbf46d9d700 1 --2- 172.21.15.55:0/1757516368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf480cb230 0x7fbf4816a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:31.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.408+0000 7fbf4659c700 1 --2- 172.21.15.55:0/1757516368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf480ccc00 0x7fbf4815b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:31.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.408+0000 7fbf4759e700 1 -- 172.21.15.55:0/1757516368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf480cb230 msgr2=0x7fbf4816a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:31.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.408+0000 7fbf4759e700 1 --2- 172.21.15.55:0/1757516368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf480cb230 0x7fbf4816a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.408+0000 7fbf4759e700 1 -- 172.21.15.55:0/1757516368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf480ccc00 msgr2=0x7fbf4815b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:31.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.408+0000 7fbf4759e700 1 --2- 172.21.15.55:0/1757516368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf480ccc00 0x7fbf4815b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.408+0000 7fbf4759e700 1 -- 172.21.15.55:0/1757516368 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf380094f0 con 0x7fbf48159a60 2023-10-06T03:33:31.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.408+0000 7fbf4759e700 1 --2- 172.21.15.55:0/1757516368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf48159a60 0x7fbf48154d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000d180 tx=0x7fbf40009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:31.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.408+0000 7fbf2effd700 1 -- 172.21.15.55:0/1757516368 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4000dcc0 con 0x7fbf48159a60 2023-10-06T03:33:31.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.408+0000 7fbf2effd700 1 -- 172.21.15.55:0/1757516368 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4000f040 con 0x7fbf48159a60 2023-10-06T03:33:31.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.408+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1757516368 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf480b3240 con 0x7fbf48159a60 2023-10-06T03:33:31.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.408+0000 7fbf2effd700 1 -- 172.21.15.55:0/1757516368 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40015690 con 0x7fbf48159a60 2023-10-06T03:33:31.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.409+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1757516368 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4816ae10 con 0x7fbf48159a60 2023-10-06T03:33:31.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.409+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1757516368 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf480b3740 con 0x7fbf48159a60 2023-10-06T03:33:31.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.410+0000 7fbf2effd700 1 -- 172.21.15.55:0/1757516368 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbf4001e3b0 con 0x7fbf48159a60 2023-10-06T03:33:31.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.411+0000 7fbf2effd700 1 --2- 172.21.15.55:0/1757516368 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbf34066fb0 0x7fbf34069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:31.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.411+0000 7fbf2effd700 1 -- 172.21.15.55:0/1757516368 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbf40084bf0 con 0x7fbf48159a60 2023-10-06T03:33:31.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.412+0000 7fbf46d9d700 1 --2- 172.21.15.55:0/1757516368 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbf34066fb0 0x7fbf34069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:31.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.412+0000 7fbf46d9d700 1 --2- 172.21.15.55:0/1757516368 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbf34066fb0 0x7fbf34069460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbf480cc410 tx=0x7fbf38017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:31.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.415+0000 7fbf2effd700 1 -- 172.21.15.55:0/1757516368 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf40051c60 con 0x7fbf48159a60 2023-10-06T03:33:31.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.618+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1757516368 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbf480ca470 con 0x7fbf48159a60 2023-10-06T03:33:31.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.618+0000 7fbf2effd700 1 -- 172.21.15.55:0/1757516368 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbf4001a0a0 con 0x7fbf48159a60 2023-10-06T03:33:31.620 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T03:33:31.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.621+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1757516368 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbf34066fb0 msgr2=0x7fbf34069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:31.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.621+0000 7fbf4c9fe700 1 --2- 172.21.15.55:0/1757516368 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbf34066fb0 0x7fbf34069460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbf480cc410 tx=0x7fbf38017000 comp rx=0 tx=0).stop 2023-10-06T03:33:31.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.621+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1757516368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf48159a60 msgr2=0x7fbf48154d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:31.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.621+0000 7fbf4c9fe700 1 --2- 172.21.15.55:0/1757516368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf48159a60 0x7fbf48154d10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000d180 tx=0x7fbf40009a50 comp rx=0 tx=0).stop 2023-10-06T03:33:31.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.621+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1757516368 shutdown_connections 2023-10-06T03:33:31.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.621+0000 7fbf4c9fe700 1 --2- 172.21.15.55:0/1757516368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbf48159a60 0x7fbf48154d10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.621+0000 7fbf4c9fe700 1 --2- 172.21.15.55:0/1757516368 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbf34066fb0 0x7fbf34069460 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.621+0000 7fbf4c9fe700 1 --2- 172.21.15.55:0/1757516368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbf480cb230 0x7fbf4816a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.621+0000 7fbf4c9fe700 1 --2- 172.21.15.55:0/1757516368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbf480ccc00 0x7fbf4815b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.621+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1757516368 >> 172.21.15.55:0/1757516368 conn(0x7fbf4806baa0 msgr2=0x7fbf481583a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:31.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.621+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1757516368 shutdown_connections 2023-10-06T03:33:31.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.621+0000 7fbf4c9fe700 1 -- 172.21.15.55:0/1757516368 wait complete. 2023-10-06T03:33:31.640 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:33:31.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.824+0000 7fb4d159d700 1 Processor -- start 2023-10-06T03:33:31.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.824+0000 7fb4d159d700 1 -- start start 2023-10-06T03:33:31.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.826+0000 7fb4d159d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb4cc157270 0x7fb4cc068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.826+0000 7fb4d159d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4cc0688d0 0x7fb4cc0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.826+0000 7fb4d159d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4cc15a980 con 0x7fb4cc0b0a00 2023-10-06T03:33:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.826+0000 7fb4d159d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4cc15e310 con 0x7fb4cc157270 2023-10-06T03:33:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.826+0000 7fb4d159d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc068d20 con 0x7fb4cc0688d0 2023-10-06T03:33:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.826+0000 7fb4cbfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb4cc157270 0x7fb4cc068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.826+0000 7fb4d0d9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb4cc0b0a00 0x7fb4cc06a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44058/0 (socket says 172.21.15.55:44058) 2023-10-06T03:33:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.826+0000 7fb4d0d9c700 1 -- 172.21.15.55:0/298327021 learned_addr learned my addr 172.21.15.55:0/298327021 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.826+0000 7fb4cb7fe700 1 --2- 172.21.15.55:0/298327021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4cc0688d0 0x7fb4cc0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.826+0000 7fb4caffd700 1 -- 172.21.15.55:0/298327021 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153233934 0 0) 0x7fb4cc15a980 con 0x7fb4cc0b0a00 2023-10-06T03:33:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.826+0000 7fb4caffd700 1 -- 172.21.15.55:0/298327021 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4b4003570 con 0x7fb4cc0b0a00 2023-10-06T03:33:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.826+0000 7fb4cbfff700 1 -- 172.21.15.55:0/298327021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4cc0688d0 msgr2=0x7fb4cc0b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.826+0000 7fb4cbfff700 1 --2- 172.21.15.55:0/298327021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4cc0688d0 0x7fb4cc0b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.827+0000 7fb4cbfff700 1 -- 172.21.15.55:0/298327021 >> v1:172.21.15.55:6789/0 conn(0x7fb4cc0b0a00 legacy=0x7fb4cc06a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.827+0000 7fb4cbfff700 1 -- 172.21.15.55:0/298327021 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc15a050 con 0x7fb4cc157270 2023-10-06T03:33:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.827+0000 7fb4cbfff700 1 --2- 172.21.15.55:0/298327021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4cc157270 0x7fb4cc068390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0098e0 tx=0x7fb4bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1c91c732de5c6e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.827+0000 7fb4caffd700 1 -- 172.21.15.55:0/298327021 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc00b640 con 0x7fb4cc157270 2023-10-06T03:33:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.827+0000 7fb4caffd700 1 -- 172.21.15.55:0/298327021 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc00b7a0 con 0x7fb4cc157270 2023-10-06T03:33:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.827+0000 7fb4caffd700 1 -- 172.21.15.55:0/298327021 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc00bb40 con 0x7fb4cc157270 2023-10-06T03:33:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.827+0000 7fb4d159d700 1 -- 172.21.15.55:0/298327021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4cc157270 msgr2=0x7fb4cc068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.828+0000 7fb4d159d700 1 --2- 172.21.15.55:0/298327021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4cc157270 0x7fb4cc068390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0098e0 tx=0x7fb4bc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.828+0000 7fb4d159d700 1 -- 172.21.15.55:0/298327021 shutdown_connections 2023-10-06T03:33:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.828+0000 7fb4d159d700 1 --2- 172.21.15.55:0/298327021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4cc157270 0x7fb4cc068390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.828+0000 7fb4d159d700 1 --2- 172.21.15.55:0/298327021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4cc0688d0 0x7fb4cc0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.828+0000 7fb4d159d700 1 -- 172.21.15.55:0/298327021 >> 172.21.15.55:0/298327021 conn(0x7fb4cc0cd310 msgr2=0x7fb4cc0cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.828+0000 7fb4d159d700 1 -- 172.21.15.55:0/298327021 shutdown_connections 2023-10-06T03:33:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.828+0000 7fb4d159d700 1 -- 172.21.15.55:0/298327021 wait complete. 2023-10-06T03:33:31.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.829+0000 7fb4d159d700 1 Processor -- start 2023-10-06T03:33:31.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.829+0000 7fb4d159d700 1 -- start start 2023-10-06T03:33:31.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.830+0000 7fb4d159d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4cc0688d0 0x7fb4cc16c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:31.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.830+0000 7fb4d159d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4cc0b0a00 0x7fb4cc1511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:31.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.830+0000 7fb4d159d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4cc157270 0x7fb4cc151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:31.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.830+0000 7fb4d159d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc15d8e0 con 0x7fb4cc0b0a00 2023-10-06T03:33:31.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.830+0000 7fb4d159d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc15ccd0 con 0x7fb4cc157270 2023-10-06T03:33:31.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.830+0000 7fb4d159d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4cc15cfb0 con 0x7fb4cc0688d0 2023-10-06T03:33:31.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.830+0000 7fb4cb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4cc0b0a00 0x7fb4cc1511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:31.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.830+0000 7fb4cb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4cc0b0a00 0x7fb4cc1511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59440/0 (socket says 172.21.15.55:59440) 2023-10-06T03:33:31.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.830+0000 7fb4cb7fe700 1 -- 172.21.15.55:0/2146023917 learned_addr learned my addr 172.21.15.55:0/2146023917 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:31.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.831+0000 7fb4cbfff700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4cc0688d0 0x7fb4cc16c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:31.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.831+0000 7fb4d0d9c700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4cc157270 0x7fb4cc151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:31.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.831+0000 7fb4cb7fe700 1 -- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4cc0688d0 msgr2=0x7fb4cc16c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:31.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.831+0000 7fb4cb7fe700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4cc0688d0 0x7fb4cc16c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.831+0000 7fb4cb7fe700 1 -- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4cc157270 msgr2=0x7fb4cc151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:31.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.831+0000 7fb4cb7fe700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4cc157270 0x7fb4cc151700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:31.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.831+0000 7fb4cb7fe700 1 -- 172.21.15.55:0/2146023917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4bc009420 con 0x7fb4cc0b0a00 2023-10-06T03:33:31.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.831+0000 7fb4d0d9c700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4cc157270 0x7fb4cc151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:31.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.831+0000 7fb4cb7fe700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4cc0b0a00 0x7fb4cc1511c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000d890 tx=0x7fb4c0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:31.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.831+0000 7fb4b3fff700 1 -- 172.21.15.55:0/2146023917 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c000b600 con 0x7fb4cc0b0a00 2023-10-06T03:33:31.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.831+0000 7fb4b3fff700 1 -- 172.21.15.55:0/2146023917 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c000b760 con 0x7fb4cc0b0a00 2023-10-06T03:33:31.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.831+0000 7fb4d159d700 1 -- 172.21.15.55:0/2146023917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4cc15d290 con 0x7fb4cc0b0a00 2023-10-06T03:33:31.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.831+0000 7fb4b3fff700 1 -- 172.21.15.55:0/2146023917 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c000bae0 con 0x7fb4cc0b0a00 2023-10-06T03:33:31.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.831+0000 7fb4d159d700 1 -- 172.21.15.55:0/2146023917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4cc16cc20 con 0x7fb4cc0b0a00 2023-10-06T03:33:31.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.832+0000 7fb4d159d700 1 -- 172.21.15.55:0/2146023917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4cc04c3c0 con 0x7fb4cc0b0a00 2023-10-06T03:33:31.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.833+0000 7fb4b3fff700 1 -- 172.21.15.55:0/2146023917 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4c001e460 con 0x7fb4cc0b0a00 2023-10-06T03:33:31.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.834+0000 7fb4b3fff700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb4b8066f30 0x7fb4b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:31.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.834+0000 7fb4b3fff700 1 -- 172.21.15.55:0/2146023917 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb4c0084e10 con 0x7fb4cc0b0a00 2023-10-06T03:33:31.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.835+0000 7fb4cbfff700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb4b8066f30 0x7fb4b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:31.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.835+0000 7fb4cbfff700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb4b8066f30 0x7fb4b80693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0093f0 tx=0x7fb4bc017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:31.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:31.838+0000 7fb4b3fff700 1 -- 172.21.15.55:0/2146023917 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4c0088030 con 0x7fb4cc0b0a00 2023-10-06T03:33:32.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.085+0000 7fb4d159d700 1 -- 172.21.15.55:0/2146023917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4cc15a330 con 0x7fb4cc0b0a00 2023-10-06T03:33:32.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.086+0000 7fb4b3fff700 1 -- 172.21.15.55:0/2146023917 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7fb4c0051e80 con 0x7fb4cc0b0a00 2023-10-06T03:33:32.087 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:32.087 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2677235076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":2677235076},{"type":"v1","addr":"172.21.15.55:6835","nonce":2677235076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1030747394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1030747394},{"type":"v1","addr":"172.21.15.55:6834","nonce":1030747394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2040975046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2040975046},{"type":"v1","addr":"172.21.15.163:6836","nonce":2040975046}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:33:15.383124+0000","modified":"2023-10-06T03:33:18.908835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/1910887369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1910887369},{"type":"v1","addr":"172.21.15.163:6837","nonce":1910887369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:33:32.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.088+0000 7fb4d159d700 1 -- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb4b8066f30 msgr2=0x7fb4b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:32.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.088+0000 7fb4d159d700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb4b8066f30 0x7fb4b80693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0093f0 tx=0x7fb4bc017000 comp rx=0 tx=0).stop 2023-10-06T03:33:32.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.088+0000 7fb4d159d700 1 -- 172.21.15.55:0/2146023917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4cc0b0a00 msgr2=0x7fb4cc1511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:32.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.088+0000 7fb4d159d700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4cc0b0a00 0x7fb4cc1511c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb4c000d890 tx=0x7fb4c0008040 comp rx=0 tx=0).stop 2023-10-06T03:33:32.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.088+0000 7fb4d159d700 1 -- 172.21.15.55:0/2146023917 shutdown_connections 2023-10-06T03:33:32.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.088+0000 7fb4d159d700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4cc157270 0x7fb4cc151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.088+0000 7fb4d159d700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb4b8066f30 0x7fb4b80693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.088+0000 7fb4d159d700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4cc0688d0 0x7fb4cc16c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.088+0000 7fb4d159d700 1 --2- 172.21.15.55:0/2146023917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4cc0b0a00 0x7fb4cc1511c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.088+0000 7fb4d159d700 1 -- 172.21.15.55:0/2146023917 >> 172.21.15.55:0/2146023917 conn(0x7fb4cc0cd310 msgr2=0x7fb4cc06c510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:32.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.088+0000 7fb4d159d700 1 -- 172.21.15.55:0/2146023917 shutdown_connections 2023-10-06T03:33:32.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.088+0000 7fb4d159d700 1 -- 172.21.15.55:0/2146023917 wait complete. 2023-10-06T03:33:32.091 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 8 2023-10-06T03:33:32.107 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:32.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.292+0000 7f76acb9d700 1 Processor -- start 2023-10-06T03:33:32.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.293+0000 7f76acb9d700 1 -- start start 2023-10-06T03:33:32.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.294+0000 7f76acb9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76a81566e0 0x7f76a80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:32.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.294+0000 7f76acb9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a80ca860 con 0x7f76a8156310 2023-10-06T03:33:32.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.294+0000 7f76acb9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a80cab40 con 0x7f76a80cb900 2023-10-06T03:33:32.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.294+0000 7f76acb9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a80cae20 con 0x7f76a81566e0 2023-10-06T03:33:32.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.294+0000 7f76a6ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76a81566e0 0x7f76a80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:32.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.294+0000 7f76a77fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f76a8156310 0x7f76a80c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44062/0 (socket says 172.21.15.55:44062) 2023-10-06T03:33:32.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.294+0000 7f76a6ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76a81566e0 0x7f76a80ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59444/0 (socket says 172.21.15.55:59444) 2023-10-06T03:33:32.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.294+0000 7f76a77fe700 1 -- 172.21.15.55:0/2952190831 learned_addr learned my addr 172.21.15.55:0/2952190831 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:32.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.294+0000 7f76a6ffd700 1 -- 172.21.15.55:0/2952190831 >> v1:172.21.15.163:6789/0 conn(0x7f76a80cb900 legacy=0x7f76a80c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:32.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.295+0000 7f76a6ffd700 1 -- 172.21.15.55:0/2952190831 >> v1:172.21.15.55:6789/0 conn(0x7f76a8156310 legacy=0x7f76a80c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:32.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.295+0000 7f76a6ffd700 1 -- 172.21.15.55:0/2952190831 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a80cb100 con 0x7f76a81566e0 2023-10-06T03:33:32.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.295+0000 7f76a67fc700 1 -- 172.21.15.55:0/2952190831 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845403339 0 0) 0x7f76a80ca860 con 0x7f76a8156310 2023-10-06T03:33:32.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.295+0000 7f76a6ffd700 1 --2- 172.21.15.55:0/2952190831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a81566e0 0x7f76a80ae000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f769c0098e0 tx=0x7f769c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723e76f37ceeba51 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:32.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.295+0000 7f76a67fc700 1 -- 172.21.15.55:0/2952190831 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c009e80 con 0x7f76a81566e0 2023-10-06T03:33:32.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.295+0000 7f76a67fc700 1 -- 172.21.15.55:0/2952190831 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769c00b860 con 0x7f76a81566e0 2023-10-06T03:33:32.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.295+0000 7f76a67fc700 1 -- 172.21.15.55:0/2952190831 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769c00bbe0 con 0x7f76a81566e0 2023-10-06T03:33:32.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.295+0000 7f76acb9d700 1 -- 172.21.15.55:0/2952190831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a81566e0 msgr2=0x7f76a80ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:32.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.295+0000 7f76acb9d700 1 --2- 172.21.15.55:0/2952190831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a81566e0 0x7f76a80ae000 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f769c0098e0 tx=0x7f769c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.296+0000 7f76acb9d700 1 -- 172.21.15.55:0/2952190831 shutdown_connections 2023-10-06T03:33:32.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.296+0000 7f76acb9d700 1 --2- 172.21.15.55:0/2952190831 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a81566e0 0x7f76a80ae000 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.296+0000 7f76acb9d700 1 -- 172.21.15.55:0/2952190831 >> 172.21.15.55:0/2952190831 conn(0x7f76a80cc1f0 msgr2=0x7f76a80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:32.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.296+0000 7f76acb9d700 1 -- 172.21.15.55:0/2952190831 shutdown_connections 2023-10-06T03:33:32.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.296+0000 7f76acb9d700 1 -- 172.21.15.55:0/2952190831 wait complete. 2023-10-06T03:33:32.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.297+0000 7f76acb9d700 1 Processor -- start 2023-10-06T03:33:32.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.297+0000 7f76acb9d700 1 -- start start 2023-10-06T03:33:32.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.298+0000 7f76acb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a80cb900 0x7f76a80b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:32.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.298+0000 7f76acb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76a8156310 0x7f76a816de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:32.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.298+0000 7f76acb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a81566e0 0x7f76a816cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:32.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.298+0000 7f76acb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a816e3c0 con 0x7f76a81566e0 2023-10-06T03:33:32.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.298+0000 7f76acb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a814bca0 con 0x7f76a80cb900 2023-10-06T03:33:32.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.298+0000 7f76acb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a814c7a0 con 0x7f76a8156310 2023-10-06T03:33:32.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.298+0000 7f76a7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a81566e0 0x7f76a816cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:32.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.298+0000 7f76a7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a81566e0 0x7f76a816cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59448/0 (socket says 172.21.15.55:59448) 2023-10-06T03:33:32.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.298+0000 7f76a7fff700 1 -- 172.21.15.55:0/1561906400 learned_addr learned my addr 172.21.15.55:0/1561906400 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:32.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.298+0000 7f76a6ffd700 1 --2- 172.21.15.55:0/1561906400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76a8156310 0x7f76a816de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:32.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.298+0000 7f76a77fe700 1 --2- 172.21.15.55:0/1561906400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a80cb900 0x7f76a80b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:32.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.299+0000 7f76a7fff700 1 -- 172.21.15.55:0/1561906400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76a8156310 msgr2=0x7f76a816de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:32.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.299+0000 7f76a7fff700 1 --2- 172.21.15.55:0/1561906400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76a8156310 0x7f76a816de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.299+0000 7f76a7fff700 1 -- 172.21.15.55:0/1561906400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a80cb900 msgr2=0x7f76a80b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:32.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.299+0000 7f76a7fff700 1 --2- 172.21.15.55:0/1561906400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a80cb900 0x7f76a80b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.299+0000 7f76a7fff700 1 -- 172.21.15.55:0/1561906400 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c009420 con 0x7f76a81566e0 2023-10-06T03:33:32.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.299+0000 7f76a7fff700 1 --2- 172.21.15.55:0/1561906400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a81566e0 0x7f76a816cd20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76a000c840 tx=0x7f76a0001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:32.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.299+0000 7f768f7fe700 1 -- 172.21.15.55:0/1561906400 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0012630 con 0x7f76a81566e0 2023-10-06T03:33:32.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.299+0000 7f768f7fe700 1 -- 172.21.15.55:0/1561906400 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a00127b0 con 0x7f76a81566e0 2023-10-06T03:33:32.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.299+0000 7f768f7fe700 1 -- 172.21.15.55:0/1561906400 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0012b70 con 0x7f76a81566e0 2023-10-06T03:33:32.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.299+0000 7f76acb9d700 1 -- 172.21.15.55:0/1561906400 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a80b1dc0 con 0x7f76a81566e0 2023-10-06T03:33:32.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.299+0000 7f76acb9d700 1 -- 172.21.15.55:0/1561906400 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a816d260 con 0x7f76a81566e0 2023-10-06T03:33:32.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.300+0000 7f76acb9d700 1 -- 172.21.15.55:0/1561906400 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a804c3c0 con 0x7f76a81566e0 2023-10-06T03:33:32.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.301+0000 7f768f7fe700 1 -- 172.21.15.55:0/1561906400 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f76a001a3a0 con 0x7f76a81566e0 2023-10-06T03:33:32.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.302+0000 7f768f7fe700 1 --2- 172.21.15.55:0/1561906400 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7694066f30 0x7f76940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:32.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.302+0000 7f768f7fe700 1 -- 172.21.15.55:0/1561906400 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f76a0084840 con 0x7f76a81566e0 2023-10-06T03:33:32.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.303+0000 7f76a77fe700 1 --2- 172.21.15.55:0/1561906400 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7694066f30 0x7f76940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:32.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.303+0000 7f76a77fe700 1 --2- 172.21.15.55:0/1561906400 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7694066f30 0x7f76940693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7698007fd0 tx=0x7f7698007d10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:32.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.306+0000 7f768f7fe700 1 -- 172.21.15.55:0/1561906400 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f76a00518b0 con 0x7f76a81566e0 2023-10-06T03:33:32.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.507+0000 7f76acb9d700 1 -- 172.21.15.55:0/1561906400 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76a814c8e0 con 0x7f76a81566e0 2023-10-06T03:33:32.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.508+0000 7f768f7fe700 1 -- 172.21.15.55:0/1561906400 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f76a0054ed0 con 0x7f76a81566e0 2023-10-06T03:33:32.511 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:32.511 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:16.470817+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:32.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.510+0000 7f76acb9d700 1 -- 172.21.15.55:0/1561906400 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7694066f30 msgr2=0x7f76940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:32.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.510+0000 7f76acb9d700 1 --2- 172.21.15.55:0/1561906400 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7694066f30 0x7f76940693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7698007fd0 tx=0x7f7698007d10 comp rx=0 tx=0).stop 2023-10-06T03:33:32.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.510+0000 7f76acb9d700 1 -- 172.21.15.55:0/1561906400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a81566e0 msgr2=0x7f76a816cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:32.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.510+0000 7f76acb9d700 1 --2- 172.21.15.55:0/1561906400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a81566e0 0x7f76a816cd20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76a000c840 tx=0x7f76a0001670 comp rx=0 tx=0).stop 2023-10-06T03:33:32.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.510+0000 7f76acb9d700 1 -- 172.21.15.55:0/1561906400 shutdown_connections 2023-10-06T03:33:32.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.510+0000 7f76acb9d700 1 --2- 172.21.15.55:0/1561906400 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76a81566e0 0x7f76a816cd20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.510+0000 7f76acb9d700 1 --2- 172.21.15.55:0/1561906400 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76a80cb900 0x7f76a80b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.510+0000 7f76acb9d700 1 --2- 172.21.15.55:0/1561906400 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7694066f30 0x7f76940693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.510+0000 7f76acb9d700 1 --2- 172.21.15.55:0/1561906400 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76a8156310 0x7f76a816de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.510+0000 7f76acb9d700 1 -- 172.21.15.55:0/1561906400 >> 172.21.15.55:0/1561906400 conn(0x7f76a80cc1f0 msgr2=0x7f76a80c1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:32.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.511+0000 7f76acb9d700 1 -- 172.21.15.55:0/1561906400 shutdown_connections 2023-10-06T03:33:32.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.511+0000 7f76acb9d700 1 -- 172.21.15.55:0/1561906400 wait complete. 2023-10-06T03:33:32.531 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:33:32.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.717+0000 7fe8e8f9d700 1 Processor -- start 2023-10-06T03:33:32.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.717+0000 7fe8e8f9d700 1 -- start start 2023-10-06T03:33:32.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.718+0000 7fe8e8f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe8e4157270 0x7fe8e4068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:32.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.718+0000 7fe8e8f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8e40688d0 0x7fe8e40b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:32.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.718+0000 7fe8e8f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e415a980 con 0x7fe8e40b0a00 2023-10-06T03:33:32.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.718+0000 7fe8e8f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e415e310 con 0x7fe8e40688d0 2023-10-06T03:33:32.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.718+0000 7fe8e8f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e4068d50 con 0x7fe8e4157270 2023-10-06T03:33:32.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.718+0000 7fe8e2ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8e40688d0 0x7fe8e40b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:32.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.718+0000 7fe8e2ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8e40688d0 0x7fe8e40b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59450/0 (socket says 172.21.15.55:59450) 2023-10-06T03:33:32.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.718+0000 7fe8e2ffd700 1 -- 172.21.15.55:0/1288220242 learned_addr learned my addr 172.21.15.55:0/1288220242 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:32.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.718+0000 7fe8e3fff700 1 --1- 172.21.15.55:0/1288220242 >> v1:172.21.15.55:6789/0 conn(0x7fe8e40b0a00 0x7fe8e406a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44074/0 (socket says 172.21.15.55:44074) 2023-10-06T03:33:32.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.719+0000 7fe8e37fe700 1 --2- 172.21.15.55:0/1288220242 >> v2:172.21.15.163:3300/0 conn(0x7fe8e4157270 0x7fe8e4068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:32.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.719+0000 7fe8e2ffd700 1 -- 172.21.15.55:0/1288220242 >> v2:172.21.15.163:3300/0 conn(0x7fe8e4157270 msgr2=0x7fe8e4068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:32.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.719+0000 7fe8e2ffd700 1 --2- 172.21.15.55:0/1288220242 >> v2:172.21.15.163:3300/0 conn(0x7fe8e4157270 0x7fe8e4068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.719+0000 7fe8e2ffd700 1 -- 172.21.15.55:0/1288220242 >> v1:172.21.15.55:6789/0 conn(0x7fe8e40b0a00 legacy=0x7fe8e406a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:32.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.719+0000 7fe8e2ffd700 1 -- 172.21.15.55:0/1288220242 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e415a050 con 0x7fe8e40688d0 2023-10-06T03:33:32.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.719+0000 7fe8e27fc700 1 -- 172.21.15.55:0/1288220242 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614646881 0 0) 0x7fe8e415a980 con 0x7fe8e40b0a00 2023-10-06T03:33:32.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.719+0000 7fe8e2ffd700 1 --2- 172.21.15.55:0/1288220242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8e40688d0 0x7fe8e40b04c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40b3360 tx=0x7fe8d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c0c38eefcdcaf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:32.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.719+0000 7fe8e27fc700 1 -- 172.21.15.55:0/1288220242 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d800b820 con 0x7fe8e40688d0 2023-10-06T03:33:32.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.719+0000 7fe8e27fc700 1 -- 172.21.15.55:0/1288220242 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d800b980 con 0x7fe8e40688d0 2023-10-06T03:33:32.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.719+0000 7fe8e27fc700 1 -- 172.21.15.55:0/1288220242 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d800bd20 con 0x7fe8e40688d0 2023-10-06T03:33:32.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.720+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/1288220242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8e40688d0 msgr2=0x7fe8e40b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:32.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.720+0000 7fe8e8f9d700 1 --2- 172.21.15.55:0/1288220242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8e40688d0 0x7fe8e40b04c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40b3360 tx=0x7fe8d8009250 comp rx=0 tx=0).stop 2023-10-06T03:33:32.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.720+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/1288220242 shutdown_connections 2023-10-06T03:33:32.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.720+0000 7fe8e8f9d700 1 --2- 172.21.15.55:0/1288220242 >> v2:172.21.15.163:3300/0 conn(0x7fe8e4157270 0x7fe8e4068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.720+0000 7fe8e8f9d700 1 --2- 172.21.15.55:0/1288220242 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8e40688d0 0x7fe8e40b04c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.720+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/1288220242 >> 172.21.15.55:0/1288220242 conn(0x7fe8e40cd310 msgr2=0x7fe8e40cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:32.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.720+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/1288220242 shutdown_connections 2023-10-06T03:33:32.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.720+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/1288220242 wait complete. 2023-10-06T03:33:32.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.721+0000 7fe8e8f9d700 1 Processor -- start 2023-10-06T03:33:32.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.722+0000 7fe8e8f9d700 1 -- start start 2023-10-06T03:33:32.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.722+0000 7fe8e8f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8e40688d0 0x7fe8e4141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:32.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.722+0000 7fe8e8f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8e40b0a00 0x7fe8e4153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:32.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.722+0000 7fe8e8f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8e4157270 0x7fe8e41586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:32.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.722+0000 7fe8e8f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e416e5f0 con 0x7fe8e4157270 2023-10-06T03:33:32.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.722+0000 7fe8e8f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e416d9e0 con 0x7fe8e40688d0 2023-10-06T03:33:32.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.722+0000 7fe8e8f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e416dcc0 con 0x7fe8e40b0a00 2023-10-06T03:33:32.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.723+0000 7fe8e3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8e4157270 0x7fe8e41586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:32.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.723+0000 7fe8e3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8e4157270 0x7fe8e41586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:59452/0 (socket says 172.21.15.55:59452) 2023-10-06T03:33:32.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.723+0000 7fe8e3fff700 1 -- 172.21.15.55:0/3132148824 learned_addr learned my addr 172.21.15.55:0/3132148824 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:32.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.723+0000 7fe8e2ffd700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8e40b0a00 0x7fe8e4153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:32.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.723+0000 7fe8e37fe700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8e40688d0 0x7fe8e4141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:32.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.723+0000 7fe8e3fff700 1 -- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8e40b0a00 msgr2=0x7fe8e4153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:32.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.723+0000 7fe8e3fff700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8e40b0a00 0x7fe8e4153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.723+0000 7fe8e3fff700 1 -- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8e40688d0 msgr2=0x7fe8e4141c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:32.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.723+0000 7fe8e3fff700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8e40688d0 0x7fe8e4141c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.723+0000 7fe8e3fff700 1 -- 172.21.15.55:0/3132148824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d80094f0 con 0x7fe8e4157270 2023-10-06T03:33:32.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.723+0000 7fe8e2ffd700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8e40b0a00 0x7fe8e4153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:32.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.723+0000 7fe8e37fe700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8e40688d0 0x7fe8e4141c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:32.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.723+0000 7fe8e3fff700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8e4157270 0x7fe8e41586e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00cd00 tx=0x7fe8dc001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:32.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.723+0000 7fe8cb7fe700 1 -- 172.21.15.55:0/3132148824 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc005df0 con 0x7fe8e4157270 2023-10-06T03:33:32.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.724+0000 7fe8cb7fe700 1 -- 172.21.15.55:0/3132148824 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc00a960 con 0x7fe8e4157270 2023-10-06T03:33:32.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.724+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/3132148824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e416dfa0 con 0x7fe8e4157270 2023-10-06T03:33:32.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.724+0000 7fe8cb7fe700 1 -- 172.21.15.55:0/3132148824 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00ad20 con 0x7fe8e4157270 2023-10-06T03:33:32.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.724+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/3132148824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e4152ab0 con 0x7fe8e4157270 2023-10-06T03:33:32.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.725+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/3132148824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e404c3c0 con 0x7fe8e4157270 2023-10-06T03:33:32.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.726+0000 7fe8cb7fe700 1 -- 172.21.15.55:0/3132148824 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8dc007300 con 0x7fe8e4157270 2023-10-06T03:33:32.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.726+0000 7fe8cb7fe700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe8d0066f30 0x7fe8d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:32.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.726+0000 7fe8cb7fe700 1 -- 172.21.15.55:0/3132148824 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe8dc084880 con 0x7fe8e4157270 2023-10-06T03:33:32.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.727+0000 7fe8e37fe700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe8d0066f30 0x7fe8d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:32.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.727+0000 7fe8e37fe700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe8d0066f30 0x7fe8d00693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009990 tx=0x7fe8d4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:32.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.730+0000 7fe8cb7fe700 1 -- 172.21.15.55:0/3132148824 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8dc051a10 con 0x7fe8e4157270 2023-10-06T03:33:32.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.980+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/3132148824 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8e416e730 con 0x7fe8e4157270 2023-10-06T03:33:32.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.981+0000 7fe8cb7fe700 1 -- 172.21.15.55:0/3132148824 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7fe8e416e730 con 0x7fe8e4157270 2023-10-06T03:33:32.983 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:32.983 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2677235076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":2677235076},{"type":"v1","addr":"172.21.15.55:6835","nonce":2677235076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1030747394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1030747394},{"type":"v1","addr":"172.21.15.55:6834","nonce":1030747394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2040975046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2040975046},{"type":"v1","addr":"172.21.15.163:6836","nonce":2040975046}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:33:15.383124+0000","modified":"2023-10-06T03:33:18.908835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/1910887369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1910887369},{"type":"v1","addr":"172.21.15.163:6837","nonce":1910887369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:33:32.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.983+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe8d0066f30 msgr2=0x7fe8d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:32.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.983+0000 7fe8e8f9d700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe8d0066f30 0x7fe8d00693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009990 tx=0x7fe8d4008040 comp rx=0 tx=0).stop 2023-10-06T03:33:32.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.983+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/3132148824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8e4157270 msgr2=0x7fe8e41586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:32.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.983+0000 7fe8e8f9d700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8e4157270 0x7fe8e41586e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00cd00 tx=0x7fe8dc001ac0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.983+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/3132148824 shutdown_connections 2023-10-06T03:33:32.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.983+0000 7fe8e8f9d700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8e4157270 0x7fe8e41586e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.983+0000 7fe8e8f9d700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8e40688d0 0x7fe8e4141c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.983+0000 7fe8e8f9d700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe8d0066f30 0x7fe8d00693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.983+0000 7fe8e8f9d700 1 --2- 172.21.15.55:0/3132148824 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8e40b0a00 0x7fe8e4153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:32.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.983+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/3132148824 >> 172.21.15.55:0/3132148824 conn(0x7fe8e40cd310 msgr2=0x7fe8e4150750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:32.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.983+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/3132148824 shutdown_connections 2023-10-06T03:33:32.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:32.983+0000 7fe8e8f9d700 1 -- 172.21.15.55:0/3132148824 wait complete. 2023-10-06T03:33:32.987 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 8 2023-10-06T03:33:33.405 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:33:15.383124+0000', 'modified': '2023-10-06T03:33:18.908835+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6837/1910887369', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 1910887369}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 1910887369}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:33:33.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:33:33.406 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T03:33:33.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.594+0000 7f0f95dff700 1 Processor -- start 2023-10-06T03:33:33.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.594+0000 7f0f95dff700 1 -- start start 2023-10-06T03:33:33.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.596+0000 7f0f95dff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0f900b1420 0x7f0f900ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.596+0000 7f0f95dff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f900cc300 con 0x7f0f900cbf30 2023-10-06T03:33:33.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.596+0000 7f0f95dff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f900ad370 con 0x7f0f900ccf90 2023-10-06T03:33:33.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.596+0000 7f0f95dff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f90156bf0 con 0x7f0f900b1420 2023-10-06T03:33:33.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.596+0000 7f0f8f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0f900cbf30 0x7f0f900b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38086/0 (socket says 172.21.15.55:38086) 2023-10-06T03:33:33.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.597+0000 7f0f8f7fe700 1 -- 172.21.15.55:0/2104280568 learned_addr learned my addr 172.21.15.55:0/2104280568 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:33.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.597+0000 7f0f8effd700 1 --2- 172.21.15.55:0/2104280568 >> v2:172.21.15.163:3300/0 conn(0x7f0f900b1420 0x7f0f900ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.597+0000 7f0f8e7fc700 1 -- 172.21.15.55:0/2104280568 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077524388 0 0) 0x7f0f900cc300 con 0x7f0f900cbf30 2023-10-06T03:33:33.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.597+0000 7f0f8e7fc700 1 -- 172.21.15.55:0/2104280568 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f78003570 con 0x7f0f900cbf30 2023-10-06T03:33:33.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.597+0000 7f0f8effd700 1 -- 172.21.15.55:0/2104280568 >> v1:172.21.15.163:6789/0 conn(0x7f0f900ccf90 legacy=0x7f0f900c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:33.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.597+0000 7f0f8e7fc700 1 -- 172.21.15.55:0/2104280568 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652191537 0 0) 0x7f0f900ad370 con 0x7f0f900ccf90 2023-10-06T03:33:33.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.597+0000 7f0f8effd700 1 -- 172.21.15.55:0/2104280568 >> v1:172.21.15.55:6789/0 conn(0x7f0f900cbf30 legacy=0x7f0f900b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:33.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.597+0000 7f0f8effd700 1 -- 172.21.15.55:0/2104280568 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f900ad650 con 0x7f0f900b1420 2023-10-06T03:33:33.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.598+0000 7f0f8effd700 1 --2- 172.21.15.55:0/2104280568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f900b1420 0x7f0f900ae410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0f840098e0 tx=0x7f0f840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ded71e35fea7d2ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:33.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.598+0000 7f0f8e7fc700 1 -- 172.21.15.55:0/2104280568 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400b640 con 0x7f0f900b1420 2023-10-06T03:33:33.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.598+0000 7f0f8e7fc700 1 -- 172.21.15.55:0/2104280568 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8400b7a0 con 0x7f0f900b1420 2023-10-06T03:33:33.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.598+0000 7f0f8e7fc700 1 -- 172.21.15.55:0/2104280568 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8400bb40 con 0x7f0f900b1420 2023-10-06T03:33:33.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.598+0000 7f0f95dff700 1 -- 172.21.15.55:0/2104280568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f900b1420 msgr2=0x7f0f900ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:33.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.598+0000 7f0f95dff700 1 --2- 172.21.15.55:0/2104280568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f900b1420 0x7f0f900ae410 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0f840098e0 tx=0x7f0f840091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.598+0000 7f0f95dff700 1 -- 172.21.15.55:0/2104280568 shutdown_connections 2023-10-06T03:33:33.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.598+0000 7f0f95dff700 1 --2- 172.21.15.55:0/2104280568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f900b1420 0x7f0f900ae410 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.599+0000 7f0f95dff700 1 -- 172.21.15.55:0/2104280568 >> 172.21.15.55:0/2104280568 conn(0x7f0f9006a320 msgr2=0x7f0f900cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:33.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.599+0000 7f0f95dff700 1 -- 172.21.15.55:0/2104280568 shutdown_connections 2023-10-06T03:33:33.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.599+0000 7f0f95dff700 1 -- 172.21.15.55:0/2104280568 wait complete. 2023-10-06T03:33:33.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.600+0000 7f0f95dff700 1 Processor -- start 2023-10-06T03:33:33.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.600+0000 7f0f95dff700 1 -- start start 2023-10-06T03:33:33.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.601+0000 7f0f95dff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f900b1420 0x7f0f900a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.601+0000 7f0f95dff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f900cbf30 0x7f0f90169fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.601+0000 7f0f95dff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f900ccf90 0x7f0f9016a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.601+0000 7f0f95dff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900a6780 con 0x7f0f900ccf90 2023-10-06T03:33:33.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.601+0000 7f0f95dff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900a6a60 con 0x7f0f900cbf30 2023-10-06T03:33:33.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.601+0000 7f0f95dff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900a6d40 con 0x7f0f900b1420 2023-10-06T03:33:33.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.601+0000 7f0f8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f900ccf90 0x7f0f9016a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.601+0000 7f0f8f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f900b1420 0x7f0f900a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.601+0000 7f0f8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f900ccf90 0x7f0f9016a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51700/0 (socket says 172.21.15.55:51700) 2023-10-06T03:33:33.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.601+0000 7f0f8ffff700 1 -- 172.21.15.55:0/4281895606 learned_addr learned my addr 172.21.15.55:0/4281895606 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:33.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.601+0000 7f0f8effd700 1 --2- 172.21.15.55:0/4281895606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f900cbf30 0x7f0f90169fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.602+0000 7f0f8f7fe700 1 -- 172.21.15.55:0/4281895606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f900cbf30 msgr2=0x7f0f90169fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:33.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.602+0000 7f0f8f7fe700 1 --2- 172.21.15.55:0/4281895606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f900cbf30 0x7f0f90169fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.602+0000 7f0f8f7fe700 1 -- 172.21.15.55:0/4281895606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f900ccf90 msgr2=0x7f0f9016a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:33.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.602+0000 7f0f8f7fe700 1 --2- 172.21.15.55:0/4281895606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f900ccf90 0x7f0f9016a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.602+0000 7f0f8f7fe700 1 -- 172.21.15.55:0/4281895606 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f84009420 con 0x7f0f900b1420 2023-10-06T03:33:33.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.602+0000 7f0f8ffff700 1 --2- 172.21.15.55:0/4281895606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f900ccf90 0x7f0f9016a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:33.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.602+0000 7f0f8f7fe700 1 --2- 172.21.15.55:0/4281895606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f900b1420 0x7f0f900a5e10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f800058d0 tx=0x7f0f80007d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:33.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.602+0000 7f0f777fe700 1 -- 172.21.15.55:0/4281895606 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80005b30 con 0x7f0f900b1420 2023-10-06T03:33:33.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.602+0000 7f0f777fe700 1 -- 172.21.15.55:0/4281895606 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f80005cd0 con 0x7f0f900b1420 2023-10-06T03:33:33.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.602+0000 7f0f95dff700 1 -- 172.21.15.55:0/4281895606 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f900c6850 con 0x7f0f900b1420 2023-10-06T03:33:33.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.603+0000 7f0f777fe700 1 -- 172.21.15.55:0/4281895606 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80007320 con 0x7f0f900b1420 2023-10-06T03:33:33.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.603+0000 7f0f95dff700 1 -- 172.21.15.55:0/4281895606 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f900a86b0 con 0x7f0f900b1420 2023-10-06T03:33:33.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.605+0000 7f0f777fe700 1 -- 172.21.15.55:0/4281895606 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0f80007690 con 0x7f0f900b1420 2023-10-06T03:33:33.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.605+0000 7f0f777fe700 1 --2- 172.21.15.55:0/4281895606 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0f7c066f30 0x7f0f7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.606+0000 7f0f777fe700 1 -- 172.21.15.55:0/4281895606 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0f80083950 con 0x7f0f900b1420 2023-10-06T03:33:33.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.606+0000 7f0f8effd700 1 --2- 172.21.15.55:0/4281895606 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0f7c066f30 0x7f0f7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.607+0000 7f0f8effd700 1 --2- 172.21.15.55:0/4281895606 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0f7c066f30 0x7f0f7c0693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0f900b1820 tx=0x7f0f84017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:33.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.608+0000 7f0fa446db80 1 Processor -- start 2023-10-06T03:33:33.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.608+0000 7f0fa446db80 1 -- start start 2023-10-06T03:33:33.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.609+0000 7f0fa446db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f431c840 0x5608f431cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.609+0000 7f0fa446db80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431dcc0 0x5608f436c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.609+0000 7f0fa446db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f436cc00 0x5608f436d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.609+0000 7f0fa446db80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608f4273a70 con 0x5608f436cc00 2023-10-06T03:33:33.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.609+0000 7f0fa446db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608f4271270 con 0x5608f431c840 2023-10-06T03:33:33.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.609+0000 7f0fa446db80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608f426d060 con 0x5608f431dcc0 2023-10-06T03:33:33.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.609+0000 7f0f8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f436cc00 0x5608f436d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.609+0000 7f0f8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f436cc00 0x5608f436d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51704/0 (socket says 172.21.15.55:51704) 2023-10-06T03:33:33.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.609+0000 7f0f8ffff700 1 -- 172.21.15.55:0/2434015250 learned_addr learned my addr 172.21.15.55:0/2434015250 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:33.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.610+0000 7f0f8ffff700 1 -- 172.21.15.55:0/2434015250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431dcc0 msgr2=0x5608f436c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:33.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.610+0000 7f0f8ffff700 1 --2- 172.21.15.55:0/2434015250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431dcc0 0x5608f436c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.610+0000 7f0f8f7fe700 1 --2- 172.21.15.55:0/2434015250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f431c840 0x5608f431cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.610+0000 7f0f8ffff700 1 -- 172.21.15.55:0/2434015250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f431c840 msgr2=0x5608f431cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:33.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.610+0000 7f0f8ffff700 1 --2- 172.21.15.55:0/2434015250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f431c840 0x5608f431cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.610+0000 7f0f8ffff700 1 -- 172.21.15.55:0/2434015250 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608f4107270 con 0x5608f436cc00 2023-10-06T03:33:33.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.610+0000 7f0f8ffff700 1 --2- 172.21.15.55:0/2434015250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f436cc00 0x5608f436d050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800f440 tx=0x7f0f88009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64483c4173b89e8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:33.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.610+0000 7f0f74ff9700 1 -- 172.21.15.55:0/2434015250 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8800fe70 con 0x5608f436cc00 2023-10-06T03:33:33.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.610+0000 7f0f74ff9700 1 -- 172.21.15.55:0/2434015250 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f88013600 con 0x5608f436cc00 2023-10-06T03:33:33.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.610+0000 7f0f74ff9700 1 -- 172.21.15.55:0/2434015250 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f880139c0 con 0x5608f436cc00 2023-10-06T03:33:33.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.611+0000 7f0fa446db80 1 -- 172.21.15.55:0/2434015250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f436cc00 msgr2=0x5608f436d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:33.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.611+0000 7f0fa446db80 1 --2- 172.21.15.55:0/2434015250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f436cc00 0x5608f436d050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800f440 tx=0x7f0f88009040 comp rx=0 tx=0).stop 2023-10-06T03:33:33.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.611+0000 7f0fa446db80 1 -- 172.21.15.55:0/2434015250 shutdown_connections 2023-10-06T03:33:33.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.611+0000 7f0fa446db80 1 --2- 172.21.15.55:0/2434015250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f436cc00 0x5608f436d050 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.611+0000 7f0fa446db80 1 --2- 172.21.15.55:0/2434015250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f431c840 0x5608f431cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.611+0000 7f0fa446db80 1 --2- 172.21.15.55:0/2434015250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431dcc0 0x5608f436c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.611+0000 7f0fa446db80 1 -- 172.21.15.55:0/2434015250 >> 172.21.15.55:0/2434015250 conn(0x5608f431e160 msgr2=0x5608f431e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:33.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.611+0000 7f0fa446db80 1 -- 172.21.15.55:0/2434015250 shutdown_connections 2023-10-06T03:33:33.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.611+0000 7f0fa446db80 1 -- 172.21.15.55:0/2434015250 wait complete. 2023-10-06T03:33:33.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.612+0000 7f0fa446db80 1 -- start start 2023-10-06T03:33:33.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.612+0000 7f0fa446db80 1 Processor -- start 2023-10-06T03:33:33.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.613+0000 7f0fa446db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f431c840 0x5608f4376d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.613+0000 7f0fa446db80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431dcc0 0x5608f4377290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.613+0000 7f0fa446db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f436cc00 0x5608f43632e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.613+0000 7f0fa446db80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608f4218890 con 0x5608f436cc00 2023-10-06T03:33:33.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.613+0000 7f0fa446db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608f4233ea0 con 0x5608f431c840 2023-10-06T03:33:33.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.613+0000 7f0fa446db80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608f40efe30 con 0x5608f431dcc0 2023-10-06T03:33:33.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.613+0000 7f0fa446db80 10 client.0 _ll_register_callbacks cb 0x5608f42dfe70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:33:33.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.613+0000 7f0f8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f436cc00 0x5608f43632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.613+0000 7f0f8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f436cc00 0x5608f43632e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51708/0 (socket says 172.21.15.55:51708) 2023-10-06T03:33:33.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.613+0000 7f0f8ffff700 1 -- 172.21.15.55:0/3538668934 learned_addr learned my addr 172.21.15.55:0/3538668934 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:33.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.613+0000 7f0f8f7fe700 1 --2- 172.21.15.55:0/3538668934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f431c840 0x5608f4376d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.613+0000 7f0f8effd700 1 --2- 172.21.15.55:0/3538668934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431dcc0 0x5608f4377290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.614+0000 7f0f8ffff700 1 -- 172.21.15.55:0/3538668934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431dcc0 msgr2=0x5608f4377290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:33.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.614+0000 7f0f8ffff700 1 --2- 172.21.15.55:0/3538668934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431dcc0 0x5608f4377290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.614+0000 7f0f8ffff700 1 -- 172.21.15.55:0/3538668934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f431c840 msgr2=0x5608f4376d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:33.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.614+0000 7f0f8ffff700 1 --2- 172.21.15.55:0/3538668934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f431c840 0x5608f4376d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.614+0000 7f0f8ffff700 1 -- 172.21.15.55:0/3538668934 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f880020d0 con 0x5608f436cc00 2023-10-06T03:33:33.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.614+0000 7f0f8ffff700 1 --2- 172.21.15.55:0/3538668934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f436cc00 0x5608f43632e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0f88002010 tx=0x7f0f88019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff6d8648e17a53b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:33.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.614+0000 7f0f6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:33:33.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.614+0000 7f0f6dffb700 1 -- 172.21.15.55:0/3538668934 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88022070 con 0x5608f436cc00 2023-10-06T03:33:33.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.614+0000 7f0fa446db80 1 -- 172.21.15.55:0/3538668934 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608f4311940 con 0x5608f436cc00 2023-10-06T03:33:33.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.614+0000 7f0f6dffb700 1 -- 172.21.15.55:0/3538668934 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0f8801c420 con 0x5608f436cc00 2023-10-06T03:33:33.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.614+0000 7f0f6dffb700 1 -- 172.21.15.55:0/3538668934 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f8801c7f0 con 0x5608f436cc00 2023-10-06T03:33:33.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.614+0000 7f0fa446db80 10 client.4692 fetch_fsmap learned FSMap version 8 2023-10-06T03:33:33.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.614+0000 7f0fa446db80 1 -- 172.21.15.55:0/3538668934 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5608f40a9fe0 con 0x5608f436cc00 2023-10-06T03:33:33.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.615+0000 7f0f6dffb700 1 -- 172.21.15.55:0/3538668934 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5608f4311940 con 0x5608f436cc00 2023-10-06T03:33:33.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.615+0000 7f0f6dffb700 1 -- 172.21.15.55:0/3538668934 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0f88012070 con 0x5608f436cc00 2023-10-06T03:33:33.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.615+0000 7f0fa446db80 10 client.4692 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T03:33:33.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.615+0000 7f0fa446db80 10 client.4692 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T03:33:33.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.615+0000 7f0fa446db80 20 client.4692 populate_metadata read hostname 'smithi055' 2023-10-06T03:33:33.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.615+0000 7f0fa446db80 1 --2- 172.21.15.55:0/3538668934 >> [v2:172.21.15.163:6835/1910887369,v1:172.21.15.163:6837/1910887369] conn(0x5608f4363ca0 0x5608f435cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.615+0000 7f0fa446db80 4 client.4692 mds_command: new command op to 4288 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:33:33.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.615+0000 7f0fa446db80 1 -- 172.21.15.55:0/3538668934 --> [v2:172.21.15.163:6835/1910887369,v1:172.21.15.163:6837/1910887369] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608f42ee870 con 0x5608f4363ca0 2023-10-06T03:33:33.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.615+0000 7f0f8f7fe700 1 --2- 172.21.15.55:0/3538668934 >> [v2:172.21.15.163:6835/1910887369,v1:172.21.15.163:6837/1910887369] conn(0x5608f4363ca0 0x5608f435cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.616+0000 7f0f8f7fe700 1 --2- 172.21.15.55:0/3538668934 >> [v2:172.21.15.163:6835/1910887369,v1:172.21.15.163:6837/1910887369] conn(0x5608f4363ca0 0x5608f435cb60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c7799b3ee3bf612 server_cookie=d7ede34e26184e2c in_seq=0 out_seq=0 2023-10-06T03:33:33.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.616+0000 7f0f6dffb700 10 client.4692 ms_handle_connect on v2:172.21.15.163:6835/1910887369 2023-10-06T03:33:33.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.616+0000 7f0f6dffb700 1 -- 172.21.15.55:0/3538668934 <== mds.0 v2:172.21.15.163:6835/1910887369 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5608f42ee870 con 0x5608f4363ca0 2023-10-06T03:33:33.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.616+0000 7f0f6dffb700 10 client.4692 handle_command_reply: tid=0 2023-10-06T03:33:33.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.617+0000 7f0fa446db80 1 client.4692 shutdown 2023-10-06T03:33:33.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.617+0000 7f0fa446db80 1 -- 172.21.15.55:0/3538668934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f436cc00 msgr2=0x5608f43632e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:33.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.617+0000 7f0fa446db80 1 --2- 172.21.15.55:0/3538668934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f436cc00 0x5608f43632e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0f88002010 tx=0x7f0f88019000 comp rx=0 tx=0).stop 2023-10-06T03:33:33.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.617+0000 7f0fa446db80 1 -- 172.21.15.55:0/3538668934 shutdown_connections 2023-10-06T03:33:33.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.617+0000 7f0fa446db80 1 --2- 172.21.15.55:0/3538668934 >> [v2:172.21.15.163:6835/1910887369,v1:172.21.15.163:6837/1910887369] conn(0x5608f4363ca0 0x5608f435cb60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.617+0000 7f0fa446db80 1 --2- 172.21.15.55:0/3538668934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f436cc00 0x5608f43632e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.617+0000 7f0fa446db80 1 --2- 172.21.15.55:0/3538668934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f431c840 0x5608f4376d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.617+0000 7f0fa446db80 1 --2- 172.21.15.55:0/3538668934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431dcc0 0x5608f4377290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.617+0000 7f0f6dffb700 0 client.4692 ms_handle_reset on v2:172.21.15.163:6835/1910887369 2023-10-06T03:33:33.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.617+0000 7f0fa446db80 1 -- 172.21.15.55:0/3538668934 >> 172.21.15.55:0/3538668934 conn(0x5608f431e160 msgr2=0x5608f4360360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:33.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.617+0000 7f0fa446db80 1 -- 172.21.15.55:0/3538668934 shutdown_connections 2023-10-06T03:33:33.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.617+0000 7f0fa446db80 1 -- 172.21.15.55:0/3538668934 wait complete. 2023-10-06T03:33:33.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.617+0000 7f0fa446db80 20 client.4692 trim_cache size 0 max 16384 2023-10-06T03:33:33.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.635+0000 7f0fa446db80 1 Processor -- start 2023-10-06T03:33:33.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.635+0000 7f0fa446db80 1 -- start start 2023-10-06T03:33:33.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.636+0000 7f0fa446db80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431c840 0x5608f435cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.636+0000 7f0fa446db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f431dcc0 0x5608f4376d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.636+0000 7f0fa446db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f436cc00 0x5608f4377290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.636+0000 7f0fa446db80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608f4317a10 con 0x5608f431dcc0 2023-10-06T03:33:33.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.636+0000 7f0fa446db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608f436a6d0 con 0x5608f436cc00 2023-10-06T03:33:33.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.636+0000 7f0fa446db80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608f42fc380 con 0x5608f431c840 2023-10-06T03:33:33.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.636+0000 7f0f8effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f431dcc0 0x5608f4376d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.636+0000 7f0f8effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f431dcc0 0x5608f4376d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51718/0 (socket says 172.21.15.55:51718) 2023-10-06T03:33:33.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.636+0000 7f0f8effd700 1 -- 172.21.15.55:0/3136553525 learned_addr learned my addr 172.21.15.55:0/3136553525 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:33.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.636+0000 7f0f8f7fe700 1 --2- 172.21.15.55:0/3136553525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431c840 0x5608f435cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.637+0000 7f0f8ffff700 1 --2- 172.21.15.55:0/3136553525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f436cc00 0x5608f4377290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.637+0000 7f0f8effd700 1 -- 172.21.15.55:0/3136553525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431c840 msgr2=0x5608f435cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:33.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.637+0000 7f0f8effd700 1 --2- 172.21.15.55:0/3136553525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431c840 0x5608f435cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.637+0000 7f0f8effd700 1 -- 172.21.15.55:0/3136553525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f436cc00 msgr2=0x5608f4377290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.637+0000 7f0f8effd700 1 --2- 172.21.15.55:0/3136553525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f436cc00 0x5608f4377290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.637+0000 7f0f8effd700 1 -- 172.21.15.55:0/3136553525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608f423b780 con 0x5608f431dcc0 2023-10-06T03:33:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.637+0000 7f0f8f7fe700 1 --2- 172.21.15.55:0/3136553525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431c840 0x5608f435cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.637+0000 7f0f8ffff700 1 --2- 172.21.15.55:0/3136553525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f436cc00 0x5608f4377290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.637+0000 7f0f8effd700 1 --2- 172.21.15.55:0/3136553525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f431dcc0 0x5608f4376d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0f90064290 tx=0x7f0f84022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d6129ae5676d5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.637+0000 7f0f757fa700 1 -- 172.21.15.55:0/3136553525 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8402c050 con 0x5608f431dcc0 2023-10-06T03:33:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.637+0000 7f0f757fa700 1 -- 172.21.15.55:0/3136553525 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8402d040 con 0x5608f431dcc0 2023-10-06T03:33:33.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.637+0000 7f0f757fa700 1 -- 172.21.15.55:0/3136553525 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8401abb0 con 0x5608f431dcc0 2023-10-06T03:33:33.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.638+0000 7f0fa446db80 1 -- 172.21.15.55:0/3136553525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f431dcc0 msgr2=0x5608f4376d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:33.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.638+0000 7f0fa446db80 1 --2- 172.21.15.55:0/3136553525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f431dcc0 0x5608f4376d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0f90064290 tx=0x7f0f84022000 comp rx=0 tx=0).stop 2023-10-06T03:33:33.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.638+0000 7f0fa446db80 1 -- 172.21.15.55:0/3136553525 shutdown_connections 2023-10-06T03:33:33.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.638+0000 7f0fa446db80 1 --2- 172.21.15.55:0/3136553525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f436cc00 0x5608f4377290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.638+0000 7f0fa446db80 1 --2- 172.21.15.55:0/3136553525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431c840 0x5608f435cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.638+0000 7f0fa446db80 1 --2- 172.21.15.55:0/3136553525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f431dcc0 0x5608f4376d50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.638+0000 7f0fa446db80 1 -- 172.21.15.55:0/3136553525 >> 172.21.15.55:0/3136553525 conn(0x5608f431e160 msgr2=0x5608f431ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:33.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.638+0000 7f0fa446db80 1 -- 172.21.15.55:0/3136553525 shutdown_connections 2023-10-06T03:33:33.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.638+0000 7f0fa446db80 1 -- 172.21.15.55:0/3136553525 wait complete. 2023-10-06T03:33:33.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.638+0000 7f0fa446db80 1 -- start start 2023-10-06T03:33:33.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.639+0000 7f0fa446db80 1 Processor -- start 2023-10-06T03:33:33.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0fa446db80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f431c840 0x5608f4376d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0fa446db80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431dcc0 0x5608f4377290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0fa446db80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f436cc00 0x5608f4382040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0fa446db80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608f4382f90 con 0x5608f431c840 2023-10-06T03:33:33.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0fa446db80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608f4368150 con 0x5608f436cc00 2023-10-06T03:33:33.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0fa446db80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5608f4359c80 con 0x5608f431dcc0 2023-10-06T03:33:33.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0fa446db80 10 client.0 _ll_register_callbacks cb 0x5608f42dfe70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:33:33.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0f8f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f431c840 0x5608f4376d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0f8f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f431c840 0x5608f4376d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51724/0 (socket says 172.21.15.55:51724) 2023-10-06T03:33:33.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0f8f7fe700 1 -- 172.21.15.55:0/2756378993 learned_addr learned my addr 172.21.15.55:0/2756378993 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:33.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0f8effd700 1 --2- 172.21.15.55:0/2756378993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431dcc0 0x5608f4377290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0f8ffff700 1 --2- 172.21.15.55:0/2756378993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f436cc00 0x5608f4382040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0f8f7fe700 1 -- 172.21.15.55:0/2756378993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431dcc0 msgr2=0x5608f4377290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:33.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0f8f7fe700 1 --2- 172.21.15.55:0/2756378993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431dcc0 0x5608f4377290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0f8f7fe700 1 -- 172.21.15.55:0/2756378993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f436cc00 msgr2=0x5608f4382040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:33.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0f8f7fe700 1 --2- 172.21.15.55:0/2756378993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f436cc00 0x5608f4382040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0f8f7fe700 1 -- 172.21.15.55:0/2756378993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f900ad650 con 0x5608f431c840 2023-10-06T03:33:33.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0f8f7fe700 1 --2- 172.21.15.55:0/2756378993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f431c840 0x5608f4376d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000cb20 tx=0x7f0f80081bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e5e500cea73092 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:33.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.640+0000 7f0f6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:33:33.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.641+0000 7f0f6dffb700 1 -- 172.21.15.55:0/2756378993 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8005b620 con 0x5608f431c840 2023-10-06T03:33:33.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.641+0000 7f0f6dffb700 1 -- 172.21.15.55:0/2756378993 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0f80052ad0 con 0x5608f431c840 2023-10-06T03:33:33.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.641+0000 7f0fa446db80 1 -- 172.21.15.55:0/2756378993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f88013600 con 0x5608f431c840 2023-10-06T03:33:33.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.641+0000 7f0f6dffb700 1 -- 172.21.15.55:0/2756378993 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f8007ee30 con 0x5608f431c840 2023-10-06T03:33:33.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.641+0000 7f0fa446db80 10 client.4698 fetch_fsmap learned FSMap version 8 2023-10-06T03:33:33.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.641+0000 7f0fa446db80 1 -- 172.21.15.55:0/2756378993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5608f4222470 con 0x5608f431c840 2023-10-06T03:33:33.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.641+0000 7f0f6dffb700 1 -- 172.21.15.55:0/2756378993 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f88013600 con 0x5608f431c840 2023-10-06T03:33:33.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.641+0000 7f0f6dffb700 1 -- 172.21.15.55:0/2756378993 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0f80053c20 con 0x5608f431c840 2023-10-06T03:33:33.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.641+0000 7f0fa446db80 10 client.4698 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T03:33:33.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.641+0000 7f0fa446db80 10 client.4698 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T03:33:33.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.641+0000 7f0fa446db80 20 client.4698 populate_metadata read hostname 'smithi055' 2023-10-06T03:33:33.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.641+0000 7f0fa446db80 1 --2- 172.21.15.55:0/2756378993 >> [v2:172.21.15.163:6835/1910887369,v1:172.21.15.163:6837/1910887369] conn(0x5608f4382580 0x5608f4366f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:33.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.641+0000 7f0fa446db80 4 client.4698 mds_command: new command op to 4288 tid=0[{"prefix": "status"}] 2023-10-06T03:33:33.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.641+0000 7f0fa446db80 1 -- 172.21.15.55:0/2756378993 --> [v2:172.21.15.163:6835/1910887369,v1:172.21.15.163:6837/1910887369] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5608f420fe00 con 0x5608f4382580 2023-10-06T03:33:33.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.642+0000 7f0f8effd700 1 --2- 172.21.15.55:0/2756378993 >> [v2:172.21.15.163:6835/1910887369,v1:172.21.15.163:6837/1910887369] conn(0x5608f4382580 0x5608f4366f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:33.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.642+0000 7f0f8effd700 1 --2- 172.21.15.55:0/2756378993 >> [v2:172.21.15.163:6835/1910887369,v1:172.21.15.163:6837/1910887369] conn(0x5608f4382580 0x5608f4366f10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21d3e68d363b71c1 server_cookie=113a8ff77299975e in_seq=0 out_seq=0 2023-10-06T03:33:33.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.642+0000 7f0f6dffb700 10 client.4698 ms_handle_connect on v2:172.21.15.163:6835/1910887369 2023-10-06T03:33:33.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.642+0000 7f0f6dffb700 1 -- 172.21.15.55:0/2756378993 <== mds.0 v2:172.21.15.163:6835/1910887369 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5608f420fe00 con 0x5608f4382580 2023-10-06T03:33:33.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.642+0000 7f0f6dffb700 10 client.4698 handle_command_reply: tid=0 2023-10-06T03:33:33.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.642+0000 7f0fa446db80 1 client.4698 shutdown 2023-10-06T03:33:33.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0fa446db80 1 -- 172.21.15.55:0/2756378993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f431c840 msgr2=0x5608f4376d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:33.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0fa446db80 1 --2- 172.21.15.55:0/2756378993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f431c840 0x5608f4376d50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000cb20 tx=0x7f0f80081bd0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0fa446db80 1 -- 172.21.15.55:0/2756378993 shutdown_connections 2023-10-06T03:33:33.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0fa446db80 1 --2- 172.21.15.55:0/2756378993 >> [v2:172.21.15.163:6835/1910887369,v1:172.21.15.163:6837/1910887369] conn(0x5608f4382580 0x5608f4366f10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0fa446db80 1 --2- 172.21.15.55:0/2756378993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5608f436cc00 0x5608f4382040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0fa446db80 1 --2- 172.21.15.55:0/2756378993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5608f431c840 0x5608f4376d50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0f6dffb700 0 client.4698 ms_handle_reset on v2:172.21.15.163:6835/1910887369 2023-10-06T03:33:33.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0fa446db80 1 --2- 172.21.15.55:0/2756378993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5608f431dcc0 0x5608f4377290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0fa446db80 1 -- 172.21.15.55:0/2756378993 >> 172.21.15.55:0/2756378993 conn(0x5608f431e160 msgr2=0x5608f431ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:33.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0fa446db80 1 -- 172.21.15.55:0/2756378993 shutdown_connections 2023-10-06T03:33:33.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0fa446db80 1 -- 172.21.15.55:0/2756378993 wait complete. 2023-10-06T03:33:33.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0fa446db80 20 client.4698 trim_cache size 0 max 16384 2023-10-06T03:33:33.652 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:33:33.652 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "8132c8ba-ee1c-41a2-b781-913925034ab6", 2023-10-06T03:33:33.652 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2023-10-06T03:33:33.652 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 4288, 2023-10-06T03:33:33.652 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2023-10-06T03:33:33.653 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2023-10-06T03:33:33.653 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2023-10-06T03:33:33.653 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 17.774125761000001, 2023-10-06T03:33:33.653 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 8, 2023-10-06T03:33:33.653 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 21, 2023-10-06T03:33:33.653 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 21, 2023-10-06T03:33:33.653 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 23.988746615 2023-10-06T03:33:33.653 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:33:33.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0f95dff700 1 -- 172.21.15.55:0/4281895606 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0f7c066f30 msgr2=0x7f0f7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:33.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0f95dff700 1 --2- 172.21.15.55:0/4281895606 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0f7c066f30 0x7f0f7c0693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0f900b1820 tx=0x7f0f84017000 comp rx=0 tx=0).stop 2023-10-06T03:33:33.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0f95dff700 1 -- 172.21.15.55:0/4281895606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f900b1420 msgr2=0x7f0f900a5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:33.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.643+0000 7f0f95dff700 1 --2- 172.21.15.55:0/4281895606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f900b1420 0x7f0f900a5e10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f800058d0 tx=0x7f0f80007d10 comp rx=0 tx=0).stop 2023-10-06T03:33:33.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.644+0000 7f0f95dff700 1 -- 172.21.15.55:0/4281895606 shutdown_connections 2023-10-06T03:33:33.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.644+0000 7f0f95dff700 1 --2- 172.21.15.55:0/4281895606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0f900ccf90 0x7f0f9016a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.644+0000 7f0f95dff700 1 --2- 172.21.15.55:0/4281895606 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0f7c066f30 0x7f0f7c0693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.644+0000 7f0f95dff700 1 --2- 172.21.15.55:0/4281895606 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0f900b1420 0x7f0f900a5e10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.644+0000 7f0f95dff700 1 --2- 172.21.15.55:0/4281895606 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0f900cbf30 0x7f0f90169fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:33.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.644+0000 7f0f95dff700 1 -- 172.21.15.55:0/4281895606 >> 172.21.15.55:0/4281895606 conn(0x7f0f9006a320 msgr2=0x7f0f900afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:33.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.644+0000 7f0f95dff700 1 -- 172.21.15.55:0/4281895606 shutdown_connections 2023-10-06T03:33:33.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:33.644+0000 7f0f95dff700 1 -- 172.21.15.55:0/4281895606 wait complete. 2023-10-06T03:33:33.663 INFO:teuthology.run_tasks:Running task kclient... 2023-10-06T03:33:33.685 INFO:tasks.kclient:Mounting kernel clients... 2023-10-06T03:33:33.686 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-06T03:33:33.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:33.686 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T03:33:33.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:33.724 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link delete ceph-brx 2023-10-06T03:33:33.828 INFO:teuthology.orchestra.run.smithi055.stderr:Cannot find device "ceph-brx" 2023-10-06T03:33:33.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:33:33.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:33.831 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T03:33:33.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:33.868 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link delete ceph-brx 2023-10-06T03:33:33.971 INFO:teuthology.orchestra.run.smithi055.stderr:Cannot find device "ceph-brx" 2023-10-06T03:33:33.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:33:33.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:33.975 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-06T03:33:34.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:34.018 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2023-10-06T03:33:34.125 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2023-10-06T03:33:34.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:33:34.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:34.128 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-06T03:33:34.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:34.165 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2023-10-06T03:33:34.269 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2023-10-06T03:33:34.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T03:33:34.272 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-10-06T03:33:34.272 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:33:34.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.456+0000 7f2ea15ff700 1 Processor -- start 2023-10-06T03:33:34.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.456+0000 7f2ea15ff700 1 -- start start 2023-10-06T03:33:34.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2ea15ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2e9c0cbf30 0x7f2e9c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:34.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2ea15ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2e9c0ccf90 0x7f2e9c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:34.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2ea15ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c0cc300 con 0x7f2e9c15a270 2023-10-06T03:33:34.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2ea15ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c156bf0 con 0x7f2e9c0ccf90 2023-10-06T03:33:34.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2ea15ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c0ae950 con 0x7f2e9c0cbf30 2023-10-06T03:33:34.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2e9a7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2e9c0ccf90 0x7f2e9c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:34.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2e9a7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2e9c0ccf90 0x7f2e9c0b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51726/0 (socket says 172.21.15.55:51726) 2023-10-06T03:33:34.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2e9a7fc700 1 -- 172.21.15.55:0/3486417114 learned_addr learned my addr 172.21.15.55:0/3486417114 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:34.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2e9b7fe700 1 --1- 172.21.15.55:0/3486417114 >> v1:172.21.15.55:6789/0 conn(0x7f2e9c15a270 0x7f2e9c0c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38090/0 (socket says 172.21.15.55:38090) 2023-10-06T03:33:34.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2e9affd700 1 --2- 172.21.15.55:0/3486417114 >> v2:172.21.15.163:3300/0 conn(0x7f2e9c0cbf30 0x7f2e9c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:34.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2e99ffb700 1 -- 172.21.15.55:0/3486417114 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290456076 0 0) 0x7f2e9c0cc300 con 0x7f2e9c15a270 2023-10-06T03:33:34.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2e9a7fc700 1 -- 172.21.15.55:0/3486417114 >> v2:172.21.15.163:3300/0 conn(0x7f2e9c0cbf30 msgr2=0x7f2e9c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:34.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2e9a7fc700 1 --2- 172.21.15.55:0/3486417114 >> v2:172.21.15.163:3300/0 conn(0x7f2e9c0cbf30 0x7f2e9c0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:34.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2e9a7fc700 1 -- 172.21.15.55:0/3486417114 >> v1:172.21.15.55:6789/0 conn(0x7f2e9c15a270 legacy=0x7f2e9c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:34.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.458+0000 7f2e9a7fc700 1 -- 172.21.15.55:0/3486417114 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0ad370 con 0x7f2e9c0ccf90 2023-10-06T03:33:34.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.459+0000 7f2e9a7fc700 1 --2- 172.21.15.55:0/3486417114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e9c0ccf90 0x7f2e9c0b0d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c063d40 tx=0x7f2e90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a640a0c8a0f7cb2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:34.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.459+0000 7f2e99ffb700 1 -- 172.21.15.55:0/3486417114 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90009ea0 con 0x7f2e9c0ccf90 2023-10-06T03:33:34.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.459+0000 7f2e99ffb700 1 -- 172.21.15.55:0/3486417114 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9000b820 con 0x7f2e9c0ccf90 2023-10-06T03:33:34.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.459+0000 7f2e99ffb700 1 -- 172.21.15.55:0/3486417114 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9000bbc0 con 0x7f2e9c0ccf90 2023-10-06T03:33:34.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.459+0000 7f2ea15ff700 1 -- 172.21.15.55:0/3486417114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e9c0ccf90 msgr2=0x7f2e9c0b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:34.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.459+0000 7f2ea15ff700 1 --2- 172.21.15.55:0/3486417114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e9c0ccf90 0x7f2e9c0b0d10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c063d40 tx=0x7f2e90009250 comp rx=0 tx=0).stop 2023-10-06T03:33:34.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.460+0000 7f2ea15ff700 1 -- 172.21.15.55:0/3486417114 shutdown_connections 2023-10-06T03:33:34.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.460+0000 7f2ea15ff700 1 --2- 172.21.15.55:0/3486417114 >> v2:172.21.15.163:3300/0 conn(0x7f2e9c0cbf30 0x7f2e9c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:34.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.460+0000 7f2ea15ff700 1 --2- 172.21.15.55:0/3486417114 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e9c0ccf90 0x7f2e9c0b0d10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:34.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.460+0000 7f2ea15ff700 1 -- 172.21.15.55:0/3486417114 >> 172.21.15.55:0/3486417114 conn(0x7f2e9c06a320 msgr2=0x7f2e9c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:34.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.460+0000 7f2ea15ff700 1 -- 172.21.15.55:0/3486417114 shutdown_connections 2023-10-06T03:33:34.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.460+0000 7f2ea15ff700 1 -- 172.21.15.55:0/3486417114 wait complete. 2023-10-06T03:33:34.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.461+0000 7f2ea15ff700 1 Processor -- start 2023-10-06T03:33:34.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.461+0000 7f2ea15ff700 1 -- start start 2023-10-06T03:33:34.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.462+0000 7f2ea15ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e9c0cbf30 0x7f2e9c158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:34.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.462+0000 7f2ea15ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c0ccf90 0x7f2e9c146420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:34.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.462+0000 7f2ea15ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c15a270 0x7f2e9c14e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:34.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.462+0000 7f2ea15ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c14f3e0 con 0x7f2e9c0cbf30 2023-10-06T03:33:34.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.462+0000 7f2ea15ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c158770 con 0x7f2e9c0ccf90 2023-10-06T03:33:34.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.462+0000 7f2ea15ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c146960 con 0x7f2e9c15a270 2023-10-06T03:33:34.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.462+0000 7f2e9affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e9c0cbf30 0x7f2e9c158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:34.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.462+0000 7f2e9affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e9c0cbf30 0x7f2e9c158c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51738/0 (socket says 172.21.15.55:51738) 2023-10-06T03:33:34.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.462+0000 7f2e9affd700 1 -- 172.21.15.55:0/2998248083 learned_addr learned my addr 172.21.15.55:0/2998248083 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:34.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.462+0000 7f2e9a7fc700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c0ccf90 0x7f2e9c146420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:34.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.462+0000 7f2e9b7fe700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c15a270 0x7f2e9c14e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:34.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.463+0000 7f2e9affd700 1 -- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c15a270 msgr2=0x7f2e9c14e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:34.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.463+0000 7f2e9affd700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c15a270 0x7f2e9c14e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:34.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.463+0000 7f2e9affd700 1 -- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c0ccf90 msgr2=0x7f2e9c146420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:34.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.463+0000 7f2e9affd700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c0ccf90 0x7f2e9c146420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:34.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.463+0000 7f2e9affd700 1 -- 172.21.15.55:0/2998248083 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e900094f0 con 0x7f2e9c0cbf30 2023-10-06T03:33:34.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.463+0000 7f2e9b7fe700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c15a270 0x7f2e9c14e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:34.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.463+0000 7f2e9affd700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e9c0cbf30 0x7f2e9c158c00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00e930 tx=0x7f2e8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:34.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.463+0000 7f2e82ffd700 1 -- 172.21.15.55:0/2998248083 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c013680 con 0x7f2e9c0cbf30 2023-10-06T03:33:34.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.463+0000 7f2e82ffd700 1 -- 172.21.15.55:0/2998248083 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c0137e0 con 0x7f2e9c0cbf30 2023-10-06T03:33:34.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.463+0000 7f2ea15ff700 1 -- 172.21.15.55:0/2998248083 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c157b60 con 0x7f2e9c0cbf30 2023-10-06T03:33:34.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.463+0000 7f2e82ffd700 1 -- 172.21.15.55:0/2998248083 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c013b80 con 0x7f2e9c0cbf30 2023-10-06T03:33:34.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.463+0000 7f2ea15ff700 1 -- 172.21.15.55:0/2998248083 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c158120 con 0x7f2e9c0cbf30 2023-10-06T03:33:34.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.464+0000 7f2ea15ff700 1 -- 172.21.15.55:0/2998248083 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c158a50 con 0x7f2e9c0cbf30 2023-10-06T03:33:34.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.465+0000 7f2e82ffd700 1 -- 172.21.15.55:0/2998248083 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e8c00c460 con 0x7f2e9c0cbf30 2023-10-06T03:33:34.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.466+0000 7f2e82ffd700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2e88066f30 0x7f2e880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:34.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.466+0000 7f2e82ffd700 1 -- 172.21.15.55:0/2998248083 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2e8c0848a0 con 0x7f2e9c0cbf30 2023-10-06T03:33:34.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.466+0000 7f2e9a7fc700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2e88066f30 0x7f2e880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:34.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.467+0000 7f2e9a7fc700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2e88066f30 0x7f2e880693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c063d40 tx=0x7f2e90017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:34.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.470+0000 7f2e82ffd700 1 -- 172.21.15.55:0/2998248083 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e8c051910 con 0x7f2e9c0cbf30 2023-10-06T03:33:34.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.718+0000 7f2ea15ff700 1 -- 172.21.15.55:0/2998248083 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2e9c04c3c0 con 0x7f2e9c0cbf30 2023-10-06T03:33:34.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.719+0000 7f2e82ffd700 1 -- 172.21.15.55:0/2998248083 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2e8c054f30 con 0x7f2e9c0cbf30 2023-10-06T03:33:34.721 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:33:34.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.721+0000 7f2ea15ff700 1 -- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2e88066f30 msgr2=0x7f2e880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:34.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.721+0000 7f2ea15ff700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2e88066f30 0x7f2e880693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c063d40 tx=0x7f2e90017000 comp rx=0 tx=0).stop 2023-10-06T03:33:34.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.721+0000 7f2ea15ff700 1 -- 172.21.15.55:0/2998248083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e9c0cbf30 msgr2=0x7f2e9c158c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:34.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.721+0000 7f2ea15ff700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e9c0cbf30 0x7f2e9c158c00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c00e930 tx=0x7f2e8c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:34.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.721+0000 7f2ea15ff700 1 -- 172.21.15.55:0/2998248083 shutdown_connections 2023-10-06T03:33:34.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.721+0000 7f2ea15ff700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2e88066f30 0x7f2e880693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:34.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.721+0000 7f2ea15ff700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e9c15a270 0x7f2e9c14e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:34.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.721+0000 7f2ea15ff700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e9c0cbf30 0x7f2e9c158c00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:34.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.721+0000 7f2ea15ff700 1 --2- 172.21.15.55:0/2998248083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e9c0ccf90 0x7f2e9c146420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:34.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.721+0000 7f2ea15ff700 1 -- 172.21.15.55:0/2998248083 >> 172.21.15.55:0/2998248083 conn(0x7f2e9c06a320 msgr2=0x7f2e9c155fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:34.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.721+0000 7f2ea15ff700 1 -- 172.21.15.55:0/2998248083 shutdown_connections 2023-10-06T03:33:34.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:34.721+0000 7f2ea15ff700 1 -- 172.21.15.55:0/2998248083 wait complete. 2023-10-06T03:33:34.741 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T03:33:34.741 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:33:34.741 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2023-10-06T03:33:34.742 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:33:34.742 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:33:34.742 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T03:33:34.742 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T03:33:34.742 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:33:34.742 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:33:34.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:34.743 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2023-10-06T03:33:34.780 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:33:34.780 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:33:34.780 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:33:34.780 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:34.780 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T03:33:34.780 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:34.780 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:33:34.780 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:34.781 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2023-10-06T03:33:34.781 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:33:34.781 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:34.781 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2023-10-06T03:33:34.781 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:33:34.781 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:34.781 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2023-10-06T03:33:34.781 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T03:33:34.781 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42227sec preferred_lft 42227sec 2023-10-06T03:33:34.782 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link 2023-10-06T03:33:34.782 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:34.782 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:33:34.782 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:34.782 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2023-10-06T03:33:34.783 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T03:33:34.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:34.783 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:33:34.783 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:33:34.783 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add name ceph-brx type bridge 2023-10-06T03:33:34.783 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip addr flush dev ceph-brx 2023-10-06T03:33:34.783 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set ceph-brx up 2023-10-06T03:33:34.783 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T03:33:34.783 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:33:35.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:35.245 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:33:35.383 INFO:teuthology.orchestra.run.smithi055.stdout:1 2023-10-06T03:33:35.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:35.387 DEBUG:teuthology.orchestra.run.smithi055:> route 2023-10-06T03:33:35.441 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2023-10-06T03:33:35.441 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:33:35.441 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T03:33:35.441 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T03:33:35.441 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:33:35.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:35.443 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:33:35.443 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:33:35.443 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T03:33:35.443 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T03:33:35.445 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T03:33:35.445 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:33:36.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:36.192 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T03:33:36.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:36.229 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list-id 2023-10-06T03:33:36.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:36.306 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:33:36.306 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:33:36.306 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:33:36.306 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T03:33:36.306 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:33:36.476 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T03:33:36.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:36.477 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:33:36.477 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:33:36.477 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T03:33:36.477 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T03:33:36.477 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T03:33:36.477 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T03:33:36.477 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T03:33:36.477 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:33:36.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:36.725 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:33:36.725 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:33:36.725 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2023-10-06T03:33:36.726 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T03:33:36.726 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:33:36.885 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:33:36.886 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:33:36.923 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:33:36.924 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:33:36.924 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:33:37.002 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T03:33:37.003 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:33:37.003 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:33:37.149 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:33:37.149 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:33:37.149 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:33:37.643 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:33:37.643 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:33:37.644 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:33:37.644 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:33:37.644 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:33:37.667 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:33:37.668 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:33:37.668 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:33:37.668 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:33:37.668 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:33:37.668 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T03:33:37.668 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T03:33:37.668 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T03:33:37.673 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:33:37.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:37.673 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:33:37.737 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-10-06T03:33:37.737 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:33:37.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.922+0000 7fadc119d700 1 Processor -- start 2023-10-06T03:33:37.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.922+0000 7fadc119d700 1 -- start start 2023-10-06T03:33:37.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.923+0000 7fadc119d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadbc1597f0 0x7fadbc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:37.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.923+0000 7fadc119d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fadbc15d230 0x7fadbc15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:37.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.923+0000 7fadc119d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadbc15dbd0 con 0x7fadbc159420 2023-10-06T03:33:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.923+0000 7fadc119d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc0b11d0 con 0x7fadbc15d230 2023-10-06T03:33:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.923+0000 7fadc119d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0b14b0 con 0x7fadbc1597f0 2023-10-06T03:33:37.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.923+0000 7fadbbfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fadbc159420 0x7fadbc0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38098/0 (socket says 172.21.15.55:38098) 2023-10-06T03:33:37.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.923+0000 7fadbbfff700 1 -- 172.21.15.55:0/1911242404 learned_addr learned my addr 172.21.15.55:0/1911242404 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:37.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.924+0000 7fadbb7fe700 1 --2- 172.21.15.55:0/1911242404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadbc1597f0 0x7fadbc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:37.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.924+0000 7fadc099c700 1 --2- 172.21.15.55:0/1911242404 >> v2:172.21.15.163:3300/0 conn(0x7fadbc15d230 0x7fadbc15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:37.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.924+0000 7fadbaffd700 1 -- 172.21.15.55:0/1911242404 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075864544 0 0) 0x7fadac001110 con 0x7fadbc159420 2023-10-06T03:33:37.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.924+0000 7fadbaffd700 1 -- 172.21.15.55:0/1911242404 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fada4003570 con 0x7fadbc159420 2023-10-06T03:33:37.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.924+0000 7fadbb7fe700 1 -- 172.21.15.55:0/1911242404 >> v2:172.21.15.163:3300/0 conn(0x7fadbc15d230 msgr2=0x7fadbc15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:37.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.924+0000 7fadbb7fe700 1 --2- 172.21.15.55:0/1911242404 >> v2:172.21.15.163:3300/0 conn(0x7fadbc15d230 0x7fadbc15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:37.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.924+0000 7fadbb7fe700 1 -- 172.21.15.55:0/1911242404 >> v1:172.21.15.55:6789/0 conn(0x7fadbc159420 legacy=0x7fadbc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:37.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.924+0000 7fadbb7fe700 1 -- 172.21.15.55:0/1911242404 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc0b1790 con 0x7fadbc1597f0 2023-10-06T03:33:37.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.925+0000 7fadbb7fe700 1 --2- 172.21.15.55:0/1911242404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadbc1597f0 0x7fadbc0b2270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fadb00098e0 tx=0x7fadb00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe01f09d6ab3fcb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:37.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.925+0000 7fadbaffd700 1 -- 172.21.15.55:0/1911242404 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0011630 con 0x7fadbc1597f0 2023-10-06T03:33:37.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.925+0000 7fadbaffd700 1 -- 172.21.15.55:0/1911242404 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb0011790 con 0x7fadbc1597f0 2023-10-06T03:33:37.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.925+0000 7fadbaffd700 1 -- 172.21.15.55:0/1911242404 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0011b30 con 0x7fadbc1597f0 2023-10-06T03:33:37.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.925+0000 7fadc119d700 1 -- 172.21.15.55:0/1911242404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadbc1597f0 msgr2=0x7fadbc0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:37.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.925+0000 7fadc119d700 1 --2- 172.21.15.55:0/1911242404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadbc1597f0 0x7fadbc0b2270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fadb00098e0 tx=0x7fadb00091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:37.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.925+0000 7fadc119d700 1 -- 172.21.15.55:0/1911242404 shutdown_connections 2023-10-06T03:33:37.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.925+0000 7fadc119d700 1 --2- 172.21.15.55:0/1911242404 >> v2:172.21.15.163:3300/0 conn(0x7fadbc15d230 0x7fadbc15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:37.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.925+0000 7fadc119d700 1 --2- 172.21.15.55:0/1911242404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadbc1597f0 0x7fadbc0b2270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:37.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.925+0000 7fadc119d700 1 -- 172.21.15.55:0/1911242404 >> 172.21.15.55:0/1911242404 conn(0x7fadbc0d0660 msgr2=0x7fadbc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:37.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.926+0000 7fadc119d700 1 -- 172.21.15.55:0/1911242404 shutdown_connections 2023-10-06T03:33:37.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.926+0000 7fadc119d700 1 -- 172.21.15.55:0/1911242404 wait complete. 2023-10-06T03:33:37.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.927+0000 7fadc119d700 1 Processor -- start 2023-10-06T03:33:37.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.927+0000 7fadc119d700 1 -- start start 2023-10-06T03:33:37.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.928+0000 7fadc119d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadbc159420 0x7fadbc0c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:37.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.928+0000 7fadc119d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadbc1597f0 0x7fadbc1580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:37.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.928+0000 7fadc119d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadbc15d230 0x7fadbc1585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:37.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.928+0000 7fadc119d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc16d270 con 0x7fadbc15d230 2023-10-06T03:33:37.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.928+0000 7fadc119d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc16ddf0 con 0x7fadbc1597f0 2023-10-06T03:33:37.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.928+0000 7fadc119d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc16d4c0 con 0x7fadbc159420 2023-10-06T03:33:37.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.928+0000 7fadc099c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadbc15d230 0x7fadbc1585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:37.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.928+0000 7fadc099c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadbc15d230 0x7fadbc1585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51746/0 (socket says 172.21.15.55:51746) 2023-10-06T03:33:37.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.928+0000 7fadc099c700 1 -- 172.21.15.55:0/382236227 learned_addr learned my addr 172.21.15.55:0/382236227 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:37.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.928+0000 7fadbb7fe700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadbc1597f0 0x7fadbc1580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:37.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.929+0000 7fadbbfff700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadbc159420 0x7fadbc0c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:37.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.929+0000 7fadc099c700 1 -- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadbc159420 msgr2=0x7fadbc0c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:37.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.929+0000 7fadc099c700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadbc159420 0x7fadbc0c6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:37.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.929+0000 7fadc099c700 1 -- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadbc1597f0 msgr2=0x7fadbc1580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:37.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.929+0000 7fadc099c700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadbc1597f0 0x7fadbc1580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:37.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.929+0000 7fadc099c700 1 -- 172.21.15.55:0/382236227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb0009420 con 0x7fadbc15d230 2023-10-06T03:33:37.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.929+0000 7fadbbfff700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadbc159420 0x7fadbc0c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:37.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.929+0000 7fadc099c700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadbc15d230 0x7fadbc1585f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009df0 tx=0x7fadb400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:37.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.929+0000 7fada3fff700 1 -- 172.21.15.55:0/382236227 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb400cb50 con 0x7fadbc15d230 2023-10-06T03:33:37.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.929+0000 7fada3fff700 1 -- 172.21.15.55:0/382236227 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb400f040 con 0x7fadbc15d230 2023-10-06T03:33:37.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.929+0000 7fadc119d700 1 -- 172.21.15.55:0/382236227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc16d7a0 con 0x7fadbc15d230 2023-10-06T03:33:37.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.929+0000 7fada3fff700 1 -- 172.21.15.55:0/382236227 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4014480 con 0x7fadbc15d230 2023-10-06T03:33:37.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.930+0000 7fadc119d700 1 -- 172.21.15.55:0/382236227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc0c7390 con 0x7fadbc15d230 2023-10-06T03:33:37.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.931+0000 7fadc119d700 1 -- 172.21.15.55:0/382236227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc16e180 con 0x7fadbc15d230 2023-10-06T03:33:37.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.932+0000 7fada3fff700 1 -- 172.21.15.55:0/382236227 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fadb4014830 con 0x7fadbc15d230 2023-10-06T03:33:37.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.933+0000 7fada3fff700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fada8066f30 0x7fada80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:37.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.933+0000 7fada3fff700 1 -- 172.21.15.55:0/382236227 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fadb40850c0 con 0x7fadbc15d230 2023-10-06T03:33:37.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.933+0000 7fadbbfff700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fada8066f30 0x7fada80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:37.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.934+0000 7fadbbfff700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fada8066f30 0x7fada80693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fadac001270 tx=0x7fadac009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:37.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:37.938+0000 7fada3fff700 1 -- 172.21.15.55:0/382236227 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fadb4052130 con 0x7fadbc15d230 2023-10-06T03:33:38.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:38.185+0000 7fadc119d700 1 -- 172.21.15.55:0/382236227 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fadbc0c7580 con 0x7fadbc15d230 2023-10-06T03:33:38.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:38.185+0000 7fada3fff700 1 -- 172.21.15.55:0/382236227 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fadb4055750 con 0x7fadbc15d230 2023-10-06T03:33:38.188 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:33:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:38.187+0000 7fadc119d700 1 -- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fada8066f30 msgr2=0x7fada80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:38.187+0000 7fadc119d700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fada8066f30 0x7fada80693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fadac001270 tx=0x7fadac009040 comp rx=0 tx=0).stop 2023-10-06T03:33:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:38.187+0000 7fadc119d700 1 -- 172.21.15.55:0/382236227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadbc15d230 msgr2=0x7fadbc1585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:38.187+0000 7fadc119d700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadbc15d230 0x7fadbc1585f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009df0 tx=0x7fadb400b040 comp rx=0 tx=0).stop 2023-10-06T03:33:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:38.188+0000 7fadc119d700 1 -- 172.21.15.55:0/382236227 shutdown_connections 2023-10-06T03:33:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:38.188+0000 7fadc119d700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadbc15d230 0x7fadbc1585f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:38.188+0000 7fadc119d700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fada8066f30 0x7fada80693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:38.188+0000 7fadc119d700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadbc159420 0x7fadbc0c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:38.188+0000 7fadc119d700 1 --2- 172.21.15.55:0/382236227 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadbc1597f0 0x7fadbc1580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:38.188+0000 7fadc119d700 1 -- 172.21.15.55:0/382236227 >> 172.21.15.55:0/382236227 conn(0x7fadbc0d0660 msgr2=0x7fadbc156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:38.188+0000 7fadc119d700 1 -- 172.21.15.55:0/382236227 shutdown_connections 2023-10-06T03:33:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:38.188+0000 7fadc119d700 1 -- 172.21.15.55:0/382236227 wait complete. 2023-10-06T03:33:38.208 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T03:33:38.208 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:33:38.208 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2023-10-06T03:33:38.208 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-06T03:33:38.208 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-06T03:33:38.208 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T03:33:38.209 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T03:33:38.209 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:33:38.209 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-06T03:33:38.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:38.209 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2023-10-06T03:33:38.246 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:33:38.247 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:33:38.247 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:33:38.247 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:38.247 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T03:33:38.247 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:38.247 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:33:38.247 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:38.247 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2023-10-06T03:33:38.247 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:33:38.247 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:38.248 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2023-10-06T03:33:38.248 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:33:38.248 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:38.248 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2023-10-06T03:33:38.248 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T03:33:38.248 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42223sec preferred_lft 42223sec 2023-10-06T03:33:38.248 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link 2023-10-06T03:33:38.248 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:38.249 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:33:38.249 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:38.249 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2023-10-06T03:33:38.249 INFO:teuthology.orchestra.run.smithi055.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:33:38.249 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether f2:5d:53:d2:00:0a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:38.249 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:33:38.249 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:38.249 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::1099:8eff:fe6d:102/64 scope link 2023-10-06T03:33:38.249 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:38.249 INFO:teuthology.orchestra.run.smithi055.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:33:38.250 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether f2:5d:53:d2:00:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:33:38.250 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f05d:53ff:fed2:a/64 scope link tentative 2023-10-06T03:33:38.250 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:38.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:38.250 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:33:38.346 INFO:teuthology.orchestra.run.smithi055.stdout:1 2023-10-06T03:33:38.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:38.350 DEBUG:teuthology.orchestra.run.smithi055:> route 2023-10-06T03:33:38.387 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2023-10-06T03:33:38.388 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:33:38.388 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T03:33:38.388 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T03:33:38.388 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:33:38.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:38.389 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:33:38.389 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:33:38.389 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T03:33:38.389 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T03:33:38.390 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T03:33:38.390 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:33:38.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:38.588 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T03:33:38.625 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:33:38.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:38.625 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list-id 2023-10-06T03:33:38.701 INFO:teuthology.orchestra.run.smithi055.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T03:33:38.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:38.702 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:33:38.703 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:33:38.703 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:33:38.703 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-06T03:33:38.703 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:33:38.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:38.872 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T03:33:38.935 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:33:38.935 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:33:38.935 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:33:38.935 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:38.936 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T03:33:38.936 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:38.936 INFO:teuthology.orchestra.run.smithi055.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:33:38.936 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether be:0a:81:ea:e4:4c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T03:33:38.936 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T03:33:38.936 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:38.936 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::bc0a:81ff:feea:e44c/64 scope link 2023-10-06T03:33:38.936 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:38.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:38.938 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T03:33:39.000 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:33:39.000 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:33:39.001 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:33:39.001 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:39.001 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T03:33:39.001 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:39.001 INFO:teuthology.orchestra.run.smithi055.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:33:39.001 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether be:0a:81:ea:e4:4c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T03:33:39.001 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T03:33:39.001 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:39.001 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::bc0a:81ff:feea:e44c/64 scope link 2023-10-06T03:33:39.001 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:39.003 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-06T03:33:39.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:39.003 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:33:39.004 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:33:39.004 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-06T03:33:39.004 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T03:33:39.004 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-06T03:33:39.004 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-06T03:33:39.004 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-06T03:33:39.004 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:33:39.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:39.245 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:33:39.245 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:33:39.245 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.1 up 2023-10-06T03:33:39.245 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T03:33:39.245 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:33:39.404 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:33:39.405 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-06T03:33:39.442 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-06T03:33:39.442 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:33:39.442 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-06T03:33:39.522 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T03:33:39.522 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:33:39.522 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T03:33:39.655 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T03:33:39.655 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-06T03:33:39.655 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:33:39.671 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T03:33:39.671 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-06T03:33:39.671 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:33:39.672 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:33:39.672 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:33:39.693 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-10-06T03:33:39.693 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-10-06T03:33:39.693 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:33:39.693 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:33:39.693 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:33:39.693 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T03:33:39.694 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T03:33:39.694 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T03:33:39.699 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:33:39.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:39.699 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-06T03:33:39.761 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-10-06T03:33:39.762 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:33:39.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.946+0000 7f17d9b9d700 1 Processor -- start 2023-10-06T03:33:39.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.947+0000 7f17d9b9d700 1 -- start start 2023-10-06T03:33:39.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.948+0000 7f17d9b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d415d560 0x7f17d4164680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:39.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.948+0000 7f17d9b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d40aa610 con 0x7f17d415ca80 2023-10-06T03:33:39.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.948+0000 7f17d9b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d40abab0 con 0x7f17d40ad690 2023-10-06T03:33:39.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.948+0000 7f17d9b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d40a9a00 con 0x7f17d415d560 2023-10-06T03:33:39.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.948+0000 7f17d8b9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f17d415ca80 0x7f17d40acf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38106/0 (socket says 172.21.15.55:38106) 2023-10-06T03:33:39.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.948+0000 7f17d8b9b700 1 -- 172.21.15.55:0/1724334306 learned_addr learned my addr 172.21.15.55:0/1724334306 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:39.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.949+0000 7f17d939c700 1 --2- 172.21.15.55:0/1724334306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d415d560 0x7f17d4164680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:39.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.949+0000 7f17d37fe700 1 -- 172.21.15.55:0/1724334306 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655373205 0 0) 0x7f17d40aa610 con 0x7f17d415ca80 2023-10-06T03:33:39.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.949+0000 7f17d37fe700 1 -- 172.21.15.55:0/1724334306 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17c4003570 con 0x7f17d415ca80 2023-10-06T03:33:39.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.949+0000 7f17d37fe700 1 -- 172.21.15.55:0/1724334306 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 961620119 0 0) 0x7f17d40abab0 con 0x7f17d40ad690 2023-10-06T03:33:39.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.949+0000 7f17d37fe700 1 -- 172.21.15.55:0/1724334306 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17d40aa610 con 0x7f17d40ad690 2023-10-06T03:33:39.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.949+0000 7f17d939c700 1 -- 172.21.15.55:0/1724334306 >> v1:172.21.15.163:6789/0 conn(0x7f17d40ad690 legacy=0x7f17d415ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:39.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.949+0000 7f17d37fe700 1 -- 172.21.15.55:0/1724334306 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2498093309 0 0) 0x7f17c4003570 con 0x7f17d415ca80 2023-10-06T03:33:39.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.949+0000 7f17d939c700 1 -- 172.21.15.55:0/1724334306 >> v1:172.21.15.55:6789/0 conn(0x7f17d415ca80 legacy=0x7f17d40acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:39.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.949+0000 7f17d939c700 1 -- 172.21.15.55:0/1724334306 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d40a9ce0 con 0x7f17d415d560 2023-10-06T03:33:39.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.949+0000 7f17d939c700 1 --2- 172.21.15.55:0/1724334306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d415d560 0x7f17d4164680 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009e90 tx=0x7f17cc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40033342a42920fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:39.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.950+0000 7f17d37fe700 1 -- 172.21.15.55:0/1724334306 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc00e070 con 0x7f17d415d560 2023-10-06T03:33:39.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.950+0000 7f17d37fe700 1 -- 172.21.15.55:0/1724334306 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17cc00b870 con 0x7f17d415d560 2023-10-06T03:33:39.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.950+0000 7f17d37fe700 1 -- 172.21.15.55:0/1724334306 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8001470 con 0x7f17d415d560 2023-10-06T03:33:39.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.950+0000 7f17d9b9d700 1 -- 172.21.15.55:0/1724334306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d415d560 msgr2=0x7f17d4164680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:39.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.950+0000 7f17d9b9d700 1 --2- 172.21.15.55:0/1724334306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d415d560 0x7f17d4164680 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009e90 tx=0x7f17cc009780 comp rx=0 tx=0).stop 2023-10-06T03:33:39.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.950+0000 7f17d9b9d700 1 -- 172.21.15.55:0/1724334306 shutdown_connections 2023-10-06T03:33:39.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.950+0000 7f17d9b9d700 1 --2- 172.21.15.55:0/1724334306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d415d560 0x7f17d4164680 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:39.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.950+0000 7f17d9b9d700 1 -- 172.21.15.55:0/1724334306 >> 172.21.15.55:0/1724334306 conn(0x7f17d4068e20 msgr2=0x7f17d40cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:39.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.950+0000 7f17d9b9d700 1 -- 172.21.15.55:0/1724334306 shutdown_connections 2023-10-06T03:33:39.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.950+0000 7f17d9b9d700 1 -- 172.21.15.55:0/1724334306 wait complete. 2023-10-06T03:33:39.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.952+0000 7f17d9b9d700 1 Processor -- start 2023-10-06T03:33:39.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.952+0000 7f17d9b9d700 1 -- start start 2023-10-06T03:33:39.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d9b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d40ad690 0x7f17d4154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:39.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d9b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17d415ca80 0x7f17d40cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:39.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d9b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d415d560 0x7f17d40cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:39.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d9b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d41550a0 con 0x7f17d415ca80 2023-10-06T03:33:39.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d9b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d41575f0 con 0x7f17d415d560 2023-10-06T03:33:39.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d9b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d41578d0 con 0x7f17d40ad690 2023-10-06T03:33:39.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17d415ca80 0x7f17d40cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:39.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d939c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d415d560 0x7f17d40cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:39.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17d415ca80 0x7f17d40cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51748/0 (socket says 172.21.15.55:51748) 2023-10-06T03:33:39.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d3fff700 1 -- 172.21.15.55:0/694597335 learned_addr learned my addr 172.21.15.55:0/694597335 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:39.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d8b9b700 1 --2- 172.21.15.55:0/694597335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d40ad690 0x7f17d4154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:39.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d3fff700 1 -- 172.21.15.55:0/694597335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d40ad690 msgr2=0x7f17d4154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:39.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d3fff700 1 --2- 172.21.15.55:0/694597335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d40ad690 0x7f17d4154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:39.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d3fff700 1 -- 172.21.15.55:0/694597335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d415d560 msgr2=0x7f17d40cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:39.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d3fff700 1 --2- 172.21.15.55:0/694597335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d415d560 0x7f17d40cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:39.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d3fff700 1 -- 172.21.15.55:0/694597335 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc009a20 con 0x7f17d415ca80 2023-10-06T03:33:39.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.953+0000 7f17d939c700 1 --2- 172.21.15.55:0/694597335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d415d560 0x7f17d40cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:39.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.954+0000 7f17d3fff700 1 --2- 172.21.15.55:0/694597335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17d415ca80 0x7f17d40cabc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c790 tx=0x7f17c0007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:39.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.954+0000 7f17d0ff9700 1 -- 172.21.15.55:0/694597335 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0010510 con 0x7f17d415ca80 2023-10-06T03:33:39.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.954+0000 7f17d0ff9700 1 -- 172.21.15.55:0/694597335 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c0010670 con 0x7f17d415ca80 2023-10-06T03:33:39.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.954+0000 7f17d0ff9700 1 -- 172.21.15.55:0/694597335 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0010a30 con 0x7f17d415ca80 2023-10-06T03:33:39.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.954+0000 7f17d9b9d700 1 -- 172.21.15.55:0/694597335 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d4157bb0 con 0x7f17d415ca80 2023-10-06T03:33:39.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.954+0000 7f17d9b9d700 1 -- 172.21.15.55:0/694597335 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d415a400 con 0x7f17d415ca80 2023-10-06T03:33:39.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.955+0000 7f17d9b9d700 1 -- 172.21.15.55:0/694597335 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d415a6b0 con 0x7f17d415ca80 2023-10-06T03:33:39.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.956+0000 7f17d0ff9700 1 -- 172.21.15.55:0/694597335 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f17c001b3b0 con 0x7f17d415ca80 2023-10-06T03:33:39.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.957+0000 7f17d0ff9700 1 --2- 172.21.15.55:0/694597335 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f17b8066f30 0x7f17b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:39.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.957+0000 7f17d0ff9700 1 -- 172.21.15.55:0/694597335 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f17c0083f30 con 0x7f17d415ca80 2023-10-06T03:33:39.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.957+0000 7f17d8b9b700 1 --2- 172.21.15.55:0/694597335 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f17b8066f30 0x7f17b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:39.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.958+0000 7f17d8b9b700 1 --2- 172.21.15.55:0/694597335 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f17b8066f30 0x7f17b80693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f17c80011e0 tx=0x7f17c8001a00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:39.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:39.961+0000 7f17d0ff9700 1 -- 172.21.15.55:0/694597335 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17d415a6b0 con 0x7f17d415ca80 2023-10-06T03:33:40.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:40.209+0000 7f17d9b9d700 1 -- 172.21.15.55:0/694597335 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f17d415a6b0 con 0x7f17d415ca80 2023-10-06T03:33:40.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:40.210+0000 7f17d0ff9700 1 -- 172.21.15.55:0/694597335 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f17d415a6b0 con 0x7f17d415ca80 2023-10-06T03:33:40.211 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:33:40.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:40.212+0000 7f17d9b9d700 1 -- 172.21.15.55:0/694597335 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f17b8066f30 msgr2=0x7f17b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:40.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:40.212+0000 7f17d9b9d700 1 --2- 172.21.15.55:0/694597335 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f17b8066f30 0x7f17b80693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f17c80011e0 tx=0x7f17c8001a00 comp rx=0 tx=0).stop 2023-10-06T03:33:40.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:40.212+0000 7f17d9b9d700 1 -- 172.21.15.55:0/694597335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17d415ca80 msgr2=0x7f17d40cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:40.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:40.212+0000 7f17d9b9d700 1 --2- 172.21.15.55:0/694597335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17d415ca80 0x7f17d40cabc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c790 tx=0x7f17c0007e10 comp rx=0 tx=0).stop 2023-10-06T03:33:40.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:40.212+0000 7f17d9b9d700 1 -- 172.21.15.55:0/694597335 shutdown_connections 2023-10-06T03:33:40.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:40.212+0000 7f17d9b9d700 1 --2- 172.21.15.55:0/694597335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17d415d560 0x7f17d40cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:40.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:40.212+0000 7f17d9b9d700 1 --2- 172.21.15.55:0/694597335 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f17b8066f30 0x7f17b80693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:40.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:40.212+0000 7f17d9b9d700 1 --2- 172.21.15.55:0/694597335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17d40ad690 0x7f17d4154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:40.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:40.212+0000 7f17d9b9d700 1 --2- 172.21.15.55:0/694597335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17d415ca80 0x7f17d40cabc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:40.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:40.212+0000 7f17d9b9d700 1 -- 172.21.15.55:0/694597335 >> 172.21.15.55:0/694597335 conn(0x7f17d4068e20 msgr2=0x7f17d4069a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:40.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:40.212+0000 7f17d9b9d700 1 -- 172.21.15.55:0/694597335 shutdown_connections 2023-10-06T03:33:40.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:40.212+0000 7f17d9b9d700 1 -- 172.21.15.55:0/694597335 wait complete. 2023-10-06T03:33:40.231 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T03:33:40.231 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:33:40.231 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-06T03:33:40.231 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T03:33:40.231 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T03:33:40.231 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T03:33:40.231 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T03:33:40.232 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:33:40.232 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:33:40.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:40.232 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-06T03:33:40.272 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:33:40.272 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:33:40.273 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:33:40.273 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:40.273 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-06T03:33:40.273 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:40.273 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:33:40.273 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:40.273 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-06T03:33:40.273 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:33:40.273 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:40.273 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-06T03:33:40.274 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:33:40.274 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:40.274 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:33:40.274 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:40.274 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:33:40.274 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42246sec preferred_lft 42246sec 2023-10-06T03:33:40.274 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-06T03:33:40.274 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:40.275 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T03:33:40.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:40.276 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-06T03:33:40.276 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-06T03:33:40.276 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add name ceph-brx type bridge 2023-10-06T03:33:40.276 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip addr flush dev ceph-brx 2023-10-06T03:33:40.276 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set ceph-brx up 2023-10-06T03:33:40.276 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T03:33:40.276 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-06T03:33:40.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:40.738 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:33:40.877 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-06T03:33:40.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:40.880 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-06T03:33:40.934 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-06T03:33:40.934 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:33:40.934 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:33:40.934 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:33:40.935 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:33:40.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:40.936 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-06T03:33:40.936 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-06T03:33:40.936 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:33:40.936 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:33:40.936 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:33:40.936 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-06T03:33:41.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:41.624 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-06T03:33:41.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:41.660 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list-id 2023-10-06T03:33:41.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:41.737 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-06T03:33:41.737 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-06T03:33:41.737 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:33:41.738 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-06T03:33:41.738 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-06T03:33:41.903 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-06T03:33:41.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:41.903 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-06T03:33:41.903 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-06T03:33:41.903 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-06T03:33:41.903 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T03:33:41.903 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T03:33:41.904 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T03:33:41.904 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T03:33:41.904 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-06T03:33:41.914 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:33:41.920 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:33:42.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:42.162 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-06T03:33:42.162 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-06T03:33:42.162 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.0 up 2023-10-06T03:33:42.162 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T03:33:42.162 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-06T03:33:42.320 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:33:42.320 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T03:33:42.359 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T03:33:42.359 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:33:42.359 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:33:42.437 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T03:33:42.437 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:33:42.437 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T03:33:42.587 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T03:33:42.587 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T03:33:42.587 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2023-10-06T03:33:43.027 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T03:33:43.027 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T03:33:43.027 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2023-10-06T03:33:43.027 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:33:43.028 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:33:43.052 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-10-06T03:33:43.052 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-10-06T03:33:43.052 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2023-10-06T03:33:43.052 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:33:43.052 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:33:43.052 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T03:33:43.052 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T03:33:43.052 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T03:33:43.057 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:33:43.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:43.057 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T03:33:43.118 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-10-06T03:33:43.118 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:33:43.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.314+0000 7f9145f9d700 1 Processor -- start 2023-10-06T03:33:43.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.314+0000 7f9145f9d700 1 -- start start 2023-10-06T03:33:43.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.316+0000 7f9145f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f914006a0a0 0x7f914015b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:43.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.316+0000 7f9145f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f914006dd20 0x7f914015b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:43.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.316+0000 7f9145f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f914006ae70 con 0x7f914006d950 2023-10-06T03:33:43.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.316+0000 7f9145f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91400c8a60 con 0x7f914006dd20 2023-10-06T03:33:43.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.316+0000 7f9145f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914006a540 con 0x7f914006a0a0 2023-10-06T03:33:43.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.316+0000 7f914579c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f914006dd20 0x7f914015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:43.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.316+0000 7f914579c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f914006dd20 0x7f914015b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36016/0 (socket says 172.21.15.55:36016) 2023-10-06T03:33:43.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.316+0000 7f914579c700 1 -- 172.21.15.55:0/3957539578 learned_addr learned my addr 172.21.15.55:0/3957539578 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:43.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.316+0000 7f913ffff700 1 --2- 172.21.15.55:0/3957539578 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f914006a0a0 0x7f914015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:43.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.316+0000 7f914579c700 1 -- 172.21.15.55:0/3957539578 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f914006a0a0 msgr2=0x7f914015b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:43.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.316+0000 7f914579c700 1 --2- 172.21.15.55:0/3957539578 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f914006a0a0 0x7f914015b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:43.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.316+0000 7f914579c700 1 -- 172.21.15.55:0/3957539578 >> v1:172.21.15.163:6789/0 conn(0x7f914006d950 legacy=0x7f91400c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:43.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.317+0000 7f914579c700 1 -- 172.21.15.55:0/3957539578 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914006a820 con 0x7f914006dd20 2023-10-06T03:33:43.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.316+0000 7f913f7fe700 1 -- 172.21.15.55:0/3957539578 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540183510 0 0) 0x7f914006ae70 con 0x7f914006d950 2023-10-06T03:33:43.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.317+0000 7f914579c700 1 --2- 172.21.15.55:0/3957539578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f914006dd20 0x7f914015b950 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9138009f20 tx=0x7f9138009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bdaa678308fdfa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:43.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.317+0000 7f913f7fe700 1 -- 172.21.15.55:0/3957539578 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913800e070 con 0x7f914006dd20 2023-10-06T03:33:43.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.317+0000 7f913f7fe700 1 -- 172.21.15.55:0/3957539578 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913800b9d0 con 0x7f914006dd20 2023-10-06T03:33:43.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.317+0000 7f913f7fe700 1 -- 172.21.15.55:0/3957539578 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91380092e0 con 0x7f914006dd20 2023-10-06T03:33:43.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.317+0000 7f9145f9d700 1 -- 172.21.15.55:0/3957539578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f914006dd20 msgr2=0x7f914015b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:43.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.317+0000 7f9145f9d700 1 --2- 172.21.15.55:0/3957539578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f914006dd20 0x7f914015b950 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9138009f20 tx=0x7f9138009840 comp rx=0 tx=0).stop 2023-10-06T03:33:43.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.318+0000 7f9145f9d700 1 -- 172.21.15.55:0/3957539578 shutdown_connections 2023-10-06T03:33:43.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.318+0000 7f9145f9d700 1 --2- 172.21.15.55:0/3957539578 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f914006dd20 0x7f914015b950 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:43.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.318+0000 7f9145f9d700 1 --2- 172.21.15.55:0/3957539578 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f914006a0a0 0x7f914015b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:43.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.318+0000 7f9145f9d700 1 -- 172.21.15.55:0/3957539578 >> 172.21.15.55:0/3957539578 conn(0x7f91400cc1f0 msgr2=0x7f91400cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:43.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.318+0000 7f9145f9d700 1 -- 172.21.15.55:0/3957539578 shutdown_connections 2023-10-06T03:33:43.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.318+0000 7f9145f9d700 1 -- 172.21.15.55:0/3957539578 wait complete. 2023-10-06T03:33:43.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.319+0000 7f9145f9d700 1 Processor -- start 2023-10-06T03:33:43.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.319+0000 7f9145f9d700 1 -- start start 2023-10-06T03:33:43.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.320+0000 7f9145f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f914006a0a0 0x7f91400affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:43.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.320+0000 7f9145f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f914006d950 0x7f91400b0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:43.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.320+0000 7f9145f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f914006dd20 0x7f9140167530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:43.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.320+0000 7f9145f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91400b1820 con 0x7f914006d950 2023-10-06T03:33:43.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.320+0000 7f9145f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91400b0c10 con 0x7f914006dd20 2023-10-06T03:33:43.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.320+0000 7f9145f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91400b0ef0 con 0x7f914006a0a0 2023-10-06T03:33:43.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.320+0000 7f913ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f914006d950 0x7f91400b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:43.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.320+0000 7f913ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f914006d950 0x7f91400b0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36022/0 (socket says 172.21.15.55:36022) 2023-10-06T03:33:43.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.320+0000 7f913ffff700 1 -- 172.21.15.55:0/1835771069 learned_addr learned my addr 172.21.15.55:0/1835771069 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:43.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.321+0000 7f9144f9b700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f914006a0a0 0x7f91400affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:43.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.321+0000 7f914579c700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f914006dd20 0x7f9140167530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:43.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.321+0000 7f913ffff700 1 -- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f914006a0a0 msgr2=0x7f91400affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:43.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.321+0000 7f913ffff700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f914006a0a0 0x7f91400affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:43.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.321+0000 7f913ffff700 1 -- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f914006dd20 msgr2=0x7f9140167530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:43.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.321+0000 7f913ffff700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f914006dd20 0x7f9140167530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:43.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.321+0000 7f913ffff700 1 -- 172.21.15.55:0/1835771069 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9138009ae0 con 0x7f914006d950 2023-10-06T03:33:43.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.321+0000 7f9144f9b700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f914006a0a0 0x7f91400affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:43.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.321+0000 7f914579c700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f914006dd20 0x7f9140167530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:43.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.321+0000 7f913ffff700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f914006d950 0x7f91400b0500 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f912c00e930 tx=0x7f912c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:43.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.321+0000 7f913cff9700 1 -- 172.21.15.55:0/1835771069 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c00eed0 con 0x7f914006d950 2023-10-06T03:33:43.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.321+0000 7f913cff9700 1 -- 172.21.15.55:0/1835771069 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c00c850 con 0x7f914006d950 2023-10-06T03:33:43.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.321+0000 7f913cff9700 1 -- 172.21.15.55:0/1835771069 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c00cbf0 con 0x7f914006d950 2023-10-06T03:33:43.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.321+0000 7f9145f9d700 1 -- 172.21.15.55:0/1835771069 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91400b11d0 con 0x7f914006d950 2023-10-06T03:33:43.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.322+0000 7f9145f9d700 1 -- 172.21.15.55:0/1835771069 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914013bfb0 con 0x7f914006d950 2023-10-06T03:33:43.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.322+0000 7f9145f9d700 1 -- 172.21.15.55:0/1835771069 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914013c230 con 0x7f914006d950 2023-10-06T03:33:43.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.323+0000 7f913cff9700 1 -- 172.21.15.55:0/1835771069 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f912c013470 con 0x7f914006d950 2023-10-06T03:33:43.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.324+0000 7f913cff9700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9124066f30 0x7f91240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:43.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.324+0000 7f913cff9700 1 -- 172.21.15.55:0/1835771069 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f912c084960 con 0x7f914006d950 2023-10-06T03:33:43.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.325+0000 7f9144f9b700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9124066f30 0x7f91240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:43.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.325+0000 7f9144f9b700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9124066f30 0x7f91240693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f91340011e0 tx=0x7f9134009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:43.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.329+0000 7f913cff9700 1 -- 172.21.15.55:0/1835771069 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f912c0519d0 con 0x7f914006d950 2023-10-06T03:33:43.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.575+0000 7f9145f9d700 1 -- 172.21.15.55:0/1835771069 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f914004c3c0 con 0x7f914006d950 2023-10-06T03:33:43.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.576+0000 7f913cff9700 1 -- 172.21.15.55:0/1835771069 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f912c054ff0 con 0x7f914006d950 2023-10-06T03:33:43.578 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:33:43.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.578+0000 7f9145f9d700 1 -- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9124066f30 msgr2=0x7f91240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:43.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.578+0000 7f9145f9d700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9124066f30 0x7f91240693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f91340011e0 tx=0x7f9134009040 comp rx=0 tx=0).stop 2023-10-06T03:33:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.578+0000 7f9145f9d700 1 -- 172.21.15.55:0/1835771069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f914006d950 msgr2=0x7f91400b0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.578+0000 7f9145f9d700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f914006d950 0x7f91400b0500 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f912c00e930 tx=0x7f912c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.578+0000 7f9145f9d700 1 -- 172.21.15.55:0/1835771069 shutdown_connections 2023-10-06T03:33:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.578+0000 7f9145f9d700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f914006dd20 0x7f9140167530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.578+0000 7f9145f9d700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9124066f30 0x7f91240693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.578+0000 7f9145f9d700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f914006a0a0 0x7f91400affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:43.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.578+0000 7f9145f9d700 1 --2- 172.21.15.55:0/1835771069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f914006d950 0x7f91400b0500 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:43.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.578+0000 7f9145f9d700 1 -- 172.21.15.55:0/1835771069 >> 172.21.15.55:0/1835771069 conn(0x7f91400cc1f0 msgr2=0x7f91400c2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:43.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.578+0000 7f9145f9d700 1 -- 172.21.15.55:0/1835771069 shutdown_connections 2023-10-06T03:33:43.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:43.578+0000 7f9145f9d700 1 -- 172.21.15.55:0/1835771069 wait complete. 2023-10-06T03:33:43.597 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T03:33:43.597 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:33:43.597 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi163.front.sepia.ceph.com 2023-10-06T03:33:43.597 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-06T03:33:43.597 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-06T03:33:43.597 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T03:33:43.598 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T03:33:43.598 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:33:43.598 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-06T03:33:43.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:43.598 DEBUG:teuthology.orchestra.run.smithi163:> ip addr 2023-10-06T03:33:43.637 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:33:43.637 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:33:43.637 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:33:43.637 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:43.638 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-06T03:33:43.638 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:43.638 INFO:teuthology.orchestra.run.smithi163.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:33:43.638 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:43.638 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f0 2023-10-06T03:33:43.638 INFO:teuthology.orchestra.run.smithi163.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:33:43.638 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:d9:38:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:43.638 INFO:teuthology.orchestra.run.smithi163.stdout: altname enp7s0f1 2023-10-06T03:33:43.638 INFO:teuthology.orchestra.run.smithi163.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:33:43.638 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5a brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:43.638 INFO:teuthology.orchestra.run.smithi163.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:33:43.638 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 0c:c4:7a:88:7d:5b brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:43.639 INFO:teuthology.orchestra.run.smithi163.stdout: inet 172.21.15.163/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T03:33:43.639 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft 42243sec preferred_lft 42243sec 2023-10-06T03:33:43.639 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::ec4:7aff:fe88:7d5b/64 scope link 2023-10-06T03:33:43.639 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:43.639 INFO:teuthology.orchestra.run.smithi163.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:33:43.639 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 32:6c:06:ef:f1:a6 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:33:43.639 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:33:43.639 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:43.639 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::20bd:49ff:fef6:dfcc/64 scope link 2023-10-06T03:33:43.639 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:43.639 INFO:teuthology.orchestra.run.smithi163.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:33:43.639 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether 32:6c:06:ef:f1:a6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T03:33:43.640 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::306c:6ff:feef:f1a6/64 scope link tentative 2023-10-06T03:33:43.640 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:43.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:43.641 DEBUG:teuthology.orchestra.run.smithi163:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:33:43.742 INFO:teuthology.orchestra.run.smithi163.stdout:1 2023-10-06T03:33:43.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:43.745 DEBUG:teuthology.orchestra.run.smithi163:> route 2023-10-06T03:33:43.783 INFO:teuthology.orchestra.run.smithi163.stdout:Kernel IP routing table 2023-10-06T03:33:43.784 INFO:teuthology.orchestra.run.smithi163.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:33:43.784 INFO:teuthology.orchestra.run.smithi163.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T03:33:43.784 INFO:teuthology.orchestra.run.smithi163.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T03:33:43.784 INFO:teuthology.orchestra.run.smithi163.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:33:43.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:43.785 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-06T03:33:43.786 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-06T03:33:43.786 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T03:33:43.786 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T03:33:43.786 DEBUG:teuthology.orchestra.run.smithi163:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T03:33:43.786 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-06T03:33:43.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:43.981 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-06T03:33:44.017 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T03:33:44.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:44.018 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list-id 2023-10-06T03:33:44.094 INFO:teuthology.orchestra.run.smithi163.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-06T03:33:44.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:44.095 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-06T03:33:44.095 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-06T03:33:44.095 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T03:33:44.095 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-06T03:33:44.095 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-06T03:33:44.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:44.260 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T03:33:44.321 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:33:44.321 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:33:44.321 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:33:44.321 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:44.322 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-06T03:33:44.322 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:44.322 INFO:teuthology.orchestra.run.smithi163.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:33:44.322 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether aa:67:71:9f:9f:b4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T03:33:44.322 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T03:33:44.322 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:44.322 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::a867:71ff:fe9f:9fb4/64 scope link 2023-10-06T03:33:44.322 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:44.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:44.324 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T03:33:44.385 INFO:teuthology.orchestra.run.smithi163.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:33:44.385 INFO:teuthology.orchestra.run.smithi163.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:33:44.385 INFO:teuthology.orchestra.run.smithi163.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:33:44.385 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:44.385 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 ::1/128 scope host 2023-10-06T03:33:44.385 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:44.385 INFO:teuthology.orchestra.run.smithi163.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:33:44.385 INFO:teuthology.orchestra.run.smithi163.stdout: link/ether aa:67:71:9f:9f:b4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T03:33:44.386 INFO:teuthology.orchestra.run.smithi163.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T03:33:44.386 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:44.386 INFO:teuthology.orchestra.run.smithi163.stdout: inet6 fe80::a867:71ff:fe9f:9fb4/64 scope link 2023-10-06T03:33:44.386 INFO:teuthology.orchestra.run.smithi163.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:33:44.387 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-06T03:33:44.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:44.388 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-06T03:33:44.388 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-06T03:33:44.388 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-06T03:33:44.388 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T03:33:44.388 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-06T03:33:44.388 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-06T03:33:44.388 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-06T03:33:44.388 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-06T03:33:44.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:44.614 DEBUG:teuthology.orchestra.run.smithi163:> (cd / && exec bash -c ' 2023-10-06T03:33:44.614 DEBUG:teuthology.orchestra.run.smithi163:> set -e 2023-10-06T03:33:44.614 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set brx.1 up 2023-10-06T03:33:44.614 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T03:33:44.614 DEBUG:teuthology.orchestra.run.smithi163:> ') 2023-10-06T03:33:44.772 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:33:44.772 DEBUG:teuthology.orchestra.run.smithi163:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-06T03:33:44.809 INFO:teuthology.orchestra.run.smithi163.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-06T03:33:44.809 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:33:44.809 DEBUG:teuthology.orchestra.run.smithi163:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-06T03:33:44.884 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T03:33:44.884 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:33:44.884 DEBUG:teuthology.orchestra.run.smithi163:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T03:33:45.018 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T03:33:45.018 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-06T03:33:45.018 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2023-10-06T03:33:45.034 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T03:33:45.035 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-06T03:33:45.035 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2023-10-06T03:33:45.035 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:33:45.035 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:33:45.055 INFO:teuthology.orchestra.run.smithi163.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-10-06T03:33:45.055 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-10-06T03:33:45.055 INFO:teuthology.orchestra.run.smithi163.stdout:invalid new device string format 2023-10-06T03:33:45.055 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:33:45.055 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:33:45.055 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T03:33:45.055 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T03:33:45.056 INFO:teuthology.orchestra.run.smithi163.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T03:33:45.059 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:33:45.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:45.059 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-06T03:33:45.120 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-06T03:33:45.140 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:45.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.340+0000 7f377e79d700 1 Processor -- start 2023-10-06T03:33:45.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.340+0000 7f377e79d700 1 -- start start 2023-10-06T03:33:45.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.341+0000 7f377e79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3778150180 0x7f3778157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:45.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.341+0000 7f377e79d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37780c7410 0x7f3778069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.341+0000 7f377e79d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377806e2a0 con 0x7f3778150550 2023-10-06T03:33:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.341+0000 7f377e79d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377806b2e0 con 0x7f37780c7410 2023-10-06T03:33:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.341+0000 7f377e79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37780c78e0 con 0x7f3778150180 2023-10-06T03:33:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.341+0000 7f377cf9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3778150550 0x7f37780c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59466/0 (socket says 172.21.15.55:59466) 2023-10-06T03:33:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.341+0000 7f377cf9a700 1 -- 172.21.15.55:0/1909942211 learned_addr learned my addr 172.21.15.55:0/1909942211 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.342+0000 7f377d79b700 1 --2- 172.21.15.55:0/1909942211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3778150180 0x7f3778157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.342+0000 7f377df9c700 1 --2- 172.21.15.55:0/1909942211 >> v2:172.21.15.163:3300/0 conn(0x7f37780c7410 0x7f3778069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.342+0000 7f376ffff700 1 -- 172.21.15.55:0/1909942211 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128678814 0 0) 0x7f377806e2a0 con 0x7f3778150550 2023-10-06T03:33:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.342+0000 7f376ffff700 1 -- 172.21.15.55:0/1909942211 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3764004680 con 0x7f3778150550 2023-10-06T03:33:45.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.342+0000 7f376ffff700 1 -- 172.21.15.55:0/1909942211 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3955695312 0 0) 0x7f3764004680 con 0x7f3778150550 2023-10-06T03:33:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.342+0000 7f377df9c700 1 -- 172.21.15.55:0/1909942211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3778150180 msgr2=0x7f3778157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.342+0000 7f377df9c700 1 --2- 172.21.15.55:0/1909942211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3778150180 0x7f3778157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.342+0000 7f377df9c700 1 -- 172.21.15.55:0/1909942211 >> v1:172.21.15.55:6789/0 conn(0x7f3778150550 legacy=0x7f37780c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.342+0000 7f377df9c700 1 -- 172.21.15.55:0/1909942211 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37780c7bc0 con 0x7f37780c7410 2023-10-06T03:33:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.342+0000 7f377d79b700 1 --2- 172.21.15.55:0/1909942211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3778150180 0x7f3778157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:45.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.342+0000 7f377df9c700 1 --2- 172.21.15.55:0/1909942211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37780c7410 0x7f3778069f50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3770009e60 tx=0x7f3770009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c68cb1f563bf2bf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:45.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.343+0000 7f376ffff700 1 -- 172.21.15.55:0/1909942211 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377000e070 con 0x7f37780c7410 2023-10-06T03:33:45.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.343+0000 7f376ffff700 1 -- 172.21.15.55:0/1909942211 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377000b7a0 con 0x7f37780c7410 2023-10-06T03:33:45.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.343+0000 7f376ffff700 1 -- 172.21.15.55:0/1909942211 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768001470 con 0x7f37780c7410 2023-10-06T03:33:45.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.343+0000 7f377e79d700 1 -- 172.21.15.55:0/1909942211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37780c7410 msgr2=0x7f3778069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:45.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.343+0000 7f377e79d700 1 --2- 172.21.15.55:0/1909942211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37780c7410 0x7f3778069f50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3770009e60 tx=0x7f3770009780 comp rx=0 tx=0).stop 2023-10-06T03:33:45.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.343+0000 7f377e79d700 1 -- 172.21.15.55:0/1909942211 shutdown_connections 2023-10-06T03:33:45.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.343+0000 7f377e79d700 1 --2- 172.21.15.55:0/1909942211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37780c7410 0x7f3778069f50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.343+0000 7f377e79d700 1 --2- 172.21.15.55:0/1909942211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3778150180 0x7f3778157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.343+0000 7f377e79d700 1 -- 172.21.15.55:0/1909942211 >> 172.21.15.55:0/1909942211 conn(0x7f37780cc1f0 msgr2=0x7f37780cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:45.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.344+0000 7f377e79d700 1 -- 172.21.15.55:0/1909942211 shutdown_connections 2023-10-06T03:33:45.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.344+0000 7f377e79d700 1 -- 172.21.15.55:0/1909942211 wait complete. 2023-10-06T03:33:45.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.345+0000 7f377e79d700 1 Processor -- start 2023-10-06T03:33:45.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.345+0000 7f377e79d700 1 -- start start 2023-10-06T03:33:45.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.346+0000 7f377e79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37780c7410 0x7f377816a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:45.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.346+0000 7f377e79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3778150180 0x7f3778154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:45.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.346+0000 7f377e79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3778150550 0x7f3778155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:45.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.346+0000 7f377e79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37780ae5f0 con 0x7f37780c7410 2023-10-06T03:33:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.346+0000 7f377e79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37780ad9e0 con 0x7f3778150550 2023-10-06T03:33:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.346+0000 7f377e79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37780adcc0 con 0x7f3778150180 2023-10-06T03:33:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.346+0000 7f377d79b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37780c7410 0x7f377816a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.346+0000 7f377d79b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37780c7410 0x7f377816a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36028/0 (socket says 172.21.15.55:36028) 2023-10-06T03:33:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.346+0000 7f377d79b700 1 -- 172.21.15.55:0/4107318195 learned_addr learned my addr 172.21.15.55:0/4107318195 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.346+0000 7f377cf9a700 1 --2- 172.21.15.55:0/4107318195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3778150180 0x7f3778154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.346+0000 7f377df9c700 1 --2- 172.21.15.55:0/4107318195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3778150550 0x7f3778155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.347+0000 7f377d79b700 1 -- 172.21.15.55:0/4107318195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3778150180 msgr2=0x7f3778154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.347+0000 7f377d79b700 1 --2- 172.21.15.55:0/4107318195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3778150180 0x7f3778154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.347+0000 7f377d79b700 1 -- 172.21.15.55:0/4107318195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3778150550 msgr2=0x7f3778155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:45.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.347+0000 7f377d79b700 1 --2- 172.21.15.55:0/4107318195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3778150550 0x7f3778155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.347+0000 7f377d79b700 1 -- 172.21.15.55:0/4107318195 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3774009420 con 0x7f37780c7410 2023-10-06T03:33:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.347+0000 7f377d79b700 1 --2- 172.21.15.55:0/4107318195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37780c7410 0x7f377816a0c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f377400ec80 tx=0x7f37740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.347+0000 7f376d7fa700 1 -- 172.21.15.55:0/4107318195 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377400c8f0 con 0x7f37780c7410 2023-10-06T03:33:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.347+0000 7f376d7fa700 1 -- 172.21.15.55:0/4107318195 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377400ca50 con 0x7f37780c7410 2023-10-06T03:33:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.347+0000 7f376d7fa700 1 -- 172.21.15.55:0/4107318195 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f377400cdd0 con 0x7f37780c7410 2023-10-06T03:33:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.347+0000 7f377e79d700 1 -- 172.21.15.55:0/4107318195 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3770009a20 con 0x7f37780c7410 2023-10-06T03:33:45.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.347+0000 7f377e79d700 1 -- 172.21.15.55:0/4107318195 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37780ae280 con 0x7f37780c7410 2023-10-06T03:33:45.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.348+0000 7f377e79d700 1 -- 172.21.15.55:0/4107318195 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377804c3c0 con 0x7f37780c7410 2023-10-06T03:33:45.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.349+0000 7f376d7fa700 1 -- 172.21.15.55:0/4107318195 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3774013680 con 0x7f37780c7410 2023-10-06T03:33:45.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.350+0000 7f376d7fa700 1 --2- 172.21.15.55:0/4107318195 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f375c066f30 0x7f375c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:45.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.350+0000 7f376d7fa700 1 -- 172.21.15.55:0/4107318195 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3774084a70 con 0x7f37780c7410 2023-10-06T03:33:45.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.351+0000 7f377cf9a700 1 --2- 172.21.15.55:0/4107318195 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f375c066f30 0x7f375c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:45.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.351+0000 7f377cf9a700 1 --2- 172.21.15.55:0/4107318195 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f375c066f30 0x7f375c0693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f37781493e0 tx=0x7f3768001970 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:45.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.354+0000 7f376d7fa700 1 -- 172.21.15.55:0/4107318195 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3774051ae0 con 0x7f37780c7410 2023-10-06T03:33:45.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.563+0000 7f377e79d700 1 -- 172.21.15.55:0/4107318195 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f377806ada0 con 0x7f37780c7410 2023-10-06T03:33:45.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.564+0000 7f376d7fa700 1 -- 172.21.15.55:0/4107318195 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f3774055100 con 0x7f37780c7410 2023-10-06T03:33:45.566 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:45.566 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:16.470817+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:45.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.566+0000 7f377e79d700 1 -- 172.21.15.55:0/4107318195 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f375c066f30 msgr2=0x7f375c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:45.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.566+0000 7f377e79d700 1 --2- 172.21.15.55:0/4107318195 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f375c066f30 0x7f375c0693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f37781493e0 tx=0x7f3768001970 comp rx=0 tx=0).stop 2023-10-06T03:33:45.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.566+0000 7f377e79d700 1 -- 172.21.15.55:0/4107318195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37780c7410 msgr2=0x7f377816a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:45.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.566+0000 7f377e79d700 1 --2- 172.21.15.55:0/4107318195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37780c7410 0x7f377816a0c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f377400ec80 tx=0x7f37740091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.566+0000 7f377e79d700 1 -- 172.21.15.55:0/4107318195 shutdown_connections 2023-10-06T03:33:45.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.566+0000 7f377e79d700 1 --2- 172.21.15.55:0/4107318195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3778150550 0x7f3778155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.566+0000 7f377e79d700 1 --2- 172.21.15.55:0/4107318195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37780c7410 0x7f377816a0c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.566+0000 7f377e79d700 1 --2- 172.21.15.55:0/4107318195 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f375c066f30 0x7f375c0693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.566+0000 7f377e79d700 1 --2- 172.21.15.55:0/4107318195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3778150180 0x7f3778154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.566+0000 7f377e79d700 1 -- 172.21.15.55:0/4107318195 >> 172.21.15.55:0/4107318195 conn(0x7f37780cc1f0 msgr2=0x7f3778159800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:45.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.567+0000 7f377e79d700 1 -- 172.21.15.55:0/4107318195 shutdown_connections 2023-10-06T03:33:45.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.567+0000 7f377e79d700 1 -- 172.21.15.55:0/4107318195 wait complete. 2023-10-06T03:33:45.585 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:33:13.543361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:33:14.341572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:33:45.586 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:33:45.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.775+0000 7fc12339d700 1 Processor -- start 2023-10-06T03:33:45.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.775+0000 7fc12339d700 1 -- start start 2023-10-06T03:33:45.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.777+0000 7fc12339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11c159a60 0x7fc11c0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:45.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.777+0000 7fc12339d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc11c0ccc00 0x7fc11c069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:45.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.777+0000 7fc12339d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c0c9cb0 con 0x7fc11c06a020 2023-10-06T03:33:45.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.777+0000 7fc12339d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11c06dfc0 con 0x7fc11c0ccc00 2023-10-06T03:33:45.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.777+0000 7fc12339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0c9f30 con 0x7fc11c159a60 2023-10-06T03:33:45.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.777+0000 7fc12239b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11c159a60 0x7fc11c0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:45.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.777+0000 7fc121b9a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc11c0ccc00 0x7fc11c069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:45.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.777+0000 7fc122b9c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fc11c06a020 0x7fc11c0cb230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:47810/0 (socket says 172.21.15.55:47810) 2023-10-06T03:33:45.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.777+0000 7fc122b9c700 1 -- 172.21.15.55:0/1888032620 learned_addr learned my addr 172.21.15.55:0/1888032620 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:45.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.777+0000 7fc12239b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11c159a60 0x7fc11c0c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:34626/0 (socket says 172.21.15.55:34626) 2023-10-06T03:33:45.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.778+0000 7fc121399700 1 -- 172.21.15.55:0/1888032620 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938182833 0 0) 0x7fc11c0c9cb0 con 0x7fc11c06a020 2023-10-06T03:33:45.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.778+0000 7fc121399700 1 -- 172.21.15.55:0/1888032620 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc108003570 con 0x7fc11c06a020 2023-10-06T03:33:45.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.778+0000 7fc121b9a700 1 -- 172.21.15.55:0/1888032620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11c159a60 msgr2=0x7fc11c0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:45.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.778+0000 7fc121b9a700 1 --2- 172.21.15.55:0/1888032620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11c159a60 0x7fc11c0c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.778+0000 7fc121b9a700 1 -- 172.21.15.55:0/1888032620 >> v1:172.21.15.163:6789/0 conn(0x7fc11c06a020 legacy=0x7fc11c0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:45.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.778+0000 7fc121b9a700 1 -- 172.21.15.55:0/1888032620 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c0ca470 con 0x7fc11c0ccc00 2023-10-06T03:33:45.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.778+0000 7fc12239b700 1 --2- 172.21.15.55:0/1888032620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11c159a60 0x7fc11c0c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:45.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.778+0000 7fc121b9a700 1 --2- 172.21.15.55:0/1888032620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc11c0ccc00 0x7fc11c069ae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0098e0 tx=0x7fc10c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a69d1a620dec7e6c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:45.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.778+0000 7fc121399700 1 -- 172.21.15.55:0/1888032620 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c011630 con 0x7fc11c0ccc00 2023-10-06T03:33:45.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.778+0000 7fc121399700 1 -- 172.21.15.55:0/1888032620 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c011790 con 0x7fc11c0ccc00 2023-10-06T03:33:45.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.778+0000 7fc121399700 1 -- 172.21.15.55:0/1888032620 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c011b30 con 0x7fc11c0ccc00 2023-10-06T03:33:45.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.779+0000 7fc12339d700 1 -- 172.21.15.55:0/1888032620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc11c0ccc00 msgr2=0x7fc11c069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:45.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.779+0000 7fc12339d700 1 --2- 172.21.15.55:0/1888032620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc11c0ccc00 0x7fc11c069ae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0098e0 tx=0x7fc10c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.779+0000 7fc12339d700 1 -- 172.21.15.55:0/1888032620 shutdown_connections 2023-10-06T03:33:45.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.779+0000 7fc12339d700 1 --2- 172.21.15.55:0/1888032620 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11c159a60 0x7fc11c0c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.779+0000 7fc12339d700 1 --2- 172.21.15.55:0/1888032620 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc11c0ccc00 0x7fc11c069ae0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.779+0000 7fc12339d700 1 -- 172.21.15.55:0/1888032620 >> 172.21.15.55:0/1888032620 conn(0x7fc11c06baa0 msgr2=0x7fc11c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:45.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.779+0000 7fc12339d700 1 -- 172.21.15.55:0/1888032620 shutdown_connections 2023-10-06T03:33:45.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.779+0000 7fc12339d700 1 -- 172.21.15.55:0/1888032620 wait complete. 2023-10-06T03:33:45.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.780+0000 7fc12339d700 1 Processor -- start 2023-10-06T03:33:45.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.781+0000 7fc12339d700 1 -- start start 2023-10-06T03:33:45.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.781+0000 7fc12339d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc11c06a020 0x7fc11c14c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:45.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.781+0000 7fc12339d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc11c0ccc00 0x7fc11c0cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:45.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.781+0000 7fc12339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11c159a60 0x7fc11c0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:45.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.781+0000 7fc12339d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0c6b00 con 0x7fc11c06a020 2023-10-06T03:33:45.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.781+0000 7fc12339d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0c6de0 con 0x7fc11c0ccc00 2023-10-06T03:33:45.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.781+0000 7fc12339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0c70c0 con 0x7fc11c159a60 2023-10-06T03:33:45.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.782+0000 7fc12239b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc11c06a020 0x7fc11c14c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:45.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.782+0000 7fc12239b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc11c06a020 0x7fc11c14c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36038/0 (socket says 172.21.15.55:36038) 2023-10-06T03:33:45.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.782+0000 7fc12239b700 1 -- 172.21.15.55:0/3374466097 learned_addr learned my addr 172.21.15.55:0/3374466097 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:45.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.782+0000 7fc121b9a700 1 --2- 172.21.15.55:0/3374466097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc11c0ccc00 0x7fc11c0cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:45.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.782+0000 7fc122b9c700 1 --2- 172.21.15.55:0/3374466097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11c159a60 0x7fc11c0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:45.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.782+0000 7fc12239b700 1 -- 172.21.15.55:0/3374466097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11c159a60 msgr2=0x7fc11c0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:45.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.782+0000 7fc12239b700 1 --2- 172.21.15.55:0/3374466097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11c159a60 0x7fc11c0cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.782+0000 7fc12239b700 1 -- 172.21.15.55:0/3374466097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc11c0ccc00 msgr2=0x7fc11c0cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:45.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.782+0000 7fc12239b700 1 --2- 172.21.15.55:0/3374466097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc11c0ccc00 0x7fc11c0cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.782+0000 7fc12239b700 1 -- 172.21.15.55:0/3374466097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc118009420 con 0x7fc11c06a020 2023-10-06T03:33:45.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.783+0000 7fc12239b700 1 --2- 172.21.15.55:0/3374466097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc11c06a020 0x7fc11c14c110 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc11800ec80 tx=0x7fc1180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:45.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.783+0000 7fc1127fc700 1 -- 172.21.15.55:0/3374466097 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11800c850 con 0x7fc11c06a020 2023-10-06T03:33:45.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.783+0000 7fc1127fc700 1 -- 172.21.15.55:0/3374466097 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11800c9b0 con 0x7fc11c06a020 2023-10-06T03:33:45.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.783+0000 7fc12339d700 1 -- 172.21.15.55:0/3374466097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10c009420 con 0x7fc11c06a020 2023-10-06T03:33:45.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.783+0000 7fc1127fc700 1 -- 172.21.15.55:0/3374466097 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11800cd30 con 0x7fc11c06a020 2023-10-06T03:33:45.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.783+0000 7fc12339d700 1 -- 172.21.15.55:0/3374466097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c14c650 con 0x7fc11c06a020 2023-10-06T03:33:45.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.784+0000 7fc12339d700 1 -- 172.21.15.55:0/3374466097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c0c6c40 con 0x7fc11c06a020 2023-10-06T03:33:45.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.785+0000 7fc1127fc700 1 -- 172.21.15.55:0/3374466097 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc118013680 con 0x7fc11c06a020 2023-10-06T03:33:45.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.786+0000 7fc1127fc700 1 --2- 172.21.15.55:0/3374466097 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc100066f30 0x7fc1000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:45.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.786+0000 7fc1127fc700 1 -- 172.21.15.55:0/3374466097 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc1180848d0 con 0x7fc11c06a020 2023-10-06T03:33:45.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.786+0000 7fc121b9a700 1 --2- 172.21.15.55:0/3374466097 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc100066f30 0x7fc1000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:45.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.787+0000 7fc121b9a700 1 --2- 172.21.15.55:0/3374466097 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc100066f30 0x7fc1000693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0091e0 tx=0x7fc10c017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:45.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.790+0000 7fc1127fc700 1 -- 172.21.15.55:0/3374466097 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc11c0c6c40 con 0x7fc11c06a020 2023-10-06T03:33:45.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.990+0000 7fc12339d700 1 -- 172.21.15.55:0/3374466097 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc11c0c6f20 con 0x7fc11c06a020 2023-10-06T03:33:45.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.990+0000 7fc1127fc700 1 -- 172.21.15.55:0/3374466097 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc11c0c6f20 con 0x7fc11c06a020 2023-10-06T03:33:45.992 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:33:45.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.993+0000 7fc12339d700 1 -- 172.21.15.55:0/3374466097 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc100066f30 msgr2=0x7fc1000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:45.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.993+0000 7fc12339d700 1 --2- 172.21.15.55:0/3374466097 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc100066f30 0x7fc1000693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc10c0091e0 tx=0x7fc10c017000 comp rx=0 tx=0).stop 2023-10-06T03:33:45.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.993+0000 7fc12339d700 1 -- 172.21.15.55:0/3374466097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc11c06a020 msgr2=0x7fc11c14c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:45.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.993+0000 7fc12339d700 1 --2- 172.21.15.55:0/3374466097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc11c06a020 0x7fc11c14c110 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc11800ec80 tx=0x7fc1180091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.993+0000 7fc12339d700 1 -- 172.21.15.55:0/3374466097 shutdown_connections 2023-10-06T03:33:45.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.993+0000 7fc12339d700 1 --2- 172.21.15.55:0/3374466097 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc100066f30 0x7fc1000693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.993+0000 7fc12339d700 1 --2- 172.21.15.55:0/3374466097 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc11c159a60 0x7fc11c0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.993+0000 7fc12339d700 1 --2- 172.21.15.55:0/3374466097 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc11c06a020 0x7fc11c14c110 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.993+0000 7fc12339d700 1 --2- 172.21.15.55:0/3374466097 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc11c0ccc00 0x7fc11c0cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:45.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.993+0000 7fc12339d700 1 -- 172.21.15.55:0/3374466097 >> 172.21.15.55:0/3374466097 conn(0x7fc11c06baa0 msgr2=0x7fc11c1577e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:45.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.993+0000 7fc12339d700 1 -- 172.21.15.55:0/3374466097 shutdown_connections 2023-10-06T03:33:45.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:45.993+0000 7fc12339d700 1 -- 172.21.15.55:0/3374466097 wait complete. 2023-10-06T03:33:46.012 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:33:46.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.203+0000 7fc261d9d700 1 Processor -- start 2023-10-06T03:33:46.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.203+0000 7fc261d9d700 1 -- start start 2023-10-06T03:33:46.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.205+0000 7fc261d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc25c0cb230 0x7fc25c0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:46.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.205+0000 7fc261d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25c0c9f30 con 0x7fc25c159a60 2023-10-06T03:33:46.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.205+0000 7fc261d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc25c0ca470 con 0x7fc25c0ccc00 2023-10-06T03:33:46.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.205+0000 7fc261d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c0cbbd0 con 0x7fc25c0cb230 2023-10-06T03:33:46.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.205+0000 7fc260d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc25c159a60 0x7fc25c0c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59480/0 (socket says 172.21.15.55:59480) 2023-10-06T03:33:46.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.205+0000 7fc260d9b700 1 -- 172.21.15.55:0/3513316323 learned_addr learned my addr 172.21.15.55:0/3513316323 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:46.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.205+0000 7fc26159c700 1 --2- 172.21.15.55:0/3513316323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc25c0cb230 0x7fc25c0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:46.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.205+0000 7fc25b7fe700 1 -- 172.21.15.55:0/3513316323 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861950088 0 0) 0x7fc25c0c9f30 con 0x7fc25c159a60 2023-10-06T03:33:46.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.206+0000 7fc25b7fe700 1 -- 172.21.15.55:0/3513316323 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc24c003570 con 0x7fc25c159a60 2023-10-06T03:33:46.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.206+0000 7fc25b7fe700 1 -- 172.21.15.55:0/3513316323 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824780075 0 0) 0x7fc25c0ca470 con 0x7fc25c0ccc00 2023-10-06T03:33:46.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.206+0000 7fc25b7fe700 1 -- 172.21.15.55:0/3513316323 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc25c0c9f30 con 0x7fc25c0ccc00 2023-10-06T03:33:46.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.206+0000 7fc26159c700 1 -- 172.21.15.55:0/3513316323 >> v1:172.21.15.163:6789/0 conn(0x7fc25c0ccc00 legacy=0x7fc25c069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:46.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.206+0000 7fc25b7fe700 1 -- 172.21.15.55:0/3513316323 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2033832091 0 0) 0x7fc24c003570 con 0x7fc25c159a60 2023-10-06T03:33:46.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.206+0000 7fc26159c700 1 -- 172.21.15.55:0/3513316323 >> v1:172.21.15.55:6789/0 conn(0x7fc25c159a60 legacy=0x7fc25c0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:46.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.206+0000 7fc26159c700 1 -- 172.21.15.55:0/3513316323 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc25c0ca750 con 0x7fc25c0cb230 2023-10-06T03:33:46.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.206+0000 7fc26159c700 1 --2- 172.21.15.55:0/3513316323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc25c0cb230 0x7fc25c0cb690 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc254009e90 tx=0x7fc254009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37af3acf43bfeae6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:46.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.206+0000 7fc25b7fe700 1 -- 172.21.15.55:0/3513316323 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25400e070 con 0x7fc25c0cb230 2023-10-06T03:33:46.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.206+0000 7fc25b7fe700 1 -- 172.21.15.55:0/3513316323 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25400b900 con 0x7fc25c0cb230 2023-10-06T03:33:46.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.206+0000 7fc25b7fe700 1 -- 172.21.15.55:0/3513316323 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc250001470 con 0x7fc25c0cb230 2023-10-06T03:33:46.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.207+0000 7fc261d9d700 1 -- 172.21.15.55:0/3513316323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc25c0cb230 msgr2=0x7fc25c0cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:46.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.207+0000 7fc261d9d700 1 --2- 172.21.15.55:0/3513316323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc25c0cb230 0x7fc25c0cb690 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc254009e90 tx=0x7fc254009780 comp rx=0 tx=0).stop 2023-10-06T03:33:46.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.207+0000 7fc261d9d700 1 -- 172.21.15.55:0/3513316323 shutdown_connections 2023-10-06T03:33:46.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.207+0000 7fc261d9d700 1 --2- 172.21.15.55:0/3513316323 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc25c0cb230 0x7fc25c0cb690 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.207+0000 7fc261d9d700 1 -- 172.21.15.55:0/3513316323 >> 172.21.15.55:0/3513316323 conn(0x7fc25c06baa0 msgr2=0x7fc25c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:46.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.207+0000 7fc261d9d700 1 -- 172.21.15.55:0/3513316323 shutdown_connections 2023-10-06T03:33:46.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.207+0000 7fc261d9d700 1 -- 172.21.15.55:0/3513316323 wait complete. 2023-10-06T03:33:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.208+0000 7fc261d9d700 1 Processor -- start 2023-10-06T03:33:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.209+0000 7fc261d9d700 1 -- start start 2023-10-06T03:33:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.209+0000 7fc261d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc25c0cb230 0x7fc25c155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:46.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.209+0000 7fc261d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc25c0ccc00 0x7fc25c0ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:46.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc261d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc25c159a60 0x7fc25c1522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:46.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc261d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c15e5d0 con 0x7fc25c0ccc00 2023-10-06T03:33:46.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc261d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c155b60 con 0x7fc25c0cb230 2023-10-06T03:33:46.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc261d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc25c155e40 con 0x7fc25c159a60 2023-10-06T03:33:46.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc25bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc25c0ccc00 0x7fc25c0ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:46.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc260d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc25c0cb230 0x7fc25c155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:46.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc260d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc25c0cb230 0x7fc25c155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:57582/0 (socket says 172.21.15.55:57582) 2023-10-06T03:33:46.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc260d9b700 1 -- 172.21.15.55:0/4084139996 learned_addr learned my addr 172.21.15.55:0/4084139996 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:46.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc25bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc25c0ccc00 0x7fc25c0ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36042/0 (socket says 172.21.15.55:36042) 2023-10-06T03:33:46.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc26159c700 1 --2- 172.21.15.55:0/4084139996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc25c159a60 0x7fc25c1522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:46.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc25bfff700 1 -- 172.21.15.55:0/4084139996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc25c159a60 msgr2=0x7fc25c1522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc25bfff700 1 --2- 172.21.15.55:0/4084139996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc25c159a60 0x7fc25c1522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc25bfff700 1 -- 172.21.15.55:0/4084139996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc25c0cb230 msgr2=0x7fc25c155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc25bfff700 1 --2- 172.21.15.55:0/4084139996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc25c0cb230 0x7fc25c155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc25bfff700 1 -- 172.21.15.55:0/4084139996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc254009a20 con 0x7fc25c0ccc00 2023-10-06T03:33:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.210+0000 7fc260d9b700 1 --2- 172.21.15.55:0/4084139996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc25c0cb230 0x7fc25c155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.211+0000 7fc25bfff700 1 --2- 172.21.15.55:0/4084139996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc25c0ccc00 0x7fc25c0ce8c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c560 tx=0x7fc248007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.211+0000 7fc258ff9700 1 -- 172.21.15.55:0/4084139996 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248011590 con 0x7fc25c0ccc00 2023-10-06T03:33:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.211+0000 7fc258ff9700 1 -- 172.21.15.55:0/4084139996 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2480116f0 con 0x7fc25c0ccc00 2023-10-06T03:33:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.211+0000 7fc258ff9700 1 -- 172.21.15.55:0/4084139996 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248011ab0 con 0x7fc25c0ccc00 2023-10-06T03:33:46.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.211+0000 7fc261d9d700 1 -- 172.21.15.55:0/4084139996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc25c156120 con 0x7fc25c0ccc00 2023-10-06T03:33:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.211+0000 7fc261d9d700 1 -- 172.21.15.55:0/4084139996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc25c0cee00 con 0x7fc25c0ccc00 2023-10-06T03:33:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.212+0000 7fc261d9d700 1 -- 172.21.15.55:0/4084139996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25c0cf020 con 0x7fc25c0ccc00 2023-10-06T03:33:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.213+0000 7fc258ff9700 1 -- 172.21.15.55:0/4084139996 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc24801c460 con 0x7fc25c0ccc00 2023-10-06T03:33:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.214+0000 7fc258ff9700 1 --2- 172.21.15.55:0/4084139996 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc240066f30 0x7fc2400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.214+0000 7fc258ff9700 1 -- 172.21.15.55:0/4084139996 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc248084270 con 0x7fc25c0ccc00 2023-10-06T03:33:46.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.215+0000 7fc260d9b700 1 --2- 172.21.15.55:0/4084139996 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc240066f30 0x7fc2400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:46.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.215+0000 7fc260d9b700 1 --2- 172.21.15.55:0/4084139996 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc240066f30 0x7fc2400693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc250007fd0 tx=0x7fc250001ab0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:46.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.218+0000 7fc258ff9700 1 -- 172.21.15.55:0/4084139996 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2480512e0 con 0x7fc25c0ccc00 2023-10-06T03:33:46.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.423+0000 7fc261d9d700 1 -- 172.21.15.55:0/4084139996 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc25c15e710 con 0x7fc25c0ccc00 2023-10-06T03:33:46.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.424+0000 7fc258ff9700 1 -- 172.21.15.55:0/4084139996 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc248054900 con 0x7fc25c0ccc00 2023-10-06T03:33:46.425 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:33:46.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.426+0000 7fc261d9d700 1 -- 172.21.15.55:0/4084139996 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc240066f30 msgr2=0x7fc2400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.426+0000 7fc261d9d700 1 --2- 172.21.15.55:0/4084139996 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc240066f30 0x7fc2400693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc250007fd0 tx=0x7fc250001ab0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.426+0000 7fc261d9d700 1 -- 172.21.15.55:0/4084139996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc25c0ccc00 msgr2=0x7fc25c0ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.426+0000 7fc261d9d700 1 --2- 172.21.15.55:0/4084139996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc25c0ccc00 0x7fc25c0ce8c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c560 tx=0x7fc248007d50 comp rx=0 tx=0).stop 2023-10-06T03:33:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.426+0000 7fc261d9d700 1 -- 172.21.15.55:0/4084139996 shutdown_connections 2023-10-06T03:33:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.426+0000 7fc261d9d700 1 --2- 172.21.15.55:0/4084139996 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc240066f30 0x7fc2400693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.426+0000 7fc261d9d700 1 --2- 172.21.15.55:0/4084139996 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc25c159a60 0x7fc25c1522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.426+0000 7fc261d9d700 1 --2- 172.21.15.55:0/4084139996 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc25c0cb230 0x7fc25c155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.426+0000 7fc261d9d700 1 --2- 172.21.15.55:0/4084139996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc25c0ccc00 0x7fc25c0ce8c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.426+0000 7fc261d9d700 1 -- 172.21.15.55:0/4084139996 >> 172.21.15.55:0/4084139996 conn(0x7fc25c06baa0 msgr2=0x7fc25c06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:46.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.426+0000 7fc261d9d700 1 -- 172.21.15.55:0/4084139996 shutdown_connections 2023-10-06T03:33:46.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.426+0000 7fc261d9d700 1 -- 172.21.15.55:0/4084139996 wait complete. 2023-10-06T03:33:46.446 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:33:46.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.634+0000 7f343319d700 1 Processor -- start 2023-10-06T03:33:46.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.634+0000 7f343319d700 1 -- start start 2023-10-06T03:33:46.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.636+0000 7f343319d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f342c06dc00 0x7f342c1627e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:46.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.636+0000 7f343319d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c0c6820 con 0x7f342c1407e0 2023-10-06T03:33:46.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.636+0000 7f343319d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c0c6de0 con 0x7f342c140bb0 2023-10-06T03:33:46.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.636+0000 7f343319d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c141270 con 0x7f342c06dc00 2023-10-06T03:33:46.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.636+0000 7f343219b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f342c1407e0 0x7f342c06a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59492/0 (socket says 172.21.15.55:59492) 2023-10-06T03:33:46.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.636+0000 7f343219b700 1 -- 172.21.15.55:0/3808258691 learned_addr learned my addr 172.21.15.55:0/3808258691 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:46.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.636+0000 7f343299c700 1 --2- 172.21.15.55:0/3808258691 >> v2:172.21.15.55:3300/0 conn(0x7f342c06dc00 0x7f342c1627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:46.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.637+0000 7f343299c700 1 -- 172.21.15.55:0/3808258691 >> v1:172.21.15.163:6789/0 conn(0x7f342c140bb0 legacy=0x7f342c06d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:46.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.637+0000 7f343299c700 1 -- 172.21.15.55:0/3808258691 >> v1:172.21.15.55:6789/0 conn(0x7f342c1407e0 legacy=0x7f342c06a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:46.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.637+0000 7f343299c700 1 -- 172.21.15.55:0/3808258691 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c06acf0 con 0x7f342c06dc00 2023-10-06T03:33:46.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.637+0000 7f343299c700 1 --2- 172.21.15.55:0/3808258691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f342c06dc00 0x7f342c1627e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3424009e60 tx=0x7f3424009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da078f3ae483293 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:46.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.638+0000 7f3431199700 1 -- 172.21.15.55:0/3808258691 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342400e070 con 0x7f342c06dc00 2023-10-06T03:33:46.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.638+0000 7f3431199700 1 -- 172.21.15.55:0/3808258691 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342400bc30 con 0x7f342c06dc00 2023-10-06T03:33:46.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.638+0000 7f3431199700 1 -- 172.21.15.55:0/3808258691 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424012430 con 0x7f342c06dc00 2023-10-06T03:33:46.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.638+0000 7f343319d700 1 -- 172.21.15.55:0/3808258691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f342c06dc00 msgr2=0x7f342c1627e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:46.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.638+0000 7f343319d700 1 --2- 172.21.15.55:0/3808258691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f342c06dc00 0x7f342c1627e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3424009e60 tx=0x7f3424009780 comp rx=0 tx=0).stop 2023-10-06T03:33:46.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.638+0000 7f343319d700 1 -- 172.21.15.55:0/3808258691 shutdown_connections 2023-10-06T03:33:46.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.638+0000 7f343319d700 1 --2- 172.21.15.55:0/3808258691 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f342c06dc00 0x7f342c1627e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.638+0000 7f343319d700 1 -- 172.21.15.55:0/3808258691 >> 172.21.15.55:0/3808258691 conn(0x7f342c0cc1e0 msgr2=0x7f342c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:46.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.639+0000 7f343319d700 1 -- 172.21.15.55:0/3808258691 shutdown_connections 2023-10-06T03:33:46.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.639+0000 7f343319d700 1 -- 172.21.15.55:0/3808258691 wait complete. 2023-10-06T03:33:46.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.640+0000 7f343319d700 1 Processor -- start 2023-10-06T03:33:46.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.640+0000 7f343319d700 1 -- start start 2023-10-06T03:33:46.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.641+0000 7f343319d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f342c06dc00 0x7f342c132f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:46.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.641+0000 7f343319d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f342c1407e0 0x7f342c1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:46.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.641+0000 7f343319d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f342c140bb0 0x7f342c159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:46.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.641+0000 7f343319d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c15a9a0 con 0x7f342c140bb0 2023-10-06T03:33:46.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.641+0000 7f343319d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c159d90 con 0x7f342c1407e0 2023-10-06T03:33:46.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.641+0000 7f343319d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c15a070 con 0x7f342c06dc00 2023-10-06T03:33:46.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.641+0000 7f343299c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f342c140bb0 0x7f342c159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:46.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.641+0000 7f343299c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f342c140bb0 0x7f342c159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36066/0 (socket says 172.21.15.55:36066) 2023-10-06T03:33:46.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.641+0000 7f343299c700 1 -- 172.21.15.55:0/350995951 learned_addr learned my addr 172.21.15.55:0/350995951 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:46.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.642+0000 7f343219b700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f342c06dc00 0x7f342c132f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:46.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.642+0000 7f343299c700 1 -- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f342c06dc00 msgr2=0x7f342c132f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:46.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.642+0000 7f343199a700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f342c1407e0 0x7f342c1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:46.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.642+0000 7f343299c700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f342c06dc00 0x7f342c132f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.642+0000 7f343299c700 1 -- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f342c1407e0 msgr2=0x7f342c1591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:46.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.642+0000 7f343299c700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f342c1407e0 0x7f342c1591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.642+0000 7f343299c700 1 -- 172.21.15.55:0/350995951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3424009a20 con 0x7f342c140bb0 2023-10-06T03:33:46.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.642+0000 7f343219b700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f342c06dc00 0x7f342c132f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:46.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.642+0000 7f343199a700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f342c1407e0 0x7f342c1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:46.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.642+0000 7f343299c700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f342c140bb0 0x7f342c159710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f34240097c0 tx=0x7f3424019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:46.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.642+0000 7f34227fc700 1 -- 172.21.15.55:0/350995951 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342401e630 con 0x7f342c140bb0 2023-10-06T03:33:46.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.642+0000 7f34227fc700 1 -- 172.21.15.55:0/350995951 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342401e790 con 0x7f342c140bb0 2023-10-06T03:33:46.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.642+0000 7f343319d700 1 -- 172.21.15.55:0/350995951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c15a350 con 0x7f342c140bb0 2023-10-06T03:33:46.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.642+0000 7f34227fc700 1 -- 172.21.15.55:0/350995951 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342401eb50 con 0x7f342c140bb0 2023-10-06T03:33:46.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.643+0000 7f343319d700 1 -- 172.21.15.55:0/350995951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c16da20 con 0x7f342c140bb0 2023-10-06T03:33:46.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.644+0000 7f343319d700 1 -- 172.21.15.55:0/350995951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c15ab10 con 0x7f342c140bb0 2023-10-06T03:33:46.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.644+0000 7f34227fc700 1 -- 172.21.15.55:0/350995951 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3424024070 con 0x7f342c140bb0 2023-10-06T03:33:46.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.646+0000 7f34227fc700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3410066f30 0x7f34100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:46.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.646+0000 7f34227fc700 1 -- 172.21.15.55:0/350995951 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f34240870a0 con 0x7f342c140bb0 2023-10-06T03:33:46.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.646+0000 7f343219b700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3410066f30 0x7f34100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:46.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.647+0000 7f343219b700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3410066f30 0x7f34100693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f342c0aeb50 tx=0x7f3428009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:46.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.650+0000 7f34227fc700 1 -- 172.21.15.55:0/350995951 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3424054110 con 0x7f342c140bb0 2023-10-06T03:33:46.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.849+0000 7f343319d700 1 -- 172.21.15.55:0/350995951 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f342c06afd0 con 0x7f342c140bb0 2023-10-06T03:33:46.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.850+0000 7f34227fc700 1 -- 172.21.15.55:0/350995951 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3424057730 con 0x7f342c140bb0 2023-10-06T03:33:46.851 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T03:33:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.852+0000 7f343319d700 1 -- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3410066f30 msgr2=0x7f34100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:46.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.852+0000 7f343319d700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3410066f30 0x7f34100693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f342c0aeb50 tx=0x7f3428009040 comp rx=0 tx=0).stop 2023-10-06T03:33:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.852+0000 7f343319d700 1 -- 172.21.15.55:0/350995951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f342c140bb0 msgr2=0x7f342c159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.852+0000 7f343319d700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f342c140bb0 0x7f342c159710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f34240097c0 tx=0x7f3424019000 comp rx=0 tx=0).stop 2023-10-06T03:33:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.852+0000 7f343319d700 1 -- 172.21.15.55:0/350995951 shutdown_connections 2023-10-06T03:33:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.852+0000 7f343319d700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f342c140bb0 0x7f342c159710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.852+0000 7f343319d700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3410066f30 0x7f34100693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.852+0000 7f343319d700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f342c06dc00 0x7f342c132f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.852+0000 7f343319d700 1 --2- 172.21.15.55:0/350995951 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f342c1407e0 0x7f342c1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.852+0000 7f343319d700 1 -- 172.21.15.55:0/350995951 >> 172.21.15.55:0/350995951 conn(0x7f342c0cc1e0 msgr2=0x7f342c0c1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.852+0000 7f343319d700 1 -- 172.21.15.55:0/350995951 shutdown_connections 2023-10-06T03:33:46.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:46.852+0000 7f343319d700 1 -- 172.21.15.55:0/350995951 wait complete. 2023-10-06T03:33:46.871 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:33:47.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.060+0000 7f7f2919d700 1 Processor -- start 2023-10-06T03:33:47.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.060+0000 7f7f2919d700 1 -- start start 2023-10-06T03:33:47.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.061+0000 7f7f2919d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7f240cb1a0 0x7f7f2415a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.061+0000 7f7f2919d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f240ae0d0 0x7f7f240ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.061+0000 7f7f2919d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2406ebd0 con 0x7f7f2415a8a0 2023-10-06T03:33:47.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.061+0000 7f7f2919d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f2415b2e0 con 0x7f7f240cb1a0 2023-10-06T03:33:47.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.061+0000 7f7f2919d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240aea30 con 0x7f7f240ae0d0 2023-10-06T03:33:47.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.061+0000 7f7f23fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7f240cb1a0 0x7f7f2415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.061+0000 7f7f23fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7f240cb1a0 0x7f7f2415a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36080/0 (socket says 172.21.15.55:36080) 2023-10-06T03:33:47.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.061+0000 7f7f23fff700 1 -- 172.21.15.55:0/868015369 learned_addr learned my addr 172.21.15.55:0/868015369 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:47.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.061+0000 7f7f237fe700 1 --1- 172.21.15.55:0/868015369 >> v1:172.21.15.55:6789/0 conn(0x7f7f2415a8a0 0x7f7f2406f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59494/0 (socket says 172.21.15.55:59494) 2023-10-06T03:33:47.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.062+0000 7f7f2899c700 1 --2- 172.21.15.55:0/868015369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f240ae0d0 0x7f7f240ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.062+0000 7f7f23fff700 1 -- 172.21.15.55:0/868015369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f240ae0d0 msgr2=0x7f7f240ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:47.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.062+0000 7f7f23fff700 1 --2- 172.21.15.55:0/868015369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f240ae0d0 0x7f7f240ae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.062+0000 7f7f23fff700 1 -- 172.21.15.55:0/868015369 >> v1:172.21.15.55:6789/0 conn(0x7f7f2415a8a0 legacy=0x7f7f2406f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.062+0000 7f7f23fff700 1 -- 172.21.15.55:0/868015369 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2415ad20 con 0x7f7f240cb1a0 2023-10-06T03:33:47.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.062+0000 7f7f23fff700 1 --2- 172.21.15.55:0/868015369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f240cb1a0 0x7f7f2415a360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7f140099b0 tx=0x7f7f14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d83b54ece172db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:47.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.062+0000 7f7f22ffd700 1 -- 172.21.15.55:0/868015369 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f14009ea0 con 0x7f7f240cb1a0 2023-10-06T03:33:47.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.063+0000 7f7f22ffd700 1 -- 172.21.15.55:0/868015369 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1400b8f0 con 0x7f7f240cb1a0 2023-10-06T03:33:47.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.063+0000 7f7f22ffd700 1 -- 172.21.15.55:0/868015369 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400bc90 con 0x7f7f240cb1a0 2023-10-06T03:33:47.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.063+0000 7f7f2919d700 1 -- 172.21.15.55:0/868015369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f240cb1a0 msgr2=0x7f7f2415a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.063+0000 7f7f2919d700 1 --2- 172.21.15.55:0/868015369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f240cb1a0 0x7f7f2415a360 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7f140099b0 tx=0x7f7f14009250 comp rx=0 tx=0).stop 2023-10-06T03:33:47.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.063+0000 7f7f2919d700 1 -- 172.21.15.55:0/868015369 shutdown_connections 2023-10-06T03:33:47.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.063+0000 7f7f2919d700 1 --2- 172.21.15.55:0/868015369 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f240ae0d0 0x7f7f240ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.063+0000 7f7f2919d700 1 --2- 172.21.15.55:0/868015369 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f240cb1a0 0x7f7f2415a360 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.063+0000 7f7f2919d700 1 -- 172.21.15.55:0/868015369 >> 172.21.15.55:0/868015369 conn(0x7f7f240cc1e0 msgr2=0x7f7f240cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:47.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.063+0000 7f7f2919d700 1 -- 172.21.15.55:0/868015369 shutdown_connections 2023-10-06T03:33:47.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.063+0000 7f7f2919d700 1 -- 172.21.15.55:0/868015369 wait complete. 2023-10-06T03:33:47.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.065+0000 7f7f2919d700 1 Processor -- start 2023-10-06T03:33:47.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.065+0000 7f7f2919d700 1 -- start start 2023-10-06T03:33:47.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f2919d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f240ae0d0 0x7f7f2414c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f2919d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f240cb1a0 0x7f7f240c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f2919d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f240c4180 0x7f7f24153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f2919d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24156d90 con 0x7f7f240c4180 2023-10-06T03:33:47.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f2919d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f241572e0 con 0x7f7f240ae0d0 2023-10-06T03:33:47.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f2919d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240c4760 con 0x7f7f240cb1a0 2023-10-06T03:33:47.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f2899c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f240c4180 0x7f7f24153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f2899c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f240c4180 0x7f7f24153160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36092/0 (socket says 172.21.15.55:36092) 2023-10-06T03:33:47.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f2899c700 1 -- 172.21.15.55:0/2115295339 learned_addr learned my addr 172.21.15.55:0/2115295339 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:47.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f237fe700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f240cb1a0 0x7f7f240c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f23fff700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f240ae0d0 0x7f7f2414c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f2899c700 1 -- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f240cb1a0 msgr2=0x7f7f240c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f2899c700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f240cb1a0 0x7f7f240c3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f2899c700 1 -- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f240ae0d0 msgr2=0x7f7f2414c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f2899c700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f240ae0d0 0x7f7f2414c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f2899c700 1 -- 172.21.15.55:0/2115295339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f140094f0 con 0x7f7f240c4180 2023-10-06T03:33:47.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f237fe700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f240cb1a0 0x7f7f240c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:47.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.066+0000 7f7f23fff700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f240ae0d0 0x7f7f2414c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:47.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.067+0000 7f7f2899c700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f240c4180 0x7f7f24153160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009e90 tx=0x7f7f1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:47.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.067+0000 7f7f0bfff700 1 -- 172.21.15.55:0/2115295339 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00cc20 con 0x7f7f240c4180 2023-10-06T03:33:47.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.067+0000 7f7f0bfff700 1 -- 172.21.15.55:0/2115295339 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c00f040 con 0x7f7f240c4180 2023-10-06T03:33:47.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.067+0000 7f7f0bfff700 1 -- 172.21.15.55:0/2115295339 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c014480 con 0x7f7f240c4180 2023-10-06T03:33:47.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.067+0000 7f7f2919d700 1 -- 172.21.15.55:0/2115295339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f240c4a40 con 0x7f7f240c4180 2023-10-06T03:33:47.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.067+0000 7f7f2919d700 1 -- 172.21.15.55:0/2115295339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f24155e70 con 0x7f7f240c4180 2023-10-06T03:33:47.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.068+0000 7f7f2919d700 1 -- 172.21.15.55:0/2115295339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f240c4c60 con 0x7f7f240c4180 2023-10-06T03:33:47.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.069+0000 7f7f0bfff700 1 -- 172.21.15.55:0/2115295339 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f1c014830 con 0x7f7f240c4180 2023-10-06T03:33:47.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.070+0000 7f7f0bfff700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7f10066f30 0x7f7f100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.070+0000 7f7f0bfff700 1 -- 172.21.15.55:0/2115295339 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7f1c085150 con 0x7f7f240c4180 2023-10-06T03:33:47.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.070+0000 7f7f23fff700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7f10066f30 0x7f7f100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.071+0000 7f7f23fff700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7f10066f30 0x7f7f100693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7f140099b0 tx=0x7f7f14017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:47.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.074+0000 7f7f0bfff700 1 -- 172.21.15.55:0/2115295339 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f1c055c50 con 0x7f7f240c4180 2023-10-06T03:33:47.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.277+0000 7f7f2919d700 1 -- 172.21.15.55:0/2115295339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f2415b000 con 0x7f7f240c4180 2023-10-06T03:33:47.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.277+0000 7f7f0bfff700 1 -- 172.21.15.55:0/2115295339 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7f1c0557e0 con 0x7f7f240c4180 2023-10-06T03:33:47.279 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T03:33:47.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.280+0000 7f7f2919d700 1 -- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7f10066f30 msgr2=0x7f7f100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.280+0000 7f7f2919d700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7f10066f30 0x7f7f100693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7f140099b0 tx=0x7f7f14017000 comp rx=0 tx=0).stop 2023-10-06T03:33:47.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.280+0000 7f7f2919d700 1 -- 172.21.15.55:0/2115295339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f240c4180 msgr2=0x7f7f24153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.280+0000 7f7f2919d700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f240c4180 0x7f7f24153160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009e90 tx=0x7f7f1c00b040 comp rx=0 tx=0).stop 2023-10-06T03:33:47.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.280+0000 7f7f2919d700 1 -- 172.21.15.55:0/2115295339 shutdown_connections 2023-10-06T03:33:47.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.280+0000 7f7f2919d700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f240c4180 0x7f7f24153160 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.280+0000 7f7f2919d700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f240ae0d0 0x7f7f2414c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.280+0000 7f7f2919d700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7f10066f30 0x7f7f100693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.280+0000 7f7f2919d700 1 --2- 172.21.15.55:0/2115295339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f240cb1a0 0x7f7f240c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.280+0000 7f7f2919d700 1 -- 172.21.15.55:0/2115295339 >> 172.21.15.55:0/2115295339 conn(0x7f7f240cc1e0 msgr2=0x7f7f240c7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:47.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.280+0000 7f7f2919d700 1 -- 172.21.15.55:0/2115295339 shutdown_connections 2023-10-06T03:33:47.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.280+0000 7f7f2919d700 1 -- 172.21.15.55:0/2115295339 wait complete. 2023-10-06T03:33:47.298 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:47.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.484+0000 7fefc6b9d700 1 Processor -- start 2023-10-06T03:33:47.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.484+0000 7fefc6b9d700 1 -- start start 2023-10-06T03:33:47.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.486+0000 7fefc6b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fefc01597f0 0x7fefc00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.486+0000 7fefc6b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc00b11d0 con 0x7fefc0159420 2023-10-06T03:33:47.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.486+0000 7fefc6b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefc00b14b0 con 0x7fefc015d230 2023-10-06T03:33:47.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.486+0000 7fefc6b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc00b1790 con 0x7fefc01597f0 2023-10-06T03:33:47.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.486+0000 7fefc5b9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fefc0159420 0x7fefc00b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59502/0 (socket says 172.21.15.55:59502) 2023-10-06T03:33:47.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.486+0000 7fefc5b9b700 1 -- 172.21.15.55:0/754657428 learned_addr learned my addr 172.21.15.55:0/754657428 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:47.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.486+0000 7fefc539a700 1 --2- 172.21.15.55:0/754657428 >> v2:172.21.15.55:3300/0 conn(0x7fefc01597f0 0x7fefc00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.487+0000 7fefc4b99700 1 -- 172.21.15.55:0/754657428 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652003113 0 0) 0x7fefc00b11d0 con 0x7fefc0159420 2023-10-06T03:33:47.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.487+0000 7fefc539a700 1 -- 172.21.15.55:0/754657428 >> v1:172.21.15.163:6789/0 conn(0x7fefc015d230 legacy=0x7fefc015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.487+0000 7fefc539a700 1 -- 172.21.15.55:0/754657428 >> v1:172.21.15.55:6789/0 conn(0x7fefc0159420 legacy=0x7fefc00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.487+0000 7fefc539a700 1 -- 172.21.15.55:0/754657428 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc00b1a70 con 0x7fefc01597f0 2023-10-06T03:33:47.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.487+0000 7fefc539a700 1 --2- 172.21.15.55:0/754657428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc01597f0 0x7fefc00b2270 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fefb00098e0 tx=0x7fefb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c07d0ad7a11904 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:47.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.487+0000 7fefc4b99700 1 -- 172.21.15.55:0/754657428 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb0009e80 con 0x7fefc01597f0 2023-10-06T03:33:47.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.487+0000 7fefc4b99700 1 -- 172.21.15.55:0/754657428 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb000b860 con 0x7fefc01597f0 2023-10-06T03:33:47.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.487+0000 7fefc4b99700 1 -- 172.21.15.55:0/754657428 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb000bbe0 con 0x7fefc01597f0 2023-10-06T03:33:47.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.488+0000 7fefc6b9d700 1 -- 172.21.15.55:0/754657428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc01597f0 msgr2=0x7fefc00b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.488+0000 7fefc6b9d700 1 --2- 172.21.15.55:0/754657428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc01597f0 0x7fefc00b2270 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fefb00098e0 tx=0x7fefb00091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.488+0000 7fefc6b9d700 1 -- 172.21.15.55:0/754657428 shutdown_connections 2023-10-06T03:33:47.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.488+0000 7fefc6b9d700 1 --2- 172.21.15.55:0/754657428 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc01597f0 0x7fefc00b2270 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.488+0000 7fefc6b9d700 1 -- 172.21.15.55:0/754657428 >> 172.21.15.55:0/754657428 conn(0x7fefc00d0660 msgr2=0x7fefc00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:47.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.488+0000 7fefc6b9d700 1 -- 172.21.15.55:0/754657428 shutdown_connections 2023-10-06T03:33:47.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.488+0000 7fefc6b9d700 1 -- 172.21.15.55:0/754657428 wait complete. 2023-10-06T03:33:47.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.490+0000 7fefc6b9d700 1 Processor -- start 2023-10-06T03:33:47.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.490+0000 7fefc6b9d700 1 -- start start 2023-10-06T03:33:47.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.491+0000 7fefc6b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc0159420 0x7fefc0153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.491+0000 7fefc6b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefc01597f0 0x7fefc0068ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.491+0000 7fefc6b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefc015d230 0x7fefc0069220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.491+0000 7fefc6b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc00c6120 con 0x7fefc0159420 2023-10-06T03:33:47.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.491+0000 7fefc6b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc00c5510 con 0x7fefc015d230 2023-10-06T03:33:47.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.491+0000 7fefc6b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc00c57f0 con 0x7fefc01597f0 2023-10-06T03:33:47.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.491+0000 7fefc539a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefc01597f0 0x7fefc0068ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.491+0000 7fefc539a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefc01597f0 0x7fefc0068ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:34696/0 (socket says 172.21.15.55:34696) 2023-10-06T03:33:47.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.491+0000 7fefc539a700 1 -- 172.21.15.55:0/971394010 learned_addr learned my addr 172.21.15.55:0/971394010 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:47.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.491+0000 7fefc5b9b700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc0159420 0x7fefc0153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.492+0000 7fefc539a700 1 -- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefc015d230 msgr2=0x7fefc0069220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.492+0000 7fefc639c700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefc015d230 0x7fefc0069220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.492+0000 7fefc539a700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefc015d230 0x7fefc0069220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.492+0000 7fefc539a700 1 -- 172.21.15.55:0/971394010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc0159420 msgr2=0x7fefc0153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.492+0000 7fefc539a700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc0159420 0x7fefc0153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.492+0000 7fefc539a700 1 -- 172.21.15.55:0/971394010 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb0009420 con 0x7fefc01597f0 2023-10-06T03:33:47.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.492+0000 7fefc639c700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefc015d230 0x7fefc0069220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:47.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.492+0000 7fefc5b9b700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc0159420 0x7fefc0153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:47.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.493+0000 7fefc539a700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefc01597f0 0x7fefc0068ce0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fefb0013000 tx=0x7fefb0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:47.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.493+0000 7fefb5ffb700 1 -- 172.21.15.55:0/971394010 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb001f070 con 0x7fefc01597f0 2023-10-06T03:33:47.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.493+0000 7fefb5ffb700 1 -- 172.21.15.55:0/971394010 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb0000c90 con 0x7fefc01597f0 2023-10-06T03:33:47.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.493+0000 7fefc6b9d700 1 -- 172.21.15.55:0/971394010 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc00c5ad0 con 0x7fefc01597f0 2023-10-06T03:33:47.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.493+0000 7fefb5ffb700 1 -- 172.21.15.55:0/971394010 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb0025420 con 0x7fefc01597f0 2023-10-06T03:33:47.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.493+0000 7fefc6b9d700 1 -- 172.21.15.55:0/971394010 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc0169650 con 0x7fefc01597f0 2023-10-06T03:33:47.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.494+0000 7fefc6b9d700 1 -- 172.21.15.55:0/971394010 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc00c6290 con 0x7fefc01597f0 2023-10-06T03:33:47.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.495+0000 7fefb5ffb700 1 -- 172.21.15.55:0/971394010 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fefb0023070 con 0x7fefc01597f0 2023-10-06T03:33:47.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.496+0000 7fefb5ffb700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fefa4066f30 0x7fefa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.497+0000 7fefb5ffb700 1 -- 172.21.15.55:0/971394010 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fefb0085fe0 con 0x7fefc01597f0 2023-10-06T03:33:47.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.497+0000 7fefc5b9b700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fefa4066f30 0x7fefa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.498+0000 7fefc5b9b700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fefa4066f30 0x7fefa40693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fefbc007fd0 tx=0x7fefbc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:47.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.501+0000 7fefb5ffb700 1 -- 172.21.15.55:0/971394010 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fefb00530f0 con 0x7fefc01597f0 2023-10-06T03:33:47.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.707+0000 7fefc6b9d700 1 -- 172.21.15.55:0/971394010 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefc00c5930 con 0x7fefc01597f0 2023-10-06T03:33:47.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.708+0000 7fefb5ffb700 1 -- 172.21.15.55:0/971394010 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7fefc00c5930 con 0x7fefc01597f0 2023-10-06T03:33:47.710 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:47.710 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:16.470817+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:47.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.711+0000 7fefc6b9d700 1 -- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fefa4066f30 msgr2=0x7fefa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.711+0000 7fefc6b9d700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fefa4066f30 0x7fefa40693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fefbc007fd0 tx=0x7fefbc009040 comp rx=0 tx=0).stop 2023-10-06T03:33:47.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.711+0000 7fefc6b9d700 1 -- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefc01597f0 msgr2=0x7fefc0068ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.711+0000 7fefc6b9d700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefc01597f0 0x7fefc0068ce0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fefb0013000 tx=0x7fefb0017000 comp rx=0 tx=0).stop 2023-10-06T03:33:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.711+0000 7fefc6b9d700 1 -- 172.21.15.55:0/971394010 shutdown_connections 2023-10-06T03:33:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.711+0000 7fefc6b9d700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefc015d230 0x7fefc0069220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.711+0000 7fefc6b9d700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefc0159420 0x7fefc0153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.711+0000 7fefc6b9d700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fefa4066f30 0x7fefa40693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.711+0000 7fefc6b9d700 1 --2- 172.21.15.55:0/971394010 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefc01597f0 0x7fefc0068ce0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.711+0000 7fefc6b9d700 1 -- 172.21.15.55:0/971394010 >> 172.21.15.55:0/971394010 conn(0x7fefc00d0660 msgr2=0x7fefc00b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:47.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.711+0000 7fefc6b9d700 1 -- 172.21.15.55:0/971394010 shutdown_connections 2023-10-06T03:33:47.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.711+0000 7fefc6b9d700 1 -- 172.21.15.55:0/971394010 wait complete. 2023-10-06T03:33:47.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:33:13.543361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:33:14.341572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:33:47.730 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:33:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.918+0000 7f1c1b59e700 1 Processor -- start 2023-10-06T03:33:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.919+0000 7f1c1b59e700 1 -- start start 2023-10-06T03:33:47.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.920+0000 7f1c1b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1c159a70 0x7f1c1c0c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.920+0000 7f1c1b59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1c1c068aa0 0x7f1c1c0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.920+0000 7f1c1b59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c1c0ca920 con 0x7f1c1c06bfa0 2023-10-06T03:33:47.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.920+0000 7f1c1b59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c1c06af00 con 0x7f1c1c068aa0 2023-10-06T03:33:47.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.920+0000 7f1c1b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c06b1e0 con 0x7f1c1c159a70 2023-10-06T03:33:47.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.921+0000 7f1c1a59c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1c159a70 0x7f1c1c0c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.921+0000 7f1c1ad9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1c1c068aa0 0x7f1c1c0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.921+0000 7f1c19d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1c1c06bfa0 0x7f1c1c068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:47840/0 (socket says 172.21.15.55:47840) 2023-10-06T03:33:47.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.921+0000 7f1c19d9b700 1 -- 172.21.15.55:0/453715337 learned_addr learned my addr 172.21.15.55:0/453715337 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.921+0000 7f1c1ad9d700 1 -- 172.21.15.55:0/453715337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1c159a70 msgr2=0x7f1c1c0c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.921+0000 7f1c1ad9d700 1 --2- 172.21.15.55:0/453715337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1c159a70 0x7f1c1c0c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.921+0000 7f1c1ad9d700 1 -- 172.21.15.55:0/453715337 >> v1:172.21.15.163:6789/0 conn(0x7f1c1c06bfa0 legacy=0x7f1c1c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.921+0000 7f1c1ad9d700 1 -- 172.21.15.55:0/453715337 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c06b4c0 con 0x7f1c1c068aa0 2023-10-06T03:33:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.921+0000 7f1c1a59c700 1 --2- 172.21.15.55:0/453715337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1c159a70 0x7f1c1c0c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.921+0000 7f1c1ad9d700 1 --2- 172.21.15.55:0/453715337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c1c068aa0 0x7f1c1c0c8830 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009e60 tx=0x7f1c14009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6c99f4982ce7706 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.922+0000 7f1c1959a700 1 -- 172.21.15.55:0/453715337 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1400e070 con 0x7f1c1c068aa0 2023-10-06T03:33:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.922+0000 7f1c1959a700 1 -- 172.21.15.55:0/453715337 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1400bb60 con 0x7f1c1c068aa0 2023-10-06T03:33:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.922+0000 7f1c1959a700 1 -- 172.21.15.55:0/453715337 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14012430 con 0x7f1c1c068aa0 2023-10-06T03:33:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.922+0000 7f1c1b59e700 1 -- 172.21.15.55:0/453715337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c1c068aa0 msgr2=0x7f1c1c0c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.922+0000 7f1c1b59e700 1 --2- 172.21.15.55:0/453715337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c1c068aa0 0x7f1c1c0c8830 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1c14009e60 tx=0x7f1c14009780 comp rx=0 tx=0).stop 2023-10-06T03:33:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.922+0000 7f1c1b59e700 1 -- 172.21.15.55:0/453715337 shutdown_connections 2023-10-06T03:33:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.922+0000 7f1c1b59e700 1 --2- 172.21.15.55:0/453715337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1c159a70 0x7f1c1c0c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.922+0000 7f1c1b59e700 1 --2- 172.21.15.55:0/453715337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c1c068aa0 0x7f1c1c0c8830 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.922+0000 7f1c1b59e700 1 -- 172.21.15.55:0/453715337 >> 172.21.15.55:0/453715337 conn(0x7f1c1c0cddc0 msgr2=0x7f1c1c0c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.922+0000 7f1c1b59e700 1 -- 172.21.15.55:0/453715337 shutdown_connections 2023-10-06T03:33:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.922+0000 7f1c1b59e700 1 -- 172.21.15.55:0/453715337 wait complete. 2023-10-06T03:33:47.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.924+0000 7f1c1b59e700 1 Processor -- start 2023-10-06T03:33:47.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.924+0000 7f1c1b59e700 1 -- start start 2023-10-06T03:33:47.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.925+0000 7f1c1b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1c068aa0 0x7f1c1c06e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.925+0000 7f1c1b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c06bfa0 0x7f1c1c15c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.925+0000 7f1c1b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c1c159a70 0x7f1c1c154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.925+0000 7f1c1b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c15d9d0 con 0x7f1c1c06bfa0 2023-10-06T03:33:47.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.925+0000 7f1c1b59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c15d130 con 0x7f1c1c159a70 2023-10-06T03:33:47.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.925+0000 7f1c1b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c15d410 con 0x7f1c1c068aa0 2023-10-06T03:33:47.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.925+0000 7f1c19d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c06bfa0 0x7f1c1c15c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.925+0000 7f1c19d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c06bfa0 0x7f1c1c15c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36112/0 (socket says 172.21.15.55:36112) 2023-10-06T03:33:47.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.925+0000 7f1c19d9b700 1 -- 172.21.15.55:0/1730231890 learned_addr learned my addr 172.21.15.55:0/1730231890 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:47.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.925+0000 7f1c1a59c700 1 --2- 172.21.15.55:0/1730231890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1c068aa0 0x7f1c1c06e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.925+0000 7f1c1ad9d700 1 --2- 172.21.15.55:0/1730231890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c1c159a70 0x7f1c1c154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.926+0000 7f1c19d9b700 1 -- 172.21.15.55:0/1730231890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1c068aa0 msgr2=0x7f1c1c06e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.926+0000 7f1c19d9b700 1 --2- 172.21.15.55:0/1730231890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1c068aa0 0x7f1c1c06e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.926+0000 7f1c19d9b700 1 -- 172.21.15.55:0/1730231890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c1c159a70 msgr2=0x7f1c1c154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:47.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.926+0000 7f1c19d9b700 1 --2- 172.21.15.55:0/1730231890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c1c159a70 0x7f1c1c154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:47.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.926+0000 7f1c19d9b700 1 -- 172.21.15.55:0/1730231890 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c009420 con 0x7f1c1c06bfa0 2023-10-06T03:33:47.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.926+0000 7f1c19d9b700 1 --2- 172.21.15.55:0/1730231890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c06bfa0 0x7f1c1c15c650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000c590 tx=0x7f1c10001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:47.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.926+0000 7f1c0a7fc700 1 -- 172.21.15.55:0/1730231890 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10005b80 con 0x7f1c1c06bfa0 2023-10-06T03:33:47.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.926+0000 7f1c0a7fc700 1 -- 172.21.15.55:0/1730231890 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c10005ce0 con 0x7f1c1c06bfa0 2023-10-06T03:33:47.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.926+0000 7f1c0a7fc700 1 -- 172.21.15.55:0/1730231890 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c10004ad0 con 0x7f1c1c06bfa0 2023-10-06T03:33:47.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.926+0000 7f1c1b59e700 1 -- 172.21.15.55:0/1730231890 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c14009a20 con 0x7f1c1c06bfa0 2023-10-06T03:33:47.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.926+0000 7f1c1b59e700 1 -- 172.21.15.55:0/1730231890 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1c155ef0 con 0x7f1c1c06bfa0 2023-10-06T03:33:47.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.927+0000 7f1c1b59e700 1 -- 172.21.15.55:0/1730231890 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1c1560b0 con 0x7f1c1c06bfa0 2023-10-06T03:33:47.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.928+0000 7f1c0a7fc700 1 -- 172.21.15.55:0/1730231890 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c100070d0 con 0x7f1c1c06bfa0 2023-10-06T03:33:47.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.930+0000 7f1c0a7fc700 1 --2- 172.21.15.55:0/1730231890 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1bfc066f30 0x7f1bfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:47.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.930+0000 7f1c0a7fc700 1 -- 172.21.15.55:0/1730231890 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1c100839a0 con 0x7f1c1c06bfa0 2023-10-06T03:33:47.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.931+0000 7f1c1a59c700 1 --2- 172.21.15.55:0/1730231890 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1bfc066f30 0x7f1bfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:47.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.931+0000 7f1c1a59c700 1 --2- 172.21.15.55:0/1730231890 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1bfc066f30 0x7f1bfc0693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009f40 tx=0x7f1c0c0091a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:47.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:47.934+0000 7f1c0a7fc700 1 -- 172.21.15.55:0/1730231890 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c1004b5a0 con 0x7f1c1c06bfa0 2023-10-06T03:33:48.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.136+0000 7f1c1b59e700 1 -- 172.21.15.55:0/1730231890 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1c1c0c7c20 con 0x7f1c1c06bfa0 2023-10-06T03:33:48.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.137+0000 7f1c0a7fc700 1 -- 172.21.15.55:0/1730231890 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1c1c1560b0 con 0x7f1c1c06bfa0 2023-10-06T03:33:48.138 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:33:48.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.139+0000 7f1c1b59e700 1 -- 172.21.15.55:0/1730231890 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1bfc066f30 msgr2=0x7f1bfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:48.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.139+0000 7f1c1b59e700 1 --2- 172.21.15.55:0/1730231890 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1bfc066f30 0x7f1bfc0693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c009f40 tx=0x7f1c0c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.139+0000 7f1c1b59e700 1 -- 172.21.15.55:0/1730231890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c06bfa0 msgr2=0x7f1c1c15c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:48.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.139+0000 7f1c1b59e700 1 --2- 172.21.15.55:0/1730231890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c06bfa0 0x7f1c1c15c650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1c1000c590 tx=0x7f1c10001070 comp rx=0 tx=0).stop 2023-10-06T03:33:48.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.139+0000 7f1c1b59e700 1 -- 172.21.15.55:0/1730231890 shutdown_connections 2023-10-06T03:33:48.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.139+0000 7f1c1b59e700 1 --2- 172.21.15.55:0/1730231890 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c1c159a70 0x7f1c1c154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.139+0000 7f1c1b59e700 1 --2- 172.21.15.55:0/1730231890 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1bfc066f30 0x7f1bfc0693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.139+0000 7f1c1b59e700 1 --2- 172.21.15.55:0/1730231890 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1c068aa0 0x7f1c1c06e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.139+0000 7f1c1b59e700 1 --2- 172.21.15.55:0/1730231890 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c1c06bfa0 0x7f1c1c15c650 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.139+0000 7f1c1b59e700 1 -- 172.21.15.55:0/1730231890 >> 172.21.15.55:0/1730231890 conn(0x7f1c1c0cddc0 msgr2=0x7f1c1c1552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:48.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.139+0000 7f1c1b59e700 1 -- 172.21.15.55:0/1730231890 shutdown_connections 2023-10-06T03:33:48.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.139+0000 7f1c1b59e700 1 -- 172.21.15.55:0/1730231890 wait complete. 2023-10-06T03:33:48.157 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:33:48.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.346+0000 7f3d6b59e700 1 Processor -- start 2023-10-06T03:33:48.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.346+0000 7f3d6b59e700 1 -- start start 2023-10-06T03:33:48.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.347+0000 7f3d6b59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3d6c159a60 0x7f3d6c0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.347+0000 7f3d6b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3d6c0ccc00 0x7f3d6c069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.347+0000 7f3d6b59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6c0c9cb0 con 0x7f3d6c06a020 2023-10-06T03:33:48.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.347+0000 7f3d6b59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6c06dfc0 con 0x7f3d6c0ccc00 2023-10-06T03:33:48.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.347+0000 7f3d6b59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6c0c9f30 con 0x7f3d6c159a60 2023-10-06T03:33:48.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.347+0000 7f3d69d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3d6c0ccc00 0x7f3d6c069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:48.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.347+0000 7f3d69d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3d6c0ccc00 0x7f3d6c069ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36116/0 (socket says 172.21.15.55:36116) 2023-10-06T03:33:48.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.347+0000 7f3d69d9b700 1 -- 172.21.15.55:0/31616721 learned_addr learned my addr 172.21.15.55:0/31616721 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:48.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.348+0000 7f3d6a59c700 1 --2- 172.21.15.55:0/31616721 >> v2:172.21.15.163:3300/0 conn(0x7f3d6c159a60 0x7f3d6c0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:48.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.348+0000 7f3d69d9b700 1 -- 172.21.15.55:0/31616721 >> v2:172.21.15.163:3300/0 conn(0x7f3d6c159a60 msgr2=0x7f3d6c0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:48.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.348+0000 7f3d69d9b700 1 --2- 172.21.15.55:0/31616721 >> v2:172.21.15.163:3300/0 conn(0x7f3d6c159a60 0x7f3d6c0c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.348+0000 7f3d69d9b700 1 -- 172.21.15.55:0/31616721 >> v1:172.21.15.163:6789/0 conn(0x7f3d6c06a020 legacy=0x7f3d6c0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:48.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.348+0000 7f3d69d9b700 1 -- 172.21.15.55:0/31616721 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c0ca470 con 0x7f3d6c0ccc00 2023-10-06T03:33:48.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.348+0000 7f3d69d9b700 1 --2- 172.21.15.55:0/31616721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6c0ccc00 0x7f3d6c069ae0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c06e510 tx=0x7f3d60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cacdc3e3eac04a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:48.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.348+0000 7f3d6959a700 1 -- 172.21.15.55:0/31616721 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000b8f0 con 0x7f3d6c0ccc00 2023-10-06T03:33:48.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.348+0000 7f3d6959a700 1 -- 172.21.15.55:0/31616721 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6000ba50 con 0x7f3d6c0ccc00 2023-10-06T03:33:48.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.349+0000 7f3d6959a700 1 -- 172.21.15.55:0/31616721 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000bdf0 con 0x7f3d6c0ccc00 2023-10-06T03:33:48.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.349+0000 7f3d6b59e700 1 -- 172.21.15.55:0/31616721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6c0ccc00 msgr2=0x7f3d6c069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:48.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.349+0000 7f3d6b59e700 1 --2- 172.21.15.55:0/31616721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6c0ccc00 0x7f3d6c069ae0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c06e510 tx=0x7f3d60009250 comp rx=0 tx=0).stop 2023-10-06T03:33:48.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.349+0000 7f3d6b59e700 1 -- 172.21.15.55:0/31616721 shutdown_connections 2023-10-06T03:33:48.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.349+0000 7f3d6b59e700 1 --2- 172.21.15.55:0/31616721 >> v2:172.21.15.163:3300/0 conn(0x7f3d6c159a60 0x7f3d6c0c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.349+0000 7f3d6b59e700 1 --2- 172.21.15.55:0/31616721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6c0ccc00 0x7f3d6c069ae0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.349+0000 7f3d6b59e700 1 -- 172.21.15.55:0/31616721 >> 172.21.15.55:0/31616721 conn(0x7f3d6c06baa0 msgr2=0x7f3d6c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:48.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.349+0000 7f3d6b59e700 1 -- 172.21.15.55:0/31616721 shutdown_connections 2023-10-06T03:33:48.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.349+0000 7f3d6b59e700 1 -- 172.21.15.55:0/31616721 wait complete. 2023-10-06T03:33:48.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.351+0000 7f3d6b59e700 1 Processor -- start 2023-10-06T03:33:48.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.351+0000 7f3d6b59e700 1 -- start start 2023-10-06T03:33:48.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d6b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6c06a020 0x7f3d6c0c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d6b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d6c0ccc00 0x7f3d6c0c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d6b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d6c159a60 0x7f3d6c16c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d6b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0c67c0 con 0x7f3d6c06a020 2023-10-06T03:33:48.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d6b59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0c5c80 con 0x7f3d6c0ccc00 2023-10-06T03:33:48.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d6b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c16a0d0 con 0x7f3d6c159a60 2023-10-06T03:33:48.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d6a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6c06a020 0x7f3d6c0c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:48.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d6a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6c06a020 0x7f3d6c0c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36120/0 (socket says 172.21.15.55:36120) 2023-10-06T03:33:48.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d6a59c700 1 -- 172.21.15.55:0/3988967146 learned_addr learned my addr 172.21.15.55:0/3988967146 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:48.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d6a59c700 1 -- 172.21.15.55:0/3988967146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d6c159a60 msgr2=0x7f3d6c16c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:48.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d6a59c700 1 --2- 172.21.15.55:0/3988967146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d6c159a60 0x7f3d6c16c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d6a59c700 1 -- 172.21.15.55:0/3988967146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d6c0ccc00 msgr2=0x7f3d6c0c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:48.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d69d9b700 1 --2- 172.21.15.55:0/3988967146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d6c0ccc00 0x7f3d6c0c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:48.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d6a59c700 1 --2- 172.21.15.55:0/3988967146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d6c0ccc00 0x7f3d6c0c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d6a59c700 1 -- 172.21.15.55:0/3988967146 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d600094f0 con 0x7f3d6c06a020 2023-10-06T03:33:48.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.352+0000 7f3d69d9b700 1 --2- 172.21.15.55:0/3988967146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d6c0ccc00 0x7f3d6c0c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:48.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.353+0000 7f3d6a59c700 1 --2- 172.21.15.55:0/3988967146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6c06a020 0x7f3d6c0c77a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00d860 tx=0x7f3d5c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:48.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.353+0000 7f3d5a7fc700 1 -- 172.21.15.55:0/3988967146 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c0125f0 con 0x7f3d6c06a020 2023-10-06T03:33:48.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.353+0000 7f3d5a7fc700 1 -- 172.21.15.55:0/3988967146 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c012750 con 0x7f3d6c06a020 2023-10-06T03:33:48.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.353+0000 7f3d5a7fc700 1 -- 172.21.15.55:0/3988967146 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c012b10 con 0x7f3d6c06a020 2023-10-06T03:33:48.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.353+0000 7f3d6b59e700 1 -- 172.21.15.55:0/3988967146 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6c0c5f20 con 0x7f3d6c06a020 2023-10-06T03:33:48.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.353+0000 7f3d6b59e700 1 -- 172.21.15.55:0/3988967146 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6c0c64e0 con 0x7f3d6c06a020 2023-10-06T03:33:48.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.354+0000 7f3d6b59e700 1 -- 172.21.15.55:0/3988967146 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6c06eba0 con 0x7f3d6c06a020 2023-10-06T03:33:48.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.355+0000 7f3d5a7fc700 1 -- 172.21.15.55:0/3988967146 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d5c01e460 con 0x7f3d6c06a020 2023-10-06T03:33:48.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.356+0000 7f3d5a7fc700 1 --2- 172.21.15.55:0/3988967146 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3d4c066f30 0x7f3d4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.356+0000 7f3d5a7fc700 1 -- 172.21.15.55:0/3988967146 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3d5c084d50 con 0x7f3d6c06a020 2023-10-06T03:33:48.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.356+0000 7f3d69d9b700 1 --2- 172.21.15.55:0/3988967146 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3d4c066f30 0x7f3d4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:48.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.357+0000 7f3d69d9b700 1 --2- 172.21.15.55:0/3988967146 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3d4c066f30 0x7f3d4c0693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c06e510 tx=0x7f3d60017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:48.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.360+0000 7f3d5a7fc700 1 -- 172.21.15.55:0/3988967146 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d5c051dc0 con 0x7f3d6c06a020 2023-10-06T03:33:48.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.559+0000 7f3d6b59e700 1 -- 172.21.15.55:0/3988967146 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d6c0ca750 con 0x7f3d6c06a020 2023-10-06T03:33:48.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.559+0000 7f3d5a7fc700 1 -- 172.21.15.55:0/3988967146 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d5c0553e0 con 0x7f3d6c06a020 2023-10-06T03:33:48.561 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:33:48.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.562+0000 7f3d6b59e700 1 -- 172.21.15.55:0/3988967146 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3d4c066f30 msgr2=0x7f3d4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:48.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.562+0000 7f3d6b59e700 1 --2- 172.21.15.55:0/3988967146 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3d4c066f30 0x7f3d4c0693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c06e510 tx=0x7f3d60017000 comp rx=0 tx=0).stop 2023-10-06T03:33:48.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.562+0000 7f3d6b59e700 1 -- 172.21.15.55:0/3988967146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6c06a020 msgr2=0x7f3d6c0c77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:48.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.562+0000 7f3d6b59e700 1 --2- 172.21.15.55:0/3988967146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6c06a020 0x7f3d6c0c77a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00d860 tx=0x7f3d5c008040 comp rx=0 tx=0).stop 2023-10-06T03:33:48.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.562+0000 7f3d6b59e700 1 -- 172.21.15.55:0/3988967146 shutdown_connections 2023-10-06T03:33:48.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.562+0000 7f3d6b59e700 1 --2- 172.21.15.55:0/3988967146 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3d4c066f30 0x7f3d4c0693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.562+0000 7f3d6b59e700 1 --2- 172.21.15.55:0/3988967146 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d6c159a60 0x7f3d6c16c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.562+0000 7f3d6b59e700 1 --2- 172.21.15.55:0/3988967146 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d6c06a020 0x7f3d6c0c77a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.562+0000 7f3d6b59e700 1 --2- 172.21.15.55:0/3988967146 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d6c0ccc00 0x7f3d6c0c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.562+0000 7f3d6b59e700 1 -- 172.21.15.55:0/3988967146 >> 172.21.15.55:0/3988967146 conn(0x7f3d6c06baa0 msgr2=0x7f3d6c06a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:48.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.562+0000 7f3d6b59e700 1 -- 172.21.15.55:0/3988967146 shutdown_connections 2023-10-06T03:33:48.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.562+0000 7f3d6b59e700 1 -- 172.21.15.55:0/3988967146 wait complete. 2023-10-06T03:33:48.580 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:33:48.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.770+0000 7f507cb9d700 1 Processor -- start 2023-10-06T03:33:48.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.770+0000 7f507cb9d700 1 -- start start 2023-10-06T03:33:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.771+0000 7f507cb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50780aef00 0x7f507815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.771+0000 7f507cb9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f507806e470 0x7f507806e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.771+0000 7f507cb9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50780aea70 con 0x7f507815a3b0 2023-10-06T03:33:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.771+0000 7f507cb9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507806ee00 con 0x7f507806e470 2023-10-06T03:33:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.771+0000 7f507cb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50780ae480 con 0x7f50780aef00 2023-10-06T03:33:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.772+0000 7f50777fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f507815a3b0 0x7f507815a780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59508/0 (socket says 172.21.15.55:59508) 2023-10-06T03:33:48.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.772+0000 7f50777fe700 1 -- 172.21.15.55:0/1004012056 learned_addr learned my addr 172.21.15.55:0/1004012056 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.772+0000 7f5076ffd700 1 --2- 172.21.15.55:0/1004012056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50780aef00 0x7f507815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.772+0000 7f5077fff700 1 --2- 172.21.15.55:0/1004012056 >> v2:172.21.15.163:3300/0 conn(0x7f507806e470 0x7f507806e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:48.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.772+0000 7f50767fc700 1 -- 172.21.15.55:0/1004012056 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753292549 0 0) 0x7f50780aea70 con 0x7f507815a3b0 2023-10-06T03:33:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.772+0000 7f50767fc700 1 -- 172.21.15.55:0/1004012056 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5060003570 con 0x7f507815a3b0 2023-10-06T03:33:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.772+0000 7f50767fc700 1 -- 172.21.15.55:0/1004012056 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1651776252 0 0) 0x7f5060003570 con 0x7f507815a3b0 2023-10-06T03:33:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.772+0000 7f50767fc700 1 -- 172.21.15.55:0/1004012056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50780aef00 msgr2=0x7f507815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.772+0000 7f50767fc700 1 --2- 172.21.15.55:0/1004012056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50780aef00 0x7f507815d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.772+0000 7f50767fc700 1 -- 172.21.15.55:0/1004012056 >> v2:172.21.15.163:3300/0 conn(0x7f507806e470 msgr2=0x7f507806e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.772+0000 7f50767fc700 1 --2- 172.21.15.55:0/1004012056 >> v2:172.21.15.163:3300/0 conn(0x7f507806e470 0x7f507806e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.772+0000 7f50767fc700 1 -- 172.21.15.55:0/1004012056 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507815dd30 con 0x7f507815a3b0 2023-10-06T03:33:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.772+0000 7f50767fc700 1 -- 172.21.15.55:0/1004012056 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f5068001470 con 0x7f507815a3b0 2023-10-06T03:33:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.772+0000 7f50767fc700 1 -- 172.21.15.55:0/1004012056 >> v1:172.21.15.55:6789/0 conn(0x7f507815a3b0 legacy=0x7f507815a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.773+0000 7f50767fc700 1 --2- 172.21.15.55:0/1004012056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5060003a60 0x7f5060003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.773+0000 7f50767fc700 1 --2- 172.21.15.55:0/1004012056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5060004810 0x7f5060004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.773+0000 7f50767fc700 1 --2- 172.21.15.55:0/1004012056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50780aef00 0x7f50600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.773+0000 7f50767fc700 1 -- 172.21.15.55:0/1004012056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507806ee00 con 0x7f5060003a60 2023-10-06T03:33:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.773+0000 7f50767fc700 1 -- 172.21.15.55:0/1004012056 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50780ae480 con 0x7f5060004810 2023-10-06T03:33:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.773+0000 7f50767fc700 1 -- 172.21.15.55:0/1004012056 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506000ba60 con 0x7f50780aef00 2023-10-06T03:33:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.773+0000 7f50777fe700 1 --2- 172.21.15.55:0/1004012056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5060003a60 0x7f5060003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.773+0000 7f5077fff700 1 --2- 172.21.15.55:0/1004012056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50780aef00 0x7f50600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.773+0000 7f5076ffd700 1 --2- 172.21.15.55:0/1004012056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5060004810 0x7f5060004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.773+0000 7f50777fe700 1 -- 172.21.15.55:0/1004012056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50780aef00 msgr2=0x7f50600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.773+0000 7f50777fe700 1 --2- 172.21.15.55:0/1004012056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50780aef00 0x7f50600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.773+0000 7f50777fe700 1 -- 172.21.15.55:0/1004012056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5060004810 msgr2=0x7f5060004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.773+0000 7f50777fe700 1 --2- 172.21.15.55:0/1004012056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5060004810 0x7f5060004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.773+0000 7f50777fe700 1 -- 172.21.15.55:0/1004012056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5060003570 con 0x7f5060003a60 2023-10-06T03:33:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.774+0000 7f50777fe700 1 --2- 172.21.15.55:0/1004012056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5060003a60 0x7f5060003e70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f506800c3f0 tx=0x7f5068001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44d960ef23a672f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.774+0000 7f50767fc700 1 -- 172.21.15.55:0/1004012056 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506800cc80 con 0x7f5060003a60 2023-10-06T03:33:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.774+0000 7f50767fc700 1 -- 172.21.15.55:0/1004012056 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506800ce20 con 0x7f5060003a60 2023-10-06T03:33:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.774+0000 7f50767fc700 1 -- 172.21.15.55:0/1004012056 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068005b70 con 0x7f5060003a60 2023-10-06T03:33:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.774+0000 7f507cb9d700 1 -- 172.21.15.55:0/1004012056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5060003a60 msgr2=0x7f5060003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.774+0000 7f507cb9d700 1 --2- 172.21.15.55:0/1004012056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5060003a60 0x7f5060003e70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f506800c3f0 tx=0x7f5068001e90 comp rx=0 tx=0).stop 2023-10-06T03:33:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.774+0000 7f5077fff700 1 -- 172.21.15.55:0/1004012056 reap_dead start 2023-10-06T03:33:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.774+0000 7f507cb9d700 1 -- 172.21.15.55:0/1004012056 shutdown_connections 2023-10-06T03:33:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.774+0000 7f507cb9d700 1 -- 172.21.15.55:0/1004012056 >> 172.21.15.55:0/1004012056 conn(0x7f50780c9f70 msgr2=0x7f507806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:48.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.775+0000 7f507cb9d700 1 -- 172.21.15.55:0/1004012056 shutdown_connections 2023-10-06T03:33:48.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.775+0000 7f507cb9d700 1 -- 172.21.15.55:0/1004012056 wait complete. 2023-10-06T03:33:48.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.776+0000 7f507cb9d700 1 Processor -- start 2023-10-06T03:33:48.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.776+0000 7f507cb9d700 1 -- start start 2023-10-06T03:33:48.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.777+0000 7f507cb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5060003a60 0x7f50781517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.777+0000 7f507cb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50780c3ae0 0x7f50780c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.777+0000 7f507cb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5078156e10 0x7f5078157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.777+0000 7f507cb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50780c5190 con 0x7f50780c3ae0 2023-10-06T03:33:48.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.777+0000 7f507cb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50780c5750 con 0x7f5078156e10 2023-10-06T03:33:48.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.777+0000 7f507cb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50780c6c60 con 0x7f5060003a60 2023-10-06T03:33:48.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.777+0000 7f5076ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50780c3ae0 0x7f50780c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:48.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.777+0000 7f5076ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50780c3ae0 0x7f50780c3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36138/0 (socket says 172.21.15.55:36138) 2023-10-06T03:33:48.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.777+0000 7f5076ffd700 1 -- 172.21.15.55:0/1133754999 learned_addr learned my addr 172.21.15.55:0/1133754999 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:48.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.778+0000 7f50777fe700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5060003a60 0x7f50781517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:48.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.778+0000 7f5077fff700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5078156e10 0x7f5078157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:48.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.778+0000 7f5076ffd700 1 -- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5060003a60 msgr2=0x7f50781517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:48.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.778+0000 7f5076ffd700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5060003a60 0x7f50781517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.778+0000 7f5076ffd700 1 -- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5078156e10 msgr2=0x7f5078157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:48.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.778+0000 7f5076ffd700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5078156e10 0x7f5078157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.778+0000 7f5076ffd700 1 -- 172.21.15.55:0/1133754999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50780c45e0 con 0x7f50780c3ae0 2023-10-06T03:33:48.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.778+0000 7f50777fe700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5060003a60 0x7f50781517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:48.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.778+0000 7f5076ffd700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50780c3ae0 0x7f50780c3f50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f506c011d40 tx=0x7f506c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:48.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.778+0000 7f505f7fe700 1 -- 172.21.15.55:0/1133754999 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c00fa40 con 0x7f50780c3ae0 2023-10-06T03:33:48.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.778+0000 7f505f7fe700 1 -- 172.21.15.55:0/1133754999 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c00fba0 con 0x7f50780c3ae0 2023-10-06T03:33:48.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.778+0000 7f505f7fe700 1 -- 172.21.15.55:0/1133754999 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c014440 con 0x7f50780c3ae0 2023-10-06T03:33:48.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.778+0000 7f507cb9d700 1 -- 172.21.15.55:0/1133754999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50780c48c0 con 0x7f50780c3ae0 2023-10-06T03:33:48.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.779+0000 7f507cb9d700 1 -- 172.21.15.55:0/1133754999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5078151d10 con 0x7f50780c3ae0 2023-10-06T03:33:48.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.780+0000 7f507cb9d700 1 -- 172.21.15.55:0/1133754999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50780c58c0 con 0x7f50780c3ae0 2023-10-06T03:33:48.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.780+0000 7f505f7fe700 1 -- 172.21.15.55:0/1133754999 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f506c014870 con 0x7f50780c3ae0 2023-10-06T03:33:48.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.782+0000 7f505f7fe700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5064066f30 0x7f50640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:48.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.782+0000 7f505f7fe700 1 -- 172.21.15.55:0/1133754999 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f506c018030 con 0x7f50780c3ae0 2023-10-06T03:33:48.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.782+0000 7f50777fe700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5064066f30 0x7f50640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:48.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.783+0000 7f50777fe700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5064066f30 0x7f50640693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f506800c3f0 tx=0x7f5068015000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:48.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.786+0000 7f505f7fe700 1 -- 172.21.15.55:0/1133754999 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f506c0521f0 con 0x7f50780c3ae0 2023-10-06T03:33:48.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.985+0000 7f507cb9d700 1 -- 172.21.15.55:0/1133754999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f507815e010 con 0x7f50780c3ae0 2023-10-06T03:33:48.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.986+0000 7f505f7fe700 1 -- 172.21.15.55:0/1133754999 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f506c022020 con 0x7f50780c3ae0 2023-10-06T03:33:48.988 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T03:33:48.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.988+0000 7f507cb9d700 1 -- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5064066f30 msgr2=0x7f50640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:48.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.988+0000 7f507cb9d700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5064066f30 0x7f50640693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f506800c3f0 tx=0x7f5068015000 comp rx=0 tx=0).stop 2023-10-06T03:33:48.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.988+0000 7f507cb9d700 1 -- 172.21.15.55:0/1133754999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50780c3ae0 msgr2=0x7f50780c3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:48.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.988+0000 7f507cb9d700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50780c3ae0 0x7f50780c3f50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f506c011d40 tx=0x7f506c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.988+0000 7f507cb9d700 1 -- 172.21.15.55:0/1133754999 shutdown_connections 2023-10-06T03:33:48.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.988+0000 7f507cb9d700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5078156e10 0x7f5078157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.988+0000 7f507cb9d700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5064066f30 0x7f50640693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.988+0000 7f507cb9d700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5060003a60 0x7f50781517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.988+0000 7f507cb9d700 1 --2- 172.21.15.55:0/1133754999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50780c3ae0 0x7f50780c3f50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:48.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.988+0000 7f507cb9d700 1 -- 172.21.15.55:0/1133754999 >> 172.21.15.55:0/1133754999 conn(0x7f50780c9f70 msgr2=0x7f5078152720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:48.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.988+0000 7f507cb9d700 1 -- 172.21.15.55:0/1133754999 shutdown_connections 2023-10-06T03:33:48.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:48.988+0000 7f507cb9d700 1 -- 172.21.15.55:0/1133754999 wait complete. 2023-10-06T03:33:49.007 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:33:49.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.195+0000 7f8ca6d9d700 1 Processor -- start 2023-10-06T03:33:49.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.195+0000 7f8ca6d9d700 1 -- start start 2023-10-06T03:33:49.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.196+0000 7f8ca6d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8ca00b4e70 0x7f8ca00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:49.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.196+0000 7f8ca6d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ca00b16d0 0x7f8ca00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:49.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.196+0000 7f8ca6d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca00b2080 con 0x7f8ca015d240 2023-10-06T03:33:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.196+0000 7f8ca6d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca00cfb00 con 0x7f8ca00b16d0 2023-10-06T03:33:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.196+0000 7f8ca6d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca00b5c40 con 0x7f8ca00b4e70 2023-10-06T03:33:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.197+0000 7f8ca559a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ca00b16d0 0x7f8ca00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.197+0000 7f8ca659c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8ca015d240 0x7f8ca015d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59514/0 (socket says 172.21.15.55:59514) 2023-10-06T03:33:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.197+0000 7f8ca559a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ca00b16d0 0x7f8ca00b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36152/0 (socket says 172.21.15.55:36152) 2023-10-06T03:33:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.197+0000 7f8ca659c700 1 -- 172.21.15.55:0/954600953 learned_addr learned my addr 172.21.15.55:0/954600953 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.197+0000 7f8ca5d9b700 1 --2- 172.21.15.55:0/954600953 >> v2:172.21.15.163:3300/0 conn(0x7f8ca00b4e70 0x7f8ca00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.197+0000 7f8ca559a700 1 -- 172.21.15.55:0/954600953 >> v2:172.21.15.163:3300/0 conn(0x7f8ca00b4e70 msgr2=0x7f8ca00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.197+0000 7f8ca559a700 1 --2- 172.21.15.55:0/954600953 >> v2:172.21.15.163:3300/0 conn(0x7f8ca00b4e70 0x7f8ca00ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.197+0000 7f8ca559a700 1 -- 172.21.15.55:0/954600953 >> v1:172.21.15.55:6789/0 conn(0x7f8ca015d240 legacy=0x7f8ca015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.197+0000 7f8ca559a700 1 -- 172.21.15.55:0/954600953 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca00b5f20 con 0x7f8ca00b16d0 2023-10-06T03:33:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.197+0000 7f8ca5d9b700 1 --2- 172.21.15.55:0/954600953 >> v2:172.21.15.163:3300/0 conn(0x7f8ca00b4e70 0x7f8ca00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:49.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.197+0000 7f8ca4d99700 1 -- 172.21.15.55:0/954600953 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935843901 0 0) 0x7f8ca00b2080 con 0x7f8ca015d240 2023-10-06T03:33:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.197+0000 7f8ca559a700 1 --2- 172.21.15.55:0/954600953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ca00b16d0 0x7f8ca00b1b40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0064910 tx=0x7f8c900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f69bdb7b0a66110 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.198+0000 7f8ca4d99700 1 -- 172.21.15.55:0/954600953 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9000b6b0 con 0x7f8ca00b16d0 2023-10-06T03:33:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.198+0000 7f8ca4d99700 1 -- 172.21.15.55:0/954600953 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9000b810 con 0x7f8ca00b16d0 2023-10-06T03:33:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.198+0000 7f8ca4d99700 1 -- 172.21.15.55:0/954600953 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9000bbb0 con 0x7f8ca00b16d0 2023-10-06T03:33:49.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.198+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/954600953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ca00b16d0 msgr2=0x7f8ca00b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:49.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.198+0000 7f8ca6d9d700 1 --2- 172.21.15.55:0/954600953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ca00b16d0 0x7f8ca00b1b40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0064910 tx=0x7f8c900091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.198+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/954600953 shutdown_connections 2023-10-06T03:33:49.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.198+0000 7f8ca6d9d700 1 --2- 172.21.15.55:0/954600953 >> v2:172.21.15.163:3300/0 conn(0x7f8ca00b4e70 0x7f8ca00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.198+0000 7f8ca6d9d700 1 --2- 172.21.15.55:0/954600953 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ca00b16d0 0x7f8ca00b1b40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.198+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/954600953 >> 172.21.15.55:0/954600953 conn(0x7f8ca00d10c0 msgr2=0x7f8ca00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:49.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.198+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/954600953 shutdown_connections 2023-10-06T03:33:49.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.199+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/954600953 wait complete. 2023-10-06T03:33:49.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.200+0000 7f8ca6d9d700 1 Processor -- start 2023-10-06T03:33:49.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.200+0000 7f8ca6d9d700 1 -- start start 2023-10-06T03:33:49.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.201+0000 7f8ca6d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00b16d0 0x7f8ca00cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.201+0000 7f8ca6d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ca00b4e70 0x7f8ca0145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.201+0000 7f8ca6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ca015d240 0x7f8ca0149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.201+0000 7f8ca6d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca01482e0 con 0x7f8ca015d240 2023-10-06T03:33:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.201+0000 7f8ca6d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca01476d0 con 0x7f8ca00b16d0 2023-10-06T03:33:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.201+0000 7f8ca6d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca01479b0 con 0x7f8ca00b4e70 2023-10-06T03:33:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.201+0000 7f8ca659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ca015d240 0x7f8ca0149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.201+0000 7f8ca659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ca015d240 0x7f8ca0149670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36160/0 (socket says 172.21.15.55:36160) 2023-10-06T03:33:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.201+0000 7f8ca659c700 1 -- 172.21.15.55:0/1892414475 learned_addr learned my addr 172.21.15.55:0/1892414475 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.201+0000 7f8ca559a700 1 --2- 172.21.15.55:0/1892414475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ca00b4e70 0x7f8ca0145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.201+0000 7f8ca5d9b700 1 --2- 172.21.15.55:0/1892414475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00b16d0 0x7f8ca00cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.202+0000 7f8ca659c700 1 -- 172.21.15.55:0/1892414475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ca00b4e70 msgr2=0x7f8ca0145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:49.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.202+0000 7f8ca659c700 1 --2- 172.21.15.55:0/1892414475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ca00b4e70 0x7f8ca0145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.202+0000 7f8ca659c700 1 -- 172.21.15.55:0/1892414475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00b16d0 msgr2=0x7f8ca00cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.202+0000 7f8ca659c700 1 --2- 172.21.15.55:0/1892414475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00b16d0 0x7f8ca00cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.202+0000 7f8ca659c700 1 -- 172.21.15.55:0/1892414475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c90009420 con 0x7f8ca015d240 2023-10-06T03:33:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.202+0000 7f8ca659c700 1 --2- 172.21.15.55:0/1892414475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ca015d240 0x7f8ca0149670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800cbf0 tx=0x7f8c980019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.202+0000 7f8c95ffb700 1 -- 172.21.15.55:0/1892414475 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9800a9c0 con 0x7f8ca015d240 2023-10-06T03:33:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.202+0000 7f8c95ffb700 1 -- 172.21.15.55:0/1892414475 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9800ab20 con 0x7f8ca015d240 2023-10-06T03:33:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.202+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/1892414475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca0147c90 con 0x7f8ca015d240 2023-10-06T03:33:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.202+0000 7f8c95ffb700 1 -- 172.21.15.55:0/1892414475 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c980070f0 con 0x7f8ca015d240 2023-10-06T03:33:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.202+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/1892414475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca00cc2e0 con 0x7f8ca015d240 2023-10-06T03:33:49.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.203+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/1892414475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca0148450 con 0x7f8ca015d240 2023-10-06T03:33:49.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.204+0000 7f8c95ffb700 1 -- 172.21.15.55:0/1892414475 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c98007520 con 0x7f8ca015d240 2023-10-06T03:33:49.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.205+0000 7f8c95ffb700 1 --2- 172.21.15.55:0/1892414475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8c84066f60 0x7f8c84069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:49.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.205+0000 7f8c95ffb700 1 -- 172.21.15.55:0/1892414475 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8c98084b00 con 0x7f8ca015d240 2023-10-06T03:33:49.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.205+0000 7f8ca5d9b700 1 --2- 172.21.15.55:0/1892414475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8c84066f60 0x7f8c84069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:49.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.206+0000 7f8ca5d9b700 1 --2- 172.21.15.55:0/1892414475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8c84066f60 0x7f8c84069410 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c006f00 tx=0x7f8c9c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:49.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.209+0000 7f8c95ffb700 1 -- 172.21.15.55:0/1892414475 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c98086e60 con 0x7f8ca015d240 2023-10-06T03:33:49.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.411+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/1892414475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ca004bb80 con 0x7f8ca015d240 2023-10-06T03:33:49.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.411+0000 7f8c95ffb700 1 -- 172.21.15.55:0/1892414475 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c98051bf0 con 0x7f8ca015d240 2023-10-06T03:33:49.413 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T03:33:49.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.413+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/1892414475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8c84066f60 msgr2=0x7f8c84069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:49.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.413+0000 7f8ca6d9d700 1 --2- 172.21.15.55:0/1892414475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8c84066f60 0x7f8c84069410 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c006f00 tx=0x7f8c9c008040 comp rx=0 tx=0).stop 2023-10-06T03:33:49.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.413+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/1892414475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ca015d240 msgr2=0x7f8ca0149670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:49.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.413+0000 7f8ca6d9d700 1 --2- 172.21.15.55:0/1892414475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ca015d240 0x7f8ca0149670 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800cbf0 tx=0x7f8c980019b0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.414+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/1892414475 shutdown_connections 2023-10-06T03:33:49.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.414+0000 7f8ca6d9d700 1 --2- 172.21.15.55:0/1892414475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ca015d240 0x7f8ca0149670 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.414+0000 7f8ca6d9d700 1 --2- 172.21.15.55:0/1892414475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ca00b16d0 0x7f8ca00cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.414+0000 7f8ca6d9d700 1 --2- 172.21.15.55:0/1892414475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8c84066f60 0x7f8c84069410 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.414+0000 7f8ca6d9d700 1 --2- 172.21.15.55:0/1892414475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ca00b4e70 0x7f8ca0145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.414+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/1892414475 >> 172.21.15.55:0/1892414475 conn(0x7f8ca00d10c0 msgr2=0x7f8ca015c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:49.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.414+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/1892414475 shutdown_connections 2023-10-06T03:33:49.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.414+0000 7f8ca6d9d700 1 -- 172.21.15.55:0/1892414475 wait complete. 2023-10-06T03:33:49.432 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:49.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.618+0000 7f38a6b9d700 1 Processor -- start 2023-10-06T03:33:49.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.618+0000 7f38a6b9d700 1 -- start start 2023-10-06T03:33:49.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.619+0000 7f38a6b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f38a01566e0 0x7f38a00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:49.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.619+0000 7f38a6b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38a00cb900 0x7f38a00ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:49.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.619+0000 7f38a6b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a00ca860 con 0x7f38a0156310 2023-10-06T03:33:49.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.619+0000 7f38a6b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a00cab40 con 0x7f38a01566e0 2023-10-06T03:33:49.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.619+0000 7f38a6b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00cae20 con 0x7f38a00cb900 2023-10-06T03:33:49.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.619+0000 7f38a539a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f38a01566e0 0x7f38a00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:49.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.620+0000 7f38a539a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f38a01566e0 0x7f38a00ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36170/0 (socket says 172.21.15.55:36170) 2023-10-06T03:33:49.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.620+0000 7f38a539a700 1 -- 172.21.15.55:0/2316418195 learned_addr learned my addr 172.21.15.55:0/2316418195 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:49.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.620+0000 7f38a639c700 1 --2- 172.21.15.55:0/2316418195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38a00cb900 0x7f38a00ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:49.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.620+0000 7f38a539a700 1 -- 172.21.15.55:0/2316418195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38a00cb900 msgr2=0x7f38a00ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:49.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.620+0000 7f38a539a700 1 --2- 172.21.15.55:0/2316418195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38a00cb900 0x7f38a00ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.620+0000 7f38a539a700 1 -- 172.21.15.55:0/2316418195 >> v1:172.21.15.163:6789/0 conn(0x7f38a0156310 legacy=0x7f38a00c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:49.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.620+0000 7f38a539a700 1 -- 172.21.15.55:0/2316418195 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a00cb100 con 0x7f38a01566e0 2023-10-06T03:33:49.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.620+0000 7f38a539a700 1 --2- 172.21.15.55:0/2316418195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a01566e0 0x7f38a00ae000 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f38900099b0 tx=0x7f3890009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a598f38d22d9c19 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:49.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.620+0000 7f38a4b99700 1 -- 172.21.15.55:0/2316418195 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890009ea0 con 0x7f38a01566e0 2023-10-06T03:33:49.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.621+0000 7f38a4b99700 1 -- 172.21.15.55:0/2316418195 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389000b860 con 0x7f38a01566e0 2023-10-06T03:33:49.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.621+0000 7f38a4b99700 1 -- 172.21.15.55:0/2316418195 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389000bc00 con 0x7f38a01566e0 2023-10-06T03:33:49.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.621+0000 7f38a6b9d700 1 -- 172.21.15.55:0/2316418195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a01566e0 msgr2=0x7f38a00ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:49.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.621+0000 7f38a6b9d700 1 --2- 172.21.15.55:0/2316418195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a01566e0 0x7f38a00ae000 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f38900099b0 tx=0x7f3890009250 comp rx=0 tx=0).stop 2023-10-06T03:33:49.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.621+0000 7f38a6b9d700 1 -- 172.21.15.55:0/2316418195 shutdown_connections 2023-10-06T03:33:49.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.621+0000 7f38a6b9d700 1 --2- 172.21.15.55:0/2316418195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38a00cb900 0x7f38a00ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.621+0000 7f38a6b9d700 1 --2- 172.21.15.55:0/2316418195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a01566e0 0x7f38a00ae000 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.621+0000 7f38a6b9d700 1 -- 172.21.15.55:0/2316418195 >> 172.21.15.55:0/2316418195 conn(0x7f38a00cc1f0 msgr2=0x7f38a00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:49.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.621+0000 7f38a6b9d700 1 -- 172.21.15.55:0/2316418195 shutdown_connections 2023-10-06T03:33:49.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.621+0000 7f38a6b9d700 1 -- 172.21.15.55:0/2316418195 wait complete. 2023-10-06T03:33:49.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.623+0000 7f38a6b9d700 1 Processor -- start 2023-10-06T03:33:49.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.623+0000 7f38a6b9d700 1 -- start start 2023-10-06T03:33:49.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a6b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38a00cb900 0x7f38a0150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:49.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a6b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38a0156310 0x7f38a0148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:49.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a6b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a01566e0 0x7f38a0147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:49.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a6b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a016a7f0 con 0x7f38a01566e0 2023-10-06T03:33:49.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a6b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a0150da0 con 0x7f38a00cb900 2023-10-06T03:33:49.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a6b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a0169be0 con 0x7f38a0156310 2023-10-06T03:33:49.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a639c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a01566e0 0x7f38a0147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:49.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a639c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a01566e0 0x7f38a0147dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36184/0 (socket says 172.21.15.55:36184) 2023-10-06T03:33:49.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a639c700 1 -- 172.21.15.55:0/3676626957 learned_addr learned my addr 172.21.15.55:0/3676626957 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:49.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a539a700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38a0156310 0x7f38a0148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:49.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a5b9b700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38a00cb900 0x7f38a0150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:49.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a639c700 1 -- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38a0156310 msgr2=0x7f38a0148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:49.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a639c700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38a0156310 0x7f38a0148ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a639c700 1 -- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38a00cb900 msgr2=0x7f38a0150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:49.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a639c700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38a00cb900 0x7f38a0150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a639c700 1 -- 172.21.15.55:0/3676626957 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38900094f0 con 0x7f38a01566e0 2023-10-06T03:33:49.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a539a700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38a0156310 0x7f38a0148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:49.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.624+0000 7f38a639c700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a01566e0 0x7f38a0147dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3898009df0 tx=0x7f389800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:49.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.625+0000 7f3895ffb700 1 -- 172.21.15.55:0/3676626957 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389800caf0 con 0x7f38a01566e0 2023-10-06T03:33:49.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.625+0000 7f3895ffb700 1 -- 172.21.15.55:0/3676626957 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389800f040 con 0x7f38a01566e0 2023-10-06T03:33:49.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.625+0000 7f38a6b9d700 1 -- 172.21.15.55:0/3676626957 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a0169ec0 con 0x7f38a01566e0 2023-10-06T03:33:49.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.625+0000 7f3895ffb700 1 -- 172.21.15.55:0/3676626957 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389800cec0 con 0x7f38a01566e0 2023-10-06T03:33:49.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.625+0000 7f38a6b9d700 1 -- 172.21.15.55:0/3676626957 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a016a480 con 0x7f38a01566e0 2023-10-06T03:33:49.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.626+0000 7f38a6b9d700 1 -- 172.21.15.55:0/3676626957 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a0150fc0 con 0x7f38a01566e0 2023-10-06T03:33:49.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.627+0000 7f3895ffb700 1 -- 172.21.15.55:0/3676626957 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3898014690 con 0x7f38a01566e0 2023-10-06T03:33:49.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.628+0000 7f3895ffb700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3884066f30 0x7f38840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:49.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.628+0000 7f3895ffb700 1 -- 172.21.15.55:0/3676626957 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f38980850c0 con 0x7f38a01566e0 2023-10-06T03:33:49.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.628+0000 7f38a5b9b700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3884066f30 0x7f38840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:49.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.629+0000 7f38a5b9b700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3884066f30 0x7f38840693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f389c003860 tx=0x7f389c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:49.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.632+0000 7f3895ffb700 1 -- 172.21.15.55:0/3676626957 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38a0150fc0 con 0x7f38a01566e0 2023-10-06T03:33:49.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.836+0000 7f38a6b9d700 1 -- 172.21.15.55:0/3676626957 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38a0150fc0 con 0x7f38a01566e0 2023-10-06T03:33:49.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.837+0000 7f3895ffb700 1 -- 172.21.15.55:0/3676626957 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f38a0150fc0 con 0x7f38a01566e0 2023-10-06T03:33:49.838 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:49.839 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:16.470817+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:49.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.839+0000 7f38a6b9d700 1 -- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3884066f30 msgr2=0x7f38840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:49.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.839+0000 7f38a6b9d700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3884066f30 0x7f38840693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f389c003860 tx=0x7f389c009040 comp rx=0 tx=0).stop 2023-10-06T03:33:49.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.839+0000 7f38a6b9d700 1 -- 172.21.15.55:0/3676626957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a01566e0 msgr2=0x7f38a0147dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:49.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.839+0000 7f38a6b9d700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a01566e0 0x7f38a0147dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3898009df0 tx=0x7f389800b040 comp rx=0 tx=0).stop 2023-10-06T03:33:49.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.839+0000 7f38a6b9d700 1 -- 172.21.15.55:0/3676626957 shutdown_connections 2023-10-06T03:33:49.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.839+0000 7f38a6b9d700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38a01566e0 0x7f38a0147dc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.839+0000 7f38a6b9d700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38a00cb900 0x7f38a0150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.839+0000 7f38a6b9d700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3884066f30 0x7f38840693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.839+0000 7f38a6b9d700 1 --2- 172.21.15.55:0/3676626957 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38a0156310 0x7f38a0148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:49.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.839+0000 7f38a6b9d700 1 -- 172.21.15.55:0/3676626957 >> 172.21.15.55:0/3676626957 conn(0x7f38a00cc1f0 msgr2=0x7f38a006ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:49.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.839+0000 7f38a6b9d700 1 -- 172.21.15.55:0/3676626957 shutdown_connections 2023-10-06T03:33:49.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:49.839+0000 7f38a6b9d700 1 -- 172.21.15.55:0/3676626957 wait complete. 2023-10-06T03:33:49.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:33:13.543361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:33:14.341572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:33:49.858 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:33:50.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.047+0000 7f54f359e700 1 Processor -- start 2023-10-06T03:33:50.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.047+0000 7f54f359e700 1 -- start start 2023-10-06T03:33:50.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.048+0000 7f54f359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f54f41407e0 0x7f54f406a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.048+0000 7f54f359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f54f406dc00 0x7f54f406a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.048+0000 7f54f359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f40c6820 con 0x7f54f4140bb0 2023-10-06T03:33:50.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.048+0000 7f54f359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f40c6de0 con 0x7f54f406dc00 2023-10-06T03:33:50.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.048+0000 7f54f359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f4141270 con 0x7f54f41407e0 2023-10-06T03:33:50.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.048+0000 7f54f1d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f54f4140bb0 0x7f54f406d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59528/0 (socket says 172.21.15.55:59528) 2023-10-06T03:33:50.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.048+0000 7f54f1d9b700 1 -- 172.21.15.55:0/2869414521 learned_addr learned my addr 172.21.15.55:0/2869414521 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:50.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.048+0000 7f54f2d9d700 1 --2- 172.21.15.55:0/2869414521 >> v2:172.21.15.55:3300/0 conn(0x7f54f406dc00 0x7f54f406a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.049+0000 7f54f259c700 1 --2- 172.21.15.55:0/2869414521 >> v2:172.21.15.163:3300/0 conn(0x7f54f41407e0 0x7f54f406a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.049+0000 7f54f2d9d700 1 -- 172.21.15.55:0/2869414521 >> v2:172.21.15.163:3300/0 conn(0x7f54f41407e0 msgr2=0x7f54f406a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.049+0000 7f54f2d9d700 1 --2- 172.21.15.55:0/2869414521 >> v2:172.21.15.163:3300/0 conn(0x7f54f41407e0 0x7f54f406a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.049+0000 7f54f2d9d700 1 -- 172.21.15.55:0/2869414521 >> v1:172.21.15.55:6789/0 conn(0x7f54f4140bb0 legacy=0x7f54f406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.049+0000 7f54f2d9d700 1 -- 172.21.15.55:0/2869414521 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f406acf0 con 0x7f54f406dc00 2023-10-06T03:33:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.049+0000 7f54f159a700 1 -- 172.21.15.55:0/2869414521 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017884173 0 0) 0x7f54f40c6820 con 0x7f54f4140bb0 2023-10-06T03:33:50.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.049+0000 7f54f2d9d700 1 --2- 172.21.15.55:0/2869414521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f406dc00 0x7f54f406a6c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f54f406f9a0 tx=0x7f54ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ecd608960063c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:50.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.049+0000 7f54f159a700 1 -- 172.21.15.55:0/2869414521 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00e070 con 0x7f54f406dc00 2023-10-06T03:33:50.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.050+0000 7f54f159a700 1 -- 172.21.15.55:0/2869414521 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ec00bb60 con 0x7f54f406dc00 2023-10-06T03:33:50.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.050+0000 7f54f159a700 1 -- 172.21.15.55:0/2869414521 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec012430 con 0x7f54f406dc00 2023-10-06T03:33:50.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.050+0000 7f54f359e700 1 -- 172.21.15.55:0/2869414521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f406dc00 msgr2=0x7f54f406a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.050+0000 7f54f359e700 1 --2- 172.21.15.55:0/2869414521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f406dc00 0x7f54f406a6c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f54f406f9a0 tx=0x7f54ec009780 comp rx=0 tx=0).stop 2023-10-06T03:33:50.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.050+0000 7f54f359e700 1 -- 172.21.15.55:0/2869414521 shutdown_connections 2023-10-06T03:33:50.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.050+0000 7f54f359e700 1 --2- 172.21.15.55:0/2869414521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f406dc00 0x7f54f406a6c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.050+0000 7f54f359e700 1 --2- 172.21.15.55:0/2869414521 >> v2:172.21.15.163:3300/0 conn(0x7f54f41407e0 0x7f54f406a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.050+0000 7f54f359e700 1 -- 172.21.15.55:0/2869414521 >> 172.21.15.55:0/2869414521 conn(0x7f54f40cc1e0 msgr2=0x7f54f40cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:50.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.050+0000 7f54f359e700 1 -- 172.21.15.55:0/2869414521 shutdown_connections 2023-10-06T03:33:50.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.050+0000 7f54f359e700 1 -- 172.21.15.55:0/2869414521 wait complete. 2023-10-06T03:33:50.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.052+0000 7f54f359e700 1 Processor -- start 2023-10-06T03:33:50.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.052+0000 7f54f359e700 1 -- start start 2023-10-06T03:33:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.053+0000 7f54f359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f406dc00 0x7f54f413a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.053+0000 7f54f359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54f41407e0 0x7f54f40ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.053+0000 7f54f359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54f4140bb0 0x7f54f40ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.053+0000 7f54f359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f4169e80 con 0x7f54f406dc00 2023-10-06T03:33:50.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.053+0000 7f54f359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f416ac90 con 0x7f54f4140bb0 2023-10-06T03:33:50.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.053+0000 7f54f359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f416a080 con 0x7f54f41407e0 2023-10-06T03:33:50.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.053+0000 7f54f259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f406dc00 0x7f54f413a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.053+0000 7f54f259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f406dc00 0x7f54f413a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36204/0 (socket says 172.21.15.55:36204) 2023-10-06T03:33:50.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.053+0000 7f54f259c700 1 -- 172.21.15.55:0/2903450010 learned_addr learned my addr 172.21.15.55:0/2903450010 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:50.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.053+0000 7f54f259c700 1 -- 172.21.15.55:0/2903450010 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54f41407e0 msgr2=0x7f54f40ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.053+0000 7f54f259c700 1 --2- 172.21.15.55:0/2903450010 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54f41407e0 0x7f54f40ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.053+0000 7f54f259c700 1 -- 172.21.15.55:0/2903450010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54f4140bb0 msgr2=0x7f54f40ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.053+0000 7f54f259c700 1 --2- 172.21.15.55:0/2903450010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54f4140bb0 0x7f54f40ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.053+0000 7f54f259c700 1 -- 172.21.15.55:0/2903450010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec009a20 con 0x7f54f406dc00 2023-10-06T03:33:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.054+0000 7f54f259c700 1 --2- 172.21.15.55:0/2903450010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f406dc00 0x7f54f413a220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f54e400ec80 tx=0x7f54e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.054+0000 7f54e27fc700 1 -- 172.21.15.55:0/2903450010 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e400c9d0 con 0x7f54f406dc00 2023-10-06T03:33:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.054+0000 7f54e27fc700 1 -- 172.21.15.55:0/2903450010 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e400cb30 con 0x7f54f406dc00 2023-10-06T03:33:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.054+0000 7f54e27fc700 1 -- 172.21.15.55:0/2903450010 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4013470 con 0x7f54f406dc00 2023-10-06T03:33:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.054+0000 7f54f359e700 1 -- 172.21.15.55:0/2903450010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f416a360 con 0x7f54f406dc00 2023-10-06T03:33:50.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.054+0000 7f54f359e700 1 -- 172.21.15.55:0/2903450010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f416a920 con 0x7f54f406dc00 2023-10-06T03:33:50.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.055+0000 7f54f359e700 1 -- 172.21.15.55:0/2903450010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f406c450 con 0x7f54f406dc00 2023-10-06T03:33:50.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.056+0000 7f54e27fc700 1 -- 172.21.15.55:0/2903450010 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f54e4013820 con 0x7f54f406dc00 2023-10-06T03:33:50.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.058+0000 7f54e27fc700 1 --2- 172.21.15.55:0/2903450010 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f54d4066f30 0x7f54d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.058+0000 7f54e27fc700 1 -- 172.21.15.55:0/2903450010 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f54e4084ac0 con 0x7f54f406dc00 2023-10-06T03:33:50.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.058+0000 7f54f1d9b700 1 --2- 172.21.15.55:0/2903450010 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f54d4066f30 0x7f54d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.059+0000 7f54f1d9b700 1 --2- 172.21.15.55:0/2903450010 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f54d4066f30 0x7f54d40693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f54f406fd30 tx=0x7f54e8007bd0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:50.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.062+0000 7f54e27fc700 1 -- 172.21.15.55:0/2903450010 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54e4051b30 con 0x7f54f406dc00 2023-10-06T03:33:50.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.263+0000 7f54f359e700 1 -- 172.21.15.55:0/2903450010 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f54f406afd0 con 0x7f54f406dc00 2023-10-06T03:33:50.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.263+0000 7f54e27fc700 1 -- 172.21.15.55:0/2903450010 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f54e4055150 con 0x7f54f406dc00 2023-10-06T03:33:50.265 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:33:50.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.265+0000 7f54f359e700 1 -- 172.21.15.55:0/2903450010 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f54d4066f30 msgr2=0x7f54d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.265+0000 7f54f359e700 1 --2- 172.21.15.55:0/2903450010 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f54d4066f30 0x7f54d40693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f54f406fd30 tx=0x7f54e8007bd0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.265+0000 7f54f359e700 1 -- 172.21.15.55:0/2903450010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f406dc00 msgr2=0x7f54f413a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.265+0000 7f54f359e700 1 --2- 172.21.15.55:0/2903450010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f406dc00 0x7f54f413a220 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f54e400ec80 tx=0x7f54e40091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.266+0000 7f54f359e700 1 -- 172.21.15.55:0/2903450010 shutdown_connections 2023-10-06T03:33:50.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.266+0000 7f54f359e700 1 --2- 172.21.15.55:0/2903450010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54f4140bb0 0x7f54f40ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.266+0000 7f54f359e700 1 --2- 172.21.15.55:0/2903450010 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f406dc00 0x7f54f413a220 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.266+0000 7f54f359e700 1 --2- 172.21.15.55:0/2903450010 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f54d4066f30 0x7f54d40693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.266+0000 7f54f359e700 1 --2- 172.21.15.55:0/2903450010 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54f41407e0 0x7f54f40ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.266+0000 7f54f359e700 1 -- 172.21.15.55:0/2903450010 >> 172.21.15.55:0/2903450010 conn(0x7f54f40cc1e0 msgr2=0x7f54f40c1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:50.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.266+0000 7f54f359e700 1 -- 172.21.15.55:0/2903450010 shutdown_connections 2023-10-06T03:33:50.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.266+0000 7f54f359e700 1 -- 172.21.15.55:0/2903450010 wait complete. 2023-10-06T03:33:50.284 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:33:50.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.472+0000 7f341939d700 1 Processor -- start 2023-10-06T03:33:50.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.472+0000 7f341939d700 1 -- start start 2023-10-06T03:33:50.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.474+0000 7f341939d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f34140c9db0 0x7f341406a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.474+0000 7f341939d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341406e150 con 0x7f3414153900 2023-10-06T03:33:50.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.474+0000 7f341939d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34140c7800 con 0x7f3414157130 2023-10-06T03:33:50.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.474+0000 7f341939d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34140c7610 con 0x7f34140c9db0 2023-10-06T03:33:50.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.474+0000 7f34137fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f34140c9db0 0x7f341406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.474+0000 7f34137fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f34140c9db0 0x7f341406a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36206/0 (socket says 172.21.15.55:36206) 2023-10-06T03:33:50.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.474+0000 7f34137fe700 1 -- 172.21.15.55:0/2502490636 learned_addr learned my addr 172.21.15.55:0/2502490636 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:50.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.474+0000 7f3413fff700 1 --1- 172.21.15.55:0/2502490636 >> v1:172.21.15.55:6789/0 conn(0x7f3414153900 0x7f34140c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59544/0 (socket says 172.21.15.55:59544) 2023-10-06T03:33:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.474+0000 7f34137fe700 1 -- 172.21.15.55:0/2502490636 >> v1:172.21.15.163:6789/0 conn(0x7f3414157130 legacy=0x7f341406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.474+0000 7f34137fe700 1 -- 172.21.15.55:0/2502490636 >> v1:172.21.15.55:6789/0 conn(0x7f3414153900 legacy=0x7f34140c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.474+0000 7f34137fe700 1 -- 172.21.15.55:0/2502490636 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34140c7ae0 con 0x7f34140c9db0 2023-10-06T03:33:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.475+0000 7f34137fe700 1 --2- 172.21.15.55:0/2502490636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34140c9db0 0x7f341406a000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f34080099b0 tx=0x7f3408009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7428f97e8d4e93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.476+0000 7f3412ffd700 1 -- 172.21.15.55:0/2502490636 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340800b820 con 0x7f34140c9db0 2023-10-06T03:33:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.476+0000 7f3412ffd700 1 -- 172.21.15.55:0/2502490636 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340800b980 con 0x7f34140c9db0 2023-10-06T03:33:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.476+0000 7f3412ffd700 1 -- 172.21.15.55:0/2502490636 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340800bd20 con 0x7f34140c9db0 2023-10-06T03:33:50.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.476+0000 7f341939d700 1 -- 172.21.15.55:0/2502490636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34140c9db0 msgr2=0x7f341406a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.476+0000 7f341939d700 1 --2- 172.21.15.55:0/2502490636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34140c9db0 0x7f341406a000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f34080099b0 tx=0x7f3408009250 comp rx=0 tx=0).stop 2023-10-06T03:33:50.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.476+0000 7f341939d700 1 -- 172.21.15.55:0/2502490636 shutdown_connections 2023-10-06T03:33:50.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.476+0000 7f341939d700 1 --2- 172.21.15.55:0/2502490636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34140c9db0 0x7f341406a000 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.476+0000 7f341939d700 1 -- 172.21.15.55:0/2502490636 >> 172.21.15.55:0/2502490636 conn(0x7f34140cc1e0 msgr2=0x7f34140cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:50.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.476+0000 7f341939d700 1 -- 172.21.15.55:0/2502490636 shutdown_connections 2023-10-06T03:33:50.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.476+0000 7f341939d700 1 -- 172.21.15.55:0/2502490636 wait complete. 2023-10-06T03:33:50.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.479+0000 7f341939d700 1 Processor -- start 2023-10-06T03:33:50.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.479+0000 7f341939d700 1 -- start start 2023-10-06T03:33:50.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f341939d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34140c9db0 0x7f34141593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f341939d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3414153900 0x7f34140af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f341939d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3414157130 0x7f34140abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f341939d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34140afaf0 con 0x7f3414153900 2023-10-06T03:33:50.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f341939d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34140ac020 con 0x7f34140c9db0 2023-10-06T03:33:50.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f341939d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3414157730 con 0x7f3414157130 2023-10-06T03:33:50.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f34137fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3414153900 0x7f34140af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f34137fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3414153900 0x7f34140af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36214/0 (socket says 172.21.15.55:36214) 2023-10-06T03:33:50.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f34137fe700 1 -- 172.21.15.55:0/2862168404 learned_addr learned my addr 172.21.15.55:0/2862168404 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:50.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f3413fff700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34140c9db0 0x7f34141593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f3418b9c700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3414157130 0x7f34140abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f34137fe700 1 -- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3414157130 msgr2=0x7f34140abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f34137fe700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3414157130 0x7f34140abae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f34137fe700 1 -- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34140c9db0 msgr2=0x7f34141593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f34137fe700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34140c9db0 0x7f34141593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f34137fe700 1 -- 172.21.15.55:0/2862168404 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34080094f0 con 0x7f3414153900 2023-10-06T03:33:50.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.480+0000 7f3413fff700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34140c9db0 0x7f34141593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:50.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.481+0000 7f34137fe700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3414153900 0x7f34140af5b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f341406ce80 tx=0x7f3408017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:50.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.481+0000 7f33fbfff700 1 -- 172.21.15.55:0/2862168404 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340801d070 con 0x7f3414153900 2023-10-06T03:33:50.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.481+0000 7f33fbfff700 1 -- 172.21.15.55:0/2862168404 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3408021410 con 0x7f3414153900 2023-10-06T03:33:50.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.481+0000 7f341939d700 1 -- 172.21.15.55:0/2862168404 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34140ac250 con 0x7f3414153900 2023-10-06T03:33:50.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.481+0000 7f33fbfff700 1 -- 172.21.15.55:0/2862168404 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3408021790 con 0x7f3414153900 2023-10-06T03:33:50.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.481+0000 7f341939d700 1 -- 172.21.15.55:0/2862168404 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34140ac810 con 0x7f3414153900 2023-10-06T03:33:50.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.482+0000 7f341939d700 1 -- 172.21.15.55:0/2862168404 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341406d5b0 con 0x7f3414153900 2023-10-06T03:33:50.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.483+0000 7f33fbfff700 1 -- 172.21.15.55:0/2862168404 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f34080103f0 con 0x7f3414153900 2023-10-06T03:33:50.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.484+0000 7f33fbfff700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3400066f30 0x7f34000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.484+0000 7f33fbfff700 1 -- 172.21.15.55:0/2862168404 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3408085bf0 con 0x7f3414153900 2023-10-06T03:33:50.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.484+0000 7f3413fff700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3400066f30 0x7f34000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.485+0000 7f3413fff700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3400066f30 0x7f34000693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3404007fd0 tx=0x7f3404007e10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:50.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.489+0000 7f33fbfff700 1 -- 172.21.15.55:0/2862168404 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3408052c60 con 0x7f3414153900 2023-10-06T03:33:50.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.692+0000 7f341939d700 1 -- 172.21.15.55:0/2862168404 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34140acb50 con 0x7f3414153900 2023-10-06T03:33:50.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.693+0000 7f33fbfff700 1 -- 172.21.15.55:0/2862168404 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3408026020 con 0x7f3414153900 2023-10-06T03:33:50.694 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:33:50.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.695+0000 7f341939d700 1 -- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3400066f30 msgr2=0x7f34000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.695+0000 7f341939d700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3400066f30 0x7f34000693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3404007fd0 tx=0x7f3404007e10 comp rx=0 tx=0).stop 2023-10-06T03:33:50.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.695+0000 7f341939d700 1 -- 172.21.15.55:0/2862168404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3414153900 msgr2=0x7f34140af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.695+0000 7f341939d700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3414153900 0x7f34140af5b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f341406ce80 tx=0x7f3408017000 comp rx=0 tx=0).stop 2023-10-06T03:33:50.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.695+0000 7f341939d700 1 -- 172.21.15.55:0/2862168404 shutdown_connections 2023-10-06T03:33:50.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.695+0000 7f341939d700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3400066f30 0x7f34000693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.695+0000 7f341939d700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3414157130 0x7f34140abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.695+0000 7f341939d700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34140c9db0 0x7f34141593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.698 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.695+0000 7f341939d700 1 --2- 172.21.15.55:0/2862168404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3414153900 0x7f34140af5b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.698 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.695+0000 7f341939d700 1 -- 172.21.15.55:0/2862168404 >> 172.21.15.55:0/2862168404 conn(0x7f34140cc1e0 msgr2=0x7f341406a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:50.698 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.695+0000 7f341939d700 1 -- 172.21.15.55:0/2862168404 shutdown_connections 2023-10-06T03:33:50.698 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.695+0000 7f341939d700 1 -- 172.21.15.55:0/2862168404 wait complete. 2023-10-06T03:33:50.715 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:33:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.903+0000 7ff3cb59d700 1 Processor -- start 2023-10-06T03:33:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.903+0000 7ff3cb59d700 1 -- start start 2023-10-06T03:33:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.904+0000 7ff3cb59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff3c406f060 0x7ff3c40ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.904+0000 7ff3cb59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3c406f4b0 0x7ff3c40ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.904+0000 7ff3cb59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c406ebd0 con 0x7ff3c40cb1a0 2023-10-06T03:33:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.904+0000 7ff3cb59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c415b2e0 con 0x7ff3c406f060 2023-10-06T03:33:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.904+0000 7ff3cb59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c415aa70 con 0x7ff3c406f4b0 2023-10-06T03:33:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.905+0000 7ff3c9d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff3c406f060 0x7ff3c40ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.905+0000 7ff3c9d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff3c406f060 0x7ff3c40ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36220/0 (socket says 172.21.15.55:36220) 2023-10-06T03:33:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.905+0000 7ff3c9d9a700 1 -- 172.21.15.55:0/1545655756 learned_addr learned my addr 172.21.15.55:0/1545655756 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.905+0000 7ff3cad9c700 1 --2- 172.21.15.55:0/1545655756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3c406f4b0 0x7ff3c40ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.905+0000 7ff3c9d9a700 1 -- 172.21.15.55:0/1545655756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3c406f4b0 msgr2=0x7ff3c40ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.905+0000 7ff3c9d9a700 1 --2- 172.21.15.55:0/1545655756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3c406f4b0 0x7ff3c40ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.905+0000 7ff3c9d9a700 1 -- 172.21.15.55:0/1545655756 >> v1:172.21.15.163:6789/0 conn(0x7ff3c40cb1a0 legacy=0x7ff3c415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.905+0000 7ff3c9d9a700 1 -- 172.21.15.55:0/1545655756 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c415ad20 con 0x7ff3c406f060 2023-10-06T03:33:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.905+0000 7ff3c9d9a700 1 --2- 172.21.15.55:0/1545655756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c406f060 0x7ff3c40ae0d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40099b0 tx=0x7ff3b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43e684029663c49 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:50.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.906+0000 7ff3c9599700 1 -- 172.21.15.55:0/1545655756 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400b8f0 con 0x7ff3c406f060 2023-10-06T03:33:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.906+0000 7ff3c9599700 1 -- 172.21.15.55:0/1545655756 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b400ba50 con 0x7ff3c406f060 2023-10-06T03:33:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.906+0000 7ff3c9599700 1 -- 172.21.15.55:0/1545655756 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400bdf0 con 0x7ff3c406f060 2023-10-06T03:33:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.906+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1545655756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c406f060 msgr2=0x7ff3c40ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.906+0000 7ff3cb59d700 1 --2- 172.21.15.55:0/1545655756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c406f060 0x7ff3c40ae0d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40099b0 tx=0x7ff3b4009250 comp rx=0 tx=0).stop 2023-10-06T03:33:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.906+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1545655756 shutdown_connections 2023-10-06T03:33:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.906+0000 7ff3cb59d700 1 --2- 172.21.15.55:0/1545655756 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3c406f4b0 0x7ff3c40ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.906+0000 7ff3cb59d700 1 --2- 172.21.15.55:0/1545655756 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c406f060 0x7ff3c40ae0d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.906+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1545655756 >> 172.21.15.55:0/1545655756 conn(0x7ff3c40cc1e0 msgr2=0x7ff3c40cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.906+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1545655756 shutdown_connections 2023-10-06T03:33:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.906+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1545655756 wait complete. 2023-10-06T03:33:50.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.908+0000 7ff3cb59d700 1 Processor -- start 2023-10-06T03:33:50.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.908+0000 7ff3cb59d700 1 -- start start 2023-10-06T03:33:50.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.909+0000 7ff3cb59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3c406f060 0x7ff3c415c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.909+0000 7ff3cb59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c406f4b0 0x7ff3c414f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.909+0000 7ff3cb59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3c40cb1a0 0x7ff3c41502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.909+0000 7ff3cb59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40c4eb0 con 0x7ff3c406f4b0 2023-10-06T03:33:50.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.909+0000 7ff3cb59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c40c48d0 con 0x7ff3c406f060 2023-10-06T03:33:50.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.909+0000 7ff3cb59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c415ccc0 con 0x7ff3c40cb1a0 2023-10-06T03:33:50.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.909+0000 7ff3c9d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c406f4b0 0x7ff3c414f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.909+0000 7ff3c9d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c406f4b0 0x7ff3c414f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36226/0 (socket says 172.21.15.55:36226) 2023-10-06T03:33:50.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.909+0000 7ff3c9d9a700 1 -- 172.21.15.55:0/1630761172 learned_addr learned my addr 172.21.15.55:0/1630761172 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:50.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.909+0000 7ff3c9d9a700 1 -- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3c40cb1a0 msgr2=0x7ff3c41502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.909+0000 7ff3cad9c700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3c40cb1a0 0x7ff3c41502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.910+0000 7ff3c9d9a700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3c40cb1a0 0x7ff3c41502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.909+0000 7ff3ca59b700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3c406f060 0x7ff3c415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.910+0000 7ff3c9d9a700 1 -- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3c406f060 msgr2=0x7ff3c415c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.910+0000 7ff3c9d9a700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3c406f060 0x7ff3c415c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.910+0000 7ff3c9d9a700 1 -- 172.21.15.55:0/1630761172 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b40094f0 con 0x7ff3c406f4b0 2023-10-06T03:33:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.910+0000 7ff3cad9c700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3c40cb1a0 0x7ff3c41502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.910+0000 7ff3ca59b700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3c406f060 0x7ff3c415c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.910+0000 7ff3c9d9a700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c406f4b0 0x7ff3c414f0e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff3c406ce80 tx=0x7ff3b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.910+0000 7ff3ba7fc700 1 -- 172.21.15.55:0/1630761172 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4024440 con 0x7ff3c406f4b0 2023-10-06T03:33:50.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.910+0000 7ff3ba7fc700 1 -- 172.21.15.55:0/1630761172 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b40245c0 con 0x7ff3c406f4b0 2023-10-06T03:33:50.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.910+0000 7ff3ba7fc700 1 -- 172.21.15.55:0/1630761172 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4024980 con 0x7ff3c406f4b0 2023-10-06T03:33:50.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.910+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1630761172 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c40c4b40 con 0x7ff3c406f4b0 2023-10-06T03:33:50.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.910+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1630761172 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c415cfa0 con 0x7ff3c406f4b0 2023-10-06T03:33:50.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.911+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1630761172 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c40c5020 con 0x7ff3c406f4b0 2023-10-06T03:33:50.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.912+0000 7ff3ba7fc700 1 -- 172.21.15.55:0/1630761172 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff3b401a420 con 0x7ff3c406f4b0 2023-10-06T03:33:50.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.913+0000 7ff3ba7fc700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff3a8066f30 0x7ff3a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:50.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.913+0000 7ff3ba7fc700 1 -- 172.21.15.55:0/1630761172 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff3b40863e0 con 0x7ff3c406f4b0 2023-10-06T03:33:50.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.914+0000 7ff3ca59b700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff3a8066f30 0x7ff3a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:50.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.914+0000 7ff3ca59b700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff3a8066f30 0x7ff3a80693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00017a0 tx=0x7ff3c0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:50.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:50.918+0000 7ff3ba7fc700 1 -- 172.21.15.55:0/1630761172 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3b4053450 con 0x7ff3c406f4b0 2023-10-06T03:33:51.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.121+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1630761172 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff3c415b000 con 0x7ff3c406f4b0 2023-10-06T03:33:51.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.122+0000 7ff3ba7fc700 1 -- 172.21.15.55:0/1630761172 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff3b4056a70 con 0x7ff3c406f4b0 2023-10-06T03:33:51.123 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T03:33:51.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.124+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff3a8066f30 msgr2=0x7ff3a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:51.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.124+0000 7ff3cb59d700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff3a8066f30 0x7ff3a80693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00017a0 tx=0x7ff3c0009040 comp rx=0 tx=0).stop 2023-10-06T03:33:51.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.124+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1630761172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c406f4b0 msgr2=0x7ff3c414f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:51.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.124+0000 7ff3cb59d700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c406f4b0 0x7ff3c414f0e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff3c406ce80 tx=0x7ff3b4017000 comp rx=0 tx=0).stop 2023-10-06T03:33:51.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.124+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1630761172 shutdown_connections 2023-10-06T03:33:51.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.124+0000 7ff3cb59d700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff3a8066f30 0x7ff3a80693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.124+0000 7ff3cb59d700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff3c40cb1a0 0x7ff3c41502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.124+0000 7ff3cb59d700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3c406f060 0x7ff3c415c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.124+0000 7ff3cb59d700 1 --2- 172.21.15.55:0/1630761172 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3c406f4b0 0x7ff3c414f0e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.124+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1630761172 >> 172.21.15.55:0/1630761172 conn(0x7ff3c40cc1e0 msgr2=0x7ff3c40b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:51.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.124+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1630761172 shutdown_connections 2023-10-06T03:33:51.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.124+0000 7ff3cb59d700 1 -- 172.21.15.55:0/1630761172 wait complete. 2023-10-06T03:33:51.143 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:33:51.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.331+0000 7fac1bfff700 1 Processor -- start 2023-10-06T03:33:51.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.331+0000 7fac1bfff700 1 -- start start 2023-10-06T03:33:51.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.332+0000 7fac1bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c0ccc00 0x7fac1c069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:51.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.332+0000 7fac1bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fac1c06a020 0x7fac1c0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:51.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.333+0000 7fac1bfff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac1c0c9f30 con 0x7fac1c159a60 2023-10-06T03:33:51.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.333+0000 7fac1bfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1c06dfc0 con 0x7fac1c06a020 2023-10-06T03:33:51.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.333+0000 7fac1bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c0ca470 con 0x7fac1c0ccc00 2023-10-06T03:33:51.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.333+0000 7fac1b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fac1c06a020 0x7fac1c0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:51.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.333+0000 7fac1b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fac1c06a020 0x7fac1c0cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36234/0 (socket says 172.21.15.55:36234) 2023-10-06T03:33:51.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.333+0000 7fac1b7fe700 1 -- 172.21.15.55:0/4244665299 learned_addr learned my addr 172.21.15.55:0/4244665299 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:51.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.333+0000 7fac1a7fc700 1 --2- 172.21.15.55:0/4244665299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c0ccc00 0x7fac1c069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:51.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.333+0000 7fac1b7fe700 1 -- 172.21.15.55:0/4244665299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c0ccc00 msgr2=0x7fac1c069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:51.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.333+0000 7fac1b7fe700 1 --2- 172.21.15.55:0/4244665299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c0ccc00 0x7fac1c069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.333+0000 7fac1b7fe700 1 -- 172.21.15.55:0/4244665299 >> v1:172.21.15.163:6789/0 conn(0x7fac1c159a60 legacy=0x7fac1c0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:51.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.333+0000 7fac1b7fe700 1 -- 172.21.15.55:0/4244665299 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c0ca750 con 0x7fac1c06a020 2023-10-06T03:33:51.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.334+0000 7fac1b7fe700 1 --2- 172.21.15.55:0/4244665299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1c06a020 0x7fac1c0cb230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac14009f20 tx=0x7fac14009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388810d477f25836 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:51.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.334+0000 7fac19ffb700 1 -- 172.21.15.55:0/4244665299 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1400e070 con 0x7fac1c06a020 2023-10-06T03:33:51.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.334+0000 7fac19ffb700 1 -- 172.21.15.55:0/4244665299 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1400ba60 con 0x7fac1c06a020 2023-10-06T03:33:51.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.334+0000 7fac19ffb700 1 -- 172.21.15.55:0/4244665299 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac140092e0 con 0x7fac1c06a020 2023-10-06T03:33:51.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.334+0000 7fac1bfff700 1 -- 172.21.15.55:0/4244665299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1c06a020 msgr2=0x7fac1c0cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:51.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.334+0000 7fac1bfff700 1 --2- 172.21.15.55:0/4244665299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1c06a020 0x7fac1c0cb230 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac14009f20 tx=0x7fac14009840 comp rx=0 tx=0).stop 2023-10-06T03:33:51.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.335+0000 7fac1bfff700 1 -- 172.21.15.55:0/4244665299 shutdown_connections 2023-10-06T03:33:51.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.335+0000 7fac1bfff700 1 --2- 172.21.15.55:0/4244665299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1c06a020 0x7fac1c0cb230 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.335+0000 7fac1bfff700 1 --2- 172.21.15.55:0/4244665299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c0ccc00 0x7fac1c069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.335+0000 7fac1bfff700 1 -- 172.21.15.55:0/4244665299 >> 172.21.15.55:0/4244665299 conn(0x7fac1c06baa0 msgr2=0x7fac1c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:51.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.335+0000 7fac1bfff700 1 -- 172.21.15.55:0/4244665299 shutdown_connections 2023-10-06T03:33:51.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.335+0000 7fac1bfff700 1 -- 172.21.15.55:0/4244665299 wait complete. 2023-10-06T03:33:51.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.336+0000 7fac1bfff700 1 Processor -- start 2023-10-06T03:33:51.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.336+0000 7fac1bfff700 1 -- start start 2023-10-06T03:33:51.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.337+0000 7fac1bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c06a020 0x7fac1c154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:51.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.337+0000 7fac1bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c0ccc00 0x7fac1c0b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:51.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.337+0000 7fac1bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1c159a60 0x7fac1c0b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:51.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.337+0000 7fac1bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c14ca60 con 0x7fac1c159a60 2023-10-06T03:33:51.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.337+0000 7fac1bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c15c6a0 con 0x7fac1c0ccc00 2023-10-06T03:33:51.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.337+0000 7fac1bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c0b3710 con 0x7fac1c06a020 2023-10-06T03:33:51.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.337+0000 7fac1b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1c159a60 0x7fac1c0b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:51.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.338+0000 7fac1a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c0ccc00 0x7fac1c0b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:51.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.338+0000 7fac1b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1c159a60 0x7fac1c0b31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36240/0 (socket says 172.21.15.55:36240) 2023-10-06T03:33:51.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.338+0000 7fac1a7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c0ccc00 0x7fac1c0b2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:57742/0 (socket says 172.21.15.55:57742) 2023-10-06T03:33:51.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.338+0000 7fac1b7fe700 1 -- 172.21.15.55:0/1656325999 learned_addr learned my addr 172.21.15.55:0/1656325999 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:51.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.338+0000 7fac1affd700 1 --2- 172.21.15.55:0/1656325999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c06a020 0x7fac1c154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:51.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.338+0000 7fac1b7fe700 1 -- 172.21.15.55:0/1656325999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c06a020 msgr2=0x7fac1c154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:51.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.338+0000 7fac1b7fe700 1 --2- 172.21.15.55:0/1656325999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c06a020 0x7fac1c154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.338+0000 7fac1b7fe700 1 -- 172.21.15.55:0/1656325999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c0ccc00 msgr2=0x7fac1c0b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:51.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.338+0000 7fac1b7fe700 1 --2- 172.21.15.55:0/1656325999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c0ccc00 0x7fac1c0b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.338+0000 7fac1b7fe700 1 -- 172.21.15.55:0/1656325999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac14009ae0 con 0x7fac1c159a60 2023-10-06T03:33:51.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.338+0000 7fac1a7fc700 1 --2- 172.21.15.55:0/1656325999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c0ccc00 0x7fac1c0b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:51.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.338+0000 7fac1b7fe700 1 --2- 172.21.15.55:0/1656325999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1c159a60 0x7fac1c0b31d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fac1400b610 tx=0x7fac14018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:51.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.338+0000 7fac02ffd700 1 -- 172.21.15.55:0/1656325999 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac140032f0 con 0x7fac1c159a60 2023-10-06T03:33:51.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.339+0000 7fac02ffd700 1 -- 172.21.15.55:0/1656325999 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac14008d20 con 0x7fac1c159a60 2023-10-06T03:33:51.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.339+0000 7fac02ffd700 1 -- 172.21.15.55:0/1656325999 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1401d630 con 0x7fac1c159a60 2023-10-06T03:33:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.339+0000 7fac1bfff700 1 -- 172.21.15.55:0/1656325999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c0b3f40 con 0x7fac1c159a60 2023-10-06T03:33:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.339+0000 7fac1bfff700 1 -- 172.21.15.55:0/1656325999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c0b3c60 con 0x7fac1c159a60 2023-10-06T03:33:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.340+0000 7fac1bfff700 1 -- 172.21.15.55:0/1656325999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac1c15c810 con 0x7fac1c159a60 2023-10-06T03:33:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.340+0000 7fac02ffd700 1 -- 172.21.15.55:0/1656325999 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac14003040 con 0x7fac1c159a60 2023-10-06T03:33:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.342+0000 7fac02ffd700 1 --2- 172.21.15.55:0/1656325999 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fac08066f30 0x7fac080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.342+0000 7fac02ffd700 1 -- 172.21.15.55:0/1656325999 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fac14086200 con 0x7fac1c159a60 2023-10-06T03:33:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.342+0000 7fac1affd700 1 --2- 172.21.15.55:0/1656325999 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fac08066f30 0x7fac080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:51.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.343+0000 7fac1affd700 1 --2- 172.21.15.55:0/1656325999 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fac08066f30 0x7fac080693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0c9e60 tx=0x7fac0c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:51.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.345+0000 7fac02ffd700 1 -- 172.21.15.55:0/1656325999 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac14053270 con 0x7fac1c159a60 2023-10-06T03:33:51.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.548+0000 7fac1bfff700 1 -- 172.21.15.55:0/1656325999 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac1c0caa30 con 0x7fac1c159a60 2023-10-06T03:33:51.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.549+0000 7fac02ffd700 1 -- 172.21.15.55:0/1656325999 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac14056890 con 0x7fac1c159a60 2023-10-06T03:33:51.550 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T03:33:51.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.551+0000 7fac1bfff700 1 -- 172.21.15.55:0/1656325999 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fac08066f30 msgr2=0x7fac080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.551+0000 7fac1bfff700 1 --2- 172.21.15.55:0/1656325999 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fac08066f30 0x7fac080693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0c9e60 tx=0x7fac0c009040 comp rx=0 tx=0).stop 2023-10-06T03:33:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.551+0000 7fac1bfff700 1 -- 172.21.15.55:0/1656325999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1c159a60 msgr2=0x7fac1c0b31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.551+0000 7fac1bfff700 1 --2- 172.21.15.55:0/1656325999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1c159a60 0x7fac1c0b31d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fac1400b610 tx=0x7fac14018000 comp rx=0 tx=0).stop 2023-10-06T03:33:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.551+0000 7fac1bfff700 1 -- 172.21.15.55:0/1656325999 shutdown_connections 2023-10-06T03:33:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.551+0000 7fac1bfff700 1 --2- 172.21.15.55:0/1656325999 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1c159a60 0x7fac1c0b31d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.551+0000 7fac1bfff700 1 --2- 172.21.15.55:0/1656325999 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fac08066f30 0x7fac080693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.551+0000 7fac1bfff700 1 --2- 172.21.15.55:0/1656325999 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1c06a020 0x7fac1c154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.551+0000 7fac1bfff700 1 --2- 172.21.15.55:0/1656325999 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1c0ccc00 0x7fac1c0b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.551+0000 7fac1bfff700 1 -- 172.21.15.55:0/1656325999 >> 172.21.15.55:0/1656325999 conn(0x7fac1c06baa0 msgr2=0x7fac1c159e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.551+0000 7fac1bfff700 1 -- 172.21.15.55:0/1656325999 shutdown_connections 2023-10-06T03:33:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.551+0000 7fac1bfff700 1 -- 172.21.15.55:0/1656325999 wait complete. 2023-10-06T03:33:51.570 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:51.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.756+0000 7f5175f9d700 1 Processor -- start 2023-10-06T03:33:51.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.757+0000 7f5175f9d700 1 -- start start 2023-10-06T03:33:51.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.758+0000 7f5175f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5170142ea0 0x7f51700b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:51.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.758+0000 7f5175f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f51700ad870 0x7f517006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:51.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.758+0000 7f5175f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51700b40f0 con 0x7f51700b3b70 2023-10-06T03:33:51.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.758+0000 7f5175f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5170069640 con 0x7f51700ad870 2023-10-06T03:33:51.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.758+0000 7f5175f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5170069920 con 0x7f5170142ea0 2023-10-06T03:33:51.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.758+0000 7f516ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f51700ad870 0x7f517006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:51.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.758+0000 7f516ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f51700ad870 0x7f517006b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36254/0 (socket says 172.21.15.55:36254) 2023-10-06T03:33:51.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.758+0000 7f516ffff700 1 -- 172.21.15.55:0/2894676526 learned_addr learned my addr 172.21.15.55:0/2894676526 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:51.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.758+0000 7f5174f9b700 1 --2- 172.21.15.55:0/2894676526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5170142ea0 0x7f51700b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:51.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.759+0000 7f516ffff700 1 -- 172.21.15.55:0/2894676526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5170142ea0 msgr2=0x7f51700b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:51.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.759+0000 7f516ffff700 1 --2- 172.21.15.55:0/2894676526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5170142ea0 0x7f51700b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.759+0000 7f516ffff700 1 -- 172.21.15.55:0/2894676526 >> v1:172.21.15.163:6789/0 conn(0x7f51700b3b70 legacy=0x7f517006cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:51.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.759+0000 7f516ffff700 1 -- 172.21.15.55:0/2894676526 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5170069c00 con 0x7f51700ad870 2023-10-06T03:33:51.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.759+0000 7f516ffff700 1 --2- 172.21.15.55:0/2894676526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51700ad870 0x7f517006b9e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f515c0099b0 tx=0x7f515c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6872eaaeff7c6d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:51.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.759+0000 7f516f7fe700 1 -- 172.21.15.55:0/2894676526 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c00b860 con 0x7f51700ad870 2023-10-06T03:33:51.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.759+0000 7f516f7fe700 1 -- 172.21.15.55:0/2894676526 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c00b9c0 con 0x7f51700ad870 2023-10-06T03:33:51.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.759+0000 7f516f7fe700 1 -- 172.21.15.55:0/2894676526 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c00bd60 con 0x7f51700ad870 2023-10-06T03:33:51.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.760+0000 7f5175f9d700 1 -- 172.21.15.55:0/2894676526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51700ad870 msgr2=0x7f517006b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:51.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.760+0000 7f5175f9d700 1 --2- 172.21.15.55:0/2894676526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51700ad870 0x7f517006b9e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f515c0099b0 tx=0x7f515c009250 comp rx=0 tx=0).stop 2023-10-06T03:33:51.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.760+0000 7f5175f9d700 1 -- 172.21.15.55:0/2894676526 shutdown_connections 2023-10-06T03:33:51.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.760+0000 7f5175f9d700 1 --2- 172.21.15.55:0/2894676526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5170142ea0 0x7f51700b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.760+0000 7f5175f9d700 1 --2- 172.21.15.55:0/2894676526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51700ad870 0x7f517006b9e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.760+0000 7f5175f9d700 1 -- 172.21.15.55:0/2894676526 >> 172.21.15.55:0/2894676526 conn(0x7f51700ce430 msgr2=0x7f51700ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:51.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.760+0000 7f5175f9d700 1 -- 172.21.15.55:0/2894676526 shutdown_connections 2023-10-06T03:33:51.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.760+0000 7f5175f9d700 1 -- 172.21.15.55:0/2894676526 wait complete. 2023-10-06T03:33:51.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.761+0000 7f5175f9d700 1 Processor -- start 2023-10-06T03:33:51.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.762+0000 7f5175f9d700 1 -- start start 2023-10-06T03:33:51.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.762+0000 7f5175f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51700b3b70 0x7f5170158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:51.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f5175f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5170142ea0 0x7f5170140f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:51.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f5175f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5170141470 0x7f517015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:51.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f5175f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5170141910 con 0x7f5170141470 2023-10-06T03:33:51.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f5175f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517015bfb0 con 0x7f5170142ea0 2023-10-06T03:33:51.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f5175f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517015c200 con 0x7f51700b3b70 2023-10-06T03:33:51.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f517579c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5170141470 0x7f517015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:51.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f517579c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5170141470 0x7f517015d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36256/0 (socket says 172.21.15.55:36256) 2023-10-06T03:33:51.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f517579c700 1 -- 172.21.15.55:0/4083310584 learned_addr learned my addr 172.21.15.55:0/4083310584 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:51.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f516ffff700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5170142ea0 0x7f5170140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:51.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f5174f9b700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51700b3b70 0x7f5170158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:51.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f517579c700 1 -- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51700b3b70 msgr2=0x7f5170158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:51.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f517579c700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51700b3b70 0x7f5170158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f517579c700 1 -- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5170142ea0 msgr2=0x7f5170140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:51.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f517579c700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5170142ea0 0x7f5170140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f517579c700 1 -- 172.21.15.55:0/4083310584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515c0094f0 con 0x7f5170141470 2023-10-06T03:33:51.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f5174f9b700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51700b3b70 0x7f5170158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:51.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.763+0000 7f517579c700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5170141470 0x7f517015d8d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f516800c8e0 tx=0x7f5168001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:51.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.764+0000 7f516cff9700 1 -- 172.21.15.55:0/4083310584 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516800ce80 con 0x7f5170141470 2023-10-06T03:33:51.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.764+0000 7f516cff9700 1 -- 172.21.15.55:0/4083310584 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516800a7e0 con 0x7f5170141470 2023-10-06T03:33:51.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.764+0000 7f5175f9d700 1 -- 172.21.15.55:0/4083310584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517015c4e0 con 0x7f5170141470 2023-10-06T03:33:51.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.764+0000 7f516cff9700 1 -- 172.21.15.55:0/4083310584 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516800ab80 con 0x7f5170141470 2023-10-06T03:33:51.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.764+0000 7f5175f9d700 1 -- 172.21.15.55:0/4083310584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517015caa0 con 0x7f5170141470 2023-10-06T03:33:51.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.765+0000 7f5175f9d700 1 -- 172.21.15.55:0/4083310584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5170159520 con 0x7f5170141470 2023-10-06T03:33:51.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.766+0000 7f516cff9700 1 -- 172.21.15.55:0/4083310584 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5168012420 con 0x7f5170141470 2023-10-06T03:33:51.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.767+0000 7f516cff9700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5154066f30 0x7f51540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:51.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.767+0000 7f516cff9700 1 -- 172.21.15.55:0/4083310584 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5168084a80 con 0x7f5170141470 2023-10-06T03:33:51.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.767+0000 7f5174f9b700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5154066f30 0x7f51540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:51.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.768+0000 7f5174f9b700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5154066f30 0x7f51540693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5164009990 tx=0x7f5164008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:51.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.770+0000 7f516cff9700 1 -- 172.21.15.55:0/4083310584 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5168051af0 con 0x7f5170141470 2023-10-06T03:33:51.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.974+0000 7f5175f9d700 1 -- 172.21.15.55:0/4083310584 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f517004c3c0 con 0x7f5170141470 2023-10-06T03:33:51.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.975+0000 7f516cff9700 1 -- 172.21.15.55:0/4083310584 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f5168055110 con 0x7f5170141470 2023-10-06T03:33:51.977 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:51.977 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:16.470817+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:51.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.978+0000 7f5175f9d700 1 -- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5154066f30 msgr2=0x7f51540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:51.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.978+0000 7f5175f9d700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5154066f30 0x7f51540693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5164009990 tx=0x7f5164008040 comp rx=0 tx=0).stop 2023-10-06T03:33:51.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.978+0000 7f5175f9d700 1 -- 172.21.15.55:0/4083310584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5170141470 msgr2=0x7f517015d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:51.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.978+0000 7f5175f9d700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5170141470 0x7f517015d8d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f516800c8e0 tx=0x7f5168001710 comp rx=0 tx=0).stop 2023-10-06T03:33:51.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.978+0000 7f5175f9d700 1 -- 172.21.15.55:0/4083310584 shutdown_connections 2023-10-06T03:33:51.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.978+0000 7f5175f9d700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5170141470 0x7f517015d8d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.978+0000 7f5175f9d700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5154066f30 0x7f51540693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.978+0000 7f5175f9d700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51700b3b70 0x7f5170158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.978+0000 7f5175f9d700 1 --2- 172.21.15.55:0/4083310584 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5170142ea0 0x7f5170140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:51.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.978+0000 7f5175f9d700 1 -- 172.21.15.55:0/4083310584 >> 172.21.15.55:0/4083310584 conn(0x7f51700ce430 msgr2=0x7f51700b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:51.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.978+0000 7f5175f9d700 1 -- 172.21.15.55:0/4083310584 shutdown_connections 2023-10-06T03:33:51.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:51.978+0000 7f5175f9d700 1 -- 172.21.15.55:0/4083310584 wait complete. 2023-10-06T03:33:51.997 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:33:13.543361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:33:14.341572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:33:51.997 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:33:52.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.186+0000 7f21bffff700 1 Processor -- start 2023-10-06T03:33:52.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.187+0000 7f21bffff700 1 -- start start 2023-10-06T03:33:52.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.188+0000 7f21bffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f21b80c83f0 0x7f21b80c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:52.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.188+0000 7f21bffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21b80ccee0 0x7f21b81561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:52.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.188+0000 7f21bffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f21b80c87c0 0x7f21b8156710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:52.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.188+0000 7f21bffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b8156c50 con 0x7f21b80c83f0 2023-10-06T03:33:52.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.188+0000 7f21bffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b815b0d0 con 0x7f21b80c87c0 2023-10-06T03:33:52.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.188+0000 7f21bffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b80c7a60 con 0x7f21b80ccee0 2023-10-06T03:33:52.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.188+0000 7f21bdd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f21b80c83f0 0x7f21b80c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:52.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.188+0000 7f21bdd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f21b80c83f0 0x7f21b80c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36270/0 (socket says 172.21.15.55:36270) 2023-10-06T03:33:52.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.188+0000 7f21bdd9b700 1 -- 172.21.15.55:0/2563723816 learned_addr learned my addr 172.21.15.55:0/2563723816 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:52.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.188+0000 7f21be59c700 1 --2- 172.21.15.55:0/2563723816 >> v2:172.21.15.163:3300/0 conn(0x7f21b80c87c0 0x7f21b8156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:52.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.189+0000 7f21bd59a700 1 --2- 172.21.15.55:0/2563723816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21b80ccee0 0x7f21b81561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:52.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.189+0000 7f21bdd9b700 1 -- 172.21.15.55:0/2563723816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21b80ccee0 msgr2=0x7f21b81561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:52.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.189+0000 7f21bdd9b700 1 --2- 172.21.15.55:0/2563723816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21b80ccee0 0x7f21b81561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.189+0000 7f21bdd9b700 1 -- 172.21.15.55:0/2563723816 >> v2:172.21.15.163:3300/0 conn(0x7f21b80c87c0 msgr2=0x7f21b8156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:52.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.189+0000 7f21bdd9b700 1 --2- 172.21.15.55:0/2563723816 >> v2:172.21.15.163:3300/0 conn(0x7f21b80c87c0 0x7f21b8156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.189+0000 7f21bdd9b700 1 -- 172.21.15.55:0/2563723816 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b815a4c0 con 0x7f21b80c83f0 2023-10-06T03:33:52.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.189+0000 7f21be59c700 1 --2- 172.21.15.55:0/2563723816 >> v2:172.21.15.163:3300/0 conn(0x7f21b80c87c0 0x7f21b8156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:52.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.189+0000 7f21bd59a700 1 --2- 172.21.15.55:0/2563723816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21b80ccee0 0x7f21b81561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:52.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.189+0000 7f21bdd9b700 1 --2- 172.21.15.55:0/2563723816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21b80c83f0 0x7f21b80c7520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f21b80cc340 tx=0x7f21b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571639450bbc3cdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:52.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.189+0000 7f21bcd99700 1 -- 172.21.15.55:0/2563723816 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b400b860 con 0x7f21b80c83f0 2023-10-06T03:33:52.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.189+0000 7f21bcd99700 1 -- 172.21.15.55:0/2563723816 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b400b9c0 con 0x7f21b80c83f0 2023-10-06T03:33:52.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.189+0000 7f21bcd99700 1 -- 172.21.15.55:0/2563723816 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b400bd60 con 0x7f21b80c83f0 2023-10-06T03:33:52.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.190+0000 7f21bffff700 1 -- 172.21.15.55:0/2563723816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21b80c83f0 msgr2=0x7f21b80c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:52.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.190+0000 7f21bffff700 1 --2- 172.21.15.55:0/2563723816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21b80c83f0 0x7f21b80c7520 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f21b80cc340 tx=0x7f21b4009250 comp rx=0 tx=0).stop 2023-10-06T03:33:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.190+0000 7f21bffff700 1 -- 172.21.15.55:0/2563723816 shutdown_connections 2023-10-06T03:33:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.190+0000 7f21bffff700 1 --2- 172.21.15.55:0/2563723816 >> v2:172.21.15.163:3300/0 conn(0x7f21b80c87c0 0x7f21b8156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.190+0000 7f21bffff700 1 --2- 172.21.15.55:0/2563723816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21b80c83f0 0x7f21b80c7520 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.190+0000 7f21bffff700 1 --2- 172.21.15.55:0/2563723816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21b80ccee0 0x7f21b81561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.190+0000 7f21bffff700 1 -- 172.21.15.55:0/2563723816 >> 172.21.15.55:0/2563723816 conn(0x7f21b8068e10 msgr2=0x7f21b80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.190+0000 7f21bffff700 1 -- 172.21.15.55:0/2563723816 shutdown_connections 2023-10-06T03:33:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.190+0000 7f21bffff700 1 -- 172.21.15.55:0/2563723816 wait complete. 2023-10-06T03:33:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.191+0000 7f21bffff700 1 Processor -- start 2023-10-06T03:33:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.192+0000 7f21bffff700 1 -- start start 2023-10-06T03:33:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.192+0000 7f21bffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21b80c83f0 0x7f21b81490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:52.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.192+0000 7f21bffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21b80c87c0 0x7f21b80af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.192+0000 7f21bffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21b80ccee0 0x7f21b80afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.192+0000 7f21bffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b80c3320 con 0x7f21b80c83f0 2023-10-06T03:33:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.192+0000 7f21bffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b816e350 con 0x7f21b80ccee0 2023-10-06T03:33:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.192+0000 7f21bffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b816dec0 con 0x7f21b80c87c0 2023-10-06T03:33:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.193+0000 7f21bdd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21b80c83f0 0x7f21b81490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.193+0000 7f21bdd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21b80c83f0 0x7f21b81490b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36276/0 (socket says 172.21.15.55:36276) 2023-10-06T03:33:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.193+0000 7f21bdd9b700 1 -- 172.21.15.55:0/1018182452 learned_addr learned my addr 172.21.15.55:0/1018182452 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.193+0000 7f21bd59a700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21b80c87c0 0x7f21b80af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:52.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.193+0000 7f21be59c700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21b80ccee0 0x7f21b80afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.193+0000 7f21bdd9b700 1 -- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21b80c87c0 msgr2=0x7f21b80af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.193+0000 7f21bdd9b700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21b80c87c0 0x7f21b80af600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.193+0000 7f21bdd9b700 1 -- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21b80ccee0 msgr2=0x7f21b80afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.193+0000 7f21bdd9b700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21b80ccee0 0x7f21b80afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.193+0000 7f21bdd9b700 1 -- 172.21.15.55:0/1018182452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b40094f0 con 0x7f21b80c83f0 2023-10-06T03:33:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.193+0000 7f21bd59a700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21b80c87c0 0x7f21b80af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.193+0000 7f21bdd9b700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21b80c83f0 0x7f21b81490b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f21b40090b0 tx=0x7f21b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.194+0000 7f21adffb700 1 -- 172.21.15.55:0/1018182452 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b401a450 con 0x7f21b80c83f0 2023-10-06T03:33:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.194+0000 7f21adffb700 1 -- 172.21.15.55:0/1018182452 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b401a5b0 con 0x7f21b80c83f0 2023-10-06T03:33:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.194+0000 7f21bffff700 1 -- 172.21.15.55:0/1018182452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b816d590 con 0x7f21b80c83f0 2023-10-06T03:33:52.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.194+0000 7f21adffb700 1 -- 172.21.15.55:0/1018182452 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b401a970 con 0x7f21b80c83f0 2023-10-06T03:33:52.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.194+0000 7f21bffff700 1 -- 172.21.15.55:0/1018182452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b816db50 con 0x7f21b80c83f0 2023-10-06T03:33:52.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.195+0000 7f21bffff700 1 -- 172.21.15.55:0/1018182452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b816e1a0 con 0x7f21b80c83f0 2023-10-06T03:33:52.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.196+0000 7f21adffb700 1 -- 172.21.15.55:0/1018182452 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f21b401ad20 con 0x7f21b80c83f0 2023-10-06T03:33:52.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.197+0000 7f21adffb700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f219c066f30 0x7f219c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:52.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.197+0000 7f21adffb700 1 -- 172.21.15.55:0/1018182452 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f21b4085dc0 con 0x7f21b80c83f0 2023-10-06T03:33:52.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.197+0000 7f21bd59a700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f219c066f30 0x7f219c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:52.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.198+0000 7f21bd59a700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f219c066f30 0x7f219c0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f21b80cbd60 tx=0x7f21a8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:52.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.201+0000 7f21adffb700 1 -- 172.21.15.55:0/1018182452 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21b4052e30 con 0x7f21b80c83f0 2023-10-06T03:33:52.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.403+0000 7f21bffff700 1 -- 172.21.15.55:0/1018182452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f21b804c3c0 con 0x7f21b80c83f0 2023-10-06T03:33:52.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.403+0000 7f21adffb700 1 -- 172.21.15.55:0/1018182452 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f21b40200a0 con 0x7f21b80c83f0 2023-10-06T03:33:52.405 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:33:52.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.406+0000 7f21bffff700 1 -- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f219c066f30 msgr2=0x7f219c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:52.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.406+0000 7f21bffff700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f219c066f30 0x7f219c0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f21b80cbd60 tx=0x7f21a8008040 comp rx=0 tx=0).stop 2023-10-06T03:33:52.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.406+0000 7f21bffff700 1 -- 172.21.15.55:0/1018182452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21b80c83f0 msgr2=0x7f21b81490b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:52.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.406+0000 7f21bffff700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21b80c83f0 0x7f21b81490b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f21b40090b0 tx=0x7f21b4017000 comp rx=0 tx=0).stop 2023-10-06T03:33:52.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.406+0000 7f21bffff700 1 -- 172.21.15.55:0/1018182452 shutdown_connections 2023-10-06T03:33:52.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.406+0000 7f21bffff700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21b80ccee0 0x7f21b80afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.406+0000 7f21bffff700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21b80c83f0 0x7f21b81490b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.406+0000 7f21bffff700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f219c066f30 0x7f219c0693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.406+0000 7f21bffff700 1 --2- 172.21.15.55:0/1018182452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21b80c87c0 0x7f21b80af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.406+0000 7f21bffff700 1 -- 172.21.15.55:0/1018182452 >> 172.21.15.55:0/1018182452 conn(0x7f21b8068e10 msgr2=0x7f21b80b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:52.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.406+0000 7f21bffff700 1 -- 172.21.15.55:0/1018182452 shutdown_connections 2023-10-06T03:33:52.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.406+0000 7f21bffff700 1 -- 172.21.15.55:0/1018182452 wait complete. 2023-10-06T03:33:52.425 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:33:52.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.615+0000 7fba26f9d700 1 Processor -- start 2023-10-06T03:33:52.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.615+0000 7fba26f9d700 1 -- start start 2023-10-06T03:33:52.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.616+0000 7fba26f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fba200cb1a0 0x7fba2015a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:52.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.616+0000 7fba26f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fba200ae0d0 0x7fba200ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:52.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.616+0000 7fba26f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2006ebd0 con 0x7fba2015a8a0 2023-10-06T03:33:52.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.616+0000 7fba26f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2015b2e0 con 0x7fba200cb1a0 2023-10-06T03:33:52.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.616+0000 7fba26f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba200aea10 con 0x7fba200ae0d0 2023-10-06T03:33:52.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.616+0000 7fba25f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fba200cb1a0 0x7fba2015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:52.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.616+0000 7fba2579a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fba2015a8a0 0x7fba2006f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59554/0 (socket says 172.21.15.55:59554) 2023-10-06T03:33:52.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.616+0000 7fba25f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fba200cb1a0 0x7fba2015a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36284/0 (socket says 172.21.15.55:36284) 2023-10-06T03:33:52.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.616+0000 7fba2579a700 1 -- 172.21.15.55:0/2956114344 learned_addr learned my addr 172.21.15.55:0/2956114344 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:52.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.617+0000 7fba2679c700 1 --2- 172.21.15.55:0/2956114344 >> v2:172.21.15.163:3300/0 conn(0x7fba200ae0d0 0x7fba200ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:52.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.617+0000 7fba24f99700 1 -- 172.21.15.55:0/2956114344 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174444082 0 0) 0x7fba2006ebd0 con 0x7fba2015a8a0 2023-10-06T03:33:52.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.617+0000 7fba25f9b700 1 -- 172.21.15.55:0/2956114344 >> v2:172.21.15.163:3300/0 conn(0x7fba200ae0d0 msgr2=0x7fba200ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:52.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.617+0000 7fba25f9b700 1 --2- 172.21.15.55:0/2956114344 >> v2:172.21.15.163:3300/0 conn(0x7fba200ae0d0 0x7fba200ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.617+0000 7fba25f9b700 1 -- 172.21.15.55:0/2956114344 >> v1:172.21.15.55:6789/0 conn(0x7fba2015a8a0 legacy=0x7fba2006f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:52.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.617+0000 7fba25f9b700 1 -- 172.21.15.55:0/2956114344 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2015ad20 con 0x7fba200cb1a0 2023-10-06T03:33:52.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.617+0000 7fba25f9b700 1 --2- 172.21.15.55:0/2956114344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba200cb1a0 0x7fba2015a360 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fba200c67b0 tx=0x7fba1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607e71df18d3b2af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:52.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.617+0000 7fba24f99700 1 -- 172.21.15.55:0/2956114344 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c00b740 con 0x7fba200cb1a0 2023-10-06T03:33:52.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.617+0000 7fba24f99700 1 -- 172.21.15.55:0/2956114344 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c00b8a0 con 0x7fba200cb1a0 2023-10-06T03:33:52.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.617+0000 7fba24f99700 1 -- 172.21.15.55:0/2956114344 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c00bc40 con 0x7fba200cb1a0 2023-10-06T03:33:52.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.618+0000 7fba26f9d700 1 -- 172.21.15.55:0/2956114344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba200cb1a0 msgr2=0x7fba2015a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:52.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.618+0000 7fba26f9d700 1 --2- 172.21.15.55:0/2956114344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba200cb1a0 0x7fba2015a360 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fba200c67b0 tx=0x7fba1c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.618+0000 7fba26f9d700 1 -- 172.21.15.55:0/2956114344 shutdown_connections 2023-10-06T03:33:52.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.618+0000 7fba26f9d700 1 --2- 172.21.15.55:0/2956114344 >> v2:172.21.15.163:3300/0 conn(0x7fba200ae0d0 0x7fba200ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.618+0000 7fba26f9d700 1 --2- 172.21.15.55:0/2956114344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba200cb1a0 0x7fba2015a360 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.618+0000 7fba26f9d700 1 -- 172.21.15.55:0/2956114344 >> 172.21.15.55:0/2956114344 conn(0x7fba200cc1e0 msgr2=0x7fba200cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:52.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.618+0000 7fba26f9d700 1 -- 172.21.15.55:0/2956114344 shutdown_connections 2023-10-06T03:33:52.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.618+0000 7fba26f9d700 1 -- 172.21.15.55:0/2956114344 wait complete. 2023-10-06T03:33:52.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.619+0000 7fba26f9d700 1 Processor -- start 2023-10-06T03:33:52.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.619+0000 7fba26f9d700 1 -- start start 2023-10-06T03:33:52.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.620+0000 7fba26f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba200ae0d0 0x7fba200aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:52.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.620+0000 7fba26f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba200cb1a0 0x7fba201551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.620+0000 7fba26f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba200adc00 0x7fba200b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.620+0000 7fba26f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200b03e0 con 0x7fba200cb1a0 2023-10-06T03:33:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.620+0000 7fba26f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200af7d0 con 0x7fba200adc00 2023-10-06T03:33:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.620+0000 7fba26f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba200afab0 con 0x7fba200ae0d0 2023-10-06T03:33:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.621+0000 7fba2579a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba200cb1a0 0x7fba201551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.621+0000 7fba2579a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba200cb1a0 0x7fba201551d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36292/0 (socket says 172.21.15.55:36292) 2023-10-06T03:33:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.621+0000 7fba2579a700 1 -- 172.21.15.55:0/3934766958 learned_addr learned my addr 172.21.15.55:0/3934766958 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.621+0000 7fba25f9b700 1 --2- 172.21.15.55:0/3934766958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba200ae0d0 0x7fba200aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.621+0000 7fba2679c700 1 --2- 172.21.15.55:0/3934766958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba200adc00 0x7fba200b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:52.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.621+0000 7fba2579a700 1 -- 172.21.15.55:0/3934766958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba200ae0d0 msgr2=0x7fba200aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:52.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.621+0000 7fba2579a700 1 --2- 172.21.15.55:0/3934766958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba200ae0d0 0x7fba200aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.621+0000 7fba2579a700 1 -- 172.21.15.55:0/3934766958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba200adc00 msgr2=0x7fba200b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:52.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.621+0000 7fba2579a700 1 --2- 172.21.15.55:0/3934766958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba200adc00 0x7fba200b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.621+0000 7fba2579a700 1 -- 172.21.15.55:0/3934766958 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1c009420 con 0x7fba200cb1a0 2023-10-06T03:33:52.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.621+0000 7fba2579a700 1 --2- 172.21.15.55:0/3934766958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba200cb1a0 0x7fba201551d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fba1000c5b0 tx=0x7fba10001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:52.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.622+0000 7fba15ffb700 1 -- 172.21.15.55:0/3934766958 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10005b30 con 0x7fba200cb1a0 2023-10-06T03:33:52.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.622+0000 7fba15ffb700 1 -- 172.21.15.55:0/3934766958 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba10005c90 con 0x7fba200cb1a0 2023-10-06T03:33:52.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.622+0000 7fba15ffb700 1 -- 172.21.15.55:0/3934766958 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10007300 con 0x7fba200cb1a0 2023-10-06T03:33:52.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.622+0000 7fba26f9d700 1 -- 172.21.15.55:0/3934766958 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba200afd90 con 0x7fba200cb1a0 2023-10-06T03:33:52.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.622+0000 7fba26f9d700 1 -- 172.21.15.55:0/3934766958 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba200af4c0 con 0x7fba200cb1a0 2023-10-06T03:33:52.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.623+0000 7fba26f9d700 1 -- 172.21.15.55:0/3934766958 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba200b0550 con 0x7fba200cb1a0 2023-10-06T03:33:52.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.624+0000 7fba15ffb700 1 -- 172.21.15.55:0/3934766958 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba1001b3b0 con 0x7fba200cb1a0 2023-10-06T03:33:52.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.624+0000 7fba15ffb700 1 --2- 172.21.15.55:0/3934766958 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fba04066f30 0x7fba040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:52.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.625+0000 7fba15ffb700 1 -- 172.21.15.55:0/3934766958 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fba10083b30 con 0x7fba200cb1a0 2023-10-06T03:33:52.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.625+0000 7fba25f9b700 1 --2- 172.21.15.55:0/3934766958 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fba04066f30 0x7fba040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:52.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.626+0000 7fba25f9b700 1 --2- 172.21.15.55:0/3934766958 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fba04066f30 0x7fba040693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0093f0 tx=0x7fba1c017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:52.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.629+0000 7fba15ffb700 1 -- 172.21.15.55:0/3934766958 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba10050ba0 con 0x7fba200cb1a0 2023-10-06T03:33:52.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.834+0000 7fba26f9d700 1 -- 172.21.15.55:0/3934766958 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba2015b000 con 0x7fba200cb1a0 2023-10-06T03:33:52.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.834+0000 7fba15ffb700 1 -- 172.21.15.55:0/3934766958 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba100170a0 con 0x7fba200cb1a0 2023-10-06T03:33:52.836 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:33:52.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.837+0000 7fba26f9d700 1 -- 172.21.15.55:0/3934766958 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fba04066f30 msgr2=0x7fba040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:52.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.837+0000 7fba26f9d700 1 --2- 172.21.15.55:0/3934766958 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fba04066f30 0x7fba040693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0093f0 tx=0x7fba1c017000 comp rx=0 tx=0).stop 2023-10-06T03:33:52.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.837+0000 7fba26f9d700 1 -- 172.21.15.55:0/3934766958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba200cb1a0 msgr2=0x7fba201551d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:52.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.837+0000 7fba26f9d700 1 --2- 172.21.15.55:0/3934766958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba200cb1a0 0x7fba201551d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fba1000c5b0 tx=0x7fba10001070 comp rx=0 tx=0).stop 2023-10-06T03:33:52.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.837+0000 7fba26f9d700 1 -- 172.21.15.55:0/3934766958 shutdown_connections 2023-10-06T03:33:52.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.837+0000 7fba26f9d700 1 --2- 172.21.15.55:0/3934766958 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba200adc00 0x7fba200b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.837+0000 7fba26f9d700 1 --2- 172.21.15.55:0/3934766958 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fba04066f30 0x7fba040693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.837+0000 7fba26f9d700 1 --2- 172.21.15.55:0/3934766958 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba200ae0d0 0x7fba200aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.837+0000 7fba26f9d700 1 --2- 172.21.15.55:0/3934766958 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba200cb1a0 0x7fba201551d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:52.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.837+0000 7fba26f9d700 1 -- 172.21.15.55:0/3934766958 >> 172.21.15.55:0/3934766958 conn(0x7fba200cc1e0 msgr2=0x7fba200c12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:52.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.837+0000 7fba26f9d700 1 -- 172.21.15.55:0/3934766958 shutdown_connections 2023-10-06T03:33:52.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:52.837+0000 7fba26f9d700 1 -- 172.21.15.55:0/3934766958 wait complete. 2023-10-06T03:33:52.856 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:33:53.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.044+0000 7fba8719d700 1 Processor -- start 2023-10-06T03:33:53.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.044+0000 7fba8719d700 1 -- start start 2023-10-06T03:33:53.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.045+0000 7fba8719d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fba800b4e70 0x7fba800ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.045+0000 7fba8719d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba800cfb00 con 0x7fba800b1b40 2023-10-06T03:33:53.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.046+0000 7fba8719d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba8015e060 con 0x7fba800b16d0 2023-10-06T03:33:53.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.046+0000 7fba8719d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba800b5c40 con 0x7fba800b4e70 2023-10-06T03:33:53.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.046+0000 7fba8699c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fba800b1b40 0x7fba8015d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59566/0 (socket says 172.21.15.55:59566) 2023-10-06T03:33:53.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.046+0000 7fba8699c700 1 -- 172.21.15.55:0/72482061 learned_addr learned my addr 172.21.15.55:0/72482061 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:53.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.046+0000 7fba8619b700 1 --2- 172.21.15.55:0/72482061 >> v2:172.21.15.163:3300/0 conn(0x7fba800b4e70 0x7fba800ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.046+0000 7fba85199700 1 -- 172.21.15.55:0/72482061 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084825422 0 0) 0x7fba800cfb00 con 0x7fba800b1b40 2023-10-06T03:33:53.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.046+0000 7fba85199700 1 -- 172.21.15.55:0/72482061 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba6c003570 con 0x7fba800b1b40 2023-10-06T03:33:53.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.046+0000 7fba85199700 1 -- 172.21.15.55:0/72482061 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1236256352 0 0) 0x7fba8015e060 con 0x7fba800b16d0 2023-10-06T03:33:53.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.046+0000 7fba85199700 1 -- 172.21.15.55:0/72482061 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba800cfb00 con 0x7fba800b16d0 2023-10-06T03:33:53.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.047+0000 7fba8619b700 1 -- 172.21.15.55:0/72482061 >> v1:172.21.15.163:6789/0 conn(0x7fba800b16d0 legacy=0x7fba8015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.047+0000 7fba8619b700 1 -- 172.21.15.55:0/72482061 >> v1:172.21.15.55:6789/0 conn(0x7fba800b1b40 legacy=0x7fba8015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.047+0000 7fba85199700 1 -- 172.21.15.55:0/72482061 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2957522556 0 0) 0x7fba6c003570 con 0x7fba800b1b40 2023-10-06T03:33:53.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.047+0000 7fba8619b700 1 -- 172.21.15.55:0/72482061 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba800b5f20 con 0x7fba800b4e70 2023-10-06T03:33:53.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.047+0000 7fba8619b700 1 --2- 172.21.15.55:0/72482061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba800b4e70 0x7fba800ccff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0098e0 tx=0x7fba7c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fec2bb934a502ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:53.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.047+0000 7fba85199700 1 -- 172.21.15.55:0/72482061 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba780017f0 con 0x7fba800b4e70 2023-10-06T03:33:53.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.047+0000 7fba85199700 1 -- 172.21.15.55:0/72482061 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c011420 con 0x7fba800b4e70 2023-10-06T03:33:53.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.047+0000 7fba85199700 1 -- 172.21.15.55:0/72482061 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c0117c0 con 0x7fba800b4e70 2023-10-06T03:33:53.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.048+0000 7fba8719d700 1 -- 172.21.15.55:0/72482061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba800b4e70 msgr2=0x7fba800ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.048+0000 7fba8719d700 1 --2- 172.21.15.55:0/72482061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba800b4e70 0x7fba800ccff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0098e0 tx=0x7fba7c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.048+0000 7fba8719d700 1 -- 172.21.15.55:0/72482061 shutdown_connections 2023-10-06T03:33:53.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.048+0000 7fba8719d700 1 --2- 172.21.15.55:0/72482061 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba800b4e70 0x7fba800ccff0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.048+0000 7fba8719d700 1 -- 172.21.15.55:0/72482061 >> 172.21.15.55:0/72482061 conn(0x7fba800d10c0 msgr2=0x7fba800affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:53.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.048+0000 7fba8719d700 1 -- 172.21.15.55:0/72482061 shutdown_connections 2023-10-06T03:33:53.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.048+0000 7fba8719d700 1 -- 172.21.15.55:0/72482061 wait complete. 2023-10-06T03:33:53.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.049+0000 7fba8719d700 1 Processor -- start 2023-10-06T03:33:53.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.049+0000 7fba8719d700 1 -- start start 2023-10-06T03:33:53.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.050+0000 7fba8719d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba800b16d0 0x7fba800c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.050+0000 7fba8719d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba800b1b40 0x7fba800c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.050+0000 7fba8719d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba800b4e70 0x7fba800c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.050+0000 7fba8719d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba80154a20 con 0x7fba800b1b40 2023-10-06T03:33:53.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.050+0000 7fba8719d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba800c6b10 con 0x7fba800b4e70 2023-10-06T03:33:53.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.050+0000 7fba8719d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba800c5f00 con 0x7fba800b16d0 2023-10-06T03:33:53.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.051+0000 7fba8599a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba800b1b40 0x7fba800c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.051+0000 7fba8619b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba800b16d0 0x7fba800c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.051+0000 7fba8599a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba800b1b40 0x7fba800c7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36306/0 (socket says 172.21.15.55:36306) 2023-10-06T03:33:53.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.051+0000 7fba8619b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba800b16d0 0x7fba800c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:34906/0 (socket says 172.21.15.55:34906) 2023-10-06T03:33:53.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.051+0000 7fba8599a700 1 -- 172.21.15.55:0/1023899892 learned_addr learned my addr 172.21.15.55:0/1023899892 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:53.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.051+0000 7fba8699c700 1 --2- 172.21.15.55:0/1023899892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba800b4e70 0x7fba800c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.051+0000 7fba8599a700 1 -- 172.21.15.55:0/1023899892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba800b16d0 msgr2=0x7fba800c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.051+0000 7fba8599a700 1 --2- 172.21.15.55:0/1023899892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba800b16d0 0x7fba800c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.051+0000 7fba8599a700 1 -- 172.21.15.55:0/1023899892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba800b4e70 msgr2=0x7fba800c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.051+0000 7fba8599a700 1 --2- 172.21.15.55:0/1023899892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba800b4e70 0x7fba800c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.051+0000 7fba8599a700 1 -- 172.21.15.55:0/1023899892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c009420 con 0x7fba800b1b40 2023-10-06T03:33:53.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.051+0000 7fba8599a700 1 --2- 172.21.15.55:0/1023899892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba800b1b40 0x7fba800c7710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fba7000c790 tx=0x7fba70007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:53.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.052+0000 7fba767fc700 1 -- 172.21.15.55:0/1023899892 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70010510 con 0x7fba800b1b40 2023-10-06T03:33:53.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.052+0000 7fba767fc700 1 -- 172.21.15.55:0/1023899892 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba70010670 con 0x7fba800b1b40 2023-10-06T03:33:53.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.052+0000 7fba8719d700 1 -- 172.21.15.55:0/1023899892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba800c61e0 con 0x7fba800b1b40 2023-10-06T03:33:53.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.052+0000 7fba767fc700 1 -- 172.21.15.55:0/1023899892 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70010a10 con 0x7fba800b1b40 2023-10-06T03:33:53.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.052+0000 7fba8719d700 1 -- 172.21.15.55:0/1023899892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba800c67a0 con 0x7fba800b1b40 2023-10-06T03:33:53.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.053+0000 7fba8719d700 1 -- 172.21.15.55:0/1023899892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba80088990 con 0x7fba800b1b40 2023-10-06T03:33:53.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.054+0000 7fba767fc700 1 -- 172.21.15.55:0/1023899892 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba7001b3b0 con 0x7fba800b1b40 2023-10-06T03:33:53.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.054+0000 7fba767fc700 1 --2- 172.21.15.55:0/1023899892 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fba64066f30 0x7fba640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.055+0000 7fba767fc700 1 -- 172.21.15.55:0/1023899892 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fba70083fd0 con 0x7fba800b1b40 2023-10-06T03:33:53.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.055+0000 7fba8619b700 1 --2- 172.21.15.55:0/1023899892 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fba64066f30 0x7fba640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.056+0000 7fba8619b700 1 --2- 172.21.15.55:0/1023899892 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fba64066f30 0x7fba640693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0093f0 tx=0x7fba7c016000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:53.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.058+0000 7fba767fc700 1 -- 172.21.15.55:0/1023899892 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba70051040 con 0x7fba800b1b40 2023-10-06T03:33:53.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.256+0000 7fba8719d700 1 -- 172.21.15.55:0/1023899892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba800c6c50 con 0x7fba800b1b40 2023-10-06T03:33:53.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.257+0000 7fba767fc700 1 -- 172.21.15.55:0/1023899892 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba700170a0 con 0x7fba800b1b40 2023-10-06T03:33:53.258 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T03:33:53.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.259+0000 7fba8719d700 1 -- 172.21.15.55:0/1023899892 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fba64066f30 msgr2=0x7fba640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.259+0000 7fba8719d700 1 --2- 172.21.15.55:0/1023899892 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fba64066f30 0x7fba640693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0093f0 tx=0x7fba7c016000 comp rx=0 tx=0).stop 2023-10-06T03:33:53.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.259+0000 7fba8719d700 1 -- 172.21.15.55:0/1023899892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba800b1b40 msgr2=0x7fba800c7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.259+0000 7fba8719d700 1 --2- 172.21.15.55:0/1023899892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba800b1b40 0x7fba800c7710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fba7000c790 tx=0x7fba70007e10 comp rx=0 tx=0).stop 2023-10-06T03:33:53.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.259+0000 7fba8719d700 1 -- 172.21.15.55:0/1023899892 shutdown_connections 2023-10-06T03:33:53.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.259+0000 7fba8719d700 1 --2- 172.21.15.55:0/1023899892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fba800b4e70 0x7fba800c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.259+0000 7fba8719d700 1 --2- 172.21.15.55:0/1023899892 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fba64066f30 0x7fba640693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.259+0000 7fba8719d700 1 --2- 172.21.15.55:0/1023899892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fba800b16d0 0x7fba800c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.259+0000 7fba8719d700 1 --2- 172.21.15.55:0/1023899892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fba800b1b40 0x7fba800c7710 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.259+0000 7fba8719d700 1 -- 172.21.15.55:0/1023899892 >> 172.21.15.55:0/1023899892 conn(0x7fba800d10c0 msgr2=0x7fba80069370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:53.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.259+0000 7fba8719d700 1 -- 172.21.15.55:0/1023899892 shutdown_connections 2023-10-06T03:33:53.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.259+0000 7fba8719d700 1 -- 172.21.15.55:0/1023899892 wait complete. 2023-10-06T03:33:53.278 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:33:53.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.470+0000 7f55ecd9d700 1 Processor -- start 2023-10-06T03:33:53.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.470+0000 7f55ecd9d700 1 -- start start 2023-10-06T03:33:53.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.471+0000 7f55ecd9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f55e8159a60 0x7f55e80c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.471+0000 7f55ecd9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55e80ccc00 0x7f55e8069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.471+0000 7f55ecd9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e80c9cb0 con 0x7f55e806a020 2023-10-06T03:33:53.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.471+0000 7f55ecd9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e806dfc0 con 0x7f55e8159a60 2023-10-06T03:33:53.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.471+0000 7f55ecd9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e80c9f30 con 0x7f55e80ccc00 2023-10-06T03:33:53.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.471+0000 7f55e7fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f55e806a020 0x7f55e80cb230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45290/0 (socket says 172.21.15.55:45290) 2023-10-06T03:33:53.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.471+0000 7f55e7fff700 1 -- 172.21.15.55:0/1302259534 learned_addr learned my addr 172.21.15.55:0/1302259534 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:53.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.471+0000 7f55e6ffd700 1 --2- 172.21.15.55:0/1302259534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55e80ccc00 0x7f55e8069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.471+0000 7f55e77fe700 1 --2- 172.21.15.55:0/1302259534 >> v2:172.21.15.163:3300/0 conn(0x7f55e8159a60 0x7f55e80c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.472+0000 7f55e67fc700 1 -- 172.21.15.55:0/1302259534 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900227302 0 0) 0x7f55e80c9cb0 con 0x7f55e806a020 2023-10-06T03:33:53.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.472+0000 7f55e67fc700 1 -- 172.21.15.55:0/1302259534 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55d0003570 con 0x7f55e806a020 2023-10-06T03:33:53.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.472+0000 7f55e6ffd700 1 -- 172.21.15.55:0/1302259534 >> v2:172.21.15.163:3300/0 conn(0x7f55e8159a60 msgr2=0x7f55e80c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:53.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.472+0000 7f55e6ffd700 1 --2- 172.21.15.55:0/1302259534 >> v2:172.21.15.163:3300/0 conn(0x7f55e8159a60 0x7f55e80c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.472+0000 7f55e6ffd700 1 -- 172.21.15.55:0/1302259534 >> v1:172.21.15.55:6789/0 conn(0x7f55e806a020 legacy=0x7f55e80cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.472+0000 7f55e6ffd700 1 -- 172.21.15.55:0/1302259534 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e80ca470 con 0x7f55e80ccc00 2023-10-06T03:33:53.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.472+0000 7f55e6ffd700 1 --2- 172.21.15.55:0/1302259534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55e80ccc00 0x7f55e8069ae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0098e0 tx=0x7f55dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0fd2e3817d8b683 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:53.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.473+0000 7f55e67fc700 1 -- 172.21.15.55:0/1302259534 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc011630 con 0x7f55e80ccc00 2023-10-06T03:33:53.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.473+0000 7f55e67fc700 1 -- 172.21.15.55:0/1302259534 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc011790 con 0x7f55e80ccc00 2023-10-06T03:33:53.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.473+0000 7f55e67fc700 1 -- 172.21.15.55:0/1302259534 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc011b30 con 0x7f55e80ccc00 2023-10-06T03:33:53.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.473+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1302259534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55e80ccc00 msgr2=0x7f55e8069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.473+0000 7f55ecd9d700 1 --2- 172.21.15.55:0/1302259534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55e80ccc00 0x7f55e8069ae0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0098e0 tx=0x7f55dc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.473+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1302259534 shutdown_connections 2023-10-06T03:33:53.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.473+0000 7f55ecd9d700 1 --2- 172.21.15.55:0/1302259534 >> v2:172.21.15.163:3300/0 conn(0x7f55e8159a60 0x7f55e80c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.473+0000 7f55ecd9d700 1 --2- 172.21.15.55:0/1302259534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55e80ccc00 0x7f55e8069ae0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.473+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1302259534 >> 172.21.15.55:0/1302259534 conn(0x7f55e806baa0 msgr2=0x7f55e806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:53.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.473+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1302259534 shutdown_connections 2023-10-06T03:33:53.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.473+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1302259534 wait complete. 2023-10-06T03:33:53.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.475+0000 7f55ecd9d700 1 Processor -- start 2023-10-06T03:33:53.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.475+0000 7f55ecd9d700 1 -- start start 2023-10-06T03:33:53.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55ecd9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e806a020 0x7f55e81526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55ecd9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55e80ccc00 0x7f55e8169ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55ecd9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55e8159a60 0x7f55e80b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55ecd9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8154ae0 con 0x7f55e806a020 2023-10-06T03:33:53.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55ecd9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e81550a0 con 0x7f55e8159a60 2023-10-06T03:33:53.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55ecd9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e816a7a0 con 0x7f55e80ccc00 2023-10-06T03:33:53.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55e77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e806a020 0x7f55e81526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55e77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e806a020 0x7f55e81526c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51420/0 (socket says 172.21.15.55:51420) 2023-10-06T03:33:53.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55e77fe700 1 -- 172.21.15.55:0/1031622309 learned_addr learned my addr 172.21.15.55:0/1031622309 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:53.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55e77fe700 1 -- 172.21.15.55:0/1031622309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55e80ccc00 msgr2=0x7f55e8169ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55e77fe700 1 --2- 172.21.15.55:0/1031622309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55e80ccc00 0x7f55e8169ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55e77fe700 1 -- 172.21.15.55:0/1031622309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55e8159a60 msgr2=0x7f55e80b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55e7fff700 1 --2- 172.21.15.55:0/1031622309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55e8159a60 0x7f55e80b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55e77fe700 1 --2- 172.21.15.55:0/1031622309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55e8159a60 0x7f55e80b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55e77fe700 1 -- 172.21.15.55:0/1031622309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc009420 con 0x7f55e806a020 2023-10-06T03:33:53.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.476+0000 7f55e7fff700 1 --2- 172.21.15.55:0/1031622309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55e8159a60 0x7f55e80b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:53.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.477+0000 7f55e77fe700 1 --2- 172.21.15.55:0/1031622309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e806a020 0x7f55e81526c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f55d800e960 tx=0x7f55d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:53.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.477+0000 7f55cf7fe700 1 -- 172.21.15.55:0/1031622309 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d800c850 con 0x7f55e806a020 2023-10-06T03:33:53.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.477+0000 7f55cf7fe700 1 -- 172.21.15.55:0/1031622309 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d800c9b0 con 0x7f55e806a020 2023-10-06T03:33:53.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.477+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1031622309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e816aa80 con 0x7f55e806a020 2023-10-06T03:33:53.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.477+0000 7f55cf7fe700 1 -- 172.21.15.55:0/1031622309 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d800cd50 con 0x7f55e806a020 2023-10-06T03:33:53.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.477+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1031622309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e80b1610 con 0x7f55e806a020 2023-10-06T03:33:53.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.478+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1031622309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e8155380 con 0x7f55e806a020 2023-10-06T03:33:53.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.479+0000 7f55cf7fe700 1 -- 172.21.15.55:0/1031622309 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f55d8013680 con 0x7f55e806a020 2023-10-06T03:33:53.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.480+0000 7f55cf7fe700 1 --2- 172.21.15.55:0/1031622309 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f55d4066f30 0x7f55d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.480+0000 7f55cf7fe700 1 -- 172.21.15.55:0/1031622309 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f55d80848c0 con 0x7f55e806a020 2023-10-06T03:33:53.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.480+0000 7f55e6ffd700 1 --2- 172.21.15.55:0/1031622309 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f55d4066f30 0x7f55d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.481+0000 7f55e6ffd700 1 --2- 172.21.15.55:0/1031622309 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f55d4066f30 0x7f55d40693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f55e8167830 tx=0x7f55dc017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:53.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.484+0000 7f55cf7fe700 1 -- 172.21.15.55:0/1031622309 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55d80519d0 con 0x7f55e806a020 2023-10-06T03:33:53.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.686+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1031622309 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f55e80ca750 con 0x7f55e806a020 2023-10-06T03:33:53.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.687+0000 7f55cf7fe700 1 -- 172.21.15.55:0/1031622309 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f55d8054ff0 con 0x7f55e806a020 2023-10-06T03:33:53.688 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T03:33:53.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.689+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1031622309 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f55d4066f30 msgr2=0x7f55d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.689+0000 7f55ecd9d700 1 --2- 172.21.15.55:0/1031622309 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f55d4066f30 0x7f55d40693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f55e8167830 tx=0x7f55dc017000 comp rx=0 tx=0).stop 2023-10-06T03:33:53.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.689+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1031622309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e806a020 msgr2=0x7f55e81526c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.689+0000 7f55ecd9d700 1 --2- 172.21.15.55:0/1031622309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e806a020 0x7f55e81526c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f55d800e960 tx=0x7f55d80091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.689+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1031622309 shutdown_connections 2023-10-06T03:33:53.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.689+0000 7f55ecd9d700 1 --2- 172.21.15.55:0/1031622309 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55e8159a60 0x7f55e80b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.689+0000 7f55ecd9d700 1 --2- 172.21.15.55:0/1031622309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55e806a020 0x7f55e81526c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.689+0000 7f55ecd9d700 1 --2- 172.21.15.55:0/1031622309 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f55d4066f30 0x7f55d40693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.689+0000 7f55ecd9d700 1 --2- 172.21.15.55:0/1031622309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55e80ccc00 0x7f55e8169ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.689+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1031622309 >> 172.21.15.55:0/1031622309 conn(0x7f55e806baa0 msgr2=0x7f55e806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:53.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.689+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1031622309 shutdown_connections 2023-10-06T03:33:53.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.689+0000 7f55ecd9d700 1 -- 172.21.15.55:0/1031622309 wait complete. 2023-10-06T03:33:53.717 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-10-06T03:33:53.717 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T03:33:53.717 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-06T03:33:53.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.902+0000 7f2b9a99d700 1 Processor -- start 2023-10-06T03:33:53.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.902+0000 7f2b9a99d700 1 -- start start 2023-10-06T03:33:53.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9a99d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b94156310 0x7f2b940c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9a99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b941566e0 0x7f2b940ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9a99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2b940ca300 0x7f2b940ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9a99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b940ca860 con 0x7f2b940ca300 2023-10-06T03:33:53.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9a99d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b940cab40 con 0x7f2b94156310 2023-10-06T03:33:53.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9a99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b940cae20 con 0x7f2b941566e0 2023-10-06T03:33:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9a19c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2b940ca300 0x7f2b940ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9a19c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2b940ca300 0x7f2b940ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51426/0 (socket says 172.21.15.55:51426) 2023-10-06T03:33:53.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9a19c700 1 -- 172.21.15.55:0/2416025282 learned_addr learned my addr 172.21.15.55:0/2416025282 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9999b700 1 --2- 172.21.15.55:0/2416025282 >> v2:172.21.15.163:3300/0 conn(0x7f2b94156310 0x7f2b940c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9919a700 1 --2- 172.21.15.55:0/2416025282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b941566e0 0x7f2b940ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9a19c700 1 -- 172.21.15.55:0/2416025282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b941566e0 msgr2=0x7f2b940ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9a19c700 1 --2- 172.21.15.55:0/2416025282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b941566e0 0x7f2b940ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9a19c700 1 -- 172.21.15.55:0/2416025282 >> v2:172.21.15.163:3300/0 conn(0x7f2b94156310 msgr2=0x7f2b940c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.904+0000 7f2b9a19c700 1 --2- 172.21.15.55:0/2416025282 >> v2:172.21.15.163:3300/0 conn(0x7f2b94156310 0x7f2b940c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.905+0000 7f2b9a19c700 1 -- 172.21.15.55:0/2416025282 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b940cb100 con 0x7f2b940ca300 2023-10-06T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.905+0000 7f2b9a19c700 1 --2- 172.21.15.55:0/2416025282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b940ca300 0x7f2b940ae540 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b940cce20 tx=0x7f2b8c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a89f068c1ea33f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.905+0000 7f2b98999700 1 -- 172.21.15.55:0/2416025282 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c010070 con 0x7f2b940ca300 2023-10-06T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.905+0000 7f2b98999700 1 -- 172.21.15.55:0/2416025282 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c00de90 con 0x7f2b940ca300 2023-10-06T03:33:53.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.905+0000 7f2b98999700 1 -- 172.21.15.55:0/2416025282 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c00b320 con 0x7f2b940ca300 2023-10-06T03:33:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.906+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2416025282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b940ca300 msgr2=0x7f2b940ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.906+0000 7f2b9a99d700 1 --2- 172.21.15.55:0/2416025282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b940ca300 0x7f2b940ae540 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b940cce20 tx=0x7f2b8c00ba50 comp rx=0 tx=0).stop 2023-10-06T03:33:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.906+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2416025282 shutdown_connections 2023-10-06T03:33:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.906+0000 7f2b9a99d700 1 --2- 172.21.15.55:0/2416025282 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b940ca300 0x7f2b940ae540 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.906+0000 7f2b9a99d700 1 --2- 172.21.15.55:0/2416025282 >> v2:172.21.15.163:3300/0 conn(0x7f2b94156310 0x7f2b940c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.906+0000 7f2b9a99d700 1 --2- 172.21.15.55:0/2416025282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b941566e0 0x7f2b940ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.906+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2416025282 >> 172.21.15.55:0/2416025282 conn(0x7f2b940cc1f0 msgr2=0x7f2b940cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.906+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2416025282 shutdown_connections 2023-10-06T03:33:53.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.906+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2416025282 wait complete. 2023-10-06T03:33:53.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.907+0000 7f2b9a99d700 1 Processor -- start 2023-10-06T03:33:53.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.907+0000 7f2b9a99d700 1 -- start start 2023-10-06T03:33:53.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.908+0000 7f2b9a99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b940ca300 0x7f2b940c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.908+0000 7f2b9a99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b94156310 0x7f2b94158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.908+0000 7f2b9a99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b941566e0 0x7f2b94158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.908+0000 7f2b9a99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9416de90 con 0x7f2b940ca300 2023-10-06T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.908+0000 7f2b9a99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9416cfa0 con 0x7f2b94156310 2023-10-06T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.908+0000 7f2b9a99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9416d280 con 0x7f2b941566e0 2023-10-06T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.909+0000 7f2b9999b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b940ca300 0x7f2b940c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.909+0000 7f2b9999b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b940ca300 0x7f2b940c7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51440/0 (socket says 172.21.15.55:51440) 2023-10-06T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.909+0000 7f2b9999b700 1 -- 172.21.15.55:0/2748191590 learned_addr learned my addr 172.21.15.55:0/2748191590 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.909+0000 7f2b9919a700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b94156310 0x7f2b94158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.909+0000 7f2b9a19c700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b941566e0 0x7f2b94158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.909+0000 7f2b9999b700 1 -- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b941566e0 msgr2=0x7f2b94158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.909+0000 7f2b9999b700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b941566e0 0x7f2b94158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.909+0000 7f2b9999b700 1 -- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b94156310 msgr2=0x7f2b94158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:53.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.909+0000 7f2b9999b700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b94156310 0x7f2b94158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:53.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.909+0000 7f2b9999b700 1 -- 172.21.15.55:0/2748191590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c00bcf0 con 0x7f2b940ca300 2023-10-06T03:33:53.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.909+0000 7f2b9a19c700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b941566e0 0x7f2b94158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:53.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.909+0000 7f2b9999b700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b940ca300 0x7f2b940c7900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000d890 tx=0x7f2b90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:53.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.910+0000 7f2b89ffb700 1 -- 172.21.15.55:0/2748191590 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b900125f0 con 0x7f2b940ca300 2023-10-06T03:33:53.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.910+0000 7f2b89ffb700 1 -- 172.21.15.55:0/2748191590 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b90012750 con 0x7f2b940ca300 2023-10-06T03:33:53.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.910+0000 7f2b89ffb700 1 -- 172.21.15.55:0/2748191590 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90012ad0 con 0x7f2b940ca300 2023-10-06T03:33:53.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.910+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2748191590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9416d560 con 0x7f2b940ca300 2023-10-06T03:33:53.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.910+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2748191590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9416db20 con 0x7f2b940ca300 2023-10-06T03:33:53.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.911+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2748191590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9404c3c0 con 0x7f2b940ca300 2023-10-06T03:33:53.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.912+0000 7f2b89ffb700 1 -- 172.21.15.55:0/2748191590 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b9001e460 con 0x7f2b940ca300 2023-10-06T03:33:53.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.912+0000 7f2b89ffb700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2b78066f30 0x7f2b780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:53.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.912+0000 7f2b89ffb700 1 -- 172.21.15.55:0/2748191590 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2b90084ca0 con 0x7f2b940ca300 2023-10-06T03:33:53.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.913+0000 7f2b9919a700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2b78066f30 0x7f2b780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.913+0000 7f2b9919a700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2b78066f30 0x7f2b780693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2b940adaa0 tx=0x7f2b84008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:53.916+0000 7f2b89ffb700 1 -- 172.21.15.55:0/2748191590 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b90051db0 con 0x7f2b940ca300 2023-10-06T03:33:54.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.198+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2748191590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f2b941525b0 con 0x7f2b940ca300 2023-10-06T03:33:54.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.688+0000 7f2b89ffb700 1 -- 172.21.15.55:0/2748191590 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2b900553d0 con 0x7f2b940ca300 2023-10-06T03:33:54.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.690+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2b78066f30 msgr2=0x7f2b780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:54.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.690+0000 7f2b9a99d700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2b78066f30 0x7f2b780693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2b940adaa0 tx=0x7f2b84008040 comp rx=0 tx=0).stop 2023-10-06T03:33:54.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.690+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2748191590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b940ca300 msgr2=0x7f2b940c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:54.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.690+0000 7f2b9a99d700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b940ca300 0x7f2b940c7900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000d890 tx=0x7f2b90008040 comp rx=0 tx=0).stop 2023-10-06T03:33:54.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.690+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2748191590 shutdown_connections 2023-10-06T03:33:54.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.690+0000 7f2b9a99d700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2b78066f30 0x7f2b780693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:54.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.690+0000 7f2b9a99d700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b941566e0 0x7f2b94158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:54.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.690+0000 7f2b9a99d700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b940ca300 0x7f2b940c7900 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:54.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.690+0000 7f2b9a99d700 1 --2- 172.21.15.55:0/2748191590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b94156310 0x7f2b94158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:54.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.690+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2748191590 >> 172.21.15.55:0/2748191590 conn(0x7f2b940cc1f0 msgr2=0x7f2b94151ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:54.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.691+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2748191590 shutdown_connections 2023-10-06T03:33:54.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.691+0000 7f2b9a99d700 1 -- 172.21.15.55:0/2748191590 wait complete. 2023-10-06T03:33:54.711 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:33:54.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.902+0000 7f96bf59e700 1 Processor -- start 2023-10-06T03:33:54.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.902+0000 7f96bf59e700 1 -- start start 2023-10-06T03:33:54.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.903+0000 7f96bf59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f96c00b16d0 0x7f96c00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:54.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.903+0000 7f96bf59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c00b2080 con 0x7f96c00b4e70 2023-10-06T03:33:54.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.903+0000 7f96bf59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96c00cfb00 con 0x7f96c00b33b0 2023-10-06T03:33:54.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.903+0000 7f96bf59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c00b5c40 con 0x7f96c00b16d0 2023-10-06T03:33:54.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.904+0000 7f96be59c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f96c00b4e70 0x7f96c00ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45306/0 (socket says 172.21.15.55:45306) 2023-10-06T03:33:54.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.904+0000 7f96bdd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f96c00b16d0 0x7f96c00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:54.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.904+0000 7f96be59c700 1 -- 172.21.15.55:0/2838099335 learned_addr learned my addr 172.21.15.55:0/2838099335 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:54.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.904+0000 7f96bd59a700 1 -- 172.21.15.55:0/2838099335 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1701170405 0 0) 0x7f96c00b2080 con 0x7f96c00b4e70 2023-10-06T03:33:54.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.904+0000 7f96bd59a700 1 -- 172.21.15.55:0/2838099335 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96a8003570 con 0x7f96c00b4e70 2023-10-06T03:33:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.904+0000 7f96bdd9b700 1 -- 172.21.15.55:0/2838099335 >> v1:172.21.15.163:6789/0 conn(0x7f96c00b33b0 legacy=0x7f96c00b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.904+0000 7f96bdd9b700 1 -- 172.21.15.55:0/2838099335 >> v1:172.21.15.55:6789/0 conn(0x7f96c00b4e70 legacy=0x7f96c00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.904+0000 7f96bdd9b700 1 -- 172.21.15.55:0/2838099335 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c00b5f20 con 0x7f96c00b16d0 2023-10-06T03:33:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.904+0000 7f96bdd9b700 1 --2- 172.21.15.55:0/2838099335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c00b16d0 0x7f96c00b1b40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f96b40098e0 tx=0x7f96b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35e18ceffe7de03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.905+0000 7f96bd59a700 1 -- 172.21.15.55:0/2838099335 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b400b640 con 0x7f96c00b16d0 2023-10-06T03:33:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.905+0000 7f96bd59a700 1 -- 172.21.15.55:0/2838099335 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b400b7a0 con 0x7f96c00b16d0 2023-10-06T03:33:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.905+0000 7f96bd59a700 1 -- 172.21.15.55:0/2838099335 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b400bb40 con 0x7f96c00b16d0 2023-10-06T03:33:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.905+0000 7f96bf59e700 1 -- 172.21.15.55:0/2838099335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c00b16d0 msgr2=0x7f96c00b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:54.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.905+0000 7f96bf59e700 1 --2- 172.21.15.55:0/2838099335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c00b16d0 0x7f96c00b1b40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f96b40098e0 tx=0x7f96b40091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.905+0000 7f96bf59e700 1 -- 172.21.15.55:0/2838099335 shutdown_connections 2023-10-06T03:33:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.905+0000 7f96bf59e700 1 --2- 172.21.15.55:0/2838099335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c00b16d0 0x7f96c00b1b40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.905+0000 7f96bf59e700 1 -- 172.21.15.55:0/2838099335 >> 172.21.15.55:0/2838099335 conn(0x7f96c00d10c0 msgr2=0x7f96c00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.905+0000 7f96bf59e700 1 -- 172.21.15.55:0/2838099335 shutdown_connections 2023-10-06T03:33:54.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.905+0000 7f96bf59e700 1 -- 172.21.15.55:0/2838099335 wait complete. 2023-10-06T03:33:54.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.907+0000 7f96bf59e700 1 Processor -- start 2023-10-06T03:33:54.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.907+0000 7f96bf59e700 1 -- start start 2023-10-06T03:33:54.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.908+0000 7f96bf59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c00b16d0 0x7f96c0157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:54.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.908+0000 7f96bf59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c00b33b0 0x7f96c015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:54.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.908+0000 7f96bf59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00b4e70 0x7f96c00cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:54.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.908+0000 7f96bf59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c015e200 con 0x7f96c00b33b0 2023-10-06T03:33:54.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.908+0000 7f96bf59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c015ad90 con 0x7f96c00b4e70 2023-10-06T03:33:54.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.908+0000 7f96bf59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c015b070 con 0x7f96c00b16d0 2023-10-06T03:33:54.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.908+0000 7f96bdd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c00b33b0 0x7f96c015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:54.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.908+0000 7f96bdd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c00b33b0 0x7f96c015dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51454/0 (socket says 172.21.15.55:51454) 2023-10-06T03:33:54.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.908+0000 7f96bdd9b700 1 -- 172.21.15.55:0/430610568 learned_addr learned my addr 172.21.15.55:0/430610568 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:54.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.908+0000 7f96bed9d700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00b4e70 0x7f96c00cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:54.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.908+0000 7f96be59c700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c00b16d0 0x7f96c0157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:54.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.908+0000 7f96bdd9b700 1 -- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c00b16d0 msgr2=0x7f96c0157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:54.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.908+0000 7f96bdd9b700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c00b16d0 0x7f96c0157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:54.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.909+0000 7f96bdd9b700 1 -- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00b4e70 msgr2=0x7f96c00cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:54.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.909+0000 7f96bdd9b700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00b4e70 0x7f96c00cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:54.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.909+0000 7f96bdd9b700 1 -- 172.21.15.55:0/430610568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b4009420 con 0x7f96c00b33b0 2023-10-06T03:33:54.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.909+0000 7f96bed9d700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00b4e70 0x7f96c00cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:54.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.909+0000 7f96bdd9b700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c00b33b0 0x7f96c015dcc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f96b40093f0 tx=0x7f96b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:54.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.909+0000 7f96ae7fc700 1 -- 172.21.15.55:0/430610568 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4000c40 con 0x7f96c00b33b0 2023-10-06T03:33:54.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.909+0000 7f96ae7fc700 1 -- 172.21.15.55:0/430610568 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b400b910 con 0x7f96c00b33b0 2023-10-06T03:33:54.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.909+0000 7f96ae7fc700 1 -- 172.21.15.55:0/430610568 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4025670 con 0x7f96c00b33b0 2023-10-06T03:33:54.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.909+0000 7f96bf59e700 1 -- 172.21.15.55:0/430610568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c015b350 con 0x7f96c00b33b0 2023-10-06T03:33:54.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.909+0000 7f96bf59e700 1 -- 172.21.15.55:0/430610568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c015baa0 con 0x7f96c00b33b0 2023-10-06T03:33:54.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.910+0000 7f96bf59e700 1 -- 172.21.15.55:0/430610568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c015b610 con 0x7f96c00b33b0 2023-10-06T03:33:54.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.911+0000 7f96ae7fc700 1 -- 172.21.15.55:0/430610568 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f96b4023070 con 0x7f96c00b33b0 2023-10-06T03:33:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.912+0000 7f96ae7fc700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f96a0066f30 0x7f96a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.912+0000 7f96ae7fc700 1 -- 172.21.15.55:0/430610568 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f96b4085970 con 0x7f96c00b33b0 2023-10-06T03:33:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.913+0000 7f96be59c700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f96a0066f30 0x7f96a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:54.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.913+0000 7f96be59c700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f96a0066f30 0x7f96a00693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f96c0159cb0 tx=0x7f96b0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:54.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:54.916+0000 7f96ae7fc700 1 -- 172.21.15.55:0/430610568 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96b4052a80 con 0x7f96c00b33b0 2023-10-06T03:33:55.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:55.119+0000 7f96bf59e700 1 -- 172.21.15.55:0/430610568 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f96c015bcc0 con 0x7f96c00b33b0 2023-10-06T03:33:55.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:55.123+0000 7f96ae7fc700 1 -- 172.21.15.55:0/430610568 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f96b40150c0 con 0x7f96c00b33b0 2023-10-06T03:33:55.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:55.126+0000 7f96bf59e700 1 -- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f96a0066f30 msgr2=0x7f96a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:55.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:55.126+0000 7f96bf59e700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f96a0066f30 0x7f96a00693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f96c0159cb0 tx=0x7f96b0009040 comp rx=0 tx=0).stop 2023-10-06T03:33:55.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:55.126+0000 7f96bf59e700 1 -- 172.21.15.55:0/430610568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c00b33b0 msgr2=0x7f96c015dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:55.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:55.126+0000 7f96bf59e700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c00b33b0 0x7f96c015dcc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f96b40093f0 tx=0x7f96b4017000 comp rx=0 tx=0).stop 2023-10-06T03:33:55.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:55.126+0000 7f96bf59e700 1 -- 172.21.15.55:0/430610568 shutdown_connections 2023-10-06T03:33:55.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:55.126+0000 7f96bf59e700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96c00b4e70 0x7f96c00cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:55.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:55.126+0000 7f96bf59e700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f96a0066f30 0x7f96a00693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:55.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:55.126+0000 7f96bf59e700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96c00b16d0 0x7f96c0157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:55.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:55.126+0000 7f96bf59e700 1 --2- 172.21.15.55:0/430610568 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96c00b33b0 0x7f96c015dcc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:55.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:55.126+0000 7f96bf59e700 1 -- 172.21.15.55:0/430610568 >> 172.21.15.55:0/430610568 conn(0x7f96c00d10c0 msgr2=0x7f96c0068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:55.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:55.126+0000 7f96bf59e700 1 -- 172.21.15.55:0/430610568 shutdown_connections 2023-10-06T03:33:55.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:55.126+0000 7f96bf59e700 1 -- 172.21.15.55:0/430610568 wait complete. 2023-10-06T03:33:55.144 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:33:55.144 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:33:55.223 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:33:55.313 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:33:55.313 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:33:55.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:33:55.313 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-06T03:33:55.400 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T03:33:55.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:55.401 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:33:55.449 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-06T03:33:55.449 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:33:55.449 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-06T03:33:55.578 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-06T03:33:55.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:55.579 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T03:33:55.639 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-06T03:33:55.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:33:55.640 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-06T03:33:55.720 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T03:33:55.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:55.720 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T03:33:55.758 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-06T03:33:55.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:33:55.758 DEBUG:teuthology.orchestra.run.smithi163:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-06T03:33:55.838 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi163.front.sepia.ceph.com 2023-10-06T03:33:55.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:33:55.839 DEBUG:teuthology.orchestra.run.smithi163:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T03:33:55.876 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:33:55.876 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:33:56.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.062+0000 7fe153fff700 1 Processor -- start 2023-10-06T03:33:56.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.062+0000 7fe153fff700 1 -- start start 2023-10-06T03:33:56.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.064+0000 7fe153fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe1540c6a70 0x7fe15406acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.064+0000 7fe153fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe15406a840 con 0x7fe1540696c0 2023-10-06T03:33:56.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.064+0000 7fe153fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe154069c30 con 0x7fe1540ca660 2023-10-06T03:33:56.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.064+0000 7fe153fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15406b210 con 0x7fe1540c6a70 2023-10-06T03:33:56.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.064+0000 7fe1527fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe1540696c0 0x7fe1540c6e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45312/0 (socket says 172.21.15.55:45312) 2023-10-06T03:33:56.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.064+0000 7fe1527fc700 1 -- 172.21.15.55:0/2454325441 learned_addr learned my addr 172.21.15.55:0/2454325441 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:56.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.064+0000 7fe152ffd700 1 --2- 172.21.15.55:0/2454325441 >> v2:172.21.15.163:3300/0 conn(0x7fe1540c6a70 0x7fe15406acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.064+0000 7fe151ffb700 1 -- 172.21.15.55:0/2454325441 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833287437 0 0) 0x7fe15406a840 con 0x7fe1540696c0 2023-10-06T03:33:56.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.064+0000 7fe151ffb700 1 -- 172.21.15.55:0/2454325441 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe13c003570 con 0x7fe1540696c0 2023-10-06T03:33:56.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.064+0000 7fe151ffb700 1 -- 172.21.15.55:0/2454325441 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202316745 0 0) 0x7fe154069c30 con 0x7fe1540ca660 2023-10-06T03:33:56.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.064+0000 7fe151ffb700 1 -- 172.21.15.55:0/2454325441 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe15406a840 con 0x7fe1540ca660 2023-10-06T03:33:56.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.065+0000 7fe152ffd700 1 -- 172.21.15.55:0/2454325441 >> v1:172.21.15.163:6789/0 conn(0x7fe1540ca660 legacy=0x7fe15406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.065+0000 7fe152ffd700 1 -- 172.21.15.55:0/2454325441 >> v1:172.21.15.55:6789/0 conn(0x7fe1540696c0 legacy=0x7fe1540c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.065+0000 7fe152ffd700 1 -- 172.21.15.55:0/2454325441 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe154069f10 con 0x7fe1540c6a70 2023-10-06T03:33:56.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.065+0000 7fe152ffd700 1 --2- 172.21.15.55:0/2454325441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1540c6a70 0x7fe15406acd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe1440098e0 tx=0x7fe1440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb99b6f66312daf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:56.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.065+0000 7fe151ffb700 1 -- 172.21.15.55:0/2454325441 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe144011630 con 0x7fe1540c6a70 2023-10-06T03:33:56.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.065+0000 7fe151ffb700 1 -- 172.21.15.55:0/2454325441 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe144011790 con 0x7fe1540c6a70 2023-10-06T03:33:56.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.065+0000 7fe151ffb700 1 -- 172.21.15.55:0/2454325441 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe144011b10 con 0x7fe1540c6a70 2023-10-06T03:33:56.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.066+0000 7fe153fff700 1 -- 172.21.15.55:0/2454325441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1540c6a70 msgr2=0x7fe15406acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.066+0000 7fe153fff700 1 --2- 172.21.15.55:0/2454325441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1540c6a70 0x7fe15406acd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe1440098e0 tx=0x7fe1440091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.066+0000 7fe153fff700 1 -- 172.21.15.55:0/2454325441 shutdown_connections 2023-10-06T03:33:56.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.066+0000 7fe153fff700 1 --2- 172.21.15.55:0/2454325441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1540c6a70 0x7fe15406acd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.066+0000 7fe153fff700 1 -- 172.21.15.55:0/2454325441 >> 172.21.15.55:0/2454325441 conn(0x7fe1540cc1f0 msgr2=0x7fe1540cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:56.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.066+0000 7fe153fff700 1 -- 172.21.15.55:0/2454325441 shutdown_connections 2023-10-06T03:33:56.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.066+0000 7fe153fff700 1 -- 172.21.15.55:0/2454325441 wait complete. 2023-10-06T03:33:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.067+0000 7fe153fff700 1 Processor -- start 2023-10-06T03:33:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.067+0000 7fe153fff700 1 -- start start 2023-10-06T03:33:56.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.068+0000 7fe153fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1540696c0 0x7fe15415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.068+0000 7fe153fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1540c6a70 0x7fe15406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.068+0000 7fe153fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1540ca660 0x7fe1540b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.068+0000 7fe153fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15415a1d0 con 0x7fe1540ca660 2023-10-06T03:33:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.068+0000 7fe153fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1541595c0 con 0x7fe1540c6a70 2023-10-06T03:33:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.068+0000 7fe153fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1541598a0 con 0x7fe1540696c0 2023-10-06T03:33:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.068+0000 7fe1537fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1540ca660 0x7fe1540b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.068+0000 7fe1527fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1540c6a70 0x7fe15406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.069+0000 7fe152ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1540696c0 0x7fe15415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.069+0000 7fe1527fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1540c6a70 0x7fe15406e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:40020/0 (socket says 172.21.15.55:40020) 2023-10-06T03:33:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.069+0000 7fe1527fc700 1 -- 172.21.15.55:0/706596763 learned_addr learned my addr 172.21.15.55:0/706596763 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:56.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.069+0000 7fe152ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1540696c0 0x7fe15415a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:58210/0 (socket says 172.21.15.55:58210) 2023-10-06T03:33:56.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.069+0000 7fe1537fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1540ca660 0x7fe1540b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51466/0 (socket says 172.21.15.55:51466) 2023-10-06T03:33:56.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.069+0000 7fe1527fc700 1 -- 172.21.15.55:0/706596763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1540696c0 msgr2=0x7fe15415a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.069+0000 7fe1527fc700 1 --2- 172.21.15.55:0/706596763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1540696c0 0x7fe15415a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.069+0000 7fe1527fc700 1 -- 172.21.15.55:0/706596763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1540ca660 msgr2=0x7fe1540b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.069+0000 7fe1527fc700 1 --2- 172.21.15.55:0/706596763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1540ca660 0x7fe1540b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.069+0000 7fe1527fc700 1 -- 172.21.15.55:0/706596763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe144009420 con 0x7fe1540c6a70 2023-10-06T03:33:56.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.069+0000 7fe152ffd700 1 --2- 172.21.15.55:0/706596763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1540696c0 0x7fe15415a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:56.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.069+0000 7fe1537fe700 1 --2- 172.21.15.55:0/706596763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1540ca660 0x7fe1540b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:56.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.069+0000 7fe1527fc700 1 --2- 172.21.15.55:0/706596763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1540c6a70 0x7fe15406e770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe14800c790 tx=0x7fe148007e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:56.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.070+0000 7fe13affd700 1 -- 172.21.15.55:0/706596763 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148005b30 con 0x7fe1540c6a70 2023-10-06T03:33:56.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.070+0000 7fe13affd700 1 -- 172.21.15.55:0/706596763 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe148005c90 con 0x7fe1540c6a70 2023-10-06T03:33:56.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.070+0000 7fe13affd700 1 -- 172.21.15.55:0/706596763 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe148007320 con 0x7fe1540c6a70 2023-10-06T03:33:56.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.070+0000 7fe153fff700 1 -- 172.21.15.55:0/706596763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe154159b80 con 0x7fe1540c6a70 2023-10-06T03:33:56.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.070+0000 7fe153fff700 1 -- 172.21.15.55:0/706596763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15406ecb0 con 0x7fe1540c6a70 2023-10-06T03:33:56.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.071+0000 7fe153fff700 1 -- 172.21.15.55:0/706596763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15415a380 con 0x7fe1540c6a70 2023-10-06T03:33:56.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.072+0000 7fe13affd700 1 -- 172.21.15.55:0/706596763 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe14801b3b0 con 0x7fe1540c6a70 2023-10-06T03:33:56.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.073+0000 7fe13affd700 1 --2- 172.21.15.55:0/706596763 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe140066f30 0x7fe1400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.073+0000 7fe13affd700 1 -- 172.21.15.55:0/706596763 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe148083c10 con 0x7fe1540c6a70 2023-10-06T03:33:56.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.073+0000 7fe152ffd700 1 --2- 172.21.15.55:0/706596763 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe140066f30 0x7fe1400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.074+0000 7fe152ffd700 1 --2- 172.21.15.55:0/706596763 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe140066f30 0x7fe1400693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe1440093f0 tx=0x7fe144017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:56.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.077+0000 7fe13affd700 1 -- 172.21.15.55:0/706596763 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe15415a380 con 0x7fe1540c6a70 2023-10-06T03:33:56.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.323+0000 7fe153fff700 1 -- 172.21.15.55:0/706596763 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe15415a340 con 0x7fe1540c6a70 2023-10-06T03:33:56.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.324+0000 7fe13affd700 1 -- 172.21.15.55:0/706596763 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7fe148017020 con 0x7fe1540c6a70 2023-10-06T03:33:56.325 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:56.325 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2677235076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":2677235076},{"type":"v1","addr":"172.21.15.55:6835","nonce":2677235076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1030747394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1030747394},{"type":"v1","addr":"172.21.15.55:6834","nonce":1030747394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2040975046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2040975046},{"type":"v1","addr":"172.21.15.163:6836","nonce":2040975046}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:33:15.383124+0000","modified":"2023-10-06T03:33:18.908835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/1910887369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1910887369},{"type":"v1","addr":"172.21.15.163:6837","nonce":1910887369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:33:56.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.326+0000 7fe153fff700 1 -- 172.21.15.55:0/706596763 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe140066f30 msgr2=0x7fe1400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.326+0000 7fe153fff700 1 --2- 172.21.15.55:0/706596763 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe140066f30 0x7fe1400693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe1440093f0 tx=0x7fe144017000 comp rx=0 tx=0).stop 2023-10-06T03:33:56.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.326+0000 7fe153fff700 1 -- 172.21.15.55:0/706596763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1540c6a70 msgr2=0x7fe15406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.326+0000 7fe153fff700 1 --2- 172.21.15.55:0/706596763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1540c6a70 0x7fe15406e770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe14800c790 tx=0x7fe148007e20 comp rx=0 tx=0).stop 2023-10-06T03:33:56.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.326+0000 7fe153fff700 1 -- 172.21.15.55:0/706596763 shutdown_connections 2023-10-06T03:33:56.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.326+0000 7fe153fff700 1 --2- 172.21.15.55:0/706596763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1540ca660 0x7fe1540b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.326+0000 7fe153fff700 1 --2- 172.21.15.55:0/706596763 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe140066f30 0x7fe1400693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.326+0000 7fe153fff700 1 --2- 172.21.15.55:0/706596763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1540696c0 0x7fe15415a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.326+0000 7fe153fff700 1 --2- 172.21.15.55:0/706596763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1540c6a70 0x7fe15406e770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.326+0000 7fe153fff700 1 -- 172.21.15.55:0/706596763 >> 172.21.15.55:0/706596763 conn(0x7fe1540cc1f0 msgr2=0x7fe15415d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:56.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.326+0000 7fe153fff700 1 -- 172.21.15.55:0/706596763 shutdown_connections 2023-10-06T03:33:56.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.326+0000 7fe153fff700 1 -- 172.21.15.55:0/706596763 wait complete. 2023-10-06T03:33:56.329 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 8 2023-10-06T03:33:56.346 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:56.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.531+0000 7fa4c8b9d700 1 Processor -- start 2023-10-06T03:33:56.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.531+0000 7fa4c8b9d700 1 -- start start 2023-10-06T03:33:56.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.532+0000 7fa4c8b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4c41597f0 0x7fa4c40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.532+0000 7fa4c8b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4c415d230 0x7fa4c415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.532+0000 7fa4c8b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c415dbb0 con 0x7fa4c4159420 2023-10-06T03:33:56.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.532+0000 7fa4c8b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c40b11d0 con 0x7fa4c41597f0 2023-10-06T03:33:56.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.532+0000 7fa4c8b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40b14b0 con 0x7fa4c415d230 2023-10-06T03:33:56.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.533+0000 7fa4c2ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4c41597f0 0x7fa4c40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.533+0000 7fa4c2ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4c41597f0 0x7fa4c40b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51474/0 (socket says 172.21.15.55:51474) 2023-10-06T03:33:56.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.533+0000 7fa4c2ffd700 1 -- 172.21.15.55:0/929715375 learned_addr learned my addr 172.21.15.55:0/929715375 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:56.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.533+0000 7fa4c37fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa4c4159420 0x7fa4c40b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45326/0 (socket says 172.21.15.55:45326) 2023-10-06T03:33:56.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.533+0000 7fa4c2ffd700 1 -- 172.21.15.55:0/929715375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4c415d230 msgr2=0x7fa4c415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.533+0000 7fa4c3fff700 1 --2- 172.21.15.55:0/929715375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4c415d230 0x7fa4c415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.533+0000 7fa4c2ffd700 1 --2- 172.21.15.55:0/929715375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4c415d230 0x7fa4c415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.533+0000 7fa4c2ffd700 1 -- 172.21.15.55:0/929715375 >> v1:172.21.15.55:6789/0 conn(0x7fa4c4159420 legacy=0x7fa4c40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.533+0000 7fa4c27fc700 1 -- 172.21.15.55:0/929715375 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241119773 0 0) 0x7fa4c415dbb0 con 0x7fa4c4159420 2023-10-06T03:33:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.533+0000 7fa4c2ffd700 1 -- 172.21.15.55:0/929715375 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c40b1790 con 0x7fa4c41597f0 2023-10-06T03:33:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.533+0000 7fa4c3fff700 1 --2- 172.21.15.55:0/929715375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4c415d230 0x7fa4c415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.533+0000 7fa4c2ffd700 1 --2- 172.21.15.55:0/929715375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c41597f0 0x7fa4c40b2270 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80099b0 tx=0x7fa4b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804c8966c0717a46 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.534+0000 7fa4c27fc700 1 -- 172.21.15.55:0/929715375 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b800b860 con 0x7fa4c41597f0 2023-10-06T03:33:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.534+0000 7fa4c27fc700 1 -- 172.21.15.55:0/929715375 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b800b9c0 con 0x7fa4c41597f0 2023-10-06T03:33:56.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.534+0000 7fa4c27fc700 1 -- 172.21.15.55:0/929715375 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b800bd60 con 0x7fa4c41597f0 2023-10-06T03:33:56.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.534+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/929715375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c41597f0 msgr2=0x7fa4c40b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.534+0000 7fa4c8b9d700 1 --2- 172.21.15.55:0/929715375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c41597f0 0x7fa4c40b2270 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa4b80099b0 tx=0x7fa4b8009250 comp rx=0 tx=0).stop 2023-10-06T03:33:56.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.534+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/929715375 shutdown_connections 2023-10-06T03:33:56.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.534+0000 7fa4c8b9d700 1 --2- 172.21.15.55:0/929715375 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4c415d230 0x7fa4c415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.534+0000 7fa4c8b9d700 1 --2- 172.21.15.55:0/929715375 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c41597f0 0x7fa4c40b2270 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.534+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/929715375 >> 172.21.15.55:0/929715375 conn(0x7fa4c40d0660 msgr2=0x7fa4c40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:56.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.534+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/929715375 shutdown_connections 2023-10-06T03:33:56.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.534+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/929715375 wait complete. 2023-10-06T03:33:56.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.536+0000 7fa4c8b9d700 1 Processor -- start 2023-10-06T03:33:56.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.536+0000 7fa4c8b9d700 1 -- start start 2023-10-06T03:33:56.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c8b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4c4159420 0x7fa4c415c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c8b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c41597f0 0x7fa4c40ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c8b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4c415d230 0x7fa4c40c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c8b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40ca9d0 con 0x7fa4c41597f0 2023-10-06T03:33:56.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c8b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40c5af0 con 0x7fa4c4159420 2023-10-06T03:33:56.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c8b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c40c88d0 con 0x7fa4c415d230 2023-10-06T03:33:56.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c41597f0 0x7fa4c40ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c41597f0 0x7fa4c40ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51478/0 (socket says 172.21.15.55:51478) 2023-10-06T03:33:56.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c2ffd700 1 -- 172.21.15.55:0/987042793 learned_addr learned my addr 172.21.15.55:0/987042793 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:56.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c3fff700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4c415d230 0x7fa4c40c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c37fe700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4c4159420 0x7fa4c415c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c2ffd700 1 -- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4c415d230 msgr2=0x7fa4c40c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c2ffd700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4c415d230 0x7fa4c40c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c2ffd700 1 -- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4c4159420 msgr2=0x7fa4c415c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c2ffd700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4c4159420 0x7fa4c415c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c2ffd700 1 -- 172.21.15.55:0/987042793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b80094f0 con 0x7fa4c41597f0 2023-10-06T03:33:56.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c37fe700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4c4159420 0x7fa4c415c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:56.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.537+0000 7fa4c3fff700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4c415d230 0x7fa4c40c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:56.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.538+0000 7fa4c2ffd700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c41597f0 0x7fa4c40ca490 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4089340 tx=0x7fa4b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:56.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.538+0000 7fa4ab7fe700 1 -- 172.21.15.55:0/987042793 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b801a420 con 0x7fa4c41597f0 2023-10-06T03:33:56.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.538+0000 7fa4ab7fe700 1 -- 172.21.15.55:0/987042793 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b801a5a0 con 0x7fa4c41597f0 2023-10-06T03:33:56.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.538+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/987042793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c40c8b70 con 0x7fa4c41597f0 2023-10-06T03:33:56.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.538+0000 7fa4ab7fe700 1 -- 172.21.15.55:0/987042793 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b801a960 con 0x7fa4c41597f0 2023-10-06T03:33:56.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.538+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/987042793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c40c5d90 con 0x7fa4c41597f0 2023-10-06T03:33:56.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.539+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/987042793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c404c3c0 con 0x7fa4c41597f0 2023-10-06T03:33:56.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.540+0000 7fa4ab7fe700 1 -- 172.21.15.55:0/987042793 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa4b801ad50 con 0x7fa4c41597f0 2023-10-06T03:33:56.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.541+0000 7fa4ab7fe700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa4b0066f30 0x7fa4b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.541+0000 7fa4ab7fe700 1 -- 172.21.15.55:0/987042793 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa4b80862c0 con 0x7fa4c41597f0 2023-10-06T03:33:56.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.541+0000 7fa4c37fe700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa4b0066f30 0x7fa4b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.542+0000 7fa4c37fe700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa4b0066f30 0x7fa4b00693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40018d0 tx=0x7fa4b4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:56.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.545+0000 7fa4ab7fe700 1 -- 172.21.15.55:0/987042793 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4b8053330 con 0x7fa4c41597f0 2023-10-06T03:33:56.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.751+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/987042793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4c40b1a70 con 0x7fa4c41597f0 2023-10-06T03:33:56.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.752+0000 7fa4ab7fe700 1 -- 172.21.15.55:0/987042793 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7fa4b8056950 con 0x7fa4c41597f0 2023-10-06T03:33:56.754 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:56.754 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:16.470817+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:56.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.754+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa4b0066f30 msgr2=0x7fa4b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.754+0000 7fa4c8b9d700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa4b0066f30 0x7fa4b00693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa4b40018d0 tx=0x7fa4b4009040 comp rx=0 tx=0).stop 2023-10-06T03:33:56.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.754+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/987042793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c41597f0 msgr2=0x7fa4c40ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.754+0000 7fa4c8b9d700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c41597f0 0x7fa4c40ca490 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4089340 tx=0x7fa4b8017000 comp rx=0 tx=0).stop 2023-10-06T03:33:56.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.755+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/987042793 shutdown_connections 2023-10-06T03:33:56.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.755+0000 7fa4c8b9d700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa4b0066f30 0x7fa4b00693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.755+0000 7fa4c8b9d700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4c415d230 0x7fa4c40c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.755+0000 7fa4c8b9d700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4c4159420 0x7fa4c415c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.755+0000 7fa4c8b9d700 1 --2- 172.21.15.55:0/987042793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4c41597f0 0x7fa4c40ca490 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.755+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/987042793 >> 172.21.15.55:0/987042793 conn(0x7fa4c40d0660 msgr2=0x7fa4c4156230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:56.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.755+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/987042793 shutdown_connections 2023-10-06T03:33:56.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.755+0000 7fa4c8b9d700 1 -- 172.21.15.55:0/987042793 wait complete. 2023-10-06T03:33:56.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:33:13.543361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:33:14.341572+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:33:56.775 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:33:56.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.962+0000 7f8f149fe700 1 Processor -- start 2023-10-06T03:33:56.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.962+0000 7f8f149fe700 1 -- start start 2023-10-06T03:33:56.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.963+0000 7f8f149fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8f10159a60 0x7f8f100c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.963+0000 7f8f149fe700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f100c9cb0 con 0x7f8f100ccc00 2023-10-06T03:33:56.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.963+0000 7f8f149fe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f100c9f30 con 0x7f8f100cb230 2023-10-06T03:33:56.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.963+0000 7f8f149fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f1006dfc0 con 0x7f8f10159a60 2023-10-06T03:33:56.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.963+0000 7f8f0e59c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8f100ccc00 0x7f8f10069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45340/0 (socket says 172.21.15.55:45340) 2023-10-06T03:33:56.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.963+0000 7f8f0e59c700 1 -- 172.21.15.55:0/2393567628 learned_addr learned my addr 172.21.15.55:0/2393567628 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:56.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0ed9d700 1 --2- 172.21.15.55:0/2393567628 >> v2:172.21.15.163:3300/0 conn(0x7f8f10159a60 0x7f8f100c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 -- 172.21.15.55:0/2393567628 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295716746 0 0) 0x7f8f100c9cb0 con 0x7f8f100ccc00 2023-10-06T03:33:56.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 -- 172.21.15.55:0/2393567628 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ef8003570 con 0x7f8f100ccc00 2023-10-06T03:33:56.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 -- 172.21.15.55:0/2393567628 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3291299922 0 0) 0x7f8ef8003570 con 0x7f8f100ccc00 2023-10-06T03:33:56.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 -- 172.21.15.55:0/2393567628 >> v2:172.21.15.163:3300/0 conn(0x7f8f10159a60 msgr2=0x7f8f100c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:56.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 --2- 172.21.15.55:0/2393567628 >> v2:172.21.15.163:3300/0 conn(0x7f8f10159a60 0x7f8f100c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 -- 172.21.15.55:0/2393567628 >> v1:172.21.15.163:6789/0 conn(0x7f8f100cb230 legacy=0x7f8f1006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 -- 172.21.15.55:0/2393567628 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f100ca470 con 0x7f8f100ccc00 2023-10-06T03:33:56.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 -- 172.21.15.55:0/2393567628 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f8f04001470 con 0x7f8f100ccc00 2023-10-06T03:33:56.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 -- 172.21.15.55:0/2393567628 >> v1:172.21.15.55:6789/0 conn(0x7f8f100ccc00 legacy=0x7f8f10069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 --2- 172.21.15.55:0/2393567628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ef8003a60 0x7f8ef8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 --2- 172.21.15.55:0/2393567628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ef8004810 0x7f8ef8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 --2- 172.21.15.55:0/2393567628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef80051c0 0x7f8ef800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 -- 172.21.15.55:0/2393567628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f1006dfc0 con 0x7f8ef8003a60 2023-10-06T03:33:56.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 -- 172.21.15.55:0/2393567628 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef800c0d0 con 0x7f8ef80051c0 2023-10-06T03:33:56.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.964+0000 7f8f0dd9b700 1 -- 172.21.15.55:0/2393567628 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef800c290 con 0x7f8ef8004810 2023-10-06T03:33:56.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.965+0000 7f8f0ed9d700 1 --2- 172.21.15.55:0/2393567628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ef8003a60 0x7f8ef8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.965+0000 7f8f0f59e700 1 --2- 172.21.15.55:0/2393567628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef80051c0 0x7f8ef800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.965+0000 7f8f0e59c700 1 --2- 172.21.15.55:0/2393567628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ef8004810 0x7f8ef8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.965+0000 7f8f0ed9d700 1 -- 172.21.15.55:0/2393567628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ef8004810 msgr2=0x7f8ef8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:56.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.965+0000 7f8f0ed9d700 1 --2- 172.21.15.55:0/2393567628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ef8004810 0x7f8ef8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.965+0000 7f8f0ed9d700 1 -- 172.21.15.55:0/2393567628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef80051c0 msgr2=0x7f8ef800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:56.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.965+0000 7f8f0ed9d700 1 --2- 172.21.15.55:0/2393567628 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef80051c0 0x7f8ef800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.965+0000 7f8f0ed9d700 1 -- 172.21.15.55:0/2393567628 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef8003570 con 0x7f8ef8003a60 2023-10-06T03:33:56.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.965+0000 7f8f0f59e700 1 -- 172.21.15.55:0/2393567628 reap_dead start 2023-10-06T03:33:56.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.965+0000 7f8f0e59c700 1 --2- 172.21.15.55:0/2393567628 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ef8004810 0x7f8ef8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:56.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.965+0000 7f8f0ed9d700 1 --2- 172.21.15.55:0/2393567628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ef8003a60 0x7f8ef8003e70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009c00 tx=0x7f8f000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306401eb29e483dd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:56.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.965+0000 7f8f0dd9b700 1 -- 172.21.15.55:0/2393567628 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0000e710 con 0x7f8ef8003a60 2023-10-06T03:33:56.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.965+0000 7f8f0dd9b700 1 -- 172.21.15.55:0/2393567628 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0000e870 con 0x7f8ef8003a60 2023-10-06T03:33:56.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.965+0000 7f8f0dd9b700 1 -- 172.21.15.55:0/2393567628 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0000ec30 con 0x7f8ef8003a60 2023-10-06T03:33:56.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.966+0000 7f8f149fe700 1 -- 172.21.15.55:0/2393567628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ef8003a60 msgr2=0x7f8ef8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.966+0000 7f8f149fe700 1 --2- 172.21.15.55:0/2393567628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ef8003a60 0x7f8ef8003e70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009c00 tx=0x7f8f000091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.966+0000 7f8f149fe700 1 -- 172.21.15.55:0/2393567628 shutdown_connections 2023-10-06T03:33:56.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.966+0000 7f8f149fe700 1 --2- 172.21.15.55:0/2393567628 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ef8003a60 0x7f8ef8003e70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.966+0000 7f8f149fe700 1 -- 172.21.15.55:0/2393567628 >> 172.21.15.55:0/2393567628 conn(0x7f8f1006baa0 msgr2=0x7f8f1006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:56.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.966+0000 7f8f149fe700 1 -- 172.21.15.55:0/2393567628 shutdown_connections 2023-10-06T03:33:56.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.966+0000 7f8f149fe700 1 -- 172.21.15.55:0/2393567628 wait complete. 2023-10-06T03:33:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.967+0000 7f8f149fe700 1 Processor -- start 2023-10-06T03:33:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.968+0000 7f8f149fe700 1 -- start start 2023-10-06T03:33:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.968+0000 7f8f149fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef8003a60 0x7f8f100a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.968+0000 7f8f149fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f100a9e10 0x7f8f100b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.968+0000 7f8f149fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f100b37b0 0x7f8f100b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.968+0000 7f8f149fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100b21d0 con 0x7f8f100b37b0 2023-10-06T03:33:56.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.968+0000 7f8f149fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100b24b0 con 0x7f8ef8003a60 2023-10-06T03:33:56.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.968+0000 7f8f149fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100b2790 con 0x7f8f100a9e10 2023-10-06T03:33:56.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.969+0000 7f8f0f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f100b37b0 0x7f8f100b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.969+0000 7f8f0f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f100b37b0 0x7f8f100b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51498/0 (socket says 172.21.15.55:51498) 2023-10-06T03:33:56.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.969+0000 7f8f0f59e700 1 -- 172.21.15.55:0/2727341634 learned_addr learned my addr 172.21.15.55:0/2727341634 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:56.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.969+0000 7f8f0ed9d700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef8003a60 0x7f8f100a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.969+0000 7f8f0e59c700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f100a9e10 0x7f8f100b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.969+0000 7f8f0f59e700 1 -- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f100a9e10 msgr2=0x7f8f100b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.969+0000 7f8f0f59e700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f100a9e10 0x7f8f100b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.969+0000 7f8f0f59e700 1 -- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef8003a60 msgr2=0x7f8f100a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.969+0000 7f8f0f59e700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef8003a60 0x7f8f100a98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.969+0000 7f8f0f59e700 1 -- 172.21.15.55:0/2727341634 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f100b2a70 con 0x7f8f100b37b0 2023-10-06T03:33:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.969+0000 7f8f0e59c700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f100a9e10 0x7f8f100b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.969+0000 7f8f0f59e700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f100b37b0 0x7f8f100b3c30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f1015b1e0 tx=0x7f8f08009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.970+0000 7f8ef6ffd700 1 -- 172.21.15.55:0/2727341634 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f08006db0 con 0x7f8f100b37b0 2023-10-06T03:33:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.970+0000 7f8ef6ffd700 1 -- 172.21.15.55:0/2727341634 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f080030d0 con 0x7f8f100b37b0 2023-10-06T03:33:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.970+0000 7f8ef6ffd700 1 -- 172.21.15.55:0/2727341634 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f08001580 con 0x7f8f100b37b0 2023-10-06T03:33:56.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.970+0000 7f8f149fe700 1 -- 172.21.15.55:0/2727341634 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f100b2d50 con 0x7f8f100b37b0 2023-10-06T03:33:56.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.970+0000 7f8f149fe700 1 -- 172.21.15.55:0/2727341634 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f10147ba0 con 0x7f8f100b37b0 2023-10-06T03:33:56.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.971+0000 7f8f149fe700 1 -- 172.21.15.55:0/2727341634 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f100b2f90 con 0x7f8f100b37b0 2023-10-06T03:33:56.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.972+0000 7f8ef6ffd700 1 -- 172.21.15.55:0/2727341634 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f08001930 con 0x7f8f100b37b0 2023-10-06T03:33:56.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.973+0000 7f8ef6ffd700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8efc066f30 0x7f8efc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:56.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.973+0000 7f8ef6ffd700 1 -- 172.21.15.55:0/2727341634 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8f08083a90 con 0x7f8f100b37b0 2023-10-06T03:33:56.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.973+0000 7f8f0ed9d700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8efc066f30 0x7f8efc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:56.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.974+0000 7f8f0ed9d700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8efc066f30 0x7f8efc0693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f000091e0 tx=0x7f8f0001a000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:56.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:56.977+0000 7f8ef6ffd700 1 -- 172.21.15.55:0/2727341634 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f100b2f90 con 0x7f8f100b37b0 2023-10-06T03:33:57.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.176+0000 7f8f149fe700 1 -- 172.21.15.55:0/2727341634 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8f101580e0 con 0x7f8f100b37b0 2023-10-06T03:33:57.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.176+0000 7f8ef6ffd700 1 -- 172.21.15.55:0/2727341634 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8f0801d0a0 con 0x7f8f100b37b0 2023-10-06T03:33:57.178 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:33:57.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.179+0000 7f8f149fe700 1 -- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8efc066f30 msgr2=0x7f8efc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:57.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.179+0000 7f8f149fe700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8efc066f30 0x7f8efc0693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f000091e0 tx=0x7f8f0001a000 comp rx=0 tx=0).stop 2023-10-06T03:33:57.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.179+0000 7f8f149fe700 1 -- 172.21.15.55:0/2727341634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f100b37b0 msgr2=0x7f8f100b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:57.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.179+0000 7f8f149fe700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f100b37b0 0x7f8f100b3c30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8f1015b1e0 tx=0x7f8f08009040 comp rx=0 tx=0).stop 2023-10-06T03:33:57.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.179+0000 7f8f149fe700 1 -- 172.21.15.55:0/2727341634 shutdown_connections 2023-10-06T03:33:57.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.179+0000 7f8f149fe700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f100b37b0 0x7f8f100b3c30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.179+0000 7f8f149fe700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ef8003a60 0x7f8f100a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.179+0000 7f8f149fe700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8efc066f30 0x7f8efc0693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.179+0000 7f8f149fe700 1 --2- 172.21.15.55:0/2727341634 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f100a9e10 0x7f8f100b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.179+0000 7f8f149fe700 1 -- 172.21.15.55:0/2727341634 >> 172.21.15.55:0/2727341634 conn(0x7f8f1006baa0 msgr2=0x7f8f1006a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:57.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.179+0000 7f8f149fe700 1 -- 172.21.15.55:0/2727341634 shutdown_connections 2023-10-06T03:33:57.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.179+0000 7f8f149fe700 1 -- 172.21.15.55:0/2727341634 wait complete. 2023-10-06T03:33:57.198 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:33:57.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.386+0000 7f2bacd9d700 1 Processor -- start 2023-10-06T03:33:57.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.386+0000 7f2bacd9d700 1 -- start start 2023-10-06T03:33:57.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.387+0000 7f2bacd9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba8159a60 0x7f2ba80c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.387+0000 7f2bacd9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2ba80cb230 0x7f2ba80cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.387+0000 7f2bacd9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba80c9cb0 con 0x7f2ba80ccc00 2023-10-06T03:33:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.387+0000 7f2bacd9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba80cbbd0 con 0x7f2ba80cb230 2023-10-06T03:33:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.387+0000 7f2bacd9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba806dfc0 con 0x7f2ba8159a60 2023-10-06T03:33:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.387+0000 7f2ba7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2ba80cb230 0x7f2ba80cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.388+0000 7f2ba7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2ba80cb230 0x7f2ba80cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51514/0 (socket says 172.21.15.55:51514) 2023-10-06T03:33:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.388+0000 7f2ba7fff700 1 -- 172.21.15.55:0/68992663 learned_addr learned my addr 172.21.15.55:0/68992663 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.388+0000 7f2ba77fe700 1 --2- 172.21.15.55:0/68992663 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba8159a60 0x7f2ba80c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.388+0000 7f2ba7fff700 1 -- 172.21.15.55:0/68992663 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba8159a60 msgr2=0x7f2ba80c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.388+0000 7f2ba7fff700 1 --2- 172.21.15.55:0/68992663 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba8159a60 0x7f2ba80c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.388+0000 7f2ba7fff700 1 -- 172.21.15.55:0/68992663 >> v1:172.21.15.163:6789/0 conn(0x7f2ba80ccc00 legacy=0x7f2ba8069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.388+0000 7f2ba7fff700 1 -- 172.21.15.55:0/68992663 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba80c9f30 con 0x7f2ba80cb230 2023-10-06T03:33:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.388+0000 7f2ba67fc700 1 -- 172.21.15.55:0/68992663 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745516796 0 0) 0x7f2ba80c9cb0 con 0x7f2ba80ccc00 2023-10-06T03:33:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.388+0000 7f2ba7fff700 1 --2- 172.21.15.55:0/68992663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba80cb230 0x7f2ba80cb690 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0009f50 tx=0x7f2ba0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74dbc8553f0d6954 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.388+0000 7f2ba67fc700 1 -- 172.21.15.55:0/68992663 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba000e070 con 0x7f2ba80cb230 2023-10-06T03:33:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.389+0000 7f2ba67fc700 1 -- 172.21.15.55:0/68992663 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba000ba50 con 0x7f2ba80cb230 2023-10-06T03:33:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.389+0000 7f2ba67fc700 1 -- 172.21.15.55:0/68992663 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba00092e0 con 0x7f2ba80cb230 2023-10-06T03:33:57.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.389+0000 7f2bacd9d700 1 -- 172.21.15.55:0/68992663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba80cb230 msgr2=0x7f2ba80cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:57.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.389+0000 7f2bacd9d700 1 --2- 172.21.15.55:0/68992663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba80cb230 0x7f2ba80cb690 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0009f50 tx=0x7f2ba0009840 comp rx=0 tx=0).stop 2023-10-06T03:33:57.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.389+0000 7f2bacd9d700 1 -- 172.21.15.55:0/68992663 shutdown_connections 2023-10-06T03:33:57.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.389+0000 7f2bacd9d700 1 --2- 172.21.15.55:0/68992663 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba80cb230 0x7f2ba80cb690 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.389+0000 7f2bacd9d700 1 --2- 172.21.15.55:0/68992663 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba8159a60 0x7f2ba80c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.389+0000 7f2bacd9d700 1 -- 172.21.15.55:0/68992663 >> 172.21.15.55:0/68992663 conn(0x7f2ba806baa0 msgr2=0x7f2ba806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.389+0000 7f2bacd9d700 1 -- 172.21.15.55:0/68992663 shutdown_connections 2023-10-06T03:33:57.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.389+0000 7f2bacd9d700 1 -- 172.21.15.55:0/68992663 wait complete. 2023-10-06T03:33:57.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.391+0000 7f2bacd9d700 1 Processor -- start 2023-10-06T03:33:57.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.391+0000 7f2bacd9d700 1 -- start start 2023-10-06T03:33:57.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.392+0000 7f2bacd9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba80cb230 0x7f2ba816f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.392+0000 7f2bacd9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba80ccc00 0x7f2ba816ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.392+0000 7f2bacd9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba8159a60 0x7f2ba8154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.392+0000 7f2bacd9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba816f290 con 0x7f2ba8159a60 2023-10-06T03:33:57.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.392+0000 7f2bacd9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba8170670 con 0x7f2ba80ccc00 2023-10-06T03:33:57.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.392+0000 7f2bacd9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba816fa60 con 0x7f2ba80cb230 2023-10-06T03:33:57.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.392+0000 7f2ba7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba8159a60 0x7f2ba8154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.392+0000 7f2ba7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba8159a60 0x7f2ba8154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51522/0 (socket says 172.21.15.55:51522) 2023-10-06T03:33:57.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.392+0000 7f2ba7fff700 1 -- 172.21.15.55:0/1239488347 learned_addr learned my addr 172.21.15.55:0/1239488347 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:57.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.392+0000 7f2ba77fe700 1 --2- 172.21.15.55:0/1239488347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba80cb230 0x7f2ba816f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.392+0000 7f2ba6ffd700 1 --2- 172.21.15.55:0/1239488347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba80ccc00 0x7f2ba816ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.392+0000 7f2ba7fff700 1 -- 172.21.15.55:0/1239488347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba80cb230 msgr2=0x7f2ba816f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.392+0000 7f2ba7fff700 1 --2- 172.21.15.55:0/1239488347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba80cb230 0x7f2ba816f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.393+0000 7f2ba7fff700 1 -- 172.21.15.55:0/1239488347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba80ccc00 msgr2=0x7f2ba816ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.393+0000 7f2ba7fff700 1 --2- 172.21.15.55:0/1239488347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba80ccc00 0x7f2ba816ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.393+0000 7f2ba7fff700 1 -- 172.21.15.55:0/1239488347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba0009ae0 con 0x7f2ba8159a60 2023-10-06T03:33:57.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.393+0000 7f2ba7fff700 1 --2- 172.21.15.55:0/1239488347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba8159a60 0x7f2ba8154ae0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2ba806cdc0 tx=0x7f2ba0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.393+0000 7f2b8f7fe700 1 -- 172.21.15.55:0/1239488347 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba000be90 con 0x7f2ba8159a60 2023-10-06T03:33:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.393+0000 7f2b8f7fe700 1 -- 172.21.15.55:0/1239488347 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba0008d60 con 0x7f2ba8159a60 2023-10-06T03:33:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.393+0000 7f2bacd9d700 1 -- 172.21.15.55:0/1239488347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba816fd40 con 0x7f2ba8159a60 2023-10-06T03:33:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.393+0000 7f2b8f7fe700 1 -- 172.21.15.55:0/1239488347 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba001d670 con 0x7f2ba8159a60 2023-10-06T03:33:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.393+0000 7f2bacd9d700 1 -- 172.21.15.55:0/1239488347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba8170300 con 0x7f2ba8159a60 2023-10-06T03:33:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.394+0000 7f2bacd9d700 1 -- 172.21.15.55:0/1239488347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba815a3d0 con 0x7f2ba8159a60 2023-10-06T03:33:57.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.395+0000 7f2b8f7fe700 1 -- 172.21.15.55:0/1239488347 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2ba0003040 con 0x7f2ba8159a60 2023-10-06T03:33:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.396+0000 7f2b8f7fe700 1 --2- 172.21.15.55:0/1239488347 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2b94066f30 0x7f2b940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.396+0000 7f2b8f7fe700 1 -- 172.21.15.55:0/1239488347 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2ba00862a0 con 0x7f2ba8159a60 2023-10-06T03:33:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.397+0000 7f2ba77fe700 1 --2- 172.21.15.55:0/1239488347 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2b94066f30 0x7f2b940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.397+0000 7f2ba77fe700 1 --2- 172.21.15.55:0/1239488347 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2b94066f30 0x7f2b940693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009d10 tx=0x7f2b980091a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:57.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.400+0000 7f2b8f7fe700 1 -- 172.21.15.55:0/1239488347 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ba0053310 con 0x7f2ba8159a60 2023-10-06T03:33:57.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.605+0000 7f2bacd9d700 1 -- 172.21.15.55:0/1239488347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ba80ca470 con 0x7f2ba8159a60 2023-10-06T03:33:57.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.605+0000 7f2b8f7fe700 1 -- 172.21.15.55:0/1239488347 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ba0056930 con 0x7f2ba8159a60 2023-10-06T03:33:57.607 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:33:57.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.608+0000 7f2bacd9d700 1 -- 172.21.15.55:0/1239488347 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2b94066f30 msgr2=0x7f2b940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:57.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.608+0000 7f2bacd9d700 1 --2- 172.21.15.55:0/1239488347 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2b94066f30 0x7f2b940693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009d10 tx=0x7f2b980091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.608+0000 7f2bacd9d700 1 -- 172.21.15.55:0/1239488347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba8159a60 msgr2=0x7f2ba8154ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.608+0000 7f2bacd9d700 1 --2- 172.21.15.55:0/1239488347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba8159a60 0x7f2ba8154ae0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2ba806cdc0 tx=0x7f2ba0018000 comp rx=0 tx=0).stop 2023-10-06T03:33:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.608+0000 7f2bacd9d700 1 -- 172.21.15.55:0/1239488347 shutdown_connections 2023-10-06T03:33:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.608+0000 7f2bacd9d700 1 --2- 172.21.15.55:0/1239488347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba8159a60 0x7f2ba8154ae0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.608+0000 7f2bacd9d700 1 --2- 172.21.15.55:0/1239488347 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2b94066f30 0x7f2b940693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.608+0000 7f2bacd9d700 1 --2- 172.21.15.55:0/1239488347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba80cb230 0x7f2ba816f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.608+0000 7f2bacd9d700 1 --2- 172.21.15.55:0/1239488347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba80ccc00 0x7f2ba816ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.608+0000 7f2bacd9d700 1 -- 172.21.15.55:0/1239488347 >> 172.21.15.55:0/1239488347 conn(0x7f2ba806baa0 msgr2=0x7f2ba8153f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.608+0000 7f2bacd9d700 1 -- 172.21.15.55:0/1239488347 shutdown_connections 2023-10-06T03:33:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.608+0000 7f2bacd9d700 1 -- 172.21.15.55:0/1239488347 wait complete. 2023-10-06T03:33:57.626 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:33:57.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.814+0000 7f2d0219d700 1 Processor -- start 2023-10-06T03:33:57.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.814+0000 7f2d0219d700 1 -- start start 2023-10-06T03:33:57.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.815+0000 7f2d0219d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2cfc159a10 0x7f2cfc0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.815+0000 7f2d0219d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cfc0aac80 0x7f2cfc15a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.815+0000 7f2d0219d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cfc15aa30 con 0x7f2cfc0c81e0 2023-10-06T03:33:57.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.815+0000 7f2d0219d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cfc0caee0 con 0x7f2cfc159a10 2023-10-06T03:33:57.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.815+0000 7f2d0219d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc0c7db0 con 0x7f2cfc0aac80 2023-10-06T03:33:57.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.816+0000 7f2d0099a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2cfc0c81e0 0x7f2cfc159de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45354/0 (socket says 172.21.15.55:45354) 2023-10-06T03:33:57.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.816+0000 7f2d0099a700 1 -- 172.21.15.55:0/127054399 learned_addr learned my addr 172.21.15.55:0/127054399 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:57.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.816+0000 7f2d0119b700 1 --2- 172.21.15.55:0/127054399 >> v2:172.21.15.163:3300/0 conn(0x7f2cfc159a10 0x7f2cfc0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.816+0000 7f2d0199c700 1 --2- 172.21.15.55:0/127054399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cfc0aac80 0x7f2cfc15a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.816+0000 7f2cf3fff700 1 -- 172.21.15.55:0/127054399 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512464193 0 0) 0x7f2cfc15aa30 con 0x7f2cfc0c81e0 2023-10-06T03:33:57.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.816+0000 7f2cf3fff700 1 -- 172.21.15.55:0/127054399 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ce8003570 con 0x7f2cfc0c81e0 2023-10-06T03:33:57.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2cf3fff700 1 -- 172.21.15.55:0/127054399 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3537291513 0 0) 0x7f2ce8003570 con 0x7f2cfc0c81e0 2023-10-06T03:33:57.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2cf3fff700 1 -- 172.21.15.55:0/127054399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cfc0aac80 msgr2=0x7f2cfc15a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:57.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2cf3fff700 1 --2- 172.21.15.55:0/127054399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cfc0aac80 0x7f2cfc15a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2cf3fff700 1 -- 172.21.15.55:0/127054399 >> v2:172.21.15.163:3300/0 conn(0x7f2cfc159a10 msgr2=0x7f2cfc0ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:57.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2cf3fff700 1 --2- 172.21.15.55:0/127054399 >> v2:172.21.15.163:3300/0 conn(0x7f2cfc159a10 0x7f2cfc0ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2cf3fff700 1 -- 172.21.15.55:0/127054399 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc0c89a0 con 0x7f2cfc0c81e0 2023-10-06T03:33:57.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2cf3fff700 1 -- 172.21.15.55:0/127054399 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f2cec001470 con 0x7f2cfc0c81e0 2023-10-06T03:33:57.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2cf3fff700 1 -- 172.21.15.55:0/127054399 >> v1:172.21.15.55:6789/0 conn(0x7f2cfc0c81e0 legacy=0x7f2cfc159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:57.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2cf3fff700 1 --2- 172.21.15.55:0/127054399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ce8003a60 0x7f2ce8003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2cf3fff700 1 --2- 172.21.15.55:0/127054399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ce8004810 0x7f2ce8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2cf3fff700 1 --2- 172.21.15.55:0/127054399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ce80051c0 0x7f2ce800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2cf3fff700 1 -- 172.21.15.55:0/127054399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc0c7db0 con 0x7f2ce8003a60 2023-10-06T03:33:57.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2cf3fff700 1 -- 172.21.15.55:0/127054399 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce800bfa0 con 0x7f2ce8004810 2023-10-06T03:33:57.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2cf3fff700 1 -- 172.21.15.55:0/127054399 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce800c110 con 0x7f2ce80051c0 2023-10-06T03:33:57.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2d0119b700 1 --2- 172.21.15.55:0/127054399 >> v2:172.21.15.163:3300/0 conn(0x7f2cfc159a10 0x7f2cfc0ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:57.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2d0199c700 1 --2- 172.21.15.55:0/127054399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cfc0aac80 0x7f2cfc15a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:33:57.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2d0119b700 1 --2- 172.21.15.55:0/127054399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ce8003a60 0x7f2ce8003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2d0099a700 1 --2- 172.21.15.55:0/127054399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ce8004810 0x7f2ce8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.817+0000 7f2d0199c700 1 --2- 172.21.15.55:0/127054399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ce80051c0 0x7f2ce800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.818+0000 7f2d0119b700 1 -- 172.21.15.55:0/127054399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ce80051c0 msgr2=0x7f2ce800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:57.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.818+0000 7f2d0119b700 1 --2- 172.21.15.55:0/127054399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ce80051c0 0x7f2ce800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.818+0000 7f2d0119b700 1 -- 172.21.15.55:0/127054399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ce8004810 msgr2=0x7f2ce8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:57.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.818+0000 7f2d0119b700 1 --2- 172.21.15.55:0/127054399 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ce8004810 0x7f2ce8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.818+0000 7f2d0119b700 1 -- 172.21.15.55:0/127054399 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce8003570 con 0x7f2ce8003a60 2023-10-06T03:33:57.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.818+0000 7f2d0119b700 1 --2- 172.21.15.55:0/127054399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ce8003a60 0x7f2ce8003e90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009db0 tx=0x7f2cf80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b140fca180a954 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:57.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.818+0000 7f2cf3fff700 1 -- 172.21.15.55:0/127054399 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8009740 con 0x7f2ce8003a60 2023-10-06T03:33:57.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.818+0000 7f2cf3fff700 1 -- 172.21.15.55:0/127054399 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf800e920 con 0x7f2ce8003a60 2023-10-06T03:33:57.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.818+0000 7f2cf3fff700 1 -- 172.21.15.55:0/127054399 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf800eca0 con 0x7f2ce8003a60 2023-10-06T03:33:57.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.819+0000 7f2d0219d700 1 -- 172.21.15.55:0/127054399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ce8003a60 msgr2=0x7f2ce8003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:57.824 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.819+0000 7f2d0219d700 1 --2- 172.21.15.55:0/127054399 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ce8003a60 0x7f2ce8003e90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009db0 tx=0x7f2cf80091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.824 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.819+0000 7f2d0199c700 1 -- 172.21.15.55:0/127054399 reap_dead start 2023-10-06T03:33:57.824 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.819+0000 7f2d0219d700 1 -- 172.21.15.55:0/127054399 shutdown_connections 2023-10-06T03:33:57.824 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.819+0000 7f2d0219d700 1 -- 172.21.15.55:0/127054399 >> 172.21.15.55:0/127054399 conn(0x7f2cfc0cc1e0 msgr2=0x7f2cfc0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:57.824 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.819+0000 7f2d0219d700 1 -- 172.21.15.55:0/127054399 shutdown_connections 2023-10-06T03:33:57.824 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.819+0000 7f2d0219d700 1 -- 172.21.15.55:0/127054399 wait complete. 2023-10-06T03:33:57.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.820+0000 7f2d0219d700 1 Processor -- start 2023-10-06T03:33:57.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.820+0000 7f2d0219d700 1 -- start start 2023-10-06T03:33:57.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.821+0000 7f2d0219d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ce8003a60 0x7f2cfc153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.821+0000 7f2d0219d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cfc0ad420 0x7f2cfc0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.821+0000 7f2d0219d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc15dfc0 0x7f2cfc15e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.821+0000 7f2d0219d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc0addd0 con 0x7f2cfc15dfc0 2023-10-06T03:33:57.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.821+0000 7f2d0219d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc0af950 con 0x7f2ce8003a60 2023-10-06T03:33:57.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.821+0000 7f2d0219d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc15e950 con 0x7f2cfc0ad420 2023-10-06T03:33:57.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.821+0000 7f2d0199c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc15dfc0 0x7f2cfc15e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.821+0000 7f2d0199c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc15dfc0 0x7f2cfc15e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51540/0 (socket says 172.21.15.55:51540) 2023-10-06T03:33:57.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.821+0000 7f2d0199c700 1 -- 172.21.15.55:0/723886643 learned_addr learned my addr 172.21.15.55:0/723886643 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:57.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.822+0000 7f2d0099a700 1 --2- 172.21.15.55:0/723886643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cfc0ad420 0x7f2cfc0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.822+0000 7f2d0119b700 1 --2- 172.21.15.55:0/723886643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ce8003a60 0x7f2cfc153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.822+0000 7f2d0199c700 1 -- 172.21.15.55:0/723886643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cfc0ad420 msgr2=0x7f2cfc0ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:57.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.822+0000 7f2d0199c700 1 --2- 172.21.15.55:0/723886643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cfc0ad420 0x7f2cfc0ad890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.822+0000 7f2d0199c700 1 -- 172.21.15.55:0/723886643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ce8003a60 msgr2=0x7f2cfc153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:57.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.822+0000 7f2d0199c700 1 --2- 172.21.15.55:0/723886643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ce8003a60 0x7f2cfc153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:57.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.822+0000 7f2d0199c700 1 -- 172.21.15.55:0/723886643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc0adfc0 con 0x7f2cfc15dfc0 2023-10-06T03:33:57.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.822+0000 7f2d0199c700 1 --2- 172.21.15.55:0/723886643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc15dfc0 0x7f2cfc15e410 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0ab3d0 tx=0x7f2cf400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:57.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.822+0000 7f2cf17fa700 1 -- 172.21.15.55:0/723886643 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400e070 con 0x7f2cfc15dfc0 2023-10-06T03:33:57.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.822+0000 7f2cf17fa700 1 -- 172.21.15.55:0/723886643 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf4011db0 con 0x7f2cfc15dfc0 2023-10-06T03:33:57.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.822+0000 7f2d0219d700 1 -- 172.21.15.55:0/723886643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cfc15cc10 con 0x7f2cfc15dfc0 2023-10-06T03:33:57.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.822+0000 7f2cf17fa700 1 -- 172.21.15.55:0/723886643 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf40166a0 con 0x7f2cfc15dfc0 2023-10-06T03:33:57.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.823+0000 7f2d0219d700 1 -- 172.21.15.55:0/723886643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cfc0af300 con 0x7f2cfc15dfc0 2023-10-06T03:33:57.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.823+0000 7f2d0219d700 1 -- 172.21.15.55:0/723886643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cfc0afae0 con 0x7f2cfc15dfc0 2023-10-06T03:33:57.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.824+0000 7f2cf17fa700 1 -- 172.21.15.55:0/723886643 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2cf4016a50 con 0x7f2cfc15dfc0 2023-10-06T03:33:57.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.825+0000 7f2cf17fa700 1 --2- 172.21.15.55:0/723886643 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2ce0066f30 0x7f2ce00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:57.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.825+0000 7f2cf17fa700 1 -- 172.21.15.55:0/723886643 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2cf4085410 con 0x7f2cfc15dfc0 2023-10-06T03:33:57.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.826+0000 7f2d0119b700 1 --2- 172.21.15.55:0/723886643 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2ce0066f30 0x7f2ce00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:57.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.826+0000 7f2d0119b700 1 --2- 172.21.15.55:0/723886643 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2ce0066f30 0x7f2ce00693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800f010 tx=0x7f2cf8019000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:57.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:57.829+0000 7f2cf17fa700 1 -- 172.21.15.55:0/723886643 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cf4052480 con 0x7f2cfc15dfc0 2023-10-06T03:33:58.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.030+0000 7f2d0219d700 1 -- 172.21.15.55:0/723886643 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2cfc0c7510 con 0x7f2cfc15dfc0 2023-10-06T03:33:58.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.031+0000 7f2cf17fa700 1 -- 172.21.15.55:0/723886643 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2cf4055aa0 con 0x7f2cfc15dfc0 2023-10-06T03:33:58.033 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T03:33:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.033+0000 7f2d0219d700 1 -- 172.21.15.55:0/723886643 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2ce0066f30 msgr2=0x7f2ce00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.033+0000 7f2d0219d700 1 --2- 172.21.15.55:0/723886643 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2ce0066f30 0x7f2ce00693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800f010 tx=0x7f2cf8019000 comp rx=0 tx=0).stop 2023-10-06T03:33:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.033+0000 7f2d0219d700 1 -- 172.21.15.55:0/723886643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc15dfc0 msgr2=0x7f2cfc15e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.033+0000 7f2d0219d700 1 --2- 172.21.15.55:0/723886643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc15dfc0 0x7f2cfc15e410 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0ab3d0 tx=0x7f2cf400b040 comp rx=0 tx=0).stop 2023-10-06T03:33:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.033+0000 7f2d0219d700 1 -- 172.21.15.55:0/723886643 shutdown_connections 2023-10-06T03:33:58.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.033+0000 7f2d0219d700 1 --2- 172.21.15.55:0/723886643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2cfc15dfc0 0x7f2cfc15e410 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.033+0000 7f2d0219d700 1 --2- 172.21.15.55:0/723886643 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ce8003a60 0x7f2cfc153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.033+0000 7f2d0219d700 1 --2- 172.21.15.55:0/723886643 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2ce0066f30 0x7f2ce00693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.033+0000 7f2d0219d700 1 --2- 172.21.15.55:0/723886643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2cfc0ad420 0x7f2cfc0ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.033+0000 7f2d0219d700 1 -- 172.21.15.55:0/723886643 >> 172.21.15.55:0/723886643 conn(0x7f2cfc0cc1e0 msgr2=0x7f2cfc069560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:58.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.034+0000 7f2d0219d700 1 -- 172.21.15.55:0/723886643 shutdown_connections 2023-10-06T03:33:58.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.034+0000 7f2d0219d700 1 -- 172.21.15.55:0/723886643 wait complete. 2023-10-06T03:33:58.052 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:33:58.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.241+0000 7f653ad9d700 1 Processor -- start 2023-10-06T03:33:58.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.241+0000 7f653ad9d700 1 -- start start 2023-10-06T03:33:58.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.242+0000 7f653ad9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f65340cb1a0 0x7f653415a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:58.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.242+0000 7f653ad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f65340ae0d0 0x7f65340ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:58.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.242+0000 7f653ad9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f653406ebd0 con 0x7f653415a8a0 2023-10-06T03:33:58.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.242+0000 7f653ad9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653415b2e0 con 0x7f65340ae0d0 2023-10-06T03:33:58.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.242+0000 7f653ad9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65340aea10 con 0x7f65340cb1a0 2023-10-06T03:33:58.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.242+0000 7f653a59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f65340ae0d0 0x7f65340ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:58.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.243+0000 7f653a59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f65340ae0d0 0x7f65340ae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51556/0 (socket says 172.21.15.55:51556) 2023-10-06T03:33:58.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.243+0000 7f653a59c700 1 -- 172.21.15.55:0/946063189 learned_addr learned my addr 172.21.15.55:0/946063189 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:58.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.243+0000 7f6539d9b700 1 --2- 172.21.15.55:0/946063189 >> v2:172.21.15.163:3300/0 conn(0x7f65340cb1a0 0x7f653415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:58.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.243+0000 7f653a59c700 1 -- 172.21.15.55:0/946063189 >> v2:172.21.15.163:3300/0 conn(0x7f65340cb1a0 msgr2=0x7f653415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:58.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.243+0000 7f653a59c700 1 --2- 172.21.15.55:0/946063189 >> v2:172.21.15.163:3300/0 conn(0x7f65340cb1a0 0x7f653415a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.243+0000 7f653a59c700 1 -- 172.21.15.55:0/946063189 >> v1:172.21.15.163:6789/0 conn(0x7f653415a8a0 legacy=0x7f653406f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:58.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.243+0000 7f653a59c700 1 -- 172.21.15.55:0/946063189 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653415ad20 con 0x7f65340ae0d0 2023-10-06T03:33:58.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.243+0000 7f653a59c700 1 --2- 172.21.15.55:0/946063189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65340ae0d0 0x7f65340ae4d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f65340cce10 tx=0x7f652c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e2c74c178aca26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:58.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.243+0000 7f6538d99700 1 -- 172.21.15.55:0/946063189 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c00e070 con 0x7f65340ae0d0 2023-10-06T03:33:58.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.244+0000 7f6538d99700 1 -- 172.21.15.55:0/946063189 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652c00bae0 con 0x7f65340ae0d0 2023-10-06T03:33:58.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.244+0000 7f6538d99700 1 -- 172.21.15.55:0/946063189 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f652c0092e0 con 0x7f65340ae0d0 2023-10-06T03:33:58.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.244+0000 7f653ad9d700 1 -- 172.21.15.55:0/946063189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65340ae0d0 msgr2=0x7f65340ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:58.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.244+0000 7f653ad9d700 1 --2- 172.21.15.55:0/946063189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65340ae0d0 0x7f65340ae4d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f65340cce10 tx=0x7f652c009840 comp rx=0 tx=0).stop 2023-10-06T03:33:58.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.244+0000 7f653ad9d700 1 -- 172.21.15.55:0/946063189 shutdown_connections 2023-10-06T03:33:58.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.244+0000 7f653ad9d700 1 --2- 172.21.15.55:0/946063189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65340ae0d0 0x7f65340ae4d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.244+0000 7f653ad9d700 1 --2- 172.21.15.55:0/946063189 >> v2:172.21.15.163:3300/0 conn(0x7f65340cb1a0 0x7f653415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.244+0000 7f653ad9d700 1 -- 172.21.15.55:0/946063189 >> 172.21.15.55:0/946063189 conn(0x7f65340cc1e0 msgr2=0x7f65340cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:58.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.244+0000 7f653ad9d700 1 -- 172.21.15.55:0/946063189 shutdown_connections 2023-10-06T03:33:58.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.244+0000 7f653ad9d700 1 -- 172.21.15.55:0/946063189 wait complete. 2023-10-06T03:33:58.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.246+0000 7f653ad9d700 1 Processor -- start 2023-10-06T03:33:58.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.246+0000 7f653ad9d700 1 -- start start 2023-10-06T03:33:58.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653ad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65340ae0d0 0x7f6534167070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:58.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653ad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65340cb1a0 0x7f6534157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:58.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653ad9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6534158270 0x7f65340c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:58.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653ad9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65341587f0 con 0x7f65340cb1a0 2023-10-06T03:33:58.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653ad9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6534158ad0 con 0x7f65340ae0d0 2023-10-06T03:33:58.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653ad9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6534158db0 con 0x7f6534158270 2023-10-06T03:33:58.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653959a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65340cb1a0 0x7f6534157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:58.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f6539d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65340ae0d0 0x7f6534167070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:58.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653959a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65340cb1a0 0x7f6534157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51570/0 (socket says 172.21.15.55:51570) 2023-10-06T03:33:58.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653959a700 1 -- 172.21.15.55:0/2641394087 learned_addr learned my addr 172.21.15.55:0/2641394087 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:58.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653a59c700 1 --2- 172.21.15.55:0/2641394087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6534158270 0x7f65340c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:58.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653959a700 1 -- 172.21.15.55:0/2641394087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6534158270 msgr2=0x7f65340c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:58.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653959a700 1 --2- 172.21.15.55:0/2641394087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6534158270 0x7f65340c5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653959a700 1 -- 172.21.15.55:0/2641394087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65340ae0d0 msgr2=0x7f6534167070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:58.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653959a700 1 --2- 172.21.15.55:0/2641394087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65340ae0d0 0x7f6534167070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653959a700 1 -- 172.21.15.55:0/2641394087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652c009ae0 con 0x7f65340cb1a0 2023-10-06T03:33:58.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.247+0000 7f653a59c700 1 --2- 172.21.15.55:0/2641394087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6534158270 0x7f65340c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:33:58.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.248+0000 7f653959a700 1 --2- 172.21.15.55:0/2641394087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65340cb1a0 0x7f6534157d30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f652400c4e0 tx=0x7f652400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:58.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.248+0000 7f6529ffb700 1 -- 172.21.15.55:0/2641394087 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6524011590 con 0x7f65340cb1a0 2023-10-06T03:33:58.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.248+0000 7f6529ffb700 1 -- 172.21.15.55:0/2641394087 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6524011710 con 0x7f65340cb1a0 2023-10-06T03:33:58.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.248+0000 7f653ad9d700 1 -- 172.21.15.55:0/2641394087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6534159090 con 0x7f65340cb1a0 2023-10-06T03:33:58.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.248+0000 7f6529ffb700 1 -- 172.21.15.55:0/2641394087 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6524011ad0 con 0x7f65340cb1a0 2023-10-06T03:33:58.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.248+0000 7f653ad9d700 1 -- 172.21.15.55:0/2641394087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65341675b0 con 0x7f65340cb1a0 2023-10-06T03:33:58.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.249+0000 7f653ad9d700 1 -- 172.21.15.55:0/2641394087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65341595b0 con 0x7f65340cb1a0 2023-10-06T03:33:58.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.250+0000 7f6529ffb700 1 -- 172.21.15.55:0/2641394087 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f652401b3b0 con 0x7f65340cb1a0 2023-10-06T03:33:58.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.251+0000 7f6529ffb700 1 --2- 172.21.15.55:0/2641394087 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6518066f30 0x7f65180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:58.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.251+0000 7f6529ffb700 1 -- 172.21.15.55:0/2641394087 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6524083b60 con 0x7f65340cb1a0 2023-10-06T03:33:58.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.251+0000 7f6539d9b700 1 --2- 172.21.15.55:0/2641394087 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6518066f30 0x7f65180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:58.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.252+0000 7f6539d9b700 1 --2- 172.21.15.55:0/2641394087 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6518066f30 0x7f65180693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6530006fd0 tx=0x7f6530008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:58.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.255+0000 7f6529ffb700 1 -- 172.21.15.55:0/2641394087 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65341595b0 con 0x7f65340cb1a0 2023-10-06T03:33:58.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.458+0000 7f653ad9d700 1 -- 172.21.15.55:0/2641394087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65341595b0 con 0x7f65340cb1a0 2023-10-06T03:33:58.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.459+0000 7f6529ffb700 1 -- 172.21.15.55:0/2641394087 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65341595b0 con 0x7f65340cb1a0 2023-10-06T03:33:58.461 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T03:33:58.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.461+0000 7f653ad9d700 1 -- 172.21.15.55:0/2641394087 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6518066f30 msgr2=0x7f65180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:58.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.461+0000 7f653ad9d700 1 --2- 172.21.15.55:0/2641394087 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6518066f30 0x7f65180693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6530006fd0 tx=0x7f6530008040 comp rx=0 tx=0).stop 2023-10-06T03:33:58.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.461+0000 7f653ad9d700 1 -- 172.21.15.55:0/2641394087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65340cb1a0 msgr2=0x7f6534157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:58.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.461+0000 7f653ad9d700 1 --2- 172.21.15.55:0/2641394087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65340cb1a0 0x7f6534157d30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f652400c4e0 tx=0x7f652400c7f0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.461+0000 7f653ad9d700 1 -- 172.21.15.55:0/2641394087 shutdown_connections 2023-10-06T03:33:58.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.461+0000 7f653ad9d700 1 --2- 172.21.15.55:0/2641394087 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6518066f30 0x7f65180693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.461+0000 7f653ad9d700 1 --2- 172.21.15.55:0/2641394087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6534158270 0x7f65340c5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.461+0000 7f653ad9d700 1 --2- 172.21.15.55:0/2641394087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f65340ae0d0 0x7f6534167070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.461+0000 7f653ad9d700 1 --2- 172.21.15.55:0/2641394087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f65340cb1a0 0x7f6534157d30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.461+0000 7f653ad9d700 1 -- 172.21.15.55:0/2641394087 >> 172.21.15.55:0/2641394087 conn(0x7f65340cc1e0 msgr2=0x7f65340b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:58.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.461+0000 7f653ad9d700 1 -- 172.21.15.55:0/2641394087 shutdown_connections 2023-10-06T03:33:58.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.461+0000 7f653ad9d700 1 -- 172.21.15.55:0/2641394087 wait complete. 2023-10-06T03:33:58.480 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:33:58.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.665+0000 7f0d4ed9d700 1 Processor -- start 2023-10-06T03:33:58.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.665+0000 7f0d4ed9d700 1 -- start start 2023-10-06T03:33:58.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.666+0000 7f0d4ed9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d48156310 0x7f0d480c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:58.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.666+0000 7f0d4ed9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0d480ae710 0x7f0d480c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:58.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.666+0000 7f0d4ed9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d480ca860 con 0x7f0d481566e0 2023-10-06T03:33:58.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.666+0000 7f0d4ed9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d480cab40 con 0x7f0d480ae710 2023-10-06T03:33:58.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.666+0000 7f0d4ed9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d480cae20 con 0x7f0d48156310 2023-10-06T03:33:58.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.667+0000 7f0d4e59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0d480ae710 0x7f0d480c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:58.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.667+0000 7f0d4e59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0d480ae710 0x7f0d480c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51580/0 (socket says 172.21.15.55:51580) 2023-10-06T03:33:58.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.667+0000 7f0d4e59c700 1 -- 172.21.15.55:0/835093063 learned_addr learned my addr 172.21.15.55:0/835093063 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:58.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.667+0000 7f0d4dd9b700 1 --2- 172.21.15.55:0/835093063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d48156310 0x7f0d480c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.667+0000 7f0d4e59c700 1 -- 172.21.15.55:0/835093063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d48156310 msgr2=0x7f0d480c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:33:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.667+0000 7f0d4e59c700 1 --2- 172.21.15.55:0/835093063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d48156310 0x7f0d480c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.667+0000 7f0d4e59c700 1 -- 172.21.15.55:0/835093063 >> v1:172.21.15.163:6789/0 conn(0x7f0d481566e0 legacy=0x7f0d480ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:58.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.667+0000 7f0d4e59c700 1 -- 172.21.15.55:0/835093063 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d480cb100 con 0x7f0d480ae710 2023-10-06T03:33:58.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.668+0000 7f0d4e59c700 1 --2- 172.21.15.55:0/835093063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d480ae710 0x7f0d480c6830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009f20 tx=0x7f0d40009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57e1739dc58e0a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:58.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.668+0000 7f0d4cd99700 1 -- 172.21.15.55:0/835093063 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4000e070 con 0x7f0d480ae710 2023-10-06T03:33:58.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.668+0000 7f0d4cd99700 1 -- 172.21.15.55:0/835093063 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4000b9d0 con 0x7f0d480ae710 2023-10-06T03:33:58.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.668+0000 7f0d4cd99700 1 -- 172.21.15.55:0/835093063 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d400092e0 con 0x7f0d480ae710 2023-10-06T03:33:58.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.668+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/835093063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d480ae710 msgr2=0x7f0d480c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:58.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.668+0000 7f0d4ed9d700 1 --2- 172.21.15.55:0/835093063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d480ae710 0x7f0d480c6830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009f20 tx=0x7f0d40009840 comp rx=0 tx=0).stop 2023-10-06T03:33:58.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.668+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/835093063 shutdown_connections 2023-10-06T03:33:58.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.668+0000 7f0d4ed9d700 1 --2- 172.21.15.55:0/835093063 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d480ae710 0x7f0d480c6830 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.668+0000 7f0d4ed9d700 1 --2- 172.21.15.55:0/835093063 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d48156310 0x7f0d480c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.668+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/835093063 >> 172.21.15.55:0/835093063 conn(0x7f0d480cc1f0 msgr2=0x7f0d480cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:58.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.669+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/835093063 shutdown_connections 2023-10-06T03:33:58.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.669+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/835093063 wait complete. 2023-10-06T03:33:58.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.670+0000 7f0d4ed9d700 1 Processor -- start 2023-10-06T03:33:58.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.670+0000 7f0d4ed9d700 1 -- start start 2023-10-06T03:33:58.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.671+0000 7f0d4ed9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d480ae710 0x7f0d480aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:58.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.671+0000 7f0d4ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d48156310 0x7f0d4814f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:58.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.671+0000 7f0d4ed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d481566e0 0x7f0d480c2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:58.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.671+0000 7f0d4ed9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4815b4a0 con 0x7f0d48156310 2023-10-06T03:33:58.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.671+0000 7f0d4ed9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d480b0840 con 0x7f0d481566e0 2023-10-06T03:33:58.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.671+0000 7f0d4ed9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d480b0b20 con 0x7f0d480ae710 2023-10-06T03:33:58.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.671+0000 7f0d4d59a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d48156310 0x7f0d4814f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:58.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.672+0000 7f0d4dd9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d480ae710 0x7f0d480aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:58.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.672+0000 7f0d4d59a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d48156310 0x7f0d4814f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51590/0 (socket says 172.21.15.55:51590) 2023-10-06T03:33:58.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.672+0000 7f0d4d59a700 1 -- 172.21.15.55:0/289317398 learned_addr learned my addr 172.21.15.55:0/289317398 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:58.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.672+0000 7f0d4e59c700 1 --2- 172.21.15.55:0/289317398 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d481566e0 0x7f0d480c2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:58.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.672+0000 7f0d4d59a700 1 -- 172.21.15.55:0/289317398 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d480ae710 msgr2=0x7f0d480aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:58.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.672+0000 7f0d4d59a700 1 --2- 172.21.15.55:0/289317398 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d480ae710 0x7f0d480aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.672+0000 7f0d4d59a700 1 -- 172.21.15.55:0/289317398 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d481566e0 msgr2=0x7f0d480c2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:58.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.672+0000 7f0d4d59a700 1 --2- 172.21.15.55:0/289317398 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d481566e0 0x7f0d480c2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.672+0000 7f0d4d59a700 1 -- 172.21.15.55:0/289317398 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d40009ae0 con 0x7f0d48156310 2023-10-06T03:33:58.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.672+0000 7f0d4d59a700 1 --2- 172.21.15.55:0/289317398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d48156310 0x7f0d4814f470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c7c0 tx=0x7f0d38007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:58.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.672+0000 7f0d3dffb700 1 -- 172.21.15.55:0/289317398 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38011590 con 0x7f0d48156310 2023-10-06T03:33:58.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.672+0000 7f0d3dffb700 1 -- 172.21.15.55:0/289317398 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d380116f0 con 0x7f0d48156310 2023-10-06T03:33:58.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.672+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/289317398 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d480b0e00 con 0x7f0d48156310 2023-10-06T03:33:58.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.672+0000 7f0d3dffb700 1 -- 172.21.15.55:0/289317398 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38011ab0 con 0x7f0d48156310 2023-10-06T03:33:58.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.673+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/289317398 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4814f9b0 con 0x7f0d48156310 2023-10-06T03:33:58.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.673+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/289317398 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4815b270 con 0x7f0d48156310 2023-10-06T03:33:58.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.674+0000 7f0d3dffb700 1 -- 172.21.15.55:0/289317398 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d3801b3b0 con 0x7f0d48156310 2023-10-06T03:33:58.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.675+0000 7f0d3dffb700 1 --2- 172.21.15.55:0/289317398 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0d2c066f30 0x7f0d2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:58.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.675+0000 7f0d3dffb700 1 -- 172.21.15.55:0/289317398 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0d38083b60 con 0x7f0d48156310 2023-10-06T03:33:58.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.676+0000 7f0d4dd9b700 1 --2- 172.21.15.55:0/289317398 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0d2c066f30 0x7f0d2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:58.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.676+0000 7f0d4dd9b700 1 --2- 172.21.15.55:0/289317398 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0d2c066f30 0x7f0d2c0693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0d480c3100 tx=0x7f0d440091a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:58.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.679+0000 7f0d3dffb700 1 -- 172.21.15.55:0/289317398 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d38050bd0 con 0x7f0d48156310 2023-10-06T03:33:58.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.924+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/289317398 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d480b0ff0 con 0x7f0d48156310 2023-10-06T03:33:58.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.924+0000 7f0d3dffb700 1 -- 172.21.15.55:0/289317398 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f0d38017070 con 0x7f0d48156310 2023-10-06T03:33:58.926 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:58.926 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2677235076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":2677235076},{"type":"v1","addr":"172.21.15.55:6835","nonce":2677235076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1030747394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1030747394},{"type":"v1","addr":"172.21.15.55:6834","nonce":1030747394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2040975046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2040975046},{"type":"v1","addr":"172.21.15.163:6836","nonce":2040975046}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:33:15.383124+0000","modified":"2023-10-06T03:33:18.908835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/1910887369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1910887369},{"type":"v1","addr":"172.21.15.163:6837","nonce":1910887369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:33:58.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.927+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/289317398 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0d2c066f30 msgr2=0x7f0d2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:58.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.927+0000 7f0d4ed9d700 1 --2- 172.21.15.55:0/289317398 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0d2c066f30 0x7f0d2c0693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0d480c3100 tx=0x7f0d440091a0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.927+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/289317398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d48156310 msgr2=0x7f0d4814f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:58.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.927+0000 7f0d4ed9d700 1 --2- 172.21.15.55:0/289317398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d48156310 0x7f0d4814f470 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c7c0 tx=0x7f0d38007f50 comp rx=0 tx=0).stop 2023-10-06T03:33:58.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.927+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/289317398 shutdown_connections 2023-10-06T03:33:58.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.927+0000 7f0d4ed9d700 1 --2- 172.21.15.55:0/289317398 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d481566e0 0x7f0d480c2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.927+0000 7f0d4ed9d700 1 --2- 172.21.15.55:0/289317398 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0d2c066f30 0x7f0d2c0693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.927+0000 7f0d4ed9d700 1 --2- 172.21.15.55:0/289317398 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d480ae710 0x7f0d480aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.927+0000 7f0d4ed9d700 1 --2- 172.21.15.55:0/289317398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d48156310 0x7f0d4814f470 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:58.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.927+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/289317398 >> 172.21.15.55:0/289317398 conn(0x7f0d480cc1f0 msgr2=0x7f0d480c0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:58.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.927+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/289317398 shutdown_connections 2023-10-06T03:33:58.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:58.927+0000 7f0d4ed9d700 1 -- 172.21.15.55:0/289317398 wait complete. 2023-10-06T03:33:58.930 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 8 2023-10-06T03:33:58.946 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:33:59.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.132+0000 7f83db59d700 1 Processor -- start 2023-10-06T03:33:59.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.132+0000 7f83db59d700 1 -- start start 2023-10-06T03:33:59.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.133+0000 7f83db59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d406c0f0 0x7f83d41642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:59.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.134+0000 7f83db59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d40b40f0 con 0x7f83d40ad870 2023-10-06T03:33:59.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.134+0000 7f83db59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d4069640 con 0x7f83d4142ea0 2023-10-06T03:33:59.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.134+0000 7f83db59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d4069920 con 0x7f83d406c0f0 2023-10-06T03:33:59.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.134+0000 7f83d9d9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f83d40ad870 0x7f83d406b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45360/0 (socket says 172.21.15.55:45360) 2023-10-06T03:33:59.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.134+0000 7f83d9d9a700 1 -- 172.21.15.55:0/2070493662 learned_addr learned my addr 172.21.15.55:0/2070493662 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:59.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.134+0000 7f83dad9c700 1 --2- 172.21.15.55:0/2070493662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d406c0f0 0x7f83d41642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:59.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.134+0000 7f83d9599700 1 -- 172.21.15.55:0/2070493662 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100836543 0 0) 0x7f83d40b40f0 con 0x7f83d40ad870 2023-10-06T03:33:59.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.134+0000 7f83d9599700 1 -- 172.21.15.55:0/2070493662 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83c0003570 con 0x7f83d40ad870 2023-10-06T03:33:59.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.134+0000 7f83d9599700 1 -- 172.21.15.55:0/2070493662 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928655160 0 0) 0x7f83d4069640 con 0x7f83d4142ea0 2023-10-06T03:33:59.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.134+0000 7f83d9599700 1 -- 172.21.15.55:0/2070493662 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83d40b40f0 con 0x7f83d4142ea0 2023-10-06T03:33:59.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.135+0000 7f83dad9c700 1 -- 172.21.15.55:0/2070493662 >> v1:172.21.15.163:6789/0 conn(0x7f83d4142ea0 legacy=0x7f83d40b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:59.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.135+0000 7f83dad9c700 1 -- 172.21.15.55:0/2070493662 >> v1:172.21.15.55:6789/0 conn(0x7f83d40ad870 legacy=0x7f83d406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:59.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.135+0000 7f83dad9c700 1 -- 172.21.15.55:0/2070493662 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d4069c00 con 0x7f83d406c0f0 2023-10-06T03:33:59.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.135+0000 7f83dad9c700 1 --2- 172.21.15.55:0/2070493662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d406c0f0 0x7f83d41642e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009e90 tx=0x7f83cc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66252cf98cc45da4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:59.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.135+0000 7f83d9599700 1 -- 172.21.15.55:0/2070493662 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc00e070 con 0x7f83d406c0f0 2023-10-06T03:33:59.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.135+0000 7f83d9599700 1 -- 172.21.15.55:0/2070493662 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc00bb80 con 0x7f83d406c0f0 2023-10-06T03:33:59.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.135+0000 7f83d9599700 1 -- 172.21.15.55:0/2070493662 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc012430 con 0x7f83d406c0f0 2023-10-06T03:33:59.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.136+0000 7f83db59d700 1 -- 172.21.15.55:0/2070493662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d406c0f0 msgr2=0x7f83d41642e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:59.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.136+0000 7f83db59d700 1 --2- 172.21.15.55:0/2070493662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d406c0f0 0x7f83d41642e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009e90 tx=0x7f83cc009780 comp rx=0 tx=0).stop 2023-10-06T03:33:59.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.136+0000 7f83db59d700 1 -- 172.21.15.55:0/2070493662 shutdown_connections 2023-10-06T03:33:59.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.136+0000 7f83db59d700 1 --2- 172.21.15.55:0/2070493662 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d406c0f0 0x7f83d41642e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.136+0000 7f83db59d700 1 -- 172.21.15.55:0/2070493662 >> 172.21.15.55:0/2070493662 conn(0x7f83d40ce430 msgr2=0x7f83d40ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:59.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.136+0000 7f83db59d700 1 -- 172.21.15.55:0/2070493662 shutdown_connections 2023-10-06T03:33:59.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.136+0000 7f83db59d700 1 -- 172.21.15.55:0/2070493662 wait complete. 2023-10-06T03:33:59.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.137+0000 7f83db59d700 1 Processor -- start 2023-10-06T03:33:59.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.137+0000 7f83db59d700 1 -- start start 2023-10-06T03:33:59.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.138+0000 7f83db59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d406c0f0 0x7f83d40c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:59.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.138+0000 7f83db59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d4142ea0 0x7f83d40c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:59.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.138+0000 7f83db59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d416a610 0x7f83d415c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:59.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.138+0000 7f83db59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40c7e40 con 0x7f83d416a610 2023-10-06T03:33:59.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.138+0000 7f83db59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40c8120 con 0x7f83d4142ea0 2023-10-06T03:33:59.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.138+0000 7f83db59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40c8400 con 0x7f83d406c0f0 2023-10-06T03:33:59.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.138+0000 7f83dad9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d416a610 0x7f83d415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:59.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.138+0000 7f83dad9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d416a610 0x7f83d415c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51594/0 (socket says 172.21.15.55:51594) 2023-10-06T03:33:59.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.138+0000 7f83dad9c700 1 -- 172.21.15.55:0/2147075022 learned_addr learned my addr 172.21.15.55:0/2147075022 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:59.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.139+0000 7f83da59b700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d406c0f0 0x7f83d40c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:59.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.139+0000 7f83d9d9a700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d4142ea0 0x7f83d40c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:59.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.139+0000 7f83dad9c700 1 -- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d406c0f0 msgr2=0x7f83d40c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:59.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.139+0000 7f83dad9c700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d406c0f0 0x7f83d40c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.139+0000 7f83dad9c700 1 -- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d4142ea0 msgr2=0x7f83d40c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:59.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.139+0000 7f83dad9c700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d4142ea0 0x7f83d40c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.139+0000 7f83dad9c700 1 -- 172.21.15.55:0/2147075022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc009a20 con 0x7f83d416a610 2023-10-06T03:33:59.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.139+0000 7f83d9d9a700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d4142ea0 0x7f83d40c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:59.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.139+0000 7f83dad9c700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d416a610 0x7f83d415c910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f83cc013000 tx=0x7f83cc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:59.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.139+0000 7f83ca7fc700 1 -- 172.21.15.55:0/2147075022 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc012e00 con 0x7f83d416a610 2023-10-06T03:33:59.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.139+0000 7f83ca7fc700 1 -- 172.21.15.55:0/2147075022 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc01b630 con 0x7f83d416a610 2023-10-06T03:33:59.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.139+0000 7f83db59d700 1 -- 172.21.15.55:0/2147075022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d40c86e0 con 0x7f83d416a610 2023-10-06T03:33:59.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.139+0000 7f83ca7fc700 1 -- 172.21.15.55:0/2147075022 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc01b9f0 con 0x7f83d416a610 2023-10-06T03:33:59.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.140+0000 7f83db59d700 1 -- 172.21.15.55:0/2147075022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d416aa70 con 0x7f83d416a610 2023-10-06T03:33:59.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.141+0000 7f83db59d700 1 -- 172.21.15.55:0/2147075022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d404c3c0 con 0x7f83d416a610 2023-10-06T03:33:59.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.141+0000 7f83ca7fc700 1 -- 172.21.15.55:0/2147075022 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f83cc0253f0 con 0x7f83d416a610 2023-10-06T03:33:59.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.143+0000 7f83ca7fc700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f83b8066f30 0x7f83b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:59.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.143+0000 7f83ca7fc700 1 -- 172.21.15.55:0/2147075022 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f83cc086110 con 0x7f83d416a610 2023-10-06T03:33:59.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.143+0000 7f83da59b700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f83b8066f30 0x7f83b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:59.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.144+0000 7f83da59b700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f83b8066f30 0x7f83b80693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f83d40c9510 tx=0x7f83d0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:59.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.147+0000 7f83ca7fc700 1 -- 172.21.15.55:0/2147075022 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83cc056cc0 con 0x7f83d416a610 2023-10-06T03:33:59.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.349+0000 7f83db59d700 1 -- 172.21.15.55:0/2147075022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83d40b3d40 con 0x7f83d416a610 2023-10-06T03:33:59.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.350+0000 7f83ca7fc700 1 -- 172.21.15.55:0/2147075022 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f83cc056850 con 0x7f83d416a610 2023-10-06T03:33:59.352 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:59.352 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:16.470817+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:33:59.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.352+0000 7f83db59d700 1 -- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f83b8066f30 msgr2=0x7f83b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:59.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.352+0000 7f83db59d700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f83b8066f30 0x7f83b80693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f83d40c9510 tx=0x7f83d0009040 comp rx=0 tx=0).stop 2023-10-06T03:33:59.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.352+0000 7f83db59d700 1 -- 172.21.15.55:0/2147075022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d416a610 msgr2=0x7f83d415c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:59.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.352+0000 7f83db59d700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d416a610 0x7f83d415c910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f83cc013000 tx=0x7f83cc019000 comp rx=0 tx=0).stop 2023-10-06T03:33:59.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.352+0000 7f83db59d700 1 -- 172.21.15.55:0/2147075022 shutdown_connections 2023-10-06T03:33:59.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.352+0000 7f83db59d700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d416a610 0x7f83d415c910 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.352+0000 7f83db59d700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f83b8066f30 0x7f83b80693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.352+0000 7f83db59d700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d406c0f0 0x7f83d40c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.352+0000 7f83db59d700 1 --2- 172.21.15.55:0/2147075022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d4142ea0 0x7f83d40c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.352+0000 7f83db59d700 1 -- 172.21.15.55:0/2147075022 >> 172.21.15.55:0/2147075022 conn(0x7f83d40ce430 msgr2=0x7f83d40b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:59.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.353+0000 7f83db59d700 1 -- 172.21.15.55:0/2147075022 shutdown_connections 2023-10-06T03:33:59.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.353+0000 7f83db59d700 1 -- 172.21.15.55:0/2147075022 wait complete. 2023-10-06T03:33:59.372 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:33:59.372 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:33:59.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.558+0000 7ff921d9d700 1 Processor -- start 2023-10-06T03:33:59.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.558+0000 7ff921d9d700 1 -- start start 2023-10-06T03:33:59.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.559+0000 7ff921d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff91c15d230 0x7ff91c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:59.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.560+0000 7ff921d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff91c0b2980 con 0x7ff91c159420 2023-10-06T03:33:59.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.560+0000 7ff921d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff91c15db70 con 0x7ff91c1597f0 2023-10-06T03:33:59.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.560+0000 7ff921d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91c0b11d0 con 0x7ff91c15d230 2023-10-06T03:33:59.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.560+0000 7ff920d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff91c159420 0x7ff91c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45364/0 (socket says 172.21.15.55:45364) 2023-10-06T03:33:59.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.560+0000 7ff920d9b700 1 -- 172.21.15.55:0/407763798 learned_addr learned my addr 172.21.15.55:0/407763798 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:33:59.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.560+0000 7ff92159c700 1 --2- 172.21.15.55:0/407763798 >> v2:172.21.15.163:3300/0 conn(0x7ff91c15d230 0x7ff91c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:59.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.560+0000 7ff91b7fe700 1 -- 172.21.15.55:0/407763798 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897974048 0 0) 0x7ff91c0b2980 con 0x7ff91c159420 2023-10-06T03:33:59.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.560+0000 7ff91b7fe700 1 -- 172.21.15.55:0/407763798 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff90c003570 con 0x7ff91c159420 2023-10-06T03:33:59.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.561+0000 7ff92159c700 1 -- 172.21.15.55:0/407763798 >> v1:172.21.15.163:6789/0 conn(0x7ff91c1597f0 legacy=0x7ff91c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:59.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.561+0000 7ff91b7fe700 1 -- 172.21.15.55:0/407763798 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051233627 0 0) 0x7ff91c15db70 con 0x7ff91c1597f0 2023-10-06T03:33:59.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.561+0000 7ff92159c700 1 -- 172.21.15.55:0/407763798 >> v1:172.21.15.55:6789/0 conn(0x7ff91c159420 legacy=0x7ff91c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:59.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.561+0000 7ff92159c700 1 -- 172.21.15.55:0/407763798 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c0b14b0 con 0x7ff91c15d230 2023-10-06T03:33:59.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.561+0000 7ff92159c700 1 --2- 172.21.15.55:0/407763798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff91c15d230 0x7ff91c15d630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff914009e60 tx=0x7ff914009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2042e2e20d6fdfb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:59.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.561+0000 7ff91b7fe700 1 -- 172.21.15.55:0/407763798 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91400e070 con 0x7ff91c15d230 2023-10-06T03:33:59.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.561+0000 7ff91b7fe700 1 -- 172.21.15.55:0/407763798 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91400bb60 con 0x7ff91c15d230 2023-10-06T03:33:59.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.561+0000 7ff91b7fe700 1 -- 172.21.15.55:0/407763798 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914012430 con 0x7ff91c15d230 2023-10-06T03:33:59.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.562+0000 7ff921d9d700 1 -- 172.21.15.55:0/407763798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff91c15d230 msgr2=0x7ff91c15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:59.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.562+0000 7ff921d9d700 1 --2- 172.21.15.55:0/407763798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff91c15d230 0x7ff91c15d630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff914009e60 tx=0x7ff914009780 comp rx=0 tx=0).stop 2023-10-06T03:33:59.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.562+0000 7ff921d9d700 1 -- 172.21.15.55:0/407763798 shutdown_connections 2023-10-06T03:33:59.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.562+0000 7ff921d9d700 1 --2- 172.21.15.55:0/407763798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff91c15d230 0x7ff91c15d630 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.562+0000 7ff921d9d700 1 -- 172.21.15.55:0/407763798 >> 172.21.15.55:0/407763798 conn(0x7ff91c0d0660 msgr2=0x7ff91c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:59.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.562+0000 7ff921d9d700 1 -- 172.21.15.55:0/407763798 shutdown_connections 2023-10-06T03:33:59.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.562+0000 7ff921d9d700 1 -- 172.21.15.55:0/407763798 wait complete. 2023-10-06T03:33:59.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.563+0000 7ff921d9d700 1 Processor -- start 2023-10-06T03:33:59.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.563+0000 7ff921d9d700 1 -- start start 2023-10-06T03:33:59.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.564+0000 7ff921d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff91c159420 0x7ff91c0c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:59.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.564+0000 7ff921d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff91c1597f0 0x7ff91c15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:59.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.564+0000 7ff921d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff91c15d230 0x7ff91c0c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:59.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.564+0000 7ff921d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c15a220 con 0x7ff91c159420 2023-10-06T03:33:59.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.564+0000 7ff921d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c15a4e0 con 0x7ff91c1597f0 2023-10-06T03:33:59.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.564+0000 7ff921d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c15a7c0 con 0x7ff91c15d230 2023-10-06T03:33:59.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.565+0000 7ff920d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff91c159420 0x7ff91c0c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:59.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.565+0000 7ff920d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff91c159420 0x7ff91c0c6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51606/0 (socket says 172.21.15.55:51606) 2023-10-06T03:33:59.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.565+0000 7ff920d9b700 1 -- 172.21.15.55:0/310077870 learned_addr learned my addr 172.21.15.55:0/310077870 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:33:59.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.565+0000 7ff92159c700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff91c15d230 0x7ff91c0c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:59.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.565+0000 7ff920d9b700 1 -- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff91c15d230 msgr2=0x7ff91c0c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:59.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.565+0000 7ff91bfff700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff91c1597f0 0x7ff91c15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:59.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.565+0000 7ff920d9b700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff91c15d230 0x7ff91c0c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.565+0000 7ff920d9b700 1 -- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff91c1597f0 msgr2=0x7ff91c15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:59.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.565+0000 7ff920d9b700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff91c1597f0 0x7ff91c15b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.565+0000 7ff920d9b700 1 -- 172.21.15.55:0/310077870 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff914009a20 con 0x7ff91c159420 2023-10-06T03:33:59.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.565+0000 7ff92159c700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff91c15d230 0x7ff91c0c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:59.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.565+0000 7ff91bfff700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff91c1597f0 0x7ff91c15b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:33:59.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.565+0000 7ff920d9b700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff91c159420 0x7ff91c0c6390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff9100011e0 tx=0x7ff910007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:59.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.565+0000 7ff918ff9700 1 -- 172.21.15.55:0/310077870 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910005b30 con 0x7ff91c159420 2023-10-06T03:33:59.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.566+0000 7ff918ff9700 1 -- 172.21.15.55:0/310077870 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff910005c90 con 0x7ff91c159420 2023-10-06T03:33:59.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.566+0000 7ff918ff9700 1 -- 172.21.15.55:0/310077870 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9100072e0 con 0x7ff91c159420 2023-10-06T03:33:59.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.566+0000 7ff921d9d700 1 -- 172.21.15.55:0/310077870 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91c15aaa0 con 0x7ff91c159420 2023-10-06T03:33:59.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.566+0000 7ff921d9d700 1 -- 172.21.15.55:0/310077870 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c0c68d0 con 0x7ff91c159420 2023-10-06T03:33:59.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.567+0000 7ff921d9d700 1 -- 172.21.15.55:0/310077870 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91c04c3c0 con 0x7ff91c159420 2023-10-06T03:33:59.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.568+0000 7ff918ff9700 1 -- 172.21.15.55:0/310077870 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff910007650 con 0x7ff91c159420 2023-10-06T03:33:59.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.569+0000 7ff918ff9700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff900066f30 0x7ff9000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:33:59.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.569+0000 7ff918ff9700 1 -- 172.21.15.55:0/310077870 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff910083b50 con 0x7ff91c159420 2023-10-06T03:33:59.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.569+0000 7ff91bfff700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff900066f30 0x7ff9000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:33:59.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.570+0000 7ff91bfff700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff900066f30 0x7ff9000693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0af880 tx=0x7ff908009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:33:59.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.573+0000 7ff918ff9700 1 -- 172.21.15.55:0/310077870 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff91004b5a0 con 0x7ff91c159420 2023-10-06T03:33:59.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.822+0000 7ff921d9d700 1 -- 172.21.15.55:0/310077870 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff91c0b1790 con 0x7ff91c159420 2023-10-06T03:33:59.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.822+0000 7ff918ff9700 1 -- 172.21.15.55:0/310077870 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7ff910017020 con 0x7ff91c159420 2023-10-06T03:33:59.824 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:33:59.824 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2677235076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":2677235076},{"type":"v1","addr":"172.21.15.55:6835","nonce":2677235076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1030747394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1030747394},{"type":"v1","addr":"172.21.15.55:6834","nonce":1030747394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2040975046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2040975046},{"type":"v1","addr":"172.21.15.163:6836","nonce":2040975046}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:33:15.383124+0000","modified":"2023-10-06T03:33:18.908835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/1910887369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1910887369},{"type":"v1","addr":"172.21.15.163:6837","nonce":1910887369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:33:59.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.825+0000 7ff921d9d700 1 -- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff900066f30 msgr2=0x7ff9000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:59.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.825+0000 7ff921d9d700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff900066f30 0x7ff9000693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff91c0af880 tx=0x7ff908009040 comp rx=0 tx=0).stop 2023-10-06T03:33:59.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.825+0000 7ff921d9d700 1 -- 172.21.15.55:0/310077870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff91c159420 msgr2=0x7ff91c0c6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:33:59.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.825+0000 7ff921d9d700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff91c159420 0x7ff91c0c6390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff9100011e0 tx=0x7ff910007df0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.825+0000 7ff921d9d700 1 -- 172.21.15.55:0/310077870 shutdown_connections 2023-10-06T03:33:59.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.825+0000 7ff921d9d700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff900066f30 0x7ff9000693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.825+0000 7ff921d9d700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff91c15d230 0x7ff91c0c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.825+0000 7ff921d9d700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff91c159420 0x7ff91c0c6390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.825+0000 7ff921d9d700 1 --2- 172.21.15.55:0/310077870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff91c1597f0 0x7ff91c15b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:33:59.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.825+0000 7ff921d9d700 1 -- 172.21.15.55:0/310077870 >> 172.21.15.55:0/310077870 conn(0x7ff91c0d0660 msgr2=0x7ff91c0699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:33:59.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.825+0000 7ff921d9d700 1 -- 172.21.15.55:0/310077870 shutdown_connections 2023-10-06T03:33:59.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:33:59.825+0000 7ff921d9d700 1 -- 172.21.15.55:0/310077870 wait complete. 2023-10-06T03:33:59.827 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 8 2023-10-06T03:33:59.843 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:34:00.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.028+0000 7f47a7fff700 1 Processor -- start 2023-10-06T03:34:00.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.028+0000 7f47a7fff700 1 -- start start 2023-10-06T03:34:00.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.029+0000 7f47a7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f47a8150550 0x7f47a8157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.029+0000 7f47a7fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f47a80c7410 0x7f47a8069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.029+0000 7f47a7fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a806e2a0 con 0x7f47a8150180 2023-10-06T03:34:00.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.029+0000 7f47a7fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a806b2e0 con 0x7f47a8150550 2023-10-06T03:34:00.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.029+0000 7f47a7fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a80c78e0 con 0x7f47a80c7410 2023-10-06T03:34:00.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.030+0000 7f47a67fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f47a8150550 0x7f47a8157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.030+0000 7f47a6ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f47a8150180 0x7f47a80c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45368/0 (socket says 172.21.15.55:45368) 2023-10-06T03:34:00.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.030+0000 7f47a6ffd700 1 -- 172.21.15.55:0/2262243604 learned_addr learned my addr 172.21.15.55:0/2262243604 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:00.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.030+0000 7f47a5ffb700 1 -- 172.21.15.55:0/2262243604 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2946777701 0 0) 0x7f47a806e2a0 con 0x7f47a8150180 2023-10-06T03:34:00.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.030+0000 7f47a77fe700 1 --2- 172.21.15.55:0/2262243604 >> v2:172.21.15.163:3300/0 conn(0x7f47a80c7410 0x7f47a8069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.030+0000 7f47a67fc700 1 -- 172.21.15.55:0/2262243604 >> v2:172.21.15.163:3300/0 conn(0x7f47a80c7410 msgr2=0x7f47a8069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:00.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.030+0000 7f47a67fc700 1 --2- 172.21.15.55:0/2262243604 >> v2:172.21.15.163:3300/0 conn(0x7f47a80c7410 0x7f47a8069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.030+0000 7f47a67fc700 1 -- 172.21.15.55:0/2262243604 >> v1:172.21.15.55:6789/0 conn(0x7f47a8150180 legacy=0x7f47a80c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.030+0000 7f47a67fc700 1 -- 172.21.15.55:0/2262243604 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80c7bc0 con 0x7f47a8150550 2023-10-06T03:34:00.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.030+0000 7f47a77fe700 1 --2- 172.21.15.55:0/2262243604 >> v2:172.21.15.163:3300/0 conn(0x7f47a80c7410 0x7f47a8069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:00.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.031+0000 7f47a67fc700 1 --2- 172.21.15.55:0/2262243604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47a8150550 0x7f47a8157770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f47a8156270 tx=0x7f479c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b28c96198d974f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:00.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.031+0000 7f47a5ffb700 1 -- 172.21.15.55:0/2262243604 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c00b6b0 con 0x7f47a8150550 2023-10-06T03:34:00.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.031+0000 7f47a5ffb700 1 -- 172.21.15.55:0/2262243604 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c00b810 con 0x7f47a8150550 2023-10-06T03:34:00.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.031+0000 7f47a5ffb700 1 -- 172.21.15.55:0/2262243604 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c00bbb0 con 0x7f47a8150550 2023-10-06T03:34:00.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.031+0000 7f47a7fff700 1 -- 172.21.15.55:0/2262243604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47a8150550 msgr2=0x7f47a8157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.031+0000 7f47a7fff700 1 --2- 172.21.15.55:0/2262243604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47a8150550 0x7f47a8157770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f47a8156270 tx=0x7f479c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.031+0000 7f47a7fff700 1 -- 172.21.15.55:0/2262243604 shutdown_connections 2023-10-06T03:34:00.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.031+0000 7f47a7fff700 1 --2- 172.21.15.55:0/2262243604 >> v2:172.21.15.163:3300/0 conn(0x7f47a80c7410 0x7f47a8069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.031+0000 7f47a7fff700 1 --2- 172.21.15.55:0/2262243604 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47a8150550 0x7f47a8157770 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.031+0000 7f47a7fff700 1 -- 172.21.15.55:0/2262243604 >> 172.21.15.55:0/2262243604 conn(0x7f47a80cc1f0 msgr2=0x7f47a80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:00.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.031+0000 7f47a7fff700 1 -- 172.21.15.55:0/2262243604 shutdown_connections 2023-10-06T03:34:00.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.032+0000 7f47a7fff700 1 -- 172.21.15.55:0/2262243604 wait complete. 2023-10-06T03:34:00.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.033+0000 7f47a7fff700 1 Processor -- start 2023-10-06T03:34:00.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.033+0000 7f47a7fff700 1 -- start start 2023-10-06T03:34:00.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.034+0000 7f47a7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47a80c7410 0x7f47a80c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.034+0000 7f47a7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47a8150180 0x7f47a80af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.034+0000 7f47a7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47a8150550 0x7f47a80c2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.034+0000 7f47a7fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8169c90 con 0x7f47a8150180 2023-10-06T03:34:00.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.034+0000 7f47a7fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8169f70 con 0x7f47a8150550 2023-10-06T03:34:00.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.034+0000 7f47a7fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a816a250 con 0x7f47a80c7410 2023-10-06T03:34:00.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.034+0000 7f47a67fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47a8150180 0x7f47a80af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.034+0000 7f47a67fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47a8150180 0x7f47a80af6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51624/0 (socket says 172.21.15.55:51624) 2023-10-06T03:34:00.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.034+0000 7f47a67fc700 1 -- 172.21.15.55:0/4187578782 learned_addr learned my addr 172.21.15.55:0/4187578782 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:00.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.034+0000 7f47a77fe700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47a8150550 0x7f47a80c2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.034+0000 7f47a6ffd700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47a80c7410 0x7f47a80c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.035+0000 7f47a67fc700 1 -- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47a80c7410 msgr2=0x7f47a80c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.035+0000 7f47a67fc700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47a80c7410 0x7f47a80c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.035+0000 7f47a67fc700 1 -- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47a8150550 msgr2=0x7f47a80c2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.035+0000 7f47a67fc700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47a8150550 0x7f47a80c2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.035+0000 7f47a67fc700 1 -- 172.21.15.55:0/4187578782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c009420 con 0x7f47a8150180 2023-10-06T03:34:00.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.035+0000 7f47a77fe700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47a8150550 0x7f47a80c2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:00.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.035+0000 7f47a67fc700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47a8150180 0x7f47a80af6a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47a806fe40 tx=0x7f479c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:00.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.035+0000 7f478effd700 1 -- 172.21.15.55:0/4187578782 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c011420 con 0x7f47a8150180 2023-10-06T03:34:00.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.035+0000 7f478effd700 1 -- 172.21.15.55:0/4187578782 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c011580 con 0x7f47a8150180 2023-10-06T03:34:00.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.035+0000 7f47a7fff700 1 -- 172.21.15.55:0/4187578782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a816a530 con 0x7f47a8150180 2023-10-06T03:34:00.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.035+0000 7f478effd700 1 -- 172.21.15.55:0/4187578782 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c011920 con 0x7f47a8150180 2023-10-06T03:34:00.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.035+0000 7f47a7fff700 1 -- 172.21.15.55:0/4187578782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a8169a20 con 0x7f47a8150180 2023-10-06T03:34:00.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.036+0000 7f47a7fff700 1 -- 172.21.15.55:0/4187578782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a804c3c0 con 0x7f47a8150180 2023-10-06T03:34:00.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.037+0000 7f478effd700 1 -- 172.21.15.55:0/4187578782 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f479c011d10 con 0x7f47a8150180 2023-10-06T03:34:00.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.038+0000 7f478effd700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4794066f30 0x7f47940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.038+0000 7f478effd700 1 -- 172.21.15.55:0/4187578782 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f479c0861a0 con 0x7f47a8150180 2023-10-06T03:34:00.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.038+0000 7f47a6ffd700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4794066f30 0x7f47940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.039+0000 7f47a6ffd700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4794066f30 0x7f47940693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f479800c870 tx=0x7f4798009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:00.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.042+0000 7f478effd700 1 -- 172.21.15.55:0/4187578782 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f479c053290 con 0x7f47a8150180 2023-10-06T03:34:00.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.290+0000 7f47a7fff700 1 -- 172.21.15.55:0/4187578782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47a806ada0 con 0x7f47a8150180 2023-10-06T03:34:00.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.290+0000 7f478effd700 1 -- 172.21.15.55:0/4187578782 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4597 (secure 0 0 0) 0x7f479c0568b0 con 0x7f47a8150180 2023-10-06T03:34:00.292 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:00.292 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4232,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2677235076","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":2677235076},{"type":"v1","addr":"172.21.15.55:6835","nonce":2677235076}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1030747394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1030747394},{"type":"v1","addr":"172.21.15.55:6834","nonce":1030747394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2040975046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":2040975046},{"type":"v1","addr":"172.21.15.163:6836","nonce":2040975046}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:33:15.383124+0000","modified":"2023-10-06T03:33:18.908835+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6837/1910887369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1910887369},{"type":"v1","addr":"172.21.15.163:6837","nonce":1910887369}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T03:34:00.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.293+0000 7f47a7fff700 1 -- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4794066f30 msgr2=0x7f47940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.293+0000 7f47a7fff700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4794066f30 0x7f47940693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f479800c870 tx=0x7f4798009040 comp rx=0 tx=0).stop 2023-10-06T03:34:00.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.293+0000 7f47a7fff700 1 -- 172.21.15.55:0/4187578782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47a8150180 msgr2=0x7f47a80af6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.293+0000 7f47a7fff700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47a8150180 0x7f47a80af6a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f47a806fe40 tx=0x7f479c017000 comp rx=0 tx=0).stop 2023-10-06T03:34:00.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.293+0000 7f47a7fff700 1 -- 172.21.15.55:0/4187578782 shutdown_connections 2023-10-06T03:34:00.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.293+0000 7f47a7fff700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47a8150550 0x7f47a80c2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.293+0000 7f47a7fff700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4794066f30 0x7f47940693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.293+0000 7f47a7fff700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47a80c7410 0x7f47a80c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.293+0000 7f47a7fff700 1 --2- 172.21.15.55:0/4187578782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47a8150180 0x7f47a80af6a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.293+0000 7f47a7fff700 1 -- 172.21.15.55:0/4187578782 >> 172.21.15.55:0/4187578782 conn(0x7f47a80cc1f0 msgr2=0x7f47a8154d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:00.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.293+0000 7f47a7fff700 1 -- 172.21.15.55:0/4187578782 shutdown_connections 2023-10-06T03:34:00.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.293+0000 7f47a7fff700 1 -- 172.21.15.55:0/4187578782 wait complete. 2023-10-06T03:34:00.296 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 8 2023-10-06T03:34:00.312 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:34:00.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.498+0000 7f255d79d700 1 Processor -- start 2023-10-06T03:34:00.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.499+0000 7f255d79d700 1 -- start start 2023-10-06T03:34:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.500+0000 7f255d79d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2558156310 0x7f25580c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.500+0000 7f255d79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25580ae710 0x7f25580c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.500+0000 7f255d79d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25580ca860 con 0x7f25581566e0 2023-10-06T03:34:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.500+0000 7f255d79d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25580cab40 con 0x7f2558156310 2023-10-06T03:34:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.500+0000 7f255d79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25580cae20 con 0x7f25580ae710 2023-10-06T03:34:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.500+0000 7f255cf9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25580ae710 0x7f25580c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.500+0000 7f2557fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2558156310 0x7f25580c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.500+0000 7f255cf9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25580ae710 0x7f25580c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:58372/0 (socket says 172.21.15.55:58372) 2023-10-06T03:34:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.500+0000 7f255cf9c700 1 -- 172.21.15.55:0/780982831 learned_addr learned my addr 172.21.15.55:0/780982831 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.501+0000 7f2557fff700 1 -- 172.21.15.55:0/780982831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25580ae710 msgr2=0x7f25580c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.501+0000 7f2557fff700 1 --2- 172.21.15.55:0/780982831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25580ae710 0x7f25580c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.501+0000 7f2557fff700 1 -- 172.21.15.55:0/780982831 >> v1:172.21.15.163:6789/0 conn(0x7f25581566e0 legacy=0x7f25580ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.501+0000 7f2557fff700 1 -- 172.21.15.55:0/780982831 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25580cb100 con 0x7f2558156310 2023-10-06T03:34:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.501+0000 7f2556ffd700 1 -- 172.21.15.55:0/780982831 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928406374 0 0) 0x7f25580ca860 con 0x7f25581566e0 2023-10-06T03:34:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.501+0000 7f255cf9c700 1 --2- 172.21.15.55:0/780982831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25580ae710 0x7f25580c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:34:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.501+0000 7f2557fff700 1 --2- 172.21.15.55:0/780982831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2558156310 0x7f25580c9dc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f25480098e0 tx=0x7f25480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abeb23c405486031 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:00.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.501+0000 7f2556ffd700 1 -- 172.21.15.55:0/780982831 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254800b640 con 0x7f2558156310 2023-10-06T03:34:00.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.501+0000 7f2556ffd700 1 -- 172.21.15.55:0/780982831 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254800b7a0 con 0x7f2558156310 2023-10-06T03:34:00.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.501+0000 7f2556ffd700 1 -- 172.21.15.55:0/780982831 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254800bb40 con 0x7f2558156310 2023-10-06T03:34:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.502+0000 7f255d79d700 1 -- 172.21.15.55:0/780982831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2558156310 msgr2=0x7f25580c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.502+0000 7f255d79d700 1 --2- 172.21.15.55:0/780982831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2558156310 0x7f25580c9dc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f25480098e0 tx=0x7f25480091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.502+0000 7f255d79d700 1 -- 172.21.15.55:0/780982831 shutdown_connections 2023-10-06T03:34:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.502+0000 7f255d79d700 1 --2- 172.21.15.55:0/780982831 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25580ae710 0x7f25580c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.502+0000 7f255d79d700 1 --2- 172.21.15.55:0/780982831 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2558156310 0x7f25580c9dc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.502+0000 7f255d79d700 1 -- 172.21.15.55:0/780982831 >> 172.21.15.55:0/780982831 conn(0x7f25580cc1f0 msgr2=0x7f25580cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.502+0000 7f255d79d700 1 -- 172.21.15.55:0/780982831 shutdown_connections 2023-10-06T03:34:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.502+0000 7f255d79d700 1 -- 172.21.15.55:0/780982831 wait complete. 2023-10-06T03:34:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.504+0000 7f255d79d700 1 Processor -- start 2023-10-06T03:34:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.504+0000 7f255d79d700 1 -- start start 2023-10-06T03:34:00.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f255d79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25580ae710 0x7f25580c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f255d79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2558156310 0x7f25580c14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f255d79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25581566e0 0x7f25581440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f255d79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2558145100 con 0x7f25580ae710 2023-10-06T03:34:00.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f255d79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25580c66d0 con 0x7f25581566e0 2023-10-06T03:34:00.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f255d79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25580c69b0 con 0x7f2558156310 2023-10-06T03:34:00.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f2557fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25580ae710 0x7f25580c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f2557fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25580ae710 0x7f25580c0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51636/0 (socket says 172.21.15.55:51636) 2023-10-06T03:34:00.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f2557fff700 1 -- 172.21.15.55:0/2914794449 learned_addr learned my addr 172.21.15.55:0/2914794449 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:00.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f255cf9c700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25581566e0 0x7f25581440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f2557fff700 1 -- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2558156310 msgr2=0x7f25580c14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f25577fe700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2558156310 0x7f25580c14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f2557fff700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2558156310 0x7f25580c14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f2557fff700 1 -- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25581566e0 msgr2=0x7f25581440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f2557fff700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25581566e0 0x7f25581440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f2557fff700 1 -- 172.21.15.55:0/2914794449 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2550009ae0 con 0x7f25580ae710 2023-10-06T03:34:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f255cf9c700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25581566e0 0x7f25581440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.505+0000 7f25577fe700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2558156310 0x7f25580c14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.506+0000 7f2557fff700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25580ae710 0x7f25580c0f90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f25480093f0 tx=0x7f2548017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.506+0000 7f253ffff700 1 -- 172.21.15.55:0/2914794449 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254800bb20 con 0x7f25580ae710 2023-10-06T03:34:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.506+0000 7f253ffff700 1 -- 172.21.15.55:0/2914794449 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2548011420 con 0x7f25580ae710 2023-10-06T03:34:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.506+0000 7f255d79d700 1 -- 172.21.15.55:0/2914794449 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2548009420 con 0x7f25580ae710 2023-10-06T03:34:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.506+0000 7f253ffff700 1 -- 172.21.15.55:0/2914794449 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2548011800 con 0x7f25580ae710 2023-10-06T03:34:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.506+0000 7f255d79d700 1 -- 172.21.15.55:0/2914794449 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25580c62f0 con 0x7f25580ae710 2023-10-06T03:34:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.507+0000 7f255d79d700 1 -- 172.21.15.55:0/2914794449 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25580c6540 con 0x7f25580ae710 2023-10-06T03:34:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.508+0000 7f253ffff700 1 -- 172.21.15.55:0/2914794449 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2548011c30 con 0x7f25580ae710 2023-10-06T03:34:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.509+0000 7f253ffff700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2544066f30 0x7f25440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.509+0000 7f253ffff700 1 -- 172.21.15.55:0/2914794449 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f25480860d0 con 0x7f25580ae710 2023-10-06T03:34:00.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.509+0000 7f25577fe700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2544066f30 0x7f25440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.510+0000 7f25577fe700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2544066f30 0x7f25440693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f25580ae0d0 tx=0x7f254c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:00.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.514+0000 7f253ffff700 1 -- 172.21.15.55:0/2914794449 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2548053140 con 0x7f25580ae710 2023-10-06T03:34:00.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.718+0000 7f255d79d700 1 -- 172.21.15.55:0/2914794449 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25580c6ed0 con 0x7f25580ae710 2023-10-06T03:34:00.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.723+0000 7f253ffff700 1 -- 172.21.15.55:0/2914794449 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f25580c6ed0 con 0x7f25580ae710 2023-10-06T03:34:00.724 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:00.724 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":21,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:33:16.470817+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:33:13.543361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T03:33:14.341572+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:34:00.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.725+0000 7f255d79d700 1 -- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2544066f30 msgr2=0x7f25440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.725+0000 7f255d79d700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2544066f30 0x7f25440693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f25580ae0d0 tx=0x7f254c009040 comp rx=0 tx=0).stop 2023-10-06T03:34:00.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.725+0000 7f255d79d700 1 -- 172.21.15.55:0/2914794449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25580ae710 msgr2=0x7f25580c0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.725+0000 7f255d79d700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25580ae710 0x7f25580c0f90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f25480093f0 tx=0x7f2548017000 comp rx=0 tx=0).stop 2023-10-06T03:34:00.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.725+0000 7f255d79d700 1 -- 172.21.15.55:0/2914794449 shutdown_connections 2023-10-06T03:34:00.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.725+0000 7f255d79d700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25581566e0 0x7f25581440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.725+0000 7f255d79d700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25580ae710 0x7f25580c0f90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.725+0000 7f255d79d700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2544066f30 0x7f25440693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.725+0000 7f255d79d700 1 --2- 172.21.15.55:0/2914794449 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2558156310 0x7f25580c14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.725+0000 7f255d79d700 1 -- 172.21.15.55:0/2914794449 >> 172.21.15.55:0/2914794449 conn(0x7f25580cc1f0 msgr2=0x7f255814b530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:00.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.725+0000 7f255d79d700 1 -- 172.21.15.55:0/2914794449 shutdown_connections 2023-10-06T03:34:00.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.725+0000 7f255d79d700 1 -- 172.21.15.55:0/2914794449 wait complete. 2023-10-06T03:34:00.744 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:34:00.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.931+0000 7f541b59e700 1 Processor -- start 2023-10-06T03:34:00.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.931+0000 7f541b59e700 1 -- start start 2023-10-06T03:34:00.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.932+0000 7f541b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f541c1597f0 0x7f541c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.932+0000 7f541b59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c0b11d0 con 0x7f541c15d230 2023-10-06T03:34:00.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.932+0000 7f541b59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541c0b14b0 con 0x7f541c159420 2023-10-06T03:34:00.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.932+0000 7f541b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0b1790 con 0x7f541c1597f0 2023-10-06T03:34:00.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.933+0000 7f541ad9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f541c15d230 0x7f541c0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45380/0 (socket says 172.21.15.55:45380) 2023-10-06T03:34:00.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.933+0000 7f541ad9d700 1 -- 172.21.15.55:0/895654333 learned_addr learned my addr 172.21.15.55:0/895654333 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:00.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.933+0000 7f5419d9b700 1 --2- 172.21.15.55:0/895654333 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f541c1597f0 0x7f541c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.933+0000 7f541959a700 1 -- 172.21.15.55:0/895654333 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984281209 0 0) 0x7f541c0b11d0 con 0x7f541c15d230 2023-10-06T03:34:00.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.933+0000 7f541959a700 1 -- 172.21.15.55:0/895654333 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5404003570 con 0x7f541c15d230 2023-10-06T03:34:00.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.933+0000 7f541959a700 1 -- 172.21.15.55:0/895654333 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2049353870 0 0) 0x7f5404003570 con 0x7f541c15d230 2023-10-06T03:34:00.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.933+0000 7f541959a700 1 -- 172.21.15.55:0/895654333 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f541c1597f0 msgr2=0x7f541c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:00.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.933+0000 7f541959a700 1 --2- 172.21.15.55:0/895654333 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f541c1597f0 0x7f541c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.933+0000 7f541959a700 1 -- 172.21.15.55:0/895654333 >> v1:172.21.15.163:6789/0 conn(0x7f541c159420 legacy=0x7f541c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.933+0000 7f541959a700 1 -- 172.21.15.55:0/895654333 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c0b1a70 con 0x7f541c15d230 2023-10-06T03:34:00.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.933+0000 7f541959a700 1 -- 172.21.15.55:0/895654333 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f54140017f0 con 0x7f541c15d230 2023-10-06T03:34:00.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.933+0000 7f541959a700 1 -- 172.21.15.55:0/895654333 >> v1:172.21.15.55:6789/0 conn(0x7f541c15d230 legacy=0x7f541c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.934+0000 7f541959a700 1 --2- 172.21.15.55:0/895654333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5404003a90 0x7f5404003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.934+0000 7f541959a700 1 --2- 172.21.15.55:0/895654333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5404004810 0x7f5404004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.934+0000 7f541959a700 1 --2- 172.21.15.55:0/895654333 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f541c1597f0 0x7f54040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.934+0000 7f541959a700 1 -- 172.21.15.55:0/895654333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0b1790 con 0x7f5404004810 2023-10-06T03:34:00.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.934+0000 7f541959a700 1 -- 172.21.15.55:0/895654333 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540400bb40 con 0x7f5404003a90 2023-10-06T03:34:00.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.934+0000 7f541959a700 1 -- 172.21.15.55:0/895654333 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540400bd00 con 0x7f541c1597f0 2023-10-06T03:34:00.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.934+0000 7f5419d9b700 1 --2- 172.21.15.55:0/895654333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5404004810 0x7f5404004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.934+0000 7f541ad9d700 1 --2- 172.21.15.55:0/895654333 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f541c1597f0 0x7f54040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.934+0000 7f541a59c700 1 --2- 172.21.15.55:0/895654333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5404003a90 0x7f5404003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.934+0000 7f5419d9b700 1 -- 172.21.15.55:0/895654333 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f541c1597f0 msgr2=0x7f54040051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:00.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.934+0000 7f5419d9b700 1 --2- 172.21.15.55:0/895654333 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f541c1597f0 0x7f54040051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.934+0000 7f5419d9b700 1 -- 172.21.15.55:0/895654333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5404003a90 msgr2=0x7f5404003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:00.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.934+0000 7f5419d9b700 1 --2- 172.21.15.55:0/895654333 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5404003a90 0x7f5404003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.934+0000 7f5419d9b700 1 -- 172.21.15.55:0/895654333 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5404003570 con 0x7f5404004810 2023-10-06T03:34:00.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.935+0000 7f5419d9b700 1 --2- 172.21.15.55:0/895654333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5404004810 0x7f5404004c80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5410009c00 tx=0x7f54100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f9ef01018d4077 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:00.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.935+0000 7f541959a700 1 -- 172.21.15.55:0/895654333 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541000e710 con 0x7f5404004810 2023-10-06T03:34:00.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.935+0000 7f541959a700 1 -- 172.21.15.55:0/895654333 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541000e870 con 0x7f5404004810 2023-10-06T03:34:00.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.935+0000 7f541959a700 1 -- 172.21.15.55:0/895654333 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541000ec30 con 0x7f5404004810 2023-10-06T03:34:00.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.935+0000 7f541b59e700 1 -- 172.21.15.55:0/895654333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5404004810 msgr2=0x7f5404004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.935+0000 7f541b59e700 1 --2- 172.21.15.55:0/895654333 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5404004810 0x7f5404004c80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5410009c00 tx=0x7f54100091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.935+0000 7f541ad9d700 1 -- 172.21.15.55:0/895654333 reap_dead start 2023-10-06T03:34:00.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.935+0000 7f541b59e700 1 -- 172.21.15.55:0/895654333 shutdown_connections 2023-10-06T03:34:00.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.935+0000 7f541b59e700 1 -- 172.21.15.55:0/895654333 >> 172.21.15.55:0/895654333 conn(0x7f541c0d0660 msgr2=0x7f541c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:00.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.936+0000 7f541b59e700 1 -- 172.21.15.55:0/895654333 shutdown_connections 2023-10-06T03:34:00.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.936+0000 7f541b59e700 1 -- 172.21.15.55:0/895654333 wait complete. 2023-10-06T03:34:00.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.937+0000 7f541b59e700 1 Processor -- start 2023-10-06T03:34:00.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.937+0000 7f541b59e700 1 -- start start 2023-10-06T03:34:00.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.938+0000 7f541b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5404004810 0x7f541c15ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.938+0000 7f541b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f541c15a580 0x7f541c157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.938+0000 7f541b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f541c0c71c0 0x7f541c0c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.938+0000 7f541b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c15b570 con 0x7f541c15a580 2023-10-06T03:34:00.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.938+0000 7f541b59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0c9880 con 0x7f5404004810 2023-10-06T03:34:00.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.938+0000 7f541b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541c0c7b50 con 0x7f541c0c71c0 2023-10-06T03:34:00.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.938+0000 7f5419d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f541c15a580 0x7f541c157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.938+0000 7f5419d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f541c15a580 0x7f541c157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51656/0 (socket says 172.21.15.55:51656) 2023-10-06T03:34:00.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.938+0000 7f5419d9b700 1 -- 172.21.15.55:0/2879725527 learned_addr learned my addr 172.21.15.55:0/2879725527 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:00.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.938+0000 7f541a59c700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5404004810 0x7f541c15ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.938+0000 7f541ad9d700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f541c0c71c0 0x7f541c0c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.939+0000 7f5419d9b700 1 -- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f541c0c71c0 msgr2=0x7f541c0c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.939+0000 7f5419d9b700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f541c0c71c0 0x7f541c0c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.939+0000 7f5419d9b700 1 -- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5404004810 msgr2=0x7f541c15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:00.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.939+0000 7f5419d9b700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5404004810 0x7f541c15ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:00.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.939+0000 7f5419d9b700 1 -- 172.21.15.55:0/2879725527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541c1577b0 con 0x7f541c15a580 2023-10-06T03:34:00.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.939+0000 7f541a59c700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5404004810 0x7f541c15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:00.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.939+0000 7f5419d9b700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f541c15a580 0x7f541c157270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f541000e4b0 tx=0x7f541001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:00.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.939+0000 7f540a7fc700 1 -- 172.21.15.55:0/2879725527 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410022070 con 0x7f541c15a580 2023-10-06T03:34:00.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.939+0000 7f540a7fc700 1 -- 172.21.15.55:0/2879725527 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541000e9e0 con 0x7f541c15a580 2023-10-06T03:34:00.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.939+0000 7f540a7fc700 1 -- 172.21.15.55:0/2879725527 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410027470 con 0x7f541c15a580 2023-10-06T03:34:00.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.939+0000 7f541b59e700 1 -- 172.21.15.55:0/2879725527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541c0c9550 con 0x7f541c15a580 2023-10-06T03:34:00.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.939+0000 7f541b59e700 1 -- 172.21.15.55:0/2879725527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541c15afb0 con 0x7f541c15a580 2023-10-06T03:34:00.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.940+0000 7f541b59e700 1 -- 172.21.15.55:0/2879725527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541c0b3ee0 con 0x7f541c15a580 2023-10-06T03:34:00.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.941+0000 7f540a7fc700 1 -- 172.21.15.55:0/2879725527 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5410027880 con 0x7f541c15a580 2023-10-06T03:34:00.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.942+0000 7f540a7fc700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f53fc066f30 0x7f53fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:00.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.942+0000 7f540a7fc700 1 -- 172.21.15.55:0/2879725527 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5410086250 con 0x7f541c15a580 2023-10-06T03:34:00.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.943+0000 7f541a59c700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f53fc066f30 0x7f53fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:00.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.944+0000 7f541a59c700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f53fc066f30 0x7f53fc0693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f540c000c00 tx=0x7f540c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:00.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:00.946+0000 7f540a7fc700 1 -- 172.21.15.55:0/2879725527 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5410053370 con 0x7f541c15a580 2023-10-06T03:34:01.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.193+0000 7f541b59e700 1 -- 172.21.15.55:0/2879725527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f541c04c410 con 0x7f541c15a580 2023-10-06T03:34:01.194 INFO:tasks.ceph.mon.a.smithi055.stderr:2023-10-06T03:34:01.193+0000 7fe524ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:34:01.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.201+0000 7f540a7fc700 1 -- 172.21.15.55:0/2879725527 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5410056990 con 0x7f541c15a580 2023-10-06T03:34:01.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.204+0000 7f541b59e700 1 -- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f53fc066f30 msgr2=0x7f53fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:01.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.204+0000 7f541b59e700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f53fc066f30 0x7f53fc0693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f540c000c00 tx=0x7f540c009040 comp rx=0 tx=0).stop 2023-10-06T03:34:01.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.204+0000 7f541b59e700 1 -- 172.21.15.55:0/2879725527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f541c15a580 msgr2=0x7f541c157270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:01.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.204+0000 7f541b59e700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f541c15a580 0x7f541c157270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f541000e4b0 tx=0x7f541001a000 comp rx=0 tx=0).stop 2023-10-06T03:34:01.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.204+0000 7f541b59e700 1 -- 172.21.15.55:0/2879725527 shutdown_connections 2023-10-06T03:34:01.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.204+0000 7f541b59e700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f53fc066f30 0x7f53fc0693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:01.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.204+0000 7f541b59e700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f541c0c71c0 0x7f541c0c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:01.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.204+0000 7f541b59e700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5404004810 0x7f541c15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:01.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.204+0000 7f541b59e700 1 --2- 172.21.15.55:0/2879725527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f541c15a580 0x7f541c157270 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:01.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.204+0000 7f541b59e700 1 -- 172.21.15.55:0/2879725527 >> 172.21.15.55:0/2879725527 conn(0x7f541c0d0660 msgr2=0x7f541c0b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:01.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.204+0000 7f541b59e700 1 -- 172.21.15.55:0/2879725527 shutdown_connections 2023-10-06T03:34:01.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.204+0000 7f541b59e700 1 -- 172.21.15.55:0/2879725527 wait complete. 2023-10-06T03:34:01.207 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:34:01.211 INFO:tasks.ceph.mds.b.smithi163.stderr: -2130> 2023-10-06T03:33:20.660+0000 7fb0dedf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:34:01.223 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:34:01.254 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T03:34:01.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.411+0000 7fb760d9d700 1 Processor -- start 2023-10-06T03:34:01.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.411+0000 7fb760d9d700 1 -- start start 2023-10-06T03:34:01.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.412+0000 7fb760d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb75c0cb1a0 0x7fb75c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:01.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.412+0000 7fb760d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb75c15a8a0 0x7fb75c06f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:01.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.412+0000 7fb760d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb75c06f5a0 0x7fb75c0ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:01.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.412+0000 7fb760d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb75c15b2e0 con 0x7fb75c06f5a0 2023-10-06T03:34:01.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.413+0000 7fb760d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb75c06ebd0 con 0x7fb75c0cb1a0 2023-10-06T03:34:01.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.413+0000 7fb760d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c15ad20 con 0x7fb75c15a8a0 2023-10-06T03:34:01.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.413+0000 7fb75bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb75c06f5a0 0x7fb75c0ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:01.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.413+0000 7fb75bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb75c06f5a0 0x7fb75c0ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51666/0 (socket says 172.21.15.55:51666) 2023-10-06T03:34:01.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.413+0000 7fb75bfff700 1 -- 172.21.15.55:0/113978861 learned_addr learned my addr 172.21.15.55:0/113978861 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:01.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.413+0000 7fb75affd700 1 --2- 172.21.15.55:0/113978861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb75c15a8a0 0x7fb75c06f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:01.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.413+0000 7fb75b7fe700 1 --2- 172.21.15.55:0/113978861 >> v2:172.21.15.163:3300/0 conn(0x7fb75c0cb1a0 0x7fb75c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:01.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.413+0000 7fb75bfff700 1 -- 172.21.15.55:0/113978861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb75c15a8a0 msgr2=0x7fb75c06f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:01.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.413+0000 7fb75bfff700 1 --2- 172.21.15.55:0/113978861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb75c15a8a0 0x7fb75c06f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:01.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.413+0000 7fb75bfff700 1 -- 172.21.15.55:0/113978861 >> v2:172.21.15.163:3300/0 conn(0x7fb75c0cb1a0 msgr2=0x7fb75c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:01.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.413+0000 7fb75bfff700 1 --2- 172.21.15.55:0/113978861 >> v2:172.21.15.163:3300/0 conn(0x7fb75c0cb1a0 0x7fb75c15a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:01.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.413+0000 7fb75bfff700 1 -- 172.21.15.55:0/113978861 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c15b000 con 0x7fb75c06f5a0 2023-10-06T03:34:01.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.414+0000 7fb75bfff700 1 --2- 172.21.15.55:0/113978861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb75c06f5a0 0x7fb75c0ae0d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0cce10 tx=0x7fb754009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fd7a7a84f46df2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:01.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.414+0000 7fb75a7fc700 1 -- 172.21.15.55:0/113978861 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75400e070 con 0x7fb75c06f5a0 2023-10-06T03:34:01.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.414+0000 7fb75a7fc700 1 -- 172.21.15.55:0/113978861 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75400ba00 con 0x7fb75c06f5a0 2023-10-06T03:34:01.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.414+0000 7fb75a7fc700 1 -- 172.21.15.55:0/113978861 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7540092e0 con 0x7fb75c06f5a0 2023-10-06T03:34:01.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.414+0000 7fb760d9d700 1 -- 172.21.15.55:0/113978861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb75c06f5a0 msgr2=0x7fb75c0ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:01.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.414+0000 7fb760d9d700 1 --2- 172.21.15.55:0/113978861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb75c06f5a0 0x7fb75c0ae0d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0cce10 tx=0x7fb754009840 comp rx=0 tx=0).stop 2023-10-06T03:34:01.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.415+0000 7fb760d9d700 1 -- 172.21.15.55:0/113978861 shutdown_connections 2023-10-06T03:34:01.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.415+0000 7fb760d9d700 1 --2- 172.21.15.55:0/113978861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb75c06f5a0 0x7fb75c0ae0d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:01.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.415+0000 7fb760d9d700 1 --2- 172.21.15.55:0/113978861 >> v2:172.21.15.163:3300/0 conn(0x7fb75c0cb1a0 0x7fb75c15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:01.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.415+0000 7fb760d9d700 1 --2- 172.21.15.55:0/113978861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb75c15a8a0 0x7fb75c06f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:01.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.415+0000 7fb760d9d700 1 -- 172.21.15.55:0/113978861 >> 172.21.15.55:0/113978861 conn(0x7fb75c0cc1e0 msgr2=0x7fb75c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:01.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.415+0000 7fb760d9d700 1 -- 172.21.15.55:0/113978861 shutdown_connections 2023-10-06T03:34:01.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.415+0000 7fb760d9d700 1 -- 172.21.15.55:0/113978861 wait complete. 2023-10-06T03:34:01.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.416+0000 7fb760d9d700 1 Processor -- start 2023-10-06T03:34:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.416+0000 7fb760d9d700 1 -- start start 2023-10-06T03:34:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.417+0000 7fb760d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb75c06f5a0 0x7fb75c148f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.417+0000 7fb760d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb75c0cb1a0 0x7fb75c16d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.417+0000 7fb760d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb75c1596b0 0x7fb75c169870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.417+0000 7fb760d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c149490 con 0x7fb75c1596b0 2023-10-06T03:34:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.417+0000 7fb760d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c157d50 con 0x7fb75c06f5a0 2023-10-06T03:34:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.417+0000 7fb760d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c159c70 con 0x7fb75c0cb1a0 2023-10-06T03:34:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.417+0000 7fb75bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb75c1596b0 0x7fb75c169870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.417+0000 7fb75bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb75c1596b0 0x7fb75c169870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51674/0 (socket says 172.21.15.55:51674) 2023-10-06T03:34:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.417+0000 7fb75bfff700 1 -- 172.21.15.55:0/2649399086 learned_addr learned my addr 172.21.15.55:0/2649399086 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:01.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.418+0000 7fb75b7fe700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb75c06f5a0 0x7fb75c148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.418+0000 7fb75affd700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb75c0cb1a0 0x7fb75c16d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.418+0000 7fb75bfff700 1 -- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb75c0cb1a0 msgr2=0x7fb75c16d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.418+0000 7fb75bfff700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb75c0cb1a0 0x7fb75c16d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.418+0000 7fb75bfff700 1 -- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb75c06f5a0 msgr2=0x7fb75c148f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.418+0000 7fb75bfff700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb75c06f5a0 0x7fb75c148f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.418+0000 7fb75bfff700 1 -- 172.21.15.55:0/2649399086 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb754009ae0 con 0x7fb75c1596b0 2023-10-06T03:34:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.418+0000 7fb75b7fe700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb75c06f5a0 0x7fb75c148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.418+0000 7fb75bfff700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb75c1596b0 0x7fb75c169870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb75400c010 tx=0x7fb754018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.418+0000 7fb7437fe700 1 -- 172.21.15.55:0/2649399086 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754008c30 con 0x7fb75c1596b0 2023-10-06T03:34:01.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.418+0000 7fb7437fe700 1 -- 172.21.15.55:0/2649399086 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb754008db0 con 0x7fb75c1596b0 2023-10-06T03:34:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.418+0000 7fb7437fe700 1 -- 172.21.15.55:0/2649399086 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75401d670 con 0x7fb75c1596b0 2023-10-06T03:34:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.418+0000 7fb760d9d700 1 -- 172.21.15.55:0/2649399086 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75c159f50 con 0x7fb75c1596b0 2023-10-06T03:34:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.418+0000 7fb760d9d700 1 -- 172.21.15.55:0/2649399086 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75c1579e0 con 0x7fb75c1596b0 2023-10-06T03:34:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.419+0000 7fb760d9d700 1 -- 172.21.15.55:0/2649399086 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75c06c450 con 0x7fb75c1596b0 2023-10-06T03:34:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.420+0000 7fb7437fe700 1 -- 172.21.15.55:0/2649399086 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb754003040 con 0x7fb75c1596b0 2023-10-06T03:34:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.421+0000 7fb7437fe700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb748066f30 0x7fb7480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:01.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.421+0000 7fb7437fe700 1 -- 172.21.15.55:0/2649399086 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fb7540862c0 con 0x7fb75c1596b0 2023-10-06T03:34:01.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.422+0000 7fb75b7fe700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb748066f30 0x7fb7480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:01.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.422+0000 7fb75b7fe700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb748066f30 0x7fb7480693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009d80 tx=0x7fb74c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:01.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.425+0000 7fb7437fe700 1 -- 172.21.15.55:0/2649399086 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb754056d90 con 0x7fb75c1596b0 2023-10-06T03:34:01.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:01.672+0000 7fb760d9d700 1 -- 172.21.15.55:0/2649399086 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb75c06e860 con 0x7fb75c1596b0 2023-10-06T03:34:02.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.203+0000 7fb7437fe700 1 -- 172.21.15.55:0/2649399086 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb754009940 con 0x7fb75c1596b0 2023-10-06T03:34:02.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.206+0000 7fb760d9d700 1 -- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb748066f30 msgr2=0x7fb7480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:02.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.206+0000 7fb760d9d700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb748066f30 0x7fb7480693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009d80 tx=0x7fb74c008040 comp rx=0 tx=0).stop 2023-10-06T03:34:02.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.206+0000 7fb760d9d700 1 -- 172.21.15.55:0/2649399086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb75c1596b0 msgr2=0x7fb75c169870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:02.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.206+0000 7fb760d9d700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb75c1596b0 0x7fb75c169870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb75400c010 tx=0x7fb754018000 comp rx=0 tx=0).stop 2023-10-06T03:34:02.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.206+0000 7fb760d9d700 1 -- 172.21.15.55:0/2649399086 shutdown_connections 2023-10-06T03:34:02.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.206+0000 7fb760d9d700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb75c1596b0 0x7fb75c169870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:02.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.206+0000 7fb760d9d700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb75c06f5a0 0x7fb75c148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:02.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.206+0000 7fb760d9d700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb748066f30 0x7fb7480693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:02.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.206+0000 7fb760d9d700 1 --2- 172.21.15.55:0/2649399086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb75c0cb1a0 0x7fb75c16d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:02.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.206+0000 7fb760d9d700 1 -- 172.21.15.55:0/2649399086 >> 172.21.15.55:0/2649399086 conn(0x7fb75c0cc1e0 msgr2=0x7fb75c0aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:02.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.206+0000 7fb760d9d700 1 -- 172.21.15.55:0/2649399086 shutdown_connections 2023-10-06T03:34:02.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.206+0000 7fb760d9d700 1 -- 172.21.15.55:0/2649399086 wait complete. 2023-10-06T03:34:02.230 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:34:02.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.424+0000 7f147c9fe700 1 Processor -- start 2023-10-06T03:34:02.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.424+0000 7f147c9fe700 1 -- start start 2023-10-06T03:34:02.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.425+0000 7f147c9fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1478153850 0x7f14781596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:02.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.425+0000 7f147c9fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1478159bf0 0x7f1478157280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:02.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.425+0000 7f147c9fe700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f147815a480 con 0x7f1478153480 2023-10-06T03:34:02.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.425+0000 7f147c9fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f147815a070 con 0x7f1478153850 2023-10-06T03:34:02.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.425+0000 7f147c9fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14780afb40 con 0x7f1478159bf0 2023-10-06T03:34:02.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.425+0000 7f147659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1478153850 0x7f14781596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:02.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.426+0000 7f147659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1478153850 0x7f14781596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51680/0 (socket says 172.21.15.55:51680) 2023-10-06T03:34:02.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.426+0000 7f147659c700 1 -- 172.21.15.55:0/824233718 learned_addr learned my addr 172.21.15.55:0/824233718 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:02.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.426+0000 7f147659c700 1 -- 172.21.15.55:0/824233718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1478159bf0 msgr2=0x7f1478157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:02.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.426+0000 7f147759e700 1 --2- 172.21.15.55:0/824233718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1478159bf0 0x7f1478157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:02.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.426+0000 7f147659c700 1 --2- 172.21.15.55:0/824233718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1478159bf0 0x7f1478157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:02.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.426+0000 7f1475d9b700 1 -- 172.21.15.55:0/824233718 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395435045 0 0) 0x7f147815a480 con 0x7f1478153480 2023-10-06T03:34:02.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.426+0000 7f147659c700 1 -- 172.21.15.55:0/824233718 >> v1:172.21.15.55:6789/0 conn(0x7f1478153480 legacy=0x7f14780ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:02.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.426+0000 7f147659c700 1 -- 172.21.15.55:0/824233718 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14780afe20 con 0x7f1478153850 2023-10-06T03:34:02.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.426+0000 7f147759e700 1 --2- 172.21.15.55:0/824233718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1478159bf0 0x7f1478157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:02.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.426+0000 7f147659c700 1 --2- 172.21.15.55:0/824233718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1478153850 0x7f14781596b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f146c00bc40 tx=0x7f146c00b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4019d863c2a541 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:02.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.427+0000 7f1475d9b700 1 -- 172.21.15.55:0/824233718 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c00db40 con 0x7f1478153850 2023-10-06T03:34:02.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.427+0000 7f1475d9b700 1 -- 172.21.15.55:0/824233718 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146c00dca0 con 0x7f1478153850 2023-10-06T03:34:02.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.427+0000 7f1475d9b700 1 -- 172.21.15.55:0/824233718 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c017640 con 0x7f1478153850 2023-10-06T03:34:02.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.427+0000 7f147c9fe700 1 -- 172.21.15.55:0/824233718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1478153850 msgr2=0x7f14781596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:02.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.427+0000 7f147c9fe700 1 --2- 172.21.15.55:0/824233718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1478153850 0x7f14781596b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f146c00bc40 tx=0x7f146c00b4e0 comp rx=0 tx=0).stop 2023-10-06T03:34:02.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.427+0000 7f147c9fe700 1 -- 172.21.15.55:0/824233718 shutdown_connections 2023-10-06T03:34:02.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.427+0000 7f147c9fe700 1 --2- 172.21.15.55:0/824233718 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1478159bf0 0x7f1478157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:02.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.427+0000 7f147c9fe700 1 --2- 172.21.15.55:0/824233718 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1478153850 0x7f14781596b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:02.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.427+0000 7f147c9fe700 1 -- 172.21.15.55:0/824233718 >> 172.21.15.55:0/824233718 conn(0x7f1478068e20 msgr2=0x7f14780cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:02.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.427+0000 7f147c9fe700 1 -- 172.21.15.55:0/824233718 shutdown_connections 2023-10-06T03:34:02.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.427+0000 7f147c9fe700 1 -- 172.21.15.55:0/824233718 wait complete. 2023-10-06T03:34:02.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.429+0000 7f147c9fe700 1 Processor -- start 2023-10-06T03:34:02.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.429+0000 7f147c9fe700 1 -- start start 2023-10-06T03:34:02.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.430+0000 7f147c9fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1478153480 0x7f1478153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:02.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.430+0000 7f147c9fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1478153850 0x7f147814e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:02.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.430+0000 7f147c9fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1478159bf0 0x7f14781560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:02.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.430+0000 7f147c9fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14781541d0 con 0x7f1478153480 2023-10-06T03:34:02.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.430+0000 7f147c9fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1478154640 con 0x7f1478153850 2023-10-06T03:34:02.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.430+0000 7f147c9fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14780ad640 con 0x7f1478159bf0 2023-10-06T03:34:02.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.430+0000 7f1476d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1478153480 0x7f1478153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:02.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.430+0000 7f1476d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1478153480 0x7f1478153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51686/0 (socket says 172.21.15.55:51686) 2023-10-06T03:34:02.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.430+0000 7f1476d9d700 1 -- 172.21.15.55:0/2136417342 learned_addr learned my addr 172.21.15.55:0/2136417342 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:02.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.430+0000 7f147659c700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1478153850 0x7f147814e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:02.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.430+0000 7f147759e700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1478159bf0 0x7f14781560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:02.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.430+0000 7f1476d9d700 1 -- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1478159bf0 msgr2=0x7f14781560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:02.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.431+0000 7f1476d9d700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1478159bf0 0x7f14781560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:02.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.431+0000 7f1476d9d700 1 -- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1478153850 msgr2=0x7f147814e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:02.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.431+0000 7f1476d9d700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1478153850 0x7f147814e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:02.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.431+0000 7f1476d9d700 1 -- 172.21.15.55:0/2136417342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c00b780 con 0x7f1478153480 2023-10-06T03:34:02.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.431+0000 7f147659c700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1478153850 0x7f147814e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:02.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.431+0000 7f147759e700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1478159bf0 0x7f14781560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:02.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.431+0000 7f1476d9d700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1478153480 0x7f1478153c90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f146800c7e0 tx=0x7f1468007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:02.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.431+0000 7f145effd700 1 -- 172.21.15.55:0/2136417342 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468012700 con 0x7f1478153480 2023-10-06T03:34:02.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.431+0000 7f145effd700 1 -- 172.21.15.55:0/2136417342 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1468012880 con 0x7f1478153480 2023-10-06T03:34:02.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.431+0000 7f145effd700 1 -- 172.21.15.55:0/2136417342 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468012c40 con 0x7f1478153480 2023-10-06T03:34:02.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.431+0000 7f147c9fe700 1 -- 172.21.15.55:0/2136417342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1478154b70 con 0x7f1478153480 2023-10-06T03:34:02.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.431+0000 7f147c9fe700 1 -- 172.21.15.55:0/2136417342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1478154890 con 0x7f1478153480 2023-10-06T03:34:02.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.432+0000 7f147c9fe700 1 -- 172.21.15.55:0/2136417342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f147804c3c0 con 0x7f1478153480 2023-10-06T03:34:02.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.433+0000 7f145effd700 1 -- 172.21.15.55:0/2136417342 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f146801b400 con 0x7f1478153480 2023-10-06T03:34:02.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.434+0000 7f145effd700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1464066f30 0x7f14640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:02.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.434+0000 7f145effd700 1 -- 172.21.15.55:0/2136417342 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f1468084330 con 0x7f1478153480 2023-10-06T03:34:02.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.434+0000 7f147659c700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1464066f30 0x7f14640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:02.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.435+0000 7f147659c700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1464066f30 0x7f14640693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f147806f1d0 tx=0x7f146c019000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:02.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.438+0000 7f145effd700 1 -- 172.21.15.55:0/2136417342 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1468051370 con 0x7f1478153480 2023-10-06T03:34:02.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:02.642+0000 7f147c9fe700 1 -- 172.21.15.55:0/2136417342 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f14780b0100 con 0x7f1478153480 2023-10-06T03:34:03.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.222+0000 7f145effd700 1 -- 172.21.15.55:0/2136417342 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1468054990 con 0x7f1478153480 2023-10-06T03:34:03.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.226+0000 7f147c9fe700 1 -- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1464066f30 msgr2=0x7f14640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:03.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.226+0000 7f147c9fe700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1464066f30 0x7f14640693e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f147806f1d0 tx=0x7f146c019000 comp rx=0 tx=0).stop 2023-10-06T03:34:03.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.226+0000 7f147c9fe700 1 -- 172.21.15.55:0/2136417342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1478153480 msgr2=0x7f1478153c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.226+0000 7f147c9fe700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1478153480 0x7f1478153c90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f146800c7e0 tx=0x7f1468007df0 comp rx=0 tx=0).stop 2023-10-06T03:34:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.226+0000 7f147c9fe700 1 -- 172.21.15.55:0/2136417342 shutdown_connections 2023-10-06T03:34:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.226+0000 7f147c9fe700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1464066f30 0x7f14640693e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.226+0000 7f147c9fe700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1478159bf0 0x7f14781560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.226+0000 7f147c9fe700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1478153480 0x7f1478153c90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.226+0000 7f147c9fe700 1 --2- 172.21.15.55:0/2136417342 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1478153850 0x7f147814e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:03.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.226+0000 7f147c9fe700 1 -- 172.21.15.55:0/2136417342 >> 172.21.15.55:0/2136417342 conn(0x7f1478068e20 msgr2=0x7f147806dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.227+0000 7f147c9fe700 1 -- 172.21.15.55:0/2136417342 shutdown_connections 2023-10-06T03:34:03.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.227+0000 7f147c9fe700 1 -- 172.21.15.55:0/2136417342 wait complete. 2023-10-06T03:34:03.232 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:34:03.251 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:34:03.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.443+0000 7f47d659d700 1 Processor -- start 2023-10-06T03:34:03.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.443+0000 7f47d659d700 1 -- start start 2023-10-06T03:34:03.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.445+0000 7f47d659d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f47d015b050 0x7f47d00ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:03.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.445+0000 7f47d659d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f47d015bb30 0x7f47d00ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:03.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.445+0000 7f47d659d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d0157a50 con 0x7f47d00cc8d0 2023-10-06T03:34:03.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.445+0000 7f47d659d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d0158f10 con 0x7f47d015bb30 2023-10-06T03:34:03.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.445+0000 7f47d659d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d01591b0 con 0x7f47d015b050 2023-10-06T03:34:03.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.445+0000 7f47d5d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f47d015bb30 0x7f47d00ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:03.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.445+0000 7f47d5d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f47d015bb30 0x7f47d00ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53598/0 (socket says 172.21.15.55:53598) 2023-10-06T03:34:03.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.445+0000 7f47d5d9c700 1 -- 172.21.15.55:0/1441311225 learned_addr learned my addr 172.21.15.55:0/1441311225 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:03.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.445+0000 7f47d559b700 1 --2- 172.21.15.55:0/1441311225 >> v2:172.21.15.163:3300/0 conn(0x7f47d015b050 0x7f47d00ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:03.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.445+0000 7f47d5d9c700 1 -- 172.21.15.55:0/1441311225 >> v2:172.21.15.163:3300/0 conn(0x7f47d015b050 msgr2=0x7f47d00ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:03.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.445+0000 7f47d5d9c700 1 --2- 172.21.15.55:0/1441311225 >> v2:172.21.15.163:3300/0 conn(0x7f47d015b050 0x7f47d00ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:03.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.445+0000 7f47d5d9c700 1 -- 172.21.15.55:0/1441311225 >> v1:172.21.15.163:6789/0 conn(0x7f47d00cc8d0 legacy=0x7f47d015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.445+0000 7f47d5d9c700 1 -- 172.21.15.55:0/1441311225 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d00ca9e0 con 0x7f47d015bb30 2023-10-06T03:34:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.446+0000 7f47d5d9c700 1 --2- 172.21.15.55:0/1441311225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47d015bb30 0x7f47d00ada30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f47d0064f60 tx=0x7f47c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21737895c242e1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.446+0000 7f47c7fff700 1 -- 172.21.15.55:0/1441311225 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c800e070 con 0x7f47d015bb30 2023-10-06T03:34:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.446+0000 7f47c7fff700 1 -- 172.21.15.55:0/1441311225 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c800ba10 con 0x7f47d015bb30 2023-10-06T03:34:03.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.446+0000 7f47c7fff700 1 -- 172.21.15.55:0/1441311225 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c80092e0 con 0x7f47d015bb30 2023-10-06T03:34:03.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.446+0000 7f47d659d700 1 -- 172.21.15.55:0/1441311225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47d015bb30 msgr2=0x7f47d00ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:03.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.446+0000 7f47d659d700 1 --2- 172.21.15.55:0/1441311225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47d015bb30 0x7f47d00ada30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f47d0064f60 tx=0x7f47c8009840 comp rx=0 tx=0).stop 2023-10-06T03:34:03.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.447+0000 7f47d659d700 1 -- 172.21.15.55:0/1441311225 shutdown_connections 2023-10-06T03:34:03.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.447+0000 7f47d659d700 1 --2- 172.21.15.55:0/1441311225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47d015bb30 0x7f47d00ada30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:03.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.447+0000 7f47d659d700 1 --2- 172.21.15.55:0/1441311225 >> v2:172.21.15.163:3300/0 conn(0x7f47d015b050 0x7f47d00ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:03.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.447+0000 7f47d659d700 1 -- 172.21.15.55:0/1441311225 >> 172.21.15.55:0/1441311225 conn(0x7f47d0068e20 msgr2=0x7f47d00cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:03.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.447+0000 7f47d659d700 1 -- 172.21.15.55:0/1441311225 shutdown_connections 2023-10-06T03:34:03.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.447+0000 7f47d659d700 1 -- 172.21.15.55:0/1441311225 wait complete. 2023-10-06T03:34:03.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.448+0000 7f47d659d700 1 Processor -- start 2023-10-06T03:34:03.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.448+0000 7f47d659d700 1 -- start start 2023-10-06T03:34:03.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.449+0000 7f47d659d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47d00cc8d0 0x7f47d0150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:03.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.449+0000 7f47d659d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47d015b050 0x7f47d00c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:03.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.449+0000 7f47d659d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47d015bb30 0x7f47d00c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:03.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.449+0000 7f47d659d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00c3fb0 con 0x7f47d00cc8d0 2023-10-06T03:34:03.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.449+0000 7f47d659d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d0167fb0 con 0x7f47d015b050 2023-10-06T03:34:03.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.449+0000 7f47d659d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d00c25c0 con 0x7f47d015bb30 2023-10-06T03:34:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.449+0000 7f47d559b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47d00cc8d0 0x7f47d0150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47d559b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47d00cc8d0 0x7f47d0150ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53606/0 (socket says 172.21.15.55:53606) 2023-10-06T03:34:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47d559b700 1 -- 172.21.15.55:0/3707789103 learned_addr learned my addr 172.21.15.55:0/3707789103 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47d5d9c700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47d015bb30 0x7f47d00c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47d4d9a700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47d015b050 0x7f47d00c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47d559b700 1 -- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47d015bb30 msgr2=0x7f47d00c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47d559b700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47d015bb30 0x7f47d00c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47d559b700 1 -- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47d015b050 msgr2=0x7f47d00c3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47d559b700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47d015b050 0x7f47d00c3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:03.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47d559b700 1 -- 172.21.15.55:0/3707789103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c8009ae0 con 0x7f47d00cc8d0 2023-10-06T03:34:03.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47d5d9c700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47d015bb30 0x7f47d00c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:03.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47d4d9a700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47d015b050 0x7f47d00c3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:03.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47d559b700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47d00cc8d0 0x7f47d0150ee0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00d890 tx=0x7f47cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:03.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47c57fa700 1 -- 172.21.15.55:0/3707789103 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00b600 con 0x7f47d00cc8d0 2023-10-06T03:34:03.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47c57fa700 1 -- 172.21.15.55:0/3707789103 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc00b760 con 0x7f47d00cc8d0 2023-10-06T03:34:03.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47d659d700 1 -- 172.21.15.55:0/3707789103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d01676c0 con 0x7f47d00cc8d0 2023-10-06T03:34:03.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.450+0000 7f47c57fa700 1 -- 172.21.15.55:0/3707789103 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00bae0 con 0x7f47d00cc8d0 2023-10-06T03:34:03.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.451+0000 7f47d659d700 1 -- 172.21.15.55:0/3707789103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d00c3d80 con 0x7f47d00cc8d0 2023-10-06T03:34:03.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.452+0000 7f47d659d700 1 -- 172.21.15.55:0/3707789103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d006bdd0 con 0x7f47d00cc8d0 2023-10-06T03:34:03.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.452+0000 7f47c57fa700 1 -- 172.21.15.55:0/3707789103 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f47cc01e460 con 0x7f47d00cc8d0 2023-10-06T03:34:03.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.454+0000 7f47c57fa700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f47b4066f30 0x7f47b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:03.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.454+0000 7f47c57fa700 1 -- 172.21.15.55:0/3707789103 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5764+0+0 (secure 0 0 0) 0x7f47cc084c50 con 0x7f47d00cc8d0 2023-10-06T03:34:03.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.454+0000 7f47d4d9a700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f47b4066f30 0x7f47b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:03.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.455+0000 7f47d4d9a700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f47b4066f30 0x7f47b40693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f47c0006fd0 tx=0x7f47c0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:03.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.458+0000 7f47c57fa700 1 -- 172.21.15.55:0/3707789103 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47cc051de0 con 0x7f47d00cc8d0 2023-10-06T03:34:03.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:03.665+0000 7f47d659d700 1 -- 172.21.15.55:0/3707789103 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f47d004c3c0 con 0x7f47d00cc8d0 2023-10-06T03:34:04.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.228+0000 7f47c57fa700 1 -- 172.21.15.55:0/3707789103 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f47cc055400 con 0x7f47d00cc8d0 2023-10-06T03:34:04.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.232+0000 7f47d659d700 1 -- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f47b4066f30 msgr2=0x7f47b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:04.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.232+0000 7f47d659d700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f47b4066f30 0x7f47b40693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f47c0006fd0 tx=0x7f47c0008040 comp rx=0 tx=0).stop 2023-10-06T03:34:04.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.232+0000 7f47d659d700 1 -- 172.21.15.55:0/3707789103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47d00cc8d0 msgr2=0x7f47d0150ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:04.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.232+0000 7f47d659d700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47d00cc8d0 0x7f47d0150ee0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00d890 tx=0x7f47cc008040 comp rx=0 tx=0).stop 2023-10-06T03:34:04.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.234+0000 7f47d659d700 1 -- 172.21.15.55:0/3707789103 shutdown_connections 2023-10-06T03:34:04.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.234+0000 7f47d659d700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f47b4066f30 0x7f47b40693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:04.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.234+0000 7f47d659d700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47d015bb30 0x7f47d00c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:04.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.234+0000 7f47d659d700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47d00cc8d0 0x7f47d0150ee0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:04.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.234+0000 7f47d659d700 1 --2- 172.21.15.55:0/3707789103 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47d015b050 0x7f47d00c3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:04.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.234+0000 7f47d659d700 1 -- 172.21.15.55:0/3707789103 >> 172.21.15.55:0/3707789103 conn(0x7f47d0068e20 msgr2=0x7f47d006d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:04.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.235+0000 7f47d659d700 1 -- 172.21.15.55:0/3707789103 shutdown_connections 2023-10-06T03:34:04.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.235+0000 7f47d659d700 1 -- 172.21.15.55:0/3707789103 wait complete. 2023-10-06T03:34:04.238 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2023-10-06T03:34:04.256 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:34:04.257 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:34:04.257 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:34:04.257 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:34:04.257 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:34:04.257 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:34:04.258 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:34:04.258 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:34:04.258 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:34:04.258 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:34:04.258 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:34:04.258 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:34:04.258 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:34:04.259 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:34:04.259 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:34:04.259 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:34:04.272 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:34:04.273 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:34:04.273 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:34:04.274 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:34:04.275 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:34:04.275 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:34:04.276 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:34:04.276 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:34:04.318 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:34:04.318 DEBUG:teuthology.parallel:result is None 2023-10-06T03:34:04.320 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:34:04.320 DEBUG:teuthology.parallel:result is None 2023-10-06T03:34:04.321 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:34:04.321 DEBUG:teuthology.parallel:result is None 2023-10-06T03:34:04.323 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:34:04.323 DEBUG:teuthology.parallel:result is None 2023-10-06T03:34:04.323 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:34:04.457 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-10-06T03:34:04.457 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T03:34:04.459 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T03:34:04.460 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2023-10-06T03:34:04.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.522+0000 7ff18cb9d700 1 Processor -- start 2023-10-06T03:34:04.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.523+0000 7ff18cb9d700 1 -- start start 2023-10-06T03:34:04.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.524+0000 7ff18cb9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff1881597f0 0x7ff1880b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:04.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.524+0000 7ff18cb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff18815d230 0x7ff18815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:04.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.524+0000 7ff18cb9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff18815dbb0 con 0x7ff188159420 2023-10-06T03:34:04.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.524+0000 7ff18cb9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1880b11d0 con 0x7ff1881597f0 2023-10-06T03:34:04.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.524+0000 7ff18cb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1880b14b0 con 0x7ff18815d230 2023-10-06T03:34:04.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.524+0000 7ff186ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff1881597f0 0x7ff1880b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:04.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.524+0000 7ff186ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff1881597f0 0x7ff1880b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53658/0 (socket says 172.21.15.55:53658) 2023-10-06T03:34:04.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.524+0000 7ff186ffd700 1 -- 172.21.15.55:0/1455162337 learned_addr learned my addr 172.21.15.55:0/1455162337 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:04.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.524+0000 7ff187fff700 1 --2- 172.21.15.55:0/1455162337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff18815d230 0x7ff18815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:04.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.525+0000 7ff186ffd700 1 -- 172.21.15.55:0/1455162337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff18815d230 msgr2=0x7ff18815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:04.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.525+0000 7ff186ffd700 1 --2- 172.21.15.55:0/1455162337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff18815d230 0x7ff18815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:04.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.525+0000 7ff186ffd700 1 -- 172.21.15.55:0/1455162337 >> v1:172.21.15.163:6789/0 conn(0x7ff188159420 legacy=0x7ff1880b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:04.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.525+0000 7ff186ffd700 1 -- 172.21.15.55:0/1455162337 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1880b1790 con 0x7ff1881597f0 2023-10-06T03:34:04.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.525+0000 7ff186ffd700 1 --2- 172.21.15.55:0/1455162337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1881597f0 0x7ff1880b2270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0099b0 tx=0x7ff17c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c1ae8ed0500bbe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:04.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.525+0000 7ff1867fc700 1 -- 172.21.15.55:0/1455162337 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c009ea0 con 0x7ff1881597f0 2023-10-06T03:34:04.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.525+0000 7ff1867fc700 1 -- 172.21.15.55:0/1455162337 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c00b860 con 0x7ff1881597f0 2023-10-06T03:34:04.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.525+0000 7ff1867fc700 1 -- 172.21.15.55:0/1455162337 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00bc00 con 0x7ff1881597f0 2023-10-06T03:34:04.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.526+0000 7ff18cb9d700 1 -- 172.21.15.55:0/1455162337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1881597f0 msgr2=0x7ff1880b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:04.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.526+0000 7ff18cb9d700 1 --2- 172.21.15.55:0/1455162337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1881597f0 0x7ff1880b2270 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0099b0 tx=0x7ff17c009250 comp rx=0 tx=0).stop 2023-10-06T03:34:04.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.526+0000 7ff18cb9d700 1 -- 172.21.15.55:0/1455162337 shutdown_connections 2023-10-06T03:34:04.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.526+0000 7ff18cb9d700 1 --2- 172.21.15.55:0/1455162337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff18815d230 0x7ff18815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:04.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.526+0000 7ff18cb9d700 1 --2- 172.21.15.55:0/1455162337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1881597f0 0x7ff1880b2270 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:04.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.526+0000 7ff18cb9d700 1 -- 172.21.15.55:0/1455162337 >> 172.21.15.55:0/1455162337 conn(0x7ff1880d0660 msgr2=0x7ff1880d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:04.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.526+0000 7ff18cb9d700 1 -- 172.21.15.55:0/1455162337 shutdown_connections 2023-10-06T03:34:04.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.526+0000 7ff18cb9d700 1 -- 172.21.15.55:0/1455162337 wait complete. 2023-10-06T03:34:04.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.527+0000 7ff18cb9d700 1 Processor -- start 2023-10-06T03:34:04.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.527+0000 7ff18cb9d700 1 -- start start 2023-10-06T03:34:04.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.528+0000 7ff18cb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff188159420 0x7ff188146150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:04.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.528+0000 7ff18cb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1881597f0 0x7ff188155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:04.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.528+0000 7ff18cb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff18815d230 0x7ff1881560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:04.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.528+0000 7ff18cb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1880c5400 con 0x7ff1881597f0 2023-10-06T03:34:04.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.528+0000 7ff18cb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18816be20 con 0x7ff18815d230 2023-10-06T03:34:04.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.528+0000 7ff18cb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff188146690 con 0x7ff188159420 2023-10-06T03:34:04.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff186ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1881597f0 0x7ff188155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:04.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff186ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1881597f0 0x7ff188155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53670/0 (socket says 172.21.15.55:53670) 2023-10-06T03:34:04.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff186ffd700 1 -- 172.21.15.55:0/4260074726 learned_addr learned my addr 172.21.15.55:0/4260074726 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:04.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff187fff700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff18815d230 0x7ff1881560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:04.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff1877fe700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff188159420 0x7ff188146150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:04.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff186ffd700 1 -- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff188159420 msgr2=0x7ff188146150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:04.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff186ffd700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff188159420 0x7ff188146150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:04.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff186ffd700 1 -- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff18815d230 msgr2=0x7ff1881560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:04.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff186ffd700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff18815d230 0x7ff1881560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:04.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff186ffd700 1 -- 172.21.15.55:0/4260074726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0094f0 con 0x7ff1881597f0 2023-10-06T03:34:04.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff1877fe700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff188159420 0x7ff188146150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:04.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff186ffd700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1881597f0 0x7ff188155ba0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff188089340 tx=0x7ff17c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:04.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff16f7fe700 1 -- 172.21.15.55:0/4260074726 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c00bbe0 con 0x7ff1881597f0 2023-10-06T03:34:04.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff16f7fe700 1 -- 172.21.15.55:0/4260074726 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c011420 con 0x7ff1881597f0 2023-10-06T03:34:04.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff16f7fe700 1 -- 172.21.15.55:0/4260074726 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0117c0 con 0x7ff1881597f0 2023-10-06T03:34:04.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.529+0000 7ff18cb9d700 1 -- 172.21.15.55:0/4260074726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18816b210 con 0x7ff1881597f0 2023-10-06T03:34:04.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.530+0000 7ff18cb9d700 1 -- 172.21.15.55:0/4260074726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18816b7d0 con 0x7ff1881597f0 2023-10-06T03:34:04.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.531+0000 7ff18cb9d700 1 -- 172.21.15.55:0/4260074726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18816bf90 con 0x7ff1881597f0 2023-10-06T03:34:04.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.532+0000 7ff16f7fe700 1 -- 172.21.15.55:0/4260074726 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff17c011bf0 con 0x7ff1881597f0 2023-10-06T03:34:04.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.532+0000 7ff16f7fe700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff174066f30 0x7ff1740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:04.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.532+0000 7ff16f7fe700 1 -- 172.21.15.55:0/4260074726 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5292+0+0 (secure 0 0 0) 0x7ff17c085d30 con 0x7ff1881597f0 2023-10-06T03:34:04.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.533+0000 7ff1877fe700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff174066f30 0x7ff1740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:04.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.533+0000 7ff1877fe700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff174066f30 0x7ff1740693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff178007fd0 tx=0x7ff178009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:04.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.537+0000 7ff16f7fe700 1 -- 172.21.15.55:0/4260074726 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff17c0530a0 con 0x7ff1881597f0 2023-10-06T03:34:04.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:04.753+0000 7ff18cb9d700 1 -- 172.21.15.55:0/4260074726 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff18804c3c0 con 0x7ff1881597f0 2023-10-06T03:34:05.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.251+0000 7ff16f7fe700 1 -- 172.21.15.55:0/4260074726 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff17c0566c0 con 0x7ff1881597f0 2023-10-06T03:34:05.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.254+0000 7ff18cb9d700 1 -- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff174066f30 msgr2=0x7ff1740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:05.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.254+0000 7ff18cb9d700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff174066f30 0x7ff1740693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff178007fd0 tx=0x7ff178009040 comp rx=0 tx=0).stop 2023-10-06T03:34:05.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.254+0000 7ff18cb9d700 1 -- 172.21.15.55:0/4260074726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1881597f0 msgr2=0x7ff188155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:05.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.254+0000 7ff18cb9d700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1881597f0 0x7ff188155ba0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff188089340 tx=0x7ff17c017000 comp rx=0 tx=0).stop 2023-10-06T03:34:05.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.254+0000 7ff18cb9d700 1 -- 172.21.15.55:0/4260074726 shutdown_connections 2023-10-06T03:34:05.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.254+0000 7ff18cb9d700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff18815d230 0x7ff1881560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.254+0000 7ff18cb9d700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff174066f30 0x7ff1740693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.254+0000 7ff18cb9d700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff188159420 0x7ff188146150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.254+0000 7ff18cb9d700 1 --2- 172.21.15.55:0/4260074726 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1881597f0 0x7ff188155ba0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.254+0000 7ff18cb9d700 1 -- 172.21.15.55:0/4260074726 >> 172.21.15.55:0/4260074726 conn(0x7ff1880d0660 msgr2=0x7ff188155080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:05.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.254+0000 7ff18cb9d700 1 -- 172.21.15.55:0/4260074726 shutdown_connections 2023-10-06T03:34:05.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.254+0000 7ff18cb9d700 1 -- 172.21.15.55:0/4260074726 wait complete. 2023-10-06T03:34:05.257 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2023-10-06T03:34:05.275 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:34:05.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.460+0000 7fdb1199d700 1 Processor -- start 2023-10-06T03:34:05.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.460+0000 7fdb1199d700 1 -- start start 2023-10-06T03:34:05.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.461+0000 7fdb1199d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdb0c156310 0x7fdb0c0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:05.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.461+0000 7fdb1199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb0c0ae710 0x7fdb0c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:05.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.461+0000 7fdb1199d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0c0ca860 con 0x7fdb0c1566e0 2023-10-06T03:34:05.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.461+0000 7fdb1199d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0c0cab40 con 0x7fdb0c156310 2023-10-06T03:34:05.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.461+0000 7fdb1199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0cae20 con 0x7fdb0c0ae710 2023-10-06T03:34:05.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.462+0000 7fdb1099b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdb0c156310 0x7fdb0c0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:05.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.462+0000 7fdb1099b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdb0c156310 0x7fdb0c0c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53684/0 (socket says 172.21.15.55:53684) 2023-10-06T03:34:05.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.462+0000 7fdb1099b700 1 -- 172.21.15.55:0/1814596345 learned_addr learned my addr 172.21.15.55:0/1814596345 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:05.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.462+0000 7fdb1119c700 1 --2- 172.21.15.55:0/1814596345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb0c0ae710 0x7fdb0c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:05.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.462+0000 7fdb1099b700 1 -- 172.21.15.55:0/1814596345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb0c0ae710 msgr2=0x7fdb0c0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:05.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.462+0000 7fdb1099b700 1 --2- 172.21.15.55:0/1814596345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb0c0ae710 0x7fdb0c0c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.462+0000 7fdb1099b700 1 -- 172.21.15.55:0/1814596345 >> v1:172.21.15.163:6789/0 conn(0x7fdb0c1566e0 legacy=0x7fdb0c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:05.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.462+0000 7fdb1099b700 1 -- 172.21.15.55:0/1814596345 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c0cb100 con 0x7fdb0c156310 2023-10-06T03:34:05.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.462+0000 7fdb1119c700 1 --2- 172.21.15.55:0/1814596345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb0c0ae710 0x7fdb0c0c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:05.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.462+0000 7fdb1099b700 1 --2- 172.21.15.55:0/1814596345 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0c156310 0x7fdb0c0c9dc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000bc50 tx=0x7fdb0000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbb70feb703a683 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:05.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.462+0000 7fdb0b7fe700 1 -- 172.21.15.55:0/1814596345 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0000be60 con 0x7fdb0c156310 2023-10-06T03:34:05.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.463+0000 7fdb0b7fe700 1 -- 172.21.15.55:0/1814596345 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0000de00 con 0x7fdb0c156310 2023-10-06T03:34:05.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.463+0000 7fdb0b7fe700 1 -- 172.21.15.55:0/1814596345 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb000196b0 con 0x7fdb0c156310 2023-10-06T03:34:05.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.463+0000 7fdb1199d700 1 -- 172.21.15.55:0/1814596345 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0c156310 msgr2=0x7fdb0c0c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.463+0000 7fdb1199d700 1 --2- 172.21.15.55:0/1814596345 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0c156310 0x7fdb0c0c9dc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000bc50 tx=0x7fdb0000b4f0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.463+0000 7fdb1199d700 1 -- 172.21.15.55:0/1814596345 shutdown_connections 2023-10-06T03:34:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.463+0000 7fdb1199d700 1 --2- 172.21.15.55:0/1814596345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb0c0ae710 0x7fdb0c0c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.463+0000 7fdb1199d700 1 --2- 172.21.15.55:0/1814596345 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0c156310 0x7fdb0c0c9dc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.463+0000 7fdb1199d700 1 -- 172.21.15.55:0/1814596345 >> 172.21.15.55:0/1814596345 conn(0x7fdb0c0cc1f0 msgr2=0x7fdb0c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.463+0000 7fdb1199d700 1 -- 172.21.15.55:0/1814596345 shutdown_connections 2023-10-06T03:34:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.463+0000 7fdb1199d700 1 -- 172.21.15.55:0/1814596345 wait complete. 2023-10-06T03:34:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.465+0000 7fdb1199d700 1 Processor -- start 2023-10-06T03:34:05.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.465+0000 7fdb1199d700 1 -- start start 2023-10-06T03:34:05.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1199d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0c0ae710 0x7fdb0c0c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:05.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1199d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb0c156310 0x7fdb0c0b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:05.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb0c1566e0 0x7fdb0c169af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:05.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1199d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0c87d0 con 0x7fdb0c0ae710 2023-10-06T03:34:05.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1199d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c16b080 con 0x7fdb0c156310 2023-10-06T03:34:05.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c0b14b0 con 0x7fdb0c1566e0 2023-10-06T03:34:05.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1099b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0c0ae710 0x7fdb0c0c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:05.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1099b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0c0ae710 0x7fdb0c0c8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53700/0 (socket says 172.21.15.55:53700) 2023-10-06T03:34:05.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1099b700 1 -- 172.21.15.55:0/2972166209 learned_addr learned my addr 172.21.15.55:0/2972166209 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:05.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb0bfff700 1 --2- 172.21.15.55:0/2972166209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb0c156310 0x7fdb0c0b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:05.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1119c700 1 --2- 172.21.15.55:0/2972166209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb0c1566e0 0x7fdb0c169af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:05.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1099b700 1 -- 172.21.15.55:0/2972166209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb0c1566e0 msgr2=0x7fdb0c169af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:05.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1099b700 1 --2- 172.21.15.55:0/2972166209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb0c1566e0 0x7fdb0c169af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1099b700 1 -- 172.21.15.55:0/2972166209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb0c156310 msgr2=0x7fdb0c0b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1099b700 1 --2- 172.21.15.55:0/2972166209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb0c156310 0x7fdb0c0b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.466+0000 7fdb1099b700 1 -- 172.21.15.55:0/2972166209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0000b790 con 0x7fdb0c0ae710 2023-10-06T03:34:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.467+0000 7fdb1099b700 1 --2- 172.21.15.55:0/2972166209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0c0ae710 0x7fdb0c0c8290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000b760 tx=0x7fdb0001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.467+0000 7fdb08ff9700 1 -- 172.21.15.55:0/2972166209 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb000108d0 con 0x7fdb0c0ae710 2023-10-06T03:34:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.467+0000 7fdb08ff9700 1 -- 172.21.15.55:0/2972166209 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb00010a30 con 0x7fdb0c0ae710 2023-10-06T03:34:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.467+0000 7fdb08ff9700 1 -- 172.21.15.55:0/2972166209 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00010df0 con 0x7fdb0c0ae710 2023-10-06T03:34:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.467+0000 7fdb1199d700 1 -- 172.21.15.55:0/2972166209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c0b0ef0 con 0x7fdb0c0ae710 2023-10-06T03:34:05.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.467+0000 7fdb1199d700 1 -- 172.21.15.55:0/2972166209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c16ad10 con 0x7fdb0c0ae710 2023-10-06T03:34:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.468+0000 7fdb1199d700 1 -- 172.21.15.55:0/2972166209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0c04c3c0 con 0x7fdb0c0ae710 2023-10-06T03:34:05.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.469+0000 7fdb08ff9700 1 -- 172.21.15.55:0/2972166209 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb00022620 con 0x7fdb0c0ae710 2023-10-06T03:34:05.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.470+0000 7fdb08ff9700 1 --2- 172.21.15.55:0/2972166209 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fdaf0066f30 0x7fdaf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:05.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.471+0000 7fdb08ff9700 1 -- 172.21.15.55:0/2972166209 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fdb00014070 con 0x7fdb0c0ae710 2023-10-06T03:34:05.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.471+0000 7fdb0bfff700 1 --2- 172.21.15.55:0/2972166209 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fdaf0066f30 0x7fdaf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:05.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.472+0000 7fdb0bfff700 1 --2- 172.21.15.55:0/2972166209 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fdaf0066f30 0x7fdaf00693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c06b360 tx=0x7fdaf8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:05.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.475+0000 7fdb08ff9700 1 -- 172.21.15.55:0/2972166209 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb000536d0 con 0x7fdb0c0ae710 2023-10-06T03:34:05.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.760+0000 7fdb1199d700 1 -- 172.21.15.55:0/2972166209 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fdb0c0c9930 con 0x7fdb0c0ae710 2023-10-06T03:34:05.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.761+0000 7fdb08ff9700 1 -- 172.21.15.55:0/2972166209 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fdb00056cf0 con 0x7fdb0c0ae710 2023-10-06T03:34:05.762 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:05.762 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:34:05.762 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2023-10-06T03:34:05.762 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.762 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2023-10-06T03:34:05.762 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9li2RaGRAAgRjtbw3h40vylEPjolzlpw==", 2023-10-06T03:34:05.763 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.763 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:34:05.763 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:34:05.763 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:34:05.763 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.763 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.763 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.763 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.763 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2023-10-06T03:34:05.763 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lXZu7LxAAu3QAtdDxtD/AuuqGG12sDw==", 2023-10-06T03:34:05.763 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.764 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:34:05.764 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:34:05.764 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:34:05.764 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.764 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.764 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.764 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.764 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2023-10-06T03:34:05.764 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lS22fJBAAZi6CfyLyJAFgIkaarad1mA==", 2023-10-06T03:34:05.764 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.764 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:34:05.764 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:34:05.765 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:34:05.765 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.765 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.765 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.765 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.765 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2023-10-06T03:34:05.765 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9ll0RIOhAAGtGdPv4Y2bge7xIv3eu2NQ==", 2023-10-06T03:34:05.765 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.765 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:34:05.765 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:34:05.765 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:34:05.766 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.766 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.766 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.766 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.766 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2023-10-06T03:34:05.766 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDHfx9lmTeaDRAAuycc9i7fPj8xrDkuZCAGVg==", 2023-10-06T03:34:05.766 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.766 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:34:05.766 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:34:05.766 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.766 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.767 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.767 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.767 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2023-10-06T03:34:05.767 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDKfx9lxOdXEhAA9S8tdOvhnmBeEbutqnUsZg==", 2023-10-06T03:34:05.767 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.767 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:34:05.767 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:34:05.767 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.767 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.768 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.768 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.768 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2023-10-06T03:34:05.768 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDNfx9l76I+GRAAjaj0gM7An9BCyGmfQgaDFw==", 2023-10-06T03:34:05.768 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.768 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:34:05.768 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:34:05.768 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.768 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.768 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.768 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.768 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2023-10-06T03:34:05.769 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDQfx9lsPK+HxAAzQCV0EuBenqOkUpiQW9uog==", 2023-10-06T03:34:05.769 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.769 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:34:05.769 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:34:05.769 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.769 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.769 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.769 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.769 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2023-10-06T03:34:05.769 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDWfx9lSHNrMxAAIn7awDmljO2cRtKyDalWnw==", 2023-10-06T03:34:05.769 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.770 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:34:05.770 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:34:05.770 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.770 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.770 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.770 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.770 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2023-10-06T03:34:05.770 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDZfx9lFoWuOBAAhzQOt3YChvuv0MhrspOjPw==", 2023-10-06T03:34:05.770 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.770 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:34:05.770 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:34:05.770 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.771 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.771 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.771 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.771 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2023-10-06T03:34:05.771 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDcfx9lB8iSOhAAwamSgGIXZ8owFE4FdbOOBw==", 2023-10-06T03:34:05.771 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.771 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:34:05.771 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:34:05.771 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.771 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.771 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.771 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.772 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2023-10-06T03:34:05.772 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDgfx9lUb3mABAAJ75UO2UQg2r+l+tDYVn0Dg==", 2023-10-06T03:34:05.772 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.772 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:34:05.772 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:34:05.772 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.772 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.772 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.772 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.772 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2023-10-06T03:34:05.772 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lfNvCBxAA1ZJq4XsWTZHVY0u0NdOqIw==", 2023-10-06T03:34:05.772 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.773 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:34:05.773 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:34:05.773 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:34:05.773 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:34:05.773 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.773 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.773 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.773 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2023-10-06T03:34:05.773 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lnvkoFhAAafxsOmItxmwUr1GJrGYJ6g==", 2023-10-06T03:34:05.773 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.773 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:34:05.774 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:34:05.774 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:34:05.774 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:34:05.774 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.774 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.774 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.774 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2023-10-06T03:34:05.774 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9l+9b4DhAAc5XHdikWbXsEblSE3FGScQ==", 2023-10-06T03:34:05.774 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.774 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:34:05.774 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:34:05.774 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:34:05.775 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:34:05.775 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.775 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.775 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.775 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2023-10-06T03:34:05.775 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9luSXrHBAAg6Ufa1Ep/EOgsQtuOPQgYg==", 2023-10-06T03:34:05.775 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.775 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:34:05.775 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:34:05.775 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:34:05.775 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:34:05.775 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.776 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.776 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.776 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2023-10-06T03:34:05.776 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC+fx9lEb50FxAAt0Oq28gsXtQvol7TV8trgg==", 2023-10-06T03:34:05.776 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.776 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:34:05.776 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:34:05.776 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:34:05.776 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.776 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.776 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.776 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.777 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-mds", 2023-10-06T03:34:05.777 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDnfx9lDjGXHBAA22fJN5W/9alaUlHzJUjLwg==", 2023-10-06T03:34:05.777 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.777 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-mds" 2023-10-06T03:34:05.777 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.777 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.777 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.777 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-mgr", 2023-10-06T03:34:05.777 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDnfx9lJDuXHBAABtcdXaWib0dp8DbQiZXSiw==", 2023-10-06T03:34:05.777 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.777 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-06T03:34:05.777 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.778 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.778 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.778 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-osd", 2023-10-06T03:34:05.778 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDnfx9lfUOXHBAAoxVXy4uLKJAAqSpF4FOp1Q==", 2023-10-06T03:34:05.778 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.778 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-osd" 2023-10-06T03:34:05.778 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.778 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.778 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.778 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-rbd", 2023-10-06T03:34:05.778 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDnfx9lv0uXHBAA+K51W5pYpVIw//Vf0xt0WA==", 2023-10-06T03:34:05.778 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.779 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-06T03:34:05.779 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.779 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.779 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.779 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-06T03:34:05.779 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDnfx9lplaXHBAA4oQG/jVhJGQuM4biV5nZYQ==", 2023-10-06T03:34:05.779 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.779 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-06T03:34:05.779 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.779 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.779 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.780 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.bootstrap-rgw", 2023-10-06T03:34:05.780 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDnfx9li1+XHBAANaEYJ/0YhhArYI/IBkRAzQ==", 2023-10-06T03:34:05.780 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.780 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-06T03:34:05.780 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.780 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.780 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.780 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2023-10-06T03:34:05.780 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lQAANEhAAsNiWPqCNEyUS3IELXSC1Cw==", 2023-10-06T03:34:05.780 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.780 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:34:05.780 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T03:34:05.780 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.781 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.781 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:05.781 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:05.781 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2023-10-06T03:34:05.781 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lCwU2ABAAMijxXd8hwgk3XzcdP4NZyg==", 2023-10-06T03:34:05.781 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:34:05.781 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:34:05.781 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T03:34:05.781 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:34:05.781 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.781 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:05.781 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2023-10-06T03:34:05.782 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:34:05.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.763+0000 7fdb1199d700 1 -- 172.21.15.55:0/2972166209 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fdaf0066f30 msgr2=0x7fdaf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:05.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.763+0000 7fdb1199d700 1 --2- 172.21.15.55:0/2972166209 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fdaf0066f30 0x7fdaf00693e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c06b360 tx=0x7fdaf8009040 comp rx=0 tx=0).stop 2023-10-06T03:34:05.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.763+0000 7fdb1199d700 1 -- 172.21.15.55:0/2972166209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0c0ae710 msgr2=0x7fdb0c0c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:05.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.763+0000 7fdb1199d700 1 --2- 172.21.15.55:0/2972166209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0c0ae710 0x7fdb0c0c8290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000b760 tx=0x7fdb0001b000 comp rx=0 tx=0).stop 2023-10-06T03:34:05.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.763+0000 7fdb1199d700 1 -- 172.21.15.55:0/2972166209 shutdown_connections 2023-10-06T03:34:05.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.763+0000 7fdb1199d700 1 --2- 172.21.15.55:0/2972166209 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fdaf0066f30 0x7fdaf00693e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.763+0000 7fdb1199d700 1 --2- 172.21.15.55:0/2972166209 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdb0c1566e0 0x7fdb0c169af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.763+0000 7fdb1199d700 1 --2- 172.21.15.55:0/2972166209 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdb0c0ae710 0x7fdb0c0c8290 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.763+0000 7fdb1199d700 1 --2- 172.21.15.55:0/2972166209 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdb0c156310 0x7fdb0c0b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.763+0000 7fdb1199d700 1 -- 172.21.15.55:0/2972166209 >> 172.21.15.55:0/2972166209 conn(0x7fdb0c0cc1f0 msgr2=0x7fdb0c151c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:05.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.763+0000 7fdb1199d700 1 -- 172.21.15.55:0/2972166209 shutdown_connections 2023-10-06T03:34:05.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.763+0000 7fdb1199d700 1 -- 172.21.15.55:0/2972166209 wait complete. 2023-10-06T03:34:05.784 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-06T03:34:05.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.970+0000 7f475e99d700 1 Processor -- start 2023-10-06T03:34:05.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.970+0000 7f475e99d700 1 -- start start 2023-10-06T03:34:05.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.972+0000 7f475e99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4758157270 0x7f4758068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:05.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.972+0000 7f475e99d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4758068d40 con 0x7f47580688d0 2023-10-06T03:34:05.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.972+0000 7f475e99d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f475815a980 con 0x7f47580b0bd0 2023-10-06T03:34:05.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.972+0000 7f475e99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475815a050 con 0x7f4758157270 2023-10-06T03:34:05.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.972+0000 7f475d19a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f47580688d0 0x7f47580b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38142/0 (socket says 172.21.15.55:38142) 2023-10-06T03:34:05.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.972+0000 7f475d19a700 1 -- 172.21.15.55:0/2193723409 learned_addr learned my addr 172.21.15.55:0/2193723409 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.972+0000 7f475d99b700 1 --2- 172.21.15.55:0/2193723409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4758157270 0x7f4758068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.972+0000 7f475c999700 1 -- 172.21.15.55:0/2193723409 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964087528 0 0) 0x7f4758068d40 con 0x7f47580688d0 2023-10-06T03:34:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.972+0000 7f475c999700 1 -- 172.21.15.55:0/2193723409 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4744003570 con 0x7f47580688d0 2023-10-06T03:34:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.973+0000 7f475c999700 1 -- 172.21.15.55:0/2193723409 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771284317 0 0) 0x7f475815a980 con 0x7f47580b0bd0 2023-10-06T03:34:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.973+0000 7f475c999700 1 -- 172.21.15.55:0/2193723409 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4758068d40 con 0x7f47580b0bd0 2023-10-06T03:34:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.973+0000 7f475d99b700 1 -- 172.21.15.55:0/2193723409 >> v1:172.21.15.163:6789/0 conn(0x7f47580b0bd0 legacy=0x7f475815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.973+0000 7f475d99b700 1 -- 172.21.15.55:0/2193723409 >> v1:172.21.15.55:6789/0 conn(0x7f47580688d0 legacy=0x7f47580b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.973+0000 7f475d99b700 1 -- 172.21.15.55:0/2193723409 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475815a330 con 0x7f4758157270 2023-10-06T03:34:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.973+0000 7f475d99b700 1 --2- 172.21.15.55:0/2193723409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4758157270 0x7f4758068390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4754009910 tx=0x7f47540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b753ff2b79ddce50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.973+0000 7f475c999700 1 -- 172.21.15.55:0/2193723409 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754011630 con 0x7f4758157270 2023-10-06T03:34:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.973+0000 7f475c999700 1 -- 172.21.15.55:0/2193723409 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4754011790 con 0x7f4758157270 2023-10-06T03:34:05.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.973+0000 7f475c999700 1 -- 172.21.15.55:0/2193723409 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754011b10 con 0x7f4758157270 2023-10-06T03:34:05.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.974+0000 7f475e99d700 1 -- 172.21.15.55:0/2193723409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4758157270 msgr2=0x7f4758068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:05.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.974+0000 7f475e99d700 1 --2- 172.21.15.55:0/2193723409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4758157270 0x7f4758068390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4754009910 tx=0x7f47540091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.974+0000 7f475e99d700 1 -- 172.21.15.55:0/2193723409 shutdown_connections 2023-10-06T03:34:05.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.974+0000 7f475e99d700 1 --2- 172.21.15.55:0/2193723409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4758157270 0x7f4758068390 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.974+0000 7f475e99d700 1 -- 172.21.15.55:0/2193723409 >> 172.21.15.55:0/2193723409 conn(0x7f47580cd310 msgr2=0x7f47580cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:05.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.974+0000 7f475e99d700 1 -- 172.21.15.55:0/2193723409 shutdown_connections 2023-10-06T03:34:05.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.974+0000 7f475e99d700 1 -- 172.21.15.55:0/2193723409 wait complete. 2023-10-06T03:34:05.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.975+0000 7f475e99d700 1 Processor -- start 2023-10-06T03:34:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.976+0000 7f475e99d700 1 -- start start 2023-10-06T03:34:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.976+0000 7f475e99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47580688d0 0x7f475815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.976+0000 7f475e99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47580b0bd0 0x7f475815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475e99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4758157270 0x7f4758154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475e99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4758143360 con 0x7f47580b0bd0 2023-10-06T03:34:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475e99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4758142750 con 0x7f4758157270 2023-10-06T03:34:05.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475e99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4758142a30 con 0x7f47580688d0 2023-10-06T03:34:05.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475d19a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47580b0bd0 0x7f475815b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:05.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475d19a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47580b0bd0 0x7f475815b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53704/0 (socket says 172.21.15.55:53704) 2023-10-06T03:34:05.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475d19a700 1 -- 172.21.15.55:0/1568950967 learned_addr learned my addr 172.21.15.55:0/1568950967 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:05.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475d99b700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47580688d0 0x7f475815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:05.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475e19c700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4758157270 0x7f4758154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:05.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475d19a700 1 -- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47580688d0 msgr2=0x7f475815aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:05.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475d19a700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47580688d0 0x7f475815aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475d19a700 1 -- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4758157270 msgr2=0x7f4758154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:05.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475d19a700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4758157270 0x7f4758154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:05.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475d19a700 1 -- 172.21.15.55:0/1568950967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4754009420 con 0x7f47580b0bd0 2023-10-06T03:34:05.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475e19c700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4758157270 0x7f4758154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:05.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.977+0000 7f475d19a700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47580b0bd0 0x7f475815b3f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f474800c790 tx=0x7f4748007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:05.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.978+0000 7f474dffb700 1 -- 172.21.15.55:0/1568950967 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748005b30 con 0x7f47580b0bd0 2023-10-06T03:34:05.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.978+0000 7f474dffb700 1 -- 172.21.15.55:0/1568950967 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4748005c90 con 0x7f47580b0bd0 2023-10-06T03:34:05.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.978+0000 7f475e99d700 1 -- 172.21.15.55:0/1568950967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4758142d10 con 0x7f47580b0bd0 2023-10-06T03:34:05.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.978+0000 7f474dffb700 1 -- 172.21.15.55:0/1568950967 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748007300 con 0x7f47580b0bd0 2023-10-06T03:34:05.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.978+0000 7f475e99d700 1 -- 172.21.15.55:0/1568950967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4758154bb0 con 0x7f47580b0bd0 2023-10-06T03:34:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.979+0000 7f475e99d700 1 -- 172.21.15.55:0/1568950967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475804c3c0 con 0x7f47580b0bd0 2023-10-06T03:34:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.980+0000 7f474dffb700 1 -- 172.21.15.55:0/1568950967 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f474801b400 con 0x7f47580b0bd0 2023-10-06T03:34:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.980+0000 7f474dffb700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f473c066f30 0x7f473c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.981+0000 7f474dffb700 1 -- 172.21.15.55:0/1568950967 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f4748083f20 con 0x7f47580b0bd0 2023-10-06T03:34:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.981+0000 7f475d99b700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f473c066f30 0x7f473c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:05.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.982+0000 7f475d99b700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f473c066f30 0x7f473c0693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f47540091e0 tx=0x7f4754017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:05.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:05.985+0000 7f474dffb700 1 -- 172.21.15.55:0/1568950967 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4748051410 con 0x7f47580b0bd0 2023-10-06T03:34:06.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.266+0000 7f475e99d700 1 -- 172.21.15.55:0/1568950967 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f475815a610 con 0x7f47580b0bd0 2023-10-06T03:34:06.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.270+0000 7f474dffb700 1 -- 172.21.15.55:0/1568950967 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f47480076f0 con 0x7f47580b0bd0 2023-10-06T03:34:06.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.272+0000 7f475e99d700 1 -- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f473c066f30 msgr2=0x7f473c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:06.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.272+0000 7f475e99d700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f473c066f30 0x7f473c0693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f47540091e0 tx=0x7f4754017000 comp rx=0 tx=0).stop 2023-10-06T03:34:06.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.272+0000 7f475e99d700 1 -- 172.21.15.55:0/1568950967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47580b0bd0 msgr2=0x7f475815b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:06.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.272+0000 7f475e99d700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47580b0bd0 0x7f475815b3f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f474800c790 tx=0x7f4748007ea0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.272+0000 7f475e99d700 1 -- 172.21.15.55:0/1568950967 shutdown_connections 2023-10-06T03:34:06.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.272+0000 7f475e99d700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4758157270 0x7f4758154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.272+0000 7f475e99d700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f473c066f30 0x7f473c0693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.272+0000 7f475e99d700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47580688d0 0x7f475815aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.272+0000 7f475e99d700 1 --2- 172.21.15.55:0/1568950967 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47580b0bd0 0x7f475815b3f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.272+0000 7f475e99d700 1 -- 172.21.15.55:0/1568950967 >> 172.21.15.55:0/1568950967 conn(0x7f47580cd310 msgr2=0x7f47580ac210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:06.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.272+0000 7f475e99d700 1 -- 172.21.15.55:0/1568950967 shutdown_connections 2023-10-06T03:34:06.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.272+0000 7f475e99d700 1 -- 172.21.15.55:0/1568950967 wait complete. 2023-10-06T03:34:06.291 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-06T03:34:06.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.477+0000 7f323bfff700 1 Processor -- start 2023-10-06T03:34:06.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.477+0000 7f323bfff700 1 -- start start 2023-10-06T03:34:06.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.478+0000 7f323bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f323c0ae710 0x7f323c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:06.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.478+0000 7f323bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323c0ca860 con 0x7f323c156310 2023-10-06T03:34:06.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.478+0000 7f323bfff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323c0cab40 con 0x7f323c1566e0 2023-10-06T03:34:06.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.478+0000 7f323bfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323c0cae20 con 0x7f323c0ae710 2023-10-06T03:34:06.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.479+0000 7f323affd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f323c156310 0x7f323c0c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38156/0 (socket says 172.21.15.55:38156) 2023-10-06T03:34:06.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.479+0000 7f323affd700 1 -- 172.21.15.55:0/4127901695 learned_addr learned my addr 172.21.15.55:0/4127901695 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:06.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.479+0000 7f323b7fe700 1 --2- 172.21.15.55:0/4127901695 >> v2:172.21.15.163:3300/0 conn(0x7f323c0ae710 0x7f323c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:06.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.479+0000 7f3239ffb700 1 -- 172.21.15.55:0/4127901695 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35636959 0 0) 0x7f323c0ca860 con 0x7f323c156310 2023-10-06T03:34:06.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.479+0000 7f3239ffb700 1 -- 172.21.15.55:0/4127901695 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3224003570 con 0x7f323c156310 2023-10-06T03:34:06.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.479+0000 7f3239ffb700 1 -- 172.21.15.55:0/4127901695 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1995569025 0 0) 0x7f3224003570 con 0x7f323c156310 2023-10-06T03:34:06.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.479+0000 7f3239ffb700 1 -- 172.21.15.55:0/4127901695 >> v2:172.21.15.163:3300/0 conn(0x7f323c0ae710 msgr2=0x7f323c0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:06.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.479+0000 7f3239ffb700 1 --2- 172.21.15.55:0/4127901695 >> v2:172.21.15.163:3300/0 conn(0x7f323c0ae710 0x7f323c0c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.479+0000 7f3239ffb700 1 -- 172.21.15.55:0/4127901695 >> v1:172.21.15.163:6789/0 conn(0x7f323c1566e0 legacy=0x7f323c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:06.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f3239ffb700 1 -- 172.21.15.55:0/4127901695 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c0cb100 con 0x7f323c156310 2023-10-06T03:34:06.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f3239ffb700 1 -- 172.21.15.55:0/4127901695 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f322c001470 con 0x7f323c156310 2023-10-06T03:34:06.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f3239ffb700 1 -- 172.21.15.55:0/4127901695 >> v1:172.21.15.55:6789/0 conn(0x7f323c156310 legacy=0x7f323c0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:06.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f3239ffb700 1 --2- 172.21.15.55:0/4127901695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224003ab0 0x7f3224003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:06.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f3239ffb700 1 --2- 172.21.15.55:0/4127901695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3224004810 0x7f3224004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:06.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f3239ffb700 1 --2- 172.21.15.55:0/4127901695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3224005210 0x7f322400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:06.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f3239ffb700 1 -- 172.21.15.55:0/4127901695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c0cae20 con 0x7f3224003ab0 2023-10-06T03:34:06.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f3239ffb700 1 -- 172.21.15.55:0/4127901695 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322400c130 con 0x7f3224005210 2023-10-06T03:34:06.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f3239ffb700 1 -- 172.21.15.55:0/4127901695 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322400c2f0 con 0x7f3224004810 2023-10-06T03:34:06.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f323affd700 1 --2- 172.21.15.55:0/4127901695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224003ab0 0x7f3224003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:06.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f323a7fc700 1 --2- 172.21.15.55:0/4127901695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3224004810 0x7f3224004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:06.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f323b7fe700 1 --2- 172.21.15.55:0/4127901695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3224005210 0x7f322400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:06.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f323affd700 1 -- 172.21.15.55:0/4127901695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3224004810 msgr2=0x7f3224004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:06.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f323affd700 1 --2- 172.21.15.55:0/4127901695 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3224004810 0x7f3224004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f323affd700 1 -- 172.21.15.55:0/4127901695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3224005210 msgr2=0x7f322400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:06.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f323affd700 1 --2- 172.21.15.55:0/4127901695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3224005210 0x7f322400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f323affd700 1 -- 172.21.15.55:0/4127901695 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3224003570 con 0x7f3224003ab0 2023-10-06T03:34:06.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f323b7fe700 1 --2- 172.21.15.55:0/4127901695 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3224005210 0x7f322400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:06.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.480+0000 7f323b7fe700 1 -- 172.21.15.55:0/4127901695 reap_dead start 2023-10-06T03:34:06.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.481+0000 7f323affd700 1 --2- 172.21.15.55:0/4127901695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224003ab0 0x7f3224003ec0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f322c00c2a0 tx=0x7f323c0caa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744025d4ede8ae0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:06.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.481+0000 7f3239ffb700 1 -- 172.21.15.55:0/4127901695 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c00cb70 con 0x7f3224003ab0 2023-10-06T03:34:06.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.481+0000 7f3239ffb700 1 -- 172.21.15.55:0/4127901695 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322c00ccd0 con 0x7f3224003ab0 2023-10-06T03:34:06.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.481+0000 7f3239ffb700 1 -- 172.21.15.55:0/4127901695 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c005b70 con 0x7f3224003ab0 2023-10-06T03:34:06.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.481+0000 7f323bfff700 1 -- 172.21.15.55:0/4127901695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224003ab0 msgr2=0x7f3224003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:06.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.481+0000 7f323bfff700 1 --2- 172.21.15.55:0/4127901695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224003ab0 0x7f3224003ec0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f322c00c2a0 tx=0x7f323c0caa30 comp rx=0 tx=0).stop 2023-10-06T03:34:06.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.481+0000 7f323bfff700 1 -- 172.21.15.55:0/4127901695 shutdown_connections 2023-10-06T03:34:06.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.481+0000 7f323bfff700 1 --2- 172.21.15.55:0/4127901695 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224003ab0 0x7f3224003ec0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.481+0000 7f323bfff700 1 -- 172.21.15.55:0/4127901695 >> 172.21.15.55:0/4127901695 conn(0x7f323c0cc1f0 msgr2=0x7f323c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:06.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.482+0000 7f323bfff700 1 -- 172.21.15.55:0/4127901695 shutdown_connections 2023-10-06T03:34:06.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.482+0000 7f323bfff700 1 -- 172.21.15.55:0/4127901695 wait complete. 2023-10-06T03:34:06.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.483+0000 7f323bfff700 1 Processor -- start 2023-10-06T03:34:06.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.483+0000 7f323bfff700 1 -- start start 2023-10-06T03:34:06.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.484+0000 7f323bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224003ab0 0x7f323c148060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:06.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.484+0000 7f323bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f323c147060 0x7f323c0b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:06.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.484+0000 7f323bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f323c1485a0 0x7f323c0b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:06.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.484+0000 7f323bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c14ae90 con 0x7f3224003ab0 2023-10-06T03:34:06.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.484+0000 7f323bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c14b170 con 0x7f323c1485a0 2023-10-06T03:34:06.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.484+0000 7f323bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323c14b450 con 0x7f323c147060 2023-10-06T03:34:06.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.484+0000 7f323affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224003ab0 0x7f323c148060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:06.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.484+0000 7f323affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224003ab0 0x7f323c148060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53726/0 (socket says 172.21.15.55:53726) 2023-10-06T03:34:06.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.484+0000 7f323affd700 1 -- 172.21.15.55:0/1945231388 learned_addr learned my addr 172.21.15.55:0/1945231388 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:06.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.484+0000 7f323a7fc700 1 --2- 172.21.15.55:0/1945231388 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f323c147060 0x7f323c0b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:06.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.484+0000 7f323b7fe700 1 --2- 172.21.15.55:0/1945231388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f323c1485a0 0x7f323c0b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:06.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.485+0000 7f323affd700 1 -- 172.21.15.55:0/1945231388 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f323c147060 msgr2=0x7f323c0b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:06.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.485+0000 7f323affd700 1 --2- 172.21.15.55:0/1945231388 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f323c147060 0x7f323c0b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.485+0000 7f323affd700 1 -- 172.21.15.55:0/1945231388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f323c1485a0 msgr2=0x7f323c0b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.485+0000 7f323affd700 1 --2- 172.21.15.55:0/1945231388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f323c1485a0 0x7f323c0b0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.485+0000 7f323affd700 1 -- 172.21.15.55:0/1945231388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c14b730 con 0x7f3224003ab0 2023-10-06T03:34:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.485+0000 7f323affd700 1 --2- 172.21.15.55:0/1945231388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224003ab0 0x7f323c148060 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f322c001ea0 tx=0x7f322c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.485+0000 7f3222ffd700 1 -- 172.21.15.55:0/1945231388 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c022070 con 0x7f3224003ab0 2023-10-06T03:34:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.485+0000 7f3222ffd700 1 -- 172.21.15.55:0/1945231388 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322c00c8c0 con 0x7f3224003ab0 2023-10-06T03:34:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.485+0000 7f323bfff700 1 -- 172.21.15.55:0/1945231388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323c14aab0 con 0x7f3224003ab0 2023-10-06T03:34:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.485+0000 7f3222ffd700 1 -- 172.21.15.55:0/1945231388 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c00cb70 con 0x7f3224003ab0 2023-10-06T03:34:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.485+0000 7f323bfff700 1 -- 172.21.15.55:0/1945231388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c14cac0 con 0x7f3224003ab0 2023-10-06T03:34:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.486+0000 7f323bfff700 1 -- 172.21.15.55:0/1945231388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f323c14c750 con 0x7f3224003ab0 2023-10-06T03:34:06.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.487+0000 7f3222ffd700 1 -- 172.21.15.55:0/1945231388 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f322c01e430 con 0x7f3224003ab0 2023-10-06T03:34:06.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.488+0000 7f3222ffd700 1 --2- 172.21.15.55:0/1945231388 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3228066f30 0x7f32280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:06.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.488+0000 7f3222ffd700 1 -- 172.21.15.55:0/1945231388 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f322c084e70 con 0x7f3224003ab0 2023-10-06T03:34:06.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.488+0000 7f323a7fc700 1 --2- 172.21.15.55:0/1945231388 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3228066f30 0x7f32280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:06.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.489+0000 7f323a7fc700 1 --2- 172.21.15.55:0/1945231388 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3228066f30 0x7f32280693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f323c168d40 tx=0x7f3230009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:06.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.493+0000 7f3222ffd700 1 -- 172.21.15.55:0/1945231388 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f322c052230 con 0x7f3224003ab0 2023-10-06T03:34:06.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.777+0000 7f323bfff700 1 -- 172.21.15.55:0/1945231388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f323c14b8c0 con 0x7f3224003ab0 2023-10-06T03:34:06.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.785+0000 7f3222ffd700 1 -- 172.21.15.55:0/1945231388 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f322c017020 con 0x7f3224003ab0 2023-10-06T03:34:06.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.787+0000 7f323bfff700 1 -- 172.21.15.55:0/1945231388 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3228066f30 msgr2=0x7f32280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:06.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.787+0000 7f323bfff700 1 --2- 172.21.15.55:0/1945231388 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3228066f30 0x7f32280693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f323c168d40 tx=0x7f3230009040 comp rx=0 tx=0).stop 2023-10-06T03:34:06.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.787+0000 7f323bfff700 1 -- 172.21.15.55:0/1945231388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224003ab0 msgr2=0x7f323c148060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:06.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.787+0000 7f323bfff700 1 --2- 172.21.15.55:0/1945231388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224003ab0 0x7f323c148060 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f322c001ea0 tx=0x7f322c015000 comp rx=0 tx=0).stop 2023-10-06T03:34:06.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.787+0000 7f323bfff700 1 -- 172.21.15.55:0/1945231388 shutdown_connections 2023-10-06T03:34:06.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.787+0000 7f323bfff700 1 --2- 172.21.15.55:0/1945231388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f323c1485a0 0x7f323c0b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.787+0000 7f323bfff700 1 --2- 172.21.15.55:0/1945231388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3224003ab0 0x7f323c148060 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.787+0000 7f323bfff700 1 --2- 172.21.15.55:0/1945231388 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3228066f30 0x7f32280693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.787+0000 7f323bfff700 1 --2- 172.21.15.55:0/1945231388 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f323c147060 0x7f323c0b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.787+0000 7f323bfff700 1 -- 172.21.15.55:0/1945231388 >> 172.21.15.55:0/1945231388 conn(0x7f323c0cc1f0 msgr2=0x7f323c151b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:06.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.787+0000 7f323bfff700 1 -- 172.21.15.55:0/1945231388 shutdown_connections 2023-10-06T03:34:06.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.787+0000 7f323bfff700 1 -- 172.21.15.55:0/1945231388 wait complete. 2023-10-06T03:34:06.806 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-06T03:34:06.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.992+0000 7f53f619d700 1 Processor -- start 2023-10-06T03:34:06.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.992+0000 7f53f619d700 1 -- start start 2023-10-06T03:34:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.993+0000 7f53f619d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53f01566e0 0x7f53f00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.993+0000 7f53f619d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f53f00cb900 0x7f53f00ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.993+0000 7f53f619d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f00ca860 con 0x7f53f0156310 2023-10-06T03:34:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.993+0000 7f53f619d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f00cab40 con 0x7f53f00cb900 2023-10-06T03:34:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.993+0000 7f53f619d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f00cae20 con 0x7f53f01566e0 2023-10-06T03:34:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.993+0000 7f53f599c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f53f00cb900 0x7f53f00ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.994+0000 7f53f519b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f53f0156310 0x7f53f00c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38164/0 (socket says 172.21.15.55:38164) 2023-10-06T03:34:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.994+0000 7f53f519b700 1 -- 172.21.15.55:0/3625202561 learned_addr learned my addr 172.21.15.55:0/3625202561 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:06.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.994+0000 7f53f499a700 1 --2- 172.21.15.55:0/3625202561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53f01566e0 0x7f53f00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:06.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.994+0000 7f53e7fff700 1 -- 172.21.15.55:0/3625202561 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239624846 0 0) 0x7f53f00ca860 con 0x7f53f0156310 2023-10-06T03:34:06.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.994+0000 7f53e7fff700 1 -- 172.21.15.55:0/3625202561 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53dc003570 con 0x7f53f0156310 2023-10-06T03:34:06.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.994+0000 7f53f599c700 1 -- 172.21.15.55:0/3625202561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53f01566e0 msgr2=0x7f53f00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:06.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.994+0000 7f53f599c700 1 --2- 172.21.15.55:0/3625202561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53f01566e0 0x7f53f00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:06.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.994+0000 7f53f599c700 1 -- 172.21.15.55:0/3625202561 >> v1:172.21.15.55:6789/0 conn(0x7f53f0156310 legacy=0x7f53f00c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:06.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.994+0000 7f53f599c700 1 -- 172.21.15.55:0/3625202561 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f00cb100 con 0x7f53f00cb900 2023-10-06T03:34:06.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.994+0000 7f53f599c700 1 --2- 172.21.15.55:0/3625202561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f00cb900 0x7f53f00ae540 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009e60 tx=0x7f53e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746451377a90a02d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:06.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.995+0000 7f53e7fff700 1 -- 172.21.15.55:0/3625202561 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e800e070 con 0x7f53f00cb900 2023-10-06T03:34:06.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.995+0000 7f53e7fff700 1 -- 172.21.15.55:0/3625202561 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e800bab0 con 0x7f53f00cb900 2023-10-06T03:34:06.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.995+0000 7f53e7fff700 1 -- 172.21.15.55:0/3625202561 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e800be30 con 0x7f53f00cb900 2023-10-06T03:34:07.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.995+0000 7f53f619d700 1 -- 172.21.15.55:0/3625202561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f00cb900 msgr2=0x7f53f00ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:07.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.995+0000 7f53f619d700 1 --2- 172.21.15.55:0/3625202561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f00cb900 0x7f53f00ae540 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009e60 tx=0x7f53e8009780 comp rx=0 tx=0).stop 2023-10-06T03:34:07.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.995+0000 7f53f619d700 1 -- 172.21.15.55:0/3625202561 shutdown_connections 2023-10-06T03:34:07.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.995+0000 7f53f619d700 1 --2- 172.21.15.55:0/3625202561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f00cb900 0x7f53f00ae540 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.995+0000 7f53f619d700 1 --2- 172.21.15.55:0/3625202561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53f01566e0 0x7f53f00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.995+0000 7f53f619d700 1 -- 172.21.15.55:0/3625202561 >> 172.21.15.55:0/3625202561 conn(0x7f53f00cc1f0 msgr2=0x7f53f00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:07.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.995+0000 7f53f619d700 1 -- 172.21.15.55:0/3625202561 shutdown_connections 2023-10-06T03:34:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.995+0000 7f53f619d700 1 -- 172.21.15.55:0/3625202561 wait complete. 2023-10-06T03:34:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.997+0000 7f53f619d700 1 Processor -- start 2023-10-06T03:34:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.997+0000 7f53f619d700 1 -- start start 2023-10-06T03:34:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.998+0000 7f53f619d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53f00cb900 0x7f53f014fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.998+0000 7f53f619d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53f0156310 0x7f53f015b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.998+0000 7f53f619d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f01566e0 0x7f53f015b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.998+0000 7f53f619d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f0169550 con 0x7f53f01566e0 2023-10-06T03:34:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.998+0000 7f53f619d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f014ffd0 con 0x7f53f00cb900 2023-10-06T03:34:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.998+0000 7f53f619d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f00afd60 con 0x7f53f0156310 2023-10-06T03:34:07.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.998+0000 7f53f599c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f01566e0 0x7f53f015b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:07.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.998+0000 7f53f599c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f01566e0 0x7f53f015b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53740/0 (socket says 172.21.15.55:53740) 2023-10-06T03:34:07.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.998+0000 7f53f599c700 1 -- 172.21.15.55:0/1798361527 learned_addr learned my addr 172.21.15.55:0/1798361527 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:07.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.998+0000 7f53f519b700 1 --2- 172.21.15.55:0/1798361527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53f00cb900 0x7f53f014fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:07.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.998+0000 7f53f499a700 1 --2- 172.21.15.55:0/1798361527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53f0156310 0x7f53f015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:07.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.999+0000 7f53f599c700 1 -- 172.21.15.55:0/1798361527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53f0156310 msgr2=0x7f53f015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:07.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.999+0000 7f53f599c700 1 --2- 172.21.15.55:0/1798361527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53f0156310 0x7f53f015b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.999+0000 7f53f599c700 1 -- 172.21.15.55:0/1798361527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53f00cb900 msgr2=0x7f53f014fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:07.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.999+0000 7f53f599c700 1 --2- 172.21.15.55:0/1798361527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53f00cb900 0x7f53f014fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.999+0000 7f53f599c700 1 -- 172.21.15.55:0/1798361527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e8009a20 con 0x7f53f01566e0 2023-10-06T03:34:07.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.999+0000 7f53f599c700 1 --2- 172.21.15.55:0/1798361527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f01566e0 0x7f53f015b7b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f53f006fe40 tx=0x7f53e8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:07.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.999+0000 7f53e57fa700 1 -- 172.21.15.55:0/1798361527 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8027470 con 0x7f53f01566e0 2023-10-06T03:34:07.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.999+0000 7f53e57fa700 1 -- 172.21.15.55:0/1798361527 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e8027610 con 0x7f53f01566e0 2023-10-06T03:34:07.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.999+0000 7f53f619d700 1 -- 172.21.15.55:0/1798361527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f00b0040 con 0x7f53f01566e0 2023-10-06T03:34:07.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.999+0000 7f53e57fa700 1 -- 172.21.15.55:0/1798361527 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e80279d0 con 0x7f53f01566e0 2023-10-06T03:34:07.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:06.999+0000 7f53f619d700 1 -- 172.21.15.55:0/1798361527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f00b0600 con 0x7f53f01566e0 2023-10-06T03:34:07.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.001+0000 7f53f619d700 1 -- 172.21.15.55:0/1798361527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f01696a0 con 0x7f53f01566e0 2023-10-06T03:34:07.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.001+0000 7f53e57fa700 1 -- 172.21.15.55:0/1798361527 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f53e8024070 con 0x7f53f01566e0 2023-10-06T03:34:07.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.002+0000 7f53e57fa700 1 --2- 172.21.15.55:0/1798361527 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f53d4066f30 0x7f53d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:07.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.002+0000 7f53e57fa700 1 -- 172.21.15.55:0/1798361527 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f53e8086e00 con 0x7f53f01566e0 2023-10-06T03:34:07.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.002+0000 7f53f519b700 1 --2- 172.21.15.55:0/1798361527 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f53d4066f30 0x7f53d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:07.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.003+0000 7f53f519b700 1 --2- 172.21.15.55:0/1798361527 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f53d4066f30 0x7f53d40693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0058d0 tx=0x7f53ec009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:07.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.007+0000 7f53e57fa700 1 -- 172.21.15.55:0/1798361527 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f53f01696a0 con 0x7f53f01566e0 2023-10-06T03:34:07.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.291+0000 7f53f619d700 1 -- 172.21.15.55:0/1798361527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f53f01696a0 con 0x7f53f01566e0 2023-10-06T03:34:07.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.294+0000 7f53e57fa700 1 -- 172.21.15.55:0/1798361527 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f53f01696a0 con 0x7f53f01566e0 2023-10-06T03:34:07.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.296+0000 7f53f619d700 1 -- 172.21.15.55:0/1798361527 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f53d4066f30 msgr2=0x7f53d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:07.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.296+0000 7f53f619d700 1 --2- 172.21.15.55:0/1798361527 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f53d4066f30 0x7f53d40693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0058d0 tx=0x7f53ec009040 comp rx=0 tx=0).stop 2023-10-06T03:34:07.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.296+0000 7f53f619d700 1 -- 172.21.15.55:0/1798361527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f01566e0 msgr2=0x7f53f015b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:07.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.296+0000 7f53f619d700 1 --2- 172.21.15.55:0/1798361527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f01566e0 0x7f53f015b7b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f53f006fe40 tx=0x7f53e8019000 comp rx=0 tx=0).stop 2023-10-06T03:34:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.297+0000 7f53f619d700 1 -- 172.21.15.55:0/1798361527 shutdown_connections 2023-10-06T03:34:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.297+0000 7f53f619d700 1 --2- 172.21.15.55:0/1798361527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f53f01566e0 0x7f53f015b7b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.297+0000 7f53f619d700 1 --2- 172.21.15.55:0/1798361527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f53f00cb900 0x7f53f014fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.297+0000 7f53f619d700 1 --2- 172.21.15.55:0/1798361527 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f53d4066f30 0x7f53d40693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.297+0000 7f53f619d700 1 --2- 172.21.15.55:0/1798361527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f53f0156310 0x7f53f015b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.297+0000 7f53f619d700 1 -- 172.21.15.55:0/1798361527 >> 172.21.15.55:0/1798361527 conn(0x7f53f00cc1f0 msgr2=0x7f53f00c0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.297+0000 7f53f619d700 1 -- 172.21.15.55:0/1798361527 shutdown_connections 2023-10-06T03:34:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.297+0000 7f53f619d700 1 -- 172.21.15.55:0/1798361527 wait complete. 2023-10-06T03:34:07.316 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-06T03:34:07.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.502+0000 7f940f59e700 1 Processor -- start 2023-10-06T03:34:07.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.502+0000 7f940f59e700 1 -- start start 2023-10-06T03:34:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.503+0000 7f940f59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9410150550 0x7f9410157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.503+0000 7f940f59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941006e2a0 con 0x7f94100c7410 2023-10-06T03:34:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.503+0000 7f940f59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941006b2e0 con 0x7f9410150180 2023-10-06T03:34:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.503+0000 7f940f59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94100c78e0 con 0x7f9410150550 2023-10-06T03:34:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.504+0000 7f940ed9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f94100c7410 0x7f9410069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38170/0 (socket says 172.21.15.55:38170) 2023-10-06T03:34:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.504+0000 7f940ed9d700 1 -- 172.21.15.55:0/3934124094 learned_addr learned my addr 172.21.15.55:0/3934124094 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:07.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.504+0000 7f940dd9b700 1 --2- 172.21.15.55:0/3934124094 >> v2:172.21.15.163:3300/0 conn(0x7f9410150550 0x7f9410157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:07.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.504+0000 7f940d59a700 1 -- 172.21.15.55:0/3934124094 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802698216 0 0) 0x7f941006e2a0 con 0x7f94100c7410 2023-10-06T03:34:07.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.504+0000 7f940d59a700 1 -- 172.21.15.55:0/3934124094 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93f8003570 con 0x7f94100c7410 2023-10-06T03:34:07.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.504+0000 7f940dd9b700 1 -- 172.21.15.55:0/3934124094 >> v1:172.21.15.163:6789/0 conn(0x7f9410150180 legacy=0x7f94100c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:07.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.504+0000 7f940dd9b700 1 -- 172.21.15.55:0/3934124094 >> v1:172.21.15.55:6789/0 conn(0x7f94100c7410 legacy=0x7f9410069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:07.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.504+0000 7f940dd9b700 1 -- 172.21.15.55:0/3934124094 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94100c7bc0 con 0x7f9410150550 2023-10-06T03:34:07.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.505+0000 7f940dd9b700 1 --2- 172.21.15.55:0/3934124094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9410150550 0x7f9410157770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f94040098e0 tx=0x7f94040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c37531f77878433 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.505+0000 7f940d59a700 1 -- 172.21.15.55:0/3934124094 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940400b640 con 0x7f9410150550 2023-10-06T03:34:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.505+0000 7f940d59a700 1 -- 172.21.15.55:0/3934124094 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940400b7a0 con 0x7f9410150550 2023-10-06T03:34:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.505+0000 7f940d59a700 1 -- 172.21.15.55:0/3934124094 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940400bb40 con 0x7f9410150550 2023-10-06T03:34:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.505+0000 7f940f59e700 1 -- 172.21.15.55:0/3934124094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9410150550 msgr2=0x7f9410157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.505+0000 7f940f59e700 1 --2- 172.21.15.55:0/3934124094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9410150550 0x7f9410157770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f94040098e0 tx=0x7f94040091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.505+0000 7f940f59e700 1 -- 172.21.15.55:0/3934124094 shutdown_connections 2023-10-06T03:34:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.505+0000 7f940f59e700 1 --2- 172.21.15.55:0/3934124094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9410150550 0x7f9410157770 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.505+0000 7f940f59e700 1 -- 172.21.15.55:0/3934124094 >> 172.21.15.55:0/3934124094 conn(0x7f94100cc1f0 msgr2=0x7f94100cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.506+0000 7f940f59e700 1 -- 172.21.15.55:0/3934124094 shutdown_connections 2023-10-06T03:34:07.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.506+0000 7f940f59e700 1 -- 172.21.15.55:0/3934124094 wait complete. 2023-10-06T03:34:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.507+0000 7f940f59e700 1 Processor -- start 2023-10-06T03:34:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.507+0000 7f940f59e700 1 -- start start 2023-10-06T03:34:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.508+0000 7f940f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94100c7410 0x7f94100b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.508+0000 7f940f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9410150180 0x7f9410169190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.508+0000 7f940f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9410150550 0x7f94100aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.508+0000 7f940f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94100b0f10 con 0x7f9410150550 2023-10-06T03:34:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.508+0000 7f940f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94100b0540 con 0x7f9410150180 2023-10-06T03:34:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.508+0000 7f940f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94101696d0 con 0x7f94100c7410 2023-10-06T03:34:07.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.508+0000 7f940ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9410150550 0x7f94100aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.508+0000 7f940ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9410150550 0x7f94100aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53746/0 (socket says 172.21.15.55:53746) 2023-10-06T03:34:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.508+0000 7f940ed9d700 1 -- 172.21.15.55:0/2771230248 learned_addr learned my addr 172.21.15.55:0/2771230248 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.509+0000 7f940e59c700 1 --2- 172.21.15.55:0/2771230248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94100c7410 0x7f94100b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.509+0000 7f940dd9b700 1 --2- 172.21.15.55:0/2771230248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9410150180 0x7f9410169190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.509+0000 7f940ed9d700 1 -- 172.21.15.55:0/2771230248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94100c7410 msgr2=0x7f94100b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.509+0000 7f940ed9d700 1 --2- 172.21.15.55:0/2771230248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94100c7410 0x7f94100b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.509+0000 7f940ed9d700 1 -- 172.21.15.55:0/2771230248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9410150180 msgr2=0x7f9410169190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.509+0000 7f940ed9d700 1 --2- 172.21.15.55:0/2771230248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9410150180 0x7f9410169190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.509+0000 7f940ed9d700 1 -- 172.21.15.55:0/2771230248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9404009420 con 0x7f9410150550 2023-10-06T03:34:07.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.509+0000 7f940ed9d700 1 --2- 172.21.15.55:0/2771230248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9410150550 0x7f94100aef90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f940800cc70 tx=0x7f9408001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:07.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.509+0000 7f93fe7fc700 1 -- 172.21.15.55:0/2771230248 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940800ce80 con 0x7f9410150550 2023-10-06T03:34:07.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.509+0000 7f93fe7fc700 1 -- 172.21.15.55:0/2771230248 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940800ab30 con 0x7f9410150550 2023-10-06T03:34:07.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.509+0000 7f93fe7fc700 1 -- 172.21.15.55:0/2771230248 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94080070f0 con 0x7f9410150550 2023-10-06T03:34:07.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.509+0000 7f940f59e700 1 -- 172.21.15.55:0/2771230248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94100af930 con 0x7f9410150550 2023-10-06T03:34:07.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.509+0000 7f940f59e700 1 -- 172.21.15.55:0/2771230248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94100afef0 con 0x7f9410150550 2023-10-06T03:34:07.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.510+0000 7f940f59e700 1 -- 172.21.15.55:0/2771230248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94100b06d0 con 0x7f9410150550 2023-10-06T03:34:07.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.511+0000 7f93fe7fc700 1 -- 172.21.15.55:0/2771230248 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9408007520 con 0x7f9410150550 2023-10-06T03:34:07.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.512+0000 7f93fe7fc700 1 --2- 172.21.15.55:0/2771230248 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f93f0066f30 0x7f93f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:07.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.512+0000 7f93fe7fc700 1 -- 172.21.15.55:0/2771230248 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f94080848c0 con 0x7f9410150550 2023-10-06T03:34:07.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.513+0000 7f940e59c700 1 --2- 172.21.15.55:0/2771230248 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f93f0066f30 0x7f93f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:07.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.513+0000 7f940e59c700 1 --2- 172.21.15.55:0/2771230248 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f93f0066f30 0x7f93f00693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9400003860 tx=0x7f9400009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:07.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.516+0000 7f93fe7fc700 1 -- 172.21.15.55:0/2771230248 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9408051c70 con 0x7f9410150550 2023-10-06T03:34:07.800 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.798+0000 7f940f59e700 1 -- 172.21.15.55:0/2771230248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f941004c3c0 con 0x7f9410150550 2023-10-06T03:34:07.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.806+0000 7f93fe7fc700 1 -- 172.21.15.55:0/2771230248 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9408055290 con 0x7f9410150550 2023-10-06T03:34:07.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.809+0000 7f940f59e700 1 -- 172.21.15.55:0/2771230248 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f93f0066f30 msgr2=0x7f93f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:07.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.809+0000 7f940f59e700 1 --2- 172.21.15.55:0/2771230248 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f93f0066f30 0x7f93f00693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9400003860 tx=0x7f9400009040 comp rx=0 tx=0).stop 2023-10-06T03:34:07.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.809+0000 7f940f59e700 1 -- 172.21.15.55:0/2771230248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9410150550 msgr2=0x7f94100aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:07.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.809+0000 7f940f59e700 1 --2- 172.21.15.55:0/2771230248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9410150550 0x7f94100aef90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f940800cc70 tx=0x7f9408001aa0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.809+0000 7f940f59e700 1 -- 172.21.15.55:0/2771230248 shutdown_connections 2023-10-06T03:34:07.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.809+0000 7f940f59e700 1 --2- 172.21.15.55:0/2771230248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9410150550 0x7f94100aef90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.809+0000 7f940f59e700 1 --2- 172.21.15.55:0/2771230248 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f93f0066f30 0x7f93f00693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.809+0000 7f940f59e700 1 --2- 172.21.15.55:0/2771230248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f94100c7410 0x7f94100b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.809+0000 7f940f59e700 1 --2- 172.21.15.55:0/2771230248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9410150180 0x7f9410169190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:07.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.809+0000 7f940f59e700 1 -- 172.21.15.55:0/2771230248 >> 172.21.15.55:0/2771230248 conn(0x7f94100cc1f0 msgr2=0x7f9410158790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:07.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.809+0000 7f940f59e700 1 -- 172.21.15.55:0/2771230248 shutdown_connections 2023-10-06T03:34:07.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:07.809+0000 7f940f59e700 1 -- 172.21.15.55:0/2771230248 wait complete. 2023-10-06T03:34:07.827 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-06T03:34:08.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.014+0000 7ff51c9fe700 1 Processor -- start 2023-10-06T03:34:08.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.014+0000 7ff51c9fe700 1 -- start start 2023-10-06T03:34:08.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.015+0000 7ff51c9fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff518068aa0 0x7ff5180b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:08.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.016+0000 7ff51c9fe700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51815a980 con 0x7ff518157270 2023-10-06T03:34:08.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.016+0000 7ff51c9fe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51815a050 con 0x7ff5180b0a00 2023-10-06T03:34:08.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.016+0000 7ff51c9fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51815a330 con 0x7ff518068aa0 2023-10-06T03:34:08.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.016+0000 7ff516d9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff518157270 0x7ff518068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38180/0 (socket says 172.21.15.55:38180) 2023-10-06T03:34:08.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.016+0000 7ff516d9d700 1 -- 172.21.15.55:0/1427765035 learned_addr learned my addr 172.21.15.55:0/1427765035 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:08.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.016+0000 7ff51659c700 1 --2- 172.21.15.55:0/1427765035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff518068aa0 0x7ff5180b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:08.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.016+0000 7ff515d9b700 1 -- 172.21.15.55:0/1427765035 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237780291 0 0) 0x7ff51815a980 con 0x7ff518157270 2023-10-06T03:34:08.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.016+0000 7ff515d9b700 1 -- 172.21.15.55:0/1427765035 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff500003570 con 0x7ff518157270 2023-10-06T03:34:08.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.017+0000 7ff515d9b700 1 -- 172.21.15.55:0/1427765035 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4200027073 0 0) 0x7ff51815a050 con 0x7ff5180b0a00 2023-10-06T03:34:08.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.017+0000 7ff515d9b700 1 -- 172.21.15.55:0/1427765035 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff51815a980 con 0x7ff5180b0a00 2023-10-06T03:34:08.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.017+0000 7ff51659c700 1 -- 172.21.15.55:0/1427765035 >> v1:172.21.15.163:6789/0 conn(0x7ff5180b0a00 legacy=0x7ff51815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:08.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.017+0000 7ff515d9b700 1 -- 172.21.15.55:0/1427765035 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1705162581 0 0) 0x7ff500003570 con 0x7ff518157270 2023-10-06T03:34:08.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.017+0000 7ff51659c700 1 -- 172.21.15.55:0/1427765035 >> v1:172.21.15.55:6789/0 conn(0x7ff518157270 legacy=0x7ff518068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:08.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.017+0000 7ff51659c700 1 -- 172.21.15.55:0/1427765035 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51815a610 con 0x7ff518068aa0 2023-10-06T03:34:08.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.017+0000 7ff51659c700 1 --2- 172.21.15.55:0/1427765035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff518068aa0 0x7ff5180b04c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009910 tx=0x7ff50c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f2c8a146d891b1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:08.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.017+0000 7ff515d9b700 1 -- 172.21.15.55:0/1427765035 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508001470 con 0x7ff518068aa0 2023-10-06T03:34:08.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.017+0000 7ff515d9b700 1 -- 172.21.15.55:0/1427765035 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c011420 con 0x7ff518068aa0 2023-10-06T03:34:08.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.017+0000 7ff515d9b700 1 -- 172.21.15.55:0/1427765035 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0117c0 con 0x7ff518068aa0 2023-10-06T03:34:08.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.018+0000 7ff51c9fe700 1 -- 172.21.15.55:0/1427765035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff518068aa0 msgr2=0x7ff5180b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:08.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.018+0000 7ff51c9fe700 1 --2- 172.21.15.55:0/1427765035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff518068aa0 0x7ff5180b04c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009910 tx=0x7ff50c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.018+0000 7ff51c9fe700 1 -- 172.21.15.55:0/1427765035 shutdown_connections 2023-10-06T03:34:08.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.018+0000 7ff51c9fe700 1 --2- 172.21.15.55:0/1427765035 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff518068aa0 0x7ff5180b04c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.018+0000 7ff51c9fe700 1 -- 172.21.15.55:0/1427765035 >> 172.21.15.55:0/1427765035 conn(0x7ff5180cd310 msgr2=0x7ff5180cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:08.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.018+0000 7ff51c9fe700 1 -- 172.21.15.55:0/1427765035 shutdown_connections 2023-10-06T03:34:08.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.018+0000 7ff51c9fe700 1 -- 172.21.15.55:0/1427765035 wait complete. 2023-10-06T03:34:08.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.020+0000 7ff51c9fe700 1 Processor -- start 2023-10-06T03:34:08.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.020+0000 7ff51c9fe700 1 -- start start 2023-10-06T03:34:08.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff51c9fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff518068aa0 0x7ff518155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:08.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff51c9fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5180b0a00 0x7ff51814ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:08.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff51c9fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff518157270 0x7ff5180c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:08.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff51c9fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff518153e80 con 0x7ff518068aa0 2023-10-06T03:34:08.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff51c9fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff518154160 con 0x7ff518157270 2023-10-06T03:34:08.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff51c9fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff518154440 con 0x7ff5180b0a00 2023-10-06T03:34:08.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff516d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff518068aa0 0x7ff518155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:08.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff516d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff518068aa0 0x7ff518155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53756/0 (socket says 172.21.15.55:53756) 2023-10-06T03:34:08.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff516d9d700 1 -- 172.21.15.55:0/49739862 learned_addr learned my addr 172.21.15.55:0/49739862 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:08.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff516d9d700 1 -- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5180b0a00 msgr2=0x7ff51814ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:08.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff51659c700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5180b0a00 0x7ff51814ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:08.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff51759e700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff518157270 0x7ff5180c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:08.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff516d9d700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5180b0a00 0x7ff51814ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff516d9d700 1 -- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff518157270 msgr2=0x7ff5180c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:08.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff516d9d700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff518157270 0x7ff5180c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff516d9d700 1 -- 172.21.15.55:0/49739862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c009420 con 0x7ff518068aa0 2023-10-06T03:34:08.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff51759e700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff518157270 0x7ff5180c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:08.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.021+0000 7ff51659c700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5180b0a00 0x7ff51814ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:08.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.022+0000 7ff516d9d700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff518068aa0 0x7ff518155950 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff50800c810 tx=0x7ff508001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:08.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.022+0000 7ff4feffd700 1 -- 172.21.15.55:0/49739862 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508007360 con 0x7ff518068aa0 2023-10-06T03:34:08.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.022+0000 7ff4feffd700 1 -- 172.21.15.55:0/49739862 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff508007500 con 0x7ff518068aa0 2023-10-06T03:34:08.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.022+0000 7ff4feffd700 1 -- 172.21.15.55:0/49739862 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff508004a80 con 0x7ff518068aa0 2023-10-06T03:34:08.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.022+0000 7ff51c9fe700 1 -- 172.21.15.55:0/49739862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff518154720 con 0x7ff518068aa0 2023-10-06T03:34:08.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.022+0000 7ff51c9fe700 1 -- 172.21.15.55:0/49739862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff518155e90 con 0x7ff518068aa0 2023-10-06T03:34:08.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.023+0000 7ff51c9fe700 1 -- 172.21.15.55:0/49739862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51816a4f0 con 0x7ff518068aa0 2023-10-06T03:34:08.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.024+0000 7ff4feffd700 1 -- 172.21.15.55:0/49739862 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff50801c3b0 con 0x7ff518068aa0 2023-10-06T03:34:08.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.026+0000 7ff4feffd700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff504066fb0 0x7ff504069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:08.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.026+0000 7ff4feffd700 1 -- 172.21.15.55:0/49739862 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff508083ab0 con 0x7ff518068aa0 2023-10-06T03:34:08.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.026+0000 7ff51659c700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff504066fb0 0x7ff504069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:08.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.027+0000 7ff51659c700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff504066fb0 0x7ff504069460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff518156930 tx=0x7ff50c016000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:08.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.030+0000 7ff4feffd700 1 -- 172.21.15.55:0/49739862 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff50804c5a0 con 0x7ff518068aa0 2023-10-06T03:34:08.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.312+0000 7ff51c9fe700 1 -- 172.21.15.55:0/49739862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7ff51804c3c0 con 0x7ff518068aa0 2023-10-06T03:34:08.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.315+0000 7ff4feffd700 1 -- 172.21.15.55:0/49739862 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff5080180a0 con 0x7ff518068aa0 2023-10-06T03:34:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.318+0000 7ff51c9fe700 1 -- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff504066fb0 msgr2=0x7ff504069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.318+0000 7ff51c9fe700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff504066fb0 0x7ff504069460 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff518156930 tx=0x7ff50c016000 comp rx=0 tx=0).stop 2023-10-06T03:34:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.318+0000 7ff51c9fe700 1 -- 172.21.15.55:0/49739862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff518068aa0 msgr2=0x7ff518155950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.318+0000 7ff51c9fe700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff518068aa0 0x7ff518155950 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff50800c810 tx=0x7ff508001a10 comp rx=0 tx=0).stop 2023-10-06T03:34:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.318+0000 7ff51c9fe700 1 -- 172.21.15.55:0/49739862 shutdown_connections 2023-10-06T03:34:08.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.318+0000 7ff51c9fe700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff518157270 0x7ff5180c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.318+0000 7ff51c9fe700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff518068aa0 0x7ff518155950 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.318+0000 7ff51c9fe700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff504066fb0 0x7ff504069460 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.318+0000 7ff51c9fe700 1 --2- 172.21.15.55:0/49739862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5180b0a00 0x7ff51814ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.318+0000 7ff51c9fe700 1 -- 172.21.15.55:0/49739862 >> 172.21.15.55:0/49739862 conn(0x7ff5180cd310 msgr2=0x7ff5180aa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.318+0000 7ff51c9fe700 1 -- 172.21.15.55:0/49739862 shutdown_connections 2023-10-06T03:34:08.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.318+0000 7ff51c9fe700 1 -- 172.21.15.55:0/49739862 wait complete. 2023-10-06T03:34:08.337 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-06T03:34:08.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.523+0000 7fa6feb9d700 1 Processor -- start 2023-10-06T03:34:08.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.524+0000 7fa6feb9d700 1 -- start start 2023-10-06T03:34:08.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.525+0000 7fa6feb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f8142ea0 0x7fa6f80b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:08.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.525+0000 7fa6feb9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa6f806c0f0 0x7fa6f806a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:08.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.525+0000 7fa6feb9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f80b40f0 con 0x7fa6f80ad870 2023-10-06T03:34:08.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.525+0000 7fa6feb9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f8069640 con 0x7fa6f806c0f0 2023-10-06T03:34:08.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.525+0000 7fa6feb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8069920 con 0x7fa6f8142ea0 2023-10-06T03:34:08.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.525+0000 7fa6fd39a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa6f80ad870 0x7fa6f806b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38184/0 (socket says 172.21.15.55:38184) 2023-10-06T03:34:08.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.525+0000 7fa6fd39a700 1 -- 172.21.15.55:0/3116418169 learned_addr learned my addr 172.21.15.55:0/3116418169 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:08.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.525+0000 7fa6fe39c700 1 --2- 172.21.15.55:0/3116418169 >> v2:172.21.15.163:3300/0 conn(0x7fa6f806c0f0 0x7fa6f806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:08.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.526+0000 7fa6fdb9b700 1 --2- 172.21.15.55:0/3116418169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f8142ea0 0x7fa6f80b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:08.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.526+0000 7fa6fcb99700 1 -- 172.21.15.55:0/3116418169 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188793940 0 0) 0x7fa6f80b40f0 con 0x7fa6f80ad870 2023-10-06T03:34:08.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.526+0000 7fa6fcb99700 1 -- 172.21.15.55:0/3116418169 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e4003570 con 0x7fa6f80ad870 2023-10-06T03:34:08.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.526+0000 7fa6fe39c700 1 -- 172.21.15.55:0/3116418169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f8142ea0 msgr2=0x7fa6f80b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:08.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.526+0000 7fa6fe39c700 1 --2- 172.21.15.55:0/3116418169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f8142ea0 0x7fa6f80b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.526+0000 7fa6fe39c700 1 -- 172.21.15.55:0/3116418169 >> v1:172.21.15.55:6789/0 conn(0x7fa6f80ad870 legacy=0x7fa6f806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:08.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.526+0000 7fa6fe39c700 1 -- 172.21.15.55:0/3116418169 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f8069c00 con 0x7fa6f806c0f0 2023-10-06T03:34:08.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.526+0000 7fa6fe39c700 1 --2- 172.21.15.55:0/3116418169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f806c0f0 0x7fa6f806a120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009e60 tx=0x7fa6f0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a3057b5cedbd920 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:08.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.527+0000 7fa6fcb99700 1 -- 172.21.15.55:0/3116418169 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000e070 con 0x7fa6f806c0f0 2023-10-06T03:34:08.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.527+0000 7fa6fcb99700 1 -- 172.21.15.55:0/3116418169 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f000bab0 con 0x7fa6f806c0f0 2023-10-06T03:34:08.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.527+0000 7fa6fcb99700 1 -- 172.21.15.55:0/3116418169 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000be70 con 0x7fa6f806c0f0 2023-10-06T03:34:08.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.527+0000 7fa6feb9d700 1 -- 172.21.15.55:0/3116418169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f806c0f0 msgr2=0x7fa6f806a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:08.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.527+0000 7fa6feb9d700 1 --2- 172.21.15.55:0/3116418169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f806c0f0 0x7fa6f806a120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009e60 tx=0x7fa6f0009780 comp rx=0 tx=0).stop 2023-10-06T03:34:08.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.527+0000 7fa6feb9d700 1 -- 172.21.15.55:0/3116418169 shutdown_connections 2023-10-06T03:34:08.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.527+0000 7fa6feb9d700 1 --2- 172.21.15.55:0/3116418169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f806c0f0 0x7fa6f806a120 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.527+0000 7fa6feb9d700 1 --2- 172.21.15.55:0/3116418169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f8142ea0 0x7fa6f80b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.527+0000 7fa6feb9d700 1 -- 172.21.15.55:0/3116418169 >> 172.21.15.55:0/3116418169 conn(0x7fa6f80ce430 msgr2=0x7fa6f80ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:08.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.527+0000 7fa6feb9d700 1 -- 172.21.15.55:0/3116418169 shutdown_connections 2023-10-06T03:34:08.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.527+0000 7fa6feb9d700 1 -- 172.21.15.55:0/3116418169 wait complete. 2023-10-06T03:34:08.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.529+0000 7fa6feb9d700 1 Processor -- start 2023-10-06T03:34:08.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.529+0000 7fa6feb9d700 1 -- start start 2023-10-06T03:34:08.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6feb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f806c0f0 0x7fa6f806a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:08.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6feb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f8142ea0 0x7fa6f813b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:08.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6feb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f806ac30 0x7fa6f813b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:08.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6feb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80c89d0 con 0x7fa6f8142ea0 2023-10-06T03:34:08.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6feb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f813de60 con 0x7fa6f806c0f0 2023-10-06T03:34:08.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6feb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80c8380 con 0x7fa6f806ac30 2023-10-06T03:34:08.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6fd39a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f8142ea0 0x7fa6f813b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:08.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6fd39a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f8142ea0 0x7fa6f813b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53770/0 (socket says 172.21.15.55:53770) 2023-10-06T03:34:08.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6fd39a700 1 -- 172.21.15.55:0/2893993382 learned_addr learned my addr 172.21.15.55:0/2893993382 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:08.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6fe39c700 1 --2- 172.21.15.55:0/2893993382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f806ac30 0x7fa6f813b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6fdb9b700 1 --2- 172.21.15.55:0/2893993382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f806c0f0 0x7fa6f806a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6fd39a700 1 -- 172.21.15.55:0/2893993382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f806ac30 msgr2=0x7fa6f813b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6fd39a700 1 --2- 172.21.15.55:0/2893993382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f806ac30 0x7fa6f813b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6fd39a700 1 -- 172.21.15.55:0/2893993382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f806c0f0 msgr2=0x7fa6f806a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.530+0000 7fa6fd39a700 1 --2- 172.21.15.55:0/2893993382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f806c0f0 0x7fa6f806a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.531+0000 7fa6fd39a700 1 -- 172.21.15.55:0/2893993382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f0009a20 con 0x7fa6f8142ea0 2023-10-06T03:34:08.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.531+0000 7fa6fd39a700 1 --2- 172.21.15.55:0/2893993382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f8142ea0 0x7fa6f813b120 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c7c0 tx=0x7fa6e8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:08.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.531+0000 7fa6edffb700 1 -- 172.21.15.55:0/2893993382 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8005b30 con 0x7fa6f8142ea0 2023-10-06T03:34:08.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.531+0000 7fa6edffb700 1 -- 172.21.15.55:0/2893993382 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e8005c90 con 0x7fa6f8142ea0 2023-10-06T03:34:08.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.531+0000 7fa6feb9d700 1 -- 172.21.15.55:0/2893993382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f80c8660 con 0x7fa6f8142ea0 2023-10-06T03:34:08.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.531+0000 7fa6edffb700 1 -- 172.21.15.55:0/2893993382 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8007340 con 0x7fa6f8142ea0 2023-10-06T03:34:08.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.531+0000 7fa6feb9d700 1 -- 172.21.15.55:0/2893993382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f813d350 con 0x7fa6f8142ea0 2023-10-06T03:34:08.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.532+0000 7fa6feb9d700 1 -- 172.21.15.55:0/2893993382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f80c8b80 con 0x7fa6f8142ea0 2023-10-06T03:34:08.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.533+0000 7fa6edffb700 1 -- 172.21.15.55:0/2893993382 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa6e801b400 con 0x7fa6f8142ea0 2023-10-06T03:34:08.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.534+0000 7fa6edffb700 1 --2- 172.21.15.55:0/2893993382 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa6dc066f30 0x7fa6dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:08.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.534+0000 7fa6edffb700 1 -- 172.21.15.55:0/2893993382 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa6e8083e80 con 0x7fa6f8142ea0 2023-10-06T03:34:08.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.535+0000 7fa6fdb9b700 1 --2- 172.21.15.55:0/2893993382 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa6dc066f30 0x7fa6dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:08.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.535+0000 7fa6fdb9b700 1 --2- 172.21.15.55:0/2893993382 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa6dc066f30 0x7fa6dc0693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009ca0 tx=0x7fa6f4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:08.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.538+0000 7fa6edffb700 1 -- 172.21.15.55:0/2893993382 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6f80c8b80 con 0x7fa6f8142ea0 2023-10-06T03:34:08.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.820+0000 7fa6feb9d700 1 -- 172.21.15.55:0/2893993382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa6f804c3c0 con 0x7fa6f8142ea0 2023-10-06T03:34:08.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.827+0000 7fa6edffb700 1 -- 172.21.15.55:0/2893993382 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa6e80512c0 con 0x7fa6f8142ea0 2023-10-06T03:34:08.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.829+0000 7fa6feb9d700 1 -- 172.21.15.55:0/2893993382 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa6dc066f30 msgr2=0x7fa6dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:08.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.829+0000 7fa6feb9d700 1 --2- 172.21.15.55:0/2893993382 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa6dc066f30 0x7fa6dc0693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4009ca0 tx=0x7fa6f4008040 comp rx=0 tx=0).stop 2023-10-06T03:34:08.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.829+0000 7fa6feb9d700 1 -- 172.21.15.55:0/2893993382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f8142ea0 msgr2=0x7fa6f813b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:08.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.829+0000 7fa6feb9d700 1 --2- 172.21.15.55:0/2893993382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f8142ea0 0x7fa6f813b120 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa6e800c7c0 tx=0x7fa6e8007e00 comp rx=0 tx=0).stop 2023-10-06T03:34:08.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.830+0000 7fa6feb9d700 1 -- 172.21.15.55:0/2893993382 shutdown_connections 2023-10-06T03:34:08.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.830+0000 7fa6feb9d700 1 --2- 172.21.15.55:0/2893993382 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa6dc066f30 0x7fa6dc0693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.830+0000 7fa6feb9d700 1 --2- 172.21.15.55:0/2893993382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f806ac30 0x7fa6f813b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.830+0000 7fa6feb9d700 1 --2- 172.21.15.55:0/2893993382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f806c0f0 0x7fa6f806a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.830+0000 7fa6feb9d700 1 --2- 172.21.15.55:0/2893993382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f8142ea0 0x7fa6f813b120 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:08.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.830+0000 7fa6feb9d700 1 -- 172.21.15.55:0/2893993382 >> 172.21.15.55:0/2893993382 conn(0x7fa6f80ce430 msgr2=0x7fa6f8141d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:08.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.830+0000 7fa6feb9d700 1 -- 172.21.15.55:0/2893993382 shutdown_connections 2023-10-06T03:34:08.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:08.830+0000 7fa6feb9d700 1 -- 172.21.15.55:0/2893993382 wait complete. 2023-10-06T03:34:08.849 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:34:09.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.035+0000 7f26e759e700 1 Processor -- start 2023-10-06T03:34:09.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.035+0000 7f26e759e700 1 -- start start 2023-10-06T03:34:09.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.036+0000 7f26e759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26e81593b0 0x7f26e8159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.036+0000 7f26e759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26e80ceaa0 0x7f26e80b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.036+0000 7f26e759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f26e80b1740 0x7f26e80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.036+0000 7f26e759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e80b2100 con 0x7f26e80ceaa0 2023-10-06T03:34:09.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.036+0000 7f26e759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e80b6570 con 0x7f26e80b1740 2023-10-06T03:34:09.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.036+0000 7f26e759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80cfe20 con 0x7f26e81593b0 2023-10-06T03:34:09.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.036+0000 7f26dffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26e80ceaa0 0x7f26e80b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.037+0000 7f26dffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26e80ceaa0 0x7f26e80b6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53772/0 (socket says 172.21.15.55:53772) 2023-10-06T03:34:09.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.037+0000 7f26dffff700 1 -- 172.21.15.55:0/2090503374 learned_addr learned my addr 172.21.15.55:0/2090503374 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:09.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.037+0000 7f26e6d9d700 1 --2- 172.21.15.55:0/2090503374 >> v2:172.21.15.163:3300/0 conn(0x7f26e80b1740 0x7f26e80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.037+0000 7f26e659c700 1 --2- 172.21.15.55:0/2090503374 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26e81593b0 0x7f26e8159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.037+0000 7f26dffff700 1 -- 172.21.15.55:0/2090503374 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26e81593b0 msgr2=0x7f26e8159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:09.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.037+0000 7f26dffff700 1 --2- 172.21.15.55:0/2090503374 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26e81593b0 0x7f26e8159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.037+0000 7f26dffff700 1 -- 172.21.15.55:0/2090503374 >> v2:172.21.15.163:3300/0 conn(0x7f26e80b1740 msgr2=0x7f26e80b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:09.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.037+0000 7f26dffff700 1 --2- 172.21.15.55:0/2090503374 >> v2:172.21.15.163:3300/0 conn(0x7f26e80b1740 0x7f26e80b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.037+0000 7f26dffff700 1 -- 172.21.15.55:0/2090503374 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e80cf210 con 0x7f26e80ceaa0 2023-10-06T03:34:09.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.037+0000 7f26dffff700 1 --2- 172.21.15.55:0/2090503374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e80ceaa0 0x7f26e80b6030 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f26e80b24b0 tx=0x7f26d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9ef9efdddebd2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:09.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.037+0000 7f26e5d9b700 1 -- 172.21.15.55:0/2090503374 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d000b640 con 0x7f26e80ceaa0 2023-10-06T03:34:09.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.038+0000 7f26e5d9b700 1 -- 172.21.15.55:0/2090503374 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d000b7a0 con 0x7f26e80ceaa0 2023-10-06T03:34:09.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.038+0000 7f26e5d9b700 1 -- 172.21.15.55:0/2090503374 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d000bb60 con 0x7f26e80ceaa0 2023-10-06T03:34:09.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.038+0000 7f26e759e700 1 -- 172.21.15.55:0/2090503374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e80ceaa0 msgr2=0x7f26e80b6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:09.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.038+0000 7f26e759e700 1 --2- 172.21.15.55:0/2090503374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e80ceaa0 0x7f26e80b6030 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f26e80b24b0 tx=0x7f26d0009250 comp rx=0 tx=0).stop 2023-10-06T03:34:09.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.038+0000 7f26e759e700 1 -- 172.21.15.55:0/2090503374 shutdown_connections 2023-10-06T03:34:09.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.038+0000 7f26e759e700 1 --2- 172.21.15.55:0/2090503374 >> v2:172.21.15.163:3300/0 conn(0x7f26e80b1740 0x7f26e80b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.038+0000 7f26e759e700 1 --2- 172.21.15.55:0/2090503374 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26e81593b0 0x7f26e8159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.038+0000 7f26e759e700 1 --2- 172.21.15.55:0/2090503374 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e80ceaa0 0x7f26e80b6030 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.038+0000 7f26e759e700 1 -- 172.21.15.55:0/2090503374 >> 172.21.15.55:0/2090503374 conn(0x7f26e80d1130 msgr2=0x7f26e80b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:09.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.038+0000 7f26e759e700 1 -- 172.21.15.55:0/2090503374 shutdown_connections 2023-10-06T03:34:09.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.038+0000 7f26e759e700 1 -- 172.21.15.55:0/2090503374 wait complete. 2023-10-06T03:34:09.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.040+0000 7f26e759e700 1 Processor -- start 2023-10-06T03:34:09.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.040+0000 7f26e759e700 1 -- start start 2023-10-06T03:34:09.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.041+0000 7f26e759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e80b1740 0x7f26e8149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.041+0000 7f26e759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26e80ceaa0 0x7f26e815e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.041+0000 7f26e759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26e81593b0 0x7f26e815eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.041+0000 7f26e759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e816a8a0 con 0x7f26e80b1740 2023-10-06T03:34:09.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.041+0000 7f26e759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e816d6f0 con 0x7f26e80ceaa0 2023-10-06T03:34:09.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.041+0000 7f26e759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e8155450 con 0x7f26e81593b0 2023-10-06T03:34:09.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.041+0000 7f26e659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e80b1740 0x7f26e8149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.041+0000 7f26e659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e80b1740 0x7f26e8149080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53776/0 (socket says 172.21.15.55:53776) 2023-10-06T03:34:09.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.041+0000 7f26e659c700 1 -- 172.21.15.55:0/4042851731 learned_addr learned my addr 172.21.15.55:0/4042851731 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:09.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.041+0000 7f26dffff700 1 --2- 172.21.15.55:0/4042851731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26e80ceaa0 0x7f26e815e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.041+0000 7f26e6d9d700 1 --2- 172.21.15.55:0/4042851731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26e81593b0 0x7f26e815eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.042+0000 7f26e659c700 1 -- 172.21.15.55:0/4042851731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26e81593b0 msgr2=0x7f26e815eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:09.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.042+0000 7f26e659c700 1 --2- 172.21.15.55:0/4042851731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26e81593b0 0x7f26e815eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.042+0000 7f26e659c700 1 -- 172.21.15.55:0/4042851731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26e80ceaa0 msgr2=0x7f26e815e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:09.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.042+0000 7f26e659c700 1 --2- 172.21.15.55:0/4042851731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26e80ceaa0 0x7f26e815e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.042+0000 7f26e659c700 1 -- 172.21.15.55:0/4042851731 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d00094f0 con 0x7f26e80b1740 2023-10-06T03:34:09.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.042+0000 7f26e659c700 1 --2- 172.21.15.55:0/4042851731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e80b1740 0x7f26e8149080 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f26d800e960 tx=0x7f26d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:09.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.042+0000 7f26de7fc700 1 -- 172.21.15.55:0/4042851731 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d800c850 con 0x7f26e80b1740 2023-10-06T03:34:09.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.042+0000 7f26de7fc700 1 -- 172.21.15.55:0/4042851731 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d800c9b0 con 0x7f26e80b1740 2023-10-06T03:34:09.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.042+0000 7f26de7fc700 1 -- 172.21.15.55:0/4042851731 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d800cd50 con 0x7f26e80b1740 2023-10-06T03:34:09.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.042+0000 7f26e759e700 1 -- 172.21.15.55:0/4042851731 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e816a5c0 con 0x7f26e80b1740 2023-10-06T03:34:09.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.042+0000 7f26e759e700 1 -- 172.21.15.55:0/4042851731 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e816d380 con 0x7f26e80b1740 2023-10-06T03:34:09.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.043+0000 7f26e759e700 1 -- 172.21.15.55:0/4042851731 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e816d9d0 con 0x7f26e80b1740 2023-10-06T03:34:09.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.044+0000 7f26de7fc700 1 -- 172.21.15.55:0/4042851731 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26d8013680 con 0x7f26e80b1740 2023-10-06T03:34:09.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.045+0000 7f26de7fc700 1 --2- 172.21.15.55:0/4042851731 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f26c8067000 0x7f26c80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.045+0000 7f26de7fc700 1 -- 172.21.15.55:0/4042851731 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f26d80845d0 con 0x7f26e80b1740 2023-10-06T03:34:09.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.046+0000 7f26dffff700 1 --2- 172.21.15.55:0/4042851731 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f26c8067000 0x7f26c80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.046+0000 7f26dffff700 1 --2- 172.21.15.55:0/4042851731 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f26c8067000 0x7f26c80694b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f26e80b24b0 tx=0x7f26d0017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:09.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.049+0000 7f26de7fc700 1 -- 172.21.15.55:0/4042851731 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26d8051a10 con 0x7f26e80b1740 2023-10-06T03:34:09.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.257+0000 7f26e759e700 1 -- 172.21.15.55:0/4042851731 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26e804c3c0 con 0x7f26e80b1740 2023-10-06T03:34:09.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.258+0000 7f26de7fc700 1 -- 172.21.15.55:0/4042851731 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+13698 (secure 0 0 0) 0x7f26d8055030 con 0x7f26e80b1740 2023-10-06T03:34:09.259 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:09.259 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":28,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:34:05.262821+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:34:09.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.260+0000 7f26e759e700 1 -- 172.21.15.55:0/4042851731 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f26c8067000 msgr2=0x7f26c80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:09.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.260+0000 7f26e759e700 1 --2- 172.21.15.55:0/4042851731 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f26c8067000 0x7f26c80694b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f26e80b24b0 tx=0x7f26d0017000 comp rx=0 tx=0).stop 2023-10-06T03:34:09.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.260+0000 7f26e759e700 1 -- 172.21.15.55:0/4042851731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e80b1740 msgr2=0x7f26e8149080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:09.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.260+0000 7f26e759e700 1 --2- 172.21.15.55:0/4042851731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e80b1740 0x7f26e8149080 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f26d800e960 tx=0x7f26d80091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.260+0000 7f26e759e700 1 -- 172.21.15.55:0/4042851731 shutdown_connections 2023-10-06T03:34:09.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.260+0000 7f26e759e700 1 --2- 172.21.15.55:0/4042851731 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f26c8067000 0x7f26c80694b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.260+0000 7f26e759e700 1 --2- 172.21.15.55:0/4042851731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26e81593b0 0x7f26e815eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.260+0000 7f26e759e700 1 --2- 172.21.15.55:0/4042851731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26e80b1740 0x7f26e8149080 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.260+0000 7f26e759e700 1 --2- 172.21.15.55:0/4042851731 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26e80ceaa0 0x7f26e815e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.260+0000 7f26e759e700 1 -- 172.21.15.55:0/4042851731 >> 172.21.15.55:0/4042851731 conn(0x7f26e80d1130 msgr2=0x7f26e8157120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:09.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.261+0000 7f26e759e700 1 -- 172.21.15.55:0/4042851731 shutdown_connections 2023-10-06T03:34:09.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.261+0000 7f26e759e700 1 -- 172.21.15.55:0/4042851731 wait complete. 2023-10-06T03:34:09.280 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T03:34:09.280 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:34:09.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.469+0000 7f3931d9d700 1 Processor -- start 2023-10-06T03:34:09.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.470+0000 7f3931d9d700 1 -- start start 2023-10-06T03:34:09.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.471+0000 7f3931d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f392c0b4e70 0x7f392c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.471+0000 7f3931d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f392c0b16d0 0x7f392c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.471+0000 7f3931d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f392c15d240 0x7f392c15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.471+0000 7f3931d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392c0b2080 con 0x7f392c15d240 2023-10-06T03:34:09.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.471+0000 7f3931d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392c0cfb00 con 0x7f392c0b16d0 2023-10-06T03:34:09.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.471+0000 7f3931d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c0b5c40 con 0x7f392c0b4e70 2023-10-06T03:34:09.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.471+0000 7f393159c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f392c15d240 0x7f392c15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.471+0000 7f393159c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f392c15d240 0x7f392c15d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53782/0 (socket says 172.21.15.55:53782) 2023-10-06T03:34:09.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.471+0000 7f393159c700 1 -- 172.21.15.55:0/3892005380 learned_addr learned my addr 172.21.15.55:0/3892005380 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:09.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.471+0000 7f3930d9b700 1 --2- 172.21.15.55:0/3892005380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f392c0b4e70 0x7f392c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.471+0000 7f392bfff700 1 --2- 172.21.15.55:0/3892005380 >> v2:172.21.15.163:3300/0 conn(0x7f392c0b16d0 0x7f392c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.471+0000 7f393159c700 1 -- 172.21.15.55:0/3892005380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f392c0b4e70 msgr2=0x7f392c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:09.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.472+0000 7f393159c700 1 --2- 172.21.15.55:0/3892005380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f392c0b4e70 0x7f392c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.472+0000 7f393159c700 1 -- 172.21.15.55:0/3892005380 >> v2:172.21.15.163:3300/0 conn(0x7f392c0b16d0 msgr2=0x7f392c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:09.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.472+0000 7f393159c700 1 --2- 172.21.15.55:0/3892005380 >> v2:172.21.15.163:3300/0 conn(0x7f392c0b16d0 0x7f392c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.472+0000 7f393159c700 1 -- 172.21.15.55:0/3892005380 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c0b5f20 con 0x7f392c15d240 2023-10-06T03:34:09.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.472+0000 7f393159c700 1 --2- 172.21.15.55:0/3892005380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392c15d240 0x7f392c15d6c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f392c089440 tx=0x7f3924009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212101d077789b67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:09.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.472+0000 7f392b7fe700 1 -- 172.21.15.55:0/3892005380 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392400e070 con 0x7f392c15d240 2023-10-06T03:34:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.472+0000 7f392b7fe700 1 -- 172.21.15.55:0/3892005380 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392400ba50 con 0x7f392c15d240 2023-10-06T03:34:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.472+0000 7f392b7fe700 1 -- 172.21.15.55:0/3892005380 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39240092e0 con 0x7f392c15d240 2023-10-06T03:34:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.473+0000 7f3931d9d700 1 -- 172.21.15.55:0/3892005380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392c15d240 msgr2=0x7f392c15d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.473+0000 7f3931d9d700 1 --2- 172.21.15.55:0/3892005380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392c15d240 0x7f392c15d6c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f392c089440 tx=0x7f3924009840 comp rx=0 tx=0).stop 2023-10-06T03:34:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.473+0000 7f3931d9d700 1 -- 172.21.15.55:0/3892005380 shutdown_connections 2023-10-06T03:34:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.473+0000 7f3931d9d700 1 --2- 172.21.15.55:0/3892005380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392c15d240 0x7f392c15d6c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.473+0000 7f3931d9d700 1 --2- 172.21.15.55:0/3892005380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f392c0b4e70 0x7f392c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.473+0000 7f3931d9d700 1 --2- 172.21.15.55:0/3892005380 >> v2:172.21.15.163:3300/0 conn(0x7f392c0b16d0 0x7f392c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.473+0000 7f3931d9d700 1 -- 172.21.15.55:0/3892005380 >> 172.21.15.55:0/3892005380 conn(0x7f392c0d10c0 msgr2=0x7f392c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.473+0000 7f3931d9d700 1 -- 172.21.15.55:0/3892005380 shutdown_connections 2023-10-06T03:34:09.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.473+0000 7f3931d9d700 1 -- 172.21.15.55:0/3892005380 wait complete. 2023-10-06T03:34:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.474+0000 7f3931d9d700 1 Processor -- start 2023-10-06T03:34:09.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.474+0000 7f3931d9d700 1 -- start start 2023-10-06T03:34:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.475+0000 7f3931d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392c0b16d0 0x7f392c155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.475+0000 7f3931d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f392c0b4e70 0x7f392c14fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.475+0000 7f3931d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f392c15d240 0x7f392c150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.475+0000 7f3931d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c150780 con 0x7f392c0b16d0 2023-10-06T03:34:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.475+0000 7f3931d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c151700 con 0x7f392c0b4e70 2023-10-06T03:34:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.475+0000 7f3931d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c152220 con 0x7f392c15d240 2023-10-06T03:34:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.475+0000 7f3930d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392c0b16d0 0x7f392c155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.476+0000 7f3930d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392c0b16d0 0x7f392c155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53798/0 (socket says 172.21.15.55:53798) 2023-10-06T03:34:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.476+0000 7f3930d9b700 1 -- 172.21.15.55:0/2303044927 learned_addr learned my addr 172.21.15.55:0/2303044927 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.476+0000 7f392bfff700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f392c0b4e70 0x7f392c14fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.476+0000 7f393159c700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f392c15d240 0x7f392c150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.476+0000 7f3930d9b700 1 -- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f392c15d240 msgr2=0x7f392c150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.476+0000 7f3930d9b700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f392c15d240 0x7f392c150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.476+0000 7f3930d9b700 1 -- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f392c0b4e70 msgr2=0x7f392c14fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.476+0000 7f3930d9b700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f392c0b4e70 0x7f392c14fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.476+0000 7f3930d9b700 1 -- 172.21.15.55:0/2303044927 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3924009ae0 con 0x7f392c0b16d0 2023-10-06T03:34:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.476+0000 7f393159c700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f392c15d240 0x7f392c150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.476+0000 7f3930d9b700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392c0b16d0 0x7f392c155f80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f392000d8c0 tx=0x7f3920008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.476+0000 7f3928ff9700 1 -- 172.21.15.55:0/2303044927 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39200125f0 con 0x7f392c0b16d0 2023-10-06T03:34:09.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.477+0000 7f3928ff9700 1 -- 172.21.15.55:0/2303044927 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3920012770 con 0x7f392c0b16d0 2023-10-06T03:34:09.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.477+0000 7f3928ff9700 1 -- 172.21.15.55:0/2303044927 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920012b30 con 0x7f392c0b16d0 2023-10-06T03:34:09.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.477+0000 7f3931d9d700 1 -- 172.21.15.55:0/2303044927 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392c15efd0 con 0x7f392c0b16d0 2023-10-06T03:34:09.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.477+0000 7f3931d9d700 1 -- 172.21.15.55:0/2303044927 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392c151990 con 0x7f392c0b16d0 2023-10-06T03:34:09.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.478+0000 7f3931d9d700 1 -- 172.21.15.55:0/2303044927 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392c0cf510 con 0x7f392c0b16d0 2023-10-06T03:34:09.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.479+0000 7f3928ff9700 1 -- 172.21.15.55:0/2303044927 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f392001e460 con 0x7f392c0b16d0 2023-10-06T03:34:09.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.479+0000 7f3928ff9700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3910066f30 0x7f39100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.479+0000 7f3928ff9700 1 -- 172.21.15.55:0/2303044927 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3920084810 con 0x7f392c0b16d0 2023-10-06T03:34:09.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.480+0000 7f392bfff700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3910066f30 0x7f39100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.481+0000 7f392bfff700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3910066f30 0x7f39100693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f392c1603c0 tx=0x7f3918008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:09.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.484+0000 7f3928ff9700 1 -- 172.21.15.55:0/2303044927 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3920051bd0 con 0x7f392c0b16d0 2023-10-06T03:34:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.686+0000 7f3931d9d700 1 -- 172.21.15.55:0/2303044927 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f392c151fa0 con 0x7f392c0b16d0 2023-10-06T03:34:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.686+0000 7f3928ff9700 1 -- 172.21.15.55:0/2303044927 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7f39200551f0 con 0x7f392c0b16d0 2023-10-06T03:34:09.688 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:34:09.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.688+0000 7f3931d9d700 1 -- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3910066f30 msgr2=0x7f39100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:09.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.688+0000 7f3931d9d700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3910066f30 0x7f39100693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f392c1603c0 tx=0x7f3918008040 comp rx=0 tx=0).stop 2023-10-06T03:34:09.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.688+0000 7f3931d9d700 1 -- 172.21.15.55:0/2303044927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392c0b16d0 msgr2=0x7f392c155f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:09.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.688+0000 7f3931d9d700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392c0b16d0 0x7f392c155f80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f392000d8c0 tx=0x7f3920008040 comp rx=0 tx=0).stop 2023-10-06T03:34:09.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.688+0000 7f3931d9d700 1 -- 172.21.15.55:0/2303044927 shutdown_connections 2023-10-06T03:34:09.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.688+0000 7f3931d9d700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3910066f30 0x7f39100693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.688+0000 7f3931d9d700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f392c15d240 0x7f392c150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.688+0000 7f3931d9d700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f392c0b16d0 0x7f392c155f80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.688+0000 7f3931d9d700 1 --2- 172.21.15.55:0/2303044927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f392c0b4e70 0x7f392c14fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.688+0000 7f3931d9d700 1 -- 172.21.15.55:0/2303044927 >> 172.21.15.55:0/2303044927 conn(0x7f392c0d10c0 msgr2=0x7f392c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:09.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.689+0000 7f3931d9d700 1 -- 172.21.15.55:0/2303044927 shutdown_connections 2023-10-06T03:34:09.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.689+0000 7f3931d9d700 1 -- 172.21.15.55:0/2303044927 wait complete. 2023-10-06T03:34:09.708 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:34:09.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.896+0000 7f90b4f9d700 1 Processor -- start 2023-10-06T03:34:09.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.896+0000 7f90b4f9d700 1 -- start start 2023-10-06T03:34:09.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.897+0000 7f90b4f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f90b00cb1a0 0x7f90b015a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.897+0000 7f90b4f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90b015a8a0 0x7f90b006f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.898+0000 7f90b4f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b006ebd0 con 0x7f90b006f5a0 2023-10-06T03:34:09.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.898+0000 7f90b4f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b015b2e0 con 0x7f90b00cb1a0 2023-10-06T03:34:09.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.898+0000 7f90b4f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b015ad20 con 0x7f90b015a8a0 2023-10-06T03:34:09.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.898+0000 7f90af7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f90b00cb1a0 0x7f90b015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.898+0000 7f90af7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f90b00cb1a0 0x7f90b015a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53812/0 (socket says 172.21.15.55:53812) 2023-10-06T03:34:09.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.898+0000 7f90af7fe700 1 -- 172.21.15.55:0/2723571710 learned_addr learned my addr 172.21.15.55:0/2723571710 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:09.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.898+0000 7f90affff700 1 --1- 172.21.15.55:0/2723571710 >> v1:172.21.15.55:6789/0 conn(0x7f90b006f5a0 0x7f90b00ae0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38186/0 (socket says 172.21.15.55:38186) 2023-10-06T03:34:09.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.898+0000 7f90af7fe700 1 -- 172.21.15.55:0/2723571710 >> v2:172.21.15.163:3300/0 conn(0x7f90b015a8a0 msgr2=0x7f90b006f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:09.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.898+0000 7f90af7fe700 1 --2- 172.21.15.55:0/2723571710 >> v2:172.21.15.163:3300/0 conn(0x7f90b015a8a0 0x7f90b006f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.898+0000 7f90af7fe700 1 -- 172.21.15.55:0/2723571710 >> v1:172.21.15.55:6789/0 conn(0x7f90b006f5a0 legacy=0x7f90b00ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:09.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.898+0000 7f90af7fe700 1 -- 172.21.15.55:0/2723571710 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b015b000 con 0x7f90b00cb1a0 2023-10-06T03:34:09.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.898+0000 7f90ae7fc700 1 -- 172.21.15.55:0/2723571710 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903211413 0 0) 0x7f90b006ebd0 con 0x7f90b006f5a0 2023-10-06T03:34:09.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.899+0000 7f90af7fe700 1 --2- 172.21.15.55:0/2723571710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90b00cb1a0 0x7f90b015a360 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f90b0070d30 tx=0x7f90a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b932c06d8126734f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:09.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.899+0000 7f90ae7fc700 1 -- 172.21.15.55:0/2723571710 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a000b8f0 con 0x7f90b00cb1a0 2023-10-06T03:34:09.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.899+0000 7f90ae7fc700 1 -- 172.21.15.55:0/2723571710 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a000ba50 con 0x7f90b00cb1a0 2023-10-06T03:34:09.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.899+0000 7f90ae7fc700 1 -- 172.21.15.55:0/2723571710 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a000bdf0 con 0x7f90b00cb1a0 2023-10-06T03:34:09.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.899+0000 7f90b4f9d700 1 -- 172.21.15.55:0/2723571710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90b00cb1a0 msgr2=0x7f90b015a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:09.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.899+0000 7f90b4f9d700 1 --2- 172.21.15.55:0/2723571710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90b00cb1a0 0x7f90b015a360 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f90b0070d30 tx=0x7f90a0009250 comp rx=0 tx=0).stop 2023-10-06T03:34:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.899+0000 7f90b4f9d700 1 -- 172.21.15.55:0/2723571710 shutdown_connections 2023-10-06T03:34:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.899+0000 7f90b4f9d700 1 --2- 172.21.15.55:0/2723571710 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90b00cb1a0 0x7f90b015a360 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.899+0000 7f90b4f9d700 1 --2- 172.21.15.55:0/2723571710 >> v2:172.21.15.163:3300/0 conn(0x7f90b015a8a0 0x7f90b006f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.900+0000 7f90b4f9d700 1 -- 172.21.15.55:0/2723571710 >> 172.21.15.55:0/2723571710 conn(0x7f90b00cc1e0 msgr2=0x7f90b00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.900+0000 7f90b4f9d700 1 -- 172.21.15.55:0/2723571710 shutdown_connections 2023-10-06T03:34:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.900+0000 7f90b4f9d700 1 -- 172.21.15.55:0/2723571710 wait complete. 2023-10-06T03:34:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.901+0000 7f90b4f9d700 1 Processor -- start 2023-10-06T03:34:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.901+0000 7f90b4f9d700 1 -- start start 2023-10-06T03:34:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.902+0000 7f90b4f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90b006f5a0 0x7f90b0156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.902+0000 7f90b4f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90b00cb1a0 0x7f90b016ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.902+0000 7f90b4f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b00c4740 0x7f90b014d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.902+0000 7f90b4f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00c5b80 con 0x7f90b00cb1a0 2023-10-06T03:34:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.902+0000 7f90b4f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00c4f70 con 0x7f90b00c4740 2023-10-06T03:34:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.902+0000 7f90b4f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00c5250 con 0x7f90b006f5a0 2023-10-06T03:34:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.902+0000 7f90aeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90b00cb1a0 0x7f90b016ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.902+0000 7f90aeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90b00cb1a0 0x7f90b016ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53826/0 (socket says 172.21.15.55:53826) 2023-10-06T03:34:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.902+0000 7f90aeffd700 1 -- 172.21.15.55:0/882583622 learned_addr learned my addr 172.21.15.55:0/882583622 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.903+0000 7f90aeffd700 1 -- 172.21.15.55:0/882583622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90b006f5a0 msgr2=0x7f90b0156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.903+0000 7f90aeffd700 1 --2- 172.21.15.55:0/882583622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90b006f5a0 0x7f90b0156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.903+0000 7f90aeffd700 1 -- 172.21.15.55:0/882583622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b00c4740 msgr2=0x7f90b014d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:09.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.903+0000 7f90affff700 1 --2- 172.21.15.55:0/882583622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b00c4740 0x7f90b014d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.903+0000 7f90aeffd700 1 --2- 172.21.15.55:0/882583622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b00c4740 0x7f90b014d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:09.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.903+0000 7f90aeffd700 1 -- 172.21.15.55:0/882583622 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a00094f0 con 0x7f90b00cb1a0 2023-10-06T03:34:09.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.903+0000 7f90affff700 1 --2- 172.21.15.55:0/882583622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b00c4740 0x7f90b014d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:09.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.903+0000 7f90aeffd700 1 --2- 172.21.15.55:0/882583622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90b00cb1a0 0x7f90b016ba00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f90a400b810 tx=0x7f90a40091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:09.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.903+0000 7f90977fe700 1 -- 172.21.15.55:0/882583622 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a400f570 con 0x7f90b00cb1a0 2023-10-06T03:34:09.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.903+0000 7f90977fe700 1 -- 172.21.15.55:0/882583622 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a400f6d0 con 0x7f90b00cb1a0 2023-10-06T03:34:09.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.903+0000 7f90b4f9d700 1 -- 172.21.15.55:0/882583622 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b00c5530 con 0x7f90b00cb1a0 2023-10-06T03:34:09.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.903+0000 7f90977fe700 1 -- 172.21.15.55:0/882583622 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a400fa90 con 0x7f90b00cb1a0 2023-10-06T03:34:09.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.903+0000 7f90b4f9d700 1 -- 172.21.15.55:0/882583622 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b00c4bc0 con 0x7f90b00cb1a0 2023-10-06T03:34:09.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.905+0000 7f90b4f9d700 1 -- 172.21.15.55:0/882583622 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b00c59d0 con 0x7f90b00cb1a0 2023-10-06T03:34:09.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.905+0000 7f90977fe700 1 -- 172.21.15.55:0/882583622 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90a401a460 con 0x7f90b00cb1a0 2023-10-06T03:34:09.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.906+0000 7f90977fe700 1 --2- 172.21.15.55:0/882583622 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f909c066f30 0x7f909c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:09.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.906+0000 7f90977fe700 1 -- 172.21.15.55:0/882583622 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f90a40840c0 con 0x7f90b00cb1a0 2023-10-06T03:34:09.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.907+0000 7f90af7fe700 1 --2- 172.21.15.55:0/882583622 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f909c066f30 0x7f909c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:09.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.907+0000 7f90af7fe700 1 --2- 172.21.15.55:0/882583622 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f909c066f30 0x7f909c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90b0070d30 tx=0x7f90a0017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:09.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:09.910+0000 7f90977fe700 1 -- 172.21.15.55:0/882583622 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90a4051500 con 0x7f90b00cb1a0 2023-10-06T03:34:10.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.112+0000 7f90b4f9d700 1 -- 172.21.15.55:0/882583622 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f90b00aef80 con 0x7f90b00cb1a0 2023-10-06T03:34:10.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.113+0000 7f90977fe700 1 -- 172.21.15.55:0/882583622 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90b00aef80 con 0x7f90b00cb1a0 2023-10-06T03:34:10.114 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:34:10.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.115+0000 7f90b4f9d700 1 -- 172.21.15.55:0/882583622 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f909c066f30 msgr2=0x7f909c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:10.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.115+0000 7f90b4f9d700 1 --2- 172.21.15.55:0/882583622 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f909c066f30 0x7f909c0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90b0070d30 tx=0x7f90a0017000 comp rx=0 tx=0).stop 2023-10-06T03:34:10.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.115+0000 7f90b4f9d700 1 -- 172.21.15.55:0/882583622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90b00cb1a0 msgr2=0x7f90b016ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:10.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.115+0000 7f90b4f9d700 1 --2- 172.21.15.55:0/882583622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90b00cb1a0 0x7f90b016ba00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f90a400b810 tx=0x7f90a40091d0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.115+0000 7f90b4f9d700 1 -- 172.21.15.55:0/882583622 shutdown_connections 2023-10-06T03:34:10.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.115+0000 7f90b4f9d700 1 --2- 172.21.15.55:0/882583622 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90b00c4740 0x7f90b014d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.115+0000 7f90b4f9d700 1 --2- 172.21.15.55:0/882583622 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f909c066f30 0x7f909c0693e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.115+0000 7f90b4f9d700 1 --2- 172.21.15.55:0/882583622 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90b006f5a0 0x7f90b0156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.115+0000 7f90b4f9d700 1 --2- 172.21.15.55:0/882583622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90b00cb1a0 0x7f90b016ba00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.115+0000 7f90b4f9d700 1 -- 172.21.15.55:0/882583622 >> 172.21.15.55:0/882583622 conn(0x7f90b00cc1e0 msgr2=0x7f90b00c0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:10.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.115+0000 7f90b4f9d700 1 -- 172.21.15.55:0/882583622 shutdown_connections 2023-10-06T03:34:10.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.115+0000 7f90b4f9d700 1 -- 172.21.15.55:0/882583622 wait complete. 2023-10-06T03:34:10.134 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:34:10.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.319+0000 7ff38eb9d700 1 Processor -- start 2023-10-06T03:34:10.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.319+0000 7ff38eb9d700 1 -- start start 2023-10-06T03:34:10.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.320+0000 7ff38eb9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff3880b02a0 0x7ff38815bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:10.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.320+0000 7ff38eb9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38806ae70 con 0x7ff38806a0a0 2023-10-06T03:34:10.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.320+0000 7ff38eb9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3880c8a60 con 0x7ff38806d950 2023-10-06T03:34:10.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.320+0000 7ff38eb9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38806a540 con 0x7ff3880b02a0 2023-10-06T03:34:10.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.320+0000 7ff38d39a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff38806a0a0 0x7ff38815b410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38188/0 (socket says 172.21.15.55:38188) 2023-10-06T03:34:10.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.321+0000 7ff38d39a700 1 -- 172.21.15.55:0/3225021530 learned_addr learned my addr 172.21.15.55:0/3225021530 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:10.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.321+0000 7ff38e39c700 1 --2- 172.21.15.55:0/3225021530 >> v2:172.21.15.163:3300/0 conn(0x7ff3880b02a0 0x7ff38815bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:10.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.321+0000 7ff38cb99700 1 -- 172.21.15.55:0/3225021530 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760191807 0 0) 0x7ff38806ae70 con 0x7ff38806a0a0 2023-10-06T03:34:10.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.321+0000 7ff38cb99700 1 -- 172.21.15.55:0/3225021530 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff374003570 con 0x7ff38806a0a0 2023-10-06T03:34:10.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.321+0000 7ff38e39c700 1 -- 172.21.15.55:0/3225021530 >> v1:172.21.15.163:6789/0 conn(0x7ff38806d950 legacy=0x7ff3880c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:10.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.321+0000 7ff38e39c700 1 -- 172.21.15.55:0/3225021530 >> v1:172.21.15.55:6789/0 conn(0x7ff38806a0a0 legacy=0x7ff38815b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:10.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.321+0000 7ff38e39c700 1 -- 172.21.15.55:0/3225021530 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38806a820 con 0x7ff3880b02a0 2023-10-06T03:34:10.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.322+0000 7ff38e39c700 1 --2- 172.21.15.55:0/3225021530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3880b02a0 0x7ff38815bb20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff380009e60 tx=0x7ff380009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf8a62e1ce9da0b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:10.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.325+0000 7ff38cb99700 1 -- 172.21.15.55:0/3225021530 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38000e070 con 0x7ff3880b02a0 2023-10-06T03:34:10.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.325+0000 7ff38cb99700 1 -- 172.21.15.55:0/3225021530 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38000bb60 con 0x7ff3880b02a0 2023-10-06T03:34:10.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.325+0000 7ff38cb99700 1 -- 172.21.15.55:0/3225021530 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380012430 con 0x7ff3880b02a0 2023-10-06T03:34:10.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.326+0000 7ff38eb9d700 1 -- 172.21.15.55:0/3225021530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3880b02a0 msgr2=0x7ff38815bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:10.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.326+0000 7ff38eb9d700 1 --2- 172.21.15.55:0/3225021530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3880b02a0 0x7ff38815bb20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff380009e60 tx=0x7ff380009780 comp rx=0 tx=0).stop 2023-10-06T03:34:10.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.326+0000 7ff38eb9d700 1 -- 172.21.15.55:0/3225021530 shutdown_connections 2023-10-06T03:34:10.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.326+0000 7ff38eb9d700 1 --2- 172.21.15.55:0/3225021530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff3880b02a0 0x7ff38815bb20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.326+0000 7ff38eb9d700 1 -- 172.21.15.55:0/3225021530 >> 172.21.15.55:0/3225021530 conn(0x7ff3880cc1f0 msgr2=0x7ff3880cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:10.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.326+0000 7ff38eb9d700 1 -- 172.21.15.55:0/3225021530 shutdown_connections 2023-10-06T03:34:10.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.326+0000 7ff38eb9d700 1 -- 172.21.15.55:0/3225021530 wait complete. 2023-10-06T03:34:10.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.327+0000 7ff38eb9d700 1 Processor -- start 2023-10-06T03:34:10.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.328+0000 7ff38eb9d700 1 -- start start 2023-10-06T03:34:10.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.328+0000 7ff38eb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff38806a0a0 0x7ff388165a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:10.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.328+0000 7ff38eb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff38806d950 0x7ff388142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:10.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38eb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3880b02a0 0x7ff3880c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:10.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38eb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38815a950 con 0x7ff3880b02a0 2023-10-06T03:34:10.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38eb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3880c46e0 con 0x7ff38806d950 2023-10-06T03:34:10.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38eb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3881431a0 con 0x7ff38806a0a0 2023-10-06T03:34:10.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38e39c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3880b02a0 0x7ff3880c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:10.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38e39c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3880b02a0 0x7ff3880c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53842/0 (socket says 172.21.15.55:53842) 2023-10-06T03:34:10.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38e39c700 1 -- 172.21.15.55:0/4187400860 learned_addr learned my addr 172.21.15.55:0/4187400860 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:10.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38d39a700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff38806d950 0x7ff388142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:10.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38db9b700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff38806a0a0 0x7ff388165a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:10.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38e39c700 1 -- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff38806a0a0 msgr2=0x7ff388165a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:10.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38e39c700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff38806a0a0 0x7ff388165a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38e39c700 1 -- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff38806d950 msgr2=0x7ff388142c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:10.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38e39c700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff38806d950 0x7ff388142c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38e39c700 1 -- 172.21.15.55:0/4187400860 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff380009a20 con 0x7ff3880b02a0 2023-10-06T03:34:10.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38db9b700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff38806a0a0 0x7ff388165a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:10.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.329+0000 7ff38e39c700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3880b02a0 0x7ff3880c8bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff38806fe40 tx=0x7ff380019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:10.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.330+0000 7ff37dffb700 1 -- 172.21.15.55:0/4187400860 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380012e00 con 0x7ff3880b02a0 2023-10-06T03:34:10.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.330+0000 7ff37dffb700 1 -- 172.21.15.55:0/4187400860 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38001d630 con 0x7ff3880b02a0 2023-10-06T03:34:10.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.330+0000 7ff37dffb700 1 -- 172.21.15.55:0/4187400860 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38001d9f0 con 0x7ff3880b02a0 2023-10-06T03:34:10.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.330+0000 7ff38eb9d700 1 -- 172.21.15.55:0/4187400860 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38815a670 con 0x7ff3880b02a0 2023-10-06T03:34:10.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.330+0000 7ff38eb9d700 1 -- 172.21.15.55:0/4187400860 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3880c4370 con 0x7ff3880b02a0 2023-10-06T03:34:10.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.331+0000 7ff38eb9d700 1 -- 172.21.15.55:0/4187400860 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38804c3c0 con 0x7ff3880b02a0 2023-10-06T03:34:10.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.332+0000 7ff37dffb700 1 -- 172.21.15.55:0/4187400860 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff380024070 con 0x7ff3880b02a0 2023-10-06T03:34:10.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.332+0000 7ff37dffb700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff36c066f30 0x7ff36c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:10.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.332+0000 7ff37dffb700 1 -- 172.21.15.55:0/4187400860 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff380086c30 con 0x7ff3880b02a0 2023-10-06T03:34:10.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.333+0000 7ff38db9b700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff36c066f30 0x7ff36c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:10.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.334+0000 7ff38db9b700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff36c066f30 0x7ff36c0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff384001840 tx=0x7ff384009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:10.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.337+0000 7ff37dffb700 1 -- 172.21.15.55:0/4187400860 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3800540a0 con 0x7ff3880b02a0 2023-10-06T03:34:10.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.539+0000 7ff38eb9d700 1 -- 172.21.15.55:0/4187400860 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff3880c4820 con 0x7ff3880b02a0 2023-10-06T03:34:10.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.540+0000 7ff37dffb700 1 -- 172.21.15.55:0/4187400860 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) v1 ==== 84+0+126 (secure 0 0 0) 0x7ff38002c090 con 0x7ff3880b02a0 2023-10-06T03:34:10.541 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:10.541 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:34:10.541 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:10.542 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2023-10-06T03:34:10.542 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": "rbd" 2023-10-06T03:34:10.542 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:34:10.542 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:10.542 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 2, 2023-10-06T03:34:10.542 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2023-10-06T03:34:10.542 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:10.542 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:34:10.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.542+0000 7ff38eb9d700 1 -- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff36c066f30 msgr2=0x7ff36c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:10.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.542+0000 7ff38eb9d700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff36c066f30 0x7ff36c0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff384001840 tx=0x7ff384009040 comp rx=0 tx=0).stop 2023-10-06T03:34:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.542+0000 7ff38eb9d700 1 -- 172.21.15.55:0/4187400860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3880b02a0 msgr2=0x7ff3880c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.542+0000 7ff38eb9d700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3880b02a0 0x7ff3880c8bf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff38806fe40 tx=0x7ff380019000 comp rx=0 tx=0).stop 2023-10-06T03:34:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.542+0000 7ff38eb9d700 1 -- 172.21.15.55:0/4187400860 shutdown_connections 2023-10-06T03:34:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.542+0000 7ff38eb9d700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3880b02a0 0x7ff3880c8bf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.542+0000 7ff38eb9d700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff36c066f30 0x7ff36c0693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.542+0000 7ff38eb9d700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff38806a0a0 0x7ff388165a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.542+0000 7ff38eb9d700 1 --2- 172.21.15.55:0/4187400860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff38806d950 0x7ff388142c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.542+0000 7ff38eb9d700 1 -- 172.21.15.55:0/4187400860 >> 172.21.15.55:0/4187400860 conn(0x7ff3880cc1f0 msgr2=0x7ff3880b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.542+0000 7ff38eb9d700 1 -- 172.21.15.55:0/4187400860 shutdown_connections 2023-10-06T03:34:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.542+0000 7ff38eb9d700 1 -- 172.21.15.55:0/4187400860 wait complete. 2023-10-06T03:34:10.561 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:34:10.561 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:34:10.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.751+0000 7ff5dcf9d700 1 Processor -- start 2023-10-06T03:34:10.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.751+0000 7ff5dcf9d700 1 -- start start 2023-10-06T03:34:10.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.752+0000 7ff5dcf9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5d80b03c0 0x7ff5d80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:10.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.752+0000 7ff5dcf9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5d81591e0 0x7ff5d8159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:10.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.752+0000 7ff5dcf9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d80cfa00 con 0x7ff5d80b0790 2023-10-06T03:34:10.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.752+0000 7ff5dcf9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d80cb580 con 0x7ff5d81591e0 2023-10-06T03:34:10.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.752+0000 7ff5dcf9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d80cf160 con 0x7ff5d80b03c0 2023-10-06T03:34:10.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.753+0000 7ff5d77fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5d80b03c0 0x7ff5d80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:10.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.753+0000 7ff5d7fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5d81591e0 0x7ff5d8159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:10.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.753+0000 7ff5d77fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5d80b03c0 0x7ff5d80cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:43560/0 (socket says 172.21.15.55:43560) 2023-10-06T03:34:10.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.753+0000 7ff5d77fe700 1 -- 172.21.15.55:0/4242818039 learned_addr learned my addr 172.21.15.55:0/4242818039 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:10.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.753+0000 7ff5d7fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5d81591e0 0x7ff5d8159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:40318/0 (socket says 172.21.15.55:40318) 2023-10-06T03:34:10.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.753+0000 7ff5d6ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff5d80b0790 0x7ff5d80b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:34908/0 (socket says 172.21.15.55:34908) 2023-10-06T03:34:10.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.753+0000 7ff5d77fe700 1 -- 172.21.15.55:0/4242818039 >> v2:172.21.15.163:3300/0 conn(0x7ff5d81591e0 msgr2=0x7ff5d8159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:10.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.753+0000 7ff5d77fe700 1 --2- 172.21.15.55:0/4242818039 >> v2:172.21.15.163:3300/0 conn(0x7ff5d81591e0 0x7ff5d8159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.753+0000 7ff5d77fe700 1 -- 172.21.15.55:0/4242818039 >> v1:172.21.15.163:6789/0 conn(0x7ff5d80b0790 legacy=0x7ff5d80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:10.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.753+0000 7ff5d77fe700 1 -- 172.21.15.55:0/4242818039 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d80cf440 con 0x7ff5d80b03c0 2023-10-06T03:34:10.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.754+0000 7ff5d77fe700 1 --2- 172.21.15.55:0/4242818039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5d80b03c0 0x7ff5d80cea70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80099b0 tx=0x7ff5c8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7fc4997a89885cb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:10.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.754+0000 7ff5d67fc700 1 -- 172.21.15.55:0/4242818039 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c8009ea0 con 0x7ff5d80b03c0 2023-10-06T03:34:10.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.754+0000 7ff5d67fc700 1 -- 172.21.15.55:0/4242818039 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c800b860 con 0x7ff5d80b03c0 2023-10-06T03:34:10.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.754+0000 7ff5d67fc700 1 -- 172.21.15.55:0/4242818039 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800bc00 con 0x7ff5d80b03c0 2023-10-06T03:34:10.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.754+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/4242818039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5d80b03c0 msgr2=0x7ff5d80cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:10.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.755+0000 7ff5dcf9d700 1 --2- 172.21.15.55:0/4242818039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5d80b03c0 0x7ff5d80cea70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80099b0 tx=0x7ff5c8009250 comp rx=0 tx=0).stop 2023-10-06T03:34:10.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.755+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/4242818039 shutdown_connections 2023-10-06T03:34:10.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.755+0000 7ff5dcf9d700 1 --2- 172.21.15.55:0/4242818039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5d80b03c0 0x7ff5d80cea70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.755+0000 7ff5dcf9d700 1 --2- 172.21.15.55:0/4242818039 >> v2:172.21.15.163:3300/0 conn(0x7ff5d81591e0 0x7ff5d8159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.755+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/4242818039 >> 172.21.15.55:0/4242818039 conn(0x7ff5d8063230 msgr2=0x7ff5d8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:10.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.755+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/4242818039 shutdown_connections 2023-10-06T03:34:10.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.755+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/4242818039 wait complete. 2023-10-06T03:34:10.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.756+0000 7ff5dcf9d700 1 Processor -- start 2023-10-06T03:34:10.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.756+0000 7ff5dcf9d700 1 -- start start 2023-10-06T03:34:10.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.757+0000 7ff5dcf9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d80b03c0 0x7ff5d80af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:10.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.757+0000 7ff5dcf9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5d80b0790 0x7ff5d80c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:10.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.757+0000 7ff5dcf9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5d81591e0 0x7ff5d815c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:10.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.757+0000 7ff5dcf9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d80c70a0 con 0x7ff5d80b03c0 2023-10-06T03:34:10.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.757+0000 7ff5dcf9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d80c3fe0 con 0x7ff5d80b0790 2023-10-06T03:34:10.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.757+0000 7ff5dcf9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d80c6800 con 0x7ff5d81591e0 2023-10-06T03:34:10.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.758+0000 7ff5d77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d80b03c0 0x7ff5d80af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:10.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.758+0000 7ff5d77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d80b03c0 0x7ff5d80af550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53850/0 (socket says 172.21.15.55:53850) 2023-10-06T03:34:10.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.758+0000 7ff5d77fe700 1 -- 172.21.15.55:0/657624327 learned_addr learned my addr 172.21.15.55:0/657624327 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:10.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.758+0000 7ff5d6ffd700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5d80b0790 0x7ff5d80c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:10.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.758+0000 7ff5d7fff700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5d81591e0 0x7ff5d815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:10.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.758+0000 7ff5d77fe700 1 -- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5d81591e0 msgr2=0x7ff5d815c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:10.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.758+0000 7ff5d77fe700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5d81591e0 0x7ff5d815c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.758+0000 7ff5d77fe700 1 -- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5d80b0790 msgr2=0x7ff5d80c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:10.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.758+0000 7ff5d77fe700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5d80b0790 0x7ff5d80c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:10.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.758+0000 7ff5d77fe700 1 -- 172.21.15.55:0/657624327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c80094f0 con 0x7ff5d80b03c0 2023-10-06T03:34:10.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.758+0000 7ff5d7fff700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5d81591e0 0x7ff5d815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:10.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.758+0000 7ff5d77fe700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d80b03c0 0x7ff5d80af550 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80cf2a0 tx=0x7ff5c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:10.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.759+0000 7ff5bf7fe700 1 -- 172.21.15.55:0/657624327 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800bbe0 con 0x7ff5d80b03c0 2023-10-06T03:34:10.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.759+0000 7ff5bf7fe700 1 -- 172.21.15.55:0/657624327 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c8011420 con 0x7ff5d80b03c0 2023-10-06T03:34:10.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.759+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/657624327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d80c6ae0 con 0x7ff5d80b03c0 2023-10-06T03:34:10.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.759+0000 7ff5bf7fe700 1 -- 172.21.15.55:0/657624327 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c80117c0 con 0x7ff5d80b03c0 2023-10-06T03:34:10.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.759+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/657624327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d80afa90 con 0x7ff5d80b03c0 2023-10-06T03:34:10.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.760+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/657624327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d804c3c0 con 0x7ff5d80b03c0 2023-10-06T03:34:10.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.761+0000 7ff5bf7fe700 1 -- 172.21.15.55:0/657624327 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5c8011bf0 con 0x7ff5d80b03c0 2023-10-06T03:34:10.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.761+0000 7ff5bf7fe700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff5c4066f30 0x7ff5c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:10.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.762+0000 7ff5bf7fe700 1 -- 172.21.15.55:0/657624327 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5206+0+0 (secure 0 0 0) 0x7ff5c80858d0 con 0x7ff5d80b03c0 2023-10-06T03:34:10.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.762+0000 7ff5d6ffd700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff5c4066f30 0x7ff5c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:10.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.763+0000 7ff5d6ffd700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff5c4066f30 0x7ff5c40693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc007fd0 tx=0x7ff5cc007d70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:10.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.766+0000 7ff5bf7fe700 1 -- 172.21.15.55:0/657624327 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5c8052d10 con 0x7ff5d80b03c0 2023-10-06T03:34:10.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:10.966+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/657624327 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff5d80cd5f0 con 0x7ff5d80b03c0 2023-10-06T03:34:11.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.504+0000 7ff5bf7fe700 1 -- 172.21.15.55:0/657624327 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff5c8056330 con 0x7ff5d80b03c0 2023-10-06T03:34:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.509+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff5c4066f30 msgr2=0x7ff5c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.509+0000 7ff5dcf9d700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff5c4066f30 0x7ff5c40693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc007fd0 tx=0x7ff5cc007d70 comp rx=0 tx=0).stop 2023-10-06T03:34:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.509+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/657624327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d80b03c0 msgr2=0x7ff5d80af550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:11.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.509+0000 7ff5dcf9d700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d80b03c0 0x7ff5d80af550 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80cf2a0 tx=0x7ff5c8017000 comp rx=0 tx=0).stop 2023-10-06T03:34:11.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.509+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/657624327 shutdown_connections 2023-10-06T03:34:11.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.509+0000 7ff5dcf9d700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff5c4066f30 0x7ff5c40693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:11.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.509+0000 7ff5dcf9d700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5d81591e0 0x7ff5d815c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.509+0000 7ff5dcf9d700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d80b03c0 0x7ff5d80af550 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.509+0000 7ff5dcf9d700 1 --2- 172.21.15.55:0/657624327 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5d80b0790 0x7ff5d80c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.509+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/657624327 >> 172.21.15.55:0/657624327 conn(0x7ff5d8063230 msgr2=0x7ff5d80b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.509+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/657624327 shutdown_connections 2023-10-06T03:34:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.509+0000 7ff5dcf9d700 1 -- 172.21.15.55:0/657624327 wait complete. 2023-10-06T03:34:11.514 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2023-10-06T03:34:11.534 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:34:11.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.729+0000 7f734359e700 1 Processor -- start 2023-10-06T03:34:11.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.729+0000 7f734359e700 1 -- start start 2023-10-06T03:34:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.730+0000 7f734359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f73441595d0 0x7f73440ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.730+0000 7f734359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734415d930 0x7f73440b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.731+0000 7f734359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73440b10f0 con 0x7f73440cf480 2023-10-06T03:34:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.731+0000 7f734359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73440cd530 con 0x7f73441595d0 2023-10-06T03:34:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.731+0000 7f734359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734415e000 con 0x7f734415d930 2023-10-06T03:34:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.731+0000 7f734259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f73441595d0 0x7f73440ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.731+0000 7f734259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f73441595d0 0x7f73440ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53860/0 (socket says 172.21.15.55:53860) 2023-10-06T03:34:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.731+0000 7f734259c700 1 -- 172.21.15.55:0/3144547292 learned_addr learned my addr 172.21.15.55:0/3144547292 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.731+0000 7f7341d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f73440cf480 0x7f734415d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38204/0 (socket says 172.21.15.55:38204) 2023-10-06T03:34:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.731+0000 7f7342d9d700 1 --2- 172.21.15.55:0/3144547292 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734415d930 0x7f73440b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.731+0000 7f734259c700 1 -- 172.21.15.55:0/3144547292 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734415d930 msgr2=0x7f73440b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.731+0000 7f734259c700 1 --2- 172.21.15.55:0/3144547292 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734415d930 0x7f73440b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.731+0000 7f734259c700 1 -- 172.21.15.55:0/3144547292 >> v1:172.21.15.55:6789/0 conn(0x7f73440cf480 legacy=0x7f734415d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.731+0000 7f734259c700 1 -- 172.21.15.55:0/3144547292 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73440cf990 con 0x7f73441595d0 2023-10-06T03:34:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.731+0000 7f734159a700 1 -- 172.21.15.55:0/3144547292 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 721889685 0 0) 0x7f73440b10f0 con 0x7f73440cf480 2023-10-06T03:34:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.732+0000 7f734259c700 1 --2- 172.21.15.55:0/3144547292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73441595d0 0x7f73440ccff0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7334009990 tx=0x7f7334009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aceb7fc27e7d20d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.732+0000 7f734159a700 1 -- 172.21.15.55:0/3144547292 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334005c10 con 0x7f73441595d0 2023-10-06T03:34:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.732+0000 7f734159a700 1 -- 172.21.15.55:0/3144547292 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7334005d70 con 0x7f73441595d0 2023-10-06T03:34:11.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.732+0000 7f734159a700 1 -- 172.21.15.55:0/3144547292 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733400b880 con 0x7f73441595d0 2023-10-06T03:34:11.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.732+0000 7f734359e700 1 -- 172.21.15.55:0/3144547292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73441595d0 msgr2=0x7f73440ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.732+0000 7f734359e700 1 --2- 172.21.15.55:0/3144547292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73441595d0 0x7f73440ccff0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7334009990 tx=0x7f7334009220 comp rx=0 tx=0).stop 2023-10-06T03:34:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.733+0000 7f734359e700 1 -- 172.21.15.55:0/3144547292 shutdown_connections 2023-10-06T03:34:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.733+0000 7f734359e700 1 --2- 172.21.15.55:0/3144547292 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734415d930 0x7f73440b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.733+0000 7f734359e700 1 --2- 172.21.15.55:0/3144547292 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73441595d0 0x7f73440ccff0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.733+0000 7f734359e700 1 -- 172.21.15.55:0/3144547292 >> 172.21.15.55:0/3144547292 conn(0x7f73440d05f0 msgr2=0x7f73440d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.733+0000 7f734359e700 1 -- 172.21.15.55:0/3144547292 shutdown_connections 2023-10-06T03:34:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.733+0000 7f734359e700 1 -- 172.21.15.55:0/3144547292 wait complete. 2023-10-06T03:34:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.734+0000 7f734359e700 1 Processor -- start 2023-10-06T03:34:11.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.734+0000 7f734359e700 1 -- start start 2023-10-06T03:34:11.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.735+0000 7f734359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73440cf480 0x7f73441523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:11.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.735+0000 7f734359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73441595d0 0x7f7344155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:11.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.735+0000 7f734359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734415d930 0x7f73441535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:11.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.735+0000 7f734359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73440c65f0 con 0x7f73441595d0 2023-10-06T03:34:11.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.735+0000 7f734359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7344156e30 con 0x7f73440cf480 2023-10-06T03:34:11.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.735+0000 7f734359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73440ccad0 con 0x7f734415d930 2023-10-06T03:34:11.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.735+0000 7f7341d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73441595d0 0x7f7344155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:11.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.735+0000 7f7341d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73441595d0 0x7f7344155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53872/0 (socket says 172.21.15.55:53872) 2023-10-06T03:34:11.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.735+0000 7f7341d9b700 1 -- 172.21.15.55:0/3140401197 learned_addr learned my addr 172.21.15.55:0/3140401197 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:11.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.736+0000 7f7342d9d700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734415d930 0x7f73441535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:11.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.736+0000 7f734259c700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73440cf480 0x7f73441523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:11.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.736+0000 7f7341d9b700 1 -- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734415d930 msgr2=0x7f73441535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:11.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.736+0000 7f7341d9b700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734415d930 0x7f73441535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:11.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.736+0000 7f7341d9b700 1 -- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73440cf480 msgr2=0x7f73441523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:11.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.736+0000 7f7341d9b700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73440cf480 0x7f73441523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:11.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.736+0000 7f7341d9b700 1 -- 172.21.15.55:0/3140401197 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7334009480 con 0x7f73441595d0 2023-10-06T03:34:11.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.736+0000 7f734259c700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73440cf480 0x7f73441523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:11.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.736+0000 7f7341d9b700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73441595d0 0x7f7344155640 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f733800c540 tx=0x7f733800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:11.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.736+0000 7f73327fc700 1 -- 172.21.15.55:0/3140401197 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7338005a70 con 0x7f73441595d0 2023-10-06T03:34:11.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.736+0000 7f73327fc700 1 -- 172.21.15.55:0/3140401197 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7338005bf0 con 0x7f73441595d0 2023-10-06T03:34:11.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.736+0000 7f73327fc700 1 -- 172.21.15.55:0/3140401197 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73380070d0 con 0x7f73441595d0 2023-10-06T03:34:11.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.736+0000 7f734359e700 1 -- 172.21.15.55:0/3140401197 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73440c5d50 con 0x7f73441595d0 2023-10-06T03:34:11.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.736+0000 7f734359e700 1 -- 172.21.15.55:0/3140401197 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73440c6310 con 0x7f73441595d0 2023-10-06T03:34:11.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.737+0000 7f734359e700 1 -- 172.21.15.55:0/3140401197 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734404c3c0 con 0x7f73441595d0 2023-10-06T03:34:11.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.739+0000 7f73327fc700 1 -- 172.21.15.55:0/3140401197 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7338007480 con 0x7f73441595d0 2023-10-06T03:34:11.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.740+0000 7f73327fc700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7324066f30 0x7f73240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:11.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.740+0000 7f73327fc700 1 -- 172.21.15.55:0/3140401197 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f7338083980 con 0x7f73441595d0 2023-10-06T03:34:11.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.741+0000 7f734259c700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7324066f30 0x7f73240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:11.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.741+0000 7f734259c700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7324066f30 0x7f73240693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7334009260 tx=0x7f7334016000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:11.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.745+0000 7f73327fc700 1 -- 172.21.15.55:0/3140401197 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7338050bd0 con 0x7f73441595d0 2023-10-06T03:34:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:11.947+0000 7f734359e700 1 -- 172.21.15.55:0/3140401197 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7344156f70 con 0x7f73441595d0 2023-10-06T03:34:12.046 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:34:12.051 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:34:12.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.505+0000 7f73327fc700 1 -- 172.21.15.55:0/3140401197 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7338017070 con 0x7f73441595d0 2023-10-06T03:34:12.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.510+0000 7f734359e700 1 -- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7324066f30 msgr2=0x7f73240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:12.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.510+0000 7f734359e700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7324066f30 0x7f73240693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7334009260 tx=0x7f7334016000 comp rx=0 tx=0).stop 2023-10-06T03:34:12.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.510+0000 7f734359e700 1 -- 172.21.15.55:0/3140401197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73441595d0 msgr2=0x7f7344155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:12.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.510+0000 7f734359e700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73441595d0 0x7f7344155640 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f733800c540 tx=0x7f733800c850 comp rx=0 tx=0).stop 2023-10-06T03:34:12.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.510+0000 7f734359e700 1 -- 172.21.15.55:0/3140401197 shutdown_connections 2023-10-06T03:34:12.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.510+0000 7f734359e700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7324066f30 0x7f73240693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:12.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.510+0000 7f734359e700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734415d930 0x7f73441535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:12.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.510+0000 7f734359e700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73440cf480 0x7f73441523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:12.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.510+0000 7f734359e700 1 --2- 172.21.15.55:0/3140401197 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73441595d0 0x7f7344155640 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:12.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.510+0000 7f734359e700 1 -- 172.21.15.55:0/3140401197 >> 172.21.15.55:0/3140401197 conn(0x7f73440d05f0 msgr2=0x7f7344152cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:12.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.512+0000 7f734359e700 1 -- 172.21.15.55:0/3140401197 shutdown_connections 2023-10-06T03:34:12.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.512+0000 7f734359e700 1 -- 172.21.15.55:0/3140401197 wait complete. 2023-10-06T03:34:12.522 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2023-10-06T03:34:12.547 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:34:12.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.737+0000 7f356759e700 1 Processor -- start 2023-10-06T03:34:12.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.738+0000 7f356759e700 1 -- start start 2023-10-06T03:34:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.739+0000 7f356759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35681407e0 0x7f356806a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.739+0000 7f356759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3568140bb0 0x7f356806a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.739+0000 7f356759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f356806d4f0 0x7f356806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.739+0000 7f356759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35680c6820 con 0x7f3568140bb0 2023-10-06T03:34:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.739+0000 7f356759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35680c6de0 con 0x7f356806d4f0 2023-10-06T03:34:12.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.739+0000 7f356759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356806de70 con 0x7f35681407e0 2023-10-06T03:34:12.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.739+0000 7f3565d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3568140bb0 0x7f356806a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:12.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.739+0000 7f3565d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3568140bb0 0x7f356806a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53884/0 (socket says 172.21.15.55:53884) 2023-10-06T03:34:12.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.739+0000 7f3565d9b700 1 -- 172.21.15.55:0/4157008029 learned_addr learned my addr 172.21.15.55:0/4157008029 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:12.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.739+0000 7f3566d9d700 1 --2- 172.21.15.55:0/4157008029 >> v2:172.21.15.163:3300/0 conn(0x7f356806d4f0 0x7f356806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:12.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.739+0000 7f356659c700 1 --2- 172.21.15.55:0/4157008029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35681407e0 0x7f356806a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:12.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.740+0000 7f3565d9b700 1 -- 172.21.15.55:0/4157008029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35681407e0 msgr2=0x7f356806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:12.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.740+0000 7f3565d9b700 1 --2- 172.21.15.55:0/4157008029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35681407e0 0x7f356806a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:12.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.740+0000 7f3565d9b700 1 -- 172.21.15.55:0/4157008029 >> v2:172.21.15.163:3300/0 conn(0x7f356806d4f0 msgr2=0x7f356806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:12.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.740+0000 7f3565d9b700 1 --2- 172.21.15.55:0/4157008029 >> v2:172.21.15.163:3300/0 conn(0x7f356806d4f0 0x7f356806d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:12.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.740+0000 7f3565d9b700 1 -- 172.21.15.55:0/4157008029 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3568141270 con 0x7f3568140bb0 2023-10-06T03:34:12.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.740+0000 7f3565d9b700 1 --2- 172.21.15.55:0/4157008029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3568140bb0 0x7f356806a6c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f356806ac00 tx=0x7f355c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1a11a1d48f6062 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:12.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.740+0000 7f356559a700 1 -- 172.21.15.55:0/4157008029 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c00b640 con 0x7f3568140bb0 2023-10-06T03:34:12.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.740+0000 7f356559a700 1 -- 172.21.15.55:0/4157008029 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c00b7a0 con 0x7f3568140bb0 2023-10-06T03:34:12.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.741+0000 7f356559a700 1 -- 172.21.15.55:0/4157008029 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c00bb60 con 0x7f3568140bb0 2023-10-06T03:34:12.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.741+0000 7f356759e700 1 -- 172.21.15.55:0/4157008029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3568140bb0 msgr2=0x7f356806a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:12.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.741+0000 7f356759e700 1 --2- 172.21.15.55:0/4157008029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3568140bb0 0x7f356806a6c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f356806ac00 tx=0x7f355c009250 comp rx=0 tx=0).stop 2023-10-06T03:34:12.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.741+0000 7f356759e700 1 -- 172.21.15.55:0/4157008029 shutdown_connections 2023-10-06T03:34:12.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.741+0000 7f356759e700 1 --2- 172.21.15.55:0/4157008029 >> v2:172.21.15.163:3300/0 conn(0x7f356806d4f0 0x7f356806d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:12.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.741+0000 7f356759e700 1 --2- 172.21.15.55:0/4157008029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35681407e0 0x7f356806a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:12.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.741+0000 7f356759e700 1 --2- 172.21.15.55:0/4157008029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3568140bb0 0x7f356806a6c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:12.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.741+0000 7f356759e700 1 -- 172.21.15.55:0/4157008029 >> 172.21.15.55:0/4157008029 conn(0x7f35680cc1e0 msgr2=0x7f35680cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:12.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.741+0000 7f356759e700 1 -- 172.21.15.55:0/4157008029 shutdown_connections 2023-10-06T03:34:12.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.741+0000 7f356759e700 1 -- 172.21.15.55:0/4157008029 wait complete. 2023-10-06T03:34:12.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.743+0000 7f356759e700 1 Processor -- start 2023-10-06T03:34:12.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.743+0000 7f356759e700 1 -- start start 2023-10-06T03:34:12.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f356759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f356806d4f0 0x7f3568157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:12.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f356759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35681407e0 0x7f356815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:12.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f356759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3568140bb0 0x7f356813b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:12.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f356759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35680c5cf0 con 0x7f356806d4f0 2023-10-06T03:34:12.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f356759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35680c6180 con 0x7f3568140bb0 2023-10-06T03:34:12.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f356759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356815dca0 con 0x7f35681407e0 2023-10-06T03:34:12.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f356659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f356806d4f0 0x7f3568157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:12.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f356659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f356806d4f0 0x7f3568157010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53888/0 (socket says 172.21.15.55:53888) 2023-10-06T03:34:12.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f356659c700 1 -- 172.21.15.55:0/209301795 learned_addr learned my addr 172.21.15.55:0/209301795 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:12.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f356659c700 1 -- 172.21.15.55:0/209301795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35681407e0 msgr2=0x7f356815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:12.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f356659c700 1 --2- 172.21.15.55:0/209301795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35681407e0 0x7f356815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:12.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f3566d9d700 1 --2- 172.21.15.55:0/209301795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3568140bb0 0x7f356813b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:12.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f356659c700 1 -- 172.21.15.55:0/209301795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3568140bb0 msgr2=0x7f356813b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:12.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f356659c700 1 --2- 172.21.15.55:0/209301795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3568140bb0 0x7f356813b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:12.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.744+0000 7f356659c700 1 -- 172.21.15.55:0/209301795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c0094f0 con 0x7f356806d4f0 2023-10-06T03:34:12.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.745+0000 7f356659c700 1 --2- 172.21.15.55:0/209301795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f356806d4f0 0x7f3568157010 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f355800ec80 tx=0x7f35580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.745+0000 7f35567fc700 1 -- 172.21.15.55:0/209301795 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355800c9d0 con 0x7f356806d4f0 2023-10-06T03:34:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.745+0000 7f35567fc700 1 -- 172.21.15.55:0/209301795 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355800cb50 con 0x7f356806d4f0 2023-10-06T03:34:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.745+0000 7f356759e700 1 -- 172.21.15.55:0/209301795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35680c56a0 con 0x7f356806d4f0 2023-10-06T03:34:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.745+0000 7f35567fc700 1 -- 172.21.15.55:0/209301795 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558013470 con 0x7f356806d4f0 2023-10-06T03:34:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.745+0000 7f356759e700 1 -- 172.21.15.55:0/209301795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3568157550 con 0x7f356806d4f0 2023-10-06T03:34:12.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.746+0000 7f356759e700 1 -- 172.21.15.55:0/209301795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35680c5fd0 con 0x7f356806d4f0 2023-10-06T03:34:12.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.747+0000 7f35567fc700 1 -- 172.21.15.55:0/209301795 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3558013860 con 0x7f356806d4f0 2023-10-06T03:34:12.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.748+0000 7f35567fc700 1 --2- 172.21.15.55:0/209301795 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3548066f30 0x7f35480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:12.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.748+0000 7f35567fc700 1 -- 172.21.15.55:0/209301795 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f3558085220 con 0x7f356806d4f0 2023-10-06T03:34:12.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.749+0000 7f3565d9b700 1 --2- 172.21.15.55:0/209301795 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3548066f30 0x7f35480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:12.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.749+0000 7f3565d9b700 1 --2- 172.21.15.55:0/209301795 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3548066f30 0x7f35480693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f356806ac00 tx=0x7f355c017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:12.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:12.751+0000 7f35567fc700 1 -- 172.21.15.55:0/209301795 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3558052370 con 0x7f356806d4f0 2023-10-06T03:34:13.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.001+0000 7f356759e700 1 -- 172.21.15.55:0/209301795 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f356806acf0 con 0x7f356806d4f0 2023-10-06T03:34:13.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.521+0000 7f35567fc700 1 -- 172.21.15.55:0/209301795 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3558055990 con 0x7f356806d4f0 2023-10-06T03:34:13.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.531+0000 7f356759e700 1 -- 172.21.15.55:0/209301795 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3548066f30 msgr2=0x7f35480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:13.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.531+0000 7f356759e700 1 --2- 172.21.15.55:0/209301795 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3548066f30 0x7f35480693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f356806ac00 tx=0x7f355c017000 comp rx=0 tx=0).stop 2023-10-06T03:34:13.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.531+0000 7f356759e700 1 -- 172.21.15.55:0/209301795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f356806d4f0 msgr2=0x7f3568157010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:13.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.531+0000 7f356759e700 1 --2- 172.21.15.55:0/209301795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f356806d4f0 0x7f3568157010 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f355800ec80 tx=0x7f35580091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.532+0000 7f356759e700 1 -- 172.21.15.55:0/209301795 shutdown_connections 2023-10-06T03:34:13.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.533+0000 7f356759e700 1 --2- 172.21.15.55:0/209301795 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3568140bb0 0x7f356813b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.533+0000 7f356759e700 1 --2- 172.21.15.55:0/209301795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f356806d4f0 0x7f3568157010 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.533+0000 7f356759e700 1 --2- 172.21.15.55:0/209301795 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3548066f30 0x7f35480693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.535+0000 7f356759e700 1 --2- 172.21.15.55:0/209301795 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35681407e0 0x7f356815d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.535+0000 7f356759e700 1 -- 172.21.15.55:0/209301795 >> 172.21.15.55:0/209301795 conn(0x7f35680cc1e0 msgr2=0x7f35680c7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:13.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.542+0000 7f356759e700 1 -- 172.21.15.55:0/209301795 shutdown_connections 2023-10-06T03:34:13.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.542+0000 7f356759e700 1 -- 172.21.15.55:0/209301795 wait complete. 2023-10-06T03:34:13.549 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 5 and data pool 6 2023-10-06T03:34:13.570 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:34:13.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.757+0000 7ff315d9d700 1 Processor -- start 2023-10-06T03:34:13.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.757+0000 7ff315d9d700 1 -- start start 2023-10-06T03:34:13.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.758+0000 7ff315d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff310150180 0x7ff310157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:13.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.758+0000 7ff315d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff310150550 0x7ff3100c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:13.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.758+0000 7ff315d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31006e2a0 con 0x7ff3100c7240 2023-10-06T03:34:13.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.758+0000 7ff315d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3100c76c0 con 0x7ff310150180 2023-10-06T03:34:13.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.758+0000 7ff315d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31006b2e0 con 0x7ff310150550 2023-10-06T03:34:13.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.758+0000 7ff314d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff310150180 0x7ff310157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:13.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.758+0000 7ff314d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff310150180 0x7ff310157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39748/0 (socket says 172.21.15.55:39748) 2023-10-06T03:34:13.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.758+0000 7ff314d9b700 1 -- 172.21.15.55:0/3740299719 learned_addr learned my addr 172.21.15.55:0/3740299719 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:13.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.758+0000 7ff31559c700 1 --1- 172.21.15.55:0/3740299719 >> v1:172.21.15.55:6789/0 conn(0x7ff3100c7240 0x7ff310069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37954/0 (socket says 172.21.15.55:37954) 2023-10-06T03:34:13.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.759+0000 7ff30ffff700 1 --2- 172.21.15.55:0/3740299719 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff310150550 0x7ff3100c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:13.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.759+0000 7ff314d9b700 1 -- 172.21.15.55:0/3740299719 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff310150550 msgr2=0x7ff3100c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:13.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.759+0000 7ff314d9b700 1 --2- 172.21.15.55:0/3740299719 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff310150550 0x7ff3100c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.759+0000 7ff314d9b700 1 -- 172.21.15.55:0/3740299719 >> v1:172.21.15.55:6789/0 conn(0x7ff3100c7240 legacy=0x7ff310069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:13.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.759+0000 7ff314d9b700 1 -- 172.21.15.55:0/3740299719 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3100c78e0 con 0x7ff310150180 2023-10-06T03:34:13.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.759+0000 7ff30ffff700 1 --2- 172.21.15.55:0/3740299719 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff310150550 0x7ff3100c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:13.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.759+0000 7ff314d9b700 1 --2- 172.21.15.55:0/3740299719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff310150180 0x7ff310157770 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff30400bcb0 tx=0x7ff30400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d94c8be4fbd3b37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:13.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.759+0000 7ff30f7fe700 1 -- 172.21.15.55:0/3740299719 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30400dd80 con 0x7ff310150180 2023-10-06T03:34:13.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.759+0000 7ff30f7fe700 1 -- 172.21.15.55:0/3740299719 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff304019480 con 0x7ff310150180 2023-10-06T03:34:13.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.759+0000 7ff30f7fe700 1 -- 172.21.15.55:0/3740299719 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304019820 con 0x7ff310150180 2023-10-06T03:34:13.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.760+0000 7ff315d9d700 1 -- 172.21.15.55:0/3740299719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff310150180 msgr2=0x7ff310157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:13.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.760+0000 7ff315d9d700 1 --2- 172.21.15.55:0/3740299719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff310150180 0x7ff310157770 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff30400bcb0 tx=0x7ff30400b4f0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.760+0000 7ff315d9d700 1 -- 172.21.15.55:0/3740299719 shutdown_connections 2023-10-06T03:34:13.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.760+0000 7ff315d9d700 1 --2- 172.21.15.55:0/3740299719 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff310150180 0x7ff310157770 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.760+0000 7ff315d9d700 1 --2- 172.21.15.55:0/3740299719 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff310150550 0x7ff3100c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.760+0000 7ff315d9d700 1 -- 172.21.15.55:0/3740299719 >> 172.21.15.55:0/3740299719 conn(0x7ff3100cc1f0 msgr2=0x7ff3100cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.760+0000 7ff315d9d700 1 -- 172.21.15.55:0/3740299719 shutdown_connections 2023-10-06T03:34:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.760+0000 7ff315d9d700 1 -- 172.21.15.55:0/3740299719 wait complete. 2023-10-06T03:34:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.762+0000 7ff315d9d700 1 Processor -- start 2023-10-06T03:34:13.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.762+0000 7ff315d9d700 1 -- start start 2023-10-06T03:34:13.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff315d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3100c7240 0x7ff31014b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff315d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff310150180 0x7ff3100aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff315d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff310150550 0x7ff310158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff315d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3100b0620 con 0x7ff3100c7240 2023-10-06T03:34:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff315d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3100afa10 con 0x7ff310150180 2023-10-06T03:34:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff315d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3100afcf0 con 0x7ff310150550 2023-10-06T03:34:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff314d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3100c7240 0x7ff31014b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff314d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3100c7240 0x7ff31014b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39752/0 (socket says 172.21.15.55:39752) 2023-10-06T03:34:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff314d9b700 1 -- 172.21.15.55:0/2177916233 learned_addr learned my addr 172.21.15.55:0/2177916233 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff314d9b700 1 -- 172.21.15.55:0/2177916233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff310150550 msgr2=0x7ff310158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff30ffff700 1 --2- 172.21.15.55:0/2177916233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff310150180 0x7ff3100aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:13.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff314d9b700 1 --2- 172.21.15.55:0/2177916233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff310150550 0x7ff310158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff314d9b700 1 -- 172.21.15.55:0/2177916233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff310150180 msgr2=0x7ff3100aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff314d9b700 1 --2- 172.21.15.55:0/2177916233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff310150180 0x7ff3100aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff314d9b700 1 -- 172.21.15.55:0/2177916233 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30400b790 con 0x7ff3100c7240 2023-10-06T03:34:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.763+0000 7ff30ffff700 1 --2- 172.21.15.55:0/2177916233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff310150180 0x7ff3100aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.764+0000 7ff314d9b700 1 --2- 172.21.15.55:0/2177916233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3100c7240 0x7ff31014b800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff30400b530 tx=0x7ff30401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.764+0000 7ff30cff9700 1 -- 172.21.15.55:0/2177916233 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304006bc0 con 0x7ff3100c7240 2023-10-06T03:34:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.764+0000 7ff30cff9700 1 -- 172.21.15.55:0/2177916233 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3040107d0 con 0x7ff3100c7240 2023-10-06T03:34:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.764+0000 7ff315d9d700 1 -- 172.21.15.55:0/2177916233 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3100affd0 con 0x7ff3100c7240 2023-10-06T03:34:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.764+0000 7ff30cff9700 1 -- 172.21.15.55:0/2177916233 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304010b90 con 0x7ff3100c7240 2023-10-06T03:34:13.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.764+0000 7ff315d9d700 1 -- 172.21.15.55:0/2177916233 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3100ad540 con 0x7ff3100c7240 2023-10-06T03:34:13.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.765+0000 7ff315d9d700 1 -- 172.21.15.55:0/2177916233 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31004c3c0 con 0x7ff3100c7240 2023-10-06T03:34:13.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.766+0000 7ff30cff9700 1 -- 172.21.15.55:0/2177916233 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff304022410 con 0x7ff3100c7240 2023-10-06T03:34:13.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.767+0000 7ff30cff9700 1 --2- 172.21.15.55:0/2177916233 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff2f4066f30 0x7ff2f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:13.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.767+0000 7ff30cff9700 1 -- 172.21.15.55:0/2177916233 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff304014070 con 0x7ff3100c7240 2023-10-06T03:34:13.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.767+0000 7ff30ffff700 1 --2- 172.21.15.55:0/2177916233 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff2f4066f30 0x7ff2f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:13.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.768+0000 7ff30ffff700 1 --2- 172.21.15.55:0/2177916233 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff2f4066f30 0x7ff2f40693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc006f00 tx=0x7ff2fc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:13.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.771+0000 7ff30cff9700 1 -- 172.21.15.55:0/2177916233 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3040536b0 con 0x7ff3100c7240 2023-10-06T03:34:13.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.973+0000 7ff315d9d700 1 -- 172.21.15.55:0/2177916233 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff31006ada0 con 0x7ff3100c7240 2023-10-06T03:34:13.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.974+0000 7ff30cff9700 1 -- 172.21.15.55:0/2177916233 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16927 (secure 0 0 0) 0x7ff304056cd0 con 0x7ff3100c7240 2023-10-06T03:34:13.975 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:13.976 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":31,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:34:13.002684+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:34:11.006235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T03:34:11.987296+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:34:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.976+0000 7ff315d9d700 1 -- 172.21.15.55:0/2177916233 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff2f4066f30 msgr2=0x7ff2f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.976+0000 7ff315d9d700 1 --2- 172.21.15.55:0/2177916233 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff2f4066f30 0x7ff2f40693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc006f00 tx=0x7ff2fc008040 comp rx=0 tx=0).stop 2023-10-06T03:34:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.976+0000 7ff315d9d700 1 -- 172.21.15.55:0/2177916233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3100c7240 msgr2=0x7ff31014b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.976+0000 7ff315d9d700 1 --2- 172.21.15.55:0/2177916233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3100c7240 0x7ff31014b800 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff30400b530 tx=0x7ff30401b000 comp rx=0 tx=0).stop 2023-10-06T03:34:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.976+0000 7ff315d9d700 1 -- 172.21.15.55:0/2177916233 shutdown_connections 2023-10-06T03:34:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.976+0000 7ff315d9d700 1 --2- 172.21.15.55:0/2177916233 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff2f4066f30 0x7ff2f40693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.976+0000 7ff315d9d700 1 --2- 172.21.15.55:0/2177916233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff310150550 0x7ff310158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.976+0000 7ff315d9d700 1 --2- 172.21.15.55:0/2177916233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff3100c7240 0x7ff31014b800 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.976+0000 7ff315d9d700 1 --2- 172.21.15.55:0/2177916233 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff310150180 0x7ff3100aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:13.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.976+0000 7ff315d9d700 1 -- 172.21.15.55:0/2177916233 >> 172.21.15.55:0/2177916233 conn(0x7ff3100cc1f0 msgr2=0x7ff3101559f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:13.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.976+0000 7ff315d9d700 1 -- 172.21.15.55:0/2177916233 shutdown_connections 2023-10-06T03:34:13.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:13.976+0000 7ff315d9d700 1 -- 172.21.15.55:0/2177916233 wait complete. 2023-10-06T03:34:13.995 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:34:14.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.180+0000 7feff699d700 1 Processor -- start 2023-10-06T03:34:14.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.180+0000 7feff699d700 1 -- start start 2023-10-06T03:34:14.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.182+0000 7feff699d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff0150550 0x7feff0157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:14.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.182+0000 7feff699d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff006e2a0 con 0x7feff00c7410 2023-10-06T03:34:14.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.182+0000 7feff699d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff006b2e0 con 0x7feff0150180 2023-10-06T03:34:14.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.182+0000 7feff699d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff00c78e0 con 0x7feff0150550 2023-10-06T03:34:14.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.182+0000 7feff619c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7feff00c7410 0x7feff0069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37964/0 (socket says 172.21.15.55:37964) 2023-10-06T03:34:14.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.182+0000 7feff619c700 1 -- 172.21.15.55:0/1895728956 learned_addr learned my addr 172.21.15.55:0/1895728956 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:14.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.182+0000 7feff519a700 1 --2- 172.21.15.55:0/1895728956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff0150550 0x7feff0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:14.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.182+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994744847 0 0) 0x7feff006e2a0 con 0x7feff00c7410 2023-10-06T03:34:14.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefdc003570 con 0x7feff00c7410 2023-10-06T03:34:14.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806846073 0 0) 0x7feff006b2e0 con 0x7feff0150180 2023-10-06T03:34:14.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feff006e2a0 con 0x7feff0150180 2023-10-06T03:34:14.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2062594819 0 0) 0x7fefdc003570 con 0x7feff00c7410 2023-10-06T03:34:14.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff0150550 msgr2=0x7feff0157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:14.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 --2- 172.21.15.55:0/1895728956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff0150550 0x7feff0157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 >> v1:172.21.15.163:6789/0 conn(0x7feff0150180 legacy=0x7feff00c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:14.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff00c7bc0 con 0x7feff00c7410 2023-10-06T03:34:14.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fefe80017f0 con 0x7feff00c7410 2023-10-06T03:34:14.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 >> v1:172.21.15.55:6789/0 conn(0x7feff00c7410 legacy=0x7feff0069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:14.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 --2- 172.21.15.55:0/1895728956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc003c20 0x7fefdc004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:14.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 --2- 172.21.15.55:0/1895728956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff0150550 0x7fefdc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:14.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 --2- 172.21.15.55:0/1895728956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefdc004d50 0x7fefdc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:14.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00c78e0 con 0x7fefdc004d50 2023-10-06T03:34:14.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc00bbf0 con 0x7fefdc003c20 2023-10-06T03:34:14.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc00bd60 con 0x7feff0150550 2023-10-06T03:34:14.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff619c700 1 --2- 172.21.15.55:0/1895728956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefdc004d50 0x7fefdc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:14.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff599b700 1 --2- 172.21.15.55:0/1895728956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc003c20 0x7fefdc004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:14.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.183+0000 7feff519a700 1 --2- 172.21.15.55:0/1895728956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff0150550 0x7fefdc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:14.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.184+0000 7feff619c700 1 -- 172.21.15.55:0/1895728956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff0150550 msgr2=0x7fefdc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:14.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.184+0000 7feff619c700 1 --2- 172.21.15.55:0/1895728956 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff0150550 0x7fefdc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.184+0000 7feff619c700 1 -- 172.21.15.55:0/1895728956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc003c20 msgr2=0x7fefdc004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:14.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.184+0000 7feff619c700 1 --2- 172.21.15.55:0/1895728956 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc003c20 0x7fefdc004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.184+0000 7feff619c700 1 -- 172.21.15.55:0/1895728956 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc003570 con 0x7fefdc004d50 2023-10-06T03:34:14.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.184+0000 7feff619c700 1 --2- 172.21.15.55:0/1895728956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefdc004d50 0x7fefdc0051c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feff00aed30 tx=0x7fefe80020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69dd6326e3bd531c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:14.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.184+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe80104a0 con 0x7fefdc004d50 2023-10-06T03:34:14.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.184+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe8010600 con 0x7fefdc004d50 2023-10-06T03:34:14.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.184+0000 7feff4999700 1 -- 172.21.15.55:0/1895728956 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8010980 con 0x7fefdc004d50 2023-10-06T03:34:14.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.185+0000 7feff699d700 1 -- 172.21.15.55:0/1895728956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefdc004d50 msgr2=0x7fefdc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:14.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.185+0000 7feff699d700 1 --2- 172.21.15.55:0/1895728956 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefdc004d50 0x7fefdc0051c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7feff00aed30 tx=0x7fefe80020c0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.185+0000 7feff619c700 1 -- 172.21.15.55:0/1895728956 reap_dead start 2023-10-06T03:34:14.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.185+0000 7feff699d700 1 -- 172.21.15.55:0/1895728956 shutdown_connections 2023-10-06T03:34:14.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.185+0000 7feff699d700 1 -- 172.21.15.55:0/1895728956 >> 172.21.15.55:0/1895728956 conn(0x7feff00cc1f0 msgr2=0x7feff00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:14.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.185+0000 7feff699d700 1 -- 172.21.15.55:0/1895728956 shutdown_connections 2023-10-06T03:34:14.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.185+0000 7feff699d700 1 -- 172.21.15.55:0/1895728956 wait complete. 2023-10-06T03:34:14.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.186+0000 7feff699d700 1 Processor -- start 2023-10-06T03:34:14.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.186+0000 7feff699d700 1 -- start start 2023-10-06T03:34:14.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.187+0000 7feff699d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc004d50 0x7feff00c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:14.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.187+0000 7feff699d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff00b0030 0x7feff00b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:14.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.187+0000 7feff699d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feff0165e10 0x7feff00c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:14.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.187+0000 7feff699d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00aef90 con 0x7feff0165e10 2023-10-06T03:34:14.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.187+0000 7feff699d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff00af270 con 0x7fefdc004d50 2023-10-06T03:34:14.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.187+0000 7feff699d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff00af550 con 0x7feff00b0030 2023-10-06T03:34:14.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.187+0000 7feff619c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feff0165e10 0x7feff00c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:14.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.188+0000 7feff619c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feff0165e10 0x7feff00c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39768/0 (socket says 172.21.15.55:39768) 2023-10-06T03:34:14.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.188+0000 7feff619c700 1 -- 172.21.15.55:0/3897269133 learned_addr learned my addr 172.21.15.55:0/3897269133 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:14.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.188+0000 7feff519a700 1 --2- 172.21.15.55:0/3897269133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff00b0030 0x7feff00b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:14.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.188+0000 7feff599b700 1 --2- 172.21.15.55:0/3897269133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc004d50 0x7feff00c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:14.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.188+0000 7feff619c700 1 -- 172.21.15.55:0/3897269133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff00b0030 msgr2=0x7feff00b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:14.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.188+0000 7feff619c700 1 --2- 172.21.15.55:0/3897269133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff00b0030 0x7feff00b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.188+0000 7feff619c700 1 -- 172.21.15.55:0/3897269133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc004d50 msgr2=0x7feff00c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:14.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.188+0000 7feff619c700 1 --2- 172.21.15.55:0/3897269133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc004d50 0x7feff00c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.188+0000 7feff619c700 1 -- 172.21.15.55:0/3897269133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff00af830 con 0x7feff0165e10 2023-10-06T03:34:14.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.188+0000 7feff619c700 1 --2- 172.21.15.55:0/3897269133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feff0165e10 0x7feff00c4560 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fefe8002100 tx=0x7fefe8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:14.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.188+0000 7fefe5ffb700 1 -- 172.21.15.55:0/3897269133 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8023070 con 0x7feff0165e10 2023-10-06T03:34:14.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.188+0000 7fefe5ffb700 1 -- 172.21.15.55:0/3897269133 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe8010cf0 con 0x7feff0165e10 2023-10-06T03:34:14.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.188+0000 7feff699d700 1 -- 172.21.15.55:0/3897269133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff00afb10 con 0x7feff0165e10 2023-10-06T03:34:14.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.188+0000 7fefe5ffb700 1 -- 172.21.15.55:0/3897269133 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe80104a0 con 0x7feff0165e10 2023-10-06T03:34:14.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.189+0000 7feff699d700 1 -- 172.21.15.55:0/3897269133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff00b1820 con 0x7feff0165e10 2023-10-06T03:34:14.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.189+0000 7feff699d700 1 -- 172.21.15.55:0/3897269133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff00afd00 con 0x7feff0165e10 2023-10-06T03:34:14.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.190+0000 7fefe5ffb700 1 -- 172.21.15.55:0/3897269133 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fefe801b460 con 0x7feff0165e10 2023-10-06T03:34:14.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.191+0000 7fefe5ffb700 1 --2- 172.21.15.55:0/3897269133 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fefd4066f30 0x7fefd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:14.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.191+0000 7fefe5ffb700 1 -- 172.21.15.55:0/3897269133 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fefe8087ba0 con 0x7feff0165e10 2023-10-06T03:34:14.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.191+0000 7feff599b700 1 --2- 172.21.15.55:0/3897269133 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fefd4066f30 0x7fefd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:14.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.192+0000 7feff599b700 1 --2- 172.21.15.55:0/3897269133 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fefd4066f30 0x7fefd40693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fefec003b80 tx=0x7fefec009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:14.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.195+0000 7fefe5ffb700 1 -- 172.21.15.55:0/3897269133 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fefe8054cb0 con 0x7feff0165e10 2023-10-06T03:34:14.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.399+0000 7feff699d700 1 -- 172.21.15.55:0/3897269133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feff004c3c0 con 0x7feff0165e10 2023-10-06T03:34:14.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.400+0000 7fefe5ffb700 1 -- 172.21.15.55:0/3897269133 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+16927 (secure 0 0 0) 0x7fefe80582d0 con 0x7feff0165e10 2023-10-06T03:34:14.402 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:14.402 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":31,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:34:13.002684+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:34:11.006235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T03:34:11.987296+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:34:14.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.403+0000 7feff699d700 1 -- 172.21.15.55:0/3897269133 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fefd4066f30 msgr2=0x7fefd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:14.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.403+0000 7feff699d700 1 --2- 172.21.15.55:0/3897269133 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fefd4066f30 0x7fefd40693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fefec003b80 tx=0x7fefec009040 comp rx=0 tx=0).stop 2023-10-06T03:34:14.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.403+0000 7feff699d700 1 -- 172.21.15.55:0/3897269133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feff0165e10 msgr2=0x7feff00c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:14.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.403+0000 7feff699d700 1 --2- 172.21.15.55:0/3897269133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feff0165e10 0x7feff00c4560 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fefe8002100 tx=0x7fefe8018000 comp rx=0 tx=0).stop 2023-10-06T03:34:14.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.403+0000 7feff699d700 1 -- 172.21.15.55:0/3897269133 shutdown_connections 2023-10-06T03:34:14.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.403+0000 7feff699d700 1 --2- 172.21.15.55:0/3897269133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feff0165e10 0x7feff00c4560 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.403+0000 7feff699d700 1 --2- 172.21.15.55:0/3897269133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefdc004d50 0x7feff00c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.403+0000 7feff699d700 1 --2- 172.21.15.55:0/3897269133 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fefd4066f30 0x7fefd40693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.403+0000 7feff699d700 1 --2- 172.21.15.55:0/3897269133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feff00b0030 0x7feff00b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.403+0000 7feff699d700 1 -- 172.21.15.55:0/3897269133 >> 172.21.15.55:0/3897269133 conn(0x7feff00cc1f0 msgr2=0x7feff00c20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:14.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.403+0000 7feff699d700 1 -- 172.21.15.55:0/3897269133 shutdown_connections 2023-10-06T03:34:14.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.403+0000 7feff699d700 1 -- 172.21.15.55:0/3897269133 wait complete. 2023-10-06T03:34:14.422 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:34:14.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.618+0000 7f5c4a79d700 1 Processor -- start 2023-10-06T03:34:14.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.618+0000 7f5c4a79d700 1 -- start start 2023-10-06T03:34:14.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.619+0000 7f5c4a79d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c440b16d0 0x7f5c440b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:14.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.619+0000 7f5c4a79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c440b33b0 0x7f5c440b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:14.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.619+0000 7f5c4a79d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c440b2080 con 0x7f5c440b4e70 2023-10-06T03:34:14.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.619+0000 7f5c4a79d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c440cfb00 con 0x7f5c440b16d0 2023-10-06T03:34:14.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.619+0000 7f5c4a79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c440b5c40 con 0x7f5c440b33b0 2023-10-06T03:34:14.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.620+0000 7f5c48f9a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c440b16d0 0x7f5c440b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:14.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.620+0000 7f5c49f9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c440b33b0 0x7f5c440b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:14.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.620+0000 7f5c48f9a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c440b16d0 0x7f5c440b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:32772/0 (socket says 172.21.15.55:32772) 2023-10-06T03:34:14.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.620+0000 7f5c48f9a700 1 -- 172.21.15.55:0/3480794981 learned_addr learned my addr 172.21.15.55:0/3480794981 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:14.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.620+0000 7f5c4979b700 1 --1- 172.21.15.55:0/3480794981 >> v1:172.21.15.163:6789/0 conn(0x7f5c440b4e70 0x7f5c440ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:42106/0 (socket says 172.21.15.55:42106) 2023-10-06T03:34:14.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.620+0000 7f5c48f9a700 1 -- 172.21.15.55:0/3480794981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c440b33b0 msgr2=0x7f5c440b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:14.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.620+0000 7f5c48f9a700 1 --2- 172.21.15.55:0/3480794981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c440b33b0 0x7f5c440b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.620+0000 7f5c48f9a700 1 -- 172.21.15.55:0/3480794981 >> v1:172.21.15.163:6789/0 conn(0x7f5c440b4e70 legacy=0x7f5c440ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.621+0000 7f5c48f9a700 1 -- 172.21.15.55:0/3480794981 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c440b5f20 con 0x7f5c440b16d0 2023-10-06T03:34:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.621+0000 7f5c49f9c700 1 --2- 172.21.15.55:0/3480794981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c440b33b0 0x7f5c440b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:34:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.621+0000 7f5c3bfff700 1 -- 172.21.15.55:0/3480794981 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3067883056 0 0) 0x7f5c440b2080 con 0x7f5c440b4e70 2023-10-06T03:34:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.621+0000 7f5c48f9a700 1 --2- 172.21.15.55:0/3480794981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c440b16d0 0x7f5c440b1b40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5c340099b0 tx=0x7f5c34009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b0ffcd881044aae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:14.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.621+0000 7f5c3bfff700 1 -- 172.21.15.55:0/3480794981 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34009ea0 con 0x7f5c440b16d0 2023-10-06T03:34:14.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.621+0000 7f5c3bfff700 1 -- 172.21.15.55:0/3480794981 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3400b860 con 0x7f5c440b16d0 2023-10-06T03:34:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.621+0000 7f5c3bfff700 1 -- 172.21.15.55:0/3480794981 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400bc20 con 0x7f5c440b16d0 2023-10-06T03:34:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.622+0000 7f5c4a79d700 1 -- 172.21.15.55:0/3480794981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c440b16d0 msgr2=0x7f5c440b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.622+0000 7f5c4a79d700 1 --2- 172.21.15.55:0/3480794981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c440b16d0 0x7f5c440b1b40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5c340099b0 tx=0x7f5c34009250 comp rx=0 tx=0).stop 2023-10-06T03:34:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.622+0000 7f5c4a79d700 1 -- 172.21.15.55:0/3480794981 shutdown_connections 2023-10-06T03:34:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.622+0000 7f5c4a79d700 1 --2- 172.21.15.55:0/3480794981 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c440b33b0 0x7f5c440b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.622+0000 7f5c4a79d700 1 --2- 172.21.15.55:0/3480794981 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c440b16d0 0x7f5c440b1b40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.622+0000 7f5c4a79d700 1 -- 172.21.15.55:0/3480794981 >> 172.21.15.55:0/3480794981 conn(0x7f5c440d10c0 msgr2=0x7f5c440affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.622+0000 7f5c4a79d700 1 -- 172.21.15.55:0/3480794981 shutdown_connections 2023-10-06T03:34:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.622+0000 7f5c4a79d700 1 -- 172.21.15.55:0/3480794981 wait complete. 2023-10-06T03:34:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.623+0000 7f5c4a79d700 1 Processor -- start 2023-10-06T03:34:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.623+0000 7f5c4a79d700 1 -- start start 2023-10-06T03:34:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.624+0000 7f5c4a79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c440b16d0 0x7f5c440a7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.624+0000 7f5c4a79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c440b33b0 0x7f5c440c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.624+0000 7f5c4a79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c440b4e70 0x7f5c440cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.624+0000 7f5c4a79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4416b150 con 0x7f5c440b4e70 2023-10-06T03:34:14.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.624+0000 7f5c4a79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c440c83e0 con 0x7f5c440b16d0 2023-10-06T03:34:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.624+0000 7f5c4a79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c440c8660 con 0x7f5c440b33b0 2023-10-06T03:34:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.624+0000 7f5c49f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c440b4e70 0x7f5c440cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.625+0000 7f5c49f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c440b4e70 0x7f5c440cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39772/0 (socket says 172.21.15.55:39772) 2023-10-06T03:34:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.625+0000 7f5c49f9c700 1 -- 172.21.15.55:0/98989762 learned_addr learned my addr 172.21.15.55:0/98989762 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.625+0000 7f5c48f9a700 1 --2- 172.21.15.55:0/98989762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c440b33b0 0x7f5c440c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.625+0000 7f5c4979b700 1 --2- 172.21.15.55:0/98989762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c440b16d0 0x7f5c440a7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.625+0000 7f5c49f9c700 1 -- 172.21.15.55:0/98989762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c440b33b0 msgr2=0x7f5c440c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.625+0000 7f5c49f9c700 1 --2- 172.21.15.55:0/98989762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c440b33b0 0x7f5c440c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.625+0000 7f5c49f9c700 1 -- 172.21.15.55:0/98989762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c440b16d0 msgr2=0x7f5c440a7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.625+0000 7f5c49f9c700 1 --2- 172.21.15.55:0/98989762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c440b16d0 0x7f5c440a7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.625+0000 7f5c49f9c700 1 -- 172.21.15.55:0/98989762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3c009a20 con 0x7f5c440b4e70 2023-10-06T03:34:14.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.625+0000 7f5c49f9c700 1 --2- 172.21.15.55:0/98989762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c440b4e70 0x7f5c440cc360 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009fc0 tx=0x7f5c3c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:14.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.625+0000 7f5c397fa700 1 -- 172.21.15.55:0/98989762 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c00cbd0 con 0x7f5c440b4e70 2023-10-06T03:34:14.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.626+0000 7f5c397fa700 1 -- 172.21.15.55:0/98989762 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c00f040 con 0x7f5c440b4e70 2023-10-06T03:34:14.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.626+0000 7f5c4a79d700 1 -- 172.21.15.55:0/98989762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c340094f0 con 0x7f5c440b4e70 2023-10-06T03:34:14.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.626+0000 7f5c397fa700 1 -- 172.21.15.55:0/98989762 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c014480 con 0x7f5c440b4e70 2023-10-06T03:34:14.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.626+0000 7f5c4a79d700 1 -- 172.21.15.55:0/98989762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c440c8c20 con 0x7f5c440b4e70 2023-10-06T03:34:14.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.627+0000 7f5c4a79d700 1 -- 172.21.15.55:0/98989762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4416af20 con 0x7f5c440b4e70 2023-10-06T03:34:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.628+0000 7f5c397fa700 1 -- 172.21.15.55:0/98989762 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c3c003020 con 0x7f5c440b4e70 2023-10-06T03:34:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.629+0000 7f5c397fa700 1 --2- 172.21.15.55:0/98989762 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c28066f30 0x7f5c280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.629+0000 7f5c397fa700 1 -- 172.21.15.55:0/98989762 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5c3c084fc0 con 0x7f5c440b4e70 2023-10-06T03:34:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.629+0000 7f5c4979b700 1 --2- 172.21.15.55:0/98989762 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c28066f30 0x7f5c280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.630+0000 7f5c4979b700 1 --2- 172.21.15.55:0/98989762 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c28066f30 0x7f5c280693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5c40007fd0 tx=0x7f5c40007d50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:14.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.633+0000 7f5c397fa700 1 -- 172.21.15.55:0/98989762 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c4416af20 con 0x7f5c440b4e70 2023-10-06T03:34:14.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:14.883+0000 7f5c4a79d700 1 -- 172.21.15.55:0/98989762 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5c4416af20 con 0x7f5c440b4e70 2023-10-06T03:34:15.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.545+0000 7f5c397fa700 1 -- 172.21.15.55:0/98989762 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5c4416af20 con 0x7f5c440b4e70 2023-10-06T03:34:15.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.548+0000 7f5c4a79d700 1 -- 172.21.15.55:0/98989762 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c28066f30 msgr2=0x7f5c280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:15.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.548+0000 7f5c4a79d700 1 --2- 172.21.15.55:0/98989762 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c28066f30 0x7f5c280693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5c40007fd0 tx=0x7f5c40007d50 comp rx=0 tx=0).stop 2023-10-06T03:34:15.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.548+0000 7f5c4a79d700 1 -- 172.21.15.55:0/98989762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c440b4e70 msgr2=0x7f5c440cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:15.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.548+0000 7f5c4a79d700 1 --2- 172.21.15.55:0/98989762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c440b4e70 0x7f5c440cc360 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009fc0 tx=0x7f5c3c00b040 comp rx=0 tx=0).stop 2023-10-06T03:34:15.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.549+0000 7f5c4a79d700 1 -- 172.21.15.55:0/98989762 shutdown_connections 2023-10-06T03:34:15.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.549+0000 7f5c4a79d700 1 --2- 172.21.15.55:0/98989762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c440b4e70 0x7f5c440cc360 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:15.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.549+0000 7f5c4a79d700 1 --2- 172.21.15.55:0/98989762 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c440b16d0 0x7f5c440a7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:15.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.549+0000 7f5c4a79d700 1 --2- 172.21.15.55:0/98989762 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c28066f30 0x7f5c280693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:15.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.549+0000 7f5c4a79d700 1 --2- 172.21.15.55:0/98989762 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c440b33b0 0x7f5c440c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:15.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.549+0000 7f5c4a79d700 1 -- 172.21.15.55:0/98989762 >> 172.21.15.55:0/98989762 conn(0x7f5c440d10c0 msgr2=0x7f5c44159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:15.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.549+0000 7f5c4a79d700 1 -- 172.21.15.55:0/98989762 shutdown_connections 2023-10-06T03:34:15.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.549+0000 7f5c4a79d700 1 -- 172.21.15.55:0/98989762 wait complete. 2023-10-06T03:34:15.570 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:34:15.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.762+0000 7fe1bd79d700 1 Processor -- start 2023-10-06T03:34:15.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.762+0000 7fe1bd79d700 1 -- start start 2023-10-06T03:34:15.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.763+0000 7fe1bd79d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1b8159a60 0x7fe1b80c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:15.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.763+0000 7fe1bd79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80cb230 0x7fe1b80cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:15.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.763+0000 7fe1bd79d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b80c9cb0 con 0x7fe1b80ccc00 2023-10-06T03:34:15.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.763+0000 7fe1bd79d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b80cbbd0 con 0x7fe1b8159a60 2023-10-06T03:34:15.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.763+0000 7fe1bd79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b806dfc0 con 0x7fe1b80cb230 2023-10-06T03:34:15.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.764+0000 7fe1b7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1b8159a60 0x7fe1b80c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:15.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.764+0000 7fe1b77fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe1b80ccc00 0x7fe1b8069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37978/0 (socket says 172.21.15.55:37978) 2023-10-06T03:34:15.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.764+0000 7fe1b77fe700 1 -- 172.21.15.55:0/3495895108 learned_addr learned my addr 172.21.15.55:0/3495895108 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:15.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.764+0000 7fe1b7fff700 1 -- 172.21.15.55:0/3495895108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80cb230 msgr2=0x7fe1b80cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:15.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.764+0000 7fe1bcf9c700 1 --2- 172.21.15.55:0/3495895108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80cb230 0x7fe1b80cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:15.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.764+0000 7fe1b6ffd700 1 -- 172.21.15.55:0/3495895108 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336757354 0 0) 0x7fe1b80c9cb0 con 0x7fe1b80ccc00 2023-10-06T03:34:15.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.764+0000 7fe1b7fff700 1 --2- 172.21.15.55:0/3495895108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80cb230 0x7fe1b80cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:15.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.764+0000 7fe1b7fff700 1 -- 172.21.15.55:0/3495895108 >> v1:172.21.15.55:6789/0 conn(0x7fe1b80ccc00 legacy=0x7fe1b8069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:15.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.764+0000 7fe1b7fff700 1 -- 172.21.15.55:0/3495895108 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b80c9f30 con 0x7fe1b8159a60 2023-10-06T03:34:15.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.764+0000 7fe1bcf9c700 1 --2- 172.21.15.55:0/3495895108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80cb230 0x7fe1b80cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:15.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.765+0000 7fe1b7fff700 1 --2- 172.21.15.55:0/3495895108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8159a60 0x7fe1b80c9770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009910 tx=0x7fe1a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5d4bb990b98785 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:15.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.765+0000 7fe1b6ffd700 1 -- 172.21.15.55:0/3495895108 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a800b8b0 con 0x7fe1b8159a60 2023-10-06T03:34:15.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.765+0000 7fe1b6ffd700 1 -- 172.21.15.55:0/3495895108 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a800ba10 con 0x7fe1b8159a60 2023-10-06T03:34:15.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.765+0000 7fe1b6ffd700 1 -- 172.21.15.55:0/3495895108 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a800bdb0 con 0x7fe1b8159a60 2023-10-06T03:34:15.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.765+0000 7fe1bd79d700 1 -- 172.21.15.55:0/3495895108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8159a60 msgr2=0x7fe1b80c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:15.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.765+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/3495895108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8159a60 0x7fe1b80c9770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009910 tx=0x7fe1a80091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:15.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.765+0000 7fe1bd79d700 1 -- 172.21.15.55:0/3495895108 shutdown_connections 2023-10-06T03:34:15.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.765+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/3495895108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80cb230 0x7fe1b80cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:15.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.765+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/3495895108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8159a60 0x7fe1b80c9770 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:15.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.765+0000 7fe1bd79d700 1 -- 172.21.15.55:0/3495895108 >> 172.21.15.55:0/3495895108 conn(0x7fe1b806baa0 msgr2=0x7fe1b806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:15.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.766+0000 7fe1bd79d700 1 -- 172.21.15.55:0/3495895108 shutdown_connections 2023-10-06T03:34:15.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.766+0000 7fe1bd79d700 1 -- 172.21.15.55:0/3495895108 wait complete. 2023-10-06T03:34:15.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.767+0000 7fe1bd79d700 1 Processor -- start 2023-10-06T03:34:15.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.767+0000 7fe1bd79d700 1 -- start start 2023-10-06T03:34:15.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.768+0000 7fe1bd79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80cb230 0x7fe1b80a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:15.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.768+0000 7fe1bd79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1b80ccc00 0x7fe1b80b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:15.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.768+0000 7fe1bd79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8159a60 0x7fe1b8169c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:15.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.768+0000 7fe1bd79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80b4300 con 0x7fe1b8159a60 2023-10-06T03:34:15.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.768+0000 7fe1bd79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80c7560 con 0x7fe1b80ccc00 2023-10-06T03:34:15.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.768+0000 7fe1bd79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80c6c90 con 0x7fe1b80cb230 2023-10-06T03:34:15.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.768+0000 7fe1bcf9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8159a60 0x7fe1b8169c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:15.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.768+0000 7fe1bcf9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8159a60 0x7fe1b8169c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39792/0 (socket says 172.21.15.55:39792) 2023-10-06T03:34:15.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.768+0000 7fe1bcf9c700 1 -- 172.21.15.55:0/2996894833 learned_addr learned my addr 172.21.15.55:0/2996894833 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:15.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.768+0000 7fe1b77fe700 1 --2- 172.21.15.55:0/2996894833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1b80ccc00 0x7fe1b80b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:15.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.768+0000 7fe1b7fff700 1 --2- 172.21.15.55:0/2996894833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80cb230 0x7fe1b80a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:15.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.769+0000 7fe1bcf9c700 1 -- 172.21.15.55:0/2996894833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80cb230 msgr2=0x7fe1b80a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:15.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.769+0000 7fe1bcf9c700 1 --2- 172.21.15.55:0/2996894833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80cb230 0x7fe1b80a98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:15.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.769+0000 7fe1bcf9c700 1 -- 172.21.15.55:0/2996894833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1b80ccc00 msgr2=0x7fe1b80b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:15.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.769+0000 7fe1bcf9c700 1 --2- 172.21.15.55:0/2996894833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1b80ccc00 0x7fe1b80b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:15.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.769+0000 7fe1bcf9c700 1 -- 172.21.15.55:0/2996894833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a8009420 con 0x7fe1b8159a60 2023-10-06T03:34:15.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.769+0000 7fe1bcf9c700 1 --2- 172.21.15.55:0/2996894833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8159a60 0x7fe1b8169c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000a380 tx=0x7fe1b0007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:15.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.769+0000 7fe19ffff700 1 -- 172.21.15.55:0/2996894833 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0015440 con 0x7fe1b8159a60 2023-10-06T03:34:15.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.769+0000 7fe19ffff700 1 -- 172.21.15.55:0/2996894833 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b00155a0 con 0x7fe1b8159a60 2023-10-06T03:34:15.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.769+0000 7fe1bd79d700 1 -- 172.21.15.55:0/2996894833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b80c5f30 con 0x7fe1b8159a60 2023-10-06T03:34:15.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.769+0000 7fe19ffff700 1 -- 172.21.15.55:0/2996894833 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0015960 con 0x7fe1b8159a60 2023-10-06T03:34:15.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.770+0000 7fe1bd79d700 1 -- 172.21.15.55:0/2996894833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b80c71f0 con 0x7fe1b8159a60 2023-10-06T03:34:15.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.770+0000 7fe1bd79d700 1 -- 172.21.15.55:0/2996894833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b806eba0 con 0x7fe1b8159a60 2023-10-06T03:34:15.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.771+0000 7fe19ffff700 1 -- 172.21.15.55:0/2996894833 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1b000d440 con 0x7fe1b8159a60 2023-10-06T03:34:15.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.772+0000 7fe19ffff700 1 --2- 172.21.15.55:0/2996894833 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1a4066f30 0x7fe1a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:15.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.772+0000 7fe19ffff700 1 -- 172.21.15.55:0/2996894833 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe1b00851f0 con 0x7fe1b8159a60 2023-10-06T03:34:15.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.773+0000 7fe1b7fff700 1 --2- 172.21.15.55:0/2996894833 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1a4066f30 0x7fe1a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:15.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.773+0000 7fe1b7fff700 1 --2- 172.21.15.55:0/2996894833 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1a4066f30 0x7fe1a40693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009860 tx=0x7fe1a8017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:15.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:15.777+0000 7fe19ffff700 1 -- 172.21.15.55:0/2996894833 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1b0052260 con 0x7fe1b8159a60 2023-10-06T03:34:16.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.026+0000 7fe1bd79d700 1 -- 172.21.15.55:0/2996894833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe1b80c76a0 con 0x7fe1b8159a60 2023-10-06T03:34:16.257 INFO:tasks.ceph.mds.d.smithi163.stderr:2023-10-06T03:34:16.254+0000 7f31223f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:34:16.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.557+0000 7fe19ffff700 1 -- 172.21.15.55:0/2996894833 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe1b0055880 con 0x7fe1b8159a60 2023-10-06T03:34:16.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.560+0000 7fe1bd79d700 1 -- 172.21.15.55:0/2996894833 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1a4066f30 msgr2=0x7fe1a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:16.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.560+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/2996894833 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1a4066f30 0x7fe1a40693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009860 tx=0x7fe1a8017000 comp rx=0 tx=0).stop 2023-10-06T03:34:16.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.560+0000 7fe1bd79d700 1 -- 172.21.15.55:0/2996894833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8159a60 msgr2=0x7fe1b8169c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:16.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.560+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/2996894833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8159a60 0x7fe1b8169c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000a380 tx=0x7fe1b0007900 comp rx=0 tx=0).stop 2023-10-06T03:34:16.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.560+0000 7fe1bd79d700 1 -- 172.21.15.55:0/2996894833 shutdown_connections 2023-10-06T03:34:16.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.560+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/2996894833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b8159a60 0x7fe1b8169c80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:16.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.560+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/2996894833 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1a4066f30 0x7fe1a40693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:16.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.560+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/2996894833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80cb230 0x7fe1b80a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:16.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.560+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/2996894833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1b80ccc00 0x7fe1b80b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:16.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.560+0000 7fe1bd79d700 1 -- 172.21.15.55:0/2996894833 >> 172.21.15.55:0/2996894833 conn(0x7fe1b806baa0 msgr2=0x7fe1b80ce600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:16.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.560+0000 7fe1bd79d700 1 -- 172.21.15.55:0/2996894833 shutdown_connections 2023-10-06T03:34:16.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.560+0000 7fe1bd79d700 1 -- 172.21.15.55:0/2996894833 wait complete. 2023-10-06T03:34:16.582 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:34:16.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.771+0000 7feda499d700 1 Processor -- start 2023-10-06T03:34:16.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.771+0000 7feda499d700 1 -- start start 2023-10-06T03:34:16.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.772+0000 7feda499d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feda0150180 0x7feda0157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:16.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.772+0000 7feda499d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feda0150550 0x7feda00c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:16.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.772+0000 7feda499d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda00c7240 0x7feda0069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:16.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.772+0000 7feda499d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda00c76a0 con 0x7feda0150550 2023-10-06T03:34:16.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.772+0000 7feda499d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda006e2a0 con 0x7feda0150180 2023-10-06T03:34:16.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.772+0000 7feda499d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda006b2e0 con 0x7feda00c7240 2023-10-06T03:34:16.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.773+0000 7fed9effd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feda0150550 0x7feda00c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:16.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.773+0000 7fed9effd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feda0150550 0x7feda00c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39806/0 (socket says 172.21.15.55:39806) 2023-10-06T03:34:16.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.773+0000 7fed9effd700 1 -- 172.21.15.55:0/2647636107 learned_addr learned my addr 172.21.15.55:0/2647636107 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:16.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.773+0000 7fed9f7fe700 1 --2- 172.21.15.55:0/2647636107 >> v2:172.21.15.163:3300/0 conn(0x7feda0150180 0x7feda0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:16.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.773+0000 7fed9ffff700 1 --2- 172.21.15.55:0/2647636107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda00c7240 0x7feda0069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:16.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.773+0000 7fed9effd700 1 -- 172.21.15.55:0/2647636107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda00c7240 msgr2=0x7feda0069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:16.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.773+0000 7fed9effd700 1 --2- 172.21.15.55:0/2647636107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda00c7240 0x7feda0069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:16.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.773+0000 7fed9effd700 1 -- 172.21.15.55:0/2647636107 >> v2:172.21.15.163:3300/0 conn(0x7feda0150180 msgr2=0x7feda0157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:16.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.773+0000 7fed9effd700 1 --2- 172.21.15.55:0/2647636107 >> v2:172.21.15.163:3300/0 conn(0x7feda0150180 0x7feda0157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:16.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.773+0000 7fed9effd700 1 -- 172.21.15.55:0/2647636107 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda00c78e0 con 0x7feda0150550 2023-10-06T03:34:16.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.773+0000 7fed9ffff700 1 --2- 172.21.15.55:0/2647636107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda00c7240 0x7feda0069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:16.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.773+0000 7fed9effd700 1 --2- 172.21.15.55:0/2647636107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda0150550 0x7feda00c6d00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feda006fd20 tx=0x7fed94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c063abee1dcdd2a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:16.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.774+0000 7fed9e7fc700 1 -- 172.21.15.55:0/2647636107 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94009e70 con 0x7feda0150550 2023-10-06T03:34:16.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.774+0000 7fed9e7fc700 1 -- 172.21.15.55:0/2647636107 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9400b860 con 0x7feda0150550 2023-10-06T03:34:16.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.774+0000 7fed9e7fc700 1 -- 172.21.15.55:0/2647636107 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9400bc00 con 0x7feda0150550 2023-10-06T03:34:16.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.774+0000 7feda499d700 1 -- 172.21.15.55:0/2647636107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda0150550 msgr2=0x7feda00c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:16.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.774+0000 7feda499d700 1 --2- 172.21.15.55:0/2647636107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda0150550 0x7feda00c6d00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feda006fd20 tx=0x7fed94009250 comp rx=0 tx=0).stop 2023-10-06T03:34:16.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.774+0000 7feda499d700 1 -- 172.21.15.55:0/2647636107 shutdown_connections 2023-10-06T03:34:16.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.774+0000 7feda499d700 1 --2- 172.21.15.55:0/2647636107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda00c7240 0x7feda0069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:16.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.774+0000 7feda499d700 1 --2- 172.21.15.55:0/2647636107 >> v2:172.21.15.163:3300/0 conn(0x7feda0150180 0x7feda0157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:16.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.774+0000 7feda499d700 1 --2- 172.21.15.55:0/2647636107 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda0150550 0x7feda00c6d00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:16.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.774+0000 7feda499d700 1 -- 172.21.15.55:0/2647636107 >> 172.21.15.55:0/2647636107 conn(0x7feda00cc1f0 msgr2=0x7feda00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:16.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.774+0000 7feda499d700 1 -- 172.21.15.55:0/2647636107 shutdown_connections 2023-10-06T03:34:16.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.774+0000 7feda499d700 1 -- 172.21.15.55:0/2647636107 wait complete. 2023-10-06T03:34:16.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.776+0000 7feda499d700 1 Processor -- start 2023-10-06T03:34:16.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.776+0000 7feda499d700 1 -- start start 2023-10-06T03:34:16.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.777+0000 7feda499d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda00c7240 0x7feda0141810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:16.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.777+0000 7feda499d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda0150180 0x7feda00c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:16.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.777+0000 7feda499d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feda0150550 0x7feda00b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:16.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.777+0000 7feda499d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda00c32a0 con 0x7feda0150180 2023-10-06T03:34:16.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.777+0000 7feda499d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda00c3540 con 0x7feda0150550 2023-10-06T03:34:16.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.777+0000 7feda499d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda00b13f0 con 0x7feda00c7240 2023-10-06T03:34:16.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.777+0000 7fed9effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda0150180 0x7feda00c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:16.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.777+0000 7fed9effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda0150180 0x7feda00c57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39818/0 (socket says 172.21.15.55:39818) 2023-10-06T03:34:16.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.777+0000 7fed9effd700 1 -- 172.21.15.55:0/228629613 learned_addr learned my addr 172.21.15.55:0/228629613 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:16.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.777+0000 7fed9effd700 1 -- 172.21.15.55:0/228629613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda00c7240 msgr2=0x7feda0141810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:16.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.777+0000 7fed9effd700 1 --2- 172.21.15.55:0/228629613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda00c7240 0x7feda0141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:16.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.777+0000 7fed9effd700 1 -- 172.21.15.55:0/228629613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feda0150550 msgr2=0x7feda00b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:16.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.777+0000 7fed9effd700 1 --2- 172.21.15.55:0/228629613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feda0150550 0x7feda00b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:16.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.777+0000 7fed9effd700 1 -- 172.21.15.55:0/228629613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed940094f0 con 0x7feda0150180 2023-10-06T03:34:16.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.778+0000 7fed9effd700 1 --2- 172.21.15.55:0/228629613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda0150180 0x7feda00c57f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fed940090b0 tx=0x7fed94017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:16.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.778+0000 7fed877fe700 1 -- 172.21.15.55:0/228629613 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9400bbe0 con 0x7feda0150180 2023-10-06T03:34:16.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.778+0000 7fed877fe700 1 -- 172.21.15.55:0/228629613 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9401f070 con 0x7feda0150180 2023-10-06T03:34:16.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.778+0000 7feda499d700 1 -- 172.21.15.55:0/228629613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda00b16d0 con 0x7feda0150180 2023-10-06T03:34:16.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.778+0000 7fed877fe700 1 -- 172.21.15.55:0/228629613 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed940266d0 con 0x7feda0150180 2023-10-06T03:34:16.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.778+0000 7feda499d700 1 -- 172.21.15.55:0/228629613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda0141d50 con 0x7feda0150180 2023-10-06T03:34:16.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.779+0000 7feda499d700 1 -- 172.21.15.55:0/228629613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda004bbe0 con 0x7feda0150180 2023-10-06T03:34:16.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.780+0000 7fed877fe700 1 -- 172.21.15.55:0/228629613 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed9401a430 con 0x7feda0150180 2023-10-06T03:34:16.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.781+0000 7fed877fe700 1 --2- 172.21.15.55:0/228629613 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fed8c066f30 0x7fed8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:16.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.781+0000 7fed877fe700 1 -- 172.21.15.55:0/228629613 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fed94021070 con 0x7feda0150180 2023-10-06T03:34:16.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.781+0000 7fed9f7fe700 1 --2- 172.21.15.55:0/228629613 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fed8c066f30 0x7fed8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:16.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.782+0000 7fed9f7fe700 1 --2- 172.21.15.55:0/228629613 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fed8c066f30 0x7fed8c0693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed90009bb0 tx=0x7fed900091a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:16.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:16.785+0000 7fed877fe700 1 -- 172.21.15.55:0/228629613 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed94053630 con 0x7feda0150180 2023-10-06T03:34:17.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.035+0000 7feda499d700 1 -- 172.21.15.55:0/228629613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feda004c3c0 con 0x7feda0150180 2023-10-06T03:34:17.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.036+0000 7fed877fe700 1 -- 172.21.15.55:0/228629613 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fed94056c50 con 0x7feda0150180 2023-10-06T03:34:17.037 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:17.037 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1532010025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1532010025},{"type":"v1","addr":"172.21.15.55:6834","nonce":1532010025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4258042685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":4258042685},{"type":"v1","addr":"172.21.15.55:6835","nonce":4258042685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/430119478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":430119478},{"type":"v1","addr":"172.21.15.163:6836","nonce":430119478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:34:13.003040+0000","modified":"2023-10-06T03:34:16.548740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4903},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4903":{"gid":4903,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6837/1606384901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1606384901},{"type":"v1","addr":"172.21.15.163:6837","nonce":1606384901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T03:34:17.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.038+0000 7feda499d700 1 -- 172.21.15.55:0/228629613 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fed8c066f30 msgr2=0x7fed8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.038+0000 7feda499d700 1 --2- 172.21.15.55:0/228629613 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fed8c066f30 0x7fed8c0693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fed90009bb0 tx=0x7fed900091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.038+0000 7feda499d700 1 -- 172.21.15.55:0/228629613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda0150180 msgr2=0x7feda00c57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.038+0000 7feda499d700 1 --2- 172.21.15.55:0/228629613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda0150180 0x7feda00c57f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fed940090b0 tx=0x7fed94017000 comp rx=0 tx=0).stop 2023-10-06T03:34:17.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.038+0000 7feda499d700 1 -- 172.21.15.55:0/228629613 shutdown_connections 2023-10-06T03:34:17.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.038+0000 7feda499d700 1 --2- 172.21.15.55:0/228629613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feda0150550 0x7feda00b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.038+0000 7feda499d700 1 --2- 172.21.15.55:0/228629613 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fed8c066f30 0x7fed8c0693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.038+0000 7feda499d700 1 --2- 172.21.15.55:0/228629613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feda00c7240 0x7feda0141810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.038+0000 7feda499d700 1 --2- 172.21.15.55:0/228629613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feda0150180 0x7feda00c57f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.038+0000 7feda499d700 1 -- 172.21.15.55:0/228629613 >> 172.21.15.55:0/228629613 conn(0x7feda00cc1f0 msgr2=0x7feda0158de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:17.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.038+0000 7feda499d700 1 -- 172.21.15.55:0/228629613 shutdown_connections 2023-10-06T03:34:17.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.038+0000 7feda499d700 1 -- 172.21.15.55:0/228629613 wait complete. 2023-10-06T03:34:17.041 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 17 2023-10-06T03:34:17.057 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:34:17.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.243+0000 7f1d8239d700 1 Processor -- start 2023-10-06T03:34:17.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.243+0000 7f1d8239d700 1 -- start start 2023-10-06T03:34:17.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.245+0000 7f1d8239d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1d7c0b0bd0 0x7f1d7c1646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:17.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.245+0000 7f1d8239d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7c15a980 con 0x7f1d7c068aa0 2023-10-06T03:34:17.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.245+0000 7f1d8239d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7c15a050 con 0x7f1d7c157270 2023-10-06T03:34:17.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.245+0000 7f1d8239d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7c15e310 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.245+0000 7f1d80b9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1d7c068aa0 0x7f1d7c0b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37992/0 (socket says 172.21.15.55:37992) 2023-10-06T03:34:17.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.245+0000 7f1d80b9a700 1 -- 172.21.15.55:0/3771965904 learned_addr learned my addr 172.21.15.55:0/3771965904 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:17.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.245+0000 7f1d81b9c700 1 --2- 172.21.15.55:0/3771965904 >> v2:172.21.15.163:3300/0 conn(0x7f1d7c0b0bd0 0x7f1d7c1646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:17.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.245+0000 7f1d73fff700 1 -- 172.21.15.55:0/3771965904 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532636618 0 0) 0x7f1d7c15a980 con 0x7f1d7c068aa0 2023-10-06T03:34:17.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.245+0000 7f1d73fff700 1 -- 172.21.15.55:0/3771965904 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d68003570 con 0x7f1d7c068aa0 2023-10-06T03:34:17.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.246+0000 7f1d81b9c700 1 -- 172.21.15.55:0/3771965904 >> v1:172.21.15.163:6789/0 conn(0x7f1d7c157270 legacy=0x7f1d7c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.246+0000 7f1d81b9c700 1 -- 172.21.15.55:0/3771965904 >> v1:172.21.15.55:6789/0 conn(0x7f1d7c068aa0 legacy=0x7f1d7c0b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.246+0000 7f1d81b9c700 1 -- 172.21.15.55:0/3771965904 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c15a330 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.246+0000 7f1d81b9c700 1 --2- 172.21.15.55:0/3771965904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7c0b0bd0 0x7f1d7c1646a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009e60 tx=0x7f1d74009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d43e8ffcddc78055 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:17.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.246+0000 7f1d73fff700 1 -- 172.21.15.55:0/3771965904 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7400e070 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.246+0000 7f1d73fff700 1 -- 172.21.15.55:0/3771965904 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7400bb60 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.246+0000 7f1d73fff700 1 -- 172.21.15.55:0/3771965904 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74012430 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.247+0000 7f1d8239d700 1 -- 172.21.15.55:0/3771965904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7c0b0bd0 msgr2=0x7f1d7c1646a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.247+0000 7f1d8239d700 1 --2- 172.21.15.55:0/3771965904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7c0b0bd0 0x7f1d7c1646a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009e60 tx=0x7f1d74009780 comp rx=0 tx=0).stop 2023-10-06T03:34:17.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.247+0000 7f1d8239d700 1 -- 172.21.15.55:0/3771965904 shutdown_connections 2023-10-06T03:34:17.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.247+0000 7f1d8239d700 1 --2- 172.21.15.55:0/3771965904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7c0b0bd0 0x7f1d7c1646a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.247+0000 7f1d8239d700 1 -- 172.21.15.55:0/3771965904 >> 172.21.15.55:0/3771965904 conn(0x7f1d7c0cd310 msgr2=0x7f1d7c0cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:17.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.247+0000 7f1d8239d700 1 -- 172.21.15.55:0/3771965904 shutdown_connections 2023-10-06T03:34:17.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.247+0000 7f1d8239d700 1 -- 172.21.15.55:0/3771965904 wait complete. 2023-10-06T03:34:17.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.248+0000 7f1d8239d700 1 Processor -- start 2023-10-06T03:34:17.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.249+0000 7f1d8239d700 1 -- start start 2023-10-06T03:34:17.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.249+0000 7f1d8239d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7c068aa0 0x7f1d7c152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:17.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d8239d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d7c0b0bd0 0x7f1d7c159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:17.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d8239d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d7c157270 0x7f1d7c159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:17.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d8239d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c15c610 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d8239d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0adb40 con 0x7f1d7c068aa0 2023-10-06T03:34:17.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d8239d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0ade20 con 0x7f1d7c157270 2023-10-06T03:34:17.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d80b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d7c0b0bd0 0x7f1d7c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:17.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d80b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d7c0b0bd0 0x7f1d7c159010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39820/0 (socket says 172.21.15.55:39820) 2023-10-06T03:34:17.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d80b9a700 1 -- 172.21.15.55:0/4152769101 learned_addr learned my addr 172.21.15.55:0/4152769101 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:17.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d81b9c700 1 --2- 172.21.15.55:0/4152769101 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d7c157270 0x7f1d7c159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d8139b700 1 --2- 172.21.15.55:0/4152769101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7c068aa0 0x7f1d7c152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d80b9a700 1 -- 172.21.15.55:0/4152769101 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d7c157270 msgr2=0x7f1d7c159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d80b9a700 1 --2- 172.21.15.55:0/4152769101 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d7c157270 0x7f1d7c159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d80b9a700 1 -- 172.21.15.55:0/4152769101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7c068aa0 msgr2=0x7f1d7c152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d80b9a700 1 --2- 172.21.15.55:0/4152769101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7c068aa0 0x7f1d7c152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.250+0000 7f1d80b9a700 1 -- 172.21.15.55:0/4152769101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d74009a20 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.251+0000 7f1d80b9a700 1 --2- 172.21.15.55:0/4152769101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d7c0b0bd0 0x7f1d7c159010 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00c7c0 tx=0x7f1d6c007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.251+0000 7f1d717fa700 1 -- 172.21.15.55:0/4152769101 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c005c00 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.251+0000 7f1d717fa700 1 -- 172.21.15.55:0/4152769101 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c005d60 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.251+0000 7f1d717fa700 1 -- 172.21.15.55:0/4152769101 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c007320 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.251+0000 7f1d8239d700 1 -- 172.21.15.55:0/4152769101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7c0ae100 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.251+0000 7f1d8239d700 1 -- 172.21.15.55:0/4152769101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c1697d0 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.252+0000 7f1d8239d700 1 -- 172.21.15.55:0/4152769101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7c04c3c0 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.253+0000 7f1d717fa700 1 -- 172.21.15.55:0/4152769101 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d6c01b3b0 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.254+0000 7f1d717fa700 1 --2- 172.21.15.55:0/4152769101 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1d60066f30 0x7f1d600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.254+0000 7f1d717fa700 1 -- 172.21.15.55:0/4152769101 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1d6c083b70 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.254+0000 7f1d8139b700 1 --2- 172.21.15.55:0/4152769101 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1d60066f30 0x7f1d600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:17.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.255+0000 7f1d8139b700 1 --2- 172.21.15.55:0/4152769101 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1d60066f30 0x7f1d600693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c159d10 tx=0x7f1d78009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:17.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.258+0000 7f1d717fa700 1 -- 172.21.15.55:0/4152769101 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d6c050c80 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.460+0000 7f1d8239d700 1 -- 172.21.15.55:0/4152769101 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d7c15a610 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.461+0000 7f1d717fa700 1 -- 172.21.15.55:0/4152769101 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16914 (secure 0 0 0) 0x7f1d6c017070 con 0x7f1d7c0b0bd0 2023-10-06T03:34:17.463 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:17.463 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":32,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:34:14.516683+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:34:11.006235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T03:34:11.987296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:34:17.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.464+0000 7f1d8239d700 1 -- 172.21.15.55:0/4152769101 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1d60066f30 msgr2=0x7f1d600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.464+0000 7f1d8239d700 1 --2- 172.21.15.55:0/4152769101 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1d60066f30 0x7f1d600693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c159d10 tx=0x7f1d78009040 comp rx=0 tx=0).stop 2023-10-06T03:34:17.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.464+0000 7f1d8239d700 1 -- 172.21.15.55:0/4152769101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d7c0b0bd0 msgr2=0x7f1d7c159010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.464+0000 7f1d8239d700 1 --2- 172.21.15.55:0/4152769101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d7c0b0bd0 0x7f1d7c159010 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c00c7c0 tx=0x7f1d6c007dd0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.464+0000 7f1d8239d700 1 -- 172.21.15.55:0/4152769101 shutdown_connections 2023-10-06T03:34:17.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.464+0000 7f1d8239d700 1 --2- 172.21.15.55:0/4152769101 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1d60066f30 0x7f1d600693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.464+0000 7f1d8239d700 1 --2- 172.21.15.55:0/4152769101 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d7c157270 0x7f1d7c159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.464+0000 7f1d8239d700 1 --2- 172.21.15.55:0/4152769101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7c068aa0 0x7f1d7c152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.464+0000 7f1d8239d700 1 --2- 172.21.15.55:0/4152769101 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d7c0b0bd0 0x7f1d7c159010 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.464+0000 7f1d8239d700 1 -- 172.21.15.55:0/4152769101 >> 172.21.15.55:0/4152769101 conn(0x7f1d7c0cd310 msgr2=0x7f1d7c152cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:17.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.464+0000 7f1d8239d700 1 -- 172.21.15.55:0/4152769101 shutdown_connections 2023-10-06T03:34:17.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.464+0000 7f1d8239d700 1 -- 172.21.15.55:0/4152769101 wait complete. 2023-10-06T03:34:17.485 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:34:17.485 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:34:17.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.671+0000 7f9e6679d700 1 Processor -- start 2023-10-06T03:34:17.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.672+0000 7f9e6679d700 1 -- start start 2023-10-06T03:34:17.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.673+0000 7f9e6679d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9e60156310 0x7f9e600c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:17.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.673+0000 7f9e6679d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e600ca860 con 0x7f9e600ae710 2023-10-06T03:34:17.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.673+0000 7f9e6679d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e600cab40 con 0x7f9e601566e0 2023-10-06T03:34:17.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.673+0000 7f9e6679d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e600cae20 con 0x7f9e60156310 2023-10-06T03:34:17.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.673+0000 7f9e65f9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9e600ae710 0x7f9e600c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38006/0 (socket says 172.21.15.55:38006) 2023-10-06T03:34:17.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.673+0000 7f9e65f9c700 1 -- 172.21.15.55:0/2913317944 learned_addr learned my addr 172.21.15.55:0/2913317944 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:17.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.673+0000 7f9e6579b700 1 --2- 172.21.15.55:0/2913317944 >> v2:172.21.15.163:3300/0 conn(0x7f9e60156310 0x7f9e600c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.674+0000 7f9e57fff700 1 -- 172.21.15.55:0/2913317944 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851141094 0 0) 0x7f9e600ca860 con 0x7f9e600ae710 2023-10-06T03:34:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.674+0000 7f9e57fff700 1 -- 172.21.15.55:0/2913317944 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e4c003570 con 0x7f9e600ae710 2023-10-06T03:34:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.674+0000 7f9e57fff700 1 -- 172.21.15.55:0/2913317944 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224346046 0 0) 0x7f9e600cab40 con 0x7f9e601566e0 2023-10-06T03:34:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.674+0000 7f9e57fff700 1 -- 172.21.15.55:0/2913317944 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e600ca860 con 0x7f9e601566e0 2023-10-06T03:34:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.674+0000 7f9e6579b700 1 -- 172.21.15.55:0/2913317944 >> v1:172.21.15.163:6789/0 conn(0x7f9e601566e0 legacy=0x7f9e600ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.674+0000 7f9e6579b700 1 -- 172.21.15.55:0/2913317944 >> v1:172.21.15.55:6789/0 conn(0x7f9e600ae710 legacy=0x7f9e600c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.674+0000 7f9e6579b700 1 -- 172.21.15.55:0/2913317944 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e600cb100 con 0x7f9e60156310 2023-10-06T03:34:17.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.674+0000 7f9e6579b700 1 --2- 172.21.15.55:0/2913317944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e60156310 0x7f9e600c9dc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0098e0 tx=0x7f9e5c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=628c6c7b581bf9e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:17.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.675+0000 7f9e57fff700 1 -- 172.21.15.55:0/2913317944 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c011630 con 0x7f9e60156310 2023-10-06T03:34:17.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.675+0000 7f9e57fff700 1 -- 172.21.15.55:0/2913317944 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5c011790 con 0x7f9e60156310 2023-10-06T03:34:17.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.675+0000 7f9e57fff700 1 -- 172.21.15.55:0/2913317944 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5c011b30 con 0x7f9e60156310 2023-10-06T03:34:17.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.675+0000 7f9e6679d700 1 -- 172.21.15.55:0/2913317944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e60156310 msgr2=0x7f9e600c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.675+0000 7f9e6679d700 1 --2- 172.21.15.55:0/2913317944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e60156310 0x7f9e600c9dc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0098e0 tx=0x7f9e5c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.675+0000 7f9e6679d700 1 -- 172.21.15.55:0/2913317944 shutdown_connections 2023-10-06T03:34:17.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.675+0000 7f9e6679d700 1 --2- 172.21.15.55:0/2913317944 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e60156310 0x7f9e600c9dc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.675+0000 7f9e6679d700 1 -- 172.21.15.55:0/2913317944 >> 172.21.15.55:0/2913317944 conn(0x7f9e600cc1f0 msgr2=0x7f9e600cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:17.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.675+0000 7f9e6679d700 1 -- 172.21.15.55:0/2913317944 shutdown_connections 2023-10-06T03:34:17.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.675+0000 7f9e6679d700 1 -- 172.21.15.55:0/2913317944 wait complete. 2023-10-06T03:34:17.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.677+0000 7f9e6679d700 1 Processor -- start 2023-10-06T03:34:17.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.677+0000 7f9e6679d700 1 -- start start 2023-10-06T03:34:17.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.678+0000 7f9e6679d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e600ae710 0x7f9e60143ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:17.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.678+0000 7f9e6679d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e60156310 0x7f9e601598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:17.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.678+0000 7f9e6679d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e601566e0 0x7f9e60159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:17.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.678+0000 7f9e6679d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e60154ed0 con 0x7f9e601566e0 2023-10-06T03:34:17.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.678+0000 7f9e6679d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e60157000 con 0x7f9e60156310 2023-10-06T03:34:17.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.678+0000 7f9e6679d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e601572e0 con 0x7f9e600ae710 2023-10-06T03:34:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.678+0000 7f9e65f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e601566e0 0x7f9e60159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.678+0000 7f9e65f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e601566e0 0x7f9e60159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39826/0 (socket says 172.21.15.55:39826) 2023-10-06T03:34:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.678+0000 7f9e65f9c700 1 -- 172.21.15.55:0/3051645623 learned_addr learned my addr 172.21.15.55:0/3051645623 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.678+0000 7f9e64f9a700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e60156310 0x7f9e601598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.678+0000 7f9e6579b700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e600ae710 0x7f9e60143ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.679+0000 7f9e65f9c700 1 -- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e600ae710 msgr2=0x7f9e60143ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.679+0000 7f9e65f9c700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e600ae710 0x7f9e60143ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.679+0000 7f9e65f9c700 1 -- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e60156310 msgr2=0x7f9e601598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.679+0000 7f9e65f9c700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e60156310 0x7f9e601598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.679+0000 7f9e65f9c700 1 -- 172.21.15.55:0/3051645623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c009420 con 0x7f9e601566e0 2023-10-06T03:34:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.679+0000 7f9e64f9a700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e60156310 0x7f9e601598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.679+0000 7f9e65f9c700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e601566e0 0x7f9e60159e20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9e58001460 tx=0x7f9e58001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.679+0000 7f9e557fa700 1 -- 172.21.15.55:0/3051645623 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5800a8b0 con 0x7f9e601566e0 2023-10-06T03:34:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.679+0000 7f9e557fa700 1 -- 172.21.15.55:0/3051645623 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5800aa10 con 0x7f9e601566e0 2023-10-06T03:34:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.679+0000 7f9e557fa700 1 -- 172.21.15.55:0/3051645623 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5800add0 con 0x7f9e601566e0 2023-10-06T03:34:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.679+0000 7f9e6679d700 1 -- 172.21.15.55:0/3051645623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e601575c0 con 0x7f9e601566e0 2023-10-06T03:34:17.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.679+0000 7f9e6679d700 1 -- 172.21.15.55:0/3051645623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e60144010 con 0x7f9e601566e0 2023-10-06T03:34:17.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.680+0000 7f9e6679d700 1 -- 172.21.15.55:0/3051645623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e60157ae0 con 0x7f9e601566e0 2023-10-06T03:34:17.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.681+0000 7f9e557fa700 1 -- 172.21.15.55:0/3051645623 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e58007300 con 0x7f9e601566e0 2023-10-06T03:34:17.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.682+0000 7f9e557fa700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9e44066f30 0x7f9e440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:17.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.682+0000 7f9e557fa700 1 -- 172.21.15.55:0/3051645623 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9e58084b30 con 0x7f9e601566e0 2023-10-06T03:34:17.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.682+0000 7f9e6579b700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9e44066f30 0x7f9e440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:17.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.683+0000 7f9e6579b700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9e44066f30 0x7f9e440693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0093f0 tx=0x7f9e5c017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:17.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.686+0000 7f9e557fa700 1 -- 172.21.15.55:0/3051645623 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e60157ae0 con 0x7f9e601566e0 2023-10-06T03:34:17.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.884+0000 7f9e6679d700 1 -- 172.21.15.55:0/3051645623 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e60157ae0 con 0x7f9e44066f30 2023-10-06T03:34:17.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.896+0000 7f9e557fa700 1 -- 172.21.15.55:0/3051645623 <== mgr.4114 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308964 (secure 0 0 0) 0x7f9e60157ae0 con 0x7f9e44066f30 2023-10-06T03:34:17.901 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:17.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.900+0000 7f9e6679d700 1 -- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9e44066f30 msgr2=0x7f9e440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.900+0000 7f9e6679d700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9e44066f30 0x7f9e440693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c0093f0 tx=0x7f9e5c017000 comp rx=0 tx=0).stop 2023-10-06T03:34:17.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.900+0000 7f9e6679d700 1 -- 172.21.15.55:0/3051645623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e601566e0 msgr2=0x7f9e60159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:17.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.900+0000 7f9e6679d700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e601566e0 0x7f9e60159e20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9e58001460 tx=0x7f9e58001a20 comp rx=0 tx=0).stop 2023-10-06T03:34:17.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.900+0000 7f9e6679d700 1 -- 172.21.15.55:0/3051645623 shutdown_connections 2023-10-06T03:34:17.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.900+0000 7f9e6679d700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e601566e0 0x7f9e60159e20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.900+0000 7f9e6679d700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9e44066f30 0x7f9e440693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.900+0000 7f9e6679d700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e600ae710 0x7f9e60143ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.900+0000 7f9e6679d700 1 --2- 172.21.15.55:0/3051645623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e60156310 0x7f9e601598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:17.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.900+0000 7f9e6679d700 1 -- 172.21.15.55:0/3051645623 >> 172.21.15.55:0/3051645623 conn(0x7f9e600cc1f0 msgr2=0x7f9e60151c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:17.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.900+0000 7f9e6679d700 1 -- 172.21.15.55:0/3051645623 shutdown_connections 2023-10-06T03:34:17.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:17.901+0000 7f9e6679d700 1 -- 172.21.15.55:0/3051645623 wait complete. 2023-10-06T03:34:17.906 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:34:17.921 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2023-10-06T03:34:16.503849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398360,"kb_used_data":3672,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752576360,"statfs":{"total":773094113280,"available":770638192640,"internally_reserved":0,"allocated":3760128,"data_stored":2184408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.002301"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.511338+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:27.486440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.512011+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:38.277766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.511012+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:10.048229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.511933+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:47.323964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509810+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:17.520030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.512665+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:05.851589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.516769+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:04.506250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.514172+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:17.351579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510350+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:22.590165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.521902+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:01.442206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.520741+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:53.934064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510363+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:53.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.520333+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:38.277766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.507121+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:50.641614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.517941+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:53.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.514353+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:38.277766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.513164+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:10.048229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509840+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:28.353727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510480+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:43.771510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510338+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:44.589878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.518386+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:32.081718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509021+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:19.240735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.513444+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:23.077437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510723+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:25.638673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509780+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:47.323964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510560+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:09.006005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510517+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:10.034971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.520041+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:23.077437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510111+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:06.270734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509226+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:43.771510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.515227+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:48.475963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.511747+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:55.396883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.513200+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:27.486440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.523661+0000","last_change":"2023-10-06T03:34:12.523661+0000","last_active":"2023-10-06T03:34:12.523661+0000","last_peered":"2023-10-06T03:34:12.523661+0000","last_clean":"2023-10-06T03:34:12.523661+0000","last_became_active":"2023-10-06T03:34:12.523343+0000","last_became_peered":"2023-10-06T03:34:12.523343+0000","last_unstale":"2023-10-06T03:34:12.523661+0000","last_undegraded":"2023-10-06T03:34:12.523661+0000","last_fullsized":"2023-10-06T03:34:12.523661+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:23.796615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.511540+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:48.522789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.519828+0000","last_change":"2023-10-06T03:34:12.519828+0000","last_active":"2023-10-06T03:34:12.519828+0000","last_peered":"2023-10-06T03:34:12.519828+0000","last_clean":"2023-10-06T03:34:12.519828+0000","last_became_active":"2023-10-06T03:34:12.519596+0000","last_became_peered":"2023-10-06T03:34:12.519596+0000","last_unstale":"2023-10-06T03:34:12.519828+0000","last_undegraded":"2023-10-06T03:34:12.519828+0000","last_fullsized":"2023-10-06T03:34:12.519828+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:01.889807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.508223+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:25.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.526656+0000","last_change":"2023-10-06T03:34:12.526656+0000","last_active":"2023-10-06T03:34:12.526656+0000","last_peered":"2023-10-06T03:34:12.526656+0000","last_clean":"2023-10-06T03:34:12.526656+0000","last_became_active":"2023-10-06T03:34:12.525776+0000","last_became_peered":"2023-10-06T03:34:12.525776+0000","last_unstale":"2023-10-06T03:34:12.526656+0000","last_undegraded":"2023-10-06T03:34:12.526656+0000","last_fullsized":"2023-10-06T03:34:12.526656+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:27.342656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.512262+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:53.934064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.524322+0000","last_change":"2023-10-06T03:34:12.524322+0000","last_active":"2023-10-06T03:34:12.524322+0000","last_peered":"2023-10-06T03:34:12.524322+0000","last_clean":"2023-10-06T03:34:12.524322+0000","last_became_active":"2023-10-06T03:34:12.524037+0000","last_became_peered":"2023-10-06T03:34:12.524037+0000","last_unstale":"2023-10-06T03:34:12.524322+0000","last_undegraded":"2023-10-06T03:34:12.524322+0000","last_fullsized":"2023-10-06T03:34:12.524322+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:59.774790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509777+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:04.506250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.527023+0000","last_change":"2023-10-06T03:34:12.527023+0000","last_active":"2023-10-06T03:34:12.527023+0000","last_peered":"2023-10-06T03:34:12.527023+0000","last_clean":"2023-10-06T03:34:12.527023+0000","last_became_active":"2023-10-06T03:34:12.526443+0000","last_became_peered":"2023-10-06T03:34:12.526443+0000","last_unstale":"2023-10-06T03:34:12.527023+0000","last_undegraded":"2023-10-06T03:34:12.527023+0000","last_fullsized":"2023-10-06T03:34:12.527023+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:54.385812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.517298+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:22.590165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.532851+0000","last_change":"2023-10-06T03:34:12.532851+0000","last_active":"2023-10-06T03:34:12.532851+0000","last_peered":"2023-10-06T03:34:12.532851+0000","last_clean":"2023-10-06T03:34:12.532851+0000","last_became_active":"2023-10-06T03:34:12.532517+0000","last_became_peered":"2023-10-06T03:34:12.532517+0000","last_unstale":"2023-10-06T03:34:12.532851+0000","last_undegraded":"2023-10-06T03:34:12.532851+0000","last_fullsized":"2023-10-06T03:34:12.532851+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:04.840518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.508297+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:17.622122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.527549+0000","last_change":"2023-10-06T03:34:12.527549+0000","last_active":"2023-10-06T03:34:12.527549+0000","last_peered":"2023-10-06T03:34:12.527549+0000","last_clean":"2023-10-06T03:34:12.527549+0000","last_became_active":"2023-10-06T03:34:12.527236+0000","last_became_peered":"2023-10-06T03:34:12.527236+0000","last_unstale":"2023-10-06T03:34:12.527549+0000","last_undegraded":"2023-10-06T03:34:12.527549+0000","last_fullsized":"2023-10-06T03:34:12.527549+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:43.578807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.516316+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:30.492483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.516603+0000","last_change":"2023-10-06T03:34:12.516603+0000","last_active":"2023-10-06T03:34:12.516603+0000","last_peered":"2023-10-06T03:34:12.516603+0000","last_clean":"2023-10-06T03:34:12.516603+0000","last_became_active":"2023-10-06T03:34:12.516217+0000","last_became_peered":"2023-10-06T03:34:12.516217+0000","last_unstale":"2023-10-06T03:34:12.516603+0000","last_undegraded":"2023-10-06T03:34:12.516603+0000","last_fullsized":"2023-10-06T03:34:12.516603+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:16.340508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510060+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:24.431295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.523775+0000","last_change":"2023-10-06T03:34:12.523775+0000","last_active":"2023-10-06T03:34:12.523775+0000","last_peered":"2023-10-06T03:34:12.523775+0000","last_clean":"2023-10-06T03:34:12.523775+0000","last_became_active":"2023-10-06T03:34:12.523512+0000","last_became_peered":"2023-10-06T03:34:12.523512+0000","last_unstale":"2023-10-06T03:34:12.523775+0000","last_undegraded":"2023-10-06T03:34:12.523775+0000","last_fullsized":"2023-10-06T03:34:12.523775+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:21.579094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.516255+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:17.351579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.528843+0000","last_change":"2023-10-06T03:34:12.528843+0000","last_active":"2023-10-06T03:34:12.528843+0000","last_peered":"2023-10-06T03:34:12.528843+0000","last_clean":"2023-10-06T03:34:12.528843+0000","last_became_active":"2023-10-06T03:34:12.528575+0000","last_became_peered":"2023-10-06T03:34:12.528575+0000","last_unstale":"2023-10-06T03:34:12.528843+0000","last_undegraded":"2023-10-06T03:34:12.528843+0000","last_fullsized":"2023-10-06T03:34:12.528843+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:07.994934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509203+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:32.081718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.529757+0000","last_change":"2023-10-06T03:34:12.529757+0000","last_active":"2023-10-06T03:34:12.529757+0000","last_peered":"2023-10-06T03:34:12.529757+0000","last_clean":"2023-10-06T03:34:12.529757+0000","last_became_active":"2023-10-06T03:34:12.529371+0000","last_became_peered":"2023-10-06T03:34:12.529371+0000","last_unstale":"2023-10-06T03:34:12.529757+0000","last_undegraded":"2023-10-06T03:34:12.529757+0000","last_fullsized":"2023-10-06T03:34:12.529757+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:17.340207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.508885+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:35.437314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.528656+0000","last_change":"2023-10-06T03:34:12.528656+0000","last_active":"2023-10-06T03:34:12.528656+0000","last_peered":"2023-10-06T03:34:12.528656+0000","last_clean":"2023-10-06T03:34:12.528656+0000","last_became_active":"2023-10-06T03:34:12.528361+0000","last_became_peered":"2023-10-06T03:34:12.528361+0000","last_unstale":"2023-10-06T03:34:12.528656+0000","last_undegraded":"2023-10-06T03:34:12.528656+0000","last_fullsized":"2023-10-06T03:34:12.528656+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:31.070647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509144+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:54.559785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.523998+0000","last_change":"2023-10-06T03:34:12.523998+0000","last_active":"2023-10-06T03:34:12.523998+0000","last_peered":"2023-10-06T03:34:12.523998+0000","last_clean":"2023-10-06T03:34:12.523998+0000","last_became_active":"2023-10-06T03:34:12.523688+0000","last_became_peered":"2023-10-06T03:34:12.523688+0000","last_unstale":"2023-10-06T03:34:12.523998+0000","last_undegraded":"2023-10-06T03:34:12.523998+0000","last_fullsized":"2023-10-06T03:34:12.523998+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:52.611432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.513908+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:01.442206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.530728+0000","last_change":"2023-10-06T03:34:12.530728+0000","last_active":"2023-10-06T03:34:12.530728+0000","last_peered":"2023-10-06T03:34:12.530728+0000","last_clean":"2023-10-06T03:34:12.530728+0000","last_became_active":"2023-10-06T03:34:12.530458+0000","last_became_peered":"2023-10-06T03:34:12.530458+0000","last_unstale":"2023-10-06T03:34:12.530728+0000","last_undegraded":"2023-10-06T03:34:12.530728+0000","last_fullsized":"2023-10-06T03:34:12.530728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:09.423072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.513144+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:53.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.517112+0000","last_change":"2023-10-06T03:34:12.517112+0000","last_active":"2023-10-06T03:34:12.517112+0000","last_peered":"2023-10-06T03:34:12.517112+0000","last_clean":"2023-10-06T03:34:12.517112+0000","last_became_active":"2023-10-06T03:34:12.516831+0000","last_became_peered":"2023-10-06T03:34:12.516831+0000","last_unstale":"2023-10-06T03:34:12.517112+0000","last_undegraded":"2023-10-06T03:34:12.517112+0000","last_fullsized":"2023-10-06T03:34:12.517112+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:48.752619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.521103+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:17.622122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.532903+0000","last_change":"2023-10-06T03:34:12.532903+0000","last_active":"2023-10-06T03:34:12.532903+0000","last_peered":"2023-10-06T03:34:12.532903+0000","last_clean":"2023-10-06T03:34:12.532903+0000","last_became_active":"2023-10-06T03:34:12.532576+0000","last_became_peered":"2023-10-06T03:34:12.532576+0000","last_unstale":"2023-10-06T03:34:12.532903+0000","last_undegraded":"2023-10-06T03:34:12.532903+0000","last_fullsized":"2023-10-06T03:34:12.532903+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.514884+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:48.475963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.532478+0000","last_change":"2023-10-06T03:34:12.532478+0000","last_active":"2023-10-06T03:34:12.532478+0000","last_peered":"2023-10-06T03:34:12.532478+0000","last_clean":"2023-10-06T03:34:12.532478+0000","last_became_active":"2023-10-06T03:34:12.532204+0000","last_became_peered":"2023-10-06T03:34:12.532204+0000","last_unstale":"2023-10-06T03:34:12.532478+0000","last_undegraded":"2023-10-06T03:34:12.532478+0000","last_fullsized":"2023-10-06T03:34:12.532478+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.521299+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:48.475963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.527219+0000","last_change":"2023-10-06T03:34:12.527219+0000","last_active":"2023-10-06T03:34:12.527219+0000","last_peered":"2023-10-06T03:34:12.527219+0000","last_clean":"2023-10-06T03:34:12.527219+0000","last_became_active":"2023-10-06T03:34:12.526985+0000","last_became_peered":"2023-10-06T03:34:12.526985+0000","last_unstale":"2023-10-06T03:34:12.527219+0000","last_undegraded":"2023-10-06T03:34:12.527219+0000","last_fullsized":"2023-10-06T03:34:12.527219+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:29.481412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.507630+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:01.766184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.524489+0000","last_change":"2023-10-06T03:34:12.524489+0000","last_active":"2023-10-06T03:34:12.524489+0000","last_peered":"2023-10-06T03:34:12.524489+0000","last_clean":"2023-10-06T03:34:12.524489+0000","last_became_active":"2023-10-06T03:34:12.524226+0000","last_became_peered":"2023-10-06T03:34:12.524226+0000","last_unstale":"2023-10-06T03:34:12.524489+0000","last_undegraded":"2023-10-06T03:34:12.524489+0000","last_fullsized":"2023-10-06T03:34:12.524489+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:37.934361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.511450+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:10.034971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.532410+0000","last_change":"2023-10-06T03:34:12.532410+0000","last_active":"2023-10-06T03:34:12.532410+0000","last_peered":"2023-10-06T03:34:12.532410+0000","last_clean":"2023-10-06T03:34:12.532410+0000","last_became_active":"2023-10-06T03:34:12.532154+0000","last_became_peered":"2023-10-06T03:34:12.532154+0000","last_unstale":"2023-10-06T03:34:12.532410+0000","last_undegraded":"2023-10-06T03:34:12.532410+0000","last_fullsized":"2023-10-06T03:34:12.532410+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:05.259663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.512940+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:04.506250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.519345+0000","last_change":"2023-10-06T03:34:12.519345+0000","last_active":"2023-10-06T03:34:12.519345+0000","last_peered":"2023-10-06T03:34:12.519345+0000","last_clean":"2023-10-06T03:34:12.519345+0000","last_became_active":"2023-10-06T03:34:12.519053+0000","last_became_peered":"2023-10-06T03:34:12.519053+0000","last_unstale":"2023-10-06T03:34:12.519345+0000","last_undegraded":"2023-10-06T03:34:12.519345+0000","last_fullsized":"2023-10-06T03:34:12.519345+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:34.904232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.508577+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:05.851589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.522297+0000","last_change":"2023-10-06T03:34:12.522297+0000","last_active":"2023-10-06T03:34:12.522297+0000","last_peered":"2023-10-06T03:34:12.522297+0000","last_clean":"2023-10-06T03:34:12.522297+0000","last_became_active":"2023-10-06T03:34:12.522006+0000","last_became_peered":"2023-10-06T03:34:12.522006+0000","last_unstale":"2023-10-06T03:34:12.522297+0000","last_undegraded":"2023-10-06T03:34:12.522297+0000","last_fullsized":"2023-10-06T03:34:12.522297+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:39.716255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.519867+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:27.486440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.527419+0000","last_change":"2023-10-06T03:34:12.527419+0000","last_active":"2023-10-06T03:34:12.527419+0000","last_peered":"2023-10-06T03:34:12.527419+0000","last_clean":"2023-10-06T03:34:12.527419+0000","last_became_active":"2023-10-06T03:34:12.527141+0000","last_became_peered":"2023-10-06T03:34:12.527141+0000","last_unstale":"2023-10-06T03:34:12.527419+0000","last_undegraded":"2023-10-06T03:34:12.527419+0000","last_fullsized":"2023-10-06T03:34:12.527419+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:17.340207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.520568+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:01.766184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.521014+0000","last_change":"2023-10-06T03:34:12.521014+0000","last_active":"2023-10-06T03:34:12.521014+0000","last_peered":"2023-10-06T03:34:12.521014+0000","last_clean":"2023-10-06T03:34:12.521014+0000","last_became_active":"2023-10-06T03:34:12.520088+0000","last_became_peered":"2023-10-06T03:34:12.520088+0000","last_unstale":"2023-10-06T03:34:12.521014+0000","last_undegraded":"2023-10-06T03:34:12.521014+0000","last_fullsized":"2023-10-06T03:34:12.521014+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:34.651158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.507767+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:03.345245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.525259+0000","last_change":"2023-10-06T03:34:12.525259+0000","last_active":"2023-10-06T03:34:12.525259+0000","last_peered":"2023-10-06T03:34:12.525259+0000","last_clean":"2023-10-06T03:34:12.525259+0000","last_became_active":"2023-10-06T03:34:12.524902+0000","last_became_peered":"2023-10-06T03:34:12.524902+0000","last_unstale":"2023-10-06T03:34:12.525259+0000","last_undegraded":"2023-10-06T03:34:12.525259+0000","last_fullsized":"2023-10-06T03:34:12.525259+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:37.697037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.506037+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:34:11.506037+0000","last_peered":"2023-10-06T03:34:11.506037+0000","last_clean":"2023-10-06T03:34:11.506037+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:34:11.506037+0000","last_undegraded":"2023-10-06T03:34:11.506037+0000","last_fullsized":"2023-10-06T03:34:11.506037+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.513004+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:22.590165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.525282+0000","last_change":"2023-10-06T03:34:12.525282+0000","last_active":"2023-10-06T03:34:12.525282+0000","last_peered":"2023-10-06T03:34:12.525282+0000","last_clean":"2023-10-06T03:34:12.525282+0000","last_became_active":"2023-10-06T03:34:12.525020+0000","last_became_peered":"2023-10-06T03:34:12.525020+0000","last_unstale":"2023-10-06T03:34:12.525282+0000","last_undegraded":"2023-10-06T03:34:12.525282+0000","last_fullsized":"2023-10-06T03:34:12.525282+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:03.495179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.509800+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:34:11.509800+0000","last_peered":"2023-10-06T03:34:11.509800+0000","last_clean":"2023-10-06T03:34:11.509800+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:34:11.509800+0000","last_undegraded":"2023-10-06T03:34:11.509800+0000","last_fullsized":"2023-10-06T03:34:11.509800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.513140+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:54.559785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.529984+0000","last_change":"2023-10-06T03:34:12.529984+0000","last_active":"2023-10-06T03:34:12.529984+0000","last_peered":"2023-10-06T03:34:12.529984+0000","last_clean":"2023-10-06T03:34:12.529984+0000","last_became_active":"2023-10-06T03:34:12.529440+0000","last_became_peered":"2023-10-06T03:34:12.529440+0000","last_unstale":"2023-10-06T03:34:12.529984+0000","last_undegraded":"2023-10-06T03:34:12.529984+0000","last_fullsized":"2023-10-06T03:34:12.529984+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:04.840518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.512520+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:34:11.512520+0000","last_peered":"2023-10-06T03:34:11.512520+0000","last_clean":"2023-10-06T03:34:11.512520+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:34:11.512520+0000","last_undegraded":"2023-10-06T03:34:11.512520+0000","last_fullsized":"2023-10-06T03:34:11.512520+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":170,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.511358+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:34:11.511358+0000","last_peered":"2023-10-06T03:34:11.511358+0000","last_clean":"2023-10-06T03:34:11.511358+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:34:11.511358+0000","last_undegraded":"2023-10-06T03:34:11.511358+0000","last_fullsized":"2023-10-06T03:34:11.511358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.515713+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:01.442206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.526945+0000","last_change":"2023-10-06T03:34:12.526945+0000","last_active":"2023-10-06T03:34:12.526945+0000","last_peered":"2023-10-06T03:34:12.526945+0000","last_clean":"2023-10-06T03:34:12.526945+0000","last_became_active":"2023-10-06T03:34:12.526661+0000","last_became_peered":"2023-10-06T03:34:12.526661+0000","last_unstale":"2023-10-06T03:34:12.526945+0000","last_undegraded":"2023-10-06T03:34:12.526945+0000","last_fullsized":"2023-10-06T03:34:12.526945+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:36.885264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.511645+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:34:11.511645+0000","last_peered":"2023-10-06T03:34:11.511645+0000","last_clean":"2023-10-06T03:34:11.511645+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:34:11.511645+0000","last_undegraded":"2023-10-06T03:34:11.511645+0000","last_fullsized":"2023-10-06T03:34:11.511645+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.518813+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:10.048229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.529818+0000","last_change":"2023-10-06T03:34:12.529818+0000","last_active":"2023-10-06T03:34:12.529818+0000","last_peered":"2023-10-06T03:34:12.529818+0000","last_clean":"2023-10-06T03:34:12.529818+0000","last_became_active":"2023-10-06T03:34:12.529370+0000","last_became_peered":"2023-10-06T03:34:12.529370+0000","last_unstale":"2023-10-06T03:34:12.529818+0000","last_undegraded":"2023-10-06T03:34:12.529818+0000","last_fullsized":"2023-10-06T03:34:12.529818+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:54.385812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.511314+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:34:11.511314+0000","last_peered":"2023-10-06T03:34:11.511314+0000","last_clean":"2023-10-06T03:34:11.511314+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:34:11.511314+0000","last_undegraded":"2023-10-06T03:34:11.511314+0000","last_fullsized":"2023-10-06T03:34:11.511314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.511459+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:23.077437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.516230+0000","last_change":"2023-10-06T03:34:12.516230+0000","last_active":"2023-10-06T03:34:12.516230+0000","last_peered":"2023-10-06T03:34:12.516230+0000","last_clean":"2023-10-06T03:34:12.516230+0000","last_became_active":"2023-10-06T03:34:12.515968+0000","last_became_peered":"2023-10-06T03:34:12.515968+0000","last_unstale":"2023-10-06T03:34:12.516230+0000","last_undegraded":"2023-10-06T03:34:12.516230+0000","last_fullsized":"2023-10-06T03:34:12.516230+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.510836+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:34:11.510836+0000","last_peered":"2023-10-06T03:34:11.510836+0000","last_clean":"2023-10-06T03:34:11.510836+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:34:11.510836+0000","last_undegraded":"2023-10-06T03:34:11.510836+0000","last_fullsized":"2023-10-06T03:34:11.510836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.512235+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:18.351278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.519229+0000","last_change":"2023-10-06T03:34:12.519229+0000","last_active":"2023-10-06T03:34:12.519229+0000","last_peered":"2023-10-06T03:34:12.519229+0000","last_clean":"2023-10-06T03:34:12.519229+0000","last_became_active":"2023-10-06T03:34:12.518975+0000","last_became_peered":"2023-10-06T03:34:12.518975+0000","last_unstale":"2023-10-06T03:34:12.519229+0000","last_undegraded":"2023-10-06T03:34:12.519229+0000","last_fullsized":"2023-10-06T03:34:12.519229+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:24.627602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.506386+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:34:11.506386+0000","last_peered":"2023-10-06T03:34:11.506386+0000","last_clean":"2023-10-06T03:34:11.506386+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:34:11.506386+0000","last_undegraded":"2023-10-06T03:34:11.506386+0000","last_fullsized":"2023-10-06T03:34:11.506386+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.508478+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:18.351278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.514751+0000","last_change":"2023-10-06T03:34:12.514751+0000","last_active":"2023-10-06T03:34:12.514751+0000","last_peered":"2023-10-06T03:34:12.514751+0000","last_clean":"2023-10-06T03:34:12.514751+0000","last_became_active":"2023-10-06T03:34:12.514468+0000","last_became_peered":"2023-10-06T03:34:12.514468+0000","last_unstale":"2023-10-06T03:34:12.514751+0000","last_undegraded":"2023-10-06T03:34:12.514751+0000","last_fullsized":"2023-10-06T03:34:12.514751+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:07.994934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.506724+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:34:11.506724+0000","last_peered":"2023-10-06T03:34:11.506724+0000","last_clean":"2023-10-06T03:34:11.506724+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:34:11.506724+0000","last_undegraded":"2023-10-06T03:34:11.506724+0000","last_fullsized":"2023-10-06T03:34:11.506724+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298180,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073660,"statfs":{"total":96636764160,"available":96331427840,"internally_reserved":0,"allocated":913408,"data_stored":715338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59099999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.622}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.746}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":125948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.77200000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.78700000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.55700000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.83499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302280,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069560,"statfs":{"total":96636764160,"available":96327229440,"internally_reserved":0,"allocated":917504,"data_stored":716316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.63600000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.747}]},{"osd":6,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.623}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301696,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070144,"statfs":{"total":96636764160,"available":96327827456,"internally_reserved":0,"allocated":319488,"data_stored":124970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.76800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301696,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070144,"statfs":{"total":96636764160,"available":96327827456,"internally_reserved":0,"allocated":319488,"data_stored":124970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.51300000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.95399999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.73999999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.873}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.90200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301700,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070140,"statfs":{"total":96636764160,"available":96327823360,"internally_reserved":0,"allocated":323584,"data_stored":125948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44400000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.57099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":125948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.621}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.55500000000000005}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.747},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.77600000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.69799999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297600,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074240,"statfs":{"total":96636764160,"available":96332021760,"internally_reserved":0,"allocated":319488,"data_stored":124970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.629}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.56899999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.71499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:34:17.934 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:34:18.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.147+0000 7f40709fe700 1 Processor -- start 2023-10-06T03:34:18.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.147+0000 7f40709fe700 1 -- start start 2023-10-06T03:34:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.148+0000 7f40709fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f406c140cd0 0x7f406c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.148+0000 7f40709fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f406c1410a0 0x7f406c06bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.148+0000 7f40709fe700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406c0b1500 con 0x7f406c0ccd10 2023-10-06T03:34:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.148+0000 7f40709fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406c0cda20 con 0x7f406c140cd0 2023-10-06T03:34:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.148+0000 7f40709fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406c0cdd00 con 0x7f406c1410a0 2023-10-06T03:34:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.149+0000 7f406ad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f406c140cd0 0x7f406c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.149+0000 7f406b59e700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f406c0ccd10 0x7f406c0cd170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38010/0 (socket says 172.21.15.55:38010) 2023-10-06T03:34:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.149+0000 7f406b59e700 1 -- 172.21.15.55:0/1247743512 learned_addr learned my addr 172.21.15.55:0/1247743512 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.149+0000 7f4063fff700 1 --2- 172.21.15.55:0/1247743512 >> v2:172.21.15.163:3300/0 conn(0x7f406c1410a0 0x7f406c06bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.149+0000 7f406ad9d700 1 -- 172.21.15.55:0/1247743512 >> v2:172.21.15.163:3300/0 conn(0x7f406c1410a0 msgr2=0x7f406c06bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.149+0000 7f406ad9d700 1 --2- 172.21.15.55:0/1247743512 >> v2:172.21.15.163:3300/0 conn(0x7f406c1410a0 0x7f406c06bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.149+0000 7f406ad9d700 1 -- 172.21.15.55:0/1247743512 >> v1:172.21.15.55:6789/0 conn(0x7f406c0ccd10 legacy=0x7f406c0cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.149+0000 7f406ad9d700 1 -- 172.21.15.55:0/1247743512 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c0cdfe0 con 0x7f406c140cd0 2023-10-06T03:34:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.149+0000 7f406a59c700 1 -- 172.21.15.55:0/1247743512 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044645986 0 0) 0x7f406c0b1500 con 0x7f406c0ccd10 2023-10-06T03:34:18.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.149+0000 7f406ad9d700 1 --2- 172.21.15.55:0/1247743512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c140cd0 0x7f406c06b9e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f405c0098e0 tx=0x7f405c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156453bd05da433a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:18.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.150+0000 7f406a59c700 1 -- 172.21.15.55:0/1247743512 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c009e80 con 0x7f406c140cd0 2023-10-06T03:34:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.150+0000 7f406a59c700 1 -- 172.21.15.55:0/1247743512 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c00b820 con 0x7f406c140cd0 2023-10-06T03:34:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.150+0000 7f406a59c700 1 -- 172.21.15.55:0/1247743512 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c00bba0 con 0x7f406c140cd0 2023-10-06T03:34:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.150+0000 7f40709fe700 1 -- 172.21.15.55:0/1247743512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c140cd0 msgr2=0x7f406c06b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.150+0000 7f40709fe700 1 --2- 172.21.15.55:0/1247743512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c140cd0 0x7f406c06b9e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f405c0098e0 tx=0x7f405c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.150+0000 7f40709fe700 1 -- 172.21.15.55:0/1247743512 shutdown_connections 2023-10-06T03:34:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.150+0000 7f40709fe700 1 --2- 172.21.15.55:0/1247743512 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c140cd0 0x7f406c06b9e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.150+0000 7f40709fe700 1 --2- 172.21.15.55:0/1247743512 >> v2:172.21.15.163:3300/0 conn(0x7f406c1410a0 0x7f406c06bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.150+0000 7f40709fe700 1 -- 172.21.15.55:0/1247743512 >> 172.21.15.55:0/1247743512 conn(0x7f406c0ce430 msgr2=0x7f406c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.150+0000 7f40709fe700 1 -- 172.21.15.55:0/1247743512 shutdown_connections 2023-10-06T03:34:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.150+0000 7f40709fe700 1 -- 172.21.15.55:0/1247743512 wait complete. 2023-10-06T03:34:18.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.152+0000 7f40709fe700 1 Processor -- start 2023-10-06T03:34:18.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.152+0000 7f40709fe700 1 -- start start 2023-10-06T03:34:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f40709fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f406c0ccd10 0x7f406c0af780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f40709fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c140cd0 0x7f406c148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f40709fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f406c1410a0 0x7f406c0aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f40709fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c0af2f0 con 0x7f406c140cd0 2023-10-06T03:34:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f40709fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c0b0c00 con 0x7f406c1410a0 2023-10-06T03:34:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f40709fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406c0aece0 con 0x7f406c0ccd10 2023-10-06T03:34:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f4063fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c140cd0 0x7f406c148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f4063fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c140cd0 0x7f406c148d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39842/0 (socket says 172.21.15.55:39842) 2023-10-06T03:34:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f4063fff700 1 -- 172.21.15.55:0/1338946815 learned_addr learned my addr 172.21.15.55:0/1338946815 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f406b59e700 1 --2- 172.21.15.55:0/1338946815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f406c1410a0 0x7f406c0aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f4063fff700 1 -- 172.21.15.55:0/1338946815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f406c0ccd10 msgr2=0x7f406c0af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:18.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f4063fff700 1 --2- 172.21.15.55:0/1338946815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f406c0ccd10 0x7f406c0af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f4063fff700 1 -- 172.21.15.55:0/1338946815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f406c1410a0 msgr2=0x7f406c0aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:18.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f4063fff700 1 --2- 172.21.15.55:0/1338946815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f406c1410a0 0x7f406c0aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f4063fff700 1 -- 172.21.15.55:0/1338946815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c009420 con 0x7f406c140cd0 2023-10-06T03:34:18.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.153+0000 7f406b59e700 1 --2- 172.21.15.55:0/1338946815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f406c1410a0 0x7f406c0aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:18.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.154+0000 7f4063fff700 1 --2- 172.21.15.55:0/1338946815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c140cd0 0x7f406c148d20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f405400d860 tx=0x7f4054008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:18.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.154+0000 7f4062ffd700 1 -- 172.21.15.55:0/1338946815 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054009610 con 0x7f406c140cd0 2023-10-06T03:34:18.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.154+0000 7f4062ffd700 1 -- 172.21.15.55:0/1338946815 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4054009770 con 0x7f406c140cd0 2023-10-06T03:34:18.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.154+0000 7f4062ffd700 1 -- 172.21.15.55:0/1338946815 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054009b30 con 0x7f406c140cd0 2023-10-06T03:34:18.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.154+0000 7f40709fe700 1 -- 172.21.15.55:0/1338946815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406c0b0920 con 0x7f406c140cd0 2023-10-06T03:34:18.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.154+0000 7f40709fe700 1 -- 172.21.15.55:0/1338946815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c149260 con 0x7f406c140cd0 2023-10-06T03:34:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.155+0000 7f40709fe700 1 -- 172.21.15.55:0/1338946815 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406c04c3c0 con 0x7f406c140cd0 2023-10-06T03:34:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.156+0000 7f4062ffd700 1 -- 172.21.15.55:0/1338946815 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f405401e460 con 0x7f406c140cd0 2023-10-06T03:34:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.157+0000 7f4062ffd700 1 --2- 172.21.15.55:0/1338946815 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f404c066f30 0x7f404c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:18.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.157+0000 7f4062ffd700 1 -- 172.21.15.55:0/1338946815 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4054084cb0 con 0x7f406c140cd0 2023-10-06T03:34:18.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.157+0000 7f406ad9d700 1 --2- 172.21.15.55:0/1338946815 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f404c066f30 0x7f404c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:18.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.158+0000 7f406ad9d700 1 --2- 172.21.15.55:0/1338946815 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f404c066f30 0x7f404c0693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f405c0098e0 tx=0x7f405c017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:18.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.161+0000 7f4062ffd700 1 -- 172.21.15.55:0/1338946815 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4054051dc0 con 0x7f406c140cd0 2023-10-06T03:34:18.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.360+0000 7f40709fe700 1 -- 172.21.15.55:0/1338946815 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f406c0af430 con 0x7f404c066f30 2023-10-06T03:34:18.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.370+0000 7f4062ffd700 1 -- 172.21.15.55:0/1338946815 <== mgr.4114 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+308964 (secure 0 0 0) 0x7f406c0af430 con 0x7f404c066f30 2023-10-06T03:34:18.375 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:18.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.374+0000 7f40709fe700 1 -- 172.21.15.55:0/1338946815 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f404c066f30 msgr2=0x7f404c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:18.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.374+0000 7f40709fe700 1 --2- 172.21.15.55:0/1338946815 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f404c066f30 0x7f404c0693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f405c0098e0 tx=0x7f405c017000 comp rx=0 tx=0).stop 2023-10-06T03:34:18.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.374+0000 7f40709fe700 1 -- 172.21.15.55:0/1338946815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c140cd0 msgr2=0x7f406c148d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:18.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.374+0000 7f40709fe700 1 --2- 172.21.15.55:0/1338946815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c140cd0 0x7f406c148d20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f405400d860 tx=0x7f4054008040 comp rx=0 tx=0).stop 2023-10-06T03:34:18.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.374+0000 7f40709fe700 1 -- 172.21.15.55:0/1338946815 shutdown_connections 2023-10-06T03:34:18.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.374+0000 7f40709fe700 1 --2- 172.21.15.55:0/1338946815 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f406c1410a0 0x7f406c0aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.374+0000 7f40709fe700 1 --2- 172.21.15.55:0/1338946815 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f404c066f30 0x7f404c0693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.374+0000 7f40709fe700 1 --2- 172.21.15.55:0/1338946815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f406c0ccd10 0x7f406c0af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.374+0000 7f40709fe700 1 --2- 172.21.15.55:0/1338946815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f406c140cd0 0x7f406c148d20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.374+0000 7f40709fe700 1 -- 172.21.15.55:0/1338946815 >> 172.21.15.55:0/1338946815 conn(0x7f406c0ce430 msgr2=0x7f406c13cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:18.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.374+0000 7f40709fe700 1 -- 172.21.15.55:0/1338946815 shutdown_connections 2023-10-06T03:34:18.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.374+0000 7f40709fe700 1 -- 172.21.15.55:0/1338946815 wait complete. 2023-10-06T03:34:18.379 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:34:18.393 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2023-10-06T03:34:16.503849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398360,"kb_used_data":3672,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752576360,"statfs":{"total":773094113280,"available":770638192640,"internally_reserved":0,"allocated":3760128,"data_stored":2184408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.002301"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.511338+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:27.486440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.512011+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:38.277766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.511012+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:10.048229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.511933+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:47.323964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509810+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:12:17.520030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.512665+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:05.851589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.516769+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:04.506250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.514172+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:17.351579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510350+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:22.590165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.521902+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:01.442206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.520741+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:53.934064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510363+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:53.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.520333+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:38.277766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.507121+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:50.641614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.517941+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:53.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.514353+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:38.277766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.513164+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:10.048229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509840+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:28.353727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510480+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:43.771510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510338+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:44.589878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.518386+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:32.081718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509021+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:19.240735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.513444+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:23.077437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510723+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:25.638673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509780+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:47.323964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510560+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:09.006005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510517+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:10.034971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.520041+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:23.077437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510111+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:06.270734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509226+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:43.771510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.515227+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:48.475963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.511747+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:55.396883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.513200+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:27.486440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.523661+0000","last_change":"2023-10-06T03:34:12.523661+0000","last_active":"2023-10-06T03:34:12.523661+0000","last_peered":"2023-10-06T03:34:12.523661+0000","last_clean":"2023-10-06T03:34:12.523661+0000","last_became_active":"2023-10-06T03:34:12.523343+0000","last_became_peered":"2023-10-06T03:34:12.523343+0000","last_unstale":"2023-10-06T03:34:12.523661+0000","last_undegraded":"2023-10-06T03:34:12.523661+0000","last_fullsized":"2023-10-06T03:34:12.523661+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:23.796615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.511540+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:48.522789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.519828+0000","last_change":"2023-10-06T03:34:12.519828+0000","last_active":"2023-10-06T03:34:12.519828+0000","last_peered":"2023-10-06T03:34:12.519828+0000","last_clean":"2023-10-06T03:34:12.519828+0000","last_became_active":"2023-10-06T03:34:12.519596+0000","last_became_peered":"2023-10-06T03:34:12.519596+0000","last_unstale":"2023-10-06T03:34:12.519828+0000","last_undegraded":"2023-10-06T03:34:12.519828+0000","last_fullsized":"2023-10-06T03:34:12.519828+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:01.889807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.508223+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:25.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.526656+0000","last_change":"2023-10-06T03:34:12.526656+0000","last_active":"2023-10-06T03:34:12.526656+0000","last_peered":"2023-10-06T03:34:12.526656+0000","last_clean":"2023-10-06T03:34:12.526656+0000","last_became_active":"2023-10-06T03:34:12.525776+0000","last_became_peered":"2023-10-06T03:34:12.525776+0000","last_unstale":"2023-10-06T03:34:12.526656+0000","last_undegraded":"2023-10-06T03:34:12.526656+0000","last_fullsized":"2023-10-06T03:34:12.526656+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:27.342656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.512262+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:53.934064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.524322+0000","last_change":"2023-10-06T03:34:12.524322+0000","last_active":"2023-10-06T03:34:12.524322+0000","last_peered":"2023-10-06T03:34:12.524322+0000","last_clean":"2023-10-06T03:34:12.524322+0000","last_became_active":"2023-10-06T03:34:12.524037+0000","last_became_peered":"2023-10-06T03:34:12.524037+0000","last_unstale":"2023-10-06T03:34:12.524322+0000","last_undegraded":"2023-10-06T03:34:12.524322+0000","last_fullsized":"2023-10-06T03:34:12.524322+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:59.774790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509777+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:04.506250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.527023+0000","last_change":"2023-10-06T03:34:12.527023+0000","last_active":"2023-10-06T03:34:12.527023+0000","last_peered":"2023-10-06T03:34:12.527023+0000","last_clean":"2023-10-06T03:34:12.527023+0000","last_became_active":"2023-10-06T03:34:12.526443+0000","last_became_peered":"2023-10-06T03:34:12.526443+0000","last_unstale":"2023-10-06T03:34:12.527023+0000","last_undegraded":"2023-10-06T03:34:12.527023+0000","last_fullsized":"2023-10-06T03:34:12.527023+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:54.385812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.517298+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:22.590165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.532851+0000","last_change":"2023-10-06T03:34:12.532851+0000","last_active":"2023-10-06T03:34:12.532851+0000","last_peered":"2023-10-06T03:34:12.532851+0000","last_clean":"2023-10-06T03:34:12.532851+0000","last_became_active":"2023-10-06T03:34:12.532517+0000","last_became_peered":"2023-10-06T03:34:12.532517+0000","last_unstale":"2023-10-06T03:34:12.532851+0000","last_undegraded":"2023-10-06T03:34:12.532851+0000","last_fullsized":"2023-10-06T03:34:12.532851+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:04.840518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.508297+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:17.622122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.527549+0000","last_change":"2023-10-06T03:34:12.527549+0000","last_active":"2023-10-06T03:34:12.527549+0000","last_peered":"2023-10-06T03:34:12.527549+0000","last_clean":"2023-10-06T03:34:12.527549+0000","last_became_active":"2023-10-06T03:34:12.527236+0000","last_became_peered":"2023-10-06T03:34:12.527236+0000","last_unstale":"2023-10-06T03:34:12.527549+0000","last_undegraded":"2023-10-06T03:34:12.527549+0000","last_fullsized":"2023-10-06T03:34:12.527549+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:43.578807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.516316+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:30.492483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.516603+0000","last_change":"2023-10-06T03:34:12.516603+0000","last_active":"2023-10-06T03:34:12.516603+0000","last_peered":"2023-10-06T03:34:12.516603+0000","last_clean":"2023-10-06T03:34:12.516603+0000","last_became_active":"2023-10-06T03:34:12.516217+0000","last_became_peered":"2023-10-06T03:34:12.516217+0000","last_unstale":"2023-10-06T03:34:12.516603+0000","last_undegraded":"2023-10-06T03:34:12.516603+0000","last_fullsized":"2023-10-06T03:34:12.516603+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:16.340508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.510060+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:35:24.431295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.523775+0000","last_change":"2023-10-06T03:34:12.523775+0000","last_active":"2023-10-06T03:34:12.523775+0000","last_peered":"2023-10-06T03:34:12.523775+0000","last_clean":"2023-10-06T03:34:12.523775+0000","last_became_active":"2023-10-06T03:34:12.523512+0000","last_became_peered":"2023-10-06T03:34:12.523512+0000","last_unstale":"2023-10-06T03:34:12.523775+0000","last_undegraded":"2023-10-06T03:34:12.523775+0000","last_fullsized":"2023-10-06T03:34:12.523775+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:21.579094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.516255+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:17.351579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.528843+0000","last_change":"2023-10-06T03:34:12.528843+0000","last_active":"2023-10-06T03:34:12.528843+0000","last_peered":"2023-10-06T03:34:12.528843+0000","last_clean":"2023-10-06T03:34:12.528843+0000","last_became_active":"2023-10-06T03:34:12.528575+0000","last_became_peered":"2023-10-06T03:34:12.528575+0000","last_unstale":"2023-10-06T03:34:12.528843+0000","last_undegraded":"2023-10-06T03:34:12.528843+0000","last_fullsized":"2023-10-06T03:34:12.528843+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:07.994934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509203+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:32.081718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.529757+0000","last_change":"2023-10-06T03:34:12.529757+0000","last_active":"2023-10-06T03:34:12.529757+0000","last_peered":"2023-10-06T03:34:12.529757+0000","last_clean":"2023-10-06T03:34:12.529757+0000","last_became_active":"2023-10-06T03:34:12.529371+0000","last_became_peered":"2023-10-06T03:34:12.529371+0000","last_unstale":"2023-10-06T03:34:12.529757+0000","last_undegraded":"2023-10-06T03:34:12.529757+0000","last_fullsized":"2023-10-06T03:34:12.529757+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:17.340207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.508885+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:35.437314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.528656+0000","last_change":"2023-10-06T03:34:12.528656+0000","last_active":"2023-10-06T03:34:12.528656+0000","last_peered":"2023-10-06T03:34:12.528656+0000","last_clean":"2023-10-06T03:34:12.528656+0000","last_became_active":"2023-10-06T03:34:12.528361+0000","last_became_peered":"2023-10-06T03:34:12.528361+0000","last_unstale":"2023-10-06T03:34:12.528656+0000","last_undegraded":"2023-10-06T03:34:12.528656+0000","last_fullsized":"2023-10-06T03:34:12.528656+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:31.070647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.509144+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:54.559785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.523998+0000","last_change":"2023-10-06T03:34:12.523998+0000","last_active":"2023-10-06T03:34:12.523998+0000","last_peered":"2023-10-06T03:34:12.523998+0000","last_clean":"2023-10-06T03:34:12.523998+0000","last_became_active":"2023-10-06T03:34:12.523688+0000","last_became_peered":"2023-10-06T03:34:12.523688+0000","last_unstale":"2023-10-06T03:34:12.523998+0000","last_undegraded":"2023-10-06T03:34:12.523998+0000","last_fullsized":"2023-10-06T03:34:12.523998+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:52.611432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.513908+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:01.442206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.530728+0000","last_change":"2023-10-06T03:34:12.530728+0000","last_active":"2023-10-06T03:34:12.530728+0000","last_peered":"2023-10-06T03:34:12.530728+0000","last_clean":"2023-10-06T03:34:12.530728+0000","last_became_active":"2023-10-06T03:34:12.530458+0000","last_became_peered":"2023-10-06T03:34:12.530458+0000","last_unstale":"2023-10-06T03:34:12.530728+0000","last_undegraded":"2023-10-06T03:34:12.530728+0000","last_fullsized":"2023-10-06T03:34:12.530728+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:09.423072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.513144+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:53.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.517112+0000","last_change":"2023-10-06T03:34:12.517112+0000","last_active":"2023-10-06T03:34:12.517112+0000","last_peered":"2023-10-06T03:34:12.517112+0000","last_clean":"2023-10-06T03:34:12.517112+0000","last_became_active":"2023-10-06T03:34:12.516831+0000","last_became_peered":"2023-10-06T03:34:12.516831+0000","last_unstale":"2023-10-06T03:34:12.517112+0000","last_undegraded":"2023-10-06T03:34:12.517112+0000","last_fullsized":"2023-10-06T03:34:12.517112+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:48.752619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.521103+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:17.622122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.532903+0000","last_change":"2023-10-06T03:34:12.532903+0000","last_active":"2023-10-06T03:34:12.532903+0000","last_peered":"2023-10-06T03:34:12.532903+0000","last_clean":"2023-10-06T03:34:12.532903+0000","last_became_active":"2023-10-06T03:34:12.532576+0000","last_became_peered":"2023-10-06T03:34:12.532576+0000","last_unstale":"2023-10-06T03:34:12.532903+0000","last_undegraded":"2023-10-06T03:34:12.532903+0000","last_fullsized":"2023-10-06T03:34:12.532903+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.514884+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:48.475963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.532478+0000","last_change":"2023-10-06T03:34:12.532478+0000","last_active":"2023-10-06T03:34:12.532478+0000","last_peered":"2023-10-06T03:34:12.532478+0000","last_clean":"2023-10-06T03:34:12.532478+0000","last_became_active":"2023-10-06T03:34:12.532204+0000","last_became_peered":"2023-10-06T03:34:12.532204+0000","last_unstale":"2023-10-06T03:34:12.532478+0000","last_undegraded":"2023-10-06T03:34:12.532478+0000","last_fullsized":"2023-10-06T03:34:12.532478+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.521299+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:48.475963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.527219+0000","last_change":"2023-10-06T03:34:12.527219+0000","last_active":"2023-10-06T03:34:12.527219+0000","last_peered":"2023-10-06T03:34:12.527219+0000","last_clean":"2023-10-06T03:34:12.527219+0000","last_became_active":"2023-10-06T03:34:12.526985+0000","last_became_peered":"2023-10-06T03:34:12.526985+0000","last_unstale":"2023-10-06T03:34:12.527219+0000","last_undegraded":"2023-10-06T03:34:12.527219+0000","last_fullsized":"2023-10-06T03:34:12.527219+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:29.481412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.507630+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:01.766184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.524489+0000","last_change":"2023-10-06T03:34:12.524489+0000","last_active":"2023-10-06T03:34:12.524489+0000","last_peered":"2023-10-06T03:34:12.524489+0000","last_clean":"2023-10-06T03:34:12.524489+0000","last_became_active":"2023-10-06T03:34:12.524226+0000","last_became_peered":"2023-10-06T03:34:12.524226+0000","last_unstale":"2023-10-06T03:34:12.524489+0000","last_undegraded":"2023-10-06T03:34:12.524489+0000","last_fullsized":"2023-10-06T03:34:12.524489+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:37.934361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.511450+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:10.034971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.532410+0000","last_change":"2023-10-06T03:34:12.532410+0000","last_active":"2023-10-06T03:34:12.532410+0000","last_peered":"2023-10-06T03:34:12.532410+0000","last_clean":"2023-10-06T03:34:12.532410+0000","last_became_active":"2023-10-06T03:34:12.532154+0000","last_became_peered":"2023-10-06T03:34:12.532154+0000","last_unstale":"2023-10-06T03:34:12.532410+0000","last_undegraded":"2023-10-06T03:34:12.532410+0000","last_fullsized":"2023-10-06T03:34:12.532410+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:05.259663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.512940+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:04.506250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.519345+0000","last_change":"2023-10-06T03:34:12.519345+0000","last_active":"2023-10-06T03:34:12.519345+0000","last_peered":"2023-10-06T03:34:12.519345+0000","last_clean":"2023-10-06T03:34:12.519345+0000","last_became_active":"2023-10-06T03:34:12.519053+0000","last_became_peered":"2023-10-06T03:34:12.519053+0000","last_unstale":"2023-10-06T03:34:12.519345+0000","last_undegraded":"2023-10-06T03:34:12.519345+0000","last_fullsized":"2023-10-06T03:34:12.519345+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:34.904232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.508577+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:05.851589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.522297+0000","last_change":"2023-10-06T03:34:12.522297+0000","last_active":"2023-10-06T03:34:12.522297+0000","last_peered":"2023-10-06T03:34:12.522297+0000","last_clean":"2023-10-06T03:34:12.522297+0000","last_became_active":"2023-10-06T03:34:12.522006+0000","last_became_peered":"2023-10-06T03:34:12.522006+0000","last_unstale":"2023-10-06T03:34:12.522297+0000","last_undegraded":"2023-10-06T03:34:12.522297+0000","last_fullsized":"2023-10-06T03:34:12.522297+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:39.716255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.519867+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:27.486440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.527419+0000","last_change":"2023-10-06T03:34:12.527419+0000","last_active":"2023-10-06T03:34:12.527419+0000","last_peered":"2023-10-06T03:34:12.527419+0000","last_clean":"2023-10-06T03:34:12.527419+0000","last_became_active":"2023-10-06T03:34:12.527141+0000","last_became_peered":"2023-10-06T03:34:12.527141+0000","last_unstale":"2023-10-06T03:34:12.527419+0000","last_undegraded":"2023-10-06T03:34:12.527419+0000","last_fullsized":"2023-10-06T03:34:12.527419+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:17.340207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.520568+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:01.766184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.521014+0000","last_change":"2023-10-06T03:34:12.521014+0000","last_active":"2023-10-06T03:34:12.521014+0000","last_peered":"2023-10-06T03:34:12.521014+0000","last_clean":"2023-10-06T03:34:12.521014+0000","last_became_active":"2023-10-06T03:34:12.520088+0000","last_became_peered":"2023-10-06T03:34:12.520088+0000","last_unstale":"2023-10-06T03:34:12.521014+0000","last_undegraded":"2023-10-06T03:34:12.521014+0000","last_fullsized":"2023-10-06T03:34:12.521014+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:34.651158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.507767+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:03.345245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.525259+0000","last_change":"2023-10-06T03:34:12.525259+0000","last_active":"2023-10-06T03:34:12.525259+0000","last_peered":"2023-10-06T03:34:12.525259+0000","last_clean":"2023-10-06T03:34:12.525259+0000","last_became_active":"2023-10-06T03:34:12.524902+0000","last_became_peered":"2023-10-06T03:34:12.524902+0000","last_unstale":"2023-10-06T03:34:12.525259+0000","last_undegraded":"2023-10-06T03:34:12.525259+0000","last_fullsized":"2023-10-06T03:34:12.525259+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:37.697037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.506037+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:34:11.506037+0000","last_peered":"2023-10-06T03:34:11.506037+0000","last_clean":"2023-10-06T03:34:11.506037+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:34:11.506037+0000","last_undegraded":"2023-10-06T03:34:11.506037+0000","last_fullsized":"2023-10-06T03:34:11.506037+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.513004+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:22.590165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.525282+0000","last_change":"2023-10-06T03:34:12.525282+0000","last_active":"2023-10-06T03:34:12.525282+0000","last_peered":"2023-10-06T03:34:12.525282+0000","last_clean":"2023-10-06T03:34:12.525282+0000","last_became_active":"2023-10-06T03:34:12.525020+0000","last_became_peered":"2023-10-06T03:34:12.525020+0000","last_unstale":"2023-10-06T03:34:12.525282+0000","last_undegraded":"2023-10-06T03:34:12.525282+0000","last_fullsized":"2023-10-06T03:34:12.525282+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:03.495179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.509800+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:34:11.509800+0000","last_peered":"2023-10-06T03:34:11.509800+0000","last_clean":"2023-10-06T03:34:11.509800+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:34:11.509800+0000","last_undegraded":"2023-10-06T03:34:11.509800+0000","last_fullsized":"2023-10-06T03:34:11.509800+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.513140+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:54.559785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.529984+0000","last_change":"2023-10-06T03:34:12.529984+0000","last_active":"2023-10-06T03:34:12.529984+0000","last_peered":"2023-10-06T03:34:12.529984+0000","last_clean":"2023-10-06T03:34:12.529984+0000","last_became_active":"2023-10-06T03:34:12.529440+0000","last_became_peered":"2023-10-06T03:34:12.529440+0000","last_unstale":"2023-10-06T03:34:12.529984+0000","last_undegraded":"2023-10-06T03:34:12.529984+0000","last_fullsized":"2023-10-06T03:34:12.529984+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:04.840518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.512520+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:34:11.512520+0000","last_peered":"2023-10-06T03:34:11.512520+0000","last_clean":"2023-10-06T03:34:11.512520+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:34:11.512520+0000","last_undegraded":"2023-10-06T03:34:11.512520+0000","last_fullsized":"2023-10-06T03:34:11.512520+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":170,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.511358+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:34:11.511358+0000","last_peered":"2023-10-06T03:34:11.511358+0000","last_clean":"2023-10-06T03:34:11.511358+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:34:11.511358+0000","last_undegraded":"2023-10-06T03:34:11.511358+0000","last_fullsized":"2023-10-06T03:34:11.511358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.515713+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:01.442206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.526945+0000","last_change":"2023-10-06T03:34:12.526945+0000","last_active":"2023-10-06T03:34:12.526945+0000","last_peered":"2023-10-06T03:34:12.526945+0000","last_clean":"2023-10-06T03:34:12.526945+0000","last_became_active":"2023-10-06T03:34:12.526661+0000","last_became_peered":"2023-10-06T03:34:12.526661+0000","last_unstale":"2023-10-06T03:34:12.526945+0000","last_undegraded":"2023-10-06T03:34:12.526945+0000","last_fullsized":"2023-10-06T03:34:12.526945+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:36.885264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.511645+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:34:11.511645+0000","last_peered":"2023-10-06T03:34:11.511645+0000","last_clean":"2023-10-06T03:34:11.511645+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:34:11.511645+0000","last_undegraded":"2023-10-06T03:34:11.511645+0000","last_fullsized":"2023-10-06T03:34:11.511645+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.518813+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:10.048229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.529818+0000","last_change":"2023-10-06T03:34:12.529818+0000","last_active":"2023-10-06T03:34:12.529818+0000","last_peered":"2023-10-06T03:34:12.529818+0000","last_clean":"2023-10-06T03:34:12.529818+0000","last_became_active":"2023-10-06T03:34:12.529370+0000","last_became_peered":"2023-10-06T03:34:12.529370+0000","last_unstale":"2023-10-06T03:34:12.529818+0000","last_undegraded":"2023-10-06T03:34:12.529818+0000","last_fullsized":"2023-10-06T03:34:12.529818+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:54.385812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.511314+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:34:11.511314+0000","last_peered":"2023-10-06T03:34:11.511314+0000","last_clean":"2023-10-06T03:34:11.511314+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:34:11.511314+0000","last_undegraded":"2023-10-06T03:34:11.511314+0000","last_fullsized":"2023-10-06T03:34:11.511314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.511459+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:23.077437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.516230+0000","last_change":"2023-10-06T03:34:12.516230+0000","last_active":"2023-10-06T03:34:12.516230+0000","last_peered":"2023-10-06T03:34:12.516230+0000","last_clean":"2023-10-06T03:34:12.516230+0000","last_became_active":"2023-10-06T03:34:12.515968+0000","last_became_peered":"2023-10-06T03:34:12.515968+0000","last_unstale":"2023-10-06T03:34:12.516230+0000","last_undegraded":"2023-10-06T03:34:12.516230+0000","last_fullsized":"2023-10-06T03:34:12.516230+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.510836+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:34:11.510836+0000","last_peered":"2023-10-06T03:34:11.510836+0000","last_clean":"2023-10-06T03:34:11.510836+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:34:11.510836+0000","last_undegraded":"2023-10-06T03:34:11.510836+0000","last_fullsized":"2023-10-06T03:34:11.510836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.512235+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:18.351278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.519229+0000","last_change":"2023-10-06T03:34:12.519229+0000","last_active":"2023-10-06T03:34:12.519229+0000","last_peered":"2023-10-06T03:34:12.519229+0000","last_clean":"2023-10-06T03:34:12.519229+0000","last_became_active":"2023-10-06T03:34:12.518975+0000","last_became_peered":"2023-10-06T03:34:12.518975+0000","last_unstale":"2023-10-06T03:34:12.519229+0000","last_undegraded":"2023-10-06T03:34:12.519229+0000","last_fullsized":"2023-10-06T03:34:12.519229+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:24.627602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.506386+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:34:11.506386+0000","last_peered":"2023-10-06T03:34:11.506386+0000","last_clean":"2023-10-06T03:34:11.506386+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:34:11.506386+0000","last_undegraded":"2023-10-06T03:34:11.506386+0000","last_fullsized":"2023-10-06T03:34:11.506386+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:34:12.497789+0000","last_change":"2023-10-06T03:34:12.508478+0000","last_active":"2023-10-06T03:34:12.497789+0000","last_peered":"2023-10-06T03:34:12.497789+0000","last_clean":"2023-10-06T03:34:12.497789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:34:12.497789+0000","last_undegraded":"2023-10-06T03:34:12.497789+0000","last_fullsized":"2023-10-06T03:34:12.497789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:18.351278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-06T03:34:12.514751+0000","last_change":"2023-10-06T03:34:12.514751+0000","last_active":"2023-10-06T03:34:12.514751+0000","last_peered":"2023-10-06T03:34:12.514751+0000","last_clean":"2023-10-06T03:34:12.514751+0000","last_became_active":"2023-10-06T03:34:12.514468+0000","last_became_peered":"2023-10-06T03:34:12.514468+0000","last_unstale":"2023-10-06T03:34:12.514751+0000","last_undegraded":"2023-10-06T03:34:12.514751+0000","last_fullsized":"2023-10-06T03:34:12.514751+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:07.994934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-06T03:34:11.506724+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:34:11.506724+0000","last_peered":"2023-10-06T03:34:11.506724+0000","last_clean":"2023-10-06T03:34:11.506724+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:34:11.506724+0000","last_undegraded":"2023-10-06T03:34:11.506724+0000","last_fullsized":"2023-10-06T03:34:11.506724+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298180,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073660,"statfs":{"total":96636764160,"available":96331427840,"internally_reserved":0,"allocated":913408,"data_stored":715338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59099999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.622}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.746}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":125948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.77200000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.78700000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.55700000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.83499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302280,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069560,"statfs":{"total":96636764160,"available":96327229440,"internally_reserved":0,"allocated":917504,"data_stored":716316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.54100000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.63600000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.747}]},{"osd":6,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.623}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301696,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070144,"statfs":{"total":96636764160,"available":96327827456,"internally_reserved":0,"allocated":319488,"data_stored":124970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.76800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301696,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070144,"statfs":{"total":96636764160,"available":96327827456,"internally_reserved":0,"allocated":319488,"data_stored":124970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.51300000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.95399999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.73999999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.873}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.90200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301700,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070140,"statfs":{"total":96636764160,"available":96327823360,"internally_reserved":0,"allocated":323584,"data_stored":125948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.44400000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.57099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297604,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074236,"statfs":{"total":96636764160,"available":96332017664,"internally_reserved":0,"allocated":323584,"data_stored":125948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.621}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.55500000000000005}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.747},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.77600000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.69799999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297600,"kb_used_data":312,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074240,"statfs":{"total":96636764160,"available":96332021760,"internally_reserved":0,"allocated":319488,"data_stored":124970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.629}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.56899999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.71499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:34:18.400 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:34:18.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.620+0000 7fbd4a59d700 1 Processor -- start 2023-10-06T03:34:18.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.620+0000 7fbd4a59d700 1 -- start start 2023-10-06T03:34:18.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.621+0000 7fbd4a59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4415d230 0x7fbd4415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:18.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.621+0000 7fbd4a59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd440b2980 con 0x7fbd44159420 2023-10-06T03:34:18.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.621+0000 7fbd4a59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4415dbb0 con 0x7fbd441597f0 2023-10-06T03:34:18.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.621+0000 7fbd4a59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd440b11d0 con 0x7fbd4415d230 2023-10-06T03:34:18.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.622+0000 7fbd4959b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbd44159420 0x7fbd440b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38012/0 (socket says 172.21.15.55:38012) 2023-10-06T03:34:18.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.622+0000 7fbd4959b700 1 -- 172.21.15.55:0/3425488174 learned_addr learned my addr 172.21.15.55:0/3425488174 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:18.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.622+0000 7fbd49d9c700 1 --2- 172.21.15.55:0/3425488174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4415d230 0x7fbd4415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:18.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.622+0000 7fbd3bfff700 1 -- 172.21.15.55:0/3425488174 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162666852 0 0) 0x7fbd440b2980 con 0x7fbd44159420 2023-10-06T03:34:18.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.622+0000 7fbd3bfff700 1 -- 172.21.15.55:0/3425488174 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd30003570 con 0x7fbd44159420 2023-10-06T03:34:18.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.622+0000 7fbd3bfff700 1 -- 172.21.15.55:0/3425488174 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1843793584 0 0) 0x7fbd30003570 con 0x7fbd44159420 2023-10-06T03:34:18.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.622+0000 7fbd3bfff700 1 -- 172.21.15.55:0/3425488174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4415d230 msgr2=0x7fbd4415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:18.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.622+0000 7fbd3bfff700 1 --2- 172.21.15.55:0/3425488174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4415d230 0x7fbd4415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd3bfff700 1 -- 172.21.15.55:0/3425488174 >> v1:172.21.15.163:6789/0 conn(0x7fbd441597f0 legacy=0x7fbd440b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:18.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd3bfff700 1 -- 172.21.15.55:0/3425488174 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd440b14b0 con 0x7fbd44159420 2023-10-06T03:34:18.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd3bfff700 1 -- 172.21.15.55:0/3425488174 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fbd40001470 con 0x7fbd44159420 2023-10-06T03:34:18.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd3bfff700 1 -- 172.21.15.55:0/3425488174 >> v1:172.21.15.55:6789/0 conn(0x7fbd44159420 legacy=0x7fbd440b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:18.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd3bfff700 1 --2- 172.21.15.55:0/3425488174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd30003a90 0x7fbd30003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:18.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd3bfff700 1 --2- 172.21.15.55:0/3425488174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd30004810 0x7fbd30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:18.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd3bfff700 1 --2- 172.21.15.55:0/3425488174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4415d230 0x7fbd300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:18.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd3bfff700 1 -- 172.21.15.55:0/3425488174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd440b11d0 con 0x7fbd30004810 2023-10-06T03:34:18.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd3bfff700 1 -- 172.21.15.55:0/3425488174 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3000bb10 con 0x7fbd30003a90 2023-10-06T03:34:18.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd3bfff700 1 -- 172.21.15.55:0/3425488174 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3000bcd0 con 0x7fbd4415d230 2023-10-06T03:34:18.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd48d9a700 1 --2- 172.21.15.55:0/3425488174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd30004810 0x7fbd30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:18.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd4959b700 1 --2- 172.21.15.55:0/3425488174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd30003a90 0x7fbd30003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:18.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd49d9c700 1 --2- 172.21.15.55:0/3425488174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4415d230 0x7fbd300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:18.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd48d9a700 1 -- 172.21.15.55:0/3425488174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4415d230 msgr2=0x7fbd300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:18.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd48d9a700 1 --2- 172.21.15.55:0/3425488174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd4415d230 0x7fbd300051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd48d9a700 1 -- 172.21.15.55:0/3425488174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd30003a90 msgr2=0x7fbd30003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:18.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd48d9a700 1 --2- 172.21.15.55:0/3425488174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd30003a90 0x7fbd30003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.623+0000 7fbd48d9a700 1 -- 172.21.15.55:0/3425488174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd30003570 con 0x7fbd30004810 2023-10-06T03:34:18.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.624+0000 7fbd48d9a700 1 --2- 172.21.15.55:0/3425488174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd30004810 0x7fbd30004c80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbd34001930 tx=0x7fbd34007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f972d3a7e11d5c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:18.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.624+0000 7fbd3bfff700 1 -- 172.21.15.55:0/3425488174 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400c5d0 con 0x7fbd30004810 2023-10-06T03:34:18.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.624+0000 7fbd3bfff700 1 -- 172.21.15.55:0/3425488174 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3400c750 con 0x7fbd30004810 2023-10-06T03:34:18.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.624+0000 7fbd3bfff700 1 -- 172.21.15.55:0/3425488174 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400cb10 con 0x7fbd30004810 2023-10-06T03:34:18.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.624+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3425488174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd30004810 msgr2=0x7fbd30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:18.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.624+0000 7fbd4a59d700 1 --2- 172.21.15.55:0/3425488174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd30004810 0x7fbd30004c80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbd34001930 tx=0x7fbd34007d10 comp rx=0 tx=0).stop 2023-10-06T03:34:18.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.624+0000 7fbd49d9c700 1 -- 172.21.15.55:0/3425488174 reap_dead start 2023-10-06T03:34:18.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.625+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3425488174 shutdown_connections 2023-10-06T03:34:18.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.625+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3425488174 >> 172.21.15.55:0/3425488174 conn(0x7fbd440d0660 msgr2=0x7fbd440d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:18.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.625+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3425488174 shutdown_connections 2023-10-06T03:34:18.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.625+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3425488174 wait complete. 2023-10-06T03:34:18.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.626+0000 7fbd4a59d700 1 Processor -- start 2023-10-06T03:34:18.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.626+0000 7fbd4a59d700 1 -- start start 2023-10-06T03:34:18.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.627+0000 7fbd4a59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd30004810 0x7fbd44155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:18.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.627+0000 7fbd4a59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd440c6b30 0x7fbd440c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:18.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.627+0000 7fbd4a59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd44153f70 0x7fbd44154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.627+0000 7fbd4a59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd44154950 con 0x7fbd30004810 2023-10-06T03:34:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.627+0000 7fbd4a59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4416df20 con 0x7fbd44153f70 2023-10-06T03:34:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.627+0000 7fbd4a59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4415cb00 con 0x7fbd440c6b30 2023-10-06T03:34:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.627+0000 7fbd4959b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd30004810 0x7fbd44155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.627+0000 7fbd4959b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd30004810 0x7fbd44155f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39868/0 (socket says 172.21.15.55:39868) 2023-10-06T03:34:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.627+0000 7fbd4959b700 1 -- 172.21.15.55:0/3891393994 learned_addr learned my addr 172.21.15.55:0/3891393994 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.628+0000 7fbd49d9c700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd44153f70 0x7fbd44154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.628+0000 7fbd48d9a700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd440c6b30 0x7fbd440c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.628+0000 7fbd4959b700 1 -- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd440c6b30 msgr2=0x7fbd440c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.628+0000 7fbd4959b700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd440c6b30 0x7fbd440c6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.628+0000 7fbd4959b700 1 -- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd44153f70 msgr2=0x7fbd44154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.628+0000 7fbd4959b700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd44153f70 0x7fbd44154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.628+0000 7fbd4959b700 1 -- 172.21.15.55:0/3891393994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd440c95c0 con 0x7fbd30004810 2023-10-06T03:34:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.628+0000 7fbd49d9c700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd44153f70 0x7fbd44154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.628+0000 7fbd4959b700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd30004810 0x7fbd44155f40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbd400073f0 tx=0x7fbd40001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.628+0000 7fbd397fa700 1 -- 172.21.15.55:0/3891393994 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40018030 con 0x7fbd30004810 2023-10-06T03:34:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.628+0000 7fbd397fa700 1 -- 172.21.15.55:0/3891393994 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4000f660 con 0x7fbd30004810 2023-10-06T03:34:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.628+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3891393994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4416d310 con 0x7fbd30004810 2023-10-06T03:34:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.628+0000 7fbd397fa700 1 -- 172.21.15.55:0/3891393994 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4000fa00 con 0x7fbd30004810 2023-10-06T03:34:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.628+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3891393994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4416d8d0 con 0x7fbd30004810 2023-10-06T03:34:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.629+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3891393994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4404c3c0 con 0x7fbd30004810 2023-10-06T03:34:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.630+0000 7fbd397fa700 1 -- 172.21.15.55:0/3891393994 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd4001b070 con 0x7fbd30004810 2023-10-06T03:34:18.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.631+0000 7fbd397fa700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbd28066f30 0x7fbd280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:18.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.631+0000 7fbd397fa700 1 -- 172.21.15.55:0/3891393994 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbd40019070 con 0x7fbd30004810 2023-10-06T03:34:18.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.631+0000 7fbd48d9a700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbd28066f30 0x7fbd280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:18.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.632+0000 7fbd48d9a700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbd28066f30 0x7fbd280693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbd34008fd0 tx=0x7fbd34015000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:18.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.635+0000 7fbd397fa700 1 -- 172.21.15.55:0/3891393994 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd400550a0 con 0x7fbd30004810 2023-10-06T03:34:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.835+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3891393994 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbd440b1790 con 0x7fbd28066f30 2023-10-06T03:34:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.845+0000 7fbd397fa700 1 -- 172.21.15.55:0/3891393994 <== mgr.4114 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311808 (secure 0 0 0) 0x7fbd440b1790 con 0x7fbd28066f30 2023-10-06T03:34:18.850 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.849+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbd28066f30 msgr2=0x7fbd280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.849+0000 7fbd4a59d700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbd28066f30 0x7fbd280693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbd34008fd0 tx=0x7fbd34015000 comp rx=0 tx=0).stop 2023-10-06T03:34:18.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.849+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3891393994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd30004810 msgr2=0x7fbd44155f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.849+0000 7fbd4a59d700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd30004810 0x7fbd44155f40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbd400073f0 tx=0x7fbd40001e70 comp rx=0 tx=0).stop 2023-10-06T03:34:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.849+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3891393994 shutdown_connections 2023-10-06T03:34:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.849+0000 7fbd4a59d700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd44153f70 0x7fbd44154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.849+0000 7fbd4a59d700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd30004810 0x7fbd44155f40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.849+0000 7fbd4a59d700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbd28066f30 0x7fbd280693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.849+0000 7fbd4a59d700 1 --2- 172.21.15.55:0/3891393994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd440c6b30 0x7fbd440c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.849+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3891393994 >> 172.21.15.55:0/3891393994 conn(0x7fbd440d0660 msgr2=0x7fbd44155180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.849+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3891393994 shutdown_connections 2023-10-06T03:34:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:18.849+0000 7fbd4a59d700 1 -- 172.21.15.55:0/3891393994 wait complete. 2023-10-06T03:34:18.853 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:34:18.868 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-10-06T03:34:18.504345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398712,"kb_used_data":4024,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752576008,"statfs":{"total":773094113280,"available":770637832192,"internally_reserved":0,"allocated":4120576,"data_stored":2352004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001321"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.548919+0000","last_change":"2023-10-06T03:34:13.543351+0000","last_active":"2023-10-06T03:34:13.548919+0000","last_peered":"2023-10-06T03:34:13.548919+0000","last_clean":"2023-10-06T03:34:13.548919+0000","last_became_active":"2023-10-06T03:34:13.542886+0000","last_became_peered":"2023-10-06T03:34:13.542886+0000","last_unstale":"2023-10-06T03:34:13.548919+0000","last_undegraded":"2023-10-06T03:34:13.548919+0000","last_fullsized":"2023-10-06T03:34:13.548919+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:25.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541689+0000","last_change":"2023-10-06T03:34:13.538209+0000","last_active":"2023-10-06T03:34:13.541689+0000","last_peered":"2023-10-06T03:34:13.541689+0000","last_clean":"2023-10-06T03:34:13.541689+0000","last_became_active":"2023-10-06T03:34:13.537849+0000","last_became_peered":"2023-10-06T03:34:13.537849+0000","last_unstale":"2023-10-06T03:34:13.541689+0000","last_undegraded":"2023-10-06T03:34:13.541689+0000","last_fullsized":"2023-10-06T03:34:13.541689+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:38.708108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550150+0000","last_change":"2023-10-06T03:34:13.547917+0000","last_active":"2023-10-06T03:34:13.550150+0000","last_peered":"2023-10-06T03:34:13.550150+0000","last_clean":"2023-10-06T03:34:13.550150+0000","last_became_active":"2023-10-06T03:34:13.546914+0000","last_became_peered":"2023-10-06T03:34:13.546914+0000","last_unstale":"2023-10-06T03:34:13.550150+0000","last_undegraded":"2023-10-06T03:34:13.550150+0000","last_fullsized":"2023-10-06T03:34:13.550150+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.785861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544849+0000","last_change":"2023-10-06T03:34:13.538592+0000","last_active":"2023-10-06T03:34:13.544849+0000","last_peered":"2023-10-06T03:34:13.544849+0000","last_clean":"2023-10-06T03:34:13.544849+0000","last_became_active":"2023-10-06T03:34:13.538156+0000","last_became_peered":"2023-10-06T03:34:13.538156+0000","last_unstale":"2023-10-06T03:34:13.544849+0000","last_undegraded":"2023-10-06T03:34:13.544849+0000","last_fullsized":"2023-10-06T03:34:13.544849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:10.434143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543863+0000","last_change":"2023-10-06T03:34:13.534736+0000","last_active":"2023-10-06T03:34:13.543863+0000","last_peered":"2023-10-06T03:34:13.543863+0000","last_clean":"2023-10-06T03:34:13.543863+0000","last_became_active":"2023-10-06T03:34:13.534493+0000","last_became_peered":"2023-10-06T03:34:13.534493+0000","last_unstale":"2023-10-06T03:34:13.543863+0000","last_undegraded":"2023-10-06T03:34:13.543863+0000","last_fullsized":"2023-10-06T03:34:13.543863+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:02.900878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.546850+0000","last_change":"2023-10-06T03:34:13.541709+0000","last_active":"2023-10-06T03:34:13.546850+0000","last_peered":"2023-10-06T03:34:13.546850+0000","last_clean":"2023-10-06T03:34:13.546850+0000","last_became_active":"2023-10-06T03:34:13.541399+0000","last_became_peered":"2023-10-06T03:34:13.541399+0000","last_unstale":"2023-10-06T03:34:13.546850+0000","last_undegraded":"2023-10-06T03:34:13.546850+0000","last_fullsized":"2023-10-06T03:34:13.546850+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:22.590165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557767+0000","last_change":"2023-10-06T03:34:13.545667+0000","last_active":"2023-10-06T03:34:13.557767+0000","last_peered":"2023-10-06T03:34:13.557767+0000","last_clean":"2023-10-06T03:34:13.557767+0000","last_became_active":"2023-10-06T03:34:13.545306+0000","last_became_peered":"2023-10-06T03:34:13.545306+0000","last_unstale":"2023-10-06T03:34:13.557767+0000","last_undegraded":"2023-10-06T03:34:13.557767+0000","last_fullsized":"2023-10-06T03:34:13.557767+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:38.708108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550618+0000","last_change":"2023-10-06T03:34:13.549359+0000","last_active":"2023-10-06T03:34:13.550618+0000","last_peered":"2023-10-06T03:34:13.550618+0000","last_clean":"2023-10-06T03:34:13.550618+0000","last_became_active":"2023-10-06T03:34:13.548692+0000","last_became_peered":"2023-10-06T03:34:13.548692+0000","last_unstale":"2023-10-06T03:34:13.550618+0000","last_undegraded":"2023-10-06T03:34:13.550618+0000","last_fullsized":"2023-10-06T03:34:13.550618+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:35.437314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.549190+0000","last_change":"2023-10-06T03:34:13.546572+0000","last_active":"2023-10-06T03:34:13.549190+0000","last_peered":"2023-10-06T03:34:13.549190+0000","last_clean":"2023-10-06T03:34:13.549190+0000","last_became_active":"2023-10-06T03:34:13.546313+0000","last_became_peered":"2023-10-06T03:34:13.546313+0000","last_unstale":"2023-10-06T03:34:13.549190+0000","last_undegraded":"2023-10-06T03:34:13.549190+0000","last_fullsized":"2023-10-06T03:34:13.549190+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:17.351579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560723+0000","last_change":"2023-10-06T03:34:13.558859+0000","last_active":"2023-10-06T03:34:13.560723+0000","last_peered":"2023-10-06T03:34:13.560723+0000","last_clean":"2023-10-06T03:34:13.560723+0000","last_became_active":"2023-10-06T03:34:13.558102+0000","last_became_peered":"2023-10-06T03:34:13.558102+0000","last_unstale":"2023-10-06T03:34:13.560723+0000","last_undegraded":"2023-10-06T03:34:13.560723+0000","last_fullsized":"2023-10-06T03:34:13.560723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:17.622122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559631+0000","last_change":"2023-10-06T03:34:13.548391+0000","last_active":"2023-10-06T03:34:13.559631+0000","last_peered":"2023-10-06T03:34:13.559631+0000","last_clean":"2023-10-06T03:34:13.559631+0000","last_became_active":"2023-10-06T03:34:13.548117+0000","last_became_peered":"2023-10-06T03:34:13.548117+0000","last_unstale":"2023-10-06T03:34:13.559631+0000","last_undegraded":"2023-10-06T03:34:13.559631+0000","last_fullsized":"2023-10-06T03:34:13.559631+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:17.351579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541952+0000","last_change":"2023-10-06T03:34:13.533970+0000","last_active":"2023-10-06T03:34:13.541952+0000","last_peered":"2023-10-06T03:34:13.541952+0000","last_clean":"2023-10-06T03:34:13.541952+0000","last_became_active":"2023-10-06T03:34:13.533643+0000","last_became_peered":"2023-10-06T03:34:13.533643+0000","last_unstale":"2023-10-06T03:34:13.541952+0000","last_undegraded":"2023-10-06T03:34:13.541952+0000","last_fullsized":"2023-10-06T03:34:13.541952+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:38.277766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559747+0000","last_change":"2023-10-06T03:34:13.548687+0000","last_active":"2023-10-06T03:34:13.559747+0000","last_peered":"2023-10-06T03:34:13.559747+0000","last_clean":"2023-10-06T03:34:13.559747+0000","last_became_active":"2023-10-06T03:34:13.547089+0000","last_became_peered":"2023-10-06T03:34:13.547089+0000","last_unstale":"2023-10-06T03:34:13.559747+0000","last_undegraded":"2023-10-06T03:34:13.559747+0000","last_fullsized":"2023-10-06T03:34:13.559747+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:35.437314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539582+0000","last_change":"2023-10-06T03:34:13.533974+0000","last_active":"2023-10-06T03:34:13.539582+0000","last_peered":"2023-10-06T03:34:13.539582+0000","last_clean":"2023-10-06T03:34:13.539582+0000","last_became_active":"2023-10-06T03:34:13.533686+0000","last_became_peered":"2023-10-06T03:34:13.533686+0000","last_unstale":"2023-10-06T03:34:13.539582+0000","last_undegraded":"2023-10-06T03:34:13.539582+0000","last_fullsized":"2023-10-06T03:34:13.539582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:15.558498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557756+0000","last_change":"2023-10-06T03:34:13.545608+0000","last_active":"2023-10-06T03:34:13.557756+0000","last_peered":"2023-10-06T03:34:13.557756+0000","last_clean":"2023-10-06T03:34:13.557756+0000","last_became_active":"2023-10-06T03:34:13.544797+0000","last_became_peered":"2023-10-06T03:34:13.544797+0000","last_unstale":"2023-10-06T03:34:13.557756+0000","last_undegraded":"2023-10-06T03:34:13.557756+0000","last_fullsized":"2023-10-06T03:34:13.557756+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:50.641614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557053+0000","last_change":"2023-10-06T03:34:13.555245+0000","last_active":"2023-10-06T03:34:13.557053+0000","last_peered":"2023-10-06T03:34:13.557053+0000","last_clean":"2023-10-06T03:34:13.557053+0000","last_became_active":"2023-10-06T03:34:13.555025+0000","last_became_peered":"2023-10-06T03:34:13.555025+0000","last_unstale":"2023-10-06T03:34:13.557053+0000","last_undegraded":"2023-10-06T03:34:13.557053+0000","last_fullsized":"2023-10-06T03:34:13.557053+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:25.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.552765+0000","last_change":"2023-10-06T03:34:13.549273+0000","last_active":"2023-10-06T03:34:13.552765+0000","last_peered":"2023-10-06T03:34:13.552765+0000","last_clean":"2023-10-06T03:34:13.552765+0000","last_became_active":"2023-10-06T03:34:13.548990+0000","last_became_peered":"2023-10-06T03:34:13.548990+0000","last_unstale":"2023-10-06T03:34:13.552765+0000","last_undegraded":"2023-10-06T03:34:13.552765+0000","last_fullsized":"2023-10-06T03:34:13.552765+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:17.622122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541045+0000","last_change":"2023-10-06T03:34:13.538176+0000","last_active":"2023-10-06T03:34:13.541045+0000","last_peered":"2023-10-06T03:34:13.541045+0000","last_clean":"2023-10-06T03:34:13.541045+0000","last_became_active":"2023-10-06T03:34:13.537927+0000","last_became_peered":"2023-10-06T03:34:13.537927+0000","last_unstale":"2023-10-06T03:34:13.541045+0000","last_undegraded":"2023-10-06T03:34:13.541045+0000","last_fullsized":"2023-10-06T03:34:13.541045+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:48.522789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544460+0000","last_change":"2023-10-06T03:34:13.539248+0000","last_active":"2023-10-06T03:34:13.544460+0000","last_peered":"2023-10-06T03:34:13.544460+0000","last_clean":"2023-10-06T03:34:13.544460+0000","last_became_active":"2023-10-06T03:34:13.539008+0000","last_became_peered":"2023-10-06T03:34:13.539008+0000","last_unstale":"2023-10-06T03:34:13.544460+0000","last_undegraded":"2023-10-06T03:34:13.544460+0000","last_fullsized":"2023-10-06T03:34:13.544460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:28.353727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541088+0000","last_change":"2023-10-06T03:34:13.537228+0000","last_active":"2023-10-06T03:34:13.541088+0000","last_peered":"2023-10-06T03:34:13.541088+0000","last_clean":"2023-10-06T03:34:13.541088+0000","last_became_active":"2023-10-06T03:34:13.536971+0000","last_became_peered":"2023-10-06T03:34:13.536971+0000","last_unstale":"2023-10-06T03:34:13.541088+0000","last_undegraded":"2023-10-06T03:34:13.541088+0000","last_fullsized":"2023-10-06T03:34:13.541088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:09.006005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557807+0000","last_change":"2023-10-06T03:34:13.547135+0000","last_active":"2023-10-06T03:34:13.557807+0000","last_peered":"2023-10-06T03:34:13.557807+0000","last_clean":"2023-10-06T03:34:13.557807+0000","last_became_active":"2023-10-06T03:34:13.546876+0000","last_became_peered":"2023-10-06T03:34:13.546876+0000","last_unstale":"2023-10-06T03:34:13.557807+0000","last_undegraded":"2023-10-06T03:34:13.557807+0000","last_fullsized":"2023-10-06T03:34:13.557807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:03.345245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539603+0000","last_change":"2023-10-06T03:34:13.535259+0000","last_active":"2023-10-06T03:34:13.539603+0000","last_peered":"2023-10-06T03:34:13.539603+0000","last_clean":"2023-10-06T03:34:13.539603+0000","last_became_active":"2023-10-06T03:34:13.535006+0000","last_became_peered":"2023-10-06T03:34:13.535006+0000","last_unstale":"2023-10-06T03:34:13.539603+0000","last_undegraded":"2023-10-06T03:34:13.539603+0000","last_fullsized":"2023-10-06T03:34:13.539603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:15.748952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.552796+0000","last_change":"2023-10-06T03:34:13.549634+0000","last_active":"2023-10-06T03:34:13.552796+0000","last_peered":"2023-10-06T03:34:13.552796+0000","last_clean":"2023-10-06T03:34:13.552796+0000","last_became_active":"2023-10-06T03:34:13.549343+0000","last_became_peered":"2023-10-06T03:34:13.549343+0000","last_unstale":"2023-10-06T03:34:13.552796+0000","last_undegraded":"2023-10-06T03:34:13.552796+0000","last_fullsized":"2023-10-06T03:34:13.552796+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:50.641614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544186+0000","last_change":"2023-10-06T03:34:13.537144+0000","last_active":"2023-10-06T03:34:13.544186+0000","last_peered":"2023-10-06T03:34:13.544186+0000","last_clean":"2023-10-06T03:34:13.544186+0000","last_became_active":"2023-10-06T03:34:13.536727+0000","last_became_peered":"2023-10-06T03:34:13.536727+0000","last_unstale":"2023-10-06T03:34:13.544186+0000","last_undegraded":"2023-10-06T03:34:13.544186+0000","last_fullsized":"2023-10-06T03:34:13.544186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:09.006005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.537520+0000","last_change":"2023-10-06T03:34:13.529906+0000","last_active":"2023-10-06T03:34:13.537520+0000","last_peered":"2023-10-06T03:34:13.537520+0000","last_clean":"2023-10-06T03:34:13.537520+0000","last_became_active":"2023-10-06T03:34:13.529637+0000","last_became_peered":"2023-10-06T03:34:13.529637+0000","last_unstale":"2023-10-06T03:34:13.537520+0000","last_undegraded":"2023-10-06T03:34:13.537520+0000","last_fullsized":"2023-10-06T03:34:13.537520+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:10.434143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.558046+0000","last_change":"2023-10-06T03:34:13.555751+0000","last_active":"2023-10-06T03:34:13.558046+0000","last_peered":"2023-10-06T03:34:13.558046+0000","last_clean":"2023-10-06T03:34:13.558046+0000","last_became_active":"2023-10-06T03:34:13.555469+0000","last_became_peered":"2023-10-06T03:34:13.555469+0000","last_unstale":"2023-10-06T03:34:13.558046+0000","last_undegraded":"2023-10-06T03:34:13.558046+0000","last_fullsized":"2023-10-06T03:34:13.558046+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:53.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540269+0000","last_change":"2023-10-06T03:34:13.534759+0000","last_active":"2023-10-06T03:34:13.540269+0000","last_peered":"2023-10-06T03:34:13.540269+0000","last_clean":"2023-10-06T03:34:13.540269+0000","last_became_active":"2023-10-06T03:34:13.534458+0000","last_became_peered":"2023-10-06T03:34:13.534458+0000","last_unstale":"2023-10-06T03:34:13.540269+0000","last_undegraded":"2023-10-06T03:34:13.540269+0000","last_fullsized":"2023-10-06T03:34:13.540269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:55.396883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559551+0000","last_change":"2023-10-06T03:34:13.547737+0000","last_active":"2023-10-06T03:34:13.559551+0000","last_peered":"2023-10-06T03:34:13.559551+0000","last_clean":"2023-10-06T03:34:13.559551+0000","last_became_active":"2023-10-06T03:34:13.547463+0000","last_became_peered":"2023-10-06T03:34:13.547463+0000","last_unstale":"2023-10-06T03:34:13.559551+0000","last_undegraded":"2023-10-06T03:34:13.559551+0000","last_fullsized":"2023-10-06T03:34:13.559551+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:30.492483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540458+0000","last_change":"2023-10-06T03:34:13.536475+0000","last_active":"2023-10-06T03:34:13.540458+0000","last_peered":"2023-10-06T03:34:13.540458+0000","last_clean":"2023-10-06T03:34:13.540458+0000","last_became_active":"2023-10-06T03:34:13.536202+0000","last_became_peered":"2023-10-06T03:34:13.536202+0000","last_unstale":"2023-10-06T03:34:13.540458+0000","last_undegraded":"2023-10-06T03:34:13.540458+0000","last_fullsized":"2023-10-06T03:34:13.540458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:02.900878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.537621+0000","last_change":"2023-10-06T03:34:13.530628+0000","last_active":"2023-10-06T03:34:13.537621+0000","last_peered":"2023-10-06T03:34:13.537621+0000","last_clean":"2023-10-06T03:34:13.537621+0000","last_became_active":"2023-10-06T03:34:13.530318+0000","last_became_peered":"2023-10-06T03:34:13.530318+0000","last_unstale":"2023-10-06T03:34:13.537621+0000","last_undegraded":"2023-10-06T03:34:13.537621+0000","last_fullsized":"2023-10-06T03:34:13.537621+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:18.351278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550254+0000","last_change":"2023-10-06T03:34:13.547733+0000","last_active":"2023-10-06T03:34:13.550254+0000","last_peered":"2023-10-06T03:34:13.550254+0000","last_clean":"2023-10-06T03:34:13.550254+0000","last_became_active":"2023-10-06T03:34:13.547402+0000","last_became_peered":"2023-10-06T03:34:13.547402+0000","last_unstale":"2023-10-06T03:34:13.550254+0000","last_undegraded":"2023-10-06T03:34:13.550254+0000","last_fullsized":"2023-10-06T03:34:13.550254+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:05.808082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.547236+0000","last_change":"2023-10-06T03:34:13.540168+0000","last_active":"2023-10-06T03:34:13.547236+0000","last_peered":"2023-10-06T03:34:13.547236+0000","last_clean":"2023-10-06T03:34:13.547236+0000","last_became_active":"2023-10-06T03:34:13.539910+0000","last_became_peered":"2023-10-06T03:34:13.539910+0000","last_unstale":"2023-10-06T03:34:13.547236+0000","last_undegraded":"2023-10-06T03:34:13.547236+0000","last_fullsized":"2023-10-06T03:34:13.547236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:04.506250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557194+0000","last_change":"2023-10-06T03:34:13.556269+0000","last_active":"2023-10-06T03:34:13.557194+0000","last_peered":"2023-10-06T03:34:13.557194+0000","last_clean":"2023-10-06T03:34:13.557194+0000","last_became_active":"2023-10-06T03:34:13.555974+0000","last_became_peered":"2023-10-06T03:34:13.555974+0000","last_unstale":"2023-10-06T03:34:13.557194+0000","last_undegraded":"2023-10-06T03:34:13.557194+0000","last_fullsized":"2023-10-06T03:34:13.557194+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:03.345245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"31'4","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544660+0000","last_change":"2023-10-06T03:34:12.523661+0000","last_active":"2023-10-06T03:34:13.544660+0000","last_peered":"2023-10-06T03:34:13.544660+0000","last_clean":"2023-10-06T03:34:13.544660+0000","last_became_active":"2023-10-06T03:34:12.523343+0000","last_became_peered":"2023-10-06T03:34:12.523343+0000","last_unstale":"2023-10-06T03:34:13.544660+0000","last_undegraded":"2023-10-06T03:34:13.544660+0000","last_fullsized":"2023-10-06T03:34:13.544660+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:23.796615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.549209+0000","last_change":"2023-10-06T03:34:13.546825+0000","last_active":"2023-10-06T03:34:13.549209+0000","last_peered":"2023-10-06T03:34:13.549209+0000","last_clean":"2023-10-06T03:34:13.549209+0000","last_became_active":"2023-10-06T03:34:13.546561+0000","last_became_peered":"2023-10-06T03:34:13.546561+0000","last_unstale":"2023-10-06T03:34:13.549209+0000","last_undegraded":"2023-10-06T03:34:13.549209+0000","last_fullsized":"2023-10-06T03:34:13.549209+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:10.048229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540711+0000","last_change":"2023-10-06T03:34:12.519828+0000","last_active":"2023-10-06T03:34:13.540711+0000","last_peered":"2023-10-06T03:34:13.540711+0000","last_clean":"2023-10-06T03:34:13.540711+0000","last_became_active":"2023-10-06T03:34:12.519596+0000","last_became_peered":"2023-10-06T03:34:12.519596+0000","last_unstale":"2023-10-06T03:34:13.540711+0000","last_undegraded":"2023-10-06T03:34:13.540711+0000","last_fullsized":"2023-10-06T03:34:13.540711+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:01.889807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539250+0000","last_change":"2023-10-06T03:34:13.533928+0000","last_active":"2023-10-06T03:34:13.539250+0000","last_peered":"2023-10-06T03:34:13.539250+0000","last_clean":"2023-10-06T03:34:13.539250+0000","last_became_active":"2023-10-06T03:34:13.533642+0000","last_became_peered":"2023-10-06T03:34:13.533642+0000","last_unstale":"2023-10-06T03:34:13.539250+0000","last_undegraded":"2023-10-06T03:34:13.539250+0000","last_fullsized":"2023-10-06T03:34:13.539250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:25.840403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.545261+0000","last_change":"2023-10-06T03:34:12.526656+0000","last_active":"2023-10-06T03:34:13.545261+0000","last_peered":"2023-10-06T03:34:13.545261+0000","last_clean":"2023-10-06T03:34:13.545261+0000","last_became_active":"2023-10-06T03:34:12.525776+0000","last_became_peered":"2023-10-06T03:34:12.525776+0000","last_unstale":"2023-10-06T03:34:13.545261+0000","last_undegraded":"2023-10-06T03:34:13.545261+0000","last_fullsized":"2023-10-06T03:34:13.545261+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:27.342656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.548898+0000","last_change":"2023-10-06T03:34:13.543146+0000","last_active":"2023-10-06T03:34:13.548898+0000","last_peered":"2023-10-06T03:34:13.548898+0000","last_clean":"2023-10-06T03:34:13.548898+0000","last_became_active":"2023-10-06T03:34:13.542460+0000","last_became_peered":"2023-10-06T03:34:13.542460+0000","last_unstale":"2023-10-06T03:34:13.548898+0000","last_undegraded":"2023-10-06T03:34:13.548898+0000","last_fullsized":"2023-10-06T03:34:13.548898+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.785861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.552296+0000","last_change":"2023-10-06T03:34:12.524322+0000","last_active":"2023-10-06T03:34:13.552296+0000","last_peered":"2023-10-06T03:34:13.552296+0000","last_clean":"2023-10-06T03:34:13.552296+0000","last_became_active":"2023-10-06T03:34:12.524037+0000","last_became_peered":"2023-10-06T03:34:12.524037+0000","last_unstale":"2023-10-06T03:34:13.552296+0000","last_undegraded":"2023-10-06T03:34:13.552296+0000","last_fullsized":"2023-10-06T03:34:13.552296+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:59.774790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543775+0000","last_change":"2023-10-06T03:34:13.539137+0000","last_active":"2023-10-06T03:34:13.543775+0000","last_peered":"2023-10-06T03:34:13.543775+0000","last_clean":"2023-10-06T03:34:13.543775+0000","last_became_active":"2023-10-06T03:34:13.538833+0000","last_became_peered":"2023-10-06T03:34:13.538833+0000","last_unstale":"2023-10-06T03:34:13.543775+0000","last_undegraded":"2023-10-06T03:34:13.543775+0000","last_fullsized":"2023-10-06T03:34:13.543775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:27.486440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559924+0000","last_change":"2023-10-06T03:34:12.527023+0000","last_active":"2023-10-06T03:34:13.559924+0000","last_peered":"2023-10-06T03:34:13.559924+0000","last_clean":"2023-10-06T03:34:13.559924+0000","last_became_active":"2023-10-06T03:34:12.526443+0000","last_became_peered":"2023-10-06T03:34:12.526443+0000","last_unstale":"2023-10-06T03:34:13.559924+0000","last_undegraded":"2023-10-06T03:34:13.559924+0000","last_fullsized":"2023-10-06T03:34:13.559924+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:54.385812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557742+0000","last_change":"2023-10-06T03:34:13.544656+0000","last_active":"2023-10-06T03:34:13.557742+0000","last_peered":"2023-10-06T03:34:13.557742+0000","last_clean":"2023-10-06T03:34:13.557742+0000","last_became_active":"2023-10-06T03:34:13.544249+0000","last_became_peered":"2023-10-06T03:34:13.544249+0000","last_unstale":"2023-10-06T03:34:13.557742+0000","last_undegraded":"2023-10-06T03:34:13.557742+0000","last_fullsized":"2023-10-06T03:34:13.557742+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:01.766184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.552592+0000","last_change":"2023-10-06T03:34:12.532851+0000","last_active":"2023-10-06T03:34:13.552592+0000","last_peered":"2023-10-06T03:34:13.552592+0000","last_clean":"2023-10-06T03:34:13.552592+0000","last_became_active":"2023-10-06T03:34:12.532517+0000","last_became_peered":"2023-10-06T03:34:12.532517+0000","last_unstale":"2023-10-06T03:34:13.552592+0000","last_undegraded":"2023-10-06T03:34:13.552592+0000","last_fullsized":"2023-10-06T03:34:13.552592+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:04.840518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.549683+0000","last_change":"2023-10-06T03:34:13.546209+0000","last_active":"2023-10-06T03:34:13.549683+0000","last_peered":"2023-10-06T03:34:13.549683+0000","last_clean":"2023-10-06T03:34:13.549683+0000","last_became_active":"2023-10-06T03:34:13.545940+0000","last_became_peered":"2023-10-06T03:34:13.545940+0000","last_unstale":"2023-10-06T03:34:13.549683+0000","last_undegraded":"2023-10-06T03:34:13.549683+0000","last_fullsized":"2023-10-06T03:34:13.549683+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:44.937279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.528389+0000","last_change":"2023-10-06T03:34:12.527549+0000","last_active":"2023-10-06T03:34:13.528389+0000","last_peered":"2023-10-06T03:34:13.528389+0000","last_clean":"2023-10-06T03:34:13.528389+0000","last_became_active":"2023-10-06T03:34:12.527236+0000","last_became_peered":"2023-10-06T03:34:12.527236+0000","last_unstale":"2023-10-06T03:34:13.528389+0000","last_undegraded":"2023-10-06T03:34:13.528389+0000","last_fullsized":"2023-10-06T03:34:13.528389+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:43.578807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.542212+0000","last_change":"2023-10-06T03:34:13.535907+0000","last_active":"2023-10-06T03:34:13.542212+0000","last_peered":"2023-10-06T03:34:13.542212+0000","last_clean":"2023-10-06T03:34:13.542212+0000","last_became_active":"2023-10-06T03:34:13.535054+0000","last_became_peered":"2023-10-06T03:34:13.535054+0000","last_unstale":"2023-10-06T03:34:13.542212+0000","last_undegraded":"2023-10-06T03:34:13.542212+0000","last_fullsized":"2023-10-06T03:34:13.542212+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:15.558498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.523244+0000","last_change":"2023-10-06T03:34:12.516603+0000","last_active":"2023-10-06T03:34:13.523244+0000","last_peered":"2023-10-06T03:34:13.523244+0000","last_clean":"2023-10-06T03:34:13.523244+0000","last_became_active":"2023-10-06T03:34:12.516217+0000","last_became_peered":"2023-10-06T03:34:12.516217+0000","last_unstale":"2023-10-06T03:34:13.523244+0000","last_undegraded":"2023-10-06T03:34:13.523244+0000","last_fullsized":"2023-10-06T03:34:13.523244+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:16.340508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544831+0000","last_change":"2023-10-06T03:34:13.538400+0000","last_active":"2023-10-06T03:34:13.544831+0000","last_peered":"2023-10-06T03:34:13.544831+0000","last_clean":"2023-10-06T03:34:13.544831+0000","last_became_active":"2023-10-06T03:34:13.537683+0000","last_became_peered":"2023-10-06T03:34:13.537683+0000","last_unstale":"2023-10-06T03:34:13.544831+0000","last_undegraded":"2023-10-06T03:34:13.544831+0000","last_fullsized":"2023-10-06T03:34:13.544831+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:06.270734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.526789+0000","last_change":"2023-10-06T03:34:12.523775+0000","last_active":"2023-10-06T03:34:13.526789+0000","last_peered":"2023-10-06T03:34:13.526789+0000","last_clean":"2023-10-06T03:34:13.526789+0000","last_became_active":"2023-10-06T03:34:12.523512+0000","last_became_peered":"2023-10-06T03:34:12.523512+0000","last_unstale":"2023-10-06T03:34:13.526789+0000","last_undegraded":"2023-10-06T03:34:13.526789+0000","last_fullsized":"2023-10-06T03:34:13.526789+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:21.579094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543961+0000","last_change":"2023-10-06T03:34:13.541278+0000","last_active":"2023-10-06T03:34:13.543961+0000","last_peered":"2023-10-06T03:34:13.543961+0000","last_clean":"2023-10-06T03:34:13.543961+0000","last_became_active":"2023-10-06T03:34:13.541008+0000","last_became_peered":"2023-10-06T03:34:13.541008+0000","last_unstale":"2023-10-06T03:34:13.543961+0000","last_undegraded":"2023-10-06T03:34:13.543961+0000","last_fullsized":"2023-10-06T03:34:13.543961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:38.708108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539800+0000","last_change":"2023-10-06T03:34:12.528843+0000","last_active":"2023-10-06T03:34:13.539800+0000","last_peered":"2023-10-06T03:34:13.539800+0000","last_clean":"2023-10-06T03:34:13.539800+0000","last_became_active":"2023-10-06T03:34:12.528575+0000","last_became_peered":"2023-10-06T03:34:12.528575+0000","last_unstale":"2023-10-06T03:34:13.539800+0000","last_undegraded":"2023-10-06T03:34:13.539800+0000","last_fullsized":"2023-10-06T03:34:13.539800+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:07.994934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540805+0000","last_change":"2023-10-06T03:34:13.535849+0000","last_active":"2023-10-06T03:34:13.540805+0000","last_peered":"2023-10-06T03:34:13.540805+0000","last_clean":"2023-10-06T03:34:13.540805+0000","last_became_active":"2023-10-06T03:34:13.535582+0000","last_became_peered":"2023-10-06T03:34:13.535582+0000","last_unstale":"2023-10-06T03:34:13.540805+0000","last_undegraded":"2023-10-06T03:34:13.540805+0000","last_fullsized":"2023-10-06T03:34:13.540805+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:53.934064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"31'3","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560103+0000","last_change":"2023-10-06T03:34:12.529757+0000","last_active":"2023-10-06T03:34:13.560103+0000","last_peered":"2023-10-06T03:34:13.560103+0000","last_clean":"2023-10-06T03:34:13.560103+0000","last_became_active":"2023-10-06T03:34:12.529371+0000","last_became_peered":"2023-10-06T03:34:12.529371+0000","last_unstale":"2023-10-06T03:34:13.560103+0000","last_undegraded":"2023-10-06T03:34:13.560103+0000","last_fullsized":"2023-10-06T03:34:13.560103+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:17.340207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540076+0000","last_change":"2023-10-06T03:34:13.534519+0000","last_active":"2023-10-06T03:34:13.540076+0000","last_peered":"2023-10-06T03:34:13.540076+0000","last_clean":"2023-10-06T03:34:13.540076+0000","last_became_active":"2023-10-06T03:34:13.534278+0000","last_became_peered":"2023-10-06T03:34:13.534278+0000","last_unstale":"2023-10-06T03:34:13.540076+0000","last_undegraded":"2023-10-06T03:34:13.540076+0000","last_fullsized":"2023-10-06T03:34:13.540076+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:51.583498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"31'6","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.561812+0000","last_change":"2023-10-06T03:34:12.528656+0000","last_active":"2023-10-06T03:34:13.561812+0000","last_peered":"2023-10-06T03:34:13.561812+0000","last_clean":"2023-10-06T03:34:13.561812+0000","last_became_active":"2023-10-06T03:34:12.528361+0000","last_became_peered":"2023-10-06T03:34:12.528361+0000","last_unstale":"2023-10-06T03:34:13.561812+0000","last_undegraded":"2023-10-06T03:34:13.561812+0000","last_fullsized":"2023-10-06T03:34:13.561812+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:31.070647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557081+0000","last_change":"2023-10-06T03:34:13.555313+0000","last_active":"2023-10-06T03:34:13.557081+0000","last_peered":"2023-10-06T03:34:13.557081+0000","last_clean":"2023-10-06T03:34:13.557081+0000","last_became_active":"2023-10-06T03:34:13.555027+0000","last_became_peered":"2023-10-06T03:34:13.555027+0000","last_unstale":"2023-10-06T03:34:13.557081+0000","last_undegraded":"2023-10-06T03:34:13.557081+0000","last_fullsized":"2023-10-06T03:34:13.557081+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:01.442206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550514+0000","last_change":"2023-10-06T03:34:12.523998+0000","last_active":"2023-10-06T03:34:13.550514+0000","last_peered":"2023-10-06T03:34:13.550514+0000","last_clean":"2023-10-06T03:34:13.550514+0000","last_became_active":"2023-10-06T03:34:12.523688+0000","last_became_peered":"2023-10-06T03:34:12.523688+0000","last_unstale":"2023-10-06T03:34:13.550514+0000","last_undegraded":"2023-10-06T03:34:13.550514+0000","last_fullsized":"2023-10-06T03:34:13.550514+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:52.611432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.542751+0000","last_change":"2023-10-06T03:34:13.540598+0000","last_active":"2023-10-06T03:34:13.542751+0000","last_peered":"2023-10-06T03:34:13.542751+0000","last_clean":"2023-10-06T03:34:13.542751+0000","last_became_active":"2023-10-06T03:34:13.540350+0000","last_became_peered":"2023-10-06T03:34:13.540350+0000","last_unstale":"2023-10-06T03:34:13.542751+0000","last_undegraded":"2023-10-06T03:34:13.542751+0000","last_fullsized":"2023-10-06T03:34:13.542751+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:03.345245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.527766+0000","last_change":"2023-10-06T03:34:12.530728+0000","last_active":"2023-10-06T03:34:13.527766+0000","last_peered":"2023-10-06T03:34:13.527766+0000","last_clean":"2023-10-06T03:34:13.527766+0000","last_became_active":"2023-10-06T03:34:12.530458+0000","last_became_peered":"2023-10-06T03:34:12.530458+0000","last_unstale":"2023-10-06T03:34:13.527766+0000","last_undegraded":"2023-10-06T03:34:13.527766+0000","last_fullsized":"2023-10-06T03:34:13.527766+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:09.423072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550408+0000","last_change":"2023-10-06T03:34:13.547951+0000","last_active":"2023-10-06T03:34:13.550408+0000","last_peered":"2023-10-06T03:34:13.550408+0000","last_clean":"2023-10-06T03:34:13.550408+0000","last_became_active":"2023-10-06T03:34:13.547567+0000","last_became_peered":"2023-10-06T03:34:13.547567+0000","last_unstale":"2023-10-06T03:34:13.550408+0000","last_undegraded":"2023-10-06T03:34:13.550408+0000","last_fullsized":"2023-10-06T03:34:13.550408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:01.766184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.521120+0000","last_change":"2023-10-06T03:34:12.517112+0000","last_active":"2023-10-06T03:34:13.521120+0000","last_peered":"2023-10-06T03:34:13.521120+0000","last_clean":"2023-10-06T03:34:13.521120+0000","last_became_active":"2023-10-06T03:34:12.516831+0000","last_became_peered":"2023-10-06T03:34:12.516831+0000","last_unstale":"2023-10-06T03:34:13.521120+0000","last_undegraded":"2023-10-06T03:34:13.521120+0000","last_fullsized":"2023-10-06T03:34:13.521120+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:48.752619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.542193+0000","last_change":"2023-10-06T03:34:13.535453+0000","last_active":"2023-10-06T03:34:13.542193+0000","last_peered":"2023-10-06T03:34:13.542193+0000","last_clean":"2023-10-06T03:34:13.542193+0000","last_became_active":"2023-10-06T03:34:13.534677+0000","last_became_peered":"2023-10-06T03:34:13.534677+0000","last_unstale":"2023-10-06T03:34:13.542193+0000","last_undegraded":"2023-10-06T03:34:13.542193+0000","last_fullsized":"2023-10-06T03:34:13.542193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:19.240735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"31'8","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.556402+0000","last_change":"2023-10-06T03:34:12.532903+0000","last_active":"2023-10-06T03:34:13.556402+0000","last_peered":"2023-10-06T03:34:13.556402+0000","last_clean":"2023-10-06T03:34:13.556402+0000","last_became_active":"2023-10-06T03:34:12.532576+0000","last_became_peered":"2023-10-06T03:34:12.532576+0000","last_unstale":"2023-10-06T03:34:13.556402+0000","last_undegraded":"2023-10-06T03:34:13.556402+0000","last_fullsized":"2023-10-06T03:34:13.556402+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539800+0000","last_change":"2023-10-06T03:34:13.531821+0000","last_active":"2023-10-06T03:34:13.539800+0000","last_peered":"2023-10-06T03:34:13.539800+0000","last_clean":"2023-10-06T03:34:13.539800+0000","last_became_active":"2023-10-06T03:34:13.531527+0000","last_became_peered":"2023-10-06T03:34:13.531527+0000","last_unstale":"2023-10-06T03:34:13.539800+0000","last_undegraded":"2023-10-06T03:34:13.539800+0000","last_fullsized":"2023-10-06T03:34:13.539800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:05.808082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560016+0000","last_change":"2023-10-06T03:34:12.532478+0000","last_active":"2023-10-06T03:34:13.560016+0000","last_peered":"2023-10-06T03:34:13.560016+0000","last_clean":"2023-10-06T03:34:13.560016+0000","last_became_active":"2023-10-06T03:34:12.532204+0000","last_became_peered":"2023-10-06T03:34:12.532204+0000","last_unstale":"2023-10-06T03:34:13.560016+0000","last_undegraded":"2023-10-06T03:34:13.560016+0000","last_fullsized":"2023-10-06T03:34:13.560016+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560141+0000","last_change":"2023-10-06T03:34:13.556710+0000","last_active":"2023-10-06T03:34:13.560141+0000","last_peered":"2023-10-06T03:34:13.560141+0000","last_clean":"2023-10-06T03:34:13.560141+0000","last_became_active":"2023-10-06T03:34:13.554487+0000","last_became_peered":"2023-10-06T03:34:13.554487+0000","last_unstale":"2023-10-06T03:34:13.560141+0000","last_undegraded":"2023-10-06T03:34:13.560141+0000","last_fullsized":"2023-10-06T03:34:13.560141+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:25.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560376+0000","last_change":"2023-10-06T03:34:12.527219+0000","last_active":"2023-10-06T03:34:13.560376+0000","last_peered":"2023-10-06T03:34:13.560376+0000","last_clean":"2023-10-06T03:34:13.560376+0000","last_became_active":"2023-10-06T03:34:12.526985+0000","last_became_peered":"2023-10-06T03:34:12.526985+0000","last_unstale":"2023-10-06T03:34:13.560376+0000","last_undegraded":"2023-10-06T03:34:13.560376+0000","last_fullsized":"2023-10-06T03:34:13.560376+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:29.481412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543498+0000","last_change":"2023-10-06T03:34:13.537130+0000","last_active":"2023-10-06T03:34:13.543498+0000","last_peered":"2023-10-06T03:34:13.543498+0000","last_clean":"2023-10-06T03:34:13.543498+0000","last_became_active":"2023-10-06T03:34:13.536871+0000","last_became_peered":"2023-10-06T03:34:13.536871+0000","last_unstale":"2023-10-06T03:34:13.543498+0000","last_undegraded":"2023-10-06T03:34:13.543498+0000","last_fullsized":"2023-10-06T03:34:13.543498+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:21.602441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.553427+0000","last_change":"2023-10-06T03:34:12.524489+0000","last_active":"2023-10-06T03:34:13.553427+0000","last_peered":"2023-10-06T03:34:13.553427+0000","last_clean":"2023-10-06T03:34:13.553427+0000","last_became_active":"2023-10-06T03:34:12.524226+0000","last_became_peered":"2023-10-06T03:34:12.524226+0000","last_unstale":"2023-10-06T03:34:13.553427+0000","last_undegraded":"2023-10-06T03:34:13.553427+0000","last_fullsized":"2023-10-06T03:34:13.553427+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:37.934361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.536949+0000","last_change":"2023-10-06T03:34:13.531913+0000","last_active":"2023-10-06T03:34:13.536949+0000","last_peered":"2023-10-06T03:34:13.536949+0000","last_clean":"2023-10-06T03:34:13.536949+0000","last_became_active":"2023-10-06T03:34:13.531611+0000","last_became_peered":"2023-10-06T03:34:13.531611+0000","last_unstale":"2023-10-06T03:34:13.536949+0000","last_undegraded":"2023-10-06T03:34:13.536949+0000","last_fullsized":"2023-10-06T03:34:13.536949+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:44.589878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.528863+0000","last_change":"2023-10-06T03:34:12.532410+0000","last_active":"2023-10-06T03:34:13.528863+0000","last_peered":"2023-10-06T03:34:13.528863+0000","last_clean":"2023-10-06T03:34:13.528863+0000","last_became_active":"2023-10-06T03:34:12.532154+0000","last_became_peered":"2023-10-06T03:34:12.532154+0000","last_unstale":"2023-10-06T03:34:13.528863+0000","last_undegraded":"2023-10-06T03:34:13.528863+0000","last_fullsized":"2023-10-06T03:34:13.528863+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:05.259663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.549613+0000","last_change":"2023-10-06T03:34:13.546456+0000","last_active":"2023-10-06T03:34:13.549613+0000","last_peered":"2023-10-06T03:34:13.549613+0000","last_clean":"2023-10-06T03:34:13.549613+0000","last_became_active":"2023-10-06T03:34:13.546118+0000","last_became_peered":"2023-10-06T03:34:13.546118+0000","last_unstale":"2023-10-06T03:34:13.549613+0000","last_undegraded":"2023-10-06T03:34:13.549613+0000","last_fullsized":"2023-10-06T03:34:13.549613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:30.492483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550619+0000","last_change":"2023-10-06T03:34:12.519345+0000","last_active":"2023-10-06T03:34:13.550619+0000","last_peered":"2023-10-06T03:34:13.550619+0000","last_clean":"2023-10-06T03:34:13.550619+0000","last_became_active":"2023-10-06T03:34:12.519053+0000","last_became_peered":"2023-10-06T03:34:12.519053+0000","last_unstale":"2023-10-06T03:34:13.550619+0000","last_undegraded":"2023-10-06T03:34:13.550619+0000","last_fullsized":"2023-10-06T03:34:13.550619+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:34.904232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540779+0000","last_change":"2023-10-06T03:34:13.535161+0000","last_active":"2023-10-06T03:34:13.540779+0000","last_peered":"2023-10-06T03:34:13.540779+0000","last_clean":"2023-10-06T03:34:13.540779+0000","last_became_active":"2023-10-06T03:34:13.534909+0000","last_became_peered":"2023-10-06T03:34:13.534909+0000","last_unstale":"2023-10-06T03:34:13.540779+0000","last_undegraded":"2023-10-06T03:34:13.540779+0000","last_fullsized":"2023-10-06T03:34:13.540779+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:48.475963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.524923+0000","last_change":"2023-10-06T03:34:12.522297+0000","last_active":"2023-10-06T03:34:13.524923+0000","last_peered":"2023-10-06T03:34:13.524923+0000","last_clean":"2023-10-06T03:34:13.524923+0000","last_became_active":"2023-10-06T03:34:12.522006+0000","last_became_peered":"2023-10-06T03:34:12.522006+0000","last_unstale":"2023-10-06T03:34:13.524923+0000","last_undegraded":"2023-10-06T03:34:13.524923+0000","last_fullsized":"2023-10-06T03:34:13.524923+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:39.716255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559638+0000","last_change":"2023-10-06T03:34:13.547565+0000","last_active":"2023-10-06T03:34:13.559638+0000","last_peered":"2023-10-06T03:34:13.559638+0000","last_clean":"2023-10-06T03:34:13.559638+0000","last_became_active":"2023-10-06T03:34:13.547290+0000","last_became_peered":"2023-10-06T03:34:13.547290+0000","last_unstale":"2023-10-06T03:34:13.559638+0000","last_undegraded":"2023-10-06T03:34:13.559638+0000","last_fullsized":"2023-10-06T03:34:13.559638+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:05.808082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.531326+0000","last_change":"2023-10-06T03:34:12.527419+0000","last_active":"2023-10-06T03:34:13.531326+0000","last_peered":"2023-10-06T03:34:13.531326+0000","last_clean":"2023-10-06T03:34:13.531326+0000","last_became_active":"2023-10-06T03:34:12.527141+0000","last_became_peered":"2023-10-06T03:34:12.527141+0000","last_unstale":"2023-10-06T03:34:13.531326+0000","last_undegraded":"2023-10-06T03:34:13.531326+0000","last_fullsized":"2023-10-06T03:34:13.531326+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:17.340207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.542230+0000","last_change":"2023-10-06T03:34:13.536079+0000","last_active":"2023-10-06T03:34:13.542230+0000","last_peered":"2023-10-06T03:34:13.542230+0000","last_clean":"2023-10-06T03:34:13.542230+0000","last_became_active":"2023-10-06T03:34:13.535653+0000","last_became_peered":"2023-10-06T03:34:13.535653+0000","last_unstale":"2023-10-06T03:34:13.542230+0000","last_undegraded":"2023-10-06T03:34:13.542230+0000","last_fullsized":"2023-10-06T03:34:13.542230+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:25.840403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.525414+0000","last_change":"2023-10-06T03:34:12.521014+0000","last_active":"2023-10-06T03:34:13.525414+0000","last_peered":"2023-10-06T03:34:13.525414+0000","last_clean":"2023-10-06T03:34:13.525414+0000","last_became_active":"2023-10-06T03:34:12.520088+0000","last_became_peered":"2023-10-06T03:34:12.520088+0000","last_unstale":"2023-10-06T03:34:13.525414+0000","last_undegraded":"2023-10-06T03:34:13.525414+0000","last_fullsized":"2023-10-06T03:34:13.525414+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:34.651158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539620+0000","last_change":"2023-10-06T03:34:13.535691+0000","last_active":"2023-10-06T03:34:13.539620+0000","last_peered":"2023-10-06T03:34:13.539620+0000","last_clean":"2023-10-06T03:34:13.539620+0000","last_became_active":"2023-10-06T03:34:13.535365+0000","last_became_peered":"2023-10-06T03:34:13.535365+0000","last_unstale":"2023-10-06T03:34:13.539620+0000","last_undegraded":"2023-10-06T03:34:13.539620+0000","last_fullsized":"2023-10-06T03:34:13.539620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:08.739478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.523084+0000","last_change":"2023-10-06T03:34:12.525259+0000","last_active":"2023-10-06T03:34:13.523084+0000","last_peered":"2023-10-06T03:34:13.523084+0000","last_clean":"2023-10-06T03:34:13.523084+0000","last_became_active":"2023-10-06T03:34:12.524902+0000","last_became_peered":"2023-10-06T03:34:12.524902+0000","last_unstale":"2023-10-06T03:34:13.523084+0000","last_undegraded":"2023-10-06T03:34:13.523084+0000","last_fullsized":"2023-10-06T03:34:13.523084+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:37.697037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.524780+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:34:13.524780+0000","last_peered":"2023-10-06T03:34:13.524780+0000","last_clean":"2023-10-06T03:34:13.524780+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:34:13.524780+0000","last_undegraded":"2023-10-06T03:34:13.524780+0000","last_fullsized":"2023-10-06T03:34:13.524780+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.545347+0000","last_change":"2023-10-06T03:34:13.543841+0000","last_active":"2023-10-06T03:34:13.545347+0000","last_peered":"2023-10-06T03:34:13.545347+0000","last_clean":"2023-10-06T03:34:13.545347+0000","last_became_active":"2023-10-06T03:34:13.543580+0000","last_became_peered":"2023-10-06T03:34:13.543580+0000","last_unstale":"2023-10-06T03:34:13.545347+0000","last_undegraded":"2023-10-06T03:34:13.545347+0000","last_fullsized":"2023-10-06T03:34:13.545347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:53.934064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.524324+0000","last_change":"2023-10-06T03:34:12.525282+0000","last_active":"2023-10-06T03:34:13.524324+0000","last_peered":"2023-10-06T03:34:13.524324+0000","last_clean":"2023-10-06T03:34:13.524324+0000","last_became_active":"2023-10-06T03:34:12.525020+0000","last_became_peered":"2023-10-06T03:34:12.525020+0000","last_unstale":"2023-10-06T03:34:13.524324+0000","last_undegraded":"2023-10-06T03:34:13.524324+0000","last_fullsized":"2023-10-06T03:34:13.524324+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:03.495179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.524356+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:34:13.524356+0000","last_peered":"2023-10-06T03:34:13.524356+0000","last_clean":"2023-10-06T03:34:13.524356+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:34:13.524356+0000","last_undegraded":"2023-10-06T03:34:13.524356+0000","last_fullsized":"2023-10-06T03:34:13.524356+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.558168+0000","last_change":"2023-10-06T03:34:13.555991+0000","last_active":"2023-10-06T03:34:13.558168+0000","last_peered":"2023-10-06T03:34:13.558168+0000","last_clean":"2023-10-06T03:34:13.558168+0000","last_became_active":"2023-10-06T03:34:13.555086+0000","last_became_peered":"2023-10-06T03:34:13.555086+0000","last_unstale":"2023-10-06T03:34:13.558168+0000","last_undegraded":"2023-10-06T03:34:13.558168+0000","last_fullsized":"2023-10-06T03:34:13.558168+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:27.486440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559924+0000","last_change":"2023-10-06T03:34:12.529984+0000","last_active":"2023-10-06T03:34:13.559924+0000","last_peered":"2023-10-06T03:34:13.559924+0000","last_clean":"2023-10-06T03:34:13.559924+0000","last_became_active":"2023-10-06T03:34:12.529440+0000","last_became_peered":"2023-10-06T03:34:12.529440+0000","last_unstale":"2023-10-06T03:34:13.559924+0000","last_undegraded":"2023-10-06T03:34:13.559924+0000","last_fullsized":"2023-10-06T03:34:13.559924+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:04.840518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.527887+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:34:13.527887+0000","last_peered":"2023-10-06T03:34:13.527887+0000","last_clean":"2023-10-06T03:34:13.527887+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:34:13.527887+0000","last_undegraded":"2023-10-06T03:34:13.527887+0000","last_fullsized":"2023-10-06T03:34:13.527887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.531546+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:34:13.531546+0000","last_peered":"2023-10-06T03:34:13.531546+0000","last_clean":"2023-10-06T03:34:13.531546+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:34:13.531546+0000","last_undegraded":"2023-10-06T03:34:13.531546+0000","last_fullsized":"2023-10-06T03:34:13.531546+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543979+0000","last_change":"2023-10-06T03:34:13.541987+0000","last_active":"2023-10-06T03:34:13.543979+0000","last_peered":"2023-10-06T03:34:13.543979+0000","last_clean":"2023-10-06T03:34:13.543979+0000","last_became_active":"2023-10-06T03:34:13.541710+0000","last_became_peered":"2023-10-06T03:34:13.541710+0000","last_unstale":"2023-10-06T03:34:13.543979+0000","last_undegraded":"2023-10-06T03:34:13.543979+0000","last_fullsized":"2023-10-06T03:34:13.543979+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.785861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.525061+0000","last_change":"2023-10-06T03:34:12.526945+0000","last_active":"2023-10-06T03:34:13.525061+0000","last_peered":"2023-10-06T03:34:13.525061+0000","last_clean":"2023-10-06T03:34:13.525061+0000","last_became_active":"2023-10-06T03:34:12.526661+0000","last_became_peered":"2023-10-06T03:34:12.526661+0000","last_unstale":"2023-10-06T03:34:13.525061+0000","last_undegraded":"2023-10-06T03:34:13.525061+0000","last_fullsized":"2023-10-06T03:34:13.525061+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:36.885264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.530135+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:34:13.530135+0000","last_peered":"2023-10-06T03:34:13.530135+0000","last_clean":"2023-10-06T03:34:13.530135+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:34:13.530135+0000","last_undegraded":"2023-10-06T03:34:13.530135+0000","last_fullsized":"2023-10-06T03:34:13.530135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559864+0000","last_change":"2023-10-06T03:34:13.553850+0000","last_active":"2023-10-06T03:34:13.559864+0000","last_peered":"2023-10-06T03:34:13.559864+0000","last_clean":"2023-10-06T03:34:13.559864+0000","last_became_active":"2023-10-06T03:34:13.553534+0000","last_became_peered":"2023-10-06T03:34:13.553534+0000","last_unstale":"2023-10-06T03:34:13.559864+0000","last_undegraded":"2023-10-06T03:34:13.559864+0000","last_fullsized":"2023-10-06T03:34:13.559864+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.785861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.530519+0000","last_change":"2023-10-06T03:34:12.529818+0000","last_active":"2023-10-06T03:34:13.530519+0000","last_peered":"2023-10-06T03:34:13.530519+0000","last_clean":"2023-10-06T03:34:13.530519+0000","last_became_active":"2023-10-06T03:34:12.529370+0000","last_became_peered":"2023-10-06T03:34:12.529370+0000","last_unstale":"2023-10-06T03:34:13.530519+0000","last_undegraded":"2023-10-06T03:34:13.530519+0000","last_fullsized":"2023-10-06T03:34:13.530519+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:54.385812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.531571+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:34:13.531571+0000","last_peered":"2023-10-06T03:34:13.531571+0000","last_clean":"2023-10-06T03:34:13.531571+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:34:13.531571+0000","last_undegraded":"2023-10-06T03:34:13.531571+0000","last_fullsized":"2023-10-06T03:34:13.531571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541671+0000","last_change":"2023-10-06T03:34:13.534808+0000","last_active":"2023-10-06T03:34:13.541671+0000","last_peered":"2023-10-06T03:34:13.541671+0000","last_clean":"2023-10-06T03:34:13.541671+0000","last_became_active":"2023-10-06T03:34:13.534549+0000","last_became_peered":"2023-10-06T03:34:13.534549+0000","last_unstale":"2023-10-06T03:34:13.541671+0000","last_undegraded":"2023-10-06T03:34:13.541671+0000","last_fullsized":"2023-10-06T03:34:13.541671+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:50.641614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.522197+0000","last_change":"2023-10-06T03:34:12.516230+0000","last_active":"2023-10-06T03:34:13.522197+0000","last_peered":"2023-10-06T03:34:13.522197+0000","last_clean":"2023-10-06T03:34:13.522197+0000","last_became_active":"2023-10-06T03:34:12.515968+0000","last_became_peered":"2023-10-06T03:34:12.515968+0000","last_unstale":"2023-10-06T03:34:13.522197+0000","last_undegraded":"2023-10-06T03:34:13.522197+0000","last_fullsized":"2023-10-06T03:34:13.522197+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.529216+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:34:13.529216+0000","last_peered":"2023-10-06T03:34:13.529216+0000","last_clean":"2023-10-06T03:34:13.529216+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:34:13.529216+0000","last_undegraded":"2023-10-06T03:34:13.529216+0000","last_fullsized":"2023-10-06T03:34:13.529216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.546934+0000","last_change":"2023-10-06T03:34:13.540397+0000","last_active":"2023-10-06T03:34:13.546934+0000","last_peered":"2023-10-06T03:34:13.546934+0000","last_clean":"2023-10-06T03:34:13.546934+0000","last_became_active":"2023-10-06T03:34:13.540142+0000","last_became_peered":"2023-10-06T03:34:13.540142+0000","last_unstale":"2023-10-06T03:34:13.546934+0000","last_undegraded":"2023-10-06T03:34:13.546934+0000","last_fullsized":"2023-10-06T03:34:13.546934+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:32.081718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.521373+0000","last_change":"2023-10-06T03:34:12.519229+0000","last_active":"2023-10-06T03:34:13.521373+0000","last_peered":"2023-10-06T03:34:13.521373+0000","last_clean":"2023-10-06T03:34:13.521373+0000","last_became_active":"2023-10-06T03:34:12.518975+0000","last_became_peered":"2023-10-06T03:34:12.518975+0000","last_unstale":"2023-10-06T03:34:13.521373+0000","last_undegraded":"2023-10-06T03:34:13.521373+0000","last_fullsized":"2023-10-06T03:34:13.521373+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:24.627602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.521342+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:34:13.521342+0000","last_peered":"2023-10-06T03:34:13.521342+0000","last_clean":"2023-10-06T03:34:13.521342+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:34:13.521342+0000","last_undegraded":"2023-10-06T03:34:13.521342+0000","last_fullsized":"2023-10-06T03:34:13.521342+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.537670+0000","last_change":"2023-10-06T03:34:13.532662+0000","last_active":"2023-10-06T03:34:13.537670+0000","last_peered":"2023-10-06T03:34:13.537670+0000","last_clean":"2023-10-06T03:34:13.537670+0000","last_became_active":"2023-10-06T03:34:13.532351+0000","last_became_peered":"2023-10-06T03:34:13.532351+0000","last_unstale":"2023-10-06T03:34:13.537670+0000","last_undegraded":"2023-10-06T03:34:13.537670+0000","last_fullsized":"2023-10-06T03:34:13.537670+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:23.077437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.523003+0000","last_change":"2023-10-06T03:34:12.514751+0000","last_active":"2023-10-06T03:34:13.523003+0000","last_peered":"2023-10-06T03:34:13.523003+0000","last_clean":"2023-10-06T03:34:13.523003+0000","last_became_active":"2023-10-06T03:34:12.514468+0000","last_became_peered":"2023-10-06T03:34:12.514468+0000","last_unstale":"2023-10-06T03:34:13.523003+0000","last_undegraded":"2023-10-06T03:34:13.523003+0000","last_fullsized":"2023-10-06T03:34:13.523003+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:07.994934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.521805+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:34:13.521805+0000","last_peered":"2023-10-06T03:34:13.521805+0000","last_clean":"2023-10-06T03:34:13.521805+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:34:13.521805+0000","last_undegraded":"2023-10-06T03:34:13.521805+0000","last_fullsized":"2023-10-06T03:34:13.521805+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073608,"statfs":{"total":96636764160,"available":96331374592,"internally_reserved":0,"allocated":966656,"data_stored":737317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59099999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.622}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.746}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":146875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.77200000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.625}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.68300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302320,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069520,"statfs":{"total":96636764160,"available":96327188480,"internally_reserved":0,"allocated":958464,"data_stored":736779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.45500000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.42799999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.40799999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.56299999999999994}]},{"osd":5,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.54900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301736,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070104,"statfs":{"total":96636764160,"available":96327786496,"internally_reserved":0,"allocated":360448,"data_stored":145377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.439},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":0.85899999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.71699999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.93700000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301736,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070104,"statfs":{"total":96636764160,"available":96327786496,"internally_reserved":0,"allocated":360448,"data_stored":145413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.442}]},{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.5}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.91000000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.94199999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.98799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301752,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070088,"statfs":{"total":96636764160,"available":96327770112,"internally_reserved":0,"allocated":376832,"data_stored":147891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.33100000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.442}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.51800000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.80000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":368640,"data_stored":146897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.433}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.45200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.55500000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.748},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.77900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":145455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.877}]},{"osd":2,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.78000000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.751}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":1.01}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:34:18.876 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T03:34:19.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.093+0000 7fede359e700 1 Processor -- start 2023-10-06T03:34:19.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.093+0000 7fede359e700 1 -- start start 2023-10-06T03:34:19.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.094+0000 7fede359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede40cabc0 0x7fede4164680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:19.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.094+0000 7fede359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede40c9090 con 0x7fede406b2a0 2023-10-06T03:34:19.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.094+0000 7fede359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede40c9370 con 0x7fede4159aa0 2023-10-06T03:34:19.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.094+0000 7fede359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede40ca130 con 0x7fede40cabc0 2023-10-06T03:34:19.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.095+0000 7fede1d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fede406b2a0 0x7fede40ca4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38020/0 (socket says 172.21.15.55:38020) 2023-10-06T03:34:19.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.095+0000 7fede1d9b700 1 -- 172.21.15.55:0/979810553 learned_addr learned my addr 172.21.15.55:0/979810553 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:19.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.095+0000 7fede2d9d700 1 --2- 172.21.15.55:0/979810553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede40cabc0 0x7fede4164680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:19.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.095+0000 7fede159a700 1 -- 172.21.15.55:0/979810553 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388141677 0 0) 0x7fede40c9090 con 0x7fede406b2a0 2023-10-06T03:34:19.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.095+0000 7fede159a700 1 -- 172.21.15.55:0/979810553 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedcc003570 con 0x7fede406b2a0 2023-10-06T03:34:19.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.095+0000 7fede159a700 1 -- 172.21.15.55:0/979810553 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 161361976 0 0) 0x7fede40c9370 con 0x7fede4159aa0 2023-10-06T03:34:19.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.095+0000 7fede159a700 1 -- 172.21.15.55:0/979810553 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fede40c9090 con 0x7fede4159aa0 2023-10-06T03:34:19.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.095+0000 7fede2d9d700 1 -- 172.21.15.55:0/979810553 >> v1:172.21.15.163:6789/0 conn(0x7fede4159aa0 legacy=0x7fede40cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:19.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.095+0000 7fede2d9d700 1 -- 172.21.15.55:0/979810553 >> v1:172.21.15.55:6789/0 conn(0x7fede406b2a0 legacy=0x7fede40ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:19.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.095+0000 7fede159a700 1 -- 172.21.15.55:0/979810553 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2194672921 0 0) 0x7fedcc003570 con 0x7fede406b2a0 2023-10-06T03:34:19.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.095+0000 7fede2d9d700 1 -- 172.21.15.55:0/979810553 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede40c9650 con 0x7fede40cabc0 2023-10-06T03:34:19.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.096+0000 7fede2d9d700 1 --2- 172.21.15.55:0/979810553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede40cabc0 0x7fede4164680 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feddc009e90 tx=0x7feddc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0c661fe5d80f5b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:19.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.096+0000 7fede159a700 1 -- 172.21.15.55:0/979810553 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc00e070 con 0x7fede40cabc0 2023-10-06T03:34:19.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.096+0000 7fede159a700 1 -- 172.21.15.55:0/979810553 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feddc00b870 con 0x7fede40cabc0 2023-10-06T03:34:19.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.096+0000 7fede159a700 1 -- 172.21.15.55:0/979810553 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8001470 con 0x7fede40cabc0 2023-10-06T03:34:19.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.096+0000 7fede359e700 1 -- 172.21.15.55:0/979810553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede40cabc0 msgr2=0x7fede4164680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:19.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.096+0000 7fede359e700 1 --2- 172.21.15.55:0/979810553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede40cabc0 0x7fede4164680 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feddc009e90 tx=0x7feddc009780 comp rx=0 tx=0).stop 2023-10-06T03:34:19.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.096+0000 7fede359e700 1 -- 172.21.15.55:0/979810553 shutdown_connections 2023-10-06T03:34:19.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.096+0000 7fede359e700 1 --2- 172.21.15.55:0/979810553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede40cabc0 0x7fede4164680 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.096+0000 7fede359e700 1 -- 172.21.15.55:0/979810553 >> 172.21.15.55:0/979810553 conn(0x7fede406baa0 msgr2=0x7fede406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:19.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.097+0000 7fede359e700 1 -- 172.21.15.55:0/979810553 shutdown_connections 2023-10-06T03:34:19.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.097+0000 7fede359e700 1 -- 172.21.15.55:0/979810553 wait complete. 2023-10-06T03:34:19.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.098+0000 7fede359e700 1 Processor -- start 2023-10-06T03:34:19.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.098+0000 7fede359e700 1 -- start start 2023-10-06T03:34:19.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.099+0000 7fede359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fede406b2a0 0x7fede4151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:19.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.099+0000 7fede359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fede40cabc0 0x7fede40c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:19.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.099+0000 7fede359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede4159aa0 0x7fede406d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:19.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.099+0000 7fede359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede4151860 con 0x7fede406b2a0 2023-10-06T03:34:19.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.099+0000 7fede359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede415e540 con 0x7fede40cabc0 2023-10-06T03:34:19.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.099+0000 7fede359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede415eb00 con 0x7fede4159aa0 2023-10-06T03:34:19.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.099+0000 7fede259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fede406b2a0 0x7fede4151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:19.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.099+0000 7fede259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fede406b2a0 0x7fede4151320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39876/0 (socket says 172.21.15.55:39876) 2023-10-06T03:34:19.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.099+0000 7fede259c700 1 -- 172.21.15.55:0/2792633746 learned_addr learned my addr 172.21.15.55:0/2792633746 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:19.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.099+0000 7fede1d9b700 1 --2- 172.21.15.55:0/2792633746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fede40cabc0 0x7fede40c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:19.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.099+0000 7fede2d9d700 1 --2- 172.21.15.55:0/2792633746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede4159aa0 0x7fede406d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:19.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.100+0000 7fede259c700 1 -- 172.21.15.55:0/2792633746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede4159aa0 msgr2=0x7fede406d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:19.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.100+0000 7fede259c700 1 --2- 172.21.15.55:0/2792633746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede4159aa0 0x7fede406d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.100+0000 7fede259c700 1 -- 172.21.15.55:0/2792633746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fede40cabc0 msgr2=0x7fede40c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:19.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.100+0000 7fede259c700 1 --2- 172.21.15.55:0/2792633746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fede40cabc0 0x7fede40c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.100+0000 7fede259c700 1 -- 172.21.15.55:0/2792633746 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc009a20 con 0x7fede406b2a0 2023-10-06T03:34:19.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.100+0000 7fede259c700 1 --2- 172.21.15.55:0/2792633746 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fede406b2a0 0x7fede4151320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fedd400c5b0 tx=0x7fedd4007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:19.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.100+0000 7fedd27fc700 1 -- 172.21.15.55:0/2792633746 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4010510 con 0x7fede406b2a0 2023-10-06T03:34:19.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.100+0000 7fedd27fc700 1 -- 172.21.15.55:0/2792633746 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd4010670 con 0x7fede406b2a0 2023-10-06T03:34:19.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.100+0000 7fedd27fc700 1 -- 172.21.15.55:0/2792633746 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4010a30 con 0x7fede406b2a0 2023-10-06T03:34:19.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.100+0000 7fede359e700 1 -- 172.21.15.55:0/2792633746 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede40c8240 con 0x7fede406b2a0 2023-10-06T03:34:19.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.100+0000 7fede359e700 1 -- 172.21.15.55:0/2792633746 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede415e790 con 0x7fede406b2a0 2023-10-06T03:34:19.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.101+0000 7fede359e700 1 -- 172.21.15.55:0/2792633746 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede406eba0 con 0x7fede406b2a0 2023-10-06T03:34:19.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.102+0000 7fedd27fc700 1 -- 172.21.15.55:0/2792633746 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fedd401b400 con 0x7fede406b2a0 2023-10-06T03:34:19.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.103+0000 7fedd27fc700 1 --2- 172.21.15.55:0/2792633746 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fedc4066f30 0x7fedc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:19.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.103+0000 7fedd27fc700 1 -- 172.21.15.55:0/2792633746 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fedd40843c0 con 0x7fede406b2a0 2023-10-06T03:34:19.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.104+0000 7fede1d9b700 1 --2- 172.21.15.55:0/2792633746 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fedc4066f30 0x7fedc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:19.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.104+0000 7fede1d9b700 1 --2- 172.21.15.55:0/2792633746 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fedc4066f30 0x7fedc40693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fedd8001ed0 tx=0x7fedd8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:19.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.107+0000 7fedd27fc700 1 -- 172.21.15.55:0/2792633746 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedd4051430 con 0x7fede406b2a0 2023-10-06T03:34:19.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.388+0000 7fede359e700 1 -- 172.21.15.55:0/2792633746 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fede40c9930 con 0x7fede406b2a0 2023-10-06T03:34:19.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.388+0000 7fedd27fc700 1 -- 172.21.15.55:0/2792633746 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1564 (secure 0 0 0) 0x7fedd4054a50 con 0x7fede406b2a0 2023-10-06T03:34:19.390 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:19.390 INFO:teuthology.orchestra.run.smithi055.stdout:{"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":83,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":32,"num_osds":8,"num_up_osds":8,"osd_up_since":1696563187,"num_in_osds":8,"osd_in_since":1696563178,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":23,"data_bytes":592674,"bytes_used":2456281088,"bytes_avail":770637832192,"bytes_total":773094113280,"write_bytes_sec":2218,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"d","status":"up:active","gid":4903}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2023-10-06T03:34:08.502132+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T03:34:19.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.391+0000 7fede359e700 1 -- 172.21.15.55:0/2792633746 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fedc4066f30 msgr2=0x7fedc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:19.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.391+0000 7fede359e700 1 --2- 172.21.15.55:0/2792633746 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fedc4066f30 0x7fedc40693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fedd8001ed0 tx=0x7fedd8009040 comp rx=0 tx=0).stop 2023-10-06T03:34:19.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.391+0000 7fede359e700 1 -- 172.21.15.55:0/2792633746 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fede406b2a0 msgr2=0x7fede4151320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:19.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.391+0000 7fede359e700 1 --2- 172.21.15.55:0/2792633746 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fede406b2a0 0x7fede4151320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fedd400c5b0 tx=0x7fedd4007e10 comp rx=0 tx=0).stop 2023-10-06T03:34:19.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.391+0000 7fede359e700 1 -- 172.21.15.55:0/2792633746 shutdown_connections 2023-10-06T03:34:19.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.391+0000 7fede359e700 1 --2- 172.21.15.55:0/2792633746 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fedc4066f30 0x7fedc40693e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.391+0000 7fede359e700 1 --2- 172.21.15.55:0/2792633746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede4159aa0 0x7fede406d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.391+0000 7fede359e700 1 --2- 172.21.15.55:0/2792633746 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fede406b2a0 0x7fede4151320 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.391+0000 7fede359e700 1 --2- 172.21.15.55:0/2792633746 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fede40cabc0 0x7fede40c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.391+0000 7fede359e700 1 -- 172.21.15.55:0/2792633746 >> 172.21.15.55:0/2792633746 conn(0x7fede406baa0 msgr2=0x7fede4154100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:19.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.391+0000 7fede359e700 1 -- 172.21.15.55:0/2792633746 shutdown_connections 2023-10-06T03:34:19.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.391+0000 7fede359e700 1 -- 172.21.15.55:0/2792633746 wait complete. 2023-10-06T03:34:19.409 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T03:34:19.410 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:34:19.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.597+0000 7fcf0f59d700 1 Processor -- start 2023-10-06T03:34:19.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.597+0000 7fcf0f59d700 1 -- start start 2023-10-06T03:34:19.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.598+0000 7fcf0f59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcf08142ea0 0x7fcf080b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:19.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.598+0000 7fcf0f59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf0806c0f0 0x7fcf0806a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:19.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.598+0000 7fcf0f59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf080b40f0 con 0x7fcf080ad870 2023-10-06T03:34:19.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.598+0000 7fcf0f59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf08069640 con 0x7fcf08142ea0 2023-10-06T03:34:19.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.598+0000 7fcf0f59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf08069920 con 0x7fcf0806c0f0 2023-10-06T03:34:19.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.598+0000 7fcf0e59b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcf08142ea0 0x7fcf080b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:19.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.598+0000 7fcf0dd9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcf080ad870 0x7fcf0806b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38034/0 (socket says 172.21.15.55:38034) 2023-10-06T03:34:19.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.599+0000 7fcf0dd9a700 1 -- 172.21.15.55:0/4280773864 learned_addr learned my addr 172.21.15.55:0/4280773864 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:19.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.599+0000 7fcf0ed9c700 1 --2- 172.21.15.55:0/4280773864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf0806c0f0 0x7fcf0806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:19.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.599+0000 7fcf0d599700 1 -- 172.21.15.55:0/4280773864 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836298928 0 0) 0x7fcf080b40f0 con 0x7fcf080ad870 2023-10-06T03:34:19.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.599+0000 7fcf0d599700 1 -- 172.21.15.55:0/4280773864 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcef4003570 con 0x7fcf080ad870 2023-10-06T03:34:19.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.599+0000 7fcf0e59b700 1 -- 172.21.15.55:0/4280773864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf0806c0f0 msgr2=0x7fcf0806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:19.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.599+0000 7fcf0e59b700 1 --2- 172.21.15.55:0/4280773864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf0806c0f0 0x7fcf0806a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.599+0000 7fcf0e59b700 1 -- 172.21.15.55:0/4280773864 >> v1:172.21.15.55:6789/0 conn(0x7fcf080ad870 legacy=0x7fcf0806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:19.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.599+0000 7fcf0e59b700 1 -- 172.21.15.55:0/4280773864 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf08069c00 con 0x7fcf08142ea0 2023-10-06T03:34:19.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.599+0000 7fcf0e59b700 1 --2- 172.21.15.55:0/4280773864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf08142ea0 0x7fcf080b3630 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcf040098e0 tx=0x7fcf040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782324f7b6155b2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:19.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.599+0000 7fcf0d599700 1 -- 172.21.15.55:0/4280773864 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf04011630 con 0x7fcf08142ea0 2023-10-06T03:34:19.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.600+0000 7fcf0d599700 1 -- 172.21.15.55:0/4280773864 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf04011790 con 0x7fcf08142ea0 2023-10-06T03:34:19.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.600+0000 7fcf0d599700 1 -- 172.21.15.55:0/4280773864 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf04011b30 con 0x7fcf08142ea0 2023-10-06T03:34:19.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.600+0000 7fcf0f59d700 1 -- 172.21.15.55:0/4280773864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf08142ea0 msgr2=0x7fcf080b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:19.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.600+0000 7fcf0f59d700 1 --2- 172.21.15.55:0/4280773864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf08142ea0 0x7fcf080b3630 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcf040098e0 tx=0x7fcf040091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.600+0000 7fcf0f59d700 1 -- 172.21.15.55:0/4280773864 shutdown_connections 2023-10-06T03:34:19.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.600+0000 7fcf0f59d700 1 --2- 172.21.15.55:0/4280773864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf0806c0f0 0x7fcf0806a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.600+0000 7fcf0f59d700 1 --2- 172.21.15.55:0/4280773864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf08142ea0 0x7fcf080b3630 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.600+0000 7fcf0f59d700 1 -- 172.21.15.55:0/4280773864 >> 172.21.15.55:0/4280773864 conn(0x7fcf080ce430 msgr2=0x7fcf080ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:19.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.600+0000 7fcf0f59d700 1 -- 172.21.15.55:0/4280773864 shutdown_connections 2023-10-06T03:34:19.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.600+0000 7fcf0f59d700 1 -- 172.21.15.55:0/4280773864 wait complete. 2023-10-06T03:34:19.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.602+0000 7fcf0f59d700 1 Processor -- start 2023-10-06T03:34:19.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.602+0000 7fcf0f59d700 1 -- start start 2023-10-06T03:34:19.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.603+0000 7fcf0f59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf0806c0f0 0x7fcf080c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:19.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.603+0000 7fcf0f59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf08142ea0 0x7fcf080c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:19.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.603+0000 7fcf0f59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf08169ad0 0x7fcf08169f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:19.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.603+0000 7fcf0f59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0816b740 con 0x7fcf0806c0f0 2023-10-06T03:34:19.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.603+0000 7fcf0f59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf081403a0 con 0x7fcf08169ad0 2023-10-06T03:34:19.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.603+0000 7fcf0f59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf080c9ef0 con 0x7fcf08142ea0 2023-10-06T03:34:19.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.603+0000 7fcf0e59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf0806c0f0 0x7fcf080c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:19.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.603+0000 7fcf0e59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf0806c0f0 0x7fcf080c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39900/0 (socket says 172.21.15.55:39900) 2023-10-06T03:34:19.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.603+0000 7fcf0e59b700 1 -- 172.21.15.55:0/1041489591 learned_addr learned my addr 172.21.15.55:0/1041489591 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:19.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.603+0000 7fcf0e59b700 1 -- 172.21.15.55:0/1041489591 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf08142ea0 msgr2=0x7fcf080c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:19.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.603+0000 7fcf0e59b700 1 --2- 172.21.15.55:0/1041489591 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf08142ea0 0x7fcf080c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.603+0000 7fcf0e59b700 1 -- 172.21.15.55:0/1041489591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf08169ad0 msgr2=0x7fcf08169f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:19.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.603+0000 7fcf0e59b700 1 --2- 172.21.15.55:0/1041489591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf08169ad0 0x7fcf08169f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.603+0000 7fcf0e59b700 1 -- 172.21.15.55:0/1041489591 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04009420 con 0x7fcf0806c0f0 2023-10-06T03:34:19.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.604+0000 7fcf0e59b700 1 --2- 172.21.15.55:0/1041489591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf0806c0f0 0x7fcf080c9180 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009ed0 tx=0x7fcf04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:19.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.604+0000 7fcefe7fc700 1 -- 172.21.15.55:0/1041489591 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf04011900 con 0x7fcf0806c0f0 2023-10-06T03:34:19.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.604+0000 7fcefe7fc700 1 -- 172.21.15.55:0/1041489591 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf04011a60 con 0x7fcf0806c0f0 2023-10-06T03:34:19.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.604+0000 7fcefe7fc700 1 -- 172.21.15.55:0/1041489591 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf04026680 con 0x7fcf0806c0f0 2023-10-06T03:34:19.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.604+0000 7fcf0f59d700 1 -- 172.21.15.55:0/1041489591 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0816ae10 con 0x7fcf0806c0f0 2023-10-06T03:34:19.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.604+0000 7fcf0f59d700 1 -- 172.21.15.55:0/1041489591 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0816b3d0 con 0x7fcf0806c0f0 2023-10-06T03:34:19.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.605+0000 7fcf0f59d700 1 -- 172.21.15.55:0/1041489591 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0804c3c0 con 0x7fcf0806c0f0 2023-10-06T03:34:19.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.606+0000 7fcefe7fc700 1 -- 172.21.15.55:0/1041489591 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcf04023030 con 0x7fcf0806c0f0 2023-10-06T03:34:19.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.607+0000 7fcefe7fc700 1 --2- 172.21.15.55:0/1041489591 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fceec066f30 0x7fceec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:19.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.607+0000 7fcefe7fc700 1 -- 172.21.15.55:0/1041489591 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcf04085de0 con 0x7fcf0806c0f0 2023-10-06T03:34:19.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.608+0000 7fcf0dd9a700 1 --2- 172.21.15.55:0/1041489591 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fceec066f30 0x7fceec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:19.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.608+0000 7fcf0dd9a700 1 --2- 172.21.15.55:0/1041489591 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fceec066f30 0x7fceec0693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcf080b12a0 tx=0x7fcef80079d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:19.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.611+0000 7fcefe7fc700 1 -- 172.21.15.55:0/1041489591 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf04052ed0 con 0x7fcf0806c0f0 2023-10-06T03:34:19.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.812+0000 7fcf0f59d700 1 -- 172.21.15.55:0/1041489591 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf080b2d80 con 0x7fceec066f30 2023-10-06T03:34:19.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.822+0000 7fcefe7fc700 1 -- 172.21.15.55:0/1041489591 <== mgr.4114 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311808 (secure 0 0 0) 0x7fcf080b2d80 con 0x7fceec066f30 2023-10-06T03:34:19.827 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:19.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.826+0000 7fcf0f59d700 1 -- 172.21.15.55:0/1041489591 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fceec066f30 msgr2=0x7fceec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:19.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.826+0000 7fcf0f59d700 1 --2- 172.21.15.55:0/1041489591 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fceec066f30 0x7fceec0693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcf080b12a0 tx=0x7fcef80079d0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.826+0000 7fcf0f59d700 1 -- 172.21.15.55:0/1041489591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf0806c0f0 msgr2=0x7fcf080c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:19.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.826+0000 7fcf0f59d700 1 --2- 172.21.15.55:0/1041489591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf0806c0f0 0x7fcf080c9180 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009ed0 tx=0x7fcf04017000 comp rx=0 tx=0).stop 2023-10-06T03:34:19.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.826+0000 7fcf0f59d700 1 -- 172.21.15.55:0/1041489591 shutdown_connections 2023-10-06T03:34:19.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.826+0000 7fcf0f59d700 1 --2- 172.21.15.55:0/1041489591 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf08169ad0 0x7fcf08169f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.826+0000 7fcf0f59d700 1 --2- 172.21.15.55:0/1041489591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf0806c0f0 0x7fcf080c9180 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.826+0000 7fcf0f59d700 1 --2- 172.21.15.55:0/1041489591 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fceec066f30 0x7fceec0693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.826+0000 7fcf0f59d700 1 --2- 172.21.15.55:0/1041489591 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf08142ea0 0x7fcf080c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:19.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.826+0000 7fcf0f59d700 1 -- 172.21.15.55:0/1041489591 >> 172.21.15.55:0/1041489591 conn(0x7fcf080ce430 msgr2=0x7fcf0806b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:19.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.827+0000 7fcf0f59d700 1 -- 172.21.15.55:0/1041489591 shutdown_connections 2023-10-06T03:34:19.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:19.827+0000 7fcf0f59d700 1 -- 172.21.15.55:0/1041489591 wait complete. 2023-10-06T03:34:19.831 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:34:19.845 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-10-06T03:34:18.504345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398712,"kb_used_data":4024,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752576008,"statfs":{"total":773094113280,"available":770637832192,"internally_reserved":0,"allocated":4120576,"data_stored":2352004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001321"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.548919+0000","last_change":"2023-10-06T03:34:13.543351+0000","last_active":"2023-10-06T03:34:13.548919+0000","last_peered":"2023-10-06T03:34:13.548919+0000","last_clean":"2023-10-06T03:34:13.548919+0000","last_became_active":"2023-10-06T03:34:13.542886+0000","last_became_peered":"2023-10-06T03:34:13.542886+0000","last_unstale":"2023-10-06T03:34:13.548919+0000","last_undegraded":"2023-10-06T03:34:13.548919+0000","last_fullsized":"2023-10-06T03:34:13.548919+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:25.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541689+0000","last_change":"2023-10-06T03:34:13.538209+0000","last_active":"2023-10-06T03:34:13.541689+0000","last_peered":"2023-10-06T03:34:13.541689+0000","last_clean":"2023-10-06T03:34:13.541689+0000","last_became_active":"2023-10-06T03:34:13.537849+0000","last_became_peered":"2023-10-06T03:34:13.537849+0000","last_unstale":"2023-10-06T03:34:13.541689+0000","last_undegraded":"2023-10-06T03:34:13.541689+0000","last_fullsized":"2023-10-06T03:34:13.541689+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:38.708108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550150+0000","last_change":"2023-10-06T03:34:13.547917+0000","last_active":"2023-10-06T03:34:13.550150+0000","last_peered":"2023-10-06T03:34:13.550150+0000","last_clean":"2023-10-06T03:34:13.550150+0000","last_became_active":"2023-10-06T03:34:13.546914+0000","last_became_peered":"2023-10-06T03:34:13.546914+0000","last_unstale":"2023-10-06T03:34:13.550150+0000","last_undegraded":"2023-10-06T03:34:13.550150+0000","last_fullsized":"2023-10-06T03:34:13.550150+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.785861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544849+0000","last_change":"2023-10-06T03:34:13.538592+0000","last_active":"2023-10-06T03:34:13.544849+0000","last_peered":"2023-10-06T03:34:13.544849+0000","last_clean":"2023-10-06T03:34:13.544849+0000","last_became_active":"2023-10-06T03:34:13.538156+0000","last_became_peered":"2023-10-06T03:34:13.538156+0000","last_unstale":"2023-10-06T03:34:13.544849+0000","last_undegraded":"2023-10-06T03:34:13.544849+0000","last_fullsized":"2023-10-06T03:34:13.544849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:10.434143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543863+0000","last_change":"2023-10-06T03:34:13.534736+0000","last_active":"2023-10-06T03:34:13.543863+0000","last_peered":"2023-10-06T03:34:13.543863+0000","last_clean":"2023-10-06T03:34:13.543863+0000","last_became_active":"2023-10-06T03:34:13.534493+0000","last_became_peered":"2023-10-06T03:34:13.534493+0000","last_unstale":"2023-10-06T03:34:13.543863+0000","last_undegraded":"2023-10-06T03:34:13.543863+0000","last_fullsized":"2023-10-06T03:34:13.543863+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:02.900878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.546850+0000","last_change":"2023-10-06T03:34:13.541709+0000","last_active":"2023-10-06T03:34:13.546850+0000","last_peered":"2023-10-06T03:34:13.546850+0000","last_clean":"2023-10-06T03:34:13.546850+0000","last_became_active":"2023-10-06T03:34:13.541399+0000","last_became_peered":"2023-10-06T03:34:13.541399+0000","last_unstale":"2023-10-06T03:34:13.546850+0000","last_undegraded":"2023-10-06T03:34:13.546850+0000","last_fullsized":"2023-10-06T03:34:13.546850+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:22.590165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557767+0000","last_change":"2023-10-06T03:34:13.545667+0000","last_active":"2023-10-06T03:34:13.557767+0000","last_peered":"2023-10-06T03:34:13.557767+0000","last_clean":"2023-10-06T03:34:13.557767+0000","last_became_active":"2023-10-06T03:34:13.545306+0000","last_became_peered":"2023-10-06T03:34:13.545306+0000","last_unstale":"2023-10-06T03:34:13.557767+0000","last_undegraded":"2023-10-06T03:34:13.557767+0000","last_fullsized":"2023-10-06T03:34:13.557767+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:38.708108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550618+0000","last_change":"2023-10-06T03:34:13.549359+0000","last_active":"2023-10-06T03:34:13.550618+0000","last_peered":"2023-10-06T03:34:13.550618+0000","last_clean":"2023-10-06T03:34:13.550618+0000","last_became_active":"2023-10-06T03:34:13.548692+0000","last_became_peered":"2023-10-06T03:34:13.548692+0000","last_unstale":"2023-10-06T03:34:13.550618+0000","last_undegraded":"2023-10-06T03:34:13.550618+0000","last_fullsized":"2023-10-06T03:34:13.550618+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:35.437314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.549190+0000","last_change":"2023-10-06T03:34:13.546572+0000","last_active":"2023-10-06T03:34:13.549190+0000","last_peered":"2023-10-06T03:34:13.549190+0000","last_clean":"2023-10-06T03:34:13.549190+0000","last_became_active":"2023-10-06T03:34:13.546313+0000","last_became_peered":"2023-10-06T03:34:13.546313+0000","last_unstale":"2023-10-06T03:34:13.549190+0000","last_undegraded":"2023-10-06T03:34:13.549190+0000","last_fullsized":"2023-10-06T03:34:13.549190+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:17.351579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560723+0000","last_change":"2023-10-06T03:34:13.558859+0000","last_active":"2023-10-06T03:34:13.560723+0000","last_peered":"2023-10-06T03:34:13.560723+0000","last_clean":"2023-10-06T03:34:13.560723+0000","last_became_active":"2023-10-06T03:34:13.558102+0000","last_became_peered":"2023-10-06T03:34:13.558102+0000","last_unstale":"2023-10-06T03:34:13.560723+0000","last_undegraded":"2023-10-06T03:34:13.560723+0000","last_fullsized":"2023-10-06T03:34:13.560723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:17.622122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559631+0000","last_change":"2023-10-06T03:34:13.548391+0000","last_active":"2023-10-06T03:34:13.559631+0000","last_peered":"2023-10-06T03:34:13.559631+0000","last_clean":"2023-10-06T03:34:13.559631+0000","last_became_active":"2023-10-06T03:34:13.548117+0000","last_became_peered":"2023-10-06T03:34:13.548117+0000","last_unstale":"2023-10-06T03:34:13.559631+0000","last_undegraded":"2023-10-06T03:34:13.559631+0000","last_fullsized":"2023-10-06T03:34:13.559631+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:17.351579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541952+0000","last_change":"2023-10-06T03:34:13.533970+0000","last_active":"2023-10-06T03:34:13.541952+0000","last_peered":"2023-10-06T03:34:13.541952+0000","last_clean":"2023-10-06T03:34:13.541952+0000","last_became_active":"2023-10-06T03:34:13.533643+0000","last_became_peered":"2023-10-06T03:34:13.533643+0000","last_unstale":"2023-10-06T03:34:13.541952+0000","last_undegraded":"2023-10-06T03:34:13.541952+0000","last_fullsized":"2023-10-06T03:34:13.541952+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:38.277766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559747+0000","last_change":"2023-10-06T03:34:13.548687+0000","last_active":"2023-10-06T03:34:13.559747+0000","last_peered":"2023-10-06T03:34:13.559747+0000","last_clean":"2023-10-06T03:34:13.559747+0000","last_became_active":"2023-10-06T03:34:13.547089+0000","last_became_peered":"2023-10-06T03:34:13.547089+0000","last_unstale":"2023-10-06T03:34:13.559747+0000","last_undegraded":"2023-10-06T03:34:13.559747+0000","last_fullsized":"2023-10-06T03:34:13.559747+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:35.437314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539582+0000","last_change":"2023-10-06T03:34:13.533974+0000","last_active":"2023-10-06T03:34:13.539582+0000","last_peered":"2023-10-06T03:34:13.539582+0000","last_clean":"2023-10-06T03:34:13.539582+0000","last_became_active":"2023-10-06T03:34:13.533686+0000","last_became_peered":"2023-10-06T03:34:13.533686+0000","last_unstale":"2023-10-06T03:34:13.539582+0000","last_undegraded":"2023-10-06T03:34:13.539582+0000","last_fullsized":"2023-10-06T03:34:13.539582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:15.558498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557756+0000","last_change":"2023-10-06T03:34:13.545608+0000","last_active":"2023-10-06T03:34:13.557756+0000","last_peered":"2023-10-06T03:34:13.557756+0000","last_clean":"2023-10-06T03:34:13.557756+0000","last_became_active":"2023-10-06T03:34:13.544797+0000","last_became_peered":"2023-10-06T03:34:13.544797+0000","last_unstale":"2023-10-06T03:34:13.557756+0000","last_undegraded":"2023-10-06T03:34:13.557756+0000","last_fullsized":"2023-10-06T03:34:13.557756+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:50.641614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557053+0000","last_change":"2023-10-06T03:34:13.555245+0000","last_active":"2023-10-06T03:34:13.557053+0000","last_peered":"2023-10-06T03:34:13.557053+0000","last_clean":"2023-10-06T03:34:13.557053+0000","last_became_active":"2023-10-06T03:34:13.555025+0000","last_became_peered":"2023-10-06T03:34:13.555025+0000","last_unstale":"2023-10-06T03:34:13.557053+0000","last_undegraded":"2023-10-06T03:34:13.557053+0000","last_fullsized":"2023-10-06T03:34:13.557053+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:25.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.552765+0000","last_change":"2023-10-06T03:34:13.549273+0000","last_active":"2023-10-06T03:34:13.552765+0000","last_peered":"2023-10-06T03:34:13.552765+0000","last_clean":"2023-10-06T03:34:13.552765+0000","last_became_active":"2023-10-06T03:34:13.548990+0000","last_became_peered":"2023-10-06T03:34:13.548990+0000","last_unstale":"2023-10-06T03:34:13.552765+0000","last_undegraded":"2023-10-06T03:34:13.552765+0000","last_fullsized":"2023-10-06T03:34:13.552765+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:17.622122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541045+0000","last_change":"2023-10-06T03:34:13.538176+0000","last_active":"2023-10-06T03:34:13.541045+0000","last_peered":"2023-10-06T03:34:13.541045+0000","last_clean":"2023-10-06T03:34:13.541045+0000","last_became_active":"2023-10-06T03:34:13.537927+0000","last_became_peered":"2023-10-06T03:34:13.537927+0000","last_unstale":"2023-10-06T03:34:13.541045+0000","last_undegraded":"2023-10-06T03:34:13.541045+0000","last_fullsized":"2023-10-06T03:34:13.541045+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:48.522789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544460+0000","last_change":"2023-10-06T03:34:13.539248+0000","last_active":"2023-10-06T03:34:13.544460+0000","last_peered":"2023-10-06T03:34:13.544460+0000","last_clean":"2023-10-06T03:34:13.544460+0000","last_became_active":"2023-10-06T03:34:13.539008+0000","last_became_peered":"2023-10-06T03:34:13.539008+0000","last_unstale":"2023-10-06T03:34:13.544460+0000","last_undegraded":"2023-10-06T03:34:13.544460+0000","last_fullsized":"2023-10-06T03:34:13.544460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:28.353727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541088+0000","last_change":"2023-10-06T03:34:13.537228+0000","last_active":"2023-10-06T03:34:13.541088+0000","last_peered":"2023-10-06T03:34:13.541088+0000","last_clean":"2023-10-06T03:34:13.541088+0000","last_became_active":"2023-10-06T03:34:13.536971+0000","last_became_peered":"2023-10-06T03:34:13.536971+0000","last_unstale":"2023-10-06T03:34:13.541088+0000","last_undegraded":"2023-10-06T03:34:13.541088+0000","last_fullsized":"2023-10-06T03:34:13.541088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:09.006005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557807+0000","last_change":"2023-10-06T03:34:13.547135+0000","last_active":"2023-10-06T03:34:13.557807+0000","last_peered":"2023-10-06T03:34:13.557807+0000","last_clean":"2023-10-06T03:34:13.557807+0000","last_became_active":"2023-10-06T03:34:13.546876+0000","last_became_peered":"2023-10-06T03:34:13.546876+0000","last_unstale":"2023-10-06T03:34:13.557807+0000","last_undegraded":"2023-10-06T03:34:13.557807+0000","last_fullsized":"2023-10-06T03:34:13.557807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:03.345245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539603+0000","last_change":"2023-10-06T03:34:13.535259+0000","last_active":"2023-10-06T03:34:13.539603+0000","last_peered":"2023-10-06T03:34:13.539603+0000","last_clean":"2023-10-06T03:34:13.539603+0000","last_became_active":"2023-10-06T03:34:13.535006+0000","last_became_peered":"2023-10-06T03:34:13.535006+0000","last_unstale":"2023-10-06T03:34:13.539603+0000","last_undegraded":"2023-10-06T03:34:13.539603+0000","last_fullsized":"2023-10-06T03:34:13.539603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:15.748952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.552796+0000","last_change":"2023-10-06T03:34:13.549634+0000","last_active":"2023-10-06T03:34:13.552796+0000","last_peered":"2023-10-06T03:34:13.552796+0000","last_clean":"2023-10-06T03:34:13.552796+0000","last_became_active":"2023-10-06T03:34:13.549343+0000","last_became_peered":"2023-10-06T03:34:13.549343+0000","last_unstale":"2023-10-06T03:34:13.552796+0000","last_undegraded":"2023-10-06T03:34:13.552796+0000","last_fullsized":"2023-10-06T03:34:13.552796+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:50.641614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544186+0000","last_change":"2023-10-06T03:34:13.537144+0000","last_active":"2023-10-06T03:34:13.544186+0000","last_peered":"2023-10-06T03:34:13.544186+0000","last_clean":"2023-10-06T03:34:13.544186+0000","last_became_active":"2023-10-06T03:34:13.536727+0000","last_became_peered":"2023-10-06T03:34:13.536727+0000","last_unstale":"2023-10-06T03:34:13.544186+0000","last_undegraded":"2023-10-06T03:34:13.544186+0000","last_fullsized":"2023-10-06T03:34:13.544186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:09.006005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.537520+0000","last_change":"2023-10-06T03:34:13.529906+0000","last_active":"2023-10-06T03:34:13.537520+0000","last_peered":"2023-10-06T03:34:13.537520+0000","last_clean":"2023-10-06T03:34:13.537520+0000","last_became_active":"2023-10-06T03:34:13.529637+0000","last_became_peered":"2023-10-06T03:34:13.529637+0000","last_unstale":"2023-10-06T03:34:13.537520+0000","last_undegraded":"2023-10-06T03:34:13.537520+0000","last_fullsized":"2023-10-06T03:34:13.537520+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:10.434143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.558046+0000","last_change":"2023-10-06T03:34:13.555751+0000","last_active":"2023-10-06T03:34:13.558046+0000","last_peered":"2023-10-06T03:34:13.558046+0000","last_clean":"2023-10-06T03:34:13.558046+0000","last_became_active":"2023-10-06T03:34:13.555469+0000","last_became_peered":"2023-10-06T03:34:13.555469+0000","last_unstale":"2023-10-06T03:34:13.558046+0000","last_undegraded":"2023-10-06T03:34:13.558046+0000","last_fullsized":"2023-10-06T03:34:13.558046+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:53.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540269+0000","last_change":"2023-10-06T03:34:13.534759+0000","last_active":"2023-10-06T03:34:13.540269+0000","last_peered":"2023-10-06T03:34:13.540269+0000","last_clean":"2023-10-06T03:34:13.540269+0000","last_became_active":"2023-10-06T03:34:13.534458+0000","last_became_peered":"2023-10-06T03:34:13.534458+0000","last_unstale":"2023-10-06T03:34:13.540269+0000","last_undegraded":"2023-10-06T03:34:13.540269+0000","last_fullsized":"2023-10-06T03:34:13.540269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:55.396883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559551+0000","last_change":"2023-10-06T03:34:13.547737+0000","last_active":"2023-10-06T03:34:13.559551+0000","last_peered":"2023-10-06T03:34:13.559551+0000","last_clean":"2023-10-06T03:34:13.559551+0000","last_became_active":"2023-10-06T03:34:13.547463+0000","last_became_peered":"2023-10-06T03:34:13.547463+0000","last_unstale":"2023-10-06T03:34:13.559551+0000","last_undegraded":"2023-10-06T03:34:13.559551+0000","last_fullsized":"2023-10-06T03:34:13.559551+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:30.492483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540458+0000","last_change":"2023-10-06T03:34:13.536475+0000","last_active":"2023-10-06T03:34:13.540458+0000","last_peered":"2023-10-06T03:34:13.540458+0000","last_clean":"2023-10-06T03:34:13.540458+0000","last_became_active":"2023-10-06T03:34:13.536202+0000","last_became_peered":"2023-10-06T03:34:13.536202+0000","last_unstale":"2023-10-06T03:34:13.540458+0000","last_undegraded":"2023-10-06T03:34:13.540458+0000","last_fullsized":"2023-10-06T03:34:13.540458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:02.900878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.537621+0000","last_change":"2023-10-06T03:34:13.530628+0000","last_active":"2023-10-06T03:34:13.537621+0000","last_peered":"2023-10-06T03:34:13.537621+0000","last_clean":"2023-10-06T03:34:13.537621+0000","last_became_active":"2023-10-06T03:34:13.530318+0000","last_became_peered":"2023-10-06T03:34:13.530318+0000","last_unstale":"2023-10-06T03:34:13.537621+0000","last_undegraded":"2023-10-06T03:34:13.537621+0000","last_fullsized":"2023-10-06T03:34:13.537621+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:18.351278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550254+0000","last_change":"2023-10-06T03:34:13.547733+0000","last_active":"2023-10-06T03:34:13.550254+0000","last_peered":"2023-10-06T03:34:13.550254+0000","last_clean":"2023-10-06T03:34:13.550254+0000","last_became_active":"2023-10-06T03:34:13.547402+0000","last_became_peered":"2023-10-06T03:34:13.547402+0000","last_unstale":"2023-10-06T03:34:13.550254+0000","last_undegraded":"2023-10-06T03:34:13.550254+0000","last_fullsized":"2023-10-06T03:34:13.550254+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:05.808082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.547236+0000","last_change":"2023-10-06T03:34:13.540168+0000","last_active":"2023-10-06T03:34:13.547236+0000","last_peered":"2023-10-06T03:34:13.547236+0000","last_clean":"2023-10-06T03:34:13.547236+0000","last_became_active":"2023-10-06T03:34:13.539910+0000","last_became_peered":"2023-10-06T03:34:13.539910+0000","last_unstale":"2023-10-06T03:34:13.547236+0000","last_undegraded":"2023-10-06T03:34:13.547236+0000","last_fullsized":"2023-10-06T03:34:13.547236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:04.506250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557194+0000","last_change":"2023-10-06T03:34:13.556269+0000","last_active":"2023-10-06T03:34:13.557194+0000","last_peered":"2023-10-06T03:34:13.557194+0000","last_clean":"2023-10-06T03:34:13.557194+0000","last_became_active":"2023-10-06T03:34:13.555974+0000","last_became_peered":"2023-10-06T03:34:13.555974+0000","last_unstale":"2023-10-06T03:34:13.557194+0000","last_undegraded":"2023-10-06T03:34:13.557194+0000","last_fullsized":"2023-10-06T03:34:13.557194+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:03.345245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"31'4","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544660+0000","last_change":"2023-10-06T03:34:12.523661+0000","last_active":"2023-10-06T03:34:13.544660+0000","last_peered":"2023-10-06T03:34:13.544660+0000","last_clean":"2023-10-06T03:34:13.544660+0000","last_became_active":"2023-10-06T03:34:12.523343+0000","last_became_peered":"2023-10-06T03:34:12.523343+0000","last_unstale":"2023-10-06T03:34:13.544660+0000","last_undegraded":"2023-10-06T03:34:13.544660+0000","last_fullsized":"2023-10-06T03:34:13.544660+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:23.796615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.549209+0000","last_change":"2023-10-06T03:34:13.546825+0000","last_active":"2023-10-06T03:34:13.549209+0000","last_peered":"2023-10-06T03:34:13.549209+0000","last_clean":"2023-10-06T03:34:13.549209+0000","last_became_active":"2023-10-06T03:34:13.546561+0000","last_became_peered":"2023-10-06T03:34:13.546561+0000","last_unstale":"2023-10-06T03:34:13.549209+0000","last_undegraded":"2023-10-06T03:34:13.549209+0000","last_fullsized":"2023-10-06T03:34:13.549209+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:10.048229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540711+0000","last_change":"2023-10-06T03:34:12.519828+0000","last_active":"2023-10-06T03:34:13.540711+0000","last_peered":"2023-10-06T03:34:13.540711+0000","last_clean":"2023-10-06T03:34:13.540711+0000","last_became_active":"2023-10-06T03:34:12.519596+0000","last_became_peered":"2023-10-06T03:34:12.519596+0000","last_unstale":"2023-10-06T03:34:13.540711+0000","last_undegraded":"2023-10-06T03:34:13.540711+0000","last_fullsized":"2023-10-06T03:34:13.540711+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:01.889807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539250+0000","last_change":"2023-10-06T03:34:13.533928+0000","last_active":"2023-10-06T03:34:13.539250+0000","last_peered":"2023-10-06T03:34:13.539250+0000","last_clean":"2023-10-06T03:34:13.539250+0000","last_became_active":"2023-10-06T03:34:13.533642+0000","last_became_peered":"2023-10-06T03:34:13.533642+0000","last_unstale":"2023-10-06T03:34:13.539250+0000","last_undegraded":"2023-10-06T03:34:13.539250+0000","last_fullsized":"2023-10-06T03:34:13.539250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:25.840403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.545261+0000","last_change":"2023-10-06T03:34:12.526656+0000","last_active":"2023-10-06T03:34:13.545261+0000","last_peered":"2023-10-06T03:34:13.545261+0000","last_clean":"2023-10-06T03:34:13.545261+0000","last_became_active":"2023-10-06T03:34:12.525776+0000","last_became_peered":"2023-10-06T03:34:12.525776+0000","last_unstale":"2023-10-06T03:34:13.545261+0000","last_undegraded":"2023-10-06T03:34:13.545261+0000","last_fullsized":"2023-10-06T03:34:13.545261+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:27.342656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.548898+0000","last_change":"2023-10-06T03:34:13.543146+0000","last_active":"2023-10-06T03:34:13.548898+0000","last_peered":"2023-10-06T03:34:13.548898+0000","last_clean":"2023-10-06T03:34:13.548898+0000","last_became_active":"2023-10-06T03:34:13.542460+0000","last_became_peered":"2023-10-06T03:34:13.542460+0000","last_unstale":"2023-10-06T03:34:13.548898+0000","last_undegraded":"2023-10-06T03:34:13.548898+0000","last_fullsized":"2023-10-06T03:34:13.548898+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.785861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.552296+0000","last_change":"2023-10-06T03:34:12.524322+0000","last_active":"2023-10-06T03:34:13.552296+0000","last_peered":"2023-10-06T03:34:13.552296+0000","last_clean":"2023-10-06T03:34:13.552296+0000","last_became_active":"2023-10-06T03:34:12.524037+0000","last_became_peered":"2023-10-06T03:34:12.524037+0000","last_unstale":"2023-10-06T03:34:13.552296+0000","last_undegraded":"2023-10-06T03:34:13.552296+0000","last_fullsized":"2023-10-06T03:34:13.552296+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:59.774790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543775+0000","last_change":"2023-10-06T03:34:13.539137+0000","last_active":"2023-10-06T03:34:13.543775+0000","last_peered":"2023-10-06T03:34:13.543775+0000","last_clean":"2023-10-06T03:34:13.543775+0000","last_became_active":"2023-10-06T03:34:13.538833+0000","last_became_peered":"2023-10-06T03:34:13.538833+0000","last_unstale":"2023-10-06T03:34:13.543775+0000","last_undegraded":"2023-10-06T03:34:13.543775+0000","last_fullsized":"2023-10-06T03:34:13.543775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:27.486440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559924+0000","last_change":"2023-10-06T03:34:12.527023+0000","last_active":"2023-10-06T03:34:13.559924+0000","last_peered":"2023-10-06T03:34:13.559924+0000","last_clean":"2023-10-06T03:34:13.559924+0000","last_became_active":"2023-10-06T03:34:12.526443+0000","last_became_peered":"2023-10-06T03:34:12.526443+0000","last_unstale":"2023-10-06T03:34:13.559924+0000","last_undegraded":"2023-10-06T03:34:13.559924+0000","last_fullsized":"2023-10-06T03:34:13.559924+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:54.385812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557742+0000","last_change":"2023-10-06T03:34:13.544656+0000","last_active":"2023-10-06T03:34:13.557742+0000","last_peered":"2023-10-06T03:34:13.557742+0000","last_clean":"2023-10-06T03:34:13.557742+0000","last_became_active":"2023-10-06T03:34:13.544249+0000","last_became_peered":"2023-10-06T03:34:13.544249+0000","last_unstale":"2023-10-06T03:34:13.557742+0000","last_undegraded":"2023-10-06T03:34:13.557742+0000","last_fullsized":"2023-10-06T03:34:13.557742+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:01.766184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.552592+0000","last_change":"2023-10-06T03:34:12.532851+0000","last_active":"2023-10-06T03:34:13.552592+0000","last_peered":"2023-10-06T03:34:13.552592+0000","last_clean":"2023-10-06T03:34:13.552592+0000","last_became_active":"2023-10-06T03:34:12.532517+0000","last_became_peered":"2023-10-06T03:34:12.532517+0000","last_unstale":"2023-10-06T03:34:13.552592+0000","last_undegraded":"2023-10-06T03:34:13.552592+0000","last_fullsized":"2023-10-06T03:34:13.552592+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:04.840518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.549683+0000","last_change":"2023-10-06T03:34:13.546209+0000","last_active":"2023-10-06T03:34:13.549683+0000","last_peered":"2023-10-06T03:34:13.549683+0000","last_clean":"2023-10-06T03:34:13.549683+0000","last_became_active":"2023-10-06T03:34:13.545940+0000","last_became_peered":"2023-10-06T03:34:13.545940+0000","last_unstale":"2023-10-06T03:34:13.549683+0000","last_undegraded":"2023-10-06T03:34:13.549683+0000","last_fullsized":"2023-10-06T03:34:13.549683+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:44.937279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.528389+0000","last_change":"2023-10-06T03:34:12.527549+0000","last_active":"2023-10-06T03:34:13.528389+0000","last_peered":"2023-10-06T03:34:13.528389+0000","last_clean":"2023-10-06T03:34:13.528389+0000","last_became_active":"2023-10-06T03:34:12.527236+0000","last_became_peered":"2023-10-06T03:34:12.527236+0000","last_unstale":"2023-10-06T03:34:13.528389+0000","last_undegraded":"2023-10-06T03:34:13.528389+0000","last_fullsized":"2023-10-06T03:34:13.528389+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:43.578807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.542212+0000","last_change":"2023-10-06T03:34:13.535907+0000","last_active":"2023-10-06T03:34:13.542212+0000","last_peered":"2023-10-06T03:34:13.542212+0000","last_clean":"2023-10-06T03:34:13.542212+0000","last_became_active":"2023-10-06T03:34:13.535054+0000","last_became_peered":"2023-10-06T03:34:13.535054+0000","last_unstale":"2023-10-06T03:34:13.542212+0000","last_undegraded":"2023-10-06T03:34:13.542212+0000","last_fullsized":"2023-10-06T03:34:13.542212+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:15.558498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.523244+0000","last_change":"2023-10-06T03:34:12.516603+0000","last_active":"2023-10-06T03:34:13.523244+0000","last_peered":"2023-10-06T03:34:13.523244+0000","last_clean":"2023-10-06T03:34:13.523244+0000","last_became_active":"2023-10-06T03:34:12.516217+0000","last_became_peered":"2023-10-06T03:34:12.516217+0000","last_unstale":"2023-10-06T03:34:13.523244+0000","last_undegraded":"2023-10-06T03:34:13.523244+0000","last_fullsized":"2023-10-06T03:34:13.523244+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:16.340508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544831+0000","last_change":"2023-10-06T03:34:13.538400+0000","last_active":"2023-10-06T03:34:13.544831+0000","last_peered":"2023-10-06T03:34:13.544831+0000","last_clean":"2023-10-06T03:34:13.544831+0000","last_became_active":"2023-10-06T03:34:13.537683+0000","last_became_peered":"2023-10-06T03:34:13.537683+0000","last_unstale":"2023-10-06T03:34:13.544831+0000","last_undegraded":"2023-10-06T03:34:13.544831+0000","last_fullsized":"2023-10-06T03:34:13.544831+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:06.270734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.526789+0000","last_change":"2023-10-06T03:34:12.523775+0000","last_active":"2023-10-06T03:34:13.526789+0000","last_peered":"2023-10-06T03:34:13.526789+0000","last_clean":"2023-10-06T03:34:13.526789+0000","last_became_active":"2023-10-06T03:34:12.523512+0000","last_became_peered":"2023-10-06T03:34:12.523512+0000","last_unstale":"2023-10-06T03:34:13.526789+0000","last_undegraded":"2023-10-06T03:34:13.526789+0000","last_fullsized":"2023-10-06T03:34:13.526789+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:21.579094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543961+0000","last_change":"2023-10-06T03:34:13.541278+0000","last_active":"2023-10-06T03:34:13.543961+0000","last_peered":"2023-10-06T03:34:13.543961+0000","last_clean":"2023-10-06T03:34:13.543961+0000","last_became_active":"2023-10-06T03:34:13.541008+0000","last_became_peered":"2023-10-06T03:34:13.541008+0000","last_unstale":"2023-10-06T03:34:13.543961+0000","last_undegraded":"2023-10-06T03:34:13.543961+0000","last_fullsized":"2023-10-06T03:34:13.543961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:38.708108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539800+0000","last_change":"2023-10-06T03:34:12.528843+0000","last_active":"2023-10-06T03:34:13.539800+0000","last_peered":"2023-10-06T03:34:13.539800+0000","last_clean":"2023-10-06T03:34:13.539800+0000","last_became_active":"2023-10-06T03:34:12.528575+0000","last_became_peered":"2023-10-06T03:34:12.528575+0000","last_unstale":"2023-10-06T03:34:13.539800+0000","last_undegraded":"2023-10-06T03:34:13.539800+0000","last_fullsized":"2023-10-06T03:34:13.539800+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:07.994934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540805+0000","last_change":"2023-10-06T03:34:13.535849+0000","last_active":"2023-10-06T03:34:13.540805+0000","last_peered":"2023-10-06T03:34:13.540805+0000","last_clean":"2023-10-06T03:34:13.540805+0000","last_became_active":"2023-10-06T03:34:13.535582+0000","last_became_peered":"2023-10-06T03:34:13.535582+0000","last_unstale":"2023-10-06T03:34:13.540805+0000","last_undegraded":"2023-10-06T03:34:13.540805+0000","last_fullsized":"2023-10-06T03:34:13.540805+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:53.934064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"31'3","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560103+0000","last_change":"2023-10-06T03:34:12.529757+0000","last_active":"2023-10-06T03:34:13.560103+0000","last_peered":"2023-10-06T03:34:13.560103+0000","last_clean":"2023-10-06T03:34:13.560103+0000","last_became_active":"2023-10-06T03:34:12.529371+0000","last_became_peered":"2023-10-06T03:34:12.529371+0000","last_unstale":"2023-10-06T03:34:13.560103+0000","last_undegraded":"2023-10-06T03:34:13.560103+0000","last_fullsized":"2023-10-06T03:34:13.560103+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:17.340207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540076+0000","last_change":"2023-10-06T03:34:13.534519+0000","last_active":"2023-10-06T03:34:13.540076+0000","last_peered":"2023-10-06T03:34:13.540076+0000","last_clean":"2023-10-06T03:34:13.540076+0000","last_became_active":"2023-10-06T03:34:13.534278+0000","last_became_peered":"2023-10-06T03:34:13.534278+0000","last_unstale":"2023-10-06T03:34:13.540076+0000","last_undegraded":"2023-10-06T03:34:13.540076+0000","last_fullsized":"2023-10-06T03:34:13.540076+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:51.583498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"31'6","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.561812+0000","last_change":"2023-10-06T03:34:12.528656+0000","last_active":"2023-10-06T03:34:13.561812+0000","last_peered":"2023-10-06T03:34:13.561812+0000","last_clean":"2023-10-06T03:34:13.561812+0000","last_became_active":"2023-10-06T03:34:12.528361+0000","last_became_peered":"2023-10-06T03:34:12.528361+0000","last_unstale":"2023-10-06T03:34:13.561812+0000","last_undegraded":"2023-10-06T03:34:13.561812+0000","last_fullsized":"2023-10-06T03:34:13.561812+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:31.070647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557081+0000","last_change":"2023-10-06T03:34:13.555313+0000","last_active":"2023-10-06T03:34:13.557081+0000","last_peered":"2023-10-06T03:34:13.557081+0000","last_clean":"2023-10-06T03:34:13.557081+0000","last_became_active":"2023-10-06T03:34:13.555027+0000","last_became_peered":"2023-10-06T03:34:13.555027+0000","last_unstale":"2023-10-06T03:34:13.557081+0000","last_undegraded":"2023-10-06T03:34:13.557081+0000","last_fullsized":"2023-10-06T03:34:13.557081+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:01.442206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550514+0000","last_change":"2023-10-06T03:34:12.523998+0000","last_active":"2023-10-06T03:34:13.550514+0000","last_peered":"2023-10-06T03:34:13.550514+0000","last_clean":"2023-10-06T03:34:13.550514+0000","last_became_active":"2023-10-06T03:34:12.523688+0000","last_became_peered":"2023-10-06T03:34:12.523688+0000","last_unstale":"2023-10-06T03:34:13.550514+0000","last_undegraded":"2023-10-06T03:34:13.550514+0000","last_fullsized":"2023-10-06T03:34:13.550514+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:52.611432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.542751+0000","last_change":"2023-10-06T03:34:13.540598+0000","last_active":"2023-10-06T03:34:13.542751+0000","last_peered":"2023-10-06T03:34:13.542751+0000","last_clean":"2023-10-06T03:34:13.542751+0000","last_became_active":"2023-10-06T03:34:13.540350+0000","last_became_peered":"2023-10-06T03:34:13.540350+0000","last_unstale":"2023-10-06T03:34:13.542751+0000","last_undegraded":"2023-10-06T03:34:13.542751+0000","last_fullsized":"2023-10-06T03:34:13.542751+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:03.345245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.527766+0000","last_change":"2023-10-06T03:34:12.530728+0000","last_active":"2023-10-06T03:34:13.527766+0000","last_peered":"2023-10-06T03:34:13.527766+0000","last_clean":"2023-10-06T03:34:13.527766+0000","last_became_active":"2023-10-06T03:34:12.530458+0000","last_became_peered":"2023-10-06T03:34:12.530458+0000","last_unstale":"2023-10-06T03:34:13.527766+0000","last_undegraded":"2023-10-06T03:34:13.527766+0000","last_fullsized":"2023-10-06T03:34:13.527766+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:09.423072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550408+0000","last_change":"2023-10-06T03:34:13.547951+0000","last_active":"2023-10-06T03:34:13.550408+0000","last_peered":"2023-10-06T03:34:13.550408+0000","last_clean":"2023-10-06T03:34:13.550408+0000","last_became_active":"2023-10-06T03:34:13.547567+0000","last_became_peered":"2023-10-06T03:34:13.547567+0000","last_unstale":"2023-10-06T03:34:13.550408+0000","last_undegraded":"2023-10-06T03:34:13.550408+0000","last_fullsized":"2023-10-06T03:34:13.550408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:01.766184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.521120+0000","last_change":"2023-10-06T03:34:12.517112+0000","last_active":"2023-10-06T03:34:13.521120+0000","last_peered":"2023-10-06T03:34:13.521120+0000","last_clean":"2023-10-06T03:34:13.521120+0000","last_became_active":"2023-10-06T03:34:12.516831+0000","last_became_peered":"2023-10-06T03:34:12.516831+0000","last_unstale":"2023-10-06T03:34:13.521120+0000","last_undegraded":"2023-10-06T03:34:13.521120+0000","last_fullsized":"2023-10-06T03:34:13.521120+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:48.752619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.542193+0000","last_change":"2023-10-06T03:34:13.535453+0000","last_active":"2023-10-06T03:34:13.542193+0000","last_peered":"2023-10-06T03:34:13.542193+0000","last_clean":"2023-10-06T03:34:13.542193+0000","last_became_active":"2023-10-06T03:34:13.534677+0000","last_became_peered":"2023-10-06T03:34:13.534677+0000","last_unstale":"2023-10-06T03:34:13.542193+0000","last_undegraded":"2023-10-06T03:34:13.542193+0000","last_fullsized":"2023-10-06T03:34:13.542193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:19.240735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"31'8","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.556402+0000","last_change":"2023-10-06T03:34:12.532903+0000","last_active":"2023-10-06T03:34:13.556402+0000","last_peered":"2023-10-06T03:34:13.556402+0000","last_clean":"2023-10-06T03:34:13.556402+0000","last_became_active":"2023-10-06T03:34:12.532576+0000","last_became_peered":"2023-10-06T03:34:12.532576+0000","last_unstale":"2023-10-06T03:34:13.556402+0000","last_undegraded":"2023-10-06T03:34:13.556402+0000","last_fullsized":"2023-10-06T03:34:13.556402+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539800+0000","last_change":"2023-10-06T03:34:13.531821+0000","last_active":"2023-10-06T03:34:13.539800+0000","last_peered":"2023-10-06T03:34:13.539800+0000","last_clean":"2023-10-06T03:34:13.539800+0000","last_became_active":"2023-10-06T03:34:13.531527+0000","last_became_peered":"2023-10-06T03:34:13.531527+0000","last_unstale":"2023-10-06T03:34:13.539800+0000","last_undegraded":"2023-10-06T03:34:13.539800+0000","last_fullsized":"2023-10-06T03:34:13.539800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:05.808082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560016+0000","last_change":"2023-10-06T03:34:12.532478+0000","last_active":"2023-10-06T03:34:13.560016+0000","last_peered":"2023-10-06T03:34:13.560016+0000","last_clean":"2023-10-06T03:34:13.560016+0000","last_became_active":"2023-10-06T03:34:12.532204+0000","last_became_peered":"2023-10-06T03:34:12.532204+0000","last_unstale":"2023-10-06T03:34:13.560016+0000","last_undegraded":"2023-10-06T03:34:13.560016+0000","last_fullsized":"2023-10-06T03:34:13.560016+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560141+0000","last_change":"2023-10-06T03:34:13.556710+0000","last_active":"2023-10-06T03:34:13.560141+0000","last_peered":"2023-10-06T03:34:13.560141+0000","last_clean":"2023-10-06T03:34:13.560141+0000","last_became_active":"2023-10-06T03:34:13.554487+0000","last_became_peered":"2023-10-06T03:34:13.554487+0000","last_unstale":"2023-10-06T03:34:13.560141+0000","last_undegraded":"2023-10-06T03:34:13.560141+0000","last_fullsized":"2023-10-06T03:34:13.560141+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:25.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560376+0000","last_change":"2023-10-06T03:34:12.527219+0000","last_active":"2023-10-06T03:34:13.560376+0000","last_peered":"2023-10-06T03:34:13.560376+0000","last_clean":"2023-10-06T03:34:13.560376+0000","last_became_active":"2023-10-06T03:34:12.526985+0000","last_became_peered":"2023-10-06T03:34:12.526985+0000","last_unstale":"2023-10-06T03:34:13.560376+0000","last_undegraded":"2023-10-06T03:34:13.560376+0000","last_fullsized":"2023-10-06T03:34:13.560376+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:29.481412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543498+0000","last_change":"2023-10-06T03:34:13.537130+0000","last_active":"2023-10-06T03:34:13.543498+0000","last_peered":"2023-10-06T03:34:13.543498+0000","last_clean":"2023-10-06T03:34:13.543498+0000","last_became_active":"2023-10-06T03:34:13.536871+0000","last_became_peered":"2023-10-06T03:34:13.536871+0000","last_unstale":"2023-10-06T03:34:13.543498+0000","last_undegraded":"2023-10-06T03:34:13.543498+0000","last_fullsized":"2023-10-06T03:34:13.543498+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:21.602441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.553427+0000","last_change":"2023-10-06T03:34:12.524489+0000","last_active":"2023-10-06T03:34:13.553427+0000","last_peered":"2023-10-06T03:34:13.553427+0000","last_clean":"2023-10-06T03:34:13.553427+0000","last_became_active":"2023-10-06T03:34:12.524226+0000","last_became_peered":"2023-10-06T03:34:12.524226+0000","last_unstale":"2023-10-06T03:34:13.553427+0000","last_undegraded":"2023-10-06T03:34:13.553427+0000","last_fullsized":"2023-10-06T03:34:13.553427+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:37.934361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.536949+0000","last_change":"2023-10-06T03:34:13.531913+0000","last_active":"2023-10-06T03:34:13.536949+0000","last_peered":"2023-10-06T03:34:13.536949+0000","last_clean":"2023-10-06T03:34:13.536949+0000","last_became_active":"2023-10-06T03:34:13.531611+0000","last_became_peered":"2023-10-06T03:34:13.531611+0000","last_unstale":"2023-10-06T03:34:13.536949+0000","last_undegraded":"2023-10-06T03:34:13.536949+0000","last_fullsized":"2023-10-06T03:34:13.536949+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:44.589878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.528863+0000","last_change":"2023-10-06T03:34:12.532410+0000","last_active":"2023-10-06T03:34:13.528863+0000","last_peered":"2023-10-06T03:34:13.528863+0000","last_clean":"2023-10-06T03:34:13.528863+0000","last_became_active":"2023-10-06T03:34:12.532154+0000","last_became_peered":"2023-10-06T03:34:12.532154+0000","last_unstale":"2023-10-06T03:34:13.528863+0000","last_undegraded":"2023-10-06T03:34:13.528863+0000","last_fullsized":"2023-10-06T03:34:13.528863+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:05.259663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.549613+0000","last_change":"2023-10-06T03:34:13.546456+0000","last_active":"2023-10-06T03:34:13.549613+0000","last_peered":"2023-10-06T03:34:13.549613+0000","last_clean":"2023-10-06T03:34:13.549613+0000","last_became_active":"2023-10-06T03:34:13.546118+0000","last_became_peered":"2023-10-06T03:34:13.546118+0000","last_unstale":"2023-10-06T03:34:13.549613+0000","last_undegraded":"2023-10-06T03:34:13.549613+0000","last_fullsized":"2023-10-06T03:34:13.549613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:30.492483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550619+0000","last_change":"2023-10-06T03:34:12.519345+0000","last_active":"2023-10-06T03:34:13.550619+0000","last_peered":"2023-10-06T03:34:13.550619+0000","last_clean":"2023-10-06T03:34:13.550619+0000","last_became_active":"2023-10-06T03:34:12.519053+0000","last_became_peered":"2023-10-06T03:34:12.519053+0000","last_unstale":"2023-10-06T03:34:13.550619+0000","last_undegraded":"2023-10-06T03:34:13.550619+0000","last_fullsized":"2023-10-06T03:34:13.550619+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:34.904232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540779+0000","last_change":"2023-10-06T03:34:13.535161+0000","last_active":"2023-10-06T03:34:13.540779+0000","last_peered":"2023-10-06T03:34:13.540779+0000","last_clean":"2023-10-06T03:34:13.540779+0000","last_became_active":"2023-10-06T03:34:13.534909+0000","last_became_peered":"2023-10-06T03:34:13.534909+0000","last_unstale":"2023-10-06T03:34:13.540779+0000","last_undegraded":"2023-10-06T03:34:13.540779+0000","last_fullsized":"2023-10-06T03:34:13.540779+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:48.475963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.524923+0000","last_change":"2023-10-06T03:34:12.522297+0000","last_active":"2023-10-06T03:34:13.524923+0000","last_peered":"2023-10-06T03:34:13.524923+0000","last_clean":"2023-10-06T03:34:13.524923+0000","last_became_active":"2023-10-06T03:34:12.522006+0000","last_became_peered":"2023-10-06T03:34:12.522006+0000","last_unstale":"2023-10-06T03:34:13.524923+0000","last_undegraded":"2023-10-06T03:34:13.524923+0000","last_fullsized":"2023-10-06T03:34:13.524923+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:39.716255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559638+0000","last_change":"2023-10-06T03:34:13.547565+0000","last_active":"2023-10-06T03:34:13.559638+0000","last_peered":"2023-10-06T03:34:13.559638+0000","last_clean":"2023-10-06T03:34:13.559638+0000","last_became_active":"2023-10-06T03:34:13.547290+0000","last_became_peered":"2023-10-06T03:34:13.547290+0000","last_unstale":"2023-10-06T03:34:13.559638+0000","last_undegraded":"2023-10-06T03:34:13.559638+0000","last_fullsized":"2023-10-06T03:34:13.559638+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:05.808082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.531326+0000","last_change":"2023-10-06T03:34:12.527419+0000","last_active":"2023-10-06T03:34:13.531326+0000","last_peered":"2023-10-06T03:34:13.531326+0000","last_clean":"2023-10-06T03:34:13.531326+0000","last_became_active":"2023-10-06T03:34:12.527141+0000","last_became_peered":"2023-10-06T03:34:12.527141+0000","last_unstale":"2023-10-06T03:34:13.531326+0000","last_undegraded":"2023-10-06T03:34:13.531326+0000","last_fullsized":"2023-10-06T03:34:13.531326+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:17.340207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.542230+0000","last_change":"2023-10-06T03:34:13.536079+0000","last_active":"2023-10-06T03:34:13.542230+0000","last_peered":"2023-10-06T03:34:13.542230+0000","last_clean":"2023-10-06T03:34:13.542230+0000","last_became_active":"2023-10-06T03:34:13.535653+0000","last_became_peered":"2023-10-06T03:34:13.535653+0000","last_unstale":"2023-10-06T03:34:13.542230+0000","last_undegraded":"2023-10-06T03:34:13.542230+0000","last_fullsized":"2023-10-06T03:34:13.542230+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:25.840403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.525414+0000","last_change":"2023-10-06T03:34:12.521014+0000","last_active":"2023-10-06T03:34:13.525414+0000","last_peered":"2023-10-06T03:34:13.525414+0000","last_clean":"2023-10-06T03:34:13.525414+0000","last_became_active":"2023-10-06T03:34:12.520088+0000","last_became_peered":"2023-10-06T03:34:12.520088+0000","last_unstale":"2023-10-06T03:34:13.525414+0000","last_undegraded":"2023-10-06T03:34:13.525414+0000","last_fullsized":"2023-10-06T03:34:13.525414+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:34.651158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539620+0000","last_change":"2023-10-06T03:34:13.535691+0000","last_active":"2023-10-06T03:34:13.539620+0000","last_peered":"2023-10-06T03:34:13.539620+0000","last_clean":"2023-10-06T03:34:13.539620+0000","last_became_active":"2023-10-06T03:34:13.535365+0000","last_became_peered":"2023-10-06T03:34:13.535365+0000","last_unstale":"2023-10-06T03:34:13.539620+0000","last_undegraded":"2023-10-06T03:34:13.539620+0000","last_fullsized":"2023-10-06T03:34:13.539620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:08.739478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.523084+0000","last_change":"2023-10-06T03:34:12.525259+0000","last_active":"2023-10-06T03:34:13.523084+0000","last_peered":"2023-10-06T03:34:13.523084+0000","last_clean":"2023-10-06T03:34:13.523084+0000","last_became_active":"2023-10-06T03:34:12.524902+0000","last_became_peered":"2023-10-06T03:34:12.524902+0000","last_unstale":"2023-10-06T03:34:13.523084+0000","last_undegraded":"2023-10-06T03:34:13.523084+0000","last_fullsized":"2023-10-06T03:34:13.523084+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:37.697037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.524780+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:34:13.524780+0000","last_peered":"2023-10-06T03:34:13.524780+0000","last_clean":"2023-10-06T03:34:13.524780+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:34:13.524780+0000","last_undegraded":"2023-10-06T03:34:13.524780+0000","last_fullsized":"2023-10-06T03:34:13.524780+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.545347+0000","last_change":"2023-10-06T03:34:13.543841+0000","last_active":"2023-10-06T03:34:13.545347+0000","last_peered":"2023-10-06T03:34:13.545347+0000","last_clean":"2023-10-06T03:34:13.545347+0000","last_became_active":"2023-10-06T03:34:13.543580+0000","last_became_peered":"2023-10-06T03:34:13.543580+0000","last_unstale":"2023-10-06T03:34:13.545347+0000","last_undegraded":"2023-10-06T03:34:13.545347+0000","last_fullsized":"2023-10-06T03:34:13.545347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:53.934064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.524324+0000","last_change":"2023-10-06T03:34:12.525282+0000","last_active":"2023-10-06T03:34:13.524324+0000","last_peered":"2023-10-06T03:34:13.524324+0000","last_clean":"2023-10-06T03:34:13.524324+0000","last_became_active":"2023-10-06T03:34:12.525020+0000","last_became_peered":"2023-10-06T03:34:12.525020+0000","last_unstale":"2023-10-06T03:34:13.524324+0000","last_undegraded":"2023-10-06T03:34:13.524324+0000","last_fullsized":"2023-10-06T03:34:13.524324+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:03.495179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.524356+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:34:13.524356+0000","last_peered":"2023-10-06T03:34:13.524356+0000","last_clean":"2023-10-06T03:34:13.524356+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:34:13.524356+0000","last_undegraded":"2023-10-06T03:34:13.524356+0000","last_fullsized":"2023-10-06T03:34:13.524356+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.558168+0000","last_change":"2023-10-06T03:34:13.555991+0000","last_active":"2023-10-06T03:34:13.558168+0000","last_peered":"2023-10-06T03:34:13.558168+0000","last_clean":"2023-10-06T03:34:13.558168+0000","last_became_active":"2023-10-06T03:34:13.555086+0000","last_became_peered":"2023-10-06T03:34:13.555086+0000","last_unstale":"2023-10-06T03:34:13.558168+0000","last_undegraded":"2023-10-06T03:34:13.558168+0000","last_fullsized":"2023-10-06T03:34:13.558168+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:27.486440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559924+0000","last_change":"2023-10-06T03:34:12.529984+0000","last_active":"2023-10-06T03:34:13.559924+0000","last_peered":"2023-10-06T03:34:13.559924+0000","last_clean":"2023-10-06T03:34:13.559924+0000","last_became_active":"2023-10-06T03:34:12.529440+0000","last_became_peered":"2023-10-06T03:34:12.529440+0000","last_unstale":"2023-10-06T03:34:13.559924+0000","last_undegraded":"2023-10-06T03:34:13.559924+0000","last_fullsized":"2023-10-06T03:34:13.559924+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:04.840518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.527887+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:34:13.527887+0000","last_peered":"2023-10-06T03:34:13.527887+0000","last_clean":"2023-10-06T03:34:13.527887+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:34:13.527887+0000","last_undegraded":"2023-10-06T03:34:13.527887+0000","last_fullsized":"2023-10-06T03:34:13.527887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.531546+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:34:13.531546+0000","last_peered":"2023-10-06T03:34:13.531546+0000","last_clean":"2023-10-06T03:34:13.531546+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:34:13.531546+0000","last_undegraded":"2023-10-06T03:34:13.531546+0000","last_fullsized":"2023-10-06T03:34:13.531546+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543979+0000","last_change":"2023-10-06T03:34:13.541987+0000","last_active":"2023-10-06T03:34:13.543979+0000","last_peered":"2023-10-06T03:34:13.543979+0000","last_clean":"2023-10-06T03:34:13.543979+0000","last_became_active":"2023-10-06T03:34:13.541710+0000","last_became_peered":"2023-10-06T03:34:13.541710+0000","last_unstale":"2023-10-06T03:34:13.543979+0000","last_undegraded":"2023-10-06T03:34:13.543979+0000","last_fullsized":"2023-10-06T03:34:13.543979+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.785861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.525061+0000","last_change":"2023-10-06T03:34:12.526945+0000","last_active":"2023-10-06T03:34:13.525061+0000","last_peered":"2023-10-06T03:34:13.525061+0000","last_clean":"2023-10-06T03:34:13.525061+0000","last_became_active":"2023-10-06T03:34:12.526661+0000","last_became_peered":"2023-10-06T03:34:12.526661+0000","last_unstale":"2023-10-06T03:34:13.525061+0000","last_undegraded":"2023-10-06T03:34:13.525061+0000","last_fullsized":"2023-10-06T03:34:13.525061+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:36.885264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.530135+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:34:13.530135+0000","last_peered":"2023-10-06T03:34:13.530135+0000","last_clean":"2023-10-06T03:34:13.530135+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:34:13.530135+0000","last_undegraded":"2023-10-06T03:34:13.530135+0000","last_fullsized":"2023-10-06T03:34:13.530135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559864+0000","last_change":"2023-10-06T03:34:13.553850+0000","last_active":"2023-10-06T03:34:13.559864+0000","last_peered":"2023-10-06T03:34:13.559864+0000","last_clean":"2023-10-06T03:34:13.559864+0000","last_became_active":"2023-10-06T03:34:13.553534+0000","last_became_peered":"2023-10-06T03:34:13.553534+0000","last_unstale":"2023-10-06T03:34:13.559864+0000","last_undegraded":"2023-10-06T03:34:13.559864+0000","last_fullsized":"2023-10-06T03:34:13.559864+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.785861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.530519+0000","last_change":"2023-10-06T03:34:12.529818+0000","last_active":"2023-10-06T03:34:13.530519+0000","last_peered":"2023-10-06T03:34:13.530519+0000","last_clean":"2023-10-06T03:34:13.530519+0000","last_became_active":"2023-10-06T03:34:12.529370+0000","last_became_peered":"2023-10-06T03:34:12.529370+0000","last_unstale":"2023-10-06T03:34:13.530519+0000","last_undegraded":"2023-10-06T03:34:13.530519+0000","last_fullsized":"2023-10-06T03:34:13.530519+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:54.385812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.531571+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:34:13.531571+0000","last_peered":"2023-10-06T03:34:13.531571+0000","last_clean":"2023-10-06T03:34:13.531571+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:34:13.531571+0000","last_undegraded":"2023-10-06T03:34:13.531571+0000","last_fullsized":"2023-10-06T03:34:13.531571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541671+0000","last_change":"2023-10-06T03:34:13.534808+0000","last_active":"2023-10-06T03:34:13.541671+0000","last_peered":"2023-10-06T03:34:13.541671+0000","last_clean":"2023-10-06T03:34:13.541671+0000","last_became_active":"2023-10-06T03:34:13.534549+0000","last_became_peered":"2023-10-06T03:34:13.534549+0000","last_unstale":"2023-10-06T03:34:13.541671+0000","last_undegraded":"2023-10-06T03:34:13.541671+0000","last_fullsized":"2023-10-06T03:34:13.541671+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:50.641614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.522197+0000","last_change":"2023-10-06T03:34:12.516230+0000","last_active":"2023-10-06T03:34:13.522197+0000","last_peered":"2023-10-06T03:34:13.522197+0000","last_clean":"2023-10-06T03:34:13.522197+0000","last_became_active":"2023-10-06T03:34:12.515968+0000","last_became_peered":"2023-10-06T03:34:12.515968+0000","last_unstale":"2023-10-06T03:34:13.522197+0000","last_undegraded":"2023-10-06T03:34:13.522197+0000","last_fullsized":"2023-10-06T03:34:13.522197+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.529216+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:34:13.529216+0000","last_peered":"2023-10-06T03:34:13.529216+0000","last_clean":"2023-10-06T03:34:13.529216+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:34:13.529216+0000","last_undegraded":"2023-10-06T03:34:13.529216+0000","last_fullsized":"2023-10-06T03:34:13.529216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.546934+0000","last_change":"2023-10-06T03:34:13.540397+0000","last_active":"2023-10-06T03:34:13.546934+0000","last_peered":"2023-10-06T03:34:13.546934+0000","last_clean":"2023-10-06T03:34:13.546934+0000","last_became_active":"2023-10-06T03:34:13.540142+0000","last_became_peered":"2023-10-06T03:34:13.540142+0000","last_unstale":"2023-10-06T03:34:13.546934+0000","last_undegraded":"2023-10-06T03:34:13.546934+0000","last_fullsized":"2023-10-06T03:34:13.546934+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:32.081718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.521373+0000","last_change":"2023-10-06T03:34:12.519229+0000","last_active":"2023-10-06T03:34:13.521373+0000","last_peered":"2023-10-06T03:34:13.521373+0000","last_clean":"2023-10-06T03:34:13.521373+0000","last_became_active":"2023-10-06T03:34:12.518975+0000","last_became_peered":"2023-10-06T03:34:12.518975+0000","last_unstale":"2023-10-06T03:34:13.521373+0000","last_undegraded":"2023-10-06T03:34:13.521373+0000","last_fullsized":"2023-10-06T03:34:13.521373+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:24.627602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.521342+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:34:13.521342+0000","last_peered":"2023-10-06T03:34:13.521342+0000","last_clean":"2023-10-06T03:34:13.521342+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:34:13.521342+0000","last_undegraded":"2023-10-06T03:34:13.521342+0000","last_fullsized":"2023-10-06T03:34:13.521342+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.537670+0000","last_change":"2023-10-06T03:34:13.532662+0000","last_active":"2023-10-06T03:34:13.537670+0000","last_peered":"2023-10-06T03:34:13.537670+0000","last_clean":"2023-10-06T03:34:13.537670+0000","last_became_active":"2023-10-06T03:34:13.532351+0000","last_became_peered":"2023-10-06T03:34:13.532351+0000","last_unstale":"2023-10-06T03:34:13.537670+0000","last_undegraded":"2023-10-06T03:34:13.537670+0000","last_fullsized":"2023-10-06T03:34:13.537670+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:23.077437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.523003+0000","last_change":"2023-10-06T03:34:12.514751+0000","last_active":"2023-10-06T03:34:13.523003+0000","last_peered":"2023-10-06T03:34:13.523003+0000","last_clean":"2023-10-06T03:34:13.523003+0000","last_became_active":"2023-10-06T03:34:12.514468+0000","last_became_peered":"2023-10-06T03:34:12.514468+0000","last_unstale":"2023-10-06T03:34:13.523003+0000","last_undegraded":"2023-10-06T03:34:13.523003+0000","last_fullsized":"2023-10-06T03:34:13.523003+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:07.994934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.521805+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:34:13.521805+0000","last_peered":"2023-10-06T03:34:13.521805+0000","last_clean":"2023-10-06T03:34:13.521805+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:34:13.521805+0000","last_undegraded":"2023-10-06T03:34:13.521805+0000","last_fullsized":"2023-10-06T03:34:13.521805+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073608,"statfs":{"total":96636764160,"available":96331374592,"internally_reserved":0,"allocated":966656,"data_stored":737317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59099999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.622}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.746}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":146875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.77200000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.625}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.68300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302320,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069520,"statfs":{"total":96636764160,"available":96327188480,"internally_reserved":0,"allocated":958464,"data_stored":736779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.45500000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.42799999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.40799999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.56299999999999994}]},{"osd":5,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.54900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301736,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070104,"statfs":{"total":96636764160,"available":96327786496,"internally_reserved":0,"allocated":360448,"data_stored":145377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.439},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":0.85899999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.71699999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.93700000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301736,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070104,"statfs":{"total":96636764160,"available":96327786496,"internally_reserved":0,"allocated":360448,"data_stored":145413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.442}]},{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.5}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.91000000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.94199999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.98799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301752,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070088,"statfs":{"total":96636764160,"available":96327770112,"internally_reserved":0,"allocated":376832,"data_stored":147891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.33100000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.442}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.51800000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.80000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":368640,"data_stored":146897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.433}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.45200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.55500000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.748},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.77900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":145455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.877}]},{"osd":2,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.78000000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.751}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":1.01}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:34:22.853 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:34:23.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.050+0000 7f7a4f59e700 1 Processor -- start 2023-10-06T03:34:23.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.050+0000 7f7a4f59e700 1 -- start start 2023-10-06T03:34:23.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.051+0000 7f7a4f59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7a50156af0 0x7f7a500c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:23.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.051+0000 7f7a4f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a500c7610 0x7f7a500ca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:23.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.051+0000 7f7a4f59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a500c7a30 con 0x7f7a50155720 2023-10-06T03:34:23.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.051+0000 7f7a4f59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5006dd20 con 0x7f7a50156af0 2023-10-06T03:34:23.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.051+0000 7f7a4f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5006e000 con 0x7f7a500c7610 2023-10-06T03:34:23.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.052+0000 7f7a4ed9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a500c7610 0x7f7a500ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:23.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.052+0000 7f7a4dd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7a50156af0 0x7f7a500c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:23.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.052+0000 7f7a4ed9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a500c7610 0x7f7a500ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:45742/0 (socket says 172.21.15.55:45742) 2023-10-06T03:34:23.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.052+0000 7f7a4ed9d700 1 -- 172.21.15.55:0/1162864254 learned_addr learned my addr 172.21.15.55:0/1162864254 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:23.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.052+0000 7f7a4dd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7a50156af0 0x7f7a500c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:32904/0 (socket says 172.21.15.55:32904) 2023-10-06T03:34:23.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.052+0000 7f7a4e59c700 1 --1- 172.21.15.55:0/1162864254 >> v1:172.21.15.163:6789/0 conn(0x7f7a50155720 0x7f7a5006f120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:42152/0 (socket says 172.21.15.55:42152) 2023-10-06T03:34:23.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.052+0000 7f7a4ed9d700 1 -- 172.21.15.55:0/1162864254 >> v2:172.21.15.163:3300/0 conn(0x7f7a50156af0 msgr2=0x7f7a500c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:23.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.052+0000 7f7a4ed9d700 1 --2- 172.21.15.55:0/1162864254 >> v2:172.21.15.163:3300/0 conn(0x7f7a50156af0 0x7f7a500c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.052+0000 7f7a4ed9d700 1 -- 172.21.15.55:0/1162864254 >> v1:172.21.15.163:6789/0 conn(0x7f7a50155720 legacy=0x7f7a5006f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:23.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.052+0000 7f7a4ed9d700 1 -- 172.21.15.55:0/1162864254 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5006e2e0 con 0x7f7a500c7610 2023-10-06T03:34:23.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.052+0000 7f7a4dd9b700 1 --2- 172.21.15.55:0/1162864254 >> v2:172.21.15.163:3300/0 conn(0x7f7a50156af0 0x7f7a500c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:34:23.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.052+0000 7f7a4ed9d700 1 --2- 172.21.15.55:0/1162864254 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a500c7610 0x7f7a500ca630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009f20 tx=0x7f7a48009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7664568211474943 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:23.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.053+0000 7f7a4d59a700 1 -- 172.21.15.55:0/1162864254 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4800e070 con 0x7f7a500c7610 2023-10-06T03:34:23.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.053+0000 7f7a4d59a700 1 -- 172.21.15.55:0/1162864254 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4800ba80 con 0x7f7a500c7610 2023-10-06T03:34:23.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.053+0000 7f7a4d59a700 1 -- 172.21.15.55:0/1162864254 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a480092e0 con 0x7f7a500c7610 2023-10-06T03:34:23.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.053+0000 7f7a4f59e700 1 -- 172.21.15.55:0/1162864254 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a500c7610 msgr2=0x7f7a500ca630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:23.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.053+0000 7f7a4f59e700 1 --2- 172.21.15.55:0/1162864254 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a500c7610 0x7f7a500ca630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009f20 tx=0x7f7a48009840 comp rx=0 tx=0).stop 2023-10-06T03:34:23.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.053+0000 7f7a4f59e700 1 -- 172.21.15.55:0/1162864254 shutdown_connections 2023-10-06T03:34:23.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.053+0000 7f7a4f59e700 1 --2- 172.21.15.55:0/1162864254 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a500c7610 0x7f7a500ca630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.053+0000 7f7a4f59e700 1 --2- 172.21.15.55:0/1162864254 >> v2:172.21.15.163:3300/0 conn(0x7f7a50156af0 0x7f7a500c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.053+0000 7f7a4f59e700 1 -- 172.21.15.55:0/1162864254 >> 172.21.15.55:0/1162864254 conn(0x7f7a500cc1f0 msgr2=0x7f7a500cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:23.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.054+0000 7f7a4f59e700 1 -- 172.21.15.55:0/1162864254 shutdown_connections 2023-10-06T03:34:23.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.054+0000 7f7a4f59e700 1 -- 172.21.15.55:0/1162864254 wait complete. 2023-10-06T03:34:23.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.055+0000 7f7a4f59e700 1 Processor -- start 2023-10-06T03:34:23.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.055+0000 7f7a4f59e700 1 -- start start 2023-10-06T03:34:23.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.056+0000 7f7a4f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a500c7610 0x7f7a500c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:23.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.056+0000 7f7a4f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a50155720 0x7f7a50144c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:23.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.056+0000 7f7a4f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a50156af0 0x7f7a50153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:23.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.056+0000 7f7a4f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500c35d0 con 0x7f7a50156af0 2023-10-06T03:34:23.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.056+0000 7f7a4f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500c65d0 con 0x7f7a50155720 2023-10-06T03:34:23.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.056+0000 7f7a4f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5015c6d0 con 0x7f7a500c7610 2023-10-06T03:34:23.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.056+0000 7f7a4ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a50156af0 0x7f7a50153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:23.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.056+0000 7f7a4ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a50156af0 0x7f7a50153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48596/0 (socket says 172.21.15.55:48596) 2023-10-06T03:34:23.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.056+0000 7f7a4ed9d700 1 -- 172.21.15.55:0/2149842475 learned_addr learned my addr 172.21.15.55:0/2149842475 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:23.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.056+0000 7f7a4e59c700 1 --2- 172.21.15.55:0/2149842475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a500c7610 0x7f7a500c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:23.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.057+0000 7f7a4dd9b700 1 --2- 172.21.15.55:0/2149842475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a50155720 0x7f7a50144c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:23.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.057+0000 7f7a4ed9d700 1 -- 172.21.15.55:0/2149842475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a500c7610 msgr2=0x7f7a500c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:23.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.057+0000 7f7a4ed9d700 1 --2- 172.21.15.55:0/2149842475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a500c7610 0x7f7a500c5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.057+0000 7f7a4ed9d700 1 -- 172.21.15.55:0/2149842475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a50155720 msgr2=0x7f7a50144c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:23.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.057+0000 7f7a4ed9d700 1 --2- 172.21.15.55:0/2149842475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a50155720 0x7f7a50144c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.057+0000 7f7a4ed9d700 1 -- 172.21.15.55:0/2149842475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4400b6d0 con 0x7f7a50156af0 2023-10-06T03:34:23.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.057+0000 7f7a4ed9d700 1 --2- 172.21.15.55:0/2149842475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a50156af0 0x7f7a50153990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800b6c0 tx=0x7f7a48018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:23.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.057+0000 7f7a3e7fc700 1 -- 172.21.15.55:0/2149842475 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a480032f0 con 0x7f7a50156af0 2023-10-06T03:34:23.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.057+0000 7f7a3e7fc700 1 -- 172.21.15.55:0/2149842475 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a48008d20 con 0x7f7a50156af0 2023-10-06T03:34:23.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.057+0000 7f7a3e7fc700 1 -- 172.21.15.55:0/2149842475 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48026670 con 0x7f7a50156af0 2023-10-06T03:34:23.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.057+0000 7f7a4f59e700 1 -- 172.21.15.55:0/2149842475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a48009ae0 con 0x7f7a50156af0 2023-10-06T03:34:23.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.058+0000 7f7a4f59e700 1 -- 172.21.15.55:0/2149842475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a500c5d30 con 0x7f7a50156af0 2023-10-06T03:34:23.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.059+0000 7f7a4f59e700 1 -- 172.21.15.55:0/2149842475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5015c880 con 0x7f7a50156af0 2023-10-06T03:34:23.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.059+0000 7f7a3e7fc700 1 -- 172.21.15.55:0/2149842475 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a48003040 con 0x7f7a50156af0 2023-10-06T03:34:23.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.061+0000 7f7a3e7fc700 1 --2- 172.21.15.55:0/2149842475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7a30066f30 0x7f7a300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:23.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.061+0000 7f7a3e7fc700 1 -- 172.21.15.55:0/2149842475 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7a480865e0 con 0x7f7a50156af0 2023-10-06T03:34:23.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.061+0000 7f7a4e59c700 1 --2- 172.21.15.55:0/2149842475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7a30066f30 0x7f7a300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:23.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.062+0000 7f7a4e59c700 1 --2- 172.21.15.55:0/2149842475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7a30066f30 0x7f7a300693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7a40004fd0 tx=0x7f7a40007eb0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:23.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.065+0000 7f7a3e7fc700 1 -- 172.21.15.55:0/2149842475 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a5015c880 con 0x7f7a50156af0 2023-10-06T03:34:23.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.262+0000 7f7a4f59e700 1 -- 172.21.15.55:0/2149842475 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a5015c840 con 0x7f7a30066f30 2023-10-06T03:34:23.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.272+0000 7f7a3e7fc700 1 -- 172.21.15.55:0/2149842475 <== mgr.4114 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311808 (secure 0 0 0) 0x7f7a5015c840 con 0x7f7a30066f30 2023-10-06T03:34:23.277 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:23.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.276+0000 7f7a4f59e700 1 -- 172.21.15.55:0/2149842475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7a30066f30 msgr2=0x7f7a300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:23.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.276+0000 7f7a4f59e700 1 --2- 172.21.15.55:0/2149842475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7a30066f30 0x7f7a300693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7a40004fd0 tx=0x7f7a40007eb0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.276+0000 7f7a4f59e700 1 -- 172.21.15.55:0/2149842475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a50156af0 msgr2=0x7f7a50153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:23.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.276+0000 7f7a4f59e700 1 --2- 172.21.15.55:0/2149842475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a50156af0 0x7f7a50153990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800b6c0 tx=0x7f7a48018000 comp rx=0 tx=0).stop 2023-10-06T03:34:23.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.276+0000 7f7a4f59e700 1 -- 172.21.15.55:0/2149842475 shutdown_connections 2023-10-06T03:34:23.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.276+0000 7f7a4f59e700 1 --2- 172.21.15.55:0/2149842475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a50156af0 0x7f7a50153990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.276+0000 7f7a4f59e700 1 --2- 172.21.15.55:0/2149842475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7a30066f30 0x7f7a300693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.277+0000 7f7a4f59e700 1 --2- 172.21.15.55:0/2149842475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a500c7610 0x7f7a500c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.277+0000 7f7a4f59e700 1 --2- 172.21.15.55:0/2149842475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a50155720 0x7f7a50144c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.277+0000 7f7a4f59e700 1 -- 172.21.15.55:0/2149842475 >> 172.21.15.55:0/2149842475 conn(0x7f7a500cc1f0 msgr2=0x7f7a500c11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:23.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.277+0000 7f7a4f59e700 1 -- 172.21.15.55:0/2149842475 shutdown_connections 2023-10-06T03:34:23.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.277+0000 7f7a4f59e700 1 -- 172.21.15.55:0/2149842475 wait complete. 2023-10-06T03:34:23.280 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:34:23.295 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-10-06T03:34:22.505350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2398712,"kb_used_data":4024,"kb_used_omap":0,"kb_used_meta":2394624,"kb_avail":752576008,"statfs":{"total":773094113280,"available":770637832192,"internally_reserved":0,"allocated":4120576,"data_stored":2352004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2452094976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.987067"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.548919+0000","last_change":"2023-10-06T03:34:13.543351+0000","last_active":"2023-10-06T03:34:13.548919+0000","last_peered":"2023-10-06T03:34:13.548919+0000","last_clean":"2023-10-06T03:34:13.548919+0000","last_became_active":"2023-10-06T03:34:13.542886+0000","last_became_peered":"2023-10-06T03:34:13.542886+0000","last_unstale":"2023-10-06T03:34:13.548919+0000","last_undegraded":"2023-10-06T03:34:13.548919+0000","last_fullsized":"2023-10-06T03:34:13.548919+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:25.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541689+0000","last_change":"2023-10-06T03:34:13.538209+0000","last_active":"2023-10-06T03:34:13.541689+0000","last_peered":"2023-10-06T03:34:13.541689+0000","last_clean":"2023-10-06T03:34:13.541689+0000","last_became_active":"2023-10-06T03:34:13.537849+0000","last_became_peered":"2023-10-06T03:34:13.537849+0000","last_unstale":"2023-10-06T03:34:13.541689+0000","last_undegraded":"2023-10-06T03:34:13.541689+0000","last_fullsized":"2023-10-06T03:34:13.541689+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:38.708108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550150+0000","last_change":"2023-10-06T03:34:13.547917+0000","last_active":"2023-10-06T03:34:13.550150+0000","last_peered":"2023-10-06T03:34:13.550150+0000","last_clean":"2023-10-06T03:34:13.550150+0000","last_became_active":"2023-10-06T03:34:13.546914+0000","last_became_peered":"2023-10-06T03:34:13.546914+0000","last_unstale":"2023-10-06T03:34:13.550150+0000","last_undegraded":"2023-10-06T03:34:13.550150+0000","last_fullsized":"2023-10-06T03:34:13.550150+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.785861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544849+0000","last_change":"2023-10-06T03:34:13.538592+0000","last_active":"2023-10-06T03:34:13.544849+0000","last_peered":"2023-10-06T03:34:13.544849+0000","last_clean":"2023-10-06T03:34:13.544849+0000","last_became_active":"2023-10-06T03:34:13.538156+0000","last_became_peered":"2023-10-06T03:34:13.538156+0000","last_unstale":"2023-10-06T03:34:13.544849+0000","last_undegraded":"2023-10-06T03:34:13.544849+0000","last_fullsized":"2023-10-06T03:34:13.544849+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:10.434143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543863+0000","last_change":"2023-10-06T03:34:13.534736+0000","last_active":"2023-10-06T03:34:13.543863+0000","last_peered":"2023-10-06T03:34:13.543863+0000","last_clean":"2023-10-06T03:34:13.543863+0000","last_became_active":"2023-10-06T03:34:13.534493+0000","last_became_peered":"2023-10-06T03:34:13.534493+0000","last_unstale":"2023-10-06T03:34:13.543863+0000","last_undegraded":"2023-10-06T03:34:13.543863+0000","last_fullsized":"2023-10-06T03:34:13.543863+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:02.900878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.546850+0000","last_change":"2023-10-06T03:34:13.541709+0000","last_active":"2023-10-06T03:34:13.546850+0000","last_peered":"2023-10-06T03:34:13.546850+0000","last_clean":"2023-10-06T03:34:13.546850+0000","last_became_active":"2023-10-06T03:34:13.541399+0000","last_became_peered":"2023-10-06T03:34:13.541399+0000","last_unstale":"2023-10-06T03:34:13.546850+0000","last_undegraded":"2023-10-06T03:34:13.546850+0000","last_fullsized":"2023-10-06T03:34:13.546850+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:22.590165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557767+0000","last_change":"2023-10-06T03:34:13.545667+0000","last_active":"2023-10-06T03:34:13.557767+0000","last_peered":"2023-10-06T03:34:13.557767+0000","last_clean":"2023-10-06T03:34:13.557767+0000","last_became_active":"2023-10-06T03:34:13.545306+0000","last_became_peered":"2023-10-06T03:34:13.545306+0000","last_unstale":"2023-10-06T03:34:13.557767+0000","last_undegraded":"2023-10-06T03:34:13.557767+0000","last_fullsized":"2023-10-06T03:34:13.557767+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:38.708108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550618+0000","last_change":"2023-10-06T03:34:13.549359+0000","last_active":"2023-10-06T03:34:13.550618+0000","last_peered":"2023-10-06T03:34:13.550618+0000","last_clean":"2023-10-06T03:34:13.550618+0000","last_became_active":"2023-10-06T03:34:13.548692+0000","last_became_peered":"2023-10-06T03:34:13.548692+0000","last_unstale":"2023-10-06T03:34:13.550618+0000","last_undegraded":"2023-10-06T03:34:13.550618+0000","last_fullsized":"2023-10-06T03:34:13.550618+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:35.437314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.549190+0000","last_change":"2023-10-06T03:34:13.546572+0000","last_active":"2023-10-06T03:34:13.549190+0000","last_peered":"2023-10-06T03:34:13.549190+0000","last_clean":"2023-10-06T03:34:13.549190+0000","last_became_active":"2023-10-06T03:34:13.546313+0000","last_became_peered":"2023-10-06T03:34:13.546313+0000","last_unstale":"2023-10-06T03:34:13.549190+0000","last_undegraded":"2023-10-06T03:34:13.549190+0000","last_fullsized":"2023-10-06T03:34:13.549190+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:17.351579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560723+0000","last_change":"2023-10-06T03:34:13.558859+0000","last_active":"2023-10-06T03:34:13.560723+0000","last_peered":"2023-10-06T03:34:13.560723+0000","last_clean":"2023-10-06T03:34:13.560723+0000","last_became_active":"2023-10-06T03:34:13.558102+0000","last_became_peered":"2023-10-06T03:34:13.558102+0000","last_unstale":"2023-10-06T03:34:13.560723+0000","last_undegraded":"2023-10-06T03:34:13.560723+0000","last_fullsized":"2023-10-06T03:34:13.560723+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:17.622122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559631+0000","last_change":"2023-10-06T03:34:13.548391+0000","last_active":"2023-10-06T03:34:13.559631+0000","last_peered":"2023-10-06T03:34:13.559631+0000","last_clean":"2023-10-06T03:34:13.559631+0000","last_became_active":"2023-10-06T03:34:13.548117+0000","last_became_peered":"2023-10-06T03:34:13.548117+0000","last_unstale":"2023-10-06T03:34:13.559631+0000","last_undegraded":"2023-10-06T03:34:13.559631+0000","last_fullsized":"2023-10-06T03:34:13.559631+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:17.351579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541952+0000","last_change":"2023-10-06T03:34:13.533970+0000","last_active":"2023-10-06T03:34:13.541952+0000","last_peered":"2023-10-06T03:34:13.541952+0000","last_clean":"2023-10-06T03:34:13.541952+0000","last_became_active":"2023-10-06T03:34:13.533643+0000","last_became_peered":"2023-10-06T03:34:13.533643+0000","last_unstale":"2023-10-06T03:34:13.541952+0000","last_undegraded":"2023-10-06T03:34:13.541952+0000","last_fullsized":"2023-10-06T03:34:13.541952+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:38.277766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559747+0000","last_change":"2023-10-06T03:34:13.548687+0000","last_active":"2023-10-06T03:34:13.559747+0000","last_peered":"2023-10-06T03:34:13.559747+0000","last_clean":"2023-10-06T03:34:13.559747+0000","last_became_active":"2023-10-06T03:34:13.547089+0000","last_became_peered":"2023-10-06T03:34:13.547089+0000","last_unstale":"2023-10-06T03:34:13.559747+0000","last_undegraded":"2023-10-06T03:34:13.559747+0000","last_fullsized":"2023-10-06T03:34:13.559747+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:59:35.437314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539582+0000","last_change":"2023-10-06T03:34:13.533974+0000","last_active":"2023-10-06T03:34:13.539582+0000","last_peered":"2023-10-06T03:34:13.539582+0000","last_clean":"2023-10-06T03:34:13.539582+0000","last_became_active":"2023-10-06T03:34:13.533686+0000","last_became_peered":"2023-10-06T03:34:13.533686+0000","last_unstale":"2023-10-06T03:34:13.539582+0000","last_undegraded":"2023-10-06T03:34:13.539582+0000","last_fullsized":"2023-10-06T03:34:13.539582+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:15.558498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557756+0000","last_change":"2023-10-06T03:34:13.545608+0000","last_active":"2023-10-06T03:34:13.557756+0000","last_peered":"2023-10-06T03:34:13.557756+0000","last_clean":"2023-10-06T03:34:13.557756+0000","last_became_active":"2023-10-06T03:34:13.544797+0000","last_became_peered":"2023-10-06T03:34:13.544797+0000","last_unstale":"2023-10-06T03:34:13.557756+0000","last_undegraded":"2023-10-06T03:34:13.557756+0000","last_fullsized":"2023-10-06T03:34:13.557756+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:50.641614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557053+0000","last_change":"2023-10-06T03:34:13.555245+0000","last_active":"2023-10-06T03:34:13.557053+0000","last_peered":"2023-10-06T03:34:13.557053+0000","last_clean":"2023-10-06T03:34:13.557053+0000","last_became_active":"2023-10-06T03:34:13.555025+0000","last_became_peered":"2023-10-06T03:34:13.555025+0000","last_unstale":"2023-10-06T03:34:13.557053+0000","last_undegraded":"2023-10-06T03:34:13.557053+0000","last_fullsized":"2023-10-06T03:34:13.557053+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:25.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.552765+0000","last_change":"2023-10-06T03:34:13.549273+0000","last_active":"2023-10-06T03:34:13.552765+0000","last_peered":"2023-10-06T03:34:13.552765+0000","last_clean":"2023-10-06T03:34:13.552765+0000","last_became_active":"2023-10-06T03:34:13.548990+0000","last_became_peered":"2023-10-06T03:34:13.548990+0000","last_unstale":"2023-10-06T03:34:13.552765+0000","last_undegraded":"2023-10-06T03:34:13.552765+0000","last_fullsized":"2023-10-06T03:34:13.552765+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:17.622122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541045+0000","last_change":"2023-10-06T03:34:13.538176+0000","last_active":"2023-10-06T03:34:13.541045+0000","last_peered":"2023-10-06T03:34:13.541045+0000","last_clean":"2023-10-06T03:34:13.541045+0000","last_became_active":"2023-10-06T03:34:13.537927+0000","last_became_peered":"2023-10-06T03:34:13.537927+0000","last_unstale":"2023-10-06T03:34:13.541045+0000","last_undegraded":"2023-10-06T03:34:13.541045+0000","last_fullsized":"2023-10-06T03:34:13.541045+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:48.522789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544460+0000","last_change":"2023-10-06T03:34:13.539248+0000","last_active":"2023-10-06T03:34:13.544460+0000","last_peered":"2023-10-06T03:34:13.544460+0000","last_clean":"2023-10-06T03:34:13.544460+0000","last_became_active":"2023-10-06T03:34:13.539008+0000","last_became_peered":"2023-10-06T03:34:13.539008+0000","last_unstale":"2023-10-06T03:34:13.544460+0000","last_undegraded":"2023-10-06T03:34:13.544460+0000","last_fullsized":"2023-10-06T03:34:13.544460+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:28.353727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541088+0000","last_change":"2023-10-06T03:34:13.537228+0000","last_active":"2023-10-06T03:34:13.541088+0000","last_peered":"2023-10-06T03:34:13.541088+0000","last_clean":"2023-10-06T03:34:13.541088+0000","last_became_active":"2023-10-06T03:34:13.536971+0000","last_became_peered":"2023-10-06T03:34:13.536971+0000","last_unstale":"2023-10-06T03:34:13.541088+0000","last_undegraded":"2023-10-06T03:34:13.541088+0000","last_fullsized":"2023-10-06T03:34:13.541088+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:09.006005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557807+0000","last_change":"2023-10-06T03:34:13.547135+0000","last_active":"2023-10-06T03:34:13.557807+0000","last_peered":"2023-10-06T03:34:13.557807+0000","last_clean":"2023-10-06T03:34:13.557807+0000","last_became_active":"2023-10-06T03:34:13.546876+0000","last_became_peered":"2023-10-06T03:34:13.546876+0000","last_unstale":"2023-10-06T03:34:13.557807+0000","last_undegraded":"2023-10-06T03:34:13.557807+0000","last_fullsized":"2023-10-06T03:34:13.557807+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:03.345245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539603+0000","last_change":"2023-10-06T03:34:13.535259+0000","last_active":"2023-10-06T03:34:13.539603+0000","last_peered":"2023-10-06T03:34:13.539603+0000","last_clean":"2023-10-06T03:34:13.539603+0000","last_became_active":"2023-10-06T03:34:13.535006+0000","last_became_peered":"2023-10-06T03:34:13.535006+0000","last_unstale":"2023-10-06T03:34:13.539603+0000","last_undegraded":"2023-10-06T03:34:13.539603+0000","last_fullsized":"2023-10-06T03:34:13.539603+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:06:15.748952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.552796+0000","last_change":"2023-10-06T03:34:13.549634+0000","last_active":"2023-10-06T03:34:13.552796+0000","last_peered":"2023-10-06T03:34:13.552796+0000","last_clean":"2023-10-06T03:34:13.552796+0000","last_became_active":"2023-10-06T03:34:13.549343+0000","last_became_peered":"2023-10-06T03:34:13.549343+0000","last_unstale":"2023-10-06T03:34:13.552796+0000","last_undegraded":"2023-10-06T03:34:13.552796+0000","last_fullsized":"2023-10-06T03:34:13.552796+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:50.641614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544186+0000","last_change":"2023-10-06T03:34:13.537144+0000","last_active":"2023-10-06T03:34:13.544186+0000","last_peered":"2023-10-06T03:34:13.544186+0000","last_clean":"2023-10-06T03:34:13.544186+0000","last_became_active":"2023-10-06T03:34:13.536727+0000","last_became_peered":"2023-10-06T03:34:13.536727+0000","last_unstale":"2023-10-06T03:34:13.544186+0000","last_undegraded":"2023-10-06T03:34:13.544186+0000","last_fullsized":"2023-10-06T03:34:13.544186+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:09.006005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.537520+0000","last_change":"2023-10-06T03:34:13.529906+0000","last_active":"2023-10-06T03:34:13.537520+0000","last_peered":"2023-10-06T03:34:13.537520+0000","last_clean":"2023-10-06T03:34:13.537520+0000","last_became_active":"2023-10-06T03:34:13.529637+0000","last_became_peered":"2023-10-06T03:34:13.529637+0000","last_unstale":"2023-10-06T03:34:13.537520+0000","last_undegraded":"2023-10-06T03:34:13.537520+0000","last_fullsized":"2023-10-06T03:34:13.537520+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:10.434143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.558046+0000","last_change":"2023-10-06T03:34:13.555751+0000","last_active":"2023-10-06T03:34:13.558046+0000","last_peered":"2023-10-06T03:34:13.558046+0000","last_clean":"2023-10-06T03:34:13.558046+0000","last_became_active":"2023-10-06T03:34:13.555469+0000","last_became_peered":"2023-10-06T03:34:13.555469+0000","last_unstale":"2023-10-06T03:34:13.558046+0000","last_undegraded":"2023-10-06T03:34:13.558046+0000","last_fullsized":"2023-10-06T03:34:13.558046+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:53.622503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540269+0000","last_change":"2023-10-06T03:34:13.534759+0000","last_active":"2023-10-06T03:34:13.540269+0000","last_peered":"2023-10-06T03:34:13.540269+0000","last_clean":"2023-10-06T03:34:13.540269+0000","last_became_active":"2023-10-06T03:34:13.534458+0000","last_became_peered":"2023-10-06T03:34:13.534458+0000","last_unstale":"2023-10-06T03:34:13.540269+0000","last_undegraded":"2023-10-06T03:34:13.540269+0000","last_fullsized":"2023-10-06T03:34:13.540269+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:55.396883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559551+0000","last_change":"2023-10-06T03:34:13.547737+0000","last_active":"2023-10-06T03:34:13.559551+0000","last_peered":"2023-10-06T03:34:13.559551+0000","last_clean":"2023-10-06T03:34:13.559551+0000","last_became_active":"2023-10-06T03:34:13.547463+0000","last_became_peered":"2023-10-06T03:34:13.547463+0000","last_unstale":"2023-10-06T03:34:13.559551+0000","last_undegraded":"2023-10-06T03:34:13.559551+0000","last_fullsized":"2023-10-06T03:34:13.559551+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:30.492483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540458+0000","last_change":"2023-10-06T03:34:13.536475+0000","last_active":"2023-10-06T03:34:13.540458+0000","last_peered":"2023-10-06T03:34:13.540458+0000","last_clean":"2023-10-06T03:34:13.540458+0000","last_became_active":"2023-10-06T03:34:13.536202+0000","last_became_peered":"2023-10-06T03:34:13.536202+0000","last_unstale":"2023-10-06T03:34:13.540458+0000","last_undegraded":"2023-10-06T03:34:13.540458+0000","last_fullsized":"2023-10-06T03:34:13.540458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:02.900878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.537621+0000","last_change":"2023-10-06T03:34:13.530628+0000","last_active":"2023-10-06T03:34:13.537621+0000","last_peered":"2023-10-06T03:34:13.537621+0000","last_clean":"2023-10-06T03:34:13.537621+0000","last_became_active":"2023-10-06T03:34:13.530318+0000","last_became_peered":"2023-10-06T03:34:13.530318+0000","last_unstale":"2023-10-06T03:34:13.537621+0000","last_undegraded":"2023-10-06T03:34:13.537621+0000","last_fullsized":"2023-10-06T03:34:13.537621+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:18.351278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550254+0000","last_change":"2023-10-06T03:34:13.547733+0000","last_active":"2023-10-06T03:34:13.550254+0000","last_peered":"2023-10-06T03:34:13.550254+0000","last_clean":"2023-10-06T03:34:13.550254+0000","last_became_active":"2023-10-06T03:34:13.547402+0000","last_became_peered":"2023-10-06T03:34:13.547402+0000","last_unstale":"2023-10-06T03:34:13.550254+0000","last_undegraded":"2023-10-06T03:34:13.550254+0000","last_fullsized":"2023-10-06T03:34:13.550254+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:05.808082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.547236+0000","last_change":"2023-10-06T03:34:13.540168+0000","last_active":"2023-10-06T03:34:13.547236+0000","last_peered":"2023-10-06T03:34:13.547236+0000","last_clean":"2023-10-06T03:34:13.547236+0000","last_became_active":"2023-10-06T03:34:13.539910+0000","last_became_peered":"2023-10-06T03:34:13.539910+0000","last_unstale":"2023-10-06T03:34:13.547236+0000","last_undegraded":"2023-10-06T03:34:13.547236+0000","last_fullsized":"2023-10-06T03:34:13.547236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:04.506250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557194+0000","last_change":"2023-10-06T03:34:13.556269+0000","last_active":"2023-10-06T03:34:13.557194+0000","last_peered":"2023-10-06T03:34:13.557194+0000","last_clean":"2023-10-06T03:34:13.557194+0000","last_became_active":"2023-10-06T03:34:13.555974+0000","last_became_peered":"2023-10-06T03:34:13.555974+0000","last_unstale":"2023-10-06T03:34:13.557194+0000","last_undegraded":"2023-10-06T03:34:13.557194+0000","last_fullsized":"2023-10-06T03:34:13.557194+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:03.345245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"31'4","reported_seq":14,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544660+0000","last_change":"2023-10-06T03:34:12.523661+0000","last_active":"2023-10-06T03:34:13.544660+0000","last_peered":"2023-10-06T03:34:13.544660+0000","last_clean":"2023-10-06T03:34:13.544660+0000","last_became_active":"2023-10-06T03:34:12.523343+0000","last_became_peered":"2023-10-06T03:34:12.523343+0000","last_unstale":"2023-10-06T03:34:13.544660+0000","last_undegraded":"2023-10-06T03:34:13.544660+0000","last_fullsized":"2023-10-06T03:34:13.544660+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:23.796615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.549209+0000","last_change":"2023-10-06T03:34:13.546825+0000","last_active":"2023-10-06T03:34:13.549209+0000","last_peered":"2023-10-06T03:34:13.549209+0000","last_clean":"2023-10-06T03:34:13.549209+0000","last_became_active":"2023-10-06T03:34:13.546561+0000","last_became_peered":"2023-10-06T03:34:13.546561+0000","last_unstale":"2023-10-06T03:34:13.549209+0000","last_undegraded":"2023-10-06T03:34:13.549209+0000","last_fullsized":"2023-10-06T03:34:13.549209+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:19:10.048229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540711+0000","last_change":"2023-10-06T03:34:12.519828+0000","last_active":"2023-10-06T03:34:13.540711+0000","last_peered":"2023-10-06T03:34:13.540711+0000","last_clean":"2023-10-06T03:34:13.540711+0000","last_became_active":"2023-10-06T03:34:12.519596+0000","last_became_peered":"2023-10-06T03:34:12.519596+0000","last_unstale":"2023-10-06T03:34:13.540711+0000","last_undegraded":"2023-10-06T03:34:13.540711+0000","last_fullsized":"2023-10-06T03:34:13.540711+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:28:01.889807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539250+0000","last_change":"2023-10-06T03:34:13.533928+0000","last_active":"2023-10-06T03:34:13.539250+0000","last_peered":"2023-10-06T03:34:13.539250+0000","last_clean":"2023-10-06T03:34:13.539250+0000","last_became_active":"2023-10-06T03:34:13.533642+0000","last_became_peered":"2023-10-06T03:34:13.533642+0000","last_unstale":"2023-10-06T03:34:13.539250+0000","last_undegraded":"2023-10-06T03:34:13.539250+0000","last_fullsized":"2023-10-06T03:34:13.539250+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:25.840403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.545261+0000","last_change":"2023-10-06T03:34:12.526656+0000","last_active":"2023-10-06T03:34:13.545261+0000","last_peered":"2023-10-06T03:34:13.545261+0000","last_clean":"2023-10-06T03:34:13.545261+0000","last_became_active":"2023-10-06T03:34:12.525776+0000","last_became_peered":"2023-10-06T03:34:12.525776+0000","last_unstale":"2023-10-06T03:34:13.545261+0000","last_undegraded":"2023-10-06T03:34:13.545261+0000","last_fullsized":"2023-10-06T03:34:13.545261+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:27.342656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.548898+0000","last_change":"2023-10-06T03:34:13.543146+0000","last_active":"2023-10-06T03:34:13.548898+0000","last_peered":"2023-10-06T03:34:13.548898+0000","last_clean":"2023-10-06T03:34:13.548898+0000","last_became_active":"2023-10-06T03:34:13.542460+0000","last_became_peered":"2023-10-06T03:34:13.542460+0000","last_unstale":"2023-10-06T03:34:13.548898+0000","last_undegraded":"2023-10-06T03:34:13.548898+0000","last_fullsized":"2023-10-06T03:34:13.548898+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.785861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.552296+0000","last_change":"2023-10-06T03:34:12.524322+0000","last_active":"2023-10-06T03:34:13.552296+0000","last_peered":"2023-10-06T03:34:13.552296+0000","last_clean":"2023-10-06T03:34:13.552296+0000","last_became_active":"2023-10-06T03:34:12.524037+0000","last_became_peered":"2023-10-06T03:34:12.524037+0000","last_unstale":"2023-10-06T03:34:13.552296+0000","last_undegraded":"2023-10-06T03:34:13.552296+0000","last_fullsized":"2023-10-06T03:34:13.552296+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:57:59.774790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543775+0000","last_change":"2023-10-06T03:34:13.539137+0000","last_active":"2023-10-06T03:34:13.543775+0000","last_peered":"2023-10-06T03:34:13.543775+0000","last_clean":"2023-10-06T03:34:13.543775+0000","last_became_active":"2023-10-06T03:34:13.538833+0000","last_became_peered":"2023-10-06T03:34:13.538833+0000","last_unstale":"2023-10-06T03:34:13.543775+0000","last_undegraded":"2023-10-06T03:34:13.543775+0000","last_fullsized":"2023-10-06T03:34:13.543775+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:27.486440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559924+0000","last_change":"2023-10-06T03:34:12.527023+0000","last_active":"2023-10-06T03:34:13.559924+0000","last_peered":"2023-10-06T03:34:13.559924+0000","last_clean":"2023-10-06T03:34:13.559924+0000","last_became_active":"2023-10-06T03:34:12.526443+0000","last_became_peered":"2023-10-06T03:34:12.526443+0000","last_unstale":"2023-10-06T03:34:13.559924+0000","last_undegraded":"2023-10-06T03:34:13.559924+0000","last_fullsized":"2023-10-06T03:34:13.559924+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:54.385812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557742+0000","last_change":"2023-10-06T03:34:13.544656+0000","last_active":"2023-10-06T03:34:13.557742+0000","last_peered":"2023-10-06T03:34:13.557742+0000","last_clean":"2023-10-06T03:34:13.557742+0000","last_became_active":"2023-10-06T03:34:13.544249+0000","last_became_peered":"2023-10-06T03:34:13.544249+0000","last_unstale":"2023-10-06T03:34:13.557742+0000","last_undegraded":"2023-10-06T03:34:13.557742+0000","last_fullsized":"2023-10-06T03:34:13.557742+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:01.766184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.552592+0000","last_change":"2023-10-06T03:34:12.532851+0000","last_active":"2023-10-06T03:34:13.552592+0000","last_peered":"2023-10-06T03:34:13.552592+0000","last_clean":"2023-10-06T03:34:13.552592+0000","last_became_active":"2023-10-06T03:34:12.532517+0000","last_became_peered":"2023-10-06T03:34:12.532517+0000","last_unstale":"2023-10-06T03:34:13.552592+0000","last_undegraded":"2023-10-06T03:34:13.552592+0000","last_fullsized":"2023-10-06T03:34:13.552592+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:04.840518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.549683+0000","last_change":"2023-10-06T03:34:13.546209+0000","last_active":"2023-10-06T03:34:13.549683+0000","last_peered":"2023-10-06T03:34:13.549683+0000","last_clean":"2023-10-06T03:34:13.549683+0000","last_became_active":"2023-10-06T03:34:13.545940+0000","last_became_peered":"2023-10-06T03:34:13.545940+0000","last_unstale":"2023-10-06T03:34:13.549683+0000","last_undegraded":"2023-10-06T03:34:13.549683+0000","last_fullsized":"2023-10-06T03:34:13.549683+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:44.937279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.528389+0000","last_change":"2023-10-06T03:34:12.527549+0000","last_active":"2023-10-06T03:34:13.528389+0000","last_peered":"2023-10-06T03:34:13.528389+0000","last_clean":"2023-10-06T03:34:13.528389+0000","last_became_active":"2023-10-06T03:34:12.527236+0000","last_became_peered":"2023-10-06T03:34:12.527236+0000","last_unstale":"2023-10-06T03:34:13.528389+0000","last_undegraded":"2023-10-06T03:34:13.528389+0000","last_fullsized":"2023-10-06T03:34:13.528389+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:43.578807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.542212+0000","last_change":"2023-10-06T03:34:13.535907+0000","last_active":"2023-10-06T03:34:13.542212+0000","last_peered":"2023-10-06T03:34:13.542212+0000","last_clean":"2023-10-06T03:34:13.542212+0000","last_became_active":"2023-10-06T03:34:13.535054+0000","last_became_peered":"2023-10-06T03:34:13.535054+0000","last_unstale":"2023-10-06T03:34:13.542212+0000","last_undegraded":"2023-10-06T03:34:13.542212+0000","last_fullsized":"2023-10-06T03:34:13.542212+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:15.558498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.523244+0000","last_change":"2023-10-06T03:34:12.516603+0000","last_active":"2023-10-06T03:34:13.523244+0000","last_peered":"2023-10-06T03:34:13.523244+0000","last_clean":"2023-10-06T03:34:13.523244+0000","last_became_active":"2023-10-06T03:34:12.516217+0000","last_became_peered":"2023-10-06T03:34:12.516217+0000","last_unstale":"2023-10-06T03:34:13.523244+0000","last_undegraded":"2023-10-06T03:34:13.523244+0000","last_fullsized":"2023-10-06T03:34:13.523244+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:16.340508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.544831+0000","last_change":"2023-10-06T03:34:13.538400+0000","last_active":"2023-10-06T03:34:13.544831+0000","last_peered":"2023-10-06T03:34:13.544831+0000","last_clean":"2023-10-06T03:34:13.544831+0000","last_became_active":"2023-10-06T03:34:13.537683+0000","last_became_peered":"2023-10-06T03:34:13.537683+0000","last_unstale":"2023-10-06T03:34:13.544831+0000","last_undegraded":"2023-10-06T03:34:13.544831+0000","last_fullsized":"2023-10-06T03:34:13.544831+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:06.270734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.526789+0000","last_change":"2023-10-06T03:34:12.523775+0000","last_active":"2023-10-06T03:34:13.526789+0000","last_peered":"2023-10-06T03:34:13.526789+0000","last_clean":"2023-10-06T03:34:13.526789+0000","last_became_active":"2023-10-06T03:34:12.523512+0000","last_became_peered":"2023-10-06T03:34:12.523512+0000","last_unstale":"2023-10-06T03:34:13.526789+0000","last_undegraded":"2023-10-06T03:34:13.526789+0000","last_fullsized":"2023-10-06T03:34:13.526789+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:21.579094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543961+0000","last_change":"2023-10-06T03:34:13.541278+0000","last_active":"2023-10-06T03:34:13.543961+0000","last_peered":"2023-10-06T03:34:13.543961+0000","last_clean":"2023-10-06T03:34:13.543961+0000","last_became_active":"2023-10-06T03:34:13.541008+0000","last_became_peered":"2023-10-06T03:34:13.541008+0000","last_unstale":"2023-10-06T03:34:13.543961+0000","last_undegraded":"2023-10-06T03:34:13.543961+0000","last_fullsized":"2023-10-06T03:34:13.543961+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:38.708108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539800+0000","last_change":"2023-10-06T03:34:12.528843+0000","last_active":"2023-10-06T03:34:13.539800+0000","last_peered":"2023-10-06T03:34:13.539800+0000","last_clean":"2023-10-06T03:34:13.539800+0000","last_became_active":"2023-10-06T03:34:12.528575+0000","last_became_peered":"2023-10-06T03:34:12.528575+0000","last_unstale":"2023-10-06T03:34:13.539800+0000","last_undegraded":"2023-10-06T03:34:13.539800+0000","last_fullsized":"2023-10-06T03:34:13.539800+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:07.994934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540805+0000","last_change":"2023-10-06T03:34:13.535849+0000","last_active":"2023-10-06T03:34:13.540805+0000","last_peered":"2023-10-06T03:34:13.540805+0000","last_clean":"2023-10-06T03:34:13.540805+0000","last_became_active":"2023-10-06T03:34:13.535582+0000","last_became_peered":"2023-10-06T03:34:13.535582+0000","last_unstale":"2023-10-06T03:34:13.540805+0000","last_undegraded":"2023-10-06T03:34:13.540805+0000","last_fullsized":"2023-10-06T03:34:13.540805+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:53.934064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"31'3","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560103+0000","last_change":"2023-10-06T03:34:12.529757+0000","last_active":"2023-10-06T03:34:13.560103+0000","last_peered":"2023-10-06T03:34:13.560103+0000","last_clean":"2023-10-06T03:34:13.560103+0000","last_became_active":"2023-10-06T03:34:12.529371+0000","last_became_peered":"2023-10-06T03:34:12.529371+0000","last_unstale":"2023-10-06T03:34:13.560103+0000","last_undegraded":"2023-10-06T03:34:13.560103+0000","last_fullsized":"2023-10-06T03:34:13.560103+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:17.340207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540076+0000","last_change":"2023-10-06T03:34:13.534519+0000","last_active":"2023-10-06T03:34:13.540076+0000","last_peered":"2023-10-06T03:34:13.540076+0000","last_clean":"2023-10-06T03:34:13.540076+0000","last_became_active":"2023-10-06T03:34:13.534278+0000","last_became_peered":"2023-10-06T03:34:13.534278+0000","last_unstale":"2023-10-06T03:34:13.540076+0000","last_undegraded":"2023-10-06T03:34:13.540076+0000","last_fullsized":"2023-10-06T03:34:13.540076+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:51:51.583498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"31'6","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.561812+0000","last_change":"2023-10-06T03:34:12.528656+0000","last_active":"2023-10-06T03:34:13.561812+0000","last_peered":"2023-10-06T03:34:13.561812+0000","last_clean":"2023-10-06T03:34:13.561812+0000","last_became_active":"2023-10-06T03:34:12.528361+0000","last_became_peered":"2023-10-06T03:34:12.528361+0000","last_unstale":"2023-10-06T03:34:13.561812+0000","last_undegraded":"2023-10-06T03:34:13.561812+0000","last_fullsized":"2023-10-06T03:34:13.561812+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:31.070647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.557081+0000","last_change":"2023-10-06T03:34:13.555313+0000","last_active":"2023-10-06T03:34:13.557081+0000","last_peered":"2023-10-06T03:34:13.557081+0000","last_clean":"2023-10-06T03:34:13.557081+0000","last_became_active":"2023-10-06T03:34:13.555027+0000","last_became_peered":"2023-10-06T03:34:13.555027+0000","last_unstale":"2023-10-06T03:34:13.557081+0000","last_undegraded":"2023-10-06T03:34:13.557081+0000","last_fullsized":"2023-10-06T03:34:13.557081+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:01.442206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550514+0000","last_change":"2023-10-06T03:34:12.523998+0000","last_active":"2023-10-06T03:34:13.550514+0000","last_peered":"2023-10-06T03:34:13.550514+0000","last_clean":"2023-10-06T03:34:13.550514+0000","last_became_active":"2023-10-06T03:34:12.523688+0000","last_became_peered":"2023-10-06T03:34:12.523688+0000","last_unstale":"2023-10-06T03:34:13.550514+0000","last_undegraded":"2023-10-06T03:34:13.550514+0000","last_fullsized":"2023-10-06T03:34:13.550514+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:52.611432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.542751+0000","last_change":"2023-10-06T03:34:13.540598+0000","last_active":"2023-10-06T03:34:13.542751+0000","last_peered":"2023-10-06T03:34:13.542751+0000","last_clean":"2023-10-06T03:34:13.542751+0000","last_became_active":"2023-10-06T03:34:13.540350+0000","last_became_peered":"2023-10-06T03:34:13.540350+0000","last_unstale":"2023-10-06T03:34:13.542751+0000","last_undegraded":"2023-10-06T03:34:13.542751+0000","last_fullsized":"2023-10-06T03:34:13.542751+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:24:03.345245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.527766+0000","last_change":"2023-10-06T03:34:12.530728+0000","last_active":"2023-10-06T03:34:13.527766+0000","last_peered":"2023-10-06T03:34:13.527766+0000","last_clean":"2023-10-06T03:34:13.527766+0000","last_became_active":"2023-10-06T03:34:12.530458+0000","last_became_peered":"2023-10-06T03:34:12.530458+0000","last_unstale":"2023-10-06T03:34:13.527766+0000","last_undegraded":"2023-10-06T03:34:13.527766+0000","last_fullsized":"2023-10-06T03:34:13.527766+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:36:09.423072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550408+0000","last_change":"2023-10-06T03:34:13.547951+0000","last_active":"2023-10-06T03:34:13.550408+0000","last_peered":"2023-10-06T03:34:13.550408+0000","last_clean":"2023-10-06T03:34:13.550408+0000","last_became_active":"2023-10-06T03:34:13.547567+0000","last_became_peered":"2023-10-06T03:34:13.547567+0000","last_unstale":"2023-10-06T03:34:13.550408+0000","last_undegraded":"2023-10-06T03:34:13.550408+0000","last_fullsized":"2023-10-06T03:34:13.550408+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:19:01.766184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.521120+0000","last_change":"2023-10-06T03:34:12.517112+0000","last_active":"2023-10-06T03:34:13.521120+0000","last_peered":"2023-10-06T03:34:13.521120+0000","last_clean":"2023-10-06T03:34:13.521120+0000","last_became_active":"2023-10-06T03:34:12.516831+0000","last_became_peered":"2023-10-06T03:34:12.516831+0000","last_unstale":"2023-10-06T03:34:13.521120+0000","last_undegraded":"2023-10-06T03:34:13.521120+0000","last_fullsized":"2023-10-06T03:34:13.521120+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:48.752619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.542193+0000","last_change":"2023-10-06T03:34:13.535453+0000","last_active":"2023-10-06T03:34:13.542193+0000","last_peered":"2023-10-06T03:34:13.542193+0000","last_clean":"2023-10-06T03:34:13.542193+0000","last_became_active":"2023-10-06T03:34:13.534677+0000","last_became_peered":"2023-10-06T03:34:13.534677+0000","last_unstale":"2023-10-06T03:34:13.542193+0000","last_undegraded":"2023-10-06T03:34:13.542193+0000","last_fullsized":"2023-10-06T03:34:13.542193+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:19.240735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"31'8","reported_seq":18,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.556402+0000","last_change":"2023-10-06T03:34:12.532903+0000","last_active":"2023-10-06T03:34:13.556402+0000","last_peered":"2023-10-06T03:34:13.556402+0000","last_clean":"2023-10-06T03:34:13.556402+0000","last_became_active":"2023-10-06T03:34:12.532576+0000","last_became_peered":"2023-10-06T03:34:12.532576+0000","last_unstale":"2023-10-06T03:34:13.556402+0000","last_undegraded":"2023-10-06T03:34:13.556402+0000","last_fullsized":"2023-10-06T03:34:13.556402+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539800+0000","last_change":"2023-10-06T03:34:13.531821+0000","last_active":"2023-10-06T03:34:13.539800+0000","last_peered":"2023-10-06T03:34:13.539800+0000","last_clean":"2023-10-06T03:34:13.539800+0000","last_became_active":"2023-10-06T03:34:13.531527+0000","last_became_peered":"2023-10-06T03:34:13.531527+0000","last_unstale":"2023-10-06T03:34:13.539800+0000","last_undegraded":"2023-10-06T03:34:13.539800+0000","last_fullsized":"2023-10-06T03:34:13.539800+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:05.808082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560016+0000","last_change":"2023-10-06T03:34:12.532478+0000","last_active":"2023-10-06T03:34:13.560016+0000","last_peered":"2023-10-06T03:34:13.560016+0000","last_clean":"2023-10-06T03:34:13.560016+0000","last_became_active":"2023-10-06T03:34:12.532204+0000","last_became_peered":"2023-10-06T03:34:12.532204+0000","last_unstale":"2023-10-06T03:34:13.560016+0000","last_undegraded":"2023-10-06T03:34:13.560016+0000","last_fullsized":"2023-10-06T03:34:13.560016+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560141+0000","last_change":"2023-10-06T03:34:13.556710+0000","last_active":"2023-10-06T03:34:13.560141+0000","last_peered":"2023-10-06T03:34:13.560141+0000","last_clean":"2023-10-06T03:34:13.560141+0000","last_became_active":"2023-10-06T03:34:13.554487+0000","last_became_peered":"2023-10-06T03:34:13.554487+0000","last_unstale":"2023-10-06T03:34:13.560141+0000","last_undegraded":"2023-10-06T03:34:13.560141+0000","last_fullsized":"2023-10-06T03:34:13.560141+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:50:25.467790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.560376+0000","last_change":"2023-10-06T03:34:12.527219+0000","last_active":"2023-10-06T03:34:13.560376+0000","last_peered":"2023-10-06T03:34:13.560376+0000","last_clean":"2023-10-06T03:34:13.560376+0000","last_became_active":"2023-10-06T03:34:12.526985+0000","last_became_peered":"2023-10-06T03:34:12.526985+0000","last_unstale":"2023-10-06T03:34:13.560376+0000","last_undegraded":"2023-10-06T03:34:13.560376+0000","last_fullsized":"2023-10-06T03:34:13.560376+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:29.481412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543498+0000","last_change":"2023-10-06T03:34:13.537130+0000","last_active":"2023-10-06T03:34:13.543498+0000","last_peered":"2023-10-06T03:34:13.543498+0000","last_clean":"2023-10-06T03:34:13.543498+0000","last_became_active":"2023-10-06T03:34:13.536871+0000","last_became_peered":"2023-10-06T03:34:13.536871+0000","last_unstale":"2023-10-06T03:34:13.543498+0000","last_undegraded":"2023-10-06T03:34:13.543498+0000","last_fullsized":"2023-10-06T03:34:13.543498+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:21.602441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"31'1","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.553427+0000","last_change":"2023-10-06T03:34:12.524489+0000","last_active":"2023-10-06T03:34:13.553427+0000","last_peered":"2023-10-06T03:34:13.553427+0000","last_clean":"2023-10-06T03:34:13.553427+0000","last_became_active":"2023-10-06T03:34:12.524226+0000","last_became_peered":"2023-10-06T03:34:12.524226+0000","last_unstale":"2023-10-06T03:34:13.553427+0000","last_undegraded":"2023-10-06T03:34:13.553427+0000","last_fullsized":"2023-10-06T03:34:13.553427+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:37.934361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.536949+0000","last_change":"2023-10-06T03:34:13.531913+0000","last_active":"2023-10-06T03:34:13.536949+0000","last_peered":"2023-10-06T03:34:13.536949+0000","last_clean":"2023-10-06T03:34:13.536949+0000","last_became_active":"2023-10-06T03:34:13.531611+0000","last_became_peered":"2023-10-06T03:34:13.531611+0000","last_unstale":"2023-10-06T03:34:13.536949+0000","last_undegraded":"2023-10-06T03:34:13.536949+0000","last_fullsized":"2023-10-06T03:34:13.536949+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:44.589878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.528863+0000","last_change":"2023-10-06T03:34:12.532410+0000","last_active":"2023-10-06T03:34:13.528863+0000","last_peered":"2023-10-06T03:34:13.528863+0000","last_clean":"2023-10-06T03:34:13.528863+0000","last_became_active":"2023-10-06T03:34:12.532154+0000","last_became_peered":"2023-10-06T03:34:12.532154+0000","last_unstale":"2023-10-06T03:34:13.528863+0000","last_undegraded":"2023-10-06T03:34:13.528863+0000","last_fullsized":"2023-10-06T03:34:13.528863+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:41:05.259663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.549613+0000","last_change":"2023-10-06T03:34:13.546456+0000","last_active":"2023-10-06T03:34:13.549613+0000","last_peered":"2023-10-06T03:34:13.549613+0000","last_clean":"2023-10-06T03:34:13.549613+0000","last_became_active":"2023-10-06T03:34:13.546118+0000","last_became_peered":"2023-10-06T03:34:13.546118+0000","last_unstale":"2023-10-06T03:34:13.549613+0000","last_undegraded":"2023-10-06T03:34:13.549613+0000","last_fullsized":"2023-10-06T03:34:13.549613+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:30.492483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.550619+0000","last_change":"2023-10-06T03:34:12.519345+0000","last_active":"2023-10-06T03:34:13.550619+0000","last_peered":"2023-10-06T03:34:13.550619+0000","last_clean":"2023-10-06T03:34:13.550619+0000","last_became_active":"2023-10-06T03:34:12.519053+0000","last_became_peered":"2023-10-06T03:34:12.519053+0000","last_unstale":"2023-10-06T03:34:13.550619+0000","last_undegraded":"2023-10-06T03:34:13.550619+0000","last_fullsized":"2023-10-06T03:34:13.550619+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:34.904232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.540779+0000","last_change":"2023-10-06T03:34:13.535161+0000","last_active":"2023-10-06T03:34:13.540779+0000","last_peered":"2023-10-06T03:34:13.540779+0000","last_clean":"2023-10-06T03:34:13.540779+0000","last_became_active":"2023-10-06T03:34:13.534909+0000","last_became_peered":"2023-10-06T03:34:13.534909+0000","last_unstale":"2023-10-06T03:34:13.540779+0000","last_undegraded":"2023-10-06T03:34:13.540779+0000","last_fullsized":"2023-10-06T03:34:13.540779+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:48.475963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.524923+0000","last_change":"2023-10-06T03:34:12.522297+0000","last_active":"2023-10-06T03:34:13.524923+0000","last_peered":"2023-10-06T03:34:13.524923+0000","last_clean":"2023-10-06T03:34:13.524923+0000","last_became_active":"2023-10-06T03:34:12.522006+0000","last_became_peered":"2023-10-06T03:34:12.522006+0000","last_unstale":"2023-10-06T03:34:13.524923+0000","last_undegraded":"2023-10-06T03:34:13.524923+0000","last_fullsized":"2023-10-06T03:34:13.524923+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:52:39.716255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559638+0000","last_change":"2023-10-06T03:34:13.547565+0000","last_active":"2023-10-06T03:34:13.559638+0000","last_peered":"2023-10-06T03:34:13.559638+0000","last_clean":"2023-10-06T03:34:13.559638+0000","last_became_active":"2023-10-06T03:34:13.547290+0000","last_became_peered":"2023-10-06T03:34:13.547290+0000","last_unstale":"2023-10-06T03:34:13.559638+0000","last_undegraded":"2023-10-06T03:34:13.559638+0000","last_fullsized":"2023-10-06T03:34:13.559638+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:05.808082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.531326+0000","last_change":"2023-10-06T03:34:12.527419+0000","last_active":"2023-10-06T03:34:13.531326+0000","last_peered":"2023-10-06T03:34:13.531326+0000","last_clean":"2023-10-06T03:34:13.531326+0000","last_became_active":"2023-10-06T03:34:12.527141+0000","last_became_peered":"2023-10-06T03:34:12.527141+0000","last_unstale":"2023-10-06T03:34:13.531326+0000","last_undegraded":"2023-10-06T03:34:13.531326+0000","last_fullsized":"2023-10-06T03:34:13.531326+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:17.340207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.542230+0000","last_change":"2023-10-06T03:34:13.536079+0000","last_active":"2023-10-06T03:34:13.542230+0000","last_peered":"2023-10-06T03:34:13.542230+0000","last_clean":"2023-10-06T03:34:13.542230+0000","last_became_active":"2023-10-06T03:34:13.535653+0000","last_became_peered":"2023-10-06T03:34:13.535653+0000","last_unstale":"2023-10-06T03:34:13.542230+0000","last_undegraded":"2023-10-06T03:34:13.542230+0000","last_fullsized":"2023-10-06T03:34:13.542230+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:25.840403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.525414+0000","last_change":"2023-10-06T03:34:12.521014+0000","last_active":"2023-10-06T03:34:13.525414+0000","last_peered":"2023-10-06T03:34:13.525414+0000","last_clean":"2023-10-06T03:34:13.525414+0000","last_became_active":"2023-10-06T03:34:12.520088+0000","last_became_peered":"2023-10-06T03:34:12.520088+0000","last_unstale":"2023-10-06T03:34:13.525414+0000","last_undegraded":"2023-10-06T03:34:13.525414+0000","last_fullsized":"2023-10-06T03:34:13.525414+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:34.651158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.539620+0000","last_change":"2023-10-06T03:34:13.535691+0000","last_active":"2023-10-06T03:34:13.539620+0000","last_peered":"2023-10-06T03:34:13.539620+0000","last_clean":"2023-10-06T03:34:13.539620+0000","last_became_active":"2023-10-06T03:34:13.535365+0000","last_became_peered":"2023-10-06T03:34:13.535365+0000","last_unstale":"2023-10-06T03:34:13.539620+0000","last_undegraded":"2023-10-06T03:34:13.539620+0000","last_fullsized":"2023-10-06T03:34:13.539620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:08.739478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.523084+0000","last_change":"2023-10-06T03:34:12.525259+0000","last_active":"2023-10-06T03:34:13.523084+0000","last_peered":"2023-10-06T03:34:13.523084+0000","last_clean":"2023-10-06T03:34:13.523084+0000","last_became_active":"2023-10-06T03:34:12.524902+0000","last_became_peered":"2023-10-06T03:34:12.524902+0000","last_unstale":"2023-10-06T03:34:13.523084+0000","last_undegraded":"2023-10-06T03:34:13.523084+0000","last_fullsized":"2023-10-06T03:34:13.523084+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:37.697037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.524780+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:34:13.524780+0000","last_peered":"2023-10-06T03:34:13.524780+0000","last_clean":"2023-10-06T03:34:13.524780+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:34:13.524780+0000","last_undegraded":"2023-10-06T03:34:13.524780+0000","last_fullsized":"2023-10-06T03:34:13.524780+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.545347+0000","last_change":"2023-10-06T03:34:13.543841+0000","last_active":"2023-10-06T03:34:13.545347+0000","last_peered":"2023-10-06T03:34:13.545347+0000","last_clean":"2023-10-06T03:34:13.545347+0000","last_became_active":"2023-10-06T03:34:13.543580+0000","last_became_peered":"2023-10-06T03:34:13.543580+0000","last_unstale":"2023-10-06T03:34:13.545347+0000","last_undegraded":"2023-10-06T03:34:13.545347+0000","last_fullsized":"2023-10-06T03:34:13.545347+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:53.934064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.524324+0000","last_change":"2023-10-06T03:34:12.525282+0000","last_active":"2023-10-06T03:34:13.524324+0000","last_peered":"2023-10-06T03:34:13.524324+0000","last_clean":"2023-10-06T03:34:13.524324+0000","last_became_active":"2023-10-06T03:34:12.525020+0000","last_became_peered":"2023-10-06T03:34:12.525020+0000","last_unstale":"2023-10-06T03:34:13.524324+0000","last_undegraded":"2023-10-06T03:34:13.524324+0000","last_fullsized":"2023-10-06T03:34:13.524324+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:03.495179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.524356+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:34:13.524356+0000","last_peered":"2023-10-06T03:34:13.524356+0000","last_clean":"2023-10-06T03:34:13.524356+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:34:13.524356+0000","last_undegraded":"2023-10-06T03:34:13.524356+0000","last_fullsized":"2023-10-06T03:34:13.524356+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.558168+0000","last_change":"2023-10-06T03:34:13.555991+0000","last_active":"2023-10-06T03:34:13.558168+0000","last_peered":"2023-10-06T03:34:13.558168+0000","last_clean":"2023-10-06T03:34:13.558168+0000","last_became_active":"2023-10-06T03:34:13.555086+0000","last_became_peered":"2023-10-06T03:34:13.555086+0000","last_unstale":"2023-10-06T03:34:13.558168+0000","last_undegraded":"2023-10-06T03:34:13.558168+0000","last_fullsized":"2023-10-06T03:34:13.558168+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:27.486440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"31'2","reported_seq":12,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559924+0000","last_change":"2023-10-06T03:34:12.529984+0000","last_active":"2023-10-06T03:34:13.559924+0000","last_peered":"2023-10-06T03:34:13.559924+0000","last_clean":"2023-10-06T03:34:13.559924+0000","last_became_active":"2023-10-06T03:34:12.529440+0000","last_became_peered":"2023-10-06T03:34:12.529440+0000","last_unstale":"2023-10-06T03:34:13.559924+0000","last_undegraded":"2023-10-06T03:34:13.559924+0000","last_fullsized":"2023-10-06T03:34:13.559924+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:04.840518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":22,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.527887+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:34:13.527887+0000","last_peered":"2023-10-06T03:34:13.527887+0000","last_clean":"2023-10-06T03:34:13.527887+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:34:13.527887+0000","last_undegraded":"2023-10-06T03:34:13.527887+0000","last_fullsized":"2023-10-06T03:34:13.527887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":172,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.531546+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:34:13.531546+0000","last_peered":"2023-10-06T03:34:13.531546+0000","last_clean":"2023-10-06T03:34:13.531546+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:34:13.531546+0000","last_undegraded":"2023-10-06T03:34:13.531546+0000","last_fullsized":"2023-10-06T03:34:13.531546+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.543979+0000","last_change":"2023-10-06T03:34:13.541987+0000","last_active":"2023-10-06T03:34:13.543979+0000","last_peered":"2023-10-06T03:34:13.543979+0000","last_clean":"2023-10-06T03:34:13.543979+0000","last_became_active":"2023-10-06T03:34:13.541710+0000","last_became_peered":"2023-10-06T03:34:13.541710+0000","last_unstale":"2023-10-06T03:34:13.543979+0000","last_undegraded":"2023-10-06T03:34:13.543979+0000","last_fullsized":"2023-10-06T03:34:13.543979+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.785861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.525061+0000","last_change":"2023-10-06T03:34:12.526945+0000","last_active":"2023-10-06T03:34:13.525061+0000","last_peered":"2023-10-06T03:34:13.525061+0000","last_clean":"2023-10-06T03:34:13.525061+0000","last_became_active":"2023-10-06T03:34:12.526661+0000","last_became_peered":"2023-10-06T03:34:12.526661+0000","last_unstale":"2023-10-06T03:34:13.525061+0000","last_undegraded":"2023-10-06T03:34:13.525061+0000","last_fullsized":"2023-10-06T03:34:13.525061+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:36.885264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.530135+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:34:13.530135+0000","last_peered":"2023-10-06T03:34:13.530135+0000","last_clean":"2023-10-06T03:34:13.530135+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:34:13.530135+0000","last_undegraded":"2023-10-06T03:34:13.530135+0000","last_fullsized":"2023-10-06T03:34:13.530135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.559864+0000","last_change":"2023-10-06T03:34:13.553850+0000","last_active":"2023-10-06T03:34:13.559864+0000","last_peered":"2023-10-06T03:34:13.559864+0000","last_clean":"2023-10-06T03:34:13.559864+0000","last_became_active":"2023-10-06T03:34:13.553534+0000","last_became_peered":"2023-10-06T03:34:13.553534+0000","last_unstale":"2023-10-06T03:34:13.559864+0000","last_undegraded":"2023-10-06T03:34:13.559864+0000","last_fullsized":"2023-10-06T03:34:13.559864+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:00.785861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.530519+0000","last_change":"2023-10-06T03:34:12.529818+0000","last_active":"2023-10-06T03:34:13.530519+0000","last_peered":"2023-10-06T03:34:13.530519+0000","last_clean":"2023-10-06T03:34:13.530519+0000","last_became_active":"2023-10-06T03:34:12.529370+0000","last_became_peered":"2023-10-06T03:34:12.529370+0000","last_unstale":"2023-10-06T03:34:13.530519+0000","last_undegraded":"2023-10-06T03:34:13.530519+0000","last_fullsized":"2023-10-06T03:34:13.530519+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:54.385812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":23,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.531571+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:34:13.531571+0000","last_peered":"2023-10-06T03:34:13.531571+0000","last_clean":"2023-10-06T03:34:13.531571+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:34:13.531571+0000","last_undegraded":"2023-10-06T03:34:13.531571+0000","last_fullsized":"2023-10-06T03:34:13.531571+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.541671+0000","last_change":"2023-10-06T03:34:13.534808+0000","last_active":"2023-10-06T03:34:13.541671+0000","last_peered":"2023-10-06T03:34:13.541671+0000","last_clean":"2023-10-06T03:34:13.541671+0000","last_became_active":"2023-10-06T03:34:13.534549+0000","last_became_peered":"2023-10-06T03:34:13.534549+0000","last_unstale":"2023-10-06T03:34:13.541671+0000","last_undegraded":"2023-10-06T03:34:13.541671+0000","last_fullsized":"2023-10-06T03:34:13.541671+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:50.641614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.522197+0000","last_change":"2023-10-06T03:34:12.516230+0000","last_active":"2023-10-06T03:34:13.522197+0000","last_peered":"2023-10-06T03:34:13.522197+0000","last_clean":"2023-10-06T03:34:13.522197+0000","last_became_active":"2023-10-06T03:34:12.515968+0000","last_became_peered":"2023-10-06T03:34:12.515968+0000","last_unstale":"2023-10-06T03:34:13.522197+0000","last_undegraded":"2023-10-06T03:34:13.522197+0000","last_fullsized":"2023-10-06T03:34:13.522197+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:47.511718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.529216+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:34:13.529216+0000","last_peered":"2023-10-06T03:34:13.529216+0000","last_clean":"2023-10-06T03:34:13.529216+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:34:13.529216+0000","last_undegraded":"2023-10-06T03:34:13.529216+0000","last_fullsized":"2023-10-06T03:34:13.529216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.546934+0000","last_change":"2023-10-06T03:34:13.540397+0000","last_active":"2023-10-06T03:34:13.546934+0000","last_peered":"2023-10-06T03:34:13.546934+0000","last_clean":"2023-10-06T03:34:13.546934+0000","last_became_active":"2023-10-06T03:34:13.540142+0000","last_became_peered":"2023-10-06T03:34:13.540142+0000","last_unstale":"2023-10-06T03:34:13.546934+0000","last_undegraded":"2023-10-06T03:34:13.546934+0000","last_fullsized":"2023-10-06T03:34:13.546934+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:32.081718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.521373+0000","last_change":"2023-10-06T03:34:12.519229+0000","last_active":"2023-10-06T03:34:13.521373+0000","last_peered":"2023-10-06T03:34:13.521373+0000","last_clean":"2023-10-06T03:34:13.521373+0000","last_became_active":"2023-10-06T03:34:12.518975+0000","last_became_peered":"2023-10-06T03:34:12.518975+0000","last_unstale":"2023-10-06T03:34:13.521373+0000","last_undegraded":"2023-10-06T03:34:13.521373+0000","last_fullsized":"2023-10-06T03:34:13.521373+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:37:24.627602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.521342+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:34:13.521342+0000","last_peered":"2023-10-06T03:34:13.521342+0000","last_clean":"2023-10-06T03:34:13.521342+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:34:13.521342+0000","last_undegraded":"2023-10-06T03:34:13.521342+0000","last_fullsized":"2023-10-06T03:34:13.521342+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.537670+0000","last_change":"2023-10-06T03:34:13.532662+0000","last_active":"2023-10-06T03:34:13.537670+0000","last_peered":"2023-10-06T03:34:13.537670+0000","last_clean":"2023-10-06T03:34:13.537670+0000","last_became_active":"2023-10-06T03:34:13.532351+0000","last_became_peered":"2023-10-06T03:34:13.532351+0000","last_unstale":"2023-10-06T03:34:13.537670+0000","last_undegraded":"2023-10-06T03:34:13.537670+0000","last_fullsized":"2023-10-06T03:34:13.537670+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:12.497789+0000","last_clean_scrub_stamp":"2023-10-06T03:34:12.497789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:32:23.077437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.523003+0000","last_change":"2023-10-06T03:34:12.514751+0000","last_active":"2023-10-06T03:34:13.523003+0000","last_peered":"2023-10-06T03:34:13.523003+0000","last_clean":"2023-10-06T03:34:13.523003+0000","last_became_active":"2023-10-06T03:34:12.514468+0000","last_became_peered":"2023-10-06T03:34:12.514468+0000","last_unstale":"2023-10-06T03:34:13.523003+0000","last_undegraded":"2023-10-06T03:34:13.523003+0000","last_fullsized":"2023-10-06T03:34:13.523003+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:34:11.486718+0000","last_clean_scrub_stamp":"2023-10-06T03:34:11.486718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:07.994934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":25,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-10-06T03:34:13.521805+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:34:13.521805+0000","last_peered":"2023-10-06T03:34:13.521805+0000","last_clean":"2023-10-06T03:34:13.521805+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:34:13.521805+0000","last_undegraded":"2023-10-06T03:34:13.521805+0000","last_fullsized":"2023-10-06T03:34:13.521805+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073608,"statfs":{"total":96636764160,"available":96331374592,"internally_reserved":0,"allocated":966656,"data_stored":737317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59099999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.622}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.746}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":146875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":3.3319999999999999,"5min":3.3319999999999999,"15min":3.3319999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.3300000000000001,"5min":3.3300000000000001,"15min":3.3300000000000001},"last":0.77200000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.625}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.68300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302320,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069520,"statfs":{"total":96636764160,"available":96327188480,"internally_reserved":0,"allocated":958464,"data_stored":736779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.45500000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.42799999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.40799999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.56299999999999994}]},{"osd":5,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:34:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.54900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301736,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070104,"statfs":{"total":96636764160,"available":96327786496,"internally_reserved":0,"allocated":360448,"data_stored":145377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.4329999999999998,"5min":2.4329999999999998,"15min":2.4329999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":0.45900000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.439},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.2320000000000002,"5min":2.2320000000000002,"15min":2.2320000000000002},"last":0.85899999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.71699999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.70699999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.93700000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301736,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070104,"statfs":{"total":96636764160,"available":96327786496,"internally_reserved":0,"allocated":360448,"data_stored":145413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.442}]},{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.48599999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.5}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.91000000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.94199999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.98799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301752,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94070088,"statfs":{"total":96636764160,"available":96327770112,"internally_reserved":0,"allocated":376832,"data_stored":147891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.33100000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.442}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.51800000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.80000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":360,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":368640,"data_stored":146897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.433}]},{"osd":1,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.45200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.55500000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.748},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:34:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.77900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297644,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074196,"statfs":{"total":96636764160,"available":96331976704,"internally_reserved":0,"allocated":364544,"data_stored":145455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.877}]},{"osd":2,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.78000000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.751}]},{"osd":6,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":1.01}]},{"osd":7,"last update":"Fri Oct 6 03:34:11 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.66900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:34:23.302 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:34:23.302 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:34:23.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.529+0000 7f0c6939d700 1 Processor -- start 2023-10-06T03:34:23.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.529+0000 7f0c6939d700 1 -- start start 2023-10-06T03:34:23.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c6939d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0c640c7840 0x7f0c64159920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:23.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c6939d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0c640c7c10 0x7f0c64159e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:23.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c6939d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6406e510 con 0x7f0c6415a3a0 2023-10-06T03:34:23.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c6939d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c640c7220 con 0x7f0c640c7c10 2023-10-06T03:34:23.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c6939d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6406df20 con 0x7f0c640c7840 2023-10-06T03:34:23.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c637fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0c640c7c10 0x7f0c64159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:23.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c637fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0c640c7c10 0x7f0c64159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48602/0 (socket says 172.21.15.55:48602) 2023-10-06T03:34:23.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c68b9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0c6415a3a0 0x7f0c6406e9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36336/0 (socket says 172.21.15.55:36336) 2023-10-06T03:34:23.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c637fe700 1 -- 172.21.15.55:0/3155867192 learned_addr learned my addr 172.21.15.55:0/3155867192 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:23.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c63fff700 1 --2- 172.21.15.55:0/3155867192 >> v2:172.21.15.163:3300/0 conn(0x7f0c640c7840 0x7f0c64159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c637fe700 1 -- 172.21.15.55:0/3155867192 >> v2:172.21.15.163:3300/0 conn(0x7f0c640c7840 msgr2=0x7f0c64159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c637fe700 1 --2- 172.21.15.55:0/3155867192 >> v2:172.21.15.163:3300/0 conn(0x7f0c640c7840 0x7f0c64159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c637fe700 1 -- 172.21.15.55:0/3155867192 >> v1:172.21.15.55:6789/0 conn(0x7f0c6415a3a0 legacy=0x7f0c6406e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c637fe700 1 -- 172.21.15.55:0/3155867192 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c64157040 con 0x7f0c640c7c10 2023-10-06T03:34:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c62ffd700 1 -- 172.21.15.55:0/3155867192 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596843961 0 0) 0x7f0c6406e510 con 0x7f0c6415a3a0 2023-10-06T03:34:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.531+0000 7f0c63fff700 1 --2- 172.21.15.55:0/3155867192 >> v2:172.21.15.163:3300/0 conn(0x7f0c640c7840 0x7f0c64159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.532+0000 7f0c637fe700 1 --2- 172.21.15.55:0/3155867192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c640c7c10 0x7f0c64159e60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0c6406c110 tx=0x7f0c58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cb7ef18ec8f060 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:23.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.532+0000 7f0c62ffd700 1 -- 172.21.15.55:0/3155867192 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5800b8f0 con 0x7f0c640c7c10 2023-10-06T03:34:23.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.532+0000 7f0c62ffd700 1 -- 172.21.15.55:0/3155867192 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5800ba50 con 0x7f0c640c7c10 2023-10-06T03:34:23.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.532+0000 7f0c62ffd700 1 -- 172.21.15.55:0/3155867192 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5800bdf0 con 0x7f0c640c7c10 2023-10-06T03:34:23.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.532+0000 7f0c6939d700 1 -- 172.21.15.55:0/3155867192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c640c7c10 msgr2=0x7f0c64159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:23.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.532+0000 7f0c6939d700 1 --2- 172.21.15.55:0/3155867192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c640c7c10 0x7f0c64159e60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0c6406c110 tx=0x7f0c58009250 comp rx=0 tx=0).stop 2023-10-06T03:34:23.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.532+0000 7f0c6939d700 1 -- 172.21.15.55:0/3155867192 shutdown_connections 2023-10-06T03:34:23.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.532+0000 7f0c6939d700 1 --2- 172.21.15.55:0/3155867192 >> v2:172.21.15.163:3300/0 conn(0x7f0c640c7840 0x7f0c64159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.533+0000 7f0c6939d700 1 --2- 172.21.15.55:0/3155867192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c640c7c10 0x7f0c64159e60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.533+0000 7f0c6939d700 1 -- 172.21.15.55:0/3155867192 >> 172.21.15.55:0/3155867192 conn(0x7f0c64068e20 msgr2=0x7f0c640cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:23.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.533+0000 7f0c6939d700 1 -- 172.21.15.55:0/3155867192 shutdown_connections 2023-10-06T03:34:23.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.533+0000 7f0c6939d700 1 -- 172.21.15.55:0/3155867192 wait complete. 2023-10-06T03:34:23.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.534+0000 7f0c6939d700 1 Processor -- start 2023-10-06T03:34:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.534+0000 7f0c6939d700 1 -- start start 2023-10-06T03:34:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.535+0000 7f0c6939d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c640c7840 0x7f0c640cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.535+0000 7f0c6939d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c640c7c10 0x7f0c64154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.535+0000 7f0c6939d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c6415a3a0 0x7f0c640c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.535+0000 7f0c6939d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c640c42c0 con 0x7f0c640c7840 2023-10-06T03:34:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.535+0000 7f0c6939d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c640ccb40 con 0x7f0c6415a3a0 2023-10-06T03:34:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.535+0000 7f0c6939d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c640c36b0 con 0x7f0c640c7c10 2023-10-06T03:34:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.535+0000 7f0c63fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c640c7840 0x7f0c640cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.535+0000 7f0c63fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c640c7840 0x7f0c640cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48614/0 (socket says 172.21.15.55:48614) 2023-10-06T03:34:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.535+0000 7f0c63fff700 1 -- 172.21.15.55:0/1846495711 learned_addr learned my addr 172.21.15.55:0/1846495711 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:23.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.536+0000 7f0c637fe700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c640c7c10 0x7f0c64154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:23.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.536+0000 7f0c68b9c700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c6415a3a0 0x7f0c640c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:23.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.536+0000 7f0c63fff700 1 -- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c640c7c10 msgr2=0x7f0c64154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:23.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.536+0000 7f0c63fff700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c640c7c10 0x7f0c64154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.536+0000 7f0c63fff700 1 -- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c6415a3a0 msgr2=0x7f0c640c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:23.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.536+0000 7f0c63fff700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c6415a3a0 0x7f0c640c4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.536+0000 7f0c63fff700 1 -- 172.21.15.55:0/1846495711 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c580094f0 con 0x7f0c640c7840 2023-10-06T03:34:23.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.536+0000 7f0c637fe700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c640c7c10 0x7f0c64154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:23.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.536+0000 7f0c63fff700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c640c7840 0x7f0c640cc600 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400c850 tx=0x7f0c54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:23.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.536+0000 7f0c4bfff700 1 -- 172.21.15.55:0/1846495711 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54011600 con 0x7f0c640c7840 2023-10-06T03:34:23.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.536+0000 7f0c4bfff700 1 -- 172.21.15.55:0/1846495711 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c54011760 con 0x7f0c640c7840 2023-10-06T03:34:23.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.536+0000 7f0c4bfff700 1 -- 172.21.15.55:0/1846495711 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54011b00 con 0x7f0c640c7840 2023-10-06T03:34:23.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.536+0000 7f0c6939d700 1 -- 172.21.15.55:0/1846495711 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c640c3990 con 0x7f0c640c7840 2023-10-06T03:34:23.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.536+0000 7f0c6939d700 1 -- 172.21.15.55:0/1846495711 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c640c3f50 con 0x7f0c640c7840 2023-10-06T03:34:23.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.537+0000 7f0c6939d700 1 -- 172.21.15.55:0/1846495711 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6404c3c0 con 0x7f0c640c7840 2023-10-06T03:34:23.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.538+0000 7f0c4bfff700 1 -- 172.21.15.55:0/1846495711 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c540073b0 con 0x7f0c640c7840 2023-10-06T03:34:23.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.539+0000 7f0c4bfff700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0c50066f30 0x7f0c500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:23.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.539+0000 7f0c4bfff700 1 -- 172.21.15.55:0/1846495711 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0c54084730 con 0x7f0c640c7840 2023-10-06T03:34:23.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.540+0000 7f0c637fe700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0c50066f30 0x7f0c500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:23.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.540+0000 7f0c637fe700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0c50066f30 0x7f0c500693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0c6406c110 tx=0x7f0c58017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:23.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.543+0000 7f0c4bfff700 1 -- 172.21.15.55:0/1846495711 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c54055230 con 0x7f0c640c7840 2023-10-06T03:34:23.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.823+0000 7f0c6939d700 1 -- 172.21.15.55:0/1846495711 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0c640c45a0 con 0x7f0c640c7840 2023-10-06T03:34:23.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.827+0000 7f0c4bfff700 1 -- 172.21.15.55:0/1846495711 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0c54054dc0 con 0x7f0c640c7840 2023-10-06T03:34:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.829+0000 7f0c6939d700 1 -- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0c50066f30 msgr2=0x7f0c500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.829+0000 7f0c6939d700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0c50066f30 0x7f0c500693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0c6406c110 tx=0x7f0c58017000 comp rx=0 tx=0).stop 2023-10-06T03:34:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.829+0000 7f0c6939d700 1 -- 172.21.15.55:0/1846495711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c640c7840 msgr2=0x7f0c640cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.829+0000 7f0c6939d700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c640c7840 0x7f0c640cc600 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0c5400c850 tx=0x7f0c54008040 comp rx=0 tx=0).stop 2023-10-06T03:34:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.829+0000 7f0c6939d700 1 -- 172.21.15.55:0/1846495711 shutdown_connections 2023-10-06T03:34:23.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.829+0000 7f0c6939d700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c6415a3a0 0x7f0c640c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.829+0000 7f0c6939d700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c640c7840 0x7f0c640cc600 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.829+0000 7f0c6939d700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0c50066f30 0x7f0c500693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.829+0000 7f0c6939d700 1 --2- 172.21.15.55:0/1846495711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c640c7c10 0x7f0c64154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.829+0000 7f0c6939d700 1 -- 172.21.15.55:0/1846495711 >> 172.21.15.55:0/1846495711 conn(0x7f0c64068e20 msgr2=0x7f0c64156740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.830+0000 7f0c6939d700 1 -- 172.21.15.55:0/1846495711 shutdown_connections 2023-10-06T03:34:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:23.830+0000 7f0c6939d700 1 -- 172.21.15.55:0/1846495711 wait complete. 2023-10-06T03:34:23.833 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2023-10-06T03:34:23.849 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:34:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.034+0000 7fdde6b9d700 1 Processor -- start 2023-10-06T03:34:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.034+0000 7fdde6b9d700 1 -- start start 2023-10-06T03:34:24.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.035+0000 7fdde6b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdde01595a0 0x7fdde00b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.035+0000 7fdde6b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdde00c8830 0x7fdde0159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.035+0000 7fdde6b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdde00b1410 con 0x7fdde00c6c60 2023-10-06T03:34:24.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.035+0000 7fdde6b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdde00b2b90 con 0x7fdde01595a0 2023-10-06T03:34:24.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.035+0000 7fdde6b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde0063230 con 0x7fdde00c8830 2023-10-06T03:34:24.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.036+0000 7fdde639c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdde00c8830 0x7fdde0159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.036+0000 7fdde5b9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdde01595a0 0x7fdde00b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.036+0000 7fdde639c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdde00c8830 0x7fdde0159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:41210/0 (socket says 172.21.15.55:41210) 2023-10-06T03:34:24.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.036+0000 7fdde639c700 1 -- 172.21.15.55:0/1399812735 learned_addr learned my addr 172.21.15.55:0/1399812735 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:24.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.036+0000 7fdde539a700 1 --1- 172.21.15.55:0/1399812735 >> v1:172.21.15.163:6789/0 conn(0x7fdde00c6c60 0x7fdde00c8120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:41962/0 (socket says 172.21.15.55:41962) 2023-10-06T03:34:24.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.036+0000 7fdde4b99700 1 -- 172.21.15.55:0/1399812735 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3278042217 0 0) 0x7fdde00b1410 con 0x7fdde00c6c60 2023-10-06T03:34:24.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.036+0000 7fdde639c700 1 -- 172.21.15.55:0/1399812735 >> v2:172.21.15.163:3300/0 conn(0x7fdde01595a0 msgr2=0x7fdde00b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:24.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.036+0000 7fdde639c700 1 --2- 172.21.15.55:0/1399812735 >> v2:172.21.15.163:3300/0 conn(0x7fdde01595a0 0x7fdde00b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.036+0000 7fdde639c700 1 -- 172.21.15.55:0/1399812735 >> v1:172.21.15.163:6789/0 conn(0x7fdde00c6c60 legacy=0x7fdde00c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.036+0000 7fdde639c700 1 -- 172.21.15.55:0/1399812735 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde00634f0 con 0x7fdde00c8830 2023-10-06T03:34:24.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.036+0000 7fdde5b9b700 1 --2- 172.21.15.55:0/1399812735 >> v2:172.21.15.163:3300/0 conn(0x7fdde01595a0 0x7fdde00b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:34:24.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.037+0000 7fdde639c700 1 --2- 172.21.15.55:0/1399812735 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdde00c8830 0x7fdde0159970 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009f20 tx=0x7fddd8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61b730ad7cde8721 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:24.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.037+0000 7fdde4b99700 1 -- 172.21.15.55:0/1399812735 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd800e070 con 0x7fdde00c8830 2023-10-06T03:34:24.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.037+0000 7fdde4b99700 1 -- 172.21.15.55:0/1399812735 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd800b9d0 con 0x7fdde00c8830 2023-10-06T03:34:24.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.037+0000 7fdde4b99700 1 -- 172.21.15.55:0/1399812735 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd80092e0 con 0x7fdde00c8830 2023-10-06T03:34:24.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.038+0000 7fdde6b9d700 1 -- 172.21.15.55:0/1399812735 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdde00c8830 msgr2=0x7fdde0159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.038+0000 7fdde6b9d700 1 --2- 172.21.15.55:0/1399812735 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdde00c8830 0x7fdde0159970 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009f20 tx=0x7fddd8009840 comp rx=0 tx=0).stop 2023-10-06T03:34:24.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.038+0000 7fdde6b9d700 1 -- 172.21.15.55:0/1399812735 shutdown_connections 2023-10-06T03:34:24.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.038+0000 7fdde6b9d700 1 --2- 172.21.15.55:0/1399812735 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdde00c8830 0x7fdde0159970 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.038+0000 7fdde6b9d700 1 --2- 172.21.15.55:0/1399812735 >> v2:172.21.15.163:3300/0 conn(0x7fdde01595a0 0x7fdde00b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.038+0000 7fdde6b9d700 1 -- 172.21.15.55:0/1399812735 >> 172.21.15.55:0/1399812735 conn(0x7fdde006ddb0 msgr2=0x7fdde006e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:24.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.038+0000 7fdde6b9d700 1 -- 172.21.15.55:0/1399812735 shutdown_connections 2023-10-06T03:34:24.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.038+0000 7fdde6b9d700 1 -- 172.21.15.55:0/1399812735 wait complete. 2023-10-06T03:34:24.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.039+0000 7fdde6b9d700 1 Processor -- start 2023-10-06T03:34:24.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.040+0000 7fdde6b9d700 1 -- start start 2023-10-06T03:34:24.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.040+0000 7fdde6b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdde00c6c60 0x7fdde013c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.040+0000 7fdde6b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdde00c8830 0x7fdde014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.040+0000 7fdde6b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde01595a0 0x7fdde0159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.040+0000 7fdde6b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde014cdd0 con 0x7fdde01595a0 2023-10-06T03:34:24.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.040+0000 7fdde6b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdde014dbf0 con 0x7fdde00c8830 2023-10-06T03:34:24.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.040+0000 7fdde6b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdde014cfe0 con 0x7fdde00c6c60 2023-10-06T03:34:24.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.041+0000 7fdde639c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde01595a0 0x7fdde0159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.041+0000 7fdde639c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde01595a0 0x7fdde0159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48626/0 (socket says 172.21.15.55:48626) 2023-10-06T03:34:24.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.041+0000 7fdde639c700 1 -- 172.21.15.55:0/806529801 learned_addr learned my addr 172.21.15.55:0/806529801 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:24.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.041+0000 7fdde539a700 1 --2- 172.21.15.55:0/806529801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdde00c8830 0x7fdde014e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.041+0000 7fdde5b9b700 1 --2- 172.21.15.55:0/806529801 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdde00c6c60 0x7fdde013c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.041+0000 7fdde639c700 1 -- 172.21.15.55:0/806529801 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdde00c6c60 msgr2=0x7fdde013c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.041+0000 7fdde639c700 1 --2- 172.21.15.55:0/806529801 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdde00c6c60 0x7fdde013c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.041+0000 7fdde639c700 1 -- 172.21.15.55:0/806529801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdde00c8830 msgr2=0x7fdde014e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.041+0000 7fdde639c700 1 --2- 172.21.15.55:0/806529801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdde00c8830 0x7fdde014e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.041+0000 7fdde639c700 1 -- 172.21.15.55:0/806529801 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc009420 con 0x7fdde01595a0 2023-10-06T03:34:24.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.042+0000 7fdde639c700 1 --2- 172.21.15.55:0/806529801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde01595a0 0x7fdde0159970 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fddd8012000 tx=0x7fddd8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:24.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.042+0000 7fddd5ffb700 1 -- 172.21.15.55:0/806529801 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd8008c70 con 0x7fdde01595a0 2023-10-06T03:34:24.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.042+0000 7fddd5ffb700 1 -- 172.21.15.55:0/806529801 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd8008dd0 con 0x7fdde01595a0 2023-10-06T03:34:24.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.042+0000 7fddd5ffb700 1 -- 172.21.15.55:0/806529801 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd80266b0 con 0x7fdde01595a0 2023-10-06T03:34:24.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.042+0000 7fdde6b9d700 1 -- 172.21.15.55:0/806529801 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd8009ae0 con 0x7fdde01595a0 2023-10-06T03:34:24.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.042+0000 7fdde6b9d700 1 -- 172.21.15.55:0/806529801 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde014d5a0 con 0x7fdde01595a0 2023-10-06T03:34:24.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.043+0000 7fdde6b9d700 1 -- 172.21.15.55:0/806529801 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdde014dda0 con 0x7fdde01595a0 2023-10-06T03:34:24.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.044+0000 7fddd5ffb700 1 -- 172.21.15.55:0/806529801 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fddd8003040 con 0x7fdde01595a0 2023-10-06T03:34:24.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.045+0000 7fddd5ffb700 1 --2- 172.21.15.55:0/806529801 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fddc4066f30 0x7fddc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.045+0000 7fddd5ffb700 1 -- 172.21.15.55:0/806529801 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fddd8086640 con 0x7fdde01595a0 2023-10-06T03:34:24.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.046+0000 7fdde5b9b700 1 --2- 172.21.15.55:0/806529801 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fddc4066f30 0x7fddc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.046+0000 7fdde5b9b700 1 --2- 172.21.15.55:0/806529801 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fddc4066f30 0x7fddc40693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0093f0 tx=0x7fdddc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:24.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.050+0000 7fddd5ffb700 1 -- 172.21.15.55:0/806529801 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdde014dda0 con 0x7fdde01595a0 2023-10-06T03:34:24.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.297+0000 7fdde6b9d700 1 -- 172.21.15.55:0/806529801 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdde014dd60 con 0x7fdde01595a0 2023-10-06T03:34:24.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.297+0000 7fddd5ffb700 1 -- 172.21.15.55:0/806529801 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fddd8053760 con 0x7fdde01595a0 2023-10-06T03:34:24.300 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:34:24.301 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1532010025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1532010025},{"type":"v1","addr":"172.21.15.55:6834","nonce":1532010025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4258042685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":4258042685},{"type":"v1","addr":"172.21.15.55:6835","nonce":4258042685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/430119478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":430119478},{"type":"v1","addr":"172.21.15.163:6836","nonce":430119478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:34:13.003040+0000","modified":"2023-10-06T03:34:16.548740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4903},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4903":{"gid":4903,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6837/1606384901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1606384901},{"type":"v1","addr":"172.21.15.163:6837","nonce":1606384901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T03:34:24.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.300+0000 7fdde6b9d700 1 -- 172.21.15.55:0/806529801 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fddc4066f30 msgr2=0x7fddc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.300+0000 7fdde6b9d700 1 --2- 172.21.15.55:0/806529801 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fddc4066f30 0x7fddc40693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdddc0093f0 tx=0x7fdddc008040 comp rx=0 tx=0).stop 2023-10-06T03:34:24.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.300+0000 7fdde6b9d700 1 -- 172.21.15.55:0/806529801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde01595a0 msgr2=0x7fdde0159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.300+0000 7fdde6b9d700 1 --2- 172.21.15.55:0/806529801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde01595a0 0x7fdde0159970 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fddd8012000 tx=0x7fddd8018000 comp rx=0 tx=0).stop 2023-10-06T03:34:24.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.300+0000 7fdde6b9d700 1 -- 172.21.15.55:0/806529801 shutdown_connections 2023-10-06T03:34:24.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.300+0000 7fdde6b9d700 1 --2- 172.21.15.55:0/806529801 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdde01595a0 0x7fdde0159970 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.300+0000 7fdde6b9d700 1 --2- 172.21.15.55:0/806529801 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fddc4066f30 0x7fddc40693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.300+0000 7fdde6b9d700 1 --2- 172.21.15.55:0/806529801 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdde00c6c60 0x7fdde013c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.300+0000 7fdde6b9d700 1 --2- 172.21.15.55:0/806529801 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdde00c8830 0x7fdde014e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.300+0000 7fdde6b9d700 1 -- 172.21.15.55:0/806529801 >> 172.21.15.55:0/806529801 conn(0x7fdde006ddb0 msgr2=0x7fdde00c70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:24.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.300+0000 7fdde6b9d700 1 -- 172.21.15.55:0/806529801 shutdown_connections 2023-10-06T03:34:24.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.300+0000 7fdde6b9d700 1 -- 172.21.15.55:0/806529801 wait complete. 2023-10-06T03:34:24.304 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 17 2023-10-06T03:34:24.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:34:13.003040+0000', 'modified': '2023-10-06T03:34:16.548740+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4903}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4903': {'gid': 4903, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.163:6837/1606384901', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6835', 'nonce': 1606384901}, {'type': 'v1', 'addr': '172.21.15.163:6837', 'nonce': 1606384901}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:34:24.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:34:24.722 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T03:34:24.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.917+0000 7f01ca39d700 1 Processor -- start 2023-10-06T03:34:24.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.917+0000 7f01ca39d700 1 -- start start 2023-10-06T03:34:24.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.919+0000 7f01ca39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01c40c6c60 0x7f01c40c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.919+0000 7f01ca39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c40b1410 con 0x7f01c41595a0 2023-10-06T03:34:24.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.919+0000 7f01ca39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c4063230 con 0x7f01c406e8c0 2023-10-06T03:34:24.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.919+0000 7f01ca39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c40b2b90 con 0x7f01c40c6c60 2023-10-06T03:34:24.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.919+0000 7f01c8b9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01c40c6c60 0x7f01c40c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.919+0000 7f01c939b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f01c41595a0 0x7f01c40b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36348/0 (socket says 172.21.15.55:36348) 2023-10-06T03:34:24.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.919+0000 7f01c939b700 1 -- 172.21.15.55:0/2590416078 learned_addr learned my addr 172.21.15.55:0/2590416078 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:24.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.919+0000 7f01c8b9a700 1 -- 172.21.15.55:0/2590416078 >> v1:172.21.15.163:6789/0 conn(0x7f01c406e8c0 legacy=0x7f01c4159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:24.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.919+0000 7f01c8b9a700 1 -- 172.21.15.55:0/2590416078 >> v1:172.21.15.55:6789/0 conn(0x7f01c41595a0 legacy=0x7f01c40b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.919+0000 7f01c8b9a700 1 -- 172.21.15.55:0/2590416078 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c40634f0 con 0x7f01c40c6c60 2023-10-06T03:34:24.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.919+0000 7f01bbfff700 1 -- 172.21.15.55:0/2590416078 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437400668 0 0) 0x7f01c40b1410 con 0x7f01c41595a0 2023-10-06T03:34:24.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.920+0000 7f01c8b9a700 1 --2- 172.21.15.55:0/2590416078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c40c6c60 0x7f01c40c8120 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f01b40098e0 tx=0x7f01b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17df5008ac8217be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:24.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.920+0000 7f01bbfff700 1 -- 172.21.15.55:0/2590416078 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b4009e80 con 0x7f01c40c6c60 2023-10-06T03:34:24.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.920+0000 7f01bbfff700 1 -- 172.21.15.55:0/2590416078 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b400b820 con 0x7f01c40c6c60 2023-10-06T03:34:24.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.920+0000 7f01bbfff700 1 -- 172.21.15.55:0/2590416078 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b400bba0 con 0x7f01c40c6c60 2023-10-06T03:34:24.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.920+0000 7f01ca39d700 1 -- 172.21.15.55:0/2590416078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c40c6c60 msgr2=0x7f01c40c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.920+0000 7f01ca39d700 1 --2- 172.21.15.55:0/2590416078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c40c6c60 0x7f01c40c8120 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f01b40098e0 tx=0x7f01b40091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.920+0000 7f01ca39d700 1 -- 172.21.15.55:0/2590416078 shutdown_connections 2023-10-06T03:34:24.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.920+0000 7f01ca39d700 1 --2- 172.21.15.55:0/2590416078 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c40c6c60 0x7f01c40c8120 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.920+0000 7f01ca39d700 1 -- 172.21.15.55:0/2590416078 >> 172.21.15.55:0/2590416078 conn(0x7f01c406ddb0 msgr2=0x7f01c406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:24.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.921+0000 7f01ca39d700 1 -- 172.21.15.55:0/2590416078 shutdown_connections 2023-10-06T03:34:24.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.921+0000 7f01ca39d700 1 -- 172.21.15.55:0/2590416078 wait complete. 2023-10-06T03:34:24.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.922+0000 7f01ca39d700 1 Processor -- start 2023-10-06T03:34:24.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.922+0000 7f01ca39d700 1 -- start start 2023-10-06T03:34:24.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.923+0000 7f01ca39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c406e8c0 0x7f01c4160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.923+0000 7f01ca39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01c40c6c60 0x7f01c413daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.923+0000 7f01ca39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01c41595a0 0x7f01c4151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.923+0000 7f01ca39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c413d660 con 0x7f01c406e8c0 2023-10-06T03:34:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.923+0000 7f01ca39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c41609b0 con 0x7f01c41595a0 2023-10-06T03:34:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.923+0000 7f01ca39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c413ca50 con 0x7f01c40c6c60 2023-10-06T03:34:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.923+0000 7f01c939b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c406e8c0 0x7f01c4160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.923+0000 7f01c939b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c406e8c0 0x7f01c4160470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48638/0 (socket says 172.21.15.55:48638) 2023-10-06T03:34:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.923+0000 7f01c939b700 1 -- 172.21.15.55:0/4181466490 learned_addr learned my addr 172.21.15.55:0/4181466490 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.923+0000 7f01c9b9c700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01c41595a0 0x7f01c4151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.923+0000 7f01c8b9a700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01c40c6c60 0x7f01c413daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.924+0000 7f01c939b700 1 -- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01c40c6c60 msgr2=0x7f01c413daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.924+0000 7f01c939b700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01c40c6c60 0x7f01c413daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.924+0000 7f01c939b700 1 -- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01c41595a0 msgr2=0x7f01c4151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.924+0000 7f01c939b700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01c41595a0 0x7f01c4151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.924+0000 7f01c939b700 1 -- 172.21.15.55:0/4181466490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b4009420 con 0x7f01c406e8c0 2023-10-06T03:34:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.924+0000 7f01c9b9c700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01c41595a0 0x7f01c4151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.924+0000 7f01c939b700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c406e8c0 0x7f01c4160470 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f01c000c5b0 tx=0x7f01c0001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.924+0000 7f01b97fa700 1 -- 172.21.15.55:0/4181466490 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c0005b30 con 0x7f01c406e8c0 2023-10-06T03:34:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.924+0000 7f01b97fa700 1 -- 172.21.15.55:0/4181466490 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c0005c90 con 0x7f01c406e8c0 2023-10-06T03:34:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.924+0000 7f01b97fa700 1 -- 172.21.15.55:0/4181466490 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c0007320 con 0x7f01c406e8c0 2023-10-06T03:34:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.924+0000 7f01ca39d700 1 -- 172.21.15.55:0/4181466490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c413cd30 con 0x7f01c406e8c0 2023-10-06T03:34:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.924+0000 7f01ca39d700 1 -- 172.21.15.55:0/4181466490 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c413d2f0 con 0x7f01c406e8c0 2023-10-06T03:34:24.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.926+0000 7f01b97fa700 1 -- 172.21.15.55:0/4181466490 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01c001b3b0 con 0x7f01c406e8c0 2023-10-06T03:34:24.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.927+0000 7f01b97fa700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f01a8066f30 0x7f01a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.927+0000 7f01b97fa700 1 -- 172.21.15.55:0/4181466490 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f01c0083b60 con 0x7f01c406e8c0 2023-10-06T03:34:24.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.928+0000 7f01c8b9a700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f01a8066f30 0x7f01a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.928+0000 7f01c8b9a700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f01a8066f30 0x7f01a80693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f01c40c70c0 tx=0x7f01b4016000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:24.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.929+0000 7f01d9dc1b80 1 Processor -- start 2023-10-06T03:34:24.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.930+0000 7f01d9dc1b80 1 -- start start 2023-10-06T03:34:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.930+0000 7f01d9dc1b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252c9e50 0x5586252a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.930+0000 7f01d9dc1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252aa2c0 0x5586252a5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.930+0000 7f01d9dc1b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aba10 0x5586252a5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.930+0000 7f01d9dc1b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586251ffa70 con 0x5586252aba10 2023-10-06T03:34:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.930+0000 7f01d9dc1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586251fd270 con 0x5586252aa2c0 2023-10-06T03:34:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.930+0000 7f01d9dc1b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586251f9060 con 0x5586252c9e50 2023-10-06T03:34:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.931+0000 7f01c9b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aba10 0x5586252a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.931+0000 7f01c9b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aba10 0x5586252a5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48650/0 (socket says 172.21.15.55:48650) 2023-10-06T03:34:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.931+0000 7f01c9b9c700 1 -- 172.21.15.55:0/2075582883 learned_addr learned my addr 172.21.15.55:0/2075582883 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.931+0000 7f01c8b9a700 1 --2- 172.21.15.55:0/2075582883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252aa2c0 0x5586252a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.931+0000 7f01c939b700 1 --2- 172.21.15.55:0/2075582883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252c9e50 0x5586252a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.931+0000 7f01c9b9c700 1 -- 172.21.15.55:0/2075582883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252c9e50 msgr2=0x5586252a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.931+0000 7f01c9b9c700 1 --2- 172.21.15.55:0/2075582883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252c9e50 0x5586252a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.931+0000 7f01c9b9c700 1 -- 172.21.15.55:0/2075582883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252aa2c0 msgr2=0x5586252a5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.931+0000 7f01c9b9c700 1 --2- 172.21.15.55:0/2075582883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252aa2c0 0x5586252a5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.931+0000 7f01c9b9c700 1 -- 172.21.15.55:0/2075582883 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558625093270 con 0x5586252aba10 2023-10-06T03:34:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.931+0000 7f01c9b9c700 1 --2- 172.21.15.55:0/2075582883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aba10 0x5586252a5740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f01bc002e00 tx=0x7f01bc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c119ce2e0bdde server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.932+0000 7f01ae7fc700 1 -- 172.21.15.55:0/2075582883 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc018070 con 0x5586252aba10 2023-10-06T03:34:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.932+0000 7f01ae7fc700 1 -- 172.21.15.55:0/2075582883 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01bc00e990 con 0x5586252aba10 2023-10-06T03:34:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.932+0000 7f01ae7fc700 1 -- 172.21.15.55:0/2075582883 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc00ed30 con 0x5586252aba10 2023-10-06T03:34:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.932+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/2075582883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aba10 msgr2=0x5586252a5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.932+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/2075582883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aba10 0x5586252a5740 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f01bc002e00 tx=0x7f01bc009040 comp rx=0 tx=0).stop 2023-10-06T03:34:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.932+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/2075582883 shutdown_connections 2023-10-06T03:34:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.932+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/2075582883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aba10 0x5586252a5740 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.932+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/2075582883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252c9e50 0x5586252a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.932+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/2075582883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252aa2c0 0x5586252a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.932+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/2075582883 >> 172.21.15.55:0/2075582883 conn(0x5586252a9e80 msgr2=0x5586252aa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.932+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/2075582883 shutdown_connections 2023-10-06T03:34:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.932+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/2075582883 wait complete. 2023-10-06T03:34:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.933+0000 7f01d9dc1b80 1 -- start start 2023-10-06T03:34:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.933+0000 7f01d9dc1b80 1 Processor -- start 2023-10-06T03:34:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.934+0000 7f01d9dc1b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aa2c0 0x5586252f2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.934+0000 7f01d9dc1b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aba10 0x5586252f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.934+0000 7f01d9dc1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252c9e50 0x5586252b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.934+0000 7f01d9dc1b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586251a4890 con 0x5586252aba10 2023-10-06T03:34:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.934+0000 7f01d9dc1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586251bfea0 con 0x5586252c9e50 2023-10-06T03:34:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.934+0000 7f01d9dc1b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55862507be30 con 0x5586252aa2c0 2023-10-06T03:34:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.934+0000 7f01d9dc1b80 10 client.0 _ll_register_callbacks cb 0x55862526c020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:34:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.935+0000 7f01c8b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aba10 0x5586252f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.935+0000 7f01c8b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aba10 0x5586252f3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48656/0 (socket says 172.21.15.55:48656) 2023-10-06T03:34:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.935+0000 7f01c8b9a700 1 -- 172.21.15.55:0/3211900531 learned_addr learned my addr 172.21.15.55:0/3211900531 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.935+0000 7f01c9b9c700 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252c9e50 0x5586252b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.935+0000 7f01c939b700 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aa2c0 0x5586252f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.935+0000 7f01c8b9a700 1 -- 172.21.15.55:0/3211900531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aa2c0 msgr2=0x5586252f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:24.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.935+0000 7f01c8b9a700 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aa2c0 0x5586252f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.935+0000 7f01c8b9a700 1 -- 172.21.15.55:0/3211900531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252c9e50 msgr2=0x5586252b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:24.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.935+0000 7f01c8b9a700 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252c9e50 0x5586252b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.935+0000 7f01c8b9a700 1 -- 172.21.15.55:0/3211900531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01bc00f040 con 0x5586252aba10 2023-10-06T03:34:24.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.935+0000 7f01c9b9c700 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252c9e50 0x5586252b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:24.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.935+0000 7f01c939b700 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aa2c0 0x5586252f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:24.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.935+0000 7f01c8b9a700 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aba10 0x5586252f3b50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f01b40093f0 tx=0x7f01b401f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8011bf2f19f89f7f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:24.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.935+0000 7f01a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:34:24.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.936+0000 7f01a37fe700 1 -- 172.21.15.55:0/3211900531 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b402d070 con 0x5586252aba10 2023-10-06T03:34:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.936+0000 7f01a37fe700 1 -- 172.21.15.55:0/3211900531 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f01b4026450 con 0x5586252aba10 2023-10-06T03:34:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.936+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/3211900531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55862529d940 con 0x5586252aba10 2023-10-06T03:34:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.936+0000 7f01a37fe700 1 -- 172.21.15.55:0/3211900531 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01b4026860 con 0x5586252aba10 2023-10-06T03:34:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.936+0000 7f01d9dc1b80 10 client.5202 fetch_fsmap learned FSMap version 17 2023-10-06T03:34:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.936+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/3211900531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x558625035fe0 con 0x5586252aba10 2023-10-06T03:34:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.936+0000 7f01a37fe700 1 -- 172.21.15.55:0/3211900531 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55862529d940 con 0x5586252aba10 2023-10-06T03:34:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.936+0000 7f01a37fe700 1 -- 172.21.15.55:0/3211900531 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f01b40269c0 con 0x5586252aba10 2023-10-06T03:34:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.936+0000 7f01d9dc1b80 10 client.5202 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T03:34:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.936+0000 7f01d9dc1b80 10 client.5202 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T03:34:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.936+0000 7f01d9dc1b80 20 client.5202 populate_metadata read hostname 'smithi055' 2023-10-06T03:34:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.936+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.163:6835/1606384901,v1:172.21.15.163:6837/1606384901] conn(0x5586252d7370 0x5586252d7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.936+0000 7f01d9dc1b80 4 client.5202 mds_command: new command op to 4903 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:34:24.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.936+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/3211900531 --> [v2:172.21.15.163:6835/1606384901,v1:172.21.15.163:6837/1606384901] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55862527a870 con 0x5586252d7370 2023-10-06T03:34:24.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.937+0000 7f01c939b700 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.163:6835/1606384901,v1:172.21.15.163:6837/1606384901] conn(0x5586252d7370 0x5586252d7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.937+0000 7f01c939b700 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.163:6835/1606384901,v1:172.21.15.163:6837/1606384901] conn(0x5586252d7370 0x5586252d7740 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d11bfd8144d3f2eb server_cookie=a551d6bc2a8cfd1c in_seq=0 out_seq=0 2023-10-06T03:34:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.937+0000 7f01a37fe700 10 client.5202 ms_handle_connect on v2:172.21.15.163:6835/1606384901 2023-10-06T03:34:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.938+0000 7f01a37fe700 1 -- 172.21.15.55:0/3211900531 <== mds.0 v2:172.21.15.163:6835/1606384901 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55862527a870 con 0x5586252d7370 2023-10-06T03:34:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.938+0000 7f01a37fe700 10 client.5202 handle_command_reply: tid=0 2023-10-06T03:34:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.938+0000 7f01d9dc1b80 1 client.5202 shutdown 2023-10-06T03:34:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.938+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/3211900531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aba10 msgr2=0x5586252f3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.938+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aba10 0x5586252f3b50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f01b40093f0 tx=0x7f01b401f040 comp rx=0 tx=0).stop 2023-10-06T03:34:24.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.939+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/3211900531 shutdown_connections 2023-10-06T03:34:24.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.939+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.163:6835/1606384901,v1:172.21.15.163:6837/1606384901] conn(0x5586252d7370 0x5586252d7740 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.939+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252c9e50 0x5586252b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.939+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aa2c0 0x5586252f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.939+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/3211900531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aba10 0x5586252f3b50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.939+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/3211900531 >> 172.21.15.55:0/3211900531 conn(0x5586252a9e80 msgr2=0x5586252aad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:24.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.939+0000 7f01a37fe700 0 client.5202 ms_handle_reset on v2:172.21.15.163:6835/1606384901 2023-10-06T03:34:24.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.939+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/3211900531 shutdown_connections 2023-10-06T03:34:24.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.939+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/3211900531 wait complete. 2023-10-06T03:34:24.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.939+0000 7f01d9dc1b80 20 client.5202 trim_cache size 0 max 16384 2023-10-06T03:34:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.956+0000 7f01d9dc1b80 1 Processor -- start 2023-10-06T03:34:24.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.957+0000 7f01d9dc1b80 1 -- start start 2023-10-06T03:34:24.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.957+0000 7f01d9dc1b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aa2c0 0x5586252b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.957+0000 7f01d9dc1b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aba10 0x5586252ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.957+0000 7f01d9dc1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252c9e50 0x5586252b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.957+0000 7f01d9dc1b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586252a6820 con 0x5586252aa2c0 2023-10-06T03:34:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.957+0000 7f01d9dc1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586252a9cb0 con 0x5586252c9e50 2023-10-06T03:34:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.957+0000 7f01d9dc1b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586252c7d20 con 0x5586252aba10 2023-10-06T03:34:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01c939b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aa2c0 0x5586252b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01c939b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aa2c0 0x5586252b85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48668/0 (socket says 172.21.15.55:48668) 2023-10-06T03:34:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01c939b700 1 -- 172.21.15.55:0/61418231 learned_addr learned my addr 172.21.15.55:0/61418231 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01c9b9c700 1 --2- 172.21.15.55:0/61418231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252c9e50 0x5586252b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01c8b9a700 1 --2- 172.21.15.55:0/61418231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aba10 0x5586252ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01c939b700 1 -- 172.21.15.55:0/61418231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aba10 msgr2=0x5586252ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01c939b700 1 --2- 172.21.15.55:0/61418231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aba10 0x5586252ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01c939b700 1 -- 172.21.15.55:0/61418231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252c9e50 msgr2=0x5586252b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01c939b700 1 --2- 172.21.15.55:0/61418231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252c9e50 0x5586252b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01c939b700 1 -- 172.21.15.55:0/61418231 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586252a6170 con 0x5586252aa2c0 2023-10-06T03:34:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01c9b9c700 1 --2- 172.21.15.55:0/61418231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252c9e50 0x5586252b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01c8b9a700 1 --2- 172.21.15.55:0/61418231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aba10 0x5586252ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01c939b700 1 --2- 172.21.15.55:0/61418231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aa2c0 0x5586252b85f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f01c0054630 tx=0x7f01c0081e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b2ec3f929f209e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01aeffd700 1 -- 172.21.15.55:0/61418231 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c0054930 con 0x5586252aa2c0 2023-10-06T03:34:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01aeffd700 1 -- 172.21.15.55:0/61418231 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c0057b80 con 0x5586252aa2c0 2023-10-06T03:34:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.958+0000 7f01aeffd700 1 -- 172.21.15.55:0/61418231 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c005b8d0 con 0x5586252aa2c0 2023-10-06T03:34:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.959+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/61418231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aa2c0 msgr2=0x5586252b85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.959+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/61418231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aa2c0 0x5586252b85f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f01c0054630 tx=0x7f01c0081e80 comp rx=0 tx=0).stop 2023-10-06T03:34:24.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.959+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/61418231 shutdown_connections 2023-10-06T03:34:24.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.959+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/61418231 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252c9e50 0x5586252b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.959+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/61418231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252aa2c0 0x5586252b85f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.959+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/61418231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aba10 0x5586252ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.959+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/61418231 >> 172.21.15.55:0/61418231 conn(0x5586252a9e80 msgr2=0x5586252aa690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:24.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.959+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/61418231 shutdown_connections 2023-10-06T03:34:24.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.959+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/61418231 wait complete. 2023-10-06T03:34:24.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.960+0000 7f01d9dc1b80 1 -- start start 2023-10-06T03:34:24.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.960+0000 7f01d9dc1b80 1 Processor -- start 2023-10-06T03:34:24.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.961+0000 7f01d9dc1b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aa2c0 0x5586252c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.961+0000 7f01d9dc1b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252aba10 0x5586252ef660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.961+0000 7f01d9dc1b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252c9e50 0x5586252f1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.961+0000 7f01d9dc1b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586252c8d30 con 0x5586252c9e50 2023-10-06T03:34:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.961+0000 7f01d9dc1b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558625355700 con 0x5586252aba10 2023-10-06T03:34:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.961+0000 7f01d9dc1b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586252c9160 con 0x5586252aa2c0 2023-10-06T03:34:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.961+0000 7f01d9dc1b80 10 client.0 _ll_register_callbacks cb 0x55862526c020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:34:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.961+0000 7f01c9b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252c9e50 0x5586252f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.961+0000 7f01c939b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aa2c0 0x5586252c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.961+0000 7f01c8b9a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252aba10 0x5586252ef660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.961+0000 7f01c939b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aa2c0 0x5586252c87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:41272/0 (socket says 172.21.15.55:41272) 2023-10-06T03:34:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.961+0000 7f01c939b700 1 -- 172.21.15.55:0/1384975487 learned_addr learned my addr 172.21.15.55:0/1384975487 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.961+0000 7f01c9b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252c9e50 0x5586252f1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48670/0 (socket says 172.21.15.55:48670) 2023-10-06T03:34:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.962+0000 7f01c9b9c700 1 -- 172.21.15.55:0/1384975487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aa2c0 msgr2=0x5586252c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.962+0000 7f01c9b9c700 1 --2- 172.21.15.55:0/1384975487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aa2c0 0x5586252c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.962+0000 7f01c9b9c700 1 -- 172.21.15.55:0/1384975487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252aba10 msgr2=0x5586252ef660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.962+0000 7f01c9b9c700 1 --2- 172.21.15.55:0/1384975487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252aba10 0x5586252ef660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.962+0000 7f01c9b9c700 1 -- 172.21.15.55:0/1384975487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c413d2f0 con 0x5586252c9e50 2023-10-06T03:34:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.962+0000 7f01c9b9c700 1 --2- 172.21.15.55:0/1384975487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252c9e50 0x5586252f1730 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f01bc017000 tx=0x7f01bc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1692bec31e09462c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.962+0000 7f01a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:34:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.962+0000 7f01a37fe700 1 -- 172.21.15.55:0/1384975487 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc0029e0 con 0x5586252c9e50 2023-10-06T03:34:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.962+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/1384975487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01bc00e990 con 0x5586252c9e50 2023-10-06T03:34:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.962+0000 7f01a37fe700 1 -- 172.21.15.55:0/1384975487 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f01bc006db0 con 0x5586252c9e50 2023-10-06T03:34:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.962+0000 7f01a37fe700 1 -- 172.21.15.55:0/1384975487 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01bc002540 con 0x5586252c9e50 2023-10-06T03:34:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.962+0000 7f01d9dc1b80 10 client.5208 fetch_fsmap learned FSMap version 17 2023-10-06T03:34:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.962+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/1384975487 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5586251c7780 con 0x5586252c9e50 2023-10-06T03:34:24.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.963+0000 7f01a37fe700 1 -- 172.21.15.55:0/1384975487 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01bc00e990 con 0x5586252c9e50 2023-10-06T03:34:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.963+0000 7f01a37fe700 1 -- 172.21.15.55:0/1384975487 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f01bc014690 con 0x5586252c9e50 2023-10-06T03:34:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.963+0000 7f01d9dc1b80 10 client.5208 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T03:34:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.963+0000 7f01d9dc1b80 10 client.5208 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T03:34:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.963+0000 7f01d9dc1b80 20 client.5208 populate_metadata read hostname 'smithi055' 2023-10-06T03:34:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.963+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/1384975487 >> [v2:172.21.15.163:6835/1606384901,v1:172.21.15.163:6837/1606384901] conn(0x5586252f1c70 0x5586252f2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.963+0000 7f01d9dc1b80 4 client.5208 mds_command: new command op to 4903 tid=0[{"prefix": "status"}] 2023-10-06T03:34:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.963+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/1384975487 --> [v2:172.21.15.163:6835/1606384901,v1:172.21.15.163:6837/1606384901] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55862519be00 con 0x5586252f1c70 2023-10-06T03:34:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.963+0000 7f01c939b700 1 --2- 172.21.15.55:0/1384975487 >> [v2:172.21.15.163:6835/1606384901,v1:172.21.15.163:6837/1606384901] conn(0x5586252f1c70 0x5586252f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01c939b700 1 --2- 172.21.15.55:0/1384975487 >> [v2:172.21.15.163:6835/1606384901,v1:172.21.15.163:6837/1606384901] conn(0x5586252f1c70 0x5586252f2040 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=229096fcfb0fc2e3 server_cookie=1ed2a40f32d7bc40 in_seq=0 out_seq=0 2023-10-06T03:34:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01a37fe700 10 client.5208 ms_handle_connect on v2:172.21.15.163:6835/1606384901 2023-10-06T03:34:24.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01a37fe700 1 -- 172.21.15.55:0/1384975487 <== mds.0 v2:172.21.15.163:6835/1606384901 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55862519be00 con 0x5586252f1c70 2023-10-06T03:34:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01a37fe700 10 client.5208 handle_command_reply: tid=0 2023-10-06T03:34:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01d9dc1b80 1 client.5208 shutdown 2023-10-06T03:34:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/1384975487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252c9e50 msgr2=0x5586252f1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/1384975487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252c9e50 0x5586252f1730 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f01bc017000 tx=0x7f01bc01a000 comp rx=0 tx=0).stop 2023-10-06T03:34:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/1384975487 shutdown_connections 2023-10-06T03:34:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/1384975487 >> [v2:172.21.15.163:6835/1606384901,v1:172.21.15.163:6837/1606384901] conn(0x5586252f1c70 0x5586252f2040 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/1384975487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5586252c9e50 0x5586252f1730 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/1384975487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5586252aa2c0 0x5586252c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01d9dc1b80 1 --2- 172.21.15.55:0/1384975487 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5586252aba10 0x5586252ef660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01a37fe700 0 client.5208 ms_handle_reset on v2:172.21.15.163:6835/1606384901 2023-10-06T03:34:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/1384975487 >> 172.21.15.55:0/1384975487 conn(0x5586252a9e80 msgr2=0x5586252a8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/1384975487 shutdown_connections 2023-10-06T03:34:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.964+0000 7f01d9dc1b80 1 -- 172.21.15.55:0/1384975487 wait complete. 2023-10-06T03:34:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.965+0000 7f01d9dc1b80 20 client.5208 trim_cache size 0 max 16384 2023-10-06T03:34:24.973 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:34:24.973 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "8132c8ba-ee1c-41a2-b781-913925034ab6", 2023-10-06T03:34:24.973 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2023-10-06T03:34:24.973 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 4903, 2023-10-06T03:34:24.973 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2023-10-06T03:34:24.973 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2023-10-06T03:34:24.973 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2023-10-06T03:34:24.974 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 11.447813168, 2023-10-06T03:34:24.974 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 17, 2023-10-06T03:34:24.974 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 32, 2023-10-06T03:34:24.974 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 32, 2023-10-06T03:34:24.974 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 20.501944949999999 2023-10-06T03:34:24.974 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.965+0000 7f01ca39d700 1 -- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f01a8066f30 msgr2=0x7f01a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.965+0000 7f01ca39d700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f01a8066f30 0x7f01a80693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f01c40c70c0 tx=0x7f01b4016000 comp rx=0 tx=0).stop 2023-10-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.965+0000 7f01ca39d700 1 -- 172.21.15.55:0/4181466490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c406e8c0 msgr2=0x7f01c4160470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.965+0000 7f01ca39d700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c406e8c0 0x7f01c4160470 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f01c000c5b0 tx=0x7f01c0001070 comp rx=0 tx=0).stop 2023-10-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.965+0000 7f01ca39d700 1 -- 172.21.15.55:0/4181466490 shutdown_connections 2023-10-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.965+0000 7f01ca39d700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01c41595a0 0x7f01c4151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.965+0000 7f01ca39d700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01c406e8c0 0x7f01c4160470 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.965+0000 7f01ca39d700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f01a8066f30 0x7f01a80693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.965+0000 7f01ca39d700 1 --2- 172.21.15.55:0/4181466490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01c40c6c60 0x7f01c413daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:24.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.965+0000 7f01ca39d700 1 -- 172.21.15.55:0/4181466490 >> 172.21.15.55:0/4181466490 conn(0x7f01c406ddb0 msgr2=0x7f01c4157200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:24.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.965+0000 7f01ca39d700 1 -- 172.21.15.55:0/4181466490 shutdown_connections 2023-10-06T03:34:24.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:24.965+0000 7f01ca39d700 1 -- 172.21.15.55:0/4181466490 wait complete. 2023-10-06T03:34:24.984 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:34:25.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.168+0000 7f60f2d9d700 1 Processor -- start 2023-10-06T03:34:25.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.168+0000 7f60f2d9d700 1 -- start start 2023-10-06T03:34:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.170+0000 7f60f2d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60ec156310 0x7f60ec0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.170+0000 7f60f2d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f60ec1566e0 0x7f60ec0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.170+0000 7f60f2d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60ec0ca860 con 0x7f60ec0ca300 2023-10-06T03:34:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.170+0000 7f60f2d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60ec0cab40 con 0x7f60ec1566e0 2023-10-06T03:34:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.170+0000 7f60f2d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec0cae20 con 0x7f60ec156310 2023-10-06T03:34:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.170+0000 7f60f159a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f60ec1566e0 0x7f60ec0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.170+0000 7f60f159a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f60ec1566e0 0x7f60ec0ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48684/0 (socket says 172.21.15.55:48684) 2023-10-06T03:34:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.170+0000 7f60f159a700 1 -- 172.21.15.55:0/3014388911 learned_addr learned my addr 172.21.15.55:0/3014388911 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:25.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.170+0000 7f60f259c700 1 --1- 172.21.15.55:0/3014388911 >> v1:172.21.15.55:6789/0 conn(0x7f60ec0ca300 0x7f60ec0c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36350/0 (socket says 172.21.15.55:36350) 2023-10-06T03:34:25.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.170+0000 7f60f1d9b700 1 --2- 172.21.15.55:0/3014388911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60ec156310 0x7f60ec0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:25.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.170+0000 7f60f159a700 1 -- 172.21.15.55:0/3014388911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60ec156310 msgr2=0x7f60ec0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:25.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.170+0000 7f60f159a700 1 --2- 172.21.15.55:0/3014388911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60ec156310 0x7f60ec0c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:25.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.170+0000 7f60f159a700 1 -- 172.21.15.55:0/3014388911 >> v1:172.21.15.55:6789/0 conn(0x7f60ec0ca300 legacy=0x7f60ec0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:25.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.170+0000 7f60f0d99700 1 -- 172.21.15.55:0/3014388911 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2847725516 0 0) 0x7f60ec0ca860 con 0x7f60ec0ca300 2023-10-06T03:34:25.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.171+0000 7f60f159a700 1 -- 172.21.15.55:0/3014388911 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec0cb100 con 0x7f60ec1566e0 2023-10-06T03:34:25.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.171+0000 7f60f159a700 1 --2- 172.21.15.55:0/3014388911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60ec1566e0 0x7f60ec0ae000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0099b0 tx=0x7f60dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ddf7bad3f0b046 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:25.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.171+0000 7f60f0d99700 1 -- 172.21.15.55:0/3014388911 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00b860 con 0x7f60ec1566e0 2023-10-06T03:34:25.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.171+0000 7f60f0d99700 1 -- 172.21.15.55:0/3014388911 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc00b9c0 con 0x7f60ec1566e0 2023-10-06T03:34:25.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.171+0000 7f60f0d99700 1 -- 172.21.15.55:0/3014388911 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00bd60 con 0x7f60ec1566e0 2023-10-06T03:34:25.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.171+0000 7f60f2d9d700 1 -- 172.21.15.55:0/3014388911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60ec1566e0 msgr2=0x7f60ec0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:25.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.171+0000 7f60f2d9d700 1 --2- 172.21.15.55:0/3014388911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60ec1566e0 0x7f60ec0ae000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0099b0 tx=0x7f60dc009250 comp rx=0 tx=0).stop 2023-10-06T03:34:25.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.172+0000 7f60f2d9d700 1 -- 172.21.15.55:0/3014388911 shutdown_connections 2023-10-06T03:34:25.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.172+0000 7f60f2d9d700 1 --2- 172.21.15.55:0/3014388911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60ec156310 0x7f60ec0c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:25.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.172+0000 7f60f2d9d700 1 --2- 172.21.15.55:0/3014388911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60ec1566e0 0x7f60ec0ae000 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:25.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.172+0000 7f60f2d9d700 1 -- 172.21.15.55:0/3014388911 >> 172.21.15.55:0/3014388911 conn(0x7f60ec0cc1f0 msgr2=0x7f60ec0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:25.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.172+0000 7f60f2d9d700 1 -- 172.21.15.55:0/3014388911 shutdown_connections 2023-10-06T03:34:25.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.172+0000 7f60f2d9d700 1 -- 172.21.15.55:0/3014388911 wait complete. 2023-10-06T03:34:25.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.173+0000 7f60f2d9d700 1 Processor -- start 2023-10-06T03:34:25.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.173+0000 7f60f2d9d700 1 -- start start 2023-10-06T03:34:25.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.174+0000 7f60f2d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60ec0ca300 0x7f60ec15d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:25.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.174+0000 7f60f2d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60ec156310 0x7f60ec15dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:25.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.174+0000 7f60f2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60ec1566e0 0x7f60ec13cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:25.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.174+0000 7f60f2d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec15c110 con 0x7f60ec156310 2023-10-06T03:34:25.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.174+0000 7f60f2d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec15c3f0 con 0x7f60ec1566e0 2023-10-06T03:34:25.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.174+0000 7f60f2d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60ec15c6d0 con 0x7f60ec0ca300 2023-10-06T03:34:25.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.174+0000 7f60f159a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60ec156310 0x7f60ec15dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:25.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.174+0000 7f60f159a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60ec156310 0x7f60ec15dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48694/0 (socket says 172.21.15.55:48694) 2023-10-06T03:34:25.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.174+0000 7f60f159a700 1 -- 172.21.15.55:0/2159577849 learned_addr learned my addr 172.21.15.55:0/2159577849 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:25.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.175+0000 7f60f1d9b700 1 --2- 172.21.15.55:0/2159577849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60ec0ca300 0x7f60ec15d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:25.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.175+0000 7f60f259c700 1 --2- 172.21.15.55:0/2159577849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60ec1566e0 0x7f60ec13cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:25.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.175+0000 7f60f159a700 1 -- 172.21.15.55:0/2159577849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60ec0ca300 msgr2=0x7f60ec15d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:25.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.175+0000 7f60f159a700 1 --2- 172.21.15.55:0/2159577849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60ec0ca300 0x7f60ec15d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:25.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.175+0000 7f60f159a700 1 -- 172.21.15.55:0/2159577849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60ec1566e0 msgr2=0x7f60ec13cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:25.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.175+0000 7f60f159a700 1 --2- 172.21.15.55:0/2159577849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60ec1566e0 0x7f60ec13cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:25.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.175+0000 7f60f159a700 1 -- 172.21.15.55:0/2159577849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60dc0094f0 con 0x7f60ec156310 2023-10-06T03:34:25.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.175+0000 7f60f159a700 1 --2- 172.21.15.55:0/2159577849 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60ec156310 0x7f60ec15dc60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009290 tx=0x7f60dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:25.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.175+0000 7f60e1ffb700 1 -- 172.21.15.55:0/2159577849 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc024440 con 0x7f60ec156310 2023-10-06T03:34:25.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.175+0000 7f60e1ffb700 1 -- 172.21.15.55:0/2159577849 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc0245c0 con 0x7f60ec156310 2023-10-06T03:34:25.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.175+0000 7f60e1ffb700 1 -- 172.21.15.55:0/2159577849 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc024980 con 0x7f60ec156310 2023-10-06T03:34:25.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.175+0000 7f60f2d9d700 1 -- 172.21.15.55:0/2159577849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60ec15c9b0 con 0x7f60ec156310 2023-10-06T03:34:25.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.176+0000 7f60f2d9d700 1 -- 172.21.15.55:0/2159577849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60ec15d6f0 con 0x7f60ec156310 2023-10-06T03:34:25.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.177+0000 7f60f2d9d700 1 -- 172.21.15.55:0/2159577849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60ec15ced0 con 0x7f60ec156310 2023-10-06T03:34:25.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.177+0000 7f60e1ffb700 1 -- 172.21.15.55:0/2159577849 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f60dc01a420 con 0x7f60ec156310 2023-10-06T03:34:25.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.178+0000 7f60e1ffb700 1 --2- 172.21.15.55:0/2159577849 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f60d0066f60 0x7f60d0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:25.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.178+0000 7f60e1ffb700 1 -- 172.21.15.55:0/2159577849 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f60dc086530 con 0x7f60ec156310 2023-10-06T03:34:25.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.179+0000 7f60f1d9b700 1 --2- 172.21.15.55:0/2159577849 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f60d0066f60 0x7f60d0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:25.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.180+0000 7f60f1d9b700 1 --2- 172.21.15.55:0/2159577849 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f60d0066f60 0x7f60d0069410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f60e8006fd0 tx=0x7f60e8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:25.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.183+0000 7f60e1ffb700 1 -- 172.21.15.55:0/2159577849 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60ec15ced0 con 0x7f60ec156310 2023-10-06T03:34:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.430+0000 7f60f2d9d700 1 -- 172.21.15.55:0/2159577849 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f60ec15ced0 con 0x7f60ec156310 2023-10-06T03:34:25.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.430+0000 7f60e1ffb700 1 -- 172.21.15.55:0/2159577849 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f60ec15ced0 con 0x7f60ec156310 2023-10-06T03:34:25.433 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:34:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.432+0000 7f60f2d9d700 1 -- 172.21.15.55:0/2159577849 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f60d0066f60 msgr2=0x7f60d0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.432+0000 7f60f2d9d700 1 --2- 172.21.15.55:0/2159577849 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f60d0066f60 0x7f60d0069410 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f60e8006fd0 tx=0x7f60e8008040 comp rx=0 tx=0).stop 2023-10-06T03:34:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.432+0000 7f60f2d9d700 1 -- 172.21.15.55:0/2159577849 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60ec156310 msgr2=0x7f60ec15dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.432+0000 7f60f2d9d700 1 --2- 172.21.15.55:0/2159577849 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60ec156310 0x7f60ec15dc60 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009290 tx=0x7f60dc017000 comp rx=0 tx=0).stop 2023-10-06T03:34:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.432+0000 7f60f2d9d700 1 -- 172.21.15.55:0/2159577849 shutdown_connections 2023-10-06T03:34:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.432+0000 7f60f2d9d700 1 --2- 172.21.15.55:0/2159577849 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60ec1566e0 0x7f60ec13cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.432+0000 7f60f2d9d700 1 --2- 172.21.15.55:0/2159577849 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f60d0066f60 0x7f60d0069410 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.432+0000 7f60f2d9d700 1 --2- 172.21.15.55:0/2159577849 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60ec0ca300 0x7f60ec15d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:25.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.432+0000 7f60f2d9d700 1 --2- 172.21.15.55:0/2159577849 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60ec156310 0x7f60ec15dc60 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:25.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.432+0000 7f60f2d9d700 1 -- 172.21.15.55:0/2159577849 >> 172.21.15.55:0/2159577849 conn(0x7f60ec0cc1f0 msgr2=0x7f60ec06ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:25.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.432+0000 7f60f2d9d700 1 -- 172.21.15.55:0/2159577849 shutdown_connections 2023-10-06T03:34:25.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:25.432+0000 7f60f2d9d700 1 -- 172.21.15.55:0/2159577849 wait complete. 2023-10-06T03:34:25.452 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:34:25.452 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:34:25.452 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2023-10-06T03:34:25.453 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:34:25.453 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:34:25.453 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:34:25.453 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:34:25.453 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:34:25.453 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:34:25.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:34:25.453 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2023-10-06T03:34:25.491 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:34:25.491 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:34:25.491 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:34:25.491 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:34:25.492 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T03:34:25.492 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:34:25.492 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:34:25.492 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:34:25.492 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2023-10-06T03:34:25.492 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:34:25.492 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:34:25.492 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2023-10-06T03:34:25.492 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:34:25.493 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:34:25.493 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2023-10-06T03:34:25.493 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T03:34:25.493 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 42176sec preferred_lft 42176sec 2023-10-06T03:34:25.493 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link 2023-10-06T03:34:25.493 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:34:25.493 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:34:25.493 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:34:25.493 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2023-10-06T03:34:25.493 INFO:teuthology.orchestra.run.smithi055.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:34:25.494 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff 2023-10-06T03:34:25.494 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:34:25.494 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:34:25.494 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::1099:8eff:fe6d:102/64 scope link 2023-10-06T03:34:25.494 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:34:25.494 INFO:teuthology.orchestra.run.smithi055.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:34:25.494 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether f2:5d:53:d2:00:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:34:25.494 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f05d:53ff:fed2:a/64 scope link 2023-10-06T03:34:25.494 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:34:25.494 INFO:teuthology.orchestra.run.smithi055.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:34:25.494 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:34:25.494 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::e037:33ff:feb7:6c8f/64 scope link 2023-10-06T03:34:25.495 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:34:25.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:34:25.495 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:34:25.593 INFO:teuthology.orchestra.run.smithi055.stdout:1 2023-10-06T03:34:25.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:34:25.596 DEBUG:teuthology.orchestra.run.smithi055:> route 2023-10-06T03:34:25.634 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2023-10-06T03:34:25.635 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:34:25.635 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T03:34:25.635 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T03:34:25.635 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:34:25.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:34:25.636 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:34:25.636 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:34:25.636 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T03:34:25.636 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T03:34:25.636 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T03:34:25.636 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:34:25.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:34:25.835 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T03:34:25.871 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:34:25.871 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:34:25.872 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:34:25.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:34:25.872 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2023-10-06T03:34:25.974 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:34:25.974 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:34:26.010 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:34:26.011 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:34:26.011 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:34:26.087 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T03:34:26.087 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:34:26.087 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:34:26.219 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:34:26.219 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:34:26.219 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:34:26.235 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:34:26.235 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:34:26.235 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:34:26.235 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:34:26.236 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:34:26.257 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:34:26.258 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:34:26.258 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:34:26.258 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:34:26.258 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:34:26.258 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T03:34:26.258 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T03:34:26.258 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T03:34:26.262 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:34:26.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:34:26.262 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:34:26.325 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:34:26.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.512+0000 7f4024b9d700 1 Processor -- start 2023-10-06T03:34:26.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.512+0000 7f4024b9d700 1 -- start start 2023-10-06T03:34:26.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.513+0000 7f4024b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4020159420 0x7f40200b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:26.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.513+0000 7f4024b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40200b2980 con 0x7f40201597f0 2023-10-06T03:34:26.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.513+0000 7f4024b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40200b11d0 con 0x7f402015d230 2023-10-06T03:34:26.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.513+0000 7f4024b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40200b14b0 con 0x7f4020159420 2023-10-06T03:34:26.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.514+0000 7f401effd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f40201597f0 0x7f40200b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36364/0 (socket says 172.21.15.55:36364) 2023-10-06T03:34:26.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.514+0000 7f401effd700 1 -- 172.21.15.55:0/2304009741 learned_addr learned my addr 172.21.15.55:0/2304009741 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:26.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.514+0000 7f401f7fe700 1 --2- 172.21.15.55:0/2304009741 >> v2:172.21.15.163:3300/0 conn(0x7f4020159420 0x7f40200b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:26.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.514+0000 7f401e7fc700 1 -- 172.21.15.55:0/2304009741 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734707660 0 0) 0x7f40200b2980 con 0x7f40201597f0 2023-10-06T03:34:26.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.514+0000 7f401e7fc700 1 -- 172.21.15.55:0/2304009741 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4008003570 con 0x7f40201597f0 2023-10-06T03:34:26.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.514+0000 7f401f7fe700 1 -- 172.21.15.55:0/2304009741 >> v1:172.21.15.163:6789/0 conn(0x7f402015d230 legacy=0x7f402015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:26.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.514+0000 7f401f7fe700 1 -- 172.21.15.55:0/2304009741 >> v1:172.21.15.55:6789/0 conn(0x7f40201597f0 legacy=0x7f40200b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:26.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.514+0000 7f401f7fe700 1 -- 172.21.15.55:0/2304009741 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40200b1790 con 0x7f4020159420 2023-10-06T03:34:26.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.515+0000 7f401f7fe700 1 --2- 172.21.15.55:0/2304009741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4020159420 0x7f40200b5f40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f40100098e0 tx=0x7f40100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c7366af86e94521 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:26.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.515+0000 7f401e7fc700 1 -- 172.21.15.55:0/2304009741 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401000b640 con 0x7f4020159420 2023-10-06T03:34:26.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.515+0000 7f401e7fc700 1 -- 172.21.15.55:0/2304009741 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401000b7a0 con 0x7f4020159420 2023-10-06T03:34:26.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.515+0000 7f401e7fc700 1 -- 172.21.15.55:0/2304009741 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401000bb40 con 0x7f4020159420 2023-10-06T03:34:26.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.515+0000 7f4024b9d700 1 -- 172.21.15.55:0/2304009741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4020159420 msgr2=0x7f40200b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:26.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.515+0000 7f4024b9d700 1 --2- 172.21.15.55:0/2304009741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4020159420 0x7f40200b5f40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f40100098e0 tx=0x7f40100091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:26.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.516+0000 7f4024b9d700 1 -- 172.21.15.55:0/2304009741 shutdown_connections 2023-10-06T03:34:26.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.516+0000 7f4024b9d700 1 --2- 172.21.15.55:0/2304009741 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4020159420 0x7f40200b5f40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:26.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.516+0000 7f4024b9d700 1 -- 172.21.15.55:0/2304009741 >> 172.21.15.55:0/2304009741 conn(0x7f40200d0660 msgr2=0x7f40200d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:26.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.516+0000 7f4024b9d700 1 -- 172.21.15.55:0/2304009741 shutdown_connections 2023-10-06T03:34:26.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.516+0000 7f4024b9d700 1 -- 172.21.15.55:0/2304009741 wait complete. 2023-10-06T03:34:26.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.517+0000 7f4024b9d700 1 Processor -- start 2023-10-06T03:34:26.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.517+0000 7f4024b9d700 1 -- start start 2023-10-06T03:34:26.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.518+0000 7f4024b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4020159420 0x7f402014e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:26.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.518+0000 7f4024b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40201597f0 0x7f40201538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:26.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.518+0000 7f4024b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402015d230 0x7f4020155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:26.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.518+0000 7f4024b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40201543a0 con 0x7f40201597f0 2023-10-06T03:34:26.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.518+0000 7f4024b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402015a670 con 0x7f402015d230 2023-10-06T03:34:26.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.518+0000 7f4024b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4020159d40 con 0x7f4020159420 2023-10-06T03:34:26.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.518+0000 7f401effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40201597f0 0x7f40201538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:26.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f401effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40201597f0 0x7f40201538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48702/0 (socket says 172.21.15.55:48702) 2023-10-06T03:34:26.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f401effd700 1 -- 172.21.15.55:0/182527108 learned_addr learned my addr 172.21.15.55:0/182527108 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:26.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f401f7fe700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4020159420 0x7f402014e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:26.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f401ffff700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402015d230 0x7f4020155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:26.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f401effd700 1 -- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4020159420 msgr2=0x7f402014e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:26.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f401effd700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4020159420 0x7f402014e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:26.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f401effd700 1 -- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402015d230 msgr2=0x7f4020155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:26.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f401effd700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402015d230 0x7f4020155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:26.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f401effd700 1 -- 172.21.15.55:0/182527108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4010009420 con 0x7f40201597f0 2023-10-06T03:34:26.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f401ffff700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402015d230 0x7f4020155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:26.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f401effd700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40201597f0 0x7f40201538e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f401400c7c0 tx=0x7f4014007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:26.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f40077fe700 1 -- 172.21.15.55:0/182527108 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014005c00 con 0x7f40201597f0 2023-10-06T03:34:26.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f40077fe700 1 -- 172.21.15.55:0/182527108 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4014005d60 con 0x7f40201597f0 2023-10-06T03:34:26.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f40077fe700 1 -- 172.21.15.55:0/182527108 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014007300 con 0x7f40201597f0 2023-10-06T03:34:26.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.519+0000 7f4024b9d700 1 -- 172.21.15.55:0/182527108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402015a020 con 0x7f40201597f0 2023-10-06T03:34:26.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.520+0000 7f4024b9d700 1 -- 172.21.15.55:0/182527108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402014e960 con 0x7f40201597f0 2023-10-06T03:34:26.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.521+0000 7f4024b9d700 1 -- 172.21.15.55:0/182527108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402015a7e0 con 0x7f40201597f0 2023-10-06T03:34:26.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.522+0000 7f40077fe700 1 -- 172.21.15.55:0/182527108 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f401401b3b0 con 0x7f40201597f0 2023-10-06T03:34:26.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.523+0000 7f40077fe700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f400c066f30 0x7f400c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:26.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.523+0000 7f40077fe700 1 -- 172.21.15.55:0/182527108 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4014083b90 con 0x7f40201597f0 2023-10-06T03:34:26.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.524+0000 7f401f7fe700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f400c066f30 0x7f400c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:26.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.524+0000 7f401f7fe700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f400c066f30 0x7f400c0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f40100093f0 tx=0x7f4010017000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:26.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.528+0000 7f40077fe700 1 -- 172.21.15.55:0/182527108 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4014050ca0 con 0x7f40201597f0 2023-10-06T03:34:26.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.758+0000 7f4024b9d700 1 -- 172.21.15.55:0/182527108 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f402004c3c0 con 0x7f400c066f30 2023-10-06T03:34:26.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.760+0000 7f40077fe700 1 -- 172.21.15.55:0/182527108 <== mgr.4114 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f402004c3c0 con 0x7f400c066f30 2023-10-06T03:34:26.763 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:34:26.763 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:34:26.763 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephfs" 2023-10-06T03:34:26.763 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:34:26.763 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:34:26.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.762+0000 7f4024b9d700 1 -- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f400c066f30 msgr2=0x7f400c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:26.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.762+0000 7f4024b9d700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f400c066f30 0x7f400c0693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f40100093f0 tx=0x7f4010017000 comp rx=0 tx=0).stop 2023-10-06T03:34:26.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.762+0000 7f4024b9d700 1 -- 172.21.15.55:0/182527108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40201597f0 msgr2=0x7f40201538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:26.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.762+0000 7f4024b9d700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40201597f0 0x7f40201538e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f401400c7c0 tx=0x7f4014007dd0 comp rx=0 tx=0).stop 2023-10-06T03:34:26.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.762+0000 7f4024b9d700 1 -- 172.21.15.55:0/182527108 shutdown_connections 2023-10-06T03:34:26.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.762+0000 7f4024b9d700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f402015d230 0x7f4020155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:26.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.762+0000 7f4024b9d700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f400c066f30 0x7f400c0693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:26.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.762+0000 7f4024b9d700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4020159420 0x7f402014e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:26.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.762+0000 7f4024b9d700 1 --2- 172.21.15.55:0/182527108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40201597f0 0x7f40201538e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:26.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.762+0000 7f4024b9d700 1 -- 172.21.15.55:0/182527108 >> 172.21.15.55:0/182527108 conn(0x7f40200d0660 msgr2=0x7f402015d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:26.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.762+0000 7f4024b9d700 1 -- 172.21.15.55:0/182527108 shutdown_connections 2023-10-06T03:34:26.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.763+0000 7f4024b9d700 1 -- 172.21.15.55:0/182527108 wait complete. 2023-10-06T03:34:26.783 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T03:34:26.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.969+0000 7f8a2219d700 1 Processor -- start 2023-10-06T03:34:26.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.970+0000 7f8a2219d700 1 -- start start 2023-10-06T03:34:26.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.971+0000 7f8a2219d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8a1c0cb1a0 0x7f8a1c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:26.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.971+0000 7f8a2219d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a1c0ae0d0 0x7f8a1c0ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:26.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.971+0000 7f8a2219d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1c06ebd0 con 0x7f8a1c15a8a0 2023-10-06T03:34:26.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.971+0000 7f8a2219d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a1c15b2e0 con 0x7f8a1c0ae0d0 2023-10-06T03:34:26.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.971+0000 7f8a2219d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a1c0aea10 con 0x7f8a1c0cb1a0 2023-10-06T03:34:26.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.971+0000 7f8a2199c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a1c0ae0d0 0x7f8a1c0ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:26.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.971+0000 7f8a2099a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8a1c15a8a0 0x7f8a1c06f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36376/0 (socket says 172.21.15.55:36376) 2023-10-06T03:34:26.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.971+0000 7f8a2099a700 1 -- 172.21.15.55:0/1639652727 learned_addr learned my addr 172.21.15.55:0/1639652727 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:26.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.971+0000 7f8a2199c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a1c0ae0d0 0x7f8a1c0ae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48712/0 (socket says 172.21.15.55:48712) 2023-10-06T03:34:26.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.971+0000 7f8a2119b700 1 --2- 172.21.15.55:0/1639652727 >> v2:172.21.15.163:3300/0 conn(0x7f8a1c0cb1a0 0x7f8a1c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:26.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.972+0000 7f8a2199c700 1 -- 172.21.15.55:0/1639652727 >> v2:172.21.15.163:3300/0 conn(0x7f8a1c0cb1a0 msgr2=0x7f8a1c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:26.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.972+0000 7f8a2199c700 1 --2- 172.21.15.55:0/1639652727 >> v2:172.21.15.163:3300/0 conn(0x7f8a1c0cb1a0 0x7f8a1c15a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:26.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.972+0000 7f8a2199c700 1 -- 172.21.15.55:0/1639652727 >> v1:172.21.15.55:6789/0 conn(0x7f8a1c15a8a0 legacy=0x7f8a1c06f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:26.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.972+0000 7f8a13fff700 1 -- 172.21.15.55:0/1639652727 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304123108 0 0) 0x7f8a1c06ebd0 con 0x7f8a1c15a8a0 2023-10-06T03:34:26.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.972+0000 7f8a2199c700 1 -- 172.21.15.55:0/1639652727 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c15ad20 con 0x7f8a1c0ae0d0 2023-10-06T03:34:26.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.972+0000 7f8a2199c700 1 --2- 172.21.15.55:0/1639652727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a1c0ae0d0 0x7f8a1c0ae4d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0cce10 tx=0x7f8a1400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43355391b72ee818 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:26.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.972+0000 7f8a13fff700 1 -- 172.21.15.55:0/1639652727 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14010070 con 0x7f8a1c0ae0d0 2023-10-06T03:34:26.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.972+0000 7f8a13fff700 1 -- 172.21.15.55:0/1639652727 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1400de50 con 0x7f8a1c0ae0d0 2023-10-06T03:34:26.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.972+0000 7f8a13fff700 1 -- 172.21.15.55:0/1639652727 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400b320 con 0x7f8a1c0ae0d0 2023-10-06T03:34:26.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.973+0000 7f8a2219d700 1 -- 172.21.15.55:0/1639652727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a1c0ae0d0 msgr2=0x7f8a1c0ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:26.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.973+0000 7f8a2219d700 1 --2- 172.21.15.55:0/1639652727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a1c0ae0d0 0x7f8a1c0ae4d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0cce10 tx=0x7f8a1400b990 comp rx=0 tx=0).stop 2023-10-06T03:34:26.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.973+0000 7f8a2219d700 1 -- 172.21.15.55:0/1639652727 shutdown_connections 2023-10-06T03:34:26.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.973+0000 7f8a2219d700 1 --2- 172.21.15.55:0/1639652727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a1c0ae0d0 0x7f8a1c0ae4d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:26.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.973+0000 7f8a2219d700 1 --2- 172.21.15.55:0/1639652727 >> v2:172.21.15.163:3300/0 conn(0x7f8a1c0cb1a0 0x7f8a1c15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:26.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.973+0000 7f8a2219d700 1 -- 172.21.15.55:0/1639652727 >> 172.21.15.55:0/1639652727 conn(0x7f8a1c0cc1e0 msgr2=0x7f8a1c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:26.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.973+0000 7f8a2219d700 1 -- 172.21.15.55:0/1639652727 shutdown_connections 2023-10-06T03:34:26.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.973+0000 7f8a2219d700 1 -- 172.21.15.55:0/1639652727 wait complete. 2023-10-06T03:34:26.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.974+0000 7f8a2219d700 1 Processor -- start 2023-10-06T03:34:26.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.974+0000 7f8a2219d700 1 -- start start 2023-10-06T03:34:26.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.975+0000 7f8a2219d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a1c0ae0d0 0x7f8a1c1588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:26.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.975+0000 7f8a2219d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a1c0cb1a0 0x7f8a1c15ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:26.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.975+0000 7f8a2219d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a1c14ceb0 0x7f8a1c15dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:26.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.975+0000 7f8a2219d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c159be0 con 0x7f8a1c0cb1a0 2023-10-06T03:34:26.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.975+0000 7f8a2219d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c15d370 con 0x7f8a1c14ceb0 2023-10-06T03:34:26.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.975+0000 7f8a2219d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c158fd0 con 0x7f8a1c0ae0d0 2023-10-06T03:34:26.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.976+0000 7f8a2099a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a1c0cb1a0 0x7f8a1c15ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:26.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.976+0000 7f8a2099a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a1c0cb1a0 0x7f8a1c15ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48720/0 (socket says 172.21.15.55:48720) 2023-10-06T03:34:26.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.976+0000 7f8a2099a700 1 -- 172.21.15.55:0/1061555603 learned_addr learned my addr 172.21.15.55:0/1061555603 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:26.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.976+0000 7f8a2119b700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a1c0ae0d0 0x7f8a1c1588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:26.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.976+0000 7f8a2199c700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a1c14ceb0 0x7f8a1c15dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:26.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.976+0000 7f8a2099a700 1 -- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a1c0ae0d0 msgr2=0x7f8a1c1588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:26.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.976+0000 7f8a2099a700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a1c0ae0d0 0x7f8a1c1588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:26.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.976+0000 7f8a2099a700 1 -- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a1c14ceb0 msgr2=0x7f8a1c15dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:26.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.976+0000 7f8a2099a700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a1c14ceb0 0x7f8a1c15dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:26.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.976+0000 7f8a2099a700 1 -- 172.21.15.55:0/1061555603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1400bc30 con 0x7f8a1c0cb1a0 2023-10-06T03:34:26.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.976+0000 7f8a2199c700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a1c14ceb0 0x7f8a1c15dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:26.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.976+0000 7f8a2099a700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a1c0cb1a0 0x7f8a1c15ce30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00c5b0 tx=0x7f8a0c001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:26.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.977+0000 7f8a117fa700 1 -- 172.21.15.55:0/1061555603 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c005b30 con 0x7f8a1c0cb1a0 2023-10-06T03:34:26.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.977+0000 7f8a117fa700 1 -- 172.21.15.55:0/1061555603 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0c005c90 con 0x7f8a1c0cb1a0 2023-10-06T03:34:26.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.977+0000 7f8a2219d700 1 -- 172.21.15.55:0/1061555603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1c1592b0 con 0x7f8a1c0cb1a0 2023-10-06T03:34:26.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.977+0000 7f8a117fa700 1 -- 172.21.15.55:0/1061555603 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c007300 con 0x7f8a1c0cb1a0 2023-10-06T03:34:26.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.977+0000 7f8a2219d700 1 -- 172.21.15.55:0/1061555603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1c159870 con 0x7f8a1c0cb1a0 2023-10-06T03:34:26.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.978+0000 7f8a2219d700 1 -- 172.21.15.55:0/1061555603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1c06c450 con 0x7f8a1c0cb1a0 2023-10-06T03:34:26.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.979+0000 7f8a117fa700 1 -- 172.21.15.55:0/1061555603 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a0c01b3b0 con 0x7f8a1c0cb1a0 2023-10-06T03:34:26.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.980+0000 7f8a117fa700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a00066f30 0x7f8a000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:26.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.980+0000 7f8a117fa700 1 -- 172.21.15.55:0/1061555603 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8a0c083c60 con 0x7f8a1c0cb1a0 2023-10-06T03:34:26.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.980+0000 7f8a2119b700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a00066f30 0x7f8a000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:26.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.981+0000 7f8a2119b700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a00066f30 0x7f8a000693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8a18006fd0 tx=0x7f8a18008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:26.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:26.984+0000 7f8a117fa700 1 -- 172.21.15.55:0/1061555603 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a0c050cd0 con 0x7f8a1c0cb1a0 2023-10-06T03:34:27.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.241+0000 7f8a2219d700 1 -- 172.21.15.55:0/1061555603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f8a1c15b030 con 0x7f8a1c0cb1a0 2023-10-06T03:34:27.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.590+0000 7f8a117fa700 1 -- 172.21.15.55:0/1061555603 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 6) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f8a0c0510a0 con 0x7f8a1c0cb1a0 2023-10-06T03:34:27.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.591+0000 7f8a117fa700 1 -- 172.21.15.55:0/1061555603 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7f8a0c06fae0 con 0x7f8a1c0cb1a0 2023-10-06T03:34:27.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.594+0000 7f8a2219d700 1 -- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a00066f30 msgr2=0x7f8a000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:27.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.594+0000 7f8a2219d700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a00066f30 0x7f8a000693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8a18006fd0 tx=0x7f8a18008040 comp rx=0 tx=0).stop 2023-10-06T03:34:27.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.594+0000 7f8a2219d700 1 -- 172.21.15.55:0/1061555603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a1c0cb1a0 msgr2=0x7f8a1c15ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:27.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.594+0000 7f8a2219d700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a1c0cb1a0 0x7f8a1c15ce30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00c5b0 tx=0x7f8a0c001070 comp rx=0 tx=0).stop 2023-10-06T03:34:27.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.594+0000 7f8a2219d700 1 -- 172.21.15.55:0/1061555603 shutdown_connections 2023-10-06T03:34:27.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.594+0000 7f8a2219d700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a1c14ceb0 0x7f8a1c15dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:27.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.594+0000 7f8a2219d700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a00066f30 0x7f8a000693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:27.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.594+0000 7f8a2219d700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a1c0ae0d0 0x7f8a1c1588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:27.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.594+0000 7f8a2219d700 1 --2- 172.21.15.55:0/1061555603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a1c0cb1a0 0x7f8a1c15ce30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:27.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.594+0000 7f8a2219d700 1 -- 172.21.15.55:0/1061555603 >> 172.21.15.55:0/1061555603 conn(0x7f8a1c0cc1e0 msgr2=0x7f8a1c0c1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:27.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.594+0000 7f8a2219d700 1 -- 172.21.15.55:0/1061555603 shutdown_connections 2023-10-06T03:34:27.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.594+0000 7f8a2219d700 1 -- 172.21.15.55:0/1061555603 wait complete. 2023-10-06T03:34:27.615 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T03:34:27.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.807+0000 7f6deffff700 1 Processor -- start 2023-10-06T03:34:27.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.807+0000 7f6deffff700 1 -- start start 2023-10-06T03:34:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.808+0000 7f6deffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6df00b4e70 0x7f6df00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.808+0000 7f6deffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df00cfb00 con 0x7f6df00b1b40 2023-10-06T03:34:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.808+0000 7f6deffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6df015e060 con 0x7f6df00b16d0 2023-10-06T03:34:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.808+0000 7f6deffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6df00b5c40 con 0x7f6df00b4e70 2023-10-06T03:34:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.809+0000 7f6deeffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6df00b4e70 0x7f6df00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:27.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.809+0000 7f6def7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6df00b1b40 0x7f6df015d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36382/0 (socket says 172.21.15.55:36382) 2023-10-06T03:34:27.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.809+0000 7f6deeffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6df00b4e70 0x7f6df00ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48748/0 (socket says 172.21.15.55:48748) 2023-10-06T03:34:27.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.809+0000 7f6def7fe700 1 -- 172.21.15.55:0/1106263623 learned_addr learned my addr 172.21.15.55:0/1106263623 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:27.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.809+0000 7f6dedffb700 1 -- 172.21.15.55:0/1106263623 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369835267 0 0) 0x7f6df00cfb00 con 0x7f6df00b1b40 2023-10-06T03:34:27.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.809+0000 7f6dedffb700 1 -- 172.21.15.55:0/1106263623 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dd8003570 con 0x7f6df00b1b40 2023-10-06T03:34:27.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.809+0000 7f6dedffb700 1 -- 172.21.15.55:0/1106263623 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432868378 0 0) 0x7f6df015e060 con 0x7f6df00b16d0 2023-10-06T03:34:27.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.809+0000 7f6deeffd700 1 -- 172.21.15.55:0/1106263623 >> v1:172.21.15.163:6789/0 conn(0x7f6df00b16d0 legacy=0x7f6df015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:27.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.809+0000 7f6deeffd700 1 -- 172.21.15.55:0/1106263623 >> v1:172.21.15.55:6789/0 conn(0x7f6df00b1b40 legacy=0x7f6df015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:27.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.809+0000 7f6deeffd700 1 -- 172.21.15.55:0/1106263623 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df00b5f20 con 0x7f6df00b4e70 2023-10-06T03:34:27.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.810+0000 7f6deeffd700 1 --2- 172.21.15.55:0/1106263623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df00b4e70 0x7f6df00ccff0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6de00098e0 tx=0x7f6de00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4c3231f5b77ec0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:27.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.810+0000 7f6dedffb700 1 -- 172.21.15.55:0/1106263623 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de000b640 con 0x7f6df00b4e70 2023-10-06T03:34:27.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.810+0000 7f6dedffb700 1 -- 172.21.15.55:0/1106263623 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de000b7a0 con 0x7f6df00b4e70 2023-10-06T03:34:27.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.810+0000 7f6dedffb700 1 -- 172.21.15.55:0/1106263623 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de000bb40 con 0x7f6df00b4e70 2023-10-06T03:34:27.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.810+0000 7f6deffff700 1 -- 172.21.15.55:0/1106263623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df00b4e70 msgr2=0x7f6df00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:27.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.810+0000 7f6deffff700 1 --2- 172.21.15.55:0/1106263623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df00b4e70 0x7f6df00ccff0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6de00098e0 tx=0x7f6de00091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:27.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.811+0000 7f6deffff700 1 -- 172.21.15.55:0/1106263623 shutdown_connections 2023-10-06T03:34:27.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.811+0000 7f6deffff700 1 --2- 172.21.15.55:0/1106263623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df00b4e70 0x7f6df00ccff0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:27.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.811+0000 7f6deffff700 1 -- 172.21.15.55:0/1106263623 >> 172.21.15.55:0/1106263623 conn(0x7f6df00d10c0 msgr2=0x7f6df00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:27.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.811+0000 7f6deffff700 1 -- 172.21.15.55:0/1106263623 shutdown_connections 2023-10-06T03:34:27.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.811+0000 7f6deffff700 1 -- 172.21.15.55:0/1106263623 wait complete. 2023-10-06T03:34:27.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.812+0000 7f6deffff700 1 Processor -- start 2023-10-06T03:34:27.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.812+0000 7f6deffff700 1 -- start start 2023-10-06T03:34:27.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.813+0000 7f6deffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6df00b16d0 0x7f6df00c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:27.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.813+0000 7f6deffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df00b1b40 0x7f6df0158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:27.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.813+0000 7f6deffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6df00b4e70 0x7f6df01591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:27.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.813+0000 7f6deffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df00c5d60 con 0x7f6df00b1b40 2023-10-06T03:34:27.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.813+0000 7f6deffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6df00c61f0 con 0x7f6df00b4e70 2023-10-06T03:34:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.813+0000 7f6deffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6df016ad90 con 0x7f6df00b16d0 2023-10-06T03:34:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.813+0000 7f6dee7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df00b1b40 0x7f6df0158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.813+0000 7f6dee7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df00b1b40 0x7f6df0158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48758/0 (socket says 172.21.15.55:48758) 2023-10-06T03:34:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.814+0000 7f6dee7fc700 1 -- 172.21.15.55:0/1623342674 learned_addr learned my addr 172.21.15.55:0/1623342674 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.814+0000 7f6deeffd700 1 --2- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6df00b16d0 0x7f6df00c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.814+0000 7f6def7fe700 1 --2- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6df00b4e70 0x7f6df01591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:27.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.814+0000 7f6dee7fc700 1 -- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6df00b16d0 msgr2=0x7f6df00c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:27.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.814+0000 7f6dee7fc700 1 --2- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6df00b16d0 0x7f6df00c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:27.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.814+0000 7f6dee7fc700 1 -- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6df00b4e70 msgr2=0x7f6df01591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:27.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.814+0000 7f6dee7fc700 1 --2- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6df00b4e70 0x7f6df01591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:27.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.814+0000 7f6dee7fc700 1 -- 172.21.15.55:0/1623342674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de0009420 con 0x7f6df00b1b40 2023-10-06T03:34:27.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.814+0000 7f6dee7fc700 1 --2- 172.21.15.55:0/1623342674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df00b1b40 0x7f6df0158cb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6de400c7e0 tx=0x7f6de4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:27.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.815+0000 7f6dd6ffd700 1 -- 172.21.15.55:0/1623342674 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de40105c0 con 0x7f6df00b1b40 2023-10-06T03:34:27.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.815+0000 7f6dd6ffd700 1 -- 172.21.15.55:0/1623342674 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de4010740 con 0x7f6df00b1b40 2023-10-06T03:34:27.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.815+0000 7f6dd6ffd700 1 -- 172.21.15.55:0/1623342674 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de4010b00 con 0x7f6df00b1b40 2023-10-06T03:34:27.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.815+0000 7f6deffff700 1 -- 172.21.15.55:0/1623342674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df00c5430 con 0x7f6df00b1b40 2023-10-06T03:34:27.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.815+0000 7f6deffff700 1 -- 172.21.15.55:0/1623342674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df00c59f0 con 0x7f6df00b1b40 2023-10-06T03:34:27.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.816+0000 7f6deffff700 1 -- 172.21.15.55:0/1623342674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6df0088990 con 0x7f6df00b1b40 2023-10-06T03:34:27.824 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.817+0000 7f6dd6ffd700 1 -- 172.21.15.55:0/1623342674 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 6) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f6de401b3b0 con 0x7f6df00b1b40 2023-10-06T03:34:27.824 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.818+0000 7f6dd6ffd700 1 --2- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6ddc066f80 0x7f6ddc069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:27.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.818+0000 7f6dd6ffd700 1 -- 172.21.15.55:0/1623342674 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6de4083d40 con 0x7f6df00b1b40 2023-10-06T03:34:27.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.818+0000 7f6deeffd700 1 -- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6ddc066f80 msgr2=0x7f6ddc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:34:27.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.818+0000 7f6deeffd700 1 --2- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6ddc066f80 0x7f6ddc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:34:27.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:27.822+0000 7f6dd6ffd700 1 -- 172.21.15.55:0/1623342674 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6de4050db0 con 0x7f6df00b1b40 2023-10-06T03:34:28.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.019+0000 7f6deeffd700 1 -- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6ddc066f80 msgr2=0x7f6ddc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:34:28.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.019+0000 7f6deeffd700 1 --2- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6ddc066f80 0x7f6ddc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:34:28.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.027+0000 7f6deffff700 1 -- 172.21.15.55:0/1623342674 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f6df004bb80 con 0x7f6df00b1b40 2023-10-06T03:34:28.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.035+0000 7f6dd6ffd700 1 -- 172.21.15.55:0/1623342674 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6de4017070 con 0x7f6df00b1b40 2023-10-06T03:34:28.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.037+0000 7f6deffff700 1 -- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6ddc066f80 msgr2=0x7f6ddc069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:34:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.037+0000 7f6deffff700 1 --2- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6ddc066f80 0x7f6ddc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.037+0000 7f6deffff700 1 -- 172.21.15.55:0/1623342674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df00b1b40 msgr2=0x7f6df0158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.037+0000 7f6deffff700 1 --2- 172.21.15.55:0/1623342674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df00b1b40 0x7f6df0158cb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6de400c7e0 tx=0x7f6de4007df0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.037+0000 7f6deffff700 1 -- 172.21.15.55:0/1623342674 shutdown_connections 2023-10-06T03:34:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.037+0000 7f6deffff700 1 --2- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6df00b4e70 0x7f6df01591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.037+0000 7f6deffff700 1 --2- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f6ddc066f80 0x7f6ddc069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.037+0000 7f6deffff700 1 --2- 172.21.15.55:0/1623342674 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6df00b16d0 0x7f6df00c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.037+0000 7f6deffff700 1 --2- 172.21.15.55:0/1623342674 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6df00b1b40 0x7f6df0158cb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.037+0000 7f6deffff700 1 -- 172.21.15.55:0/1623342674 >> 172.21.15.55:0/1623342674 conn(0x7f6df00d10c0 msgr2=0x7f6df0159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.037+0000 7f6deffff700 1 -- 172.21.15.55:0/1623342674 shutdown_connections 2023-10-06T03:34:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.037+0000 7f6deffff700 1 -- 172.21.15.55:0/1623342674 wait complete. 2023-10-06T03:34:28.057 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T03:34:28.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.247+0000 7f36d519d700 1 Processor -- start 2023-10-06T03:34:28.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.247+0000 7f36d519d700 1 -- start start 2023-10-06T03:34:28.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.248+0000 7f36d519d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f36d0159a60 0x7f36d00c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:28.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.248+0000 7f36d519d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d00c9cb0 con 0x7f36d00ccc00 2023-10-06T03:34:28.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.248+0000 7f36d519d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d00c9f30 con 0x7f36d00cb230 2023-10-06T03:34:28.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.248+0000 7f36d519d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d006dfc0 con 0x7f36d0159a60 2023-10-06T03:34:28.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.248+0000 7f36cffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f36d0159a60 0x7f36d00c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:28.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.249+0000 7f36cffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f36d0159a60 0x7f36d00c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48762/0 (socket says 172.21.15.55:48762) 2023-10-06T03:34:28.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.248+0000 7f36cf7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f36d00ccc00 0x7f36d0069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36398/0 (socket says 172.21.15.55:36398) 2023-10-06T03:34:28.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.249+0000 7f36cffff700 1 -- 172.21.15.55:0/1732342777 learned_addr learned my addr 172.21.15.55:0/1732342777 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:28.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.249+0000 7f36cffff700 1 -- 172.21.15.55:0/1732342777 >> v1:172.21.15.163:6789/0 conn(0x7f36d00cb230 legacy=0x7f36d006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:28.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.249+0000 7f36cffff700 1 -- 172.21.15.55:0/1732342777 >> v1:172.21.15.55:6789/0 conn(0x7f36d00ccc00 legacy=0x7f36d0069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:28.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.249+0000 7f36cffff700 1 -- 172.21.15.55:0/1732342777 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d00ca470 con 0x7f36d0159a60 2023-10-06T03:34:28.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.249+0000 7f36ceffd700 1 -- 172.21.15.55:0/1732342777 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558998184 0 0) 0x7f36d00c9cb0 con 0x7f36d00ccc00 2023-10-06T03:34:28.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.249+0000 7f36cffff700 1 --2- 172.21.15.55:0/1732342777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36d0159a60 0x7f36d00c9770 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f36c00099b0 tx=0x7f36c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a3c045b4bbcf60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:28.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.250+0000 7f36ceffd700 1 -- 172.21.15.55:0/1732342777 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c000b8f0 con 0x7f36d0159a60 2023-10-06T03:34:28.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.250+0000 7f36ceffd700 1 -- 172.21.15.55:0/1732342777 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c000ba50 con 0x7f36d0159a60 2023-10-06T03:34:28.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.250+0000 7f36ceffd700 1 -- 172.21.15.55:0/1732342777 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c000bdf0 con 0x7f36d0159a60 2023-10-06T03:34:28.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.250+0000 7f36d519d700 1 -- 172.21.15.55:0/1732342777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36d0159a60 msgr2=0x7f36d00c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:28.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.250+0000 7f36d519d700 1 --2- 172.21.15.55:0/1732342777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36d0159a60 0x7f36d00c9770 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f36c00099b0 tx=0x7f36c0009250 comp rx=0 tx=0).stop 2023-10-06T03:34:28.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.251+0000 7f36d519d700 1 -- 172.21.15.55:0/1732342777 shutdown_connections 2023-10-06T03:34:28.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.251+0000 7f36d519d700 1 --2- 172.21.15.55:0/1732342777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36d0159a60 0x7f36d00c9770 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.251+0000 7f36d519d700 1 -- 172.21.15.55:0/1732342777 >> 172.21.15.55:0/1732342777 conn(0x7f36d006baa0 msgr2=0x7f36d006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:28.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.251+0000 7f36d519d700 1 -- 172.21.15.55:0/1732342777 shutdown_connections 2023-10-06T03:34:28.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.251+0000 7f36d519d700 1 -- 172.21.15.55:0/1732342777 wait complete. 2023-10-06T03:34:28.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.252+0000 7f36d519d700 1 Processor -- start 2023-10-06T03:34:28.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.252+0000 7f36d519d700 1 -- start start 2023-10-06T03:34:28.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.253+0000 7f36d519d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d00cb230 0x7f36d016a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:28.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.253+0000 7f36d519d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36d00ccc00 0x7f36d014e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:28.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.253+0000 7f36d519d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36d0159a60 0x7f36d00aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:28.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.253+0000 7f36d519d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d014e760 con 0x7f36d0159a60 2023-10-06T03:34:28.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.253+0000 7f36d519d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d00c7150 con 0x7f36d00ccc00 2023-10-06T03:34:28.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.253+0000 7f36d519d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d00aacd0 con 0x7f36d00cb230 2023-10-06T03:34:28.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.253+0000 7f36d499c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36d0159a60 0x7f36d00aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:28.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.253+0000 7f36cffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d00cb230 0x7f36d016a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:28.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.253+0000 7f36cf7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36d00ccc00 0x7f36d014e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:28.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.253+0000 7f36d499c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36d0159a60 0x7f36d00aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48778/0 (socket says 172.21.15.55:48778) 2023-10-06T03:34:28.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.253+0000 7f36d499c700 1 -- 172.21.15.55:0/3556475484 learned_addr learned my addr 172.21.15.55:0/3556475484 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:28.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.254+0000 7f36cffff700 1 -- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36d00ccc00 msgr2=0x7f36d014e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:28.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.254+0000 7f36cffff700 1 --2- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36d00ccc00 0x7f36d014e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.254+0000 7f36cffff700 1 -- 172.21.15.55:0/3556475484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36d0159a60 msgr2=0x7f36d00aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:28.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.254+0000 7f36cffff700 1 --2- 172.21.15.55:0/3556475484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36d0159a60 0x7f36d00aa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.254+0000 7f36cffff700 1 -- 172.21.15.55:0/3556475484 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c00094f0 con 0x7f36d00cb230 2023-10-06T03:34:28.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.254+0000 7f36d499c700 1 --2- 172.21.15.55:0/3556475484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36d0159a60 0x7f36d00aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:34:28.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.254+0000 7f36cffff700 1 --2- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d00cb230 0x7f36d016a820 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36c0013000 tx=0x7f36c0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:28.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.255+0000 7f36b7fff700 1 -- 172.21.15.55:0/3556475484 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c001f070 con 0x7f36d00cb230 2023-10-06T03:34:28.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.255+0000 7f36b7fff700 1 -- 172.21.15.55:0/3556475484 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c0011bb0 con 0x7f36d00cb230 2023-10-06T03:34:28.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.255+0000 7f36b7fff700 1 -- 172.21.15.55:0/3556475484 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0025420 con 0x7f36d00cb230 2023-10-06T03:34:28.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.255+0000 7f36d519d700 1 -- 172.21.15.55:0/3556475484 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d00aafb0 con 0x7f36d00cb230 2023-10-06T03:34:28.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.255+0000 7f36d519d700 1 -- 172.21.15.55:0/3556475484 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d00c6de0 con 0x7f36d00cb230 2023-10-06T03:34:28.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.256+0000 7f36d519d700 1 -- 172.21.15.55:0/3556475484 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d006eba0 con 0x7f36d00cb230 2023-10-06T03:34:28.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.257+0000 7f36b7fff700 1 -- 172.21.15.55:0/3556475484 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 6) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f36c0023070 con 0x7f36d00cb230 2023-10-06T03:34:28.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.257+0000 7f36b7fff700 1 --2- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f36bc066f80 0x7f36bc069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:28.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.257+0000 7f36b7fff700 1 -- 172.21.15.55:0/3556475484 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f36c0085690 con 0x7f36d00cb230 2023-10-06T03:34:28.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.258+0000 7f36cf7fe700 1 -- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f36bc066f80 msgr2=0x7f36bc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:34:28.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.258+0000 7f36cf7fe700 1 --2- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f36bc066f80 0x7f36bc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:34:28.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.262+0000 7f36b7fff700 1 -- 172.21.15.55:0/3556475484 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36c00527b0 con 0x7f36d00cb230 2023-10-06T03:34:28.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.459+0000 7f36cf7fe700 1 -- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f36bc066f80 msgr2=0x7f36bc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:34:28.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.459+0000 7f36cf7fe700 1 --2- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f36bc066f80 0x7f36bc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:34:28.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.467+0000 7f36d519d700 1 -- 172.21.15.55:0/3556475484 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f36d00ca750 con 0x7f36d00cb230 2023-10-06T03:34:28.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.472+0000 7f36b7fff700 1 -- 172.21.15.55:0/3556475484 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f36c0055dd0 con 0x7f36d00cb230 2023-10-06T03:34:28.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.475+0000 7f36d519d700 1 -- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f36bc066f80 msgr2=0x7f36bc069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:34:28.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.475+0000 7f36d519d700 1 --2- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f36bc066f80 0x7f36bc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.475+0000 7f36d519d700 1 -- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d00cb230 msgr2=0x7f36d016a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.475+0000 7f36d519d700 1 --2- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d00cb230 0x7f36d016a820 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36c0013000 tx=0x7f36c0017000 comp rx=0 tx=0).stop 2023-10-06T03:34:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.475+0000 7f36d519d700 1 -- 172.21.15.55:0/3556475484 shutdown_connections 2023-10-06T03:34:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.475+0000 7f36d519d700 1 --2- 172.21.15.55:0/3556475484 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36d0159a60 0x7f36d00aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.475+0000 7f36d519d700 1 --2- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f36bc066f80 0x7f36bc069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.475+0000 7f36d519d700 1 --2- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36d00cb230 0x7f36d016a820 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.475+0000 7f36d519d700 1 --2- 172.21.15.55:0/3556475484 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36d00ccc00 0x7f36d014e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.475+0000 7f36d519d700 1 -- 172.21.15.55:0/3556475484 >> 172.21.15.55:0/3556475484 conn(0x7f36d006baa0 msgr2=0x7f36d0156ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.475+0000 7f36d519d700 1 -- 172.21.15.55:0/3556475484 shutdown_connections 2023-10-06T03:34:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.475+0000 7f36d519d700 1 -- 172.21.15.55:0/3556475484 wait complete. 2023-10-06T03:34:28.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:34:28.495 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T03:34:28.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:34:28.545 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-10-06T03:34:28.587 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir0 1M 2023-10-06T03:34:28.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.821+0000 7f5c8cd9d700 1 Processor -- start 2023-10-06T03:34:28.824 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.821+0000 7f5c8cd9d700 1 -- start start 2023-10-06T03:34:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.822+0000 7f5c8cd9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c880b03c0 0x7f5c880cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.822+0000 7f5c8cd9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c880cfa00 con 0x7f5c881591e0 2023-10-06T03:34:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.823+0000 7f5c8cd9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c880cb580 con 0x7f5c880b0790 2023-10-06T03:34:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.823+0000 7f5c8cd9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c880cf160 con 0x7f5c880b03c0 2023-10-06T03:34:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.823+0000 7f5c87fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5c881591e0 0x7f5c88159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36414/0 (socket says 172.21.15.55:36414) 2023-10-06T03:34:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.823+0000 7f5c87fff700 1 -- 172.21.15.55:0/141805032 learned_addr learned my addr 172.21.15.55:0/141805032 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:28.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.823+0000 7f5c877fe700 1 --2- 172.21.15.55:0/141805032 >> v2:172.21.15.163:3300/0 conn(0x7f5c880b03c0 0x7f5c880cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:28.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.823+0000 7f5c867fc700 1 -- 172.21.15.55:0/141805032 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589286340 0 0) 0x7f5c880cfa00 con 0x7f5c881591e0 2023-10-06T03:34:28.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.823+0000 7f5c867fc700 1 -- 172.21.15.55:0/141805032 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c70003570 con 0x7f5c881591e0 2023-10-06T03:34:28.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.823+0000 7f5c867fc700 1 -- 172.21.15.55:0/141805032 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061743515 0 0) 0x7f5c880cb580 con 0x7f5c880b0790 2023-10-06T03:34:28.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.823+0000 7f5c867fc700 1 -- 172.21.15.55:0/141805032 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c880cfa00 con 0x7f5c880b0790 2023-10-06T03:34:28.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.824+0000 7f5c877fe700 1 -- 172.21.15.55:0/141805032 >> v1:172.21.15.163:6789/0 conn(0x7f5c880b0790 legacy=0x7f5c880b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:28.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.824+0000 7f5c877fe700 1 -- 172.21.15.55:0/141805032 >> v1:172.21.15.55:6789/0 conn(0x7f5c881591e0 legacy=0x7f5c88159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:28.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.824+0000 7f5c877fe700 1 -- 172.21.15.55:0/141805032 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c880cf440 con 0x7f5c880b03c0 2023-10-06T03:34:28.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.824+0000 7f5c877fe700 1 --2- 172.21.15.55:0/141805032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c880b03c0 0x7f5c880cea70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009910 tx=0x7f5c780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de35370ef6920bc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:28.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.824+0000 7f5c867fc700 1 -- 172.21.15.55:0/141805032 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78011630 con 0x7f5c880b03c0 2023-10-06T03:34:28.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.824+0000 7f5c867fc700 1 -- 172.21.15.55:0/141805032 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c78011790 con 0x7f5c880b03c0 2023-10-06T03:34:28.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.824+0000 7f5c867fc700 1 -- 172.21.15.55:0/141805032 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78011b30 con 0x7f5c880b03c0 2023-10-06T03:34:28.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.825+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/141805032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c880b03c0 msgr2=0x7f5c880cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:28.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.825+0000 7f5c8cd9d700 1 --2- 172.21.15.55:0/141805032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c880b03c0 0x7f5c880cea70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009910 tx=0x7f5c780091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.825+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/141805032 shutdown_connections 2023-10-06T03:34:28.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.825+0000 7f5c8cd9d700 1 --2- 172.21.15.55:0/141805032 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c880b03c0 0x7f5c880cea70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.825+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/141805032 >> 172.21.15.55:0/141805032 conn(0x7f5c88063230 msgr2=0x7f5c88063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:28.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.825+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/141805032 shutdown_connections 2023-10-06T03:34:28.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.825+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/141805032 wait complete. 2023-10-06T03:34:28.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.826+0000 7f5c8cd9d700 1 Processor -- start 2023-10-06T03:34:28.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.826+0000 7f5c8cd9d700 1 -- start start 2023-10-06T03:34:28.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.827+0000 7f5c8cd9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c880b03c0 0x7f5c8815eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:28.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.827+0000 7f5c8cd9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c880b0790 0x7f5c8815acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:28.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.827+0000 7f5c8cd9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c881591e0 0x7f5c8816d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:28.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.827+0000 7f5c8cd9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8815a840 con 0x7f5c880b0790 2023-10-06T03:34:28.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.827+0000 7f5c8cd9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8816e1d0 con 0x7f5c881591e0 2023-10-06T03:34:28.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.827+0000 7f5c8cd9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c88159f50 con 0x7f5c880b03c0 2023-10-06T03:34:28.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.828+0000 7f5c86ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c880b0790 0x7f5c8815acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.828+0000 7f5c87fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c881591e0 0x7f5c8816d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.828+0000 7f5c86ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c880b0790 0x7f5c8815acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48794/0 (socket says 172.21.15.55:48794) 2023-10-06T03:34:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.828+0000 7f5c86ffd700 1 -- 172.21.15.55:0/1325033954 learned_addr learned my addr 172.21.15.55:0/1325033954 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.828+0000 7f5c877fe700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c880b03c0 0x7f5c8815eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.828+0000 7f5c86ffd700 1 -- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c880b03c0 msgr2=0x7f5c8815eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.828+0000 7f5c86ffd700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c880b03c0 0x7f5c8815eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.828+0000 7f5c86ffd700 1 -- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c881591e0 msgr2=0x7f5c8816d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.828+0000 7f5c86ffd700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c881591e0 0x7f5c8816d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.828+0000 7f5c86ffd700 1 -- 172.21.15.55:0/1325033954 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c78009420 con 0x7f5c880b0790 2023-10-06T03:34:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.828+0000 7f5c877fe700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c880b03c0 0x7f5c8815eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:34:28.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.828+0000 7f5c87fff700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c881591e0 0x7f5c8816d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:34:28.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.828+0000 7f5c86ffd700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c880b0790 0x7f5c8815acd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00c790 tx=0x7f5c7c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:28.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.829+0000 7f5c6f7fe700 1 -- 172.21.15.55:0/1325033954 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c010510 con 0x7f5c880b0790 2023-10-06T03:34:28.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.829+0000 7f5c6f7fe700 1 -- 172.21.15.55:0/1325033954 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c010690 con 0x7f5c880b0790 2023-10-06T03:34:28.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.829+0000 7f5c6f7fe700 1 -- 172.21.15.55:0/1325033954 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c010a50 con 0x7f5c880b0790 2023-10-06T03:34:28.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.829+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/1325033954 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8815a1f0 con 0x7f5c880b0790 2023-10-06T03:34:28.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.829+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/1325033954 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8816ded0 con 0x7f5c880b0790 2023-10-06T03:34:28.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.830+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/1325033954 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8804c3c0 con 0x7f5c880b0790 2023-10-06T03:34:28.834 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.831+0000 7f5c6f7fe700 1 -- 172.21.15.55:0/1325033954 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 6) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f5c7c01b3b0 con 0x7f5c880b0790 2023-10-06T03:34:28.834 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.832+0000 7f5c6f7fe700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 0x7f5c74069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:28.834 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.832+0000 7f5c6f7fe700 1 -- 172.21.15.55:0/1325033954 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5c7c083ec0 con 0x7f5c880b0790 2023-10-06T03:34:28.834 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.832+0000 7f5c877fe700 1 -- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 msgr2=0x7f5c74069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:34:28.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.832+0000 7f5c877fe700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 0x7f5c74069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:34:28.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:28.836+0000 7f5c6f7fe700 1 -- 172.21.15.55:0/1325033954 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c7c050fd0 con 0x7f5c880b0790 2023-10-06T03:34:28.887 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:28.886+0000 7fec1c2aa180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:34:28.968 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:28.966+0000 7f63f00f6180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:34:29.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:29.033+0000 7f5c877fe700 1 -- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 msgr2=0x7f5c74069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:34:29.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:29.033+0000 7f5c877fe700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 0x7f5c74069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:34:29.050 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:29.050+0000 7fec1c2aa180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:34:29.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:29.068+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/1325033954 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c88158fc0 con 0x7f5c74066f80 2023-10-06T03:34:29.136 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:29.135+0000 7fec1c2aa180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:34:29.143 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:29.141+0000 7f63f00f6180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:34:29.233 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:29.232+0000 7f63f00f6180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:34:29.358 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:29.357+0000 7fec1c2aa180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:34:29.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:29.435+0000 7f5c877fe700 1 -- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 msgr2=0x7f5c74069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:34:29.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:29.435+0000 7f5c877fe700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 0x7f5c74069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T03:34:29.466 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:29.465+0000 7f63f00f6180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:34:29.537 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:29.537+0000 7fec1c2aa180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:34:29.634 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:29.633+0000 7fec1c2aa180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:34:29.657 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:29.655+0000 7f63f00f6180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:34:29.754 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:29.752+0000 7f63f00f6180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:34:29.795 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:29.794+0000 7fec1c2aa180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:34:29.894 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:29.893+0000 7fec1c2aa180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:34:29.921 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:29.920+0000 7f63f00f6180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:34:30.028 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:30.027+0000 7f63f00f6180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:34:30.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:30.236+0000 7f5c877fe700 1 -- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 msgr2=0x7f5c74069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:34:30.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:30.236+0000 7f5c877fe700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 0x7f5c74069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T03:34:30.376 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:30.375+0000 7fec1c2aa180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:34:30.470 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:30.469+0000 7fec1c2aa180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:34:30.534 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:30.533+0000 7f63f00f6180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:34:30.561 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:30.560+0000 7fec1c2aa180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:34:30.633 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:30.632+0000 7f63f00f6180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:34:30.728 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:30.727+0000 7f63f00f6180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:34:31.185 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:31.184+0000 7fec1c2aa180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:34:31.355 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:31.354+0000 7fec1c2aa180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:34:31.383 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:31.382+0000 7f63f00f6180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:34:31.449 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:31.449+0000 7fec1c2aa180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:34:31.563 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:31.561+0000 7f63f00f6180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:34:31.663 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:31.661+0000 7f63f00f6180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:34:31.744 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:31.743+0000 7fec1c2aa180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:34:31.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:31.837+0000 7f5c877fe700 1 -- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 msgr2=0x7f5c74069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:34:31.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:31.837+0000 7f5c877fe700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 0x7f5c74069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T03:34:31.839 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:31.837+0000 7fec1c2aa180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:34:31.964 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:31.963+0000 7f63f00f6180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:34:32.065 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:32.063+0000 7f63f00f6180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:34:32.537 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:32.536+0000 7fec1c2aa180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:34:32.801 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:32.800+0000 7f63f00f6180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:34:32.822 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:32.822+0000 7fec1c2aa180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:34:32.913 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:32.912+0000 7fec1c2aa180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:34:33.066 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.065+0000 7fec1c2aa180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:34:33.093 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:33.092+0000 7f63f00f6180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:34:33.168 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.168+0000 7fec1c2aa180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:34:33.189 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:33.188+0000 7f63f00f6180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:34:33.351 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:33.349+0000 7f63f00f6180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:34:33.355 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.354+0000 7fec1c2aa180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:34:33.446 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.445+0000 7fec1c2aa180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:34:33.459 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:33.458+0000 7f63f00f6180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:34:33.538 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.537+0000 7fec1c2aa180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:34:33.629 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.628+0000 7fec1c2aa180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:34:33.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:33.642+0000 7f5c6f7fe700 1 -- 172.21.15.55:0/1325033954 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 7) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f5c7c0513a0 con 0x7f5c880b0790 2023-10-06T03:34:33.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:33.642+0000 7f5c6f7fe700 1 -- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 msgr2=0x7f5c74069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:34:33.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:33.642+0000 7f5c6f7fe700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 0x7f5c74069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:33.658 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:33.657+0000 7f63f00f6180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:34:33.764 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:33.763+0000 7f63f00f6180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:34:33.770 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.769+0000 7feb915f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:34:33.771 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.769+0000 7feb915f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:34:33.771 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.769+0000 7feb915f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:34:33.771 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.769+0000 7feb915f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:34:33.771 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.769+0000 7feb915f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:34:33.771 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.770+0000 7feb8e5f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:34:33.771 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.770+0000 7feb8e5f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:34:33.771 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.770+0000 7feb8e5f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:34:33.771 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.770+0000 7feb8e5f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:34:33.771 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:34:33.770+0000 7feb8e5f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:34:33.861 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:33.859+0000 7f63f00f6180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:34:33.958 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:34:33.957+0000 7f63f00f6180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:34:34.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.647+0000 7f5c6f7fe700 1 -- 172.21.15.55:0/1325033954 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mgrmap(e 8) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f5c7c05be80 con 0x7f5c880b0790 2023-10-06T03:34:34.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.648+0000 7f5c6f7fe700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 0x7f5c74069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:34.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.648+0000 7f5c6f7fe700 1 -- 172.21.15.55:0/1325033954 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c88158fc0 con 0x7f5c74066f80 2023-10-06T03:34:34.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.648+0000 7f5c877fe700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 0x7f5c74069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:34.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.651+0000 7f5c877fe700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 0x7f5c74069430 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009130 tx=0x7f5c78017000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:34.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.757+0000 7f5c6f7fe700 1 -- 172.21.15.55:0/1325033954 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f5c88158fc0 con 0x7f5c74066f80 2023-10-06T03:34:34.758 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path /snap_test_dir1/dir0 2023-10-06T03:34:34.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.759+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 msgr2=0x7f5c74069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:34.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.759+0000 7f5c8cd9d700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 0x7f5c74069430 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009130 tx=0x7f5c78017000 comp rx=0 tx=0).stop 2023-10-06T03:34:34.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.760+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/1325033954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c880b0790 msgr2=0x7f5c8815acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:34.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.760+0000 7f5c8cd9d700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c880b0790 0x7f5c8815acd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c00c790 tx=0x7f5c7c007e10 comp rx=0 tx=0).stop 2023-10-06T03:34:34.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.760+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/1325033954 shutdown_connections 2023-10-06T03:34:34.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.760+0000 7f5c8cd9d700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c881591e0 0x7f5c8816d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:34.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.760+0000 7f5c8cd9d700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5c74066f80 0x7f5c74069430 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:34.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.760+0000 7f5c8cd9d700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c880b03c0 0x7f5c8815eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:34.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.760+0000 7f5c8cd9d700 1 --2- 172.21.15.55:0/1325033954 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c880b0790 0x7f5c8815acd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:34.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.760+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/1325033954 >> 172.21.15.55:0/1325033954 conn(0x7f5c88063230 msgr2=0x7f5c8815d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:34.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.760+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/1325033954 shutdown_connections 2023-10-06T03:34:34.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:34.760+0000 7f5c8cd9d700 1 -- 172.21.15.55:0/1325033954 wait complete. 2023-10-06T03:34:34.780 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-10-06T03:34:34.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:34:34.781 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-10-06T03:34:34.820 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir1 1M 2023-10-06T03:34:35.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.051+0000 7fe0c5b9d700 1 Processor -- start 2023-10-06T03:34:35.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.051+0000 7fe0c5b9d700 1 -- start start 2023-10-06T03:34:35.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.053+0000 7fe0c5b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe0c00cc8d0 0x7fe0c015b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:35.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.053+0000 7fe0c5b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c0157a50 con 0x7fe0c015b050 2023-10-06T03:34:35.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.053+0000 7fe0c5b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c0158f10 con 0x7fe0c015b960 2023-10-06T03:34:35.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.053+0000 7fe0c5b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c01591b0 con 0x7fe0c00cc8d0 2023-10-06T03:34:35.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.053+0000 7fe0c4b9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe0c015b050 0x7fe0c00ca100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35606/0 (socket says 172.21.15.55:35606) 2023-10-06T03:34:35.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.053+0000 7fe0c4b9b700 1 -- 172.21.15.55:0/3545345709 learned_addr learned my addr 172.21.15.55:0/3545345709 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:35.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.053+0000 7fe0bffff700 1 --2- 172.21.15.55:0/3545345709 >> v2:172.21.15.163:3300/0 conn(0x7fe0c00cc8d0 0x7fe0c015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:35.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.053+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/3545345709 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445542383 0 0) 0x7fe0c0157a50 con 0x7fe0c015b050 2023-10-06T03:34:35.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/3545345709 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0b0003570 con 0x7fe0c015b050 2023-10-06T03:34:35.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/3545345709 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3643436351 0 0) 0x7fe0b0003570 con 0x7fe0c015b050 2023-10-06T03:34:35.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/3545345709 >> v2:172.21.15.163:3300/0 conn(0x7fe0c00cc8d0 msgr2=0x7fe0c015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:35.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bf7fe700 1 --2- 172.21.15.55:0/3545345709 >> v2:172.21.15.163:3300/0 conn(0x7fe0c00cc8d0 0x7fe0c015b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/3545345709 >> v1:172.21.15.163:6789/0 conn(0x7fe0c015b960 legacy=0x7fe0c00ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:35.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/3545345709 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c00ca9e0 con 0x7fe0c015b050 2023-10-06T03:34:35.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/3545345709 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fe0b4001470 con 0x7fe0c015b050 2023-10-06T03:34:35.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/3545345709 >> v1:172.21.15.55:6789/0 conn(0x7fe0c015b050 legacy=0x7fe0c00ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:35.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bf7fe700 1 --2- 172.21.15.55:0/3545345709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0003a60 0x7fe0b0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:35.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bf7fe700 1 --2- 172.21.15.55:0/3545345709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0b0004810 0x7fe0b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:35.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bf7fe700 1 --2- 172.21.15.55:0/3545345709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0b00051c0 0x7fe0b000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:35.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/3545345709 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c01591b0 con 0x7fe0b0004810 2023-10-06T03:34:35.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/3545345709 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b000c050 con 0x7fe0b0003a60 2023-10-06T03:34:35.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/3545345709 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b000c210 con 0x7fe0b00051c0 2023-10-06T03:34:35.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0bffff700 1 --2- 172.21.15.55:0/3545345709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0b0004810 0x7fe0b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:35.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0c4b9b700 1 --2- 172.21.15.55:0/3545345709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0003a60 0x7fe0b0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:35.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.054+0000 7fe0c539c700 1 --2- 172.21.15.55:0/3545345709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0b00051c0 0x7fe0b000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:35.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.055+0000 7fe0bffff700 1 -- 172.21.15.55:0/3545345709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0b00051c0 msgr2=0x7fe0b000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:35.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.055+0000 7fe0bffff700 1 --2- 172.21.15.55:0/3545345709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0b00051c0 0x7fe0b000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.055+0000 7fe0bffff700 1 -- 172.21.15.55:0/3545345709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0003a60 msgr2=0x7fe0b0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:35.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.055+0000 7fe0bffff700 1 --2- 172.21.15.55:0/3545345709 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0003a60 0x7fe0b0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.055+0000 7fe0bffff700 1 -- 172.21.15.55:0/3545345709 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b0003570 con 0x7fe0b0004810 2023-10-06T03:34:35.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.055+0000 7fe0c539c700 1 -- 172.21.15.55:0/3545345709 reap_dead start 2023-10-06T03:34:35.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.055+0000 7fe0bffff700 1 --2- 172.21.15.55:0/3545345709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0b0004810 0x7fe0b0004c80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009c00 tx=0x7fe0ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45af2d75c8e7928a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:35.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.055+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/3545345709 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00e710 con 0x7fe0b0004810 2023-10-06T03:34:35.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.055+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/3545345709 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac00e870 con 0x7fe0b0004810 2023-10-06T03:34:35.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.055+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/3545345709 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00ec30 con 0x7fe0b0004810 2023-10-06T03:34:35.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.056+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/3545345709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0b0004810 msgr2=0x7fe0b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:35.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.056+0000 7fe0c5b9d700 1 --2- 172.21.15.55:0/3545345709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0b0004810 0x7fe0b0004c80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009c00 tx=0x7fe0ac0091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.056+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/3545345709 shutdown_connections 2023-10-06T03:34:35.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.056+0000 7fe0c5b9d700 1 --2- 172.21.15.55:0/3545345709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0b0004810 0x7fe0b0004c80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.056+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/3545345709 >> 172.21.15.55:0/3545345709 conn(0x7fe0c0068e20 msgr2=0x7fe0c00cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:35.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.056+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/3545345709 shutdown_connections 2023-10-06T03:34:35.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.056+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/3545345709 wait complete. 2023-10-06T03:34:35.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.057+0000 7fe0c5b9d700 1 Processor -- start 2023-10-06T03:34:35.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.057+0000 7fe0c5b9d700 1 -- start start 2023-10-06T03:34:35.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.058+0000 7fe0c5b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0004810 0x7fe0c0167af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:35.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.058+0000 7fe0c5b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c014ede0 0x7fe0c014ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:35.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.058+0000 7fe0c5b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c0069c10 0x7fe0c006a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:35.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.058+0000 7fe0c5b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c006a5d0 con 0x7fe0c0069c10 2023-10-06T03:34:35.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.058+0000 7fe0c5b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c0168380 con 0x7fe0b0004810 2023-10-06T03:34:35.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.058+0000 7fe0c5b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c0168660 con 0x7fe0c014ede0 2023-10-06T03:34:35.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.059+0000 7fe0c539c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c0069c10 0x7fe0c006a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:35.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.059+0000 7fe0c4b9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0004810 0x7fe0c0167af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:35.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.059+0000 7fe0c539c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c0069c10 0x7fe0c006a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44080/0 (socket says 172.21.15.55:44080) 2023-10-06T03:34:35.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.059+0000 7fe0c539c700 1 -- 172.21.15.55:0/875408138 learned_addr learned my addr 172.21.15.55:0/875408138 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:35.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.059+0000 7fe0bffff700 1 --2- 172.21.15.55:0/875408138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c014ede0 0x7fe0c014ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:35.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.059+0000 7fe0c539c700 1 -- 172.21.15.55:0/875408138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c014ede0 msgr2=0x7fe0c014ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:35.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.059+0000 7fe0c539c700 1 --2- 172.21.15.55:0/875408138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c014ede0 0x7fe0c014ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.059+0000 7fe0c539c700 1 -- 172.21.15.55:0/875408138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0004810 msgr2=0x7fe0c0167af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:35.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.059+0000 7fe0c539c700 1 --2- 172.21.15.55:0/875408138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0004810 0x7fe0c0167af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.059+0000 7fe0c539c700 1 -- 172.21.15.55:0/875408138 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c0168940 con 0x7fe0c0069c10 2023-10-06T03:34:35.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.059+0000 7fe0c539c700 1 --2- 172.21.15.55:0/875408138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c0069c10 0x7fe0c006a090 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00b2260 tx=0x7fe0b8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:35.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.059+0000 7fe0bcff9700 1 -- 172.21.15.55:0/875408138 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b80032e0 con 0x7fe0c0069c10 2023-10-06T03:34:35.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.060+0000 7fe0bcff9700 1 -- 172.21.15.55:0/875408138 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b8003480 con 0x7fe0c0069c10 2023-10-06T03:34:35.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.060+0000 7fe0bcff9700 1 -- 172.21.15.55:0/875408138 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8006df0 con 0x7fe0c0069c10 2023-10-06T03:34:35.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.060+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/875408138 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c014f710 con 0x7fe0c0069c10 2023-10-06T03:34:35.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.060+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/875408138 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c0168030 con 0x7fe0c0069c10 2023-10-06T03:34:35.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.061+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/875408138 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c004c3c0 con 0x7fe0c0069c10 2023-10-06T03:34:35.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.061+0000 7fe0bcff9700 1 -- 172.21.15.55:0/875408138 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) v1 ==== 84595+0+0 (secure 0 0 0) 0x7fe0b8001850 con 0x7fe0c0069c10 2023-10-06T03:34:35.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.062+0000 7fe0bcff9700 1 --2- 172.21.15.55:0/875408138 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe0a4066dd0 0x7fe0a4069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:35.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.062+0000 7fe0bcff9700 1 -- 172.21.15.55:0/875408138 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fe0b80839f0 con 0x7fe0c0069c10 2023-10-06T03:34:35.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.063+0000 7fe0c4b9b700 1 --2- 172.21.15.55:0/875408138 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe0a4066dd0 0x7fe0a4069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:35.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.063+0000 7fe0c4b9b700 1 --2- 172.21.15.55:0/875408138 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe0a4066dd0 0x7fe0a4069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0066e40 tx=0x7fe0b4007040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:35.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.067+0000 7fe0bcff9700 1 -- 172.21.15.55:0/875408138 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0b804d5a0 con 0x7fe0c0069c10 2023-10-06T03:34:35.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.294+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/875408138 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0c00cacc0 con 0x7fe0a4066dd0 2023-10-06T03:34:35.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.388+0000 7fe0bcff9700 1 -- 172.21.15.55:0/875408138 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe0c00cacc0 con 0x7fe0a4066dd0 2023-10-06T03:34:35.389 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path /snap_test_dir1/dir1 2023-10-06T03:34:35.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.390+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/875408138 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe0a4066dd0 msgr2=0x7fe0a4069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:35.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.390+0000 7fe0c5b9d700 1 --2- 172.21.15.55:0/875408138 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe0a4066dd0 0x7fe0a4069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0066e40 tx=0x7fe0b4007040 comp rx=0 tx=0).stop 2023-10-06T03:34:35.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.390+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/875408138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c0069c10 msgr2=0x7fe0c006a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:35.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.390+0000 7fe0c5b9d700 1 --2- 172.21.15.55:0/875408138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c0069c10 0x7fe0c006a090 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00b2260 tx=0x7fe0b8009040 comp rx=0 tx=0).stop 2023-10-06T03:34:35.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.390+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/875408138 shutdown_connections 2023-10-06T03:34:35.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.390+0000 7fe0c5b9d700 1 --2- 172.21.15.55:0/875408138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c0069c10 0x7fe0c006a090 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.390+0000 7fe0c5b9d700 1 --2- 172.21.15.55:0/875408138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0004810 0x7fe0c0167af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.390+0000 7fe0c5b9d700 1 --2- 172.21.15.55:0/875408138 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe0a4066dd0 0x7fe0a4069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.391+0000 7fe0c5b9d700 1 --2- 172.21.15.55:0/875408138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c014ede0 0x7fe0c014ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.391+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/875408138 >> 172.21.15.55:0/875408138 conn(0x7fe0c0068e20 msgr2=0x7fe0c00b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:35.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.391+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/875408138 shutdown_connections 2023-10-06T03:34:35.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.391+0000 7fe0c5b9d700 1 -- 172.21.15.55:0/875408138 wait complete. 2023-10-06T03:34:35.409 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-10-06T03:34:35.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:34:35.410 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-10-06T03:34:35.449 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir2 1M 2023-10-06T03:34:35.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.681+0000 7fa14d9ff700 1 Processor -- start 2023-10-06T03:34:35.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.681+0000 7fa14d9ff700 1 -- start start 2023-10-06T03:34:35.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.682+0000 7fa14d9ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148157740 0x7fa1480adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:35.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.682+0000 7fa14d9ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa14806dc20 con 0x7fa1480b2100 2023-10-06T03:34:35.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.682+0000 7fa14d9ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1480ae910 con 0x7fa148068aa0 2023-10-06T03:34:35.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.682+0000 7fa14d9ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1480acc00 con 0x7fa148157740 2023-10-06T03:34:35.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.683+0000 7fa1467fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa1480b2100 0x7fa148068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35616/0 (socket says 172.21.15.55:35616) 2023-10-06T03:34:35.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.683+0000 7fa1467fc700 1 -- 172.21.15.55:0/4249995843 learned_addr learned my addr 172.21.15.55:0/4249995843 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:35.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.683+0000 7fa146ffd700 1 --2- 172.21.15.55:0/4249995843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148157740 0x7fa1480adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:35.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.683+0000 7fa145ffb700 1 -- 172.21.15.55:0/4249995843 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446669296 0 0) 0x7fa14806dc20 con 0x7fa1480b2100 2023-10-06T03:34:35.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.683+0000 7fa145ffb700 1 -- 172.21.15.55:0/4249995843 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa130003570 con 0x7fa1480b2100 2023-10-06T03:34:35.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.683+0000 7fa146ffd700 1 -- 172.21.15.55:0/4249995843 >> v1:172.21.15.163:6789/0 conn(0x7fa148068aa0 legacy=0x7fa14806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:35.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.683+0000 7fa146ffd700 1 -- 172.21.15.55:0/4249995843 >> v1:172.21.15.55:6789/0 conn(0x7fa1480b2100 legacy=0x7fa148068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:35.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.683+0000 7fa146ffd700 1 -- 172.21.15.55:0/4249995843 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1480aebf0 con 0x7fa148157740 2023-10-06T03:34:35.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.684+0000 7fa146ffd700 1 --2- 172.21.15.55:0/4249995843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148157740 0x7fa1480adfc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa138009910 tx=0x7fa1380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9572ee4d032c6c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:35.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.684+0000 7fa145ffb700 1 -- 172.21.15.55:0/4249995843 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13800b680 con 0x7fa148157740 2023-10-06T03:34:35.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.684+0000 7fa145ffb700 1 -- 172.21.15.55:0/4249995843 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13800b7e0 con 0x7fa148157740 2023-10-06T03:34:35.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.684+0000 7fa145ffb700 1 -- 172.21.15.55:0/4249995843 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13800bba0 con 0x7fa148157740 2023-10-06T03:34:35.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.684+0000 7fa14d9ff700 1 -- 172.21.15.55:0/4249995843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148157740 msgr2=0x7fa1480adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:35.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.684+0000 7fa14d9ff700 1 --2- 172.21.15.55:0/4249995843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148157740 0x7fa1480adfc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa138009910 tx=0x7fa1380091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.684+0000 7fa14d9ff700 1 -- 172.21.15.55:0/4249995843 shutdown_connections 2023-10-06T03:34:35.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.685+0000 7fa14d9ff700 1 --2- 172.21.15.55:0/4249995843 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148157740 0x7fa1480adfc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.685+0000 7fa14d9ff700 1 -- 172.21.15.55:0/4249995843 >> 172.21.15.55:0/4249995843 conn(0x7fa1480c8f90 msgr2=0x7fa1480c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:35.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.685+0000 7fa14d9ff700 1 -- 172.21.15.55:0/4249995843 shutdown_connections 2023-10-06T03:34:35.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.685+0000 7fa14d9ff700 1 -- 172.21.15.55:0/4249995843 wait complete. 2023-10-06T03:34:35.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.686+0000 7fa14d9ff700 1 Processor -- start 2023-10-06T03:34:35.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.686+0000 7fa14d9ff700 1 -- start start 2023-10-06T03:34:35.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.687+0000 7fa14d9ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa148068aa0 0x7fa1480b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:35.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.687+0000 7fa14d9ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1480b2100 0x7fa148169c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:35.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.687+0000 7fa14d9ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148157740 0x7fa1481489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:35.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.687+0000 7fa14d9ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14816a4e0 con 0x7fa1480b2100 2023-10-06T03:34:35.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.687+0000 7fa14d9ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14816a7c0 con 0x7fa148068aa0 2023-10-06T03:34:35.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.687+0000 7fa14d9ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14816aaa0 con 0x7fa148157740 2023-10-06T03:34:35.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.687+0000 7fa1467fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1480b2100 0x7fa148169c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:35.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.687+0000 7fa1467fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1480b2100 0x7fa148169c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44092/0 (socket says 172.21.15.55:44092) 2023-10-06T03:34:35.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.687+0000 7fa1467fc700 1 -- 172.21.15.55:0/2695404572 learned_addr learned my addr 172.21.15.55:0/2695404572 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:35.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.688+0000 7fa146ffd700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa148068aa0 0x7fa1480b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:35.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.688+0000 7fa1477fe700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148157740 0x7fa1481489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:35.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.688+0000 7fa1467fc700 1 -- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148157740 msgr2=0x7fa1481489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:35.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.688+0000 7fa1467fc700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148157740 0x7fa1481489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.688+0000 7fa1467fc700 1 -- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa148068aa0 msgr2=0x7fa1480b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:35.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.688+0000 7fa1467fc700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa148068aa0 0x7fa1480b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:35.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.688+0000 7fa1467fc700 1 -- 172.21.15.55:0/2695404572 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa138009420 con 0x7fa1480b2100 2023-10-06T03:34:35.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.688+0000 7fa1477fe700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148157740 0x7fa1481489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:35.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.688+0000 7fa1467fc700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1480b2100 0x7fa148169c90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00c7c0 tx=0x7fa13c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:35.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.688+0000 7fa12effd700 1 -- 172.21.15.55:0/2695404572 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c005b30 con 0x7fa1480b2100 2023-10-06T03:34:35.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.688+0000 7fa12effd700 1 -- 172.21.15.55:0/2695404572 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c005cb0 con 0x7fa1480b2100 2023-10-06T03:34:35.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.688+0000 7fa14d9ff700 1 -- 172.21.15.55:0/2695404572 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14816ad80 con 0x7fa1480b2100 2023-10-06T03:34:35.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.688+0000 7fa12effd700 1 -- 172.21.15.55:0/2695404572 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c007320 con 0x7fa1480b2100 2023-10-06T03:34:35.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.688+0000 7fa14d9ff700 1 -- 172.21.15.55:0/2695404572 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14816a1d0 con 0x7fa1480b2100 2023-10-06T03:34:35.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.689+0000 7fa14d9ff700 1 -- 172.21.15.55:0/2695404572 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1480718b0 con 0x7fa1480b2100 2023-10-06T03:34:35.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.690+0000 7fa12effd700 1 -- 172.21.15.55:0/2695404572 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) v1 ==== 84595+0+0 (secure 0 0 0) 0x7fa13c01b3b0 con 0x7fa1480b2100 2023-10-06T03:34:35.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.691+0000 7fa12effd700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa134066dd0 0x7fa134069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:35.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.691+0000 7fa12effd700 1 -- 172.21.15.55:0/2695404572 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fa13c083a60 con 0x7fa1480b2100 2023-10-06T03:34:35.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.691+0000 7fa146ffd700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa134066dd0 0x7fa134069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:35.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.692+0000 7fa146ffd700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa134066dd0 0x7fa134069280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa1380093f0 tx=0x7fa138017000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:35.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.696+0000 7fa12effd700 1 -- 172.21.15.55:0/2695404572 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa13c050c50 con 0x7fa1480b2100 2023-10-06T03:34:35.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:35.920+0000 7fa14d9ff700 1 -- 172.21.15.55:0/2695404572 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1480678d0 con 0x7fa134066dd0 2023-10-06T03:34:36.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.001+0000 7fa12effd700 1 -- 172.21.15.55:0/2695404572 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa1480678d0 con 0x7fa134066dd0 2023-10-06T03:34:36.002 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path /snap_test_dir1/dir2 2023-10-06T03:34:36.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.003+0000 7fa14d9ff700 1 -- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa134066dd0 msgr2=0x7fa134069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:36.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.003+0000 7fa14d9ff700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa134066dd0 0x7fa134069280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa1380093f0 tx=0x7fa138017000 comp rx=0 tx=0).stop 2023-10-06T03:34:36.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.003+0000 7fa14d9ff700 1 -- 172.21.15.55:0/2695404572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1480b2100 msgr2=0x7fa148169c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:36.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.003+0000 7fa14d9ff700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1480b2100 0x7fa148169c90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00c7c0 tx=0x7fa13c007e00 comp rx=0 tx=0).stop 2023-10-06T03:34:36.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.003+0000 7fa14d9ff700 1 -- 172.21.15.55:0/2695404572 shutdown_connections 2023-10-06T03:34:36.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.003+0000 7fa14d9ff700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa134066dd0 0x7fa134069280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.003+0000 7fa14d9ff700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148157740 0x7fa1481489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.003+0000 7fa14d9ff700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa148068aa0 0x7fa1480b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.003+0000 7fa14d9ff700 1 --2- 172.21.15.55:0/2695404572 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1480b2100 0x7fa148169c90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.003+0000 7fa14d9ff700 1 -- 172.21.15.55:0/2695404572 >> 172.21.15.55:0/2695404572 conn(0x7fa1480c8f90 msgr2=0x7fa148156a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:36.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.003+0000 7fa14d9ff700 1 -- 172.21.15.55:0/2695404572 shutdown_connections 2023-10-06T03:34:36.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.003+0000 7fa14d9ff700 1 -- 172.21.15.55:0/2695404572 wait complete. 2023-10-06T03:34:36.022 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-10-06T03:34:36.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:34:36.022 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-10-06T03:34:36.061 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir3 1M 2023-10-06T03:34:36.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.294+0000 7f226f59e700 1 Processor -- start 2023-10-06T03:34:36.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.294+0000 7f226f59e700 1 -- start start 2023-10-06T03:34:36.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.295+0000 7f226f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f227015ca70 0x7f22701645a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:36.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.295+0000 7f226f59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2270159db0 con 0x7f22700cb560 2023-10-06T03:34:36.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.295+0000 7f226f59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227006a470 con 0x7f22700ccb40 2023-10-06T03:34:36.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.295+0000 7f226f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227006aa30 con 0x7f227015ca70 2023-10-06T03:34:36.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.295+0000 7f226e59c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f22700cb560 0x7f22700cc430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35622/0 (socket says 172.21.15.55:35622) 2023-10-06T03:34:36.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.295+0000 7f226e59c700 1 -- 172.21.15.55:0/2235758121 learned_addr learned my addr 172.21.15.55:0/2235758121 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:36.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226ed9d700 1 --2- 172.21.15.55:0/2235758121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f227015ca70 0x7f22701645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:36.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826682038 0 0) 0x7f2270159db0 con 0x7f22700cb560 2023-10-06T03:34:36.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2258003570 con 0x7f22700cb560 2023-10-06T03:34:36.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148380687 0 0) 0x7f227006a470 con 0x7f22700ccb40 2023-10-06T03:34:36.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2270159db0 con 0x7f22700ccb40 2023-10-06T03:34:36.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3969192234 0 0) 0x7f2258003570 con 0x7f22700cb560 2023-10-06T03:34:36.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f227015ca70 msgr2=0x7f22701645a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:36.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 --2- 172.21.15.55:0/2235758121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f227015ca70 0x7f22701645a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 >> v1:172.21.15.163:6789/0 conn(0x7f22700ccb40 legacy=0x7f22700697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:36.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227006a750 con 0x7f22700cb560 2023-10-06T03:34:36.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f2260001470 con 0x7f22700cb560 2023-10-06T03:34:36.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 >> v1:172.21.15.55:6789/0 conn(0x7f22700cb560 legacy=0x7f22700cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:36.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 --2- 172.21.15.55:0/2235758121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2258003c20 0x7f2258004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:36.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 --2- 172.21.15.55:0/2235758121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2258004810 0x7f2258004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:36.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 --2- 172.21.15.55:0/2235758121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f227015ca70 0x7f22580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:36.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227006aa30 con 0x7f2258003c20 2023-10-06T03:34:36.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225800bb40 con 0x7f2258004810 2023-10-06T03:34:36.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.296+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225800bd00 con 0x7f227015ca70 2023-10-06T03:34:36.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.297+0000 7f226e59c700 1 --2- 172.21.15.55:0/2235758121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2258003c20 0x7f2258004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:36.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.297+0000 7f226dd9b700 1 --2- 172.21.15.55:0/2235758121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2258004810 0x7f2258004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:36.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.297+0000 7f226ed9d700 1 --2- 172.21.15.55:0/2235758121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f227015ca70 0x7f22580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:36.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.297+0000 7f226e59c700 1 -- 172.21.15.55:0/2235758121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f227015ca70 msgr2=0x7f22580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:36.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.297+0000 7f226e59c700 1 --2- 172.21.15.55:0/2235758121 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f227015ca70 0x7f22580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.297+0000 7f226e59c700 1 -- 172.21.15.55:0/2235758121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2258004810 msgr2=0x7f2258004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:36.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.297+0000 7f226e59c700 1 --2- 172.21.15.55:0/2235758121 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2258004810 0x7f2258004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.297+0000 7f226e59c700 1 -- 172.21.15.55:0/2235758121 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2258003570 con 0x7f2258003c20 2023-10-06T03:34:36.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.297+0000 7f226e59c700 1 --2- 172.21.15.55:0/2235758121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2258003c20 0x7f2258004010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f226000c2d0 tx=0x7f2260001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6bb2de87aa42bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:36.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.298+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226000cbc0 con 0x7f2258003c20 2023-10-06T03:34:36.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.298+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226000cd20 con 0x7f2258003c20 2023-10-06T03:34:36.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.298+0000 7f226d59a700 1 -- 172.21.15.55:0/2235758121 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2260005b70 con 0x7f2258003c20 2023-10-06T03:34:36.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.298+0000 7f226f59e700 1 -- 172.21.15.55:0/2235758121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2258003c20 msgr2=0x7f2258004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:36.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.298+0000 7f226f59e700 1 --2- 172.21.15.55:0/2235758121 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2258003c20 0x7f2258004010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f226000c2d0 tx=0x7f2260001e10 comp rx=0 tx=0).stop 2023-10-06T03:34:36.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.298+0000 7f226ed9d700 1 -- 172.21.15.55:0/2235758121 reap_dead start 2023-10-06T03:34:36.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.298+0000 7f226f59e700 1 -- 172.21.15.55:0/2235758121 shutdown_connections 2023-10-06T03:34:36.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.298+0000 7f226f59e700 1 -- 172.21.15.55:0/2235758121 >> 172.21.15.55:0/2235758121 conn(0x7f227006baf0 msgr2=0x7f227006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:36.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.298+0000 7f226f59e700 1 -- 172.21.15.55:0/2235758121 shutdown_connections 2023-10-06T03:34:36.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.298+0000 7f226f59e700 1 -- 172.21.15.55:0/2235758121 wait complete. 2023-10-06T03:34:36.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.300+0000 7f226f59e700 1 Processor -- start 2023-10-06T03:34:36.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.300+0000 7f226f59e700 1 -- start start 2023-10-06T03:34:36.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2258003c20 0x7f227016a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:36.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22700b37c0 0x7f22700b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:36.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22700c5700 0x7f22700c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:36.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22700c6550 con 0x7f22700c5700 2023-10-06T03:34:36.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22700c60b0 con 0x7f22700b37c0 2023-10-06T03:34:36.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22700c79f0 con 0x7f2258003c20 2023-10-06T03:34:36.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22700c5700 0x7f22700c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:36.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22700c5700 0x7f22700c5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44106/0 (socket says 172.21.15.55:44106) 2023-10-06T03:34:36.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226ed9d700 1 -- 172.21.15.55:0/2221114959 learned_addr learned my addr 172.21.15.55:0/2221114959 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:36.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226dd9b700 1 --2- 172.21.15.55:0/2221114959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22700b37c0 0x7f22700b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:36.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226e59c700 1 --2- 172.21.15.55:0/2221114959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2258003c20 0x7f227016a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:36.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226ed9d700 1 -- 172.21.15.55:0/2221114959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2258003c20 msgr2=0x7f227016a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:36.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226ed9d700 1 --2- 172.21.15.55:0/2221114959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2258003c20 0x7f227016a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226ed9d700 1 -- 172.21.15.55:0/2221114959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22700b37c0 msgr2=0x7f22700b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:36.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226ed9d700 1 --2- 172.21.15.55:0/2221114959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22700b37c0 0x7f22700b2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.301+0000 7f226ed9d700 1 -- 172.21.15.55:0/2221114959 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22700c6de0 con 0x7f22700c5700 2023-10-06T03:34:36.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.302+0000 7f226ed9d700 1 --2- 172.21.15.55:0/2221114959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22700c5700 0x7f22700c5b70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f22700b4650 tx=0x7f226800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:36.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.302+0000 7f225e7fc700 1 -- 172.21.15.55:0/2221114959 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226800e040 con 0x7f22700c5700 2023-10-06T03:34:36.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.302+0000 7f225e7fc700 1 -- 172.21.15.55:0/2221114959 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226800f040 con 0x7f22700c5700 2023-10-06T03:34:36.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.302+0000 7f225e7fc700 1 -- 172.21.15.55:0/2221114959 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226800fe20 con 0x7f22700c5700 2023-10-06T03:34:36.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.302+0000 7f226f59e700 1 -- 172.21.15.55:0/2221114959 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22700c70c0 con 0x7f22700c5700 2023-10-06T03:34:36.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.302+0000 7f226f59e700 1 -- 172.21.15.55:0/2221114959 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22700c7680 con 0x7f22700c5700 2023-10-06T03:34:36.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.303+0000 7f226f59e700 1 -- 172.21.15.55:0/2221114959 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227004c3c0 con 0x7f22700c5700 2023-10-06T03:34:36.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.304+0000 7f225e7fc700 1 -- 172.21.15.55:0/2221114959 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 8) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f2268015640 con 0x7f22700c5700 2023-10-06T03:34:36.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.305+0000 7f225e7fc700 1 --2- 172.21.15.55:0/2221114959 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2250066dd0 0x7f2250069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:36.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.305+0000 7f225e7fc700 1 -- 172.21.15.55:0/2221114959 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f2268084f20 con 0x7f22700c5700 2023-10-06T03:34:36.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.305+0000 7f226e59c700 1 --2- 172.21.15.55:0/2221114959 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2250066dd0 0x7f2250069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:36.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.306+0000 7f226e59c700 1 --2- 172.21.15.55:0/2221114959 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2250066dd0 0x7f2250069280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2260008fd0 tx=0x7f2260015000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:36.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.309+0000 7f225e7fc700 1 -- 172.21.15.55:0/2221114959 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2268052110 con 0x7f22700c5700 2023-10-06T03:34:36.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.494+0000 7f225e7fc700 1 -- 172.21.15.55:0/2221114959 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f22680524e0 con 0x7f22700c5700 2023-10-06T03:34:36.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.542+0000 7f226f59e700 1 -- 172.21.15.55:0/2221114959 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f22700c7b30 con 0x7f2250066dd0 2023-10-06T03:34:36.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.616+0000 7f225e7fc700 1 -- 172.21.15.55:0/2221114959 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f22700c7b30 con 0x7f2250066dd0 2023-10-06T03:34:36.617 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path /snap_test_dir1/dir3 2023-10-06T03:34:36.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.618+0000 7f226f59e700 1 -- 172.21.15.55:0/2221114959 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2250066dd0 msgr2=0x7f2250069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:36.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.618+0000 7f226f59e700 1 --2- 172.21.15.55:0/2221114959 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2250066dd0 0x7f2250069280 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2260008fd0 tx=0x7f2260015000 comp rx=0 tx=0).stop 2023-10-06T03:34:36.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.618+0000 7f226f59e700 1 -- 172.21.15.55:0/2221114959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22700c5700 msgr2=0x7f22700c5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:36.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.618+0000 7f226f59e700 1 --2- 172.21.15.55:0/2221114959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22700c5700 0x7f22700c5b70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f22700b4650 tx=0x7f226800b040 comp rx=0 tx=0).stop 2023-10-06T03:34:36.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.618+0000 7f226f59e700 1 -- 172.21.15.55:0/2221114959 shutdown_connections 2023-10-06T03:34:36.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.618+0000 7f226f59e700 1 --2- 172.21.15.55:0/2221114959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22700c5700 0x7f22700c5b70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.618+0000 7f226f59e700 1 --2- 172.21.15.55:0/2221114959 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2250066dd0 0x7f2250069280 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.618+0000 7f226f59e700 1 --2- 172.21.15.55:0/2221114959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2258003c20 0x7f227016a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.618+0000 7f226f59e700 1 --2- 172.21.15.55:0/2221114959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22700b37c0 0x7f22700b2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.618+0000 7f226f59e700 1 -- 172.21.15.55:0/2221114959 >> 172.21.15.55:0/2221114959 conn(0x7f227006baf0 msgr2=0x7f22700ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:36.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.618+0000 7f226f59e700 1 -- 172.21.15.55:0/2221114959 shutdown_connections 2023-10-06T03:34:36.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.618+0000 7f226f59e700 1 -- 172.21.15.55:0/2221114959 wait complete. 2023-10-06T03:34:36.637 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-10-06T03:34:36.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:34:36.637 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-10-06T03:34:36.677 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir4 1M 2023-10-06T03:34:36.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.908+0000 7f68d759e700 1 Processor -- start 2023-10-06T03:34:36.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.909+0000 7f68d759e700 1 -- start start 2023-10-06T03:34:36.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.910+0000 7f68d759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d815b050 0x7f68d80ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:36.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.910+0000 7f68d759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f68d815bb30 0x7f68d80ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:36.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.910+0000 7f68d759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d8157a50 con 0x7f68d80cc8d0 2023-10-06T03:34:36.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.910+0000 7f68d759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d8158f10 con 0x7f68d815bb30 2023-10-06T03:34:36.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.910+0000 7f68d759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d81591b0 con 0x7f68d815b050 2023-10-06T03:34:36.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.910+0000 7f68d6d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f68d815bb30 0x7f68d80ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:36.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.910+0000 7f68d5d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f68d80cc8d0 0x7f68d815b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35634/0 (socket says 172.21.15.55:35634) 2023-10-06T03:34:36.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.910+0000 7f68d5d9b700 1 -- 172.21.15.55:0/1330032625 learned_addr learned my addr 172.21.15.55:0/1330032625 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:36.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.910+0000 7f68d659c700 1 --2- 172.21.15.55:0/1330032625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d815b050 0x7f68d80ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:36.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.911+0000 7f68d6d9d700 1 -- 172.21.15.55:0/1330032625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d815b050 msgr2=0x7f68d80ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:36.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.911+0000 7f68d6d9d700 1 --2- 172.21.15.55:0/1330032625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d815b050 0x7f68d80ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.911+0000 7f68d6d9d700 1 -- 172.21.15.55:0/1330032625 >> v1:172.21.15.55:6789/0 conn(0x7f68d80cc8d0 legacy=0x7f68d815b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:36.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.911+0000 7f68d6d9d700 1 -- 172.21.15.55:0/1330032625 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d80ca9e0 con 0x7f68d815bb30 2023-10-06T03:34:36.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.911+0000 7f68d6d9d700 1 --2- 172.21.15.55:0/1330032625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d815bb30 0x7f68d80ada30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009e60 tx=0x7f68d0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69b26dbadbe9818 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:36.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.911+0000 7f68d559a700 1 -- 172.21.15.55:0/1330032625 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d000e070 con 0x7f68d815bb30 2023-10-06T03:34:36.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.911+0000 7f68d559a700 1 -- 172.21.15.55:0/1330032625 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d000bc30 con 0x7f68d815bb30 2023-10-06T03:34:36.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.911+0000 7f68d559a700 1 -- 172.21.15.55:0/1330032625 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d0012430 con 0x7f68d815bb30 2023-10-06T03:34:36.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.912+0000 7f68d759e700 1 -- 172.21.15.55:0/1330032625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d815bb30 msgr2=0x7f68d80ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:36.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.912+0000 7f68d759e700 1 --2- 172.21.15.55:0/1330032625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d815bb30 0x7f68d80ada30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009e60 tx=0x7f68d0009780 comp rx=0 tx=0).stop 2023-10-06T03:34:36.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.912+0000 7f68d759e700 1 -- 172.21.15.55:0/1330032625 shutdown_connections 2023-10-06T03:34:36.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.912+0000 7f68d759e700 1 --2- 172.21.15.55:0/1330032625 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d815bb30 0x7f68d80ada30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.912+0000 7f68d759e700 1 --2- 172.21.15.55:0/1330032625 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d815b050 0x7f68d80ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.912+0000 7f68d759e700 1 -- 172.21.15.55:0/1330032625 >> 172.21.15.55:0/1330032625 conn(0x7f68d8068e20 msgr2=0x7f68d80cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:36.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.912+0000 7f68d759e700 1 -- 172.21.15.55:0/1330032625 shutdown_connections 2023-10-06T03:34:36.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.912+0000 7f68d759e700 1 -- 172.21.15.55:0/1330032625 wait complete. 2023-10-06T03:34:36.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.913+0000 7f68d759e700 1 Processor -- start 2023-10-06T03:34:36.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.914+0000 7f68d759e700 1 -- start start 2023-10-06T03:34:36.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.914+0000 7f68d759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d80cc8d0 0x7f68d80a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:36.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.914+0000 7f68d759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d815b050 0x7f68d80b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:36.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.914+0000 7f68d759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d815bb30 0x7f68d80b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:36.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.914+0000 7f68d759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d80a5880 con 0x7f68d80cc8d0 2023-10-06T03:34:36.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.914+0000 7f68d759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d815efa0 con 0x7f68d815bb30 2023-10-06T03:34:36.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.914+0000 7f68d759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d815e390 con 0x7f68d815b050 2023-10-06T03:34:36.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.915+0000 7f68d659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d80cc8d0 0x7f68d80a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:36.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.915+0000 7f68d659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d80cc8d0 0x7f68d80a6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44122/0 (socket says 172.21.15.55:44122) 2023-10-06T03:34:36.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.915+0000 7f68d659c700 1 -- 172.21.15.55:0/1343185427 learned_addr learned my addr 172.21.15.55:0/1343185427 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:36.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.915+0000 7f68d5d9b700 1 --2- 172.21.15.55:0/1343185427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d815b050 0x7f68d80b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:36.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.915+0000 7f68d6d9d700 1 --2- 172.21.15.55:0/1343185427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d815bb30 0x7f68d80b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:36.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.915+0000 7f68d659c700 1 -- 172.21.15.55:0/1343185427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d815b050 msgr2=0x7f68d80b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:36.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.915+0000 7f68d659c700 1 --2- 172.21.15.55:0/1343185427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d815b050 0x7f68d80b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.915+0000 7f68d659c700 1 -- 172.21.15.55:0/1343185427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d815bb30 msgr2=0x7f68d80b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:36.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.915+0000 7f68d659c700 1 --2- 172.21.15.55:0/1343185427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d815bb30 0x7f68d80b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:36.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.915+0000 7f68d659c700 1 -- 172.21.15.55:0/1343185427 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d0009a20 con 0x7f68d80cc8d0 2023-10-06T03:34:36.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.915+0000 7f68d659c700 1 --2- 172.21.15.55:0/1343185427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d80cc8d0 0x7f68d80a6a20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f68c800d890 tx=0x7f68c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:36.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.916+0000 7f68c67fc700 1 -- 172.21.15.55:0/1343185427 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c800b600 con 0x7f68d80cc8d0 2023-10-06T03:34:36.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.916+0000 7f68c67fc700 1 -- 172.21.15.55:0/1343185427 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c800b760 con 0x7f68d80cc8d0 2023-10-06T03:34:36.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.916+0000 7f68c67fc700 1 -- 172.21.15.55:0/1343185427 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c800bae0 con 0x7f68d80cc8d0 2023-10-06T03:34:36.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.916+0000 7f68d759e700 1 -- 172.21.15.55:0/1343185427 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d815e670 con 0x7f68d80cc8d0 2023-10-06T03:34:36.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.916+0000 7f68d759e700 1 -- 172.21.15.55:0/1343185427 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d815ec30 con 0x7f68d80cc8d0 2023-10-06T03:34:36.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.917+0000 7f68d759e700 1 -- 172.21.15.55:0/1343185427 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d804c3c0 con 0x7f68d80cc8d0 2023-10-06T03:34:36.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.918+0000 7f68c67fc700 1 -- 172.21.15.55:0/1343185427 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f68c801e460 con 0x7f68d80cc8d0 2023-10-06T03:34:36.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.919+0000 7f68c67fc700 1 --2- 172.21.15.55:0/1343185427 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f68b8067200 0x7f68b80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:36.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.919+0000 7f68c67fc700 1 -- 172.21.15.55:0/1343185427 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f68c8085130 con 0x7f68d80cc8d0 2023-10-06T03:34:36.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.919+0000 7f68d5d9b700 1 --2- 172.21.15.55:0/1343185427 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f68b8067200 0x7f68b80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:36.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.920+0000 7f68d5d9b700 1 --2- 172.21.15.55:0/1343185427 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f68b8067200 0x7f68b80696b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f68d80b1770 tx=0x7f68cc009040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:36.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:36.923+0000 7f68c67fc700 1 -- 172.21.15.55:0/1343185427 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68c8051f40 con 0x7f68d80cc8d0 2023-10-06T03:34:37.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.150+0000 7f68d759e700 1 -- 172.21.15.55:0/1343185427 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f68d80cacc0 con 0x7f68b8067200 2023-10-06T03:34:37.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.223+0000 7f68c67fc700 1 -- 172.21.15.55:0/1343185427 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f68d80cacc0 con 0x7f68b8067200 2023-10-06T03:34:37.225 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path /snap_test_dir1/dir4 2023-10-06T03:34:37.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.226+0000 7f68d759e700 1 -- 172.21.15.55:0/1343185427 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f68b8067200 msgr2=0x7f68b80696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:37.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.226+0000 7f68d759e700 1 --2- 172.21.15.55:0/1343185427 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f68b8067200 0x7f68b80696b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f68d80b1770 tx=0x7f68cc009040 comp rx=0 tx=0).stop 2023-10-06T03:34:37.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.226+0000 7f68d759e700 1 -- 172.21.15.55:0/1343185427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d80cc8d0 msgr2=0x7f68d80a6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:37.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.226+0000 7f68d759e700 1 --2- 172.21.15.55:0/1343185427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d80cc8d0 0x7f68d80a6a20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f68c800d890 tx=0x7f68c8008040 comp rx=0 tx=0).stop 2023-10-06T03:34:37.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.226+0000 7f68d759e700 1 -- 172.21.15.55:0/1343185427 shutdown_connections 2023-10-06T03:34:37.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.226+0000 7f68d759e700 1 --2- 172.21.15.55:0/1343185427 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68d815bb30 0x7f68d80b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:37.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.226+0000 7f68d759e700 1 --2- 172.21.15.55:0/1343185427 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68d80cc8d0 0x7f68d80a6a20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:37.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.226+0000 7f68d759e700 1 --2- 172.21.15.55:0/1343185427 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f68b8067200 0x7f68b80696b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:37.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.226+0000 7f68d759e700 1 --2- 172.21.15.55:0/1343185427 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68d815b050 0x7f68d80b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:37.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.226+0000 7f68d759e700 1 -- 172.21.15.55:0/1343185427 >> 172.21.15.55:0/1343185427 conn(0x7f68d8068e20 msgr2=0x7f68d806d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:37.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.226+0000 7f68d759e700 1 -- 172.21.15.55:0/1343185427 shutdown_connections 2023-10-06T03:34:37.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.226+0000 7f68d759e700 1 -- 172.21.15.55:0/1343185427 wait complete. 2023-10-06T03:34:37.244 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-10-06T03:34:37.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:34:37.244 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-10-06T03:34:37.284 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir5 1M 2023-10-06T03:34:37.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.515+0000 7fb31979d700 1 Processor -- start 2023-10-06T03:34:37.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.515+0000 7fb31979d700 1 -- start start 2023-10-06T03:34:37.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.516+0000 7fb31979d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb3140b0760 0x7fb3140b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:37.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.516+0000 7fb31979d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb3140ace60 0x7fb3140ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:37.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.516+0000 7fb31979d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3140b0570 con 0x7fb314156350 2023-10-06T03:34:37.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.516+0000 7fb31979d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3140b1140 con 0x7fb3140ace60 2023-10-06T03:34:37.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.517+0000 7fb31979d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3140af4d0 con 0x7fb3140b0760 2023-10-06T03:34:37.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.517+0000 7fb318f9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb3140ace60 0x7fb3140ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:37.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.517+0000 7fb318f9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb3140ace60 0x7fb3140ad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44126/0 (socket says 172.21.15.55:44126) 2023-10-06T03:34:37.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.517+0000 7fb318f9c700 1 -- 172.21.15.55:0/2301233548 learned_addr learned my addr 172.21.15.55:0/2301233548 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:37.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.517+0000 7fb3137fe700 1 --2- 172.21.15.55:0/2301233548 >> v2:172.21.15.163:3300/0 conn(0x7fb3140b0760 0x7fb3140b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:37.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.517+0000 7fb318f9c700 1 -- 172.21.15.55:0/2301233548 >> v2:172.21.15.163:3300/0 conn(0x7fb3140b0760 msgr2=0x7fb3140b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:37.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.517+0000 7fb318f9c700 1 --2- 172.21.15.55:0/2301233548 >> v2:172.21.15.163:3300/0 conn(0x7fb3140b0760 0x7fb3140b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:37.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.517+0000 7fb318f9c700 1 -- 172.21.15.55:0/2301233548 >> v1:172.21.15.163:6789/0 conn(0x7fb314156350 legacy=0x7fb314157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:37.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.517+0000 7fb318f9c700 1 -- 172.21.15.55:0/2301233548 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3140af7b0 con 0x7fb3140ace60 2023-10-06T03:34:37.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.517+0000 7fb318f9c700 1 --2- 172.21.15.55:0/2301233548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3140ace60 0x7fb3140ad260 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb3140642a0 tx=0x7fb30c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445239f9d3730a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:37.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.518+0000 7fb312ffd700 1 -- 172.21.15.55:0/2301233548 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c0092e0 con 0x7fb3140ace60 2023-10-06T03:34:37.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.518+0000 7fb312ffd700 1 -- 172.21.15.55:0/2301233548 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c00ba50 con 0x7fb3140ace60 2023-10-06T03:34:37.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.518+0000 7fb312ffd700 1 -- 172.21.15.55:0/2301233548 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c00be10 con 0x7fb3140ace60 2023-10-06T03:34:37.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.518+0000 7fb31979d700 1 -- 172.21.15.55:0/2301233548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3140ace60 msgr2=0x7fb3140ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:37.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.518+0000 7fb31979d700 1 --2- 172.21.15.55:0/2301233548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3140ace60 0x7fb3140ad260 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb3140642a0 tx=0x7fb30c009840 comp rx=0 tx=0).stop 2023-10-06T03:34:37.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.518+0000 7fb31979d700 1 -- 172.21.15.55:0/2301233548 shutdown_connections 2023-10-06T03:34:37.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.518+0000 7fb31979d700 1 --2- 172.21.15.55:0/2301233548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3140ace60 0x7fb3140ad260 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:37.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.518+0000 7fb31979d700 1 --2- 172.21.15.55:0/2301233548 >> v2:172.21.15.163:3300/0 conn(0x7fb3140b0760 0x7fb3140b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:37.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.518+0000 7fb31979d700 1 -- 172.21.15.55:0/2301233548 >> 172.21.15.55:0/2301233548 conn(0x7fb314068e20 msgr2=0x7fb3140cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:37.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.518+0000 7fb31979d700 1 -- 172.21.15.55:0/2301233548 shutdown_connections 2023-10-06T03:34:37.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.518+0000 7fb31979d700 1 -- 172.21.15.55:0/2301233548 wait complete. 2023-10-06T03:34:37.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.520+0000 7fb31979d700 1 Processor -- start 2023-10-06T03:34:37.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.520+0000 7fb31979d700 1 -- start start 2023-10-06T03:34:37.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.521+0000 7fb31979d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3140ace60 0x7fb3140cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:37.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.521+0000 7fb31979d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3140b0760 0x7fb31415c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:37.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.521+0000 7fb31979d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb314156350 0x7fb314142be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:37.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.521+0000 7fb31979d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3140c94d0 con 0x7fb3140ace60 2023-10-06T03:34:37.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.521+0000 7fb31979d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31415ce00 con 0x7fb3140b0760 2023-10-06T03:34:37.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.521+0000 7fb31979d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31415d0a0 con 0x7fb314156350 2023-10-06T03:34:37.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.521+0000 7fb313fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3140ace60 0x7fb3140cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:37.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.521+0000 7fb313fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3140ace60 0x7fb3140cbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44134/0 (socket says 172.21.15.55:44134) 2023-10-06T03:34:37.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.521+0000 7fb313fff700 1 -- 172.21.15.55:0/4233439351 learned_addr learned my addr 172.21.15.55:0/4233439351 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:37.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.521+0000 7fb318f9c700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb314156350 0x7fb314142be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:37.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.521+0000 7fb3137fe700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3140b0760 0x7fb31415c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:37.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.522+0000 7fb313fff700 1 -- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb314156350 msgr2=0x7fb314142be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:37.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.522+0000 7fb313fff700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb314156350 0x7fb314142be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:37.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.522+0000 7fb313fff700 1 -- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3140b0760 msgr2=0x7fb31415c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:37.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.522+0000 7fb313fff700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3140b0760 0x7fb31415c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:37.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.522+0000 7fb313fff700 1 -- 172.21.15.55:0/4233439351 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30c009ae0 con 0x7fb3140ace60 2023-10-06T03:34:37.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.522+0000 7fb318f9c700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb314156350 0x7fb314142be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:37.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.522+0000 7fb313fff700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3140ace60 0x7fb3140cbb00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb30400c590 tx=0x7fb30400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:37.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.522+0000 7fb2fbfff700 1 -- 172.21.15.55:0/4233439351 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304011590 con 0x7fb3140ace60 2023-10-06T03:34:37.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.522+0000 7fb2fbfff700 1 -- 172.21.15.55:0/4233439351 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb304011710 con 0x7fb3140ace60 2023-10-06T03:34:37.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.522+0000 7fb2fbfff700 1 -- 172.21.15.55:0/4233439351 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304011ad0 con 0x7fb3140ace60 2023-10-06T03:34:37.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.522+0000 7fb31979d700 1 -- 172.21.15.55:0/4233439351 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31415d380 con 0x7fb3140ace60 2023-10-06T03:34:37.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.522+0000 7fb31979d700 1 -- 172.21.15.55:0/4233439351 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb314143120 con 0x7fb3140ace60 2023-10-06T03:34:37.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.523+0000 7fb31979d700 1 -- 172.21.15.55:0/4233439351 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31404c3c0 con 0x7fb3140ace60 2023-10-06T03:34:37.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.524+0000 7fb2fbfff700 1 -- 172.21.15.55:0/4233439351 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7fb30401b3b0 con 0x7fb3140ace60 2023-10-06T03:34:37.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.525+0000 7fb2fbfff700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb300067130 0x7fb3000695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:37.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.525+0000 7fb2fbfff700 1 -- 172.21.15.55:0/4233439351 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fb304084100 con 0x7fb3140ace60 2023-10-06T03:34:37.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.525+0000 7fb3137fe700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb300067130 0x7fb3000695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:37.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.526+0000 7fb3137fe700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb300067130 0x7fb3000695e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb31415bdf0 tx=0x7fb308008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:37.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.529+0000 7fb2fbfff700 1 -- 172.21.15.55:0/4233439351 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb304050df0 con 0x7fb3140ace60 2023-10-06T03:34:37.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.766+0000 7fb31979d700 1 -- 172.21.15.55:0/4233439351 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3140c9610 con 0x7fb300067130 2023-10-06T03:34:37.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.861+0000 7fb2fbfff700 1 -- 172.21.15.55:0/4233439351 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb30800b900 con 0x7fb300067130 2023-10-06T03:34:37.862 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path /snap_test_dir1/dir5 2023-10-06T03:34:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.863+0000 7fb31979d700 1 -- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb300067130 msgr2=0x7fb3000695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.863+0000 7fb31979d700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb300067130 0x7fb3000695e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb31415bdf0 tx=0x7fb308008040 comp rx=0 tx=0).stop 2023-10-06T03:34:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.863+0000 7fb31979d700 1 -- 172.21.15.55:0/4233439351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3140ace60 msgr2=0x7fb3140cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.863+0000 7fb31979d700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3140ace60 0x7fb3140cbb00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb30400c590 tx=0x7fb30400c8a0 comp rx=0 tx=0).stop 2023-10-06T03:34:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.863+0000 7fb31979d700 1 -- 172.21.15.55:0/4233439351 shutdown_connections 2023-10-06T03:34:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.863+0000 7fb31979d700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb300067130 0x7fb3000695e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.863+0000 7fb31979d700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb314156350 0x7fb314142be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.863+0000 7fb31979d700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3140ace60 0x7fb3140cbb00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:37.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.863+0000 7fb31979d700 1 --2- 172.21.15.55:0/4233439351 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3140b0760 0x7fb31415c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.863+0000 7fb31979d700 1 -- 172.21.15.55:0/4233439351 >> 172.21.15.55:0/4233439351 conn(0x7fb314068e20 msgr2=0x7fb314156720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.863+0000 7fb31979d700 1 -- 172.21.15.55:0/4233439351 shutdown_connections 2023-10-06T03:34:37.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:37.863+0000 7fb31979d700 1 -- 172.21.15.55:0/4233439351 wait complete. 2023-10-06T03:34:37.882 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-10-06T03:34:37.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:34:37.882 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-10-06T03:34:37.922 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir6 1M 2023-10-06T03:34:38.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.151+0000 7fc69559d700 1 Processor -- start 2023-10-06T03:34:38.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.152+0000 7fc69559d700 1 -- start start 2023-10-06T03:34:38.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.153+0000 7fc69559d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc6900b03c0 0x7fc6900cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:38.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.153+0000 7fc69559d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6900cfa00 con 0x7fc6901591e0 2023-10-06T03:34:38.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.153+0000 7fc69559d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6900cb580 con 0x7fc6900b0790 2023-10-06T03:34:38.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.153+0000 7fc69559d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6900cf160 con 0x7fc6900b03c0 2023-10-06T03:34:38.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.153+0000 7fc68ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc6900b03c0 0x7fc6900cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:38.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.153+0000 7fc694d9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc6901591e0 0x7fc690159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35648/0 (socket says 172.21.15.55:35648) 2023-10-06T03:34:38.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.153+0000 7fc694d9c700 1 -- 172.21.15.55:0/2968322897 learned_addr learned my addr 172.21.15.55:0/2968322897 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:38.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.154+0000 7fc68ffff700 1 -- 172.21.15.55:0/2968322897 >> v1:172.21.15.163:6789/0 conn(0x7fc6900b0790 legacy=0x7fc6900b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:38.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.154+0000 7fc68ffff700 1 -- 172.21.15.55:0/2968322897 >> v1:172.21.15.55:6789/0 conn(0x7fc6901591e0 legacy=0x7fc690159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:38.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.154+0000 7fc68ffff700 1 -- 172.21.15.55:0/2968322897 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6900cf440 con 0x7fc6900b03c0 2023-10-06T03:34:38.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.154+0000 7fc68ffff700 1 --2- 172.21.15.55:0/2968322897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6900b03c0 0x7fc6900cea70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc680009910 tx=0x7fc6800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a68395f7f7c9f9ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:38.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.154+0000 7fc68effd700 1 -- 172.21.15.55:0/2968322897 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68000b820 con 0x7fc6900b03c0 2023-10-06T03:34:38.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.154+0000 7fc68effd700 1 -- 172.21.15.55:0/2968322897 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68000b980 con 0x7fc6900b03c0 2023-10-06T03:34:38.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.154+0000 7fc68effd700 1 -- 172.21.15.55:0/2968322897 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68000bd20 con 0x7fc6900b03c0 2023-10-06T03:34:38.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.155+0000 7fc69559d700 1 -- 172.21.15.55:0/2968322897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6900b03c0 msgr2=0x7fc6900cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:38.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.155+0000 7fc69559d700 1 --2- 172.21.15.55:0/2968322897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6900b03c0 0x7fc6900cea70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc680009910 tx=0x7fc6800091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:38.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.155+0000 7fc69559d700 1 -- 172.21.15.55:0/2968322897 shutdown_connections 2023-10-06T03:34:38.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.155+0000 7fc69559d700 1 --2- 172.21.15.55:0/2968322897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6900b03c0 0x7fc6900cea70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:38.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.155+0000 7fc69559d700 1 -- 172.21.15.55:0/2968322897 >> 172.21.15.55:0/2968322897 conn(0x7fc690063230 msgr2=0x7fc690063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:38.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.155+0000 7fc69559d700 1 -- 172.21.15.55:0/2968322897 shutdown_connections 2023-10-06T03:34:38.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.155+0000 7fc69559d700 1 -- 172.21.15.55:0/2968322897 wait complete. 2023-10-06T03:34:38.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.156+0000 7fc69559d700 1 Processor -- start 2023-10-06T03:34:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.156+0000 7fc69559d700 1 -- start start 2023-10-06T03:34:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.157+0000 7fc69559d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6900b03c0 0x7fc69014a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.157+0000 7fc69559d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6900b0790 0x7fc6900c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.157+0000 7fc69559d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6901591e0 0x7fc6900c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.157+0000 7fc69559d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6900c7c20 con 0x7fc6900b0790 2023-10-06T03:34:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.157+0000 7fc69559d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6900c7640 con 0x7fc6901591e0 2023-10-06T03:34:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.157+0000 7fc69559d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6900c4d50 con 0x7fc6900b03c0 2023-10-06T03:34:38.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.158+0000 7fc68f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6900b0790 0x7fc6900c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:38.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.158+0000 7fc68f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6900b0790 0x7fc6900c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44162/0 (socket says 172.21.15.55:44162) 2023-10-06T03:34:38.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.158+0000 7fc68f7fe700 1 -- 172.21.15.55:0/3505766195 learned_addr learned my addr 172.21.15.55:0/3505766195 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:38.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.158+0000 7fc694d9c700 1 --2- 172.21.15.55:0/3505766195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6901591e0 0x7fc6900c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:38.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.158+0000 7fc68ffff700 1 --2- 172.21.15.55:0/3505766195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6900b03c0 0x7fc69014a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:38.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.158+0000 7fc68f7fe700 1 -- 172.21.15.55:0/3505766195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6900b03c0 msgr2=0x7fc69014a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:38.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.158+0000 7fc68f7fe700 1 --2- 172.21.15.55:0/3505766195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6900b03c0 0x7fc69014a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:38.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.158+0000 7fc68f7fe700 1 -- 172.21.15.55:0/3505766195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6901591e0 msgr2=0x7fc6900c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:38.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.158+0000 7fc68f7fe700 1 --2- 172.21.15.55:0/3505766195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6901591e0 0x7fc6900c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:38.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.158+0000 7fc68f7fe700 1 -- 172.21.15.55:0/3505766195 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc680009420 con 0x7fc6900b0790 2023-10-06T03:34:38.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.158+0000 7fc68f7fe700 1 --2- 172.21.15.55:0/3505766195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6900b0790 0x7fc6900c99f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc68400c590 tx=0x7fc68400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:38.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.159+0000 7fc677fff700 1 -- 172.21.15.55:0/3505766195 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684011590 con 0x7fc6900b0790 2023-10-06T03:34:38.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.159+0000 7fc677fff700 1 -- 172.21.15.55:0/3505766195 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc684011710 con 0x7fc6900b0790 2023-10-06T03:34:38.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.159+0000 7fc69559d700 1 -- 172.21.15.55:0/3505766195 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6900c78b0 con 0x7fc6900b0790 2023-10-06T03:34:38.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.159+0000 7fc677fff700 1 -- 172.21.15.55:0/3505766195 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684011ad0 con 0x7fc6900b0790 2023-10-06T03:34:38.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.159+0000 7fc69559d700 1 -- 172.21.15.55:0/3505766195 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69014ade0 con 0x7fc6900b0790 2023-10-06T03:34:38.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.160+0000 7fc69559d700 1 -- 172.21.15.55:0/3505766195 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6900c7dd0 con 0x7fc6900b0790 2023-10-06T03:34:38.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.161+0000 7fc677fff700 1 -- 172.21.15.55:0/3505766195 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7fc68401b3b0 con 0x7fc6900b0790 2023-10-06T03:34:38.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.161+0000 7fc677fff700 1 --2- 172.21.15.55:0/3505766195 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc67c067130 0x7fc67c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:38.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.162+0000 7fc677fff700 1 -- 172.21.15.55:0/3505766195 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fc684083fc0 con 0x7fc6900b0790 2023-10-06T03:34:38.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.162+0000 7fc68ffff700 1 --2- 172.21.15.55:0/3505766195 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc67c067130 0x7fc67c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:38.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.163+0000 7fc68ffff700 1 --2- 172.21.15.55:0/3505766195 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc67c067130 0x7fc67c0695e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc6800093f0 tx=0x7fc680017000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:38.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.166+0000 7fc677fff700 1 -- 172.21.15.55:0/3505766195 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6900c7dd0 con 0x7fc6900b0790 2023-10-06T03:34:38.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.394+0000 7fc69559d700 1 -- 172.21.15.55:0/3505766195 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc69004c3c0 con 0x7fc67c067130 2023-10-06T03:34:38.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.490+0000 7fc677fff700 1 -- 172.21.15.55:0/3505766195 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc69004c3c0 con 0x7fc67c067130 2023-10-06T03:34:38.525 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path /snap_test_dir1/dir6 2023-10-06T03:34:38.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.492+0000 7fc69559d700 1 -- 172.21.15.55:0/3505766195 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc67c067130 msgr2=0x7fc67c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:38.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.492+0000 7fc69559d700 1 --2- 172.21.15.55:0/3505766195 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc67c067130 0x7fc67c0695e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc6800093f0 tx=0x7fc680017000 comp rx=0 tx=0).stop 2023-10-06T03:34:38.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.492+0000 7fc69559d700 1 -- 172.21.15.55:0/3505766195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6900b0790 msgr2=0x7fc6900c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:38.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.492+0000 7fc69559d700 1 --2- 172.21.15.55:0/3505766195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6900b0790 0x7fc6900c99f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc68400c590 tx=0x7fc68400c8a0 comp rx=0 tx=0).stop 2023-10-06T03:34:38.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.492+0000 7fc69559d700 1 -- 172.21.15.55:0/3505766195 shutdown_connections 2023-10-06T03:34:38.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.492+0000 7fc69559d700 1 --2- 172.21.15.55:0/3505766195 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6901591e0 0x7fc6900c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:38.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.492+0000 7fc69559d700 1 --2- 172.21.15.55:0/3505766195 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc67c067130 0x7fc67c0695e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:38.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.492+0000 7fc69559d700 1 --2- 172.21.15.55:0/3505766195 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6900b03c0 0x7fc69014a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:38.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.492+0000 7fc69559d700 1 --2- 172.21.15.55:0/3505766195 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6900b0790 0x7fc6900c99f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:38.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.492+0000 7fc69559d700 1 -- 172.21.15.55:0/3505766195 >> 172.21.15.55:0/3505766195 conn(0x7fc690063230 msgr2=0x7fc6901589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:38.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.493+0000 7fc69559d700 1 -- 172.21.15.55:0/3505766195 shutdown_connections 2023-10-06T03:34:38.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.493+0000 7fc69559d700 1 -- 172.21.15.55:0/3505766195 wait complete. 2023-10-06T03:34:38.528 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-10-06T03:34:38.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:34:38.528 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-10-06T03:34:38.568 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir7 1M 2023-10-06T03:34:38.801 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.800+0000 7f987d79d700 1 Processor -- start 2023-10-06T03:34:38.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.800+0000 7f987d79d700 1 -- start start 2023-10-06T03:34:38.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.801+0000 7f987d79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98780b03c0 0x7f98780cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:38.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.801+0000 7f987d79d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98781591e0 0x7f9878159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:38.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.801+0000 7f987d79d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98780cfa00 con 0x7f98780b0790 2023-10-06T03:34:38.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.801+0000 7f987d79d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98780cb580 con 0x7f98781591e0 2023-10-06T03:34:38.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.801+0000 7f987d79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98780cf160 con 0x7f98780b03c0 2023-10-06T03:34:38.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.802+0000 7f987cf9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98781591e0 0x7f9878159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:38.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.802+0000 7f987cf9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98781591e0 0x7f9878159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44172/0 (socket says 172.21.15.55:44172) 2023-10-06T03:34:38.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.802+0000 7f987cf9c700 1 -- 172.21.15.55:0/1390823589 learned_addr learned my addr 172.21.15.55:0/1390823589 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:38.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.802+0000 7f9877fff700 1 --2- 172.21.15.55:0/1390823589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98780b03c0 0x7f98780cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:38.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.802+0000 7f9876ffd700 1 -- 172.21.15.55:0/1390823589 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1072278796 0 0) 0x7f98780cfa00 con 0x7f98780b0790 2023-10-06T03:34:38.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.802+0000 7f987cf9c700 1 -- 172.21.15.55:0/1390823589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98780b03c0 msgr2=0x7f98780cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:38.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.802+0000 7f987cf9c700 1 --2- 172.21.15.55:0/1390823589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98780b03c0 0x7f98780cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:38.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.802+0000 7f987cf9c700 1 -- 172.21.15.55:0/1390823589 >> v1:172.21.15.163:6789/0 conn(0x7f98780b0790 legacy=0x7f98780b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:38.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.802+0000 7f987cf9c700 1 -- 172.21.15.55:0/1390823589 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98780cf440 con 0x7f98781591e0 2023-10-06T03:34:38.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.802+0000 7f987cf9c700 1 --2- 172.21.15.55:0/1390823589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98781591e0 0x7f9878159620 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9870009f20 tx=0x7f9870009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2768aed2f5b92db7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:38.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.803+0000 7f9876ffd700 1 -- 172.21.15.55:0/1390823589 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987000e070 con 0x7f98781591e0 2023-10-06T03:34:38.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.803+0000 7f9876ffd700 1 -- 172.21.15.55:0/1390823589 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987000b9d0 con 0x7f98781591e0 2023-10-06T03:34:38.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.803+0000 7f9876ffd700 1 -- 172.21.15.55:0/1390823589 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98700092e0 con 0x7f98781591e0 2023-10-06T03:34:38.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.803+0000 7f987d79d700 1 -- 172.21.15.55:0/1390823589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98781591e0 msgr2=0x7f9878159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:38.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.803+0000 7f987d79d700 1 --2- 172.21.15.55:0/1390823589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98781591e0 0x7f9878159620 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9870009f20 tx=0x7f9870009840 comp rx=0 tx=0).stop 2023-10-06T03:34:38.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.803+0000 7f987d79d700 1 -- 172.21.15.55:0/1390823589 shutdown_connections 2023-10-06T03:34:38.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.803+0000 7f987d79d700 1 --2- 172.21.15.55:0/1390823589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98781591e0 0x7f9878159620 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:38.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.803+0000 7f987d79d700 1 --2- 172.21.15.55:0/1390823589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98780b03c0 0x7f98780cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:38.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.803+0000 7f987d79d700 1 -- 172.21.15.55:0/1390823589 >> 172.21.15.55:0/1390823589 conn(0x7f9878063230 msgr2=0x7f9878063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:38.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.803+0000 7f987d79d700 1 -- 172.21.15.55:0/1390823589 shutdown_connections 2023-10-06T03:34:38.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.803+0000 7f987d79d700 1 -- 172.21.15.55:0/1390823589 wait complete. 2023-10-06T03:34:38.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.805+0000 7f987d79d700 1 Processor -- start 2023-10-06T03:34:38.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.805+0000 7f987d79d700 1 -- start start 2023-10-06T03:34:38.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.806+0000 7f987d79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98780b03c0 0x7f9878148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:38.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.806+0000 7f987d79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98780b0790 0x7f98780b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:38.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.806+0000 7f987d79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98781591e0 0x7f98780b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:38.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.806+0000 7f987d79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987816df00 con 0x7f98780b0790 2023-10-06T03:34:38.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.806+0000 7f987d79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98781699c0 con 0x7f98781591e0 2023-10-06T03:34:38.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.806+0000 7f987d79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987816a220 con 0x7f98780b03c0 2023-10-06T03:34:38.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.806+0000 7f98777fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98780b0790 0x7f98780b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:38.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.806+0000 7f98777fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98780b0790 0x7f98780b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44182/0 (socket says 172.21.15.55:44182) 2023-10-06T03:34:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.806+0000 7f98777fe700 1 -- 172.21.15.55:0/1131174670 learned_addr learned my addr 172.21.15.55:0/1131174670 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.806+0000 7f987cf9c700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98781591e0 0x7f98780b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.806+0000 7f9877fff700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98780b03c0 0x7f9878148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.807+0000 7f98777fe700 1 -- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98780b03c0 msgr2=0x7f9878148940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.807+0000 7f98777fe700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98780b03c0 0x7f9878148940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.807+0000 7f98777fe700 1 -- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98781591e0 msgr2=0x7f98780b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.807+0000 7f98777fe700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98781591e0 0x7f98780b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:38.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.807+0000 7f98777fe700 1 -- 172.21.15.55:0/1131174670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9870009ae0 con 0x7f98780b0790 2023-10-06T03:34:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.807+0000 7f987cf9c700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98781591e0 0x7f98780b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.807+0000 7f98777fe700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98780b0790 0x7f98780b4910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c810 tx=0x7f986c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.807+0000 7f985ffff700 1 -- 172.21.15.55:0/1131174670 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c012700 con 0x7f98780b0790 2023-10-06T03:34:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.807+0000 7f985ffff700 1 -- 172.21.15.55:0/1131174670 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c0128a0 con 0x7f98780b0790 2023-10-06T03:34:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.807+0000 7f987d79d700 1 -- 172.21.15.55:0/1131174670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f987816dc20 con 0x7f98780b0790 2023-10-06T03:34:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.807+0000 7f985ffff700 1 -- 172.21.15.55:0/1131174670 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c012c60 con 0x7f98780b0790 2023-10-06T03:34:38.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.807+0000 7f987d79d700 1 -- 172.21.15.55:0/1131174670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9878169eb0 con 0x7f98780b0790 2023-10-06T03:34:38.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.808+0000 7f987d79d700 1 -- 172.21.15.55:0/1131174670 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987804c3c0 con 0x7f98780b0790 2023-10-06T03:34:38.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.809+0000 7f985ffff700 1 -- 172.21.15.55:0/1131174670 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f986c01b400 con 0x7f98780b0790 2023-10-06T03:34:38.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.810+0000 7f985ffff700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9864067130 0x7f98640695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:38.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.810+0000 7f985ffff700 1 -- 172.21.15.55:0/1131174670 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f986c0847d0 con 0x7f98780b0790 2023-10-06T03:34:38.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.810+0000 7f9877fff700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9864067130 0x7f98640695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:38.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.811+0000 7f9877fff700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9864067130 0x7f98640695e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9868009ca0 tx=0x7f9868008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:38.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:38.814+0000 7f985ffff700 1 -- 172.21.15.55:0/1131174670 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f986c0514c0 con 0x7f98780b0790 2023-10-06T03:34:39.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.043+0000 7f987d79d700 1 -- 172.21.15.55:0/1131174670 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f987816a360 con 0x7f9864067130 2023-10-06T03:34:39.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.113+0000 7f985ffff700 1 -- 172.21.15.55:0/1131174670 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f987816a360 con 0x7f9864067130 2023-10-06T03:34:39.114 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path /snap_test_dir1/dir7 2023-10-06T03:34:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.115+0000 7f987d79d700 1 -- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9864067130 msgr2=0x7f98640695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.115+0000 7f987d79d700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9864067130 0x7f98640695e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9868009ca0 tx=0x7f9868008040 comp rx=0 tx=0).stop 2023-10-06T03:34:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.115+0000 7f987d79d700 1 -- 172.21.15.55:0/1131174670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98780b0790 msgr2=0x7f98780b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.115+0000 7f987d79d700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98780b0790 0x7f98780b4910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c810 tx=0x7f986c007df0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.115+0000 7f987d79d700 1 -- 172.21.15.55:0/1131174670 shutdown_connections 2023-10-06T03:34:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.115+0000 7f987d79d700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98781591e0 0x7f98780b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.115+0000 7f987d79d700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9864067130 0x7f98640695e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.115+0000 7f987d79d700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98780b03c0 0x7f9878148940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.115+0000 7f987d79d700 1 --2- 172.21.15.55:0/1131174670 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98780b0790 0x7f98780b4910 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.115+0000 7f987d79d700 1 -- 172.21.15.55:0/1131174670 >> 172.21.15.55:0/1131174670 conn(0x7f9878063230 msgr2=0x7f98780699c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:39.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.115+0000 7f987d79d700 1 -- 172.21.15.55:0/1131174670 shutdown_connections 2023-10-06T03:34:39.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.115+0000 7f987d79d700 1 -- 172.21.15.55:0/1131174670 wait complete. 2023-10-06T03:34:39.134 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-10-06T03:34:39.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:34:39.135 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-10-06T03:34:39.175 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir8 1M 2023-10-06T03:34:39.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.406+0000 7f90f699d700 1 Processor -- start 2023-10-06T03:34:39.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.406+0000 7f90f699d700 1 -- start start 2023-10-06T03:34:39.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.407+0000 7f90f699d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90f00b0790 0x7f90f00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:39.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.407+0000 7f90f699d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f90f01591e0 0x7f90f0159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:39.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.407+0000 7f90f699d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f00cfa00 con 0x7f90f00b03c0 2023-10-06T03:34:39.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.407+0000 7f90f699d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f00cb580 con 0x7f90f01591e0 2023-10-06T03:34:39.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.407+0000 7f90f699d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f00cf160 con 0x7f90f00b0790 2023-10-06T03:34:39.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.408+0000 7f90f519a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90f00b0790 0x7f90f00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:39.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.408+0000 7f90f599b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f90f00b03c0 0x7f90f00b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:43000/0 (socket says 172.21.15.55:43000) 2023-10-06T03:34:39.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.408+0000 7f90f599b700 1 -- 172.21.15.55:0/3368797736 learned_addr learned my addr 172.21.15.55:0/3368797736 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:39.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.408+0000 7f90f619c700 1 --2- 172.21.15.55:0/3368797736 >> v2:172.21.15.163:3300/0 conn(0x7f90f01591e0 0x7f90f0159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:39.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.408+0000 7f90f4999700 1 -- 172.21.15.55:0/3368797736 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69064676 0 0) 0x7f90f00cfa00 con 0x7f90f00b03c0 2023-10-06T03:34:39.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.408+0000 7f90f4999700 1 -- 172.21.15.55:0/3368797736 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90dc003570 con 0x7f90f00b03c0 2023-10-06T03:34:39.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.408+0000 7f90f619c700 1 -- 172.21.15.55:0/3368797736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90f00b0790 msgr2=0x7f90f00cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:39.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.408+0000 7f90f619c700 1 --2- 172.21.15.55:0/3368797736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90f00b0790 0x7f90f00cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.408+0000 7f90f619c700 1 -- 172.21.15.55:0/3368797736 >> v1:172.21.15.163:6789/0 conn(0x7f90f00b03c0 legacy=0x7f90f00b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:39.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.408+0000 7f90f619c700 1 -- 172.21.15.55:0/3368797736 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f00cf440 con 0x7f90f01591e0 2023-10-06T03:34:39.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.408+0000 7f90f519a700 1 --2- 172.21.15.55:0/3368797736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90f00b0790 0x7f90f00cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:34:39.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.409+0000 7f90f619c700 1 --2- 172.21.15.55:0/3368797736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f01591e0 0x7f90f0159620 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009e60 tx=0x7f90e8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=178f3f311cbeb0fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:39.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.409+0000 7f90f4999700 1 -- 172.21.15.55:0/3368797736 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e800e070 con 0x7f90f01591e0 2023-10-06T03:34:39.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.409+0000 7f90f4999700 1 -- 172.21.15.55:0/3368797736 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e800bab0 con 0x7f90f01591e0 2023-10-06T03:34:39.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.409+0000 7f90f4999700 1 -- 172.21.15.55:0/3368797736 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e800be70 con 0x7f90f01591e0 2023-10-06T03:34:39.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.409+0000 7f90f699d700 1 -- 172.21.15.55:0/3368797736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f01591e0 msgr2=0x7f90f0159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:39.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.409+0000 7f90f699d700 1 --2- 172.21.15.55:0/3368797736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f01591e0 0x7f90f0159620 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009e60 tx=0x7f90e8009780 comp rx=0 tx=0).stop 2023-10-06T03:34:39.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.410+0000 7f90f699d700 1 -- 172.21.15.55:0/3368797736 shutdown_connections 2023-10-06T03:34:39.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.410+0000 7f90f699d700 1 --2- 172.21.15.55:0/3368797736 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f01591e0 0x7f90f0159620 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.410+0000 7f90f699d700 1 --2- 172.21.15.55:0/3368797736 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90f00b0790 0x7f90f00cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.410+0000 7f90f699d700 1 -- 172.21.15.55:0/3368797736 >> 172.21.15.55:0/3368797736 conn(0x7f90f0063230 msgr2=0x7f90f0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:39.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.410+0000 7f90f699d700 1 -- 172.21.15.55:0/3368797736 shutdown_connections 2023-10-06T03:34:39.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.410+0000 7f90f699d700 1 -- 172.21.15.55:0/3368797736 wait complete. 2023-10-06T03:34:39.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.411+0000 7f90f699d700 1 Processor -- start 2023-10-06T03:34:39.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.411+0000 7f90f699d700 1 -- start start 2023-10-06T03:34:39.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.412+0000 7f90f699d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90f00b03c0 0x7f90f0157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:39.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.412+0000 7f90f699d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00b0790 0x7f90f015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:39.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.412+0000 7f90f699d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f01591e0 0x7f90f015a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:39.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.412+0000 7f90f699d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f015c4b0 con 0x7f90f00b0790 2023-10-06T03:34:39.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.412+0000 7f90f699d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f015cd10 con 0x7f90f01591e0 2023-10-06T03:34:39.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.412+0000 7f90f699d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f0157960 con 0x7f90f00b03c0 2023-10-06T03:34:39.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.412+0000 7f90f519a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00b0790 0x7f90f015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:39.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.413+0000 7f90f599b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90f00b03c0 0x7f90f0157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:39.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.413+0000 7f90f519a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00b0790 0x7f90f015d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44192/0 (socket says 172.21.15.55:44192) 2023-10-06T03:34:39.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.413+0000 7f90f519a700 1 -- 172.21.15.55:0/2596238684 learned_addr learned my addr 172.21.15.55:0/2596238684 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:39.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.413+0000 7f90f619c700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f01591e0 0x7f90f015a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:39.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.413+0000 7f90f519a700 1 -- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90f00b03c0 msgr2=0x7f90f0157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:39.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.413+0000 7f90f519a700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90f00b03c0 0x7f90f0157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.413+0000 7f90f519a700 1 -- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f01591e0 msgr2=0x7f90f015a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:39.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.413+0000 7f90f519a700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f01591e0 0x7f90f015a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.413+0000 7f90f519a700 1 -- 172.21.15.55:0/2596238684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e0009420 con 0x7f90f00b0790 2023-10-06T03:34:39.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.413+0000 7f90f599b700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90f00b03c0 0x7f90f0157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:34:39.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.413+0000 7f90f619c700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f01591e0 0x7f90f015a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:34:39.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.413+0000 7f90f519a700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00b0790 0x7f90f015d1a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f90e000ec80 tx=0x7f90e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:39.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.414+0000 7f90e5ffb700 1 -- 172.21.15.55:0/2596238684 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e000c920 con 0x7f90f00b0790 2023-10-06T03:34:39.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.414+0000 7f90e5ffb700 1 -- 172.21.15.55:0/2596238684 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e000ca80 con 0x7f90f00b0790 2023-10-06T03:34:39.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.414+0000 7f90e5ffb700 1 -- 172.21.15.55:0/2596238684 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e000ce00 con 0x7f90f00b0790 2023-10-06T03:34:39.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.414+0000 7f90f699d700 1 -- 172.21.15.55:0/2596238684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e8009a20 con 0x7f90f00b0790 2023-10-06T03:34:39.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.414+0000 7f90f699d700 1 -- 172.21.15.55:0/2596238684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f015c6c0 con 0x7f90f00b0790 2023-10-06T03:34:39.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.415+0000 7f90f699d700 1 -- 172.21.15.55:0/2596238684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f004c3c0 con 0x7f90f00b0790 2023-10-06T03:34:39.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.416+0000 7f90e5ffb700 1 -- 172.21.15.55:0/2596238684 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f90e0013680 con 0x7f90f00b0790 2023-10-06T03:34:39.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.416+0000 7f90e5ffb700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f90d4067130 0x7f90d40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:39.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.416+0000 7f90e5ffb700 1 -- 172.21.15.55:0/2596238684 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f90e00852a0 con 0x7f90f00b0790 2023-10-06T03:34:39.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.417+0000 7f90f599b700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f90d4067130 0x7f90d40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:39.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.418+0000 7f90f599b700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f90d4067130 0x7f90d40695e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f90ec007fd0 tx=0x7f90ec009040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:39.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.420+0000 7f90e5ffb700 1 -- 172.21.15.55:0/2596238684 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90e0051f90 con 0x7f90f00b0790 2023-10-06T03:34:39.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.651+0000 7f90f699d700 1 -- 172.21.15.55:0/2596238684 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f90f015ce50 con 0x7f90d4067130 2023-10-06T03:34:39.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.713+0000 7f90e5ffb700 1 -- 172.21.15.55:0/2596238684 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f90f015ce50 con 0x7f90d4067130 2023-10-06T03:34:39.715 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path /snap_test_dir1/dir8 2023-10-06T03:34:39.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.715+0000 7f90f699d700 1 -- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f90d4067130 msgr2=0x7f90d40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:39.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.715+0000 7f90f699d700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f90d4067130 0x7f90d40695e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f90ec007fd0 tx=0x7f90ec009040 comp rx=0 tx=0).stop 2023-10-06T03:34:39.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.715+0000 7f90f699d700 1 -- 172.21.15.55:0/2596238684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00b0790 msgr2=0x7f90f015d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:39.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.715+0000 7f90f699d700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00b0790 0x7f90f015d1a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f90e000ec80 tx=0x7f90e00091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.715+0000 7f90f699d700 1 -- 172.21.15.55:0/2596238684 shutdown_connections 2023-10-06T03:34:39.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.715+0000 7f90f699d700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f90f01591e0 0x7f90f015a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.715+0000 7f90f699d700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f90d4067130 0x7f90d40695e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.715+0000 7f90f699d700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f90f00b03c0 0x7f90f0157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.715+0000 7f90f699d700 1 --2- 172.21.15.55:0/2596238684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f90f00b0790 0x7f90f015d1a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:39.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.716+0000 7f90f699d700 1 -- 172.21.15.55:0/2596238684 >> 172.21.15.55:0/2596238684 conn(0x7f90f0063230 msgr2=0x7f90f00b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:39.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.716+0000 7f90f699d700 1 -- 172.21.15.55:0/2596238684 shutdown_connections 2023-10-06T03:34:39.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:39.716+0000 7f90f699d700 1 -- 172.21.15.55:0/2596238684 wait complete. 2023-10-06T03:34:39.734 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-10-06T03:34:39.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:34:39.735 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-10-06T03:34:39.773 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir9 1M 2023-10-06T03:34:40.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.004+0000 7fd9e4b9d700 1 Processor -- start 2023-10-06T03:34:40.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.004+0000 7fd9e4b9d700 1 -- start start 2023-10-06T03:34:40.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.005+0000 7fd9e4b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00cca20 0x7fd9e00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:40.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.005+0000 7fd9e4b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e00cfa00 con 0x7fd9e00b03c0 2023-10-06T03:34:40.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.005+0000 7fd9e4b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e00cb580 con 0x7fd9e00b0790 2023-10-06T03:34:40.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.005+0000 7fd9e4b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00cf160 con 0x7fd9e00cca20 2023-10-06T03:34:40.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.005+0000 7fd9df7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd9e00b03c0 0x7fd9e00b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35662/0 (socket says 172.21.15.55:35662) 2023-10-06T03:34:40.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.006+0000 7fd9df7fe700 1 -- 172.21.15.55:0/3034999107 learned_addr learned my addr 172.21.15.55:0/3034999107 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:40.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.006+0000 7fd9dffff700 1 --2- 172.21.15.55:0/3034999107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00cca20 0x7fd9e00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:40.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.006+0000 7fd9de7fc700 1 -- 172.21.15.55:0/3034999107 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723589699 0 0) 0x7fd9e00cfa00 con 0x7fd9e00b03c0 2023-10-06T03:34:40.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.006+0000 7fd9de7fc700 1 -- 172.21.15.55:0/3034999107 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9c8003570 con 0x7fd9e00b03c0 2023-10-06T03:34:40.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.006+0000 7fd9de7fc700 1 -- 172.21.15.55:0/3034999107 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2951160553 0 0) 0x7fd9c8003570 con 0x7fd9e00b03c0 2023-10-06T03:34:40.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.006+0000 7fd9dffff700 1 -- 172.21.15.55:0/3034999107 >> v1:172.21.15.163:6789/0 conn(0x7fd9e00b0790 legacy=0x7fd9e01591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:40.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.006+0000 7fd9dffff700 1 -- 172.21.15.55:0/3034999107 >> v1:172.21.15.55:6789/0 conn(0x7fd9e00b03c0 legacy=0x7fd9e00b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:40.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.006+0000 7fd9dffff700 1 -- 172.21.15.55:0/3034999107 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00cf440 con 0x7fd9e00cca20 2023-10-06T03:34:40.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.007+0000 7fd9dffff700 1 --2- 172.21.15.55:0/3034999107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00cca20 0x7fd9e00cea70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009ec0 tx=0x7fd9d8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=810fcaeb636938db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:40.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.007+0000 7fd9de7fc700 1 -- 172.21.15.55:0/3034999107 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d800e070 con 0x7fd9e00cca20 2023-10-06T03:34:40.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.007+0000 7fd9de7fc700 1 -- 172.21.15.55:0/3034999107 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d800b920 con 0x7fd9e00cca20 2023-10-06T03:34:40.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.007+0000 7fd9de7fc700 1 -- 172.21.15.55:0/3034999107 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0001470 con 0x7fd9e00cca20 2023-10-06T03:34:40.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.008+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3034999107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00cca20 msgr2=0x7fd9e00cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:40.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.008+0000 7fd9e4b9d700 1 --2- 172.21.15.55:0/3034999107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00cca20 0x7fd9e00cea70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009ec0 tx=0x7fd9d8009780 comp rx=0 tx=0).stop 2023-10-06T03:34:40.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.008+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3034999107 shutdown_connections 2023-10-06T03:34:40.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.008+0000 7fd9e4b9d700 1 --2- 172.21.15.55:0/3034999107 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00cca20 0x7fd9e00cea70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.008+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3034999107 >> 172.21.15.55:0/3034999107 conn(0x7fd9e0063230 msgr2=0x7fd9e0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:40.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.008+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3034999107 shutdown_connections 2023-10-06T03:34:40.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.008+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3034999107 wait complete. 2023-10-06T03:34:40.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.009+0000 7fd9e4b9d700 1 Processor -- start 2023-10-06T03:34:40.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.009+0000 7fd9e4b9d700 1 -- start start 2023-10-06T03:34:40.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.010+0000 7fd9e4b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00b03c0 0x7fd9e00c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:40.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.010+0000 7fd9e4b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e00b0790 0x7fd9e00c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:40.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.010+0000 7fd9e4b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00cca20 0x7fd9e01699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:40.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.010+0000 7fd9e4b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00c75a0 con 0x7fd9e00b03c0 2023-10-06T03:34:40.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.010+0000 7fd9e4b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00c77f0 con 0x7fd9e00b0790 2023-10-06T03:34:40.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.010+0000 7fd9e4b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00c7ad0 con 0x7fd9e00cca20 2023-10-06T03:34:40.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.010+0000 7fd9df7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00b03c0 0x7fd9e00c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:40.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.010+0000 7fd9df7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00b03c0 0x7fd9e00c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44196/0 (socket says 172.21.15.55:44196) 2023-10-06T03:34:40.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.010+0000 7fd9df7fe700 1 -- 172.21.15.55:0/3377435991 learned_addr learned my addr 172.21.15.55:0/3377435991 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:40.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.010+0000 7fd9deffd700 1 --2- 172.21.15.55:0/3377435991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e00b0790 0x7fd9e00c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:40.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.011+0000 7fd9dffff700 1 --2- 172.21.15.55:0/3377435991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00cca20 0x7fd9e01699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:40.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.011+0000 7fd9df7fe700 1 -- 172.21.15.55:0/3377435991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00cca20 msgr2=0x7fd9e01699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:40.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.011+0000 7fd9df7fe700 1 --2- 172.21.15.55:0/3377435991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00cca20 0x7fd9e01699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.011+0000 7fd9df7fe700 1 -- 172.21.15.55:0/3377435991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e00b0790 msgr2=0x7fd9e00c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:40.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.011+0000 7fd9df7fe700 1 --2- 172.21.15.55:0/3377435991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e00b0790 0x7fd9e00c65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.011+0000 7fd9df7fe700 1 -- 172.21.15.55:0/3377435991 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d8009a20 con 0x7fd9e00b03c0 2023-10-06T03:34:40.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.011+0000 7fd9df7fe700 1 --2- 172.21.15.55:0/3377435991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00b03c0 0x7fd9e00c8890 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c7f0 tx=0x7fd9d00019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:40.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.011+0000 7fd9c77fe700 1 -- 172.21.15.55:0/3377435991 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0007360 con 0x7fd9e00b03c0 2023-10-06T03:34:40.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.011+0000 7fd9c77fe700 1 -- 172.21.15.55:0/3377435991 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d00074e0 con 0x7fd9e00b03c0 2023-10-06T03:34:40.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.011+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3377435991 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e00c7db0 con 0x7fd9e00b03c0 2023-10-06T03:34:40.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.011+0000 7fd9c77fe700 1 -- 172.21.15.55:0/3377435991 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d0004830 con 0x7fd9e00b03c0 2023-10-06T03:34:40.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.011+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3377435991 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e00c8370 con 0x7fd9e00b03c0 2023-10-06T03:34:40.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.012+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3377435991 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e00c8590 con 0x7fd9e00b03c0 2023-10-06T03:34:40.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.013+0000 7fd9c77fe700 1 -- 172.21.15.55:0/3377435991 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7fd9d0004c20 con 0x7fd9e00b03c0 2023-10-06T03:34:40.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.014+0000 7fd9c77fe700 1 --2- 172.21.15.55:0/3377435991 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd9cc067130 0x7fd9cc0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:40.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.014+0000 7fd9c77fe700 1 -- 172.21.15.55:0/3377435991 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fd9d0083cb0 con 0x7fd9e00b03c0 2023-10-06T03:34:40.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.015+0000 7fd9deffd700 1 --2- 172.21.15.55:0/3377435991 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd9cc067130 0x7fd9cc0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:40.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.015+0000 7fd9deffd700 1 --2- 172.21.15.55:0/3377435991 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd9cc067130 0x7fd9cc0695e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00b35f0 tx=0x7fd9d4007ea0 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:40.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.018+0000 7fd9c77fe700 1 -- 172.21.15.55:0/3377435991 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9d0050a20 con 0x7fd9e00b03c0 2023-10-06T03:34:40.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.245+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3377435991 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9e004c3c0 con 0x7fd9cc067130 2023-10-06T03:34:40.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.326+0000 7fd9c77fe700 1 -- 172.21.15.55:0/3377435991 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fd9e004c3c0 con 0x7fd9cc067130 2023-10-06T03:34:40.328 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path /snap_test_dir1/dir9 2023-10-06T03:34:40.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.328+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3377435991 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd9cc067130 msgr2=0x7fd9cc0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:40.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.328+0000 7fd9e4b9d700 1 --2- 172.21.15.55:0/3377435991 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd9cc067130 0x7fd9cc0695e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00b35f0 tx=0x7fd9d4007ea0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.328+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3377435991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00b03c0 msgr2=0x7fd9e00c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:40.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.328+0000 7fd9e4b9d700 1 --2- 172.21.15.55:0/3377435991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00b03c0 0x7fd9e00c8890 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd9d000c7f0 tx=0x7fd9d00019b0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.328+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3377435991 shutdown_connections 2023-10-06T03:34:40.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.328+0000 7fd9e4b9d700 1 --2- 172.21.15.55:0/3377435991 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd9cc067130 0x7fd9cc0695e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.328+0000 7fd9e4b9d700 1 --2- 172.21.15.55:0/3377435991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00cca20 0x7fd9e01699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.328+0000 7fd9e4b9d700 1 --2- 172.21.15.55:0/3377435991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00b03c0 0x7fd9e00c8890 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.328+0000 7fd9e4b9d700 1 --2- 172.21.15.55:0/3377435991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e00b0790 0x7fd9e00c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.328+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3377435991 >> 172.21.15.55:0/3377435991 conn(0x7fd9e0063230 msgr2=0x7fd9e00b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:40.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.328+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3377435991 shutdown_connections 2023-10-06T03:34:40.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.328+0000 7fd9e4b9d700 1 -- 172.21.15.55:0/3377435991 wait complete. 2023-10-06T03:34:40.347 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-10-06T03:34:40.347 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir0 1M 2023-10-06T03:34:40.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.538+0000 7f72d6f9d700 1 Processor -- start 2023-10-06T03:34:40.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.538+0000 7f72d6f9d700 1 -- start start 2023-10-06T03:34:40.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.539+0000 7f72d6f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d00b0790 0x7f72d00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:40.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.539+0000 7f72d6f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f72d01591e0 0x7f72d0159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:40.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.539+0000 7f72d6f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d00cfa00 con 0x7f72d00b03c0 2023-10-06T03:34:40.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.539+0000 7f72d6f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d00cb580 con 0x7f72d01591e0 2023-10-06T03:34:40.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.539+0000 7f72d6f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00cf160 con 0x7f72d00b0790 2023-10-06T03:34:40.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.539+0000 7f72d679c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f72d01591e0 0x7f72d0159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:40.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.540+0000 7f72d679c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f72d01591e0 0x7f72d0159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44212/0 (socket says 172.21.15.55:44212) 2023-10-06T03:34:40.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.540+0000 7f72d679c700 1 -- 172.21.15.55:0/3327789722 learned_addr learned my addr 172.21.15.55:0/3327789722 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:40.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.540+0000 7f72d579a700 1 --2- 172.21.15.55:0/3327789722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d00b0790 0x7f72d00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:40.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.540+0000 7f72d679c700 1 -- 172.21.15.55:0/3327789722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d00b0790 msgr2=0x7f72d00cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:40.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.540+0000 7f72d679c700 1 --2- 172.21.15.55:0/3327789722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d00b0790 0x7f72d00cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.540+0000 7f72d679c700 1 -- 172.21.15.55:0/3327789722 >> v1:172.21.15.163:6789/0 conn(0x7f72d00b03c0 legacy=0x7f72d00b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:40.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.540+0000 7f72d679c700 1 -- 172.21.15.55:0/3327789722 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d00cf440 con 0x7f72d01591e0 2023-10-06T03:34:40.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.540+0000 7f72d579a700 1 --2- 172.21.15.55:0/3327789722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d00b0790 0x7f72d00cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:40.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.540+0000 7f72d679c700 1 --2- 172.21.15.55:0/3327789722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d01591e0 0x7f72d0159620 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009f20 tx=0x7f72c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82e77bf64b8558f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:40.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.540+0000 7f72d4f99700 1 -- 172.21.15.55:0/3327789722 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c800e070 con 0x7f72d01591e0 2023-10-06T03:34:40.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.540+0000 7f72d4f99700 1 -- 172.21.15.55:0/3327789722 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c800ba40 con 0x7f72d01591e0 2023-10-06T03:34:40.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.540+0000 7f72d4f99700 1 -- 172.21.15.55:0/3327789722 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c80092e0 con 0x7f72d01591e0 2023-10-06T03:34:40.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.541+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3327789722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d01591e0 msgr2=0x7f72d0159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:40.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.541+0000 7f72d6f9d700 1 --2- 172.21.15.55:0/3327789722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d01591e0 0x7f72d0159620 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f72c8009f20 tx=0x7f72c8009840 comp rx=0 tx=0).stop 2023-10-06T03:34:40.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.541+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3327789722 shutdown_connections 2023-10-06T03:34:40.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.541+0000 7f72d6f9d700 1 --2- 172.21.15.55:0/3327789722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d01591e0 0x7f72d0159620 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.541+0000 7f72d6f9d700 1 --2- 172.21.15.55:0/3327789722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d00b0790 0x7f72d00cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.541+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3327789722 >> 172.21.15.55:0/3327789722 conn(0x7f72d0063230 msgr2=0x7f72d0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:40.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.541+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3327789722 shutdown_connections 2023-10-06T03:34:40.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.541+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3327789722 wait complete. 2023-10-06T03:34:40.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.542+0000 7f72d6f9d700 1 Processor -- start 2023-10-06T03:34:40.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.543+0000 7f72d6f9d700 1 -- start start 2023-10-06T03:34:40.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.543+0000 7f72d6f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d00b03c0 0x7f72d0166a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:40.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.543+0000 7f72d6f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d00b0790 0x7f72d0169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:40.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d6f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d01591e0 0x7f72d00c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:40.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d6f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00c7a50 con 0x7f72d00b03c0 2023-10-06T03:34:40.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d6f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00c7cc0 con 0x7f72d01591e0 2023-10-06T03:34:40.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d6f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d00c7fa0 con 0x7f72d00b0790 2023-10-06T03:34:40.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d5f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d00b03c0 0x7f72d0166a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:40.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d5f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d00b03c0 0x7f72d0166a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44216/0 (socket says 172.21.15.55:44216) 2023-10-06T03:34:40.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d5f9b700 1 -- 172.21.15.55:0/3817091008 learned_addr learned my addr 172.21.15.55:0/3817091008 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:40.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d579a700 1 --2- 172.21.15.55:0/3817091008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d00b0790 0x7f72d0169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:40.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d679c700 1 --2- 172.21.15.55:0/3817091008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d01591e0 0x7f72d00c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:40.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d5f9b700 1 -- 172.21.15.55:0/3817091008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d00b0790 msgr2=0x7f72d0169cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:40.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d5f9b700 1 --2- 172.21.15.55:0/3817091008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d00b0790 0x7f72d0169cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d5f9b700 1 -- 172.21.15.55:0/3817091008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d01591e0 msgr2=0x7f72d00c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:40.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d5f9b700 1 --2- 172.21.15.55:0/3817091008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d01591e0 0x7f72d00c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d5f9b700 1 -- 172.21.15.55:0/3817091008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c8009ae0 con 0x7f72d00b03c0 2023-10-06T03:34:40.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.544+0000 7f72d5f9b700 1 --2- 172.21.15.55:0/3817091008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d00b03c0 0x7f72d0166a30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00b5e0 tx=0x7f72cc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:40.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.545+0000 7f72c5ffb700 1 -- 172.21.15.55:0/3817091008 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc00f510 con 0x7f72d00b03c0 2023-10-06T03:34:40.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.545+0000 7f72c5ffb700 1 -- 172.21.15.55:0/3817091008 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc00f670 con 0x7f72d00b03c0 2023-10-06T03:34:40.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.545+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3817091008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d00c8280 con 0x7f72d00b03c0 2023-10-06T03:34:40.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.545+0000 7f72c5ffb700 1 -- 172.21.15.55:0/3817091008 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc00fa30 con 0x7f72d00b03c0 2023-10-06T03:34:40.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.545+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3817091008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d00c8840 con 0x7f72d00b03c0 2023-10-06T03:34:40.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.546+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3817091008 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d004c3c0 con 0x7f72d00b03c0 2023-10-06T03:34:40.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.547+0000 7f72c5ffb700 1 -- 172.21.15.55:0/3817091008 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f72cc01a400 con 0x7f72d00b03c0 2023-10-06T03:34:40.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.548+0000 7f72c5ffb700 1 --2- 172.21.15.55:0/3817091008 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f72b4067130 0x7f72b40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:40.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.548+0000 7f72c5ffb700 1 -- 172.21.15.55:0/3817091008 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f72cc0846e0 con 0x7f72d00b03c0 2023-10-06T03:34:40.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.548+0000 7f72d579a700 1 --2- 172.21.15.55:0/3817091008 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f72b4067130 0x7f72b40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:40.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.549+0000 7f72d579a700 1 --2- 172.21.15.55:0/3817091008 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f72b4067130 0x7f72b40695e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f72d00c7550 tx=0x7f72c0006f90 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:40.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.552+0000 7f72c5ffb700 1 -- 172.21.15.55:0/3817091008 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72cc051450 con 0x7f72d00b03c0 2023-10-06T03:34:40.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.781+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3817091008 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f72d00cad50 con 0x7f72b4067130 2023-10-06T03:34:40.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.836+0000 7f72c5ffb700 1 -- 172.21.15.55:0/3817091008 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f72d00cad50 con 0x7f72b4067130 2023-10-06T03:34:40.837 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-10-06T03:34:40.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.838+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3817091008 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f72b4067130 msgr2=0x7f72b40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:40.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.838+0000 7f72d6f9d700 1 --2- 172.21.15.55:0/3817091008 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f72b4067130 0x7f72b40695e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f72d00c7550 tx=0x7f72c0006f90 comp rx=0 tx=0).stop 2023-10-06T03:34:40.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.838+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3817091008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d00b03c0 msgr2=0x7f72d0166a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:40.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.838+0000 7f72d6f9d700 1 --2- 172.21.15.55:0/3817091008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d00b03c0 0x7f72d0166a30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00b5e0 tx=0x7f72cc009120 comp rx=0 tx=0).stop 2023-10-06T03:34:40.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.838+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3817091008 shutdown_connections 2023-10-06T03:34:40.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.838+0000 7f72d6f9d700 1 --2- 172.21.15.55:0/3817091008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72d01591e0 0x7f72d00c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.838+0000 7f72d6f9d700 1 --2- 172.21.15.55:0/3817091008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72d00b03c0 0x7f72d0166a30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.838+0000 7f72d6f9d700 1 --2- 172.21.15.55:0/3817091008 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f72b4067130 0x7f72b40695e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.838+0000 7f72d6f9d700 1 --2- 172.21.15.55:0/3817091008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72d00b0790 0x7f72d0169cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:40.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.838+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3817091008 >> 172.21.15.55:0/3817091008 conn(0x7f72d0063230 msgr2=0x7f72d00d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:40.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.839+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3817091008 shutdown_connections 2023-10-06T03:34:40.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:40.839+0000 7f72d6f9d700 1 -- 172.21.15.55:0/3817091008 wait complete. 2023-10-06T03:34:40.857 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-10-06T03:34:40.857 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir1 1M 2023-10-06T03:34:41.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.047+0000 7f8998b9d700 1 Processor -- start 2023-10-06T03:34:41.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.047+0000 7f8998b9d700 1 -- start start 2023-10-06T03:34:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.049+0000 7f8998b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89940685e0 0x7f899415a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.049+0000 7f8998b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8994154000 0x7f899406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.049+0000 7f8998b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89940c9480 con 0x7f899415d3f0 2023-10-06T03:34:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.049+0000 7f8998b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899415d880 con 0x7f8994154000 2023-10-06T03:34:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.049+0000 7f8998b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899415db60 con 0x7f89940685e0 2023-10-06T03:34:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.049+0000 7f8993fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8994154000 0x7f899406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.049+0000 7f8993fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8994154000 0x7f899406eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44218/0 (socket says 172.21.15.55:44218) 2023-10-06T03:34:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.049+0000 7f8993fff700 1 -- 172.21.15.55:0/3090406702 learned_addr learned my addr 172.21.15.55:0/3090406702 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:41.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.049+0000 7f89937fe700 1 --2- 172.21.15.55:0/3090406702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89940685e0 0x7f899415a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:41.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.049+0000 7f8993fff700 1 -- 172.21.15.55:0/3090406702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89940685e0 msgr2=0x7f899415a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:41.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.049+0000 7f8993fff700 1 --2- 172.21.15.55:0/3090406702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89940685e0 0x7f899415a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.049+0000 7f8993fff700 1 -- 172.21.15.55:0/3090406702 >> v1:172.21.15.163:6789/0 conn(0x7f899415d3f0 legacy=0x7f89941538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:41.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.049+0000 7f8993fff700 1 -- 172.21.15.55:0/3090406702 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899415de40 con 0x7f8994154000 2023-10-06T03:34:41.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.050+0000 7f8993fff700 1 --2- 172.21.15.55:0/3090406702 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8994154000 0x7f899406eb00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f898c009f20 tx=0x7f898c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde3afece6ddca4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:41.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.050+0000 7f89927fc700 1 -- 172.21.15.55:0/3090406702 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c00e070 con 0x7f8994154000 2023-10-06T03:34:41.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.050+0000 7f89927fc700 1 -- 172.21.15.55:0/3090406702 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898c00bb10 con 0x7f8994154000 2023-10-06T03:34:41.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.050+0000 7f89927fc700 1 -- 172.21.15.55:0/3090406702 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c0092e0 con 0x7f8994154000 2023-10-06T03:34:41.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.050+0000 7f8998b9d700 1 -- 172.21.15.55:0/3090406702 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8994154000 msgr2=0x7f899406eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:41.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.050+0000 7f8998b9d700 1 --2- 172.21.15.55:0/3090406702 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8994154000 0x7f899406eb00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f898c009f20 tx=0x7f898c009840 comp rx=0 tx=0).stop 2023-10-06T03:34:41.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.051+0000 7f8998b9d700 1 -- 172.21.15.55:0/3090406702 shutdown_connections 2023-10-06T03:34:41.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.051+0000 7f8998b9d700 1 --2- 172.21.15.55:0/3090406702 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8994154000 0x7f899406eb00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.051+0000 7f8998b9d700 1 --2- 172.21.15.55:0/3090406702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f89940685e0 0x7f899415a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.051+0000 7f8998b9d700 1 -- 172.21.15.55:0/3090406702 >> 172.21.15.55:0/3090406702 conn(0x7f899406b420 msgr2=0x7f89940cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:41.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.051+0000 7f8998b9d700 1 -- 172.21.15.55:0/3090406702 shutdown_connections 2023-10-06T03:34:41.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.051+0000 7f8998b9d700 1 -- 172.21.15.55:0/3090406702 wait complete. 2023-10-06T03:34:41.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.052+0000 7f8998b9d700 1 Processor -- start 2023-10-06T03:34:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.052+0000 7f8998b9d700 1 -- start start 2023-10-06T03:34:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.053+0000 7f8998b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89940685e0 0x7f89940b2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.053+0000 7f8998b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8994154000 0x7f8994159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.053+0000 7f8998b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f899415d3f0 0x7f8994158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.053+0000 7f8998b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89940b2720 con 0x7f8994154000 2023-10-06T03:34:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.053+0000 7f8998b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8994159ba0 con 0x7f89940685e0 2023-10-06T03:34:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.053+0000 7f8998b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899416ac40 con 0x7f899415d3f0 2023-10-06T03:34:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.053+0000 7f8992ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8994154000 0x7f8994159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.053+0000 7f8992ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8994154000 0x7f8994159660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44232/0 (socket says 172.21.15.55:44232) 2023-10-06T03:34:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.053+0000 7f8992ffd700 1 -- 172.21.15.55:0/805783613 learned_addr learned my addr 172.21.15.55:0/805783613 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:41.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.054+0000 7f8993fff700 1 --2- 172.21.15.55:0/805783613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f899415d3f0 0x7f8994158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.054+0000 7f89937fe700 1 --2- 172.21.15.55:0/805783613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89940685e0 0x7f89940b2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.054+0000 7f8992ffd700 1 -- 172.21.15.55:0/805783613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f899415d3f0 msgr2=0x7f8994158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.054+0000 7f8992ffd700 1 --2- 172.21.15.55:0/805783613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f899415d3f0 0x7f8994158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.054+0000 7f8992ffd700 1 -- 172.21.15.55:0/805783613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89940685e0 msgr2=0x7f89940b2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.054+0000 7f8992ffd700 1 --2- 172.21.15.55:0/805783613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89940685e0 0x7f89940b2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.054+0000 7f8992ffd700 1 -- 172.21.15.55:0/805783613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c009ae0 con 0x7f8994154000 2023-10-06T03:34:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.054+0000 7f8992ffd700 1 --2- 172.21.15.55:0/805783613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8994154000 0x7f8994159660 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f898800c540 tx=0x7f8988001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.054+0000 7f897b7fe700 1 -- 172.21.15.55:0/805783613 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988011590 con 0x7f8994154000 2023-10-06T03:34:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.054+0000 7f897b7fe700 1 -- 172.21.15.55:0/805783613 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89880116f0 con 0x7f8994154000 2023-10-06T03:34:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.054+0000 7f897b7fe700 1 -- 172.21.15.55:0/805783613 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988011ab0 con 0x7f8994154000 2023-10-06T03:34:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.054+0000 7f8998b9d700 1 -- 172.21.15.55:0/805783613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899416a030 con 0x7f8994154000 2023-10-06T03:34:41.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.054+0000 7f8998b9d700 1 -- 172.21.15.55:0/805783613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899416a5f0 con 0x7f8994154000 2023-10-06T03:34:41.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.055+0000 7f8998b9d700 1 -- 172.21.15.55:0/805783613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899416add0 con 0x7f8994154000 2023-10-06T03:34:41.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.056+0000 7f897b7fe700 1 -- 172.21.15.55:0/805783613 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f898801b3b0 con 0x7f8994154000 2023-10-06T03:34:41.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.057+0000 7f897b7fe700 1 --2- 172.21.15.55:0/805783613 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8980067130 0x7f89800695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:41.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.057+0000 7f897b7fe700 1 -- 172.21.15.55:0/805783613 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f8988083fe0 con 0x7f8994154000 2023-10-06T03:34:41.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.057+0000 7f89937fe700 1 --2- 172.21.15.55:0/805783613 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8980067130 0x7f89800695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:41.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.058+0000 7f89937fe700 1 --2- 172.21.15.55:0/805783613 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8980067130 0x7f89800695e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8984006fd0 tx=0x7f8984008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:41.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.061+0000 7f897b7fe700 1 -- 172.21.15.55:0/805783613 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8988050da0 con 0x7f8994154000 2023-10-06T03:34:41.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.289+0000 7f8998b9d700 1 -- 172.21.15.55:0/805783613 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f899415e120 con 0x7f8980067130 2023-10-06T03:34:41.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.357+0000 7f897b7fe700 1 -- 172.21.15.55:0/805783613 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f899415e120 con 0x7f8980067130 2023-10-06T03:34:41.358 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-10-06T03:34:41.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.359+0000 7f8998b9d700 1 -- 172.21.15.55:0/805783613 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8980067130 msgr2=0x7f89800695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:41.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.359+0000 7f8998b9d700 1 --2- 172.21.15.55:0/805783613 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8980067130 0x7f89800695e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8984006fd0 tx=0x7f8984008040 comp rx=0 tx=0).stop 2023-10-06T03:34:41.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.359+0000 7f8998b9d700 1 -- 172.21.15.55:0/805783613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8994154000 msgr2=0x7f8994159660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:41.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.359+0000 7f8998b9d700 1 --2- 172.21.15.55:0/805783613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8994154000 0x7f8994159660 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f898800c540 tx=0x7f8988001070 comp rx=0 tx=0).stop 2023-10-06T03:34:41.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.360+0000 7f8998b9d700 1 -- 172.21.15.55:0/805783613 shutdown_connections 2023-10-06T03:34:41.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.360+0000 7f8998b9d700 1 --2- 172.21.15.55:0/805783613 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8980067130 0x7f89800695e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.360+0000 7f8998b9d700 1 --2- 172.21.15.55:0/805783613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f899415d3f0 0x7f8994158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.360+0000 7f8998b9d700 1 --2- 172.21.15.55:0/805783613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f89940685e0 0x7f89940b2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.360+0000 7f8998b9d700 1 --2- 172.21.15.55:0/805783613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8994154000 0x7f8994159660 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.360+0000 7f8998b9d700 1 -- 172.21.15.55:0/805783613 >> 172.21.15.55:0/805783613 conn(0x7f899406b420 msgr2=0x7f89940c9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:41.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.360+0000 7f8998b9d700 1 -- 172.21.15.55:0/805783613 shutdown_connections 2023-10-06T03:34:41.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.360+0000 7f8998b9d700 1 -- 172.21.15.55:0/805783613 wait complete. 2023-10-06T03:34:41.378 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-10-06T03:34:41.378 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir2 1M 2023-10-06T03:34:41.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.569+0000 7f8a8579d700 1 Processor -- start 2023-10-06T03:34:41.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.569+0000 7f8a8579d700 1 -- start start 2023-10-06T03:34:41.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.571+0000 7f8a8579d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a801597d0 0x7f8a800afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:41.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.571+0000 7f8a8579d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a800ac0e0 0x7f8a8006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:41.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.571+0000 7f8a8579d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8006e510 con 0x7f8a8006eee0 2023-10-06T03:34:41.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.571+0000 7f8a8579d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8015e450 con 0x7f8a800ac0e0 2023-10-06T03:34:41.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.571+0000 7f8a8579d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80069180 con 0x7f8a801597d0 2023-10-06T03:34:41.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.571+0000 7f8a7f7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a800ac0e0 0x7f8a8006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:41.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.571+0000 7f8a84f9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8a8006eee0 0x7f8a800a98d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35672/0 (socket says 172.21.15.55:35672) 2023-10-06T03:34:41.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.571+0000 7f8a84f9c700 1 -- 172.21.15.55:0/3550624233 learned_addr learned my addr 172.21.15.55:0/3550624233 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:41.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.571+0000 7f8a7ffff700 1 --2- 172.21.15.55:0/3550624233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a801597d0 0x7f8a800afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:41.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.571+0000 7f8a7effd700 1 -- 172.21.15.55:0/3550624233 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795714291 0 0) 0x7f8a8006e510 con 0x7f8a8006eee0 2023-10-06T03:34:41.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.571+0000 7f8a7effd700 1 -- 172.21.15.55:0/3550624233 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a68003570 con 0x7f8a8006eee0 2023-10-06T03:34:41.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.571+0000 7f8a7f7fe700 1 -- 172.21.15.55:0/3550624233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a801597d0 msgr2=0x7f8a800afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:41.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.572+0000 7f8a7f7fe700 1 --2- 172.21.15.55:0/3550624233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a801597d0 0x7f8a800afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.572+0000 7f8a7f7fe700 1 -- 172.21.15.55:0/3550624233 >> v1:172.21.15.55:6789/0 conn(0x7f8a8006eee0 legacy=0x7f8a800a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:41.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.572+0000 7f8a7f7fe700 1 -- 172.21.15.55:0/3550624233 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8006dec0 con 0x7f8a800ac0e0 2023-10-06T03:34:41.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.572+0000 7f8a7f7fe700 1 --2- 172.21.15.55:0/3550624233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a800ac0e0 0x7f8a8006e9a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8a740098e0 tx=0x7f8a740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c88716591795f00 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:41.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.572+0000 7f8a7effd700 1 -- 172.21.15.55:0/3550624233 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400b640 con 0x7f8a800ac0e0 2023-10-06T03:34:41.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.572+0000 7f8a7effd700 1 -- 172.21.15.55:0/3550624233 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7400b7a0 con 0x7f8a800ac0e0 2023-10-06T03:34:41.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.572+0000 7f8a7effd700 1 -- 172.21.15.55:0/3550624233 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7400bb40 con 0x7f8a800ac0e0 2023-10-06T03:34:41.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.573+0000 7f8a8579d700 1 -- 172.21.15.55:0/3550624233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a800ac0e0 msgr2=0x7f8a8006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:41.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.573+0000 7f8a8579d700 1 --2- 172.21.15.55:0/3550624233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a800ac0e0 0x7f8a8006e9a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8a740098e0 tx=0x7f8a740091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.573+0000 7f8a8579d700 1 -- 172.21.15.55:0/3550624233 shutdown_connections 2023-10-06T03:34:41.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.573+0000 7f8a8579d700 1 --2- 172.21.15.55:0/3550624233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a801597d0 0x7f8a800afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.573+0000 7f8a8579d700 1 --2- 172.21.15.55:0/3550624233 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a800ac0e0 0x7f8a8006e9a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.573+0000 7f8a8579d700 1 -- 172.21.15.55:0/3550624233 >> 172.21.15.55:0/3550624233 conn(0x7f8a8006adf0 msgr2=0x7f8a800ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:41.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.573+0000 7f8a8579d700 1 -- 172.21.15.55:0/3550624233 shutdown_connections 2023-10-06T03:34:41.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.573+0000 7f8a8579d700 1 -- 172.21.15.55:0/3550624233 wait complete. 2023-10-06T03:34:41.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.574+0000 7f8a8579d700 1 Processor -- start 2023-10-06T03:34:41.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.574+0000 7f8a8579d700 1 -- start start 2023-10-06T03:34:41.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.575+0000 7f8a8579d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a8006eee0 0x7f8a800c23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:41.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.575+0000 7f8a8579d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a801597d0 0x7f8a80157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:41.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.575+0000 7f8a8579d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8015d890 0x7f8a801699f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:41.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.575+0000 7f8a8579d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80159ba0 con 0x7f8a801597d0 2023-10-06T03:34:41.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.575+0000 7f8a8579d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a80159df0 con 0x7f8a8015d890 2023-10-06T03:34:41.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.575+0000 7f8a8579d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8015a0d0 con 0x7f8a8006eee0 2023-10-06T03:34:41.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.575+0000 7f8a7f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a801597d0 0x7f8a80157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:41.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.575+0000 7f8a7f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a801597d0 0x7f8a80157490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44250/0 (socket says 172.21.15.55:44250) 2023-10-06T03:34:41.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.575+0000 7f8a7f7fe700 1 -- 172.21.15.55:0/505470188 learned_addr learned my addr 172.21.15.55:0/505470188 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:41.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.576+0000 7f8a7ffff700 1 --2- 172.21.15.55:0/505470188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a8006eee0 0x7f8a800c23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:41.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.576+0000 7f8a84f9c700 1 --2- 172.21.15.55:0/505470188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8015d890 0x7f8a801699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:41.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.576+0000 7f8a7f7fe700 1 -- 172.21.15.55:0/505470188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a8006eee0 msgr2=0x7f8a800c23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:41.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.576+0000 7f8a7f7fe700 1 --2- 172.21.15.55:0/505470188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a8006eee0 0x7f8a800c23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.576+0000 7f8a7f7fe700 1 -- 172.21.15.55:0/505470188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8015d890 msgr2=0x7f8a801699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:41.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.576+0000 7f8a7f7fe700 1 --2- 172.21.15.55:0/505470188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8015d890 0x7f8a801699f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.576+0000 7f8a7f7fe700 1 -- 172.21.15.55:0/505470188 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a74009420 con 0x7f8a801597d0 2023-10-06T03:34:41.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.576+0000 7f8a7f7fe700 1 --2- 172.21.15.55:0/505470188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a801597d0 0x7f8a80157490 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8a8006dad0 tx=0x7f8a74017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:41.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.576+0000 7f8a67fff700 1 -- 172.21.15.55:0/505470188 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7401bda0 con 0x7f8a801597d0 2023-10-06T03:34:41.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.576+0000 7f8a67fff700 1 -- 172.21.15.55:0/505470188 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a74011420 con 0x7f8a801597d0 2023-10-06T03:34:41.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.576+0000 7f8a67fff700 1 -- 172.21.15.55:0/505470188 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a740117e0 con 0x7f8a801597d0 2023-10-06T03:34:41.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.576+0000 7f8a8579d700 1 -- 172.21.15.55:0/505470188 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8015a3b0 con 0x7f8a801597d0 2023-10-06T03:34:41.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.577+0000 7f8a8579d700 1 -- 172.21.15.55:0/505470188 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a800c2910 con 0x7f8a801597d0 2023-10-06T03:34:41.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.578+0000 7f8a8579d700 1 -- 172.21.15.55:0/505470188 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8015a8d0 con 0x7f8a801597d0 2023-10-06T03:34:41.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.578+0000 7f8a67fff700 1 -- 172.21.15.55:0/505470188 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f8a74011b90 con 0x7f8a801597d0 2023-10-06T03:34:41.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.579+0000 7f8a67fff700 1 --2- 172.21.15.55:0/505470188 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a6c067130 0x7f8a6c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:41.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.579+0000 7f8a67fff700 1 -- 172.21.15.55:0/505470188 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f8a740862d0 con 0x7f8a801597d0 2023-10-06T03:34:41.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.580+0000 7f8a7ffff700 1 --2- 172.21.15.55:0/505470188 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a6c067130 0x7f8a6c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:41.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.580+0000 7f8a7ffff700 1 --2- 172.21.15.55:0/505470188 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a6c067130 0x7f8a6c0695e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8a70006fd0 tx=0x7f8a70008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:41.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.583+0000 7f8a67fff700 1 -- 172.21.15.55:0/505470188 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a8015a8d0 con 0x7f8a801597d0 2023-10-06T03:34:41.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.812+0000 7f8a8579d700 1 -- 172.21.15.55:0/505470188 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a80159f60 con 0x7f8a6c067130 2023-10-06T03:34:41.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.881+0000 7f8a67fff700 1 -- 172.21.15.55:0/505470188 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8a80159f60 con 0x7f8a6c067130 2023-10-06T03:34:41.883 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-10-06T03:34:41.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.884+0000 7f8a8579d700 1 -- 172.21.15.55:0/505470188 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a6c067130 msgr2=0x7f8a6c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:41.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.884+0000 7f8a8579d700 1 --2- 172.21.15.55:0/505470188 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a6c067130 0x7f8a6c0695e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8a70006fd0 tx=0x7f8a70008040 comp rx=0 tx=0).stop 2023-10-06T03:34:41.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.884+0000 7f8a8579d700 1 -- 172.21.15.55:0/505470188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a801597d0 msgr2=0x7f8a80157490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:41.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.884+0000 7f8a8579d700 1 --2- 172.21.15.55:0/505470188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a801597d0 0x7f8a80157490 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8a8006dad0 tx=0x7f8a74017000 comp rx=0 tx=0).stop 2023-10-06T03:34:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.884+0000 7f8a8579d700 1 -- 172.21.15.55:0/505470188 shutdown_connections 2023-10-06T03:34:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.884+0000 7f8a8579d700 1 --2- 172.21.15.55:0/505470188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a8015d890 0x7f8a801699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.884+0000 7f8a8579d700 1 --2- 172.21.15.55:0/505470188 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a6c067130 0x7f8a6c0695e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.884+0000 7f8a8579d700 1 --2- 172.21.15.55:0/505470188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a8006eee0 0x7f8a800c23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.884+0000 7f8a8579d700 1 --2- 172.21.15.55:0/505470188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a801597d0 0x7f8a80157490 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.884+0000 7f8a8579d700 1 -- 172.21.15.55:0/505470188 >> 172.21.15.55:0/505470188 conn(0x7f8a8006adf0 msgr2=0x7f8a800afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.884+0000 7f8a8579d700 1 -- 172.21.15.55:0/505470188 shutdown_connections 2023-10-06T03:34:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:41.884+0000 7f8a8579d700 1 -- 172.21.15.55:0/505470188 wait complete. 2023-10-06T03:34:41.902 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-10-06T03:34:41.903 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir3 1M 2023-10-06T03:34:42.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.093+0000 7f2f63fff700 1 Processor -- start 2023-10-06T03:34:42.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.093+0000 7f2f63fff700 1 -- start start 2023-10-06T03:34:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.095+0000 7f2f63fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f640ac0e0 0x7f2f640afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.095+0000 7f2f63fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2f6406f0b0 0x7f2f6415d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.095+0000 7f2f63fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6406e510 con 0x7f2f641597d0 2023-10-06T03:34:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.095+0000 7f2f63fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f64069180 con 0x7f2f6406f0b0 2023-10-06T03:34:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.095+0000 7f2f63fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6406dec0 con 0x7f2f640ac0e0 2023-10-06T03:34:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.095+0000 7f2f637fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2f6406f0b0 0x7f2f6415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.095+0000 7f2f637fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2f6406f0b0 0x7f2f6415d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44258/0 (socket says 172.21.15.55:44258) 2023-10-06T03:34:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.095+0000 7f2f637fe700 1 -- 172.21.15.55:0/1360260448 learned_addr learned my addr 172.21.15.55:0/1360260448 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.095+0000 7f2f627fc700 1 --2- 172.21.15.55:0/1360260448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f640ac0e0 0x7f2f640afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.095+0000 7f2f637fe700 1 -- 172.21.15.55:0/1360260448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f640ac0e0 msgr2=0x7f2f640afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.095+0000 7f2f637fe700 1 --2- 172.21.15.55:0/1360260448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f640ac0e0 0x7f2f640afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.095+0000 7f2f637fe700 1 -- 172.21.15.55:0/1360260448 >> v1:172.21.15.163:6789/0 conn(0x7f2f641597d0 legacy=0x7f2f6406e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.095+0000 7f2f637fe700 1 -- 172.21.15.55:0/1360260448 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6406e1a0 con 0x7f2f6406f0b0 2023-10-06T03:34:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.096+0000 7f2f637fe700 1 --2- 172.21.15.55:0/1360260448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6406f0b0 0x7f2f6415d440 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009f20 tx=0x7f2f5c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3a54f1ee57d44a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:42.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.096+0000 7f2f61ffb700 1 -- 172.21.15.55:0/1360260448 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c00e070 con 0x7f2f6406f0b0 2023-10-06T03:34:42.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.096+0000 7f2f61ffb700 1 -- 172.21.15.55:0/1360260448 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5c00ba80 con 0x7f2f6406f0b0 2023-10-06T03:34:42.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.096+0000 7f2f61ffb700 1 -- 172.21.15.55:0/1360260448 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c0092e0 con 0x7f2f6406f0b0 2023-10-06T03:34:42.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.096+0000 7f2f63fff700 1 -- 172.21.15.55:0/1360260448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6406f0b0 msgr2=0x7f2f6415d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:42.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.096+0000 7f2f63fff700 1 --2- 172.21.15.55:0/1360260448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6406f0b0 0x7f2f6415d440 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009f20 tx=0x7f2f5c009840 comp rx=0 tx=0).stop 2023-10-06T03:34:42.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.096+0000 7f2f63fff700 1 -- 172.21.15.55:0/1360260448 shutdown_connections 2023-10-06T03:34:42.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.096+0000 7f2f63fff700 1 --2- 172.21.15.55:0/1360260448 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6406f0b0 0x7f2f6415d440 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.096+0000 7f2f63fff700 1 --2- 172.21.15.55:0/1360260448 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f640ac0e0 0x7f2f640afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.096+0000 7f2f63fff700 1 -- 172.21.15.55:0/1360260448 >> 172.21.15.55:0/1360260448 conn(0x7f2f6406adf0 msgr2=0x7f2f640ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:42.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.097+0000 7f2f63fff700 1 -- 172.21.15.55:0/1360260448 shutdown_connections 2023-10-06T03:34:42.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.097+0000 7f2f63fff700 1 -- 172.21.15.55:0/1360260448 wait complete. 2023-10-06T03:34:42.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.098+0000 7f2f63fff700 1 Processor -- start 2023-10-06T03:34:42.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.098+0000 7f2f63fff700 1 -- start start 2023-10-06T03:34:42.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.099+0000 7f2f63fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6406f0b0 0x7f2f6414a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:42.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.099+0000 7f2f63fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f641597d0 0x7f2f640c23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:42.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.099+0000 7f2f63fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f640c2910 0x7f2f640ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:42.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.099+0000 7f2f63fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6414abb0 con 0x7f2f6406f0b0 2023-10-06T03:34:42.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.099+0000 7f2f63fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f641697f0 con 0x7f2f641597d0 2023-10-06T03:34:42.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.099+0000 7f2f63fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f640c3960 con 0x7f2f640c2910 2023-10-06T03:34:42.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.099+0000 7f2f62ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6406f0b0 0x7f2f6414a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:42.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.099+0000 7f2f62ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6406f0b0 0x7f2f6414a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44264/0 (socket says 172.21.15.55:44264) 2023-10-06T03:34:42.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.099+0000 7f2f62ffd700 1 -- 172.21.15.55:0/3224931452 learned_addr learned my addr 172.21.15.55:0/3224931452 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:42.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.100+0000 7f2f627fc700 1 --2- 172.21.15.55:0/3224931452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f641597d0 0x7f2f640c23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:42.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.100+0000 7f2f637fe700 1 --2- 172.21.15.55:0/3224931452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f640c2910 0x7f2f640ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:42.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.100+0000 7f2f62ffd700 1 -- 172.21.15.55:0/3224931452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f640c2910 msgr2=0x7f2f640ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:42.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.100+0000 7f2f62ffd700 1 --2- 172.21.15.55:0/3224931452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f640c2910 0x7f2f640ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.100+0000 7f2f62ffd700 1 -- 172.21.15.55:0/3224931452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f641597d0 msgr2=0x7f2f640c23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:42.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.100+0000 7f2f62ffd700 1 --2- 172.21.15.55:0/3224931452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f641597d0 0x7f2f640c23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.100+0000 7f2f62ffd700 1 -- 172.21.15.55:0/3224931452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c009ae0 con 0x7f2f6406f0b0 2023-10-06T03:34:42.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.100+0000 7f2f62ffd700 1 --2- 172.21.15.55:0/3224931452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6406f0b0 0x7f2f6414a670 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400c4e0 tx=0x7f2f5400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:42.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.100+0000 7f2f4affd700 1 -- 172.21.15.55:0/3224931452 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f54011590 con 0x7f2f6406f0b0 2023-10-06T03:34:42.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.100+0000 7f2f4affd700 1 -- 172.21.15.55:0/3224931452 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f54011710 con 0x7f2f6406f0b0 2023-10-06T03:34:42.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.100+0000 7f2f4affd700 1 -- 172.21.15.55:0/3224931452 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f54011ad0 con 0x7f2f6406f0b0 2023-10-06T03:34:42.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.100+0000 7f2f63fff700 1 -- 172.21.15.55:0/3224931452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f640c3c40 con 0x7f2f6406f0b0 2023-10-06T03:34:42.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.100+0000 7f2f63fff700 1 -- 172.21.15.55:0/3224931452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f640c4200 con 0x7f2f6406f0b0 2023-10-06T03:34:42.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.101+0000 7f2f63fff700 1 -- 172.21.15.55:0/3224931452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6404c3c0 con 0x7f2f6406f0b0 2023-10-06T03:34:42.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.102+0000 7f2f4affd700 1 -- 172.21.15.55:0/3224931452 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f2f5401b3b0 con 0x7f2f6406f0b0 2023-10-06T03:34:42.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.103+0000 7f2f4affd700 1 --2- 172.21.15.55:0/3224931452 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2f50067130 0x7f2f500695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:42.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.103+0000 7f2f4affd700 1 -- 172.21.15.55:0/3224931452 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f2f540840e0 con 0x7f2f6406f0b0 2023-10-06T03:34:42.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.104+0000 7f2f627fc700 1 --2- 172.21.15.55:0/3224931452 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2f50067130 0x7f2f500695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:42.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.104+0000 7f2f627fc700 1 --2- 172.21.15.55:0/3224931452 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2f50067130 0x7f2f500695e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2f6415e7f0 tx=0x7f2f58008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:42.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.107+0000 7f2f4affd700 1 -- 172.21.15.55:0/3224931452 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2f54050dd0 con 0x7f2f6406f0b0 2023-10-06T03:34:42.163 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:34:42.167 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:34:42.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.341+0000 7f2f63fff700 1 -- 172.21.15.55:0/3224931452 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f640cd4a0 con 0x7f2f50067130 2023-10-06T03:34:42.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.404+0000 7f2f4affd700 1 -- 172.21.15.55:0/3224931452 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f2f640cd4a0 con 0x7f2f50067130 2023-10-06T03:34:42.405 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-10-06T03:34:42.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.406+0000 7f2f63fff700 1 -- 172.21.15.55:0/3224931452 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2f50067130 msgr2=0x7f2f500695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:42.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.406+0000 7f2f63fff700 1 --2- 172.21.15.55:0/3224931452 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2f50067130 0x7f2f500695e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2f6415e7f0 tx=0x7f2f58008040 comp rx=0 tx=0).stop 2023-10-06T03:34:42.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.406+0000 7f2f63fff700 1 -- 172.21.15.55:0/3224931452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6406f0b0 msgr2=0x7f2f6414a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:42.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.406+0000 7f2f63fff700 1 --2- 172.21.15.55:0/3224931452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6406f0b0 0x7f2f6414a670 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400c4e0 tx=0x7f2f5400c8a0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.406+0000 7f2f63fff700 1 -- 172.21.15.55:0/3224931452 shutdown_connections 2023-10-06T03:34:42.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.406+0000 7f2f63fff700 1 --2- 172.21.15.55:0/3224931452 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2f50067130 0x7f2f500695e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.406+0000 7f2f63fff700 1 --2- 172.21.15.55:0/3224931452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2f640c2910 0x7f2f640ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.406+0000 7f2f63fff700 1 --2- 172.21.15.55:0/3224931452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2f6406f0b0 0x7f2f6414a670 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.406+0000 7f2f63fff700 1 --2- 172.21.15.55:0/3224931452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2f641597d0 0x7f2f640c23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.406+0000 7f2f63fff700 1 -- 172.21.15.55:0/3224931452 >> 172.21.15.55:0/3224931452 conn(0x7f2f6406adf0 msgr2=0x7f2f64158830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:42.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.406+0000 7f2f63fff700 1 -- 172.21.15.55:0/3224931452 shutdown_connections 2023-10-06T03:34:42.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.406+0000 7f2f63fff700 1 -- 172.21.15.55:0/3224931452 wait complete. 2023-10-06T03:34:42.425 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-10-06T03:34:42.425 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir4 1M 2023-10-06T03:34:42.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.616+0000 7f88c659d700 1 Processor -- start 2023-10-06T03:34:42.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.616+0000 7f88c659d700 1 -- start start 2023-10-06T03:34:42.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.618+0000 7f88c659d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c00cc8d0 0x7f88c015b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:42.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.618+0000 7f88c659d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f88c015b960 0x7f88c00ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:42.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.618+0000 7f88c659d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c0157a50 con 0x7f88c015b050 2023-10-06T03:34:42.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.618+0000 7f88c659d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c0158f10 con 0x7f88c015b960 2023-10-06T03:34:42.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.618+0000 7f88c659d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c01591b0 con 0x7f88c00cc8d0 2023-10-06T03:34:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.618+0000 7f88c5d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f88c015b960 0x7f88c00ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.618+0000 7f88c5d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f88c015b960 0x7f88c00ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44276/0 (socket says 172.21.15.55:44276) 2023-10-06T03:34:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.618+0000 7f88c5d9c700 1 -- 172.21.15.55:0/2165276253 learned_addr learned my addr 172.21.15.55:0/2165276253 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.618+0000 7f88c4d9a700 1 --2- 172.21.15.55:0/2165276253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c00cc8d0 0x7f88c015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.618+0000 7f88c5d9c700 1 -- 172.21.15.55:0/2165276253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c00cc8d0 msgr2=0x7f88c015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.618+0000 7f88c5d9c700 1 --2- 172.21.15.55:0/2165276253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c00cc8d0 0x7f88c015b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.618+0000 7f88c5d9c700 1 -- 172.21.15.55:0/2165276253 >> v1:172.21.15.163:6789/0 conn(0x7f88c015b050 legacy=0x7f88c00ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.618+0000 7f88c5d9c700 1 -- 172.21.15.55:0/2165276253 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c00ca9e0 con 0x7f88c015b960 2023-10-06T03:34:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.618+0000 7f88c4d9a700 1 --2- 172.21.15.55:0/2165276253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c00cc8d0 0x7f88c015b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.619+0000 7f88c5d9c700 1 --2- 172.21.15.55:0/2165276253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c015b960 0x7f88c00ada30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009f20 tx=0x7f88b8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbaa9ae82c98e076 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.619+0000 7f88b7fff700 1 -- 172.21.15.55:0/2165276253 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b800e070 con 0x7f88c015b960 2023-10-06T03:34:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.619+0000 7f88b7fff700 1 -- 172.21.15.55:0/2165276253 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b800ba80 con 0x7f88c015b960 2023-10-06T03:34:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.619+0000 7f88b7fff700 1 -- 172.21.15.55:0/2165276253 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80092e0 con 0x7f88c015b960 2023-10-06T03:34:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.619+0000 7f88c659d700 1 -- 172.21.15.55:0/2165276253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c015b960 msgr2=0x7f88c00ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:42.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.619+0000 7f88c659d700 1 --2- 172.21.15.55:0/2165276253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c015b960 0x7f88c00ada30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009f20 tx=0x7f88b8009840 comp rx=0 tx=0).stop 2023-10-06T03:34:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.619+0000 7f88c659d700 1 -- 172.21.15.55:0/2165276253 shutdown_connections 2023-10-06T03:34:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.619+0000 7f88c659d700 1 --2- 172.21.15.55:0/2165276253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c015b960 0x7f88c00ada30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.619+0000 7f88c659d700 1 --2- 172.21.15.55:0/2165276253 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c00cc8d0 0x7f88c015b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.619+0000 7f88c659d700 1 -- 172.21.15.55:0/2165276253 >> 172.21.15.55:0/2165276253 conn(0x7f88c0068e20 msgr2=0x7f88c00cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.620+0000 7f88c659d700 1 -- 172.21.15.55:0/2165276253 shutdown_connections 2023-10-06T03:34:42.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.620+0000 7f88c659d700 1 -- 172.21.15.55:0/2165276253 wait complete. 2023-10-06T03:34:42.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.621+0000 7f88c659d700 1 Processor -- start 2023-10-06T03:34:42.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.621+0000 7f88c659d700 1 -- start start 2023-10-06T03:34:42.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.622+0000 7f88c659d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cc8d0 0x7f88c00a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:42.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.622+0000 7f88c659d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c015b050 0x7f88c015e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:42.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.622+0000 7f88c659d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c015b960 0x7f88c0149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:42.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.622+0000 7f88c659d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c0156420 con 0x7f88c00cc8d0 2023-10-06T03:34:42.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.622+0000 7f88c659d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c016d1c0 con 0x7f88c015b960 2023-10-06T03:34:42.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.622+0000 7f88c659d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c016d4a0 con 0x7f88c015b050 2023-10-06T03:34:42.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.622+0000 7f88c559b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cc8d0 0x7f88c00a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:42.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.622+0000 7f88c559b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cc8d0 0x7f88c00a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44282/0 (socket says 172.21.15.55:44282) 2023-10-06T03:34:42.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.622+0000 7f88c559b700 1 -- 172.21.15.55:0/1122067325 learned_addr learned my addr 172.21.15.55:0/1122067325 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:42.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.623+0000 7f88c4d9a700 1 --2- 172.21.15.55:0/1122067325 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c015b050 0x7f88c015e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:42.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.623+0000 7f88c5d9c700 1 --2- 172.21.15.55:0/1122067325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c015b960 0x7f88c0149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.623+0000 7f88c559b700 1 -- 172.21.15.55:0/1122067325 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c015b050 msgr2=0x7f88c015e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.623+0000 7f88c559b700 1 --2- 172.21.15.55:0/1122067325 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c015b050 0x7f88c015e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.623+0000 7f88c559b700 1 -- 172.21.15.55:0/1122067325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c015b960 msgr2=0x7f88c0149a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.623+0000 7f88c559b700 1 --2- 172.21.15.55:0/1122067325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c015b960 0x7f88c0149a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.623+0000 7f88c559b700 1 -- 172.21.15.55:0/1122067325 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b8009ae0 con 0x7f88c00cc8d0 2023-10-06T03:34:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.623+0000 7f88c559b700 1 --2- 172.21.15.55:0/1122067325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cc8d0 0x7f88c00a54c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00c4e0 tx=0x7f88bc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.623+0000 7f88b57fa700 1 -- 172.21.15.55:0/1122067325 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc011590 con 0x7f88c00cc8d0 2023-10-06T03:34:42.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.623+0000 7f88b57fa700 1 -- 172.21.15.55:0/1122067325 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc011710 con 0x7f88c00cc8d0 2023-10-06T03:34:42.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.623+0000 7f88c659d700 1 -- 172.21.15.55:0/1122067325 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c016d780 con 0x7f88c00cc8d0 2023-10-06T03:34:42.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.623+0000 7f88b57fa700 1 -- 172.21.15.55:0/1122067325 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc011ad0 con 0x7f88c00cc8d0 2023-10-06T03:34:42.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.624+0000 7f88c659d700 1 -- 172.21.15.55:0/1122067325 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c016ce40 con 0x7f88c00cc8d0 2023-10-06T03:34:42.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.624+0000 7f88c659d700 1 -- 172.21.15.55:0/1122067325 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c004c3c0 con 0x7f88c00cc8d0 2023-10-06T03:34:42.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.625+0000 7f88b57fa700 1 -- 172.21.15.55:0/1122067325 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f88bc01b3b0 con 0x7f88c00cc8d0 2023-10-06T03:34:42.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.626+0000 7f88b57fa700 1 --2- 172.21.15.55:0/1122067325 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f88a4067130 0x7f88a40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:42.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.626+0000 7f88b57fa700 1 -- 172.21.15.55:0/1122067325 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f88bc083fb0 con 0x7f88c00cc8d0 2023-10-06T03:34:42.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.627+0000 7f88c4d9a700 1 --2- 172.21.15.55:0/1122067325 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f88a4067130 0x7f88a40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:42.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.627+0000 7f88c4d9a700 1 --2- 172.21.15.55:0/1122067325 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f88a4067130 0x7f88a40695e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f88b0006f00 tx=0x7f88b0008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:42.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.630+0000 7f88b57fa700 1 -- 172.21.15.55:0/1122067325 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f88bc050ca0 con 0x7f88c00cc8d0 2023-10-06T03:34:42.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.862+0000 7f88c659d700 1 -- 172.21.15.55:0/1122067325 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f88c00cba20 con 0x7f88a4067130 2023-10-06T03:34:42.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.929+0000 7f88b57fa700 1 -- 172.21.15.55:0/1122067325 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f88c00cba20 con 0x7f88a4067130 2023-10-06T03:34:42.931 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-10-06T03:34:42.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.931+0000 7f88c659d700 1 -- 172.21.15.55:0/1122067325 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f88a4067130 msgr2=0x7f88a40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:42.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.931+0000 7f88c659d700 1 --2- 172.21.15.55:0/1122067325 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f88a4067130 0x7f88a40695e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f88b0006f00 tx=0x7f88b0008040 comp rx=0 tx=0).stop 2023-10-06T03:34:42.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.931+0000 7f88c659d700 1 -- 172.21.15.55:0/1122067325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cc8d0 msgr2=0x7f88c00a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:42.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.931+0000 7f88c659d700 1 --2- 172.21.15.55:0/1122067325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cc8d0 0x7f88c00a54c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00c4e0 tx=0x7f88bc00c7f0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.931+0000 7f88c659d700 1 -- 172.21.15.55:0/1122067325 shutdown_connections 2023-10-06T03:34:42.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.931+0000 7f88c659d700 1 --2- 172.21.15.55:0/1122067325 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c015b960 0x7f88c0149a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.931+0000 7f88c659d700 1 --2- 172.21.15.55:0/1122067325 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c00cc8d0 0x7f88c00a54c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.931+0000 7f88c659d700 1 --2- 172.21.15.55:0/1122067325 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f88a4067130 0x7f88a40695e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.931+0000 7f88c659d700 1 --2- 172.21.15.55:0/1122067325 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c015b050 0x7f88c015e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:42.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.931+0000 7f88c659d700 1 -- 172.21.15.55:0/1122067325 >> 172.21.15.55:0/1122067325 conn(0x7f88c0068e20 msgr2=0x7f88c00b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:42.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.931+0000 7f88c659d700 1 -- 172.21.15.55:0/1122067325 shutdown_connections 2023-10-06T03:34:42.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:42.931+0000 7f88c659d700 1 -- 172.21.15.55:0/1122067325 wait complete. 2023-10-06T03:34:42.950 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-10-06T03:34:42.951 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir5 1M 2023-10-06T03:34:43.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.141+0000 7f5febfff700 1 Processor -- start 2023-10-06T03:34:43.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.142+0000 7f5febfff700 1 -- start start 2023-10-06T03:34:43.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.143+0000 7f5febfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fec153850 0x7f5fec1596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:43.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.143+0000 7f5febfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5fec159bf0 0x7f5fec157280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:43.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.143+0000 7f5febfff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fec15a050 con 0x7f5fec153480 2023-10-06T03:34:43.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.143+0000 7f5febfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fec15a480 con 0x7f5fec159bf0 2023-10-06T03:34:43.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.143+0000 7f5febfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0afb40 con 0x7f5fec153850 2023-10-06T03:34:43.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.143+0000 7f5feb7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5fec159bf0 0x7f5fec157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:43.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.143+0000 7f5feb7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5fec159bf0 0x7f5fec157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39784/0 (socket says 172.21.15.55:39784) 2023-10-06T03:34:43.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.143+0000 7f5feb7fe700 1 -- 172.21.15.55:0/1183590850 learned_addr learned my addr 172.21.15.55:0/1183590850 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:43.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.143+0000 7f5fea7fc700 1 --2- 172.21.15.55:0/1183590850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fec153850 0x7f5fec1596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:43.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.144+0000 7f5feb7fe700 1 -- 172.21.15.55:0/1183590850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fec153850 msgr2=0x7f5fec1596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:43.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.144+0000 7f5feb7fe700 1 --2- 172.21.15.55:0/1183590850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fec153850 0x7f5fec1596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.144+0000 7f5feb7fe700 1 -- 172.21.15.55:0/1183590850 >> v1:172.21.15.163:6789/0 conn(0x7f5fec153480 legacy=0x7f5fec0ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:43.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.144+0000 7f5feb7fe700 1 -- 172.21.15.55:0/1183590850 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec0afe20 con 0x7f5fec159bf0 2023-10-06T03:34:43.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.144+0000 7f5fea7fc700 1 --2- 172.21.15.55:0/1183590850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fec153850 0x7f5fec1596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:43.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.144+0000 7f5feb7fe700 1 --2- 172.21.15.55:0/1183590850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fec159bf0 0x7f5fec157280 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009f20 tx=0x7f5fe4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275a922ab8faab73 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:43.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.144+0000 7f5fe9ffb700 1 -- 172.21.15.55:0/1183590850 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe400e070 con 0x7f5fec159bf0 2023-10-06T03:34:43.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.144+0000 7f5fe9ffb700 1 -- 172.21.15.55:0/1183590850 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe400ba80 con 0x7f5fec159bf0 2023-10-06T03:34:43.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.144+0000 7f5fe9ffb700 1 -- 172.21.15.55:0/1183590850 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40092e0 con 0x7f5fec159bf0 2023-10-06T03:34:43.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.145+0000 7f5febfff700 1 -- 172.21.15.55:0/1183590850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fec159bf0 msgr2=0x7f5fec157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:43.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.145+0000 7f5febfff700 1 --2- 172.21.15.55:0/1183590850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fec159bf0 0x7f5fec157280 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009f20 tx=0x7f5fe4009840 comp rx=0 tx=0).stop 2023-10-06T03:34:43.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.145+0000 7f5febfff700 1 -- 172.21.15.55:0/1183590850 shutdown_connections 2023-10-06T03:34:43.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.145+0000 7f5febfff700 1 --2- 172.21.15.55:0/1183590850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fec159bf0 0x7f5fec157280 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.145+0000 7f5febfff700 1 --2- 172.21.15.55:0/1183590850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fec153850 0x7f5fec1596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.145+0000 7f5febfff700 1 -- 172.21.15.55:0/1183590850 >> 172.21.15.55:0/1183590850 conn(0x7f5fec068e20 msgr2=0x7f5fec0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:43.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.145+0000 7f5febfff700 1 -- 172.21.15.55:0/1183590850 shutdown_connections 2023-10-06T03:34:43.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.145+0000 7f5febfff700 1 -- 172.21.15.55:0/1183590850 wait complete. 2023-10-06T03:34:43.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.146+0000 7f5febfff700 1 Processor -- start 2023-10-06T03:34:43.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.146+0000 7f5febfff700 1 -- start start 2023-10-06T03:34:43.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.147+0000 7f5febfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fec153480 0x7f5fec140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:43.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.147+0000 7f5febfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fec153850 0x7f5fec0c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:43.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.147+0000 7f5febfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fec159bf0 0x7f5fec0c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:43.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.147+0000 7f5febfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0cb5b0 con 0x7f5fec159bf0 2023-10-06T03:34:43.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.147+0000 7f5febfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec154f00 con 0x7f5fec153480 2023-10-06T03:34:43.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.147+0000 7f5febfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec1545d0 con 0x7f5fec153850 2023-10-06T03:34:43.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.148+0000 7f5feb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fec159bf0 0x7f5fec0c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:43.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.148+0000 7f5feb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fec159bf0 0x7f5fec0c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39794/0 (socket says 172.21.15.55:39794) 2023-10-06T03:34:43.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.148+0000 7f5feb7fe700 1 -- 172.21.15.55:0/2158843891 learned_addr learned my addr 172.21.15.55:0/2158843891 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:43.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.148+0000 7f5fea7fc700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fec153850 0x7f5fec0c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:43.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.148+0000 7f5feaffd700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fec153480 0x7f5fec140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:43.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.148+0000 7f5feb7fe700 1 -- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fec153850 msgr2=0x7f5fec0c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:43.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.148+0000 7f5feb7fe700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fec153850 0x7f5fec0c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.148+0000 7f5feb7fe700 1 -- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fec153480 msgr2=0x7f5fec140580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:43.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.148+0000 7f5feb7fe700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fec153480 0x7f5fec140580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.148+0000 7f5feb7fe700 1 -- 172.21.15.55:0/2158843891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe4009ae0 con 0x7f5fec159bf0 2023-10-06T03:34:43.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.148+0000 7f5feaffd700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fec153480 0x7f5fec140580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:43.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.148+0000 7f5feb7fe700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fec159bf0 0x7f5fec0c59c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400b6c0 tx=0x7f5fe4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:43.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.148+0000 7f5fd2ffd700 1 -- 172.21.15.55:0/2158843891 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40032f0 con 0x7f5fec159bf0 2023-10-06T03:34:43.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.149+0000 7f5fd2ffd700 1 -- 172.21.15.55:0/2158843891 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe4008e40 con 0x7f5fec159bf0 2023-10-06T03:34:43.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.149+0000 7f5fd2ffd700 1 -- 172.21.15.55:0/2158843891 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4025690 con 0x7f5fec159bf0 2023-10-06T03:34:43.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.149+0000 7f5febfff700 1 -- 172.21.15.55:0/2158843891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fec1548b0 con 0x7f5fec159bf0 2023-10-06T03:34:43.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.149+0000 7f5febfff700 1 -- 172.21.15.55:0/2158843891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fec140ac0 con 0x7f5fec159bf0 2023-10-06T03:34:43.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.150+0000 7f5febfff700 1 -- 172.21.15.55:0/2158843891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fec140ce0 con 0x7f5fec159bf0 2023-10-06T03:34:43.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.150+0000 7f5fd2ffd700 1 -- 172.21.15.55:0/2158843891 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f5fe4003040 con 0x7f5fec159bf0 2023-10-06T03:34:43.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.152+0000 7f5fd2ffd700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5fd8067130 0x7f5fd80695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.152+0000 7f5fd2ffd700 1 -- 172.21.15.55:0/2158843891 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f5fe40869c0 con 0x7f5fec159bf0 2023-10-06T03:34:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.152+0000 7f5feaffd700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5fd8067130 0x7f5fd80695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.153+0000 7f5feaffd700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5fd8067130 0x7f5fd80695e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0adb80 tx=0x7f5fdc009040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:43.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.156+0000 7f5fd2ffd700 1 -- 172.21.15.55:0/2158843891 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fe4053650 con 0x7f5fec159bf0 2023-10-06T03:34:43.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.386+0000 7f5febfff700 1 -- 172.21.15.55:0/2158843891 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fec04c3c0 con 0x7f5fd8067130 2023-10-06T03:34:43.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.451+0000 7f5fd2ffd700 1 -- 172.21.15.55:0/2158843891 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f5fec04c3c0 con 0x7f5fd8067130 2023-10-06T03:34:43.453 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-10-06T03:34:43.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.453+0000 7f5febfff700 1 -- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5fd8067130 msgr2=0x7f5fd80695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:43.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.453+0000 7f5febfff700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5fd8067130 0x7f5fd80695e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0adb80 tx=0x7f5fdc009040 comp rx=0 tx=0).stop 2023-10-06T03:34:43.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.453+0000 7f5febfff700 1 -- 172.21.15.55:0/2158843891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fec159bf0 msgr2=0x7f5fec0c59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:43.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.453+0000 7f5febfff700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fec159bf0 0x7f5fec0c59c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400b6c0 tx=0x7f5fe4018000 comp rx=0 tx=0).stop 2023-10-06T03:34:43.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.453+0000 7f5febfff700 1 -- 172.21.15.55:0/2158843891 shutdown_connections 2023-10-06T03:34:43.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.453+0000 7f5febfff700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fec159bf0 0x7f5fec0c59c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.453+0000 7f5febfff700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fec153480 0x7f5fec140580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.453+0000 7f5febfff700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5fd8067130 0x7f5fd80695e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.453+0000 7f5febfff700 1 --2- 172.21.15.55:0/2158843891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fec153850 0x7f5fec0c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.453+0000 7f5febfff700 1 -- 172.21.15.55:0/2158843891 >> 172.21.15.55:0/2158843891 conn(0x7f5fec068e20 msgr2=0x7f5fec06ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.454+0000 7f5febfff700 1 -- 172.21.15.55:0/2158843891 shutdown_connections 2023-10-06T03:34:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.454+0000 7f5febfff700 1 -- 172.21.15.55:0/2158843891 wait complete. 2023-10-06T03:34:43.472 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-10-06T03:34:43.473 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir6 1M 2023-10-06T03:34:43.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.665+0000 7f2274b9d700 1 Processor -- start 2023-10-06T03:34:43.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.665+0000 7f2274b9d700 1 -- start start 2023-10-06T03:34:43.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.666+0000 7f2274b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22700ccb40 0x7f22700697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:43.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.666+0000 7f2274b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2270159db0 con 0x7f2270069cf0 2023-10-06T03:34:43.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.666+0000 7f2274b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227006a470 con 0x7f22700cb560 2023-10-06T03:34:43.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.666+0000 7f2274b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227006aa30 con 0x7f22700ccb40 2023-10-06T03:34:43.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.666+0000 7f226effd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22700ccb40 0x7f22700697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:43.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.666+0000 7f226ffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2270069cf0 0x7f227015ba60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:54270/0 (socket says 172.21.15.55:54270) 2023-10-06T03:34:43.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.666+0000 7f226ffff700 1 -- 172.21.15.55:0/1174692083 learned_addr learned my addr 172.21.15.55:0/1174692083 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:34:43.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.667+0000 7f226effd700 1 -- 172.21.15.55:0/1174692083 >> v1:172.21.15.163:6789/0 conn(0x7f22700cb560 legacy=0x7f22700cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:43.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.667+0000 7f226effd700 1 -- 172.21.15.55:0/1174692083 >> v1:172.21.15.55:6789/0 conn(0x7f2270069cf0 legacy=0x7f227015ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:43.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.667+0000 7f226effd700 1 -- 172.21.15.55:0/1174692083 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227006a750 con 0x7f22700ccb40 2023-10-06T03:34:43.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.667+0000 7f226e7fc700 1 -- 172.21.15.55:0/1174692083 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228226028 0 0) 0x7f2270159db0 con 0x7f2270069cf0 2023-10-06T03:34:43.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.667+0000 7f226effd700 1 --2- 172.21.15.55:0/1174692083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22700ccb40 0x7f22700697b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2264009910 tx=0x7f22640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382298b9ee2810a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:43.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.667+0000 7f226e7fc700 1 -- 172.21.15.55:0/1174692083 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400b820 con 0x7f22700ccb40 2023-10-06T03:34:43.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.667+0000 7f226e7fc700 1 -- 172.21.15.55:0/1174692083 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226400b980 con 0x7f22700ccb40 2023-10-06T03:34:43.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.667+0000 7f226e7fc700 1 -- 172.21.15.55:0/1174692083 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400bd20 con 0x7f22700ccb40 2023-10-06T03:34:43.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.668+0000 7f2274b9d700 1 -- 172.21.15.55:0/1174692083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22700ccb40 msgr2=0x7f22700697b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:43.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.668+0000 7f2274b9d700 1 --2- 172.21.15.55:0/1174692083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22700ccb40 0x7f22700697b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2264009910 tx=0x7f22640091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.668+0000 7f2274b9d700 1 -- 172.21.15.55:0/1174692083 shutdown_connections 2023-10-06T03:34:43.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.668+0000 7f2274b9d700 1 --2- 172.21.15.55:0/1174692083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22700ccb40 0x7f22700697b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.668+0000 7f2274b9d700 1 -- 172.21.15.55:0/1174692083 >> 172.21.15.55:0/1174692083 conn(0x7f227006baf0 msgr2=0x7f227006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:43.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.668+0000 7f2274b9d700 1 -- 172.21.15.55:0/1174692083 shutdown_connections 2023-10-06T03:34:43.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.668+0000 7f2274b9d700 1 -- 172.21.15.55:0/1174692083 wait complete. 2023-10-06T03:34:43.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.669+0000 7f2274b9d700 1 Processor -- start 2023-10-06T03:34:43.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.670+0000 7f2274b9d700 1 -- start start 2023-10-06T03:34:43.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.670+0000 7f2274b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2270069cf0 0x7f22701585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:43.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.670+0000 7f2274b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22700cb560 0x7f22700ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:43.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.670+0000 7f2274b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22700ccb40 0x7f22700afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:43.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.671+0000 7f2274b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22700af990 con 0x7f2270069cf0 2023-10-06T03:34:43.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.671+0000 7f2274b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22700b1620 con 0x7f22700ccb40 2023-10-06T03:34:43.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.671+0000 7f2274b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22700af0f0 con 0x7f22700cb560 2023-10-06T03:34:43.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.671+0000 7f226f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2270069cf0 0x7f22701585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.671+0000 7f226f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2270069cf0 0x7f22701585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39814/0 (socket says 172.21.15.55:39814) 2023-10-06T03:34:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.671+0000 7f226f7fe700 1 -- 172.21.15.55:0/2741682077 learned_addr learned my addr 172.21.15.55:0/2741682077 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.671+0000 7f226effd700 1 --2- 172.21.15.55:0/2741682077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22700cb560 0x7f22700ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.671+0000 7f226ffff700 1 --2- 172.21.15.55:0/2741682077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22700ccb40 0x7f22700afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.671+0000 7f226f7fe700 1 -- 172.21.15.55:0/2741682077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22700cb560 msgr2=0x7f22700ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.671+0000 7f226f7fe700 1 --2- 172.21.15.55:0/2741682077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22700cb560 0x7f22700ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.671+0000 7f226f7fe700 1 -- 172.21.15.55:0/2741682077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22700ccb40 msgr2=0x7f22700afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:43.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.671+0000 7f226f7fe700 1 --2- 172.21.15.55:0/2741682077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22700ccb40 0x7f22700afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.671+0000 7f226f7fe700 1 -- 172.21.15.55:0/2741682077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264009420 con 0x7f2270069cf0 2023-10-06T03:34:43.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.671+0000 7f226f7fe700 1 --2- 172.21.15.55:0/2741682077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2270069cf0 0x7f22701585b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f226000c590 tx=0x7f226000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:43.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.672+0000 7f22577fe700 1 -- 172.21.15.55:0/2741682077 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2260011590 con 0x7f2270069cf0 2023-10-06T03:34:43.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.672+0000 7f22577fe700 1 -- 172.21.15.55:0/2741682077 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22600116f0 con 0x7f2270069cf0 2023-10-06T03:34:43.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.672+0000 7f22577fe700 1 -- 172.21.15.55:0/2741682077 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2260011ab0 con 0x7f2270069cf0 2023-10-06T03:34:43.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.672+0000 7f2274b9d700 1 -- 172.21.15.55:0/2741682077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22700af3d0 con 0x7f2270069cf0 2023-10-06T03:34:43.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.672+0000 7f2274b9d700 1 -- 172.21.15.55:0/2741682077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22700aed90 con 0x7f2270069cf0 2023-10-06T03:34:43.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.673+0000 7f2274b9d700 1 -- 172.21.15.55:0/2741682077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22700b13f0 con 0x7f2270069cf0 2023-10-06T03:34:43.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.674+0000 7f22577fe700 1 -- 172.21.15.55:0/2741682077 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f226001b3b0 con 0x7f2270069cf0 2023-10-06T03:34:43.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.675+0000 7f22577fe700 1 --2- 172.21.15.55:0/2741682077 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f225c067130 0x7f225c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:43.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.675+0000 7f22577fe700 1 -- 172.21.15.55:0/2741682077 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f2260084050 con 0x7f2270069cf0 2023-10-06T03:34:43.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.675+0000 7f226effd700 1 --2- 172.21.15.55:0/2741682077 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f225c067130 0x7f225c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:43.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.676+0000 7f226effd700 1 --2- 172.21.15.55:0/2741682077 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f225c067130 0x7f225c0695e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f22700ccf40 tx=0x7f2264017000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:43.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.680+0000 7f22577fe700 1 -- 172.21.15.55:0/2741682077 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22700b13f0 con 0x7f2270069cf0 2023-10-06T03:34:43.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.910+0000 7f2274b9d700 1 -- 172.21.15.55:0/2741682077 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f22700b1760 con 0x7f225c067130 2023-10-06T03:34:43.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.977+0000 7f22577fe700 1 -- 172.21.15.55:0/2741682077 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f226400bed0 con 0x7f225c067130 2023-10-06T03:34:43.979 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-10-06T03:34:43.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.980+0000 7f2274b9d700 1 -- 172.21.15.55:0/2741682077 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f225c067130 msgr2=0x7f225c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:43.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.980+0000 7f2274b9d700 1 --2- 172.21.15.55:0/2741682077 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f225c067130 0x7f225c0695e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f22700ccf40 tx=0x7f2264017000 comp rx=0 tx=0).stop 2023-10-06T03:34:43.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.980+0000 7f2274b9d700 1 -- 172.21.15.55:0/2741682077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2270069cf0 msgr2=0x7f22701585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:43.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.980+0000 7f2274b9d700 1 --2- 172.21.15.55:0/2741682077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2270069cf0 0x7f22701585b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f226000c590 tx=0x7f226000c950 comp rx=0 tx=0).stop 2023-10-06T03:34:43.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.980+0000 7f2274b9d700 1 -- 172.21.15.55:0/2741682077 shutdown_connections 2023-10-06T03:34:43.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.980+0000 7f2274b9d700 1 --2- 172.21.15.55:0/2741682077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22700ccb40 0x7f22700afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.980+0000 7f2274b9d700 1 --2- 172.21.15.55:0/2741682077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2270069cf0 0x7f22701585b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.980+0000 7f2274b9d700 1 --2- 172.21.15.55:0/2741682077 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f225c067130 0x7f225c0695e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.980+0000 7f2274b9d700 1 --2- 172.21.15.55:0/2741682077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22700cb560 0x7f22700ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:43.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.980+0000 7f2274b9d700 1 -- 172.21.15.55:0/2741682077 >> 172.21.15.55:0/2741682077 conn(0x7f227006baf0 msgr2=0x7f22700c9470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:43.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.980+0000 7f2274b9d700 1 -- 172.21.15.55:0/2741682077 shutdown_connections 2023-10-06T03:34:43.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:43.980+0000 7f2274b9d700 1 -- 172.21.15.55:0/2741682077 wait complete. 2023-10-06T03:34:43.999 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-10-06T03:34:43.999 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir7 1M 2023-10-06T03:34:44.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.189+0000 7f8b3d19d700 1 Processor -- start 2023-10-06T03:34:44.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.189+0000 7f8b3d19d700 1 -- start start 2023-10-06T03:34:44.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.190+0000 7f8b3d19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8b380ae180 0x7f8b380acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:44.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.190+0000 7f8b3d19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b380656e0 0x7f8b380ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:44.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.190+0000 7f8b3d19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b3806b290 0x7f8b381594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:44.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.191+0000 7f8b3d19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b380aedf0 con 0x7f8b3806b290 2023-10-06T03:34:44.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.191+0000 7f8b3d19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b380aeb10 con 0x7f8b380ae180 2023-10-06T03:34:44.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.191+0000 7f8b3d19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b380af540 con 0x7f8b380656e0 2023-10-06T03:34:44.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.191+0000 7f8b3c99c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b3806b290 0x7f8b381594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:44.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.191+0000 7f8b3c99c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b3806b290 0x7f8b381594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39828/0 (socket says 172.21.15.55:39828) 2023-10-06T03:34:44.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.191+0000 7f8b3c99c700 1 -- 172.21.15.55:0/3756139115 learned_addr learned my addr 172.21.15.55:0/3756139115 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:44.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.191+0000 7f8b3c99c700 1 -- 172.21.15.55:0/3756139115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b380656e0 msgr2=0x7f8b380ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:44.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.191+0000 7f8b3c99c700 1 --2- 172.21.15.55:0/3756139115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b380656e0 0x7f8b380ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.191+0000 7f8b3c99c700 1 -- 172.21.15.55:0/3756139115 >> v2:172.21.15.163:3300/0 conn(0x7f8b380ae180 msgr2=0x7f8b380acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:44.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.191+0000 7f8b3c99c700 1 --2- 172.21.15.55:0/3756139115 >> v2:172.21.15.163:3300/0 conn(0x7f8b380ae180 0x7f8b380acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.191+0000 7f8b3c99c700 1 -- 172.21.15.55:0/3756139115 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b380af820 con 0x7f8b3806b290 2023-10-06T03:34:44.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.192+0000 7f8b3c99c700 1 --2- 172.21.15.55:0/3756139115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b3806b290 0x7f8b381594b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8b38067820 tx=0x7f8b30009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c516b58f9922a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:44.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.192+0000 7f8b36ffd700 1 -- 172.21.15.55:0/3756139115 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3000e070 con 0x7f8b3806b290 2023-10-06T03:34:44.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.192+0000 7f8b36ffd700 1 -- 172.21.15.55:0/3756139115 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3000ba50 con 0x7f8b3806b290 2023-10-06T03:34:44.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.192+0000 7f8b36ffd700 1 -- 172.21.15.55:0/3756139115 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b300092e0 con 0x7f8b3806b290 2023-10-06T03:34:44.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.192+0000 7f8b3d19d700 1 -- 172.21.15.55:0/3756139115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b3806b290 msgr2=0x7f8b381594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:44.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.192+0000 7f8b3d19d700 1 --2- 172.21.15.55:0/3756139115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b3806b290 0x7f8b381594b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8b38067820 tx=0x7f8b30009840 comp rx=0 tx=0).stop 2023-10-06T03:34:44.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.192+0000 7f8b3d19d700 1 -- 172.21.15.55:0/3756139115 shutdown_connections 2023-10-06T03:34:44.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.192+0000 7f8b3d19d700 1 --2- 172.21.15.55:0/3756139115 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b3806b290 0x7f8b381594b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.192+0000 7f8b3d19d700 1 --2- 172.21.15.55:0/3756139115 >> v2:172.21.15.163:3300/0 conn(0x7f8b380ae180 0x7f8b380acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.192+0000 7f8b3d19d700 1 --2- 172.21.15.55:0/3756139115 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b380656e0 0x7f8b380ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.192+0000 7f8b3d19d700 1 -- 172.21.15.55:0/3756139115 >> 172.21.15.55:0/3756139115 conn(0x7f8b3806f6a0 msgr2=0x7f8b3806ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:44.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.192+0000 7f8b3d19d700 1 -- 172.21.15.55:0/3756139115 shutdown_connections 2023-10-06T03:34:44.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.193+0000 7f8b3d19d700 1 -- 172.21.15.55:0/3756139115 wait complete. 2023-10-06T03:34:44.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.194+0000 7f8b3d19d700 1 Processor -- start 2023-10-06T03:34:44.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.194+0000 7f8b3d19d700 1 -- start start 2023-10-06T03:34:44.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.195+0000 7f8b3d19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b380656e0 0x7f8b380cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:44.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.195+0000 7f8b3d19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b3806b290 0x7f8b380c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:44.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.195+0000 7f8b3d19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b380ae180 0x7f8b3816cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:44.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.195+0000 7f8b3d19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b380c5de0 con 0x7f8b3806b290 2023-10-06T03:34:44.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.195+0000 7f8b3d19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b380c60c0 con 0x7f8b380656e0 2023-10-06T03:34:44.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.195+0000 7f8b3d19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b380c63a0 con 0x7f8b380ae180 2023-10-06T03:34:44.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.195+0000 7f8b377fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b3806b290 0x7f8b380c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:44.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.195+0000 7f8b377fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b3806b290 0x7f8b380c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39840/0 (socket says 172.21.15.55:39840) 2023-10-06T03:34:44.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.195+0000 7f8b377fe700 1 -- 172.21.15.55:0/875583480 learned_addr learned my addr 172.21.15.55:0/875583480 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:44.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.195+0000 7f8b3c99c700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b380ae180 0x7f8b3816cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:44.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.196+0000 7f8b377fe700 1 -- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b380ae180 msgr2=0x7f8b3816cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:44.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.196+0000 7f8b377fe700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b380ae180 0x7f8b3816cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.196+0000 7f8b377fe700 1 -- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b380656e0 msgr2=0x7f8b380cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:44.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.196+0000 7f8b37fff700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b380656e0 0x7f8b380cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:44.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.196+0000 7f8b377fe700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b380656e0 0x7f8b380cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.196+0000 7f8b377fe700 1 -- 172.21.15.55:0/875583480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b30009ae0 con 0x7f8b3806b290 2023-10-06T03:34:44.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.196+0000 7f8b37fff700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b380656e0 0x7f8b380cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:34:44.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.196+0000 7f8b377fe700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b3806b290 0x7f8b380c8150 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00b7e0 tx=0x7f8b2c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:44.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.196+0000 7f8b1ffff700 1 -- 172.21.15.55:0/875583480 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c00f570 con 0x7f8b3806b290 2023-10-06T03:34:44.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.196+0000 7f8b1ffff700 1 -- 172.21.15.55:0/875583480 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c00f6d0 con 0x7f8b3806b290 2023-10-06T03:34:44.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.196+0000 7f8b1ffff700 1 -- 172.21.15.55:0/875583480 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c00fa90 con 0x7f8b3806b290 2023-10-06T03:34:44.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.196+0000 7f8b3d19d700 1 -- 172.21.15.55:0/875583480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b380c6680 con 0x7f8b3806b290 2023-10-06T03:34:44.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.196+0000 7f8b3d19d700 1 -- 172.21.15.55:0/875583480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b380c8690 con 0x7f8b3806b290 2023-10-06T03:34:44.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.197+0000 7f8b3d19d700 1 -- 172.21.15.55:0/875583480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3804c3c0 con 0x7f8b3806b290 2023-10-06T03:34:44.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.199+0000 7f8b1ffff700 1 -- 172.21.15.55:0/875583480 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f8b2c01a460 con 0x7f8b3806b290 2023-10-06T03:34:44.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.199+0000 7f8b1ffff700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b24067130 0x7f8b240695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:44.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.199+0000 7f8b1ffff700 1 -- 172.21.15.55:0/875583480 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7f8b2c084900 con 0x7f8b3806b290 2023-10-06T03:34:44.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.200+0000 7f8b37fff700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b24067130 0x7f8b240695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:44.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.201+0000 7f8b37fff700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b24067130 0x7f8b240695e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8b28006fd0 tx=0x7f8b28008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:44.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.203+0000 7f8b1ffff700 1 -- 172.21.15.55:0/875583480 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b2c0515f0 con 0x7f8b3806b290 2023-10-06T03:34:44.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.433+0000 7f8b3d19d700 1 -- 172.21.15.55:0/875583480 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b380c6b20 con 0x7f8b24067130 2023-10-06T03:34:44.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.495+0000 7f8b1ffff700 1 -- 172.21.15.55:0/875583480 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8b380c6b20 con 0x7f8b24067130 2023-10-06T03:34:44.497 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-10-06T03:34:44.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.497+0000 7f8b3d19d700 1 -- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b24067130 msgr2=0x7f8b240695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:44.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.497+0000 7f8b3d19d700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b24067130 0x7f8b240695e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8b28006fd0 tx=0x7f8b28008040 comp rx=0 tx=0).stop 2023-10-06T03:34:44.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.497+0000 7f8b3d19d700 1 -- 172.21.15.55:0/875583480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b3806b290 msgr2=0x7f8b380c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:44.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.497+0000 7f8b3d19d700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b3806b290 0x7f8b380c8150 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00b7e0 tx=0x7f8b2c009220 comp rx=0 tx=0).stop 2023-10-06T03:34:44.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.497+0000 7f8b3d19d700 1 -- 172.21.15.55:0/875583480 shutdown_connections 2023-10-06T03:34:44.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.497+0000 7f8b3d19d700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b24067130 0x7f8b240695e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.497+0000 7f8b3d19d700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b380ae180 0x7f8b3816cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.497+0000 7f8b3d19d700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b380656e0 0x7f8b380cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.497+0000 7f8b3d19d700 1 --2- 172.21.15.55:0/875583480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b3806b290 0x7f8b380c8150 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.497+0000 7f8b3d19d700 1 -- 172.21.15.55:0/875583480 >> 172.21.15.55:0/875583480 conn(0x7f8b3806f6a0 msgr2=0x7f8b38068b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:44.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.498+0000 7f8b3d19d700 1 -- 172.21.15.55:0/875583480 shutdown_connections 2023-10-06T03:34:44.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.498+0000 7f8b3d19d700 1 -- 172.21.15.55:0/875583480 wait complete. 2023-10-06T03:34:44.517 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-10-06T03:34:44.517 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir8 1M 2023-10-06T03:34:44.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.709+0000 7feaba19d700 1 Processor -- start 2023-10-06T03:34:44.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.709+0000 7feaba19d700 1 -- start start 2023-10-06T03:34:44.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.710+0000 7feaba19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feab4156350 0x7feab4157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:44.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.711+0000 7feaba19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feab40b0760 0x7feab40b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:44.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.711+0000 7feaba19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab40b0570 con 0x7feab4159f20 2023-10-06T03:34:44.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.711+0000 7feaba19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab40b1140 con 0x7feab4156350 2023-10-06T03:34:44.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.711+0000 7feaba19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab41578b0 con 0x7feab40b0760 2023-10-06T03:34:44.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.711+0000 7feab919b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feab4156350 0x7feab4157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:44.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.711+0000 7feab919b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feab4156350 0x7feab4157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39854/0 (socket says 172.21.15.55:39854) 2023-10-06T03:34:44.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.711+0000 7feab919b700 1 -- 172.21.15.55:0/2153725858 learned_addr learned my addr 172.21.15.55:0/2153725858 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:44.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.711+0000 7feab919b700 1 -- 172.21.15.55:0/2153725858 >> v2:172.21.15.163:3300/0 conn(0x7feab40b0760 msgr2=0x7feab40b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:44.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.711+0000 7feab919b700 1 --2- 172.21.15.55:0/2153725858 >> v2:172.21.15.163:3300/0 conn(0x7feab40b0760 0x7feab40b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.711+0000 7feab919b700 1 -- 172.21.15.55:0/2153725858 >> v1:172.21.15.163:6789/0 conn(0x7feab4159f20 legacy=0x7feab415a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:44.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.711+0000 7feab919b700 1 -- 172.21.15.55:0/2153725858 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab40af4d0 con 0x7feab4156350 2023-10-06T03:34:44.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.712+0000 7feab919b700 1 --2- 172.21.15.55:0/2153725858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab4156350 0x7feab4157370 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7feab40642a0 tx=0x7feab000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91ae4b224cfec1a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:44.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.712+0000 7feaabfff700 1 -- 172.21.15.55:0/2153725858 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab000dc10 con 0x7feab4156350 2023-10-06T03:34:44.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.712+0000 7feaabfff700 1 -- 172.21.15.55:0/2153725858 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab000dd70 con 0x7feab4156350 2023-10-06T03:34:44.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.712+0000 7feaabfff700 1 -- 172.21.15.55:0/2153725858 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab00106c0 con 0x7feab4156350 2023-10-06T03:34:44.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.712+0000 7feaba19d700 1 -- 172.21.15.55:0/2153725858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab4156350 msgr2=0x7feab4157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:44.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.712+0000 7feaba19d700 1 --2- 172.21.15.55:0/2153725858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab4156350 0x7feab4157370 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7feab40642a0 tx=0x7feab000b4f0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.712+0000 7feaba19d700 1 -- 172.21.15.55:0/2153725858 shutdown_connections 2023-10-06T03:34:44.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.712+0000 7feaba19d700 1 --2- 172.21.15.55:0/2153725858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab4156350 0x7feab4157370 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.712+0000 7feaba19d700 1 --2- 172.21.15.55:0/2153725858 >> v2:172.21.15.163:3300/0 conn(0x7feab40b0760 0x7feab40b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.712+0000 7feaba19d700 1 -- 172.21.15.55:0/2153725858 >> 172.21.15.55:0/2153725858 conn(0x7feab4068e20 msgr2=0x7feab40cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:44.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.712+0000 7feaba19d700 1 -- 172.21.15.55:0/2153725858 shutdown_connections 2023-10-06T03:34:44.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.712+0000 7feaba19d700 1 -- 172.21.15.55:0/2153725858 wait complete. 2023-10-06T03:34:44.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.714+0000 7feaba19d700 1 Processor -- start 2023-10-06T03:34:44.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.714+0000 7feaba19d700 1 -- start start 2023-10-06T03:34:44.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.715+0000 7feaba19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab40b0760 0x7feab414e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:44.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.715+0000 7feaba19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feab4156350 0x7feab413dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:44.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.715+0000 7feaba19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feab4159f20 0x7feab4150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:44.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.715+0000 7feaba19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab414e8b0 con 0x7feab40b0760 2023-10-06T03:34:44.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.715+0000 7feaba19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab4168de0 con 0x7feab4159f20 2023-10-06T03:34:44.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.715+0000 7feaba19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab4150800 con 0x7feab4156350 2023-10-06T03:34:44.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.715+0000 7feab919b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab40b0760 0x7feab414e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:44.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.715+0000 7feab919b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab40b0760 0x7feab414e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39860/0 (socket says 172.21.15.55:39860) 2023-10-06T03:34:44.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.715+0000 7feab919b700 1 -- 172.21.15.55:0/3408160531 learned_addr learned my addr 172.21.15.55:0/3408160531 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:44.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.715+0000 7feab999c700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feab4159f20 0x7feab4150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:44.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.715+0000 7feab899a700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feab4156350 0x7feab413dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:44.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.716+0000 7feab919b700 1 -- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feab4156350 msgr2=0x7feab413dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:44.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.716+0000 7feab919b700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feab4156350 0x7feab413dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.716+0000 7feab919b700 1 -- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feab4159f20 msgr2=0x7feab4150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:44.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.716+0000 7feab919b700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feab4159f20 0x7feab4150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:44.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.716+0000 7feab919b700 1 -- 172.21.15.55:0/3408160531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab000b790 con 0x7feab40b0760 2023-10-06T03:34:44.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.716+0000 7feab999c700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feab4159f20 0x7feab4150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:34:44.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.716+0000 7feab919b700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab40b0760 0x7feab414e020 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feab0003080 tx=0x7feab001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:44.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.716+0000 7feaa97fa700 1 -- 172.21.15.55:0/3408160531 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab0019580 con 0x7feab40b0760 2023-10-06T03:34:44.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.716+0000 7feaa97fa700 1 -- 172.21.15.55:0/3408160531 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab0019700 con 0x7feab40b0760 2023-10-06T03:34:44.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.716+0000 7feaa97fa700 1 -- 172.21.15.55:0/3408160531 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab0019ac0 con 0x7feab40b0760 2023-10-06T03:34:44.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.716+0000 7feaba19d700 1 -- 172.21.15.55:0/3408160531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab4150ae0 con 0x7feab40b0760 2023-10-06T03:34:44.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.716+0000 7feaba19d700 1 -- 172.21.15.55:0/3408160531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab4169030 con 0x7feab40b0760 2023-10-06T03:34:44.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.717+0000 7feaba19d700 1 -- 172.21.15.55:0/3408160531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab4159ae0 con 0x7feab40b0760 2023-10-06T03:34:44.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.718+0000 7feaa97fa700 1 -- 172.21.15.55:0/3408160531 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7feab0021070 con 0x7feab40b0760 2023-10-06T03:34:44.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.719+0000 7feaa97fa700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fea98067130 0x7fea980695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:44.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.719+0000 7feaa97fa700 1 -- 172.21.15.55:0/3408160531 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7feab0014070 con 0x7feab40b0760 2023-10-06T03:34:44.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.720+0000 7feab899a700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fea98067130 0x7fea980695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:44.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.720+0000 7feab899a700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fea98067130 0x7fea980695e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feab40ae550 tx=0x7feaa4008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:44.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.723+0000 7feaa97fa700 1 -- 172.21.15.55:0/3408160531 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feab0054f30 con 0x7feab40b0760 2023-10-06T03:34:44.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:44.954+0000 7feaba19d700 1 -- 172.21.15.55:0/3408160531 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7feab404c3c0 con 0x7fea98067130 2023-10-06T03:34:45.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.020+0000 7feaa97fa700 1 -- 172.21.15.55:0/3408160531 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7feab404c3c0 con 0x7fea98067130 2023-10-06T03:34:45.021 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-10-06T03:34:45.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.022+0000 7feaba19d700 1 -- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fea98067130 msgr2=0x7fea980695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:45.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.022+0000 7feaba19d700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fea98067130 0x7fea980695e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feab40ae550 tx=0x7feaa4008040 comp rx=0 tx=0).stop 2023-10-06T03:34:45.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.022+0000 7feaba19d700 1 -- 172.21.15.55:0/3408160531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab40b0760 msgr2=0x7feab414e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:45.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.022+0000 7feaba19d700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab40b0760 0x7feab414e020 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7feab0003080 tx=0x7feab001a000 comp rx=0 tx=0).stop 2023-10-06T03:34:45.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.022+0000 7feaba19d700 1 -- 172.21.15.55:0/3408160531 shutdown_connections 2023-10-06T03:34:45.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.022+0000 7feaba19d700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feab4159f20 0x7feab4150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:45.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.022+0000 7feaba19d700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feab40b0760 0x7feab414e020 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:45.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.022+0000 7feaba19d700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fea98067130 0x7fea980695e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:45.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.022+0000 7feaba19d700 1 --2- 172.21.15.55:0/3408160531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feab4156350 0x7feab413dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:45.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.022+0000 7feaba19d700 1 -- 172.21.15.55:0/3408160531 >> 172.21.15.55:0/3408160531 conn(0x7feab4068e20 msgr2=0x7feab41552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:45.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.022+0000 7feaba19d700 1 -- 172.21.15.55:0/3408160531 shutdown_connections 2023-10-06T03:34:45.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.022+0000 7feaba19d700 1 -- 172.21.15.55:0/3408160531 wait complete. 2023-10-06T03:34:45.041 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-10-06T03:34:45.041 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir9 1M 2023-10-06T03:34:45.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.233+0000 7fa78359e700 1 Processor -- start 2023-10-06T03:34:45.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.233+0000 7fa78359e700 1 -- start start 2023-10-06T03:34:45.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.235+0000 7fa78359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa78415b050 0x7fa7840ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:45.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.235+0000 7fa78359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa78415bb30 0x7fa7840ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:45.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.235+0000 7fa78359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa784157a50 con 0x7fa7840cc8d0 2023-10-06T03:34:45.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.235+0000 7fa78359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa784158f10 con 0x7fa78415bb30 2023-10-06T03:34:45.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.235+0000 7fa78359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7841591b0 con 0x7fa78415b050 2023-10-06T03:34:45.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.235+0000 7fa782d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa78415bb30 0x7fa7840ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:45.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.235+0000 7fa782d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa78415bb30 0x7fa7840ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39872/0 (socket says 172.21.15.55:39872) 2023-10-06T03:34:45.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.235+0000 7fa782d9d700 1 -- 172.21.15.55:0/2082237398 learned_addr learned my addr 172.21.15.55:0/2082237398 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:45.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.235+0000 7fa78259c700 1 --2- 172.21.15.55:0/2082237398 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa78415b050 0x7fa7840ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:45.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.235+0000 7fa782d9d700 1 -- 172.21.15.55:0/2082237398 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa78415b050 msgr2=0x7fa7840ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:34:45.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.235+0000 7fa782d9d700 1 --2- 172.21.15.55:0/2082237398 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa78415b050 0x7fa7840ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:45.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.235+0000 7fa782d9d700 1 -- 172.21.15.55:0/2082237398 >> v1:172.21.15.163:6789/0 conn(0x7fa7840cc8d0 legacy=0x7fa78415b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:45.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.236+0000 7fa782d9d700 1 -- 172.21.15.55:0/2082237398 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7840ca9e0 con 0x7fa78415bb30 2023-10-06T03:34:45.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.236+0000 7fa782d9d700 1 --2- 172.21.15.55:0/2082237398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa78415bb30 0x7fa7840ada30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009f20 tx=0x7fa77c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c516af9b948f418 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:45.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.236+0000 7fa78159a700 1 -- 172.21.15.55:0/2082237398 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c00e070 con 0x7fa78415bb30 2023-10-06T03:34:45.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.236+0000 7fa78159a700 1 -- 172.21.15.55:0/2082237398 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77c00b9d0 con 0x7fa78415bb30 2023-10-06T03:34:45.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.236+0000 7fa78159a700 1 -- 172.21.15.55:0/2082237398 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0092e0 con 0x7fa78415bb30 2023-10-06T03:34:45.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.237+0000 7fa78359e700 1 -- 172.21.15.55:0/2082237398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa78415bb30 msgr2=0x7fa7840ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:45.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.237+0000 7fa78359e700 1 --2- 172.21.15.55:0/2082237398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa78415bb30 0x7fa7840ada30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009f20 tx=0x7fa77c009840 comp rx=0 tx=0).stop 2023-10-06T03:34:45.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.237+0000 7fa78359e700 1 -- 172.21.15.55:0/2082237398 shutdown_connections 2023-10-06T03:34:45.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.237+0000 7fa78359e700 1 --2- 172.21.15.55:0/2082237398 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa78415bb30 0x7fa7840ada30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:45.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.237+0000 7fa78359e700 1 --2- 172.21.15.55:0/2082237398 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa78415b050 0x7fa7840ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:45.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.237+0000 7fa78359e700 1 -- 172.21.15.55:0/2082237398 >> 172.21.15.55:0/2082237398 conn(0x7fa784068e20 msgr2=0x7fa7840cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:45.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.237+0000 7fa78359e700 1 -- 172.21.15.55:0/2082237398 shutdown_connections 2023-10-06T03:34:45.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.237+0000 7fa78359e700 1 -- 172.21.15.55:0/2082237398 wait complete. 2023-10-06T03:34:45.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.238+0000 7fa78359e700 1 Processor -- start 2023-10-06T03:34:45.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.238+0000 7fa78359e700 1 -- start start 2023-10-06T03:34:45.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.239+0000 7fa78359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7840cc8d0 0x7fa784144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:45.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.239+0000 7fa78359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78415b050 0x7fa7840c2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:45.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.239+0000 7fa78359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa78415bb30 0x7fa784153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:45.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.239+0000 7fa78359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7840c6030 con 0x7fa7840cc8d0 2023-10-06T03:34:45.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.239+0000 7fa78359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7840c61c0 con 0x7fa78415b050 2023-10-06T03:34:45.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.239+0000 7fa78359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7840c2af0 con 0x7fa78415bb30 2023-10-06T03:34:45.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.239+0000 7fa78259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7840cc8d0 0x7fa784144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:45.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.239+0000 7fa78259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7840cc8d0 0x7fa784144bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39878/0 (socket says 172.21.15.55:39878) 2023-10-06T03:34:45.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.239+0000 7fa78259c700 1 -- 172.21.15.55:0/2369166287 learned_addr learned my addr 172.21.15.55:0/2369166287 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:34:45.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.240+0000 7fa781d9b700 1 --2- 172.21.15.55:0/2369166287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78415b050 0x7fa7840c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:45.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.240+0000 7fa782d9d700 1 --2- 172.21.15.55:0/2369166287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa78415bb30 0x7fa784153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:45.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.240+0000 7fa78259c700 1 -- 172.21.15.55:0/2369166287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa78415bb30 msgr2=0x7fa784153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:45.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.240+0000 7fa78259c700 1 --2- 172.21.15.55:0/2369166287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa78415bb30 0x7fa784153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:45.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.240+0000 7fa78259c700 1 -- 172.21.15.55:0/2369166287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78415b050 msgr2=0x7fa7840c2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:45.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.240+0000 7fa78259c700 1 --2- 172.21.15.55:0/2369166287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78415b050 0x7fa7840c2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:45.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.240+0000 7fa78259c700 1 -- 172.21.15.55:0/2369166287 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c009ae0 con 0x7fa7840cc8d0 2023-10-06T03:34:45.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.240+0000 7fa78259c700 1 --2- 172.21.15.55:0/2369166287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7840cc8d0 0x7fa784144bf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa77400ec80 tx=0x7fa7740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:45.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.240+0000 7fa7727fc700 1 -- 172.21.15.55:0/2369166287 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77400c9d0 con 0x7fa7840cc8d0 2023-10-06T03:34:45.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.240+0000 7fa7727fc700 1 -- 172.21.15.55:0/2369166287 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77400cb30 con 0x7fa7840cc8d0 2023-10-06T03:34:45.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.240+0000 7fa7727fc700 1 -- 172.21.15.55:0/2369166287 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774013470 con 0x7fa7840cc8d0 2023-10-06T03:34:45.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.240+0000 7fa78359e700 1 -- 172.21.15.55:0/2369166287 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7840c2dd0 con 0x7fa7840cc8d0 2023-10-06T03:34:45.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.241+0000 7fa78359e700 1 -- 172.21.15.55:0/2369166287 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7840c3390 con 0x7fa7840cc8d0 2023-10-06T03:34:45.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.241+0000 7fa78359e700 1 -- 172.21.15.55:0/2369166287 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7841451f0 con 0x7fa7840cc8d0 2023-10-06T03:34:45.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.242+0000 7fa7727fc700 1 -- 172.21.15.55:0/2369166287 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 9) v1 ==== 84916+0+0 (secure 0 0 0) 0x7fa774013820 con 0x7fa7840cc8d0 2023-10-06T03:34:45.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.244+0000 7fa7727fc700 1 --2- 172.21.15.55:0/2369166287 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa764067130 0x7fa7640695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:34:45.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.244+0000 7fa7727fc700 1 -- 172.21.15.55:0/2369166287 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6316+0+0 (secure 0 0 0) 0x7fa774084fa0 con 0x7fa7840cc8d0 2023-10-06T03:34:45.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.244+0000 7fa781d9b700 1 --2- 172.21.15.55:0/2369166287 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa764067130 0x7fa7640695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:34:45.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.245+0000 7fa781d9b700 1 --2- 172.21.15.55:0/2369166287 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa764067130 0x7fa7640695e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa7840c1620 tx=0x7fa778009040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:34:45.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.247+0000 7fa7727fc700 1 -- 172.21.15.55:0/2369166287 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa774051c90 con 0x7fa7840cc8d0 2023-10-06T03:34:45.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.479+0000 7fa78359e700 1 -- 172.21.15.55:0/2369166287 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7840c2f60 con 0x7fa764067130 2023-10-06T03:34:45.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.545+0000 7fa7727fc700 1 -- 172.21.15.55:0/2369166287 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa7840c2f60 con 0x7fa764067130 2023-10-06T03:34:45.546 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-10-06T03:34:45.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.548+0000 7fa78359e700 1 -- 172.21.15.55:0/2369166287 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa764067130 msgr2=0x7fa7640695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:45.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.548+0000 7fa78359e700 1 --2- 172.21.15.55:0/2369166287 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa764067130 0x7fa7640695e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa7840c1620 tx=0x7fa778009040 comp rx=0 tx=0).stop 2023-10-06T03:34:45.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.548+0000 7fa78359e700 1 -- 172.21.15.55:0/2369166287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7840cc8d0 msgr2=0x7fa784144bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:34:45.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.548+0000 7fa78359e700 1 --2- 172.21.15.55:0/2369166287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7840cc8d0 0x7fa784144bf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa77400ec80 tx=0x7fa7740091a0 comp rx=0 tx=0).stop 2023-10-06T03:34:45.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.548+0000 7fa78359e700 1 -- 172.21.15.55:0/2369166287 shutdown_connections 2023-10-06T03:34:45.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.548+0000 7fa78359e700 1 --2- 172.21.15.55:0/2369166287 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa764067130 0x7fa7640695e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:45.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.548+0000 7fa78359e700 1 --2- 172.21.15.55:0/2369166287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa78415bb30 0x7fa784153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:45.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.548+0000 7fa78359e700 1 --2- 172.21.15.55:0/2369166287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7840cc8d0 0x7fa784144bf0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:45.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.548+0000 7fa78359e700 1 --2- 172.21.15.55:0/2369166287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa78415b050 0x7fa7840c2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:34:45.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.548+0000 7fa78359e700 1 -- 172.21.15.55:0/2369166287 >> 172.21.15.55:0/2369166287 conn(0x7fa784068e20 msgr2=0x7fa78415a030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:34:45.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.548+0000 7fa78359e700 1 -- 172.21.15.55:0/2369166287 shutdown_connections 2023-10-06T03:34:45.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:34:45.548+0000 7fa78359e700 1 -- 172.21.15.55:0/2369166287 wait complete. 2023-10-06T03:34:45.567 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-10-06T03:35:00.035 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:35:00.032+0000 7feb7d5cf700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:35:00.035 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:35:00.032+0000 7feb7d5cf700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:35:00.035 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:35:00.032+0000 7feb7d5cf700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:35:00.035 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:35:00.032+0000 7feb7d5cf700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:35:00.035 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:35:00.033+0000 7feb7d5cf700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:35:12.303 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:35:12.309 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:35:42.413 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:35:42.419 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:36:12.518 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:36:12.523 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:36:42.639 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:36:42.644 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:37:12.762 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:37:12.768 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:37:42.903 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:37:42.908 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:38:13.060 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:38:13.065 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:38:43.183 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:38:43.189 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:39:00.463 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:39:00.462+0000 7feb7cdce700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:39:00.463 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:39:00.462+0000 7feb7cdce700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:39:00.463 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:39:00.462+0000 7feb7cdce700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:39:00.464 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:39:00.462+0000 7feb7cdce700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:39:00.464 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:39:00.462+0000 7feb7cdce700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:39:13.295 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:39:13.300 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:39:43.402 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:39:43.407 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:40:13.528 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:40:13.533 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:40:43.640 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:40:43.646 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:41:13.884 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:41:13.890 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:41:43.994 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:41:44.000 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:42:14.107 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:42:14.112 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:42:44.236 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:42:44.242 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:43:14.344 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:43:14.349 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:43:44.455 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:43:44.460 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:44:14.560 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:44:14.566 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:44:44.673 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:44:44.679 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:45:14.789 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:45:14.795 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:45:20.568 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir0 1M 2023-10-06T03:45:20.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.772+0000 7faa2499d700 1 Processor -- start 2023-10-06T03:45:20.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.772+0000 7faa2499d700 1 -- start start 2023-10-06T03:45:20.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.773+0000 7faa2499d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faa200b03c0 0x7faa200cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:20.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.773+0000 7faa2499d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa201591e0 0x7faa20159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:20.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.773+0000 7faa2499d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa200cfa00 con 0x7faa200b0790 2023-10-06T03:45:20.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.773+0000 7faa2499d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa200cb580 con 0x7faa200b03c0 2023-10-06T03:45:20.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.773+0000 7faa2499d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa200cf160 con 0x7faa201591e0 2023-10-06T03:45:20.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.774+0000 7faa1ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa201591e0 0x7faa20159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:20.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.774+0000 7faa1ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa201591e0 0x7faa20159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:33758/0 (socket says 172.21.15.55:33758) 2023-10-06T03:45:20.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.774+0000 7faa1ffff700 1 -- 172.21.15.55:0/203142241 learned_addr learned my addr 172.21.15.55:0/203142241 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:20.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.774+0000 7faa1effd700 1 --1- 172.21.15.55:0/203142241 >> v1:172.21.15.163:6789/0 conn(0x7faa200b0790 0x7faa200b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:35036/0 (socket says 172.21.15.55:35036) 2023-10-06T03:45:20.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.774+0000 7faa1f7fe700 1 --2- 172.21.15.55:0/203142241 >> v2:172.21.15.163:3300/0 conn(0x7faa200b03c0 0x7faa200cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:20.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.774+0000 7faa1ffff700 1 -- 172.21.15.55:0/203142241 >> v2:172.21.15.163:3300/0 conn(0x7faa200b03c0 msgr2=0x7faa200cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:20.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.774+0000 7faa1ffff700 1 --2- 172.21.15.55:0/203142241 >> v2:172.21.15.163:3300/0 conn(0x7faa200b03c0 0x7faa200cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:20.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.774+0000 7faa1ffff700 1 -- 172.21.15.55:0/203142241 >> v1:172.21.15.163:6789/0 conn(0x7faa200b0790 legacy=0x7faa200b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:20.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.774+0000 7faa1ffff700 1 -- 172.21.15.55:0/203142241 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa200cf440 con 0x7faa201591e0 2023-10-06T03:45:20.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.775+0000 7faa1ffff700 1 --2- 172.21.15.55:0/203142241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa201591e0 0x7faa20159620 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faa18009f50 tx=0x7faa18009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f7b242f9d1e6e6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:20.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.775+0000 7faa1e7fc700 1 -- 172.21.15.55:0/203142241 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1800e070 con 0x7faa201591e0 2023-10-06T03:45:20.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.775+0000 7faa1e7fc700 1 -- 172.21.15.55:0/203142241 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1800b9c0 con 0x7faa201591e0 2023-10-06T03:45:20.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.775+0000 7faa1e7fc700 1 -- 172.21.15.55:0/203142241 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180092e0 con 0x7faa201591e0 2023-10-06T03:45:20.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.776+0000 7faa2499d700 1 -- 172.21.15.55:0/203142241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa201591e0 msgr2=0x7faa20159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:20.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.776+0000 7faa2499d700 1 --2- 172.21.15.55:0/203142241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa201591e0 0x7faa20159620 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faa18009f50 tx=0x7faa18009840 comp rx=0 tx=0).stop 2023-10-06T03:45:20.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.776+0000 7faa2499d700 1 -- 172.21.15.55:0/203142241 shutdown_connections 2023-10-06T03:45:20.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.776+0000 7faa2499d700 1 --2- 172.21.15.55:0/203142241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa201591e0 0x7faa20159620 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:20.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.776+0000 7faa2499d700 1 --2- 172.21.15.55:0/203142241 >> v2:172.21.15.163:3300/0 conn(0x7faa200b03c0 0x7faa200cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:20.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.776+0000 7faa2499d700 1 -- 172.21.15.55:0/203142241 >> 172.21.15.55:0/203142241 conn(0x7faa20063230 msgr2=0x7faa20063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:20.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.776+0000 7faa2499d700 1 -- 172.21.15.55:0/203142241 shutdown_connections 2023-10-06T03:45:20.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.776+0000 7faa2499d700 1 -- 172.21.15.55:0/203142241 wait complete. 2023-10-06T03:45:20.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.777+0000 7faa2499d700 1 Processor -- start 2023-10-06T03:45:20.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.777+0000 7faa2499d700 1 -- start start 2023-10-06T03:45:20.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.778+0000 7faa2499d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa200b03c0 0x7faa200c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:20.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.778+0000 7faa2499d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa200b0790 0x7faa20158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:20.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.778+0000 7faa2499d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa201591e0 0x7faa200a8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:20.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.778+0000 7faa2499d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa200a8340 con 0x7faa200b03c0 2023-10-06T03:45:20.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.778+0000 7faa2499d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa200a77f0 con 0x7faa200b0790 2023-10-06T03:45:20.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.778+0000 7faa2499d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa200c7350 con 0x7faa201591e0 2023-10-06T03:45:20.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.779+0000 7faa1f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa200b03c0 0x7faa200c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:20.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.779+0000 7faa1f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa200b03c0 0x7faa200c6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37474/0 (socket says 172.21.15.55:37474) 2023-10-06T03:45:20.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.779+0000 7faa1f7fe700 1 -- 172.21.15.55:0/1017376012 learned_addr learned my addr 172.21.15.55:0/1017376012 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:20.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.779+0000 7faa1effd700 1 --2- 172.21.15.55:0/1017376012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa200b0790 0x7faa20158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:20.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.779+0000 7faa1ffff700 1 --2- 172.21.15.55:0/1017376012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa201591e0 0x7faa200a8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:20.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.779+0000 7faa1f7fe700 1 -- 172.21.15.55:0/1017376012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa201591e0 msgr2=0x7faa200a8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:20.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.779+0000 7faa1f7fe700 1 --2- 172.21.15.55:0/1017376012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa201591e0 0x7faa200a8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:20.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.779+0000 7faa1f7fe700 1 -- 172.21.15.55:0/1017376012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa200b0790 msgr2=0x7faa20158980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:20.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.779+0000 7faa1f7fe700 1 --2- 172.21.15.55:0/1017376012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa200b0790 0x7faa20158980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:20.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.779+0000 7faa1f7fe700 1 -- 172.21.15.55:0/1017376012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa18009ae0 con 0x7faa200b03c0 2023-10-06T03:45:20.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.779+0000 7faa1f7fe700 1 --2- 172.21.15.55:0/1017376012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa200b03c0 0x7faa200c6e10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faa1000e930 tx=0x7faa100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:20.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.779+0000 7faa077fe700 1 -- 172.21.15.55:0/1017376012 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10013680 con 0x7faa200b03c0 2023-10-06T03:45:20.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.780+0000 7faa077fe700 1 -- 172.21.15.55:0/1017376012 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa100137e0 con 0x7faa200b03c0 2023-10-06T03:45:20.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.780+0000 7faa077fe700 1 -- 172.21.15.55:0/1017376012 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10013b80 con 0x7faa200b03c0 2023-10-06T03:45:20.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.780+0000 7faa2499d700 1 -- 172.21.15.55:0/1017376012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa200a7aa0 con 0x7faa200b03c0 2023-10-06T03:45:20.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.780+0000 7faa2499d700 1 -- 172.21.15.55:0/1017376012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa200a8060 con 0x7faa200b03c0 2023-10-06T03:45:20.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.781+0000 7faa2499d700 1 -- 172.21.15.55:0/1017376012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa2015af60 con 0x7faa200b03c0 2023-10-06T03:45:20.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.782+0000 7faa077fe700 1 -- 172.21.15.55:0/1017376012 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7faa1000c3e0 con 0x7faa200b03c0 2023-10-06T03:45:20.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.782+0000 7faa077fe700 1 --2- 172.21.15.55:0/1017376012 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7faa0c067130 0x7faa0c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:20.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.783+0000 7faa077fe700 1 -- 172.21.15.55:0/1017376012 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7faa10084c10 con 0x7faa200b03c0 2023-10-06T03:45:20.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.783+0000 7faa1effd700 1 --2- 172.21.15.55:0/1017376012 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7faa0c067130 0x7faa0c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:20.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.784+0000 7faa1effd700 1 --2- 172.21.15.55:0/1017376012 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7faa0c067130 0x7faa0c0695e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faa14004fd0 tx=0x7faa14007eb0 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:20.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:20.787+0000 7faa077fe700 1 -- 172.21.15.55:0/1017376012 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa10051980 con 0x7faa200b03c0 2023-10-06T03:45:21.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.018+0000 7faa2499d700 1 -- 172.21.15.55:0/1017376012 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7faa2004c3c0 con 0x7faa0c067130 2023-10-06T03:45:21.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.092+0000 7faa077fe700 1 -- 172.21.15.55:0/1017376012 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7faa2004c3c0 con 0x7faa0c067130 2023-10-06T03:45:21.095 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-10-06T03:45:21.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.095+0000 7faa2499d700 1 -- 172.21.15.55:0/1017376012 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7faa0c067130 msgr2=0x7faa0c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:21.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.095+0000 7faa2499d700 1 --2- 172.21.15.55:0/1017376012 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7faa0c067130 0x7faa0c0695e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faa14004fd0 tx=0x7faa14007eb0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.095+0000 7faa2499d700 1 -- 172.21.15.55:0/1017376012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa200b03c0 msgr2=0x7faa200c6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:21.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.095+0000 7faa2499d700 1 --2- 172.21.15.55:0/1017376012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa200b03c0 0x7faa200c6e10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7faa1000e930 tx=0x7faa100091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.095+0000 7faa2499d700 1 -- 172.21.15.55:0/1017376012 shutdown_connections 2023-10-06T03:45:21.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.095+0000 7faa2499d700 1 --2- 172.21.15.55:0/1017376012 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7faa0c067130 0x7faa0c0695e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.095+0000 7faa2499d700 1 --2- 172.21.15.55:0/1017376012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa201591e0 0x7faa200a8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.095+0000 7faa2499d700 1 --2- 172.21.15.55:0/1017376012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa200b03c0 0x7faa200c6e10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.095+0000 7faa2499d700 1 --2- 172.21.15.55:0/1017376012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa200b0790 0x7faa20158980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.095+0000 7faa2499d700 1 -- 172.21.15.55:0/1017376012 >> 172.21.15.55:0/1017376012 conn(0x7faa20063230 msgr2=0x7faa200b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:21.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.095+0000 7faa2499d700 1 -- 172.21.15.55:0/1017376012 shutdown_connections 2023-10-06T03:45:21.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.095+0000 7faa2499d700 1 -- 172.21.15.55:0/1017376012 wait complete. 2023-10-06T03:45:21.115 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-10-06T03:45:21.115 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir1 1M 2023-10-06T03:45:21.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.304+0000 7f265a59d700 1 Processor -- start 2023-10-06T03:45:21.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.304+0000 7f265a59d700 1 -- start start 2023-10-06T03:45:21.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.305+0000 7f265a59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2654153480 0x7f26540ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:21.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.305+0000 7f265a59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2654153850 0x7f26541596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:21.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.305+0000 7f265a59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265415a050 con 0x7f2654159bf0 2023-10-06T03:45:21.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.305+0000 7f265a59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26540aec90 con 0x7f2654153850 2023-10-06T03:45:21.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.305+0000 7f265a59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f265415a480 con 0x7f2654153480 2023-10-06T03:45:21.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.306+0000 7f2658d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2654153850 0x7f26541596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:21.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.306+0000 7f2658d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2654153850 0x7f26541596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37490/0 (socket says 172.21.15.55:37490) 2023-10-06T03:45:21.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.306+0000 7f2658d9a700 1 -- 172.21.15.55:0/3214413162 learned_addr learned my addr 172.21.15.55:0/3214413162 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:21.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.306+0000 7f2659d9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2654159bf0 0x7f26541560b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37780/0 (socket says 172.21.15.55:37780) 2023-10-06T03:45:21.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.306+0000 7f265959b700 1 --2- 172.21.15.55:0/3214413162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2654153480 0x7f26540ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:21.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.306+0000 7f264bfff700 1 -- 172.21.15.55:0/3214413162 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958646468 0 0) 0x7f264c001490 con 0x7f2654159bf0 2023-10-06T03:45:21.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.306+0000 7f2658d9a700 1 -- 172.21.15.55:0/3214413162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2654153480 msgr2=0x7f26540ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:21.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.306+0000 7f2658d9a700 1 --2- 172.21.15.55:0/3214413162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2654153480 0x7f26540ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.306+0000 7f2658d9a700 1 -- 172.21.15.55:0/3214413162 >> v1:172.21.15.55:6789/0 conn(0x7f2654159bf0 legacy=0x7f26541560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:21.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.306+0000 7f2658d9a700 1 -- 172.21.15.55:0/3214413162 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26540afb40 con 0x7f2654153850 2023-10-06T03:45:21.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.307+0000 7f2658d9a700 1 --2- 172.21.15.55:0/3214413162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654153850 0x7f26541596b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f26440099b0 tx=0x7f2644009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72143989537f054 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:21.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.307+0000 7f264bfff700 1 -- 172.21.15.55:0/3214413162 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2644009ea0 con 0x7f2654153850 2023-10-06T03:45:21.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.307+0000 7f264bfff700 1 -- 172.21.15.55:0/3214413162 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264400b860 con 0x7f2654153850 2023-10-06T03:45:21.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.307+0000 7f264bfff700 1 -- 172.21.15.55:0/3214413162 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264400bc00 con 0x7f2654153850 2023-10-06T03:45:21.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.307+0000 7f265a59d700 1 -- 172.21.15.55:0/3214413162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654153850 msgr2=0x7f26541596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:21.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.307+0000 7f265a59d700 1 --2- 172.21.15.55:0/3214413162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654153850 0x7f26541596b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f26440099b0 tx=0x7f2644009250 comp rx=0 tx=0).stop 2023-10-06T03:45:21.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.307+0000 7f265a59d700 1 -- 172.21.15.55:0/3214413162 shutdown_connections 2023-10-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.307+0000 7f265a59d700 1 --2- 172.21.15.55:0/3214413162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2654153480 0x7f26540ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.307+0000 7f265a59d700 1 --2- 172.21.15.55:0/3214413162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654153850 0x7f26541596b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.307+0000 7f265a59d700 1 -- 172.21.15.55:0/3214413162 >> 172.21.15.55:0/3214413162 conn(0x7f2654068e20 msgr2=0x7f26540cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.307+0000 7f265a59d700 1 -- 172.21.15.55:0/3214413162 shutdown_connections 2023-10-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.307+0000 7f265a59d700 1 -- 172.21.15.55:0/3214413162 wait complete. 2023-10-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.309+0000 7f265a59d700 1 Processor -- start 2023-10-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.309+0000 7f265a59d700 1 -- start start 2023-10-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.310+0000 7f265a59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654153480 0x7f26540c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.310+0000 7f265a59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2654153850 0x7f26540ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.310+0000 7f265a59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2654159bf0 0x7f2654151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:21.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.310+0000 7f265a59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26540cadf0 con 0x7f2654153480 2023-10-06T03:45:21.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.310+0000 7f265a59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26540cb0d0 con 0x7f2654153850 2023-10-06T03:45:21.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.310+0000 7f265a59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26540cb3b0 con 0x7f2654159bf0 2023-10-06T03:45:21.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.310+0000 7f265959b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654153480 0x7f26540c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:21.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.310+0000 7f265959b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654153480 0x7f26540c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37496/0 (socket says 172.21.15.55:37496) 2023-10-06T03:45:21.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.310+0000 7f265959b700 1 -- 172.21.15.55:0/3070375260 learned_addr learned my addr 172.21.15.55:0/3070375260 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:21.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.310+0000 7f2659d9c700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2654159bf0 0x7f2654151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:21.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.310+0000 7f2658d9a700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2654153850 0x7f26540ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:21.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.311+0000 7f265959b700 1 -- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2654159bf0 msgr2=0x7f2654151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:21.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.311+0000 7f265959b700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2654159bf0 0x7f2654151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.311+0000 7f265959b700 1 -- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2654153850 msgr2=0x7f26540ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.311+0000 7f265959b700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2654153850 0x7f26540ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.311+0000 7f265959b700 1 -- 172.21.15.55:0/3070375260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26440094f0 con 0x7f2654153480 2023-10-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.311+0000 7f2659d9c700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2654159bf0 0x7f2654151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.311+0000 7f2658d9a700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2654153850 0x7f26540ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.311+0000 7f265959b700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654153480 0x7f26540c7820 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f265000ec80 tx=0x7f26500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.311+0000 7f26497fa700 1 -- 172.21.15.55:0/3070375260 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265000c9d0 con 0x7f2654153480 2023-10-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.311+0000 7f26497fa700 1 -- 172.21.15.55:0/3070375260 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265000cb30 con 0x7f2654153480 2023-10-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.311+0000 7f265a59d700 1 -- 172.21.15.55:0/3070375260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26540cb690 con 0x7f2654153480 2023-10-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.311+0000 7f26497fa700 1 -- 172.21.15.55:0/3070375260 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650013470 con 0x7f2654153480 2023-10-06T03:45:21.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.311+0000 7f265a59d700 1 -- 172.21.15.55:0/3070375260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26540c9e20 con 0x7f2654153480 2023-10-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.312+0000 7f265a59d700 1 -- 172.21.15.55:0/3070375260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f265416a0c0 con 0x7f2654153480 2023-10-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.313+0000 7f26497fa700 1 -- 172.21.15.55:0/3070375260 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f2650013820 con 0x7f2654153480 2023-10-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.314+0000 7f26497fa700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2638067160 0x7f2638069610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.314+0000 7f26497fa700 1 -- 172.21.15.55:0/3070375260 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f2650084f20 con 0x7f2654153480 2023-10-06T03:45:21.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.314+0000 7f2658d9a700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2638067160 0x7f2638069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:21.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.315+0000 7f2658d9a700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2638067160 0x7f2638069610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f26440099b0 tx=0x7f2644017000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:21.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.318+0000 7f26497fa700 1 -- 172.21.15.55:0/3070375260 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f265416a0c0 con 0x7f2654153480 2023-10-06T03:45:21.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.546+0000 7f265a59d700 1 -- 172.21.15.55:0/3070375260 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f265406f880 con 0x7f2638067160 2023-10-06T03:45:21.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.619+0000 7f26497fa700 1 -- 172.21.15.55:0/3070375260 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f265406f880 con 0x7f2638067160 2023-10-06T03:45:21.621 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-10-06T03:45:21.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.621+0000 7f265a59d700 1 -- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2638067160 msgr2=0x7f2638069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:21.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.621+0000 7f265a59d700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2638067160 0x7f2638069610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f26440099b0 tx=0x7f2644017000 comp rx=0 tx=0).stop 2023-10-06T03:45:21.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.621+0000 7f265a59d700 1 -- 172.21.15.55:0/3070375260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654153480 msgr2=0x7f26540c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:21.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.621+0000 7f265a59d700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654153480 0x7f26540c7820 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f265000ec80 tx=0x7f26500091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.621+0000 7f265a59d700 1 -- 172.21.15.55:0/3070375260 shutdown_connections 2023-10-06T03:45:21.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.621+0000 7f265a59d700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f2638067160 0x7f2638069610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.621+0000 7f265a59d700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2654159bf0 0x7f2654151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.621+0000 7f265a59d700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2654153480 0x7f26540c7820 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.621+0000 7f265a59d700 1 --2- 172.21.15.55:0/3070375260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2654153850 0x7f26540ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.621+0000 7f265a59d700 1 -- 172.21.15.55:0/3070375260 >> 172.21.15.55:0/3070375260 conn(0x7f2654068e20 msgr2=0x7f2654156a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:21.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.621+0000 7f265a59d700 1 -- 172.21.15.55:0/3070375260 shutdown_connections 2023-10-06T03:45:21.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.621+0000 7f265a59d700 1 -- 172.21.15.55:0/3070375260 wait complete. 2023-10-06T03:45:21.641 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-10-06T03:45:21.641 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir2 1M 2023-10-06T03:45:21.834 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.832+0000 7f4e3299d700 1 Processor -- start 2023-10-06T03:45:21.834 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.832+0000 7f4e3299d700 1 -- start start 2023-10-06T03:45:21.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.833+0000 7f4e3299d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4e2c0ac0e0 0x7f4e2c0afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:21.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.833+0000 7f4e3299d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2c06e510 con 0x7f4e2c1597d0 2023-10-06T03:45:21.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.833+0000 7f4e3299d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2c069180 con 0x7f4e2c06f0b0 2023-10-06T03:45:21.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.833+0000 7f4e3299d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e2c06dec0 con 0x7f4e2c0ac0e0 2023-10-06T03:45:21.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.834+0000 7f4e3199b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4e2c1597d0 0x7f4e2c06e9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37792/0 (socket says 172.21.15.55:37792) 2023-10-06T03:45:21.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.834+0000 7f4e3199b700 1 -- 172.21.15.55:0/601603173 learned_addr learned my addr 172.21.15.55:0/601603173 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:21.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.834+0000 7f4e3119a700 1 --2- 172.21.15.55:0/601603173 >> v2:172.21.15.55:3300/0 conn(0x7f4e2c0ac0e0 0x7f4e2c0afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:21.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.834+0000 7f4e30999700 1 -- 172.21.15.55:0/601603173 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2439606493 0 0) 0x7f4e2c06e510 con 0x7f4e2c1597d0 2023-10-06T03:45:21.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.834+0000 7f4e3119a700 1 -- 172.21.15.55:0/601603173 >> v1:172.21.15.163:6789/0 conn(0x7f4e2c06f0b0 legacy=0x7f4e2c15d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:21.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.834+0000 7f4e3119a700 1 -- 172.21.15.55:0/601603173 >> v1:172.21.15.55:6789/0 conn(0x7f4e2c1597d0 legacy=0x7f4e2c06e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:21.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.834+0000 7f4e3119a700 1 -- 172.21.15.55:0/601603173 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c06e1a0 con 0x7f4e2c0ac0e0 2023-10-06T03:45:21.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.834+0000 7f4e3119a700 1 --2- 172.21.15.55:0/601603173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c0ac0e0 0x7f4e2c0afa40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0098e0 tx=0x7f4e1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de9851c449e807c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:21.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.835+0000 7f4e30999700 1 -- 172.21.15.55:0/601603173 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c009e80 con 0x7f4e2c0ac0e0 2023-10-06T03:45:21.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.835+0000 7f4e30999700 1 -- 172.21.15.55:0/601603173 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c00b860 con 0x7f4e2c0ac0e0 2023-10-06T03:45:21.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.835+0000 7f4e30999700 1 -- 172.21.15.55:0/601603173 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00bbe0 con 0x7f4e2c0ac0e0 2023-10-06T03:45:21.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.835+0000 7f4e3299d700 1 -- 172.21.15.55:0/601603173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c0ac0e0 msgr2=0x7f4e2c0afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:21.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.835+0000 7f4e3299d700 1 --2- 172.21.15.55:0/601603173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c0ac0e0 0x7f4e2c0afa40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c0098e0 tx=0x7f4e1c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.835+0000 7f4e3299d700 1 -- 172.21.15.55:0/601603173 shutdown_connections 2023-10-06T03:45:21.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.835+0000 7f4e3299d700 1 --2- 172.21.15.55:0/601603173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c0ac0e0 0x7f4e2c0afa40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.835+0000 7f4e3299d700 1 -- 172.21.15.55:0/601603173 >> 172.21.15.55:0/601603173 conn(0x7f4e2c06adf0 msgr2=0x7f4e2c0ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:21.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.835+0000 7f4e3299d700 1 -- 172.21.15.55:0/601603173 shutdown_connections 2023-10-06T03:45:21.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.835+0000 7f4e3299d700 1 -- 172.21.15.55:0/601603173 wait complete. 2023-10-06T03:45:21.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.837+0000 7f4e3299d700 1 Processor -- start 2023-10-06T03:45:21.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.837+0000 7f4e3299d700 1 -- start start 2023-10-06T03:45:21.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3299d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e2c06f0b0 0x7f4e2c0ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:21.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3299d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c1597d0 0x7f4e2c1455b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:21.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3299d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e2c158700 0x7f4e2c158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:21.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3299d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c0ae4b0 con 0x7f4e2c1597d0 2023-10-06T03:45:21.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3299d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c1590b0 con 0x7f4e2c06f0b0 2023-10-06T03:45:21.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3299d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2c16afd0 con 0x7f4e2c158700 2023-10-06T03:45:21.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3119a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c1597d0 0x7f4e2c1455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:21.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3119a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c1597d0 0x7f4e2c1455b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37526/0 (socket says 172.21.15.55:37526) 2023-10-06T03:45:21.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3119a700 1 -- 172.21.15.55:0/3463062181 learned_addr learned my addr 172.21.15.55:0/3463062181 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:21.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3219c700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e2c158700 0x7f4e2c158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:21.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3119a700 1 -- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e2c158700 msgr2=0x7f4e2c158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:21.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3119a700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e2c158700 0x7f4e2c158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3199b700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e2c06f0b0 0x7f4e2c0ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:21.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3119a700 1 -- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e2c06f0b0 msgr2=0x7f4e2c0ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:21.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3119a700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e2c06f0b0 0x7f4e2c0ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:21.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3119a700 1 -- 172.21.15.55:0/3463062181 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1c009420 con 0x7f4e2c1597d0 2023-10-06T03:45:21.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3219c700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e2c158700 0x7f4e2c158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:45:21.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.838+0000 7f4e3199b700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e2c06f0b0 0x7f4e2c0ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:45:21.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.839+0000 7f4e3119a700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c1597d0 0x7f4e2c1455b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c06dad0 tx=0x7f4e1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:21.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.839+0000 7f4e21ffb700 1 -- 172.21.15.55:0/3463062181 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c01f070 con 0x7f4e2c1597d0 2023-10-06T03:45:21.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.839+0000 7f4e21ffb700 1 -- 172.21.15.55:0/3463062181 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1c011420 con 0x7f4e2c1597d0 2023-10-06T03:45:21.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.839+0000 7f4e3299d700 1 -- 172.21.15.55:0/3463062181 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2c0acdd0 con 0x7f4e2c1597d0 2023-10-06T03:45:21.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.839+0000 7f4e21ffb700 1 -- 172.21.15.55:0/3463062181 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c00bbe0 con 0x7f4e2c1597d0 2023-10-06T03:45:21.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.839+0000 7f4e3299d700 1 -- 172.21.15.55:0/3463062181 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2c0ac780 con 0x7f4e2c1597d0 2023-10-06T03:45:21.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.840+0000 7f4e3299d700 1 -- 172.21.15.55:0/3463062181 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e2c0acf60 con 0x7f4e2c1597d0 2023-10-06T03:45:21.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.841+0000 7f4e21ffb700 1 -- 172.21.15.55:0/3463062181 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f4e1c011a50 con 0x7f4e2c1597d0 2023-10-06T03:45:21.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.842+0000 7f4e21ffb700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4e10067130 0x7f4e100695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:21.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.842+0000 7f4e21ffb700 1 -- 172.21.15.55:0/3463062181 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f4e1c086180 con 0x7f4e2c1597d0 2023-10-06T03:45:21.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.843+0000 7f4e3199b700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4e10067130 0x7f4e100695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:21.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.843+0000 7f4e3199b700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4e10067130 0x7f4e100695e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4e28000c00 tx=0x7f4e28009040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:21.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:21.846+0000 7f4e21ffb700 1 -- 172.21.15.55:0/3463062181 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e1c052e50 con 0x7f4e2c1597d0 2023-10-06T03:45:22.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.075+0000 7f4e3299d700 1 -- 172.21.15.55:0/3463062181 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e2c0cd4a0 con 0x7f4e10067130 2023-10-06T03:45:22.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.154+0000 7f4e21ffb700 1 -- 172.21.15.55:0/3463062181 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4e2c0cd4a0 con 0x7f4e10067130 2023-10-06T03:45:22.156 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-10-06T03:45:22.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.156+0000 7f4e3299d700 1 -- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4e10067130 msgr2=0x7f4e100695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:22.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.156+0000 7f4e3299d700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4e10067130 0x7f4e100695e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4e28000c00 tx=0x7f4e28009040 comp rx=0 tx=0).stop 2023-10-06T03:45:22.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.156+0000 7f4e3299d700 1 -- 172.21.15.55:0/3463062181 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c1597d0 msgr2=0x7f4e2c1455b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:22.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.156+0000 7f4e3299d700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c1597d0 0x7f4e2c1455b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c06dad0 tx=0x7f4e1c017000 comp rx=0 tx=0).stop 2023-10-06T03:45:22.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.156+0000 7f4e3299d700 1 -- 172.21.15.55:0/3463062181 shutdown_connections 2023-10-06T03:45:22.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.156+0000 7f4e3299d700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4e10067130 0x7f4e100695e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.156+0000 7f4e3299d700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4e2c158700 0x7f4e2c158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.156+0000 7f4e3299d700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4e2c06f0b0 0x7f4e2c0ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.156+0000 7f4e3299d700 1 --2- 172.21.15.55:0/3463062181 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4e2c1597d0 0x7f4e2c1455b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.156+0000 7f4e3299d700 1 -- 172.21.15.55:0/3463062181 >> 172.21.15.55:0/3463062181 conn(0x7f4e2c06adf0 msgr2=0x7f4e2c15bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:22.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.156+0000 7f4e3299d700 1 -- 172.21.15.55:0/3463062181 shutdown_connections 2023-10-06T03:45:22.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.156+0000 7f4e3299d700 1 -- 172.21.15.55:0/3463062181 wait complete. 2023-10-06T03:45:22.176 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-10-06T03:45:22.176 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir3 1M 2023-10-06T03:45:22.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.366+0000 7f519b59d700 1 Processor -- start 2023-10-06T03:45:22.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.366+0000 7f519b59d700 1 -- start start 2023-10-06T03:45:22.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.367+0000 7f519b59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5194156350 0x7f5194157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:22.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.367+0000 7f519b59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51940b0760 0x7f51940b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:22.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.367+0000 7f519b59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5194159f20 0x7f519415a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:22.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.367+0000 7f519b59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51940b1110 con 0x7f5194156350 2023-10-06T03:45:22.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.367+0000 7f519b59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51940b0570 con 0x7f5194159f20 2023-10-06T03:45:22.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.367+0000 7f519b59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51941578b0 con 0x7f51940b0760 2023-10-06T03:45:22.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.368+0000 7f519a59b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5194156350 0x7f5194157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:22.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.368+0000 7f519a59b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5194156350 0x7f5194157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37540/0 (socket says 172.21.15.55:37540) 2023-10-06T03:45:22.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.368+0000 7f519a59b700 1 -- 172.21.15.55:0/661934293 learned_addr learned my addr 172.21.15.55:0/661934293 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:22.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.368+0000 7f519a59b700 1 -- 172.21.15.55:0/661934293 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51940b0760 msgr2=0x7f51940b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:22.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.368+0000 7f519a59b700 1 --2- 172.21.15.55:0/661934293 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51940b0760 0x7f51940b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.368+0000 7f519a59b700 1 -- 172.21.15.55:0/661934293 >> v2:172.21.15.163:3300/0 conn(0x7f5194159f20 msgr2=0x7f519415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:22.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.368+0000 7f519a59b700 1 --2- 172.21.15.55:0/661934293 >> v2:172.21.15.163:3300/0 conn(0x7f5194159f20 0x7f519415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.368+0000 7f519a59b700 1 -- 172.21.15.55:0/661934293 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51940af4d0 con 0x7f5194156350 2023-10-06T03:45:22.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.368+0000 7f519a59b700 1 --2- 172.21.15.55:0/661934293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194156350 0x7f5194157370 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5194067f50 tx=0x7f5190009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6709bf2145a0d479 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:22.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.368+0000 7f5199599700 1 -- 172.21.15.55:0/661934293 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519000b860 con 0x7f5194156350 2023-10-06T03:45:22.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.369+0000 7f5199599700 1 -- 172.21.15.55:0/661934293 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519000b9c0 con 0x7f5194156350 2023-10-06T03:45:22.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.369+0000 7f5199599700 1 -- 172.21.15.55:0/661934293 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519000bd60 con 0x7f5194156350 2023-10-06T03:45:22.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.369+0000 7f519b59d700 1 -- 172.21.15.55:0/661934293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194156350 msgr2=0x7f5194157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:22.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.369+0000 7f519b59d700 1 --2- 172.21.15.55:0/661934293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194156350 0x7f5194157370 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5194067f50 tx=0x7f5190009250 comp rx=0 tx=0).stop 2023-10-06T03:45:22.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.369+0000 7f519b59d700 1 -- 172.21.15.55:0/661934293 shutdown_connections 2023-10-06T03:45:22.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.369+0000 7f519b59d700 1 --2- 172.21.15.55:0/661934293 >> v2:172.21.15.163:3300/0 conn(0x7f5194159f20 0x7f519415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.369+0000 7f519b59d700 1 --2- 172.21.15.55:0/661934293 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5194156350 0x7f5194157370 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.369+0000 7f519b59d700 1 --2- 172.21.15.55:0/661934293 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51940b0760 0x7f51940b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.369+0000 7f519b59d700 1 -- 172.21.15.55:0/661934293 >> 172.21.15.55:0/661934293 conn(0x7f5194068e20 msgr2=0x7f51940cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:22.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.369+0000 7f519b59d700 1 -- 172.21.15.55:0/661934293 shutdown_connections 2023-10-06T03:45:22.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.369+0000 7f519b59d700 1 -- 172.21.15.55:0/661934293 wait complete. 2023-10-06T03:45:22.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.371+0000 7f519b59d700 1 Processor -- start 2023-10-06T03:45:22.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.371+0000 7f519b59d700 1 -- start start 2023-10-06T03:45:22.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.372+0000 7f519b59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51940b0760 0x7f519414cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:22.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.372+0000 7f519b59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5194156350 0x7f51940cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:22.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.372+0000 7f519b59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194159f20 0x7f51940cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:22.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.372+0000 7f519b59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519414d330 con 0x7f51940b0760 2023-10-06T03:45:22.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.372+0000 7f519b59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519414eaa0 con 0x7f5194159f20 2023-10-06T03:45:22.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.372+0000 7f519b59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51940cbdc0 con 0x7f5194156350 2023-10-06T03:45:22.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.372+0000 7f519a59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51940b0760 0x7f519414cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:22.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.372+0000 7f519a59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51940b0760 0x7f519414cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37546/0 (socket says 172.21.15.55:37546) 2023-10-06T03:45:22.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.372+0000 7f519a59b700 1 -- 172.21.15.55:0/478880022 learned_addr learned my addr 172.21.15.55:0/478880022 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:22.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.372+0000 7f519a59b700 1 -- 172.21.15.55:0/478880022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5194156350 msgr2=0x7f51940cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:22.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.372+0000 7f519a59b700 1 --2- 172.21.15.55:0/478880022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5194156350 0x7f51940cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.372+0000 7f519a59b700 1 -- 172.21.15.55:0/478880022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194159f20 msgr2=0x7f51940cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:22.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.372+0000 7f519a59b700 1 --2- 172.21.15.55:0/478880022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194159f20 0x7f51940cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.372+0000 7f519a59b700 1 -- 172.21.15.55:0/478880022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900094f0 con 0x7f51940b0760 2023-10-06T03:45:22.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.373+0000 7f519a59b700 1 --2- 172.21.15.55:0/478880022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51940b0760 0x7f519414cdf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5190009290 tx=0x7f5190017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:22.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.373+0000 7f518a7fc700 1 -- 172.21.15.55:0/478880022 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190011e20 con 0x7f51940b0760 2023-10-06T03:45:22.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.373+0000 7f518a7fc700 1 -- 172.21.15.55:0/478880022 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519001a660 con 0x7f51940b0760 2023-10-06T03:45:22.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.373+0000 7f518a7fc700 1 -- 172.21.15.55:0/478880022 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519001aa20 con 0x7f51940b0760 2023-10-06T03:45:22.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.373+0000 7f519b59d700 1 -- 172.21.15.55:0/478880022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5194150160 con 0x7f51940b0760 2023-10-06T03:45:22.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.373+0000 7f519b59d700 1 -- 172.21.15.55:0/478880022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5194150720 con 0x7f51940b0760 2023-10-06T03:45:22.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.374+0000 7f519b59d700 1 -- 172.21.15.55:0/478880022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519404c3c0 con 0x7f51940b0760 2023-10-06T03:45:22.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.375+0000 7f518a7fc700 1 -- 172.21.15.55:0/478880022 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f5190010380 con 0x7f51940b0760 2023-10-06T03:45:22.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.376+0000 7f518a7fc700 1 --2- 172.21.15.55:0/478880022 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5178067160 0x7f5178069610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:22.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.376+0000 7f518a7fc700 1 -- 172.21.15.55:0/478880022 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f5190085f70 con 0x7f51940b0760 2023-10-06T03:45:22.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.376+0000 7f5199d9a700 1 --2- 172.21.15.55:0/478880022 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5178067160 0x7f5178069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:22.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.377+0000 7f5199d9a700 1 --2- 172.21.15.55:0/478880022 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5178067160 0x7f5178069610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f519406f170 tx=0x7f5184006d10 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:22.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.380+0000 7f518a7fc700 1 -- 172.21.15.55:0/478880022 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5190052c40 con 0x7f51940b0760 2023-10-06T03:45:22.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.609+0000 7f519b59d700 1 -- 172.21.15.55:0/478880022 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f51940cc050 con 0x7f5178067160 2023-10-06T03:45:22.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.680+0000 7f518a7fc700 1 -- 172.21.15.55:0/478880022 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f51940cc050 con 0x7f5178067160 2023-10-06T03:45:22.683 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-10-06T03:45:22.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.683+0000 7f519b59d700 1 -- 172.21.15.55:0/478880022 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5178067160 msgr2=0x7f5178069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:22.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.683+0000 7f519b59d700 1 --2- 172.21.15.55:0/478880022 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5178067160 0x7f5178069610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f519406f170 tx=0x7f5184006d10 comp rx=0 tx=0).stop 2023-10-06T03:45:22.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.683+0000 7f519b59d700 1 -- 172.21.15.55:0/478880022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51940b0760 msgr2=0x7f519414cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:22.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.683+0000 7f519b59d700 1 --2- 172.21.15.55:0/478880022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51940b0760 0x7f519414cdf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5190009290 tx=0x7f5190017000 comp rx=0 tx=0).stop 2023-10-06T03:45:22.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.683+0000 7f519b59d700 1 -- 172.21.15.55:0/478880022 shutdown_connections 2023-10-06T03:45:22.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.683+0000 7f519b59d700 1 --2- 172.21.15.55:0/478880022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5194159f20 0x7f51940cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.683+0000 7f519b59d700 1 --2- 172.21.15.55:0/478880022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51940b0760 0x7f519414cdf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.683+0000 7f519b59d700 1 --2- 172.21.15.55:0/478880022 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f5178067160 0x7f5178069610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.683+0000 7f519b59d700 1 --2- 172.21.15.55:0/478880022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5194156350 0x7f51940cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.683+0000 7f519b59d700 1 -- 172.21.15.55:0/478880022 >> 172.21.15.55:0/478880022 conn(0x7f5194068e20 msgr2=0x7f51940add60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:22.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.683+0000 7f519b59d700 1 -- 172.21.15.55:0/478880022 shutdown_connections 2023-10-06T03:45:22.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.683+0000 7f519b59d700 1 -- 172.21.15.55:0/478880022 wait complete. 2023-10-06T03:45:22.703 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-10-06T03:45:22.703 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir4 1M 2023-10-06T03:45:22.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.894+0000 7fbfd759e700 1 Processor -- start 2023-10-06T03:45:22.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.894+0000 7fbfd759e700 1 -- start start 2023-10-06T03:45:22.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.895+0000 7fbfd759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfd8156350 0x7fbfd8157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:22.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.895+0000 7fbfd759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbfd80b0bd0 0x7fbfd815a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:22.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.895+0000 7fbfd759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfd80b0570 con 0x7fbfd80b0760 2023-10-06T03:45:22.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.895+0000 7fbfd759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd81578b0 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.895+0000 7fbfd759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd80af4d0 con 0x7fbfd8156350 2023-10-06T03:45:22.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.896+0000 7fbfd659c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfd8156350 0x7fbfd8157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:22.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.896+0000 7fbfd6d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbfd80b0bd0 0x7fbfd815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:22.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.896+0000 7fbfd5d9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fbfd80b0760 0x7fbfd8159f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:35048/0 (socket says 172.21.15.55:35048) 2023-10-06T03:45:22.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.896+0000 7fbfd5d9b700 1 -- 172.21.15.55:0/1890154463 learned_addr learned my addr 172.21.15.55:0/1890154463 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.896+0000 7fbfd559a700 1 -- 172.21.15.55:0/1890154463 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455231942 0 0) 0x7fbfd80b0570 con 0x7fbfd80b0760 2023-10-06T03:45:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.896+0000 7fbfd559a700 1 -- 172.21.15.55:0/1890154463 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfc0003570 con 0x7fbfd80b0760 2023-10-06T03:45:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.896+0000 7fbfd6d9d700 1 -- 172.21.15.55:0/1890154463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfd8156350 msgr2=0x7fbfd8157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.896+0000 7fbfd6d9d700 1 --2- 172.21.15.55:0/1890154463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfd8156350 0x7fbfd8157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.896+0000 7fbfd6d9d700 1 -- 172.21.15.55:0/1890154463 >> v1:172.21.15.163:6789/0 conn(0x7fbfd80b0760 legacy=0x7fbfd8159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.896+0000 7fbfd6d9d700 1 -- 172.21.15.55:0/1890154463 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd80af7b0 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.897+0000 7fbfd6d9d700 1 --2- 172.21.15.55:0/1890154463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd80b0bd0 0x7fbfd815a630 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009e60 tx=0x7fbfd0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3819a1a9555951 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.897+0000 7fbfd559a700 1 -- 172.21.15.55:0/1890154463 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000bab0 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.897+0000 7fbfd559a700 1 -- 172.21.15.55:0/1890154463 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd000bc10 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.897+0000 7fbfd559a700 1 -- 172.21.15.55:0/1890154463 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0011460 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.898+0000 7fbfd759e700 1 -- 172.21.15.55:0/1890154463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd80b0bd0 msgr2=0x7fbfd815a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.898+0000 7fbfd759e700 1 --2- 172.21.15.55:0/1890154463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd80b0bd0 0x7fbfd815a630 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0009e60 tx=0x7fbfd0009780 comp rx=0 tx=0).stop 2023-10-06T03:45:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.898+0000 7fbfd759e700 1 -- 172.21.15.55:0/1890154463 shutdown_connections 2023-10-06T03:45:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.898+0000 7fbfd759e700 1 --2- 172.21.15.55:0/1890154463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfd8156350 0x7fbfd8157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.898+0000 7fbfd759e700 1 --2- 172.21.15.55:0/1890154463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd80b0bd0 0x7fbfd815a630 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.898+0000 7fbfd759e700 1 -- 172.21.15.55:0/1890154463 >> 172.21.15.55:0/1890154463 conn(0x7fbfd8068e20 msgr2=0x7fbfd80cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:22.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.898+0000 7fbfd759e700 1 -- 172.21.15.55:0/1890154463 shutdown_connections 2023-10-06T03:45:22.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.898+0000 7fbfd759e700 1 -- 172.21.15.55:0/1890154463 wait complete. 2023-10-06T03:45:22.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.899+0000 7fbfd759e700 1 Processor -- start 2023-10-06T03:45:22.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.899+0000 7fbfd759e700 1 -- start start 2023-10-06T03:45:22.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.900+0000 7fbfd759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd80b0760 0x7fbfd80cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:22.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.900+0000 7fbfd759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfd80b0bd0 0x7fbfd8150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:22.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.900+0000 7fbfd759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfd8156350 0x7fbfd80c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:22.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.900+0000 7fbfd759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd80ac360 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.900+0000 7fbfd759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd80abd30 con 0x7fbfd80b0760 2023-10-06T03:45:22.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.900+0000 7fbfd759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd80cbb80 con 0x7fbfd8156350 2023-10-06T03:45:22.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.901+0000 7fbfd5d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfd80b0bd0 0x7fbfd8150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:22.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.901+0000 7fbfd5d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfd80b0bd0 0x7fbfd8150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37552/0 (socket says 172.21.15.55:37552) 2023-10-06T03:45:22.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.901+0000 7fbfd5d9b700 1 -- 172.21.15.55:0/934199412 learned_addr learned my addr 172.21.15.55:0/934199412 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:22.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.901+0000 7fbfd659c700 1 --2- 172.21.15.55:0/934199412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd80b0760 0x7fbfd80cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:22.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.901+0000 7fbfd6d9d700 1 --2- 172.21.15.55:0/934199412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfd8156350 0x7fbfd80c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:22.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.901+0000 7fbfd5d9b700 1 -- 172.21.15.55:0/934199412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfd8156350 msgr2=0x7fbfd80c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:22.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.901+0000 7fbfd5d9b700 1 --2- 172.21.15.55:0/934199412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfd8156350 0x7fbfd80c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.901+0000 7fbfd5d9b700 1 -- 172.21.15.55:0/934199412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd80b0760 msgr2=0x7fbfd80cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:22.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.901+0000 7fbfd5d9b700 1 --2- 172.21.15.55:0/934199412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd80b0760 0x7fbfd80cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:22.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.901+0000 7fbfd5d9b700 1 -- 172.21.15.55:0/934199412 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd0009a20 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.901+0000 7fbfd5d9b700 1 --2- 172.21.15.55:0/934199412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfd80b0bd0 0x7fbfd8150030 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00c590 tx=0x7fbfcc007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:22.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.902+0000 7fbfc67fc700 1 -- 172.21.15.55:0/934199412 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc005b30 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.902+0000 7fbfc67fc700 1 -- 172.21.15.55:0/934199412 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfcc005cb0 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.902+0000 7fbfc67fc700 1 -- 172.21.15.55:0/934199412 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc007320 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.902+0000 7fbfd759e700 1 -- 172.21.15.55:0/934199412 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd80cbe60 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.902+0000 7fbfd759e700 1 -- 172.21.15.55:0/934199412 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd80cb820 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.903+0000 7fbfd759e700 1 -- 172.21.15.55:0/934199412 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd804c3c0 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.904+0000 7fbfc67fc700 1 -- 172.21.15.55:0/934199412 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fbfcc01b3b0 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.905+0000 7fbfc67fc700 1 --2- 172.21.15.55:0/934199412 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbfb8067130 0x7fbfb80695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:22.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.905+0000 7fbfc67fc700 1 -- 172.21.15.55:0/934199412 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fbfcc083fe0 con 0x7fbfd80b0bd0 2023-10-06T03:45:22.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.905+0000 7fbfd659c700 1 --2- 172.21.15.55:0/934199412 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbfb8067130 0x7fbfb80695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:22.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.906+0000 7fbfd659c700 1 --2- 172.21.15.55:0/934199412 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbfb8067130 0x7fbfb80695e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009db0 tx=0x7fbfc80091a0 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:22.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:22.908+0000 7fbfc67fc700 1 -- 172.21.15.55:0/934199412 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfcc050cb0 con 0x7fbfd80b0bd0 2023-10-06T03:45:23.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.141+0000 7fbfd759e700 1 -- 172.21.15.55:0/934199412 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfd80afa90 con 0x7fbfb8067130 2023-10-06T03:45:23.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.215+0000 7fbfc67fc700 1 -- 172.21.15.55:0/934199412 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fbfd80afa90 con 0x7fbfb8067130 2023-10-06T03:45:23.217 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-10-06T03:45:23.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.217+0000 7fbfd759e700 1 -- 172.21.15.55:0/934199412 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbfb8067130 msgr2=0x7fbfb80695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:23.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.217+0000 7fbfd759e700 1 --2- 172.21.15.55:0/934199412 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbfb8067130 0x7fbfb80695e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009db0 tx=0x7fbfc80091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.217+0000 7fbfd759e700 1 -- 172.21.15.55:0/934199412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfd80b0bd0 msgr2=0x7fbfd8150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:23.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.217+0000 7fbfd759e700 1 --2- 172.21.15.55:0/934199412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfd80b0bd0 0x7fbfd8150030 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc00c590 tx=0x7fbfcc007e00 comp rx=0 tx=0).stop 2023-10-06T03:45:23.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.217+0000 7fbfd759e700 1 -- 172.21.15.55:0/934199412 shutdown_connections 2023-10-06T03:45:23.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.217+0000 7fbfd759e700 1 --2- 172.21.15.55:0/934199412 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbfb8067130 0x7fbfb80695e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.217+0000 7fbfd759e700 1 --2- 172.21.15.55:0/934199412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfd8156350 0x7fbfd80c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.217+0000 7fbfd759e700 1 --2- 172.21.15.55:0/934199412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfd80b0760 0x7fbfd80cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.217+0000 7fbfd759e700 1 --2- 172.21.15.55:0/934199412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfd80b0bd0 0x7fbfd8150030 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.217+0000 7fbfd759e700 1 -- 172.21.15.55:0/934199412 >> 172.21.15.55:0/934199412 conn(0x7fbfd8068e20 msgr2=0x7fbfd8151f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:23.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.217+0000 7fbfd759e700 1 -- 172.21.15.55:0/934199412 shutdown_connections 2023-10-06T03:45:23.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.217+0000 7fbfd759e700 1 -- 172.21.15.55:0/934199412 wait complete. 2023-10-06T03:45:23.237 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-10-06T03:45:23.237 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir5 1M 2023-10-06T03:45:23.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.428+0000 7fb54e39d700 1 Processor -- start 2023-10-06T03:45:23.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.428+0000 7fb54e39d700 1 -- start start 2023-10-06T03:45:23.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.429+0000 7fb54e39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb548153480 0x7fb5480ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:23.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.429+0000 7fb54e39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5480aec90 con 0x7fb548159dc0 2023-10-06T03:45:23.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.429+0000 7fb54e39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54815a480 con 0x7fb548153850 2023-10-06T03:45:23.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.429+0000 7fb54e39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5480afb40 con 0x7fb548153480 2023-10-06T03:45:23.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.429+0000 7fb54d39b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb548153480 0x7fb5480ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:23.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.429+0000 7fb54d39b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb548153480 0x7fb5480ae750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54650/0 (socket says 172.21.15.55:54650) 2023-10-06T03:45:23.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.429+0000 7fb54db9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb548159dc0 0x7fb548157280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35396/0 (socket says 172.21.15.55:35396) 2023-10-06T03:45:23.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.429+0000 7fb54d39b700 1 -- 172.21.15.55:0/1664442299 learned_addr learned my addr 172.21.15.55:0/1664442299 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:23.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.429+0000 7fb54d39b700 1 -- 172.21.15.55:0/1664442299 >> v1:172.21.15.163:6789/0 conn(0x7fb548153850 legacy=0x7fb5481596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:23.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.429+0000 7fb54d39b700 1 -- 172.21.15.55:0/1664442299 >> v1:172.21.15.55:6789/0 conn(0x7fb548159dc0 legacy=0x7fb548157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:23.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.429+0000 7fb54d39b700 1 -- 172.21.15.55:0/1664442299 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5480afe20 con 0x7fb548153480 2023-10-06T03:45:23.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.430+0000 7fb54d39b700 1 --2- 172.21.15.55:0/1664442299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb548153480 0x7fb5480ae750 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb5440099b0 tx=0x7fb544009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b237f12b24977b6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:23.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.430+0000 7fb53ffff700 1 -- 172.21.15.55:0/1664442299 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54400b820 con 0x7fb548153480 2023-10-06T03:45:23.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.430+0000 7fb53ffff700 1 -- 172.21.15.55:0/1664442299 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54400b980 con 0x7fb548153480 2023-10-06T03:45:23.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.430+0000 7fb53ffff700 1 -- 172.21.15.55:0/1664442299 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54400bd20 con 0x7fb548153480 2023-10-06T03:45:23.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.430+0000 7fb54e39d700 1 -- 172.21.15.55:0/1664442299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb548153480 msgr2=0x7fb5480ae750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:23.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.430+0000 7fb54e39d700 1 --2- 172.21.15.55:0/1664442299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb548153480 0x7fb5480ae750 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb5440099b0 tx=0x7fb544009250 comp rx=0 tx=0).stop 2023-10-06T03:45:23.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.430+0000 7fb54e39d700 1 -- 172.21.15.55:0/1664442299 shutdown_connections 2023-10-06T03:45:23.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.430+0000 7fb54e39d700 1 --2- 172.21.15.55:0/1664442299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb548153480 0x7fb5480ae750 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.430+0000 7fb54e39d700 1 -- 172.21.15.55:0/1664442299 >> 172.21.15.55:0/1664442299 conn(0x7fb548068e20 msgr2=0x7fb5480cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:23.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.431+0000 7fb54e39d700 1 -- 172.21.15.55:0/1664442299 shutdown_connections 2023-10-06T03:45:23.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.431+0000 7fb54e39d700 1 -- 172.21.15.55:0/1664442299 wait complete. 2023-10-06T03:45:23.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.432+0000 7fb54e39d700 1 Processor -- start 2023-10-06T03:45:23.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.432+0000 7fb54e39d700 1 -- start start 2023-10-06T03:45:23.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54e39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb548153480 0x7fb54815a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:23.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54e39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb548153850 0x7fb5480c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:23.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54e39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb548159dc0 0x7fb548149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:23.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54e39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54815c6e0 con 0x7fb548153480 2023-10-06T03:45:23.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54e39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480c7b60 con 0x7fb548159dc0 2023-10-06T03:45:23.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54e39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480c7e40 con 0x7fb548153850 2023-10-06T03:45:23.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54d39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb548153480 0x7fb54815a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:23.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54d39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb548153480 0x7fb54815a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54656/0 (socket says 172.21.15.55:54656) 2023-10-06T03:45:23.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54d39b700 1 -- 172.21.15.55:0/1788661083 learned_addr learned my addr 172.21.15.55:0/1788661083 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:23.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54d39b700 1 -- 172.21.15.55:0/1788661083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb548153850 msgr2=0x7fb5480c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:23.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54d39b700 1 --2- 172.21.15.55:0/1788661083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb548153850 0x7fb5480c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54d39b700 1 -- 172.21.15.55:0/1788661083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb548159dc0 msgr2=0x7fb548149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:23.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54d39b700 1 --2- 172.21.15.55:0/1788661083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb548159dc0 0x7fb548149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54d39b700 1 -- 172.21.15.55:0/1788661083 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5440094f0 con 0x7fb548153480 2023-10-06T03:45:23.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54d39b700 1 --2- 172.21.15.55:0/1788661083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb548153480 0x7fb54815a790 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb54400c010 tx=0x7fb544016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:23.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb53d7fa700 1 -- 172.21.15.55:0/1788661083 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544011e10 con 0x7fb548153480 2023-10-06T03:45:23.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb53d7fa700 1 -- 172.21.15.55:0/1788661083 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb544020070 con 0x7fb548153480 2023-10-06T03:45:23.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb53d7fa700 1 -- 172.21.15.55:0/1788661083 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54401b6a0 con 0x7fb548153480 2023-10-06T03:45:23.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54e39d700 1 -- 172.21.15.55:0/1788661083 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb548148c90 con 0x7fb548153480 2023-10-06T03:45:23.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.434+0000 7fb54e39d700 1 -- 172.21.15.55:0/1788661083 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb548149250 con 0x7fb548153480 2023-10-06T03:45:23.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.435+0000 7fb54e39d700 1 -- 172.21.15.55:0/1788661083 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5480c82f0 con 0x7fb548153480 2023-10-06T03:45:23.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.436+0000 7fb53d7fa700 1 -- 172.21.15.55:0/1788661083 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fb544022030 con 0x7fb548153480 2023-10-06T03:45:23.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.437+0000 7fb53d7fa700 1 --2- 172.21.15.55:0/1788661083 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb52c067130 0x7fb52c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:23.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.437+0000 7fb53d7fa700 1 -- 172.21.15.55:0/1788661083 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fb544086b80 con 0x7fb548153480 2023-10-06T03:45:23.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.438+0000 7fb54cb9a700 1 --2- 172.21.15.55:0/1788661083 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb52c067130 0x7fb52c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:23.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.438+0000 7fb54cb9a700 1 --2- 172.21.15.55:0/1788661083 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb52c067130 0x7fb52c0695e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb538007fd0 tx=0x7fb538007b20 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:23.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.441+0000 7fb53d7fa700 1 -- 172.21.15.55:0/1788661083 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb544053850 con 0x7fb548153480 2023-10-06T03:45:23.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.671+0000 7fb54e39d700 1 -- 172.21.15.55:0/1788661083 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5480c82f0 con 0x7fb52c067130 2023-10-06T03:45:23.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.750+0000 7fb53d7fa700 1 -- 172.21.15.55:0/1788661083 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fb5480c82f0 con 0x7fb52c067130 2023-10-06T03:45:23.753 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-10-06T03:45:23.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.752+0000 7fb54e39d700 1 -- 172.21.15.55:0/1788661083 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb52c067130 msgr2=0x7fb52c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:23.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.752+0000 7fb54e39d700 1 --2- 172.21.15.55:0/1788661083 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb52c067130 0x7fb52c0695e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb538007fd0 tx=0x7fb538007b20 comp rx=0 tx=0).stop 2023-10-06T03:45:23.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.752+0000 7fb54e39d700 1 -- 172.21.15.55:0/1788661083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb548153480 msgr2=0x7fb54815a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:23.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.752+0000 7fb54e39d700 1 --2- 172.21.15.55:0/1788661083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb548153480 0x7fb54815a790 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb54400c010 tx=0x7fb544016000 comp rx=0 tx=0).stop 2023-10-06T03:45:23.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.753+0000 7fb54e39d700 1 -- 172.21.15.55:0/1788661083 shutdown_connections 2023-10-06T03:45:23.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.753+0000 7fb54e39d700 1 --2- 172.21.15.55:0/1788661083 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb548159dc0 0x7fb548149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.753+0000 7fb54e39d700 1 --2- 172.21.15.55:0/1788661083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb548153480 0x7fb54815a790 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.753+0000 7fb54e39d700 1 --2- 172.21.15.55:0/1788661083 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb52c067130 0x7fb52c0695e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.753+0000 7fb54e39d700 1 --2- 172.21.15.55:0/1788661083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb548153850 0x7fb5480c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.753+0000 7fb54e39d700 1 -- 172.21.15.55:0/1788661083 >> 172.21.15.55:0/1788661083 conn(0x7fb548068e20 msgr2=0x7fb54806dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:23.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.753+0000 7fb54e39d700 1 -- 172.21.15.55:0/1788661083 shutdown_connections 2023-10-06T03:45:23.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.753+0000 7fb54e39d700 1 -- 172.21.15.55:0/1788661083 wait complete. 2023-10-06T03:45:23.774 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-10-06T03:45:23.774 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir6 1M 2023-10-06T03:45:23.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.964+0000 7f0c5f19d700 1 Processor -- start 2023-10-06T03:45:23.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.964+0000 7f0c5f19d700 1 -- start start 2023-10-06T03:45:23.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.965+0000 7f0c5f19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c5815b050 0x7f0c580ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:23.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5f19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0c580cc8d0 0x7f0c5815b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:23.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5f19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c58157a50 con 0x7f0c5815b960 2023-10-06T03:45:23.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5f19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c581591b0 con 0x7f0c580cc8d0 2023-10-06T03:45:23.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5f19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580ca9e0 con 0x7f0c5815b050 2023-10-06T03:45:23.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5d99a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0c580cc8d0 0x7f0c5815b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:23.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5d99a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0c580cc8d0 0x7f0c5815b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54664/0 (socket says 172.21.15.55:54664) 2023-10-06T03:45:23.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5d99a700 1 -- 172.21.15.55:0/2391847810 learned_addr learned my addr 172.21.15.55:0/2391847810 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:23.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5e19b700 1 --2- 172.21.15.55:0/2391847810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c5815b050 0x7f0c580ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:23.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5d99a700 1 -- 172.21.15.55:0/2391847810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c5815b050 msgr2=0x7f0c580ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:23.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5d99a700 1 --2- 172.21.15.55:0/2391847810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c5815b050 0x7f0c580ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5d99a700 1 -- 172.21.15.55:0/2391847810 >> v1:172.21.15.163:6789/0 conn(0x7f0c5815b960 legacy=0x7f0c58157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:23.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5d99a700 1 -- 172.21.15.55:0/2391847810 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c580cacc0 con 0x7f0c580cc8d0 2023-10-06T03:45:23.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5d99a700 1 --2- 172.21.15.55:0/2391847810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c580cc8d0 0x7f0c5815b420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0c480099b0 tx=0x7f0c48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc54e35080f828c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:23.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5d199700 1 -- 172.21.15.55:0/2391847810 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4800b860 con 0x7f0c580cc8d0 2023-10-06T03:45:23.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5d199700 1 -- 172.21.15.55:0/2391847810 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4800b9c0 con 0x7f0c580cc8d0 2023-10-06T03:45:23.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5d199700 1 -- 172.21.15.55:0/2391847810 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4800bd60 con 0x7f0c580cc8d0 2023-10-06T03:45:23.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5f19d700 1 -- 172.21.15.55:0/2391847810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c580cc8d0 msgr2=0x7f0c5815b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:23.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5f19d700 1 --2- 172.21.15.55:0/2391847810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c580cc8d0 0x7f0c5815b420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0c480099b0 tx=0x7f0c48009250 comp rx=0 tx=0).stop 2023-10-06T03:45:23.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5f19d700 1 -- 172.21.15.55:0/2391847810 shutdown_connections 2023-10-06T03:45:23.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5f19d700 1 --2- 172.21.15.55:0/2391847810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c5815b050 0x7f0c580ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5f19d700 1 --2- 172.21.15.55:0/2391847810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c580cc8d0 0x7f0c5815b420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.967+0000 7f0c5f19d700 1 -- 172.21.15.55:0/2391847810 >> 172.21.15.55:0/2391847810 conn(0x7f0c58068e20 msgr2=0x7f0c580cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:23.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.969+0000 7f0c5f19d700 1 -- 172.21.15.55:0/2391847810 shutdown_connections 2023-10-06T03:45:23.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.969+0000 7f0c5f19d700 1 -- 172.21.15.55:0/2391847810 wait complete. 2023-10-06T03:45:23.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.970+0000 7f0c5f19d700 1 Processor -- start 2023-10-06T03:45:23.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.970+0000 7f0c5f19d700 1 -- start start 2023-10-06T03:45:23.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5f19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c580cc8d0 0x7f0c580b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:23.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5f19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c5815b050 0x7f0c581537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:23.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5f19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c5815b960 0x7f0c5816d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:23.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5f19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580c6820 con 0x7f0c5815b960 2023-10-06T03:45:23.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5f19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c580c62f0 con 0x7f0c5815b050 2023-10-06T03:45:23.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5f19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5815edd0 con 0x7f0c580cc8d0 2023-10-06T03:45:23.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5e99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c5815b960 0x7f0c5816d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:23.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5e99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c5815b960 0x7f0c5816d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54680/0 (socket says 172.21.15.55:54680) 2023-10-06T03:45:23.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5e99c700 1 -- 172.21.15.55:0/1680308861 learned_addr learned my addr 172.21.15.55:0/1680308861 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:23.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5e19b700 1 --2- 172.21.15.55:0/1680308861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c580cc8d0 0x7f0c580b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:23.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5d99a700 1 --2- 172.21.15.55:0/1680308861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c5815b050 0x7f0c581537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:23.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5e99c700 1 -- 172.21.15.55:0/1680308861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c580cc8d0 msgr2=0x7f0c580b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:23.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5e99c700 1 --2- 172.21.15.55:0/1680308861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c580cc8d0 0x7f0c580b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5e99c700 1 -- 172.21.15.55:0/1680308861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c5815b050 msgr2=0x7f0c581537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:23.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5e99c700 1 --2- 172.21.15.55:0/1680308861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c5815b050 0x7f0c581537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:23.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5e99c700 1 -- 172.21.15.55:0/1680308861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c480094f0 con 0x7f0c5815b960 2023-10-06T03:45:23.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5e99c700 1 --2- 172.21.15.55:0/1680308861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c5815b960 0x7f0c5816d170 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000c8e0 tx=0x7f0c50001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:23.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c4e7fc700 1 -- 172.21.15.55:0/1680308861 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50012630 con 0x7f0c5815b960 2023-10-06T03:45:23.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c4e7fc700 1 -- 172.21.15.55:0/1680308861 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c50012790 con 0x7f0c5815b960 2023-10-06T03:45:23.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5f19d700 1 -- 172.21.15.55:0/1680308861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c580c6010 con 0x7f0c5815b960 2023-10-06T03:45:23.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c4e7fc700 1 -- 172.21.15.55:0/1680308861 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c50012b50 con 0x7f0c5815b960 2023-10-06T03:45:23.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.971+0000 7f0c5f19d700 1 -- 172.21.15.55:0/1680308861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5815eae0 con 0x7f0c5815b960 2023-10-06T03:45:23.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.973+0000 7f0c5f19d700 1 -- 172.21.15.55:0/1680308861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5816cfe0 con 0x7f0c5815b960 2023-10-06T03:45:23.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.973+0000 7f0c4e7fc700 1 -- 172.21.15.55:0/1680308861 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f0c5001a3a0 con 0x7f0c5815b960 2023-10-06T03:45:23.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.975+0000 7f0c4e7fc700 1 --2- 172.21.15.55:0/1680308861 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0c3c067130 0x7f0c3c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:23.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.975+0000 7f0c4e7fc700 1 -- 172.21.15.55:0/1680308861 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f0c50084d60 con 0x7f0c5815b960 2023-10-06T03:45:23.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.975+0000 7f0c5e19b700 1 --2- 172.21.15.55:0/1680308861 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0c3c067130 0x7f0c3c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:23.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.976+0000 7f0c5e19b700 1 --2- 172.21.15.55:0/1680308861 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0c3c067130 0x7f0c3c0695e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0c54006fd0 tx=0x7f0c54008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:23.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:23.979+0000 7f0c4e7fc700 1 -- 172.21.15.55:0/1680308861 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c50051a30 con 0x7f0c5815b960 2023-10-06T03:45:24.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.208+0000 7f0c5f19d700 1 -- 172.21.15.55:0/1680308861 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c5804c3c0 con 0x7f0c3c067130 2023-10-06T03:45:24.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.281+0000 7f0c4e7fc700 1 -- 172.21.15.55:0/1680308861 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f0c5804c3c0 con 0x7f0c3c067130 2023-10-06T03:45:24.283 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-10-06T03:45:24.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.284+0000 7f0c5f19d700 1 -- 172.21.15.55:0/1680308861 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0c3c067130 msgr2=0x7f0c3c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:24.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.284+0000 7f0c5f19d700 1 --2- 172.21.15.55:0/1680308861 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0c3c067130 0x7f0c3c0695e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0c54006fd0 tx=0x7f0c54008040 comp rx=0 tx=0).stop 2023-10-06T03:45:24.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.284+0000 7f0c5f19d700 1 -- 172.21.15.55:0/1680308861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c5815b960 msgr2=0x7f0c5816d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:24.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.284+0000 7f0c5f19d700 1 --2- 172.21.15.55:0/1680308861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c5815b960 0x7f0c5816d170 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000c8e0 tx=0x7f0c50001710 comp rx=0 tx=0).stop 2023-10-06T03:45:24.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.284+0000 7f0c5f19d700 1 -- 172.21.15.55:0/1680308861 shutdown_connections 2023-10-06T03:45:24.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.284+0000 7f0c5f19d700 1 --2- 172.21.15.55:0/1680308861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c5815b960 0x7f0c5816d170 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.284+0000 7f0c5f19d700 1 --2- 172.21.15.55:0/1680308861 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0c3c067130 0x7f0c3c0695e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.284+0000 7f0c5f19d700 1 --2- 172.21.15.55:0/1680308861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c580cc8d0 0x7f0c580b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.284+0000 7f0c5f19d700 1 --2- 172.21.15.55:0/1680308861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c5815b050 0x7f0c581537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.284+0000 7f0c5f19d700 1 -- 172.21.15.55:0/1680308861 >> 172.21.15.55:0/1680308861 conn(0x7f0c58068e20 msgr2=0x7f0c5806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:24.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.284+0000 7f0c5f19d700 1 -- 172.21.15.55:0/1680308861 shutdown_connections 2023-10-06T03:45:24.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.284+0000 7f0c5f19d700 1 -- 172.21.15.55:0/1680308861 wait complete. 2023-10-06T03:45:24.303 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-10-06T03:45:24.303 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir7 1M 2023-10-06T03:45:24.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.496+0000 7f0a2a79d700 1 Processor -- start 2023-10-06T03:45:24.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.496+0000 7f0a2a79d700 1 -- start start 2023-10-06T03:45:24.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.497+0000 7f0a2a79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a2415b050 0x7f0a240ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:24.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.497+0000 7f0a2a79d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0a240cc8d0 0x7f0a2415b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:24.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.497+0000 7f0a2a79d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a24157a50 con 0x7f0a2415b960 2023-10-06T03:45:24.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.497+0000 7f0a2a79d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a241591b0 con 0x7f0a240cc8d0 2023-10-06T03:45:24.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.497+0000 7f0a2a79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240ca9e0 con 0x7f0a2415b050 2023-10-06T03:45:24.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.498+0000 7f0a28f9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0a240cc8d0 0x7f0a2415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:24.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.498+0000 7f0a29f9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0a2415b960 0x7f0a24157f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35404/0 (socket says 172.21.15.55:35404) 2023-10-06T03:45:24.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.498+0000 7f0a29f9c700 1 -- 172.21.15.55:0/604628367 learned_addr learned my addr 172.21.15.55:0/604628367 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:24.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.498+0000 7f0a2979b700 1 --2- 172.21.15.55:0/604628367 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a2415b050 0x7f0a240ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:24.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.498+0000 7f0a1bfff700 1 -- 172.21.15.55:0/604628367 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022784795 0 0) 0x7f0a24157a50 con 0x7f0a2415b960 2023-10-06T03:45:24.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.498+0000 7f0a1bfff700 1 -- 172.21.15.55:0/604628367 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a10003570 con 0x7f0a2415b960 2023-10-06T03:45:24.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.498+0000 7f0a28f9a700 1 -- 172.21.15.55:0/604628367 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a2415b050 msgr2=0x7f0a240ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:24.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.498+0000 7f0a28f9a700 1 --2- 172.21.15.55:0/604628367 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a2415b050 0x7f0a240ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.498+0000 7f0a28f9a700 1 -- 172.21.15.55:0/604628367 >> v1:172.21.15.55:6789/0 conn(0x7f0a2415b960 legacy=0x7f0a24157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:24.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.498+0000 7f0a28f9a700 1 -- 172.21.15.55:0/604628367 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a240cacc0 con 0x7f0a240cc8d0 2023-10-06T03:45:24.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.499+0000 7f0a28f9a700 1 --2- 172.21.15.55:0/604628367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a240cc8d0 0x7f0a2415b420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0a140098e0 tx=0x7f0a140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe82047219f648e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.499+0000 7f0a1bfff700 1 -- 172.21.15.55:0/604628367 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14011630 con 0x7f0a240cc8d0 2023-10-06T03:45:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.499+0000 7f0a1bfff700 1 -- 172.21.15.55:0/604628367 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a14011790 con 0x7f0a240cc8d0 2023-10-06T03:45:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.499+0000 7f0a1bfff700 1 -- 172.21.15.55:0/604628367 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14011b30 con 0x7f0a240cc8d0 2023-10-06T03:45:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.499+0000 7f0a2a79d700 1 -- 172.21.15.55:0/604628367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a240cc8d0 msgr2=0x7f0a2415b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.499+0000 7f0a2a79d700 1 --2- 172.21.15.55:0/604628367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a240cc8d0 0x7f0a2415b420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0a140098e0 tx=0x7f0a140091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.499+0000 7f0a2a79d700 1 -- 172.21.15.55:0/604628367 shutdown_connections 2023-10-06T03:45:24.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.499+0000 7f0a2a79d700 1 --2- 172.21.15.55:0/604628367 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a2415b050 0x7f0a240ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.499+0000 7f0a2a79d700 1 --2- 172.21.15.55:0/604628367 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a240cc8d0 0x7f0a2415b420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.499+0000 7f0a2a79d700 1 -- 172.21.15.55:0/604628367 >> 172.21.15.55:0/604628367 conn(0x7f0a24068e20 msgr2=0x7f0a240cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.500+0000 7f0a2a79d700 1 -- 172.21.15.55:0/604628367 shutdown_connections 2023-10-06T03:45:24.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.500+0000 7f0a2a79d700 1 -- 172.21.15.55:0/604628367 wait complete. 2023-10-06T03:45:24.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.501+0000 7f0a2a79d700 1 Processor -- start 2023-10-06T03:45:24.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.501+0000 7f0a2a79d700 1 -- start start 2023-10-06T03:45:24.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.502+0000 7f0a2a79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a240cc8d0 0x7f0a240c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:24.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.502+0000 7f0a2a79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a2415b050 0x7f0a24142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:24.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.502+0000 7f0a2a79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a2415b960 0x7f0a240c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:24.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.502+0000 7f0a2a79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240c2b30 con 0x7f0a2415b960 2023-10-06T03:45:24.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.502+0000 7f0a2a79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a240c3cc0 con 0x7f0a240cc8d0 2023-10-06T03:45:24.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.502+0000 7f0a2a79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a241432b0 con 0x7f0a2415b050 2023-10-06T03:45:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.502+0000 7f0a29f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a2415b960 0x7f0a240c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.503+0000 7f0a2979b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a240cc8d0 0x7f0a240c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.502+0000 7f0a29f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a2415b960 0x7f0a240c1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54698/0 (socket says 172.21.15.55:54698) 2023-10-06T03:45:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.503+0000 7f0a29f9c700 1 -- 172.21.15.55:0/2646607410 learned_addr learned my addr 172.21.15.55:0/2646607410 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.503+0000 7f0a28f9a700 1 --2- 172.21.15.55:0/2646607410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a2415b050 0x7f0a24142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.503+0000 7f0a29f9c700 1 -- 172.21.15.55:0/2646607410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a2415b050 msgr2=0x7f0a24142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:24.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.503+0000 7f0a29f9c700 1 --2- 172.21.15.55:0/2646607410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a2415b050 0x7f0a24142d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.503+0000 7f0a29f9c700 1 -- 172.21.15.55:0/2646607410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a240cc8d0 msgr2=0x7f0a240c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:24.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.503+0000 7f0a29f9c700 1 --2- 172.21.15.55:0/2646607410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a240cc8d0 0x7f0a240c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.503+0000 7f0a29f9c700 1 -- 172.21.15.55:0/2646607410 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a14009420 con 0x7f0a2415b960 2023-10-06T03:45:24.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.503+0000 7f0a29f9c700 1 --2- 172.21.15.55:0/2646607410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a2415b960 0x7f0a240c1a80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c001460 tx=0x7f0a1c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:24.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.503+0000 7f0a197fa700 1 -- 172.21.15.55:0/2646607410 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c00a890 con 0x7f0a2415b960 2023-10-06T03:45:24.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.503+0000 7f0a197fa700 1 -- 172.21.15.55:0/2646607410 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1c00a9f0 con 0x7f0a2415b960 2023-10-06T03:45:24.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.503+0000 7f0a2a79d700 1 -- 172.21.15.55:0/2646607410 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a240c1fc0 con 0x7f0a2415b960 2023-10-06T03:45:24.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.503+0000 7f0a197fa700 1 -- 172.21.15.55:0/2646607410 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c00ad70 con 0x7f0a2415b960 2023-10-06T03:45:24.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.504+0000 7f0a2a79d700 1 -- 172.21.15.55:0/2646607410 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a240c2570 con 0x7f0a2415b960 2023-10-06T03:45:24.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.505+0000 7f0a2a79d700 1 -- 172.21.15.55:0/2646607410 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a240c3a90 con 0x7f0a2415b960 2023-10-06T03:45:24.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.506+0000 7f0a197fa700 1 -- 172.21.15.55:0/2646607410 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f0a1c007300 con 0x7f0a2415b960 2023-10-06T03:45:24.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.507+0000 7f0a197fa700 1 --2- 172.21.15.55:0/2646607410 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0a08067160 0x7f0a08069610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:24.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.507+0000 7f0a197fa700 1 -- 172.21.15.55:0/2646607410 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f0a1c084fa0 con 0x7f0a2415b960 2023-10-06T03:45:24.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.507+0000 7f0a2979b700 1 --2- 172.21.15.55:0/2646607410 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0a08067160 0x7f0a08069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:24.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.508+0000 7f0a2979b700 1 --2- 172.21.15.55:0/2646607410 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0a08067160 0x7f0a08069610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0a20010e50 tx=0x7f0a20012040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:24.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.511+0000 7f0a197fa700 1 -- 172.21.15.55:0/2646607410 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a240c3a90 con 0x7f0a2415b960 2023-10-06T03:45:24.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.745+0000 7f0a2a79d700 1 -- 172.21.15.55:0/2646607410 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a240c3a90 con 0x7f0a08067160 2023-10-06T03:45:24.824 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.823+0000 7f0a197fa700 1 -- 172.21.15.55:0/2646607410 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f0a240c3a90 con 0x7f0a08067160 2023-10-06T03:45:24.824 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-10-06T03:45:24.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.825+0000 7f0a2a79d700 1 -- 172.21.15.55:0/2646607410 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0a08067160 msgr2=0x7f0a08069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:24.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.825+0000 7f0a2a79d700 1 --2- 172.21.15.55:0/2646607410 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0a08067160 0x7f0a08069610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0a20010e50 tx=0x7f0a20012040 comp rx=0 tx=0).stop 2023-10-06T03:45:24.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.825+0000 7f0a2a79d700 1 -- 172.21.15.55:0/2646607410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a2415b960 msgr2=0x7f0a240c1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:24.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.825+0000 7f0a2a79d700 1 --2- 172.21.15.55:0/2646607410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a2415b960 0x7f0a240c1a80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c001460 tx=0x7f0a1c001ae0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.825+0000 7f0a2a79d700 1 -- 172.21.15.55:0/2646607410 shutdown_connections 2023-10-06T03:45:24.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.825+0000 7f0a2a79d700 1 --2- 172.21.15.55:0/2646607410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a2415b960 0x7f0a240c1a80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.825+0000 7f0a2a79d700 1 --2- 172.21.15.55:0/2646607410 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a240cc8d0 0x7f0a240c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.825+0000 7f0a2a79d700 1 --2- 172.21.15.55:0/2646607410 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0a08067160 0x7f0a08069610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.825+0000 7f0a2a79d700 1 --2- 172.21.15.55:0/2646607410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a2415b050 0x7f0a24142d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:24.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.825+0000 7f0a2a79d700 1 -- 172.21.15.55:0/2646607410 >> 172.21.15.55:0/2646607410 conn(0x7f0a24068e20 msgr2=0x7f0a2406d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:24.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.825+0000 7f0a2a79d700 1 -- 172.21.15.55:0/2646607410 shutdown_connections 2023-10-06T03:45:24.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:24.825+0000 7f0a2a79d700 1 -- 172.21.15.55:0/2646607410 wait complete. 2023-10-06T03:45:24.844 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-10-06T03:45:24.844 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir8 1M 2023-10-06T03:45:25.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.035+0000 7fed29f9d700 1 Processor -- start 2023-10-06T03:45:25.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.035+0000 7fed29f9d700 1 -- start start 2023-10-06T03:45:25.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed29f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fed24153480 0x7fed240ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:25.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed29f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed24153850 0x7fed241596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:25.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed29f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fed24159bf0 0x7fed240cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:25.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed29f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2415a050 con 0x7fed24159bf0 2023-10-06T03:45:25.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed29f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed240aec90 con 0x7fed24153480 2023-10-06T03:45:25.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed29f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2415a480 con 0x7fed24153850 2023-10-06T03:45:25.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed2979c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fed24159bf0 0x7fed240cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:25.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed2979c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fed24159bf0 0x7fed240cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54712/0 (socket says 172.21.15.55:54712) 2023-10-06T03:45:25.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed2979c700 1 -- 172.21.15.55:0/3788585970 learned_addr learned my addr 172.21.15.55:0/3788585970 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:25.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed28f9b700 1 --2- 172.21.15.55:0/3788585970 >> v2:172.21.15.163:3300/0 conn(0x7fed24153480 0x7fed240ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:25.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed23fff700 1 --2- 172.21.15.55:0/3788585970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed24153850 0x7fed241596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:25.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed2979c700 1 -- 172.21.15.55:0/3788585970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed24153850 msgr2=0x7fed241596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:25.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed2979c700 1 --2- 172.21.15.55:0/3788585970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed24153850 0x7fed241596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed2979c700 1 -- 172.21.15.55:0/3788585970 >> v2:172.21.15.163:3300/0 conn(0x7fed24153480 msgr2=0x7fed240ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:25.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed2979c700 1 --2- 172.21.15.55:0/3788585970 >> v2:172.21.15.163:3300/0 conn(0x7fed24153480 0x7fed240ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.037+0000 7fed2979c700 1 -- 172.21.15.55:0/3788585970 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed240afb40 con 0x7fed24159bf0 2023-10-06T03:45:25.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.038+0000 7fed2979c700 1 --2- 172.21.15.55:0/3788585970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed24159bf0 0x7fed240cc8d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fed2406cf00 tx=0x7fed1c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131784aa8dafee05 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:25.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.038+0000 7fed237fe700 1 -- 172.21.15.55:0/3788585970 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c00e070 con 0x7fed24159bf0 2023-10-06T03:45:25.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.038+0000 7fed237fe700 1 -- 172.21.15.55:0/3788585970 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1c00b970 con 0x7fed24159bf0 2023-10-06T03:45:25.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.038+0000 7fed237fe700 1 -- 172.21.15.55:0/3788585970 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0092e0 con 0x7fed24159bf0 2023-10-06T03:45:25.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.038+0000 7fed29f9d700 1 -- 172.21.15.55:0/3788585970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed24159bf0 msgr2=0x7fed240cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:25.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.038+0000 7fed29f9d700 1 --2- 172.21.15.55:0/3788585970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed24159bf0 0x7fed240cc8d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fed2406cf00 tx=0x7fed1c009840 comp rx=0 tx=0).stop 2023-10-06T03:45:25.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.039+0000 7fed29f9d700 1 -- 172.21.15.55:0/3788585970 shutdown_connections 2023-10-06T03:45:25.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.039+0000 7fed29f9d700 1 --2- 172.21.15.55:0/3788585970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed24159bf0 0x7fed240cc8d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.039+0000 7fed29f9d700 1 --2- 172.21.15.55:0/3788585970 >> v2:172.21.15.163:3300/0 conn(0x7fed24153480 0x7fed240ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.039+0000 7fed29f9d700 1 --2- 172.21.15.55:0/3788585970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed24153850 0x7fed241596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.039+0000 7fed29f9d700 1 -- 172.21.15.55:0/3788585970 >> 172.21.15.55:0/3788585970 conn(0x7fed24068e20 msgr2=0x7fed240cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:25.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.039+0000 7fed29f9d700 1 -- 172.21.15.55:0/3788585970 shutdown_connections 2023-10-06T03:45:25.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.039+0000 7fed29f9d700 1 -- 172.21.15.55:0/3788585970 wait complete. 2023-10-06T03:45:25.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.040+0000 7fed29f9d700 1 Processor -- start 2023-10-06T03:45:25.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.040+0000 7fed29f9d700 1 -- start start 2023-10-06T03:45:25.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.041+0000 7fed29f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed24153480 0x7fed24142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:25.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.041+0000 7fed29f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed24153850 0x7fed240a7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:25.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.041+0000 7fed29f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed24159bf0 0x7fed240a8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:25.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.041+0000 7fed29f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2416a9c0 con 0x7fed24153480 2023-10-06T03:45:25.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.041+0000 7fed29f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed24169b40 con 0x7fed24159bf0 2023-10-06T03:45:25.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.041+0000 7fed29f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed24169db0 con 0x7fed24153850 2023-10-06T03:45:25.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.041+0000 7fed28f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed24153480 0x7fed24142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:25.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.041+0000 7fed28f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed24153480 0x7fed24142bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54714/0 (socket says 172.21.15.55:54714) 2023-10-06T03:45:25.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.041+0000 7fed28f9b700 1 -- 172.21.15.55:0/983909637 learned_addr learned my addr 172.21.15.55:0/983909637 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:25.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.042+0000 7fed2979c700 1 --2- 172.21.15.55:0/983909637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed24159bf0 0x7fed240a8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:25.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.042+0000 7fed23fff700 1 --2- 172.21.15.55:0/983909637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed24153850 0x7fed240a7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:25.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.042+0000 7fed28f9b700 1 -- 172.21.15.55:0/983909637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed24153850 msgr2=0x7fed240a7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:25.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.042+0000 7fed28f9b700 1 --2- 172.21.15.55:0/983909637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed24153850 0x7fed240a7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.042+0000 7fed28f9b700 1 -- 172.21.15.55:0/983909637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed24159bf0 msgr2=0x7fed240a8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:25.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.042+0000 7fed28f9b700 1 --2- 172.21.15.55:0/983909637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed24159bf0 0x7fed240a8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.042+0000 7fed28f9b700 1 -- 172.21.15.55:0/983909637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c009ae0 con 0x7fed24153480 2023-10-06T03:45:25.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.042+0000 7fed28f9b700 1 --2- 172.21.15.55:0/983909637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed24153480 0x7fed24142bc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fed1800e930 tx=0x7fed180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:25.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.042+0000 7fed20ff9700 1 -- 172.21.15.55:0/983909637 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18013680 con 0x7fed24153480 2023-10-06T03:45:25.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.042+0000 7fed20ff9700 1 -- 172.21.15.55:0/983909637 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed180137e0 con 0x7fed24153480 2023-10-06T03:45:25.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.042+0000 7fed20ff9700 1 -- 172.21.15.55:0/983909637 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18013b80 con 0x7fed24153480 2023-10-06T03:45:25.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.042+0000 7fed29f9d700 1 -- 172.21.15.55:0/983909637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2416a090 con 0x7fed24153480 2023-10-06T03:45:25.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.043+0000 7fed29f9d700 1 -- 172.21.15.55:0/983909637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2416a650 con 0x7fed24153480 2023-10-06T03:45:25.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.044+0000 7fed29f9d700 1 -- 172.21.15.55:0/983909637 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed240cb9a0 con 0x7fed24153480 2023-10-06T03:45:25.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.044+0000 7fed20ff9700 1 -- 172.21.15.55:0/983909637 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fed1800c3e0 con 0x7fed24153480 2023-10-06T03:45:25.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.046+0000 7fed20ff9700 1 --2- 172.21.15.55:0/983909637 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fed08067130 0x7fed080695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:25.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.046+0000 7fed20ff9700 1 -- 172.21.15.55:0/983909637 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fed18084c30 con 0x7fed24153480 2023-10-06T03:45:25.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.046+0000 7fed23fff700 1 --2- 172.21.15.55:0/983909637 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fed08067130 0x7fed080695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:25.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.047+0000 7fed23fff700 1 --2- 172.21.15.55:0/983909637 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fed08067130 0x7fed080695e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed240a9bf0 tx=0x7fed100091a0 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:25.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.050+0000 7fed20ff9700 1 -- 172.21.15.55:0/983909637 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed18051900 con 0x7fed24153480 2023-10-06T03:45:25.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.282+0000 7fed29f9d700 1 -- 172.21.15.55:0/983909637 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fed2416ab00 con 0x7fed08067130 2023-10-06T03:45:25.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.357+0000 7fed20ff9700 1 -- 172.21.15.55:0/983909637 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fed10009ad0 con 0x7fed08067130 2023-10-06T03:45:25.358 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-10-06T03:45:25.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.359+0000 7fed29f9d700 1 -- 172.21.15.55:0/983909637 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fed08067130 msgr2=0x7fed080695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:25.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.359+0000 7fed29f9d700 1 --2- 172.21.15.55:0/983909637 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fed08067130 0x7fed080695e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fed240a9bf0 tx=0x7fed100091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.359+0000 7fed29f9d700 1 -- 172.21.15.55:0/983909637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed24153480 msgr2=0x7fed24142bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:25.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.359+0000 7fed29f9d700 1 --2- 172.21.15.55:0/983909637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed24153480 0x7fed24142bc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fed1800e930 tx=0x7fed180091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.359+0000 7fed29f9d700 1 -- 172.21.15.55:0/983909637 shutdown_connections 2023-10-06T03:45:25.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.359+0000 7fed29f9d700 1 --2- 172.21.15.55:0/983909637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed24159bf0 0x7fed240a8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.359+0000 7fed29f9d700 1 --2- 172.21.15.55:0/983909637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed24153480 0x7fed24142bc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.359+0000 7fed29f9d700 1 --2- 172.21.15.55:0/983909637 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fed08067130 0x7fed080695e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.359+0000 7fed29f9d700 1 --2- 172.21.15.55:0/983909637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed24153850 0x7fed240a7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.359+0000 7fed29f9d700 1 -- 172.21.15.55:0/983909637 >> 172.21.15.55:0/983909637 conn(0x7fed24068e20 msgr2=0x7fed240cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:25.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.359+0000 7fed29f9d700 1 -- 172.21.15.55:0/983909637 shutdown_connections 2023-10-06T03:45:25.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.359+0000 7fed29f9d700 1 -- 172.21.15.55:0/983909637 wait complete. 2023-10-06T03:45:25.378 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-10-06T03:45:25.378 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir9 1M 2023-10-06T03:45:25.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.570+0000 7f8e85d9d700 1 Processor -- start 2023-10-06T03:45:25.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.571+0000 7f8e85d9d700 1 -- start start 2023-10-06T03:45:25.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.572+0000 7f8e85d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e800b03c0 0x7f8e800cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:25.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.572+0000 7f8e85d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e800cfa00 con 0x7f8e801591e0 2023-10-06T03:45:25.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.572+0000 7f8e85d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e800cb580 con 0x7f8e800b0790 2023-10-06T03:45:25.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.572+0000 7f8e85d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e800cf160 con 0x7f8e800b03c0 2023-10-06T03:45:25.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.572+0000 7f8e84d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e800b03c0 0x7f8e800cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:25.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.572+0000 7f8e8559c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8e801591e0 0x7f8e80159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35408/0 (socket says 172.21.15.55:35408) 2023-10-06T03:45:25.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.572+0000 7f8e8559c700 1 -- 172.21.15.55:0/240936161 learned_addr learned my addr 172.21.15.55:0/240936161 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:25.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.572+0000 7f8e84d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8e800b03c0 0x7f8e800cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54716/0 (socket says 172.21.15.55:54716) 2023-10-06T03:45:25.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.573+0000 7f8e84d9b700 1 -- 172.21.15.55:0/240936161 >> v1:172.21.15.163:6789/0 conn(0x7f8e800b0790 legacy=0x7f8e800b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:25.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.573+0000 7f8e84d9b700 1 -- 172.21.15.55:0/240936161 >> v1:172.21.15.55:6789/0 conn(0x7f8e801591e0 legacy=0x7f8e80159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:25.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.573+0000 7f8e84d9b700 1 -- 172.21.15.55:0/240936161 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e800cf440 con 0x7f8e800b03c0 2023-10-06T03:45:25.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.573+0000 7f8e7f7fe700 1 -- 172.21.15.55:0/240936161 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058488029 0 0) 0x7f8e800cfa00 con 0x7f8e801591e0 2023-10-06T03:45:25.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.573+0000 7f8e84d9b700 1 --2- 172.21.15.55:0/240936161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e800b03c0 0x7f8e800cea70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009910 tx=0x7f8e740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2cccebc7d05dbf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:25.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.573+0000 7f8e7f7fe700 1 -- 172.21.15.55:0/240936161 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400b820 con 0x7f8e800b03c0 2023-10-06T03:45:25.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.573+0000 7f8e7f7fe700 1 -- 172.21.15.55:0/240936161 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7400b980 con 0x7f8e800b03c0 2023-10-06T03:45:25.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.573+0000 7f8e7f7fe700 1 -- 172.21.15.55:0/240936161 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7400bd20 con 0x7f8e800b03c0 2023-10-06T03:45:25.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.574+0000 7f8e85d9d700 1 -- 172.21.15.55:0/240936161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e800b03c0 msgr2=0x7f8e800cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:25.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.574+0000 7f8e85d9d700 1 --2- 172.21.15.55:0/240936161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e800b03c0 0x7f8e800cea70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8e74009910 tx=0x7f8e740091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.574+0000 7f8e85d9d700 1 -- 172.21.15.55:0/240936161 shutdown_connections 2023-10-06T03:45:25.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.574+0000 7f8e85d9d700 1 --2- 172.21.15.55:0/240936161 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e800b03c0 0x7f8e800cea70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.574+0000 7f8e85d9d700 1 -- 172.21.15.55:0/240936161 >> 172.21.15.55:0/240936161 conn(0x7f8e80063230 msgr2=0x7f8e80063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:25.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.574+0000 7f8e85d9d700 1 -- 172.21.15.55:0/240936161 shutdown_connections 2023-10-06T03:45:25.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.574+0000 7f8e85d9d700 1 -- 172.21.15.55:0/240936161 wait complete. 2023-10-06T03:45:25.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.575+0000 7f8e85d9d700 1 Processor -- start 2023-10-06T03:45:25.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.575+0000 7f8e85d9d700 1 -- start start 2023-10-06T03:45:25.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.576+0000 7f8e85d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e800b03c0 0x7f8e80155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:25.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.576+0000 7f8e85d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e800b0790 0x7f8e8015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:25.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.576+0000 7f8e85d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e801591e0 0x7f8e800ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:25.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.576+0000 7f8e85d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e800c85f0 con 0x7f8e800b0790 2023-10-06T03:45:25.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.576+0000 7f8e85d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e801558c0 con 0x7f8e800b03c0 2023-10-06T03:45:25.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.576+0000 7f8e85d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8015c6c0 con 0x7f8e801591e0 2023-10-06T03:45:25.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.577+0000 7f8e7ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e800b0790 0x7f8e8015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:25.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.577+0000 7f8e7ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e800b0790 0x7f8e8015bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54728/0 (socket says 172.21.15.55:54728) 2023-10-06T03:45:25.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.577+0000 7f8e7ffff700 1 -- 172.21.15.55:0/2109612547 learned_addr learned my addr 172.21.15.55:0/2109612547 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:25.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.577+0000 7f8e84d9b700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e800b03c0 0x7f8e80155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:25.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.577+0000 7f8e8559c700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e801591e0 0x7f8e800ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:25.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.577+0000 7f8e7ffff700 1 -- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e801591e0 msgr2=0x7f8e800ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:25.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.577+0000 7f8e7ffff700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e801591e0 0x7f8e800ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.577+0000 7f8e7ffff700 1 -- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e800b03c0 msgr2=0x7f8e80155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:25.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.577+0000 7f8e7ffff700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e800b03c0 0x7f8e80155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.577+0000 7f8e7ffff700 1 -- 172.21.15.55:0/2109612547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e74009420 con 0x7f8e800b0790 2023-10-06T03:45:25.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.577+0000 7f8e84d9b700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e800b03c0 0x7f8e80155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:45:25.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.578+0000 7f8e7ffff700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e800b0790 0x7f8e8015bc20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00c590 tx=0x7f8e6c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:25.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.578+0000 7f8e7cff9700 1 -- 172.21.15.55:0/2109612547 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c011590 con 0x7f8e800b0790 2023-10-06T03:45:25.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.578+0000 7f8e7cff9700 1 -- 172.21.15.55:0/2109612547 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6c011710 con 0x7f8e800b0790 2023-10-06T03:45:25.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.578+0000 7f8e85d9d700 1 -- 172.21.15.55:0/2109612547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8015c9a0 con 0x7f8e800b0790 2023-10-06T03:45:25.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.578+0000 7f8e7cff9700 1 -- 172.21.15.55:0/2109612547 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c011ad0 con 0x7f8e800b0790 2023-10-06T03:45:25.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.578+0000 7f8e85d9d700 1 -- 172.21.15.55:0/2109612547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8015cf60 con 0x7f8e800b0790 2023-10-06T03:45:25.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.579+0000 7f8e85d9d700 1 -- 172.21.15.55:0/2109612547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e800c8760 con 0x7f8e800b0790 2023-10-06T03:45:25.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.580+0000 7f8e7cff9700 1 -- 172.21.15.55:0/2109612547 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f8e6c01b3b0 con 0x7f8e800b0790 2023-10-06T03:45:25.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.581+0000 7f8e7cff9700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8e64067130 0x7f8e640695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:25.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.581+0000 7f8e7cff9700 1 -- 172.21.15.55:0/2109612547 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f8e6c083fd0 con 0x7f8e800b0790 2023-10-06T03:45:25.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.581+0000 7f8e84d9b700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8e64067130 0x7f8e640695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:25.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.582+0000 7f8e84d9b700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8e64067130 0x7f8e640695e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e740093f0 tx=0x7f8e74017000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:25.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.585+0000 7f8e7cff9700 1 -- 172.21.15.55:0/2109612547 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e6c050ca0 con 0x7f8e800b0790 2023-10-06T03:45:25.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.820+0000 7f8e85d9d700 1 -- 172.21.15.55:0/2109612547 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e8004c3c0 con 0x7f8e64067130 2023-10-06T03:45:25.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.894+0000 7f8e7cff9700 1 -- 172.21.15.55:0/2109612547 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f8e8004c3c0 con 0x7f8e64067130 2023-10-06T03:45:25.896 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-10-06T03:45:25.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.896+0000 7f8e85d9d700 1 -- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8e64067130 msgr2=0x7f8e640695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:25.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.896+0000 7f8e85d9d700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8e64067130 0x7f8e640695e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8e740093f0 tx=0x7f8e74017000 comp rx=0 tx=0).stop 2023-10-06T03:45:25.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.896+0000 7f8e85d9d700 1 -- 172.21.15.55:0/2109612547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e800b0790 msgr2=0x7f8e8015bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:25.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.896+0000 7f8e85d9d700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e800b0790 0x7f8e8015bc20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00c590 tx=0x7f8e6c00c8a0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.896+0000 7f8e85d9d700 1 -- 172.21.15.55:0/2109612547 shutdown_connections 2023-10-06T03:45:25.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.896+0000 7f8e85d9d700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8e64067130 0x7f8e640695e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.896+0000 7f8e85d9d700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e801591e0 0x7f8e800ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.896+0000 7f8e85d9d700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e800b03c0 0x7f8e80155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.896+0000 7f8e85d9d700 1 --2- 172.21.15.55:0/2109612547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e800b0790 0x7f8e8015bc20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:25.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.896+0000 7f8e85d9d700 1 -- 172.21.15.55:0/2109612547 >> 172.21.15.55:0/2109612547 conn(0x7f8e80063230 msgr2=0x7f8e801588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:25.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.896+0000 7f8e85d9d700 1 -- 172.21.15.55:0/2109612547 shutdown_connections 2023-10-06T03:45:25.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:25.897+0000 7f8e85d9d700 1 -- 172.21.15.55:0/2109612547 wait complete. 2023-10-06T03:45:25.915 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-10-06T03:45:25.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:25.915 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-06T03:45:25.957 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:25.958 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:25.958 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:25.958 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:25.958 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:25.958 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:25.958 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:25.958 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:25.958 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:25.958 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:25.958 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:25.959 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-03_35_00_UTC', 'scheduled-2023-10-06-03_36_00_UTC', 'scheduled-2023-10-06-03_37_00_UTC', 'scheduled-2023-10-06-03_38_00_UTC', 'scheduled-2023-10-06-03_39_00_UTC', 'scheduled-2023-10-06-03_40_00_UTC', 'scheduled-2023-10-06-03_41_00_UTC', 'scheduled-2023-10-06-03_42_00_UTC', 'scheduled-2023-10-06-03_43_00_UTC', 'scheduled-2023-10-06-03_44_00_UTC', 'scheduled-2023-10-06-03_45_00_UTC'] 2023-10-06T03:45:25.959 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir0 2023-10-06T03:45:26.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.190+0000 7feb82b9d700 1 Processor -- start 2023-10-06T03:45:26.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.190+0000 7feb82b9d700 1 -- start start 2023-10-06T03:45:26.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.191+0000 7feb82b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feb7c0ac0e0 0x7feb7c0afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:26.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb82b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feb7c06f0b0 0x7feb7c15d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:26.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb82b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7c06e510 con 0x7feb7c1597d0 2023-10-06T03:45:26.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb82b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb7c069180 con 0x7feb7c0ac0e0 2023-10-06T03:45:26.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb82b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb7c06dec0 con 0x7feb7c06f0b0 2023-10-06T03:45:26.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb8139a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feb7c0ac0e0 0x7feb7c0afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:26.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb8139a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7feb7c0ac0e0 0x7feb7c0afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54730/0 (socket says 172.21.15.55:54730) 2023-10-06T03:45:26.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb8139a700 1 -- 172.21.15.55:0/794338088 learned_addr learned my addr 172.21.15.55:0/794338088 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:26.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb81b9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7feb7c1597d0 0x7feb7c06e9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35414/0 (socket says 172.21.15.55:35414) 2023-10-06T03:45:26.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb8239c700 1 --2- 172.21.15.55:0/794338088 >> v2:172.21.15.163:3300/0 conn(0x7feb7c06f0b0 0x7feb7c15d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:26.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb8139a700 1 -- 172.21.15.55:0/794338088 >> v2:172.21.15.163:3300/0 conn(0x7feb7c06f0b0 msgr2=0x7feb7c15d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:26.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb8139a700 1 --2- 172.21.15.55:0/794338088 >> v2:172.21.15.163:3300/0 conn(0x7feb7c06f0b0 0x7feb7c15d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb8139a700 1 -- 172.21.15.55:0/794338088 >> v1:172.21.15.55:6789/0 conn(0x7feb7c1597d0 legacy=0x7feb7c06e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:26.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb8139a700 1 -- 172.21.15.55:0/794338088 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c06e1a0 con 0x7feb7c0ac0e0 2023-10-06T03:45:26.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb8239c700 1 --2- 172.21.15.55:0/794338088 >> v2:172.21.15.163:3300/0 conn(0x7feb7c06f0b0 0x7feb7c15d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:45:26.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.192+0000 7feb80b99700 1 -- 172.21.15.55:0/794338088 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489148851 0 0) 0x7feb7c06e510 con 0x7feb7c1597d0 2023-10-06T03:45:26.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.193+0000 7feb8139a700 1 --2- 172.21.15.55:0/794338088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb7c0ac0e0 0x7feb7c0afa40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7feb7c06e480 tx=0x7feb6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4342acac90df29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:26.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.193+0000 7feb80b99700 1 -- 172.21.15.55:0/794338088 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c00b860 con 0x7feb7c0ac0e0 2023-10-06T03:45:26.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.193+0000 7feb80b99700 1 -- 172.21.15.55:0/794338088 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6c00b9c0 con 0x7feb7c0ac0e0 2023-10-06T03:45:26.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.193+0000 7feb80b99700 1 -- 172.21.15.55:0/794338088 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6c00bd60 con 0x7feb7c0ac0e0 2023-10-06T03:45:26.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.193+0000 7feb82b9d700 1 -- 172.21.15.55:0/794338088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb7c0ac0e0 msgr2=0x7feb7c0afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:26.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.193+0000 7feb82b9d700 1 --2- 172.21.15.55:0/794338088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb7c0ac0e0 0x7feb7c0afa40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7feb7c06e480 tx=0x7feb6c009250 comp rx=0 tx=0).stop 2023-10-06T03:45:26.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.193+0000 7feb82b9d700 1 -- 172.21.15.55:0/794338088 shutdown_connections 2023-10-06T03:45:26.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.193+0000 7feb82b9d700 1 --2- 172.21.15.55:0/794338088 >> v2:172.21.15.163:3300/0 conn(0x7feb7c06f0b0 0x7feb7c15d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.193+0000 7feb82b9d700 1 --2- 172.21.15.55:0/794338088 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb7c0ac0e0 0x7feb7c0afa40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.193+0000 7feb82b9d700 1 -- 172.21.15.55:0/794338088 >> 172.21.15.55:0/794338088 conn(0x7feb7c06adf0 msgr2=0x7feb7c0ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:26.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.194+0000 7feb82b9d700 1 -- 172.21.15.55:0/794338088 shutdown_connections 2023-10-06T03:45:26.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.194+0000 7feb82b9d700 1 -- 172.21.15.55:0/794338088 wait complete. 2023-10-06T03:45:26.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.195+0000 7feb82b9d700 1 Processor -- start 2023-10-06T03:45:26.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.195+0000 7feb82b9d700 1 -- start start 2023-10-06T03:45:26.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.196+0000 7feb82b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb7c06f0b0 0x7feb7c0c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:26.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.196+0000 7feb82b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb7c1597d0 0x7feb7c14a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:26.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.196+0000 7feb82b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb7c15e7b0 0x7feb7c068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:26.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.196+0000 7feb82b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c1695a0 con 0x7feb7c06f0b0 2023-10-06T03:45:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.196+0000 7feb82b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c0c2d70 con 0x7feb7c1597d0 2023-10-06T03:45:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.196+0000 7feb82b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7c0c21f0 con 0x7feb7c15e7b0 2023-10-06T03:45:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.196+0000 7feb81b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb7c06f0b0 0x7feb7c0c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.197+0000 7feb81b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb7c06f0b0 0x7feb7c0c19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54740/0 (socket says 172.21.15.55:54740) 2023-10-06T03:45:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.197+0000 7feb81b9b700 1 -- 172.21.15.55:0/2870223880 learned_addr learned my addr 172.21.15.55:0/2870223880 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.197+0000 7feb8139a700 1 --2- 172.21.15.55:0/2870223880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb7c1597d0 0x7feb7c14a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.197+0000 7feb8239c700 1 --2- 172.21.15.55:0/2870223880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb7c15e7b0 0x7feb7c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.197+0000 7feb81b9b700 1 -- 172.21.15.55:0/2870223880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb7c15e7b0 msgr2=0x7feb7c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.197+0000 7feb81b9b700 1 --2- 172.21.15.55:0/2870223880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb7c15e7b0 0x7feb7c068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.197+0000 7feb81b9b700 1 -- 172.21.15.55:0/2870223880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb7c1597d0 msgr2=0x7feb7c14a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.197+0000 7feb81b9b700 1 --2- 172.21.15.55:0/2870223880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb7c1597d0 0x7feb7c14a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.197+0000 7feb81b9b700 1 -- 172.21.15.55:0/2870223880 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6c0094f0 con 0x7feb7c06f0b0 2023-10-06T03:45:26.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.197+0000 7feb81b9b700 1 --2- 172.21.15.55:0/2870223880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb7c06f0b0 0x7feb7c0c19c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7feb7800c5b0 tx=0x7feb78001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:26.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.197+0000 7feb71ffb700 1 -- 172.21.15.55:0/2870223880 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb78005a70 con 0x7feb7c06f0b0 2023-10-06T03:45:26.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.198+0000 7feb71ffb700 1 -- 172.21.15.55:0/2870223880 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb78005bd0 con 0x7feb7c06f0b0 2023-10-06T03:45:26.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.198+0000 7feb82b9d700 1 -- 172.21.15.55:0/2870223880 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7c0c24d0 con 0x7feb7c06f0b0 2023-10-06T03:45:26.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.198+0000 7feb71ffb700 1 -- 172.21.15.55:0/2870223880 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb780070d0 con 0x7feb7c06f0b0 2023-10-06T03:45:26.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.198+0000 7feb82b9d700 1 -- 172.21.15.55:0/2870223880 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb7c0c2a90 con 0x7feb7c06f0b0 2023-10-06T03:45:26.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.199+0000 7feb82b9d700 1 -- 172.21.15.55:0/2870223880 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7c1696e0 con 0x7feb7c06f0b0 2023-10-06T03:45:26.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.200+0000 7feb71ffb700 1 -- 172.21.15.55:0/2870223880 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7feb78007440 con 0x7feb7c06f0b0 2023-10-06T03:45:26.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.200+0000 7feb71ffb700 1 --2- 172.21.15.55:0/2870223880 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7feb60067130 0x7feb600695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:26.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.200+0000 7feb71ffb700 1 -- 172.21.15.55:0/2870223880 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7feb78083f90 con 0x7feb7c06f0b0 2023-10-06T03:45:26.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.201+0000 7feb8139a700 1 --2- 172.21.15.55:0/2870223880 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7feb60067130 0x7feb600695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:26.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.201+0000 7feb8139a700 1 --2- 172.21.15.55:0/2870223880 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7feb60067130 0x7feb600695e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb7c06e480 tx=0x7feb6c017000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:26.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.205+0000 7feb71ffb700 1 -- 172.21.15.55:0/2870223880 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb78050c60 con 0x7feb7c06f0b0 2023-10-06T03:45:26.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.430+0000 7feb82b9d700 1 -- 172.21.15.55:0/2870223880 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb7c0cd4a0 con 0x7feb60067130 2023-10-06T03:45:26.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.441+0000 7feb71ffb700 1 -- 172.21.15.55:0/2870223880 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7feb7c0cd4a0 con 0x7feb60067130 2023-10-06T03:45:26.442 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:26.442 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:34", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:26.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.443+0000 7feb82b9d700 1 -- 172.21.15.55:0/2870223880 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7feb60067130 msgr2=0x7feb600695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:26.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.443+0000 7feb82b9d700 1 --2- 172.21.15.55:0/2870223880 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7feb60067130 0x7feb600695e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb7c06e480 tx=0x7feb6c017000 comp rx=0 tx=0).stop 2023-10-06T03:45:26.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.443+0000 7feb82b9d700 1 -- 172.21.15.55:0/2870223880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb7c06f0b0 msgr2=0x7feb7c0c19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:26.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.443+0000 7feb82b9d700 1 --2- 172.21.15.55:0/2870223880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb7c06f0b0 0x7feb7c0c19c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7feb7800c5b0 tx=0x7feb78001040 comp rx=0 tx=0).stop 2023-10-06T03:45:26.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.443+0000 7feb82b9d700 1 -- 172.21.15.55:0/2870223880 shutdown_connections 2023-10-06T03:45:26.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.443+0000 7feb82b9d700 1 --2- 172.21.15.55:0/2870223880 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7feb60067130 0x7feb600695e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.443+0000 7feb82b9d700 1 --2- 172.21.15.55:0/2870223880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb7c15e7b0 0x7feb7c068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.443+0000 7feb82b9d700 1 --2- 172.21.15.55:0/2870223880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb7c06f0b0 0x7feb7c0c19c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.443+0000 7feb82b9d700 1 --2- 172.21.15.55:0/2870223880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb7c1597d0 0x7feb7c14a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.443+0000 7feb82b9d700 1 -- 172.21.15.55:0/2870223880 >> 172.21.15.55:0/2870223880 conn(0x7feb7c06adf0 msgr2=0x7feb7c15b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:26.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.443+0000 7feb82b9d700 1 -- 172.21.15.55:0/2870223880 shutdown_connections 2023-10-06T03:45:26.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.443+0000 7feb82b9d700 1 -- 172.21.15.55:0/2870223880 wait complete. 2023-10-06T03:45:26.463 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:34", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:26.463 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T03:34:34', 'first': '2023-10-06T03:35:00', 'last': '2023-10-06T03:45:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T03:45:26.463 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T03:45:26.463 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T03:45:26.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:26.464 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-06T03:45:26.502 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:26.503 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:26.503 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:26.503 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:26.503 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:26.503 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:26.503 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:26.503 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:26.503 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:26.503 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:26.503 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:26.504 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-03_35_00_UTC', 'scheduled-2023-10-06-03_36_00_UTC', 'scheduled-2023-10-06-03_37_00_UTC', 'scheduled-2023-10-06-03_38_00_UTC', 'scheduled-2023-10-06-03_39_00_UTC', 'scheduled-2023-10-06-03_40_00_UTC', 'scheduled-2023-10-06-03_41_00_UTC', 'scheduled-2023-10-06-03_42_00_UTC', 'scheduled-2023-10-06-03_43_00_UTC', 'scheduled-2023-10-06-03_44_00_UTC', 'scheduled-2023-10-06-03_45_00_UTC'] 2023-10-06T03:45:26.504 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir1 2023-10-06T03:45:26.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.734+0000 7fd26619d700 1 Processor -- start 2023-10-06T03:45:26.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.734+0000 7fd26619d700 1 -- start start 2023-10-06T03:45:26.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.735+0000 7fd26619d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd2600656e0 0x7fd2600ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:26.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.735+0000 7fd26619d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd26006b290 0x7fd2600b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.735+0000 7fd26619d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2600aeb10 con 0x7fd2600ae180 2023-10-06T03:45:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.735+0000 7fd26619d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2600aedf0 con 0x7fd2600656e0 2023-10-06T03:45:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.735+0000 7fd26619d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600af540 con 0x7fd26006b290 2023-10-06T03:45:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.735+0000 7fd26519b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd2600ae180 0x7fd2600acb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:55338/0 (socket says 172.21.15.55:55338) 2023-10-06T03:45:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.735+0000 7fd26519b700 1 -- 172.21.15.55:0/1967767761 learned_addr learned my addr 172.21.15.55:0/1967767761 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.735+0000 7fd26599c700 1 --2- 172.21.15.55:0/1967767761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd26006b290 0x7fd2600b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.735+0000 7fd26499a700 1 --2- 172.21.15.55:0/1967767761 >> v2:172.21.15.163:3300/0 conn(0x7fd2600656e0 0x7fd2600ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.736+0000 7fd257fff700 1 -- 172.21.15.55:0/1967767761 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030075981 0 0) 0x7fd2600aeb10 con 0x7fd2600ae180 2023-10-06T03:45:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.736+0000 7fd257fff700 1 -- 172.21.15.55:0/1967767761 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd24c003570 con 0x7fd2600ae180 2023-10-06T03:45:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.736+0000 7fd26599c700 1 -- 172.21.15.55:0/1967767761 >> v2:172.21.15.163:3300/0 conn(0x7fd2600656e0 msgr2=0x7fd2600ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.736+0000 7fd26599c700 1 --2- 172.21.15.55:0/1967767761 >> v2:172.21.15.163:3300/0 conn(0x7fd2600656e0 0x7fd2600ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.736+0000 7fd26599c700 1 -- 172.21.15.55:0/1967767761 >> v1:172.21.15.163:6789/0 conn(0x7fd2600ae180 legacy=0x7fd2600acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.736+0000 7fd26599c700 1 -- 172.21.15.55:0/1967767761 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2600af820 con 0x7fd26006b290 2023-10-06T03:45:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.736+0000 7fd26599c700 1 --2- 172.21.15.55:0/1967767761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd26006b290 0x7fd2600b1110 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd258009e60 tx=0x7fd258009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43d41c24a5b81c39 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.736+0000 7fd257fff700 1 -- 172.21.15.55:0/1967767761 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25800e070 con 0x7fd26006b290 2023-10-06T03:45:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.736+0000 7fd257fff700 1 -- 172.21.15.55:0/1967767761 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25800bab0 con 0x7fd26006b290 2023-10-06T03:45:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.736+0000 7fd257fff700 1 -- 172.21.15.55:0/1967767761 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25800be70 con 0x7fd26006b290 2023-10-06T03:45:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.737+0000 7fd26619d700 1 -- 172.21.15.55:0/1967767761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd26006b290 msgr2=0x7fd2600b1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.737+0000 7fd26619d700 1 --2- 172.21.15.55:0/1967767761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd26006b290 0x7fd2600b1110 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd258009e60 tx=0x7fd258009780 comp rx=0 tx=0).stop 2023-10-06T03:45:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.737+0000 7fd26619d700 1 -- 172.21.15.55:0/1967767761 shutdown_connections 2023-10-06T03:45:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.737+0000 7fd26619d700 1 --2- 172.21.15.55:0/1967767761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd26006b290 0x7fd2600b1110 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.737+0000 7fd26619d700 1 --2- 172.21.15.55:0/1967767761 >> v2:172.21.15.163:3300/0 conn(0x7fd2600656e0 0x7fd2600ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.737+0000 7fd26619d700 1 -- 172.21.15.55:0/1967767761 >> 172.21.15.55:0/1967767761 conn(0x7fd26006f6a0 msgr2=0x7fd26006ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.737+0000 7fd26619d700 1 -- 172.21.15.55:0/1967767761 shutdown_connections 2023-10-06T03:45:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.737+0000 7fd26619d700 1 -- 172.21.15.55:0/1967767761 wait complete. 2023-10-06T03:45:26.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.738+0000 7fd26619d700 1 Processor -- start 2023-10-06T03:45:26.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.739+0000 7fd26619d700 1 -- start start 2023-10-06T03:45:26.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.739+0000 7fd26619d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2600656e0 0x7fd2600c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:26.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.739+0000 7fd26619d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd26006b290 0x7fd2600cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:26.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.739+0000 7fd26619d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2600ae180 0x7fd26016a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:26.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.739+0000 7fd26619d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600c4be0 con 0x7fd2600ae180 2023-10-06T03:45:26.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.739+0000 7fd26619d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600cbfc0 con 0x7fd26006b290 2023-10-06T03:45:26.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.739+0000 7fd26619d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600c2270 con 0x7fd2600656e0 2023-10-06T03:45:26.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.740+0000 7fd26599c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2600ae180 0x7fd26016a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:26.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.740+0000 7fd26599c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2600ae180 0x7fd26016a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54746/0 (socket says 172.21.15.55:54746) 2023-10-06T03:45:26.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.740+0000 7fd26599c700 1 -- 172.21.15.55:0/1072347943 learned_addr learned my addr 172.21.15.55:0/1072347943 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:26.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.740+0000 7fd26519b700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2600656e0 0x7fd2600c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:26.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.740+0000 7fd26499a700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd26006b290 0x7fd2600cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:26.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.740+0000 7fd26599c700 1 -- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2600656e0 msgr2=0x7fd2600c46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:26.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.740+0000 7fd26599c700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2600656e0 0x7fd2600c46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.740+0000 7fd26599c700 1 -- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd26006b290 msgr2=0x7fd2600cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:26.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.740+0000 7fd26599c700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd26006b290 0x7fd2600cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.740+0000 7fd26599c700 1 -- 172.21.15.55:0/1072347943 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd258009a20 con 0x7fd2600ae180 2023-10-06T03:45:26.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.740+0000 7fd26499a700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd26006b290 0x7fd2600cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:45:26.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.740+0000 7fd26519b700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2600656e0 0x7fd2600c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:45:26.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.741+0000 7fd26599c700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2600ae180 0x7fd26016a2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd258013000 tx=0x7fd258019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:26.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.741+0000 7fd2557fa700 1 -- 172.21.15.55:0/1072347943 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd258027470 con 0x7fd2600ae180 2023-10-06T03:45:26.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.741+0000 7fd2557fa700 1 -- 172.21.15.55:0/1072347943 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2580275f0 con 0x7fd2600ae180 2023-10-06T03:45:26.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.741+0000 7fd26619d700 1 -- 172.21.15.55:0/1072347943 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26016a830 con 0x7fd2600ae180 2023-10-06T03:45:26.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.741+0000 7fd2557fa700 1 -- 172.21.15.55:0/1072347943 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2580279b0 con 0x7fd2600ae180 2023-10-06T03:45:26.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.741+0000 7fd26619d700 1 -- 172.21.15.55:0/1072347943 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2600c1660 con 0x7fd2600ae180 2023-10-06T03:45:26.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.742+0000 7fd26619d700 1 -- 172.21.15.55:0/1072347943 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2600c2400 con 0x7fd2600ae180 2023-10-06T03:45:26.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.743+0000 7fd2557fa700 1 -- 172.21.15.55:0/1072347943 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fd258024070 con 0x7fd2600ae180 2023-10-06T03:45:26.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.744+0000 7fd2557fa700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd244067130 0x7fd2440695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:26.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.744+0000 7fd2557fa700 1 -- 172.21.15.55:0/1072347943 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fd2580875a0 con 0x7fd2600ae180 2023-10-06T03:45:26.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.744+0000 7fd26519b700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd244067130 0x7fd2440695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:26.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.745+0000 7fd26519b700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd244067130 0x7fd2440695e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2600ae8d0 tx=0x7fd25c009040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:26.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.748+0000 7fd2557fa700 1 -- 172.21.15.55:0/1072347943 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd258054270 con 0x7fd2600ae180 2023-10-06T03:45:26.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.981+0000 7fd26619d700 1 -- 172.21.15.55:0/1072347943 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd26004c3c0 con 0x7fd244067130 2023-10-06T03:45:26.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.991+0000 7fd2557fa700 1 -- 172.21.15.55:0/1072347943 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fd26004c3c0 con 0x7fd244067130 2023-10-06T03:45:26.992 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:26.992 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:35", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:26.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.993+0000 7fd26619d700 1 -- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd244067130 msgr2=0x7fd2440695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:26.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.993+0000 7fd26619d700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd244067130 0x7fd2440695e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2600ae8d0 tx=0x7fd25c009040 comp rx=0 tx=0).stop 2023-10-06T03:45:26.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.993+0000 7fd26619d700 1 -- 172.21.15.55:0/1072347943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2600ae180 msgr2=0x7fd26016a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:26.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.993+0000 7fd26619d700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2600ae180 0x7fd26016a2f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd258013000 tx=0x7fd258019000 comp rx=0 tx=0).stop 2023-10-06T03:45:26.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.993+0000 7fd26619d700 1 -- 172.21.15.55:0/1072347943 shutdown_connections 2023-10-06T03:45:26.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.993+0000 7fd26619d700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2600ae180 0x7fd26016a2f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.993+0000 7fd26619d700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fd244067130 0x7fd2440695e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.993+0000 7fd26619d700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2600656e0 0x7fd2600c46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.993+0000 7fd26619d700 1 --2- 172.21.15.55:0/1072347943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd26006b290 0x7fd2600cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:26.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.993+0000 7fd26619d700 1 -- 172.21.15.55:0/1072347943 >> 172.21.15.55:0/1072347943 conn(0x7fd26006f6a0 msgr2=0x7fd260068c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:26.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.994+0000 7fd26619d700 1 -- 172.21.15.55:0/1072347943 shutdown_connections 2023-10-06T03:45:26.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:26.994+0000 7fd26619d700 1 -- 172.21.15.55:0/1072347943 wait complete. 2023-10-06T03:45:27.012 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:35", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:27.012 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T03:34:35', 'first': '2023-10-06T03:35:00', 'last': '2023-10-06T03:45:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T03:45:27.012 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T03:45:27.012 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T03:45:27.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:27.013 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-06T03:45:27.052 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:27.052 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_01_UTC 2023-10-06T03:45:27.052 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:27.052 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:27.052 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:27.053 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:27.053 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:27.053 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:27.053 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:27.053 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:27.053 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:27.054 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-03_35_00_UTC', 'scheduled-2023-10-06-03_36_01_UTC', 'scheduled-2023-10-06-03_37_00_UTC', 'scheduled-2023-10-06-03_38_00_UTC', 'scheduled-2023-10-06-03_39_00_UTC', 'scheduled-2023-10-06-03_40_00_UTC', 'scheduled-2023-10-06-03_41_00_UTC', 'scheduled-2023-10-06-03_42_00_UTC', 'scheduled-2023-10-06-03_43_00_UTC', 'scheduled-2023-10-06-03_44_00_UTC', 'scheduled-2023-10-06-03_45_00_UTC'] 2023-10-06T03:45:27.054 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir2 2023-10-06T03:45:27.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.284+0000 7fe4d9f9d700 1 Processor -- start 2023-10-06T03:45:27.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.284+0000 7fe4d9f9d700 1 -- start start 2023-10-06T03:45:27.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.285+0000 7fe4d9f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe4d4156590 0x7fe4d40cc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:27.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.285+0000 7fe4d9f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe4d406ee30 0x7fe4d40c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:27.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.285+0000 7fe4d9f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d40cce90 con 0x7fe4d40cc880 2023-10-06T03:45:27.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.285+0000 7fe4d9f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d415afd0 con 0x7fe4d4156590 2023-10-06T03:45:27.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.285+0000 7fe4d9f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d415ab40 con 0x7fe4d406ee30 2023-10-06T03:45:27.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.285+0000 7fe4d8f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe4d4156590 0x7fe4d40cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:27.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.285+0000 7fe4d8f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe4d4156590 0x7fe4d40cc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54758/0 (socket says 172.21.15.55:54758) 2023-10-06T03:45:27.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.285+0000 7fe4d8f9b700 1 -- 172.21.15.55:0/3315291355 learned_addr learned my addr 172.21.15.55:0/3315291355 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:27.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.286+0000 7fe4d8f9b700 1 -- 172.21.15.55:0/3315291355 >> v2:172.21.15.163:3300/0 conn(0x7fe4d406ee30 msgr2=0x7fe4d40c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:27.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.286+0000 7fe4d8f9b700 1 --2- 172.21.15.55:0/3315291355 >> v2:172.21.15.163:3300/0 conn(0x7fe4d406ee30 0x7fe4d40c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:27.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.286+0000 7fe4d8f9b700 1 -- 172.21.15.55:0/3315291355 >> v1:172.21.15.163:6789/0 conn(0x7fe4d40cc880 legacy=0x7fe4d406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:27.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.286+0000 7fe4d8f9b700 1 -- 172.21.15.55:0/3315291355 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d415a210 con 0x7fe4d4156590 2023-10-06T03:45:27.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.286+0000 7fe4d8f9b700 1 --2- 172.21.15.55:0/3315291355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d4156590 0x7fe4d40cc340 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40c79e0 tx=0x7fe4d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a23d6c3e56b834 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:27.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.286+0000 7fe4cb7fe700 1 -- 172.21.15.55:0/3315291355 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d00092e0 con 0x7fe4d4156590 2023-10-06T03:45:27.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.286+0000 7fe4cb7fe700 1 -- 172.21.15.55:0/3315291355 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d000ba10 con 0x7fe4d4156590 2023-10-06T03:45:27.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.286+0000 7fe4cb7fe700 1 -- 172.21.15.55:0/3315291355 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000bdd0 con 0x7fe4d4156590 2023-10-06T03:45:27.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.287+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/3315291355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d4156590 msgr2=0x7fe4d40cc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:27.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.287+0000 7fe4d9f9d700 1 --2- 172.21.15.55:0/3315291355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d4156590 0x7fe4d40cc340 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40c79e0 tx=0x7fe4d0009840 comp rx=0 tx=0).stop 2023-10-06T03:45:27.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.287+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/3315291355 shutdown_connections 2023-10-06T03:45:27.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.287+0000 7fe4d9f9d700 1 --2- 172.21.15.55:0/3315291355 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d4156590 0x7fe4d40cc340 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:27.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.287+0000 7fe4d9f9d700 1 --2- 172.21.15.55:0/3315291355 >> v2:172.21.15.163:3300/0 conn(0x7fe4d406ee30 0x7fe4d40c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:27.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.287+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/3315291355 >> 172.21.15.55:0/3315291355 conn(0x7fe4d406b420 msgr2=0x7fe4d40cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:27.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.287+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/3315291355 shutdown_connections 2023-10-06T03:45:27.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.287+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/3315291355 wait complete. 2023-10-06T03:45:27.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.289+0000 7fe4d9f9d700 1 Processor -- start 2023-10-06T03:45:27.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.289+0000 7fe4d9f9d700 1 -- start start 2023-10-06T03:45:27.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d9f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d406ee30 0x7fe4d40c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:27.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d9f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d40cc880 0x7fe4d40ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:27.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d9f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4d4156590 0x7fe4d40ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:27.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d9f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40c5870 con 0x7fe4d406ee30 2023-10-06T03:45:27.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d9f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40c5b50 con 0x7fe4d4156590 2023-10-06T03:45:27.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d9f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40c5e30 con 0x7fe4d40cc880 2023-10-06T03:45:27.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d8f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d406ee30 0x7fe4d40c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:27.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d8f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d406ee30 0x7fe4d40c6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54770/0 (socket says 172.21.15.55:54770) 2023-10-06T03:45:27.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d8f9b700 1 -- 172.21.15.55:0/2749705544 learned_addr learned my addr 172.21.15.55:0/2749705544 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:27.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d8f9b700 1 -- 172.21.15.55:0/2749705544 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d40cc880 msgr2=0x7fe4d40ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:27.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d8f9b700 1 --2- 172.21.15.55:0/2749705544 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d40cc880 0x7fe4d40ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:27.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d8f9b700 1 -- 172.21.15.55:0/2749705544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4d4156590 msgr2=0x7fe4d40ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:27.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d8f9b700 1 --2- 172.21.15.55:0/2749705544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4d4156590 0x7fe4d40ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:27.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d8f9b700 1 -- 172.21.15.55:0/2749705544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d0009ae0 con 0x7fe4d406ee30 2023-10-06T03:45:27.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.290+0000 7fe4d8f9b700 1 --2- 172.21.15.55:0/2749705544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d406ee30 0x7fe4d40c6b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4070fe0 tx=0x7fe4d0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:27.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.291+0000 7fe4c8ff9700 1 -- 172.21.15.55:0/2749705544 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d001e070 con 0x7fe4d406ee30 2023-10-06T03:45:27.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.291+0000 7fe4c8ff9700 1 -- 172.21.15.55:0/2749705544 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d0022410 con 0x7fe4d406ee30 2023-10-06T03:45:27.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.291+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/2749705544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d40c6110 con 0x7fe4d406ee30 2023-10-06T03:45:27.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.291+0000 7fe4c8ff9700 1 -- 172.21.15.55:0/2749705544 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d00227d0 con 0x7fe4d406ee30 2023-10-06T03:45:27.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.291+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/2749705544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d40c5490 con 0x7fe4d406ee30 2023-10-06T03:45:27.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.292+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/2749705544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d404c3c0 con 0x7fe4d406ee30 2023-10-06T03:45:27.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.293+0000 7fe4c8ff9700 1 -- 172.21.15.55:0/2749705544 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fe4d00193f0 con 0x7fe4d406ee30 2023-10-06T03:45:27.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.294+0000 7fe4c8ff9700 1 --2- 172.21.15.55:0/2749705544 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe4c0067130 0x7fe4c00695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:27.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.294+0000 7fe4c8ff9700 1 -- 172.21.15.55:0/2749705544 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fe4d00867c0 con 0x7fe4d406ee30 2023-10-06T03:45:27.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.294+0000 7fe4cbfff700 1 --2- 172.21.15.55:0/2749705544 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe4c0067130 0x7fe4c00695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:27.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.295+0000 7fe4cbfff700 1 --2- 172.21.15.55:0/2749705544 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe4c0067130 0x7fe4c00695e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4069290 tx=0x7fe4c4006c30 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:27.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.298+0000 7fe4c8ff9700 1 -- 172.21.15.55:0/2749705544 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4d0053380 con 0x7fe4d406ee30 2023-10-06T03:45:27.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.528+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/2749705544 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4d40cdb90 con 0x7fe4c0067130 2023-10-06T03:45:27.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.537+0000 7fe4c8ff9700 1 -- 172.21.15.55:0/2749705544 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fe4d40cdb90 con 0x7fe4c0067130 2023-10-06T03:45:27.539 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:27.539 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:35", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:27.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.540+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/2749705544 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe4c0067130 msgr2=0x7fe4c00695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:27.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.540+0000 7fe4d9f9d700 1 --2- 172.21.15.55:0/2749705544 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe4c0067130 0x7fe4c00695e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4069290 tx=0x7fe4c4006c30 comp rx=0 tx=0).stop 2023-10-06T03:45:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.540+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/2749705544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d406ee30 msgr2=0x7fe4d40c6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.540+0000 7fe4d9f9d700 1 --2- 172.21.15.55:0/2749705544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d406ee30 0x7fe4d40c6b90 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4070fe0 tx=0x7fe4d0019000 comp rx=0 tx=0).stop 2023-10-06T03:45:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.540+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/2749705544 shutdown_connections 2023-10-06T03:45:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.540+0000 7fe4d9f9d700 1 --2- 172.21.15.55:0/2749705544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4d4156590 0x7fe4d40ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.540+0000 7fe4d9f9d700 1 --2- 172.21.15.55:0/2749705544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4d406ee30 0x7fe4d40c6b90 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.540+0000 7fe4d9f9d700 1 --2- 172.21.15.55:0/2749705544 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe4c0067130 0x7fe4c00695e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.540+0000 7fe4d9f9d700 1 --2- 172.21.15.55:0/2749705544 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4d40cc880 0x7fe4d40ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.540+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/2749705544 >> 172.21.15.55:0/2749705544 conn(0x7fe4d406b420 msgr2=0x7fe4d41513f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.540+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/2749705544 shutdown_connections 2023-10-06T03:45:27.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.540+0000 7fe4d9f9d700 1 -- 172.21.15.55:0/2749705544 wait complete. 2023-10-06T03:45:27.558 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:35", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:27.558 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T03:34:35', 'first': '2023-10-06T03:35:00', 'last': '2023-10-06T03:45:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T03:45:27.559 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T03:45:27.559 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T03:45:27.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:27.559 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-06T03:45:27.598 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:27.598 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:27.598 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:27.598 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:27.598 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:27.598 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:27.598 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:27.599 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:27.599 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:27.599 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:27.599 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:27.600 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-03_35_00_UTC', 'scheduled-2023-10-06-03_36_00_UTC', 'scheduled-2023-10-06-03_37_00_UTC', 'scheduled-2023-10-06-03_38_00_UTC', 'scheduled-2023-10-06-03_39_00_UTC', 'scheduled-2023-10-06-03_40_00_UTC', 'scheduled-2023-10-06-03_41_00_UTC', 'scheduled-2023-10-06-03_42_00_UTC', 'scheduled-2023-10-06-03_43_00_UTC', 'scheduled-2023-10-06-03_44_00_UTC', 'scheduled-2023-10-06-03_45_00_UTC'] 2023-10-06T03:45:27.600 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir3 2023-10-06T03:45:27.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.831+0000 7f01fd39d700 1 Processor -- start 2023-10-06T03:45:27.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.831+0000 7f01fd39d700 1 -- start start 2023-10-06T03:45:27.834 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.832+0000 7f01fd39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f01f80b0790 0x7f01f80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:27.834 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.832+0000 7f01fd39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01f81591e0 0x7f01f8159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:27.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.832+0000 7f01fd39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f80cfa00 con 0x7f01f80b03c0 2023-10-06T03:45:27.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.832+0000 7f01fd39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f80cb580 con 0x7f01f81591e0 2023-10-06T03:45:27.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.832+0000 7f01fd39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f80cf160 con 0x7f01f80b0790 2023-10-06T03:45:27.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.833+0000 7f01f7fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f01f80b03c0 0x7f01f80b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35430/0 (socket says 172.21.15.55:35430) 2023-10-06T03:45:27.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.833+0000 7f01f7fff700 1 -- 172.21.15.55:0/2407484238 learned_addr learned my addr 172.21.15.55:0/2407484238 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:27.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.833+0000 7f01fcb9c700 1 --2- 172.21.15.55:0/2407484238 >> v2:172.21.15.55:3300/0 conn(0x7f01f81591e0 0x7f01f8159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:27.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.833+0000 7f01f77fe700 1 --2- 172.21.15.55:0/2407484238 >> v2:172.21.15.163:3300/0 conn(0x7f01f80b0790 0x7f01f80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:27.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.833+0000 7f01f6ffd700 1 -- 172.21.15.55:0/2407484238 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896914870 0 0) 0x7f01f80cfa00 con 0x7f01f80b03c0 2023-10-06T03:45:27.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.833+0000 7f01f6ffd700 1 -- 172.21.15.55:0/2407484238 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01e0003570 con 0x7f01f80b03c0 2023-10-06T03:45:27.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.833+0000 7f01fcb9c700 1 -- 172.21.15.55:0/2407484238 >> v2:172.21.15.163:3300/0 conn(0x7f01f80b0790 msgr2=0x7f01f80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:27.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.833+0000 7f01fcb9c700 1 --2- 172.21.15.55:0/2407484238 >> v2:172.21.15.163:3300/0 conn(0x7f01f80b0790 0x7f01f80cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:27.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.833+0000 7f01fcb9c700 1 -- 172.21.15.55:0/2407484238 >> v1:172.21.15.55:6789/0 conn(0x7f01f80b03c0 legacy=0x7f01f80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:27.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.833+0000 7f01fcb9c700 1 -- 172.21.15.55:0/2407484238 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f80cf440 con 0x7f01f81591e0 2023-10-06T03:45:27.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.833+0000 7f01fcb9c700 1 --2- 172.21.15.55:0/2407484238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f81591e0 0x7f01f8159600 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009e60 tx=0x7f01f0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe19e039313c13e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:27.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.834+0000 7f01f6ffd700 1 -- 172.21.15.55:0/2407484238 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f000e070 con 0x7f01f81591e0 2023-10-06T03:45:27.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.834+0000 7f01f6ffd700 1 -- 172.21.15.55:0/2407484238 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f000bb20 con 0x7f01f81591e0 2023-10-06T03:45:27.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.834+0000 7f01f6ffd700 1 -- 172.21.15.55:0/2407484238 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0012430 con 0x7f01f81591e0 2023-10-06T03:45:27.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.834+0000 7f01fd39d700 1 -- 172.21.15.55:0/2407484238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f81591e0 msgr2=0x7f01f8159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:27.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.834+0000 7f01fd39d700 1 --2- 172.21.15.55:0/2407484238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f81591e0 0x7f01f8159600 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009e60 tx=0x7f01f0009780 comp rx=0 tx=0).stop 2023-10-06T03:45:27.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.834+0000 7f01fd39d700 1 -- 172.21.15.55:0/2407484238 shutdown_connections 2023-10-06T03:45:27.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.834+0000 7f01fd39d700 1 --2- 172.21.15.55:0/2407484238 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f81591e0 0x7f01f8159600 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:27.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.834+0000 7f01fd39d700 1 --2- 172.21.15.55:0/2407484238 >> v2:172.21.15.163:3300/0 conn(0x7f01f80b0790 0x7f01f80cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:27.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.834+0000 7f01fd39d700 1 -- 172.21.15.55:0/2407484238 >> 172.21.15.55:0/2407484238 conn(0x7f01f8063230 msgr2=0x7f01f8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:27.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.834+0000 7f01fd39d700 1 -- 172.21.15.55:0/2407484238 shutdown_connections 2023-10-06T03:45:27.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.834+0000 7f01fd39d700 1 -- 172.21.15.55:0/2407484238 wait complete. 2023-10-06T03:45:27.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.836+0000 7f01fd39d700 1 Processor -- start 2023-10-06T03:45:27.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.836+0000 7f01fd39d700 1 -- start start 2023-10-06T03:45:27.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.837+0000 7f01fd39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f80b03c0 0x7f01f8169c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:27.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.837+0000 7f01fd39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f80b0790 0x7f01f80c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:27.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.837+0000 7f01fd39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f81591e0 0x7f01f8168990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:27.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.837+0000 7f01fd39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f816a180 con 0x7f01f80b0790 2023-10-06T03:45:27.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.837+0000 7f01fd39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f80c64a0 con 0x7f01f80b03c0 2023-10-06T03:45:27.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.837+0000 7f01fd39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f80c6720 con 0x7f01f81591e0 2023-10-06T03:45:27.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.837+0000 7f01f77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f80b0790 0x7f01f80c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:27.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.837+0000 7f01f77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f80b0790 0x7f01f80c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54800/0 (socket says 172.21.15.55:54800) 2023-10-06T03:45:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.837+0000 7f01f77fe700 1 -- 172.21.15.55:0/1765754291 learned_addr learned my addr 172.21.15.55:0/1765754291 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.837+0000 7f01f7fff700 1 --2- 172.21.15.55:0/1765754291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f80b03c0 0x7f01f8169c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.837+0000 7f01fcb9c700 1 --2- 172.21.15.55:0/1765754291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f81591e0 0x7f01f8168990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.838+0000 7f01f77fe700 1 -- 172.21.15.55:0/1765754291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f81591e0 msgr2=0x7f01f8168990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.838+0000 7f01f77fe700 1 --2- 172.21.15.55:0/1765754291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f81591e0 0x7f01f8168990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.838+0000 7f01f77fe700 1 -- 172.21.15.55:0/1765754291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f80b03c0 msgr2=0x7f01f8169c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.838+0000 7f01f77fe700 1 --2- 172.21.15.55:0/1765754291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f80b03c0 0x7f01f8169c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.838+0000 7f01f77fe700 1 -- 172.21.15.55:0/1765754291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f0009a20 con 0x7f01f80b0790 2023-10-06T03:45:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.838+0000 7f01f77fe700 1 --2- 172.21.15.55:0/1765754291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f80b0790 0x7f01f80c5f60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00d8c0 tx=0x7f01ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.838+0000 7f01dffff700 1 -- 172.21.15.55:0/1765754291 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00de60 con 0x7f01f80b0790 2023-10-06T03:45:27.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.838+0000 7f01dffff700 1 -- 172.21.15.55:0/1765754291 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec00b780 con 0x7f01f80b0790 2023-10-06T03:45:27.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.838+0000 7f01dffff700 1 -- 172.21.15.55:0/1765754291 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00bb20 con 0x7f01f80b0790 2023-10-06T03:45:27.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.838+0000 7f01fd39d700 1 -- 172.21.15.55:0/1765754291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f80c6a00 con 0x7f01f80b0790 2023-10-06T03:45:27.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.838+0000 7f01fd39d700 1 -- 172.21.15.55:0/1765754291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f8155b20 con 0x7f01f80b0790 2023-10-06T03:45:27.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.839+0000 7f01fd39d700 1 -- 172.21.15.55:0/1765754291 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f80c6f20 con 0x7f01f80b0790 2023-10-06T03:45:27.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.840+0000 7f01dffff700 1 -- 172.21.15.55:0/1765754291 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f01ec01e460 con 0x7f01f80b0790 2023-10-06T03:45:27.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.841+0000 7f01dffff700 1 --2- 172.21.15.55:0/1765754291 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f01e4067130 0x7f01e40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:27.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.841+0000 7f01dffff700 1 -- 172.21.15.55:0/1765754291 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f01ec085230 con 0x7f01f80b0790 2023-10-06T03:45:27.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.842+0000 7f01f7fff700 1 --2- 172.21.15.55:0/1765754291 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f01e4067130 0x7f01e40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:27.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.843+0000 7f01f7fff700 1 --2- 172.21.15.55:0/1765754291 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f01e4067130 0x7f01e40695e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f01e80058d0 tx=0x7f01e8009040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:27.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:27.846+0000 7f01dffff700 1 -- 172.21.15.55:0/1765754291 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01f80c6f20 con 0x7f01f80b0790 2023-10-06T03:45:28.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.071+0000 7f01fd39d700 1 -- 172.21.15.55:0/1765754291 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01f80c6f20 con 0x7f01e4067130 2023-10-06T03:45:28.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.081+0000 7f01dffff700 1 -- 172.21.15.55:0/1765754291 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f01f80c6f20 con 0x7f01e4067130 2023-10-06T03:45:28.083 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:28.083 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:36", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:28.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.084+0000 7f01fd39d700 1 -- 172.21.15.55:0/1765754291 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f01e4067130 msgr2=0x7f01e40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:28.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.084+0000 7f01fd39d700 1 --2- 172.21.15.55:0/1765754291 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f01e4067130 0x7f01e40695e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f01e80058d0 tx=0x7f01e8009040 comp rx=0 tx=0).stop 2023-10-06T03:45:28.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.084+0000 7f01fd39d700 1 -- 172.21.15.55:0/1765754291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f80b0790 msgr2=0x7f01f80c5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:28.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.084+0000 7f01fd39d700 1 --2- 172.21.15.55:0/1765754291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f80b0790 0x7f01f80c5f60 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00d8c0 tx=0x7f01ec008040 comp rx=0 tx=0).stop 2023-10-06T03:45:28.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.084+0000 7f01fd39d700 1 -- 172.21.15.55:0/1765754291 shutdown_connections 2023-10-06T03:45:28.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.084+0000 7f01fd39d700 1 --2- 172.21.15.55:0/1765754291 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f01e4067130 0x7f01e40695e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.084+0000 7f01fd39d700 1 --2- 172.21.15.55:0/1765754291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f81591e0 0x7f01f8168990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.084+0000 7f01fd39d700 1 --2- 172.21.15.55:0/1765754291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f80b03c0 0x7f01f8169c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.084+0000 7f01fd39d700 1 --2- 172.21.15.55:0/1765754291 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f80b0790 0x7f01f80c5f60 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.084+0000 7f01fd39d700 1 -- 172.21.15.55:0/1765754291 >> 172.21.15.55:0/1765754291 conn(0x7f01f8063230 msgr2=0x7f01f80cba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:28.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.084+0000 7f01fd39d700 1 -- 172.21.15.55:0/1765754291 shutdown_connections 2023-10-06T03:45:28.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.084+0000 7f01fd39d700 1 -- 172.21.15.55:0/1765754291 wait complete. 2023-10-06T03:45:28.102 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:36", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:28.103 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T03:34:36', 'first': '2023-10-06T03:35:00', 'last': '2023-10-06T03:45:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T03:45:28.103 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T03:45:28.103 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T03:45:28.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:28.103 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-06T03:45:28.142 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_01_UTC 2023-10-06T03:45:28.142 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:28.142 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:28.142 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:28.142 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:28.143 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:28.143 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:28.143 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:28.143 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:28.143 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:28.143 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:28.144 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-03_35_01_UTC', 'scheduled-2023-10-06-03_36_00_UTC', 'scheduled-2023-10-06-03_37_00_UTC', 'scheduled-2023-10-06-03_38_00_UTC', 'scheduled-2023-10-06-03_39_00_UTC', 'scheduled-2023-10-06-03_40_00_UTC', 'scheduled-2023-10-06-03_41_00_UTC', 'scheduled-2023-10-06-03_42_00_UTC', 'scheduled-2023-10-06-03_43_00_UTC', 'scheduled-2023-10-06-03_44_00_UTC', 'scheduled-2023-10-06-03_45_00_UTC'] 2023-10-06T03:45:28.144 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir4 2023-10-06T03:45:28.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.374+0000 7f0ac5b9d700 1 Processor -- start 2023-10-06T03:45:28.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.374+0000 7f0ac5b9d700 1 -- start start 2023-10-06T03:45:28.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.375+0000 7f0ac5b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0ac00b03c0 0x7f0ac00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:28.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.375+0000 7f0ac5b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ac00b0790 0x7f0ac00b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:28.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.375+0000 7f0ac5b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac00cfa00 con 0x7f0ac01591e0 2023-10-06T03:45:28.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.375+0000 7f0ac5b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac00cb580 con 0x7f0ac00b03c0 2023-10-06T03:45:28.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.375+0000 7f0ac5b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00b2de0 con 0x7f0ac00b0790 2023-10-06T03:45:28.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.376+0000 7f0ac4b9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0ac00b03c0 0x7f0ac00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:28.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.376+0000 7f0ac4b9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0ac00b03c0 0x7f0ac00cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54816/0 (socket says 172.21.15.55:54816) 2023-10-06T03:45:28.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.376+0000 7f0ac4b9b700 1 -- 172.21.15.55:0/2504945830 learned_addr learned my addr 172.21.15.55:0/2504945830 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:28.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.376+0000 7f0ac539c700 1 --1- 172.21.15.55:0/2504945830 >> v1:172.21.15.55:6789/0 conn(0x7f0ac01591e0 0x7f0ac0159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35440/0 (socket says 172.21.15.55:35440) 2023-10-06T03:45:28.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.376+0000 7f0abffff700 1 --2- 172.21.15.55:0/2504945830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ac00b0790 0x7f0ac00b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:28.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.376+0000 7f0ac4b9b700 1 -- 172.21.15.55:0/2504945830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ac00b0790 msgr2=0x7f0ac00b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:28.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.376+0000 7f0ac4b9b700 1 --2- 172.21.15.55:0/2504945830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ac00b0790 0x7f0ac00b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.376+0000 7f0ac4b9b700 1 -- 172.21.15.55:0/2504945830 >> v1:172.21.15.55:6789/0 conn(0x7f0ac01591e0 legacy=0x7f0ac0159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:28.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.376+0000 7f0ac4b9b700 1 -- 172.21.15.55:0/2504945830 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac00cf160 con 0x7f0ac00b03c0 2023-10-06T03:45:28.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.376+0000 7f0ac4b9b700 1 --2- 172.21.15.55:0/2504945830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac00b03c0 0x7f0ac00cea70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40099b0 tx=0x7f0ab4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e7baa69f808551 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:28.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.377+0000 7f0abf7fe700 1 -- 172.21.15.55:0/2504945830 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400b860 con 0x7f0ac00b03c0 2023-10-06T03:45:28.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.377+0000 7f0abf7fe700 1 -- 172.21.15.55:0/2504945830 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab400b9c0 con 0x7f0ac00b03c0 2023-10-06T03:45:28.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.377+0000 7f0abf7fe700 1 -- 172.21.15.55:0/2504945830 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400bd60 con 0x7f0ac00b03c0 2023-10-06T03:45:28.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.377+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/2504945830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac00b03c0 msgr2=0x7f0ac00cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:28.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.377+0000 7f0ac5b9d700 1 --2- 172.21.15.55:0/2504945830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac00b03c0 0x7f0ac00cea70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40099b0 tx=0x7f0ab4009250 comp rx=0 tx=0).stop 2023-10-06T03:45:28.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.377+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/2504945830 shutdown_connections 2023-10-06T03:45:28.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.377+0000 7f0ac5b9d700 1 --2- 172.21.15.55:0/2504945830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac00b03c0 0x7f0ac00cea70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.377+0000 7f0ac5b9d700 1 --2- 172.21.15.55:0/2504945830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ac00b0790 0x7f0ac00b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.377+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/2504945830 >> 172.21.15.55:0/2504945830 conn(0x7f0ac0063230 msgr2=0x7f0ac0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:28.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.377+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/2504945830 shutdown_connections 2023-10-06T03:45:28.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.377+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/2504945830 wait complete. 2023-10-06T03:45:28.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.379+0000 7f0ac5b9d700 1 Processor -- start 2023-10-06T03:45:28.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.379+0000 7f0ac5b9d700 1 -- start start 2023-10-06T03:45:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.380+0000 7f0ac5b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac00b03c0 0x7f0ac014ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.380+0000 7f0ac5b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ac00b0790 0x7f0ac015a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.380+0000 7f0ac5b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ac01591e0 0x7f0ac00b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.380+0000 7f0ac5b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac016da60 con 0x7f0ac00b03c0 2023-10-06T03:45:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.380+0000 7f0ac5b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac015b170 con 0x7f0ac01591e0 2023-10-06T03:45:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.380+0000 7f0ac5b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac016d130 con 0x7f0ac00b0790 2023-10-06T03:45:28.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.380+0000 7f0ac4b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac00b03c0 0x7f0ac014ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.380+0000 7f0ac4b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac00b03c0 0x7f0ac014ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54818/0 (socket says 172.21.15.55:54818) 2023-10-06T03:45:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.380+0000 7f0ac4b9b700 1 -- 172.21.15.55:0/1484853880 learned_addr learned my addr 172.21.15.55:0/1484853880 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.380+0000 7f0ac4b9b700 1 -- 172.21.15.55:0/1484853880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ac00b0790 msgr2=0x7f0ac015a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.380+0000 7f0ac4b9b700 1 --2- 172.21.15.55:0/1484853880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ac00b0790 0x7f0ac015a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.380+0000 7f0ac4b9b700 1 -- 172.21.15.55:0/1484853880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ac01591e0 msgr2=0x7f0ac00b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.380+0000 7f0ac4b9b700 1 --2- 172.21.15.55:0/1484853880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ac01591e0 0x7f0ac00b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.380+0000 7f0ac4b9b700 1 -- 172.21.15.55:0/1484853880 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab40094f0 con 0x7f0ac00b03c0 2023-10-06T03:45:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.381+0000 7f0ac4b9b700 1 --2- 172.21.15.55:0/1484853880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac00b03c0 0x7f0ac014ba70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0ac00cfe40 tx=0x7f0ab4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.381+0000 7f0abcff9700 1 -- 172.21.15.55:0/1484853880 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4024440 con 0x7f0ac00b03c0 2023-10-06T03:45:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.381+0000 7f0abcff9700 1 -- 172.21.15.55:0/1484853880 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab40245c0 con 0x7f0ac00b03c0 2023-10-06T03:45:28.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.381+0000 7f0abcff9700 1 -- 172.21.15.55:0/1484853880 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4024980 con 0x7f0ac00b03c0 2023-10-06T03:45:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.381+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/1484853880 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac016d410 con 0x7f0ac00b03c0 2023-10-06T03:45:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.381+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/1484853880 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac016a8d0 con 0x7f0ac00b03c0 2023-10-06T03:45:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.382+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/1484853880 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac004c3c0 con 0x7f0ac00b03c0 2023-10-06T03:45:28.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.383+0000 7f0abcff9700 1 -- 172.21.15.55:0/1484853880 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f0ab401a420 con 0x7f0ac00b03c0 2023-10-06T03:45:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.384+0000 7f0abcff9700 1 --2- 172.21.15.55:0/1484853880 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0aa4067130 0x7f0aa40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.384+0000 7f0abcff9700 1 -- 172.21.15.55:0/1484853880 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f0ab40869a0 con 0x7f0ac00b03c0 2023-10-06T03:45:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.385+0000 7f0abffff700 1 --2- 172.21.15.55:0/1484853880 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0aa4067130 0x7f0aa40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:28.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.385+0000 7f0abffff700 1 --2- 172.21.15.55:0/1484853880 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0aa4067130 0x7f0aa40695e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0157310 tx=0x7f0aac008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:28.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.389+0000 7f0abcff9700 1 -- 172.21.15.55:0/1484853880 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ab4053670 con 0x7f0ac00b03c0 2023-10-06T03:45:28.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.619+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/1484853880 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ac016dba0 con 0x7f0aa4067130 2023-10-06T03:45:28.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.629+0000 7f0abcff9700 1 -- 172.21.15.55:0/1484853880 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f0aac009640 con 0x7f0aa4067130 2023-10-06T03:45:28.631 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:28.631 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:37", "first": "2023-10-06T03:35:01", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:28.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.632+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/1484853880 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0aa4067130 msgr2=0x7f0aa40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:28.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.632+0000 7f0ac5b9d700 1 --2- 172.21.15.55:0/1484853880 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0aa4067130 0x7f0aa40695e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0157310 tx=0x7f0aac008040 comp rx=0 tx=0).stop 2023-10-06T03:45:28.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.632+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/1484853880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac00b03c0 msgr2=0x7f0ac014ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:28.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.632+0000 7f0ac5b9d700 1 --2- 172.21.15.55:0/1484853880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac00b03c0 0x7f0ac014ba70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0ac00cfe40 tx=0x7f0ab4017000 comp rx=0 tx=0).stop 2023-10-06T03:45:28.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.632+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/1484853880 shutdown_connections 2023-10-06T03:45:28.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.632+0000 7f0ac5b9d700 1 --2- 172.21.15.55:0/1484853880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ac01591e0 0x7f0ac00b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.632+0000 7f0ac5b9d700 1 --2- 172.21.15.55:0/1484853880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ac00b03c0 0x7f0ac014ba70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.632+0000 7f0ac5b9d700 1 --2- 172.21.15.55:0/1484853880 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0aa4067130 0x7f0aa40695e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.632+0000 7f0ac5b9d700 1 --2- 172.21.15.55:0/1484853880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ac00b0790 0x7f0ac015a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.632+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/1484853880 >> 172.21.15.55:0/1484853880 conn(0x7f0ac0063230 msgr2=0x7f0ac01579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:28.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.632+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/1484853880 shutdown_connections 2023-10-06T03:45:28.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.632+0000 7f0ac5b9d700 1 -- 172.21.15.55:0/1484853880 wait complete. 2023-10-06T03:45:28.651 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:37", "first": "2023-10-06T03:35:01", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:28.652 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T03:34:37', 'first': '2023-10-06T03:35:01', 'last': '2023-10-06T03:45:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T03:45:28.652 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T03:45:28.652 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T03:45:28.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:28.652 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-06T03:45:28.691 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:28.691 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:28.692 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:28.692 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:28.692 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:28.692 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:28.692 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:28.692 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:28.693 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:28.693 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:28.693 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:28.694 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-03_35_00_UTC', 'scheduled-2023-10-06-03_36_00_UTC', 'scheduled-2023-10-06-03_37_00_UTC', 'scheduled-2023-10-06-03_38_00_UTC', 'scheduled-2023-10-06-03_39_00_UTC', 'scheduled-2023-10-06-03_40_00_UTC', 'scheduled-2023-10-06-03_41_00_UTC', 'scheduled-2023-10-06-03_42_00_UTC', 'scheduled-2023-10-06-03_43_00_UTC', 'scheduled-2023-10-06-03_44_00_UTC', 'scheduled-2023-10-06-03_45_00_UTC'] 2023-10-06T03:45:28.694 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir5 2023-10-06T03:45:28.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.925+0000 7f24f8d9d700 1 Processor -- start 2023-10-06T03:45:28.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.925+0000 7f24f8d9d700 1 -- start start 2023-10-06T03:45:28.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.926+0000 7f24f8d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40b03c0 0x7f24f40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:28.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.926+0000 7f24f8d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f24f40b0790 0x7f24f40b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:28.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.926+0000 7f24f8d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f40cfa00 con 0x7f24f41591e0 2023-10-06T03:45:28.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.926+0000 7f24f8d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f40cb580 con 0x7f24f40b0790 2023-10-06T03:45:28.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.926+0000 7f24f8d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40b2de0 con 0x7f24f40b03c0 2023-10-06T03:45:28.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.927+0000 7f24f2ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f24f40b0790 0x7f24f40b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:28.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.927+0000 7f24f2ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f24f40b0790 0x7f24f40b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54826/0 (socket says 172.21.15.55:54826) 2023-10-06T03:45:28.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.927+0000 7f24f2ffd700 1 -- 172.21.15.55:0/1790578391 learned_addr learned my addr 172.21.15.55:0/1790578391 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:28.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.927+0000 7f24f37fe700 1 --2- 172.21.15.55:0/1790578391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40b03c0 0x7f24f40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:28.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.927+0000 7f24f2ffd700 1 -- 172.21.15.55:0/1790578391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40b03c0 msgr2=0x7f24f40cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:28.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.927+0000 7f24f2ffd700 1 --2- 172.21.15.55:0/1790578391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40b03c0 0x7f24f40cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.927+0000 7f24f2ffd700 1 -- 172.21.15.55:0/1790578391 >> v1:172.21.15.163:6789/0 conn(0x7f24f41591e0 legacy=0x7f24f4159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:28.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.927+0000 7f24f2ffd700 1 -- 172.21.15.55:0/1790578391 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f40cf160 con 0x7f24f40b0790 2023-10-06T03:45:28.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.927+0000 7f24f2ffd700 1 --2- 172.21.15.55:0/1790578391 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24f40b0790 0x7f24f40b28a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f24e80099b0 tx=0x7f24e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cdabf10a70b3d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:28.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.927+0000 7f24f27fc700 1 -- 172.21.15.55:0/1790578391 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8009ea0 con 0x7f24f40b0790 2023-10-06T03:45:28.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.928+0000 7f24f27fc700 1 -- 172.21.15.55:0/1790578391 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e800b860 con 0x7f24f40b0790 2023-10-06T03:45:28.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.928+0000 7f24f27fc700 1 -- 172.21.15.55:0/1790578391 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e800bc00 con 0x7f24f40b0790 2023-10-06T03:45:28.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.928+0000 7f24f8d9d700 1 -- 172.21.15.55:0/1790578391 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24f40b0790 msgr2=0x7f24f40b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:28.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.928+0000 7f24f8d9d700 1 --2- 172.21.15.55:0/1790578391 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24f40b0790 0x7f24f40b28a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f24e80099b0 tx=0x7f24e8009250 comp rx=0 tx=0).stop 2023-10-06T03:45:28.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.928+0000 7f24f8d9d700 1 -- 172.21.15.55:0/1790578391 shutdown_connections 2023-10-06T03:45:28.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.928+0000 7f24f8d9d700 1 --2- 172.21.15.55:0/1790578391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40b03c0 0x7f24f40cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.928+0000 7f24f8d9d700 1 --2- 172.21.15.55:0/1790578391 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24f40b0790 0x7f24f40b28a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.928+0000 7f24f8d9d700 1 -- 172.21.15.55:0/1790578391 >> 172.21.15.55:0/1790578391 conn(0x7f24f4063230 msgr2=0x7f24f4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:28.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.928+0000 7f24f8d9d700 1 -- 172.21.15.55:0/1790578391 shutdown_connections 2023-10-06T03:45:28.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.928+0000 7f24f8d9d700 1 -- 172.21.15.55:0/1790578391 wait complete. 2023-10-06T03:45:28.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.930+0000 7f24f8d9d700 1 Processor -- start 2023-10-06T03:45:28.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.930+0000 7f24f8d9d700 1 -- start start 2023-10-06T03:45:28.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.931+0000 7f24f8d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f40b03c0 0x7f24f40c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:28.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.931+0000 7f24f8d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40b0790 0x7f24f4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:28.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.931+0000 7f24f8d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24f41591e0 0x7f24f415f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:28.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.931+0000 7f24f8d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40c4e90 con 0x7f24f41591e0 2023-10-06T03:45:28.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.931+0000 7f24f8d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40c5170 con 0x7f24f40b03c0 2023-10-06T03:45:28.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.931+0000 7f24f8d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40c5450 con 0x7f24f40b0790 2023-10-06T03:45:28.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.931+0000 7f24f3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24f41591e0 0x7f24f415f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:28.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.931+0000 7f24f3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24f41591e0 0x7f24f415f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54836/0 (socket says 172.21.15.55:54836) 2023-10-06T03:45:28.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.931+0000 7f24f3fff700 1 -- 172.21.15.55:0/3153675682 learned_addr learned my addr 172.21.15.55:0/3153675682 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:28.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.931+0000 7f24f37fe700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f40b03c0 0x7f24f40c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:28.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.931+0000 7f24f2ffd700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40b0790 0x7f24f4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:28.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.931+0000 7f24f3fff700 1 -- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40b0790 msgr2=0x7f24f4155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:28.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.931+0000 7f24f3fff700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40b0790 0x7f24f4155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.932+0000 7f24f3fff700 1 -- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f40b03c0 msgr2=0x7f24f40c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:28.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.932+0000 7f24f3fff700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f40b03c0 0x7f24f40c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:28.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.932+0000 7f24f3fff700 1 -- 172.21.15.55:0/3153675682 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e80094f0 con 0x7f24f41591e0 2023-10-06T03:45:28.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.932+0000 7f24f37fe700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f40b03c0 0x7f24f40c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:45:28.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.932+0000 7f24f3fff700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24f41591e0 0x7f24f415f400 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00c8e0 tx=0x7f24ec001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:28.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.932+0000 7f24db7fe700 1 -- 172.21.15.55:0/3153675682 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec00ce80 con 0x7f24f41591e0 2023-10-06T03:45:28.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.932+0000 7f24db7fe700 1 -- 172.21.15.55:0/3153675682 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec00a7e0 con 0x7f24f41591e0 2023-10-06T03:45:28.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.932+0000 7f24db7fe700 1 -- 172.21.15.55:0/3153675682 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec00ab80 con 0x7f24f41591e0 2023-10-06T03:45:28.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.932+0000 7f24f8d9d700 1 -- 172.21.15.55:0/3153675682 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f40c5730 con 0x7f24f41591e0 2023-10-06T03:45:28.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.932+0000 7f24f8d9d700 1 -- 172.21.15.55:0/3153675682 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f4156520 con 0x7f24f41591e0 2023-10-06T03:45:28.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.933+0000 7f24f8d9d700 1 -- 172.21.15.55:0/3153675682 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f40c8ba0 con 0x7f24f41591e0 2023-10-06T03:45:28.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.934+0000 7f24db7fe700 1 -- 172.21.15.55:0/3153675682 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f24ec012420 con 0x7f24f41591e0 2023-10-06T03:45:28.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.935+0000 7f24db7fe700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f24e0067130 0x7f24e00695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:28.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.935+0000 7f24db7fe700 1 -- 172.21.15.55:0/3153675682 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f24ec084ef0 con 0x7f24f41591e0 2023-10-06T03:45:28.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.935+0000 7f24f37fe700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f24e0067130 0x7f24e00695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:28.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.936+0000 7f24f37fe700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f24e0067130 0x7f24e00695e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f24f40c40a0 tx=0x7f24e4008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:28.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:28.939+0000 7f24db7fe700 1 -- 172.21.15.55:0/3153675682 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24ec051bc0 con 0x7f24f41591e0 2023-10-06T03:45:29.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.165+0000 7f24f8d9d700 1 -- 172.21.15.55:0/3153675682 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24f40c55c0 con 0x7f24e0067130 2023-10-06T03:45:29.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.177+0000 7f24db7fe700 1 -- 172.21.15.55:0/3153675682 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f24f40c55c0 con 0x7f24e0067130 2023-10-06T03:45:29.179 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:29.179 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:37", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:29.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.180+0000 7f24f8d9d700 1 -- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f24e0067130 msgr2=0x7f24e00695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:29.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.180+0000 7f24f8d9d700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f24e0067130 0x7f24e00695e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f24f40c40a0 tx=0x7f24e4008040 comp rx=0 tx=0).stop 2023-10-06T03:45:29.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.180+0000 7f24f8d9d700 1 -- 172.21.15.55:0/3153675682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24f41591e0 msgr2=0x7f24f415f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:29.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.180+0000 7f24f8d9d700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24f41591e0 0x7f24f415f400 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00c8e0 tx=0x7f24ec001710 comp rx=0 tx=0).stop 2023-10-06T03:45:29.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.180+0000 7f24f8d9d700 1 -- 172.21.15.55:0/3153675682 shutdown_connections 2023-10-06T03:45:29.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.180+0000 7f24f8d9d700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24f41591e0 0x7f24f415f400 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:29.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.180+0000 7f24f8d9d700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24f40b03c0 0x7f24f40c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:29.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.180+0000 7f24f8d9d700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f24e0067130 0x7f24e00695e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:29.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.180+0000 7f24f8d9d700 1 --2- 172.21.15.55:0/3153675682 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24f40b0790 0x7f24f4155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:29.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.180+0000 7f24f8d9d700 1 -- 172.21.15.55:0/3153675682 >> 172.21.15.55:0/3153675682 conn(0x7f24f4063230 msgr2=0x7f24f40699f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:29.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.180+0000 7f24f8d9d700 1 -- 172.21.15.55:0/3153675682 shutdown_connections 2023-10-06T03:45:29.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.180+0000 7f24f8d9d700 1 -- 172.21.15.55:0/3153675682 wait complete. 2023-10-06T03:45:29.199 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:37", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:29.199 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T03:34:37', 'first': '2023-10-06T03:35:00', 'last': '2023-10-06T03:45:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T03:45:29.199 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T03:45:29.199 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T03:45:29.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:29.200 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-06T03:45:29.239 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_01_UTC 2023-10-06T03:45:29.239 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:29.240 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:29.240 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:29.240 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:29.240 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:29.240 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:29.240 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:29.240 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:29.240 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:29.240 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:29.241 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-03_35_01_UTC', 'scheduled-2023-10-06-03_36_00_UTC', 'scheduled-2023-10-06-03_37_00_UTC', 'scheduled-2023-10-06-03_38_00_UTC', 'scheduled-2023-10-06-03_39_00_UTC', 'scheduled-2023-10-06-03_40_00_UTC', 'scheduled-2023-10-06-03_41_00_UTC', 'scheduled-2023-10-06-03_42_00_UTC', 'scheduled-2023-10-06-03_43_00_UTC', 'scheduled-2023-10-06-03_44_00_UTC', 'scheduled-2023-10-06-03_45_00_UTC'] 2023-10-06T03:45:29.241 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir6 2023-10-06T03:45:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.470+0000 7ffbcf59d700 1 Processor -- start 2023-10-06T03:45:29.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.470+0000 7ffbcf59d700 1 -- start start 2023-10-06T03:45:29.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.471+0000 7ffbcf59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffbc8156350 0x7ffbc8157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:29.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.472+0000 7ffbcf59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc80b0760 0x7ffbc80b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:29.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.472+0000 7ffbcf59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc80b1110 con 0x7ffbc8159f20 2023-10-06T03:45:29.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.472+0000 7ffbcf59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc80b0570 con 0x7ffbc8156350 2023-10-06T03:45:29.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.472+0000 7ffbcf59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc81578b0 con 0x7ffbc80b0760 2023-10-06T03:45:29.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.472+0000 7ffbced9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ffbc8159f20 0x7ffbc815a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35450/0 (socket says 172.21.15.55:35450) 2023-10-06T03:45:29.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.472+0000 7ffbced9c700 1 -- 172.21.15.55:0/3622477140 learned_addr learned my addr 172.21.15.55:0/3622477140 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:29.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.472+0000 7ffbce59b700 1 --2- 172.21.15.55:0/3622477140 >> v2:172.21.15.55:3300/0 conn(0x7ffbc8156350 0x7ffbc8157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:29.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.472+0000 7ffbcdd9a700 1 --2- 172.21.15.55:0/3622477140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc80b0760 0x7ffbc80b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:29.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.472+0000 7ffbcd599700 1 -- 172.21.15.55:0/3622477140 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144623762 0 0) 0x7ffbc80b1110 con 0x7ffbc8159f20 2023-10-06T03:45:29.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.472+0000 7ffbcd599700 1 -- 172.21.15.55:0/3622477140 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbb4003570 con 0x7ffbc8159f20 2023-10-06T03:45:29.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.472+0000 7ffbce59b700 1 -- 172.21.15.55:0/3622477140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc80b0760 msgr2=0x7ffbc80b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:29.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.472+0000 7ffbce59b700 1 --2- 172.21.15.55:0/3622477140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc80b0760 0x7ffbc80b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:29.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.472+0000 7ffbce59b700 1 -- 172.21.15.55:0/3622477140 >> v1:172.21.15.55:6789/0 conn(0x7ffbc8159f20 legacy=0x7ffbc815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:29.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.472+0000 7ffbce59b700 1 -- 172.21.15.55:0/3622477140 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc80af4d0 con 0x7ffbc8156350 2023-10-06T03:45:29.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.473+0000 7ffbce59b700 1 --2- 172.21.15.55:0/3622477140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbc8156350 0x7ffbc8157370 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40098e0 tx=0x7ffbc40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f51850724db002 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:29.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.473+0000 7ffbcd599700 1 -- 172.21.15.55:0/3622477140 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc400b640 con 0x7ffbc8156350 2023-10-06T03:45:29.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.473+0000 7ffbcd599700 1 -- 172.21.15.55:0/3622477140 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc400b7a0 con 0x7ffbc8156350 2023-10-06T03:45:29.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.473+0000 7ffbcd599700 1 -- 172.21.15.55:0/3622477140 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc400bb40 con 0x7ffbc8156350 2023-10-06T03:45:29.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.473+0000 7ffbcf59d700 1 -- 172.21.15.55:0/3622477140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbc8156350 msgr2=0x7ffbc8157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:29.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.473+0000 7ffbcf59d700 1 --2- 172.21.15.55:0/3622477140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbc8156350 0x7ffbc8157370 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40098e0 tx=0x7ffbc40091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:29.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.473+0000 7ffbcf59d700 1 -- 172.21.15.55:0/3622477140 shutdown_connections 2023-10-06T03:45:29.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.473+0000 7ffbcf59d700 1 --2- 172.21.15.55:0/3622477140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbc8156350 0x7ffbc8157370 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:29.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.473+0000 7ffbcf59d700 1 --2- 172.21.15.55:0/3622477140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc80b0760 0x7ffbc80b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:29.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.474+0000 7ffbcf59d700 1 -- 172.21.15.55:0/3622477140 >> 172.21.15.55:0/3622477140 conn(0x7ffbc8068e20 msgr2=0x7ffbc80cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:29.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.474+0000 7ffbcf59d700 1 -- 172.21.15.55:0/3622477140 shutdown_connections 2023-10-06T03:45:29.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.474+0000 7ffbcf59d700 1 -- 172.21.15.55:0/3622477140 wait complete. 2023-10-06T03:45:29.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.475+0000 7ffbcf59d700 1 Processor -- start 2023-10-06T03:45:29.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.475+0000 7ffbcf59d700 1 -- start start 2023-10-06T03:45:29.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.476+0000 7ffbcf59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc80b0760 0x7ffbc81510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:29.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.476+0000 7ffbcf59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc8156350 0x7ffbc80caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:29.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.476+0000 7ffbcf59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbc8159f20 0x7ffbc8154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:29.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.476+0000 7ffbcf59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc816a940 con 0x7ffbc8159f20 2023-10-06T03:45:29.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.476+0000 7ffbcf59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc80cb8b0 con 0x7ffbc80b0760 2023-10-06T03:45:29.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.476+0000 7ffbcf59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc816a2f0 con 0x7ffbc8156350 2023-10-06T03:45:29.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.476+0000 7ffbced9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbc8159f20 0x7ffbc8154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:29.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.476+0000 7ffbced9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbc8159f20 0x7ffbc8154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54850/0 (socket says 172.21.15.55:54850) 2023-10-06T03:45:29.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.476+0000 7ffbced9c700 1 -- 172.21.15.55:0/386037373 learned_addr learned my addr 172.21.15.55:0/386037373 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:29.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.477+0000 7ffbcdd9a700 1 --2- 172.21.15.55:0/386037373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc8156350 0x7ffbc80caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:29.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.477+0000 7ffbce59b700 1 --2- 172.21.15.55:0/386037373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc80b0760 0x7ffbc81510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:29.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.477+0000 7ffbced9c700 1 -- 172.21.15.55:0/386037373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc8156350 msgr2=0x7ffbc80caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:29.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.477+0000 7ffbced9c700 1 --2- 172.21.15.55:0/386037373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc8156350 0x7ffbc80caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:29.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.477+0000 7ffbced9c700 1 -- 172.21.15.55:0/386037373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc80b0760 msgr2=0x7ffbc81510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:29.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.477+0000 7ffbced9c700 1 --2- 172.21.15.55:0/386037373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc80b0760 0x7ffbc81510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:29.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.477+0000 7ffbced9c700 1 -- 172.21.15.55:0/386037373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc4009420 con 0x7ffbc8159f20 2023-10-06T03:45:29.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.477+0000 7ffbced9c700 1 --2- 172.21.15.55:0/386037373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbc8159f20 0x7ffbc8154c50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0001460 tx=0x7ffbc0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:29.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.477+0000 7ffbbe7fc700 1 -- 172.21.15.55:0/386037373 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc000a870 con 0x7ffbc8159f20 2023-10-06T03:45:29.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.477+0000 7ffbbe7fc700 1 -- 172.21.15.55:0/386037373 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc000a9d0 con 0x7ffbc8159f20 2023-10-06T03:45:29.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.477+0000 7ffbcf59d700 1 -- 172.21.15.55:0/386037373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc816a5d0 con 0x7ffbc8159f20 2023-10-06T03:45:29.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.477+0000 7ffbbe7fc700 1 -- 172.21.15.55:0/386037373 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc000ad50 con 0x7ffbc8159f20 2023-10-06T03:45:29.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.478+0000 7ffbcf59d700 1 -- 172.21.15.55:0/386037373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc80caae0 con 0x7ffbc8159f20 2023-10-06T03:45:29.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.478+0000 7ffbcf59d700 1 -- 172.21.15.55:0/386037373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc804c3c0 con 0x7ffbc8159f20 2023-10-06T03:45:29.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.479+0000 7ffbbe7fc700 1 -- 172.21.15.55:0/386037373 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7ffbc0007300 con 0x7ffbc8159f20 2023-10-06T03:45:29.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.480+0000 7ffbbe7fc700 1 --2- 172.21.15.55:0/386037373 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ffbac067130 0x7ffbac0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:29.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.480+0000 7ffbbe7fc700 1 -- 172.21.15.55:0/386037373 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7ffbc0084ea0 con 0x7ffbc8159f20 2023-10-06T03:45:29.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.481+0000 7ffbce59b700 1 --2- 172.21.15.55:0/386037373 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ffbac067130 0x7ffbac0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:29.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.481+0000 7ffbce59b700 1 --2- 172.21.15.55:0/386037373 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ffbac067130 0x7ffbac0695e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40093f0 tx=0x7ffbc4017000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:29.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.484+0000 7ffbbe7fc700 1 -- 172.21.15.55:0/386037373 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffbc0051ca0 con 0x7ffbc8159f20 2023-10-06T03:45:29.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.714+0000 7ffbcf59d700 1 -- 172.21.15.55:0/386037373 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffbc816aa80 con 0x7ffbac067130 2023-10-06T03:45:29.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.724+0000 7ffbbe7fc700 1 -- 172.21.15.55:0/386037373 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7ffbc816aa80 con 0x7ffbac067130 2023-10-06T03:45:29.726 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:29.726 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:38", "first": "2023-10-06T03:35:01", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.727+0000 7ffbcf59d700 1 -- 172.21.15.55:0/386037373 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ffbac067130 msgr2=0x7ffbac0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.727+0000 7ffbcf59d700 1 --2- 172.21.15.55:0/386037373 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ffbac067130 0x7ffbac0695e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40093f0 tx=0x7ffbc4017000 comp rx=0 tx=0).stop 2023-10-06T03:45:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.727+0000 7ffbcf59d700 1 -- 172.21.15.55:0/386037373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbc8159f20 msgr2=0x7ffbc8154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.727+0000 7ffbcf59d700 1 --2- 172.21.15.55:0/386037373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbc8159f20 0x7ffbc8154c50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0001460 tx=0x7ffbc0001a40 comp rx=0 tx=0).stop 2023-10-06T03:45:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.727+0000 7ffbcf59d700 1 -- 172.21.15.55:0/386037373 shutdown_connections 2023-10-06T03:45:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.727+0000 7ffbcf59d700 1 --2- 172.21.15.55:0/386037373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbc8159f20 0x7ffbc8154c50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.727+0000 7ffbcf59d700 1 --2- 172.21.15.55:0/386037373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbc80b0760 0x7ffbc81510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.727+0000 7ffbcf59d700 1 --2- 172.21.15.55:0/386037373 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ffbac067130 0x7ffbac0695e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.727+0000 7ffbcf59d700 1 --2- 172.21.15.55:0/386037373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbc8156350 0x7ffbc80caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.727+0000 7ffbcf59d700 1 -- 172.21.15.55:0/386037373 >> 172.21.15.55:0/386037373 conn(0x7ffbc8068e20 msgr2=0x7ffbc8152ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:29.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.727+0000 7ffbcf59d700 1 -- 172.21.15.55:0/386037373 shutdown_connections 2023-10-06T03:45:29.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:29.727+0000 7ffbcf59d700 1 -- 172.21.15.55:0/386037373 wait complete. 2023-10-06T03:45:29.746 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:38", "first": "2023-10-06T03:35:01", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:29.746 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T03:34:38', 'first': '2023-10-06T03:35:01', 'last': '2023-10-06T03:45:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T03:45:29.746 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T03:45:29.746 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T03:45:29.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:29.746 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-06T03:45:29.785 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:29.785 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:29.786 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:29.786 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:29.786 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:29.786 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:29.786 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:29.786 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:29.786 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:29.786 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:29.786 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:29.788 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-03_35_00_UTC', 'scheduled-2023-10-06-03_36_00_UTC', 'scheduled-2023-10-06-03_37_00_UTC', 'scheduled-2023-10-06-03_38_00_UTC', 'scheduled-2023-10-06-03_39_00_UTC', 'scheduled-2023-10-06-03_40_00_UTC', 'scheduled-2023-10-06-03_41_00_UTC', 'scheduled-2023-10-06-03_42_00_UTC', 'scheduled-2023-10-06-03_43_00_UTC', 'scheduled-2023-10-06-03_44_00_UTC', 'scheduled-2023-10-06-03_45_00_UTC'] 2023-10-06T03:45:29.788 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir7 2023-10-06T03:45:30.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.018+0000 7fe1bd79d700 1 Processor -- start 2023-10-06T03:45:30.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.019+0000 7fe1bd79d700 1 -- start start 2023-10-06T03:45:30.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.020+0000 7fe1bd79d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1b80685e0 0x7fe1b815a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:30.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.020+0000 7fe1bd79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b815d3f0 0x7fe1b81538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:30.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.020+0000 7fe1bd79d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe1b80689b0 0x7fe1b8153e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:30.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.020+0000 7fe1bd79d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b80c9480 con 0x7fe1b80685e0 2023-10-06T03:45:30.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.020+0000 7fe1bd79d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b815d880 con 0x7fe1b80689b0 2023-10-06T03:45:30.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.020+0000 7fe1bd79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b815db60 con 0x7fe1b815d3f0 2023-10-06T03:45:30.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.020+0000 7fe1b7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1b80685e0 0x7fe1b815a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:30.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.020+0000 7fe1b7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1b80685e0 0x7fe1b815a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54852/0 (socket says 172.21.15.55:54852) 2023-10-06T03:45:30.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.020+0000 7fe1b7fff700 1 -- 172.21.15.55:0/1129254656 learned_addr learned my addr 172.21.15.55:0/1129254656 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:30.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.021+0000 7fe1b77fe700 1 --2- 172.21.15.55:0/1129254656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b815d3f0 0x7fe1b81538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:30.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.021+0000 7fe1bcf9c700 1 --2- 172.21.15.55:0/1129254656 >> v2:172.21.15.163:3300/0 conn(0x7fe1b80689b0 0x7fe1b8153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:30.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.021+0000 7fe1b7fff700 1 -- 172.21.15.55:0/1129254656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b815d3f0 msgr2=0x7fe1b81538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:30.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.021+0000 7fe1b7fff700 1 --2- 172.21.15.55:0/1129254656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b815d3f0 0x7fe1b81538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.021+0000 7fe1b7fff700 1 -- 172.21.15.55:0/1129254656 >> v2:172.21.15.163:3300/0 conn(0x7fe1b80689b0 msgr2=0x7fe1b8153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:30.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.021+0000 7fe1b7fff700 1 --2- 172.21.15.55:0/1129254656 >> v2:172.21.15.163:3300/0 conn(0x7fe1b80689b0 0x7fe1b8153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.021+0000 7fe1b7fff700 1 -- 172.21.15.55:0/1129254656 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b815de40 con 0x7fe1b80685e0 2023-10-06T03:45:30.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.021+0000 7fe1bcf9c700 1 --2- 172.21.15.55:0/1129254656 >> v2:172.21.15.163:3300/0 conn(0x7fe1b80689b0 0x7fe1b8153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:45:30.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.021+0000 7fe1b7fff700 1 --2- 172.21.15.55:0/1129254656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b80685e0 0x7fe1b815a6f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe1b806e1a0 tx=0x7fe1a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3021193cdb647a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:30.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.021+0000 7fe1b6ffd700 1 -- 172.21.15.55:0/1129254656 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8009e70 con 0x7fe1b80685e0 2023-10-06T03:45:30.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.021+0000 7fe1b6ffd700 1 -- 172.21.15.55:0/1129254656 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a800b8f0 con 0x7fe1b80685e0 2023-10-06T03:45:30.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.021+0000 7fe1b6ffd700 1 -- 172.21.15.55:0/1129254656 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a800bc90 con 0x7fe1b80685e0 2023-10-06T03:45:30.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.022+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1129254656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b80685e0 msgr2=0x7fe1b815a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:30.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.022+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/1129254656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b80685e0 0x7fe1b815a6f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe1b806e1a0 tx=0x7fe1a8009250 comp rx=0 tx=0).stop 2023-10-06T03:45:30.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.022+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1129254656 shutdown_connections 2023-10-06T03:45:30.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.022+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/1129254656 >> v2:172.21.15.163:3300/0 conn(0x7fe1b80689b0 0x7fe1b8153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.022+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/1129254656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b80685e0 0x7fe1b815a6f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.022+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/1129254656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b815d3f0 0x7fe1b81538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.022+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1129254656 >> 172.21.15.55:0/1129254656 conn(0x7fe1b806b420 msgr2=0x7fe1b80cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:30.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.022+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1129254656 shutdown_connections 2023-10-06T03:45:30.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.022+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1129254656 wait complete. 2023-10-06T03:45:30.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.024+0000 7fe1bd79d700 1 Processor -- start 2023-10-06T03:45:30.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.024+0000 7fe1bd79d700 1 -- start start 2023-10-06T03:45:30.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.024+0000 7fe1bd79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b80685e0 0x7fe1b8145ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:30.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.024+0000 7fe1bd79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80689b0 0x7fe1b8169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:30.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1bd79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1b815d3f0 0x7fe1b816a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:30.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1bd79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b81544f0 con 0x7fe1b80685e0 2023-10-06T03:45:30.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1bd79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b816d8e0 con 0x7fe1b815d3f0 2023-10-06T03:45:30.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1bd79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b816d2b0 con 0x7fe1b80689b0 2023-10-06T03:45:30.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1b7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b80685e0 0x7fe1b8145ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:30.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1b7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b80685e0 0x7fe1b8145ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54868/0 (socket says 172.21.15.55:54868) 2023-10-06T03:45:30.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1b7fff700 1 -- 172.21.15.55:0/1451507330 learned_addr learned my addr 172.21.15.55:0/1451507330 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:30.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1bcf9c700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1b815d3f0 0x7fe1b816a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:30.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1b77fe700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80689b0 0x7fe1b8169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:30.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1b7fff700 1 -- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80689b0 msgr2=0x7fe1b8169cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:30.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1b7fff700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80689b0 0x7fe1b8169cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1b7fff700 1 -- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1b815d3f0 msgr2=0x7fe1b816a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:30.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1b7fff700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1b815d3f0 0x7fe1b816a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1b7fff700 1 -- 172.21.15.55:0/1451507330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a80094f0 con 0x7fe1b80685e0 2023-10-06T03:45:30.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1bcf9c700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1b815d3f0 0x7fe1b816a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:45:30.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.025+0000 7fe1b7fff700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b80685e0 0x7fe1b8145ec0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009290 tx=0x7fe1a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:30.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.026+0000 7fe19ffff700 1 -- 172.21.15.55:0/1451507330 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8026480 con 0x7fe1b80685e0 2023-10-06T03:45:30.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.026+0000 7fe19ffff700 1 -- 172.21.15.55:0/1451507330 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a801f070 con 0x7fe1b80685e0 2023-10-06T03:45:30.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.026+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1451507330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b816a7e0 con 0x7fe1b80685e0 2023-10-06T03:45:30.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.026+0000 7fe19ffff700 1 -- 172.21.15.55:0/1451507330 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8026850 con 0x7fe1b80685e0 2023-10-06T03:45:30.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.026+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1451507330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b816d570 con 0x7fe1b80685e0 2023-10-06T03:45:30.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.027+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1451507330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b804c3c0 con 0x7fe1b80685e0 2023-10-06T03:45:30.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.028+0000 7fe19ffff700 1 -- 172.21.15.55:0/1451507330 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fe1a801a430 con 0x7fe1b80685e0 2023-10-06T03:45:30.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.028+0000 7fe19ffff700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1a4067130 0x7fe1a40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:30.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.029+0000 7fe19ffff700 1 -- 172.21.15.55:0/1451507330 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fe1a8021070 con 0x7fe1b80685e0 2023-10-06T03:45:30.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.029+0000 7fe1b77fe700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1a4067130 0x7fe1a40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:30.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.030+0000 7fe1b77fe700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1a4067130 0x7fe1a40695e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80c61c0 tx=0x7fe1ac008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:30.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.033+0000 7fe19ffff700 1 -- 172.21.15.55:0/1451507330 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1a80537a0 con 0x7fe1b80685e0 2023-10-06T03:45:30.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.264+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1451507330 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe1b80c9c60 con 0x7fe1a4067130 2023-10-06T03:45:30.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.274+0000 7fe19ffff700 1 -- 172.21.15.55:0/1451507330 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fe1b80c9c60 con 0x7fe1a4067130 2023-10-06T03:45:30.276 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:30.276 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:39", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:30.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.277+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1a4067130 msgr2=0x7fe1a40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:30.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.277+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1a4067130 0x7fe1a40695e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80c61c0 tx=0x7fe1ac008040 comp rx=0 tx=0).stop 2023-10-06T03:45:30.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.277+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1451507330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b80685e0 msgr2=0x7fe1b8145ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:30.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.277+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b80685e0 0x7fe1b8145ec0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8009290 tx=0x7fe1a8017000 comp rx=0 tx=0).stop 2023-10-06T03:45:30.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.277+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1451507330 shutdown_connections 2023-10-06T03:45:30.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.277+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1b815d3f0 0x7fe1b816a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.277+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1b80685e0 0x7fe1b8145ec0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.277+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fe1a4067130 0x7fe1a40695e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.277+0000 7fe1bd79d700 1 --2- 172.21.15.55:0/1451507330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1b80689b0 0x7fe1b8169cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.277+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1451507330 >> 172.21.15.55:0/1451507330 conn(0x7fe1b806b420 msgr2=0x7fe1b81518d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:30.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.277+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1451507330 shutdown_connections 2023-10-06T03:45:30.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.277+0000 7fe1bd79d700 1 -- 172.21.15.55:0/1451507330 wait complete. 2023-10-06T03:45:30.296 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:39", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:30.296 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T03:34:39', 'first': '2023-10-06T03:35:00', 'last': '2023-10-06T03:45:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T03:45:30.297 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T03:45:30.297 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T03:45:30.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:30.297 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-06T03:45:30.336 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:30.337 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:30.337 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:30.337 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:30.337 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:30.337 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:30.337 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:30.337 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:30.337 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:30.337 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:30.338 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:30.338 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-03_35_00_UTC', 'scheduled-2023-10-06-03_36_00_UTC', 'scheduled-2023-10-06-03_37_00_UTC', 'scheduled-2023-10-06-03_38_00_UTC', 'scheduled-2023-10-06-03_39_00_UTC', 'scheduled-2023-10-06-03_40_00_UTC', 'scheduled-2023-10-06-03_41_00_UTC', 'scheduled-2023-10-06-03_42_00_UTC', 'scheduled-2023-10-06-03_43_00_UTC', 'scheduled-2023-10-06-03_44_00_UTC', 'scheduled-2023-10-06-03_45_00_UTC'] 2023-10-06T03:45:30.339 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir8 2023-10-06T03:45:30.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.569+0000 7f7ee09fe700 1 Processor -- start 2023-10-06T03:45:30.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.569+0000 7f7ee09fe700 1 -- start start 2023-10-06T03:45:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.571+0000 7f7ee09fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7edc0b0c00 0x7f7edc0ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.571+0000 7f7ee09fe700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc0b0570 con 0x7f7edc156350 2023-10-06T03:45:30.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.571+0000 7f7ee09fe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc0af4d0 con 0x7f7edc0b0760 2023-10-06T03:45:30.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.571+0000 7f7ee09fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7edc0af7b0 con 0x7f7edc0b0c00 2023-10-06T03:45:30.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.571+0000 7f7edad9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7edc156350 0x7f7edc157370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35452/0 (socket says 172.21.15.55:35452) 2023-10-06T03:45:30.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.571+0000 7f7edad9d700 1 -- 172.21.15.55:0/913147480 learned_addr learned my addr 172.21.15.55:0/913147480 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:30.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.571+0000 7f7edb59e700 1 --2- 172.21.15.55:0/913147480 >> v2:172.21.15.163:3300/0 conn(0x7f7edc0b0c00 0x7f7edc0ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:30.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.571+0000 7f7ed9d9b700 1 -- 172.21.15.55:0/913147480 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3314469317 0 0) 0x7f7edc0b0570 con 0x7f7edc156350 2023-10-06T03:45:30.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.572+0000 7f7ed9d9b700 1 -- 172.21.15.55:0/913147480 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec4003570 con 0x7f7edc156350 2023-10-06T03:45:30.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.572+0000 7f7ed9d9b700 1 -- 172.21.15.55:0/913147480 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645898697 0 0) 0x7f7edc0af4d0 con 0x7f7edc0b0760 2023-10-06T03:45:30.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.572+0000 7f7ed9d9b700 1 -- 172.21.15.55:0/913147480 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7edc0b0570 con 0x7f7edc0b0760 2023-10-06T03:45:30.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.572+0000 7f7ed9d9b700 1 -- 172.21.15.55:0/913147480 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3495202836 0 0) 0x7f7ec4003570 con 0x7f7edc156350 2023-10-06T03:45:30.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.572+0000 7f7edb59e700 1 -- 172.21.15.55:0/913147480 >> v1:172.21.15.163:6789/0 conn(0x7f7edc0b0760 legacy=0x7f7edc0ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:30.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.572+0000 7f7edb59e700 1 -- 172.21.15.55:0/913147480 >> v1:172.21.15.55:6789/0 conn(0x7f7edc156350 legacy=0x7f7edc157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:30.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.572+0000 7f7edb59e700 1 -- 172.21.15.55:0/913147480 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc0afa90 con 0x7f7edc0b0c00 2023-10-06T03:45:30.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.572+0000 7f7edb59e700 1 --2- 172.21.15.55:0/913147480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0b0c00 0x7f7edc0ad570 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009e60 tx=0x7f7ed4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a610f6992f1a4ac2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:30.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.573+0000 7f7ed9d9b700 1 -- 172.21.15.55:0/913147480 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed400e070 con 0x7f7edc0b0c00 2023-10-06T03:45:30.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.573+0000 7f7ed9d9b700 1 -- 172.21.15.55:0/913147480 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed400b7a0 con 0x7f7edc0b0c00 2023-10-06T03:45:30.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.573+0000 7f7ed9d9b700 1 -- 172.21.15.55:0/913147480 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc001470 con 0x7f7edc0b0c00 2023-10-06T03:45:30.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.573+0000 7f7ee09fe700 1 -- 172.21.15.55:0/913147480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0b0c00 msgr2=0x7f7edc0ad570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:30.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.573+0000 7f7ee09fe700 1 --2- 172.21.15.55:0/913147480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0b0c00 0x7f7edc0ad570 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009e60 tx=0x7f7ed4009780 comp rx=0 tx=0).stop 2023-10-06T03:45:30.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.573+0000 7f7ee09fe700 1 -- 172.21.15.55:0/913147480 shutdown_connections 2023-10-06T03:45:30.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.573+0000 7f7ee09fe700 1 --2- 172.21.15.55:0/913147480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0b0c00 0x7f7edc0ad570 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.573+0000 7f7ee09fe700 1 -- 172.21.15.55:0/913147480 >> 172.21.15.55:0/913147480 conn(0x7f7edc068e20 msgr2=0x7f7edc0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:30.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.573+0000 7f7ee09fe700 1 -- 172.21.15.55:0/913147480 shutdown_connections 2023-10-06T03:45:30.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.573+0000 7f7ee09fe700 1 -- 172.21.15.55:0/913147480 wait complete. 2023-10-06T03:45:30.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.575+0000 7f7ee09fe700 1 Processor -- start 2023-10-06T03:45:30.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.575+0000 7f7ee09fe700 1 -- start start 2023-10-06T03:45:30.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.576+0000 7f7ee09fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0b0760 0x7f7edc0cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:30.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.576+0000 7f7ee09fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7edc0b0c00 0x7f7edc0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:30.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.576+0000 7f7ee09fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7edc156350 0x7f7edc155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:30.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.576+0000 7f7ee09fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0cb4f0 con 0x7f7edc0b0c00 2023-10-06T03:45:30.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.576+0000 7f7ee09fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0ca5d0 con 0x7f7edc0b0760 2023-10-06T03:45:30.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.576+0000 7f7ee09fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0ca870 con 0x7f7edc156350 2023-10-06T03:45:30.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.576+0000 7f7eda59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7edc0b0c00 0x7f7edc0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:30.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.576+0000 7f7eda59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7edc0b0c00 0x7f7edc0ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54882/0 (socket says 172.21.15.55:54882) 2023-10-06T03:45:30.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.576+0000 7f7eda59c700 1 -- 172.21.15.55:0/699590086 learned_addr learned my addr 172.21.15.55:0/699590086 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:30.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.576+0000 7f7edb59e700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7edc156350 0x7f7edc155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:30.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.576+0000 7f7edad9d700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0b0760 0x7f7edc0cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:30.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.576+0000 7f7eda59c700 1 -- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7edc156350 msgr2=0x7f7edc155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:30.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.576+0000 7f7eda59c700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7edc156350 0x7f7edc155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.576+0000 7f7eda59c700 1 -- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0b0760 msgr2=0x7f7edc0cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:30.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.577+0000 7f7eda59c700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0b0760 0x7f7edc0cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.577+0000 7f7eda59c700 1 -- 172.21.15.55:0/699590086 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed4009a20 con 0x7f7edc0b0c00 2023-10-06T03:45:30.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.577+0000 7f7edad9d700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0b0760 0x7f7edc0cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:45:30.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.577+0000 7f7eda59c700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7edc0b0c00 0x7f7edc0ca090 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000c580 tx=0x7f7ed0007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:30.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.577+0000 7f7ec2ffd700 1 -- 172.21.15.55:0/699590086 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0010510 con 0x7f7edc0b0c00 2023-10-06T03:45:30.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.577+0000 7f7ec2ffd700 1 -- 172.21.15.55:0/699590086 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed0010670 con 0x7f7edc0b0c00 2023-10-06T03:45:30.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.577+0000 7f7ee09fe700 1 -- 172.21.15.55:0/699590086 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc0cab50 con 0x7f7edc0b0c00 2023-10-06T03:45:30.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.577+0000 7f7ec2ffd700 1 -- 172.21.15.55:0/699590086 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0010a30 con 0x7f7edc0b0c00 2023-10-06T03:45:30.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.577+0000 7f7ee09fe700 1 -- 172.21.15.55:0/699590086 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc0cb110 con 0x7f7edc0b0c00 2023-10-06T03:45:30.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.578+0000 7f7ee09fe700 1 -- 172.21.15.55:0/699590086 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7edc0cb630 con 0x7f7edc0b0c00 2023-10-06T03:45:30.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.579+0000 7f7ec2ffd700 1 -- 172.21.15.55:0/699590086 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f7ed001b400 con 0x7f7edc0b0c00 2023-10-06T03:45:30.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.580+0000 7f7ec2ffd700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7ec8067130 0x7f7ec80695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:30.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.580+0000 7f7ec2ffd700 1 -- 172.21.15.55:0/699590086 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f7ed0084750 con 0x7f7edc0b0c00 2023-10-06T03:45:30.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.580+0000 7f7edad9d700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7ec8067130 0x7f7ec80695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:30.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.581+0000 7f7edad9d700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7ec8067130 0x7f7ec80695e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0011e0 tx=0x7f7ecc000bc0 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:30.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.584+0000 7f7ec2ffd700 1 -- 172.21.15.55:0/699590086 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7edc0cb630 con 0x7f7edc0b0c00 2023-10-06T03:45:30.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.815+0000 7f7ee09fe700 1 -- 172.21.15.55:0/699590086 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7edc0cb630 con 0x7f7ec8067130 2023-10-06T03:45:30.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.825+0000 7f7ec2ffd700 1 -- 172.21.15.55:0/699590086 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f7edc0cb630 con 0x7f7ec8067130 2023-10-06T03:45:30.827 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:30.827 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:39", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:30.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.828+0000 7f7ee09fe700 1 -- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7ec8067130 msgr2=0x7f7ec80695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:30.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.828+0000 7f7ee09fe700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7ec8067130 0x7f7ec80695e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0011e0 tx=0x7f7ecc000bc0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.828+0000 7f7ee09fe700 1 -- 172.21.15.55:0/699590086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7edc0b0c00 msgr2=0x7f7edc0ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:30.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.828+0000 7f7ee09fe700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7edc0b0c00 0x7f7edc0ca090 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000c580 tx=0x7f7ed0007e10 comp rx=0 tx=0).stop 2023-10-06T03:45:30.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.828+0000 7f7ee09fe700 1 -- 172.21.15.55:0/699590086 shutdown_connections 2023-10-06T03:45:30.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.828+0000 7f7ee09fe700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7ec8067130 0x7f7ec80695e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.828+0000 7f7ee09fe700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7edc156350 0x7f7edc155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.828+0000 7f7ee09fe700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7edc0b0760 0x7f7edc0cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.828+0000 7f7ee09fe700 1 --2- 172.21.15.55:0/699590086 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7edc0b0c00 0x7f7edc0ca090 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:30.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.828+0000 7f7ee09fe700 1 -- 172.21.15.55:0/699590086 >> 172.21.15.55:0/699590086 conn(0x7f7edc068e20 msgr2=0x7f7edc0add60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:30.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.828+0000 7f7ee09fe700 1 -- 172.21.15.55:0/699590086 shutdown_connections 2023-10-06T03:45:30.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:30.828+0000 7f7ee09fe700 1 -- 172.21.15.55:0/699590086 wait complete. 2023-10-06T03:45:30.847 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:39", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:30.847 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T03:34:39', 'first': '2023-10-06T03:35:00', 'last': '2023-10-06T03:45:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T03:45:30.847 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T03:45:30.848 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T03:45:30.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:30.848 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-06T03:45:30.887 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:30.887 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:30.887 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:30.887 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:30.887 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:30.887 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:30.887 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:30.888 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:30.888 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:30.888 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:30.888 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:30.889 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-03_35_00_UTC', 'scheduled-2023-10-06-03_36_00_UTC', 'scheduled-2023-10-06-03_37_00_UTC', 'scheduled-2023-10-06-03_38_00_UTC', 'scheduled-2023-10-06-03_39_00_UTC', 'scheduled-2023-10-06-03_40_00_UTC', 'scheduled-2023-10-06-03_41_00_UTC', 'scheduled-2023-10-06-03_42_00_UTC', 'scheduled-2023-10-06-03_43_00_UTC', 'scheduled-2023-10-06-03_44_00_UTC', 'scheduled-2023-10-06-03_45_00_UTC'] 2023-10-06T03:45:30.889 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir9 2023-10-06T03:45:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.118+0000 7f7f8199d700 1 Processor -- start 2023-10-06T03:45:31.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.118+0000 7f7f8199d700 1 -- start start 2023-10-06T03:45:31.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.119+0000 7f7f8199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f7c0cb560 0x7f7f7c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:31.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.119+0000 7f7f8199d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f7c0cc970 0x7f7f7c0697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:31.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.119+0000 7f7f8199d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7c0ccde0 con 0x7f7f7c069cf0 2023-10-06T03:45:31.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.120+0000 7f7f8199d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7c06aa30 con 0x7f7f7c0cc970 2023-10-06T03:45:31.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.120+0000 7f7f8199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c159db0 con 0x7f7f7c0cb560 2023-10-06T03:45:31.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.120+0000 7f7f8099b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f7c0cb560 0x7f7f7c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:31.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.120+0000 7f7f7bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f7c0cc970 0x7f7f7c0697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:31.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.120+0000 7f7f8119c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f7f7c069cf0 0x7f7f7c06d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:55372/0 (socket says 172.21.15.55:55372) 2023-10-06T03:45:31.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.120+0000 7f7f8099b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f7c0cb560 0x7f7f7c0cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:51418/0 (socket says 172.21.15.55:51418) 2023-10-06T03:45:31.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.120+0000 7f7f8119c700 1 -- 172.21.15.55:0/2016897666 learned_addr learned my addr 172.21.15.55:0/2016897666 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:31.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.120+0000 7f7f8099b700 1 -- 172.21.15.55:0/2016897666 >> v2:172.21.15.163:3300/0 conn(0x7f7f7c0cc970 msgr2=0x7f7f7c0697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:31.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.120+0000 7f7f8099b700 1 --2- 172.21.15.55:0/2016897666 >> v2:172.21.15.163:3300/0 conn(0x7f7f7c0cc970 0x7f7f7c0697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.120+0000 7f7f8099b700 1 -- 172.21.15.55:0/2016897666 >> v1:172.21.15.163:6789/0 conn(0x7f7f7c069cf0 legacy=0x7f7f7c06d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.120+0000 7f7f8099b700 1 -- 172.21.15.55:0/2016897666 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c06a470 con 0x7f7f7c0cb560 2023-10-06T03:45:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.120+0000 7f7f7b7fe700 1 -- 172.21.15.55:0/2016897666 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100553286 0 0) 0x7f7f7c0ccde0 con 0x7f7f7c069cf0 2023-10-06T03:45:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.121+0000 7f7f8099b700 1 --2- 172.21.15.55:0/2016897666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f7c0cb560 0x7f7f7c0cc430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7f700098e0 tx=0x7f7f700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d95df5bbea00b30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.121+0000 7f7f7b7fe700 1 -- 172.21.15.55:0/2016897666 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7000b640 con 0x7f7f7c0cb560 2023-10-06T03:45:31.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.121+0000 7f7f7b7fe700 1 -- 172.21.15.55:0/2016897666 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7000b7a0 con 0x7f7f7c0cb560 2023-10-06T03:45:31.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.121+0000 7f7f7b7fe700 1 -- 172.21.15.55:0/2016897666 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7000bb40 con 0x7f7f7c0cb560 2023-10-06T03:45:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.122+0000 7f7f8199d700 1 -- 172.21.15.55:0/2016897666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f7c0cb560 msgr2=0x7f7f7c0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.122+0000 7f7f8199d700 1 --2- 172.21.15.55:0/2016897666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f7c0cb560 0x7f7f7c0cc430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7f700098e0 tx=0x7f7f700091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.122+0000 7f7f8199d700 1 -- 172.21.15.55:0/2016897666 shutdown_connections 2023-10-06T03:45:31.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.122+0000 7f7f8199d700 1 --2- 172.21.15.55:0/2016897666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f7c0cb560 0x7f7f7c0cc430 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.122+0000 7f7f8199d700 1 --2- 172.21.15.55:0/2016897666 >> v2:172.21.15.163:3300/0 conn(0x7f7f7c0cc970 0x7f7f7c0697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.122+0000 7f7f8199d700 1 -- 172.21.15.55:0/2016897666 >> 172.21.15.55:0/2016897666 conn(0x7f7f7c06baf0 msgr2=0x7f7f7c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:31.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.122+0000 7f7f8199d700 1 -- 172.21.15.55:0/2016897666 shutdown_connections 2023-10-06T03:45:31.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.122+0000 7f7f8199d700 1 -- 172.21.15.55:0/2016897666 wait complete. 2023-10-06T03:45:31.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.123+0000 7f7f8199d700 1 Processor -- start 2023-10-06T03:45:31.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.123+0000 7f7f8199d700 1 -- start start 2023-10-06T03:45:31.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.124+0000 7f7f8199d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f7c069cf0 0x7f7f7c157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:31.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.124+0000 7f7f8199d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f7c0cb560 0x7f7f7c0b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:31.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.124+0000 7f7f8199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f7c0cc970 0x7f7f7c0ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:31.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.124+0000 7f7f8199d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c1580c0 con 0x7f7f7c0cb560 2023-10-06T03:45:31.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.124+0000 7f7f8199d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0b31f0 con 0x7f7f7c069cf0 2023-10-06T03:45:31.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.124+0000 7f7f8199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c1585b0 con 0x7f7f7c0cc970 2023-10-06T03:45:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.125+0000 7f7f7bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f7c0cb560 0x7f7f7c0b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.125+0000 7f7f7bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f7c0cb560 0x7f7f7c0b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54894/0 (socket says 172.21.15.55:54894) 2023-10-06T03:45:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.125+0000 7f7f7bfff700 1 -- 172.21.15.55:0/3859956636 learned_addr learned my addr 172.21.15.55:0/3859956636 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.125+0000 7f7f8099b700 1 --2- 172.21.15.55:0/3859956636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f7c069cf0 0x7f7f7c157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.125+0000 7f7f8119c700 1 --2- 172.21.15.55:0/3859956636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f7c0cc970 0x7f7f7c0ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.125+0000 7f7f7bfff700 1 -- 172.21.15.55:0/3859956636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f7c0cc970 msgr2=0x7f7f7c0ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.125+0000 7f7f7bfff700 1 --2- 172.21.15.55:0/3859956636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f7c0cc970 0x7f7f7c0ab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.125+0000 7f7f7bfff700 1 -- 172.21.15.55:0/3859956636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f7c069cf0 msgr2=0x7f7f7c157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.125+0000 7f7f7bfff700 1 --2- 172.21.15.55:0/3859956636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f7c069cf0 0x7f7f7c157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.125+0000 7f7f7bfff700 1 -- 172.21.15.55:0/3859956636 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f70009420 con 0x7f7f7c0cb560 2023-10-06T03:45:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.125+0000 7f7f7bfff700 1 --2- 172.21.15.55:0/3859956636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f7c0cb560 0x7f7f7c0b00e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800e930 tx=0x7f7f680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:31.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.126+0000 7f7f78ff9700 1 -- 172.21.15.55:0/3859956636 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68013680 con 0x7f7f7c0cb560 2023-10-06T03:45:31.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.126+0000 7f7f78ff9700 1 -- 172.21.15.55:0/3859956636 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f680137e0 con 0x7f7f7c0cb560 2023-10-06T03:45:31.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.126+0000 7f7f78ff9700 1 -- 172.21.15.55:0/3859956636 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68013b80 con 0x7f7f7c0cb560 2023-10-06T03:45:31.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.126+0000 7f7f8199d700 1 -- 172.21.15.55:0/3859956636 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c0a9c70 con 0x7f7f7c0cb560 2023-10-06T03:45:31.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.126+0000 7f7f8199d700 1 -- 172.21.15.55:0/3859956636 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c0aa2b0 con 0x7f7f7c0cb560 2023-10-06T03:45:31.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.127+0000 7f7f8199d700 1 -- 172.21.15.55:0/3859956636 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7c04c3c0 con 0x7f7f7c0cb560 2023-10-06T03:45:31.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.128+0000 7f7f78ff9700 1 -- 172.21.15.55:0/3859956636 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f7f6800c3e0 con 0x7f7f7c0cb560 2023-10-06T03:45:31.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.128+0000 7f7f78ff9700 1 --2- 172.21.15.55:0/3859956636 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7f60067130 0x7f7f600695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:31.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.128+0000 7f7f78ff9700 1 -- 172.21.15.55:0/3859956636 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f7f68084cb0 con 0x7f7f7c0cb560 2023-10-06T03:45:31.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.129+0000 7f7f8099b700 1 --2- 172.21.15.55:0/3859956636 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7f60067130 0x7f7f600695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:31.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.129+0000 7f7f8099b700 1 --2- 172.21.15.55:0/3859956636 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7f60067130 0x7f7f600695e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f700093f0 tx=0x7f7f70017000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:31.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.133+0000 7f7f78ff9700 1 -- 172.21.15.55:0/3859956636 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f68051980 con 0x7f7f7c0cb560 2023-10-06T03:45:31.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.359+0000 7f7f8199d700 1 -- 172.21.15.55:0/3859956636 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f7c06a7e0 con 0x7f7f60067130 2023-10-06T03:45:31.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.370+0000 7f7f78ff9700 1 -- 172.21.15.55:0/3859956636 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f7f7c06a7e0 con 0x7f7f60067130 2023-10-06T03:45:31.371 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:31.371 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:40", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:31.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.372+0000 7f7f8199d700 1 -- 172.21.15.55:0/3859956636 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7f60067130 msgr2=0x7f7f600695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:31.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.372+0000 7f7f8199d700 1 --2- 172.21.15.55:0/3859956636 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7f60067130 0x7f7f600695e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f700093f0 tx=0x7f7f70017000 comp rx=0 tx=0).stop 2023-10-06T03:45:31.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.372+0000 7f7f8199d700 1 -- 172.21.15.55:0/3859956636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f7c0cb560 msgr2=0x7f7f7c0b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:31.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.372+0000 7f7f8199d700 1 --2- 172.21.15.55:0/3859956636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f7c0cb560 0x7f7f7c0b00e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800e930 tx=0x7f7f680091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.372+0000 7f7f8199d700 1 -- 172.21.15.55:0/3859956636 shutdown_connections 2023-10-06T03:45:31.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.372+0000 7f7f8199d700 1 --2- 172.21.15.55:0/3859956636 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f7f60067130 0x7f7f600695e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.372+0000 7f7f8199d700 1 --2- 172.21.15.55:0/3859956636 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f7c0cc970 0x7f7f7c0ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.372+0000 7f7f8199d700 1 --2- 172.21.15.55:0/3859956636 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f7c069cf0 0x7f7f7c157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.372+0000 7f7f8199d700 1 --2- 172.21.15.55:0/3859956636 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f7c0cb560 0x7f7f7c0b00e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.372+0000 7f7f8199d700 1 -- 172.21.15.55:0/3859956636 >> 172.21.15.55:0/3859956636 conn(0x7f7f7c06baf0 msgr2=0x7f7f7c0c9380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:31.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.372+0000 7f7f8199d700 1 -- 172.21.15.55:0/3859956636 shutdown_connections 2023-10-06T03:45:31.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.372+0000 7f7f8199d700 1 -- 172.21.15.55:0/3859956636 wait complete. 2023-10-06T03:45:31.391 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:34:40", "first": "2023-10-06T03:35:00", "last": "2023-10-06T03:45:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-06T03:45:31.391 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T03:34:40', 'first': '2023-10-06T03:35:00', 'last': '2023-10-06T03:45:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-06T03:45:31.391 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T03:45:31.391 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T03:45:31.392 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir0 1M 2023-10-06T03:45:31.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.582+0000 7f8a6e79d700 1 Processor -- start 2023-10-06T03:45:31.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.582+0000 7f8a6e79d700 1 -- start start 2023-10-06T03:45:31.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.584+0000 7f8a6e79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a680cb560 0x7f8a680cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:31.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.584+0000 7f8a6e79d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8a680cc970 0x7f8a680697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:31.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.584+0000 7f8a6e79d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a68069cf0 0x7f8a6806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:31.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.584+0000 7f8a6e79d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6806aa30 con 0x7f8a68069cf0 2023-10-06T03:45:31.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.584+0000 7f8a6e79d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a680ccde0 con 0x7f8a680cc970 2023-10-06T03:45:31.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.584+0000 7f8a6e79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68159db0 con 0x7f8a680cb560 2023-10-06T03:45:31.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.584+0000 7f8a6df9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a68069cf0 0x7f8a6806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:31.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.584+0000 7f8a6df9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a68069cf0 0x7f8a6806d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54896/0 (socket says 172.21.15.55:54896) 2023-10-06T03:45:31.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.584+0000 7f8a6df9c700 1 -- 172.21.15.55:0/2410556058 learned_addr learned my addr 172.21.15.55:0/2410556058 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:31.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.584+0000 7f8a6d79b700 1 --2- 172.21.15.55:0/2410556058 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a680cb560 0x7f8a680cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:31.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.584+0000 7f8a6cf9a700 1 --2- 172.21.15.55:0/2410556058 >> v2:172.21.15.163:3300/0 conn(0x7f8a680cc970 0x7f8a680697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:31.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.585+0000 7f8a6df9c700 1 -- 172.21.15.55:0/2410556058 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a680cb560 msgr2=0x7f8a680cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:31.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.585+0000 7f8a6df9c700 1 --2- 172.21.15.55:0/2410556058 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a680cb560 0x7f8a680cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.585+0000 7f8a6df9c700 1 -- 172.21.15.55:0/2410556058 >> v2:172.21.15.163:3300/0 conn(0x7f8a680cc970 msgr2=0x7f8a680697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:31.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.585+0000 7f8a6df9c700 1 --2- 172.21.15.55:0/2410556058 >> v2:172.21.15.163:3300/0 conn(0x7f8a680cc970 0x7f8a680697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.585+0000 7f8a6df9c700 1 -- 172.21.15.55:0/2410556058 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6806a470 con 0x7f8a68069cf0 2023-10-06T03:45:31.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.585+0000 7f8a6df9c700 1 --2- 172.21.15.55:0/2410556058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a68069cf0 0x7f8a6806d0e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8a680cf6f0 tx=0x7f8a60009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28f45714df6a031 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:31.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.585+0000 7f8a5ffff700 1 -- 172.21.15.55:0/2410556058 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a6000e070 con 0x7f8a68069cf0 2023-10-06T03:45:31.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.585+0000 7f8a5ffff700 1 -- 172.21.15.55:0/2410556058 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a6000ba40 con 0x7f8a68069cf0 2023-10-06T03:45:31.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.585+0000 7f8a5ffff700 1 -- 172.21.15.55:0/2410556058 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a600092e0 con 0x7f8a68069cf0 2023-10-06T03:45:31.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.586+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2410556058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a68069cf0 msgr2=0x7f8a6806d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:31.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.586+0000 7f8a6e79d700 1 --2- 172.21.15.55:0/2410556058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a68069cf0 0x7f8a6806d0e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8a680cf6f0 tx=0x7f8a60009840 comp rx=0 tx=0).stop 2023-10-06T03:45:31.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.586+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2410556058 shutdown_connections 2023-10-06T03:45:31.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.586+0000 7f8a6e79d700 1 --2- 172.21.15.55:0/2410556058 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a68069cf0 0x7f8a6806d0e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.586+0000 7f8a6e79d700 1 --2- 172.21.15.55:0/2410556058 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a680cb560 0x7f8a680cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.586+0000 7f8a6e79d700 1 --2- 172.21.15.55:0/2410556058 >> v2:172.21.15.163:3300/0 conn(0x7f8a680cc970 0x7f8a680697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.586+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2410556058 >> 172.21.15.55:0/2410556058 conn(0x7f8a6806baf0 msgr2=0x7f8a6806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:31.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.586+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2410556058 shutdown_connections 2023-10-06T03:45:31.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.586+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2410556058 wait complete. 2023-10-06T03:45:31.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.587+0000 7f8a6e79d700 1 Processor -- start 2023-10-06T03:45:31.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.587+0000 7f8a6e79d700 1 -- start start 2023-10-06T03:45:31.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.588+0000 7f8a6e79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a68069cf0 0x7f8a680b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:31.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.588+0000 7f8a6e79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a680cb560 0x7f8a68169180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:31.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.588+0000 7f8a6e79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a680cc970 0x7f8a681696c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:31.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.588+0000 7f8a6e79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68157ad0 con 0x7f8a680cb560 2023-10-06T03:45:31.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.588+0000 7f8a6e79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a680aafe0 con 0x7f8a680cc970 2023-10-06T03:45:31.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.588+0000 7f8a6e79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a680b0790 con 0x7f8a68069cf0 2023-10-06T03:45:31.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.589+0000 7f8a6cf9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a680cb560 0x7f8a68169180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:31.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.589+0000 7f8a6cf9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a680cb560 0x7f8a68169180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54910/0 (socket says 172.21.15.55:54910) 2023-10-06T03:45:31.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.589+0000 7f8a6cf9a700 1 -- 172.21.15.55:0/2830868745 learned_addr learned my addr 172.21.15.55:0/2830868745 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.589+0000 7f8a6d79b700 1 --2- 172.21.15.55:0/2830868745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a68069cf0 0x7f8a680b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.589+0000 7f8a6df9c700 1 --2- 172.21.15.55:0/2830868745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a680cc970 0x7f8a681696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.589+0000 7f8a6cf9a700 1 -- 172.21.15.55:0/2830868745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a68069cf0 msgr2=0x7f8a680b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.589+0000 7f8a6cf9a700 1 --2- 172.21.15.55:0/2830868745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a68069cf0 0x7f8a680b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.589+0000 7f8a6cf9a700 1 -- 172.21.15.55:0/2830868745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a680cc970 msgr2=0x7f8a681696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.589+0000 7f8a6cf9a700 1 --2- 172.21.15.55:0/2830868745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a680cc970 0x7f8a681696c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.589+0000 7f8a6cf9a700 1 -- 172.21.15.55:0/2830868745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a60009ae0 con 0x7f8a680cb560 2023-10-06T03:45:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.589+0000 7f8a6cf9a700 1 --2- 172.21.15.55:0/2830868745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a680cb560 0x7f8a68169180 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800d8c0 tx=0x7f8a58008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.590+0000 7f8a5d7fa700 1 -- 172.21.15.55:0/2830868745 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a580125f0 con 0x7f8a680cb560 2023-10-06T03:45:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.590+0000 7f8a5d7fa700 1 -- 172.21.15.55:0/2830868745 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a58012750 con 0x7f8a680cb560 2023-10-06T03:45:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.590+0000 7f8a5d7fa700 1 -- 172.21.15.55:0/2830868745 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58012af0 con 0x7f8a680cb560 2023-10-06T03:45:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.590+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2830868745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a680aa410 con 0x7f8a680cb560 2023-10-06T03:45:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.590+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2830868745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a680aa6b0 con 0x7f8a680cb560 2023-10-06T03:45:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.591+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2830868745 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a680ab190 con 0x7f8a680cb560 2023-10-06T03:45:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.592+0000 7f8a5d7fa700 1 -- 172.21.15.55:0/2830868745 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f8a5801e460 con 0x7f8a680cb560 2023-10-06T03:45:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.592+0000 7f8a5d7fa700 1 --2- 172.21.15.55:0/2830868745 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a4c067130 0x7f8a4c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.593+0000 7f8a5d7fa700 1 -- 172.21.15.55:0/2830868745 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f8a580850f0 con 0x7f8a680cb560 2023-10-06T03:45:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.593+0000 7f8a6d79b700 1 --2- 172.21.15.55:0/2830868745 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a4c067130 0x7f8a4c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.594+0000 7f8a6d79b700 1 --2- 172.21.15.55:0/2830868745 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a4c067130 0x7f8a4c0695e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8a64006fd0 tx=0x7f8a64008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:31.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.596+0000 7f8a5d7fa700 1 -- 172.21.15.55:0/2830868745 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a680ab190 con 0x7f8a680cb560 2023-10-06T03:45:31.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.827+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2830868745 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a6806a7e0 con 0x7f8a4c067130 2023-10-06T03:45:31.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.911+0000 7f8a5d7fa700 1 -- 172.21.15.55:0/2830868745 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f8a6806a7e0 con 0x7f8a4c067130 2023-10-06T03:45:31.913 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-10-06T03:45:31.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.914+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2830868745 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a4c067130 msgr2=0x7f8a4c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:31.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.914+0000 7f8a6e79d700 1 --2- 172.21.15.55:0/2830868745 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a4c067130 0x7f8a4c0695e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8a64006fd0 tx=0x7f8a64008040 comp rx=0 tx=0).stop 2023-10-06T03:45:31.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.914+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2830868745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a680cb560 msgr2=0x7f8a68169180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:31.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.914+0000 7f8a6e79d700 1 --2- 172.21.15.55:0/2830868745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a680cb560 0x7f8a68169180 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800d8c0 tx=0x7f8a58008040 comp rx=0 tx=0).stop 2023-10-06T03:45:31.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.914+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2830868745 shutdown_connections 2023-10-06T03:45:31.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.914+0000 7f8a6e79d700 1 --2- 172.21.15.55:0/2830868745 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a680cc970 0x7f8a681696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.914+0000 7f8a6e79d700 1 --2- 172.21.15.55:0/2830868745 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8a4c067130 0x7f8a4c0695e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.914+0000 7f8a6e79d700 1 --2- 172.21.15.55:0/2830868745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a68069cf0 0x7f8a680b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.914+0000 7f8a6e79d700 1 --2- 172.21.15.55:0/2830868745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a680cb560 0x7f8a68169180 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:31.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.914+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2830868745 >> 172.21.15.55:0/2830868745 conn(0x7f8a6806baf0 msgr2=0x7f8a680ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:31.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.914+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2830868745 shutdown_connections 2023-10-06T03:45:31.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:31.914+0000 7f8a6e79d700 1 -- 172.21.15.55:0/2830868745 wait complete. 2023-10-06T03:45:31.932 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-10-06T03:45:31.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:31.933 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-06T03:45:31.971 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:31.971 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:31.971 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:31.972 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:31.972 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:31.972 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:31.972 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:31.972 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:31.972 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:31.972 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:31.972 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:31.973 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:31.973 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:31.973 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_35_00_UTC) 2023-10-06T03:45:32.059 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:32.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:32.060 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_36_00_UTC) 2023-10-06T03:45:32.103 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:32.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:32.104 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_37_00_UTC) 2023-10-06T03:45:32.189 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:32.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:32.190 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_38_00_UTC) 2023-10-06T03:45:32.275 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:32.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:32.275 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_39_00_UTC) 2023-10-06T03:45:32.361 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:32.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:32.361 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_40_00_UTC) 2023-10-06T03:45:32.447 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:32.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:32.448 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_41_00_UTC) 2023-10-06T03:45:32.534 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:32.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:32.534 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_42_00_UTC) 2023-10-06T03:45:32.580 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:32.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:32.580 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_43_00_UTC) 2023-10-06T03:45:32.665 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:32.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:32.666 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_44_00_UTC) 2023-10-06T03:45:32.752 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:32.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:32.752 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-06-03_45_00_UTC) 2023-10-06T03:45:32.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:32.838 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-10-06T03:45:32.920 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir1 1M 2023-10-06T03:45:33.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.150+0000 7f3b5359e700 1 Processor -- start 2023-10-06T03:45:33.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.150+0000 7f3b5359e700 1 -- start start 2023-10-06T03:45:33.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b5359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b54153480 0x7f3b540ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:33.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b5359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3b54159dc0 0x7f3b54157280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:33.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b5359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b540aec90 con 0x7f3b54153850 2023-10-06T03:45:33.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b5359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5415a480 con 0x7f3b54159dc0 2023-10-06T03:45:33.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b5359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b540afb40 con 0x7f3b54153480 2023-10-06T03:45:33.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b52d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3b54159dc0 0x7f3b54157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:33.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b52d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3b54159dc0 0x7f3b54157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52154/0 (socket says 172.21.15.55:52154) 2023-10-06T03:45:33.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b52d9d700 1 -- 172.21.15.55:0/2173744309 learned_addr learned my addr 172.21.15.55:0/2173744309 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:33.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b51d9b700 1 --1- 172.21.15.55:0/2173744309 >> v1:172.21.15.55:6789/0 conn(0x7f3b54153850 0x7f3b541596b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39834/0 (socket says 172.21.15.55:39834) 2023-10-06T03:45:33.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b5259c700 1 --2- 172.21.15.55:0/2173744309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b54153480 0x7f3b540ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:33.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b52d9d700 1 -- 172.21.15.55:0/2173744309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b54153480 msgr2=0x7f3b540ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:33.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b52d9d700 1 --2- 172.21.15.55:0/2173744309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b54153480 0x7f3b540ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:33.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b52d9d700 1 -- 172.21.15.55:0/2173744309 >> v1:172.21.15.55:6789/0 conn(0x7f3b54153850 legacy=0x7f3b541596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:33.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b52d9d700 1 -- 172.21.15.55:0/2173744309 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b540afe20 con 0x7f3b54159dc0 2023-10-06T03:45:33.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.152+0000 7f3b5159a700 1 -- 172.21.15.55:0/2173744309 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920188705 0 0) 0x7f3b540aec90 con 0x7f3b54153850 2023-10-06T03:45:33.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.153+0000 7f3b52d9d700 1 --2- 172.21.15.55:0/2173744309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b54159dc0 0x7f3b54157280 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009f20 tx=0x7f3b4c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f419ab63322341c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:33.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.153+0000 7f3b5159a700 1 -- 172.21.15.55:0/2173744309 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c00e070 con 0x7f3b54159dc0 2023-10-06T03:45:33.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.153+0000 7f3b5159a700 1 -- 172.21.15.55:0/2173744309 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c00ba80 con 0x7f3b54159dc0 2023-10-06T03:45:33.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.153+0000 7f3b5159a700 1 -- 172.21.15.55:0/2173744309 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c0092e0 con 0x7f3b54159dc0 2023-10-06T03:45:33.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.153+0000 7f3b5359e700 1 -- 172.21.15.55:0/2173744309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b54159dc0 msgr2=0x7f3b54157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:33.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.153+0000 7f3b5359e700 1 --2- 172.21.15.55:0/2173744309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b54159dc0 0x7f3b54157280 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009f20 tx=0x7f3b4c009840 comp rx=0 tx=0).stop 2023-10-06T03:45:33.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.154+0000 7f3b5359e700 1 -- 172.21.15.55:0/2173744309 shutdown_connections 2023-10-06T03:45:33.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.154+0000 7f3b5359e700 1 --2- 172.21.15.55:0/2173744309 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b54159dc0 0x7f3b54157280 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:33.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.154+0000 7f3b5359e700 1 --2- 172.21.15.55:0/2173744309 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b54153480 0x7f3b540ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:33.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.154+0000 7f3b5359e700 1 -- 172.21.15.55:0/2173744309 >> 172.21.15.55:0/2173744309 conn(0x7f3b54068e20 msgr2=0x7f3b540cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:33.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.154+0000 7f3b5359e700 1 -- 172.21.15.55:0/2173744309 shutdown_connections 2023-10-06T03:45:33.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.154+0000 7f3b5359e700 1 -- 172.21.15.55:0/2173744309 wait complete. 2023-10-06T03:45:33.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.155+0000 7f3b5359e700 1 Processor -- start 2023-10-06T03:45:33.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.155+0000 7f3b5359e700 1 -- start start 2023-10-06T03:45:33.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.156+0000 7f3b5359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b54153480 0x7f3b5414bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:33.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.156+0000 7f3b5359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b54153850 0x7f3b540a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:33.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.156+0000 7f3b5359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b54159dc0 0x7f3b540acb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:33.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.156+0000 7f3b5359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b540a8b00 con 0x7f3b54159dc0 2023-10-06T03:45:33.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.156+0000 7f3b5359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b540ac670 con 0x7f3b54153480 2023-10-06T03:45:33.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.156+0000 7f3b5359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b540aba60 con 0x7f3b54153850 2023-10-06T03:45:33.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.156+0000 7f3b52d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b54159dc0 0x7f3b540acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:33.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.157+0000 7f3b52d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b54159dc0 0x7f3b540acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52166/0 (socket says 172.21.15.55:52166) 2023-10-06T03:45:33.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.157+0000 7f3b52d9d700 1 -- 172.21.15.55:0/1368988698 learned_addr learned my addr 172.21.15.55:0/1368988698 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:33.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.157+0000 7f3b51d9b700 1 --2- 172.21.15.55:0/1368988698 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b54153850 0x7f3b540a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:33.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.157+0000 7f3b5259c700 1 --2- 172.21.15.55:0/1368988698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b54153480 0x7f3b5414bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:33.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.157+0000 7f3b52d9d700 1 -- 172.21.15.55:0/1368988698 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b54153850 msgr2=0x7f3b540a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:33.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.157+0000 7f3b52d9d700 1 --2- 172.21.15.55:0/1368988698 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b54153850 0x7f3b540a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:33.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.157+0000 7f3b52d9d700 1 -- 172.21.15.55:0/1368988698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b54153480 msgr2=0x7f3b5414bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:33.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.157+0000 7f3b52d9d700 1 --2- 172.21.15.55:0/1368988698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b54153480 0x7f3b5414bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:33.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.157+0000 7f3b52d9d700 1 -- 172.21.15.55:0/1368988698 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c009ae0 con 0x7f3b54159dc0 2023-10-06T03:45:33.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.157+0000 7f3b52d9d700 1 --2- 172.21.15.55:0/1368988698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b54159dc0 0x7f3b540acb00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c012000 tx=0x7f3b4c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:33.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.157+0000 7f3b427fc700 1 -- 172.21.15.55:0/1368988698 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c0032f0 con 0x7f3b54159dc0 2023-10-06T03:45:33.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.157+0000 7f3b427fc700 1 -- 172.21.15.55:0/1368988698 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c025460 con 0x7f3b54159dc0 2023-10-06T03:45:33.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.157+0000 7f3b5359e700 1 -- 172.21.15.55:0/1368988698 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b540abd40 con 0x7f3b54159dc0 2023-10-06T03:45:33.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.157+0000 7f3b427fc700 1 -- 172.21.15.55:0/1368988698 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c025820 con 0x7f3b54159dc0 2023-10-06T03:45:33.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.158+0000 7f3b5359e700 1 -- 172.21.15.55:0/1368988698 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b540ac300 con 0x7f3b54159dc0 2023-10-06T03:45:33.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.159+0000 7f3b5359e700 1 -- 172.21.15.55:0/1368988698 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b540ac800 con 0x7f3b54159dc0 2023-10-06T03:45:33.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.159+0000 7f3b427fc700 1 -- 172.21.15.55:0/1368988698 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f3b4c003040 con 0x7f3b54159dc0 2023-10-06T03:45:33.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.160+0000 7f3b427fc700 1 --2- 172.21.15.55:0/1368988698 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3b340671b0 0x7f3b34069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:33.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.161+0000 7f3b427fc700 1 -- 172.21.15.55:0/1368988698 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7f3b4c0869e0 con 0x7f3b54159dc0 2023-10-06T03:45:33.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.161+0000 7f3b5259c700 1 --2- 172.21.15.55:0/1368988698 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3b340671b0 0x7f3b34069660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:33.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.162+0000 7f3b5259c700 1 --2- 172.21.15.55:0/1368988698 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3b340671b0 0x7f3b34069660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3b44006fd0 tx=0x7f3b44008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:33.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.165+0000 7f3b427fc700 1 -- 172.21.15.55:0/1368988698 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b4c0535a0 con 0x7f3b54159dc0 2023-10-06T03:45:33.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.392+0000 7f3b5359e700 1 -- 172.21.15.55:0/1368988698 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b5404c3c0 con 0x7f3b340671b0 2023-10-06T03:45:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.481+0000 7f3b427fc700 1 -- 172.21.15.55:0/1368988698 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f3b5404c3c0 con 0x7f3b340671b0 2023-10-06T03:45:33.483 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-10-06T03:45:33.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.483+0000 7f3b5359e700 1 -- 172.21.15.55:0/1368988698 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3b340671b0 msgr2=0x7f3b34069660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:33.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.483+0000 7f3b5359e700 1 --2- 172.21.15.55:0/1368988698 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3b340671b0 0x7f3b34069660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3b44006fd0 tx=0x7f3b44008040 comp rx=0 tx=0).stop 2023-10-06T03:45:33.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.483+0000 7f3b5359e700 1 -- 172.21.15.55:0/1368988698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b54159dc0 msgr2=0x7f3b540acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:33.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.483+0000 7f3b5359e700 1 --2- 172.21.15.55:0/1368988698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b54159dc0 0x7f3b540acb00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c012000 tx=0x7f3b4c018000 comp rx=0 tx=0).stop 2023-10-06T03:45:33.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.483+0000 7f3b5359e700 1 -- 172.21.15.55:0/1368988698 shutdown_connections 2023-10-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.483+0000 7f3b5359e700 1 --2- 172.21.15.55:0/1368988698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b54159dc0 0x7f3b540acb00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.483+0000 7f3b5359e700 1 --2- 172.21.15.55:0/1368988698 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b54153480 0x7f3b5414bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.483+0000 7f3b5359e700 1 --2- 172.21.15.55:0/1368988698 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f3b340671b0 0x7f3b34069660 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.483+0000 7f3b5359e700 1 --2- 172.21.15.55:0/1368988698 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b54153850 0x7f3b540a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.483+0000 7f3b5359e700 1 -- 172.21.15.55:0/1368988698 >> 172.21.15.55:0/1368988698 conn(0x7f3b54068e20 msgr2=0x7f3b5406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.483+0000 7f3b5359e700 1 -- 172.21.15.55:0/1368988698 shutdown_connections 2023-10-06T03:45:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:33.483+0000 7f3b5359e700 1 -- 172.21.15.55:0/1368988698 wait complete. 2023-10-06T03:45:33.502 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-10-06T03:45:33.502 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:33.503 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-06T03:45:33.541 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:33.541 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:33.541 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:33.542 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:33.542 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:33.542 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:33.542 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:33.542 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:33.542 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:33.542 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:33.542 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:33.543 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:33.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:33.543 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_35_00_UTC) 2023-10-06T03:45:33.630 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:33.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:33.631 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_36_00_UTC) 2023-10-06T03:45:33.675 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:33.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:33.675 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_37_00_UTC) 2023-10-06T03:45:33.760 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:33.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:33.760 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_38_00_UTC) 2023-10-06T03:45:33.846 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:33.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:33.846 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_39_00_UTC) 2023-10-06T03:45:33.931 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:33.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:33.932 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_40_00_UTC) 2023-10-06T03:45:34.017 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:34.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:34.017 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_41_00_UTC) 2023-10-06T03:45:34.102 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:34.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:34.103 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_42_00_UTC) 2023-10-06T03:45:34.187 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:34.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:34.188 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_43_00_UTC) 2023-10-06T03:45:34.272 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:34.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:34.273 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_44_00_UTC) 2023-10-06T03:45:34.356 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:34.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:34.357 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-06-03_45_00_UTC) 2023-10-06T03:45:34.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:34.441 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-10-06T03:45:34.522 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir2 1M 2023-10-06T03:45:34.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.754+0000 7fa40759d700 1 Processor -- start 2023-10-06T03:45:34.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.754+0000 7fa40759d700 1 -- start start 2023-10-06T03:45:34.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.755+0000 7fa40759d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4000b03c0 0x7fa4000cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:34.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.755+0000 7fa40759d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa4001591e0 0x7fa400159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:34.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.755+0000 7fa40759d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4000cfa00 con 0x7fa4000b0790 2023-10-06T03:45:34.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.755+0000 7fa40759d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4000cb580 con 0x7fa4000b03c0 2023-10-06T03:45:34.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.755+0000 7fa40759d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4000cf160 con 0x7fa4001591e0 2023-10-06T03:45:34.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.756+0000 7fa40659b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4000b03c0 0x7fa4000cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:34.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.756+0000 7fa40659b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4000b03c0 0x7fa4000cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52172/0 (socket says 172.21.15.55:52172) 2023-10-06T03:45:34.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.756+0000 7fa40659b700 1 -- 172.21.15.55:0/2732743262 learned_addr learned my addr 172.21.15.55:0/2732743262 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:34.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.756+0000 7fa405d9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa4000b0790 0x7fa4000b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39840/0 (socket says 172.21.15.55:39840) 2023-10-06T03:45:34.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.756+0000 7fa40659b700 1 -- 172.21.15.55:0/2732743262 >> v2:172.21.15.163:3300/0 conn(0x7fa4001591e0 msgr2=0x7fa400159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:34.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.756+0000 7fa406d9c700 1 --2- 172.21.15.55:0/2732743262 >> v2:172.21.15.163:3300/0 conn(0x7fa4001591e0 0x7fa400159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:34.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.756+0000 7fa405599700 1 -- 172.21.15.55:0/2732743262 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804430583 0 0) 0x7fa4000cfa00 con 0x7fa4000b0790 2023-10-06T03:45:34.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.756+0000 7fa40659b700 1 --2- 172.21.15.55:0/2732743262 >> v2:172.21.15.163:3300/0 conn(0x7fa4001591e0 0x7fa400159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:34.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.756+0000 7fa40659b700 1 -- 172.21.15.55:0/2732743262 >> v1:172.21.15.55:6789/0 conn(0x7fa4000b0790 legacy=0x7fa4000b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:34.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.756+0000 7fa40659b700 1 -- 172.21.15.55:0/2732743262 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4000cf440 con 0x7fa4000b03c0 2023-10-06T03:45:34.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.756+0000 7fa406d9c700 1 --2- 172.21.15.55:0/2732743262 >> v2:172.21.15.163:3300/0 conn(0x7fa4001591e0 0x7fa400159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:45:34.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.757+0000 7fa40659b700 1 --2- 172.21.15.55:0/2732743262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4000b03c0 0x7fa4000cea70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0099b0 tx=0x7fa3fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c041bf3f37eb164 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:34.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.757+0000 7fa405599700 1 -- 172.21.15.55:0/2732743262 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc00b820 con 0x7fa4000b03c0 2023-10-06T03:45:34.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.757+0000 7fa405599700 1 -- 172.21.15.55:0/2732743262 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc00b980 con 0x7fa4000b03c0 2023-10-06T03:45:34.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.757+0000 7fa405599700 1 -- 172.21.15.55:0/2732743262 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc00bd40 con 0x7fa4000b03c0 2023-10-06T03:45:34.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.757+0000 7fa40759d700 1 -- 172.21.15.55:0/2732743262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4000b03c0 msgr2=0x7fa4000cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:34.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.757+0000 7fa40759d700 1 --2- 172.21.15.55:0/2732743262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4000b03c0 0x7fa4000cea70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0099b0 tx=0x7fa3fc009250 comp rx=0 tx=0).stop 2023-10-06T03:45:34.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.757+0000 7fa40759d700 1 -- 172.21.15.55:0/2732743262 shutdown_connections 2023-10-06T03:45:34.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.757+0000 7fa40759d700 1 --2- 172.21.15.55:0/2732743262 >> v2:172.21.15.163:3300/0 conn(0x7fa4001591e0 0x7fa400159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:34.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.757+0000 7fa40759d700 1 --2- 172.21.15.55:0/2732743262 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4000b03c0 0x7fa4000cea70 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:34.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.757+0000 7fa40759d700 1 -- 172.21.15.55:0/2732743262 >> 172.21.15.55:0/2732743262 conn(0x7fa400063230 msgr2=0x7fa400063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:34.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.757+0000 7fa40759d700 1 -- 172.21.15.55:0/2732743262 shutdown_connections 2023-10-06T03:45:34.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.758+0000 7fa40759d700 1 -- 172.21.15.55:0/2732743262 wait complete. 2023-10-06T03:45:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.759+0000 7fa40759d700 1 Processor -- start 2023-10-06T03:45:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.759+0000 7fa40759d700 1 -- start start 2023-10-06T03:45:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.760+0000 7fa40759d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4000b03c0 0x7fa4000c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.760+0000 7fa40759d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4000b0790 0x7fa4000c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.760+0000 7fa40759d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4001591e0 0x7fa4001696f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:34.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.760+0000 7fa40759d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000c5940 con 0x7fa4000b03c0 2023-10-06T03:45:34.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.760+0000 7fa40759d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000ca0d0 con 0x7fa4000b0790 2023-10-06T03:45:34.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.760+0000 7fa40759d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000c48a0 con 0x7fa4001591e0 2023-10-06T03:45:34.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.760+0000 7fa40659b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4000b03c0 0x7fa4000c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:34.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.760+0000 7fa40659b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4000b03c0 0x7fa4000c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52182/0 (socket says 172.21.15.55:52182) 2023-10-06T03:45:34.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.760+0000 7fa40659b700 1 -- 172.21.15.55:0/1391907761 learned_addr learned my addr 172.21.15.55:0/1391907761 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:34.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.760+0000 7fa405d9a700 1 --2- 172.21.15.55:0/1391907761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4000b0790 0x7fa4000c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:34.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.760+0000 7fa406d9c700 1 --2- 172.21.15.55:0/1391907761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4001591e0 0x7fa4001696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:34.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.761+0000 7fa40659b700 1 -- 172.21.15.55:0/1391907761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4001591e0 msgr2=0x7fa4001696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:34.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.761+0000 7fa40659b700 1 --2- 172.21.15.55:0/1391907761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4001591e0 0x7fa4001696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:34.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.761+0000 7fa40659b700 1 -- 172.21.15.55:0/1391907761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4000b0790 msgr2=0x7fa4000c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:34.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.761+0000 7fa40659b700 1 --2- 172.21.15.55:0/1391907761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4000b0790 0x7fa4000c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:34.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.761+0000 7fa40659b700 1 -- 172.21.15.55:0/1391907761 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc0094f0 con 0x7fa4000b03c0 2023-10-06T03:45:34.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.761+0000 7fa40659b700 1 --2- 172.21.15.55:0/1391907761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4000b03c0 0x7fa4000c5f30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa4000cfe40 tx=0x7fa3fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:34.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.761+0000 7fa3f67fc700 1 -- 172.21.15.55:0/1391907761 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc01f070 con 0x7fa4000b03c0 2023-10-06T03:45:34.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.761+0000 7fa3f67fc700 1 -- 172.21.15.55:0/1391907761 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc00baf0 con 0x7fa4000b03c0 2023-10-06T03:45:34.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.761+0000 7fa40759d700 1 -- 172.21.15.55:0/1391907761 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4000c4b80 con 0x7fa4000b03c0 2023-10-06T03:45:34.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.761+0000 7fa3f67fc700 1 -- 172.21.15.55:0/1391907761 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc025420 con 0x7fa4000b03c0 2023-10-06T03:45:34.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.761+0000 7fa40759d700 1 -- 172.21.15.55:0/1391907761 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4000c5140 con 0x7fa4000b03c0 2023-10-06T03:45:34.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.762+0000 7fa40759d700 1 -- 172.21.15.55:0/1391907761 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40004c3c0 con 0x7fa4000b03c0 2023-10-06T03:45:34.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.763+0000 7fa3f67fc700 1 -- 172.21.15.55:0/1391907761 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fa3fc023070 con 0x7fa4000b03c0 2023-10-06T03:45:34.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.764+0000 7fa3f67fc700 1 --2- 172.21.15.55:0/1391907761 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa3e4067130 0x7fa3e40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:34.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.764+0000 7fa3f67fc700 1 -- 172.21.15.55:0/1391907761 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6345+0+0 (secure 0 0 0) 0x7fa3fc086050 con 0x7fa4000b03c0 2023-10-06T03:45:34.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.764+0000 7fa405d9a700 1 --2- 172.21.15.55:0/1391907761 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa3e4067130 0x7fa3e40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:34.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.765+0000 7fa405d9a700 1 --2- 172.21.15.55:0/1391907761 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa3e4067130 0x7fa3e40695e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa4000b51f0 tx=0x7fa3f0007bf0 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:34.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:34.768+0000 7fa3f67fc700 1 -- 172.21.15.55:0/1391907761 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa3fc0b75a0 con 0x7fa4000b03c0 2023-10-06T03:45:35.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:35.047+0000 7fa40759d700 1 -- 172.21.15.55:0/1391907761 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4000cad50 con 0x7fa3e4067130 2023-10-06T03:45:35.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:35.168+0000 7fa3f67fc700 1 -- 172.21.15.55:0/1391907761 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fa4000cad50 con 0x7fa3e4067130 2023-10-06T03:45:35.169 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-10-06T03:45:35.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:35.170+0000 7fa40759d700 1 -- 172.21.15.55:0/1391907761 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa3e4067130 msgr2=0x7fa3e40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:35.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:35.170+0000 7fa40759d700 1 --2- 172.21.15.55:0/1391907761 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa3e4067130 0x7fa3e40695e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa4000b51f0 tx=0x7fa3f0007bf0 comp rx=0 tx=0).stop 2023-10-06T03:45:35.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:35.170+0000 7fa40759d700 1 -- 172.21.15.55:0/1391907761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4000b03c0 msgr2=0x7fa4000c5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:35.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:35.170+0000 7fa40759d700 1 --2- 172.21.15.55:0/1391907761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4000b03c0 0x7fa4000c5f30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa4000cfe40 tx=0x7fa3fc017000 comp rx=0 tx=0).stop 2023-10-06T03:45:35.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:35.170+0000 7fa40759d700 1 -- 172.21.15.55:0/1391907761 shutdown_connections 2023-10-06T03:45:35.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:35.170+0000 7fa40759d700 1 --2- 172.21.15.55:0/1391907761 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa3e4067130 0x7fa3e40695e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:35.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:35.170+0000 7fa40759d700 1 --2- 172.21.15.55:0/1391907761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4001591e0 0x7fa4001696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:35.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:35.170+0000 7fa40759d700 1 --2- 172.21.15.55:0/1391907761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4000b03c0 0x7fa4000c5f30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:35.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:35.170+0000 7fa40759d700 1 --2- 172.21.15.55:0/1391907761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4000b0790 0x7fa4000c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:35.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:35.170+0000 7fa40759d700 1 -- 172.21.15.55:0/1391907761 >> 172.21.15.55:0/1391907761 conn(0x7fa400063230 msgr2=0x7fa400069600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:35.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:35.170+0000 7fa40759d700 1 -- 172.21.15.55:0/1391907761 shutdown_connections 2023-10-06T03:45:35.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:35.170+0000 7fa40759d700 1 -- 172.21.15.55:0/1391907761 wait complete. 2023-10-06T03:45:35.189 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-10-06T03:45:35.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:35.189 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-06T03:45:35.228 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:35.228 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_01_UTC 2023-10-06T03:45:35.228 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:35.228 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:35.228 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:35.229 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:35.229 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:35.229 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:35.229 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:35.229 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:35.229 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:35.230 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:35.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:35.230 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_35_00_UTC) 2023-10-06T03:45:35.319 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_36_01_UTC 2023-10-06T03:45:35.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:35.319 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_36_01_UTC) 2023-10-06T03:45:35.409 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:35.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:35.409 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_37_00_UTC) 2023-10-06T03:45:35.484 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:35.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:35.484 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_38_00_UTC) 2023-10-06T03:45:35.527 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:35.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:35.528 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_39_00_UTC) 2023-10-06T03:45:35.612 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:35.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:35.612 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_40_00_UTC) 2023-10-06T03:45:35.696 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:35.696 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:35.697 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_41_00_UTC) 2023-10-06T03:45:35.780 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:35.780 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:35.780 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_42_00_UTC) 2023-10-06T03:45:35.869 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:35.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:35.869 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_43_00_UTC) 2023-10-06T03:45:35.914 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:35.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:35.914 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_44_00_UTC) 2023-10-06T03:45:35.997 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:35.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:35.998 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-06-03_45_00_UTC) 2023-10-06T03:45:36.081 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:36.081 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-10-06T03:45:36.164 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir3 1M 2023-10-06T03:45:36.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.395+0000 7fc143fff700 1 Processor -- start 2023-10-06T03:45:36.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.395+0000 7fc143fff700 1 -- start start 2023-10-06T03:45:36.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.396+0000 7fc143fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc1440b0760 0x7fc1440b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:36.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.396+0000 7fc143fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc1440ace60 0x7fc1440ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:36.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.397+0000 7fc143fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1440b0570 con 0x7fc144156350 2023-10-06T03:45:36.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.397+0000 7fc143fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1440b1140 con 0x7fc1440ace60 2023-10-06T03:45:36.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.397+0000 7fc143fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1440af4d0 con 0x7fc1440b0760 2023-10-06T03:45:36.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.397+0000 7fc142ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc144156350 0x7fc144157370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39856/0 (socket says 172.21.15.55:39856) 2023-10-06T03:45:36.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.397+0000 7fc142ffd700 1 -- 172.21.15.55:0/3358975371 learned_addr learned my addr 172.21.15.55:0/3358975371 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:36.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.397+0000 7fc1437fe700 1 --2- 172.21.15.55:0/3358975371 >> v2:172.21.15.55:3300/0 conn(0x7fc1440ace60 0x7fc1440ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:36.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.397+0000 7fc141ffb700 1 -- 172.21.15.55:0/3358975371 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228223267 0 0) 0x7fc1440b0570 con 0x7fc144156350 2023-10-06T03:45:36.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.397+0000 7fc1437fe700 1 -- 172.21.15.55:0/3358975371 >> v2:172.21.15.163:3300/0 conn(0x7fc1440b0760 msgr2=0x7fc1440b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:36.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.397+0000 7fc1437fe700 1 --2- 172.21.15.55:0/3358975371 >> v2:172.21.15.163:3300/0 conn(0x7fc1440b0760 0x7fc1440b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:36.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.397+0000 7fc1437fe700 1 -- 172.21.15.55:0/3358975371 >> v1:172.21.15.55:6789/0 conn(0x7fc144156350 legacy=0x7fc144157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:36.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.397+0000 7fc1437fe700 1 -- 172.21.15.55:0/3358975371 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1440af7b0 con 0x7fc1440ace60 2023-10-06T03:45:36.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.398+0000 7fc1437fe700 1 --2- 172.21.15.55:0/3358975371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1440ace60 0x7fc1440ad260 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc1440642a0 tx=0x7fc13c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f584d29c702ec4b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:36.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.398+0000 7fc141ffb700 1 -- 172.21.15.55:0/3358975371 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c00e070 con 0x7fc1440ace60 2023-10-06T03:45:36.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.398+0000 7fc141ffb700 1 -- 172.21.15.55:0/3358975371 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13c00bc30 con 0x7fc1440ace60 2023-10-06T03:45:36.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.398+0000 7fc141ffb700 1 -- 172.21.15.55:0/3358975371 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13c012430 con 0x7fc1440ace60 2023-10-06T03:45:36.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.398+0000 7fc143fff700 1 -- 172.21.15.55:0/3358975371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1440ace60 msgr2=0x7fc1440ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:36.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.398+0000 7fc143fff700 1 --2- 172.21.15.55:0/3358975371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1440ace60 0x7fc1440ad260 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc1440642a0 tx=0x7fc13c009780 comp rx=0 tx=0).stop 2023-10-06T03:45:36.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.398+0000 7fc143fff700 1 -- 172.21.15.55:0/3358975371 shutdown_connections 2023-10-06T03:45:36.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.398+0000 7fc143fff700 1 --2- 172.21.15.55:0/3358975371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1440ace60 0x7fc1440ad260 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:36.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.398+0000 7fc143fff700 1 --2- 172.21.15.55:0/3358975371 >> v2:172.21.15.163:3300/0 conn(0x7fc1440b0760 0x7fc1440b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:36.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.398+0000 7fc143fff700 1 -- 172.21.15.55:0/3358975371 >> 172.21.15.55:0/3358975371 conn(0x7fc144068e20 msgr2=0x7fc1440cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:36.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.398+0000 7fc143fff700 1 -- 172.21.15.55:0/3358975371 shutdown_connections 2023-10-06T03:45:36.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.398+0000 7fc143fff700 1 -- 172.21.15.55:0/3358975371 wait complete. 2023-10-06T03:45:36.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.400+0000 7fc143fff700 1 Processor -- start 2023-10-06T03:45:36.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.400+0000 7fc143fff700 1 -- start start 2023-10-06T03:45:36.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.401+0000 7fc143fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1440ace60 0x7fc14414fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:36.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.401+0000 7fc143fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1440b0760 0x7fc14406dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:36.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.401+0000 7fc143fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc144156350 0x7fc14406e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:36.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.401+0000 7fc143fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1440a5df0 con 0x7fc1440b0760 2023-10-06T03:45:36.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.401+0000 7fc143fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14406f2b0 con 0x7fc144156350 2023-10-06T03:45:36.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.401+0000 7fc143fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14406e6a0 con 0x7fc1440ace60 2023-10-06T03:45:36.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.401+0000 7fc1427fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1440b0760 0x7fc14406dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:36.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.401+0000 7fc1427fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1440b0760 0x7fc14406dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52196/0 (socket says 172.21.15.55:52196) 2023-10-06T03:45:36.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.401+0000 7fc1427fc700 1 -- 172.21.15.55:0/248155557 learned_addr learned my addr 172.21.15.55:0/248155557 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:36.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.401+0000 7fc142ffd700 1 --2- 172.21.15.55:0/248155557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1440ace60 0x7fc14414fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:36.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.401+0000 7fc1437fe700 1 --2- 172.21.15.55:0/248155557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc144156350 0x7fc14406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:36.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.402+0000 7fc1427fc700 1 -- 172.21.15.55:0/248155557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1440ace60 msgr2=0x7fc14414fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:36.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.402+0000 7fc1427fc700 1 --2- 172.21.15.55:0/248155557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1440ace60 0x7fc14414fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:36.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.402+0000 7fc1427fc700 1 -- 172.21.15.55:0/248155557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc144156350 msgr2=0x7fc14406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:36.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.402+0000 7fc1427fc700 1 --2- 172.21.15.55:0/248155557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc144156350 0x7fc14406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:36.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.402+0000 7fc1427fc700 1 -- 172.21.15.55:0/248155557 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c009a20 con 0x7fc1440b0760 2023-10-06T03:45:36.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.402+0000 7fc1427fc700 1 --2- 172.21.15.55:0/248155557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1440b0760 0x7fc14406dac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc13800b810 tx=0x7fc1380091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:36.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.402+0000 7fc12affd700 1 -- 172.21.15.55:0/248155557 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13800f510 con 0x7fc1440b0760 2023-10-06T03:45:36.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.402+0000 7fc12affd700 1 -- 172.21.15.55:0/248155557 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13800f670 con 0x7fc1440b0760 2023-10-06T03:45:36.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.402+0000 7fc12affd700 1 -- 172.21.15.55:0/248155557 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13800fa30 con 0x7fc1440b0760 2023-10-06T03:45:36.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.402+0000 7fc143fff700 1 -- 172.21.15.55:0/248155557 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14406e980 con 0x7fc1440b0760 2023-10-06T03:45:36.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.402+0000 7fc143fff700 1 -- 172.21.15.55:0/248155557 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14406ef40 con 0x7fc1440b0760 2023-10-06T03:45:36.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.403+0000 7fc143fff700 1 -- 172.21.15.55:0/248155557 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14406f420 con 0x7fc1440b0760 2023-10-06T03:45:36.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.404+0000 7fc12affd700 1 -- 172.21.15.55:0/248155557 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fc13801a400 con 0x7fc1440b0760 2023-10-06T03:45:36.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.405+0000 7fc12affd700 1 --2- 172.21.15.55:0/248155557 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc130067130 0x7fc1300695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:36.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.406+0000 7fc12affd700 1 -- 172.21.15.55:0/248155557 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6753+0+0 (secure 0 0 0) 0x7fc1380847e0 con 0x7fc1440b0760 2023-10-06T03:45:36.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.406+0000 7fc142ffd700 1 --2- 172.21.15.55:0/248155557 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc130067130 0x7fc1300695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:36.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.406+0000 7fc142ffd700 1 --2- 172.21.15.55:0/248155557 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc130067130 0x7fc1300695e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc134007160 tx=0x7fc134007f50 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:36.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.409+0000 7fc12affd700 1 -- 172.21.15.55:0/248155557 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc138051310 con 0x7fc1440b0760 2023-10-06T03:45:36.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.638+0000 7fc143fff700 1 -- 172.21.15.55:0/248155557 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc14404c3c0 con 0x7fc130067130 2023-10-06T03:45:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.714+0000 7fc12affd700 1 -- 172.21.15.55:0/248155557 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fc14404c3c0 con 0x7fc130067130 2023-10-06T03:45:36.716 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-10-06T03:45:36.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.717+0000 7fc143fff700 1 -- 172.21.15.55:0/248155557 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc130067130 msgr2=0x7fc1300695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:36.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.717+0000 7fc143fff700 1 --2- 172.21.15.55:0/248155557 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc130067130 0x7fc1300695e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc134007160 tx=0x7fc134007f50 comp rx=0 tx=0).stop 2023-10-06T03:45:36.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.717+0000 7fc143fff700 1 -- 172.21.15.55:0/248155557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1440b0760 msgr2=0x7fc14406dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:36.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.717+0000 7fc143fff700 1 --2- 172.21.15.55:0/248155557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1440b0760 0x7fc14406dac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc13800b810 tx=0x7fc1380091d0 comp rx=0 tx=0).stop 2023-10-06T03:45:36.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.717+0000 7fc143fff700 1 -- 172.21.15.55:0/248155557 shutdown_connections 2023-10-06T03:45:36.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.717+0000 7fc143fff700 1 --2- 172.21.15.55:0/248155557 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc144156350 0x7fc14406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:36.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.717+0000 7fc143fff700 1 --2- 172.21.15.55:0/248155557 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fc130067130 0x7fc1300695e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:36.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.717+0000 7fc143fff700 1 --2- 172.21.15.55:0/248155557 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1440ace60 0x7fc14414fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:36.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.717+0000 7fc143fff700 1 --2- 172.21.15.55:0/248155557 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1440b0760 0x7fc14406dac0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:36.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.717+0000 7fc143fff700 1 -- 172.21.15.55:0/248155557 >> 172.21.15.55:0/248155557 conn(0x7fc144068e20 msgr2=0x7fc144151c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:36.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.717+0000 7fc143fff700 1 -- 172.21.15.55:0/248155557 shutdown_connections 2023-10-06T03:45:36.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:36.717+0000 7fc143fff700 1 -- 172.21.15.55:0/248155557 wait complete. 2023-10-06T03:45:36.736 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-10-06T03:45:36.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:36.736 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-06T03:45:36.774 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:36.775 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:36.775 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:36.775 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:36.775 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:36.775 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:36.775 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:36.775 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:36.775 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:36.775 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:36.776 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:36.777 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:36.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:36.777 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_35_00_UTC) 2023-10-06T03:45:36.886 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:36.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:36.887 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_36_00_UTC) 2023-10-06T03:45:36.931 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:36.931 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:36.932 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_37_00_UTC) 2023-10-06T03:45:37.031 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:37.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:37.032 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_38_00_UTC) 2023-10-06T03:45:37.075 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:37.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:37.075 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_39_00_UTC) 2023-10-06T03:45:37.159 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:37.159 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:37.159 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_40_00_UTC) 2023-10-06T03:45:37.243 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:37.243 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:37.243 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_41_00_UTC) 2023-10-06T03:45:37.327 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:37.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:37.327 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_42_00_UTC) 2023-10-06T03:45:37.411 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:37.411 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:37.411 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_43_00_UTC) 2023-10-06T03:45:37.495 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:37.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:37.495 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_44_00_UTC) 2023-10-06T03:45:37.579 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:37.579 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:37.579 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-06-03_45_00_UTC) 2023-10-06T03:45:37.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:37.664 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-10-06T03:45:37.744 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir4 1M 2023-10-06T03:45:37.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.983+0000 7f62e099d700 1 Processor -- start 2023-10-06T03:45:37.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.983+0000 7f62e099d700 1 -- start start 2023-10-06T03:45:37.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.985+0000 7f62e099d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f62dc156350 0x7f62dc157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:37.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.985+0000 7f62e099d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62dc0b0760 0x7f62dc0b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.985+0000 7f62e099d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62dc0b1110 con 0x7f62dc159f20 2023-10-06T03:45:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.985+0000 7f62e099d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62dc0b0570 con 0x7f62dc156350 2023-10-06T03:45:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.985+0000 7f62e099d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc1578b0 con 0x7f62dc0b0760 2023-10-06T03:45:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.985+0000 7f62daffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62dc0b0760 0x7f62dc0b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.985+0000 7f62daffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62dc0b0760 0x7f62dc0b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:38482/0 (socket says 172.21.15.55:38482) 2023-10-06T03:45:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.985+0000 7f62daffd700 1 -- 172.21.15.55:0/976188170 learned_addr learned my addr 172.21.15.55:0/976188170 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:37.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.985+0000 7f62db7fe700 1 --2- 172.21.15.55:0/976188170 >> v2:172.21.15.163:3300/0 conn(0x7f62dc156350 0x7f62dc157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:37.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.986+0000 7f62daffd700 1 -- 172.21.15.55:0/976188170 >> v2:172.21.15.163:3300/0 conn(0x7f62dc156350 msgr2=0x7f62dc157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:37.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.986+0000 7f62daffd700 1 --2- 172.21.15.55:0/976188170 >> v2:172.21.15.163:3300/0 conn(0x7f62dc156350 0x7f62dc157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:37.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.986+0000 7f62daffd700 1 -- 172.21.15.55:0/976188170 >> v1:172.21.15.163:6789/0 conn(0x7f62dc159f20 legacy=0x7f62dc15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:37.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.986+0000 7f62daffd700 1 -- 172.21.15.55:0/976188170 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62dc0af4d0 con 0x7f62dc0b0760 2023-10-06T03:45:37.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.986+0000 7f62da7fc700 1 -- 172.21.15.55:0/976188170 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727344891 0 0) 0x7f62dc0b1110 con 0x7f62dc159f20 2023-10-06T03:45:37.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.986+0000 7f62daffd700 1 --2- 172.21.15.55:0/976188170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62dc0b0760 0x7f62dc0b0bd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f62d00099b0 tx=0x7f62d0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaccbf44a5131a48 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:37.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.986+0000 7f62da7fc700 1 -- 172.21.15.55:0/976188170 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0009ea0 con 0x7f62dc0b0760 2023-10-06T03:45:37.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.986+0000 7f62da7fc700 1 -- 172.21.15.55:0/976188170 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d000b860 con 0x7f62dc0b0760 2023-10-06T03:45:37.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.986+0000 7f62da7fc700 1 -- 172.21.15.55:0/976188170 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d000bc00 con 0x7f62dc0b0760 2023-10-06T03:45:37.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.987+0000 7f62e099d700 1 -- 172.21.15.55:0/976188170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62dc0b0760 msgr2=0x7f62dc0b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:37.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.987+0000 7f62e099d700 1 --2- 172.21.15.55:0/976188170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62dc0b0760 0x7f62dc0b0bd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f62d00099b0 tx=0x7f62d0009250 comp rx=0 tx=0).stop 2023-10-06T03:45:37.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.987+0000 7f62e099d700 1 -- 172.21.15.55:0/976188170 shutdown_connections 2023-10-06T03:45:37.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.987+0000 7f62e099d700 1 --2- 172.21.15.55:0/976188170 >> v2:172.21.15.163:3300/0 conn(0x7f62dc156350 0x7f62dc157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:37.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.987+0000 7f62e099d700 1 --2- 172.21.15.55:0/976188170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62dc0b0760 0x7f62dc0b0bd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:37.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.987+0000 7f62e099d700 1 -- 172.21.15.55:0/976188170 >> 172.21.15.55:0/976188170 conn(0x7f62dc068e20 msgr2=0x7f62dc0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:37.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.987+0000 7f62e099d700 1 -- 172.21.15.55:0/976188170 shutdown_connections 2023-10-06T03:45:37.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.987+0000 7f62e099d700 1 -- 172.21.15.55:0/976188170 wait complete. 2023-10-06T03:45:37.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.988+0000 7f62e099d700 1 Processor -- start 2023-10-06T03:45:37.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.988+0000 7f62e099d700 1 -- start start 2023-10-06T03:45:37.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.989+0000 7f62e099d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62dc0b0760 0x7f62dc151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:37.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.989+0000 7f62e099d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62dc156350 0x7f62dc151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:37.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.989+0000 7f62e099d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62dc159f20 0x7f62dc152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:37.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.989+0000 7f62e099d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc16a1a0 con 0x7f62dc156350 2023-10-06T03:45:37.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.989+0000 7f62e099d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc1517d0 con 0x7f62dc159f20 2023-10-06T03:45:37.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.989+0000 7f62e099d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc0ab1c0 con 0x7f62dc0b0760 2023-10-06T03:45:37.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.990+0000 7f62daffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62dc156350 0x7f62dc151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:37.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.990+0000 7f62daffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62dc156350 0x7f62dc151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52198/0 (socket says 172.21.15.55:52198) 2023-10-06T03:45:37.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.990+0000 7f62daffd700 1 -- 172.21.15.55:0/439416441 learned_addr learned my addr 172.21.15.55:0/439416441 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:37.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.990+0000 7f62db7fe700 1 --2- 172.21.15.55:0/439416441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62dc0b0760 0x7f62dc151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:37.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.990+0000 7f62dbfff700 1 --2- 172.21.15.55:0/439416441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62dc159f20 0x7f62dc152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:37.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.990+0000 7f62daffd700 1 -- 172.21.15.55:0/439416441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62dc0b0760 msgr2=0x7f62dc151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:37.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.990+0000 7f62daffd700 1 --2- 172.21.15.55:0/439416441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62dc0b0760 0x7f62dc151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:37.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.990+0000 7f62daffd700 1 -- 172.21.15.55:0/439416441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62dc159f20 msgr2=0x7f62dc152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:37.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.990+0000 7f62daffd700 1 --2- 172.21.15.55:0/439416441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62dc159f20 0x7f62dc152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:37.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.990+0000 7f62daffd700 1 -- 172.21.15.55:0/439416441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d00094f0 con 0x7f62dc156350 2023-10-06T03:45:37.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.990+0000 7f62daffd700 1 --2- 172.21.15.55:0/439416441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62dc156350 0x7f62dc151af0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f62dc1579f0 tx=0x7f62d0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:37.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.991+0000 7f62c37fe700 1 -- 172.21.15.55:0/439416441 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0000d10 con 0x7f62dc156350 2023-10-06T03:45:37.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.991+0000 7f62c37fe700 1 -- 172.21.15.55:0/439416441 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d000bbe0 con 0x7f62dc156350 2023-10-06T03:45:37.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.991+0000 7f62e099d700 1 -- 172.21.15.55:0/439416441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc0ab4a0 con 0x7f62dc156350 2023-10-06T03:45:37.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.991+0000 7f62c37fe700 1 -- 172.21.15.55:0/439416441 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0011650 con 0x7f62dc156350 2023-10-06T03:45:37.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.991+0000 7f62e099d700 1 -- 172.21.15.55:0/439416441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc0aba60 con 0x7f62dc156350 2023-10-06T03:45:37.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.992+0000 7f62e099d700 1 -- 172.21.15.55:0/439416441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62dc16a310 con 0x7f62dc156350 2023-10-06T03:45:37.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.993+0000 7f62c37fe700 1 -- 172.21.15.55:0/439416441 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f62d0011a80 con 0x7f62dc156350 2023-10-06T03:45:37.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.994+0000 7f62c37fe700 1 --2- 172.21.15.55:0/439416441 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f62c8067130 0x7f62c80695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:37.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.994+0000 7f62c37fe700 1 -- 172.21.15.55:0/439416441 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6785+0+0 (secure 0 0 0) 0x7f62d0086110 con 0x7f62dc156350 2023-10-06T03:45:37.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.994+0000 7f62db7fe700 1 --2- 172.21.15.55:0/439416441 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f62c8067130 0x7f62c80695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:37.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.995+0000 7f62db7fe700 1 --2- 172.21.15.55:0/439416441 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f62c8067130 0x7f62c80695e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009d10 tx=0x7f62cc0091a0 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:38.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:37.998+0000 7f62c37fe700 1 -- 172.21.15.55:0/439416441 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62d0052ca0 con 0x7f62dc156350 2023-10-06T03:45:38.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:38.226+0000 7f62e099d700 1 -- 172.21.15.55:0/439416441 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f62dc0cbfd0 con 0x7f62c8067130 2023-10-06T03:45:38.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:38.322+0000 7f62c37fe700 1 -- 172.21.15.55:0/439416441 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f62dc0cbfd0 con 0x7f62c8067130 2023-10-06T03:45:38.323 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-10-06T03:45:38.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:38.324+0000 7f62e099d700 1 -- 172.21.15.55:0/439416441 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f62c8067130 msgr2=0x7f62c80695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:38.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:38.324+0000 7f62e099d700 1 --2- 172.21.15.55:0/439416441 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f62c8067130 0x7f62c80695e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009d10 tx=0x7f62cc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:38.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:38.324+0000 7f62e099d700 1 -- 172.21.15.55:0/439416441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62dc156350 msgr2=0x7f62dc151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:38.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:38.324+0000 7f62e099d700 1 --2- 172.21.15.55:0/439416441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62dc156350 0x7f62dc151af0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f62dc1579f0 tx=0x7f62d0017000 comp rx=0 tx=0).stop 2023-10-06T03:45:38.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:38.324+0000 7f62e099d700 1 -- 172.21.15.55:0/439416441 shutdown_connections 2023-10-06T03:45:38.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:38.324+0000 7f62e099d700 1 --2- 172.21.15.55:0/439416441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62dc159f20 0x7f62dc152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:38.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:38.324+0000 7f62e099d700 1 --2- 172.21.15.55:0/439416441 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f62c8067130 0x7f62c80695e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:38.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:38.324+0000 7f62e099d700 1 --2- 172.21.15.55:0/439416441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62dc0b0760 0x7f62dc151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:38.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:38.324+0000 7f62e099d700 1 --2- 172.21.15.55:0/439416441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62dc156350 0x7f62dc151af0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:38.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:38.324+0000 7f62e099d700 1 -- 172.21.15.55:0/439416441 >> 172.21.15.55:0/439416441 conn(0x7f62dc068e20 msgr2=0x7f62dc0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:38.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:38.324+0000 7f62e099d700 1 -- 172.21.15.55:0/439416441 shutdown_connections 2023-10-06T03:45:38.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:38.324+0000 7f62e099d700 1 -- 172.21.15.55:0/439416441 wait complete. 2023-10-06T03:45:38.343 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-10-06T03:45:38.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:38.343 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-06T03:45:38.383 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_01_UTC 2023-10-06T03:45:38.383 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:38.383 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:38.383 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:38.384 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:38.384 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:38.384 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:38.384 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:38.384 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:38.384 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:38.384 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:38.384 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_35_01_UTC 2023-10-06T03:45:38.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:38.385 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_35_01_UTC) 2023-10-06T03:45:38.477 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:38.477 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:38.477 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_36_00_UTC) 2023-10-06T03:45:38.522 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:38.522 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:38.522 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_37_00_UTC) 2023-10-06T03:45:38.608 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:38.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:38.609 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_38_00_UTC) 2023-10-06T03:45:38.698 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:38.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:38.698 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_39_00_UTC) 2023-10-06T03:45:38.743 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:38.743 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:38.743 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_40_00_UTC) 2023-10-06T03:45:38.830 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:38.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:38.831 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_41_00_UTC) 2023-10-06T03:45:38.927 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:38.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:38.927 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_42_00_UTC) 2023-10-06T03:45:38.986 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:38.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:38.986 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_43_00_UTC) 2023-10-06T03:45:39.029 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:39.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:39.030 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_44_00_UTC) 2023-10-06T03:45:39.116 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:39.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:39.116 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-06-03_45_00_UTC) 2023-10-06T03:45:39.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:39.206 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-10-06T03:45:39.258 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir5 1M 2023-10-06T03:45:39.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.448+0000 7f54f659d700 1 Processor -- start 2023-10-06T03:45:39.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.449+0000 7f54f659d700 1 -- start start 2023-10-06T03:45:39.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.450+0000 7f54f659d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f54f00b0760 0x7f54f00b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:39.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.450+0000 7f54f659d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f54f00ace60 0x7f54f00ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:39.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.450+0000 7f54f659d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f00b0570 con 0x7f54f0156350 2023-10-06T03:45:39.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.450+0000 7f54f659d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f00b1140 con 0x7f54f00ace60 2023-10-06T03:45:39.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.450+0000 7f54f659d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f00af4d0 con 0x7f54f00b0760 2023-10-06T03:45:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.450+0000 7f54f5d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f54f00ace60 0x7f54f00ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.450+0000 7f54f559b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f54f0156350 0x7f54f0157370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39872/0 (socket says 172.21.15.55:39872) 2023-10-06T03:45:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.450+0000 7f54f559b700 1 -- 172.21.15.55:0/4044955694 learned_addr learned my addr 172.21.15.55:0/4044955694 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.450+0000 7f54f4d9a700 1 --2- 172.21.15.55:0/4044955694 >> v2:172.21.15.163:3300/0 conn(0x7f54f00b0760 0x7f54f00b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.451+0000 7f54e7fff700 1 -- 172.21.15.55:0/4044955694 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408175460 0 0) 0x7f54f00b0570 con 0x7f54f0156350 2023-10-06T03:45:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.451+0000 7f54e7fff700 1 -- 172.21.15.55:0/4044955694 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54dc003570 con 0x7f54f0156350 2023-10-06T03:45:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.451+0000 7f54f5d9c700 1 -- 172.21.15.55:0/4044955694 >> v2:172.21.15.163:3300/0 conn(0x7f54f00b0760 msgr2=0x7f54f00b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.451+0000 7f54f5d9c700 1 --2- 172.21.15.55:0/4044955694 >> v2:172.21.15.163:3300/0 conn(0x7f54f00b0760 0x7f54f00b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:39.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.451+0000 7f54f5d9c700 1 -- 172.21.15.55:0/4044955694 >> v1:172.21.15.55:6789/0 conn(0x7f54f0156350 legacy=0x7f54f0157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:39.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.451+0000 7f54f5d9c700 1 -- 172.21.15.55:0/4044955694 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f00af7b0 con 0x7f54f00ace60 2023-10-06T03:45:39.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.451+0000 7f54f5d9c700 1 --2- 172.21.15.55:0/4044955694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f00ace60 0x7f54f00ad260 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f54f00642a0 tx=0x7f54e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c007f78196149c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:39.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.451+0000 7f54e7fff700 1 -- 172.21.15.55:0/4044955694 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e800e070 con 0x7f54f00ace60 2023-10-06T03:45:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.451+0000 7f54e7fff700 1 -- 172.21.15.55:0/4044955694 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e800bab0 con 0x7f54f00ace60 2023-10-06T03:45:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.451+0000 7f54e7fff700 1 -- 172.21.15.55:0/4044955694 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e800be70 con 0x7f54f00ace60 2023-10-06T03:45:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.452+0000 7f54f659d700 1 -- 172.21.15.55:0/4044955694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f00ace60 msgr2=0x7f54f00ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.452+0000 7f54f659d700 1 --2- 172.21.15.55:0/4044955694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f00ace60 0x7f54f00ad260 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f54f00642a0 tx=0x7f54e8009780 comp rx=0 tx=0).stop 2023-10-06T03:45:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.452+0000 7f54f659d700 1 -- 172.21.15.55:0/4044955694 shutdown_connections 2023-10-06T03:45:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.452+0000 7f54f659d700 1 --2- 172.21.15.55:0/4044955694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f00ace60 0x7f54f00ad260 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.452+0000 7f54f659d700 1 --2- 172.21.15.55:0/4044955694 >> v2:172.21.15.163:3300/0 conn(0x7f54f00b0760 0x7f54f00b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.452+0000 7f54f659d700 1 -- 172.21.15.55:0/4044955694 >> 172.21.15.55:0/4044955694 conn(0x7f54f0068e20 msgr2=0x7f54f00cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.452+0000 7f54f659d700 1 -- 172.21.15.55:0/4044955694 shutdown_connections 2023-10-06T03:45:39.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.452+0000 7f54f659d700 1 -- 172.21.15.55:0/4044955694 wait complete. 2023-10-06T03:45:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.454+0000 7f54f659d700 1 Processor -- start 2023-10-06T03:45:39.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.454+0000 7f54f659d700 1 -- start start 2023-10-06T03:45:39.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f659d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f00ace60 0x7f54f006e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:39.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f659d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54f00b0760 0x7f54f014b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:39.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f659d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54f0156350 0x7f54f0149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:39.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f659d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f014beb0 con 0x7f54f00ace60 2023-10-06T03:45:39.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f659d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f014c190 con 0x7f54f0156350 2023-10-06T03:45:39.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f659d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f014c470 con 0x7f54f00b0760 2023-10-06T03:45:39.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f559b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f00ace60 0x7f54f006e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:39.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f559b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f00ace60 0x7f54f006e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52220/0 (socket says 172.21.15.55:52220) 2023-10-06T03:45:39.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f559b700 1 -- 172.21.15.55:0/2335810519 learned_addr learned my addr 172.21.15.55:0/2335810519 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:39.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f5d9c700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54f0156350 0x7f54f0149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:39.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f4d9a700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54f00b0760 0x7f54f014b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:39.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f559b700 1 -- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54f00b0760 msgr2=0x7f54f014b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:39.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f559b700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54f00b0760 0x7f54f014b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:39.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f559b700 1 -- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54f0156350 msgr2=0x7f54f0149db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:39.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f559b700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54f0156350 0x7f54f0149db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:39.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f559b700 1 -- 172.21.15.55:0/2335810519 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e8009a20 con 0x7f54f00ace60 2023-10-06T03:45:39.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.455+0000 7f54f5d9c700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54f0156350 0x7f54f0149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:45:39.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.456+0000 7f54f4d9a700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54f00b0760 0x7f54f014b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:45:39.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.456+0000 7f54f559b700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f00ace60 0x7f54f006e710 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00c7c0 tx=0x7f54ec007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:39.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.456+0000 7f54e57fa700 1 -- 172.21.15.55:0/2335810519 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec005b30 con 0x7f54f00ace60 2023-10-06T03:45:39.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.456+0000 7f54e57fa700 1 -- 172.21.15.55:0/2335810519 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ec005c90 con 0x7f54f00ace60 2023-10-06T03:45:39.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.456+0000 7f54f659d700 1 -- 172.21.15.55:0/2335810519 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f014c750 con 0x7f54f00ace60 2023-10-06T03:45:39.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.456+0000 7f54e57fa700 1 -- 172.21.15.55:0/2335810519 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec007340 con 0x7f54f00ace60 2023-10-06T03:45:39.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.456+0000 7f54f659d700 1 -- 172.21.15.55:0/2335810519 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f014ea60 con 0x7f54f00ace60 2023-10-06T03:45:39.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.457+0000 7f54f659d700 1 -- 172.21.15.55:0/2335810519 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f006a520 con 0x7f54f00ace60 2023-10-06T03:45:39.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.458+0000 7f54e57fa700 1 -- 172.21.15.55:0/2335810519 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f54ec01b400 con 0x7f54f00ace60 2023-10-06T03:45:39.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.459+0000 7f54e57fa700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f54d4067130 0x7f54d40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:39.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.459+0000 7f54e57fa700 1 -- 172.21.15.55:0/2335810519 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6881+0+0 (secure 0 0 0) 0x7f54ec084890 con 0x7f54f00ace60 2023-10-06T03:45:39.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.459+0000 7f54f4d9a700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f54d4067130 0x7f54d40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:39.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.460+0000 7f54f4d9a700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f54d4067130 0x7f54d40695e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f54e0006f00 tx=0x7f54e0008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:39.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.463+0000 7f54e57fa700 1 -- 172.21.15.55:0/2335810519 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54ec0513c0 con 0x7f54f00ace60 2023-10-06T03:45:39.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.694+0000 7f54f659d700 1 -- 172.21.15.55:0/2335810519 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f54f00b2200 con 0x7f54d4067130 2023-10-06T03:45:39.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.772+0000 7f54e57fa700 1 -- 172.21.15.55:0/2335810519 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f54f00b2200 con 0x7f54d4067130 2023-10-06T03:45:39.774 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-10-06T03:45:39.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.774+0000 7f54f659d700 1 -- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f54d4067130 msgr2=0x7f54d40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:39.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.774+0000 7f54f659d700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f54d4067130 0x7f54d40695e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f54e0006f00 tx=0x7f54e0008040 comp rx=0 tx=0).stop 2023-10-06T03:45:39.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.774+0000 7f54f659d700 1 -- 172.21.15.55:0/2335810519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f00ace60 msgr2=0x7f54f006e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:39.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.774+0000 7f54f659d700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f00ace60 0x7f54f006e710 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00c7c0 tx=0x7f54ec007e00 comp rx=0 tx=0).stop 2023-10-06T03:45:39.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.774+0000 7f54f659d700 1 -- 172.21.15.55:0/2335810519 shutdown_connections 2023-10-06T03:45:39.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.774+0000 7f54f659d700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54f0156350 0x7f54f0149db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:39.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.774+0000 7f54f659d700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54f00ace60 0x7f54f006e710 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:39.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.774+0000 7f54f659d700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f54d4067130 0x7f54d40695e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:39.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.775+0000 7f54f659d700 1 --2- 172.21.15.55:0/2335810519 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54f00b0760 0x7f54f014b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:39.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.775+0000 7f54f659d700 1 -- 172.21.15.55:0/2335810519 >> 172.21.15.55:0/2335810519 conn(0x7f54f0068e20 msgr2=0x7f54f0069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:39.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.775+0000 7f54f659d700 1 -- 172.21.15.55:0/2335810519 shutdown_connections 2023-10-06T03:45:39.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:39.775+0000 7f54f659d700 1 -- 172.21.15.55:0/2335810519 wait complete. 2023-10-06T03:45:39.793 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-10-06T03:45:39.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:39.793 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-06T03:45:39.832 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:39.832 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:39.832 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:39.832 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:39.832 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:39.832 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:39.832 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:39.832 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:39.832 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:39.833 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:39.833 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:39.834 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:39.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:39.834 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_35_00_UTC) 2023-10-06T03:45:39.946 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:39.946 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:39.946 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_36_00_UTC) 2023-10-06T03:45:40.006 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:40.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:40.006 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_37_00_UTC) 2023-10-06T03:45:40.050 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:40.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:40.051 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_38_00_UTC) 2023-10-06T03:45:40.135 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:40.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:40.136 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_39_00_UTC) 2023-10-06T03:45:40.226 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:40.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:40.226 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_40_00_UTC) 2023-10-06T03:45:40.270 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:40.270 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:40.271 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_41_00_UTC) 2023-10-06T03:45:40.354 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:40.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:40.354 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_42_00_UTC) 2023-10-06T03:45:40.439 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:40.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:40.440 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_43_00_UTC) 2023-10-06T03:45:40.523 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:40.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:40.524 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_44_00_UTC) 2023-10-06T03:45:40.608 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:40.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:40.609 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-06-03_45_00_UTC) 2023-10-06T03:45:40.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:40.704 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-10-06T03:45:40.745 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir6 1M 2023-10-06T03:45:40.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.978+0000 7f826cd9d700 1 Processor -- start 2023-10-06T03:45:40.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.978+0000 7f826cd9d700 1 -- start start 2023-10-06T03:45:40.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.979+0000 7f826cd9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82680b0790 0x7f82680cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:40.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.979+0000 7f826cd9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f82681591e0 0x7f8268159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:40.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.980+0000 7f826cd9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82680cfa00 con 0x7f82680b03c0 2023-10-06T03:45:40.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.980+0000 7f826cd9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82680cb580 con 0x7f82681591e0 2023-10-06T03:45:40.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.980+0000 7f826cd9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82680cf160 con 0x7f82680b0790 2023-10-06T03:45:40.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.980+0000 7f8266ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82680b0790 0x7f82680cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:40.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.980+0000 7f8266ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82680b0790 0x7f82680cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:38518/0 (socket says 172.21.15.55:38518) 2023-10-06T03:45:40.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.980+0000 7f8266ffd700 1 -- 172.21.15.55:0/2470854556 learned_addr learned my addr 172.21.15.55:0/2470854556 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:40.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.980+0000 7f8267fff700 1 --2- 172.21.15.55:0/2470854556 >> v2:172.21.15.163:3300/0 conn(0x7f82681591e0 0x7f8268159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:40.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.980+0000 7f82677fe700 1 --1- 172.21.15.55:0/2470854556 >> v1:172.21.15.163:6789/0 conn(0x7f82680b03c0 0x7f82680b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:34826/0 (socket says 172.21.15.55:34826) 2023-10-06T03:45:40.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.980+0000 7f82667fc700 1 -- 172.21.15.55:0/2470854556 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977614327 0 0) 0x7f82680cfa00 con 0x7f82680b03c0 2023-10-06T03:45:40.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.980+0000 7f8266ffd700 1 -- 172.21.15.55:0/2470854556 >> v2:172.21.15.163:3300/0 conn(0x7f82681591e0 msgr2=0x7f8268159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:40.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.980+0000 7f8266ffd700 1 --2- 172.21.15.55:0/2470854556 >> v2:172.21.15.163:3300/0 conn(0x7f82681591e0 0x7f8268159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:40.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.981+0000 7f8266ffd700 1 -- 172.21.15.55:0/2470854556 >> v1:172.21.15.163:6789/0 conn(0x7f82680b03c0 legacy=0x7f82680b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:40.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.981+0000 7f8266ffd700 1 -- 172.21.15.55:0/2470854556 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82680cf440 con 0x7f82680b0790 2023-10-06T03:45:40.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.981+0000 7f8267fff700 1 --2- 172.21.15.55:0/2470854556 >> v2:172.21.15.163:3300/0 conn(0x7f82681591e0 0x7f8268159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:45:40.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.981+0000 7f8266ffd700 1 --2- 172.21.15.55:0/2470854556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82680b0790 0x7f82680cea70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f825c0099b0 tx=0x7f825c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f9b996c027da369 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:40.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.981+0000 7f82667fc700 1 -- 172.21.15.55:0/2470854556 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c009ea0 con 0x7f82680b0790 2023-10-06T03:45:40.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.981+0000 7f82667fc700 1 -- 172.21.15.55:0/2470854556 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c00b860 con 0x7f82680b0790 2023-10-06T03:45:40.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.981+0000 7f82667fc700 1 -- 172.21.15.55:0/2470854556 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c00bc00 con 0x7f82680b0790 2023-10-06T03:45:40.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.982+0000 7f826cd9d700 1 -- 172.21.15.55:0/2470854556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82680b0790 msgr2=0x7f82680cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:40.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.982+0000 7f826cd9d700 1 --2- 172.21.15.55:0/2470854556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82680b0790 0x7f82680cea70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f825c0099b0 tx=0x7f825c009250 comp rx=0 tx=0).stop 2023-10-06T03:45:40.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.982+0000 7f826cd9d700 1 -- 172.21.15.55:0/2470854556 shutdown_connections 2023-10-06T03:45:40.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.982+0000 7f826cd9d700 1 --2- 172.21.15.55:0/2470854556 >> v2:172.21.15.163:3300/0 conn(0x7f82681591e0 0x7f8268159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:40.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.982+0000 7f826cd9d700 1 --2- 172.21.15.55:0/2470854556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82680b0790 0x7f82680cea70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:40.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.982+0000 7f826cd9d700 1 -- 172.21.15.55:0/2470854556 >> 172.21.15.55:0/2470854556 conn(0x7f8268063230 msgr2=0x7f8268063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:40.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.982+0000 7f826cd9d700 1 -- 172.21.15.55:0/2470854556 shutdown_connections 2023-10-06T03:45:40.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.982+0000 7f826cd9d700 1 -- 172.21.15.55:0/2470854556 wait complete. 2023-10-06T03:45:40.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.983+0000 7f826cd9d700 1 Processor -- start 2023-10-06T03:45:40.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.983+0000 7f826cd9d700 1 -- start start 2023-10-06T03:45:40.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.984+0000 7f826cd9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82680b03c0 0x7f82680af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:40.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.984+0000 7f826cd9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82680b0790 0x7f826816a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:40.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.984+0000 7f826cd9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82681591e0 0x7f826816a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:40.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.984+0000 7f826cd9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82680afdd0 con 0x7f82680b03c0 2023-10-06T03:45:40.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.984+0000 7f826cd9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82680c5d00 con 0x7f82680b0790 2023-10-06T03:45:40.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.984+0000 7f826cd9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82680c50f0 con 0x7f82681591e0 2023-10-06T03:45:40.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.984+0000 7f82677fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82680b03c0 0x7f82680af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:40.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.985+0000 7f82677fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82680b03c0 0x7f82680af890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52222/0 (socket says 172.21.15.55:52222) 2023-10-06T03:45:40.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.985+0000 7f82677fe700 1 -- 172.21.15.55:0/3327238021 learned_addr learned my addr 172.21.15.55:0/3327238021 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:40.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.985+0000 7f8267fff700 1 --2- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82681591e0 0x7f826816a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:40.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.985+0000 7f8266ffd700 1 --2- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82680b0790 0x7f826816a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:40.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.985+0000 7f8267fff700 1 -- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82680b0790 msgr2=0x7f826816a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:40.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.985+0000 7f8267fff700 1 --2- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82680b0790 0x7f826816a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:40.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.985+0000 7f8267fff700 1 -- 172.21.15.55:0/3327238021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82680b03c0 msgr2=0x7f82680af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:40.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.985+0000 7f8267fff700 1 --2- 172.21.15.55:0/3327238021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82680b03c0 0x7f82680af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:40.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.985+0000 7f8267fff700 1 -- 172.21.15.55:0/3327238021 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8260009a20 con 0x7f82681591e0 2023-10-06T03:45:40.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.985+0000 7f8267fff700 1 --2- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82681591e0 0x7f826816a5c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8260009f10 tx=0x7f826000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:40.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.986+0000 7f824f7fe700 1 -- 172.21.15.55:0/3327238021 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826000cb70 con 0x7f82681591e0 2023-10-06T03:45:40.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.986+0000 7f824f7fe700 1 -- 172.21.15.55:0/3327238021 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826000f040 con 0x7f82681591e0 2023-10-06T03:45:40.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.986+0000 7f824f7fe700 1 -- 172.21.15.55:0/3327238021 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260014480 con 0x7f82681591e0 2023-10-06T03:45:40.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.986+0000 7f826cd9d700 1 -- 172.21.15.55:0/3327238021 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f825c0094f0 con 0x7f82681591e0 2023-10-06T03:45:40.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.986+0000 7f826cd9d700 1 -- 172.21.15.55:0/3327238021 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82680c56b0 con 0x7f82681591e0 2023-10-06T03:45:40.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.987+0000 7f826cd9d700 1 -- 172.21.15.55:0/3327238021 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82680c5e70 con 0x7f82681591e0 2023-10-06T03:45:40.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.988+0000 7f824f7fe700 1 -- 172.21.15.55:0/3327238021 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f8260003020 con 0x7f82681591e0 2023-10-06T03:45:40.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.988+0000 7f824f7fe700 1 --2- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8254067130 0x7f82540695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:40.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.989+0000 7f824f7fe700 1 -- 172.21.15.55:0/3327238021 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f8260085720 con 0x7f82681591e0 2023-10-06T03:45:40.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.989+0000 7f82677fe700 1 --2- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8254067130 0x7f82540695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:40.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.989+0000 7f82677fe700 1 --2- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8254067130 0x7f82540695e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8258007fd0 tx=0x7f8258001040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:40.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:40.994+0000 7f824f7fe700 1 -- 172.21.15.55:0/3327238021 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8260052080 con 0x7f82681591e0 2023-10-06T03:45:41.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:41.226+0000 7f826cd9d700 1 -- 172.21.15.55:0/3327238021 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f826804c3c0 con 0x7f8254067130 2023-10-06T03:45:41.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:41.302+0000 7f824f7fe700 1 -- 172.21.15.55:0/3327238021 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f826804c3c0 con 0x7f8254067130 2023-10-06T03:45:41.304 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-10-06T03:45:41.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:41.305+0000 7f826cd9d700 1 -- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8254067130 msgr2=0x7f82540695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:41.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:41.305+0000 7f826cd9d700 1 --2- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8254067130 0x7f82540695e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8258007fd0 tx=0x7f8258001040 comp rx=0 tx=0).stop 2023-10-06T03:45:41.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:41.305+0000 7f826cd9d700 1 -- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82681591e0 msgr2=0x7f826816a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:41.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:41.305+0000 7f826cd9d700 1 --2- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82681591e0 0x7f826816a5c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8260009f10 tx=0x7f826000b040 comp rx=0 tx=0).stop 2023-10-06T03:45:41.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:41.305+0000 7f826cd9d700 1 -- 172.21.15.55:0/3327238021 shutdown_connections 2023-10-06T03:45:41.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:41.305+0000 7f826cd9d700 1 --2- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8254067130 0x7f82540695e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:41.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:41.305+0000 7f826cd9d700 1 --2- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82681591e0 0x7f826816a5c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:41.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:41.305+0000 7f826cd9d700 1 --2- 172.21.15.55:0/3327238021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82680b03c0 0x7f82680af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:41.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:41.305+0000 7f826cd9d700 1 --2- 172.21.15.55:0/3327238021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82680b0790 0x7f826816a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:41.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:41.305+0000 7f826cd9d700 1 -- 172.21.15.55:0/3327238021 >> 172.21.15.55:0/3327238021 conn(0x7f8268063230 msgr2=0x7f8268068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:41.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:41.305+0000 7f826cd9d700 1 -- 172.21.15.55:0/3327238021 shutdown_connections 2023-10-06T03:45:41.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:41.305+0000 7f826cd9d700 1 -- 172.21.15.55:0/3327238021 wait complete. 2023-10-06T03:45:41.324 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-10-06T03:45:41.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:41.324 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-06T03:45:41.362 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_01_UTC 2023-10-06T03:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:41.363 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:41.364 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_35_01_UTC 2023-10-06T03:45:41.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:41.365 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_35_01_UTC) 2023-10-06T03:45:41.452 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:41.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:41.452 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_36_00_UTC) 2023-10-06T03:45:41.496 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:41.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:41.496 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_37_00_UTC) 2023-10-06T03:45:41.583 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:41.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:41.583 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_38_00_UTC) 2023-10-06T03:45:41.627 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:41.627 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:41.627 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_39_00_UTC) 2023-10-06T03:45:41.726 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:41.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:41.726 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_40_00_UTC) 2023-10-06T03:45:41.778 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:41.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:41.779 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_41_00_UTC) 2023-10-06T03:45:41.824 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:41.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:41.824 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_42_00_UTC) 2023-10-06T03:45:41.937 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:41.937 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:41.937 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_43_00_UTC) 2023-10-06T03:45:41.982 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:41.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:41.982 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_44_00_UTC) 2023-10-06T03:45:42.066 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:42.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:42.066 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-06-03_45_00_UTC) 2023-10-06T03:45:42.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:42.150 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-10-06T03:45:42.231 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir7 1M 2023-10-06T03:45:42.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.461+0000 7f9bd759e700 1 Processor -- start 2023-10-06T03:45:42.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.462+0000 7f9bd759e700 1 -- start start 2023-10-06T03:45:42.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.463+0000 7f9bd759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9bd815b050 0x7f9bd80ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:42.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.463+0000 7f9bd759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9bd80cc8d0 0x7f9bd815b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:42.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.463+0000 7f9bd759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd8157a50 con 0x7f9bd815b960 2023-10-06T03:45:42.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.463+0000 7f9bd759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd81591b0 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.463+0000 7f9bd759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd80ca9e0 con 0x7f9bd815b050 2023-10-06T03:45:42.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.463+0000 7f9bd5d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9bd80cc8d0 0x7f9bd815b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:42.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.463+0000 7f9bd6d9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9bd815b960 0x7f9bd8157f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39882/0 (socket says 172.21.15.55:39882) 2023-10-06T03:45:42.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.463+0000 7f9bd6d9d700 1 -- 172.21.15.55:0/858174659 learned_addr learned my addr 172.21.15.55:0/858174659 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:42.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.463+0000 7f9bd659c700 1 --2- 172.21.15.55:0/858174659 >> v2:172.21.15.163:3300/0 conn(0x7f9bd815b050 0x7f9bd80ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:42.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.464+0000 7f9bd5d9b700 1 -- 172.21.15.55:0/858174659 >> v2:172.21.15.163:3300/0 conn(0x7f9bd815b050 msgr2=0x7f9bd80ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:42.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.464+0000 7f9bd5d9b700 1 --2- 172.21.15.55:0/858174659 >> v2:172.21.15.163:3300/0 conn(0x7f9bd815b050 0x7f9bd80ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:42.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.464+0000 7f9bd5d9b700 1 -- 172.21.15.55:0/858174659 >> v1:172.21.15.55:6789/0 conn(0x7f9bd815b960 legacy=0x7f9bd8157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:42.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.464+0000 7f9bd5d9b700 1 -- 172.21.15.55:0/858174659 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd80cacc0 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.464+0000 7f9bd559a700 1 -- 172.21.15.55:0/858174659 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68126581 0 0) 0x7f9bd8157a50 con 0x7f9bd815b960 2023-10-06T03:45:42.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.464+0000 7f9bd5d9b700 1 --2- 172.21.15.55:0/858174659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bd80cc8d0 0x7f9bd815b420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8064f60 tx=0x7f9bcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a546201128b0aea2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:42.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.464+0000 7f9bd559a700 1 -- 172.21.15.55:0/858174659 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc011630 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.464+0000 7f9bd559a700 1 -- 172.21.15.55:0/858174659 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bcc011790 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.464+0000 7f9bd559a700 1 -- 172.21.15.55:0/858174659 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc011b30 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.465+0000 7f9bd759e700 1 -- 172.21.15.55:0/858174659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bd80cc8d0 msgr2=0x7f9bd815b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:42.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.465+0000 7f9bd759e700 1 --2- 172.21.15.55:0/858174659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bd80cc8d0 0x7f9bd815b420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8064f60 tx=0x7f9bcc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:42.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.465+0000 7f9bd759e700 1 -- 172.21.15.55:0/858174659 shutdown_connections 2023-10-06T03:45:42.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.465+0000 7f9bd759e700 1 --2- 172.21.15.55:0/858174659 >> v2:172.21.15.163:3300/0 conn(0x7f9bd815b050 0x7f9bd80ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:42.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.465+0000 7f9bd759e700 1 --2- 172.21.15.55:0/858174659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bd80cc8d0 0x7f9bd815b420 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:42.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.465+0000 7f9bd759e700 1 -- 172.21.15.55:0/858174659 >> 172.21.15.55:0/858174659 conn(0x7f9bd8068e20 msgr2=0x7f9bd80cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:42.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.465+0000 7f9bd759e700 1 -- 172.21.15.55:0/858174659 shutdown_connections 2023-10-06T03:45:42.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.465+0000 7f9bd759e700 1 -- 172.21.15.55:0/858174659 wait complete. 2023-10-06T03:45:42.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.467+0000 7f9bd759e700 1 Processor -- start 2023-10-06T03:45:42.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.467+0000 7f9bd759e700 1 -- start start 2023-10-06T03:45:42.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bd80cc8d0 0x7f9bd8145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:42.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bd815b050 0x7f9bd8159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:42.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bd815b960 0x7f9bd815a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:42.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd8169860 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd814f340 con 0x7f9bd815b050 2023-10-06T03:45:42.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80afb50 con 0x7f9bd815b960 2023-10-06T03:45:42.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bd80cc8d0 0x7f9bd8145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:42.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bd80cc8d0 0x7f9bd8145cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52232/0 (socket says 172.21.15.55:52232) 2023-10-06T03:45:42.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd659c700 1 -- 172.21.15.55:0/337701579 learned_addr learned my addr 172.21.15.55:0/337701579 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:42.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd5d9b700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bd815b050 0x7f9bd8159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:42.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd6d9d700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bd815b960 0x7f9bd815a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:42.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd659c700 1 -- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bd815b960 msgr2=0x7f9bd815a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:42.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd659c700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bd815b960 0x7f9bd815a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:42.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd659c700 1 -- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bd815b050 msgr2=0x7f9bd8159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:42.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd659c700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bd815b050 0x7f9bd8159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd659c700 1 -- 172.21.15.55:0/337701579 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc009420 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.468+0000 7f9bd6d9d700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bd815b960 0x7f9bd815a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:45:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.469+0000 7f9bd659c700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bd80cc8d0 0x7f9bd8145cf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800ec80 tx=0x7f9bc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.469+0000 7f9bc67fc700 1 -- 172.21.15.55:0/337701579 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc800c970 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.469+0000 7f9bc67fc700 1 -- 172.21.15.55:0/337701579 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc800cad0 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.469+0000 7f9bd759e700 1 -- 172.21.15.55:0/337701579 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd80af290 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.469+0000 7f9bc67fc700 1 -- 172.21.15.55:0/337701579 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc800ce70 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.469+0000 7f9bd759e700 1 -- 172.21.15.55:0/337701579 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd80c88e0 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.470+0000 7f9bd759e700 1 -- 172.21.15.55:0/337701579 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd80af420 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.471+0000 7f9bc67fc700 1 -- 172.21.15.55:0/337701579 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f9bc8013680 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.472+0000 7f9bc67fc700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9bb8067130 0x7f9bb80695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:42.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.472+0000 7f9bc67fc700 1 -- 172.21.15.55:0/337701579 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6849+0+0 (secure 0 0 0) 0x7f9bc80850d0 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.472+0000 7f9bd5d9b700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9bb8067130 0x7f9bb80695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:42.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.473+0000 7f9bd5d9b700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9bb8067130 0x7f9bb80695e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0093f0 tx=0x7f9bcc017000 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:42.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.476+0000 7f9bc67fc700 1 -- 172.21.15.55:0/337701579 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bd80af420 con 0x7f9bd80cc8d0 2023-10-06T03:45:42.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.713+0000 7f9bd759e700 1 -- 172.21.15.55:0/337701579 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bd80af670 con 0x7f9bb8067130 2023-10-06T03:45:42.801 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.800+0000 7f9bc67fc700 1 -- 172.21.15.55:0/337701579 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f9bd80af670 con 0x7f9bb8067130 2023-10-06T03:45:42.802 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-10-06T03:45:42.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.802+0000 7f9bd759e700 1 -- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9bb8067130 msgr2=0x7f9bb80695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:42.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.802+0000 7f9bd759e700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9bb8067130 0x7f9bb80695e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc0093f0 tx=0x7f9bcc017000 comp rx=0 tx=0).stop 2023-10-06T03:45:42.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.802+0000 7f9bd759e700 1 -- 172.21.15.55:0/337701579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bd80cc8d0 msgr2=0x7f9bd8145cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:42.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.802+0000 7f9bd759e700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bd80cc8d0 0x7f9bd8145cf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800ec80 tx=0x7f9bc80091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:42.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.802+0000 7f9bd759e700 1 -- 172.21.15.55:0/337701579 shutdown_connections 2023-10-06T03:45:42.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.802+0000 7f9bd759e700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f9bb8067130 0x7f9bb80695e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:42.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.802+0000 7f9bd759e700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9bd815b960 0x7f9bd815a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:42.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.802+0000 7f9bd759e700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9bd80cc8d0 0x7f9bd8145cf0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:42.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.802+0000 7f9bd759e700 1 --2- 172.21.15.55:0/337701579 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9bd815b050 0x7f9bd8159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:42.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.802+0000 7f9bd759e700 1 -- 172.21.15.55:0/337701579 >> 172.21.15.55:0/337701579 conn(0x7f9bd8068e20 msgr2=0x7f9bd80694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:42.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.803+0000 7f9bd759e700 1 -- 172.21.15.55:0/337701579 shutdown_connections 2023-10-06T03:45:42.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:42.803+0000 7f9bd759e700 1 -- 172.21.15.55:0/337701579 wait complete. 2023-10-06T03:45:42.821 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-10-06T03:45:42.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:42.821 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-06T03:45:42.860 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:42.860 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:42.860 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:42.861 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:42.861 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:42.861 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:42.861 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:42.861 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:42.861 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:42.861 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:42.861 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:42.862 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:42.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:42.862 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_35_00_UTC) 2023-10-06T03:45:42.953 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:42.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:42.954 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_36_00_UTC) 2023-10-06T03:45:43.002 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:43.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:43.002 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_37_00_UTC) 2023-10-06T03:45:43.087 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:43.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:43.088 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_38_00_UTC) 2023-10-06T03:45:43.172 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:43.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:43.173 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_39_00_UTC) 2023-10-06T03:45:43.258 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:43.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:43.259 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_40_00_UTC) 2023-10-06T03:45:43.344 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:43.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:43.344 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_41_00_UTC) 2023-10-06T03:45:43.430 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:43.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:43.430 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_42_00_UTC) 2023-10-06T03:45:43.516 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:43.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:43.516 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_43_00_UTC) 2023-10-06T03:45:43.601 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:43.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:43.601 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_44_00_UTC) 2023-10-06T03:45:43.703 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:43.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:43.703 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-06-03_45_00_UTC) 2023-10-06T03:45:43.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:43.752 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-10-06T03:45:43.833 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir8 1M 2023-10-06T03:45:44.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.064+0000 7fbb37fff700 1 Processor -- start 2023-10-06T03:45:44.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.064+0000 7fbb37fff700 1 -- start start 2023-10-06T03:45:44.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.066+0000 7fbb37fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbb380ccb40 0x7fbb380697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:44.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.066+0000 7fbb37fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb38069cf0 0x7fbb3815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:44.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.066+0000 7fbb37fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb38159db0 con 0x7fbb380cb560 2023-10-06T03:45:44.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.066+0000 7fbb37fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3806aa30 con 0x7fbb38069cf0 2023-10-06T03:45:44.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.066+0000 7fbb37fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3806a470 con 0x7fbb380ccb40 2023-10-06T03:45:44.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.066+0000 7fbb377fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb38069cf0 0x7fbb3815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:44.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.066+0000 7fbb377fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb38069cf0 0x7fbb3815ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52934/0 (socket says 172.21.15.55:52934) 2023-10-06T03:45:44.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.066+0000 7fbb377fe700 1 -- 172.21.15.55:0/2591492889 learned_addr learned my addr 172.21.15.55:0/2591492889 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:44.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.066+0000 7fbb367fc700 1 --2- 172.21.15.55:0/2591492889 >> v2:172.21.15.163:3300/0 conn(0x7fbb380ccb40 0x7fbb380697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:44.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.066+0000 7fbb377fe700 1 -- 172.21.15.55:0/2591492889 >> v2:172.21.15.163:3300/0 conn(0x7fbb380ccb40 msgr2=0x7fbb380697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:44.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.066+0000 7fbb377fe700 1 --2- 172.21.15.55:0/2591492889 >> v2:172.21.15.163:3300/0 conn(0x7fbb380ccb40 0x7fbb380697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:44.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.066+0000 7fbb377fe700 1 -- 172.21.15.55:0/2591492889 >> v1:172.21.15.163:6789/0 conn(0x7fbb380cb560 legacy=0x7fbb380cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:44.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.066+0000 7fbb377fe700 1 -- 172.21.15.55:0/2591492889 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3806a750 con 0x7fbb38069cf0 2023-10-06T03:45:44.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.067+0000 7fbb377fe700 1 --2- 172.21.15.55:0/2591492889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb38069cf0 0x7fbb3815ba60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009f20 tx=0x7fbb30009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddafa8c67827d0c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:44.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.067+0000 7fbb35ffb700 1 -- 172.21.15.55:0/2591492889 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3000e070 con 0x7fbb38069cf0 2023-10-06T03:45:44.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.067+0000 7fbb35ffb700 1 -- 172.21.15.55:0/2591492889 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3000b990 con 0x7fbb38069cf0 2023-10-06T03:45:44.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.067+0000 7fbb35ffb700 1 -- 172.21.15.55:0/2591492889 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb300092e0 con 0x7fbb38069cf0 2023-10-06T03:45:44.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.067+0000 7fbb37fff700 1 -- 172.21.15.55:0/2591492889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb38069cf0 msgr2=0x7fbb3815ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:44.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.067+0000 7fbb37fff700 1 --2- 172.21.15.55:0/2591492889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb38069cf0 0x7fbb3815ba60 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbb30009f20 tx=0x7fbb30009840 comp rx=0 tx=0).stop 2023-10-06T03:45:44.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.068+0000 7fbb37fff700 1 -- 172.21.15.55:0/2591492889 shutdown_connections 2023-10-06T03:45:44.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.068+0000 7fbb37fff700 1 --2- 172.21.15.55:0/2591492889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb38069cf0 0x7fbb3815ba60 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:44.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.068+0000 7fbb37fff700 1 --2- 172.21.15.55:0/2591492889 >> v2:172.21.15.163:3300/0 conn(0x7fbb380ccb40 0x7fbb380697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:44.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.068+0000 7fbb37fff700 1 -- 172.21.15.55:0/2591492889 >> 172.21.15.55:0/2591492889 conn(0x7fbb3806baf0 msgr2=0x7fbb3806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:44.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.068+0000 7fbb37fff700 1 -- 172.21.15.55:0/2591492889 shutdown_connections 2023-10-06T03:45:44.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.068+0000 7fbb37fff700 1 -- 172.21.15.55:0/2591492889 wait complete. 2023-10-06T03:45:44.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.069+0000 7fbb37fff700 1 Processor -- start 2023-10-06T03:45:44.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.069+0000 7fbb37fff700 1 -- start start 2023-10-06T03:45:44.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.070+0000 7fbb37fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb38069cf0 0x7fbb38155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:44.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.070+0000 7fbb37fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb380cb560 0x7fbb380b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:44.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.070+0000 7fbb37fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb380ccb40 0x7fbb3814bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:44.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.070+0000 7fbb37fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380c7d00 con 0x7fbb380cb560 2023-10-06T03:45:44.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.070+0000 7fbb37fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380adf30 con 0x7fbb380ccb40 2023-10-06T03:45:44.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.070+0000 7fbb37fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb380ae210 con 0x7fbb38069cf0 2023-10-06T03:45:44.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.070+0000 7fbb367fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb380cb560 0x7fbb380b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:44.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.070+0000 7fbb367fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb380cb560 0x7fbb380b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52946/0 (socket says 172.21.15.55:52946) 2023-10-06T03:45:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.070+0000 7fbb367fc700 1 -- 172.21.15.55:0/2725795250 learned_addr learned my addr 172.21.15.55:0/2725795250 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.070+0000 7fbb377fe700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb380ccb40 0x7fbb3814bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.070+0000 7fbb36ffd700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb38069cf0 0x7fbb38155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.071+0000 7fbb367fc700 1 -- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb38069cf0 msgr2=0x7fbb38155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.071+0000 7fbb367fc700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb38069cf0 0x7fbb38155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.071+0000 7fbb367fc700 1 -- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb380ccb40 msgr2=0x7fbb3814bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.071+0000 7fbb367fc700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb380ccb40 0x7fbb3814bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.071+0000 7fbb367fc700 1 -- 172.21.15.55:0/2725795250 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb30009ae0 con 0x7fbb380cb560 2023-10-06T03:45:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.071+0000 7fbb36ffd700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb38069cf0 0x7fbb38155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:45:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.071+0000 7fbb367fc700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb380cb560 0x7fbb380b0470 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00e8d0 tx=0x7fbb2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:44.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.071+0000 7fbb1effd700 1 -- 172.21.15.55:0/2725795250 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c013680 con 0x7fbb380cb560 2023-10-06T03:45:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.071+0000 7fbb1effd700 1 -- 172.21.15.55:0/2725795250 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c0137e0 con 0x7fbb380cb560 2023-10-06T03:45:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.071+0000 7fbb1effd700 1 -- 172.21.15.55:0/2725795250 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c013ba0 con 0x7fbb380cb560 2023-10-06T03:45:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.071+0000 7fbb37fff700 1 -- 172.21.15.55:0/2725795250 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb380ae4f0 con 0x7fbb380cb560 2023-10-06T03:45:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.071+0000 7fbb37fff700 1 -- 172.21.15.55:0/2725795250 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb38155c80 con 0x7fbb380cb560 2023-10-06T03:45:44.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.072+0000 7fbb37fff700 1 -- 172.21.15.55:0/2725795250 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb38155ea0 con 0x7fbb380cb560 2023-10-06T03:45:44.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.073+0000 7fbb1effd700 1 -- 172.21.15.55:0/2725795250 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fbb2c00c400 con 0x7fbb380cb560 2023-10-06T03:45:44.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.074+0000 7fbb1effd700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbb24067130 0x7fbb240695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:44.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.074+0000 7fbb1effd700 1 -- 172.21.15.55:0/2725795250 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6833+0+0 (secure 0 0 0) 0x7fbb2c084e90 con 0x7fbb380cb560 2023-10-06T03:45:44.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.075+0000 7fbb36ffd700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbb24067130 0x7fbb240695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:44.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.075+0000 7fbb36ffd700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbb24067130 0x7fbb240695e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb28007fd0 tx=0x7fbb28009040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:44.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.078+0000 7fbb1effd700 1 -- 172.21.15.55:0/2725795250 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb2c051970 con 0x7fbb380cb560 2023-10-06T03:45:44.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.313+0000 7fbb37fff700 1 -- 172.21.15.55:0/2725795250 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb380ae990 con 0x7fbb24067130 2023-10-06T03:45:44.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.401+0000 7fbb1effd700 1 -- 172.21.15.55:0/2725795250 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fbb380ae990 con 0x7fbb24067130 2023-10-06T03:45:44.403 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-10-06T03:45:44.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.404+0000 7fbb37fff700 1 -- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbb24067130 msgr2=0x7fbb240695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:44.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.404+0000 7fbb37fff700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbb24067130 0x7fbb240695e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb28007fd0 tx=0x7fbb28009040 comp rx=0 tx=0).stop 2023-10-06T03:45:44.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.404+0000 7fbb37fff700 1 -- 172.21.15.55:0/2725795250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb380cb560 msgr2=0x7fbb380b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:44.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.404+0000 7fbb37fff700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb380cb560 0x7fbb380b0470 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c00e8d0 tx=0x7fbb2c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:44.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.404+0000 7fbb37fff700 1 -- 172.21.15.55:0/2725795250 shutdown_connections 2023-10-06T03:45:44.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.404+0000 7fbb37fff700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb380ccb40 0x7fbb3814bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:44.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.404+0000 7fbb37fff700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbb24067130 0x7fbb240695e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:44.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.404+0000 7fbb37fff700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb38069cf0 0x7fbb38155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:44.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.404+0000 7fbb37fff700 1 --2- 172.21.15.55:0/2725795250 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb380cb560 0x7fbb380b0470 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:44.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.404+0000 7fbb37fff700 1 -- 172.21.15.55:0/2725795250 >> 172.21.15.55:0/2725795250 conn(0x7fbb3806baf0 msgr2=0x7fbb3806ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:44.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.404+0000 7fbb37fff700 1 -- 172.21.15.55:0/2725795250 shutdown_connections 2023-10-06T03:45:44.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:44.404+0000 7fbb37fff700 1 -- 172.21.15.55:0/2725795250 wait complete. 2023-10-06T03:45:44.423 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-10-06T03:45:44.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:44.423 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-06T03:45:44.462 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:44.462 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:44.462 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:44.462 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:44.462 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:44.462 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:44.462 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:44.463 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:44.463 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:44.463 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:44.463 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:44.464 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:44.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:44.464 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_35_00_UTC) 2023-10-06T03:45:44.550 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:44.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:44.550 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_36_00_UTC) 2023-10-06T03:45:44.635 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:44.635 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:44.636 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_37_00_UTC) 2023-10-06T03:45:44.750 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:44.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:44.750 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_38_00_UTC) 2023-10-06T03:45:44.814 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:44.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:44.815 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_39_00_UTC) 2023-10-06T03:45:44.877 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:44.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:44.878 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_40_00_UTC) 2023-10-06T03:45:44.895 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:45:44.921 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:44.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:44.921 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_41_00_UTC) 2023-10-06T03:45:44.965 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:45:45.010 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:45.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:45.010 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_42_00_UTC) 2023-10-06T03:45:45.093 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:45.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:45.094 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_43_00_UTC) 2023-10-06T03:45:45.178 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:45.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:45.178 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_44_00_UTC) 2023-10-06T03:45:45.271 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:45.271 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:45.271 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-06-03_45_00_UTC) 2023-10-06T03:45:45.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:45.315 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-10-06T03:45:45.396 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir9 1M 2023-10-06T03:45:45.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.627+0000 7f092339d700 1 Processor -- start 2023-10-06T03:45:45.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.627+0000 7f092339d700 1 -- start start 2023-10-06T03:45:45.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.629+0000 7f092339d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f091c0ccb40 0x7f091c0697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:45.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.629+0000 7f092339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f091c069cf0 0x7f091c15ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:45.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.629+0000 7f092339d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091c159db0 con 0x7f091c0cb560 2023-10-06T03:45:45.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.629+0000 7f092339d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091c06aa30 con 0x7f091c0ccb40 2023-10-06T03:45:45.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.629+0000 7f092339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c06a470 con 0x7f091c069cf0 2023-10-06T03:45:45.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.629+0000 7f092239b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f091c0cb560 0x7f091c0cc430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42336/0 (socket says 172.21.15.55:42336) 2023-10-06T03:45:45.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.629+0000 7f092239b700 1 -- 172.21.15.55:0/3604509473 learned_addr learned my addr 172.21.15.55:0/3604509473 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:45.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.629+0000 7f0922b9c700 1 --2- 172.21.15.55:0/3604509473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f091c069cf0 0x7f091c15ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:45.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.629+0000 7f0921b9a700 1 --2- 172.21.15.55:0/3604509473 >> v2:172.21.15.163:3300/0 conn(0x7f091c0ccb40 0x7f091c0697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:45.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.629+0000 7f0921399700 1 -- 172.21.15.55:0/3604509473 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511390949 0 0) 0x7f091c159db0 con 0x7f091c0cb560 2023-10-06T03:45:45.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.629+0000 7f0921399700 1 -- 172.21.15.55:0/3604509473 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0908003570 con 0x7f091c0cb560 2023-10-06T03:45:45.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.630+0000 7f0922b9c700 1 -- 172.21.15.55:0/3604509473 >> v2:172.21.15.163:3300/0 conn(0x7f091c0ccb40 msgr2=0x7f091c0697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:45.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.630+0000 7f0922b9c700 1 --2- 172.21.15.55:0/3604509473 >> v2:172.21.15.163:3300/0 conn(0x7f091c0ccb40 0x7f091c0697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:45.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.630+0000 7f0922b9c700 1 -- 172.21.15.55:0/3604509473 >> v1:172.21.15.55:6789/0 conn(0x7f091c0cb560 legacy=0x7f091c0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:45.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.630+0000 7f0922b9c700 1 -- 172.21.15.55:0/3604509473 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c06a750 con 0x7f091c069cf0 2023-10-06T03:45:45.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.630+0000 7f0922b9c700 1 --2- 172.21.15.55:0/3604509473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f091c069cf0 0x7f091c15ba60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0914009e90 tx=0x7f0914009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1cd20593cdb97e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:45.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.630+0000 7f0921399700 1 -- 172.21.15.55:0/3604509473 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091400e070 con 0x7f091c069cf0 2023-10-06T03:45:45.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.630+0000 7f0921399700 1 -- 172.21.15.55:0/3604509473 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091400bb40 con 0x7f091c069cf0 2023-10-06T03:45:45.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.630+0000 7f0921399700 1 -- 172.21.15.55:0/3604509473 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914012430 con 0x7f091c069cf0 2023-10-06T03:45:45.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.631+0000 7f092339d700 1 -- 172.21.15.55:0/3604509473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f091c069cf0 msgr2=0x7f091c15ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:45.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.631+0000 7f092339d700 1 --2- 172.21.15.55:0/3604509473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f091c069cf0 0x7f091c15ba60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0914009e90 tx=0x7f0914009780 comp rx=0 tx=0).stop 2023-10-06T03:45:45.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.631+0000 7f092339d700 1 -- 172.21.15.55:0/3604509473 shutdown_connections 2023-10-06T03:45:45.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.631+0000 7f092339d700 1 --2- 172.21.15.55:0/3604509473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f091c069cf0 0x7f091c15ba60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:45.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.631+0000 7f092339d700 1 --2- 172.21.15.55:0/3604509473 >> v2:172.21.15.163:3300/0 conn(0x7f091c0ccb40 0x7f091c0697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:45.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.631+0000 7f092339d700 1 -- 172.21.15.55:0/3604509473 >> 172.21.15.55:0/3604509473 conn(0x7f091c06baf0 msgr2=0x7f091c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:45.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.631+0000 7f092339d700 1 -- 172.21.15.55:0/3604509473 shutdown_connections 2023-10-06T03:45:45.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.631+0000 7f092339d700 1 -- 172.21.15.55:0/3604509473 wait complete. 2023-10-06T03:45:45.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.632+0000 7f092339d700 1 Processor -- start 2023-10-06T03:45:45.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.632+0000 7f092339d700 1 -- start start 2023-10-06T03:45:45.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.633+0000 7f092339d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f091c069cf0 0x7f091c16a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:45.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.633+0000 7f092339d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f091c0cb560 0x7f091c14f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:45.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.633+0000 7f092339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f091c0ccb40 0x7f091c0c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:45.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.633+0000 7f092339d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0aec40 con 0x7f091c069cf0 2023-10-06T03:45:45.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.633+0000 7f092339d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0ae030 con 0x7f091c0cb560 2023-10-06T03:45:45.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.633+0000 7f092339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c0ae310 con 0x7f091c0ccb40 2023-10-06T03:45:45.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.634+0000 7f092239b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f091c069cf0 0x7f091c16a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:45.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.634+0000 7f092239b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f091c069cf0 0x7f091c16a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52954/0 (socket says 172.21.15.55:52954) 2023-10-06T03:45:45.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.634+0000 7f092239b700 1 -- 172.21.15.55:0/1361398460 learned_addr learned my addr 172.21.15.55:0/1361398460 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:45.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.634+0000 7f0922b9c700 1 --2- 172.21.15.55:0/1361398460 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f091c0ccb40 0x7f091c0c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:45.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.634+0000 7f0921b9a700 1 --2- 172.21.15.55:0/1361398460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f091c0cb560 0x7f091c14f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:45.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.634+0000 7f092239b700 1 -- 172.21.15.55:0/1361398460 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f091c0ccb40 msgr2=0x7f091c0c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:45.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.634+0000 7f092239b700 1 --2- 172.21.15.55:0/1361398460 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f091c0ccb40 0x7f091c0c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:45.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.634+0000 7f092239b700 1 -- 172.21.15.55:0/1361398460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f091c0cb560 msgr2=0x7f091c14f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:45.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.634+0000 7f092239b700 1 --2- 172.21.15.55:0/1361398460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f091c0cb560 0x7f091c14f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:45.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.634+0000 7f092239b700 1 -- 172.21.15.55:0/1361398460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0914009a20 con 0x7f091c069cf0 2023-10-06T03:45:45.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.634+0000 7f092239b700 1 --2- 172.21.15.55:0/1361398460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f091c069cf0 0x7f091c16a8b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f091800c7c0 tx=0x7f0918007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:45.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.634+0000 7f09127fc700 1 -- 172.21.15.55:0/1361398460 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918005b30 con 0x7f091c069cf0 2023-10-06T03:45:45.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.635+0000 7f09127fc700 1 -- 172.21.15.55:0/1361398460 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0918005c90 con 0x7f091c069cf0 2023-10-06T03:45:45.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.635+0000 7f092339d700 1 -- 172.21.15.55:0/1361398460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c0ae5f0 con 0x7f091c069cf0 2023-10-06T03:45:45.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.635+0000 7f09127fc700 1 -- 172.21.15.55:0/1361398460 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918007320 con 0x7f091c069cf0 2023-10-06T03:45:45.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.635+0000 7f092339d700 1 -- 172.21.15.55:0/1361398460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c14f650 con 0x7f091c069cf0 2023-10-06T03:45:45.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.636+0000 7f092339d700 1 -- 172.21.15.55:0/1361398460 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c04c3c0 con 0x7f091c069cf0 2023-10-06T03:45:45.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.637+0000 7f09127fc700 1 -- 172.21.15.55:0/1361398460 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f091801b400 con 0x7f091c069cf0 2023-10-06T03:45:45.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.638+0000 7f09127fc700 1 --2- 172.21.15.55:0/1361398460 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0900067130 0x7f09000695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:45.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.638+0000 7f09127fc700 1 -- 172.21.15.55:0/1361398460 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6881+0+0 (secure 0 0 0) 0x7f0918084800 con 0x7f091c069cf0 2023-10-06T03:45:45.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.638+0000 7f0921b9a700 1 --2- 172.21.15.55:0/1361398460 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0900067130 0x7f09000695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:45.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.639+0000 7f0921b9a700 1 --2- 172.21.15.55:0/1361398460 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0900067130 0x7f09000695e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f091c0c68c0 tx=0x7f090c008040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:45.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.642+0000 7f09127fc700 1 -- 172.21.15.55:0/1361398460 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0918051330 con 0x7f091c069cf0 2023-10-06T03:45:45.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.884+0000 7f092339d700 1 -- 172.21.15.55:0/1361398460 --> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f091c15b260 con 0x7f0900067130 2023-10-06T03:45:45.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.973+0000 7f09127fc700 1 -- 172.21.15.55:0/1361398460 <== mgr.4844 v2:172.21.15.163:6832/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f091c15b260 con 0x7f0900067130 2023-10-06T03:45:45.975 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-10-06T03:45:45.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.976+0000 7f092339d700 1 -- 172.21.15.55:0/1361398460 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0900067130 msgr2=0x7f09000695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:45.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.976+0000 7f092339d700 1 --2- 172.21.15.55:0/1361398460 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0900067130 0x7f09000695e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f091c0c68c0 tx=0x7f090c008040 comp rx=0 tx=0).stop 2023-10-06T03:45:45.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.976+0000 7f092339d700 1 -- 172.21.15.55:0/1361398460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f091c069cf0 msgr2=0x7f091c16a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:45.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.976+0000 7f092339d700 1 --2- 172.21.15.55:0/1361398460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f091c069cf0 0x7f091c16a8b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f091800c7c0 tx=0x7f0918007e00 comp rx=0 tx=0).stop 2023-10-06T03:45:45.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.976+0000 7f092339d700 1 -- 172.21.15.55:0/1361398460 shutdown_connections 2023-10-06T03:45:45.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.976+0000 7f092339d700 1 --2- 172.21.15.55:0/1361398460 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0900067130 0x7f09000695e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:45.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.976+0000 7f092339d700 1 --2- 172.21.15.55:0/1361398460 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f091c0ccb40 0x7f091c0c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:45.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.976+0000 7f092339d700 1 --2- 172.21.15.55:0/1361398460 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f091c069cf0 0x7f091c16a8b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:45.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.976+0000 7f092339d700 1 --2- 172.21.15.55:0/1361398460 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f091c0cb560 0x7f091c14f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:45.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.976+0000 7f092339d700 1 -- 172.21.15.55:0/1361398460 >> 172.21.15.55:0/1361398460 conn(0x7f091c06baf0 msgr2=0x7f091c06d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:45.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.976+0000 7f092339d700 1 -- 172.21.15.55:0/1361398460 shutdown_connections 2023-10-06T03:45:45.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:45.976+0000 7f092339d700 1 -- 172.21.15.55:0/1361398460 wait complete. 2023-10-06T03:45:45.995 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-10-06T03:45:45.995 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:45.995 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-06T03:45:46.033 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:46.033 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:46.034 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:46.034 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:46.034 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:46.034 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:46.034 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:46.034 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:46.034 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:46.034 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:46.034 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:46.035 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_35_00_UTC 2023-10-06T03:45:46.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:46.036 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_35_00_UTC) 2023-10-06T03:45:46.122 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_36_00_UTC 2023-10-06T03:45:46.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:46.122 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_36_00_UTC) 2023-10-06T03:45:46.205 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_37_00_UTC 2023-10-06T03:45:46.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:46.205 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_37_00_UTC) 2023-10-06T03:45:46.289 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_38_00_UTC 2023-10-06T03:45:46.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:46.290 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_38_00_UTC) 2023-10-06T03:45:46.374 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_39_00_UTC 2023-10-06T03:45:46.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:46.374 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_39_00_UTC) 2023-10-06T03:45:46.458 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_40_00_UTC 2023-10-06T03:45:46.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:46.459 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_40_00_UTC) 2023-10-06T03:45:46.543 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_41_00_UTC 2023-10-06T03:45:46.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:46.544 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_41_00_UTC) 2023-10-06T03:45:46.628 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_42_00_UTC 2023-10-06T03:45:46.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:46.628 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_42_00_UTC) 2023-10-06T03:45:46.724 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_43_00_UTC 2023-10-06T03:45:46.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:46.725 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_43_00_UTC) 2023-10-06T03:45:46.768 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_44_00_UTC 2023-10-06T03:45:46.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:46.768 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_44_00_UTC) 2023-10-06T03:45:46.851 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_45_00_UTC 2023-10-06T03:45:46.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:46.851 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-06-03_45_00_UTC) 2023-10-06T03:45:46.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:46.935 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-10-06T03:45:47.015 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T03:45:47.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.279+0000 7fb737fff700 1 Processor -- start 2023-10-06T03:45:47.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.280+0000 7fb737fff700 1 -- start start 2023-10-06T03:45:47.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.281+0000 7fb737fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb738140bb0 0x7fb73806d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:47.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.281+0000 7fb737fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7380c6820 con 0x7fb7381407e0 2023-10-06T03:45:47.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.281+0000 7fb737fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7380c6de0 con 0x7fb73806da30 2023-10-06T03:45:47.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.281+0000 7fb737fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73806deb0 con 0x7fb738140bb0 2023-10-06T03:45:47.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.281+0000 7fb736ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb7381407e0 0x7fb73806a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42350/0 (socket says 172.21.15.55:42350) 2023-10-06T03:45:47.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.281+0000 7fb736ffd700 1 -- 172.21.15.55:0/4187671763 learned_addr learned my addr 172.21.15.55:0/4187671763 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:47.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.281+0000 7fb7367fc700 1 --2- 172.21.15.55:0/4187671763 >> v2:172.21.15.163:3300/0 conn(0x7fb738140bb0 0x7fb73806d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:47.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.282+0000 7fb735ffb700 1 -- 172.21.15.55:0/4187671763 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132189777 0 0) 0x7fb7380c6820 con 0x7fb7381407e0 2023-10-06T03:45:47.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.282+0000 7fb735ffb700 1 -- 172.21.15.55:0/4187671763 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb720003570 con 0x7fb7381407e0 2023-10-06T03:45:47.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.282+0000 7fb735ffb700 1 -- 172.21.15.55:0/4187671763 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432054433 0 0) 0x7fb7380c6de0 con 0x7fb73806da30 2023-10-06T03:45:47.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.282+0000 7fb735ffb700 1 -- 172.21.15.55:0/4187671763 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7380c6820 con 0x7fb73806da30 2023-10-06T03:45:47.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.282+0000 7fb7367fc700 1 -- 172.21.15.55:0/4187671763 >> v1:172.21.15.163:6789/0 conn(0x7fb73806da30 legacy=0x7fb7380c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:47.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.282+0000 7fb7367fc700 1 -- 172.21.15.55:0/4187671763 >> v1:172.21.15.55:6789/0 conn(0x7fb7381407e0 legacy=0x7fb73806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:47.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.282+0000 7fb7367fc700 1 -- 172.21.15.55:0/4187671763 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb738141270 con 0x7fb738140bb0 2023-10-06T03:45:47.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.282+0000 7fb7367fc700 1 --2- 172.21.15.55:0/4187671763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb738140bb0 0x7fb73806d4f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0098e0 tx=0x7fb72c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adb26fa484b83269 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:47.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.283+0000 7fb735ffb700 1 -- 172.21.15.55:0/4187671763 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c011630 con 0x7fb738140bb0 2023-10-06T03:45:47.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.283+0000 7fb735ffb700 1 -- 172.21.15.55:0/4187671763 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c011790 con 0x7fb738140bb0 2023-10-06T03:45:47.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.283+0000 7fb735ffb700 1 -- 172.21.15.55:0/4187671763 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c011b30 con 0x7fb738140bb0 2023-10-06T03:45:47.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.283+0000 7fb737fff700 1 -- 172.21.15.55:0/4187671763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb738140bb0 msgr2=0x7fb73806d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:47.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.283+0000 7fb737fff700 1 --2- 172.21.15.55:0/4187671763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb738140bb0 0x7fb73806d4f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0098e0 tx=0x7fb72c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:47.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.283+0000 7fb737fff700 1 -- 172.21.15.55:0/4187671763 shutdown_connections 2023-10-06T03:45:47.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.283+0000 7fb737fff700 1 --2- 172.21.15.55:0/4187671763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb738140bb0 0x7fb73806d4f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:47.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.283+0000 7fb737fff700 1 -- 172.21.15.55:0/4187671763 >> 172.21.15.55:0/4187671763 conn(0x7fb7380cc1e0 msgr2=0x7fb7380cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:47.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.283+0000 7fb737fff700 1 -- 172.21.15.55:0/4187671763 shutdown_connections 2023-10-06T03:45:47.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.283+0000 7fb737fff700 1 -- 172.21.15.55:0/4187671763 wait complete. 2023-10-06T03:45:47.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.285+0000 7fb737fff700 1 Processor -- start 2023-10-06T03:45:47.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.285+0000 7fb737fff700 1 -- start start 2023-10-06T03:45:47.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb737fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb73806da30 0x7fb738167410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:47.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb737fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7381407e0 0x7fb7381325a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:47.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb737fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb738140bb0 0x7fb7380a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:47.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb737fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73813d910 con 0x7fb738140bb0 2023-10-06T03:45:47.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb737fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73813ff90 con 0x7fb7381407e0 2023-10-06T03:45:47.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb737fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73813ccd0 con 0x7fb73806da30 2023-10-06T03:45:47.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb7377fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb738140bb0 0x7fb7380a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:47.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb7377fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb738140bb0 0x7fb7380a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52960/0 (socket says 172.21.15.55:52960) 2023-10-06T03:45:47.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb7377fe700 1 -- 172.21.15.55:0/847349785 learned_addr learned my addr 172.21.15.55:0/847349785 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:47.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb736ffd700 1 --2- 172.21.15.55:0/847349785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb73806da30 0x7fb738167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:47.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb7367fc700 1 --2- 172.21.15.55:0/847349785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7381407e0 0x7fb7381325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:47.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb7377fe700 1 -- 172.21.15.55:0/847349785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb73806da30 msgr2=0x7fb738167410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:47.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb7377fe700 1 --2- 172.21.15.55:0/847349785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb73806da30 0x7fb738167410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:47.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb7377fe700 1 -- 172.21.15.55:0/847349785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7381407e0 msgr2=0x7fb7381325a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:47.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb7377fe700 1 --2- 172.21.15.55:0/847349785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7381407e0 0x7fb7381325a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:47.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.286+0000 7fb7377fe700 1 -- 172.21.15.55:0/847349785 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c009420 con 0x7fb738140bb0 2023-10-06T03:45:47.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.287+0000 7fb7377fe700 1 --2- 172.21.15.55:0/847349785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb738140bb0 0x7fb7380a6f00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb73000c910 tx=0x7fb7300027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:47.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.287+0000 7fb71effd700 1 -- 172.21.15.55:0/847349785 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb730012630 con 0x7fb738140bb0 2023-10-06T03:45:47.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.287+0000 7fb71effd700 1 -- 172.21.15.55:0/847349785 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7300127b0 con 0x7fb738140bb0 2023-10-06T03:45:47.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.287+0000 7fb737fff700 1 -- 172.21.15.55:0/847349785 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb738167950 con 0x7fb738140bb0 2023-10-06T03:45:47.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.287+0000 7fb71effd700 1 -- 172.21.15.55:0/847349785 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb730012b70 con 0x7fb738140bb0 2023-10-06T03:45:47.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.287+0000 7fb737fff700 1 -- 172.21.15.55:0/847349785 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73813d250 con 0x7fb738140bb0 2023-10-06T03:45:47.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.288+0000 7fb737fff700 1 -- 172.21.15.55:0/847349785 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73813da50 con 0x7fb738140bb0 2023-10-06T03:45:47.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.289+0000 7fb71effd700 1 -- 172.21.15.55:0/847349785 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 12) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fb73001b400 con 0x7fb738140bb0 2023-10-06T03:45:47.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.290+0000 7fb71effd700 1 --2- 172.21.15.55:0/847349785 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb724067130 0x7fb7240695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:47.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.290+0000 7fb71effd700 1 -- 172.21.15.55:0/847349785 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6881+0+0 (secure 0 0 0) 0x7fb7300850e0 con 0x7fb738140bb0 2023-10-06T03:45:47.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.290+0000 7fb736ffd700 1 --2- 172.21.15.55:0/847349785 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb724067130 0x7fb7240695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:47.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.291+0000 7fb736ffd700 1 --2- 172.21.15.55:0/847349785 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb724067130 0x7fb7240695e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb728007fd0 tx=0x7fb728009040 comp rx=0 tx=0).ready entity=mgr.4844 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:47.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.294+0000 7fb71effd700 1 -- 172.21.15.55:0/847349785 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb73813da50 con 0x7fb738140bb0 2023-10-06T03:45:47.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.551+0000 7fb737fff700 1 -- 172.21.15.55:0/847349785 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb73813da50 con 0x7fb738140bb0 2023-10-06T03:45:47.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.702+0000 7fb71effd700 1 -- 172.21.15.55:0/847349785 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 13) v1 ==== 84911+0+0 (secure 0 0 0) 0x7fb730051f60 con 0x7fb738140bb0 2023-10-06T03:45:47.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.704+0000 7fb71effd700 1 -- 172.21.15.55:0/847349785 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v13) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb730051b90 con 0x7fb738140bb0 2023-10-06T03:45:47.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.708+0000 7fb737fff700 1 -- 172.21.15.55:0/847349785 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb724067130 msgr2=0x7fb7240695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:47.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.708+0000 7fb737fff700 1 --2- 172.21.15.55:0/847349785 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb724067130 0x7fb7240695e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb728007fd0 tx=0x7fb728009040 comp rx=0 tx=0).stop 2023-10-06T03:45:47.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.708+0000 7fb737fff700 1 -- 172.21.15.55:0/847349785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb738140bb0 msgr2=0x7fb7380a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:47.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.708+0000 7fb737fff700 1 --2- 172.21.15.55:0/847349785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb738140bb0 0x7fb7380a6f00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb73000c910 tx=0x7fb7300027a0 comp rx=0 tx=0).stop 2023-10-06T03:45:47.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.708+0000 7fb737fff700 1 -- 172.21.15.55:0/847349785 shutdown_connections 2023-10-06T03:45:47.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.708+0000 7fb737fff700 1 --2- 172.21.15.55:0/847349785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb738140bb0 0x7fb7380a6f00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:47.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.708+0000 7fb737fff700 1 --2- 172.21.15.55:0/847349785 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb724067130 0x7fb7240695e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:47.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.708+0000 7fb737fff700 1 --2- 172.21.15.55:0/847349785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb73806da30 0x7fb738167410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:47.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.708+0000 7fb737fff700 1 --2- 172.21.15.55:0/847349785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7381407e0 0x7fb7381325a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:47.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.708+0000 7fb737fff700 1 -- 172.21.15.55:0/847349785 >> 172.21.15.55:0/847349785 conn(0x7fb7380cc1e0 msgr2=0x7fb7380cc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:47.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.709+0000 7fb737fff700 1 -- 172.21.15.55:0/847349785 shutdown_connections 2023-10-06T03:45:47.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:47.709+0000 7fb737fff700 1 -- 172.21.15.55:0/847349785 wait complete. 2023-10-06T03:45:47.732 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:45:47.732 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:45:47.814 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:45:47.899 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:45:47.899 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:45:47.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:45:47.899 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:45:47.994 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T03:45:47.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:45:47.994 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:45:48.037 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:45:48.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.268+0000 7fa369f9d700 1 Processor -- start 2023-10-06T03:45:48.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.268+0000 7fa369f9d700 1 -- start start 2023-10-06T03:45:48.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.269+0000 7fa369f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3641597f0 0x7fa3640b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:48.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.269+0000 7fa369f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa36415d230 0x7fa36415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:48.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.269+0000 7fa369f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36415dbd0 con 0x7fa364159420 2023-10-06T03:45:48.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.269+0000 7fa369f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3640b11d0 con 0x7fa36415d230 2023-10-06T03:45:48.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.269+0000 7fa369f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3640b14b0 con 0x7fa3641597f0 2023-10-06T03:45:48.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.270+0000 7fa368f9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa364159420 0x7fa3640b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42352/0 (socket says 172.21.15.55:42352) 2023-10-06T03:45:48.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.270+0000 7fa368f9b700 1 -- 172.21.15.55:0/1710365147 learned_addr learned my addr 172.21.15.55:0/1710365147 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:48.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.270+0000 7fa36979c700 1 --2- 172.21.15.55:0/1710365147 >> v2:172.21.15.55:3300/0 conn(0x7fa36415d230 0x7fa36415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:48.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.270+0000 7fa363fff700 1 --2- 172.21.15.55:0/1710365147 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3641597f0 0x7fa3640b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:48.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.270+0000 7fa3637fe700 1 -- 172.21.15.55:0/1710365147 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417556305 0 0) 0x7fa358001110 con 0x7fa364159420 2023-10-06T03:45:48.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.270+0000 7fa3637fe700 1 -- 172.21.15.55:0/1710365147 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa354003570 con 0x7fa364159420 2023-10-06T03:45:48.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.270+0000 7fa36979c700 1 -- 172.21.15.55:0/1710365147 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3641597f0 msgr2=0x7fa3640b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:48.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.270+0000 7fa36979c700 1 --2- 172.21.15.55:0/1710365147 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3641597f0 0x7fa3640b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.270+0000 7fa36979c700 1 -- 172.21.15.55:0/1710365147 >> v1:172.21.15.55:6789/0 conn(0x7fa364159420 legacy=0x7fa3640b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:48.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.270+0000 7fa36979c700 1 -- 172.21.15.55:0/1710365147 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3640b1790 con 0x7fa36415d230 2023-10-06T03:45:48.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.270+0000 7fa36979c700 1 --2- 172.21.15.55:0/1710365147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa36415d230 0x7fa36415d690 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009e60 tx=0x7fa35c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7f799eb618a8a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:48.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.271+0000 7fa3637fe700 1 -- 172.21.15.55:0/1710365147 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00e070 con 0x7fa36415d230 2023-10-06T03:45:48.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.271+0000 7fa3637fe700 1 -- 172.21.15.55:0/1710365147 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c00bab0 con 0x7fa36415d230 2023-10-06T03:45:48.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.271+0000 7fa3637fe700 1 -- 172.21.15.55:0/1710365147 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00be70 con 0x7fa36415d230 2023-10-06T03:45:48.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.271+0000 7fa369f9d700 1 -- 172.21.15.55:0/1710365147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa36415d230 msgr2=0x7fa36415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:48.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.271+0000 7fa369f9d700 1 --2- 172.21.15.55:0/1710365147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa36415d230 0x7fa36415d690 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009e60 tx=0x7fa35c009780 comp rx=0 tx=0).stop 2023-10-06T03:45:48.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.271+0000 7fa369f9d700 1 -- 172.21.15.55:0/1710365147 shutdown_connections 2023-10-06T03:45:48.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.271+0000 7fa369f9d700 1 --2- 172.21.15.55:0/1710365147 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa36415d230 0x7fa36415d690 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.271+0000 7fa369f9d700 1 --2- 172.21.15.55:0/1710365147 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3641597f0 0x7fa3640b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.271+0000 7fa369f9d700 1 -- 172.21.15.55:0/1710365147 >> 172.21.15.55:0/1710365147 conn(0x7fa3640d0660 msgr2=0x7fa3640d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:48.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.272+0000 7fa369f9d700 1 -- 172.21.15.55:0/1710365147 shutdown_connections 2023-10-06T03:45:48.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.272+0000 7fa369f9d700 1 -- 172.21.15.55:0/1710365147 wait complete. 2023-10-06T03:45:48.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.273+0000 7fa369f9d700 1 Processor -- start 2023-10-06T03:45:48.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.273+0000 7fa369f9d700 1 -- start start 2023-10-06T03:45:48.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.274+0000 7fa369f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa364159420 0x7fa364151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:48.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.274+0000 7fa369f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3641597f0 0x7fa364159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:48.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.274+0000 7fa369f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa36415d230 0x7fa3640b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:48.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.274+0000 7fa369f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36415b250 con 0x7fa364159420 2023-10-06T03:45:48.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.274+0000 7fa369f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36415a5b0 con 0x7fa36415d230 2023-10-06T03:45:48.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.274+0000 7fa369f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36415a890 con 0x7fa3641597f0 2023-10-06T03:45:48.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.274+0000 7fa368f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa364159420 0x7fa364151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:48.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.274+0000 7fa368f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa364159420 0x7fa364151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52982/0 (socket says 172.21.15.55:52982) 2023-10-06T03:45:48.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.274+0000 7fa368f9b700 1 -- 172.21.15.55:0/2988472303 learned_addr learned my addr 172.21.15.55:0/2988472303 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:48.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.274+0000 7fa36979c700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa36415d230 0x7fa3640b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:48.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.274+0000 7fa363fff700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3641597f0 0x7fa364159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:48.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.275+0000 7fa368f9b700 1 -- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3641597f0 msgr2=0x7fa364159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:48.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.275+0000 7fa368f9b700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3641597f0 0x7fa364159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.275+0000 7fa368f9b700 1 -- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa36415d230 msgr2=0x7fa3640b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:48.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.275+0000 7fa368f9b700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa36415d230 0x7fa3640b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.275+0000 7fa368f9b700 1 -- 172.21.15.55:0/2988472303 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c009a20 con 0x7fa364159420 2023-10-06T03:45:48.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.275+0000 7fa363fff700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3641597f0 0x7fa364159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:45:48.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.275+0000 7fa368f9b700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa364159420 0x7fa364151d40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa35800c7c0 tx=0x7fa358007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:48.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.275+0000 7fa360ff9700 1 -- 172.21.15.55:0/2988472303 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358005b30 con 0x7fa364159420 2023-10-06T03:45:48.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.275+0000 7fa360ff9700 1 -- 172.21.15.55:0/2988472303 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa358005cb0 con 0x7fa364159420 2023-10-06T03:45:48.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.275+0000 7fa360ff9700 1 -- 172.21.15.55:0/2988472303 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358007320 con 0x7fa364159420 2023-10-06T03:45:48.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.275+0000 7fa369f9d700 1 -- 172.21.15.55:0/2988472303 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36415ab70 con 0x7fa364159420 2023-10-06T03:45:48.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.276+0000 7fa369f9d700 1 -- 172.21.15.55:0/2988472303 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa364152280 con 0x7fa364159420 2023-10-06T03:45:48.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.277+0000 7fa369f9d700 1 -- 172.21.15.55:0/2988472303 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36404c3c0 con 0x7fa364159420 2023-10-06T03:45:48.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.278+0000 7fa360ff9700 1 -- 172.21.15.55:0/2988472303 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) v1 ==== 84911+0+0 (secure 0 0 0) 0x7fa35801b3b0 con 0x7fa364159420 2023-10-06T03:45:48.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.278+0000 7fa360ff9700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa3480670e0 0x7fa348069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:48.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.279+0000 7fa360ff9700 1 -- 172.21.15.55:0/2988472303 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6833+0+0 (secure 0 0 0) 0x7fa3580840f0 con 0x7fa364159420 2023-10-06T03:45:48.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.279+0000 7fa363fff700 1 -- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa3480670e0 msgr2=0x7fa348069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:48.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.279+0000 7fa363fff700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa3480670e0 0x7fa348069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:45:48.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.283+0000 7fa360ff9700 1 -- 172.21.15.55:0/2988472303 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa358050bd0 con 0x7fa364159420 2023-10-06T03:45:48.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.480+0000 7fa363fff700 1 -- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa3480670e0 msgr2=0x7fa348069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:48.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.480+0000 7fa363fff700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa3480670e0 0x7fa348069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:45:48.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.552+0000 7fa369f9d700 1 -- 172.21.15.55:0/2988472303 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa36415dbd0 con 0x7fa364159420 2023-10-06T03:45:48.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.553+0000 7fa360ff9700 1 -- 172.21.15.55:0/2988472303 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fa36415dbd0 con 0x7fa364159420 2023-10-06T03:45:48.554 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:48.555 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1532010025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1532010025},{"type":"v1","addr":"172.21.15.55:6834","nonce":1532010025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4258042685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":4258042685},{"type":"v1","addr":"172.21.15.55:6835","nonce":4258042685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/430119478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":430119478},{"type":"v1","addr":"172.21.15.163:6836","nonce":430119478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:34:13.003040+0000","modified":"2023-10-06T03:34:16.548740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4903},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4903":{"gid":4903,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6837/1606384901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1606384901},{"type":"v1","addr":"172.21.15.163:6837","nonce":1606384901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T03:45:48.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.555+0000 7fa369f9d700 1 -- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa3480670e0 msgr2=0x7fa348069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:45:48.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.555+0000 7fa369f9d700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa3480670e0 0x7fa348069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.555+0000 7fa369f9d700 1 -- 172.21.15.55:0/2988472303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa364159420 msgr2=0x7fa364151d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:48.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.555+0000 7fa369f9d700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa364159420 0x7fa364151d40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa35800c7c0 tx=0x7fa358007c70 comp rx=0 tx=0).stop 2023-10-06T03:45:48.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.555+0000 7fa369f9d700 1 -- 172.21.15.55:0/2988472303 shutdown_connections 2023-10-06T03:45:48.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.555+0000 7fa369f9d700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa36415d230 0x7fa3640b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.555+0000 7fa369f9d700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa364159420 0x7fa364151d40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.555+0000 7fa369f9d700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fa3480670e0 0x7fa348069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.555+0000 7fa369f9d700 1 --2- 172.21.15.55:0/2988472303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa3641597f0 0x7fa364159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.556+0000 7fa369f9d700 1 -- 172.21.15.55:0/2988472303 >> 172.21.15.55:0/2988472303 conn(0x7fa3640d0660 msgr2=0x7fa364157220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:48.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.556+0000 7fa369f9d700 1 -- 172.21.15.55:0/2988472303 shutdown_connections 2023-10-06T03:45:48.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.556+0000 7fa369f9d700 1 -- 172.21.15.55:0/2988472303 wait complete. 2023-10-06T03:45:48.558 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 17 2023-10-06T03:45:48.577 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:45:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.787+0000 7ff9affff700 1 Processor -- start 2023-10-06T03:45:48.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.787+0000 7ff9affff700 1 -- start start 2023-10-06T03:45:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.788+0000 7ff9affff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9b0159420 0x7ff9b00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.788+0000 7ff9affff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff9b01597f0 0x7ff9b00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.788+0000 7ff9affff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff9b015d230 0x7ff9b015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.788+0000 7ff9affff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b015dbd0 con 0x7ff9b01597f0 2023-10-06T03:45:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.788+0000 7ff9affff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b00b11d0 con 0x7ff9b015d230 2023-10-06T03:45:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.788+0000 7ff9affff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00b14b0 con 0x7ff9b0159420 2023-10-06T03:45:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.788+0000 7ff9ae7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff9b01597f0 0x7ff9b00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.788+0000 7ff9ae7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff9b01597f0 0x7ff9b00b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52984/0 (socket says 172.21.15.55:52984) 2023-10-06T03:45:48.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.788+0000 7ff9ae7fc700 1 -- 172.21.15.55:0/2512891445 learned_addr learned my addr 172.21.15.55:0/2512891445 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.788+0000 7ff9af7fe700 1 --2- 172.21.15.55:0/2512891445 >> v2:172.21.15.163:3300/0 conn(0x7ff9b015d230 0x7ff9b015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.789+0000 7ff9aeffd700 1 --2- 172.21.15.55:0/2512891445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9b0159420 0x7ff9b00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.789+0000 7ff9ae7fc700 1 -- 172.21.15.55:0/2512891445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9b0159420 msgr2=0x7ff9b00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.789+0000 7ff9ae7fc700 1 --2- 172.21.15.55:0/2512891445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9b0159420 0x7ff9b00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.789+0000 7ff9ae7fc700 1 -- 172.21.15.55:0/2512891445 >> v2:172.21.15.163:3300/0 conn(0x7ff9b015d230 msgr2=0x7ff9b015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.789+0000 7ff9ae7fc700 1 --2- 172.21.15.55:0/2512891445 >> v2:172.21.15.163:3300/0 conn(0x7ff9b015d230 0x7ff9b015d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.789+0000 7ff9ae7fc700 1 -- 172.21.15.55:0/2512891445 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b00b1790 con 0x7ff9b01597f0 2023-10-06T03:45:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.789+0000 7ff9af7fe700 1 --2- 172.21.15.55:0/2512891445 >> v2:172.21.15.163:3300/0 conn(0x7ff9b015d230 0x7ff9b015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:45:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.789+0000 7ff9ae7fc700 1 --2- 172.21.15.55:0/2512891445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b01597f0 0x7ff9b00b2270 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00d1720 tx=0x7ff9a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf41aa4a8333695 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.790+0000 7ff9adffb700 1 -- 172.21.15.55:0/2512891445 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a400b640 con 0x7ff9b01597f0 2023-10-06T03:45:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.790+0000 7ff9adffb700 1 -- 172.21.15.55:0/2512891445 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a400b7a0 con 0x7ff9b01597f0 2023-10-06T03:45:48.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.790+0000 7ff9adffb700 1 -- 172.21.15.55:0/2512891445 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a400bb60 con 0x7ff9b01597f0 2023-10-06T03:45:48.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.790+0000 7ff9affff700 1 -- 172.21.15.55:0/2512891445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b01597f0 msgr2=0x7ff9b00b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.790+0000 7ff9affff700 1 --2- 172.21.15.55:0/2512891445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b01597f0 0x7ff9b00b2270 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00d1720 tx=0x7ff9a4009250 comp rx=0 tx=0).stop 2023-10-06T03:45:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.790+0000 7ff9affff700 1 -- 172.21.15.55:0/2512891445 shutdown_connections 2023-10-06T03:45:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.790+0000 7ff9affff700 1 --2- 172.21.15.55:0/2512891445 >> v2:172.21.15.163:3300/0 conn(0x7ff9b015d230 0x7ff9b015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.790+0000 7ff9affff700 1 --2- 172.21.15.55:0/2512891445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9b0159420 0x7ff9b00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.790+0000 7ff9affff700 1 --2- 172.21.15.55:0/2512891445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b01597f0 0x7ff9b00b2270 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.790+0000 7ff9affff700 1 -- 172.21.15.55:0/2512891445 >> 172.21.15.55:0/2512891445 conn(0x7ff9b00d0660 msgr2=0x7ff9b00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.790+0000 7ff9affff700 1 -- 172.21.15.55:0/2512891445 shutdown_connections 2023-10-06T03:45:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.790+0000 7ff9affff700 1 -- 172.21.15.55:0/2512891445 wait complete. 2023-10-06T03:45:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.792+0000 7ff9affff700 1 Processor -- start 2023-10-06T03:45:48.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.792+0000 7ff9affff700 1 -- start start 2023-10-06T03:45:48.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.793+0000 7ff9affff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9b0159420 0x7ff9b0150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:48.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.793+0000 7ff9affff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b01597f0 0x7ff9b0151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:48.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.793+0000 7ff9affff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9b015d230 0x7ff9b014d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:48.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.793+0000 7ff9affff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b0151950 con 0x7ff9b01597f0 2023-10-06T03:45:48.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.793+0000 7ff9affff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b0151b80 con 0x7ff9b0159420 2023-10-06T03:45:48.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.793+0000 7ff9affff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b0151e60 con 0x7ff9b015d230 2023-10-06T03:45:48.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.793+0000 7ff9ae7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b01597f0 0x7ff9b0151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:48.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.793+0000 7ff9ae7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b01597f0 0x7ff9b0151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52988/0 (socket says 172.21.15.55:52988) 2023-10-06T03:45:48.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.793+0000 7ff9ae7fc700 1 -- 172.21.15.55:0/3593924819 learned_addr learned my addr 172.21.15.55:0/3593924819 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:48.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.793+0000 7ff9aeffd700 1 --2- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9b0159420 0x7ff9b0150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:48.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.793+0000 7ff9af7fe700 1 --2- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9b015d230 0x7ff9b014d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:48.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.794+0000 7ff9ae7fc700 1 -- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9b015d230 msgr2=0x7ff9b014d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:48.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.794+0000 7ff9ae7fc700 1 --2- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9b015d230 0x7ff9b014d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.794+0000 7ff9ae7fc700 1 -- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9b0159420 msgr2=0x7ff9b0150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:48.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.794+0000 7ff9ae7fc700 1 --2- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9b0159420 0x7ff9b0150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:48.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.794+0000 7ff9ae7fc700 1 -- 172.21.15.55:0/3593924819 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a8009a20 con 0x7ff9b01597f0 2023-10-06T03:45:48.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.794+0000 7ff9ae7fc700 1 --2- 172.21.15.55:0/3593924819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b01597f0 0x7ff9b0151410 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0089340 tx=0x7ff9a4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:48.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.794+0000 7ff996ffd700 1 -- 172.21.15.55:0/3593924819 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4000d10 con 0x7ff9b01597f0 2023-10-06T03:45:48.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.794+0000 7ff996ffd700 1 -- 172.21.15.55:0/3593924819 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a400bb20 con 0x7ff9b01597f0 2023-10-06T03:45:48.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.794+0000 7ff9affff700 1 -- 172.21.15.55:0/3593924819 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a40094f0 con 0x7ff9b01597f0 2023-10-06T03:45:48.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.794+0000 7ff996ffd700 1 -- 172.21.15.55:0/3593924819 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4011670 con 0x7ff9b01597f0 2023-10-06T03:45:48.799 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.795+0000 7ff9affff700 1 -- 172.21.15.55:0/3593924819 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b0152420 con 0x7ff9b01597f0 2023-10-06T03:45:48.799 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.796+0000 7ff9affff700 1 -- 172.21.15.55:0/3593924819 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b004c3c0 con 0x7ff9b01597f0 2023-10-06T03:45:48.800 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.796+0000 7ff996ffd700 1 -- 172.21.15.55:0/3593924819 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) v1 ==== 84911+0+0 (secure 0 0 0) 0x7ff9a40119e0 con 0x7ff9b01597f0 2023-10-06T03:45:48.800 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.798+0000 7ff996ffd700 1 --2- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff99c0670e0 0x7ff99c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:48.800 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.798+0000 7ff996ffd700 1 -- 172.21.15.55:0/3593924819 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6801+0+0 (secure 0 0 0) 0x7ff9a4086530 con 0x7ff9b01597f0 2023-10-06T03:45:48.800 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.798+0000 7ff9aeffd700 1 -- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff99c0670e0 msgr2=0x7ff99c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:48.800 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.798+0000 7ff9aeffd700 1 --2- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff99c0670e0 0x7ff99c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:45:48.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.803+0000 7ff996ffd700 1 -- 172.21.15.55:0/3593924819 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9a4053030 con 0x7ff9b01597f0 2023-10-06T03:45:49.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.999+0000 7ff9aeffd700 1 -- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff99c0670e0 msgr2=0x7ff99c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:49.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:48.999+0000 7ff9aeffd700 1 --2- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff99c0670e0 0x7ff99c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:45:49.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.006+0000 7ff9affff700 1 -- 172.21.15.55:0/3593924819 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9b00b1a70 con 0x7ff9b01597f0 2023-10-06T03:45:49.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.007+0000 7ff996ffd700 1 -- 172.21.15.55:0/3593924819 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18049 (secure 0 0 0) 0x7ff9a4056650 con 0x7ff9b01597f0 2023-10-06T03:45:49.009 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:49.009 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":50,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:45:48.685718+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:34:11.006235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T03:34:11.987296+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":50,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6832/85191":"2023-10-07T03:34:33.631661+0000","172.21.15.163:0/2869540037":"2023-10-07T03:34:33.631661+0000","172.21.15.163:0/2697678523":"2023-10-07T03:34:33.631661+0000","172.21.15.163:0/65446995":"2023-10-07T03:34:33.631661+0000","172.21.15.163:6833/85191":"2023-10-07T03:34:33.631661+0000","172.21.15.163:0/2882606989":"2023-10-07T03:34:33.631661+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":5,"length":3},{"begin":10,"length":5},{"begin":17,"length":3},{"begin":24,"length":3},{"begin":29,"length":3},{"begin":34,"length":3},{"begin":39,"length":4},{"begin":45,"length":3},{"begin":50,"length":4},{"begin":56,"length":3},{"begin":61,"length":4},{"begin":67,"length":3},{"begin":73,"length":3},{"begin":78,"length":3},{"begin":83,"length":3},{"begin":88,"length":3},{"begin":94,"length":3},{"begin":99,"length":3},{"begin":103,"length":3},{"begin":109,"length":3},{"begin":155,"length":66}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":64,"length":1},{"begin":75,"length":1},{"begin":85,"length":1},{"begin":96,"length":1},{"begin":217,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:45:49.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.009+0000 7ff9affff700 1 -- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff99c0670e0 msgr2=0x7ff99c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:45:49.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.009+0000 7ff9affff700 1 --2- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff99c0670e0 0x7ff99c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.009+0000 7ff9affff700 1 -- 172.21.15.55:0/3593924819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b01597f0 msgr2=0x7ff9b0151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:49.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.009+0000 7ff9affff700 1 --2- 172.21.15.55:0/3593924819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b01597f0 0x7ff9b0151410 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0089340 tx=0x7ff9a4017000 comp rx=0 tx=0).stop 2023-10-06T03:45:49.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.009+0000 7ff9affff700 1 -- 172.21.15.55:0/3593924819 shutdown_connections 2023-10-06T03:45:49.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.009+0000 7ff9affff700 1 --2- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff99c0670e0 0x7ff99c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.009+0000 7ff9affff700 1 --2- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff9b015d230 0x7ff9b014d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.009+0000 7ff9affff700 1 --2- 172.21.15.55:0/3593924819 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff9b0159420 0x7ff9b0150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.009+0000 7ff9affff700 1 --2- 172.21.15.55:0/3593924819 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff9b01597f0 0x7ff9b0151410 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.009+0000 7ff9affff700 1 -- 172.21.15.55:0/3593924819 >> 172.21.15.55:0/3593924819 conn(0x7ff9b00d0660 msgr2=0x7ff9b00d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.010+0000 7ff9affff700 1 -- 172.21.15.55:0/3593924819 shutdown_connections 2023-10-06T03:45:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.010+0000 7ff9affff700 1 -- 172.21.15.55:0/3593924819 wait complete. 2023-10-06T03:45:49.029 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:34:11.006235+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:34:11.987296+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 50, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:45:49.030 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:45:49.055 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:49.052+0000 7f911da9b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:45:49.118 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:49.116+0000 7f71e5a54180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:45:49.218 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:49.216+0000 7f911da9b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:45:49.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.222+0000 7f0b34b9d700 1 Processor -- start 2023-10-06T03:45:49.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.222+0000 7f0b34b9d700 1 -- start start 2023-10-06T03:45:49.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.223+0000 7f0b34b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0b300b1b40 0x7f0b300b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:49.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.223+0000 7f0b34b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b300cfb00 con 0x7f0b300b4e70 2023-10-06T03:45:49.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.223+0000 7f0b34b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b300b5c40 con 0x7f0b300b16d0 2023-10-06T03:45:49.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.223+0000 7f0b34b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b300b5f20 con 0x7f0b300b1b40 2023-10-06T03:45:49.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.224+0000 7f0b2f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0b300b4e70 0x7f0b300ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42356/0 (socket says 172.21.15.55:42356) 2023-10-06T03:45:49.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.224+0000 7f0b2f7fe700 1 -- 172.21.15.55:0/1175846599 learned_addr learned my addr 172.21.15.55:0/1175846599 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:49.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.224+0000 7f0b2ffff700 1 --2- 172.21.15.55:0/1175846599 >> v2:172.21.15.163:3300/0 conn(0x7f0b300b1b40 0x7f0b300b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:49.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.224+0000 7f0b2e7fc700 1 -- 172.21.15.55:0/1175846599 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550612505 0 0) 0x7f0b300cfb00 con 0x7f0b300b4e70 2023-10-06T03:45:49.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.224+0000 7f0b2e7fc700 1 -- 172.21.15.55:0/1175846599 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b18003570 con 0x7f0b300b4e70 2023-10-06T03:45:49.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.224+0000 7f0b2ffff700 1 -- 172.21.15.55:0/1175846599 >> v1:172.21.15.163:6789/0 conn(0x7f0b300b16d0 legacy=0x7f0b300b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:49.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.224+0000 7f0b2ffff700 1 -- 172.21.15.55:0/1175846599 >> v1:172.21.15.55:6789/0 conn(0x7f0b300b4e70 legacy=0x7f0b300ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:49.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.224+0000 7f0b2ffff700 1 -- 172.21.15.55:0/1175846599 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b300b6200 con 0x7f0b300b1b40 2023-10-06T03:45:49.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.225+0000 7f0b2ffff700 1 --2- 172.21.15.55:0/1175846599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b300b1b40 0x7f0b300b3ac0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009de0 tx=0x7f0b28009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ef1f4f0a2493698 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:49.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.225+0000 7f0b2e7fc700 1 -- 172.21.15.55:0/1175846599 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2800e070 con 0x7f0b300b1b40 2023-10-06T03:45:49.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.226+0000 7f0b2e7fc700 1 -- 172.21.15.55:0/1175846599 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2800bae0 con 0x7f0b300b1b40 2023-10-06T03:45:49.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.226+0000 7f0b2e7fc700 1 -- 172.21.15.55:0/1175846599 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2800be80 con 0x7f0b300b1b40 2023-10-06T03:45:49.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.226+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1175846599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b300b1b40 msgr2=0x7f0b300b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:49.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.226+0000 7f0b34b9d700 1 --2- 172.21.15.55:0/1175846599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b300b1b40 0x7f0b300b3ac0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009de0 tx=0x7f0b28009780 comp rx=0 tx=0).stop 2023-10-06T03:45:49.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.226+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1175846599 shutdown_connections 2023-10-06T03:45:49.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.226+0000 7f0b34b9d700 1 --2- 172.21.15.55:0/1175846599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b300b1b40 0x7f0b300b3ac0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.226+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1175846599 >> 172.21.15.55:0/1175846599 conn(0x7f0b300d10c0 msgr2=0x7f0b300affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:49.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.226+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1175846599 shutdown_connections 2023-10-06T03:45:49.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.226+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1175846599 wait complete. 2023-10-06T03:45:49.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.228+0000 7f0b34b9d700 1 Processor -- start 2023-10-06T03:45:49.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.228+0000 7f0b34b9d700 1 -- start start 2023-10-06T03:45:49.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.229+0000 7f0b34b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b300b16d0 0x7f0b3015c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:49.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.229+0000 7f0b34b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300b1b40 0x7f0b3016cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:49.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.229+0000 7f0b34b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b300b4e70 0x7f0b301574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:49.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.229+0000 7f0b34b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3015cb90 con 0x7f0b300b1b40 2023-10-06T03:45:49.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.229+0000 7f0b34b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3016a150 con 0x7f0b300b16d0 2023-10-06T03:45:49.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.229+0000 7f0b34b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3016a430 con 0x7f0b300b4e70 2023-10-06T03:45:49.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.229+0000 7f0b2effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300b1b40 0x7f0b3016cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:49.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.229+0000 7f0b2effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300b1b40 0x7f0b3016cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52996/0 (socket says 172.21.15.55:52996) 2023-10-06T03:45:49.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.229+0000 7f0b2effd700 1 -- 172.21.15.55:0/1526825268 learned_addr learned my addr 172.21.15.55:0/1526825268 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:49.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.229+0000 7f0b2f7fe700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b300b16d0 0x7f0b3015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:49.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.230+0000 7f0b2ffff700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b300b4e70 0x7f0b301574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:49.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.230+0000 7f0b2effd700 1 -- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b300b4e70 msgr2=0x7f0b301574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:49.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.230+0000 7f0b2effd700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b300b4e70 0x7f0b301574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.230+0000 7f0b2effd700 1 -- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b300b16d0 msgr2=0x7f0b3015c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:49.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.230+0000 7f0b2effd700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b300b16d0 0x7f0b3015c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.230+0000 7f0b2effd700 1 -- 172.21.15.55:0/1526825268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b28009a20 con 0x7f0b300b1b40 2023-10-06T03:45:49.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.230+0000 7f0b2ffff700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b300b4e70 0x7f0b301574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:45:49.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.230+0000 7f0b2effd700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300b1b40 0x7f0b3016cae0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400c810 tx=0x7f0b24007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:49.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.230+0000 7f0b177fe700 1 -- 172.21.15.55:0/1526825268 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24012630 con 0x7f0b300b1b40 2023-10-06T03:45:49.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.230+0000 7f0b177fe700 1 -- 172.21.15.55:0/1526825268 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b240127b0 con 0x7f0b300b1b40 2023-10-06T03:45:49.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.230+0000 7f0b177fe700 1 -- 172.21.15.55:0/1526825268 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24012b70 con 0x7f0b300b1b40 2023-10-06T03:45:49.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.230+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1526825268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3016a710 con 0x7f0b300b1b40 2023-10-06T03:45:49.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.231+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1526825268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3016acd0 con 0x7f0b300b1b40 2023-10-06T03:45:49.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.232+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1526825268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b30088990 con 0x7f0b300b1b40 2023-10-06T03:45:49.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.233+0000 7f0b177fe700 1 -- 172.21.15.55:0/1526825268 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) v1 ==== 84911+0+0 (secure 0 0 0) 0x7f0b2401b400 con 0x7f0b300b1b40 2023-10-06T03:45:49.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.234+0000 7f0b177fe700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0b1c0670e0 0x7f0b1c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:49.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.234+0000 7f0b177fe700 1 -- 172.21.15.55:0/1526825268 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6801+0+0 (secure 0 0 0) 0x7f0b240848a0 con 0x7f0b300b1b40 2023-10-06T03:45:49.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.234+0000 7f0b2f7fe700 1 -- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0b1c0670e0 msgr2=0x7f0b1c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:49.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.234+0000 7f0b2f7fe700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0b1c0670e0 0x7f0b1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:45:49.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.238+0000 7f0b177fe700 1 -- 172.21.15.55:0/1526825268 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b24051440 con 0x7f0b300b1b40 2023-10-06T03:45:49.295 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:49.294+0000 7f71e5a54180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:45:49.321 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:49.319+0000 7f911da9b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:45:49.391 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:49.389+0000 7f71e5a54180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:45:49.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.435+0000 7f0b2f7fe700 1 -- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0b1c0670e0 msgr2=0x7f0b1c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:49.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.435+0000 7f0b2f7fe700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0b1c0670e0 0x7f0b1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:45:49.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.450+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1526825268 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0b3004bb80 con 0x7f0b300b1b40 2023-10-06T03:45:49.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.450+0000 7f0b177fe700 1 -- 172.21.15.55:0/1526825268 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v50) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0b24054a60 con 0x7f0b300b1b40 2023-10-06T03:45:49.453 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:45:49.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.453+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0b1c0670e0 msgr2=0x7f0b1c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:45:49.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.453+0000 7f0b34b9d700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0b1c0670e0 0x7f0b1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.453+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1526825268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300b1b40 msgr2=0x7f0b3016cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.453+0000 7f0b34b9d700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300b1b40 0x7f0b3016cae0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400c810 tx=0x7f0b24007ee0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.453+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1526825268 shutdown_connections 2023-10-06T03:45:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.453+0000 7f0b34b9d700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f0b1c0670e0 0x7f0b1c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.453+0000 7f0b34b9d700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0b300b4e70 0x7f0b301574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.453+0000 7f0b34b9d700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0b300b16d0 0x7f0b3015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.453+0000 7f0b34b9d700 1 --2- 172.21.15.55:0/1526825268 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0b300b1b40 0x7f0b3016cae0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.453+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1526825268 >> 172.21.15.55:0/1526825268 conn(0x7f0b300d10c0 msgr2=0x7f0b301590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.453+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1526825268 shutdown_connections 2023-10-06T03:45:49.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.453+0000 7f0b34b9d700 1 -- 172.21.15.55:0/1526825268 wait complete. 2023-10-06T03:45:49.473 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:45:49.550 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:49.548+0000 7f911da9b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:45:49.640 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:49.638+0000 7f71e5a54180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:45:49.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.679+0000 7f1e9619d700 1 Processor -- start 2023-10-06T03:45:49.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.679+0000 7f1e9619d700 1 -- start start 2023-10-06T03:45:49.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.680+0000 7f1e9619d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e900b4e70 0x7f1e900ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.680+0000 7f1e9619d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1e900b1b40 0x7f1e9015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.680+0000 7f1e9619d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e900cfb00 con 0x7f1e900b16d0 2023-10-06T03:45:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.680+0000 7f1e9619d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e900b5c40 con 0x7f1e900b1b40 2023-10-06T03:45:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.680+0000 7f1e9619d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e900b5f20 con 0x7f1e900b4e70 2023-10-06T03:45:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.680+0000 7f1e9599c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1e900b1b40 0x7f1e9015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.680+0000 7f1e9599c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1e900b1b40 0x7f1e9015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53002/0 (socket says 172.21.15.55:53002) 2023-10-06T03:45:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.680+0000 7f1e9599c700 1 -- 172.21.15.55:0/405130027 learned_addr learned my addr 172.21.15.55:0/405130027 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:49.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.680+0000 7f1e9519b700 1 --2- 172.21.15.55:0/405130027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e900b4e70 0x7f1e900ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.681+0000 7f1e9599c700 1 -- 172.21.15.55:0/405130027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e900b4e70 msgr2=0x7f1e900ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.681+0000 7f1e9599c700 1 --2- 172.21.15.55:0/405130027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e900b4e70 0x7f1e900ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.681+0000 7f1e9599c700 1 -- 172.21.15.55:0/405130027 >> v1:172.21.15.163:6789/0 conn(0x7f1e900b16d0 legacy=0x7f1e9015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.681+0000 7f1e9599c700 1 -- 172.21.15.55:0/405130027 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e900b6200 con 0x7f1e900b1b40 2023-10-06T03:45:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.681+0000 7f1e87fff700 1 -- 172.21.15.55:0/405130027 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2643216685 0 0) 0x7f1e900cfb00 con 0x7f1e900b16d0 2023-10-06T03:45:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.681+0000 7f1e9599c700 1 --2- 172.21.15.55:0/405130027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e900b1b40 0x7f1e9015d950 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009f20 tx=0x7f1e88009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556295a6a114d3cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.681+0000 7f1e87fff700 1 -- 172.21.15.55:0/405130027 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8800e070 con 0x7f1e900b1b40 2023-10-06T03:45:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.681+0000 7f1e87fff700 1 -- 172.21.15.55:0/405130027 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8800b9d0 con 0x7f1e900b1b40 2023-10-06T03:45:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.681+0000 7f1e87fff700 1 -- 172.21.15.55:0/405130027 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e880092e0 con 0x7f1e900b1b40 2023-10-06T03:45:49.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.682+0000 7f1e9619d700 1 -- 172.21.15.55:0/405130027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e900b1b40 msgr2=0x7f1e9015d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.682+0000 7f1e9619d700 1 --2- 172.21.15.55:0/405130027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e900b1b40 0x7f1e9015d950 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009f20 tx=0x7f1e88009840 comp rx=0 tx=0).stop 2023-10-06T03:45:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.682+0000 7f1e9619d700 1 -- 172.21.15.55:0/405130027 shutdown_connections 2023-10-06T03:45:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.682+0000 7f1e9619d700 1 --2- 172.21.15.55:0/405130027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e900b1b40 0x7f1e9015d950 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.682+0000 7f1e9619d700 1 --2- 172.21.15.55:0/405130027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e900b4e70 0x7f1e900ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.682+0000 7f1e9619d700 1 -- 172.21.15.55:0/405130027 >> 172.21.15.55:0/405130027 conn(0x7f1e900d10c0 msgr2=0x7f1e900affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.682+0000 7f1e9619d700 1 -- 172.21.15.55:0/405130027 shutdown_connections 2023-10-06T03:45:49.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.682+0000 7f1e9619d700 1 -- 172.21.15.55:0/405130027 wait complete. 2023-10-06T03:45:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.684+0000 7f1e9619d700 1 Processor -- start 2023-10-06T03:45:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.684+0000 7f1e9619d700 1 -- start start 2023-10-06T03:45:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.684+0000 7f1e9619d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e900b16d0 0x7f1e900af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.684+0000 7f1e9619d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e900b1b40 0x7f1e900c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.685+0000 7f1e9619d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e900b4e70 0x7f1e900ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.685+0000 7f1e9619d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e900af5a0 con 0x7f1e900b4e70 2023-10-06T03:45:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.685+0000 7f1e9619d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e900c9e10 con 0x7f1e900b16d0 2023-10-06T03:45:49.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.685+0000 7f1e9619d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e900ca0f0 con 0x7f1e900b1b40 2023-10-06T03:45:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.685+0000 7f1e9599c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e900b4e70 0x7f1e900ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.685+0000 7f1e9599c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e900b4e70 0x7f1e900ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53010/0 (socket says 172.21.15.55:53010) 2023-10-06T03:45:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.685+0000 7f1e9599c700 1 -- 172.21.15.55:0/1726986820 learned_addr learned my addr 172.21.15.55:0/1726986820 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.685+0000 7f1e9519b700 1 --2- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e900b16d0 0x7f1e900af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.685+0000 7f1e9499a700 1 --2- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e900b1b40 0x7f1e900c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.685+0000 7f1e9599c700 1 -- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e900b1b40 msgr2=0x7f1e900c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.685+0000 7f1e9599c700 1 --2- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e900b1b40 0x7f1e900c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.685+0000 7f1e9599c700 1 -- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e900b16d0 msgr2=0x7f1e900af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:49.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.685+0000 7f1e9599c700 1 --2- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e900b16d0 0x7f1e900af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.685+0000 7f1e9599c700 1 -- 172.21.15.55:0/1726986820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e88009ae0 con 0x7f1e900b4e70 2023-10-06T03:45:49.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.686+0000 7f1e9599c700 1 --2- 172.21.15.55:0/1726986820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e900b4e70 0x7f1e900ca700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1e90070dd0 tx=0x7f1e88018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:49.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.686+0000 7f1e857fa700 1 -- 172.21.15.55:0/1726986820 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e88008bc0 con 0x7f1e900b4e70 2023-10-06T03:45:49.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.686+0000 7f1e857fa700 1 -- 172.21.15.55:0/1726986820 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e88008d40 con 0x7f1e900b4e70 2023-10-06T03:45:49.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.686+0000 7f1e9619d700 1 -- 172.21.15.55:0/1726986820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e900ca3d0 con 0x7f1e900b4e70 2023-10-06T03:45:49.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.686+0000 7f1e857fa700 1 -- 172.21.15.55:0/1726986820 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e880266b0 con 0x7f1e900b4e70 2023-10-06T03:45:49.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.686+0000 7f1e9619d700 1 -- 172.21.15.55:0/1726986820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e900c7ea0 con 0x7f1e900b4e70 2023-10-06T03:45:49.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.687+0000 7f1e9619d700 1 -- 172.21.15.55:0/1726986820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e900b1510 con 0x7f1e900b4e70 2023-10-06T03:45:49.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.688+0000 7f1e857fa700 1 -- 172.21.15.55:0/1726986820 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) v1 ==== 84911+0+0 (secure 0 0 0) 0x7f1e88003040 con 0x7f1e900b4e70 2023-10-06T03:45:49.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.689+0000 7f1e857fa700 1 --2- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1e740670e0 0x7f1e74069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:49.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.689+0000 7f1e857fa700 1 -- 172.21.15.55:0/1726986820 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6801+0+0 (secure 0 0 0) 0x7f1e88087280 con 0x7f1e900b4e70 2023-10-06T03:45:49.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.689+0000 7f1e9519b700 1 -- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1e740670e0 msgr2=0x7f1e74069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:49.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.690+0000 7f1e9519b700 1 --2- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1e740670e0 0x7f1e74069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:45:49.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.693+0000 7f1e857fa700 1 -- 172.21.15.55:0/1726986820 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e88053da0 con 0x7f1e900b4e70 2023-10-06T03:45:49.735 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:49.733+0000 7f911da9b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:45:49.829 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:49.827+0000 7f911da9b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:45:49.832 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:49.831+0000 7f71e5a54180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:45:49.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.891+0000 7f1e9519b700 1 -- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1e740670e0 msgr2=0x7f1e74069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:49.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.891+0000 7f1e9519b700 1 --2- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1e740670e0 0x7f1e74069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:45:49.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.900+0000 7f1e9619d700 1 -- 172.21.15.55:0/1726986820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e900b5a20 con 0x7f1e900b4e70 2023-10-06T03:45:49.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.900+0000 7f1e857fa700 1 -- 172.21.15.55:0/1726986820 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e880573c0 con 0x7f1e900b4e70 2023-10-06T03:45:49.902 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:45:49.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.903+0000 7f1e9619d700 1 -- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1e740670e0 msgr2=0x7f1e74069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:45:49.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.903+0000 7f1e9619d700 1 --2- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1e740670e0 0x7f1e74069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.903+0000 7f1e9619d700 1 -- 172.21.15.55:0/1726986820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e900b4e70 msgr2=0x7f1e900ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:49.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.903+0000 7f1e9619d700 1 --2- 172.21.15.55:0/1726986820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e900b4e70 0x7f1e900ca700 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1e90070dd0 tx=0x7f1e88018000 comp rx=0 tx=0).stop 2023-10-06T03:45:49.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.903+0000 7f1e9619d700 1 -- 172.21.15.55:0/1726986820 shutdown_connections 2023-10-06T03:45:49.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.903+0000 7f1e9619d700 1 --2- 172.21.15.55:0/1726986820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1e900b4e70 0x7f1e900ca700 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.903+0000 7f1e9619d700 1 --2- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1e900b16d0 0x7f1e900af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.903+0000 7f1e9619d700 1 --2- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f1e740670e0 0x7f1e74069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.903+0000 7f1e9619d700 1 --2- 172.21.15.55:0/1726986820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1e900b1b40 0x7f1e900c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:49.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.903+0000 7f1e9619d700 1 -- 172.21.15.55:0/1726986820 >> 172.21.15.55:0/1726986820 conn(0x7f1e900d10c0 msgr2=0x7f1e900b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:49.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.903+0000 7f1e9619d700 1 -- 172.21.15.55:0/1726986820 shutdown_connections 2023-10-06T03:45:49.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:49.903+0000 7f1e9619d700 1 -- 172.21.15.55:0/1726986820 wait complete. 2023-10-06T03:45:49.922 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:45:49.929 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:49.927+0000 7f71e5a54180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:45:49.991 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:49.989+0000 7f911da9b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:45:50.096 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:50.094+0000 7f911da9b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:45:50.104 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:50.103+0000 7f71e5a54180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:45:50.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.123+0000 7fb30359e700 1 Processor -- start 2023-10-06T03:45:50.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.123+0000 7fb30359e700 1 -- start start 2023-10-06T03:45:50.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb30359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3040b4e70 0x7fb3040ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:50.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb30359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb3040b16d0 0x7fb3040b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:50.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb30359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb30415d240 0x7fb30415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:50.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb30359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3040b2080 con 0x7fb3040b16d0 2023-10-06T03:45:50.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb30359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3040cfb00 con 0x7fb30415d240 2023-10-06T03:45:50.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb30359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3040b5c40 con 0x7fb3040b4e70 2023-10-06T03:45:50.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb301d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb3040b16d0 0x7fb3040b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:50.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb30259c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3040b4e70 0x7fb3040ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:50.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb301d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb3040b16d0 0x7fb3040b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53020/0 (socket says 172.21.15.55:53020) 2023-10-06T03:45:50.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb302d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb30415d240 0x7fb30415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:50.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb301d9b700 1 -- 172.21.15.55:0/3133214697 learned_addr learned my addr 172.21.15.55:0/3133214697 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:50.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb301d9b700 1 -- 172.21.15.55:0/3133214697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3040b4e70 msgr2=0x7fb3040ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:50.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb301d9b700 1 --2- 172.21.15.55:0/3133214697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3040b4e70 0x7fb3040ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb301d9b700 1 -- 172.21.15.55:0/3133214697 >> v2:172.21.15.163:3300/0 conn(0x7fb30415d240 msgr2=0x7fb30415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:50.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb301d9b700 1 --2- 172.21.15.55:0/3133214697 >> v2:172.21.15.163:3300/0 conn(0x7fb30415d240 0x7fb30415d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.125+0000 7fb301d9b700 1 -- 172.21.15.55:0/3133214697 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3040b5f20 con 0x7fb3040b16d0 2023-10-06T03:45:50.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.126+0000 7fb301d9b700 1 --2- 172.21.15.55:0/3133214697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3040b16d0 0x7fb3040b1b40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb30415d180 tx=0x7fb2f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c67d49417cd3848 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:50.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.126+0000 7fb30159a700 1 -- 172.21.15.55:0/3133214697 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f800b640 con 0x7fb3040b16d0 2023-10-06T03:45:50.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.126+0000 7fb30159a700 1 -- 172.21.15.55:0/3133214697 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f800b7a0 con 0x7fb3040b16d0 2023-10-06T03:45:50.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.126+0000 7fb30159a700 1 -- 172.21.15.55:0/3133214697 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f800bb60 con 0x7fb3040b16d0 2023-10-06T03:45:50.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.127+0000 7fb30359e700 1 -- 172.21.15.55:0/3133214697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3040b16d0 msgr2=0x7fb3040b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:50.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.127+0000 7fb30359e700 1 --2- 172.21.15.55:0/3133214697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3040b16d0 0x7fb3040b1b40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb30415d180 tx=0x7fb2f8009250 comp rx=0 tx=0).stop 2023-10-06T03:45:50.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.127+0000 7fb30359e700 1 -- 172.21.15.55:0/3133214697 shutdown_connections 2023-10-06T03:45:50.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.127+0000 7fb30359e700 1 --2- 172.21.15.55:0/3133214697 >> v2:172.21.15.163:3300/0 conn(0x7fb30415d240 0x7fb30415d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.127+0000 7fb30359e700 1 --2- 172.21.15.55:0/3133214697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3040b4e70 0x7fb3040ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.127+0000 7fb30359e700 1 --2- 172.21.15.55:0/3133214697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3040b16d0 0x7fb3040b1b40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.127+0000 7fb30359e700 1 -- 172.21.15.55:0/3133214697 >> 172.21.15.55:0/3133214697 conn(0x7fb3040d10c0 msgr2=0x7fb3040affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:50.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.127+0000 7fb30359e700 1 -- 172.21.15.55:0/3133214697 shutdown_connections 2023-10-06T03:45:50.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.127+0000 7fb30359e700 1 -- 172.21.15.55:0/3133214697 wait complete. 2023-10-06T03:45:50.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.128+0000 7fb30359e700 1 Processor -- start 2023-10-06T03:45:50.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.129+0000 7fb30359e700 1 -- start start 2023-10-06T03:45:50.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.129+0000 7fb30359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3040b16d0 0x7fb304148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb30359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3040b4e70 0x7fb3040c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb30359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb30415d240 0x7fb30415e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb30359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3040c6360 con 0x7fb3040b4e70 2023-10-06T03:45:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb30359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3040ca840 con 0x7fb3040b16d0 2023-10-06T03:45:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb30359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3040c65b0 con 0x7fb30415d240 2023-10-06T03:45:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb301d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3040b4e70 0x7fb3040c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb30259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3040b16d0 0x7fb304148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb30259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3040b16d0 0x7fb304148f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:50866/0 (socket says 172.21.15.55:50866) 2023-10-06T03:45:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb30259c700 1 -- 172.21.15.55:0/738006707 learned_addr learned my addr 172.21.15.55:0/738006707 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb302d9d700 1 --2- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb30415d240 0x7fb30415e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb301d9b700 1 -- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb30415d240 msgr2=0x7fb30415e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:50.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb301d9b700 1 --2- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb30415d240 0x7fb30415e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb301d9b700 1 -- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3040b16d0 msgr2=0x7fb304148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:50.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb301d9b700 1 --2- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3040b16d0 0x7fb304148f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.130+0000 7fb301d9b700 1 -- 172.21.15.55:0/738006707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f80094f0 con 0x7fb3040b4e70 2023-10-06T03:45:50.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.131+0000 7fb301d9b700 1 --2- 172.21.15.55:0/738006707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3040b4e70 0x7fb3040c9df0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb304089440 tx=0x7fb2f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:50.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.131+0000 7fb2f27fc700 1 -- 172.21.15.55:0/738006707 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f800b910 con 0x7fb3040b4e70 2023-10-06T03:45:50.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.131+0000 7fb2f27fc700 1 -- 172.21.15.55:0/738006707 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f800ba70 con 0x7fb3040b4e70 2023-10-06T03:45:50.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.131+0000 7fb2f27fc700 1 -- 172.21.15.55:0/738006707 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8011650 con 0x7fb3040b4e70 2023-10-06T03:45:50.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.131+0000 7fb30359e700 1 -- 172.21.15.55:0/738006707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3040c6890 con 0x7fb3040b4e70 2023-10-06T03:45:50.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.131+0000 7fb30359e700 1 -- 172.21.15.55:0/738006707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3041494c0 con 0x7fb3040b4e70 2023-10-06T03:45:50.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.132+0000 7fb30359e700 1 -- 172.21.15.55:0/738006707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb304088990 con 0x7fb3040b4e70 2023-10-06T03:45:50.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.133+0000 7fb2f27fc700 1 -- 172.21.15.55:0/738006707 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) v1 ==== 84911+0+0 (secure 0 0 0) 0x7fb2f8011a00 con 0x7fb3040b4e70 2023-10-06T03:45:50.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.134+0000 7fb2f27fc700 1 --2- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb2e40670e0 0x7fb2e4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:50.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.134+0000 7fb2f27fc700 1 -- 172.21.15.55:0/738006707 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6753+0+0 (secure 0 0 0) 0x7fb2f8085ce0 con 0x7fb3040b4e70 2023-10-06T03:45:50.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.134+0000 7fb30259c700 1 -- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb2e40670e0 msgr2=0x7fb2e4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:50.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.134+0000 7fb30259c700 1 --2- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb2e40670e0 0x7fb2e4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:45:50.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.138+0000 7fb2f27fc700 1 -- 172.21.15.55:0/738006707 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2f8052810 con 0x7fb3040b4e70 2023-10-06T03:45:50.213 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:50.211+0000 7f71e5a54180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:45:50.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.335+0000 7fb30259c700 1 -- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb2e40670e0 msgr2=0x7fb2e4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:50.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.335+0000 7fb30259c700 1 --2- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb2e40670e0 0x7fb2e4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:45:50.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.342+0000 7fb30359e700 1 -- 172.21.15.55:0/738006707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3040cf6f0 con 0x7fb3040b4e70 2023-10-06T03:45:50.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.342+0000 7fb2f27fc700 1 -- 172.21.15.55:0/738006707 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb2f8015020 con 0x7fb3040b4e70 2023-10-06T03:45:50.344 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T03:45:50.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.344+0000 7fb30359e700 1 -- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb2e40670e0 msgr2=0x7fb2e4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:45:50.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.344+0000 7fb30359e700 1 --2- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb2e40670e0 0x7fb2e4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.344+0000 7fb30359e700 1 -- 172.21.15.55:0/738006707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3040b4e70 msgr2=0x7fb3040c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:50.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.344+0000 7fb30359e700 1 --2- 172.21.15.55:0/738006707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3040b4e70 0x7fb3040c9df0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb304089440 tx=0x7fb2f8017000 comp rx=0 tx=0).stop 2023-10-06T03:45:50.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.344+0000 7fb30359e700 1 -- 172.21.15.55:0/738006707 shutdown_connections 2023-10-06T03:45:50.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.344+0000 7fb30359e700 1 --2- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fb2e40670e0 0x7fb2e4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.344+0000 7fb30359e700 1 --2- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb30415d240 0x7fb30415e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.344+0000 7fb30359e700 1 --2- 172.21.15.55:0/738006707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3040b16d0 0x7fb304148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.344+0000 7fb30359e700 1 --2- 172.21.15.55:0/738006707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3040b4e70 0x7fb3040c9df0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.344+0000 7fb30359e700 1 -- 172.21.15.55:0/738006707 >> 172.21.15.55:0/738006707 conn(0x7fb3040d10c0 msgr2=0x7fb3040b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:50.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.345+0000 7fb30359e700 1 -- 172.21.15.55:0/738006707 shutdown_connections 2023-10-06T03:45:50.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.345+0000 7fb30359e700 1 -- 172.21.15.55:0/738006707 wait complete. 2023-10-06T03:45:50.364 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:45:50.588 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:50.586+0000 7f911da9b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:45:50.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.605+0000 7f4a3f59d700 1 Processor -- start 2023-10-06T03:45:50.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.605+0000 7f4a3f59d700 1 -- start start 2023-10-06T03:45:50.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.606+0000 7f4a3f59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4a380b4e70 0x7f4a380ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:50.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.607+0000 7f4a3f59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a380b1b40 0x7f4a3815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:50.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.607+0000 7f4a3f59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a380cfb00 con 0x7f4a380b16d0 2023-10-06T03:45:50.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.607+0000 7f4a3f59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a380b5c40 con 0x7f4a380b4e70 2023-10-06T03:45:50.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.607+0000 7f4a3f59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380b5f20 con 0x7f4a380b1b40 2023-10-06T03:45:50.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.607+0000 7f4a3e59b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4a380b4e70 0x7f4a380ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:50.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.607+0000 7f4a3e59b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4a380b4e70 0x7f4a380ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53044/0 (socket says 172.21.15.55:53044) 2023-10-06T03:45:50.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.607+0000 7f4a3e59b700 1 -- 172.21.15.55:0/398520743 learned_addr learned my addr 172.21.15.55:0/398520743 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:50.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.607+0000 7f4a3ed9c700 1 --2- 172.21.15.55:0/398520743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a380b1b40 0x7f4a3815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:50.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.607+0000 7f4a3e59b700 1 -- 172.21.15.55:0/398520743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a380b1b40 msgr2=0x7f4a3815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:50.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.607+0000 7f4a3e59b700 1 --2- 172.21.15.55:0/398520743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a380b1b40 0x7f4a3815d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.607+0000 7f4a3d599700 1 -- 172.21.15.55:0/398520743 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804446027 0 0) 0x7f4a380cfb00 con 0x7f4a380b16d0 2023-10-06T03:45:50.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.607+0000 7f4a3e59b700 1 -- 172.21.15.55:0/398520743 >> v1:172.21.15.163:6789/0 conn(0x7f4a380b16d0 legacy=0x7f4a3815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:50.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.607+0000 7f4a3e59b700 1 -- 172.21.15.55:0/398520743 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a380b6200 con 0x7f4a380b4e70 2023-10-06T03:45:50.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.608+0000 7f4a3e59b700 1 --2- 172.21.15.55:0/398520743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a380b4e70 0x7f4a380ccff0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4a340099b0 tx=0x7f4a34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c02cadd86c0349 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:50.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.608+0000 7f4a3d599700 1 -- 172.21.15.55:0/398520743 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a34009ea0 con 0x7f4a380b4e70 2023-10-06T03:45:50.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.608+0000 7f4a3d599700 1 -- 172.21.15.55:0/398520743 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3400b860 con 0x7f4a380b4e70 2023-10-06T03:45:50.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.608+0000 7f4a3d599700 1 -- 172.21.15.55:0/398520743 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3400bc00 con 0x7f4a380b4e70 2023-10-06T03:45:50.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.608+0000 7f4a3f59d700 1 -- 172.21.15.55:0/398520743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a380b4e70 msgr2=0x7f4a380ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:50.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.608+0000 7f4a3f59d700 1 --2- 172.21.15.55:0/398520743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a380b4e70 0x7f4a380ccff0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4a340099b0 tx=0x7f4a34009250 comp rx=0 tx=0).stop 2023-10-06T03:45:50.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.609+0000 7f4a3f59d700 1 -- 172.21.15.55:0/398520743 shutdown_connections 2023-10-06T03:45:50.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.609+0000 7f4a3f59d700 1 --2- 172.21.15.55:0/398520743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a380b1b40 0x7f4a3815d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.609+0000 7f4a3f59d700 1 --2- 172.21.15.55:0/398520743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a380b4e70 0x7f4a380ccff0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.609+0000 7f4a3f59d700 1 -- 172.21.15.55:0/398520743 >> 172.21.15.55:0/398520743 conn(0x7f4a380d10c0 msgr2=0x7f4a380affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:50.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.609+0000 7f4a3f59d700 1 -- 172.21.15.55:0/398520743 shutdown_connections 2023-10-06T03:45:50.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.609+0000 7f4a3f59d700 1 -- 172.21.15.55:0/398520743 wait complete. 2023-10-06T03:45:50.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.610+0000 7f4a3f59d700 1 Processor -- start 2023-10-06T03:45:50.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.610+0000 7f4a3f59d700 1 -- start start 2023-10-06T03:45:50.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.611+0000 7f4a3f59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a380b16d0 0x7f4a3816a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:50.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.611+0000 7f4a3f59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a380b1b40 0x7f4a380c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:50.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.611+0000 7f4a3f59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a380b4e70 0x7f4a381578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:50.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.611+0000 7f4a3f59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380ca1b0 con 0x7f4a380b4e70 2023-10-06T03:45:50.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.611+0000 7f4a3f59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380cbda0 con 0x7f4a380b16d0 2023-10-06T03:45:50.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.611+0000 7f4a3f59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a380ca450 con 0x7f4a380b1b40 2023-10-06T03:45:50.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.612+0000 7f4a3e59b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a380b16d0 0x7f4a3816a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:50.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.612+0000 7f4a3dd9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a380b1b40 0x7f4a380c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:50.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.612+0000 7f4a3ed9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a380b4e70 0x7f4a381578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:50.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.612+0000 7f4a3e59b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a380b16d0 0x7f4a3816a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:50876/0 (socket says 172.21.15.55:50876) 2023-10-06T03:45:50.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.612+0000 7f4a3dd9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a380b1b40 0x7f4a380c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:37812/0 (socket says 172.21.15.55:37812) 2023-10-06T03:45:50.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.612+0000 7f4a3e59b700 1 -- 172.21.15.55:0/1942846475 learned_addr learned my addr 172.21.15.55:0/1942846475 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:50.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.612+0000 7f4a3ed9c700 1 -- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a380b1b40 msgr2=0x7f4a380c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:50.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.612+0000 7f4a3ed9c700 1 --2- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a380b1b40 0x7f4a380c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.612+0000 7f4a3ed9c700 1 -- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a380b16d0 msgr2=0x7f4a3816a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:50.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.612+0000 7f4a3ed9c700 1 --2- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a380b16d0 0x7f4a3816a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.612+0000 7f4a3ed9c700 1 -- 172.21.15.55:0/1942846475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a340094f0 con 0x7f4a380b4e70 2023-10-06T03:45:50.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.612+0000 7f4a3e59b700 1 --2- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a380b16d0 0x7f4a3816a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:45:50.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.612+0000 7f4a3dd9a700 1 --2- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a380b1b40 0x7f4a380c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:45:50.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.613+0000 7f4a3ed9c700 1 --2- 172.21.15.55:0/1942846475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a380b4e70 0x7f4a381578b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009e50 tx=0x7f4a3000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:50.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.613+0000 7f4a2e7fc700 1 -- 172.21.15.55:0/1942846475 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000caa0 con 0x7f4a380b4e70 2023-10-06T03:45:50.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.613+0000 7f4a2e7fc700 1 -- 172.21.15.55:0/1942846475 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3000f040 con 0x7f4a380b4e70 2023-10-06T03:45:50.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.613+0000 7f4a2e7fc700 1 -- 172.21.15.55:0/1942846475 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000ce50 con 0x7f4a380b4e70 2023-10-06T03:45:50.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.613+0000 7f4a3f59d700 1 -- 172.21.15.55:0/1942846475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a380c9d60 con 0x7f4a380b4e70 2023-10-06T03:45:50.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.613+0000 7f4a3f59d700 1 -- 172.21.15.55:0/1942846475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a380cc2d0 con 0x7f4a380b4e70 2023-10-06T03:45:50.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.614+0000 7f4a3f59d700 1 -- 172.21.15.55:0/1942846475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a380ce100 con 0x7f4a380b4e70 2023-10-06T03:45:50.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.616+0000 7f4a2e7fc700 1 -- 172.21.15.55:0/1942846475 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) v1 ==== 84911+0+0 (secure 0 0 0) 0x7f4a30014690 con 0x7f4a380b4e70 2023-10-06T03:45:50.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.617+0000 7f4a2e7fc700 1 --2- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4a1c0670e0 0x7f4a1c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:50.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.617+0000 7f4a2e7fc700 1 -- 172.21.15.55:0/1942846475 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6753+0+0 (secure 0 0 0) 0x7f4a30085950 con 0x7f4a380b4e70 2023-10-06T03:45:50.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.617+0000 7f4a3e59b700 1 -- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4a1c0670e0 msgr2=0x7f4a1c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:50.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.617+0000 7f4a3e59b700 1 --2- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4a1c0670e0 0x7f4a1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:45:50.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.621+0000 7f4a2e7fc700 1 -- 172.21.15.55:0/1942846475 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a30052410 con 0x7f4a380b4e70 2023-10-06T03:45:50.687 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:50.685+0000 7f911da9b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:45:50.732 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:50.731+0000 7f71e5a54180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:45:50.780 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:50.778+0000 7f911da9b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:45:50.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.818+0000 7f4a3e59b700 1 -- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4a1c0670e0 msgr2=0x7f4a1c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:50.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.818+0000 7f4a3e59b700 1 --2- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4a1c0670e0 0x7f4a1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:45:50.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.827+0000 7f4a3f59d700 1 -- 172.21.15.55:0/1942846475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a380cc4f0 con 0x7f4a380b4e70 2023-10-06T03:45:50.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.828+0000 7f4a2e7fc700 1 -- 172.21.15.55:0/1942846475 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a30055a30 con 0x7f4a380b4e70 2023-10-06T03:45:50.829 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T03:45:50.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.830+0000 7f4a3f59d700 1 -- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4a1c0670e0 msgr2=0x7f4a1c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:45:50.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.830+0000 7f4a3f59d700 1 --2- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4a1c0670e0 0x7f4a1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.830+0000 7f4a3f59d700 1 -- 172.21.15.55:0/1942846475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a380b4e70 msgr2=0x7f4a381578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:50.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.830+0000 7f4a3f59d700 1 --2- 172.21.15.55:0/1942846475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a380b4e70 0x7f4a381578b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009e50 tx=0x7f4a3000b040 comp rx=0 tx=0).stop 2023-10-06T03:45:50.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.830+0000 7f4a3f59d700 1 -- 172.21.15.55:0/1942846475 shutdown_connections 2023-10-06T03:45:50.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.830+0000 7f4a3f59d700 1 --2- 172.21.15.55:0/1942846475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a380b4e70 0x7f4a381578b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.830+0000 7f4a3f59d700 1 --2- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a380b16d0 0x7f4a3816a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.830+0000 7f4a3f59d700 1 --2- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f4a1c0670e0 0x7f4a1c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.830+0000 7f4a3f59d700 1 --2- 172.21.15.55:0/1942846475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a380b1b40 0x7f4a380c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:50.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.830+0000 7f4a3f59d700 1 -- 172.21.15.55:0/1942846475 >> 172.21.15.55:0/1942846475 conn(0x7f4a380d10c0 msgr2=0x7f4a3815ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:50.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.830+0000 7f4a3f59d700 1 -- 172.21.15.55:0/1942846475 shutdown_connections 2023-10-06T03:45:50.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:50.830+0000 7f4a3f59d700 1 -- 172.21.15.55:0/1942846475 wait complete. 2023-10-06T03:45:50.833 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:50.831+0000 7f71e5a54180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:45:50.851 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:45:50.932 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:50.930+0000 7f71e5a54180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:45:51.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.043+0000 7fbe05b9d700 1 Processor -- start 2023-10-06T03:45:51.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.043+0000 7fbe05b9d700 1 -- start start 2023-10-06T03:45:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.044+0000 7fbe05b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe0015d230 0x7fbe0015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.044+0000 7fbe05b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe000b2980 con 0x7fbe00159420 2023-10-06T03:45:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.044+0000 7fbe05b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe0015dbb0 con 0x7fbe001597f0 2023-10-06T03:45:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.044+0000 7fbe05b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000b11d0 con 0x7fbe0015d230 2023-10-06T03:45:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.045+0000 7fbe04b9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbe00159420 0x7fbe000b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42364/0 (socket says 172.21.15.55:42364) 2023-10-06T03:45:51.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.045+0000 7fbe04b9b700 1 -- 172.21.15.55:0/270550506 learned_addr learned my addr 172.21.15.55:0/270550506 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:51.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.045+0000 7fbe0539c700 1 --2- 172.21.15.55:0/270550506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe0015d230 0x7fbe0015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.045+0000 7fbdff7fe700 1 -- 172.21.15.55:0/270550506 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937056291 0 0) 0x7fbe000b2980 con 0x7fbe00159420 2023-10-06T03:45:51.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.045+0000 7fbdff7fe700 1 -- 172.21.15.55:0/270550506 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdf0003570 con 0x7fbe00159420 2023-10-06T03:45:51.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.045+0000 7fbdff7fe700 1 -- 172.21.15.55:0/270550506 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67330007 0 0) 0x7fbe0015dbb0 con 0x7fbe001597f0 2023-10-06T03:45:51.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.045+0000 7fbe0539c700 1 -- 172.21.15.55:0/270550506 >> v1:172.21.15.163:6789/0 conn(0x7fbe001597f0 legacy=0x7fbe000b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.045+0000 7fbe0539c700 1 -- 172.21.15.55:0/270550506 >> v1:172.21.15.55:6789/0 conn(0x7fbe00159420 legacy=0x7fbe000b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.045+0000 7fbe0539c700 1 -- 172.21.15.55:0/270550506 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe000b14b0 con 0x7fbe0015d230 2023-10-06T03:45:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.046+0000 7fbe0539c700 1 --2- 172.21.15.55:0/270550506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe0015d230 0x7fbe0015d670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009e90 tx=0x7fbdf8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f4911a36f262581 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.046+0000 7fbdff7fe700 1 -- 172.21.15.55:0/270550506 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf800e070 con 0x7fbe0015d230 2023-10-06T03:45:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.046+0000 7fbdff7fe700 1 -- 172.21.15.55:0/270550506 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf800bc30 con 0x7fbe0015d230 2023-10-06T03:45:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.046+0000 7fbdff7fe700 1 -- 172.21.15.55:0/270550506 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8012430 con 0x7fbe0015d230 2023-10-06T03:45:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.046+0000 7fbe05b9d700 1 -- 172.21.15.55:0/270550506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe0015d230 msgr2=0x7fbe0015d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.046+0000 7fbe05b9d700 1 --2- 172.21.15.55:0/270550506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe0015d230 0x7fbe0015d670 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009e90 tx=0x7fbdf8009780 comp rx=0 tx=0).stop 2023-10-06T03:45:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.046+0000 7fbe05b9d700 1 -- 172.21.15.55:0/270550506 shutdown_connections 2023-10-06T03:45:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.046+0000 7fbe05b9d700 1 --2- 172.21.15.55:0/270550506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe0015d230 0x7fbe0015d670 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.046+0000 7fbe05b9d700 1 -- 172.21.15.55:0/270550506 >> 172.21.15.55:0/270550506 conn(0x7fbe000d0660 msgr2=0x7fbe000d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:51.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.047+0000 7fbe05b9d700 1 -- 172.21.15.55:0/270550506 shutdown_connections 2023-10-06T03:45:51.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.047+0000 7fbe05b9d700 1 -- 172.21.15.55:0/270550506 wait complete. 2023-10-06T03:45:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.048+0000 7fbe05b9d700 1 Processor -- start 2023-10-06T03:45:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.048+0000 7fbe05b9d700 1 -- start start 2023-10-06T03:45:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.049+0000 7fbe05b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe00159420 0x7fbe001448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.049+0000 7fbe05b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe001597f0 0x7fbe0015a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.049+0000 7fbe05b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0015d230 0x7fbe00168a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.049+0000 7fbe05b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000c5730 con 0x7fbe0015d230 2023-10-06T03:45:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.049+0000 7fbe05b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000b5e80 con 0x7fbe001597f0 2023-10-06T03:45:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.049+0000 7fbe05b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe000b5ba0 con 0x7fbe00159420 2023-10-06T03:45:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.049+0000 7fbe04b9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe00159420 0x7fbe001448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.049+0000 7fbe0539c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0015d230 0x7fbe00168a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.049+0000 7fbe0539c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0015d230 0x7fbe00168a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53058/0 (socket says 172.21.15.55:53058) 2023-10-06T03:45:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.049+0000 7fbe04b9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe00159420 0x7fbe001448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:37830/0 (socket says 172.21.15.55:37830) 2023-10-06T03:45:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.049+0000 7fbe0539c700 1 -- 172.21.15.55:0/1599764993 learned_addr learned my addr 172.21.15.55:0/1599764993 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.049+0000 7fbdfffff700 1 --2- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe001597f0 0x7fbe0015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.050+0000 7fbe0539c700 1 -- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe00159420 msgr2=0x7fbe001448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.050+0000 7fbe0539c700 1 --2- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe00159420 0x7fbe001448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.050+0000 7fbe0539c700 1 -- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe001597f0 msgr2=0x7fbe0015a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.050+0000 7fbe0539c700 1 --2- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe001597f0 0x7fbe0015a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.050+0000 7fbe0539c700 1 -- 172.21.15.55:0/1599764993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf8009a20 con 0x7fbe0015d230 2023-10-06T03:45:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.050+0000 7fbe0539c700 1 --2- 172.21.15.55:0/1599764993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0015d230 0x7fbe00168a30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbe00089340 tx=0x7fbdf8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.050+0000 7fbdfcff9700 1 -- 172.21.15.55:0/1599764993 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf801b420 con 0x7fbe0015d230 2023-10-06T03:45:51.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.050+0000 7fbdfcff9700 1 -- 172.21.15.55:0/1599764993 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf801b5a0 con 0x7fbe0015d230 2023-10-06T03:45:51.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.050+0000 7fbe05b9d700 1 -- 172.21.15.55:0/1599764993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0015ab30 con 0x7fbe0015d230 2023-10-06T03:45:51.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.050+0000 7fbdfcff9700 1 -- 172.21.15.55:0/1599764993 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf801b960 con 0x7fbe0015d230 2023-10-06T03:45:51.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.050+0000 7fbe05b9d700 1 -- 172.21.15.55:0/1599764993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0015b0f0 con 0x7fbe0015d230 2023-10-06T03:45:51.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.052+0000 7fbe05b9d700 1 -- 172.21.15.55:0/1599764993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0015b2e0 con 0x7fbe0015d230 2023-10-06T03:45:51.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.052+0000 7fbdfcff9700 1 -- 172.21.15.55:0/1599764993 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) v1 ==== 84911+0+0 (secure 0 0 0) 0x7fbdf801bcd0 con 0x7fbe0015d230 2023-10-06T03:45:51.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.053+0000 7fbdfcff9700 1 --2- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbde40670e0 0x7fbde4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.053+0000 7fbdfcff9700 1 -- 172.21.15.55:0/1599764993 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7057+0+0 (secure 0 0 0) 0x7fbdf8086c00 con 0x7fbe0015d230 2023-10-06T03:45:51.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.053+0000 7fbe04b9b700 1 -- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbde40670e0 msgr2=0x7fbde4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:51.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.053+0000 7fbe04b9b700 1 --2- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbde40670e0 0x7fbde4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:45:51.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.058+0000 7fbdfcff9700 1 -- 172.21.15.55:0/1599764993 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbdf808d080 con 0x7fbe0015d230 2023-10-06T03:45:51.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.254+0000 7fbe04b9b700 1 -- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbde40670e0 msgr2=0x7fbde4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:51.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.254+0000 7fbe04b9b700 1 --2- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbde40670e0 0x7fbde4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:45:51.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.304+0000 7fbe05b9d700 1 -- 172.21.15.55:0/1599764993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe0004c3c0 con 0x7fbe0015d230 2023-10-06T03:45:51.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.305+0000 7fbdfcff9700 1 -- 172.21.15.55:0/1599764993 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4600 (secure 0 0 0) 0x7fbdf802a090 con 0x7fbe0015d230 2023-10-06T03:45:51.306 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:51.307 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1532010025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1532010025},{"type":"v1","addr":"172.21.15.55:6834","nonce":1532010025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4258042685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":4258042685},{"type":"v1","addr":"172.21.15.55:6835","nonce":4258042685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/430119478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":430119478},{"type":"v1","addr":"172.21.15.163:6836","nonce":430119478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:34:13.003040+0000","modified":"2023-10-06T03:34:16.548740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4903},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4903":{"gid":4903,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6837/1606384901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1606384901},{"type":"v1","addr":"172.21.15.163:6837","nonce":1606384901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T03:45:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.307+0000 7fbe05b9d700 1 -- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbde40670e0 msgr2=0x7fbde4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:45:51.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.307+0000 7fbe05b9d700 1 --2- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbde40670e0 0x7fbde4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.307+0000 7fbe05b9d700 1 -- 172.21.15.55:0/1599764993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0015d230 msgr2=0x7fbe00168a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:51.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.307+0000 7fbe05b9d700 1 --2- 172.21.15.55:0/1599764993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0015d230 0x7fbe00168a30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbe00089340 tx=0x7fbdf8019000 comp rx=0 tx=0).stop 2023-10-06T03:45:51.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.307+0000 7fbe05b9d700 1 -- 172.21.15.55:0/1599764993 shutdown_connections 2023-10-06T03:45:51.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.307+0000 7fbe05b9d700 1 --2- 172.21.15.55:0/1599764993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0015d230 0x7fbe00168a30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.307+0000 7fbe05b9d700 1 --2- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbde40670e0 0x7fbde4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.307+0000 7fbe05b9d700 1 --2- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe00159420 0x7fbe001448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.307+0000 7fbe05b9d700 1 --2- 172.21.15.55:0/1599764993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe001597f0 0x7fbe0015a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.307+0000 7fbe05b9d700 1 -- 172.21.15.55:0/1599764993 >> 172.21.15.55:0/1599764993 conn(0x7fbe000d0660 msgr2=0x7fbe000d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:51.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.308+0000 7fbe05b9d700 1 -- 172.21.15.55:0/1599764993 shutdown_connections 2023-10-06T03:45:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.308+0000 7fbe05b9d700 1 -- 172.21.15.55:0/1599764993 wait complete. 2023-10-06T03:45:51.311 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 17 2023-10-06T03:45:51.328 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:45:51.415 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:51.413+0000 7f911da9b180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:45:51.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.545+0000 7efe3719d700 1 Processor -- start 2023-10-06T03:45:51.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.546+0000 7efe3719d700 1 -- start start 2023-10-06T03:45:51.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.547+0000 7efe3719d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe30156310 0x7efe300c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.547+0000 7efe3719d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efe301566e0 0x7efe300ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.547+0000 7efe3719d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efe300ca300 0x7efe300ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.547+0000 7efe3719d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe300ca860 con 0x7efe300ca300 2023-10-06T03:45:51.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.547+0000 7efe3719d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe300cab40 con 0x7efe301566e0 2023-10-06T03:45:51.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.547+0000 7efe3719d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe300cae20 con 0x7efe30156310 2023-10-06T03:45:51.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.547+0000 7efe3699c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efe300ca300 0x7efe300ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.547+0000 7efe3699c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efe300ca300 0x7efe300ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53070/0 (socket says 172.21.15.55:53070) 2023-10-06T03:45:51.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.547+0000 7efe3699c700 1 -- 172.21.15.55:0/2111569990 learned_addr learned my addr 172.21.15.55:0/2111569990 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:51.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.547+0000 7efe3599a700 1 --2- 172.21.15.55:0/2111569990 >> v2:172.21.15.163:3300/0 conn(0x7efe301566e0 0x7efe300ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.547+0000 7efe3619b700 1 --2- 172.21.15.55:0/2111569990 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe30156310 0x7efe300c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.548+0000 7efe3699c700 1 -- 172.21.15.55:0/2111569990 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe30156310 msgr2=0x7efe300c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:51.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.548+0000 7efe3699c700 1 --2- 172.21.15.55:0/2111569990 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe30156310 0x7efe300c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.548+0000 7efe3699c700 1 -- 172.21.15.55:0/2111569990 >> v2:172.21.15.163:3300/0 conn(0x7efe301566e0 msgr2=0x7efe300ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:51.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.548+0000 7efe3699c700 1 --2- 172.21.15.55:0/2111569990 >> v2:172.21.15.163:3300/0 conn(0x7efe301566e0 0x7efe300ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.548+0000 7efe3699c700 1 -- 172.21.15.55:0/2111569990 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300cb100 con 0x7efe300ca300 2023-10-06T03:45:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.548+0000 7efe3599a700 1 --2- 172.21.15.55:0/2111569990 >> v2:172.21.15.163:3300/0 conn(0x7efe301566e0 0x7efe300ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:45:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.548+0000 7efe3699c700 1 --2- 172.21.15.55:0/2111569990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe300ca300 0x7efe300ae540 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7efe3006d1e0 tx=0x7efe2800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d35a65425b1c5d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.548+0000 7efe35199700 1 -- 172.21.15.55:0/2111569990 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2800b0d0 con 0x7efe300ca300 2023-10-06T03:45:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.548+0000 7efe35199700 1 -- 172.21.15.55:0/2111569990 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe28010070 con 0x7efe300ca300 2023-10-06T03:45:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.548+0000 7efe35199700 1 -- 172.21.15.55:0/2111569990 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2800b480 con 0x7efe300ca300 2023-10-06T03:45:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.549+0000 7efe3719d700 1 -- 172.21.15.55:0/2111569990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe300ca300 msgr2=0x7efe300ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.549+0000 7efe3719d700 1 --2- 172.21.15.55:0/2111569990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe300ca300 0x7efe300ae540 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7efe3006d1e0 tx=0x7efe2800ba50 comp rx=0 tx=0).stop 2023-10-06T03:45:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.549+0000 7efe3719d700 1 -- 172.21.15.55:0/2111569990 shutdown_connections 2023-10-06T03:45:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.549+0000 7efe3719d700 1 --2- 172.21.15.55:0/2111569990 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe300ca300 0x7efe300ae540 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.549+0000 7efe3719d700 1 --2- 172.21.15.55:0/2111569990 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe30156310 0x7efe300c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.549+0000 7efe3719d700 1 --2- 172.21.15.55:0/2111569990 >> v2:172.21.15.163:3300/0 conn(0x7efe301566e0 0x7efe300ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.549+0000 7efe3719d700 1 -- 172.21.15.55:0/2111569990 >> 172.21.15.55:0/2111569990 conn(0x7efe300cc1f0 msgr2=0x7efe300cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:51.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.549+0000 7efe3719d700 1 -- 172.21.15.55:0/2111569990 shutdown_connections 2023-10-06T03:45:51.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.549+0000 7efe3719d700 1 -- 172.21.15.55:0/2111569990 wait complete. 2023-10-06T03:45:51.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.551+0000 7efe3719d700 1 Processor -- start 2023-10-06T03:45:51.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.551+0000 7efe3719d700 1 -- start start 2023-10-06T03:45:51.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.552+0000 7efe3719d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe300ca300 0x7efe30151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.552+0000 7efe3719d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe30156310 0x7efe3014c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.552+0000 7efe3719d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe301566e0 0x7efe3014e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.552+0000 7efe3719d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3014c860 con 0x7efe30156310 2023-10-06T03:45:51.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.552+0000 7efe3719d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe301511d0 con 0x7efe300ca300 2023-10-06T03:45:51.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.552+0000 7efe3719d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe30150ea0 con 0x7efe301566e0 2023-10-06T03:45:51.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.552+0000 7efe3599a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe30156310 0x7efe3014c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.552+0000 7efe3599a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe30156310 0x7efe3014c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53084/0 (socket says 172.21.15.55:53084) 2023-10-06T03:45:51.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.552+0000 7efe3599a700 1 -- 172.21.15.55:0/2930235269 learned_addr learned my addr 172.21.15.55:0/2930235269 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.552+0000 7efe3699c700 1 --2- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe301566e0 0x7efe3014e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.552+0000 7efe3619b700 1 --2- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe300ca300 0x7efe30151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.553+0000 7efe3599a700 1 -- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe301566e0 msgr2=0x7efe3014e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.553+0000 7efe3599a700 1 --2- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe301566e0 0x7efe3014e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.553+0000 7efe3599a700 1 -- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe300ca300 msgr2=0x7efe30151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.553+0000 7efe3599a700 1 --2- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe300ca300 0x7efe30151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.553+0000 7efe3599a700 1 -- 172.21.15.55:0/2930235269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2000f610 con 0x7efe30156310 2023-10-06T03:45:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.553+0000 7efe3599a700 1 --2- 172.21.15.55:0/2930235269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe30156310 0x7efe3014c320 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efe20009de0 tx=0x7efe20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.553+0000 7efe267fc700 1 -- 172.21.15.55:0/2930235269 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe20013ab0 con 0x7efe30156310 2023-10-06T03:45:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.553+0000 7efe267fc700 1 -- 172.21.15.55:0/2930235269 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe20015040 con 0x7efe30156310 2023-10-06T03:45:51.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.553+0000 7efe3719d700 1 -- 172.21.15.55:0/2930235269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2800bcf0 con 0x7efe30156310 2023-10-06T03:45:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.553+0000 7efe267fc700 1 -- 172.21.15.55:0/2930235269 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe20013e60 con 0x7efe30156310 2023-10-06T03:45:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.553+0000 7efe3719d700 1 -- 172.21.15.55:0/2930235269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe300b1720 con 0x7efe30156310 2023-10-06T03:45:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.554+0000 7efe3719d700 1 -- 172.21.15.55:0/2930235269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3006e930 con 0x7efe30156310 2023-10-06T03:45:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.555+0000 7efe267fc700 1 -- 172.21.15.55:0/2930235269 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) v1 ==== 84911+0+0 (secure 0 0 0) 0x7efe2001b610 con 0x7efe30156310 2023-10-06T03:45:51.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.557+0000 7efe267fc700 1 --2- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7efe140670e0 0x7efe14069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.557+0000 7efe267fc700 1 -- 172.21.15.55:0/2930235269 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7057+0+0 (secure 0 0 0) 0x7efe20089420 con 0x7efe30156310 2023-10-06T03:45:51.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.557+0000 7efe3619b700 1 -- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7efe140670e0 msgr2=0x7efe14069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:51.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.557+0000 7efe3619b700 1 --2- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7efe140670e0 0x7efe14069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:45:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.561+0000 7efe267fc700 1 -- 172.21.15.55:0/2930235269 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe20055d10 con 0x7efe30156310 2023-10-06T03:45:51.589 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:51.587+0000 7f911da9b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:45:51.599 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:51.597+0000 7f71e5a54180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:45:51.684 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:51.682+0000 7f911da9b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:45:51.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.758+0000 7efe3619b700 1 -- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7efe140670e0 msgr2=0x7efe14069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:51.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.758+0000 7efe3619b700 1 --2- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7efe140670e0 0x7efe14069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:45:51.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.767+0000 7efe3719d700 1 -- 172.21.15.55:0/2930235269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe3004c3c0 con 0x7efe30156310 2023-10-06T03:45:51.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.768+0000 7efe267fc700 1 -- 172.21.15.55:0/2930235269 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+18436 (secure 0 0 0) 0x7efe20059330 con 0x7efe30156310 2023-10-06T03:45:51.770 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:51.770 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":52,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:45:50.654836+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:34:11.006235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T03:34:11.987296+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6832/85191":"2023-10-07T03:34:33.631661+0000","172.21.15.163:0/2869540037":"2023-10-07T03:34:33.631661+0000","172.21.15.163:0/2697678523":"2023-10-07T03:34:33.631661+0000","172.21.15.163:0/65446995":"2023-10-07T03:34:33.631661+0000","172.21.15.163:6833/85191":"2023-10-07T03:34:33.631661+0000","172.21.15.163:0/2882606989":"2023-10-07T03:34:33.631661+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":5,"length":1},{"begin":10,"length":4},{"begin":17,"length":1},{"begin":24,"length":2},{"begin":29,"length":1},{"begin":34,"length":2},{"begin":39,"length":1},{"begin":45,"length":2},{"begin":50,"length":1},{"begin":56,"length":2},{"begin":61,"length":2},{"begin":67,"length":2},{"begin":73,"length":1},{"begin":78,"length":2},{"begin":83,"length":1},{"begin":88,"length":2},{"begin":94,"length":1},{"begin":99,"length":2},{"begin":103,"length":2},{"begin":109,"length":1},{"begin":155,"length":33}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:45:51.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.770+0000 7efe3719d700 1 -- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7efe140670e0 msgr2=0x7efe14069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:45:51.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.770+0000 7efe3719d700 1 --2- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7efe140670e0 0x7efe14069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.770+0000 7efe3719d700 1 -- 172.21.15.55:0/2930235269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe30156310 msgr2=0x7efe3014c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:51.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.770+0000 7efe3719d700 1 --2- 172.21.15.55:0/2930235269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe30156310 0x7efe3014c320 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efe20009de0 tx=0x7efe20008040 comp rx=0 tx=0).stop 2023-10-06T03:45:51.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.770+0000 7efe3719d700 1 -- 172.21.15.55:0/2930235269 shutdown_connections 2023-10-06T03:45:51.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.770+0000 7efe3719d700 1 --2- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7efe140670e0 0x7efe14069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.770+0000 7efe3719d700 1 --2- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efe301566e0 0x7efe3014e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.770+0000 7efe3719d700 1 --2- 172.21.15.55:0/2930235269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe300ca300 0x7efe30151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.770+0000 7efe3719d700 1 --2- 172.21.15.55:0/2930235269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe30156310 0x7efe3014c320 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.770+0000 7efe3719d700 1 -- 172.21.15.55:0/2930235269 >> 172.21.15.55:0/2930235269 conn(0x7efe300cc1f0 msgr2=0x7efe300c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:51.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.770+0000 7efe3719d700 1 -- 172.21.15.55:0/2930235269 shutdown_connections 2023-10-06T03:45:51.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.770+0000 7efe3719d700 1 -- 172.21.15.55:0/2930235269 wait complete. 2023-10-06T03:45:51.780 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:51.779+0000 7f71e5a54180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:45:51.790 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:45:51.790 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:45:51.882 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:51.881+0000 7f71e5a54180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:45:51.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.977+0000 7ff254fff700 1 Processor -- start 2023-10-06T03:45:51.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.978+0000 7ff254fff700 1 -- start start 2023-10-06T03:45:51.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.979+0000 7ff254fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2500b1850 0x7ff2500b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.979+0000 7ff254fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2500ae680 0x7ff2501572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.979+0000 7ff254fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2500cb300 con 0x7ff2500aead0 2023-10-06T03:45:51.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.979+0000 7ff254fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25006fc30 con 0x7ff2500ae680 2023-10-06T03:45:51.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.979+0000 7ff254fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2500cc260 con 0x7ff2500b1850 2023-10-06T03:45:51.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.979+0000 7ff24e59c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2500b1850 0x7ff2500b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.979+0000 7ff24dd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2500ae680 0x7ff2501572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.979+0000 7ff24e59c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2500b1850 0x7ff2500b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:37856/0 (socket says 172.21.15.55:37856) 2023-10-06T03:45:51.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.979+0000 7ff24ed9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff2500aead0 0x7ff25015b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:60644/0 (socket says 172.21.15.55:60644) 2023-10-06T03:45:51.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.979+0000 7ff24dd9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2500ae680 0x7ff2501572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:50900/0 (socket says 172.21.15.55:50900) 2023-10-06T03:45:51.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.979+0000 7ff24e59c700 1 -- 172.21.15.55:0/481309765 learned_addr learned my addr 172.21.15.55:0/481309765 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:51.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.980+0000 7ff24d59a700 1 -- 172.21.15.55:0/481309765 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663791154 0 0) 0x7ff2500cb300 con 0x7ff2500aead0 2023-10-06T03:45:51.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.980+0000 7ff24e59c700 1 -- 172.21.15.55:0/481309765 >> v2:172.21.15.163:3300/0 conn(0x7ff2500ae680 msgr2=0x7ff2501572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:51.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.980+0000 7ff24e59c700 1 --2- 172.21.15.55:0/481309765 >> v2:172.21.15.163:3300/0 conn(0x7ff2500ae680 0x7ff2501572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.980+0000 7ff24e59c700 1 -- 172.21.15.55:0/481309765 >> v1:172.21.15.163:6789/0 conn(0x7ff2500aead0 legacy=0x7ff25015b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:51.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.980+0000 7ff24e59c700 1 -- 172.21.15.55:0/481309765 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2500cb5e0 con 0x7ff2500b1850 2023-10-06T03:45:51.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.980+0000 7ff24dd9b700 1 --2- 172.21.15.55:0/481309765 >> v2:172.21.15.163:3300/0 conn(0x7ff2500ae680 0x7ff2501572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:45:51.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.980+0000 7ff24e59c700 1 --2- 172.21.15.55:0/481309765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2500b1850 0x7ff2500b1c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff2400099b0 tx=0x7ff240009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=780e8e03a6e2dc8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:51.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.980+0000 7ff24d59a700 1 -- 172.21.15.55:0/481309765 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240009ea0 con 0x7ff2500b1850 2023-10-06T03:45:51.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.980+0000 7ff24d59a700 1 -- 172.21.15.55:0/481309765 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24000b860 con 0x7ff2500b1850 2023-10-06T03:45:51.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.980+0000 7ff24d59a700 1 -- 172.21.15.55:0/481309765 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24000bc00 con 0x7ff2500b1850 2023-10-06T03:45:51.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.981+0000 7ff254fff700 1 -- 172.21.15.55:0/481309765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2500b1850 msgr2=0x7ff2500b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:51.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.981+0000 7ff254fff700 1 --2- 172.21.15.55:0/481309765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2500b1850 0x7ff2500b1c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff2400099b0 tx=0x7ff240009250 comp rx=0 tx=0).stop 2023-10-06T03:45:51.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.981+0000 7ff254fff700 1 -- 172.21.15.55:0/481309765 shutdown_connections 2023-10-06T03:45:51.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.981+0000 7ff254fff700 1 --2- 172.21.15.55:0/481309765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2500b1850 0x7ff2500b1c20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.981+0000 7ff254fff700 1 --2- 172.21.15.55:0/481309765 >> v2:172.21.15.163:3300/0 conn(0x7ff2500ae680 0x7ff2501572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.981+0000 7ff254fff700 1 -- 172.21.15.55:0/481309765 >> 172.21.15.55:0/481309765 conn(0x7ff25006a320 msgr2=0x7ff2500cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:51.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.981+0000 7ff254fff700 1 -- 172.21.15.55:0/481309765 shutdown_connections 2023-10-06T03:45:51.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.981+0000 7ff254fff700 1 -- 172.21.15.55:0/481309765 wait complete. 2023-10-06T03:45:51.985 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:51.979+0000 7f911da9b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:45:51.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.983+0000 7ff254fff700 1 Processor -- start 2023-10-06T03:45:51.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.983+0000 7ff254fff700 1 -- start start 2023-10-06T03:45:51.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.984+0000 7ff254fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2500ae680 0x7ff250159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.984+0000 7ff254fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2500aead0 0x7ff250169c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.984+0000 7ff254fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2500b1850 0x7ff250152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.984+0000 7ff254fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff250158fe0 con 0x7ff2500aead0 2023-10-06T03:45:51.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.984+0000 7ff254fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2501599b0 con 0x7ff2500ae680 2023-10-06T03:45:51.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.984+0000 7ff254fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2501583d0 con 0x7ff2500b1850 2023-10-06T03:45:51.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.984+0000 7ff24dd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2500aead0 0x7ff250169c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.984+0000 7ff24dd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2500aead0 0x7ff250169c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53092/0 (socket says 172.21.15.55:53092) 2023-10-06T03:45:51.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.984+0000 7ff24ed9d700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2500b1850 0x7ff250152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.984+0000 7ff24e59c700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2500ae680 0x7ff250159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:51.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.984+0000 7ff24dd9b700 1 -- 172.21.15.55:0/3226352614 learned_addr learned my addr 172.21.15.55:0/3226352614 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:51.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.985+0000 7ff24dd9b700 1 -- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2500b1850 msgr2=0x7ff250152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:51.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.985+0000 7ff24dd9b700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2500b1850 0x7ff250152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.985+0000 7ff24dd9b700 1 -- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2500ae680 msgr2=0x7ff250159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:51.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.985+0000 7ff24dd9b700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2500ae680 0x7ff250159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:51.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.985+0000 7ff24dd9b700 1 -- 172.21.15.55:0/3226352614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24400d8d0 con 0x7ff2500aead0 2023-10-06T03:45:51.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.985+0000 7ff24e59c700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2500ae680 0x7ff250159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:45:51.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.985+0000 7ff24dd9b700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2500aead0 0x7ff250169c60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff24400df50 tx=0x7ff24400f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:51.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.985+0000 7ff23e7fc700 1 -- 172.21.15.55:0/3226352614 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24400d2e0 con 0x7ff2500aead0 2023-10-06T03:45:51.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.985+0000 7ff23e7fc700 1 -- 172.21.15.55:0/3226352614 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff244013040 con 0x7ff2500aead0 2023-10-06T03:45:51.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.985+0000 7ff254fff700 1 -- 172.21.15.55:0/3226352614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2400094f0 con 0x7ff2500aead0 2023-10-06T03:45:51.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.985+0000 7ff23e7fc700 1 -- 172.21.15.55:0/3226352614 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244010b50 con 0x7ff2500aead0 2023-10-06T03:45:51.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.985+0000 7ff254fff700 1 -- 172.21.15.55:0/3226352614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff250158990 con 0x7ff2500aead0 2023-10-06T03:45:51.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.986+0000 7ff254fff700 1 -- 172.21.15.55:0/3226352614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff250159150 con 0x7ff2500aead0 2023-10-06T03:45:51.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.988+0000 7ff23e7fc700 1 -- 172.21.15.55:0/3226352614 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) v1 ==== 84911+0+0 (secure 0 0 0) 0x7ff244018480 con 0x7ff2500aead0 2023-10-06T03:45:51.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.989+0000 7ff23e7fc700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff2300670e0 0x7ff230069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:51.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.989+0000 7ff23e7fc700 1 -- 172.21.15.55:0/3226352614 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7057+0+0 (secure 0 0 0) 0x7ff244088f90 con 0x7ff2500aead0 2023-10-06T03:45:51.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.989+0000 7ff24e59c700 1 -- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff2300670e0 msgr2=0x7ff230069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:51.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.989+0000 7ff24e59c700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff2300670e0 0x7ff230069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:45:51.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:51.993+0000 7ff23e7fc700 1 -- 172.21.15.55:0/3226352614 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff244055880 con 0x7ff2500aead0 2023-10-06T03:45:52.080 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:52.078+0000 7f911da9b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:45:52.185 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:52.184+0000 7f71e5a54180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:45:52.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.190+0000 7ff24e59c700 1 -- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff2300670e0 msgr2=0x7ff230069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:52.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.190+0000 7ff24e59c700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff2300670e0 0x7ff230069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:45:52.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.239+0000 7ff254fff700 1 -- 172.21.15.55:0/3226352614 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff25004c3c0 con 0x7ff2500aead0 2023-10-06T03:45:52.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.239+0000 7ff23e7fc700 1 -- 172.21.15.55:0/3226352614 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4600 (secure 0 0 0) 0x7ff244058ea0 con 0x7ff2500aead0 2023-10-06T03:45:52.241 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:52.241 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1532010025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1532010025},{"type":"v1","addr":"172.21.15.55:6834","nonce":1532010025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4258042685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":4258042685},{"type":"v1","addr":"172.21.15.55:6835","nonce":4258042685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/430119478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":430119478},{"type":"v1","addr":"172.21.15.163:6836","nonce":430119478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:34:13.003040+0000","modified":"2023-10-06T03:34:16.548740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4903},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4903":{"gid":4903,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6837/1606384901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1606384901},{"type":"v1","addr":"172.21.15.163:6837","nonce":1606384901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T03:45:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.242+0000 7ff254fff700 1 -- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff2300670e0 msgr2=0x7ff230069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:45:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.242+0000 7ff254fff700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff2300670e0 0x7ff230069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.242+0000 7ff254fff700 1 -- 172.21.15.55:0/3226352614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2500aead0 msgr2=0x7ff250169c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.242+0000 7ff254fff700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2500aead0 0x7ff250169c60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff24400df50 tx=0x7ff24400f040 comp rx=0 tx=0).stop 2023-10-06T03:45:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.242+0000 7ff254fff700 1 -- 172.21.15.55:0/3226352614 shutdown_connections 2023-10-06T03:45:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.242+0000 7ff254fff700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7ff2300670e0 0x7ff230069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.242+0000 7ff254fff700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2500b1850 0x7ff250152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.242+0000 7ff254fff700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2500ae680 0x7ff250159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.242+0000 7ff254fff700 1 --2- 172.21.15.55:0/3226352614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2500aead0 0x7ff250169c60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.242+0000 7ff254fff700 1 -- 172.21.15.55:0/3226352614 >> 172.21.15.55:0/3226352614 conn(0x7ff25006a320 msgr2=0x7ff2500ac1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:52.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.242+0000 7ff254fff700 1 -- 172.21.15.55:0/3226352614 shutdown_connections 2023-10-06T03:45:52.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.242+0000 7ff254fff700 1 -- 172.21.15.55:0/3226352614 wait complete. 2023-10-06T03:45:52.245 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 17 2023-10-06T03:45:52.261 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:45:52.287 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:52.286+0000 7f71e5a54180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:45:52.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.449+0000 7f8b5d7ff700 1 Processor -- start 2023-10-06T03:45:52.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.449+0000 7f8b5d7ff700 1 -- start start 2023-10-06T03:45:52.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.450+0000 7f8b5d7ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b580cbf30 0x7f8b580ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:52.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.450+0000 7f8b5d7ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8b580b1420 0x7f8b580c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:52.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.450+0000 7f8b5d7ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b580cc300 con 0x7f8b580ccf90 2023-10-06T03:45:52.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.450+0000 7f8b5d7ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b58156bf0 con 0x7f8b580cbf30 2023-10-06T03:45:52.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.450+0000 7f8b5d7ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b580ae950 con 0x7f8b580b1420 2023-10-06T03:45:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.451+0000 7f8b56ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b580cbf30 0x7f8b580ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.451+0000 7f8b567fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8b580ccf90 0x7f8b580b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42370/0 (socket says 172.21.15.55:42370) 2023-10-06T03:45:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.451+0000 7f8b567fc700 1 -- 172.21.15.55:0/1788877885 learned_addr learned my addr 172.21.15.55:0/1788877885 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.451+0000 7f8b56ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8b580cbf30 0x7f8b580ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53102/0 (socket says 172.21.15.55:53102) 2023-10-06T03:45:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.451+0000 7f8b577fe700 1 --2- 172.21.15.55:0/1788877885 >> v2:172.21.15.163:3300/0 conn(0x7f8b580b1420 0x7f8b580c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.451+0000 7f8b56ffd700 1 -- 172.21.15.55:0/1788877885 >> v2:172.21.15.163:3300/0 conn(0x7f8b580b1420 msgr2=0x7f8b580c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.451+0000 7f8b56ffd700 1 --2- 172.21.15.55:0/1788877885 >> v2:172.21.15.163:3300/0 conn(0x7f8b580b1420 0x7f8b580c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.451+0000 7f8b56ffd700 1 -- 172.21.15.55:0/1788877885 >> v1:172.21.15.55:6789/0 conn(0x7f8b580ccf90 legacy=0x7f8b580b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.451+0000 7f8b56ffd700 1 -- 172.21.15.55:0/1788877885 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b580ad370 con 0x7f8b580cbf30 2023-10-06T03:45:52.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.451+0000 7f8b55ffb700 1 -- 172.21.15.55:0/1788877885 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3633653897 0 0) 0x7f8b580cc300 con 0x7f8b580ccf90 2023-10-06T03:45:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.452+0000 7f8b56ffd700 1 --2- 172.21.15.55:0/1788877885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b580cbf30 0x7f8b580ae410 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8b580b19e0 tx=0x7f8b480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce4f889c56144d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.452+0000 7f8b55ffb700 1 -- 172.21.15.55:0/1788877885 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48011630 con 0x7f8b580cbf30 2023-10-06T03:45:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.452+0000 7f8b55ffb700 1 -- 172.21.15.55:0/1788877885 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b48011790 con 0x7f8b580cbf30 2023-10-06T03:45:52.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.452+0000 7f8b55ffb700 1 -- 172.21.15.55:0/1788877885 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48011b30 con 0x7f8b580cbf30 2023-10-06T03:45:52.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.452+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/1788877885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b580cbf30 msgr2=0x7f8b580ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:52.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.452+0000 7f8b5d7ff700 1 --2- 172.21.15.55:0/1788877885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b580cbf30 0x7f8b580ae410 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8b580b19e0 tx=0x7f8b480091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.452+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/1788877885 shutdown_connections 2023-10-06T03:45:52.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.452+0000 7f8b5d7ff700 1 --2- 172.21.15.55:0/1788877885 >> v2:172.21.15.163:3300/0 conn(0x7f8b580b1420 0x7f8b580c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.452+0000 7f8b5d7ff700 1 --2- 172.21.15.55:0/1788877885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b580cbf30 0x7f8b580ae410 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.452+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/1788877885 >> 172.21.15.55:0/1788877885 conn(0x7f8b5806a320 msgr2=0x7f8b580cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:52.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.453+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/1788877885 shutdown_connections 2023-10-06T03:45:52.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.453+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/1788877885 wait complete. 2023-10-06T03:45:52.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.454+0000 7f8b5d7ff700 1 Processor -- start 2023-10-06T03:45:52.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.454+0000 7f8b5d7ff700 1 -- start start 2023-10-06T03:45:52.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.455+0000 7f8b5d7ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b580b1420 0x7f8b581559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:52.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.455+0000 7f8b5d7ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b580cbf30 0x7f8b5815db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:52.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.455+0000 7f8b5d7ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b580ccf90 0x7f8b580a8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:52.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.455+0000 7f8b5d7ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b580ab4a0 con 0x7f8b580ccf90 2023-10-06T03:45:52.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.455+0000 7f8b5d7ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b580aabe0 con 0x7f8b580cbf30 2023-10-06T03:45:52.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.455+0000 7f8b5d7ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b580aae50 con 0x7f8b580b1420 2023-10-06T03:45:52.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.455+0000 7f8b577fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b580ccf90 0x7f8b580a8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:52.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.455+0000 7f8b56ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b580b1420 0x7f8b581559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:52.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.455+0000 7f8b577fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b580ccf90 0x7f8b580a8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53106/0 (socket says 172.21.15.55:53106) 2023-10-06T03:45:52.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.455+0000 7f8b577fe700 1 -- 172.21.15.55:0/229654867 learned_addr learned my addr 172.21.15.55:0/229654867 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:52.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.455+0000 7f8b567fc700 1 --2- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b580cbf30 0x7f8b5815db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:52.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.456+0000 7f8b577fe700 1 -- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b580b1420 msgr2=0x7f8b581559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:52.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.456+0000 7f8b577fe700 1 --2- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b580b1420 0x7f8b581559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.456+0000 7f8b577fe700 1 -- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b580cbf30 msgr2=0x7f8b5815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:52.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.456+0000 7f8b577fe700 1 --2- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b580cbf30 0x7f8b5815db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.456+0000 7f8b577fe700 1 -- 172.21.15.55:0/229654867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b48009420 con 0x7f8b580ccf90 2023-10-06T03:45:52.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.456+0000 7f8b577fe700 1 --2- 172.21.15.55:0/229654867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b580ccf90 0x7f8b580a8a90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009ec0 tx=0x7f8b5000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:52.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.456+0000 7f8b3effd700 1 -- 172.21.15.55:0/229654867 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5000cb70 con 0x7f8b580ccf90 2023-10-06T03:45:52.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.456+0000 7f8b3effd700 1 -- 172.21.15.55:0/229654867 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5000f040 con 0x7f8b580ccf90 2023-10-06T03:45:52.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.456+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/229654867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b580ab130 con 0x7f8b580ccf90 2023-10-06T03:45:52.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.456+0000 7f8b3effd700 1 -- 172.21.15.55:0/229654867 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b50014480 con 0x7f8b580ccf90 2023-10-06T03:45:52.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.457+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/229654867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b580a9470 con 0x7f8b580ccf90 2023-10-06T03:45:52.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.457+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/229654867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b580b08c0 con 0x7f8b580ccf90 2023-10-06T03:45:52.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.458+0000 7f8b3effd700 1 -- 172.21.15.55:0/229654867 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) v1 ==== 84911+0+0 (secure 0 0 0) 0x7f8b50014830 con 0x7f8b580ccf90 2023-10-06T03:45:52.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.459+0000 7f8b3effd700 1 --2- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b440670e0 0x7f8b44069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:52.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.459+0000 7f8b3effd700 1 -- 172.21.15.55:0/229654867 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7057+0+0 (secure 0 0 0) 0x7f8b50085b10 con 0x7f8b580ccf90 2023-10-06T03:45:52.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.459+0000 7f8b56ffd700 1 -- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b440670e0 msgr2=0x7f8b44069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:52.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.460+0000 7f8b56ffd700 1 --2- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b440670e0 0x7f8b44069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:45:52.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.466+0000 7f8b3effd700 1 -- 172.21.15.55:0/229654867 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b50052400 con 0x7f8b580ccf90 2023-10-06T03:45:52.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.661+0000 7f8b56ffd700 1 -- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b440670e0 msgr2=0x7f8b44069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:52.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.661+0000 7f8b56ffd700 1 --2- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b440670e0 0x7f8b44069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:45:52.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.717+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/229654867 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b580ab650 con 0x7f8b580ccf90 2023-10-06T03:45:52.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.718+0000 7f8b3effd700 1 -- 172.21.15.55:0/229654867 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4600 (secure 0 0 0) 0x7f8b50083e60 con 0x7f8b580ccf90 2023-10-06T03:45:52.720 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:52.720 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1532010025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1532010025},{"type":"v1","addr":"172.21.15.55:6834","nonce":1532010025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4258042685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":4258042685},{"type":"v1","addr":"172.21.15.55:6835","nonce":4258042685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/430119478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":430119478},{"type":"v1","addr":"172.21.15.163:6836","nonce":430119478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:34:13.003040+0000","modified":"2023-10-06T03:34:16.548740+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4903},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4903":{"gid":4903,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.163:6837/1606384901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6835","nonce":1606384901},{"type":"v1","addr":"172.21.15.163:6837","nonce":1606384901}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T03:45:52.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.721+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b440670e0 msgr2=0x7f8b44069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:45:52.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.721+0000 7f8b5d7ff700 1 --2- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b440670e0 0x7f8b44069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.721+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/229654867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b580ccf90 msgr2=0x7f8b580a8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:52.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.721+0000 7f8b5d7ff700 1 --2- 172.21.15.55:0/229654867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b580ccf90 0x7f8b580a8a90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009ec0 tx=0x7f8b5000b040 comp rx=0 tx=0).stop 2023-10-06T03:45:52.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.721+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/229654867 shutdown_connections 2023-10-06T03:45:52.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.721+0000 7f8b5d7ff700 1 --2- 172.21.15.55:0/229654867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8b580ccf90 0x7f8b580a8a90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.721+0000 7f8b5d7ff700 1 --2- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7f8b440670e0 0x7f8b44069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.721+0000 7f8b5d7ff700 1 --2- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8b580b1420 0x7f8b581559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.721+0000 7f8b5d7ff700 1 --2- 172.21.15.55:0/229654867 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8b580cbf30 0x7f8b5815db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.721+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/229654867 >> 172.21.15.55:0/229654867 conn(0x7f8b5806a320 msgr2=0x7f8b5806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:52.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.721+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/229654867 shutdown_connections 2023-10-06T03:45:52.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.721+0000 7f8b5d7ff700 1 -- 172.21.15.55:0/229654867 wait complete. 2023-10-06T03:45:52.724 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 17 2023-10-06T03:45:52.742 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:45:52.794 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:52.792+0000 7f911da9b180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:45:52.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.970+0000 7fdd689fe700 1 Processor -- start 2023-10-06T03:45:52.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.970+0000 7fdd689fe700 1 -- start start 2023-10-06T03:45:52.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.971+0000 7fdd689fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdd641597f0 0x7fdd640b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:52.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.971+0000 7fdd689fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdd6415d230 0x7fdd6415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.971+0000 7fdd689fe700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6415db70 con 0x7fdd64159420 2023-10-06T03:45:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.971+0000 7fdd689fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd640b11d0 con 0x7fdd641597f0 2023-10-06T03:45:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.971+0000 7fdd689fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd640b14b0 con 0x7fdd6415d230 2023-10-06T03:45:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.971+0000 7fdd6259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdd641597f0 0x7fdd640b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.971+0000 7fdd6259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdd641597f0 0x7fdd640b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53114/0 (socket says 172.21.15.55:53114) 2023-10-06T03:45:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.971+0000 7fdd6259c700 1 -- 172.21.15.55:0/2062515214 learned_addr learned my addr 172.21.15.55:0/2062515214 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.971+0000 7fdd62d9d700 1 --1- 172.21.15.55:0/2062515214 >> v1:172.21.15.55:6789/0 conn(0x7fdd64159420 0x7fdd640b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42376/0 (socket says 172.21.15.55:42376) 2023-10-06T03:45:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.972+0000 7fdd6359e700 1 --2- 172.21.15.55:0/2062515214 >> v2:172.21.15.163:3300/0 conn(0x7fdd6415d230 0x7fdd6415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:52.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.972+0000 7fdd6259c700 1 -- 172.21.15.55:0/2062515214 >> v2:172.21.15.163:3300/0 conn(0x7fdd6415d230 msgr2=0x7fdd6415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:52.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.972+0000 7fdd6259c700 1 --2- 172.21.15.55:0/2062515214 >> v2:172.21.15.163:3300/0 conn(0x7fdd6415d230 0x7fdd6415d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.972+0000 7fdd6259c700 1 -- 172.21.15.55:0/2062515214 >> v1:172.21.15.55:6789/0 conn(0x7fdd64159420 legacy=0x7fdd640b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:52.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.972+0000 7fdd6259c700 1 -- 172.21.15.55:0/2062515214 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd640b1790 con 0x7fdd641597f0 2023-10-06T03:45:52.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.972+0000 7fdd61d9b700 1 -- 172.21.15.55:0/2062515214 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969450911 0 0) 0x7fdd6415db70 con 0x7fdd64159420 2023-10-06T03:45:52.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.972+0000 7fdd6259c700 1 --2- 172.21.15.55:0/2062515214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641597f0 0x7fdd640b2270 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdd6415d170 tx=0x7fdd58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c90d241bd75a091 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:52.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.972+0000 7fdd61d9b700 1 -- 172.21.15.55:0/2062515214 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800b860 con 0x7fdd641597f0 2023-10-06T03:45:52.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.973+0000 7fdd61d9b700 1 -- 172.21.15.55:0/2062515214 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5800b9c0 con 0x7fdd641597f0 2023-10-06T03:45:52.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.973+0000 7fdd61d9b700 1 -- 172.21.15.55:0/2062515214 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800bd60 con 0x7fdd641597f0 2023-10-06T03:45:52.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.973+0000 7fdd689fe700 1 -- 172.21.15.55:0/2062515214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641597f0 msgr2=0x7fdd640b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:52.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.973+0000 7fdd689fe700 1 --2- 172.21.15.55:0/2062515214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641597f0 0x7fdd640b2270 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdd6415d170 tx=0x7fdd58009250 comp rx=0 tx=0).stop 2023-10-06T03:45:52.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.973+0000 7fdd689fe700 1 -- 172.21.15.55:0/2062515214 shutdown_connections 2023-10-06T03:45:52.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.973+0000 7fdd689fe700 1 --2- 172.21.15.55:0/2062515214 >> v2:172.21.15.163:3300/0 conn(0x7fdd6415d230 0x7fdd6415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.973+0000 7fdd689fe700 1 --2- 172.21.15.55:0/2062515214 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641597f0 0x7fdd640b2270 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.973+0000 7fdd689fe700 1 -- 172.21.15.55:0/2062515214 >> 172.21.15.55:0/2062515214 conn(0x7fdd640d0660 msgr2=0x7fdd640d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:52.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.973+0000 7fdd689fe700 1 -- 172.21.15.55:0/2062515214 shutdown_connections 2023-10-06T03:45:52.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.973+0000 7fdd689fe700 1 -- 172.21.15.55:0/2062515214 wait complete. 2023-10-06T03:45:52.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.975+0000 7fdd689fe700 1 Processor -- start 2023-10-06T03:45:52.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.975+0000 7fdd689fe700 1 -- start start 2023-10-06T03:45:52.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd689fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd64159420 0x7fdd64147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:52.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd689fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641597f0 0x7fdd640c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:52.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd689fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd6415d230 0x7fdd64169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:52.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd689fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6415bf90 con 0x7fdd641597f0 2023-10-06T03:45:52.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd689fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd641476a0 con 0x7fdd6415d230 2023-10-06T03:45:52.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd689fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd640c5940 con 0x7fdd64159420 2023-10-06T03:45:52.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd6259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641597f0 0x7fdd640c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:52.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd6259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641597f0 0x7fdd640c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53120/0 (socket says 172.21.15.55:53120) 2023-10-06T03:45:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd6259c700 1 -- 172.21.15.55:0/1014789269 learned_addr learned my addr 172.21.15.55:0/1014789269 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd62d9d700 1 --2- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd64159420 0x7fdd64147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd6359e700 1 --2- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd6415d230 0x7fdd64169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd6259c700 1 -- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd64159420 msgr2=0x7fdd64147160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd6259c700 1 --2- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd64159420 0x7fdd64147160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd6259c700 1 -- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd6415d230 msgr2=0x7fdd64169cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd6259c700 1 --2- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd6415d230 0x7fdd64169cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.976+0000 7fdd6259c700 1 -- 172.21.15.55:0/1014789269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd580094f0 con 0x7fdd641597f0 2023-10-06T03:45:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.977+0000 7fdd6259c700 1 --2- 172.21.15.55:0/1014789269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641597f0 0x7fdd640c5400 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdd64089340 tx=0x7fdd58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:52.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.977+0000 7fdd4affd700 1 -- 172.21.15.55:0/1014789269 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58011e00 con 0x7fdd641597f0 2023-10-06T03:45:52.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.977+0000 7fdd4affd700 1 -- 172.21.15.55:0/1014789269 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd58024440 con 0x7fdd641597f0 2023-10-06T03:45:52.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.977+0000 7fdd4affd700 1 -- 172.21.15.55:0/1014789269 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58024800 con 0x7fdd641597f0 2023-10-06T03:45:52.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.977+0000 7fdd689fe700 1 -- 172.21.15.55:0/1014789269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6415bb00 con 0x7fdd641597f0 2023-10-06T03:45:52.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.977+0000 7fdd689fe700 1 -- 172.21.15.55:0/1014789269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6415b1d0 con 0x7fdd641597f0 2023-10-06T03:45:52.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.979+0000 7fdd689fe700 1 -- 172.21.15.55:0/1014789269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6404c3c0 con 0x7fdd641597f0 2023-10-06T03:45:52.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.979+0000 7fdd4affd700 1 -- 172.21.15.55:0/1014789269 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) v1 ==== 84911+0+0 (secure 0 0 0) 0x7fdd5802d440 con 0x7fdd641597f0 2023-10-06T03:45:52.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.981+0000 7fdd4affd700 1 --2- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fdd500670e0 0x7fdd50069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:52.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.981+0000 7fdd4affd700 1 -- 172.21.15.55:0/1014789269 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7057+0+0 (secure 0 0 0) 0x7fdd580868b0 con 0x7fdd641597f0 2023-10-06T03:45:52.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.981+0000 7fdd62d9d700 1 -- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fdd500670e0 msgr2=0x7fdd50069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:52.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.981+0000 7fdd62d9d700 1 --2- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fdd500670e0 0x7fdd50069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:45:52.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:52.985+0000 7fdd4affd700 1 -- 172.21.15.55:0/1014789269 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd580532b0 con 0x7fdd641597f0 2023-10-06T03:45:53.062 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:53.061+0000 7f71e5a54180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:45:53.083 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:53.081+0000 7f911da9b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:45:53.176 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:53.174+0000 7f911da9b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:45:53.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.182+0000 7fdd62d9d700 1 -- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fdd500670e0 msgr2=0x7fdd50069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:53.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.182+0000 7fdd62d9d700 1 --2- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fdd500670e0 0x7fdd50069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:45:53.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.204+0000 7fdd689fe700 1 -- 172.21.15.55:0/1014789269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd6406a160 con 0x7fdd641597f0 2023-10-06T03:45:53.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.205+0000 7fdd4affd700 1 -- 172.21.15.55:0/1014789269 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+18436 (secure 0 0 0) 0x7fdd580568d0 con 0x7fdd641597f0 2023-10-06T03:45:53.207 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:45:53.207 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":52,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:45:50.654836+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:34:11.006235+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T03:34:11.987296+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6832/85191":"2023-10-07T03:34:33.631661+0000","172.21.15.163:0/2869540037":"2023-10-07T03:34:33.631661+0000","172.21.15.163:0/2697678523":"2023-10-07T03:34:33.631661+0000","172.21.15.163:0/65446995":"2023-10-07T03:34:33.631661+0000","172.21.15.163:6833/85191":"2023-10-07T03:34:33.631661+0000","172.21.15.163:0/2882606989":"2023-10-07T03:34:33.631661+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":5,"length":1},{"begin":10,"length":4},{"begin":17,"length":1},{"begin":24,"length":2},{"begin":29,"length":1},{"begin":34,"length":2},{"begin":39,"length":1},{"begin":45,"length":2},{"begin":50,"length":1},{"begin":56,"length":2},{"begin":61,"length":2},{"begin":67,"length":2},{"begin":73,"length":1},{"begin":78,"length":2},{"begin":83,"length":1},{"begin":88,"length":2},{"begin":94,"length":1},{"begin":99,"length":2},{"begin":103,"length":2},{"begin":109,"length":1},{"begin":155,"length":33}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:45:53.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.207+0000 7fdd689fe700 1 -- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fdd500670e0 msgr2=0x7fdd50069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:45:53.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.207+0000 7fdd689fe700 1 --2- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fdd500670e0 0x7fdd50069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.207+0000 7fdd689fe700 1 -- 172.21.15.55:0/1014789269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641597f0 msgr2=0x7fdd640c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:53.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.207+0000 7fdd689fe700 1 --2- 172.21.15.55:0/1014789269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641597f0 0x7fdd640c5400 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdd64089340 tx=0x7fdd58017000 comp rx=0 tx=0).stop 2023-10-06T03:45:53.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.208+0000 7fdd689fe700 1 -- 172.21.15.55:0/1014789269 shutdown_connections 2023-10-06T03:45:53.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.208+0000 7fdd689fe700 1 --2- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdd6415d230 0x7fdd64169cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.208+0000 7fdd689fe700 1 --2- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fdd500670e0 0x7fdd50069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.208+0000 7fdd689fe700 1 --2- 172.21.15.55:0/1014789269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdd64159420 0x7fdd64147160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.208+0000 7fdd689fe700 1 --2- 172.21.15.55:0/1014789269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdd641597f0 0x7fdd640c5400 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.208+0000 7fdd689fe700 1 -- 172.21.15.55:0/1014789269 >> 172.21.15.55:0/1014789269 conn(0x7fdd640d0660 msgr2=0x7fdd64157ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:53.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.208+0000 7fdd689fe700 1 -- 172.21.15.55:0/1014789269 shutdown_connections 2023-10-06T03:45:53.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.208+0000 7fdd689fe700 1 -- 172.21.15.55:0/1014789269 wait complete. 2023-10-06T03:45:53.230 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:45:53.331 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:53.329+0000 7f911da9b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:45:53.360 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:53.358+0000 7f71e5a54180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:45:53.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.419+0000 7fbc7e79d700 1 Processor -- start 2023-10-06T03:45:53.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.420+0000 7fbc7e79d700 1 -- start start 2023-10-06T03:45:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.421+0000 7fbc7e79d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbc781595a0 0x7fbc780b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.421+0000 7fbc7e79d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbc780c6c60 0x7fbc780c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.421+0000 7fbc7e79d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc780b1410 con 0x7fbc780b1de0 2023-10-06T03:45:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.421+0000 7fbc7e79d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc780b2b90 con 0x7fbc780c6c60 2023-10-06T03:45:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.421+0000 7fbc7e79d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc78063230 con 0x7fbc781595a0 2023-10-06T03:45:53.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.422+0000 7fbc7cf9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbc780c6c60 0x7fbc780c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:53.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.422+0000 7fbc7df9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbc780b1de0 0x7fbc7815d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57188/0 (socket says 172.21.15.55:57188) 2023-10-06T03:45:53.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.422+0000 7fbc7df9c700 1 -- 172.21.15.55:0/382024269 learned_addr learned my addr 172.21.15.55:0/382024269 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:53.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.422+0000 7fbc7d79b700 1 --2- 172.21.15.55:0/382024269 >> v2:172.21.15.163:3300/0 conn(0x7fbc781595a0 0x7fbc780b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:53.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.422+0000 7fbc6ffff700 1 -- 172.21.15.55:0/382024269 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945980902 0 0) 0x7fbc780b1410 con 0x7fbc780b1de0 2023-10-06T03:45:53.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.422+0000 7fbc7cf9a700 1 -- 172.21.15.55:0/382024269 >> v2:172.21.15.163:3300/0 conn(0x7fbc781595a0 msgr2=0x7fbc780b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:53.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.422+0000 7fbc7cf9a700 1 --2- 172.21.15.55:0/382024269 >> v2:172.21.15.163:3300/0 conn(0x7fbc781595a0 0x7fbc780b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.422+0000 7fbc7cf9a700 1 -- 172.21.15.55:0/382024269 >> v1:172.21.15.55:6789/0 conn(0x7fbc780b1de0 legacy=0x7fbc7815d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:53.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.422+0000 7fbc7cf9a700 1 -- 172.21.15.55:0/382024269 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc780634f0 con 0x7fbc780c6c60 2023-10-06T03:45:53.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.422+0000 7fbc7cf9a700 1 --2- 172.21.15.55:0/382024269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc780c6c60 0x7fbc780c8120 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbc780c8d10 tx=0x7fbc680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6ebb044db6c029 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:53.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.423+0000 7fbc6ffff700 1 -- 172.21.15.55:0/382024269 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68011630 con 0x7fbc780c6c60 2023-10-06T03:45:53.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.423+0000 7fbc6ffff700 1 -- 172.21.15.55:0/382024269 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc68011790 con 0x7fbc780c6c60 2023-10-06T03:45:53.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.423+0000 7fbc6ffff700 1 -- 172.21.15.55:0/382024269 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc68011b30 con 0x7fbc780c6c60 2023-10-06T03:45:53.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.423+0000 7fbc7e79d700 1 -- 172.21.15.55:0/382024269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc780c6c60 msgr2=0x7fbc780c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:53.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.423+0000 7fbc7e79d700 1 --2- 172.21.15.55:0/382024269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc780c6c60 0x7fbc780c8120 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbc780c8d10 tx=0x7fbc680091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.424+0000 7fbc7e79d700 1 -- 172.21.15.55:0/382024269 shutdown_connections 2023-10-06T03:45:53.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.424+0000 7fbc7e79d700 1 --2- 172.21.15.55:0/382024269 >> v2:172.21.15.163:3300/0 conn(0x7fbc781595a0 0x7fbc780b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.424+0000 7fbc7e79d700 1 --2- 172.21.15.55:0/382024269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc780c6c60 0x7fbc780c8120 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.424+0000 7fbc7e79d700 1 -- 172.21.15.55:0/382024269 >> 172.21.15.55:0/382024269 conn(0x7fbc7806ddb0 msgr2=0x7fbc7806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:53.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.424+0000 7fbc7e79d700 1 -- 172.21.15.55:0/382024269 shutdown_connections 2023-10-06T03:45:53.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.424+0000 7fbc7e79d700 1 -- 172.21.15.55:0/382024269 wait complete. 2023-10-06T03:45:53.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.425+0000 7fbc7e79d700 1 Processor -- start 2023-10-06T03:45:53.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.425+0000 7fbc7e79d700 1 -- start start 2023-10-06T03:45:53.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.426+0000 7fbc7e79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc780b1de0 0x7fbc78151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:53.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.426+0000 7fbc7e79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc780c6c60 0x7fbc781570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:53.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.426+0000 7fbc7e79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc781595a0 0x7fbc78157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:53.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.426+0000 7fbc7e79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc780c4030 con 0x7fbc780b1de0 2023-10-06T03:45:53.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.426+0000 7fbc7e79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc780c54a0 con 0x7fbc780c6c60 2023-10-06T03:45:53.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.426+0000 7fbc7e79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc780c5740 con 0x7fbc781595a0 2023-10-06T03:45:53.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.426+0000 7fbc7d79b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc780b1de0 0x7fbc78151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:53.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.426+0000 7fbc7d79b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc780b1de0 0x7fbc78151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52926/0 (socket says 172.21.15.55:52926) 2023-10-06T03:45:53.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.426+0000 7fbc7d79b700 1 -- 172.21.15.55:0/2905236997 learned_addr learned my addr 172.21.15.55:0/2905236997 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:53.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.427+0000 7fbc7cf9a700 1 --2- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc780c6c60 0x7fbc781570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:53.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.427+0000 7fbc7df9c700 1 --2- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc781595a0 0x7fbc78157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:53.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.427+0000 7fbc7d79b700 1 -- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc781595a0 msgr2=0x7fbc78157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:53.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.427+0000 7fbc7d79b700 1 --2- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc781595a0 0x7fbc78157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.427+0000 7fbc7d79b700 1 -- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc780c6c60 msgr2=0x7fbc781570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:53.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.427+0000 7fbc7d79b700 1 --2- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc780c6c60 0x7fbc781570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.427+0000 7fbc7d79b700 1 -- 172.21.15.55:0/2905236997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc68009420 con 0x7fbc780b1de0 2023-10-06T03:45:53.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.427+0000 7fbc7d79b700 1 --2- 172.21.15.55:0/2905236997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc780b1de0 0x7fbc78151c30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400e930 tx=0x7fbc740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:53.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.427+0000 7fbc6d7fa700 1 -- 172.21.15.55:0/2905236997 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74013680 con 0x7fbc780b1de0 2023-10-06T03:45:53.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.428+0000 7fbc6d7fa700 1 -- 172.21.15.55:0/2905236997 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc740137e0 con 0x7fbc780b1de0 2023-10-06T03:45:53.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.428+0000 7fbc7e79d700 1 -- 172.21.15.55:0/2905236997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc780c5a20 con 0x7fbc780b1de0 2023-10-06T03:45:53.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.428+0000 7fbc6d7fa700 1 -- 172.21.15.55:0/2905236997 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74013b80 con 0x7fbc780b1de0 2023-10-06T03:45:53.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.428+0000 7fbc7e79d700 1 -- 172.21.15.55:0/2905236997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc780c5fe0 con 0x7fbc780b1de0 2023-10-06T03:45:53.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.429+0000 7fbc7e79d700 1 -- 172.21.15.55:0/2905236997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc780c6350 con 0x7fbc780b1de0 2023-10-06T03:45:53.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.430+0000 7fbc6d7fa700 1 -- 172.21.15.55:0/2905236997 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 13) v1 ==== 84911+0+0 (secure 0 0 0) 0x7fbc7400c460 con 0x7fbc780b1de0 2023-10-06T03:45:53.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.431+0000 7fbc6d7fa700 1 --2- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbc5c0670e0 0x7fbc5c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:53.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.431+0000 7fbc6d7fa700 1 -- 172.21.15.55:0/2905236997 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7057+0+0 (secure 0 0 0) 0x7fbc74084fb0 con 0x7fbc780b1de0 2023-10-06T03:45:53.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.431+0000 7fbc7cf9a700 1 -- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbc5c0670e0 msgr2=0x7fbc5c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:53.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.431+0000 7fbc7cf9a700 1 --2- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbc5c0670e0 0x7fbc5c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:45:53.435 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:53.433+0000 7f911da9b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:45:53.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.435+0000 7fbc6d7fa700 1 -- 172.21.15.55:0/2905236997 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc740519b0 con 0x7fbc780b1de0 2023-10-06T03:45:53.463 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:53.462+0000 7f71e5a54180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:45:53.623 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:53.621+0000 7f71e5a54180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:45:53.626 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:53.624+0000 7f911da9b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:45:53.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.632+0000 7fbc7cf9a700 1 -- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbc5c0670e0 msgr2=0x7fbc5c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6832/85191 2023-10-06T03:45:53.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.632+0000 7fbc7cf9a700 1 --2- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbc5c0670e0 0x7fbc5c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:45:53.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.690+0000 7fbc7e79d700 1 -- 172.21.15.55:0/2905236997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbc7804c3c0 con 0x7fbc780b1de0 2023-10-06T03:45:53.692 INFO:tasks.ceph.mon.a.smithi055.stderr:2023-10-06T03:45:53.691+0000 7fe524ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:45:53.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.697+0000 7fbc6d7fa700 1 -- 172.21.15.55:0/2905236997 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbc74054fd0 con 0x7fbc780b1de0 2023-10-06T03:45:53.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.700+0000 7fbc7e79d700 1 -- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbc5c0670e0 msgr2=0x7fbc5c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:45:53.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.700+0000 7fbc7e79d700 1 --2- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbc5c0670e0 0x7fbc5c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.700+0000 7fbc7e79d700 1 -- 172.21.15.55:0/2905236997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc780b1de0 msgr2=0x7fbc78151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:53.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.700+0000 7fbc7e79d700 1 --2- 172.21.15.55:0/2905236997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc780b1de0 0x7fbc78151c30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400e930 tx=0x7fbc740091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.700+0000 7fbc7e79d700 1 -- 172.21.15.55:0/2905236997 shutdown_connections 2023-10-06T03:45:53.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.700+0000 7fbc7e79d700 1 --2- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:6832/85191,v1:172.21.15.163:6833/85191] conn(0x7fbc5c0670e0 0x7fbc5c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.700+0000 7fbc7e79d700 1 --2- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc781595a0 0x7fbc78157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.700+0000 7fbc7e79d700 1 --2- 172.21.15.55:0/2905236997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc780b1de0 0x7fbc78151c30 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.700+0000 7fbc7e79d700 1 --2- 172.21.15.55:0/2905236997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc780c6c60 0x7fbc781570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.700+0000 7fbc7e79d700 1 -- 172.21.15.55:0/2905236997 >> 172.21.15.55:0/2905236997 conn(0x7fbc7806ddb0 msgr2=0x7fbc781585b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:53.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.700+0000 7fbc7e79d700 1 -- 172.21.15.55:0/2905236997 shutdown_connections 2023-10-06T03:45:53.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.701+0000 7fbc7e79d700 1 -- 172.21.15.55:0/2905236997 wait complete. 2023-10-06T03:45:53.704 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:45:53.722 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:45:53.723 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:53.720+0000 7f911da9b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:45:53.734 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:53.733+0000 7f71e5a54180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:45:53.759 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2023-10-06T03:45:53.818 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:53.816+0000 7f911da9b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:45:53.911 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:53.909+0000 7f911da9b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:45:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.914+0000 7faecf59e700 1 Processor -- start 2023-10-06T03:45:53.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.914+0000 7faecf59e700 1 -- start start 2023-10-06T03:45:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.915+0000 7faecf59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00b16d0 0x7faed00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.915+0000 7faecf59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faed00b33b0 0x7faed00b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.915+0000 7faecf59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed00b2080 con 0x7faed00b4e70 2023-10-06T03:45:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.915+0000 7faecf59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed00cfb00 con 0x7faed00b33b0 2023-10-06T03:45:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.915+0000 7faecf59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed00b5c40 con 0x7faed00b16d0 2023-10-06T03:45:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.915+0000 7faeced9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faed00b33b0 0x7faed00b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.915+0000 7faecdd9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00b16d0 0x7faed00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:53.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.915+0000 7faece59c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7faed00b4e70 0x7faed00ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:46932/0 (socket says 172.21.15.55:46932) 2023-10-06T03:45:53.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.915+0000 7faece59c700 1 -- 172.21.15.55:0/1048669535 learned_addr learned my addr 172.21.15.55:0/1048669535 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:53.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.916+0000 7faeced9d700 1 -- 172.21.15.55:0/1048669535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00b16d0 msgr2=0x7faed00b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:53.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.916+0000 7faeced9d700 1 --2- 172.21.15.55:0/1048669535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00b16d0 0x7faed00b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.916+0000 7faeced9d700 1 -- 172.21.15.55:0/1048669535 >> v1:172.21.15.163:6789/0 conn(0x7faed00b4e70 legacy=0x7faed00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.916+0000 7faeced9d700 1 -- 172.21.15.55:0/1048669535 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00b5f20 con 0x7faed00b33b0 2023-10-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.916+0000 7faecdd9b700 1 --2- 172.21.15.55:0/1048669535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00b16d0 0x7faed00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.916+0000 7faeced9d700 1 --2- 172.21.15.55:0/1048669535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed00b33b0 0x7faed00b3830 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faec8009e60 tx=0x7faec8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39c975184b282128 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.917+0000 7faecd59a700 1 -- 172.21.15.55:0/1048669535 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec800e070 con 0x7faed00b33b0 2023-10-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.917+0000 7faecd59a700 1 -- 172.21.15.55:0/1048669535 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec800bb60 con 0x7faed00b33b0 2023-10-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.917+0000 7faecd59a700 1 -- 172.21.15.55:0/1048669535 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8012430 con 0x7faed00b33b0 2023-10-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.917+0000 7faecf59e700 1 -- 172.21.15.55:0/1048669535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed00b33b0 msgr2=0x7faed00b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.917+0000 7faecf59e700 1 --2- 172.21.15.55:0/1048669535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed00b33b0 0x7faed00b3830 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faec8009e60 tx=0x7faec8009780 comp rx=0 tx=0).stop 2023-10-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.918+0000 7faecf59e700 1 -- 172.21.15.55:0/1048669535 shutdown_connections 2023-10-06T03:45:53.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.918+0000 7faecf59e700 1 --2- 172.21.15.55:0/1048669535 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed00b33b0 0x7faed00b3830 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.918+0000 7faecf59e700 1 --2- 172.21.15.55:0/1048669535 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00b16d0 0x7faed00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.918+0000 7faecf59e700 1 -- 172.21.15.55:0/1048669535 >> 172.21.15.55:0/1048669535 conn(0x7faed00d10c0 msgr2=0x7faed00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:53.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.918+0000 7faecf59e700 1 -- 172.21.15.55:0/1048669535 shutdown_connections 2023-10-06T03:45:53.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.918+0000 7faecf59e700 1 -- 172.21.15.55:0/1048669535 wait complete. 2023-10-06T03:45:53.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.919+0000 7faecf59e700 1 Processor -- start 2023-10-06T03:45:53.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.919+0000 7faecf59e700 1 -- start start 2023-10-06T03:45:53.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.920+0000 7faecf59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed00b16d0 0x7faed015e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:53.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.920+0000 7faecf59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed00b33b0 0x7faed016d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:53.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.920+0000 7faecf59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00b4e70 0x7faed016d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:53.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.920+0000 7faecf59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00c7e00 con 0x7faed00b33b0 2023-10-06T03:45:53.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.920+0000 7faecf59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00c80e0 con 0x7faed00b16d0 2023-10-06T03:45:53.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.920+0000 7faecf59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed00c83c0 con 0x7faed00b4e70 2023-10-06T03:45:53.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.921+0000 7faece59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed00b16d0 0x7faed015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:53.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.921+0000 7faece59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed00b16d0 0x7faed015e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:34186/0 (socket says 172.21.15.55:34186) 2023-10-06T03:45:53.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.921+0000 7faece59c700 1 -- 172.21.15.55:0/117509513 learned_addr learned my addr 172.21.15.55:0/117509513 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:53.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.921+0000 7faecdd9b700 1 --2- 172.21.15.55:0/117509513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed00b33b0 0x7faed016d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:53.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.922+0000 7faecdd9b700 1 -- 172.21.15.55:0/117509513 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00b4e70 msgr2=0x7faed016d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:53.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.922+0000 7faecdd9b700 1 --2- 172.21.15.55:0/117509513 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00b4e70 0x7faed016d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.922+0000 7faecdd9b700 1 -- 172.21.15.55:0/117509513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed00b16d0 msgr2=0x7faed015e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:53.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.922+0000 7faecdd9b700 1 --2- 172.21.15.55:0/117509513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed00b16d0 0x7faed015e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:53.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.922+0000 7faecdd9b700 1 -- 172.21.15.55:0/117509513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec4009420 con 0x7faed00b33b0 2023-10-06T03:45:53.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.923+0000 7faecdd9b700 1 --2- 172.21.15.55:0/117509513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed00b33b0 0x7faed016d310 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faec400ec80 tx=0x7faec40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:53.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.925+0000 7faebe7fc700 1 -- 172.21.15.55:0/117509513 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec400c9d0 con 0x7faed00b33b0 2023-10-06T03:45:53.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.926+0000 7faecf59e700 1 -- 172.21.15.55:0/117509513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec8009a20 con 0x7faed00b33b0 2023-10-06T03:45:53.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.926+0000 7faebe7fc700 1 -- 172.21.15.55:0/117509513 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec400cb30 con 0x7faed00b33b0 2023-10-06T03:45:53.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.926+0000 7faebe7fc700 1 -- 172.21.15.55:0/117509513 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec4013470 con 0x7faed00b33b0 2023-10-06T03:45:53.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.926+0000 7faecf59e700 1 -- 172.21.15.55:0/117509513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed00c8980 con 0x7faed00b33b0 2023-10-06T03:45:53.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.927+0000 7faecf59e700 1 -- 172.21.15.55:0/117509513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed00c8ba0 con 0x7faed00b33b0 2023-10-06T03:45:53.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.928+0000 7faebe7fc700 1 -- 172.21.15.55:0/117509513 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 14) v1 ==== 84451+0+0 (secure 0 0 0) 0x7faec40138a0 con 0x7faed00b33b0 2023-10-06T03:45:53.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.930+0000 7faebe7fc700 1 -- 172.21.15.55:0/117509513 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7096+0+0 (secure 0 0 0) 0x7faec4084a60 con 0x7faed00b33b0 2023-10-06T03:45:53.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:53.933+0000 7faebe7fc700 1 -- 172.21.15.55:0/117509513 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faec4051a30 con 0x7faed00b33b0 2023-10-06T03:45:53.939 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:53.938+0000 7f71e5a54180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:45:54.001 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:53.999+0000 7f9099644700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:45:54.001 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:53.999+0000 7f9099644700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:45:54.001 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:53.999+0000 7f9099644700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:45:54.001 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:53.999+0000 7f9099644700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:45:54.001 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:53.999+0000 7f9099644700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:45:54.003 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:54.001+0000 7f9097640700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:45:54.003 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:54.001+0000 7f9097640700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:45:54.003 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:54.001+0000 7f9097640700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:45:54.003 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:54.001+0000 7f9097640700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:45:54.003 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:45:54.001+0000 7f9097640700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:45:54.042 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:54.041+0000 7f71e5a54180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:45:54.137 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:54.135+0000 7f71e5a54180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:45:54.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.189+0000 7faecf59e700 1 -- 172.21.15.55:0/117509513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faed004bb80 con 0x7faed00b33b0 2023-10-06T03:45:54.233 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:45:54.232+0000 7f71e5a54180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:45:54.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.719+0000 7faebe7fc700 1 -- 172.21.15.55:0/117509513 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7faec4055050 con 0x7faed00b33b0 2023-10-06T03:45:54.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.722+0000 7faecf59e700 1 -- 172.21.15.55:0/117509513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed00b33b0 msgr2=0x7faed016d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:54.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.722+0000 7faecf59e700 1 --2- 172.21.15.55:0/117509513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed00b33b0 0x7faed016d310 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7faec400ec80 tx=0x7faec40091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:54.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.722+0000 7faecf59e700 1 -- 172.21.15.55:0/117509513 shutdown_connections 2023-10-06T03:45:54.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.722+0000 7faecf59e700 1 --2- 172.21.15.55:0/117509513 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00b4e70 0x7faed016d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:54.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.722+0000 7faecf59e700 1 --2- 172.21.15.55:0/117509513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faed00b16d0 0x7faed015e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:54.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.722+0000 7faecf59e700 1 --2- 172.21.15.55:0/117509513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed00b33b0 0x7faed016d310 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:54.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.722+0000 7faecf59e700 1 -- 172.21.15.55:0/117509513 >> 172.21.15.55:0/117509513 conn(0x7faed00d10c0 msgr2=0x7faed0158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:54.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.722+0000 7faecf59e700 1 -- 172.21.15.55:0/117509513 shutdown_connections 2023-10-06T03:45:54.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.723+0000 7faecf59e700 1 -- 172.21.15.55:0/117509513 wait complete. 2023-10-06T03:45:54.743 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:45:54.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.979+0000 7f24255ff700 1 Processor -- start 2023-10-06T03:45:54.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.979+0000 7f24255ff700 1 -- start start 2023-10-06T03:45:54.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.980+0000 7f24255ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2420068890 0x7f24200c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:54.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.980+0000 7f24255ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242015a9b0 con 0x7f24201563b0 2023-10-06T03:45:54.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.980+0000 7f24255ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242015ac90 con 0x7f242015a440 2023-10-06T03:45:54.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.980+0000 7f24255ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242015af70 con 0x7f2420068890 2023-10-06T03:45:54.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.981+0000 7f241effd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f24201563b0 0x7f2420156780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57194/0 (socket says 172.21.15.55:57194) 2023-10-06T03:45:54.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.981+0000 7f241effd700 1 -- 172.21.15.55:0/2357725120 learned_addr learned my addr 172.21.15.55:0/2357725120 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:54.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.981+0000 7f241f7fe700 1 --2- 172.21.15.55:0/2357725120 >> v2:172.21.15.55:3300/0 conn(0x7f2420068890 0x7f24200c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:54.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.981+0000 7f241dffb700 1 -- 172.21.15.55:0/2357725120 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804091193 0 0) 0x7f242015a9b0 con 0x7f24201563b0 2023-10-06T03:45:54.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.981+0000 7f241dffb700 1 -- 172.21.15.55:0/2357725120 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2408003570 con 0x7f24201563b0 2023-10-06T03:45:54.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.981+0000 7f241f7fe700 1 -- 172.21.15.55:0/2357725120 >> v1:172.21.15.163:6789/0 conn(0x7f242015a440 legacy=0x7f24200acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:54.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.981+0000 7f241f7fe700 1 -- 172.21.15.55:0/2357725120 >> v1:172.21.15.55:6789/0 conn(0x7f24201563b0 legacy=0x7f2420156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:54.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.981+0000 7f241f7fe700 1 -- 172.21.15.55:0/2357725120 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242015b250 con 0x7f2420068890 2023-10-06T03:45:54.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.982+0000 7f241f7fe700 1 --2- 172.21.15.55:0/2357725120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2420068890 0x7f24200c8230 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2418009e60 tx=0x7f2418009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107154fb3f3f95b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:54.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.982+0000 7f241dffb700 1 -- 172.21.15.55:0/2357725120 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241800e070 con 0x7f2420068890 2023-10-06T03:45:54.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.982+0000 7f241dffb700 1 -- 172.21.15.55:0/2357725120 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241800bb60 con 0x7f2420068890 2023-10-06T03:45:54.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.982+0000 7f241dffb700 1 -- 172.21.15.55:0/2357725120 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418012430 con 0x7f2420068890 2023-10-06T03:45:54.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.982+0000 7f24255ff700 1 -- 172.21.15.55:0/2357725120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2420068890 msgr2=0x7f24200c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:54.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.982+0000 7f24255ff700 1 --2- 172.21.15.55:0/2357725120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2420068890 0x7f24200c8230 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f2418009e60 tx=0x7f2418009780 comp rx=0 tx=0).stop 2023-10-06T03:45:54.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.982+0000 7f24255ff700 1 -- 172.21.15.55:0/2357725120 shutdown_connections 2023-10-06T03:45:54.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.982+0000 7f24255ff700 1 --2- 172.21.15.55:0/2357725120 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2420068890 0x7f24200c8230 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:54.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.982+0000 7f24255ff700 1 -- 172.21.15.55:0/2357725120 >> 172.21.15.55:0/2357725120 conn(0x7f24200c8f90 msgr2=0x7f24200c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:54.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.982+0000 7f24255ff700 1 -- 172.21.15.55:0/2357725120 shutdown_connections 2023-10-06T03:45:54.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.983+0000 7f24255ff700 1 -- 172.21.15.55:0/2357725120 wait complete. 2023-10-06T03:45:54.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.984+0000 7f24255ff700 1 Processor -- start 2023-10-06T03:45:54.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.984+0000 7f24255ff700 1 -- start start 2023-10-06T03:45:54.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.985+0000 7f24255ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2420068890 0x7f24201597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:54.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.985+0000 7f24255ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24201563b0 0x7f242016ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:54.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.985+0000 7f24255ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f242015a440 0x7f242016d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:54.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.985+0000 7f24255ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2420154330 con 0x7f2420068890 2023-10-06T03:45:54.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.985+0000 7f24255ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2420159ce0 con 0x7f242015a440 2023-10-06T03:45:54.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.985+0000 7f24255ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24200c2950 con 0x7f24201563b0 2023-10-06T03:45:54.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.985+0000 7f241effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2420068890 0x7f24201597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:54.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.985+0000 7f241effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2420068890 0x7f24201597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52952/0 (socket says 172.21.15.55:52952) 2023-10-06T03:45:54.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.985+0000 7f241effd700 1 -- 172.21.15.55:0/1456224588 learned_addr learned my addr 172.21.15.55:0/1456224588 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:54.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.986+0000 7f241effd700 1 -- 172.21.15.55:0/1456224588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24201563b0 msgr2=0x7f242016ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:54.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.986+0000 7f241effd700 1 --2- 172.21.15.55:0/1456224588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24201563b0 0x7f242016ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:54.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.986+0000 7f241effd700 1 -- 172.21.15.55:0/1456224588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f242015a440 msgr2=0x7f242016d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:54.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.986+0000 7f241effd700 1 --2- 172.21.15.55:0/1456224588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f242015a440 0x7f242016d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:54.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.986+0000 7f241effd700 1 -- 172.21.15.55:0/1456224588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2418009a20 con 0x7f2420068890 2023-10-06T03:45:54.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.986+0000 7f241effd700 1 --2- 172.21.15.55:0/1456224588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2420068890 0x7f24201597a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f241000c7c0 tx=0x7f2410007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:54.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.986+0000 7f2406ffd700 1 -- 172.21.15.55:0/1456224588 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2410005b30 con 0x7f2420068890 2023-10-06T03:45:54.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.986+0000 7f2406ffd700 1 -- 172.21.15.55:0/1456224588 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2410005c90 con 0x7f2420068890 2023-10-06T03:45:54.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.986+0000 7f2406ffd700 1 -- 172.21.15.55:0/1456224588 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2410007340 con 0x7f2420068890 2023-10-06T03:45:54.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.986+0000 7f24255ff700 1 -- 172.21.15.55:0/1456224588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24200c2c30 con 0x7f2420068890 2023-10-06T03:45:54.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.986+0000 7f24255ff700 1 -- 172.21.15.55:0/1456224588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2420153fc0 con 0x7f2420068890 2023-10-06T03:45:54.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.987+0000 7f24255ff700 1 -- 172.21.15.55:0/1456224588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24200718b0 con 0x7f2420068890 2023-10-06T03:45:54.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.988+0000 7f2406ffd700 1 -- 172.21.15.55:0/1456224588 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 15) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f241001b400 con 0x7f2420068890 2023-10-06T03:45:54.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.989+0000 7f2406ffd700 1 --2- 172.21.15.55:0/1456224588 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f240c066d80 0x7f240c069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:54.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.990+0000 7f2406ffd700 1 -- 172.21.15.55:0/1456224588 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7096+0+0 (secure 0 0 0) 0x7f2410084330 con 0x7f2420068890 2023-10-06T03:45:54.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.990+0000 7f241e7fc700 1 --2- 172.21.15.55:0/1456224588 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f240c066d80 0x7f240c069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:54.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.991+0000 7f241e7fc700 1 --2- 172.21.15.55:0/1456224588 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f240c066d80 0x7f240c069230 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f24200c7d00 tx=0x7f2414007b40 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:54.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:54.994+0000 7f2406ffd700 1 -- 172.21.15.55:0/1456224588 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2410051290 con 0x7f2420068890 2023-10-06T03:45:55.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:55.199+0000 7f24255ff700 1 -- 172.21.15.55:0/1456224588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f242006d6a0 con 0x7f2420068890 2023-10-06T03:45:55.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:55.930+0000 7f2406ffd700 1 -- 172.21.15.55:0/1456224588 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7f24100548b0 con 0x7f2420068890 2023-10-06T03:45:55.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:55.934+0000 7f24255ff700 1 -- 172.21.15.55:0/1456224588 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f240c066d80 msgr2=0x7f240c069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:55.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:55.934+0000 7f24255ff700 1 --2- 172.21.15.55:0/1456224588 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f240c066d80 0x7f240c069230 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f24200c7d00 tx=0x7f2414007b40 comp rx=0 tx=0).stop 2023-10-06T03:45:55.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:55.934+0000 7f24255ff700 1 -- 172.21.15.55:0/1456224588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2420068890 msgr2=0x7f24201597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:55.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:55.934+0000 7f24255ff700 1 --2- 172.21.15.55:0/1456224588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2420068890 0x7f24201597a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f241000c7c0 tx=0x7f2410007e00 comp rx=0 tx=0).stop 2023-10-06T03:45:55.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:55.935+0000 7f24255ff700 1 -- 172.21.15.55:0/1456224588 shutdown_connections 2023-10-06T03:45:55.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:55.935+0000 7f24255ff700 1 --2- 172.21.15.55:0/1456224588 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f240c066d80 0x7f240c069230 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:55.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:55.935+0000 7f24255ff700 1 --2- 172.21.15.55:0/1456224588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f242015a440 0x7f242016d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:55.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:55.935+0000 7f24255ff700 1 --2- 172.21.15.55:0/1456224588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2420068890 0x7f24201597a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:55.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:55.935+0000 7f24255ff700 1 --2- 172.21.15.55:0/1456224588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24201563b0 0x7f242016ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:55.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:55.935+0000 7f24255ff700 1 -- 172.21.15.55:0/1456224588 >> 172.21.15.55:0/1456224588 conn(0x7f24200c8f90 msgr2=0x7f24200ca5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:55.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:55.935+0000 7f24255ff700 1 -- 172.21.15.55:0/1456224588 shutdown_connections 2023-10-06T03:45:55.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:55.935+0000 7f24255ff700 1 -- 172.21.15.55:0/1456224588 wait complete. 2023-10-06T03:45:55.938 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:45:55.960 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:45:56.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.152+0000 7fae1e59d700 1 Processor -- start 2023-10-06T03:45:56.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.153+0000 7fae1e59d700 1 -- start start 2023-10-06T03:45:56.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.154+0000 7fae1e59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fae1815b050 0x7fae180ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:56.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.154+0000 7fae1e59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fae180cc8d0 0x7fae1815b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:56.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.154+0000 7fae1e59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae1815b960 0x7fae18157f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:56.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.154+0000 7fae1e59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae18157a50 con 0x7fae180cc8d0 2023-10-06T03:45:56.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.154+0000 7fae1e59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae181591b0 con 0x7fae1815b050 2023-10-06T03:45:56.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.154+0000 7fae1e59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae180ca9e0 con 0x7fae1815b960 2023-10-06T03:45:56.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.154+0000 7fae1cd9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fae180cc8d0 0x7fae1815b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:56.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.154+0000 7fae1cd9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fae180cc8d0 0x7fae1815b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52968/0 (socket says 172.21.15.55:52968) 2023-10-06T03:45:56.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.154+0000 7fae1cd9a700 1 -- 172.21.15.55:0/2718053070 learned_addr learned my addr 172.21.15.55:0/2718053070 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:56.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.154+0000 7fae1d59b700 1 --2- 172.21.15.55:0/2718053070 >> v2:172.21.15.163:3300/0 conn(0x7fae1815b050 0x7fae180ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:56.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.155+0000 7fae1dd9c700 1 --2- 172.21.15.55:0/2718053070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae1815b960 0x7fae18157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:56.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.155+0000 7fae1cd9a700 1 -- 172.21.15.55:0/2718053070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae1815b960 msgr2=0x7fae18157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:56.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.155+0000 7fae1cd9a700 1 --2- 172.21.15.55:0/2718053070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae1815b960 0x7fae18157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:56.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.155+0000 7fae1cd9a700 1 -- 172.21.15.55:0/2718053070 >> v2:172.21.15.163:3300/0 conn(0x7fae1815b050 msgr2=0x7fae180ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:56.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.155+0000 7fae1cd9a700 1 --2- 172.21.15.55:0/2718053070 >> v2:172.21.15.163:3300/0 conn(0x7fae1815b050 0x7fae180ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:56.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.155+0000 7fae1cd9a700 1 -- 172.21.15.55:0/2718053070 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae180cacc0 con 0x7fae180cc8d0 2023-10-06T03:45:56.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.155+0000 7fae1d59b700 1 --2- 172.21.15.55:0/2718053070 >> v2:172.21.15.163:3300/0 conn(0x7fae1815b050 0x7fae180ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:45:56.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.155+0000 7fae1cd9a700 1 --2- 172.21.15.55:0/2718053070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae180cc8d0 0x7fae1815b420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fae18064f60 tx=0x7fae08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298743f2b8df697e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:56.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.155+0000 7fae0ffff700 1 -- 172.21.15.55:0/2718053070 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0800b860 con 0x7fae180cc8d0 2023-10-06T03:45:56.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.155+0000 7fae0ffff700 1 -- 172.21.15.55:0/2718053070 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0800b9c0 con 0x7fae180cc8d0 2023-10-06T03:45:56.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.155+0000 7fae0ffff700 1 -- 172.21.15.55:0/2718053070 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0800bd60 con 0x7fae180cc8d0 2023-10-06T03:45:56.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.156+0000 7fae1e59d700 1 -- 172.21.15.55:0/2718053070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae180cc8d0 msgr2=0x7fae1815b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:56.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.156+0000 7fae1e59d700 1 --2- 172.21.15.55:0/2718053070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae180cc8d0 0x7fae1815b420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fae18064f60 tx=0x7fae08009250 comp rx=0 tx=0).stop 2023-10-06T03:45:56.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.156+0000 7fae1e59d700 1 -- 172.21.15.55:0/2718053070 shutdown_connections 2023-10-06T03:45:56.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.156+0000 7fae1e59d700 1 --2- 172.21.15.55:0/2718053070 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae1815b960 0x7fae18157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:56.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.156+0000 7fae1e59d700 1 --2- 172.21.15.55:0/2718053070 >> v2:172.21.15.163:3300/0 conn(0x7fae1815b050 0x7fae180ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:56.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.156+0000 7fae1e59d700 1 --2- 172.21.15.55:0/2718053070 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae180cc8d0 0x7fae1815b420 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:56.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.156+0000 7fae1e59d700 1 -- 172.21.15.55:0/2718053070 >> 172.21.15.55:0/2718053070 conn(0x7fae18068e20 msgr2=0x7fae180cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:56.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.156+0000 7fae1e59d700 1 -- 172.21.15.55:0/2718053070 shutdown_connections 2023-10-06T03:45:56.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.156+0000 7fae1e59d700 1 -- 172.21.15.55:0/2718053070 wait complete. 2023-10-06T03:45:56.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.157+0000 7fae1e59d700 1 Processor -- start 2023-10-06T03:45:56.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.158+0000 7fae1e59d700 1 -- start start 2023-10-06T03:45:56.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.158+0000 7fae1e59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae180cc8d0 0x7fae18149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:56.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.158+0000 7fae1e59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1815b050 0x7fae180a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:56.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.158+0000 7fae1e59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae1815b960 0x7fae180a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:56.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1e59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae181565e0 con 0x7fae1815b050 2023-10-06T03:45:56.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1e59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1814a200 con 0x7fae1815b960 2023-10-06T03:45:56.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1e59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae180c8800 con 0x7fae180cc8d0 2023-10-06T03:45:56.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1cd9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1815b050 0x7fae180a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:56.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1cd9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1815b050 0x7fae180a6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52978/0 (socket says 172.21.15.55:52978) 2023-10-06T03:45:56.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1cd9a700 1 -- 172.21.15.55:0/931435043 learned_addr learned my addr 172.21.15.55:0/931435043 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:56.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1d59b700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae180cc8d0 0x7fae18149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:56.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1dd9c700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae1815b960 0x7fae180a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:56.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1cd9a700 1 -- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae180cc8d0 msgr2=0x7fae18149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:56.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1cd9a700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae180cc8d0 0x7fae18149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:56.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1cd9a700 1 -- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae1815b960 msgr2=0x7fae180a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:56.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1cd9a700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae1815b960 0x7fae180a68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:56.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1cd9a700 1 -- 172.21.15.55:0/931435043 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae080094f0 con 0x7fae1815b050 2023-10-06T03:45:56.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1d59b700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae180cc8d0 0x7fae18149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:45:56.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.159+0000 7fae1cd9a700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1815b050 0x7fae180a6370 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fae080090b0 tx=0x7fae08017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:56.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.160+0000 7fae0d7fa700 1 -- 172.21.15.55:0/931435043 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08026480 con 0x7fae1815b050 2023-10-06T03:45:56.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.160+0000 7fae0d7fa700 1 -- 172.21.15.55:0/931435043 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0801f070 con 0x7fae1815b050 2023-10-06T03:45:56.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.160+0000 7fae1e59d700 1 -- 172.21.15.55:0/931435043 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae180c5520 con 0x7fae1815b050 2023-10-06T03:45:56.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.160+0000 7fae0d7fa700 1 -- 172.21.15.55:0/931435043 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08026830 con 0x7fae1815b050 2023-10-06T03:45:56.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.160+0000 7fae1e59d700 1 -- 172.21.15.55:0/931435043 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae180c5ae0 con 0x7fae1815b050 2023-10-06T03:45:56.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.161+0000 7fae1e59d700 1 -- 172.21.15.55:0/931435043 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae180c5cd0 con 0x7fae1815b050 2023-10-06T03:45:56.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.162+0000 7fae0d7fa700 1 -- 172.21.15.55:0/931435043 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 15) v1 ==== 84578+0+0 (secure 0 0 0) 0x7fae08026ba0 con 0x7fae1815b050 2023-10-06T03:45:56.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.162+0000 7fae0d7fa700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fadfc066d80 0x7fadfc069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:56.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.162+0000 7fae0d7fa700 1 -- 172.21.15.55:0/931435043 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6658+0+0 (secure 0 0 0) 0x7fae08021070 con 0x7fae1815b050 2023-10-06T03:45:56.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.163+0000 7fae1d59b700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fadfc066d80 0x7fadfc069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:56.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.164+0000 7fae1d59b700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fadfc066d80 0x7fadfc069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fae14006f00 tx=0x7fae14008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:56.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.167+0000 7fae0d7fa700 1 -- 172.21.15.55:0/931435043 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae080534c0 con 0x7fae1815b050 2023-10-06T03:45:56.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.369+0000 7fae1e59d700 1 -- 172.21.15.55:0/931435043 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fae1804c3c0 con 0x7fae1815b050 2023-10-06T03:45:56.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.726+0000 7fae0d7fa700 1 -- 172.21.15.55:0/931435043 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae08086120 con 0x7fae1815b050 2023-10-06T03:45:56.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.943+0000 7fae0d7fa700 1 -- 172.21.15.55:0/931435043 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7fae080699d0 con 0x7fae1815b050 2023-10-06T03:45:56.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.947+0000 7fae1e59d700 1 -- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fadfc066d80 msgr2=0x7fadfc069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:56.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.947+0000 7fae1e59d700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fadfc066d80 0x7fadfc069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fae14006f00 tx=0x7fae14008040 comp rx=0 tx=0).stop 2023-10-06T03:45:56.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.947+0000 7fae1e59d700 1 -- 172.21.15.55:0/931435043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1815b050 msgr2=0x7fae180a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:56.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.947+0000 7fae1e59d700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1815b050 0x7fae180a6370 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fae080090b0 tx=0x7fae08017000 comp rx=0 tx=0).stop 2023-10-06T03:45:56.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.947+0000 7fae1e59d700 1 -- 172.21.15.55:0/931435043 shutdown_connections 2023-10-06T03:45:56.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.947+0000 7fae1e59d700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fadfc066d80 0x7fadfc069230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:56.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.947+0000 7fae1e59d700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae1815b960 0x7fae180a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.947+0000 7fae1e59d700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae180cc8d0 0x7fae18149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.947+0000 7fae1e59d700 1 --2- 172.21.15.55:0/931435043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae1815b050 0x7fae180a6370 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.947+0000 7fae1e59d700 1 -- 172.21.15.55:0/931435043 >> 172.21.15.55:0/931435043 conn(0x7fae18068e20 msgr2=0x7fae18156980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.948+0000 7fae1e59d700 1 -- 172.21.15.55:0/931435043 shutdown_connections 2023-10-06T03:45:56.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:56.948+0000 7fae1e59d700 1 -- 172.21.15.55:0/931435043 wait complete. 2023-10-06T03:45:56.951 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2023-10-06T03:45:56.970 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:45:57.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.161+0000 7f0763fff700 1 Processor -- start 2023-10-06T03:45:57.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.161+0000 7f0763fff700 1 -- start start 2023-10-06T03:45:57.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.162+0000 7f0763fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f07640b4e70 0x7f07640ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:57.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.162+0000 7f0763fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f07640b1b40 0x7f076415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:57.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.162+0000 7f0763fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07640cfb00 con 0x7f07640b16d0 2023-10-06T03:45:57.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.162+0000 7f0763fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07640b5c40 con 0x7f07640b1b40 2023-10-06T03:45:57.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.162+0000 7f0763fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07640b5f20 con 0x7f07640b4e70 2023-10-06T03:45:57.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.162+0000 7f07627fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f07640b16d0 0x7f076415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57202/0 (socket says 172.21.15.55:57202) 2023-10-06T03:45:57.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.162+0000 7f07627fc700 1 -- 172.21.15.55:0/3040107264 learned_addr learned my addr 172.21.15.55:0/3040107264 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:57.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.162+0000 7f07637fe700 1 --2- 172.21.15.55:0/3040107264 >> v2:172.21.15.55:3300/0 conn(0x7f07640b1b40 0x7f076415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:57.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.163+0000 7f0762ffd700 1 --2- 172.21.15.55:0/3040107264 >> v2:172.21.15.163:3300/0 conn(0x7f07640b4e70 0x7f07640ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:57.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.163+0000 7f0761ffb700 1 -- 172.21.15.55:0/3040107264 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1740229754 0 0) 0x7f07640cfb00 con 0x7f07640b16d0 2023-10-06T03:45:57.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.163+0000 7f0761ffb700 1 -- 172.21.15.55:0/3040107264 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f074c003570 con 0x7f07640b16d0 2023-10-06T03:45:57.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.163+0000 7f07637fe700 1 -- 172.21.15.55:0/3040107264 >> v2:172.21.15.163:3300/0 conn(0x7f07640b4e70 msgr2=0x7f07640ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:57.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.163+0000 7f07637fe700 1 --2- 172.21.15.55:0/3040107264 >> v2:172.21.15.163:3300/0 conn(0x7f07640b4e70 0x7f07640ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.163+0000 7f07637fe700 1 -- 172.21.15.55:0/3040107264 >> v1:172.21.15.55:6789/0 conn(0x7f07640b16d0 legacy=0x7f076415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:57.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.163+0000 7f07637fe700 1 -- 172.21.15.55:0/3040107264 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07640b6200 con 0x7f07640b1b40 2023-10-06T03:45:57.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.163+0000 7f0762ffd700 1 --2- 172.21.15.55:0/3040107264 >> v2:172.21.15.163:3300/0 conn(0x7f07640b4e70 0x7f07640ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:45:57.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.163+0000 7f07637fe700 1 --2- 172.21.15.55:0/3040107264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07640b1b40 0x7f076415d950 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0764064910 tx=0x7f075c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585efcb2a6c605fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:57.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.163+0000 7f0761ffb700 1 -- 172.21.15.55:0/3040107264 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c00e070 con 0x7f07640b1b40 2023-10-06T03:45:57.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.163+0000 7f0761ffb700 1 -- 172.21.15.55:0/3040107264 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c00bb60 con 0x7f07640b1b40 2023-10-06T03:45:57.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.163+0000 7f0761ffb700 1 -- 172.21.15.55:0/3040107264 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c012430 con 0x7f07640b1b40 2023-10-06T03:45:57.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.164+0000 7f0763fff700 1 -- 172.21.15.55:0/3040107264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07640b1b40 msgr2=0x7f076415d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:57.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.164+0000 7f0763fff700 1 --2- 172.21.15.55:0/3040107264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07640b1b40 0x7f076415d950 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0764064910 tx=0x7f075c009780 comp rx=0 tx=0).stop 2023-10-06T03:45:57.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.164+0000 7f0763fff700 1 -- 172.21.15.55:0/3040107264 shutdown_connections 2023-10-06T03:45:57.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.164+0000 7f0763fff700 1 --2- 172.21.15.55:0/3040107264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07640b1b40 0x7f076415d950 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.164+0000 7f0763fff700 1 --2- 172.21.15.55:0/3040107264 >> v2:172.21.15.163:3300/0 conn(0x7f07640b4e70 0x7f07640ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.164+0000 7f0763fff700 1 -- 172.21.15.55:0/3040107264 >> 172.21.15.55:0/3040107264 conn(0x7f07640d10c0 msgr2=0x7f07640affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:57.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.164+0000 7f0763fff700 1 -- 172.21.15.55:0/3040107264 shutdown_connections 2023-10-06T03:45:57.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.164+0000 7f0763fff700 1 -- 172.21.15.55:0/3040107264 wait complete. 2023-10-06T03:45:57.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.166+0000 7f0763fff700 1 Processor -- start 2023-10-06T03:45:57.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.166+0000 7f0763fff700 1 -- start start 2023-10-06T03:45:57.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f0763fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07640b16d0 0x7f076416a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:57.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f0763fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07640b1b40 0x7f07640c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:57.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f0763fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07640b4e70 0x7f076414bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:57.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f0763fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07640c6280 con 0x7f07640b1b40 2023-10-06T03:45:57.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f0763fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076416a930 con 0x7f07640b4e70 2023-10-06T03:45:57.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f0763fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07640c5390 con 0x7f07640b16d0 2023-10-06T03:45:57.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f07627fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07640b1b40 0x7f07640c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:57.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f07627fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07640b1b40 0x7f07640c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52990/0 (socket says 172.21.15.55:52990) 2023-10-06T03:45:57.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f07627fc700 1 -- 172.21.15.55:0/3029073790 learned_addr learned my addr 172.21.15.55:0/3029073790 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:57.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f0762ffd700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07640b16d0 0x7f076416a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:57.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f07637fe700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07640b4e70 0x7f076414bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:57.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f07627fc700 1 -- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07640b16d0 msgr2=0x7f076416a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:57.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f07627fc700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07640b16d0 0x7f076416a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f07627fc700 1 -- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07640b4e70 msgr2=0x7f076414bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:57.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f07627fc700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07640b4e70 0x7f076414bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.167+0000 7f07627fc700 1 -- 172.21.15.55:0/3029073790 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c009a20 con 0x7f07640b1b40 2023-10-06T03:45:57.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.168+0000 7f07637fe700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07640b4e70 0x7f076414bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:45:57.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.168+0000 7f07627fc700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07640b1b40 0x7f07640c6710 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f075800c7c0 tx=0x7f0758007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:57.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.168+0000 7f074affd700 1 -- 172.21.15.55:0/3029073790 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758005b30 con 0x7f07640b1b40 2023-10-06T03:45:57.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.168+0000 7f074affd700 1 -- 172.21.15.55:0/3029073790 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0758005c90 con 0x7f07640b1b40 2023-10-06T03:45:57.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.168+0000 7f074affd700 1 -- 172.21.15.55:0/3029073790 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758007340 con 0x7f07640b1b40 2023-10-06T03:45:57.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.168+0000 7f0763fff700 1 -- 172.21.15.55:0/3029073790 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07640c5670 con 0x7f07640b1b40 2023-10-06T03:45:57.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.169+0000 7f0763fff700 1 -- 172.21.15.55:0/3029073790 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07640c5c30 con 0x7f07640b1b40 2023-10-06T03:45:57.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.170+0000 7f0763fff700 1 -- 172.21.15.55:0/3029073790 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0764088990 con 0x7f07640b1b40 2023-10-06T03:45:57.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.172+0000 7f074affd700 1 -- 172.21.15.55:0/3029073790 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f075801b400 con 0x7f07640b1b40 2023-10-06T03:45:57.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.174+0000 7f074affd700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0750066f30 0x7f07500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:57.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.174+0000 7f074affd700 1 -- 172.21.15.55:0/3029073790 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6215+0+0 (secure 0 0 0) 0x7f0758084320 con 0x7f07640b1b40 2023-10-06T03:45:57.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.175+0000 7f0762ffd700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0750066f30 0x7f07500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:57.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.180+0000 7f0762ffd700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0750066f30 0x7f07500693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0754006f00 tx=0x7f0754008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:57.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.181+0000 7f074affd700 1 -- 172.21.15.55:0/3029073790 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0758051370 con 0x7f07640b1b40 2023-10-06T03:45:57.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.384+0000 7f0763fff700 1 -- 172.21.15.55:0/3029073790 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f07640c63c0 con 0x7f07640b1b40 2023-10-06T03:45:57.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.388+0000 7f074affd700 1 -- 172.21.15.55:0/3029073790 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0758054990 con 0x7f07640b1b40 2023-10-06T03:45:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.390+0000 7f0763fff700 1 -- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0750066f30 msgr2=0x7f07500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.390+0000 7f0763fff700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0750066f30 0x7f07500693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0754006f00 tx=0x7f0754008040 comp rx=0 tx=0).stop 2023-10-06T03:45:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.390+0000 7f0763fff700 1 -- 172.21.15.55:0/3029073790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07640b1b40 msgr2=0x7f07640c6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.390+0000 7f0763fff700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07640b1b40 0x7f07640c6710 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f075800c7c0 tx=0x7f0758007e00 comp rx=0 tx=0).stop 2023-10-06T03:45:57.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.390+0000 7f0763fff700 1 -- 172.21.15.55:0/3029073790 shutdown_connections 2023-10-06T03:45:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.390+0000 7f0763fff700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0750066f30 0x7f07500693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.390+0000 7f0763fff700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f07640b4e70 0x7f076414bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.390+0000 7f0763fff700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f07640b16d0 0x7f076416a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.390+0000 7f0763fff700 1 --2- 172.21.15.55:0/3029073790 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f07640b1b40 0x7f07640c6710 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.390+0000 7f0763fff700 1 -- 172.21.15.55:0/3029073790 >> 172.21.15.55:0/3029073790 conn(0x7f07640d10c0 msgr2=0x7f0764068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.391+0000 7f0763fff700 1 -- 172.21.15.55:0/3029073790 shutdown_connections 2023-10-06T03:45:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.391+0000 7f0763fff700 1 -- 172.21.15.55:0/3029073790 wait complete. 2023-10-06T03:45:57.410 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T03:45:57.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.598+0000 7f370559d700 1 Processor -- start 2023-10-06T03:45:57.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.598+0000 7f370559d700 1 -- start start 2023-10-06T03:45:57.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.599+0000 7f370559d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37000b16d0 0x7f37000b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:57.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.599+0000 7f370559d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37000b33b0 0x7f37000b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:57.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.599+0000 7f370559d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37000b2080 con 0x7f37000b4e70 2023-10-06T03:45:57.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.599+0000 7f370559d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37000cfb00 con 0x7f37000b33b0 2023-10-06T03:45:57.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.599+0000 7f370559d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37000b5c40 con 0x7f37000b16d0 2023-10-06T03:45:57.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.600+0000 7f3704d9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37000b33b0 0x7f37000b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:57.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.600+0000 7f36ff7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37000b16d0 0x7f37000b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:57.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.600+0000 7f3704d9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37000b33b0 0x7f37000b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:34236/0 (socket says 172.21.15.55:34236) 2023-10-06T03:45:57.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.600+0000 7f3704d9c700 1 -- 172.21.15.55:0/1595720617 learned_addr learned my addr 172.21.15.55:0/1595720617 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:57.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.600+0000 7f36feffd700 1 -- 172.21.15.55:0/1595720617 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281733957 0 0) 0x7f37000b2080 con 0x7f37000b4e70 2023-10-06T03:45:57.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.600+0000 7f3704d9c700 1 -- 172.21.15.55:0/1595720617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37000b16d0 msgr2=0x7f37000b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:57.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.600+0000 7f3704d9c700 1 --2- 172.21.15.55:0/1595720617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37000b16d0 0x7f37000b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.600+0000 7f3704d9c700 1 -- 172.21.15.55:0/1595720617 >> v1:172.21.15.163:6789/0 conn(0x7f37000b4e70 legacy=0x7f37000ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:57.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.600+0000 7f3704d9c700 1 -- 172.21.15.55:0/1595720617 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37000b5f20 con 0x7f37000b33b0 2023-10-06T03:45:57.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.600+0000 7f36ff7fe700 1 --2- 172.21.15.55:0/1595720617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37000b16d0 0x7f37000b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:45:57.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.601+0000 7f3704d9c700 1 --2- 172.21.15.55:0/1595720617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37000b33b0 0x7f37000b3830 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009f20 tx=0x7f36f8009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8ec66e16cfef327 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:57.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.601+0000 7f36feffd700 1 -- 172.21.15.55:0/1595720617 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f800e070 con 0x7f37000b33b0 2023-10-06T03:45:57.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.601+0000 7f36feffd700 1 -- 172.21.15.55:0/1595720617 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f800b9d0 con 0x7f37000b33b0 2023-10-06T03:45:57.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.601+0000 7f36feffd700 1 -- 172.21.15.55:0/1595720617 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f80092e0 con 0x7f37000b33b0 2023-10-06T03:45:57.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.602+0000 7f370559d700 1 -- 172.21.15.55:0/1595720617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37000b33b0 msgr2=0x7f37000b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:57.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.602+0000 7f370559d700 1 --2- 172.21.15.55:0/1595720617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37000b33b0 0x7f37000b3830 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009f20 tx=0x7f36f8009840 comp rx=0 tx=0).stop 2023-10-06T03:45:57.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.602+0000 7f370559d700 1 -- 172.21.15.55:0/1595720617 shutdown_connections 2023-10-06T03:45:57.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.602+0000 7f370559d700 1 --2- 172.21.15.55:0/1595720617 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37000b33b0 0x7f37000b3830 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.602+0000 7f370559d700 1 --2- 172.21.15.55:0/1595720617 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37000b16d0 0x7f37000b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.602+0000 7f370559d700 1 -- 172.21.15.55:0/1595720617 >> 172.21.15.55:0/1595720617 conn(0x7f37000d10c0 msgr2=0x7f37000affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:57.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.602+0000 7f370559d700 1 -- 172.21.15.55:0/1595720617 shutdown_connections 2023-10-06T03:45:57.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.602+0000 7f370559d700 1 -- 172.21.15.55:0/1595720617 wait complete. 2023-10-06T03:45:57.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.603+0000 7f370559d700 1 Processor -- start 2023-10-06T03:45:57.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.603+0000 7f370559d700 1 -- start start 2023-10-06T03:45:57.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.604+0000 7f370559d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37000b16d0 0x7f37000ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:57.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.604+0000 7f370559d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37000b33b0 0x7f3700068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:57.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.604+0000 7f370559d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37000b4e70 0x7f3700068f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:57.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.604+0000 7f370559d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37000c6660 con 0x7f37000b16d0 2023-10-06T03:45:57.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.604+0000 7f370559d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370016dfa0 con 0x7f37000b33b0 2023-10-06T03:45:57.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.604+0000 7f370559d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37000c5d30 con 0x7f37000b4e70 2023-10-06T03:45:57.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.605+0000 7f36fffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37000b16d0 0x7f37000ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:57.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.605+0000 7f36fffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37000b16d0 0x7f37000ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52992/0 (socket says 172.21.15.55:52992) 2023-10-06T03:45:57.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.605+0000 7f36fffff700 1 -- 172.21.15.55:0/1304515548 learned_addr learned my addr 172.21.15.55:0/1304515548 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:57.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.605+0000 7f36ff7fe700 1 --2- 172.21.15.55:0/1304515548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37000b33b0 0x7f3700068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.605+0000 7f3704d9c700 1 --2- 172.21.15.55:0/1304515548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37000b4e70 0x7f3700068f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.605+0000 7f36fffff700 1 -- 172.21.15.55:0/1304515548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37000b4e70 msgr2=0x7f3700068f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.605+0000 7f36fffff700 1 --2- 172.21.15.55:0/1304515548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37000b4e70 0x7f3700068f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.605+0000 7f36fffff700 1 -- 172.21.15.55:0/1304515548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37000b33b0 msgr2=0x7f3700068a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.605+0000 7f36fffff700 1 --2- 172.21.15.55:0/1304515548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37000b33b0 0x7f3700068a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.605+0000 7f36fffff700 1 -- 172.21.15.55:0/1304515548 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36f4009420 con 0x7f37000b16d0 2023-10-06T03:45:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.605+0000 7f36fffff700 1 --2- 172.21.15.55:0/1304515548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37000b16d0 0x7f37000ca250 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f36f000c530 tx=0x7f36f000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.606+0000 7f36e7fff700 1 -- 172.21.15.55:0/1304515548 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0011590 con 0x7f37000b16d0 2023-10-06T03:45:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.606+0000 7f36e7fff700 1 -- 172.21.15.55:0/1304515548 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36f0011710 con 0x7f37000b16d0 2023-10-06T03:45:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.606+0000 7f370559d700 1 -- 172.21.15.55:0/1304515548 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36f8009ae0 con 0x7f37000b16d0 2023-10-06T03:45:57.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.606+0000 7f36e7fff700 1 -- 172.21.15.55:0/1304515548 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f0011ad0 con 0x7f37000b16d0 2023-10-06T03:45:57.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.606+0000 7f370559d700 1 -- 172.21.15.55:0/1304515548 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37000c62f0 con 0x7f37000b16d0 2023-10-06T03:45:57.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.607+0000 7f370559d700 1 -- 172.21.15.55:0/1304515548 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3700088990 con 0x7f37000b16d0 2023-10-06T03:45:57.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.608+0000 7f36e7fff700 1 -- 172.21.15.55:0/1304515548 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f36f001b3b0 con 0x7f37000b16d0 2023-10-06T03:45:57.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.609+0000 7f36e7fff700 1 --2- 172.21.15.55:0/1304515548 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f36ec066f30 0x7f36ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:57.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.609+0000 7f36e7fff700 1 -- 172.21.15.55:0/1304515548 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6215+0+0 (secure 0 0 0) 0x7f36f0083bc0 con 0x7f37000b16d0 2023-10-06T03:45:57.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.609+0000 7f36ff7fe700 1 --2- 172.21.15.55:0/1304515548 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f36ec066f30 0x7f36ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:57.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.610+0000 7f36ff7fe700 1 --2- 172.21.15.55:0/1304515548 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f36ec066f30 0x7f36ec0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f37000c7b40 tx=0x7f36f40091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:57.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.613+0000 7f36e7fff700 1 -- 172.21.15.55:0/1304515548 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36f0050c30 con 0x7f37000b16d0 2023-10-06T03:45:57.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.817+0000 7f370559d700 1 -- 172.21.15.55:0/1304515548 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f370004bb80 con 0x7f37000b16d0 2023-10-06T03:45:57.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.821+0000 7f36e7fff700 1 -- 172.21.15.55:0/1304515548 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v14) v1 ==== 115+0+0 (secure 0 0 0) 0x7f36f0017070 con 0x7f37000b16d0 2023-10-06T03:45:57.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.823+0000 7f370559d700 1 -- 172.21.15.55:0/1304515548 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f36ec066f30 msgr2=0x7f36ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:57.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.823+0000 7f370559d700 1 --2- 172.21.15.55:0/1304515548 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f36ec066f30 0x7f36ec0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f37000c7b40 tx=0x7f36f40091a0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.823+0000 7f370559d700 1 -- 172.21.15.55:0/1304515548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37000b16d0 msgr2=0x7f37000ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:57.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.823+0000 7f370559d700 1 --2- 172.21.15.55:0/1304515548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37000b16d0 0x7f37000ca250 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f36f000c530 tx=0x7f36f000c840 comp rx=0 tx=0).stop 2023-10-06T03:45:57.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.823+0000 7f370559d700 1 -- 172.21.15.55:0/1304515548 shutdown_connections 2023-10-06T03:45:57.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.823+0000 7f370559d700 1 --2- 172.21.15.55:0/1304515548 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f36ec066f30 0x7f36ec0693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.823+0000 7f370559d700 1 --2- 172.21.15.55:0/1304515548 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37000b4e70 0x7f3700068f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.823+0000 7f370559d700 1 --2- 172.21.15.55:0/1304515548 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37000b16d0 0x7f37000ca250 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.823+0000 7f370559d700 1 --2- 172.21.15.55:0/1304515548 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37000b33b0 0x7f3700068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:57.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.823+0000 7f370559d700 1 -- 172.21.15.55:0/1304515548 >> 172.21.15.55:0/1304515548 conn(0x7f37000d10c0 msgr2=0x7f3700158ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:57.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.823+0000 7f370559d700 1 -- 172.21.15.55:0/1304515548 shutdown_connections 2023-10-06T03:45:57.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:57.823+0000 7f370559d700 1 -- 172.21.15.55:0/1304515548 wait complete. 2023-10-06T03:45:57.842 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T03:45:58.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.030+0000 7ffab519d700 1 Processor -- start 2023-10-06T03:45:58.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.030+0000 7ffab519d700 1 -- start start 2023-10-06T03:45:58.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffab519d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffab00cb1a0 0x7ffab015a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:58.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffab519d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffab015a8a0 0x7ffab006f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:58.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffab519d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffab006f5a0 0x7ffab00ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffab519d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab015b2e0 con 0x7ffab006f5a0 2023-10-06T03:45:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffab519d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab006ebd0 con 0x7ffab00cb1a0 2023-10-06T03:45:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffab519d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab015ad20 con 0x7ffab015a8a0 2023-10-06T03:45:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffab499c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffab006f5a0 0x7ffab00ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffab499c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffab006f5a0 0x7ffab00ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52998/0 (socket says 172.21.15.55:52998) 2023-10-06T03:45:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffab499c700 1 -- 172.21.15.55:0/746867056 learned_addr learned my addr 172.21.15.55:0/746867056 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffaaffff700 1 --2- 172.21.15.55:0/746867056 >> v2:172.21.15.163:3300/0 conn(0x7ffab00cb1a0 0x7ffab015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffaaf7fe700 1 --2- 172.21.15.55:0/746867056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffab015a8a0 0x7ffab006f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffab499c700 1 -- 172.21.15.55:0/746867056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffab015a8a0 msgr2=0x7ffab006f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffab499c700 1 --2- 172.21.15.55:0/746867056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffab015a8a0 0x7ffab006f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffab499c700 1 -- 172.21.15.55:0/746867056 >> v2:172.21.15.163:3300/0 conn(0x7ffab00cb1a0 msgr2=0x7ffab015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:58.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffab499c700 1 --2- 172.21.15.55:0/746867056 >> v2:172.21.15.163:3300/0 conn(0x7ffab00cb1a0 0x7ffab015a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.032+0000 7ffab499c700 1 -- 172.21.15.55:0/746867056 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab015b000 con 0x7ffab006f5a0 2023-10-06T03:45:58.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.033+0000 7ffab499c700 1 --2- 172.21.15.55:0/746867056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffab006f5a0 0x7ffab00ae0d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ffab00cce10 tx=0x7ffaa8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d070572cbd185ea8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:58.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.033+0000 7ffaaeffd700 1 -- 172.21.15.55:0/746867056 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa800e070 con 0x7ffab006f5a0 2023-10-06T03:45:58.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.033+0000 7ffaaeffd700 1 -- 172.21.15.55:0/746867056 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa800bae0 con 0x7ffab006f5a0 2023-10-06T03:45:58.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.033+0000 7ffaaeffd700 1 -- 172.21.15.55:0/746867056 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa80092e0 con 0x7ffab006f5a0 2023-10-06T03:45:58.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.034+0000 7ffab519d700 1 -- 172.21.15.55:0/746867056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffab006f5a0 msgr2=0x7ffab00ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:58.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.034+0000 7ffab519d700 1 --2- 172.21.15.55:0/746867056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffab006f5a0 0x7ffab00ae0d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ffab00cce10 tx=0x7ffaa8009840 comp rx=0 tx=0).stop 2023-10-06T03:45:58.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.034+0000 7ffab519d700 1 -- 172.21.15.55:0/746867056 shutdown_connections 2023-10-06T03:45:58.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.034+0000 7ffab519d700 1 --2- 172.21.15.55:0/746867056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffab006f5a0 0x7ffab00ae0d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.034+0000 7ffab519d700 1 --2- 172.21.15.55:0/746867056 >> v2:172.21.15.163:3300/0 conn(0x7ffab00cb1a0 0x7ffab015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.034+0000 7ffab519d700 1 --2- 172.21.15.55:0/746867056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffab015a8a0 0x7ffab006f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.034+0000 7ffab519d700 1 -- 172.21.15.55:0/746867056 >> 172.21.15.55:0/746867056 conn(0x7ffab00cc1e0 msgr2=0x7ffab00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:58.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.034+0000 7ffab519d700 1 -- 172.21.15.55:0/746867056 shutdown_connections 2023-10-06T03:45:58.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.034+0000 7ffab519d700 1 -- 172.21.15.55:0/746867056 wait complete. 2023-10-06T03:45:58.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.035+0000 7ffab519d700 1 Processor -- start 2023-10-06T03:45:58.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.035+0000 7ffab519d700 1 -- start start 2023-10-06T03:45:58.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.036+0000 7ffab519d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffab006f5a0 0x7ffab0152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:58.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.036+0000 7ffab519d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffab00cb1a0 0x7ffab015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:58.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.036+0000 7ffab519d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffab00c4480 0x7ffab015d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:58.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.036+0000 7ffab519d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab015d200 con 0x7ffab006f5a0 2023-10-06T03:45:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.036+0000 7ffab519d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab0152650 con 0x7ffab00c4480 2023-10-06T03:45:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.036+0000 7ffab519d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab015c160 con 0x7ffab00cb1a0 2023-10-06T03:45:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.036+0000 7ffaaffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffab006f5a0 0x7ffab0152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.036+0000 7ffaaffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffab006f5a0 0x7ffab0152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53004/0 (socket says 172.21.15.55:53004) 2023-10-06T03:45:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.036+0000 7ffaaffff700 1 -- 172.21.15.55:0/1423495478 learned_addr learned my addr 172.21.15.55:0/1423495478 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.037+0000 7ffab499c700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffab00c4480 0x7ffab015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.037+0000 7ffaaf7fe700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffab00cb1a0 0x7ffab015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.037+0000 7ffaaffff700 1 -- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffab00cb1a0 msgr2=0x7ffab015bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.037+0000 7ffaaffff700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffab00cb1a0 0x7ffab015bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.037+0000 7ffaaffff700 1 -- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffab00c4480 msgr2=0x7ffab015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.037+0000 7ffaaffff700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffab00c4480 0x7ffab015d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.037+0000 7ffaaffff700 1 -- 172.21.15.55:0/1423495478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa8009ae0 con 0x7ffab006f5a0 2023-10-06T03:45:58.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.037+0000 7ffaaf7fe700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffab00cb1a0 0x7ffab015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:45:58.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.037+0000 7ffaaffff700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffab006f5a0 0x7ffab0152110 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000d8c0 tx=0x7ffaa0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:58.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.037+0000 7ffa97fff700 1 -- 172.21.15.55:0/1423495478 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa000de60 con 0x7ffab006f5a0 2023-10-06T03:45:58.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.037+0000 7ffa97fff700 1 -- 172.21.15.55:0/1423495478 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa000b780 con 0x7ffab006f5a0 2023-10-06T03:45:58.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.037+0000 7ffab519d700 1 -- 172.21.15.55:0/1423495478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab015c8d0 con 0x7ffab006f5a0 2023-10-06T03:45:58.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.037+0000 7ffa97fff700 1 -- 172.21.15.55:0/1423495478 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa000bb20 con 0x7ffab006f5a0 2023-10-06T03:45:58.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.038+0000 7ffab519d700 1 -- 172.21.15.55:0/1423495478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab015ce90 con 0x7ffab006f5a0 2023-10-06T03:45:58.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.039+0000 7ffab519d700 1 -- 172.21.15.55:0/1423495478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab006c450 con 0x7ffab006f5a0 2023-10-06T03:45:58.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.039+0000 7ffa97fff700 1 -- 172.21.15.55:0/1423495478 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffaa001e460 con 0x7ffab006f5a0 2023-10-06T03:45:58.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.041+0000 7ffa97fff700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffa9c066f30 0x7ffa9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:58.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.041+0000 7ffa97fff700 1 -- 172.21.15.55:0/1423495478 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6215+0+0 (secure 0 0 0) 0x7ffaa0084de0 con 0x7ffab006f5a0 2023-10-06T03:45:58.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.041+0000 7ffaaf7fe700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffa9c066f30 0x7ffa9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:58.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.042+0000 7ffaaf7fe700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffa9c066f30 0x7ffa9c0693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffab006fd30 tx=0x7ffaa4008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:58.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.045+0000 7ffa97fff700 1 -- 172.21.15.55:0/1423495478 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffaa0051e50 con 0x7ffab006f5a0 2023-10-06T03:45:58.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.246+0000 7ffab519d700 1 -- 172.21.15.55:0/1423495478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7ffab006e860 con 0x7ffab006f5a0 2023-10-06T03:45:58.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.251+0000 7ffa97fff700 1 -- 172.21.15.55:0/1423495478 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v15) v1 ==== 120+0+0 (secure 0 0 0) 0x7ffaa0055470 con 0x7ffab006f5a0 2023-10-06T03:45:58.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.254+0000 7ffab519d700 1 -- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffa9c066f30 msgr2=0x7ffa9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:58.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.254+0000 7ffab519d700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffa9c066f30 0x7ffa9c0693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffab006fd30 tx=0x7ffaa4008040 comp rx=0 tx=0).stop 2023-10-06T03:45:58.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.254+0000 7ffab519d700 1 -- 172.21.15.55:0/1423495478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffab006f5a0 msgr2=0x7ffab0152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:58.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.254+0000 7ffab519d700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffab006f5a0 0x7ffab0152110 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000d8c0 tx=0x7ffaa0008040 comp rx=0 tx=0).stop 2023-10-06T03:45:58.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.254+0000 7ffab519d700 1 -- 172.21.15.55:0/1423495478 shutdown_connections 2023-10-06T03:45:58.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.254+0000 7ffab519d700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffa9c066f30 0x7ffa9c0693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.254+0000 7ffab519d700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffab00c4480 0x7ffab015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.254+0000 7ffab519d700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffab006f5a0 0x7ffab0152110 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.254+0000 7ffab519d700 1 --2- 172.21.15.55:0/1423495478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffab00cb1a0 0x7ffab015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.254+0000 7ffab519d700 1 -- 172.21.15.55:0/1423495478 >> 172.21.15.55:0/1423495478 conn(0x7ffab00cc1e0 msgr2=0x7ffab00aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:58.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.254+0000 7ffab519d700 1 -- 172.21.15.55:0/1423495478 shutdown_connections 2023-10-06T03:45:58.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.254+0000 7ffab519d700 1 -- 172.21.15.55:0/1423495478 wait complete. 2023-10-06T03:45:58.277 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-06T03:45:58.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.463+0000 7fc344f9d700 1 Processor -- start 2023-10-06T03:45:58.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.463+0000 7fc344f9d700 1 -- start start 2023-10-06T03:45:58.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.464+0000 7fc344f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc340156310 0x7fc3400c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:58.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.465+0000 7fc344f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc3400ae710 0x7fc3400c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:58.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.465+0000 7fc344f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3400ca860 con 0x7fc3401566e0 2023-10-06T03:45:58.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.465+0000 7fc344f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3400cab40 con 0x7fc3400ae710 2023-10-06T03:45:58.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.465+0000 7fc344f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3400cae20 con 0x7fc340156310 2023-10-06T03:45:58.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.465+0000 7fc33effd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc3401566e0 0x7fc3400ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57204/0 (socket says 172.21.15.55:57204) 2023-10-06T03:45:58.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.465+0000 7fc33effd700 1 -- 172.21.15.55:0/750474521 learned_addr learned my addr 172.21.15.55:0/750474521 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:45:58.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.465+0000 7fc33ffff700 1 --2- 172.21.15.55:0/750474521 >> v2:172.21.15.55:3300/0 conn(0x7fc3400ae710 0x7fc3400c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:58.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.465+0000 7fc33f7fe700 1 --2- 172.21.15.55:0/750474521 >> v2:172.21.15.163:3300/0 conn(0x7fc340156310 0x7fc3400c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:58.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.465+0000 7fc33ffff700 1 -- 172.21.15.55:0/750474521 >> v2:172.21.15.163:3300/0 conn(0x7fc340156310 msgr2=0x7fc3400c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:58.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.465+0000 7fc33e7fc700 1 -- 172.21.15.55:0/750474521 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407833186 0 0) 0x7fc3400ca860 con 0x7fc3401566e0 2023-10-06T03:45:58.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.465+0000 7fc33ffff700 1 --2- 172.21.15.55:0/750474521 >> v2:172.21.15.163:3300/0 conn(0x7fc340156310 0x7fc3400c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.465+0000 7fc33ffff700 1 -- 172.21.15.55:0/750474521 >> v1:172.21.15.55:6789/0 conn(0x7fc3401566e0 legacy=0x7fc3400ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:58.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.465+0000 7fc33ffff700 1 -- 172.21.15.55:0/750474521 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3400cb100 con 0x7fc3400ae710 2023-10-06T03:45:58.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.466+0000 7fc33ffff700 1 --2- 172.21.15.55:0/750474521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3400ae710 0x7fc3400c6830 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc3400cce20 tx=0x7fc338009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e853e09618546 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:58.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.466+0000 7fc33e7fc700 1 -- 172.21.15.55:0/750474521 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33800e070 con 0x7fc3400ae710 2023-10-06T03:45:58.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.466+0000 7fc33e7fc700 1 -- 172.21.15.55:0/750474521 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33800bc30 con 0x7fc3400ae710 2023-10-06T03:45:58.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.466+0000 7fc33e7fc700 1 -- 172.21.15.55:0/750474521 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338012430 con 0x7fc3400ae710 2023-10-06T03:45:58.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.466+0000 7fc344f9d700 1 -- 172.21.15.55:0/750474521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3400ae710 msgr2=0x7fc3400c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:58.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.466+0000 7fc344f9d700 1 --2- 172.21.15.55:0/750474521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3400ae710 0x7fc3400c6830 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc3400cce20 tx=0x7fc338009780 comp rx=0 tx=0).stop 2023-10-06T03:45:58.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.466+0000 7fc344f9d700 1 -- 172.21.15.55:0/750474521 shutdown_connections 2023-10-06T03:45:58.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.467+0000 7fc344f9d700 1 --2- 172.21.15.55:0/750474521 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3400ae710 0x7fc3400c6830 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.467+0000 7fc344f9d700 1 --2- 172.21.15.55:0/750474521 >> v2:172.21.15.163:3300/0 conn(0x7fc340156310 0x7fc3400c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.467+0000 7fc344f9d700 1 -- 172.21.15.55:0/750474521 >> 172.21.15.55:0/750474521 conn(0x7fc3400cc1f0 msgr2=0x7fc3400cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:58.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.467+0000 7fc344f9d700 1 -- 172.21.15.55:0/750474521 shutdown_connections 2023-10-06T03:45:58.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.467+0000 7fc344f9d700 1 -- 172.21.15.55:0/750474521 wait complete. 2023-10-06T03:45:58.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.468+0000 7fc344f9d700 1 Processor -- start 2023-10-06T03:45:58.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.468+0000 7fc344f9d700 1 -- start start 2023-10-06T03:45:58.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.469+0000 7fc344f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3400ae710 0x7fc340168140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:58.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.469+0000 7fc344f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc340156310 0x7fc340146c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:58.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.469+0000 7fc344f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3401566e0 0x7fc3400b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:58.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.469+0000 7fc344f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc340168680 con 0x7fc3400ae710 2023-10-06T03:45:58.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.469+0000 7fc344f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc340157ea0 con 0x7fc340156310 2023-10-06T03:45:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.469+0000 7fc344f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc340158180 con 0x7fc3401566e0 2023-10-06T03:45:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.469+0000 7fc33f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3400ae710 0x7fc340168140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.469+0000 7fc33f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3400ae710 0x7fc340168140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53024/0 (socket says 172.21.15.55:53024) 2023-10-06T03:45:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.469+0000 7fc33f7fe700 1 -- 172.21.15.55:0/3041908534 learned_addr learned my addr 172.21.15.55:0/3041908534 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.470+0000 7fc33effd700 1 --2- 172.21.15.55:0/3041908534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc340156310 0x7fc340146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.470+0000 7fc33ffff700 1 --2- 172.21.15.55:0/3041908534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3401566e0 0x7fc3400b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.470+0000 7fc33f7fe700 1 -- 172.21.15.55:0/3041908534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3401566e0 msgr2=0x7fc3400b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.470+0000 7fc33f7fe700 1 --2- 172.21.15.55:0/3041908534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3401566e0 0x7fc3400b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.470+0000 7fc33f7fe700 1 -- 172.21.15.55:0/3041908534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc340156310 msgr2=0x7fc340146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.470+0000 7fc33f7fe700 1 --2- 172.21.15.55:0/3041908534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc340156310 0x7fc340146c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:58.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.470+0000 7fc33f7fe700 1 -- 172.21.15.55:0/3041908534 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc338009a20 con 0x7fc3400ae710 2023-10-06T03:45:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.470+0000 7fc33f7fe700 1 --2- 172.21.15.55:0/3041908534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3400ae710 0x7fc340168140 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc33000d860 tx=0x7fc330008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.470+0000 7fc3277fe700 1 -- 172.21.15.55:0/3041908534 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330009610 con 0x7fc3400ae710 2023-10-06T03:45:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.470+0000 7fc3277fe700 1 -- 172.21.15.55:0/3041908534 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc330009770 con 0x7fc3400ae710 2023-10-06T03:45:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.470+0000 7fc3277fe700 1 -- 172.21.15.55:0/3041908534 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330009b30 con 0x7fc3400ae710 2023-10-06T03:45:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.470+0000 7fc344f9d700 1 -- 172.21.15.55:0/3041908534 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc340158460 con 0x7fc3400ae710 2023-10-06T03:45:58.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.471+0000 7fc344f9d700 1 -- 172.21.15.55:0/3041908534 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc340158a20 con 0x7fc3400ae710 2023-10-06T03:45:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.472+0000 7fc344f9d700 1 -- 172.21.15.55:0/3041908534 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3400c69a0 con 0x7fc3400ae710 2023-10-06T03:45:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.472+0000 7fc3277fe700 1 -- 172.21.15.55:0/3041908534 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc33001e460 con 0x7fc3400ae710 2023-10-06T03:45:58.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.473+0000 7fc3277fe700 1 --2- 172.21.15.55:0/3041908534 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc32c066f30 0x7fc32c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:58.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.473+0000 7fc3277fe700 1 -- 172.21.15.55:0/3041908534 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6215+0+0 (secure 0 0 0) 0x7fc330084d50 con 0x7fc3400ae710 2023-10-06T03:45:58.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.474+0000 7fc33effd700 1 --2- 172.21.15.55:0/3041908534 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc32c066f30 0x7fc32c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:58.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.474+0000 7fc33effd700 1 --2- 172.21.15.55:0/3041908534 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc32c066f30 0x7fc32c0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc34015a8b0 tx=0x7fc334009040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:58.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.477+0000 7fc3277fe700 1 -- 172.21.15.55:0/3041908534 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc330088030 con 0x7fc3400ae710 2023-10-06T03:45:58.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:58.764+0000 7fc344f9d700 1 -- 172.21.15.55:0/3041908534 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fc340152610 con 0x7fc3400ae710 2023-10-06T03:45:59.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.735+0000 7fc3277fe700 1 -- 172.21.15.55:0/3041908534 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v453) v1 ==== 158+0+0 (secure 0 0 0) 0x7fc330051dc0 con 0x7fc3400ae710 2023-10-06T03:45:59.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.738+0000 7fc344f9d700 1 -- 172.21.15.55:0/3041908534 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc32c066f30 msgr2=0x7fc32c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:59.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.738+0000 7fc344f9d700 1 --2- 172.21.15.55:0/3041908534 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc32c066f30 0x7fc32c0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc34015a8b0 tx=0x7fc334009040 comp rx=0 tx=0).stop 2023-10-06T03:45:59.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.738+0000 7fc344f9d700 1 -- 172.21.15.55:0/3041908534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3400ae710 msgr2=0x7fc340168140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:59.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.738+0000 7fc344f9d700 1 --2- 172.21.15.55:0/3041908534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3400ae710 0x7fc340168140 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc33000d860 tx=0x7fc330008040 comp rx=0 tx=0).stop 2023-10-06T03:45:59.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.738+0000 7fc344f9d700 1 -- 172.21.15.55:0/3041908534 shutdown_connections 2023-10-06T03:45:59.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.738+0000 7fc344f9d700 1 --2- 172.21.15.55:0/3041908534 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc32c066f30 0x7fc32c0693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:59.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.738+0000 7fc344f9d700 1 --2- 172.21.15.55:0/3041908534 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3401566e0 0x7fc3400b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:59.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.738+0000 7fc344f9d700 1 --2- 172.21.15.55:0/3041908534 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3400ae710 0x7fc340168140 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:59.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.738+0000 7fc344f9d700 1 --2- 172.21.15.55:0/3041908534 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc340156310 0x7fc340146c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:59.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.738+0000 7fc344f9d700 1 -- 172.21.15.55:0/3041908534 >> 172.21.15.55:0/3041908534 conn(0x7fc3400cc1f0 msgr2=0x7fc340151d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:59.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.739+0000 7fc344f9d700 1 -- 172.21.15.55:0/3041908534 shutdown_connections 2023-10-06T03:45:59.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.739+0000 7fc344f9d700 1 -- 172.21.15.55:0/3041908534 wait complete. 2023-10-06T03:45:59.761 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-10-06T03:45:59.761 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-10-06T03:45:59.761 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T03:45:59.761 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-06T03:45:59.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.953+0000 7fcf1eb9d700 1 Processor -- start 2023-10-06T03:45:59.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.953+0000 7fcf1eb9d700 1 -- start start 2023-10-06T03:45:59.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.955+0000 7fcf1eb9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcf18156310 0x7fcf180c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:59.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.955+0000 7fcf1eb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf181566e0 0x7fcf180ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:59.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.955+0000 7fcf1eb9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf180ca860 con 0x7fcf180ca300 2023-10-06T03:45:59.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.955+0000 7fcf1eb9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf180cab40 con 0x7fcf18156310 2023-10-06T03:45:59.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.955+0000 7fcf1eb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf180cae20 con 0x7fcf181566e0 2023-10-06T03:45:59.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.955+0000 7fcf1d39a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf181566e0 0x7fcf180ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:59.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.955+0000 7fcf1db9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcf18156310 0x7fcf180c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:59.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.955+0000 7fcf1db9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcf18156310 0x7fcf180c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:34312/0 (socket says 172.21.15.55:34312) 2023-10-06T03:45:59.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.955+0000 7fcf1d39a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf181566e0 0x7fcf180ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:37922/0 (socket says 172.21.15.55:37922) 2023-10-06T03:45:59.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.955+0000 7fcf1db9b700 1 -- 172.21.15.55:0/2275932468 learned_addr learned my addr 172.21.15.55:0/2275932468 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:59.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.956+0000 7fcf1db9b700 1 -- 172.21.15.55:0/2275932468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf181566e0 msgr2=0x7fcf180ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:45:59.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.956+0000 7fcf1db9b700 1 --2- 172.21.15.55:0/2275932468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf181566e0 0x7fcf180ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:59.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.956+0000 7fcf1db9b700 1 -- 172.21.15.55:0/2275932468 >> v1:172.21.15.163:6789/0 conn(0x7fcf180ca300 legacy=0x7fcf180c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:59.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.956+0000 7fcf1db9b700 1 -- 172.21.15.55:0/2275932468 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf180cb100 con 0x7fcf18156310 2023-10-06T03:45:59.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.956+0000 7fcf1cb99700 1 -- 172.21.15.55:0/2275932468 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186316375 0 0) 0x7fcf180ca860 con 0x7fcf180ca300 2023-10-06T03:45:59.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.956+0000 7fcf1d39a700 1 --2- 172.21.15.55:0/2275932468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf181566e0 0x7fcf180ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:45:59.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.956+0000 7fcf1db9b700 1 --2- 172.21.15.55:0/2275932468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf18156310 0x7fcf180c9dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcf140099b0 tx=0x7fcf14009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=733bc0db62f7d065 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:59.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.956+0000 7fcf1cb99700 1 -- 172.21.15.55:0/2275932468 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf14009ea0 con 0x7fcf18156310 2023-10-06T03:45:59.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.956+0000 7fcf1cb99700 1 -- 172.21.15.55:0/2275932468 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1400b860 con 0x7fcf18156310 2023-10-06T03:45:59.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.956+0000 7fcf1cb99700 1 -- 172.21.15.55:0/2275932468 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1400bc00 con 0x7fcf18156310 2023-10-06T03:45:59.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.957+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/2275932468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf18156310 msgr2=0x7fcf180c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:59.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.957+0000 7fcf1eb9d700 1 --2- 172.21.15.55:0/2275932468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf18156310 0x7fcf180c9dc0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcf140099b0 tx=0x7fcf14009250 comp rx=0 tx=0).stop 2023-10-06T03:45:59.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.957+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/2275932468 shutdown_connections 2023-10-06T03:45:59.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.957+0000 7fcf1eb9d700 1 --2- 172.21.15.55:0/2275932468 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf18156310 0x7fcf180c9dc0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:59.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.957+0000 7fcf1eb9d700 1 --2- 172.21.15.55:0/2275932468 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf181566e0 0x7fcf180ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:59.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.957+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/2275932468 >> 172.21.15.55:0/2275932468 conn(0x7fcf180cc1f0 msgr2=0x7fcf180cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:45:59.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.957+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/2275932468 shutdown_connections 2023-10-06T03:45:59.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.957+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/2275932468 wait complete. 2023-10-06T03:45:59.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.958+0000 7fcf1eb9d700 1 Processor -- start 2023-10-06T03:45:59.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.959+0000 7fcf1eb9d700 1 -- start start 2023-10-06T03:45:59.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1eb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf180ca300 0x7fcf1815f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:59.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1eb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf18156310 0x7fcf180c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:59.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1eb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf181566e0 0x7fcf1813ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:59.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1eb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1815dee0 con 0x7fcf181566e0 2023-10-06T03:45:59.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1eb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1815d310 con 0x7fcf180ca300 2023-10-06T03:45:59.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1eb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf180c37e0 con 0x7fcf18156310 2023-10-06T03:45:59.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1e39c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf181566e0 0x7fcf1813ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1e39c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf181566e0 0x7fcf1813ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53030/0 (socket says 172.21.15.55:53030) 2023-10-06T03:45:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1e39c700 1 -- 172.21.15.55:0/426925418 learned_addr learned my addr 172.21.15.55:0/426925418 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:45:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1db9b700 1 --2- 172.21.15.55:0/426925418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf180ca300 0x7fcf1815f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1d39a700 1 --2- 172.21.15.55:0/426925418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf18156310 0x7fcf180c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1e39c700 1 -- 172.21.15.55:0/426925418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf18156310 msgr2=0x7fcf180c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1e39c700 1 --2- 172.21.15.55:0/426925418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf18156310 0x7fcf180c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1e39c700 1 -- 172.21.15.55:0/426925418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf180ca300 msgr2=0x7fcf1815f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:45:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1e39c700 1 --2- 172.21.15.55:0/426925418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf180ca300 0x7fcf1815f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:45:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.960+0000 7fcf1e39c700 1 -- 172.21.15.55:0/426925418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0800b6d0 con 0x7fcf181566e0 2023-10-06T03:45:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.961+0000 7fcf1e39c700 1 --2- 172.21.15.55:0/426925418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf181566e0 0x7fcf1813ec10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000c890 tx=0x7fcf10001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.961+0000 7fcf0dffb700 1 -- 172.21.15.55:0/426925418 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10012630 con 0x7fcf181566e0 2023-10-06T03:45:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.961+0000 7fcf0dffb700 1 -- 172.21.15.55:0/426925418 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf100127b0 con 0x7fcf181566e0 2023-10-06T03:45:59.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.961+0000 7fcf0dffb700 1 -- 172.21.15.55:0/426925418 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10012b70 con 0x7fcf181566e0 2023-10-06T03:45:59.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.961+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/426925418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf140094f0 con 0x7fcf181566e0 2023-10-06T03:45:59.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.961+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/426925418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1815d890 con 0x7fcf181566e0 2023-10-06T03:45:59.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.962+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/426925418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1815e090 con 0x7fcf181566e0 2023-10-06T03:45:59.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.963+0000 7fcf0dffb700 1 -- 172.21.15.55:0/426925418 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcf10003030 con 0x7fcf181566e0 2023-10-06T03:45:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.964+0000 7fcf0dffb700 1 --2- 172.21.15.55:0/426925418 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcefc066f30 0x7fcefc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:45:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.965+0000 7fcf0dffb700 1 -- 172.21.15.55:0/426925418 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6215+0+0 (secure 0 0 0) 0x7fcf10084720 con 0x7fcf181566e0 2023-10-06T03:45:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.965+0000 7fcf1db9b700 1 --2- 172.21.15.55:0/426925418 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcefc066f30 0x7fcefc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:45:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.966+0000 7fcf1db9b700 1 --2- 172.21.15.55:0/426925418 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcefc066f30 0x7fcefc0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcf140099b0 tx=0x7fcf14017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:45:59.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:45:59.969+0000 7fcf0dffb700 1 -- 172.21.15.55:0/426925418 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf1815e090 con 0x7fcf181566e0 2023-10-06T03:46:00.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.250+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/426925418 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7fcf1804c3c0 con 0x7fcf181566e0 2023-10-06T03:46:00.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.737+0000 7fcf0dffb700 1 -- 172.21.15.55:0/426925418 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v454) v1 ==== 157+0+0 (secure 0 0 0) 0x7fcf100516f0 con 0x7fcf181566e0 2023-10-06T03:46:00.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.739+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/426925418 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcefc066f30 msgr2=0x7fcefc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:00.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.739+0000 7fcf1eb9d700 1 --2- 172.21.15.55:0/426925418 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcefc066f30 0x7fcefc0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcf140099b0 tx=0x7fcf14017000 comp rx=0 tx=0).stop 2023-10-06T03:46:00.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.739+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/426925418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf181566e0 msgr2=0x7fcf1813ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:00.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.739+0000 7fcf1eb9d700 1 --2- 172.21.15.55:0/426925418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf181566e0 0x7fcf1813ec10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000c890 tx=0x7fcf10001630 comp rx=0 tx=0).stop 2023-10-06T03:46:00.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.740+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/426925418 shutdown_connections 2023-10-06T03:46:00.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.740+0000 7fcf1eb9d700 1 --2- 172.21.15.55:0/426925418 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcefc066f30 0x7fcefc0693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:00.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.740+0000 7fcf1eb9d700 1 --2- 172.21.15.55:0/426925418 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcf181566e0 0x7fcf1813ec10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:00.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.740+0000 7fcf1eb9d700 1 --2- 172.21.15.55:0/426925418 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcf180ca300 0x7fcf1815f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:00.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.740+0000 7fcf1eb9d700 1 --2- 172.21.15.55:0/426925418 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcf18156310 0x7fcf180c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:00.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.740+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/426925418 >> 172.21.15.55:0/426925418 conn(0x7fcf180cc1f0 msgr2=0x7fcf18151980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:00.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.740+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/426925418 shutdown_connections 2023-10-06T03:46:00.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.740+0000 7fcf1eb9d700 1 -- 172.21.15.55:0/426925418 wait complete. 2023-10-06T03:46:00.761 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:46:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.951+0000 7fd302f9d700 1 Processor -- start 2023-10-06T03:46:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.951+0000 7fd302f9d700 1 -- start start 2023-10-06T03:46:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.952+0000 7fd302f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2fc0cb1a0 0x7fd2fc15a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.952+0000 7fd302f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd2fc15a8a0 0x7fd2fc06f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.952+0000 7fd302f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd2fc06f5a0 0x7fd2fc0ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.952+0000 7fd302f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2fc15b2e0 con 0x7fd2fc15a8a0 2023-10-06T03:46:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.952+0000 7fd302f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2fc06ebd0 con 0x7fd2fc06f5a0 2023-10-06T03:46:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.952+0000 7fd302f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc15ad20 con 0x7fd2fc0cb1a0 2023-10-06T03:46:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.953+0000 7fd30179a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd2fc15a8a0 0x7fd2fc06f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.953+0000 7fd30179a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd2fc15a8a0 0x7fd2fc06f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53036/0 (socket says 172.21.15.55:53036) 2023-10-06T03:46:00.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.953+0000 7fd30179a700 1 -- 172.21.15.55:0/3028329616 learned_addr learned my addr 172.21.15.55:0/3028329616 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:00.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.953+0000 7fd301f9b700 1 --2- 172.21.15.55:0/3028329616 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2fc0cb1a0 0x7fd2fc15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:00.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.953+0000 7fd30279c700 1 --2- 172.21.15.55:0/3028329616 >> v2:172.21.15.163:3300/0 conn(0x7fd2fc06f5a0 0x7fd2fc0ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:00.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.953+0000 7fd30179a700 1 -- 172.21.15.55:0/3028329616 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2fc0cb1a0 msgr2=0x7fd2fc15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:00.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.953+0000 7fd30179a700 1 --2- 172.21.15.55:0/3028329616 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2fc0cb1a0 0x7fd2fc15a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:00.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.953+0000 7fd30179a700 1 -- 172.21.15.55:0/3028329616 >> v2:172.21.15.163:3300/0 conn(0x7fd2fc06f5a0 msgr2=0x7fd2fc0ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:00.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.953+0000 7fd30179a700 1 --2- 172.21.15.55:0/3028329616 >> v2:172.21.15.163:3300/0 conn(0x7fd2fc06f5a0 0x7fd2fc0ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:00.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.953+0000 7fd30179a700 1 -- 172.21.15.55:0/3028329616 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc15b000 con 0x7fd2fc15a8a0 2023-10-06T03:46:00.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.953+0000 7fd30179a700 1 --2- 172.21.15.55:0/3028329616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2fc15a8a0 0x7fd2fc06f060 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc06d580 tx=0x7fd2ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d8e860ad9e30c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:00.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.953+0000 7fd300f99700 1 -- 172.21.15.55:0/3028329616 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec00b8f0 con 0x7fd2fc15a8a0 2023-10-06T03:46:00.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.954+0000 7fd300f99700 1 -- 172.21.15.55:0/3028329616 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec00ba50 con 0x7fd2fc15a8a0 2023-10-06T03:46:00.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.954+0000 7fd300f99700 1 -- 172.21.15.55:0/3028329616 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec00bdf0 con 0x7fd2fc15a8a0 2023-10-06T03:46:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.954+0000 7fd302f9d700 1 -- 172.21.15.55:0/3028329616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2fc15a8a0 msgr2=0x7fd2fc06f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.954+0000 7fd302f9d700 1 --2- 172.21.15.55:0/3028329616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2fc15a8a0 0x7fd2fc06f060 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc06d580 tx=0x7fd2ec009250 comp rx=0 tx=0).stop 2023-10-06T03:46:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.954+0000 7fd302f9d700 1 -- 172.21.15.55:0/3028329616 shutdown_connections 2023-10-06T03:46:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.954+0000 7fd302f9d700 1 --2- 172.21.15.55:0/3028329616 >> v2:172.21.15.163:3300/0 conn(0x7fd2fc06f5a0 0x7fd2fc0ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.954+0000 7fd302f9d700 1 --2- 172.21.15.55:0/3028329616 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2fc0cb1a0 0x7fd2fc15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.954+0000 7fd302f9d700 1 --2- 172.21.15.55:0/3028329616 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2fc15a8a0 0x7fd2fc06f060 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.954+0000 7fd302f9d700 1 -- 172.21.15.55:0/3028329616 >> 172.21.15.55:0/3028329616 conn(0x7fd2fc0cc1e0 msgr2=0x7fd2fc0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.954+0000 7fd302f9d700 1 -- 172.21.15.55:0/3028329616 shutdown_connections 2023-10-06T03:46:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.954+0000 7fd302f9d700 1 -- 172.21.15.55:0/3028329616 wait complete. 2023-10-06T03:46:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.956+0000 7fd302f9d700 1 Processor -- start 2023-10-06T03:46:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.956+0000 7fd302f9d700 1 -- start start 2023-10-06T03:46:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd302f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2fc06f5a0 0x7fd2fc153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd302f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2fc0cb1a0 0x7fd2fc15be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd302f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2fc15c3a0 0x7fd2fc169e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd302f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0aef80 con 0x7fd2fc0cb1a0 2023-10-06T03:46:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd302f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0afb00 con 0x7fd2fc15c3a0 2023-10-06T03:46:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd302f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0af1d0 con 0x7fd2fc06f5a0 2023-10-06T03:46:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd30179a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2fc0cb1a0 0x7fd2fc15be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd30179a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2fc0cb1a0 0x7fd2fc15be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53050/0 (socket says 172.21.15.55:53050) 2023-10-06T03:46:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd30179a700 1 -- 172.21.15.55:0/403769793 learned_addr learned my addr 172.21.15.55:0/403769793 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd301f9b700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2fc06f5a0 0x7fd2fc153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd30279c700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2fc15c3a0 0x7fd2fc169e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd30179a700 1 -- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2fc06f5a0 msgr2=0x7fd2fc153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd30179a700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2fc06f5a0 0x7fd2fc153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd30179a700 1 -- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2fc15c3a0 msgr2=0x7fd2fc169e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd30179a700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2fc15c3a0 0x7fd2fc169e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:00.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd30179a700 1 -- 172.21.15.55:0/403769793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec0094f0 con 0x7fd2fc0cb1a0 2023-10-06T03:46:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.957+0000 7fd30279c700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2fc15c3a0 0x7fd2fc169e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:46:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.958+0000 7fd30179a700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2fc0cb1a0 0x7fd2fc15be60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc06ce80 tx=0x7fd2ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.958+0000 7fd2f1ffb700 1 -- 172.21.15.55:0/403769793 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec011e60 con 0x7fd2fc0cb1a0 2023-10-06T03:46:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.958+0000 7fd2f1ffb700 1 -- 172.21.15.55:0/403769793 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec01a630 con 0x7fd2fc0cb1a0 2023-10-06T03:46:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.958+0000 7fd2f1ffb700 1 -- 172.21.15.55:0/403769793 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec01a9f0 con 0x7fd2fc0cb1a0 2023-10-06T03:46:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.958+0000 7fd302f9d700 1 -- 172.21.15.55:0/403769793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc0af4b0 con 0x7fd2fc0cb1a0 2023-10-06T03:46:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.958+0000 7fd302f9d700 1 -- 172.21.15.55:0/403769793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc153ff0 con 0x7fd2fc0cb1a0 2023-10-06T03:46:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.959+0000 7fd302f9d700 1 -- 172.21.15.55:0/403769793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2fc0afc70 con 0x7fd2fc0cb1a0 2023-10-06T03:46:00.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.960+0000 7fd2f1ffb700 1 -- 172.21.15.55:0/403769793 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2ec0233e0 con 0x7fd2fc0cb1a0 2023-10-06T03:46:00.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.961+0000 7fd2f1ffb700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd2e0066f30 0x7fd2e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:00.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.961+0000 7fd2f1ffb700 1 -- 172.21.15.55:0/403769793 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6215+0+0 (secure 0 0 0) 0x7fd2ec085e50 con 0x7fd2fc0cb1a0 2023-10-06T03:46:00.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.961+0000 7fd301f9b700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd2e0066f30 0x7fd2e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:00.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.962+0000 7fd301f9b700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd2e0066f30 0x7fd2e00693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8006fd0 tx=0x7fd2f8008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:00.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:00.965+0000 7fd2f1ffb700 1 -- 172.21.15.55:0/403769793 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2ec052e20 con 0x7fd2fc0cb1a0 2023-10-06T03:46:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.165+0000 7fd302f9d700 1 -- 172.21.15.55:0/403769793 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd2fc04bb80 con 0x7fd2fc0cb1a0 2023-10-06T03:46:01.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.171+0000 7fd2f1ffb700 1 -- 172.21.15.55:0/403769793 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd2ec0150c0 con 0x7fd2fc0cb1a0 2023-10-06T03:46:01.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.173+0000 7fd302f9d700 1 -- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd2e0066f30 msgr2=0x7fd2e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:01.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.173+0000 7fd302f9d700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd2e0066f30 0x7fd2e00693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8006fd0 tx=0x7fd2f8008040 comp rx=0 tx=0).stop 2023-10-06T03:46:01.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.173+0000 7fd302f9d700 1 -- 172.21.15.55:0/403769793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2fc0cb1a0 msgr2=0x7fd2fc15be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:01.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.173+0000 7fd302f9d700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2fc0cb1a0 0x7fd2fc15be60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc06ce80 tx=0x7fd2ec017000 comp rx=0 tx=0).stop 2023-10-06T03:46:01.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.173+0000 7fd302f9d700 1 -- 172.21.15.55:0/403769793 shutdown_connections 2023-10-06T03:46:01.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.173+0000 7fd302f9d700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd2e0066f30 0x7fd2e00693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:01.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.173+0000 7fd302f9d700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd2fc15c3a0 0x7fd2fc169e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:01.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.173+0000 7fd302f9d700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd2fc06f5a0 0x7fd2fc153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:01.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.173+0000 7fd302f9d700 1 --2- 172.21.15.55:0/403769793 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd2fc0cb1a0 0x7fd2fc15be60 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:01.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.173+0000 7fd302f9d700 1 -- 172.21.15.55:0/403769793 >> 172.21.15.55:0/403769793 conn(0x7fd2fc0cc1e0 msgr2=0x7fd2fc0c0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:01.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.173+0000 7fd302f9d700 1 -- 172.21.15.55:0/403769793 shutdown_connections 2023-10-06T03:46:01.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.173+0000 7fd302f9d700 1 -- 172.21.15.55:0/403769793 wait complete. 2023-10-06T03:46:01.192 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:46:01.193 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:46:01.271 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:46:01.352 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:46:01.353 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:46:01.353 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:46:01.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.539+0000 7f8ab339d700 1 Processor -- start 2023-10-06T03:46:01.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.539+0000 7f8ab339d700 1 -- start start 2023-10-06T03:46:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.541+0000 7f8ab339d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8aac150550 0x7f8aac157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.541+0000 7f8ab339d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8aac0c7410 0x7f8aac069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.541+0000 7f8ab339d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aac06e2a0 con 0x7f8aac150180 2023-10-06T03:46:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.541+0000 7f8ab339d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac06b2e0 con 0x7f8aac0c7410 2023-10-06T03:46:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.541+0000 7f8ab339d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac0c78e0 con 0x7f8aac150550 2023-10-06T03:46:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.541+0000 7f8ab239b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8aac150180 0x7f8aac0c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57206/0 (socket says 172.21.15.55:57206) 2023-10-06T03:46:01.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.541+0000 7f8ab239b700 1 -- 172.21.15.55:0/2289695212 learned_addr learned my addr 172.21.15.55:0/2289695212 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.541+0000 7f8ab2b9c700 1 --2- 172.21.15.55:0/2289695212 >> v2:172.21.15.55:3300/0 conn(0x7f8aac0c7410 0x7f8aac069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.541+0000 7f8ab1b9a700 1 --2- 172.21.15.55:0/2289695212 >> v2:172.21.15.163:3300/0 conn(0x7f8aac150550 0x7f8aac157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.541+0000 7f8ab1399700 1 -- 172.21.15.55:0/2289695212 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157040958 0 0) 0x7f8aac06e2a0 con 0x7f8aac150180 2023-10-06T03:46:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.541+0000 7f8ab1399700 1 -- 172.21.15.55:0/2289695212 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a98003570 con 0x7f8aac150180 2023-10-06T03:46:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.542+0000 7f8ab2b9c700 1 -- 172.21.15.55:0/2289695212 >> v2:172.21.15.163:3300/0 conn(0x7f8aac150550 msgr2=0x7f8aac157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.542+0000 7f8ab2b9c700 1 --2- 172.21.15.55:0/2289695212 >> v2:172.21.15.163:3300/0 conn(0x7f8aac150550 0x7f8aac157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.542+0000 7f8ab2b9c700 1 -- 172.21.15.55:0/2289695212 >> v1:172.21.15.55:6789/0 conn(0x7f8aac150180 legacy=0x7f8aac0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.542+0000 7f8ab2b9c700 1 -- 172.21.15.55:0/2289695212 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac0c7bc0 con 0x7f8aac0c7410 2023-10-06T03:46:01.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.542+0000 7f8ab2b9c700 1 --2- 172.21.15.55:0/2289695212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8aac0c7410 0x7f8aac069f50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8aac06d1e0 tx=0x7f8aa400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdf4f66d4311434 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.542+0000 7f8ab1399700 1 -- 172.21.15.55:0/2289695212 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400f070 con 0x7f8aac0c7410 2023-10-06T03:46:01.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.542+0000 7f8ab1399700 1 -- 172.21.15.55:0/2289695212 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa400a2e0 con 0x7f8aac0c7410 2023-10-06T03:46:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.542+0000 7f8ab1399700 1 -- 172.21.15.55:0/2289695212 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa400cbc0 con 0x7f8aac0c7410 2023-10-06T03:46:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.543+0000 7f8ab339d700 1 -- 172.21.15.55:0/2289695212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8aac0c7410 msgr2=0x7f8aac069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.543+0000 7f8ab339d700 1 --2- 172.21.15.55:0/2289695212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8aac0c7410 0x7f8aac069f50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8aac06d1e0 tx=0x7f8aa400a890 comp rx=0 tx=0).stop 2023-10-06T03:46:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.543+0000 7f8ab339d700 1 -- 172.21.15.55:0/2289695212 shutdown_connections 2023-10-06T03:46:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.543+0000 7f8ab339d700 1 --2- 172.21.15.55:0/2289695212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8aac0c7410 0x7f8aac069f50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.543+0000 7f8ab339d700 1 --2- 172.21.15.55:0/2289695212 >> v2:172.21.15.163:3300/0 conn(0x7f8aac150550 0x7f8aac157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.543+0000 7f8ab339d700 1 -- 172.21.15.55:0/2289695212 >> 172.21.15.55:0/2289695212 conn(0x7f8aac0cc1f0 msgr2=0x7f8aac0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.543+0000 7f8ab339d700 1 -- 172.21.15.55:0/2289695212 shutdown_connections 2023-10-06T03:46:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.543+0000 7f8ab339d700 1 -- 172.21.15.55:0/2289695212 wait complete. 2023-10-06T03:46:01.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.544+0000 7f8ab339d700 1 Processor -- start 2023-10-06T03:46:01.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.545+0000 7f8ab339d700 1 -- start start 2023-10-06T03:46:01.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.545+0000 7f8ab339d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8aac0c7410 0x7f8aac16a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:01.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.545+0000 7f8ab339d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8aac150180 0x7f8aac16d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:01.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.545+0000 7f8ab339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aac150550 0x7f8aac0aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:01.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.546+0000 7f8ab339d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac16aa30 con 0x7f8aac150180 2023-10-06T03:46:01.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.546+0000 7f8ab339d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac0b1500 con 0x7f8aac0c7410 2023-10-06T03:46:01.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.546+0000 7f8ab339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac0b00a0 con 0x7f8aac150550 2023-10-06T03:46:01.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.546+0000 7f8ab1b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8aac150180 0x7f8aac16d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:01.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.546+0000 7f8ab1b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8aac150180 0x7f8aac16d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53060/0 (socket says 172.21.15.55:53060) 2023-10-06T03:46:01.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.546+0000 7f8ab1b9a700 1 -- 172.21.15.55:0/123705963 learned_addr learned my addr 172.21.15.55:0/123705963 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:01.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.546+0000 7f8ab239b700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8aac0c7410 0x7f8aac16a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:01.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.546+0000 7f8ab2b9c700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aac150550 0x7f8aac0aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:01.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.546+0000 7f8ab1b9a700 1 -- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aac150550 msgr2=0x7f8aac0aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:01.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.546+0000 7f8ab1b9a700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aac150550 0x7f8aac0aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:01.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.546+0000 7f8ab1b9a700 1 -- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8aac0c7410 msgr2=0x7f8aac16a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:01.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.546+0000 7f8ab1b9a700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8aac0c7410 0x7f8aac16a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:01.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.546+0000 7f8ab1b9a700 1 -- 172.21.15.55:0/123705963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa400ab30 con 0x7f8aac150180 2023-10-06T03:46:01.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.546+0000 7f8ab2b9c700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aac150550 0x7f8aac0aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:46:01.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.547+0000 7f8ab1b9a700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8aac150180 0x7f8aac16d040 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00ecb0 tx=0x7f8a9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:01.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.547+0000 7f8aa27fc700 1 -- 172.21.15.55:0/123705963 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c00c920 con 0x7f8aac150180 2023-10-06T03:46:01.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.547+0000 7f8aa27fc700 1 -- 172.21.15.55:0/123705963 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9c00ca80 con 0x7f8aac150180 2023-10-06T03:46:01.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.547+0000 7f8aa27fc700 1 -- 172.21.15.55:0/123705963 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c00ce20 con 0x7f8aac150180 2023-10-06T03:46:01.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.547+0000 7f8ab339d700 1 -- 172.21.15.55:0/123705963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac0b1ca0 con 0x7f8aac150180 2023-10-06T03:46:01.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.547+0000 7f8ab339d700 1 -- 172.21.15.55:0/123705963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac0afdc0 con 0x7f8aac150180 2023-10-06T03:46:01.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.548+0000 7f8ab339d700 1 -- 172.21.15.55:0/123705963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aac0b1af0 con 0x7f8aac150180 2023-10-06T03:46:01.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.549+0000 7f8aa27fc700 1 -- 172.21.15.55:0/123705963 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a9c013680 con 0x7f8aac150180 2023-10-06T03:46:01.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.550+0000 7f8aa27fc700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a90066f30 0x7f8a900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:01.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.550+0000 7f8aa27fc700 1 -- 172.21.15.55:0/123705963 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6215+0+0 (secure 0 0 0) 0x7f8a9c084b20 con 0x7f8aac150180 2023-10-06T03:46:01.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.550+0000 7f8ab239b700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a90066f30 0x7f8a900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:01.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.551+0000 7f8ab239b700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a90066f30 0x7f8a900693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8007fd0 tx=0x7f8aa8009040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:01.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.554+0000 7f8aa27fc700 1 -- 172.21.15.55:0/123705963 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a9c051af0 con 0x7f8aac150180 2023-10-06T03:46:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.805+0000 7f8ab339d700 1 -- 172.21.15.55:0/123705963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8aac04c3c0 con 0x7f8aac150180 2023-10-06T03:46:01.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.806+0000 7f8aa27fc700 1 -- 172.21.15.55:0/123705963 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f8a9c017de0 con 0x7f8aac150180 2023-10-06T03:46:01.807 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:46:01.807 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4894,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1532010025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1532010025},{"type":"v1","addr":"172.21.15.55:6834","nonce":1532010025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4897,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4258042685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":4258042685},{"type":"v1","addr":"172.21.15.55:6835","nonce":4258042685}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4900,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/430119478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":430119478},{"type":"v1","addr":"172.21.15.163:6836","nonce":430119478}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5533,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/2861144485","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2861144485},{"type":"v1","addr":"172.21.15.163:6833","nonce":2861144485}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-10-06T03:46:01.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.808+0000 7f8ab339d700 1 -- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a90066f30 msgr2=0x7f8a900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:01.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.808+0000 7f8ab339d700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a90066f30 0x7f8a900693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8007fd0 tx=0x7f8aa8009040 comp rx=0 tx=0).stop 2023-10-06T03:46:01.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.808+0000 7f8ab339d700 1 -- 172.21.15.55:0/123705963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8aac150180 msgr2=0x7f8aac16d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:01.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.808+0000 7f8ab339d700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8aac150180 0x7f8aac16d040 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00ecb0 tx=0x7f8a9c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:46:01.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.808+0000 7f8ab339d700 1 -- 172.21.15.55:0/123705963 shutdown_connections 2023-10-06T03:46:01.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.808+0000 7f8ab339d700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a90066f30 0x7f8a900693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:01.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.808+0000 7f8ab339d700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8aac150550 0x7f8aac0aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:01.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.808+0000 7f8ab339d700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8aac0c7410 0x7f8aac16a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:01.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.808+0000 7f8ab339d700 1 --2- 172.21.15.55:0/123705963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8aac150180 0x7f8aac16d040 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:01.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.808+0000 7f8ab339d700 1 -- 172.21.15.55:0/123705963 >> 172.21.15.55:0/123705963 conn(0x7f8aac0cc1f0 msgr2=0x7f8aac0c1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:01.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.808+0000 7f8ab339d700 1 -- 172.21.15.55:0/123705963 shutdown_connections 2023-10-06T03:46:01.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:01.808+0000 7f8ab339d700 1 -- 172.21.15.55:0/123705963 wait complete. 2023-10-06T03:46:01.811 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 19 2023-10-06T03:46:01.828 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:46:01.828 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:46:01.828 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:46:01.829 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:46:01.829 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:46:01.829 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:46:01.829 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:46:01.829 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:46:01.829 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:46:01.830 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:46:01.830 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:46:01.830 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:46:01.830 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:46:01.830 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:46:01.831 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:46:01.831 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:46:01.845 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:46:01.846 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:46:01.846 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:46:01.847 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:46:01.848 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:46:01.848 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:46:01.848 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:46:01.849 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:46:01.891 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:46:01.891 DEBUG:teuthology.parallel:result is None 2023-10-06T03:46:01.892 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:46:01.892 DEBUG:teuthology.parallel:result is None 2023-10-06T03:46:01.894 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:46:01.894 DEBUG:teuthology.parallel:result is None 2023-10-06T03:46:01.895 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:46:01.895 DEBUG:teuthology.parallel:result is None 2023-10-06T03:46:01.895 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:46:02.029 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T03:46:02.039 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-10-06T03:46:02.039 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T03:46:02.040 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2023-10-06T03:46:02.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.092+0000 7f752b59e700 1 Processor -- start 2023-10-06T03:46:02.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.092+0000 7f752b59e700 1 -- start start 2023-10-06T03:46:02.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f752b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f752c142ea0 0x7f752c0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:02.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f752b59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f752c06c0f0 0x7f752c06a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:02.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f752b59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752c0b40f0 con 0x7f752c0ad870 2023-10-06T03:46:02.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f752b59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752c069640 con 0x7f752c142ea0 2023-10-06T03:46:02.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f752b59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752c069920 con 0x7f752c06c0f0 2023-10-06T03:46:02.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f752a59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f752c142ea0 0x7f752c0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f7529d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f752c0ad870 0x7f752c06b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57212/0 (socket says 172.21.15.55:57212) 2023-10-06T03:46:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f7529d9b700 1 -- 172.21.15.55:0/2321588545 learned_addr learned my addr 172.21.15.55:0/2321588545 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f752a59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f752c142ea0 0x7f752c0b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53094/0 (socket says 172.21.15.55:53094) 2023-10-06T03:46:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f752ad9d700 1 --2- 172.21.15.55:0/2321588545 >> v2:172.21.15.163:3300/0 conn(0x7f752c06c0f0 0x7f752c06a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f752a59c700 1 -- 172.21.15.55:0/2321588545 >> v2:172.21.15.163:3300/0 conn(0x7f752c06c0f0 msgr2=0x7f752c06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f752a59c700 1 --2- 172.21.15.55:0/2321588545 >> v2:172.21.15.163:3300/0 conn(0x7f752c06c0f0 0x7f752c06a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f752959a700 1 -- 172.21.15.55:0/2321588545 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847191829 0 0) 0x7f752c0b40f0 con 0x7f752c0ad870 2023-10-06T03:46:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f752a59c700 1 -- 172.21.15.55:0/2321588545 >> v1:172.21.15.55:6789/0 conn(0x7f752c0ad870 legacy=0x7f752c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.094+0000 7f752a59c700 1 -- 172.21.15.55:0/2321588545 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c069c00 con 0x7f752c142ea0 2023-10-06T03:46:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.095+0000 7f752a59c700 1 --2- 172.21.15.55:0/2321588545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f752c142ea0 0x7f752c0b3630 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f752c071330 tx=0x7f751c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a609bff97c10e220 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:02.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.095+0000 7f752959a700 1 -- 172.21.15.55:0/2321588545 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c011630 con 0x7f752c142ea0 2023-10-06T03:46:02.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.095+0000 7f752959a700 1 -- 172.21.15.55:0/2321588545 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751c011790 con 0x7f752c142ea0 2023-10-06T03:46:02.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.095+0000 7f752959a700 1 -- 172.21.15.55:0/2321588545 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c011b30 con 0x7f752c142ea0 2023-10-06T03:46:02.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.095+0000 7f752b59e700 1 -- 172.21.15.55:0/2321588545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f752c142ea0 msgr2=0x7f752c0b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:02.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.095+0000 7f752b59e700 1 --2- 172.21.15.55:0/2321588545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f752c142ea0 0x7f752c0b3630 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f752c071330 tx=0x7f751c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.096+0000 7f752b59e700 1 -- 172.21.15.55:0/2321588545 shutdown_connections 2023-10-06T03:46:02.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.096+0000 7f752b59e700 1 --2- 172.21.15.55:0/2321588545 >> v2:172.21.15.163:3300/0 conn(0x7f752c06c0f0 0x7f752c06a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.096+0000 7f752b59e700 1 --2- 172.21.15.55:0/2321588545 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f752c142ea0 0x7f752c0b3630 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.096+0000 7f752b59e700 1 -- 172.21.15.55:0/2321588545 >> 172.21.15.55:0/2321588545 conn(0x7f752c0ce430 msgr2=0x7f752c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:02.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.096+0000 7f752b59e700 1 -- 172.21.15.55:0/2321588545 shutdown_connections 2023-10-06T03:46:02.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.096+0000 7f752b59e700 1 -- 172.21.15.55:0/2321588545 wait complete. 2023-10-06T03:46:02.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.097+0000 7f752b59e700 1 Processor -- start 2023-10-06T03:46:02.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.097+0000 7f752b59e700 1 -- start start 2023-10-06T03:46:02.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.098+0000 7f752b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f752c06c0f0 0x7f752c0acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:02.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.098+0000 7f752b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f752c142ea0 0x7f752c0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.098+0000 7f752b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f752c140120 0x7f752c0cd100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.098+0000 7f752b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c16a850 con 0x7f752c142ea0 2023-10-06T03:46:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.098+0000 7f752b59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c15d200 con 0x7f752c140120 2023-10-06T03:46:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.098+0000 7f752b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c15d850 con 0x7f752c06c0f0 2023-10-06T03:46:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.098+0000 7f7529d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f752c142ea0 0x7f752c0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.098+0000 7f7529d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f752c142ea0 0x7f752c0ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53104/0 (socket says 172.21.15.55:53104) 2023-10-06T03:46:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.098+0000 7f7529d9b700 1 -- 172.21.15.55:0/4279870870 learned_addr learned my addr 172.21.15.55:0/4279870870 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.098+0000 7f752ad9d700 1 --2- 172.21.15.55:0/4279870870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f752c140120 0x7f752c0cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.098+0000 7f752a59c700 1 --2- 172.21.15.55:0/4279870870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f752c06c0f0 0x7f752c0acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.099+0000 7f7529d9b700 1 -- 172.21.15.55:0/4279870870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f752c06c0f0 msgr2=0x7f752c0acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.099+0000 7f7529d9b700 1 --2- 172.21.15.55:0/4279870870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f752c06c0f0 0x7f752c0acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.099+0000 7f7529d9b700 1 -- 172.21.15.55:0/4279870870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f752c140120 msgr2=0x7f752c0cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.099+0000 7f7529d9b700 1 --2- 172.21.15.55:0/4279870870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f752c140120 0x7f752c0cd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.099+0000 7f7529d9b700 1 -- 172.21.15.55:0/4279870870 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c009420 con 0x7f752c142ea0 2023-10-06T03:46:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.099+0000 7f7529d9b700 1 --2- 172.21.15.55:0/4279870870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f752c142ea0 0x7f752c0ccbc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f752000c840 tx=0x7f7520007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.099+0000 7f751a7fc700 1 -- 172.21.15.55:0/4279870870 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520005c00 con 0x7f752c142ea0 2023-10-06T03:46:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.099+0000 7f751a7fc700 1 -- 172.21.15.55:0/4279870870 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7520005d60 con 0x7f752c142ea0 2023-10-06T03:46:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.099+0000 7f751a7fc700 1 -- 172.21.15.55:0/4279870870 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520007300 con 0x7f752c142ea0 2023-10-06T03:46:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.099+0000 7f752b59e700 1 -- 172.21.15.55:0/4279870870 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752c0ca2f0 con 0x7f752c142ea0 2023-10-06T03:46:02.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.099+0000 7f752b59e700 1 -- 172.21.15.55:0/4279870870 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c16a470 con 0x7f752c142ea0 2023-10-06T03:46:02.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.101+0000 7f752b59e700 1 -- 172.21.15.55:0/4279870870 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f752c16a990 con 0x7f752c142ea0 2023-10-06T03:46:02.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.101+0000 7f751a7fc700 1 -- 172.21.15.55:0/4279870870 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f752001b3b0 con 0x7f752c142ea0 2023-10-06T03:46:02.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.102+0000 7f751a7fc700 1 --2- 172.21.15.55:0/4279870870 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f750c066f30 0x7f750c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:02.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.102+0000 7f751a7fc700 1 -- 172.21.15.55:0/4279870870 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6215+0+0 (secure 0 0 0) 0x7f7520083bb0 con 0x7f752c142ea0 2023-10-06T03:46:02.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.103+0000 7f752a59c700 1 --2- 172.21.15.55:0/4279870870 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f750c066f30 0x7f750c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:02.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.103+0000 7f752a59c700 1 --2- 172.21.15.55:0/4279870870 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f750c066f30 0x7f750c0693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f751c0091e0 tx=0x7f751c017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:02.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.106+0000 7f751a7fc700 1 -- 172.21.15.55:0/4279870870 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f752c16a990 con 0x7f752c142ea0 2023-10-06T03:46:02.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.326+0000 7f752b59e700 1 -- 172.21.15.55:0/4279870870 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f752c16a990 con 0x7f752c142ea0 2023-10-06T03:46:02.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.759+0000 7f751a7fc700 1 -- 172.21.15.55:0/4279870870 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7f752c16a990 con 0x7f752c142ea0 2023-10-06T03:46:02.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.762+0000 7f752b59e700 1 -- 172.21.15.55:0/4279870870 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f750c066f30 msgr2=0x7f750c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.762+0000 7f752b59e700 1 --2- 172.21.15.55:0/4279870870 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f750c066f30 0x7f750c0693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f751c0091e0 tx=0x7f751c017000 comp rx=0 tx=0).stop 2023-10-06T03:46:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.762+0000 7f752b59e700 1 -- 172.21.15.55:0/4279870870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f752c142ea0 msgr2=0x7f752c0ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.762+0000 7f752b59e700 1 --2- 172.21.15.55:0/4279870870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f752c142ea0 0x7f752c0ccbc0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f752000c840 tx=0x7f7520007db0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.762+0000 7f752b59e700 1 -- 172.21.15.55:0/4279870870 shutdown_connections 2023-10-06T03:46:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.762+0000 7f752b59e700 1 --2- 172.21.15.55:0/4279870870 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f750c066f30 0x7f750c0693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.762+0000 7f752b59e700 1 --2- 172.21.15.55:0/4279870870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f752c140120 0x7f752c0cd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.762+0000 7f752b59e700 1 --2- 172.21.15.55:0/4279870870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f752c06c0f0 0x7f752c0acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.762+0000 7f752b59e700 1 --2- 172.21.15.55:0/4279870870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f752c142ea0 0x7f752c0ccbc0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.762+0000 7f752b59e700 1 -- 172.21.15.55:0/4279870870 >> 172.21.15.55:0/4279870870 conn(0x7f752c0ce430 msgr2=0x7f752c06a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:02.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.763+0000 7f752b59e700 1 -- 172.21.15.55:0/4279870870 shutdown_connections 2023-10-06T03:46:02.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.763+0000 7f752b59e700 1 -- 172.21.15.55:0/4279870870 wait complete. 2023-10-06T03:46:02.766 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2023-10-06T03:46:02.787 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:46:02.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.973+0000 7ff1c0d9d700 1 Processor -- start 2023-10-06T03:46:02.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.973+0000 7ff1c0d9d700 1 -- start start 2023-10-06T03:46:02.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.974+0000 7ff1c0d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff1bc150550 0x7ff1bc157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:02.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.974+0000 7ff1c0d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1bc06e2a0 con 0x7ff1bc0c7410 2023-10-06T03:46:02.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.974+0000 7ff1c0d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1bc06b2e0 con 0x7ff1bc150180 2023-10-06T03:46:02.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.974+0000 7ff1c0d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1bc0c78e0 con 0x7ff1bc150550 2023-10-06T03:46:02.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.974+0000 7ff1bbfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff1bc0c7410 0x7ff1bc069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57222/0 (socket says 172.21.15.55:57222) 2023-10-06T03:46:02.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.974+0000 7ff1baffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff1bc150550 0x7ff1bc157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:02.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.974+0000 7ff1bbfff700 1 -- 172.21.15.55:0/3971611594 learned_addr learned my addr 172.21.15.55:0/3971611594 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:02.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.975+0000 7ff1ba7fc700 1 -- 172.21.15.55:0/3971611594 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167755681 0 0) 0x7ff1bc06e2a0 con 0x7ff1bc0c7410 2023-10-06T03:46:02.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.975+0000 7ff1ba7fc700 1 -- 172.21.15.55:0/3971611594 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1a4003570 con 0x7ff1bc0c7410 2023-10-06T03:46:02.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.975+0000 7ff1baffd700 1 -- 172.21.15.55:0/3971611594 >> v1:172.21.15.163:6789/0 conn(0x7ff1bc150180 legacy=0x7ff1bc0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:02.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.975+0000 7ff1baffd700 1 -- 172.21.15.55:0/3971611594 >> v1:172.21.15.55:6789/0 conn(0x7ff1bc0c7410 legacy=0x7ff1bc069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:02.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.975+0000 7ff1baffd700 1 -- 172.21.15.55:0/3971611594 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc0c7bc0 con 0x7ff1bc150550 2023-10-06T03:46:02.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.975+0000 7ff1baffd700 1 --2- 172.21.15.55:0/3971611594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1bc150550 0x7ff1bc157770 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00098e0 tx=0x7ff1b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11850245209c4a91 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:02.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.975+0000 7ff1ba7fc700 1 -- 172.21.15.55:0/3971611594 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b000f6b0 con 0x7ff1bc150550 2023-10-06T03:46:02.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.975+0000 7ff1ba7fc700 1 -- 172.21.15.55:0/3971611594 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b000f810 con 0x7ff1bc150550 2023-10-06T03:46:02.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.975+0000 7ff1ba7fc700 1 -- 172.21.15.55:0/3971611594 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b000fbb0 con 0x7ff1bc150550 2023-10-06T03:46:02.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.976+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3971611594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1bc150550 msgr2=0x7ff1bc157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:02.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.976+0000 7ff1c0d9d700 1 --2- 172.21.15.55:0/3971611594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1bc150550 0x7ff1bc157770 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00098e0 tx=0x7ff1b00091a0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.976+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3971611594 shutdown_connections 2023-10-06T03:46:02.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.976+0000 7ff1c0d9d700 1 --2- 172.21.15.55:0/3971611594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1bc150550 0x7ff1bc157770 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.976+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3971611594 >> 172.21.15.55:0/3971611594 conn(0x7ff1bc0cc1f0 msgr2=0x7ff1bc0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:02.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.976+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3971611594 shutdown_connections 2023-10-06T03:46:02.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.976+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3971611594 wait complete. 2023-10-06T03:46:02.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.977+0000 7ff1c0d9d700 1 Processor -- start 2023-10-06T03:46:02.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.978+0000 7ff1c0d9d700 1 -- start start 2023-10-06T03:46:02.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.978+0000 7ff1c0d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1bc0c7410 0x7ff1bc0adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:02.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.978+0000 7ff1c0d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1bc150180 0x7ff1bc14b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:02.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1c0d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1bc150550 0x7ff1bc0af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:02.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1c0d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc0b0a20 con 0x7ff1bc150550 2023-10-06T03:46:02.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1c0d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc14b810 con 0x7ff1bc0c7410 2023-10-06T03:46:02.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1c0d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc14d590 con 0x7ff1bc150180 2023-10-06T03:46:02.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1bbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1bc150550 0x7ff1bc0af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:02.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1bbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1bc150550 0x7ff1bc0af970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53110/0 (socket says 172.21.15.55:53110) 2023-10-06T03:46:02.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1bbfff700 1 -- 172.21.15.55:0/3830723558 learned_addr learned my addr 172.21.15.55:0/3830723558 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:02.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1bb7fe700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1bc0c7410 0x7ff1bc0adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:02.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1baffd700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1bc150180 0x7ff1bc14b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:02.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1bbfff700 1 -- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1bc150180 msgr2=0x7ff1bc14b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:02.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1bbfff700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1bc150180 0x7ff1bc14b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1bbfff700 1 -- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1bc0c7410 msgr2=0x7ff1bc0adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:02.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1bbfff700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1bc0c7410 0x7ff1bc0adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:02.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1bbfff700 1 -- 172.21.15.55:0/3830723558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b0009420 con 0x7ff1bc150550 2023-10-06T03:46:02.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1baffd700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1bc150180 0x7ff1bc14b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:46:02.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.979+0000 7ff1bbfff700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1bc150550 0x7ff1bc0af970 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400cc70 tx=0x7ff1b4001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:02.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.980+0000 7ff1a37fe700 1 -- 172.21.15.55:0/3830723558 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b400ce80 con 0x7ff1bc150550 2023-10-06T03:46:02.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.980+0000 7ff1a37fe700 1 -- 172.21.15.55:0/3830723558 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1b400ab30 con 0x7ff1bc150550 2023-10-06T03:46:02.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.980+0000 7ff1a37fe700 1 -- 172.21.15.55:0/3830723558 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40070f0 con 0x7ff1bc150550 2023-10-06T03:46:02.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.980+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3830723558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1bc14d260 con 0x7ff1bc150550 2023-10-06T03:46:02.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.980+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3830723558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1bc0b0180 con 0x7ff1bc150550 2023-10-06T03:46:02.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.981+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3830723558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1bc14d700 con 0x7ff1bc150550 2023-10-06T03:46:02.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.982+0000 7ff1a37fe700 1 -- 172.21.15.55:0/3830723558 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1b40074e0 con 0x7ff1bc150550 2023-10-06T03:46:02.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.983+0000 7ff1a37fe700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff1a8066f30 0x7ff1a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:02.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.983+0000 7ff1a37fe700 1 -- 172.21.15.55:0/3830723558 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5570+0+0 (secure 0 0 0) 0x7ff1b4084a00 con 0x7ff1bc150550 2023-10-06T03:46:02.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.983+0000 7ff1bb7fe700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff1a8066f30 0x7ff1a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:02.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.984+0000 7ff1bb7fe700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff1a8066f30 0x7ff1a80693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac003860 tx=0x7ff1ac009040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:02.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:02.987+0000 7ff1a37fe700 1 -- 172.21.15.55:0/3830723558 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1b4051c40 con 0x7ff1bc150550 2023-10-06T03:46:03.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.271+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3830723558 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff1bc04c3c0 con 0x7ff1bc150550 2023-10-06T03:46:03.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.272+0000 7ff1a37fe700 1 -- 172.21.15.55:0/3830723558 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff1b4055260 con 0x7ff1bc150550 2023-10-06T03:46:03.273 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:46:03.273 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:46:03.273 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2023-10-06T03:46:03.274 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.274 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2023-10-06T03:46:03.274 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9li2RaGRAAgRjtbw3h40vylEPjolzlpw==", 2023-10-06T03:46:03.274 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.274 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:46:03.274 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:46:03.274 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:46:03.274 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.274 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.274 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.275 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.275 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2023-10-06T03:46:03.275 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lXZu7LxAAu3QAtdDxtD/AuuqGG12sDw==", 2023-10-06T03:46:03.275 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.275 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:46:03.275 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:46:03.275 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:46:03.275 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.275 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.275 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.275 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.275 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2023-10-06T03:46:03.275 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lS22fJBAAZi6CfyLyJAFgIkaarad1mA==", 2023-10-06T03:46:03.276 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.276 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:46:03.276 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:46:03.276 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:46:03.276 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.276 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.276 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.276 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.276 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2023-10-06T03:46:03.276 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9ll0RIOhAAGtGdPv4Y2bge7xIv3eu2NQ==", 2023-10-06T03:46:03.276 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.277 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:46:03.277 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:46:03.277 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:46:03.277 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.277 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.277 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.277 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.277 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2023-10-06T03:46:03.277 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDHfx9lmTeaDRAAuycc9i7fPj8xrDkuZCAGVg==", 2023-10-06T03:46:03.277 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.277 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:46:03.277 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:46:03.278 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.278 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.278 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.278 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.279 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2023-10-06T03:46:03.279 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDKfx9lxOdXEhAA9S8tdOvhnmBeEbutqnUsZg==", 2023-10-06T03:46:03.279 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.279 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:46:03.279 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:46:03.279 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.279 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.279 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.279 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.279 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2023-10-06T03:46:03.279 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDNfx9l76I+GRAAjaj0gM7An9BCyGmfQgaDFw==", 2023-10-06T03:46:03.280 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.280 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:46:03.280 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:46:03.280 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.280 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.280 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.280 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.280 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2023-10-06T03:46:03.280 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDQfx9lsPK+HxAAzQCV0EuBenqOkUpiQW9uog==", 2023-10-06T03:46:03.280 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.280 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:46:03.280 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:46:03.281 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.281 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.281 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.281 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.281 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2023-10-06T03:46:03.281 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDWfx9lSHNrMxAAIn7awDmljO2cRtKyDalWnw==", 2023-10-06T03:46:03.281 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.281 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:46:03.281 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:46:03.281 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.281 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.281 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.282 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.282 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2023-10-06T03:46:03.282 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDZfx9lFoWuOBAAhzQOt3YChvuv0MhrspOjPw==", 2023-10-06T03:46:03.282 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.282 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:46:03.282 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:46:03.282 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.282 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.282 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.282 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.283 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2023-10-06T03:46:03.283 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDcfx9lB8iSOhAAwamSgGIXZ8owFE4FdbOOBw==", 2023-10-06T03:46:03.283 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.283 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:46:03.283 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:46:03.283 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.283 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.283 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.283 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.283 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2023-10-06T03:46:03.283 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDgfx9lUb3mABAAJ75UO2UQg2r+l+tDYVn0Dg==", 2023-10-06T03:46:03.283 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.284 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:46:03.284 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:46:03.284 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.284 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.284 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.284 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.284 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2023-10-06T03:46:03.284 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lfNvCBxAA1ZJq4XsWTZHVY0u0NdOqIw==", 2023-10-06T03:46:03.284 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.284 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:46:03.284 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2023-10-06T03:46:03.284 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:46:03.285 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.285 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.285 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.285 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2023-10-06T03:46:03.285 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lnvkoFhAAafxsOmItxmwUr1GJrGYJ6g==", 2023-10-06T03:46:03.285 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.285 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:46:03.285 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:46:03.285 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:46:03.285 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:46:03.285 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.285 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.286 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.286 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2023-10-06T03:46:03.286 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9l+9b4DhAAc5XHdikWbXsEblSE3FGScQ==", 2023-10-06T03:46:03.286 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.286 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:46:03.286 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:46:03.286 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:46:03.286 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:46:03.286 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.286 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.286 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.286 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2023-10-06T03:46:03.287 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9luSXrHBAAg6Ufa1Ep/EOgsQtuOPQgYg==", 2023-10-06T03:46:03.287 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.287 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:46:03.287 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:46:03.287 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:46:03.287 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:46:03.287 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.287 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.287 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.287 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2023-10-06T03:46:03.287 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC+fx9lEb50FxAAt0Oq28gsXtQvol7TV8trgg==", 2023-10-06T03:46:03.287 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.288 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:46:03.288 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:46:03.288 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:46:03.288 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.288 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.288 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.288 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.288 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2023-10-06T03:46:03.288 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lQAANEhAAsNiWPqCNEyUS3IELXSC1Cw==", 2023-10-06T03:46:03.288 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.288 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:46:03.288 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T03:46:03.289 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.289 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.289 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:03.289 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:03.289 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2023-10-06T03:46:03.289 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lCwU2ABAAMijxXd8hwgk3XzcdP4NZyg==", 2023-10-06T03:46:03.289 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:46:03.289 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:46:03.289 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T03:46:03.289 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:46:03.289 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.289 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:03.290 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2023-10-06T03:46:03.290 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:46:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.274+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff1a8066f30 msgr2=0x7ff1a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.274+0000 7ff1c0d9d700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff1a8066f30 0x7ff1a80693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac003860 tx=0x7ff1ac009040 comp rx=0 tx=0).stop 2023-10-06T03:46:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.274+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3830723558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1bc150550 msgr2=0x7ff1bc0af970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.274+0000 7ff1c0d9d700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1bc150550 0x7ff1bc0af970 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400cc70 tx=0x7ff1b4001aa0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.274+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3830723558 shutdown_connections 2023-10-06T03:46:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.274+0000 7ff1c0d9d700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff1a8066f30 0x7ff1a80693e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.274+0000 7ff1c0d9d700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff1bc150550 0x7ff1bc0af970 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.274+0000 7ff1c0d9d700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff1bc0c7410 0x7ff1bc0adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.274+0000 7ff1c0d9d700 1 --2- 172.21.15.55:0/3830723558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff1bc150180 0x7ff1bc14b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.274+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3830723558 >> 172.21.15.55:0/3830723558 conn(0x7ff1bc0cc1f0 msgr2=0x7ff1bc0c8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:03.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.274+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3830723558 shutdown_connections 2023-10-06T03:46:03.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.274+0000 7ff1c0d9d700 1 -- 172.21.15.55:0/3830723558 wait complete. 2023-10-06T03:46:03.293 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:46:03.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.478+0000 7f8af979d700 1 Processor -- start 2023-10-06T03:46:03.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.478+0000 7f8af979d700 1 -- start start 2023-10-06T03:46:03.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.479+0000 7f8af979d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8af4142ea0 0x7f8af40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:03.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.479+0000 7f8af979d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af40ad870 0x7f8af406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:03.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.479+0000 7f8af979d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af40b40f0 con 0x7f8af40b3b70 2023-10-06T03:46:03.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.479+0000 7f8af979d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af4069640 con 0x7f8af4142ea0 2023-10-06T03:46:03.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.479+0000 7f8af979d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af4069920 con 0x7f8af40ad870 2023-10-06T03:46:03.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.480+0000 7f8af3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8af4142ea0 0x7f8af40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:03.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.480+0000 7f8af3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8af4142ea0 0x7f8af40b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50002/0 (socket says 172.21.15.55:50002) 2023-10-06T03:46:03.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.480+0000 7f8af3fff700 1 -- 172.21.15.55:0/1676506039 learned_addr learned my addr 172.21.15.55:0/1676506039 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:03.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.480+0000 7f8af37fe700 1 --2- 172.21.15.55:0/1676506039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af40ad870 0x7f8af406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:03.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.480+0000 7f8af3fff700 1 -- 172.21.15.55:0/1676506039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af40ad870 msgr2=0x7f8af406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:03.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.480+0000 7f8af3fff700 1 --2- 172.21.15.55:0/1676506039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af40ad870 0x7f8af406b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.480+0000 7f8af3fff700 1 -- 172.21.15.55:0/1676506039 >> v1:172.21.15.163:6789/0 conn(0x7f8af40b3b70 legacy=0x7f8af406cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:03.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.480+0000 7f8af3fff700 1 -- 172.21.15.55:0/1676506039 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af4069c00 con 0x7f8af4142ea0 2023-10-06T03:46:03.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.480+0000 7f8af3fff700 1 --2- 172.21.15.55:0/1676506039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af4142ea0 0x7f8af40b3630 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40099b0 tx=0x7f8ae4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d748c9d07bd66d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:03.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.481+0000 7f8af2ffd700 1 -- 172.21.15.55:0/1676506039 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4009ea0 con 0x7f8af4142ea0 2023-10-06T03:46:03.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.481+0000 7f8af2ffd700 1 -- 172.21.15.55:0/1676506039 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae400b860 con 0x7f8af4142ea0 2023-10-06T03:46:03.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.481+0000 7f8af2ffd700 1 -- 172.21.15.55:0/1676506039 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400bc00 con 0x7f8af4142ea0 2023-10-06T03:46:03.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.481+0000 7f8af979d700 1 -- 172.21.15.55:0/1676506039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af4142ea0 msgr2=0x7f8af40b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:03.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.481+0000 7f8af979d700 1 --2- 172.21.15.55:0/1676506039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af4142ea0 0x7f8af40b3630 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8ae40099b0 tx=0x7f8ae4009250 comp rx=0 tx=0).stop 2023-10-06T03:46:03.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.481+0000 7f8af979d700 1 -- 172.21.15.55:0/1676506039 shutdown_connections 2023-10-06T03:46:03.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.481+0000 7f8af979d700 1 --2- 172.21.15.55:0/1676506039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af4142ea0 0x7f8af40b3630 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.481+0000 7f8af979d700 1 --2- 172.21.15.55:0/1676506039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af40ad870 0x7f8af406b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.481+0000 7f8af979d700 1 -- 172.21.15.55:0/1676506039 >> 172.21.15.55:0/1676506039 conn(0x7f8af40ce430 msgr2=0x7f8af40ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:03.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.481+0000 7f8af979d700 1 -- 172.21.15.55:0/1676506039 shutdown_connections 2023-10-06T03:46:03.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.481+0000 7f8af979d700 1 -- 172.21.15.55:0/1676506039 wait complete. 2023-10-06T03:46:03.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.483+0000 7f8af979d700 1 Processor -- start 2023-10-06T03:46:03.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.483+0000 7f8af979d700 1 -- start start 2023-10-06T03:46:03.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af979d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af40b3b70 0x7f8af41590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af979d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af4142ea0 0x7f8af4141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af979d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af415c600 0x7f8af40aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af979d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af415ca70 con 0x7f8af40b3b70 2023-10-06T03:46:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af979d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af415cd50 con 0x7f8af4142ea0 2023-10-06T03:46:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af979d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af415d030 con 0x7f8af415c600 2023-10-06T03:46:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af40b3b70 0x7f8af41590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af40b3b70 0x7f8af41590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50012/0 (socket says 172.21.15.55:50012) 2023-10-06T03:46:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af3fff700 1 -- 172.21.15.55:0/2445938133 learned_addr learned my addr 172.21.15.55:0/2445938133 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af37fe700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af4142ea0 0x7f8af4141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af8f9c700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af415c600 0x7f8af40aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:03.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af3fff700 1 -- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af415c600 msgr2=0x7f8af40aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:03.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af3fff700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af415c600 0x7f8af40aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af3fff700 1 -- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af4142ea0 msgr2=0x7f8af4141160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:03.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af3fff700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af4142ea0 0x7f8af4141160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af3fff700 1 -- 172.21.15.55:0/2445938133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae40094f0 con 0x7f8af40b3b70 2023-10-06T03:46:03.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af37fe700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af4142ea0 0x7f8af4141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:46:03.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.484+0000 7f8af8f9c700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af415c600 0x7f8af40aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:46:03.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.485+0000 7f8af3fff700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af40b3b70 0x7f8af41590a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8af406f0d0 tx=0x7f8ae4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:03.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.485+0000 7f8adbfff700 1 -- 172.21.15.55:0/2445938133 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae400bbe0 con 0x7f8af40b3b70 2023-10-06T03:46:03.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.485+0000 7f8adbfff700 1 -- 172.21.15.55:0/2445938133 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae4011420 con 0x7f8af40b3b70 2023-10-06T03:46:03.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.485+0000 7f8adbfff700 1 -- 172.21.15.55:0/2445938133 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae40117c0 con 0x7f8af40b3b70 2023-10-06T03:46:03.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.485+0000 7f8af979d700 1 -- 172.21.15.55:0/2445938133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af415d310 con 0x7f8af40b3b70 2023-10-06T03:46:03.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.485+0000 7f8af979d700 1 -- 172.21.15.55:0/2445938133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af41595e0 con 0x7f8af40b3b70 2023-10-06T03:46:03.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.486+0000 7f8af979d700 1 -- 172.21.15.55:0/2445938133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af415d810 con 0x7f8af40b3b70 2023-10-06T03:46:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.487+0000 7f8adbfff700 1 -- 172.21.15.55:0/2445938133 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ae4011bf0 con 0x7f8af40b3b70 2023-10-06T03:46:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.489+0000 7f8adbfff700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8ae0066f30 0x7f8ae00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.489+0000 7f8adbfff700 1 -- 172.21.15.55:0/2445938133 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5570+0+0 (secure 0 0 0) 0x7f8ae4085e80 con 0x7f8af40b3b70 2023-10-06T03:46:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.489+0000 7f8af37fe700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8ae0066f30 0x7f8ae00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:03.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.490+0000 7f8af37fe700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8ae0066f30 0x7f8ae00693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8006f00 tx=0x7f8ae8008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:03.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.493+0000 7f8adbfff700 1 -- 172.21.15.55:0/2445938133 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ae40530d0 con 0x7f8af40b3b70 2023-10-06T03:46:03.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.698+0000 7f8af979d700 1 -- 172.21.15.55:0/2445938133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8af404c3c0 con 0x7f8af40b3b70 2023-10-06T03:46:03.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.699+0000 7f8adbfff700 1 -- 172.21.15.55:0/2445938133 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+14249 (secure 0 0 0) 0x7f8ae40566f0 con 0x7f8af40b3b70 2023-10-06T03:46:03.701 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:46:03.701 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":60,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:46:02.767763+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:46:03.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.701+0000 7f8af979d700 1 -- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8ae0066f30 msgr2=0x7f8ae00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:03.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.701+0000 7f8af979d700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8ae0066f30 0x7f8ae00693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8006f00 tx=0x7f8ae8008040 comp rx=0 tx=0).stop 2023-10-06T03:46:03.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.701+0000 7f8af979d700 1 -- 172.21.15.55:0/2445938133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af40b3b70 msgr2=0x7f8af41590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:03.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.701+0000 7f8af979d700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af40b3b70 0x7f8af41590a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8af406f0d0 tx=0x7f8ae4017000 comp rx=0 tx=0).stop 2023-10-06T03:46:03.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.702+0000 7f8af979d700 1 -- 172.21.15.55:0/2445938133 shutdown_connections 2023-10-06T03:46:03.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.702+0000 7f8af979d700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8ae0066f30 0x7f8ae00693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.702+0000 7f8af979d700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af415c600 0x7f8af40aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.702+0000 7f8af979d700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af40b3b70 0x7f8af41590a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.702+0000 7f8af979d700 1 --2- 172.21.15.55:0/2445938133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af4142ea0 0x7f8af4141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.702+0000 7f8af979d700 1 -- 172.21.15.55:0/2445938133 >> 172.21.15.55:0/2445938133 conn(0x7f8af40ce430 msgr2=0x7f8af40cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:03.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.702+0000 7f8af979d700 1 -- 172.21.15.55:0/2445938133 shutdown_connections 2023-10-06T03:46:03.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.702+0000 7f8af979d700 1 -- 172.21.15.55:0/2445938133 wait complete. 2023-10-06T03:46:03.721 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T03:46:03.721 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:46:03.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.912+0000 7f7f5899d700 1 Processor -- start 2023-10-06T03:46:03.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.912+0000 7f7f5899d700 1 -- start start 2023-10-06T03:46:03.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.913+0000 7f7f5899d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7f540cb1a0 0x7f7f5415a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:03.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.913+0000 7f7f5899d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f540ae0d0 0x7f7f540ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:03.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.913+0000 7f7f5899d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f5406ebd0 con 0x7f7f5415a8a0 2023-10-06T03:46:03.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.913+0000 7f7f5899d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f5415b2e0 con 0x7f7f540cb1a0 2023-10-06T03:46:03.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.913+0000 7f7f5899d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f540aea30 con 0x7f7f540ae0d0 2023-10-06T03:46:03.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.913+0000 7f7f52ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7f5415a8a0 0x7f7f5406f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39438/0 (socket says 172.21.15.55:39438) 2023-10-06T03:46:03.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.913+0000 7f7f52ffd700 1 -- 172.21.15.55:0/1912339017 learned_addr learned my addr 172.21.15.55:0/1912339017 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:03.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.913+0000 7f7f537fe700 1 --2- 172.21.15.55:0/1912339017 >> v2:172.21.15.163:3300/0 conn(0x7f7f540cb1a0 0x7f7f5415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:03.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.913+0000 7f7f53fff700 1 --2- 172.21.15.55:0/1912339017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f540ae0d0 0x7f7f540ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:03.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.914+0000 7f7f527fc700 1 -- 172.21.15.55:0/1912339017 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279672376 0 0) 0x7f7f5406ebd0 con 0x7f7f5415a8a0 2023-10-06T03:46:03.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.914+0000 7f7f527fc700 1 -- 172.21.15.55:0/1912339017 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f3c003570 con 0x7f7f5415a8a0 2023-10-06T03:46:03.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.914+0000 7f7f53fff700 1 -- 172.21.15.55:0/1912339017 >> v2:172.21.15.163:3300/0 conn(0x7f7f540cb1a0 msgr2=0x7f7f5415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:03.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.914+0000 7f7f53fff700 1 --2- 172.21.15.55:0/1912339017 >> v2:172.21.15.163:3300/0 conn(0x7f7f540cb1a0 0x7f7f5415a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.914+0000 7f7f53fff700 1 -- 172.21.15.55:0/1912339017 >> v1:172.21.15.55:6789/0 conn(0x7f7f5415a8a0 legacy=0x7f7f5406f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:03.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.914+0000 7f7f53fff700 1 -- 172.21.15.55:0/1912339017 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5415ad20 con 0x7f7f540ae0d0 2023-10-06T03:46:03.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.914+0000 7f7f537fe700 1 --2- 172.21.15.55:0/1912339017 >> v2:172.21.15.163:3300/0 conn(0x7f7f540cb1a0 0x7f7f5415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:46:03.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.914+0000 7f7f53fff700 1 --2- 172.21.15.55:0/1912339017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f540ae0d0 0x7f7f540ae4f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009e60 tx=0x7f7f4c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e687e7ba2f01fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:03.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.915+0000 7f7f527fc700 1 -- 172.21.15.55:0/1912339017 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00e070 con 0x7f7f540ae0d0 2023-10-06T03:46:03.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.915+0000 7f7f527fc700 1 -- 172.21.15.55:0/1912339017 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c00bb40 con 0x7f7f540ae0d0 2023-10-06T03:46:03.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.915+0000 7f7f527fc700 1 -- 172.21.15.55:0/1912339017 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c012430 con 0x7f7f540ae0d0 2023-10-06T03:46:03.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.915+0000 7f7f5899d700 1 -- 172.21.15.55:0/1912339017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f540ae0d0 msgr2=0x7f7f540ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:03.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.915+0000 7f7f5899d700 1 --2- 172.21.15.55:0/1912339017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f540ae0d0 0x7f7f540ae4f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009e60 tx=0x7f7f4c009780 comp rx=0 tx=0).stop 2023-10-06T03:46:03.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.915+0000 7f7f5899d700 1 -- 172.21.15.55:0/1912339017 shutdown_connections 2023-10-06T03:46:03.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.915+0000 7f7f5899d700 1 --2- 172.21.15.55:0/1912339017 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f540ae0d0 0x7f7f540ae4f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.915+0000 7f7f5899d700 1 --2- 172.21.15.55:0/1912339017 >> v2:172.21.15.163:3300/0 conn(0x7f7f540cb1a0 0x7f7f5415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.915+0000 7f7f5899d700 1 -- 172.21.15.55:0/1912339017 >> 172.21.15.55:0/1912339017 conn(0x7f7f540cc1e0 msgr2=0x7f7f540cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:03.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.916+0000 7f7f5899d700 1 -- 172.21.15.55:0/1912339017 shutdown_connections 2023-10-06T03:46:03.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.916+0000 7f7f5899d700 1 -- 172.21.15.55:0/1912339017 wait complete. 2023-10-06T03:46:03.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.917+0000 7f7f5899d700 1 Processor -- start 2023-10-06T03:46:03.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.917+0000 7f7f5899d700 1 -- start start 2023-10-06T03:46:03.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.918+0000 7f7f5899d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f540ae0d0 0x7f7f541592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:03.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.918+0000 7f7f5899d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f540cb1a0 0x7f7f541582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:03.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.918+0000 7f7f5899d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f541587e0 0x7f7f54165320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:03.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.918+0000 7f7f5899d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54158c80 con 0x7f7f541587e0 2023-10-06T03:46:03.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.918+0000 7f7f5899d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f54166070 con 0x7f7f540ae0d0 2023-10-06T03:46:03.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.918+0000 7f7f5899d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f540c3290 con 0x7f7f540cb1a0 2023-10-06T03:46:03.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.918+0000 7f7f53fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f541587e0 0x7f7f54165320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:03.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.918+0000 7f7f53fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f541587e0 0x7f7f54165320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50028/0 (socket says 172.21.15.55:50028) 2023-10-06T03:46:03.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.918+0000 7f7f53fff700 1 -- 172.21.15.55:0/1138949960 learned_addr learned my addr 172.21.15.55:0/1138949960 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:03.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.919+0000 7f7f537fe700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f540ae0d0 0x7f7f541592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:03.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.919+0000 7f7f52ffd700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f540cb1a0 0x7f7f541582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:03.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.919+0000 7f7f53fff700 1 -- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f540cb1a0 msgr2=0x7f7f541582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:03.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.919+0000 7f7f53fff700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f540cb1a0 0x7f7f541582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.919+0000 7f7f53fff700 1 -- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f540ae0d0 msgr2=0x7f7f541592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:03.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.919+0000 7f7f53fff700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f540ae0d0 0x7f7f541592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:03.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.919+0000 7f7f53fff700 1 -- 172.21.15.55:0/1138949960 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f44009420 con 0x7f7f541587e0 2023-10-06T03:46:03.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.919+0000 7f7f52ffd700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f540cb1a0 0x7f7f541582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:46:03.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.919+0000 7f7f53fff700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f541587e0 0x7f7f54165320 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7f540aeb70 tx=0x7f7f4c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:03.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.919+0000 7f7f3b7fe700 1 -- 172.21.15.55:0/1138949960 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c027470 con 0x7f7f541587e0 2023-10-06T03:46:03.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.919+0000 7f7f3b7fe700 1 -- 172.21.15.55:0/1138949960 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c0275f0 con 0x7f7f541587e0 2023-10-06T03:46:03.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.919+0000 7f7f5899d700 1 -- 172.21.15.55:0/1138949960 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4c009a20 con 0x7f7f541587e0 2023-10-06T03:46:03.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.919+0000 7f7f3b7fe700 1 -- 172.21.15.55:0/1138949960 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0279b0 con 0x7f7f541587e0 2023-10-06T03:46:03.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.920+0000 7f7f5899d700 1 -- 172.21.15.55:0/1138949960 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f540c3850 con 0x7f7f541587e0 2023-10-06T03:46:03.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.921+0000 7f7f5899d700 1 -- 172.21.15.55:0/1138949960 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5406c450 con 0x7f7f541587e0 2023-10-06T03:46:03.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.921+0000 7f7f3b7fe700 1 -- 172.21.15.55:0/1138949960 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7f4c024070 con 0x7f7f541587e0 2023-10-06T03:46:03.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.922+0000 7f7f3b7fe700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7f40066f30 0x7f7f400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:03.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.922+0000 7f7f3b7fe700 1 -- 172.21.15.55:0/1138949960 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5570+0+0 (secure 0 0 0) 0x7f7f4c086fe0 con 0x7f7f541587e0 2023-10-06T03:46:03.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.923+0000 7f7f537fe700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7f40066f30 0x7f7f400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:03.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.924+0000 7f7f537fe700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7f40066f30 0x7f7f400693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7f440093f0 tx=0x7f7f44008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:03.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:03.926+0000 7f7f3b7fe700 1 -- 172.21.15.55:0/1138949960 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f4c054250 con 0x7f7f541587e0 2023-10-06T03:46:04.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.128+0000 7f7f5899d700 1 -- 172.21.15.55:0/1138949960 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7f5415b000 con 0x7f7f541587e0 2023-10-06T03:46:04.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.128+0000 7f7f3b7fe700 1 -- 172.21.15.55:0/1138949960 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v60) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7f4c057870 con 0x7f7f541587e0 2023-10-06T03:46:04.130 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:46:04.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.130+0000 7f7f5899d700 1 -- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7f40066f30 msgr2=0x7f7f400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:04.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.130+0000 7f7f5899d700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7f40066f30 0x7f7f400693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7f440093f0 tx=0x7f7f44008040 comp rx=0 tx=0).stop 2023-10-06T03:46:04.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.130+0000 7f7f5899d700 1 -- 172.21.15.55:0/1138949960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f541587e0 msgr2=0x7f7f54165320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:04.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.130+0000 7f7f5899d700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f541587e0 0x7f7f54165320 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7f540aeb70 tx=0x7f7f4c019000 comp rx=0 tx=0).stop 2023-10-06T03:46:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.130+0000 7f7f5899d700 1 -- 172.21.15.55:0/1138949960 shutdown_connections 2023-10-06T03:46:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.130+0000 7f7f5899d700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7f40066f30 0x7f7f400693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.130+0000 7f7f5899d700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7f541587e0 0x7f7f54165320 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.130+0000 7f7f5899d700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7f540ae0d0 0x7f7f541592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.130+0000 7f7f5899d700 1 --2- 172.21.15.55:0/1138949960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7f540cb1a0 0x7f7f541582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.130+0000 7f7f5899d700 1 -- 172.21.15.55:0/1138949960 >> 172.21.15.55:0/1138949960 conn(0x7f7f540cc1e0 msgr2=0x7f7f540c75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.131+0000 7f7f5899d700 1 -- 172.21.15.55:0/1138949960 shutdown_connections 2023-10-06T03:46:04.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.131+0000 7f7f5899d700 1 -- 172.21.15.55:0/1138949960 wait complete. 2023-10-06T03:46:04.149 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:46:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.339+0000 7f6db759e700 1 Processor -- start 2023-10-06T03:46:04.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.339+0000 7f6db759e700 1 -- start start 2023-10-06T03:46:04.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.341+0000 7f6db759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 0x7f6db80c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.341+0000 7f6db759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db80ca920 con 0x7f6db806bfa0 2023-10-06T03:46:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.341+0000 7f6db759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db806af00 con 0x7f6db8068aa0 2023-10-06T03:46:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.341+0000 7f6db759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db806b1e0 con 0x7f6db8159a70 2023-10-06T03:46:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.341+0000 7f6db5d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6db806bfa0 0x7f6db8068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39446/0 (socket says 172.21.15.55:39446) 2023-10-06T03:46:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.341+0000 7f6db5d9b700 1 -- 172.21.15.55:0/843530738 learned_addr learned my addr 172.21.15.55:0/843530738 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.341+0000 7f6db559a700 1 -- 172.21.15.55:0/843530738 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989261826 0 0) 0x7f6db80ca920 con 0x7f6db806bfa0 2023-10-06T03:46:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.341+0000 7f6db559a700 1 -- 172.21.15.55:0/843530738 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6da0003570 con 0x7f6db806bfa0 2023-10-06T03:46:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.341+0000 7f6db659c700 1 --2- 172.21.15.55:0/843530738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 0x7f6db80c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db559a700 1 -- 172.21.15.55:0/843530738 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2769653454 0 0) 0x7f6da0003570 con 0x7f6db806bfa0 2023-10-06T03:46:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db559a700 1 -- 172.21.15.55:0/843530738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 msgr2=0x7f6db80c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:04.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db559a700 1 --2- 172.21.15.55:0/843530738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 0x7f6db80c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db559a700 1 -- 172.21.15.55:0/843530738 >> v1:172.21.15.163:6789/0 conn(0x7f6db8068aa0 legacy=0x7f6db80caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db559a700 1 -- 172.21.15.55:0/843530738 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db806b4c0 con 0x7f6db806bfa0 2023-10-06T03:46:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db559a700 1 -- 172.21.15.55:0/843530738 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f6dac0015f0 con 0x7f6db806bfa0 2023-10-06T03:46:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db559a700 1 -- 172.21.15.55:0/843530738 >> v1:172.21.15.55:6789/0 conn(0x7f6db806bfa0 legacy=0x7f6db8068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db559a700 1 --2- 172.21.15.55:0/843530738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 0x7f6da0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db559a700 1 --2- 172.21.15.55:0/843530738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6da0004060 0x7f6da0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db559a700 1 --2- 172.21.15.55:0/843530738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da0004d50 0x7f6da00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db559a700 1 -- 172.21.15.55:0/843530738 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db806b1e0 con 0x7f6da0004d50 2023-10-06T03:46:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db559a700 1 -- 172.21.15.55:0/843530738 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da000bc20 con 0x7f6da0004060 2023-10-06T03:46:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db559a700 1 -- 172.21.15.55:0/843530738 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da000bde0 con 0x7f6db8159a70 2023-10-06T03:46:04.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db659c700 1 --2- 172.21.15.55:0/843530738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 0x7f6da0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db5d9b700 1 --2- 172.21.15.55:0/843530738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6da0004060 0x7f6da0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.342+0000 7f6db6d9d700 1 --2- 172.21.15.55:0/843530738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da0004d50 0x7f6da00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.343+0000 7f6db659c700 1 -- 172.21.15.55:0/843530738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6da0004060 msgr2=0x7f6da0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:04.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.343+0000 7f6db659c700 1 --2- 172.21.15.55:0/843530738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6da0004060 0x7f6da0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.343+0000 7f6db659c700 1 -- 172.21.15.55:0/843530738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da0004d50 msgr2=0x7f6da00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:04.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.343+0000 7f6db659c700 1 --2- 172.21.15.55:0/843530738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da0004d50 0x7f6da00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.343+0000 7f6db659c700 1 -- 172.21.15.55:0/843530738 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da0003570 con 0x7f6db8159a70 2023-10-06T03:46:04.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.343+0000 7f6db6d9d700 1 --2- 172.21.15.55:0/843530738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da0004d50 0x7f6da00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:46:04.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.343+0000 7f6db5d9b700 1 --2- 172.21.15.55:0/843530738 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6da0004060 0x7f6da0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:46:04.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.343+0000 7f6db659c700 1 --2- 172.21.15.55:0/843530738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 0x7f6da0003a90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009c00 tx=0x7f6da80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=140c4c1b16802705 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:04.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.343+0000 7f6db559a700 1 -- 172.21.15.55:0/843530738 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800e710 con 0x7f6db8159a70 2023-10-06T03:46:04.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.344+0000 7f6db559a700 1 -- 172.21.15.55:0/843530738 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da800e870 con 0x7f6db8159a70 2023-10-06T03:46:04.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.344+0000 7f6db559a700 1 -- 172.21.15.55:0/843530738 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da800ec30 con 0x7f6db8159a70 2023-10-06T03:46:04.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.344+0000 7f6db759e700 1 -- 172.21.15.55:0/843530738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 msgr2=0x7f6da0003a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:04.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.344+0000 7f6db759e700 1 --2- 172.21.15.55:0/843530738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 0x7f6da0003a90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6da8009c00 tx=0x7f6da80091a0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.344+0000 7f6db6d9d700 1 -- 172.21.15.55:0/843530738 reap_dead start 2023-10-06T03:46:04.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.344+0000 7f6db759e700 1 -- 172.21.15.55:0/843530738 shutdown_connections 2023-10-06T03:46:04.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.344+0000 7f6db759e700 1 -- 172.21.15.55:0/843530738 >> 172.21.15.55:0/843530738 conn(0x7f6db80cddc0 msgr2=0x7f6db80c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.344+0000 7f6db759e700 1 -- 172.21.15.55:0/843530738 shutdown_connections 2023-10-06T03:46:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.344+0000 7f6db759e700 1 -- 172.21.15.55:0/843530738 wait complete. 2023-10-06T03:46:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.346+0000 7f6db759e700 1 Processor -- start 2023-10-06T03:46:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.346+0000 7f6db759e700 1 -- start start 2023-10-06T03:46:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 0x7f6db8159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db81509d0 0x7f6db815d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db80c5dd0 0x7f6db80c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db815c6c0 con 0x7f6db81509d0 2023-10-06T03:46:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db80c6780 con 0x7f6db80c5dd0 2023-10-06T03:46:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db81576f0 con 0x7f6db8159a70 2023-10-06T03:46:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db5d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db81509d0 0x7f6db815d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db5d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db81509d0 0x7f6db815d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50058/0 (socket says 172.21.15.55:50058) 2023-10-06T03:46:04.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db5d9b700 1 -- 172.21.15.55:0/3294824527 learned_addr learned my addr 172.21.15.55:0/3294824527 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:04.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db5d9b700 1 -- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 msgr2=0x7f6db8159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:04.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db659c700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 0x7f6db8159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db6d9d700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db80c5dd0 0x7f6db80c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db5d9b700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 0x7f6db8159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db5d9b700 1 -- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db80c5dd0 msgr2=0x7f6db80c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:04.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db5d9b700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db80c5dd0 0x7f6db80c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db5d9b700 1 -- 172.21.15.55:0/3294824527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db815cf40 con 0x7f6db81509d0 2023-10-06T03:46:04.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db659c700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 0x7f6db8159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:46:04.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.347+0000 7f6db6d9d700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db80c5dd0 0x7f6db80c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:46:04.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.348+0000 7f6db5d9b700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db81509d0 0x7f6db815d3d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6dac001260 tx=0x7f6dac009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:04.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.348+0000 7f6da67fc700 1 -- 172.21.15.55:0/3294824527 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00dbb0 con 0x7f6db81509d0 2023-10-06T03:46:04.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.348+0000 7f6da67fc700 1 -- 172.21.15.55:0/3294824527 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac00dd10 con 0x7f6db81509d0 2023-10-06T03:46:04.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.348+0000 7f6db759e700 1 -- 172.21.15.55:0/3294824527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db815a380 con 0x7f6db81509d0 2023-10-06T03:46:04.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.348+0000 7f6da67fc700 1 -- 172.21.15.55:0/3294824527 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0072e0 con 0x7f6db81509d0 2023-10-06T03:46:04.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.348+0000 7f6db759e700 1 -- 172.21.15.55:0/3294824527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db8157410 con 0x7f6db81509d0 2023-10-06T03:46:04.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.349+0000 7f6db759e700 1 -- 172.21.15.55:0/3294824527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db8063d10 con 0x7f6db81509d0 2023-10-06T03:46:04.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.350+0000 7f6da67fc700 1 -- 172.21.15.55:0/3294824527 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6dac0057f0 con 0x7f6db81509d0 2023-10-06T03:46:04.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.351+0000 7f6da67fc700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6d98066f30 0x7f6d980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.351+0000 7f6da67fc700 1 -- 172.21.15.55:0/3294824527 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5570+0+0 (secure 0 0 0) 0x7f6dac084a90 con 0x7f6db81509d0 2023-10-06T03:46:04.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.352+0000 7f6db659c700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6d98066f30 0x7f6d980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.352+0000 7f6db659c700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6d98066f30 0x7f6d980693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6da80091e0 tx=0x7f6da801a000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:04.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.355+0000 7f6da67fc700 1 -- 172.21.15.55:0/3294824527 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6dac051ce0 con 0x7f6db81509d0 2023-10-06T03:46:04.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.557+0000 7f6db759e700 1 -- 172.21.15.55:0/3294824527 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6db815d0d0 con 0x7f6db81509d0 2023-10-06T03:46:04.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.557+0000 7f6da67fc700 1 -- 172.21.15.55:0/3294824527 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6dac055300 con 0x7f6db81509d0 2023-10-06T03:46:04.559 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:46:04.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.560+0000 7f6db759e700 1 -- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6d98066f30 msgr2=0x7f6d980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:04.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.560+0000 7f6db759e700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6d98066f30 0x7f6d980693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6da80091e0 tx=0x7f6da801a000 comp rx=0 tx=0).stop 2023-10-06T03:46:04.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.560+0000 7f6db759e700 1 -- 172.21.15.55:0/3294824527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db81509d0 msgr2=0x7f6db815d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:04.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.560+0000 7f6db759e700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db81509d0 0x7f6db815d3d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6dac001260 tx=0x7f6dac009040 comp rx=0 tx=0).stop 2023-10-06T03:46:04.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.560+0000 7f6db759e700 1 -- 172.21.15.55:0/3294824527 shutdown_connections 2023-10-06T03:46:04.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.560+0000 7f6db759e700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6d98066f30 0x7f6d980693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.560+0000 7f6db759e700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db80c5dd0 0x7f6db80c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.560+0000 7f6db759e700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db8159a70 0x7f6db8159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.560+0000 7f6db759e700 1 --2- 172.21.15.55:0/3294824527 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db81509d0 0x7f6db815d3d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.560+0000 7f6db759e700 1 -- 172.21.15.55:0/3294824527 >> 172.21.15.55:0/3294824527 conn(0x7f6db80cddc0 msgr2=0x7f6db806e500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:04.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.560+0000 7f6db759e700 1 -- 172.21.15.55:0/3294824527 shutdown_connections 2023-10-06T03:46:04.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.560+0000 7f6db759e700 1 -- 172.21.15.55:0/3294824527 wait complete. 2023-10-06T03:46:04.579 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:46:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.766+0000 7fccd5d9d700 1 Processor -- start 2023-10-06T03:46:04.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.766+0000 7fccd5d9d700 1 -- start start 2023-10-06T03:46:04.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.767+0000 7fccd5d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fccd0142ea0 0x7fccd00b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.767+0000 7fccd5d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fccd00ad870 0x7fccd006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.767+0000 7fccd5d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccd00b3b70 0x7fccd006bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.767+0000 7fccd5d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd00b40f0 con 0x7fccd00ad870 2023-10-06T03:46:04.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.767+0000 7fccd5d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccd0069640 con 0x7fccd0142ea0 2023-10-06T03:46:04.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.767+0000 7fccd5d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd0069920 con 0x7fccd00b3b70 2023-10-06T03:46:04.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.767+0000 7fcccffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fccd00ad870 0x7fccd006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.768+0000 7fcccffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fccd00ad870 0x7fccd006b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50060/0 (socket says 172.21.15.55:50060) 2023-10-06T03:46:04.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.768+0000 7fcccffff700 1 -- 172.21.15.55:0/304862349 learned_addr learned my addr 172.21.15.55:0/304862349 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.768+0000 7fccd4d9b700 1 --2- 172.21.15.55:0/304862349 >> v2:172.21.15.163:3300/0 conn(0x7fccd0142ea0 0x7fccd00b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.768+0000 7fccd559c700 1 --2- 172.21.15.55:0/304862349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccd00b3b70 0x7fccd006bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.768+0000 7fcccffff700 1 -- 172.21.15.55:0/304862349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccd00b3b70 msgr2=0x7fccd006bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.768+0000 7fcccffff700 1 --2- 172.21.15.55:0/304862349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccd00b3b70 0x7fccd006bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.768+0000 7fcccffff700 1 -- 172.21.15.55:0/304862349 >> v2:172.21.15.163:3300/0 conn(0x7fccd0142ea0 msgr2=0x7fccd00b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.768+0000 7fcccffff700 1 --2- 172.21.15.55:0/304862349 >> v2:172.21.15.163:3300/0 conn(0x7fccd0142ea0 0x7fccd00b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.768+0000 7fcccffff700 1 -- 172.21.15.55:0/304862349 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd0069c00 con 0x7fccd00ad870 2023-10-06T03:46:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.768+0000 7fcccffff700 1 --2- 172.21.15.55:0/304862349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccd00ad870 0x7fccd006b9e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fccd0071240 tx=0x7fccbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b24410ce5c5d8d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.768+0000 7fcccf7fe700 1 -- 172.21.15.55:0/304862349 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc00b640 con 0x7fccd00ad870 2023-10-06T03:46:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.769+0000 7fcccf7fe700 1 -- 172.21.15.55:0/304862349 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccbc00b7a0 con 0x7fccd00ad870 2023-10-06T03:46:04.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.769+0000 7fcccf7fe700 1 -- 172.21.15.55:0/304862349 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc00bb60 con 0x7fccd00ad870 2023-10-06T03:46:04.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.769+0000 7fccd5d9d700 1 -- 172.21.15.55:0/304862349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccd00ad870 msgr2=0x7fccd006b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:04.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.769+0000 7fccd5d9d700 1 --2- 172.21.15.55:0/304862349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccd00ad870 0x7fccd006b9e0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fccd0071240 tx=0x7fccbc009250 comp rx=0 tx=0).stop 2023-10-06T03:46:04.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.769+0000 7fccd5d9d700 1 -- 172.21.15.55:0/304862349 shutdown_connections 2023-10-06T03:46:04.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.769+0000 7fccd5d9d700 1 --2- 172.21.15.55:0/304862349 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccd00b3b70 0x7fccd006bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.769+0000 7fccd5d9d700 1 --2- 172.21.15.55:0/304862349 >> v2:172.21.15.163:3300/0 conn(0x7fccd0142ea0 0x7fccd00b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.769+0000 7fccd5d9d700 1 --2- 172.21.15.55:0/304862349 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccd00ad870 0x7fccd006b9e0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.769+0000 7fccd5d9d700 1 -- 172.21.15.55:0/304862349 >> 172.21.15.55:0/304862349 conn(0x7fccd00ce430 msgr2=0x7fccd00ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:04.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.769+0000 7fccd5d9d700 1 -- 172.21.15.55:0/304862349 shutdown_connections 2023-10-06T03:46:04.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.769+0000 7fccd5d9d700 1 -- 172.21.15.55:0/304862349 wait complete. 2023-10-06T03:46:04.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.771+0000 7fccd5d9d700 1 Processor -- start 2023-10-06T03:46:04.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.771+0000 7fccd5d9d700 1 -- start start 2023-10-06T03:46:04.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fccd5d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccd00b3b70 0x7fccd015c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fccd5d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccd0142ea0 0x7fccd015e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fccd5d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccd00acee0 0x7fccd0169870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fccd5d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00b0080 con 0x7fccd0142ea0 2023-10-06T03:46:04.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fccd5d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd015cef0 con 0x7fccd00acee0 2023-10-06T03:46:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fccd5d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd00c9350 con 0x7fccd00b3b70 2023-10-06T03:46:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fcccffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccd0142ea0 0x7fccd015e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fcccffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccd0142ea0 0x7fccd015e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50074/0 (socket says 172.21.15.55:50074) 2023-10-06T03:46:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fcccffff700 1 -- 172.21.15.55:0/4048492966 learned_addr learned my addr 172.21.15.55:0/4048492966 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fccd4d9b700 1 --2- 172.21.15.55:0/4048492966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccd00b3b70 0x7fccd015c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fcccffff700 1 -- 172.21.15.55:0/4048492966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccd00b3b70 msgr2=0x7fccd015c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fcccffff700 1 --2- 172.21.15.55:0/4048492966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccd00b3b70 0x7fccd015c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fccd559c700 1 --2- 172.21.15.55:0/4048492966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccd00acee0 0x7fccd0169870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fcccffff700 1 -- 172.21.15.55:0/4048492966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccd00acee0 msgr2=0x7fccd0169870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fcccffff700 1 --2- 172.21.15.55:0/4048492966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccd00acee0 0x7fccd0169870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.772+0000 7fcccffff700 1 -- 172.21.15.55:0/4048492966 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc0094f0 con 0x7fccd0142ea0 2023-10-06T03:46:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.773+0000 7fcccffff700 1 --2- 172.21.15.55:0/4048492966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccd0142ea0 0x7fccd015e9c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fccbc006fd0 tx=0x7fccbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.773+0000 7fccccff9700 1 -- 172.21.15.55:0/4048492966 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc000d10 con 0x7fccd0142ea0 2023-10-06T03:46:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.773+0000 7fccccff9700 1 -- 172.21.15.55:0/4048492966 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccbc00bb20 con 0x7fccd0142ea0 2023-10-06T03:46:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.773+0000 7fccd5d9d700 1 -- 172.21.15.55:0/4048492966 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd015d100 con 0x7fccd0142ea0 2023-10-06T03:46:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.773+0000 7fccccff9700 1 -- 172.21.15.55:0/4048492966 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc011670 con 0x7fccd0142ea0 2023-10-06T03:46:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.773+0000 7fccd5d9d700 1 -- 172.21.15.55:0/4048492966 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd00c8d00 con 0x7fccd0142ea0 2023-10-06T03:46:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.774+0000 7fccd5d9d700 1 -- 172.21.15.55:0/4048492966 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd00c9500 con 0x7fccd0142ea0 2023-10-06T03:46:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.775+0000 7fccccff9700 1 -- 172.21.15.55:0/4048492966 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fccbc0119e0 con 0x7fccd0142ea0 2023-10-06T03:46:04.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.776+0000 7fccccff9700 1 --2- 172.21.15.55:0/4048492966 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fccb4066f30 0x7fccb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:04.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.776+0000 7fccccff9700 1 -- 172.21.15.55:0/4048492966 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5570+0+0 (secure 0 0 0) 0x7fccbc085df0 con 0x7fccd0142ea0 2023-10-06T03:46:04.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.776+0000 7fccd4d9b700 1 --2- 172.21.15.55:0/4048492966 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fccb4066f30 0x7fccb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:04.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.777+0000 7fccd4d9b700 1 --2- 172.21.15.55:0/4048492966 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fccb4066f30 0x7fccb40693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fccc4006f00 tx=0x7fccc4008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:04.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.780+0000 7fccccff9700 1 -- 172.21.15.55:0/4048492966 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fccd00c9500 con 0x7fccd0142ea0 2023-10-06T03:46:04.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.987+0000 7fccd5d9d700 1 -- 172.21.15.55:0/4048492966 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fccd00c9500 con 0x7fccd0142ea0 2023-10-06T03:46:04.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.988+0000 7fccccff9700 1 -- 172.21.15.55:0/4048492966 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) v1 ==== 84+0+126 (secure 0 0 0) 0x7fccd00c9500 con 0x7fccd0142ea0 2023-10-06T03:46:04.989 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:46:04.990 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:46:04.990 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:04.990 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2023-10-06T03:46:04.990 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": "rbd" 2023-10-06T03:46:04.990 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:46:04.990 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:04.990 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 2, 2023-10-06T03:46:04.990 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2023-10-06T03:46:04.990 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:04.990 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:46:04.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.990+0000 7fccd5d9d700 1 -- 172.21.15.55:0/4048492966 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fccb4066f30 msgr2=0x7fccb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:04.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.990+0000 7fccd5d9d700 1 --2- 172.21.15.55:0/4048492966 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fccb4066f30 0x7fccb40693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fccc4006f00 tx=0x7fccc4008040 comp rx=0 tx=0).stop 2023-10-06T03:46:04.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.990+0000 7fccd5d9d700 1 -- 172.21.15.55:0/4048492966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccd0142ea0 msgr2=0x7fccd015e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:04.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.990+0000 7fccd5d9d700 1 --2- 172.21.15.55:0/4048492966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccd0142ea0 0x7fccd015e9c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fccbc006fd0 tx=0x7fccbc017000 comp rx=0 tx=0).stop 2023-10-06T03:46:04.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.990+0000 7fccd5d9d700 1 -- 172.21.15.55:0/4048492966 shutdown_connections 2023-10-06T03:46:04.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.990+0000 7fccd5d9d700 1 --2- 172.21.15.55:0/4048492966 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fccb4066f30 0x7fccb40693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.990+0000 7fccd5d9d700 1 --2- 172.21.15.55:0/4048492966 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fccd00acee0 0x7fccd0169870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.990+0000 7fccd5d9d700 1 --2- 172.21.15.55:0/4048492966 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fccd00b3b70 0x7fccd015c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.990+0000 7fccd5d9d700 1 --2- 172.21.15.55:0/4048492966 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fccd0142ea0 0x7fccd015e9c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:04.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.990+0000 7fccd5d9d700 1 -- 172.21.15.55:0/4048492966 >> 172.21.15.55:0/4048492966 conn(0x7fccd00ce430 msgr2=0x7fccd0141db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:04.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.990+0000 7fccd5d9d700 1 -- 172.21.15.55:0/4048492966 shutdown_connections 2023-10-06T03:46:04.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:04.990+0000 7fccd5d9d700 1 -- 172.21.15.55:0/4048492966 wait complete. 2023-10-06T03:46:05.009 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:46:05.009 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:46:05.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.200+0000 7f765a99d700 1 Processor -- start 2023-10-06T03:46:05.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.200+0000 7f765a99d700 1 -- start start 2023-10-06T03:46:05.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.201+0000 7f765a99d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f76540b03c0 0x7f76540cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:05.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.201+0000 7f765a99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76540b0790 0x7f76540b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:05.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.201+0000 7f765a99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76541591e0 0x7f7654159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:05.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.201+0000 7f765a99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76540cfa00 con 0x7f76540b0790 2023-10-06T03:46:05.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.201+0000 7f765a99d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76540cb580 con 0x7f76540b03c0 2023-10-06T03:46:05.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.201+0000 7f765a99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76540b2de0 con 0x7f76541591e0 2023-10-06T03:46:05.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.202+0000 7f765919a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76540b0790 0x7f76540b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:05.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.202+0000 7f765919a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f76540b0790 0x7f76540b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50088/0 (socket says 172.21.15.55:50088) 2023-10-06T03:46:05.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.202+0000 7f765919a700 1 -- 172.21.15.55:0/3859948550 learned_addr learned my addr 172.21.15.55:0/3859948550 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:05.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.202+0000 7f765999b700 1 --2- 172.21.15.55:0/3859948550 >> v2:172.21.15.163:3300/0 conn(0x7f76540b03c0 0x7f76540cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:05.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.202+0000 7f765a19c700 1 --2- 172.21.15.55:0/3859948550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76541591e0 0x7f7654159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:05.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.202+0000 7f765919a700 1 -- 172.21.15.55:0/3859948550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76541591e0 msgr2=0x7f7654159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:05.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.202+0000 7f765919a700 1 --2- 172.21.15.55:0/3859948550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76541591e0 0x7f7654159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:05.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.202+0000 7f765919a700 1 -- 172.21.15.55:0/3859948550 >> v2:172.21.15.163:3300/0 conn(0x7f76540b03c0 msgr2=0x7f76540cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:05.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.202+0000 7f765919a700 1 --2- 172.21.15.55:0/3859948550 >> v2:172.21.15.163:3300/0 conn(0x7f76540b03c0 0x7f76540cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:05.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.202+0000 7f765919a700 1 -- 172.21.15.55:0/3859948550 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76540cf160 con 0x7f76540b0790 2023-10-06T03:46:05.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.202+0000 7f765919a700 1 --2- 172.21.15.55:0/3859948550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76540b0790 0x7f76540b28a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f76440099b0 tx=0x7f7644009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46b974435848d8f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.202+0000 7f7658999700 1 -- 172.21.15.55:0/3859948550 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764400b820 con 0x7f76540b0790 2023-10-06T03:46:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.203+0000 7f7658999700 1 -- 172.21.15.55:0/3859948550 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764400b980 con 0x7f76540b0790 2023-10-06T03:46:05.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.203+0000 7f7658999700 1 -- 172.21.15.55:0/3859948550 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764400bd20 con 0x7f76540b0790 2023-10-06T03:46:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.203+0000 7f765a99d700 1 -- 172.21.15.55:0/3859948550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76540b0790 msgr2=0x7f76540b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.203+0000 7f765a99d700 1 --2- 172.21.15.55:0/3859948550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76540b0790 0x7f76540b28a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f76440099b0 tx=0x7f7644009250 comp rx=0 tx=0).stop 2023-10-06T03:46:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.203+0000 7f765a99d700 1 -- 172.21.15.55:0/3859948550 shutdown_connections 2023-10-06T03:46:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.203+0000 7f765a99d700 1 --2- 172.21.15.55:0/3859948550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76541591e0 0x7f7654159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.203+0000 7f765a99d700 1 --2- 172.21.15.55:0/3859948550 >> v2:172.21.15.163:3300/0 conn(0x7f76540b03c0 0x7f76540cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.203+0000 7f765a99d700 1 --2- 172.21.15.55:0/3859948550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76540b0790 0x7f76540b28a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.203+0000 7f765a99d700 1 -- 172.21.15.55:0/3859948550 >> 172.21.15.55:0/3859948550 conn(0x7f7654063230 msgr2=0x7f7654063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.203+0000 7f765a99d700 1 -- 172.21.15.55:0/3859948550 shutdown_connections 2023-10-06T03:46:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.203+0000 7f765a99d700 1 -- 172.21.15.55:0/3859948550 wait complete. 2023-10-06T03:46:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.205+0000 7f765a99d700 1 Processor -- start 2023-10-06T03:46:05.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.205+0000 7f765a99d700 1 -- start start 2023-10-06T03:46:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765a99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76540b03c0 0x7f76540c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765a99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76540b0790 0x7f7654158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765a99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76541591e0 0x7f76540a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765a99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7654169810 con 0x7f76540b0790 2023-10-06T03:46:05.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765a99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76540c84a0 con 0x7f76540b03c0 2023-10-06T03:46:05.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765a99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76540a8110 con 0x7f76541591e0 2023-10-06T03:46:05.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765919a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76540b0790 0x7f7654158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:05.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765919a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76540b0790 0x7f7654158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50104/0 (socket says 172.21.15.55:50104) 2023-10-06T03:46:05.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765919a700 1 -- 172.21.15.55:0/955461633 learned_addr learned my addr 172.21.15.55:0/955461633 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:05.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765999b700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76540b03c0 0x7f76540c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:05.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765a19c700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76541591e0 0x7f76540a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:05.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765919a700 1 -- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76541591e0 msgr2=0x7f76540a83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:05.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765919a700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76541591e0 0x7f76540a83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:05.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765919a700 1 -- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76540b03c0 msgr2=0x7f76540c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:05.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765919a700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76540b03c0 0x7f76540c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:05.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765919a700 1 -- 172.21.15.55:0/955461633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76440094f0 con 0x7f76540b0790 2023-10-06T03:46:05.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.206+0000 7f765999b700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76540b03c0 0x7f76540c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:46:05.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.207+0000 7f765919a700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76540b0790 0x7f7654158980 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f76440094c0 tx=0x7f7644017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:05.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.207+0000 7f7649ffb700 1 -- 172.21.15.55:0/955461633 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644024440 con 0x7f76540b0790 2023-10-06T03:46:05.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.207+0000 7f7649ffb700 1 -- 172.21.15.55:0/955461633 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76440245a0 con 0x7f76540b0790 2023-10-06T03:46:05.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.207+0000 7f7649ffb700 1 -- 172.21.15.55:0/955461633 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644024960 con 0x7f76540b0790 2023-10-06T03:46:05.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.207+0000 7f765a99d700 1 -- 172.21.15.55:0/955461633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76540c8f50 con 0x7f76540b0790 2023-10-06T03:46:05.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.207+0000 7f765a99d700 1 -- 172.21.15.55:0/955461633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76540c8a10 con 0x7f76540b0790 2023-10-06T03:46:05.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.208+0000 7f765a99d700 1 -- 172.21.15.55:0/955461633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76540c8d90 con 0x7f76540b0790 2023-10-06T03:46:05.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.209+0000 7f7649ffb700 1 -- 172.21.15.55:0/955461633 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f764401a420 con 0x7f76540b0790 2023-10-06T03:46:05.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.210+0000 7f7649ffb700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7638066f30 0x7f76380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:05.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.210+0000 7f7649ffb700 1 -- 172.21.15.55:0/955461633 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5570+0+0 (secure 0 0 0) 0x7f7644086310 con 0x7f76540b0790 2023-10-06T03:46:05.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.211+0000 7f765999b700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7638066f30 0x7f76380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:05.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.211+0000 7f765999b700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7638066f30 0x7f76380693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7650006fd0 tx=0x7f7650008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:05.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.215+0000 7f7649ffb700 1 -- 172.21.15.55:0/955461633 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7644053680 con 0x7f76540b0790 2023-10-06T03:46:05.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.416+0000 7f765a99d700 1 -- 172.21.15.55:0/955461633 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f765404c3c0 con 0x7f76540b0790 2023-10-06T03:46:05.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.776+0000 7f7649ffb700 1 -- 172.21.15.55:0/955461633 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v61) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7644056ca0 con 0x7f76540b0790 2023-10-06T03:46:05.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.783+0000 7f765a99d700 1 -- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7638066f30 msgr2=0x7f76380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:05.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.783+0000 7f765a99d700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7638066f30 0x7f76380693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7650006fd0 tx=0x7f7650008040 comp rx=0 tx=0).stop 2023-10-06T03:46:05.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.783+0000 7f765a99d700 1 -- 172.21.15.55:0/955461633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76540b0790 msgr2=0x7f7654158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:05.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.783+0000 7f765a99d700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76540b0790 0x7f7654158980 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f76440094c0 tx=0x7f7644017000 comp rx=0 tx=0).stop 2023-10-06T03:46:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.784+0000 7f765a99d700 1 -- 172.21.15.55:0/955461633 shutdown_connections 2023-10-06T03:46:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.784+0000 7f765a99d700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7638066f30 0x7f76380693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.784+0000 7f765a99d700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76541591e0 0x7f76540a83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.784+0000 7f765a99d700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f76540b03c0 0x7f76540c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.784+0000 7f765a99d700 1 --2- 172.21.15.55:0/955461633 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76540b0790 0x7f7654158980 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.784+0000 7f765a99d700 1 -- 172.21.15.55:0/955461633 >> 172.21.15.55:0/955461633 conn(0x7f7654063230 msgr2=0x7f765415d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.784+0000 7f765a99d700 1 -- 172.21.15.55:0/955461633 shutdown_connections 2023-10-06T03:46:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.784+0000 7f765a99d700 1 -- 172.21.15.55:0/955461633 wait complete. 2023-10-06T03:46:05.786 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2023-10-06T03:46:05.804 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:46:05.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.998+0000 7f184bfff700 1 Processor -- start 2023-10-06T03:46:05.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.998+0000 7f184bfff700 1 -- start start 2023-10-06T03:46:06.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.999+0000 7f184bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f184c06af90 0x7f184c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:06.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.999+0000 7f184bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184c06b360 0x7f184c0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:06.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.999+0000 7f184bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f184c15e020 0x7f184c06d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:06.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.999+0000 7f184bfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184c0c9860 con 0x7f184c06af90 2023-10-06T03:46:06.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.999+0000 7f184bfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f184c0c8cd0 con 0x7f184c15e020 2023-10-06T03:46:06.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.999+0000 7f184bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184c0cd200 con 0x7f184c06b360 2023-10-06T03:46:06.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:05.999+0000 7f184affd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f184c06af90 0x7f184c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:06.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.000+0000 7f184affd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f184c06af90 0x7f184c0c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50118/0 (socket says 172.21.15.55:50118) 2023-10-06T03:46:06.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.000+0000 7f184affd700 1 -- 172.21.15.55:0/3469322438 learned_addr learned my addr 172.21.15.55:0/3469322438 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:06.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.000+0000 7f184affd700 1 -- 172.21.15.55:0/3469322438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184c06b360 msgr2=0x7f184c0ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:06.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.000+0000 7f184b7fe700 1 --2- 172.21.15.55:0/3469322438 >> v2:172.21.15.163:3300/0 conn(0x7f184c15e020 0x7f184c06d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:06.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.000+0000 7f184a7fc700 1 --2- 172.21.15.55:0/3469322438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184c06b360 0x7f184c0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:06.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.000+0000 7f184affd700 1 --2- 172.21.15.55:0/3469322438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184c06b360 0x7f184c0ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:06.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.000+0000 7f184affd700 1 -- 172.21.15.55:0/3469322438 >> v2:172.21.15.163:3300/0 conn(0x7f184c15e020 msgr2=0x7f184c06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:06.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.000+0000 7f184affd700 1 --2- 172.21.15.55:0/3469322438 >> v2:172.21.15.163:3300/0 conn(0x7f184c15e020 0x7f184c06d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:06.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.000+0000 7f184affd700 1 -- 172.21.15.55:0/3469322438 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184c15e4c0 con 0x7f184c06af90 2023-10-06T03:46:06.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.000+0000 7f184b7fe700 1 --2- 172.21.15.55:0/3469322438 >> v2:172.21.15.163:3300/0 conn(0x7f184c15e020 0x7f184c06d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:46:06.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.000+0000 7f184a7fc700 1 --2- 172.21.15.55:0/3469322438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184c06b360 0x7f184c0ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:46:06.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.000+0000 7f184affd700 1 --2- 172.21.15.55:0/3469322438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f184c06af90 0x7f184c0c9cf0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f184c06c840 tx=0x7f183c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4937e54ac4a8857 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:06.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.000+0000 7f1849ffb700 1 -- 172.21.15.55:0/3469322438 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c00b8f0 con 0x7f184c06af90 2023-10-06T03:46:06.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.001+0000 7f1849ffb700 1 -- 172.21.15.55:0/3469322438 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c00ba50 con 0x7f184c06af90 2023-10-06T03:46:06.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.001+0000 7f1849ffb700 1 -- 172.21.15.55:0/3469322438 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c00bdf0 con 0x7f184c06af90 2023-10-06T03:46:06.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.001+0000 7f184bfff700 1 -- 172.21.15.55:0/3469322438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f184c06af90 msgr2=0x7f184c0c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:06.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.001+0000 7f184bfff700 1 --2- 172.21.15.55:0/3469322438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f184c06af90 0x7f184c0c9cf0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f184c06c840 tx=0x7f183c009250 comp rx=0 tx=0).stop 2023-10-06T03:46:06.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.001+0000 7f184bfff700 1 -- 172.21.15.55:0/3469322438 shutdown_connections 2023-10-06T03:46:06.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.001+0000 7f184bfff700 1 --2- 172.21.15.55:0/3469322438 >> v2:172.21.15.163:3300/0 conn(0x7f184c15e020 0x7f184c06d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:06.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.001+0000 7f184bfff700 1 --2- 172.21.15.55:0/3469322438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f184c06af90 0x7f184c0c9cf0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:06.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.001+0000 7f184bfff700 1 --2- 172.21.15.55:0/3469322438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184c06b360 0x7f184c0ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:06.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.001+0000 7f184bfff700 1 -- 172.21.15.55:0/3469322438 >> 172.21.15.55:0/3469322438 conn(0x7f184c06baa0 msgr2=0x7f184c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:06.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.001+0000 7f184bfff700 1 -- 172.21.15.55:0/3469322438 shutdown_connections 2023-10-06T03:46:06.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.001+0000 7f184bfff700 1 -- 172.21.15.55:0/3469322438 wait complete. 2023-10-06T03:46:06.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.003+0000 7f184bfff700 1 Processor -- start 2023-10-06T03:46:06.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.003+0000 7f184bfff700 1 -- start start 2023-10-06T03:46:06.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.004+0000 7f184bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f184c06af90 0x7f184c16d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:06.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.004+0000 7f184bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184c06b360 0x7f184c15c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:06.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.004+0000 7f184bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f184c15e020 0x7f184c1540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:06.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.004+0000 7f184bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c0addc0 con 0x7f184c15e020 2023-10-06T03:46:06.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.004+0000 7f184bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184c0ae0a0 con 0x7f184c06af90 2023-10-06T03:46:06.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.004+0000 7f184bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184c16da60 con 0x7f184c06b360 2023-10-06T03:46:06.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.004+0000 7f184b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f184c15e020 0x7f184c1540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:06.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.004+0000 7f184b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f184c15e020 0x7f184c1540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50130/0 (socket says 172.21.15.55:50130) 2023-10-06T03:46:06.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.004+0000 7f184b7fe700 1 -- 172.21.15.55:0/3278285002 learned_addr learned my addr 172.21.15.55:0/3278285002 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:06.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.004+0000 7f184a7fc700 1 --2- 172.21.15.55:0/3278285002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184c06b360 0x7f184c15c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.004+0000 7f184affd700 1 --2- 172.21.15.55:0/3278285002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f184c06af90 0x7f184c16d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.005+0000 7f184b7fe700 1 -- 172.21.15.55:0/3278285002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184c06b360 msgr2=0x7f184c15c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.005+0000 7f184b7fe700 1 --2- 172.21.15.55:0/3278285002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184c06b360 0x7f184c15c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.005+0000 7f184b7fe700 1 -- 172.21.15.55:0/3278285002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f184c06af90 msgr2=0x7f184c16d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.005+0000 7f184b7fe700 1 --2- 172.21.15.55:0/3278285002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f184c06af90 0x7f184c16d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.005+0000 7f184b7fe700 1 -- 172.21.15.55:0/3278285002 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c0094f0 con 0x7f184c15e020 2023-10-06T03:46:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.005+0000 7f184b7fe700 1 --2- 172.21.15.55:0/3278285002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f184c15e020 0x7f184c1540f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f184400cf20 tx=0x7f1844007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.005+0000 7f1832ffd700 1 -- 172.21.15.55:0/3278285002 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184400ace0 con 0x7f184c15e020 2023-10-06T03:46:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.005+0000 7f1832ffd700 1 -- 172.21.15.55:0/3278285002 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184400ae60 con 0x7f184c15e020 2023-10-06T03:46:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.005+0000 7f184bfff700 1 -- 172.21.15.55:0/3278285002 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f184c16dd40 con 0x7f184c15e020 2023-10-06T03:46:06.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.005+0000 7f1832ffd700 1 -- 172.21.15.55:0/3278285002 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844016670 con 0x7f184c15e020 2023-10-06T03:46:06.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.005+0000 7f184bfff700 1 -- 172.21.15.55:0/3278285002 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184c154630 con 0x7f184c15e020 2023-10-06T03:46:06.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.006+0000 7f184bfff700 1 -- 172.21.15.55:0/3278285002 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f184c0ae230 con 0x7f184c15e020 2023-10-06T03:46:06.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.007+0000 7f1832ffd700 1 -- 172.21.15.55:0/3278285002 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1844016a60 con 0x7f184c15e020 2023-10-06T03:46:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.008+0000 7f1832ffd700 1 --2- 172.21.15.55:0/3278285002 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1838066f30 0x7f18380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.008+0000 7f1832ffd700 1 -- 172.21.15.55:0/3278285002 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f1844012070 con 0x7f184c15e020 2023-10-06T03:46:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.009+0000 7f184affd700 1 --2- 172.21.15.55:0/3278285002 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1838066f30 0x7f18380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:06.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.010+0000 7f184affd700 1 --2- 172.21.15.55:0/3278285002 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1838066f30 0x7f18380693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f184c06c840 tx=0x7f183c017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:06.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.013+0000 7f1832ffd700 1 -- 172.21.15.55:0/3278285002 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f184404e5a0 con 0x7f184c15e020 2023-10-06T03:46:06.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.217+0000 7f184bfff700 1 -- 172.21.15.55:0/3278285002 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f184c04c3c0 con 0x7f184c15e020 2023-10-06T03:46:06.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.786+0000 7f1832ffd700 1 -- 172.21.15.55:0/3278285002 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v62) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1844052000 con 0x7f184c15e020 2023-10-06T03:46:06.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.791+0000 7f184bfff700 1 -- 172.21.15.55:0/3278285002 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1838066f30 msgr2=0x7f18380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:06.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.791+0000 7f184bfff700 1 --2- 172.21.15.55:0/3278285002 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1838066f30 0x7f18380693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f184c06c840 tx=0x7f183c017000 comp rx=0 tx=0).stop 2023-10-06T03:46:06.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.791+0000 7f184bfff700 1 -- 172.21.15.55:0/3278285002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f184c15e020 msgr2=0x7f184c1540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:06.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.791+0000 7f184bfff700 1 --2- 172.21.15.55:0/3278285002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f184c15e020 0x7f184c1540f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f184400cf20 tx=0x7f1844007860 comp rx=0 tx=0).stop 2023-10-06T03:46:06.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.806+0000 7f184bfff700 1 -- 172.21.15.55:0/3278285002 shutdown_connections 2023-10-06T03:46:06.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.806+0000 7f184bfff700 1 --2- 172.21.15.55:0/3278285002 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1838066f30 0x7f18380693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:06.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.806+0000 7f184bfff700 1 --2- 172.21.15.55:0/3278285002 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f184c15e020 0x7f184c1540f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:06.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.806+0000 7f184bfff700 1 --2- 172.21.15.55:0/3278285002 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f184c06af90 0x7f184c16d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:06.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.806+0000 7f184bfff700 1 --2- 172.21.15.55:0/3278285002 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f184c06b360 0x7f184c15c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:06.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.806+0000 7f184bfff700 1 -- 172.21.15.55:0/3278285002 >> 172.21.15.55:0/3278285002 conn(0x7f184c06baa0 msgr2=0x7f184c0c4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:06.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.808+0000 7f184bfff700 1 -- 172.21.15.55:0/3278285002 shutdown_connections 2023-10-06T03:46:06.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:06.808+0000 7f184bfff700 1 -- 172.21.15.55:0/3278285002 wait complete. 2023-10-06T03:46:06.814 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2023-10-06T03:46:06.833 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:46:07.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.023+0000 7f3561f9d700 1 Processor -- start 2023-10-06T03:46:07.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.023+0000 7f3561f9d700 1 -- start start 2023-10-06T03:46:07.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.024+0000 7f3561f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f355c0b4e70 0x7f355c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:07.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.024+0000 7f3561f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f355c0b16d0 0x7f355c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:07.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.024+0000 7f3561f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f355c15d240 0x7f355c15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:07.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.024+0000 7f3561f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355c0b2080 con 0x7f355c0b16d0 2023-10-06T03:46:07.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.024+0000 7f3561f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355c0cfb00 con 0x7f355c0b4e70 2023-10-06T03:46:07.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.024+0000 7f3561f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c0b5c40 con 0x7f355c15d240 2023-10-06T03:46:07.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.025+0000 7f355bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f355c0b16d0 0x7f355c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:07.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.025+0000 7f355bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f355c0b16d0 0x7f355c0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50142/0 (socket says 172.21.15.55:50142) 2023-10-06T03:46:07.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.025+0000 7f355bfff700 1 -- 172.21.15.55:0/1691683574 learned_addr learned my addr 172.21.15.55:0/1691683574 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:07.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.025+0000 7f3560f9b700 1 --2- 172.21.15.55:0/1691683574 >> v2:172.21.15.163:3300/0 conn(0x7f355c0b4e70 0x7f355c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:07.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.025+0000 7f356179c700 1 --2- 172.21.15.55:0/1691683574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f355c15d240 0x7f355c15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:07.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.025+0000 7f355bfff700 1 -- 172.21.15.55:0/1691683574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f355c15d240 msgr2=0x7f355c15d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:07.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.025+0000 7f355bfff700 1 --2- 172.21.15.55:0/1691683574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f355c15d240 0x7f355c15d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:07.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.025+0000 7f355bfff700 1 -- 172.21.15.55:0/1691683574 >> v2:172.21.15.163:3300/0 conn(0x7f355c0b4e70 msgr2=0x7f355c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:07.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.025+0000 7f355bfff700 1 --2- 172.21.15.55:0/1691683574 >> v2:172.21.15.163:3300/0 conn(0x7f355c0b4e70 0x7f355c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:07.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.025+0000 7f355bfff700 1 -- 172.21.15.55:0/1691683574 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c0b5f20 con 0x7f355c0b16d0 2023-10-06T03:46:07.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.025+0000 7f3560f9b700 1 --2- 172.21.15.55:0/1691683574 >> v2:172.21.15.163:3300/0 conn(0x7f355c0b4e70 0x7f355c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:46:07.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.026+0000 7f355bfff700 1 --2- 172.21.15.55:0/1691683574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f355c0b16d0 0x7f355c0b1b40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f355c064910 tx=0x7f3548009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4ea1db299038d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:07.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.026+0000 7f355b7fe700 1 -- 172.21.15.55:0/1691683574 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354800b860 con 0x7f355c0b16d0 2023-10-06T03:46:07.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.026+0000 7f355b7fe700 1 -- 172.21.15.55:0/1691683574 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354800b9c0 con 0x7f355c0b16d0 2023-10-06T03:46:07.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.026+0000 7f355b7fe700 1 -- 172.21.15.55:0/1691683574 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354800bd60 con 0x7f355c0b16d0 2023-10-06T03:46:07.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.026+0000 7f3561f9d700 1 -- 172.21.15.55:0/1691683574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f355c0b16d0 msgr2=0x7f355c0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:07.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.026+0000 7f3561f9d700 1 --2- 172.21.15.55:0/1691683574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f355c0b16d0 0x7f355c0b1b40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f355c064910 tx=0x7f3548009250 comp rx=0 tx=0).stop 2023-10-06T03:46:07.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.026+0000 7f3561f9d700 1 -- 172.21.15.55:0/1691683574 shutdown_connections 2023-10-06T03:46:07.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.026+0000 7f3561f9d700 1 --2- 172.21.15.55:0/1691683574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f355c15d240 0x7f355c15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:07.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.026+0000 7f3561f9d700 1 --2- 172.21.15.55:0/1691683574 >> v2:172.21.15.163:3300/0 conn(0x7f355c0b4e70 0x7f355c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:07.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.026+0000 7f3561f9d700 1 --2- 172.21.15.55:0/1691683574 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f355c0b16d0 0x7f355c0b1b40 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:07.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.027+0000 7f3561f9d700 1 -- 172.21.15.55:0/1691683574 >> 172.21.15.55:0/1691683574 conn(0x7f355c0d10c0 msgr2=0x7f355c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:07.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.027+0000 7f3561f9d700 1 -- 172.21.15.55:0/1691683574 shutdown_connections 2023-10-06T03:46:07.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.027+0000 7f3561f9d700 1 -- 172.21.15.55:0/1691683574 wait complete. 2023-10-06T03:46:07.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.028+0000 7f3561f9d700 1 Processor -- start 2023-10-06T03:46:07.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.028+0000 7f3561f9d700 1 -- start start 2023-10-06T03:46:07.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.029+0000 7f3561f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f355c0b16d0 0x7f355c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:07.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.029+0000 7f3561f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f355c0b4e70 0x7f355c14afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:07.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.029+0000 7f3561f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f355c15d240 0x7f355c0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:07.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.029+0000 7f3561f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c15c480 con 0x7f355c0b16d0 2023-10-06T03:46:07.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.029+0000 7f3561f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c15ce50 con 0x7f355c0b4e70 2023-10-06T03:46:07.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.029+0000 7f3561f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c15b870 con 0x7f355c15d240 2023-10-06T03:46:07.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.029+0000 7f3560f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f355c0b16d0 0x7f355c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:07.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.029+0000 7f3560f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f355c0b16d0 0x7f355c15c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50148/0 (socket says 172.21.15.55:50148) 2023-10-06T03:46:07.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.029+0000 7f3560f9b700 1 -- 172.21.15.55:0/871536543 learned_addr learned my addr 172.21.15.55:0/871536543 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.030+0000 7f355bfff700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f355c0b4e70 0x7f355c14afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.030+0000 7f356179c700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f355c15d240 0x7f355c0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.030+0000 7f3560f9b700 1 -- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f355c15d240 msgr2=0x7f355c0cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.030+0000 7f3560f9b700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f355c15d240 0x7f355c0cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:07.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.030+0000 7f3560f9b700 1 -- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f355c0b4e70 msgr2=0x7f355c14afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:07.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.030+0000 7f3560f9b700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f355c0b4e70 0x7f355c14afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:07.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.030+0000 7f3560f9b700 1 -- 172.21.15.55:0/871536543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35480094f0 con 0x7f355c0b16d0 2023-10-06T03:46:07.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.030+0000 7f355bfff700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f355c0b4e70 0x7f355c14afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:46:07.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.030+0000 7f3560f9b700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f355c0b16d0 0x7f355c15c910 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f355000e8d0 tx=0x7f3550008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:07.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.030+0000 7f3558ff9700 1 -- 172.21.15.55:0/871536543 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355000ee70 con 0x7f355c0b16d0 2023-10-06T03:46:07.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.030+0000 7f3558ff9700 1 -- 172.21.15.55:0/871536543 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355000c5e0 con 0x7f355c0b16d0 2023-10-06T03:46:07.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.030+0000 7f3558ff9700 1 -- 172.21.15.55:0/871536543 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355000c980 con 0x7f355c0b16d0 2023-10-06T03:46:07.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.030+0000 7f3561f9d700 1 -- 172.21.15.55:0/871536543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355c15bb50 con 0x7f355c0b16d0 2023-10-06T03:46:07.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.031+0000 7f3561f9d700 1 -- 172.21.15.55:0/871536543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355c15c110 con 0x7f355c0b16d0 2023-10-06T03:46:07.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.032+0000 7f3561f9d700 1 -- 172.21.15.55:0/871536543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355c15c760 con 0x7f355c0b16d0 2023-10-06T03:46:07.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.033+0000 7f3558ff9700 1 -- 172.21.15.55:0/871536543 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3550013410 con 0x7f355c0b16d0 2023-10-06T03:46:07.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.033+0000 7f3558ff9700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3540066f30 0x7f35400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:07.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.034+0000 7f3558ff9700 1 -- 172.21.15.55:0/871536543 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f3550084630 con 0x7f355c0b16d0 2023-10-06T03:46:07.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.034+0000 7f355bfff700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3540066f30 0x7f35400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:07.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.035+0000 7f355bfff700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3540066f30 0x7f35400693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f355c064910 tx=0x7f3548017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:07.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.039+0000 7f3558ff9700 1 -- 172.21.15.55:0/871536543 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3550051590 con 0x7f355c0b16d0 2023-10-06T03:46:07.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.289+0000 7f3561f9d700 1 -- 172.21.15.55:0/871536543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f355c0b11a0 con 0x7f355c0b16d0 2023-10-06T03:46:07.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.787+0000 7f3558ff9700 1 -- 172.21.15.55:0/871536543 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3550054bb0 con 0x7f355c0b16d0 2023-10-06T03:46:07.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.808+0000 7f3561f9d700 1 -- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3540066f30 msgr2=0x7f35400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:07.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.808+0000 7f3561f9d700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3540066f30 0x7f35400693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f355c064910 tx=0x7f3548017000 comp rx=0 tx=0).stop 2023-10-06T03:46:07.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.808+0000 7f3561f9d700 1 -- 172.21.15.55:0/871536543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f355c0b16d0 msgr2=0x7f355c15c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:07.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.808+0000 7f3561f9d700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f355c0b16d0 0x7f355c15c910 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f355000e8d0 tx=0x7f3550008040 comp rx=0 tx=0).stop 2023-10-06T03:46:07.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.810+0000 7f3561f9d700 1 -- 172.21.15.55:0/871536543 shutdown_connections 2023-10-06T03:46:07.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.810+0000 7f3561f9d700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3540066f30 0x7f35400693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:07.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.810+0000 7f3561f9d700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f355c15d240 0x7f355c0cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:07.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.810+0000 7f3561f9d700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f355c0b16d0 0x7f355c15c910 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:07.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.810+0000 7f3561f9d700 1 --2- 172.21.15.55:0/871536543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f355c0b4e70 0x7f355c14afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:07.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.810+0000 7f3561f9d700 1 -- 172.21.15.55:0/871536543 >> 172.21.15.55:0/871536543 conn(0x7f355c0d10c0 msgr2=0x7f355c1591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:07.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.813+0000 7f3561f9d700 1 -- 172.21.15.55:0/871536543 shutdown_connections 2023-10-06T03:46:07.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:07.813+0000 7f3561f9d700 1 -- 172.21.15.55:0/871536543 wait complete. 2023-10-06T03:46:07.818 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 7 and data pool 8 2023-10-06T03:46:07.841 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:46:08.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.027+0000 7f2491f9d700 1 Processor -- start 2023-10-06T03:46:08.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.027+0000 7f2491f9d700 1 -- start start 2023-10-06T03:46:08.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.029+0000 7f2491f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f248c156310 0x7f248c0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.029+0000 7f2491f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f248c0ae710 0x7f248c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.029+0000 7f2491f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248c0ca860 con 0x7f248c1566e0 2023-10-06T03:46:08.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.029+0000 7f2491f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248c0cab40 con 0x7f248c0ae710 2023-10-06T03:46:08.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.029+0000 7f2491f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248c0cae20 con 0x7f248c156310 2023-10-06T03:46:08.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.029+0000 7f249179c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f248c0ae710 0x7f248c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.029+0000 7f248bfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f248c1566e0 0x7f248c0ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39460/0 (socket says 172.21.15.55:39460) 2023-10-06T03:46:08.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.029+0000 7f248bfff700 1 -- 172.21.15.55:0/2986890654 learned_addr learned my addr 172.21.15.55:0/2986890654 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:08.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.029+0000 7f2490f9b700 1 --2- 172.21.15.55:0/2986890654 >> v2:172.21.15.163:3300/0 conn(0x7f248c156310 0x7f248c0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.029+0000 7f249179c700 1 -- 172.21.15.55:0/2986890654 >> v2:172.21.15.163:3300/0 conn(0x7f248c156310 msgr2=0x7f248c0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:08.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.029+0000 7f249179c700 1 --2- 172.21.15.55:0/2986890654 >> v2:172.21.15.163:3300/0 conn(0x7f248c156310 0x7f248c0c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.029+0000 7f249179c700 1 -- 172.21.15.55:0/2986890654 >> v1:172.21.15.55:6789/0 conn(0x7f248c1566e0 legacy=0x7f248c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.030+0000 7f249179c700 1 -- 172.21.15.55:0/2986890654 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c0cb100 con 0x7f248c0ae710 2023-10-06T03:46:08.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.029+0000 7f248b7fe700 1 -- 172.21.15.55:0/2986890654 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804705678 0 0) 0x7f248c0ca860 con 0x7f248c1566e0 2023-10-06T03:46:08.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.030+0000 7f249179c700 1 --2- 172.21.15.55:0/2986890654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f248c0ae710 0x7f248c0c6830 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f248c0cce20 tx=0x7f2484009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a67f89bb646c25e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:08.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.030+0000 7f248b7fe700 1 -- 172.21.15.55:0/2986890654 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248400e070 con 0x7f248c0ae710 2023-10-06T03:46:08.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.030+0000 7f248b7fe700 1 -- 172.21.15.55:0/2986890654 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248400bc30 con 0x7f248c0ae710 2023-10-06T03:46:08.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.030+0000 7f248b7fe700 1 -- 172.21.15.55:0/2986890654 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484012430 con 0x7f248c0ae710 2023-10-06T03:46:08.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.031+0000 7f2491f9d700 1 -- 172.21.15.55:0/2986890654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f248c0ae710 msgr2=0x7f248c0c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.031+0000 7f2491f9d700 1 --2- 172.21.15.55:0/2986890654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f248c0ae710 0x7f248c0c6830 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f248c0cce20 tx=0x7f2484009780 comp rx=0 tx=0).stop 2023-10-06T03:46:08.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.031+0000 7f2491f9d700 1 -- 172.21.15.55:0/2986890654 shutdown_connections 2023-10-06T03:46:08.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.031+0000 7f2491f9d700 1 --2- 172.21.15.55:0/2986890654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f248c0ae710 0x7f248c0c6830 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.031+0000 7f2491f9d700 1 --2- 172.21.15.55:0/2986890654 >> v2:172.21.15.163:3300/0 conn(0x7f248c156310 0x7f248c0c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.031+0000 7f2491f9d700 1 -- 172.21.15.55:0/2986890654 >> 172.21.15.55:0/2986890654 conn(0x7f248c0cc1f0 msgr2=0x7f248c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:08.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.031+0000 7f2491f9d700 1 -- 172.21.15.55:0/2986890654 shutdown_connections 2023-10-06T03:46:08.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.031+0000 7f2491f9d700 1 -- 172.21.15.55:0/2986890654 wait complete. 2023-10-06T03:46:08.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.032+0000 7f2491f9d700 1 Processor -- start 2023-10-06T03:46:08.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.032+0000 7f2491f9d700 1 -- start start 2023-10-06T03:46:08.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.033+0000 7f2491f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f248c0ae710 0x7f248c14c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.033+0000 7f2491f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f248c156310 0x7f248c15de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.033+0000 7f2491f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f248c1566e0 0x7f248c15e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.033+0000 7f2491f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c15bf20 con 0x7f248c0ae710 2023-10-06T03:46:08.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.033+0000 7f2491f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c151b30 con 0x7f248c156310 2023-10-06T03:46:08.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.033+0000 7f2491f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c15b310 con 0x7f248c1566e0 2023-10-06T03:46:08.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.033+0000 7f2490f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f248c0ae710 0x7f248c14c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f2490f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f248c0ae710 0x7f248c14c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50162/0 (socket says 172.21.15.55:50162) 2023-10-06T03:46:08.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f2490f9b700 1 -- 172.21.15.55:0/1525116154 learned_addr learned my addr 172.21.15.55:0/1525116154 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:08.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f248bfff700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f248c156310 0x7f248c15de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f249179c700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f248c1566e0 0x7f248c15e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f2490f9b700 1 -- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f248c1566e0 msgr2=0x7f248c15e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f2490f9b700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f248c1566e0 0x7f248c15e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f2490f9b700 1 -- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f248c156310 msgr2=0x7f248c15de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f2490f9b700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f248c156310 0x7f248c15de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f2490f9b700 1 -- 172.21.15.55:0/1525116154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2484009a20 con 0x7f248c0ae710 2023-10-06T03:46:08.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f248bfff700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f248c156310 0x7f248c15de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:46:08.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f2490f9b700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f248c0ae710 0x7f248c14c200 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f248000d890 tx=0x7f2480008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:08.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f2488ff9700 1 -- 172.21.15.55:0/1525116154 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248000b600 con 0x7f248c0ae710 2023-10-06T03:46:08.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f2488ff9700 1 -- 172.21.15.55:0/1525116154 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248000b760 con 0x7f248c0ae710 2023-10-06T03:46:08.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f2491f9d700 1 -- 172.21.15.55:0/1525116154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248c15b5f0 con 0x7f248c0ae710 2023-10-06T03:46:08.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.034+0000 7f2488ff9700 1 -- 172.21.15.55:0/1525116154 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248000bae0 con 0x7f248c0ae710 2023-10-06T03:46:08.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.035+0000 7f2491f9d700 1 -- 172.21.15.55:0/1525116154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248c15bbb0 con 0x7f248c0ae710 2023-10-06T03:46:08.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.035+0000 7f2491f9d700 1 -- 172.21.15.55:0/1525116154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248c14d9e0 con 0x7f248c0ae710 2023-10-06T03:46:08.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.037+0000 7f2488ff9700 1 -- 172.21.15.55:0/1525116154 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f248001e460 con 0x7f248c0ae710 2023-10-06T03:46:08.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.037+0000 7f2488ff9700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2470066f30 0x7f24700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.038+0000 7f2488ff9700 1 -- 172.21.15.55:0/1525116154 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f2480084eb0 con 0x7f248c0ae710 2023-10-06T03:46:08.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.038+0000 7f248bfff700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2470066f30 0x7f24700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.039+0000 7f248bfff700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2470066f30 0x7f24700693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f248c150fa0 tx=0x7f2478009040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:08.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.042+0000 7f2488ff9700 1 -- 172.21.15.55:0/1525116154 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2480051e50 con 0x7f248c0ae710 2023-10-06T03:46:08.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.247+0000 7f2491f9d700 1 -- 172.21.15.55:0/1525116154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f248c155920 con 0x7f248c0ae710 2023-10-06T03:46:08.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.248+0000 7f2488ff9700 1 -- 172.21.15.55:0/1525116154 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17478 (secure 0 0 0) 0x7f2480055470 con 0x7f248c0ae710 2023-10-06T03:46:08.250 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:46:08.251 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":63,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:46:07.291685+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:46:05.457264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T03:46:06.258505+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:46:08.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.250+0000 7f2491f9d700 1 -- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2470066f30 msgr2=0x7f24700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.250+0000 7f2491f9d700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2470066f30 0x7f24700693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f248c150fa0 tx=0x7f2478009040 comp rx=0 tx=0).stop 2023-10-06T03:46:08.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.250+0000 7f2491f9d700 1 -- 172.21.15.55:0/1525116154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f248c0ae710 msgr2=0x7f248c14c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.250+0000 7f2491f9d700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f248c0ae710 0x7f248c14c200 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f248000d890 tx=0x7f2480008040 comp rx=0 tx=0).stop 2023-10-06T03:46:08.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.250+0000 7f2491f9d700 1 -- 172.21.15.55:0/1525116154 shutdown_connections 2023-10-06T03:46:08.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.250+0000 7f2491f9d700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2470066f30 0x7f24700693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.250+0000 7f2491f9d700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f248c1566e0 0x7f248c15e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.250+0000 7f2491f9d700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f248c0ae710 0x7f248c14c200 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.250+0000 7f2491f9d700 1 --2- 172.21.15.55:0/1525116154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f248c156310 0x7f248c15de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.250+0000 7f2491f9d700 1 -- 172.21.15.55:0/1525116154 >> 172.21.15.55:0/1525116154 conn(0x7f248c0cc1f0 msgr2=0x7f248c0c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:08.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.251+0000 7f2491f9d700 1 -- 172.21.15.55:0/1525116154 shutdown_connections 2023-10-06T03:46:08.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.251+0000 7f2491f9d700 1 -- 172.21.15.55:0/1525116154 wait complete. 2023-10-06T03:46:08.271 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:46:08.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.455+0000 7f686a79d700 1 Processor -- start 2023-10-06T03:46:08.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.455+0000 7f686a79d700 1 -- start start 2023-10-06T03:46:08.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.456+0000 7f686a79d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6864157270 0x7f6864068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f686a79d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f68640688d0 0x7f68640b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f686a79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68640b0a00 0x7f686406a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f686a79d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6864068d50 con 0x7f6864157270 2023-10-06T03:46:08.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f686a79d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f686415a980 con 0x7f68640688d0 2023-10-06T03:46:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f686a79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686415a050 con 0x7f68640b0a00 2023-10-06T03:46:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f686979b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6864157270 0x7f6864068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f686979b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6864157270 0x7f6864068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50176/0 (socket says 172.21.15.55:50176) 2023-10-06T03:46:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f686979b700 1 -- 172.21.15.55:0/215569096 learned_addr learned my addr 172.21.15.55:0/215569096 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f6869f9c700 1 --2- 172.21.15.55:0/215569096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68640b0a00 0x7f686406a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f6868f9a700 1 --2- 172.21.15.55:0/215569096 >> v2:172.21.15.163:3300/0 conn(0x7f68640688d0 0x7f68640b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f686979b700 1 -- 172.21.15.55:0/215569096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68640b0a00 msgr2=0x7f686406a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f686979b700 1 --2- 172.21.15.55:0/215569096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68640b0a00 0x7f686406a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f686979b700 1 -- 172.21.15.55:0/215569096 >> v2:172.21.15.163:3300/0 conn(0x7f68640688d0 msgr2=0x7f68640b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f686979b700 1 --2- 172.21.15.55:0/215569096 >> v2:172.21.15.163:3300/0 conn(0x7f68640688d0 0x7f68640b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.457+0000 7f686979b700 1 -- 172.21.15.55:0/215569096 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686415a330 con 0x7f6864157270 2023-10-06T03:46:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.458+0000 7f686979b700 1 --2- 172.21.15.55:0/215569096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6864157270 0x7f6864068390 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f686406ed40 tx=0x7f6860009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f500ad462e617b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.458+0000 7f685bfff700 1 -- 172.21.15.55:0/215569096 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686000b860 con 0x7f6864157270 2023-10-06T03:46:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.458+0000 7f685bfff700 1 -- 172.21.15.55:0/215569096 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686000b9c0 con 0x7f6864157270 2023-10-06T03:46:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.458+0000 7f685bfff700 1 -- 172.21.15.55:0/215569096 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686000bd60 con 0x7f6864157270 2023-10-06T03:46:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.458+0000 7f686a79d700 1 -- 172.21.15.55:0/215569096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6864157270 msgr2=0x7f6864068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.458+0000 7f686a79d700 1 --2- 172.21.15.55:0/215569096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6864157270 0x7f6864068390 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f686406ed40 tx=0x7f6860009250 comp rx=0 tx=0).stop 2023-10-06T03:46:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.459+0000 7f686a79d700 1 -- 172.21.15.55:0/215569096 shutdown_connections 2023-10-06T03:46:08.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.459+0000 7f686a79d700 1 --2- 172.21.15.55:0/215569096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68640b0a00 0x7f686406a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.459+0000 7f686a79d700 1 --2- 172.21.15.55:0/215569096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6864157270 0x7f6864068390 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.459+0000 7f686a79d700 1 --2- 172.21.15.55:0/215569096 >> v2:172.21.15.163:3300/0 conn(0x7f68640688d0 0x7f68640b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.459+0000 7f686a79d700 1 -- 172.21.15.55:0/215569096 >> 172.21.15.55:0/215569096 conn(0x7f68640cd310 msgr2=0x7f68640cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:08.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.459+0000 7f686a79d700 1 -- 172.21.15.55:0/215569096 shutdown_connections 2023-10-06T03:46:08.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.459+0000 7f686a79d700 1 -- 172.21.15.55:0/215569096 wait complete. 2023-10-06T03:46:08.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.460+0000 7f686a79d700 1 Processor -- start 2023-10-06T03:46:08.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.460+0000 7f686a79d700 1 -- start start 2023-10-06T03:46:08.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.461+0000 7f686a79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68640688d0 0x7f68640cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.461+0000 7f686a79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68640b0a00 0x7f68640af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.461+0000 7f686a79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6864157270 0x7f68640c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.461+0000 7f686a79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68641466b0 con 0x7f68640b0a00 2023-10-06T03:46:08.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.461+0000 7f686a79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686416dd20 con 0x7f68640688d0 2023-10-06T03:46:08.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.461+0000 7f686a79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686416d110 con 0x7f6864157270 2023-10-06T03:46:08.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.461+0000 7f6868f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68640b0a00 0x7f68640af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.462+0000 7f6868f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68640b0a00 0x7f68640af0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50188/0 (socket says 172.21.15.55:50188) 2023-10-06T03:46:08.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.462+0000 7f6868f9a700 1 -- 172.21.15.55:0/1513417880 learned_addr learned my addr 172.21.15.55:0/1513417880 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:08.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.462+0000 7f686979b700 1 --2- 172.21.15.55:0/1513417880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68640688d0 0x7f68640cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.462+0000 7f6869f9c700 1 --2- 172.21.15.55:0/1513417880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6864157270 0x7f68640c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.462+0000 7f6868f9a700 1 -- 172.21.15.55:0/1513417880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6864157270 msgr2=0x7f68640c20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.462+0000 7f6868f9a700 1 --2- 172.21.15.55:0/1513417880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6864157270 0x7f68640c20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.462+0000 7f6868f9a700 1 -- 172.21.15.55:0/1513417880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68640688d0 msgr2=0x7f68640cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.462+0000 7f6868f9a700 1 --2- 172.21.15.55:0/1513417880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68640688d0 0x7f68640cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.462+0000 7f6868f9a700 1 -- 172.21.15.55:0/1513417880 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68600094f0 con 0x7f68640b0a00 2023-10-06T03:46:08.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.462+0000 7f6868f9a700 1 --2- 172.21.15.55:0/1513417880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68640b0a00 0x7f68640af0b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f685400d890 tx=0x7f6854008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:08.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.462+0000 7f68597fa700 1 -- 172.21.15.55:0/1513417880 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68540125f0 con 0x7f68640b0a00 2023-10-06T03:46:08.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.463+0000 7f68597fa700 1 -- 172.21.15.55:0/1513417880 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6854012750 con 0x7f68640b0a00 2023-10-06T03:46:08.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.463+0000 7f686a79d700 1 -- 172.21.15.55:0/1513417880 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686416d3f0 con 0x7f68640b0a00 2023-10-06T03:46:08.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.463+0000 7f68597fa700 1 -- 172.21.15.55:0/1513417880 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854012af0 con 0x7f68640b0a00 2023-10-06T03:46:08.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.463+0000 7f686a79d700 1 -- 172.21.15.55:0/1513417880 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686416d9b0 con 0x7f68640b0a00 2023-10-06T03:46:08.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.464+0000 7f686a79d700 1 -- 172.21.15.55:0/1513417880 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686404c3c0 con 0x7f68640b0a00 2023-10-06T03:46:08.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.465+0000 7f68597fa700 1 -- 172.21.15.55:0/1513417880 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f685401e460 con 0x7f68640b0a00 2023-10-06T03:46:08.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.465+0000 7f68597fa700 1 --2- 172.21.15.55:0/1513417880 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6848066f30 0x7f68480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.465+0000 7f68597fa700 1 -- 172.21.15.55:0/1513417880 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f6854084e10 con 0x7f68640b0a00 2023-10-06T03:46:08.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.466+0000 7f686979b700 1 --2- 172.21.15.55:0/1513417880 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6848066f30 0x7f68480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.467+0000 7f686979b700 1 --2- 172.21.15.55:0/1513417880 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6848066f30 0x7f68480693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f686406ed40 tx=0x7f6860017000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:08.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.470+0000 7f68597fa700 1 -- 172.21.15.55:0/1513417880 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6854088030 con 0x7f68640b0a00 2023-10-06T03:46:08.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.673+0000 7f686a79d700 1 -- 172.21.15.55:0/1513417880 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f686415a610 con 0x7f68640b0a00 2023-10-06T03:46:08.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.674+0000 7f68597fa700 1 -- 172.21.15.55:0/1513417880 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+17478 (secure 0 0 0) 0x7f68540557a0 con 0x7f68640b0a00 2023-10-06T03:46:08.677 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:46:08.677 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":63,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:46:07.291685+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:46:05.457264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T03:46:06.258505+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:46:08.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.676+0000 7f686a79d700 1 -- 172.21.15.55:0/1513417880 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6848066f30 msgr2=0x7f68480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.676+0000 7f686a79d700 1 --2- 172.21.15.55:0/1513417880 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6848066f30 0x7f68480693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f686406ed40 tx=0x7f6860017000 comp rx=0 tx=0).stop 2023-10-06T03:46:08.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.676+0000 7f686a79d700 1 -- 172.21.15.55:0/1513417880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68640b0a00 msgr2=0x7f68640af0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.676+0000 7f686a79d700 1 --2- 172.21.15.55:0/1513417880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68640b0a00 0x7f68640af0b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f685400d890 tx=0x7f6854008040 comp rx=0 tx=0).stop 2023-10-06T03:46:08.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.676+0000 7f686a79d700 1 -- 172.21.15.55:0/1513417880 shutdown_connections 2023-10-06T03:46:08.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.676+0000 7f686a79d700 1 --2- 172.21.15.55:0/1513417880 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6848066f30 0x7f68480693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.676+0000 7f686a79d700 1 --2- 172.21.15.55:0/1513417880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6864157270 0x7f68640c20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.676+0000 7f686a79d700 1 --2- 172.21.15.55:0/1513417880 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68640688d0 0x7f68640cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.676+0000 7f686a79d700 1 --2- 172.21.15.55:0/1513417880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68640b0a00 0x7f68640af0b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.676+0000 7f686a79d700 1 -- 172.21.15.55:0/1513417880 >> 172.21.15.55:0/1513417880 conn(0x7f68640cd310 msgr2=0x7f6864158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:08.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.677+0000 7f686a79d700 1 -- 172.21.15.55:0/1513417880 shutdown_connections 2023-10-06T03:46:08.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.677+0000 7f686a79d700 1 -- 172.21.15.55:0/1513417880 wait complete. 2023-10-06T03:46:08.697 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:46:08.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.889+0000 7fd9ce99d700 1 Processor -- start 2023-10-06T03:46:08.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.889+0000 7fd9ce99d700 1 -- start start 2023-10-06T03:46:08.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.890+0000 7fd9ce99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9c80ae7e0 0x7fd9c80c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.890+0000 7fd9ce99d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c806ebd0 con 0x7fd9c80cb1a0 2023-10-06T03:46:08.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.890+0000 7fd9ce99d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c815aa70 con 0x7fd9c806f060 2023-10-06T03:46:08.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.890+0000 7fd9ce99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c815b2e0 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.890+0000 7fd9cd99b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd9c80cb1a0 0x7fd9c815a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39472/0 (socket says 172.21.15.55:39472) 2023-10-06T03:46:08.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.890+0000 7fd9cd99b700 1 -- 172.21.15.55:0/1190125814 learned_addr learned my addr 172.21.15.55:0/1190125814 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:08.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.890+0000 7fd9ce19c700 1 --2- 172.21.15.55:0/1190125814 >> v2:172.21.15.55:3300/0 conn(0x7fd9c80ae7e0 0x7fd9c80c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.891+0000 7fd9cc999700 1 -- 172.21.15.55:0/1190125814 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563604331 0 0) 0x7fd9c806ebd0 con 0x7fd9c80cb1a0 2023-10-06T03:46:08.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.891+0000 7fd9cc999700 1 -- 172.21.15.55:0/1190125814 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9b4003570 con 0x7fd9c80cb1a0 2023-10-06T03:46:08.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.891+0000 7fd9ce19c700 1 -- 172.21.15.55:0/1190125814 >> v1:172.21.15.163:6789/0 conn(0x7fd9c806f060 legacy=0x7fd9c80ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:08.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.891+0000 7fd9ce19c700 1 -- 172.21.15.55:0/1190125814 >> v1:172.21.15.55:6789/0 conn(0x7fd9c80cb1a0 legacy=0x7fd9c815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.891+0000 7fd9ce19c700 1 -- 172.21.15.55:0/1190125814 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c815ad20 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.891+0000 7fd9ce19c700 1 --2- 172.21.15.55:0/1190125814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9c80ae7e0 0x7fd9c80c9db0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009e60 tx=0x7fd9c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c956fe60375406c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:08.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.891+0000 7fd9cc999700 1 -- 172.21.15.55:0/1190125814 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c000e070 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.891+0000 7fd9cc999700 1 -- 172.21.15.55:0/1190125814 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c000bbf0 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.891+0000 7fd9cc999700 1 -- 172.21.15.55:0/1190125814 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0012430 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.892+0000 7fd9ce99d700 1 -- 172.21.15.55:0/1190125814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9c80ae7e0 msgr2=0x7fd9c80c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.892+0000 7fd9ce99d700 1 --2- 172.21.15.55:0/1190125814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9c80ae7e0 0x7fd9c80c9db0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009e60 tx=0x7fd9c0009780 comp rx=0 tx=0).stop 2023-10-06T03:46:08.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.892+0000 7fd9ce99d700 1 -- 172.21.15.55:0/1190125814 shutdown_connections 2023-10-06T03:46:08.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.892+0000 7fd9ce99d700 1 --2- 172.21.15.55:0/1190125814 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9c80ae7e0 0x7fd9c80c9db0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.892+0000 7fd9ce99d700 1 -- 172.21.15.55:0/1190125814 >> 172.21.15.55:0/1190125814 conn(0x7fd9c80cc1e0 msgr2=0x7fd9c80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:08.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.892+0000 7fd9ce99d700 1 -- 172.21.15.55:0/1190125814 shutdown_connections 2023-10-06T03:46:08.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.892+0000 7fd9ce99d700 1 -- 172.21.15.55:0/1190125814 wait complete. 2023-10-06T03:46:08.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.893+0000 7fd9ce99d700 1 Processor -- start 2023-10-06T03:46:08.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.894+0000 7fd9ce99d700 1 -- start start 2023-10-06T03:46:08.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.894+0000 7fd9ce99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9c806f060 0x7fd9c80c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.894+0000 7fd9ce99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9c80ae7e0 0x7fd9c814e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.894+0000 7fd9ce99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9c80cb1a0 0x7fd9c814ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.894+0000 7fd9ce99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c80c37d0 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.894+0000 7fd9ce99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c80c3a70 con 0x7fd9c80cb1a0 2023-10-06T03:46:08.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.894+0000 7fd9ce99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c80c3d50 con 0x7fd9c806f060 2023-10-06T03:46:08.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.895+0000 7fd9cd19a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9c80ae7e0 0x7fd9c814e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.895+0000 7fd9cd19a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9c80ae7e0 0x7fd9c814e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50194/0 (socket says 172.21.15.55:50194) 2023-10-06T03:46:08.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.895+0000 7fd9cd19a700 1 -- 172.21.15.55:0/2973171492 learned_addr learned my addr 172.21.15.55:0/2973171492 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:08.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.895+0000 7fd9ce19c700 1 --2- 172.21.15.55:0/2973171492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9c80cb1a0 0x7fd9c814ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.895+0000 7fd9cd99b700 1 --2- 172.21.15.55:0/2973171492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9c806f060 0x7fd9c80c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.895+0000 7fd9cd19a700 1 -- 172.21.15.55:0/2973171492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9c806f060 msgr2=0x7fd9c80c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.895+0000 7fd9cd19a700 1 --2- 172.21.15.55:0/2973171492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9c806f060 0x7fd9c80c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.895+0000 7fd9cd19a700 1 -- 172.21.15.55:0/2973171492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9c80cb1a0 msgr2=0x7fd9c814ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:08.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.895+0000 7fd9cd19a700 1 --2- 172.21.15.55:0/2973171492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9c80cb1a0 0x7fd9c814ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:08.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.895+0000 7fd9cd19a700 1 -- 172.21.15.55:0/2973171492 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c0009a20 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.895+0000 7fd9cd19a700 1 --2- 172.21.15.55:0/2973171492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9c80ae7e0 0x7fd9c814e090 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800c590 tx=0x7fd9b800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:08.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.896+0000 7fd9bdffb700 1 -- 172.21.15.55:0/2973171492 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8011590 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.896+0000 7fd9bdffb700 1 -- 172.21.15.55:0/2973171492 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b8011710 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.896+0000 7fd9bdffb700 1 -- 172.21.15.55:0/2973171492 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8011ad0 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.896+0000 7fd9ce99d700 1 -- 172.21.15.55:0/2973171492 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c80c4030 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.896+0000 7fd9ce99d700 1 -- 172.21.15.55:0/2973171492 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c80c45f0 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.897+0000 7fd9ce99d700 1 -- 172.21.15.55:0/2973171492 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c806c450 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.898+0000 7fd9bdffb700 1 -- 172.21.15.55:0/2973171492 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9b801b3b0 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.898+0000 7fd9bdffb700 1 --2- 172.21.15.55:0/2973171492 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9ac066f30 0x7fd9ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:08.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.899+0000 7fd9bdffb700 1 -- 172.21.15.55:0/2973171492 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7fd9b8083c70 con 0x7fd9c80ae7e0 2023-10-06T03:46:08.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.899+0000 7fd9cd99b700 1 --2- 172.21.15.55:0/2973171492 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9ac066f30 0x7fd9ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:08.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.900+0000 7fd9cd99b700 1 --2- 172.21.15.55:0/2973171492 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9ac066f30 0x7fd9ac0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4007fd0 tx=0x7fd9c4009040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:08.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:08.903+0000 7fd9bdffb700 1 -- 172.21.15.55:0/2973171492 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9b8050c10 con 0x7fd9c80ae7e0 2023-10-06T03:46:09.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:09.154+0000 7fd9ce99d700 1 -- 172.21.15.55:0/2973171492 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd9c815b000 con 0x7fd9c80ae7e0 2023-10-06T03:46:09.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:09.814+0000 7fd9bdffb700 1 -- 172.21.15.55:0/2973171492 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd9b8017070 con 0x7fd9c80ae7e0 2023-10-06T03:46:09.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:09.817+0000 7fd9ce99d700 1 -- 172.21.15.55:0/2973171492 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9ac066f30 msgr2=0x7fd9ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:09.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:09.817+0000 7fd9ce99d700 1 --2- 172.21.15.55:0/2973171492 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9ac066f30 0x7fd9ac0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4007fd0 tx=0x7fd9c4009040 comp rx=0 tx=0).stop 2023-10-06T03:46:09.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:09.817+0000 7fd9ce99d700 1 -- 172.21.15.55:0/2973171492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9c80ae7e0 msgr2=0x7fd9c814e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:09.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:09.817+0000 7fd9ce99d700 1 --2- 172.21.15.55:0/2973171492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9c80ae7e0 0x7fd9c814e090 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800c590 tx=0x7fd9b800c8a0 comp rx=0 tx=0).stop 2023-10-06T03:46:09.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:09.818+0000 7fd9ce99d700 1 -- 172.21.15.55:0/2973171492 shutdown_connections 2023-10-06T03:46:09.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:09.818+0000 7fd9ce99d700 1 --2- 172.21.15.55:0/2973171492 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9ac066f30 0x7fd9ac0693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:09.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:09.818+0000 7fd9ce99d700 1 --2- 172.21.15.55:0/2973171492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9c80cb1a0 0x7fd9c814ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:09.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:09.818+0000 7fd9ce99d700 1 --2- 172.21.15.55:0/2973171492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9c806f060 0x7fd9c80c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:09.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:09.818+0000 7fd9ce99d700 1 --2- 172.21.15.55:0/2973171492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9c80ae7e0 0x7fd9c814e090 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:09.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:09.818+0000 7fd9ce99d700 1 -- 172.21.15.55:0/2973171492 >> 172.21.15.55:0/2973171492 conn(0x7fd9c80cc1e0 msgr2=0x7fd9c80c7090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:09.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:09.818+0000 7fd9ce99d700 1 -- 172.21.15.55:0/2973171492 shutdown_connections 2023-10-06T03:46:09.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:09.818+0000 7fd9ce99d700 1 -- 172.21.15.55:0/2973171492 wait complete. 2023-10-06T03:46:09.841 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:46:10.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.033+0000 7f5203fff700 1 Processor -- start 2023-10-06T03:46:10.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.034+0000 7f5203fff700 1 -- start start 2023-10-06T03:46:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.035+0000 7f5203fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f52040cb1a0 0x7f520415a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.035+0000 7f5203fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f52040ae0d0 0x7f52040ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.035+0000 7f5203fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f520406ebd0 con 0x7f520415a8a0 2023-10-06T03:46:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.035+0000 7f5203fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f520415b2e0 con 0x7f52040cb1a0 2023-10-06T03:46:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.035+0000 7f5203fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52040aea10 con 0x7f52040ae0d0 2023-10-06T03:46:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.035+0000 7f5202ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f52040cb1a0 0x7f520415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.035+0000 7f5202ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f52040cb1a0 0x7f520415a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50202/0 (socket says 172.21.15.55:50202) 2023-10-06T03:46:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.035+0000 7f5202ffd700 1 -- 172.21.15.55:0/3641097913 learned_addr learned my addr 172.21.15.55:0/3641097913 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.035+0000 7f52027fc700 1 --1- 172.21.15.55:0/3641097913 >> v1:172.21.15.55:6789/0 conn(0x7f520415a8a0 0x7f520406f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39486/0 (socket says 172.21.15.55:39486) 2023-10-06T03:46:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.036+0000 7f52037fe700 1 --2- 172.21.15.55:0/3641097913 >> v2:172.21.15.163:3300/0 conn(0x7f52040ae0d0 0x7f52040ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:10.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.036+0000 7f5202ffd700 1 -- 172.21.15.55:0/3641097913 >> v2:172.21.15.163:3300/0 conn(0x7f52040ae0d0 msgr2=0x7f52040ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:10.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.036+0000 7f5202ffd700 1 --2- 172.21.15.55:0/3641097913 >> v2:172.21.15.163:3300/0 conn(0x7f52040ae0d0 0x7f52040ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:10.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.036+0000 7f5202ffd700 1 -- 172.21.15.55:0/3641097913 >> v1:172.21.15.55:6789/0 conn(0x7f520415a8a0 legacy=0x7f520406f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:10.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.036+0000 7f5202ffd700 1 -- 172.21.15.55:0/3641097913 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520415ad20 con 0x7f52040cb1a0 2023-10-06T03:46:10.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.036+0000 7f52037fe700 1 --2- 172.21.15.55:0/3641097913 >> v2:172.21.15.163:3300/0 conn(0x7f52040ae0d0 0x7f52040ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:46:10.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.036+0000 7f5202ffd700 1 --2- 172.21.15.55:0/3641097913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52040cb1a0 0x7f520415a360 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f52040c67b0 tx=0x7f51f400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e073148f44ccd6b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:10.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.036+0000 7f5201ffb700 1 -- 172.21.15.55:0/3641097913 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f400dce0 con 0x7f52040cb1a0 2023-10-06T03:46:10.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.036+0000 7f5201ffb700 1 -- 172.21.15.55:0/3641097913 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f400de40 con 0x7f52040cb1a0 2023-10-06T03:46:10.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.036+0000 7f5201ffb700 1 -- 172.21.15.55:0/3641097913 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40196d0 con 0x7f52040cb1a0 2023-10-06T03:46:10.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.037+0000 7f5203fff700 1 -- 172.21.15.55:0/3641097913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52040cb1a0 msgr2=0x7f520415a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:10.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.037+0000 7f5203fff700 1 --2- 172.21.15.55:0/3641097913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52040cb1a0 0x7f520415a360 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f52040c67b0 tx=0x7f51f400b4f0 comp rx=0 tx=0).stop 2023-10-06T03:46:10.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.037+0000 7f5203fff700 1 -- 172.21.15.55:0/3641097913 shutdown_connections 2023-10-06T03:46:10.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.037+0000 7f5203fff700 1 --2- 172.21.15.55:0/3641097913 >> v2:172.21.15.163:3300/0 conn(0x7f52040ae0d0 0x7f52040ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:10.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.037+0000 7f5203fff700 1 --2- 172.21.15.55:0/3641097913 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52040cb1a0 0x7f520415a360 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:10.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.037+0000 7f5203fff700 1 -- 172.21.15.55:0/3641097913 >> 172.21.15.55:0/3641097913 conn(0x7f52040cc1e0 msgr2=0x7f52040cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:10.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.037+0000 7f5203fff700 1 -- 172.21.15.55:0/3641097913 shutdown_connections 2023-10-06T03:46:10.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.037+0000 7f5203fff700 1 -- 172.21.15.55:0/3641097913 wait complete. 2023-10-06T03:46:10.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.039+0000 7f5203fff700 1 Processor -- start 2023-10-06T03:46:10.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.039+0000 7f5203fff700 1 -- start start 2023-10-06T03:46:10.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f5203fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52040ae0d0 0x7f5204155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:10.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f5203fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52040cb1a0 0x7f5204169fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:10.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f5203fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52040aef80 0x7f520415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:10.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f5203fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520416b040 con 0x7f52040aef80 2023-10-06T03:46:10.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f5203fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52041557a0 con 0x7f52040cb1a0 2023-10-06T03:46:10.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f5203fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520416aa80 con 0x7f52040ae0d0 2023-10-06T03:46:10.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f52037fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52040aef80 0x7f520415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:10.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f52037fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52040aef80 0x7f520415e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50214/0 (socket says 172.21.15.55:50214) 2023-10-06T03:46:10.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f52037fe700 1 -- 172.21.15.55:0/3021060267 learned_addr learned my addr 172.21.15.55:0/3021060267 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:10.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f5202ffd700 1 --2- 172.21.15.55:0/3021060267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52040ae0d0 0x7f5204155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:10.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f52027fc700 1 --2- 172.21.15.55:0/3021060267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52040cb1a0 0x7f5204169fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:10.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f52037fe700 1 -- 172.21.15.55:0/3021060267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52040ae0d0 msgr2=0x7f5204155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:10.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f52037fe700 1 --2- 172.21.15.55:0/3021060267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52040ae0d0 0x7f5204155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:10.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f52037fe700 1 -- 172.21.15.55:0/3021060267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52040cb1a0 msgr2=0x7f5204169fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:10.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f52037fe700 1 --2- 172.21.15.55:0/3021060267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52040cb1a0 0x7f5204169fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:10.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.040+0000 7f52037fe700 1 -- 172.21.15.55:0/3021060267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f400b790 con 0x7f52040aef80 2023-10-06T03:46:10.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.041+0000 7f52037fe700 1 --2- 172.21.15.55:0/3021060267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52040aef80 0x7f520415e8c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00a380 tx=0x7f51fc007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:10.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.041+0000 7f51eaffd700 1 -- 172.21.15.55:0/3021060267 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc015440 con 0x7f52040aef80 2023-10-06T03:46:10.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.041+0000 7f51eaffd700 1 -- 172.21.15.55:0/3021060267 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51fc0155a0 con 0x7f52040aef80 2023-10-06T03:46:10.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.041+0000 7f51eaffd700 1 -- 172.21.15.55:0/3021060267 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc015960 con 0x7f52040aef80 2023-10-06T03:46:10.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.041+0000 7f5203fff700 1 -- 172.21.15.55:0/3021060267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520416ad60 con 0x7f52040aef80 2023-10-06T03:46:10.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.041+0000 7f5203fff700 1 -- 172.21.15.55:0/3021060267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52041571e0 con 0x7f52040aef80 2023-10-06T03:46:10.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.042+0000 7f5203fff700 1 -- 172.21.15.55:0/3021060267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5204156d60 con 0x7f52040aef80 2023-10-06T03:46:10.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.043+0000 7f51eaffd700 1 -- 172.21.15.55:0/3021060267 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f51fc00d440 con 0x7f52040aef80 2023-10-06T03:46:10.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.044+0000 7f51eaffd700 1 --2- 172.21.15.55:0/3021060267 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f51f0066f30 0x7f51f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:10.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.044+0000 7f51eaffd700 1 -- 172.21.15.55:0/3021060267 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f51fc0852c0 con 0x7f52040aef80 2023-10-06T03:46:10.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.045+0000 7f5202ffd700 1 --2- 172.21.15.55:0/3021060267 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f51f0066f30 0x7f51f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:10.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.045+0000 7f5202ffd700 1 --2- 172.21.15.55:0/3021060267 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f51f0066f30 0x7f51f00693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f52040c67b0 tx=0x7f51f401b000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:10.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.048+0000 7f51eaffd700 1 -- 172.21.15.55:0/3021060267 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5204156d60 con 0x7f52040aef80 2023-10-06T03:46:10.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.297+0000 7f5203fff700 1 -- 172.21.15.55:0/3021060267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5204156d60 con 0x7f52040aef80 2023-10-06T03:46:10.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.827+0000 7f51eaffd700 1 -- 172.21.15.55:0/3021060267 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5204156d60 con 0x7f52040aef80 2023-10-06T03:46:10.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.829+0000 7f5203fff700 1 -- 172.21.15.55:0/3021060267 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f51f0066f30 msgr2=0x7f51f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:10.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.829+0000 7f5203fff700 1 --2- 172.21.15.55:0/3021060267 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f51f0066f30 0x7f51f00693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f52040c67b0 tx=0x7f51f401b000 comp rx=0 tx=0).stop 2023-10-06T03:46:10.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.829+0000 7f5203fff700 1 -- 172.21.15.55:0/3021060267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52040aef80 msgr2=0x7f520415e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:10.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.829+0000 7f5203fff700 1 --2- 172.21.15.55:0/3021060267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52040aef80 0x7f520415e8c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00a380 tx=0x7f51fc007900 comp rx=0 tx=0).stop 2023-10-06T03:46:10.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.830+0000 7f5203fff700 1 -- 172.21.15.55:0/3021060267 shutdown_connections 2023-10-06T03:46:10.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.830+0000 7f5203fff700 1 --2- 172.21.15.55:0/3021060267 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f51f0066f30 0x7f51f00693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:10.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.830+0000 7f5203fff700 1 --2- 172.21.15.55:0/3021060267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f52040aef80 0x7f520415e8c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:10.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.830+0000 7f5203fff700 1 --2- 172.21.15.55:0/3021060267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f52040ae0d0 0x7f5204155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:10.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.830+0000 7f5203fff700 1 --2- 172.21.15.55:0/3021060267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f52040cb1a0 0x7f5204169fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:10.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.830+0000 7f5203fff700 1 -- 172.21.15.55:0/3021060267 >> 172.21.15.55:0/3021060267 conn(0x7f52040cc1e0 msgr2=0x7f5204155aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:10.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.830+0000 7f5203fff700 1 -- 172.21.15.55:0/3021060267 shutdown_connections 2023-10-06T03:46:10.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:10.830+0000 7f5203fff700 1 -- 172.21.15.55:0/3021060267 wait complete. 2023-10-06T03:46:10.852 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:46:11.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.039+0000 7f0c9359d700 1 Processor -- start 2023-10-06T03:46:11.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.039+0000 7f0c9359d700 1 -- start start 2023-10-06T03:46:11.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.040+0000 7f0c9359d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0c8c159420 0x7f0c8c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.040+0000 7f0c9359d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0c8c1597f0 0x7f0c8c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.040+0000 7f0c9359d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8c0b11d0 con 0x7f0c8c15d230 2023-10-06T03:46:11.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.040+0000 7f0c9359d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c8c0b14b0 con 0x7f0c8c1597f0 2023-10-06T03:46:11.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.040+0000 7f0c9359d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c8c0b1790 con 0x7f0c8c159420 2023-10-06T03:46:11.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.041+0000 7f0c91d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0c8c1597f0 0x7f0c8c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.041+0000 7f0c91d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0c8c1597f0 0x7f0c8c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50220/0 (socket says 172.21.15.55:50220) 2023-10-06T03:46:11.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.041+0000 7f0c91d9a700 1 -- 172.21.15.55:0/3708291281 learned_addr learned my addr 172.21.15.55:0/3708291281 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:11.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.041+0000 7f0c91d9a700 1 -- 172.21.15.55:0/3708291281 >> v2:172.21.15.163:3300/0 conn(0x7f0c8c159420 msgr2=0x7f0c8c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:11.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.041+0000 7f0c91d9a700 1 --2- 172.21.15.55:0/3708291281 >> v2:172.21.15.163:3300/0 conn(0x7f0c8c159420 0x7f0c8c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.041+0000 7f0c91d9a700 1 -- 172.21.15.55:0/3708291281 >> v1:172.21.15.163:6789/0 conn(0x7f0c8c15d230 legacy=0x7f0c8c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:11.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.041+0000 7f0c91d9a700 1 -- 172.21.15.55:0/3708291281 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c0b1a70 con 0x7f0c8c1597f0 2023-10-06T03:46:11.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.041+0000 7f0c91d9a700 1 --2- 172.21.15.55:0/3708291281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8c1597f0 0x7f0c8c0b2270 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0b2f20 tx=0x7f0c7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f033474fac541400 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:11.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.041+0000 7f0c91599700 1 -- 172.21.15.55:0/3708291281 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c00b820 con 0x7f0c8c1597f0 2023-10-06T03:46:11.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.042+0000 7f0c91599700 1 -- 172.21.15.55:0/3708291281 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c00b980 con 0x7f0c8c1597f0 2023-10-06T03:46:11.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.042+0000 7f0c91599700 1 -- 172.21.15.55:0/3708291281 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c00bd20 con 0x7f0c8c1597f0 2023-10-06T03:46:11.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.042+0000 7f0c9359d700 1 -- 172.21.15.55:0/3708291281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8c1597f0 msgr2=0x7f0c8c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:11.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.042+0000 7f0c9359d700 1 --2- 172.21.15.55:0/3708291281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8c1597f0 0x7f0c8c0b2270 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0b2f20 tx=0x7f0c7c009250 comp rx=0 tx=0).stop 2023-10-06T03:46:11.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.042+0000 7f0c9359d700 1 -- 172.21.15.55:0/3708291281 shutdown_connections 2023-10-06T03:46:11.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.042+0000 7f0c9359d700 1 --2- 172.21.15.55:0/3708291281 >> v2:172.21.15.163:3300/0 conn(0x7f0c8c159420 0x7f0c8c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.042+0000 7f0c9359d700 1 --2- 172.21.15.55:0/3708291281 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8c1597f0 0x7f0c8c0b2270 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.042+0000 7f0c9359d700 1 -- 172.21.15.55:0/3708291281 >> 172.21.15.55:0/3708291281 conn(0x7f0c8c0d0660 msgr2=0x7f0c8c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:11.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.042+0000 7f0c9359d700 1 -- 172.21.15.55:0/3708291281 shutdown_connections 2023-10-06T03:46:11.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.042+0000 7f0c9359d700 1 -- 172.21.15.55:0/3708291281 wait complete. 2023-10-06T03:46:11.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.044+0000 7f0c9359d700 1 Processor -- start 2023-10-06T03:46:11.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.044+0000 7f0c9359d700 1 -- start start 2023-10-06T03:46:11.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.045+0000 7f0c9359d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c8c159420 0x7f0c8c169980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.045+0000 7f0c9359d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c8c1597f0 0x7f0c8c15c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.045+0000 7f0c9359d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8c15d230 0x7f0c8c153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.045+0000 7f0c9359d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c16d030 con 0x7f0c8c15d230 2023-10-06T03:46:11.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.045+0000 7f0c9359d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c169f90 con 0x7f0c8c159420 2023-10-06T03:46:11.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.045+0000 7f0c9359d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c16a270 con 0x7f0c8c1597f0 2023-10-06T03:46:11.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.045+0000 7f0c92d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8c15d230 0x7f0c8c153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.045+0000 7f0c92d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8c15d230 0x7f0c8c153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50228/0 (socket says 172.21.15.55:50228) 2023-10-06T03:46:11.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.045+0000 7f0c92d9c700 1 -- 172.21.15.55:0/1661568935 learned_addr learned my addr 172.21.15.55:0/1661568935 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:11.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.045+0000 7f0c9259b700 1 --2- 172.21.15.55:0/1661568935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c8c159420 0x7f0c8c169980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.045+0000 7f0c91d9a700 1 --2- 172.21.15.55:0/1661568935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c8c1597f0 0x7f0c8c15c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.046+0000 7f0c92d9c700 1 -- 172.21.15.55:0/1661568935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c8c1597f0 msgr2=0x7f0c8c15c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.046+0000 7f0c92d9c700 1 --2- 172.21.15.55:0/1661568935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c8c1597f0 0x7f0c8c15c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.046+0000 7f0c92d9c700 1 -- 172.21.15.55:0/1661568935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c8c159420 msgr2=0x7f0c8c169980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.046+0000 7f0c92d9c700 1 --2- 172.21.15.55:0/1661568935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c8c159420 0x7f0c8c169980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.046+0000 7f0c92d9c700 1 -- 172.21.15.55:0/1661568935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c0094f0 con 0x7f0c8c15d230 2023-10-06T03:46:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.046+0000 7f0c92d9c700 1 --2- 172.21.15.55:0/1661568935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8c15d230 0x7f0c8c153f20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400bb80 tx=0x7f0c84009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.046+0000 7f0c827fc700 1 -- 172.21.15.55:0/1661568935 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8400c970 con 0x7f0c8c15d230 2023-10-06T03:46:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.046+0000 7f0c827fc700 1 -- 172.21.15.55:0/1661568935 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8400cad0 con 0x7f0c8c15d230 2023-10-06T03:46:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.046+0000 7f0c827fc700 1 -- 172.21.15.55:0/1661568935 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8400ce90 con 0x7f0c8c15d230 2023-10-06T03:46:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.046+0000 7f0c9359d700 1 -- 172.21.15.55:0/1661568935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8c16a550 con 0x7f0c8c15d230 2023-10-06T03:46:11.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.046+0000 7f0c9359d700 1 -- 172.21.15.55:0/1661568935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c15c9d0 con 0x7f0c8c15d230 2023-10-06T03:46:11.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.047+0000 7f0c9359d700 1 -- 172.21.15.55:0/1661568935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8c16d1a0 con 0x7f0c8c15d230 2023-10-06T03:46:11.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.048+0000 7f0c827fc700 1 -- 172.21.15.55:0/1661568935 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c84014620 con 0x7f0c8c15d230 2023-10-06T03:46:11.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.049+0000 7f0c827fc700 1 --2- 172.21.15.55:0/1661568935 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0c70066f30 0x7f0c700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.049+0000 7f0c827fc700 1 -- 172.21.15.55:0/1661568935 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f0c84084940 con 0x7f0c8c15d230 2023-10-06T03:46:11.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.049+0000 7f0c9259b700 1 --2- 172.21.15.55:0/1661568935 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0c70066f30 0x7f0c700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.050+0000 7f0c9259b700 1 --2- 172.21.15.55:0/1661568935 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0c70066f30 0x7f0c700693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0c88006fd0 tx=0x7f0c88008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:11.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.053+0000 7f0c827fc700 1 -- 172.21.15.55:0/1661568935 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c8c16d1a0 con 0x7f0c8c15d230 2023-10-06T03:46:11.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.298+0000 7f0c9359d700 1 -- 172.21.15.55:0/1661568935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c8c16d170 con 0x7f0c8c15d230 2023-10-06T03:46:11.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.299+0000 7f0c827fc700 1 -- 172.21.15.55:0/1661568935 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f0c84051840 con 0x7f0c8c15d230 2023-10-06T03:46:11.302 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:46:11.302 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3493866751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3493866751},{"type":"v1","addr":"172.21.15.163:6836","nonce":3493866751}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3246061279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3246061279},{"type":"v1","addr":"172.21.15.55:6835","nonce":3246061279}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1366768080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1366768080},{"type":"v1","addr":"172.21.15.55:6834","nonce":1366768080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:46:07.292050+0000","modified":"2023-10-06T03:46:10.815261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6833/2287859778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2287859778},{"type":"v1","addr":"172.21.15.163:6833","nonce":2287859778}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T03:46:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.301+0000 7f0c9359d700 1 -- 172.21.15.55:0/1661568935 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0c70066f30 msgr2=0x7f0c700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.301+0000 7f0c9359d700 1 --2- 172.21.15.55:0/1661568935 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0c70066f30 0x7f0c700693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0c88006fd0 tx=0x7f0c88008040 comp rx=0 tx=0).stop 2023-10-06T03:46:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.301+0000 7f0c9359d700 1 -- 172.21.15.55:0/1661568935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8c15d230 msgr2=0x7f0c8c153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.301+0000 7f0c9359d700 1 --2- 172.21.15.55:0/1661568935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8c15d230 0x7f0c8c153f20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400bb80 tx=0x7f0c84009810 comp rx=0 tx=0).stop 2023-10-06T03:46:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.301+0000 7f0c9359d700 1 -- 172.21.15.55:0/1661568935 shutdown_connections 2023-10-06T03:46:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.301+0000 7f0c9359d700 1 --2- 172.21.15.55:0/1661568935 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0c70066f30 0x7f0c700693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.301+0000 7f0c9359d700 1 --2- 172.21.15.55:0/1661568935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0c8c15d230 0x7f0c8c153f20 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.301+0000 7f0c9359d700 1 --2- 172.21.15.55:0/1661568935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0c8c159420 0x7f0c8c169980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.301+0000 7f0c9359d700 1 --2- 172.21.15.55:0/1661568935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0c8c1597f0 0x7f0c8c15c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.301+0000 7f0c9359d700 1 -- 172.21.15.55:0/1661568935 >> 172.21.15.55:0/1661568935 conn(0x7f0c8c0d0660 msgr2=0x7f0c8c1553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:11.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.301+0000 7f0c9359d700 1 -- 172.21.15.55:0/1661568935 shutdown_connections 2023-10-06T03:46:11.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.301+0000 7f0c9359d700 1 -- 172.21.15.55:0/1661568935 wait complete. 2023-10-06T03:46:11.305 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 26 2023-10-06T03:46:11.321 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:46:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.509+0000 7f97f7fff700 1 Processor -- start 2023-10-06T03:46:11.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.509+0000 7f97f7fff700 1 -- start start 2023-10-06T03:46:11.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.510+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97f81595a0 0x7f97f80b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.510+0000 7f97f7fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f97f80c8830 0x7f97f8159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.510+0000 7f97f7fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f80b1410 con 0x7f97f80c6c60 2023-10-06T03:46:11.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.510+0000 7f97f7fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f80b2b90 con 0x7f97f80c8830 2023-10-06T03:46:11.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.510+0000 7f97f7fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8063230 con 0x7f97f81595a0 2023-10-06T03:46:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.511+0000 7f97f6ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97f81595a0 0x7f97f80b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.511+0000 7f97f77fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f97f80c8830 0x7f97f8159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.511+0000 7f97f77fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f97f80c8830 0x7f97f8159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:56820/0 (socket says 172.21.15.55:56820) 2023-10-06T03:46:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.511+0000 7f97f77fe700 1 -- 172.21.15.55:0/2561525960 learned_addr learned my addr 172.21.15.55:0/2561525960 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.511+0000 7f97f6ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97f81595a0 0x7f97f80b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:46592/0 (socket says 172.21.15.55:46592) 2023-10-06T03:46:11.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.511+0000 7f97f67fc700 1 --1- 172.21.15.55:0/2561525960 >> v1:172.21.15.163:6789/0 conn(0x7f97f80c6c60 0x7f97f80c8120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:37382/0 (socket says 172.21.15.55:37382) 2023-10-06T03:46:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.512+0000 7f97f77fe700 1 -- 172.21.15.55:0/2561525960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97f81595a0 msgr2=0x7f97f80b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.512+0000 7f97f77fe700 1 --2- 172.21.15.55:0/2561525960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97f81595a0 0x7f97f80b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.512+0000 7f97f77fe700 1 -- 172.21.15.55:0/2561525960 >> v1:172.21.15.163:6789/0 conn(0x7f97f80c6c60 legacy=0x7f97f80c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.512+0000 7f97f77fe700 1 -- 172.21.15.55:0/2561525960 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f80634f0 con 0x7f97f80c8830 2023-10-06T03:46:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.512+0000 7f97f5ffb700 1 -- 172.21.15.55:0/2561525960 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265444060 0 0) 0x7f97f80b1410 con 0x7f97f80c6c60 2023-10-06T03:46:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.512+0000 7f97f6ffd700 1 --2- 172.21.15.55:0/2561525960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97f81595a0 0x7f97f80b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:46:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.512+0000 7f97f77fe700 1 --2- 172.21.15.55:0/2561525960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80c8830 0x7f97f8159970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009f20 tx=0x7f97f0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50a5c9a3a61c4d15 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:11.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.512+0000 7f97f5ffb700 1 -- 172.21.15.55:0/2561525960 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f000e070 con 0x7f97f80c8830 2023-10-06T03:46:11.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.512+0000 7f97f5ffb700 1 -- 172.21.15.55:0/2561525960 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f000b9d0 con 0x7f97f80c8830 2023-10-06T03:46:11.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.512+0000 7f97f5ffb700 1 -- 172.21.15.55:0/2561525960 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00092e0 con 0x7f97f80c8830 2023-10-06T03:46:11.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.513+0000 7f97f7fff700 1 -- 172.21.15.55:0/2561525960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80c8830 msgr2=0x7f97f8159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:11.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.513+0000 7f97f7fff700 1 --2- 172.21.15.55:0/2561525960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80c8830 0x7f97f8159970 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009f20 tx=0x7f97f0009840 comp rx=0 tx=0).stop 2023-10-06T03:46:11.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.513+0000 7f97f7fff700 1 -- 172.21.15.55:0/2561525960 shutdown_connections 2023-10-06T03:46:11.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.513+0000 7f97f7fff700 1 --2- 172.21.15.55:0/2561525960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97f81595a0 0x7f97f80b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.513+0000 7f97f7fff700 1 --2- 172.21.15.55:0/2561525960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f80c8830 0x7f97f8159970 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.513+0000 7f97f7fff700 1 -- 172.21.15.55:0/2561525960 >> 172.21.15.55:0/2561525960 conn(0x7f97f806ddb0 msgr2=0x7f97f806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:11.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.513+0000 7f97f7fff700 1 -- 172.21.15.55:0/2561525960 shutdown_connections 2023-10-06T03:46:11.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.513+0000 7f97f7fff700 1 -- 172.21.15.55:0/2561525960 wait complete. 2023-10-06T03:46:11.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.514+0000 7f97f7fff700 1 Processor -- start 2023-10-06T03:46:11.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.515+0000 7f97f7fff700 1 -- start start 2023-10-06T03:46:11.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.515+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97f80c6c60 0x7f97f81519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97f80c8830 0x7f97f813c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f81595a0 0x7f97f815c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f7fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f814cda0 con 0x7f97f80c8830 2023-10-06T03:46:11.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f7fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f815e8b0 con 0x7f97f81595a0 2023-10-06T03:46:11.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f7fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f815eb90 con 0x7f97f80c6c60 2023-10-06T03:46:11.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f67fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97f80c8830 0x7f97f813c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f67fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97f80c8830 0x7f97f813c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50242/0 (socket says 172.21.15.55:50242) 2023-10-06T03:46:11.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f67fc700 1 -- 172.21.15.55:0/107214909 learned_addr learned my addr 172.21.15.55:0/107214909 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:11.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f6ffd700 1 --2- 172.21.15.55:0/107214909 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97f80c6c60 0x7f97f81519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f77fe700 1 --2- 172.21.15.55:0/107214909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f81595a0 0x7f97f815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f67fc700 1 -- 172.21.15.55:0/107214909 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97f80c6c60 msgr2=0x7f97f81519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:11.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f67fc700 1 --2- 172.21.15.55:0/107214909 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97f80c6c60 0x7f97f81519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f67fc700 1 -- 172.21.15.55:0/107214909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f81595a0 msgr2=0x7f97f815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:11.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f67fc700 1 --2- 172.21.15.55:0/107214909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f81595a0 0x7f97f815c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.516+0000 7f97f67fc700 1 -- 172.21.15.55:0/107214909 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8009420 con 0x7f97f80c8830 2023-10-06T03:46:11.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.517+0000 7f97f67fc700 1 --2- 172.21.15.55:0/107214909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97f80c8830 0x7f97f813c800 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0011e0 tx=0x7f97ec007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:11.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.517+0000 7f97deffd700 1 -- 172.21.15.55:0/107214909 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec005a70 con 0x7f97f80c8830 2023-10-06T03:46:11.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.517+0000 7f97deffd700 1 -- 172.21.15.55:0/107214909 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec005bd0 con 0x7f97f80c8830 2023-10-06T03:46:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.517+0000 7f97f7fff700 1 -- 172.21.15.55:0/107214909 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f0009ae0 con 0x7f97f80c8830 2023-10-06T03:46:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.517+0000 7f97deffd700 1 -- 172.21.15.55:0/107214909 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec0070d0 con 0x7f97f80c8830 2023-10-06T03:46:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.517+0000 7f97f7fff700 1 -- 172.21.15.55:0/107214909 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f814cff0 con 0x7f97f80c8830 2023-10-06T03:46:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.518+0000 7f97f7fff700 1 -- 172.21.15.55:0/107214909 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f804c3c0 con 0x7f97f80c8830 2023-10-06T03:46:11.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.519+0000 7f97deffd700 1 -- 172.21.15.55:0/107214909 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97ec007480 con 0x7f97f80c8830 2023-10-06T03:46:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.520+0000 7f97deffd700 1 --2- 172.21.15.55:0/107214909 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97e4066f30 0x7f97e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.520+0000 7f97deffd700 1 -- 172.21.15.55:0/107214909 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f97ec083cb0 con 0x7f97f80c8830 2023-10-06T03:46:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.521+0000 7f97f6ffd700 1 --2- 172.21.15.55:0/107214909 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97e4066f30 0x7f97e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.521+0000 7f97f6ffd700 1 --2- 172.21.15.55:0/107214909 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97e4066f30 0x7f97e40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009ea0 tx=0x7f97e80091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:11.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.524+0000 7f97deffd700 1 -- 172.21.15.55:0/107214909 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97ec050c50 con 0x7f97f80c8830 2023-10-06T03:46:11.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.727+0000 7f97f7fff700 1 -- 172.21.15.55:0/107214909 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97f80637d0 con 0x7f97f80c8830 2023-10-06T03:46:11.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.728+0000 7f97deffd700 1 -- 172.21.15.55:0/107214909 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+17465 (secure 0 0 0) 0x7f97ec017070 con 0x7f97f80c8830 2023-10-06T03:46:11.731 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:46:11.731 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":64,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:46:08.783394+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:46:05.457264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T03:46:06.258505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:46:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.730+0000 7f97f7fff700 1 -- 172.21.15.55:0/107214909 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97e4066f30 msgr2=0x7f97e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.731+0000 7f97f7fff700 1 --2- 172.21.15.55:0/107214909 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97e4066f30 0x7f97e40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009ea0 tx=0x7f97e80091a0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.731+0000 7f97f7fff700 1 -- 172.21.15.55:0/107214909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97f80c8830 msgr2=0x7f97f813c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.731+0000 7f97f7fff700 1 --2- 172.21.15.55:0/107214909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97f80c8830 0x7f97f813c800 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0011e0 tx=0x7f97ec007e10 comp rx=0 tx=0).stop 2023-10-06T03:46:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.731+0000 7f97f7fff700 1 -- 172.21.15.55:0/107214909 shutdown_connections 2023-10-06T03:46:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.731+0000 7f97f7fff700 1 --2- 172.21.15.55:0/107214909 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97e4066f30 0x7f97e40693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.731+0000 7f97f7fff700 1 --2- 172.21.15.55:0/107214909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97f81595a0 0x7f97f815c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.731+0000 7f97f7fff700 1 --2- 172.21.15.55:0/107214909 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97f80c6c60 0x7f97f81519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.731+0000 7f97f7fff700 1 --2- 172.21.15.55:0/107214909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97f80c8830 0x7f97f813c800 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.731+0000 7f97f7fff700 1 -- 172.21.15.55:0/107214909 >> 172.21.15.55:0/107214909 conn(0x7f97f806ddb0 msgr2=0x7f97f8154630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.731+0000 7f97f7fff700 1 -- 172.21.15.55:0/107214909 shutdown_connections 2023-10-06T03:46:11.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.731+0000 7f97f7fff700 1 -- 172.21.15.55:0/107214909 wait complete. 2023-10-06T03:46:11.751 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:46:11.751 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:46:11.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.936+0000 7f3c7a99d700 1 Processor -- start 2023-10-06T03:46:11.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.937+0000 7f3c7a99d700 1 -- start start 2023-10-06T03:46:11.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.938+0000 7f3c7a99d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3c74157270 0x7f3c74068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.938+0000 7f3c7a99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c740688d0 0x7f3c740b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.938+0000 7f3c7a99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3c740b0a00 0x7f3c7406a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.938+0000 7f3c7a99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c7415a980 con 0x7f3c740b0a00 2023-10-06T03:46:11.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.938+0000 7f3c7a99d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c74068d20 con 0x7f3c74157270 2023-10-06T03:46:11.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.938+0000 7f3c7a99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7415a050 con 0x7f3c740688d0 2023-10-06T03:46:11.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.938+0000 7f3c7a19c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3c740b0a00 0x7f3c7406a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.938+0000 7f3c7a19c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3c740b0a00 0x7f3c7406a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50250/0 (socket says 172.21.15.55:50250) 2023-10-06T03:46:11.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.938+0000 7f3c7a19c700 1 -- 172.21.15.55:0/1453717561 learned_addr learned my addr 172.21.15.55:0/1453717561 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:11.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.938+0000 7f3c7999b700 1 --2- 172.21.15.55:0/1453717561 >> v2:172.21.15.163:3300/0 conn(0x7f3c74157270 0x7f3c74068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.938+0000 7f3c7919a700 1 --2- 172.21.15.55:0/1453717561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c740688d0 0x7f3c740b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.939+0000 7f3c7a19c700 1 -- 172.21.15.55:0/1453717561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c740688d0 msgr2=0x7f3c740b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.939+0000 7f3c7a19c700 1 --2- 172.21.15.55:0/1453717561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c740688d0 0x7f3c740b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.939+0000 7f3c7a19c700 1 -- 172.21.15.55:0/1453717561 >> v2:172.21.15.163:3300/0 conn(0x7f3c74157270 msgr2=0x7f3c74068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.939+0000 7f3c7a19c700 1 --2- 172.21.15.55:0/1453717561 >> v2:172.21.15.163:3300/0 conn(0x7f3c74157270 0x7f3c74068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.939+0000 7f3c7a19c700 1 -- 172.21.15.55:0/1453717561 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7415a330 con 0x7f3c740b0a00 2023-10-06T03:46:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.939+0000 7f3c7a19c700 1 --2- 172.21.15.55:0/1453717561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c740b0a00 0x7f3c7406a7e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3c740b3360 tx=0x7f3c6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4170d49ca76ad476 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.939+0000 7f3c78999700 1 -- 172.21.15.55:0/1453717561 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c00e070 con 0x7f3c740b0a00 2023-10-06T03:46:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.939+0000 7f3c78999700 1 -- 172.21.15.55:0/1453717561 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c00b970 con 0x7f3c740b0a00 2023-10-06T03:46:11.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.939+0000 7f3c78999700 1 -- 172.21.15.55:0/1453717561 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0092e0 con 0x7f3c740b0a00 2023-10-06T03:46:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.940+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1453717561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c740b0a00 msgr2=0x7f3c7406a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:11.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.940+0000 7f3c7a99d700 1 --2- 172.21.15.55:0/1453717561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c740b0a00 0x7f3c7406a7e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f3c740b3360 tx=0x7f3c6c009840 comp rx=0 tx=0).stop 2023-10-06T03:46:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.940+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1453717561 shutdown_connections 2023-10-06T03:46:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.940+0000 7f3c7a99d700 1 --2- 172.21.15.55:0/1453717561 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c740b0a00 0x7f3c7406a7e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.940+0000 7f3c7a99d700 1 --2- 172.21.15.55:0/1453717561 >> v2:172.21.15.163:3300/0 conn(0x7f3c74157270 0x7f3c74068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.940+0000 7f3c7a99d700 1 --2- 172.21.15.55:0/1453717561 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c740688d0 0x7f3c740b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.940+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1453717561 >> 172.21.15.55:0/1453717561 conn(0x7f3c740cd310 msgr2=0x7f3c740cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.940+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1453717561 shutdown_connections 2023-10-06T03:46:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.940+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1453717561 wait complete. 2023-10-06T03:46:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.942+0000 7f3c7a99d700 1 Processor -- start 2023-10-06T03:46:11.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.942+0000 7f3c7a99d700 1 -- start start 2023-10-06T03:46:11.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7a99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c740688d0 0x7f3c740ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7a99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c740b0a00 0x7f3c740aae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7a99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c74157270 0x7f3c740ab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7a99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c740aa270 con 0x7f3c74157270 2023-10-06T03:46:11.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7a99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c74169650 con 0x7f3c740b0a00 2023-10-06T03:46:11.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7a99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c74169910 con 0x7f3c740688d0 2023-10-06T03:46:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7a19c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c74157270 0x7f3c740ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7a19c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c74157270 0x7f3c740ab350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50254/0 (socket says 172.21.15.55:50254) 2023-10-06T03:46:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7a19c700 1 -- 172.21.15.55:0/1058248314 learned_addr learned my addr 172.21.15.55:0/1058248314 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7999b700 1 --2- 172.21.15.55:0/1058248314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c740688d0 0x7f3c740ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7919a700 1 --2- 172.21.15.55:0/1058248314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c740b0a00 0x7f3c740aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7a19c700 1 -- 172.21.15.55:0/1058248314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c740688d0 msgr2=0x7f3c740ae6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7a19c700 1 --2- 172.21.15.55:0/1058248314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c740688d0 0x7f3c740ae6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7a19c700 1 -- 172.21.15.55:0/1058248314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c740b0a00 msgr2=0x7f3c740aae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7a19c700 1 --2- 172.21.15.55:0/1058248314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c740b0a00 0x7f3c740aae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.943+0000 7f3c7a19c700 1 -- 172.21.15.55:0/1058248314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c009ae0 con 0x7f3c74157270 2023-10-06T03:46:11.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.944+0000 7f3c7a19c700 1 --2- 172.21.15.55:0/1058248314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c74157270 0x7f3c740ab350 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3c74068ed0 tx=0x7f3c6c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:11.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.944+0000 7f3c69ffb700 1 -- 172.21.15.55:0/1058248314 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c008b80 con 0x7f3c74157270 2023-10-06T03:46:11.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.944+0000 7f3c69ffb700 1 -- 172.21.15.55:0/1058248314 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c008ce0 con 0x7f3c74157270 2023-10-06T03:46:11.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.944+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1058248314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c74169bf0 con 0x7f3c74157270 2023-10-06T03:46:11.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.944+0000 7f3c69ffb700 1 -- 172.21.15.55:0/1058248314 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0266b0 con 0x7f3c74157270 2023-10-06T03:46:11.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.944+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1058248314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7416a1b0 con 0x7f3c74157270 2023-10-06T03:46:11.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.945+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1058248314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7415f210 con 0x7f3c74157270 2023-10-06T03:46:11.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.946+0000 7f3c69ffb700 1 -- 172.21.15.55:0/1058248314 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c6c003040 con 0x7f3c74157270 2023-10-06T03:46:11.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.947+0000 7f3c69ffb700 1 --2- 172.21.15.55:0/1058248314 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3c58066f30 0x7f3c580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:11.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.947+0000 7f3c69ffb700 1 -- 172.21.15.55:0/1058248314 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f3c6c0866e0 con 0x7f3c74157270 2023-10-06T03:46:11.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.947+0000 7f3c7999b700 1 --2- 172.21.15.55:0/1058248314 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3c58066f30 0x7f3c580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:11.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.948+0000 7f3c7999b700 1 --2- 172.21.15.55:0/1058248314 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3c58066f30 0x7f3c580693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009ca0 tx=0x7f3c70008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:11.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:11.951+0000 7f3c69ffb700 1 -- 172.21.15.55:0/1058248314 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c6c053690 con 0x7f3c74157270 2023-10-06T03:46:12.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.149+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1058248314 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c7415a610 con 0x7f3c58066f30 2023-10-06T03:46:12.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.161+0000 7f3c69ffb700 1 -- 172.21.15.55:0/1058248314 <== mgr.5452 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311497 (secure 0 0 0) 0x7f3c7415a610 con 0x7f3c58066f30 2023-10-06T03:46:12.168 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:46:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.166+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1058248314 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3c58066f30 msgr2=0x7f3c580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.166+0000 7f3c7a99d700 1 --2- 172.21.15.55:0/1058248314 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3c58066f30 0x7f3c580693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009ca0 tx=0x7f3c70008040 comp rx=0 tx=0).stop 2023-10-06T03:46:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.166+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1058248314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c74157270 msgr2=0x7f3c740ab350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.166+0000 7f3c7a99d700 1 --2- 172.21.15.55:0/1058248314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c74157270 0x7f3c740ab350 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3c74068ed0 tx=0x7f3c6c018000 comp rx=0 tx=0).stop 2023-10-06T03:46:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.166+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1058248314 shutdown_connections 2023-10-06T03:46:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.166+0000 7f3c7a99d700 1 --2- 172.21.15.55:0/1058248314 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3c58066f30 0x7f3c580693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.166+0000 7f3c7a99d700 1 --2- 172.21.15.55:0/1058248314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c74157270 0x7f3c740ab350 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.166+0000 7f3c7a99d700 1 --2- 172.21.15.55:0/1058248314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c740688d0 0x7f3c740ae6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.166+0000 7f3c7a99d700 1 --2- 172.21.15.55:0/1058248314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c740b0a00 0x7f3c740aae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.166+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1058248314 >> 172.21.15.55:0/1058248314 conn(0x7f3c740cd310 msgr2=0x7f3c74155d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.166+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1058248314 shutdown_connections 2023-10-06T03:46:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.166+0000 7f3c7a99d700 1 -- 172.21.15.55:0/1058248314 wait complete. 2023-10-06T03:46:12.172 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:46:12.186 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T03:46:11.939059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2434616,"kb_used_data":7160,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752540104,"statfs":{"total":773094113280,"available":770601066496,"internally_reserved":0,"allocated":7331840,"data_stored":4191372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001507"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.806717+0000","last_change":"2023-10-06T03:46:07.804657+0000","last_active":"2023-10-06T03:46:07.806717+0000","last_peered":"2023-10-06T03:46:07.806717+0000","last_clean":"2023-10-06T03:46:07.806717+0000","last_became_active":"2023-10-06T03:46:07.804382+0000","last_became_peered":"2023-10-06T03:46:07.804382+0000","last_unstale":"2023-10-06T03:46:07.806717+0000","last_undegraded":"2023-10-06T03:46:07.806717+0000","last_fullsized":"2023-10-06T03:46:07.806717+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:59.481973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.815654+0000","last_change":"2023-10-06T03:46:07.808232+0000","last_active":"2023-10-06T03:46:07.815654+0000","last_peered":"2023-10-06T03:46:07.815654+0000","last_clean":"2023-10-06T03:46:07.815654+0000","last_became_active":"2023-10-06T03:46:07.807987+0000","last_became_peered":"2023-10-06T03:46:07.807987+0000","last_unstale":"2023-10-06T03:46:07.815654+0000","last_undegraded":"2023-10-06T03:46:07.815654+0000","last_fullsized":"2023-10-06T03:46:07.815654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:22.338715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.819136+0000","last_change":"2023-10-06T03:46:07.814899+0000","last_active":"2023-10-06T03:46:07.819136+0000","last_peered":"2023-10-06T03:46:07.819136+0000","last_clean":"2023-10-06T03:46:07.819136+0000","last_became_active":"2023-10-06T03:46:07.814574+0000","last_became_peered":"2023-10-06T03:46:07.814574+0000","last_unstale":"2023-10-06T03:46:07.819136+0000","last_undegraded":"2023-10-06T03:46:07.819136+0000","last_fullsized":"2023-10-06T03:46:07.819136+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:24.995759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814858+0000","last_change":"2023-10-06T03:46:07.809583+0000","last_active":"2023-10-06T03:46:07.814858+0000","last_peered":"2023-10-06T03:46:07.814858+0000","last_clean":"2023-10-06T03:46:07.814858+0000","last_became_active":"2023-10-06T03:46:07.809323+0000","last_became_peered":"2023-10-06T03:46:07.809323+0000","last_unstale":"2023-10-06T03:46:07.814858+0000","last_undegraded":"2023-10-06T03:46:07.814858+0000","last_fullsized":"2023-10-06T03:46:07.814858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:35.844726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.816447+0000","last_change":"2023-10-06T03:46:07.814855+0000","last_active":"2023-10-06T03:46:07.816447+0000","last_peered":"2023-10-06T03:46:07.816447+0000","last_clean":"2023-10-06T03:46:07.816447+0000","last_became_active":"2023-10-06T03:46:07.814589+0000","last_became_peered":"2023-10-06T03:46:07.814589+0000","last_unstale":"2023-10-06T03:46:07.816447+0000","last_undegraded":"2023-10-06T03:46:07.816447+0000","last_fullsized":"2023-10-06T03:46:07.816447+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:43.430204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.808111+0000","last_change":"2023-10-06T03:46:07.805282+0000","last_active":"2023-10-06T03:46:07.808111+0000","last_peered":"2023-10-06T03:46:07.808111+0000","last_clean":"2023-10-06T03:46:07.808111+0000","last_became_active":"2023-10-06T03:46:07.805000+0000","last_became_peered":"2023-10-06T03:46:07.805000+0000","last_unstale":"2023-10-06T03:46:07.808111+0000","last_undegraded":"2023-10-06T03:46:07.808111+0000","last_fullsized":"2023-10-06T03:46:07.808111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:21.445776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.811405+0000","last_change":"2023-10-06T03:46:07.803595+0000","last_active":"2023-10-06T03:46:07.811405+0000","last_peered":"2023-10-06T03:46:07.811405+0000","last_clean":"2023-10-06T03:46:07.811405+0000","last_became_active":"2023-10-06T03:46:07.802830+0000","last_became_peered":"2023-10-06T03:46:07.802830+0000","last_unstale":"2023-10-06T03:46:07.811405+0000","last_undegraded":"2023-10-06T03:46:07.811405+0000","last_fullsized":"2023-10-06T03:46:07.811405+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:19.826311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.825347+0000","last_change":"2023-10-06T03:46:07.822899+0000","last_active":"2023-10-06T03:46:07.825347+0000","last_peered":"2023-10-06T03:46:07.825347+0000","last_clean":"2023-10-06T03:46:07.825347+0000","last_became_active":"2023-10-06T03:46:07.822585+0000","last_became_peered":"2023-10-06T03:46:07.822585+0000","last_unstale":"2023-10-06T03:46:07.825347+0000","last_undegraded":"2023-10-06T03:46:07.825347+0000","last_fullsized":"2023-10-06T03:46:07.825347+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:49.832412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.819815+0000","last_change":"2023-10-06T03:46:07.808808+0000","last_active":"2023-10-06T03:46:07.819815+0000","last_peered":"2023-10-06T03:46:07.819815+0000","last_clean":"2023-10-06T03:46:07.819815+0000","last_became_active":"2023-10-06T03:46:07.808510+0000","last_became_peered":"2023-10-06T03:46:07.808510+0000","last_unstale":"2023-10-06T03:46:07.819815+0000","last_undegraded":"2023-10-06T03:46:07.819815+0000","last_fullsized":"2023-10-06T03:46:07.819815+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:34.272462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.822048+0000","last_change":"2023-10-06T03:46:07.816274+0000","last_active":"2023-10-06T03:46:07.822048+0000","last_peered":"2023-10-06T03:46:07.822048+0000","last_clean":"2023-10-06T03:46:07.822048+0000","last_became_active":"2023-10-06T03:46:07.815962+0000","last_became_peered":"2023-10-06T03:46:07.815962+0000","last_unstale":"2023-10-06T03:46:07.822048+0000","last_undegraded":"2023-10-06T03:46:07.822048+0000","last_fullsized":"2023-10-06T03:46:07.822048+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:46.633052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.803942+0000","last_change":"2023-10-06T03:46:07.797756+0000","last_active":"2023-10-06T03:46:07.803942+0000","last_peered":"2023-10-06T03:46:07.803942+0000","last_clean":"2023-10-06T03:46:07.803942+0000","last_became_active":"2023-10-06T03:46:07.797498+0000","last_became_peered":"2023-10-06T03:46:07.797498+0000","last_unstale":"2023-10-06T03:46:07.803942+0000","last_undegraded":"2023-10-06T03:46:07.803942+0000","last_fullsized":"2023-10-06T03:46:07.803942+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:21.428503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.821375+0000","last_change":"2023-10-06T03:46:07.816385+0000","last_active":"2023-10-06T03:46:07.821375+0000","last_peered":"2023-10-06T03:46:07.821375+0000","last_clean":"2023-10-06T03:46:07.821375+0000","last_became_active":"2023-10-06T03:46:07.816139+0000","last_became_peered":"2023-10-06T03:46:07.816139+0000","last_unstale":"2023-10-06T03:46:07.821375+0000","last_undegraded":"2023-10-06T03:46:07.821375+0000","last_fullsized":"2023-10-06T03:46:07.821375+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:45.782682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.811764+0000","last_change":"2023-10-06T03:46:07.808806+0000","last_active":"2023-10-06T03:46:07.811764+0000","last_peered":"2023-10-06T03:46:07.811764+0000","last_clean":"2023-10-06T03:46:07.811764+0000","last_became_active":"2023-10-06T03:46:07.808553+0000","last_became_peered":"2023-10-06T03:46:07.808553+0000","last_unstale":"2023-10-06T03:46:07.811764+0000","last_undegraded":"2023-10-06T03:46:07.811764+0000","last_fullsized":"2023-10-06T03:46:07.811764+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:30.038870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.808026+0000","last_change":"2023-10-06T03:46:07.805495+0000","last_active":"2023-10-06T03:46:07.808026+0000","last_peered":"2023-10-06T03:46:07.808026+0000","last_clean":"2023-10-06T03:46:07.808026+0000","last_became_active":"2023-10-06T03:46:07.802639+0000","last_became_peered":"2023-10-06T03:46:07.802639+0000","last_unstale":"2023-10-06T03:46:07.808026+0000","last_undegraded":"2023-10-06T03:46:07.808026+0000","last_fullsized":"2023-10-06T03:46:07.808026+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:34.747585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.819413+0000","last_change":"2023-10-06T03:46:07.816804+0000","last_active":"2023-10-06T03:46:07.819413+0000","last_peered":"2023-10-06T03:46:07.819413+0000","last_clean":"2023-10-06T03:46:07.819413+0000","last_became_active":"2023-10-06T03:46:07.816489+0000","last_became_peered":"2023-10-06T03:46:07.816489+0000","last_unstale":"2023-10-06T03:46:07.819413+0000","last_undegraded":"2023-10-06T03:46:07.819413+0000","last_fullsized":"2023-10-06T03:46:07.819413+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:07.612249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.811799+0000","last_change":"2023-10-06T03:46:07.808278+0000","last_active":"2023-10-06T03:46:07.811799+0000","last_peered":"2023-10-06T03:46:07.811799+0000","last_clean":"2023-10-06T03:46:07.811799+0000","last_became_active":"2023-10-06T03:46:07.807962+0000","last_became_peered":"2023-10-06T03:46:07.807962+0000","last_unstale":"2023-10-06T03:46:07.811799+0000","last_undegraded":"2023-10-06T03:46:07.811799+0000","last_fullsized":"2023-10-06T03:46:07.811799+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:37.140556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.811353+0000","last_change":"2023-10-06T03:46:07.805483+0000","last_active":"2023-10-06T03:46:07.811353+0000","last_peered":"2023-10-06T03:46:07.811353+0000","last_clean":"2023-10-06T03:46:07.811353+0000","last_became_active":"2023-10-06T03:46:07.804915+0000","last_became_peered":"2023-10-06T03:46:07.804915+0000","last_unstale":"2023-10-06T03:46:07.811353+0000","last_undegraded":"2023-10-06T03:46:07.811353+0000","last_fullsized":"2023-10-06T03:46:07.811353+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:13.040618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.815674+0000","last_change":"2023-10-06T03:46:07.808815+0000","last_active":"2023-10-06T03:46:07.815674+0000","last_peered":"2023-10-06T03:46:07.815674+0000","last_clean":"2023-10-06T03:46:07.815674+0000","last_became_active":"2023-10-06T03:46:07.808578+0000","last_became_peered":"2023-10-06T03:46:07.808578+0000","last_unstale":"2023-10-06T03:46:07.815674+0000","last_undegraded":"2023-10-06T03:46:07.815674+0000","last_fullsized":"2023-10-06T03:46:07.815674+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:36.465082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.818657+0000","last_change":"2023-10-06T03:46:07.815410+0000","last_active":"2023-10-06T03:46:07.818657+0000","last_peered":"2023-10-06T03:46:07.818657+0000","last_clean":"2023-10-06T03:46:07.818657+0000","last_became_active":"2023-10-06T03:46:07.815144+0000","last_became_peered":"2023-10-06T03:46:07.815144+0000","last_unstale":"2023-10-06T03:46:07.818657+0000","last_undegraded":"2023-10-06T03:46:07.818657+0000","last_fullsized":"2023-10-06T03:46:07.818657+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:18.036383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814292+0000","last_change":"2023-10-06T03:46:07.812304+0000","last_active":"2023-10-06T03:46:07.814292+0000","last_peered":"2023-10-06T03:46:07.814292+0000","last_clean":"2023-10-06T03:46:07.814292+0000","last_became_active":"2023-10-06T03:46:07.812063+0000","last_became_peered":"2023-10-06T03:46:07.812063+0000","last_unstale":"2023-10-06T03:46:07.814292+0000","last_undegraded":"2023-10-06T03:46:07.814292+0000","last_fullsized":"2023-10-06T03:46:07.814292+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:46.892188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.821496+0000","last_change":"2023-10-06T03:46:07.817343+0000","last_active":"2023-10-06T03:46:07.821496+0000","last_peered":"2023-10-06T03:46:07.821496+0000","last_clean":"2023-10-06T03:46:07.821496+0000","last_became_active":"2023-10-06T03:46:07.817071+0000","last_became_peered":"2023-10-06T03:46:07.817071+0000","last_unstale":"2023-10-06T03:46:07.821496+0000","last_undegraded":"2023-10-06T03:46:07.821496+0000","last_fullsized":"2023-10-06T03:46:07.821496+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:07.722180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.825293+0000","last_change":"2023-10-06T03:46:07.823474+0000","last_active":"2023-10-06T03:46:07.825293+0000","last_peered":"2023-10-06T03:46:07.825293+0000","last_clean":"2023-10-06T03:46:07.825293+0000","last_became_active":"2023-10-06T03:46:07.822448+0000","last_became_peered":"2023-10-06T03:46:07.822448+0000","last_unstale":"2023-10-06T03:46:07.825293+0000","last_undegraded":"2023-10-06T03:46:07.825293+0000","last_fullsized":"2023-10-06T03:46:07.825293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:51.772579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.800432+0000","last_change":"2023-10-06T03:46:07.796586+0000","last_active":"2023-10-06T03:46:07.800432+0000","last_peered":"2023-10-06T03:46:07.800432+0000","last_clean":"2023-10-06T03:46:07.800432+0000","last_became_active":"2023-10-06T03:46:07.796299+0000","last_became_peered":"2023-10-06T03:46:07.796299+0000","last_unstale":"2023-10-06T03:46:07.800432+0000","last_undegraded":"2023-10-06T03:46:07.800432+0000","last_fullsized":"2023-10-06T03:46:07.800432+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:59.672250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.806376+0000","last_change":"2023-10-06T03:46:07.802200+0000","last_active":"2023-10-06T03:46:07.806376+0000","last_peered":"2023-10-06T03:46:07.806376+0000","last_clean":"2023-10-06T03:46:07.806376+0000","last_became_active":"2023-10-06T03:46:07.801913+0000","last_became_peered":"2023-10-06T03:46:07.801913+0000","last_unstale":"2023-10-06T03:46:07.806376+0000","last_undegraded":"2023-10-06T03:46:07.806376+0000","last_fullsized":"2023-10-06T03:46:07.806376+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:01.444525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.818140+0000","last_change":"2023-10-06T03:46:07.813785+0000","last_active":"2023-10-06T03:46:07.818140+0000","last_peered":"2023-10-06T03:46:07.818140+0000","last_clean":"2023-10-06T03:46:07.818140+0000","last_became_active":"2023-10-06T03:46:07.813463+0000","last_became_peered":"2023-10-06T03:46:07.813463+0000","last_unstale":"2023-10-06T03:46:07.818140+0000","last_undegraded":"2023-10-06T03:46:07.818140+0000","last_fullsized":"2023-10-06T03:46:07.818140+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:44.254939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.819805+0000","last_change":"2023-10-06T03:46:07.808918+0000","last_active":"2023-10-06T03:46:07.819805+0000","last_peered":"2023-10-06T03:46:07.819805+0000","last_clean":"2023-10-06T03:46:07.819805+0000","last_became_active":"2023-10-06T03:46:07.808626+0000","last_became_peered":"2023-10-06T03:46:07.808626+0000","last_unstale":"2023-10-06T03:46:07.819805+0000","last_undegraded":"2023-10-06T03:46:07.819805+0000","last_fullsized":"2023-10-06T03:46:07.819805+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:48.641104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.802130+0000","last_change":"2023-10-06T03:46:07.800026+0000","last_active":"2023-10-06T03:46:07.802130+0000","last_peered":"2023-10-06T03:46:07.802130+0000","last_clean":"2023-10-06T03:46:07.802130+0000","last_became_active":"2023-10-06T03:46:07.799723+0000","last_became_peered":"2023-10-06T03:46:07.799723+0000","last_unstale":"2023-10-06T03:46:07.802130+0000","last_undegraded":"2023-10-06T03:46:07.802130+0000","last_fullsized":"2023-10-06T03:46:07.802130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:03.157309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814000+0000","last_change":"2023-10-06T03:46:07.803959+0000","last_active":"2023-10-06T03:46:07.814000+0000","last_peered":"2023-10-06T03:46:07.814000+0000","last_clean":"2023-10-06T03:46:07.814000+0000","last_became_active":"2023-10-06T03:46:07.801538+0000","last_became_peered":"2023-10-06T03:46:07.801538+0000","last_unstale":"2023-10-06T03:46:07.814000+0000","last_undegraded":"2023-10-06T03:46:07.814000+0000","last_fullsized":"2023-10-06T03:46:07.814000+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:46.878707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.803395+0000","last_change":"2023-10-06T03:46:07.798753+0000","last_active":"2023-10-06T03:46:07.803395+0000","last_peered":"2023-10-06T03:46:07.803395+0000","last_clean":"2023-10-06T03:46:07.803395+0000","last_became_active":"2023-10-06T03:46:07.798435+0000","last_became_peered":"2023-10-06T03:46:07.798435+0000","last_unstale":"2023-10-06T03:46:07.803395+0000","last_undegraded":"2023-10-06T03:46:07.803395+0000","last_fullsized":"2023-10-06T03:46:07.803395+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:59.029330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814887+0000","last_change":"2023-10-06T03:46:07.810635+0000","last_active":"2023-10-06T03:46:07.814887+0000","last_peered":"2023-10-06T03:46:07.814887+0000","last_clean":"2023-10-06T03:46:07.814887+0000","last_became_active":"2023-10-06T03:46:07.810115+0000","last_became_peered":"2023-10-06T03:46:07.810115+0000","last_unstale":"2023-10-06T03:46:07.814887+0000","last_undegraded":"2023-10-06T03:46:07.814887+0000","last_fullsized":"2023-10-06T03:46:07.814887+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:17:25.155702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.811386+0000","last_change":"2023-10-06T03:46:07.803398+0000","last_active":"2023-10-06T03:46:07.811386+0000","last_peered":"2023-10-06T03:46:07.811386+0000","last_clean":"2023-10-06T03:46:07.811386+0000","last_became_active":"2023-10-06T03:46:07.802453+0000","last_became_peered":"2023-10-06T03:46:07.802453+0000","last_unstale":"2023-10-06T03:46:07.811386+0000","last_undegraded":"2023-10-06T03:46:07.811386+0000","last_fullsized":"2023-10-06T03:46:07.811386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:08.592648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817209+0000","last_change":"2023-10-06T03:46:07.812420+0000","last_active":"2023-10-06T03:46:07.817209+0000","last_peered":"2023-10-06T03:46:07.817209+0000","last_clean":"2023-10-06T03:46:07.817209+0000","last_became_active":"2023-10-06T03:46:07.812142+0000","last_became_peered":"2023-10-06T03:46:07.812142+0000","last_unstale":"2023-10-06T03:46:07.817209+0000","last_undegraded":"2023-10-06T03:46:07.817209+0000","last_fullsized":"2023-10-06T03:46:07.817209+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:07.965204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.807623+0000","last_change":"2023-10-06T03:46:07.806249+0000","last_active":"2023-10-06T03:46:07.807623+0000","last_peered":"2023-10-06T03:46:07.807623+0000","last_clean":"2023-10-06T03:46:07.807623+0000","last_became_active":"2023-10-06T03:46:07.804978+0000","last_became_peered":"2023-10-06T03:46:07.804978+0000","last_unstale":"2023-10-06T03:46:07.807623+0000","last_undegraded":"2023-10-06T03:46:07.807623+0000","last_fullsized":"2023-10-06T03:46:07.807623+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:21.695190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"63'4","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817757+0000","last_change":"2023-10-06T03:46:06.797810+0000","last_active":"2023-10-06T03:46:07.817757+0000","last_peered":"2023-10-06T03:46:07.817757+0000","last_clean":"2023-10-06T03:46:07.817757+0000","last_became_active":"2023-10-06T03:46:06.796496+0000","last_became_peered":"2023-10-06T03:46:06.796496+0000","last_unstale":"2023-10-06T03:46:07.817757+0000","last_undegraded":"2023-10-06T03:46:07.817757+0000","last_fullsized":"2023-10-06T03:46:07.817757+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:50.161636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817136+0000","last_change":"2023-10-06T03:46:07.814556+0000","last_active":"2023-10-06T03:46:07.817136+0000","last_peered":"2023-10-06T03:46:07.817136+0000","last_clean":"2023-10-06T03:46:07.817136+0000","last_became_active":"2023-10-06T03:46:07.814308+0000","last_became_peered":"2023-10-06T03:46:07.814308+0000","last_unstale":"2023-10-06T03:46:07.817136+0000","last_undegraded":"2023-10-06T03:46:07.817136+0000","last_fullsized":"2023-10-06T03:46:07.817136+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:30.038870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.821840+0000","last_change":"2023-10-06T03:46:06.806361+0000","last_active":"2023-10-06T03:46:07.821840+0000","last_peered":"2023-10-06T03:46:07.821840+0000","last_clean":"2023-10-06T03:46:07.821840+0000","last_became_active":"2023-10-06T03:46:06.806058+0000","last_became_peered":"2023-10-06T03:46:06.806058+0000","last_unstale":"2023-10-06T03:46:07.821840+0000","last_undegraded":"2023-10-06T03:46:07.821840+0000","last_fullsized":"2023-10-06T03:46:07.821840+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:39.977632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814129+0000","last_change":"2023-10-06T03:46:07.811437+0000","last_active":"2023-10-06T03:46:07.814129+0000","last_peered":"2023-10-06T03:46:07.814129+0000","last_clean":"2023-10-06T03:46:07.814129+0000","last_became_active":"2023-10-06T03:46:07.811159+0000","last_became_peered":"2023-10-06T03:46:07.811159+0000","last_unstale":"2023-10-06T03:46:07.814129+0000","last_undegraded":"2023-10-06T03:46:07.814129+0000","last_fullsized":"2023-10-06T03:46:07.814129+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:43.069156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.822012+0000","last_change":"2023-10-06T03:46:06.798822+0000","last_active":"2023-10-06T03:46:07.822012+0000","last_peered":"2023-10-06T03:46:07.822012+0000","last_clean":"2023-10-06T03:46:07.822012+0000","last_became_active":"2023-10-06T03:46:06.796111+0000","last_became_peered":"2023-10-06T03:46:06.796111+0000","last_unstale":"2023-10-06T03:46:07.822012+0000","last_undegraded":"2023-10-06T03:46:07.822012+0000","last_fullsized":"2023-10-06T03:46:07.822012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:48.864196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.822730+0000","last_change":"2023-10-06T03:46:07.820140+0000","last_active":"2023-10-06T03:46:07.822730+0000","last_peered":"2023-10-06T03:46:07.822730+0000","last_clean":"2023-10-06T03:46:07.822730+0000","last_became_active":"2023-10-06T03:46:07.819719+0000","last_became_peered":"2023-10-06T03:46:07.819719+0000","last_unstale":"2023-10-06T03:46:07.822730+0000","last_undegraded":"2023-10-06T03:46:07.822730+0000","last_fullsized":"2023-10-06T03:46:07.822730+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:03.905822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.820682+0000","last_change":"2023-10-06T03:46:06.805041+0000","last_active":"2023-10-06T03:46:07.820682+0000","last_peered":"2023-10-06T03:46:07.820682+0000","last_clean":"2023-10-06T03:46:07.820682+0000","last_became_active":"2023-10-06T03:46:06.804709+0000","last_became_peered":"2023-10-06T03:46:06.804709+0000","last_unstale":"2023-10-06T03:46:07.820682+0000","last_undegraded":"2023-10-06T03:46:07.820682+0000","last_fullsized":"2023-10-06T03:46:07.820682+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:06.713183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.812650+0000","last_change":"2023-10-06T03:46:07.808640+0000","last_active":"2023-10-06T03:46:07.812650+0000","last_peered":"2023-10-06T03:46:07.812650+0000","last_clean":"2023-10-06T03:46:07.812650+0000","last_became_active":"2023-10-06T03:46:07.808335+0000","last_became_peered":"2023-10-06T03:46:07.808335+0000","last_unstale":"2023-10-06T03:46:07.812650+0000","last_undegraded":"2023-10-06T03:46:07.812650+0000","last_fullsized":"2023-10-06T03:46:07.812650+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:25.900934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.818998+0000","last_change":"2023-10-06T03:46:06.811044+0000","last_active":"2023-10-06T03:46:07.818998+0000","last_peered":"2023-10-06T03:46:07.818998+0000","last_clean":"2023-10-06T03:46:07.818998+0000","last_became_active":"2023-10-06T03:46:06.810769+0000","last_became_peered":"2023-10-06T03:46:06.810769+0000","last_unstale":"2023-10-06T03:46:07.818998+0000","last_undegraded":"2023-10-06T03:46:07.818998+0000","last_fullsized":"2023-10-06T03:46:07.818998+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:34.265734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.807961+0000","last_change":"2023-10-06T03:46:07.805807+0000","last_active":"2023-10-06T03:46:07.807961+0000","last_peered":"2023-10-06T03:46:07.807961+0000","last_clean":"2023-10-06T03:46:07.807961+0000","last_became_active":"2023-10-06T03:46:07.804914+0000","last_became_peered":"2023-10-06T03:46:07.804914+0000","last_unstale":"2023-10-06T03:46:07.807961+0000","last_undegraded":"2023-10-06T03:46:07.807961+0000","last_fullsized":"2023-10-06T03:46:07.807961+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:05.437972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.819472+0000","last_change":"2023-10-06T03:46:06.799430+0000","last_active":"2023-10-06T03:46:07.819472+0000","last_peered":"2023-10-06T03:46:07.819472+0000","last_clean":"2023-10-06T03:46:07.819472+0000","last_became_active":"2023-10-06T03:46:06.799144+0000","last_became_peered":"2023-10-06T03:46:06.799144+0000","last_unstale":"2023-10-06T03:46:07.819472+0000","last_undegraded":"2023-10-06T03:46:07.819472+0000","last_fullsized":"2023-10-06T03:46:07.819472+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:20.686193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.820024+0000","last_change":"2023-10-06T03:46:07.809525+0000","last_active":"2023-10-06T03:46:07.820024+0000","last_peered":"2023-10-06T03:46:07.820024+0000","last_clean":"2023-10-06T03:46:07.820024+0000","last_became_active":"2023-10-06T03:46:07.809231+0000","last_became_peered":"2023-10-06T03:46:07.809231+0000","last_unstale":"2023-10-06T03:46:07.820024+0000","last_undegraded":"2023-10-06T03:46:07.820024+0000","last_fullsized":"2023-10-06T03:46:07.820024+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:08:48.093050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.790915+0000","last_change":"2023-10-06T03:46:06.799556+0000","last_active":"2023-10-06T03:46:07.790915+0000","last_peered":"2023-10-06T03:46:07.790915+0000","last_clean":"2023-10-06T03:46:07.790915+0000","last_became_active":"2023-10-06T03:46:06.798555+0000","last_became_peered":"2023-10-06T03:46:06.798555+0000","last_unstale":"2023-10-06T03:46:07.790915+0000","last_undegraded":"2023-10-06T03:46:07.790915+0000","last_fullsized":"2023-10-06T03:46:07.790915+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:12:18.719894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817705+0000","last_change":"2023-10-06T03:46:07.811054+0000","last_active":"2023-10-06T03:46:07.817705+0000","last_peered":"2023-10-06T03:46:07.817705+0000","last_clean":"2023-10-06T03:46:07.817705+0000","last_became_active":"2023-10-06T03:46:07.810678+0000","last_became_peered":"2023-10-06T03:46:07.810678+0000","last_unstale":"2023-10-06T03:46:07.817705+0000","last_undegraded":"2023-10-06T03:46:07.817705+0000","last_fullsized":"2023-10-06T03:46:07.817705+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:35.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.790930+0000","last_change":"2023-10-06T03:46:06.811527+0000","last_active":"2023-10-06T03:46:07.790930+0000","last_peered":"2023-10-06T03:46:07.790930+0000","last_clean":"2023-10-06T03:46:07.790930+0000","last_became_active":"2023-10-06T03:46:06.811221+0000","last_became_peered":"2023-10-06T03:46:06.811221+0000","last_unstale":"2023-10-06T03:46:07.790930+0000","last_undegraded":"2023-10-06T03:46:07.790930+0000","last_fullsized":"2023-10-06T03:46:07.790930+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:20.686193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.824380+0000","last_change":"2023-10-06T03:46:07.820458+0000","last_active":"2023-10-06T03:46:07.824380+0000","last_peered":"2023-10-06T03:46:07.824380+0000","last_clean":"2023-10-06T03:46:07.824380+0000","last_became_active":"2023-10-06T03:46:07.820218+0000","last_became_peered":"2023-10-06T03:46:07.820218+0000","last_unstale":"2023-10-06T03:46:07.824380+0000","last_undegraded":"2023-10-06T03:46:07.824380+0000","last_fullsized":"2023-10-06T03:46:07.824380+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:08:48.093050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.794304+0000","last_change":"2023-10-06T03:46:06.797122+0000","last_active":"2023-10-06T03:46:07.794304+0000","last_peered":"2023-10-06T03:46:07.794304+0000","last_clean":"2023-10-06T03:46:07.794304+0000","last_became_active":"2023-10-06T03:46:06.796755+0000","last_became_peered":"2023-10-06T03:46:06.796755+0000","last_unstale":"2023-10-06T03:46:07.794304+0000","last_undegraded":"2023-10-06T03:46:07.794304+0000","last_fullsized":"2023-10-06T03:46:07.794304+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:05.828405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817116+0000","last_change":"2023-10-06T03:46:07.812345+0000","last_active":"2023-10-06T03:46:07.817116+0000","last_peered":"2023-10-06T03:46:07.817116+0000","last_clean":"2023-10-06T03:46:07.817116+0000","last_became_active":"2023-10-06T03:46:07.812008+0000","last_became_peered":"2023-10-06T03:46:07.812008+0000","last_unstale":"2023-10-06T03:46:07.817116+0000","last_undegraded":"2023-10-06T03:46:07.817116+0000","last_fullsized":"2023-10-06T03:46:07.817116+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:59.481973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.800590+0000","last_change":"2023-10-06T03:46:06.809388+0000","last_active":"2023-10-06T03:46:07.800590+0000","last_peered":"2023-10-06T03:46:07.800590+0000","last_clean":"2023-10-06T03:46:07.800590+0000","last_became_active":"2023-10-06T03:46:06.809152+0000","last_became_peered":"2023-10-06T03:46:06.809152+0000","last_unstale":"2023-10-06T03:46:07.800590+0000","last_undegraded":"2023-10-06T03:46:07.800590+0000","last_fullsized":"2023-10-06T03:46:07.800590+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:34.265734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.800626+0000","last_change":"2023-10-06T03:46:07.797062+0000","last_active":"2023-10-06T03:46:07.800626+0000","last_peered":"2023-10-06T03:46:07.800626+0000","last_clean":"2023-10-06T03:46:07.800626+0000","last_became_active":"2023-10-06T03:46:07.796767+0000","last_became_peered":"2023-10-06T03:46:07.796767+0000","last_unstale":"2023-10-06T03:46:07.800626+0000","last_undegraded":"2023-10-06T03:46:07.800626+0000","last_fullsized":"2023-10-06T03:46:07.800626+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.631695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"63'3","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814848+0000","last_change":"2023-10-06T03:46:06.809953+0000","last_active":"2023-10-06T03:46:07.814848+0000","last_peered":"2023-10-06T03:46:07.814848+0000","last_clean":"2023-10-06T03:46:07.814848+0000","last_became_active":"2023-10-06T03:46:06.809646+0000","last_became_peered":"2023-10-06T03:46:06.809646+0000","last_unstale":"2023-10-06T03:46:07.814848+0000","last_undegraded":"2023-10-06T03:46:07.814848+0000","last_fullsized":"2023-10-06T03:46:07.814848+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:06.956208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.816313+0000","last_change":"2023-10-06T03:46:07.811277+0000","last_active":"2023-10-06T03:46:07.816313+0000","last_peered":"2023-10-06T03:46:07.816313+0000","last_clean":"2023-10-06T03:46:07.816313+0000","last_became_active":"2023-10-06T03:46:07.810989+0000","last_became_peered":"2023-10-06T03:46:07.810989+0000","last_unstale":"2023-10-06T03:46:07.816313+0000","last_undegraded":"2023-10-06T03:46:07.816313+0000","last_fullsized":"2023-10-06T03:46:07.816313+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:28.197821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"63'6","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.827579+0000","last_change":"2023-10-06T03:46:06.804616+0000","last_active":"2023-10-06T03:46:07.827579+0000","last_peered":"2023-10-06T03:46:07.827579+0000","last_clean":"2023-10-06T03:46:07.827579+0000","last_became_active":"2023-10-06T03:46:06.804341+0000","last_became_peered":"2023-10-06T03:46:06.804341+0000","last_unstale":"2023-10-06T03:46:07.827579+0000","last_undegraded":"2023-10-06T03:46:07.827579+0000","last_fullsized":"2023-10-06T03:46:07.827579+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:20.436780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817387+0000","last_change":"2023-10-06T03:46:07.809963+0000","last_active":"2023-10-06T03:46:07.817387+0000","last_peered":"2023-10-06T03:46:07.817387+0000","last_clean":"2023-10-06T03:46:07.817387+0000","last_became_active":"2023-10-06T03:46:07.809559+0000","last_became_peered":"2023-10-06T03:46:07.809559+0000","last_unstale":"2023-10-06T03:46:07.817387+0000","last_undegraded":"2023-10-06T03:46:07.817387+0000","last_fullsized":"2023-10-06T03:46:07.817387+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:46.633052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.819115+0000","last_change":"2023-10-06T03:46:06.809067+0000","last_active":"2023-10-06T03:46:07.819115+0000","last_peered":"2023-10-06T03:46:07.819115+0000","last_clean":"2023-10-06T03:46:07.819115+0000","last_became_active":"2023-10-06T03:46:06.808762+0000","last_became_peered":"2023-10-06T03:46:06.808762+0000","last_unstale":"2023-10-06T03:46:07.819115+0000","last_undegraded":"2023-10-06T03:46:07.819115+0000","last_fullsized":"2023-10-06T03:46:07.819115+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:34.835730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.816517+0000","last_change":"2023-10-06T03:46:07.805716+0000","last_active":"2023-10-06T03:46:07.816517+0000","last_peered":"2023-10-06T03:46:07.816517+0000","last_clean":"2023-10-06T03:46:07.816517+0000","last_became_active":"2023-10-06T03:46:07.805388+0000","last_became_peered":"2023-10-06T03:46:07.805388+0000","last_unstale":"2023-10-06T03:46:07.816517+0000","last_undegraded":"2023-10-06T03:46:07.816517+0000","last_fullsized":"2023-10-06T03:46:07.816517+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:09.430960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.791181+0000","last_change":"2023-10-06T03:46:06.796800+0000","last_active":"2023-10-06T03:46:07.791181+0000","last_peered":"2023-10-06T03:46:07.791181+0000","last_clean":"2023-10-06T03:46:07.791181+0000","last_became_active":"2023-10-06T03:46:06.796502+0000","last_became_peered":"2023-10-06T03:46:06.796502+0000","last_unstale":"2023-10-06T03:46:07.791181+0000","last_undegraded":"2023-10-06T03:46:07.791181+0000","last_fullsized":"2023-10-06T03:46:07.791181+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:47.041953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.820044+0000","last_change":"2023-10-06T03:46:07.810983+0000","last_active":"2023-10-06T03:46:07.820044+0000","last_peered":"2023-10-06T03:46:07.820044+0000","last_clean":"2023-10-06T03:46:07.820044+0000","last_became_active":"2023-10-06T03:46:07.810683+0000","last_became_peered":"2023-10-06T03:46:07.810683+0000","last_unstale":"2023-10-06T03:46:07.820044+0000","last_undegraded":"2023-10-06T03:46:07.820044+0000","last_fullsized":"2023-10-06T03:46:07.820044+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:07:51.709094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.794763+0000","last_change":"2023-10-06T03:46:06.799700+0000","last_active":"2023-10-06T03:46:07.794763+0000","last_peered":"2023-10-06T03:46:07.794763+0000","last_clean":"2023-10-06T03:46:07.794763+0000","last_became_active":"2023-10-06T03:46:06.799413+0000","last_became_peered":"2023-10-06T03:46:06.799413+0000","last_unstale":"2023-10-06T03:46:07.794763+0000","last_undegraded":"2023-10-06T03:46:07.794763+0000","last_fullsized":"2023-10-06T03:46:07.794763+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:26.208223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.811333+0000","last_change":"2023-10-06T03:46:07.804891+0000","last_active":"2023-10-06T03:46:07.811333+0000","last_peered":"2023-10-06T03:46:07.811333+0000","last_clean":"2023-10-06T03:46:07.811333+0000","last_became_active":"2023-10-06T03:46:07.804524+0000","last_became_peered":"2023-10-06T03:46:07.804524+0000","last_unstale":"2023-10-06T03:46:07.811333+0000","last_undegraded":"2023-10-06T03:46:07.811333+0000","last_fullsized":"2023-10-06T03:46:07.811333+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:43.349472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"63'8","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.826181+0000","last_change":"2023-10-06T03:46:06.803940+0000","last_active":"2023-10-06T03:46:07.826181+0000","last_peered":"2023-10-06T03:46:07.826181+0000","last_clean":"2023-10-06T03:46:07.826181+0000","last_became_active":"2023-10-06T03:46:06.803679+0000","last_became_peered":"2023-10-06T03:46:06.803679+0000","last_unstale":"2023-10-06T03:46:07.826181+0000","last_undegraded":"2023-10-06T03:46:07.826181+0000","last_fullsized":"2023-10-06T03:46:07.826181+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:58.472976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.825364+0000","last_change":"2023-10-06T03:46:07.823327+0000","last_active":"2023-10-06T03:46:07.825364+0000","last_peered":"2023-10-06T03:46:07.825364+0000","last_clean":"2023-10-06T03:46:07.825364+0000","last_became_active":"2023-10-06T03:46:07.823062+0000","last_became_peered":"2023-10-06T03:46:07.823062+0000","last_unstale":"2023-10-06T03:46:07.825364+0000","last_undegraded":"2023-10-06T03:46:07.825364+0000","last_fullsized":"2023-10-06T03:46:07.825364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:06.386235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.825688+0000","last_change":"2023-10-06T03:46:06.813181+0000","last_active":"2023-10-06T03:46:07.825688+0000","last_peered":"2023-10-06T03:46:07.825688+0000","last_clean":"2023-10-06T03:46:07.825688+0000","last_became_active":"2023-10-06T03:46:06.811641+0000","last_became_peered":"2023-10-06T03:46:06.811641+0000","last_unstale":"2023-10-06T03:46:07.825688+0000","last_undegraded":"2023-10-06T03:46:07.825688+0000","last_fullsized":"2023-10-06T03:46:07.825688+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:04.428975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.802095+0000","last_change":"2023-10-06T03:46:07.799765+0000","last_active":"2023-10-06T03:46:07.802095+0000","last_peered":"2023-10-06T03:46:07.802095+0000","last_clean":"2023-10-06T03:46:07.802095+0000","last_became_active":"2023-10-06T03:46:07.799489+0000","last_became_peered":"2023-10-06T03:46:07.799489+0000","last_unstale":"2023-10-06T03:46:07.802095+0000","last_undegraded":"2023-10-06T03:46:07.802095+0000","last_fullsized":"2023-10-06T03:46:07.802095+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:43.878689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.825737+0000","last_change":"2023-10-06T03:46:06.801829+0000","last_active":"2023-10-06T03:46:07.825737+0000","last_peered":"2023-10-06T03:46:07.825737+0000","last_clean":"2023-10-06T03:46:07.825737+0000","last_became_active":"2023-10-06T03:46:06.801568+0000","last_became_peered":"2023-10-06T03:46:06.801568+0000","last_unstale":"2023-10-06T03:46:07.825737+0000","last_undegraded":"2023-10-06T03:46:07.825737+0000","last_fullsized":"2023-10-06T03:46:07.825737+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:57.549516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.816554+0000","last_change":"2023-10-06T03:46:07.812835+0000","last_active":"2023-10-06T03:46:07.816554+0000","last_peered":"2023-10-06T03:46:07.816554+0000","last_clean":"2023-10-06T03:46:07.816554+0000","last_became_active":"2023-10-06T03:46:07.812564+0000","last_became_peered":"2023-10-06T03:46:07.812564+0000","last_unstale":"2023-10-06T03:46:07.816554+0000","last_undegraded":"2023-10-06T03:46:07.816554+0000","last_fullsized":"2023-10-06T03:46:07.816554+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:07:51.709094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.822409+0000","last_change":"2023-10-06T03:46:06.797393+0000","last_active":"2023-10-06T03:46:07.822409+0000","last_peered":"2023-10-06T03:46:07.822409+0000","last_clean":"2023-10-06T03:46:07.822409+0000","last_became_active":"2023-10-06T03:46:06.797117+0000","last_became_peered":"2023-10-06T03:46:06.797117+0000","last_unstale":"2023-10-06T03:46:07.822409+0000","last_undegraded":"2023-10-06T03:46:07.822409+0000","last_fullsized":"2023-10-06T03:46:07.822409+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:10.143016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.805053+0000","last_change":"2023-10-06T03:46:07.804195+0000","last_active":"2023-10-06T03:46:07.805053+0000","last_peered":"2023-10-06T03:46:07.805053+0000","last_clean":"2023-10-06T03:46:07.805053+0000","last_became_active":"2023-10-06T03:46:07.803833+0000","last_became_peered":"2023-10-06T03:46:07.803833+0000","last_unstale":"2023-10-06T03:46:07.805053+0000","last_undegraded":"2023-10-06T03:46:07.805053+0000","last_fullsized":"2023-10-06T03:46:07.805053+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:07.965204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.791011+0000","last_change":"2023-10-06T03:46:06.804991+0000","last_active":"2023-10-06T03:46:07.791011+0000","last_peered":"2023-10-06T03:46:07.791011+0000","last_clean":"2023-10-06T03:46:07.791011+0000","last_became_active":"2023-10-06T03:46:06.804659+0000","last_became_peered":"2023-10-06T03:46:06.804659+0000","last_unstale":"2023-10-06T03:46:07.791011+0000","last_undegraded":"2023-10-06T03:46:07.791011+0000","last_fullsized":"2023-10-06T03:46:07.791011+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:47.476452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817369+0000","last_change":"2023-10-06T03:46:07.812769+0000","last_active":"2023-10-06T03:46:07.817369+0000","last_peered":"2023-10-06T03:46:07.817369+0000","last_clean":"2023-10-06T03:46:07.817369+0000","last_became_active":"2023-10-06T03:46:07.812499+0000","last_became_peered":"2023-10-06T03:46:07.812499+0000","last_unstale":"2023-10-06T03:46:07.817369+0000","last_undegraded":"2023-10-06T03:46:07.817369+0000","last_fullsized":"2023-10-06T03:46:07.817369+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:21.445776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.818272+0000","last_change":"2023-10-06T03:46:06.789733+0000","last_active":"2023-10-06T03:46:07.818272+0000","last_peered":"2023-10-06T03:46:07.818272+0000","last_clean":"2023-10-06T03:46:07.818272+0000","last_became_active":"2023-10-06T03:46:06.789507+0000","last_became_peered":"2023-10-06T03:46:06.789507+0000","last_unstale":"2023-10-06T03:46:07.818272+0000","last_undegraded":"2023-10-06T03:46:07.818272+0000","last_fullsized":"2023-10-06T03:46:07.818272+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:19.914456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814272+0000","last_change":"2023-10-06T03:46:07.813250+0000","last_active":"2023-10-06T03:46:07.814272+0000","last_peered":"2023-10-06T03:46:07.814272+0000","last_clean":"2023-10-06T03:46:07.814272+0000","last_became_active":"2023-10-06T03:46:07.812947+0000","last_became_peered":"2023-10-06T03:46:07.812947+0000","last_unstale":"2023-10-06T03:46:07.814272+0000","last_undegraded":"2023-10-06T03:46:07.814272+0000","last_fullsized":"2023-10-06T03:46:07.814272+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:43.817906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.805049+0000","last_change":"2023-10-06T03:46:06.803264+0000","last_active":"2023-10-06T03:46:07.805049+0000","last_peered":"2023-10-06T03:46:07.805049+0000","last_clean":"2023-10-06T03:46:07.805049+0000","last_became_active":"2023-10-06T03:46:06.802868+0000","last_became_peered":"2023-10-06T03:46:06.802868+0000","last_unstale":"2023-10-06T03:46:07.805049+0000","last_undegraded":"2023-10-06T03:46:07.805049+0000","last_fullsized":"2023-10-06T03:46:07.805049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:42.869693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.803365+0000","last_change":"2023-10-06T03:46:07.798274+0000","last_active":"2023-10-06T03:46:07.803365+0000","last_peered":"2023-10-06T03:46:07.803365+0000","last_clean":"2023-10-06T03:46:07.803365+0000","last_became_active":"2023-10-06T03:46:07.797981+0000","last_became_peered":"2023-10-06T03:46:07.797981+0000","last_unstale":"2023-10-06T03:46:07.803365+0000","last_undegraded":"2023-10-06T03:46:07.803365+0000","last_fullsized":"2023-10-06T03:46:07.803365+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:23.659815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.791184+0000","last_change":"2023-10-06T03:46:06.811340+0000","last_active":"2023-10-06T03:46:07.791184+0000","last_peered":"2023-10-06T03:46:07.791184+0000","last_clean":"2023-10-06T03:46:07.791184+0000","last_became_active":"2023-10-06T03:46:06.811058+0000","last_became_peered":"2023-10-06T03:46:06.811058+0000","last_unstale":"2023-10-06T03:46:07.791184+0000","last_undegraded":"2023-10-06T03:46:07.791184+0000","last_fullsized":"2023-10-06T03:46:07.791184+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:24:00.975835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.822365+0000","last_change":"2023-10-06T03:46:07.820590+0000","last_active":"2023-10-06T03:46:07.822365+0000","last_peered":"2023-10-06T03:46:07.822365+0000","last_clean":"2023-10-06T03:46:07.822365+0000","last_became_active":"2023-10-06T03:46:07.820281+0000","last_became_peered":"2023-10-06T03:46:07.820281+0000","last_unstale":"2023-10-06T03:46:07.822365+0000","last_undegraded":"2023-10-06T03:46:07.822365+0000","last_fullsized":"2023-10-06T03:46:07.822365+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:19.132012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.787836+0000","last_change":"2023-10-06T03:46:06.799614+0000","last_active":"2023-10-06T03:46:07.787836+0000","last_peered":"2023-10-06T03:46:07.787836+0000","last_clean":"2023-10-06T03:46:07.787836+0000","last_became_active":"2023-10-06T03:46:06.799350+0000","last_became_peered":"2023-10-06T03:46:06.799350+0000","last_unstale":"2023-10-06T03:46:07.787836+0000","last_undegraded":"2023-10-06T03:46:07.787836+0000","last_fullsized":"2023-10-06T03:46:07.787836+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:46.103432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817404+0000","last_change":"2023-10-06T03:46:07.813338+0000","last_active":"2023-10-06T03:46:07.817404+0000","last_peered":"2023-10-06T03:46:07.817404+0000","last_clean":"2023-10-06T03:46:07.817404+0000","last_became_active":"2023-10-06T03:46:07.812987+0000","last_became_peered":"2023-10-06T03:46:07.812987+0000","last_unstale":"2023-10-06T03:46:07.817404+0000","last_undegraded":"2023-10-06T03:46:07.817404+0000","last_fullsized":"2023-10-06T03:46:07.817404+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:03.818415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.800087+0000","last_change":"2023-10-06T03:46:06.804932+0000","last_active":"2023-10-06T03:46:07.800087+0000","last_peered":"2023-10-06T03:46:07.800087+0000","last_clean":"2023-10-06T03:46:07.800087+0000","last_became_active":"2023-10-06T03:46:06.804646+0000","last_became_peered":"2023-10-06T03:46:06.804646+0000","last_unstale":"2023-10-06T03:46:07.800087+0000","last_undegraded":"2023-10-06T03:46:07.800087+0000","last_fullsized":"2023-10-06T03:46:07.800087+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:18.123015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":53,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.798980+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:46:07.798980+0000","last_peered":"2023-10-06T03:46:07.798980+0000","last_clean":"2023-10-06T03:46:07.798980+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:46:07.798980+0000","last_undegraded":"2023-10-06T03:46:07.798980+0000","last_fullsized":"2023-10-06T03:46:07.798980+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.816937+0000","last_change":"2023-10-06T03:46:07.809839+0000","last_active":"2023-10-06T03:46:07.816937+0000","last_peered":"2023-10-06T03:46:07.816937+0000","last_clean":"2023-10-06T03:46:07.816937+0000","last_became_active":"2023-10-06T03:46:07.809550+0000","last_became_peered":"2023-10-06T03:46:07.809550+0000","last_unstale":"2023-10-06T03:46:07.816937+0000","last_undegraded":"2023-10-06T03:46:07.816937+0000","last_fullsized":"2023-10-06T03:46:07.816937+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:22.619791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.805361+0000","last_change":"2023-10-06T03:46:06.810362+0000","last_active":"2023-10-06T03:46:07.805361+0000","last_peered":"2023-10-06T03:46:07.805361+0000","last_clean":"2023-10-06T03:46:07.805361+0000","last_became_active":"2023-10-06T03:46:06.810062+0000","last_became_peered":"2023-10-06T03:46:06.810062+0000","last_unstale":"2023-10-06T03:46:07.805361+0000","last_undegraded":"2023-10-06T03:46:07.805361+0000","last_fullsized":"2023-10-06T03:46:07.805361+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:02.148312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":52,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.801643+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:46:07.801643+0000","last_peered":"2023-10-06T03:46:07.801643+0000","last_clean":"2023-10-06T03:46:07.801643+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:46:07.801643+0000","last_undegraded":"2023-10-06T03:46:07.801643+0000","last_fullsized":"2023-10-06T03:46:07.801643+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.804832+0000","last_change":"2023-10-06T03:46:07.800207+0000","last_active":"2023-10-06T03:46:07.804832+0000","last_peered":"2023-10-06T03:46:07.804832+0000","last_clean":"2023-10-06T03:46:07.804832+0000","last_became_active":"2023-10-06T03:46:07.799926+0000","last_became_peered":"2023-10-06T03:46:07.799926+0000","last_unstale":"2023-10-06T03:46:07.804832+0000","last_undegraded":"2023-10-06T03:46:07.804832+0000","last_fullsized":"2023-10-06T03:46:07.804832+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:22.619791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.794176+0000","last_change":"2023-10-06T03:46:06.798222+0000","last_active":"2023-10-06T03:46:07.794176+0000","last_peered":"2023-10-06T03:46:07.794176+0000","last_clean":"2023-10-06T03:46:07.794176+0000","last_became_active":"2023-10-06T03:46:06.797962+0000","last_became_peered":"2023-10-06T03:46:06.797962+0000","last_unstale":"2023-10-06T03:46:07.794176+0000","last_undegraded":"2023-10-06T03:46:07.794176+0000","last_fullsized":"2023-10-06T03:46:07.794176+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:36.131560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":52,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.793396+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:46:07.793396+0000","last_peered":"2023-10-06T03:46:07.793396+0000","last_clean":"2023-10-06T03:46:07.793396+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:46:07.793396+0000","last_undegraded":"2023-10-06T03:46:07.793396+0000","last_fullsized":"2023-10-06T03:46:07.793396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'139","reported_seq":280,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.793125+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:46:07.793125+0000","last_peered":"2023-10-06T03:46:07.793125+0000","last_clean":"2023-10-06T03:46:07.793125+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:46:07.793125+0000","last_undegraded":"2023-10-06T03:46:07.793125+0000","last_fullsized":"2023-10-06T03:46:07.793125+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.821126+0000","last_change":"2023-10-06T03:46:07.815733+0000","last_active":"2023-10-06T03:46:07.821126+0000","last_peered":"2023-10-06T03:46:07.821126+0000","last_clean":"2023-10-06T03:46:07.821126+0000","last_became_active":"2023-10-06T03:46:07.815479+0000","last_became_peered":"2023-10-06T03:46:07.815479+0000","last_unstale":"2023-10-06T03:46:07.821126+0000","last_undegraded":"2023-10-06T03:46:07.821126+0000","last_fullsized":"2023-10-06T03:46:07.821126+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:35.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.792341+0000","last_change":"2023-10-06T03:46:06.797846+0000","last_active":"2023-10-06T03:46:07.792341+0000","last_peered":"2023-10-06T03:46:07.792341+0000","last_clean":"2023-10-06T03:46:07.792341+0000","last_became_active":"2023-10-06T03:46:06.797563+0000","last_became_peered":"2023-10-06T03:46:06.797563+0000","last_unstale":"2023-10-06T03:46:07.792341+0000","last_undegraded":"2023-10-06T03:46:07.792341+0000","last_fullsized":"2023-10-06T03:46:07.792341+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:21.329719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":53,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.793545+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:46:07.793545+0000","last_peered":"2023-10-06T03:46:07.793545+0000","last_clean":"2023-10-06T03:46:07.793545+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:46:07.793545+0000","last_undegraded":"2023-10-06T03:46:07.793545+0000","last_fullsized":"2023-10-06T03:46:07.793545+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814121+0000","last_change":"2023-10-06T03:46:07.803828+0000","last_active":"2023-10-06T03:46:07.814121+0000","last_peered":"2023-10-06T03:46:07.814121+0000","last_clean":"2023-10-06T03:46:07.814121+0000","last_became_active":"2023-10-06T03:46:07.803446+0000","last_became_peered":"2023-10-06T03:46:07.803446+0000","last_unstale":"2023-10-06T03:46:07.814121+0000","last_undegraded":"2023-10-06T03:46:07.814121+0000","last_fullsized":"2023-10-06T03:46:07.814121+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:52.882135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.797130+0000","last_change":"2023-10-06T03:46:06.810579+0000","last_active":"2023-10-06T03:46:07.797130+0000","last_peered":"2023-10-06T03:46:07.797130+0000","last_clean":"2023-10-06T03:46:07.797130+0000","last_became_active":"2023-10-06T03:46:06.808824+0000","last_became_peered":"2023-10-06T03:46:06.808824+0000","last_unstale":"2023-10-06T03:46:07.797130+0000","last_undegraded":"2023-10-06T03:46:07.797130+0000","last_fullsized":"2023-10-06T03:46:07.797130+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:08.478426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.793042+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:46:07.793042+0000","last_peered":"2023-10-06T03:46:07.793042+0000","last_clean":"2023-10-06T03:46:07.793042+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:46:07.793042+0000","last_undegraded":"2023-10-06T03:46:07.793042+0000","last_fullsized":"2023-10-06T03:46:07.793042+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817190+0000","last_change":"2023-10-06T03:46:07.812134+0000","last_active":"2023-10-06T03:46:07.817190+0000","last_peered":"2023-10-06T03:46:07.817190+0000","last_clean":"2023-10-06T03:46:07.817190+0000","last_became_active":"2023-10-06T03:46:07.811727+0000","last_became_peered":"2023-10-06T03:46:07.811727+0000","last_unstale":"2023-10-06T03:46:07.817190+0000","last_undegraded":"2023-10-06T03:46:07.817190+0000","last_fullsized":"2023-10-06T03:46:07.817190+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:37.140556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.820756+0000","last_change":"2023-10-06T03:46:06.812006+0000","last_active":"2023-10-06T03:46:07.820756+0000","last_peered":"2023-10-06T03:46:07.820756+0000","last_clean":"2023-10-06T03:46:07.820756+0000","last_became_active":"2023-10-06T03:46:06.811704+0000","last_became_peered":"2023-10-06T03:46:06.811704+0000","last_unstale":"2023-10-06T03:46:07.820756+0000","last_undegraded":"2023-10-06T03:46:07.820756+0000","last_fullsized":"2023-10-06T03:46:07.820756+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:10.219880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.796727+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:46:07.796727+0000","last_peered":"2023-10-06T03:46:07.796727+0000","last_clean":"2023-10-06T03:46:07.796727+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:46:07.796727+0000","last_undegraded":"2023-10-06T03:46:07.796727+0000","last_fullsized":"2023-10-06T03:46:07.796727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.820029+0000","last_change":"2023-10-06T03:46:07.812090+0000","last_active":"2023-10-06T03:46:07.820029+0000","last_peered":"2023-10-06T03:46:07.820029+0000","last_clean":"2023-10-06T03:46:07.820029+0000","last_became_active":"2023-10-06T03:46:07.811801+0000","last_became_peered":"2023-10-06T03:46:07.811801+0000","last_unstale":"2023-10-06T03:46:07.820029+0000","last_undegraded":"2023-10-06T03:46:07.820029+0000","last_fullsized":"2023-10-06T03:46:07.820029+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:28.197821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.797208+0000","last_change":"2023-10-06T03:46:06.793867+0000","last_active":"2023-10-06T03:46:07.797208+0000","last_peered":"2023-10-06T03:46:07.797208+0000","last_clean":"2023-10-06T03:46:07.797208+0000","last_became_active":"2023-10-06T03:46:06.793571+0000","last_became_peered":"2023-10-06T03:46:06.793571+0000","last_unstale":"2023-10-06T03:46:07.797208+0000","last_undegraded":"2023-10-06T03:46:07.797208+0000","last_fullsized":"2023-10-06T03:46:07.797208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:42.060159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.790374+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:46:07.790374+0000","last_peered":"2023-10-06T03:46:07.790374+0000","last_clean":"2023-10-06T03:46:07.790374+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:46:07.790374+0000","last_undegraded":"2023-10-06T03:46:07.790374+0000","last_fullsized":"2023-10-06T03:46:07.790374+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.808056+0000","last_change":"2023-10-06T03:46:07.803252+0000","last_active":"2023-10-06T03:46:07.808056+0000","last_peered":"2023-10-06T03:46:07.808056+0000","last_clean":"2023-10-06T03:46:07.808056+0000","last_became_active":"2023-10-06T03:46:07.802963+0000","last_became_peered":"2023-10-06T03:46:07.802963+0000","last_unstale":"2023-10-06T03:46:07.808056+0000","last_undegraded":"2023-10-06T03:46:07.808056+0000","last_fullsized":"2023-10-06T03:46:07.808056+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:41.541412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.797591+0000","last_change":"2023-10-06T03:46:06.803826+0000","last_active":"2023-10-06T03:46:07.797591+0000","last_peered":"2023-10-06T03:46:07.797591+0000","last_clean":"2023-10-06T03:46:07.797591+0000","last_became_active":"2023-10-06T03:46:06.803582+0000","last_became_peered":"2023-10-06T03:46:06.803582+0000","last_unstale":"2023-10-06T03:46:07.797591+0000","last_undegraded":"2023-10-06T03:46:07.797591+0000","last_fullsized":"2023-10-06T03:46:07.797591+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:21.329719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.790948+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:46:07.790948+0000","last_peered":"2023-10-06T03:46:07.790948+0000","last_clean":"2023-10-06T03:46:07.790948+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:46:07.790948+0000","last_undegraded":"2023-10-06T03:46:07.790948+0000","last_fullsized":"2023-10-06T03:46:07.790948+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575005,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310904,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060936,"statfs":{"total":96636764160,"available":96318398464,"internally_reserved":0,"allocated":1359872,"data_stored":965756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71499999999999997,"15min":0.68000000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.34000000000000002},"max":{"1min":1.0029999999999999,"5min":1.004,"15min":1.6100000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72699999999999998,"15min":0.70699999999999996},"min":{"1min":0.501,"5min":0.47999999999999998,"15min":0.47499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.5649999999999999},"last":0.624}]},{"osd":1,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77600000000000002,"15min":0.73999999999999999},"min":{"1min":0.56499999999999995,"5min":0.434,"15min":0.42199999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.5469999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78200000000000003,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.036,"5min":1.036,"15min":1.0580000000000001},"last":0.745}]},{"osd":2,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71099999999999997,"15min":0.68799999999999994},"min":{"1min":0.54400000000000004,"5min":0.49299999999999999,"15min":0.46500000000000002},"max":{"1min":0.90300000000000002,"5min":0.93000000000000005,"15min":1.5900000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.749,"15min":0.73399999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.501},"max":{"1min":0.97599999999999998,"5min":0.97699999999999998,"15min":1.044},"last":0.495}]},{"osd":3,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73699999999999999,"15min":0.71999999999999997},"min":{"1min":0.58199999999999996,"5min":0.45900000000000002,"15min":0.39500000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.6279999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75600000000000001,"15min":0.73899999999999999},"min":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.46600000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.0409999999999999},"last":0.75800000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.745,"5min":0.86599999999999999,"15min":1.0049999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.60499999999999998,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.40999999999999998,"15min":0.39400000000000002},"max":{"1min":0.78800000000000003,"5min":0.86099999999999999,"15min":0.95699999999999996},"last":0.59999999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.66000000000000003},"min":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.36799999999999999},"max":{"1min":0.93799999999999994,"5min":0.96999999999999997,"15min":1.069},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.69599999999999995},"min":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.436},"max":{"1min":0.88200000000000001,"5min":0.91900000000000004,"15min":1.044},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71099999999999997,"15min":0.73499999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.996},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69999999999999996,"15min":0.71699999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.0209999999999999},"last":0.72599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575006,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302136,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069704,"statfs":{"total":96636764160,"available":96327376896,"internally_reserved":0,"allocated":770048,"data_stored":376866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71599999999999997,"15min":0.68700000000000006},"min":{"1min":0.56100000000000005,"5min":0.38800000000000001,"15min":0.313},"max":{"1min":0.86599999999999999,"5min":0.95799999999999996,"15min":0.98299999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73699999999999999,"15min":0.70699999999999996},"min":{"1min":0.625,"5min":0.437,"15min":0.39500000000000002},"max":{"1min":0.98599999999999999,"5min":1.0069999999999999,"15min":1.0449999999999999},"last":0.94899999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.755,"15min":0.71999999999999997},"min":{"1min":0.58299999999999996,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96799999999999997,"5min":0.98199999999999998,"15min":1.3180000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72799999999999998,"15min":0.70399999999999996},"min":{"1min":0.64900000000000002,"5min":0.378,"15min":0.35899999999999999},"max":{"1min":0.85399999999999998,"5min":0.995,"15min":0.995},"last":0.90900000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.624,"5min":0.48199999999999998,"15min":0.46000000000000002},"max":{"1min":0.92600000000000005,"5min":0.96399999999999997,"15min":3.3319999999999999},"last":0.995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.75900000000000001,"15min":0.77700000000000002},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.97799999999999998,"5min":1.0409999999999999,"15min":3.5139999999999998},"last":0.748}]},{"osd":3,"last update":"Fri Oct 6 03:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76800000000000002,"15min":0.745},"min":{"1min":0.624,"5min":0.45500000000000002,"15min":0.44600000000000001},"max":{"1min":0.92200000000000004,"5min":1.0109999999999999,"15min":1.024},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78000000000000003,"15min":0.80400000000000005},"min":{"1min":0.66800000000000004,"5min":0.55000000000000004,"15min":0.45200000000000001},"max":{"1min":0.93999999999999995,"5min":1.03,"15min":3.3300000000000001},"last":0.79500000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56899999999999995,"15min":0.60799999999999998},"min":{"1min":0.44400000000000001,"5min":0.376,"15min":0.34799999999999998},"max":{"1min":0.70699999999999996,"5min":0.92600000000000005,"15min":1.0680000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63900000000000001,"15min":0.66000000000000003},"min":{"1min":0.50600000000000001,"5min":0.434,"15min":0.36499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":1.2929999999999999},"last":0.88300000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.69099999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41099999999999998},"max":{"1min":0.85999999999999999,"5min":0.95599999999999996,"15min":1.3340000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.54600000000000004,"5min":0.42899999999999999,"15min":0.41199999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":1.2370000000000001},"last":1.0269999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.72699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92300000000000004,"5min":0.94399999999999995,"15min":1.619},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70599999999999996,"15min":0.73299999999999998},"min":{"1min":0.51000000000000001,"5min":0.48299999999999998,"15min":0.47099999999999997},"max":{"1min":0.89000000000000001,"5min":0.98299999999999998,"15min":1.417},"last":1.0509999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302716,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069124,"statfs":{"total":96636764160,"available":96326782976,"internally_reserved":0,"allocated":1363968,"data_stored":966770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.5,"15min":0.48399999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.73599999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.37},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35599999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.2889999999999999},"last":0.438}]},{"osd":2,"last update":"Fri Oct 6 03:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50700000000000001,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.32700000000000001,"15min":0.26100000000000001},"max":{"1min":0.67200000000000004,"5min":0.84799999999999998,"15min":0.999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70599999999999996,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.42399999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.59699999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82099999999999995,"5min":0.82699999999999996,"15min":0.93999999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.54700000000000004},"min":{"1min":0.40899999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":1.254},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76700000000000002,"15min":0.753},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.034,"5min":1.1659999999999999,"15min":1.3600000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82399999999999995,"15min":0.79200000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.2,"5min":1.2,"15min":1.476},"last":0.59199999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80800000000000005,"15min":0.79500000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.45400000000000001},"max":{"1min":0.996,"5min":1.0640000000000001,"15min":1.601},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81399999999999995,"15min":0.79600000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.081,"5min":1.159,"15min":1.5189999999999999},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.81799999999999995,"15min":0.81200000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0609999999999999,"5min":1.1830000000000001,"15min":1.244},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.83799999999999997,"15min":0.81200000000000006},"min":{"1min":0.57299999999999995,"5min":0.53200000000000003,"15min":0.505},"max":{"1min":1.137,"5min":1.224,"15min":1.399},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.85999999999999999,"15min":0.82899999999999996},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.56599999999999995},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.84799999999999998,"15min":0.82699999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.51200000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.556},"last":0.622}]}]},{"osd":0,"up_from":13,"seq":55834575007,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306228,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065612,"statfs":{"total":96636764160,"available":96323186688,"internally_reserved":0,"allocated":765952,"data_stored":375320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.48499999999999999,"15min":0.50600000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":2.3220000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.34499999999999997,"5min":0.33600000000000002,"15min":0.29599999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":2.1440000000000001},"last":0.35399999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53900000000000003,"15min":0.58199999999999996},"min":{"1min":0.379,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77000000000000002,"5min":0.77700000000000002,"15min":2.4329999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69099999999999995,"5min":0.84699999999999998,"15min":2.1819999999999999},"last":0.42999999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.51700000000000002,"15min":0.53700000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.28399999999999997},"max":{"1min":0.61199999999999999,"5min":0.71799999999999997,"15min":2.2709999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60799999999999998,"15min":0.63800000000000001},"min":{"1min":0.434,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":2.4700000000000002},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.83099999999999996},"min":{"1min":0.55200000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97999999999999998,"5min":1.1259999999999999,"15min":1.393},"last":0.878},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0189999999999999,"5min":1.262,"15min":2.2320000000000002},"last":0.90700000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.81899999999999995,"15min":0.83999999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.054,"5min":1.1499999999999999,"15min":2.3660000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.82599999999999996,"15min":0.80700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.503},"max":{"1min":1.0800000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.92200000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.81499999999999995,"15min":0.80200000000000005},"min":{"1min":0.53100000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0349999999999999,"5min":1.726,"15min":1.726},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.85499999999999998,"15min":0.86899999999999999},"min":{"1min":0.60399999999999998,"5min":0.57899999999999996,"15min":0.56999999999999995},"max":{"1min":0.996,"5min":1.843,"15min":2.4009999999999998},"last":0.78200000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.84499999999999997,"15min":0.82099999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.48499999999999999},"max":{"1min":1.036,"5min":1.1850000000000001,"15min":1.2689999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.069,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.94999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306232,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065608,"statfs":{"total":96636764160,"available":96323182592,"internally_reserved":0,"allocated":770048,"data_stored":376800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.49299999999999999,"15min":0.45800000000000002},"min":{"1min":0.41199999999999998,"5min":0.311,"15min":0.28000000000000003},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.439},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54100000000000004,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.83599999999999997,"5min":0.86199999999999999,"15min":0.89300000000000002},"last":0.52700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48399999999999999,"15min":0.48699999999999999},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.629,"5min":0.79900000000000004,"15min":0.90000000000000002},"last":0.503},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.38600000000000001,"5min":0.38,"15min":0.34200000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":1.556},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.44900000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76300000000000001,"5min":0.98699999999999999,"15min":1.6040000000000001},"last":0.624},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.56799999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68799999999999994,"5min":0.91400000000000003,"15min":1.1910000000000001},"last":0.49099999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82899999999999996,"15min":0.80500000000000005},"min":{"1min":0.67100000000000004,"5min":0.56999999999999995,"15min":0.439},"max":{"1min":1.0149999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84799999999999998,"15min":0.82799999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55600000000000005},"max":{"1min":1.1279999999999999,"5min":1.1419999999999999,"15min":1.2270000000000001},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83099999999999996,"15min":0.82099999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.52200000000000002},"max":{"1min":1.169,"5min":1.177,"15min":1.3},"last":0.753},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.85599999999999998,"15min":0.84099999999999997},"min":{"1min":0.624,"5min":0.61299999999999999,"15min":0.51700000000000002},"max":{"1min":1.1339999999999999,"5min":1.1779999999999999,"15min":1.3069999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.83099999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.53600000000000003},"max":{"1min":1.111,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83299999999999996,"15min":0.80400000000000005},"min":{"1min":0.60599999999999998,"5min":0.59199999999999997,"15min":0.51100000000000001},"max":{"1min":1.0649999999999999,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.60999999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.85799999999999998,"15min":0.82399999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.54900000000000004},"max":{"1min":1.4379999999999999,"5min":1.4570000000000001,"15min":1.4990000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.879,"15min":0.85499999999999998},"min":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.497},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.67200000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575006,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306240,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065600,"statfs":{"total":96636764160,"available":96323174400,"internally_reserved":0,"allocated":778240,"data_stored":377800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47099999999999997,"15min":0.46500000000000002},"min":{"1min":0.33900000000000002,"5min":0.30099999999999999,"15min":0.28699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.5,"15min":0.48899999999999999},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":1.1699999999999999},"last":0.439}]},{"osd":1,"last update":"Fri Oct 6 03:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.48499999999999999,"15min":0.46500000000000002},"min":{"1min":0.35599999999999998,"5min":0.316,"15min":0.29299999999999998},"max":{"1min":0.66100000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54500000000000004,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.38300000000000001,"15min":0.33800000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.83899999999999997},"last":0.54500000000000004}]},{"osd":2,"last update":"Fri Oct 6 03:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.55400000000000005},"min":{"1min":0.42299999999999999,"5min":0.41799999999999998,"15min":0.34599999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.85599999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.61099999999999999,"15min":0.59699999999999998},"min":{"1min":0.47099999999999997,"5min":0.46000000000000002,"15min":0.38700000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.97899999999999998},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78600000000000003,"15min":0.79400000000000004},"min":{"1min":0.57399999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.028,"5min":1.2030000000000001,"15min":1.4099999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84299999999999997,"15min":0.82999999999999996},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.51300000000000001},"max":{"1min":0.96999999999999997,"5min":1.1859999999999999,"15min":1.377},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 03:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80800000000000005,"15min":0.78800000000000003},"min":{"1min":0.58899999999999997,"5min":0.54300000000000004,"15min":0.47099999999999997},"max":{"1min":1.05,"5min":1.256,"15min":1.256},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.48099999999999998},"max":{"1min":0.97899999999999998,"5min":1.1439999999999999,"15min":1.306},"last":0.69799999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82899999999999996,"15min":0.81299999999999994},"min":{"1min":0.58199999999999996,"5min":0.57099999999999995,"15min":0.48599999999999999},"max":{"1min":1.089,"5min":1.292,"15min":1.323},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.49299999999999999},"max":{"1min":1.1040000000000001,"5min":1.248,"15min":1.6020000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82799999999999996,"15min":0.81899999999999995},"min":{"1min":0.63200000000000001,"5min":0.58499999999999996,"15min":0.505},"max":{"1min":1.0700000000000001,"5min":1.2370000000000001,"15min":1.365},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81000000000000005,"15min":0.80900000000000005},"min":{"1min":0.61899999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.96999999999999997,"5min":1.1180000000000001,"15min":1.2330000000000001},"last":0.57299999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302132,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069708,"statfs":{"total":96636764160,"available":96327380992,"internally_reserved":0,"allocated":765952,"data_stored":376796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.41799999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.89200000000000002,"5min":1.0109999999999999,"15min":1.3120000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.83999999999999997,"5min":0.96399999999999997,"15min":1.01},"last":0.70999999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.74399999999999999,"15min":0.72499999999999998},"min":{"1min":0.35899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96499999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.747,"15min":0.74199999999999999},"min":{"1min":0.49299999999999999,"5min":0.44,"15min":0.41299999999999998},"max":{"1min":0.94999999999999996,"5min":0.96999999999999997,"15min":1.3380000000000001},"last":0.56200000000000006}]},{"osd":2,"last update":"Fri Oct 6 03:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.747,"15min":0.73799999999999999},"min":{"1min":0.44500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.028,"5min":1.028,"15min":1.1759999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.56999999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90200000000000002,"5min":1.022,"15min":1.3089999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.72799999999999998,"15min":0.72599999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94899999999999995,"5min":1.0049999999999999,"15min":1.0409999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77300000000000002,"15min":0.76900000000000002},"min":{"1min":0.58199999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.995,"5min":0.995,"15min":1.1259999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.62,"15min":0.64200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.085},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.623,"15min":0.64200000000000002},"min":{"1min":0.44700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91900000000000004,"5min":0.95399999999999996,"15min":1.1439999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.66000000000000003},"min":{"1min":0.44700000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90800000000000003,"5min":1.018,"15min":1.0880000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67000000000000004,"15min":0.70999999999999996},"min":{"1min":0.48799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":1.361},"last":0.74099999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64700000000000002,"15min":0.69499999999999995},"min":{"1min":0.46500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89400000000000002,"5min":0.92400000000000004,"15min":1.1950000000000001},"last":0.751},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.72599999999999998},"min":{"1min":0.51800000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0760000000000001,"5min":1.115,"15min":1.1439999999999999},"last":0.73099999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575006,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298028,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073812,"statfs":{"total":96636764160,"available":96331583488,"internally_reserved":0,"allocated":757760,"data_stored":375264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70399999999999996,"15min":0.69899999999999995},"min":{"1min":0.41499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88900000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.71299999999999997,"15min":0.69299999999999995},"min":{"1min":0.44600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82699999999999996,"5min":0.96899999999999997,"15min":1},"last":0.626}]},{"osd":1,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70999999999999996,"15min":0.70099999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86199999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.47199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82899999999999996,"5min":0.94799999999999995,"15min":0.95799999999999996},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77000000000000002,"15min":0.75700000000000001},"min":{"1min":0.502,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97299999999999998,"5min":1.02,"15min":1.022},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.76600000000000001,"15min":0.77600000000000002},"min":{"1min":0.55000000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91500000000000004,"5min":1.006,"15min":1.006},"last":0.61399999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.505,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.871,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.74299999999999999,"15min":0.74399999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.85399999999999998,"5min":0.999,"15min":0.999},"last":0.67500000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.63},"min":{"1min":0.40400000000000003,"5min":0.40000000000000002,"15min":0.38300000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.024},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60199999999999998,"15min":0.64700000000000002},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.36799999999999999},"max":{"1min":0.749,"5min":0.93999999999999995,"15min":1.175},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61899999999999999,"15min":0.66100000000000003},"min":{"1min":0.46800000000000003,"5min":0.38500000000000001,"15min":0.34999999999999998},"max":{"1min":0.77900000000000003,"5min":0.92700000000000005,"15min":1.1180000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67500000000000004,"15min":0.70899999999999996},"min":{"1min":0.504,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89000000000000001,"5min":0.96399999999999997,"15min":1.0489999999999999},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65100000000000002,"15min":0.68899999999999995},"min":{"1min":0.496,"5min":0.40899999999999997,"15min":0.40699999999999997},"max":{"1min":0.77200000000000002,"5min":1.016,"15min":1.2589999999999999},"last":0.746},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69599999999999995,"15min":0.73299999999999998},"min":{"1min":0.51300000000000001,"5min":0.436,"15min":0.435},"max":{"1min":0.88500000000000001,"5min":0.97599999999999998,"15min":1.1020000000000001},"last":0.70899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:46:12.197 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:46:12.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.412+0000 7f3b1099d700 1 Processor -- start 2023-10-06T03:46:12.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.412+0000 7f3b1099d700 1 -- start start 2023-10-06T03:46:12.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.414+0000 7f3b1099d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3b0c159420 0x7f3b0c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:12.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.414+0000 7f3b1099d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b0c1597f0 0x7f3b0c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.414+0000 7f3b1099d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0c0b11d0 con 0x7f3b0c15d230 2023-10-06T03:46:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.414+0000 7f3b1099d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0c0b14b0 con 0x7f3b0c159420 2023-10-06T03:46:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.414+0000 7f3b1099d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c0b1790 con 0x7f3b0c1597f0 2023-10-06T03:46:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.414+0000 7f3b0b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3b0c159420 0x7f3b0c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.414+0000 7f3b0b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3b0c159420 0x7f3b0c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50266/0 (socket says 172.21.15.55:50266) 2023-10-06T03:46:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.414+0000 7f3b0b7fe700 1 -- 172.21.15.55:0/1249063594 learned_addr learned my addr 172.21.15.55:0/1249063594 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.414+0000 7f3b0affd700 1 --2- 172.21.15.55:0/1249063594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b0c1597f0 0x7f3b0c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.414+0000 7f3b0b7fe700 1 -- 172.21.15.55:0/1249063594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b0c1597f0 msgr2=0x7f3b0c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.414+0000 7f3b0b7fe700 1 --2- 172.21.15.55:0/1249063594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b0c1597f0 0x7f3b0c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.414+0000 7f3b0b7fe700 1 -- 172.21.15.55:0/1249063594 >> v1:172.21.15.163:6789/0 conn(0x7f3b0c15d230 legacy=0x7f3b0c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:12.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.414+0000 7f3b0b7fe700 1 -- 172.21.15.55:0/1249063594 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c0b1a70 con 0x7f3b0c159420 2023-10-06T03:46:12.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.414+0000 7f3b0b7fe700 1 --2- 172.21.15.55:0/1249063594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b0c159420 0x7f3b0c0b5f40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0099b0 tx=0x7f3afc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0aea210efa4776 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:12.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.415+0000 7f3b0a7fc700 1 -- 172.21.15.55:0/1249063594 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc009ea0 con 0x7f3b0c159420 2023-10-06T03:46:12.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.415+0000 7f3b0a7fc700 1 -- 172.21.15.55:0/1249063594 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc00b860 con 0x7f3b0c159420 2023-10-06T03:46:12.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.415+0000 7f3b0a7fc700 1 -- 172.21.15.55:0/1249063594 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc00bc00 con 0x7f3b0c159420 2023-10-06T03:46:12.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.415+0000 7f3b1099d700 1 -- 172.21.15.55:0/1249063594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b0c159420 msgr2=0x7f3b0c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:12.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.415+0000 7f3b1099d700 1 --2- 172.21.15.55:0/1249063594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b0c159420 0x7f3b0c0b5f40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0099b0 tx=0x7f3afc009250 comp rx=0 tx=0).stop 2023-10-06T03:46:12.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.415+0000 7f3b1099d700 1 -- 172.21.15.55:0/1249063594 shutdown_connections 2023-10-06T03:46:12.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.415+0000 7f3b1099d700 1 --2- 172.21.15.55:0/1249063594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b0c159420 0x7f3b0c0b5f40 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.415+0000 7f3b1099d700 1 --2- 172.21.15.55:0/1249063594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b0c1597f0 0x7f3b0c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.415+0000 7f3b1099d700 1 -- 172.21.15.55:0/1249063594 >> 172.21.15.55:0/1249063594 conn(0x7f3b0c0d0660 msgr2=0x7f3b0c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.415+0000 7f3b1099d700 1 -- 172.21.15.55:0/1249063594 shutdown_connections 2023-10-06T03:46:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.415+0000 7f3b1099d700 1 -- 172.21.15.55:0/1249063594 wait complete. 2023-10-06T03:46:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.417+0000 7f3b1099d700 1 Processor -- start 2023-10-06T03:46:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.417+0000 7f3b1099d700 1 -- start start 2023-10-06T03:46:12.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b1099d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b0c159420 0x7f3b0c0cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:12.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b1099d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b0c1597f0 0x7f3b0c0cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:12.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b1099d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b0c15d230 0x7f3b0c0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:12.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b1099d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c153fa0 con 0x7f3b0c159420 2023-10-06T03:46:12.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b1099d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c0cff00 con 0x7f3b0c1597f0 2023-10-06T03:46:12.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b1099d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c0d01e0 con 0x7f3b0c15d230 2023-10-06T03:46:12.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b0b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b0c159420 0x7f3b0c0cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:12.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b0b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b0c159420 0x7f3b0c0cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50280/0 (socket says 172.21.15.55:50280) 2023-10-06T03:46:12.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b0b7fe700 1 -- 172.21.15.55:0/3882611059 learned_addr learned my addr 172.21.15.55:0/3882611059 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:12.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b0affd700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b0c1597f0 0x7f3b0c0cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:12.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b0bfff700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b0c15d230 0x7f3b0c0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:12.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b0b7fe700 1 -- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b0c15d230 msgr2=0x7f3b0c0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b0b7fe700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b0c15d230 0x7f3b0c0cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b0b7fe700 1 -- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b0c1597f0 msgr2=0x7f3b0c0cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b0b7fe700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b0c1597f0 0x7f3b0c0cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b0b7fe700 1 -- 172.21.15.55:0/3882611059 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3afc0094f0 con 0x7f3b0c159420 2023-10-06T03:46:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b0affd700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b0c1597f0 0x7f3b0c0cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:46:12.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b0b7fe700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b0c159420 0x7f3b0c0cbe00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c089340 tx=0x7f3afc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3af37fe700 1 -- 172.21.15.55:0/3882611059 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc00bbe0 con 0x7f3b0c159420 2023-10-06T03:46:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3af37fe700 1 -- 172.21.15.55:0/3882611059 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3afc011420 con 0x7f3b0c159420 2023-10-06T03:46:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3af37fe700 1 -- 172.21.15.55:0/3882611059 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0117c0 con 0x7f3b0c159420 2023-10-06T03:46:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b1099d700 1 -- 172.21.15.55:0/3882611059 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c15a620 con 0x7f3b0c159420 2023-10-06T03:46:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.419+0000 7f3b1099d700 1 -- 172.21.15.55:0/3882611059 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c15abe0 con 0x7f3b0c159420 2023-10-06T03:46:12.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.420+0000 7f3b1099d700 1 -- 172.21.15.55:0/3882611059 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0c0d03a0 con 0x7f3b0c159420 2023-10-06T03:46:12.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.422+0000 7f3af37fe700 1 -- 172.21.15.55:0/3882611059 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3afc011bf0 con 0x7f3b0c159420 2023-10-06T03:46:12.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.423+0000 7f3af37fe700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3af8066f30 0x7f3af80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:12.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.423+0000 7f3af37fe700 1 -- 172.21.15.55:0/3882611059 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f3afc086230 con 0x7f3b0c159420 2023-10-06T03:46:12.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.423+0000 7f3b0affd700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3af8066f30 0x7f3af80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:12.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.423+0000 7f3b0affd700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3af8066f30 0x7f3af80693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c1547a0 tx=0x7f3b00008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:12.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.427+0000 7f3af37fe700 1 -- 172.21.15.55:0/3882611059 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3afc053130 con 0x7f3b0c159420 2023-10-06T03:46:12.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.623+0000 7f3b1099d700 1 -- 172.21.15.55:0/3882611059 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b0c04c3c0 con 0x7f3af8066f30 2023-10-06T03:46:12.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.633+0000 7f3af37fe700 1 -- 172.21.15.55:0/3882611059 <== mgr.5452 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311497 (secure 0 0 0) 0x7f3b0c04c3c0 con 0x7f3af8066f30 2023-10-06T03:46:12.639 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:46:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.637+0000 7f3b1099d700 1 -- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3af8066f30 msgr2=0x7f3af80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.637+0000 7f3b1099d700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3af8066f30 0x7f3af80693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c1547a0 tx=0x7f3b00008040 comp rx=0 tx=0).stop 2023-10-06T03:46:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.637+0000 7f3b1099d700 1 -- 172.21.15.55:0/3882611059 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b0c159420 msgr2=0x7f3b0c0cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.637+0000 7f3b1099d700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b0c159420 0x7f3b0c0cbe00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c089340 tx=0x7f3afc017000 comp rx=0 tx=0).stop 2023-10-06T03:46:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.637+0000 7f3b1099d700 1 -- 172.21.15.55:0/3882611059 shutdown_connections 2023-10-06T03:46:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.637+0000 7f3b1099d700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3af8066f30 0x7f3af80693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.637+0000 7f3b1099d700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b0c15d230 0x7f3b0c0cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.637+0000 7f3b1099d700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b0c159420 0x7f3b0c0cbe00 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.637+0000 7f3b1099d700 1 --2- 172.21.15.55:0/3882611059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b0c1597f0 0x7f3b0c0cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.637+0000 7f3b1099d700 1 -- 172.21.15.55:0/3882611059 >> 172.21.15.55:0/3882611059 conn(0x7f3b0c0d0660 msgr2=0x7f3b0c0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:12.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.637+0000 7f3b1099d700 1 -- 172.21.15.55:0/3882611059 shutdown_connections 2023-10-06T03:46:12.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.637+0000 7f3b1099d700 1 -- 172.21.15.55:0/3882611059 wait complete. 2023-10-06T03:46:12.643 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:46:12.658 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T03:46:11.939059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":142,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2434616,"kb_used_data":7160,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752540104,"statfs":{"total":773094113280,"available":770601066496,"internally_reserved":0,"allocated":7331840,"data_stored":4191372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001507"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.806717+0000","last_change":"2023-10-06T03:46:07.804657+0000","last_active":"2023-10-06T03:46:07.806717+0000","last_peered":"2023-10-06T03:46:07.806717+0000","last_clean":"2023-10-06T03:46:07.806717+0000","last_became_active":"2023-10-06T03:46:07.804382+0000","last_became_peered":"2023-10-06T03:46:07.804382+0000","last_unstale":"2023-10-06T03:46:07.806717+0000","last_undegraded":"2023-10-06T03:46:07.806717+0000","last_fullsized":"2023-10-06T03:46:07.806717+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:59.481973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.815654+0000","last_change":"2023-10-06T03:46:07.808232+0000","last_active":"2023-10-06T03:46:07.815654+0000","last_peered":"2023-10-06T03:46:07.815654+0000","last_clean":"2023-10-06T03:46:07.815654+0000","last_became_active":"2023-10-06T03:46:07.807987+0000","last_became_peered":"2023-10-06T03:46:07.807987+0000","last_unstale":"2023-10-06T03:46:07.815654+0000","last_undegraded":"2023-10-06T03:46:07.815654+0000","last_fullsized":"2023-10-06T03:46:07.815654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:22.338715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.819136+0000","last_change":"2023-10-06T03:46:07.814899+0000","last_active":"2023-10-06T03:46:07.819136+0000","last_peered":"2023-10-06T03:46:07.819136+0000","last_clean":"2023-10-06T03:46:07.819136+0000","last_became_active":"2023-10-06T03:46:07.814574+0000","last_became_peered":"2023-10-06T03:46:07.814574+0000","last_unstale":"2023-10-06T03:46:07.819136+0000","last_undegraded":"2023-10-06T03:46:07.819136+0000","last_fullsized":"2023-10-06T03:46:07.819136+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:24.995759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814858+0000","last_change":"2023-10-06T03:46:07.809583+0000","last_active":"2023-10-06T03:46:07.814858+0000","last_peered":"2023-10-06T03:46:07.814858+0000","last_clean":"2023-10-06T03:46:07.814858+0000","last_became_active":"2023-10-06T03:46:07.809323+0000","last_became_peered":"2023-10-06T03:46:07.809323+0000","last_unstale":"2023-10-06T03:46:07.814858+0000","last_undegraded":"2023-10-06T03:46:07.814858+0000","last_fullsized":"2023-10-06T03:46:07.814858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:35.844726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.816447+0000","last_change":"2023-10-06T03:46:07.814855+0000","last_active":"2023-10-06T03:46:07.816447+0000","last_peered":"2023-10-06T03:46:07.816447+0000","last_clean":"2023-10-06T03:46:07.816447+0000","last_became_active":"2023-10-06T03:46:07.814589+0000","last_became_peered":"2023-10-06T03:46:07.814589+0000","last_unstale":"2023-10-06T03:46:07.816447+0000","last_undegraded":"2023-10-06T03:46:07.816447+0000","last_fullsized":"2023-10-06T03:46:07.816447+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:32:43.430204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.808111+0000","last_change":"2023-10-06T03:46:07.805282+0000","last_active":"2023-10-06T03:46:07.808111+0000","last_peered":"2023-10-06T03:46:07.808111+0000","last_clean":"2023-10-06T03:46:07.808111+0000","last_became_active":"2023-10-06T03:46:07.805000+0000","last_became_peered":"2023-10-06T03:46:07.805000+0000","last_unstale":"2023-10-06T03:46:07.808111+0000","last_undegraded":"2023-10-06T03:46:07.808111+0000","last_fullsized":"2023-10-06T03:46:07.808111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:21.445776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.811405+0000","last_change":"2023-10-06T03:46:07.803595+0000","last_active":"2023-10-06T03:46:07.811405+0000","last_peered":"2023-10-06T03:46:07.811405+0000","last_clean":"2023-10-06T03:46:07.811405+0000","last_became_active":"2023-10-06T03:46:07.802830+0000","last_became_peered":"2023-10-06T03:46:07.802830+0000","last_unstale":"2023-10-06T03:46:07.811405+0000","last_undegraded":"2023-10-06T03:46:07.811405+0000","last_fullsized":"2023-10-06T03:46:07.811405+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:19.826311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.825347+0000","last_change":"2023-10-06T03:46:07.822899+0000","last_active":"2023-10-06T03:46:07.825347+0000","last_peered":"2023-10-06T03:46:07.825347+0000","last_clean":"2023-10-06T03:46:07.825347+0000","last_became_active":"2023-10-06T03:46:07.822585+0000","last_became_peered":"2023-10-06T03:46:07.822585+0000","last_unstale":"2023-10-06T03:46:07.825347+0000","last_undegraded":"2023-10-06T03:46:07.825347+0000","last_fullsized":"2023-10-06T03:46:07.825347+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:49.832412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.819815+0000","last_change":"2023-10-06T03:46:07.808808+0000","last_active":"2023-10-06T03:46:07.819815+0000","last_peered":"2023-10-06T03:46:07.819815+0000","last_clean":"2023-10-06T03:46:07.819815+0000","last_became_active":"2023-10-06T03:46:07.808510+0000","last_became_peered":"2023-10-06T03:46:07.808510+0000","last_unstale":"2023-10-06T03:46:07.819815+0000","last_undegraded":"2023-10-06T03:46:07.819815+0000","last_fullsized":"2023-10-06T03:46:07.819815+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:09:34.272462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.822048+0000","last_change":"2023-10-06T03:46:07.816274+0000","last_active":"2023-10-06T03:46:07.822048+0000","last_peered":"2023-10-06T03:46:07.822048+0000","last_clean":"2023-10-06T03:46:07.822048+0000","last_became_active":"2023-10-06T03:46:07.815962+0000","last_became_peered":"2023-10-06T03:46:07.815962+0000","last_unstale":"2023-10-06T03:46:07.822048+0000","last_undegraded":"2023-10-06T03:46:07.822048+0000","last_fullsized":"2023-10-06T03:46:07.822048+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:46.633052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.803942+0000","last_change":"2023-10-06T03:46:07.797756+0000","last_active":"2023-10-06T03:46:07.803942+0000","last_peered":"2023-10-06T03:46:07.803942+0000","last_clean":"2023-10-06T03:46:07.803942+0000","last_became_active":"2023-10-06T03:46:07.797498+0000","last_became_peered":"2023-10-06T03:46:07.797498+0000","last_unstale":"2023-10-06T03:46:07.803942+0000","last_undegraded":"2023-10-06T03:46:07.803942+0000","last_fullsized":"2023-10-06T03:46:07.803942+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:21.428503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.821375+0000","last_change":"2023-10-06T03:46:07.816385+0000","last_active":"2023-10-06T03:46:07.821375+0000","last_peered":"2023-10-06T03:46:07.821375+0000","last_clean":"2023-10-06T03:46:07.821375+0000","last_became_active":"2023-10-06T03:46:07.816139+0000","last_became_peered":"2023-10-06T03:46:07.816139+0000","last_unstale":"2023-10-06T03:46:07.821375+0000","last_undegraded":"2023-10-06T03:46:07.821375+0000","last_fullsized":"2023-10-06T03:46:07.821375+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:45.782682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.811764+0000","last_change":"2023-10-06T03:46:07.808806+0000","last_active":"2023-10-06T03:46:07.811764+0000","last_peered":"2023-10-06T03:46:07.811764+0000","last_clean":"2023-10-06T03:46:07.811764+0000","last_became_active":"2023-10-06T03:46:07.808553+0000","last_became_peered":"2023-10-06T03:46:07.808553+0000","last_unstale":"2023-10-06T03:46:07.811764+0000","last_undegraded":"2023-10-06T03:46:07.811764+0000","last_fullsized":"2023-10-06T03:46:07.811764+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:30.038870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.808026+0000","last_change":"2023-10-06T03:46:07.805495+0000","last_active":"2023-10-06T03:46:07.808026+0000","last_peered":"2023-10-06T03:46:07.808026+0000","last_clean":"2023-10-06T03:46:07.808026+0000","last_became_active":"2023-10-06T03:46:07.802639+0000","last_became_peered":"2023-10-06T03:46:07.802639+0000","last_unstale":"2023-10-06T03:46:07.808026+0000","last_undegraded":"2023-10-06T03:46:07.808026+0000","last_fullsized":"2023-10-06T03:46:07.808026+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:52:34.747585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.819413+0000","last_change":"2023-10-06T03:46:07.816804+0000","last_active":"2023-10-06T03:46:07.819413+0000","last_peered":"2023-10-06T03:46:07.819413+0000","last_clean":"2023-10-06T03:46:07.819413+0000","last_became_active":"2023-10-06T03:46:07.816489+0000","last_became_peered":"2023-10-06T03:46:07.816489+0000","last_unstale":"2023-10-06T03:46:07.819413+0000","last_undegraded":"2023-10-06T03:46:07.819413+0000","last_fullsized":"2023-10-06T03:46:07.819413+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:48:07.612249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.811799+0000","last_change":"2023-10-06T03:46:07.808278+0000","last_active":"2023-10-06T03:46:07.811799+0000","last_peered":"2023-10-06T03:46:07.811799+0000","last_clean":"2023-10-06T03:46:07.811799+0000","last_became_active":"2023-10-06T03:46:07.807962+0000","last_became_peered":"2023-10-06T03:46:07.807962+0000","last_unstale":"2023-10-06T03:46:07.811799+0000","last_undegraded":"2023-10-06T03:46:07.811799+0000","last_fullsized":"2023-10-06T03:46:07.811799+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:37.140556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.811353+0000","last_change":"2023-10-06T03:46:07.805483+0000","last_active":"2023-10-06T03:46:07.811353+0000","last_peered":"2023-10-06T03:46:07.811353+0000","last_clean":"2023-10-06T03:46:07.811353+0000","last_became_active":"2023-10-06T03:46:07.804915+0000","last_became_peered":"2023-10-06T03:46:07.804915+0000","last_unstale":"2023-10-06T03:46:07.811353+0000","last_undegraded":"2023-10-06T03:46:07.811353+0000","last_fullsized":"2023-10-06T03:46:07.811353+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:13.040618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.815674+0000","last_change":"2023-10-06T03:46:07.808815+0000","last_active":"2023-10-06T03:46:07.815674+0000","last_peered":"2023-10-06T03:46:07.815674+0000","last_clean":"2023-10-06T03:46:07.815674+0000","last_became_active":"2023-10-06T03:46:07.808578+0000","last_became_peered":"2023-10-06T03:46:07.808578+0000","last_unstale":"2023-10-06T03:46:07.815674+0000","last_undegraded":"2023-10-06T03:46:07.815674+0000","last_fullsized":"2023-10-06T03:46:07.815674+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:36.465082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.818657+0000","last_change":"2023-10-06T03:46:07.815410+0000","last_active":"2023-10-06T03:46:07.818657+0000","last_peered":"2023-10-06T03:46:07.818657+0000","last_clean":"2023-10-06T03:46:07.818657+0000","last_became_active":"2023-10-06T03:46:07.815144+0000","last_became_peered":"2023-10-06T03:46:07.815144+0000","last_unstale":"2023-10-06T03:46:07.818657+0000","last_undegraded":"2023-10-06T03:46:07.818657+0000","last_fullsized":"2023-10-06T03:46:07.818657+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:56:18.036383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814292+0000","last_change":"2023-10-06T03:46:07.812304+0000","last_active":"2023-10-06T03:46:07.814292+0000","last_peered":"2023-10-06T03:46:07.814292+0000","last_clean":"2023-10-06T03:46:07.814292+0000","last_became_active":"2023-10-06T03:46:07.812063+0000","last_became_peered":"2023-10-06T03:46:07.812063+0000","last_unstale":"2023-10-06T03:46:07.814292+0000","last_undegraded":"2023-10-06T03:46:07.814292+0000","last_fullsized":"2023-10-06T03:46:07.814292+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:46.892188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.821496+0000","last_change":"2023-10-06T03:46:07.817343+0000","last_active":"2023-10-06T03:46:07.821496+0000","last_peered":"2023-10-06T03:46:07.821496+0000","last_clean":"2023-10-06T03:46:07.821496+0000","last_became_active":"2023-10-06T03:46:07.817071+0000","last_became_peered":"2023-10-06T03:46:07.817071+0000","last_unstale":"2023-10-06T03:46:07.821496+0000","last_undegraded":"2023-10-06T03:46:07.821496+0000","last_fullsized":"2023-10-06T03:46:07.821496+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:07.722180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.825293+0000","last_change":"2023-10-06T03:46:07.823474+0000","last_active":"2023-10-06T03:46:07.825293+0000","last_peered":"2023-10-06T03:46:07.825293+0000","last_clean":"2023-10-06T03:46:07.825293+0000","last_became_active":"2023-10-06T03:46:07.822448+0000","last_became_peered":"2023-10-06T03:46:07.822448+0000","last_unstale":"2023-10-06T03:46:07.825293+0000","last_undegraded":"2023-10-06T03:46:07.825293+0000","last_fullsized":"2023-10-06T03:46:07.825293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:02:51.772579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.800432+0000","last_change":"2023-10-06T03:46:07.796586+0000","last_active":"2023-10-06T03:46:07.800432+0000","last_peered":"2023-10-06T03:46:07.800432+0000","last_clean":"2023-10-06T03:46:07.800432+0000","last_became_active":"2023-10-06T03:46:07.796299+0000","last_became_peered":"2023-10-06T03:46:07.796299+0000","last_unstale":"2023-10-06T03:46:07.800432+0000","last_undegraded":"2023-10-06T03:46:07.800432+0000","last_fullsized":"2023-10-06T03:46:07.800432+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:34:59.672250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.806376+0000","last_change":"2023-10-06T03:46:07.802200+0000","last_active":"2023-10-06T03:46:07.806376+0000","last_peered":"2023-10-06T03:46:07.806376+0000","last_clean":"2023-10-06T03:46:07.806376+0000","last_became_active":"2023-10-06T03:46:07.801913+0000","last_became_peered":"2023-10-06T03:46:07.801913+0000","last_unstale":"2023-10-06T03:46:07.806376+0000","last_undegraded":"2023-10-06T03:46:07.806376+0000","last_fullsized":"2023-10-06T03:46:07.806376+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:01.444525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.818140+0000","last_change":"2023-10-06T03:46:07.813785+0000","last_active":"2023-10-06T03:46:07.818140+0000","last_peered":"2023-10-06T03:46:07.818140+0000","last_clean":"2023-10-06T03:46:07.818140+0000","last_became_active":"2023-10-06T03:46:07.813463+0000","last_became_peered":"2023-10-06T03:46:07.813463+0000","last_unstale":"2023-10-06T03:46:07.818140+0000","last_undegraded":"2023-10-06T03:46:07.818140+0000","last_fullsized":"2023-10-06T03:46:07.818140+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:44.254939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.819805+0000","last_change":"2023-10-06T03:46:07.808918+0000","last_active":"2023-10-06T03:46:07.819805+0000","last_peered":"2023-10-06T03:46:07.819805+0000","last_clean":"2023-10-06T03:46:07.819805+0000","last_became_active":"2023-10-06T03:46:07.808626+0000","last_became_peered":"2023-10-06T03:46:07.808626+0000","last_unstale":"2023-10-06T03:46:07.819805+0000","last_undegraded":"2023-10-06T03:46:07.819805+0000","last_fullsized":"2023-10-06T03:46:07.819805+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:01:48.641104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.802130+0000","last_change":"2023-10-06T03:46:07.800026+0000","last_active":"2023-10-06T03:46:07.802130+0000","last_peered":"2023-10-06T03:46:07.802130+0000","last_clean":"2023-10-06T03:46:07.802130+0000","last_became_active":"2023-10-06T03:46:07.799723+0000","last_became_peered":"2023-10-06T03:46:07.799723+0000","last_unstale":"2023-10-06T03:46:07.802130+0000","last_undegraded":"2023-10-06T03:46:07.802130+0000","last_fullsized":"2023-10-06T03:46:07.802130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:03.157309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814000+0000","last_change":"2023-10-06T03:46:07.803959+0000","last_active":"2023-10-06T03:46:07.814000+0000","last_peered":"2023-10-06T03:46:07.814000+0000","last_clean":"2023-10-06T03:46:07.814000+0000","last_became_active":"2023-10-06T03:46:07.801538+0000","last_became_peered":"2023-10-06T03:46:07.801538+0000","last_unstale":"2023-10-06T03:46:07.814000+0000","last_undegraded":"2023-10-06T03:46:07.814000+0000","last_fullsized":"2023-10-06T03:46:07.814000+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:46.878707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.803395+0000","last_change":"2023-10-06T03:46:07.798753+0000","last_active":"2023-10-06T03:46:07.803395+0000","last_peered":"2023-10-06T03:46:07.803395+0000","last_clean":"2023-10-06T03:46:07.803395+0000","last_became_active":"2023-10-06T03:46:07.798435+0000","last_became_peered":"2023-10-06T03:46:07.798435+0000","last_unstale":"2023-10-06T03:46:07.803395+0000","last_undegraded":"2023-10-06T03:46:07.803395+0000","last_fullsized":"2023-10-06T03:46:07.803395+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:59.029330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814887+0000","last_change":"2023-10-06T03:46:07.810635+0000","last_active":"2023-10-06T03:46:07.814887+0000","last_peered":"2023-10-06T03:46:07.814887+0000","last_clean":"2023-10-06T03:46:07.814887+0000","last_became_active":"2023-10-06T03:46:07.810115+0000","last_became_peered":"2023-10-06T03:46:07.810115+0000","last_unstale":"2023-10-06T03:46:07.814887+0000","last_undegraded":"2023-10-06T03:46:07.814887+0000","last_fullsized":"2023-10-06T03:46:07.814887+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:17:25.155702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.811386+0000","last_change":"2023-10-06T03:46:07.803398+0000","last_active":"2023-10-06T03:46:07.811386+0000","last_peered":"2023-10-06T03:46:07.811386+0000","last_clean":"2023-10-06T03:46:07.811386+0000","last_became_active":"2023-10-06T03:46:07.802453+0000","last_became_peered":"2023-10-06T03:46:07.802453+0000","last_unstale":"2023-10-06T03:46:07.811386+0000","last_undegraded":"2023-10-06T03:46:07.811386+0000","last_fullsized":"2023-10-06T03:46:07.811386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:08.592648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817209+0000","last_change":"2023-10-06T03:46:07.812420+0000","last_active":"2023-10-06T03:46:07.817209+0000","last_peered":"2023-10-06T03:46:07.817209+0000","last_clean":"2023-10-06T03:46:07.817209+0000","last_became_active":"2023-10-06T03:46:07.812142+0000","last_became_peered":"2023-10-06T03:46:07.812142+0000","last_unstale":"2023-10-06T03:46:07.817209+0000","last_undegraded":"2023-10-06T03:46:07.817209+0000","last_fullsized":"2023-10-06T03:46:07.817209+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:07.965204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.807623+0000","last_change":"2023-10-06T03:46:07.806249+0000","last_active":"2023-10-06T03:46:07.807623+0000","last_peered":"2023-10-06T03:46:07.807623+0000","last_clean":"2023-10-06T03:46:07.807623+0000","last_became_active":"2023-10-06T03:46:07.804978+0000","last_became_peered":"2023-10-06T03:46:07.804978+0000","last_unstale":"2023-10-06T03:46:07.807623+0000","last_undegraded":"2023-10-06T03:46:07.807623+0000","last_fullsized":"2023-10-06T03:46:07.807623+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:21.695190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"63'4","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817757+0000","last_change":"2023-10-06T03:46:06.797810+0000","last_active":"2023-10-06T03:46:07.817757+0000","last_peered":"2023-10-06T03:46:07.817757+0000","last_clean":"2023-10-06T03:46:07.817757+0000","last_became_active":"2023-10-06T03:46:06.796496+0000","last_became_peered":"2023-10-06T03:46:06.796496+0000","last_unstale":"2023-10-06T03:46:07.817757+0000","last_undegraded":"2023-10-06T03:46:07.817757+0000","last_fullsized":"2023-10-06T03:46:07.817757+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:50.161636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817136+0000","last_change":"2023-10-06T03:46:07.814556+0000","last_active":"2023-10-06T03:46:07.817136+0000","last_peered":"2023-10-06T03:46:07.817136+0000","last_clean":"2023-10-06T03:46:07.817136+0000","last_became_active":"2023-10-06T03:46:07.814308+0000","last_became_peered":"2023-10-06T03:46:07.814308+0000","last_unstale":"2023-10-06T03:46:07.817136+0000","last_undegraded":"2023-10-06T03:46:07.817136+0000","last_fullsized":"2023-10-06T03:46:07.817136+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:30.038870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.821840+0000","last_change":"2023-10-06T03:46:06.806361+0000","last_active":"2023-10-06T03:46:07.821840+0000","last_peered":"2023-10-06T03:46:07.821840+0000","last_clean":"2023-10-06T03:46:07.821840+0000","last_became_active":"2023-10-06T03:46:06.806058+0000","last_became_peered":"2023-10-06T03:46:06.806058+0000","last_unstale":"2023-10-06T03:46:07.821840+0000","last_undegraded":"2023-10-06T03:46:07.821840+0000","last_fullsized":"2023-10-06T03:46:07.821840+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:47:39.977632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814129+0000","last_change":"2023-10-06T03:46:07.811437+0000","last_active":"2023-10-06T03:46:07.814129+0000","last_peered":"2023-10-06T03:46:07.814129+0000","last_clean":"2023-10-06T03:46:07.814129+0000","last_became_active":"2023-10-06T03:46:07.811159+0000","last_became_peered":"2023-10-06T03:46:07.811159+0000","last_unstale":"2023-10-06T03:46:07.814129+0000","last_undegraded":"2023-10-06T03:46:07.814129+0000","last_fullsized":"2023-10-06T03:46:07.814129+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:43.069156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.822012+0000","last_change":"2023-10-06T03:46:06.798822+0000","last_active":"2023-10-06T03:46:07.822012+0000","last_peered":"2023-10-06T03:46:07.822012+0000","last_clean":"2023-10-06T03:46:07.822012+0000","last_became_active":"2023-10-06T03:46:06.796111+0000","last_became_peered":"2023-10-06T03:46:06.796111+0000","last_unstale":"2023-10-06T03:46:07.822012+0000","last_undegraded":"2023-10-06T03:46:07.822012+0000","last_fullsized":"2023-10-06T03:46:07.822012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:06:48.864196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.822730+0000","last_change":"2023-10-06T03:46:07.820140+0000","last_active":"2023-10-06T03:46:07.822730+0000","last_peered":"2023-10-06T03:46:07.822730+0000","last_clean":"2023-10-06T03:46:07.822730+0000","last_became_active":"2023-10-06T03:46:07.819719+0000","last_became_peered":"2023-10-06T03:46:07.819719+0000","last_unstale":"2023-10-06T03:46:07.822730+0000","last_undegraded":"2023-10-06T03:46:07.822730+0000","last_fullsized":"2023-10-06T03:46:07.822730+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:03.905822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.820682+0000","last_change":"2023-10-06T03:46:06.805041+0000","last_active":"2023-10-06T03:46:07.820682+0000","last_peered":"2023-10-06T03:46:07.820682+0000","last_clean":"2023-10-06T03:46:07.820682+0000","last_became_active":"2023-10-06T03:46:06.804709+0000","last_became_peered":"2023-10-06T03:46:06.804709+0000","last_unstale":"2023-10-06T03:46:07.820682+0000","last_undegraded":"2023-10-06T03:46:07.820682+0000","last_fullsized":"2023-10-06T03:46:07.820682+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:35:06.713183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.812650+0000","last_change":"2023-10-06T03:46:07.808640+0000","last_active":"2023-10-06T03:46:07.812650+0000","last_peered":"2023-10-06T03:46:07.812650+0000","last_clean":"2023-10-06T03:46:07.812650+0000","last_became_active":"2023-10-06T03:46:07.808335+0000","last_became_peered":"2023-10-06T03:46:07.808335+0000","last_unstale":"2023-10-06T03:46:07.812650+0000","last_undegraded":"2023-10-06T03:46:07.812650+0000","last_fullsized":"2023-10-06T03:46:07.812650+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:49:25.900934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.818998+0000","last_change":"2023-10-06T03:46:06.811044+0000","last_active":"2023-10-06T03:46:07.818998+0000","last_peered":"2023-10-06T03:46:07.818998+0000","last_clean":"2023-10-06T03:46:07.818998+0000","last_became_active":"2023-10-06T03:46:06.810769+0000","last_became_peered":"2023-10-06T03:46:06.810769+0000","last_unstale":"2023-10-06T03:46:07.818998+0000","last_undegraded":"2023-10-06T03:46:07.818998+0000","last_fullsized":"2023-10-06T03:46:07.818998+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:34.265734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.807961+0000","last_change":"2023-10-06T03:46:07.805807+0000","last_active":"2023-10-06T03:46:07.807961+0000","last_peered":"2023-10-06T03:46:07.807961+0000","last_clean":"2023-10-06T03:46:07.807961+0000","last_became_active":"2023-10-06T03:46:07.804914+0000","last_became_peered":"2023-10-06T03:46:07.804914+0000","last_unstale":"2023-10-06T03:46:07.807961+0000","last_undegraded":"2023-10-06T03:46:07.807961+0000","last_fullsized":"2023-10-06T03:46:07.807961+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:05.437972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.819472+0000","last_change":"2023-10-06T03:46:06.799430+0000","last_active":"2023-10-06T03:46:07.819472+0000","last_peered":"2023-10-06T03:46:07.819472+0000","last_clean":"2023-10-06T03:46:07.819472+0000","last_became_active":"2023-10-06T03:46:06.799144+0000","last_became_peered":"2023-10-06T03:46:06.799144+0000","last_unstale":"2023-10-06T03:46:07.819472+0000","last_undegraded":"2023-10-06T03:46:07.819472+0000","last_fullsized":"2023-10-06T03:46:07.819472+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:20.686193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.820024+0000","last_change":"2023-10-06T03:46:07.809525+0000","last_active":"2023-10-06T03:46:07.820024+0000","last_peered":"2023-10-06T03:46:07.820024+0000","last_clean":"2023-10-06T03:46:07.820024+0000","last_became_active":"2023-10-06T03:46:07.809231+0000","last_became_peered":"2023-10-06T03:46:07.809231+0000","last_unstale":"2023-10-06T03:46:07.820024+0000","last_undegraded":"2023-10-06T03:46:07.820024+0000","last_fullsized":"2023-10-06T03:46:07.820024+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:08:48.093050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.790915+0000","last_change":"2023-10-06T03:46:06.799556+0000","last_active":"2023-10-06T03:46:07.790915+0000","last_peered":"2023-10-06T03:46:07.790915+0000","last_clean":"2023-10-06T03:46:07.790915+0000","last_became_active":"2023-10-06T03:46:06.798555+0000","last_became_peered":"2023-10-06T03:46:06.798555+0000","last_unstale":"2023-10-06T03:46:07.790915+0000","last_undegraded":"2023-10-06T03:46:07.790915+0000","last_fullsized":"2023-10-06T03:46:07.790915+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:12:18.719894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817705+0000","last_change":"2023-10-06T03:46:07.811054+0000","last_active":"2023-10-06T03:46:07.817705+0000","last_peered":"2023-10-06T03:46:07.817705+0000","last_clean":"2023-10-06T03:46:07.817705+0000","last_became_active":"2023-10-06T03:46:07.810678+0000","last_became_peered":"2023-10-06T03:46:07.810678+0000","last_unstale":"2023-10-06T03:46:07.817705+0000","last_undegraded":"2023-10-06T03:46:07.817705+0000","last_fullsized":"2023-10-06T03:46:07.817705+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:35.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.790930+0000","last_change":"2023-10-06T03:46:06.811527+0000","last_active":"2023-10-06T03:46:07.790930+0000","last_peered":"2023-10-06T03:46:07.790930+0000","last_clean":"2023-10-06T03:46:07.790930+0000","last_became_active":"2023-10-06T03:46:06.811221+0000","last_became_peered":"2023-10-06T03:46:06.811221+0000","last_unstale":"2023-10-06T03:46:07.790930+0000","last_undegraded":"2023-10-06T03:46:07.790930+0000","last_fullsized":"2023-10-06T03:46:07.790930+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:20.686193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.824380+0000","last_change":"2023-10-06T03:46:07.820458+0000","last_active":"2023-10-06T03:46:07.824380+0000","last_peered":"2023-10-06T03:46:07.824380+0000","last_clean":"2023-10-06T03:46:07.824380+0000","last_became_active":"2023-10-06T03:46:07.820218+0000","last_became_peered":"2023-10-06T03:46:07.820218+0000","last_unstale":"2023-10-06T03:46:07.824380+0000","last_undegraded":"2023-10-06T03:46:07.824380+0000","last_fullsized":"2023-10-06T03:46:07.824380+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:08:48.093050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.794304+0000","last_change":"2023-10-06T03:46:06.797122+0000","last_active":"2023-10-06T03:46:07.794304+0000","last_peered":"2023-10-06T03:46:07.794304+0000","last_clean":"2023-10-06T03:46:07.794304+0000","last_became_active":"2023-10-06T03:46:06.796755+0000","last_became_peered":"2023-10-06T03:46:06.796755+0000","last_unstale":"2023-10-06T03:46:07.794304+0000","last_undegraded":"2023-10-06T03:46:07.794304+0000","last_fullsized":"2023-10-06T03:46:07.794304+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:05.828405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817116+0000","last_change":"2023-10-06T03:46:07.812345+0000","last_active":"2023-10-06T03:46:07.817116+0000","last_peered":"2023-10-06T03:46:07.817116+0000","last_clean":"2023-10-06T03:46:07.817116+0000","last_became_active":"2023-10-06T03:46:07.812008+0000","last_became_peered":"2023-10-06T03:46:07.812008+0000","last_unstale":"2023-10-06T03:46:07.817116+0000","last_undegraded":"2023-10-06T03:46:07.817116+0000","last_fullsized":"2023-10-06T03:46:07.817116+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:59.481973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.800590+0000","last_change":"2023-10-06T03:46:06.809388+0000","last_active":"2023-10-06T03:46:07.800590+0000","last_peered":"2023-10-06T03:46:07.800590+0000","last_clean":"2023-10-06T03:46:07.800590+0000","last_became_active":"2023-10-06T03:46:06.809152+0000","last_became_peered":"2023-10-06T03:46:06.809152+0000","last_unstale":"2023-10-06T03:46:07.800590+0000","last_undegraded":"2023-10-06T03:46:07.800590+0000","last_fullsized":"2023-10-06T03:46:07.800590+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:34.265734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.800626+0000","last_change":"2023-10-06T03:46:07.797062+0000","last_active":"2023-10-06T03:46:07.800626+0000","last_peered":"2023-10-06T03:46:07.800626+0000","last_clean":"2023-10-06T03:46:07.800626+0000","last_became_active":"2023-10-06T03:46:07.796767+0000","last_became_peered":"2023-10-06T03:46:07.796767+0000","last_unstale":"2023-10-06T03:46:07.800626+0000","last_undegraded":"2023-10-06T03:46:07.800626+0000","last_fullsized":"2023-10-06T03:46:07.800626+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:31.631695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"63'3","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814848+0000","last_change":"2023-10-06T03:46:06.809953+0000","last_active":"2023-10-06T03:46:07.814848+0000","last_peered":"2023-10-06T03:46:07.814848+0000","last_clean":"2023-10-06T03:46:07.814848+0000","last_became_active":"2023-10-06T03:46:06.809646+0000","last_became_peered":"2023-10-06T03:46:06.809646+0000","last_unstale":"2023-10-06T03:46:07.814848+0000","last_undegraded":"2023-10-06T03:46:07.814848+0000","last_fullsized":"2023-10-06T03:46:07.814848+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:06.956208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.816313+0000","last_change":"2023-10-06T03:46:07.811277+0000","last_active":"2023-10-06T03:46:07.816313+0000","last_peered":"2023-10-06T03:46:07.816313+0000","last_clean":"2023-10-06T03:46:07.816313+0000","last_became_active":"2023-10-06T03:46:07.810989+0000","last_became_peered":"2023-10-06T03:46:07.810989+0000","last_unstale":"2023-10-06T03:46:07.816313+0000","last_undegraded":"2023-10-06T03:46:07.816313+0000","last_fullsized":"2023-10-06T03:46:07.816313+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:28.197821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"63'6","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.827579+0000","last_change":"2023-10-06T03:46:06.804616+0000","last_active":"2023-10-06T03:46:07.827579+0000","last_peered":"2023-10-06T03:46:07.827579+0000","last_clean":"2023-10-06T03:46:07.827579+0000","last_became_active":"2023-10-06T03:46:06.804341+0000","last_became_peered":"2023-10-06T03:46:06.804341+0000","last_unstale":"2023-10-06T03:46:07.827579+0000","last_undegraded":"2023-10-06T03:46:07.827579+0000","last_fullsized":"2023-10-06T03:46:07.827579+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:20.436780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817387+0000","last_change":"2023-10-06T03:46:07.809963+0000","last_active":"2023-10-06T03:46:07.817387+0000","last_peered":"2023-10-06T03:46:07.817387+0000","last_clean":"2023-10-06T03:46:07.817387+0000","last_became_active":"2023-10-06T03:46:07.809559+0000","last_became_peered":"2023-10-06T03:46:07.809559+0000","last_unstale":"2023-10-06T03:46:07.817387+0000","last_undegraded":"2023-10-06T03:46:07.817387+0000","last_fullsized":"2023-10-06T03:46:07.817387+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:13:46.633052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.819115+0000","last_change":"2023-10-06T03:46:06.809067+0000","last_active":"2023-10-06T03:46:07.819115+0000","last_peered":"2023-10-06T03:46:07.819115+0000","last_clean":"2023-10-06T03:46:07.819115+0000","last_became_active":"2023-10-06T03:46:06.808762+0000","last_became_peered":"2023-10-06T03:46:06.808762+0000","last_unstale":"2023-10-06T03:46:07.819115+0000","last_undegraded":"2023-10-06T03:46:07.819115+0000","last_fullsized":"2023-10-06T03:46:07.819115+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:33:34.835730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.816517+0000","last_change":"2023-10-06T03:46:07.805716+0000","last_active":"2023-10-06T03:46:07.816517+0000","last_peered":"2023-10-06T03:46:07.816517+0000","last_clean":"2023-10-06T03:46:07.816517+0000","last_became_active":"2023-10-06T03:46:07.805388+0000","last_became_peered":"2023-10-06T03:46:07.805388+0000","last_unstale":"2023-10-06T03:46:07.816517+0000","last_undegraded":"2023-10-06T03:46:07.816517+0000","last_fullsized":"2023-10-06T03:46:07.816517+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:34:09.430960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.791181+0000","last_change":"2023-10-06T03:46:06.796800+0000","last_active":"2023-10-06T03:46:07.791181+0000","last_peered":"2023-10-06T03:46:07.791181+0000","last_clean":"2023-10-06T03:46:07.791181+0000","last_became_active":"2023-10-06T03:46:06.796502+0000","last_became_peered":"2023-10-06T03:46:06.796502+0000","last_unstale":"2023-10-06T03:46:07.791181+0000","last_undegraded":"2023-10-06T03:46:07.791181+0000","last_fullsized":"2023-10-06T03:46:07.791181+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:47.041953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.820044+0000","last_change":"2023-10-06T03:46:07.810983+0000","last_active":"2023-10-06T03:46:07.820044+0000","last_peered":"2023-10-06T03:46:07.820044+0000","last_clean":"2023-10-06T03:46:07.820044+0000","last_became_active":"2023-10-06T03:46:07.810683+0000","last_became_peered":"2023-10-06T03:46:07.810683+0000","last_unstale":"2023-10-06T03:46:07.820044+0000","last_undegraded":"2023-10-06T03:46:07.820044+0000","last_fullsized":"2023-10-06T03:46:07.820044+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:07:51.709094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.794763+0000","last_change":"2023-10-06T03:46:06.799700+0000","last_active":"2023-10-06T03:46:07.794763+0000","last_peered":"2023-10-06T03:46:07.794763+0000","last_clean":"2023-10-06T03:46:07.794763+0000","last_became_active":"2023-10-06T03:46:06.799413+0000","last_became_peered":"2023-10-06T03:46:06.799413+0000","last_unstale":"2023-10-06T03:46:07.794763+0000","last_undegraded":"2023-10-06T03:46:07.794763+0000","last_fullsized":"2023-10-06T03:46:07.794763+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:26.208223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.811333+0000","last_change":"2023-10-06T03:46:07.804891+0000","last_active":"2023-10-06T03:46:07.811333+0000","last_peered":"2023-10-06T03:46:07.811333+0000","last_clean":"2023-10-06T03:46:07.811333+0000","last_became_active":"2023-10-06T03:46:07.804524+0000","last_became_peered":"2023-10-06T03:46:07.804524+0000","last_unstale":"2023-10-06T03:46:07.811333+0000","last_undegraded":"2023-10-06T03:46:07.811333+0000","last_fullsized":"2023-10-06T03:46:07.811333+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:43.349472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"63'8","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.826181+0000","last_change":"2023-10-06T03:46:06.803940+0000","last_active":"2023-10-06T03:46:07.826181+0000","last_peered":"2023-10-06T03:46:07.826181+0000","last_clean":"2023-10-06T03:46:07.826181+0000","last_became_active":"2023-10-06T03:46:06.803679+0000","last_became_peered":"2023-10-06T03:46:06.803679+0000","last_unstale":"2023-10-06T03:46:07.826181+0000","last_undegraded":"2023-10-06T03:46:07.826181+0000","last_fullsized":"2023-10-06T03:46:07.826181+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:58.472976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.825364+0000","last_change":"2023-10-06T03:46:07.823327+0000","last_active":"2023-10-06T03:46:07.825364+0000","last_peered":"2023-10-06T03:46:07.825364+0000","last_clean":"2023-10-06T03:46:07.825364+0000","last_became_active":"2023-10-06T03:46:07.823062+0000","last_became_peered":"2023-10-06T03:46:07.823062+0000","last_unstale":"2023-10-06T03:46:07.825364+0000","last_undegraded":"2023-10-06T03:46:07.825364+0000","last_fullsized":"2023-10-06T03:46:07.825364+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:27:06.386235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.825688+0000","last_change":"2023-10-06T03:46:06.813181+0000","last_active":"2023-10-06T03:46:07.825688+0000","last_peered":"2023-10-06T03:46:07.825688+0000","last_clean":"2023-10-06T03:46:07.825688+0000","last_became_active":"2023-10-06T03:46:06.811641+0000","last_became_peered":"2023-10-06T03:46:06.811641+0000","last_unstale":"2023-10-06T03:46:07.825688+0000","last_undegraded":"2023-10-06T03:46:07.825688+0000","last_fullsized":"2023-10-06T03:46:07.825688+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:38:04.428975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.802095+0000","last_change":"2023-10-06T03:46:07.799765+0000","last_active":"2023-10-06T03:46:07.802095+0000","last_peered":"2023-10-06T03:46:07.802095+0000","last_clean":"2023-10-06T03:46:07.802095+0000","last_became_active":"2023-10-06T03:46:07.799489+0000","last_became_peered":"2023-10-06T03:46:07.799489+0000","last_unstale":"2023-10-06T03:46:07.802095+0000","last_undegraded":"2023-10-06T03:46:07.802095+0000","last_fullsized":"2023-10-06T03:46:07.802095+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:43.878689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.825737+0000","last_change":"2023-10-06T03:46:06.801829+0000","last_active":"2023-10-06T03:46:07.825737+0000","last_peered":"2023-10-06T03:46:07.825737+0000","last_clean":"2023-10-06T03:46:07.825737+0000","last_became_active":"2023-10-06T03:46:06.801568+0000","last_became_peered":"2023-10-06T03:46:06.801568+0000","last_unstale":"2023-10-06T03:46:07.825737+0000","last_undegraded":"2023-10-06T03:46:07.825737+0000","last_fullsized":"2023-10-06T03:46:07.825737+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:05:57.549516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.816554+0000","last_change":"2023-10-06T03:46:07.812835+0000","last_active":"2023-10-06T03:46:07.816554+0000","last_peered":"2023-10-06T03:46:07.816554+0000","last_clean":"2023-10-06T03:46:07.816554+0000","last_became_active":"2023-10-06T03:46:07.812564+0000","last_became_peered":"2023-10-06T03:46:07.812564+0000","last_unstale":"2023-10-06T03:46:07.816554+0000","last_undegraded":"2023-10-06T03:46:07.816554+0000","last_fullsized":"2023-10-06T03:46:07.816554+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:07:51.709094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"63'1","reported_seq":11,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.822409+0000","last_change":"2023-10-06T03:46:06.797393+0000","last_active":"2023-10-06T03:46:07.822409+0000","last_peered":"2023-10-06T03:46:07.822409+0000","last_clean":"2023-10-06T03:46:07.822409+0000","last_became_active":"2023-10-06T03:46:06.797117+0000","last_became_peered":"2023-10-06T03:46:06.797117+0000","last_unstale":"2023-10-06T03:46:07.822409+0000","last_undegraded":"2023-10-06T03:46:07.822409+0000","last_fullsized":"2023-10-06T03:46:07.822409+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:10.143016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.805053+0000","last_change":"2023-10-06T03:46:07.804195+0000","last_active":"2023-10-06T03:46:07.805053+0000","last_peered":"2023-10-06T03:46:07.805053+0000","last_clean":"2023-10-06T03:46:07.805053+0000","last_became_active":"2023-10-06T03:46:07.803833+0000","last_became_peered":"2023-10-06T03:46:07.803833+0000","last_unstale":"2023-10-06T03:46:07.805053+0000","last_undegraded":"2023-10-06T03:46:07.805053+0000","last_fullsized":"2023-10-06T03:46:07.805053+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:30:07.965204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.791011+0000","last_change":"2023-10-06T03:46:06.804991+0000","last_active":"2023-10-06T03:46:07.791011+0000","last_peered":"2023-10-06T03:46:07.791011+0000","last_clean":"2023-10-06T03:46:07.791011+0000","last_became_active":"2023-10-06T03:46:06.804659+0000","last_became_peered":"2023-10-06T03:46:06.804659+0000","last_unstale":"2023-10-06T03:46:07.791011+0000","last_undegraded":"2023-10-06T03:46:07.791011+0000","last_fullsized":"2023-10-06T03:46:07.791011+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:47.476452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817369+0000","last_change":"2023-10-06T03:46:07.812769+0000","last_active":"2023-10-06T03:46:07.817369+0000","last_peered":"2023-10-06T03:46:07.817369+0000","last_clean":"2023-10-06T03:46:07.817369+0000","last_became_active":"2023-10-06T03:46:07.812499+0000","last_became_peered":"2023-10-06T03:46:07.812499+0000","last_unstale":"2023-10-06T03:46:07.817369+0000","last_undegraded":"2023-10-06T03:46:07.817369+0000","last_fullsized":"2023-10-06T03:46:07.817369+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:21.445776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.818272+0000","last_change":"2023-10-06T03:46:06.789733+0000","last_active":"2023-10-06T03:46:07.818272+0000","last_peered":"2023-10-06T03:46:07.818272+0000","last_clean":"2023-10-06T03:46:07.818272+0000","last_became_active":"2023-10-06T03:46:06.789507+0000","last_became_peered":"2023-10-06T03:46:06.789507+0000","last_unstale":"2023-10-06T03:46:07.818272+0000","last_undegraded":"2023-10-06T03:46:07.818272+0000","last_fullsized":"2023-10-06T03:46:07.818272+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:28:19.914456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814272+0000","last_change":"2023-10-06T03:46:07.813250+0000","last_active":"2023-10-06T03:46:07.814272+0000","last_peered":"2023-10-06T03:46:07.814272+0000","last_clean":"2023-10-06T03:46:07.814272+0000","last_became_active":"2023-10-06T03:46:07.812947+0000","last_became_peered":"2023-10-06T03:46:07.812947+0000","last_unstale":"2023-10-06T03:46:07.814272+0000","last_undegraded":"2023-10-06T03:46:07.814272+0000","last_fullsized":"2023-10-06T03:46:07.814272+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:43.817906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.805049+0000","last_change":"2023-10-06T03:46:06.803264+0000","last_active":"2023-10-06T03:46:07.805049+0000","last_peered":"2023-10-06T03:46:07.805049+0000","last_clean":"2023-10-06T03:46:07.805049+0000","last_became_active":"2023-10-06T03:46:06.802868+0000","last_became_peered":"2023-10-06T03:46:06.802868+0000","last_unstale":"2023-10-06T03:46:07.805049+0000","last_undegraded":"2023-10-06T03:46:07.805049+0000","last_fullsized":"2023-10-06T03:46:07.805049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:42.869693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.803365+0000","last_change":"2023-10-06T03:46:07.798274+0000","last_active":"2023-10-06T03:46:07.803365+0000","last_peered":"2023-10-06T03:46:07.803365+0000","last_clean":"2023-10-06T03:46:07.803365+0000","last_became_active":"2023-10-06T03:46:07.797981+0000","last_became_peered":"2023-10-06T03:46:07.797981+0000","last_unstale":"2023-10-06T03:46:07.803365+0000","last_undegraded":"2023-10-06T03:46:07.803365+0000","last_fullsized":"2023-10-06T03:46:07.803365+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:57:23.659815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.791184+0000","last_change":"2023-10-06T03:46:06.811340+0000","last_active":"2023-10-06T03:46:07.791184+0000","last_peered":"2023-10-06T03:46:07.791184+0000","last_clean":"2023-10-06T03:46:07.791184+0000","last_became_active":"2023-10-06T03:46:06.811058+0000","last_became_peered":"2023-10-06T03:46:06.811058+0000","last_unstale":"2023-10-06T03:46:07.791184+0000","last_undegraded":"2023-10-06T03:46:07.791184+0000","last_fullsized":"2023-10-06T03:46:07.791184+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:24:00.975835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.822365+0000","last_change":"2023-10-06T03:46:07.820590+0000","last_active":"2023-10-06T03:46:07.822365+0000","last_peered":"2023-10-06T03:46:07.822365+0000","last_clean":"2023-10-06T03:46:07.822365+0000","last_became_active":"2023-10-06T03:46:07.820281+0000","last_became_peered":"2023-10-06T03:46:07.820281+0000","last_unstale":"2023-10-06T03:46:07.822365+0000","last_undegraded":"2023-10-06T03:46:07.822365+0000","last_fullsized":"2023-10-06T03:46:07.822365+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:19.132012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.787836+0000","last_change":"2023-10-06T03:46:06.799614+0000","last_active":"2023-10-06T03:46:07.787836+0000","last_peered":"2023-10-06T03:46:07.787836+0000","last_clean":"2023-10-06T03:46:07.787836+0000","last_became_active":"2023-10-06T03:46:06.799350+0000","last_became_peered":"2023-10-06T03:46:06.799350+0000","last_unstale":"2023-10-06T03:46:07.787836+0000","last_undegraded":"2023-10-06T03:46:07.787836+0000","last_fullsized":"2023-10-06T03:46:07.787836+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:46.103432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817404+0000","last_change":"2023-10-06T03:46:07.813338+0000","last_active":"2023-10-06T03:46:07.817404+0000","last_peered":"2023-10-06T03:46:07.817404+0000","last_clean":"2023-10-06T03:46:07.817404+0000","last_became_active":"2023-10-06T03:46:07.812987+0000","last_became_peered":"2023-10-06T03:46:07.812987+0000","last_unstale":"2023-10-06T03:46:07.817404+0000","last_undegraded":"2023-10-06T03:46:07.817404+0000","last_fullsized":"2023-10-06T03:46:07.817404+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:03.818415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.800087+0000","last_change":"2023-10-06T03:46:06.804932+0000","last_active":"2023-10-06T03:46:07.800087+0000","last_peered":"2023-10-06T03:46:07.800087+0000","last_clean":"2023-10-06T03:46:07.800087+0000","last_became_active":"2023-10-06T03:46:06.804646+0000","last_became_peered":"2023-10-06T03:46:06.804646+0000","last_unstale":"2023-10-06T03:46:07.800087+0000","last_undegraded":"2023-10-06T03:46:07.800087+0000","last_fullsized":"2023-10-06T03:46:07.800087+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:36:18.123015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":53,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.798980+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:46:07.798980+0000","last_peered":"2023-10-06T03:46:07.798980+0000","last_clean":"2023-10-06T03:46:07.798980+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:46:07.798980+0000","last_undegraded":"2023-10-06T03:46:07.798980+0000","last_fullsized":"2023-10-06T03:46:07.798980+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.816937+0000","last_change":"2023-10-06T03:46:07.809839+0000","last_active":"2023-10-06T03:46:07.816937+0000","last_peered":"2023-10-06T03:46:07.816937+0000","last_clean":"2023-10-06T03:46:07.816937+0000","last_became_active":"2023-10-06T03:46:07.809550+0000","last_became_peered":"2023-10-06T03:46:07.809550+0000","last_unstale":"2023-10-06T03:46:07.816937+0000","last_undegraded":"2023-10-06T03:46:07.816937+0000","last_fullsized":"2023-10-06T03:46:07.816937+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:22.619791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.805361+0000","last_change":"2023-10-06T03:46:06.810362+0000","last_active":"2023-10-06T03:46:07.805361+0000","last_peered":"2023-10-06T03:46:07.805361+0000","last_clean":"2023-10-06T03:46:07.805361+0000","last_became_active":"2023-10-06T03:46:06.810062+0000","last_became_peered":"2023-10-06T03:46:06.810062+0000","last_unstale":"2023-10-06T03:46:07.805361+0000","last_undegraded":"2023-10-06T03:46:07.805361+0000","last_fullsized":"2023-10-06T03:46:07.805361+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:42:02.148312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":52,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.801643+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:46:07.801643+0000","last_peered":"2023-10-06T03:46:07.801643+0000","last_clean":"2023-10-06T03:46:07.801643+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:46:07.801643+0000","last_undegraded":"2023-10-06T03:46:07.801643+0000","last_fullsized":"2023-10-06T03:46:07.801643+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.804832+0000","last_change":"2023-10-06T03:46:07.800207+0000","last_active":"2023-10-06T03:46:07.804832+0000","last_peered":"2023-10-06T03:46:07.804832+0000","last_clean":"2023-10-06T03:46:07.804832+0000","last_became_active":"2023-10-06T03:46:07.799926+0000","last_became_peered":"2023-10-06T03:46:07.799926+0000","last_unstale":"2023-10-06T03:46:07.804832+0000","last_undegraded":"2023-10-06T03:46:07.804832+0000","last_fullsized":"2023-10-06T03:46:07.804832+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:06:22.619791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.794176+0000","last_change":"2023-10-06T03:46:06.798222+0000","last_active":"2023-10-06T03:46:07.794176+0000","last_peered":"2023-10-06T03:46:07.794176+0000","last_clean":"2023-10-06T03:46:07.794176+0000","last_became_active":"2023-10-06T03:46:06.797962+0000","last_became_peered":"2023-10-06T03:46:06.797962+0000","last_unstale":"2023-10-06T03:46:07.794176+0000","last_undegraded":"2023-10-06T03:46:07.794176+0000","last_fullsized":"2023-10-06T03:46:07.794176+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:36.131560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":52,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.793396+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:46:07.793396+0000","last_peered":"2023-10-06T03:46:07.793396+0000","last_clean":"2023-10-06T03:46:07.793396+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:46:07.793396+0000","last_undegraded":"2023-10-06T03:46:07.793396+0000","last_fullsized":"2023-10-06T03:46:07.793396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"54'139","reported_seq":280,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.793125+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:46:07.793125+0000","last_peered":"2023-10-06T03:46:07.793125+0000","last_clean":"2023-10-06T03:46:07.793125+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:46:07.793125+0000","last_undegraded":"2023-10-06T03:46:07.793125+0000","last_fullsized":"2023-10-06T03:46:07.793125+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.821126+0000","last_change":"2023-10-06T03:46:07.815733+0000","last_active":"2023-10-06T03:46:07.821126+0000","last_peered":"2023-10-06T03:46:07.821126+0000","last_clean":"2023-10-06T03:46:07.821126+0000","last_became_active":"2023-10-06T03:46:07.815479+0000","last_became_peered":"2023-10-06T03:46:07.815479+0000","last_unstale":"2023-10-06T03:46:07.821126+0000","last_undegraded":"2023-10-06T03:46:07.821126+0000","last_fullsized":"2023-10-06T03:46:07.821126+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:35.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.792341+0000","last_change":"2023-10-06T03:46:06.797846+0000","last_active":"2023-10-06T03:46:07.792341+0000","last_peered":"2023-10-06T03:46:07.792341+0000","last_clean":"2023-10-06T03:46:07.792341+0000","last_became_active":"2023-10-06T03:46:06.797563+0000","last_became_peered":"2023-10-06T03:46:06.797563+0000","last_unstale":"2023-10-06T03:46:07.792341+0000","last_undegraded":"2023-10-06T03:46:07.792341+0000","last_fullsized":"2023-10-06T03:46:07.792341+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:21.329719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":53,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.793545+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:46:07.793545+0000","last_peered":"2023-10-06T03:46:07.793545+0000","last_clean":"2023-10-06T03:46:07.793545+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:46:07.793545+0000","last_undegraded":"2023-10-06T03:46:07.793545+0000","last_fullsized":"2023-10-06T03:46:07.793545+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.814121+0000","last_change":"2023-10-06T03:46:07.803828+0000","last_active":"2023-10-06T03:46:07.814121+0000","last_peered":"2023-10-06T03:46:07.814121+0000","last_clean":"2023-10-06T03:46:07.814121+0000","last_became_active":"2023-10-06T03:46:07.803446+0000","last_became_peered":"2023-10-06T03:46:07.803446+0000","last_unstale":"2023-10-06T03:46:07.814121+0000","last_undegraded":"2023-10-06T03:46:07.814121+0000","last_fullsized":"2023-10-06T03:46:07.814121+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:52.882135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.797130+0000","last_change":"2023-10-06T03:46:06.810579+0000","last_active":"2023-10-06T03:46:07.797130+0000","last_peered":"2023-10-06T03:46:07.797130+0000","last_clean":"2023-10-06T03:46:07.797130+0000","last_became_active":"2023-10-06T03:46:06.808824+0000","last_became_peered":"2023-10-06T03:46:06.808824+0000","last_unstale":"2023-10-06T03:46:07.797130+0000","last_undegraded":"2023-10-06T03:46:07.797130+0000","last_fullsized":"2023-10-06T03:46:07.797130+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:22:08.478426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.793042+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:46:07.793042+0000","last_peered":"2023-10-06T03:46:07.793042+0000","last_clean":"2023-10-06T03:46:07.793042+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:46:07.793042+0000","last_undegraded":"2023-10-06T03:46:07.793042+0000","last_fullsized":"2023-10-06T03:46:07.793042+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.817190+0000","last_change":"2023-10-06T03:46:07.812134+0000","last_active":"2023-10-06T03:46:07.817190+0000","last_peered":"2023-10-06T03:46:07.817190+0000","last_clean":"2023-10-06T03:46:07.817190+0000","last_became_active":"2023-10-06T03:46:07.811727+0000","last_became_peered":"2023-10-06T03:46:07.811727+0000","last_unstale":"2023-10-06T03:46:07.817190+0000","last_undegraded":"2023-10-06T03:46:07.817190+0000","last_fullsized":"2023-10-06T03:46:07.817190+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:52:37.140556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"63'2","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.820756+0000","last_change":"2023-10-06T03:46:06.812006+0000","last_active":"2023-10-06T03:46:07.820756+0000","last_peered":"2023-10-06T03:46:07.820756+0000","last_clean":"2023-10-06T03:46:07.820756+0000","last_became_active":"2023-10-06T03:46:06.811704+0000","last_became_peered":"2023-10-06T03:46:06.811704+0000","last_unstale":"2023-10-06T03:46:07.820756+0000","last_undegraded":"2023-10-06T03:46:07.820756+0000","last_fullsized":"2023-10-06T03:46:07.820756+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:52:10.219880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.796727+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:46:07.796727+0000","last_peered":"2023-10-06T03:46:07.796727+0000","last_clean":"2023-10-06T03:46:07.796727+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:46:07.796727+0000","last_undegraded":"2023-10-06T03:46:07.796727+0000","last_fullsized":"2023-10-06T03:46:07.796727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.820029+0000","last_change":"2023-10-06T03:46:07.812090+0000","last_active":"2023-10-06T03:46:07.820029+0000","last_peered":"2023-10-06T03:46:07.820029+0000","last_clean":"2023-10-06T03:46:07.820029+0000","last_became_active":"2023-10-06T03:46:07.811801+0000","last_became_peered":"2023-10-06T03:46:07.811801+0000","last_unstale":"2023-10-06T03:46:07.820029+0000","last_undegraded":"2023-10-06T03:46:07.820029+0000","last_fullsized":"2023-10-06T03:46:07.820029+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:03:28.197821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.797208+0000","last_change":"2023-10-06T03:46:06.793867+0000","last_active":"2023-10-06T03:46:07.797208+0000","last_peered":"2023-10-06T03:46:07.797208+0000","last_clean":"2023-10-06T03:46:07.797208+0000","last_became_active":"2023-10-06T03:46:06.793571+0000","last_became_peered":"2023-10-06T03:46:06.793571+0000","last_unstale":"2023-10-06T03:46:07.797208+0000","last_undegraded":"2023-10-06T03:46:07.797208+0000","last_fullsized":"2023-10-06T03:46:07.797208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:42.060159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.790374+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:46:07.790374+0000","last_peered":"2023-10-06T03:46:07.790374+0000","last_clean":"2023-10-06T03:46:07.790374+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:46:07.790374+0000","last_undegraded":"2023-10-06T03:46:07.790374+0000","last_fullsized":"2023-10-06T03:46:07.790374+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.808056+0000","last_change":"2023-10-06T03:46:07.803252+0000","last_active":"2023-10-06T03:46:07.808056+0000","last_peered":"2023-10-06T03:46:07.808056+0000","last_clean":"2023-10-06T03:46:07.808056+0000","last_became_active":"2023-10-06T03:46:07.802963+0000","last_became_peered":"2023-10-06T03:46:07.802963+0000","last_unstale":"2023-10-06T03:46:07.808056+0000","last_undegraded":"2023-10-06T03:46:07.808056+0000","last_fullsized":"2023-10-06T03:46:07.808056+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:06.773916+0000","last_clean_scrub_stamp":"2023-10-06T03:46:06.773916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:41.541412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.797591+0000","last_change":"2023-10-06T03:46:06.803826+0000","last_active":"2023-10-06T03:46:07.797591+0000","last_peered":"2023-10-06T03:46:07.797591+0000","last_clean":"2023-10-06T03:46:07.797591+0000","last_became_active":"2023-10-06T03:46:06.803582+0000","last_became_peered":"2023-10-06T03:46:06.803582+0000","last_unstale":"2023-10-06T03:46:07.797591+0000","last_undegraded":"2023-10-06T03:46:07.797591+0000","last_fullsized":"2023-10-06T03:46:07.797591+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:46:05.764920+0000","last_clean_scrub_stamp":"2023-10-06T03:46:05.764920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:21.329719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":57,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-10-06T03:46:07.790948+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:46:07.790948+0000","last_peered":"2023-10-06T03:46:07.790948+0000","last_clean":"2023-10-06T03:46:07.790948+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:46:07.790948+0000","last_undegraded":"2023-10-06T03:46:07.790948+0000","last_fullsized":"2023-10-06T03:46:07.790948+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575005,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310904,"kb_used_data":1328,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060936,"statfs":{"total":96636764160,"available":96318398464,"internally_reserved":0,"allocated":1359872,"data_stored":965756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71499999999999997,"15min":0.68000000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.34000000000000002},"max":{"1min":1.0029999999999999,"5min":1.004,"15min":1.6100000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72699999999999998,"15min":0.70699999999999996},"min":{"1min":0.501,"5min":0.47999999999999998,"15min":0.47499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.5649999999999999},"last":0.624}]},{"osd":1,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77600000000000002,"15min":0.73999999999999999},"min":{"1min":0.56499999999999995,"5min":0.434,"15min":0.42199999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.5469999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78200000000000003,"15min":0.77300000000000002},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.036,"5min":1.036,"15min":1.0580000000000001},"last":0.745}]},{"osd":2,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.71099999999999997,"15min":0.68799999999999994},"min":{"1min":0.54400000000000004,"5min":0.49299999999999999,"15min":0.46500000000000002},"max":{"1min":0.90300000000000002,"5min":0.93000000000000005,"15min":1.5900000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.749,"15min":0.73399999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.501},"max":{"1min":0.97599999999999998,"5min":0.97699999999999998,"15min":1.044},"last":0.495}]},{"osd":3,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73699999999999999,"15min":0.71999999999999997},"min":{"1min":0.58199999999999996,"5min":0.45900000000000002,"15min":0.39500000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.6279999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75600000000000001,"15min":0.73899999999999999},"min":{"1min":0.55400000000000005,"5min":0.53900000000000003,"15min":0.46600000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.0409999999999999},"last":0.75800000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57199999999999995,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.745,"5min":0.86599999999999999,"15min":1.0049999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.60499999999999998,"15min":0.63700000000000001},"min":{"1min":0.46100000000000002,"5min":0.40999999999999998,"15min":0.39400000000000002},"max":{"1min":0.78800000000000003,"5min":0.86099999999999999,"15min":0.95699999999999996},"last":0.59999999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.66000000000000003},"min":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.36799999999999999},"max":{"1min":0.93799999999999994,"5min":0.96999999999999997,"15min":1.069},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.69599999999999995},"min":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.436},"max":{"1min":0.88200000000000001,"5min":0.91900000000000004,"15min":1.044},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71099999999999997,"15min":0.73499999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.996},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69999999999999996,"15min":0.71699999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.0209999999999999},"last":0.72599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575006,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302136,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069704,"statfs":{"total":96636764160,"available":96327376896,"internally_reserved":0,"allocated":770048,"data_stored":376866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71599999999999997,"15min":0.68700000000000006},"min":{"1min":0.56100000000000005,"5min":0.38800000000000001,"15min":0.313},"max":{"1min":0.86599999999999999,"5min":0.95799999999999996,"15min":0.98299999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73699999999999999,"15min":0.70699999999999996},"min":{"1min":0.625,"5min":0.437,"15min":0.39500000000000002},"max":{"1min":0.98599999999999999,"5min":1.0069999999999999,"15min":1.0449999999999999},"last":0.94899999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.755,"15min":0.71999999999999997},"min":{"1min":0.58299999999999996,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96799999999999997,"5min":0.98199999999999998,"15min":1.3180000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72799999999999998,"15min":0.70399999999999996},"min":{"1min":0.64900000000000002,"5min":0.378,"15min":0.35899999999999999},"max":{"1min":0.85399999999999998,"5min":0.995,"15min":0.995},"last":0.90900000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.624,"5min":0.48199999999999998,"15min":0.46000000000000002},"max":{"1min":0.92600000000000005,"5min":0.96399999999999997,"15min":3.3319999999999999},"last":0.995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.75900000000000001,"15min":0.77700000000000002},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.43099999999999999},"max":{"1min":0.97799999999999998,"5min":1.0409999999999999,"15min":3.5139999999999998},"last":0.748}]},{"osd":3,"last update":"Fri Oct 6 03:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76800000000000002,"15min":0.745},"min":{"1min":0.624,"5min":0.45500000000000002,"15min":0.44600000000000001},"max":{"1min":0.92200000000000004,"5min":1.0109999999999999,"15min":1.024},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78000000000000003,"15min":0.80400000000000005},"min":{"1min":0.66800000000000004,"5min":0.55000000000000004,"15min":0.45200000000000001},"max":{"1min":0.93999999999999995,"5min":1.03,"15min":3.3300000000000001},"last":0.79500000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56899999999999995,"15min":0.60799999999999998},"min":{"1min":0.44400000000000001,"5min":0.376,"15min":0.34799999999999998},"max":{"1min":0.70699999999999996,"5min":0.92600000000000005,"15min":1.0680000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63900000000000001,"15min":0.66000000000000003},"min":{"1min":0.50600000000000001,"5min":0.434,"15min":0.36499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":1.2929999999999999},"last":0.88300000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66200000000000003,"15min":0.69099999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41099999999999998},"max":{"1min":0.85999999999999999,"5min":0.95599999999999996,"15min":1.3340000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.54600000000000004,"5min":0.42899999999999999,"15min":0.41199999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":1.2370000000000001},"last":1.0269999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:45:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68000000000000005,"15min":0.72699999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.92300000000000004,"5min":0.94399999999999995,"15min":1.619},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70599999999999996,"15min":0.73299999999999998},"min":{"1min":0.51000000000000001,"5min":0.48299999999999998,"15min":0.47099999999999997},"max":{"1min":0.89000000000000001,"5min":0.98299999999999998,"15min":1.417},"last":1.0509999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575005,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302716,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069124,"statfs":{"total":96636764160,"available":96326782976,"internally_reserved":0,"allocated":1363968,"data_stored":966770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.5,"15min":0.48399999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.73599999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.37},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35599999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.2889999999999999},"last":0.438}]},{"osd":2,"last update":"Fri Oct 6 03:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.50700000000000001,"15min":0.48799999999999999},"min":{"1min":0.34200000000000003,"5min":0.32700000000000001,"15min":0.26100000000000001},"max":{"1min":0.67200000000000004,"5min":0.84799999999999998,"15min":0.999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70599999999999996,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.42399999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.59699999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82099999999999995,"5min":0.82699999999999996,"15min":0.93999999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.54700000000000004},"min":{"1min":0.40899999999999997,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":1.254},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.76700000000000002,"15min":0.753},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.034,"5min":1.1659999999999999,"15min":1.3600000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.82399999999999995,"15min":0.79200000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.2,"5min":1.2,"15min":1.476},"last":0.59199999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80800000000000005,"15min":0.79500000000000004},"min":{"1min":0.504,"5min":0.504,"15min":0.45400000000000001},"max":{"1min":0.996,"5min":1.0640000000000001,"15min":1.601},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81399999999999995,"15min":0.79600000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.081,"5min":1.159,"15min":1.5189999999999999},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.81799999999999995,"15min":0.81200000000000006},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0609999999999999,"5min":1.1830000000000001,"15min":1.244},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.83799999999999997,"15min":0.81200000000000006},"min":{"1min":0.57299999999999995,"5min":0.53200000000000003,"15min":0.505},"max":{"1min":1.137,"5min":1.224,"15min":1.399},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:45:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.85999999999999999,"15min":0.82899999999999996},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.56599999999999995},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.84799999999999998,"15min":0.82699999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.51200000000000001},"max":{"1min":1.155,"5min":1.155,"15min":1.556},"last":0.622}]}]},{"osd":0,"up_from":13,"seq":55834575007,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306228,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065612,"statfs":{"total":96636764160,"available":96323186688,"internally_reserved":0,"allocated":765952,"data_stored":375320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.48499999999999999,"15min":0.50600000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":2.3220000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.34499999999999997,"5min":0.33600000000000002,"15min":0.29599999999999999},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":2.1440000000000001},"last":0.35399999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53900000000000003,"15min":0.58199999999999996},"min":{"1min":0.379,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77000000000000002,"5min":0.77700000000000002,"15min":2.4329999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69099999999999995,"5min":0.84699999999999998,"15min":2.1819999999999999},"last":0.42999999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.51700000000000002,"15min":0.53700000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.28399999999999997},"max":{"1min":0.61199999999999999,"5min":0.71799999999999997,"15min":2.2709999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.60799999999999998,"15min":0.63800000000000001},"min":{"1min":0.434,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":2.4700000000000002},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.83999999999999997,"15min":0.83099999999999996},"min":{"1min":0.55200000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97999999999999998,"5min":1.1259999999999999,"15min":1.393},"last":0.878},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.82699999999999996,"15min":0.83299999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0189999999999999,"5min":1.262,"15min":2.2320000000000002},"last":0.90700000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.81899999999999995,"15min":0.83999999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.054,"5min":1.1499999999999999,"15min":2.3660000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.82599999999999996,"15min":0.80700000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.503},"max":{"1min":1.0800000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.92200000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.81499999999999995,"15min":0.80200000000000005},"min":{"1min":0.53100000000000003,"5min":0.497,"15min":0.497},"max":{"1min":1.0349999999999999,"5min":1.726,"15min":1.726},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.85499999999999998,"15min":0.86899999999999999},"min":{"1min":0.60399999999999998,"5min":0.57899999999999996,"15min":0.56999999999999995},"max":{"1min":0.996,"5min":1.843,"15min":2.4009999999999998},"last":0.78200000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.84499999999999997,"15min":0.82099999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.48499999999999999},"max":{"1min":1.036,"5min":1.1850000000000001,"15min":1.2689999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.069,"5min":1.7649999999999999,"15min":1.7649999999999999},"last":0.94999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575006,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306232,"kb_used_data":752,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065608,"statfs":{"total":96636764160,"available":96323182592,"internally_reserved":0,"allocated":770048,"data_stored":376800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.49299999999999999,"15min":0.45800000000000002},"min":{"1min":0.41199999999999998,"5min":0.311,"15min":0.28000000000000003},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.439},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54100000000000004,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.83599999999999997,"5min":0.86199999999999999,"15min":0.89300000000000002},"last":0.52700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48399999999999999,"15min":0.48699999999999999},"min":{"1min":0.36399999999999999,"5min":0.32200000000000001,"15min":0.30599999999999999},"max":{"1min":0.629,"5min":0.79900000000000004,"15min":0.90000000000000002},"last":0.503},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.38600000000000001,"5min":0.38,"15min":0.34200000000000003},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":1.556},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59699999999999998,"15min":0.58999999999999997},"min":{"1min":0.44900000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76300000000000001,"5min":0.98699999999999999,"15min":1.6040000000000001},"last":0.624},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58299999999999996,"15min":0.56799999999999995},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68799999999999994,"5min":0.91400000000000003,"15min":1.1910000000000001},"last":0.49099999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82899999999999996,"15min":0.80500000000000005},"min":{"1min":0.67100000000000004,"5min":0.56999999999999995,"15min":0.439},"max":{"1min":1.0149999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84799999999999998,"15min":0.82799999999999996},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55600000000000005},"max":{"1min":1.1279999999999999,"5min":1.1419999999999999,"15min":1.2270000000000001},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83099999999999996,"15min":0.82099999999999995},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.52200000000000002},"max":{"1min":1.169,"5min":1.177,"15min":1.3},"last":0.753},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.85599999999999998,"15min":0.84099999999999997},"min":{"1min":0.624,"5min":0.61299999999999999,"15min":0.51700000000000002},"max":{"1min":1.1339999999999999,"5min":1.1779999999999999,"15min":1.3069999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.83199999999999996,"15min":0.83099999999999996},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.53600000000000003},"max":{"1min":1.111,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83299999999999996,"15min":0.80400000000000005},"min":{"1min":0.60599999999999998,"5min":0.59199999999999997,"15min":0.51100000000000001},"max":{"1min":1.0649999999999999,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.60999999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:45:39 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.85799999999999998,"15min":0.82399999999999995},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.54900000000000004},"max":{"1min":1.4379999999999999,"5min":1.4570000000000001,"15min":1.4990000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.879,"15min":0.85499999999999998},"min":{"1min":0.63200000000000001,"5min":0.60799999999999998,"15min":0.497},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.67200000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575006,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306240,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065600,"statfs":{"total":96636764160,"available":96323174400,"internally_reserved":0,"allocated":778240,"data_stored":377800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47099999999999997,"15min":0.46500000000000002},"min":{"1min":0.33900000000000002,"5min":0.30099999999999999,"15min":0.28699999999999998},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.5,"15min":0.48899999999999999},"min":{"1min":0.35799999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":1.1699999999999999},"last":0.439}]},{"osd":1,"last update":"Fri Oct 6 03:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.48499999999999999,"15min":0.46500000000000002},"min":{"1min":0.35599999999999998,"5min":0.316,"15min":0.29299999999999998},"max":{"1min":0.66100000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54500000000000004,"15min":0.53800000000000003},"min":{"1min":0.40300000000000002,"5min":0.38300000000000001,"15min":0.33800000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.83899999999999997},"last":0.54500000000000004}]},{"osd":2,"last update":"Fri Oct 6 03:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.55400000000000005},"min":{"1min":0.42299999999999999,"5min":0.41799999999999998,"15min":0.34599999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.85599999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.61099999999999999,"15min":0.59699999999999998},"min":{"1min":0.47099999999999997,"5min":0.46000000000000002,"15min":0.38700000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.97899999999999998},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78600000000000003,"15min":0.79400000000000004},"min":{"1min":0.57399999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.028,"5min":1.2030000000000001,"15min":1.4099999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84299999999999997,"15min":0.82999999999999996},"min":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.51300000000000001},"max":{"1min":0.96999999999999997,"5min":1.1859999999999999,"15min":1.377},"last":0.63}]},{"osd":5,"last update":"Fri Oct 6 03:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80800000000000005,"15min":0.78800000000000003},"min":{"1min":0.58899999999999997,"5min":0.54300000000000004,"15min":0.47099999999999997},"max":{"1min":1.05,"5min":1.256,"15min":1.256},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78900000000000003,"15min":0.79200000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.48099999999999998},"max":{"1min":0.97899999999999998,"5min":1.1439999999999999,"15min":1.306},"last":0.69799999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82899999999999996,"15min":0.81299999999999994},"min":{"1min":0.58199999999999996,"5min":0.57099999999999995,"15min":0.48599999999999999},"max":{"1min":1.089,"5min":1.292,"15min":1.323},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.85799999999999998,"15min":0.85299999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.49299999999999999},"max":{"1min":1.1040000000000001,"5min":1.248,"15min":1.6020000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:45:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.82799999999999996,"15min":0.81899999999999995},"min":{"1min":0.63200000000000001,"5min":0.58499999999999996,"15min":0.505},"max":{"1min":1.0700000000000001,"5min":1.2370000000000001,"15min":1.365},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.81000000000000005,"15min":0.80900000000000005},"min":{"1min":0.61899999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.96999999999999997,"5min":1.1180000000000001,"15min":1.2330000000000001},"last":0.57299999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575006,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302132,"kb_used_data":748,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069708,"statfs":{"total":96636764160,"available":96327380992,"internally_reserved":0,"allocated":765952,"data_stored":376796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.41799999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.89200000000000002,"5min":1.0109999999999999,"15min":1.3120000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.83999999999999997,"5min":0.96399999999999997,"15min":1.01},"last":0.70999999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.74399999999999999,"15min":0.72499999999999998},"min":{"1min":0.35899999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.96499999999999997,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.747,"15min":0.74199999999999999},"min":{"1min":0.49299999999999999,"5min":0.44,"15min":0.41299999999999998},"max":{"1min":0.94999999999999996,"5min":0.96999999999999997,"15min":1.3380000000000001},"last":0.56200000000000006}]},{"osd":2,"last update":"Fri Oct 6 03:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.747,"15min":0.73799999999999999},"min":{"1min":0.44500000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.028,"5min":1.028,"15min":1.1759999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.752,"15min":0.74199999999999999},"min":{"1min":0.56999999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90200000000000002,"5min":1.022,"15min":1.3089999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.72799999999999998,"15min":0.72599999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94899999999999995,"5min":1.0049999999999999,"15min":1.0409999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77300000000000002,"15min":0.76900000000000002},"min":{"1min":0.58199999999999996,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.995,"5min":0.995,"15min":1.1259999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.62,"15min":0.64200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.085},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.623,"15min":0.64200000000000002},"min":{"1min":0.44700000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91900000000000004,"5min":0.95399999999999996,"15min":1.1439999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.66000000000000003},"min":{"1min":0.44700000000000001,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90800000000000003,"5min":1.018,"15min":1.0880000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67000000000000004,"15min":0.70999999999999996},"min":{"1min":0.48799999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":1.361},"last":0.74099999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:45:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64700000000000002,"15min":0.69499999999999995},"min":{"1min":0.46500000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89400000000000002,"5min":0.92400000000000004,"15min":1.1950000000000001},"last":0.751},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70899999999999996,"15min":0.72599999999999998},"min":{"1min":0.51800000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0760000000000001,"5min":1.115,"15min":1.1439999999999999},"last":0.73099999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575006,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298028,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073812,"statfs":{"total":96636764160,"available":96331583488,"internally_reserved":0,"allocated":757760,"data_stored":375264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.70399999999999996,"15min":0.69899999999999995},"min":{"1min":0.41499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88900000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.71299999999999997,"15min":0.69299999999999995},"min":{"1min":0.44600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82699999999999996,"5min":0.96899999999999997,"15min":1},"last":0.626}]},{"osd":1,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70999999999999996,"15min":0.70099999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86199999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.47199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82899999999999996,"5min":0.94799999999999995,"15min":0.95799999999999996},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.77000000000000002,"15min":0.75700000000000001},"min":{"1min":0.502,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97299999999999998,"5min":1.02,"15min":1.022},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.76600000000000001,"15min":0.77600000000000002},"min":{"1min":0.55000000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91500000000000004,"5min":1.006,"15min":1.006},"last":0.61399999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.505,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.871,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.74299999999999999,"15min":0.74399999999999999},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.85399999999999998,"5min":0.999,"15min":0.999},"last":0.67500000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59799999999999998,"15min":0.63},"min":{"1min":0.40400000000000003,"5min":0.40000000000000002,"15min":0.38300000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.024},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.60199999999999998,"15min":0.64700000000000002},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.36799999999999999},"max":{"1min":0.749,"5min":0.93999999999999995,"15min":1.175},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.61899999999999999,"15min":0.66100000000000003},"min":{"1min":0.46800000000000003,"5min":0.38500000000000001,"15min":0.34999999999999998},"max":{"1min":0.77900000000000003,"5min":0.92700000000000005,"15min":1.1180000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67500000000000004,"15min":0.70899999999999996},"min":{"1min":0.504,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89000000000000001,"5min":0.96399999999999997,"15min":1.0489999999999999},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:45:36 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65100000000000002,"15min":0.68899999999999995},"min":{"1min":0.496,"5min":0.40899999999999997,"15min":0.40699999999999997},"max":{"1min":0.77200000000000002,"5min":1.016,"15min":1.2589999999999999},"last":0.746},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.69599999999999995,"15min":0.73299999999999998},"min":{"1min":0.51300000000000001,"5min":0.436,"15min":0.435},"max":{"1min":0.88500000000000001,"5min":0.97599999999999998,"15min":1.1020000000000001},"last":0.70899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:46:12.664 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:46:12.665 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:46:12.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.893+0000 7f3033fff700 1 Processor -- start 2023-10-06T03:46:12.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.894+0000 7f3033fff700 1 -- start start 2023-10-06T03:46:12.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.895+0000 7f3033fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f303406af90 0x7f30340c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:12.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.895+0000 7f3033fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f303406b360 0x7f30340ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:12.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.895+0000 7f3033fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30340c9860 con 0x7f303415e020 2023-10-06T03:46:12.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.895+0000 7f3033fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30340c8cd0 con 0x7f303406af90 2023-10-06T03:46:12.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.895+0000 7f3033fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30340cd200 con 0x7f303406b360 2023-10-06T03:46:12.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.895+0000 7f30327fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f303406b360 0x7f30340ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:12.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.895+0000 7f3032ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f303406af90 0x7f30340c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:12.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.895+0000 7f3032ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f303406af90 0x7f30340c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:56878/0 (socket says 172.21.15.55:56878) 2023-10-06T03:46:12.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.895+0000 7f30327fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f303406b360 0x7f30340ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:46624/0 (socket says 172.21.15.55:46624) 2023-10-06T03:46:12.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.895+0000 7f3032ffd700 1 -- 172.21.15.55:0/3290946160 learned_addr learned my addr 172.21.15.55:0/3290946160 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:12.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.895+0000 7f30337fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f303415e020 0x7f303406d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:37412/0 (socket says 172.21.15.55:37412) 2023-10-06T03:46:12.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.896+0000 7f30327fc700 1 -- 172.21.15.55:0/3290946160 >> v2:172.21.15.163:3300/0 conn(0x7f303406af90 msgr2=0x7f30340c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:12.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.896+0000 7f30327fc700 1 --2- 172.21.15.55:0/3290946160 >> v2:172.21.15.163:3300/0 conn(0x7f303406af90 0x7f30340c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.896+0000 7f30327fc700 1 -- 172.21.15.55:0/3290946160 >> v1:172.21.15.163:6789/0 conn(0x7f303415e020 legacy=0x7f303406d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:12.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.896+0000 7f30327fc700 1 -- 172.21.15.55:0/3290946160 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303415e4c0 con 0x7f303406b360 2023-10-06T03:46:12.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.896+0000 7f3032ffd700 1 --2- 172.21.15.55:0/3290946160 >> v2:172.21.15.163:3300/0 conn(0x7f303406af90 0x7f30340c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:46:12.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.896+0000 7f30327fc700 1 --2- 172.21.15.55:0/3290946160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f303406b360 0x7f30340ccb00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f30280098e0 tx=0x7f30280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a72b54457ea8a541 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:12.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.897+0000 7f3031ffb700 1 -- 172.21.15.55:0/3290946160 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302800b6d0 con 0x7f303406b360 2023-10-06T03:46:12.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.897+0000 7f3031ffb700 1 -- 172.21.15.55:0/3290946160 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302800b830 con 0x7f303406b360 2023-10-06T03:46:12.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.897+0000 7f3031ffb700 1 -- 172.21.15.55:0/3290946160 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302800bbd0 con 0x7f303406b360 2023-10-06T03:46:12.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.897+0000 7f3033fff700 1 -- 172.21.15.55:0/3290946160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f303406b360 msgr2=0x7f30340ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:12.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.897+0000 7f3033fff700 1 --2- 172.21.15.55:0/3290946160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f303406b360 0x7f30340ccb00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f30280098e0 tx=0x7f30280091a0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.897+0000 7f3033fff700 1 -- 172.21.15.55:0/3290946160 shutdown_connections 2023-10-06T03:46:12.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.897+0000 7f3033fff700 1 --2- 172.21.15.55:0/3290946160 >> v2:172.21.15.163:3300/0 conn(0x7f303406af90 0x7f30340c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.897+0000 7f3033fff700 1 --2- 172.21.15.55:0/3290946160 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f303406b360 0x7f30340ccb00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.897+0000 7f3033fff700 1 -- 172.21.15.55:0/3290946160 >> 172.21.15.55:0/3290946160 conn(0x7f303406baa0 msgr2=0x7f303406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:12.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.897+0000 7f3033fff700 1 -- 172.21.15.55:0/3290946160 shutdown_connections 2023-10-06T03:46:12.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.897+0000 7f3033fff700 1 -- 172.21.15.55:0/3290946160 wait complete. 2023-10-06T03:46:12.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.899+0000 7f3033fff700 1 Processor -- start 2023-10-06T03:46:12.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.899+0000 7f3033fff700 1 -- start start 2023-10-06T03:46:12.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.900+0000 7f3033fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f303406af90 0x7f30341697e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:12.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.900+0000 7f3033fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f303406b360 0x7f3034157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:12.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.900+0000 7f3033fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f303415e020 0x7f3034169f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:12.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.900+0000 7f3033fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3034158490 con 0x7f303406b360 2023-10-06T03:46:12.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.900+0000 7f3033fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3034158770 con 0x7f303415e020 2023-10-06T03:46:12.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.900+0000 7f3033fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3034158a50 con 0x7f303406af90 2023-10-06T03:46:12.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.900+0000 7f30327fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f303406b360 0x7f3034157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:12.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.900+0000 7f30327fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f303406b360 0x7f3034157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50290/0 (socket says 172.21.15.55:50290) 2023-10-06T03:46:12.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.900+0000 7f30327fc700 1 -- 172.21.15.55:0/2666420564 learned_addr learned my addr 172.21.15.55:0/2666420564 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:12.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.900+0000 7f30337fe700 1 --2- 172.21.15.55:0/2666420564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f303415e020 0x7f3034169f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:12.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.900+0000 7f3032ffd700 1 --2- 172.21.15.55:0/2666420564 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f303406af90 0x7f30341697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:12.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.901+0000 7f30327fc700 1 -- 172.21.15.55:0/2666420564 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f303406af90 msgr2=0x7f30341697e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:12.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.901+0000 7f30327fc700 1 --2- 172.21.15.55:0/2666420564 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f303406af90 0x7f30341697e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.901+0000 7f30327fc700 1 -- 172.21.15.55:0/2666420564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f303415e020 msgr2=0x7f3034169f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:12.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.901+0000 7f30327fc700 1 --2- 172.21.15.55:0/2666420564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f303415e020 0x7f3034169f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:12.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.901+0000 7f30327fc700 1 -- 172.21.15.55:0/2666420564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30240094f0 con 0x7f303406b360 2023-10-06T03:46:12.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.901+0000 7f30327fc700 1 --2- 172.21.15.55:0/2666420564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f303406b360 0x7f3034157d90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f30340cd340 tx=0x7f3028017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:12.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.901+0000 7f301affd700 1 -- 172.21.15.55:0/2666420564 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302801bcc0 con 0x7f303406b360 2023-10-06T03:46:12.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.901+0000 7f301affd700 1 -- 172.21.15.55:0/2666420564 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302801be60 con 0x7f303406b360 2023-10-06T03:46:12.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.901+0000 7f3033fff700 1 -- 172.21.15.55:0/2666420564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3028009420 con 0x7f303406b360 2023-10-06T03:46:12.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.902+0000 7f301affd700 1 -- 172.21.15.55:0/2666420564 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3028011670 con 0x7f303406b360 2023-10-06T03:46:12.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.902+0000 7f3033fff700 1 -- 172.21.15.55:0/2666420564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3034159010 con 0x7f303406b360 2023-10-06T03:46:12.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.903+0000 7f3033fff700 1 -- 172.21.15.55:0/2666420564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3034159200 con 0x7f303406b360 2023-10-06T03:46:12.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.903+0000 7f301affd700 1 -- 172.21.15.55:0/2666420564 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f30280119e0 con 0x7f303406b360 2023-10-06T03:46:12.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.904+0000 7f301affd700 1 --2- 172.21.15.55:0/2666420564 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3020066f30 0x7f30200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:12.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.904+0000 7f301affd700 1 -- 172.21.15.55:0/2666420564 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f30280860a0 con 0x7f303406b360 2023-10-06T03:46:12.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.905+0000 7f3032ffd700 1 --2- 172.21.15.55:0/2666420564 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3020066f30 0x7f30200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:12.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.905+0000 7f3032ffd700 1 --2- 172.21.15.55:0/2666420564 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3020066f30 0x7f30200693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3024006d20 tx=0x7f3024018040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:12.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:12.910+0000 7f301affd700 1 -- 172.21.15.55:0/2666420564 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3028053040 con 0x7f303406b360 2023-10-06T03:46:13.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.193+0000 7f3033fff700 1 -- 172.21.15.55:0/2666420564 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f303404c3c0 con 0x7f303406b360 2023-10-06T03:46:13.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.199+0000 7f301affd700 1 -- 172.21.15.55:0/2666420564 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3028056660 con 0x7f303406b360 2023-10-06T03:46:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.201+0000 7f3033fff700 1 -- 172.21.15.55:0/2666420564 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3020066f30 msgr2=0x7f30200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.201+0000 7f3033fff700 1 --2- 172.21.15.55:0/2666420564 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3020066f30 0x7f30200693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3024006d20 tx=0x7f3024018040 comp rx=0 tx=0).stop 2023-10-06T03:46:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.201+0000 7f3033fff700 1 -- 172.21.15.55:0/2666420564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f303406b360 msgr2=0x7f3034157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.201+0000 7f3033fff700 1 --2- 172.21.15.55:0/2666420564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f303406b360 0x7f3034157d90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f30340cd340 tx=0x7f3028017000 comp rx=0 tx=0).stop 2023-10-06T03:46:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.201+0000 7f3033fff700 1 -- 172.21.15.55:0/2666420564 shutdown_connections 2023-10-06T03:46:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.201+0000 7f3033fff700 1 --2- 172.21.15.55:0/2666420564 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3020066f30 0x7f30200693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.201+0000 7f3033fff700 1 --2- 172.21.15.55:0/2666420564 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f303415e020 0x7f3034169f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:13.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.201+0000 7f3033fff700 1 --2- 172.21.15.55:0/2666420564 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f303406af90 0x7f30341697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:13.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.201+0000 7f3033fff700 1 --2- 172.21.15.55:0/2666420564 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f303406b360 0x7f3034157d90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:13.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.201+0000 7f3033fff700 1 -- 172.21.15.55:0/2666420564 >> 172.21.15.55:0/2666420564 conn(0x7f303406baa0 msgr2=0x7f30340c3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:13.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.201+0000 7f3033fff700 1 -- 172.21.15.55:0/2666420564 shutdown_connections 2023-10-06T03:46:13.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.201+0000 7f3033fff700 1 -- 172.21.15.55:0/2666420564 wait complete. 2023-10-06T03:46:13.204 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2023-10-06T03:46:13.220 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:46:13.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.406+0000 7fbd5b59e700 1 Processor -- start 2023-10-06T03:46:13.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.406+0000 7fbd5b59e700 1 -- start start 2023-10-06T03:46:13.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.407+0000 7fbd5b59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd5c1595a0 0x7fbd5c0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:13.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.407+0000 7fbd5b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbd5c0c6c60 0x7fbd5c0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:13.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.407+0000 7fbd5b59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5c0b1410 con 0x7fbd5c0b1de0 2023-10-06T03:46:13.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.407+0000 7fbd5b59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5c0b2b90 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.407+0000 7fbd5b59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5c063230 con 0x7fbd5c1595a0 2023-10-06T03:46:13.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.408+0000 7fbd59d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbd5c0c6c60 0x7fbd5c0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:13.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.408+0000 7fbd59d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbd5c0c6c60 0x7fbd5c0c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34128/0 (socket says 172.21.15.55:34128) 2023-10-06T03:46:13.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.408+0000 7fbd59d9b700 1 -- 172.21.15.55:0/1467392976 learned_addr learned my addr 172.21.15.55:0/1467392976 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:13.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.408+0000 7fbd5a59c700 1 --2- 172.21.15.55:0/1467392976 >> v2:172.21.15.163:3300/0 conn(0x7fbd5c1595a0 0x7fbd5c0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:13.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.408+0000 7fbd59d9b700 1 -- 172.21.15.55:0/1467392976 >> v2:172.21.15.163:3300/0 conn(0x7fbd5c1595a0 msgr2=0x7fbd5c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:13.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.408+0000 7fbd59d9b700 1 --2- 172.21.15.55:0/1467392976 >> v2:172.21.15.163:3300/0 conn(0x7fbd5c1595a0 0x7fbd5c0b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:13.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.408+0000 7fbd59d9b700 1 -- 172.21.15.55:0/1467392976 >> v1:172.21.15.163:6789/0 conn(0x7fbd5c0b1de0 legacy=0x7fbd5c15d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:13.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.408+0000 7fbd59d9b700 1 -- 172.21.15.55:0/1467392976 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c0634f0 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.408+0000 7fbd59d9b700 1 --2- 172.21.15.55:0/1467392976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd5c0c6c60 0x7fbd5c0c8120 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0c8d10 tx=0x7fbd50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4257a7b40a674e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.409+0000 7fbd5959a700 1 -- 172.21.15.55:0/1467392976 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50009e70 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.409+0000 7fbd5959a700 1 -- 172.21.15.55:0/1467392976 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5000b860 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.409+0000 7fbd5959a700 1 -- 172.21.15.55:0/1467392976 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5000bc00 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.409+0000 7fbd5b59e700 1 -- 172.21.15.55:0/1467392976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd5c0c6c60 msgr2=0x7fbd5c0c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.409+0000 7fbd5b59e700 1 --2- 172.21.15.55:0/1467392976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd5c0c6c60 0x7fbd5c0c8120 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0c8d10 tx=0x7fbd50009250 comp rx=0 tx=0).stop 2023-10-06T03:46:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.409+0000 7fbd5b59e700 1 -- 172.21.15.55:0/1467392976 shutdown_connections 2023-10-06T03:46:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.409+0000 7fbd5b59e700 1 --2- 172.21.15.55:0/1467392976 >> v2:172.21.15.163:3300/0 conn(0x7fbd5c1595a0 0x7fbd5c0b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.409+0000 7fbd5b59e700 1 --2- 172.21.15.55:0/1467392976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd5c0c6c60 0x7fbd5c0c8120 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.409+0000 7fbd5b59e700 1 -- 172.21.15.55:0/1467392976 >> 172.21.15.55:0/1467392976 conn(0x7fbd5c06ddb0 msgr2=0x7fbd5c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:13.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.409+0000 7fbd5b59e700 1 -- 172.21.15.55:0/1467392976 shutdown_connections 2023-10-06T03:46:13.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.409+0000 7fbd5b59e700 1 -- 172.21.15.55:0/1467392976 wait complete. 2023-10-06T03:46:13.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.411+0000 7fbd5b59e700 1 Processor -- start 2023-10-06T03:46:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.411+0000 7fbd5b59e700 1 -- start start 2023-10-06T03:46:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.412+0000 7fbd5b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd5c0b1de0 0x7fbd5c1424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.412+0000 7fbd5b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd5c0c6c60 0x7fbd5c15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.412+0000 7fbd5b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd5c1595a0 0x7fbd5c156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.412+0000 7fbd5b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c15b950 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.412+0000 7fbd5b59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c0c3ba0 con 0x7fbd5c0b1de0 2023-10-06T03:46:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.412+0000 7fbd5b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c0c2f90 con 0x7fbd5c1595a0 2023-10-06T03:46:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.412+0000 7fbd59d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd5c0c6c60 0x7fbd5c15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.412+0000 7fbd59d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd5c0c6c60 0x7fbd5c15a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34130/0 (socket says 172.21.15.55:34130) 2023-10-06T03:46:13.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.412+0000 7fbd59d9b700 1 -- 172.21.15.55:0/2053009531 learned_addr learned my addr 172.21.15.55:0/2053009531 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.412+0000 7fbd59d9b700 1 -- 172.21.15.55:0/2053009531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd5c1595a0 msgr2=0x7fbd5c156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.412+0000 7fbd59d9b700 1 --2- 172.21.15.55:0/2053009531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd5c1595a0 0x7fbd5c156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.412+0000 7fbd59d9b700 1 -- 172.21.15.55:0/2053009531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd5c0b1de0 msgr2=0x7fbd5c1424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.412+0000 7fbd59d9b700 1 --2- 172.21.15.55:0/2053009531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd5c0b1de0 0x7fbd5c1424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.413+0000 7fbd59d9b700 1 -- 172.21.15.55:0/2053009531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd500094f0 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.413+0000 7fbd59d9b700 1 --2- 172.21.15.55:0/2053009531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd5c0c6c60 0x7fbd5c15a320 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbd50006fd0 tx=0x7fbd50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.413+0000 7fbd4a7fc700 1 -- 172.21.15.55:0/2053009531 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50011420 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.413+0000 7fbd4a7fc700 1 -- 172.21.15.55:0/2053009531 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd500115a0 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.413+0000 7fbd4a7fc700 1 -- 172.21.15.55:0/2053009531 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50011960 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.413+0000 7fbd5b59e700 1 -- 172.21.15.55:0/2053009531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5c0c3270 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.413+0000 7fbd5b59e700 1 -- 172.21.15.55:0/2053009531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5c0c3830 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.414+0000 7fbd5b59e700 1 -- 172.21.15.55:0/2053009531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5c04c3c0 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.415+0000 7fbd4a7fc700 1 -- 172.21.15.55:0/2053009531 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd500253f0 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.416+0000 7fbd4a7fc700 1 --2- 172.21.15.55:0/2053009531 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd3c066f30 0x7fbd3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:13.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.416+0000 7fbd4a7fc700 1 -- 172.21.15.55:0/2053009531 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7fbd500861f0 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.416+0000 7fbd5a59c700 1 --2- 172.21.15.55:0/2053009531 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd3c066f30 0x7fbd3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:13.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.417+0000 7fbd5a59c700 1 --2- 172.21.15.55:0/2053009531 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd3c066f30 0x7fbd3c0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009b10 tx=0x7fbd4c0091a0 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:13.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.420+0000 7fbd4a7fc700 1 -- 172.21.15.55:0/2053009531 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd500530f0 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.667+0000 7fbd5b59e700 1 -- 172.21.15.55:0/2053009531 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd5c0c3ce0 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.667+0000 7fbd4a7fc700 1 -- 172.21.15.55:0/2053009531 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fbd50056710 con 0x7fbd5c0c6c60 2023-10-06T03:46:13.669 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:46:13.669 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3493866751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3493866751},{"type":"v1","addr":"172.21.15.163:6836","nonce":3493866751}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3246061279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3246061279},{"type":"v1","addr":"172.21.15.55:6835","nonce":3246061279}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1366768080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1366768080},{"type":"v1","addr":"172.21.15.55:6834","nonce":1366768080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:46:07.292050+0000","modified":"2023-10-06T03:46:10.815261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6833/2287859778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2287859778},{"type":"v1","addr":"172.21.15.163:6833","nonce":2287859778}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T03:46:13.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.670+0000 7fbd5b59e700 1 -- 172.21.15.55:0/2053009531 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd3c066f30 msgr2=0x7fbd3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:13.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.670+0000 7fbd5b59e700 1 --2- 172.21.15.55:0/2053009531 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd3c066f30 0x7fbd3c0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009b10 tx=0x7fbd4c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:46:13.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.670+0000 7fbd5b59e700 1 -- 172.21.15.55:0/2053009531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd5c0c6c60 msgr2=0x7fbd5c15a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.670+0000 7fbd5b59e700 1 --2- 172.21.15.55:0/2053009531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd5c0c6c60 0x7fbd5c15a320 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fbd50006fd0 tx=0x7fbd50017000 comp rx=0 tx=0).stop 2023-10-06T03:46:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.670+0000 7fbd5b59e700 1 -- 172.21.15.55:0/2053009531 shutdown_connections 2023-10-06T03:46:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.670+0000 7fbd5b59e700 1 --2- 172.21.15.55:0/2053009531 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd3c066f30 0x7fbd3c0693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.670+0000 7fbd5b59e700 1 --2- 172.21.15.55:0/2053009531 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd5c1595a0 0x7fbd5c156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.670+0000 7fbd5b59e700 1 --2- 172.21.15.55:0/2053009531 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd5c0b1de0 0x7fbd5c1424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.670+0000 7fbd5b59e700 1 --2- 172.21.15.55:0/2053009531 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd5c0c6c60 0x7fbd5c15a320 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.670+0000 7fbd5b59e700 1 -- 172.21.15.55:0/2053009531 >> 172.21.15.55:0/2053009531 conn(0x7fbd5c06ddb0 msgr2=0x7fbd5c0cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.670+0000 7fbd5b59e700 1 -- 172.21.15.55:0/2053009531 shutdown_connections 2023-10-06T03:46:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:13.670+0000 7fbd5b59e700 1 -- 172.21.15.55:0/2053009531 wait complete. 2023-10-06T03:46:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 26 2023-10-06T03:46:13.765 INFO:tasks.ceph.mds.b.smithi163.stderr:2023-10-06T03:46:13.764+0000 7f95909f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:46:14.092 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:46:07.292050+0000', 'modified': '2023-10-06T03:46:10.815261+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5862}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5862': {'gid': 5862, 'name': 'b', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6833/2287859778', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6832', 'nonce': 2287859778}, {'type': 'v1', 'addr': '172.21.15.163:6833', 'nonce': 2287859778}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:46:14.092 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:46:14.092 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T03:46:14.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.282+0000 7fd023fff700 1 Processor -- start 2023-10-06T03:46:14.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.283+0000 7fd023fff700 1 -- start start 2023-10-06T03:46:14.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.284+0000 7fd023fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd02406a0a0 0x7fd02415b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.284+0000 7fd023fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02406ae70 con 0x7fd02406dd20 2023-10-06T03:46:14.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.284+0000 7fd023fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0240c8a60 con 0x7fd02406d950 2023-10-06T03:46:14.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.284+0000 7fd023fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd02406a540 con 0x7fd02406a0a0 2023-10-06T03:46:14.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.284+0000 7fd0237fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd02406dd20 0x7fd02415b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55690/0 (socket says 172.21.15.55:55690) 2023-10-06T03:46:14.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.284+0000 7fd0237fe700 1 -- 172.21.15.55:0/720104536 learned_addr learned my addr 172.21.15.55:0/720104536 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:14.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.285+0000 7fd0227fc700 1 --2- 172.21.15.55:0/720104536 >> v2:172.21.15.163:3300/0 conn(0x7fd02406a0a0 0x7fd02415b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.285+0000 7fd021ffb700 1 -- 172.21.15.55:0/720104536 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399565724 0 0) 0x7fd02406ae70 con 0x7fd02406dd20 2023-10-06T03:46:14.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.285+0000 7fd021ffb700 1 -- 172.21.15.55:0/720104536 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd00c003570 con 0x7fd02406dd20 2023-10-06T03:46:14.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.285+0000 7fd0227fc700 1 -- 172.21.15.55:0/720104536 >> v1:172.21.15.163:6789/0 conn(0x7fd02406d950 legacy=0x7fd0240c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.285+0000 7fd0227fc700 1 -- 172.21.15.55:0/720104536 >> v1:172.21.15.55:6789/0 conn(0x7fd02406dd20 legacy=0x7fd02415b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.285+0000 7fd021ffb700 1 -- 172.21.15.55:0/720104536 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3029505588 0 0) 0x7fd00c003570 con 0x7fd02406dd20 2023-10-06T03:46:14.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.285+0000 7fd0227fc700 1 -- 172.21.15.55:0/720104536 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02406a820 con 0x7fd02406a0a0 2023-10-06T03:46:14.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.285+0000 7fd0227fc700 1 --2- 172.21.15.55:0/720104536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02406a0a0 0x7fd02415b410 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd0180098e0 tx=0x7fd0180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58937865113ddda6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:14.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.286+0000 7fd021ffb700 1 -- 172.21.15.55:0/720104536 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0017f0 con 0x7fd02406a0a0 2023-10-06T03:46:14.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.286+0000 7fd021ffb700 1 -- 172.21.15.55:0/720104536 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd018011420 con 0x7fd02406a0a0 2023-10-06T03:46:14.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.286+0000 7fd021ffb700 1 -- 172.21.15.55:0/720104536 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0180117c0 con 0x7fd02406a0a0 2023-10-06T03:46:14.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.286+0000 7fd023fff700 1 -- 172.21.15.55:0/720104536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02406a0a0 msgr2=0x7fd02415b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.286+0000 7fd023fff700 1 --2- 172.21.15.55:0/720104536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02406a0a0 0x7fd02415b410 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd0180098e0 tx=0x7fd0180091a0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.286+0000 7fd023fff700 1 -- 172.21.15.55:0/720104536 shutdown_connections 2023-10-06T03:46:14.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.286+0000 7fd023fff700 1 --2- 172.21.15.55:0/720104536 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02406a0a0 0x7fd02415b410 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.286+0000 7fd023fff700 1 -- 172.21.15.55:0/720104536 >> 172.21.15.55:0/720104536 conn(0x7fd0240cc1f0 msgr2=0x7fd0240cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:14.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.286+0000 7fd023fff700 1 -- 172.21.15.55:0/720104536 shutdown_connections 2023-10-06T03:46:14.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.286+0000 7fd023fff700 1 -- 172.21.15.55:0/720104536 wait complete. 2023-10-06T03:46:14.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.288+0000 7fd023fff700 1 Processor -- start 2023-10-06T03:46:14.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.288+0000 7fd023fff700 1 -- start start 2023-10-06T03:46:14.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd023fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd02406a0a0 0x7fd02415ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd023fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd02406d950 0x7fd0240c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd023fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02406dd20 0x7fd024153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd023fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240c9dc0 con 0x7fd02406d950 2023-10-06T03:46:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd023fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240c5160 con 0x7fd02406dd20 2023-10-06T03:46:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd023fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0240c53d0 con 0x7fd02406a0a0 2023-10-06T03:46:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd0227fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd02406d950 0x7fd0240c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd0227fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd02406d950 0x7fd0240c4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34138/0 (socket says 172.21.15.55:34138) 2023-10-06T03:46:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd0227fc700 1 -- 172.21.15.55:0/344366820 learned_addr learned my addr 172.21.15.55:0/344366820 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd0237fe700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02406dd20 0x7fd024153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd022ffd700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd02406a0a0 0x7fd02415ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd0227fc700 1 -- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd02406a0a0 msgr2=0x7fd02415ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd0227fc700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd02406a0a0 0x7fd02415ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd0227fc700 1 -- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02406dd20 msgr2=0x7fd024153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd0227fc700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02406dd20 0x7fd024153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd0227fc700 1 -- 172.21.15.55:0/344366820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd018009420 con 0x7fd02406d950 2023-10-06T03:46:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.289+0000 7fd022ffd700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd02406a0a0 0x7fd02415ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:46:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.290+0000 7fd0227fc700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd02406d950 0x7fd0240c4c20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd0180091e0 tx=0x7fd018016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.290+0000 7fd00affd700 1 -- 172.21.15.55:0/344366820 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01801cc90 con 0x7fd02406d950 2023-10-06T03:46:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.290+0000 7fd00affd700 1 -- 172.21.15.55:0/344366820 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01801cdf0 con 0x7fd02406d950 2023-10-06T03:46:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.290+0000 7fd00affd700 1 -- 172.21.15.55:0/344366820 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01801a6a0 con 0x7fd02406d950 2023-10-06T03:46:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.290+0000 7fd023fff700 1 -- 172.21.15.55:0/344366820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0240c56b0 con 0x7fd02406d950 2023-10-06T03:46:14.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.290+0000 7fd023fff700 1 -- 172.21.15.55:0/344366820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0240c5c70 con 0x7fd02406d950 2023-10-06T03:46:14.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.292+0000 7fd00affd700 1 -- 172.21.15.55:0/344366820 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd01801aa90 con 0x7fd02406d950 2023-10-06T03:46:14.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.293+0000 7fd00affd700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd010066f30 0x7fd0100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.293+0000 7fd00affd700 1 -- 172.21.15.55:0/344366820 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7fd018086350 con 0x7fd02406d950 2023-10-06T03:46:14.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.294+0000 7fd022ffd700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd010066f30 0x7fd0100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.294+0000 7fd022ffd700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd010066f30 0x7fd0100693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd0240c86b0 tx=0x7fd014009040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:14.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.295+0000 7fd037b4cb80 1 Processor -- start 2023-10-06T03:46:14.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.295+0000 7fd037b4cb80 1 -- start start 2023-10-06T03:46:14.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.296+0000 7fd037b4cb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc76be50 0x55dcbc746cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.296+0000 7fd037b4cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74c2c0 0x55dcbc747200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.296+0000 7fd037b4cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74da10 0x55dcbc747740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.296+0000 7fd037b4cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbc6a1a70 con 0x55dcbc74c2c0 2023-10-06T03:46:14.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.296+0000 7fd037b4cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbc69f270 con 0x55dcbc74da10 2023-10-06T03:46:14.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.296+0000 7fd037b4cb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dcbc69b060 con 0x55dcbc76be50 2023-10-06T03:46:14.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.296+0000 7fd0227fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74c2c0 0x55dcbc747200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.296+0000 7fd0227fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74c2c0 0x55dcbc747200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34154/0 (socket says 172.21.15.55:34154) 2023-10-06T03:46:14.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.296+0000 7fd0227fc700 1 -- 172.21.15.55:0/1632740593 learned_addr learned my addr 172.21.15.55:0/1632740593 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:14.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.296+0000 7fd022ffd700 1 --2- 172.21.15.55:0/1632740593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc76be50 0x55dcbc746cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.296+0000 7fd0237fe700 1 --2- 172.21.15.55:0/1632740593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74da10 0x55dcbc747740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.297+0000 7fd0227fc700 1 -- 172.21.15.55:0/1632740593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc76be50 msgr2=0x55dcbc746cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:14.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.297+0000 7fd0227fc700 1 --2- 172.21.15.55:0/1632740593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc76be50 0x55dcbc746cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.297+0000 7fd0227fc700 1 -- 172.21.15.55:0/1632740593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74da10 msgr2=0x55dcbc747740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:14.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.297+0000 7fd0227fc700 1 --2- 172.21.15.55:0/1632740593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74da10 0x55dcbc747740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.297+0000 7fd0227fc700 1 -- 172.21.15.55:0/1632740593 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcbc535270 con 0x55dcbc74c2c0 2023-10-06T03:46:14.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.297+0000 7fd0227fc700 1 --2- 172.21.15.55:0/1632740593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74c2c0 0x55dcbc747200 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd0180093a0 tx=0x7fd018084460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18debc2faaaf510a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:14.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.297+0000 7fd008ff9700 1 -- 172.21.15.55:0/1632740593 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01805aea0 con 0x55dcbc74c2c0 2023-10-06T03:46:14.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.297+0000 7fd008ff9700 1 -- 172.21.15.55:0/1632740593 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd018057610 con 0x55dcbc74c2c0 2023-10-06T03:46:14.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.297+0000 7fd008ff9700 1 -- 172.21.15.55:0/1632740593 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018056f10 con 0x55dcbc74c2c0 2023-10-06T03:46:14.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.298+0000 7fd037b4cb80 1 -- 172.21.15.55:0/1632740593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74c2c0 msgr2=0x55dcbc747200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.298+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/1632740593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74c2c0 0x55dcbc747200 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd0180093a0 tx=0x7fd018084460 comp rx=0 tx=0).stop 2023-10-06T03:46:14.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.298+0000 7fd037b4cb80 1 -- 172.21.15.55:0/1632740593 shutdown_connections 2023-10-06T03:46:14.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.298+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/1632740593 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74da10 0x55dcbc747740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.298+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/1632740593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc76be50 0x55dcbc746cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.298+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/1632740593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74c2c0 0x55dcbc747200 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.298+0000 7fd037b4cb80 1 -- 172.21.15.55:0/1632740593 >> 172.21.15.55:0/1632740593 conn(0x55dcbc74be80 msgr2=0x55dcbc74c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:14.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.298+0000 7fd037b4cb80 1 -- 172.21.15.55:0/1632740593 shutdown_connections 2023-10-06T03:46:14.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.298+0000 7fd037b4cb80 1 -- 172.21.15.55:0/1632740593 wait complete. 2023-10-06T03:46:14.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.299+0000 7fd037b4cb80 1 -- start start 2023-10-06T03:46:14.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.299+0000 7fd037b4cb80 1 Processor -- start 2023-10-06T03:46:14.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.300+0000 7fd037b4cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74c2c0 0x55dcbc7739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.300+0000 7fd037b4cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74da10 0x55dcbc76c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.300+0000 7fd037b4cb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc76be50 0x55dcbc779290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.300+0000 7fd037b4cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbc646890 con 0x55dcbc74da10 2023-10-06T03:46:14.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.300+0000 7fd037b4cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbc661ea0 con 0x55dcbc74c2c0 2023-10-06T03:46:14.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.300+0000 7fd037b4cb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dcbc51de30 con 0x55dcbc76be50 2023-10-06T03:46:14.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.300+0000 7fd037b4cb80 10 client.0 _ll_register_callbacks cb 0x55dcbc70e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:46:14.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.300+0000 7fd0227fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74da10 0x55dcbc76c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.300+0000 7fd0227fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74da10 0x55dcbc76c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34162/0 (socket says 172.21.15.55:34162) 2023-10-06T03:46:14.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.300+0000 7fd0227fc700 1 -- 172.21.15.55:0/1470114987 learned_addr learned my addr 172.21.15.55:0/1470114987 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:14.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.300+0000 7fd022ffd700 1 --2- 172.21.15.55:0/1470114987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74c2c0 0x55dcbc7739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.300+0000 7fd0237fe700 1 --2- 172.21.15.55:0/1470114987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc76be50 0x55dcbc779290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.301+0000 7fd0227fc700 1 -- 172.21.15.55:0/1470114987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc76be50 msgr2=0x55dcbc779290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.301+0000 7fd0227fc700 1 --2- 172.21.15.55:0/1470114987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc76be50 0x55dcbc779290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.301+0000 7fd0227fc700 1 -- 172.21.15.55:0/1470114987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74c2c0 msgr2=0x55dcbc7739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.301+0000 7fd0227fc700 1 --2- 172.21.15.55:0/1470114987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74c2c0 0x55dcbc7739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.301+0000 7fd0227fc700 1 -- 172.21.15.55:0/1470114987 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0240c5c70 con 0x55dcbc74da10 2023-10-06T03:46:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.301+0000 7fd0227fc700 1 --2- 172.21.15.55:0/1470114987 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74da10 0x55dcbc76c570 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd018053410 tx=0x7fd018058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e11379f608cf66 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.301+0000 7fd0017fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:46:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.301+0000 7fd0017fa700 1 -- 172.21.15.55:0/1470114987 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd018056f10 con 0x55dcbc74da10 2023-10-06T03:46:14.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.301+0000 7fd0017fa700 1 -- 172.21.15.55:0/1470114987 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7fd0180860b0 con 0x55dcbc74da10 2023-10-06T03:46:14.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.301+0000 7fd037b4cb80 1 -- 172.21.15.55:0/1470114987 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcbc73f940 con 0x55dcbc74da10 2023-10-06T03:46:14.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.301+0000 7fd0017fa700 1 -- 172.21.15.55:0/1470114987 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd01805c0c0 con 0x55dcbc74da10 2023-10-06T03:46:14.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.301+0000 7fd037b4cb80 10 client.6000 fetch_fsmap learned FSMap version 26 2023-10-06T03:46:14.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.301+0000 7fd037b4cb80 1 -- 172.21.15.55:0/1470114987 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55dcbc4d7fe0 con 0x55dcbc74da10 2023-10-06T03:46:14.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.301+0000 7fd0017fa700 1 -- 172.21.15.55:0/1470114987 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dcbc73f940 con 0x55dcbc74da10 2023-10-06T03:46:14.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.302+0000 7fd0017fa700 1 -- 172.21.15.55:0/1470114987 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd01806d100 con 0x55dcbc74da10 2023-10-06T03:46:14.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.302+0000 7fd037b4cb80 10 client.6000 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T03:46:14.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.302+0000 7fd037b4cb80 10 client.6000 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T03:46:14.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.302+0000 7fd037b4cb80 20 client.6000 populate_metadata read hostname 'smithi055' 2023-10-06T03:46:14.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.302+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/1470114987 >> [v2:172.21.15.163:6832/2287859778,v1:172.21.15.163:6833/2287859778] conn(0x55dcbc795600 0x55dcbc7959d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.302+0000 7fd037b4cb80 4 client.6000 mds_command: new command op to 5862 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:46:14.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.302+0000 7fd037b4cb80 1 -- 172.21.15.55:0/1470114987 --> [v2:172.21.15.163:6832/2287859778,v1:172.21.15.163:6833/2287859778] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcbc71c870 con 0x55dcbc795600 2023-10-06T03:46:14.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.302+0000 7fd022ffd700 1 --2- 172.21.15.55:0/1470114987 >> [v2:172.21.15.163:6832/2287859778,v1:172.21.15.163:6833/2287859778] conn(0x55dcbc795600 0x55dcbc7959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.303+0000 7fd022ffd700 1 --2- 172.21.15.55:0/1470114987 >> [v2:172.21.15.163:6832/2287859778,v1:172.21.15.163:6833/2287859778] conn(0x55dcbc795600 0x55dcbc7959d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2cbcf901a2407ed server_cookie=9d8519c0a4ab6d8 in_seq=0 out_seq=0 2023-10-06T03:46:14.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.303+0000 7fd0017fa700 10 client.6000 ms_handle_connect on v2:172.21.15.163:6832/2287859778 2023-10-06T03:46:14.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd0017fa700 1 -- 172.21.15.55:0/1470114987 <== mds.0 v2:172.21.15.163:6832/2287859778 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55dcbc71c870 con 0x55dcbc795600 2023-10-06T03:46:14.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd0017fa700 10 client.6000 handle_command_reply: tid=0 2023-10-06T03:46:14.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd037b4cb80 1 client.6000 shutdown 2023-10-06T03:46:14.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd037b4cb80 1 -- 172.21.15.55:0/1470114987 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74da10 msgr2=0x55dcbc76c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/1470114987 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74da10 0x55dcbc76c570 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd018053410 tx=0x7fd018058000 comp rx=0 tx=0).stop 2023-10-06T03:46:14.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd037b4cb80 1 -- 172.21.15.55:0/1470114987 shutdown_connections 2023-10-06T03:46:14.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/1470114987 >> [v2:172.21.15.163:6832/2287859778,v1:172.21.15.163:6833/2287859778] conn(0x55dcbc795600 0x55dcbc7959d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/1470114987 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc76be50 0x55dcbc779290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/1470114987 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74c2c0 0x55dcbc7739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/1470114987 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74da10 0x55dcbc76c570 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd0017fa700 0 client.6000 ms_handle_reset on v2:172.21.15.163:6832/2287859778 2023-10-06T03:46:14.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd037b4cb80 1 -- 172.21.15.55:0/1470114987 >> 172.21.15.55:0/1470114987 conn(0x55dcbc74be80 msgr2=0x55dcbc74cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:14.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd037b4cb80 1 -- 172.21.15.55:0/1470114987 shutdown_connections 2023-10-06T03:46:14.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd037b4cb80 1 -- 172.21.15.55:0/1470114987 wait complete. 2023-10-06T03:46:14.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.304+0000 7fd037b4cb80 20 client.6000 trim_cache size 0 max 16384 2023-10-06T03:46:14.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.322+0000 7fd037b4cb80 1 Processor -- start 2023-10-06T03:46:14.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.322+0000 7fd037b4cb80 1 -- start start 2023-10-06T03:46:14.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.322+0000 7fd037b4cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74c2c0 0x55dcbc76c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.322+0000 7fd037b4cb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc74da10 0x55dcbc74c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.323+0000 7fd037b4cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc76be50 0x55dcbc74cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.323+0000 7fd037b4cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbc76bb40 con 0x55dcbc74c2c0 2023-10-06T03:46:14.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.323+0000 7fd037b4cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbc748fe0 con 0x55dcbc76be50 2023-10-06T03:46:14.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.323+0000 7fd037b4cb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dcbc7420f0 con 0x55dcbc74da10 2023-10-06T03:46:14.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.323+0000 7fd022ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74c2c0 0x55dcbc76c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.323+0000 7fd022ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74c2c0 0x55dcbc76c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34178/0 (socket says 172.21.15.55:34178) 2023-10-06T03:46:14.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.323+0000 7fd022ffd700 1 -- 172.21.15.55:0/2374287220 learned_addr learned my addr 172.21.15.55:0/2374287220 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:14.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.323+0000 7fd0237fe700 1 --2- 172.21.15.55:0/2374287220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc76be50 0x55dcbc74cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.323+0000 7fd022ffd700 1 -- 172.21.15.55:0/2374287220 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc74da10 msgr2=0x55dcbc74c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:14.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.323+0000 7fd022ffd700 1 --2- 172.21.15.55:0/2374287220 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc74da10 0x55dcbc74c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.323+0000 7fd022ffd700 1 -- 172.21.15.55:0/2374287220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc76be50 msgr2=0x55dcbc74cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:14.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.323+0000 7fd022ffd700 1 --2- 172.21.15.55:0/2374287220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc76be50 0x55dcbc74cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.323+0000 7fd022ffd700 1 -- 172.21.15.55:0/2374287220 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcbc669780 con 0x55dcbc74c2c0 2023-10-06T03:46:14.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.323+0000 7fd022ffd700 1 --2- 172.21.15.55:0/2374287220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74c2c0 0x55dcbc76c3f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd014007fd0 tx=0x7fd014013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b12da7ab940d873 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:14.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.324+0000 7fd0097fa700 1 -- 172.21.15.55:0/2374287220 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01402a070 con 0x55dcbc74c2c0 2023-10-06T03:46:14.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.324+0000 7fd0097fa700 1 -- 172.21.15.55:0/2374287220 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0140250e0 con 0x55dcbc74c2c0 2023-10-06T03:46:14.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.324+0000 7fd0097fa700 1 -- 172.21.15.55:0/2374287220 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014006a30 con 0x55dcbc74c2c0 2023-10-06T03:46:14.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.324+0000 7fd037b4cb80 1 -- 172.21.15.55:0/2374287220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74c2c0 msgr2=0x55dcbc76c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.324+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/2374287220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74c2c0 0x55dcbc76c3f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd014007fd0 tx=0x7fd014013000 comp rx=0 tx=0).stop 2023-10-06T03:46:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.324+0000 7fd037b4cb80 1 -- 172.21.15.55:0/2374287220 shutdown_connections 2023-10-06T03:46:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.324+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/2374287220 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc76be50 0x55dcbc74cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.324+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/2374287220 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc74c2c0 0x55dcbc76c3f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.324+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/2374287220 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc74da10 0x55dcbc74c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.324+0000 7fd037b4cb80 1 -- 172.21.15.55:0/2374287220 >> 172.21.15.55:0/2374287220 conn(0x55dcbc74be80 msgr2=0x55dcbc792a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.324+0000 7fd037b4cb80 1 -- 172.21.15.55:0/2374287220 shutdown_connections 2023-10-06T03:46:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.324+0000 7fd037b4cb80 1 -- 172.21.15.55:0/2374287220 wait complete. 2023-10-06T03:46:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.325+0000 7fd037b4cb80 1 -- start start 2023-10-06T03:46:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.325+0000 7fd037b4cb80 1 Processor -- start 2023-10-06T03:46:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.326+0000 7fd037b4cb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc74c2c0 0x55dcbc7ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.326+0000 7fd037b4cb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74da10 0x55dcbc7475b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.326+0000 7fd037b4cb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc76be50 0x55dcbc777510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.326+0000 7fd037b4cb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbc7edb20 con 0x55dcbc76be50 2023-10-06T03:46:14.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.326+0000 7fd037b4cb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcbc76ade0 con 0x55dcbc74da10 2023-10-06T03:46:14.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.326+0000 7fd037b4cb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dcbc7aa490 con 0x55dcbc74c2c0 2023-10-06T03:46:14.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.326+0000 7fd037b4cb80 10 client.0 _ll_register_callbacks cb 0x55dcbc70e020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:46:14.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.326+0000 7fd0237fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc76be50 0x55dcbc777510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.326+0000 7fd0237fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc76be50 0x55dcbc777510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34180/0 (socket says 172.21.15.55:34180) 2023-10-06T03:46:14.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.326+0000 7fd0237fe700 1 -- 172.21.15.55:0/731087402 learned_addr learned my addr 172.21.15.55:0/731087402 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:14.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.327+0000 7fd022ffd700 1 --2- 172.21.15.55:0/731087402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc74c2c0 0x55dcbc7ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.327+0000 7fd0227fc700 1 --2- 172.21.15.55:0/731087402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74da10 0x55dcbc7475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.327+0000 7fd0237fe700 1 -- 172.21.15.55:0/731087402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc74c2c0 msgr2=0x55dcbc7ecfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:14.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.327+0000 7fd0237fe700 1 --2- 172.21.15.55:0/731087402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc74c2c0 0x55dcbc7ecfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.327+0000 7fd0237fe700 1 -- 172.21.15.55:0/731087402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74da10 msgr2=0x55dcbc7475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:14.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.327+0000 7fd0237fe700 1 --2- 172.21.15.55:0/731087402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74da10 0x55dcbc7475b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.327+0000 7fd0237fe700 1 -- 172.21.15.55:0/731087402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0240c8a60 con 0x55dcbc76be50 2023-10-06T03:46:14.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.327+0000 7fd0237fe700 1 --2- 172.21.15.55:0/731087402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc76be50 0x55dcbc777510 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd01c007350 tx=0x7fd01c009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c413241e7aa81d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:14.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.327+0000 7fd0017fa700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:46:14.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.327+0000 7fd0017fa700 1 -- 172.21.15.55:0/731087402 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c017670 con 0x55dcbc76be50 2023-10-06T03:46:14.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.327+0000 7fd037b4cb80 1 -- 172.21.15.55:0/731087402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0240ca240 con 0x55dcbc76be50 2023-10-06T03:46:14.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.327+0000 7fd0017fa700 1 -- 172.21.15.55:0/731087402 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7fd01c0177d0 con 0x55dcbc76be50 2023-10-06T03:46:14.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.328+0000 7fd0017fa700 1 -- 172.21.15.55:0/731087402 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd01c017ba0 con 0x55dcbc76be50 2023-10-06T03:46:14.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.328+0000 7fd037b4cb80 10 client.6006 fetch_fsmap learned FSMap version 26 2023-10-06T03:46:14.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.328+0000 7fd037b4cb80 1 -- 172.21.15.55:0/731087402 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55dcbc650470 con 0x55dcbc76be50 2023-10-06T03:46:14.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.328+0000 7fd0017fa700 1 -- 172.21.15.55:0/731087402 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0240ca240 con 0x55dcbc76be50 2023-10-06T03:46:14.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.328+0000 7fd0017fa700 1 -- 172.21.15.55:0/731087402 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 26) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd01c017d00 con 0x55dcbc76be50 2023-10-06T03:46:14.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.328+0000 7fd037b4cb80 10 client.6006 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T03:46:14.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.328+0000 7fd037b4cb80 10 client.6006 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T03:46:14.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.328+0000 7fd037b4cb80 20 client.6006 populate_metadata read hostname 'smithi055' 2023-10-06T03:46:14.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.328+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/731087402 >> [v2:172.21.15.163:6832/2287859778,v1:172.21.15.163:6833/2287859778] conn(0x55dcbc76c4e0 0x55dcbc76c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.328+0000 7fd037b4cb80 4 client.6006 mds_command: new command op to 5862 tid=0[{"prefix": "status"}] 2023-10-06T03:46:14.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.328+0000 7fd037b4cb80 1 -- 172.21.15.55:0/731087402 --> [v2:172.21.15.163:6832/2287859778,v1:172.21.15.163:6833/2287859778] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fd018057610 con 0x55dcbc76c4e0 2023-10-06T03:46:14.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.328+0000 7fd022ffd700 1 --2- 172.21.15.55:0/731087402 >> [v2:172.21.15.163:6832/2287859778,v1:172.21.15.163:6833/2287859778] conn(0x55dcbc76c4e0 0x55dcbc76c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.329+0000 7fd022ffd700 1 --2- 172.21.15.55:0/731087402 >> [v2:172.21.15.163:6832/2287859778,v1:172.21.15.163:6833/2287859778] conn(0x55dcbc76c4e0 0x55dcbc76c8b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b09f9280b61add01 server_cookie=3d82316c66ed9141 in_seq=0 out_seq=0 2023-10-06T03:46:14.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.329+0000 7fd0017fa700 10 client.6006 ms_handle_connect on v2:172.21.15.163:6832/2287859778 2023-10-06T03:46:14.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.329+0000 7fd0017fa700 1 -- 172.21.15.55:0/731087402 <== mds.0 v2:172.21.15.163:6832/2287859778 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x7fd018057610 con 0x55dcbc76c4e0 2023-10-06T03:46:14.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.329+0000 7fd0017fa700 10 client.6006 handle_command_reply: tid=0 2023-10-06T03:46:14.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.329+0000 7fd037b4cb80 1 client.6006 shutdown 2023-10-06T03:46:14.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.329+0000 7fd037b4cb80 1 -- 172.21.15.55:0/731087402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc76be50 msgr2=0x55dcbc777510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.329+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/731087402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc76be50 0x55dcbc777510 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd01c007350 tx=0x7fd01c009040 comp rx=0 tx=0).stop 2023-10-06T03:46:14.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.330+0000 7fd037b4cb80 1 -- 172.21.15.55:0/731087402 shutdown_connections 2023-10-06T03:46:14.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.330+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/731087402 >> [v2:172.21.15.163:6832/2287859778,v1:172.21.15.163:6833/2287859778] conn(0x55dcbc76c4e0 0x55dcbc76c8b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.330+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/731087402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55dcbc76be50 0x55dcbc777510 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.330+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/731087402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55dcbc74c2c0 0x55dcbc7ecfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.330+0000 7fd037b4cb80 1 --2- 172.21.15.55:0/731087402 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55dcbc74da10 0x55dcbc7475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.330+0000 7fd037b4cb80 1 -- 172.21.15.55:0/731087402 >> 172.21.15.55:0/731087402 conn(0x55dcbc74be80 msgr2=0x55dcbc793e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:14.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.330+0000 7fd0017fa700 0 client.6006 ms_handle_reset on v2:172.21.15.163:6832/2287859778 2023-10-06T03:46:14.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.330+0000 7fd037b4cb80 1 -- 172.21.15.55:0/731087402 shutdown_connections 2023-10-06T03:46:14.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.330+0000 7fd037b4cb80 1 -- 172.21.15.55:0/731087402 wait complete. 2023-10-06T03:46:14.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.330+0000 7fd037b4cb80 20 client.6006 trim_cache size 0 max 16384 2023-10-06T03:46:14.337 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:46:14.338 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "8132c8ba-ee1c-41a2-b781-913925034ab6", 2023-10-06T03:46:14.338 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2023-10-06T03:46:14.338 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 5862, 2023-10-06T03:46:14.338 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2023-10-06T03:46:14.338 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2023-10-06T03:46:14.338 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2023-10-06T03:46:14.338 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 6.5461277620000002, 2023-10-06T03:46:14.338 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 26, 2023-10-06T03:46:14.338 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 64, 2023-10-06T03:46:14.338 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 64, 2023-10-06T03:46:14.338 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 12.294150481000001 2023-10-06T03:46:14.339 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:46:14.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.330+0000 7fd023fff700 1 -- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd010066f30 msgr2=0x7fd0100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.330+0000 7fd023fff700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd010066f30 0x7fd0100693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd0240c86b0 tx=0x7fd014009040 comp rx=0 tx=0).stop 2023-10-06T03:46:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.330+0000 7fd023fff700 1 -- 172.21.15.55:0/344366820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd02406d950 msgr2=0x7fd0240c4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.330+0000 7fd023fff700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd02406d950 0x7fd0240c4c20 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd0180091e0 tx=0x7fd018016000 comp rx=0 tx=0).stop 2023-10-06T03:46:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.331+0000 7fd023fff700 1 -- 172.21.15.55:0/344366820 shutdown_connections 2023-10-06T03:46:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.331+0000 7fd023fff700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd010066f30 0x7fd0100693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.331+0000 7fd023fff700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd02406dd20 0x7fd024153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.331+0000 7fd023fff700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd02406a0a0 0x7fd02415ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.331+0000 7fd023fff700 1 --2- 172.21.15.55:0/344366820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd02406d950 0x7fd0240c4c20 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.331+0000 7fd023fff700 1 -- 172.21.15.55:0/344366820 >> 172.21.15.55:0/344366820 conn(0x7fd0240cc1f0 msgr2=0x7fd0240b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.331+0000 7fd023fff700 1 -- 172.21.15.55:0/344366820 shutdown_connections 2023-10-06T03:46:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.331+0000 7fd023fff700 1 -- 172.21.15.55:0/344366820 wait complete. 2023-10-06T03:46:14.349 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:46:14.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.534+0000 7fd1be39d700 1 Processor -- start 2023-10-06T03:46:14.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.534+0000 7fd1be39d700 1 -- start start 2023-10-06T03:46:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.535+0000 7fd1be39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd1b8150550 0x7fd1b8157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.535+0000 7fd1be39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd1b80c7410 0x7fd1b8069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.536+0000 7fd1be39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b806e2a0 con 0x7fd1b8150180 2023-10-06T03:46:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.536+0000 7fd1be39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1b806b2e0 con 0x7fd1b8150550 2023-10-06T03:46:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.536+0000 7fd1be39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1b80c78e0 con 0x7fd1b80c7410 2023-10-06T03:46:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.536+0000 7fd1bcb9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd1b8150550 0x7fd1b8157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.536+0000 7fd1bd39b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd1b8150180 0x7fd1b80c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55696/0 (socket says 172.21.15.55:55696) 2023-10-06T03:46:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.536+0000 7fd1bcb9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd1b8150550 0x7fd1b8157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34196/0 (socket says 172.21.15.55:34196) 2023-10-06T03:46:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.536+0000 7fd1bd39b700 1 -- 172.21.15.55:0/3736159336 learned_addr learned my addr 172.21.15.55:0/3736159336 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.536+0000 7fd1bdb9c700 1 --2- 172.21.15.55:0/3736159336 >> v2:172.21.15.163:3300/0 conn(0x7fd1b80c7410 0x7fd1b8069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.536+0000 7fd1affff700 1 -- 172.21.15.55:0/3736159336 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830437976 0 0) 0x7fd1b806e2a0 con 0x7fd1b8150180 2023-10-06T03:46:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.536+0000 7fd1bcb9a700 1 -- 172.21.15.55:0/3736159336 >> v2:172.21.15.163:3300/0 conn(0x7fd1b80c7410 msgr2=0x7fd1b8069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.536+0000 7fd1bcb9a700 1 --2- 172.21.15.55:0/3736159336 >> v2:172.21.15.163:3300/0 conn(0x7fd1b80c7410 0x7fd1b8069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.536+0000 7fd1bcb9a700 1 -- 172.21.15.55:0/3736159336 >> v1:172.21.15.55:6789/0 conn(0x7fd1b8150180 legacy=0x7fd1b80c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.536+0000 7fd1bcb9a700 1 -- 172.21.15.55:0/3736159336 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b80c7bc0 con 0x7fd1b8150550 2023-10-06T03:46:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.537+0000 7fd1bcb9a700 1 --2- 172.21.15.55:0/3736159336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b8150550 0x7fd1b8157770 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8156270 tx=0x7fd1a800a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941e2d5041ebae3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.537+0000 7fd1affff700 1 -- 172.21.15.55:0/3736159336 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a800c970 con 0x7fd1b8150550 2023-10-06T03:46:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.537+0000 7fd1affff700 1 -- 172.21.15.55:0/3736159336 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a800cad0 con 0x7fd1b8150550 2023-10-06T03:46:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.537+0000 7fd1affff700 1 -- 172.21.15.55:0/3736159336 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a800ce70 con 0x7fd1b8150550 2023-10-06T03:46:14.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.537+0000 7fd1be39d700 1 -- 172.21.15.55:0/3736159336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b8150550 msgr2=0x7fd1b8157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.537+0000 7fd1be39d700 1 --2- 172.21.15.55:0/3736159336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b8150550 0x7fd1b8157770 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8156270 tx=0x7fd1a800a330 comp rx=0 tx=0).stop 2023-10-06T03:46:14.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.538+0000 7fd1be39d700 1 -- 172.21.15.55:0/3736159336 shutdown_connections 2023-10-06T03:46:14.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.538+0000 7fd1be39d700 1 --2- 172.21.15.55:0/3736159336 >> v2:172.21.15.163:3300/0 conn(0x7fd1b80c7410 0x7fd1b8069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.538+0000 7fd1be39d700 1 --2- 172.21.15.55:0/3736159336 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b8150550 0x7fd1b8157770 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.538+0000 7fd1be39d700 1 -- 172.21.15.55:0/3736159336 >> 172.21.15.55:0/3736159336 conn(0x7fd1b80cc1f0 msgr2=0x7fd1b80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.538+0000 7fd1be39d700 1 -- 172.21.15.55:0/3736159336 shutdown_connections 2023-10-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.538+0000 7fd1be39d700 1 -- 172.21.15.55:0/3736159336 wait complete. 2023-10-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.539+0000 7fd1be39d700 1 Processor -- start 2023-10-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.539+0000 7fd1be39d700 1 -- start start 2023-10-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.540+0000 7fd1be39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b80c7410 0x7fd1b814a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.540+0000 7fd1be39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1b8150180 0x7fd1b80aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.540+0000 7fd1be39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b8150550 0x7fd1b80b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.540+0000 7fd1be39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b80b0180 con 0x7fd1b80c7410 2023-10-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.540+0000 7fd1be39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b80abce0 con 0x7fd1b8150180 2023-10-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.540+0000 7fd1be39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b80af8e0 con 0x7fd1b8150550 2023-10-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.541+0000 7fd1bd39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b80c7410 0x7fd1b814a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.541+0000 7fd1bd39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b80c7410 0x7fd1b814a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34208/0 (socket says 172.21.15.55:34208) 2023-10-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.541+0000 7fd1bd39b700 1 -- 172.21.15.55:0/1156640850 learned_addr learned my addr 172.21.15.55:0/1156640850 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.541+0000 7fd1bcb9a700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1b8150180 0x7fd1b80aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.541+0000 7fd1bdb9c700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b8150550 0x7fd1b80b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.541+0000 7fd1bd39b700 1 -- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b8150550 msgr2=0x7fd1b80b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.541+0000 7fd1bd39b700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b8150550 0x7fd1b80b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.541+0000 7fd1bd39b700 1 -- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1b8150180 msgr2=0x7fd1b80aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.541+0000 7fd1bd39b700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1b8150180 0x7fd1b80aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.541+0000 7fd1bd39b700 1 -- 172.21.15.55:0/1156640850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a800a5d0 con 0x7fd1b80c7410 2023-10-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.541+0000 7fd1bdb9c700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b8150550 0x7fd1b80b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.541+0000 7fd1bd39b700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b80c7410 0x7fd1b814a680 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400c5b0 tx=0x7fd1b4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.542+0000 7fd1ad7fa700 1 -- 172.21.15.55:0/1156640850 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4005b30 con 0x7fd1b80c7410 2023-10-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.542+0000 7fd1ad7fa700 1 -- 172.21.15.55:0/1156640850 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b4005c90 con 0x7fd1b80c7410 2023-10-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.542+0000 7fd1ad7fa700 1 -- 172.21.15.55:0/1156640850 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4007300 con 0x7fd1b80c7410 2023-10-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.542+0000 7fd1be39d700 1 -- 172.21.15.55:0/1156640850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b80afbc0 con 0x7fd1b80c7410 2023-10-06T03:46:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.542+0000 7fd1be39d700 1 -- 172.21.15.55:0/1156640850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b814abc0 con 0x7fd1b80c7410 2023-10-06T03:46:14.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.543+0000 7fd1be39d700 1 -- 172.21.15.55:0/1156640850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b80abb00 con 0x7fd1b80c7410 2023-10-06T03:46:14.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.544+0000 7fd1ad7fa700 1 -- 172.21.15.55:0/1156640850 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd1b40076b0 con 0x7fd1b80c7410 2023-10-06T03:46:14.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.545+0000 7fd1ad7fa700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd19c066f30 0x7fd19c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:14.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.545+0000 7fd1ad7fa700 1 -- 172.21.15.55:0/1156640850 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7fd1b4083d50 con 0x7fd1b80c7410 2023-10-06T03:46:14.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.545+0000 7fd1bcb9a700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd19c066f30 0x7fd19c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:14.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.546+0000 7fd1bcb9a700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd19c066f30 0x7fd19c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800a5a0 tx=0x7fd1a801a000 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:14.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.549+0000 7fd1ad7fa700 1 -- 172.21.15.55:0/1156640850 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1b4050c50 con 0x7fd1b80c7410 2023-10-06T03:46:14.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.797+0000 7fd1be39d700 1 -- 172.21.15.55:0/1156640850 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd1b804c3c0 con 0x7fd1b80c7410 2023-10-06T03:46:14.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.797+0000 7fd1ad7fa700 1 -- 172.21.15.55:0/1156640850 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd1b4017070 con 0x7fd1b80c7410 2023-10-06T03:46:14.799 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:46:14.801 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.799+0000 7fd1be39d700 1 -- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd19c066f30 msgr2=0x7fd19c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.801 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.799+0000 7fd1be39d700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd19c066f30 0x7fd19c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd1a800a5a0 tx=0x7fd1a801a000 comp rx=0 tx=0).stop 2023-10-06T03:46:14.801 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.799+0000 7fd1be39d700 1 -- 172.21.15.55:0/1156640850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b80c7410 msgr2=0x7fd1b814a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:14.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.799+0000 7fd1be39d700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b80c7410 0x7fd1b814a680 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400c5b0 tx=0x7fd1b4001070 comp rx=0 tx=0).stop 2023-10-06T03:46:14.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.800+0000 7fd1be39d700 1 -- 172.21.15.55:0/1156640850 shutdown_connections 2023-10-06T03:46:14.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.800+0000 7fd1be39d700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd19c066f30 0x7fd19c0693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.800+0000 7fd1be39d700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1b8150550 0x7fd1b80b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.800+0000 7fd1be39d700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1b80c7410 0x7fd1b814a680 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.800+0000 7fd1be39d700 1 --2- 172.21.15.55:0/1156640850 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1b8150180 0x7fd1b80aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:14.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.800+0000 7fd1be39d700 1 -- 172.21.15.55:0/1156640850 >> 172.21.15.55:0/1156640850 conn(0x7fd1b80cc1f0 msgr2=0x7fd1b806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:14.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.800+0000 7fd1be39d700 1 -- 172.21.15.55:0/1156640850 shutdown_connections 2023-10-06T03:46:14.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:14.800+0000 7fd1be39d700 1 -- 172.21.15.55:0/1156640850 wait complete. 2023-10-06T03:46:14.820 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:46:14.820 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:46:14.820 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2023-10-06T03:46:14.820 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:46:14.820 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:46:14.820 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:46:14.820 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:46:14.820 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:46:14.821 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:46:14.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:46:14.821 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2023-10-06T03:46:14.859 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:46:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:46:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:46:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:46:14.859 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T03:46:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:46:14.860 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:46:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:46:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2023-10-06T03:46:14.860 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:46:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:46:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2023-10-06T03:46:14.860 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:46:14.860 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:46:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2023-10-06T03:46:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T03:46:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 41467sec preferred_lft 41467sec 2023-10-06T03:46:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link 2023-10-06T03:46:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:46:14.861 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:46:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:46:14.861 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2023-10-06T03:46:14.861 INFO:teuthology.orchestra.run.smithi055.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:46:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff 2023-10-06T03:46:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:46:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:46:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::1099:8eff:fe6d:102/64 scope link 2023-10-06T03:46:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:46:14.862 INFO:teuthology.orchestra.run.smithi055.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:46:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether f2:5d:53:d2:00:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:46:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f05d:53ff:fed2:a/64 scope link 2023-10-06T03:46:14.862 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:46:14.862 INFO:teuthology.orchestra.run.smithi055.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:46:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:46:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::e037:33ff:feb7:6c8f/64 scope link 2023-10-06T03:46:14.863 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:46:14.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:46:14.863 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:46:14.959 INFO:teuthology.orchestra.run.smithi055.stdout:1 2023-10-06T03:46:14.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:46:14.962 DEBUG:teuthology.orchestra.run.smithi055:> route 2023-10-06T03:46:15.002 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2023-10-06T03:46:15.002 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:46:15.002 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T03:46:15.003 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T03:46:15.003 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:46:15.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:46:15.003 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:46:15.003 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:46:15.003 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T03:46:15.004 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T03:46:15.004 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T03:46:15.004 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:46:15.081 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:46:15.091 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:46:15.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:46:15.203 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T03:46:15.279 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:46:15.280 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:46:15.281 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:46:15.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:46:15.281 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2023-10-06T03:46:15.383 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:46:15.383 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:46:15.419 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:46:15.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:46:15.419 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:46:15.495 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T03:46:15.495 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:46:15.495 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:46:15.628 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:46:15.628 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:46:15.628 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:46:15.644 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:46:15.644 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:46:15.644 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:46:15.644 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:46:15.645 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:46:15.666 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:46:15.667 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:46:15.667 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:46:15.667 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:46:15.667 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:46:15.667 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T03:46:15.667 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T03:46:15.667 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T03:46:15.671 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:46:15.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:46:15.671 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:46:15.734 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:46:15.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.924+0000 7f189e39d700 1 Processor -- start 2023-10-06T03:46:15.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.924+0000 7f189e39d700 1 -- start start 2023-10-06T03:46:15.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.925+0000 7f189e39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1898150180 0x7f1898157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:15.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.925+0000 7f189e39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189806e2a0 con 0x7f1898150550 2023-10-06T03:46:15.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.925+0000 7f189e39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189806b2e0 con 0x7f18980c7410 2023-10-06T03:46:15.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.925+0000 7f189e39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18980c78e0 con 0x7f1898150180 2023-10-06T03:46:15.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.925+0000 7f189cb9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1898150550 0x7f18980c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55704/0 (socket says 172.21.15.55:55704) 2023-10-06T03:46:15.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f189cb9a700 1 -- 172.21.15.55:0/3236982412 learned_addr learned my addr 172.21.15.55:0/3236982412 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:15.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f189d39b700 1 --2- 172.21.15.55:0/3236982412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1898150180 0x7f1898157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:15.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f188ffff700 1 -- 172.21.15.55:0/3236982412 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2495743228 0 0) 0x7f189806e2a0 con 0x7f1898150550 2023-10-06T03:46:15.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f188ffff700 1 -- 172.21.15.55:0/3236982412 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1884003570 con 0x7f1898150550 2023-10-06T03:46:15.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f188ffff700 1 -- 172.21.15.55:0/3236982412 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 846509500 0 0) 0x7f1884003570 con 0x7f1898150550 2023-10-06T03:46:15.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f188ffff700 1 -- 172.21.15.55:0/3236982412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1898150180 msgr2=0x7f1898157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:15.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f188ffff700 1 --2- 172.21.15.55:0/3236982412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1898150180 0x7f1898157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:15.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f188ffff700 1 -- 172.21.15.55:0/3236982412 >> v1:172.21.15.163:6789/0 conn(0x7f18980c7410 legacy=0x7f1898069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:15.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f188ffff700 1 -- 172.21.15.55:0/3236982412 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18980c7bc0 con 0x7f1898150550 2023-10-06T03:46:15.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f188ffff700 1 -- 172.21.15.55:0/3236982412 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f1888001470 con 0x7f1898150550 2023-10-06T03:46:15.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f188ffff700 1 -- 172.21.15.55:0/3236982412 >> v1:172.21.15.55:6789/0 conn(0x7f1898150550 legacy=0x7f18980c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:15.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f188ffff700 1 --2- 172.21.15.55:0/3236982412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1898150180 0x7f1884003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:15.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f188ffff700 1 --2- 172.21.15.55:0/3236982412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1884003fd0 0x7f1884004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:15.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f188ffff700 1 --2- 172.21.15.55:0/3236982412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1884004d50 0x7f18840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:15.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.926+0000 7f188ffff700 1 -- 172.21.15.55:0/3236982412 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18980c78e0 con 0x7f1884004d50 2023-10-06T03:46:15.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.927+0000 7f188ffff700 1 -- 172.21.15.55:0/3236982412 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188400bb40 con 0x7f1884003fd0 2023-10-06T03:46:15.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.927+0000 7f188ffff700 1 -- 172.21.15.55:0/3236982412 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188400bd00 con 0x7f1898150180 2023-10-06T03:46:15.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.927+0000 7f189db9c700 1 --2- 172.21.15.55:0/3236982412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1884004d50 0x7f18840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:15.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.927+0000 7f189d39b700 1 --2- 172.21.15.55:0/3236982412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1898150180 0x7f1884003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:15.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.927+0000 7f189cb9a700 1 --2- 172.21.15.55:0/3236982412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1884003fd0 0x7f1884004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:15.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.927+0000 7f189db9c700 1 -- 172.21.15.55:0/3236982412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1898150180 msgr2=0x7f1884003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:15.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.927+0000 7f189db9c700 1 --2- 172.21.15.55:0/3236982412 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1898150180 0x7f1884003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:15.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.927+0000 7f189db9c700 1 -- 172.21.15.55:0/3236982412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1884003fd0 msgr2=0x7f1884004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:15.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.927+0000 7f189db9c700 1 --2- 172.21.15.55:0/3236982412 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1884003fd0 0x7f1884004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:15.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.927+0000 7f189db9c700 1 -- 172.21.15.55:0/3236982412 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1884003570 con 0x7f1884004d50 2023-10-06T03:46:15.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.927+0000 7f189db9c700 1 --2- 172.21.15.55:0/3236982412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1884004d50 0x7f18840051c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f18980aed30 tx=0x7f1890001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc66a59e9c6f5d78 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:15.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.928+0000 7f188ffff700 1 -- 172.21.15.55:0/3236982412 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890001a10 con 0x7f1884004d50 2023-10-06T03:46:15.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.928+0000 7f188ffff700 1 -- 172.21.15.55:0/3236982412 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189000c880 con 0x7f1884004d50 2023-10-06T03:46:15.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.928+0000 7f188ffff700 1 -- 172.21.15.55:0/3236982412 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189000cc20 con 0x7f1884004d50 2023-10-06T03:46:15.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.928+0000 7f189e39d700 1 -- 172.21.15.55:0/3236982412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1884004d50 msgr2=0x7f18840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:15.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.928+0000 7f189e39d700 1 --2- 172.21.15.55:0/3236982412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1884004d50 0x7f18840051c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f18980aed30 tx=0x7f1890001700 comp rx=0 tx=0).stop 2023-10-06T03:46:15.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.928+0000 7f189db9c700 1 -- 172.21.15.55:0/3236982412 reap_dead start 2023-10-06T03:46:15.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.928+0000 7f189e39d700 1 -- 172.21.15.55:0/3236982412 shutdown_connections 2023-10-06T03:46:15.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.928+0000 7f189e39d700 1 -- 172.21.15.55:0/3236982412 >> 172.21.15.55:0/3236982412 conn(0x7f18980cc1f0 msgr2=0x7f18980cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:15.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.928+0000 7f189e39d700 1 -- 172.21.15.55:0/3236982412 shutdown_connections 2023-10-06T03:46:15.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.928+0000 7f189e39d700 1 -- 172.21.15.55:0/3236982412 wait complete. 2023-10-06T03:46:15.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.930+0000 7f189e39d700 1 Processor -- start 2023-10-06T03:46:15.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.930+0000 7f189e39d700 1 -- start start 2023-10-06T03:46:15.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189e39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1884004d50 0x7f18981416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:15.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189e39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18980c2b40 0x7f18980c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:15.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189e39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f189806a160 0x7f189806a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:15.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189e39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189806ab40 con 0x7f189806a160 2023-10-06T03:46:15.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189e39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1898155d20 con 0x7f18980c2b40 2023-10-06T03:46:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189e39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1898155110 con 0x7f1884004d50 2023-10-06T03:46:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189db9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f189806a160 0x7f189806a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189db9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f189806a160 0x7f189806a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34216/0 (socket says 172.21.15.55:34216) 2023-10-06T03:46:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189db9c700 1 -- 172.21.15.55:0/1606512827 learned_addr learned my addr 172.21.15.55:0/1606512827 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189db9c700 1 -- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1884004d50 msgr2=0x7f18981416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189cb9a700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18980c2b40 0x7f18980c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189d39b700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1884004d50 0x7f18981416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189db9c700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1884004d50 0x7f18981416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189db9c700 1 -- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18980c2b40 msgr2=0x7f18980c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189db9c700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18980c2b40 0x7f18980c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189db9c700 1 -- 172.21.15.55:0/1606512827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18981553f0 con 0x7f189806a160 2023-10-06T03:46:15.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189cb9a700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18980c2b40 0x7f18980c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:46:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.931+0000 7f189d39b700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1884004d50 0x7f18981416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:46:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.932+0000 7f189db9c700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f189806a160 0x7f189806a600 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1890004000 tx=0x7f1890015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.932+0000 7f188d7fa700 1 -- 172.21.15.55:0/1606512827 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189000c6a0 con 0x7f189806a160 2023-10-06T03:46:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.932+0000 7f188d7fa700 1 -- 172.21.15.55:0/1606512827 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189000cc00 con 0x7f189806a160 2023-10-06T03:46:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.932+0000 7f189e39d700 1 -- 172.21.15.55:0/1606512827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18981556d0 con 0x7f189806a160 2023-10-06T03:46:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.932+0000 7f188d7fa700 1 -- 172.21.15.55:0/1606512827 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890006df0 con 0x7f189806a160 2023-10-06T03:46:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.932+0000 7f189e39d700 1 -- 172.21.15.55:0/1606512827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18980c5db0 con 0x7f189806a160 2023-10-06T03:46:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.933+0000 7f189e39d700 1 -- 172.21.15.55:0/1606512827 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1898155eb0 con 0x7f189806a160 2023-10-06T03:46:15.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.934+0000 7f188d7fa700 1 -- 172.21.15.55:0/1606512827 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f189001e640 con 0x7f189806a160 2023-10-06T03:46:15.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.935+0000 7f188d7fa700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f187c066f60 0x7f187c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:15.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.935+0000 7f188d7fa700 1 -- 172.21.15.55:0/1606512827 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f1890086460 con 0x7f189806a160 2023-10-06T03:46:15.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.935+0000 7f189d39b700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f187c066f60 0x7f187c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:15.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.936+0000 7f189d39b700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f187c066f60 0x7f187c069410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f18940093f0 tx=0x7f1894008040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:15.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:15.939+0000 7f188d7fa700 1 -- 172.21.15.55:0/1606512827 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1890053360 con 0x7f189806a160 2023-10-06T03:46:16.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.171+0000 7f189e39d700 1 -- 172.21.15.55:0/1606512827 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f189804c3c0 con 0x7f187c066f60 2023-10-06T03:46:16.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.173+0000 7f188d7fa700 1 -- 172.21.15.55:0/1606512827 <== mgr.5452 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f189804c3c0 con 0x7f187c066f60 2023-10-06T03:46:16.174 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:46:16.174 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:46:16.174 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephfs" 2023-10-06T03:46:16.174 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:46:16.175 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:46:16.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.175+0000 7f189e39d700 1 -- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f187c066f60 msgr2=0x7f187c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:16.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.175+0000 7f189e39d700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f187c066f60 0x7f187c069410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f18940093f0 tx=0x7f1894008040 comp rx=0 tx=0).stop 2023-10-06T03:46:16.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.175+0000 7f189e39d700 1 -- 172.21.15.55:0/1606512827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f189806a160 msgr2=0x7f189806a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:16.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.175+0000 7f189e39d700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f189806a160 0x7f189806a600 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1890004000 tx=0x7f1890015000 comp rx=0 tx=0).stop 2023-10-06T03:46:16.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.175+0000 7f189e39d700 1 -- 172.21.15.55:0/1606512827 shutdown_connections 2023-10-06T03:46:16.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.175+0000 7f189e39d700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f187c066f60 0x7f187c069410 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.175+0000 7f189e39d700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f189806a160 0x7f189806a600 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.175+0000 7f189e39d700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1884004d50 0x7f18981416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.175+0000 7f189e39d700 1 --2- 172.21.15.55:0/1606512827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18980c2b40 0x7f18980c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.175+0000 7f189e39d700 1 -- 172.21.15.55:0/1606512827 >> 172.21.15.55:0/1606512827 conn(0x7f18980cc1f0 msgr2=0x7f18980c8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:16.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.175+0000 7f189e39d700 1 -- 172.21.15.55:0/1606512827 shutdown_connections 2023-10-06T03:46:16.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.175+0000 7f189e39d700 1 -- 172.21.15.55:0/1606512827 wait complete. 2023-10-06T03:46:16.194 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T03:46:16.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.381+0000 7fc0a7fff700 1 Processor -- start 2023-10-06T03:46:16.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.382+0000 7fc0a7fff700 1 -- start start 2023-10-06T03:46:16.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.383+0000 7fc0a7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a80ae7e0 0x7fc0a80c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:16.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.383+0000 7fc0a7fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a806ebd0 con 0x7fc0a806f060 2023-10-06T03:46:16.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.383+0000 7fc0a7fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a815aa70 con 0x7fc0a80cb1a0 2023-10-06T03:46:16.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.383+0000 7fc0a7fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a815b2e0 con 0x7fc0a80ae7e0 2023-10-06T03:46:16.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.383+0000 7fc0a67fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc0a806f060 0x7fc0a80ae0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55708/0 (socket says 172.21.15.55:55708) 2023-10-06T03:46:16.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.383+0000 7fc0a67fc700 1 -- 172.21.15.55:0/2792886601 learned_addr learned my addr 172.21.15.55:0/2792886601 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:16.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.383+0000 7fc0a77fe700 1 --2- 172.21.15.55:0/2792886601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a80ae7e0 0x7fc0a80c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:16.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.384+0000 7fc0a5ffb700 1 -- 172.21.15.55:0/2792886601 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223492982 0 0) 0x7fc0a806ebd0 con 0x7fc0a806f060 2023-10-06T03:46:16.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.384+0000 7fc0a5ffb700 1 -- 172.21.15.55:0/2792886601 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc090003570 con 0x7fc0a806f060 2023-10-06T03:46:16.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.384+0000 7fc0a5ffb700 1 -- 172.21.15.55:0/2792886601 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330368894 0 0) 0x7fc0a815aa70 con 0x7fc0a80cb1a0 2023-10-06T03:46:16.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.384+0000 7fc0a5ffb700 1 -- 172.21.15.55:0/2792886601 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0a806ebd0 con 0x7fc0a80cb1a0 2023-10-06T03:46:16.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.384+0000 7fc0a77fe700 1 -- 172.21.15.55:0/2792886601 >> v1:172.21.15.163:6789/0 conn(0x7fc0a80cb1a0 legacy=0x7fc0a815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:16.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.384+0000 7fc0a77fe700 1 -- 172.21.15.55:0/2792886601 >> v1:172.21.15.55:6789/0 conn(0x7fc0a806f060 legacy=0x7fc0a80ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:16.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.384+0000 7fc0a77fe700 1 -- 172.21.15.55:0/2792886601 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a815ad20 con 0x7fc0a80ae7e0 2023-10-06T03:46:16.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.384+0000 7fc0a77fe700 1 --2- 172.21.15.55:0/2792886601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a80ae7e0 0x7fc0a80c9db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009e90 tx=0x7fc0a0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd4a181c41fb8ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:16.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.384+0000 7fc0a5ffb700 1 -- 172.21.15.55:0/2792886601 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a000e070 con 0x7fc0a80ae7e0 2023-10-06T03:46:16.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.385+0000 7fc0a5ffb700 1 -- 172.21.15.55:0/2792886601 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a000bc10 con 0x7fc0a80ae7e0 2023-10-06T03:46:16.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.385+0000 7fc0a5ffb700 1 -- 172.21.15.55:0/2792886601 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0012430 con 0x7fc0a80ae7e0 2023-10-06T03:46:16.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.385+0000 7fc0a7fff700 1 -- 172.21.15.55:0/2792886601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a80ae7e0 msgr2=0x7fc0a80c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:16.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.385+0000 7fc0a7fff700 1 --2- 172.21.15.55:0/2792886601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a80ae7e0 0x7fc0a80c9db0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009e90 tx=0x7fc0a0009780 comp rx=0 tx=0).stop 2023-10-06T03:46:16.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.385+0000 7fc0a7fff700 1 -- 172.21.15.55:0/2792886601 shutdown_connections 2023-10-06T03:46:16.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.385+0000 7fc0a7fff700 1 --2- 172.21.15.55:0/2792886601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a80ae7e0 0x7fc0a80c9db0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.385+0000 7fc0a7fff700 1 -- 172.21.15.55:0/2792886601 >> 172.21.15.55:0/2792886601 conn(0x7fc0a80cc1e0 msgr2=0x7fc0a80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:16.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.385+0000 7fc0a7fff700 1 -- 172.21.15.55:0/2792886601 shutdown_connections 2023-10-06T03:46:16.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.385+0000 7fc0a7fff700 1 -- 172.21.15.55:0/2792886601 wait complete. 2023-10-06T03:46:16.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.387+0000 7fc0a7fff700 1 Processor -- start 2023-10-06T03:46:16.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.387+0000 7fc0a7fff700 1 -- start start 2023-10-06T03:46:16.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0a806f060 0x7fc0a814e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:16.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a80ae7e0 0x7fc0a814eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:16.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0a80cb1a0 0x7fc0a8157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:16.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a7fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a814e710 con 0x7fc0a806f060 2023-10-06T03:46:16.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a7fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a814dd40 con 0x7fc0a80cb1a0 2023-10-06T03:46:16.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a7fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a814d130 con 0x7fc0a80ae7e0 2023-10-06T03:46:16.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0a806f060 0x7fc0a814e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:16.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0a806f060 0x7fc0a814e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34232/0 (socket says 172.21.15.55:34232) 2023-10-06T03:46:16.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a6ffd700 1 -- 172.21.15.55:0/4108374713 learned_addr learned my addr 172.21.15.55:0/4108374713 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:16.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a77fe700 1 --2- 172.21.15.55:0/4108374713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0a80cb1a0 0x7fc0a8157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:16.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a67fc700 1 --2- 172.21.15.55:0/4108374713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a80ae7e0 0x7fc0a814eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:16.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a6ffd700 1 -- 172.21.15.55:0/4108374713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a80ae7e0 msgr2=0x7fc0a814eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:16.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a6ffd700 1 --2- 172.21.15.55:0/4108374713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a80ae7e0 0x7fc0a814eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a6ffd700 1 -- 172.21.15.55:0/4108374713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0a80cb1a0 msgr2=0x7fc0a8157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:16.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a6ffd700 1 --2- 172.21.15.55:0/4108374713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0a80cb1a0 0x7fc0a8157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.388+0000 7fc0a6ffd700 1 -- 172.21.15.55:0/4108374713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a0009a20 con 0x7fc0a806f060 2023-10-06T03:46:16.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.389+0000 7fc0a6ffd700 1 --2- 172.21.15.55:0/4108374713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0a806f060 0x7fc0a814e1d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc09800c5c0 tx=0x7fc098007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:16.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.389+0000 7fc08effd700 1 -- 172.21.15.55:0/4108374713 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc098010510 con 0x7fc0a806f060 2023-10-06T03:46:16.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.389+0000 7fc08effd700 1 -- 172.21.15.55:0/4108374713 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc098010690 con 0x7fc0a806f060 2023-10-06T03:46:16.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.389+0000 7fc08effd700 1 -- 172.21.15.55:0/4108374713 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc098010a50 con 0x7fc0a806f060 2023-10-06T03:46:16.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.389+0000 7fc0a7fff700 1 -- 172.21.15.55:0/4108374713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0a814d410 con 0x7fc0a806f060 2023-10-06T03:46:16.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.389+0000 7fc0a7fff700 1 -- 172.21.15.55:0/4108374713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a814d9d0 con 0x7fc0a806f060 2023-10-06T03:46:16.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.390+0000 7fc0a7fff700 1 -- 172.21.15.55:0/4108374713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a814e020 con 0x7fc0a806f060 2023-10-06T03:46:16.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.391+0000 7fc08effd700 1 -- 172.21.15.55:0/4108374713 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 16) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc09801b400 con 0x7fc0a806f060 2023-10-06T03:46:16.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.392+0000 7fc08effd700 1 --2- 172.21.15.55:0/4108374713 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc094066f30 0x7fc0940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:16.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.392+0000 7fc08effd700 1 -- 172.21.15.55:0/4108374713 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7fc0980845b0 con 0x7fc0a806f060 2023-10-06T03:46:16.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.392+0000 7fc0a67fc700 1 --2- 172.21.15.55:0/4108374713 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc094066f30 0x7fc0940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:16.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.393+0000 7fc0a67fc700 1 --2- 172.21.15.55:0/4108374713 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc094066f30 0x7fc0940693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8153e40 tx=0x7fc09c009040 comp rx=0 tx=0).ready entity=mgr.5452 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:16.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.396+0000 7fc08effd700 1 -- 172.21.15.55:0/4108374713 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0980514b0 con 0x7fc0a806f060 2023-10-06T03:46:16.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.655+0000 7fc0a7fff700 1 -- 172.21.15.55:0/4108374713 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fc0a815b000 con 0x7fc0a806f060 2023-10-06T03:46:16.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.681+0000 7fc08effd700 1 -- 172.21.15.55:0/4108374713 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 17) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fc098051880 con 0x7fc0a806f060 2023-10-06T03:46:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.683+0000 7fc08effd700 1 -- 172.21.15.55:0/4108374713 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v17) v1 ==== 92+0+0 (secure 0 0 0) 0x7fc0980514b0 con 0x7fc0a806f060 2023-10-06T03:46:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.685+0000 7fc0a7fff700 1 -- 172.21.15.55:0/4108374713 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc094066f30 msgr2=0x7fc0940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.685+0000 7fc0a7fff700 1 --2- 172.21.15.55:0/4108374713 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc094066f30 0x7fc0940693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8153e40 tx=0x7fc09c009040 comp rx=0 tx=0).stop 2023-10-06T03:46:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.685+0000 7fc0a7fff700 1 -- 172.21.15.55:0/4108374713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0a806f060 msgr2=0x7fc0a814e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.685+0000 7fc0a7fff700 1 --2- 172.21.15.55:0/4108374713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0a806f060 0x7fc0a814e1d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc09800c5c0 tx=0x7fc098007dd0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.685+0000 7fc0a7fff700 1 -- 172.21.15.55:0/4108374713 shutdown_connections 2023-10-06T03:46:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.685+0000 7fc0a7fff700 1 --2- 172.21.15.55:0/4108374713 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc094066f30 0x7fc0940693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.685+0000 7fc0a7fff700 1 --2- 172.21.15.55:0/4108374713 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc0a80cb1a0 0x7fc0a8157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.685+0000 7fc0a7fff700 1 --2- 172.21.15.55:0/4108374713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc0a806f060 0x7fc0a814e1d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.685+0000 7fc0a7fff700 1 --2- 172.21.15.55:0/4108374713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc0a80ae7e0 0x7fc0a814eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.685+0000 7fc0a7fff700 1 -- 172.21.15.55:0/4108374713 >> 172.21.15.55:0/4108374713 conn(0x7fc0a80cc1e0 msgr2=0x7fc0a80b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.686+0000 7fc0a7fff700 1 -- 172.21.15.55:0/4108374713 shutdown_connections 2023-10-06T03:46:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.686+0000 7fc0a7fff700 1 -- 172.21.15.55:0/4108374713 wait complete. 2023-10-06T03:46:16.705 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T03:46:16.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.899+0000 7f7d0159d700 1 Processor -- start 2023-10-06T03:46:16.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.899+0000 7f7d0159d700 1 -- start start 2023-10-06T03:46:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.900+0000 7f7d0159d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cfc0b16d0 0x7f7cfc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.900+0000 7f7d0159d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7cfc0b33b0 0x7f7cfc0b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.900+0000 7f7d0159d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc0b2080 con 0x7f7cfc0b4e70 2023-10-06T03:46:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.900+0000 7f7d0159d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cfc0cfb00 con 0x7f7cfc0b33b0 2023-10-06T03:46:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.900+0000 7f7d0159d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc0b5c40 con 0x7f7cfc0b16d0 2023-10-06T03:46:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.901+0000 7f7cfbfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7cfc0b4e70 0x7f7cfc0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55712/0 (socket says 172.21.15.55:55712) 2023-10-06T03:46:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.901+0000 7f7cfbfff700 1 -- 172.21.15.55:0/2344767569 learned_addr learned my addr 172.21.15.55:0/2344767569 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:16.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.901+0000 7f7cfb7fe700 1 --2- 172.21.15.55:0/2344767569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cfc0b16d0 0x7f7cfc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.901+0000 7f7d00d9c700 1 --2- 172.21.15.55:0/2344767569 >> v2:172.21.15.55:3300/0 conn(0x7f7cfc0b33b0 0x7f7cfc0b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.901+0000 7f7cfaffd700 1 -- 172.21.15.55:0/2344767569 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588121272 0 0) 0x7f7cfc0b2080 con 0x7f7cfc0b4e70 2023-10-06T03:46:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.901+0000 7f7cfaffd700 1 -- 172.21.15.55:0/2344767569 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ce4003570 con 0x7f7cfc0b4e70 2023-10-06T03:46:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.901+0000 7f7cfb7fe700 1 -- 172.21.15.55:0/2344767569 >> v2:172.21.15.55:3300/0 conn(0x7f7cfc0b33b0 msgr2=0x7f7cfc0b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.901+0000 7f7cfb7fe700 1 --2- 172.21.15.55:0/2344767569 >> v2:172.21.15.55:3300/0 conn(0x7f7cfc0b33b0 0x7f7cfc0b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.901+0000 7f7cfb7fe700 1 -- 172.21.15.55:0/2344767569 >> v1:172.21.15.55:6789/0 conn(0x7f7cfc0b4e70 legacy=0x7f7cfc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.901+0000 7f7cfb7fe700 1 -- 172.21.15.55:0/2344767569 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc0b5f20 con 0x7f7cfc0b16d0 2023-10-06T03:46:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.901+0000 7f7d00d9c700 1 --2- 172.21.15.55:0/2344767569 >> v2:172.21.15.55:3300/0 conn(0x7f7cfc0b33b0 0x7f7cfc0b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:46:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.902+0000 7f7cfb7fe700 1 --2- 172.21.15.55:0/2344767569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cfc0b16d0 0x7f7cfc0b1b40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00098e0 tx=0x7f7cf00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=959701c783948fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.902+0000 7f7cfaffd700 1 -- 172.21.15.55:0/2344767569 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0011630 con 0x7f7cfc0b16d0 2023-10-06T03:46:16.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.902+0000 7f7cfaffd700 1 -- 172.21.15.55:0/2344767569 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf0011790 con 0x7f7cfc0b16d0 2023-10-06T03:46:16.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.902+0000 7f7cfaffd700 1 -- 172.21.15.55:0/2344767569 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0011b10 con 0x7f7cfc0b16d0 2023-10-06T03:46:16.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.903+0000 7f7d0159d700 1 -- 172.21.15.55:0/2344767569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cfc0b16d0 msgr2=0x7f7cfc0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:16.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.903+0000 7f7d0159d700 1 --2- 172.21.15.55:0/2344767569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cfc0b16d0 0x7f7cfc0b1b40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00098e0 tx=0x7f7cf00091a0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.903+0000 7f7d0159d700 1 -- 172.21.15.55:0/2344767569 shutdown_connections 2023-10-06T03:46:16.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.903+0000 7f7d0159d700 1 --2- 172.21.15.55:0/2344767569 >> v2:172.21.15.55:3300/0 conn(0x7f7cfc0b33b0 0x7f7cfc0b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.903+0000 7f7d0159d700 1 --2- 172.21.15.55:0/2344767569 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cfc0b16d0 0x7f7cfc0b1b40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.903+0000 7f7d0159d700 1 -- 172.21.15.55:0/2344767569 >> 172.21.15.55:0/2344767569 conn(0x7f7cfc0d10c0 msgr2=0x7f7cfc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:16.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.903+0000 7f7d0159d700 1 -- 172.21.15.55:0/2344767569 shutdown_connections 2023-10-06T03:46:16.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.903+0000 7f7d0159d700 1 -- 172.21.15.55:0/2344767569 wait complete. 2023-10-06T03:46:16.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.905+0000 7f7d0159d700 1 Processor -- start 2023-10-06T03:46:16.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.905+0000 7f7d0159d700 1 -- start start 2023-10-06T03:46:16.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.907+0000 7f7d0159d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cfc0b16d0 0x7f7cfc1576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.907+0000 7f7d0159d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cfc0b33b0 0x7f7cfc15d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.907+0000 7f7d0159d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0b4e70 0x7f7cfc15d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.907+0000 7f7d0159d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc15df10 con 0x7f7cfc0b4e70 2023-10-06T03:46:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.907+0000 7f7d0159d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc15b410 con 0x7f7cfc0b33b0 2023-10-06T03:46:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.907+0000 7f7d0159d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc15ab00 con 0x7f7cfc0b16d0 2023-10-06T03:46:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.907+0000 7f7d00d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0b4e70 0x7f7cfc15d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.907+0000 7f7d00d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0b4e70 0x7f7cfc15d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34248/0 (socket says 172.21.15.55:34248) 2023-10-06T03:46:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.907+0000 7f7d00d9c700 1 -- 172.21.15.55:0/2485653384 learned_addr learned my addr 172.21.15.55:0/2485653384 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.907+0000 7f7cfbfff700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cfc0b16d0 0x7f7cfc1576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.907+0000 7f7cfb7fe700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cfc0b33b0 0x7f7cfc15d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:16.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.908+0000 7f7d00d9c700 1 -- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cfc0b16d0 msgr2=0x7f7cfc1576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.908+0000 7f7d00d9c700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cfc0b16d0 0x7f7cfc1576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.908+0000 7f7d00d9c700 1 -- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cfc0b33b0 msgr2=0x7f7cfc15d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.908+0000 7f7d00d9c700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cfc0b33b0 0x7f7cfc15d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.908+0000 7f7d00d9c700 1 -- 172.21.15.55:0/2485653384 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf4009a20 con 0x7f7cfc0b4e70 2023-10-06T03:46:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.908+0000 7f7cfb7fe700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cfc0b33b0 0x7f7cfc15d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:46:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.908+0000 7f7d00d9c700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0b4e70 0x7f7cfc15d9d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009f20 tx=0x7f7cf400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.908+0000 7f7ce3fff700 1 -- 172.21.15.55:0/2485653384 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf400cbd0 con 0x7f7cfc0b4e70 2023-10-06T03:46:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.908+0000 7f7ce3fff700 1 -- 172.21.15.55:0/2485653384 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf400f040 con 0x7f7cfc0b4e70 2023-10-06T03:46:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.908+0000 7f7d0159d700 1 -- 172.21.15.55:0/2485653384 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf0009420 con 0x7f7cfc0b4e70 2023-10-06T03:46:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.908+0000 7f7ce3fff700 1 -- 172.21.15.55:0/2485653384 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4014480 con 0x7f7cfc0b4e70 2023-10-06T03:46:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.908+0000 7f7d0159d700 1 -- 172.21.15.55:0/2485653384 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc15adc0 con 0x7f7cfc0b4e70 2023-10-06T03:46:16.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.909+0000 7f7d0159d700 1 -- 172.21.15.55:0/2485653384 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc088990 con 0x7f7cfc0b4e70 2023-10-06T03:46:16.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.910+0000 7f7ce3fff700 1 -- 172.21.15.55:0/2485653384 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f7cf4003020 con 0x7f7cfc0b4e70 2023-10-06T03:46:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.912+0000 7f7ce3fff700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ce8066f80 0x7f7ce8069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.912+0000 7f7ce3fff700 1 -- 172.21.15.55:0/2485653384 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f7cf4085220 con 0x7f7cfc0b4e70 2023-10-06T03:46:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.912+0000 7f7cfbfff700 1 -- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ce8066f80 msgr2=0x7f7ce8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:46:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.912+0000 7f7cfbfff700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ce8066f80 0x7f7ce8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:46:16.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:16.916+0000 7f7ce3fff700 1 -- 172.21.15.55:0/2485653384 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cf4052120 con 0x7f7cfc0b4e70 2023-10-06T03:46:17.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.113+0000 7f7cfbfff700 1 -- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ce8066f80 msgr2=0x7f7ce8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:46:17.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.113+0000 7f7cfbfff700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ce8066f80 0x7f7ce8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:46:17.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.121+0000 7f7d0159d700 1 -- 172.21.15.55:0/2485653384 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f7cfc04bb80 con 0x7f7cfc0b4e70 2023-10-06T03:46:17.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.125+0000 7f7ce3fff700 1 -- 172.21.15.55:0/2485653384 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v17)=0 v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7cf4055740 con 0x7f7cfc0b4e70 2023-10-06T03:46:17.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.127+0000 7f7d0159d700 1 -- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ce8066f80 msgr2=0x7f7ce8069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:46:17.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.127+0000 7f7d0159d700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ce8066f80 0x7f7ce8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.127+0000 7f7d0159d700 1 -- 172.21.15.55:0/2485653384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0b4e70 msgr2=0x7f7cfc15d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:17.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.127+0000 7f7d0159d700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0b4e70 0x7f7cfc15d9d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009f20 tx=0x7f7cf400b040 comp rx=0 tx=0).stop 2023-10-06T03:46:17.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.127+0000 7f7d0159d700 1 -- 172.21.15.55:0/2485653384 shutdown_connections 2023-10-06T03:46:17.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.127+0000 7f7d0159d700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ce8066f80 0x7f7ce8069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.127+0000 7f7d0159d700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7cfc0b4e70 0x7f7cfc15d9d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.127+0000 7f7d0159d700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7cfc0b16d0 0x7f7cfc1576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.127+0000 7f7d0159d700 1 --2- 172.21.15.55:0/2485653384 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7cfc0b33b0 0x7f7cfc15d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.127+0000 7f7d0159d700 1 -- 172.21.15.55:0/2485653384 >> 172.21.15.55:0/2485653384 conn(0x7f7cfc0d10c0 msgr2=0x7f7cfc069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:17.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.127+0000 7f7d0159d700 1 -- 172.21.15.55:0/2485653384 shutdown_connections 2023-10-06T03:46:17.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.127+0000 7f7d0159d700 1 -- 172.21.15.55:0/2485653384 wait complete. 2023-10-06T03:46:17.150 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T03:46:17.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.340+0000 7f82dbfff700 1 Processor -- start 2023-10-06T03:46:17.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.340+0000 7f82dbfff700 1 -- start start 2023-10-06T03:46:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.342+0000 7f82dbfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82dc0b1b40 0x7f82dc0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.342+0000 7f82dbfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc0cfb00 con 0x7f82dc0b4e70 2023-10-06T03:46:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.342+0000 7f82dbfff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc0b5c40 con 0x7f82dc0b16d0 2023-10-06T03:46:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.342+0000 7f82dbfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0b5f20 con 0x7f82dc0b1b40 2023-10-06T03:46:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.342+0000 7f82daffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f82dc0b4e70 0x7f82dc0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55724/0 (socket says 172.21.15.55:55724) 2023-10-06T03:46:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.342+0000 7f82daffd700 1 -- 172.21.15.55:0/774517382 learned_addr learned my addr 172.21.15.55:0/774517382 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:17.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.342+0000 7f82db7fe700 1 --2- 172.21.15.55:0/774517382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82dc0b1b40 0x7f82dc0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.342+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962787464 0 0) 0x7f82dc0cfb00 con 0x7f82dc0b4e70 2023-10-06T03:46:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.342+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c4003570 con 0x7f82dc0b4e70 2023-10-06T03:46:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1342322389 0 0) 0x7f82dc0b5c40 con 0x7f82dc0b16d0 2023-10-06T03:46:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82dc0cfb00 con 0x7f82dc0b16d0 2023-10-06T03:46:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 946425940 0 0) 0x7f82c4003570 con 0x7f82dc0b4e70 2023-10-06T03:46:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82dc0b1b40 msgr2=0x7f82dc0b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 --2- 172.21.15.55:0/774517382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82dc0b1b40 0x7f82dc0b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 >> v1:172.21.15.163:6789/0 conn(0x7f82dc0b16d0 legacy=0x7f82dc0b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc0b6200 con 0x7f82dc0b4e70 2023-10-06T03:46:17.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f82cc001470 con 0x7f82dc0b4e70 2023-10-06T03:46:17.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82db7fe700 1 --2- 172.21.15.55:0/774517382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82dc0b1b40 0x7f82dc0b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:46:17.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 >> v1:172.21.15.55:6789/0 conn(0x7f82dc0b4e70 legacy=0x7f82dc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:17.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 --2- 172.21.15.55:0/774517382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c4003c20 0x7f82c4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:17.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 --2- 172.21.15.55:0/774517382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4004810 0x7f82c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:17.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 --2- 172.21.15.55:0/774517382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4005210 0x7f82c400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:17.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0b5f20 con 0x7f82c4003c20 2023-10-06T03:46:17.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c400c1b0 con 0x7f82c4005210 2023-10-06T03:46:17.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c400c370 con 0x7f82c4004810 2023-10-06T03:46:17.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82daffd700 1 --2- 172.21.15.55:0/774517382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c4003c20 0x7f82c4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:17.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82da7fc700 1 --2- 172.21.15.55:0/774517382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4004810 0x7f82c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:17.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.343+0000 7f82db7fe700 1 --2- 172.21.15.55:0/774517382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4005210 0x7f82c400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:17.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.344+0000 7f82daffd700 1 -- 172.21.15.55:0/774517382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4004810 msgr2=0x7f82c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:17.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.344+0000 7f82daffd700 1 --2- 172.21.15.55:0/774517382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4004810 0x7f82c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.344+0000 7f82daffd700 1 -- 172.21.15.55:0/774517382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4005210 msgr2=0x7f82c400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:17.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.344+0000 7f82daffd700 1 --2- 172.21.15.55:0/774517382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4005210 0x7f82c400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.344+0000 7f82daffd700 1 -- 172.21.15.55:0/774517382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c4003570 con 0x7f82c4003c20 2023-10-06T03:46:17.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.344+0000 7f82daffd700 1 --2- 172.21.15.55:0/774517382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c4003c20 0x7f82c4004010 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00c1e0 tx=0x7f82cc001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d513f064cbca9740 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:17.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.344+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc00cbe0 con 0x7f82c4003c20 2023-10-06T03:46:17.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.344+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc00cd40 con 0x7f82c4003c20 2023-10-06T03:46:17.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.344+0000 7f82d9ffb700 1 -- 172.21.15.55:0/774517382 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc005b70 con 0x7f82c4003c20 2023-10-06T03:46:17.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.345+0000 7f82dbfff700 1 -- 172.21.15.55:0/774517382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c4003c20 msgr2=0x7f82c4004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:17.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.345+0000 7f82dbfff700 1 --2- 172.21.15.55:0/774517382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c4003c20 0x7f82c4004010 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00c1e0 tx=0x7f82cc001d40 comp rx=0 tx=0).stop 2023-10-06T03:46:17.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.345+0000 7f82db7fe700 1 -- 172.21.15.55:0/774517382 reap_dead start 2023-10-06T03:46:17.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.345+0000 7f82dbfff700 1 -- 172.21.15.55:0/774517382 shutdown_connections 2023-10-06T03:46:17.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.345+0000 7f82dbfff700 1 -- 172.21.15.55:0/774517382 >> 172.21.15.55:0/774517382 conn(0x7f82dc0d10c0 msgr2=0x7f82dc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:17.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.345+0000 7f82dbfff700 1 -- 172.21.15.55:0/774517382 shutdown_connections 2023-10-06T03:46:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.345+0000 7f82dbfff700 1 -- 172.21.15.55:0/774517382 wait complete. 2023-10-06T03:46:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.346+0000 7f82dbfff700 1 Processor -- start 2023-10-06T03:46:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.347+0000 7f82dbfff700 1 -- start start 2023-10-06T03:46:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.347+0000 7f82dbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4003c20 0x7f82dc1477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.347+0000 7f82dbfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82dc0a6f00 0x7f82dc0a7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.347+0000 7f82dbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82dc0c77b0 0x7f82dc0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.347+0000 7f82dbfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc16b180 con 0x7f82dc0c77b0 2023-10-06T03:46:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.347+0000 7f82dbfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0a78b0 con 0x7f82c4003c20 2023-10-06T03:46:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.347+0000 7f82dbfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0cac70 con 0x7f82dc0a6f00 2023-10-06T03:46:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.348+0000 7f82db7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82dc0c77b0 0x7f82dc0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.348+0000 7f82daffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4003c20 0x7f82dc1477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:17.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.348+0000 7f82da7fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82dc0a6f00 0x7f82dc0a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.348+0000 7f82daffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4003c20 0x7f82dc1477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:48394/0 (socket says 172.21.15.55:48394) 2023-10-06T03:46:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.348+0000 7f82daffd700 1 -- 172.21.15.55:0/4016462903 learned_addr learned my addr 172.21.15.55:0/4016462903 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.348+0000 7f82da7fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82dc0a6f00 0x7f82dc0a7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:39594/0 (socket says 172.21.15.55:39594) 2023-10-06T03:46:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.348+0000 7f82db7fe700 1 -- 172.21.15.55:0/4016462903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82dc0a6f00 msgr2=0x7f82dc0a7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.348+0000 7f82db7fe700 1 --2- 172.21.15.55:0/4016462903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82dc0a6f00 0x7f82dc0a7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.348+0000 7f82db7fe700 1 -- 172.21.15.55:0/4016462903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4003c20 msgr2=0x7f82dc1477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.348+0000 7f82db7fe700 1 --2- 172.21.15.55:0/4016462903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4003c20 0x7f82dc1477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.348+0000 7f82db7fe700 1 -- 172.21.15.55:0/4016462903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc147cf0 con 0x7f82dc0c77b0 2023-10-06T03:46:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.348+0000 7f82db7fe700 1 --2- 172.21.15.55:0/4016462903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82dc0c77b0 0x7f82dc0c7c00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0b3130 tx=0x7f82d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.349+0000 7f82c2ffd700 1 -- 172.21.15.55:0/4016462903 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d400e040 con 0x7f82dc0c77b0 2023-10-06T03:46:17.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.349+0000 7f82c2ffd700 1 -- 172.21.15.55:0/4016462903 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d400f040 con 0x7f82dc0c77b0 2023-10-06T03:46:17.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.349+0000 7f82c2ffd700 1 -- 172.21.15.55:0/4016462903 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d400fd30 con 0x7f82dc0c77b0 2023-10-06T03:46:17.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.349+0000 7f82dbfff700 1 -- 172.21.15.55:0/4016462903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc0ca340 con 0x7f82dc0c77b0 2023-10-06T03:46:17.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.349+0000 7f82dbfff700 1 -- 172.21.15.55:0/4016462903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc0ca900 con 0x7f82dc0c77b0 2023-10-06T03:46:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.350+0000 7f82dbfff700 1 -- 172.21.15.55:0/4016462903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc0cade0 con 0x7f82dc0c77b0 2023-10-06T03:46:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.351+0000 7f82c2ffd700 1 -- 172.21.15.55:0/4016462903 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f82d4015640 con 0x7f82dc0c77b0 2023-10-06T03:46:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.352+0000 7f82c2ffd700 1 --2- 172.21.15.55:0/4016462903 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f82c8066f80 0x7f82c8069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.352+0000 7f82c2ffd700 1 -- 172.21.15.55:0/4016462903 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f82d4084880 con 0x7f82dc0c77b0 2023-10-06T03:46:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.352+0000 7f82daffd700 1 -- 172.21.15.55:0/4016462903 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f82c8066f80 msgr2=0x7f82c8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:46:17.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.352+0000 7f82daffd700 1 --2- 172.21.15.55:0/4016462903 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f82c8066f80 0x7f82c8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:46:17.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.356+0000 7f82c2ffd700 1 -- 172.21.15.55:0/4016462903 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82d4051780 con 0x7f82dc0c77b0 2023-10-06T03:46:17.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.553+0000 7f82daffd700 1 -- 172.21.15.55:0/4016462903 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f82c8066f80 msgr2=0x7f82c8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:46:17.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.553+0000 7f82daffd700 1 --2- 172.21.15.55:0/4016462903 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f82c8066f80 0x7f82c8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:46:17.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.558+0000 7f82dbfff700 1 -- 172.21.15.55:0/4016462903 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f82dc04bb80 con 0x7f82dc0c77b0 2023-10-06T03:46:17.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.564+0000 7f82c2ffd700 1 -- 172.21.15.55:0/4016462903 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v18)=0 v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7f82d4054da0 con 0x7f82dc0c77b0 2023-10-06T03:46:17.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.566+0000 7f82dbfff700 1 -- 172.21.15.55:0/4016462903 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f82c8066f80 msgr2=0x7f82c8069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:46:17.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.566+0000 7f82dbfff700 1 --2- 172.21.15.55:0/4016462903 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f82c8066f80 0x7f82c8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.566+0000 7f82dbfff700 1 -- 172.21.15.55:0/4016462903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82dc0c77b0 msgr2=0x7f82dc0c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:17.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.566+0000 7f82dbfff700 1 --2- 172.21.15.55:0/4016462903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82dc0c77b0 0x7f82dc0c7c00 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f82dc0b3130 tx=0x7f82d400b040 comp rx=0 tx=0).stop 2023-10-06T03:46:17.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.566+0000 7f82dbfff700 1 -- 172.21.15.55:0/4016462903 shutdown_connections 2023-10-06T03:46:17.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.566+0000 7f82dbfff700 1 --2- 172.21.15.55:0/4016462903 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f82c8066f80 0x7f82c8069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.566+0000 7f82dbfff700 1 --2- 172.21.15.55:0/4016462903 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82dc0c77b0 0x7f82dc0c7c00 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.566+0000 7f82dbfff700 1 --2- 172.21.15.55:0/4016462903 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c4003c20 0x7f82dc1477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.566+0000 7f82dbfff700 1 --2- 172.21.15.55:0/4016462903 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82dc0a6f00 0x7f82dc0a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.566+0000 7f82dbfff700 1 -- 172.21.15.55:0/4016462903 >> 172.21.15.55:0/4016462903 conn(0x7f82dc0d10c0 msgr2=0x7f82dc0cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:17.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.566+0000 7f82dbfff700 1 -- 172.21.15.55:0/4016462903 shutdown_connections 2023-10-06T03:46:17.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.566+0000 7f82dbfff700 1 -- 172.21.15.55:0/4016462903 wait complete. 2023-10-06T03:46:17.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:17.586 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T03:46:17.631 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-10-06T03:46:17.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.865+0000 7f3a2179d700 1 Processor -- start 2023-10-06T03:46:17.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.866+0000 7f3a2179d700 1 -- start start 2023-10-06T03:46:17.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.867+0000 7f3a2179d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3a1c0b03c0 0x7f3a1c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:17.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.867+0000 7f3a2179d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a1c1591e0 0x7f3a1c159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:17.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.867+0000 7f3a2179d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1c0cfa00 con 0x7f3a1c0b0790 2023-10-06T03:46:17.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.867+0000 7f3a2179d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1c0cb580 con 0x7f3a1c0b03c0 2023-10-06T03:46:17.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.867+0000 7f3a2179d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0cf160 con 0x7f3a1c1591e0 2023-10-06T03:46:17.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.867+0000 7f3a1b7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3a1c0b0790 0x7f3a1c0b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55730/0 (socket says 172.21.15.55:55730) 2023-10-06T03:46:17.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.867+0000 7f3a1b7fe700 1 -- 172.21.15.55:0/1969717184 learned_addr learned my addr 172.21.15.55:0/1969717184 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:17.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.867+0000 7f3a1bfff700 1 --2- 172.21.15.55:0/1969717184 >> v2:172.21.15.163:3300/0 conn(0x7f3a1c0b03c0 0x7f3a1c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:17.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.867+0000 7f3a20f9c700 1 --2- 172.21.15.55:0/1969717184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a1c1591e0 0x7f3a1c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:17.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.867+0000 7f3a1affd700 1 -- 172.21.15.55:0/1969717184 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547480141 0 0) 0x7f3a1c0cfa00 con 0x7f3a1c0b0790 2023-10-06T03:46:17.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.868+0000 7f3a1affd700 1 -- 172.21.15.55:0/1969717184 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a04003570 con 0x7f3a1c0b0790 2023-10-06T03:46:17.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.868+0000 7f3a1bfff700 1 -- 172.21.15.55:0/1969717184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a1c1591e0 msgr2=0x7f3a1c159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:17.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.868+0000 7f3a1bfff700 1 --2- 172.21.15.55:0/1969717184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a1c1591e0 0x7f3a1c159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.868+0000 7f3a1bfff700 1 -- 172.21.15.55:0/1969717184 >> v1:172.21.15.55:6789/0 conn(0x7f3a1c0b0790 legacy=0x7f3a1c0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:17.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.868+0000 7f3a1bfff700 1 -- 172.21.15.55:0/1969717184 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c0cf440 con 0x7f3a1c0b03c0 2023-10-06T03:46:17.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.868+0000 7f3a1bfff700 1 --2- 172.21.15.55:0/1969717184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c0b03c0 0x7f3a1c0cea70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009910 tx=0x7f3a0c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be6d2c08e91a261f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:17.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.868+0000 7f3a1affd700 1 -- 172.21.15.55:0/1969717184 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c011630 con 0x7f3a1c0b03c0 2023-10-06T03:46:17.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.868+0000 7f3a1affd700 1 -- 172.21.15.55:0/1969717184 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0c011790 con 0x7f3a1c0b03c0 2023-10-06T03:46:17.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.868+0000 7f3a1affd700 1 -- 172.21.15.55:0/1969717184 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c011b30 con 0x7f3a1c0b03c0 2023-10-06T03:46:17.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.869+0000 7f3a2179d700 1 -- 172.21.15.55:0/1969717184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c0b03c0 msgr2=0x7f3a1c0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:17.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.869+0000 7f3a2179d700 1 --2- 172.21.15.55:0/1969717184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c0b03c0 0x7f3a1c0cea70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c009910 tx=0x7f3a0c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.869+0000 7f3a2179d700 1 -- 172.21.15.55:0/1969717184 shutdown_connections 2023-10-06T03:46:17.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.869+0000 7f3a2179d700 1 --2- 172.21.15.55:0/1969717184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a1c1591e0 0x7f3a1c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.869+0000 7f3a2179d700 1 --2- 172.21.15.55:0/1969717184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c0b03c0 0x7f3a1c0cea70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.869+0000 7f3a2179d700 1 -- 172.21.15.55:0/1969717184 >> 172.21.15.55:0/1969717184 conn(0x7f3a1c063230 msgr2=0x7f3a1c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:17.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.869+0000 7f3a2179d700 1 -- 172.21.15.55:0/1969717184 shutdown_connections 2023-10-06T03:46:17.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.869+0000 7f3a2179d700 1 -- 172.21.15.55:0/1969717184 wait complete. 2023-10-06T03:46:17.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.870+0000 7f3a2179d700 1 Processor -- start 2023-10-06T03:46:17.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.871+0000 7f3a2179d700 1 -- start start 2023-10-06T03:46:17.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.871+0000 7f3a2179d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c0b03c0 0x7f3a1c0c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:17.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.871+0000 7f3a2179d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a1c0b0790 0x7f3a1c1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:17.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.871+0000 7f3a2179d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a1c1591e0 0x7f3a1c0c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:17.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.871+0000 7f3a2179d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0c3f70 con 0x7f3a1c0b0790 2023-10-06T03:46:17.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.871+0000 7f3a2179d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0c4810 con 0x7f3a1c0b03c0 2023-10-06T03:46:17.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.871+0000 7f3a2179d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1c0c7ae0 con 0x7f3a1c1591e0 2023-10-06T03:46:17.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.872+0000 7f3a1b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a1c0b0790 0x7f3a1c1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:17.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.872+0000 7f3a1bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c0b03c0 0x7f3a1c0c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:17.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.872+0000 7f3a1bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c0b03c0 0x7f3a1c0c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:48406/0 (socket says 172.21.15.55:48406) 2023-10-06T03:46:17.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.872+0000 7f3a1bfff700 1 -- 172.21.15.55:0/4065960343 learned_addr learned my addr 172.21.15.55:0/4065960343 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:17.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.872+0000 7f3a20f9c700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a1c1591e0 0x7f3a1c0c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:17.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.872+0000 7f3a1b7fe700 1 -- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a1c1591e0 msgr2=0x7f3a1c0c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.872+0000 7f3a1b7fe700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a1c1591e0 0x7f3a1c0c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.872+0000 7f3a1b7fe700 1 -- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c0b03c0 msgr2=0x7f3a1c0c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.872+0000 7f3a1b7fe700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c0b03c0 0x7f3a1c0c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.872+0000 7f3a1b7fe700 1 -- 172.21.15.55:0/4065960343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c009420 con 0x7f3a1c0b0790 2023-10-06T03:46:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.873+0000 7f3a1b7fe700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a1c0b0790 0x7f3a1c1576e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000c5e0 tx=0x7f3a10007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.873+0000 7f3a03fff700 1 -- 172.21.15.55:0/4065960343 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10005b30 con 0x7f3a1c0b0790 2023-10-06T03:46:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.873+0000 7f3a03fff700 1 -- 172.21.15.55:0/4065960343 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a10005c90 con 0x7f3a1c0b0790 2023-10-06T03:46:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.873+0000 7f3a2179d700 1 -- 172.21.15.55:0/4065960343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1c16a360 con 0x7f3a1c0b0790 2023-10-06T03:46:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.873+0000 7f3a03fff700 1 -- 172.21.15.55:0/4065960343 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10007320 con 0x7f3a1c0b0790 2023-10-06T03:46:17.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.873+0000 7f3a2179d700 1 -- 172.21.15.55:0/4065960343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1c0c44a0 con 0x7f3a1c0b0790 2023-10-06T03:46:17.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.874+0000 7f3a2179d700 1 -- 172.21.15.55:0/4065960343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1c0c4630 con 0x7f3a1c0b0790 2023-10-06T03:46:17.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.875+0000 7f3a03fff700 1 -- 172.21.15.55:0/4065960343 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 17) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f3a1001b3b0 con 0x7f3a1c0b0790 2023-10-06T03:46:17.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.876+0000 7f3a03fff700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 0x7f3a08069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:17.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.876+0000 7f3a03fff700 1 -- 172.21.15.55:0/4065960343 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6422+0+0 (secure 0 0 0) 0x7f3a10083d50 con 0x7f3a1c0b0790 2023-10-06T03:46:17.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.876+0000 7f3a1bfff700 1 -- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 msgr2=0x7f3a08069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:46:17.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.876+0000 7f3a1bfff700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 0x7f3a08069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:46:17.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:17.880+0000 7f3a03fff700 1 -- 172.21.15.55:0/4065960343 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a10050c50 con 0x7f3a1c0b0790 2023-10-06T03:46:17.966 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:17.965+0000 7f93e6173180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:46:18.038 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:18.037+0000 7fe12257f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:46:18.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:18.077+0000 7f3a1bfff700 1 -- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 msgr2=0x7f3a08069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:46:18.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:18.077+0000 7f3a1bfff700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 0x7f3a08069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:46:18.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:18.109+0000 7f3a2179d700 1 -- 172.21.15.55:0/4065960343 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a1c04c3c0 con 0x7f3a08066f80 2023-10-06T03:46:18.130 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:18.130+0000 7f93e6173180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:46:18.210 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:18.208+0000 7fe12257f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:46:18.216 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:18.215+0000 7f93e6173180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:46:18.298 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:18.297+0000 7fe12257f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:46:18.441 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:18.440+0000 7f93e6173180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:46:18.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:18.478+0000 7f3a1bfff700 1 -- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 msgr2=0x7f3a08069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:46:18.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:18.478+0000 7f3a1bfff700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 0x7f3a08069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T03:46:18.529 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:18.527+0000 7fe12257f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:46:18.619 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:18.618+0000 7f93e6173180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:46:18.711 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:18.710+0000 7f93e6173180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:46:18.712 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:18.711+0000 7fe12257f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:46:18.807 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:18.806+0000 7fe12257f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:46:18.873 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:18.872+0000 7f93e6173180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:46:18.972 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:18.971+0000 7fe12257f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:46:18.973 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:18.972+0000 7f93e6173180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:46:19.076 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:19.075+0000 7fe12257f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:46:19.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:19.279+0000 7f3a1bfff700 1 -- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 msgr2=0x7f3a08069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:46:19.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:19.279+0000 7f3a1bfff700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 0x7f3a08069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T03:46:19.454 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:19.453+0000 7f93e6173180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:46:19.546 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:19.545+0000 7f93e6173180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:46:19.575 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:19.573+0000 7fe12257f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:46:19.636 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:19.635+0000 7f93e6173180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:46:19.672 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:19.670+0000 7fe12257f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:46:19.765 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:19.764+0000 7fe12257f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:46:20.256 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:20.254+0000 7f93e6173180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:46:20.408 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:20.407+0000 7fe12257f180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:46:20.428 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:20.427+0000 7f93e6173180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:46:20.525 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:20.524+0000 7f93e6173180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:46:20.585 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:20.584+0000 7fe12257f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:46:20.685 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:20.683+0000 7fe12257f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:46:20.820 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:20.819+0000 7f93e6173180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:46:20.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:20.881+0000 7f3a1bfff700 1 -- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 msgr2=0x7f3a08069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:46:20.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:20.881+0000 7f3a1bfff700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 0x7f3a08069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T03:46:20.917 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:20.916+0000 7f93e6173180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:46:20.985 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:20.984+0000 7fe12257f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:46:21.084 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:21.082+0000 7fe12257f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:46:21.616 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:21.615+0000 7f93e6173180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:46:21.813 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:21.812+0000 7fe12257f180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:46:21.903 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:21.902+0000 7f93e6173180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:46:21.995 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:21.994+0000 7f93e6173180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:46:22.104 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:22.102+0000 7fe12257f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:46:22.150 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.148+0000 7f93e6173180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:46:22.199 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:22.198+0000 7fe12257f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:46:22.253 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.252+0000 7f93e6173180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:46:22.360 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:22.359+0000 7fe12257f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:46:22.441 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.439+0000 7f93e6173180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:46:22.467 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:22.466+0000 7fe12257f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:46:22.576 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.531+0000 7f93e6173180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:46:22.622 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.621+0000 7f93e6173180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:46:22.662 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:22.661+0000 7fe12257f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:46:22.714 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.713+0000 7f93e6173180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:46:22.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:22.726+0000 7f3a03fff700 1 -- 172.21.15.55:0/4065960343 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 18) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f3a10051020 con 0x7f3a1c0b0790 2023-10-06T03:46:22.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:22.726+0000 7f3a03fff700 1 -- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 msgr2=0x7f3a08069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:46:22.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:22.726+0000 7f3a03fff700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 0x7f3a08069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:22.763 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:22.761+0000 7fe12257f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:46:22.851 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.850+0000 7f935b4b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:46:22.851 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.850+0000 7f935b4b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:46:22.851 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.850+0000 7f935b4b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:46:22.851 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.850+0000 7f935b4b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:46:22.851 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.850+0000 7f935b4b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:46:22.852 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.850+0000 7f93584b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:46:22.852 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.850+0000 7f93584b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:46:22.852 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.850+0000 7f93584b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:46:22.852 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.850+0000 7f93584b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:46:22.852 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:46:22.850+0000 7f93584b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:46:22.870 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:22.869+0000 7fe12257f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:46:22.966 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:46:22.965+0000 7fe12257f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:46:23.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.729+0000 7f3a03fff700 1 -- 172.21.15.55:0/4065960343 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mgrmap(e 19) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f3a1005b420 con 0x7f3a1c0b0790 2023-10-06T03:46:23.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.729+0000 7f3a03fff700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 0x7f3a08069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:23.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.729+0000 7f3a03fff700 1 -- 172.21.15.55:0/4065960343 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a1c04c3c0 con 0x7f3a08066f80 2023-10-06T03:46:23.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.733+0000 7f3a1bfff700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 0x7f3a08069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:23.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.738+0000 7f3a1bfff700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 0x7f3a08069430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c011fd0 tx=0x7f3a0c017000 comp rx=0 tx=0).ready entity=mgr.5794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:23.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.807+0000 7f3a03fff700 1 -- 172.21.15.55:0/4065960343 <== mgr.5794 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3a1c04c3c0 con 0x7f3a08066f80 2023-10-06T03:46:23.810 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path snap_test_dir1 2023-10-06T03:46:23.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.810+0000 7f3a2179d700 1 -- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 msgr2=0x7f3a08069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:23.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.810+0000 7f3a2179d700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 0x7f3a08069430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c011fd0 tx=0x7f3a0c017000 comp rx=0 tx=0).stop 2023-10-06T03:46:23.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.810+0000 7f3a2179d700 1 -- 172.21.15.55:0/4065960343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a1c0b0790 msgr2=0x7f3a1c1576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:23.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.810+0000 7f3a2179d700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a1c0b0790 0x7f3a1c1576e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000c5e0 tx=0x7f3a10007d50 comp rx=0 tx=0).stop 2023-10-06T03:46:23.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.810+0000 7f3a2179d700 1 -- 172.21.15.55:0/4065960343 shutdown_connections 2023-10-06T03:46:23.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.810+0000 7f3a2179d700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a08066f80 0x7f3a08069430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:23.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.810+0000 7f3a2179d700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a1c1591e0 0x7f3a1c0c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:23.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.810+0000 7f3a2179d700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a1c0b03c0 0x7f3a1c0c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:23.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.810+0000 7f3a2179d700 1 --2- 172.21.15.55:0/4065960343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a1c0b0790 0x7f3a1c1576e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:23.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.810+0000 7f3a2179d700 1 -- 172.21.15.55:0/4065960343 >> 172.21.15.55:0/4065960343 conn(0x7f3a1c063230 msgr2=0x7f3a1c0ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:23.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.810+0000 7f3a2179d700 1 -- 172.21.15.55:0/4065960343 shutdown_connections 2023-10-06T03:46:23.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:23.811+0000 7f3a2179d700 1 -- 172.21.15.55:0/4065960343 wait complete. 2023-10-06T03:46:23.832 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-06T03:46:23.832 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 2M 2023-10-06T03:46:24.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.025+0000 7f657359d700 1 Processor -- start 2023-10-06T03:46:24.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.025+0000 7f657359d700 1 -- start start 2023-10-06T03:46:24.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.026+0000 7f657359d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f656c156350 0x7f656c157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:24.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.026+0000 7f657359d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f656c0b0760 0x7f656c0b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:24.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.026+0000 7f657359d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656c0b1110 con 0x7f656c159f20 2023-10-06T03:46:24.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.026+0000 7f657359d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656c0b0570 con 0x7f656c156350 2023-10-06T03:46:24.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.026+0000 7f657359d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c1578b0 con 0x7f656c0b0760 2023-10-06T03:46:24.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.027+0000 7f6571d9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f656c0b0760 0x7f656c0b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:24.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.027+0000 7f6572d9c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f656c159f20 0x7f656c15a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:36372/0 (socket says 172.21.15.55:36372) 2023-10-06T03:46:24.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.027+0000 7f657259b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f656c156350 0x7f656c157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:24.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.027+0000 7f6572d9c700 1 -- 172.21.15.55:0/1348361869 learned_addr learned my addr 172.21.15.55:0/1348361869 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.027+0000 7f6571599700 1 -- 172.21.15.55:0/1348361869 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431129373 0 0) 0x7f656c0b1110 con 0x7f656c159f20 2023-10-06T03:46:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.027+0000 7f6571599700 1 -- 172.21.15.55:0/1348361869 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6558003570 con 0x7f656c159f20 2023-10-06T03:46:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.027+0000 7f657259b700 1 -- 172.21.15.55:0/1348361869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f656c0b0760 msgr2=0x7f656c0b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.027+0000 7f657259b700 1 --2- 172.21.15.55:0/1348361869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f656c0b0760 0x7f656c0b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.027+0000 7f657259b700 1 -- 172.21.15.55:0/1348361869 >> v1:172.21.15.163:6789/0 conn(0x7f656c159f20 legacy=0x7f656c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.027+0000 7f657259b700 1 -- 172.21.15.55:0/1348361869 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c0af4d0 con 0x7f656c156350 2023-10-06T03:46:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.027+0000 7f6571d9a700 1 --2- 172.21.15.55:0/1348361869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f656c0b0760 0x7f656c0b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:46:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.028+0000 7f657259b700 1 --2- 172.21.15.55:0/1348361869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f656c156350 0x7f656c157370 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f656800bb90 tx=0x7f656800b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf352059f4ec3956 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.028+0000 7f6571599700 1 -- 172.21.15.55:0/1348361869 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656800da70 con 0x7f656c156350 2023-10-06T03:46:24.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.028+0000 7f6571599700 1 -- 172.21.15.55:0/1348361869 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656800dbd0 con 0x7f656c156350 2023-10-06T03:46:24.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.028+0000 7f6571599700 1 -- 172.21.15.55:0/1348361869 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568019480 con 0x7f656c156350 2023-10-06T03:46:24.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.028+0000 7f657359d700 1 -- 172.21.15.55:0/1348361869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f656c156350 msgr2=0x7f656c157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:24.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.028+0000 7f657359d700 1 --2- 172.21.15.55:0/1348361869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f656c156350 0x7f656c157370 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f656800bb90 tx=0x7f656800b430 comp rx=0 tx=0).stop 2023-10-06T03:46:24.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.029+0000 7f657359d700 1 -- 172.21.15.55:0/1348361869 shutdown_connections 2023-10-06T03:46:24.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.029+0000 7f657359d700 1 --2- 172.21.15.55:0/1348361869 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f656c156350 0x7f656c157370 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.029+0000 7f657359d700 1 --2- 172.21.15.55:0/1348361869 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f656c0b0760 0x7f656c0b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.029+0000 7f657359d700 1 -- 172.21.15.55:0/1348361869 >> 172.21.15.55:0/1348361869 conn(0x7f656c068e20 msgr2=0x7f656c0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:24.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.029+0000 7f657359d700 1 -- 172.21.15.55:0/1348361869 shutdown_connections 2023-10-06T03:46:24.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.029+0000 7f657359d700 1 -- 172.21.15.55:0/1348361869 wait complete. 2023-10-06T03:46:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.030+0000 7f657359d700 1 Processor -- start 2023-10-06T03:46:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.030+0000 7f657359d700 1 -- start start 2023-10-06T03:46:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.031+0000 7f657359d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f656c0b0760 0x7f656c169040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.031+0000 7f657359d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f656c156350 0x7f656c15b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.031+0000 7f657359d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f656c159f20 0x7f656c142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.031+0000 7f657359d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c169580 con 0x7f656c0b0760 2023-10-06T03:46:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.031+0000 7f657359d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c15bd00 con 0x7f656c156350 2023-10-06T03:46:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.031+0000 7f657359d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c15b330 con 0x7f656c159f20 2023-10-06T03:46:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.031+0000 7f657259b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f656c0b0760 0x7f656c169040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.032+0000 7f657259b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f656c0b0760 0x7f656c169040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35054/0 (socket says 172.21.15.55:35054) 2023-10-06T03:46:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.032+0000 7f657259b700 1 -- 172.21.15.55:0/254690716 learned_addr learned my addr 172.21.15.55:0/254690716 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.032+0000 7f6572d9c700 1 --2- 172.21.15.55:0/254690716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f656c159f20 0x7f656c142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.032+0000 7f6571d9a700 1 --2- 172.21.15.55:0/254690716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f656c156350 0x7f656c15b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.032+0000 7f657259b700 1 -- 172.21.15.55:0/254690716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f656c159f20 msgr2=0x7f656c142c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.032+0000 7f657259b700 1 --2- 172.21.15.55:0/254690716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f656c159f20 0x7f656c142c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.032+0000 7f657259b700 1 -- 172.21.15.55:0/254690716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f656c156350 msgr2=0x7f656c15b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.032+0000 7f657259b700 1 --2- 172.21.15.55:0/254690716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f656c156350 0x7f656c15b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.032+0000 7f657259b700 1 -- 172.21.15.55:0/254690716 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c009420 con 0x7f656c0b0760 2023-10-06T03:46:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.032+0000 7f657259b700 1 --2- 172.21.15.55:0/254690716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f656c0b0760 0x7f656c169040 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f656800b2c0 tx=0x7f656801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.032+0000 7f65627fc700 1 -- 172.21.15.55:0/254690716 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568010680 con 0x7f656c0b0760 2023-10-06T03:46:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.032+0000 7f65627fc700 1 -- 172.21.15.55:0/254690716 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6568010800 con 0x7f656c0b0760 2023-10-06T03:46:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.033+0000 7f65627fc700 1 -- 172.21.15.55:0/254690716 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568010bc0 con 0x7f656c0b0760 2023-10-06T03:46:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.033+0000 7f657359d700 1 -- 172.21.15.55:0/254690716 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656800b6d0 con 0x7f656c0b0760 2023-10-06T03:46:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.033+0000 7f657359d700 1 -- 172.21.15.55:0/254690716 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656c15aa00 con 0x7f656c0b0760 2023-10-06T03:46:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.034+0000 7f657359d700 1 -- 172.21.15.55:0/254690716 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656c04c3c0 con 0x7f656c0b0760 2023-10-06T03:46:24.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.035+0000 7f65627fc700 1 -- 172.21.15.55:0/254690716 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 19) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f6568023070 con 0x7f656c0b0760 2023-10-06T03:46:24.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.035+0000 7f65627fc700 1 --2- 172.21.15.55:0/254690716 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6550066dd0 0x7f6550069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:24.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.035+0000 7f65627fc700 1 -- 172.21.15.55:0/254690716 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6680+0+0 (secure 0 0 0) 0x7f6568014070 con 0x7f656c0b0760 2023-10-06T03:46:24.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.036+0000 7f6571d9a700 1 --2- 172.21.15.55:0/254690716 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6550066dd0 0x7f6550069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:24.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.037+0000 7f6571d9a700 1 --2- 172.21.15.55:0/254690716 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6550066dd0 0x7f6550069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f656c0c95c0 tx=0x7f655c0091a0 comp rx=0 tx=0).ready entity=mgr.5794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:24.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.040+0000 7f65627fc700 1 -- 172.21.15.55:0/254690716 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6568055420 con 0x7f656c0b0760 2023-10-06T03:46:24.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.267+0000 7f657359d700 1 -- 172.21.15.55:0/254690716 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "2M", "target": ["mon-mgr", ""]}) v1 -- 0x7f656c15b470 con 0x7f6550066dd0 2023-10-06T03:46:24.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.331+0000 7f65627fc700 1 -- 172.21.15.55:0/254690716 <== mgr.5794 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f655c01a030 con 0x7f6550066dd0 2023-10-06T03:46:24.333 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path snap_test_dir1 2023-10-06T03:46:24.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.333+0000 7f657359d700 1 -- 172.21.15.55:0/254690716 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6550066dd0 msgr2=0x7f6550069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:24.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.333+0000 7f657359d700 1 --2- 172.21.15.55:0/254690716 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6550066dd0 0x7f6550069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f656c0c95c0 tx=0x7f655c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.333+0000 7f657359d700 1 -- 172.21.15.55:0/254690716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f656c0b0760 msgr2=0x7f656c169040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:24.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.333+0000 7f657359d700 1 --2- 172.21.15.55:0/254690716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f656c0b0760 0x7f656c169040 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f656800b2c0 tx=0x7f656801b000 comp rx=0 tx=0).stop 2023-10-06T03:46:24.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.333+0000 7f657359d700 1 -- 172.21.15.55:0/254690716 shutdown_connections 2023-10-06T03:46:24.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.333+0000 7f657359d700 1 --2- 172.21.15.55:0/254690716 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6550066dd0 0x7f6550069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.333+0000 7f657359d700 1 --2- 172.21.15.55:0/254690716 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f656c159f20 0x7f656c142c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.333+0000 7f657359d700 1 --2- 172.21.15.55:0/254690716 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f656c0b0760 0x7f656c169040 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.333+0000 7f657359d700 1 --2- 172.21.15.55:0/254690716 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f656c156350 0x7f656c15b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.333+0000 7f657359d700 1 -- 172.21.15.55:0/254690716 >> 172.21.15.55:0/254690716 conn(0x7f656c068e20 msgr2=0x7f656c153c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:24.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.333+0000 7f657359d700 1 -- 172.21.15.55:0/254690716 shutdown_connections 2023-10-06T03:46:24.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.334+0000 7f657359d700 1 -- 172.21.15.55:0/254690716 wait complete. 2023-10-06T03:46:24.353 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-06T03:46:24.353 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-06-03_47_00 in ~36s... 2023-10-06T03:46:24.353 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-06-03_48_00 in ~96s... 2023-10-06T03:46:24.353 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M', '2M'], expected_retention: [] 2023-10-06T03:46:24.354 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-10-06T03:46:24.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.544+0000 7f5505f9d700 1 Processor -- start 2023-10-06T03:46:24.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.544+0000 7f5505f9d700 1 -- start start 2023-10-06T03:46:24.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.545+0000 7f5505f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f55000b03c0 0x7f55000cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:24.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.545+0000 7f5505f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55000cfa00 con 0x7f55000b0790 2023-10-06T03:46:24.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.545+0000 7f5505f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55000cb580 con 0x7f55001591e0 2023-10-06T03:46:24.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.545+0000 7f5505f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55000cf160 con 0x7f55000b03c0 2023-10-06T03:46:24.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.546+0000 7f5504f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f55000b03c0 0x7f55000cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:24.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.546+0000 7f54fffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f55000b0790 0x7f55000b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56700/0 (socket says 172.21.15.55:56700) 2023-10-06T03:46:24.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.546+0000 7f54fffff700 1 -- 172.21.15.55:0/3179990928 learned_addr learned my addr 172.21.15.55:0/3179990928 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:46:24.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.546+0000 7f5504f9b700 1 -- 172.21.15.55:0/3179990928 >> v1:172.21.15.163:6789/0 conn(0x7f55001591e0 legacy=0x7f5500159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:46:24.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.546+0000 7f5504f9b700 1 -- 172.21.15.55:0/3179990928 >> v1:172.21.15.55:6789/0 conn(0x7f55000b0790 legacy=0x7f55000b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:24.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.546+0000 7f5504f9b700 1 -- 172.21.15.55:0/3179990928 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55000cf440 con 0x7f55000b03c0 2023-10-06T03:46:24.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.546+0000 7f5504f9b700 1 --2- 172.21.15.55:0/3179990928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55000b03c0 0x7f55000cea70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009910 tx=0x7f54f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16a48112b9d51ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:24.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.547+0000 7f54ff7fe700 1 -- 172.21.15.55:0/3179990928 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f400b7e0 con 0x7f55000b03c0 2023-10-06T03:46:24.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.547+0000 7f54ff7fe700 1 -- 172.21.15.55:0/3179990928 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f400b940 con 0x7f55000b03c0 2023-10-06T03:46:24.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.547+0000 7f54ff7fe700 1 -- 172.21.15.55:0/3179990928 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f400bce0 con 0x7f55000b03c0 2023-10-06T03:46:24.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.547+0000 7f5505f9d700 1 -- 172.21.15.55:0/3179990928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55000b03c0 msgr2=0x7f55000cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:24.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.547+0000 7f5505f9d700 1 --2- 172.21.15.55:0/3179990928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55000b03c0 0x7f55000cea70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009910 tx=0x7f54f40091a0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.547+0000 7f5505f9d700 1 -- 172.21.15.55:0/3179990928 shutdown_connections 2023-10-06T03:46:24.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.547+0000 7f5505f9d700 1 --2- 172.21.15.55:0/3179990928 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55000b03c0 0x7f55000cea70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.547+0000 7f5505f9d700 1 -- 172.21.15.55:0/3179990928 >> 172.21.15.55:0/3179990928 conn(0x7f5500063230 msgr2=0x7f5500063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:24.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.547+0000 7f5505f9d700 1 -- 172.21.15.55:0/3179990928 shutdown_connections 2023-10-06T03:46:24.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.547+0000 7f5505f9d700 1 -- 172.21.15.55:0/3179990928 wait complete. 2023-10-06T03:46:24.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.549+0000 7f5505f9d700 1 Processor -- start 2023-10-06T03:46:24.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.549+0000 7f5505f9d700 1 -- start start 2023-10-06T03:46:24.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.550+0000 7f5505f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55000b03c0 0x7f55000c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:24.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.550+0000 7f5505f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55000b0790 0x7f55000b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:24.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.550+0000 7f5505f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55001591e0 0x7f55001488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:24.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.550+0000 7f5505f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550015faa0 con 0x7f55001591e0 2023-10-06T03:46:24.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.550+0000 7f5505f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55000b3f40 con 0x7f55000b0790 2023-10-06T03:46:24.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.550+0000 7f5505f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55000b4d80 con 0x7f55000b03c0 2023-10-06T03:46:24.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.550+0000 7f550579c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55001591e0 0x7f55001488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:24.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.550+0000 7f550579c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55001591e0 0x7f55001488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35076/0 (socket says 172.21.15.55:35076) 2023-10-06T03:46:24.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.550+0000 7f550579c700 1 -- 172.21.15.55:0/3664254404 learned_addr learned my addr 172.21.15.55:0/3664254404 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:46:24.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.551+0000 7f5504f9b700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55000b03c0 0x7f55000c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:24.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.551+0000 7f54fffff700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55000b0790 0x7f55000b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:24.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.551+0000 7f550579c700 1 -- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55000b03c0 msgr2=0x7f55000c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:24.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.551+0000 7f550579c700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55000b03c0 0x7f55000c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.551+0000 7f550579c700 1 -- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55000b0790 msgr2=0x7f55000b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:24.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.551+0000 7f550579c700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55000b0790 0x7f55000b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.551+0000 7f550579c700 1 -- 172.21.15.55:0/3664254404 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f4009420 con 0x7f55001591e0 2023-10-06T03:46:24.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.551+0000 7f54fffff700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55000b0790 0x7f55000b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:46:24.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.551+0000 7f550579c700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55001591e0 0x7f55001488b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f54f800bbe0 tx=0x7f54f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:24.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.551+0000 7f54fcff9700 1 -- 172.21.15.55:0/3664254404 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80092e0 con 0x7f55001591e0 2023-10-06T03:46:24.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.551+0000 7f54fcff9700 1 -- 172.21.15.55:0/3664254404 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f800c6f0 con 0x7f55001591e0 2023-10-06T03:46:24.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.551+0000 7f5505f9d700 1 -- 172.21.15.55:0/3664254404 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55000b4170 con 0x7f55001591e0 2023-10-06T03:46:24.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.551+0000 7f54fcff9700 1 -- 172.21.15.55:0/3664254404 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f800cab0 con 0x7f55001591e0 2023-10-06T03:46:24.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.552+0000 7f5505f9d700 1 -- 172.21.15.55:0/3664254404 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55000b4730 con 0x7f55001591e0 2023-10-06T03:46:24.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.552+0000 7f5505f9d700 1 -- 172.21.15.55:0/3664254404 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55000b4f30 con 0x7f55001591e0 2023-10-06T03:46:24.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.553+0000 7f54fcff9700 1 -- 172.21.15.55:0/3664254404 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 19) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f54f8012070 con 0x7f55001591e0 2023-10-06T03:46:24.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.554+0000 7f54fcff9700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f54e4066dd0 0x7f54e4069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:46:24.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.554+0000 7f54fcff9700 1 -- 172.21.15.55:0/3664254404 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6680+0+0 (secure 0 0 0) 0x7f54f8084290 con 0x7f55001591e0 2023-10-06T03:46:24.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.555+0000 7f5504f9b700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f54e4066dd0 0x7f54e4069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:46:24.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.555+0000 7f5504f9b700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f54e4066dd0 0x7f54e4069280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f54f40030c0 tx=0x7f54f4016000 comp rx=0 tx=0).ready entity=mgr.5794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:46:24.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.558+0000 7f54fcff9700 1 -- 172.21.15.55:0/3664254404 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55000b4f30 con 0x7f55001591e0 2023-10-06T03:46:24.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.783+0000 7f54fcff9700 1 -- 172.21.15.55:0/3664254404 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 20) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f54f80516e0 con 0x7f55001591e0 2023-10-06T03:46:24.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.789+0000 7f5505f9d700 1 -- 172.21.15.55:0/3664254404 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55000b4ef0 con 0x7f54e4066dd0 2023-10-06T03:46:24.799 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.798+0000 7f54fcff9700 1 -- 172.21.15.55:0/3664254404 <== mgr.5794 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7f55000b4ef0 con 0x7f54e4066dd0 2023-10-06T03:46:24.799 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:46:24.800 INFO:teuthology.orchestra.run.smithi055.stdout:{"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-10-06T03:46:24.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.800+0000 7f5505f9d700 1 -- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f54e4066dd0 msgr2=0x7f54e4069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:24.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.800+0000 7f5505f9d700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f54e4066dd0 0x7f54e4069280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f54f40030c0 tx=0x7f54f4016000 comp rx=0 tx=0).stop 2023-10-06T03:46:24.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.800+0000 7f5505f9d700 1 -- 172.21.15.55:0/3664254404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55001591e0 msgr2=0x7f55001488b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:46:24.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.800+0000 7f5505f9d700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55001591e0 0x7f55001488b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f54f800bbe0 tx=0x7f54f8009840 comp rx=0 tx=0).stop 2023-10-06T03:46:24.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.800+0000 7f5505f9d700 1 -- 172.21.15.55:0/3664254404 shutdown_connections 2023-10-06T03:46:24.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.800+0000 7f5505f9d700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f54e4066dd0 0x7f54e4069280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.800+0000 7f5505f9d700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55001591e0 0x7f55001488b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.800+0000 7f5505f9d700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55000b03c0 0x7f55000c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.800+0000 7f5505f9d700 1 --2- 172.21.15.55:0/3664254404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55000b0790 0x7f55000b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:46:24.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.800+0000 7f5505f9d700 1 -- 172.21.15.55:0/3664254404 >> 172.21.15.55:0/3664254404 conn(0x7f5500063230 msgr2=0x7f55001589f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:46:24.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.800+0000 7f5505f9d700 1 -- 172.21.15.55:0/3664254404 shutdown_connections 2023-10-06T03:46:24.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:46:24.801+0000 7f5505f9d700 1 -- 172.21.15.55:0/3664254404 wait complete. 2023-10-06T03:46:24.820 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-10-06T03:46:24.820 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M', '2M'], 'retention': [{}, {}]} 2023-10-06T03:46:24.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:24.821 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:24.860 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:24.860 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:24.861 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:25.862 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:25.862 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:25.907 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:25.907 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:25.907 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:26.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:26.908 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:26.952 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:26.952 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:26.952 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:27.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:27.954 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:27.997 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:27.997 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:27.997 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:28.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:28.999 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:29.042 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:29.042 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:29.042 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:30.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:30.043 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:30.086 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:30.087 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:30.087 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:31.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:31.088 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:31.132 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:31.132 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:31.132 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:32.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:32.133 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:32.177 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:32.177 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:32.177 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:33.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:33.179 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:33.222 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:33.223 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:33.223 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:34.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:34.224 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:34.268 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:34.268 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:34.268 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:35.269 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:35.269 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:35.312 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:35.313 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:35.313 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:36.314 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:36.314 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:36.358 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:36.358 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:36.358 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:37.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:37.360 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:37.403 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:37.403 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:37.403 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:38.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:38.405 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:38.449 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:38.449 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:38.449 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:39.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:39.450 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:39.494 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:39.495 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:39.495 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:40.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:40.496 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:40.540 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:40.541 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:40.541 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:41.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:41.542 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:41.585 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:41.586 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:41.586 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:42.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:42.587 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:42.631 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:42.631 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:42.631 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:43.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:43.632 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:43.676 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:43.676 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:43.676 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:44.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:44.678 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:44.721 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:44.765 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:44.765 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:45.186 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:46:45.191 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:46:45.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:45.766 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:45.810 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:45.810 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:45.810 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:46.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:46.838 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:46.885 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:46.885 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:46.886 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:47.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:47.945 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:47.989 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:47.989 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:47.989 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:48.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:49.021 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:49.065 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:49.065 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:49.065 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:50.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:50.115 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:50.159 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:50.160 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:50.160 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:51.161 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:51.178 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:51.221 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:51.221 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:51.221 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:52.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:52.223 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:52.266 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:52.266 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:52.266 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:53.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:53.268 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:53.311 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:53.312 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:53.312 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:54.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:54.313 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:54.356 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:54.364 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:54.364 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:55.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:55.372 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:55.416 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:55.416 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:55.416 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:56.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:56.417 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:56.460 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:56.460 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:56.460 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:57.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:57.462 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:57.506 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:57.506 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:57.506 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:58.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:58.507 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:58.550 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:58.551 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:58.551 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:46:59.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:46:59.553 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:46:59.596 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:46:59.597 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:46:59.597 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:00.346 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:47:00.345+0000 7f934a495700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:47:00.346 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:47:00.345+0000 7f934a495700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:47:00.346 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:47:00.345+0000 7f934a495700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:47:00.346 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:47:00.345+0000 7f934a495700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:47:00.347 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:47:00.345+0000 7f934a495700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:47:00.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:00.598 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:00.640 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:00.641 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:00.641 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:00.641 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:00.641 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-06-03_47_00_UTC'] 2023-10-06T03:47:00.642 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696563984.3535988, now = 1696564020.6420527, timo = 36 2023-10-06T03:47:01.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:01.643 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:01.686 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:01.687 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:01.687 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:01.687 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:02.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:02.688 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:02.730 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:02.731 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:02.731 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:02.731 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:03.732 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:03.732 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:03.774 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:03.775 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:03.775 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:03.775 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:04.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:04.776 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:04.819 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:04.820 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:04.820 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:04.820 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:05.821 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:05.821 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:05.863 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:05.864 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:05.865 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:05.865 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:06.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:06.866 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:06.909 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:06.910 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:06.910 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:06.910 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:07.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:07.911 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:07.953 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:07.954 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:07.954 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:07.954 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:08.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:08.955 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:08.998 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:08.999 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:08.999 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:08.999 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:10.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:10.001 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:10.043 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:10.044 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:10.044 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:10.044 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:11.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:11.046 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:11.089 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:11.090 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:11.090 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:11.090 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:12.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:12.091 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:12.134 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:12.135 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:12.135 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:12.135 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:13.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:13.136 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:13.179 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:13.180 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:13.181 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:13.181 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:14.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:14.182 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:14.225 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:14.226 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:14.226 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:14.226 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:15.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:15.227 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:15.271 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:15.272 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:15.272 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:15.272 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:15.336 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:47:15.340 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:47:16.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:16.274 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:16.316 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:16.317 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:16.317 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:16.317 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:17.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:17.319 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:17.361 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:17.362 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:17.362 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:17.362 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:18.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:18.364 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:18.406 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:18.407 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:18.407 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:18.408 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:19.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:19.409 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:19.451 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:19.453 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:19.453 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:19.453 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:20.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:20.454 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:20.496 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:20.497 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:20.497 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:20.497 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:21.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:21.499 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:21.541 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:21.542 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:21.542 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:21.542 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:22.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:22.543 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:22.586 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:22.604 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:22.604 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:22.604 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:23.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:23.605 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:23.648 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:23.649 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:23.649 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:23.649 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:24.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:24.651 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:24.693 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:24.694 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:24.694 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:24.694 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:25.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:25.696 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:25.738 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:25.739 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:25.739 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:25.740 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:26.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:26.741 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:26.783 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:26.784 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:26.784 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:26.784 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:27.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:27.785 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:27.830 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:27.831 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:27.831 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:27.831 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:28.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:28.833 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:28.875 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:28.877 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:28.877 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:28.877 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:29.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:29.878 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:29.920 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:29.921 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:29.921 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:29.921 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:30.922 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:30.923 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:30.965 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:30.966 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:30.967 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:30.967 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:31.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:31.967 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:32.011 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:32.012 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:32.012 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:32.012 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:33.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:33.014 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:33.056 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:33.057 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:33.058 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:33.058 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:34.058 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:34.058 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:34.100 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:34.102 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:34.102 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:34.102 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:35.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:35.103 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:35.146 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:35.147 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:35.147 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:35.147 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:36.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:36.149 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:36.192 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:36.193 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:36.193 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:36.193 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:37.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:37.194 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:37.236 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:37.237 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:37.237 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:37.237 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:38.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:38.238 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:38.280 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:38.282 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:38.282 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:38.282 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:39.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:39.283 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:39.326 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:39.327 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:39.327 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:39.327 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:40.327 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:40.328 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:40.370 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:40.371 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:40.371 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:40.371 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:41.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:41.372 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:41.415 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:41.416 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:41.416 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:41.416 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:42.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:42.417 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:42.459 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:42.460 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:42.460 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:42.460 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:43.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:43.462 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:43.504 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:43.506 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:43.506 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:43.506 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:44.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:44.507 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:44.549 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:44.550 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:44.550 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:44.550 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:45.469 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:47:45.474 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:47:45.552 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:45.552 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:45.592 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:45.593 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:45.593 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:45.593 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:46.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:46.594 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:46.637 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:46.638 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:46.638 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:46.638 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:47.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:47.639 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:47.681 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:47.683 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:47.683 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:47.683 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:48.684 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:48.684 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:48.726 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:48.727 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:48.727 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:48.727 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:49.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:49.728 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:49.773 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:49.774 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:49.774 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:49.774 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:50.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:50.776 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:50.821 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:50.823 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:50.823 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:50.823 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:51.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:51.824 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:51.866 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:51.867 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:51.867 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:51.867 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:52.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:52.868 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:52.911 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:52.912 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:52.912 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:52.912 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:53.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:53.913 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:53.956 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:53.957 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:53.957 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:53.957 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:54.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:54.958 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:55.000 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:55.001 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:55.001 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:55.001 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:56.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:56.002 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:56.044 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:56.045 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:56.045 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:56.046 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:57.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:57.047 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:57.089 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:57.090 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:57.090 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:57.090 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:58.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:58.092 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:58.134 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:58.135 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:58.135 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:58.135 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:47:59.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:47:59.137 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:47:59.179 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:47:59.180 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:47:59.180 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:47:59.180 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:48:00.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:48:00.181 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:48:00.224 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:48:00.225 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC'} 2023-10-06T03:48:00.225 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:48:00.225 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:48:01.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:48:01.226 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:48:01.269 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:48:01.269 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_48_00_UTC 2023-10-06T03:48:01.270 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_47_00_UTC', 'scheduled-2023-10-06-03_48_00_UTC'} 2023-10-06T03:48:01.270 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-06-03_48_00_UTC'} 2023-10-06T03:48:01.270 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:48:01.270 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-06-03_48_00_UTC'] 2023-10-06T03:48:01.270 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696563984.3535988, now = 1696564081.2708635, timo = 96 2023-10-06T03:48:02.271 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-06T03:48:02.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.472+0000 7f3cff59e700 1 Processor -- start 2023-10-06T03:48:02.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.472+0000 7f3cff59e700 1 -- start start 2023-10-06T03:48:02.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.473+0000 7f3cff59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d00156350 0x7f3d00157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:02.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.473+0000 7f3cff59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3d000b0bd0 0x7f3d0015a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:02.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.473+0000 7f3cff59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d000b0570 con 0x7f3d000b0760 2023-10-06T03:48:02.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.473+0000 7f3cff59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d001578b0 con 0x7f3d000b0bd0 2023-10-06T03:48:02.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.473+0000 7f3cff59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d000af4d0 con 0x7f3d00156350 2023-10-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.474+0000 7f3cfe59c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d00156350 0x7f3d00157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.474+0000 7f3cfdd9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f3d000b0760 0x7f3d00159f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:36226/0 (socket says 172.21.15.55:36226) 2023-10-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.474+0000 7f3cfdd9b700 1 -- 172.21.15.55:0/726768635 learned_addr learned my addr 172.21.15.55:0/726768635 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:02.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.474+0000 7f3cfed9d700 1 --2- 172.21.15.55:0/726768635 >> v2:172.21.15.163:3300/0 conn(0x7f3d000b0bd0 0x7f3d0015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.474+0000 7f3cfe59c700 1 -- 172.21.15.55:0/726768635 >> v2:172.21.15.163:3300/0 conn(0x7f3d000b0bd0 msgr2=0x7f3d0015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.474+0000 7f3cfe59c700 1 --2- 172.21.15.55:0/726768635 >> v2:172.21.15.163:3300/0 conn(0x7f3d000b0bd0 0x7f3d0015a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:02.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.474+0000 7f3cfe59c700 1 -- 172.21.15.55:0/726768635 >> v1:172.21.15.163:6789/0 conn(0x7f3d000b0760 legacy=0x7f3d00159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:02.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.474+0000 7f3cfe59c700 1 -- 172.21.15.55:0/726768635 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d000af7b0 con 0x7f3d00156350 2023-10-06T03:48:02.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.474+0000 7f3cfd59a700 1 -- 172.21.15.55:0/726768635 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253967110 0 0) 0x7f3d000b0570 con 0x7f3d000b0760 2023-10-06T03:48:02.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.475+0000 7f3cfe59c700 1 --2- 172.21.15.55:0/726768635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d00156350 0x7f3d00157370 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00098e0 tx=0x7f3cf00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7b95219b0987749 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:02.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.475+0000 7f3cfd59a700 1 -- 172.21.15.55:0/726768635 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000b640 con 0x7f3d00156350 2023-10-06T03:48:02.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.475+0000 7f3cfd59a700 1 -- 172.21.15.55:0/726768635 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf000b7a0 con 0x7f3d00156350 2023-10-06T03:48:02.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.475+0000 7f3cfd59a700 1 -- 172.21.15.55:0/726768635 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000bb40 con 0x7f3d00156350 2023-10-06T03:48:02.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.476+0000 7f3cff59e700 1 -- 172.21.15.55:0/726768635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d00156350 msgr2=0x7f3d00157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:02.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.476+0000 7f3cff59e700 1 --2- 172.21.15.55:0/726768635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d00156350 0x7f3d00157370 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00098e0 tx=0x7f3cf00091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:02.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.476+0000 7f3cff59e700 1 -- 172.21.15.55:0/726768635 shutdown_connections 2023-10-06T03:48:02.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.476+0000 7f3cff59e700 1 --2- 172.21.15.55:0/726768635 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d00156350 0x7f3d00157370 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:02.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.476+0000 7f3cff59e700 1 --2- 172.21.15.55:0/726768635 >> v2:172.21.15.163:3300/0 conn(0x7f3d000b0bd0 0x7f3d0015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:02.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.476+0000 7f3cff59e700 1 -- 172.21.15.55:0/726768635 >> 172.21.15.55:0/726768635 conn(0x7f3d00068e20 msgr2=0x7f3d000cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:02.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.476+0000 7f3cff59e700 1 -- 172.21.15.55:0/726768635 shutdown_connections 2023-10-06T03:48:02.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.476+0000 7f3cff59e700 1 -- 172.21.15.55:0/726768635 wait complete. 2023-10-06T03:48:02.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.477+0000 7f3cff59e700 1 Processor -- start 2023-10-06T03:48:02.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.477+0000 7f3cff59e700 1 -- start start 2023-10-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.478+0000 7f3cff59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d000b0760 0x7f3d000c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.478+0000 7f3cff59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d000b0bd0 0x7f3d00146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.478+0000 7f3cff59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d00156350 0x7f3d0014b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.478+0000 7f3cff59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d00152360 con 0x7f3d00156350 2023-10-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.478+0000 7f3cff59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0014c450 con 0x7f3d000b0bd0 2023-10-06T03:48:02.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.478+0000 7f3cff59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0014c170 con 0x7f3d000b0760 2023-10-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.479+0000 7f3cfed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d00156350 0x7f3d0014b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.479+0000 7f3cfed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d00156350 0x7f3d0014b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46600/0 (socket says 172.21.15.55:46600) 2023-10-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.479+0000 7f3cfed9d700 1 -- 172.21.15.55:0/583445228 learned_addr learned my addr 172.21.15.55:0/583445228 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.479+0000 7f3cfe59c700 1 --2- 172.21.15.55:0/583445228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d000b0760 0x7f3d000c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.479+0000 7f3cfdd9b700 1 --2- 172.21.15.55:0/583445228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d000b0bd0 0x7f3d00146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.479+0000 7f3cfed9d700 1 -- 172.21.15.55:0/583445228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d000b0760 msgr2=0x7f3d000c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.479+0000 7f3cfed9d700 1 --2- 172.21.15.55:0/583445228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d000b0760 0x7f3d000c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.479+0000 7f3cfed9d700 1 -- 172.21.15.55:0/583445228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d000b0bd0 msgr2=0x7f3d00146b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.479+0000 7f3cfed9d700 1 --2- 172.21.15.55:0/583445228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d000b0bd0 0x7f3d00146b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:02.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.479+0000 7f3cfed9d700 1 -- 172.21.15.55:0/583445228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf0009420 con 0x7f3d00156350 2023-10-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.479+0000 7f3cfed9d700 1 --2- 172.21.15.55:0/583445228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d00156350 0x7f3d0014b820 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009e20 tx=0x7f3cf800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.480+0000 7f3cee7fc700 1 -- 172.21.15.55:0/583445228 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800caa0 con 0x7f3d00156350 2023-10-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.480+0000 7f3cee7fc700 1 -- 172.21.15.55:0/583445228 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf800f040 con 0x7f3d00156350 2023-10-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.480+0000 7f3cee7fc700 1 -- 172.21.15.55:0/583445228 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800ce70 con 0x7f3d00156350 2023-10-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.480+0000 7f3cff59e700 1 -- 172.21.15.55:0/583445228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d00151d10 con 0x7f3d00156350 2023-10-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.480+0000 7f3cff59e700 1 -- 172.21.15.55:0/583445228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0014bd60 con 0x7f3d00156350 2023-10-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.481+0000 7f3cff59e700 1 -- 172.21.15.55:0/583445228 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d00152510 con 0x7f3d00156350 2023-10-06T03:48:02.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.482+0000 7f3cee7fc700 1 -- 172.21.15.55:0/583445228 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 22) v1 ==== 84871+0+0 (secure 0 0 0) 0x7f3cf8014690 con 0x7f3d00156350 2023-10-06T03:48:02.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.482+0000 7f3cee7fc700 1 --2- 172.21.15.55:0/583445228 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3ce00670a0 0x7f3ce0069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:02.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.483+0000 7f3cee7fc700 1 -- 172.21.15.55:0/583445228 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6738+0+0 (secure 0 0 0) 0x7f3cf80854c0 con 0x7f3d00156350 2023-10-06T03:48:02.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.483+0000 7f3cfe59c700 1 --2- 172.21.15.55:0/583445228 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3ce00670a0 0x7f3ce0069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:02.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.484+0000 7f3cfe59c700 1 --2- 172.21.15.55:0/583445228 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3ce00670a0 0x7f3ce0069550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00093f0 tx=0x7f3cf0017000 comp rx=0 tx=0).ready entity=mgr.5794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:02.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.487+0000 7f3cee7fc700 1 -- 172.21.15.55:0/583445228 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cf8055af0 con 0x7f3d00156350 2023-10-06T03:48:02.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.717+0000 7f3cff59e700 1 -- 172.21.15.55:0/583445228 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d0004c3c0 con 0x7f3ce00670a0 2023-10-06T03:48:02.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.803+0000 7f3cee7fc700 1 -- 172.21.15.55:0/583445228 <== mgr.5794 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3d0004c3c0 con 0x7f3ce00670a0 2023-10-06T03:48:02.804 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path snap_test_dir1 2023-10-06T03:48:02.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.805+0000 7f3cff59e700 1 -- 172.21.15.55:0/583445228 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3ce00670a0 msgr2=0x7f3ce0069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:02.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.805+0000 7f3cff59e700 1 --2- 172.21.15.55:0/583445228 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3ce00670a0 0x7f3ce0069550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3cf00093f0 tx=0x7f3cf0017000 comp rx=0 tx=0).stop 2023-10-06T03:48:02.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.805+0000 7f3cff59e700 1 -- 172.21.15.55:0/583445228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d00156350 msgr2=0x7f3d0014b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:02.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.805+0000 7f3cff59e700 1 --2- 172.21.15.55:0/583445228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d00156350 0x7f3d0014b820 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009e20 tx=0x7f3cf800b040 comp rx=0 tx=0).stop 2023-10-06T03:48:02.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.805+0000 7f3cff59e700 1 -- 172.21.15.55:0/583445228 shutdown_connections 2023-10-06T03:48:02.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.805+0000 7f3cff59e700 1 --2- 172.21.15.55:0/583445228 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3ce00670a0 0x7f3ce0069550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:02.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.805+0000 7f3cff59e700 1 --2- 172.21.15.55:0/583445228 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d00156350 0x7f3d0014b820 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:02.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.805+0000 7f3cff59e700 1 --2- 172.21.15.55:0/583445228 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d000b0760 0x7f3d000c9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:02.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.805+0000 7f3cff59e700 1 --2- 172.21.15.55:0/583445228 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d000b0bd0 0x7f3d00146b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:02.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.805+0000 7f3cff59e700 1 -- 172.21.15.55:0/583445228 >> 172.21.15.55:0/583445228 conn(0x7f3d00068e20 msgr2=0x7f3d001546b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:02.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.805+0000 7f3cff59e700 1 -- 172.21.15.55:0/583445228 shutdown_connections 2023-10-06T03:48:02.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:02.805+0000 7f3cff59e700 1 -- 172.21.15.55:0/583445228 wait complete. 2023-10-06T03:48:02.825 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-06T03:48:02.825 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:48:02.825 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:48:02.864 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:48:02.864 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_48_00_UTC 2023-10-06T03:48:02.865 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-03_47_00_UTC 2023-10-06T03:48:02.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:48:02.865 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-03_47_00_UTC) 2023-10-06T03:48:02.949 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-03_48_00_UTC 2023-10-06T03:48:02.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:48:02.950 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-03_48_00_UTC) 2023-10-06T03:48:03.033 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:48:03.033 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-06T03:48:03.114 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T03:48:03.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.340+0000 7fb7e639d700 1 Processor -- start 2023-10-06T03:48:03.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.341+0000 7fb7e639d700 1 -- start start 2023-10-06T03:48:03.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.342+0000 7fb7e639d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb7e00b4e70 0x7fb7e00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:03.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.342+0000 7fb7e639d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7e00b16d0 0x7fb7e00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:03.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.342+0000 7fb7e639d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e00b2080 con 0x7fb7e015d240 2023-10-06T03:48:03.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.342+0000 7fb7e639d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e00cfb00 con 0x7fb7e00b4e70 2023-10-06T03:48:03.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.342+0000 7fb7e639d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00b5c40 con 0x7fb7e00b16d0 2023-10-06T03:48:03.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.343+0000 7fb7e4b9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7e00b16d0 0x7fb7e00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:03.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.343+0000 7fb7e539b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb7e00b4e70 0x7fb7e00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:03.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.343+0000 7fb7e4b9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7e00b16d0 0x7fb7e00b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:55660/0 (socket says 172.21.15.55:55660) 2023-10-06T03:48:03.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.343+0000 7fb7e4b9a700 1 -- 172.21.15.55:0/2140315382 learned_addr learned my addr 172.21.15.55:0/2140315382 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:03.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.343+0000 7fb7d7fff700 1 -- 172.21.15.55:0/2140315382 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553745422 0 0) 0x7fb7e00b2080 con 0x7fb7e015d240 2023-10-06T03:48:03.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.343+0000 7fb7e539b700 1 -- 172.21.15.55:0/2140315382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7e00b16d0 msgr2=0x7fb7e00b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:03.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.343+0000 7fb7e539b700 1 --2- 172.21.15.55:0/2140315382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7e00b16d0 0x7fb7e00b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:03.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.343+0000 7fb7e539b700 1 -- 172.21.15.55:0/2140315382 >> v1:172.21.15.163:6789/0 conn(0x7fb7e015d240 legacy=0x7fb7e015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:03.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.343+0000 7fb7e539b700 1 -- 172.21.15.55:0/2140315382 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e00b5f20 con 0x7fb7e00b4e70 2023-10-06T03:48:03.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.343+0000 7fb7e4b9a700 1 --2- 172.21.15.55:0/2140315382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7e00b16d0 0x7fb7e00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:03.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.343+0000 7fb7e539b700 1 --2- 172.21.15.55:0/2140315382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e00b4e70 0x7fb7e00ccff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0098e0 tx=0x7fb7dc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fddf3d0a9ec81cba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:03.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.344+0000 7fb7d7fff700 1 -- 172.21.15.55:0/2140315382 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00b640 con 0x7fb7e00b4e70 2023-10-06T03:48:03.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.344+0000 7fb7d7fff700 1 -- 172.21.15.55:0/2140315382 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7dc00b7a0 con 0x7fb7e00b4e70 2023-10-06T03:48:03.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.344+0000 7fb7d7fff700 1 -- 172.21.15.55:0/2140315382 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00bb40 con 0x7fb7e00b4e70 2023-10-06T03:48:03.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.344+0000 7fb7e639d700 1 -- 172.21.15.55:0/2140315382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e00b4e70 msgr2=0x7fb7e00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:03.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.344+0000 7fb7e639d700 1 --2- 172.21.15.55:0/2140315382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e00b4e70 0x7fb7e00ccff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0098e0 tx=0x7fb7dc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:03.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.344+0000 7fb7e639d700 1 -- 172.21.15.55:0/2140315382 shutdown_connections 2023-10-06T03:48:03.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.344+0000 7fb7e639d700 1 --2- 172.21.15.55:0/2140315382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e00b4e70 0x7fb7e00ccff0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:03.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.344+0000 7fb7e639d700 1 --2- 172.21.15.55:0/2140315382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7e00b16d0 0x7fb7e00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:03.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.344+0000 7fb7e639d700 1 -- 172.21.15.55:0/2140315382 >> 172.21.15.55:0/2140315382 conn(0x7fb7e00d10c0 msgr2=0x7fb7e00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:03.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.345+0000 7fb7e639d700 1 -- 172.21.15.55:0/2140315382 shutdown_connections 2023-10-06T03:48:03.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.345+0000 7fb7e639d700 1 -- 172.21.15.55:0/2140315382 wait complete. 2023-10-06T03:48:03.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.346+0000 7fb7e639d700 1 Processor -- start 2023-10-06T03:48:03.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.346+0000 7fb7e639d700 1 -- start start 2023-10-06T03:48:03.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.347+0000 7fb7e639d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7e00b16d0 0x7fb7e0169bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:03.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.347+0000 7fb7e639d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7e00b4e70 0x7fb7e014c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:03.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.347+0000 7fb7e639d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e015d240 0x7fb7e00cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:03.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.347+0000 7fb7e639d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00c8020 con 0x7fb7e00b4e70 2023-10-06T03:48:03.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.347+0000 7fb7e639d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00c7410 con 0x7fb7e015d240 2023-10-06T03:48:03.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.347+0000 7fb7e639d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e00c76f0 con 0x7fb7e00b16d0 2023-10-06T03:48:03.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.347+0000 7fb7e4b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7e00b4e70 0x7fb7e014c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:03.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.347+0000 7fb7e4b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7e00b4e70 0x7fb7e014c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54304/0 (socket says 172.21.15.55:54304) 2023-10-06T03:48:03.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.347+0000 7fb7e4b9a700 1 -- 172.21.15.55:0/604438667 learned_addr learned my addr 172.21.15.55:0/604438667 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:03.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.348+0000 7fb7e539b700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7e00b16d0 0x7fb7e0169bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:03.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.348+0000 7fb7e5b9c700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e015d240 0x7fb7e00cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:03.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.348+0000 7fb7e4b9a700 1 -- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7e00b16d0 msgr2=0x7fb7e0169bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:03.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.348+0000 7fb7e4b9a700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7e00b16d0 0x7fb7e0169bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:03.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.348+0000 7fb7e4b9a700 1 -- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e015d240 msgr2=0x7fb7e00cba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:03.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.348+0000 7fb7e4b9a700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e015d240 0x7fb7e00cba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:03.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.348+0000 7fb7e4b9a700 1 -- 172.21.15.55:0/604438667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d00094f0 con 0x7fb7e00b4e70 2023-10-06T03:48:03.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.348+0000 7fb7e539b700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7e00b16d0 0x7fb7e0169bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:03.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.348+0000 7fb7e4b9a700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7e00b4e70 0x7fb7e014c140 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000ec50 tx=0x7fb7d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:03.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.348+0000 7fb7d57fa700 1 -- 172.21.15.55:0/604438667 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000c9d0 con 0x7fb7e00b4e70 2023-10-06T03:48:03.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.348+0000 7fb7d57fa700 1 -- 172.21.15.55:0/604438667 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d000cb30 con 0x7fb7e00b4e70 2023-10-06T03:48:03.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.348+0000 7fb7d57fa700 1 -- 172.21.15.55:0/604438667 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0013470 con 0x7fb7e00b4e70 2023-10-06T03:48:03.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.348+0000 7fb7e639d700 1 -- 172.21.15.55:0/604438667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7dc009420 con 0x7fb7e00b4e70 2023-10-06T03:48:03.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.348+0000 7fb7e639d700 1 -- 172.21.15.55:0/604438667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e00c7cb0 con 0x7fb7e00b4e70 2023-10-06T03:48:03.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.349+0000 7fb7e639d700 1 -- 172.21.15.55:0/604438667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e00c81d0 con 0x7fb7e00b4e70 2023-10-06T03:48:03.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.350+0000 7fb7d57fa700 1 -- 172.21.15.55:0/604438667 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 22) v1 ==== 84871+0+0 (secure 0 0 0) 0x7fb7d0013860 con 0x7fb7e00b4e70 2023-10-06T03:48:03.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.351+0000 7fb7d57fa700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb7c40670a0 0x7fb7c4069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:03.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.351+0000 7fb7d57fa700 1 -- 172.21.15.55:0/604438667 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6738+0+0 (secure 0 0 0) 0x7fb7d0084fa0 con 0x7fb7e00b4e70 2023-10-06T03:48:03.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.352+0000 7fb7e539b700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb7c40670a0 0x7fb7c4069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:03.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.353+0000 7fb7e539b700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb7c40670a0 0x7fb7c4069550 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0091e0 tx=0x7fb7dc017000 comp rx=0 tx=0).ready entity=mgr.5794 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:03.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.356+0000 7fb7d57fa700 1 -- 172.21.15.55:0/604438667 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7e00c81d0 con 0x7fb7e00b4e70 2023-10-06T03:48:03.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.615+0000 7fb7e639d700 1 -- 172.21.15.55:0/604438667 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb7e00c8160 con 0x7fb7e00b4e70 2023-10-06T03:48:03.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.855+0000 7fb7d57fa700 1 -- 172.21.15.55:0/604438667 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 23) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fb7d0052020 con 0x7fb7e00b4e70 2023-10-06T03:48:03.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.856+0000 7fb7d57fa700 1 -- 172.21.15.55:0/604438667 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v23) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb7d007d640 con 0x7fb7e00b4e70 2023-10-06T03:48:03.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.859+0000 7fb7e639d700 1 -- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb7c40670a0 msgr2=0x7fb7c4069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.859+0000 7fb7e639d700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb7c40670a0 0x7fb7c4069550 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0091e0 tx=0x7fb7dc017000 comp rx=0 tx=0).stop 2023-10-06T03:48:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.859+0000 7fb7e639d700 1 -- 172.21.15.55:0/604438667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7e00b4e70 msgr2=0x7fb7e014c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.859+0000 7fb7e639d700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7e00b4e70 0x7fb7e014c140 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000ec50 tx=0x7fb7d0009250 comp rx=0 tx=0).stop 2023-10-06T03:48:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.859+0000 7fb7e639d700 1 -- 172.21.15.55:0/604438667 shutdown_connections 2023-10-06T03:48:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.859+0000 7fb7e639d700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb7c40670a0 0x7fb7c4069550 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.859+0000 7fb7e639d700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7e015d240 0x7fb7e00cba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.859+0000 7fb7e639d700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb7e00b16d0 0x7fb7e0169bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.859+0000 7fb7e639d700 1 --2- 172.21.15.55:0/604438667 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7e00b4e70 0x7fb7e014c140 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.859+0000 7fb7e639d700 1 -- 172.21.15.55:0/604438667 >> 172.21.15.55:0/604438667 conn(0x7fb7e00d10c0 msgr2=0x7fb7e0158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.859+0000 7fb7e639d700 1 -- 172.21.15.55:0/604438667 shutdown_connections 2023-10-06T03:48:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:03.859+0000 7fb7e639d700 1 -- 172.21.15.55:0/604438667 wait complete. 2023-10-06T03:48:03.879 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:48:03.879 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:48:03.960 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:48:04.036 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:48:04.037 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:48:04.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:48:04.037 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:48:04.124 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T03:48:04.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:48:04.124 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:48:04.162 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:48:04.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.392+0000 7fda3359e700 1 Processor -- start 2023-10-06T03:48:04.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.392+0000 7fda3359e700 1 -- start start 2023-10-06T03:48:04.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.393+0000 7fda3359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda341597f0 0x7fda340b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:04.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.393+0000 7fda3359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda340b11d0 con 0x7fda34159420 2023-10-06T03:48:04.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.393+0000 7fda3359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda340b14b0 con 0x7fda3415d230 2023-10-06T03:48:04.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.393+0000 7fda3359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda340b1790 con 0x7fda341597f0 2023-10-06T03:48:04.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.394+0000 7fda3259c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fda34159420 0x7fda340b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43390/0 (socket says 172.21.15.55:43390) 2023-10-06T03:48:04.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.394+0000 7fda3259c700 1 -- 172.21.15.55:0/2355222411 learned_addr learned my addr 172.21.15.55:0/2355222411 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:04.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.394+0000 7fda31d9b700 1 --2- 172.21.15.55:0/2355222411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda341597f0 0x7fda340b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:04.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.394+0000 7fda3159a700 1 -- 172.21.15.55:0/2355222411 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911743739 0 0) 0x7fda340b11d0 con 0x7fda34159420 2023-10-06T03:48:04.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.394+0000 7fda3159a700 1 -- 172.21.15.55:0/2355222411 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda1c003570 con 0x7fda34159420 2023-10-06T03:48:04.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.394+0000 7fda3159a700 1 -- 172.21.15.55:0/2355222411 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107035766 0 0) 0x7fda340b14b0 con 0x7fda3415d230 2023-10-06T03:48:04.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.394+0000 7fda3159a700 1 -- 172.21.15.55:0/2355222411 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda340b11d0 con 0x7fda3415d230 2023-10-06T03:48:04.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.394+0000 7fda31d9b700 1 -- 172.21.15.55:0/2355222411 >> v1:172.21.15.163:6789/0 conn(0x7fda3415d230 legacy=0x7fda340b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:04.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.394+0000 7fda31d9b700 1 -- 172.21.15.55:0/2355222411 >> v1:172.21.15.55:6789/0 conn(0x7fda34159420 legacy=0x7fda340b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:04.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.394+0000 7fda31d9b700 1 -- 172.21.15.55:0/2355222411 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda340b1a70 con 0x7fda341597f0 2023-10-06T03:48:04.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.395+0000 7fda31d9b700 1 --2- 172.21.15.55:0/2355222411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda341597f0 0x7fda340b2270 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda28009910 tx=0x7fda280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=153cd40a441701f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:04.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.395+0000 7fda3159a700 1 -- 172.21.15.55:0/2355222411 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28011630 con 0x7fda341597f0 2023-10-06T03:48:04.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.395+0000 7fda3159a700 1 -- 172.21.15.55:0/2355222411 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda28011790 con 0x7fda341597f0 2023-10-06T03:48:04.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.395+0000 7fda3159a700 1 -- 172.21.15.55:0/2355222411 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28011b30 con 0x7fda341597f0 2023-10-06T03:48:04.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.396+0000 7fda3359e700 1 -- 172.21.15.55:0/2355222411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda341597f0 msgr2=0x7fda340b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:04.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.396+0000 7fda3359e700 1 --2- 172.21.15.55:0/2355222411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda341597f0 0x7fda340b2270 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fda28009910 tx=0x7fda280091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.396+0000 7fda3359e700 1 -- 172.21.15.55:0/2355222411 shutdown_connections 2023-10-06T03:48:04.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.396+0000 7fda3359e700 1 --2- 172.21.15.55:0/2355222411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda341597f0 0x7fda340b2270 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.396+0000 7fda3359e700 1 -- 172.21.15.55:0/2355222411 >> 172.21.15.55:0/2355222411 conn(0x7fda340d0660 msgr2=0x7fda340d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:04.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.396+0000 7fda3359e700 1 -- 172.21.15.55:0/2355222411 shutdown_connections 2023-10-06T03:48:04.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.396+0000 7fda3359e700 1 -- 172.21.15.55:0/2355222411 wait complete. 2023-10-06T03:48:04.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.397+0000 7fda3359e700 1 Processor -- start 2023-10-06T03:48:04.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.397+0000 7fda3359e700 1 -- start start 2023-10-06T03:48:04.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.398+0000 7fda3359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda34159420 0x7fda341541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:04.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.398+0000 7fda3359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda341597f0 0x7fda340c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:04.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.398+0000 7fda3359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda3415d230 0x7fda340c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:04.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.398+0000 7fda3359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda340b56e0 con 0x7fda34159420 2023-10-06T03:48:04.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.398+0000 7fda3359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3415a870 con 0x7fda3415d230 2023-10-06T03:48:04.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.398+0000 7fda3359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3415ab50 con 0x7fda341597f0 2023-10-06T03:48:04.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.399+0000 7fda3259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda34159420 0x7fda341541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:04.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.399+0000 7fda3259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda34159420 0x7fda341541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54324/0 (socket says 172.21.15.55:54324) 2023-10-06T03:48:04.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.399+0000 7fda3259c700 1 -- 172.21.15.55:0/843873947 learned_addr learned my addr 172.21.15.55:0/843873947 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:04.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.399+0000 7fda31d9b700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda341597f0 0x7fda340c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:04.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.399+0000 7fda32d9d700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda3415d230 0x7fda340c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:04.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.399+0000 7fda3259c700 1 -- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda341597f0 msgr2=0x7fda340c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:04.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.399+0000 7fda3259c700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda341597f0 0x7fda340c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.399+0000 7fda3259c700 1 -- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda3415d230 msgr2=0x7fda340c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:04.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.399+0000 7fda3259c700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda3415d230 0x7fda340c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.399+0000 7fda3259c700 1 -- 172.21.15.55:0/843873947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda28009420 con 0x7fda34159420 2023-10-06T03:48:04.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.399+0000 7fda31d9b700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda341597f0 0x7fda340c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:04.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.399+0000 7fda3259c700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda34159420 0x7fda341541c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fda2400c790 tx=0x7fda24007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:04.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.400+0000 7fda227fc700 1 -- 172.21.15.55:0/843873947 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24005b30 con 0x7fda34159420 2023-10-06T03:48:04.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.400+0000 7fda227fc700 1 -- 172.21.15.55:0/843873947 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda24005cd0 con 0x7fda34159420 2023-10-06T03:48:04.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.400+0000 7fda227fc700 1 -- 172.21.15.55:0/843873947 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24007320 con 0x7fda34159420 2023-10-06T03:48:04.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.400+0000 7fda3359e700 1 -- 172.21.15.55:0/843873947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3415ae30 con 0x7fda34159420 2023-10-06T03:48:04.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.400+0000 7fda3359e700 1 -- 172.21.15.55:0/843873947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda34154700 con 0x7fda34159420 2023-10-06T03:48:04.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.401+0000 7fda3359e700 1 -- 172.21.15.55:0/843873947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3404c3c0 con 0x7fda34159420 2023-10-06T03:48:04.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.402+0000 7fda227fc700 1 -- 172.21.15.55:0/843873947 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 23) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fda24012400 con 0x7fda34159420 2023-10-06T03:48:04.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.403+0000 7fda227fc700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fda14067050 0x7fda14069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:04.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.403+0000 7fda227fc700 1 -- 172.21.15.55:0/843873947 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6738+0+0 (secure 0 0 0) 0x7fda24083e80 con 0x7fda34159420 2023-10-06T03:48:04.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.403+0000 7fda31d9b700 1 -- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fda14067050 msgr2=0x7fda14069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:04.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.403+0000 7fda31d9b700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fda14067050 0x7fda14069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:04.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.408+0000 7fda227fc700 1 -- 172.21.15.55:0/843873947 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda2404b5a0 con 0x7fda34159420 2023-10-06T03:48:04.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.604+0000 7fda31d9b700 1 -- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fda14067050 msgr2=0x7fda14069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:04.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.604+0000 7fda31d9b700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fda14067050 0x7fda14069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:04.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.656+0000 7fda3359e700 1 -- 172.21.15.55:0/843873947 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda340b1d50 con 0x7fda34159420 2023-10-06T03:48:04.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.656+0000 7fda227fc700 1 -- 172.21.15.55:0/843873947 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fda24017090 con 0x7fda34159420 2023-10-06T03:48:04.658 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:04.658 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3493866751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3493866751},{"type":"v1","addr":"172.21.15.163:6836","nonce":3493866751}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3246061279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3246061279},{"type":"v1","addr":"172.21.15.55:6835","nonce":3246061279}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1366768080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1366768080},{"type":"v1","addr":"172.21.15.55:6834","nonce":1366768080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:46:07.292050+0000","modified":"2023-10-06T03:46:10.815261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6833/2287859778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2287859778},{"type":"v1","addr":"172.21.15.163:6833","nonce":2287859778}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T03:48:04.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.658+0000 7fda3359e700 1 -- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fda14067050 msgr2=0x7fda14069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:04.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.658+0000 7fda3359e700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fda14067050 0x7fda14069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.658+0000 7fda3359e700 1 -- 172.21.15.55:0/843873947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda34159420 msgr2=0x7fda341541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:04.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.658+0000 7fda3359e700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda34159420 0x7fda341541c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fda2400c790 tx=0x7fda24007e80 comp rx=0 tx=0).stop 2023-10-06T03:48:04.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.659+0000 7fda3359e700 1 -- 172.21.15.55:0/843873947 shutdown_connections 2023-10-06T03:48:04.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.659+0000 7fda3359e700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fda14067050 0x7fda14069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.659+0000 7fda3359e700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda3415d230 0x7fda340c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.659+0000 7fda3359e700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda34159420 0x7fda341541c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.659+0000 7fda3359e700 1 --2- 172.21.15.55:0/843873947 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda341597f0 0x7fda340c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.659+0000 7fda3359e700 1 -- 172.21.15.55:0/843873947 >> 172.21.15.55:0/843873947 conn(0x7fda340d0660 msgr2=0x7fda3415d690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:04.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.659+0000 7fda3359e700 1 -- 172.21.15.55:0/843873947 shutdown_connections 2023-10-06T03:48:04.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.659+0000 7fda3359e700 1 -- 172.21.15.55:0/843873947 wait complete. 2023-10-06T03:48:04.662 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 26 2023-10-06T03:48:04.678 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:48:04.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.871+0000 7f4cbb39d700 1 Processor -- start 2023-10-06T03:48:04.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.871+0000 7f4cbb39d700 1 -- start start 2023-10-06T03:48:04.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.872+0000 7f4cbb39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb4142ea0 0x7f4cb40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:04.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.873+0000 7f4cbb39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4cb40ad870 0x7f4cb406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:04.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.873+0000 7f4cbb39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb40b40f0 con 0x7f4cb40b3b70 2023-10-06T03:48:04.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.873+0000 7f4cbb39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb4069640 con 0x7f4cb40ad870 2023-10-06T03:48:04.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.873+0000 7f4cbb39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb4069920 con 0x7f4cb4142ea0 2023-10-06T03:48:04.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.873+0000 7f4cbab9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4cb40b3b70 0x7f4cb406cf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43400/0 (socket says 172.21.15.55:43400) 2023-10-06T03:48:04.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.873+0000 7f4cbab9c700 1 -- 172.21.15.55:0/246775798 learned_addr learned my addr 172.21.15.55:0/246775798 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:04.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.873+0000 7f4cba39b700 1 --2- 172.21.15.55:0/246775798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb4142ea0 0x7f4cb40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:04.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.873+0000 7f4cb9b9a700 1 --2- 172.21.15.55:0/246775798 >> v2:172.21.15.163:3300/0 conn(0x7f4cb40ad870 0x7f4cb406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:04.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.873+0000 7f4cb9399700 1 -- 172.21.15.55:0/246775798 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923269366 0 0) 0x7f4cb40b40f0 con 0x7f4cb40b3b70 2023-10-06T03:48:04.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.873+0000 7f4cb9399700 1 -- 172.21.15.55:0/246775798 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ca0003570 con 0x7f4cb40b3b70 2023-10-06T03:48:04.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.874+0000 7f4cb9b9a700 1 -- 172.21.15.55:0/246775798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb4142ea0 msgr2=0x7f4cb40b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:04.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.874+0000 7f4cb9b9a700 1 --2- 172.21.15.55:0/246775798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb4142ea0 0x7f4cb40b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.874+0000 7f4cb9b9a700 1 -- 172.21.15.55:0/246775798 >> v1:172.21.15.55:6789/0 conn(0x7f4cb40b3b70 legacy=0x7f4cb406cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:04.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.874+0000 7f4cb9399700 1 -- 172.21.15.55:0/246775798 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3234207752 0 0) 0x7f4ca0003570 con 0x7f4cb40b3b70 2023-10-06T03:48:04.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.874+0000 7f4cb9b9a700 1 -- 172.21.15.55:0/246775798 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb4069c00 con 0x7f4cb40ad870 2023-10-06T03:48:04.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.874+0000 7f4cba39b700 1 --2- 172.21.15.55:0/246775798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb4142ea0 0x7f4cb40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:04.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.874+0000 7f4cb9b9a700 1 --2- 172.21.15.55:0/246775798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cb40ad870 0x7f4cb406b9e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40098e0 tx=0x7f4ca40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8e7034609778545 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:04.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.874+0000 7f4cb9399700 1 -- 172.21.15.55:0/246775798 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac0017f0 con 0x7f4cb40ad870 2023-10-06T03:48:04.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.874+0000 7f4cb9399700 1 -- 172.21.15.55:0/246775798 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca4011420 con 0x7f4cb40ad870 2023-10-06T03:48:04.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.874+0000 7f4cb9399700 1 -- 172.21.15.55:0/246775798 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca40117c0 con 0x7f4cb40ad870 2023-10-06T03:48:04.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.875+0000 7f4cbb39d700 1 -- 172.21.15.55:0/246775798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cb40ad870 msgr2=0x7f4cb406b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:04.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.875+0000 7f4cbb39d700 1 --2- 172.21.15.55:0/246775798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cb40ad870 0x7f4cb406b9e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40098e0 tx=0x7f4ca40091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.875+0000 7f4cbb39d700 1 -- 172.21.15.55:0/246775798 shutdown_connections 2023-10-06T03:48:04.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.875+0000 7f4cbb39d700 1 --2- 172.21.15.55:0/246775798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb4142ea0 0x7f4cb40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.875+0000 7f4cbb39d700 1 --2- 172.21.15.55:0/246775798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cb40ad870 0x7f4cb406b9e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.875+0000 7f4cbb39d700 1 -- 172.21.15.55:0/246775798 >> 172.21.15.55:0/246775798 conn(0x7f4cb40ce430 msgr2=0x7f4cb40ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:04.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.875+0000 7f4cbb39d700 1 -- 172.21.15.55:0/246775798 shutdown_connections 2023-10-06T03:48:04.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.875+0000 7f4cbb39d700 1 -- 172.21.15.55:0/246775798 wait complete. 2023-10-06T03:48:04.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.876+0000 7f4cbb39d700 1 Processor -- start 2023-10-06T03:48:04.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.877+0000 7f4cbb39d700 1 -- start start 2023-10-06T03:48:04.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.877+0000 7f4cbb39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cb40b3b70 0x7f4cb416c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:04.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.877+0000 7f4cbb39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb4142ea0 0x7f4cb415c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:04.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.878+0000 7f4cbb39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cb415cb80 0x7f4cb40cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:04.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.878+0000 7f4cbb39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb415d020 con 0x7f4cb415cb80 2023-10-06T03:48:04.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.878+0000 7f4cbb39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb416a450 con 0x7f4cb40b3b70 2023-10-06T03:48:04.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.878+0000 7f4cbb39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb416c910 con 0x7f4cb4142ea0 2023-10-06T03:48:04.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.878+0000 7f4cbab9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cb415cb80 0x7f4cb40cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:04.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.878+0000 7f4cbab9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cb415cb80 0x7f4cb40cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54336/0 (socket says 172.21.15.55:54336) 2023-10-06T03:48:04.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.878+0000 7f4cbab9c700 1 -- 172.21.15.55:0/3344230199 learned_addr learned my addr 172.21.15.55:0/3344230199 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:04.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.878+0000 7f4cba39b700 1 --2- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cb40b3b70 0x7f4cb416c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:04.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.878+0000 7f4cbab9c700 1 -- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb4142ea0 msgr2=0x7f4cb415c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:04.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.878+0000 7f4cbab9c700 1 --2- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb4142ea0 0x7f4cb415c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.878+0000 7f4cbab9c700 1 -- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cb40b3b70 msgr2=0x7f4cb416c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:04.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.878+0000 7f4cbab9c700 1 --2- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cb40b3b70 0x7f4cb416c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:04.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.878+0000 7f4cbab9c700 1 -- 172.21.15.55:0/3344230199 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb0009420 con 0x7f4cb415cb80 2023-10-06T03:48:04.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.879+0000 7f4cbab9c700 1 --2- 172.21.15.55:0/3344230199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cb415cb80 0x7f4cb40cc4e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00cbc0 tx=0x7f4cac007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:04.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.879+0000 7f4caa7fc700 1 -- 172.21.15.55:0/3344230199 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac004dc0 con 0x7f4cb415cb80 2023-10-06T03:48:04.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.879+0000 7f4caa7fc700 1 -- 172.21.15.55:0/3344230199 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cac00a640 con 0x7f4cb415cb80 2023-10-06T03:48:04.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.879+0000 7f4cbb39d700 1 -- 172.21.15.55:0/3344230199 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca4009420 con 0x7f4cb415cb80 2023-10-06T03:48:04.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.879+0000 7f4caa7fc700 1 -- 172.21.15.55:0/3344230199 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac00aa00 con 0x7f4cb415cb80 2023-10-06T03:48:04.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.879+0000 7f4cbb39d700 1 -- 172.21.15.55:0/3344230199 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb4169840 con 0x7f4cb415cb80 2023-10-06T03:48:04.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.880+0000 7f4cbb39d700 1 -- 172.21.15.55:0/3344230199 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb404c3c0 con 0x7f4cb415cb80 2023-10-06T03:48:04.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.881+0000 7f4caa7fc700 1 -- 172.21.15.55:0/3344230199 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 23) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f4cac007150 con 0x7f4cb415cb80 2023-10-06T03:48:04.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.882+0000 7f4caa7fc700 1 --2- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4c98067050 0x7f4c98069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:04.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.882+0000 7f4caa7fc700 1 -- 172.21.15.55:0/3344230199 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6738+0+0 (secure 0 0 0) 0x7f4cac084690 con 0x7f4cb415cb80 2023-10-06T03:48:04.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.882+0000 7f4cba39b700 1 -- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4c98067050 msgr2=0x7f4c98069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:04.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.882+0000 7f4cba39b700 1 --2- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4c98067050 0x7f4c98069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:04.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:04.886+0000 7f4caa7fc700 1 -- 172.21.15.55:0/3344230199 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4cac0513c0 con 0x7f4cb415cb80 2023-10-06T03:48:05.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.083+0000 7f4cba39b700 1 -- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4c98067050 msgr2=0x7f4c98069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:05.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.083+0000 7f4cba39b700 1 --2- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4c98067050 0x7f4c98069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:05.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.089+0000 7f4cbb39d700 1 -- 172.21.15.55:0/3344230199 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4cb4143270 con 0x7f4cb415cb80 2023-10-06T03:48:05.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.090+0000 7f4caa7fc700 1 -- 172.21.15.55:0/3344230199 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17934 (secure 0 0 0) 0x7f4cac018090 con 0x7f4cb415cb80 2023-10-06T03:48:05.092 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:05.092 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":67,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:47:23.822280+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:46:05.457264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T03:46:06.258505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.1a","mappings":[{"from":0,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3009002237":"2023-10-07T03:46:22.715958+0000","172.21.15.163:6837/85191":"2023-10-07T03:46:22.715958+0000","172.21.15.163:6835/85191":"2023-10-07T03:46:22.715958+0000","172.21.15.163:0/1689910889":"2023-10-07T03:46:22.715958+0000","172.21.15.163:0/2097952590":"2023-10-07T03:46:22.715958+0000","172.21.15.163:0/4134100032":"2023-10-07T03:46:22.715958+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:48:05.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.092+0000 7f4cbb39d700 1 -- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4c98067050 msgr2=0x7f4c98069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:05.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.092+0000 7f4cbb39d700 1 --2- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4c98067050 0x7f4c98069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.092+0000 7f4cbb39d700 1 -- 172.21.15.55:0/3344230199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cb415cb80 msgr2=0x7f4cb40cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:05.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.092+0000 7f4cbb39d700 1 --2- 172.21.15.55:0/3344230199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cb415cb80 0x7f4cb40cc4e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00cbc0 tx=0x7f4cac007ca0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.092+0000 7f4cbb39d700 1 -- 172.21.15.55:0/3344230199 shutdown_connections 2023-10-06T03:48:05.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.092+0000 7f4cbb39d700 1 --2- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4c98067050 0x7f4c98069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.092+0000 7f4cbb39d700 1 --2- 172.21.15.55:0/3344230199 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cb415cb80 0x7f4cb40cc4e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.092+0000 7f4cbb39d700 1 --2- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cb40b3b70 0x7f4cb416c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.092+0000 7f4cbb39d700 1 --2- 172.21.15.55:0/3344230199 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cb4142ea0 0x7f4cb415c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.092+0000 7f4cbb39d700 1 -- 172.21.15.55:0/3344230199 >> 172.21.15.55:0/3344230199 conn(0x7f4cb40ce430 msgr2=0x7f4cb4141f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:05.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.092+0000 7f4cbb39d700 1 -- 172.21.15.55:0/3344230199 shutdown_connections 2023-10-06T03:48:05.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.093+0000 7f4cbb39d700 1 -- 172.21.15.55:0/3344230199 wait complete. 2023-10-06T03:48:05.112 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:46:05.457264+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:46:06.258505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:48:05.113 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:48:05.159 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:05.157+0000 7fa553956180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:48:05.219 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:05.218+0000 7f80ccb8c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:48:05.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.305+0000 7fd7d179d700 1 Processor -- start 2023-10-06T03:48:05.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.305+0000 7fd7d179d700 1 -- start start 2023-10-06T03:48:05.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.306+0000 7fd7d179d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7cc0b16d0 0x7fd7cc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:05.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.306+0000 7fd7d179d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd7cc0b33b0 0x7fd7cc0b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:05.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.306+0000 7fd7d179d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7cc0b2080 con 0x7fd7cc0b4e70 2023-10-06T03:48:05.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.306+0000 7fd7d179d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7cc0cfb00 con 0x7fd7cc0b33b0 2023-10-06T03:48:05.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.306+0000 7fd7d179d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0b5c40 con 0x7fd7cc0b16d0 2023-10-06T03:48:05.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.306+0000 7fd7cbfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd7cc0b4e70 0x7fd7cc0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43410/0 (socket says 172.21.15.55:43410) 2023-10-06T03:48:05.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.306+0000 7fd7cbfff700 1 -- 172.21.15.55:0/2857386643 learned_addr learned my addr 172.21.15.55:0/2857386643 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:05.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.306+0000 7fd7d0f9c700 1 --2- 172.21.15.55:0/2857386643 >> v2:172.21.15.55:3300/0 conn(0x7fd7cc0b33b0 0x7fd7cc0b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:05.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.306+0000 7fd7cb7fe700 1 --2- 172.21.15.55:0/2857386643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7cc0b16d0 0x7fd7cc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:05.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.307+0000 7fd7caffd700 1 -- 172.21.15.55:0/2857386643 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2478114930 0 0) 0x7fd7cc0b2080 con 0x7fd7cc0b4e70 2023-10-06T03:48:05.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.307+0000 7fd7caffd700 1 -- 172.21.15.55:0/2857386643 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7b4003570 con 0x7fd7cc0b4e70 2023-10-06T03:48:05.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.307+0000 7fd7d0f9c700 1 -- 172.21.15.55:0/2857386643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7cc0b16d0 msgr2=0x7fd7cc0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:05.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.307+0000 7fd7d0f9c700 1 --2- 172.21.15.55:0/2857386643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7cc0b16d0 0x7fd7cc0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.307+0000 7fd7d0f9c700 1 -- 172.21.15.55:0/2857386643 >> v1:172.21.15.55:6789/0 conn(0x7fd7cc0b4e70 legacy=0x7fd7cc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:05.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.307+0000 7fd7d0f9c700 1 -- 172.21.15.55:0/2857386643 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc0b5f20 con 0x7fd7cc0b33b0 2023-10-06T03:48:05.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.307+0000 7fd7d0f9c700 1 --2- 172.21.15.55:0/2857386643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc0b33b0 0x7fd7cc0b3830 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009e60 tx=0x7fd7c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1e140922f67f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:05.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.308+0000 7fd7caffd700 1 -- 172.21.15.55:0/2857386643 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c400e070 con 0x7fd7cc0b33b0 2023-10-06T03:48:05.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.308+0000 7fd7caffd700 1 -- 172.21.15.55:0/2857386643 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c400bab0 con 0x7fd7cc0b33b0 2023-10-06T03:48:05.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.308+0000 7fd7caffd700 1 -- 172.21.15.55:0/2857386643 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c400be30 con 0x7fd7cc0b33b0 2023-10-06T03:48:05.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.308+0000 7fd7d179d700 1 -- 172.21.15.55:0/2857386643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc0b33b0 msgr2=0x7fd7cc0b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:05.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.308+0000 7fd7d179d700 1 --2- 172.21.15.55:0/2857386643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc0b33b0 0x7fd7cc0b3830 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009e60 tx=0x7fd7c4009780 comp rx=0 tx=0).stop 2023-10-06T03:48:05.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.308+0000 7fd7d179d700 1 -- 172.21.15.55:0/2857386643 shutdown_connections 2023-10-06T03:48:05.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.308+0000 7fd7d179d700 1 --2- 172.21.15.55:0/2857386643 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc0b33b0 0x7fd7cc0b3830 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.308+0000 7fd7d179d700 1 --2- 172.21.15.55:0/2857386643 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7cc0b16d0 0x7fd7cc0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.308+0000 7fd7d179d700 1 -- 172.21.15.55:0/2857386643 >> 172.21.15.55:0/2857386643 conn(0x7fd7cc0d10c0 msgr2=0x7fd7cc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:05.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.308+0000 7fd7d179d700 1 -- 172.21.15.55:0/2857386643 shutdown_connections 2023-10-06T03:48:05.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.308+0000 7fd7d179d700 1 -- 172.21.15.55:0/2857386643 wait complete. 2023-10-06T03:48:05.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.310+0000 7fd7d179d700 1 Processor -- start 2023-10-06T03:48:05.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.310+0000 7fd7d179d700 1 -- start start 2023-10-06T03:48:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.311+0000 7fd7d179d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7cc0b16d0 0x7fd7cc15bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.311+0000 7fd7d179d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7cc0b33b0 0x7fd7cc0697a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.311+0000 7fd7d179d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc0b4e70 0x7fd7cc155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.311+0000 7fd7d179d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc1699b0 con 0x7fd7cc0b4e70 2023-10-06T03:48:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.311+0000 7fd7d179d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc156780 con 0x7fd7cc0b33b0 2023-10-06T03:48:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.311+0000 7fd7d179d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc0c5f60 con 0x7fd7cc0b16d0 2023-10-06T03:48:05.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.311+0000 7fd7d0f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc0b4e70 0x7fd7cc155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:05.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.311+0000 7fd7cbfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7cc0b16d0 0x7fd7cc15bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:05.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.311+0000 7fd7cbfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7cc0b16d0 0x7fd7cc15bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:55732/0 (socket says 172.21.15.55:55732) 2023-10-06T03:48:05.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.311+0000 7fd7cbfff700 1 -- 172.21.15.55:0/4241187453 learned_addr learned my addr 172.21.15.55:0/4241187453 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:05.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.311+0000 7fd7cb7fe700 1 --2- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7cc0b33b0 0x7fd7cc0697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:05.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.312+0000 7fd7cbfff700 1 -- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7cc0b33b0 msgr2=0x7fd7cc0697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:05.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.312+0000 7fd7cbfff700 1 --2- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7cc0b33b0 0x7fd7cc0697a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.312+0000 7fd7cbfff700 1 -- 172.21.15.55:0/4241187453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc0b4e70 msgr2=0x7fd7cc155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:05.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.312+0000 7fd7cbfff700 1 --2- 172.21.15.55:0/4241187453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc0b4e70 0x7fd7cc155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.312+0000 7fd7cbfff700 1 -- 172.21.15.55:0/4241187453 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c4009a20 con 0x7fd7cc0b16d0 2023-10-06T03:48:05.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.312+0000 7fd7cbfff700 1 --2- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7cc0b16d0 0x7fd7cc15bbf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0011e0 tx=0x7fd7bc007df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:05.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.312+0000 7fd7b3fff700 1 -- 172.21.15.55:0/4241187453 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc005b30 con 0x7fd7cc0b16d0 2023-10-06T03:48:05.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.312+0000 7fd7b3fff700 1 -- 172.21.15.55:0/4241187453 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc005c90 con 0x7fd7cc0b16d0 2023-10-06T03:48:05.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.312+0000 7fd7d179d700 1 -- 172.21.15.55:0/4241187453 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7cc0c6240 con 0x7fd7cc0b16d0 2023-10-06T03:48:05.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.312+0000 7fd7b3fff700 1 -- 172.21.15.55:0/4241187453 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc007300 con 0x7fd7cc0b16d0 2023-10-06T03:48:05.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.312+0000 7fd7d179d700 1 -- 172.21.15.55:0/4241187453 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc0c6800 con 0x7fd7cc0b16d0 2023-10-06T03:48:05.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.313+0000 7fd7d179d700 1 -- 172.21.15.55:0/4241187453 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc088990 con 0x7fd7cc0b16d0 2023-10-06T03:48:05.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.314+0000 7fd7b3fff700 1 -- 172.21.15.55:0/4241187453 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 23) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fd7bc007670 con 0x7fd7cc0b16d0 2023-10-06T03:48:05.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.315+0000 7fd7b3fff700 1 --2- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd7b8067050 0x7fd7b8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:05.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.315+0000 7fd7b3fff700 1 -- 172.21.15.55:0/4241187453 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6738+0+0 (secure 0 0 0) 0x7fd7bc083f70 con 0x7fd7cc0b16d0 2023-10-06T03:48:05.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.315+0000 7fd7cb7fe700 1 -- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd7b8067050 msgr2=0x7fd7b8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:05.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.315+0000 7fd7cb7fe700 1 --2- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd7b8067050 0x7fd7b8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:05.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.320+0000 7fd7b3fff700 1 -- 172.21.15.55:0/4241187453 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7bc050c20 con 0x7fd7cc0b16d0 2023-10-06T03:48:05.326 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:05.324+0000 7fa553956180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:48:05.393 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:05.392+0000 7f80ccb8c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:48:05.412 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:05.410+0000 7fa553956180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:48:05.483 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:05.481+0000 7f80ccb8c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:48:05.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.516+0000 7fd7cb7fe700 1 -- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd7b8067050 msgr2=0x7fd7b8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:05.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.516+0000 7fd7cb7fe700 1 --2- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd7b8067050 0x7fd7b8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:05.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.520+0000 7fd7d179d700 1 -- 172.21.15.55:0/4241187453 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd7cc0cf850 con 0x7fd7cc0b16d0 2023-10-06T03:48:05.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.521+0000 7fd7b3fff700 1 -- 172.21.15.55:0/4241187453 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v67) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd7bc017020 con 0x7fd7cc0b16d0 2023-10-06T03:48:05.523 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:48:05.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.523+0000 7fd7d179d700 1 -- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd7b8067050 msgr2=0x7fd7b8069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:05.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.523+0000 7fd7d179d700 1 --2- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd7b8067050 0x7fd7b8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.523+0000 7fd7d179d700 1 -- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7cc0b16d0 msgr2=0x7fd7cc15bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:05.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.523+0000 7fd7d179d700 1 --2- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7cc0b16d0 0x7fd7cc15bbf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0011e0 tx=0x7fd7bc007df0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.523+0000 7fd7d179d700 1 -- 172.21.15.55:0/4241187453 shutdown_connections 2023-10-06T03:48:05.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.524+0000 7fd7d179d700 1 --2- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd7b8067050 0x7fd7b8069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.524+0000 7fd7d179d700 1 --2- 172.21.15.55:0/4241187453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7cc0b4e70 0x7fd7cc155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.524+0000 7fd7d179d700 1 --2- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd7cc0b16d0 0x7fd7cc15bbf0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.524+0000 7fd7d179d700 1 --2- 172.21.15.55:0/4241187453 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd7cc0b33b0 0x7fd7cc0697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.524+0000 7fd7d179d700 1 -- 172.21.15.55:0/4241187453 >> 172.21.15.55:0/4241187453 conn(0x7fd7cc0d10c0 msgr2=0x7fd7cc1590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:05.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.524+0000 7fd7d179d700 1 -- 172.21.15.55:0/4241187453 shutdown_connections 2023-10-06T03:48:05.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.524+0000 7fd7d179d700 1 -- 172.21.15.55:0/4241187453 wait complete. 2023-10-06T03:48:05.545 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:48:05.638 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:05.636+0000 7fa553956180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:48:05.721 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:05.719+0000 7f80ccb8c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:48:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.736+0000 7f281559d700 1 Processor -- start 2023-10-06T03:48:05.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.736+0000 7f281559d700 1 -- start start 2023-10-06T03:48:05.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.738+0000 7f281559d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f28100b4e70 0x7f28100ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:05.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.738+0000 7f281559d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f28100b1b40 0x7f281015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:05.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.738+0000 7f281559d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28100cfb00 con 0x7f28100b16d0 2023-10-06T03:48:05.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.738+0000 7f281559d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28100b5c40 con 0x7f28100b4e70 2023-10-06T03:48:05.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.738+0000 7f281559d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28100b5f20 con 0x7f28100b1b40 2023-10-06T03:48:05.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.738+0000 7f280ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f28100b4e70 0x7f28100ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:05.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.738+0000 7f280ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f28100b4e70 0x7f28100ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54374/0 (socket says 172.21.15.55:54374) 2023-10-06T03:48:05.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.738+0000 7f280ffff700 1 -- 172.21.15.55:0/3157882982 learned_addr learned my addr 172.21.15.55:0/3157882982 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:05.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.738+0000 7f2814d9c700 1 --2- 172.21.15.55:0/3157882982 >> v2:172.21.15.163:3300/0 conn(0x7f28100b1b40 0x7f281015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.738+0000 7f280ffff700 1 -- 172.21.15.55:0/3157882982 >> v2:172.21.15.163:3300/0 conn(0x7f28100b1b40 msgr2=0x7f281015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.738+0000 7f280ffff700 1 --2- 172.21.15.55:0/3157882982 >> v2:172.21.15.163:3300/0 conn(0x7f28100b1b40 0x7f281015d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.738+0000 7f280ffff700 1 -- 172.21.15.55:0/3157882982 >> v1:172.21.15.163:6789/0 conn(0x7f28100b16d0 legacy=0x7f281015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.738+0000 7f280ffff700 1 -- 172.21.15.55:0/3157882982 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28100b6200 con 0x7f28100b4e70 2023-10-06T03:48:05.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.738+0000 7f2814d9c700 1 --2- 172.21.15.55:0/3157882982 >> v2:172.21.15.163:3300/0 conn(0x7f28100b1b40 0x7f281015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.739+0000 7f280ffff700 1 --2- 172.21.15.55:0/3157882982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28100b4e70 0x7f28100ccff0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2810071380 tx=0x7f2800009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f4377b7efce14d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.739+0000 7f280effd700 1 -- 172.21.15.55:0/3157882982 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280000b860 con 0x7f28100b4e70 2023-10-06T03:48:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.739+0000 7f280effd700 1 -- 172.21.15.55:0/3157882982 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280000b9c0 con 0x7f28100b4e70 2023-10-06T03:48:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.739+0000 7f280effd700 1 -- 172.21.15.55:0/3157882982 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280000bd60 con 0x7f28100b4e70 2023-10-06T03:48:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.740+0000 7f281559d700 1 -- 172.21.15.55:0/3157882982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28100b4e70 msgr2=0x7f28100ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.740+0000 7f281559d700 1 --2- 172.21.15.55:0/3157882982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28100b4e70 0x7f28100ccff0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2810071380 tx=0x7f2800009250 comp rx=0 tx=0).stop 2023-10-06T03:48:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.740+0000 7f281559d700 1 -- 172.21.15.55:0/3157882982 shutdown_connections 2023-10-06T03:48:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.740+0000 7f281559d700 1 --2- 172.21.15.55:0/3157882982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28100b4e70 0x7f28100ccff0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.740+0000 7f281559d700 1 --2- 172.21.15.55:0/3157882982 >> v2:172.21.15.163:3300/0 conn(0x7f28100b1b40 0x7f281015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.740+0000 7f281559d700 1 -- 172.21.15.55:0/3157882982 >> 172.21.15.55:0/3157882982 conn(0x7f28100d10c0 msgr2=0x7f28100affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:05.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.740+0000 7f281559d700 1 -- 172.21.15.55:0/3157882982 shutdown_connections 2023-10-06T03:48:05.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.740+0000 7f281559d700 1 -- 172.21.15.55:0/3157882982 wait complete. 2023-10-06T03:48:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.741+0000 7f281559d700 1 Processor -- start 2023-10-06T03:48:05.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.742+0000 7f281559d700 1 -- start start 2023-10-06T03:48:05.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.742+0000 7f281559d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28100b16d0 0x7f28100aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:05.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.742+0000 7f281559d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28100b1b40 0x7f28100c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:05.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.742+0000 7f281559d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28100b4e70 0x7f2810169e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:05.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.743+0000 7f281559d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28100c9a20 con 0x7f28100b4e70 2023-10-06T03:48:05.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.743+0000 7f281559d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28100c6490 con 0x7f28100b16d0 2023-10-06T03:48:05.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.743+0000 7f281559d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28100c6770 con 0x7f28100b1b40 2023-10-06T03:48:05.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.743+0000 7f280ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28100b16d0 0x7f28100aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:05.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.743+0000 7f2814d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28100b4e70 0x7f2810169e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:05.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.743+0000 7f280ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28100b16d0 0x7f28100aec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:41732/0 (socket says 172.21.15.55:41732) 2023-10-06T03:48:05.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.743+0000 7f280ffff700 1 -- 172.21.15.55:0/3343871860 learned_addr learned my addr 172.21.15.55:0/3343871860 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:05.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.743+0000 7f2814d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28100b4e70 0x7f2810169e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54378/0 (socket says 172.21.15.55:54378) 2023-10-06T03:48:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.743+0000 7f280f7fe700 1 --2- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28100b1b40 0x7f28100c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.743+0000 7f2814d9c700 1 -- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28100b1b40 msgr2=0x7f28100c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.743+0000 7f2814d9c700 1 --2- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28100b1b40 0x7f28100c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.743+0000 7f2814d9c700 1 -- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28100b16d0 msgr2=0x7f28100aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.743+0000 7f2814d9c700 1 --2- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28100b16d0 0x7f28100aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.743+0000 7f2814d9c700 1 -- 172.21.15.55:0/3343871860 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28000094f0 con 0x7f28100b4e70 2023-10-06T03:48:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.744+0000 7f2814d9c700 1 --2- 172.21.15.55:0/3343871860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28100b4e70 0x7f2810169e40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2808009fa0 tx=0x7f2808009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.744+0000 7f27f7fff700 1 -- 172.21.15.55:0/3343871860 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280800dbb0 con 0x7f28100b4e70 2023-10-06T03:48:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.744+0000 7f27f7fff700 1 -- 172.21.15.55:0/3343871860 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280800f040 con 0x7f28100b4e70 2023-10-06T03:48:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.744+0000 7f281559d700 1 -- 172.21.15.55:0/3343871860 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28100c6a50 con 0x7f28100b4e70 2023-10-06T03:48:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.744+0000 7f27f7fff700 1 -- 172.21.15.55:0/3343871860 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808015440 con 0x7f28100b4e70 2023-10-06T03:48:05.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.744+0000 7f281559d700 1 -- 172.21.15.55:0/3343871860 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28100af1b0 con 0x7f28100b4e70 2023-10-06T03:48:05.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.745+0000 7f281559d700 1 -- 172.21.15.55:0/3343871860 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28100b2a20 con 0x7f28100b4e70 2023-10-06T03:48:05.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.746+0000 7f27f7fff700 1 -- 172.21.15.55:0/3343871860 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 23) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f280801e3b0 con 0x7f28100b4e70 2023-10-06T03:48:05.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.747+0000 7f27f7fff700 1 --2- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f27fc067050 0x7f27fc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:05.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.747+0000 7f27f7fff700 1 -- 172.21.15.55:0/3343871860 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6738+0+0 (secure 0 0 0) 0x7f2808085060 con 0x7f28100b4e70 2023-10-06T03:48:05.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.747+0000 7f280ffff700 1 -- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f27fc067050 msgr2=0x7f27fc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:05.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.747+0000 7f280ffff700 1 --2- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f27fc067050 0x7f27fc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:05.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.755+0000 7f27f7fff700 1 -- 172.21.15.55:0/3343871860 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2808051c00 con 0x7f28100b4e70 2023-10-06T03:48:05.819 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:05.816+0000 7fa553956180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:48:05.912 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:05.910+0000 7f80ccb8c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:48:05.913 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:05.912+0000 7fa553956180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:48:05.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.948+0000 7f280ffff700 1 -- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f27fc067050 msgr2=0x7f27fc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:05.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.948+0000 7f280ffff700 1 --2- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f27fc067050 0x7f27fc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:05.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.961+0000 7f281559d700 1 -- 172.21.15.55:0/3343871860 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f281004bb80 con 0x7f28100b4e70 2023-10-06T03:48:05.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.961+0000 7f27f7fff700 1 -- 172.21.15.55:0/3343871860 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f280801a0a0 con 0x7f28100b4e70 2023-10-06T03:48:05.963 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:48:05.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.963+0000 7f281559d700 1 -- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f27fc067050 msgr2=0x7f27fc069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:05.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.963+0000 7f281559d700 1 --2- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f27fc067050 0x7f27fc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.963+0000 7f281559d700 1 -- 172.21.15.55:0/3343871860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28100b4e70 msgr2=0x7f2810169e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:05.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.963+0000 7f281559d700 1 --2- 172.21.15.55:0/3343871860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28100b4e70 0x7f2810169e40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2808009fa0 tx=0x7f2808009990 comp rx=0 tx=0).stop 2023-10-06T03:48:05.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.963+0000 7f281559d700 1 -- 172.21.15.55:0/3343871860 shutdown_connections 2023-10-06T03:48:05.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.963+0000 7f281559d700 1 --2- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f27fc067050 0x7f27fc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.963+0000 7f281559d700 1 --2- 172.21.15.55:0/3343871860 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28100b4e70 0x7f2810169e40 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.963+0000 7f281559d700 1 --2- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28100b16d0 0x7f28100aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.963+0000 7f281559d700 1 --2- 172.21.15.55:0/3343871860 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28100b1b40 0x7f28100c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:05.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.963+0000 7f281559d700 1 -- 172.21.15.55:0/3343871860 >> 172.21.15.55:0/3343871860 conn(0x7f28100d10c0 msgr2=0x7f281015ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:05.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.964+0000 7f281559d700 1 -- 172.21.15.55:0/3343871860 shutdown_connections 2023-10-06T03:48:05.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:05.964+0000 7f281559d700 1 -- 172.21.15.55:0/3343871860 wait complete. 2023-10-06T03:48:05.983 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:48:06.011 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:06.010+0000 7f80ccb8c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:48:06.076 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:06.074+0000 7fa553956180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:48:06.177 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:06.175+0000 7fa553956180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:48:06.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.177+0000 7fe95e19d700 1 Processor -- start 2023-10-06T03:48:06.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.177+0000 7fe95e19d700 1 -- start start 2023-10-06T03:48:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.179+0000 7fe95e19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9580aef00 0x7fe95815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.179+0000 7fe95e19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9580aea70 con 0x7fe95806e470 2023-10-06T03:48:06.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.179+0000 7fe95e19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9580ae480 con 0x7fe95815a3b0 2023-10-06T03:48:06.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.179+0000 7fe95e19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95815dd30 con 0x7fe9580aef00 2023-10-06T03:48:06.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.179+0000 7fe95d99c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe95806e470 0x7fe9580c7280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43424/0 (socket says 172.21.15.55:43424) 2023-10-06T03:48:06.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.179+0000 7fe95d99c700 1 -- 172.21.15.55:0/3197209263 learned_addr learned my addr 172.21.15.55:0/3197209263 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:06.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.179+0000 7fe95c99a700 1 --2- 172.21.15.55:0/3197209263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9580aef00 0x7fe95815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:06.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.179+0000 7fe94ffff700 1 -- 172.21.15.55:0/3197209263 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361263535 0 0) 0x7fe9580aea70 con 0x7fe95806e470 2023-10-06T03:48:06.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.179+0000 7fe94ffff700 1 -- 172.21.15.55:0/3197209263 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe944003570 con 0x7fe95806e470 2023-10-06T03:48:06.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.179+0000 7fe94ffff700 1 -- 172.21.15.55:0/3197209263 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166140191 0 0) 0x7fe9580ae480 con 0x7fe95815a3b0 2023-10-06T03:48:06.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.180+0000 7fe94ffff700 1 -- 172.21.15.55:0/3197209263 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9580aea70 con 0x7fe95815a3b0 2023-10-06T03:48:06.182 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:06.180+0000 7f80ccb8c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:48:06.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.180+0000 7fe95c99a700 1 -- 172.21.15.55:0/3197209263 >> v1:172.21.15.163:6789/0 conn(0x7fe95815a3b0 legacy=0x7fe95815a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:06.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.180+0000 7fe95c99a700 1 -- 172.21.15.55:0/3197209263 >> v1:172.21.15.55:6789/0 conn(0x7fe95806e470 legacy=0x7fe9580c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:06.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.180+0000 7fe95c99a700 1 -- 172.21.15.55:0/3197209263 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95815e010 con 0x7fe9580aef00 2023-10-06T03:48:06.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.180+0000 7fe95c99a700 1 --2- 172.21.15.55:0/3197209263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9580aef00 0x7fe95815d3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe948009910 tx=0x7fe9480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98634d29dd3333ff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:06.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.180+0000 7fe94ffff700 1 -- 172.21.15.55:0/3197209263 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948011630 con 0x7fe9580aef00 2023-10-06T03:48:06.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.180+0000 7fe94ffff700 1 -- 172.21.15.55:0/3197209263 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe948011790 con 0x7fe9580aef00 2023-10-06T03:48:06.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.180+0000 7fe94ffff700 1 -- 172.21.15.55:0/3197209263 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948011b30 con 0x7fe9580aef00 2023-10-06T03:48:06.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.181+0000 7fe95e19d700 1 -- 172.21.15.55:0/3197209263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9580aef00 msgr2=0x7fe95815d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:06.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.181+0000 7fe95e19d700 1 --2- 172.21.15.55:0/3197209263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9580aef00 0x7fe95815d3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe948009910 tx=0x7fe9480091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.181+0000 7fe95e19d700 1 -- 172.21.15.55:0/3197209263 shutdown_connections 2023-10-06T03:48:06.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.181+0000 7fe95e19d700 1 --2- 172.21.15.55:0/3197209263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe9580aef00 0x7fe95815d3c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.181+0000 7fe95e19d700 1 -- 172.21.15.55:0/3197209263 >> 172.21.15.55:0/3197209263 conn(0x7fe9580c9f70 msgr2=0x7fe95806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:06.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.181+0000 7fe95e19d700 1 -- 172.21.15.55:0/3197209263 shutdown_connections 2023-10-06T03:48:06.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.181+0000 7fe95e19d700 1 -- 172.21.15.55:0/3197209263 wait complete. 2023-10-06T03:48:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.183+0000 7fe95e19d700 1 Processor -- start 2023-10-06T03:48:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.183+0000 7fe95e19d700 1 -- start start 2023-10-06T03:48:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95e19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95806e470 0x7fe9580c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95e19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9580aef00 0x7fe9580c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95e19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe95815a3b0 0x7fe958141410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95e19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580af870 con 0x7fe9580aef00 2023-10-06T03:48:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95e19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580b0110 con 0x7fe95806e470 2023-10-06T03:48:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95e19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9580c59c0 con 0x7fe95815a3b0 2023-10-06T03:48:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95c99a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9580aef00 0x7fe9580c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:06.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95d99c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe95815a3b0 0x7fe958141410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95c99a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9580aef00 0x7fe9580c5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54386/0 (socket says 172.21.15.55:54386) 2023-10-06T03:48:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95c99a700 1 -- 172.21.15.55:0/583339110 learned_addr learned my addr 172.21.15.55:0/583339110 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95d19b700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95806e470 0x7fe9580c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95c99a700 1 -- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe95815a3b0 msgr2=0x7fe958141410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95c99a700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe95815a3b0 0x7fe958141410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95c99a700 1 -- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95806e470 msgr2=0x7fe9580c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95c99a700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95806e470 0x7fe9580c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.184+0000 7fe95c99a700 1 -- 172.21.15.55:0/583339110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe948009420 con 0x7fe9580aef00 2023-10-06T03:48:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.185+0000 7fe95d99c700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe95815a3b0 0x7fe958141410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.185+0000 7fe95d19b700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95806e470 0x7fe9580c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:06.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.185+0000 7fe95c99a700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9580aef00 0x7fe9580c5480 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe948009fd0 tx=0x7fe948017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:06.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.185+0000 7fe94d7fa700 1 -- 172.21.15.55:0/583339110 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94801f070 con 0x7fe9580aef00 2023-10-06T03:48:06.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.185+0000 7fe94d7fa700 1 -- 172.21.15.55:0/583339110 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe948002f70 con 0x7fe9580aef00 2023-10-06T03:48:06.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.185+0000 7fe95e19d700 1 -- 172.21.15.55:0/583339110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9580c6240 con 0x7fe9580aef00 2023-10-06T03:48:06.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.185+0000 7fe94d7fa700 1 -- 172.21.15.55:0/583339110 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948011900 con 0x7fe9580aef00 2023-10-06T03:48:06.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.185+0000 7fe95e19d700 1 -- 172.21.15.55:0/583339110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9580afda0 con 0x7fe9580aef00 2023-10-06T03:48:06.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.186+0000 7fe95e19d700 1 -- 172.21.15.55:0/583339110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe958159e30 con 0x7fe9580aef00 2023-10-06T03:48:06.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.187+0000 7fe94d7fa700 1 -- 172.21.15.55:0/583339110 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 23) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fe948023070 con 0x7fe9580aef00 2023-10-06T03:48:06.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.189+0000 7fe94d7fa700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe93c067050 0x7fe93c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:06.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.189+0000 7fe94d7fa700 1 -- 172.21.15.55:0/583339110 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6738+0+0 (secure 0 0 0) 0x7fe948085d80 con 0x7fe9580aef00 2023-10-06T03:48:06.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.189+0000 7fe95d19b700 1 -- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe93c067050 msgr2=0x7fe93c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:06.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.189+0000 7fe95d19b700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe93c067050 0x7fe93c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:06.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.193+0000 7fe94d7fa700 1 -- 172.21.15.55:0/583339110 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe948052a30 con 0x7fe9580aef00 2023-10-06T03:48:06.289 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:06.288+0000 7f80ccb8c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:48:06.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.390+0000 7fe95d19b700 1 -- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe93c067050 msgr2=0x7fe93c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:06.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.390+0000 7fe95d19b700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe93c067050 0x7fe93c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:06.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.398+0000 7fe95e19d700 1 -- 172.21.15.55:0/583339110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9580b0250 con 0x7fe9580aef00 2023-10-06T03:48:06.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.399+0000 7fe94d7fa700 1 -- 172.21.15.55:0/583339110 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9480150a0 con 0x7fe9580aef00 2023-10-06T03:48:06.400 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T03:48:06.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.401+0000 7fe95e19d700 1 -- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe93c067050 msgr2=0x7fe93c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:06.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.401+0000 7fe95e19d700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe93c067050 0x7fe93c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.401+0000 7fe95e19d700 1 -- 172.21.15.55:0/583339110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9580aef00 msgr2=0x7fe9580c5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:06.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.401+0000 7fe95e19d700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9580aef00 0x7fe9580c5480 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe948009fd0 tx=0x7fe948017000 comp rx=0 tx=0).stop 2023-10-06T03:48:06.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.401+0000 7fe95e19d700 1 -- 172.21.15.55:0/583339110 shutdown_connections 2023-10-06T03:48:06.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.401+0000 7fe95e19d700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe93c067050 0x7fe93c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.401+0000 7fe95e19d700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe95815a3b0 0x7fe958141410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.401+0000 7fe95e19d700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe95806e470 0x7fe9580c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.401+0000 7fe95e19d700 1 --2- 172.21.15.55:0/583339110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe9580aef00 0x7fe9580c5480 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.401+0000 7fe95e19d700 1 -- 172.21.15.55:0/583339110 >> 172.21.15.55:0/583339110 conn(0x7fe9580c9f70 msgr2=0x7fe958152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:06.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.401+0000 7fe95e19d700 1 -- 172.21.15.55:0/583339110 shutdown_connections 2023-10-06T03:48:06.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.401+0000 7fe95e19d700 1 -- 172.21.15.55:0/583339110 wait complete. 2023-10-06T03:48:06.420 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:48:06.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.654+0000 7f2bc6f9d700 1 Processor -- start 2023-10-06T03:48:06.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.654+0000 7f2bc6f9d700 1 -- start start 2023-10-06T03:48:06.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.655+0000 7f2bc6f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc006e470 0x7f2bc006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:06.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.655+0000 7f2bc6f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc00aea70 con 0x7f2bc00aef00 2023-10-06T03:48:06.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.655+0000 7f2bc6f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc015dad0 con 0x7f2bc015a3b0 2023-10-06T03:48:06.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.655+0000 7f2bc6f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc006ee30 con 0x7f2bc006e470 2023-10-06T03:48:06.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.656+0000 7f2bc579a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2bc00aef00 0x7f2bc015d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43432/0 (socket says 172.21.15.55:43432) 2023-10-06T03:48:06.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.656+0000 7f2bc579a700 1 -- 172.21.15.55:0/143953384 learned_addr learned my addr 172.21.15.55:0/143953384 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:06.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.656+0000 7f2bc679c700 1 --2- 172.21.15.55:0/143953384 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc006e470 0x7f2bc006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:06.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.656+0000 7f2bc4f99700 1 -- 172.21.15.55:0/143953384 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011274760 0 0) 0x7f2bc00aea70 con 0x7f2bc00aef00 2023-10-06T03:48:06.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.656+0000 7f2bc4f99700 1 -- 172.21.15.55:0/143953384 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bac003570 con 0x7f2bc00aef00 2023-10-06T03:48:06.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.656+0000 7f2bc4f99700 1 -- 172.21.15.55:0/143953384 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498554546 0 0) 0x7f2bc015dad0 con 0x7f2bc015a3b0 2023-10-06T03:48:06.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.656+0000 7f2bc4f99700 1 -- 172.21.15.55:0/143953384 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bc00aea70 con 0x7f2bc015a3b0 2023-10-06T03:48:06.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.656+0000 7f2bc679c700 1 -- 172.21.15.55:0/143953384 >> v1:172.21.15.163:6789/0 conn(0x7f2bc015a3b0 legacy=0x7f2bc015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:06.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.656+0000 7f2bc679c700 1 -- 172.21.15.55:0/143953384 >> v1:172.21.15.55:6789/0 conn(0x7f2bc00aef00 legacy=0x7f2bc015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:06.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.656+0000 7f2bc679c700 1 -- 172.21.15.55:0/143953384 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc00ae480 con 0x7f2bc006e470 2023-10-06T03:48:06.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.657+0000 7f2bc679c700 1 --2- 172.21.15.55:0/143953384 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc006e470 0x7f2bc006e8f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2bc015dc60 tx=0x7f2bb8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e584ad873d4da0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:06.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.657+0000 7f2bc4f99700 1 -- 172.21.15.55:0/143953384 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8016070 con 0x7f2bc006e470 2023-10-06T03:48:06.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.657+0000 7f2bc4f99700 1 -- 172.21.15.55:0/143953384 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb800bab0 con 0x7f2bc006e470 2023-10-06T03:48:06.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.657+0000 7f2bc4f99700 1 -- 172.21.15.55:0/143953384 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb800e070 con 0x7f2bc006e470 2023-10-06T03:48:06.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.658+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/143953384 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc006e470 msgr2=0x7f2bc006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:06.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.658+0000 7f2bc6f9d700 1 --2- 172.21.15.55:0/143953384 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc006e470 0x7f2bc006e8f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2bc015dc60 tx=0x7f2bb8009780 comp rx=0 tx=0).stop 2023-10-06T03:48:06.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.658+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/143953384 shutdown_connections 2023-10-06T03:48:06.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.658+0000 7f2bc6f9d700 1 --2- 172.21.15.55:0/143953384 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc006e470 0x7f2bc006e8f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.658+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/143953384 >> 172.21.15.55:0/143953384 conn(0x7f2bc00c9f70 msgr2=0x7f2bc006dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:06.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.658+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/143953384 shutdown_connections 2023-10-06T03:48:06.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.658+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/143953384 wait complete. 2023-10-06T03:48:06.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.659+0000 7f2bc6f9d700 1 Processor -- start 2023-10-06T03:48:06.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.659+0000 7f2bc6f9d700 1 -- start start 2023-10-06T03:48:06.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.660+0000 7f2bc6f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc006e470 0x7f2bc00af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:06.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.660+0000 7f2bc6f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc00aef00 0x7f2bc0152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:06.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.660+0000 7f2bc6f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bc015a3b0 0x7f2bc00c2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:06.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.660+0000 7f2bc6f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc0159bf0 con 0x7f2bc015a3b0 2023-10-06T03:48:06.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.660+0000 7f2bc6f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00b0b80 con 0x7f2bc006e470 2023-10-06T03:48:06.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.660+0000 7f2bc6f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00aff70 con 0x7f2bc00aef00 2023-10-06T03:48:06.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.660+0000 7f2bc679c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bc015a3b0 0x7f2bc00c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:06.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.661+0000 7f2bc579a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc00aef00 0x7f2bc0152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:06.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.661+0000 7f2bc679c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bc015a3b0 0x7f2bc00c2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54390/0 (socket says 172.21.15.55:54390) 2023-10-06T03:48:06.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.661+0000 7f2bc679c700 1 -- 172.21.15.55:0/687965140 learned_addr learned my addr 172.21.15.55:0/687965140 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:06.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.661+0000 7f2bc679c700 1 -- 172.21.15.55:0/687965140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc00aef00 msgr2=0x7f2bc0152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:06.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.661+0000 7f2bc679c700 1 --2- 172.21.15.55:0/687965140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc00aef00 0x7f2bc0152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.661+0000 7f2bc679c700 1 -- 172.21.15.55:0/687965140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc006e470 msgr2=0x7f2bc00af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:06.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.661+0000 7f2bc679c700 1 --2- 172.21.15.55:0/687965140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc006e470 0x7f2bc00af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.661+0000 7f2bc679c700 1 -- 172.21.15.55:0/687965140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb8009a20 con 0x7f2bc015a3b0 2023-10-06T03:48:06.666 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:06.661+0000 7fa553956180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:48:06.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.661+0000 7f2bc679c700 1 --2- 172.21.15.55:0/687965140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bc015a3b0 0x7f2bc00c2800 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00714a0 tx=0x7f2bb8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:06.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.661+0000 7f2bb5ffb700 1 -- 172.21.15.55:0/687965140 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8027470 con 0x7f2bc015a3b0 2023-10-06T03:48:06.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.662+0000 7f2bb5ffb700 1 -- 172.21.15.55:0/687965140 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb80275d0 con 0x7f2bc015a3b0 2023-10-06T03:48:06.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.662+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/687965140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc00b0250 con 0x7f2bc015a3b0 2023-10-06T03:48:06.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.662+0000 7f2bb5ffb700 1 -- 172.21.15.55:0/687965140 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8027990 con 0x7f2bc015a3b0 2023-10-06T03:48:06.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.662+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/687965140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc00b0810 con 0x7f2bc015a3b0 2023-10-06T03:48:06.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.663+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/687965140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc006cfa0 con 0x7f2bc015a3b0 2023-10-06T03:48:06.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.664+0000 7f2bb5ffb700 1 -- 172.21.15.55:0/687965140 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 23) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f2bb8024070 con 0x7f2bc015a3b0 2023-10-06T03:48:06.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.664+0000 7f2bb5ffb700 1 --2- 172.21.15.55:0/687965140 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2ba4067050 0x7f2ba4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:06.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.664+0000 7f2bb5ffb700 1 -- 172.21.15.55:0/687965140 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6738+0+0 (secure 0 0 0) 0x7f2bb8087170 con 0x7f2bc015a3b0 2023-10-06T03:48:06.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.665+0000 7f2bc5f9b700 1 -- 172.21.15.55:0/687965140 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2ba4067050 msgr2=0x7f2ba4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:06.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.665+0000 7f2bc5f9b700 1 --2- 172.21.15.55:0/687965140 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2ba4067050 0x7f2ba4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:06.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.669+0000 7f2bb5ffb700 1 -- 172.21.15.55:0/687965140 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2bb8053ec0 con 0x7f2bc015a3b0 2023-10-06T03:48:06.758 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:06.756+0000 7fa553956180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:48:06.801 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:06.800+0000 7f80ccb8c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:48:06.851 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:06.849+0000 7fa553956180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:48:06.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.866+0000 7f2bc5f9b700 1 -- 172.21.15.55:0/687965140 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2ba4067050 msgr2=0x7f2ba4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:06.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.866+0000 7f2bc5f9b700 1 --2- 172.21.15.55:0/687965140 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2ba4067050 0x7f2ba4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:06.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.871+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/687965140 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bc015dd30 con 0x7f2bc015a3b0 2023-10-06T03:48:06.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.872+0000 7f2bb5ffb700 1 -- 172.21.15.55:0/687965140 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bb80574e0 con 0x7f2bc015a3b0 2023-10-06T03:48:06.874 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T03:48:06.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.874+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/687965140 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2ba4067050 msgr2=0x7f2ba4069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:06.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.874+0000 7f2bc6f9d700 1 --2- 172.21.15.55:0/687965140 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2ba4067050 0x7f2ba4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.874+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/687965140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bc015a3b0 msgr2=0x7f2bc00c2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:06.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.874+0000 7f2bc6f9d700 1 --2- 172.21.15.55:0/687965140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bc015a3b0 0x7f2bc00c2800 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00714a0 tx=0x7f2bb8018000 comp rx=0 tx=0).stop 2023-10-06T03:48:06.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.874+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/687965140 shutdown_connections 2023-10-06T03:48:06.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.874+0000 7f2bc6f9d700 1 --2- 172.21.15.55:0/687965140 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2ba4067050 0x7f2ba4069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.874+0000 7f2bc6f9d700 1 --2- 172.21.15.55:0/687965140 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2bc015a3b0 0x7f2bc00c2800 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.874+0000 7f2bc6f9d700 1 --2- 172.21.15.55:0/687965140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2bc006e470 0x7f2bc00af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.874+0000 7f2bc6f9d700 1 --2- 172.21.15.55:0/687965140 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2bc00aef00 0x7f2bc0152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:06.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.874+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/687965140 >> 172.21.15.55:0/687965140 conn(0x7f2bc00c9f70 msgr2=0x7f2bc015aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:06.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.874+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/687965140 shutdown_connections 2023-10-06T03:48:06.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:06.874+0000 7f2bc6f9d700 1 -- 172.21.15.55:0/687965140 wait complete. 2023-10-06T03:48:06.894 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:48:06.905 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:06.904+0000 7f80ccb8c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:48:07.005 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:07.004+0000 7f80ccb8c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:48:07.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.086+0000 7f803119d700 1 Processor -- start 2023-10-06T03:48:07.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.086+0000 7f803119d700 1 -- start start 2023-10-06T03:48:07.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.087+0000 7f803119d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c15d230 0x7f802c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:07.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.087+0000 7f803119d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c0b2980 con 0x7f802c1597f0 2023-10-06T03:48:07.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.087+0000 7f803119d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c15dbb0 con 0x7f802c159420 2023-10-06T03:48:07.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.087+0000 7f803119d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0b11d0 con 0x7f802c15d230 2023-10-06T03:48:07.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.087+0000 7f802b7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f802c1597f0 0x7f802c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43434/0 (socket says 172.21.15.55:43434) 2023-10-06T03:48:07.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.087+0000 7f802b7fe700 1 -- 172.21.15.55:0/4289150844 learned_addr learned my addr 172.21.15.55:0/4289150844 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:07.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.088+0000 7f803099c700 1 --2- 172.21.15.55:0/4289150844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c15d230 0x7f802c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:07.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.088+0000 7f802affd700 1 -- 172.21.15.55:0/4289150844 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663859675 0 0) 0x7f802c0b2980 con 0x7f802c1597f0 2023-10-06T03:48:07.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.088+0000 7f802affd700 1 -- 172.21.15.55:0/4289150844 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8014003570 con 0x7f802c1597f0 2023-10-06T03:48:07.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.088+0000 7f802affd700 1 -- 172.21.15.55:0/4289150844 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003347335 0 0) 0x7f802c15dbb0 con 0x7f802c159420 2023-10-06T03:48:07.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.088+0000 7f802affd700 1 -- 172.21.15.55:0/4289150844 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f802c0b2980 con 0x7f802c159420 2023-10-06T03:48:07.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.088+0000 7f803099c700 1 -- 172.21.15.55:0/4289150844 >> v1:172.21.15.163:6789/0 conn(0x7f802c159420 legacy=0x7f802c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:07.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.088+0000 7f803099c700 1 -- 172.21.15.55:0/4289150844 >> v1:172.21.15.55:6789/0 conn(0x7f802c1597f0 legacy=0x7f802c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:07.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.088+0000 7f803099c700 1 -- 172.21.15.55:0/4289150844 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0b14b0 con 0x7f802c15d230 2023-10-06T03:48:07.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.089+0000 7f803099c700 1 --2- 172.21.15.55:0/4289150844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c15d230 0x7f802c15d670 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8024009e90 tx=0x7f8024009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4c63a40f11f8488 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:07.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.089+0000 7f802affd700 1 -- 172.21.15.55:0/4289150844 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802400e070 con 0x7f802c15d230 2023-10-06T03:48:07.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.089+0000 7f802affd700 1 -- 172.21.15.55:0/4289150844 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802400bb80 con 0x7f802c15d230 2023-10-06T03:48:07.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.089+0000 7f802affd700 1 -- 172.21.15.55:0/4289150844 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024012430 con 0x7f802c15d230 2023-10-06T03:48:07.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.089+0000 7f803119d700 1 -- 172.21.15.55:0/4289150844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c15d230 msgr2=0x7f802c15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:07.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.089+0000 7f803119d700 1 --2- 172.21.15.55:0/4289150844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c15d230 0x7f802c15d670 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8024009e90 tx=0x7f8024009780 comp rx=0 tx=0).stop 2023-10-06T03:48:07.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.089+0000 7f803119d700 1 -- 172.21.15.55:0/4289150844 shutdown_connections 2023-10-06T03:48:07.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.089+0000 7f803119d700 1 --2- 172.21.15.55:0/4289150844 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c15d230 0x7f802c15d670 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.089+0000 7f803119d700 1 -- 172.21.15.55:0/4289150844 >> 172.21.15.55:0/4289150844 conn(0x7f802c0d0660 msgr2=0x7f802c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:07.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.090+0000 7f803119d700 1 -- 172.21.15.55:0/4289150844 shutdown_connections 2023-10-06T03:48:07.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.090+0000 7f803119d700 1 -- 172.21.15.55:0/4289150844 wait complete. 2023-10-06T03:48:07.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.091+0000 7f803119d700 1 Processor -- start 2023-10-06T03:48:07.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.091+0000 7f803119d700 1 -- start start 2023-10-06T03:48:07.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.092+0000 7f803119d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802c159420 0x7f802c159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:07.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.092+0000 7f803119d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c1597f0 0x7f802c150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:07.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.092+0000 7f803119d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f802c15d230 0x7f802c14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:07.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.092+0000 7f803119d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c15a280 con 0x7f802c159420 2023-10-06T03:48:07.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.092+0000 7f803119d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c15a560 con 0x7f802c15d230 2023-10-06T03:48:07.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.092+0000 7f803119d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c15a840 con 0x7f802c1597f0 2023-10-06T03:48:07.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.092+0000 7f802bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802c159420 0x7f802c159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:07.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.092+0000 7f802bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802c159420 0x7f802c159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54396/0 (socket says 172.21.15.55:54396) 2023-10-06T03:48:07.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.092+0000 7f802bfff700 1 -- 172.21.15.55:0/336946062 learned_addr learned my addr 172.21.15.55:0/336946062 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:07.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.093+0000 7f802b7fe700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c1597f0 0x7f802c150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:07.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.093+0000 7f803099c700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f802c15d230 0x7f802c14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:07.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.093+0000 7f802bfff700 1 -- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c1597f0 msgr2=0x7f802c150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:07.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.093+0000 7f802bfff700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c1597f0 0x7f802c150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.093+0000 7f802bfff700 1 -- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f802c15d230 msgr2=0x7f802c14f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:07.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.093+0000 7f802bfff700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f802c15d230 0x7f802c14f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.093+0000 7f802bfff700 1 -- 172.21.15.55:0/336946062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8024009a20 con 0x7f802c159420 2023-10-06T03:48:07.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.093+0000 7f803099c700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f802c15d230 0x7f802c14f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:07.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.093+0000 7f802bfff700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802c159420 0x7f802c159ca0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f801c00c7c0 tx=0x7f801c007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:07.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.093+0000 7f8013fff700 1 -- 172.21.15.55:0/336946062 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c016070 con 0x7f802c159420 2023-10-06T03:48:07.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.093+0000 7f8013fff700 1 -- 172.21.15.55:0/336946062 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801c011680 con 0x7f802c159420 2023-10-06T03:48:07.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.093+0000 7f803119d700 1 -- 172.21.15.55:0/336946062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802c15ab20 con 0x7f802c159420 2023-10-06T03:48:07.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.093+0000 7f8013fff700 1 -- 172.21.15.55:0/336946062 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c011a20 con 0x7f802c159420 2023-10-06T03:48:07.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.094+0000 7f803119d700 1 -- 172.21.15.55:0/336946062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c150da0 con 0x7f802c159420 2023-10-06T03:48:07.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.095+0000 7f803119d700 1 -- 172.21.15.55:0/336946062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c15b040 con 0x7f802c159420 2023-10-06T03:48:07.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.095+0000 7f8013fff700 1 -- 172.21.15.55:0/336946062 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 23) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f801c012440 con 0x7f802c159420 2023-10-06T03:48:07.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.096+0000 7f8013fff700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8018067050 0x7f8018069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:07.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.096+0000 7f8013fff700 1 -- 172.21.15.55:0/336946062 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6738+0+0 (secure 0 0 0) 0x7f801c0849b0 con 0x7f802c159420 2023-10-06T03:48:07.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.096+0000 7f802b7fe700 1 -- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8018067050 msgr2=0x7f8018069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:07.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.096+0000 7f802b7fe700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8018067050 0x7f8018069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:07.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.100+0000 7f8013fff700 1 -- 172.21.15.55:0/336946062 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f802c15b040 con 0x7f802c159420 2023-10-06T03:48:07.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.297+0000 7f802b7fe700 1 -- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8018067050 msgr2=0x7f8018069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.297+0000 7f802b7fe700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8018067050 0x7f8018069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:07.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.369+0000 7f803119d700 1 -- 172.21.15.55:0/336946062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f802c15b040 con 0x7f802c159420 2023-10-06T03:48:07.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.370+0000 7f8013fff700 1 -- 172.21.15.55:0/336946062 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f802c15b040 con 0x7f802c159420 2023-10-06T03:48:07.372 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:07.372 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3493866751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3493866751},{"type":"v1","addr":"172.21.15.163:6836","nonce":3493866751}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3246061279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3246061279},{"type":"v1","addr":"172.21.15.55:6835","nonce":3246061279}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1366768080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1366768080},{"type":"v1","addr":"172.21.15.55:6834","nonce":1366768080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:46:07.292050+0000","modified":"2023-10-06T03:46:10.815261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6833/2287859778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2287859778},{"type":"v1","addr":"172.21.15.163:6833","nonce":2287859778}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T03:48:07.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.372+0000 7f803119d700 1 -- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8018067050 msgr2=0x7f8018069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:07.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.372+0000 7f803119d700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8018067050 0x7f8018069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.372+0000 7f803119d700 1 -- 172.21.15.55:0/336946062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802c159420 msgr2=0x7f802c159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:07.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.372+0000 7f803119d700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802c159420 0x7f802c159ca0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f801c00c7c0 tx=0x7f801c007dd0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.373+0000 7f803119d700 1 -- 172.21.15.55:0/336946062 shutdown_connections 2023-10-06T03:48:07.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.373+0000 7f803119d700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8018067050 0x7f8018069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.373+0000 7f803119d700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f802c15d230 0x7f802c14f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.373+0000 7f803119d700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802c159420 0x7f802c159ca0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.373+0000 7f803119d700 1 --2- 172.21.15.55:0/336946062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802c1597f0 0x7f802c150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.373+0000 7f803119d700 1 -- 172.21.15.55:0/336946062 >> 172.21.15.55:0/336946062 conn(0x7f802c0d0660 msgr2=0x7f802c1556e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:07.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.373+0000 7f803119d700 1 -- 172.21.15.55:0/336946062 shutdown_connections 2023-10-06T03:48:07.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.373+0000 7f803119d700 1 -- 172.21.15.55:0/336946062 wait complete. 2023-10-06T03:48:07.376 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 26 2023-10-06T03:48:07.392 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:48:07.482 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:07.480+0000 7fa553956180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:48:07.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.652+0000 7f273179d700 1 Processor -- start 2023-10-06T03:48:07.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.652+0000 7f273179d700 1 -- start start 2023-10-06T03:48:07.655 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:07.652+0000 7fa553956180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:48:07.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.654+0000 7f273179d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f272c0ad870 0x7f272c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:07.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.654+0000 7f273179d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f272c06bf20 0x7f272c06a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:07.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.654+0000 7f273179d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f272c06c350 con 0x7f272c142ea0 2023-10-06T03:48:07.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.654+0000 7f273179d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f272c0b40f0 con 0x7f272c06bf20 2023-10-06T03:48:07.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.654+0000 7f273179d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c069640 con 0x7f272c0ad870 2023-10-06T03:48:07.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.654+0000 7f272bfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f272c142ea0 0x7f272c0b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43436/0 (socket says 172.21.15.55:43436) 2023-10-06T03:48:07.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.654+0000 7f272bfff700 1 -- 172.21.15.55:0/2820034304 learned_addr learned my addr 172.21.15.55:0/2820034304 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:07.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.654+0000 7f272b7fe700 1 --2- 172.21.15.55:0/2820034304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f272c0ad870 0x7f272c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:07.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.654+0000 7f2730f9c700 1 --2- 172.21.15.55:0/2820034304 >> v2:172.21.15.163:3300/0 conn(0x7f272c06bf20 0x7f272c06a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:07.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.654+0000 7f272affd700 1 -- 172.21.15.55:0/2820034304 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364333000 0 0) 0x7f272c06c350 con 0x7f272c142ea0 2023-10-06T03:48:07.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.655+0000 7f272affd700 1 -- 172.21.15.55:0/2820034304 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2714003570 con 0x7f272c142ea0 2023-10-06T03:48:07.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.655+0000 7f272b7fe700 1 -- 172.21.15.55:0/2820034304 >> v2:172.21.15.163:3300/0 conn(0x7f272c06bf20 msgr2=0x7f272c06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:07.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.655+0000 7f272b7fe700 1 --2- 172.21.15.55:0/2820034304 >> v2:172.21.15.163:3300/0 conn(0x7f272c06bf20 0x7f272c06a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.655+0000 7f272b7fe700 1 -- 172.21.15.55:0/2820034304 >> v1:172.21.15.55:6789/0 conn(0x7f272c142ea0 legacy=0x7f272c0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:07.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.655+0000 7f272b7fe700 1 -- 172.21.15.55:0/2820034304 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c069920 con 0x7f272c0ad870 2023-10-06T03:48:07.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.655+0000 7f2730f9c700 1 --2- 172.21.15.55:0/2820034304 >> v2:172.21.15.163:3300/0 conn(0x7f272c06bf20 0x7f272c06a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:07.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.655+0000 7f272b7fe700 1 --2- 172.21.15.55:0/2820034304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f272c0ad870 0x7f272c06b9e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f27200098e0 tx=0x7f27200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e65d70a4e7aabd21 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:07.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.655+0000 7f272affd700 1 -- 172.21.15.55:0/2820034304 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2720011630 con 0x7f272c0ad870 2023-10-06T03:48:07.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.655+0000 7f272affd700 1 -- 172.21.15.55:0/2820034304 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2720011790 con 0x7f272c0ad870 2023-10-06T03:48:07.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.655+0000 7f272affd700 1 -- 172.21.15.55:0/2820034304 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2720011b30 con 0x7f272c0ad870 2023-10-06T03:48:07.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.656+0000 7f273179d700 1 -- 172.21.15.55:0/2820034304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f272c0ad870 msgr2=0x7f272c06b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:07.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.656+0000 7f273179d700 1 --2- 172.21.15.55:0/2820034304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f272c0ad870 0x7f272c06b9e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f27200098e0 tx=0x7f27200091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.656+0000 7f273179d700 1 -- 172.21.15.55:0/2820034304 shutdown_connections 2023-10-06T03:48:07.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.656+0000 7f273179d700 1 --2- 172.21.15.55:0/2820034304 >> v2:172.21.15.163:3300/0 conn(0x7f272c06bf20 0x7f272c06a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.656+0000 7f273179d700 1 --2- 172.21.15.55:0/2820034304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f272c0ad870 0x7f272c06b9e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.656+0000 7f273179d700 1 -- 172.21.15.55:0/2820034304 >> 172.21.15.55:0/2820034304 conn(0x7f272c0ce430 msgr2=0x7f272c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:07.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.656+0000 7f273179d700 1 -- 172.21.15.55:0/2820034304 shutdown_connections 2023-10-06T03:48:07.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.656+0000 7f273179d700 1 -- 172.21.15.55:0/2820034304 wait complete. 2023-10-06T03:48:07.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.657+0000 7f273179d700 1 Processor -- start 2023-10-06T03:48:07.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.658+0000 7f273179d700 1 -- start start 2023-10-06T03:48:07.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.658+0000 7f273179d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f272c06bf20 0x7f272c0c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:07.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.658+0000 7f273179d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f272c142ea0 0x7f272c158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:07.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.658+0000 7f273179d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f272c0c85b0 0x7f272c15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:07.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.658+0000 7f273179d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0c8b00 con 0x7f272c06bf20 2023-10-06T03:48:07.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.658+0000 7f273179d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0c8de0 con 0x7f272c142ea0 2023-10-06T03:48:07.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.658+0000 7f273179d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0c90c0 con 0x7f272c0c85b0 2023-10-06T03:48:07.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.659+0000 7f272bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f272c06bf20 0x7f272c0c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:07.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.659+0000 7f272bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f272c06bf20 0x7f272c0c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54398/0 (socket says 172.21.15.55:54398) 2023-10-06T03:48:07.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.659+0000 7f272bfff700 1 -- 172.21.15.55:0/684860589 learned_addr learned my addr 172.21.15.55:0/684860589 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:07.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.659+0000 7f272b7fe700 1 --2- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f272c142ea0 0x7f272c158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:07.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.659+0000 7f2730f9c700 1 --2- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f272c0c85b0 0x7f272c15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:07.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.659+0000 7f272bfff700 1 -- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f272c0c85b0 msgr2=0x7f272c15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:07.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.659+0000 7f272bfff700 1 --2- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f272c0c85b0 0x7f272c15ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.659+0000 7f272bfff700 1 -- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f272c142ea0 msgr2=0x7f272c158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:07.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.659+0000 7f272bfff700 1 --2- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f272c142ea0 0x7f272c158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.659+0000 7f272bfff700 1 -- 172.21.15.55:0/684860589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2724009a20 con 0x7f272c06bf20 2023-10-06T03:48:07.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.659+0000 7f272bfff700 1 --2- 172.21.15.55:0/684860589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f272c06bf20 0x7f272c0c9dd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f271c00c590 tx=0x7f271c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:07.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.660+0000 7f2713fff700 1 -- 172.21.15.55:0/684860589 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c005cd0 con 0x7f272c06bf20 2023-10-06T03:48:07.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.660+0000 7f273179d700 1 -- 172.21.15.55:0/684860589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2720009420 con 0x7f272c06bf20 2023-10-06T03:48:07.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.660+0000 7f2713fff700 1 -- 172.21.15.55:0/684860589 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271c005e50 con 0x7f272c06bf20 2023-10-06T03:48:07.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.660+0000 7f2713fff700 1 -- 172.21.15.55:0/684860589 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c007320 con 0x7f272c06bf20 2023-10-06T03:48:07.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.660+0000 7f273179d700 1 -- 172.21.15.55:0/684860589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272c0ca310 con 0x7f272c06bf20 2023-10-06T03:48:07.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.661+0000 7f273179d700 1 -- 172.21.15.55:0/684860589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f272c04c3c0 con 0x7f272c06bf20 2023-10-06T03:48:07.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.662+0000 7f2713fff700 1 -- 172.21.15.55:0/684860589 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 23) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f271c01b3b0 con 0x7f272c06bf20 2023-10-06T03:48:07.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.663+0000 7f2713fff700 1 --2- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2718067050 0x7f2718069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:07.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.664+0000 7f272b7fe700 1 -- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2718067050 msgr2=0x7f2718069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:07.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.664+0000 7f2713fff700 1 -- 172.21.15.55:0/684860589 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6738+0+0 (secure 0 0 0) 0x7f271c0841d0 con 0x7f272c06bf20 2023-10-06T03:48:07.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.664+0000 7f272b7fe700 1 --2- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2718067050 0x7f2718069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:07.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.668+0000 7f2713fff700 1 -- 172.21.15.55:0/684860589 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f271c050f40 con 0x7f272c06bf20 2023-10-06T03:48:07.693 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:07.691+0000 7f80ccb8c180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:48:07.752 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:07.750+0000 7fa553956180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:48:07.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.865+0000 7f272b7fe700 1 -- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2718067050 msgr2=0x7f2718069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:07.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.865+0000 7f272b7fe700 1 --2- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2718067050 0x7f2718069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:07.879 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:07.877+0000 7f80ccb8c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:48:07.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.880+0000 7f273179d700 1 -- 172.21.15.55:0/684860589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f272c15bda0 con 0x7f272c06bf20 2023-10-06T03:48:07.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.881+0000 7f2713fff700 1 -- 172.21.15.55:0/684860589 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18043 (secure 0 0 0) 0x7f271c054560 con 0x7f272c06bf20 2023-10-06T03:48:07.883 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:07.883 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":68,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:48:07.857325+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:46:05.457264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T03:46:06.258505+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":68,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.1a","mappings":[{"from":0,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3009002237":"2023-10-07T03:46:22.715958+0000","172.21.15.163:6837/85191":"2023-10-07T03:46:22.715958+0000","172.21.15.163:6835/85191":"2023-10-07T03:46:22.715958+0000","172.21.15.163:0/1689910889":"2023-10-07T03:46:22.715958+0000","172.21.15.163:0/2097952590":"2023-10-07T03:46:22.715958+0000","172.21.15.163:0/4134100032":"2023-10-07T03:46:22.715958+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:48:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.884+0000 7f273179d700 1 -- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2718067050 msgr2=0x7f2718069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.884+0000 7f273179d700 1 --2- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2718067050 0x7f2718069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.884+0000 7f273179d700 1 -- 172.21.15.55:0/684860589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f272c06bf20 msgr2=0x7f272c0c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.884+0000 7f273179d700 1 --2- 172.21.15.55:0/684860589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f272c06bf20 0x7f272c0c9dd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f271c00c590 tx=0x7f271c007e00 comp rx=0 tx=0).stop 2023-10-06T03:48:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.884+0000 7f273179d700 1 -- 172.21.15.55:0/684860589 shutdown_connections 2023-10-06T03:48:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.884+0000 7f273179d700 1 --2- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2718067050 0x7f2718069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.884+0000 7f273179d700 1 --2- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f272c0c85b0 0x7f272c15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.884+0000 7f273179d700 1 --2- 172.21.15.55:0/684860589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f272c06bf20 0x7f272c0c9dd0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.884+0000 7f273179d700 1 --2- 172.21.15.55:0/684860589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f272c142ea0 0x7f272c158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:07.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.884+0000 7f273179d700 1 -- 172.21.15.55:0/684860589 >> 172.21.15.55:0/684860589 conn(0x7f272c0ce430 msgr2=0x7f272c0cc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:07.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.884+0000 7f273179d700 1 -- 172.21.15.55:0/684860589 shutdown_connections 2023-10-06T03:48:07.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:07.884+0000 7f273179d700 1 -- 172.21.15.55:0/684860589 wait complete. 2023-10-06T03:48:07.904 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:48:07.904 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:48:07.984 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:07.983+0000 7f80ccb8c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:48:08.056 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:08.054+0000 7fa553956180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:48:08.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.093+0000 7f70d53ff700 1 Processor -- start 2023-10-06T03:48:08.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.093+0000 7f70d53ff700 1 -- start start 2023-10-06T03:48:08.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.094+0000 7f70d53ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f70d00cbf30 0x7f70d00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:08.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.094+0000 7f70d53ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70d00ccf90 0x7f70d00b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:08.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.094+0000 7f70d53ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70d015a270 0x7f70d00b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:08.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.094+0000 7f70d53ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d0156bf0 con 0x7f70d00cbf30 2023-10-06T03:48:08.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.094+0000 7f70d53ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d00ae950 con 0x7f70d015a270 2023-10-06T03:48:08.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.094+0000 7f70d53ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00cc300 con 0x7f70d00ccf90 2023-10-06T03:48:08.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.095+0000 7f70ceffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f70d00cbf30 0x7f70d00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:08.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.095+0000 7f70ceffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f70d00cbf30 0x7f70d00ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54402/0 (socket says 172.21.15.55:54402) 2023-10-06T03:48:08.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.095+0000 7f70ceffd700 1 -- 172.21.15.55:0/3319282402 learned_addr learned my addr 172.21.15.55:0/3319282402 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:08.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.095+0000 7f70ce7fc700 1 --2- 172.21.15.55:0/3319282402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70d00ccf90 0x7f70d00b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:08.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.095+0000 7f70cf7fe700 1 --2- 172.21.15.55:0/3319282402 >> v2:172.21.15.163:3300/0 conn(0x7f70d015a270 0x7f70d00b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:08.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.095+0000 7f70ceffd700 1 -- 172.21.15.55:0/3319282402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70d00ccf90 msgr2=0x7f70d00b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:08.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.095+0000 7f70ceffd700 1 --2- 172.21.15.55:0/3319282402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70d00ccf90 0x7f70d00b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.095+0000 7f70ceffd700 1 -- 172.21.15.55:0/3319282402 >> v2:172.21.15.163:3300/0 conn(0x7f70d015a270 msgr2=0x7f70d00b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:08.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.095+0000 7f70ceffd700 1 --2- 172.21.15.55:0/3319282402 >> v2:172.21.15.163:3300/0 conn(0x7f70d015a270 0x7f70d00b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.095+0000 7f70ceffd700 1 -- 172.21.15.55:0/3319282402 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d00ad370 con 0x7f70d00cbf30 2023-10-06T03:48:08.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.096+0000 7f70ceffd700 1 --2- 172.21.15.55:0/3319282402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d00cbf30 0x7f70d00ae410 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f70d00b25e0 tx=0x7f70c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93be04b671592358 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:08.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.096+0000 7f70cdffb700 1 -- 172.21.15.55:0/3319282402 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c000b640 con 0x7f70d00cbf30 2023-10-06T03:48:08.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.096+0000 7f70cdffb700 1 -- 172.21.15.55:0/3319282402 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c000b7a0 con 0x7f70d00cbf30 2023-10-06T03:48:08.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.096+0000 7f70cdffb700 1 -- 172.21.15.55:0/3319282402 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c000bb40 con 0x7f70d00cbf30 2023-10-06T03:48:08.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.097+0000 7f70d53ff700 1 -- 172.21.15.55:0/3319282402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d00cbf30 msgr2=0x7f70d00ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:08.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.097+0000 7f70d53ff700 1 --2- 172.21.15.55:0/3319282402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d00cbf30 0x7f70d00ae410 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f70d00b25e0 tx=0x7f70c0009250 comp rx=0 tx=0).stop 2023-10-06T03:48:08.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.097+0000 7f70d53ff700 1 -- 172.21.15.55:0/3319282402 shutdown_connections 2023-10-06T03:48:08.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.097+0000 7f70d53ff700 1 --2- 172.21.15.55:0/3319282402 >> v2:172.21.15.163:3300/0 conn(0x7f70d015a270 0x7f70d00b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.097+0000 7f70d53ff700 1 --2- 172.21.15.55:0/3319282402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d00cbf30 0x7f70d00ae410 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.097+0000 7f70d53ff700 1 --2- 172.21.15.55:0/3319282402 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70d00ccf90 0x7f70d00b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.097+0000 7f70d53ff700 1 -- 172.21.15.55:0/3319282402 >> 172.21.15.55:0/3319282402 conn(0x7f70d006a320 msgr2=0x7f70d00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:08.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.097+0000 7f70d53ff700 1 -- 172.21.15.55:0/3319282402 shutdown_connections 2023-10-06T03:48:08.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.097+0000 7f70d53ff700 1 -- 172.21.15.55:0/3319282402 wait complete. 2023-10-06T03:48:08.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.098+0000 7f70d53ff700 1 Processor -- start 2023-10-06T03:48:08.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.099+0000 7f70d53ff700 1 -- start start 2023-10-06T03:48:08.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70d53ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d00cbf30 0x7f70d015dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:08.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70d53ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70d00ccf90 0x7f70d015e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:08.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70d53ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d015a270 0x7f70d01416d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:08.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70d53ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d016b010 con 0x7f70d00cbf30 2023-10-06T03:48:08.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70d53ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d0159080 con 0x7f70d015a270 2023-10-06T03:48:08.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70d53ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d0159360 con 0x7f70d00ccf90 2023-10-06T03:48:08.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70ceffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d00cbf30 0x7f70d015dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:08.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70ce7fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70d00ccf90 0x7f70d015e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:08.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70ceffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d00cbf30 0x7f70d015dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54408/0 (socket says 172.21.15.55:54408) 2023-10-06T03:48:08.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70ceffd700 1 -- 172.21.15.55:0/1663593803 learned_addr learned my addr 172.21.15.55:0/1663593803 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:08.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70cf7fe700 1 --2- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d015a270 0x7f70d01416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:08.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70ce7fc700 1 -- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d015a270 msgr2=0x7f70d01416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:08.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70ce7fc700 1 --2- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d015a270 0x7f70d01416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70ce7fc700 1 -- 172.21.15.55:0/1663593803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d00cbf30 msgr2=0x7f70d015dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:08.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70ce7fc700 1 --2- 172.21.15.55:0/1663593803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d00cbf30 0x7f70d015dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70ce7fc700 1 -- 172.21.15.55:0/1663593803 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c00094f0 con 0x7f70d00ccf90 2023-10-06T03:48:08.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.100+0000 7f70ceffd700 1 --2- 172.21.15.55:0/1663593803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d00cbf30 0x7f70d015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:08.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.101+0000 7f70ce7fc700 1 --2- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70d00ccf90 0x7f70d015e2c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f70c400e9d0 tx=0x7f70c40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:08.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.101+0000 7f70b6ffd700 1 -- 172.21.15.55:0/1663593803 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4013680 con 0x7f70d00ccf90 2023-10-06T03:48:08.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.101+0000 7f70b6ffd700 1 -- 172.21.15.55:0/1663593803 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c4013800 con 0x7f70d00ccf90 2023-10-06T03:48:08.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.101+0000 7f70d53ff700 1 -- 172.21.15.55:0/1663593803 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d0159640 con 0x7f70d00ccf90 2023-10-06T03:48:08.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.101+0000 7f70b6ffd700 1 -- 172.21.15.55:0/1663593803 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c4013bc0 con 0x7f70d00ccf90 2023-10-06T03:48:08.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.101+0000 7f70d53ff700 1 -- 172.21.15.55:0/1663593803 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d0159920 con 0x7f70d00ccf90 2023-10-06T03:48:08.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.102+0000 7f70d53ff700 1 -- 172.21.15.55:0/1663593803 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d016b1e0 con 0x7f70d00ccf90 2023-10-06T03:48:08.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.103+0000 7f70b6ffd700 1 -- 172.21.15.55:0/1663593803 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 23) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f70c400c490 con 0x7f70d00ccf90 2023-10-06T03:48:08.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.104+0000 7f70b6ffd700 1 --2- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70bc067050 0x7f70bc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:08.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.104+0000 7f70b6ffd700 1 -- 172.21.15.55:0/1663593803 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6794+0+0 (secure 0 0 0) 0x7f70c4084b30 con 0x7f70d00ccf90 2023-10-06T03:48:08.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.104+0000 7f70ceffd700 1 -- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70bc067050 msgr2=0x7f70bc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:08.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.104+0000 7f70ceffd700 1 --2- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70bc067050 0x7f70bc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:08.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.108+0000 7f70b6ffd700 1 -- 172.21.15.55:0/1663593803 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70d016b1e0 con 0x7f70d00ccf90 2023-10-06T03:48:08.153 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:08.151+0000 7fa553956180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:48:08.297 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:08.295+0000 7f80ccb8c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:48:08.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.305+0000 7f70ceffd700 1 -- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70bc067050 msgr2=0x7f70bc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:08.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.305+0000 7f70ceffd700 1 --2- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70bc067050 0x7f70bc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:08.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.371+0000 7f70d53ff700 1 -- 172.21.15.55:0/1663593803 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70d004c3c0 con 0x7f70d00ccf90 2023-10-06T03:48:08.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.372+0000 7f70b6ffd700 1 -- 172.21.15.55:0/1663593803 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f70c40517b0 con 0x7f70d00ccf90 2023-10-06T03:48:08.373 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:08.373 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3493866751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3493866751},{"type":"v1","addr":"172.21.15.163:6836","nonce":3493866751}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3246061279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3246061279},{"type":"v1","addr":"172.21.15.55:6835","nonce":3246061279}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1366768080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1366768080},{"type":"v1","addr":"172.21.15.55:6834","nonce":1366768080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:46:07.292050+0000","modified":"2023-10-06T03:46:10.815261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6833/2287859778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2287859778},{"type":"v1","addr":"172.21.15.163:6833","nonce":2287859778}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T03:48:08.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.374+0000 7f70d53ff700 1 -- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70bc067050 msgr2=0x7f70bc069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:08.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.374+0000 7f70d53ff700 1 --2- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70bc067050 0x7f70bc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.374+0000 7f70d53ff700 1 -- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70d00ccf90 msgr2=0x7f70d015e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:08.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.374+0000 7f70d53ff700 1 --2- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70d00ccf90 0x7f70d015e2c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f70c400e9d0 tx=0x7f70c40091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.374+0000 7f70d53ff700 1 -- 172.21.15.55:0/1663593803 shutdown_connections 2023-10-06T03:48:08.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.374+0000 7f70d53ff700 1 --2- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70bc067050 0x7f70bc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.374+0000 7f70d53ff700 1 --2- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70d015a270 0x7f70d01416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.374+0000 7f70d53ff700 1 --2- 172.21.15.55:0/1663593803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70d00cbf30 0x7f70d015dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.374+0000 7f70d53ff700 1 --2- 172.21.15.55:0/1663593803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70d00ccf90 0x7f70d015e2c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.374+0000 7f70d53ff700 1 -- 172.21.15.55:0/1663593803 >> 172.21.15.55:0/1663593803 conn(0x7f70d006a320 msgr2=0x7f70d00cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:08.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.375+0000 7f70d53ff700 1 -- 172.21.15.55:0/1663593803 shutdown_connections 2023-10-06T03:48:08.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.375+0000 7f70d53ff700 1 -- 172.21.15.55:0/1663593803 wait complete. 2023-10-06T03:48:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 26 2023-10-06T03:48:08.394 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:48:08.400 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:08.398+0000 7f80ccb8c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:48:08.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.583+0000 7f4ea5fff700 1 Processor -- start 2023-10-06T03:48:08.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.583+0000 7f4ea5fff700 1 -- start start 2023-10-06T03:48:08.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.584+0000 7f4ea5fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00b1850 0x7f4ea00b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:08.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.584+0000 7f4ea5fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4ea01579f0 0x7f4ea00abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:08.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.585+0000 7f4ea5fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea00cb300 con 0x7f4ea00ae680 2023-10-06T03:48:08.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.585+0000 7f4ea5fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea006fc30 con 0x7f4ea01579f0 2023-10-06T03:48:08.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.585+0000 7f4ea5fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea00cc260 con 0x7f4ea00b1850 2023-10-06T03:48:08.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.585+0000 7f4e9effd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4ea00ae680 0x7f4ea01572e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43452/0 (socket says 172.21.15.55:43452) 2023-10-06T03:48:08.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.585+0000 7f4e9effd700 1 -- 172.21.15.55:0/301016391 learned_addr learned my addr 172.21.15.55:0/301016391 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:08.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.585+0000 7f4e9f7fe700 1 --2- 172.21.15.55:0/301016391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00b1850 0x7f4ea00b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:08.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.585+0000 7f4e9ffff700 1 --2- 172.21.15.55:0/301016391 >> v2:172.21.15.163:3300/0 conn(0x7f4ea01579f0 0x7f4ea00abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:08.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.585+0000 7f4e9e7fc700 1 -- 172.21.15.55:0/301016391 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833316195 0 0) 0x7f4ea00cb300 con 0x7f4ea00ae680 2023-10-06T03:48:08.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.585+0000 7f4e9e7fc700 1 -- 172.21.15.55:0/301016391 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e88003570 con 0x7f4ea00ae680 2023-10-06T03:48:08.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.585+0000 7f4e9ffff700 1 -- 172.21.15.55:0/301016391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00b1850 msgr2=0x7f4ea00b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:08.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.585+0000 7f4e9ffff700 1 --2- 172.21.15.55:0/301016391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00b1850 0x7f4ea00b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.586+0000 7f4e9ffff700 1 -- 172.21.15.55:0/301016391 >> v1:172.21.15.55:6789/0 conn(0x7f4ea00ae680 legacy=0x7f4ea01572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:08.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.586+0000 7f4e9ffff700 1 -- 172.21.15.55:0/301016391 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea00cb5e0 con 0x7f4ea01579f0 2023-10-06T03:48:08.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.586+0000 7f4e9f7fe700 1 --2- 172.21.15.55:0/301016391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00b1850 0x7f4ea00b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:08.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.586+0000 7f4e9ffff700 1 --2- 172.21.15.55:0/301016391 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea01579f0 0x7f4ea00abf60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009e60 tx=0x7f4e98009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=429ec9b3031e31ad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:08.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.586+0000 7f4e9e7fc700 1 -- 172.21.15.55:0/301016391 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800e070 con 0x7f4ea01579f0 2023-10-06T03:48:08.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.586+0000 7f4e9e7fc700 1 -- 172.21.15.55:0/301016391 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9800bab0 con 0x7f4ea01579f0 2023-10-06T03:48:08.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.586+0000 7f4e9e7fc700 1 -- 172.21.15.55:0/301016391 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9800be70 con 0x7f4ea01579f0 2023-10-06T03:48:08.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.587+0000 7f4ea5fff700 1 -- 172.21.15.55:0/301016391 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea01579f0 msgr2=0x7f4ea00abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:08.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.587+0000 7f4ea5fff700 1 --2- 172.21.15.55:0/301016391 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea01579f0 0x7f4ea00abf60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009e60 tx=0x7f4e98009780 comp rx=0 tx=0).stop 2023-10-06T03:48:08.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.587+0000 7f4ea5fff700 1 -- 172.21.15.55:0/301016391 shutdown_connections 2023-10-06T03:48:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.587+0000 7f4ea5fff700 1 --2- 172.21.15.55:0/301016391 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea01579f0 0x7f4ea00abf60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.587+0000 7f4ea5fff700 1 --2- 172.21.15.55:0/301016391 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00b1850 0x7f4ea00b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.587+0000 7f4ea5fff700 1 -- 172.21.15.55:0/301016391 >> 172.21.15.55:0/301016391 conn(0x7f4ea006a320 msgr2=0x7f4ea00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.587+0000 7f4ea5fff700 1 -- 172.21.15.55:0/301016391 shutdown_connections 2023-10-06T03:48:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.587+0000 7f4ea5fff700 1 -- 172.21.15.55:0/301016391 wait complete. 2023-10-06T03:48:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.588+0000 7f4ea5fff700 1 Processor -- start 2023-10-06T03:48:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.589+0000 7f4ea5fff700 1 -- start start 2023-10-06T03:48:08.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.589+0000 7f4ea5fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00ae680 0x7f4ea01535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:08.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.589+0000 7f4ea5fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea00b1850 0x7f4ea0141020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:08.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.589+0000 7f4ea5fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea01579f0 0x7f4ea015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:08.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.589+0000 7f4ea5fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea0141560 con 0x7f4ea01579f0 2023-10-06T03:48:08.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.590+0000 7f4ea5fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea015ccf0 con 0x7f4ea00b1850 2023-10-06T03:48:08.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.590+0000 7f4ea5fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea0154870 con 0x7f4ea00ae680 2023-10-06T03:48:08.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.590+0000 7f4e9ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea01579f0 0x7f4ea015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:08.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.590+0000 7f4e9ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea01579f0 0x7f4ea015c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54418/0 (socket says 172.21.15.55:54418) 2023-10-06T03:48:08.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.590+0000 7f4e9f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00ae680 0x7f4ea01535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:08.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.590+0000 7f4e9ffff700 1 -- 172.21.15.55:0/3283645502 learned_addr learned my addr 172.21.15.55:0/3283645502 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:08.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.590+0000 7f4e9effd700 1 --2- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea00b1850 0x7f4ea0141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:08.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.590+0000 7f4e9ffff700 1 -- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00ae680 msgr2=0x7f4ea01535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:08.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.590+0000 7f4e9ffff700 1 --2- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00ae680 0x7f4ea01535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.590+0000 7f4e9ffff700 1 -- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea00b1850 msgr2=0x7f4ea0141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:08.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.590+0000 7f4e9ffff700 1 --2- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea00b1850 0x7f4ea0141020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.590+0000 7f4e9ffff700 1 -- 172.21.15.55:0/3283645502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e90009420 con 0x7f4ea01579f0 2023-10-06T03:48:08.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.591+0000 7f4e9ffff700 1 --2- 172.21.15.55:0/3283645502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea01579f0 0x7f4ea015c7b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800b610 tx=0x7f4e98019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:08.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.591+0000 7f4e877fe700 1 -- 172.21.15.55:0/3283645502 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98012dd0 con 0x7f4ea01579f0 2023-10-06T03:48:08.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.591+0000 7f4e877fe700 1 -- 172.21.15.55:0/3283645502 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e98027470 con 0x7f4ea01579f0 2023-10-06T03:48:08.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.591+0000 7f4ea5fff700 1 -- 172.21.15.55:0/3283645502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e98009a20 con 0x7f4ea01579f0 2023-10-06T03:48:08.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.591+0000 7f4e877fe700 1 -- 172.21.15.55:0/3283645502 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e98027830 con 0x7f4ea01579f0 2023-10-06T03:48:08.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.591+0000 7f4ea5fff700 1 -- 172.21.15.55:0/3283645502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea0153f40 con 0x7f4ea01579f0 2023-10-06T03:48:08.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.592+0000 7f4ea5fff700 1 -- 172.21.15.55:0/3283645502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea004c3c0 con 0x7f4ea01579f0 2023-10-06T03:48:08.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.593+0000 7f4e877fe700 1 -- 172.21.15.55:0/3283645502 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 23) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f4e98024070 con 0x7f4ea01579f0 2023-10-06T03:48:08.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.594+0000 7f4e877fe700 1 --2- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4e8c067050 0x7f4e8c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:08.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.594+0000 7f4e877fe700 1 -- 172.21.15.55:0/3283645502 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6794+0+0 (secure 0 0 0) 0x7f4e98087560 con 0x7f4ea01579f0 2023-10-06T03:48:08.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.594+0000 7f4e9f7fe700 1 -- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4e8c067050 msgr2=0x7f4e8c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:08.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.594+0000 7f4e9f7fe700 1 --2- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4e8c067050 0x7f4e8c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:08.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.598+0000 7f4e877fe700 1 -- 172.21.15.55:0/3283645502 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e98054290 con 0x7f4ea01579f0 2023-10-06T03:48:08.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.795+0000 7f4e9f7fe700 1 -- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4e8c067050 msgr2=0x7f4e8c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:08.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.795+0000 7f4e9f7fe700 1 --2- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4e8c067050 0x7f4e8c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:08.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.847+0000 7f4ea5fff700 1 -- 172.21.15.55:0/3283645502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ea00cb8c0 con 0x7f4ea01579f0 2023-10-06T03:48:08.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.847+0000 7f4e877fe700 1 -- 172.21.15.55:0/3283645502 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f4e980578b0 con 0x7f4ea01579f0 2023-10-06T03:48:08.849 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:08.849 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3493866751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3493866751},{"type":"v1","addr":"172.21.15.163:6836","nonce":3493866751}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3246061279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3246061279},{"type":"v1","addr":"172.21.15.55:6835","nonce":3246061279}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1366768080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1366768080},{"type":"v1","addr":"172.21.15.55:6834","nonce":1366768080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:46:07.292050+0000","modified":"2023-10-06T03:46:10.815261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5862},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5862":{"gid":5862,"name":"b","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6833/2287859778","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2287859778},{"type":"v1","addr":"172.21.15.163:6833","nonce":2287859778}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T03:48:08.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.849+0000 7f4ea5fff700 1 -- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4e8c067050 msgr2=0x7f4e8c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:08.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.849+0000 7f4ea5fff700 1 --2- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4e8c067050 0x7f4e8c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.849+0000 7f4ea5fff700 1 -- 172.21.15.55:0/3283645502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea01579f0 msgr2=0x7f4ea015c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:08.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.849+0000 7f4ea5fff700 1 --2- 172.21.15.55:0/3283645502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea01579f0 0x7f4ea015c7b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800b610 tx=0x7f4e98019000 comp rx=0 tx=0).stop 2023-10-06T03:48:08.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.849+0000 7f4ea5fff700 1 -- 172.21.15.55:0/3283645502 shutdown_connections 2023-10-06T03:48:08.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.849+0000 7f4ea5fff700 1 --2- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4e8c067050 0x7f4e8c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.849+0000 7f4ea5fff700 1 --2- 172.21.15.55:0/3283645502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ea01579f0 0x7f4ea015c7b0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.850+0000 7f4ea5fff700 1 --2- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ea00ae680 0x7f4ea01535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.850+0000 7f4ea5fff700 1 --2- 172.21.15.55:0/3283645502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ea00b1850 0x7f4ea0141020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:08.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.850+0000 7f4ea5fff700 1 -- 172.21.15.55:0/3283645502 >> 172.21.15.55:0/3283645502 conn(0x7f4ea006a320 msgr2=0x7f4ea014efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:08.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.850+0000 7f4ea5fff700 1 -- 172.21.15.55:0/3283645502 shutdown_connections 2023-10-06T03:48:08.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:08.850+0000 7f4ea5fff700 1 -- 172.21.15.55:0/3283645502 wait complete. 2023-10-06T03:48:08.853 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 26 2023-10-06T03:48:08.870 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:48:08.873 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:08.871+0000 7fa553956180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:48:09.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.097+0000 7fc83759e700 1 Processor -- start 2023-10-06T03:48:09.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.097+0000 7fc83759e700 1 -- start start 2023-10-06T03:48:09.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.099+0000 7fc83759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc838159420 0x7fc8380b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:09.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.099+0000 7fc83759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8380b2980 con 0x7fc83815d230 2023-10-06T03:48:09.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.099+0000 7fc83759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8380b11d0 con 0x7fc8381597f0 2023-10-06T03:48:09.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.099+0000 7fc83759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8380b14b0 con 0x7fc838159420 2023-10-06T03:48:09.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.099+0000 7fc83659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc838159420 0x7fc8380b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:09.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.099+0000 7fc836d9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc83815d230 0x7fc83815d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43460/0 (socket says 172.21.15.55:43460) 2023-10-06T03:48:09.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.099+0000 7fc836d9d700 1 -- 172.21.15.55:0/1678924487 learned_addr learned my addr 172.21.15.55:0/1678924487 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:09.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.099+0000 7fc83659c700 1 -- 172.21.15.55:0/1678924487 >> v1:172.21.15.163:6789/0 conn(0x7fc8381597f0 legacy=0x7fc8380b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:09.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.099+0000 7fc83659c700 1 -- 172.21.15.55:0/1678924487 >> v1:172.21.15.55:6789/0 conn(0x7fc83815d230 legacy=0x7fc83815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:09.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.100+0000 7fc83659c700 1 -- 172.21.15.55:0/1678924487 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8380b1790 con 0x7fc838159420 2023-10-06T03:48:09.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.099+0000 7fc83559a700 1 -- 172.21.15.55:0/1678924487 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2858818271 0 0) 0x7fc8380b2980 con 0x7fc83815d230 2023-10-06T03:48:09.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.100+0000 7fc83659c700 1 --2- 172.21.15.55:0/1678924487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc838159420 0x7fc8380b5f40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc8280098e0 tx=0x7fc8280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32240c1af09e0789 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:09.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.100+0000 7fc83559a700 1 -- 172.21.15.55:0/1678924487 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828009e80 con 0x7fc838159420 2023-10-06T03:48:09.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.100+0000 7fc83559a700 1 -- 172.21.15.55:0/1678924487 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82800b860 con 0x7fc838159420 2023-10-06T03:48:09.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.100+0000 7fc83559a700 1 -- 172.21.15.55:0/1678924487 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82800bbe0 con 0x7fc838159420 2023-10-06T03:48:09.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.101+0000 7fc83759e700 1 -- 172.21.15.55:0/1678924487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc838159420 msgr2=0x7fc8380b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:09.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.101+0000 7fc83759e700 1 --2- 172.21.15.55:0/1678924487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc838159420 0x7fc8380b5f40 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc8280098e0 tx=0x7fc8280091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.101+0000 7fc83759e700 1 -- 172.21.15.55:0/1678924487 shutdown_connections 2023-10-06T03:48:09.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.101+0000 7fc83759e700 1 --2- 172.21.15.55:0/1678924487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc838159420 0x7fc8380b5f40 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.101+0000 7fc83759e700 1 -- 172.21.15.55:0/1678924487 >> 172.21.15.55:0/1678924487 conn(0x7fc8380d0660 msgr2=0x7fc8380d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:09.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.101+0000 7fc83759e700 1 -- 172.21.15.55:0/1678924487 shutdown_connections 2023-10-06T03:48:09.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.101+0000 7fc83759e700 1 -- 172.21.15.55:0/1678924487 wait complete. 2023-10-06T03:48:09.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.102+0000 7fc83759e700 1 Processor -- start 2023-10-06T03:48:09.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.102+0000 7fc83759e700 1 -- start start 2023-10-06T03:48:09.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.103+0000 7fc83759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc838159420 0x7fc8380c5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:09.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.103+0000 7fc83759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8381597f0 0x7fc838156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:09.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.103+0000 7fc83759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc83815d230 0x7fc838157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:09.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.103+0000 7fc83759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83815ad50 con 0x7fc8381597f0 2023-10-06T03:48:09.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.103+0000 7fc83759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83815a140 con 0x7fc83815d230 2023-10-06T03:48:09.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.103+0000 7fc83759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83815a420 con 0x7fc838159420 2023-10-06T03:48:09.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.104+0000 7fc835d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8381597f0 0x7fc838156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:09.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.104+0000 7fc835d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8381597f0 0x7fc838156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54440/0 (socket says 172.21.15.55:54440) 2023-10-06T03:48:09.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.104+0000 7fc835d9b700 1 -- 172.21.15.55:0/3498481364 learned_addr learned my addr 172.21.15.55:0/3498481364 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:09.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.104+0000 7fc83659c700 1 --2- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc838159420 0x7fc8380c5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:09.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.104+0000 7fc836d9d700 1 --2- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc83815d230 0x7fc838157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:09.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.104+0000 7fc835d9b700 1 -- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc838159420 msgr2=0x7fc8380c5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:09.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.104+0000 7fc835d9b700 1 --2- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc838159420 0x7fc8380c5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.104+0000 7fc835d9b700 1 -- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc83815d230 msgr2=0x7fc838157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:09.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.104+0000 7fc835d9b700 1 --2- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc83815d230 0x7fc838157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.104+0000 7fc835d9b700 1 -- 172.21.15.55:0/3498481364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc828009420 con 0x7fc8381597f0 2023-10-06T03:48:09.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.105+0000 7fc835d9b700 1 --2- 172.21.15.55:0/3498481364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8381597f0 0x7fc838156f30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00c7c0 tx=0x7fc82c007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:09.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.105+0000 7fc8267fc700 1 -- 172.21.15.55:0/3498481364 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c011590 con 0x7fc8381597f0 2023-10-06T03:48:09.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.105+0000 7fc83759e700 1 -- 172.21.15.55:0/3498481364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83815a700 con 0x7fc8381597f0 2023-10-06T03:48:09.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.105+0000 7fc8267fc700 1 -- 172.21.15.55:0/3498481364 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82c0116f0 con 0x7fc8381597f0 2023-10-06T03:48:09.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.105+0000 7fc8267fc700 1 -- 172.21.15.55:0/3498481364 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c011ab0 con 0x7fc8381597f0 2023-10-06T03:48:09.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.105+0000 7fc83759e700 1 -- 172.21.15.55:0/3498481364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8380c5a90 con 0x7fc8381597f0 2023-10-06T03:48:09.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.106+0000 7fc83759e700 1 -- 172.21.15.55:0/3498481364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83815b110 con 0x7fc8381597f0 2023-10-06T03:48:09.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.107+0000 7fc8267fc700 1 -- 172.21.15.55:0/3498481364 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 23) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fc82c01b3b0 con 0x7fc8381597f0 2023-10-06T03:48:09.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.108+0000 7fc8267fc700 1 --2- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc818067050 0x7fc818069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:09.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.108+0000 7fc8267fc700 1 -- 172.21.15.55:0/3498481364 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6794+0+0 (secure 0 0 0) 0x7fc82c083fe0 con 0x7fc8381597f0 2023-10-06T03:48:09.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.108+0000 7fc83659c700 1 -- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc818067050 msgr2=0x7fc818069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:09.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.108+0000 7fc83659c700 1 --2- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc818067050 0x7fc818069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:09.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.113+0000 7fc8267fc700 1 -- 172.21.15.55:0/3498481364 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc83815b110 con 0x7fc8381597f0 2023-10-06T03:48:09.161 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:09.159+0000 7f80ccb8c180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:48:09.161 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:09.159+0000 7fa553956180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:48:09.254 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:09.252+0000 7fa553956180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:48:09.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.309+0000 7fc83659c700 1 -- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc818067050 msgr2=0x7fc818069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:09.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.309+0000 7fc83659c700 1 --2- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc818067050 0x7fc818069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:09.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.323+0000 7fc83759e700 1 -- 172.21.15.55:0/3498481364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc83815b110 con 0x7fc8381597f0 2023-10-06T03:48:09.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.324+0000 7fc8267fc700 1 -- 172.21.15.55:0/3498481364 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18043 (secure 0 0 0) 0x7fc83815b110 con 0x7fc8381597f0 2023-10-06T03:48:09.326 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:09.326 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":68,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:48:07.857325+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:46:05.457264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-10-06T03:46:06.258505+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":68,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"8.1a","mappings":[{"from":0,"to":1}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3009002237":"2023-10-07T03:46:22.715958+0000","172.21.15.163:6837/85191":"2023-10-07T03:46:22.715958+0000","172.21.15.163:6835/85191":"2023-10-07T03:46:22.715958+0000","172.21.15.163:0/1689910889":"2023-10-07T03:46:22.715958+0000","172.21.15.163:0/2097952590":"2023-10-07T03:46:22.715958+0000","172.21.15.163:0/4134100032":"2023-10-07T03:46:22.715958+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:48:09.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.327+0000 7fc83759e700 1 -- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc818067050 msgr2=0x7fc818069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:09.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.327+0000 7fc83759e700 1 --2- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc818067050 0x7fc818069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.327+0000 7fc83759e700 1 -- 172.21.15.55:0/3498481364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8381597f0 msgr2=0x7fc838156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:09.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.327+0000 7fc83759e700 1 --2- 172.21.15.55:0/3498481364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8381597f0 0x7fc838156f30 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00c7c0 tx=0x7fc82c007f50 comp rx=0 tx=0).stop 2023-10-06T03:48:09.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.327+0000 7fc83759e700 1 -- 172.21.15.55:0/3498481364 shutdown_connections 2023-10-06T03:48:09.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.327+0000 7fc83759e700 1 --2- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc818067050 0x7fc818069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.327+0000 7fc83759e700 1 --2- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc83815d230 0x7fc838157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.327+0000 7fc83759e700 1 --2- 172.21.15.55:0/3498481364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc838159420 0x7fc8380c5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.327+0000 7fc83759e700 1 --2- 172.21.15.55:0/3498481364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8381597f0 0x7fc838156f30 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.327+0000 7fc83759e700 1 -- 172.21.15.55:0/3498481364 >> 172.21.15.55:0/3498481364 conn(0x7fc8380d0660 msgr2=0x7fc838158610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:09.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.327+0000 7fc83759e700 1 -- 172.21.15.55:0/3498481364 shutdown_connections 2023-10-06T03:48:09.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.327+0000 7fc83759e700 1 -- 172.21.15.55:0/3498481364 wait complete. 2023-10-06T03:48:09.347 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:48:09.409 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:09.407+0000 7fa553956180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:48:09.455 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:09.453+0000 7f80ccb8c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:48:09.514 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:09.512+0000 7fa553956180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:48:09.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.536+0000 7f0de2b9d700 1 Processor -- start 2023-10-06T03:48:09.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.537+0000 7f0de2b9d700 1 -- start start 2023-10-06T03:48:09.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.538+0000 7f0de2b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0ddc0c6c60 0x7f0ddc0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:09.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.538+0000 7f0de2b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ddc0b1410 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.538+0000 7f0de2b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ddc063230 con 0x7f0ddc1595a0 2023-10-06T03:48:09.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.538+0000 7f0de2b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ddc0b2b90 con 0x7f0ddc0c6c60 2023-10-06T03:48:09.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.538+0000 7f0de239c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0ddc06e8c0 0x7f0ddc159970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43462/0 (socket says 172.21.15.55:43462) 2023-10-06T03:48:09.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.538+0000 7f0de239c700 1 -- 172.21.15.55:0/2365372203 learned_addr learned my addr 172.21.15.55:0/2365372203 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:09.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.538+0000 7f0de139a700 1 --2- 172.21.15.55:0/2365372203 >> v2:172.21.15.55:3300/0 conn(0x7f0ddc0c6c60 0x7f0ddc0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:09.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.539+0000 7f0de0b99700 1 -- 172.21.15.55:0/2365372203 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136903503 0 0) 0x7f0ddc0b1410 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.539+0000 7f0de0b99700 1 -- 172.21.15.55:0/2365372203 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dc8003570 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.539+0000 7f0de0b99700 1 -- 172.21.15.55:0/2365372203 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023824611 0 0) 0x7f0ddc063230 con 0x7f0ddc1595a0 2023-10-06T03:48:09.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.539+0000 7f0de0b99700 1 -- 172.21.15.55:0/2365372203 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ddc0b1410 con 0x7f0ddc1595a0 2023-10-06T03:48:09.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.539+0000 7f0de139a700 1 -- 172.21.15.55:0/2365372203 >> v1:172.21.15.163:6789/0 conn(0x7f0ddc1595a0 legacy=0x7f0ddc0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:09.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.539+0000 7f0de139a700 1 -- 172.21.15.55:0/2365372203 >> v1:172.21.15.55:6789/0 conn(0x7f0ddc06e8c0 legacy=0x7f0ddc159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:09.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.539+0000 7f0de139a700 1 -- 172.21.15.55:0/2365372203 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc0634f0 con 0x7f0ddc0c6c60 2023-10-06T03:48:09.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.539+0000 7f0de139a700 1 --2- 172.21.15.55:0/2365372203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ddc0c6c60 0x7f0ddc0c8120 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0098e0 tx=0x7f0dcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636be3e2f2ceb6d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:09.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.540+0000 7f0de0b99700 1 -- 172.21.15.55:0/2365372203 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc011630 con 0x7f0ddc0c6c60 2023-10-06T03:48:09.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.540+0000 7f0de0b99700 1 -- 172.21.15.55:0/2365372203 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dcc011790 con 0x7f0ddc0c6c60 2023-10-06T03:48:09.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.540+0000 7f0de0b99700 1 -- 172.21.15.55:0/2365372203 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc011b10 con 0x7f0ddc0c6c60 2023-10-06T03:48:09.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.540+0000 7f0de2b9d700 1 -- 172.21.15.55:0/2365372203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ddc0c6c60 msgr2=0x7f0ddc0c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:09.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.540+0000 7f0de2b9d700 1 --2- 172.21.15.55:0/2365372203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ddc0c6c60 0x7f0ddc0c8120 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0098e0 tx=0x7f0dcc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.540+0000 7f0de2b9d700 1 -- 172.21.15.55:0/2365372203 shutdown_connections 2023-10-06T03:48:09.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.540+0000 7f0de2b9d700 1 --2- 172.21.15.55:0/2365372203 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ddc0c6c60 0x7f0ddc0c8120 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.540+0000 7f0de2b9d700 1 -- 172.21.15.55:0/2365372203 >> 172.21.15.55:0/2365372203 conn(0x7f0ddc06ddb0 msgr2=0x7f0ddc06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:09.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.540+0000 7f0de2b9d700 1 -- 172.21.15.55:0/2365372203 shutdown_connections 2023-10-06T03:48:09.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.540+0000 7f0de2b9d700 1 -- 172.21.15.55:0/2365372203 wait complete. 2023-10-06T03:48:09.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.543+0000 7f0de2b9d700 1 Processor -- start 2023-10-06T03:48:09.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.543+0000 7f0de2b9d700 1 -- start start 2023-10-06T03:48:09.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.544+0000 7f0de2b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ddc06e8c0 0x7f0ddc0c1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:09.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.544+0000 7f0de2b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ddc0c6c60 0x7f0ddc0c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:09.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.544+0000 7f0de2b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ddc1595a0 0x7f0ddc151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:09.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.545+0000 7f0de2b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0c28d0 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.545+0000 7f0de2b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0c2030 con 0x7f0ddc0c6c60 2023-10-06T03:48:09.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.545+0000 7f0de2b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0c2310 con 0x7f0ddc1595a0 2023-10-06T03:48:09.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.545+0000 7f0de1b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ddc06e8c0 0x7f0ddc0c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:09.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.545+0000 7f0de1b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ddc06e8c0 0x7f0ddc0c1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54464/0 (socket says 172.21.15.55:54464) 2023-10-06T03:48:09.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.545+0000 7f0de1b9b700 1 -- 172.21.15.55:0/531141426 learned_addr learned my addr 172.21.15.55:0/531141426 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:09.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.545+0000 7f0de139a700 1 --2- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ddc0c6c60 0x7f0ddc0c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:09.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.545+0000 7f0de239c700 1 --2- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ddc1595a0 0x7f0ddc151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:09.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.545+0000 7f0de1b9b700 1 -- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ddc1595a0 msgr2=0x7f0ddc151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:09.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.545+0000 7f0de1b9b700 1 --2- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ddc1595a0 0x7f0ddc151fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.545+0000 7f0de1b9b700 1 -- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ddc0c6c60 msgr2=0x7f0ddc0c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:09.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.545+0000 7f0de1b9b700 1 --2- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ddc0c6c60 0x7f0ddc0c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.545+0000 7f0de1b9b700 1 -- 172.21.15.55:0/531141426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc009420 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.545+0000 7f0de1b9b700 1 --2- 172.21.15.55:0/531141426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ddc06e8c0 0x7f0ddc0c1950 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800c590 tx=0x7f0dd8007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:09.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.546+0000 7f0dd1ffb700 1 -- 172.21.15.55:0/531141426 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8010510 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.546+0000 7f0dd1ffb700 1 -- 172.21.15.55:0/531141426 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd8010670 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.546+0000 7f0dd1ffb700 1 -- 172.21.15.55:0/531141426 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8010a30 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.546+0000 7f0de2b9d700 1 -- 172.21.15.55:0/531141426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ddc0c25f0 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.546+0000 7f0de2b9d700 1 -- 172.21.15.55:0/531141426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ddc152500 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.547+0000 7f0de2b9d700 1 -- 172.21.15.55:0/531141426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ddc04c3c0 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.548+0000 7f0dd1ffb700 1 -- 172.21.15.55:0/531141426 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 23) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f0dd801b400 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.549+0000 7f0dd1ffb700 1 --2- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0dc0067050 0x7f0dc0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:09.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.549+0000 7f0dd1ffb700 1 -- 172.21.15.55:0/531141426 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6794+0+0 (secure 0 0 0) 0x7f0dd80846f0 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.549+0000 7f0de139a700 1 -- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0dc0067050 msgr2=0x7f0dc0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:09.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.549+0000 7f0de139a700 1 --2- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0dc0067050 0x7f0dc0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:09.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.553+0000 7f0dd1ffb700 1 -- 172.21.15.55:0/531141426 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0dd8051370 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.558 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:09.556+0000 7f80ccb8c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:48:09.703 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:09.701+0000 7fa553956180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:48:09.722 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:09.720+0000 7f80ccb8c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:48:09.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.750+0000 7f0de139a700 1 -- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0dc0067050 msgr2=0x7f0dc0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:09.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.750+0000 7f0de139a700 1 --2- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0dc0067050 0x7f0dc0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:09.796 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:09.794+0000 7fa553956180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:48:09.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.803+0000 7f0de2b9d700 1 -- 172.21.15.55:0/531141426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0ddc0637d0 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.805 INFO:tasks.ceph.mon.a.smithi055.stderr:2023-10-06T03:48:09.804+0000 7fe524ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:48:09.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.810+0000 7f0dd1ffb700 1 -- 172.21.15.55:0/531141426 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0dd8054990 con 0x7f0ddc06e8c0 2023-10-06T03:48:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.813+0000 7f0de2b9d700 1 -- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0dc0067050 msgr2=0x7f0dc0069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.813+0000 7f0de2b9d700 1 --2- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0dc0067050 0x7f0dc0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.813+0000 7f0de2b9d700 1 -- 172.21.15.55:0/531141426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ddc06e8c0 msgr2=0x7f0ddc0c1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.813+0000 7f0de2b9d700 1 --2- 172.21.15.55:0/531141426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ddc06e8c0 0x7f0ddc0c1950 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800c590 tx=0x7f0dd8007dc0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.813+0000 7f0de2b9d700 1 -- 172.21.15.55:0/531141426 shutdown_connections 2023-10-06T03:48:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.813+0000 7f0de2b9d700 1 --2- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0dc0067050 0x7f0dc0069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.813+0000 7f0de2b9d700 1 --2- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ddc1595a0 0x7f0ddc151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.813+0000 7f0de2b9d700 1 --2- 172.21.15.55:0/531141426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ddc06e8c0 0x7f0ddc0c1950 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.813+0000 7f0de2b9d700 1 --2- 172.21.15.55:0/531141426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ddc0c6c60 0x7f0ddc0c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:09.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.813+0000 7f0de2b9d700 1 -- 172.21.15.55:0/531141426 >> 172.21.15.55:0/531141426 conn(0x7f0ddc06ddb0 msgr2=0x7f0ddc153ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:09.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.813+0000 7f0de2b9d700 1 -- 172.21.15.55:0/531141426 shutdown_connections 2023-10-06T03:48:09.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:09.813+0000 7f0de2b9d700 1 -- 172.21.15.55:0/531141426 wait complete. 2023-10-06T03:48:09.816 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:48:09.833 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:48:09.833 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:09.831+0000 7f80ccb8c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:48:09.869 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T03:48:09.897 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:09.895+0000 7fa553956180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:48:09.991 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:09.989+0000 7fa553956180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:48:10.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.031+0000 7f2304b9d700 1 Processor -- start 2023-10-06T03:48:10.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.031+0000 7f2304b9d700 1 -- start start 2023-10-06T03:48:10.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.033+0000 7f2304b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23000b16d0 0x7f23000b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:10.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.033+0000 7f2304b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f23000b33b0 0x7f23000b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:10.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.033+0000 7f2304b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23000b2080 con 0x7f23000b4e70 2023-10-06T03:48:10.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.033+0000 7f2304b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23000cfb00 con 0x7f23000b33b0 2023-10-06T03:48:10.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.033+0000 7f2304b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23000b5c40 con 0x7f23000b16d0 2023-10-06T03:48:10.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.033+0000 7f22fffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f23000b33b0 0x7f23000b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:10.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.033+0000 7f22feffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23000b16d0 0x7f23000b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:10.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.033+0000 7f22feffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23000b16d0 0x7f23000b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:41840/0 (socket says 172.21.15.55:41840) 2023-10-06T03:48:10.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.033+0000 7f22feffd700 1 -- 172.21.15.55:0/13141988 learned_addr learned my addr 172.21.15.55:0/13141988 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:10.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.034+0000 7f22feffd700 1 -- 172.21.15.55:0/13141988 >> v2:172.21.15.55:3300/0 conn(0x7f23000b33b0 msgr2=0x7f23000b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:10.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.034+0000 7f22feffd700 1 --2- 172.21.15.55:0/13141988 >> v2:172.21.15.55:3300/0 conn(0x7f23000b33b0 0x7f23000b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:10.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.034+0000 7f22feffd700 1 -- 172.21.15.55:0/13141988 >> v1:172.21.15.163:6789/0 conn(0x7f23000b4e70 legacy=0x7f23000ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:10.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.034+0000 7f22feffd700 1 -- 172.21.15.55:0/13141988 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23000b5f20 con 0x7f23000b16d0 2023-10-06T03:48:10.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.034+0000 7f22fffff700 1 --2- 172.21.15.55:0/13141988 >> v2:172.21.15.55:3300/0 conn(0x7f23000b33b0 0x7f23000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:10.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.034+0000 7f22fe7fc700 1 -- 172.21.15.55:0/13141988 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660026698 0 0) 0x7f23000b2080 con 0x7f23000b4e70 2023-10-06T03:48:10.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.034+0000 7f22feffd700 1 --2- 172.21.15.55:0/13141988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23000b16d0 0x7f23000b1b40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2300089830 tx=0x7f22f4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef5ec28eaef6f05c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:10.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.034+0000 7f22fe7fc700 1 -- 172.21.15.55:0/13141988 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f4009e70 con 0x7f23000b16d0 2023-10-06T03:48:10.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.034+0000 7f22fe7fc700 1 -- 172.21.15.55:0/13141988 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f400b860 con 0x7f23000b16d0 2023-10-06T03:48:10.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.034+0000 7f22fe7fc700 1 -- 172.21.15.55:0/13141988 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f400bc20 con 0x7f23000b16d0 2023-10-06T03:48:10.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.035+0000 7f2304b9d700 1 -- 172.21.15.55:0/13141988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23000b16d0 msgr2=0x7f23000b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:10.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.035+0000 7f2304b9d700 1 --2- 172.21.15.55:0/13141988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23000b16d0 0x7f23000b1b40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2300089830 tx=0x7f22f4009250 comp rx=0 tx=0).stop 2023-10-06T03:48:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.035+0000 7f2304b9d700 1 -- 172.21.15.55:0/13141988 shutdown_connections 2023-10-06T03:48:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.035+0000 7f2304b9d700 1 --2- 172.21.15.55:0/13141988 >> v2:172.21.15.55:3300/0 conn(0x7f23000b33b0 0x7f23000b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.035+0000 7f2304b9d700 1 --2- 172.21.15.55:0/13141988 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23000b16d0 0x7f23000b1b40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.035+0000 7f2304b9d700 1 -- 172.21.15.55:0/13141988 >> 172.21.15.55:0/13141988 conn(0x7f23000d10c0 msgr2=0x7f23000affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.035+0000 7f2304b9d700 1 -- 172.21.15.55:0/13141988 shutdown_connections 2023-10-06T03:48:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.035+0000 7f2304b9d700 1 -- 172.21.15.55:0/13141988 wait complete. 2023-10-06T03:48:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.036+0000 7f2304b9d700 1 Processor -- start 2023-10-06T03:48:10.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.037+0000 7f2304b9d700 1 -- start start 2023-10-06T03:48:10.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.037+0000 7f2304b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23000b16d0 0x7f23000cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:10.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.037+0000 7f2304b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23000b33b0 0x7f23000c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:10.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.037+0000 7f2304b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23000b4e70 0x7f2300169d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:10.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.037+0000 7f2304b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23000ca590 con 0x7f23000b16d0 2023-10-06T03:48:10.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.037+0000 7f2304b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23000cc5e0 con 0x7f23000b4e70 2023-10-06T03:48:10.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.037+0000 7f2304b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23000cc880 con 0x7f23000b33b0 2023-10-06T03:48:10.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.038+0000 7f22ff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23000b16d0 0x7f23000cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:10.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.038+0000 7f22ff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23000b16d0 0x7f23000cb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54492/0 (socket says 172.21.15.55:54492) 2023-10-06T03:48:10.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.038+0000 7f22ff7fe700 1 -- 172.21.15.55:0/3220125294 learned_addr learned my addr 172.21.15.55:0/3220125294 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:10.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.038+0000 7f22fffff700 1 --2- 172.21.15.55:0/3220125294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23000b4e70 0x7f2300169d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:10.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.038+0000 7f22feffd700 1 --2- 172.21.15.55:0/3220125294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23000b33b0 0x7f23000c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:10.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.038+0000 7f22ff7fe700 1 -- 172.21.15.55:0/3220125294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23000b33b0 msgr2=0x7f23000c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:10.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.038+0000 7f22ff7fe700 1 --2- 172.21.15.55:0/3220125294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23000b33b0 0x7f23000c9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:10.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.038+0000 7f22ff7fe700 1 -- 172.21.15.55:0/3220125294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23000b4e70 msgr2=0x7f2300169d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:10.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.038+0000 7f22ff7fe700 1 --2- 172.21.15.55:0/3220125294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23000b4e70 0x7f2300169d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:10.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.038+0000 7f22ff7fe700 1 -- 172.21.15.55:0/3220125294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f8009a20 con 0x7f23000b16d0 2023-10-06T03:48:10.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.039+0000 7f22ff7fe700 1 --2- 172.21.15.55:0/3220125294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23000b16d0 0x7f23000cb460 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f22f000c7e0 tx=0x7f22f0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:10.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.039+0000 7f22e77fe700 1 -- 172.21.15.55:0/3220125294 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0011590 con 0x7f23000b16d0 2023-10-06T03:48:10.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.039+0000 7f22e77fe700 1 -- 172.21.15.55:0/3220125294 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f0011710 con 0x7f23000b16d0 2023-10-06T03:48:10.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.039+0000 7f22e77fe700 1 -- 172.21.15.55:0/3220125294 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0011ad0 con 0x7f23000b16d0 2023-10-06T03:48:10.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.039+0000 7f2304b9d700 1 -- 172.21.15.55:0/3220125294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f40094f0 con 0x7f23000b16d0 2023-10-06T03:48:10.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.039+0000 7f2304b9d700 1 -- 172.21.15.55:0/3220125294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23000c7580 con 0x7f23000b16d0 2023-10-06T03:48:10.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.040+0000 7f2304b9d700 1 -- 172.21.15.55:0/3220125294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2300088990 con 0x7f23000b16d0 2023-10-06T03:48:10.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.042+0000 7f22e77fe700 1 -- 172.21.15.55:0/3220125294 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 24) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f22f001b3b0 con 0x7f23000b16d0 2023-10-06T03:48:10.046 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:10.042+0000 7f80ccb8c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:48:10.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.043+0000 7f22e77fe700 1 -- 172.21.15.55:0/3220125294 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7110+0+0 (secure 0 0 0) 0x7f22f0083b00 con 0x7f23000b16d0 2023-10-06T03:48:10.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.048+0000 7f22e77fe700 1 -- 172.21.15.55:0/3220125294 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22f0050bb0 con 0x7f23000b16d0 2023-10-06T03:48:10.072 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:10.069+0000 7fa4d250a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:10.072 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:10.069+0000 7fa4d250a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:10.072 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:10.069+0000 7fa4d250a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:10.072 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:10.069+0000 7fa4d250a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:10.072 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:10.069+0000 7fa4d250a700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:10.074 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:10.072+0000 7fa4cfd05700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:10.074 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:10.072+0000 7fa4cfd05700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:10.075 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:10.072+0000 7fa4cfd05700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:10.075 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:10.072+0000 7fa4cfd05700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:10.075 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:10.072+0000 7fa4cfd05700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:10.147 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:10.145+0000 7f80ccb8c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:48:10.243 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:10.241+0000 7f80ccb8c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:48:10.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.301+0000 7f2304b9d700 1 -- 172.21.15.55:0/3220125294 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2300069b20 con 0x7f23000b16d0 2023-10-06T03:48:10.341 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:10.339+0000 7f80ccb8c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:48:10.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.879+0000 7f22e77fe700 1 -- 172.21.15.55:0/3220125294 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f22f00170a0 con 0x7f23000b16d0 2023-10-06T03:48:10.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.882+0000 7f2304b9d700 1 -- 172.21.15.55:0/3220125294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23000b16d0 msgr2=0x7f23000cb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:10.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.882+0000 7f2304b9d700 1 --2- 172.21.15.55:0/3220125294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23000b16d0 0x7f23000cb460 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f22f000c7e0 tx=0x7f22f0007df0 comp rx=0 tx=0).stop 2023-10-06T03:48:10.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.882+0000 7f2304b9d700 1 -- 172.21.15.55:0/3220125294 shutdown_connections 2023-10-06T03:48:10.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.882+0000 7f2304b9d700 1 --2- 172.21.15.55:0/3220125294 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23000b4e70 0x7f2300169d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:10.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.882+0000 7f2304b9d700 1 --2- 172.21.15.55:0/3220125294 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23000b16d0 0x7f23000cb460 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:10.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.882+0000 7f2304b9d700 1 --2- 172.21.15.55:0/3220125294 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23000b33b0 0x7f23000c9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:10.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.882+0000 7f2304b9d700 1 -- 172.21.15.55:0/3220125294 >> 172.21.15.55:0/3220125294 conn(0x7f23000d10c0 msgr2=0x7f23000affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:10.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.882+0000 7f2304b9d700 1 -- 172.21.15.55:0/3220125294 shutdown_connections 2023-10-06T03:48:10.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:10.882+0000 7f2304b9d700 1 -- 172.21.15.55:0/3220125294 wait complete. 2023-10-06T03:48:10.903 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:48:11.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.099+0000 7f8e0759e700 1 Processor -- start 2023-10-06T03:48:11.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.099+0000 7f8e0759e700 1 -- start start 2023-10-06T03:48:11.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.100+0000 7f8e0759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e081597d0 0x7f8e080afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:11.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.100+0000 7f8e0759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8e0806f0b0 0x7f8e0815d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:11.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.101+0000 7f8e0759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e0806e510 con 0x7f8e080ac0e0 2023-10-06T03:48:11.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.101+0000 7f8e0759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e08069180 con 0x7f8e0806f0b0 2023-10-06T03:48:11.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.101+0000 7f8e0759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0806dec0 con 0x7f8e081597d0 2023-10-06T03:48:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.101+0000 7f8e06d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8e0806f0b0 0x7f8e0815d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.101+0000 7f8e0659c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e081597d0 0x7f8e080afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.101+0000 7f8e06d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8e0806f0b0 0x7f8e0815d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:41848/0 (socket says 172.21.15.55:41848) 2023-10-06T03:48:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.101+0000 7f8e06d9d700 1 -- 172.21.15.55:0/1558906486 learned_addr learned my addr 172.21.15.55:0/1558906486 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.101+0000 7f8e0659c700 1 -- 172.21.15.55:0/1558906486 >> v2:172.21.15.163:3300/0 conn(0x7f8e0806f0b0 msgr2=0x7f8e0815d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.101+0000 7f8e0659c700 1 --2- 172.21.15.55:0/1558906486 >> v2:172.21.15.163:3300/0 conn(0x7f8e0806f0b0 0x7f8e0815d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.101+0000 7f8e0659c700 1 -- 172.21.15.55:0/1558906486 >> v1:172.21.15.163:6789/0 conn(0x7f8e080ac0e0 legacy=0x7f8e0806e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.102+0000 7f8e0659c700 1 -- 172.21.15.55:0/1558906486 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0806e1a0 con 0x7f8e081597d0 2023-10-06T03:48:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.102+0000 7f8e06d9d700 1 --2- 172.21.15.55:0/1558906486 >> v2:172.21.15.163:3300/0 conn(0x7f8e0806f0b0 0x7f8e0815d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:11.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.102+0000 7f8e0659c700 1 --2- 172.21.15.55:0/1558906486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e081597d0 0x7f8e080afa40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8df80098e0 tx=0x7f8df80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0eb7c7204c30135 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:11.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.102+0000 7f8e0559a700 1 -- 172.21.15.55:0/1558906486 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df800b640 con 0x7f8e081597d0 2023-10-06T03:48:11.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.102+0000 7f8e0559a700 1 -- 172.21.15.55:0/1558906486 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df800b7a0 con 0x7f8e081597d0 2023-10-06T03:48:11.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.102+0000 7f8e0559a700 1 -- 172.21.15.55:0/1558906486 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df800bb40 con 0x7f8e081597d0 2023-10-06T03:48:11.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.103+0000 7f8e0759e700 1 -- 172.21.15.55:0/1558906486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e081597d0 msgr2=0x7f8e080afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:11.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.103+0000 7f8e0759e700 1 --2- 172.21.15.55:0/1558906486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e081597d0 0x7f8e080afa40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8df80098e0 tx=0x7f8df80091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:11.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.103+0000 7f8e0759e700 1 -- 172.21.15.55:0/1558906486 shutdown_connections 2023-10-06T03:48:11.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.103+0000 7f8e0759e700 1 --2- 172.21.15.55:0/1558906486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e081597d0 0x7f8e080afa40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:11.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.103+0000 7f8e0759e700 1 --2- 172.21.15.55:0/1558906486 >> v2:172.21.15.163:3300/0 conn(0x7f8e0806f0b0 0x7f8e0815d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:11.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.103+0000 7f8e0759e700 1 -- 172.21.15.55:0/1558906486 >> 172.21.15.55:0/1558906486 conn(0x7f8e0806adf0 msgr2=0x7f8e080ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:11.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.103+0000 7f8e0759e700 1 -- 172.21.15.55:0/1558906486 shutdown_connections 2023-10-06T03:48:11.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.103+0000 7f8e0759e700 1 -- 172.21.15.55:0/1558906486 wait complete. 2023-10-06T03:48:11.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.104+0000 7f8e0759e700 1 Processor -- start 2023-10-06T03:48:11.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.104+0000 7f8e0759e700 1 -- start start 2023-10-06T03:48:11.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.105+0000 7f8e0759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e0806f0b0 0x7f8e08147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:11.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.105+0000 7f8e0759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e081597d0 0x7f8e080afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:11.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.105+0000 7f8e0759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e0815df40 0x7f8e080b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:11.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.105+0000 7f8e0759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0815dab0 con 0x7f8e0815df40 2023-10-06T03:48:11.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.105+0000 7f8e0759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e08147b80 con 0x7f8e0806f0b0 2023-10-06T03:48:11.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.105+0000 7f8e0759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0815d1c0 con 0x7f8e081597d0 2023-10-06T03:48:11.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.106+0000 7f8e06d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e0815df40 0x7f8e080b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:11.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.106+0000 7f8e06d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e0815df40 0x7f8e080b00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54498/0 (socket says 172.21.15.55:54498) 2023-10-06T03:48:11.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.106+0000 7f8e0659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e0806f0b0 0x7f8e08147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:11.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.106+0000 7f8e06d9d700 1 -- 172.21.15.55:0/2774285210 learned_addr learned my addr 172.21.15.55:0/2774285210 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:11.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.106+0000 7f8e05d9b700 1 --2- 172.21.15.55:0/2774285210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e081597d0 0x7f8e080afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:11.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.106+0000 7f8e06d9d700 1 -- 172.21.15.55:0/2774285210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e081597d0 msgr2=0x7f8e080afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:11.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.106+0000 7f8e06d9d700 1 --2- 172.21.15.55:0/2774285210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e081597d0 0x7f8e080afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:11.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.106+0000 7f8e06d9d700 1 -- 172.21.15.55:0/2774285210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e0806f0b0 msgr2=0x7f8e08147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:11.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.106+0000 7f8e06d9d700 1 --2- 172.21.15.55:0/2774285210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e0806f0b0 0x7f8e08147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:11.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.106+0000 7f8e06d9d700 1 -- 172.21.15.55:0/2774285210 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e00009ae0 con 0x7f8e0815df40 2023-10-06T03:48:11.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.106+0000 7f8e0659c700 1 --2- 172.21.15.55:0/2774285210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e0806f0b0 0x7f8e08147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:11.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.106+0000 7f8e06d9d700 1 --2- 172.21.15.55:0/2774285210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e0815df40 0x7f8e080b00a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009fd0 tx=0x7f8e0000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:11.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.107+0000 7f8df67fc700 1 -- 172.21.15.55:0/2774285210 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00018070 con 0x7f8e0815df40 2023-10-06T03:48:11.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.107+0000 7f8df67fc700 1 -- 172.21.15.55:0/2774285210 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0000f040 con 0x7f8e0815df40 2023-10-06T03:48:11.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.107+0000 7f8df67fc700 1 -- 172.21.15.55:0/2774285210 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0000cbd0 con 0x7f8e0815df40 2023-10-06T03:48:11.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.107+0000 7f8e0759e700 1 -- 172.21.15.55:0/2774285210 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df8009420 con 0x7f8e0815df40 2023-10-06T03:48:11.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.107+0000 7f8e0759e700 1 -- 172.21.15.55:0/2774285210 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0815d740 con 0x7f8e0815df40 2023-10-06T03:48:11.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.108+0000 7f8e0759e700 1 -- 172.21.15.55:0/2774285210 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0815dc20 con 0x7f8e0815df40 2023-10-06T03:48:11.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.109+0000 7f8df67fc700 1 -- 172.21.15.55:0/2774285210 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 25) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f8e00003020 con 0x7f8e0815df40 2023-10-06T03:48:11.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.110+0000 7f8df67fc700 1 --2- 172.21.15.55:0/2774285210 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8de8066d50 0x7f8de8069200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:11.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.110+0000 7f8df67fc700 1 -- 172.21.15.55:0/2774285210 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7110+0+0 (secure 0 0 0) 0x7f8e00084c80 con 0x7f8e0815df40 2023-10-06T03:48:11.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.110+0000 7f8e0659c700 1 --2- 172.21.15.55:0/2774285210 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8de8066d50 0x7f8de8069200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:11.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.111+0000 7f8e0659c700 1 --2- 172.21.15.55:0/2774285210 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8de8066d50 0x7f8de8069200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8df80093f0 tx=0x7f8df8017000 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:11.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.114+0000 7f8df67fc700 1 -- 172.21.15.55:0/2774285210 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8e00051b50 con 0x7f8e0815df40 2023-10-06T03:48:11.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:11.315+0000 7f8e0759e700 1 -- 172.21.15.55:0/2774285210 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8e0804c3c0 con 0x7f8e0815df40 2023-10-06T03:48:12.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.014+0000 7f8df67fc700 1 -- 172.21.15.55:0/2774285210 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v71) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8e00055170 con 0x7f8e0815df40 2023-10-06T03:48:12.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.018+0000 7f8e0759e700 1 -- 172.21.15.55:0/2774285210 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8de8066d50 msgr2=0x7f8de8069200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:12.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.018+0000 7f8e0759e700 1 --2- 172.21.15.55:0/2774285210 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8de8066d50 0x7f8de8069200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8df80093f0 tx=0x7f8df8017000 comp rx=0 tx=0).stop 2023-10-06T03:48:12.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.018+0000 7f8e0759e700 1 -- 172.21.15.55:0/2774285210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e0815df40 msgr2=0x7f8e080b00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:12.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.018+0000 7f8e0759e700 1 --2- 172.21.15.55:0/2774285210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e0815df40 0x7f8e080b00a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009fd0 tx=0x7f8e0000b040 comp rx=0 tx=0).stop 2023-10-06T03:48:12.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.018+0000 7f8e0759e700 1 -- 172.21.15.55:0/2774285210 shutdown_connections 2023-10-06T03:48:12.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.018+0000 7f8e0759e700 1 --2- 172.21.15.55:0/2774285210 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8de8066d50 0x7f8de8069200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:12.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.018+0000 7f8e0759e700 1 --2- 172.21.15.55:0/2774285210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8e0815df40 0x7f8e080b00a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:12.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.018+0000 7f8e0759e700 1 --2- 172.21.15.55:0/2774285210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8e0806f0b0 0x7f8e08147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:12.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.018+0000 7f8e0759e700 1 --2- 172.21.15.55:0/2774285210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8e081597d0 0x7f8e080afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:12.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.018+0000 7f8e0759e700 1 -- 172.21.15.55:0/2774285210 >> 172.21.15.55:0/2774285210 conn(0x7f8e0806adf0 msgr2=0x7f8e081588a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:12.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.018+0000 7f8e0759e700 1 -- 172.21.15.55:0/2774285210 shutdown_connections 2023-10-06T03:48:12.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.019+0000 7f8e0759e700 1 -- 172.21.15.55:0/2774285210 wait complete. 2023-10-06T03:48:12.022 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:48:12.041 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:48:12.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.243+0000 7f9afcf9d700 1 Processor -- start 2023-10-06T03:48:12.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.243+0000 7f9afcf9d700 1 -- start start 2023-10-06T03:48:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.244+0000 7f9afcf9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9af8159450 0x7f9af815d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.244+0000 7f9afcf9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9af80ce850 0x7f9af815d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.244+0000 7f9afcf9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af80cfa80 con 0x7f9af8159820 2023-10-06T03:48:12.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.244+0000 7f9afcf9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af80b3670 con 0x7f9af8159450 2023-10-06T03:48:12.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.244+0000 7f9afcf9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af80cee70 con 0x7f9af80ce850 2023-10-06T03:48:12.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.245+0000 7f9af6ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9af8159820 0x7f9af80ce140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43478/0 (socket says 172.21.15.55:43478) 2023-10-06T03:48:12.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.245+0000 7f9af6ffd700 1 -- 172.21.15.55:0/3032211977 learned_addr learned my addr 172.21.15.55:0/3032211977 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:12.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.245+0000 7f9af77fe700 1 --2- 172.21.15.55:0/3032211977 >> v2:172.21.15.55:3300/0 conn(0x7f9af8159450 0x7f9af815d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:12.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.245+0000 7f9af7fff700 1 --2- 172.21.15.55:0/3032211977 >> v2:172.21.15.163:3300/0 conn(0x7f9af80ce850 0x7f9af815d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:12.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.245+0000 7f9af67fc700 1 -- 172.21.15.55:0/3032211977 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549910484 0 0) 0x7f9af80cfa80 con 0x7f9af8159820 2023-10-06T03:48:12.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.245+0000 7f9af67fc700 1 -- 172.21.15.55:0/3032211977 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae0003570 con 0x7f9af8159820 2023-10-06T03:48:12.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.245+0000 7f9af77fe700 1 -- 172.21.15.55:0/3032211977 >> v2:172.21.15.163:3300/0 conn(0x7f9af80ce850 msgr2=0x7f9af815d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:12.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.245+0000 7f9af77fe700 1 --2- 172.21.15.55:0/3032211977 >> v2:172.21.15.163:3300/0 conn(0x7f9af80ce850 0x7f9af815d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:12.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.245+0000 7f9af77fe700 1 -- 172.21.15.55:0/3032211977 >> v1:172.21.15.55:6789/0 conn(0x7f9af8159820 legacy=0x7f9af80ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:12.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.245+0000 7f9af77fe700 1 -- 172.21.15.55:0/3032211977 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af80cf150 con 0x7f9af8159450 2023-10-06T03:48:12.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.245+0000 7f9af77fe700 1 --2- 172.21.15.55:0/3032211977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9af8159450 0x7f9af815d240 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9af80b3090 tx=0x7f9ae80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455ec3f08baa02c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:12.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.246+0000 7f9af67fc700 1 -- 172.21.15.55:0/3032211977 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8011630 con 0x7f9af8159450 2023-10-06T03:48:12.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.246+0000 7f9af67fc700 1 -- 172.21.15.55:0/3032211977 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae80117b0 con 0x7f9af8159450 2023-10-06T03:48:12.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.246+0000 7f9af67fc700 1 -- 172.21.15.55:0/3032211977 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8011b70 con 0x7f9af8159450 2023-10-06T03:48:12.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.246+0000 7f9afcf9d700 1 -- 172.21.15.55:0/3032211977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9af8159450 msgr2=0x7f9af815d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:12.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.246+0000 7f9afcf9d700 1 --2- 172.21.15.55:0/3032211977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9af8159450 0x7f9af815d240 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9af80b3090 tx=0x7f9ae80091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:12.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.246+0000 7f9afcf9d700 1 -- 172.21.15.55:0/3032211977 shutdown_connections 2023-10-06T03:48:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.246+0000 7f9afcf9d700 1 --2- 172.21.15.55:0/3032211977 >> v2:172.21.15.163:3300/0 conn(0x7f9af80ce850 0x7f9af815d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.246+0000 7f9afcf9d700 1 --2- 172.21.15.55:0/3032211977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9af8159450 0x7f9af815d240 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.246+0000 7f9afcf9d700 1 -- 172.21.15.55:0/3032211977 >> 172.21.15.55:0/3032211977 conn(0x7f9af80d0570 msgr2=0x7f9af80d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.246+0000 7f9afcf9d700 1 -- 172.21.15.55:0/3032211977 shutdown_connections 2023-10-06T03:48:12.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.246+0000 7f9afcf9d700 1 -- 172.21.15.55:0/3032211977 wait complete. 2023-10-06T03:48:12.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.248+0000 7f9afcf9d700 1 Processor -- start 2023-10-06T03:48:12.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.248+0000 7f9afcf9d700 1 -- start start 2023-10-06T03:48:12.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9afcf9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9af80ce850 0x7f9af816c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:12.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9afcf9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9af8159450 0x7f9af80c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:12.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9afcf9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9af8159820 0x7f9af80c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:12.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9afcf9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af816aad0 con 0x7f9af8159450 2023-10-06T03:48:12.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9afcf9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8169ec0 con 0x7f9af80ce850 2023-10-06T03:48:12.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9afcf9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af816a1a0 con 0x7f9af8159820 2023-10-06T03:48:12.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9af6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9af8159450 0x7f9af80c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:12.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9af6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9af8159450 0x7f9af80c7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54522/0 (socket says 172.21.15.55:54522) 2023-10-06T03:48:12.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9af6ffd700 1 -- 172.21.15.55:0/521566732 learned_addr learned my addr 172.21.15.55:0/521566732 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:12.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9af77fe700 1 --2- 172.21.15.55:0/521566732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9af80ce850 0x7f9af816c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:12.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9af7fff700 1 --2- 172.21.15.55:0/521566732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9af8159820 0x7f9af80c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:12.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9af6ffd700 1 -- 172.21.15.55:0/521566732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9af8159820 msgr2=0x7f9af80c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:12.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9af6ffd700 1 --2- 172.21.15.55:0/521566732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9af8159820 0x7f9af80c7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:12.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9af6ffd700 1 -- 172.21.15.55:0/521566732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9af80ce850 msgr2=0x7f9af816c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:12.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9af6ffd700 1 --2- 172.21.15.55:0/521566732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9af80ce850 0x7f9af816c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:12.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.249+0000 7f9af6ffd700 1 -- 172.21.15.55:0/521566732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae8009420 con 0x7f9af8159450 2023-10-06T03:48:12.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.250+0000 7f9af6ffd700 1 --2- 172.21.15.55:0/521566732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9af8159450 0x7f9af80c7350 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00c7c0 tx=0x7f9aec007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:12.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.250+0000 7f9adf7fe700 1 -- 172.21.15.55:0/521566732 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec005b30 con 0x7f9af8159450 2023-10-06T03:48:12.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.250+0000 7f9adf7fe700 1 -- 172.21.15.55:0/521566732 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec005c90 con 0x7f9af8159450 2023-10-06T03:48:12.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.250+0000 7f9adf7fe700 1 -- 172.21.15.55:0/521566732 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec007340 con 0x7f9af8159450 2023-10-06T03:48:12.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.250+0000 7f9afcf9d700 1 -- 172.21.15.55:0/521566732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af816a480 con 0x7f9af8159450 2023-10-06T03:48:12.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.250+0000 7f9afcf9d700 1 -- 172.21.15.55:0/521566732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af8169b70 con 0x7f9af8159450 2023-10-06T03:48:12.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.251+0000 7f9afcf9d700 1 -- 172.21.15.55:0/521566732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af80888b0 con 0x7f9af8159450 2023-10-06T03:48:12.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.252+0000 7f9adf7fe700 1 -- 172.21.15.55:0/521566732 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 25) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f9aec01b400 con 0x7f9af8159450 2023-10-06T03:48:12.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.253+0000 7f9adf7fe700 1 --2- 172.21.15.55:0/521566732 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9ae4066d80 0x7f9ae4069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:12.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.253+0000 7f9adf7fe700 1 -- 172.21.15.55:0/521566732 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6672+0+0 (secure 0 0 0) 0x7f9aec0841a0 con 0x7f9af8159450 2023-10-06T03:48:12.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.253+0000 7f9af77fe700 1 --2- 172.21.15.55:0/521566732 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9ae4066d80 0x7f9ae4069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:12.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.254+0000 7f9af77fe700 1 --2- 172.21.15.55:0/521566732 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9ae4066d80 0x7f9ae4069230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9af80b3090 tx=0x7f9ae8017000 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:12.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.257+0000 7f9adf7fe700 1 -- 172.21.15.55:0/521566732 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9aec0512d0 con 0x7f9af8159450 2023-10-06T03:48:12.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.461+0000 7f9afcf9d700 1 -- 172.21.15.55:0/521566732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9af804bb80 con 0x7f9af8159450 2023-10-06T03:48:12.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:12.884+0000 7f9adf7fe700 1 -- 172.21.15.55:0/521566732 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9aec0516a0 con 0x7f9af8159450 2023-10-06T03:48:13.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.015+0000 7f9adf7fe700 1 -- 172.21.15.55:0/521566732 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v72) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9aec054d60 con 0x7f9af8159450 2023-10-06T03:48:13.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.020+0000 7f9afcf9d700 1 -- 172.21.15.55:0/521566732 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9ae4066d80 msgr2=0x7f9ae4069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.020+0000 7f9afcf9d700 1 --2- 172.21.15.55:0/521566732 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9ae4066d80 0x7f9ae4069230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9af80b3090 tx=0x7f9ae8017000 comp rx=0 tx=0).stop 2023-10-06T03:48:13.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.020+0000 7f9afcf9d700 1 -- 172.21.15.55:0/521566732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9af8159450 msgr2=0x7f9af80c7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.020+0000 7f9afcf9d700 1 --2- 172.21.15.55:0/521566732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9af8159450 0x7f9af80c7350 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00c7c0 tx=0x7f9aec007e00 comp rx=0 tx=0).stop 2023-10-06T03:48:13.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.020+0000 7f9afcf9d700 1 -- 172.21.15.55:0/521566732 shutdown_connections 2023-10-06T03:48:13.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.020+0000 7f9afcf9d700 1 --2- 172.21.15.55:0/521566732 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9ae4066d80 0x7f9ae4069230 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.020+0000 7f9afcf9d700 1 --2- 172.21.15.55:0/521566732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9af8159820 0x7f9af80c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.020+0000 7f9afcf9d700 1 --2- 172.21.15.55:0/521566732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9af80ce850 0x7f9af816c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.020+0000 7f9afcf9d700 1 --2- 172.21.15.55:0/521566732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9af8159450 0x7f9af80c7350 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.020+0000 7f9afcf9d700 1 -- 172.21.15.55:0/521566732 >> 172.21.15.55:0/521566732 conn(0x7f9af80d0570 msgr2=0x7f9af80b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:13.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.020+0000 7f9afcf9d700 1 -- 172.21.15.55:0/521566732 shutdown_connections 2023-10-06T03:48:13.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.020+0000 7f9afcf9d700 1 -- 172.21.15.55:0/521566732 wait complete. 2023-10-06T03:48:13.023 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2023-10-06T03:48:13.043 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:48:13.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.232+0000 7f6451b9d700 1 Processor -- start 2023-10-06T03:48:13.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.232+0000 7f6451b9d700 1 -- start start 2023-10-06T03:48:13.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.234+0000 7f6451b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f644c159a60 0x7f644c0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.234+0000 7f6451b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644c0c9cb0 con 0x7f644c0cb230 2023-10-06T03:48:13.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.234+0000 7f6451b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644c0c9f30 con 0x7f644c0ccc00 2023-10-06T03:48:13.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.234+0000 7f6451b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c06dfc0 con 0x7f644c159a60 2023-10-06T03:48:13.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.234+0000 7f645139c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f644c0cb230 0x7f644c06d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45844/0 (socket says 172.21.15.55:45844) 2023-10-06T03:48:13.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.234+0000 7f645139c700 1 -- 172.21.15.55:0/1347827539 learned_addr learned my addr 172.21.15.55:0/1347827539 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:13.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.234+0000 7f6450b9b700 1 --2- 172.21.15.55:0/1347827539 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f644c159a60 0x7f644c0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.234+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041166384 0 0) 0x7f644c0c9cb0 con 0x7f644c0cb230 2023-10-06T03:48:13.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.234+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f643c003570 con 0x7f644c0cb230 2023-10-06T03:48:13.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110179531 0 0) 0x7f644c0c9f30 con 0x7f644c0ccc00 2023-10-06T03:48:13.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f644c0c9cb0 con 0x7f644c0ccc00 2023-10-06T03:48:13.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 977503309 0 0) 0x7f643c003570 con 0x7f644c0cb230 2023-10-06T03:48:13.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f644c159a60 msgr2=0x7f644c0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:13.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 --2- 172.21.15.55:0/1347827539 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f644c159a60 0x7f644c0c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 >> v1:172.21.15.163:6789/0 conn(0x7f644c0ccc00 legacy=0x7f644c069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c0ca470 con 0x7f644c0cb230 2023-10-06T03:48:13.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f64440017f0 con 0x7f644c0cb230 2023-10-06T03:48:13.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 >> v1:172.21.15.55:6789/0 conn(0x7f644c0cb230 legacy=0x7f644c06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 --2- 172.21.15.55:0/1347827539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c003c20 0x7f643c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 --2- 172.21.15.55:0/1347827539 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f644c159a60 0x7f643c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 --2- 172.21.15.55:0/1347827539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f643c004d50 0x7f643c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c06dfc0 con 0x7f643c003c20 2023-10-06T03:48:13.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c00bbf0 con 0x7f643c004d50 2023-10-06T03:48:13.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c00bd60 con 0x7f644c159a60 2023-10-06T03:48:13.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f6450b9b700 1 --2- 172.21.15.55:0/1347827539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c003c20 0x7f643c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f644bfff700 1 --2- 172.21.15.55:0/1347827539 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f644c159a60 0x7f643c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.235+0000 7f645139c700 1 --2- 172.21.15.55:0/1347827539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f643c004d50 0x7f643c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.236+0000 7f6450b9b700 1 -- 172.21.15.55:0/1347827539 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f644c159a60 msgr2=0x7f643c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.236+0000 7f6450b9b700 1 --2- 172.21.15.55:0/1347827539 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f644c159a60 0x7f643c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.236+0000 7f6450b9b700 1 -- 172.21.15.55:0/1347827539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f643c004d50 msgr2=0x7f643c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.236+0000 7f6450b9b700 1 --2- 172.21.15.55:0/1347827539 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f643c004d50 0x7f643c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.236+0000 7f6450b9b700 1 -- 172.21.15.55:0/1347827539 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c003570 con 0x7f643c003c20 2023-10-06T03:48:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.236+0000 7f644bfff700 1 --2- 172.21.15.55:0/1347827539 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f644c159a60 0x7f643c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.236+0000 7f6450b9b700 1 --2- 172.21.15.55:0/1347827539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c003c20 0x7f643c004010 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6440006f00 tx=0x7f64400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ade3b5cbcd6ea4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.236+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644000e710 con 0x7f643c003c20 2023-10-06T03:48:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.236+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644000e870 con 0x7f643c003c20 2023-10-06T03:48:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.236+0000 7f644b7fe700 1 -- 172.21.15.55:0/1347827539 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644000ec30 con 0x7f643c003c20 2023-10-06T03:48:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.237+0000 7f6451b9d700 1 -- 172.21.15.55:0/1347827539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c003c20 msgr2=0x7f643c004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.237+0000 7f6451b9d700 1 --2- 172.21.15.55:0/1347827539 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c003c20 0x7f643c004010 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6440006f00 tx=0x7f64400091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.237+0000 7f645139c700 1 -- 172.21.15.55:0/1347827539 reap_dead start 2023-10-06T03:48:13.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.237+0000 7f6451b9d700 1 -- 172.21.15.55:0/1347827539 shutdown_connections 2023-10-06T03:48:13.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.237+0000 7f6451b9d700 1 -- 172.21.15.55:0/1347827539 >> 172.21.15.55:0/1347827539 conn(0x7f644c06baa0 msgr2=0x7f644c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:13.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.237+0000 7f6451b9d700 1 -- 172.21.15.55:0/1347827539 shutdown_connections 2023-10-06T03:48:13.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.237+0000 7f6451b9d700 1 -- 172.21.15.55:0/1347827539 wait complete. 2023-10-06T03:48:13.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.238+0000 7f6451b9d700 1 Processor -- start 2023-10-06T03:48:13.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.238+0000 7f6451b9d700 1 -- start start 2023-10-06T03:48:13.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.239+0000 7f6451b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c003c20 0x7f644c169c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.239+0000 7f6451b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f644c0a98d0 0x7f644c0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.239+0000 7f6451b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f644c0ceaf0 0x7f644c0cef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.239+0000 7f6451b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c0cf4d0 con 0x7f643c003c20 2023-10-06T03:48:13.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.239+0000 7f6451b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c16aec0 con 0x7f644c0ceaf0 2023-10-06T03:48:13.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.239+0000 7f6451b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c0b4240 con 0x7f644c0a98d0 2023-10-06T03:48:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.239+0000 7f6450b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c003c20 0x7f644c169c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.240+0000 7f6450b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c003c20 0x7f644c169c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40854/0 (socket says 172.21.15.55:40854) 2023-10-06T03:48:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.240+0000 7f6450b9b700 1 -- 172.21.15.55:0/2820463133 learned_addr learned my addr 172.21.15.55:0/2820463133 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.240+0000 7f645139c700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f644c0ceaf0 0x7f644c0cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.240+0000 7f644bfff700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f644c0a98d0 0x7f644c0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.240+0000 7f6450b9b700 1 -- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f644c0a98d0 msgr2=0x7f644c0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.240+0000 7f6450b9b700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f644c0a98d0 0x7f644c0b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.240+0000 7f6450b9b700 1 -- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f644c0ceaf0 msgr2=0x7f644c0cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.240+0000 7f6450b9b700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f644c0ceaf0 0x7f644c0cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.240+0000 7f6450b9b700 1 -- 172.21.15.55:0/2820463133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c16a240 con 0x7f643c003c20 2023-10-06T03:48:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.240+0000 7f645139c700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f644c0ceaf0 0x7f644c0cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:13.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.240+0000 7f6450b9b700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c003c20 0x7f644c169c10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f644000e4b0 tx=0x7f644001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:13.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.240+0000 7f6448ff9700 1 -- 172.21.15.55:0/2820463133 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6440022070 con 0x7f643c003c20 2023-10-06T03:48:13.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.240+0000 7f6448ff9700 1 -- 172.21.15.55:0/2820463133 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644000e9e0 con 0x7f643c003c20 2023-10-06T03:48:13.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.240+0000 7f6451b9d700 1 -- 172.21.15.55:0/2820463133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644c16a520 con 0x7f643c003c20 2023-10-06T03:48:13.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.241+0000 7f6448ff9700 1 -- 172.21.15.55:0/2820463133 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644001c450 con 0x7f643c003c20 2023-10-06T03:48:13.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.241+0000 7f6451b9d700 1 -- 172.21.15.55:0/2820463133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644c16aae0 con 0x7f643c003c20 2023-10-06T03:48:13.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.242+0000 7f6451b9d700 1 -- 172.21.15.55:0/2820463133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644c06eba0 con 0x7f643c003c20 2023-10-06T03:48:13.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.242+0000 7f6448ff9700 1 -- 172.21.15.55:0/2820463133 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f644001c840 con 0x7f643c003c20 2023-10-06T03:48:13.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.243+0000 7f6448ff9700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6430066f30 0x7f64300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.243+0000 7f6448ff9700 1 -- 172.21.15.55:0/2820463133 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f6440086000 con 0x7f643c003c20 2023-10-06T03:48:13.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.244+0000 7f644bfff700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6430066f30 0x7f64300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.244+0000 7f644bfff700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6430066f30 0x7f64300693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6438000c00 tx=0x7f6438009040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:13.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.248+0000 7f6448ff9700 1 -- 172.21.15.55:0/2820463133 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6440053080 con 0x7f643c003c20 2023-10-06T03:48:13.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.448+0000 7f6451b9d700 1 -- 172.21.15.55:0/2820463133 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f644c0ca750 con 0x7f643c003c20 2023-10-06T03:48:13.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.453+0000 7f6448ff9700 1 -- 172.21.15.55:0/2820463133 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f64400566a0 con 0x7f643c003c20 2023-10-06T03:48:13.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.455+0000 7f6451b9d700 1 -- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6430066f30 msgr2=0x7f64300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.455+0000 7f6451b9d700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6430066f30 0x7f64300693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6438000c00 tx=0x7f6438009040 comp rx=0 tx=0).stop 2023-10-06T03:48:13.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.455+0000 7f6451b9d700 1 -- 172.21.15.55:0/2820463133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c003c20 msgr2=0x7f644c169c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.455+0000 7f6451b9d700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c003c20 0x7f644c169c10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f644000e4b0 tx=0x7f644001a000 comp rx=0 tx=0).stop 2023-10-06T03:48:13.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.455+0000 7f6451b9d700 1 -- 172.21.15.55:0/2820463133 shutdown_connections 2023-10-06T03:48:13.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.455+0000 7f6451b9d700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6430066f30 0x7f64300693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.455+0000 7f6451b9d700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f644c0ceaf0 0x7f644c0cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.455+0000 7f6451b9d700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f643c003c20 0x7f644c169c10 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.455+0000 7f6451b9d700 1 --2- 172.21.15.55:0/2820463133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f644c0a98d0 0x7f644c0b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.455+0000 7f6451b9d700 1 -- 172.21.15.55:0/2820463133 >> 172.21.15.55:0/2820463133 conn(0x7f644c06baa0 msgr2=0x7f644c1583a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:13.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.456+0000 7f6451b9d700 1 -- 172.21.15.55:0/2820463133 shutdown_connections 2023-10-06T03:48:13.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.456+0000 7f6451b9d700 1 -- 172.21.15.55:0/2820463133 wait complete. 2023-10-06T03:48:13.474 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T03:48:13.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.663+0000 7fda1219d700 1 Processor -- start 2023-10-06T03:48:13.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.663+0000 7fda1219d700 1 -- start start 2023-10-06T03:48:13.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.664+0000 7fda1219d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c15d990 0x7fda0c164620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.664+0000 7fda1219d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0c150b50 con 0x7fda0c0cc600 2023-10-06T03:48:13.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.665+0000 7fda1219d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda0c0cede0 con 0x7fda0c0cc9d0 2023-10-06T03:48:13.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.665+0000 7fda1219d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c0b2260 con 0x7fda0c15d990 2023-10-06T03:48:13.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.665+0000 7fda1119b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fda0c0cc600 0x7fda0c0b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45858/0 (socket says 172.21.15.55:45858) 2023-10-06T03:48:13.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.665+0000 7fda1119b700 1 -- 172.21.15.55:0/3471642386 learned_addr learned my addr 172.21.15.55:0/3471642386 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:13.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.665+0000 7fda1199c700 1 --2- 172.21.15.55:0/3471642386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c15d990 0x7fda0c164620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.665+0000 7fda03fff700 1 -- 172.21.15.55:0/3471642386 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091177870 0 0) 0x7fda0c150b50 con 0x7fda0c0cc600 2023-10-06T03:48:13.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.665+0000 7fda03fff700 1 -- 172.21.15.55:0/3471642386 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9f8003570 con 0x7fda0c0cc600 2023-10-06T03:48:13.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.665+0000 7fda03fff700 1 -- 172.21.15.55:0/3471642386 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2961015361 0 0) 0x7fd9f8003570 con 0x7fda0c0cc600 2023-10-06T03:48:13.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.665+0000 7fda03fff700 1 -- 172.21.15.55:0/3471642386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c15d990 msgr2=0x7fda0c164620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:13.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.665+0000 7fda03fff700 1 --2- 172.21.15.55:0/3471642386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c15d990 0x7fda0c164620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda03fff700 1 -- 172.21.15.55:0/3471642386 >> v1:172.21.15.163:6789/0 conn(0x7fda0c0cc9d0 legacy=0x7fda0c15d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda03fff700 1 -- 172.21.15.55:0/3471642386 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c0b2540 con 0x7fda0c0cc600 2023-10-06T03:48:13.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda03fff700 1 -- 172.21.15.55:0/3471642386 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fda08001470 con 0x7fda0c0cc600 2023-10-06T03:48:13.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda03fff700 1 -- 172.21.15.55:0/3471642386 >> v1:172.21.15.55:6789/0 conn(0x7fda0c0cc600 legacy=0x7fda0c0b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda03fff700 1 --2- 172.21.15.55:0/3471642386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c15d990 0x7fd9f8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda03fff700 1 --2- 172.21.15.55:0/3471642386 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f8004060 0x7fd9f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda03fff700 1 --2- 172.21.15.55:0/3471642386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9f8004d50 0x7fd9f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda03fff700 1 -- 172.21.15.55:0/3471642386 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c0b2260 con 0x7fd9f8004d50 2023-10-06T03:48:13.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda03fff700 1 -- 172.21.15.55:0/3471642386 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f800bc20 con 0x7fd9f8004060 2023-10-06T03:48:13.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda03fff700 1 -- 172.21.15.55:0/3471642386 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f800bde0 con 0x7fda0c15d990 2023-10-06T03:48:13.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda1199c700 1 --2- 172.21.15.55:0/3471642386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9f8004d50 0x7fd9f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda1119b700 1 --2- 172.21.15.55:0/3471642386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c15d990 0x7fd9f8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda1099a700 1 --2- 172.21.15.55:0/3471642386 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f8004060 0x7fd9f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda1199c700 1 -- 172.21.15.55:0/3471642386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c15d990 msgr2=0x7fd9f8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:13.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda1199c700 1 --2- 172.21.15.55:0/3471642386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c15d990 0x7fd9f8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda1199c700 1 -- 172.21.15.55:0/3471642386 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f8004060 msgr2=0x7fd9f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda1199c700 1 --2- 172.21.15.55:0/3471642386 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9f8004060 0x7fd9f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.666+0000 7fda1199c700 1 -- 172.21.15.55:0/3471642386 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f8003570 con 0x7fd9f8004d50 2023-10-06T03:48:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.667+0000 7fda1199c700 1 --2- 172.21.15.55:0/3471642386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9f8004d50 0x7fd9f80051c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0b2b20 tx=0x7fda0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf1776883ea5f64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.667+0000 7fda03fff700 1 -- 172.21.15.55:0/3471642386 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0400ea60 con 0x7fd9f8004d50 2023-10-06T03:48:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.667+0000 7fda03fff700 1 -- 172.21.15.55:0/3471642386 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0400f040 con 0x7fd9f8004d50 2023-10-06T03:48:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.667+0000 7fda03fff700 1 -- 172.21.15.55:0/3471642386 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0400ee10 con 0x7fd9f8004d50 2023-10-06T03:48:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.667+0000 7fda1219d700 1 -- 172.21.15.55:0/3471642386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9f8004d50 msgr2=0x7fd9f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.667+0000 7fda1219d700 1 --2- 172.21.15.55:0/3471642386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9f8004d50 0x7fd9f80051c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0b2b20 tx=0x7fda0400b040 comp rx=0 tx=0).stop 2023-10-06T03:48:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.667+0000 7fda1199c700 1 -- 172.21.15.55:0/3471642386 reap_dead start 2023-10-06T03:48:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.668+0000 7fda1219d700 1 -- 172.21.15.55:0/3471642386 shutdown_connections 2023-10-06T03:48:13.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.668+0000 7fda1219d700 1 -- 172.21.15.55:0/3471642386 >> 172.21.15.55:0/3471642386 conn(0x7fda0c0d0630 msgr2=0x7fda0c0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:13.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.668+0000 7fda1219d700 1 -- 172.21.15.55:0/3471642386 shutdown_connections 2023-10-06T03:48:13.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.668+0000 7fda1219d700 1 -- 172.21.15.55:0/3471642386 wait complete. 2023-10-06T03:48:13.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.669+0000 7fda1219d700 1 Processor -- start 2023-10-06T03:48:13.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.669+0000 7fda1219d700 1 -- start start 2023-10-06T03:48:13.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.670+0000 7fda1219d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9f8004d50 0x7fda0c156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.670+0000 7fda1219d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c156090 0x7fda0c152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.670+0000 7fda1219d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0c1587d0 0x7fda0c158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.670+0000 7fda1219d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c151fc0 con 0x7fd9f8004d50 2023-10-06T03:48:13.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.670+0000 7fda1219d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c1513b0 con 0x7fda0c1587d0 2023-10-06T03:48:13.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.670+0000 7fda1219d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c151690 con 0x7fda0c156090 2023-10-06T03:48:13.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.670+0000 7fda1119b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9f8004d50 0x7fda0c156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.670+0000 7fda1119b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9f8004d50 0x7fda0c156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40866/0 (socket says 172.21.15.55:40866) 2023-10-06T03:48:13.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.670+0000 7fda1119b700 1 -- 172.21.15.55:0/709156343 learned_addr learned my addr 172.21.15.55:0/709156343 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:13.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.670+0000 7fda1199c700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0c1587d0 0x7fda0c158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.670+0000 7fda1099a700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c156090 0x7fda0c152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.671+0000 7fda1119b700 1 -- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c156090 msgr2=0x7fda0c152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.671+0000 7fda1119b700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c156090 0x7fda0c152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.671+0000 7fda1119b700 1 -- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0c1587d0 msgr2=0x7fda0c158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.671+0000 7fda1119b700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0c1587d0 0x7fda0c158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.671+0000 7fda1119b700 1 -- 172.21.15.55:0/709156343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c151970 con 0x7fd9f8004d50 2023-10-06T03:48:13.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.671+0000 7fda1199c700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0c1587d0 0x7fda0c158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:13.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.671+0000 7fda1119b700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9f8004d50 0x7fda0c156f10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fda08007390 tx=0x7fda0c15ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:13.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.671+0000 7fda017fa700 1 -- 172.21.15.55:0/709156343 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08018030 con 0x7fd9f8004d50 2023-10-06T03:48:13.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.671+0000 7fda017fa700 1 -- 172.21.15.55:0/709156343 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0800f8d0 con 0x7fd9f8004d50 2023-10-06T03:48:13.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.671+0000 7fda017fa700 1 -- 172.21.15.55:0/709156343 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0800fc70 con 0x7fd9f8004d50 2023-10-06T03:48:13.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.671+0000 7fda1219d700 1 -- 172.21.15.55:0/709156343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0c151c50 con 0x7fd9f8004d50 2023-10-06T03:48:13.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.671+0000 7fda1219d700 1 -- 172.21.15.55:0/709156343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0c150ff0 con 0x7fd9f8004d50 2023-10-06T03:48:13.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.672+0000 7fda1219d700 1 -- 172.21.15.55:0/709156343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0c0889d0 con 0x7fd9f8004d50 2023-10-06T03:48:13.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.673+0000 7fda017fa700 1 -- 172.21.15.55:0/709156343 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fda0801b070 con 0x7fd9f8004d50 2023-10-06T03:48:13.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.674+0000 7fda017fa700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9f0066f30 0x7fd9f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:13.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.674+0000 7fda017fa700 1 -- 172.21.15.55:0/709156343 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6200+0+0 (secure 0 0 0) 0x7fda08019070 con 0x7fd9f8004d50 2023-10-06T03:48:13.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.675+0000 7fda1099a700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9f0066f30 0x7fd9f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:13.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.675+0000 7fda1099a700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9f0066f30 0x7fd9f00693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fda0c155d00 tx=0x7fd9fc007d30 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:13.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.679+0000 7fda017fa700 1 -- 172.21.15.55:0/709156343 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda08051780 con 0x7fd9f8004d50 2023-10-06T03:48:13.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.891+0000 7fda1219d700 1 -- 172.21.15.55:0/709156343 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fda0c0b2820 con 0x7fd9f8004d50 2023-10-06T03:48:13.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.896+0000 7fda017fa700 1 -- 172.21.15.55:0/709156343 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7fda08054da0 con 0x7fd9f8004d50 2023-10-06T03:48:13.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.898+0000 7fda1219d700 1 -- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9f0066f30 msgr2=0x7fd9f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.898+0000 7fda1219d700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9f0066f30 0x7fd9f00693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fda0c155d00 tx=0x7fd9fc007d30 comp rx=0 tx=0).stop 2023-10-06T03:48:13.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.898+0000 7fda1219d700 1 -- 172.21.15.55:0/709156343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9f8004d50 msgr2=0x7fda0c156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:13.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.898+0000 7fda1219d700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9f8004d50 0x7fda0c156f10 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fda08007390 tx=0x7fda0c15ac90 comp rx=0 tx=0).stop 2023-10-06T03:48:13.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.898+0000 7fda1219d700 1 -- 172.21.15.55:0/709156343 shutdown_connections 2023-10-06T03:48:13.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.899+0000 7fda1219d700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9f0066f30 0x7fd9f00693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.899+0000 7fda1219d700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0c1587d0 0x7fda0c158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.899+0000 7fda1219d700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9f8004d50 0x7fda0c156f10 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.899+0000 7fda1219d700 1 --2- 172.21.15.55:0/709156343 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda0c156090 0x7fda0c152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:13.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.899+0000 7fda1219d700 1 -- 172.21.15.55:0/709156343 >> 172.21.15.55:0/709156343 conn(0x7fda0c0d0630 msgr2=0x7fda0c15b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:13.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.899+0000 7fda1219d700 1 -- 172.21.15.55:0/709156343 shutdown_connections 2023-10-06T03:48:13.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:13.899+0000 7fda1219d700 1 -- 172.21.15.55:0/709156343 wait complete. 2023-10-06T03:48:13.917 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T03:48:14.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.109+0000 7f96d2b9d700 1 Processor -- start 2023-10-06T03:48:14.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.109+0000 7f96d2b9d700 1 -- start start 2023-10-06T03:48:14.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.110+0000 7f96d2b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f96cc06bfa0 0x7f96cc068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:14.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.110+0000 7f96d2b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f96cc0688d0 0x7f96cc0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:14.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.110+0000 7f96d2b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96cc068d30 con 0x7f96cc159a70 2023-10-06T03:48:14.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.110+0000 7f96d2b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96cc0ca920 con 0x7f96cc0688d0 2023-10-06T03:48:14.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.110+0000 7f96d2b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96cc06af00 con 0x7f96cc06bfa0 2023-10-06T03:48:14.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.110+0000 7f96d239c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f96cc0688d0 0x7f96cc0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:14.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.110+0000 7f96d239c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f96cc0688d0 0x7f96cc0caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40874/0 (socket says 172.21.15.55:40874) 2023-10-06T03:48:14.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.110+0000 7f96d239c700 1 -- 172.21.15.55:0/2146553871 learned_addr learned my addr 172.21.15.55:0/2146553871 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:14.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.111+0000 7f96d139a700 1 --2- 172.21.15.55:0/2146553871 >> v2:172.21.15.163:3300/0 conn(0x7f96cc06bfa0 0x7f96cc068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:14.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.111+0000 7f96d239c700 1 -- 172.21.15.55:0/2146553871 >> v2:172.21.15.163:3300/0 conn(0x7f96cc06bfa0 msgr2=0x7f96cc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:14.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.111+0000 7f96d239c700 1 --2- 172.21.15.55:0/2146553871 >> v2:172.21.15.163:3300/0 conn(0x7f96cc06bfa0 0x7f96cc068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.111+0000 7f96d239c700 1 -- 172.21.15.55:0/2146553871 >> v1:172.21.15.163:6789/0 conn(0x7f96cc159a70 legacy=0x7f96cc0c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:14.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.111+0000 7f96d239c700 1 -- 172.21.15.55:0/2146553871 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc06b1e0 con 0x7f96cc0688d0 2023-10-06T03:48:14.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.111+0000 7f96d239c700 1 --2- 172.21.15.55:0/2146553871 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96cc0688d0 0x7f96cc0caec0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f96cc064080 tx=0x7f96c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18322f22aca0e397 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:14.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.111+0000 7f96d0b99700 1 -- 172.21.15.55:0/2146553871 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c400e070 con 0x7f96cc0688d0 2023-10-06T03:48:14.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.111+0000 7f96d0b99700 1 -- 172.21.15.55:0/2146553871 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c400ba50 con 0x7f96cc0688d0 2023-10-06T03:48:14.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.111+0000 7f96d0b99700 1 -- 172.21.15.55:0/2146553871 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c40092e0 con 0x7f96cc0688d0 2023-10-06T03:48:14.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.112+0000 7f96d2b9d700 1 -- 172.21.15.55:0/2146553871 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96cc0688d0 msgr2=0x7f96cc0caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:14.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.112+0000 7f96d2b9d700 1 --2- 172.21.15.55:0/2146553871 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96cc0688d0 0x7f96cc0caec0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f96cc064080 tx=0x7f96c4009840 comp rx=0 tx=0).stop 2023-10-06T03:48:14.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.112+0000 7f96d2b9d700 1 -- 172.21.15.55:0/2146553871 shutdown_connections 2023-10-06T03:48:14.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.112+0000 7f96d2b9d700 1 --2- 172.21.15.55:0/2146553871 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96cc0688d0 0x7f96cc0caec0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.112+0000 7f96d2b9d700 1 --2- 172.21.15.55:0/2146553871 >> v2:172.21.15.163:3300/0 conn(0x7f96cc06bfa0 0x7f96cc068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.112+0000 7f96d2b9d700 1 -- 172.21.15.55:0/2146553871 >> 172.21.15.55:0/2146553871 conn(0x7f96cc0cddc0 msgr2=0x7f96cc0c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:14.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.112+0000 7f96d2b9d700 1 -- 172.21.15.55:0/2146553871 shutdown_connections 2023-10-06T03:48:14.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.112+0000 7f96d2b9d700 1 -- 172.21.15.55:0/2146553871 wait complete. 2023-10-06T03:48:14.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.114+0000 7f96d2b9d700 1 Processor -- start 2023-10-06T03:48:14.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.114+0000 7f96d2b9d700 1 -- start start 2023-10-06T03:48:14.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d2b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96cc0688d0 0x7f96cc153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:14.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d2b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96cc06bfa0 0x7f96cc14dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:14.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d2b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96cc159a70 0x7f96cc155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:14.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d2b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc14e900 con 0x7f96cc0688d0 2023-10-06T03:48:14.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d2b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc154310 con 0x7f96cc159a70 2023-10-06T03:48:14.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d2b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96cc1545f0 con 0x7f96cc06bfa0 2023-10-06T03:48:14.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d1b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96cc0688d0 0x7f96cc153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:14.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d1b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96cc0688d0 0x7f96cc153ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40880/0 (socket says 172.21.15.55:40880) 2023-10-06T03:48:14.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d1b9b700 1 -- 172.21.15.55:0/3854175131 learned_addr learned my addr 172.21.15.55:0/3854175131 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:14.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d139a700 1 --2- 172.21.15.55:0/3854175131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96cc06bfa0 0x7f96cc14dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:14.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d239c700 1 --2- 172.21.15.55:0/3854175131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96cc159a70 0x7f96cc155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:14.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d1b9b700 1 -- 172.21.15.55:0/3854175131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96cc06bfa0 msgr2=0x7f96cc14dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:14.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d1b9b700 1 --2- 172.21.15.55:0/3854175131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96cc06bfa0 0x7f96cc14dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d1b9b700 1 -- 172.21.15.55:0/3854175131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96cc159a70 msgr2=0x7f96cc155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d1b9b700 1 --2- 172.21.15.55:0/3854175131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96cc159a70 0x7f96cc155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.115+0000 7f96d1b9b700 1 -- 172.21.15.55:0/3854175131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c4009ae0 con 0x7f96cc0688d0 2023-10-06T03:48:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.116+0000 7f96d1b9b700 1 --2- 172.21.15.55:0/3854175131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96cc0688d0 0x7f96cc153ae0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f96c800c4e0 tx=0x7f96c800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.116+0000 7f96c1ffb700 1 -- 172.21.15.55:0/3854175131 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8011590 con 0x7f96cc0688d0 2023-10-06T03:48:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.116+0000 7f96c1ffb700 1 -- 172.21.15.55:0/3854175131 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c8011710 con 0x7f96cc0688d0 2023-10-06T03:48:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.116+0000 7f96d2b9d700 1 -- 172.21.15.55:0/3854175131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96cc1548d0 con 0x7f96cc0688d0 2023-10-06T03:48:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.116+0000 7f96c1ffb700 1 -- 172.21.15.55:0/3854175131 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c8011ad0 con 0x7f96cc0688d0 2023-10-06T03:48:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.116+0000 7f96d2b9d700 1 -- 172.21.15.55:0/3854175131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96cc154020 con 0x7f96cc0688d0 2023-10-06T03:48:14.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.117+0000 7f96d2b9d700 1 -- 172.21.15.55:0/3854175131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96cc0cd0e0 con 0x7f96cc0688d0 2023-10-06T03:48:14.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.118+0000 7f96c1ffb700 1 -- 172.21.15.55:0/3854175131 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f96c801b3b0 con 0x7f96cc0688d0 2023-10-06T03:48:14.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.119+0000 7f96c1ffb700 1 --2- 172.21.15.55:0/3854175131 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f96b0066f30 0x7f96b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:14.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.119+0000 7f96c1ffb700 1 -- 172.21.15.55:0/3854175131 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f96c80839e0 con 0x7f96cc0688d0 2023-10-06T03:48:14.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.119+0000 7f96d139a700 1 --2- 172.21.15.55:0/3854175131 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f96b0066f30 0x7f96b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:14.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.120+0000 7f96d139a700 1 --2- 172.21.15.55:0/3854175131 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f96b0066f30 0x7f96b00693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f96cc06f6c0 tx=0x7f96bc008040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:14.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.123+0000 7f96c1ffb700 1 -- 172.21.15.55:0/3854175131 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f96c80509c0 con 0x7f96cc0688d0 2023-10-06T03:48:14.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.323+0000 7f96d2b9d700 1 -- 172.21.15.55:0/3854175131 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f96cc068d30 con 0x7f96cc0688d0 2023-10-06T03:48:14.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.327+0000 7f96c1ffb700 1 -- 172.21.15.55:0/3854175131 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v21) v1 ==== 120+0+0 (secure 0 0 0) 0x7f96cc068d30 con 0x7f96cc0688d0 2023-10-06T03:48:14.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.329+0000 7f96d2b9d700 1 -- 172.21.15.55:0/3854175131 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f96b0066f30 msgr2=0x7f96b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:14.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.329+0000 7f96d2b9d700 1 --2- 172.21.15.55:0/3854175131 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f96b0066f30 0x7f96b00693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f96cc06f6c0 tx=0x7f96bc008040 comp rx=0 tx=0).stop 2023-10-06T03:48:14.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.329+0000 7f96d2b9d700 1 -- 172.21.15.55:0/3854175131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96cc0688d0 msgr2=0x7f96cc153ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:14.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.329+0000 7f96d2b9d700 1 --2- 172.21.15.55:0/3854175131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96cc0688d0 0x7f96cc153ae0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f96c800c4e0 tx=0x7f96c800c7f0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.329+0000 7f96d2b9d700 1 -- 172.21.15.55:0/3854175131 shutdown_connections 2023-10-06T03:48:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.329+0000 7f96d2b9d700 1 --2- 172.21.15.55:0/3854175131 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f96b0066f30 0x7f96b00693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.329+0000 7f96d2b9d700 1 --2- 172.21.15.55:0/3854175131 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96cc159a70 0x7f96cc155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.329+0000 7f96d2b9d700 1 --2- 172.21.15.55:0/3854175131 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f96cc0688d0 0x7f96cc153ae0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.329+0000 7f96d2b9d700 1 --2- 172.21.15.55:0/3854175131 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96cc06bfa0 0x7f96cc14dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.330+0000 7f96d2b9d700 1 -- 172.21.15.55:0/3854175131 >> 172.21.15.55:0/3854175131 conn(0x7f96cc0cddc0 msgr2=0x7f96cc06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.330+0000 7f96d2b9d700 1 -- 172.21.15.55:0/3854175131 shutdown_connections 2023-10-06T03:48:14.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.330+0000 7f96d2b9d700 1 -- 172.21.15.55:0/3854175131 wait complete. 2023-10-06T03:48:14.349 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-06T03:48:14.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.534+0000 7f958ffff700 1 Processor -- start 2023-10-06T03:48:14.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.534+0000 7f958ffff700 1 -- start start 2023-10-06T03:48:14.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.536+0000 7f958ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f95901566e0 0x7f95900ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:14.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.536+0000 7f958ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95900cb900 0x7f95900ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:14.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.536+0000 7f958ffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95900ca860 con 0x7f9590156310 2023-10-06T03:48:14.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.536+0000 7f958ffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95900cab40 con 0x7f95901566e0 2023-10-06T03:48:14.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.536+0000 7f958ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95900cae20 con 0x7f95900cb900 2023-10-06T03:48:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.536+0000 7f958f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95900cb900 0x7f95900ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.536+0000 7f958e7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f95901566e0 0x7f95900ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.536+0000 7f958f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95900cb900 0x7f95900ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:43696/0 (socket says 172.21.15.55:43696) 2023-10-06T03:48:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.536+0000 7f958f7fe700 1 -- 172.21.15.55:0/1820881440 learned_addr learned my addr 172.21.15.55:0/1820881440 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.537+0000 7f958dffb700 1 -- 172.21.15.55:0/1820881440 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009048168 0 0) 0x7f95900ca860 con 0x7f9590156310 2023-10-06T03:48:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.537+0000 7f958dffb700 1 -- 172.21.15.55:0/1820881440 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9578003570 con 0x7f9590156310 2023-10-06T03:48:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.537+0000 7f958f7fe700 1 -- 172.21.15.55:0/1820881440 >> v2:172.21.15.163:3300/0 conn(0x7f95901566e0 msgr2=0x7f95900ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.537+0000 7f958f7fe700 1 --2- 172.21.15.55:0/1820881440 >> v2:172.21.15.163:3300/0 conn(0x7f95901566e0 0x7f95900ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.537+0000 7f958f7fe700 1 -- 172.21.15.55:0/1820881440 >> v1:172.21.15.163:6789/0 conn(0x7f9590156310 legacy=0x7f95900c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.537+0000 7f958f7fe700 1 -- 172.21.15.55:0/1820881440 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95900cb100 con 0x7f95900cb900 2023-10-06T03:48:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.537+0000 7f958f7fe700 1 --2- 172.21.15.55:0/1820881440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95900cb900 0x7f95900ae540 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9588009f20 tx=0x7f9588009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47397cea018b294c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.537+0000 7f958dffb700 1 -- 172.21.15.55:0/1820881440 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958800e070 con 0x7f95900cb900 2023-10-06T03:48:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.537+0000 7f958dffb700 1 -- 172.21.15.55:0/1820881440 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958800b970 con 0x7f95900cb900 2023-10-06T03:48:14.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.537+0000 7f958dffb700 1 -- 172.21.15.55:0/1820881440 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95880092e0 con 0x7f95900cb900 2023-10-06T03:48:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.538+0000 7f958ffff700 1 -- 172.21.15.55:0/1820881440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95900cb900 msgr2=0x7f95900ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.538+0000 7f958ffff700 1 --2- 172.21.15.55:0/1820881440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95900cb900 0x7f95900ae540 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9588009f20 tx=0x7f9588009840 comp rx=0 tx=0).stop 2023-10-06T03:48:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.538+0000 7f958ffff700 1 -- 172.21.15.55:0/1820881440 shutdown_connections 2023-10-06T03:48:14.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.538+0000 7f958ffff700 1 --2- 172.21.15.55:0/1820881440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95900cb900 0x7f95900ae540 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.538+0000 7f958ffff700 1 --2- 172.21.15.55:0/1820881440 >> v2:172.21.15.163:3300/0 conn(0x7f95901566e0 0x7f95900ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.538+0000 7f958ffff700 1 -- 172.21.15.55:0/1820881440 >> 172.21.15.55:0/1820881440 conn(0x7f95900cc1f0 msgr2=0x7f95900cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:14.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.538+0000 7f958ffff700 1 -- 172.21.15.55:0/1820881440 shutdown_connections 2023-10-06T03:48:14.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.538+0000 7f958ffff700 1 -- 172.21.15.55:0/1820881440 wait complete. 2023-10-06T03:48:14.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.539+0000 7f958ffff700 1 Processor -- start 2023-10-06T03:48:14.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.540+0000 7f958ffff700 1 -- start start 2023-10-06T03:48:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.540+0000 7f958ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95900cb900 0x7f959014edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.540+0000 7f958ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9590156310 0x7f9590152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.540+0000 7f958ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95901566e0 0x7f95901698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.541+0000 7f958ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959016a8a0 con 0x7f95900cb900 2023-10-06T03:48:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.541+0000 7f958ffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959016a2e0 con 0x7f9590156310 2023-10-06T03:48:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.541+0000 7f958ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959016a5c0 con 0x7f95901566e0 2023-10-06T03:48:14.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.541+0000 7f958effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95900cb900 0x7f959014edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.541+0000 7f958e7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9590156310 0x7f9590152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.541+0000 7f958effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95900cb900 0x7f959014edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40892/0 (socket says 172.21.15.55:40892) 2023-10-06T03:48:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.541+0000 7f958effd700 1 -- 172.21.15.55:0/3990574858 learned_addr learned my addr 172.21.15.55:0/3990574858 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.541+0000 7f958f7fe700 1 --2- 172.21.15.55:0/3990574858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95901566e0 0x7f95901698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.541+0000 7f958effd700 1 -- 172.21.15.55:0/3990574858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95901566e0 msgr2=0x7f95901698a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.541+0000 7f958effd700 1 --2- 172.21.15.55:0/3990574858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95901566e0 0x7f95901698a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.541+0000 7f958effd700 1 -- 172.21.15.55:0/3990574858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9590156310 msgr2=0x7f9590152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.541+0000 7f958effd700 1 --2- 172.21.15.55:0/3990574858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9590156310 0x7f9590152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.541+0000 7f958effd700 1 -- 172.21.15.55:0/3990574858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9588009ae0 con 0x7f95900cb900 2023-10-06T03:48:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.541+0000 7f958e7fc700 1 --2- 172.21.15.55:0/3990574858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9590156310 0x7f9590152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.542+0000 7f958effd700 1 --2- 172.21.15.55:0/3990574858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95900cb900 0x7f959014edb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f958000c790 tx=0x7f9580007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.542+0000 7f9576ffd700 1 -- 172.21.15.55:0/3990574858 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580010510 con 0x7f95900cb900 2023-10-06T03:48:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.542+0000 7f9576ffd700 1 -- 172.21.15.55:0/3990574858 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9580010670 con 0x7f95900cb900 2023-10-06T03:48:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.542+0000 7f9576ffd700 1 -- 172.21.15.55:0/3990574858 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580010a10 con 0x7f95900cb900 2023-10-06T03:48:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.542+0000 7f958ffff700 1 -- 172.21.15.55:0/3990574858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9590152740 con 0x7f95900cb900 2023-10-06T03:48:14.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.542+0000 7f958ffff700 1 -- 172.21.15.55:0/3990574858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959014f2f0 con 0x7f95900cb900 2023-10-06T03:48:14.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.543+0000 7f958ffff700 1 -- 172.21.15.55:0/3990574858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95901525a0 con 0x7f95900cb900 2023-10-06T03:48:14.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.544+0000 7f9576ffd700 1 -- 172.21.15.55:0/3990574858 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f958001b3b0 con 0x7f95900cb900 2023-10-06T03:48:14.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.545+0000 7f9576ffd700 1 --2- 172.21.15.55:0/3990574858 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f957c066f30 0x7f957c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:14.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.545+0000 7f9576ffd700 1 -- 172.21.15.55:0/3990574858 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f9580084060 con 0x7f95900cb900 2023-10-06T03:48:14.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.546+0000 7f958e7fc700 1 --2- 172.21.15.55:0/3990574858 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f957c066f30 0x7f957c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:14.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.546+0000 7f958e7fc700 1 --2- 172.21.15.55:0/3990574858 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f957c066f30 0x7f957c0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f959015b420 tx=0x7f95840091a0 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:14.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.549+0000 7f9576ffd700 1 -- 172.21.15.55:0/3990574858 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95901525a0 con 0x7f95900cb900 2023-10-06T03:48:14.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.829+0000 7f958ffff700 1 -- 172.21.15.55:0/3990574858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f95901525a0 con 0x7f95900cb900 2023-10-06T03:48:14.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.889+0000 7f9576ffd700 1 -- 172.21.15.55:0/3990574858 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v538) v1 ==== 154+0+0 (secure 0 0 0) 0x7f95901525a0 con 0x7f95900cb900 2023-10-06T03:48:14.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.892+0000 7f958ffff700 1 -- 172.21.15.55:0/3990574858 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f957c066f30 msgr2=0x7f957c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:14.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.892+0000 7f958ffff700 1 --2- 172.21.15.55:0/3990574858 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f957c066f30 0x7f957c0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f959015b420 tx=0x7f95840091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.892+0000 7f958ffff700 1 -- 172.21.15.55:0/3990574858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95900cb900 msgr2=0x7f959014edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:14.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.892+0000 7f958ffff700 1 --2- 172.21.15.55:0/3990574858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95900cb900 0x7f959014edb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f958000c790 tx=0x7f9580007e10 comp rx=0 tx=0).stop 2023-10-06T03:48:14.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.892+0000 7f958ffff700 1 -- 172.21.15.55:0/3990574858 shutdown_connections 2023-10-06T03:48:14.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.892+0000 7f958ffff700 1 --2- 172.21.15.55:0/3990574858 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f957c066f30 0x7f957c0693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.892+0000 7f958ffff700 1 --2- 172.21.15.55:0/3990574858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95901566e0 0x7f95901698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.892+0000 7f958ffff700 1 --2- 172.21.15.55:0/3990574858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95900cb900 0x7f959014edb0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.892+0000 7f958ffff700 1 --2- 172.21.15.55:0/3990574858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9590156310 0x7f9590152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:14.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.892+0000 7f958ffff700 1 -- 172.21.15.55:0/3990574858 >> 172.21.15.55:0/3990574858 conn(0x7f95900cc1f0 msgr2=0x7f95900addb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:14.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.892+0000 7f958ffff700 1 -- 172.21.15.55:0/3990574858 shutdown_connections 2023-10-06T03:48:14.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:14.892+0000 7f958ffff700 1 -- 172.21.15.55:0/3990574858 wait complete. 2023-10-06T03:48:14.911 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-10-06T03:48:14.911 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-10-06T03:48:14.912 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T03:48:14.912 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-06T03:48:15.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.097+0000 7f9b56b9d700 1 Processor -- start 2023-10-06T03:48:15.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.097+0000 7f9b56b9d700 1 -- start start 2023-10-06T03:48:15.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.098+0000 7f9b56b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b50159420 0x7f9b500b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:15.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.098+0000 7f9b56b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9b5015d230 0x7f9b5015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:15.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.098+0000 7f9b56b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5015dbd0 con 0x7f9b501597f0 2023-10-06T03:48:15.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.098+0000 7f9b56b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b500b2980 con 0x7f9b5015d230 2023-10-06T03:48:15.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.098+0000 7f9b56b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b500b11d0 con 0x7f9b50159420 2023-10-06T03:48:15.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.099+0000 7f9b5639c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9b5015d230 0x7f9b5015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:15.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.099+0000 7f9b5539a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9b501597f0 0x7f9b500b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45868/0 (socket says 172.21.15.55:45868) 2023-10-06T03:48:15.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.099+0000 7f9b5539a700 1 -- 172.21.15.55:0/2767237340 learned_addr learned my addr 172.21.15.55:0/2767237340 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:15.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.099+0000 7f9b55b9b700 1 --2- 172.21.15.55:0/2767237340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b50159420 0x7f9b500b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:15.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.099+0000 7f9b5639c700 1 -- 172.21.15.55:0/2767237340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b50159420 msgr2=0x7f9b500b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:15.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.099+0000 7f9b5639c700 1 --2- 172.21.15.55:0/2767237340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b50159420 0x7f9b500b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:15.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.099+0000 7f9b5639c700 1 -- 172.21.15.55:0/2767237340 >> v1:172.21.15.55:6789/0 conn(0x7f9b501597f0 legacy=0x7f9b500b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:15.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.099+0000 7f9b5639c700 1 -- 172.21.15.55:0/2767237340 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b500b14b0 con 0x7f9b5015d230 2023-10-06T03:48:15.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.099+0000 7f9b54b99700 1 -- 172.21.15.55:0/2767237340 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1021305197 0 0) 0x7f9b40001110 con 0x7f9b501597f0 2023-10-06T03:48:15.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.099+0000 7f9b5639c700 1 --2- 172.21.15.55:0/2767237340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5015d230 0x7f9b5015d690 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009e60 tx=0x7f9b48009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e2ee584190e487 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:15.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.100+0000 7f9b54b99700 1 -- 172.21.15.55:0/2767237340 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4800e070 con 0x7f9b5015d230 2023-10-06T03:48:15.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.100+0000 7f9b54b99700 1 -- 172.21.15.55:0/2767237340 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4800bb60 con 0x7f9b5015d230 2023-10-06T03:48:15.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.100+0000 7f9b54b99700 1 -- 172.21.15.55:0/2767237340 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48012430 con 0x7f9b5015d230 2023-10-06T03:48:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.100+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2767237340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5015d230 msgr2=0x7f9b5015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.100+0000 7f9b56b9d700 1 --2- 172.21.15.55:0/2767237340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5015d230 0x7f9b5015d690 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009e60 tx=0x7f9b48009780 comp rx=0 tx=0).stop 2023-10-06T03:48:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.100+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2767237340 shutdown_connections 2023-10-06T03:48:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.100+0000 7f9b56b9d700 1 --2- 172.21.15.55:0/2767237340 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5015d230 0x7f9b5015d690 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.100+0000 7f9b56b9d700 1 --2- 172.21.15.55:0/2767237340 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b50159420 0x7f9b500b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.100+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2767237340 >> 172.21.15.55:0/2767237340 conn(0x7f9b500d0660 msgr2=0x7f9b500d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:15.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.100+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2767237340 shutdown_connections 2023-10-06T03:48:15.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.100+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2767237340 wait complete. 2023-10-06T03:48:15.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.102+0000 7f9b56b9d700 1 Processor -- start 2023-10-06T03:48:15.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.102+0000 7f9b56b9d700 1 -- start start 2023-10-06T03:48:15.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b56b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b50159420 0x7f9b501531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:15.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b56b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b501597f0 0x7f9b5015bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:15.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b56b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5015d230 0x7f9b50156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:15.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b56b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5015cf70 con 0x7f9b5015d230 2023-10-06T03:48:15.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b56b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b501693d0 con 0x7f9b501597f0 2023-10-06T03:48:15.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b56b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5015c6d0 con 0x7f9b50159420 2023-10-06T03:48:15.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b5639c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5015d230 0x7f9b50156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:15.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b5639c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5015d230 0x7f9b50156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40914/0 (socket says 172.21.15.55:40914) 2023-10-06T03:48:15.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b5639c700 1 -- 172.21.15.55:0/2796780051 learned_addr learned my addr 172.21.15.55:0/2796780051 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:15.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b55b9b700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b50159420 0x7f9b501531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:15.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b5539a700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b501597f0 0x7f9b5015bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:15.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b5639c700 1 -- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b50159420 msgr2=0x7f9b501531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:15.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b5639c700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b50159420 0x7f9b501531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:15.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b5639c700 1 -- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b501597f0 msgr2=0x7f9b5015bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:15.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b5639c700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b501597f0 0x7f9b5015bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:15.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b5639c700 1 -- 172.21.15.55:0/2796780051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b48009a20 con 0x7f9b5015d230 2023-10-06T03:48:15.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.103+0000 7f9b55b9b700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b50159420 0x7f9b501531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:15.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.104+0000 7f9b5639c700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5015d230 0x7f9b50156a20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9b50089340 tx=0x7f9b48019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:15.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.104+0000 7f9b45ffb700 1 -- 172.21.15.55:0/2796780051 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48027470 con 0x7f9b5015d230 2023-10-06T03:48:15.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.104+0000 7f9b45ffb700 1 -- 172.21.15.55:0/2796780051 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b48027610 con 0x7f9b5015d230 2023-10-06T03:48:15.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.104+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2796780051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5015c9b0 con 0x7f9b5015d230 2023-10-06T03:48:15.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.104+0000 7f9b45ffb700 1 -- 172.21.15.55:0/2796780051 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b480279d0 con 0x7f9b5015d230 2023-10-06T03:48:15.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.104+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2796780051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b50169610 con 0x7f9b5015d230 2023-10-06T03:48:15.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.105+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2796780051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5004c3c0 con 0x7f9b5015d230 2023-10-06T03:48:15.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.106+0000 7f9b45ffb700 1 -- 172.21.15.55:0/2796780051 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9b48024070 con 0x7f9b5015d230 2023-10-06T03:48:15.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.107+0000 7f9b45ffb700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9b34066f30 0x7f9b340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:15.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.107+0000 7f9b45ffb700 1 -- 172.21.15.55:0/2796780051 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f9b48087270 con 0x7f9b5015d230 2023-10-06T03:48:15.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.107+0000 7f9b55b9b700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9b34066f30 0x7f9b340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:15.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.108+0000 7f9b55b9b700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9b34066f30 0x7f9b340693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009d80 tx=0x7f9b4c008040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:15.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.111+0000 7f9b45ffb700 1 -- 172.21.15.55:0/2796780051 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b48054250 con 0x7f9b5015d230 2023-10-06T03:48:15.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.390+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2796780051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f9b500b1790 con 0x7f9b5015d230 2023-10-06T03:48:15.576 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:48:15.580 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:48:15.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.889+0000 7f9b45ffb700 1 -- 172.21.15.55:0/2796780051 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v539) v1 ==== 159+0+0 (secure 0 0 0) 0x7f9b48057870 con 0x7f9b5015d230 2023-10-06T03:48:15.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.892+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9b34066f30 msgr2=0x7f9b340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:15.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.892+0000 7f9b56b9d700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9b34066f30 0x7f9b340693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009d80 tx=0x7f9b4c008040 comp rx=0 tx=0).stop 2023-10-06T03:48:15.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.892+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2796780051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5015d230 msgr2=0x7f9b50156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:15.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.892+0000 7f9b56b9d700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5015d230 0x7f9b50156a20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9b50089340 tx=0x7f9b48019000 comp rx=0 tx=0).stop 2023-10-06T03:48:15.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.892+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2796780051 shutdown_connections 2023-10-06T03:48:15.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.892+0000 7f9b56b9d700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9b34066f30 0x7f9b340693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:15.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.892+0000 7f9b56b9d700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b5015d230 0x7f9b50156a20 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:15.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.892+0000 7f9b56b9d700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b50159420 0x7f9b501531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:15.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.892+0000 7f9b56b9d700 1 --2- 172.21.15.55:0/2796780051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b501597f0 0x7f9b5015bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:15.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.892+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2796780051 >> 172.21.15.55:0/2796780051 conn(0x7f9b500d0660 msgr2=0x7f9b501540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:15.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.892+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2796780051 shutdown_connections 2023-10-06T03:48:15.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:15.892+0000 7f9b56b9d700 1 -- 172.21.15.55:0/2796780051 wait complete. 2023-10-06T03:48:15.911 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:48:16.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.101+0000 7fedaf59e700 1 Processor -- start 2023-10-06T03:48:16.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.101+0000 7fedaf59e700 1 -- start start 2023-10-06T03:48:16.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.102+0000 7fedaf59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fedb015a3b0 0x7fedb015a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:16.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.102+0000 7fedaf59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb00aef00 0x7fedb015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:16.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.103+0000 7fedaf59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb00aea70 con 0x7fedb006e470 2023-10-06T03:48:16.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.103+0000 7fedaf59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb00ae480 con 0x7fedb015a3b0 2023-10-06T03:48:16.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.103+0000 7fedaf59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb015acc0 con 0x7fedb00aef00 2023-10-06T03:48:16.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.103+0000 7fedaed9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fedb006e470 0x7fedb00c7280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45884/0 (socket says 172.21.15.55:45884) 2023-10-06T03:48:16.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.103+0000 7fedaed9d700 1 -- 172.21.15.55:0/13218823 learned_addr learned my addr 172.21.15.55:0/13218823 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:16.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.103+0000 7fedae59c700 1 --2- 172.21.15.55:0/13218823 >> v2:172.21.15.55:3300/0 conn(0x7fedb015a3b0 0x7fedb015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:16.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.103+0000 7fedadd9b700 1 --2- 172.21.15.55:0/13218823 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb00aef00 0x7fedb015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:16.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.103+0000 7fedad59a700 1 -- 172.21.15.55:0/13218823 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4058484228 0 0) 0x7fedb00aea70 con 0x7fedb006e470 2023-10-06T03:48:16.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.103+0000 7fedad59a700 1 -- 172.21.15.55:0/13218823 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed98003570 con 0x7fedb006e470 2023-10-06T03:48:16.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.103+0000 7fedae59c700 1 -- 172.21.15.55:0/13218823 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb00aef00 msgr2=0x7fedb015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:16.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.103+0000 7fedae59c700 1 --2- 172.21.15.55:0/13218823 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb00aef00 0x7fedb015d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.103+0000 7fedae59c700 1 -- 172.21.15.55:0/13218823 >> v1:172.21.15.55:6789/0 conn(0x7fedb006e470 legacy=0x7fedb00c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:16.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.103+0000 7fedae59c700 1 -- 172.21.15.55:0/13218823 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb015dd30 con 0x7fedb015a3b0 2023-10-06T03:48:16.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.104+0000 7fedae59c700 1 --2- 172.21.15.55:0/13218823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb015a3b0 0x7fedb015a780 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7feda00098e0 tx=0x7feda00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb02d9d104799d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:16.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.104+0000 7fedad59a700 1 -- 172.21.15.55:0/13218823 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0011630 con 0x7fedb015a3b0 2023-10-06T03:48:16.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.104+0000 7fedad59a700 1 -- 172.21.15.55:0/13218823 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda0011790 con 0x7fedb015a3b0 2023-10-06T03:48:16.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.104+0000 7fedad59a700 1 -- 172.21.15.55:0/13218823 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0011b30 con 0x7fedb015a3b0 2023-10-06T03:48:16.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.105+0000 7fedaf59e700 1 -- 172.21.15.55:0/13218823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb015a3b0 msgr2=0x7fedb015a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:16.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.105+0000 7fedaf59e700 1 --2- 172.21.15.55:0/13218823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb015a3b0 0x7fedb015a780 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7feda00098e0 tx=0x7feda00091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.105+0000 7fedaf59e700 1 -- 172.21.15.55:0/13218823 shutdown_connections 2023-10-06T03:48:16.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.105+0000 7fedaf59e700 1 --2- 172.21.15.55:0/13218823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb015a3b0 0x7fedb015a780 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.105+0000 7fedaf59e700 1 --2- 172.21.15.55:0/13218823 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb00aef00 0x7fedb015d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.105+0000 7fedaf59e700 1 -- 172.21.15.55:0/13218823 >> 172.21.15.55:0/13218823 conn(0x7fedb00c9f70 msgr2=0x7fedb006dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:16.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.105+0000 7fedaf59e700 1 -- 172.21.15.55:0/13218823 shutdown_connections 2023-10-06T03:48:16.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.105+0000 7fedaf59e700 1 -- 172.21.15.55:0/13218823 wait complete. 2023-10-06T03:48:16.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.106+0000 7fedaf59e700 1 Processor -- start 2023-10-06T03:48:16.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.106+0000 7fedaf59e700 1 -- start start 2023-10-06T03:48:16.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.107+0000 7fedaf59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedb006e470 0x7fedb00c3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:16.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.107+0000 7fedaf59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb00aef00 0x7fedb01527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:16.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.107+0000 7fedaf59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb015a3b0 0x7fedb015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:16.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.107+0000 7fedaf59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb00c4e20 con 0x7fedb015a3b0 2023-10-06T03:48:16.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.107+0000 7fedaf59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb00c4020 con 0x7fedb006e470 2023-10-06T03:48:16.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.107+0000 7fedaf59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb00b1680 con 0x7fedb00aef00 2023-10-06T03:48:16.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.107+0000 7fedaed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb015a3b0 0x7fedb015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:16.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.108+0000 7fedaed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb015a3b0 0x7fedb015f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40934/0 (socket says 172.21.15.55:40934) 2023-10-06T03:48:16.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.108+0000 7fedaed9d700 1 -- 172.21.15.55:0/2697988500 learned_addr learned my addr 172.21.15.55:0/2697988500 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:16.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.108+0000 7fedadd9b700 1 --2- 172.21.15.55:0/2697988500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb00aef00 0x7fedb01527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:16.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.108+0000 7fedae59c700 1 --2- 172.21.15.55:0/2697988500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedb006e470 0x7fedb00c3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.108+0000 7fedaed9d700 1 -- 172.21.15.55:0/2697988500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb00aef00 msgr2=0x7fedb01527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.108+0000 7fedaed9d700 1 --2- 172.21.15.55:0/2697988500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb00aef00 0x7fedb01527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.108+0000 7fedaed9d700 1 -- 172.21.15.55:0/2697988500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedb006e470 msgr2=0x7fedb00c3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.108+0000 7fedaed9d700 1 --2- 172.21.15.55:0/2697988500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedb006e470 0x7fedb00c3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.108+0000 7fedaed9d700 1 -- 172.21.15.55:0/2697988500 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda0009420 con 0x7fedb015a3b0 2023-10-06T03:48:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.108+0000 7fedaed9d700 1 --2- 172.21.15.55:0/2697988500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb015a3b0 0x7fedb015f3d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feda8001460 tx=0x7feda8001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.108+0000 7fed9e7fc700 1 -- 172.21.15.55:0/2697988500 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda800a890 con 0x7fedb015a3b0 2023-10-06T03:48:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.109+0000 7fed9e7fc700 1 -- 172.21.15.55:0/2697988500 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda800a9f0 con 0x7fedb015a3b0 2023-10-06T03:48:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.109+0000 7fed9e7fc700 1 -- 172.21.15.55:0/2697988500 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda800ad70 con 0x7fedb015a3b0 2023-10-06T03:48:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.109+0000 7fedaf59e700 1 -- 172.21.15.55:0/2697988500 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb0152d10 con 0x7fedb015a3b0 2023-10-06T03:48:16.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.109+0000 7fedaf59e700 1 -- 172.21.15.55:0/2697988500 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb00c44f0 con 0x7fedb015a3b0 2023-10-06T03:48:16.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.110+0000 7fedaf59e700 1 -- 172.21.15.55:0/2697988500 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb006cfa0 con 0x7fedb015a3b0 2023-10-06T03:48:16.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.111+0000 7fed9e7fc700 1 -- 172.21.15.55:0/2697988500 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feda8007300 con 0x7fedb015a3b0 2023-10-06T03:48:16.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.112+0000 7fed9e7fc700 1 --2- 172.21.15.55:0/2697988500 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fed90066f30 0x7fed900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:16.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.112+0000 7fed9e7fc700 1 -- 172.21.15.55:0/2697988500 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6200+0+0 (secure 0 0 0) 0x7feda8084b60 con 0x7fedb015a3b0 2023-10-06T03:48:16.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.112+0000 7fedae59c700 1 --2- 172.21.15.55:0/2697988500 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fed90066f30 0x7fed900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:16.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.113+0000 7fedae59c700 1 --2- 172.21.15.55:0/2697988500 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fed90066f30 0x7fed900693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feda00091e0 tx=0x7feda0017000 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:16.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.116+0000 7fed9e7fc700 1 -- 172.21.15.55:0/2697988500 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feda8051b30 con 0x7fedb015a3b0 2023-10-06T03:48:16.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.320+0000 7fedaf59e700 1 -- 172.21.15.55:0/2697988500 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fedb00c28a0 con 0x7fedb015a3b0 2023-10-06T03:48:16.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.325+0000 7fed9e7fc700 1 -- 172.21.15.55:0/2697988500 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7feda8055150 con 0x7fedb015a3b0 2023-10-06T03:48:16.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.327+0000 7fedaf59e700 1 -- 172.21.15.55:0/2697988500 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fed90066f30 msgr2=0x7fed900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:16.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.327+0000 7fedaf59e700 1 --2- 172.21.15.55:0/2697988500 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fed90066f30 0x7fed900693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feda00091e0 tx=0x7feda0017000 comp rx=0 tx=0).stop 2023-10-06T03:48:16.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.327+0000 7fedaf59e700 1 -- 172.21.15.55:0/2697988500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb015a3b0 msgr2=0x7fedb015f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:16.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.327+0000 7fedaf59e700 1 --2- 172.21.15.55:0/2697988500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb015a3b0 0x7fedb015f3d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feda8001460 tx=0x7feda8001ae0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.327+0000 7fedaf59e700 1 -- 172.21.15.55:0/2697988500 shutdown_connections 2023-10-06T03:48:16.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.327+0000 7fedaf59e700 1 --2- 172.21.15.55:0/2697988500 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fed90066f30 0x7fed900693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.327+0000 7fedaf59e700 1 --2- 172.21.15.55:0/2697988500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fedb015a3b0 0x7fedb015f3d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.327+0000 7fedaf59e700 1 --2- 172.21.15.55:0/2697988500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fedb006e470 0x7fedb00c3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.327+0000 7fedaf59e700 1 --2- 172.21.15.55:0/2697988500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fedb00aef00 0x7fedb01527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.327+0000 7fedaf59e700 1 -- 172.21.15.55:0/2697988500 >> 172.21.15.55:0/2697988500 conn(0x7fedb00c9f70 msgr2=0x7fedb015b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:16.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.327+0000 7fedaf59e700 1 -- 172.21.15.55:0/2697988500 shutdown_connections 2023-10-06T03:48:16.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.327+0000 7fedaf59e700 1 -- 172.21.15.55:0/2697988500 wait complete. 2023-10-06T03:48:16.346 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:48:16.346 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:48:16.432 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:48:16.514 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:48:16.514 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:48:16.514 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:48:16.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.700+0000 7f5797fff700 1 Processor -- start 2023-10-06T03:48:16.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.700+0000 7f5797fff700 1 -- start start 2023-10-06T03:48:16.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.701+0000 7f5797fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5798150550 0x7f5798157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:16.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.701+0000 7f5797fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57980c7410 0x7f5798069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:16.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.701+0000 7f5797fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579806e2a0 con 0x7f5798150180 2023-10-06T03:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.701+0000 7f5797fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579806b2e0 con 0x7f5798150550 2023-10-06T03:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.701+0000 7f5797fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57980c78e0 con 0x7f57980c7410 2023-10-06T03:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.701+0000 7f57967fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5798150550 0x7f5798157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.701+0000 7f57967fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5798150550 0x7f5798157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40948/0 (socket says 172.21.15.55:40948) 2023-10-06T03:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.701+0000 7f57967fc700 1 -- 172.21.15.55:0/960533761 learned_addr learned my addr 172.21.15.55:0/960533761 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.702+0000 7f57977fe700 1 --2- 172.21.15.55:0/960533761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57980c7410 0x7f5798069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.702+0000 7f57967fc700 1 -- 172.21.15.55:0/960533761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57980c7410 msgr2=0x7f5798069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.702+0000 7f57967fc700 1 --2- 172.21.15.55:0/960533761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57980c7410 0x7f5798069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.702+0000 7f57967fc700 1 -- 172.21.15.55:0/960533761 >> v1:172.21.15.163:6789/0 conn(0x7f5798150180 legacy=0x7f57980c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.702+0000 7f57967fc700 1 -- 172.21.15.55:0/960533761 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57980c7bc0 con 0x7f5798150550 2023-10-06T03:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.702+0000 7f57967fc700 1 --2- 172.21.15.55:0/960533761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5798150550 0x7f5798157770 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f578c0099b0 tx=0x7f578c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f540dead179a29 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:16.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.702+0000 7f5795ffb700 1 -- 172.21.15.55:0/960533761 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c009ea0 con 0x7f5798150550 2023-10-06T03:48:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.703+0000 7f5795ffb700 1 -- 172.21.15.55:0/960533761 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c00b860 con 0x7f5798150550 2023-10-06T03:48:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.703+0000 7f5795ffb700 1 -- 172.21.15.55:0/960533761 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c00bc00 con 0x7f5798150550 2023-10-06T03:48:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.703+0000 7f5797fff700 1 -- 172.21.15.55:0/960533761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5798150550 msgr2=0x7f5798157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.703+0000 7f5797fff700 1 --2- 172.21.15.55:0/960533761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5798150550 0x7f5798157770 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f578c0099b0 tx=0x7f578c009250 comp rx=0 tx=0).stop 2023-10-06T03:48:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.703+0000 7f5797fff700 1 -- 172.21.15.55:0/960533761 shutdown_connections 2023-10-06T03:48:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.703+0000 7f5797fff700 1 --2- 172.21.15.55:0/960533761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57980c7410 0x7f5798069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.703+0000 7f5797fff700 1 --2- 172.21.15.55:0/960533761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5798150550 0x7f5798157770 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.703+0000 7f5797fff700 1 -- 172.21.15.55:0/960533761 >> 172.21.15.55:0/960533761 conn(0x7f57980cc1f0 msgr2=0x7f57980cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.703+0000 7f5797fff700 1 -- 172.21.15.55:0/960533761 shutdown_connections 2023-10-06T03:48:16.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.703+0000 7f5797fff700 1 -- 172.21.15.55:0/960533761 wait complete. 2023-10-06T03:48:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.705+0000 7f5797fff700 1 Processor -- start 2023-10-06T03:48:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.705+0000 7f5797fff700 1 -- start start 2023-10-06T03:48:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f5797fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57980c7410 0x7f579814d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f5797fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5798150180 0x7f5798069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f5797fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5798150550 0x7f57980c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f5797fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57980b02b0 con 0x7f5798150180 2023-10-06T03:48:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f5797fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57980c3340 con 0x7f5798150550 2023-10-06T03:48:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f5797fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57980c5c40 con 0x7f57980c7410 2023-10-06T03:48:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f57967fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5798150180 0x7f5798069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:16.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f57967fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5798150180 0x7f5798069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40964/0 (socket says 172.21.15.55:40964) 2023-10-06T03:48:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f57967fc700 1 -- 172.21.15.55:0/3110418317 learned_addr learned my addr 172.21.15.55:0/3110418317 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f57977fe700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5798150550 0x7f57980c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f5796ffd700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57980c7410 0x7f579814d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f57967fc700 1 -- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57980c7410 msgr2=0x7f579814d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f57967fc700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57980c7410 0x7f579814d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f57967fc700 1 -- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5798150550 msgr2=0x7f57980c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f57967fc700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5798150550 0x7f57980c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f57967fc700 1 -- 172.21.15.55:0/3110418317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c0094f0 con 0x7f5798150180 2023-10-06T03:48:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.706+0000 7f5796ffd700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57980c7410 0x7f579814d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.707+0000 7f57967fc700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5798150180 0x7f5798069e20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f579806fe40 tx=0x7f578c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.707+0000 7f577effd700 1 -- 172.21.15.55:0/3110418317 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c00bbe0 con 0x7f5798150180 2023-10-06T03:48:16.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.707+0000 7f577effd700 1 -- 172.21.15.55:0/3110418317 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c011420 con 0x7f5798150180 2023-10-06T03:48:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.707+0000 7f577effd700 1 -- 172.21.15.55:0/3110418317 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c0117c0 con 0x7f5798150180 2023-10-06T03:48:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.707+0000 7f5797fff700 1 -- 172.21.15.55:0/3110418317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57980c5f20 con 0x7f5798150180 2023-10-06T03:48:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.707+0000 7f5797fff700 1 -- 172.21.15.55:0/3110418317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57980c6200 con 0x7f5798150180 2023-10-06T03:48:16.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.708+0000 7f5797fff700 1 -- 172.21.15.55:0/3110418317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57980c3800 con 0x7f5798150180 2023-10-06T03:48:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.709+0000 7f577effd700 1 -- 172.21.15.55:0/3110418317 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f578c011bf0 con 0x7f5798150180 2023-10-06T03:48:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.710+0000 7f577effd700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5784066f30 0x7f57840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.710+0000 7f577effd700 1 -- 172.21.15.55:0/3110418317 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f578c085ff0 con 0x7f5798150180 2023-10-06T03:48:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.711+0000 7f5796ffd700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5784066f30 0x7f57840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:16.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.711+0000 7f5796ffd700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5784066f30 0x7f57840693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5788007fd0 tx=0x7f5788009040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:16.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.714+0000 7f577effd700 1 -- 172.21.15.55:0/3110418317 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f578c0b75a0 con 0x7f5798150180 2023-10-06T03:48:16.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.961+0000 7f5797fff700 1 -- 172.21.15.55:0/3110418317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f579804c3c0 con 0x7f5798150180 2023-10-06T03:48:16.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.962+0000 7f577effd700 1 -- 172.21.15.55:0/3110418317 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f578c015de0 con 0x7f5798150180 2023-10-06T03:48:16.963 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:16.963 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/3493866751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":3493866751},{"type":"v1","addr":"172.21.15.163:6836","nonce":3493866751}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5549,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/3380885785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3380885785},{"type":"v1","addr":"172.21.15.163:6833","nonce":3380885785}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":5641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3246061279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3246061279},{"type":"v1","addr":"172.21.15.55:6835","nonce":3246061279}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1366768080","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1366768080},{"type":"v1","addr":"172.21.15.55:6834","nonce":1366768080}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[]} 2023-10-06T03:48:16.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.964+0000 7f5797fff700 1 -- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5784066f30 msgr2=0x7f57840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:16.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.964+0000 7f5797fff700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5784066f30 0x7f57840693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5788007fd0 tx=0x7f5788009040 comp rx=0 tx=0).stop 2023-10-06T03:48:16.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.964+0000 7f5797fff700 1 -- 172.21.15.55:0/3110418317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5798150180 msgr2=0x7f5798069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:16.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.964+0000 7f5797fff700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5798150180 0x7f5798069e20 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f579806fe40 tx=0x7f578c017000 comp rx=0 tx=0).stop 2023-10-06T03:48:16.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.964+0000 7f5797fff700 1 -- 172.21.15.55:0/3110418317 shutdown_connections 2023-10-06T03:48:16.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.964+0000 7f5797fff700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5784066f30 0x7f57840693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.964+0000 7f5797fff700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5798150550 0x7f57980c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.964+0000 7f5797fff700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57980c7410 0x7f579814d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.964+0000 7f5797fff700 1 --2- 172.21.15.55:0/3110418317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5798150180 0x7f5798069e20 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:16.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.964+0000 7f5797fff700 1 -- 172.21.15.55:0/3110418317 >> 172.21.15.55:0/3110418317 conn(0x7f57980cc1f0 msgr2=0x7f57980c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:16.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.964+0000 7f5797fff700 1 -- 172.21.15.55:0/3110418317 shutdown_connections 2023-10-06T03:48:16.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:16.964+0000 7f5797fff700 1 -- 172.21.15.55:0/3110418317 wait complete. 2023-10-06T03:48:16.967 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 28 2023-10-06T03:48:16.983 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:48:16.983 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:48:16.984 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:48:16.984 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:48:16.984 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:48:16.984 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:48:16.984 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:48:16.984 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:48:16.985 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:48:16.985 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:48:16.985 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:48:16.985 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:48:16.985 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:48:16.985 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:48:16.986 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:48:16.986 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:48:16.998 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:48:16.999 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:48:16.999 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:48:16.999 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:48:17.004 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:48:17.005 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:48:17.005 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:48:17.005 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:48:17.043 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:48:17.043 DEBUG:teuthology.parallel:result is None 2023-10-06T03:48:17.045 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:48:17.046 DEBUG:teuthology.parallel:result is None 2023-10-06T03:48:17.050 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:48:17.050 DEBUG:teuthology.parallel:result is None 2023-10-06T03:48:17.053 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:48:17.053 DEBUG:teuthology.parallel:result is None 2023-10-06T03:48:17.053 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:48:17.180 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T03:48:17.181 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-10-06T03:48:17.187 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T03:48:17.187 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2023-10-06T03:48:17.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.248+0000 7f97dffff700 1 Processor -- start 2023-10-06T03:48:17.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.248+0000 7f97dffff700 1 -- start start 2023-10-06T03:48:17.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.250+0000 7f97dffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f97e006d950 0x7f97e00c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:17.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.250+0000 7f97dffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f97e006dd20 0x7f97e00c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:17.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.250+0000 7f97dffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e006ae70 con 0x7f97e006a0a0 2023-10-06T03:48:17.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.250+0000 7f97dffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e006a540 con 0x7f97e006dd20 2023-10-06T03:48:17.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.250+0000 7f97dffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e006a820 con 0x7f97e006d950 2023-10-06T03:48:17.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.250+0000 7f97df7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f97e006dd20 0x7f97e00c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:17.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.250+0000 7f97deffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f97e006d950 0x7f97e00c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:17.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.250+0000 7f97df7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f97e006dd20 0x7f97e00c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41002/0 (socket says 172.21.15.55:41002) 2023-10-06T03:48:17.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.250+0000 7f97df7fe700 1 -- 172.21.15.55:0/1518625591 learned_addr learned my addr 172.21.15.55:0/1518625591 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.251+0000 7f97df7fe700 1 -- 172.21.15.55:0/1518625591 >> v2:172.21.15.163:3300/0 conn(0x7f97e006d950 msgr2=0x7f97e00c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.251+0000 7f97df7fe700 1 --2- 172.21.15.55:0/1518625591 >> v2:172.21.15.163:3300/0 conn(0x7f97e006d950 0x7f97e00c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.251+0000 7f97df7fe700 1 -- 172.21.15.55:0/1518625591 >> v1:172.21.15.163:6789/0 conn(0x7f97e006a0a0 legacy=0x7f97e00c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.251+0000 7f97df7fe700 1 -- 172.21.15.55:0/1518625591 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e006ab00 con 0x7f97e006dd20 2023-10-06T03:48:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.251+0000 7f97deffd700 1 --2- 172.21.15.55:0/1518625591 >> v2:172.21.15.163:3300/0 conn(0x7f97e006d950 0x7f97e00c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.251+0000 7f97ddffb700 1 -- 172.21.15.55:0/1518625591 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056296229 0 0) 0x7f97e006ae70 con 0x7f97e006a0a0 2023-10-06T03:48:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.251+0000 7f97df7fe700 1 --2- 172.21.15.55:0/1518625591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97e006dd20 0x7f97e00c7320 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f97e00c9c70 tx=0x7f97d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2def357cb34956 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:17.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.251+0000 7f97ddffb700 1 -- 172.21.15.55:0/1518625591 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d800e070 con 0x7f97e006dd20 2023-10-06T03:48:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.251+0000 7f97ddffb700 1 -- 172.21.15.55:0/1518625591 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d800ba20 con 0x7f97e006dd20 2023-10-06T03:48:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.251+0000 7f97ddffb700 1 -- 172.21.15.55:0/1518625591 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d80092e0 con 0x7f97e006dd20 2023-10-06T03:48:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.252+0000 7f97dffff700 1 -- 172.21.15.55:0/1518625591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97e006dd20 msgr2=0x7f97e00c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.252+0000 7f97dffff700 1 --2- 172.21.15.55:0/1518625591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97e006dd20 0x7f97e00c7320 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f97e00c9c70 tx=0x7f97d8009840 comp rx=0 tx=0).stop 2023-10-06T03:48:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.252+0000 7f97dffff700 1 -- 172.21.15.55:0/1518625591 shutdown_connections 2023-10-06T03:48:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.252+0000 7f97dffff700 1 --2- 172.21.15.55:0/1518625591 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97e006dd20 0x7f97e00c7320 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.252+0000 7f97dffff700 1 --2- 172.21.15.55:0/1518625591 >> v2:172.21.15.163:3300/0 conn(0x7f97e006d950 0x7f97e00c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.252+0000 7f97dffff700 1 -- 172.21.15.55:0/1518625591 >> 172.21.15.55:0/1518625591 conn(0x7f97e00cc1f0 msgr2=0x7f97e00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.252+0000 7f97dffff700 1 -- 172.21.15.55:0/1518625591 shutdown_connections 2023-10-06T03:48:17.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.252+0000 7f97dffff700 1 -- 172.21.15.55:0/1518625591 wait complete. 2023-10-06T03:48:17.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.253+0000 7f97dffff700 1 Processor -- start 2023-10-06T03:48:17.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.253+0000 7f97dffff700 1 -- start start 2023-10-06T03:48:17.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.254+0000 7f97dffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97e006a0a0 0x7f97e015b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:17.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.254+0000 7f97dffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97e006d950 0x7f97e016ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:17.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.254+0000 7f97dffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97e006dd20 0x7f97e016d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:17.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.254+0000 7f97dffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e00ca870 con 0x7f97e006a0a0 2023-10-06T03:48:17.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.254+0000 7f97dffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e00cab10 con 0x7f97e006dd20 2023-10-06T03:48:17.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.254+0000 7f97dffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e00cadf0 con 0x7f97e006d950 2023-10-06T03:48:17.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.254+0000 7f97deffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97e006a0a0 0x7f97e015b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:17.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97deffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97e006a0a0 0x7f97e015b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41008/0 (socket says 172.21.15.55:41008) 2023-10-06T03:48:17.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97deffd700 1 -- 172.21.15.55:0/3060431626 learned_addr learned my addr 172.21.15.55:0/3060431626 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:17.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97df7fe700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97e006dd20 0x7f97e016d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:17.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97de7fc700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97e006d950 0x7f97e016ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:17.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97deffd700 1 -- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97e006d950 msgr2=0x7f97e016ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:17.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97deffd700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97e006d950 0x7f97e016ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:17.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97deffd700 1 -- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97e006dd20 msgr2=0x7f97e016d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:17.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97deffd700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97e006dd20 0x7f97e016d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:17.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97deffd700 1 -- 172.21.15.55:0/3060431626 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d0009420 con 0x7f97e006a0a0 2023-10-06T03:48:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97de7fc700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97e006d950 0x7f97e016ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97deffd700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97e006a0a0 0x7f97e015b4a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f97d000ec80 tx=0x7f97d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97c6ffd700 1 -- 172.21.15.55:0/3060431626 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d000c8f0 con 0x7f97e006a0a0 2023-10-06T03:48:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97c6ffd700 1 -- 172.21.15.55:0/3060431626 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d000ca50 con 0x7f97e006a0a0 2023-10-06T03:48:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97dffff700 1 -- 172.21.15.55:0/3060431626 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d8009ae0 con 0x7f97e006a0a0 2023-10-06T03:48:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.255+0000 7f97c6ffd700 1 -- 172.21.15.55:0/3060431626 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d000cdd0 con 0x7f97e006a0a0 2023-10-06T03:48:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.256+0000 7f97dffff700 1 -- 172.21.15.55:0/3060431626 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e00cb3b0 con 0x7f97e006a0a0 2023-10-06T03:48:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.256+0000 7f97dffff700 1 -- 172.21.15.55:0/3060431626 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e00cb720 con 0x7f97e006a0a0 2023-10-06T03:48:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.257+0000 7f97c6ffd700 1 -- 172.21.15.55:0/3060431626 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97d0013680 con 0x7f97e006a0a0 2023-10-06T03:48:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.258+0000 7f97c6ffd700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97cc066f30 0x7f97cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.258+0000 7f97c6ffd700 1 -- 172.21.15.55:0/3060431626 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f97d0084aa0 con 0x7f97e006a0a0 2023-10-06T03:48:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.259+0000 7f97de7fc700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97cc066f30 0x7f97cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.259+0000 7f97de7fc700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97cc066f30 0x7f97cc0693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f97e0152e60 tx=0x7f97d4009040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:17.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.263+0000 7f97c6ffd700 1 -- 172.21.15.55:0/3060431626 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97d0051b20 con 0x7f97e006a0a0 2023-10-06T03:48:17.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.481+0000 7f97dffff700 1 -- 172.21.15.55:0/3060431626 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f97e004c3c0 con 0x7f97e006a0a0 2023-10-06T03:48:17.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.911+0000 7f97c6ffd700 1 -- 172.21.15.55:0/3060431626 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v74) v1 ==== 97+0+0 (secure 0 0 0) 0x7f97d0055140 con 0x7f97e006a0a0 2023-10-06T03:48:17.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.914+0000 7f97dffff700 1 -- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97cc066f30 msgr2=0x7f97cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:17.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.914+0000 7f97dffff700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97cc066f30 0x7f97cc0693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f97e0152e60 tx=0x7f97d4009040 comp rx=0 tx=0).stop 2023-10-06T03:48:17.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.914+0000 7f97dffff700 1 -- 172.21.15.55:0/3060431626 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97e006a0a0 msgr2=0x7f97e015b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:17.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.914+0000 7f97dffff700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97e006a0a0 0x7f97e015b4a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f97d000ec80 tx=0x7f97d00091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:17.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.914+0000 7f97dffff700 1 -- 172.21.15.55:0/3060431626 shutdown_connections 2023-10-06T03:48:17.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.914+0000 7f97dffff700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97cc066f30 0x7f97cc0693e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:17.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.914+0000 7f97dffff700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97e006dd20 0x7f97e016d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:17.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.914+0000 7f97dffff700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97e006a0a0 0x7f97e015b4a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:17.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.914+0000 7f97dffff700 1 --2- 172.21.15.55:0/3060431626 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97e006d950 0x7f97e016ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:17.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.914+0000 7f97dffff700 1 -- 172.21.15.55:0/3060431626 >> 172.21.15.55:0/3060431626 conn(0x7f97e00cc1f0 msgr2=0x7f97e0156a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:17.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.914+0000 7f97dffff700 1 -- 172.21.15.55:0/3060431626 shutdown_connections 2023-10-06T03:48:17.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:17.914+0000 7f97dffff700 1 -- 172.21.15.55:0/3060431626 wait complete. 2023-10-06T03:48:17.917 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2023-10-06T03:48:17.936 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:48:18.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.122+0000 7f3f8d19d700 1 Processor -- start 2023-10-06T03:48:18.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.122+0000 7f3f8d19d700 1 -- start start 2023-10-06T03:48:18.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.123+0000 7f3f8d19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3f88156310 0x7f3f880c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:18.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.123+0000 7f3f8d19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3f881566e0 0x7f3f880ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:18.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.123+0000 7f3f8d19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f880ca300 0x7f3f880ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:18.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.123+0000 7f3f8d19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f880ca860 con 0x7f3f88156310 2023-10-06T03:48:18.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.124+0000 7f3f8d19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f880cab40 con 0x7f3f881566e0 2023-10-06T03:48:18.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.124+0000 7f3f8d19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880cae20 con 0x7f3f880ca300 2023-10-06T03:48:18.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.124+0000 7f3f87fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3f88156310 0x7f3f880c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:18.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.124+0000 7f3f87fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3f88156310 0x7f3f880c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41022/0 (socket says 172.21.15.55:41022) 2023-10-06T03:48:18.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.124+0000 7f3f87fff700 1 -- 172.21.15.55:0/716045961 learned_addr learned my addr 172.21.15.55:0/716045961 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:18.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.124+0000 7f3f877fe700 1 --2- 172.21.15.55:0/716045961 >> v2:172.21.15.163:3300/0 conn(0x7f3f881566e0 0x7f3f880ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:18.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.124+0000 7f3f8c99c700 1 --2- 172.21.15.55:0/716045961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f880ca300 0x7f3f880ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:18.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.124+0000 7f3f87fff700 1 -- 172.21.15.55:0/716045961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f880ca300 msgr2=0x7f3f880ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:18.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.124+0000 7f3f87fff700 1 --2- 172.21.15.55:0/716045961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f880ca300 0x7f3f880ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.124+0000 7f3f87fff700 1 -- 172.21.15.55:0/716045961 >> v2:172.21.15.163:3300/0 conn(0x7f3f881566e0 msgr2=0x7f3f880ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:18.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.124+0000 7f3f87fff700 1 --2- 172.21.15.55:0/716045961 >> v2:172.21.15.163:3300/0 conn(0x7f3f881566e0 0x7f3f880ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.124+0000 7f3f87fff700 1 -- 172.21.15.55:0/716045961 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f880cb100 con 0x7f3f88156310 2023-10-06T03:48:18.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.124+0000 7f3f877fe700 1 --2- 172.21.15.55:0/716045961 >> v2:172.21.15.163:3300/0 conn(0x7f3f881566e0 0x7f3f880ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:18.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.125+0000 7f3f87fff700 1 --2- 172.21.15.55:0/716045961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f88156310 0x7f3f880c9dc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3f8806c000 tx=0x7f3f78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e9a3d6ba63dc72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:18.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.125+0000 7f3f86ffd700 1 -- 172.21.15.55:0/716045961 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7800b860 con 0x7f3f88156310 2023-10-06T03:48:18.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.125+0000 7f3f86ffd700 1 -- 172.21.15.55:0/716045961 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7800b9c0 con 0x7f3f88156310 2023-10-06T03:48:18.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.125+0000 7f3f86ffd700 1 -- 172.21.15.55:0/716045961 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7800bd60 con 0x7f3f88156310 2023-10-06T03:48:18.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.125+0000 7f3f8d19d700 1 -- 172.21.15.55:0/716045961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f88156310 msgr2=0x7f3f880c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:18.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.125+0000 7f3f8d19d700 1 --2- 172.21.15.55:0/716045961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f88156310 0x7f3f880c9dc0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3f8806c000 tx=0x7f3f78009250 comp rx=0 tx=0).stop 2023-10-06T03:48:18.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.125+0000 7f3f8d19d700 1 -- 172.21.15.55:0/716045961 shutdown_connections 2023-10-06T03:48:18.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.125+0000 7f3f8d19d700 1 --2- 172.21.15.55:0/716045961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f880ca300 0x7f3f880ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.125+0000 7f3f8d19d700 1 --2- 172.21.15.55:0/716045961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f88156310 0x7f3f880c9dc0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.125+0000 7f3f8d19d700 1 --2- 172.21.15.55:0/716045961 >> v2:172.21.15.163:3300/0 conn(0x7f3f881566e0 0x7f3f880ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.125+0000 7f3f8d19d700 1 -- 172.21.15.55:0/716045961 >> 172.21.15.55:0/716045961 conn(0x7f3f880cc1f0 msgr2=0x7f3f880cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:18.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.126+0000 7f3f8d19d700 1 -- 172.21.15.55:0/716045961 shutdown_connections 2023-10-06T03:48:18.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.126+0000 7f3f8d19d700 1 -- 172.21.15.55:0/716045961 wait complete. 2023-10-06T03:48:18.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.127+0000 7f3f8d19d700 1 Processor -- start 2023-10-06T03:48:18.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.127+0000 7f3f8d19d700 1 -- start start 2023-10-06T03:48:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.128+0000 7f3f8d19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f880ca300 0x7f3f8815ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.128+0000 7f3f8d19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f88156310 0x7f3f88141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.128+0000 7f3f8d19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f881566e0 0x7f3f880c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.128+0000 7f3f8d19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8815e390 con 0x7f3f881566e0 2023-10-06T03:48:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.128+0000 7f3f8d19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8815cf70 con 0x7f3f880ca300 2023-10-06T03:48:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.128+0000 7f3f8d19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8815d780 con 0x7f3f88156310 2023-10-06T03:48:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.128+0000 7f3f8c99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f881566e0 0x7f3f880c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.128+0000 7f3f8c99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f881566e0 0x7f3f880c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41026/0 (socket says 172.21.15.55:41026) 2023-10-06T03:48:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.128+0000 7f3f8c99c700 1 -- 172.21.15.55:0/4229135389 learned_addr learned my addr 172.21.15.55:0/4229135389 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.128+0000 7f3f877fe700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f88156310 0x7f3f88141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:18.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.128+0000 7f3f87fff700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f880ca300 0x7f3f8815ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:18.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.129+0000 7f3f8c99c700 1 -- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f88156310 msgr2=0x7f3f88141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:18.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.129+0000 7f3f8c99c700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f88156310 0x7f3f88141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.129+0000 7f3f8c99c700 1 -- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f880ca300 msgr2=0x7f3f8815ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:18.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.129+0000 7f3f8c99c700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f880ca300 0x7f3f8815ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.129+0000 7f3f8c99c700 1 -- 172.21.15.55:0/4229135389 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f780094f0 con 0x7f3f881566e0 2023-10-06T03:48:18.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.129+0000 7f3f877fe700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f88156310 0x7f3f88141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:18.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.129+0000 7f3f8c99c700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f881566e0 0x7f3f880c51e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3f80007f20 tx=0x7f3f80009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:18.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.129+0000 7f3f6ffff700 1 -- 172.21.15.55:0/4229135389 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000ebf0 con 0x7f3f881566e0 2023-10-06T03:48:18.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.129+0000 7f3f6ffff700 1 -- 172.21.15.55:0/4229135389 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8000ed70 con 0x7f3f881566e0 2023-10-06T03:48:18.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.129+0000 7f3f6ffff700 1 -- 172.21.15.55:0/4229135389 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80014670 con 0x7f3f881566e0 2023-10-06T03:48:18.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.129+0000 7f3f8d19d700 1 -- 172.21.15.55:0/4229135389 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8815da60 con 0x7f3f881566e0 2023-10-06T03:48:18.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.129+0000 7f3f8d19d700 1 -- 172.21.15.55:0/4229135389 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8815e020 con 0x7f3f881566e0 2023-10-06T03:48:18.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.130+0000 7f3f8d19d700 1 -- 172.21.15.55:0/4229135389 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8815e540 con 0x7f3f881566e0 2023-10-06T03:48:18.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.131+0000 7f3f6ffff700 1 -- 172.21.15.55:0/4229135389 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3f800149e0 con 0x7f3f881566e0 2023-10-06T03:48:18.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.132+0000 7f3f6ffff700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3f74066f30 0x7f3f740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.132+0000 7f3f6ffff700 1 -- 172.21.15.55:0/4229135389 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f3f80084b30 con 0x7f3f881566e0 2023-10-06T03:48:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.133+0000 7f3f87fff700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3f74066f30 0x7f3f740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:18.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.133+0000 7f3f87fff700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3f74066f30 0x7f3f740693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f8806c000 tx=0x7f3f78017000 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:18.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.136+0000 7f3f6ffff700 1 -- 172.21.15.55:0/4229135389 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3f80051d70 con 0x7f3f881566e0 2023-10-06T03:48:18.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.419+0000 7f3f8d19d700 1 -- 172.21.15.55:0/4229135389 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3f8804c3c0 con 0x7f3f881566e0 2023-10-06T03:48:18.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.420+0000 7f3f6ffff700 1 -- 172.21.15.55:0/4229135389 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3f80055390 con 0x7f3f881566e0 2023-10-06T03:48:18.421 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:18.421 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:48:18.421 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2023-10-06T03:48:18.422 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.422 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2023-10-06T03:48:18.422 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9li2RaGRAAgRjtbw3h40vylEPjolzlpw==", 2023-10-06T03:48:18.422 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.422 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:18.422 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:48:18.422 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:48:18.422 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.422 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.422 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.423 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.423 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2023-10-06T03:48:18.423 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lXZu7LxAAu3QAtdDxtD/AuuqGG12sDw==", 2023-10-06T03:48:18.423 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.423 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:18.423 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:48:18.423 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:48:18.423 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.423 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.423 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.423 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.424 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2023-10-06T03:48:18.424 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lS22fJBAAZi6CfyLyJAFgIkaarad1mA==", 2023-10-06T03:48:18.424 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.424 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:18.424 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:48:18.424 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:48:18.424 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.424 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.424 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.424 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.424 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2023-10-06T03:48:18.424 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9ll0RIOhAAGtGdPv4Y2bge7xIv3eu2NQ==", 2023-10-06T03:48:18.425 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.425 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:18.425 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:48:18.425 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:48:18.425 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.425 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.425 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.425 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.425 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2023-10-06T03:48:18.425 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDHfx9lmTeaDRAAuycc9i7fPj8xrDkuZCAGVg==", 2023-10-06T03:48:18.425 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.425 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:18.426 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:18.426 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.426 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.426 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.426 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.426 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2023-10-06T03:48:18.426 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDKfx9lxOdXEhAA9S8tdOvhnmBeEbutqnUsZg==", 2023-10-06T03:48:18.426 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.426 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:18.426 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:18.426 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.427 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.427 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.427 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.427 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2023-10-06T03:48:18.427 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDNfx9l76I+GRAAjaj0gM7An9BCyGmfQgaDFw==", 2023-10-06T03:48:18.427 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.427 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:18.427 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:18.427 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.427 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.427 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.427 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.427 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2023-10-06T03:48:18.428 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDQfx9lsPK+HxAAzQCV0EuBenqOkUpiQW9uog==", 2023-10-06T03:48:18.428 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.428 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:18.428 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:18.428 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.428 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.428 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.428 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.428 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2023-10-06T03:48:18.428 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDWfx9lSHNrMxAAIn7awDmljO2cRtKyDalWnw==", 2023-10-06T03:48:18.428 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.429 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:18.429 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:18.429 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.429 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.429 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.429 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.429 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2023-10-06T03:48:18.429 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDZfx9lFoWuOBAAhzQOt3YChvuv0MhrspOjPw==", 2023-10-06T03:48:18.429 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.429 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:18.429 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:18.429 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.430 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.430 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.430 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.430 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2023-10-06T03:48:18.430 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDcfx9lB8iSOhAAwamSgGIXZ8owFE4FdbOOBw==", 2023-10-06T03:48:18.430 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.430 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:18.430 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:18.430 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.430 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.430 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.430 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.431 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2023-10-06T03:48:18.431 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDgfx9lUb3mABAAJ75UO2UQg2r+l+tDYVn0Dg==", 2023-10-06T03:48:18.431 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.431 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:18.431 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:18.431 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.431 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.431 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.431 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.431 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2023-10-06T03:48:18.431 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lfNvCBxAA1ZJq4XsWTZHVY0u0NdOqIw==", 2023-10-06T03:48:18.431 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.432 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:18.432 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2023-10-06T03:48:18.432 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:48:18.432 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.432 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.432 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.432 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2023-10-06T03:48:18.432 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lnvkoFhAAafxsOmItxmwUr1GJrGYJ6g==", 2023-10-06T03:48:18.432 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.432 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:18.432 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:48:18.432 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:48:18.433 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:48:18.433 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.433 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.433 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.433 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2023-10-06T03:48:18.433 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9l+9b4DhAAc5XHdikWbXsEblSE3FGScQ==", 2023-10-06T03:48:18.433 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.433 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:18.433 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:48:18.433 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:48:18.433 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:48:18.433 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.434 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.434 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.434 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2023-10-06T03:48:18.434 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9luSXrHBAAg6Ufa1Ep/EOgsQtuOPQgYg==", 2023-10-06T03:48:18.434 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.434 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:18.434 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:48:18.434 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:48:18.434 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:48:18.434 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.434 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.435 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.435 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2023-10-06T03:48:18.435 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC+fx9lEb50FxAAt0Oq28gsXtQvol7TV8trgg==", 2023-10-06T03:48:18.435 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.435 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:48:18.435 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:48:18.435 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:48:18.435 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.435 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.435 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.436 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.436 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2023-10-06T03:48:18.436 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lQAANEhAAsNiWPqCNEyUS3IELXSC1Cw==", 2023-10-06T03:48:18.436 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.436 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:48:18.436 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T03:48:18.436 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.436 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.436 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:18.436 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:18.436 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2023-10-06T03:48:18.437 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lCwU2ABAAMijxXd8hwgk3XzcdP4NZyg==", 2023-10-06T03:48:18.437 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:18.437 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:48:18.437 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T03:48:18.437 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:18.437 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.437 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:18.437 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2023-10-06T03:48:18.437 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:48:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.422+0000 7f3f8d19d700 1 -- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3f74066f30 msgr2=0x7f3f740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.422+0000 7f3f8d19d700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3f74066f30 0x7f3f740693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3f8806c000 tx=0x7f3f78017000 comp rx=0 tx=0).stop 2023-10-06T03:48:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.422+0000 7f3f8d19d700 1 -- 172.21.15.55:0/4229135389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f881566e0 msgr2=0x7f3f880c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.422+0000 7f3f8d19d700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f881566e0 0x7f3f880c51e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3f80007f20 tx=0x7f3f80009860 comp rx=0 tx=0).stop 2023-10-06T03:48:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.422+0000 7f3f8d19d700 1 -- 172.21.15.55:0/4229135389 shutdown_connections 2023-10-06T03:48:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.422+0000 7f3f8d19d700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3f74066f30 0x7f3f740693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.422+0000 7f3f8d19d700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3f881566e0 0x7f3f880c51e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.422+0000 7f3f8d19d700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3f880ca300 0x7f3f8815ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.422+0000 7f3f8d19d700 1 --2- 172.21.15.55:0/4229135389 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3f88156310 0x7f3f88141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.422+0000 7f3f8d19d700 1 -- 172.21.15.55:0/4229135389 >> 172.21.15.55:0/4229135389 conn(0x7f3f880cc1f0 msgr2=0x7f3f88069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.422+0000 7f3f8d19d700 1 -- 172.21.15.55:0/4229135389 shutdown_connections 2023-10-06T03:48:18.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.422+0000 7f3f8d19d700 1 -- 172.21.15.55:0/4229135389 wait complete. 2023-10-06T03:48:18.441 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:48:18.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.626+0000 7f500559d700 1 Processor -- start 2023-10-06T03:48:18.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.626+0000 7f500559d700 1 -- start start 2023-10-06T03:48:18.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.628+0000 7f500559d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5000156310 0x7f50000c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:18.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.628+0000 7f500559d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50001566e0 0x7f50000ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:18.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.628+0000 7f500559d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50000ca860 con 0x7f50000ca300 2023-10-06T03:48:18.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.628+0000 7f500559d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50000cab40 con 0x7f5000156310 2023-10-06T03:48:18.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.628+0000 7f500559d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50000cae20 con 0x7f50001566e0 2023-10-06T03:48:18.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.628+0000 7f4ffffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5000156310 0x7f50000c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:18.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.628+0000 7f5004d9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f50000ca300 0x7f50000c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45916/0 (socket says 172.21.15.55:45916) 2023-10-06T03:48:18.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.628+0000 7f5004d9c700 1 -- 172.21.15.55:0/4055679488 learned_addr learned my addr 172.21.15.55:0/4055679488 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:18.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.628+0000 7f4fff7fe700 1 --2- 172.21.15.55:0/4055679488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50001566e0 0x7f50000ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.628+0000 7f4ffeffd700 1 -- 172.21.15.55:0/4055679488 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592299354 0 0) 0x7f50000ca860 con 0x7f50000ca300 2023-10-06T03:48:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.628+0000 7f4ffffff700 1 -- 172.21.15.55:0/4055679488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50001566e0 msgr2=0x7f50000ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.628+0000 7f4ffffff700 1 --2- 172.21.15.55:0/4055679488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50001566e0 0x7f50000ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.628+0000 7f4ffffff700 1 -- 172.21.15.55:0/4055679488 >> v1:172.21.15.55:6789/0 conn(0x7f50000ca300 legacy=0x7f50000c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.629+0000 7f4ffffff700 1 -- 172.21.15.55:0/4055679488 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50000cb100 con 0x7f5000156310 2023-10-06T03:48:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.629+0000 7f4ffffff700 1 --2- 172.21.15.55:0/4055679488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5000156310 0x7f50000c9dc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00098e0 tx=0x7f4ff00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479ccbda6fa50cb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.629+0000 7f4ffeffd700 1 -- 172.21.15.55:0/4055679488 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff000b640 con 0x7f5000156310 2023-10-06T03:48:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.629+0000 7f4ffeffd700 1 -- 172.21.15.55:0/4055679488 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff000b7a0 con 0x7f5000156310 2023-10-06T03:48:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.629+0000 7f4ffeffd700 1 -- 172.21.15.55:0/4055679488 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff000bb40 con 0x7f5000156310 2023-10-06T03:48:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.630+0000 7f500559d700 1 -- 172.21.15.55:0/4055679488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5000156310 msgr2=0x7f50000c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:18.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.630+0000 7f500559d700 1 --2- 172.21.15.55:0/4055679488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5000156310 0x7f50000c9dc0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00098e0 tx=0x7f4ff00091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.630+0000 7f500559d700 1 -- 172.21.15.55:0/4055679488 shutdown_connections 2023-10-06T03:48:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.630+0000 7f500559d700 1 --2- 172.21.15.55:0/4055679488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5000156310 0x7f50000c9dc0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.630+0000 7f500559d700 1 --2- 172.21.15.55:0/4055679488 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50001566e0 0x7f50000ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.630+0000 7f500559d700 1 -- 172.21.15.55:0/4055679488 >> 172.21.15.55:0/4055679488 conn(0x7f50000cc1f0 msgr2=0x7f50000cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.630+0000 7f500559d700 1 -- 172.21.15.55:0/4055679488 shutdown_connections 2023-10-06T03:48:18.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.630+0000 7f500559d700 1 -- 172.21.15.55:0/4055679488 wait complete. 2023-10-06T03:48:18.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.631+0000 7f500559d700 1 Processor -- start 2023-10-06T03:48:18.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.631+0000 7f500559d700 1 -- start start 2023-10-06T03:48:18.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.632+0000 7f500559d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50000ca300 0x7f50000c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:18.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.632+0000 7f500559d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5000156310 0x7f5000168600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:18.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.632+0000 7f500559d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50001566e0 0x7f5000159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:18.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.632+0000 7f500559d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5000168b40 con 0x7f5000156310 2023-10-06T03:48:18.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.632+0000 7f500559d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50000c83c0 con 0x7f50000ca300 2023-10-06T03:48:18.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.632+0000 7f500559d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50000c7840 con 0x7f50001566e0 2023-10-06T03:48:18.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.633+0000 7f4fff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5000156310 0x7f5000168600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:18.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.633+0000 7f4fff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5000156310 0x7f5000168600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41048/0 (socket says 172.21.15.55:41048) 2023-10-06T03:48:18.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.633+0000 7f4fff7fe700 1 -- 172.21.15.55:0/2539965339 learned_addr learned my addr 172.21.15.55:0/2539965339 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:18.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.633+0000 7f4ffffff700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50000ca300 0x7f50000c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:18.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.633+0000 7f5004d9c700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50001566e0 0x7f5000159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:18.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.633+0000 7f4fff7fe700 1 -- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50001566e0 msgr2=0x7f5000159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:18.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.633+0000 7f4fff7fe700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50001566e0 0x7f5000159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.633+0000 7f4fff7fe700 1 -- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50000ca300 msgr2=0x7f50000c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:18.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.633+0000 7f4fff7fe700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50000ca300 0x7f50000c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.633+0000 7f4fff7fe700 1 -- 172.21.15.55:0/2539965339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff0009420 con 0x7f5000156310 2023-10-06T03:48:18.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.633+0000 7f4ffffff700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50000ca300 0x7f50000c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:18.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.633+0000 7f4fff7fe700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5000156310 0x7f5000168600 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400ec50 tx=0x7f4ff40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:18.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.633+0000 7f4fe7fff700 1 -- 172.21.15.55:0/2539965339 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff400c9d0 con 0x7f5000156310 2023-10-06T03:48:18.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.634+0000 7f4fe7fff700 1 -- 172.21.15.55:0/2539965339 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff400cb30 con 0x7f5000156310 2023-10-06T03:48:18.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.634+0000 7f500559d700 1 -- 172.21.15.55:0/2539965339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50000c7b20 con 0x7f5000156310 2023-10-06T03:48:18.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.634+0000 7f4fe7fff700 1 -- 172.21.15.55:0/2539965339 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4013470 con 0x7f5000156310 2023-10-06T03:48:18.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.634+0000 7f500559d700 1 -- 172.21.15.55:0/2539965339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50000c80e0 con 0x7f5000156310 2023-10-06T03:48:18.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.635+0000 7f500559d700 1 -- 172.21.15.55:0/2539965339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500004c3c0 con 0x7f5000156310 2023-10-06T03:48:18.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.636+0000 7f4fe7fff700 1 -- 172.21.15.55:0/2539965339 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ff4013820 con 0x7f5000156310 2023-10-06T03:48:18.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.636+0000 7f4fe7fff700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4fec066f30 0x7f4fec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:18.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.637+0000 7f4fe7fff700 1 -- 172.21.15.55:0/2539965339 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f4ff4084560 con 0x7f5000156310 2023-10-06T03:48:18.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.637+0000 7f4ffffff700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4fec066f30 0x7f4fec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:18.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.638+0000 7f4ffffff700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4fec066f30 0x7f4fec0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00093f0 tx=0x7f4ff0017000 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:18.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.641+0000 7f4fe7fff700 1 -- 172.21.15.55:0/2539965339 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ff4051840 con 0x7f5000156310 2023-10-06T03:48:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.846+0000 7f500559d700 1 -- 172.21.15.55:0/2539965339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f500006ae40 con 0x7f5000156310 2023-10-06T03:48:18.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.846+0000 7f4fe7fff700 1 -- 172.21.15.55:0/2539965339 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+14293 (secure 0 0 0) 0x7f4ff4054e60 con 0x7f5000156310 2023-10-06T03:48:18.848 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:18.848 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":76,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:48:17.917568+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:48:18.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.848+0000 7f500559d700 1 -- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4fec066f30 msgr2=0x7f4fec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:18.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.849+0000 7f500559d700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4fec066f30 0x7f4fec0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00093f0 tx=0x7f4ff0017000 comp rx=0 tx=0).stop 2023-10-06T03:48:18.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.849+0000 7f500559d700 1 -- 172.21.15.55:0/2539965339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5000156310 msgr2=0x7f5000168600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:18.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.849+0000 7f500559d700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5000156310 0x7f5000168600 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400ec50 tx=0x7f4ff40091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.849+0000 7f500559d700 1 -- 172.21.15.55:0/2539965339 shutdown_connections 2023-10-06T03:48:18.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.849+0000 7f500559d700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4fec066f30 0x7f4fec0693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.849+0000 7f500559d700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50001566e0 0x7f5000159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.849+0000 7f500559d700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50000ca300 0x7f50000c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.849+0000 7f500559d700 1 --2- 172.21.15.55:0/2539965339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5000156310 0x7f5000168600 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:18.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.849+0000 7f500559d700 1 -- 172.21.15.55:0/2539965339 >> 172.21.15.55:0/2539965339 conn(0x7f50000cc1f0 msgr2=0x7f5000151f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:18.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.849+0000 7f500559d700 1 -- 172.21.15.55:0/2539965339 shutdown_connections 2023-10-06T03:48:18.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:18.849+0000 7f500559d700 1 -- 172.21.15.55:0/2539965339 wait complete. 2023-10-06T03:48:18.868 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T03:48:18.870 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:48:19.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.060+0000 7f4be099d700 1 Processor -- start 2023-10-06T03:48:19.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.061+0000 7f4be099d700 1 -- start start 2023-10-06T03:48:19.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.062+0000 7f4be099d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4bdc068aa0 0x7f4bdc0caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.062+0000 7f4be099d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bdc0ca920 con 0x7f4bdc06bfa0 2023-10-06T03:48:19.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.062+0000 7f4be099d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bdc06af00 con 0x7f4bdc159a70 2023-10-06T03:48:19.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.062+0000 7f4be099d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bdc06b1e0 con 0x7f4bdc068aa0 2023-10-06T03:48:19.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.062+0000 7f4bdaffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4bdc06bfa0 0x7f4bdc068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45920/0 (socket says 172.21.15.55:45920) 2023-10-06T03:48:19.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.062+0000 7f4bdaffd700 1 -- 172.21.15.55:0/115093753 learned_addr learned my addr 172.21.15.55:0/115093753 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:19.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bdbfff700 1 --2- 172.21.15.55:0/115093753 >> v2:172.21.15.163:3300/0 conn(0x7f4bdc068aa0 0x7f4bdc0caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406248944 0 0) 0x7f4bdc0ca920 con 0x7f4bdc06bfa0 2023-10-06T03:48:19.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc4003570 con 0x7f4bdc06bfa0 2023-10-06T03:48:19.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129094287 0 0) 0x7f4bdc06af00 con 0x7f4bdc159a70 2023-10-06T03:48:19.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bdc0ca920 con 0x7f4bdc159a70 2023-10-06T03:48:19.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1151325078 0 0) 0x7f4bc4003570 con 0x7f4bdc06bfa0 2023-10-06T03:48:19.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 >> v2:172.21.15.163:3300/0 conn(0x7f4bdc068aa0 msgr2=0x7f4bdc0caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:19.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 --2- 172.21.15.55:0/115093753 >> v2:172.21.15.163:3300/0 conn(0x7f4bdc068aa0 0x7f4bdc0caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 >> v1:172.21.15.163:6789/0 conn(0x7f4bdc159a70 legacy=0x7f4bdc0c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc06b4c0 con 0x7f4bdc06bfa0 2023-10-06T03:48:19.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f4bd00015f0 con 0x7f4bdc06bfa0 2023-10-06T03:48:19.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 >> v1:172.21.15.55:6789/0 conn(0x7f4bdc06bfa0 legacy=0x7f4bdc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 --2- 172.21.15.55:0/115093753 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bc4003c40 0x7f4bc4004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 --2- 172.21.15.55:0/115093753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc4004810 0x7f4bc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 --2- 172.21.15.55:0/115093753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bc4005210 0x7f4bc400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc06b1e0 con 0x7f4bc4005210 2023-10-06T03:48:19.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc400c1b0 con 0x7f4bc4004810 2023-10-06T03:48:19.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc400c370 con 0x7f4bc4003c40 2023-10-06T03:48:19.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.063+0000 7f4bdbfff700 1 --2- 172.21.15.55:0/115093753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bc4005210 0x7f4bc400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.064+0000 7f4bdaffd700 1 --2- 172.21.15.55:0/115093753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc4004810 0x7f4bc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.064+0000 7f4bdb7fe700 1 --2- 172.21.15.55:0/115093753 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bc4003c40 0x7f4bc4004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.064+0000 7f4bdbfff700 1 -- 172.21.15.55:0/115093753 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bc4003c40 msgr2=0x7f4bc4004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:19.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.064+0000 7f4bdbfff700 1 --2- 172.21.15.55:0/115093753 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bc4003c40 0x7f4bc4004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.064+0000 7f4bdbfff700 1 -- 172.21.15.55:0/115093753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc4004810 msgr2=0x7f4bc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:19.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.064+0000 7f4bdbfff700 1 --2- 172.21.15.55:0/115093753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc4004810 0x7f4bc4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.064+0000 7f4bdbfff700 1 -- 172.21.15.55:0/115093753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc4003570 con 0x7f4bc4005210 2023-10-06T03:48:19.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.064+0000 7f4bdbfff700 1 -- 172.21.15.55:0/115093753 reap_dead start 2023-10-06T03:48:19.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.064+0000 7f4bdbfff700 1 --2- 172.21.15.55:0/115093753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bc4005210 0x7f4bc400ba60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40094d0 tx=0x7f4bd400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78cd6f1e0b7658b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:19.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.064+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd400ea60 con 0x7f4bc4005210 2023-10-06T03:48:19.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.064+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd400f040 con 0x7f4bc4005210 2023-10-06T03:48:19.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.064+0000 7f4bda7fc700 1 -- 172.21.15.55:0/115093753 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd400ee30 con 0x7f4bc4005210 2023-10-06T03:48:19.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.065+0000 7f4be099d700 1 -- 172.21.15.55:0/115093753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bc4005210 msgr2=0x7f4bc400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.065+0000 7f4be099d700 1 --2- 172.21.15.55:0/115093753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bc4005210 0x7f4bc400ba60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40094d0 tx=0x7f4bd400b040 comp rx=0 tx=0).stop 2023-10-06T03:48:19.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.065+0000 7f4be099d700 1 -- 172.21.15.55:0/115093753 shutdown_connections 2023-10-06T03:48:19.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.065+0000 7f4be099d700 1 --2- 172.21.15.55:0/115093753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bc4005210 0x7f4bc400ba60 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.065+0000 7f4be099d700 1 -- 172.21.15.55:0/115093753 >> 172.21.15.55:0/115093753 conn(0x7f4bdc0cddc0 msgr2=0x7f4bdc0c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:19.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.065+0000 7f4be099d700 1 -- 172.21.15.55:0/115093753 shutdown_connections 2023-10-06T03:48:19.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.065+0000 7f4be099d700 1 -- 172.21.15.55:0/115093753 wait complete. 2023-10-06T03:48:19.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.066+0000 7f4be099d700 1 Processor -- start 2023-10-06T03:48:19.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.067+0000 7f4be099d700 1 -- start start 2023-10-06T03:48:19.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.067+0000 7f4be099d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc4005210 0x7f4bdc148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4be099d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bdc0b2a30 0x7f4bdc157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4be099d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bdc1532c0 0x7f4bdc1577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4be099d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc1560e0 con 0x7f4bdc1532c0 2023-10-06T03:48:19.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4be099d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc0b2f10 con 0x7f4bc4005210 2023-10-06T03:48:19.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4be099d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc156380 con 0x7f4bdc0b2a30 2023-10-06T03:48:19.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4bdbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bdc1532c0 0x7f4bdc1577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4bdbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bdc1532c0 0x7f4bdc1577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41070/0 (socket says 172.21.15.55:41070) 2023-10-06T03:48:19.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4bdbfff700 1 -- 172.21.15.55:0/1732997675 learned_addr learned my addr 172.21.15.55:0/1732997675 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:19.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4bdaffd700 1 --2- 172.21.15.55:0/1732997675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bdc0b2a30 0x7f4bdc157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4bdb7fe700 1 --2- 172.21.15.55:0/1732997675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc4005210 0x7f4bdc148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4bdbfff700 1 -- 172.21.15.55:0/1732997675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bdc0b2a30 msgr2=0x7f4bdc157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4bdbfff700 1 --2- 172.21.15.55:0/1732997675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bdc0b2a30 0x7f4bdc157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4bdbfff700 1 -- 172.21.15.55:0/1732997675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc4005210 msgr2=0x7f4bdc148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4bdbfff700 1 --2- 172.21.15.55:0/1732997675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc4005210 0x7f4bdc148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.068+0000 7f4bdbfff700 1 -- 172.21.15.55:0/1732997675 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc156660 con 0x7f4bdc1532c0 2023-10-06T03:48:19.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.069+0000 7f4bdbfff700 1 --2- 172.21.15.55:0/1732997675 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bdc1532c0 0x7f4bdc1577b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400afd0 tx=0x7f4bd4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:19.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.069+0000 7f4bc37fe700 1 -- 172.21.15.55:0/1732997675 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd400ee80 con 0x7f4bdc1532c0 2023-10-06T03:48:19.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.069+0000 7f4bc37fe700 1 -- 172.21.15.55:0/1732997675 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd401e650 con 0x7f4bdc1532c0 2023-10-06T03:48:19.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.069+0000 7f4be099d700 1 -- 172.21.15.55:0/1732997675 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bdc152ed0 con 0x7f4bdc1532c0 2023-10-06T03:48:19.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.069+0000 7f4bc37fe700 1 -- 172.21.15.55:0/1732997675 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd401ea10 con 0x7f4bdc1532c0 2023-10-06T03:48:19.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.069+0000 7f4be099d700 1 -- 172.21.15.55:0/1732997675 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bdc149520 con 0x7f4bdc1532c0 2023-10-06T03:48:19.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.070+0000 7f4be099d700 1 -- 172.21.15.55:0/1732997675 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bdc15e5c0 con 0x7f4bdc1532c0 2023-10-06T03:48:19.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.071+0000 7f4bc37fe700 1 -- 172.21.15.55:0/1732997675 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4bd4003020 con 0x7f4bdc1532c0 2023-10-06T03:48:19.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.072+0000 7f4bc37fe700 1 --2- 172.21.15.55:0/1732997675 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4bc8066f30 0x7f4bc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.072+0000 7f4bc37fe700 1 -- 172.21.15.55:0/1732997675 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f4bd40861a0 con 0x7f4bdc1532c0 2023-10-06T03:48:19.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.072+0000 7f4bdb7fe700 1 --2- 172.21.15.55:0/1732997675 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4bc8066f30 0x7f4bc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.073+0000 7f4bdb7fe700 1 --2- 172.21.15.55:0/1732997675 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4bc8066f30 0x7f4bc80693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0b2550 tx=0x7f4bcc009040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:19.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.076+0000 7f4bc37fe700 1 -- 172.21.15.55:0/1732997675 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bd4053490 con 0x7f4bdc1532c0 2023-10-06T03:48:19.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.280+0000 7f4be099d700 1 -- 172.21.15.55:0/1732997675 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4bdc06b7a0 con 0x7f4bdc1532c0 2023-10-06T03:48:19.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.281+0000 7f4bc37fe700 1 -- 172.21.15.55:0/1732997675 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v76) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4bd4056ab0 con 0x7f4bdc1532c0 2023-10-06T03:48:19.282 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:48:19.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.283+0000 7f4be099d700 1 -- 172.21.15.55:0/1732997675 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4bc8066f30 msgr2=0x7f4bc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.283+0000 7f4be099d700 1 --2- 172.21.15.55:0/1732997675 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4bc8066f30 0x7f4bc80693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0b2550 tx=0x7f4bcc009040 comp rx=0 tx=0).stop 2023-10-06T03:48:19.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.283+0000 7f4be099d700 1 -- 172.21.15.55:0/1732997675 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bdc1532c0 msgr2=0x7f4bdc1577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.283+0000 7f4be099d700 1 --2- 172.21.15.55:0/1732997675 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bdc1532c0 0x7f4bdc1577b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400afd0 tx=0x7f4bd4019000 comp rx=0 tx=0).stop 2023-10-06T03:48:19.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.283+0000 7f4be099d700 1 -- 172.21.15.55:0/1732997675 shutdown_connections 2023-10-06T03:48:19.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.283+0000 7f4be099d700 1 --2- 172.21.15.55:0/1732997675 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4bc8066f30 0x7f4bc80693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.283+0000 7f4be099d700 1 --2- 172.21.15.55:0/1732997675 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bdc1532c0 0x7f4bdc1577b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.283+0000 7f4be099d700 1 --2- 172.21.15.55:0/1732997675 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bc4005210 0x7f4bdc148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.283+0000 7f4be099d700 1 --2- 172.21.15.55:0/1732997675 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bdc0b2a30 0x7f4bdc157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.283+0000 7f4be099d700 1 -- 172.21.15.55:0/1732997675 >> 172.21.15.55:0/1732997675 conn(0x7f4bdc0cddc0 msgr2=0x7f4bdc0cb260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:19.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.283+0000 7f4be099d700 1 -- 172.21.15.55:0/1732997675 shutdown_connections 2023-10-06T03:48:19.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.283+0000 7f4be099d700 1 -- 172.21.15.55:0/1732997675 wait complete. 2023-10-06T03:48:19.302 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:48:19.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.491+0000 7f730b59e700 1 Processor -- start 2023-10-06T03:48:19.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.491+0000 7f730b59e700 1 -- start start 2023-10-06T03:48:19.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.492+0000 7f730b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f730c158ae0 0x7f730c158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.492+0000 7f730b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f730c157940 0x7f730c06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.492+0000 7f730b59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730c0cad80 con 0x7f730c157570 2023-10-06T03:48:19.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.492+0000 7f730b59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f730c0b1dd0 con 0x7f730c157940 2023-10-06T03:48:19.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.492+0000 7f730b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730c0b1fc0 con 0x7f730c158ae0 2023-10-06T03:48:19.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.493+0000 7f730ad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f730c157940 0x7f730c06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.493+0000 7f730ad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f730c157940 0x7f730c06a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41072/0 (socket says 172.21.15.55:41072) 2023-10-06T03:48:19.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.493+0000 7f730ad9d700 1 -- 172.21.15.55:0/3062043068 learned_addr learned my addr 172.21.15.55:0/3062043068 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:19.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.493+0000 7f7309d9b700 1 --2- 172.21.15.55:0/3062043068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f730c158ae0 0x7f730c158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.493+0000 7f730ad9d700 1 -- 172.21.15.55:0/3062043068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f730c158ae0 msgr2=0x7f730c158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:19.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.493+0000 7f730ad9d700 1 --2- 172.21.15.55:0/3062043068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f730c158ae0 0x7f730c158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.493+0000 7f730ad9d700 1 -- 172.21.15.55:0/3062043068 >> v1:172.21.15.163:6789/0 conn(0x7f730c157570 legacy=0x7f730c0b16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.493+0000 7f730ad9d700 1 -- 172.21.15.55:0/3062043068 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730c0ca450 con 0x7f730c157940 2023-10-06T03:48:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.493+0000 7f730ad9d700 1 --2- 172.21.15.55:0/3062043068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f730c157940 0x7f730c06a7f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7304009f50 tx=0x7f7304009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e7601c73126ce2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.494+0000 7f730959a700 1 -- 172.21.15.55:0/3062043068 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730400e070 con 0x7f730c157940 2023-10-06T03:48:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.494+0000 7f730959a700 1 -- 172.21.15.55:0/3062043068 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730400b9c0 con 0x7f730c157940 2023-10-06T03:48:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.494+0000 7f730959a700 1 -- 172.21.15.55:0/3062043068 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73040092e0 con 0x7f730c157940 2023-10-06T03:48:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.494+0000 7f730b59e700 1 -- 172.21.15.55:0/3062043068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f730c157940 msgr2=0x7f730c06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.494+0000 7f730b59e700 1 --2- 172.21.15.55:0/3062043068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f730c157940 0x7f730c06a7f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7304009f50 tx=0x7f7304009840 comp rx=0 tx=0).stop 2023-10-06T03:48:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.494+0000 7f730b59e700 1 -- 172.21.15.55:0/3062043068 shutdown_connections 2023-10-06T03:48:19.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.494+0000 7f730b59e700 1 --2- 172.21.15.55:0/3062043068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f730c157940 0x7f730c06a7f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.494+0000 7f730b59e700 1 --2- 172.21.15.55:0/3062043068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f730c158ae0 0x7f730c158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.494+0000 7f730b59e700 1 -- 172.21.15.55:0/3062043068 >> 172.21.15.55:0/3062043068 conn(0x7f730c068390 msgr2=0x7f730c068790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:19.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.494+0000 7f730b59e700 1 -- 172.21.15.55:0/3062043068 shutdown_connections 2023-10-06T03:48:19.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.494+0000 7f730b59e700 1 -- 172.21.15.55:0/3062043068 wait complete. 2023-10-06T03:48:19.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.496+0000 7f730b59e700 1 Processor -- start 2023-10-06T03:48:19.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.496+0000 7f730b59e700 1 -- start start 2023-10-06T03:48:19.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.497+0000 7f730b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f730c157570 0x7f730c159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.497+0000 7f730b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f730c157940 0x7f730c13a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.497+0000 7f730b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f730c158ae0 0x7f730c13aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.497+0000 7f730b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c06f070 con 0x7f730c158ae0 2023-10-06T03:48:19.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.497+0000 7f730b59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c06e460 con 0x7f730c157570 2023-10-06T03:48:19.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.497+0000 7f730b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730c06e740 con 0x7f730c157940 2023-10-06T03:48:19.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.497+0000 7f730ad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f730c158ae0 0x7f730c13aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.497+0000 7f730ad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f730c158ae0 0x7f730c13aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41082/0 (socket says 172.21.15.55:41082) 2023-10-06T03:48:19.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.497+0000 7f730ad9d700 1 -- 172.21.15.55:0/4040539632 learned_addr learned my addr 172.21.15.55:0/4040539632 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:19.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.497+0000 7f7309d9b700 1 --2- 172.21.15.55:0/4040539632 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f730c157940 0x7f730c13a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.497+0000 7f730a59c700 1 --2- 172.21.15.55:0/4040539632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f730c157570 0x7f730c159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.498+0000 7f730ad9d700 1 -- 172.21.15.55:0/4040539632 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f730c157940 msgr2=0x7f730c13a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.498+0000 7f730ad9d700 1 --2- 172.21.15.55:0/4040539632 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f730c157940 0x7f730c13a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.498+0000 7f730ad9d700 1 -- 172.21.15.55:0/4040539632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f730c157570 msgr2=0x7f730c159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.498+0000 7f730ad9d700 1 --2- 172.21.15.55:0/4040539632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f730c157570 0x7f730c159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.498+0000 7f730ad9d700 1 -- 172.21.15.55:0/4040539632 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7304009ae0 con 0x7f730c158ae0 2023-10-06T03:48:19.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.498+0000 7f730ad9d700 1 --2- 172.21.15.55:0/4040539632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f730c158ae0 0x7f730c13aed0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7304009880 tx=0x7f7304018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:19.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.498+0000 7f72fa7fc700 1 -- 172.21.15.55:0/4040539632 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304008c50 con 0x7f730c158ae0 2023-10-06T03:48:19.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.498+0000 7f72fa7fc700 1 -- 172.21.15.55:0/4040539632 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7304008db0 con 0x7f730c158ae0 2023-10-06T03:48:19.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.498+0000 7f72fa7fc700 1 -- 172.21.15.55:0/4040539632 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304026690 con 0x7f730c158ae0 2023-10-06T03:48:19.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.498+0000 7f730b59e700 1 -- 172.21.15.55:0/4040539632 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730c06ea20 con 0x7f730c158ae0 2023-10-06T03:48:19.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.498+0000 7f730b59e700 1 -- 172.21.15.55:0/4040539632 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730c15a330 con 0x7f730c158ae0 2023-10-06T03:48:19.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.499+0000 7f730b59e700 1 -- 172.21.15.55:0/4040539632 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730c15a4f0 con 0x7f730c158ae0 2023-10-06T03:48:19.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.500+0000 7f72fa7fc700 1 -- 172.21.15.55:0/4040539632 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7304003040 con 0x7f730c158ae0 2023-10-06T03:48:19.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.501+0000 7f72fa7fc700 1 --2- 172.21.15.55:0/4040539632 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f72ec066f30 0x7f72ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.501+0000 7f72fa7fc700 1 -- 172.21.15.55:0/4040539632 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f7304085f10 con 0x7f730c158ae0 2023-10-06T03:48:19.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.502+0000 7f730a59c700 1 --2- 172.21.15.55:0/4040539632 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f72ec066f30 0x7f72ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.502+0000 7f730a59c700 1 --2- 172.21.15.55:0/4040539632 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f72ec066f30 0x7f72ec0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f72fc003860 tx=0x7f72fc009040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:19.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.505+0000 7f72fa7fc700 1 -- 172.21.15.55:0/4040539632 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7304053200 con 0x7f730c158ae0 2023-10-06T03:48:19.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.706+0000 7f730b59e700 1 -- 172.21.15.55:0/4040539632 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f730c0ca730 con 0x7f730c158ae0 2023-10-06T03:48:19.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.706+0000 7f72fa7fc700 1 -- 172.21.15.55:0/4040539632 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7304056820 con 0x7f730c158ae0 2023-10-06T03:48:19.708 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:48:19.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.708+0000 7f730b59e700 1 -- 172.21.15.55:0/4040539632 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f72ec066f30 msgr2=0x7f72ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.708+0000 7f730b59e700 1 --2- 172.21.15.55:0/4040539632 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f72ec066f30 0x7f72ec0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f72fc003860 tx=0x7f72fc009040 comp rx=0 tx=0).stop 2023-10-06T03:48:19.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.708+0000 7f730b59e700 1 -- 172.21.15.55:0/4040539632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f730c158ae0 msgr2=0x7f730c13aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.708+0000 7f730b59e700 1 --2- 172.21.15.55:0/4040539632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f730c158ae0 0x7f730c13aed0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7304009880 tx=0x7f7304018000 comp rx=0 tx=0).stop 2023-10-06T03:48:19.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.708+0000 7f730b59e700 1 -- 172.21.15.55:0/4040539632 shutdown_connections 2023-10-06T03:48:19.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.708+0000 7f730b59e700 1 --2- 172.21.15.55:0/4040539632 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f72ec066f30 0x7f72ec0693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.708+0000 7f730b59e700 1 --2- 172.21.15.55:0/4040539632 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f730c158ae0 0x7f730c13aed0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.708+0000 7f730b59e700 1 --2- 172.21.15.55:0/4040539632 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f730c157570 0x7f730c159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.708+0000 7f730b59e700 1 --2- 172.21.15.55:0/4040539632 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f730c157940 0x7f730c13a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.708+0000 7f730b59e700 1 -- 172.21.15.55:0/4040539632 >> 172.21.15.55:0/4040539632 conn(0x7f730c068390 msgr2=0x7f730c15d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:19.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.709+0000 7f730b59e700 1 -- 172.21.15.55:0/4040539632 shutdown_connections 2023-10-06T03:48:19.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.709+0000 7f730b59e700 1 -- 172.21.15.55:0/4040539632 wait complete. 2023-10-06T03:48:19.727 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:48:19.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.913+0000 7f111db9d700 1 Processor -- start 2023-10-06T03:48:19.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.913+0000 7f111db9d700 1 -- start start 2023-10-06T03:48:19.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.914+0000 7f111db9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1118159420 0x7f11180b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.914+0000 7f111db9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111815d230 0x7f111815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.914+0000 7f111db9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11180b2980 con 0x7f11181597f0 2023-10-06T03:48:19.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.914+0000 7f111db9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111815dbb0 con 0x7f1118159420 2023-10-06T03:48:19.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.914+0000 7f111db9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11180b11d0 con 0x7f111815d230 2023-10-06T03:48:19.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.915+0000 7f111d39c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111815d230 0x7f111815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.915+0000 7f111cb9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1118159420 0x7f11180b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.915+0000 7f111d39c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111815d230 0x7f111815d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:43860/0 (socket says 172.21.15.55:43860) 2023-10-06T03:48:19.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.915+0000 7f111d39c700 1 -- 172.21.15.55:0/1482478040 learned_addr learned my addr 172.21.15.55:0/1482478040 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:19.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.915+0000 7f111d39c700 1 -- 172.21.15.55:0/1482478040 >> v2:172.21.15.163:3300/0 conn(0x7f1118159420 msgr2=0x7f11180b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:19.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.915+0000 7f111d39c700 1 --2- 172.21.15.55:0/1482478040 >> v2:172.21.15.163:3300/0 conn(0x7f1118159420 0x7f11180b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.915+0000 7f111d39c700 1 -- 172.21.15.55:0/1482478040 >> v1:172.21.15.163:6789/0 conn(0x7f11181597f0 legacy=0x7f11180b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.915+0000 7f111d39c700 1 -- 172.21.15.55:0/1482478040 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11180b14b0 con 0x7f111815d230 2023-10-06T03:48:19.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.916+0000 7f111d39c700 1 --2- 172.21.15.55:0/1482478040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111815d230 0x7f111815d670 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f11100099b0 tx=0x7f1110009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaa56753f7b29683 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:19.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.921+0000 7f11177fe700 1 -- 172.21.15.55:0/1482478040 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110009ea0 con 0x7f111815d230 2023-10-06T03:48:19.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.921+0000 7f11177fe700 1 -- 172.21.15.55:0/1482478040 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111000b860 con 0x7f111815d230 2023-10-06T03:48:19.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.921+0000 7f11177fe700 1 -- 172.21.15.55:0/1482478040 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111000bc00 con 0x7f111815d230 2023-10-06T03:48:19.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.921+0000 7f111db9d700 1 -- 172.21.15.55:0/1482478040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111815d230 msgr2=0x7f111815d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.921+0000 7f111db9d700 1 --2- 172.21.15.55:0/1482478040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111815d230 0x7f111815d670 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f11100099b0 tx=0x7f1110009250 comp rx=0 tx=0).stop 2023-10-06T03:48:19.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.922+0000 7f111db9d700 1 -- 172.21.15.55:0/1482478040 shutdown_connections 2023-10-06T03:48:19.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.922+0000 7f111db9d700 1 --2- 172.21.15.55:0/1482478040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111815d230 0x7f111815d670 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.922+0000 7f111db9d700 1 --2- 172.21.15.55:0/1482478040 >> v2:172.21.15.163:3300/0 conn(0x7f1118159420 0x7f11180b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.922+0000 7f111db9d700 1 -- 172.21.15.55:0/1482478040 >> 172.21.15.55:0/1482478040 conn(0x7f11180d0660 msgr2=0x7f11180d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:19.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.922+0000 7f111db9d700 1 -- 172.21.15.55:0/1482478040 shutdown_connections 2023-10-06T03:48:19.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.922+0000 7f111db9d700 1 -- 172.21.15.55:0/1482478040 wait complete. 2023-10-06T03:48:19.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.923+0000 7f111db9d700 1 Processor -- start 2023-10-06T03:48:19.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.923+0000 7f111db9d700 1 -- start start 2023-10-06T03:48:19.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.924+0000 7f111db9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1118159420 0x7f111814f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.924+0000 7f111db9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11181597f0 0x7f1118150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.924+0000 7f111db9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111815d230 0x7f11181542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.924+0000 7f111db9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1118151db0 con 0x7f1118159420 2023-10-06T03:48:19.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.924+0000 7f111db9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11181511a0 con 0x7f111815d230 2023-10-06T03:48:19.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.924+0000 7f111db9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1118151480 con 0x7f11181597f0 2023-10-06T03:48:19.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.924+0000 7f111cb9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1118159420 0x7f111814f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.924+0000 7f111cb9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1118159420 0x7f111814f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41092/0 (socket says 172.21.15.55:41092) 2023-10-06T03:48:19.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.924+0000 7f111cb9b700 1 -- 172.21.15.55:0/2784654051 learned_addr learned my addr 172.21.15.55:0/2784654051 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:19.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.925+0000 7f1117fff700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11181597f0 0x7f1118150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.925+0000 7f111d39c700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111815d230 0x7f11181542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.925+0000 7f111cb9b700 1 -- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11181597f0 msgr2=0x7f1118150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.925+0000 7f111cb9b700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11181597f0 0x7f1118150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.925+0000 7f111cb9b700 1 -- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111815d230 msgr2=0x7f11181542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:19.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.925+0000 7f111cb9b700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111815d230 0x7f11181542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:19.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.925+0000 7f111cb9b700 1 -- 172.21.15.55:0/2784654051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11100094f0 con 0x7f1118159420 2023-10-06T03:48:19.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.925+0000 7f111d39c700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111815d230 0x7f11181542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:19.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.925+0000 7f1117fff700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11181597f0 0x7f1118150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:19.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.925+0000 7f111cb9b700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1118159420 0x7f111814f510 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f110c009ef0 tx=0x7f110c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:19.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.925+0000 7f1114ff9700 1 -- 172.21.15.55:0/2784654051 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c00cc50 con 0x7f1118159420 2023-10-06T03:48:19.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.925+0000 7f1114ff9700 1 -- 172.21.15.55:0/2784654051 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110c00f040 con 0x7f1118159420 2023-10-06T03:48:19.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.925+0000 7f1114ff9700 1 -- 172.21.15.55:0/2784654051 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c014480 con 0x7f1118159420 2023-10-06T03:48:19.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.925+0000 7f111db9d700 1 -- 172.21.15.55:0/2784654051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1118151760 con 0x7f1118159420 2023-10-06T03:48:19.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.926+0000 7f111db9d700 1 -- 172.21.15.55:0/2784654051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111814fa50 con 0x7f1118159420 2023-10-06T03:48:19.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.927+0000 7f111db9d700 1 -- 172.21.15.55:0/2784654051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1118151c00 con 0x7f1118159420 2023-10-06T03:48:19.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.927+0000 7f1114ff9700 1 -- 172.21.15.55:0/2784654051 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f110c0147f0 con 0x7f1118159420 2023-10-06T03:48:19.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.929+0000 7f1114ff9700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f10fc066f30 0x7f10fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:19.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.929+0000 7f1114ff9700 1 -- 172.21.15.55:0/2784654051 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f110c0850c0 con 0x7f1118159420 2023-10-06T03:48:19.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.929+0000 7f1117fff700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f10fc066f30 0x7f10fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:19.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.930+0000 7f1117fff700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f10fc066f30 0x7f10fc0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1118158b00 tx=0x7f1104009040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:19.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:19.933+0000 7f1114ff9700 1 -- 172.21.15.55:0/2784654051 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f110c052300 con 0x7f1118159420 2023-10-06T03:48:20.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.138+0000 7f111db9d700 1 -- 172.21.15.55:0/2784654051 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f111804c3c0 con 0x7f1118159420 2023-10-06T03:48:20.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.138+0000 7f1114ff9700 1 -- 172.21.15.55:0/2784654051 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) v1 ==== 84+0+126 (secure 0 0 0) 0x7f110c055920 con 0x7f1118159420 2023-10-06T03:48:20.140 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:20.141 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:48:20.141 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:20.141 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2023-10-06T03:48:20.141 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": "rbd" 2023-10-06T03:48:20.142 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:20.142 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:20.142 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 2, 2023-10-06T03:48:20.142 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2023-10-06T03:48:20.142 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:20.143 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:48:20.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.140+0000 7f111db9d700 1 -- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f10fc066f30 msgr2=0x7f10fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:20.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.140+0000 7f111db9d700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f10fc066f30 0x7f10fc0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1118158b00 tx=0x7f1104009040 comp rx=0 tx=0).stop 2023-10-06T03:48:20.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.140+0000 7f111db9d700 1 -- 172.21.15.55:0/2784654051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1118159420 msgr2=0x7f111814f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:20.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.140+0000 7f111db9d700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1118159420 0x7f111814f510 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f110c009ef0 tx=0x7f110c00b040 comp rx=0 tx=0).stop 2023-10-06T03:48:20.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.140+0000 7f111db9d700 1 -- 172.21.15.55:0/2784654051 shutdown_connections 2023-10-06T03:48:20.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.140+0000 7f111db9d700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f10fc066f30 0x7f10fc0693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:20.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.140+0000 7f111db9d700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111815d230 0x7f11181542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:20.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.140+0000 7f111db9d700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1118159420 0x7f111814f510 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:20.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.140+0000 7f111db9d700 1 --2- 172.21.15.55:0/2784654051 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11181597f0 0x7f1118150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:20.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.140+0000 7f111db9d700 1 -- 172.21.15.55:0/2784654051 >> 172.21.15.55:0/2784654051 conn(0x7f11180d0660 msgr2=0x7f1118156a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:20.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.141+0000 7f111db9d700 1 -- 172.21.15.55:0/2784654051 shutdown_connections 2023-10-06T03:48:20.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.141+0000 7f111db9d700 1 -- 172.21.15.55:0/2784654051 wait complete. 2023-10-06T03:48:20.160 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:48:20.161 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:48:20.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.351+0000 7f10bd99d700 1 Processor -- start 2023-10-06T03:48:20.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.351+0000 7f10bd99d700 1 -- start start 2023-10-06T03:48:20.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.352+0000 7f10bd99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10b80b0760 0x7f10b80b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:20.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.352+0000 7f10bd99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f10b80ace60 0x7f10b80ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:20.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.352+0000 7f10bd99d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b80b1110 con 0x7f10b8156350 2023-10-06T03:48:20.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.352+0000 7f10bd99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b80b0570 con 0x7f10b80ace60 2023-10-06T03:48:20.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.352+0000 7f10bd99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80af4d0 con 0x7f10b80b0760 2023-10-06T03:48:20.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.352+0000 7f10bd19c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f10b80ace60 0x7f10b80ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:20.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.352+0000 7f10bd19c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f10b80ace60 0x7f10b80ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41098/0 (socket says 172.21.15.55:41098) 2023-10-06T03:48:20.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.352+0000 7f10bd19c700 1 -- 172.21.15.55:0/3059180959 learned_addr learned my addr 172.21.15.55:0/3059180959 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:20.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.353+0000 7f10b7fff700 1 --2- 172.21.15.55:0/3059180959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10b80b0760 0x7f10b80b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:20.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.353+0000 7f10bd19c700 1 -- 172.21.15.55:0/3059180959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10b80b0760 msgr2=0x7f10b80b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:20.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.353+0000 7f10bd19c700 1 --2- 172.21.15.55:0/3059180959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10b80b0760 0x7f10b80b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:20.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.353+0000 7f10bd19c700 1 -- 172.21.15.55:0/3059180959 >> v1:172.21.15.163:6789/0 conn(0x7f10b8156350 legacy=0x7f10b8157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:20.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.353+0000 7f10bd19c700 1 -- 172.21.15.55:0/3059180959 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b80af7b0 con 0x7f10b80ace60 2023-10-06T03:48:20.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.353+0000 7f10bd19c700 1 --2- 172.21.15.55:0/3059180959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b80ace60 0x7f10b80ad2a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009f20 tx=0x7f10b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906777b8302e22da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:20.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.353+0000 7f10b77fe700 1 -- 172.21.15.55:0/3059180959 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b000e070 con 0x7f10b80ace60 2023-10-06T03:48:20.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.353+0000 7f10b77fe700 1 -- 172.21.15.55:0/3059180959 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b000b9d0 con 0x7f10b80ace60 2023-10-06T03:48:20.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.353+0000 7f10b77fe700 1 -- 172.21.15.55:0/3059180959 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b00092e0 con 0x7f10b80ace60 2023-10-06T03:48:20.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.354+0000 7f10bd99d700 1 -- 172.21.15.55:0/3059180959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b80ace60 msgr2=0x7f10b80ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:20.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.354+0000 7f10bd99d700 1 --2- 172.21.15.55:0/3059180959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b80ace60 0x7f10b80ad2a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009f20 tx=0x7f10b0009840 comp rx=0 tx=0).stop 2023-10-06T03:48:20.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.354+0000 7f10bd99d700 1 -- 172.21.15.55:0/3059180959 shutdown_connections 2023-10-06T03:48:20.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.354+0000 7f10bd99d700 1 --2- 172.21.15.55:0/3059180959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b80ace60 0x7f10b80ad2a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:20.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.354+0000 7f10bd99d700 1 --2- 172.21.15.55:0/3059180959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10b80b0760 0x7f10b80b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:20.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.354+0000 7f10bd99d700 1 -- 172.21.15.55:0/3059180959 >> 172.21.15.55:0/3059180959 conn(0x7f10b8068e20 msgr2=0x7f10b80cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:20.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.354+0000 7f10bd99d700 1 -- 172.21.15.55:0/3059180959 shutdown_connections 2023-10-06T03:48:20.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.354+0000 7f10bd99d700 1 -- 172.21.15.55:0/3059180959 wait complete. 2023-10-06T03:48:20.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.355+0000 7f10bd99d700 1 Processor -- start 2023-10-06T03:48:20.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.356+0000 7f10bd99d700 1 -- start start 2023-10-06T03:48:20.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.356+0000 7f10bd99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10b80ace60 0x7f10b8155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:20.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10bd99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b80b0760 0x7f10b80c1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:20.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10bd99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10b8156350 0x7f10b814f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:20.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10bd99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80c98d0 con 0x7f10b80b0760 2023-10-06T03:48:20.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10bd99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80c9bb0 con 0x7f10b8156350 2023-10-06T03:48:20.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10bd99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80c9e90 con 0x7f10b80ace60 2023-10-06T03:48:20.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10b7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b80b0760 0x7f10b80c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:20.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10b7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b80b0760 0x7f10b80c1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41100/0 (socket says 172.21.15.55:41100) 2023-10-06T03:48:20.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10b7fff700 1 -- 172.21.15.55:0/3027739136 learned_addr learned my addr 172.21.15.55:0/3027739136 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:20.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10bc99b700 1 --2- 172.21.15.55:0/3027739136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10b80ace60 0x7f10b8155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:20.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10b7fff700 1 -- 172.21.15.55:0/3027739136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10b80ace60 msgr2=0x7f10b8155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:20.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10b7fff700 1 --2- 172.21.15.55:0/3027739136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10b80ace60 0x7f10b8155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:20.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10b7fff700 1 -- 172.21.15.55:0/3027739136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10b8156350 msgr2=0x7f10b814f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:20.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10b7fff700 1 --2- 172.21.15.55:0/3027739136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10b8156350 0x7f10b814f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:20.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10b7fff700 1 -- 172.21.15.55:0/3027739136 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b0009ae0 con 0x7f10b80b0760 2023-10-06T03:48:20.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10bc99b700 1 --2- 172.21.15.55:0/3027739136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10b80ace60 0x7f10b8155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:20.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.357+0000 7f10b7fff700 1 --2- 172.21.15.55:0/3027739136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b80b0760 0x7f10b80c1070 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f10a400e930 tx=0x7f10a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:20.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.358+0000 7f10b4ff9700 1 -- 172.21.15.55:0/3027739136 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a400eed0 con 0x7f10b80b0760 2023-10-06T03:48:20.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.358+0000 7f10b4ff9700 1 -- 172.21.15.55:0/3027739136 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a400c850 con 0x7f10b80b0760 2023-10-06T03:48:20.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.358+0000 7f10b4ff9700 1 -- 172.21.15.55:0/3027739136 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a400cbf0 con 0x7f10b80b0760 2023-10-06T03:48:20.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.358+0000 7f10bd99d700 1 -- 172.21.15.55:0/3027739136 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b80ca170 con 0x7f10b80b0760 2023-10-06T03:48:20.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.358+0000 7f10bd99d700 1 -- 172.21.15.55:0/3027739136 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b81556c0 con 0x7f10b80b0760 2023-10-06T03:48:20.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.359+0000 7f10bd99d700 1 -- 172.21.15.55:0/3027739136 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b804c3c0 con 0x7f10b80b0760 2023-10-06T03:48:20.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.360+0000 7f10b4ff9700 1 -- 172.21.15.55:0/3027739136 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f10a4013470 con 0x7f10b80b0760 2023-10-06T03:48:20.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.361+0000 7f10b4ff9700 1 --2- 172.21.15.55:0/3027739136 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f109c066f30 0x7f109c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:20.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.361+0000 7f10b4ff9700 1 -- 172.21.15.55:0/3027739136 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f10a4084750 con 0x7f10b80b0760 2023-10-06T03:48:20.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.361+0000 7f10bc99b700 1 --2- 172.21.15.55:0/3027739136 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f109c066f30 0x7f109c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:20.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.362+0000 7f10bc99b700 1 --2- 172.21.15.55:0/3027739136 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f109c066f30 0x7f109c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0017a0 tx=0x7f10ac009040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:20.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.365+0000 7f10b4ff9700 1 -- 172.21.15.55:0/3027739136 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10a4051990 con 0x7f10b80b0760 2023-10-06T03:48:20.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.567+0000 7f10bd99d700 1 -- 172.21.15.55:0/3027739136 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f10b80afa90 con 0x7f10b80b0760 2023-10-06T03:48:20.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.932+0000 7f10b4ff9700 1 -- 172.21.15.55:0/3027739136 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7f10a4054fb0 con 0x7f10b80b0760 2023-10-06T03:48:20.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.935+0000 7f10bd99d700 1 -- 172.21.15.55:0/3027739136 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f109c066f30 msgr2=0x7f109c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:20.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.935+0000 7f10bd99d700 1 --2- 172.21.15.55:0/3027739136 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f109c066f30 0x7f109c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0017a0 tx=0x7f10ac009040 comp rx=0 tx=0).stop 2023-10-06T03:48:20.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.935+0000 7f10bd99d700 1 -- 172.21.15.55:0/3027739136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b80b0760 msgr2=0x7f10b80c1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:20.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.935+0000 7f10bd99d700 1 --2- 172.21.15.55:0/3027739136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b80b0760 0x7f10b80c1070 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f10a400e930 tx=0x7f10a40091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:20.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.935+0000 7f10bd99d700 1 -- 172.21.15.55:0/3027739136 shutdown_connections 2023-10-06T03:48:20.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.935+0000 7f10bd99d700 1 --2- 172.21.15.55:0/3027739136 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f109c066f30 0x7f109c0693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:20.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.935+0000 7f10bd99d700 1 --2- 172.21.15.55:0/3027739136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10b8156350 0x7f10b814f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:20.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.935+0000 7f10bd99d700 1 --2- 172.21.15.55:0/3027739136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10b80ace60 0x7f10b8155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:20.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.935+0000 7f10bd99d700 1 --2- 172.21.15.55:0/3027739136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10b80b0760 0x7f10b80c1070 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:20.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.935+0000 7f10bd99d700 1 -- 172.21.15.55:0/3027739136 >> 172.21.15.55:0/3027739136 conn(0x7f10b8068e20 msgr2=0x7f10b80704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:20.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.935+0000 7f10bd99d700 1 -- 172.21.15.55:0/3027739136 shutdown_connections 2023-10-06T03:48:20.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:20.935+0000 7f10bd99d700 1 -- 172.21.15.55:0/3027739136 wait complete. 2023-10-06T03:48:20.939 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2023-10-06T03:48:20.959 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:48:21.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.152+0000 7fb9cad9d700 1 Processor -- start 2023-10-06T03:48:21.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.152+0000 7fb9cad9d700 1 -- start start 2023-10-06T03:48:21.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.154+0000 7fb9cad9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9c41573f0 0x7fb9c406c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:21.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.154+0000 7fb9cad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb9c41577c0 0x7fb9c406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:21.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.154+0000 7fb9cad9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c415ad70 con 0x7fb9c4068390 2023-10-06T03:48:21.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.154+0000 7fb9cad9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c40c8610 con 0x7fb9c41577c0 2023-10-06T03:48:21.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.154+0000 7fb9cad9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c415a160 con 0x7fb9c41573f0 2023-10-06T03:48:21.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.154+0000 7fb9c959a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb9c41577c0 0x7fb9c406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:21.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.154+0000 7fb9c959a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb9c41577c0 0x7fb9c406cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41108/0 (socket says 172.21.15.55:41108) 2023-10-06T03:48:21.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.154+0000 7fb9c959a700 1 -- 172.21.15.55:0/1415363459 learned_addr learned my addr 172.21.15.55:0/1415363459 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:21.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.154+0000 7fb9c9d9b700 1 --2- 172.21.15.55:0/1415363459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9c41573f0 0x7fb9c406c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:21.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.154+0000 7fb9c959a700 1 -- 172.21.15.55:0/1415363459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9c41573f0 msgr2=0x7fb9c406c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:21.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.154+0000 7fb9c959a700 1 --2- 172.21.15.55:0/1415363459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9c41573f0 0x7fb9c406c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:21.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.154+0000 7fb9c959a700 1 -- 172.21.15.55:0/1415363459 >> v1:172.21.15.163:6789/0 conn(0x7fb9c4068390 legacy=0x7fb9c415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:21.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.154+0000 7fb9c959a700 1 -- 172.21.15.55:0/1415363459 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c415a440 con 0x7fb9c41577c0 2023-10-06T03:48:21.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.155+0000 7fb9c959a700 1 --2- 172.21.15.55:0/1415363459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9c41577c0 0x7fb9c406cbf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40099b0 tx=0x7fb9b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba6a8aa867fd50d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:21.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.155+0000 7fb9c8d99700 1 -- 172.21.15.55:0/1415363459 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b400b860 con 0x7fb9c41577c0 2023-10-06T03:48:21.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.155+0000 7fb9c8d99700 1 -- 172.21.15.55:0/1415363459 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b400b9c0 con 0x7fb9c41577c0 2023-10-06T03:48:21.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.155+0000 7fb9c8d99700 1 -- 172.21.15.55:0/1415363459 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b400bd60 con 0x7fb9c41577c0 2023-10-06T03:48:21.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.155+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1415363459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9c41577c0 msgr2=0x7fb9c406cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:21.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.155+0000 7fb9cad9d700 1 --2- 172.21.15.55:0/1415363459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9c41577c0 0x7fb9c406cbf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40099b0 tx=0x7fb9b4009250 comp rx=0 tx=0).stop 2023-10-06T03:48:21.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.156+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1415363459 shutdown_connections 2023-10-06T03:48:21.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.156+0000 7fb9cad9d700 1 --2- 172.21.15.55:0/1415363459 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9c41573f0 0x7fb9c406c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:21.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.156+0000 7fb9cad9d700 1 --2- 172.21.15.55:0/1415363459 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9c41577c0 0x7fb9c406cbf0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:21.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.156+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1415363459 >> 172.21.15.55:0/1415363459 conn(0x7fb9c40cddc0 msgr2=0x7fb9c40c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:21.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.156+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1415363459 shutdown_connections 2023-10-06T03:48:21.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.156+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1415363459 wait complete. 2023-10-06T03:48:21.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.157+0000 7fb9cad9d700 1 Processor -- start 2023-10-06T03:48:21.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.157+0000 7fb9cad9d700 1 -- start start 2023-10-06T03:48:21.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.158+0000 7fb9cad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9c4068390 0x7fb9c4151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:21.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.158+0000 7fb9cad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9c41573f0 0x7fb9c40b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:21.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.158+0000 7fb9cad9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9c41577c0 0x7fb9c40aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:21.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.158+0000 7fb9cad9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c41520c0 con 0x7fb9c41573f0 2023-10-06T03:48:21.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.158+0000 7fb9cad9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c40ada90 con 0x7fb9c4068390 2023-10-06T03:48:21.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.158+0000 7fb9cad9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c40ace80 con 0x7fb9c41577c0 2023-10-06T03:48:21.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.158+0000 7fb9c959a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9c41573f0 0x7fb9c40b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:21.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.158+0000 7fb9c959a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9c41573f0 0x7fb9c40b0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41112/0 (socket says 172.21.15.55:41112) 2023-10-06T03:48:21.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.158+0000 7fb9c959a700 1 -- 172.21.15.55:0/1975744159 learned_addr learned my addr 172.21.15.55:0/1975744159 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:21.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.159+0000 7fb9c9d9b700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9c4068390 0x7fb9c4151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:21.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.159+0000 7fb9c959a700 1 -- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9c41577c0 msgr2=0x7fb9c40aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:21.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.159+0000 7fb9ca59c700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9c41577c0 0x7fb9c40aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:21.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.159+0000 7fb9c959a700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9c41577c0 0x7fb9c40aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:21.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.159+0000 7fb9c959a700 1 -- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9c4068390 msgr2=0x7fb9c4151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:21.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.159+0000 7fb9c959a700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9c4068390 0x7fb9c4151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:21.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.159+0000 7fb9c959a700 1 -- 172.21.15.55:0/1975744159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b40094f0 con 0x7fb9c41573f0 2023-10-06T03:48:21.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.159+0000 7fb9c9d9b700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9c4068390 0x7fb9c4151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:21.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.159+0000 7fb9ca59c700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9c41577c0 0x7fb9c40aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:21.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.159+0000 7fb9c959a700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9c41573f0 0x7fb9c40b0a40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009290 tx=0x7fb9b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:21.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.159+0000 7fb9b9ffb700 1 -- 172.21.15.55:0/1975744159 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4024440 con 0x7fb9c41573f0 2023-10-06T03:48:21.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.159+0000 7fb9b9ffb700 1 -- 172.21.15.55:0/1975744159 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b40245c0 con 0x7fb9c41573f0 2023-10-06T03:48:21.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.159+0000 7fb9b9ffb700 1 -- 172.21.15.55:0/1975744159 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4024980 con 0x7fb9c41573f0 2023-10-06T03:48:21.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.159+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1975744159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c40ad160 con 0x7fb9c41573f0 2023-10-06T03:48:21.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.160+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1975744159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c40ad720 con 0x7fb9c41573f0 2023-10-06T03:48:21.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.161+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1975744159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c40aa8f0 con 0x7fb9c41573f0 2023-10-06T03:48:21.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.162+0000 7fb9b9ffb700 1 -- 172.21.15.55:0/1975744159 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb9b401a420 con 0x7fb9c41573f0 2023-10-06T03:48:21.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.163+0000 7fb9b9ffb700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb9a8066f30 0x7fb9a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:21.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.163+0000 7fb9b9ffb700 1 -- 172.21.15.55:0/1975744159 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5968+0+0 (secure 0 0 0) 0x7fb9b40867a0 con 0x7fb9c41573f0 2023-10-06T03:48:21.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.163+0000 7fb9c9d9b700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb9a8066f30 0x7fb9a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:21.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.164+0000 7fb9c9d9b700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb9a8066f30 0x7fb9a80693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0006fd0 tx=0x7fb9c0008040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:21.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.167+0000 7fb9b9ffb700 1 -- 172.21.15.55:0/1975744159 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9b4053910 con 0x7fb9c41573f0 2023-10-06T03:48:21.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.369+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1975744159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb9c415a720 con 0x7fb9c41573f0 2023-10-06T03:48:21.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.935+0000 7fb9b9ffb700 1 -- 172.21.15.55:0/1975744159 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb9b4056f30 con 0x7fb9c41573f0 2023-10-06T03:48:21.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.941+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb9a8066f30 msgr2=0x7fb9a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:21.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.941+0000 7fb9cad9d700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb9a8066f30 0x7fb9a80693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0006fd0 tx=0x7fb9c0008040 comp rx=0 tx=0).stop 2023-10-06T03:48:21.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.941+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1975744159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9c41573f0 msgr2=0x7fb9c40b0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:21.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.941+0000 7fb9cad9d700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9c41573f0 0x7fb9c40b0a40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009290 tx=0x7fb9b4017000 comp rx=0 tx=0).stop 2023-10-06T03:48:21.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.942+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1975744159 shutdown_connections 2023-10-06T03:48:21.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.942+0000 7fb9cad9d700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb9a8066f30 0x7fb9a80693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:21.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.942+0000 7fb9cad9d700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9c41577c0 0x7fb9c40aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:21.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.942+0000 7fb9cad9d700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9c4068390 0x7fb9c4151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:21.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.942+0000 7fb9cad9d700 1 --2- 172.21.15.55:0/1975744159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9c41573f0 0x7fb9c40b0a40 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:21.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.942+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1975744159 >> 172.21.15.55:0/1975744159 conn(0x7fb9c40cddc0 msgr2=0x7fb9c40cb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:21.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.942+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1975744159 shutdown_connections 2023-10-06T03:48:21.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:21.943+0000 7fb9cad9d700 1 -- 172.21.15.55:0/1975744159 wait complete. 2023-10-06T03:48:21.952 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2023-10-06T03:48:21.975 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:48:22.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.164+0000 7fa70579d700 1 Processor -- start 2023-10-06T03:48:22.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.165+0000 7fa70579d700 1 -- start start 2023-10-06T03:48:22.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.166+0000 7fa70579d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7001407e0 0x7fa70006a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:22.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.166+0000 7fa70579d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa700140bb0 0x7fa70006a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:22.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.166+0000 7fa70579d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7000c6820 con 0x7fa70006d4f0 2023-10-06T03:48:22.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.166+0000 7fa70579d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7000c6de0 con 0x7fa700140bb0 2023-10-06T03:48:22.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.166+0000 7fa70579d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa700141270 con 0x7fa7001407e0 2023-10-06T03:48:22.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.166+0000 7fa704f9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa70006d4f0 0x7fa70015b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45934/0 (socket says 172.21.15.55:45934) 2023-10-06T03:48:22.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.166+0000 7fa704f9c700 1 -- 172.21.15.55:0/1840087467 learned_addr learned my addr 172.21.15.55:0/1840087467 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:22.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.166+0000 7fa6ff7fe700 1 --2- 172.21.15.55:0/1840087467 >> v2:172.21.15.163:3300/0 conn(0x7fa700140bb0 0x7fa70006a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:22.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.166+0000 7fa6fffff700 1 --2- 172.21.15.55:0/1840087467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7001407e0 0x7fa70006a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:22.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 -- 172.21.15.55:0/1840087467 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951430100 0 0) 0x7fa7000c6820 con 0x7fa70006d4f0 2023-10-06T03:48:22.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 -- 172.21.15.55:0/1840087467 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e8003570 con 0x7fa70006d4f0 2023-10-06T03:48:22.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 -- 172.21.15.55:0/1840087467 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 662236825 0 0) 0x7fa6e8003570 con 0x7fa70006d4f0 2023-10-06T03:48:22.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 -- 172.21.15.55:0/1840087467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7001407e0 msgr2=0x7fa70006a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:22.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 --2- 172.21.15.55:0/1840087467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7001407e0 0x7fa70006a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:22.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 -- 172.21.15.55:0/1840087467 >> v2:172.21.15.163:3300/0 conn(0x7fa700140bb0 msgr2=0x7fa70006a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:22.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 --2- 172.21.15.55:0/1840087467 >> v2:172.21.15.163:3300/0 conn(0x7fa700140bb0 0x7fa70006a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:22.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 -- 172.21.15.55:0/1840087467 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70006acf0 con 0x7fa70006d4f0 2023-10-06T03:48:22.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 -- 172.21.15.55:0/1840087467 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fa6f80017f0 con 0x7fa70006d4f0 2023-10-06T03:48:22.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 -- 172.21.15.55:0/1840087467 >> v1:172.21.15.55:6789/0 conn(0x7fa70006d4f0 legacy=0x7fa70015b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:22.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 --2- 172.21.15.55:0/1840087467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8003a60 0x7fa6e8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:22.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 --2- 172.21.15.55:0/1840087467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6e8004810 0x7fa6e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:22.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 --2- 172.21.15.55:0/1840087467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7001407e0 0x7fa6e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:22.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 -- 172.21.15.55:0/1840087467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7000c6de0 con 0x7fa6e8003a60 2023-10-06T03:48:22.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 -- 172.21.15.55:0/1840087467 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa700141270 con 0x7fa6e8004810 2023-10-06T03:48:22.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6feffd700 1 -- 172.21.15.55:0/1840087467 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e800ba60 con 0x7fa7001407e0 2023-10-06T03:48:22.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6fffff700 1 --2- 172.21.15.55:0/1840087467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8003a60 0x7fa6e8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:22.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa6ff7fe700 1 --2- 172.21.15.55:0/1840087467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6e8004810 0x7fa6e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:22.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.167+0000 7fa704f9c700 1 --2- 172.21.15.55:0/1840087467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7001407e0 0x7fa6e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:22.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.168+0000 7fa6fffff700 1 -- 172.21.15.55:0/1840087467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7001407e0 msgr2=0x7fa6e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:22.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.168+0000 7fa6fffff700 1 --2- 172.21.15.55:0/1840087467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7001407e0 0x7fa6e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:22.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.168+0000 7fa6fffff700 1 -- 172.21.15.55:0/1840087467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6e8004810 msgr2=0x7fa6e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:22.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.168+0000 7fa6fffff700 1 --2- 172.21.15.55:0/1840087467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6e8004810 0x7fa6e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:22.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.168+0000 7fa6fffff700 1 -- 172.21.15.55:0/1840087467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e8003570 con 0x7fa6e8003a60 2023-10-06T03:48:22.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.168+0000 7fa6fffff700 1 --2- 172.21.15.55:0/1840087467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8003a60 0x7fa6e8003e70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009c30 tx=0x7fa6f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a3086dd0731bc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:22.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.168+0000 7fa6feffd700 1 -- 172.21.15.55:0/1840087467 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000e710 con 0x7fa6e8003a60 2023-10-06T03:48:22.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.168+0000 7fa6feffd700 1 -- 172.21.15.55:0/1840087467 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f000e870 con 0x7fa6e8003a60 2023-10-06T03:48:22.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.168+0000 7fa6feffd700 1 -- 172.21.15.55:0/1840087467 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000ec30 con 0x7fa6e8003a60 2023-10-06T03:48:22.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.169+0000 7fa70579d700 1 -- 172.21.15.55:0/1840087467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8003a60 msgr2=0x7fa6e8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:22.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.169+0000 7fa70579d700 1 --2- 172.21.15.55:0/1840087467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6e8003a60 0x7fa6e8003e70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009c30 tx=0x7fa6f00091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:22.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.169+0000 7fa704f9c700 1 -- 172.21.15.55:0/1840087467 reap_dead start 2023-10-06T03:48:22.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.169+0000 7fa70579d700 1 -- 172.21.15.55:0/1840087467 shutdown_connections 2023-10-06T03:48:22.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.169+0000 7fa70579d700 1 -- 172.21.15.55:0/1840087467 >> 172.21.15.55:0/1840087467 conn(0x7fa7000cc1e0 msgr2=0x7fa7000cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:22.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.169+0000 7fa70579d700 1 -- 172.21.15.55:0/1840087467 shutdown_connections 2023-10-06T03:48:22.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.169+0000 7fa70579d700 1 -- 172.21.15.55:0/1840087467 wait complete. 2023-10-06T03:48:22.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.170+0000 7fa70579d700 1 Processor -- start 2023-10-06T03:48:22.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.171+0000 7fa70579d700 1 -- start start 2023-10-06T03:48:22.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.171+0000 7fa70579d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6e8003a60 0x7fa7001639a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:22.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.171+0000 7fa70579d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa70015a950 0x7fa7001657a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:22.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.171+0000 7fa70579d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7001667e0 0x7fa70013ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:22.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.171+0000 7fa70579d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa700166c80 con 0x7fa70015a950 2023-10-06T03:48:22.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.171+0000 7fa70579d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa700167300 con 0x7fa6e8003a60 2023-10-06T03:48:22.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.171+0000 7fa70579d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7001675e0 con 0x7fa7001667e0 2023-10-06T03:48:22.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.172+0000 7fa6ff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa70015a950 0x7fa7001657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:22.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.172+0000 7fa6ff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa70015a950 0x7fa7001657a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41120/0 (socket says 172.21.15.55:41120) 2023-10-06T03:48:22.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.172+0000 7fa6ff7fe700 1 -- 172.21.15.55:0/220678879 learned_addr learned my addr 172.21.15.55:0/220678879 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:22.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.172+0000 7fa6fffff700 1 --2- 172.21.15.55:0/220678879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6e8003a60 0x7fa7001639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:22.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.172+0000 7fa704f9c700 1 --2- 172.21.15.55:0/220678879 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7001667e0 0x7fa70013ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:22.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.172+0000 7fa6ff7fe700 1 -- 172.21.15.55:0/220678879 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7001667e0 msgr2=0x7fa70013ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:22.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.172+0000 7fa6ff7fe700 1 --2- 172.21.15.55:0/220678879 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7001667e0 0x7fa70013ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:22.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.172+0000 7fa6ff7fe700 1 -- 172.21.15.55:0/220678879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6e8003a60 msgr2=0x7fa7001639a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:22.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.172+0000 7fa6ff7fe700 1 --2- 172.21.15.55:0/220678879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6e8003a60 0x7fa7001639a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:22.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.172+0000 7fa6ff7fe700 1 -- 172.21.15.55:0/220678879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7001678c0 con 0x7fa70015a950 2023-10-06T03:48:22.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.172+0000 7fa6ff7fe700 1 --2- 172.21.15.55:0/220678879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa70015a950 0x7fa7001657a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4011b40 tx=0x7fa6f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:22.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.173+0000 7fa6e7fff700 1 -- 172.21.15.55:0/220678879 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400f940 con 0x7fa70015a950 2023-10-06T03:48:22.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.173+0000 7fa6e7fff700 1 -- 172.21.15.55:0/220678879 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f400faa0 con 0x7fa70015a950 2023-10-06T03:48:22.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.173+0000 7fa6e7fff700 1 -- 172.21.15.55:0/220678879 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400fe20 con 0x7fa70015a950 2023-10-06T03:48:22.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.173+0000 7fa70579d700 1 -- 172.21.15.55:0/220678879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa700167ba0 con 0x7fa70015a950 2023-10-06T03:48:22.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.173+0000 7fa70579d700 1 -- 172.21.15.55:0/220678879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70013f260 con 0x7fa70015a950 2023-10-06T03:48:22.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.174+0000 7fa70579d700 1 -- 172.21.15.55:0/220678879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa700168040 con 0x7fa70015a950 2023-10-06T03:48:22.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.175+0000 7fa6e7fff700 1 -- 172.21.15.55:0/220678879 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa6f4014650 con 0x7fa70015a950 2023-10-06T03:48:22.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.176+0000 7fa6e7fff700 1 --2- 172.21.15.55:0/220678879 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa6ec066f30 0x7fa6ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:22.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.176+0000 7fa6e7fff700 1 -- 172.21.15.55:0/220678879 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6350+0+0 (secure 0 0 0) 0x7fa6f4018030 con 0x7fa70015a950 2023-10-06T03:48:22.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.177+0000 7fa6fffff700 1 --2- 172.21.15.55:0/220678879 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa6ec066f30 0x7fa6ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:22.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.177+0000 7fa6fffff700 1 --2- 172.21.15.55:0/220678879 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa6ec066f30 0x7fa6ec0693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00091e0 tx=0x7fa6f001a000 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:22.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.181+0000 7fa6e7fff700 1 -- 172.21.15.55:0/220678879 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6f40520c0 con 0x7fa70015a950 2023-10-06T03:48:22.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.427+0000 7fa70579d700 1 -- 172.21.15.55:0/220678879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa7001681f0 con 0x7fa70015a950 2023-10-06T03:48:22.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.972+0000 7fa6e7fff700 1 -- 172.21.15.55:0/220678879 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v31) v1 ==== 173+0+0 (secure 0 0 0) 0x7fa6f40220c0 con 0x7fa70015a950 2023-10-06T03:48:22.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.979+0000 7fa70579d700 1 -- 172.21.15.55:0/220678879 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa6ec066f30 msgr2=0x7fa6ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:22.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.980+0000 7fa70579d700 1 --2- 172.21.15.55:0/220678879 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa6ec066f30 0x7fa6ec0693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00091e0 tx=0x7fa6f001a000 comp rx=0 tx=0).stop 2023-10-06T03:48:22.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.980+0000 7fa70579d700 1 -- 172.21.15.55:0/220678879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa70015a950 msgr2=0x7fa7001657a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:22.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.980+0000 7fa70579d700 1 --2- 172.21.15.55:0/220678879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa70015a950 0x7fa7001657a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4011b40 tx=0x7fa6f40091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:22.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.980+0000 7fa70579d700 1 -- 172.21.15.55:0/220678879 shutdown_connections 2023-10-06T03:48:22.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.980+0000 7fa70579d700 1 --2- 172.21.15.55:0/220678879 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa6ec066f30 0x7fa6ec0693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:22.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.980+0000 7fa70579d700 1 --2- 172.21.15.55:0/220678879 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7001667e0 0x7fa70013ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:22.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.980+0000 7fa70579d700 1 --2- 172.21.15.55:0/220678879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6e8003a60 0x7fa7001639a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:22.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.980+0000 7fa70579d700 1 --2- 172.21.15.55:0/220678879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa70015a950 0x7fa7001657a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:22.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.980+0000 7fa70579d700 1 -- 172.21.15.55:0/220678879 >> 172.21.15.55:0/220678879 conn(0x7fa7000cc1e0 msgr2=0x7fa70006a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:22.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.982+0000 7fa70579d700 1 -- 172.21.15.55:0/220678879 shutdown_connections 2023-10-06T03:48:22.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:22.982+0000 7fa70579d700 1 -- 172.21.15.55:0/220678879 wait complete. 2023-10-06T03:48:22.988 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 9 and data pool 10 2023-10-06T03:48:23.006 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:48:23.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.193+0000 7efdb299d700 1 Processor -- start 2023-10-06T03:48:23.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.193+0000 7efdb299d700 1 -- start start 2023-10-06T03:48:23.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.194+0000 7efdb299d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efdac1597f0 0x7efdac0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:23.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.194+0000 7efdb299d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdac15d230 0x7efdac15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:23.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.194+0000 7efdb299d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdac15dbb0 con 0x7efdac159420 2023-10-06T03:48:23.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.194+0000 7efdb299d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdac0b11d0 con 0x7efdac1597f0 2023-10-06T03:48:23.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.194+0000 7efdb299d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdac0b14b0 con 0x7efdac15d230 2023-10-06T03:48:23.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.195+0000 7efdb119a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efdac1597f0 0x7efdac0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:23.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.195+0000 7efdb219c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdac15d230 0x7efdac15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:23.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.195+0000 7efdb199b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7efdac159420 0x7efdac0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:34314/0 (socket says 172.21.15.55:34314) 2023-10-06T03:48:23.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.195+0000 7efdb199b700 1 -- 172.21.15.55:0/3336782124 learned_addr learned my addr 172.21.15.55:0/3336782124 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:23.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.195+0000 7efdb119a700 1 -- 172.21.15.55:0/3336782124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdac15d230 msgr2=0x7efdac15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:23.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.195+0000 7efdb119a700 1 --2- 172.21.15.55:0/3336782124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdac15d230 0x7efdac15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.195+0000 7efdb119a700 1 -- 172.21.15.55:0/3336782124 >> v1:172.21.15.163:6789/0 conn(0x7efdac159420 legacy=0x7efdac0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:23.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.195+0000 7efdb119a700 1 -- 172.21.15.55:0/3336782124 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac0b1790 con 0x7efdac1597f0 2023-10-06T03:48:23.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.195+0000 7efdb219c700 1 --2- 172.21.15.55:0/3336782124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdac15d230 0x7efdac15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:23.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.196+0000 7efdb119a700 1 --2- 172.21.15.55:0/3336782124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdac1597f0 0x7efdac0b2270 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0098e0 tx=0x7efd9c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca349c345f276494 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:23.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.196+0000 7efdb0999700 1 -- 172.21.15.55:0/3336782124 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00b640 con 0x7efdac1597f0 2023-10-06T03:48:23.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.196+0000 7efdb0999700 1 -- 172.21.15.55:0/3336782124 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c00b7a0 con 0x7efdac1597f0 2023-10-06T03:48:23.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.196+0000 7efdb0999700 1 -- 172.21.15.55:0/3336782124 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00bb40 con 0x7efdac1597f0 2023-10-06T03:48:23.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.196+0000 7efdb299d700 1 -- 172.21.15.55:0/3336782124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdac1597f0 msgr2=0x7efdac0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:23.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.196+0000 7efdb299d700 1 --2- 172.21.15.55:0/3336782124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdac1597f0 0x7efdac0b2270 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0098e0 tx=0x7efd9c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.197+0000 7efdb299d700 1 -- 172.21.15.55:0/3336782124 shutdown_connections 2023-10-06T03:48:23.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.197+0000 7efdb299d700 1 --2- 172.21.15.55:0/3336782124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdac15d230 0x7efdac15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.197+0000 7efdb299d700 1 --2- 172.21.15.55:0/3336782124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdac1597f0 0x7efdac0b2270 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.197+0000 7efdb299d700 1 -- 172.21.15.55:0/3336782124 >> 172.21.15.55:0/3336782124 conn(0x7efdac0d0660 msgr2=0x7efdac0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:23.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.197+0000 7efdb299d700 1 -- 172.21.15.55:0/3336782124 shutdown_connections 2023-10-06T03:48:23.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.197+0000 7efdb299d700 1 -- 172.21.15.55:0/3336782124 wait complete. 2023-10-06T03:48:23.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.198+0000 7efdb299d700 1 Processor -- start 2023-10-06T03:48:23.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.198+0000 7efdb299d700 1 -- start start 2023-10-06T03:48:23.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.199+0000 7efdb299d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac159420 0x7efdac0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:23.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.199+0000 7efdb299d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdac1597f0 0x7efdac0c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:23.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.199+0000 7efdb299d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdac15d230 0x7efdac0c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:23.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.199+0000 7efdb299d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac0ca8b0 con 0x7efdac159420 2023-10-06T03:48:23.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.199+0000 7efdb299d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac068a80 con 0x7efdac1597f0 2023-10-06T03:48:23.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.199+0000 7efdb299d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdac0c8b50 con 0x7efdac15d230 2023-10-06T03:48:23.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.199+0000 7efdb199b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac159420 0x7efdac0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:23.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.199+0000 7efdb199b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac159420 0x7efdac0ca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48678/0 (socket says 172.21.15.55:48678) 2023-10-06T03:48:23.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.199+0000 7efdb199b700 1 -- 172.21.15.55:0/3983162856 learned_addr learned my addr 172.21.15.55:0/3983162856 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:23.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.199+0000 7efdb119a700 1 --2- 172.21.15.55:0/3983162856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdac1597f0 0x7efdac0c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:23.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.200+0000 7efdb219c700 1 --2- 172.21.15.55:0/3983162856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdac15d230 0x7efdac0c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:23.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.200+0000 7efdb199b700 1 -- 172.21.15.55:0/3983162856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdac15d230 msgr2=0x7efdac0c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:23.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.200+0000 7efdb199b700 1 --2- 172.21.15.55:0/3983162856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdac15d230 0x7efdac0c65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.200+0000 7efdb199b700 1 -- 172.21.15.55:0/3983162856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdac1597f0 msgr2=0x7efdac0c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:23.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.200+0000 7efdb199b700 1 --2- 172.21.15.55:0/3983162856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdac1597f0 0x7efdac0c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.200+0000 7efdb199b700 1 -- 172.21.15.55:0/3983162856 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda4009a20 con 0x7efdac159420 2023-10-06T03:48:23.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.200+0000 7efdb199b700 1 --2- 172.21.15.55:0/3983162856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac159420 0x7efdac0ca370 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efda800c7c0 tx=0x7efda8007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:23.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.200+0000 7efda1ffb700 1 -- 172.21.15.55:0/3983162856 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8005b30 con 0x7efdac159420 2023-10-06T03:48:23.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.200+0000 7efda1ffb700 1 -- 172.21.15.55:0/3983162856 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda8005cb0 con 0x7efdac159420 2023-10-06T03:48:23.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.200+0000 7efdb299d700 1 -- 172.21.15.55:0/3983162856 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9c009420 con 0x7efdac159420 2023-10-06T03:48:23.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.200+0000 7efda1ffb700 1 -- 172.21.15.55:0/3983162856 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda8007320 con 0x7efdac159420 2023-10-06T03:48:23.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.201+0000 7efdb299d700 1 -- 172.21.15.55:0/3983162856 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdac0c8840 con 0x7efdac159420 2023-10-06T03:48:23.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.202+0000 7efdb299d700 1 -- 172.21.15.55:0/3983162856 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdac04c3c0 con 0x7efdac159420 2023-10-06T03:48:23.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.203+0000 7efda1ffb700 1 -- 172.21.15.55:0/3983162856 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efda8007690 con 0x7efdac159420 2023-10-06T03:48:23.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.204+0000 7efda1ffb700 1 --2- 172.21.15.55:0/3983162856 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efd90066f30 0x7efd900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:23.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.204+0000 7efda1ffb700 1 -- 172.21.15.55:0/3983162856 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6450+0+0 (secure 0 0 0) 0x7efda8083c10 con 0x7efdac159420 2023-10-06T03:48:23.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.204+0000 7efdb119a700 1 --2- 172.21.15.55:0/3983162856 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efd90066f30 0x7efd900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:23.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.205+0000 7efdb119a700 1 --2- 172.21.15.55:0/3983162856 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efd90066f30 0x7efd900693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efdac159c70 tx=0x7efd9c017000 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:23.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.208+0000 7efda1ffb700 1 -- 172.21.15.55:0/3983162856 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efda8050bb0 con 0x7efdac159420 2023-10-06T03:48:23.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.411+0000 7efdb299d700 1 -- 172.21.15.55:0/3983162856 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdac0b37c0 con 0x7efdac159420 2023-10-06T03:48:23.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.413+0000 7efda1ffb700 1 -- 172.21.15.55:0/3983162856 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+17524 (secure 0 0 0) 0x7efda8017020 con 0x7efdac159420 2023-10-06T03:48:23.414 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:23.415 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":79,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:48:22.429094+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:48:20.608889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T03:48:21.410367+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:48:23.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.415+0000 7efdb299d700 1 -- 172.21.15.55:0/3983162856 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efd90066f30 msgr2=0x7efd900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:23.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.415+0000 7efdb299d700 1 --2- 172.21.15.55:0/3983162856 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efd90066f30 0x7efd900693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efdac159c70 tx=0x7efd9c017000 comp rx=0 tx=0).stop 2023-10-06T03:48:23.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.415+0000 7efdb299d700 1 -- 172.21.15.55:0/3983162856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac159420 msgr2=0x7efdac0ca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:23.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.415+0000 7efdb299d700 1 --2- 172.21.15.55:0/3983162856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac159420 0x7efdac0ca370 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efda800c7c0 tx=0x7efda8007f30 comp rx=0 tx=0).stop 2023-10-06T03:48:23.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.415+0000 7efdb299d700 1 -- 172.21.15.55:0/3983162856 shutdown_connections 2023-10-06T03:48:23.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.415+0000 7efdb299d700 1 --2- 172.21.15.55:0/3983162856 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efd90066f30 0x7efd900693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.415+0000 7efdb299d700 1 --2- 172.21.15.55:0/3983162856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efdac15d230 0x7efdac0c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.415+0000 7efdb299d700 1 --2- 172.21.15.55:0/3983162856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efdac159420 0x7efdac0ca370 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.415+0000 7efdb299d700 1 --2- 172.21.15.55:0/3983162856 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efdac1597f0 0x7efdac0c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.415+0000 7efdb299d700 1 -- 172.21.15.55:0/3983162856 >> 172.21.15.55:0/3983162856 conn(0x7efdac0d0660 msgr2=0x7efdac0b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:23.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.415+0000 7efdb299d700 1 -- 172.21.15.55:0/3983162856 shutdown_connections 2023-10-06T03:48:23.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.415+0000 7efdb299d700 1 -- 172.21.15.55:0/3983162856 wait complete. 2023-10-06T03:48:23.435 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:48:23.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.619+0000 7f29ab39d700 1 Processor -- start 2023-10-06T03:48:23.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.620+0000 7f29ab39d700 1 -- start start 2023-10-06T03:48:23.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.621+0000 7f29ab39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29a41595a0 0x7f29a40b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:23.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.621+0000 7f29ab39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a40b1410 con 0x7f29a40c6c60 2023-10-06T03:48:23.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.621+0000 7f29ab39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a4063230 con 0x7f29a40c8830 2023-10-06T03:48:23.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.621+0000 7f29ab39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a40b2b90 con 0x7f29a41595a0 2023-10-06T03:48:23.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.621+0000 7f29a9b9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f29a40c6c60 0x7f29a40c8120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55776/0 (socket says 172.21.15.55:55776) 2023-10-06T03:48:23.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.621+0000 7f29a9b9a700 1 -- 172.21.15.55:0/3689305211 learned_addr learned my addr 172.21.15.55:0/3689305211 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:23.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.621+0000 7f29aa39b700 1 --2- 172.21.15.55:0/3689305211 >> v2:172.21.15.55:3300/0 conn(0x7f29a41595a0 0x7f29a40b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:23.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.622+0000 7f29a9399700 1 -- 172.21.15.55:0/3689305211 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273104564 0 0) 0x7f29a40b1410 con 0x7f29a40c6c60 2023-10-06T03:48:23.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.622+0000 7f29a9399700 1 -- 172.21.15.55:0/3689305211 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2990003570 con 0x7f29a40c6c60 2023-10-06T03:48:23.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.622+0000 7f29aa39b700 1 -- 172.21.15.55:0/3689305211 >> v1:172.21.15.163:6789/0 conn(0x7f29a40c8830 legacy=0x7f29a4159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:23.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.622+0000 7f29aa39b700 1 -- 172.21.15.55:0/3689305211 >> v1:172.21.15.55:6789/0 conn(0x7f29a40c6c60 legacy=0x7f29a40c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:23.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.622+0000 7f29aa39b700 1 -- 172.21.15.55:0/3689305211 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a40634f0 con 0x7f29a41595a0 2023-10-06T03:48:23.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.622+0000 7f29aa39b700 1 --2- 172.21.15.55:0/3689305211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a41595a0 0x7f29a40b18a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f29a00098e0 tx=0x7f29a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed98250d891def5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.622+0000 7f29a9399700 1 -- 172.21.15.55:0/3689305211 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a000b640 con 0x7f29a41595a0 2023-10-06T03:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.622+0000 7f29a9399700 1 -- 172.21.15.55:0/3689305211 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a000b7a0 con 0x7f29a41595a0 2023-10-06T03:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.622+0000 7f29a9399700 1 -- 172.21.15.55:0/3689305211 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a000bb40 con 0x7f29a41595a0 2023-10-06T03:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.623+0000 7f29ab39d700 1 -- 172.21.15.55:0/3689305211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a41595a0 msgr2=0x7f29a40b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.623+0000 7f29ab39d700 1 --2- 172.21.15.55:0/3689305211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a41595a0 0x7f29a40b18a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f29a00098e0 tx=0x7f29a00091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.623+0000 7f29ab39d700 1 -- 172.21.15.55:0/3689305211 shutdown_connections 2023-10-06T03:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.623+0000 7f29ab39d700 1 --2- 172.21.15.55:0/3689305211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a41595a0 0x7f29a40b18a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.623+0000 7f29ab39d700 1 -- 172.21.15.55:0/3689305211 >> 172.21.15.55:0/3689305211 conn(0x7f29a406ddb0 msgr2=0x7f29a406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.623+0000 7f29ab39d700 1 -- 172.21.15.55:0/3689305211 shutdown_connections 2023-10-06T03:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.623+0000 7f29ab39d700 1 -- 172.21.15.55:0/3689305211 wait complete. 2023-10-06T03:48:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.624+0000 7f29ab39d700 1 Processor -- start 2023-10-06T03:48:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.625+0000 7f29ab39d700 1 -- start start 2023-10-06T03:48:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.625+0000 7f29ab39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29a40c6c60 0x7f29a415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.625+0000 7f29ab39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a40c8830 0x7f29a4154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29ab39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29a41595a0 0x7f29a415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:23.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29ab39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a415b2f0 con 0x7f29a40c8830 2023-10-06T03:48:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29ab39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a4159d10 con 0x7f29a41595a0 2023-10-06T03:48:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29ab39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a4159ff0 con 0x7f29a40c6c60 2023-10-06T03:48:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29a9b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a40c8830 0x7f29a4154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29a9b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a40c8830 0x7f29a4154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48702/0 (socket says 172.21.15.55:48702) 2023-10-06T03:48:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29a9b9a700 1 -- 172.21.15.55:0/642935595 learned_addr learned my addr 172.21.15.55:0/642935595 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29aa39b700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29a40c6c60 0x7f29a415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29aab9c700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29a41595a0 0x7f29a415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29a9b9a700 1 -- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29a40c6c60 msgr2=0x7f29a415adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29a9b9a700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29a40c6c60 0x7f29a415adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29a9b9a700 1 -- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29a41595a0 msgr2=0x7f29a415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29a9b9a700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29a41595a0 0x7f29a415d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29a9b9a700 1 -- 172.21.15.55:0/642935595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0009420 con 0x7f29a40c8830 2023-10-06T03:48:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.626+0000 7f29aab9c700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29a41595a0 0x7f29a415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.627+0000 7f29a9b9a700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a40c8830 0x7f29a4154510 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f299400c7c0 tx=0x7f2994007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.627+0000 7f299a7fc700 1 -- 172.21.15.55:0/642935595 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994005c00 con 0x7f29a40c8830 2023-10-06T03:48:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.627+0000 7f299a7fc700 1 -- 172.21.15.55:0/642935595 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2994005d60 con 0x7f29a40c8830 2023-10-06T03:48:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.627+0000 7f29ab39d700 1 -- 172.21.15.55:0/642935595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a415a2d0 con 0x7f29a40c8830 2023-10-06T03:48:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.627+0000 7f299a7fc700 1 -- 172.21.15.55:0/642935595 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994007300 con 0x7f29a40c8830 2023-10-06T03:48:23.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.627+0000 7f29ab39d700 1 -- 172.21.15.55:0/642935595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a415a910 con 0x7f29a40c8830 2023-10-06T03:48:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.628+0000 7f29ab39d700 1 -- 172.21.15.55:0/642935595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a415aaa0 con 0x7f29a40c8830 2023-10-06T03:48:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.629+0000 7f299a7fc700 1 -- 172.21.15.55:0/642935595 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f29940076b0 con 0x7f29a40c8830 2023-10-06T03:48:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.630+0000 7f299a7fc700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2988066f30 0x7f29880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:23.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.630+0000 7f299a7fc700 1 -- 172.21.15.55:0/642935595 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f2994083e00 con 0x7f29a40c8830 2023-10-06T03:48:23.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.630+0000 7f29aa39b700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2988066f30 0x7f29880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:23.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.631+0000 7f29aa39b700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2988066f30 0x7f29880693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f29a00093f0 tx=0x7f29a0017000 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:23.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.634+0000 7f299a7fc700 1 -- 172.21.15.55:0/642935595 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2994050ce0 con 0x7f29a40c8830 2023-10-06T03:48:23.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.836+0000 7f29ab39d700 1 -- 172.21.15.55:0/642935595 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29a404c3c0 con 0x7f29a40c8830 2023-10-06T03:48:23.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.837+0000 7f299a7fc700 1 -- 172.21.15.55:0/642935595 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+17524 (secure 0 0 0) 0x7f2994017070 con 0x7f29a40c8830 2023-10-06T03:48:23.839 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:23.839 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":79,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:48:22.429094+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:48:20.608889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T03:48:21.410367+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:48:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.840+0000 7f29ab39d700 1 -- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2988066f30 msgr2=0x7f29880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.840+0000 7f29ab39d700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2988066f30 0x7f29880693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f29a00093f0 tx=0x7f29a0017000 comp rx=0 tx=0).stop 2023-10-06T03:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.840+0000 7f29ab39d700 1 -- 172.21.15.55:0/642935595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a40c8830 msgr2=0x7f29a4154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.840+0000 7f29ab39d700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a40c8830 0x7f29a4154510 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f299400c7c0 tx=0x7f2994007dd0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.840+0000 7f29ab39d700 1 -- 172.21.15.55:0/642935595 shutdown_connections 2023-10-06T03:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.840+0000 7f29ab39d700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2988066f30 0x7f29880693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.840+0000 7f29ab39d700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29a41595a0 0x7f29a415d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.840+0000 7f29ab39d700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29a40c6c60 0x7f29a415adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.840+0000 7f29ab39d700 1 --2- 172.21.15.55:0/642935595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29a40c8830 0x7f29a4154510 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.840+0000 7f29ab39d700 1 -- 172.21.15.55:0/642935595 >> 172.21.15.55:0/642935595 conn(0x7f29a406ddb0 msgr2=0x7f29a40c7300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.840+0000 7f29ab39d700 1 -- 172.21.15.55:0/642935595 shutdown_connections 2023-10-06T03:48:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:23.840+0000 7f29ab39d700 1 -- 172.21.15.55:0/642935595 wait complete. 2023-10-06T03:48:23.859 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:48:24.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.049+0000 7f6e30d9d700 1 Processor -- start 2023-10-06T03:48:24.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.050+0000 7f6e30d9d700 1 -- start start 2023-10-06T03:48:24.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.051+0000 7f6e30d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6e2c0cb1a0 0x7f6e2c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:24.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.051+0000 7f6e30d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e2c15a8a0 0x7f6e2c06f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:24.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.051+0000 7f6e30d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6e2c06f5a0 0x7f6e2c0ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:24.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.051+0000 7f6e30d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e2c15b2e0 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.051+0000 7f6e30d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e2c06ebd0 con 0x7f6e2c0cb1a0 2023-10-06T03:48:24.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.051+0000 7f6e30d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c15ad20 con 0x7f6e2c15a8a0 2023-10-06T03:48:24.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.051+0000 7f6e2bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6e2c06f5a0 0x7f6e2c0ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:24.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.051+0000 7f6e2bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6e2c06f5a0 0x7f6e2c0ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48712/0 (socket says 172.21.15.55:48712) 2023-10-06T03:48:24.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.051+0000 7f6e2bfff700 1 -- 172.21.15.55:0/3226166191 learned_addr learned my addr 172.21.15.55:0/3226166191 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:24.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.052+0000 7f6e2b7fe700 1 --2- 172.21.15.55:0/3226166191 >> v2:172.21.15.163:3300/0 conn(0x7f6e2c0cb1a0 0x7f6e2c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:24.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.052+0000 7f6e2affd700 1 --2- 172.21.15.55:0/3226166191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e2c15a8a0 0x7f6e2c06f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:24.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.052+0000 7f6e2bfff700 1 -- 172.21.15.55:0/3226166191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e2c15a8a0 msgr2=0x7f6e2c06f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:24.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.052+0000 7f6e2bfff700 1 --2- 172.21.15.55:0/3226166191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e2c15a8a0 0x7f6e2c06f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:24.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.052+0000 7f6e2bfff700 1 -- 172.21.15.55:0/3226166191 >> v2:172.21.15.163:3300/0 conn(0x7f6e2c0cb1a0 msgr2=0x7f6e2c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:24.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.052+0000 7f6e2bfff700 1 --2- 172.21.15.55:0/3226166191 >> v2:172.21.15.163:3300/0 conn(0x7f6e2c0cb1a0 0x7f6e2c15a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:24.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.052+0000 7f6e2bfff700 1 -- 172.21.15.55:0/3226166191 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c15b000 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.052+0000 7f6e2affd700 1 --2- 172.21.15.55:0/3226166191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e2c15a8a0 0x7f6e2c06f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:24.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.052+0000 7f6e2bfff700 1 --2- 172.21.15.55:0/3226166191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e2c06f5a0 0x7f6e2c0ae0d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0cce10 tx=0x7f6e24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ce67de0ad36aed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:24.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.052+0000 7f6e2a7fc700 1 -- 172.21.15.55:0/3226166191 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2400e070 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.052+0000 7f6e2a7fc700 1 -- 172.21.15.55:0/3226166191 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2400bae0 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.052+0000 7f6e2a7fc700 1 -- 172.21.15.55:0/3226166191 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e240092e0 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.053+0000 7f6e30d9d700 1 -- 172.21.15.55:0/3226166191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e2c06f5a0 msgr2=0x7f6e2c0ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:24.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.053+0000 7f6e30d9d700 1 --2- 172.21.15.55:0/3226166191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e2c06f5a0 0x7f6e2c0ae0d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0cce10 tx=0x7f6e24009840 comp rx=0 tx=0).stop 2023-10-06T03:48:24.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.053+0000 7f6e30d9d700 1 -- 172.21.15.55:0/3226166191 shutdown_connections 2023-10-06T03:48:24.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.053+0000 7f6e30d9d700 1 --2- 172.21.15.55:0/3226166191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e2c06f5a0 0x7f6e2c0ae0d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:24.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.053+0000 7f6e30d9d700 1 --2- 172.21.15.55:0/3226166191 >> v2:172.21.15.163:3300/0 conn(0x7f6e2c0cb1a0 0x7f6e2c15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:24.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.053+0000 7f6e30d9d700 1 --2- 172.21.15.55:0/3226166191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e2c15a8a0 0x7f6e2c06f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:24.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.053+0000 7f6e30d9d700 1 -- 172.21.15.55:0/3226166191 >> 172.21.15.55:0/3226166191 conn(0x7f6e2c0cc1e0 msgr2=0x7f6e2c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:24.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.053+0000 7f6e30d9d700 1 -- 172.21.15.55:0/3226166191 shutdown_connections 2023-10-06T03:48:24.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.053+0000 7f6e30d9d700 1 -- 172.21.15.55:0/3226166191 wait complete. 2023-10-06T03:48:24.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.054+0000 7f6e30d9d700 1 Processor -- start 2023-10-06T03:48:24.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.055+0000 7f6e30d9d700 1 -- start start 2023-10-06T03:48:24.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.055+0000 7f6e30d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e2c06f5a0 0x7f6e2c0c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:24.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.055+0000 7f6e30d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e2c0cb1a0 0x7f6e2c0c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:24.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.055+0000 7f6e30d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e2c159610 0x7f6e2c0c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:24.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.055+0000 7f6e30d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0c47e0 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.055+0000 7f6e30d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0c4ac0 con 0x7f6e2c0cb1a0 2023-10-06T03:48:24.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.055+0000 7f6e30d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0c4da0 con 0x7f6e2c159610 2023-10-06T03:48:24.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.056+0000 7f6e2b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e2c06f5a0 0x7f6e2c0c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:24.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.056+0000 7f6e2b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e2c06f5a0 0x7f6e2c0c5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48722/0 (socket says 172.21.15.55:48722) 2023-10-06T03:48:24.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.056+0000 7f6e2b7fe700 1 -- 172.21.15.55:0/344532684 learned_addr learned my addr 172.21.15.55:0/344532684 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:24.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.056+0000 7f6e2affd700 1 --2- 172.21.15.55:0/344532684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e2c0cb1a0 0x7f6e2c0c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:24.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.056+0000 7f6e2bfff700 1 --2- 172.21.15.55:0/344532684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e2c159610 0x7f6e2c0c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:24.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.056+0000 7f6e2b7fe700 1 -- 172.21.15.55:0/344532684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e2c159610 msgr2=0x7f6e2c0c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:24.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.056+0000 7f6e2b7fe700 1 --2- 172.21.15.55:0/344532684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e2c159610 0x7f6e2c0c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:24.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.056+0000 7f6e2b7fe700 1 -- 172.21.15.55:0/344532684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e2c0cb1a0 msgr2=0x7f6e2c0c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:24.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.056+0000 7f6e2b7fe700 1 --2- 172.21.15.55:0/344532684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e2c0cb1a0 0x7f6e2c0c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:24.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.056+0000 7f6e2b7fe700 1 -- 172.21.15.55:0/344532684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e24009ae0 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.056+0000 7f6e2b7fe700 1 --2- 172.21.15.55:0/344532684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e2c06f5a0 0x7f6e2c0c5880 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00d890 tx=0x7f6e1c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:24.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.057+0000 7f6e137fe700 1 -- 172.21.15.55:0/344532684 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00b600 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.057+0000 7f6e137fe700 1 -- 172.21.15.55:0/344532684 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1c00b760 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.057+0000 7f6e137fe700 1 -- 172.21.15.55:0/344532684 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00bae0 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.057+0000 7f6e30d9d700 1 -- 172.21.15.55:0/344532684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2c0c5080 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.057+0000 7f6e30d9d700 1 -- 172.21.15.55:0/344532684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2c0c4460 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.058+0000 7f6e30d9d700 1 -- 172.21.15.55:0/344532684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2c0c56d0 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.059+0000 7f6e137fe700 1 -- 172.21.15.55:0/344532684 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e1c01e460 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.060+0000 7f6e137fe700 1 --2- 172.21.15.55:0/344532684 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6e18066f30 0x7f6e180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:24.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.060+0000 7f6e137fe700 1 -- 172.21.15.55:0/344532684 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f6e1c084fa0 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.060+0000 7f6e2affd700 1 --2- 172.21.15.55:0/344532684 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6e18066f30 0x7f6e180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:24.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.061+0000 7f6e2affd700 1 --2- 172.21.15.55:0/344532684 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6e18066f30 0x7f6e180693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0c90a0 tx=0x7f6e20008040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:24.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.064+0000 7f6e137fe700 1 -- 172.21.15.55:0/344532684 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e1c051e80 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.310+0000 7f6e30d9d700 1 -- 172.21.15.55:0/344532684 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6e2c06ceb0 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.976+0000 7f6e137fe700 1 -- 172.21.15.55:0/344532684 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6e1c0554a0 con 0x7f6e2c06f5a0 2023-10-06T03:48:24.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.979+0000 7f6e30d9d700 1 -- 172.21.15.55:0/344532684 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6e18066f30 msgr2=0x7f6e180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.979+0000 7f6e30d9d700 1 --2- 172.21.15.55:0/344532684 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6e18066f30 0x7f6e180693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c0c90a0 tx=0x7f6e20008040 comp rx=0 tx=0).stop 2023-10-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.979+0000 7f6e30d9d700 1 -- 172.21.15.55:0/344532684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e2c06f5a0 msgr2=0x7f6e2c0c5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.979+0000 7f6e30d9d700 1 --2- 172.21.15.55:0/344532684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e2c06f5a0 0x7f6e2c0c5880 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c00d890 tx=0x7f6e1c008040 comp rx=0 tx=0).stop 2023-10-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.979+0000 7f6e30d9d700 1 -- 172.21.15.55:0/344532684 shutdown_connections 2023-10-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.979+0000 7f6e30d9d700 1 --2- 172.21.15.55:0/344532684 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6e18066f30 0x7f6e180693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.979+0000 7f6e30d9d700 1 --2- 172.21.15.55:0/344532684 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e2c159610 0x7f6e2c0c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.979+0000 7f6e30d9d700 1 --2- 172.21.15.55:0/344532684 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e2c06f5a0 0x7f6e2c0c5880 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.979+0000 7f6e30d9d700 1 --2- 172.21.15.55:0/344532684 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e2c0cb1a0 0x7f6e2c0c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:24.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.979+0000 7f6e30d9d700 1 -- 172.21.15.55:0/344532684 >> 172.21.15.55:0/344532684 conn(0x7f6e2c0cc1e0 msgr2=0x7f6e2c0aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:24.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.979+0000 7f6e30d9d700 1 -- 172.21.15.55:0/344532684 shutdown_connections 2023-10-06T03:48:24.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:24.979+0000 7f6e30d9d700 1 -- 172.21.15.55:0/344532684 wait complete. 2023-10-06T03:48:25.000 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:48:25.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.194+0000 7f083ef9d700 1 Processor -- start 2023-10-06T03:48:25.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.194+0000 7f083ef9d700 1 -- start start 2023-10-06T03:48:25.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.195+0000 7f083ef9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f083815a3b0 0x7f083815a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:25.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.195+0000 7f083ef9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f08380aef00 0x7f083815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:25.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.195+0000 7f083ef9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08380aea70 con 0x7f083806e470 2023-10-06T03:48:25.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.195+0000 7f083ef9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08380ae480 con 0x7f083815a3b0 2023-10-06T03:48:25.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.195+0000 7f083ef9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f083815acc0 con 0x7f08380aef00 2023-10-06T03:48:25.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.196+0000 7f083e79c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f083806e470 0x7f08380c7280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55790/0 (socket says 172.21.15.55:55790) 2023-10-06T03:48:25.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.196+0000 7f083df9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f083815a3b0 0x7f083815a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:25.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.196+0000 7f083e79c700 1 -- 172.21.15.55:0/1624954602 learned_addr learned my addr 172.21.15.55:0/1624954602 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:25.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.196+0000 7f083d79a700 1 --2- 172.21.15.55:0/1624954602 >> v2:172.21.15.163:3300/0 conn(0x7f08380aef00 0x7f083815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:25.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.196+0000 7f083cf99700 1 -- 172.21.15.55:0/1624954602 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585076322 0 0) 0x7f08380aea70 con 0x7f083806e470 2023-10-06T03:48:25.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.196+0000 7f083df9b700 1 -- 172.21.15.55:0/1624954602 >> v2:172.21.15.163:3300/0 conn(0x7f08380aef00 msgr2=0x7f083815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:25.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.196+0000 7f083df9b700 1 --2- 172.21.15.55:0/1624954602 >> v2:172.21.15.163:3300/0 conn(0x7f08380aef00 0x7f083815d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:25.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.196+0000 7f083df9b700 1 -- 172.21.15.55:0/1624954602 >> v1:172.21.15.55:6789/0 conn(0x7f083806e470 legacy=0x7f08380c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:25.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.196+0000 7f083df9b700 1 -- 172.21.15.55:0/1624954602 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083815dd30 con 0x7f083815a3b0 2023-10-06T03:48:25.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.196+0000 7f083d79a700 1 --2- 172.21.15.55:0/1624954602 >> v2:172.21.15.163:3300/0 conn(0x7f08380aef00 0x7f083815d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:25.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.196+0000 7f083df9b700 1 --2- 172.21.15.55:0/1624954602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083815a3b0 0x7f083815a780 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f083806d7e0 tx=0x7f08340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899a8e4fdd826bd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:25.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.196+0000 7f083cf99700 1 -- 172.21.15.55:0/1624954602 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083400b6b0 con 0x7f083815a3b0 2023-10-06T03:48:25.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.197+0000 7f083cf99700 1 -- 172.21.15.55:0/1624954602 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083400b810 con 0x7f083815a3b0 2023-10-06T03:48:25.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.197+0000 7f083cf99700 1 -- 172.21.15.55:0/1624954602 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083400bbb0 con 0x7f083815a3b0 2023-10-06T03:48:25.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.197+0000 7f083ef9d700 1 -- 172.21.15.55:0/1624954602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083815a3b0 msgr2=0x7f083815a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:25.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.197+0000 7f083ef9d700 1 --2- 172.21.15.55:0/1624954602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083815a3b0 0x7f083815a780 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f083806d7e0 tx=0x7f08340091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:25.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.197+0000 7f083ef9d700 1 -- 172.21.15.55:0/1624954602 shutdown_connections 2023-10-06T03:48:25.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.197+0000 7f083ef9d700 1 --2- 172.21.15.55:0/1624954602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083815a3b0 0x7f083815a780 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:25.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.197+0000 7f083ef9d700 1 --2- 172.21.15.55:0/1624954602 >> v2:172.21.15.163:3300/0 conn(0x7f08380aef00 0x7f083815d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:25.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.197+0000 7f083ef9d700 1 -- 172.21.15.55:0/1624954602 >> 172.21.15.55:0/1624954602 conn(0x7f08380c9f70 msgr2=0x7f083806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:25.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.197+0000 7f083ef9d700 1 -- 172.21.15.55:0/1624954602 shutdown_connections 2023-10-06T03:48:25.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.197+0000 7f083ef9d700 1 -- 172.21.15.55:0/1624954602 wait complete. 2023-10-06T03:48:25.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.199+0000 7f083ef9d700 1 Processor -- start 2023-10-06T03:48:25.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.199+0000 7f083ef9d700 1 -- start start 2023-10-06T03:48:25.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.200+0000 7f083ef9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083806e470 0x7f08380c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:25.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.200+0000 7f083ef9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08380aef00 0x7f0838151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:25.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.200+0000 7f083ef9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f083815a3b0 0x7f08381545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:25.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.200+0000 7f083ef9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08380c5a10 con 0x7f083806e470 2023-10-06T03:48:25.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.200+0000 7f083ef9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0838151d80 con 0x7f08380aef00 2023-10-06T03:48:25.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.200+0000 7f083ef9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0838154e40 con 0x7f083815a3b0 2023-10-06T03:48:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.200+0000 7f083df9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083806e470 0x7f08380c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.200+0000 7f083df9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083806e470 0x7f08380c48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48738/0 (socket says 172.21.15.55:48738) 2023-10-06T03:48:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.200+0000 7f083df9b700 1 -- 172.21.15.55:0/2330666885 learned_addr learned my addr 172.21.15.55:0/2330666885 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.200+0000 7f083df9b700 1 -- 172.21.15.55:0/2330666885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f083815a3b0 msgr2=0x7f08381545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.200+0000 7f083df9b700 1 --2- 172.21.15.55:0/2330666885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f083815a3b0 0x7f08381545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.200+0000 7f083df9b700 1 -- 172.21.15.55:0/2330666885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08380aef00 msgr2=0x7f0838151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.200+0000 7f083df9b700 1 --2- 172.21.15.55:0/2330666885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08380aef00 0x7f0838151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.200+0000 7f083df9b700 1 -- 172.21.15.55:0/2330666885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0834009420 con 0x7f083806e470 2023-10-06T03:48:25.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.201+0000 7f083df9b700 1 --2- 172.21.15.55:0/2330666885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083806e470 0x7f08380c48f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0838064280 tx=0x7f0834017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:25.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.201+0000 7f082dffb700 1 -- 172.21.15.55:0/2330666885 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083400bb90 con 0x7f083806e470 2023-10-06T03:48:25.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.201+0000 7f082dffb700 1 -- 172.21.15.55:0/2330666885 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0834011420 con 0x7f083806e470 2023-10-06T03:48:25.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.201+0000 7f082dffb700 1 -- 172.21.15.55:0/2330666885 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08340117a0 con 0x7f083806e470 2023-10-06T03:48:25.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.201+0000 7f083ef9d700 1 -- 172.21.15.55:0/2330666885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0838155120 con 0x7f083806e470 2023-10-06T03:48:25.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.201+0000 7f083ef9d700 1 -- 172.21.15.55:0/2330666885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08381556e0 con 0x7f083806e470 2023-10-06T03:48:25.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.202+0000 7f083ef9d700 1 -- 172.21.15.55:0/2330666885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083806cfa0 con 0x7f083806e470 2023-10-06T03:48:25.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.203+0000 7f082dffb700 1 -- 172.21.15.55:0/2330666885 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0834011b90 con 0x7f083806e470 2023-10-06T03:48:25.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.203+0000 7f082dffb700 1 --2- 172.21.15.55:0/2330666885 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f081c066f30 0x7f081c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:25.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.204+0000 7f082dffb700 1 -- 172.21.15.55:0/2330666885 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f08340861b0 con 0x7f083806e470 2023-10-06T03:48:25.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.204+0000 7f083d79a700 1 --2- 172.21.15.55:0/2330666885 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f081c066f30 0x7f081c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:25.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.205+0000 7f083d79a700 1 --2- 172.21.15.55:0/2330666885 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f081c066f30 0x7f081c0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0828006fd0 tx=0x7f0828008040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:25.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.208+0000 7f082dffb700 1 -- 172.21.15.55:0/2330666885 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0834053090 con 0x7f083806e470 2023-10-06T03:48:25.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.453+0000 7f083ef9d700 1 -- 172.21.15.55:0/2330666885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f083815e010 con 0x7f083806e470 2023-10-06T03:48:25.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.982+0000 7f082dffb700 1 -- 172.21.15.55:0/2330666885 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f08340566b0 con 0x7f083806e470 2023-10-06T03:48:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.985+0000 7f083ef9d700 1 -- 172.21.15.55:0/2330666885 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f081c066f30 msgr2=0x7f081c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.985+0000 7f083ef9d700 1 --2- 172.21.15.55:0/2330666885 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f081c066f30 0x7f081c0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0828006fd0 tx=0x7f0828008040 comp rx=0 tx=0).stop 2023-10-06T03:48:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.985+0000 7f083ef9d700 1 -- 172.21.15.55:0/2330666885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083806e470 msgr2=0x7f08380c48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.985+0000 7f083ef9d700 1 --2- 172.21.15.55:0/2330666885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083806e470 0x7f08380c48f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0838064280 tx=0x7f0834017000 comp rx=0 tx=0).stop 2023-10-06T03:48:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.985+0000 7f083ef9d700 1 -- 172.21.15.55:0/2330666885 shutdown_connections 2023-10-06T03:48:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.985+0000 7f083ef9d700 1 --2- 172.21.15.55:0/2330666885 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f081c066f30 0x7f081c0693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.985+0000 7f083ef9d700 1 --2- 172.21.15.55:0/2330666885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f083815a3b0 0x7f08381545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.985+0000 7f083ef9d700 1 --2- 172.21.15.55:0/2330666885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f083806e470 0x7f08380c48f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:25.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.985+0000 7f083ef9d700 1 --2- 172.21.15.55:0/2330666885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08380aef00 0x7f0838151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:25.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.985+0000 7f083ef9d700 1 -- 172.21.15.55:0/2330666885 >> 172.21.15.55:0/2330666885 conn(0x7f08380c9f70 msgr2=0x7f083815b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:25.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.985+0000 7f083ef9d700 1 -- 172.21.15.55:0/2330666885 shutdown_connections 2023-10-06T03:48:25.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:25.985+0000 7f083ef9d700 1 -- 172.21.15.55:0/2330666885 wait complete. 2023-10-06T03:48:26.006 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:48:26.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.194+0000 7fcfb9d9d700 1 Processor -- start 2023-10-06T03:48:26.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.194+0000 7fcfb9d9d700 1 -- start start 2023-10-06T03:48:26.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.196+0000 7fcfb9d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcfb4068aa0 0x7fcfb40b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:26.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.196+0000 7fcfb9d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb415a980 con 0x7fcfb40b0a00 2023-10-06T03:48:26.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.196+0000 7fcfb9d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb415a050 con 0x7fcfb4157270 2023-10-06T03:48:26.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.196+0000 7fcfb9d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb415a330 con 0x7fcfb4068aa0 2023-10-06T03:48:26.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.196+0000 7fcfb959c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcfb40b0a00 0x7fcfb415d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55806/0 (socket says 172.21.15.55:55806) 2023-10-06T03:48:26.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.196+0000 7fcfb3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcfb4068aa0 0x7fcfb40b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:26.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.196+0000 7fcfb959c700 1 -- 172.21.15.55:0/1096744332 learned_addr learned my addr 172.21.15.55:0/1096744332 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:26.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.196+0000 7fcfb3fff700 1 -- 172.21.15.55:0/1096744332 >> v1:172.21.15.163:6789/0 conn(0x7fcfb4157270 legacy=0x7fcfb4068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:26.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.196+0000 7fcfb37fe700 1 -- 172.21.15.55:0/1096744332 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915128583 0 0) 0x7fcfb415a980 con 0x7fcfb40b0a00 2023-10-06T03:48:26.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.196+0000 7fcfb3fff700 1 -- 172.21.15.55:0/1096744332 >> v1:172.21.15.55:6789/0 conn(0x7fcfb40b0a00 legacy=0x7fcfb415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:26.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.196+0000 7fcfb3fff700 1 -- 172.21.15.55:0/1096744332 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb415a610 con 0x7fcfb4068aa0 2023-10-06T03:48:26.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.197+0000 7fcfb3fff700 1 --2- 172.21.15.55:0/1096744332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfb4068aa0 0x7fcfb40b04c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00098e0 tx=0x7fcfa00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9772ce818718516 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.197+0000 7fcfb37fe700 1 -- 172.21.15.55:0/1096744332 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa0009e80 con 0x7fcfb4068aa0 2023-10-06T03:48:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.197+0000 7fcfb37fe700 1 -- 172.21.15.55:0/1096744332 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa000b860 con 0x7fcfb4068aa0 2023-10-06T03:48:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.197+0000 7fcfb37fe700 1 -- 172.21.15.55:0/1096744332 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000bbe0 con 0x7fcfb4068aa0 2023-10-06T03:48:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.197+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/1096744332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfb4068aa0 msgr2=0x7fcfb40b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.197+0000 7fcfb9d9d700 1 --2- 172.21.15.55:0/1096744332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfb4068aa0 0x7fcfb40b04c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fcfa00098e0 tx=0x7fcfa00091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.198+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/1096744332 shutdown_connections 2023-10-06T03:48:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.198+0000 7fcfb9d9d700 1 --2- 172.21.15.55:0/1096744332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfb4068aa0 0x7fcfb40b04c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.198+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/1096744332 >> 172.21.15.55:0/1096744332 conn(0x7fcfb40cd310 msgr2=0x7fcfb40cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.198+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/1096744332 shutdown_connections 2023-10-06T03:48:26.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.198+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/1096744332 wait complete. 2023-10-06T03:48:26.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.199+0000 7fcfb9d9d700 1 Processor -- start 2023-10-06T03:48:26.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.199+0000 7fcfb9d9d700 1 -- start start 2023-10-06T03:48:26.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.200+0000 7fcfb9d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfb4068aa0 0x7fcfb40ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:26.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.200+0000 7fcfb9d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfb40b0a00 0x7fcfb416a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:26.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.200+0000 7fcfb9d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcfb4157270 0x7fcfb41590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:26.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.200+0000 7fcfb9d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40aac40 con 0x7fcfb4068aa0 2023-10-06T03:48:26.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.200+0000 7fcfb9d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40ab5d0 con 0x7fcfb40b0a00 2023-10-06T03:48:26.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.200+0000 7fcfb9d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb40ac560 con 0x7fcfb4157270 2023-10-06T03:48:26.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.200+0000 7fcfb8d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfb4068aa0 0x7fcfb40ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:26.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.200+0000 7fcfb8d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfb4068aa0 0x7fcfb40ac020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48762/0 (socket says 172.21.15.55:48762) 2023-10-06T03:48:26.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.200+0000 7fcfb8d9b700 1 -- 172.21.15.55:0/3796669455 learned_addr learned my addr 172.21.15.55:0/3796669455 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:26.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.201+0000 7fcfb959c700 1 --2- 172.21.15.55:0/3796669455 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcfb4157270 0x7fcfb41590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:26.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.201+0000 7fcfb3fff700 1 --2- 172.21.15.55:0/3796669455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfb40b0a00 0x7fcfb416a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:26.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.201+0000 7fcfb8d9b700 1 -- 172.21.15.55:0/3796669455 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcfb4157270 msgr2=0x7fcfb41590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:26.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.201+0000 7fcfb8d9b700 1 --2- 172.21.15.55:0/3796669455 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcfb4157270 0x7fcfb41590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.201+0000 7fcfb8d9b700 1 -- 172.21.15.55:0/3796669455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfb40b0a00 msgr2=0x7fcfb416a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:26.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.201+0000 7fcfb8d9b700 1 --2- 172.21.15.55:0/3796669455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfb40b0a00 0x7fcfb416a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.201+0000 7fcfb8d9b700 1 -- 172.21.15.55:0/3796669455 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa0009420 con 0x7fcfb4068aa0 2023-10-06T03:48:26.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.201+0000 7fcfb8d9b700 1 --2- 172.21.15.55:0/3796669455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfb4068aa0 0x7fcfb40ac020 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c4e0 tx=0x7fcfa800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:26.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.201+0000 7fcfb0ff9700 1 -- 172.21.15.55:0/3796669455 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8011590 con 0x7fcfb4068aa0 2023-10-06T03:48:26.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.201+0000 7fcfb0ff9700 1 -- 172.21.15.55:0/3796669455 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa8011710 con 0x7fcfb4068aa0 2023-10-06T03:48:26.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.201+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/3796669455 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb416ab10 con 0x7fcfb4068aa0 2023-10-06T03:48:26.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.201+0000 7fcfb0ff9700 1 -- 172.21.15.55:0/3796669455 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8011ad0 con 0x7fcfb4068aa0 2023-10-06T03:48:26.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.201+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/3796669455 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb40ab1f0 con 0x7fcfb4068aa0 2023-10-06T03:48:26.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.203+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/3796669455 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb404c3c0 con 0x7fcfb4068aa0 2023-10-06T03:48:26.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.203+0000 7fcfb0ff9700 1 -- 172.21.15.55:0/3796669455 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcfa801b3b0 con 0x7fcfb4068aa0 2023-10-06T03:48:26.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.204+0000 7fcfb0ff9700 1 --2- 172.21.15.55:0/3796669455 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcf98066f30 0x7fcf980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:26.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.204+0000 7fcfb0ff9700 1 -- 172.21.15.55:0/3796669455 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fcfa8083d60 con 0x7fcfb4068aa0 2023-10-06T03:48:26.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.205+0000 7fcfb3fff700 1 --2- 172.21.15.55:0/3796669455 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcf98066f30 0x7fcf980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:26.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.205+0000 7fcfb3fff700 1 --2- 172.21.15.55:0/3796669455 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcf98066f30 0x7fcf980693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4068ea0 tx=0x7fcfa0017000 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:26.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.209+0000 7fcfb0ff9700 1 -- 172.21.15.55:0/3796669455 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcfa8050ce0 con 0x7fcfb4068aa0 2023-10-06T03:48:26.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.460+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/3796669455 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfb40b0e80 con 0x7fcfb4068aa0 2023-10-06T03:48:26.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.461+0000 7fcfb0ff9700 1 -- 172.21.15.55:0/3796669455 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fcfa8017070 con 0x7fcfb4068aa0 2023-10-06T03:48:26.462 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:26.462 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4046797285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":4046797285},{"type":"v1","addr":"172.21.15.55:6835","nonce":4046797285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5645,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/675796170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":675796170},{"type":"v1","addr":"172.21.15.163:6836","nonce":675796170}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5648,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2543497851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2543497851},{"type":"v1","addr":"172.21.15.163:6834","nonce":2543497851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:48:22.429407+0000","modified":"2023-10-06T03:48:25.973506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6333":{"gid":6333,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3434145175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3434145175},{"type":"v1","addr":"172.21.15.55:6834","nonce":3434145175}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T03:48:26.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.463+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/3796669455 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcf98066f30 msgr2=0x7fcf980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:26.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.463+0000 7fcfb9d9d700 1 --2- 172.21.15.55:0/3796669455 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcf98066f30 0x7fcf980693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4068ea0 tx=0x7fcfa0017000 comp rx=0 tx=0).stop 2023-10-06T03:48:26.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.463+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/3796669455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfb4068aa0 msgr2=0x7fcfb40ac020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:26.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.463+0000 7fcfb9d9d700 1 --2- 172.21.15.55:0/3796669455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfb4068aa0 0x7fcfb40ac020 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800c4e0 tx=0x7fcfa800c8a0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.463+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/3796669455 shutdown_connections 2023-10-06T03:48:26.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.463+0000 7fcfb9d9d700 1 --2- 172.21.15.55:0/3796669455 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcf98066f30 0x7fcf980693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.463+0000 7fcfb9d9d700 1 --2- 172.21.15.55:0/3796669455 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcfb4157270 0x7fcfb41590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.463+0000 7fcfb9d9d700 1 --2- 172.21.15.55:0/3796669455 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcfb4068aa0 0x7fcfb40ac020 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.463+0000 7fcfb9d9d700 1 --2- 172.21.15.55:0/3796669455 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcfb40b0a00 0x7fcfb416a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.463+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/3796669455 >> 172.21.15.55:0/3796669455 conn(0x7fcfb40cd310 msgr2=0x7fcfb41562d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:26.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.463+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/3796669455 shutdown_connections 2023-10-06T03:48:26.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.463+0000 7fcfb9d9d700 1 -- 172.21.15.55:0/3796669455 wait complete. 2023-10-06T03:48:26.467 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 35 2023-10-06T03:48:26.482 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:48:26.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.668+0000 7fe4bbfff700 1 Processor -- start 2023-10-06T03:48:26.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.668+0000 7fe4bbfff700 1 -- start start 2023-10-06T03:48:26.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.669+0000 7fe4bbfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe4bc15ba00 0x7fe4bc068a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:26.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.670+0000 7fe4bbfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4bc06c8d0 con 0x7fe4bc068fd0 2023-10-06T03:48:26.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.670+0000 7fe4bbfff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4bc06cb60 con 0x7fe4bc0686c0 2023-10-06T03:48:26.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.670+0000 7fe4bbfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4bc06ce40 con 0x7fe4bc15ba00 2023-10-06T03:48:26.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.670+0000 7fe4bb7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe4bc068fd0 0x7fe4bc0b23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55814/0 (socket says 172.21.15.55:55814) 2023-10-06T03:48:26.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.670+0000 7fe4bb7fe700 1 -- 172.21.15.55:0/3859158085 learned_addr learned my addr 172.21.15.55:0/3859158085 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:26.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.670+0000 7fe4ba7fc700 1 --2- 172.21.15.55:0/3859158085 >> v2:172.21.15.163:3300/0 conn(0x7fe4bc15ba00 0x7fe4bc068a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:26.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.670+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720826823 0 0) 0x7fe4bc06c8d0 con 0x7fe4bc068fd0 2023-10-06T03:48:26.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.670+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4a4003570 con 0x7fe4bc068fd0 2023-10-06T03:48:26.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.670+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3974854176 0 0) 0x7fe4bc06cb60 con 0x7fe4bc0686c0 2023-10-06T03:48:26.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.670+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4bc06c8d0 con 0x7fe4bc0686c0 2023-10-06T03:48:26.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 858896438 0 0) 0x7fe4a4003570 con 0x7fe4bc068fd0 2023-10-06T03:48:26.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 >> v2:172.21.15.163:3300/0 conn(0x7fe4bc15ba00 msgr2=0x7fe4bc068a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:26.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4b9ffb700 1 --2- 172.21.15.55:0/3859158085 >> v2:172.21.15.163:3300/0 conn(0x7fe4bc15ba00 0x7fe4bc068a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 >> v1:172.21.15.163:6789/0 conn(0x7fe4bc0686c0 legacy=0x7fe4bc06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc15b200 con 0x7fe4bc068fd0 2023-10-06T03:48:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fe4b40017f0 con 0x7fe4bc068fd0 2023-10-06T03:48:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 >> v1:172.21.15.55:6789/0 conn(0x7fe4bc068fd0 legacy=0x7fe4bc0b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4b9ffb700 1 --2- 172.21.15.55:0/3859158085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4a4003c40 0x7fe4a4004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4b9ffb700 1 --2- 172.21.15.55:0/3859158085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4a4004810 0x7fe4a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4b9ffb700 1 --2- 172.21.15.55:0/3859158085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4a4005210 0x7fe4a400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc06ce40 con 0x7fe4a4005210 2023-10-06T03:48:26.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a400c1b0 con 0x7fe4a4003c40 2023-10-06T03:48:26.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a400c370 con 0x7fe4a4004810 2023-10-06T03:48:26.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4bb7fe700 1 --2- 172.21.15.55:0/3859158085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4a4005210 0x7fe4a400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:26.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4baffd700 1 --2- 172.21.15.55:0/3859158085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4a4003c40 0x7fe4a4004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:26.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4ba7fc700 1 --2- 172.21.15.55:0/3859158085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4a4004810 0x7fe4a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:26.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.671+0000 7fe4bb7fe700 1 -- 172.21.15.55:0/3859158085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4a4004810 msgr2=0x7fe4a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:26.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.672+0000 7fe4bb7fe700 1 --2- 172.21.15.55:0/3859158085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4a4004810 0x7fe4a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.672+0000 7fe4bb7fe700 1 -- 172.21.15.55:0/3859158085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4a4003c40 msgr2=0x7fe4a4004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:26.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.672+0000 7fe4bb7fe700 1 --2- 172.21.15.55:0/3859158085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4a4003c40 0x7fe4a4004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.672+0000 7fe4bb7fe700 1 -- 172.21.15.55:0/3859158085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a4003570 con 0x7fe4a4005210 2023-10-06T03:48:26.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.672+0000 7fe4bb7fe700 1 -- 172.21.15.55:0/3859158085 reap_dead start 2023-10-06T03:48:26.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.672+0000 7fe4bb7fe700 1 --2- 172.21.15.55:0/3859158085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4a4005210 0x7fe4a400ba60 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400a7d0 tx=0x7fe4b40020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83ff64dfd89e6bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:26.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.672+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b40070d0 con 0x7fe4a4005210 2023-10-06T03:48:26.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.672+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b4007250 con 0x7fe4a4005210 2023-10-06T03:48:26.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.672+0000 7fe4b9ffb700 1 -- 172.21.15.55:0/3859158085 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4007610 con 0x7fe4a4005210 2023-10-06T03:48:26.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.672+0000 7fe4bbfff700 1 -- 172.21.15.55:0/3859158085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4a4005210 msgr2=0x7fe4a400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:26.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.673+0000 7fe4bbfff700 1 --2- 172.21.15.55:0/3859158085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4a4005210 0x7fe4a400ba60 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400a7d0 tx=0x7fe4b40020c0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.673+0000 7fe4bbfff700 1 -- 172.21.15.55:0/3859158085 shutdown_connections 2023-10-06T03:48:26.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.673+0000 7fe4bbfff700 1 --2- 172.21.15.55:0/3859158085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4a4005210 0x7fe4a400ba60 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.673+0000 7fe4bbfff700 1 -- 172.21.15.55:0/3859158085 >> 172.21.15.55:0/3859158085 conn(0x7fe4bc0cd310 msgr2=0x7fe4bc0cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:26.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.673+0000 7fe4bbfff700 1 -- 172.21.15.55:0/3859158085 shutdown_connections 2023-10-06T03:48:26.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.673+0000 7fe4bbfff700 1 -- 172.21.15.55:0/3859158085 wait complete. 2023-10-06T03:48:26.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.674+0000 7fe4bbfff700 1 Processor -- start 2023-10-06T03:48:26.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.674+0000 7fe4bbfff700 1 -- start start 2023-10-06T03:48:26.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.675+0000 7fe4bbfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4a4005210 0x7fe4bc1513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:26.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.675+0000 7fe4bbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4bc156480 0x7fe4bc1428a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:26.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.675+0000 7fe4bbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4bc142de0 0x7fe4bc15ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:26.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.675+0000 7fe4bbfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc1455b0 con 0x7fe4bc156480 2023-10-06T03:48:26.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.675+0000 7fe4bbfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc144fa0 con 0x7fe4bc142de0 2023-10-06T03:48:26.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.675+0000 7fe4bbfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc1568f0 con 0x7fe4a4005210 2023-10-06T03:48:26.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.676+0000 7fe4ba7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4bc156480 0x7fe4bc1428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:26.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.676+0000 7fe4ba7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4bc156480 0x7fe4bc1428a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48782/0 (socket says 172.21.15.55:48782) 2023-10-06T03:48:26.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.676+0000 7fe4baffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4a4005210 0x7fe4bc1513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:26.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.676+0000 7fe4ba7fc700 1 -- 172.21.15.55:0/4093820852 learned_addr learned my addr 172.21.15.55:0/4093820852 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:26.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.676+0000 7fe4bb7fe700 1 --2- 172.21.15.55:0/4093820852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4bc142de0 0x7fe4bc15ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:26.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.676+0000 7fe4ba7fc700 1 -- 172.21.15.55:0/4093820852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4a4005210 msgr2=0x7fe4bc1513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:26.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.676+0000 7fe4ba7fc700 1 --2- 172.21.15.55:0/4093820852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4a4005210 0x7fe4bc1513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.676+0000 7fe4ba7fc700 1 -- 172.21.15.55:0/4093820852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4bc142de0 msgr2=0x7fe4bc15ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:26.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.676+0000 7fe4ba7fc700 1 --2- 172.21.15.55:0/4093820852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4bc142de0 0x7fe4bc15ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.676+0000 7fe4ba7fc700 1 -- 172.21.15.55:0/4093820852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc145240 con 0x7fe4bc156480 2023-10-06T03:48:26.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.676+0000 7fe4bb7fe700 1 --2- 172.21.15.55:0/4093820852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4bc142de0 0x7fe4bc15ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:26.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.676+0000 7fe4ba7fc700 1 --2- 172.21.15.55:0/4093820852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4bc156480 0x7fe4bc1428a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0006ac0 tx=0x7fe4b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:26.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.676+0000 7fe4a2ffd700 1 -- 172.21.15.55:0/4093820852 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b000f940 con 0x7fe4bc156480 2023-10-06T03:48:26.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.677+0000 7fe4a2ffd700 1 -- 172.21.15.55:0/4093820852 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b000faa0 con 0x7fe4bc156480 2023-10-06T03:48:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.677+0000 7fe4bbfff700 1 -- 172.21.15.55:0/4093820852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4bc156bd0 con 0x7fe4bc156480 2023-10-06T03:48:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.677+0000 7fe4a2ffd700 1 -- 172.21.15.55:0/4093820852 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b000fe20 con 0x7fe4bc156480 2023-10-06T03:48:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.677+0000 7fe4bbfff700 1 -- 172.21.15.55:0/4093820852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4bc151910 con 0x7fe4bc156480 2023-10-06T03:48:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.678+0000 7fe4bbfff700 1 -- 172.21.15.55:0/4093820852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4bc04c3c0 con 0x7fe4bc156480 2023-10-06T03:48:26.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.679+0000 7fe4a2ffd700 1 -- 172.21.15.55:0/4093820852 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe4b0014650 con 0x7fe4bc156480 2023-10-06T03:48:26.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.680+0000 7fe4a2ffd700 1 --2- 172.21.15.55:0/4093820852 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe4a8066f60 0x7fe4a8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:26.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.680+0000 7fe4a2ffd700 1 -- 172.21.15.55:0/4093820852 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fe4b0018030 con 0x7fe4bc156480 2023-10-06T03:48:26.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.680+0000 7fe4baffd700 1 --2- 172.21.15.55:0/4093820852 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe4a8066f60 0x7fe4a8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:26.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.681+0000 7fe4baffd700 1 --2- 172.21.15.55:0/4093820852 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe4a8066f60 0x7fe4a8069410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac000c00 tx=0x7fe4ac007d30 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:26.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.684+0000 7fe4a2ffd700 1 -- 172.21.15.55:0/4093820852 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4b00520a0 con 0x7fe4bc156480 2023-10-06T03:48:26.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.888+0000 7fe4bbfff700 1 -- 172.21.15.55:0/4093820852 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4bc15b4e0 con 0x7fe4bc156480 2023-10-06T03:48:26.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.889+0000 7fe4a2ffd700 1 -- 172.21.15.55:0/4093820852 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17511 (secure 0 0 0) 0x7fe4b00556c0 con 0x7fe4bc156480 2023-10-06T03:48:26.891 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:26.891 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":80,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:48:23.960459+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:48:20.608889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T03:48:21.410367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:48:26.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.892+0000 7fe4bbfff700 1 -- 172.21.15.55:0/4093820852 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe4a8066f60 msgr2=0x7fe4a8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:26.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.892+0000 7fe4bbfff700 1 --2- 172.21.15.55:0/4093820852 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe4a8066f60 0x7fe4a8069410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac000c00 tx=0x7fe4ac007d30 comp rx=0 tx=0).stop 2023-10-06T03:48:26.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.892+0000 7fe4bbfff700 1 -- 172.21.15.55:0/4093820852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4bc156480 msgr2=0x7fe4bc1428a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.892+0000 7fe4bbfff700 1 --2- 172.21.15.55:0/4093820852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4bc156480 0x7fe4bc1428a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0006ac0 tx=0x7fe4b00091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.892+0000 7fe4bbfff700 1 -- 172.21.15.55:0/4093820852 shutdown_connections 2023-10-06T03:48:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.892+0000 7fe4bbfff700 1 --2- 172.21.15.55:0/4093820852 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe4a8066f60 0x7fe4a8069410 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.892+0000 7fe4bbfff700 1 --2- 172.21.15.55:0/4093820852 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe4bc142de0 0x7fe4bc15ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.892+0000 7fe4bbfff700 1 --2- 172.21.15.55:0/4093820852 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe4a4005210 0x7fe4bc1513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.892+0000 7fe4bbfff700 1 --2- 172.21.15.55:0/4093820852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe4bc156480 0x7fe4bc1428a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.892+0000 7fe4bbfff700 1 -- 172.21.15.55:0/4093820852 >> 172.21.15.55:0/4093820852 conn(0x7fe4bc0cd310 msgr2=0x7fe4bc145f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:26.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.892+0000 7fe4bbfff700 1 -- 172.21.15.55:0/4093820852 shutdown_connections 2023-10-06T03:48:26.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:26.892+0000 7fe4bbfff700 1 -- 172.21.15.55:0/4093820852 wait complete. 2023-10-06T03:48:26.911 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:48:26.912 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:48:27.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.098+0000 7fe8f339d700 1 Processor -- start 2023-10-06T03:48:27.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.098+0000 7fe8f339d700 1 -- start start 2023-10-06T03:48:27.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.099+0000 7fe8f339d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe8ec142ea0 0x7fe8ec0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:27.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.099+0000 7fe8f339d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8ec06c0f0 0x7fe8ec06a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:27.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.099+0000 7fe8f339d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ec0b40f0 con 0x7fe8ec0ad870 2023-10-06T03:48:27.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.099+0000 7fe8f339d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ec069640 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.099+0000 7fe8f339d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ec069920 con 0x7fe8ec142ea0 2023-10-06T03:48:27.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.100+0000 7fe8f1b9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe8ec0ad870 0x7fe8ec06b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55820/0 (socket says 172.21.15.55:55820) 2023-10-06T03:48:27.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.100+0000 7fe8f2b9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8ec06c0f0 0x7fe8ec06a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:27.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.100+0000 7fe8f1b9a700 1 -- 172.21.15.55:0/4114425825 learned_addr learned my addr 172.21.15.55:0/4114425825 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:27.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.100+0000 7fe8f239b700 1 --2- 172.21.15.55:0/4114425825 >> v2:172.21.15.163:3300/0 conn(0x7fe8ec142ea0 0x7fe8ec0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:27.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.100+0000 7fe8f2b9c700 1 -- 172.21.15.55:0/4114425825 >> v2:172.21.15.163:3300/0 conn(0x7fe8ec142ea0 msgr2=0x7fe8ec0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:27.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.100+0000 7fe8f2b9c700 1 --2- 172.21.15.55:0/4114425825 >> v2:172.21.15.163:3300/0 conn(0x7fe8ec142ea0 0x7fe8ec0b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.100+0000 7fe8f2b9c700 1 -- 172.21.15.55:0/4114425825 >> v1:172.21.15.55:6789/0 conn(0x7fe8ec0ad870 legacy=0x7fe8ec06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.100+0000 7fe8f2b9c700 1 -- 172.21.15.55:0/4114425825 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec069c00 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.100+0000 7fe8f239b700 1 --2- 172.21.15.55:0/4114425825 >> v2:172.21.15.163:3300/0 conn(0x7fe8ec142ea0 0x7fe8ec0b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.100+0000 7fe8f2b9c700 1 --2- 172.21.15.55:0/4114425825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ec06c0f0 0x7fe8ec06a120 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec071240 tx=0x7fe8e4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147896f2a01a103f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.101+0000 7fe8f1399700 1 -- 172.21.15.55:0/4114425825 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e400e070 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.101+0000 7fe8f1399700 1 -- 172.21.15.55:0/4114425825 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e400bc30 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.101+0000 7fe8f1399700 1 -- 172.21.15.55:0/4114425825 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4012430 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.101+0000 7fe8f339d700 1 -- 172.21.15.55:0/4114425825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ec06c0f0 msgr2=0x7fe8ec06a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.101+0000 7fe8f339d700 1 --2- 172.21.15.55:0/4114425825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ec06c0f0 0x7fe8ec06a120 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec071240 tx=0x7fe8e4009780 comp rx=0 tx=0).stop 2023-10-06T03:48:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.101+0000 7fe8f339d700 1 -- 172.21.15.55:0/4114425825 shutdown_connections 2023-10-06T03:48:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.101+0000 7fe8f339d700 1 --2- 172.21.15.55:0/4114425825 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ec06c0f0 0x7fe8ec06a120 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.101+0000 7fe8f339d700 1 --2- 172.21.15.55:0/4114425825 >> v2:172.21.15.163:3300/0 conn(0x7fe8ec142ea0 0x7fe8ec0b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.101+0000 7fe8f339d700 1 -- 172.21.15.55:0/4114425825 >> 172.21.15.55:0/4114425825 conn(0x7fe8ec0ce430 msgr2=0x7fe8ec0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:27.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.101+0000 7fe8f339d700 1 -- 172.21.15.55:0/4114425825 shutdown_connections 2023-10-06T03:48:27.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.101+0000 7fe8f339d700 1 -- 172.21.15.55:0/4114425825 wait complete. 2023-10-06T03:48:27.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.103+0000 7fe8f339d700 1 Processor -- start 2023-10-06T03:48:27.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.103+0000 7fe8f339d700 1 -- start start 2023-10-06T03:48:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f339d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ec06c0f0 0x7fe8ec133af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f339d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8ec142ea0 0x7fe8ec0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ec0cb2f0 0x7fe8ec15c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f339d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0cb790 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f339d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0c9740 con 0x7fe8ec142ea0 2023-10-06T03:48:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec0c92b0 con 0x7fe8ec0cb2f0 2023-10-06T03:48:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f239b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ec06c0f0 0x7fe8ec133af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f239b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ec06c0f0 0x7fe8ec133af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48804/0 (socket says 172.21.15.55:48804) 2023-10-06T03:48:27.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f239b700 1 -- 172.21.15.55:0/2269901477 learned_addr learned my addr 172.21.15.55:0/2269901477 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:27.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f1b9a700 1 --2- 172.21.15.55:0/2269901477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8ec142ea0 0x7fe8ec0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:27.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f2b9c700 1 --2- 172.21.15.55:0/2269901477 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ec0cb2f0 0x7fe8ec15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:27.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f239b700 1 -- 172.21.15.55:0/2269901477 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ec0cb2f0 msgr2=0x7fe8ec15c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:27.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f239b700 1 --2- 172.21.15.55:0/2269901477 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ec0cb2f0 0x7fe8ec15c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f239b700 1 -- 172.21.15.55:0/2269901477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8ec142ea0 msgr2=0x7fe8ec0cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:27.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f239b700 1 --2- 172.21.15.55:0/2269901477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8ec142ea0 0x7fe8ec0cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.104+0000 7fe8f239b700 1 -- 172.21.15.55:0/2269901477 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e4009a20 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.105+0000 7fe8f239b700 1 --2- 172.21.15.55:0/2269901477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ec06c0f0 0x7fe8ec133af0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800d890 tx=0x7fe8e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:27.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.105+0000 7fe8e27fc700 1 -- 172.21.15.55:0/2269901477 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800b600 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.105+0000 7fe8e27fc700 1 -- 172.21.15.55:0/2269901477 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e800b760 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.105+0000 7fe8f339d700 1 -- 172.21.15.55:0/2269901477 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ec0c86e0 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.105+0000 7fe8e27fc700 1 -- 172.21.15.55:0/2269901477 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800bb00 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.105+0000 7fe8f339d700 1 -- 172.21.15.55:0/2269901477 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ec0c8c60 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.106+0000 7fe8f339d700 1 -- 172.21.15.55:0/2269901477 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ec04c3c0 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.107+0000 7fe8e27fc700 1 -- 172.21.15.55:0/2269901477 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8e8014430 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.108+0000 7fe8e27fc700 1 --2- 172.21.15.55:0/2269901477 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe8d0066f30 0x7fe8d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.108+0000 7fe8e27fc700 1 -- 172.21.15.55:0/2269901477 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fe8e8084cb0 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.108+0000 7fe8f1b9a700 1 --2- 172.21.15.55:0/2269901477 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe8d0066f30 0x7fe8d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.109+0000 7fe8f1b9a700 1 --2- 172.21.15.55:0/2269901477 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe8d0066f30 0x7fe8d00693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec15b660 tx=0x7fe8dc009040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.112+0000 7fe8e27fc700 1 -- 172.21.15.55:0/2269901477 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8e8051b90 con 0x7fe8ec06c0f0 2023-10-06T03:48:27.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.313+0000 7fe8f339d700 1 -- 172.21.15.55:0/2269901477 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe8ec143270 con 0x7fe8d0066f30 2023-10-06T03:48:27.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.325+0000 7fe8e27fc700 1 -- 172.21.15.55:0/2269901477 <== mgr.5483 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302620 (secure 0 0 0) 0x7fe8ec143270 con 0x7fe8d0066f30 2023-10-06T03:48:27.332 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:27.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.329+0000 7fe8f339d700 1 -- 172.21.15.55:0/2269901477 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe8d0066f30 msgr2=0x7fe8d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.329+0000 7fe8f339d700 1 --2- 172.21.15.55:0/2269901477 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe8d0066f30 0x7fe8d00693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec15b660 tx=0x7fe8dc009040 comp rx=0 tx=0).stop 2023-10-06T03:48:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.329+0000 7fe8f339d700 1 -- 172.21.15.55:0/2269901477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ec06c0f0 msgr2=0x7fe8ec133af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.329+0000 7fe8f339d700 1 --2- 172.21.15.55:0/2269901477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ec06c0f0 0x7fe8ec133af0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800d890 tx=0x7fe8e8008040 comp rx=0 tx=0).stop 2023-10-06T03:48:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.329+0000 7fe8f339d700 1 -- 172.21.15.55:0/2269901477 shutdown_connections 2023-10-06T03:48:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.329+0000 7fe8f339d700 1 --2- 172.21.15.55:0/2269901477 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe8d0066f30 0x7fe8d00693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.329+0000 7fe8f339d700 1 --2- 172.21.15.55:0/2269901477 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8ec0cb2f0 0x7fe8ec15c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.329+0000 7fe8f339d700 1 --2- 172.21.15.55:0/2269901477 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8ec06c0f0 0x7fe8ec133af0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.329+0000 7fe8f339d700 1 --2- 172.21.15.55:0/2269901477 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8ec142ea0 0x7fe8ec0cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.329+0000 7fe8f339d700 1 -- 172.21.15.55:0/2269901477 >> 172.21.15.55:0/2269901477 conn(0x7fe8ec0ce430 msgr2=0x7fe8ec0b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.329+0000 7fe8f339d700 1 -- 172.21.15.55:0/2269901477 shutdown_connections 2023-10-06T03:48:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.329+0000 7fe8f339d700 1 -- 172.21.15.55:0/2269901477 wait complete. 2023-10-06T03:48:27.336 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:48:27.348 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-06T03:48:26.016508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592458,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":249,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":171,"ondisk_log_size":171,"up":108,"acting":108,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444084,"kb_used_data":8436,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752530636,"statfs":{"total":773094113280,"available":770591371264,"internally_reserved":0,"allocated":8638464,"data_stored":4898454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2090,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.085491"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981925+0000","last_change":"2023-10-06T03:48:22.978687+0000","last_active":"2023-10-06T03:48:22.981925+0000","last_peered":"2023-10-06T03:48:22.981925+0000","last_clean":"2023-10-06T03:48:22.981925+0000","last_became_active":"2023-10-06T03:48:22.977640+0000","last_became_peered":"2023-10-06T03:48:22.977640+0000","last_unstale":"2023-10-06T03:48:22.981925+0000","last_undegraded":"2023-10-06T03:48:22.981925+0000","last_fullsized":"2023-10-06T03:48:22.981925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:06.518654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980965+0000","last_change":"2023-10-06T03:48:22.965813+0000","last_active":"2023-10-06T03:48:22.980965+0000","last_peered":"2023-10-06T03:48:22.980965+0000","last_clean":"2023-10-06T03:48:22.980965+0000","last_became_active":"2023-10-06T03:48:22.965532+0000","last_became_peered":"2023-10-06T03:48:22.965532+0000","last_unstale":"2023-10-06T03:48:22.980965+0000","last_undegraded":"2023-10-06T03:48:22.980965+0000","last_fullsized":"2023-10-06T03:48:22.980965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:36.717228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.985364+0000","last_change":"2023-10-06T03:48:22.980964+0000","last_active":"2023-10-06T03:48:22.985364+0000","last_peered":"2023-10-06T03:48:22.985364+0000","last_clean":"2023-10-06T03:48:22.985364+0000","last_became_active":"2023-10-06T03:48:22.980709+0000","last_became_peered":"2023-10-06T03:48:22.980709+0000","last_unstale":"2023-10-06T03:48:22.985364+0000","last_undegraded":"2023-10-06T03:48:22.985364+0000","last_fullsized":"2023-10-06T03:48:22.985364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:53.633631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.978789+0000","last_change":"2023-10-06T03:48:22.972912+0000","last_active":"2023-10-06T03:48:22.978789+0000","last_peered":"2023-10-06T03:48:22.978789+0000","last_clean":"2023-10-06T03:48:22.978789+0000","last_became_active":"2023-10-06T03:48:22.972623+0000","last_became_peered":"2023-10-06T03:48:22.972623+0000","last_unstale":"2023-10-06T03:48:22.978789+0000","last_undegraded":"2023-10-06T03:48:22.978789+0000","last_fullsized":"2023-10-06T03:48:22.978789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:44.334010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981079+0000","last_change":"2023-10-06T03:48:22.973112+0000","last_active":"2023-10-06T03:48:22.981079+0000","last_peered":"2023-10-06T03:48:22.981079+0000","last_clean":"2023-10-06T03:48:22.981079+0000","last_became_active":"2023-10-06T03:48:22.972785+0000","last_became_peered":"2023-10-06T03:48:22.972785+0000","last_unstale":"2023-10-06T03:48:22.981079+0000","last_undegraded":"2023-10-06T03:48:22.981079+0000","last_fullsized":"2023-10-06T03:48:22.981079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:14.661907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980795+0000","last_change":"2023-10-06T03:48:22.969599+0000","last_active":"2023-10-06T03:48:22.980795+0000","last_peered":"2023-10-06T03:48:22.980795+0000","last_clean":"2023-10-06T03:48:22.980795+0000","last_became_active":"2023-10-06T03:48:22.969334+0000","last_became_peered":"2023-10-06T03:48:22.969334+0000","last_unstale":"2023-10-06T03:48:22.980795+0000","last_undegraded":"2023-10-06T03:48:22.980795+0000","last_fullsized":"2023-10-06T03:48:22.980795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:33.357173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"79'4","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.992673+0000","last_change":"2023-10-06T03:48:21.959573+0000","last_active":"2023-10-06T03:48:22.992673+0000","last_peered":"2023-10-06T03:48:22.992673+0000","last_clean":"2023-10-06T03:48:22.992673+0000","last_became_active":"2023-10-06T03:48:21.959316+0000","last_became_peered":"2023-10-06T03:48:21.959316+0000","last_unstale":"2023-10-06T03:48:22.992673+0000","last_undegraded":"2023-10-06T03:48:22.992673+0000","last_fullsized":"2023-10-06T03:48:22.992673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:11.472736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.942868+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:57.671969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.942708+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:08.115823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.942314+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:49.983388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970732+0000","last_change":"2023-10-06T03:48:22.969730+0000","last_active":"2023-10-06T03:48:22.970732+0000","last_peered":"2023-10-06T03:48:22.970732+0000","last_clean":"2023-10-06T03:48:22.970732+0000","last_became_active":"2023-10-06T03:48:22.969449+0000","last_became_peered":"2023-10-06T03:48:22.969449+0000","last_unstale":"2023-10-06T03:48:22.970732+0000","last_undegraded":"2023-10-06T03:48:22.970732+0000","last_fullsized":"2023-10-06T03:48:22.970732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:41.165824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.939366+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:32.347522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981121+0000","last_change":"2023-10-06T03:48:22.969630+0000","last_active":"2023-10-06T03:48:22.981121+0000","last_peered":"2023-10-06T03:48:22.981121+0000","last_clean":"2023-10-06T03:48:22.981121+0000","last_became_active":"2023-10-06T03:48:22.969362+0000","last_became_peered":"2023-10-06T03:48:22.969362+0000","last_unstale":"2023-10-06T03:48:22.981121+0000","last_undegraded":"2023-10-06T03:48:22.981121+0000","last_fullsized":"2023-10-06T03:48:22.981121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:04.560113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.940198+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:09.475188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.940103+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:47.117650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.933917+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:12.109876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980776+0000","last_change":"2023-10-06T03:48:22.965327+0000","last_active":"2023-10-06T03:48:22.980776+0000","last_peered":"2023-10-06T03:48:22.980776+0000","last_clean":"2023-10-06T03:48:22.980776+0000","last_became_active":"2023-10-06T03:48:22.964956+0000","last_became_peered":"2023-10-06T03:48:22.964956+0000","last_unstale":"2023-10-06T03:48:22.980776+0000","last_undegraded":"2023-10-06T03:48:22.980776+0000","last_fullsized":"2023-10-06T03:48:22.980776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:10.057932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.936898+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:46.538724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.934067+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:09.048281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.937515+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:06.076610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"79'6","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.990295+0000","last_change":"2023-10-06T03:48:21.959258+0000","last_active":"2023-10-06T03:48:22.990295+0000","last_peered":"2023-10-06T03:48:22.990295+0000","last_clean":"2023-10-06T03:48:22.990295+0000","last_became_active":"2023-10-06T03:48:21.958956+0000","last_became_peered":"2023-10-06T03:48:21.958956+0000","last_unstale":"2023-10-06T03:48:22.990295+0000","last_undegraded":"2023-10-06T03:48:22.990295+0000","last_fullsized":"2023-10-06T03:48:22.990295+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:46.538724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.938296+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:14.574278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.944835+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:31.104937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.936824+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:03.746648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980740+0000","last_change":"2023-10-06T03:48:22.971271+0000","last_active":"2023-10-06T03:48:22.980740+0000","last_peered":"2023-10-06T03:48:22.980740+0000","last_clean":"2023-10-06T03:48:22.980740+0000","last_became_active":"2023-10-06T03:48:22.970969+0000","last_became_peered":"2023-10-06T03:48:22.970969+0000","last_unstale":"2023-10-06T03:48:22.980740+0000","last_undegraded":"2023-10-06T03:48:22.980740+0000","last_fullsized":"2023-10-06T03:48:22.980740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:32.235650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.933922+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:35.707577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.944482+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:02.476783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.967698+0000","last_change":"2023-10-06T03:48:22.964069+0000","last_active":"2023-10-06T03:48:22.967698+0000","last_peered":"2023-10-06T03:48:22.967698+0000","last_clean":"2023-10-06T03:48:22.967698+0000","last_became_active":"2023-10-06T03:48:22.963794+0000","last_became_peered":"2023-10-06T03:48:22.963794+0000","last_unstale":"2023-10-06T03:48:22.967698+0000","last_undegraded":"2023-10-06T03:48:22.967698+0000","last_fullsized":"2023-10-06T03:48:22.967698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:03.317508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.944325+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:38.491221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.942138+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:12.512837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.956473+0000","last_change":"2023-10-06T03:48:21.958507+0000","last_active":"2023-10-06T03:48:22.956473+0000","last_peered":"2023-10-06T03:48:22.956473+0000","last_clean":"2023-10-06T03:48:22.956473+0000","last_became_active":"2023-10-06T03:48:21.958265+0000","last_became_peered":"2023-10-06T03:48:21.958265+0000","last_unstale":"2023-10-06T03:48:22.956473+0000","last_undegraded":"2023-10-06T03:48:22.956473+0000","last_fullsized":"2023-10-06T03:48:22.956473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:57.923641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.937467+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:57.923641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.939322+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:23.949718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981040+0000","last_change":"2023-10-06T03:48:22.964301+0000","last_active":"2023-10-06T03:48:22.981040+0000","last_peered":"2023-10-06T03:48:22.981040+0000","last_clean":"2023-10-06T03:48:22.981040+0000","last_became_active":"2023-10-06T03:48:22.963870+0000","last_became_peered":"2023-10-06T03:48:22.963870+0000","last_unstale":"2023-10-06T03:48:22.981040+0000","last_undegraded":"2023-10-06T03:48:22.981040+0000","last_fullsized":"2023-10-06T03:48:22.981040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:24.959369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.939385+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:31.225998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981729+0000","last_change":"2023-10-06T03:48:22.980115+0000","last_active":"2023-10-06T03:48:22.981729+0000","last_peered":"2023-10-06T03:48:22.981729+0000","last_clean":"2023-10-06T03:48:22.981729+0000","last_became_active":"2023-10-06T03:48:22.979810+0000","last_became_peered":"2023-10-06T03:48:22.979810+0000","last_unstale":"2023-10-06T03:48:22.981729+0000","last_undegraded":"2023-10-06T03:48:22.981729+0000","last_fullsized":"2023-10-06T03:48:22.981729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:20.245038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.955562+0000","last_change":"2023-10-06T03:48:21.961740+0000","last_active":"2023-10-06T03:48:22.955562+0000","last_peered":"2023-10-06T03:48:22.955562+0000","last_clean":"2023-10-06T03:48:22.955562+0000","last_became_active":"2023-10-06T03:48:21.961381+0000","last_became_peered":"2023-10-06T03:48:21.961381+0000","last_unstale":"2023-10-06T03:48:22.955562+0000","last_undegraded":"2023-10-06T03:48:22.955562+0000","last_fullsized":"2023-10-06T03:48:22.955562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:03.746648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.936988+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:14.341349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:17.926861+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:48:17.926861+0000","last_peered":"2023-10-06T03:48:17.926861+0000","last_clean":"2023-10-06T03:48:17.926861+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:48:17.926861+0000","last_undegraded":"2023-10-06T03:48:17.926861+0000","last_fullsized":"2023-10-06T03:48:17.926861+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.939418+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:42.534981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:18.679443+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:48:18.679443+0000","last_peered":"2023-10-06T03:48:18.679443+0000","last_clean":"2023-10-06T03:48:18.679443+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:48:18.679443+0000","last_undegraded":"2023-10-06T03:48:18.679443+0000","last_fullsized":"2023-10-06T03:48:18.679443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.942113+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:39.761111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":66,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.955914+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:48:22.955914+0000","last_peered":"2023-10-06T03:48:22.955914+0000","last_clean":"2023-10-06T03:48:22.955914+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:48:22.955914+0000","last_undegraded":"2023-10-06T03:48:22.955914+0000","last_fullsized":"2023-10-06T03:48:22.955914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"70'161","reported_seq":369,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:18.679828+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:48:18.679828+0000","last_peered":"2023-10-06T03:48:18.679828+0000","last_clean":"2023-10-06T03:48:18.679828+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:48:18.679828+0000","last_undegraded":"2023-10-06T03:48:18.679828+0000","last_fullsized":"2023-10-06T03:48:18.679828+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.944301+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:49.714558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":64,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:18.679805+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:48:18.679805+0000","last_peered":"2023-10-06T03:48:18.679805+0000","last_clean":"2023-10-06T03:48:18.679805+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:48:18.679805+0000","last_undegraded":"2023-10-06T03:48:18.679805+0000","last_fullsized":"2023-10-06T03:48:18.679805+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.939401+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:05.509003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:18.679808+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:48:18.679808+0000","last_peered":"2023-10-06T03:48:18.679808+0000","last_clean":"2023-10-06T03:48:18.679808+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:48:18.679808+0000","last_undegraded":"2023-10-06T03:48:18.679808+0000","last_fullsized":"2023-10-06T03:48:18.679808+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.942488+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:05.651421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:18.680558+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:48:18.680558+0000","last_peered":"2023-10-06T03:48:18.680558+0000","last_clean":"2023-10-06T03:48:18.680558+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:48:18.680558+0000","last_undegraded":"2023-10-06T03:48:18.680558+0000","last_fullsized":"2023-10-06T03:48:18.680558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.937034+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:11.472736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:17.926237+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:48:17.926237+0000","last_peered":"2023-10-06T03:48:17.926237+0000","last_clean":"2023-10-06T03:48:17.926237+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:48:17.926237+0000","last_undegraded":"2023-10-06T03:48:17.926237+0000","last_fullsized":"2023-10-06T03:48:17.926237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.945136+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:36.809130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:17.926244+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:48:17.926244+0000","last_peered":"2023-10-06T03:48:17.926244+0000","last_clean":"2023-10-06T03:48:17.926244+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:48:17.926244+0000","last_undegraded":"2023-10-06T03:48:17.926244+0000","last_fullsized":"2023-10-06T03:48:17.926244+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":26,"acting":26,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2090,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575030,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311048,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060792,"statfs":{"total":96636764160,"available":96318251008,"internally_reserved":0,"allocated":1507328,"data_stored":1047287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71199999999999997,"15min":0.69299999999999995},"min":{"1min":0.53000000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.998,"5min":1.0029999999999999,"15min":1.6100000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.51500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.5649999999999999},"last":0.66200000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.755,"15min":0.749},"min":{"1min":0.63800000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.038,"5min":1.038,"15min":1.5469999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.59199999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.56899999999999995,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":1.5900000000000001},"last":0.496},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.56999999999999995,"5min":0.495,"15min":0.495},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73199999999999998,"15min":0.72499999999999998},"min":{"1min":0.61299999999999999,"5min":0.45900000000000002,"15min":0.39500000000000002},"max":{"1min":1.0089999999999999,"5min":1.016,"15min":1.6279999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76000000000000001,"15min":0.74399999999999999},"min":{"1min":0.624,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96099999999999997,"5min":0.98099999999999998,"15min":1.0409999999999999},"last":0.68899999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79600000000000004,"5min":0.96499999999999997,"15min":1.0049999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.628},"min":{"1min":0.46400000000000002,"5min":0.40999999999999998,"15min":0.39400000000000002},"max":{"1min":0.874,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63900000000000001,"15min":0.64800000000000002},"min":{"1min":0.49299999999999999,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":0.95599999999999996,"5min":1.018,"15min":1.069},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.46800000000000003,"15min":0.436},"max":{"1min":0.91300000000000003,"5min":1.105,"15min":1.105},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72399999999999998,"15min":0.72899999999999998},"min":{"1min":0.58199999999999996,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":1.181,"15min":1.181},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.55200000000000005,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.91500000000000004,"5min":1.02,"15min":1.0209999999999999},"last":0.77500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575031,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306376,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065464,"statfs":{"total":96636764160,"available":96323035136,"internally_reserved":0,"allocated":917504,"data_stored":457897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.49199999999999999,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.92600000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.58099999999999996,"5min":0.41299999999999998,"15min":0.39500000000000002},"max":{"1min":0.96799999999999997,"5min":1.0069999999999999,"15min":1.0449999999999999},"last":0.63700000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.54100000000000004,"5min":0.36499999999999999,"15min":0.33800000000000002},"max":{"1min":0.93100000000000005,"5min":0.98199999999999998,"15min":1.3180000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71799999999999997,"15min":0.70799999999999996},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.35899999999999999},"max":{"1min":0.89600000000000002,"5min":0.995,"15min":0.995},"last":0.76400000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.751,"15min":0.75600000000000001},"min":{"1min":0.57899999999999996,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90900000000000003,"5min":0.995,"15min":3.3319999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73899999999999999,"15min":0.75700000000000001},"min":{"1min":0.48399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94399999999999995,"5min":0.97799999999999998,"15min":3.5139999999999998},"last":0.78700000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.751,"15min":0.747},"min":{"1min":0.58799999999999997,"5min":0.45500000000000002,"15min":0.44600000000000001},"max":{"1min":0.89900000000000002,"5min":1.01,"15min":1.024},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.76300000000000001,"15min":0.78100000000000003},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.94299999999999995,"5min":0.99099999999999999,"15min":3.3300000000000001},"last":0.88600000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56699999999999995,"15min":0.58699999999999997},"min":{"1min":0.40699999999999997,"5min":0.376,"15min":0.34799999999999998},"max":{"1min":0.78100000000000003,"5min":0.92600000000000005,"15min":1.0680000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.36499999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.2929999999999999},"last":0.749}]},{"osd":5,"last update":"Fri Oct 6 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65900000000000003,"15min":0.67200000000000004},"min":{"1min":0.48799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84099999999999997,"5min":0.96499999999999997,"15min":1.3340000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.41199999999999998},"max":{"1min":0.91000000000000003,"5min":1.0269999999999999,"15min":1.2370000000000001},"last":0.94999999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.70599999999999996},"min":{"1min":0.46800000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.619},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69499999999999995,"15min":0.71499999999999997},"min":{"1min":0.46800000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86299999999999999,"5min":1.0509999999999999,"15min":1.417},"last":0.90500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575031,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307032,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064808,"statfs":{"total":96636764160,"available":96322363392,"internally_reserved":0,"allocated":1589248,"data_stored":1078628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.47999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80200000000000005,"5min":1.1319999999999999,"15min":1.2889999999999999},"last":0.432}]},{"osd":2,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.5,"15min":0.495},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.26100000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.44,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.94099999999999995},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.60199999999999998},"min":{"1min":0.50800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86699999999999999,"15min":0.93999999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.434,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77400000000000002,"5min":0.90400000000000003,"15min":1.254},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.52500000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0009999999999999,"5min":1.034,"15min":1.3600000000000001},"last":0.626},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.80900000000000005,"15min":0.79800000000000004},"min":{"1min":0.68400000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.476},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.80200000000000005,"15min":0.79300000000000004},"min":{"1min":0.621,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.601},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.59299999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.5189999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.80500000000000005,"15min":0.81000000000000005},"min":{"1min":0.64200000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82499999999999996,"15min":0.81699999999999995},"min":{"1min":0.66700000000000004,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":1.143,"5min":1.1639999999999999,"15min":1.399},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.84799999999999998,"15min":0.83399999999999996},"min":{"1min":0.66900000000000004,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1000000000000001,"5min":1.272,"15min":1.272},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.83399999999999996,"15min":0.82899999999999996},"min":{"1min":0.64400000000000002,"5min":0.53700000000000003,"15min":0.51200000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.556},"last":0.65600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575032,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306372,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065468,"statfs":{"total":96636764160,"available":96323039232,"internally_reserved":0,"allocated":913408,"data_stored":456919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48199999999999998,"15min":0.495},"min":{"1min":0.373,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":0.67800000000000005,"5min":0.81000000000000005,"15min":2.3220000000000001},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.52700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33600000000000002,"15min":0.29599999999999999},"max":{"1min":0.70299999999999996,"5min":0.78500000000000003,"15min":2.1440000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.56499999999999995},"min":{"1min":0.40500000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.76400000000000001,"5min":0.77700000000000002,"15min":2.4329999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54800000000000004,"15min":0.56299999999999994},"min":{"1min":0.40799999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.755,"5min":0.84699999999999998,"15min":2.1819999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51100000000000001,"15min":0.53000000000000003},"min":{"1min":0.41399999999999998,"5min":0.30499999999999999,"15min":0.28399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":2.2709999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61299999999999999,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81000000000000005,"5min":1.0580000000000001,"15min":2.4700000000000002},"last":0.44800000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.83099999999999996,"15min":0.83499999999999996},"min":{"1min":0.69199999999999995,"5min":0.54200000000000004,"15min":0.49399999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.393},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.80900000000000005,"15min":0.83199999999999996},"min":{"1min":0.63400000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.306,"5min":1.306,"15min":2.2320000000000002},"last":0.56399999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.80600000000000005,"15min":0.82999999999999996},"min":{"1min":0.57799999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":2.3660000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.58999999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0780000000000001,"5min":1.0800000000000001,"15min":1.6559999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77900000000000003,"15min":0.80500000000000005},"min":{"1min":0.56999999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.726},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.82399999999999995,"15min":0.85699999999999998},"min":{"1min":0.628,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1100000000000001,"5min":1.119,"15min":2.4009999999999998},"last":0.79900000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.64300000000000002,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2689999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.79800000000000004,"15min":0.83699999999999997},"min":{"1min":0.625,"5min":0.495,"15min":0.495},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.7649999999999999},"last":0.75900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575032,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306420,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065420,"statfs":{"total":96636764160,"available":96322990080,"internally_reserved":0,"allocated":962560,"data_stored":485010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48899999999999999,"15min":0.46700000000000003},"min":{"1min":0.35199999999999998,"5min":0.29699999999999999,"15min":0.28000000000000003},"max":{"1min":0.74199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53400000000000003,"15min":0.52300000000000002},"min":{"1min":0.438,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":0.86199999999999999,"15min":0.89300000000000002},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.34799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72899999999999998,"5min":0.79900000000000004,"15min":0.90000000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.372,"15min":0.34200000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.556},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.504,"5min":0.41399999999999998,"15min":0.38800000000000001},"max":{"1min":0.72899999999999998,"5min":1.78,"15min":1.78},"last":0.439},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81799999999999995,"5min":0.91400000000000003,"15min":1.1910000000000001},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.81899999999999995,"15min":0.81799999999999995},"min":{"1min":0.67600000000000005,"5min":0.51400000000000001,"15min":0.439},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.5620000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.72199999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.008,"5min":1.1419999999999999,"15min":1.2270000000000001},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.82699999999999996,"15min":0.82999999999999996},"min":{"1min":0.68100000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.1140000000000001,"5min":1.177,"15min":1.3},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84399999999999997,"15min":0.84999999999999998},"min":{"1min":0.65500000000000003,"5min":0.55500000000000005,"15min":0.51700000000000002},"max":{"1min":1.0549999999999999,"5min":1.855,"15min":1.855},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82999999999999996,"15min":0.83699999999999997},"min":{"1min":0.68400000000000005,"5min":0.54200000000000004,"15min":0.53600000000000003},"max":{"1min":0.97899999999999998,"5min":1.907,"15min":1.907},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.81899999999999995,"15min":0.81499999999999995},"min":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.51100000000000001},"max":{"1min":1.1080000000000001,"5min":1.1120000000000001,"15min":1.4750000000000001},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.84099999999999997,"15min":0.83499999999999996},"min":{"1min":0.73999999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.125,"5min":1.4379999999999999,"15min":1.4990000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.86399999999999999},"min":{"1min":0.68400000000000005,"5min":0.53100000000000003,"15min":0.497},"max":{"1min":0.99399999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.69599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575031,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306376,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065464,"statfs":{"total":96636764160,"available":96323035136,"internally_reserved":0,"allocated":917504,"data_stored":457897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.33000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56899999999999995,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":0.63800000000000001,"5min":1.04,"15min":1.1699999999999999},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47799999999999998,"15min":0.47299999999999998},"min":{"1min":0.374,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.78800000000000003,"15min":0.79500000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.33800000000000002},"max":{"1min":0.70599999999999996,"5min":0.76700000000000002,"15min":0.83899999999999997},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.34599999999999997},"max":{"1min":0.79700000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.503},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76800000000000002,"5min":0.81999999999999995,"15min":0.97899999999999998},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.78000000000000003,"15min":0.78900000000000003},"min":{"1min":0.51600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.0049999999999999,"5min":1.2030000000000001,"15min":1.4099999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.64100000000000001,"5min":0.57399999999999995,"15min":0.51300000000000001},"max":{"1min":1.0469999999999999,"5min":1.1850000000000001,"15min":1.377},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77400000000000002,"15min":0.78800000000000003},"min":{"1min":0.56699999999999995,"5min":0.51900000000000002,"15min":0.47099999999999997},"max":{"1min":1,"5min":1.256,"15min":1.256},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.61499999999999999,"5min":0.55600000000000005,"15min":0.48099999999999998},"max":{"1min":0.99199999999999999,"5min":1.1439999999999999,"15min":1.306},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81000000000000005,"15min":0.81399999999999995},"min":{"1min":0.61699999999999999,"5min":0.52000000000000002,"15min":0.48599999999999999},"max":{"1min":1.02,"5min":1.292,"15min":1.323},"last":0.629},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.82799999999999996,"15min":0.84999999999999998},"min":{"1min":0.60599999999999998,"5min":0.58599999999999997,"15min":0.49299999999999999},"max":{"1min":1.075,"5min":1.1040000000000001,"15min":1.6020000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.65500000000000003,"5min":0.57799999999999996,"15min":0.505},"max":{"1min":0.98999999999999999,"5min":1.2370000000000001,"15min":1.365},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80000000000000004,"15min":0.80800000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.46300000000000002},"max":{"1min":1.038,"5min":1.1180000000000001,"15min":1.2330000000000001},"last":0.75}]}]},{"osd":4,"up_from":13,"seq":55834575031,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302280,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069560,"statfs":{"total":96636764160,"available":96327229440,"internally_reserved":0,"allocated":917504,"data_stored":457897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.70599999999999996,"15min":0.72099999999999997},"min":{"1min":0.56899999999999995,"5min":0.316,"15min":0.316},"max":{"1min":0.96999999999999997,"5min":1.0109999999999999,"15min":1.3120000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.54800000000000004,"5min":0.316,"15min":0.316},"max":{"1min":0.91500000000000004,"5min":0.95299999999999996,"15min":1.01},"last":0.71199999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.47299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85899999999999999,"5min":1.081,"15min":1.081},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71799999999999997,"15min":0.73699999999999999},"min":{"1min":0.64600000000000002,"5min":0.44,"15min":0.41299999999999998},"max":{"1min":0.89600000000000002,"5min":0.96999999999999997,"15min":1.3380000000000001},"last":0.80700000000000005}]},{"osd":2,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.61399999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.91500000000000004,"5min":1.028,"15min":1.1759999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.747,"15min":0.746},"min":{"1min":0.629,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0109999999999999,"5min":1.022,"15min":1.3089999999999999},"last":0.877}]},{"osd":3,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.63100000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89700000000000002,"5min":1.0049999999999999,"15min":1.0409999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75800000000000001,"15min":0.76900000000000002},"min":{"1min":0.53200000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.83199999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.621,"15min":0.63},"min":{"1min":0.496,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":1.01,"15min":1.085},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.622,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.35099999999999998},"max":{"1min":0.72799999999999998,"5min":0.95399999999999996,"15min":1.1439999999999999},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.753},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.38100000000000001},"max":{"1min":0.90200000000000002,"5min":1.052,"15min":1.361},"last":0.84799999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.39900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":1.1950000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.52000000000000002,"5min":0.504,"15min":0.41299999999999998},"max":{"1min":0.90100000000000002,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.86499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575031,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298180,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073660,"statfs":{"total":96636764160,"available":96331427840,"internally_reserved":0,"allocated":913408,"data_stored":456919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.42299999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94999999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70899999999999996,"15min":0.70699999999999996},"min":{"1min":0.60599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93500000000000005,"5min":0.96899999999999997,"15min":1},"last":0.67300000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.70599999999999996,"15min":0.70799999999999996},"min":{"1min":0.46600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.60299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76400000000000001,"15min":0.77600000000000002},"min":{"1min":0.67900000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98699999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72399999999999998,"15min":0.73099999999999998},"min":{"1min":0.56599999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.996,"5min":0.996,"15min":1.6419999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.754,"15min":0.749},"min":{"1min":0.623,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0189999999999999,"5min":1.117,"15min":1.117},"last":0.68999999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.62,"15min":0.621},"min":{"1min":0.42899999999999999,"5min":0.40000000000000002,"15min":0.38300000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.024},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61799999999999999,"15min":0.63800000000000001},"min":{"1min":0.47099999999999997,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":0.94199999999999995,"5min":1.044,"15min":1.175},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.34999999999999998},"max":{"1min":1.0449999999999999,"5min":1.591,"15min":1.591},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69599999999999995,"15min":0.69899999999999995},"min":{"1min":0.47799999999999998,"5min":0.44600000000000001,"15min":0.40600000000000003},"max":{"1min":1.0389999999999999,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.746}]},{"osd":7,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67000000000000004,"15min":0.68100000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":0.97199999999999998,"5min":1.016,"15min":1.2589999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.72399999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.435},"max":{"1min":1.0880000000000001,"5min":1.278,"15min":1.278},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:48:27.356 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:48:27.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.574+0000 7fbff6d9d700 1 Processor -- start 2023-10-06T03:48:27.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.574+0000 7fbff6d9d700 1 -- start start 2023-10-06T03:48:27.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.576+0000 7fbff6d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbff0156310 0x7fbff00c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.576+0000 7fbff6d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbff01566e0 0x7fbff00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.576+0000 7fbff6d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff00ca860 con 0x7fbff00ca300 2023-10-06T03:48:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.576+0000 7fbff6d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff00cab40 con 0x7fbff0156310 2023-10-06T03:48:27.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.576+0000 7fbff6d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff00cae20 con 0x7fbff01566e0 2023-10-06T03:48:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.576+0000 7fbff5d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbff0156310 0x7fbff00c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.576+0000 7fbff659c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbff00ca300 0x7fbff00c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55828/0 (socket says 172.21.15.55:55828) 2023-10-06T03:48:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.576+0000 7fbff659c700 1 -- 172.21.15.55:0/1022153713 learned_addr learned my addr 172.21.15.55:0/1022153713 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.576+0000 7fbff559a700 1 --2- 172.21.15.55:0/1022153713 >> v2:172.21.15.163:3300/0 conn(0x7fbff01566e0 0x7fbff00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.577+0000 7fbff4d99700 1 -- 172.21.15.55:0/1022153713 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098967348 0 0) 0x7fbff00ca860 con 0x7fbff00ca300 2023-10-06T03:48:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.577+0000 7fbff4d99700 1 -- 172.21.15.55:0/1022153713 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfdc003570 con 0x7fbff00ca300 2023-10-06T03:48:27.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.577+0000 7fbff5d9b700 1 -- 172.21.15.55:0/1022153713 >> v2:172.21.15.163:3300/0 conn(0x7fbff01566e0 msgr2=0x7fbff00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:27.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.577+0000 7fbff5d9b700 1 --2- 172.21.15.55:0/1022153713 >> v2:172.21.15.163:3300/0 conn(0x7fbff01566e0 0x7fbff00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.577+0000 7fbff5d9b700 1 -- 172.21.15.55:0/1022153713 >> v1:172.21.15.55:6789/0 conn(0x7fbff00ca300 legacy=0x7fbff00c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:27.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.577+0000 7fbff5d9b700 1 -- 172.21.15.55:0/1022153713 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff00cb100 con 0x7fbff0156310 2023-10-06T03:48:27.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.577+0000 7fbff5d9b700 1 --2- 172.21.15.55:0/1022153713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff0156310 0x7fbff00c9dc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbff006c000 tx=0x7fbfec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494f1949aacf3d8a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.577+0000 7fbff4d99700 1 -- 172.21.15.55:0/1022153713 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec011630 con 0x7fbff0156310 2023-10-06T03:48:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.577+0000 7fbff4d99700 1 -- 172.21.15.55:0/1022153713 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec0117b0 con 0x7fbff0156310 2023-10-06T03:48:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.577+0000 7fbff4d99700 1 -- 172.21.15.55:0/1022153713 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec011b70 con 0x7fbff0156310 2023-10-06T03:48:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.578+0000 7fbff6d9d700 1 -- 172.21.15.55:0/1022153713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff0156310 msgr2=0x7fbff00c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.578+0000 7fbff6d9d700 1 --2- 172.21.15.55:0/1022153713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff0156310 0x7fbff00c9dc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fbff006c000 tx=0x7fbfec0091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.578+0000 7fbff6d9d700 1 -- 172.21.15.55:0/1022153713 shutdown_connections 2023-10-06T03:48:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.578+0000 7fbff6d9d700 1 --2- 172.21.15.55:0/1022153713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff0156310 0x7fbff00c9dc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.578+0000 7fbff6d9d700 1 --2- 172.21.15.55:0/1022153713 >> v2:172.21.15.163:3300/0 conn(0x7fbff01566e0 0x7fbff00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.578+0000 7fbff6d9d700 1 -- 172.21.15.55:0/1022153713 >> 172.21.15.55:0/1022153713 conn(0x7fbff00cc1f0 msgr2=0x7fbff00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.578+0000 7fbff6d9d700 1 -- 172.21.15.55:0/1022153713 shutdown_connections 2023-10-06T03:48:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.578+0000 7fbff6d9d700 1 -- 172.21.15.55:0/1022153713 wait complete. 2023-10-06T03:48:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.579+0000 7fbff6d9d700 1 Processor -- start 2023-10-06T03:48:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.580+0000 7fbff6d9d700 1 -- start start 2023-10-06T03:48:27.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff00ca300 0x7fbff014a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:27.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff6d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbff0156310 0x7fbff00c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:27.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff6d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbff01566e0 0x7fbff00c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:27.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff6d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff014aa20 con 0x7fbff00ca300 2023-10-06T03:48:27.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff6d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff015a860 con 0x7fbff0156310 2023-10-06T03:48:27.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff6d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff01597c0 con 0x7fbff01566e0 2023-10-06T03:48:27.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff5d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff00ca300 0x7fbff014a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:27.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff5d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff00ca300 0x7fbff014a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48812/0 (socket says 172.21.15.55:48812) 2023-10-06T03:48:27.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff5d9b700 1 -- 172.21.15.55:0/2288659255 learned_addr learned my addr 172.21.15.55:0/2288659255 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:27.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff659c700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbff01566e0 0x7fbff00c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:27.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff559a700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbff0156310 0x7fbff00c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:27.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff5d9b700 1 -- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbff01566e0 msgr2=0x7fbff00c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:27.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff5d9b700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbff01566e0 0x7fbff00c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff5d9b700 1 -- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbff0156310 msgr2=0x7fbff00c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:27.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff5d9b700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbff0156310 0x7fbff00c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff5d9b700 1 -- 172.21.15.55:0/2288659255 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec009420 con 0x7fbff00ca300 2023-10-06T03:48:27.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff659c700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbff01566e0 0x7fbff00c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:27.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff559a700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbff0156310 0x7fbff00c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:27.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.581+0000 7fbff5d9b700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff00ca300 0x7fbff014a4e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbfec013000 tx=0x7fbfec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:27.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.582+0000 7fbfe5ffb700 1 -- 172.21.15.55:0/2288659255 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec01f070 con 0x7fbff00ca300 2023-10-06T03:48:27.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.582+0000 7fbfe5ffb700 1 -- 172.21.15.55:0/2288659255 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec000c90 con 0x7fbff00ca300 2023-10-06T03:48:27.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.582+0000 7fbff6d9d700 1 -- 172.21.15.55:0/2288659255 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff0159aa0 con 0x7fbff00ca300 2023-10-06T03:48:27.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.582+0000 7fbfe5ffb700 1 -- 172.21.15.55:0/2288659255 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec011b50 con 0x7fbff00ca300 2023-10-06T03:48:27.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.582+0000 7fbff6d9d700 1 -- 172.21.15.55:0/2288659255 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff015a060 con 0x7fbff00ca300 2023-10-06T03:48:27.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.583+0000 7fbff6d9d700 1 -- 172.21.15.55:0/2288659255 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff004c3c0 con 0x7fbff00ca300 2023-10-06T03:48:27.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.584+0000 7fbfe5ffb700 1 -- 172.21.15.55:0/2288659255 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfec023070 con 0x7fbff00ca300 2023-10-06T03:48:27.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.585+0000 7fbfe5ffb700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbfd4066f30 0x7fbfd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:27.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.585+0000 7fbfe5ffb700 1 -- 172.21.15.55:0/2288659255 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fbfec085cf0 con 0x7fbff00ca300 2023-10-06T03:48:27.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.585+0000 7fbff559a700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbfd4066f30 0x7fbfd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:27.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.586+0000 7fbff559a700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbfd4066f30 0x7fbfd40693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0006f00 tx=0x7fbfe0008040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:27.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.589+0000 7fbfe5ffb700 1 -- 172.21.15.55:0/2288659255 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfec052bd0 con 0x7fbff00ca300 2023-10-06T03:48:27.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.786+0000 7fbff6d9d700 1 -- 172.21.15.55:0/2288659255 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbff00ca050 con 0x7fbfd4066f30 2023-10-06T03:48:27.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.796+0000 7fbfe5ffb700 1 -- 172.21.15.55:0/2288659255 <== mgr.5483 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302620 (secure 0 0 0) 0x7fbff00ca050 con 0x7fbfd4066f30 2023-10-06T03:48:27.801 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:27.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.800+0000 7fbff6d9d700 1 -- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbfd4066f30 msgr2=0x7fbfd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:27.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.800+0000 7fbff6d9d700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbfd4066f30 0x7fbfd40693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0006f00 tx=0x7fbfe0008040 comp rx=0 tx=0).stop 2023-10-06T03:48:27.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.800+0000 7fbff6d9d700 1 -- 172.21.15.55:0/2288659255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff00ca300 msgr2=0x7fbff014a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:27.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.800+0000 7fbff6d9d700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff00ca300 0x7fbff014a4e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fbfec013000 tx=0x7fbfec017000 comp rx=0 tx=0).stop 2023-10-06T03:48:27.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.800+0000 7fbff6d9d700 1 -- 172.21.15.55:0/2288659255 shutdown_connections 2023-10-06T03:48:27.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.800+0000 7fbff6d9d700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbfd4066f30 0x7fbfd40693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.800+0000 7fbff6d9d700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbff01566e0 0x7fbff00c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.800+0000 7fbff6d9d700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbff00ca300 0x7fbff014a4e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.800+0000 7fbff6d9d700 1 --2- 172.21.15.55:0/2288659255 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbff0156310 0x7fbff00c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:27.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.800+0000 7fbff6d9d700 1 -- 172.21.15.55:0/2288659255 >> 172.21.15.55:0/2288659255 conn(0x7fbff00cc1f0 msgr2=0x7fbff006a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:27.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.800+0000 7fbff6d9d700 1 -- 172.21.15.55:0/2288659255 shutdown_connections 2023-10-06T03:48:27.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:27.800+0000 7fbff6d9d700 1 -- 172.21.15.55:0/2288659255 wait complete. 2023-10-06T03:48:27.806 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:48:27.819 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-06T03:48:26.016508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592458,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":249,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":171,"ondisk_log_size":171,"up":108,"acting":108,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444084,"kb_used_data":8436,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752530636,"statfs":{"total":773094113280,"available":770591371264,"internally_reserved":0,"allocated":8638464,"data_stored":4898454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2090,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.085491"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981925+0000","last_change":"2023-10-06T03:48:22.978687+0000","last_active":"2023-10-06T03:48:22.981925+0000","last_peered":"2023-10-06T03:48:22.981925+0000","last_clean":"2023-10-06T03:48:22.981925+0000","last_became_active":"2023-10-06T03:48:22.977640+0000","last_became_peered":"2023-10-06T03:48:22.977640+0000","last_unstale":"2023-10-06T03:48:22.981925+0000","last_undegraded":"2023-10-06T03:48:22.981925+0000","last_fullsized":"2023-10-06T03:48:22.981925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:06.518654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980965+0000","last_change":"2023-10-06T03:48:22.965813+0000","last_active":"2023-10-06T03:48:22.980965+0000","last_peered":"2023-10-06T03:48:22.980965+0000","last_clean":"2023-10-06T03:48:22.980965+0000","last_became_active":"2023-10-06T03:48:22.965532+0000","last_became_peered":"2023-10-06T03:48:22.965532+0000","last_unstale":"2023-10-06T03:48:22.980965+0000","last_undegraded":"2023-10-06T03:48:22.980965+0000","last_fullsized":"2023-10-06T03:48:22.980965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:36.717228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.985364+0000","last_change":"2023-10-06T03:48:22.980964+0000","last_active":"2023-10-06T03:48:22.985364+0000","last_peered":"2023-10-06T03:48:22.985364+0000","last_clean":"2023-10-06T03:48:22.985364+0000","last_became_active":"2023-10-06T03:48:22.980709+0000","last_became_peered":"2023-10-06T03:48:22.980709+0000","last_unstale":"2023-10-06T03:48:22.985364+0000","last_undegraded":"2023-10-06T03:48:22.985364+0000","last_fullsized":"2023-10-06T03:48:22.985364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:53.633631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.978789+0000","last_change":"2023-10-06T03:48:22.972912+0000","last_active":"2023-10-06T03:48:22.978789+0000","last_peered":"2023-10-06T03:48:22.978789+0000","last_clean":"2023-10-06T03:48:22.978789+0000","last_became_active":"2023-10-06T03:48:22.972623+0000","last_became_peered":"2023-10-06T03:48:22.972623+0000","last_unstale":"2023-10-06T03:48:22.978789+0000","last_undegraded":"2023-10-06T03:48:22.978789+0000","last_fullsized":"2023-10-06T03:48:22.978789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:44.334010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981079+0000","last_change":"2023-10-06T03:48:22.973112+0000","last_active":"2023-10-06T03:48:22.981079+0000","last_peered":"2023-10-06T03:48:22.981079+0000","last_clean":"2023-10-06T03:48:22.981079+0000","last_became_active":"2023-10-06T03:48:22.972785+0000","last_became_peered":"2023-10-06T03:48:22.972785+0000","last_unstale":"2023-10-06T03:48:22.981079+0000","last_undegraded":"2023-10-06T03:48:22.981079+0000","last_fullsized":"2023-10-06T03:48:22.981079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:14.661907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980795+0000","last_change":"2023-10-06T03:48:22.969599+0000","last_active":"2023-10-06T03:48:22.980795+0000","last_peered":"2023-10-06T03:48:22.980795+0000","last_clean":"2023-10-06T03:48:22.980795+0000","last_became_active":"2023-10-06T03:48:22.969334+0000","last_became_peered":"2023-10-06T03:48:22.969334+0000","last_unstale":"2023-10-06T03:48:22.980795+0000","last_undegraded":"2023-10-06T03:48:22.980795+0000","last_fullsized":"2023-10-06T03:48:22.980795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:33.357173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"79'4","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.992673+0000","last_change":"2023-10-06T03:48:21.959573+0000","last_active":"2023-10-06T03:48:22.992673+0000","last_peered":"2023-10-06T03:48:22.992673+0000","last_clean":"2023-10-06T03:48:22.992673+0000","last_became_active":"2023-10-06T03:48:21.959316+0000","last_became_peered":"2023-10-06T03:48:21.959316+0000","last_unstale":"2023-10-06T03:48:22.992673+0000","last_undegraded":"2023-10-06T03:48:22.992673+0000","last_fullsized":"2023-10-06T03:48:22.992673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:11.472736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.942868+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:57.671969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.942708+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:08.115823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.942314+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:49.983388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970732+0000","last_change":"2023-10-06T03:48:22.969730+0000","last_active":"2023-10-06T03:48:22.970732+0000","last_peered":"2023-10-06T03:48:22.970732+0000","last_clean":"2023-10-06T03:48:22.970732+0000","last_became_active":"2023-10-06T03:48:22.969449+0000","last_became_peered":"2023-10-06T03:48:22.969449+0000","last_unstale":"2023-10-06T03:48:22.970732+0000","last_undegraded":"2023-10-06T03:48:22.970732+0000","last_fullsized":"2023-10-06T03:48:22.970732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:41.165824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.939366+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:32.347522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981121+0000","last_change":"2023-10-06T03:48:22.969630+0000","last_active":"2023-10-06T03:48:22.981121+0000","last_peered":"2023-10-06T03:48:22.981121+0000","last_clean":"2023-10-06T03:48:22.981121+0000","last_became_active":"2023-10-06T03:48:22.969362+0000","last_became_peered":"2023-10-06T03:48:22.969362+0000","last_unstale":"2023-10-06T03:48:22.981121+0000","last_undegraded":"2023-10-06T03:48:22.981121+0000","last_fullsized":"2023-10-06T03:48:22.981121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:04.560113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.940198+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:09.475188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.940103+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:47.117650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.933917+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:12.109876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980776+0000","last_change":"2023-10-06T03:48:22.965327+0000","last_active":"2023-10-06T03:48:22.980776+0000","last_peered":"2023-10-06T03:48:22.980776+0000","last_clean":"2023-10-06T03:48:22.980776+0000","last_became_active":"2023-10-06T03:48:22.964956+0000","last_became_peered":"2023-10-06T03:48:22.964956+0000","last_unstale":"2023-10-06T03:48:22.980776+0000","last_undegraded":"2023-10-06T03:48:22.980776+0000","last_fullsized":"2023-10-06T03:48:22.980776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:10.057932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.936898+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:46.538724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.934067+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:09.048281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.937515+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:06.076610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"79'6","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.990295+0000","last_change":"2023-10-06T03:48:21.959258+0000","last_active":"2023-10-06T03:48:22.990295+0000","last_peered":"2023-10-06T03:48:22.990295+0000","last_clean":"2023-10-06T03:48:22.990295+0000","last_became_active":"2023-10-06T03:48:21.958956+0000","last_became_peered":"2023-10-06T03:48:21.958956+0000","last_unstale":"2023-10-06T03:48:22.990295+0000","last_undegraded":"2023-10-06T03:48:22.990295+0000","last_fullsized":"2023-10-06T03:48:22.990295+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:46.538724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.938296+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:14.574278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.944835+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:31.104937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.936824+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:03.746648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980740+0000","last_change":"2023-10-06T03:48:22.971271+0000","last_active":"2023-10-06T03:48:22.980740+0000","last_peered":"2023-10-06T03:48:22.980740+0000","last_clean":"2023-10-06T03:48:22.980740+0000","last_became_active":"2023-10-06T03:48:22.970969+0000","last_became_peered":"2023-10-06T03:48:22.970969+0000","last_unstale":"2023-10-06T03:48:22.980740+0000","last_undegraded":"2023-10-06T03:48:22.980740+0000","last_fullsized":"2023-10-06T03:48:22.980740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:32.235650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.933922+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:35.707577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.944482+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:42:02.476783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.967698+0000","last_change":"2023-10-06T03:48:22.964069+0000","last_active":"2023-10-06T03:48:22.967698+0000","last_peered":"2023-10-06T03:48:22.967698+0000","last_clean":"2023-10-06T03:48:22.967698+0000","last_became_active":"2023-10-06T03:48:22.963794+0000","last_became_peered":"2023-10-06T03:48:22.963794+0000","last_unstale":"2023-10-06T03:48:22.967698+0000","last_undegraded":"2023-10-06T03:48:22.967698+0000","last_fullsized":"2023-10-06T03:48:22.967698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:03.317508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.944325+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:33:38.491221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.942138+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:12.512837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.956473+0000","last_change":"2023-10-06T03:48:21.958507+0000","last_active":"2023-10-06T03:48:22.956473+0000","last_peered":"2023-10-06T03:48:22.956473+0000","last_clean":"2023-10-06T03:48:22.956473+0000","last_became_active":"2023-10-06T03:48:21.958265+0000","last_became_peered":"2023-10-06T03:48:21.958265+0000","last_unstale":"2023-10-06T03:48:22.956473+0000","last_undegraded":"2023-10-06T03:48:22.956473+0000","last_fullsized":"2023-10-06T03:48:22.956473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:57.923641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.937467+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:57.923641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.939322+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:23.949718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981040+0000","last_change":"2023-10-06T03:48:22.964301+0000","last_active":"2023-10-06T03:48:22.981040+0000","last_peered":"2023-10-06T03:48:22.981040+0000","last_clean":"2023-10-06T03:48:22.981040+0000","last_became_active":"2023-10-06T03:48:22.963870+0000","last_became_peered":"2023-10-06T03:48:22.963870+0000","last_unstale":"2023-10-06T03:48:22.981040+0000","last_undegraded":"2023-10-06T03:48:22.981040+0000","last_fullsized":"2023-10-06T03:48:22.981040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:24.959369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.939385+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:31.225998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981729+0000","last_change":"2023-10-06T03:48:22.980115+0000","last_active":"2023-10-06T03:48:22.981729+0000","last_peered":"2023-10-06T03:48:22.981729+0000","last_clean":"2023-10-06T03:48:22.981729+0000","last_became_active":"2023-10-06T03:48:22.979810+0000","last_became_peered":"2023-10-06T03:48:22.979810+0000","last_unstale":"2023-10-06T03:48:22.981729+0000","last_undegraded":"2023-10-06T03:48:22.981729+0000","last_fullsized":"2023-10-06T03:48:22.981729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:20.245038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.955562+0000","last_change":"2023-10-06T03:48:21.961740+0000","last_active":"2023-10-06T03:48:22.955562+0000","last_peered":"2023-10-06T03:48:22.955562+0000","last_clean":"2023-10-06T03:48:22.955562+0000","last_became_active":"2023-10-06T03:48:21.961381+0000","last_became_peered":"2023-10-06T03:48:21.961381+0000","last_unstale":"2023-10-06T03:48:22.955562+0000","last_undegraded":"2023-10-06T03:48:22.955562+0000","last_fullsized":"2023-10-06T03:48:22.955562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:03.746648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.936988+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:14.341349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:17.926861+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:48:17.926861+0000","last_peered":"2023-10-06T03:48:17.926861+0000","last_clean":"2023-10-06T03:48:17.926861+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:48:17.926861+0000","last_undegraded":"2023-10-06T03:48:17.926861+0000","last_fullsized":"2023-10-06T03:48:17.926861+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.939418+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:35:42.534981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:18.679443+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:48:18.679443+0000","last_peered":"2023-10-06T03:48:18.679443+0000","last_clean":"2023-10-06T03:48:18.679443+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:48:18.679443+0000","last_undegraded":"2023-10-06T03:48:18.679443+0000","last_fullsized":"2023-10-06T03:48:18.679443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.942113+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:39.761111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":66,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.955914+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:48:22.955914+0000","last_peered":"2023-10-06T03:48:22.955914+0000","last_clean":"2023-10-06T03:48:22.955914+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:48:22.955914+0000","last_undegraded":"2023-10-06T03:48:22.955914+0000","last_fullsized":"2023-10-06T03:48:22.955914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"70'161","reported_seq":369,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:18.679828+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:48:18.679828+0000","last_peered":"2023-10-06T03:48:18.679828+0000","last_clean":"2023-10-06T03:48:18.679828+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:48:18.679828+0000","last_undegraded":"2023-10-06T03:48:18.679828+0000","last_fullsized":"2023-10-06T03:48:18.679828+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.944301+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:53:49.714558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":64,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:18.679805+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:48:18.679805+0000","last_peered":"2023-10-06T03:48:18.679805+0000","last_clean":"2023-10-06T03:48:18.679805+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:48:18.679805+0000","last_undegraded":"2023-10-06T03:48:18.679805+0000","last_fullsized":"2023-10-06T03:48:18.679805+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.939401+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:05.509003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:18.679808+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:48:18.679808+0000","last_peered":"2023-10-06T03:48:18.679808+0000","last_clean":"2023-10-06T03:48:18.679808+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:48:18.679808+0000","last_undegraded":"2023-10-06T03:48:18.679808+0000","last_fullsized":"2023-10-06T03:48:18.679808+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.942488+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:05.651421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:18.680558+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:48:18.680558+0000","last_peered":"2023-10-06T03:48:18.680558+0000","last_clean":"2023-10-06T03:48:18.680558+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:48:18.680558+0000","last_undegraded":"2023-10-06T03:48:18.680558+0000","last_fullsized":"2023-10-06T03:48:18.680558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.937034+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:11.472736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:17.926237+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:48:17.926237+0000","last_peered":"2023-10-06T03:48:17.926237+0000","last_clean":"2023-10-06T03:48:17.926237+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:48:17.926237+0000","last_undegraded":"2023-10-06T03:48:17.926237+0000","last_fullsized":"2023-10-06T03:48:17.926237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T03:48:21.927129+0000","last_change":"2023-10-06T03:48:21.927129+0000","last_active":"2023-10-06T03:48:21.927129+0000","last_peered":"2023-10-06T03:48:21.927129+0000","last_clean":"2023-10-06T03:48:21.927129+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:21.927129+0000","last_undegraded":"2023-10-06T03:48:21.927129+0000","last_fullsized":"2023-10-06T03:48:21.927129+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T03:48:20.917478+0000","last_change":"2023-10-06T03:48:20.945136+0000","last_active":"2023-10-06T03:48:20.917478+0000","last_peered":"2023-10-06T03:48:20.917478+0000","last_clean":"2023-10-06T03:48:20.917478+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T03:48:20.917478+0000","last_undegraded":"2023-10-06T03:48:20.917478+0000","last_fullsized":"2023-10-06T03:48:20.917478+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:28:36.809130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-06T03:48:17.926244+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:48:17.926244+0000","last_peered":"2023-10-06T03:48:17.926244+0000","last_clean":"2023-10-06T03:48:17.926244+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:48:17.926244+0000","last_undegraded":"2023-10-06T03:48:17.926244+0000","last_fullsized":"2023-10-06T03:48:17.926244+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":26,"acting":26,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2090,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575030,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311048,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060792,"statfs":{"total":96636764160,"available":96318251008,"internally_reserved":0,"allocated":1507328,"data_stored":1047287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71199999999999997,"15min":0.69299999999999995},"min":{"1min":0.53000000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.998,"5min":1.0029999999999999,"15min":1.6100000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.51500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.5649999999999999},"last":0.66200000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.755,"15min":0.749},"min":{"1min":0.63800000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.038,"5min":1.038,"15min":1.5469999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.59199999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.56899999999999995,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":1.5900000000000001},"last":0.496},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.56999999999999995,"5min":0.495,"15min":0.495},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73199999999999998,"15min":0.72499999999999998},"min":{"1min":0.61299999999999999,"5min":0.45900000000000002,"15min":0.39500000000000002},"max":{"1min":1.0089999999999999,"5min":1.016,"15min":1.6279999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76000000000000001,"15min":0.74399999999999999},"min":{"1min":0.624,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96099999999999997,"5min":0.98099999999999998,"15min":1.0409999999999999},"last":0.68899999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79600000000000004,"5min":0.96499999999999997,"15min":1.0049999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.628},"min":{"1min":0.46400000000000002,"5min":0.40999999999999998,"15min":0.39400000000000002},"max":{"1min":0.874,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63900000000000001,"15min":0.64800000000000002},"min":{"1min":0.49299999999999999,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":0.95599999999999996,"5min":1.018,"15min":1.069},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.46800000000000003,"15min":0.436},"max":{"1min":0.91300000000000003,"5min":1.105,"15min":1.105},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72399999999999998,"15min":0.72899999999999998},"min":{"1min":0.58199999999999996,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":1.181,"15min":1.181},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.55200000000000005,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.91500000000000004,"5min":1.02,"15min":1.0209999999999999},"last":0.77500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575031,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306376,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065464,"statfs":{"total":96636764160,"available":96323035136,"internally_reserved":0,"allocated":917504,"data_stored":457897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.49199999999999999,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.92600000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.72999999999999998,"15min":0.71599999999999997},"min":{"1min":0.58099999999999996,"5min":0.41299999999999998,"15min":0.39500000000000002},"max":{"1min":0.96799999999999997,"5min":1.0069999999999999,"15min":1.0449999999999999},"last":0.63700000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.54100000000000004,"5min":0.36499999999999999,"15min":0.33800000000000002},"max":{"1min":0.93100000000000005,"5min":0.98199999999999998,"15min":1.3180000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71799999999999997,"15min":0.70799999999999996},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.35899999999999999},"max":{"1min":0.89600000000000002,"5min":0.995,"15min":0.995},"last":0.76400000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.751,"15min":0.75600000000000001},"min":{"1min":0.57899999999999996,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90900000000000003,"5min":0.995,"15min":3.3319999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73899999999999999,"15min":0.75700000000000001},"min":{"1min":0.48399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94399999999999995,"5min":0.97799999999999998,"15min":3.5139999999999998},"last":0.78700000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.751,"15min":0.747},"min":{"1min":0.58799999999999997,"5min":0.45500000000000002,"15min":0.44600000000000001},"max":{"1min":0.89900000000000002,"5min":1.01,"15min":1.024},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.76300000000000001,"15min":0.78100000000000003},"min":{"1min":0.55200000000000005,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.94299999999999995,"5min":0.99099999999999999,"15min":3.3300000000000001},"last":0.88600000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56699999999999995,"15min":0.58699999999999997},"min":{"1min":0.40699999999999997,"5min":0.376,"15min":0.34799999999999998},"max":{"1min":0.78100000000000003,"5min":0.92600000000000005,"15min":1.0680000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.36499999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.2929999999999999},"last":0.749}]},{"osd":5,"last update":"Fri Oct 6 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65900000000000003,"15min":0.67200000000000004},"min":{"1min":0.48799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84099999999999997,"5min":0.96499999999999997,"15min":1.3340000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.71599999999999997,"15min":0.71699999999999997},"min":{"1min":0.45900000000000002,"5min":0.42899999999999999,"15min":0.41199999999999998},"max":{"1min":0.91000000000000003,"5min":1.0269999999999999,"15min":1.2370000000000001},"last":0.94999999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:47:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.70599999999999996},"min":{"1min":0.46800000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.619},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69499999999999995,"15min":0.71499999999999997},"min":{"1min":0.46800000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86299999999999999,"5min":1.0509999999999999,"15min":1.417},"last":0.90500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575031,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307032,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064808,"statfs":{"total":96636764160,"available":96322363392,"internally_reserved":0,"allocated":1589248,"data_stored":1078628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.47999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80200000000000005,"5min":1.1319999999999999,"15min":1.2889999999999999},"last":0.432}]},{"osd":2,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.5,"15min":0.495},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.26100000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.44,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.94099999999999995},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.60199999999999998},"min":{"1min":0.50800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86699999999999999,"15min":0.93999999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.434,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77400000000000002,"5min":0.90400000000000003,"15min":1.254},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.52500000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0009999999999999,"5min":1.034,"15min":1.3600000000000001},"last":0.626},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.80900000000000005,"15min":0.79800000000000004},"min":{"1min":0.68400000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.476},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.80200000000000005,"15min":0.79300000000000004},"min":{"1min":0.621,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.601},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.59299999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.5189999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.80500000000000005,"15min":0.81000000000000005},"min":{"1min":0.64200000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82499999999999996,"15min":0.81699999999999995},"min":{"1min":0.66700000000000004,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":1.143,"5min":1.1639999999999999,"15min":1.399},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.84799999999999998,"15min":0.83399999999999996},"min":{"1min":0.66900000000000004,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1000000000000001,"5min":1.272,"15min":1.272},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.83399999999999996,"15min":0.82899999999999996},"min":{"1min":0.64400000000000002,"5min":0.53700000000000003,"15min":0.51200000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.556},"last":0.65600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575032,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306372,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065468,"statfs":{"total":96636764160,"available":96323039232,"internally_reserved":0,"allocated":913408,"data_stored":456919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48199999999999998,"15min":0.495},"min":{"1min":0.373,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":0.67800000000000005,"5min":0.81000000000000005,"15min":2.3220000000000001},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.52700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33600000000000002,"15min":0.29599999999999999},"max":{"1min":0.70299999999999996,"5min":0.78500000000000003,"15min":2.1440000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.56499999999999995},"min":{"1min":0.40500000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.76400000000000001,"5min":0.77700000000000002,"15min":2.4329999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54800000000000004,"15min":0.56299999999999994},"min":{"1min":0.40799999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.755,"5min":0.84699999999999998,"15min":2.1819999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51100000000000001,"15min":0.53000000000000003},"min":{"1min":0.41399999999999998,"5min":0.30499999999999999,"15min":0.28399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":2.2709999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61299999999999999,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81000000000000005,"5min":1.0580000000000001,"15min":2.4700000000000002},"last":0.44800000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.83099999999999996,"15min":0.83499999999999996},"min":{"1min":0.69199999999999995,"5min":0.54200000000000004,"15min":0.49399999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.393},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.80900000000000005,"15min":0.83199999999999996},"min":{"1min":0.63400000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.306,"5min":1.306,"15min":2.2320000000000002},"last":0.56399999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.80600000000000005,"15min":0.82999999999999996},"min":{"1min":0.57799999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":2.3660000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.58999999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0780000000000001,"5min":1.0800000000000001,"15min":1.6559999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77900000000000003,"15min":0.80500000000000005},"min":{"1min":0.56999999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.726},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.82399999999999995,"15min":0.85699999999999998},"min":{"1min":0.628,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1100000000000001,"5min":1.119,"15min":2.4009999999999998},"last":0.79900000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.64300000000000002,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2689999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.79800000000000004,"15min":0.83699999999999997},"min":{"1min":0.625,"5min":0.495,"15min":0.495},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.7649999999999999},"last":0.75900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575032,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306420,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065420,"statfs":{"total":96636764160,"available":96322990080,"internally_reserved":0,"allocated":962560,"data_stored":485010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48899999999999999,"15min":0.46700000000000003},"min":{"1min":0.35199999999999998,"5min":0.29699999999999999,"15min":0.28000000000000003},"max":{"1min":0.74199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53400000000000003,"15min":0.52300000000000002},"min":{"1min":0.438,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":0.86199999999999999,"15min":0.89300000000000002},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.34799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72899999999999998,"5min":0.79900000000000004,"15min":0.90000000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.372,"15min":0.34200000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.556},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.504,"5min":0.41399999999999998,"15min":0.38800000000000001},"max":{"1min":0.72899999999999998,"5min":1.78,"15min":1.78},"last":0.439},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81799999999999995,"5min":0.91400000000000003,"15min":1.1910000000000001},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.81899999999999995,"15min":0.81799999999999995},"min":{"1min":0.67600000000000005,"5min":0.51400000000000001,"15min":0.439},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.5620000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.72199999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.008,"5min":1.1419999999999999,"15min":1.2270000000000001},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.82699999999999996,"15min":0.82999999999999996},"min":{"1min":0.68100000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.1140000000000001,"5min":1.177,"15min":1.3},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84399999999999997,"15min":0.84999999999999998},"min":{"1min":0.65500000000000003,"5min":0.55500000000000005,"15min":0.51700000000000002},"max":{"1min":1.0549999999999999,"5min":1.855,"15min":1.855},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82999999999999996,"15min":0.83699999999999997},"min":{"1min":0.68400000000000005,"5min":0.54200000000000004,"15min":0.53600000000000003},"max":{"1min":0.97899999999999998,"5min":1.907,"15min":1.907},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.81899999999999995,"15min":0.81499999999999995},"min":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.51100000000000001},"max":{"1min":1.1080000000000001,"5min":1.1120000000000001,"15min":1.4750000000000001},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.84099999999999997,"15min":0.83499999999999996},"min":{"1min":0.73999999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.125,"5min":1.4379999999999999,"15min":1.4990000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.86399999999999999},"min":{"1min":0.68400000000000005,"5min":0.53100000000000003,"15min":0.497},"max":{"1min":0.99399999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.69599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575031,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306376,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065464,"statfs":{"total":96636764160,"available":96323035136,"internally_reserved":0,"allocated":917504,"data_stored":457897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.33000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56899999999999995,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":0.63800000000000001,"5min":1.04,"15min":1.1699999999999999},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47799999999999998,"15min":0.47299999999999998},"min":{"1min":0.374,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.78800000000000003,"15min":0.79500000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.33800000000000002},"max":{"1min":0.70599999999999996,"5min":0.76700000000000002,"15min":0.83899999999999997},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.34599999999999997},"max":{"1min":0.79700000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.503},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76800000000000002,"5min":0.81999999999999995,"15min":0.97899999999999998},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.78000000000000003,"15min":0.78900000000000003},"min":{"1min":0.51600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.0049999999999999,"5min":1.2030000000000001,"15min":1.4099999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.64100000000000001,"5min":0.57399999999999995,"15min":0.51300000000000001},"max":{"1min":1.0469999999999999,"5min":1.1850000000000001,"15min":1.377},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77400000000000002,"15min":0.78800000000000003},"min":{"1min":0.56699999999999995,"5min":0.51900000000000002,"15min":0.47099999999999997},"max":{"1min":1,"5min":1.256,"15min":1.256},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.61499999999999999,"5min":0.55600000000000005,"15min":0.48099999999999998},"max":{"1min":0.99199999999999999,"5min":1.1439999999999999,"15min":1.306},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81000000000000005,"15min":0.81399999999999995},"min":{"1min":0.61699999999999999,"5min":0.52000000000000002,"15min":0.48599999999999999},"max":{"1min":1.02,"5min":1.292,"15min":1.323},"last":0.629},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.82799999999999996,"15min":0.84999999999999998},"min":{"1min":0.60599999999999998,"5min":0.58599999999999997,"15min":0.49299999999999999},"max":{"1min":1.075,"5min":1.1040000000000001,"15min":1.6020000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.65500000000000003,"5min":0.57799999999999996,"15min":0.505},"max":{"1min":0.98999999999999999,"5min":1.2370000000000001,"15min":1.365},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80000000000000004,"15min":0.80800000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.46300000000000002},"max":{"1min":1.038,"5min":1.1180000000000001,"15min":1.2330000000000001},"last":0.75}]}]},{"osd":4,"up_from":13,"seq":55834575031,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302280,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069560,"statfs":{"total":96636764160,"available":96327229440,"internally_reserved":0,"allocated":917504,"data_stored":457897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.70599999999999996,"15min":0.72099999999999997},"min":{"1min":0.56899999999999995,"5min":0.316,"15min":0.316},"max":{"1min":0.96999999999999997,"5min":1.0109999999999999,"15min":1.3120000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.54800000000000004,"5min":0.316,"15min":0.316},"max":{"1min":0.91500000000000004,"5min":0.95299999999999996,"15min":1.01},"last":0.71199999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.47299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85899999999999999,"5min":1.081,"15min":1.081},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71799999999999997,"15min":0.73699999999999999},"min":{"1min":0.64600000000000002,"5min":0.44,"15min":0.41299999999999998},"max":{"1min":0.89600000000000002,"5min":0.96999999999999997,"15min":1.3380000000000001},"last":0.80700000000000005}]},{"osd":2,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.61399999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.91500000000000004,"5min":1.028,"15min":1.1759999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.747,"15min":0.746},"min":{"1min":0.629,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0109999999999999,"5min":1.022,"15min":1.3089999999999999},"last":0.877}]},{"osd":3,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.63100000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89700000000000002,"5min":1.0049999999999999,"15min":1.0409999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75800000000000001,"15min":0.76900000000000002},"min":{"1min":0.53200000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.83199999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.621,"15min":0.63},"min":{"1min":0.496,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":1.01,"15min":1.085},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.622,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.35099999999999998},"max":{"1min":0.72799999999999998,"5min":0.95399999999999996,"15min":1.1439999999999999},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.753},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.38100000000000001},"max":{"1min":0.90200000000000002,"5min":1.052,"15min":1.361},"last":0.84799999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.39900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":1.1950000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.52000000000000002,"5min":0.504,"15min":0.41299999999999998},"max":{"1min":0.90100000000000002,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.86499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575031,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298180,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073660,"statfs":{"total":96636764160,"available":96331427840,"internally_reserved":0,"allocated":913408,"data_stored":456919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.42299999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94999999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70899999999999996,"15min":0.70699999999999996},"min":{"1min":0.60599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93500000000000005,"5min":0.96899999999999997,"15min":1},"last":0.67300000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.70599999999999996,"15min":0.70799999999999996},"min":{"1min":0.46600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.60299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76400000000000001,"15min":0.77600000000000002},"min":{"1min":0.67900000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98699999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72399999999999998,"15min":0.73099999999999998},"min":{"1min":0.56599999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.996,"5min":0.996,"15min":1.6419999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.754,"15min":0.749},"min":{"1min":0.623,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0189999999999999,"5min":1.117,"15min":1.117},"last":0.68999999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.62,"15min":0.621},"min":{"1min":0.42899999999999999,"5min":0.40000000000000002,"15min":0.38300000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.024},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61799999999999999,"15min":0.63800000000000001},"min":{"1min":0.47099999999999997,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":0.94199999999999995,"5min":1.044,"15min":1.175},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.34999999999999998},"max":{"1min":1.0449999999999999,"5min":1.591,"15min":1.591},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69599999999999995,"15min":0.69899999999999995},"min":{"1min":0.47799999999999998,"5min":0.44600000000000001,"15min":0.40600000000000003},"max":{"1min":1.0389999999999999,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.746}]},{"osd":7,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67000000000000004,"15min":0.68100000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":0.97199999999999998,"5min":1.016,"15min":1.2589999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.72399999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.435},"max":{"1min":1.0880000000000001,"5min":1.278,"15min":1.278},"last":0.76400000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:48:27.827 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:48:27.907 INFO:tasks.ceph.mds.a.smithi055.stderr:2023-10-06T03:48:27.906+0000 7f39441f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:48:28.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.046+0000 7fc115f9d700 1 Processor -- start 2023-10-06T03:48:28.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.046+0000 7fc115f9d700 1 -- start start 2023-10-06T03:48:28.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.048+0000 7fc115f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc110156310 0x7fc1100c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:28.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.048+0000 7fc115f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1100ca860 con 0x7fc1100ae710 2023-10-06T03:48:28.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.048+0000 7fc115f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1100cab40 con 0x7fc1101566e0 2023-10-06T03:48:28.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.048+0000 7fc115f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1100cae20 con 0x7fc110156310 2023-10-06T03:48:28.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.048+0000 7fc114f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc110156310 0x7fc1100c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:28.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.048+0000 7fc11579c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc1100ae710 0x7fc1100c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55830/0 (socket says 172.21.15.55:55830) 2023-10-06T03:48:28.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.048+0000 7fc11579c700 1 -- 172.21.15.55:0/2894377338 learned_addr learned my addr 172.21.15.55:0/2894377338 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:28.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.048+0000 7fc114f9b700 1 -- 172.21.15.55:0/2894377338 >> v1:172.21.15.163:6789/0 conn(0x7fc1101566e0 legacy=0x7fc1100ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:28.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.048+0000 7fc114f9b700 1 -- 172.21.15.55:0/2894377338 >> v1:172.21.15.55:6789/0 conn(0x7fc1100ae710 legacy=0x7fc1100c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:28.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.048+0000 7fc114f9b700 1 -- 172.21.15.55:0/2894377338 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1100cb100 con 0x7fc110156310 2023-10-06T03:48:28.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.049+0000 7fc114f9b700 1 --2- 172.21.15.55:0/2894377338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc110156310 0x7fc1100c9dc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc1040098e0 tx=0x7fc1040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d75528e0d5b2a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:28.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.049+0000 7fc10f7fe700 1 -- 172.21.15.55:0/2894377338 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104009e80 con 0x7fc110156310 2023-10-06T03:48:28.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.049+0000 7fc10f7fe700 1 -- 172.21.15.55:0/2894377338 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10400b760 con 0x7fc110156310 2023-10-06T03:48:28.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.049+0000 7fc10f7fe700 1 -- 172.21.15.55:0/2894377338 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10400bb00 con 0x7fc110156310 2023-10-06T03:48:28.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.049+0000 7fc115f9d700 1 -- 172.21.15.55:0/2894377338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc110156310 msgr2=0x7fc1100c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:28.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.049+0000 7fc115f9d700 1 --2- 172.21.15.55:0/2894377338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc110156310 0x7fc1100c9dc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc1040098e0 tx=0x7fc1040091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.050+0000 7fc115f9d700 1 -- 172.21.15.55:0/2894377338 shutdown_connections 2023-10-06T03:48:28.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.050+0000 7fc115f9d700 1 --2- 172.21.15.55:0/2894377338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc110156310 0x7fc1100c9dc0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.050+0000 7fc115f9d700 1 -- 172.21.15.55:0/2894377338 >> 172.21.15.55:0/2894377338 conn(0x7fc1100cc1f0 msgr2=0x7fc1100cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:28.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.050+0000 7fc115f9d700 1 -- 172.21.15.55:0/2894377338 shutdown_connections 2023-10-06T03:48:28.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.050+0000 7fc115f9d700 1 -- 172.21.15.55:0/2894377338 wait complete. 2023-10-06T03:48:28.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.051+0000 7fc115f9d700 1 Processor -- start 2023-10-06T03:48:28.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.051+0000 7fc115f9d700 1 -- start start 2023-10-06T03:48:28.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.052+0000 7fc115f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1100ae710 0x7fc1100c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:28.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.052+0000 7fc115f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc110156310 0x7fc11014eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:28.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.052+0000 7fc115f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1101566e0 0x7fc110159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:28.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.052+0000 7fc115f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc110158dd0 con 0x7fc110156310 2023-10-06T03:48:28.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.052+0000 7fc115f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1101581e0 con 0x7fc1100ae710 2023-10-06T03:48:28.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.052+0000 7fc115f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1101584a0 con 0x7fc1101566e0 2023-10-06T03:48:28.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.052+0000 7fc10ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc110156310 0x7fc11014eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:28.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.052+0000 7fc10ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc110156310 0x7fc11014eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48816/0 (socket says 172.21.15.55:48816) 2023-10-06T03:48:28.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.052+0000 7fc10ffff700 1 -- 172.21.15.55:0/3294688200 learned_addr learned my addr 172.21.15.55:0/3294688200 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:28.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.053+0000 7fc114f9b700 1 --2- 172.21.15.55:0/3294688200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1100ae710 0x7fc1100c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:28.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.053+0000 7fc11579c700 1 --2- 172.21.15.55:0/3294688200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1101566e0 0x7fc110159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:28.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.053+0000 7fc10ffff700 1 -- 172.21.15.55:0/3294688200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1101566e0 msgr2=0x7fc110159260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:28.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.053+0000 7fc10ffff700 1 --2- 172.21.15.55:0/3294688200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1101566e0 0x7fc110159260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.053+0000 7fc10ffff700 1 -- 172.21.15.55:0/3294688200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1100ae710 msgr2=0x7fc1100c1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:28.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.053+0000 7fc10ffff700 1 --2- 172.21.15.55:0/3294688200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1100ae710 0x7fc1100c1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.053+0000 7fc10ffff700 1 -- 172.21.15.55:0/3294688200 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc104009420 con 0x7fc110156310 2023-10-06T03:48:28.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.053+0000 7fc10ffff700 1 --2- 172.21.15.55:0/3294688200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc110156310 0x7fc11014eb80 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00c590 tx=0x7fc0fc007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:28.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.053+0000 7fc10cff9700 1 -- 172.21.15.55:0/3294688200 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc011660 con 0x7fc110156310 2023-10-06T03:48:28.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.053+0000 7fc10cff9700 1 -- 172.21.15.55:0/3294688200 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc0117c0 con 0x7fc110156310 2023-10-06T03:48:28.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.053+0000 7fc10cff9700 1 -- 172.21.15.55:0/3294688200 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc011b80 con 0x7fc110156310 2023-10-06T03:48:28.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.053+0000 7fc115f9d700 1 -- 172.21.15.55:0/3294688200 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc110158780 con 0x7fc110156310 2023-10-06T03:48:28.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.054+0000 7fc115f9d700 1 -- 172.21.15.55:0/3294688200 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11014f0c0 con 0x7fc110156310 2023-10-06T03:48:28.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.054+0000 7fc115f9d700 1 -- 172.21.15.55:0/3294688200 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc110158f40 con 0x7fc110156310 2023-10-06T03:48:28.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.056+0000 7fc10cff9700 1 -- 172.21.15.55:0/3294688200 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc0fc01c460 con 0x7fc110156310 2023-10-06T03:48:28.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.056+0000 7fc10cff9700 1 --2- 172.21.15.55:0/3294688200 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc0f4066f30 0x7fc0f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:28.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.057+0000 7fc10cff9700 1 -- 172.21.15.55:0/3294688200 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fc0fc0847e0 con 0x7fc110156310 2023-10-06T03:48:28.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.057+0000 7fc114f9b700 1 --2- 172.21.15.55:0/3294688200 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc0f4066f30 0x7fc0f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:28.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.058+0000 7fc114f9b700 1 --2- 172.21.15.55:0/3294688200 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc0f4066f30 0x7fc0f40693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1040091e0 tx=0x7fc104016000 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:28.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.060+0000 7fc10cff9700 1 -- 172.21.15.55:0/3294688200 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0fc0516c0 con 0x7fc110156310 2023-10-06T03:48:28.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.260+0000 7fc115f9d700 1 -- 172.21.15.55:0/3294688200 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc11004c3c0 con 0x7fc0f4066f30 2023-10-06T03:48:28.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.269+0000 7fc10cff9700 1 -- 172.21.15.55:0/3294688200 <== mgr.5483 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310900 (secure 0 0 0) 0x7fc11004c3c0 con 0x7fc0f4066f30 2023-10-06T03:48:28.275 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:28.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.274+0000 7fc115f9d700 1 -- 172.21.15.55:0/3294688200 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc0f4066f30 msgr2=0x7fc0f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:28.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.274+0000 7fc115f9d700 1 --2- 172.21.15.55:0/3294688200 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc0f4066f30 0x7fc0f40693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1040091e0 tx=0x7fc104016000 comp rx=0 tx=0).stop 2023-10-06T03:48:28.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.274+0000 7fc115f9d700 1 -- 172.21.15.55:0/3294688200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc110156310 msgr2=0x7fc11014eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:28.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.274+0000 7fc115f9d700 1 --2- 172.21.15.55:0/3294688200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc110156310 0x7fc11014eb80 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00c590 tx=0x7fc0fc007f30 comp rx=0 tx=0).stop 2023-10-06T03:48:28.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.274+0000 7fc115f9d700 1 -- 172.21.15.55:0/3294688200 shutdown_connections 2023-10-06T03:48:28.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.274+0000 7fc115f9d700 1 --2- 172.21.15.55:0/3294688200 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc0f4066f30 0x7fc0f40693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.274+0000 7fc115f9d700 1 --2- 172.21.15.55:0/3294688200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1101566e0 0x7fc110159260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.274+0000 7fc115f9d700 1 --2- 172.21.15.55:0/3294688200 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc1100ae710 0x7fc1100c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.274+0000 7fc115f9d700 1 --2- 172.21.15.55:0/3294688200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc110156310 0x7fc11014eb80 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.274+0000 7fc115f9d700 1 -- 172.21.15.55:0/3294688200 >> 172.21.15.55:0/3294688200 conn(0x7fc1100cc1f0 msgr2=0x7fc110150a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:28.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.274+0000 7fc115f9d700 1 -- 172.21.15.55:0/3294688200 shutdown_connections 2023-10-06T03:48:28.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.274+0000 7fc115f9d700 1 -- 172.21.15.55:0/3294688200 wait complete. 2023-10-06T03:48:28.278 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:48:28.293 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T03:48:28.016994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444408,"kb_used_data":8760,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752530312,"statfs":{"total":773094113280,"available":770591039488,"internally_reserved":0,"allocated":8970240,"data_stored":5069340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.082923"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981925+0000","last_change":"2023-10-06T03:48:22.978687+0000","last_active":"2023-10-06T03:48:22.981925+0000","last_peered":"2023-10-06T03:48:22.981925+0000","last_clean":"2023-10-06T03:48:22.981925+0000","last_became_active":"2023-10-06T03:48:22.977640+0000","last_became_peered":"2023-10-06T03:48:22.977640+0000","last_unstale":"2023-10-06T03:48:22.981925+0000","last_undegraded":"2023-10-06T03:48:22.981925+0000","last_fullsized":"2023-10-06T03:48:22.981925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:06.518654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980965+0000","last_change":"2023-10-06T03:48:22.965813+0000","last_active":"2023-10-06T03:48:22.980965+0000","last_peered":"2023-10-06T03:48:22.980965+0000","last_clean":"2023-10-06T03:48:22.980965+0000","last_became_active":"2023-10-06T03:48:22.965532+0000","last_became_peered":"2023-10-06T03:48:22.965532+0000","last_unstale":"2023-10-06T03:48:22.980965+0000","last_undegraded":"2023-10-06T03:48:22.980965+0000","last_fullsized":"2023-10-06T03:48:22.980965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:36.717228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.984687+0000","last_change":"2023-10-06T03:48:22.983311+0000","last_active":"2023-10-06T03:48:22.984687+0000","last_peered":"2023-10-06T03:48:22.984687+0000","last_clean":"2023-10-06T03:48:22.984687+0000","last_became_active":"2023-10-06T03:48:22.983010+0000","last_became_peered":"2023-10-06T03:48:22.983010+0000","last_unstale":"2023-10-06T03:48:22.984687+0000","last_undegraded":"2023-10-06T03:48:22.984687+0000","last_fullsized":"2023-10-06T03:48:22.984687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:12.925229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971611+0000","last_change":"2023-10-06T03:48:22.964273+0000","last_active":"2023-10-06T03:48:22.971611+0000","last_peered":"2023-10-06T03:48:22.971611+0000","last_clean":"2023-10-06T03:48:22.971611+0000","last_became_active":"2023-10-06T03:48:22.963978+0000","last_became_peered":"2023-10-06T03:48:22.963978+0000","last_unstale":"2023-10-06T03:48:22.971611+0000","last_undegraded":"2023-10-06T03:48:22.971611+0000","last_fullsized":"2023-10-06T03:48:22.971611+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:13.237222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970479+0000","last_change":"2023-10-06T03:48:22.960460+0000","last_active":"2023-10-06T03:48:22.970479+0000","last_peered":"2023-10-06T03:48:22.970479+0000","last_clean":"2023-10-06T03:48:22.970479+0000","last_became_active":"2023-10-06T03:48:22.959985+0000","last_became_peered":"2023-10-06T03:48:22.959985+0000","last_unstale":"2023-10-06T03:48:22.970479+0000","last_undegraded":"2023-10-06T03:48:22.970479+0000","last_fullsized":"2023-10-06T03:48:22.970479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:39.409932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980760+0000","last_change":"2023-10-06T03:48:22.975932+0000","last_active":"2023-10-06T03:48:22.980760+0000","last_peered":"2023-10-06T03:48:22.980760+0000","last_clean":"2023-10-06T03:48:22.980760+0000","last_became_active":"2023-10-06T03:48:22.975667+0000","last_became_peered":"2023-10-06T03:48:22.975667+0000","last_unstale":"2023-10-06T03:48:22.980760+0000","last_undegraded":"2023-10-06T03:48:22.980760+0000","last_fullsized":"2023-10-06T03:48:22.980760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:49.669934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.985364+0000","last_change":"2023-10-06T03:48:22.980964+0000","last_active":"2023-10-06T03:48:22.985364+0000","last_peered":"2023-10-06T03:48:22.985364+0000","last_clean":"2023-10-06T03:48:22.985364+0000","last_became_active":"2023-10-06T03:48:22.980709+0000","last_became_peered":"2023-10-06T03:48:22.980709+0000","last_unstale":"2023-10-06T03:48:22.985364+0000","last_undegraded":"2023-10-06T03:48:22.985364+0000","last_fullsized":"2023-10-06T03:48:22.985364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:53.633631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.976769+0000","last_change":"2023-10-06T03:48:22.973481+0000","last_active":"2023-10-06T03:48:22.976769+0000","last_peered":"2023-10-06T03:48:22.976769+0000","last_clean":"2023-10-06T03:48:22.976769+0000","last_became_active":"2023-10-06T03:48:22.973158+0000","last_became_peered":"2023-10-06T03:48:22.973158+0000","last_unstale":"2023-10-06T03:48:22.976769+0000","last_undegraded":"2023-10-06T03:48:22.976769+0000","last_fullsized":"2023-10-06T03:48:22.976769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:24.959369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980277+0000","last_change":"2023-10-06T03:48:22.978760+0000","last_active":"2023-10-06T03:48:22.980277+0000","last_peered":"2023-10-06T03:48:22.980277+0000","last_clean":"2023-10-06T03:48:22.980277+0000","last_became_active":"2023-10-06T03:48:22.978459+0000","last_became_peered":"2023-10-06T03:48:22.978459+0000","last_unstale":"2023-10-06T03:48:22.980277+0000","last_undegraded":"2023-10-06T03:48:22.980277+0000","last_fullsized":"2023-10-06T03:48:22.980277+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:30.408052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.983962+0000","last_change":"2023-10-06T03:48:22.983073+0000","last_active":"2023-10-06T03:48:22.983962+0000","last_peered":"2023-10-06T03:48:22.983962+0000","last_clean":"2023-10-06T03:48:22.983962+0000","last_became_active":"2023-10-06T03:48:22.982662+0000","last_became_peered":"2023-10-06T03:48:22.982662+0000","last_unstale":"2023-10-06T03:48:22.983962+0000","last_undegraded":"2023-10-06T03:48:22.983962+0000","last_fullsized":"2023-10-06T03:48:22.983962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:13.237222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.973340+0000","last_change":"2023-10-06T03:48:22.969513+0000","last_active":"2023-10-06T03:48:22.973340+0000","last_peered":"2023-10-06T03:48:22.973340+0000","last_clean":"2023-10-06T03:48:22.973340+0000","last_became_active":"2023-10-06T03:48:22.969211+0000","last_became_peered":"2023-10-06T03:48:22.969211+0000","last_unstale":"2023-10-06T03:48:22.973340+0000","last_undegraded":"2023-10-06T03:48:22.973340+0000","last_fullsized":"2023-10-06T03:48:22.973340+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:58.681620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.975883+0000","last_change":"2023-10-06T03:48:22.973519+0000","last_active":"2023-10-06T03:48:22.975883+0000","last_peered":"2023-10-06T03:48:22.975883+0000","last_clean":"2023-10-06T03:48:22.975883+0000","last_became_active":"2023-10-06T03:48:22.973239+0000","last_became_peered":"2023-10-06T03:48:22.973239+0000","last_unstale":"2023-10-06T03:48:22.975883+0000","last_undegraded":"2023-10-06T03:48:22.975883+0000","last_fullsized":"2023-10-06T03:48:22.975883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:04.560113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.976082+0000","last_change":"2023-10-06T03:48:22.970347+0000","last_active":"2023-10-06T03:48:22.976082+0000","last_peered":"2023-10-06T03:48:22.976082+0000","last_clean":"2023-10-06T03:48:22.976082+0000","last_became_active":"2023-10-06T03:48:22.970084+0000","last_became_peered":"2023-10-06T03:48:22.970084+0000","last_unstale":"2023-10-06T03:48:22.976082+0000","last_undegraded":"2023-10-06T03:48:22.976082+0000","last_fullsized":"2023-10-06T03:48:22.976082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:23.317887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.985566+0000","last_change":"2023-10-06T03:48:22.984621+0000","last_active":"2023-10-06T03:48:22.985566+0000","last_peered":"2023-10-06T03:48:22.985566+0000","last_clean":"2023-10-06T03:48:22.985566+0000","last_became_active":"2023-10-06T03:48:22.984339+0000","last_became_peered":"2023-10-06T03:48:22.984339+0000","last_unstale":"2023-10-06T03:48:22.985566+0000","last_undegraded":"2023-10-06T03:48:22.985566+0000","last_fullsized":"2023-10-06T03:48:22.985566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:35.725454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.984743+0000","last_change":"2023-10-06T03:48:22.983782+0000","last_active":"2023-10-06T03:48:22.984743+0000","last_peered":"2023-10-06T03:48:22.984743+0000","last_clean":"2023-10-06T03:48:22.984743+0000","last_became_active":"2023-10-06T03:48:22.982670+0000","last_became_peered":"2023-10-06T03:48:22.982670+0000","last_unstale":"2023-10-06T03:48:22.984743+0000","last_undegraded":"2023-10-06T03:48:22.984743+0000","last_fullsized":"2023-10-06T03:48:22.984743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:50.215524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.978789+0000","last_change":"2023-10-06T03:48:22.972912+0000","last_active":"2023-10-06T03:48:22.978789+0000","last_peered":"2023-10-06T03:48:22.978789+0000","last_clean":"2023-10-06T03:48:22.978789+0000","last_became_active":"2023-10-06T03:48:22.972623+0000","last_became_peered":"2023-10-06T03:48:22.972623+0000","last_unstale":"2023-10-06T03:48:22.978789+0000","last_undegraded":"2023-10-06T03:48:22.978789+0000","last_fullsized":"2023-10-06T03:48:22.978789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:44.334010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981079+0000","last_change":"2023-10-06T03:48:22.973112+0000","last_active":"2023-10-06T03:48:22.981079+0000","last_peered":"2023-10-06T03:48:22.981079+0000","last_clean":"2023-10-06T03:48:22.981079+0000","last_became_active":"2023-10-06T03:48:22.972785+0000","last_became_peered":"2023-10-06T03:48:22.972785+0000","last_unstale":"2023-10-06T03:48:22.981079+0000","last_undegraded":"2023-10-06T03:48:22.981079+0000","last_fullsized":"2023-10-06T03:48:22.981079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:14.661907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971574+0000","last_change":"2023-10-06T03:48:22.964446+0000","last_active":"2023-10-06T03:48:22.971574+0000","last_peered":"2023-10-06T03:48:22.971574+0000","last_clean":"2023-10-06T03:48:22.971574+0000","last_became_active":"2023-10-06T03:48:22.964198+0000","last_became_peered":"2023-10-06T03:48:22.964198+0000","last_unstale":"2023-10-06T03:48:22.971574+0000","last_undegraded":"2023-10-06T03:48:22.971574+0000","last_fullsized":"2023-10-06T03:48:22.971574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:13.522488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972748+0000","last_change":"2023-10-06T03:48:22.964606+0000","last_active":"2023-10-06T03:48:22.972748+0000","last_peered":"2023-10-06T03:48:22.972748+0000","last_clean":"2023-10-06T03:48:22.972748+0000","last_became_active":"2023-10-06T03:48:22.964252+0000","last_became_peered":"2023-10-06T03:48:22.964252+0000","last_unstale":"2023-10-06T03:48:22.972748+0000","last_undegraded":"2023-10-06T03:48:22.972748+0000","last_fullsized":"2023-10-06T03:48:22.972748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:59.411057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981867+0000","last_change":"2023-10-06T03:48:22.972302+0000","last_active":"2023-10-06T03:48:22.981867+0000","last_peered":"2023-10-06T03:48:22.981867+0000","last_clean":"2023-10-06T03:48:22.981867+0000","last_became_active":"2023-10-06T03:48:22.972024+0000","last_became_peered":"2023-10-06T03:48:22.972024+0000","last_unstale":"2023-10-06T03:48:22.981867+0000","last_undegraded":"2023-10-06T03:48:22.981867+0000","last_fullsized":"2023-10-06T03:48:22.981867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:00.072378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.974860+0000","last_change":"2023-10-06T03:48:22.968910+0000","last_active":"2023-10-06T03:48:22.974860+0000","last_peered":"2023-10-06T03:48:22.974860+0000","last_clean":"2023-10-06T03:48:22.974860+0000","last_became_active":"2023-10-06T03:48:22.968335+0000","last_became_peered":"2023-10-06T03:48:22.968335+0000","last_unstale":"2023-10-06T03:48:22.974860+0000","last_undegraded":"2023-10-06T03:48:22.974860+0000","last_fullsized":"2023-10-06T03:48:22.974860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:10.057932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981805+0000","last_change":"2023-10-06T03:48:22.975177+0000","last_active":"2023-10-06T03:48:22.981805+0000","last_peered":"2023-10-06T03:48:22.981805+0000","last_clean":"2023-10-06T03:48:22.981805+0000","last_became_active":"2023-10-06T03:48:22.974184+0000","last_became_peered":"2023-10-06T03:48:22.974184+0000","last_unstale":"2023-10-06T03:48:22.981805+0000","last_undegraded":"2023-10-06T03:48:22.981805+0000","last_fullsized":"2023-10-06T03:48:22.981805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:44.273052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981912+0000","last_change":"2023-10-06T03:48:22.973196+0000","last_active":"2023-10-06T03:48:22.981912+0000","last_peered":"2023-10-06T03:48:22.981912+0000","last_clean":"2023-10-06T03:48:22.981912+0000","last_became_active":"2023-10-06T03:48:22.972823+0000","last_became_peered":"2023-10-06T03:48:22.972823+0000","last_unstale":"2023-10-06T03:48:22.981912+0000","last_undegraded":"2023-10-06T03:48:22.981912+0000","last_fullsized":"2023-10-06T03:48:22.981912+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:02.782253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.985661+0000","last_change":"2023-10-06T03:48:22.984009+0000","last_active":"2023-10-06T03:48:22.985661+0000","last_peered":"2023-10-06T03:48:22.985661+0000","last_clean":"2023-10-06T03:48:22.985661+0000","last_became_active":"2023-10-06T03:48:22.983746+0000","last_became_peered":"2023-10-06T03:48:22.983746+0000","last_unstale":"2023-10-06T03:48:22.985661+0000","last_undegraded":"2023-10-06T03:48:22.985661+0000","last_fullsized":"2023-10-06T03:48:22.985661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:12.970745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.976801+0000","last_change":"2023-10-06T03:48:22.969794+0000","last_active":"2023-10-06T03:48:22.976801+0000","last_peered":"2023-10-06T03:48:22.976801+0000","last_clean":"2023-10-06T03:48:22.976801+0000","last_became_active":"2023-10-06T03:48:22.969509+0000","last_became_peered":"2023-10-06T03:48:22.969509+0000","last_unstale":"2023-10-06T03:48:22.976801+0000","last_undegraded":"2023-10-06T03:48:22.976801+0000","last_fullsized":"2023-10-06T03:48:22.976801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:00.871577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.975985+0000","last_change":"2023-10-06T03:48:22.969265+0000","last_active":"2023-10-06T03:48:22.975985+0000","last_peered":"2023-10-06T03:48:22.975985+0000","last_clean":"2023-10-06T03:48:22.975985+0000","last_became_active":"2023-10-06T03:48:22.968581+0000","last_became_peered":"2023-10-06T03:48:22.968581+0000","last_unstale":"2023-10-06T03:48:22.975985+0000","last_undegraded":"2023-10-06T03:48:22.975985+0000","last_fullsized":"2023-10-06T03:48:22.975985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:15.583929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972008+0000","last_change":"2023-10-06T03:48:22.965736+0000","last_active":"2023-10-06T03:48:22.972008+0000","last_peered":"2023-10-06T03:48:22.972008+0000","last_clean":"2023-10-06T03:48:22.972008+0000","last_became_active":"2023-10-06T03:48:22.965464+0000","last_became_peered":"2023-10-06T03:48:22.965464+0000","last_unstale":"2023-10-06T03:48:22.972008+0000","last_undegraded":"2023-10-06T03:48:22.972008+0000","last_fullsized":"2023-10-06T03:48:22.972008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:50.215524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980795+0000","last_change":"2023-10-06T03:48:22.969599+0000","last_active":"2023-10-06T03:48:22.980795+0000","last_peered":"2023-10-06T03:48:22.980795+0000","last_clean":"2023-10-06T03:48:22.980795+0000","last_became_active":"2023-10-06T03:48:22.969334+0000","last_became_peered":"2023-10-06T03:48:22.969334+0000","last_unstale":"2023-10-06T03:48:22.980795+0000","last_undegraded":"2023-10-06T03:48:22.980795+0000","last_fullsized":"2023-10-06T03:48:22.980795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:33.357173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.979756+0000","last_change":"2023-10-06T03:48:22.977428+0000","last_active":"2023-10-06T03:48:22.979756+0000","last_peered":"2023-10-06T03:48:22.979756+0000","last_clean":"2023-10-06T03:48:22.979756+0000","last_became_active":"2023-10-06T03:48:22.977087+0000","last_became_peered":"2023-10-06T03:48:22.977087+0000","last_unstale":"2023-10-06T03:48:22.979756+0000","last_undegraded":"2023-10-06T03:48:22.979756+0000","last_fullsized":"2023-10-06T03:48:22.979756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:17.224716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.975912+0000","last_change":"2023-10-06T03:48:22.974325+0000","last_active":"2023-10-06T03:48:22.975912+0000","last_peered":"2023-10-06T03:48:22.975912+0000","last_clean":"2023-10-06T03:48:22.975912+0000","last_became_active":"2023-10-06T03:48:22.974082+0000","last_became_peered":"2023-10-06T03:48:22.974082+0000","last_unstale":"2023-10-06T03:48:22.975912+0000","last_undegraded":"2023-10-06T03:48:22.975912+0000","last_fullsized":"2023-10-06T03:48:22.975912+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:14.661907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972101+0000","last_change":"2023-10-06T03:48:22.968457+0000","last_active":"2023-10-06T03:48:22.972101+0000","last_peered":"2023-10-06T03:48:22.972101+0000","last_clean":"2023-10-06T03:48:22.972101+0000","last_became_active":"2023-10-06T03:48:22.968182+0000","last_became_peered":"2023-10-06T03:48:22.968182+0000","last_unstale":"2023-10-06T03:48:22.972101+0000","last_undegraded":"2023-10-06T03:48:22.972101+0000","last_fullsized":"2023-10-06T03:48:22.972101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:47.901242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980159+0000","last_change":"2023-10-06T03:48:22.968721+0000","last_active":"2023-10-06T03:48:22.980159+0000","last_peered":"2023-10-06T03:48:22.980159+0000","last_clean":"2023-10-06T03:48:22.980159+0000","last_became_active":"2023-10-06T03:48:22.968451+0000","last_became_peered":"2023-10-06T03:48:22.968451+0000","last_unstale":"2023-10-06T03:48:22.980159+0000","last_undegraded":"2023-10-06T03:48:22.980159+0000","last_fullsized":"2023-10-06T03:48:22.980159+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:47:58.626319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970581+0000","last_change":"2023-10-06T03:48:22.963394+0000","last_active":"2023-10-06T03:48:22.970581+0000","last_peered":"2023-10-06T03:48:22.970581+0000","last_clean":"2023-10-06T03:48:22.970581+0000","last_became_active":"2023-10-06T03:48:22.963073+0000","last_became_peered":"2023-10-06T03:48:22.963073+0000","last_unstale":"2023-10-06T03:48:22.970581+0000","last_undegraded":"2023-10-06T03:48:22.970581+0000","last_fullsized":"2023-10-06T03:48:22.970581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:15.351000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"79'4","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.992673+0000","last_change":"2023-10-06T03:48:21.959573+0000","last_active":"2023-10-06T03:48:22.992673+0000","last_peered":"2023-10-06T03:48:22.992673+0000","last_clean":"2023-10-06T03:48:22.992673+0000","last_became_active":"2023-10-06T03:48:21.959316+0000","last_became_peered":"2023-10-06T03:48:21.959316+0000","last_unstale":"2023-10-06T03:48:22.992673+0000","last_undegraded":"2023-10-06T03:48:22.992673+0000","last_fullsized":"2023-10-06T03:48:22.992673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:11.472736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971988+0000","last_change":"2023-10-06T03:48:22.961954+0000","last_active":"2023-10-06T03:48:22.971988+0000","last_peered":"2023-10-06T03:48:22.971988+0000","last_clean":"2023-10-06T03:48:22.971988+0000","last_became_active":"2023-10-06T03:48:22.961642+0000","last_became_peered":"2023-10-06T03:48:22.961642+0000","last_unstale":"2023-10-06T03:48:22.971988+0000","last_undegraded":"2023-10-06T03:48:22.971988+0000","last_fullsized":"2023-10-06T03:48:22.971988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:40.770762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.986713+0000","last_change":"2023-10-06T03:48:21.954192+0000","last_active":"2023-10-06T03:48:22.986713+0000","last_peered":"2023-10-06T03:48:22.986713+0000","last_clean":"2023-10-06T03:48:22.986713+0000","last_became_active":"2023-10-06T03:48:21.953846+0000","last_became_peered":"2023-10-06T03:48:21.953846+0000","last_unstale":"2023-10-06T03:48:22.986713+0000","last_undegraded":"2023-10-06T03:48:22.986713+0000","last_fullsized":"2023-10-06T03:48:22.986713+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:38.007175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980734+0000","last_change":"2023-10-06T03:48:22.967986+0000","last_active":"2023-10-06T03:48:22.980734+0000","last_peered":"2023-10-06T03:48:22.980734+0000","last_clean":"2023-10-06T03:48:22.980734+0000","last_became_active":"2023-10-06T03:48:22.967743+0000","last_became_peered":"2023-10-06T03:48:22.967743+0000","last_unstale":"2023-10-06T03:48:22.980734+0000","last_undegraded":"2023-10-06T03:48:22.980734+0000","last_fullsized":"2023-10-06T03:48:22.980734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:46.815803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.988858+0000","last_change":"2023-10-06T03:48:21.952725+0000","last_active":"2023-10-06T03:48:22.988858+0000","last_peered":"2023-10-06T03:48:22.988858+0000","last_clean":"2023-10-06T03:48:22.988858+0000","last_became_active":"2023-10-06T03:48:21.952307+0000","last_became_peered":"2023-10-06T03:48:21.952307+0000","last_unstale":"2023-10-06T03:48:22.988858+0000","last_undegraded":"2023-10-06T03:48:22.988858+0000","last_fullsized":"2023-10-06T03:48:22.988858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:16.562701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.985371+0000","last_change":"2023-10-06T03:48:22.983206+0000","last_active":"2023-10-06T03:48:22.985371+0000","last_peered":"2023-10-06T03:48:22.985371+0000","last_clean":"2023-10-06T03:48:22.985371+0000","last_became_active":"2023-10-06T03:48:22.982830+0000","last_became_peered":"2023-10-06T03:48:22.982830+0000","last_unstale":"2023-10-06T03:48:22.985371+0000","last_undegraded":"2023-10-06T03:48:22.985371+0000","last_fullsized":"2023-10-06T03:48:22.985371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:47.901242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.987882+0000","last_change":"2023-10-06T03:48:21.954325+0000","last_active":"2023-10-06T03:48:22.987882+0000","last_peered":"2023-10-06T03:48:22.987882+0000","last_clean":"2023-10-06T03:48:22.987882+0000","last_became_active":"2023-10-06T03:48:21.954034+0000","last_became_peered":"2023-10-06T03:48:21.954034+0000","last_unstale":"2023-10-06T03:48:22.987882+0000","last_undegraded":"2023-10-06T03:48:22.987882+0000","last_fullsized":"2023-10-06T03:48:22.987882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:01.772602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970732+0000","last_change":"2023-10-06T03:48:22.969730+0000","last_active":"2023-10-06T03:48:22.970732+0000","last_peered":"2023-10-06T03:48:22.970732+0000","last_clean":"2023-10-06T03:48:22.970732+0000","last_became_active":"2023-10-06T03:48:22.969449+0000","last_became_peered":"2023-10-06T03:48:22.969449+0000","last_unstale":"2023-10-06T03:48:22.970732+0000","last_undegraded":"2023-10-06T03:48:22.970732+0000","last_fullsized":"2023-10-06T03:48:22.970732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:41.165824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.984300+0000","last_change":"2023-10-06T03:48:21.946173+0000","last_active":"2023-10-06T03:48:22.984300+0000","last_peered":"2023-10-06T03:48:22.984300+0000","last_clean":"2023-10-06T03:48:22.984300+0000","last_became_active":"2023-10-06T03:48:21.945822+0000","last_became_peered":"2023-10-06T03:48:21.945822+0000","last_unstale":"2023-10-06T03:48:22.984300+0000","last_undegraded":"2023-10-06T03:48:22.984300+0000","last_fullsized":"2023-10-06T03:48:22.984300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:46.891591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981121+0000","last_change":"2023-10-06T03:48:22.969630+0000","last_active":"2023-10-06T03:48:22.981121+0000","last_peered":"2023-10-06T03:48:22.981121+0000","last_clean":"2023-10-06T03:48:22.981121+0000","last_became_active":"2023-10-06T03:48:22.969362+0000","last_became_peered":"2023-10-06T03:48:22.969362+0000","last_unstale":"2023-10-06T03:48:22.981121+0000","last_undegraded":"2023-10-06T03:48:22.981121+0000","last_fullsized":"2023-10-06T03:48:22.981121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:04.560113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.986941+0000","last_change":"2023-10-06T03:48:21.944675+0000","last_active":"2023-10-06T03:48:22.986941+0000","last_peered":"2023-10-06T03:48:22.986941+0000","last_clean":"2023-10-06T03:48:22.986941+0000","last_became_active":"2023-10-06T03:48:21.944421+0000","last_became_peered":"2023-10-06T03:48:21.944421+0000","last_unstale":"2023-10-06T03:48:22.986941+0000","last_undegraded":"2023-10-06T03:48:22.986941+0000","last_fullsized":"2023-10-06T03:48:22.986941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:12.199710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.974865+0000","last_change":"2023-10-06T03:48:22.966222+0000","last_active":"2023-10-06T03:48:22.974865+0000","last_peered":"2023-10-06T03:48:22.974865+0000","last_clean":"2023-10-06T03:48:22.974865+0000","last_became_active":"2023-10-06T03:48:22.965935+0000","last_became_peered":"2023-10-06T03:48:22.965935+0000","last_unstale":"2023-10-06T03:48:22.974865+0000","last_undegraded":"2023-10-06T03:48:22.974865+0000","last_fullsized":"2023-10-06T03:48:22.974865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:24.367540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.961390+0000","last_change":"2023-10-06T03:48:21.951726+0000","last_active":"2023-10-06T03:48:22.961390+0000","last_peered":"2023-10-06T03:48:22.961390+0000","last_clean":"2023-10-06T03:48:22.961390+0000","last_became_active":"2023-10-06T03:48:21.951477+0000","last_became_peered":"2023-10-06T03:48:21.951477+0000","last_unstale":"2023-10-06T03:48:22.961390+0000","last_undegraded":"2023-10-06T03:48:22.961390+0000","last_fullsized":"2023-10-06T03:48:22.961390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:16.215064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.982148+0000","last_change":"2023-10-06T03:48:22.974712+0000","last_active":"2023-10-06T03:48:22.982148+0000","last_peered":"2023-10-06T03:48:22.982148+0000","last_clean":"2023-10-06T03:48:22.982148+0000","last_became_active":"2023-10-06T03:48:22.974451+0000","last_became_peered":"2023-10-06T03:48:22.974451+0000","last_unstale":"2023-10-06T03:48:22.982148+0000","last_undegraded":"2023-10-06T03:48:22.982148+0000","last_fullsized":"2023-10-06T03:48:22.982148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:50.993039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.953339+0000","last_change":"2023-10-06T03:48:21.954823+0000","last_active":"2023-10-06T03:48:22.953339+0000","last_peered":"2023-10-06T03:48:22.953339+0000","last_clean":"2023-10-06T03:48:22.953339+0000","last_became_active":"2023-10-06T03:48:21.954575+0000","last_became_peered":"2023-10-06T03:48:21.954575+0000","last_unstale":"2023-10-06T03:48:22.953339+0000","last_undegraded":"2023-10-06T03:48:22.953339+0000","last_fullsized":"2023-10-06T03:48:22.953339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:58.471188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980776+0000","last_change":"2023-10-06T03:48:22.965327+0000","last_active":"2023-10-06T03:48:22.980776+0000","last_peered":"2023-10-06T03:48:22.980776+0000","last_clean":"2023-10-06T03:48:22.980776+0000","last_became_active":"2023-10-06T03:48:22.964956+0000","last_became_peered":"2023-10-06T03:48:22.964956+0000","last_unstale":"2023-10-06T03:48:22.980776+0000","last_undegraded":"2023-10-06T03:48:22.980776+0000","last_fullsized":"2023-10-06T03:48:22.980776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:10.057932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.966607+0000","last_change":"2023-10-06T03:48:21.962473+0000","last_active":"2023-10-06T03:48:22.966607+0000","last_peered":"2023-10-06T03:48:22.966607+0000","last_clean":"2023-10-06T03:48:22.966607+0000","last_became_active":"2023-10-06T03:48:21.962144+0000","last_became_peered":"2023-10-06T03:48:21.962144+0000","last_unstale":"2023-10-06T03:48:22.966607+0000","last_undegraded":"2023-10-06T03:48:22.966607+0000","last_fullsized":"2023-10-06T03:48:22.966607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:23.381899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981836+0000","last_change":"2023-10-06T03:48:22.970666+0000","last_active":"2023-10-06T03:48:22.981836+0000","last_peered":"2023-10-06T03:48:22.981836+0000","last_clean":"2023-10-06T03:48:22.981836+0000","last_became_active":"2023-10-06T03:48:22.970339+0000","last_became_peered":"2023-10-06T03:48:22.970339+0000","last_unstale":"2023-10-06T03:48:22.981836+0000","last_undegraded":"2023-10-06T03:48:22.981836+0000","last_fullsized":"2023-10-06T03:48:22.981836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:09.125474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.953769+0000","last_change":"2023-10-06T03:48:21.956450+0000","last_active":"2023-10-06T03:48:22.953769+0000","last_peered":"2023-10-06T03:48:22.953769+0000","last_clean":"2023-10-06T03:48:22.953769+0000","last_became_active":"2023-10-06T03:48:21.956099+0000","last_became_peered":"2023-10-06T03:48:21.956099+0000","last_unstale":"2023-10-06T03:48:22.953769+0000","last_undegraded":"2023-10-06T03:48:22.953769+0000","last_fullsized":"2023-10-06T03:48:22.953769+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:03.550462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.978245+0000","last_change":"2023-10-06T03:48:22.977370+0000","last_active":"2023-10-06T03:48:22.978245+0000","last_peered":"2023-10-06T03:48:22.978245+0000","last_clean":"2023-10-06T03:48:22.978245+0000","last_became_active":"2023-10-06T03:48:22.977088+0000","last_became_peered":"2023-10-06T03:48:22.977088+0000","last_unstale":"2023-10-06T03:48:22.978245+0000","last_undegraded":"2023-10-06T03:48:22.978245+0000","last_fullsized":"2023-10-06T03:48:22.978245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:47.548376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"79'3","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.992610+0000","last_change":"2023-10-06T03:48:21.962149+0000","last_active":"2023-10-06T03:48:22.992610+0000","last_peered":"2023-10-06T03:48:22.992610+0000","last_clean":"2023-10-06T03:48:22.992610+0000","last_became_active":"2023-10-06T03:48:21.961686+0000","last_became_peered":"2023-10-06T03:48:21.961686+0000","last_unstale":"2023-10-06T03:48:22.992610+0000","last_undegraded":"2023-10-06T03:48:22.992610+0000","last_fullsized":"2023-10-06T03:48:22.992610+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:14.197511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.973148+0000","last_change":"2023-10-06T03:48:22.968754+0000","last_active":"2023-10-06T03:48:22.973148+0000","last_peered":"2023-10-06T03:48:22.973148+0000","last_clean":"2023-10-06T03:48:22.973148+0000","last_became_active":"2023-10-06T03:48:22.968469+0000","last_became_peered":"2023-10-06T03:48:22.968469+0000","last_unstale":"2023-10-06T03:48:22.973148+0000","last_undegraded":"2023-10-06T03:48:22.973148+0000","last_fullsized":"2023-10-06T03:48:22.973148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:30.408052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"79'6","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.990295+0000","last_change":"2023-10-06T03:48:21.959258+0000","last_active":"2023-10-06T03:48:22.990295+0000","last_peered":"2023-10-06T03:48:22.990295+0000","last_clean":"2023-10-06T03:48:22.990295+0000","last_became_active":"2023-10-06T03:48:21.958956+0000","last_became_peered":"2023-10-06T03:48:21.958956+0000","last_unstale":"2023-10-06T03:48:22.990295+0000","last_undegraded":"2023-10-06T03:48:22.990295+0000","last_fullsized":"2023-10-06T03:48:22.990295+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:46.538724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.977036+0000","last_change":"2023-10-06T03:48:22.974061+0000","last_active":"2023-10-06T03:48:22.977036+0000","last_peered":"2023-10-06T03:48:22.977036+0000","last_clean":"2023-10-06T03:48:22.977036+0000","last_became_active":"2023-10-06T03:48:22.973763+0000","last_became_peered":"2023-10-06T03:48:22.973763+0000","last_unstale":"2023-10-06T03:48:22.977036+0000","last_undegraded":"2023-10-06T03:48:22.977036+0000","last_fullsized":"2023-10-06T03:48:22.977036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:09.125474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.992491+0000","last_change":"2023-10-06T03:48:21.962189+0000","last_active":"2023-10-06T03:48:22.992491+0000","last_peered":"2023-10-06T03:48:22.992491+0000","last_clean":"2023-10-06T03:48:22.992491+0000","last_became_active":"2023-10-06T03:48:21.961581+0000","last_became_peered":"2023-10-06T03:48:21.961581+0000","last_unstale":"2023-10-06T03:48:22.992491+0000","last_undegraded":"2023-10-06T03:48:22.992491+0000","last_fullsized":"2023-10-06T03:48:22.992491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:59.861926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.975953+0000","last_change":"2023-10-06T03:48:22.971613+0000","last_active":"2023-10-06T03:48:22.975953+0000","last_peered":"2023-10-06T03:48:22.975953+0000","last_clean":"2023-10-06T03:48:22.975953+0000","last_became_active":"2023-10-06T03:48:22.971307+0000","last_became_peered":"2023-10-06T03:48:22.971307+0000","last_unstale":"2023-10-06T03:48:22.975953+0000","last_undegraded":"2023-10-06T03:48:22.975953+0000","last_fullsized":"2023-10-06T03:48:22.975953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:36.717228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958572+0000","last_change":"2023-10-06T03:48:21.955946+0000","last_active":"2023-10-06T03:48:22.958572+0000","last_peered":"2023-10-06T03:48:22.958572+0000","last_clean":"2023-10-06T03:48:22.958572+0000","last_became_active":"2023-10-06T03:48:21.955694+0000","last_became_peered":"2023-10-06T03:48:21.955694+0000","last_unstale":"2023-10-06T03:48:22.958572+0000","last_undegraded":"2023-10-06T03:48:22.958572+0000","last_fullsized":"2023-10-06T03:48:22.958572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:23.357889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.974849+0000","last_change":"2023-10-06T03:48:22.965598+0000","last_active":"2023-10-06T03:48:22.974849+0000","last_peered":"2023-10-06T03:48:22.974849+0000","last_clean":"2023-10-06T03:48:22.974849+0000","last_became_active":"2023-10-06T03:48:22.965115+0000","last_became_peered":"2023-10-06T03:48:22.965115+0000","last_unstale":"2023-10-06T03:48:22.974849+0000","last_undegraded":"2023-10-06T03:48:22.974849+0000","last_fullsized":"2023-10-06T03:48:22.974849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:26.181100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.964644+0000","last_change":"2023-10-06T03:48:21.959481+0000","last_active":"2023-10-06T03:48:22.964644+0000","last_peered":"2023-10-06T03:48:22.964644+0000","last_clean":"2023-10-06T03:48:22.964644+0000","last_became_active":"2023-10-06T03:48:21.959233+0000","last_became_peered":"2023-10-06T03:48:21.959233+0000","last_unstale":"2023-10-06T03:48:22.964644+0000","last_undegraded":"2023-10-06T03:48:22.964644+0000","last_fullsized":"2023-10-06T03:48:22.964644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:35.707577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980740+0000","last_change":"2023-10-06T03:48:22.971271+0000","last_active":"2023-10-06T03:48:22.980740+0000","last_peered":"2023-10-06T03:48:22.980740+0000","last_clean":"2023-10-06T03:48:22.980740+0000","last_became_active":"2023-10-06T03:48:22.970969+0000","last_became_peered":"2023-10-06T03:48:22.970969+0000","last_unstale":"2023-10-06T03:48:22.980740+0000","last_undegraded":"2023-10-06T03:48:22.980740+0000","last_fullsized":"2023-10-06T03:48:22.980740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:32.235650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"79'8","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.988873+0000","last_change":"2023-10-06T03:48:21.959296+0000","last_active":"2023-10-06T03:48:22.988873+0000","last_peered":"2023-10-06T03:48:22.988873+0000","last_clean":"2023-10-06T03:48:22.988873+0000","last_became_active":"2023-10-06T03:48:21.958948+0000","last_became_peered":"2023-10-06T03:48:21.958948+0000","last_unstale":"2023-10-06T03:48:22.988873+0000","last_undegraded":"2023-10-06T03:48:22.988873+0000","last_fullsized":"2023-10-06T03:48:22.988873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:13.652256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.983389+0000","last_change":"2023-10-06T03:48:22.979534+0000","last_active":"2023-10-06T03:48:22.983389+0000","last_peered":"2023-10-06T03:48:22.983389+0000","last_clean":"2023-10-06T03:48:22.983389+0000","last_became_active":"2023-10-06T03:48:22.979272+0000","last_became_peered":"2023-10-06T03:48:22.979272+0000","last_unstale":"2023-10-06T03:48:22.983389+0000","last_undegraded":"2023-10-06T03:48:22.983389+0000","last_fullsized":"2023-10-06T03:48:22.983389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:03.329123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.990320+0000","last_change":"2023-10-06T03:48:21.959483+0000","last_active":"2023-10-06T03:48:22.990320+0000","last_peered":"2023-10-06T03:48:22.990320+0000","last_clean":"2023-10-06T03:48:22.990320+0000","last_became_active":"2023-10-06T03:48:21.959233+0000","last_became_peered":"2023-10-06T03:48:21.959233+0000","last_unstale":"2023-10-06T03:48:22.990320+0000","last_undegraded":"2023-10-06T03:48:22.990320+0000","last_fullsized":"2023-10-06T03:48:22.990320+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:08.515631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.967698+0000","last_change":"2023-10-06T03:48:22.964069+0000","last_active":"2023-10-06T03:48:22.967698+0000","last_peered":"2023-10-06T03:48:22.967698+0000","last_clean":"2023-10-06T03:48:22.967698+0000","last_became_active":"2023-10-06T03:48:22.963794+0000","last_became_peered":"2023-10-06T03:48:22.963794+0000","last_unstale":"2023-10-06T03:48:22.967698+0000","last_undegraded":"2023-10-06T03:48:22.967698+0000","last_fullsized":"2023-10-06T03:48:22.967698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:03.317508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.959392+0000","last_change":"2023-10-06T03:48:21.949399+0000","last_active":"2023-10-06T03:48:22.959392+0000","last_peered":"2023-10-06T03:48:22.959392+0000","last_clean":"2023-10-06T03:48:22.959392+0000","last_became_active":"2023-10-06T03:48:21.949065+0000","last_became_peered":"2023-10-06T03:48:21.949065+0000","last_unstale":"2023-10-06T03:48:22.959392+0000","last_undegraded":"2023-10-06T03:48:22.959392+0000","last_fullsized":"2023-10-06T03:48:22.959392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:24.883643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.976002+0000","last_change":"2023-10-06T03:48:22.969329+0000","last_active":"2023-10-06T03:48:22.976002+0000","last_peered":"2023-10-06T03:48:22.976002+0000","last_clean":"2023-10-06T03:48:22.976002+0000","last_became_active":"2023-10-06T03:48:22.969048+0000","last_became_peered":"2023-10-06T03:48:22.969048+0000","last_unstale":"2023-10-06T03:48:22.976002+0000","last_undegraded":"2023-10-06T03:48:22.976002+0000","last_fullsized":"2023-10-06T03:48:22.976002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:15.207162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958635+0000","last_change":"2023-10-06T03:48:21.950832+0000","last_active":"2023-10-06T03:48:22.958635+0000","last_peered":"2023-10-06T03:48:22.958635+0000","last_clean":"2023-10-06T03:48:22.958635+0000","last_became_active":"2023-10-06T03:48:21.950526+0000","last_became_peered":"2023-10-06T03:48:21.950526+0000","last_unstale":"2023-10-06T03:48:22.958635+0000","last_undegraded":"2023-10-06T03:48:22.958635+0000","last_fullsized":"2023-10-06T03:48:22.958635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:41.936178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970721+0000","last_change":"2023-10-06T03:48:22.965699+0000","last_active":"2023-10-06T03:48:22.970721+0000","last_peered":"2023-10-06T03:48:22.970721+0000","last_clean":"2023-10-06T03:48:22.970721+0000","last_became_active":"2023-10-06T03:48:22.965291+0000","last_became_peered":"2023-10-06T03:48:22.965291+0000","last_unstale":"2023-10-06T03:48:22.970721+0000","last_undegraded":"2023-10-06T03:48:22.970721+0000","last_fullsized":"2023-10-06T03:48:22.970721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:58.933292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.956473+0000","last_change":"2023-10-06T03:48:21.958507+0000","last_active":"2023-10-06T03:48:22.956473+0000","last_peered":"2023-10-06T03:48:22.956473+0000","last_clean":"2023-10-06T03:48:22.956473+0000","last_became_active":"2023-10-06T03:48:21.958265+0000","last_became_peered":"2023-10-06T03:48:21.958265+0000","last_unstale":"2023-10-06T03:48:22.956473+0000","last_undegraded":"2023-10-06T03:48:22.956473+0000","last_fullsized":"2023-10-06T03:48:22.956473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:57.923641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972857+0000","last_change":"2023-10-06T03:48:22.963396+0000","last_active":"2023-10-06T03:48:22.972857+0000","last_peered":"2023-10-06T03:48:22.972857+0000","last_clean":"2023-10-06T03:48:22.972857+0000","last_became_active":"2023-10-06T03:48:22.963027+0000","last_became_peered":"2023-10-06T03:48:22.963027+0000","last_unstale":"2023-10-06T03:48:22.972857+0000","last_undegraded":"2023-10-06T03:48:22.972857+0000","last_fullsized":"2023-10-06T03:48:22.972857+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:12.482387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.965233+0000","last_change":"2023-10-06T03:48:21.960124+0000","last_active":"2023-10-06T03:48:22.965233+0000","last_peered":"2023-10-06T03:48:22.965233+0000","last_clean":"2023-10-06T03:48:22.965233+0000","last_became_active":"2023-10-06T03:48:21.959839+0000","last_became_peered":"2023-10-06T03:48:21.959839+0000","last_unstale":"2023-10-06T03:48:22.965233+0000","last_undegraded":"2023-10-06T03:48:22.965233+0000","last_fullsized":"2023-10-06T03:48:22.965233+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:19.590235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.977066+0000","last_change":"2023-10-06T03:48:22.975809+0000","last_active":"2023-10-06T03:48:22.977066+0000","last_peered":"2023-10-06T03:48:22.977066+0000","last_clean":"2023-10-06T03:48:22.977066+0000","last_became_active":"2023-10-06T03:48:22.975112+0000","last_became_peered":"2023-10-06T03:48:22.975112+0000","last_unstale":"2023-10-06T03:48:22.977066+0000","last_undegraded":"2023-10-06T03:48:22.977066+0000","last_fullsized":"2023-10-06T03:48:22.977066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:50.993039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958241+0000","last_change":"2023-10-06T03:48:21.951488+0000","last_active":"2023-10-06T03:48:22.958241+0000","last_peered":"2023-10-06T03:48:22.958241+0000","last_clean":"2023-10-06T03:48:22.958241+0000","last_became_active":"2023-10-06T03:48:21.951196+0000","last_became_peered":"2023-10-06T03:48:21.951196+0000","last_unstale":"2023-10-06T03:48:22.958241+0000","last_undegraded":"2023-10-06T03:48:22.958241+0000","last_fullsized":"2023-10-06T03:48:22.958241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:34.715803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981040+0000","last_change":"2023-10-06T03:48:22.964301+0000","last_active":"2023-10-06T03:48:22.981040+0000","last_peered":"2023-10-06T03:48:22.981040+0000","last_clean":"2023-10-06T03:48:22.981040+0000","last_became_active":"2023-10-06T03:48:22.963870+0000","last_became_peered":"2023-10-06T03:48:22.963870+0000","last_unstale":"2023-10-06T03:48:22.981040+0000","last_undegraded":"2023-10-06T03:48:22.981040+0000","last_fullsized":"2023-10-06T03:48:22.981040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:24.959369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.961484+0000","last_change":"2023-10-06T03:48:21.950936+0000","last_active":"2023-10-06T03:48:22.961484+0000","last_peered":"2023-10-06T03:48:22.961484+0000","last_clean":"2023-10-06T03:48:22.961484+0000","last_became_active":"2023-10-06T03:48:21.950639+0000","last_became_peered":"2023-10-06T03:48:21.950639+0000","last_unstale":"2023-10-06T03:48:22.961484+0000","last_undegraded":"2023-10-06T03:48:22.961484+0000","last_fullsized":"2023-10-06T03:48:22.961484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:11.961094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981729+0000","last_change":"2023-10-06T03:48:22.980115+0000","last_active":"2023-10-06T03:48:22.981729+0000","last_peered":"2023-10-06T03:48:22.981729+0000","last_clean":"2023-10-06T03:48:22.981729+0000","last_became_active":"2023-10-06T03:48:22.979810+0000","last_became_peered":"2023-10-06T03:48:22.979810+0000","last_unstale":"2023-10-06T03:48:22.981729+0000","last_undegraded":"2023-10-06T03:48:22.981729+0000","last_fullsized":"2023-10-06T03:48:22.981729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:20.245038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.955562+0000","last_change":"2023-10-06T03:48:21.961740+0000","last_active":"2023-10-06T03:48:22.955562+0000","last_peered":"2023-10-06T03:48:22.955562+0000","last_clean":"2023-10-06T03:48:22.955562+0000","last_became_active":"2023-10-06T03:48:21.961381+0000","last_became_peered":"2023-10-06T03:48:21.961381+0000","last_unstale":"2023-10-06T03:48:22.955562+0000","last_undegraded":"2023-10-06T03:48:22.955562+0000","last_fullsized":"2023-10-06T03:48:22.955562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:03.746648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971551+0000","last_change":"2023-10-06T03:48:22.962578+0000","last_active":"2023-10-06T03:48:22.971551+0000","last_peered":"2023-10-06T03:48:22.971551+0000","last_clean":"2023-10-06T03:48:22.971551+0000","last_became_active":"2023-10-06T03:48:22.962320+0000","last_became_peered":"2023-10-06T03:48:22.962320+0000","last_unstale":"2023-10-06T03:48:22.971551+0000","last_undegraded":"2023-10-06T03:48:22.971551+0000","last_fullsized":"2023-10-06T03:48:22.971551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:06.661073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.992556+0000","last_change":"2023-10-06T03:48:21.961642+0000","last_active":"2023-10-06T03:48:22.992556+0000","last_peered":"2023-10-06T03:48:22.992556+0000","last_clean":"2023-10-06T03:48:22.992556+0000","last_became_active":"2023-10-06T03:48:21.961209+0000","last_became_peered":"2023-10-06T03:48:21.961209+0000","last_unstale":"2023-10-06T03:48:22.992556+0000","last_undegraded":"2023-10-06T03:48:22.992556+0000","last_fullsized":"2023-10-06T03:48:22.992556+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:22.308236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.954065+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:48:22.954065+0000","last_peered":"2023-10-06T03:48:22.954065+0000","last_clean":"2023-10-06T03:48:22.954065+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:48:22.954065+0000","last_undegraded":"2023-10-06T03:48:22.954065+0000","last_fullsized":"2023-10-06T03:48:22.954065+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971664+0000","last_change":"2023-10-06T03:48:22.970569+0000","last_active":"2023-10-06T03:48:22.971664+0000","last_peered":"2023-10-06T03:48:22.971664+0000","last_clean":"2023-10-06T03:48:22.971664+0000","last_became_active":"2023-10-06T03:48:22.968580+0000","last_became_peered":"2023-10-06T03:48:22.968580+0000","last_unstale":"2023-10-06T03:48:22.971664+0000","last_undegraded":"2023-10-06T03:48:22.971664+0000","last_fullsized":"2023-10-06T03:48:22.971664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:04.756299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.960687+0000","last_change":"2023-10-06T03:48:21.958186+0000","last_active":"2023-10-06T03:48:22.960687+0000","last_peered":"2023-10-06T03:48:22.960687+0000","last_clean":"2023-10-06T03:48:22.960687+0000","last_became_active":"2023-10-06T03:48:21.957831+0000","last_became_peered":"2023-10-06T03:48:21.957831+0000","last_unstale":"2023-10-06T03:48:22.960687+0000","last_undegraded":"2023-10-06T03:48:22.960687+0000","last_fullsized":"2023-10-06T03:48:22.960687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:11.915578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.960769+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:48:22.960769+0000","last_peered":"2023-10-06T03:48:22.960769+0000","last_clean":"2023-10-06T03:48:22.960769+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:48:22.960769+0000","last_undegraded":"2023-10-06T03:48:22.960769+0000","last_fullsized":"2023-10-06T03:48:22.960769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980136+0000","last_change":"2023-10-06T03:48:22.968062+0000","last_active":"2023-10-06T03:48:22.980136+0000","last_peered":"2023-10-06T03:48:22.980136+0000","last_clean":"2023-10-06T03:48:22.980136+0000","last_became_active":"2023-10-06T03:48:22.967779+0000","last_became_peered":"2023-10-06T03:48:22.967779+0000","last_unstale":"2023-10-06T03:48:22.980136+0000","last_undegraded":"2023-10-06T03:48:22.980136+0000","last_fullsized":"2023-10-06T03:48:22.980136+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:14.771414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.984625+0000","last_change":"2023-10-06T03:48:21.954894+0000","last_active":"2023-10-06T03:48:22.984625+0000","last_peered":"2023-10-06T03:48:22.984625+0000","last_clean":"2023-10-06T03:48:22.984625+0000","last_became_active":"2023-10-06T03:48:21.954613+0000","last_became_peered":"2023-10-06T03:48:21.954613+0000","last_unstale":"2023-10-06T03:48:22.984625+0000","last_undegraded":"2023-10-06T03:48:22.984625+0000","last_fullsized":"2023-10-06T03:48:22.984625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:43.263401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":66,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.955914+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:48:22.955914+0000","last_peered":"2023-10-06T03:48:22.955914+0000","last_clean":"2023-10-06T03:48:22.955914+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:48:22.955914+0000","last_undegraded":"2023-10-06T03:48:22.955914+0000","last_fullsized":"2023-10-06T03:48:22.955914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"70'161","reported_seq":372,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.963654+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:48:22.963654+0000","last_peered":"2023-10-06T03:48:22.963654+0000","last_clean":"2023-10-06T03:48:22.963654+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:48:22.963654+0000","last_undegraded":"2023-10-06T03:48:22.963654+0000","last_fullsized":"2023-10-06T03:48:22.963654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971487+0000","last_change":"2023-10-06T03:48:22.967727+0000","last_active":"2023-10-06T03:48:22.971487+0000","last_peered":"2023-10-06T03:48:22.971487+0000","last_clean":"2023-10-06T03:48:22.971487+0000","last_became_active":"2023-10-06T03:48:22.967352+0000","last_became_peered":"2023-10-06T03:48:22.967352+0000","last_unstale":"2023-10-06T03:48:22.971487+0000","last_undegraded":"2023-10-06T03:48:22.971487+0000","last_fullsized":"2023-10-06T03:48:22.971487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:20.599886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958689+0000","last_change":"2023-10-06T03:48:21.954564+0000","last_active":"2023-10-06T03:48:22.958689+0000","last_peered":"2023-10-06T03:48:22.958689+0000","last_clean":"2023-10-06T03:48:22.958689+0000","last_became_active":"2023-10-06T03:48:21.954153+0000","last_became_peered":"2023-10-06T03:48:21.954153+0000","last_unstale":"2023-10-06T03:48:22.958689+0000","last_undegraded":"2023-10-06T03:48:22.958689+0000","last_fullsized":"2023-10-06T03:48:22.958689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:50.343611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":67,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.967814+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:48:22.967814+0000","last_peered":"2023-10-06T03:48:22.967814+0000","last_clean":"2023-10-06T03:48:22.967814+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:48:22.967814+0000","last_undegraded":"2023-10-06T03:48:22.967814+0000","last_fullsized":"2023-10-06T03:48:22.967814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970547+0000","last_change":"2023-10-06T03:48:22.967121+0000","last_active":"2023-10-06T03:48:22.970547+0000","last_peered":"2023-10-06T03:48:22.970547+0000","last_clean":"2023-10-06T03:48:22.970547+0000","last_became_active":"2023-10-06T03:48:22.966857+0000","last_became_peered":"2023-10-06T03:48:22.966857+0000","last_unstale":"2023-10-06T03:48:22.970547+0000","last_undegraded":"2023-10-06T03:48:22.970547+0000","last_fullsized":"2023-10-06T03:48:22.970547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:07.086261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.961262+0000","last_change":"2023-10-06T03:48:21.946145+0000","last_active":"2023-10-06T03:48:22.961262+0000","last_peered":"2023-10-06T03:48:22.961262+0000","last_clean":"2023-10-06T03:48:22.961262+0000","last_became_active":"2023-10-06T03:48:21.945883+0000","last_became_peered":"2023-10-06T03:48:21.945883+0000","last_unstale":"2023-10-06T03:48:22.961262+0000","last_undegraded":"2023-10-06T03:48:22.961262+0000","last_fullsized":"2023-10-06T03:48:22.961262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:29.398401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.963748+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:48:22.963748+0000","last_peered":"2023-10-06T03:48:22.963748+0000","last_clean":"2023-10-06T03:48:22.963748+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:48:22.963748+0000","last_undegraded":"2023-10-06T03:48:22.963748+0000","last_fullsized":"2023-10-06T03:48:22.963748+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.974827+0000","last_change":"2023-10-06T03:48:22.965384+0000","last_active":"2023-10-06T03:48:22.974827+0000","last_peered":"2023-10-06T03:48:22.974827+0000","last_clean":"2023-10-06T03:48:22.974827+0000","last_became_active":"2023-10-06T03:48:22.964719+0000","last_became_peered":"2023-10-06T03:48:22.964719+0000","last_unstale":"2023-10-06T03:48:22.974827+0000","last_undegraded":"2023-10-06T03:48:22.974827+0000","last_fullsized":"2023-10-06T03:48:22.974827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:09.525282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958144+0000","last_change":"2023-10-06T03:48:21.958097+0000","last_active":"2023-10-06T03:48:22.958144+0000","last_peered":"2023-10-06T03:48:22.958144+0000","last_clean":"2023-10-06T03:48:22.958144+0000","last_became_active":"2023-10-06T03:48:21.957822+0000","last_became_peered":"2023-10-06T03:48:21.957822+0000","last_unstale":"2023-10-06T03:48:22.958144+0000","last_undegraded":"2023-10-06T03:48:22.958144+0000","last_fullsized":"2023-10-06T03:48:22.958144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:59.062727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958612+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:48:22.958612+0000","last_peered":"2023-10-06T03:48:22.958612+0000","last_clean":"2023-10-06T03:48:22.958612+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:48:22.958612+0000","last_undegraded":"2023-10-06T03:48:22.958612+0000","last_fullsized":"2023-10-06T03:48:22.958612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972727+0000","last_change":"2023-10-06T03:48:22.961087+0000","last_active":"2023-10-06T03:48:22.972727+0000","last_peered":"2023-10-06T03:48:22.972727+0000","last_clean":"2023-10-06T03:48:22.972727+0000","last_became_active":"2023-10-06T03:48:22.960836+0000","last_became_peered":"2023-10-06T03:48:22.960836+0000","last_unstale":"2023-10-06T03:48:22.972727+0000","last_undegraded":"2023-10-06T03:48:22.972727+0000","last_fullsized":"2023-10-06T03:48:22.972727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:52.603676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.990082+0000","last_change":"2023-10-06T03:48:21.957833+0000","last_active":"2023-10-06T03:48:22.990082+0000","last_peered":"2023-10-06T03:48:22.990082+0000","last_clean":"2023-10-06T03:48:22.990082+0000","last_became_active":"2023-10-06T03:48:21.957260+0000","last_became_peered":"2023-10-06T03:48:21.957260+0000","last_unstale":"2023-10-06T03:48:22.990082+0000","last_undegraded":"2023-10-06T03:48:22.990082+0000","last_fullsized":"2023-10-06T03:48:22.990082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:31.139755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.952549+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:48:22.952549+0000","last_peered":"2023-10-06T03:48:22.952549+0000","last_clean":"2023-10-06T03:48:22.952549+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:48:22.952549+0000","last_undegraded":"2023-10-06T03:48:22.952549+0000","last_fullsized":"2023-10-06T03:48:22.952549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981603+0000","last_change":"2023-10-06T03:48:22.980129+0000","last_active":"2023-10-06T03:48:22.981603+0000","last_peered":"2023-10-06T03:48:22.981603+0000","last_clean":"2023-10-06T03:48:22.981603+0000","last_became_active":"2023-10-06T03:48:22.979872+0000","last_became_peered":"2023-10-06T03:48:22.979872+0000","last_unstale":"2023-10-06T03:48:22.981603+0000","last_undegraded":"2023-10-06T03:48:22.981603+0000","last_fullsized":"2023-10-06T03:48:22.981603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:13.209361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.989740+0000","last_change":"2023-10-06T03:48:21.955182+0000","last_active":"2023-10-06T03:48:22.989740+0000","last_peered":"2023-10-06T03:48:22.989740+0000","last_clean":"2023-10-06T03:48:22.989740+0000","last_became_active":"2023-10-06T03:48:21.954909+0000","last_became_peered":"2023-10-06T03:48:21.954909+0000","last_unstale":"2023-10-06T03:48:22.989740+0000","last_undegraded":"2023-10-06T03:48:22.989740+0000","last_fullsized":"2023-10-06T03:48:22.989740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:25.171449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.952991+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:48:22.952991+0000","last_peered":"2023-10-06T03:48:22.952991+0000","last_clean":"2023-10-06T03:48:22.952991+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:48:22.952991+0000","last_undegraded":"2023-10-06T03:48:22.952991+0000","last_fullsized":"2023-10-06T03:48:22.952991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575031,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311112,"kb_used_data":1536,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060728,"statfs":{"total":96636764160,"available":96318185472,"internally_reserved":0,"allocated":1572864,"data_stored":1076126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71199999999999997,"15min":0.69299999999999995},"min":{"1min":0.53000000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.998,"5min":1.0029999999999999,"15min":1.6100000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.51500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.5649999999999999},"last":0.68899999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.755,"15min":0.749},"min":{"1min":0.63800000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.038,"5min":1.038,"15min":1.5469999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.59199999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.56899999999999995,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":1.5900000000000001},"last":0.437},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.56999999999999995,"5min":0.495,"15min":0.495},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66500000000000004}]},{"osd":3,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73199999999999998,"15min":0.72499999999999998},"min":{"1min":0.61299999999999999,"5min":0.45900000000000002,"15min":0.39500000000000002},"max":{"1min":1.0089999999999999,"5min":1.016,"15min":1.6279999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76000000000000001,"15min":0.74399999999999999},"min":{"1min":0.624,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96099999999999997,"5min":0.98099999999999998,"15min":1.0409999999999999},"last":0.74299999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79600000000000004,"5min":0.96499999999999997,"15min":1.0049999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.628},"min":{"1min":0.46400000000000002,"5min":0.40999999999999998,"15min":0.39400000000000002},"max":{"1min":0.874,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.621}]},{"osd":5,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63900000000000001,"15min":0.64800000000000002},"min":{"1min":0.49299999999999999,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":0.95599999999999996,"5min":1.018,"15min":1.069},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.46800000000000003,"15min":0.436},"max":{"1min":0.91300000000000003,"5min":1.105,"15min":1.105},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72399999999999998,"15min":0.72899999999999998},"min":{"1min":0.58199999999999996,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":1.181,"15min":1.181},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.55200000000000005,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.91500000000000004,"5min":1.02,"15min":1.0209999999999999},"last":0.76700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575032,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306428,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065412,"statfs":{"total":96636764160,"available":96322981888,"internally_reserved":0,"allocated":970752,"data_stored":486022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.42199999999999999,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.88,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.39500000000000002},"max":{"1min":0.94799999999999995,"5min":0.98599999999999999,"15min":1.0449999999999999},"last":0.89700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.47499999999999998,"5min":0.36499999999999999,"15min":0.33800000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.3180000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.35899999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":1.073}]},{"osd":2,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.751,"15min":0.751},"min":{"1min":0.48199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.98699999999999999,"5min":0.995,"15min":3.3319999999999999},"last":0.627},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":3.5139999999999998},"last":0.95999999999999996}]},{"osd":3,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.58499999999999996,"5min":0.48299999999999998,"15min":0.44600000000000001},"max":{"1min":0.97399999999999998,"5min":1.01,"15min":1.024},"last":0.88},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76100000000000001,"15min":0.77400000000000002},"min":{"1min":0.49299999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.98799999999999999,"5min":0.99099999999999999,"15min":3.3300000000000001},"last":0.85599999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.45200000000000001,"5min":0.376,"15min":0.34799999999999998},"max":{"1min":0.91900000000000004,"5min":0.92600000000000005,"15min":1.0680000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36499999999999999},"max":{"1min":0.90700000000000003,"5min":1.0169999999999999,"15min":1.2929999999999999},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67000000000000004,"15min":0.66700000000000004},"min":{"1min":0.438,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.3340000000000001},"last":1.006},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.42899999999999999,"15min":0.41199999999999998},"max":{"1min":1.0049999999999999,"5min":1.0269999999999999,"15min":1.2370000000000001},"last":0.91400000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68600000000000005,"15min":0.69899999999999995},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.619},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.47099999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.417},"last":1.0389999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575031,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307032,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064808,"statfs":{"total":96636764160,"available":96322363392,"internally_reserved":0,"allocated":1589248,"data_stored":1078628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.47999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80200000000000005,"5min":1.1319999999999999,"15min":1.2889999999999999},"last":0.432}]},{"osd":2,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.5,"15min":0.495},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.26100000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.44,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.94099999999999995},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.60199999999999998},"min":{"1min":0.50800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86699999999999999,"15min":0.93999999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.434,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77400000000000002,"5min":0.90400000000000003,"15min":1.254},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.52500000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0009999999999999,"5min":1.034,"15min":1.3600000000000001},"last":0.626},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.80900000000000005,"15min":0.79800000000000004},"min":{"1min":0.68400000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.476},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.80200000000000005,"15min":0.79300000000000004},"min":{"1min":0.621,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.601},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.59299999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.5189999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.80500000000000005,"15min":0.81000000000000005},"min":{"1min":0.64200000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82499999999999996,"15min":0.81699999999999995},"min":{"1min":0.66700000000000004,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":1.143,"5min":1.1639999999999999,"15min":1.399},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.84799999999999998,"15min":0.83399999999999996},"min":{"1min":0.66900000000000004,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1000000000000001,"5min":1.272,"15min":1.272},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.83399999999999996,"15min":0.82899999999999996},"min":{"1min":0.64400000000000002,"5min":0.53700000000000003,"15min":0.51200000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.556},"last":0.65600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575033,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306420,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065420,"statfs":{"total":96636764160,"available":96322990080,"internally_reserved":0,"allocated":962560,"data_stored":485010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48199999999999998,"15min":0.495},"min":{"1min":0.373,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":0.67800000000000005,"5min":0.81000000000000005,"15min":2.3220000000000001},"last":0.373},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.52700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33600000000000002,"15min":0.29599999999999999},"max":{"1min":0.70299999999999996,"5min":0.78500000000000003,"15min":2.1440000000000001},"last":0.501}]},{"osd":2,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.56499999999999995},"min":{"1min":0.40500000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.76400000000000001,"5min":0.77700000000000002,"15min":2.4329999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54800000000000004,"15min":0.56299999999999994},"min":{"1min":0.40799999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.755,"5min":0.84699999999999998,"15min":2.1819999999999999},"last":0.41499999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51100000000000001,"15min":0.53000000000000003},"min":{"1min":0.41399999999999998,"5min":0.30499999999999999,"15min":0.28399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":2.2709999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61299999999999999,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81000000000000005,"5min":1.0580000000000001,"15min":2.4700000000000002},"last":0.435}]},{"osd":4,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.83099999999999996,"15min":0.83499999999999996},"min":{"1min":0.69199999999999995,"5min":0.54200000000000004,"15min":0.49399999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.393},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.80900000000000005,"15min":0.83199999999999996},"min":{"1min":0.63400000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.306,"5min":1.306,"15min":2.2320000000000002},"last":0.79200000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.80600000000000005,"15min":0.82999999999999996},"min":{"1min":0.57799999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":2.3660000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.58999999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0780000000000001,"5min":1.0800000000000001,"15min":1.6559999999999999},"last":0.77300000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77900000000000003,"15min":0.80500000000000005},"min":{"1min":0.56999999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.726},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.82399999999999995,"15min":0.85699999999999998},"min":{"1min":0.628,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1100000000000001,"5min":1.119,"15min":2.4009999999999998},"last":0.80100000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.64300000000000002,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2689999999999999},"last":0.873},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.79800000000000004,"15min":0.83699999999999997},"min":{"1min":0.625,"5min":0.495,"15min":0.495},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.7649999999999999},"last":0.76000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575032,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306420,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065420,"statfs":{"total":96636764160,"available":96322990080,"internally_reserved":0,"allocated":962560,"data_stored":485010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48899999999999999,"15min":0.46700000000000003},"min":{"1min":0.35199999999999998,"5min":0.29699999999999999,"15min":0.28000000000000003},"max":{"1min":0.74199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53400000000000003,"15min":0.52300000000000002},"min":{"1min":0.438,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":0.86199999999999999,"15min":0.89300000000000002},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.34799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72899999999999998,"5min":0.79900000000000004,"15min":0.90000000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.372,"15min":0.34200000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.556},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.504,"5min":0.41399999999999998,"15min":0.38800000000000001},"max":{"1min":0.72899999999999998,"5min":1.78,"15min":1.78},"last":0.439},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81799999999999995,"5min":0.91400000000000003,"15min":1.1910000000000001},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.81899999999999995,"15min":0.81799999999999995},"min":{"1min":0.67600000000000005,"5min":0.51400000000000001,"15min":0.439},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.5620000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.72199999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.008,"5min":1.1419999999999999,"15min":1.2270000000000001},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.82699999999999996,"15min":0.82999999999999996},"min":{"1min":0.68100000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.1140000000000001,"5min":1.177,"15min":1.3},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84399999999999997,"15min":0.84999999999999998},"min":{"1min":0.65500000000000003,"5min":0.55500000000000005,"15min":0.51700000000000002},"max":{"1min":1.0549999999999999,"5min":1.855,"15min":1.855},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82999999999999996,"15min":0.83699999999999997},"min":{"1min":0.68400000000000005,"5min":0.54200000000000004,"15min":0.53600000000000003},"max":{"1min":0.97899999999999998,"5min":1.907,"15min":1.907},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.81899999999999995,"15min":0.81499999999999995},"min":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.51100000000000001},"max":{"1min":1.1080000000000001,"5min":1.1120000000000001,"15min":1.4750000000000001},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.84099999999999997,"15min":0.83499999999999996},"min":{"1min":0.73999999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.125,"5min":1.4379999999999999,"15min":1.4990000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.86399999999999999},"min":{"1min":0.68400000000000005,"5min":0.53100000000000003,"15min":0.497},"max":{"1min":0.99399999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.69599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575032,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306424,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065416,"statfs":{"total":96636764160,"available":96322985984,"internally_reserved":0,"allocated":966656,"data_stored":485988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.33000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56899999999999995,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":0.63800000000000001,"5min":1.04,"15min":1.1699999999999999},"last":0.376}]},{"osd":1,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47799999999999998,"15min":0.47299999999999998},"min":{"1min":0.374,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.78800000000000003,"15min":0.79500000000000004},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.33800000000000002},"max":{"1min":0.70599999999999996,"5min":0.76700000000000002,"15min":0.83899999999999997},"last":0.35299999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.34599999999999997},"max":{"1min":0.79700000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76800000000000002,"5min":0.81999999999999995,"15min":0.97899999999999998},"last":0.435}]},{"osd":4,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.78000000000000003,"15min":0.78900000000000003},"min":{"1min":0.51600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.0049999999999999,"5min":1.2030000000000001,"15min":1.4099999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.64100000000000001,"5min":0.57399999999999995,"15min":0.51300000000000001},"max":{"1min":1.0469999999999999,"5min":1.1850000000000001,"15min":1.377},"last":0.79200000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77400000000000002,"15min":0.78800000000000003},"min":{"1min":0.56699999999999995,"5min":0.51900000000000002,"15min":0.47099999999999997},"max":{"1min":1,"5min":1.256,"15min":1.256},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.61499999999999999,"5min":0.55600000000000005,"15min":0.48099999999999998},"max":{"1min":0.99199999999999999,"5min":1.1439999999999999,"15min":1.306},"last":0.629}]},{"osd":6,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81000000000000005,"15min":0.81399999999999995},"min":{"1min":0.61699999999999999,"5min":0.52000000000000002,"15min":0.48599999999999999},"max":{"1min":1.02,"5min":1.292,"15min":1.323},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.82799999999999996,"15min":0.84999999999999998},"min":{"1min":0.60599999999999998,"5min":0.58599999999999997,"15min":0.49299999999999999},"max":{"1min":1.075,"5min":1.1040000000000001,"15min":1.6020000000000001},"last":0.73899999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.65500000000000003,"5min":0.57799999999999996,"15min":0.505},"max":{"1min":0.98999999999999999,"5min":1.2370000000000001,"15min":1.365},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80000000000000004,"15min":0.80800000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.46300000000000002},"max":{"1min":1.038,"5min":1.1180000000000001,"15min":1.2330000000000001},"last":0.67700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575032,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302340,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069500,"statfs":{"total":96636764160,"available":96327168000,"internally_reserved":0,"allocated":978944,"data_stored":487524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.70599999999999996,"15min":0.72099999999999997},"min":{"1min":0.56899999999999995,"5min":0.316,"15min":0.316},"max":{"1min":0.96999999999999997,"5min":1.0109999999999999,"15min":1.3120000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.54800000000000004,"5min":0.316,"15min":0.316},"max":{"1min":0.91500000000000004,"5min":0.95299999999999996,"15min":1.01},"last":0.67300000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.47299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85899999999999999,"5min":1.081,"15min":1.081},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71799999999999997,"15min":0.73699999999999999},"min":{"1min":0.64600000000000002,"5min":0.44,"15min":0.41299999999999998},"max":{"1min":0.89600000000000002,"5min":0.96999999999999997,"15min":1.3380000000000001},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.61399999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.91500000000000004,"5min":1.028,"15min":1.1759999999999999},"last":0.755},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.747,"15min":0.746},"min":{"1min":0.629,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0109999999999999,"5min":1.022,"15min":1.3089999999999999},"last":0.76600000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.63100000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89700000000000002,"5min":1.0049999999999999,"15min":1.0409999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75800000000000001,"15min":0.76900000000000002},"min":{"1min":0.53200000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.80700000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.621,"15min":0.63},"min":{"1min":0.496,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":1.01,"15min":1.085},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.622,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.35099999999999998},"max":{"1min":0.72799999999999998,"5min":0.95399999999999996,"15min":1.1439999999999999},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.38100000000000001},"max":{"1min":0.90200000000000002,"5min":1.052,"15min":1.361},"last":0.502}]},{"osd":7,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.39900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":1.1950000000000001},"last":0.628},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.52000000000000002,"5min":0.504,"15min":0.41299999999999998},"max":{"1min":0.90100000000000002,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.65000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575032,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073608,"statfs":{"total":96636764160,"available":96331374592,"internally_reserved":0,"allocated":966656,"data_stored":485032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.42299999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94999999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70899999999999996,"15min":0.70699999999999996},"min":{"1min":0.60599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93500000000000005,"5min":0.96899999999999997,"15min":1},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.70599999999999996,"15min":0.70799999999999996},"min":{"1min":0.46600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.58599999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.60299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76400000000000001,"15min":0.77600000000000002},"min":{"1min":0.67900000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98699999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.67000000000000004}]},{"osd":3,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72399999999999998,"15min":0.73099999999999998},"min":{"1min":0.56599999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.996,"5min":0.996,"15min":1.6419999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.754,"15min":0.749},"min":{"1min":0.623,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0189999999999999,"5min":1.117,"15min":1.117},"last":0.625}]},{"osd":4,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.62,"15min":0.621},"min":{"1min":0.42899999999999999,"5min":0.40000000000000002,"15min":0.38300000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.024},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61799999999999999,"15min":0.63800000000000001},"min":{"1min":0.47099999999999997,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":0.94199999999999995,"5min":1.044,"15min":1.175},"last":0.745}]},{"osd":6,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.34999999999999998},"max":{"1min":1.0449999999999999,"5min":1.591,"15min":1.591},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69599999999999995,"15min":0.69899999999999995},"min":{"1min":0.47799999999999998,"5min":0.44600000000000001,"15min":0.40600000000000003},"max":{"1min":1.0389999999999999,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67000000000000004,"15min":0.68100000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":0.97199999999999998,"5min":1.016,"15min":1.2589999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.72399999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.435},"max":{"1min":1.0880000000000001,"5min":1.278,"15min":1.278},"last":0.78100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:48:28.300 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T03:48:28.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.518+0000 7f8f41b9d700 1 Processor -- start 2023-10-06T03:48:28.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.518+0000 7f8f41b9d700 1 -- start start 2023-10-06T03:48:28.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.519+0000 7f8f41b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8f3c069f80 0x7f8f3c06deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:28.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.519+0000 7f8f41b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3c0c6810 con 0x7f8f3c1559a0 2023-10-06T03:48:28.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.519+0000 7f8f41b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3c0c7390 con 0x7f8f3c0ac210 2023-10-06T03:48:28.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.519+0000 7f8f41b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f3c0c6a60 con 0x7f8f3c069f80 2023-10-06T03:48:28.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.520+0000 7f8f40b9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8f3c1559a0 0x7f8f3c069870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55832/0 (socket says 172.21.15.55:55832) 2023-10-06T03:48:28.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.520+0000 7f8f40b9b700 1 -- 172.21.15.55:0/3014506067 learned_addr learned my addr 172.21.15.55:0/3014506067 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:28.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.520+0000 7f8f3bfff700 1 --2- 172.21.15.55:0/3014506067 >> v2:172.21.15.163:3300/0 conn(0x7f8f3c069f80 0x7f8f3c06deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:28.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.520+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039865891 0 0) 0x7f8f3c0c6810 con 0x7f8f3c1559a0 2023-10-06T03:48:28.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.520+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f2c004680 con 0x7f8f3c1559a0 2023-10-06T03:48:28.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.520+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3208379065 0 0) 0x7f8f3c0c7390 con 0x7f8f3c0ac210 2023-10-06T03:48:28.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.520+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f3c0c6810 con 0x7f8f3c0ac210 2023-10-06T03:48:28.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.520+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3047345553 0 0) 0x7f8f2c004680 con 0x7f8f3c1559a0 2023-10-06T03:48:28.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.520+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 >> v2:172.21.15.163:3300/0 conn(0x7f8f3c069f80 msgr2=0x7f8f3c06deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:28.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.520+0000 7f8f3b7fe700 1 --2- 172.21.15.55:0/3014506067 >> v2:172.21.15.163:3300/0 conn(0x7f8f3c069f80 0x7f8f3c06deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.520+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 >> v1:172.21.15.163:6789/0 conn(0x7f8f3c0ac210 legacy=0x7f8f3c06e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:28.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.520+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c0c6d40 con 0x7f8f3c1559a0 2023-10-06T03:48:28.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.520+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f8f30001470 con 0x7f8f3c1559a0 2023-10-06T03:48:28.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.520+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 >> v1:172.21.15.55:6789/0 conn(0x7f8f3c1559a0 legacy=0x7f8f3c069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:28.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f3b7fe700 1 --2- 172.21.15.55:0/3014506067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f2c004d00 0x7f8f2c0050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:28.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f3b7fe700 1 --2- 172.21.15.55:0/3014506067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f2c005920 0x7f8f2c005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:28.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f3b7fe700 1 --2- 172.21.15.55:0/3014506067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f2c0062d0 0x7f8f2c00cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:28.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c0c6a60 con 0x7f8f2c0062d0 2023-10-06T03:48:28.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c00d1c0 con 0x7f8f2c004d00 2023-10-06T03:48:28.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c00d380 con 0x7f8f2c005920 2023-10-06T03:48:28.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f4139c700 1 --2- 172.21.15.55:0/3014506067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f2c0062d0 0x7f8f2c00cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:28.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f40b9b700 1 --2- 172.21.15.55:0/3014506067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f2c004d00 0x7f8f2c0050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:28.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f3bfff700 1 --2- 172.21.15.55:0/3014506067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f2c005920 0x7f8f2c005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:28.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f4139c700 1 -- 172.21.15.55:0/3014506067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f2c005920 msgr2=0x7f8f2c005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:28.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f4139c700 1 --2- 172.21.15.55:0/3014506067 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f2c005920 0x7f8f2c005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f4139c700 1 -- 172.21.15.55:0/3014506067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f2c004d00 msgr2=0x7f8f2c0050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:28.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f4139c700 1 --2- 172.21.15.55:0/3014506067 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f2c004d00 0x7f8f2c0050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f4139c700 1 -- 172.21.15.55:0/3014506067 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c004680 con 0x7f8f2c0062d0 2023-10-06T03:48:28.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.521+0000 7f8f4139c700 1 -- 172.21.15.55:0/3014506067 reap_dead start 2023-10-06T03:48:28.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.522+0000 7f8f4139c700 1 --2- 172.21.15.55:0/3014506067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f2c0062d0 0x7f8f2c00cb70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8f34002120 tx=0x7f8f340017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2f274204ba7d7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:28.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.522+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400c980 con 0x7f8f2c0062d0 2023-10-06T03:48:28.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.522+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3400cae0 con 0x7f8f2c0062d0 2023-10-06T03:48:28.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.522+0000 7f8f3b7fe700 1 -- 172.21.15.55:0/3014506067 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400ce80 con 0x7f8f2c0062d0 2023-10-06T03:48:28.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.522+0000 7f8f41b9d700 1 -- 172.21.15.55:0/3014506067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f2c0062d0 msgr2=0x7f8f2c00cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:28.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.522+0000 7f8f41b9d700 1 --2- 172.21.15.55:0/3014506067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f2c0062d0 0x7f8f2c00cb70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8f34002120 tx=0x7f8f340017a0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.522+0000 7f8f41b9d700 1 -- 172.21.15.55:0/3014506067 shutdown_connections 2023-10-06T03:48:28.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.522+0000 7f8f41b9d700 1 --2- 172.21.15.55:0/3014506067 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f2c0062d0 0x7f8f2c00cb70 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.522+0000 7f8f41b9d700 1 -- 172.21.15.55:0/3014506067 >> 172.21.15.55:0/3014506067 conn(0x7f8f3c0ccca0 msgr2=0x7f8f3c0c8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:28.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.523+0000 7f8f41b9d700 1 -- 172.21.15.55:0/3014506067 shutdown_connections 2023-10-06T03:48:28.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.523+0000 7f8f41b9d700 1 -- 172.21.15.55:0/3014506067 wait complete. 2023-10-06T03:48:28.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.524+0000 7f8f41b9d700 1 Processor -- start 2023-10-06T03:48:28.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.524+0000 7f8f41b9d700 1 -- start start 2023-10-06T03:48:28.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.525+0000 7f8f41b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f2c0062d0 0x7f8f3c154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:28.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.525+0000 7f8f41b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f3c15b300 0x7f8f3c1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:28.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.525+0000 7f8f41b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c159720 0x7f8f3c159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:28.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.525+0000 7f8f41b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c15a0d0 con 0x7f8f3c159720 2023-10-06T03:48:28.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.525+0000 7f8f41b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c15be10 con 0x7f8f3c15b300 2023-10-06T03:48:28.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.525+0000 7f8f41b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3c154d40 con 0x7f8f2c0062d0 2023-10-06T03:48:28.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.525+0000 7f8f4139c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c159720 0x7f8f3c159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:28.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.525+0000 7f8f4139c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c159720 0x7f8f3c159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48834/0 (socket says 172.21.15.55:48834) 2023-10-06T03:48:28.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.525+0000 7f8f4139c700 1 -- 172.21.15.55:0/984329623 learned_addr learned my addr 172.21.15.55:0/984329623 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:28.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.526+0000 7f8f40b9b700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f2c0062d0 0x7f8f3c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:28.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.526+0000 7f8f3bfff700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f3c15b300 0x7f8f3c1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:28.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.526+0000 7f8f4139c700 1 -- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f2c0062d0 msgr2=0x7f8f3c154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:28.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.526+0000 7f8f4139c700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f2c0062d0 0x7f8f3c154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.526+0000 7f8f4139c700 1 -- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f3c15b300 msgr2=0x7f8f3c1582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:28.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.526+0000 7f8f4139c700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f3c15b300 0x7f8f3c1582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.526+0000 7f8f4139c700 1 -- 172.21.15.55:0/984329623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3c15b850 con 0x7f8f3c159720 2023-10-06T03:48:28.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.526+0000 7f8f40b9b700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f2c0062d0 0x7f8f3c154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:28.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.526+0000 7f8f4139c700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c159720 0x7f8f3c159b90 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8f34001be0 tx=0x7f8f34015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:28.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.526+0000 7f8f38ff9700 1 -- 172.21.15.55:0/984329623 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400c980 con 0x7f8f3c159720 2023-10-06T03:48:28.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.526+0000 7f8f38ff9700 1 -- 172.21.15.55:0/984329623 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3400c6d0 con 0x7f8f3c159720 2023-10-06T03:48:28.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.526+0000 7f8f41b9d700 1 -- 172.21.15.55:0/984329623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3c15bb30 con 0x7f8f3c159720 2023-10-06T03:48:28.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.526+0000 7f8f38ff9700 1 -- 172.21.15.55:0/984329623 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3401e400 con 0x7f8f3c159720 2023-10-06T03:48:28.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.527+0000 7f8f41b9d700 1 -- 172.21.15.55:0/984329623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3c1561f0 con 0x7f8f3c159720 2023-10-06T03:48:28.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.527+0000 7f8f41b9d700 1 -- 172.21.15.55:0/984329623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3c155d70 con 0x7f8f3c159720 2023-10-06T03:48:28.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.528+0000 7f8f38ff9700 1 -- 172.21.15.55:0/984329623 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f34019070 con 0x7f8f3c159720 2023-10-06T03:48:28.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.529+0000 7f8f38ff9700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8f20066f30 0x7f8f200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:28.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.529+0000 7f8f38ff9700 1 -- 172.21.15.55:0/984329623 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f8f340858d0 con 0x7f8f3c159720 2023-10-06T03:48:28.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.530+0000 7f8f40b9b700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8f20066f30 0x7f8f200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:28.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.530+0000 7f8f40b9b700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8f20066f30 0x7f8f200693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8f30000e10 tx=0x7f8f30009040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:28.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.533+0000 7f8f38ff9700 1 -- 172.21.15.55:0/984329623 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f3c155d70 con 0x7f8f3c159720 2023-10-06T03:48:28.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.817+0000 7f8f41b9d700 1 -- 172.21.15.55:0/984329623 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8f3c155d70 con 0x7f8f3c159720 2023-10-06T03:48:28.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.817+0000 7f8f38ff9700 1 -- 172.21.15.55:0/984329623 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1176 (secure 0 0 0) 0x7f8f340244b0 con 0x7f8f3c159720 2023-10-06T03:48:28.819 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:28.819 INFO:teuthology.orchestra.run.smithi055.stdout:{"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":933,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":80,"num_osds":8,"num_up_osds":8,"osd_up_since":1696563187,"num_in_osds":8,"osd_in_since":1696563178,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":51},{"state_name":"creating+peering","count":28},{"state_name":"active+clean","count":26}],"num_pgs":105,"num_pools":4,"num_objects":8,"data_bytes":592458,"bytes_used":2502742016,"bytes_avail":770591371264,"bytes_total":773094113280,"unknown_pgs_ratio":0.48571428656578064,"inactive_pgs_ratio":0.26666668057441711,"write_bytes_sec":805,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":35,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":6333}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-10-06T03:47:24.755649+0000","services":{}},"progress_events":{}} 2023-10-06T03:48:28.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.820+0000 7f8f41b9d700 1 -- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8f20066f30 msgr2=0x7f8f200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:28.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.820+0000 7f8f41b9d700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8f20066f30 0x7f8f200693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8f30000e10 tx=0x7f8f30009040 comp rx=0 tx=0).stop 2023-10-06T03:48:28.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.820+0000 7f8f41b9d700 1 -- 172.21.15.55:0/984329623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c159720 msgr2=0x7f8f3c159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:28.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.820+0000 7f8f41b9d700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c159720 0x7f8f3c159b90 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8f34001be0 tx=0x7f8f34015000 comp rx=0 tx=0).stop 2023-10-06T03:48:28.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.820+0000 7f8f41b9d700 1 -- 172.21.15.55:0/984329623 shutdown_connections 2023-10-06T03:48:28.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.820+0000 7f8f41b9d700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8f20066f30 0x7f8f200693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.820+0000 7f8f41b9d700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8f3c159720 0x7f8f3c159b90 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.820+0000 7f8f41b9d700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8f2c0062d0 0x7f8f3c154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.820+0000 7f8f41b9d700 1 --2- 172.21.15.55:0/984329623 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8f3c15b300 0x7f8f3c1582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:28.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.820+0000 7f8f41b9d700 1 -- 172.21.15.55:0/984329623 >> 172.21.15.55:0/984329623 conn(0x7f8f3c0ccca0 msgr2=0x7f8f3c0b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:28.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.820+0000 7f8f41b9d700 1 -- 172.21.15.55:0/984329623 shutdown_connections 2023-10-06T03:48:28.823 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:28.820+0000 7f8f41b9d700 1 -- 172.21.15.55:0/984329623 wait complete. 2023-10-06T03:48:28.839 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T03:48:28.840 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:48:29.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.027+0000 7fcd97fff700 1 Processor -- start 2023-10-06T03:48:29.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.027+0000 7fcd97fff700 1 -- start start 2023-10-06T03:48:29.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.029+0000 7fcd97fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd980ad870 0x7fcd9806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:29.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.029+0000 7fcd97fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcd9806bf20 0x7fcd9806a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:29.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.029+0000 7fcd97fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd980b40f0 con 0x7fcd98142ea0 2023-10-06T03:48:29.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.029+0000 7fcd97fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9806c3a0 con 0x7fcd980ad870 2023-10-06T03:48:29.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.029+0000 7fcd97fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd98069640 con 0x7fcd9806bf20 2023-10-06T03:48:29.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.029+0000 7fcd967fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd980ad870 0x7fcd9806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:29.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.029+0000 7fcd967fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd980ad870 0x7fcd9806b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48838/0 (socket says 172.21.15.55:48838) 2023-10-06T03:48:29.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.029+0000 7fcd967fc700 1 -- 172.21.15.55:0/534400815 learned_addr learned my addr 172.21.15.55:0/534400815 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.029+0000 7fcd977fe700 1 --2- 172.21.15.55:0/534400815 >> v2:172.21.15.163:3300/0 conn(0x7fcd9806bf20 0x7fcd9806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.029+0000 7fcd967fc700 1 -- 172.21.15.55:0/534400815 >> v2:172.21.15.163:3300/0 conn(0x7fcd9806bf20 msgr2=0x7fcd9806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.029+0000 7fcd967fc700 1 --2- 172.21.15.55:0/534400815 >> v2:172.21.15.163:3300/0 conn(0x7fcd9806bf20 0x7fcd9806a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.030+0000 7fcd967fc700 1 -- 172.21.15.55:0/534400815 >> v1:172.21.15.163:6789/0 conn(0x7fcd98142ea0 legacy=0x7fcd980b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:29.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.030+0000 7fcd967fc700 1 -- 172.21.15.55:0/534400815 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd98069920 con 0x7fcd980ad870 2023-10-06T03:48:29.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.030+0000 7fcd967fc700 1 --2- 172.21.15.55:0/534400815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd980ad870 0x7fcd9806b9e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fcd98071330 tx=0x7fcd8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2723a8bd8fa86ab9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:29.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.030+0000 7fcd95ffb700 1 -- 172.21.15.55:0/534400815 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00b860 con 0x7fcd980ad870 2023-10-06T03:48:29.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.030+0000 7fcd95ffb700 1 -- 172.21.15.55:0/534400815 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c00b9c0 con 0x7fcd980ad870 2023-10-06T03:48:29.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.030+0000 7fcd95ffb700 1 -- 172.21.15.55:0/534400815 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00bd60 con 0x7fcd980ad870 2023-10-06T03:48:29.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.031+0000 7fcd97fff700 1 -- 172.21.15.55:0/534400815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd980ad870 msgr2=0x7fcd9806b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:29.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.031+0000 7fcd97fff700 1 --2- 172.21.15.55:0/534400815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd980ad870 0x7fcd9806b9e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fcd98071330 tx=0x7fcd8c009250 comp rx=0 tx=0).stop 2023-10-06T03:48:29.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.031+0000 7fcd97fff700 1 -- 172.21.15.55:0/534400815 shutdown_connections 2023-10-06T03:48:29.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.031+0000 7fcd97fff700 1 --2- 172.21.15.55:0/534400815 >> v2:172.21.15.163:3300/0 conn(0x7fcd9806bf20 0x7fcd9806a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:29.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.031+0000 7fcd97fff700 1 --2- 172.21.15.55:0/534400815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd980ad870 0x7fcd9806b9e0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:29.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.031+0000 7fcd97fff700 1 -- 172.21.15.55:0/534400815 >> 172.21.15.55:0/534400815 conn(0x7fcd980ce430 msgr2=0x7fcd980ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:29.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.031+0000 7fcd97fff700 1 -- 172.21.15.55:0/534400815 shutdown_connections 2023-10-06T03:48:29.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.031+0000 7fcd97fff700 1 -- 172.21.15.55:0/534400815 wait complete. 2023-10-06T03:48:29.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.032+0000 7fcd97fff700 1 Processor -- start 2023-10-06T03:48:29.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.032+0000 7fcd97fff700 1 -- start start 2023-10-06T03:48:29.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.033+0000 7fcd97fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd9806bf20 0x7fcd98131d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:29.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.033+0000 7fcd97fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd98142ea0 0x7fcd9815daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:29.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.033+0000 7fcd97fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd9815dfe0 0x7fcd98140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:29.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.033+0000 7fcd97fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9813f650 con 0x7fcd9806bf20 2023-10-06T03:48:29.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.033+0000 7fcd97fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd980ae1d0 con 0x7fcd9815dfe0 2023-10-06T03:48:29.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.033+0000 7fcd97fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd980ae4b0 con 0x7fcd98142ea0 2023-10-06T03:48:29.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.034+0000 7fcd96ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd9806bf20 0x7fcd98131d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:29.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.034+0000 7fcd96ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd9806bf20 0x7fcd98131d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48844/0 (socket says 172.21.15.55:48844) 2023-10-06T03:48:29.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.034+0000 7fcd96ffd700 1 -- 172.21.15.55:0/427167673 learned_addr learned my addr 172.21.15.55:0/427167673 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:29.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.034+0000 7fcd977fe700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd9815dfe0 0x7fcd98140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:29.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.034+0000 7fcd967fc700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd98142ea0 0x7fcd9815daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:29.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.034+0000 7fcd96ffd700 1 -- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd98142ea0 msgr2=0x7fcd9815daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:29.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.034+0000 7fcd96ffd700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd98142ea0 0x7fcd9815daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.034+0000 7fcd96ffd700 1 -- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd9815dfe0 msgr2=0x7fcd98140d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.034+0000 7fcd96ffd700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd9815dfe0 0x7fcd98140d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.034+0000 7fcd96ffd700 1 -- 172.21.15.55:0/427167673 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c0094f0 con 0x7fcd9806bf20 2023-10-06T03:48:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.034+0000 7fcd977fe700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd9815dfe0 0x7fcd98140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.034+0000 7fcd967fc700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd98142ea0 0x7fcd9815daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.034+0000 7fcd96ffd700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd9806bf20 0x7fcd98131d00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800c4e0 tx=0x7fcd8800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.034+0000 7fcd7effd700 1 -- 172.21.15.55:0/427167673 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88011590 con 0x7fcd9806bf20 2023-10-06T03:48:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.035+0000 7fcd7effd700 1 -- 172.21.15.55:0/427167673 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd88011710 con 0x7fcd9806bf20 2023-10-06T03:48:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.035+0000 7fcd97fff700 1 -- 172.21.15.55:0/427167673 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd980ae790 con 0x7fcd9806bf20 2023-10-06T03:48:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.035+0000 7fcd7effd700 1 -- 172.21.15.55:0/427167673 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88011ad0 con 0x7fcd9806bf20 2023-10-06T03:48:29.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.035+0000 7fcd97fff700 1 -- 172.21.15.55:0/427167673 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd980aed50 con 0x7fcd9806bf20 2023-10-06T03:48:29.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.036+0000 7fcd97fff700 1 -- 172.21.15.55:0/427167673 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9804c3c0 con 0x7fcd9806bf20 2023-10-06T03:48:29.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.037+0000 7fcd7effd700 1 -- 172.21.15.55:0/427167673 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd8801b3b0 con 0x7fcd9806bf20 2023-10-06T03:48:29.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.037+0000 7fcd7effd700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd84066f30 0x7fcd840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:29.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.038+0000 7fcd7effd700 1 -- 172.21.15.55:0/427167673 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fcd88083e00 con 0x7fcd9806bf20 2023-10-06T03:48:29.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.038+0000 7fcd967fc700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd84066f30 0x7fcd840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:29.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.039+0000 7fcd967fc700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd84066f30 0x7fcd840693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcd980b12a0 tx=0x7fcd8c017000 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:29.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.042+0000 7fcd7effd700 1 -- 172.21.15.55:0/427167673 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd88050ce0 con 0x7fcd9806bf20 2023-10-06T03:48:29.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.241+0000 7fcd97fff700 1 -- 172.21.15.55:0/427167673 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd98069c00 con 0x7fcd84066f30 2023-10-06T03:48:29.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.251+0000 7fcd7effd700 1 -- 172.21.15.55:0/427167673 <== mgr.5483 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310900 (secure 0 0 0) 0x7fcd98069c00 con 0x7fcd84066f30 2023-10-06T03:48:29.256 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:29.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.255+0000 7fcd97fff700 1 -- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd84066f30 msgr2=0x7fcd840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:29.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.255+0000 7fcd97fff700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd84066f30 0x7fcd840693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcd980b12a0 tx=0x7fcd8c017000 comp rx=0 tx=0).stop 2023-10-06T03:48:29.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.255+0000 7fcd97fff700 1 -- 172.21.15.55:0/427167673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd9806bf20 msgr2=0x7fcd98131d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:29.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.255+0000 7fcd97fff700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd9806bf20 0x7fcd98131d00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800c4e0 tx=0x7fcd8800c8a0 comp rx=0 tx=0).stop 2023-10-06T03:48:29.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.255+0000 7fcd97fff700 1 -- 172.21.15.55:0/427167673 shutdown_connections 2023-10-06T03:48:29.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.255+0000 7fcd97fff700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd84066f30 0x7fcd840693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:29.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.255+0000 7fcd97fff700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd9815dfe0 0x7fcd98140d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:29.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.255+0000 7fcd97fff700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd9806bf20 0x7fcd98131d00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:29.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.255+0000 7fcd97fff700 1 --2- 172.21.15.55:0/427167673 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd98142ea0 0x7fcd9815daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:29.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.255+0000 7fcd97fff700 1 -- 172.21.15.55:0/427167673 >> 172.21.15.55:0/427167673 conn(0x7fcd980ce430 msgr2=0x7fcd9815f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:29.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.256+0000 7fcd97fff700 1 -- 172.21.15.55:0/427167673 shutdown_connections 2023-10-06T03:48:29.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:29.256+0000 7fcd97fff700 1 -- 172.21.15.55:0/427167673 wait complete. 2023-10-06T03:48:29.260 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:48:29.274 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T03:48:28.016994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444408,"kb_used_data":8760,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752530312,"statfs":{"total":773094113280,"available":770591039488,"internally_reserved":0,"allocated":8970240,"data_stored":5069340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.082923"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981925+0000","last_change":"2023-10-06T03:48:22.978687+0000","last_active":"2023-10-06T03:48:22.981925+0000","last_peered":"2023-10-06T03:48:22.981925+0000","last_clean":"2023-10-06T03:48:22.981925+0000","last_became_active":"2023-10-06T03:48:22.977640+0000","last_became_peered":"2023-10-06T03:48:22.977640+0000","last_unstale":"2023-10-06T03:48:22.981925+0000","last_undegraded":"2023-10-06T03:48:22.981925+0000","last_fullsized":"2023-10-06T03:48:22.981925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:06.518654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980965+0000","last_change":"2023-10-06T03:48:22.965813+0000","last_active":"2023-10-06T03:48:22.980965+0000","last_peered":"2023-10-06T03:48:22.980965+0000","last_clean":"2023-10-06T03:48:22.980965+0000","last_became_active":"2023-10-06T03:48:22.965532+0000","last_became_peered":"2023-10-06T03:48:22.965532+0000","last_unstale":"2023-10-06T03:48:22.980965+0000","last_undegraded":"2023-10-06T03:48:22.980965+0000","last_fullsized":"2023-10-06T03:48:22.980965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:36.717228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.984687+0000","last_change":"2023-10-06T03:48:22.983311+0000","last_active":"2023-10-06T03:48:22.984687+0000","last_peered":"2023-10-06T03:48:22.984687+0000","last_clean":"2023-10-06T03:48:22.984687+0000","last_became_active":"2023-10-06T03:48:22.983010+0000","last_became_peered":"2023-10-06T03:48:22.983010+0000","last_unstale":"2023-10-06T03:48:22.984687+0000","last_undegraded":"2023-10-06T03:48:22.984687+0000","last_fullsized":"2023-10-06T03:48:22.984687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:12.925229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971611+0000","last_change":"2023-10-06T03:48:22.964273+0000","last_active":"2023-10-06T03:48:22.971611+0000","last_peered":"2023-10-06T03:48:22.971611+0000","last_clean":"2023-10-06T03:48:22.971611+0000","last_became_active":"2023-10-06T03:48:22.963978+0000","last_became_peered":"2023-10-06T03:48:22.963978+0000","last_unstale":"2023-10-06T03:48:22.971611+0000","last_undegraded":"2023-10-06T03:48:22.971611+0000","last_fullsized":"2023-10-06T03:48:22.971611+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:13.237222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970479+0000","last_change":"2023-10-06T03:48:22.960460+0000","last_active":"2023-10-06T03:48:22.970479+0000","last_peered":"2023-10-06T03:48:22.970479+0000","last_clean":"2023-10-06T03:48:22.970479+0000","last_became_active":"2023-10-06T03:48:22.959985+0000","last_became_peered":"2023-10-06T03:48:22.959985+0000","last_unstale":"2023-10-06T03:48:22.970479+0000","last_undegraded":"2023-10-06T03:48:22.970479+0000","last_fullsized":"2023-10-06T03:48:22.970479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:39.409932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980760+0000","last_change":"2023-10-06T03:48:22.975932+0000","last_active":"2023-10-06T03:48:22.980760+0000","last_peered":"2023-10-06T03:48:22.980760+0000","last_clean":"2023-10-06T03:48:22.980760+0000","last_became_active":"2023-10-06T03:48:22.975667+0000","last_became_peered":"2023-10-06T03:48:22.975667+0000","last_unstale":"2023-10-06T03:48:22.980760+0000","last_undegraded":"2023-10-06T03:48:22.980760+0000","last_fullsized":"2023-10-06T03:48:22.980760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:49.669934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.985364+0000","last_change":"2023-10-06T03:48:22.980964+0000","last_active":"2023-10-06T03:48:22.985364+0000","last_peered":"2023-10-06T03:48:22.985364+0000","last_clean":"2023-10-06T03:48:22.985364+0000","last_became_active":"2023-10-06T03:48:22.980709+0000","last_became_peered":"2023-10-06T03:48:22.980709+0000","last_unstale":"2023-10-06T03:48:22.985364+0000","last_undegraded":"2023-10-06T03:48:22.985364+0000","last_fullsized":"2023-10-06T03:48:22.985364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:53.633631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.976769+0000","last_change":"2023-10-06T03:48:22.973481+0000","last_active":"2023-10-06T03:48:22.976769+0000","last_peered":"2023-10-06T03:48:22.976769+0000","last_clean":"2023-10-06T03:48:22.976769+0000","last_became_active":"2023-10-06T03:48:22.973158+0000","last_became_peered":"2023-10-06T03:48:22.973158+0000","last_unstale":"2023-10-06T03:48:22.976769+0000","last_undegraded":"2023-10-06T03:48:22.976769+0000","last_fullsized":"2023-10-06T03:48:22.976769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:24.959369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980277+0000","last_change":"2023-10-06T03:48:22.978760+0000","last_active":"2023-10-06T03:48:22.980277+0000","last_peered":"2023-10-06T03:48:22.980277+0000","last_clean":"2023-10-06T03:48:22.980277+0000","last_became_active":"2023-10-06T03:48:22.978459+0000","last_became_peered":"2023-10-06T03:48:22.978459+0000","last_unstale":"2023-10-06T03:48:22.980277+0000","last_undegraded":"2023-10-06T03:48:22.980277+0000","last_fullsized":"2023-10-06T03:48:22.980277+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:30.408052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.983962+0000","last_change":"2023-10-06T03:48:22.983073+0000","last_active":"2023-10-06T03:48:22.983962+0000","last_peered":"2023-10-06T03:48:22.983962+0000","last_clean":"2023-10-06T03:48:22.983962+0000","last_became_active":"2023-10-06T03:48:22.982662+0000","last_became_peered":"2023-10-06T03:48:22.982662+0000","last_unstale":"2023-10-06T03:48:22.983962+0000","last_undegraded":"2023-10-06T03:48:22.983962+0000","last_fullsized":"2023-10-06T03:48:22.983962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:13.237222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.973340+0000","last_change":"2023-10-06T03:48:22.969513+0000","last_active":"2023-10-06T03:48:22.973340+0000","last_peered":"2023-10-06T03:48:22.973340+0000","last_clean":"2023-10-06T03:48:22.973340+0000","last_became_active":"2023-10-06T03:48:22.969211+0000","last_became_peered":"2023-10-06T03:48:22.969211+0000","last_unstale":"2023-10-06T03:48:22.973340+0000","last_undegraded":"2023-10-06T03:48:22.973340+0000","last_fullsized":"2023-10-06T03:48:22.973340+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:58.681620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.975883+0000","last_change":"2023-10-06T03:48:22.973519+0000","last_active":"2023-10-06T03:48:22.975883+0000","last_peered":"2023-10-06T03:48:22.975883+0000","last_clean":"2023-10-06T03:48:22.975883+0000","last_became_active":"2023-10-06T03:48:22.973239+0000","last_became_peered":"2023-10-06T03:48:22.973239+0000","last_unstale":"2023-10-06T03:48:22.975883+0000","last_undegraded":"2023-10-06T03:48:22.975883+0000","last_fullsized":"2023-10-06T03:48:22.975883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:04.560113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.976082+0000","last_change":"2023-10-06T03:48:22.970347+0000","last_active":"2023-10-06T03:48:22.976082+0000","last_peered":"2023-10-06T03:48:22.976082+0000","last_clean":"2023-10-06T03:48:22.976082+0000","last_became_active":"2023-10-06T03:48:22.970084+0000","last_became_peered":"2023-10-06T03:48:22.970084+0000","last_unstale":"2023-10-06T03:48:22.976082+0000","last_undegraded":"2023-10-06T03:48:22.976082+0000","last_fullsized":"2023-10-06T03:48:22.976082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:23.317887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.985566+0000","last_change":"2023-10-06T03:48:22.984621+0000","last_active":"2023-10-06T03:48:22.985566+0000","last_peered":"2023-10-06T03:48:22.985566+0000","last_clean":"2023-10-06T03:48:22.985566+0000","last_became_active":"2023-10-06T03:48:22.984339+0000","last_became_peered":"2023-10-06T03:48:22.984339+0000","last_unstale":"2023-10-06T03:48:22.985566+0000","last_undegraded":"2023-10-06T03:48:22.985566+0000","last_fullsized":"2023-10-06T03:48:22.985566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:35.725454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.984743+0000","last_change":"2023-10-06T03:48:22.983782+0000","last_active":"2023-10-06T03:48:22.984743+0000","last_peered":"2023-10-06T03:48:22.984743+0000","last_clean":"2023-10-06T03:48:22.984743+0000","last_became_active":"2023-10-06T03:48:22.982670+0000","last_became_peered":"2023-10-06T03:48:22.982670+0000","last_unstale":"2023-10-06T03:48:22.984743+0000","last_undegraded":"2023-10-06T03:48:22.984743+0000","last_fullsized":"2023-10-06T03:48:22.984743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:50.215524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.978789+0000","last_change":"2023-10-06T03:48:22.972912+0000","last_active":"2023-10-06T03:48:22.978789+0000","last_peered":"2023-10-06T03:48:22.978789+0000","last_clean":"2023-10-06T03:48:22.978789+0000","last_became_active":"2023-10-06T03:48:22.972623+0000","last_became_peered":"2023-10-06T03:48:22.972623+0000","last_unstale":"2023-10-06T03:48:22.978789+0000","last_undegraded":"2023-10-06T03:48:22.978789+0000","last_fullsized":"2023-10-06T03:48:22.978789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:44.334010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981079+0000","last_change":"2023-10-06T03:48:22.973112+0000","last_active":"2023-10-06T03:48:22.981079+0000","last_peered":"2023-10-06T03:48:22.981079+0000","last_clean":"2023-10-06T03:48:22.981079+0000","last_became_active":"2023-10-06T03:48:22.972785+0000","last_became_peered":"2023-10-06T03:48:22.972785+0000","last_unstale":"2023-10-06T03:48:22.981079+0000","last_undegraded":"2023-10-06T03:48:22.981079+0000","last_fullsized":"2023-10-06T03:48:22.981079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:14.661907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971574+0000","last_change":"2023-10-06T03:48:22.964446+0000","last_active":"2023-10-06T03:48:22.971574+0000","last_peered":"2023-10-06T03:48:22.971574+0000","last_clean":"2023-10-06T03:48:22.971574+0000","last_became_active":"2023-10-06T03:48:22.964198+0000","last_became_peered":"2023-10-06T03:48:22.964198+0000","last_unstale":"2023-10-06T03:48:22.971574+0000","last_undegraded":"2023-10-06T03:48:22.971574+0000","last_fullsized":"2023-10-06T03:48:22.971574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:13.522488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972748+0000","last_change":"2023-10-06T03:48:22.964606+0000","last_active":"2023-10-06T03:48:22.972748+0000","last_peered":"2023-10-06T03:48:22.972748+0000","last_clean":"2023-10-06T03:48:22.972748+0000","last_became_active":"2023-10-06T03:48:22.964252+0000","last_became_peered":"2023-10-06T03:48:22.964252+0000","last_unstale":"2023-10-06T03:48:22.972748+0000","last_undegraded":"2023-10-06T03:48:22.972748+0000","last_fullsized":"2023-10-06T03:48:22.972748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:59.411057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981867+0000","last_change":"2023-10-06T03:48:22.972302+0000","last_active":"2023-10-06T03:48:22.981867+0000","last_peered":"2023-10-06T03:48:22.981867+0000","last_clean":"2023-10-06T03:48:22.981867+0000","last_became_active":"2023-10-06T03:48:22.972024+0000","last_became_peered":"2023-10-06T03:48:22.972024+0000","last_unstale":"2023-10-06T03:48:22.981867+0000","last_undegraded":"2023-10-06T03:48:22.981867+0000","last_fullsized":"2023-10-06T03:48:22.981867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:00.072378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.974860+0000","last_change":"2023-10-06T03:48:22.968910+0000","last_active":"2023-10-06T03:48:22.974860+0000","last_peered":"2023-10-06T03:48:22.974860+0000","last_clean":"2023-10-06T03:48:22.974860+0000","last_became_active":"2023-10-06T03:48:22.968335+0000","last_became_peered":"2023-10-06T03:48:22.968335+0000","last_unstale":"2023-10-06T03:48:22.974860+0000","last_undegraded":"2023-10-06T03:48:22.974860+0000","last_fullsized":"2023-10-06T03:48:22.974860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:10.057932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981805+0000","last_change":"2023-10-06T03:48:22.975177+0000","last_active":"2023-10-06T03:48:22.981805+0000","last_peered":"2023-10-06T03:48:22.981805+0000","last_clean":"2023-10-06T03:48:22.981805+0000","last_became_active":"2023-10-06T03:48:22.974184+0000","last_became_peered":"2023-10-06T03:48:22.974184+0000","last_unstale":"2023-10-06T03:48:22.981805+0000","last_undegraded":"2023-10-06T03:48:22.981805+0000","last_fullsized":"2023-10-06T03:48:22.981805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:44.273052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981912+0000","last_change":"2023-10-06T03:48:22.973196+0000","last_active":"2023-10-06T03:48:22.981912+0000","last_peered":"2023-10-06T03:48:22.981912+0000","last_clean":"2023-10-06T03:48:22.981912+0000","last_became_active":"2023-10-06T03:48:22.972823+0000","last_became_peered":"2023-10-06T03:48:22.972823+0000","last_unstale":"2023-10-06T03:48:22.981912+0000","last_undegraded":"2023-10-06T03:48:22.981912+0000","last_fullsized":"2023-10-06T03:48:22.981912+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:02.782253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.985661+0000","last_change":"2023-10-06T03:48:22.984009+0000","last_active":"2023-10-06T03:48:22.985661+0000","last_peered":"2023-10-06T03:48:22.985661+0000","last_clean":"2023-10-06T03:48:22.985661+0000","last_became_active":"2023-10-06T03:48:22.983746+0000","last_became_peered":"2023-10-06T03:48:22.983746+0000","last_unstale":"2023-10-06T03:48:22.985661+0000","last_undegraded":"2023-10-06T03:48:22.985661+0000","last_fullsized":"2023-10-06T03:48:22.985661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:12.970745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.976801+0000","last_change":"2023-10-06T03:48:22.969794+0000","last_active":"2023-10-06T03:48:22.976801+0000","last_peered":"2023-10-06T03:48:22.976801+0000","last_clean":"2023-10-06T03:48:22.976801+0000","last_became_active":"2023-10-06T03:48:22.969509+0000","last_became_peered":"2023-10-06T03:48:22.969509+0000","last_unstale":"2023-10-06T03:48:22.976801+0000","last_undegraded":"2023-10-06T03:48:22.976801+0000","last_fullsized":"2023-10-06T03:48:22.976801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:00.871577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.975985+0000","last_change":"2023-10-06T03:48:22.969265+0000","last_active":"2023-10-06T03:48:22.975985+0000","last_peered":"2023-10-06T03:48:22.975985+0000","last_clean":"2023-10-06T03:48:22.975985+0000","last_became_active":"2023-10-06T03:48:22.968581+0000","last_became_peered":"2023-10-06T03:48:22.968581+0000","last_unstale":"2023-10-06T03:48:22.975985+0000","last_undegraded":"2023-10-06T03:48:22.975985+0000","last_fullsized":"2023-10-06T03:48:22.975985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:15.583929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972008+0000","last_change":"2023-10-06T03:48:22.965736+0000","last_active":"2023-10-06T03:48:22.972008+0000","last_peered":"2023-10-06T03:48:22.972008+0000","last_clean":"2023-10-06T03:48:22.972008+0000","last_became_active":"2023-10-06T03:48:22.965464+0000","last_became_peered":"2023-10-06T03:48:22.965464+0000","last_unstale":"2023-10-06T03:48:22.972008+0000","last_undegraded":"2023-10-06T03:48:22.972008+0000","last_fullsized":"2023-10-06T03:48:22.972008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:50.215524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980795+0000","last_change":"2023-10-06T03:48:22.969599+0000","last_active":"2023-10-06T03:48:22.980795+0000","last_peered":"2023-10-06T03:48:22.980795+0000","last_clean":"2023-10-06T03:48:22.980795+0000","last_became_active":"2023-10-06T03:48:22.969334+0000","last_became_peered":"2023-10-06T03:48:22.969334+0000","last_unstale":"2023-10-06T03:48:22.980795+0000","last_undegraded":"2023-10-06T03:48:22.980795+0000","last_fullsized":"2023-10-06T03:48:22.980795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:33.357173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.979756+0000","last_change":"2023-10-06T03:48:22.977428+0000","last_active":"2023-10-06T03:48:22.979756+0000","last_peered":"2023-10-06T03:48:22.979756+0000","last_clean":"2023-10-06T03:48:22.979756+0000","last_became_active":"2023-10-06T03:48:22.977087+0000","last_became_peered":"2023-10-06T03:48:22.977087+0000","last_unstale":"2023-10-06T03:48:22.979756+0000","last_undegraded":"2023-10-06T03:48:22.979756+0000","last_fullsized":"2023-10-06T03:48:22.979756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:17.224716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.975912+0000","last_change":"2023-10-06T03:48:22.974325+0000","last_active":"2023-10-06T03:48:22.975912+0000","last_peered":"2023-10-06T03:48:22.975912+0000","last_clean":"2023-10-06T03:48:22.975912+0000","last_became_active":"2023-10-06T03:48:22.974082+0000","last_became_peered":"2023-10-06T03:48:22.974082+0000","last_unstale":"2023-10-06T03:48:22.975912+0000","last_undegraded":"2023-10-06T03:48:22.975912+0000","last_fullsized":"2023-10-06T03:48:22.975912+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:14.661907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972101+0000","last_change":"2023-10-06T03:48:22.968457+0000","last_active":"2023-10-06T03:48:22.972101+0000","last_peered":"2023-10-06T03:48:22.972101+0000","last_clean":"2023-10-06T03:48:22.972101+0000","last_became_active":"2023-10-06T03:48:22.968182+0000","last_became_peered":"2023-10-06T03:48:22.968182+0000","last_unstale":"2023-10-06T03:48:22.972101+0000","last_undegraded":"2023-10-06T03:48:22.972101+0000","last_fullsized":"2023-10-06T03:48:22.972101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:47.901242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980159+0000","last_change":"2023-10-06T03:48:22.968721+0000","last_active":"2023-10-06T03:48:22.980159+0000","last_peered":"2023-10-06T03:48:22.980159+0000","last_clean":"2023-10-06T03:48:22.980159+0000","last_became_active":"2023-10-06T03:48:22.968451+0000","last_became_peered":"2023-10-06T03:48:22.968451+0000","last_unstale":"2023-10-06T03:48:22.980159+0000","last_undegraded":"2023-10-06T03:48:22.980159+0000","last_fullsized":"2023-10-06T03:48:22.980159+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:47:58.626319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970581+0000","last_change":"2023-10-06T03:48:22.963394+0000","last_active":"2023-10-06T03:48:22.970581+0000","last_peered":"2023-10-06T03:48:22.970581+0000","last_clean":"2023-10-06T03:48:22.970581+0000","last_became_active":"2023-10-06T03:48:22.963073+0000","last_became_peered":"2023-10-06T03:48:22.963073+0000","last_unstale":"2023-10-06T03:48:22.970581+0000","last_undegraded":"2023-10-06T03:48:22.970581+0000","last_fullsized":"2023-10-06T03:48:22.970581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:15.351000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"79'4","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.992673+0000","last_change":"2023-10-06T03:48:21.959573+0000","last_active":"2023-10-06T03:48:22.992673+0000","last_peered":"2023-10-06T03:48:22.992673+0000","last_clean":"2023-10-06T03:48:22.992673+0000","last_became_active":"2023-10-06T03:48:21.959316+0000","last_became_peered":"2023-10-06T03:48:21.959316+0000","last_unstale":"2023-10-06T03:48:22.992673+0000","last_undegraded":"2023-10-06T03:48:22.992673+0000","last_fullsized":"2023-10-06T03:48:22.992673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:11.472736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971988+0000","last_change":"2023-10-06T03:48:22.961954+0000","last_active":"2023-10-06T03:48:22.971988+0000","last_peered":"2023-10-06T03:48:22.971988+0000","last_clean":"2023-10-06T03:48:22.971988+0000","last_became_active":"2023-10-06T03:48:22.961642+0000","last_became_peered":"2023-10-06T03:48:22.961642+0000","last_unstale":"2023-10-06T03:48:22.971988+0000","last_undegraded":"2023-10-06T03:48:22.971988+0000","last_fullsized":"2023-10-06T03:48:22.971988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:40.770762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.986713+0000","last_change":"2023-10-06T03:48:21.954192+0000","last_active":"2023-10-06T03:48:22.986713+0000","last_peered":"2023-10-06T03:48:22.986713+0000","last_clean":"2023-10-06T03:48:22.986713+0000","last_became_active":"2023-10-06T03:48:21.953846+0000","last_became_peered":"2023-10-06T03:48:21.953846+0000","last_unstale":"2023-10-06T03:48:22.986713+0000","last_undegraded":"2023-10-06T03:48:22.986713+0000","last_fullsized":"2023-10-06T03:48:22.986713+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:38.007175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980734+0000","last_change":"2023-10-06T03:48:22.967986+0000","last_active":"2023-10-06T03:48:22.980734+0000","last_peered":"2023-10-06T03:48:22.980734+0000","last_clean":"2023-10-06T03:48:22.980734+0000","last_became_active":"2023-10-06T03:48:22.967743+0000","last_became_peered":"2023-10-06T03:48:22.967743+0000","last_unstale":"2023-10-06T03:48:22.980734+0000","last_undegraded":"2023-10-06T03:48:22.980734+0000","last_fullsized":"2023-10-06T03:48:22.980734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:46.815803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.988858+0000","last_change":"2023-10-06T03:48:21.952725+0000","last_active":"2023-10-06T03:48:22.988858+0000","last_peered":"2023-10-06T03:48:22.988858+0000","last_clean":"2023-10-06T03:48:22.988858+0000","last_became_active":"2023-10-06T03:48:21.952307+0000","last_became_peered":"2023-10-06T03:48:21.952307+0000","last_unstale":"2023-10-06T03:48:22.988858+0000","last_undegraded":"2023-10-06T03:48:22.988858+0000","last_fullsized":"2023-10-06T03:48:22.988858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:16.562701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.985371+0000","last_change":"2023-10-06T03:48:22.983206+0000","last_active":"2023-10-06T03:48:22.985371+0000","last_peered":"2023-10-06T03:48:22.985371+0000","last_clean":"2023-10-06T03:48:22.985371+0000","last_became_active":"2023-10-06T03:48:22.982830+0000","last_became_peered":"2023-10-06T03:48:22.982830+0000","last_unstale":"2023-10-06T03:48:22.985371+0000","last_undegraded":"2023-10-06T03:48:22.985371+0000","last_fullsized":"2023-10-06T03:48:22.985371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:47.901242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.987882+0000","last_change":"2023-10-06T03:48:21.954325+0000","last_active":"2023-10-06T03:48:22.987882+0000","last_peered":"2023-10-06T03:48:22.987882+0000","last_clean":"2023-10-06T03:48:22.987882+0000","last_became_active":"2023-10-06T03:48:21.954034+0000","last_became_peered":"2023-10-06T03:48:21.954034+0000","last_unstale":"2023-10-06T03:48:22.987882+0000","last_undegraded":"2023-10-06T03:48:22.987882+0000","last_fullsized":"2023-10-06T03:48:22.987882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:01.772602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970732+0000","last_change":"2023-10-06T03:48:22.969730+0000","last_active":"2023-10-06T03:48:22.970732+0000","last_peered":"2023-10-06T03:48:22.970732+0000","last_clean":"2023-10-06T03:48:22.970732+0000","last_became_active":"2023-10-06T03:48:22.969449+0000","last_became_peered":"2023-10-06T03:48:22.969449+0000","last_unstale":"2023-10-06T03:48:22.970732+0000","last_undegraded":"2023-10-06T03:48:22.970732+0000","last_fullsized":"2023-10-06T03:48:22.970732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:41.165824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.984300+0000","last_change":"2023-10-06T03:48:21.946173+0000","last_active":"2023-10-06T03:48:22.984300+0000","last_peered":"2023-10-06T03:48:22.984300+0000","last_clean":"2023-10-06T03:48:22.984300+0000","last_became_active":"2023-10-06T03:48:21.945822+0000","last_became_peered":"2023-10-06T03:48:21.945822+0000","last_unstale":"2023-10-06T03:48:22.984300+0000","last_undegraded":"2023-10-06T03:48:22.984300+0000","last_fullsized":"2023-10-06T03:48:22.984300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:46.891591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981121+0000","last_change":"2023-10-06T03:48:22.969630+0000","last_active":"2023-10-06T03:48:22.981121+0000","last_peered":"2023-10-06T03:48:22.981121+0000","last_clean":"2023-10-06T03:48:22.981121+0000","last_became_active":"2023-10-06T03:48:22.969362+0000","last_became_peered":"2023-10-06T03:48:22.969362+0000","last_unstale":"2023-10-06T03:48:22.981121+0000","last_undegraded":"2023-10-06T03:48:22.981121+0000","last_fullsized":"2023-10-06T03:48:22.981121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:04.560113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.986941+0000","last_change":"2023-10-06T03:48:21.944675+0000","last_active":"2023-10-06T03:48:22.986941+0000","last_peered":"2023-10-06T03:48:22.986941+0000","last_clean":"2023-10-06T03:48:22.986941+0000","last_became_active":"2023-10-06T03:48:21.944421+0000","last_became_peered":"2023-10-06T03:48:21.944421+0000","last_unstale":"2023-10-06T03:48:22.986941+0000","last_undegraded":"2023-10-06T03:48:22.986941+0000","last_fullsized":"2023-10-06T03:48:22.986941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:12.199710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.974865+0000","last_change":"2023-10-06T03:48:22.966222+0000","last_active":"2023-10-06T03:48:22.974865+0000","last_peered":"2023-10-06T03:48:22.974865+0000","last_clean":"2023-10-06T03:48:22.974865+0000","last_became_active":"2023-10-06T03:48:22.965935+0000","last_became_peered":"2023-10-06T03:48:22.965935+0000","last_unstale":"2023-10-06T03:48:22.974865+0000","last_undegraded":"2023-10-06T03:48:22.974865+0000","last_fullsized":"2023-10-06T03:48:22.974865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:24.367540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.961390+0000","last_change":"2023-10-06T03:48:21.951726+0000","last_active":"2023-10-06T03:48:22.961390+0000","last_peered":"2023-10-06T03:48:22.961390+0000","last_clean":"2023-10-06T03:48:22.961390+0000","last_became_active":"2023-10-06T03:48:21.951477+0000","last_became_peered":"2023-10-06T03:48:21.951477+0000","last_unstale":"2023-10-06T03:48:22.961390+0000","last_undegraded":"2023-10-06T03:48:22.961390+0000","last_fullsized":"2023-10-06T03:48:22.961390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:16.215064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.982148+0000","last_change":"2023-10-06T03:48:22.974712+0000","last_active":"2023-10-06T03:48:22.982148+0000","last_peered":"2023-10-06T03:48:22.982148+0000","last_clean":"2023-10-06T03:48:22.982148+0000","last_became_active":"2023-10-06T03:48:22.974451+0000","last_became_peered":"2023-10-06T03:48:22.974451+0000","last_unstale":"2023-10-06T03:48:22.982148+0000","last_undegraded":"2023-10-06T03:48:22.982148+0000","last_fullsized":"2023-10-06T03:48:22.982148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:50.993039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.953339+0000","last_change":"2023-10-06T03:48:21.954823+0000","last_active":"2023-10-06T03:48:22.953339+0000","last_peered":"2023-10-06T03:48:22.953339+0000","last_clean":"2023-10-06T03:48:22.953339+0000","last_became_active":"2023-10-06T03:48:21.954575+0000","last_became_peered":"2023-10-06T03:48:21.954575+0000","last_unstale":"2023-10-06T03:48:22.953339+0000","last_undegraded":"2023-10-06T03:48:22.953339+0000","last_fullsized":"2023-10-06T03:48:22.953339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:58.471188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980776+0000","last_change":"2023-10-06T03:48:22.965327+0000","last_active":"2023-10-06T03:48:22.980776+0000","last_peered":"2023-10-06T03:48:22.980776+0000","last_clean":"2023-10-06T03:48:22.980776+0000","last_became_active":"2023-10-06T03:48:22.964956+0000","last_became_peered":"2023-10-06T03:48:22.964956+0000","last_unstale":"2023-10-06T03:48:22.980776+0000","last_undegraded":"2023-10-06T03:48:22.980776+0000","last_fullsized":"2023-10-06T03:48:22.980776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:10.057932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.966607+0000","last_change":"2023-10-06T03:48:21.962473+0000","last_active":"2023-10-06T03:48:22.966607+0000","last_peered":"2023-10-06T03:48:22.966607+0000","last_clean":"2023-10-06T03:48:22.966607+0000","last_became_active":"2023-10-06T03:48:21.962144+0000","last_became_peered":"2023-10-06T03:48:21.962144+0000","last_unstale":"2023-10-06T03:48:22.966607+0000","last_undegraded":"2023-10-06T03:48:22.966607+0000","last_fullsized":"2023-10-06T03:48:22.966607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:23.381899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981836+0000","last_change":"2023-10-06T03:48:22.970666+0000","last_active":"2023-10-06T03:48:22.981836+0000","last_peered":"2023-10-06T03:48:22.981836+0000","last_clean":"2023-10-06T03:48:22.981836+0000","last_became_active":"2023-10-06T03:48:22.970339+0000","last_became_peered":"2023-10-06T03:48:22.970339+0000","last_unstale":"2023-10-06T03:48:22.981836+0000","last_undegraded":"2023-10-06T03:48:22.981836+0000","last_fullsized":"2023-10-06T03:48:22.981836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:09.125474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.953769+0000","last_change":"2023-10-06T03:48:21.956450+0000","last_active":"2023-10-06T03:48:22.953769+0000","last_peered":"2023-10-06T03:48:22.953769+0000","last_clean":"2023-10-06T03:48:22.953769+0000","last_became_active":"2023-10-06T03:48:21.956099+0000","last_became_peered":"2023-10-06T03:48:21.956099+0000","last_unstale":"2023-10-06T03:48:22.953769+0000","last_undegraded":"2023-10-06T03:48:22.953769+0000","last_fullsized":"2023-10-06T03:48:22.953769+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:03.550462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.978245+0000","last_change":"2023-10-06T03:48:22.977370+0000","last_active":"2023-10-06T03:48:22.978245+0000","last_peered":"2023-10-06T03:48:22.978245+0000","last_clean":"2023-10-06T03:48:22.978245+0000","last_became_active":"2023-10-06T03:48:22.977088+0000","last_became_peered":"2023-10-06T03:48:22.977088+0000","last_unstale":"2023-10-06T03:48:22.978245+0000","last_undegraded":"2023-10-06T03:48:22.978245+0000","last_fullsized":"2023-10-06T03:48:22.978245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:47.548376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"79'3","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.992610+0000","last_change":"2023-10-06T03:48:21.962149+0000","last_active":"2023-10-06T03:48:22.992610+0000","last_peered":"2023-10-06T03:48:22.992610+0000","last_clean":"2023-10-06T03:48:22.992610+0000","last_became_active":"2023-10-06T03:48:21.961686+0000","last_became_peered":"2023-10-06T03:48:21.961686+0000","last_unstale":"2023-10-06T03:48:22.992610+0000","last_undegraded":"2023-10-06T03:48:22.992610+0000","last_fullsized":"2023-10-06T03:48:22.992610+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:14.197511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.973148+0000","last_change":"2023-10-06T03:48:22.968754+0000","last_active":"2023-10-06T03:48:22.973148+0000","last_peered":"2023-10-06T03:48:22.973148+0000","last_clean":"2023-10-06T03:48:22.973148+0000","last_became_active":"2023-10-06T03:48:22.968469+0000","last_became_peered":"2023-10-06T03:48:22.968469+0000","last_unstale":"2023-10-06T03:48:22.973148+0000","last_undegraded":"2023-10-06T03:48:22.973148+0000","last_fullsized":"2023-10-06T03:48:22.973148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:30.408052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"79'6","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.990295+0000","last_change":"2023-10-06T03:48:21.959258+0000","last_active":"2023-10-06T03:48:22.990295+0000","last_peered":"2023-10-06T03:48:22.990295+0000","last_clean":"2023-10-06T03:48:22.990295+0000","last_became_active":"2023-10-06T03:48:21.958956+0000","last_became_peered":"2023-10-06T03:48:21.958956+0000","last_unstale":"2023-10-06T03:48:22.990295+0000","last_undegraded":"2023-10-06T03:48:22.990295+0000","last_fullsized":"2023-10-06T03:48:22.990295+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:46.538724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.977036+0000","last_change":"2023-10-06T03:48:22.974061+0000","last_active":"2023-10-06T03:48:22.977036+0000","last_peered":"2023-10-06T03:48:22.977036+0000","last_clean":"2023-10-06T03:48:22.977036+0000","last_became_active":"2023-10-06T03:48:22.973763+0000","last_became_peered":"2023-10-06T03:48:22.973763+0000","last_unstale":"2023-10-06T03:48:22.977036+0000","last_undegraded":"2023-10-06T03:48:22.977036+0000","last_fullsized":"2023-10-06T03:48:22.977036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:09.125474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.992491+0000","last_change":"2023-10-06T03:48:21.962189+0000","last_active":"2023-10-06T03:48:22.992491+0000","last_peered":"2023-10-06T03:48:22.992491+0000","last_clean":"2023-10-06T03:48:22.992491+0000","last_became_active":"2023-10-06T03:48:21.961581+0000","last_became_peered":"2023-10-06T03:48:21.961581+0000","last_unstale":"2023-10-06T03:48:22.992491+0000","last_undegraded":"2023-10-06T03:48:22.992491+0000","last_fullsized":"2023-10-06T03:48:22.992491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:59.861926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.975953+0000","last_change":"2023-10-06T03:48:22.971613+0000","last_active":"2023-10-06T03:48:22.975953+0000","last_peered":"2023-10-06T03:48:22.975953+0000","last_clean":"2023-10-06T03:48:22.975953+0000","last_became_active":"2023-10-06T03:48:22.971307+0000","last_became_peered":"2023-10-06T03:48:22.971307+0000","last_unstale":"2023-10-06T03:48:22.975953+0000","last_undegraded":"2023-10-06T03:48:22.975953+0000","last_fullsized":"2023-10-06T03:48:22.975953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:36.717228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958572+0000","last_change":"2023-10-06T03:48:21.955946+0000","last_active":"2023-10-06T03:48:22.958572+0000","last_peered":"2023-10-06T03:48:22.958572+0000","last_clean":"2023-10-06T03:48:22.958572+0000","last_became_active":"2023-10-06T03:48:21.955694+0000","last_became_peered":"2023-10-06T03:48:21.955694+0000","last_unstale":"2023-10-06T03:48:22.958572+0000","last_undegraded":"2023-10-06T03:48:22.958572+0000","last_fullsized":"2023-10-06T03:48:22.958572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:23.357889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.974849+0000","last_change":"2023-10-06T03:48:22.965598+0000","last_active":"2023-10-06T03:48:22.974849+0000","last_peered":"2023-10-06T03:48:22.974849+0000","last_clean":"2023-10-06T03:48:22.974849+0000","last_became_active":"2023-10-06T03:48:22.965115+0000","last_became_peered":"2023-10-06T03:48:22.965115+0000","last_unstale":"2023-10-06T03:48:22.974849+0000","last_undegraded":"2023-10-06T03:48:22.974849+0000","last_fullsized":"2023-10-06T03:48:22.974849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:26.181100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.964644+0000","last_change":"2023-10-06T03:48:21.959481+0000","last_active":"2023-10-06T03:48:22.964644+0000","last_peered":"2023-10-06T03:48:22.964644+0000","last_clean":"2023-10-06T03:48:22.964644+0000","last_became_active":"2023-10-06T03:48:21.959233+0000","last_became_peered":"2023-10-06T03:48:21.959233+0000","last_unstale":"2023-10-06T03:48:22.964644+0000","last_undegraded":"2023-10-06T03:48:22.964644+0000","last_fullsized":"2023-10-06T03:48:22.964644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:35.707577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980740+0000","last_change":"2023-10-06T03:48:22.971271+0000","last_active":"2023-10-06T03:48:22.980740+0000","last_peered":"2023-10-06T03:48:22.980740+0000","last_clean":"2023-10-06T03:48:22.980740+0000","last_became_active":"2023-10-06T03:48:22.970969+0000","last_became_peered":"2023-10-06T03:48:22.970969+0000","last_unstale":"2023-10-06T03:48:22.980740+0000","last_undegraded":"2023-10-06T03:48:22.980740+0000","last_fullsized":"2023-10-06T03:48:22.980740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:32.235650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"79'8","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.988873+0000","last_change":"2023-10-06T03:48:21.959296+0000","last_active":"2023-10-06T03:48:22.988873+0000","last_peered":"2023-10-06T03:48:22.988873+0000","last_clean":"2023-10-06T03:48:22.988873+0000","last_became_active":"2023-10-06T03:48:21.958948+0000","last_became_peered":"2023-10-06T03:48:21.958948+0000","last_unstale":"2023-10-06T03:48:22.988873+0000","last_undegraded":"2023-10-06T03:48:22.988873+0000","last_fullsized":"2023-10-06T03:48:22.988873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:13.652256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.983389+0000","last_change":"2023-10-06T03:48:22.979534+0000","last_active":"2023-10-06T03:48:22.983389+0000","last_peered":"2023-10-06T03:48:22.983389+0000","last_clean":"2023-10-06T03:48:22.983389+0000","last_became_active":"2023-10-06T03:48:22.979272+0000","last_became_peered":"2023-10-06T03:48:22.979272+0000","last_unstale":"2023-10-06T03:48:22.983389+0000","last_undegraded":"2023-10-06T03:48:22.983389+0000","last_fullsized":"2023-10-06T03:48:22.983389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:03.329123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.990320+0000","last_change":"2023-10-06T03:48:21.959483+0000","last_active":"2023-10-06T03:48:22.990320+0000","last_peered":"2023-10-06T03:48:22.990320+0000","last_clean":"2023-10-06T03:48:22.990320+0000","last_became_active":"2023-10-06T03:48:21.959233+0000","last_became_peered":"2023-10-06T03:48:21.959233+0000","last_unstale":"2023-10-06T03:48:22.990320+0000","last_undegraded":"2023-10-06T03:48:22.990320+0000","last_fullsized":"2023-10-06T03:48:22.990320+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:08.515631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.967698+0000","last_change":"2023-10-06T03:48:22.964069+0000","last_active":"2023-10-06T03:48:22.967698+0000","last_peered":"2023-10-06T03:48:22.967698+0000","last_clean":"2023-10-06T03:48:22.967698+0000","last_became_active":"2023-10-06T03:48:22.963794+0000","last_became_peered":"2023-10-06T03:48:22.963794+0000","last_unstale":"2023-10-06T03:48:22.967698+0000","last_undegraded":"2023-10-06T03:48:22.967698+0000","last_fullsized":"2023-10-06T03:48:22.967698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:03.317508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.959392+0000","last_change":"2023-10-06T03:48:21.949399+0000","last_active":"2023-10-06T03:48:22.959392+0000","last_peered":"2023-10-06T03:48:22.959392+0000","last_clean":"2023-10-06T03:48:22.959392+0000","last_became_active":"2023-10-06T03:48:21.949065+0000","last_became_peered":"2023-10-06T03:48:21.949065+0000","last_unstale":"2023-10-06T03:48:22.959392+0000","last_undegraded":"2023-10-06T03:48:22.959392+0000","last_fullsized":"2023-10-06T03:48:22.959392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:24.883643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.976002+0000","last_change":"2023-10-06T03:48:22.969329+0000","last_active":"2023-10-06T03:48:22.976002+0000","last_peered":"2023-10-06T03:48:22.976002+0000","last_clean":"2023-10-06T03:48:22.976002+0000","last_became_active":"2023-10-06T03:48:22.969048+0000","last_became_peered":"2023-10-06T03:48:22.969048+0000","last_unstale":"2023-10-06T03:48:22.976002+0000","last_undegraded":"2023-10-06T03:48:22.976002+0000","last_fullsized":"2023-10-06T03:48:22.976002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:15.207162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958635+0000","last_change":"2023-10-06T03:48:21.950832+0000","last_active":"2023-10-06T03:48:22.958635+0000","last_peered":"2023-10-06T03:48:22.958635+0000","last_clean":"2023-10-06T03:48:22.958635+0000","last_became_active":"2023-10-06T03:48:21.950526+0000","last_became_peered":"2023-10-06T03:48:21.950526+0000","last_unstale":"2023-10-06T03:48:22.958635+0000","last_undegraded":"2023-10-06T03:48:22.958635+0000","last_fullsized":"2023-10-06T03:48:22.958635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:41.936178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970721+0000","last_change":"2023-10-06T03:48:22.965699+0000","last_active":"2023-10-06T03:48:22.970721+0000","last_peered":"2023-10-06T03:48:22.970721+0000","last_clean":"2023-10-06T03:48:22.970721+0000","last_became_active":"2023-10-06T03:48:22.965291+0000","last_became_peered":"2023-10-06T03:48:22.965291+0000","last_unstale":"2023-10-06T03:48:22.970721+0000","last_undegraded":"2023-10-06T03:48:22.970721+0000","last_fullsized":"2023-10-06T03:48:22.970721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:58.933292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.956473+0000","last_change":"2023-10-06T03:48:21.958507+0000","last_active":"2023-10-06T03:48:22.956473+0000","last_peered":"2023-10-06T03:48:22.956473+0000","last_clean":"2023-10-06T03:48:22.956473+0000","last_became_active":"2023-10-06T03:48:21.958265+0000","last_became_peered":"2023-10-06T03:48:21.958265+0000","last_unstale":"2023-10-06T03:48:22.956473+0000","last_undegraded":"2023-10-06T03:48:22.956473+0000","last_fullsized":"2023-10-06T03:48:22.956473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:57.923641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972857+0000","last_change":"2023-10-06T03:48:22.963396+0000","last_active":"2023-10-06T03:48:22.972857+0000","last_peered":"2023-10-06T03:48:22.972857+0000","last_clean":"2023-10-06T03:48:22.972857+0000","last_became_active":"2023-10-06T03:48:22.963027+0000","last_became_peered":"2023-10-06T03:48:22.963027+0000","last_unstale":"2023-10-06T03:48:22.972857+0000","last_undegraded":"2023-10-06T03:48:22.972857+0000","last_fullsized":"2023-10-06T03:48:22.972857+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:12.482387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.965233+0000","last_change":"2023-10-06T03:48:21.960124+0000","last_active":"2023-10-06T03:48:22.965233+0000","last_peered":"2023-10-06T03:48:22.965233+0000","last_clean":"2023-10-06T03:48:22.965233+0000","last_became_active":"2023-10-06T03:48:21.959839+0000","last_became_peered":"2023-10-06T03:48:21.959839+0000","last_unstale":"2023-10-06T03:48:22.965233+0000","last_undegraded":"2023-10-06T03:48:22.965233+0000","last_fullsized":"2023-10-06T03:48:22.965233+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:19.590235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.977066+0000","last_change":"2023-10-06T03:48:22.975809+0000","last_active":"2023-10-06T03:48:22.977066+0000","last_peered":"2023-10-06T03:48:22.977066+0000","last_clean":"2023-10-06T03:48:22.977066+0000","last_became_active":"2023-10-06T03:48:22.975112+0000","last_became_peered":"2023-10-06T03:48:22.975112+0000","last_unstale":"2023-10-06T03:48:22.977066+0000","last_undegraded":"2023-10-06T03:48:22.977066+0000","last_fullsized":"2023-10-06T03:48:22.977066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:50.993039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958241+0000","last_change":"2023-10-06T03:48:21.951488+0000","last_active":"2023-10-06T03:48:22.958241+0000","last_peered":"2023-10-06T03:48:22.958241+0000","last_clean":"2023-10-06T03:48:22.958241+0000","last_became_active":"2023-10-06T03:48:21.951196+0000","last_became_peered":"2023-10-06T03:48:21.951196+0000","last_unstale":"2023-10-06T03:48:22.958241+0000","last_undegraded":"2023-10-06T03:48:22.958241+0000","last_fullsized":"2023-10-06T03:48:22.958241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:34.715803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981040+0000","last_change":"2023-10-06T03:48:22.964301+0000","last_active":"2023-10-06T03:48:22.981040+0000","last_peered":"2023-10-06T03:48:22.981040+0000","last_clean":"2023-10-06T03:48:22.981040+0000","last_became_active":"2023-10-06T03:48:22.963870+0000","last_became_peered":"2023-10-06T03:48:22.963870+0000","last_unstale":"2023-10-06T03:48:22.981040+0000","last_undegraded":"2023-10-06T03:48:22.981040+0000","last_fullsized":"2023-10-06T03:48:22.981040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:24.959369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.961484+0000","last_change":"2023-10-06T03:48:21.950936+0000","last_active":"2023-10-06T03:48:22.961484+0000","last_peered":"2023-10-06T03:48:22.961484+0000","last_clean":"2023-10-06T03:48:22.961484+0000","last_became_active":"2023-10-06T03:48:21.950639+0000","last_became_peered":"2023-10-06T03:48:21.950639+0000","last_unstale":"2023-10-06T03:48:22.961484+0000","last_undegraded":"2023-10-06T03:48:22.961484+0000","last_fullsized":"2023-10-06T03:48:22.961484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:11.961094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981729+0000","last_change":"2023-10-06T03:48:22.980115+0000","last_active":"2023-10-06T03:48:22.981729+0000","last_peered":"2023-10-06T03:48:22.981729+0000","last_clean":"2023-10-06T03:48:22.981729+0000","last_became_active":"2023-10-06T03:48:22.979810+0000","last_became_peered":"2023-10-06T03:48:22.979810+0000","last_unstale":"2023-10-06T03:48:22.981729+0000","last_undegraded":"2023-10-06T03:48:22.981729+0000","last_fullsized":"2023-10-06T03:48:22.981729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:20.245038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.955562+0000","last_change":"2023-10-06T03:48:21.961740+0000","last_active":"2023-10-06T03:48:22.955562+0000","last_peered":"2023-10-06T03:48:22.955562+0000","last_clean":"2023-10-06T03:48:22.955562+0000","last_became_active":"2023-10-06T03:48:21.961381+0000","last_became_peered":"2023-10-06T03:48:21.961381+0000","last_unstale":"2023-10-06T03:48:22.955562+0000","last_undegraded":"2023-10-06T03:48:22.955562+0000","last_fullsized":"2023-10-06T03:48:22.955562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:03.746648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971551+0000","last_change":"2023-10-06T03:48:22.962578+0000","last_active":"2023-10-06T03:48:22.971551+0000","last_peered":"2023-10-06T03:48:22.971551+0000","last_clean":"2023-10-06T03:48:22.971551+0000","last_became_active":"2023-10-06T03:48:22.962320+0000","last_became_peered":"2023-10-06T03:48:22.962320+0000","last_unstale":"2023-10-06T03:48:22.971551+0000","last_undegraded":"2023-10-06T03:48:22.971551+0000","last_fullsized":"2023-10-06T03:48:22.971551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:06.661073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.992556+0000","last_change":"2023-10-06T03:48:21.961642+0000","last_active":"2023-10-06T03:48:22.992556+0000","last_peered":"2023-10-06T03:48:22.992556+0000","last_clean":"2023-10-06T03:48:22.992556+0000","last_became_active":"2023-10-06T03:48:21.961209+0000","last_became_peered":"2023-10-06T03:48:21.961209+0000","last_unstale":"2023-10-06T03:48:22.992556+0000","last_undegraded":"2023-10-06T03:48:22.992556+0000","last_fullsized":"2023-10-06T03:48:22.992556+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:22.308236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.954065+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:48:22.954065+0000","last_peered":"2023-10-06T03:48:22.954065+0000","last_clean":"2023-10-06T03:48:22.954065+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:48:22.954065+0000","last_undegraded":"2023-10-06T03:48:22.954065+0000","last_fullsized":"2023-10-06T03:48:22.954065+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971664+0000","last_change":"2023-10-06T03:48:22.970569+0000","last_active":"2023-10-06T03:48:22.971664+0000","last_peered":"2023-10-06T03:48:22.971664+0000","last_clean":"2023-10-06T03:48:22.971664+0000","last_became_active":"2023-10-06T03:48:22.968580+0000","last_became_peered":"2023-10-06T03:48:22.968580+0000","last_unstale":"2023-10-06T03:48:22.971664+0000","last_undegraded":"2023-10-06T03:48:22.971664+0000","last_fullsized":"2023-10-06T03:48:22.971664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:04.756299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.960687+0000","last_change":"2023-10-06T03:48:21.958186+0000","last_active":"2023-10-06T03:48:22.960687+0000","last_peered":"2023-10-06T03:48:22.960687+0000","last_clean":"2023-10-06T03:48:22.960687+0000","last_became_active":"2023-10-06T03:48:21.957831+0000","last_became_peered":"2023-10-06T03:48:21.957831+0000","last_unstale":"2023-10-06T03:48:22.960687+0000","last_undegraded":"2023-10-06T03:48:22.960687+0000","last_fullsized":"2023-10-06T03:48:22.960687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:11.915578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.960769+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:48:22.960769+0000","last_peered":"2023-10-06T03:48:22.960769+0000","last_clean":"2023-10-06T03:48:22.960769+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:48:22.960769+0000","last_undegraded":"2023-10-06T03:48:22.960769+0000","last_fullsized":"2023-10-06T03:48:22.960769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980136+0000","last_change":"2023-10-06T03:48:22.968062+0000","last_active":"2023-10-06T03:48:22.980136+0000","last_peered":"2023-10-06T03:48:22.980136+0000","last_clean":"2023-10-06T03:48:22.980136+0000","last_became_active":"2023-10-06T03:48:22.967779+0000","last_became_peered":"2023-10-06T03:48:22.967779+0000","last_unstale":"2023-10-06T03:48:22.980136+0000","last_undegraded":"2023-10-06T03:48:22.980136+0000","last_fullsized":"2023-10-06T03:48:22.980136+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:14.771414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.984625+0000","last_change":"2023-10-06T03:48:21.954894+0000","last_active":"2023-10-06T03:48:22.984625+0000","last_peered":"2023-10-06T03:48:22.984625+0000","last_clean":"2023-10-06T03:48:22.984625+0000","last_became_active":"2023-10-06T03:48:21.954613+0000","last_became_peered":"2023-10-06T03:48:21.954613+0000","last_unstale":"2023-10-06T03:48:22.984625+0000","last_undegraded":"2023-10-06T03:48:22.984625+0000","last_fullsized":"2023-10-06T03:48:22.984625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:43.263401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":66,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.955914+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:48:22.955914+0000","last_peered":"2023-10-06T03:48:22.955914+0000","last_clean":"2023-10-06T03:48:22.955914+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:48:22.955914+0000","last_undegraded":"2023-10-06T03:48:22.955914+0000","last_fullsized":"2023-10-06T03:48:22.955914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"70'161","reported_seq":372,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.963654+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:48:22.963654+0000","last_peered":"2023-10-06T03:48:22.963654+0000","last_clean":"2023-10-06T03:48:22.963654+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:48:22.963654+0000","last_undegraded":"2023-10-06T03:48:22.963654+0000","last_fullsized":"2023-10-06T03:48:22.963654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971487+0000","last_change":"2023-10-06T03:48:22.967727+0000","last_active":"2023-10-06T03:48:22.971487+0000","last_peered":"2023-10-06T03:48:22.971487+0000","last_clean":"2023-10-06T03:48:22.971487+0000","last_became_active":"2023-10-06T03:48:22.967352+0000","last_became_peered":"2023-10-06T03:48:22.967352+0000","last_unstale":"2023-10-06T03:48:22.971487+0000","last_undegraded":"2023-10-06T03:48:22.971487+0000","last_fullsized":"2023-10-06T03:48:22.971487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:20.599886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958689+0000","last_change":"2023-10-06T03:48:21.954564+0000","last_active":"2023-10-06T03:48:22.958689+0000","last_peered":"2023-10-06T03:48:22.958689+0000","last_clean":"2023-10-06T03:48:22.958689+0000","last_became_active":"2023-10-06T03:48:21.954153+0000","last_became_peered":"2023-10-06T03:48:21.954153+0000","last_unstale":"2023-10-06T03:48:22.958689+0000","last_undegraded":"2023-10-06T03:48:22.958689+0000","last_fullsized":"2023-10-06T03:48:22.958689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:50.343611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":67,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.967814+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:48:22.967814+0000","last_peered":"2023-10-06T03:48:22.967814+0000","last_clean":"2023-10-06T03:48:22.967814+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:48:22.967814+0000","last_undegraded":"2023-10-06T03:48:22.967814+0000","last_fullsized":"2023-10-06T03:48:22.967814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970547+0000","last_change":"2023-10-06T03:48:22.967121+0000","last_active":"2023-10-06T03:48:22.970547+0000","last_peered":"2023-10-06T03:48:22.970547+0000","last_clean":"2023-10-06T03:48:22.970547+0000","last_became_active":"2023-10-06T03:48:22.966857+0000","last_became_peered":"2023-10-06T03:48:22.966857+0000","last_unstale":"2023-10-06T03:48:22.970547+0000","last_undegraded":"2023-10-06T03:48:22.970547+0000","last_fullsized":"2023-10-06T03:48:22.970547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:07.086261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.961262+0000","last_change":"2023-10-06T03:48:21.946145+0000","last_active":"2023-10-06T03:48:22.961262+0000","last_peered":"2023-10-06T03:48:22.961262+0000","last_clean":"2023-10-06T03:48:22.961262+0000","last_became_active":"2023-10-06T03:48:21.945883+0000","last_became_peered":"2023-10-06T03:48:21.945883+0000","last_unstale":"2023-10-06T03:48:22.961262+0000","last_undegraded":"2023-10-06T03:48:22.961262+0000","last_fullsized":"2023-10-06T03:48:22.961262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:29.398401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.963748+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:48:22.963748+0000","last_peered":"2023-10-06T03:48:22.963748+0000","last_clean":"2023-10-06T03:48:22.963748+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:48:22.963748+0000","last_undegraded":"2023-10-06T03:48:22.963748+0000","last_fullsized":"2023-10-06T03:48:22.963748+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.974827+0000","last_change":"2023-10-06T03:48:22.965384+0000","last_active":"2023-10-06T03:48:22.974827+0000","last_peered":"2023-10-06T03:48:22.974827+0000","last_clean":"2023-10-06T03:48:22.974827+0000","last_became_active":"2023-10-06T03:48:22.964719+0000","last_became_peered":"2023-10-06T03:48:22.964719+0000","last_unstale":"2023-10-06T03:48:22.974827+0000","last_undegraded":"2023-10-06T03:48:22.974827+0000","last_fullsized":"2023-10-06T03:48:22.974827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:09.525282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958144+0000","last_change":"2023-10-06T03:48:21.958097+0000","last_active":"2023-10-06T03:48:22.958144+0000","last_peered":"2023-10-06T03:48:22.958144+0000","last_clean":"2023-10-06T03:48:22.958144+0000","last_became_active":"2023-10-06T03:48:21.957822+0000","last_became_peered":"2023-10-06T03:48:21.957822+0000","last_unstale":"2023-10-06T03:48:22.958144+0000","last_undegraded":"2023-10-06T03:48:22.958144+0000","last_fullsized":"2023-10-06T03:48:22.958144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:59.062727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958612+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:48:22.958612+0000","last_peered":"2023-10-06T03:48:22.958612+0000","last_clean":"2023-10-06T03:48:22.958612+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:48:22.958612+0000","last_undegraded":"2023-10-06T03:48:22.958612+0000","last_fullsized":"2023-10-06T03:48:22.958612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972727+0000","last_change":"2023-10-06T03:48:22.961087+0000","last_active":"2023-10-06T03:48:22.972727+0000","last_peered":"2023-10-06T03:48:22.972727+0000","last_clean":"2023-10-06T03:48:22.972727+0000","last_became_active":"2023-10-06T03:48:22.960836+0000","last_became_peered":"2023-10-06T03:48:22.960836+0000","last_unstale":"2023-10-06T03:48:22.972727+0000","last_undegraded":"2023-10-06T03:48:22.972727+0000","last_fullsized":"2023-10-06T03:48:22.972727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:52.603676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.990082+0000","last_change":"2023-10-06T03:48:21.957833+0000","last_active":"2023-10-06T03:48:22.990082+0000","last_peered":"2023-10-06T03:48:22.990082+0000","last_clean":"2023-10-06T03:48:22.990082+0000","last_became_active":"2023-10-06T03:48:21.957260+0000","last_became_peered":"2023-10-06T03:48:21.957260+0000","last_unstale":"2023-10-06T03:48:22.990082+0000","last_undegraded":"2023-10-06T03:48:22.990082+0000","last_fullsized":"2023-10-06T03:48:22.990082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:31.139755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.952549+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:48:22.952549+0000","last_peered":"2023-10-06T03:48:22.952549+0000","last_clean":"2023-10-06T03:48:22.952549+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:48:22.952549+0000","last_undegraded":"2023-10-06T03:48:22.952549+0000","last_fullsized":"2023-10-06T03:48:22.952549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981603+0000","last_change":"2023-10-06T03:48:22.980129+0000","last_active":"2023-10-06T03:48:22.981603+0000","last_peered":"2023-10-06T03:48:22.981603+0000","last_clean":"2023-10-06T03:48:22.981603+0000","last_became_active":"2023-10-06T03:48:22.979872+0000","last_became_peered":"2023-10-06T03:48:22.979872+0000","last_unstale":"2023-10-06T03:48:22.981603+0000","last_undegraded":"2023-10-06T03:48:22.981603+0000","last_fullsized":"2023-10-06T03:48:22.981603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:13.209361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.989740+0000","last_change":"2023-10-06T03:48:21.955182+0000","last_active":"2023-10-06T03:48:22.989740+0000","last_peered":"2023-10-06T03:48:22.989740+0000","last_clean":"2023-10-06T03:48:22.989740+0000","last_became_active":"2023-10-06T03:48:21.954909+0000","last_became_peered":"2023-10-06T03:48:21.954909+0000","last_unstale":"2023-10-06T03:48:22.989740+0000","last_undegraded":"2023-10-06T03:48:22.989740+0000","last_fullsized":"2023-10-06T03:48:22.989740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:25.171449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.952991+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:48:22.952991+0000","last_peered":"2023-10-06T03:48:22.952991+0000","last_clean":"2023-10-06T03:48:22.952991+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:48:22.952991+0000","last_undegraded":"2023-10-06T03:48:22.952991+0000","last_fullsized":"2023-10-06T03:48:22.952991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575031,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311112,"kb_used_data":1536,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060728,"statfs":{"total":96636764160,"available":96318185472,"internally_reserved":0,"allocated":1572864,"data_stored":1076126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71199999999999997,"15min":0.69299999999999995},"min":{"1min":0.53000000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.998,"5min":1.0029999999999999,"15min":1.6100000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.51500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.5649999999999999},"last":0.68899999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.755,"15min":0.749},"min":{"1min":0.63800000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.038,"5min":1.038,"15min":1.5469999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.59199999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.56899999999999995,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":1.5900000000000001},"last":0.437},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.56999999999999995,"5min":0.495,"15min":0.495},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.66500000000000004}]},{"osd":3,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73199999999999998,"15min":0.72499999999999998},"min":{"1min":0.61299999999999999,"5min":0.45900000000000002,"15min":0.39500000000000002},"max":{"1min":1.0089999999999999,"5min":1.016,"15min":1.6279999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76000000000000001,"15min":0.74399999999999999},"min":{"1min":0.624,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96099999999999997,"5min":0.98099999999999998,"15min":1.0409999999999999},"last":0.74299999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79600000000000004,"5min":0.96499999999999997,"15min":1.0049999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.628},"min":{"1min":0.46400000000000002,"5min":0.40999999999999998,"15min":0.39400000000000002},"max":{"1min":0.874,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.621}]},{"osd":5,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63900000000000001,"15min":0.64800000000000002},"min":{"1min":0.49299999999999999,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":0.95599999999999996,"5min":1.018,"15min":1.069},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.46800000000000003,"15min":0.436},"max":{"1min":0.91300000000000003,"5min":1.105,"15min":1.105},"last":0.72799999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72399999999999998,"15min":0.72899999999999998},"min":{"1min":0.58199999999999996,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":1.181,"15min":1.181},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.55200000000000005,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.91500000000000004,"5min":1.02,"15min":1.0209999999999999},"last":0.76700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575032,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306428,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065412,"statfs":{"total":96636764160,"available":96322981888,"internally_reserved":0,"allocated":970752,"data_stored":486022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.42199999999999999,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.88,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.39500000000000002},"max":{"1min":0.94799999999999995,"5min":0.98599999999999999,"15min":1.0449999999999999},"last":0.89700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.47499999999999998,"5min":0.36499999999999999,"15min":0.33800000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.3180000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.35899999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":1.073}]},{"osd":2,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.751,"15min":0.751},"min":{"1min":0.48199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.98699999999999999,"5min":0.995,"15min":3.3319999999999999},"last":0.627},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":3.5139999999999998},"last":0.95999999999999996}]},{"osd":3,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.58499999999999996,"5min":0.48299999999999998,"15min":0.44600000000000001},"max":{"1min":0.97399999999999998,"5min":1.01,"15min":1.024},"last":0.88},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76100000000000001,"15min":0.77400000000000002},"min":{"1min":0.49299999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.98799999999999999,"5min":0.99099999999999999,"15min":3.3300000000000001},"last":0.85599999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.45200000000000001,"5min":0.376,"15min":0.34799999999999998},"max":{"1min":0.91900000000000004,"5min":0.92600000000000005,"15min":1.0680000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36499999999999999},"max":{"1min":0.90700000000000003,"5min":1.0169999999999999,"15min":1.2929999999999999},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67000000000000004,"15min":0.66700000000000004},"min":{"1min":0.438,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.3340000000000001},"last":1.006},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.42899999999999999,"15min":0.41199999999999998},"max":{"1min":1.0049999999999999,"5min":1.0269999999999999,"15min":1.2370000000000001},"last":0.91400000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68600000000000005,"15min":0.69899999999999995},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.619},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.47099999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.417},"last":1.0389999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575031,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307032,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064808,"statfs":{"total":96636764160,"available":96322363392,"internally_reserved":0,"allocated":1589248,"data_stored":1078628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.47999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80200000000000005,"5min":1.1319999999999999,"15min":1.2889999999999999},"last":0.432}]},{"osd":2,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.5,"15min":0.495},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.26100000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.44,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.94099999999999995},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.60199999999999998},"min":{"1min":0.50800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86699999999999999,"15min":0.93999999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.434,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77400000000000002,"5min":0.90400000000000003,"15min":1.254},"last":0.38600000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.52500000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0009999999999999,"5min":1.034,"15min":1.3600000000000001},"last":0.626},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.80900000000000005,"15min":0.79800000000000004},"min":{"1min":0.68400000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.476},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.80200000000000005,"15min":0.79300000000000004},"min":{"1min":0.621,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.601},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.59299999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.5189999999999999},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.80500000000000005,"15min":0.81000000000000005},"min":{"1min":0.64200000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82499999999999996,"15min":0.81699999999999995},"min":{"1min":0.66700000000000004,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":1.143,"5min":1.1639999999999999,"15min":1.399},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.84799999999999998,"15min":0.83399999999999996},"min":{"1min":0.66900000000000004,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1000000000000001,"5min":1.272,"15min":1.272},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.83399999999999996,"15min":0.82899999999999996},"min":{"1min":0.64400000000000002,"5min":0.53700000000000003,"15min":0.51200000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.556},"last":0.65600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575033,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306420,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065420,"statfs":{"total":96636764160,"available":96322990080,"internally_reserved":0,"allocated":962560,"data_stored":485010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48199999999999998,"15min":0.495},"min":{"1min":0.373,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":0.67800000000000005,"5min":0.81000000000000005,"15min":2.3220000000000001},"last":0.373},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.52700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33600000000000002,"15min":0.29599999999999999},"max":{"1min":0.70299999999999996,"5min":0.78500000000000003,"15min":2.1440000000000001},"last":0.501}]},{"osd":2,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.56499999999999995},"min":{"1min":0.40500000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.76400000000000001,"5min":0.77700000000000002,"15min":2.4329999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54800000000000004,"15min":0.56299999999999994},"min":{"1min":0.40799999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.755,"5min":0.84699999999999998,"15min":2.1819999999999999},"last":0.41499999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51100000000000001,"15min":0.53000000000000003},"min":{"1min":0.41399999999999998,"5min":0.30499999999999999,"15min":0.28399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":2.2709999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61299999999999999,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81000000000000005,"5min":1.0580000000000001,"15min":2.4700000000000002},"last":0.435}]},{"osd":4,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.83099999999999996,"15min":0.83499999999999996},"min":{"1min":0.69199999999999995,"5min":0.54200000000000004,"15min":0.49399999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.393},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.80900000000000005,"15min":0.83199999999999996},"min":{"1min":0.63400000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.306,"5min":1.306,"15min":2.2320000000000002},"last":0.79200000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.80600000000000005,"15min":0.82999999999999996},"min":{"1min":0.57799999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":2.3660000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.58999999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0780000000000001,"5min":1.0800000000000001,"15min":1.6559999999999999},"last":0.77300000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77900000000000003,"15min":0.80500000000000005},"min":{"1min":0.56999999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.726},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.82399999999999995,"15min":0.85699999999999998},"min":{"1min":0.628,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1100000000000001,"5min":1.119,"15min":2.4009999999999998},"last":0.80100000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.64300000000000002,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2689999999999999},"last":0.873},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.79800000000000004,"15min":0.83699999999999997},"min":{"1min":0.625,"5min":0.495,"15min":0.495},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.7649999999999999},"last":0.76000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575032,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306420,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065420,"statfs":{"total":96636764160,"available":96322990080,"internally_reserved":0,"allocated":962560,"data_stored":485010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48899999999999999,"15min":0.46700000000000003},"min":{"1min":0.35199999999999998,"5min":0.29699999999999999,"15min":0.28000000000000003},"max":{"1min":0.74199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53400000000000003,"15min":0.52300000000000002},"min":{"1min":0.438,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":0.86199999999999999,"15min":0.89300000000000002},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.34799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72899999999999998,"5min":0.79900000000000004,"15min":0.90000000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.372,"15min":0.34200000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.556},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.504,"5min":0.41399999999999998,"15min":0.38800000000000001},"max":{"1min":0.72899999999999998,"5min":1.78,"15min":1.78},"last":0.439},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81799999999999995,"5min":0.91400000000000003,"15min":1.1910000000000001},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.81899999999999995,"15min":0.81799999999999995},"min":{"1min":0.67600000000000005,"5min":0.51400000000000001,"15min":0.439},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.5620000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.72199999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.008,"5min":1.1419999999999999,"15min":1.2270000000000001},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.82699999999999996,"15min":0.82999999999999996},"min":{"1min":0.68100000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.1140000000000001,"5min":1.177,"15min":1.3},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84399999999999997,"15min":0.84999999999999998},"min":{"1min":0.65500000000000003,"5min":0.55500000000000005,"15min":0.51700000000000002},"max":{"1min":1.0549999999999999,"5min":1.855,"15min":1.855},"last":0.57699999999999996}]},{"osd":6,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82999999999999996,"15min":0.83699999999999997},"min":{"1min":0.68400000000000005,"5min":0.54200000000000004,"15min":0.53600000000000003},"max":{"1min":0.97899999999999998,"5min":1.907,"15min":1.907},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.81899999999999995,"15min":0.81499999999999995},"min":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.51100000000000001},"max":{"1min":1.1080000000000001,"5min":1.1120000000000001,"15min":1.4750000000000001},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.84099999999999997,"15min":0.83499999999999996},"min":{"1min":0.73999999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.125,"5min":1.4379999999999999,"15min":1.4990000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.86399999999999999},"min":{"1min":0.68400000000000005,"5min":0.53100000000000003,"15min":0.497},"max":{"1min":0.99399999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.69599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575032,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306424,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065416,"statfs":{"total":96636764160,"available":96322985984,"internally_reserved":0,"allocated":966656,"data_stored":485988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.45800000000000002,"15min":0.46200000000000002},"min":{"1min":0.33000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56899999999999995,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.49399999999999999,"15min":0.495},"min":{"1min":0.39700000000000002,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":0.63800000000000001,"5min":1.04,"15min":1.1699999999999999},"last":0.376}]},{"osd":1,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47799999999999998,"15min":0.47299999999999998},"min":{"1min":0.374,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.78800000000000003,"15min":0.79500000000000004},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.33800000000000002},"max":{"1min":0.70599999999999996,"5min":0.76700000000000002,"15min":0.83899999999999997},"last":0.35299999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.34599999999999997},"max":{"1min":0.79700000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.38700000000000001},"max":{"1min":0.76800000000000002,"5min":0.81999999999999995,"15min":0.97899999999999998},"last":0.435}]},{"osd":4,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.78000000000000003,"15min":0.78900000000000003},"min":{"1min":0.51600000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.0049999999999999,"5min":1.2030000000000001,"15min":1.4099999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.81799999999999995,"15min":0.82799999999999996},"min":{"1min":0.64100000000000001,"5min":0.57399999999999995,"15min":0.51300000000000001},"max":{"1min":1.0469999999999999,"5min":1.1850000000000001,"15min":1.377},"last":0.79200000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77400000000000002,"15min":0.78800000000000003},"min":{"1min":0.56699999999999995,"5min":0.51900000000000002,"15min":0.47099999999999997},"max":{"1min":1,"5min":1.256,"15min":1.256},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.77100000000000002,"15min":0.78900000000000003},"min":{"1min":0.61499999999999999,"5min":0.55600000000000005,"15min":0.48099999999999998},"max":{"1min":0.99199999999999999,"5min":1.1439999999999999,"15min":1.306},"last":0.629}]},{"osd":6,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81000000000000005,"15min":0.81399999999999995},"min":{"1min":0.61699999999999999,"5min":0.52000000000000002,"15min":0.48599999999999999},"max":{"1min":1.02,"5min":1.292,"15min":1.323},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.82799999999999996,"15min":0.84999999999999998},"min":{"1min":0.60599999999999998,"5min":0.58599999999999997,"15min":0.49299999999999999},"max":{"1min":1.075,"5min":1.1040000000000001,"15min":1.6020000000000001},"last":0.73899999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:47:28 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.65500000000000003,"5min":0.57799999999999996,"15min":0.505},"max":{"1min":0.98999999999999999,"5min":1.2370000000000001,"15min":1.365},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80000000000000004,"15min":0.80800000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.46300000000000002},"max":{"1min":1.038,"5min":1.1180000000000001,"15min":1.2330000000000001},"last":0.67700000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575032,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302340,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069500,"statfs":{"total":96636764160,"available":96327168000,"internally_reserved":0,"allocated":978944,"data_stored":487524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.70599999999999996,"15min":0.72099999999999997},"min":{"1min":0.56899999999999995,"5min":0.316,"15min":0.316},"max":{"1min":0.96999999999999997,"5min":1.0109999999999999,"15min":1.3120000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.54800000000000004,"5min":0.316,"15min":0.316},"max":{"1min":0.91500000000000004,"5min":0.95299999999999996,"15min":1.01},"last":0.67300000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.47299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85899999999999999,"5min":1.081,"15min":1.081},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71799999999999997,"15min":0.73699999999999999},"min":{"1min":0.64600000000000002,"5min":0.44,"15min":0.41299999999999998},"max":{"1min":0.89600000000000002,"5min":0.96999999999999997,"15min":1.3380000000000001},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.61399999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.91500000000000004,"5min":1.028,"15min":1.1759999999999999},"last":0.755},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.747,"15min":0.746},"min":{"1min":0.629,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0109999999999999,"5min":1.022,"15min":1.3089999999999999},"last":0.76600000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.63100000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89700000000000002,"5min":1.0049999999999999,"15min":1.0409999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75800000000000001,"15min":0.76900000000000002},"min":{"1min":0.53200000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.80700000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.621,"15min":0.63},"min":{"1min":0.496,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":1.01,"15min":1.085},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.622,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.35099999999999998},"max":{"1min":0.72799999999999998,"5min":0.95399999999999996,"15min":1.1439999999999999},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.38100000000000001},"max":{"1min":0.90200000000000002,"5min":1.052,"15min":1.361},"last":0.502}]},{"osd":7,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.39900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":1.1950000000000001},"last":0.628},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.52000000000000002,"5min":0.504,"15min":0.41299999999999998},"max":{"1min":0.90100000000000002,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.65000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575032,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073608,"statfs":{"total":96636764160,"available":96331374592,"internally_reserved":0,"allocated":966656,"data_stored":485032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.42299999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94999999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70899999999999996,"15min":0.70699999999999996},"min":{"1min":0.60599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93500000000000005,"5min":0.96899999999999997,"15min":1},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.70599999999999996,"15min":0.70799999999999996},"min":{"1min":0.46600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.58599999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.60299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76400000000000001,"15min":0.77600000000000002},"min":{"1min":0.67900000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98699999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.67000000000000004}]},{"osd":3,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72399999999999998,"15min":0.73099999999999998},"min":{"1min":0.56599999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.996,"5min":0.996,"15min":1.6419999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.754,"15min":0.749},"min":{"1min":0.623,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0189999999999999,"5min":1.117,"15min":1.117},"last":0.625}]},{"osd":4,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.62,"15min":0.621},"min":{"1min":0.42899999999999999,"5min":0.40000000000000002,"15min":0.38300000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.024},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61799999999999999,"15min":0.63800000000000001},"min":{"1min":0.47099999999999997,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":0.94199999999999995,"5min":1.044,"15min":1.175},"last":0.745}]},{"osd":6,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.34999999999999998},"max":{"1min":1.0449999999999999,"5min":1.591,"15min":1.591},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69599999999999995,"15min":0.69899999999999995},"min":{"1min":0.47799999999999998,"5min":0.44600000000000001,"15min":0.40600000000000003},"max":{"1min":1.0389999999999999,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67000000000000004,"15min":0.68100000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":0.97199999999999998,"5min":1.016,"15min":1.2589999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.72399999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.435},"max":{"1min":1.0880000000000001,"5min":1.278,"15min":1.278},"last":0.78100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:48:32.282 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:48:32.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.483+0000 7fef9ffff700 1 Processor -- start 2023-10-06T03:48:32.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.483+0000 7fef9ffff700 1 -- start start 2023-10-06T03:48:32.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.484+0000 7fef9ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fefa0157770 0x7fefa0150180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:32.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.485+0000 7fef9ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fefa00c6d00 0x7fefa00c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:32.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.485+0000 7fef9ffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa006e2a0 con 0x7fefa006e730 2023-10-06T03:48:32.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.485+0000 7fef9ffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa0157b40 con 0x7fefa00c6d00 2023-10-06T03:48:32.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.485+0000 7fef9ffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa00c7690 con 0x7fefa0157770 2023-10-06T03:48:32.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.485+0000 7fef9e7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fefa00c6d00 0x7fefa00c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:32.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.485+0000 7fef9e7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fefa00c6d00 0x7fefa00c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48850/0 (socket says 172.21.15.55:48850) 2023-10-06T03:48:32.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.485+0000 7fef9e7fc700 1 -- 172.21.15.55:0/1287130595 learned_addr learned my addr 172.21.15.55:0/1287130595 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:32.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.485+0000 7fef9effd700 1 --2- 172.21.15.55:0/1287130595 >> v2:172.21.15.163:3300/0 conn(0x7fefa0157770 0x7fefa0150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:32.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.485+0000 7fef9e7fc700 1 -- 172.21.15.55:0/1287130595 >> v2:172.21.15.163:3300/0 conn(0x7fefa0157770 msgr2=0x7fefa0150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:32.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.485+0000 7fef9e7fc700 1 --2- 172.21.15.55:0/1287130595 >> v2:172.21.15.163:3300/0 conn(0x7fefa0157770 0x7fefa0150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:32.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.485+0000 7fef9e7fc700 1 -- 172.21.15.55:0/1287130595 >> v1:172.21.15.163:6789/0 conn(0x7fefa006e730 legacy=0x7fefa006eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:32.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.485+0000 7fef9e7fc700 1 -- 172.21.15.55:0/1287130595 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa006b2e0 con 0x7fefa00c6d00 2023-10-06T03:48:32.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.486+0000 7fef9e7fc700 1 --2- 172.21.15.55:0/1287130595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefa00c6d00 0x7fefa00c7150 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fefa006fd20 tx=0x7fef94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fbaa3cf0de28c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:32.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.486+0000 7fef9dffb700 1 -- 172.21.15.55:0/1287130595 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9400b860 con 0x7fefa00c6d00 2023-10-06T03:48:32.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.486+0000 7fef9dffb700 1 -- 172.21.15.55:0/1287130595 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9400b9c0 con 0x7fefa00c6d00 2023-10-06T03:48:32.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.486+0000 7fef9dffb700 1 -- 172.21.15.55:0/1287130595 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9400bd60 con 0x7fefa00c6d00 2023-10-06T03:48:32.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.486+0000 7fef9ffff700 1 -- 172.21.15.55:0/1287130595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefa00c6d00 msgr2=0x7fefa00c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:32.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.487+0000 7fef9ffff700 1 --2- 172.21.15.55:0/1287130595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefa00c6d00 0x7fefa00c7150 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fefa006fd20 tx=0x7fef94009250 comp rx=0 tx=0).stop 2023-10-06T03:48:32.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.487+0000 7fef9ffff700 1 -- 172.21.15.55:0/1287130595 shutdown_connections 2023-10-06T03:48:32.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.487+0000 7fef9ffff700 1 --2- 172.21.15.55:0/1287130595 >> v2:172.21.15.163:3300/0 conn(0x7fefa0157770 0x7fefa0150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:32.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.487+0000 7fef9ffff700 1 --2- 172.21.15.55:0/1287130595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefa00c6d00 0x7fefa00c7150 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:32.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.487+0000 7fef9ffff700 1 -- 172.21.15.55:0/1287130595 >> 172.21.15.55:0/1287130595 conn(0x7fefa00cc1f0 msgr2=0x7fefa00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:32.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.487+0000 7fef9ffff700 1 -- 172.21.15.55:0/1287130595 shutdown_connections 2023-10-06T03:48:32.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.487+0000 7fef9ffff700 1 -- 172.21.15.55:0/1287130595 wait complete. 2023-10-06T03:48:32.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.488+0000 7fef9ffff700 1 Processor -- start 2023-10-06T03:48:32.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.488+0000 7fef9ffff700 1 -- start start 2023-10-06T03:48:32.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.489+0000 7fef9ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefa006e730 0x7fefa014eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:32.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.489+0000 7fef9ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefa00c6d00 0x7fefa00c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:32.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.489+0000 7fef9ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefa0157770 0x7fefa00b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:32.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.489+0000 7fef9ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00c3e70 con 0x7fefa0157770 2023-10-06T03:48:32.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.489+0000 7fef9ffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00c32a0 con 0x7fefa006e730 2023-10-06T03:48:32.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.489+0000 7fef9ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa00c3540 con 0x7fefa00c6d00 2023-10-06T03:48:32.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.489+0000 7fef9f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefa0157770 0x7fefa00b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:32.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.489+0000 7fef9f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefa0157770 0x7fefa00b10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48864/0 (socket says 172.21.15.55:48864) 2023-10-06T03:48:32.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.489+0000 7fef9f7fe700 1 -- 172.21.15.55:0/1450075148 learned_addr learned my addr 172.21.15.55:0/1450075148 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:32.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.490+0000 7fef9effd700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefa006e730 0x7fefa014eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:32.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.490+0000 7fef9e7fc700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefa00c6d00 0x7fefa00c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:32.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.490+0000 7fef9f7fe700 1 -- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefa00c6d00 msgr2=0x7fefa00c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:32.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.490+0000 7fef9f7fe700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefa00c6d00 0x7fefa00c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:32.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.490+0000 7fef9f7fe700 1 -- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefa006e730 msgr2=0x7fefa014eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:32.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.490+0000 7fef9f7fe700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefa006e730 0x7fefa014eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:32.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.490+0000 7fef9f7fe700 1 -- 172.21.15.55:0/1450075148 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940094f0 con 0x7fefa0157770 2023-10-06T03:48:32.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.490+0000 7fef9e7fc700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefa00c6d00 0x7fefa00c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:32.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.490+0000 7fef9f7fe700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefa0157770 0x7fefa00b10c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fef9800c840 tx=0x7fef98001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:32.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.490+0000 7fef86ffd700 1 -- 172.21.15.55:0/1450075148 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef98012630 con 0x7fefa0157770 2023-10-06T03:48:32.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.490+0000 7fef86ffd700 1 -- 172.21.15.55:0/1450075148 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef980127b0 con 0x7fefa0157770 2023-10-06T03:48:32.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.490+0000 7fef9ffff700 1 -- 172.21.15.55:0/1450075148 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa00c3820 con 0x7fefa0157770 2023-10-06T03:48:32.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.490+0000 7fef86ffd700 1 -- 172.21.15.55:0/1450075148 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef98012b70 con 0x7fefa0157770 2023-10-06T03:48:32.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.491+0000 7fef9ffff700 1 -- 172.21.15.55:0/1450075148 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa014f050 con 0x7fefa0157770 2023-10-06T03:48:32.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.492+0000 7fef9ffff700 1 -- 172.21.15.55:0/1450075148 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa004c3c0 con 0x7fefa0157770 2023-10-06T03:48:32.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.493+0000 7fef86ffd700 1 -- 172.21.15.55:0/1450075148 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef98003030 con 0x7fefa0157770 2023-10-06T03:48:32.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.494+0000 7fef86ffd700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fef8c066f30 0x7fef8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:32.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.494+0000 7fef86ffd700 1 -- 172.21.15.55:0/1450075148 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fef98084820 con 0x7fefa0157770 2023-10-06T03:48:32.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.495+0000 7fef9effd700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fef8c066f30 0x7fef8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:32.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.495+0000 7fef9effd700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fef8c066f30 0x7fef8c0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fef90006fd0 tx=0x7fef90008040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:32.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.498+0000 7fef86ffd700 1 -- 172.21.15.55:0/1450075148 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef98051700 con 0x7fefa0157770 2023-10-06T03:48:32.698 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.696+0000 7fef9ffff700 1 -- 172.21.15.55:0/1450075148 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fefa00c3fb0 con 0x7fef8c066f30 2023-10-06T03:48:32.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.706+0000 7fef86ffd700 1 -- 172.21.15.55:0/1450075148 <== mgr.5483 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+310875 (secure 0 0 0) 0x7fefa00c3fb0 con 0x7fef8c066f30 2023-10-06T03:48:32.712 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:32.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.710+0000 7fef9ffff700 1 -- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fef8c066f30 msgr2=0x7fef8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:32.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.710+0000 7fef9ffff700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fef8c066f30 0x7fef8c0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fef90006fd0 tx=0x7fef90008040 comp rx=0 tx=0).stop 2023-10-06T03:48:32.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.710+0000 7fef9ffff700 1 -- 172.21.15.55:0/1450075148 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefa0157770 msgr2=0x7fefa00b10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:32.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.710+0000 7fef9ffff700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefa0157770 0x7fefa00b10c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fef9800c840 tx=0x7fef98001670 comp rx=0 tx=0).stop 2023-10-06T03:48:32.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.711+0000 7fef9ffff700 1 -- 172.21.15.55:0/1450075148 shutdown_connections 2023-10-06T03:48:32.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.711+0000 7fef9ffff700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fef8c066f30 0x7fef8c0693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:32.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.711+0000 7fef9ffff700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefa0157770 0x7fefa00b10c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:32.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.711+0000 7fef9ffff700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefa006e730 0x7fefa014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:32.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.711+0000 7fef9ffff700 1 --2- 172.21.15.55:0/1450075148 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefa00c6d00 0x7fefa00c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:32.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.711+0000 7fef9ffff700 1 -- 172.21.15.55:0/1450075148 >> 172.21.15.55:0/1450075148 conn(0x7fefa00cc1f0 msgr2=0x7fefa00adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:32.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.711+0000 7fef9ffff700 1 -- 172.21.15.55:0/1450075148 shutdown_connections 2023-10-06T03:48:32.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.711+0000 7fef9ffff700 1 -- 172.21.15.55:0/1450075148 wait complete. 2023-10-06T03:48:32.716 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:48:32.730 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-06T03:48:32.018035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":284,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444408,"kb_used_data":8760,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752530312,"statfs":{"total":773094113280,"available":770591039488,"internally_reserved":0,"allocated":8970240,"data_stored":5069340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.068495"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981925+0000","last_change":"2023-10-06T03:48:22.978687+0000","last_active":"2023-10-06T03:48:22.981925+0000","last_peered":"2023-10-06T03:48:22.981925+0000","last_clean":"2023-10-06T03:48:22.981925+0000","last_became_active":"2023-10-06T03:48:22.977640+0000","last_became_peered":"2023-10-06T03:48:22.977640+0000","last_unstale":"2023-10-06T03:48:22.981925+0000","last_undegraded":"2023-10-06T03:48:22.981925+0000","last_fullsized":"2023-10-06T03:48:22.981925+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:06.518654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980965+0000","last_change":"2023-10-06T03:48:22.965813+0000","last_active":"2023-10-06T03:48:22.980965+0000","last_peered":"2023-10-06T03:48:22.980965+0000","last_clean":"2023-10-06T03:48:22.980965+0000","last_became_active":"2023-10-06T03:48:22.965532+0000","last_became_peered":"2023-10-06T03:48:22.965532+0000","last_unstale":"2023-10-06T03:48:22.980965+0000","last_undegraded":"2023-10-06T03:48:22.980965+0000","last_fullsized":"2023-10-06T03:48:22.980965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:36.717228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.984687+0000","last_change":"2023-10-06T03:48:22.983311+0000","last_active":"2023-10-06T03:48:22.984687+0000","last_peered":"2023-10-06T03:48:22.984687+0000","last_clean":"2023-10-06T03:48:22.984687+0000","last_became_active":"2023-10-06T03:48:22.983010+0000","last_became_peered":"2023-10-06T03:48:22.983010+0000","last_unstale":"2023-10-06T03:48:22.984687+0000","last_undegraded":"2023-10-06T03:48:22.984687+0000","last_fullsized":"2023-10-06T03:48:22.984687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:12.925229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971611+0000","last_change":"2023-10-06T03:48:22.964273+0000","last_active":"2023-10-06T03:48:22.971611+0000","last_peered":"2023-10-06T03:48:22.971611+0000","last_clean":"2023-10-06T03:48:22.971611+0000","last_became_active":"2023-10-06T03:48:22.963978+0000","last_became_peered":"2023-10-06T03:48:22.963978+0000","last_unstale":"2023-10-06T03:48:22.971611+0000","last_undegraded":"2023-10-06T03:48:22.971611+0000","last_fullsized":"2023-10-06T03:48:22.971611+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:13.237222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970479+0000","last_change":"2023-10-06T03:48:22.960460+0000","last_active":"2023-10-06T03:48:22.970479+0000","last_peered":"2023-10-06T03:48:22.970479+0000","last_clean":"2023-10-06T03:48:22.970479+0000","last_became_active":"2023-10-06T03:48:22.959985+0000","last_became_peered":"2023-10-06T03:48:22.959985+0000","last_unstale":"2023-10-06T03:48:22.970479+0000","last_undegraded":"2023-10-06T03:48:22.970479+0000","last_fullsized":"2023-10-06T03:48:22.970479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:39.409932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980760+0000","last_change":"2023-10-06T03:48:22.975932+0000","last_active":"2023-10-06T03:48:22.980760+0000","last_peered":"2023-10-06T03:48:22.980760+0000","last_clean":"2023-10-06T03:48:22.980760+0000","last_became_active":"2023-10-06T03:48:22.975667+0000","last_became_peered":"2023-10-06T03:48:22.975667+0000","last_unstale":"2023-10-06T03:48:22.980760+0000","last_undegraded":"2023-10-06T03:48:22.980760+0000","last_fullsized":"2023-10-06T03:48:22.980760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:49.669934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.985364+0000","last_change":"2023-10-06T03:48:22.980964+0000","last_active":"2023-10-06T03:48:22.985364+0000","last_peered":"2023-10-06T03:48:22.985364+0000","last_clean":"2023-10-06T03:48:22.985364+0000","last_became_active":"2023-10-06T03:48:22.980709+0000","last_became_peered":"2023-10-06T03:48:22.980709+0000","last_unstale":"2023-10-06T03:48:22.985364+0000","last_undegraded":"2023-10-06T03:48:22.985364+0000","last_fullsized":"2023-10-06T03:48:22.985364+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:40:53.633631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.976769+0000","last_change":"2023-10-06T03:48:22.973481+0000","last_active":"2023-10-06T03:48:22.976769+0000","last_peered":"2023-10-06T03:48:22.976769+0000","last_clean":"2023-10-06T03:48:22.976769+0000","last_became_active":"2023-10-06T03:48:22.973158+0000","last_became_peered":"2023-10-06T03:48:22.973158+0000","last_unstale":"2023-10-06T03:48:22.976769+0000","last_undegraded":"2023-10-06T03:48:22.976769+0000","last_fullsized":"2023-10-06T03:48:22.976769+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:24.959369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980277+0000","last_change":"2023-10-06T03:48:22.978760+0000","last_active":"2023-10-06T03:48:22.980277+0000","last_peered":"2023-10-06T03:48:22.980277+0000","last_clean":"2023-10-06T03:48:22.980277+0000","last_became_active":"2023-10-06T03:48:22.978459+0000","last_became_peered":"2023-10-06T03:48:22.978459+0000","last_unstale":"2023-10-06T03:48:22.980277+0000","last_undegraded":"2023-10-06T03:48:22.980277+0000","last_fullsized":"2023-10-06T03:48:22.980277+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:30.408052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.983962+0000","last_change":"2023-10-06T03:48:22.983073+0000","last_active":"2023-10-06T03:48:22.983962+0000","last_peered":"2023-10-06T03:48:22.983962+0000","last_clean":"2023-10-06T03:48:22.983962+0000","last_became_active":"2023-10-06T03:48:22.982662+0000","last_became_peered":"2023-10-06T03:48:22.982662+0000","last_unstale":"2023-10-06T03:48:22.983962+0000","last_undegraded":"2023-10-06T03:48:22.983962+0000","last_fullsized":"2023-10-06T03:48:22.983962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:13.237222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.973340+0000","last_change":"2023-10-06T03:48:22.969513+0000","last_active":"2023-10-06T03:48:22.973340+0000","last_peered":"2023-10-06T03:48:22.973340+0000","last_clean":"2023-10-06T03:48:22.973340+0000","last_became_active":"2023-10-06T03:48:22.969211+0000","last_became_peered":"2023-10-06T03:48:22.969211+0000","last_unstale":"2023-10-06T03:48:22.973340+0000","last_undegraded":"2023-10-06T03:48:22.973340+0000","last_fullsized":"2023-10-06T03:48:22.973340+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:58.681620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.975883+0000","last_change":"2023-10-06T03:48:22.973519+0000","last_active":"2023-10-06T03:48:22.975883+0000","last_peered":"2023-10-06T03:48:22.975883+0000","last_clean":"2023-10-06T03:48:22.975883+0000","last_became_active":"2023-10-06T03:48:22.973239+0000","last_became_peered":"2023-10-06T03:48:22.973239+0000","last_unstale":"2023-10-06T03:48:22.975883+0000","last_undegraded":"2023-10-06T03:48:22.975883+0000","last_fullsized":"2023-10-06T03:48:22.975883+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:04.560113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.976082+0000","last_change":"2023-10-06T03:48:22.970347+0000","last_active":"2023-10-06T03:48:22.976082+0000","last_peered":"2023-10-06T03:48:22.976082+0000","last_clean":"2023-10-06T03:48:22.976082+0000","last_became_active":"2023-10-06T03:48:22.970084+0000","last_became_peered":"2023-10-06T03:48:22.970084+0000","last_unstale":"2023-10-06T03:48:22.976082+0000","last_undegraded":"2023-10-06T03:48:22.976082+0000","last_fullsized":"2023-10-06T03:48:22.976082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:23.317887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.985566+0000","last_change":"2023-10-06T03:48:22.984621+0000","last_active":"2023-10-06T03:48:22.985566+0000","last_peered":"2023-10-06T03:48:22.985566+0000","last_clean":"2023-10-06T03:48:22.985566+0000","last_became_active":"2023-10-06T03:48:22.984339+0000","last_became_peered":"2023-10-06T03:48:22.984339+0000","last_unstale":"2023-10-06T03:48:22.985566+0000","last_undegraded":"2023-10-06T03:48:22.985566+0000","last_fullsized":"2023-10-06T03:48:22.985566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:35.725454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.984743+0000","last_change":"2023-10-06T03:48:22.983782+0000","last_active":"2023-10-06T03:48:22.984743+0000","last_peered":"2023-10-06T03:48:22.984743+0000","last_clean":"2023-10-06T03:48:22.984743+0000","last_became_active":"2023-10-06T03:48:22.982670+0000","last_became_peered":"2023-10-06T03:48:22.982670+0000","last_unstale":"2023-10-06T03:48:22.984743+0000","last_undegraded":"2023-10-06T03:48:22.984743+0000","last_fullsized":"2023-10-06T03:48:22.984743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:50.215524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.978789+0000","last_change":"2023-10-06T03:48:22.972912+0000","last_active":"2023-10-06T03:48:22.978789+0000","last_peered":"2023-10-06T03:48:22.978789+0000","last_clean":"2023-10-06T03:48:22.978789+0000","last_became_active":"2023-10-06T03:48:22.972623+0000","last_became_peered":"2023-10-06T03:48:22.972623+0000","last_unstale":"2023-10-06T03:48:22.978789+0000","last_undegraded":"2023-10-06T03:48:22.978789+0000","last_fullsized":"2023-10-06T03:48:22.978789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:44.334010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981079+0000","last_change":"2023-10-06T03:48:22.973112+0000","last_active":"2023-10-06T03:48:22.981079+0000","last_peered":"2023-10-06T03:48:22.981079+0000","last_clean":"2023-10-06T03:48:22.981079+0000","last_became_active":"2023-10-06T03:48:22.972785+0000","last_became_peered":"2023-10-06T03:48:22.972785+0000","last_unstale":"2023-10-06T03:48:22.981079+0000","last_undegraded":"2023-10-06T03:48:22.981079+0000","last_fullsized":"2023-10-06T03:48:22.981079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:14.661907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971574+0000","last_change":"2023-10-06T03:48:22.964446+0000","last_active":"2023-10-06T03:48:22.971574+0000","last_peered":"2023-10-06T03:48:22.971574+0000","last_clean":"2023-10-06T03:48:22.971574+0000","last_became_active":"2023-10-06T03:48:22.964198+0000","last_became_peered":"2023-10-06T03:48:22.964198+0000","last_unstale":"2023-10-06T03:48:22.971574+0000","last_undegraded":"2023-10-06T03:48:22.971574+0000","last_fullsized":"2023-10-06T03:48:22.971574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:28:13.522488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972748+0000","last_change":"2023-10-06T03:48:22.964606+0000","last_active":"2023-10-06T03:48:22.972748+0000","last_peered":"2023-10-06T03:48:22.972748+0000","last_clean":"2023-10-06T03:48:22.972748+0000","last_became_active":"2023-10-06T03:48:22.964252+0000","last_became_peered":"2023-10-06T03:48:22.964252+0000","last_unstale":"2023-10-06T03:48:22.972748+0000","last_undegraded":"2023-10-06T03:48:22.972748+0000","last_fullsized":"2023-10-06T03:48:22.972748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:59.411057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981867+0000","last_change":"2023-10-06T03:48:22.972302+0000","last_active":"2023-10-06T03:48:22.981867+0000","last_peered":"2023-10-06T03:48:22.981867+0000","last_clean":"2023-10-06T03:48:22.981867+0000","last_became_active":"2023-10-06T03:48:22.972024+0000","last_became_peered":"2023-10-06T03:48:22.972024+0000","last_unstale":"2023-10-06T03:48:22.981867+0000","last_undegraded":"2023-10-06T03:48:22.981867+0000","last_fullsized":"2023-10-06T03:48:22.981867+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:00.072378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.974860+0000","last_change":"2023-10-06T03:48:22.968910+0000","last_active":"2023-10-06T03:48:22.974860+0000","last_peered":"2023-10-06T03:48:22.974860+0000","last_clean":"2023-10-06T03:48:22.974860+0000","last_became_active":"2023-10-06T03:48:22.968335+0000","last_became_peered":"2023-10-06T03:48:22.968335+0000","last_unstale":"2023-10-06T03:48:22.974860+0000","last_undegraded":"2023-10-06T03:48:22.974860+0000","last_fullsized":"2023-10-06T03:48:22.974860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:10.057932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981805+0000","last_change":"2023-10-06T03:48:22.975177+0000","last_active":"2023-10-06T03:48:22.981805+0000","last_peered":"2023-10-06T03:48:22.981805+0000","last_clean":"2023-10-06T03:48:22.981805+0000","last_became_active":"2023-10-06T03:48:22.974184+0000","last_became_peered":"2023-10-06T03:48:22.974184+0000","last_unstale":"2023-10-06T03:48:22.981805+0000","last_undegraded":"2023-10-06T03:48:22.981805+0000","last_fullsized":"2023-10-06T03:48:22.981805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:44.273052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981912+0000","last_change":"2023-10-06T03:48:22.973196+0000","last_active":"2023-10-06T03:48:22.981912+0000","last_peered":"2023-10-06T03:48:22.981912+0000","last_clean":"2023-10-06T03:48:22.981912+0000","last_became_active":"2023-10-06T03:48:22.972823+0000","last_became_peered":"2023-10-06T03:48:22.972823+0000","last_unstale":"2023-10-06T03:48:22.981912+0000","last_undegraded":"2023-10-06T03:48:22.981912+0000","last_fullsized":"2023-10-06T03:48:22.981912+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:02.782253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.985661+0000","last_change":"2023-10-06T03:48:22.984009+0000","last_active":"2023-10-06T03:48:22.985661+0000","last_peered":"2023-10-06T03:48:22.985661+0000","last_clean":"2023-10-06T03:48:22.985661+0000","last_became_active":"2023-10-06T03:48:22.983746+0000","last_became_peered":"2023-10-06T03:48:22.983746+0000","last_unstale":"2023-10-06T03:48:22.985661+0000","last_undegraded":"2023-10-06T03:48:22.985661+0000","last_fullsized":"2023-10-06T03:48:22.985661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:12.970745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.976801+0000","last_change":"2023-10-06T03:48:22.969794+0000","last_active":"2023-10-06T03:48:22.976801+0000","last_peered":"2023-10-06T03:48:22.976801+0000","last_clean":"2023-10-06T03:48:22.976801+0000","last_became_active":"2023-10-06T03:48:22.969509+0000","last_became_peered":"2023-10-06T03:48:22.969509+0000","last_unstale":"2023-10-06T03:48:22.976801+0000","last_undegraded":"2023-10-06T03:48:22.976801+0000","last_fullsized":"2023-10-06T03:48:22.976801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:40:00.871577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.975985+0000","last_change":"2023-10-06T03:48:22.969265+0000","last_active":"2023-10-06T03:48:22.975985+0000","last_peered":"2023-10-06T03:48:22.975985+0000","last_clean":"2023-10-06T03:48:22.975985+0000","last_became_active":"2023-10-06T03:48:22.968581+0000","last_became_peered":"2023-10-06T03:48:22.968581+0000","last_unstale":"2023-10-06T03:48:22.975985+0000","last_undegraded":"2023-10-06T03:48:22.975985+0000","last_fullsized":"2023-10-06T03:48:22.975985+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:15.583929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972008+0000","last_change":"2023-10-06T03:48:22.965736+0000","last_active":"2023-10-06T03:48:22.972008+0000","last_peered":"2023-10-06T03:48:22.972008+0000","last_clean":"2023-10-06T03:48:22.972008+0000","last_became_active":"2023-10-06T03:48:22.965464+0000","last_became_peered":"2023-10-06T03:48:22.965464+0000","last_unstale":"2023-10-06T03:48:22.972008+0000","last_undegraded":"2023-10-06T03:48:22.972008+0000","last_fullsized":"2023-10-06T03:48:22.972008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:06:50.215524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980795+0000","last_change":"2023-10-06T03:48:22.969599+0000","last_active":"2023-10-06T03:48:22.980795+0000","last_peered":"2023-10-06T03:48:22.980795+0000","last_clean":"2023-10-06T03:48:22.980795+0000","last_became_active":"2023-10-06T03:48:22.969334+0000","last_became_peered":"2023-10-06T03:48:22.969334+0000","last_unstale":"2023-10-06T03:48:22.980795+0000","last_undegraded":"2023-10-06T03:48:22.980795+0000","last_fullsized":"2023-10-06T03:48:22.980795+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:08:33.357173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.979756+0000","last_change":"2023-10-06T03:48:22.977428+0000","last_active":"2023-10-06T03:48:22.979756+0000","last_peered":"2023-10-06T03:48:22.979756+0000","last_clean":"2023-10-06T03:48:22.979756+0000","last_became_active":"2023-10-06T03:48:22.977087+0000","last_became_peered":"2023-10-06T03:48:22.977087+0000","last_unstale":"2023-10-06T03:48:22.979756+0000","last_undegraded":"2023-10-06T03:48:22.979756+0000","last_fullsized":"2023-10-06T03:48:22.979756+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:17.224716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.975912+0000","last_change":"2023-10-06T03:48:22.974325+0000","last_active":"2023-10-06T03:48:22.975912+0000","last_peered":"2023-10-06T03:48:22.975912+0000","last_clean":"2023-10-06T03:48:22.975912+0000","last_became_active":"2023-10-06T03:48:22.974082+0000","last_became_peered":"2023-10-06T03:48:22.974082+0000","last_unstale":"2023-10-06T03:48:22.975912+0000","last_undegraded":"2023-10-06T03:48:22.975912+0000","last_fullsized":"2023-10-06T03:48:22.975912+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:14.661907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972101+0000","last_change":"2023-10-06T03:48:22.968457+0000","last_active":"2023-10-06T03:48:22.972101+0000","last_peered":"2023-10-06T03:48:22.972101+0000","last_clean":"2023-10-06T03:48:22.972101+0000","last_became_active":"2023-10-06T03:48:22.968182+0000","last_became_peered":"2023-10-06T03:48:22.968182+0000","last_unstale":"2023-10-06T03:48:22.972101+0000","last_undegraded":"2023-10-06T03:48:22.972101+0000","last_fullsized":"2023-10-06T03:48:22.972101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:47.901242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980159+0000","last_change":"2023-10-06T03:48:22.968721+0000","last_active":"2023-10-06T03:48:22.980159+0000","last_peered":"2023-10-06T03:48:22.980159+0000","last_clean":"2023-10-06T03:48:22.980159+0000","last_became_active":"2023-10-06T03:48:22.968451+0000","last_became_peered":"2023-10-06T03:48:22.968451+0000","last_unstale":"2023-10-06T03:48:22.980159+0000","last_undegraded":"2023-10-06T03:48:22.980159+0000","last_fullsized":"2023-10-06T03:48:22.980159+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:47:58.626319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970581+0000","last_change":"2023-10-06T03:48:22.963394+0000","last_active":"2023-10-06T03:48:22.970581+0000","last_peered":"2023-10-06T03:48:22.970581+0000","last_clean":"2023-10-06T03:48:22.970581+0000","last_became_active":"2023-10-06T03:48:22.963073+0000","last_became_peered":"2023-10-06T03:48:22.963073+0000","last_unstale":"2023-10-06T03:48:22.970581+0000","last_undegraded":"2023-10-06T03:48:22.970581+0000","last_fullsized":"2023-10-06T03:48:22.970581+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:15.351000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"79'4","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.992673+0000","last_change":"2023-10-06T03:48:21.959573+0000","last_active":"2023-10-06T03:48:22.992673+0000","last_peered":"2023-10-06T03:48:22.992673+0000","last_clean":"2023-10-06T03:48:22.992673+0000","last_became_active":"2023-10-06T03:48:21.959316+0000","last_became_peered":"2023-10-06T03:48:21.959316+0000","last_unstale":"2023-10-06T03:48:22.992673+0000","last_undegraded":"2023-10-06T03:48:22.992673+0000","last_fullsized":"2023-10-06T03:48:22.992673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:11.472736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971988+0000","last_change":"2023-10-06T03:48:22.961954+0000","last_active":"2023-10-06T03:48:22.971988+0000","last_peered":"2023-10-06T03:48:22.971988+0000","last_clean":"2023-10-06T03:48:22.971988+0000","last_became_active":"2023-10-06T03:48:22.961642+0000","last_became_peered":"2023-10-06T03:48:22.961642+0000","last_unstale":"2023-10-06T03:48:22.971988+0000","last_undegraded":"2023-10-06T03:48:22.971988+0000","last_fullsized":"2023-10-06T03:48:22.971988+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:16:40.770762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.986713+0000","last_change":"2023-10-06T03:48:21.954192+0000","last_active":"2023-10-06T03:48:22.986713+0000","last_peered":"2023-10-06T03:48:22.986713+0000","last_clean":"2023-10-06T03:48:22.986713+0000","last_became_active":"2023-10-06T03:48:21.953846+0000","last_became_peered":"2023-10-06T03:48:21.953846+0000","last_unstale":"2023-10-06T03:48:22.986713+0000","last_undegraded":"2023-10-06T03:48:22.986713+0000","last_fullsized":"2023-10-06T03:48:22.986713+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:38.007175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980734+0000","last_change":"2023-10-06T03:48:22.967986+0000","last_active":"2023-10-06T03:48:22.980734+0000","last_peered":"2023-10-06T03:48:22.980734+0000","last_clean":"2023-10-06T03:48:22.980734+0000","last_became_active":"2023-10-06T03:48:22.967743+0000","last_became_peered":"2023-10-06T03:48:22.967743+0000","last_unstale":"2023-10-06T03:48:22.980734+0000","last_undegraded":"2023-10-06T03:48:22.980734+0000","last_fullsized":"2023-10-06T03:48:22.980734+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:23:46.815803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.988858+0000","last_change":"2023-10-06T03:48:21.952725+0000","last_active":"2023-10-06T03:48:22.988858+0000","last_peered":"2023-10-06T03:48:22.988858+0000","last_clean":"2023-10-06T03:48:22.988858+0000","last_became_active":"2023-10-06T03:48:21.952307+0000","last_became_peered":"2023-10-06T03:48:21.952307+0000","last_unstale":"2023-10-06T03:48:22.988858+0000","last_undegraded":"2023-10-06T03:48:22.988858+0000","last_fullsized":"2023-10-06T03:48:22.988858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:16.562701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.985371+0000","last_change":"2023-10-06T03:48:22.983206+0000","last_active":"2023-10-06T03:48:22.985371+0000","last_peered":"2023-10-06T03:48:22.985371+0000","last_clean":"2023-10-06T03:48:22.985371+0000","last_became_active":"2023-10-06T03:48:22.982830+0000","last_became_peered":"2023-10-06T03:48:22.982830+0000","last_unstale":"2023-10-06T03:48:22.985371+0000","last_undegraded":"2023-10-06T03:48:22.985371+0000","last_fullsized":"2023-10-06T03:48:22.985371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:47.901242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.987882+0000","last_change":"2023-10-06T03:48:21.954325+0000","last_active":"2023-10-06T03:48:22.987882+0000","last_peered":"2023-10-06T03:48:22.987882+0000","last_clean":"2023-10-06T03:48:22.987882+0000","last_became_active":"2023-10-06T03:48:21.954034+0000","last_became_peered":"2023-10-06T03:48:21.954034+0000","last_unstale":"2023-10-06T03:48:22.987882+0000","last_undegraded":"2023-10-06T03:48:22.987882+0000","last_fullsized":"2023-10-06T03:48:22.987882+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:01.772602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970732+0000","last_change":"2023-10-06T03:48:22.969730+0000","last_active":"2023-10-06T03:48:22.970732+0000","last_peered":"2023-10-06T03:48:22.970732+0000","last_clean":"2023-10-06T03:48:22.970732+0000","last_became_active":"2023-10-06T03:48:22.969449+0000","last_became_peered":"2023-10-06T03:48:22.969449+0000","last_unstale":"2023-10-06T03:48:22.970732+0000","last_undegraded":"2023-10-06T03:48:22.970732+0000","last_fullsized":"2023-10-06T03:48:22.970732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:41.165824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.984300+0000","last_change":"2023-10-06T03:48:21.946173+0000","last_active":"2023-10-06T03:48:22.984300+0000","last_peered":"2023-10-06T03:48:22.984300+0000","last_clean":"2023-10-06T03:48:22.984300+0000","last_became_active":"2023-10-06T03:48:21.945822+0000","last_became_peered":"2023-10-06T03:48:21.945822+0000","last_unstale":"2023-10-06T03:48:22.984300+0000","last_undegraded":"2023-10-06T03:48:22.984300+0000","last_fullsized":"2023-10-06T03:48:22.984300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:08:46.891591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981121+0000","last_change":"2023-10-06T03:48:22.969630+0000","last_active":"2023-10-06T03:48:22.981121+0000","last_peered":"2023-10-06T03:48:22.981121+0000","last_clean":"2023-10-06T03:48:22.981121+0000","last_became_active":"2023-10-06T03:48:22.969362+0000","last_became_peered":"2023-10-06T03:48:22.969362+0000","last_unstale":"2023-10-06T03:48:22.981121+0000","last_undegraded":"2023-10-06T03:48:22.981121+0000","last_fullsized":"2023-10-06T03:48:22.981121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:04.560113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.986941+0000","last_change":"2023-10-06T03:48:21.944675+0000","last_active":"2023-10-06T03:48:22.986941+0000","last_peered":"2023-10-06T03:48:22.986941+0000","last_clean":"2023-10-06T03:48:22.986941+0000","last_became_active":"2023-10-06T03:48:21.944421+0000","last_became_peered":"2023-10-06T03:48:21.944421+0000","last_unstale":"2023-10-06T03:48:22.986941+0000","last_undegraded":"2023-10-06T03:48:22.986941+0000","last_fullsized":"2023-10-06T03:48:22.986941+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:12.199710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.974865+0000","last_change":"2023-10-06T03:48:22.966222+0000","last_active":"2023-10-06T03:48:22.974865+0000","last_peered":"2023-10-06T03:48:22.974865+0000","last_clean":"2023-10-06T03:48:22.974865+0000","last_became_active":"2023-10-06T03:48:22.965935+0000","last_became_peered":"2023-10-06T03:48:22.965935+0000","last_unstale":"2023-10-06T03:48:22.974865+0000","last_undegraded":"2023-10-06T03:48:22.974865+0000","last_fullsized":"2023-10-06T03:48:22.974865+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:24.367540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.961390+0000","last_change":"2023-10-06T03:48:21.951726+0000","last_active":"2023-10-06T03:48:22.961390+0000","last_peered":"2023-10-06T03:48:22.961390+0000","last_clean":"2023-10-06T03:48:22.961390+0000","last_became_active":"2023-10-06T03:48:21.951477+0000","last_became_peered":"2023-10-06T03:48:21.951477+0000","last_unstale":"2023-10-06T03:48:22.961390+0000","last_undegraded":"2023-10-06T03:48:22.961390+0000","last_fullsized":"2023-10-06T03:48:22.961390+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:16.215064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.982148+0000","last_change":"2023-10-06T03:48:22.974712+0000","last_active":"2023-10-06T03:48:22.982148+0000","last_peered":"2023-10-06T03:48:22.982148+0000","last_clean":"2023-10-06T03:48:22.982148+0000","last_became_active":"2023-10-06T03:48:22.974451+0000","last_became_peered":"2023-10-06T03:48:22.974451+0000","last_unstale":"2023-10-06T03:48:22.982148+0000","last_undegraded":"2023-10-06T03:48:22.982148+0000","last_fullsized":"2023-10-06T03:48:22.982148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:50.993039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.953339+0000","last_change":"2023-10-06T03:48:21.954823+0000","last_active":"2023-10-06T03:48:22.953339+0000","last_peered":"2023-10-06T03:48:22.953339+0000","last_clean":"2023-10-06T03:48:22.953339+0000","last_became_active":"2023-10-06T03:48:21.954575+0000","last_became_peered":"2023-10-06T03:48:21.954575+0000","last_unstale":"2023-10-06T03:48:22.953339+0000","last_undegraded":"2023-10-06T03:48:22.953339+0000","last_fullsized":"2023-10-06T03:48:22.953339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:08:58.471188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980776+0000","last_change":"2023-10-06T03:48:22.965327+0000","last_active":"2023-10-06T03:48:22.980776+0000","last_peered":"2023-10-06T03:48:22.980776+0000","last_clean":"2023-10-06T03:48:22.980776+0000","last_became_active":"2023-10-06T03:48:22.964956+0000","last_became_peered":"2023-10-06T03:48:22.964956+0000","last_unstale":"2023-10-06T03:48:22.980776+0000","last_undegraded":"2023-10-06T03:48:22.980776+0000","last_fullsized":"2023-10-06T03:48:22.980776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:23:10.057932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.966607+0000","last_change":"2023-10-06T03:48:21.962473+0000","last_active":"2023-10-06T03:48:22.966607+0000","last_peered":"2023-10-06T03:48:22.966607+0000","last_clean":"2023-10-06T03:48:22.966607+0000","last_became_active":"2023-10-06T03:48:21.962144+0000","last_became_peered":"2023-10-06T03:48:21.962144+0000","last_unstale":"2023-10-06T03:48:22.966607+0000","last_undegraded":"2023-10-06T03:48:22.966607+0000","last_fullsized":"2023-10-06T03:48:22.966607+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:23.381899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981836+0000","last_change":"2023-10-06T03:48:22.970666+0000","last_active":"2023-10-06T03:48:22.981836+0000","last_peered":"2023-10-06T03:48:22.981836+0000","last_clean":"2023-10-06T03:48:22.981836+0000","last_became_active":"2023-10-06T03:48:22.970339+0000","last_became_peered":"2023-10-06T03:48:22.970339+0000","last_unstale":"2023-10-06T03:48:22.981836+0000","last_undegraded":"2023-10-06T03:48:22.981836+0000","last_fullsized":"2023-10-06T03:48:22.981836+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:09.125474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.953769+0000","last_change":"2023-10-06T03:48:21.956450+0000","last_active":"2023-10-06T03:48:22.953769+0000","last_peered":"2023-10-06T03:48:22.953769+0000","last_clean":"2023-10-06T03:48:22.953769+0000","last_became_active":"2023-10-06T03:48:21.956099+0000","last_became_peered":"2023-10-06T03:48:21.956099+0000","last_unstale":"2023-10-06T03:48:22.953769+0000","last_undegraded":"2023-10-06T03:48:22.953769+0000","last_fullsized":"2023-10-06T03:48:22.953769+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:03:03.550462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.978245+0000","last_change":"2023-10-06T03:48:22.977370+0000","last_active":"2023-10-06T03:48:22.978245+0000","last_peered":"2023-10-06T03:48:22.978245+0000","last_clean":"2023-10-06T03:48:22.978245+0000","last_became_active":"2023-10-06T03:48:22.977088+0000","last_became_peered":"2023-10-06T03:48:22.977088+0000","last_unstale":"2023-10-06T03:48:22.978245+0000","last_undegraded":"2023-10-06T03:48:22.978245+0000","last_fullsized":"2023-10-06T03:48:22.978245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:47.548376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"79'3","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.992610+0000","last_change":"2023-10-06T03:48:21.962149+0000","last_active":"2023-10-06T03:48:22.992610+0000","last_peered":"2023-10-06T03:48:22.992610+0000","last_clean":"2023-10-06T03:48:22.992610+0000","last_became_active":"2023-10-06T03:48:21.961686+0000","last_became_peered":"2023-10-06T03:48:21.961686+0000","last_unstale":"2023-10-06T03:48:22.992610+0000","last_undegraded":"2023-10-06T03:48:22.992610+0000","last_fullsized":"2023-10-06T03:48:22.992610+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:14.197511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.973148+0000","last_change":"2023-10-06T03:48:22.968754+0000","last_active":"2023-10-06T03:48:22.973148+0000","last_peered":"2023-10-06T03:48:22.973148+0000","last_clean":"2023-10-06T03:48:22.973148+0000","last_became_active":"2023-10-06T03:48:22.968469+0000","last_became_peered":"2023-10-06T03:48:22.968469+0000","last_unstale":"2023-10-06T03:48:22.973148+0000","last_undegraded":"2023-10-06T03:48:22.973148+0000","last_fullsized":"2023-10-06T03:48:22.973148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:30.408052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"79'6","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.990295+0000","last_change":"2023-10-06T03:48:21.959258+0000","last_active":"2023-10-06T03:48:22.990295+0000","last_peered":"2023-10-06T03:48:22.990295+0000","last_clean":"2023-10-06T03:48:22.990295+0000","last_became_active":"2023-10-06T03:48:21.958956+0000","last_became_peered":"2023-10-06T03:48:21.958956+0000","last_unstale":"2023-10-06T03:48:22.990295+0000","last_undegraded":"2023-10-06T03:48:22.990295+0000","last_fullsized":"2023-10-06T03:48:22.990295+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:46.538724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.977036+0000","last_change":"2023-10-06T03:48:22.974061+0000","last_active":"2023-10-06T03:48:22.977036+0000","last_peered":"2023-10-06T03:48:22.977036+0000","last_clean":"2023-10-06T03:48:22.977036+0000","last_became_active":"2023-10-06T03:48:22.973763+0000","last_became_peered":"2023-10-06T03:48:22.973763+0000","last_unstale":"2023-10-06T03:48:22.977036+0000","last_undegraded":"2023-10-06T03:48:22.977036+0000","last_fullsized":"2023-10-06T03:48:22.977036+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:09.125474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.992491+0000","last_change":"2023-10-06T03:48:21.962189+0000","last_active":"2023-10-06T03:48:22.992491+0000","last_peered":"2023-10-06T03:48:22.992491+0000","last_clean":"2023-10-06T03:48:22.992491+0000","last_became_active":"2023-10-06T03:48:21.961581+0000","last_became_peered":"2023-10-06T03:48:21.961581+0000","last_unstale":"2023-10-06T03:48:22.992491+0000","last_undegraded":"2023-10-06T03:48:22.992491+0000","last_fullsized":"2023-10-06T03:48:22.992491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:59.861926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.975953+0000","last_change":"2023-10-06T03:48:22.971613+0000","last_active":"2023-10-06T03:48:22.975953+0000","last_peered":"2023-10-06T03:48:22.975953+0000","last_clean":"2023-10-06T03:48:22.975953+0000","last_became_active":"2023-10-06T03:48:22.971307+0000","last_became_peered":"2023-10-06T03:48:22.971307+0000","last_unstale":"2023-10-06T03:48:22.975953+0000","last_undegraded":"2023-10-06T03:48:22.975953+0000","last_fullsized":"2023-10-06T03:48:22.975953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:36.717228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958572+0000","last_change":"2023-10-06T03:48:21.955946+0000","last_active":"2023-10-06T03:48:22.958572+0000","last_peered":"2023-10-06T03:48:22.958572+0000","last_clean":"2023-10-06T03:48:22.958572+0000","last_became_active":"2023-10-06T03:48:21.955694+0000","last_became_peered":"2023-10-06T03:48:21.955694+0000","last_unstale":"2023-10-06T03:48:22.958572+0000","last_undegraded":"2023-10-06T03:48:22.958572+0000","last_fullsized":"2023-10-06T03:48:22.958572+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:26:23.357889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.974849+0000","last_change":"2023-10-06T03:48:22.965598+0000","last_active":"2023-10-06T03:48:22.974849+0000","last_peered":"2023-10-06T03:48:22.974849+0000","last_clean":"2023-10-06T03:48:22.974849+0000","last_became_active":"2023-10-06T03:48:22.965115+0000","last_became_peered":"2023-10-06T03:48:22.965115+0000","last_unstale":"2023-10-06T03:48:22.974849+0000","last_undegraded":"2023-10-06T03:48:22.974849+0000","last_fullsized":"2023-10-06T03:48:22.974849+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:26.181100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.964644+0000","last_change":"2023-10-06T03:48:21.959481+0000","last_active":"2023-10-06T03:48:22.964644+0000","last_peered":"2023-10-06T03:48:22.964644+0000","last_clean":"2023-10-06T03:48:22.964644+0000","last_became_active":"2023-10-06T03:48:21.959233+0000","last_became_peered":"2023-10-06T03:48:21.959233+0000","last_unstale":"2023-10-06T03:48:22.964644+0000","last_undegraded":"2023-10-06T03:48:22.964644+0000","last_fullsized":"2023-10-06T03:48:22.964644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:35.707577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980740+0000","last_change":"2023-10-06T03:48:22.971271+0000","last_active":"2023-10-06T03:48:22.980740+0000","last_peered":"2023-10-06T03:48:22.980740+0000","last_clean":"2023-10-06T03:48:22.980740+0000","last_became_active":"2023-10-06T03:48:22.970969+0000","last_became_peered":"2023-10-06T03:48:22.970969+0000","last_unstale":"2023-10-06T03:48:22.980740+0000","last_undegraded":"2023-10-06T03:48:22.980740+0000","last_fullsized":"2023-10-06T03:48:22.980740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:32.235650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"79'8","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.988873+0000","last_change":"2023-10-06T03:48:21.959296+0000","last_active":"2023-10-06T03:48:22.988873+0000","last_peered":"2023-10-06T03:48:22.988873+0000","last_clean":"2023-10-06T03:48:22.988873+0000","last_became_active":"2023-10-06T03:48:21.958948+0000","last_became_peered":"2023-10-06T03:48:21.958948+0000","last_unstale":"2023-10-06T03:48:22.988873+0000","last_undegraded":"2023-10-06T03:48:22.988873+0000","last_fullsized":"2023-10-06T03:48:22.988873+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:13.652256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.983389+0000","last_change":"2023-10-06T03:48:22.979534+0000","last_active":"2023-10-06T03:48:22.983389+0000","last_peered":"2023-10-06T03:48:22.983389+0000","last_clean":"2023-10-06T03:48:22.983389+0000","last_became_active":"2023-10-06T03:48:22.979272+0000","last_became_peered":"2023-10-06T03:48:22.979272+0000","last_unstale":"2023-10-06T03:48:22.983389+0000","last_undegraded":"2023-10-06T03:48:22.983389+0000","last_fullsized":"2023-10-06T03:48:22.983389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:03.329123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.990320+0000","last_change":"2023-10-06T03:48:21.959483+0000","last_active":"2023-10-06T03:48:22.990320+0000","last_peered":"2023-10-06T03:48:22.990320+0000","last_clean":"2023-10-06T03:48:22.990320+0000","last_became_active":"2023-10-06T03:48:21.959233+0000","last_became_peered":"2023-10-06T03:48:21.959233+0000","last_unstale":"2023-10-06T03:48:22.990320+0000","last_undegraded":"2023-10-06T03:48:22.990320+0000","last_fullsized":"2023-10-06T03:48:22.990320+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:08.515631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.967698+0000","last_change":"2023-10-06T03:48:22.964069+0000","last_active":"2023-10-06T03:48:22.967698+0000","last_peered":"2023-10-06T03:48:22.967698+0000","last_clean":"2023-10-06T03:48:22.967698+0000","last_became_active":"2023-10-06T03:48:22.963794+0000","last_became_peered":"2023-10-06T03:48:22.963794+0000","last_unstale":"2023-10-06T03:48:22.967698+0000","last_undegraded":"2023-10-06T03:48:22.967698+0000","last_fullsized":"2023-10-06T03:48:22.967698+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:03.317508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"80'1","reported_seq":11,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-10-06T03:48:24.350602+0000","last_change":"2023-10-06T03:48:21.949399+0000","last_active":"2023-10-06T03:48:24.350602+0000","last_peered":"2023-10-06T03:48:24.350602+0000","last_clean":"2023-10-06T03:48:24.350602+0000","last_became_active":"2023-10-06T03:48:21.949065+0000","last_became_peered":"2023-10-06T03:48:21.949065+0000","last_unstale":"2023-10-06T03:48:24.350602+0000","last_undegraded":"2023-10-06T03:48:24.350602+0000","last_fullsized":"2023-10-06T03:48:24.350602+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:24.883643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.976002+0000","last_change":"2023-10-06T03:48:22.969329+0000","last_active":"2023-10-06T03:48:22.976002+0000","last_peered":"2023-10-06T03:48:22.976002+0000","last_clean":"2023-10-06T03:48:22.976002+0000","last_became_active":"2023-10-06T03:48:22.969048+0000","last_became_peered":"2023-10-06T03:48:22.969048+0000","last_unstale":"2023-10-06T03:48:22.976002+0000","last_undegraded":"2023-10-06T03:48:22.976002+0000","last_fullsized":"2023-10-06T03:48:22.976002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:43:15.207162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958635+0000","last_change":"2023-10-06T03:48:21.950832+0000","last_active":"2023-10-06T03:48:22.958635+0000","last_peered":"2023-10-06T03:48:22.958635+0000","last_clean":"2023-10-06T03:48:22.958635+0000","last_became_active":"2023-10-06T03:48:21.950526+0000","last_became_peered":"2023-10-06T03:48:21.950526+0000","last_unstale":"2023-10-06T03:48:22.958635+0000","last_undegraded":"2023-10-06T03:48:22.958635+0000","last_fullsized":"2023-10-06T03:48:22.958635+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:41.936178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970721+0000","last_change":"2023-10-06T03:48:22.965699+0000","last_active":"2023-10-06T03:48:22.970721+0000","last_peered":"2023-10-06T03:48:22.970721+0000","last_clean":"2023-10-06T03:48:22.970721+0000","last_became_active":"2023-10-06T03:48:22.965291+0000","last_became_peered":"2023-10-06T03:48:22.965291+0000","last_unstale":"2023-10-06T03:48:22.970721+0000","last_undegraded":"2023-10-06T03:48:22.970721+0000","last_fullsized":"2023-10-06T03:48:22.970721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:58.933292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.956473+0000","last_change":"2023-10-06T03:48:21.958507+0000","last_active":"2023-10-06T03:48:22.956473+0000","last_peered":"2023-10-06T03:48:22.956473+0000","last_clean":"2023-10-06T03:48:22.956473+0000","last_became_active":"2023-10-06T03:48:21.958265+0000","last_became_peered":"2023-10-06T03:48:21.958265+0000","last_unstale":"2023-10-06T03:48:22.956473+0000","last_undegraded":"2023-10-06T03:48:22.956473+0000","last_fullsized":"2023-10-06T03:48:22.956473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:57:57.923641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972857+0000","last_change":"2023-10-06T03:48:22.963396+0000","last_active":"2023-10-06T03:48:22.972857+0000","last_peered":"2023-10-06T03:48:22.972857+0000","last_clean":"2023-10-06T03:48:22.972857+0000","last_became_active":"2023-10-06T03:48:22.963027+0000","last_became_peered":"2023-10-06T03:48:22.963027+0000","last_unstale":"2023-10-06T03:48:22.972857+0000","last_undegraded":"2023-10-06T03:48:22.972857+0000","last_fullsized":"2023-10-06T03:48:22.972857+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:12.482387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.965233+0000","last_change":"2023-10-06T03:48:21.960124+0000","last_active":"2023-10-06T03:48:22.965233+0000","last_peered":"2023-10-06T03:48:22.965233+0000","last_clean":"2023-10-06T03:48:22.965233+0000","last_became_active":"2023-10-06T03:48:21.959839+0000","last_became_peered":"2023-10-06T03:48:21.959839+0000","last_unstale":"2023-10-06T03:48:22.965233+0000","last_undegraded":"2023-10-06T03:48:22.965233+0000","last_fullsized":"2023-10-06T03:48:22.965233+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:19.590235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.977066+0000","last_change":"2023-10-06T03:48:22.975809+0000","last_active":"2023-10-06T03:48:22.977066+0000","last_peered":"2023-10-06T03:48:22.977066+0000","last_clean":"2023-10-06T03:48:22.977066+0000","last_became_active":"2023-10-06T03:48:22.975112+0000","last_became_peered":"2023-10-06T03:48:22.975112+0000","last_unstale":"2023-10-06T03:48:22.977066+0000","last_undegraded":"2023-10-06T03:48:22.977066+0000","last_fullsized":"2023-10-06T03:48:22.977066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:02:50.993039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958241+0000","last_change":"2023-10-06T03:48:21.951488+0000","last_active":"2023-10-06T03:48:22.958241+0000","last_peered":"2023-10-06T03:48:22.958241+0000","last_clean":"2023-10-06T03:48:22.958241+0000","last_became_active":"2023-10-06T03:48:21.951196+0000","last_became_peered":"2023-10-06T03:48:21.951196+0000","last_unstale":"2023-10-06T03:48:22.958241+0000","last_undegraded":"2023-10-06T03:48:22.958241+0000","last_fullsized":"2023-10-06T03:48:22.958241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:10:34.715803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981040+0000","last_change":"2023-10-06T03:48:22.964301+0000","last_active":"2023-10-06T03:48:22.981040+0000","last_peered":"2023-10-06T03:48:22.981040+0000","last_clean":"2023-10-06T03:48:22.981040+0000","last_became_active":"2023-10-06T03:48:22.963870+0000","last_became_peered":"2023-10-06T03:48:22.963870+0000","last_unstale":"2023-10-06T03:48:22.981040+0000","last_undegraded":"2023-10-06T03:48:22.981040+0000","last_fullsized":"2023-10-06T03:48:22.981040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:42:24.959369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.961484+0000","last_change":"2023-10-06T03:48:21.950936+0000","last_active":"2023-10-06T03:48:22.961484+0000","last_peered":"2023-10-06T03:48:22.961484+0000","last_clean":"2023-10-06T03:48:22.961484+0000","last_became_active":"2023-10-06T03:48:21.950639+0000","last_became_peered":"2023-10-06T03:48:21.950639+0000","last_unstale":"2023-10-06T03:48:22.961484+0000","last_undegraded":"2023-10-06T03:48:22.961484+0000","last_fullsized":"2023-10-06T03:48:22.961484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:11.961094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981729+0000","last_change":"2023-10-06T03:48:22.980115+0000","last_active":"2023-10-06T03:48:22.981729+0000","last_peered":"2023-10-06T03:48:22.981729+0000","last_clean":"2023-10-06T03:48:22.981729+0000","last_became_active":"2023-10-06T03:48:22.979810+0000","last_became_peered":"2023-10-06T03:48:22.979810+0000","last_unstale":"2023-10-06T03:48:22.981729+0000","last_undegraded":"2023-10-06T03:48:22.981729+0000","last_fullsized":"2023-10-06T03:48:22.981729+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:43:20.245038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.955562+0000","last_change":"2023-10-06T03:48:21.961740+0000","last_active":"2023-10-06T03:48:22.955562+0000","last_peered":"2023-10-06T03:48:22.955562+0000","last_clean":"2023-10-06T03:48:22.955562+0000","last_became_active":"2023-10-06T03:48:21.961381+0000","last_became_peered":"2023-10-06T03:48:21.961381+0000","last_unstale":"2023-10-06T03:48:22.955562+0000","last_undegraded":"2023-10-06T03:48:22.955562+0000","last_fullsized":"2023-10-06T03:48:22.955562+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:03.746648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971551+0000","last_change":"2023-10-06T03:48:22.962578+0000","last_active":"2023-10-06T03:48:22.971551+0000","last_peered":"2023-10-06T03:48:22.971551+0000","last_clean":"2023-10-06T03:48:22.971551+0000","last_became_active":"2023-10-06T03:48:22.962320+0000","last_became_peered":"2023-10-06T03:48:22.962320+0000","last_unstale":"2023-10-06T03:48:22.971551+0000","last_undegraded":"2023-10-06T03:48:22.971551+0000","last_fullsized":"2023-10-06T03:48:22.971551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:06.661073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.992556+0000","last_change":"2023-10-06T03:48:21.961642+0000","last_active":"2023-10-06T03:48:22.992556+0000","last_peered":"2023-10-06T03:48:22.992556+0000","last_clean":"2023-10-06T03:48:22.992556+0000","last_became_active":"2023-10-06T03:48:21.961209+0000","last_became_peered":"2023-10-06T03:48:21.961209+0000","last_unstale":"2023-10-06T03:48:22.992556+0000","last_undegraded":"2023-10-06T03:48:22.992556+0000","last_fullsized":"2023-10-06T03:48:22.992556+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:19:22.308236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.954065+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:48:22.954065+0000","last_peered":"2023-10-06T03:48:22.954065+0000","last_clean":"2023-10-06T03:48:22.954065+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:48:22.954065+0000","last_undegraded":"2023-10-06T03:48:22.954065+0000","last_fullsized":"2023-10-06T03:48:22.954065+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971664+0000","last_change":"2023-10-06T03:48:22.970569+0000","last_active":"2023-10-06T03:48:22.971664+0000","last_peered":"2023-10-06T03:48:22.971664+0000","last_clean":"2023-10-06T03:48:22.971664+0000","last_became_active":"2023-10-06T03:48:22.968580+0000","last_became_peered":"2023-10-06T03:48:22.968580+0000","last_unstale":"2023-10-06T03:48:22.971664+0000","last_undegraded":"2023-10-06T03:48:22.971664+0000","last_fullsized":"2023-10-06T03:48:22.971664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:17:04.756299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.960687+0000","last_change":"2023-10-06T03:48:21.958186+0000","last_active":"2023-10-06T03:48:22.960687+0000","last_peered":"2023-10-06T03:48:22.960687+0000","last_clean":"2023-10-06T03:48:22.960687+0000","last_became_active":"2023-10-06T03:48:21.957831+0000","last_became_peered":"2023-10-06T03:48:21.957831+0000","last_unstale":"2023-10-06T03:48:22.960687+0000","last_undegraded":"2023-10-06T03:48:22.960687+0000","last_fullsized":"2023-10-06T03:48:22.960687+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:11.915578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.960769+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:48:22.960769+0000","last_peered":"2023-10-06T03:48:22.960769+0000","last_clean":"2023-10-06T03:48:22.960769+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:48:22.960769+0000","last_undegraded":"2023-10-06T03:48:22.960769+0000","last_fullsized":"2023-10-06T03:48:22.960769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.980136+0000","last_change":"2023-10-06T03:48:22.968062+0000","last_active":"2023-10-06T03:48:22.980136+0000","last_peered":"2023-10-06T03:48:22.980136+0000","last_clean":"2023-10-06T03:48:22.980136+0000","last_became_active":"2023-10-06T03:48:22.967779+0000","last_became_peered":"2023-10-06T03:48:22.967779+0000","last_unstale":"2023-10-06T03:48:22.980136+0000","last_undegraded":"2023-10-06T03:48:22.980136+0000","last_fullsized":"2023-10-06T03:48:22.980136+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:54:14.771414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.984625+0000","last_change":"2023-10-06T03:48:21.954894+0000","last_active":"2023-10-06T03:48:22.984625+0000","last_peered":"2023-10-06T03:48:22.984625+0000","last_clean":"2023-10-06T03:48:22.984625+0000","last_became_active":"2023-10-06T03:48:21.954613+0000","last_became_peered":"2023-10-06T03:48:21.954613+0000","last_unstale":"2023-10-06T03:48:22.984625+0000","last_undegraded":"2023-10-06T03:48:22.984625+0000","last_fullsized":"2023-10-06T03:48:22.984625+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:43.263401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":66,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.955914+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:48:22.955914+0000","last_peered":"2023-10-06T03:48:22.955914+0000","last_clean":"2023-10-06T03:48:22.955914+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:48:22.955914+0000","last_undegraded":"2023-10-06T03:48:22.955914+0000","last_fullsized":"2023-10-06T03:48:22.955914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"70'161","reported_seq":372,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.963654+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:48:22.963654+0000","last_peered":"2023-10-06T03:48:22.963654+0000","last_clean":"2023-10-06T03:48:22.963654+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:48:22.963654+0000","last_undegraded":"2023-10-06T03:48:22.963654+0000","last_fullsized":"2023-10-06T03:48:22.963654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.971487+0000","last_change":"2023-10-06T03:48:22.967727+0000","last_active":"2023-10-06T03:48:22.971487+0000","last_peered":"2023-10-06T03:48:22.971487+0000","last_clean":"2023-10-06T03:48:22.971487+0000","last_became_active":"2023-10-06T03:48:22.967352+0000","last_became_peered":"2023-10-06T03:48:22.967352+0000","last_unstale":"2023-10-06T03:48:22.971487+0000","last_undegraded":"2023-10-06T03:48:22.971487+0000","last_fullsized":"2023-10-06T03:48:22.971487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:20.599886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958689+0000","last_change":"2023-10-06T03:48:21.954564+0000","last_active":"2023-10-06T03:48:22.958689+0000","last_peered":"2023-10-06T03:48:22.958689+0000","last_clean":"2023-10-06T03:48:22.958689+0000","last_became_active":"2023-10-06T03:48:21.954153+0000","last_became_peered":"2023-10-06T03:48:21.954153+0000","last_unstale":"2023-10-06T03:48:22.958689+0000","last_undegraded":"2023-10-06T03:48:22.958689+0000","last_fullsized":"2023-10-06T03:48:22.958689+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:50.343611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":67,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.967814+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:48:22.967814+0000","last_peered":"2023-10-06T03:48:22.967814+0000","last_clean":"2023-10-06T03:48:22.967814+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:48:22.967814+0000","last_undegraded":"2023-10-06T03:48:22.967814+0000","last_fullsized":"2023-10-06T03:48:22.967814+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.970547+0000","last_change":"2023-10-06T03:48:22.967121+0000","last_active":"2023-10-06T03:48:22.970547+0000","last_peered":"2023-10-06T03:48:22.970547+0000","last_clean":"2023-10-06T03:48:22.970547+0000","last_became_active":"2023-10-06T03:48:22.966857+0000","last_became_peered":"2023-10-06T03:48:22.966857+0000","last_unstale":"2023-10-06T03:48:22.970547+0000","last_undegraded":"2023-10-06T03:48:22.970547+0000","last_fullsized":"2023-10-06T03:48:22.970547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:07.086261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.961262+0000","last_change":"2023-10-06T03:48:21.946145+0000","last_active":"2023-10-06T03:48:22.961262+0000","last_peered":"2023-10-06T03:48:22.961262+0000","last_clean":"2023-10-06T03:48:22.961262+0000","last_became_active":"2023-10-06T03:48:21.945883+0000","last_became_peered":"2023-10-06T03:48:21.945883+0000","last_unstale":"2023-10-06T03:48:22.961262+0000","last_undegraded":"2023-10-06T03:48:22.961262+0000","last_fullsized":"2023-10-06T03:48:22.961262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:00:29.398401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.963748+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:48:22.963748+0000","last_peered":"2023-10-06T03:48:22.963748+0000","last_clean":"2023-10-06T03:48:22.963748+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:48:22.963748+0000","last_undegraded":"2023-10-06T03:48:22.963748+0000","last_fullsized":"2023-10-06T03:48:22.963748+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.974827+0000","last_change":"2023-10-06T03:48:22.965384+0000","last_active":"2023-10-06T03:48:22.974827+0000","last_peered":"2023-10-06T03:48:22.974827+0000","last_clean":"2023-10-06T03:48:22.974827+0000","last_became_active":"2023-10-06T03:48:22.964719+0000","last_became_peered":"2023-10-06T03:48:22.964719+0000","last_unstale":"2023-10-06T03:48:22.974827+0000","last_undegraded":"2023-10-06T03:48:22.974827+0000","last_fullsized":"2023-10-06T03:48:22.974827+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:09.525282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958144+0000","last_change":"2023-10-06T03:48:21.958097+0000","last_active":"2023-10-06T03:48:22.958144+0000","last_peered":"2023-10-06T03:48:22.958144+0000","last_clean":"2023-10-06T03:48:22.958144+0000","last_became_active":"2023-10-06T03:48:21.957822+0000","last_became_peered":"2023-10-06T03:48:21.957822+0000","last_unstale":"2023-10-06T03:48:22.958144+0000","last_undegraded":"2023-10-06T03:48:22.958144+0000","last_fullsized":"2023-10-06T03:48:22.958144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:59.062727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.958612+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:48:22.958612+0000","last_peered":"2023-10-06T03:48:22.958612+0000","last_clean":"2023-10-06T03:48:22.958612+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:48:22.958612+0000","last_undegraded":"2023-10-06T03:48:22.958612+0000","last_fullsized":"2023-10-06T03:48:22.958612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.972727+0000","last_change":"2023-10-06T03:48:22.961087+0000","last_active":"2023-10-06T03:48:22.972727+0000","last_peered":"2023-10-06T03:48:22.972727+0000","last_clean":"2023-10-06T03:48:22.972727+0000","last_became_active":"2023-10-06T03:48:22.960836+0000","last_became_peered":"2023-10-06T03:48:22.960836+0000","last_unstale":"2023-10-06T03:48:22.972727+0000","last_undegraded":"2023-10-06T03:48:22.972727+0000","last_fullsized":"2023-10-06T03:48:22.972727+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:52.603676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"79'2","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.990082+0000","last_change":"2023-10-06T03:48:21.957833+0000","last_active":"2023-10-06T03:48:22.990082+0000","last_peered":"2023-10-06T03:48:22.990082+0000","last_clean":"2023-10-06T03:48:22.990082+0000","last_became_active":"2023-10-06T03:48:21.957260+0000","last_became_peered":"2023-10-06T03:48:21.957260+0000","last_unstale":"2023-10-06T03:48:22.990082+0000","last_undegraded":"2023-10-06T03:48:22.990082+0000","last_fullsized":"2023-10-06T03:48:22.990082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:34:31.139755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.952549+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:48:22.952549+0000","last_peered":"2023-10-06T03:48:22.952549+0000","last_clean":"2023-10-06T03:48:22.952549+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:48:22.952549+0000","last_undegraded":"2023-10-06T03:48:22.952549+0000","last_fullsized":"2023-10-06T03:48:22.952549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.981603+0000","last_change":"2023-10-06T03:48:22.980129+0000","last_active":"2023-10-06T03:48:22.981603+0000","last_peered":"2023-10-06T03:48:22.981603+0000","last_clean":"2023-10-06T03:48:22.981603+0000","last_became_active":"2023-10-06T03:48:22.979872+0000","last_became_peered":"2023-10-06T03:48:22.979872+0000","last_unstale":"2023-10-06T03:48:22.981603+0000","last_undegraded":"2023-10-06T03:48:22.981603+0000","last_fullsized":"2023-10-06T03:48:22.981603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:21.927129+0000","last_clean_scrub_stamp":"2023-10-06T03:48:21.927129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:13.209361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"79'1","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.989740+0000","last_change":"2023-10-06T03:48:21.955182+0000","last_active":"2023-10-06T03:48:22.989740+0000","last_peered":"2023-10-06T03:48:22.989740+0000","last_clean":"2023-10-06T03:48:22.989740+0000","last_became_active":"2023-10-06T03:48:21.954909+0000","last_became_peered":"2023-10-06T03:48:21.954909+0000","last_unstale":"2023-10-06T03:48:22.989740+0000","last_undegraded":"2023-10-06T03:48:22.989740+0000","last_fullsized":"2023-10-06T03:48:22.989740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:48:20.917478+0000","last_clean_scrub_stamp":"2023-10-06T03:48:20.917478+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:25.171449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-10-06T03:48:22.952991+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:48:22.952991+0000","last_peered":"2023-10-06T03:48:22.952991+0000","last_clean":"2023-10-06T03:48:22.952991+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:48:22.952991+0000","last_undegraded":"2023-10-06T03:48:22.952991+0000","last_fullsized":"2023-10-06T03:48:22.952991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575032,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311112,"kb_used_data":1536,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060728,"statfs":{"total":96636764160,"available":96318185472,"internally_reserved":0,"allocated":1572864,"data_stored":1076126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71199999999999997,"15min":0.69299999999999995},"min":{"1min":0.53000000000000003,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.998,"5min":1.0029999999999999,"15min":1.6100000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70899999999999996,"15min":0.70799999999999996},"min":{"1min":0.51500000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.5649999999999999},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.755,"15min":0.749},"min":{"1min":0.63800000000000001,"5min":0.318,"15min":0.318},"max":{"1min":1.038,"5min":1.038,"15min":1.5469999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76300000000000001,"15min":0.77000000000000002},"min":{"1min":0.59199999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.56899999999999995,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":1.5900000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.56999999999999995,"5min":0.495,"15min":0.495},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73199999999999998,"15min":0.72499999999999998},"min":{"1min":0.61299999999999999,"5min":0.45900000000000002,"15min":0.39500000000000002},"max":{"1min":1.0089999999999999,"5min":1.016,"15min":1.6279999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76000000000000001,"15min":0.74399999999999999},"min":{"1min":0.624,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.96099999999999997,"5min":0.98099999999999998,"15min":1.0409999999999999},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79600000000000004,"5min":0.96499999999999997,"15min":1.0049999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.628},"min":{"1min":0.46400000000000002,"5min":0.40999999999999998,"15min":0.39400000000000002},"max":{"1min":0.874,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.71699999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63900000000000001,"15min":0.64800000000000002},"min":{"1min":0.49299999999999999,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":0.95599999999999996,"5min":1.018,"15min":1.069},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67100000000000004,"15min":0.68600000000000005},"min":{"1min":0.495,"5min":0.46800000000000003,"15min":0.436},"max":{"1min":0.91300000000000003,"5min":1.105,"15min":1.105},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.72399999999999998,"15min":0.72899999999999998},"min":{"1min":0.58199999999999996,"5min":0.436,"15min":0.436},"max":{"1min":0.97499999999999998,"5min":1.181,"15min":1.181},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.70799999999999996},"min":{"1min":0.55200000000000005,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.91500000000000004,"5min":1.02,"15min":1.0209999999999999},"last":0.65100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575033,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306428,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065412,"statfs":{"total":96636764160,"available":96322981888,"internally_reserved":0,"allocated":970752,"data_stored":486022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.42199999999999999,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.88,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.39500000000000002},"max":{"1min":0.94799999999999995,"5min":0.98599999999999999,"15min":1.0449999999999999},"last":0.751}]},{"osd":1,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.47499999999999998,"5min":0.36499999999999999,"15min":0.33800000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.3180000000000001},"last":0.495},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.35899999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.59799999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.751,"15min":0.751},"min":{"1min":0.48199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.98699999999999999,"5min":0.995,"15min":3.3319999999999999},"last":0.628},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":3.5139999999999998},"last":0.67300000000000004}]},{"osd":3,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.58499999999999996,"5min":0.48299999999999998,"15min":0.44600000000000001},"max":{"1min":0.97399999999999998,"5min":1.01,"15min":1.024},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76100000000000001,"15min":0.77400000000000002},"min":{"1min":0.49299999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.98799999999999999,"5min":0.99099999999999999,"15min":3.3300000000000001},"last":0.70099999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.45200000000000001,"5min":0.376,"15min":0.34799999999999998},"max":{"1min":0.91900000000000004,"5min":0.92600000000000005,"15min":1.0680000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36499999999999999},"max":{"1min":0.90700000000000003,"5min":1.0169999999999999,"15min":1.2929999999999999},"last":0.79700000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67000000000000004,"15min":0.66700000000000004},"min":{"1min":0.438,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.3340000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.42899999999999999,"15min":0.41199999999999998},"max":{"1min":1.0049999999999999,"5min":1.0269999999999999,"15min":1.2370000000000001},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68600000000000005,"15min":0.69899999999999995},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.619},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.47099999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.417},"last":0.85299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575032,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307032,"kb_used_data":1552,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064808,"statfs":{"total":96636764160,"available":96322363392,"internally_reserved":0,"allocated":1589248,"data_stored":1078628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57799999999999996,"15min":0.56899999999999995},"min":{"1min":0.47999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80200000000000005,"5min":1.1319999999999999,"15min":1.2889999999999999},"last":0.5}]},{"osd":2,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.5,"15min":0.495},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.26100000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.44,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.94099999999999995},"last":0.45500000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61799999999999999,"15min":0.60199999999999998},"min":{"1min":0.50800000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":0.86699999999999999,"15min":0.93999999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.434,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77400000000000002,"5min":0.90400000000000003,"15min":1.254},"last":0.54300000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.52500000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0009999999999999,"5min":1.034,"15min":1.3600000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.80900000000000005,"15min":0.79800000000000004},"min":{"1min":0.68400000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.476},"last":0.91800000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.80200000000000005,"15min":0.79300000000000004},"min":{"1min":0.621,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":1.294,"5min":1.294,"15min":1.601},"last":0.748},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.59299999999999997,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.5189999999999999},"last":0.86499999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.80500000000000005,"15min":0.81000000000000005},"min":{"1min":0.64200000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.82499999999999996,"15min":0.81699999999999995},"min":{"1min":0.66700000000000004,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":1.143,"5min":1.1639999999999999,"15min":1.399},"last":0.80400000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:47:35 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.84799999999999998,"15min":0.83399999999999996},"min":{"1min":0.66900000000000004,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1000000000000001,"5min":1.272,"15min":1.272},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.83399999999999996,"15min":0.82899999999999996},"min":{"1min":0.64400000000000002,"5min":0.53700000000000003,"15min":0.51200000000000001},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.556},"last":0.78600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575034,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306420,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065420,"statfs":{"total":96636764160,"available":96322990080,"internally_reserved":0,"allocated":962560,"data_stored":485010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48199999999999998,"15min":0.495},"min":{"1min":0.373,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":0.67800000000000005,"5min":0.81000000000000005,"15min":2.3220000000000001},"last":0.374},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51500000000000001,"15min":0.52700000000000002},"min":{"1min":0.33800000000000002,"5min":0.33600000000000002,"15min":0.29599999999999999},"max":{"1min":0.70299999999999996,"5min":0.78500000000000003,"15min":2.1440000000000001},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.56499999999999995},"min":{"1min":0.40500000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.76400000000000001,"5min":0.77700000000000002,"15min":2.4329999999999998},"last":0.434},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54800000000000004,"15min":0.56299999999999994},"min":{"1min":0.40799999999999997,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.755,"5min":0.84699999999999998,"15min":2.1819999999999999},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.51100000000000001,"15min":0.53000000000000003},"min":{"1min":0.41399999999999998,"5min":0.30499999999999999,"15min":0.28399999999999997},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":2.2709999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61299999999999999,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.81000000000000005,"5min":1.0580000000000001,"15min":2.4700000000000002},"last":0.497}]},{"osd":4,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.83099999999999996,"15min":0.83499999999999996},"min":{"1min":0.69199999999999995,"5min":0.54200000000000004,"15min":0.49399999999999999},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.393},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.80900000000000005,"15min":0.83199999999999996},"min":{"1min":0.63400000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.306,"5min":1.306,"15min":2.2320000000000002},"last":0.76900000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.80600000000000005,"15min":0.82999999999999996},"min":{"1min":0.57799999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":2.3660000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.58999999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.0780000000000001,"5min":1.0800000000000001,"15min":1.6559999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77900000000000003,"15min":0.80500000000000005},"min":{"1min":0.56999999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.726},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.82399999999999995,"15min":0.85699999999999998},"min":{"1min":0.628,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1100000000000001,"5min":1.119,"15min":2.4009999999999998},"last":0.73799999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:47:39 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.81899999999999995,"15min":0.82199999999999995},"min":{"1min":0.64300000000000002,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2689999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.79800000000000004,"15min":0.83699999999999997},"min":{"1min":0.625,"5min":0.495,"15min":0.495},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.7649999999999999},"last":0.81899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575033,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306420,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065420,"statfs":{"total":96636764160,"available":96322990080,"internally_reserved":0,"allocated":962560,"data_stored":485010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48899999999999999,"15min":0.46700000000000003},"min":{"1min":0.35199999999999998,"5min":0.29699999999999999,"15min":0.28000000000000003},"max":{"1min":0.74199999999999999,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.53400000000000003,"15min":0.52300000000000002},"min":{"1min":0.438,"5min":0.311,"15min":0.311},"max":{"1min":0.73999999999999999,"5min":0.86199999999999999,"15min":0.89300000000000002},"last":0.47499999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.34799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72899999999999998,"5min":0.79900000000000004,"15min":0.90000000000000002},"last":0.439},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.442,"5min":0.372,"15min":0.34200000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.556},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.504,"5min":0.41399999999999998,"15min":0.38800000000000001},"max":{"1min":0.72899999999999998,"5min":1.78,"15min":1.78},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.44600000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81799999999999995,"5min":0.91400000000000003,"15min":1.1910000000000001},"last":0.42099999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.81899999999999995,"15min":0.81799999999999995},"min":{"1min":0.67600000000000005,"5min":0.51400000000000001,"15min":0.439},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.5620000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.871,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.72199999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.008,"5min":1.1419999999999999,"15min":1.2270000000000001},"last":0.69899999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.82699999999999996,"15min":0.82999999999999996},"min":{"1min":0.68100000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.1140000000000001,"5min":1.177,"15min":1.3},"last":0.749},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84399999999999997,"15min":0.84999999999999998},"min":{"1min":0.65500000000000003,"5min":0.55500000000000005,"15min":0.51700000000000002},"max":{"1min":1.0549999999999999,"5min":1.855,"15min":1.855},"last":0.76300000000000001}]},{"osd":6,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82999999999999996,"15min":0.83699999999999997},"min":{"1min":0.68400000000000005,"5min":0.54200000000000004,"15min":0.53600000000000003},"max":{"1min":0.97899999999999998,"5min":1.907,"15min":1.907},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.81899999999999995,"15min":0.81499999999999995},"min":{"1min":0.61199999999999999,"5min":0.60599999999999998,"15min":0.51100000000000001},"max":{"1min":1.1080000000000001,"5min":1.1120000000000001,"15min":1.4750000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:47:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.84099999999999997,"15min":0.83499999999999996},"min":{"1min":0.73999999999999999,"5min":0.495,"15min":0.495},"max":{"1min":1.125,"5min":1.4379999999999999,"15min":1.4990000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.86899999999999999,"15min":0.86399999999999999},"min":{"1min":0.68400000000000005,"5min":0.53100000000000003,"15min":0.497},"max":{"1min":0.99399999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.81200000000000006}]}]},{"osd":3,"up_from":13,"seq":55834575033,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306424,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065416,"statfs":{"total":96636764160,"available":96322985984,"internally_reserved":0,"allocated":966656,"data_stored":485988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.32700000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.75700000000000001,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":0.61199999999999999,"5min":1.04,"15min":1.1699999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47699999999999998,"15min":0.47799999999999998},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.63900000000000001,"5min":0.68799999999999994,"15min":0.79500000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.83899999999999997},"last":0.35299999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.34599999999999997},"max":{"1min":0.69899999999999995,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.38700000000000001},"max":{"1min":0.68600000000000005,"5min":0.81999999999999995,"15min":0.97899999999999998},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76800000000000002,"15min":0.78700000000000003},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1,"5min":1.073,"15min":1.4099999999999999},"last":0.62},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.622,"5min":0.57399999999999995,"15min":0.51300000000000001},"max":{"1min":0.96899999999999997,"5min":1.0469999999999999,"15min":1.377},"last":0.67700000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.79000000000000004},"min":{"1min":0.58999999999999997,"5min":0.51900000000000002,"15min":0.47099999999999997},"max":{"1min":1.012,"5min":1.05,"15min":1.256},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.75600000000000001,"15min":0.78700000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.48099999999999998},"max":{"1min":0.93100000000000005,"5min":1.1100000000000001,"15min":1.306},"last":0.65300000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79300000000000004,"15min":0.81399999999999995},"min":{"1min":0.56999999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"max":{"1min":0.98999999999999999,"5min":1.1399999999999999,"15min":1.323},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81799999999999995,"15min":0.84999999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.49299999999999999},"max":{"1min":1.05,"5min":1.1040000000000001,"15min":1.6020000000000001},"last":0.58599999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79100000000000004,"15min":0.81399999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86799999999999999,"5min":1.2030000000000001,"15min":1.365},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78900000000000003,"15min":0.80800000000000005},"min":{"1min":0.55800000000000005,"5min":0.54600000000000004,"15min":0.46300000000000002},"max":{"1min":1.0289999999999999,"5min":1.093,"15min":1.2330000000000001},"last":0.72099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575033,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302340,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069500,"statfs":{"total":96636764160,"available":96327168000,"internally_reserved":0,"allocated":978944,"data_stored":487524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.70599999999999996,"15min":0.72099999999999997},"min":{"1min":0.56899999999999995,"5min":0.316,"15min":0.316},"max":{"1min":0.96999999999999997,"5min":1.0109999999999999,"15min":1.3120000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.54800000000000004,"5min":0.316,"15min":0.316},"max":{"1min":0.91500000000000004,"5min":0.95299999999999996,"15min":1.01},"last":0.61099999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.47299999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.85899999999999999,"5min":1.081,"15min":1.081},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.71799999999999997,"15min":0.73699999999999999},"min":{"1min":0.64600000000000002,"5min":0.44,"15min":0.41299999999999998},"max":{"1min":0.89600000000000002,"5min":0.96999999999999997,"15min":1.3380000000000001},"last":0.64900000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71899999999999997,"15min":0.73699999999999999},"min":{"1min":0.61399999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.91500000000000004,"5min":1.028,"15min":1.1759999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.747,"15min":0.746},"min":{"1min":0.629,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0109999999999999,"5min":1.022,"15min":1.3089999999999999},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.63100000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89700000000000002,"5min":1.0049999999999999,"15min":1.0409999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.75800000000000001,"15min":0.76900000000000002},"min":{"1min":0.53200000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95099999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.63100000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.621,"15min":0.63},"min":{"1min":0.496,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.878,"5min":1.01,"15min":1.085},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.622,"15min":0.63300000000000001},"min":{"1min":0.47899999999999998,"5min":0.40899999999999997,"15min":0.35099999999999998},"max":{"1min":0.72799999999999998,"5min":0.95399999999999996,"15min":1.1439999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.45500000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.38100000000000001},"max":{"1min":0.90200000000000002,"5min":1.052,"15min":1.361},"last":0.77400000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.52300000000000002,"5min":0.46500000000000002,"15min":0.39900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":1.1950000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.72199999999999998},"min":{"1min":0.52000000000000002,"5min":0.504,"15min":0.41299999999999998},"max":{"1min":0.90100000000000002,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.66500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575033,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073608,"statfs":{"total":96636764160,"available":96331374592,"internally_reserved":0,"allocated":966656,"data_stored":485032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.42299999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.94999999999999996,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.752},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70899999999999996,"15min":0.70699999999999996},"min":{"1min":0.60599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.93500000000000005,"5min":0.96899999999999997,"15min":1},"last":0.85599999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.70599999999999996,"15min":0.70799999999999996},"min":{"1min":0.46600000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.876},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.48199999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.78700000000000003}]},{"osd":2,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.76400000000000001,"15min":0.76500000000000001},"min":{"1min":0.60299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76400000000000001,"15min":0.77600000000000002},"min":{"1min":0.67900000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.98699999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.94199999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72399999999999998,"15min":0.73099999999999998},"min":{"1min":0.56599999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.996,"5min":0.996,"15min":1.6419999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.754,"15min":0.749},"min":{"1min":0.623,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0189999999999999,"5min":1.117,"15min":1.117},"last":0.82299999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.62,"15min":0.621},"min":{"1min":0.42899999999999999,"5min":0.40000000000000002,"15min":0.38300000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.024},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61799999999999999,"15min":0.63800000000000001},"min":{"1min":0.47099999999999997,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":0.94199999999999995,"5min":1.044,"15min":1.175},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.34999999999999998},"max":{"1min":1.0449999999999999,"5min":1.591,"15min":1.591},"last":0.628},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69599999999999995,"15min":0.69899999999999995},"min":{"1min":0.47799999999999998,"5min":0.44600000000000001,"15min":0.40600000000000003},"max":{"1min":1.0389999999999999,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67000000000000004,"15min":0.68100000000000005},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":0.97199999999999998,"5min":1.016,"15min":1.2589999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.72399999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.435},"max":{"1min":1.0880000000000001,"5min":1.278,"15min":1.278},"last":0.73699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:48:32.737 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:48:32.737 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:48:32.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.962+0000 7fcd56fff700 1 Processor -- start 2023-10-06T03:48:32.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.963+0000 7fcd56fff700 1 -- start start 2023-10-06T03:48:32.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.964+0000 7fcd56fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd50068aa0 0x7fcd500c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:32.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.964+0000 7fcd56fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5015a6f0 con 0x7fcd501569f0 2023-10-06T03:48:32.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.964+0000 7fcd56fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5015a9d0 con 0x7fcd50156620 2023-10-06T03:48:32.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.964+0000 7fcd56fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500c8d30 con 0x7fcd50068aa0 2023-10-06T03:48:32.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.964+0000 7fcd4ffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcd501569f0 0x7fcd50068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55844/0 (socket says 172.21.15.55:55844) 2023-10-06T03:48:32.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.964+0000 7fcd4ffff700 1 -- 172.21.15.55:0/3427301016 learned_addr learned my addr 172.21.15.55:0/3427301016 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:32.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.964+0000 7fcd5559c700 1 --2- 172.21.15.55:0/3427301016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd50068aa0 0x7fcd500c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:32.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.964+0000 7fcd4f7fe700 1 -- 172.21.15.55:0/3427301016 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77762380 0 0) 0x7fcd5015a6f0 con 0x7fcd501569f0 2023-10-06T03:48:32.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.964+0000 7fcd4f7fe700 1 -- 172.21.15.55:0/3427301016 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd40003570 con 0x7fcd501569f0 2023-10-06T03:48:32.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd4f7fe700 1 -- 172.21.15.55:0/3427301016 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1452369704 0 0) 0x7fcd40003570 con 0x7fcd501569f0 2023-10-06T03:48:32.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd4f7fe700 1 -- 172.21.15.55:0/3427301016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd50068aa0 msgr2=0x7fcd500c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:32.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd4f7fe700 1 --2- 172.21.15.55:0/3427301016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd50068aa0 0x7fcd500c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:32.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd4f7fe700 1 -- 172.21.15.55:0/3427301016 >> v1:172.21.15.163:6789/0 conn(0x7fcd50156620 legacy=0x7fcd500ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:32.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd4f7fe700 1 -- 172.21.15.55:0/3427301016 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5015acb0 con 0x7fcd501569f0 2023-10-06T03:48:32.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd4f7fe700 1 -- 172.21.15.55:0/3427301016 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fcd3c001470 con 0x7fcd501569f0 2023-10-06T03:48:32.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd4f7fe700 1 -- 172.21.15.55:0/3427301016 >> v1:172.21.15.55:6789/0 conn(0x7fcd501569f0 legacy=0x7fcd50068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:32.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd4f7fe700 1 --2- 172.21.15.55:0/3427301016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd40003a90 0x7fcd40003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:32.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd4f7fe700 1 --2- 172.21.15.55:0/3427301016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd40004810 0x7fcd40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:32.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd4f7fe700 1 --2- 172.21.15.55:0/3427301016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd50068aa0 0x7fcd400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:32.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd4f7fe700 1 -- 172.21.15.55:0/3427301016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500c8d30 con 0x7fcd40003a90 2023-10-06T03:48:32.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd4f7fe700 1 -- 172.21.15.55:0/3427301016 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4000dd20 con 0x7fcd40004810 2023-10-06T03:48:32.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd4f7fe700 1 -- 172.21.15.55:0/3427301016 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4000dee0 con 0x7fcd50068aa0 2023-10-06T03:48:32.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd54d9b700 1 --2- 172.21.15.55:0/3427301016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd40003a90 0x7fcd40003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:32.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd4ffff700 1 --2- 172.21.15.55:0/3427301016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd40004810 0x7fcd40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:32.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.965+0000 7fcd5559c700 1 --2- 172.21.15.55:0/3427301016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd50068aa0 0x7fcd400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:32.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.966+0000 7fcd54d9b700 1 -- 172.21.15.55:0/3427301016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd50068aa0 msgr2=0x7fcd400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:32.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.966+0000 7fcd54d9b700 1 --2- 172.21.15.55:0/3427301016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd50068aa0 0x7fcd400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:32.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.966+0000 7fcd54d9b700 1 -- 172.21.15.55:0/3427301016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd40004810 msgr2=0x7fcd40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:32.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.966+0000 7fcd54d9b700 1 --2- 172.21.15.55:0/3427301016 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd40004810 0x7fcd40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:32.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.966+0000 7fcd54d9b700 1 -- 172.21.15.55:0/3427301016 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd40003570 con 0x7fcd40003a90 2023-10-06T03:48:32.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.966+0000 7fcd5559c700 1 --2- 172.21.15.55:0/3427301016 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd50068aa0 0x7fcd400051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:32.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.966+0000 7fcd54d9b700 1 --2- 172.21.15.55:0/3427301016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd40003a90 0x7fcd40003e60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fcd44001960 tx=0x7fcd44007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc311dd8287b8a60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:32.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.966+0000 7fcd4f7fe700 1 -- 172.21.15.55:0/3427301016 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4400c5d0 con 0x7fcd40003a90 2023-10-06T03:48:32.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.966+0000 7fcd4f7fe700 1 -- 172.21.15.55:0/3427301016 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4400c730 con 0x7fcd40003a90 2023-10-06T03:48:32.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.966+0000 7fcd4f7fe700 1 -- 172.21.15.55:0/3427301016 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4400cad0 con 0x7fcd40003a90 2023-10-06T03:48:32.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.967+0000 7fcd56fff700 1 -- 172.21.15.55:0/3427301016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd40003a90 msgr2=0x7fcd40003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:32.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.967+0000 7fcd56fff700 1 --2- 172.21.15.55:0/3427301016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd40003a90 0x7fcd40003e60 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fcd44001960 tx=0x7fcd44007d10 comp rx=0 tx=0).stop 2023-10-06T03:48:32.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.967+0000 7fcd5559c700 1 -- 172.21.15.55:0/3427301016 reap_dead start 2023-10-06T03:48:32.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.967+0000 7fcd56fff700 1 -- 172.21.15.55:0/3427301016 shutdown_connections 2023-10-06T03:48:32.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.967+0000 7fcd56fff700 1 -- 172.21.15.55:0/3427301016 >> 172.21.15.55:0/3427301016 conn(0x7fcd500cc540 msgr2=0x7fcd500cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:32.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.967+0000 7fcd56fff700 1 -- 172.21.15.55:0/3427301016 shutdown_connections 2023-10-06T03:48:32.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.967+0000 7fcd56fff700 1 -- 172.21.15.55:0/3427301016 wait complete. 2023-10-06T03:48:32.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.968+0000 7fcd56fff700 1 Processor -- start 2023-10-06T03:48:32.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.968+0000 7fcd56fff700 1 -- start start 2023-10-06T03:48:32.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.969+0000 7fcd56fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd40003a90 0x7fcd500c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:32.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.969+0000 7fcd56fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd500c6190 0x7fcd500c3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:32.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.969+0000 7fcd56fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd500c3a90 0x7fcd500c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:32.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.970+0000 7fcd56fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd50154c50 con 0x7fcd500c6190 2023-10-06T03:48:32.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.970+0000 7fcd56fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500bf840 con 0x7fcd40003a90 2023-10-06T03:48:32.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.970+0000 7fcd56fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd50154040 con 0x7fcd500c3a90 2023-10-06T03:48:32.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.970+0000 7fcd4ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd500c6190 0x7fcd500c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:32.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.970+0000 7fcd4ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd500c6190 0x7fcd500c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48892/0 (socket says 172.21.15.55:48892) 2023-10-06T03:48:32.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.970+0000 7fcd4ffff700 1 -- 172.21.15.55:0/926904296 learned_addr learned my addr 172.21.15.55:0/926904296 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:32.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.970+0000 7fcd4ffff700 1 -- 172.21.15.55:0/926904296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd500c3a90 msgr2=0x7fcd500c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:32.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.970+0000 7fcd4ffff700 1 --2- 172.21.15.55:0/926904296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd500c3a90 0x7fcd500c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.970+0000 7fcd4ffff700 1 -- 172.21.15.55:0/926904296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd40003a90 msgr2=0x7fcd500c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.970+0000 7fcd4ffff700 1 --2- 172.21.15.55:0/926904296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd40003a90 0x7fcd500c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.970+0000 7fcd4ffff700 1 -- 172.21.15.55:0/926904296 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50154320 con 0x7fcd500c6190 2023-10-06T03:48:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.970+0000 7fcd4ffff700 1 --2- 172.21.15.55:0/926904296 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd500c6190 0x7fcd500c3550 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c007390 tx=0x7fcd3c001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.971+0000 7fcd4cff9700 1 -- 172.21.15.55:0/926904296 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c018030 con 0x7fcd500c6190 2023-10-06T03:48:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.971+0000 7fcd4cff9700 1 -- 172.21.15.55:0/926904296 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3c00f8d0 con 0x7fcd500c6190 2023-10-06T03:48:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.971+0000 7fcd56fff700 1 -- 172.21.15.55:0/926904296 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd50154600 con 0x7fcd500c6190 2023-10-06T03:48:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.971+0000 7fcd4cff9700 1 -- 172.21.15.55:0/926904296 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3c00fc90 con 0x7fcd500c6190 2023-10-06T03:48:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.971+0000 7fcd56fff700 1 -- 172.21.15.55:0/926904296 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd50153d30 con 0x7fcd500c6190 2023-10-06T03:48:32.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.972+0000 7fcd56fff700 1 -- 172.21.15.55:0/926904296 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd500718b0 con 0x7fcd500c6190 2023-10-06T03:48:32.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.973+0000 7fcd4cff9700 1 -- 172.21.15.55:0/926904296 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd3c01b070 con 0x7fcd500c6190 2023-10-06T03:48:32.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.973+0000 7fcd4cff9700 1 --2- 172.21.15.55:0/926904296 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd34066f30 0x7fcd340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:32.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.974+0000 7fcd4cff9700 1 -- 172.21.15.55:0/926904296 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fcd3c019070 con 0x7fcd500c6190 2023-10-06T03:48:32.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.974+0000 7fcd54d9b700 1 --2- 172.21.15.55:0/926904296 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd34066f30 0x7fcd340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:32.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.975+0000 7fcd54d9b700 1 --2- 172.21.15.55:0/926904296 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd34066f30 0x7fcd340693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcd44008fd0 tx=0x7fcd44015000 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:32.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:32.977+0000 7fcd4cff9700 1 -- 172.21.15.55:0/926904296 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd3c0516d0 con 0x7fcd500c6190 2023-10-06T03:48:33.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.262+0000 7fcd56fff700 1 -- 172.21.15.55:0/926904296 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcd500c7bc0 con 0x7fcd500c6190 2023-10-06T03:48:33.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.267+0000 7fcd4cff9700 1 -- 172.21.15.55:0/926904296 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7fcd3c00fdf0 con 0x7fcd500c6190 2023-10-06T03:48:33.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.270+0000 7fcd56fff700 1 -- 172.21.15.55:0/926904296 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd34066f30 msgr2=0x7fcd340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:33.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.270+0000 7fcd56fff700 1 --2- 172.21.15.55:0/926904296 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd34066f30 0x7fcd340693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcd44008fd0 tx=0x7fcd44015000 comp rx=0 tx=0).stop 2023-10-06T03:48:33.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.270+0000 7fcd56fff700 1 -- 172.21.15.55:0/926904296 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd500c6190 msgr2=0x7fcd500c3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:33.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.270+0000 7fcd56fff700 1 --2- 172.21.15.55:0/926904296 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd500c6190 0x7fcd500c3550 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c007390 tx=0x7fcd3c001df0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.270+0000 7fcd56fff700 1 -- 172.21.15.55:0/926904296 shutdown_connections 2023-10-06T03:48:33.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.270+0000 7fcd56fff700 1 --2- 172.21.15.55:0/926904296 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd34066f30 0x7fcd340693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.270+0000 7fcd56fff700 1 --2- 172.21.15.55:0/926904296 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd500c3a90 0x7fcd500c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.270+0000 7fcd56fff700 1 --2- 172.21.15.55:0/926904296 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd40003a90 0x7fcd500c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.270+0000 7fcd56fff700 1 --2- 172.21.15.55:0/926904296 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd500c6190 0x7fcd500c3550 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.270+0000 7fcd56fff700 1 -- 172.21.15.55:0/926904296 >> 172.21.15.55:0/926904296 conn(0x7fcd500cc540 msgr2=0x7fcd50069890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:33.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.270+0000 7fcd56fff700 1 -- 172.21.15.55:0/926904296 shutdown_connections 2023-10-06T03:48:33.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.270+0000 7fcd56fff700 1 -- 172.21.15.55:0/926904296 wait complete. 2023-10-06T03:48:33.273 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2023-10-06T03:48:33.289 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:48:33.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.474+0000 7fbecb59e700 1 Processor -- start 2023-10-06T03:48:33.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.475+0000 7fbecb59e700 1 -- start start 2023-10-06T03:48:33.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.476+0000 7fbecb59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbecc1566e0 0x7fbecc0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:33.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.476+0000 7fbecb59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc0ca860 con 0x7fbecc156310 2023-10-06T03:48:33.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.476+0000 7fbecb59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc0cab40 con 0x7fbecc0cb900 2023-10-06T03:48:33.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.476+0000 7fbecb59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0cae20 con 0x7fbecc1566e0 2023-10-06T03:48:33.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.476+0000 7fbeca59c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbecc156310 0x7fbecc0c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46642/0 (socket says 172.21.15.55:46642) 2023-10-06T03:48:33.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.476+0000 7fbeca59c700 1 -- 172.21.15.55:0/2389474042 learned_addr learned my addr 172.21.15.55:0/2389474042 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:33.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.476+0000 7fbec9d9b700 1 --2- 172.21.15.55:0/2389474042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbecc1566e0 0x7fbecc0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:33.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898467067 0 0) 0x7fbecc0ca860 con 0x7fbecc156310 2023-10-06T03:48:33.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeb4003570 con 0x7fbecc156310 2023-10-06T03:48:33.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738037509 0 0) 0x7fbecc0cab40 con 0x7fbecc0cb900 2023-10-06T03:48:33.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbecc0ca860 con 0x7fbecc0cb900 2023-10-06T03:48:33.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2208304693 0 0) 0x7fbeb4003570 con 0x7fbecc156310 2023-10-06T03:48:33.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbecc1566e0 msgr2=0x7fbecc0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:33.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 --2- 172.21.15.55:0/2389474042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbecc1566e0 0x7fbecc0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 >> v1:172.21.15.163:6789/0 conn(0x7fbecc0cb900 legacy=0x7fbecc0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:33.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc0cb100 con 0x7fbecc156310 2023-10-06T03:48:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fbebc001470 con 0x7fbecc156310 2023-10-06T03:48:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 >> v1:172.21.15.55:6789/0 conn(0x7fbecc156310 legacy=0x7fbecc0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 --2- 172.21.15.55:0/2389474042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbecc1566e0 0x7fbeb4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 --2- 172.21.15.55:0/2389474042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbeb40041d0 0x7fbeb4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 --2- 172.21.15.55:0/2389474042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbeb4004d50 0x7fbeb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0cae20 con 0x7fbeb4004d50 2023-10-06T03:48:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb400bbf0 con 0x7fbeb40041d0 2023-10-06T03:48:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb400bdb0 con 0x7fbecc1566e0 2023-10-06T03:48:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbecad9d700 1 --2- 172.21.15.55:0/2389474042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbeb4004d50 0x7fbeb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.477+0000 7fbec9d9b700 1 --2- 172.21.15.55:0/2389474042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbeb40041d0 0x7fbeb4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.478+0000 7fbeca59c700 1 --2- 172.21.15.55:0/2389474042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbecc1566e0 0x7fbeb4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:33.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.478+0000 7fbecad9d700 1 -- 172.21.15.55:0/2389474042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbecc1566e0 msgr2=0x7fbeb4003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:33.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.478+0000 7fbecad9d700 1 --2- 172.21.15.55:0/2389474042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbecc1566e0 0x7fbeb4003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.478+0000 7fbecad9d700 1 -- 172.21.15.55:0/2389474042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbeb40041d0 msgr2=0x7fbeb4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:33.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.478+0000 7fbecad9d700 1 --2- 172.21.15.55:0/2389474042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbeb40041d0 0x7fbeb4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.478+0000 7fbecad9d700 1 -- 172.21.15.55:0/2389474042 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb4003570 con 0x7fbeb4004d50 2023-10-06T03:48:33.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.478+0000 7fbecad9d700 1 --2- 172.21.15.55:0/2389474042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbeb4004d50 0x7fbeb40051c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbec4002120 tx=0x7fbec40017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3f8b856ec56cc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:33.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.478+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4001ab0 con 0x7fbeb4004d50 2023-10-06T03:48:33.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.478+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec400c980 con 0x7fbeb4004d50 2023-10-06T03:48:33.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.478+0000 7fbec959a700 1 -- 172.21.15.55:0/2389474042 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec400cd20 con 0x7fbeb4004d50 2023-10-06T03:48:33.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.479+0000 7fbecb59e700 1 -- 172.21.15.55:0/2389474042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbeb4004d50 msgr2=0x7fbeb40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:33.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.479+0000 7fbecb59e700 1 --2- 172.21.15.55:0/2389474042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbeb4004d50 0x7fbeb40051c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbec4002120 tx=0x7fbec40017a0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.479+0000 7fbecad9d700 1 -- 172.21.15.55:0/2389474042 reap_dead start 2023-10-06T03:48:33.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.479+0000 7fbecb59e700 1 -- 172.21.15.55:0/2389474042 shutdown_connections 2023-10-06T03:48:33.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.479+0000 7fbecb59e700 1 -- 172.21.15.55:0/2389474042 >> 172.21.15.55:0/2389474042 conn(0x7fbecc0cc1f0 msgr2=0x7fbecc0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:33.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.479+0000 7fbecb59e700 1 -- 172.21.15.55:0/2389474042 shutdown_connections 2023-10-06T03:48:33.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.479+0000 7fbecb59e700 1 -- 172.21.15.55:0/2389474042 wait complete. 2023-10-06T03:48:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.480+0000 7fbecb59e700 1 Processor -- start 2023-10-06T03:48:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.481+0000 7fbecb59e700 1 -- start start 2023-10-06T03:48:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.481+0000 7fbecb59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbeb4004d50 0x7fbecc1509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:33.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.481+0000 7fbecb59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbecc14c3a0 0x7fbecc14e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:33.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.481+0000 7fbecb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbecc14f720 0x7fbecc14e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:33.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.481+0000 7fbecb59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc151ab0 con 0x7fbecc14f720 2023-10-06T03:48:33.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.481+0000 7fbecb59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc151210 con 0x7fbecc14c3a0 2023-10-06T03:48:33.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.481+0000 7fbecb59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc1514f0 con 0x7fbeb4004d50 2023-10-06T03:48:33.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.482+0000 7fbecad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbecc14f720 0x7fbecc14e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:33.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.482+0000 7fbecad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbecc14f720 0x7fbecc14e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46890/0 (socket says 172.21.15.55:46890) 2023-10-06T03:48:33.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.482+0000 7fbecad9d700 1 -- 172.21.15.55:0/1773875478 learned_addr learned my addr 172.21.15.55:0/1773875478 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:33.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.482+0000 7fbec9d9b700 1 --2- 172.21.15.55:0/1773875478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbecc14c3a0 0x7fbecc14e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:33.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.482+0000 7fbeca59c700 1 --2- 172.21.15.55:0/1773875478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbeb4004d50 0x7fbecc1509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:33.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.482+0000 7fbecad9d700 1 -- 172.21.15.55:0/1773875478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbeb4004d50 msgr2=0x7fbecc1509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:33.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.482+0000 7fbecad9d700 1 --2- 172.21.15.55:0/1773875478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbeb4004d50 0x7fbecc1509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.482+0000 7fbecad9d700 1 -- 172.21.15.55:0/1773875478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbecc14c3a0 msgr2=0x7fbecc14e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:33.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.482+0000 7fbecad9d700 1 --2- 172.21.15.55:0/1773875478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbecc14c3a0 0x7fbecc14e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.482+0000 7fbecad9d700 1 -- 172.21.15.55:0/1773875478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc1517d0 con 0x7fbecc14f720 2023-10-06T03:48:33.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.482+0000 7fbecad9d700 1 --2- 172.21.15.55:0/1773875478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbecc14f720 0x7fbecc14e800 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40057f0 tx=0x7fbec4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:33.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.483+0000 7fbeba7fc700 1 -- 172.21.15.55:0/1773875478 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec400c6d0 con 0x7fbecc14f720 2023-10-06T03:48:33.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.483+0000 7fbeba7fc700 1 -- 172.21.15.55:0/1773875478 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec4012e50 con 0x7fbecc14f720 2023-10-06T03:48:33.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.483+0000 7fbeba7fc700 1 -- 172.21.15.55:0/1773875478 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4006df0 con 0x7fbecc14f720 2023-10-06T03:48:33.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.483+0000 7fbecb59e700 1 -- 172.21.15.55:0/1773875478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc156dd0 con 0x7fbecc14f720 2023-10-06T03:48:33.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.483+0000 7fbecb59e700 1 -- 172.21.15.55:0/1773875478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc14ed40 con 0x7fbecc14f720 2023-10-06T03:48:33.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.484+0000 7fbecb59e700 1 -- 172.21.15.55:0/1773875478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbecc156c30 con 0x7fbecc14f720 2023-10-06T03:48:33.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.485+0000 7fbeba7fc700 1 -- 172.21.15.55:0/1773875478 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbec4003030 con 0x7fbecc14f720 2023-10-06T03:48:33.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.486+0000 7fbeba7fc700 1 --2- 172.21.15.55:0/1773875478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbeac066f30 0x7fbeac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:33.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.487+0000 7fbeba7fc700 1 -- 172.21.15.55:0/1773875478 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fbec4086ca0 con 0x7fbecc14f720 2023-10-06T03:48:33.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.487+0000 7fbeca59c700 1 --2- 172.21.15.55:0/1773875478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbeac066f30 0x7fbeac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:33.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.488+0000 7fbeca59c700 1 --2- 172.21.15.55:0/1773875478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbeac066f30 0x7fbeac0693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbebc001bb0 tx=0x7fbebc009040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:33.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.491+0000 7fbeba7fc700 1 -- 172.21.15.55:0/1773875478 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbecc156c30 con 0x7fbecc14f720 2023-10-06T03:48:33.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.737+0000 7fbecb59e700 1 -- 172.21.15.55:0/1773875478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbecc156c30 con 0x7fbecc14f720 2023-10-06T03:48:33.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.738+0000 7fbeba7fc700 1 -- 172.21.15.55:0/1773875478 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4601 (secure 0 0 0) 0x7fbecc156c30 con 0x7fbecc14f720 2023-10-06T03:48:33.740 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:33.740 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4046797285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":4046797285},{"type":"v1","addr":"172.21.15.55:6835","nonce":4046797285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5645,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/675796170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":675796170},{"type":"v1","addr":"172.21.15.163:6836","nonce":675796170}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5648,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2543497851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2543497851},{"type":"v1","addr":"172.21.15.163:6834","nonce":2543497851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:48:22.429407+0000","modified":"2023-10-06T03:48:25.973506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6333":{"gid":6333,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3434145175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3434145175},{"type":"v1","addr":"172.21.15.55:6834","nonce":3434145175}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T03:48:33.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.740+0000 7fbecb59e700 1 -- 172.21.15.55:0/1773875478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbeac066f30 msgr2=0x7fbeac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:33.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.740+0000 7fbecb59e700 1 --2- 172.21.15.55:0/1773875478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbeac066f30 0x7fbeac0693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbebc001bb0 tx=0x7fbebc009040 comp rx=0 tx=0).stop 2023-10-06T03:48:33.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.740+0000 7fbecb59e700 1 -- 172.21.15.55:0/1773875478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbecc14f720 msgr2=0x7fbecc14e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:33.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.740+0000 7fbecb59e700 1 --2- 172.21.15.55:0/1773875478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbecc14f720 0x7fbecc14e800 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbeb40057f0 tx=0x7fbec4016000 comp rx=0 tx=0).stop 2023-10-06T03:48:33.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.741+0000 7fbecb59e700 1 -- 172.21.15.55:0/1773875478 shutdown_connections 2023-10-06T03:48:33.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.741+0000 7fbecb59e700 1 --2- 172.21.15.55:0/1773875478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbeac066f30 0x7fbeac0693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.741+0000 7fbecb59e700 1 --2- 172.21.15.55:0/1773875478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbecc14f720 0x7fbecc14e800 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.741+0000 7fbecb59e700 1 --2- 172.21.15.55:0/1773875478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbeb4004d50 0x7fbecc1509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.741+0000 7fbecb59e700 1 --2- 172.21.15.55:0/1773875478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbecc14c3a0 0x7fbecc14e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:33.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.741+0000 7fbecb59e700 1 -- 172.21.15.55:0/1773875478 >> 172.21.15.55:0/1773875478 conn(0x7fbecc0cc1f0 msgr2=0x7fbecc0b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:33.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.741+0000 7fbecb59e700 1 -- 172.21.15.55:0/1773875478 shutdown_connections 2023-10-06T03:48:33.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:33.741+0000 7fbecb59e700 1 -- 172.21.15.55:0/1773875478 wait complete. 2023-10-06T03:48:33.744 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 35 2023-10-06T03:48:34.162 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:48:22.429407+0000', 'modified': '2023-10-06T03:48:25.973506+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6333}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6333': {'gid': 6333, 'name': 'a', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6834/3434145175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6832', 'nonce': 3434145175}, {'type': 'v1', 'addr': '172.21.15.55:6834', 'nonce': 3434145175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:48:34.163 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:48:34.163 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T03:48:34.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.353+0000 7fc740d9d700 1 Processor -- start 2023-10-06T03:48:34.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.354+0000 7fc740d9d700 1 -- start start 2023-10-06T03:48:34.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.355+0000 7fc740d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc73c06d950 0x7fc73c0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.355+0000 7fc740d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc73c06dd20 0x7fc73c0c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.355+0000 7fc740d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73c06ae70 con 0x7fc73c06a0a0 2023-10-06T03:48:34.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.355+0000 7fc740d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73c06a540 con 0x7fc73c06d950 2023-10-06T03:48:34.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.355+0000 7fc740d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73c06a820 con 0x7fc73c06dd20 2023-10-06T03:48:34.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.355+0000 7fc73b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc73c06d950 0x7fc73c0c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.355+0000 7fc73b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc73c06d950 0x7fc73c0c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46900/0 (socket says 172.21.15.55:46900) 2023-10-06T03:48:34.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.355+0000 7fc73b7fe700 1 -- 172.21.15.55:0/1212978407 learned_addr learned my addr 172.21.15.55:0/1212978407 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:34.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.355+0000 7fc73bfff700 1 --2- 172.21.15.55:0/1212978407 >> v2:172.21.15.163:3300/0 conn(0x7fc73c06dd20 0x7fc73c0c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.356+0000 7fc73b7fe700 1 -- 172.21.15.55:0/1212978407 >> v2:172.21.15.163:3300/0 conn(0x7fc73c06dd20 msgr2=0x7fc73c0c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:34.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.356+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/1212978407 >> v2:172.21.15.163:3300/0 conn(0x7fc73c06dd20 0x7fc73c0c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.356+0000 7fc73b7fe700 1 -- 172.21.15.55:0/1212978407 >> v1:172.21.15.163:6789/0 conn(0x7fc73c06a0a0 legacy=0x7fc73c0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.356+0000 7fc73b7fe700 1 -- 172.21.15.55:0/1212978407 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c06ab00 con 0x7fc73c06d950 2023-10-06T03:48:34.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.356+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/1212978407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73c06d950 0x7fc73c0c8350 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc73c06e420 tx=0x7fc72c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba9bdf93b4ab65f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:34.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.356+0000 7fc73a7fc700 1 -- 172.21.15.55:0/1212978407 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c009e70 con 0x7fc73c06d950 2023-10-06T03:48:34.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.356+0000 7fc73a7fc700 1 -- 172.21.15.55:0/1212978407 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c00b860 con 0x7fc73c06d950 2023-10-06T03:48:34.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.356+0000 7fc73a7fc700 1 -- 172.21.15.55:0/1212978407 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c00bc00 con 0x7fc73c06d950 2023-10-06T03:48:34.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.357+0000 7fc740d9d700 1 -- 172.21.15.55:0/1212978407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73c06d950 msgr2=0x7fc73c0c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.357+0000 7fc740d9d700 1 --2- 172.21.15.55:0/1212978407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73c06d950 0x7fc73c0c8350 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fc73c06e420 tx=0x7fc72c009250 comp rx=0 tx=0).stop 2023-10-06T03:48:34.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.357+0000 7fc740d9d700 1 -- 172.21.15.55:0/1212978407 shutdown_connections 2023-10-06T03:48:34.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.357+0000 7fc740d9d700 1 --2- 172.21.15.55:0/1212978407 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73c06d950 0x7fc73c0c8350 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.357+0000 7fc740d9d700 1 --2- 172.21.15.55:0/1212978407 >> v2:172.21.15.163:3300/0 conn(0x7fc73c06dd20 0x7fc73c0c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.357+0000 7fc740d9d700 1 -- 172.21.15.55:0/1212978407 >> 172.21.15.55:0/1212978407 conn(0x7fc73c0cc1f0 msgr2=0x7fc73c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:34.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.357+0000 7fc740d9d700 1 -- 172.21.15.55:0/1212978407 shutdown_connections 2023-10-06T03:48:34.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.357+0000 7fc740d9d700 1 -- 172.21.15.55:0/1212978407 wait complete. 2023-10-06T03:48:34.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.358+0000 7fc740d9d700 1 Processor -- start 2023-10-06T03:48:34.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.358+0000 7fc740d9d700 1 -- start start 2023-10-06T03:48:34.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.359+0000 7fc740d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc73c06a0a0 0x7fc73c150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.359+0000 7fc740d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73c06d950 0x7fc73c0c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.359+0000 7fc740d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc73c06dd20 0x7fc73c0c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.359+0000 7fc740d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c0b1090 con 0x7fc73c06d950 2023-10-06T03:48:34.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.359+0000 7fc740d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c158c30 con 0x7fc73c06dd20 2023-10-06T03:48:34.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.359+0000 7fc740d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c16da20 con 0x7fc73c06a0a0 2023-10-06T03:48:34.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.360+0000 7fc73affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73c06d950 0x7fc73c0c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.360+0000 7fc73affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73c06d950 0x7fc73c0c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46916/0 (socket says 172.21.15.55:46916) 2023-10-06T03:48:34.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.360+0000 7fc73affd700 1 -- 172.21.15.55:0/1080659339 learned_addr learned my addr 172.21.15.55:0/1080659339 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:34.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.360+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc73c06a0a0 0x7fc73c150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.360+0000 7fc73bfff700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc73c06dd20 0x7fc73c0c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.360+0000 7fc73affd700 1 -- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc73c06a0a0 msgr2=0x7fc73c150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.360+0000 7fc73affd700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc73c06a0a0 0x7fc73c150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.360+0000 7fc73affd700 1 -- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc73c06dd20 msgr2=0x7fc73c0c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.360+0000 7fc73affd700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc73c06dd20 0x7fc73c0c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.360+0000 7fc73affd700 1 -- 172.21.15.55:0/1080659339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72c0094f0 con 0x7fc73c06d950 2023-10-06T03:48:34.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.360+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc73c06a0a0 0x7fc73c150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:34.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.360+0000 7fc73affd700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73c06d950 0x7fc73c0c53d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc73000c590 tx=0x7fc73000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:34.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.361+0000 7fc7237fe700 1 -- 172.21.15.55:0/1080659339 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730011590 con 0x7fc73c06d950 2023-10-06T03:48:34.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.361+0000 7fc7237fe700 1 -- 172.21.15.55:0/1080659339 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc730011710 con 0x7fc73c06d950 2023-10-06T03:48:34.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.361+0000 7fc7237fe700 1 -- 172.21.15.55:0/1080659339 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730011ad0 con 0x7fc73c06d950 2023-10-06T03:48:34.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.361+0000 7fc740d9d700 1 -- 172.21.15.55:0/1080659339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73c16dd00 con 0x7fc73c06d950 2023-10-06T03:48:34.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.361+0000 7fc740d9d700 1 -- 172.21.15.55:0/1080659339 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73c16e2c0 con 0x7fc73c06d950 2023-10-06T03:48:34.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.363+0000 7fc7237fe700 1 -- 172.21.15.55:0/1080659339 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc73001b3b0 con 0x7fc73c06d950 2023-10-06T03:48:34.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.364+0000 7fc7237fe700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc728066f30 0x7fc7280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.364+0000 7fc7237fe700 1 -- 172.21.15.55:0/1080659339 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fc730083df0 con 0x7fc73c06d950 2023-10-06T03:48:34.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.364+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc728066f30 0x7fc7280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.365+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc728066f30 0x7fc7280693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc73c06e420 tx=0x7fc72c017000 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:34.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.366+0000 7fc750749b80 1 Processor -- start 2023-10-06T03:48:34.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.366+0000 7fc750749b80 1 -- start start 2023-10-06T03:48:34.369 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.367+0000 7fc750749b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 0x5560fb897cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.367+0000 7fc750749b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89d2c0 0x5560fb898200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.367+0000 7fc750749b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89ea10 0x5560fb898740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.367+0000 7fc750749b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560fb7f2a70 con 0x5560fb89ea10 2023-10-06T03:48:34.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.367+0000 7fc750749b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560fb7f0270 con 0x5560fb89d2c0 2023-10-06T03:48:34.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.367+0000 7fc750749b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560fb7ec060 con 0x5560fb8bce50 2023-10-06T03:48:34.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.367+0000 7fc73bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89ea10 0x5560fb898740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.367+0000 7fc73bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89ea10 0x5560fb898740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46922/0 (socket says 172.21.15.55:46922) 2023-10-06T03:48:34.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.367+0000 7fc73bfff700 1 -- 172.21.15.55:0/2161072785 learned_addr learned my addr 172.21.15.55:0/2161072785 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:34.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.367+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/2161072785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 0x5560fb897cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.367+0000 7fc73bfff700 1 -- 172.21.15.55:0/2161072785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 msgr2=0x5560fb897cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:34.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.367+0000 7fc73affd700 1 --2- 172.21.15.55:0/2161072785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89d2c0 0x5560fb898200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.367+0000 7fc73bfff700 1 --2- 172.21.15.55:0/2161072785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 0x5560fb897cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.367+0000 7fc73bfff700 1 -- 172.21.15.55:0/2161072785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89d2c0 msgr2=0x5560fb898200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:34.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.368+0000 7fc73bfff700 1 --2- 172.21.15.55:0/2161072785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89d2c0 0x5560fb898200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.368+0000 7fc73bfff700 1 -- 172.21.15.55:0/2161072785 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560fb686270 con 0x5560fb89ea10 2023-10-06T03:48:34.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.368+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/2161072785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 0x5560fb897cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:34.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.368+0000 7fc73affd700 1 --2- 172.21.15.55:0/2161072785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89d2c0 0x5560fb898200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:34.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.368+0000 7fc73bfff700 1 --2- 172.21.15.55:0/2161072785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89ea10 0x5560fb898740 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc734011500 tx=0x7fc73400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99753341551dbe63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:34.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.368+0000 7fc7217fa700 1 -- 172.21.15.55:0/2161072785 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73400e040 con 0x5560fb89ea10 2023-10-06T03:48:34.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.368+0000 7fc7217fa700 1 -- 172.21.15.55:0/2161072785 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73400f040 con 0x5560fb89ea10 2023-10-06T03:48:34.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.368+0000 7fc7217fa700 1 -- 172.21.15.55:0/2161072785 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc734015850 con 0x5560fb89ea10 2023-10-06T03:48:34.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.369+0000 7fc750749b80 1 -- 172.21.15.55:0/2161072785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89ea10 msgr2=0x5560fb898740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.369+0000 7fc750749b80 1 --2- 172.21.15.55:0/2161072785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89ea10 0x5560fb898740 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fc734011500 tx=0x7fc73400b040 comp rx=0 tx=0).stop 2023-10-06T03:48:34.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.369+0000 7fc750749b80 1 -- 172.21.15.55:0/2161072785 shutdown_connections 2023-10-06T03:48:34.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.369+0000 7fc750749b80 1 --2- 172.21.15.55:0/2161072785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89ea10 0x5560fb898740 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.369+0000 7fc750749b80 1 --2- 172.21.15.55:0/2161072785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 0x5560fb897cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.369+0000 7fc750749b80 1 --2- 172.21.15.55:0/2161072785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89d2c0 0x5560fb898200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.369+0000 7fc750749b80 1 -- 172.21.15.55:0/2161072785 >> 172.21.15.55:0/2161072785 conn(0x5560fb89ce80 msgr2=0x5560fb89d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:34.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.369+0000 7fc750749b80 1 -- 172.21.15.55:0/2161072785 shutdown_connections 2023-10-06T03:48:34.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.369+0000 7fc750749b80 1 -- 172.21.15.55:0/2161072785 wait complete. 2023-10-06T03:48:34.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.369+0000 7fc750749b80 1 -- start start 2023-10-06T03:48:34.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.370+0000 7fc750749b80 1 Processor -- start 2023-10-06T03:48:34.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc750749b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89d2c0 0x5560fb8e1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc750749b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89ea10 0x5560fb8e3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc750749b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 0x5560fb8ca290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc750749b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560fb797890 con 0x5560fb89d2c0 2023-10-06T03:48:34.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc750749b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560fb7b2ea0 con 0x5560fb89ea10 2023-10-06T03:48:34.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc750749b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560fb66ee30 con 0x5560fb8bce50 2023-10-06T03:48:34.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc750749b80 10 client.0 _ll_register_callbacks cb 0x5560fb85f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:48:34.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc73b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89d2c0 0x5560fb8e1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc73b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89d2c0 0x5560fb8e1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46934/0 (socket says 172.21.15.55:46934) 2023-10-06T03:48:34.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc73b7fe700 1 -- 172.21.15.55:0/1567720631 learned_addr learned my addr 172.21.15.55:0/1567720631 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:34.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc73bfff700 1 --2- 172.21.15.55:0/1567720631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 0x5560fb8ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc73affd700 1 --2- 172.21.15.55:0/1567720631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89ea10 0x5560fb8e3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc73b7fe700 1 -- 172.21.15.55:0/1567720631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 msgr2=0x5560fb8ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:34.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/1567720631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 0x5560fb8ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc73b7fe700 1 -- 172.21.15.55:0/1567720631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89ea10 msgr2=0x5560fb8e3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:34.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/1567720631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89ea10 0x5560fb8e3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc73b7fe700 1 -- 172.21.15.55:0/1567720631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc734011230 con 0x5560fb89d2c0 2023-10-06T03:48:34.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.371+0000 7fc73affd700 1 --2- 172.21.15.55:0/1567720631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89ea10 0x5560fb8e3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:34.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.372+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/1567720631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89d2c0 0x5560fb8e1fc0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc72c006e80 tx=0x7fc72c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc90e938e93f0836 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:34.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.372+0000 7fc719ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:48:34.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.372+0000 7fc719ffb700 1 -- 172.21.15.55:0/1567720631 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c02d070 con 0x5560fb89d2c0 2023-10-06T03:48:34.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.372+0000 7fc719ffb700 1 -- 172.21.15.55:0/1567720631 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fc72c011420 con 0x5560fb89d2c0 2023-10-06T03:48:34.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.372+0000 7fc750749b80 1 -- 172.21.15.55:0/1567720631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5560fb890940 con 0x5560fb89d2c0 2023-10-06T03:48:34.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.372+0000 7fc719ffb700 1 -- 172.21.15.55:0/1567720631 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc72c020070 con 0x5560fb89d2c0 2023-10-06T03:48:34.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.372+0000 7fc750749b80 10 client.6507 fetch_fsmap learned FSMap version 35 2023-10-06T03:48:34.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.372+0000 7fc750749b80 1 -- 172.21.15.55:0/1567720631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x5560fb628fe0 con 0x5560fb89d2c0 2023-10-06T03:48:34.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.372+0000 7fc719ffb700 1 -- 172.21.15.55:0/1567720631 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5560fb890940 con 0x5560fb89d2c0 2023-10-06T03:48:34.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.373+0000 7fc719ffb700 1 -- 172.21.15.55:0/1567720631 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 35) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc72c0117f0 con 0x5560fb89d2c0 2023-10-06T03:48:34.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.373+0000 7fc750749b80 10 client.6507 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T03:48:34.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.373+0000 7fc750749b80 10 client.6507 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:48:34.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.373+0000 7fc750749b80 20 client.6507 populate_metadata read hostname 'smithi055' 2023-10-06T03:48:34.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.373+0000 7fc750749b80 1 --2- 172.21.15.55:0/1567720631 >> [v2:172.21.15.55:6832/3434145175,v1:172.21.15.55:6834/3434145175] conn(0x5560fb8f5620 0x5560fb8f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.373+0000 7fc750749b80 4 client.6507 mds_command: new command op to 6333 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:48:34.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.373+0000 7fc750749b80 1 -- 172.21.15.55:0/1567720631 --> [v2:172.21.15.55:6832/3434145175,v1:172.21.15.55:6834/3434145175] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5560fb86d870 con 0x5560fb8f5620 2023-10-06T03:48:34.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.373+0000 7fc73affd700 1 --2- 172.21.15.55:0/1567720631 >> [v2:172.21.15.55:6832/3434145175,v1:172.21.15.55:6834/3434145175] conn(0x5560fb8f5620 0x5560fb8f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.373+0000 7fc73affd700 1 --2- 172.21.15.55:0/1567720631 >> [v2:172.21.15.55:6832/3434145175,v1:172.21.15.55:6834/3434145175] conn(0x5560fb8f5620 0x5560fb8f59f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc4080cd6cb6614 server_cookie=3bdf784ba9286aaf in_seq=0 out_seq=0 2023-10-06T03:48:34.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.373+0000 7fc719ffb700 10 client.6507 ms_handle_connect on v2:172.21.15.55:6832/3434145175 2023-10-06T03:48:34.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.374+0000 7fc719ffb700 1 -- 172.21.15.55:0/1567720631 <== mds.0 v2:172.21.15.55:6832/3434145175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5560fb86d870 con 0x5560fb8f5620 2023-10-06T03:48:34.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.374+0000 7fc719ffb700 10 client.6507 handle_command_reply: tid=0 2023-10-06T03:48:34.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.374+0000 7fc750749b80 1 client.6507 shutdown 2023-10-06T03:48:34.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.374+0000 7fc750749b80 1 -- 172.21.15.55:0/1567720631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89d2c0 msgr2=0x5560fb8e1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.374+0000 7fc750749b80 1 --2- 172.21.15.55:0/1567720631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89d2c0 0x5560fb8e1fc0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc72c006e80 tx=0x7fc72c020000 comp rx=0 tx=0).stop 2023-10-06T03:48:34.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.374+0000 7fc750749b80 1 -- 172.21.15.55:0/1567720631 shutdown_connections 2023-10-06T03:48:34.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.374+0000 7fc750749b80 1 --2- 172.21.15.55:0/1567720631 >> [v2:172.21.15.55:6832/3434145175,v1:172.21.15.55:6834/3434145175] conn(0x5560fb8f5620 0x5560fb8f59f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.375+0000 7fc750749b80 1 --2- 172.21.15.55:0/1567720631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 0x5560fb8ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.375+0000 7fc750749b80 1 --2- 172.21.15.55:0/1567720631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89d2c0 0x5560fb8e1fc0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.375+0000 7fc719ffb700 0 client.6507 ms_handle_reset on v2:172.21.15.55:6832/3434145175 2023-10-06T03:48:34.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.375+0000 7fc750749b80 1 --2- 172.21.15.55:0/1567720631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89ea10 0x5560fb8e3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.375+0000 7fc750749b80 1 -- 172.21.15.55:0/1567720631 >> 172.21.15.55:0/1567720631 conn(0x5560fb89ce80 msgr2=0x5560fb89dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:34.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.375+0000 7fc750749b80 1 -- 172.21.15.55:0/1567720631 shutdown_connections 2023-10-06T03:48:34.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.375+0000 7fc750749b80 1 -- 172.21.15.55:0/1567720631 wait complete. 2023-10-06T03:48:34.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.375+0000 7fc750749b80 20 client.6507 trim_cache size 0 max 16384 2023-10-06T03:48:34.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.392+0000 7fc750749b80 1 Processor -- start 2023-10-06T03:48:34.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.392+0000 7fc750749b80 1 -- start start 2023-10-06T03:48:34.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.393+0000 7fc750749b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89d2c0 0x5560fb8f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.393+0000 7fc750749b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89ea10 0x5560fb8e28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.393+0000 7fc750749b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 0x5560fb8e2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.393+0000 7fc750749b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560fb894280 con 0x5560fb89ea10 2023-10-06T03:48:34.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.393+0000 7fc750749b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560fb8cbae0 con 0x5560fb89d2c0 2023-10-06T03:48:34.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.393+0000 7fc750749b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560fb8ca1f0 con 0x5560fb8bce50 2023-10-06T03:48:34.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.394+0000 7fc73affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89ea10 0x5560fb8e28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.394+0000 7fc73affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89ea10 0x5560fb8e28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46944/0 (socket says 172.21.15.55:46944) 2023-10-06T03:48:34.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.394+0000 7fc73affd700 1 -- 172.21.15.55:0/4171492520 learned_addr learned my addr 172.21.15.55:0/4171492520 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:34.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.394+0000 7fc73bfff700 1 --2- 172.21.15.55:0/4171492520 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 0x5560fb8e2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.394+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/4171492520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89d2c0 0x5560fb8f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.394+0000 7fc73affd700 1 -- 172.21.15.55:0/4171492520 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 msgr2=0x5560fb8e2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:34.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.394+0000 7fc73affd700 1 --2- 172.21.15.55:0/4171492520 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 0x5560fb8e2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.394+0000 7fc73affd700 1 -- 172.21.15.55:0/4171492520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89d2c0 msgr2=0x5560fb8f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:34.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.394+0000 7fc73affd700 1 --2- 172.21.15.55:0/4171492520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89d2c0 0x5560fb8f47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.394+0000 7fc73affd700 1 -- 172.21.15.55:0/4171492520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560fb7ba780 con 0x5560fb89ea10 2023-10-06T03:48:34.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.394+0000 7fc73affd700 1 --2- 172.21.15.55:0/4171492520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89ea10 0x5560fb8e28c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc730081f80 tx=0x7fc730081ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c086d8a6aa2521 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:34.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.394+0000 7fc721ffb700 1 -- 172.21.15.55:0/4171492520 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730057be0 con 0x5560fb89ea10 2023-10-06T03:48:34.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.395+0000 7fc721ffb700 1 -- 172.21.15.55:0/4171492520 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73005b930 con 0x5560fb89ea10 2023-10-06T03:48:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.395+0000 7fc721ffb700 1 -- 172.21.15.55:0/4171492520 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730011e80 con 0x5560fb89ea10 2023-10-06T03:48:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.395+0000 7fc750749b80 1 -- 172.21.15.55:0/4171492520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89ea10 msgr2=0x5560fb8e28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.395+0000 7fc750749b80 1 --2- 172.21.15.55:0/4171492520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89ea10 0x5560fb8e28c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc730081f80 tx=0x7fc730081ee0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.395+0000 7fc750749b80 1 -- 172.21.15.55:0/4171492520 shutdown_connections 2023-10-06T03:48:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.395+0000 7fc750749b80 1 --2- 172.21.15.55:0/4171492520 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb8bce50 0x5560fb8e2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.395+0000 7fc750749b80 1 --2- 172.21.15.55:0/4171492520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb89d2c0 0x5560fb8f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.395+0000 7fc750749b80 1 --2- 172.21.15.55:0/4171492520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89ea10 0x5560fb8e28c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.395+0000 7fc750749b80 1 -- 172.21.15.55:0/4171492520 >> 172.21.15.55:0/4171492520 conn(0x5560fb89ce80 msgr2=0x5560fb899ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.395+0000 7fc750749b80 1 -- 172.21.15.55:0/4171492520 shutdown_connections 2023-10-06T03:48:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.395+0000 7fc750749b80 1 -- 172.21.15.55:0/4171492520 wait complete. 2023-10-06T03:48:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.396+0000 7fc750749b80 1 -- start start 2023-10-06T03:48:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.396+0000 7fc750749b80 1 Processor -- start 2023-10-06T03:48:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.397+0000 7fc750749b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89d2c0 0x5560fb89df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.397+0000 7fc750749b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb89ea10 0x5560fb93a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.397+0000 7fc750749b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb8bce50 0x5560fb8e3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.397+0000 7fc750749b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560fb897ca0 con 0x5560fb89d2c0 2023-10-06T03:48:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.397+0000 7fc750749b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5560fb93ac90 con 0x5560fb8bce50 2023-10-06T03:48:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.397+0000 7fc750749b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5560fb89daa0 con 0x5560fb89ea10 2023-10-06T03:48:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.397+0000 7fc750749b80 10 client.0 _ll_register_callbacks cb 0x5560fb85f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:48:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.397+0000 7fc73b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89d2c0 0x5560fb89df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.397+0000 7fc73b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89d2c0 0x5560fb89df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46948/0 (socket says 172.21.15.55:46948) 2023-10-06T03:48:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.397+0000 7fc73b7fe700 1 -- 172.21.15.55:0/3358510917 learned_addr learned my addr 172.21.15.55:0/3358510917 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.397+0000 7fc73bfff700 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb8bce50 0x5560fb8e3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.397+0000 7fc73affd700 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb89ea10 0x5560fb93a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc73b7fe700 1 -- 172.21.15.55:0/3358510917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb89ea10 msgr2=0x5560fb93a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb89ea10 0x5560fb93a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc73b7fe700 1 -- 172.21.15.55:0/3358510917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb8bce50 msgr2=0x5560fb8e3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb8bce50 0x5560fb8e3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc73b7fe700 1 -- 172.21.15.55:0/3358510917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73c16e2c0 con 0x5560fb89d2c0 2023-10-06T03:48:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc73affd700 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb89ea10 0x5560fb93a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc73bfff700 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb8bce50 0x5560fb8e3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc73b7fe700 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89d2c0 0x5560fb89df30 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0030c0 tx=0x7fc72c02a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe71b5ecbcfab4a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc719ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:48:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc719ffb700 1 -- 172.21.15.55:0/3358510917 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72c020040 con 0x5560fb89d2c0 2023-10-06T03:48:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc719ffb700 1 -- 172.21.15.55:0/3358510917 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fc72c011420 con 0x5560fb89d2c0 2023-10-06T03:48:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc750749b80 1 -- 172.21.15.55:0/3358510917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc73c0c3c40 con 0x5560fb89d2c0 2023-10-06T03:48:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc719ffb700 1 -- 172.21.15.55:0/3358510917 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc72c02d040 con 0x5560fb89d2c0 2023-10-06T03:48:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc750749b80 10 client.6513 fetch_fsmap learned FSMap version 35 2023-10-06T03:48:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.398+0000 7fc750749b80 1 -- 172.21.15.55:0/3358510917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x5560fb7a1470 con 0x5560fb89d2c0 2023-10-06T03:48:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.399+0000 7fc719ffb700 1 -- 172.21.15.55:0/3358510917 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72c04d020 con 0x5560fb89d2c0 2023-10-06T03:48:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.399+0000 7fc719ffb700 1 -- 172.21.15.55:0/3358510917 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 35) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fc72c011c60 con 0x5560fb89d2c0 2023-10-06T03:48:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.399+0000 7fc750749b80 10 client.6513 fetch_fsmap finished waiting for FSMap version 35 2023-10-06T03:48:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.399+0000 7fc750749b80 10 client.6513 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:48:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.399+0000 7fc750749b80 20 client.6513 populate_metadata read hostname 'smithi055' 2023-10-06T03:48:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.399+0000 7fc750749b80 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.55:6832/3434145175,v1:172.21.15.55:6834/3434145175] conn(0x5560fb93ee10 0x5560fb8e4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.399+0000 7fc750749b80 4 client.6513 mds_command: new command op to 6333 tid=0[{"prefix": "status"}] 2023-10-06T03:48:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.399+0000 7fc750749b80 1 -- 172.21.15.55:0/3358510917 --> [v2:172.21.15.55:6832/3434145175,v1:172.21.15.55:6834/3434145175] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fc73400f040 con 0x5560fb93ee10 2023-10-06T03:48:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.399+0000 7fc73affd700 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.55:6832/3434145175,v1:172.21.15.55:6834/3434145175] conn(0x5560fb93ee10 0x5560fb8e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc73affd700 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.55:6832/3434145175,v1:172.21.15.55:6834/3434145175] conn(0x5560fb93ee10 0x5560fb8e4090 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc33abaed7ccfbd3 server_cookie=f18939cf8a43ae5f in_seq=0 out_seq=0 2023-10-06T03:48:34.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc719ffb700 10 client.6513 ms_handle_connect on v2:172.21.15.55:6832/3434145175 2023-10-06T03:48:34.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc719ffb700 1 -- 172.21.15.55:0/3358510917 <== mds.0 v2:172.21.15.55:6832/3434145175 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x7fc73400f040 con 0x5560fb93ee10 2023-10-06T03:48:34.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc719ffb700 10 client.6513 handle_command_reply: tid=0 2023-10-06T03:48:34.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc750749b80 1 client.6513 shutdown 2023-10-06T03:48:34.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc750749b80 1 -- 172.21.15.55:0/3358510917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89d2c0 msgr2=0x5560fb89df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc750749b80 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89d2c0 0x5560fb89df30 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc72c0030c0 tx=0x7fc72c02a050 comp rx=0 tx=0).stop 2023-10-06T03:48:34.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc750749b80 1 -- 172.21.15.55:0/3358510917 shutdown_connections 2023-10-06T03:48:34.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc750749b80 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.55:6832/3434145175,v1:172.21.15.55:6834/3434145175] conn(0x5560fb93ee10 0x5560fb8e4090 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc750749b80 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5560fb8bce50 0x5560fb8e3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc750749b80 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5560fb89d2c0 0x5560fb89df30 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc719ffb700 0 client.6513 ms_handle_reset on v2:172.21.15.55:6832/3434145175 2023-10-06T03:48:34.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc750749b80 1 --2- 172.21.15.55:0/3358510917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5560fb89ea10 0x5560fb93a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc750749b80 1 -- 172.21.15.55:0/3358510917 >> 172.21.15.55:0/3358510917 conn(0x5560fb89ce80 msgr2=0x5560fb8fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:34.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc750749b80 1 -- 172.21.15.55:0/3358510917 shutdown_connections 2023-10-06T03:48:34.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc750749b80 1 -- 172.21.15.55:0/3358510917 wait complete. 2023-10-06T03:48:34.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.400+0000 7fc750749b80 20 client.6513 trim_cache size 0 max 16384 2023-10-06T03:48:34.408 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:48:34.408 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "8132c8ba-ee1c-41a2-b781-913925034ab6", 2023-10-06T03:48:34.408 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2023-10-06T03:48:34.408 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 6333, 2023-10-06T03:48:34.409 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2023-10-06T03:48:34.409 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2023-10-06T03:48:34.409 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2023-10-06T03:48:34.409 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 11.438850929999999, 2023-10-06T03:48:34.409 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 35, 2023-10-06T03:48:34.409 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 80, 2023-10-06T03:48:34.409 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 80, 2023-10-06T03:48:34.409 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 17.215259923000001 2023-10-06T03:48:34.409 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:48:34.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.401+0000 7fc740d9d700 1 -- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc728066f30 msgr2=0x7fc7280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.401+0000 7fc740d9d700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc728066f30 0x7fc7280693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc73c06e420 tx=0x7fc72c017000 comp rx=0 tx=0).stop 2023-10-06T03:48:34.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.401+0000 7fc740d9d700 1 -- 172.21.15.55:0/1080659339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73c06d950 msgr2=0x7fc73c0c53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.401+0000 7fc740d9d700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73c06d950 0x7fc73c0c53d0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fc73000c590 tx=0x7fc73000c8a0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.401+0000 7fc740d9d700 1 -- 172.21.15.55:0/1080659339 shutdown_connections 2023-10-06T03:48:34.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.401+0000 7fc740d9d700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc728066f30 0x7fc7280693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.401+0000 7fc740d9d700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc73c06dd20 0x7fc73c0c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.401+0000 7fc740d9d700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc73c06a0a0 0x7fc73c150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.401+0000 7fc740d9d700 1 --2- 172.21.15.55:0/1080659339 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc73c06d950 0x7fc73c0c53d0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.401+0000 7fc740d9d700 1 -- 172.21.15.55:0/1080659339 >> 172.21.15.55:0/1080659339 conn(0x7fc73c0cc1f0 msgr2=0x7fc73c0c9ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:34.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.401+0000 7fc740d9d700 1 -- 172.21.15.55:0/1080659339 shutdown_connections 2023-10-06T03:48:34.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.401+0000 7fc740d9d700 1 -- 172.21.15.55:0/1080659339 wait complete. 2023-10-06T03:48:34.419 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:48:34.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.605+0000 7f972759e700 1 Processor -- start 2023-10-06T03:48:34.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.605+0000 7f972759e700 1 -- start start 2023-10-06T03:48:34.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.606+0000 7f972759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9728142ea0 0x7f97280b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.606+0000 7f972759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97280b40f0 con 0x7f972806c0f0 2023-10-06T03:48:34.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.606+0000 7f972759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9728069640 con 0x7f97280ad870 2023-10-06T03:48:34.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.606+0000 7f972759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9728069920 con 0x7f9728142ea0 2023-10-06T03:48:34.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.606+0000 7f9726d9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f972806c0f0 0x7f972806a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46650/0 (socket says 172.21.15.55:46650) 2023-10-06T03:48:34.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.606+0000 7f9726d9d700 1 -- 172.21.15.55:0/1422876031 learned_addr learned my addr 172.21.15.55:0/1422876031 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:34.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.607+0000 7f972659c700 1 --2- 172.21.15.55:0/1422876031 >> v2:172.21.15.163:3300/0 conn(0x7f9728142ea0 0x7f97280b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.607+0000 7f972559a700 1 -- 172.21.15.55:0/1422876031 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559808712 0 0) 0x7f97280b40f0 con 0x7f972806c0f0 2023-10-06T03:48:34.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.607+0000 7f972559a700 1 -- 172.21.15.55:0/1422876031 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9710003570 con 0x7f972806c0f0 2023-10-06T03:48:34.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.607+0000 7f972659c700 1 -- 172.21.15.55:0/1422876031 >> v1:172.21.15.163:6789/0 conn(0x7f97280ad870 legacy=0x7f972806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.607+0000 7f972659c700 1 -- 172.21.15.55:0/1422876031 >> v1:172.21.15.55:6789/0 conn(0x7f972806c0f0 legacy=0x7f972806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.607+0000 7f972659c700 1 -- 172.21.15.55:0/1422876031 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9728069c00 con 0x7f9728142ea0 2023-10-06T03:48:34.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.607+0000 7f972659c700 1 --2- 172.21.15.55:0/1422876031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9728142ea0 0x7f97280b3630 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f97180098e0 tx=0x7f97180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b758809749889d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:34.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.608+0000 7f972559a700 1 -- 172.21.15.55:0/1422876031 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971800b640 con 0x7f9728142ea0 2023-10-06T03:48:34.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.608+0000 7f972559a700 1 -- 172.21.15.55:0/1422876031 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971800b7a0 con 0x7f9728142ea0 2023-10-06T03:48:34.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.608+0000 7f972559a700 1 -- 172.21.15.55:0/1422876031 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971800bb40 con 0x7f9728142ea0 2023-10-06T03:48:34.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.608+0000 7f972759e700 1 -- 172.21.15.55:0/1422876031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9728142ea0 msgr2=0x7f97280b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.608+0000 7f972759e700 1 --2- 172.21.15.55:0/1422876031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9728142ea0 0x7f97280b3630 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f97180098e0 tx=0x7f97180091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.608+0000 7f972759e700 1 -- 172.21.15.55:0/1422876031 shutdown_connections 2023-10-06T03:48:34.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.608+0000 7f972759e700 1 --2- 172.21.15.55:0/1422876031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9728142ea0 0x7f97280b3630 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.608+0000 7f972759e700 1 -- 172.21.15.55:0/1422876031 >> 172.21.15.55:0/1422876031 conn(0x7f97280ce430 msgr2=0x7f97280ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:34.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.608+0000 7f972759e700 1 -- 172.21.15.55:0/1422876031 shutdown_connections 2023-10-06T03:48:34.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.608+0000 7f972759e700 1 -- 172.21.15.55:0/1422876031 wait complete. 2023-10-06T03:48:34.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.610+0000 7f972759e700 1 Processor -- start 2023-10-06T03:48:34.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.610+0000 7f972759e700 1 -- start start 2023-10-06T03:48:34.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.611+0000 7f972759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f972806c0f0 0x7f972815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.611+0000 7f972759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9728142ea0 0x7f9728135050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.611+0000 7f972759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97280c75e0 0x7f97280c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.611+0000 7f972759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972815d3b0 con 0x7f97280c75e0 2023-10-06T03:48:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.611+0000 7f972759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972815c4f0 con 0x7f972806c0f0 2023-10-06T03:48:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.611+0000 7f972759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9728135590 con 0x7f9728142ea0 2023-10-06T03:48:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.611+0000 7f9726d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97280c75e0 0x7f97280c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.611+0000 7f9726d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97280c75e0 0x7f97280c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46952/0 (socket says 172.21.15.55:46952) 2023-10-06T03:48:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.611+0000 7f9726d9d700 1 -- 172.21.15.55:0/2724878821 learned_addr learned my addr 172.21.15.55:0/2724878821 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.611+0000 7f972659c700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f972806c0f0 0x7f972815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.611+0000 7f9725d9b700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9728142ea0 0x7f9728135050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.612+0000 7f9726d9d700 1 -- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9728142ea0 msgr2=0x7f9728135050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.612+0000 7f9726d9d700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9728142ea0 0x7f9728135050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.612+0000 7f9726d9d700 1 -- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f972806c0f0 msgr2=0x7f972815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.612+0000 7f9726d9d700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f972806c0f0 0x7f972815bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.612+0000 7f9726d9d700 1 -- 172.21.15.55:0/2724878821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9718009420 con 0x7f97280c75e0 2023-10-06T03:48:34.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.612+0000 7f9725d9b700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9728142ea0 0x7f9728135050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:34.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.612+0000 7f9726d9d700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97280c75e0 0x7f97280c8d90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f972000cc40 tx=0x7f9720001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:34.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.612+0000 7f97167fc700 1 -- 172.21.15.55:0/2724878821 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972000a910 con 0x7f97280c75e0 2023-10-06T03:48:34.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.612+0000 7f97167fc700 1 -- 172.21.15.55:0/2724878821 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972000aa70 con 0x7f97280c75e0 2023-10-06T03:48:34.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.612+0000 7f972759e700 1 -- 172.21.15.55:0/2724878821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972815c7a0 con 0x7f97280c75e0 2023-10-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.612+0000 7f97167fc700 1 -- 172.21.15.55:0/2724878821 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972000adf0 con 0x7f97280c75e0 2023-10-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.612+0000 7f972759e700 1 -- 172.21.15.55:0/2724878821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972815cd60 con 0x7f97280c75e0 2023-10-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.613+0000 7f972759e700 1 -- 172.21.15.55:0/2724878821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972804c3c0 con 0x7f97280c75e0 2023-10-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.614+0000 7f97167fc700 1 -- 172.21.15.55:0/2724878821 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9720007300 con 0x7f97280c75e0 2023-10-06T03:48:34.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.615+0000 7f97167fc700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9708066f30 0x7f97080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:34.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.615+0000 7f97167fc700 1 -- 172.21.15.55:0/2724878821 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f9720084d00 con 0x7f97280c75e0 2023-10-06T03:48:34.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.616+0000 7f972659c700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9708066f30 0x7f97080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:34.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.617+0000 7f972659c700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9708066f30 0x7f97080693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f97180093f0 tx=0x7f9718017000 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:34.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.619+0000 7f97167fc700 1 -- 172.21.15.55:0/2724878821 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9720051bd0 con 0x7f97280c75e0 2023-10-06T03:48:34.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.869+0000 7f972759e700 1 -- 172.21.15.55:0/2724878821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9728143270 con 0x7f97280c75e0 2023-10-06T03:48:34.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.869+0000 7f97167fc700 1 -- 172.21.15.55:0/2724878821 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7f97200551f0 con 0x7f97280c75e0 2023-10-06T03:48:34.871 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:48:34.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.872+0000 7f972759e700 1 -- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9708066f30 msgr2=0x7f97080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.872+0000 7f972759e700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9708066f30 0x7f97080693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f97180093f0 tx=0x7f9718017000 comp rx=0 tx=0).stop 2023-10-06T03:48:34.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.872+0000 7f972759e700 1 -- 172.21.15.55:0/2724878821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97280c75e0 msgr2=0x7f97280c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:34.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.872+0000 7f972759e700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97280c75e0 0x7f97280c8d90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f972000cc40 tx=0x7f9720001aa0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.872+0000 7f972759e700 1 -- 172.21.15.55:0/2724878821 shutdown_connections 2023-10-06T03:48:34.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.872+0000 7f972759e700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9708066f30 0x7f97080693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.872+0000 7f972759e700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97280c75e0 0x7f97280c8d90 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.872+0000 7f972759e700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f972806c0f0 0x7f972815bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.872+0000 7f972759e700 1 --2- 172.21.15.55:0/2724878821 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9728142ea0 0x7f9728135050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:34.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.872+0000 7f972759e700 1 -- 172.21.15.55:0/2724878821 >> 172.21.15.55:0/2724878821 conn(0x7f97280ce430 msgr2=0x7f9728141e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:34.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.872+0000 7f972759e700 1 -- 172.21.15.55:0/2724878821 shutdown_connections 2023-10-06T03:48:34.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:34.872+0000 7f972759e700 1 -- 172.21.15.55:0/2724878821 wait complete. 2023-10-06T03:48:34.891 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:48:34.891 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:48:34.891 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2023-10-06T03:48:34.891 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:48:34.891 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:48:34.891 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:48:34.891 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:48:34.891 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:48:34.891 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:48:34.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:48:34.892 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2023-10-06T03:48:34.929 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:48:34.929 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:48:34.929 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:48:34.929 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:48:34.929 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T03:48:34.929 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:48:34.930 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:48:34.930 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:48:34.930 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2023-10-06T03:48:34.930 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:48:34.930 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:48:34.930 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2023-10-06T03:48:34.930 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:48:34.930 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:48:34.930 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2023-10-06T03:48:34.930 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T03:48:34.931 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 41326sec preferred_lft 41326sec 2023-10-06T03:48:34.931 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link 2023-10-06T03:48:34.931 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:48:34.931 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:48:34.931 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:48:34.931 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2023-10-06T03:48:34.931 INFO:teuthology.orchestra.run.smithi055.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:48:34.931 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff 2023-10-06T03:48:34.931 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:48:34.931 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:48:34.931 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::1099:8eff:fe6d:102/64 scope link 2023-10-06T03:48:34.932 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:48:34.932 INFO:teuthology.orchestra.run.smithi055.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:48:34.932 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether f2:5d:53:d2:00:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:48:34.932 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f05d:53ff:fed2:a/64 scope link 2023-10-06T03:48:34.932 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:48:34.932 INFO:teuthology.orchestra.run.smithi055.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:48:34.932 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:48:34.932 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::e037:33ff:feb7:6c8f/64 scope link 2023-10-06T03:48:34.932 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:48:34.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:48:34.933 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:48:35.029 INFO:teuthology.orchestra.run.smithi055.stdout:1 2023-10-06T03:48:35.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:48:35.033 DEBUG:teuthology.orchestra.run.smithi055:> route 2023-10-06T03:48:35.072 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2023-10-06T03:48:35.072 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:48:35.072 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T03:48:35.072 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T03:48:35.072 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:48:35.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:48:35.072 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:48:35.073 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:48:35.073 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T03:48:35.073 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T03:48:35.073 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T03:48:35.073 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:48:35.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:48:35.271 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T03:48:35.308 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:48:35.308 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:48:35.308 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:48:35.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:48:35.309 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2023-10-06T03:48:35.411 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:48:35.411 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:48:35.448 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:48:35.449 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:48:35.449 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:48:35.524 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T03:48:35.524 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:48:35.525 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:48:35.657 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:48:35.658 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:48:35.658 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:48:35.674 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:48:35.674 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:48:35.674 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:48:35.674 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:48:35.675 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:48:35.696 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:48:35.696 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:48:35.696 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:48:35.696 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:48:35.696 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:48:35.696 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T03:48:35.696 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T03:48:35.696 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T03:48:35.701 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:48:35.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:48:35.701 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:48:35.765 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:48:35.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.951+0000 7fb4eab9d700 1 Processor -- start 2023-10-06T03:48:35.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.951+0000 7fb4eab9d700 1 -- start start 2023-10-06T03:48:35.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.952+0000 7fb4eab9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4e4159420 0x7fb4e40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:35.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.952+0000 7fb4eab9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb4e415d230 0x7fb4e415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.952+0000 7fb4eab9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e415dbd0 con 0x7fb4e41597f0 2023-10-06T03:48:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.952+0000 7fb4eab9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e40b2980 con 0x7fb4e415d230 2023-10-06T03:48:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.952+0000 7fb4eab9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40b11d0 con 0x7fb4e4159420 2023-10-06T03:48:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.953+0000 7fb4e939a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb4e41597f0 0x7fb4e40b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46662/0 (socket says 172.21.15.55:46662) 2023-10-06T03:48:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.953+0000 7fb4e939a700 1 -- 172.21.15.55:0/130177127 learned_addr learned my addr 172.21.15.55:0/130177127 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.953+0000 7fb4ea39c700 1 --2- 172.21.15.55:0/130177127 >> v2:172.21.15.163:3300/0 conn(0x7fb4e415d230 0x7fb4e415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.953+0000 7fb4e9b9b700 1 --2- 172.21.15.55:0/130177127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4e4159420 0x7fb4e40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.953+0000 7fb4e8b99700 1 -- 172.21.15.55:0/130177127 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920095741 0 0) 0x7fb4d4001110 con 0x7fb4e41597f0 2023-10-06T03:48:35.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.953+0000 7fb4e8b99700 1 -- 172.21.15.55:0/130177127 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4d0003570 con 0x7fb4e41597f0 2023-10-06T03:48:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.953+0000 7fb4ea39c700 1 -- 172.21.15.55:0/130177127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4e4159420 msgr2=0x7fb4e40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.953+0000 7fb4ea39c700 1 --2- 172.21.15.55:0/130177127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4e4159420 0x7fb4e40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.953+0000 7fb4ea39c700 1 -- 172.21.15.55:0/130177127 >> v1:172.21.15.55:6789/0 conn(0x7fb4e41597f0 legacy=0x7fb4e40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.953+0000 7fb4e8b99700 1 -- 172.21.15.55:0/130177127 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1065259336 0 0) 0x7fb4d0003570 con 0x7fb4e41597f0 2023-10-06T03:48:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.953+0000 7fb4ea39c700 1 -- 172.21.15.55:0/130177127 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e40b14b0 con 0x7fb4e415d230 2023-10-06T03:48:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.954+0000 7fb4ea39c700 1 --2- 172.21.15.55:0/130177127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4e415d230 0x7fb4e415d690 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009e60 tx=0x7fb4dc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c66e658bdd4495 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.954+0000 7fb4e8b99700 1 -- 172.21.15.55:0/130177127 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00e070 con 0x7fb4e415d230 2023-10-06T03:48:35.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.954+0000 7fb4e8b99700 1 -- 172.21.15.55:0/130177127 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4dc00b7a0 con 0x7fb4e415d230 2023-10-06T03:48:35.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.954+0000 7fb4e8b99700 1 -- 172.21.15.55:0/130177127 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4001600 con 0x7fb4e415d230 2023-10-06T03:48:35.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.955+0000 7fb4eab9d700 1 -- 172.21.15.55:0/130177127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4e415d230 msgr2=0x7fb4e415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:35.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.955+0000 7fb4eab9d700 1 --2- 172.21.15.55:0/130177127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4e415d230 0x7fb4e415d690 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009e60 tx=0x7fb4dc009780 comp rx=0 tx=0).stop 2023-10-06T03:48:35.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.955+0000 7fb4eab9d700 1 -- 172.21.15.55:0/130177127 shutdown_connections 2023-10-06T03:48:35.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.955+0000 7fb4eab9d700 1 --2- 172.21.15.55:0/130177127 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4e415d230 0x7fb4e415d690 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:35.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.955+0000 7fb4eab9d700 1 --2- 172.21.15.55:0/130177127 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4e4159420 0x7fb4e40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:35.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.955+0000 7fb4eab9d700 1 -- 172.21.15.55:0/130177127 >> 172.21.15.55:0/130177127 conn(0x7fb4e40d0660 msgr2=0x7fb4e40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:35.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.955+0000 7fb4eab9d700 1 -- 172.21.15.55:0/130177127 shutdown_connections 2023-10-06T03:48:35.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.955+0000 7fb4eab9d700 1 -- 172.21.15.55:0/130177127 wait complete. 2023-10-06T03:48:35.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.956+0000 7fb4eab9d700 1 Processor -- start 2023-10-06T03:48:35.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.956+0000 7fb4eab9d700 1 -- start start 2023-10-06T03:48:35.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.957+0000 7fb4eab9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e4159420 0x7fb4e416d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:35.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.957+0000 7fb4eab9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4e41597f0 0x7fb4e40c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:35.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.957+0000 7fb4eab9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4e415d230 0x7fb4e40c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:35.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.957+0000 7fb4eab9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e4153cb0 con 0x7fb4e4159420 2023-10-06T03:48:35.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.957+0000 7fb4eab9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e415c9f0 con 0x7fb4e415d230 2023-10-06T03:48:35.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.957+0000 7fb4eab9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e4169ba0 con 0x7fb4e41597f0 2023-10-06T03:48:35.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4e9b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e4159420 0x7fb4e416d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:35.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4e9b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e4159420 0x7fb4e416d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46954/0 (socket says 172.21.15.55:46954) 2023-10-06T03:48:35.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4e9b9b700 1 -- 172.21.15.55:0/1026600515 learned_addr learned my addr 172.21.15.55:0/1026600515 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:35.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4e939a700 1 --2- 172.21.15.55:0/1026600515 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4e41597f0 0x7fb4e40c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:35.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4ea39c700 1 --2- 172.21.15.55:0/1026600515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4e415d230 0x7fb4e40c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:35.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4e9b9b700 1 -- 172.21.15.55:0/1026600515 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4e41597f0 msgr2=0x7fb4e40c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:35.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4e9b9b700 1 --2- 172.21.15.55:0/1026600515 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4e41597f0 0x7fb4e40c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:35.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4e9b9b700 1 -- 172.21.15.55:0/1026600515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4e415d230 msgr2=0x7fb4e40c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:35.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4e9b9b700 1 --2- 172.21.15.55:0/1026600515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4e415d230 0x7fb4e40c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:35.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4e9b9b700 1 -- 172.21.15.55:0/1026600515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc009a20 con 0x7fb4e4159420 2023-10-06T03:48:35.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4e9b9b700 1 --2- 172.21.15.55:0/1026600515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e4159420 0x7fb4e416d8d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000ec80 tx=0x7fb4e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:35.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4d9ffb700 1 -- 172.21.15.55:0/1026600515 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000c9d0 con 0x7fb4e4159420 2023-10-06T03:48:35.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4d9ffb700 1 -- 172.21.15.55:0/1026600515 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e000cb30 con 0x7fb4e4159420 2023-10-06T03:48:35.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4d9ffb700 1 -- 172.21.15.55:0/1026600515 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0013470 con 0x7fb4e4159420 2023-10-06T03:48:35.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.958+0000 7fb4eab9d700 1 -- 172.21.15.55:0/1026600515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e4169e80 con 0x7fb4e4159420 2023-10-06T03:48:35.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.959+0000 7fb4eab9d700 1 -- 172.21.15.55:0/1026600515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e416a440 con 0x7fb4e4159420 2023-10-06T03:48:35.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.960+0000 7fb4eab9d700 1 -- 172.21.15.55:0/1026600515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e404c3c0 con 0x7fb4e4159420 2023-10-06T03:48:35.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.961+0000 7fb4d9ffb700 1 -- 172.21.15.55:0/1026600515 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb4e0013820 con 0x7fb4e4159420 2023-10-06T03:48:35.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.961+0000 7fb4d9ffb700 1 --2- 172.21.15.55:0/1026600515 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb4c8066f30 0x7fb4c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:35.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.961+0000 7fb4d9ffb700 1 -- 172.21.15.55:0/1026600515 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fb4e0084bb0 con 0x7fb4e4159420 2023-10-06T03:48:35.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.962+0000 7fb4e939a700 1 --2- 172.21.15.55:0/1026600515 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb4c8066f30 0x7fb4c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:35.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.963+0000 7fb4e939a700 1 --2- 172.21.15.55:0/1026600515 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb4c8066f30 0x7fb4c80693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40cc5e0 tx=0x7fb4d4001d20 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:35.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:35.966+0000 7fb4d9ffb700 1 -- 172.21.15.55:0/1026600515 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4e0051b30 con 0x7fb4e4159420 2023-10-06T03:48:36.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.195+0000 7fb4eab9d700 1 -- 172.21.15.55:0/1026600515 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4e40b1790 con 0x7fb4c8066f30 2023-10-06T03:48:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.198+0000 7fb4d9ffb700 1 -- 172.21.15.55:0/1026600515 <== mgr.5483 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb4e40b1790 con 0x7fb4c8066f30 2023-10-06T03:48:36.200 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:48:36.200 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:36.200 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephfs" 2023-10-06T03:48:36.200 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:36.200 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:48:36.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.200+0000 7fb4eab9d700 1 -- 172.21.15.55:0/1026600515 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb4c8066f30 msgr2=0x7fb4c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.200+0000 7fb4eab9d700 1 --2- 172.21.15.55:0/1026600515 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb4c8066f30 0x7fb4c80693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb4e40cc5e0 tx=0x7fb4d4001d20 comp rx=0 tx=0).stop 2023-10-06T03:48:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.200+0000 7fb4eab9d700 1 -- 172.21.15.55:0/1026600515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e4159420 msgr2=0x7fb4e416d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.200+0000 7fb4eab9d700 1 --2- 172.21.15.55:0/1026600515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e4159420 0x7fb4e416d8d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fb4e000ec80 tx=0x7fb4e00091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.200+0000 7fb4eab9d700 1 -- 172.21.15.55:0/1026600515 shutdown_connections 2023-10-06T03:48:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.200+0000 7fb4eab9d700 1 --2- 172.21.15.55:0/1026600515 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb4c8066f30 0x7fb4c80693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.200+0000 7fb4eab9d700 1 --2- 172.21.15.55:0/1026600515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4e415d230 0x7fb4e40c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.200+0000 7fb4eab9d700 1 --2- 172.21.15.55:0/1026600515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4e4159420 0x7fb4e416d8d0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.200+0000 7fb4eab9d700 1 --2- 172.21.15.55:0/1026600515 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4e41597f0 0x7fb4e40c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.200+0000 7fb4eab9d700 1 -- 172.21.15.55:0/1026600515 >> 172.21.15.55:0/1026600515 conn(0x7fb4e40d0660 msgr2=0x7fb4e40b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.200+0000 7fb4eab9d700 1 -- 172.21.15.55:0/1026600515 shutdown_connections 2023-10-06T03:48:36.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.200+0000 7fb4eab9d700 1 -- 172.21.15.55:0/1026600515 wait complete. 2023-10-06T03:48:36.220 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T03:48:36.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.406+0000 7f98e2d9d700 1 Processor -- start 2023-10-06T03:48:36.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.407+0000 7f98e2d9d700 1 -- start start 2023-10-06T03:48:36.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.408+0000 7f98e2d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98dc0cb1a0 0x7f98dc15a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:36.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.408+0000 7f98e2d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98dc0ae0d0 0x7f98dc0ae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:36.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.408+0000 7f98e2d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98dc06ebd0 con 0x7f98dc15a8a0 2023-10-06T03:48:36.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.408+0000 7f98e2d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc15b2e0 con 0x7f98dc0ae0d0 2023-10-06T03:48:36.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.408+0000 7f98e2d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0aea50 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.408+0000 7f98e259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98dc0ae0d0 0x7f98dc0ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:36.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.408+0000 7f98e259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f98dc0ae0d0 0x7f98dc0ae510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46968/0 (socket says 172.21.15.55:46968) 2023-10-06T03:48:36.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.408+0000 7f98e259c700 1 -- 172.21.15.55:0/2555553802 learned_addr learned my addr 172.21.15.55:0/2555553802 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:36.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.408+0000 7f98e1d9b700 1 --2- 172.21.15.55:0/2555553802 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98dc0cb1a0 0x7f98dc15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:36.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.409+0000 7f98e259c700 1 -- 172.21.15.55:0/2555553802 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98dc0cb1a0 msgr2=0x7f98dc15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:36.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.409+0000 7f98e259c700 1 --2- 172.21.15.55:0/2555553802 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98dc0cb1a0 0x7f98dc15a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.409+0000 7f98e259c700 1 -- 172.21.15.55:0/2555553802 >> v1:172.21.15.163:6789/0 conn(0x7f98dc15a8a0 legacy=0x7f98dc06f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:36.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.409+0000 7f98e259c700 1 -- 172.21.15.55:0/2555553802 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc15ad20 con 0x7f98dc0ae0d0 2023-10-06T03:48:36.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.409+0000 7f98e259c700 1 --2- 172.21.15.55:0/2555553802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98dc0ae0d0 0x7f98dc0ae510 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009f20 tx=0x7f98d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a68a6962fbd136 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:36.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.409+0000 7f98e0d99700 1 -- 172.21.15.55:0/2555553802 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d400e070 con 0x7f98dc0ae0d0 2023-10-06T03:48:36.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.409+0000 7f98e0d99700 1 -- 172.21.15.55:0/2555553802 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d400bb10 con 0x7f98dc0ae0d0 2023-10-06T03:48:36.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.409+0000 7f98e0d99700 1 -- 172.21.15.55:0/2555553802 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d40092e0 con 0x7f98dc0ae0d0 2023-10-06T03:48:36.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.410+0000 7f98e2d9d700 1 -- 172.21.15.55:0/2555553802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98dc0ae0d0 msgr2=0x7f98dc0ae510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:36.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.410+0000 7f98e2d9d700 1 --2- 172.21.15.55:0/2555553802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98dc0ae0d0 0x7f98dc0ae510 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009f20 tx=0x7f98d4009840 comp rx=0 tx=0).stop 2023-10-06T03:48:36.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.410+0000 7f98e2d9d700 1 -- 172.21.15.55:0/2555553802 shutdown_connections 2023-10-06T03:48:36.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.410+0000 7f98e2d9d700 1 --2- 172.21.15.55:0/2555553802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98dc0ae0d0 0x7f98dc0ae510 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.410+0000 7f98e2d9d700 1 --2- 172.21.15.55:0/2555553802 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98dc0cb1a0 0x7f98dc15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.410+0000 7f98e2d9d700 1 -- 172.21.15.55:0/2555553802 >> 172.21.15.55:0/2555553802 conn(0x7f98dc0cc1e0 msgr2=0x7f98dc0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:36.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.410+0000 7f98e2d9d700 1 -- 172.21.15.55:0/2555553802 shutdown_connections 2023-10-06T03:48:36.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.410+0000 7f98e2d9d700 1 -- 172.21.15.55:0/2555553802 wait complete. 2023-10-06T03:48:36.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.411+0000 7f98e2d9d700 1 Processor -- start 2023-10-06T03:48:36.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.412+0000 7f98e2d9d700 1 -- start start 2023-10-06T03:48:36.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.412+0000 7f98e2d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98dc0ae0d0 0x7f98dc0c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:36.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.412+0000 7f98e2d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98dc0cb1a0 0x7f98dc147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:36.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.412+0000 7f98e2d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98dc0c7720 0x7f98dc0c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:36.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.412+0000 7f98e2d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0c8100 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.412+0000 7f98e2d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0c3290 con 0x7f98dc0c7720 2023-10-06T03:48:36.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.412+0000 7f98e2d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc0c3b30 con 0x7f98dc0ae0d0 2023-10-06T03:48:36.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.413+0000 7f98e159a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98dc0cb1a0 0x7f98dc147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:36.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.413+0000 7f98e159a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98dc0cb1a0 0x7f98dc147e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46980/0 (socket says 172.21.15.55:46980) 2023-10-06T03:48:36.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.413+0000 7f98e159a700 1 -- 172.21.15.55:0/3463616930 learned_addr learned my addr 172.21.15.55:0/3463616930 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:36.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.413+0000 7f98e259c700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98dc0c7720 0x7f98dc0c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:36.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.413+0000 7f98e1d9b700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98dc0ae0d0 0x7f98dc0c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:36.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.413+0000 7f98e159a700 1 -- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98dc0ae0d0 msgr2=0x7f98dc0c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:36.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.413+0000 7f98e159a700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98dc0ae0d0 0x7f98dc0c3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.413+0000 7f98e159a700 1 -- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98dc0c7720 msgr2=0x7f98dc0c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:36.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.413+0000 7f98e159a700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98dc0c7720 0x7f98dc0c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.413+0000 7f98e159a700 1 -- 172.21.15.55:0/3463616930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d4009ae0 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.413+0000 7f98e1d9b700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98dc0ae0d0 0x7f98dc0c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:36.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.413+0000 7f98e159a700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98dc0cb1a0 0x7f98dc147e40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00c510 tx=0x7f98cc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:36.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.414+0000 7f98d1ffb700 1 -- 172.21.15.55:0/3463616930 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc011590 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.414+0000 7f98d1ffb700 1 -- 172.21.15.55:0/3463616930 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc011710 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.414+0000 7f98d1ffb700 1 -- 172.21.15.55:0/3463616930 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc011ad0 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.414+0000 7f98e2d9d700 1 -- 172.21.15.55:0/3463616930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc168950 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.414+0000 7f98e2d9d700 1 -- 172.21.15.55:0/3463616930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc0c37c0 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.415+0000 7f98e2d9d700 1 -- 172.21.15.55:0/3463616930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98dc0cbf50 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.416+0000 7f98d1ffb700 1 -- 172.21.15.55:0/3463616930 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 26) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98cc01b3b0 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.417+0000 7f98d1ffb700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f98c0066f30 0x7f98c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:36.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.417+0000 7f98d1ffb700 1 -- 172.21.15.55:0/3463616930 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f98cc083d70 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.417+0000 7f98e1d9b700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f98c0066f30 0x7f98c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:36.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.418+0000 7f98e1d9b700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f98c0066f30 0x7f98c00693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f98d8006fd0 tx=0x7f98d8008040 comp rx=0 tx=0).ready entity=mgr.5483 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:36.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.421+0000 7f98d1ffb700 1 -- 172.21.15.55:0/3463616930 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98cc050c50 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.678+0000 7f98e2d9d700 1 -- 172.21.15.55:0/3463616930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f98dc0c3c70 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.733+0000 7f98d1ffb700 1 -- 172.21.15.55:0/3463616930 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 27) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f98cc051020 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.733+0000 7f98d1ffb700 1 -- 172.21.15.55:0/3463616930 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v27) v1 ==== 92+0+0 (secure 0 0 0) 0x7f98cc0546e0 con 0x7f98dc0cb1a0 2023-10-06T03:48:36.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.736+0000 7f98e2d9d700 1 -- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f98c0066f30 msgr2=0x7f98c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:36.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.736+0000 7f98e2d9d700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f98c0066f30 0x7f98c00693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f98d8006fd0 tx=0x7f98d8008040 comp rx=0 tx=0).stop 2023-10-06T03:48:36.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.736+0000 7f98e2d9d700 1 -- 172.21.15.55:0/3463616930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98dc0cb1a0 msgr2=0x7f98dc147e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:36.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.736+0000 7f98e2d9d700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98dc0cb1a0 0x7f98dc147e40 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00c510 tx=0x7f98cc00c8d0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.736+0000 7f98e2d9d700 1 -- 172.21.15.55:0/3463616930 shutdown_connections 2023-10-06T03:48:36.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.736+0000 7f98e2d9d700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f98c0066f30 0x7f98c00693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.736+0000 7f98e2d9d700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f98dc0c7720 0x7f98dc0c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.736+0000 7f98e2d9d700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f98dc0ae0d0 0x7f98dc0c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.736+0000 7f98e2d9d700 1 --2- 172.21.15.55:0/3463616930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f98dc0cb1a0 0x7f98dc147e40 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.736+0000 7f98e2d9d700 1 -- 172.21.15.55:0/3463616930 >> 172.21.15.55:0/3463616930 conn(0x7f98dc0cc1e0 msgr2=0x7f98dc155f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:36.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.736+0000 7f98e2d9d700 1 -- 172.21.15.55:0/3463616930 shutdown_connections 2023-10-06T03:48:36.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.736+0000 7f98e2d9d700 1 -- 172.21.15.55:0/3463616930 wait complete. 2023-10-06T03:48:36.756 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T03:48:36.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.951+0000 7f01deb9d700 1 Processor -- start 2023-10-06T03:48:36.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.951+0000 7f01deb9d700 1 -- start start 2023-10-06T03:48:36.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.953+0000 7f01deb9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f01d80b4e70 0x7f01d80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:36.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.953+0000 7f01deb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01d80b16d0 0x7f01d80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:36.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.953+0000 7f01deb9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d80b2080 con 0x7f01d815d240 2023-10-06T03:48:36.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.953+0000 7f01deb9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d80cfb00 con 0x7f01d80b4e70 2023-10-06T03:48:36.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.953+0000 7f01deb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80b5c40 con 0x7f01d80b16d0 2023-10-06T03:48:36.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.953+0000 7f01dd39a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01d80b16d0 0x7f01d80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:36.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.953+0000 7f01ddb9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f01d80b4e70 0x7f01d80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:36.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.953+0000 7f01ddb9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f01d80b4e70 0x7f01d80ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:47724/0 (socket says 172.21.15.55:47724) 2023-10-06T03:48:36.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.953+0000 7f01dd39a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01d80b16d0 0x7f01d80b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:39448/0 (socket says 172.21.15.55:39448) 2023-10-06T03:48:36.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.953+0000 7f01ddb9b700 1 -- 172.21.15.55:0/248975326 learned_addr learned my addr 172.21.15.55:0/248975326 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:36.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.953+0000 7f01de39c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f01d815d240 0x7f01d815d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:53290/0 (socket says 172.21.15.55:53290) 2023-10-06T03:48:36.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.954+0000 7f01dd39a700 1 -- 172.21.15.55:0/248975326 >> v2:172.21.15.163:3300/0 conn(0x7f01d80b4e70 msgr2=0x7f01d80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:36.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.954+0000 7f01dd39a700 1 --2- 172.21.15.55:0/248975326 >> v2:172.21.15.163:3300/0 conn(0x7f01d80b4e70 0x7f01d80ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.954+0000 7f01dd39a700 1 -- 172.21.15.55:0/248975326 >> v1:172.21.15.163:6789/0 conn(0x7f01d815d240 legacy=0x7f01d815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:36.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.954+0000 7f01dd39a700 1 -- 172.21.15.55:0/248975326 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d80b5f20 con 0x7f01d80b16d0 2023-10-06T03:48:36.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.954+0000 7f01ddb9b700 1 --2- 172.21.15.55:0/248975326 >> v2:172.21.15.163:3300/0 conn(0x7f01d80b4e70 0x7f01d80ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.954+0000 7f01dcb99700 1 -- 172.21.15.55:0/248975326 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759232007 0 0) 0x7f01d80b2080 con 0x7f01d815d240 2023-10-06T03:48:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.954+0000 7f01dd39a700 1 --2- 172.21.15.55:0/248975326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01d80b16d0 0x7f01d80b1b40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f01c80098e0 tx=0x7f01c80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac6fbcd885f21e26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.954+0000 7f01dcb99700 1 -- 172.21.15.55:0/248975326 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c800b640 con 0x7f01d80b16d0 2023-10-06T03:48:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.954+0000 7f01dcb99700 1 -- 172.21.15.55:0/248975326 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c800b7a0 con 0x7f01d80b16d0 2023-10-06T03:48:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.955+0000 7f01dcb99700 1 -- 172.21.15.55:0/248975326 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c800bb40 con 0x7f01d80b16d0 2023-10-06T03:48:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.955+0000 7f01deb9d700 1 -- 172.21.15.55:0/248975326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01d80b16d0 msgr2=0x7f01d80b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.955+0000 7f01deb9d700 1 --2- 172.21.15.55:0/248975326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01d80b16d0 0x7f01d80b1b40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f01c80098e0 tx=0x7f01c80091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.955+0000 7f01deb9d700 1 -- 172.21.15.55:0/248975326 shutdown_connections 2023-10-06T03:48:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.955+0000 7f01deb9d700 1 --2- 172.21.15.55:0/248975326 >> v2:172.21.15.163:3300/0 conn(0x7f01d80b4e70 0x7f01d80ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.955+0000 7f01deb9d700 1 --2- 172.21.15.55:0/248975326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01d80b16d0 0x7f01d80b1b40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.955+0000 7f01deb9d700 1 -- 172.21.15.55:0/248975326 >> 172.21.15.55:0/248975326 conn(0x7f01d80d10c0 msgr2=0x7f01d80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:36.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.955+0000 7f01deb9d700 1 -- 172.21.15.55:0/248975326 shutdown_connections 2023-10-06T03:48:36.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.955+0000 7f01deb9d700 1 -- 172.21.15.55:0/248975326 wait complete. 2023-10-06T03:48:36.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.957+0000 7f01deb9d700 1 Processor -- start 2023-10-06T03:48:36.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.957+0000 7f01deb9d700 1 -- start start 2023-10-06T03:48:36.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.958+0000 7f01deb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01d80b16d0 0x7f01d8158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:36.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.958+0000 7f01deb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01d80b4e70 0x7f01d80cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:36.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.958+0000 7f01deb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01d815d240 0x7f01d80cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:36.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.958+0000 7f01deb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d816d850 con 0x7f01d80b4e70 2023-10-06T03:48:36.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.958+0000 7f01deb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d81585b0 con 0x7f01d815d240 2023-10-06T03:48:36.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.958+0000 7f01deb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d816cf20 con 0x7f01d80b16d0 2023-10-06T03:48:36.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.958+0000 7f01dd39a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01d80b4e70 0x7f01d80cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:36.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.958+0000 7f01ddb9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01d80b16d0 0x7f01d8158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:36.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.958+0000 7f01dd39a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01d80b4e70 0x7f01d80cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47004/0 (socket says 172.21.15.55:47004) 2023-10-06T03:48:36.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.958+0000 7f01dd39a700 1 -- 172.21.15.55:0/1808973492 learned_addr learned my addr 172.21.15.55:0/1808973492 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:36.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.958+0000 7f01de39c700 1 --2- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01d815d240 0x7f01d80cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:36.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.958+0000 7f01dd39a700 1 -- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01d80b16d0 msgr2=0x7f01d8158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:36.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.958+0000 7f01dd39a700 1 --2- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01d80b16d0 0x7f01d8158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.959+0000 7f01dd39a700 1 -- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01d815d240 msgr2=0x7f01d80cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:36.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.959+0000 7f01dd39a700 1 --2- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01d815d240 0x7f01d80cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:36.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.959+0000 7f01dd39a700 1 -- 172.21.15.55:0/1808973492 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d40094f0 con 0x7f01d80b4e70 2023-10-06T03:48:36.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.959+0000 7f01ddb9b700 1 --2- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01d80b16d0 0x7f01d8158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:36.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.959+0000 7f01dd39a700 1 --2- 172.21.15.55:0/1808973492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01d80b4e70 0x7f01d80cb780 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f01c80093f0 tx=0x7f01c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:36.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.959+0000 7f01cdffb700 1 -- 172.21.15.55:0/1808973492 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c801f070 con 0x7f01d80b4e70 2023-10-06T03:48:36.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.959+0000 7f01cdffb700 1 -- 172.21.15.55:0/1808973492 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c800b910 con 0x7f01d80b4e70 2023-10-06T03:48:36.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.959+0000 7f01cdffb700 1 -- 172.21.15.55:0/1808973492 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8011420 con 0x7f01d80b4e70 2023-10-06T03:48:36.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.959+0000 7f01deb9d700 1 -- 172.21.15.55:0/1808973492 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c8009420 con 0x7f01d80b4e70 2023-10-06T03:48:36.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.959+0000 7f01deb9d700 1 -- 172.21.15.55:0/1808973492 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d816d4e0 con 0x7f01d80b4e70 2023-10-06T03:48:36.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.960+0000 7f01deb9d700 1 -- 172.21.15.55:0/1808973492 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d816da00 con 0x7f01d80b4e70 2023-10-06T03:48:36.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.961+0000 7f01cdffb700 1 -- 172.21.15.55:0/1808973492 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 27) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f01c80117d0 con 0x7f01d80b4e70 2023-10-06T03:48:36.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.962+0000 7f01cdffb700 1 --2- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01bc066f80 0x7f01bc069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:36.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.962+0000 7f01cdffb700 1 -- 172.21.15.55:0/1808973492 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f01c8085c10 con 0x7f01d80b4e70 2023-10-06T03:48:36.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.962+0000 7f01ddb9b700 1 -- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01bc066f80 msgr2=0x7f01bc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:36.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.962+0000 7f01ddb9b700 1 --2- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01bc066f80 0x7f01bc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:36.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:36.967+0000 7f01cdffb700 1 -- 172.21.15.55:0/1808973492 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01d816da00 con 0x7f01d80b4e70 2023-10-06T03:48:37.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.163+0000 7f01ddb9b700 1 -- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01bc066f80 msgr2=0x7f01bc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:37.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.163+0000 7f01ddb9b700 1 --2- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01bc066f80 0x7f01bc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:37.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.171+0000 7f01deb9d700 1 -- 172.21.15.55:0/1808973492 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f01d816d990 con 0x7f01d80b4e70 2023-10-06T03:48:37.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.175+0000 7f01cdffb700 1 -- 172.21.15.55:0/1808973492 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f01c80280d0 con 0x7f01d80b4e70 2023-10-06T03:48:37.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.177+0000 7f01deb9d700 1 -- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01bc066f80 msgr2=0x7f01bc069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:37.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.177+0000 7f01deb9d700 1 --2- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01bc066f80 0x7f01bc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.177+0000 7f01deb9d700 1 -- 172.21.15.55:0/1808973492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01d80b4e70 msgr2=0x7f01d80cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:37.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.177+0000 7f01deb9d700 1 --2- 172.21.15.55:0/1808973492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01d80b4e70 0x7f01d80cb780 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f01c80093f0 tx=0x7f01c8017000 comp rx=0 tx=0).stop 2023-10-06T03:48:37.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.177+0000 7f01deb9d700 1 -- 172.21.15.55:0/1808973492 shutdown_connections 2023-10-06T03:48:37.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.177+0000 7f01deb9d700 1 --2- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01bc066f80 0x7f01bc069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.177+0000 7f01deb9d700 1 --2- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01d815d240 0x7f01d80cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.177+0000 7f01deb9d700 1 --2- 172.21.15.55:0/1808973492 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01d80b16d0 0x7f01d8158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.177+0000 7f01deb9d700 1 --2- 172.21.15.55:0/1808973492 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01d80b4e70 0x7f01d80cb780 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.177+0000 7f01deb9d700 1 -- 172.21.15.55:0/1808973492 >> 172.21.15.55:0/1808973492 conn(0x7f01d80d10c0 msgr2=0x7f01d8158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:37.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.177+0000 7f01deb9d700 1 -- 172.21.15.55:0/1808973492 shutdown_connections 2023-10-06T03:48:37.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.177+0000 7f01deb9d700 1 -- 172.21.15.55:0/1808973492 wait complete. 2023-10-06T03:48:37.199 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T03:48:37.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.389+0000 7fc43ffff700 1 Processor -- start 2023-10-06T03:48:37.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.389+0000 7fc43ffff700 1 -- start start 2023-10-06T03:48:37.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.391+0000 7fc43ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc4400b4e70 0x7fc4400ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:37.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.392+0000 7fc43ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc4400b16d0 0x7fc4400b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:37.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.392+0000 7fc43ffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4400b2080 con 0x7fc44015d240 2023-10-06T03:48:37.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.392+0000 7fc43ffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4400cfb00 con 0x7fc4400b4e70 2023-10-06T03:48:37.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.392+0000 7fc43ffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4400b5c40 con 0x7fc4400b16d0 2023-10-06T03:48:37.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.392+0000 7fc43effd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc4400b4e70 0x7fc4400ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:37.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.392+0000 7fc43effd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc4400b4e70 0x7fc4400ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47016/0 (socket says 172.21.15.55:47016) 2023-10-06T03:48:37.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.392+0000 7fc43effd700 1 -- 172.21.15.55:0/3462975402 learned_addr learned my addr 172.21.15.55:0/3462975402 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:37.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.392+0000 7fc43e7fc700 1 --2- 172.21.15.55:0/3462975402 >> v2:172.21.15.163:3300/0 conn(0x7fc4400b16d0 0x7fc4400b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:37.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.392+0000 7fc43effd700 1 -- 172.21.15.55:0/3462975402 >> v2:172.21.15.163:3300/0 conn(0x7fc4400b16d0 msgr2=0x7fc4400b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:37.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.392+0000 7fc43effd700 1 --2- 172.21.15.55:0/3462975402 >> v2:172.21.15.163:3300/0 conn(0x7fc4400b16d0 0x7fc4400b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.392+0000 7fc43effd700 1 -- 172.21.15.55:0/3462975402 >> v1:172.21.15.163:6789/0 conn(0x7fc44015d240 legacy=0x7fc44015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:37.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.392+0000 7fc43effd700 1 -- 172.21.15.55:0/3462975402 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4400b5f20 con 0x7fc4400b4e70 2023-10-06T03:48:37.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.393+0000 7fc43effd700 1 --2- 172.21.15.55:0/3462975402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4400b4e70 0x7fc4400ccff0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc440089830 tx=0x7fc430009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82016c244ef37d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:37.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.393+0000 7fc43dffb700 1 -- 172.21.15.55:0/3462975402 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000b860 con 0x7fc4400b4e70 2023-10-06T03:48:37.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.393+0000 7fc43dffb700 1 -- 172.21.15.55:0/3462975402 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43000b9c0 con 0x7fc4400b4e70 2023-10-06T03:48:37.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.393+0000 7fc43dffb700 1 -- 172.21.15.55:0/3462975402 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000bd60 con 0x7fc4400b4e70 2023-10-06T03:48:37.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.393+0000 7fc43ffff700 1 -- 172.21.15.55:0/3462975402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4400b4e70 msgr2=0x7fc4400ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:37.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.394+0000 7fc43ffff700 1 --2- 172.21.15.55:0/3462975402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4400b4e70 0x7fc4400ccff0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc440089830 tx=0x7fc430009250 comp rx=0 tx=0).stop 2023-10-06T03:48:37.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.394+0000 7fc43ffff700 1 -- 172.21.15.55:0/3462975402 shutdown_connections 2023-10-06T03:48:37.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.394+0000 7fc43ffff700 1 --2- 172.21.15.55:0/3462975402 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4400b4e70 0x7fc4400ccff0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.394+0000 7fc43ffff700 1 --2- 172.21.15.55:0/3462975402 >> v2:172.21.15.163:3300/0 conn(0x7fc4400b16d0 0x7fc4400b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.394+0000 7fc43ffff700 1 -- 172.21.15.55:0/3462975402 >> 172.21.15.55:0/3462975402 conn(0x7fc4400d10c0 msgr2=0x7fc4400affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:37.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.394+0000 7fc43ffff700 1 -- 172.21.15.55:0/3462975402 shutdown_connections 2023-10-06T03:48:37.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.394+0000 7fc43ffff700 1 -- 172.21.15.55:0/3462975402 wait complete. 2023-10-06T03:48:37.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.396+0000 7fc43ffff700 1 Processor -- start 2023-10-06T03:48:37.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.396+0000 7fc43ffff700 1 -- start start 2023-10-06T03:48:37.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.398+0000 7fc43ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4400b16d0 0x7fc4400c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:37.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.398+0000 7fc43ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4400b4e70 0x7fc440157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:37.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.398+0000 7fc43ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc44015d240 0x7fc44015e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:37.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.398+0000 7fc43ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44016d420 con 0x7fc44015d240 2023-10-06T03:48:37.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.398+0000 7fc43ffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc440157cb0 con 0x7fc4400b16d0 2023-10-06T03:48:37.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.398+0000 7fc43ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44016d0f0 con 0x7fc4400b4e70 2023-10-06T03:48:37.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.398+0000 7fc43f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc44015d240 0x7fc44015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:37.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.398+0000 7fc43effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4400b16d0 0x7fc4400c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:37.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.398+0000 7fc43f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc44015d240 0x7fc44015e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47030/0 (socket says 172.21.15.55:47030) 2023-10-06T03:48:37.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.398+0000 7fc43effd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4400b16d0 0x7fc4400c9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:47742/0 (socket says 172.21.15.55:47742) 2023-10-06T03:48:37.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.398+0000 7fc43e7fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4400b4e70 0x7fc440157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:37.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.398+0000 7fc43f7fe700 1 -- 172.21.15.55:0/198529403 learned_addr learned my addr 172.21.15.55:0/198529403 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:37.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.399+0000 7fc43f7fe700 1 -- 172.21.15.55:0/198529403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4400b4e70 msgr2=0x7fc440157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:37.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.399+0000 7fc43f7fe700 1 --2- 172.21.15.55:0/198529403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4400b4e70 0x7fc440157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.399+0000 7fc43f7fe700 1 -- 172.21.15.55:0/198529403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4400b16d0 msgr2=0x7fc4400c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:37.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.399+0000 7fc43f7fe700 1 --2- 172.21.15.55:0/198529403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4400b16d0 0x7fc4400c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.399+0000 7fc43f7fe700 1 -- 172.21.15.55:0/198529403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4300094f0 con 0x7fc44015d240 2023-10-06T03:48:37.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.399+0000 7fc43f7fe700 1 --2- 172.21.15.55:0/198529403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc44015d240 0x7fc44015e630 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc43800c840 tx=0x7fc438001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:37.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.399+0000 7fc426ffd700 1 -- 172.21.15.55:0/198529403 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438012630 con 0x7fc44015d240 2023-10-06T03:48:37.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.399+0000 7fc426ffd700 1 -- 172.21.15.55:0/198529403 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4380127b0 con 0x7fc44015d240 2023-10-06T03:48:37.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.399+0000 7fc43ffff700 1 -- 172.21.15.55:0/198529403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc440158720 con 0x7fc44015d240 2023-10-06T03:48:37.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.399+0000 7fc426ffd700 1 -- 172.21.15.55:0/198529403 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438012b70 con 0x7fc44015d240 2023-10-06T03:48:37.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.399+0000 7fc43ffff700 1 -- 172.21.15.55:0/198529403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4400c9ed0 con 0x7fc44015d240 2023-10-06T03:48:37.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.401+0000 7fc43ffff700 1 -- 172.21.15.55:0/198529403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44016d590 con 0x7fc44015d240 2023-10-06T03:48:37.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.401+0000 7fc426ffd700 1 -- 172.21.15.55:0/198529403 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 27) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fc43801a3a0 con 0x7fc44015d240 2023-10-06T03:48:37.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.403+0000 7fc426ffd700 1 --2- 172.21.15.55:0/198529403 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc42c066f80 0x7fc42c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:37.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.403+0000 7fc426ffd700 1 -- 172.21.15.55:0/198529403 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fc438084a50 con 0x7fc44015d240 2023-10-06T03:48:37.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.403+0000 7fc43effd700 1 -- 172.21.15.55:0/198529403 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc42c066f80 msgr2=0x7fc42c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:37.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.403+0000 7fc43effd700 1 --2- 172.21.15.55:0/198529403 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc42c066f80 0x7fc42c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:37.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.406+0000 7fc426ffd700 1 -- 172.21.15.55:0/198529403 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc438051930 con 0x7fc44015d240 2023-10-06T03:48:37.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.604+0000 7fc43effd700 1 -- 172.21.15.55:0/198529403 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc42c066f80 msgr2=0x7fc42c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:37.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.604+0000 7fc43effd700 1 --2- 172.21.15.55:0/198529403 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc42c066f80 0x7fc42c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:37.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.610+0000 7fc43ffff700 1 -- 172.21.15.55:0/198529403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fc44004bb80 con 0x7fc44015d240 2023-10-06T03:48:37.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.616+0000 7fc426ffd700 1 -- 172.21.15.55:0/198529403 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc438054f50 con 0x7fc44015d240 2023-10-06T03:48:37.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.618+0000 7fc43ffff700 1 -- 172.21.15.55:0/198529403 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc42c066f80 msgr2=0x7fc42c069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:37.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.618+0000 7fc43ffff700 1 --2- 172.21.15.55:0/198529403 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc42c066f80 0x7fc42c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.618+0000 7fc43ffff700 1 -- 172.21.15.55:0/198529403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc44015d240 msgr2=0x7fc44015e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:37.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.618+0000 7fc43ffff700 1 --2- 172.21.15.55:0/198529403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc44015d240 0x7fc44015e630 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc43800c840 tx=0x7fc438001670 comp rx=0 tx=0).stop 2023-10-06T03:48:37.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.618+0000 7fc43ffff700 1 -- 172.21.15.55:0/198529403 shutdown_connections 2023-10-06T03:48:37.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.618+0000 7fc43ffff700 1 --2- 172.21.15.55:0/198529403 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc42c066f80 0x7fc42c069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.618+0000 7fc43ffff700 1 --2- 172.21.15.55:0/198529403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc44015d240 0x7fc44015e630 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.618+0000 7fc43ffff700 1 --2- 172.21.15.55:0/198529403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4400b16d0 0x7fc4400c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.618+0000 7fc43ffff700 1 --2- 172.21.15.55:0/198529403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4400b4e70 0x7fc440157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.618+0000 7fc43ffff700 1 -- 172.21.15.55:0/198529403 >> 172.21.15.55:0/198529403 conn(0x7fc4400d10c0 msgr2=0x7fc440158db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:37.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.618+0000 7fc43ffff700 1 -- 172.21.15.55:0/198529403 shutdown_connections 2023-10-06T03:48:37.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.618+0000 7fc43ffff700 1 -- 172.21.15.55:0/198529403 wait complete. 2023-10-06T03:48:37.640 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:48:37.640 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T03:48:37.682 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-10-06T03:48:37.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.915+0000 7fad8cf9d700 1 Processor -- start 2023-10-06T03:48:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.915+0000 7fad8cf9d700 1 -- start start 2023-10-06T03:48:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.916+0000 7fad8cf9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fad880ccb40 0x7fad880697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.916+0000 7fad8cf9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad88069cf0 0x7fad8815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.917+0000 7fad8cf9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad88159db0 con 0x7fad880cb560 2023-10-06T03:48:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.917+0000 7fad8cf9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8806aa30 con 0x7fad880ccb40 2023-10-06T03:48:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.917+0000 7fad8cf9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8806a470 con 0x7fad88069cf0 2023-10-06T03:48:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.917+0000 7fad86ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fad880ccb40 0x7fad880697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.917+0000 7fad86ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fad880ccb40 0x7fad880697b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47040/0 (socket says 172.21.15.55:47040) 2023-10-06T03:48:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.917+0000 7fad86ffd700 1 -- 172.21.15.55:0/1394123430 learned_addr learned my addr 172.21.15.55:0/1394123430 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.917+0000 7fad87fff700 1 --2- 172.21.15.55:0/1394123430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad88069cf0 0x7fad8815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.917+0000 7fad86ffd700 1 -- 172.21.15.55:0/1394123430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad88069cf0 msgr2=0x7fad8815ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.917+0000 7fad86ffd700 1 --2- 172.21.15.55:0/1394123430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad88069cf0 0x7fad8815ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.917+0000 7fad86ffd700 1 -- 172.21.15.55:0/1394123430 >> v1:172.21.15.163:6789/0 conn(0x7fad880cb560 legacy=0x7fad880cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.917+0000 7fad86ffd700 1 -- 172.21.15.55:0/1394123430 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8806a750 con 0x7fad880ccb40 2023-10-06T03:48:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.917+0000 7fad867fc700 1 -- 172.21.15.55:0/1394123430 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549956327 0 0) 0x7fad88159db0 con 0x7fad880cb560 2023-10-06T03:48:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.918+0000 7fad86ffd700 1 --2- 172.21.15.55:0/1394123430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad880ccb40 0x7fad880697b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0099e0 tx=0x7fad7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf518a258746326c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.918+0000 7fad867fc700 1 -- 172.21.15.55:0/1394123430 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c00b820 con 0x7fad880ccb40 2023-10-06T03:48:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.918+0000 7fad867fc700 1 -- 172.21.15.55:0/1394123430 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c00b980 con 0x7fad880ccb40 2023-10-06T03:48:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.918+0000 7fad867fc700 1 -- 172.21.15.55:0/1394123430 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c00bd20 con 0x7fad880ccb40 2023-10-06T03:48:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.918+0000 7fad8cf9d700 1 -- 172.21.15.55:0/1394123430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad880ccb40 msgr2=0x7fad880697b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.918+0000 7fad8cf9d700 1 --2- 172.21.15.55:0/1394123430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad880ccb40 0x7fad880697b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0099e0 tx=0x7fad7c009250 comp rx=0 tx=0).stop 2023-10-06T03:48:37.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.919+0000 7fad8cf9d700 1 -- 172.21.15.55:0/1394123430 shutdown_connections 2023-10-06T03:48:37.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.919+0000 7fad8cf9d700 1 --2- 172.21.15.55:0/1394123430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad88069cf0 0x7fad8815ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.919+0000 7fad8cf9d700 1 --2- 172.21.15.55:0/1394123430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad880ccb40 0x7fad880697b0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.919+0000 7fad8cf9d700 1 -- 172.21.15.55:0/1394123430 >> 172.21.15.55:0/1394123430 conn(0x7fad8806baf0 msgr2=0x7fad8806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:37.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.919+0000 7fad8cf9d700 1 -- 172.21.15.55:0/1394123430 shutdown_connections 2023-10-06T03:48:37.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.919+0000 7fad8cf9d700 1 -- 172.21.15.55:0/1394123430 wait complete. 2023-10-06T03:48:37.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.920+0000 7fad8cf9d700 1 Processor -- start 2023-10-06T03:48:37.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.920+0000 7fad8cf9d700 1 -- start start 2023-10-06T03:48:37.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.921+0000 7fad8cf9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad88069cf0 0x7fad88156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:37.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.921+0000 7fad8cf9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad880cb560 0x7fad88154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:37.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.921+0000 7fad8cf9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad880ccb40 0x7fad88151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.921+0000 7fad8cf9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad88156620 con 0x7fad880ccb40 2023-10-06T03:48:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.921+0000 7fad8cf9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad88155a10 con 0x7fad88069cf0 2023-10-06T03:48:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.921+0000 7fad8cf9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad88155cf0 con 0x7fad880cb560 2023-10-06T03:48:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.922+0000 7fad877fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad88069cf0 0x7fad88156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.922+0000 7fad86ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad880cb560 0x7fad88154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.922+0000 7fad87fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad880ccb40 0x7fad88151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.922+0000 7fad86ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad880cb560 0x7fad88154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:39476/0 (socket says 172.21.15.55:39476) 2023-10-06T03:48:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.922+0000 7fad877fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad88069cf0 0x7fad88156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:47748/0 (socket says 172.21.15.55:47748) 2023-10-06T03:48:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.922+0000 7fad86ffd700 1 -- 172.21.15.55:0/2562240711 learned_addr learned my addr 172.21.15.55:0/2562240711 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.922+0000 7fad86ffd700 1 -- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad88069cf0 msgr2=0x7fad88156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:37.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.922+0000 7fad86ffd700 1 --2- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad88069cf0 0x7fad88156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.922+0000 7fad86ffd700 1 -- 172.21.15.55:0/2562240711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad880ccb40 msgr2=0x7fad88151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:37.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.922+0000 7fad86ffd700 1 --2- 172.21.15.55:0/2562240711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad880ccb40 0x7fad88151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:37.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.922+0000 7fad86ffd700 1 -- 172.21.15.55:0/2562240711 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c0094f0 con 0x7fad880cb560 2023-10-06T03:48:37.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.922+0000 7fad86ffd700 1 --2- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad880cb560 0x7fad88154f90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fad8806ce10 tx=0x7fad7c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:37.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.923+0000 7fad6f7fe700 1 -- 172.21.15.55:0/2562240711 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c01f070 con 0x7fad880cb560 2023-10-06T03:48:37.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.923+0000 7fad6f7fe700 1 -- 172.21.15.55:0/2562240711 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c002f70 con 0x7fad880cb560 2023-10-06T03:48:37.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.923+0000 7fad8cf9d700 1 -- 172.21.15.55:0/2562240711 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad88155fd0 con 0x7fad880cb560 2023-10-06T03:48:37.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.923+0000 7fad6f7fe700 1 -- 172.21.15.55:0/2562240711 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c00b820 con 0x7fad880cb560 2023-10-06T03:48:37.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.923+0000 7fad8cf9d700 1 -- 172.21.15.55:0/2562240711 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad88154ae0 con 0x7fad880cb560 2023-10-06T03:48:37.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.924+0000 7fad8cf9d700 1 -- 172.21.15.55:0/2562240711 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad88156790 con 0x7fad880cb560 2023-10-06T03:48:37.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.925+0000 7fad6f7fe700 1 -- 172.21.15.55:0/2562240711 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 27) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fad7c023070 con 0x7fad880cb560 2023-10-06T03:48:37.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.925+0000 7fad6f7fe700 1 --2- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 0x7fad74069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:37.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.926+0000 7fad6f7fe700 1 -- 172.21.15.55:0/2562240711 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fad7c085af0 con 0x7fad880cb560 2023-10-06T03:48:37.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.926+0000 7fad877fe700 1 -- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 msgr2=0x7fad74069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:37.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.926+0000 7fad877fe700 1 --2- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 0x7fad74069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:37.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:37.930+0000 7fad6f7fe700 1 -- 172.21.15.55:0/2562240711 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad7c052a70 con 0x7fad880cb560 2023-10-06T03:48:38.019 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:38.017+0000 7ff4b583f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:48:38.098 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:38.096+0000 7f957713a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:48:38.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:38.127+0000 7fad877fe700 1 -- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 msgr2=0x7fad74069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:38.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:38.127+0000 7fad877fe700 1 --2- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 0x7fad74069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:38.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:38.162+0000 7fad8cf9d700 1 -- 172.21.15.55:0/2562240711 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fad8804c430 con 0x7fad74066fb0 2023-10-06T03:48:38.181 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:38.179+0000 7ff4b583f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:48:38.268 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:38.266+0000 7ff4b583f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:48:38.269 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:38.267+0000 7f957713a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:48:38.358 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:38.355+0000 7f957713a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:48:38.493 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:38.491+0000 7ff4b583f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:48:38.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:38.528+0000 7fad877fe700 1 -- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 msgr2=0x7fad74069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:38.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:38.528+0000 7fad877fe700 1 --2- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 0x7fad74069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T03:48:38.590 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:38.587+0000 7f957713a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:48:38.670 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:38.667+0000 7ff4b583f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:48:38.762 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:38.760+0000 7ff4b583f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:48:38.773 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:38.771+0000 7f957713a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:48:38.869 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:38.867+0000 7f957713a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:48:38.925 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:38.923+0000 7ff4b583f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:48:39.030 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:39.028+0000 7ff4b583f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:48:39.038 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:39.035+0000 7f957713a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:48:39.142 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:39.140+0000 7f957713a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:48:39.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:39.329+0000 7fad877fe700 1 -- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 msgr2=0x7fad74069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:39.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:39.329+0000 7fad877fe700 1 --2- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 0x7fad74069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T03:48:39.515 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:39.513+0000 7ff4b583f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:48:39.608 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:39.606+0000 7ff4b583f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:48:39.644 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:39.641+0000 7f957713a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:48:39.698 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:39.696+0000 7ff4b583f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:48:39.741 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:39.739+0000 7f957713a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:48:39.835 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:39.833+0000 7f957713a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:48:40.318 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:40.316+0000 7ff4b583f180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:48:40.480 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:40.478+0000 7f957713a180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:48:40.490 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:40.487+0000 7ff4b583f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:48:40.585 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:40.583+0000 7ff4b583f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:48:40.658 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:40.657+0000 7f957713a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:48:40.758 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:40.755+0000 7f957713a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:48:40.879 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:40.877+0000 7ff4b583f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:48:40.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:40.931+0000 7fad877fe700 1 -- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 msgr2=0x7fad74069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:40.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:40.931+0000 7fad877fe700 1 --2- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 0x7fad74069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T03:48:40.976 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:40.974+0000 7ff4b583f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:48:41.059 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:41.057+0000 7f957713a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:48:41.159 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:41.156+0000 7f957713a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:48:41.679 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:41.677+0000 7ff4b583f180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:48:41.893 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:41.891+0000 7f957713a180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:48:41.965 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:41.962+0000 7ff4b583f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:48:42.056 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.054+0000 7ff4b583f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:48:42.183 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:42.182+0000 7f957713a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:48:42.210 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.208+0000 7ff4b583f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:48:42.280 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:42.278+0000 7f957713a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:48:42.315 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.313+0000 7ff4b583f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:48:42.439 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:42.438+0000 7f957713a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:48:42.505 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.503+0000 7ff4b583f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:48:42.983 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:42.546+0000 7f957713a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:48:42.983 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:42.742+0000 7f957713a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:48:42.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:42.796+0000 7fad6f7fe700 1 -- 172.21.15.55:0/2562240711 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mgrmap(e 28) v1 ==== 84468+0+0 (secure 0 0 0) 0x7fad7c052e40 con 0x7fad880cb560 2023-10-06T03:48:42.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:42.796+0000 7fad6f7fe700 1 -- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 msgr2=0x7fad74069460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:42.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:42.796+0000 7fad6f7fe700 1 --2- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 0x7fad74069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:42.984 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:42.844+0000 7f957713a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:48:42.984 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:42.949+0000 7f957713a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:48:42.985 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.594+0000 7ff4b583f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:48:42.985 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.690+0000 7ff4b583f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:48:42.985 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.783+0000 7ff4b583f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:48:42.985 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.926+0000 7ff42a3b6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:42.985 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.926+0000 7ff42a3b6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:42.985 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.926+0000 7ff42a3b6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:42.985 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.926+0000 7ff42a3b6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:42.986 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.926+0000 7ff42a3b6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:42.986 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.926+0000 7ff427bb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:42.986 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.926+0000 7ff427bb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:42.986 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.926+0000 7ff427bb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:42.986 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.926+0000 7ff427bb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:42.986 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:42.926+0000 7ff427bb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:43.047 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:43.045+0000 7f957713a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:48:43.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:43.804+0000 7fad6f7fe700 1 -- 172.21.15.55:0/2562240711 <== mon.2 v2:172.21.15.163:3301/0 8 ==== mgrmap(e 29) v1 ==== 84595+0+0 (secure 0 0 0) 0x7fad7c05d240 con 0x7fad880cb560 2023-10-06T03:48:43.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:43.805+0000 7fad6f7fe700 1 --2- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 0x7fad740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:43.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:43.805+0000 7fad6f7fe700 1 -- 172.21.15.55:0/2562240711 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fad8804c430 con 0x7fad74066fb0 2023-10-06T03:48:43.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:43.808+0000 7fad877fe700 1 --2- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 0x7fad740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:43.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:43.810+0000 7fad877fe700 1 --2- 172.21.15.55:0/2562240711 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fad74066fb0 0x7fad740693e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fad78007fd0 tx=0x7fad78007d30 comp rx=0 tx=0).ready entity=mgr.5759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:43.820 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:43.818+0000 7ff431bc5700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-06T03:48:43.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:43.819+0000 7fad6f7fe700 1 -- 172.21.15.55:0/2562240711 <== mgr.5759 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7fad8804c430 con 0x7fad74066fb0 2023-10-06T03:48:43.822 INFO:teuthology.orchestra.run.smithi055.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-06T03:48:43.830 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T03:48:43.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:48:43.831 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-06T03:48:43.872 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T03:48:44.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.103+0000 7f1c7a39d700 1 Processor -- start 2023-10-06T03:48:44.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.103+0000 7f1c7a39d700 1 -- start start 2023-10-06T03:48:44.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.104+0000 7f1c7a39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c740ccc00 0x7f1c74069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:44.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.104+0000 7f1c7a39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1c7406a020 0x7f1c740cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:44.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.104+0000 7f1c7a39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c740c9f30 con 0x7f1c74159a60 2023-10-06T03:48:44.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.104+0000 7f1c7a39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7406dfc0 con 0x7f1c7406a020 2023-10-06T03:48:44.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.104+0000 7f1c7a39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740ca470 con 0x7f1c740ccc00 2023-10-06T03:48:44.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.105+0000 7f1c78b9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c740ccc00 0x7f1c74069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:44.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.105+0000 7f1c79b9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1c7406a020 0x7f1c740cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:44.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.105+0000 7f1c78b9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c740ccc00 0x7f1c74069ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:39668/0 (socket says 172.21.15.55:39668) 2023-10-06T03:48:44.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.105+0000 7f1c78b9a700 1 -- 172.21.15.55:0/871157282 learned_addr learned my addr 172.21.15.55:0/871157282 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:44.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.105+0000 7f1c7939b700 1 --1- 172.21.15.55:0/871157282 >> v1:172.21.15.163:6789/0 conn(0x7f1c74159a60 0x7f1c740c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:34158/0 (socket says 172.21.15.55:34158) 2023-10-06T03:48:44.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.105+0000 7f1c78b9a700 1 -- 172.21.15.55:0/871157282 >> v2:172.21.15.163:3300/0 conn(0x7f1c7406a020 msgr2=0x7f1c740cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:44.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.105+0000 7f1c78b9a700 1 --2- 172.21.15.55:0/871157282 >> v2:172.21.15.163:3300/0 conn(0x7f1c7406a020 0x7f1c740cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:44.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.105+0000 7f1c78b9a700 1 -- 172.21.15.55:0/871157282 >> v1:172.21.15.163:6789/0 conn(0x7f1c74159a60 legacy=0x7f1c740c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:44.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.105+0000 7f1c78b9a700 1 -- 172.21.15.55:0/871157282 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c740ca750 con 0x7f1c740ccc00 2023-10-06T03:48:44.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.105+0000 7f1c79b9c700 1 --2- 172.21.15.55:0/871157282 >> v2:172.21.15.163:3300/0 conn(0x7f1c7406a020 0x7f1c740cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:44.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.106+0000 7f1c78b9a700 1 --2- 172.21.15.55:0/871157282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c740ccc00 0x7f1c74069ae0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c640099b0 tx=0x7f1c64009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd1028bb0b86d9e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:44.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.106+0000 7f1c6bfff700 1 -- 172.21.15.55:0/871157282 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c64009ea0 con 0x7f1c740ccc00 2023-10-06T03:48:44.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.106+0000 7f1c6bfff700 1 -- 172.21.15.55:0/871157282 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6400b8f0 con 0x7f1c740ccc00 2023-10-06T03:48:44.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.106+0000 7f1c6bfff700 1 -- 172.21.15.55:0/871157282 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6400bc90 con 0x7f1c740ccc00 2023-10-06T03:48:44.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.106+0000 7f1c7a39d700 1 -- 172.21.15.55:0/871157282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c740ccc00 msgr2=0x7f1c74069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:44.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.106+0000 7f1c7a39d700 1 --2- 172.21.15.55:0/871157282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c740ccc00 0x7f1c74069ae0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1c640099b0 tx=0x7f1c64009250 comp rx=0 tx=0).stop 2023-10-06T03:48:44.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.107+0000 7f1c7a39d700 1 -- 172.21.15.55:0/871157282 shutdown_connections 2023-10-06T03:48:44.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.107+0000 7f1c7a39d700 1 --2- 172.21.15.55:0/871157282 >> v2:172.21.15.163:3300/0 conn(0x7f1c7406a020 0x7f1c740cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:44.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.107+0000 7f1c7a39d700 1 --2- 172.21.15.55:0/871157282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c740ccc00 0x7f1c74069ae0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:44.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.107+0000 7f1c7a39d700 1 -- 172.21.15.55:0/871157282 >> 172.21.15.55:0/871157282 conn(0x7f1c7406baa0 msgr2=0x7f1c7406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:44.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.107+0000 7f1c7a39d700 1 -- 172.21.15.55:0/871157282 shutdown_connections 2023-10-06T03:48:44.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.107+0000 7f1c7a39d700 1 -- 172.21.15.55:0/871157282 wait complete. 2023-10-06T03:48:44.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.108+0000 7f1c7a39d700 1 Processor -- start 2023-10-06T03:48:44.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.108+0000 7f1c7a39d700 1 -- start start 2023-10-06T03:48:44.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.109+0000 7f1c7a39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c7406a020 0x7f1c740b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:44.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.109+0000 7f1c7a39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c740ccc00 0x7f1c74169d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:44.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.109+0000 7f1c7a39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c74159a60 0x7f1c740c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:44.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.109+0000 7f1c7a39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c75d0 con 0x7f1c74159a60 2023-10-06T03:48:44.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.109+0000 7f1c7a39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740b1970 con 0x7f1c740ccc00 2023-10-06T03:48:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.109+0000 7f1c7a39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c740c69c0 con 0x7f1c7406a020 2023-10-06T03:48:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.109+0000 7f1c79b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c74159a60 0x7f1c740c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.109+0000 7f1c79b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c74159a60 0x7f1c740c4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56916/0 (socket says 172.21.15.55:56916) 2023-10-06T03:48:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.109+0000 7f1c79b9c700 1 -- 172.21.15.55:0/4134529304 learned_addr learned my addr 172.21.15.55:0/4134529304 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.109+0000 7f1c78b9a700 1 --2- 172.21.15.55:0/4134529304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c740ccc00 0x7f1c74169d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.109+0000 7f1c7939b700 1 --2- 172.21.15.55:0/4134529304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c7406a020 0x7f1c740b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.110+0000 7f1c79b9c700 1 -- 172.21.15.55:0/4134529304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c7406a020 msgr2=0x7f1c740b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.110+0000 7f1c79b9c700 1 --2- 172.21.15.55:0/4134529304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c7406a020 0x7f1c740b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.110+0000 7f1c79b9c700 1 -- 172.21.15.55:0/4134529304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c740ccc00 msgr2=0x7f1c74169d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.110+0000 7f1c79b9c700 1 --2- 172.21.15.55:0/4134529304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c740ccc00 0x7f1c74169d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.110+0000 7f1c79b9c700 1 -- 172.21.15.55:0/4134529304 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c009a20 con 0x7f1c74159a60 2023-10-06T03:48:44.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.110+0000 7f1c79b9c700 1 --2- 172.21.15.55:0/4134529304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c74159a60 0x7f1c740c4400 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009fc0 tx=0x7f1c6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:44.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.110+0000 7f1c697fa700 1 -- 172.21.15.55:0/4134529304 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c00cbd0 con 0x7f1c74159a60 2023-10-06T03:48:44.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.110+0000 7f1c697fa700 1 -- 172.21.15.55:0/4134529304 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6c00f040 con 0x7f1c74159a60 2023-10-06T03:48:44.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.110+0000 7f1c697fa700 1 -- 172.21.15.55:0/4134529304 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6c014480 con 0x7f1c74159a60 2023-10-06T03:48:44.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.110+0000 7f1c7a39d700 1 -- 172.21.15.55:0/4134529304 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c640094f0 con 0x7f1c74159a60 2023-10-06T03:48:44.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.111+0000 7f1c7a39d700 1 -- 172.21.15.55:0/4134529304 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c740c6f80 con 0x7f1c74159a60 2023-10-06T03:48:44.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.111+0000 7f1c7a39d700 1 -- 172.21.15.55:0/4134529304 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7406eba0 con 0x7f1c74159a60 2023-10-06T03:48:44.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.112+0000 7f1c697fa700 1 -- 172.21.15.55:0/4134529304 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 29) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f1c6c003020 con 0x7f1c74159a60 2023-10-06T03:48:44.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.113+0000 7f1c697fa700 1 --2- 172.21.15.55:0/4134529304 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c58066dd0 0x7f1c58069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:44.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.113+0000 7f1c697fa700 1 -- 172.21.15.55:0/4134529304 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f1c6c085080 con 0x7f1c74159a60 2023-10-06T03:48:44.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.114+0000 7f1c7939b700 1 --2- 172.21.15.55:0/4134529304 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c58066dd0 0x7f1c58069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:44.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.115+0000 7f1c7939b700 1 --2- 172.21.15.55:0/4134529304 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c58066dd0 0x7f1c58069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1c70004f40 tx=0x7f1c70007eb0 comp rx=0 tx=0).ready entity=mgr.5759 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:44.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.118+0000 7f1c697fa700 1 -- 172.21.15.55:0/4134529304 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c6c0520e0 con 0x7f1c74159a60 2023-10-06T03:48:44.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.376+0000 7f1c7a39d700 1 -- 172.21.15.55:0/4134529304 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f1c740c7710 con 0x7f1c74159a60 2023-10-06T03:48:44.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.852+0000 7f1c697fa700 1 -- 172.21.15.55:0/4134529304 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 30) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f1c6c0524b0 con 0x7f1c74159a60 2023-10-06T03:48:44.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.853+0000 7f1c697fa700 1 -- 172.21.15.55:0/4134529304 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v30) v1 ==== 93+0+0 (secure 0 0 0) 0x7f1c6c055b70 con 0x7f1c74159a60 2023-10-06T03:48:44.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.856+0000 7f1c7a39d700 1 -- 172.21.15.55:0/4134529304 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c58066dd0 msgr2=0x7f1c58069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:44.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.856+0000 7f1c7a39d700 1 --2- 172.21.15.55:0/4134529304 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c58066dd0 0x7f1c58069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1c70004f40 tx=0x7f1c70007eb0 comp rx=0 tx=0).stop 2023-10-06T03:48:44.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.856+0000 7f1c7a39d700 1 -- 172.21.15.55:0/4134529304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c74159a60 msgr2=0x7f1c740c4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:44.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.856+0000 7f1c7a39d700 1 --2- 172.21.15.55:0/4134529304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c74159a60 0x7f1c740c4400 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c009fc0 tx=0x7f1c6c00b040 comp rx=0 tx=0).stop 2023-10-06T03:48:44.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.856+0000 7f1c7a39d700 1 -- 172.21.15.55:0/4134529304 shutdown_connections 2023-10-06T03:48:44.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.856+0000 7f1c7a39d700 1 --2- 172.21.15.55:0/4134529304 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c58066dd0 0x7f1c58069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:44.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.856+0000 7f1c7a39d700 1 --2- 172.21.15.55:0/4134529304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c74159a60 0x7f1c740c4400 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:44.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.856+0000 7f1c7a39d700 1 --2- 172.21.15.55:0/4134529304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c7406a020 0x7f1c740b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:44.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.856+0000 7f1c7a39d700 1 --2- 172.21.15.55:0/4134529304 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c740ccc00 0x7f1c74169d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:44.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.856+0000 7f1c7a39d700 1 -- 172.21.15.55:0/4134529304 >> 172.21.15.55:0/4134529304 conn(0x7f1c7406baa0 msgr2=0x7f1c7406a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:44.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.856+0000 7f1c7a39d700 1 -- 172.21.15.55:0/4134529304 shutdown_connections 2023-10-06T03:48:44.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:44.856+0000 7f1c7a39d700 1 -- 172.21.15.55:0/4134529304 wait complete. 2023-10-06T03:48:44.877 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:48:44.878 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:48:44.961 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:48:45.038 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:48:45.039 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:48:45.039 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:48:45.039 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:48:45.124 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T03:48:45.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:48:45.125 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:48:45.162 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:48:45.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.390+0000 7f209ffff700 1 Processor -- start 2023-10-06T03:48:45.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.391+0000 7f209ffff700 1 -- start start 2023-10-06T03:48:45.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.392+0000 7f209ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20a0159420 0x7f20a00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:45.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.392+0000 7f209ffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a00b2980 con 0x7f20a01597f0 2023-10-06T03:48:45.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.392+0000 7f209ffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a00b11d0 con 0x7f20a015d230 2023-10-06T03:48:45.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.392+0000 7f209ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a00b14b0 con 0x7f20a0159420 2023-10-06T03:48:45.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.392+0000 7f209e7fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f20a01597f0 0x7f20a00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52520/0 (socket says 172.21.15.55:52520) 2023-10-06T03:48:45.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.392+0000 7f209e7fc700 1 -- 172.21.15.55:0/2986761984 learned_addr learned my addr 172.21.15.55:0/2986761984 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:45.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.392+0000 7f209effd700 1 --2- 172.21.15.55:0/2986761984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20a0159420 0x7f20a00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:45.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.393+0000 7f209dffb700 1 -- 172.21.15.55:0/2986761984 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424236367 0 0) 0x7f20a00b11d0 con 0x7f20a015d230 2023-10-06T03:48:45.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.393+0000 7f209dffb700 1 -- 172.21.15.55:0/2986761984 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2088003570 con 0x7f20a015d230 2023-10-06T03:48:45.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.393+0000 7f209dffb700 1 -- 172.21.15.55:0/2986761984 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317427078 0 0) 0x7f20a00b2980 con 0x7f20a01597f0 2023-10-06T03:48:45.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.393+0000 7f209dffb700 1 -- 172.21.15.55:0/2986761984 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a00b11d0 con 0x7f20a01597f0 2023-10-06T03:48:45.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.393+0000 7f209effd700 1 -- 172.21.15.55:0/2986761984 >> v1:172.21.15.163:6789/0 conn(0x7f20a015d230 legacy=0x7f20a00b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:45.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.393+0000 7f209effd700 1 -- 172.21.15.55:0/2986761984 >> v1:172.21.15.55:6789/0 conn(0x7f20a01597f0 legacy=0x7f20a00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:45.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.393+0000 7f209effd700 1 -- 172.21.15.55:0/2986761984 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a00b1790 con 0x7f20a0159420 2023-10-06T03:48:45.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.393+0000 7f209effd700 1 --2- 172.21.15.55:0/2986761984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20a0159420 0x7f20a00b5f40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2090009910 tx=0x7f20900091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f09e89cf2e5c8bc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:45.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.394+0000 7f209dffb700 1 -- 172.21.15.55:0/2986761984 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090011630 con 0x7f20a0159420 2023-10-06T03:48:45.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.394+0000 7f209dffb700 1 -- 172.21.15.55:0/2986761984 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2090011790 con 0x7f20a0159420 2023-10-06T03:48:45.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.394+0000 7f209dffb700 1 -- 172.21.15.55:0/2986761984 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090011b30 con 0x7f20a0159420 2023-10-06T03:48:45.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.394+0000 7f209ffff700 1 -- 172.21.15.55:0/2986761984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20a0159420 msgr2=0x7f20a00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:45.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.394+0000 7f209ffff700 1 --2- 172.21.15.55:0/2986761984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20a0159420 0x7f20a00b5f40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2090009910 tx=0x7f20900091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:45.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.394+0000 7f209ffff700 1 -- 172.21.15.55:0/2986761984 shutdown_connections 2023-10-06T03:48:45.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.394+0000 7f209ffff700 1 --2- 172.21.15.55:0/2986761984 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20a0159420 0x7f20a00b5f40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:45.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.394+0000 7f209ffff700 1 -- 172.21.15.55:0/2986761984 >> 172.21.15.55:0/2986761984 conn(0x7f20a00d0660 msgr2=0x7f20a00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:45.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.394+0000 7f209ffff700 1 -- 172.21.15.55:0/2986761984 shutdown_connections 2023-10-06T03:48:45.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.394+0000 7f209ffff700 1 -- 172.21.15.55:0/2986761984 wait complete. 2023-10-06T03:48:45.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.396+0000 7f209ffff700 1 Processor -- start 2023-10-06T03:48:45.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.396+0000 7f209ffff700 1 -- start start 2023-10-06T03:48:45.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.397+0000 7f209ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20a0159420 0x7f20a0147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:45.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.397+0000 7f209ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20a01597f0 0x7f20a00c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:45.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.397+0000 7f209ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20a015d230 0x7f20a00c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:45.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.397+0000 7f209ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a015b7c0 con 0x7f20a01597f0 2023-10-06T03:48:45.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.397+0000 7f209ffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a01566e0 con 0x7f20a015d230 2023-10-06T03:48:45.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.397+0000 7f209ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a015ae90 con 0x7f20a0159420 2023-10-06T03:48:45.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.397+0000 7f209e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20a01597f0 0x7f20a00c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:45.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.397+0000 7f209effd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20a0159420 0x7f20a0147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:45.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.397+0000 7f209e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20a01597f0 0x7f20a00c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56930/0 (socket says 172.21.15.55:56930) 2023-10-06T03:48:45.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.397+0000 7f209e7fc700 1 -- 172.21.15.55:0/1945871119 learned_addr learned my addr 172.21.15.55:0/1945871119 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:45.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.397+0000 7f209f7fe700 1 --2- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20a015d230 0x7f20a00c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:45.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.398+0000 7f209effd700 1 -- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20a015d230 msgr2=0x7f20a00c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:45.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.398+0000 7f209effd700 1 --2- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20a015d230 0x7f20a00c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:45.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.398+0000 7f209effd700 1 -- 172.21.15.55:0/1945871119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20a01597f0 msgr2=0x7f20a00c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:45.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.398+0000 7f209effd700 1 --2- 172.21.15.55:0/1945871119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20a01597f0 0x7f20a00c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:45.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.398+0000 7f209effd700 1 -- 172.21.15.55:0/1945871119 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2090009420 con 0x7f20a0159420 2023-10-06T03:48:45.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.398+0000 7f209e7fc700 1 --2- 172.21.15.55:0/1945871119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20a01597f0 0x7f20a00c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:45.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.398+0000 7f209effd700 1 --2- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20a0159420 0x7f20a0147210 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2090016000 tx=0x7f2090017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:45.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.398+0000 7f2086ffd700 1 -- 172.21.15.55:0/1945871119 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209001f070 con 0x7f20a0159420 2023-10-06T03:48:45.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.398+0000 7f2086ffd700 1 -- 172.21.15.55:0/1945871119 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2090002f70 con 0x7f20a0159420 2023-10-06T03:48:45.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.399+0000 7f209ffff700 1 -- 172.21.15.55:0/1945871119 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a015b170 con 0x7f20a0159420 2023-10-06T03:48:45.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.399+0000 7f2086ffd700 1 -- 172.21.15.55:0/1945871119 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2090011900 con 0x7f20a0159420 2023-10-06T03:48:45.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.399+0000 7f209ffff700 1 -- 172.21.15.55:0/1945871119 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a0155980 con 0x7f20a0159420 2023-10-06T03:48:45.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.400+0000 7f209ffff700 1 -- 172.21.15.55:0/1945871119 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a004c3c0 con 0x7f20a0159420 2023-10-06T03:48:45.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.401+0000 7f2086ffd700 1 -- 172.21.15.55:0/1945871119 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 30) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f2090023070 con 0x7f20a0159420 2023-10-06T03:48:45.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.402+0000 7f2086ffd700 1 --2- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f208c0670e0 0x7f208c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:45.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.402+0000 7f2086ffd700 1 -- 172.21.15.55:0/1945871119 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f2090085a50 con 0x7f20a0159420 2023-10-06T03:48:45.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.402+0000 7f209e7fc700 1 -- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f208c0670e0 msgr2=0x7f208c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:45.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.402+0000 7f209e7fc700 1 --2- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f208c0670e0 0x7f208c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:45.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.411+0000 7f2086ffd700 1 -- 172.21.15.55:0/1945871119 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2090052670 con 0x7f20a0159420 2023-10-06T03:48:45.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.603+0000 7f209e7fc700 1 -- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f208c0670e0 msgr2=0x7f208c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:45.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.603+0000 7f209e7fc700 1 --2- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f208c0670e0 0x7f208c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:45.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.659+0000 7f209ffff700 1 -- 172.21.15.55:0/1945871119 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20a00b1a70 con 0x7f20a0159420 2023-10-06T03:48:45.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.660+0000 7f2086ffd700 1 -- 172.21.15.55:0/1945871119 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f2090015090 con 0x7f20a0159420 2023-10-06T03:48:45.662 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:45.662 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4046797285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":4046797285},{"type":"v1","addr":"172.21.15.55:6835","nonce":4046797285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5645,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/675796170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":675796170},{"type":"v1","addr":"172.21.15.163:6836","nonce":675796170}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5648,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2543497851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2543497851},{"type":"v1","addr":"172.21.15.163:6834","nonce":2543497851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:48:22.429407+0000","modified":"2023-10-06T03:48:25.973506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6333":{"gid":6333,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3434145175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3434145175},{"type":"v1","addr":"172.21.15.55:6834","nonce":3434145175}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T03:48:45.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.663+0000 7f209ffff700 1 -- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f208c0670e0 msgr2=0x7f208c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:45.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.663+0000 7f209ffff700 1 --2- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f208c0670e0 0x7f208c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:45.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.663+0000 7f209ffff700 1 -- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20a0159420 msgr2=0x7f20a0147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:45.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.663+0000 7f209ffff700 1 --2- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20a0159420 0x7f20a0147210 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2090016000 tx=0x7f2090017000 comp rx=0 tx=0).stop 2023-10-06T03:48:45.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.663+0000 7f209ffff700 1 -- 172.21.15.55:0/1945871119 shutdown_connections 2023-10-06T03:48:45.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.663+0000 7f209ffff700 1 --2- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f208c0670e0 0x7f208c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:45.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.663+0000 7f209ffff700 1 --2- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20a015d230 0x7f20a00c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:45.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.663+0000 7f209ffff700 1 --2- 172.21.15.55:0/1945871119 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20a0159420 0x7f20a0147210 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:45.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.663+0000 7f209ffff700 1 --2- 172.21.15.55:0/1945871119 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f20a01597f0 0x7f20a00c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:45.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.663+0000 7f209ffff700 1 -- 172.21.15.55:0/1945871119 >> 172.21.15.55:0/1945871119 conn(0x7f20a00d0660 msgr2=0x7f20a00b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:45.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.663+0000 7f209ffff700 1 -- 172.21.15.55:0/1945871119 shutdown_connections 2023-10-06T03:48:45.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.663+0000 7f209ffff700 1 -- 172.21.15.55:0/1945871119 wait complete. 2023-10-06T03:48:45.666 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 35 2023-10-06T03:48:45.680 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:48:45.682 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:48:45.685 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:48:45.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.878+0000 7fb5789fe700 1 Processor -- start 2023-10-06T03:48:45.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.878+0000 7fb5789fe700 1 -- start start 2023-10-06T03:48:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.879+0000 7fb5789fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb574156310 0x7fb5740c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.879+0000 7fb5789fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb5741566e0 0x7fb5740ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.879+0000 7fb5789fe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5740ca860 con 0x7fb5740ca300 2023-10-06T03:48:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.879+0000 7fb5789fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5740cab40 con 0x7fb5741566e0 2023-10-06T03:48:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.879+0000 7fb5789fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5740cae20 con 0x7fb574156310 2023-10-06T03:48:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.879+0000 7fb57259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb5741566e0 0x7fb5740ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.880+0000 7fb57259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb5741566e0 0x7fb5740ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56932/0 (socket says 172.21.15.55:56932) 2023-10-06T03:48:45.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.880+0000 7fb57259c700 1 -- 172.21.15.55:0/637771524 learned_addr learned my addr 172.21.15.55:0/637771524 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:45.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.880+0000 7fb572d9d700 1 --2- 172.21.15.55:0/637771524 >> v2:172.21.15.163:3300/0 conn(0x7fb574156310 0x7fb5740c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:45.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.880+0000 7fb57259c700 1 -- 172.21.15.55:0/637771524 >> v2:172.21.15.163:3300/0 conn(0x7fb574156310 msgr2=0x7fb5740c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:45.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.880+0000 7fb57259c700 1 --2- 172.21.15.55:0/637771524 >> v2:172.21.15.163:3300/0 conn(0x7fb574156310 0x7fb5740c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:45.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.880+0000 7fb57259c700 1 -- 172.21.15.55:0/637771524 >> v1:172.21.15.163:6789/0 conn(0x7fb5740ca300 legacy=0x7fb5740c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:45.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.880+0000 7fb57259c700 1 -- 172.21.15.55:0/637771524 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5740cb100 con 0x7fb5741566e0 2023-10-06T03:48:45.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.880+0000 7fb572d9d700 1 --2- 172.21.15.55:0/637771524 >> v2:172.21.15.163:3300/0 conn(0x7fb574156310 0x7fb5740c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:45.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.881+0000 7fb57259c700 1 --2- 172.21.15.55:0/637771524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5741566e0 0x7fb5740ae000 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb5740cce20 tx=0x7fb568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5520bb1c5b5ec93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:45.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.881+0000 7fb571d9b700 1 -- 172.21.15.55:0/637771524 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb568009e70 con 0x7fb5741566e0 2023-10-06T03:48:45.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.881+0000 7fb571d9b700 1 -- 172.21.15.55:0/637771524 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56800b860 con 0x7fb5741566e0 2023-10-06T03:48:45.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.881+0000 7fb571d9b700 1 -- 172.21.15.55:0/637771524 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56800bc00 con 0x7fb5741566e0 2023-10-06T03:48:45.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.882+0000 7fb5789fe700 1 -- 172.21.15.55:0/637771524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5741566e0 msgr2=0x7fb5740ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:45.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.882+0000 7fb5789fe700 1 --2- 172.21.15.55:0/637771524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5741566e0 0x7fb5740ae000 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb5740cce20 tx=0x7fb568009250 comp rx=0 tx=0).stop 2023-10-06T03:48:45.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.882+0000 7fb5789fe700 1 -- 172.21.15.55:0/637771524 shutdown_connections 2023-10-06T03:48:45.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.882+0000 7fb5789fe700 1 --2- 172.21.15.55:0/637771524 >> v2:172.21.15.163:3300/0 conn(0x7fb574156310 0x7fb5740c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:45.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.882+0000 7fb5789fe700 1 --2- 172.21.15.55:0/637771524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5741566e0 0x7fb5740ae000 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:45.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.882+0000 7fb5789fe700 1 -- 172.21.15.55:0/637771524 >> 172.21.15.55:0/637771524 conn(0x7fb5740cc1f0 msgr2=0x7fb5740cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:45.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.882+0000 7fb5789fe700 1 -- 172.21.15.55:0/637771524 shutdown_connections 2023-10-06T03:48:45.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.882+0000 7fb5789fe700 1 -- 172.21.15.55:0/637771524 wait complete. 2023-10-06T03:48:45.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.883+0000 7fb5789fe700 1 Processor -- start 2023-10-06T03:48:45.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.883+0000 7fb5789fe700 1 -- start start 2023-10-06T03:48:45.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.884+0000 7fb5789fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5740ca300 0x7fb574155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:45.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.884+0000 7fb5789fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb574156310 0x7fb5740aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:45.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.884+0000 7fb5789fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5741566e0 0x7fb5741405b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:45.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.884+0000 7fb5789fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740b03d0 con 0x7fb5741566e0 2023-10-06T03:48:45.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.884+0000 7fb5789fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740af7c0 con 0x7fb5740ca300 2023-10-06T03:48:45.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.884+0000 7fb5789fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740afaa0 con 0x7fb574156310 2023-10-06T03:48:45.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.885+0000 7fb572d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5740ca300 0x7fb574155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:45.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.885+0000 7fb572d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5740ca300 0x7fb574155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:41118/0 (socket says 172.21.15.55:41118) 2023-10-06T03:48:45.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.885+0000 7fb57259c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb574156310 0x7fb5740aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:45.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.885+0000 7fb572d9d700 1 -- 172.21.15.55:0/2267674404 learned_addr learned my addr 172.21.15.55:0/2267674404 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:45.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.885+0000 7fb57359e700 1 --2- 172.21.15.55:0/2267674404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5741566e0 0x7fb5741405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:45.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.885+0000 7fb572d9d700 1 -- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb574156310 msgr2=0x7fb5740aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:45.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.885+0000 7fb572d9d700 1 --2- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb574156310 0x7fb5740aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:45.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.885+0000 7fb572d9d700 1 -- 172.21.15.55:0/2267674404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5741566e0 msgr2=0x7fb5741405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:45.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.885+0000 7fb572d9d700 1 --2- 172.21.15.55:0/2267674404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5741566e0 0x7fb5741405b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:45.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.885+0000 7fb572d9d700 1 -- 172.21.15.55:0/2267674404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb564009420 con 0x7fb5740ca300 2023-10-06T03:48:45.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.885+0000 7fb57259c700 1 --2- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb574156310 0x7fb5740aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:45.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.886+0000 7fb572d9d700 1 --2- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5740ca300 0x7fb574155820 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb56400ec80 tx=0x7fb5640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:45.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.886+0000 7fb55affd700 1 -- 172.21.15.55:0/2267674404 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56400c920 con 0x7fb5740ca300 2023-10-06T03:48:45.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.886+0000 7fb55affd700 1 -- 172.21.15.55:0/2267674404 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56400ca80 con 0x7fb5740ca300 2023-10-06T03:48:45.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.886+0000 7fb5789fe700 1 -- 172.21.15.55:0/2267674404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5680094f0 con 0x7fb5740ca300 2023-10-06T03:48:45.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.886+0000 7fb55affd700 1 -- 172.21.15.55:0/2267674404 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56400ce00 con 0x7fb5740ca300 2023-10-06T03:48:45.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.886+0000 7fb5789fe700 1 -- 172.21.15.55:0/2267674404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5740b0060 con 0x7fb5740ca300 2023-10-06T03:48:45.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.887+0000 7fb5789fe700 1 -- 172.21.15.55:0/2267674404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5740af590 con 0x7fb5740ca300 2023-10-06T03:48:45.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.888+0000 7fb55affd700 1 -- 172.21.15.55:0/2267674404 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 30) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fb564013680 con 0x7fb5740ca300 2023-10-06T03:48:45.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.889+0000 7fb55affd700 1 --2- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb5600670e0 0x7fb560069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:45.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.889+0000 7fb55affd700 1 -- 172.21.15.55:0/2267674404 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6708+0+0 (secure 0 0 0) 0x7fb564085350 con 0x7fb5740ca300 2023-10-06T03:48:45.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.889+0000 7fb57259c700 1 -- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb5600670e0 msgr2=0x7fb560069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:45.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.889+0000 7fb57259c700 1 --2- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb5600670e0 0x7fb560069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:45.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:45.893+0000 7fb55affd700 1 -- 172.21.15.55:0/2267674404 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb564051f50 con 0x7fb5740ca300 2023-10-06T03:48:46.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.090+0000 7fb57259c700 1 -- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb5600670e0 msgr2=0x7fb560069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:46.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.090+0000 7fb57259c700 1 --2- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb5600670e0 0x7fb560069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:46.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.106+0000 7fb5789fe700 1 -- 172.21.15.55:0/2267674404 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb57404c3c0 con 0x7fb5740ca300 2023-10-06T03:48:46.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.107+0000 7fb55affd700 1 -- 172.21.15.55:0/2267674404 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+17883 (secure 0 0 0) 0x7fb564055570 con 0x7fb5740ca300 2023-10-06T03:48:46.109 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:46.109 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":81,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:48:42.787202+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:48:20.608889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T03:48:21.410367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/85191":"2023-10-07T03:48:42.787170+0000","172.21.15.163:6835/85191":"2023-10-07T03:48:42.787170+0000","172.21.15.163:0/596976820":"2023-10-07T03:48:42.787170+0000","172.21.15.163:0/4293045398":"2023-10-07T03:48:42.787170+0000","172.21.15.163:0/1262983566":"2023-10-07T03:48:42.787170+0000","172.21.15.163:0/3262439175":"2023-10-07T03:48:42.787170+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:48:46.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.110+0000 7fb5789fe700 1 -- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb5600670e0 msgr2=0x7fb560069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:46.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.110+0000 7fb5789fe700 1 --2- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb5600670e0 0x7fb560069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.110+0000 7fb5789fe700 1 -- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5740ca300 msgr2=0x7fb574155820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:46.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.110+0000 7fb5789fe700 1 --2- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5740ca300 0x7fb574155820 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb56400ec80 tx=0x7fb5640091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.110+0000 7fb5789fe700 1 -- 172.21.15.55:0/2267674404 shutdown_connections 2023-10-06T03:48:46.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.110+0000 7fb5789fe700 1 --2- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb5600670e0 0x7fb560069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.110+0000 7fb5789fe700 1 --2- 172.21.15.55:0/2267674404 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5741566e0 0x7fb5741405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.110+0000 7fb5789fe700 1 --2- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5740ca300 0x7fb574155820 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.110+0000 7fb5789fe700 1 --2- 172.21.15.55:0/2267674404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb574156310 0x7fb5740aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.110+0000 7fb5789fe700 1 -- 172.21.15.55:0/2267674404 >> 172.21.15.55:0/2267674404 conn(0x7fb5740cc1f0 msgr2=0x7fb57406ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:46.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.110+0000 7fb5789fe700 1 -- 172.21.15.55:0/2267674404 shutdown_connections 2023-10-06T03:48:46.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.110+0000 7fb5789fe700 1 -- 172.21.15.55:0/2267674404 wait complete. 2023-10-06T03:48:46.130 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:48:20.608889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:48:21.410367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:48:46.130 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:48:46.171 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:46.169+0000 7fe1f5450180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:48:46.222 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:46.218+0000 7f4fc328e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:48:46.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.325+0000 7f1211d9d700 1 Processor -- start 2023-10-06T03:48:46.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.326+0000 7f1211d9d700 1 -- start start 2023-10-06T03:48:46.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.327+0000 7f1211d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f120c0b4e70 0x7f120c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:46.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.327+0000 7f1211d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120c0cfb00 con 0x7f120c0b16d0 2023-10-06T03:48:46.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.327+0000 7f1211d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120c15e060 con 0x7f120c0b1b40 2023-10-06T03:48:46.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.327+0000 7f1211d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120c0b5c40 con 0x7f120c0b4e70 2023-10-06T03:48:46.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.327+0000 7f120bfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f120c0b16d0 0x7f120c15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52530/0 (socket says 172.21.15.55:52530) 2023-10-06T03:48:46.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.327+0000 7f120bfff700 1 -- 172.21.15.55:0/1958001035 learned_addr learned my addr 172.21.15.55:0/1958001035 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:46.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.327+0000 7f1210d9b700 1 --2- 172.21.15.55:0/1958001035 >> v2:172.21.15.163:3300/0 conn(0x7f120c0b4e70 0x7f120c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:46.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.328+0000 7f120b7fe700 1 -- 172.21.15.55:0/1958001035 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965478559 0 0) 0x7f120c0cfb00 con 0x7f120c0b16d0 2023-10-06T03:48:46.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.328+0000 7f1210d9b700 1 -- 172.21.15.55:0/1958001035 >> v1:172.21.15.163:6789/0 conn(0x7f120c0b1b40 legacy=0x7f120c15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:46.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.328+0000 7f1210d9b700 1 -- 172.21.15.55:0/1958001035 >> v1:172.21.15.55:6789/0 conn(0x7f120c0b16d0 legacy=0x7f120c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:46.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.328+0000 7f1210d9b700 1 -- 172.21.15.55:0/1958001035 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c0b5f20 con 0x7f120c0b4e70 2023-10-06T03:48:46.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.328+0000 7f1210d9b700 1 --2- 172.21.15.55:0/1958001035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f120c0b4e70 0x7f120c0ccff0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1200009860 tx=0x7f12000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51fb6594d7a7f6c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:46.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.328+0000 7f120b7fe700 1 -- 172.21.15.55:0/1958001035 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1200011630 con 0x7f120c0b4e70 2023-10-06T03:48:46.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.328+0000 7f120b7fe700 1 -- 172.21.15.55:0/1958001035 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1200011790 con 0x7f120c0b4e70 2023-10-06T03:48:46.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.328+0000 7f120b7fe700 1 -- 172.21.15.55:0/1958001035 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1200011b50 con 0x7f120c0b4e70 2023-10-06T03:48:46.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.329+0000 7f1211d9d700 1 -- 172.21.15.55:0/1958001035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f120c0b4e70 msgr2=0x7f120c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:46.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.329+0000 7f1211d9d700 1 --2- 172.21.15.55:0/1958001035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f120c0b4e70 0x7f120c0ccff0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1200009860 tx=0x7f12000091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.329+0000 7f1211d9d700 1 -- 172.21.15.55:0/1958001035 shutdown_connections 2023-10-06T03:48:46.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.329+0000 7f1211d9d700 1 --2- 172.21.15.55:0/1958001035 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f120c0b4e70 0x7f120c0ccff0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.329+0000 7f1211d9d700 1 -- 172.21.15.55:0/1958001035 >> 172.21.15.55:0/1958001035 conn(0x7f120c0d10c0 msgr2=0x7f120c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:46.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.329+0000 7f1211d9d700 1 -- 172.21.15.55:0/1958001035 shutdown_connections 2023-10-06T03:48:46.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.329+0000 7f1211d9d700 1 -- 172.21.15.55:0/1958001035 wait complete. 2023-10-06T03:48:46.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.330+0000 7f1211d9d700 1 Processor -- start 2023-10-06T03:48:46.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.331+0000 7f1211d9d700 1 -- start start 2023-10-06T03:48:46.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.331+0000 7f1211d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f120c0b16d0 0x7f120c0c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:46.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.331+0000 7f1211d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f120c0b1b40 0x7f120c159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:46.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.331+0000 7f1211d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f120c0b4e70 0x7f120c15a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:46.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.331+0000 7f1211d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0c6040 con 0x7f120c0b16d0 2023-10-06T03:48:46.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.331+0000 7f1211d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0c5430 con 0x7f120c0b4e70 2023-10-06T03:48:46.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.331+0000 7f1211d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0c5710 con 0x7f120c0b1b40 2023-10-06T03:48:46.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.332+0000 7f1210d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f120c0b16d0 0x7f120c0c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:46.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.332+0000 7f1210d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f120c0b16d0 0x7f120c0c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56950/0 (socket says 172.21.15.55:56950) 2023-10-06T03:48:46.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.332+0000 7f1210d9b700 1 -- 172.21.15.55:0/1235455290 learned_addr learned my addr 172.21.15.55:0/1235455290 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:46.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.332+0000 7f120bfff700 1 --2- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f120c0b1b40 0x7f120c159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:46.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.332+0000 7f121159c700 1 --2- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f120c0b4e70 0x7f120c15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:46.335 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:46.333+0000 7fe1f5450180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:48:46.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.332+0000 7f1210d9b700 1 -- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f120c0b1b40 msgr2=0x7f120c159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:46.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.332+0000 7f1210d9b700 1 --2- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f120c0b1b40 0x7f120c159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.332+0000 7f1210d9b700 1 -- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f120c0b4e70 msgr2=0x7f120c15a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:46.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.332+0000 7f1210d9b700 1 --2- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f120c0b4e70 0x7f120c15a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.332+0000 7f1210d9b700 1 -- 172.21.15.55:0/1235455290 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1200009420 con 0x7f120c0b16d0 2023-10-06T03:48:46.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.333+0000 7f1210d9b700 1 --2- 172.21.15.55:0/1235455290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f120c0b16d0 0x7f120c0c7a10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1200009fb0 tx=0x7f1200016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:46.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.333+0000 7f1208ff9700 1 -- 172.21.15.55:0/1235455290 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1200011b10 con 0x7f120c0b16d0 2023-10-06T03:48:46.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.333+0000 7f1211d9d700 1 -- 172.21.15.55:0/1235455290 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120c0c59f0 con 0x7f120c0b16d0 2023-10-06T03:48:46.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.333+0000 7f1208ff9700 1 -- 172.21.15.55:0/1235455290 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1200006a90 con 0x7f120c0b16d0 2023-10-06T03:48:46.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.333+0000 7f1208ff9700 1 -- 172.21.15.55:0/1235455290 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120001a6a0 con 0x7f120c0b16d0 2023-10-06T03:48:46.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.333+0000 7f1211d9d700 1 -- 172.21.15.55:0/1235455290 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c0c64d0 con 0x7f120c0b16d0 2023-10-06T03:48:46.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.334+0000 7f1211d9d700 1 -- 172.21.15.55:0/1235455290 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120c088990 con 0x7f120c0b16d0 2023-10-06T03:48:46.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.335+0000 7f1208ff9700 1 -- 172.21.15.55:0/1235455290 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 30) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f120001aa90 con 0x7f120c0b16d0 2023-10-06T03:48:46.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.336+0000 7f1208ff9700 1 --2- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f00670e0 0x7f11f0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:46.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.336+0000 7f1208ff9700 1 -- 172.21.15.55:0/1235455290 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f1200086c70 con 0x7f120c0b16d0 2023-10-06T03:48:46.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.336+0000 7f120bfff700 1 -- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f00670e0 msgr2=0x7f11f0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:46.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.336+0000 7f120bfff700 1 --2- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f00670e0 0x7f11f0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:46.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.344+0000 7f1208ff9700 1 -- 172.21.15.55:0/1235455290 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1200053810 con 0x7f120c0b16d0 2023-10-06T03:48:46.399 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:46.398+0000 7f4fc328e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:48:46.422 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:46.420+0000 7fe1f5450180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:48:46.488 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:46.486+0000 7f4fc328e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:48:46.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.537+0000 7f120bfff700 1 -- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f00670e0 msgr2=0x7f11f0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:46.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.537+0000 7f120bfff700 1 --2- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f00670e0 0x7f11f0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:46.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.548+0000 7f1211d9d700 1 -- 172.21.15.55:0/1235455290 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f120c0b4c50 con 0x7f120c0b16d0 2023-10-06T03:48:46.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.548+0000 7f1208ff9700 1 -- 172.21.15.55:0/1235455290 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v81) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1200029020 con 0x7f120c0b16d0 2023-10-06T03:48:46.550 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:48:46.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.551+0000 7f1211d9d700 1 -- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f00670e0 msgr2=0x7f11f0069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:46.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.551+0000 7f1211d9d700 1 --2- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f00670e0 0x7f11f0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.551+0000 7f1211d9d700 1 -- 172.21.15.55:0/1235455290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f120c0b16d0 msgr2=0x7f120c0c7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:46.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.551+0000 7f1211d9d700 1 --2- 172.21.15.55:0/1235455290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f120c0b16d0 0x7f120c0c7a10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f1200009fb0 tx=0x7f1200016000 comp rx=0 tx=0).stop 2023-10-06T03:48:46.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.551+0000 7f1211d9d700 1 -- 172.21.15.55:0/1235455290 shutdown_connections 2023-10-06T03:48:46.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.551+0000 7f1211d9d700 1 --2- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f00670e0 0x7f11f0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.551+0000 7f1211d9d700 1 --2- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f120c0b4e70 0x7f120c15a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.551+0000 7f1211d9d700 1 --2- 172.21.15.55:0/1235455290 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f120c0b16d0 0x7f120c0c7a10 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.551+0000 7f1211d9d700 1 --2- 172.21.15.55:0/1235455290 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f120c0b1b40 0x7f120c159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.551+0000 7f1211d9d700 1 -- 172.21.15.55:0/1235455290 >> 172.21.15.55:0/1235455290 conn(0x7f120c0d10c0 msgr2=0x7f120c0cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:46.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.551+0000 7f1211d9d700 1 -- 172.21.15.55:0/1235455290 shutdown_connections 2023-10-06T03:48:46.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.551+0000 7f1211d9d700 1 -- 172.21.15.55:0/1235455290 wait complete. 2023-10-06T03:48:46.572 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:48:46.646 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:46.644+0000 7fe1f5450180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:48:46.728 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:46.726+0000 7f4fc328e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:48:46.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.767+0000 7f801f39d700 1 Processor -- start 2023-10-06T03:48:46.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.767+0000 7f801f39d700 1 -- start start 2023-10-06T03:48:46.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.768+0000 7f801f39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f801815a3b0 0x7f801815a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:46.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.768+0000 7f801f39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f80180aef00 0x7f801815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:46.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.768+0000 7f801f39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80180aea70 con 0x7f801806e470 2023-10-06T03:48:46.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.768+0000 7f801f39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80180ae480 con 0x7f80180aef00 2023-10-06T03:48:46.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.768+0000 7f801f39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f801815acc0 con 0x7f801815a3b0 2023-10-06T03:48:46.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.769+0000 7f801db9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f80180aef00 0x7f801815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:46.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.769+0000 7f801db9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f80180aef00 0x7f801815d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56960/0 (socket says 172.21.15.55:56960) 2023-10-06T03:48:46.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.769+0000 7f801db9a700 1 -- 172.21.15.55:0/3385792179 learned_addr learned my addr 172.21.15.55:0/3385792179 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:46.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.769+0000 7f801e39b700 1 --2- 172.21.15.55:0/3385792179 >> v2:172.21.15.163:3300/0 conn(0x7f801815a3b0 0x7f801815a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:46.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.769+0000 7f801db9a700 1 -- 172.21.15.55:0/3385792179 >> v2:172.21.15.163:3300/0 conn(0x7f801815a3b0 msgr2=0x7f801815a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:46.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.769+0000 7f801db9a700 1 --2- 172.21.15.55:0/3385792179 >> v2:172.21.15.163:3300/0 conn(0x7f801815a3b0 0x7f801815a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.769+0000 7f801db9a700 1 -- 172.21.15.55:0/3385792179 >> v1:172.21.15.163:6789/0 conn(0x7f801806e470 legacy=0x7f80180c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:46.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.769+0000 7f801db9a700 1 -- 172.21.15.55:0/3385792179 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801815dd30 con 0x7f80180aef00 2023-10-06T03:48:46.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.770+0000 7f801db9a700 1 --2- 172.21.15.55:0/3385792179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80180aef00 0x7f801815d3c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8018063350 tx=0x7f8008009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8605a41c209e191c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:46.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.770+0000 7f801d399700 1 -- 172.21.15.55:0/3385792179 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800800b860 con 0x7f80180aef00 2023-10-06T03:48:46.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.770+0000 7f801d399700 1 -- 172.21.15.55:0/3385792179 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800800b9c0 con 0x7f80180aef00 2023-10-06T03:48:46.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.770+0000 7f801d399700 1 -- 172.21.15.55:0/3385792179 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800800bd60 con 0x7f80180aef00 2023-10-06T03:48:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.770+0000 7f801f39d700 1 -- 172.21.15.55:0/3385792179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80180aef00 msgr2=0x7f801815d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.770+0000 7f801f39d700 1 --2- 172.21.15.55:0/3385792179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80180aef00 0x7f801815d3c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8018063350 tx=0x7f8008009250 comp rx=0 tx=0).stop 2023-10-06T03:48:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.770+0000 7f801f39d700 1 -- 172.21.15.55:0/3385792179 shutdown_connections 2023-10-06T03:48:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.770+0000 7f801f39d700 1 --2- 172.21.15.55:0/3385792179 >> v2:172.21.15.163:3300/0 conn(0x7f801815a3b0 0x7f801815a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.770+0000 7f801f39d700 1 --2- 172.21.15.55:0/3385792179 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80180aef00 0x7f801815d3c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.770+0000 7f801f39d700 1 -- 172.21.15.55:0/3385792179 >> 172.21.15.55:0/3385792179 conn(0x7f80180c9f70 msgr2=0x7f801806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.771+0000 7f801f39d700 1 -- 172.21.15.55:0/3385792179 shutdown_connections 2023-10-06T03:48:46.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.771+0000 7f801f39d700 1 -- 172.21.15.55:0/3385792179 wait complete. 2023-10-06T03:48:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.772+0000 7f801f39d700 1 Processor -- start 2023-10-06T03:48:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.772+0000 7f801f39d700 1 -- start start 2023-10-06T03:48:46.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.773+0000 7f801f39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f801806e470 0x7f801814f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:46.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.773+0000 7f801f39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80180aef00 0x7f80180af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.773+0000 7f801f39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f801815a3b0 0x7f80180c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.773+0000 7f801f39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80180c5590 con 0x7f801806e470 2023-10-06T03:48:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.773+0000 7f801f39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801816df60 con 0x7f80180aef00 2023-10-06T03:48:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.773+0000 7f801f39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80180c4f40 con 0x7f801815a3b0 2023-10-06T03:48:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.773+0000 7f801e39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f801806e470 0x7f801814f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.773+0000 7f801e39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f801806e470 0x7f801814f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56962/0 (socket says 172.21.15.55:56962) 2023-10-06T03:48:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.773+0000 7f801e39b700 1 -- 172.21.15.55:0/2596379583 learned_addr learned my addr 172.21.15.55:0/2596379583 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.773+0000 7f801db9a700 1 --2- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80180aef00 0x7f80180af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.773+0000 7f801eb9c700 1 --2- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f801815a3b0 0x7f80180c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:46.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.774+0000 7f801e39b700 1 -- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f801815a3b0 msgr2=0x7f80180c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:46.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.774+0000 7f801e39b700 1 --2- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f801815a3b0 0x7f80180c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.774+0000 7f801e39b700 1 -- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80180aef00 msgr2=0x7f80180af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:46.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.774+0000 7f801e39b700 1 --2- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80180aef00 0x7f80180af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.774+0000 7f801e39b700 1 -- 172.21.15.55:0/2596379583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080094f0 con 0x7f801806e470 2023-10-06T03:48:46.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.774+0000 7f801e39b700 1 --2- 172.21.15.55:0/2596379583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f801806e470 0x7f801814f030 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f801400d860 tx=0x7f8014008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:46.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.774+0000 7f800e7fc700 1 -- 172.21.15.55:0/2596379583 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80140125f0 con 0x7f801806e470 2023-10-06T03:48:46.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.774+0000 7f800e7fc700 1 -- 172.21.15.55:0/2596379583 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8014012750 con 0x7f801806e470 2023-10-06T03:48:46.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.774+0000 7f801f39d700 1 -- 172.21.15.55:0/2596379583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80180c5220 con 0x7f801806e470 2023-10-06T03:48:46.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.774+0000 7f800e7fc700 1 -- 172.21.15.55:0/2596379583 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8014012b10 con 0x7f801806e470 2023-10-06T03:48:46.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.774+0000 7f801f39d700 1 -- 172.21.15.55:0/2596379583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80180afdb0 con 0x7f801806e470 2023-10-06T03:48:46.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.775+0000 7f801f39d700 1 -- 172.21.15.55:0/2596379583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80180c5870 con 0x7f801806e470 2023-10-06T03:48:46.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.776+0000 7f800e7fc700 1 -- 172.21.15.55:0/2596379583 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 30) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f801401e460 con 0x7f801806e470 2023-10-06T03:48:46.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.777+0000 7f800e7fc700 1 --2- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ffc0670e0 0x7f7ffc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:46.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.777+0000 7f800e7fc700 1 -- 172.21.15.55:0/2596379583 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f8014085210 con 0x7f801806e470 2023-10-06T03:48:46.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.777+0000 7f801db9a700 1 -- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ffc0670e0 msgr2=0x7f7ffc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:46.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.777+0000 7f801db9a700 1 --2- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ffc0670e0 0x7f7ffc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.785+0000 7f800e7fc700 1 -- 172.21.15.55:0/2596379583 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8014055800 con 0x7f801806e470 2023-10-06T03:48:46.823 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:46.821+0000 7fe1f5450180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:48:46.918 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:46.917+0000 7f4fc328e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:48:46.919 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:46.917+0000 7fe1f5450180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:48:46.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.978+0000 7f801db9a700 1 -- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ffc0670e0 msgr2=0x7f7ffc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:46.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.978+0000 7f801db9a700 1 --2- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ffc0670e0 0x7f7ffc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:46.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.993+0000 7f801f39d700 1 -- 172.21.15.55:0/2596379583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f801815e010 con 0x7f801806e470 2023-10-06T03:48:46.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.993+0000 7f800e7fc700 1 -- 172.21.15.55:0/2596379583 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8014055390 con 0x7f801806e470 2023-10-06T03:48:46.995 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:48:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.996+0000 7f801f39d700 1 -- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ffc0670e0 msgr2=0x7f7ffc069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.996+0000 7f801f39d700 1 --2- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ffc0670e0 0x7f7ffc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.996+0000 7f801f39d700 1 -- 172.21.15.55:0/2596379583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f801806e470 msgr2=0x7f801814f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.996+0000 7f801f39d700 1 --2- 172.21.15.55:0/2596379583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f801806e470 0x7f801814f030 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f801400d860 tx=0x7f8014008040 comp rx=0 tx=0).stop 2023-10-06T03:48:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.996+0000 7f801f39d700 1 -- 172.21.15.55:0/2596379583 shutdown_connections 2023-10-06T03:48:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.996+0000 7f801f39d700 1 --2- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7ffc0670e0 0x7f7ffc069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.996+0000 7f801f39d700 1 --2- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f801815a3b0 0x7f80180c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.996+0000 7f801f39d700 1 --2- 172.21.15.55:0/2596379583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f801806e470 0x7f801814f030 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.996+0000 7f801f39d700 1 --2- 172.21.15.55:0/2596379583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80180aef00 0x7f80180af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:46.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.996+0000 7f801f39d700 1 -- 172.21.15.55:0/2596379583 >> 172.21.15.55:0/2596379583 conn(0x7f80180c9f70 msgr2=0x7f801815bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:46.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.996+0000 7f801f39d700 1 -- 172.21.15.55:0/2596379583 shutdown_connections 2023-10-06T03:48:46.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:46.996+0000 7f801f39d700 1 -- 172.21.15.55:0/2596379583 wait complete. 2023-10-06T03:48:47.016 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:48:47.021 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:47.018+0000 7f4fc328e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:48:47.085 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:47.083+0000 7fe1f5450180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:48:47.185 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:47.183+0000 7fe1f5450180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:48:47.189 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:47.187+0000 7f4fc328e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:48:47.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.211+0000 7fedfffff700 1 Processor -- start 2023-10-06T03:48:47.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.211+0000 7fedfffff700 1 -- start start 2023-10-06T03:48:47.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.212+0000 7fedfffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fee000b4e70 0x7fee000ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:47.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.212+0000 7fedfffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fee000b16d0 0x7fee000b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:47.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.212+0000 7fedfffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee000b2080 con 0x7fee0015d240 2023-10-06T03:48:47.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.212+0000 7fedfffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee000cfb00 con 0x7fee000b4e70 2023-10-06T03:48:47.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.212+0000 7fedfffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee000b5c40 con 0x7fee000b16d0 2023-10-06T03:48:47.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.212+0000 7fedfeffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fee000b4e70 0x7fee000ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:47.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.212+0000 7fedfeffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fee000b4e70 0x7fee000ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56976/0 (socket says 172.21.15.55:56976) 2023-10-06T03:48:47.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.212+0000 7fedfeffd700 1 -- 172.21.15.55:0/3208394951 learned_addr learned my addr 172.21.15.55:0/3208394951 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:47.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.213+0000 7fedfe7fc700 1 --2- 172.21.15.55:0/3208394951 >> v2:172.21.15.163:3300/0 conn(0x7fee000b16d0 0x7fee000b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:47.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.213+0000 7fedfeffd700 1 -- 172.21.15.55:0/3208394951 >> v2:172.21.15.163:3300/0 conn(0x7fee000b16d0 msgr2=0x7fee000b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:47.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.213+0000 7fedfeffd700 1 --2- 172.21.15.55:0/3208394951 >> v2:172.21.15.163:3300/0 conn(0x7fee000b16d0 0x7fee000b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.213+0000 7fedfeffd700 1 -- 172.21.15.55:0/3208394951 >> v1:172.21.15.163:6789/0 conn(0x7fee0015d240 legacy=0x7fee0015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:47.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.213+0000 7fedfeffd700 1 -- 172.21.15.55:0/3208394951 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee000b5f20 con 0x7fee000b4e70 2023-10-06T03:48:47.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.213+0000 7fedfeffd700 1 --2- 172.21.15.55:0/3208394951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee000b4e70 0x7fee000ccff0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fee00089830 tx=0x7fedf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c3c20b02e46d5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:47.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.213+0000 7fedfdffb700 1 -- 172.21.15.55:0/3208394951 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf000b820 con 0x7fee000b4e70 2023-10-06T03:48:47.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.213+0000 7fedfdffb700 1 -- 172.21.15.55:0/3208394951 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf000b980 con 0x7fee000b4e70 2023-10-06T03:48:47.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.213+0000 7fedfdffb700 1 -- 172.21.15.55:0/3208394951 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf000bd20 con 0x7fee000b4e70 2023-10-06T03:48:47.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.214+0000 7fedfffff700 1 -- 172.21.15.55:0/3208394951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee000b4e70 msgr2=0x7fee000ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:47.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.214+0000 7fedfffff700 1 --2- 172.21.15.55:0/3208394951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee000b4e70 0x7fee000ccff0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fee00089830 tx=0x7fedf0009250 comp rx=0 tx=0).stop 2023-10-06T03:48:47.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.214+0000 7fedfffff700 1 -- 172.21.15.55:0/3208394951 shutdown_connections 2023-10-06T03:48:47.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.214+0000 7fedfffff700 1 --2- 172.21.15.55:0/3208394951 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee000b4e70 0x7fee000ccff0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.214+0000 7fedfffff700 1 --2- 172.21.15.55:0/3208394951 >> v2:172.21.15.163:3300/0 conn(0x7fee000b16d0 0x7fee000b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.214+0000 7fedfffff700 1 -- 172.21.15.55:0/3208394951 >> 172.21.15.55:0/3208394951 conn(0x7fee000d10c0 msgr2=0x7fee000affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:47.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.214+0000 7fedfffff700 1 -- 172.21.15.55:0/3208394951 shutdown_connections 2023-10-06T03:48:47.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.214+0000 7fedfffff700 1 -- 172.21.15.55:0/3208394951 wait complete. 2023-10-06T03:48:47.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.216+0000 7fedfffff700 1 Processor -- start 2023-10-06T03:48:47.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.216+0000 7fedfffff700 1 -- start start 2023-10-06T03:48:47.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.217+0000 7fedfffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee000b16d0 0x7fee00158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:47.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.217+0000 7fedfffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee000b4e70 0x7fee0014e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:47.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.217+0000 7fedfffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee0015d240 0x7fee000c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:47.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.217+0000 7fedfffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee000c9860 con 0x7fee000b16d0 2023-10-06T03:48:47.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.217+0000 7fedfffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee000caec0 con 0x7fee000b4e70 2023-10-06T03:48:47.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.217+0000 7fedfffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee00169ad0 con 0x7fee0015d240 2023-10-06T03:48:47.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.217+0000 7fedfeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee000b16d0 0x7fee00158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:47.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.217+0000 7fedfeffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee000b16d0 0x7fee00158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56982/0 (socket says 172.21.15.55:56982) 2023-10-06T03:48:47.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.217+0000 7fedfeffd700 1 -- 172.21.15.55:0/2985078702 learned_addr learned my addr 172.21.15.55:0/2985078702 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:47.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.217+0000 7fedfe7fc700 1 --2- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee000b4e70 0x7fee0014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:47.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.217+0000 7fedff7fe700 1 --2- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee0015d240 0x7fee000c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:47.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.218+0000 7fedfeffd700 1 -- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee0015d240 msgr2=0x7fee000c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:47.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.218+0000 7fedfeffd700 1 --2- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee0015d240 0x7fee000c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.218+0000 7fedfeffd700 1 -- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee000b4e70 msgr2=0x7fee0014e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:47.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.218+0000 7fedfeffd700 1 --2- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee000b4e70 0x7fee0014e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.218+0000 7fedfeffd700 1 -- 172.21.15.55:0/2985078702 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf00094f0 con 0x7fee000b16d0 2023-10-06T03:48:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.218+0000 7fedfeffd700 1 --2- 172.21.15.55:0/2985078702 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee000b16d0 0x7fee00158500 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fee00070dd0 tx=0x7fedf0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.218+0000 7fede6ffd700 1 -- 172.21.15.55:0/2985078702 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf0011e90 con 0x7fee000b16d0 2023-10-06T03:48:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.218+0000 7fede6ffd700 1 -- 172.21.15.55:0/2985078702 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf001b470 con 0x7fee000b16d0 2023-10-06T03:48:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.218+0000 7fedfffff700 1 -- 172.21.15.55:0/2985078702 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee00158b70 con 0x7fee000b16d0 2023-10-06T03:48:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.218+0000 7fede6ffd700 1 -- 172.21.15.55:0/2985078702 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf001b830 con 0x7fee000b16d0 2023-10-06T03:48:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.219+0000 7fedfffff700 1 -- 172.21.15.55:0/2985078702 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee000cbc90 con 0x7fee000b16d0 2023-10-06T03:48:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.219+0000 7fedfffff700 1 -- 172.21.15.55:0/2985078702 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee000cba60 con 0x7fee000b16d0 2023-10-06T03:48:47.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.221+0000 7fede6ffd700 1 -- 172.21.15.55:0/2985078702 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 30) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fedf001bba0 con 0x7fee000b16d0 2023-10-06T03:48:47.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.222+0000 7fede6ffd700 1 --2- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fedec0670e0 0x7fedec069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:47.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.222+0000 7fede6ffd700 1 -- 172.21.15.55:0/2985078702 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6708+0+0 (secure 0 0 0) 0x7fedf0086840 con 0x7fee000b16d0 2023-10-06T03:48:47.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.222+0000 7fedfe7fc700 1 -- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fedec0670e0 msgr2=0x7fedec069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:47.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.223+0000 7fedfe7fc700 1 --2- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fedec0670e0 0x7fedec069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:47.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.227+0000 7fede6ffd700 1 -- 172.21.15.55:0/2985078702 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee000cba60 con 0x7fee000b16d0 2023-10-06T03:48:47.298 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:47.297+0000 7f4fc328e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:48:47.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.424+0000 7fedfe7fc700 1 -- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fedec0670e0 msgr2=0x7fedec069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:47.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.424+0000 7fedfe7fc700 1 --2- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fedec0670e0 0x7fedec069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:47.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.432+0000 7fedfffff700 1 -- 172.21.15.55:0/2985078702 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee000cba60 con 0x7fee000b16d0 2023-10-06T03:48:47.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.432+0000 7fede6ffd700 1 -- 172.21.15.55:0/2985078702 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7fedf002a020 con 0x7fee000b16d0 2023-10-06T03:48:47.434 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T03:48:47.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.435+0000 7fedfffff700 1 -- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fedec0670e0 msgr2=0x7fedec069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.435+0000 7fedfffff700 1 --2- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fedec0670e0 0x7fedec069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.435+0000 7fedfffff700 1 -- 172.21.15.55:0/2985078702 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee000b16d0 msgr2=0x7fee00158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.435+0000 7fedfffff700 1 --2- 172.21.15.55:0/2985078702 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee000b16d0 0x7fee00158500 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fee00070dd0 tx=0x7fedf0016000 comp rx=0 tx=0).stop 2023-10-06T03:48:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.435+0000 7fedfffff700 1 -- 172.21.15.55:0/2985078702 shutdown_connections 2023-10-06T03:48:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.435+0000 7fedfffff700 1 --2- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fedec0670e0 0x7fedec069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.435+0000 7fedfffff700 1 --2- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee0015d240 0x7fee000c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.435+0000 7fedfffff700 1 --2- 172.21.15.55:0/2985078702 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee000b16d0 0x7fee00158500 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.435+0000 7fedfffff700 1 --2- 172.21.15.55:0/2985078702 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee000b4e70 0x7fee0014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.435+0000 7fedfffff700 1 -- 172.21.15.55:0/2985078702 >> 172.21.15.55:0/2985078702 conn(0x7fee000d10c0 msgr2=0x7fee001590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.435+0000 7fedfffff700 1 -- 172.21.15.55:0/2985078702 shutdown_connections 2023-10-06T03:48:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.435+0000 7fedfffff700 1 -- 172.21.15.55:0/2985078702 wait complete. 2023-10-06T03:48:47.455 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:48:47.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.651+0000 7f70ba79d700 1 Processor -- start 2023-10-06T03:48:47.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.651+0000 7f70ba79d700 1 -- start start 2023-10-06T03:48:47.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.652+0000 7f70ba79d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70b40b4e70 0x7f70b40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:47.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.652+0000 7f70ba79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40b16d0 0x7f70b40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:47.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.652+0000 7f70ba79d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b40b2080 con 0x7f70b415d240 2023-10-06T03:48:47.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.653+0000 7f70ba79d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b40cfb00 con 0x7f70b40b4e70 2023-10-06T03:48:47.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.653+0000 7f70ba79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40b5c40 con 0x7f70b40b16d0 2023-10-06T03:48:47.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.653+0000 7f70b8f9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40b16d0 0x7f70b40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:47.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.653+0000 7f70b979b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70b40b4e70 0x7f70b40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:47.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.653+0000 7f70b8f9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40b16d0 0x7f70b40b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:39766/0 (socket says 172.21.15.55:39766) 2023-10-06T03:48:47.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.653+0000 7f70b8f9a700 1 -- 172.21.15.55:0/3598533866 learned_addr learned my addr 172.21.15.55:0/3598533866 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.653+0000 7f70b979b700 1 -- 172.21.15.55:0/3598533866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40b16d0 msgr2=0x7f70b40b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.653+0000 7f70b979b700 1 --2- 172.21.15.55:0/3598533866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40b16d0 0x7f70b40b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.653+0000 7f70b979b700 1 -- 172.21.15.55:0/3598533866 >> v1:172.21.15.163:6789/0 conn(0x7f70b415d240 legacy=0x7f70b415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.653+0000 7f70b979b700 1 -- 172.21.15.55:0/3598533866 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b40b5f20 con 0x7f70b40b4e70 2023-10-06T03:48:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.653+0000 7f70b8f9a700 1 --2- 172.21.15.55:0/3598533866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40b16d0 0x7f70b40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.654+0000 7f70b979b700 1 --2- 172.21.15.55:0/3598533866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40b4e70 0x7f70b40ccff0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f70b00098e0 tx=0x7f70b00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98fd7ee27136fd72 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.654+0000 7f70abfff700 1 -- 172.21.15.55:0/3598533866 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b000b640 con 0x7f70b40b4e70 2023-10-06T03:48:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.654+0000 7f70abfff700 1 -- 172.21.15.55:0/3598533866 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b000b7a0 con 0x7f70b40b4e70 2023-10-06T03:48:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.654+0000 7f70abfff700 1 -- 172.21.15.55:0/3598533866 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b000bb40 con 0x7f70b40b4e70 2023-10-06T03:48:47.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.655+0000 7f70ba79d700 1 -- 172.21.15.55:0/3598533866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40b4e70 msgr2=0x7f70b40ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:47.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.655+0000 7f70ba79d700 1 --2- 172.21.15.55:0/3598533866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40b4e70 0x7f70b40ccff0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f70b00098e0 tx=0x7f70b00091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.655+0000 7f70ba79d700 1 -- 172.21.15.55:0/3598533866 shutdown_connections 2023-10-06T03:48:47.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.655+0000 7f70ba79d700 1 --2- 172.21.15.55:0/3598533866 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40b4e70 0x7f70b40ccff0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.655+0000 7f70ba79d700 1 --2- 172.21.15.55:0/3598533866 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40b16d0 0x7f70b40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.655+0000 7f70ba79d700 1 -- 172.21.15.55:0/3598533866 >> 172.21.15.55:0/3598533866 conn(0x7f70b40d10c0 msgr2=0x7f70b40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:47.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.655+0000 7f70ba79d700 1 -- 172.21.15.55:0/3598533866 shutdown_connections 2023-10-06T03:48:47.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.655+0000 7f70ba79d700 1 -- 172.21.15.55:0/3598533866 wait complete. 2023-10-06T03:48:47.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.656+0000 7f70ba79d700 1 Processor -- start 2023-10-06T03:48:47.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.656+0000 7f70ba79d700 1 -- start start 2023-10-06T03:48:47.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.657+0000 7f70ba79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40b16d0 0x7f70b40cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.657+0000 7f70ba79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40b4e70 0x7f70b4149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.657+0000 7f70ba79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70b415d240 0x7f70b40c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.657+0000 7f70ba79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b4149650 con 0x7f70b415d240 2023-10-06T03:48:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.657+0000 7f70ba79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40c9a90 con 0x7f70b40b4e70 2023-10-06T03:48:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.657+0000 7f70ba79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40c8c00 con 0x7f70b40b16d0 2023-10-06T03:48:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.657+0000 7f70b8f9a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40b4e70 0x7f70b4149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.657+0000 7f70b979b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40b16d0 0x7f70b40cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.658+0000 7f70b8f9a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40b4e70 0x7f70b4149110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:41184/0 (socket says 172.21.15.55:41184) 2023-10-06T03:48:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.658+0000 7f70b8f9a700 1 -- 172.21.15.55:0/437712891 learned_addr learned my addr 172.21.15.55:0/437712891 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.658+0000 7f70b9f9c700 1 --2- 172.21.15.55:0/437712891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70b415d240 0x7f70b40c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.658+0000 7f70b8f9a700 1 -- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40b16d0 msgr2=0x7f70b40cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.658+0000 7f70b8f9a700 1 --2- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40b16d0 0x7f70b40cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.658+0000 7f70b8f9a700 1 -- 172.21.15.55:0/437712891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70b415d240 msgr2=0x7f70b40c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.658+0000 7f70b8f9a700 1 --2- 172.21.15.55:0/437712891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70b415d240 0x7f70b40c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.658+0000 7f70b8f9a700 1 -- 172.21.15.55:0/437712891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a40094f0 con 0x7f70b40b4e70 2023-10-06T03:48:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.658+0000 7f70b8f9a700 1 --2- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40b4e70 0x7f70b4149110 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f70a400ec80 tx=0x7f70a4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.658+0000 7f70a97fa700 1 -- 172.21.15.55:0/437712891 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400c850 con 0x7f70b40b4e70 2023-10-06T03:48:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.659+0000 7f70a97fa700 1 -- 172.21.15.55:0/437712891 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a400c9b0 con 0x7f70b40b4e70 2023-10-06T03:48:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.659+0000 7f70ba79d700 1 -- 172.21.15.55:0/437712891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b0009420 con 0x7f70b40b4e70 2023-10-06T03:48:47.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.659+0000 7f70a97fa700 1 -- 172.21.15.55:0/437712891 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a400cd50 con 0x7f70b40b4e70 2023-10-06T03:48:47.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.659+0000 7f70ba79d700 1 -- 172.21.15.55:0/437712891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b40c9160 con 0x7f70b40b4e70 2023-10-06T03:48:47.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.660+0000 7f70ba79d700 1 -- 172.21.15.55:0/437712891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b4088990 con 0x7f70b40b4e70 2023-10-06T03:48:47.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.661+0000 7f70a97fa700 1 -- 172.21.15.55:0/437712891 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 30) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f70a4013680 con 0x7f70b40b4e70 2023-10-06T03:48:47.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.661+0000 7f70a97fa700 1 --2- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70980670e0 0x7f7098069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:47.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.662+0000 7f70a97fa700 1 -- 172.21.15.55:0/437712891 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f70a4084cd0 con 0x7f70b40b4e70 2023-10-06T03:48:47.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.662+0000 7f70b979b700 1 -- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70980670e0 msgr2=0x7f7098069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:47.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.662+0000 7f70b979b700 1 --2- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70980670e0 0x7f7098069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:47.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.666+0000 7f70a97fa700 1 -- 172.21.15.55:0/437712891 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70a40518d0 con 0x7f70b40b4e70 2023-10-06T03:48:47.669 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:47.667+0000 7fe1f5450180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:48:47.764 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:47.762+0000 7fe1f5450180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:48:47.810 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:47.808+0000 7f4fc328e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:48:47.858 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:47.853+0000 7fe1f5450180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:48:47.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.863+0000 7f70b979b700 1 -- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70980670e0 msgr2=0x7f7098069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:47.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.863+0000 7f70b979b700 1 --2- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70980670e0 0x7f7098069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:47.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.872+0000 7f70ba79d700 1 -- 172.21.15.55:0/437712891 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70b40c9bd0 con 0x7f70b40b4e70 2023-10-06T03:48:47.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.873+0000 7f70a97fa700 1 -- 172.21.15.55:0/437712891 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f70a40170a0 con 0x7f70b40b4e70 2023-10-06T03:48:47.874 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T03:48:47.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.875+0000 7f70ba79d700 1 -- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70980670e0 msgr2=0x7f7098069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:47.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.875+0000 7f70ba79d700 1 --2- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70980670e0 0x7f7098069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.875+0000 7f70ba79d700 1 -- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40b4e70 msgr2=0x7f70b4149110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:47.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.875+0000 7f70ba79d700 1 --2- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40b4e70 0x7f70b4149110 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f70a400ec80 tx=0x7f70a4009250 comp rx=0 tx=0).stop 2023-10-06T03:48:47.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.875+0000 7f70ba79d700 1 -- 172.21.15.55:0/437712891 shutdown_connections 2023-10-06T03:48:47.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.875+0000 7f70ba79d700 1 --2- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70980670e0 0x7f7098069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.875+0000 7f70ba79d700 1 --2- 172.21.15.55:0/437712891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70b415d240 0x7f70b40c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.875+0000 7f70ba79d700 1 --2- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f70b40b16d0 0x7f70b40cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.875+0000 7f70ba79d700 1 --2- 172.21.15.55:0/437712891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70b40b4e70 0x7f70b4149110 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:47.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.875+0000 7f70ba79d700 1 -- 172.21.15.55:0/437712891 >> 172.21.15.55:0/437712891 conn(0x7f70b40d10c0 msgr2=0x7f70b41590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:47.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.875+0000 7f70ba79d700 1 -- 172.21.15.55:0/437712891 shutdown_connections 2023-10-06T03:48:47.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:47.875+0000 7f70ba79d700 1 -- 172.21.15.55:0/437712891 wait complete. 2023-10-06T03:48:47.897 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:48:47.912 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:47.910+0000 7f4fc328e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:48:48.008 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:48.006+0000 7f4fc328e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:48:48.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.085+0000 7f7579b9d700 1 Processor -- start 2023-10-06T03:48:48.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.085+0000 7f7579b9d700 1 -- start start 2023-10-06T03:48:48.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.087+0000 7f7579b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7574159420 0x7f75740b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:48.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.087+0000 7f7579b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75741597f0 0x7f75740b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.087+0000 7f7579b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75740b11d0 con 0x7f757415d230 2023-10-06T03:48:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.087+0000 7f7579b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75740b14b0 con 0x7f7574159420 2023-10-06T03:48:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.087+0000 7f7579b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75740b1790 con 0x7f75741597f0 2023-10-06T03:48:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.087+0000 7f7578b9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7574159420 0x7f75740b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.087+0000 7f7578b9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7574159420 0x7f75740b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57006/0 (socket says 172.21.15.55:57006) 2023-10-06T03:48:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.087+0000 7f7578b9b700 1 -- 172.21.15.55:0/3956491393 learned_addr learned my addr 172.21.15.55:0/3956491393 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:48.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.087+0000 7f7573fff700 1 --2- 172.21.15.55:0/3956491393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75741597f0 0x7f75740b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:48.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.087+0000 7f7578b9b700 1 -- 172.21.15.55:0/3956491393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75741597f0 msgr2=0x7f75740b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:48.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.087+0000 7f7578b9b700 1 --2- 172.21.15.55:0/3956491393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75741597f0 0x7f75740b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.087+0000 7f7578b9b700 1 -- 172.21.15.55:0/3956491393 >> v1:172.21.15.163:6789/0 conn(0x7f757415d230 legacy=0x7f75740b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:48.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.087+0000 7f7578b9b700 1 -- 172.21.15.55:0/3956491393 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75740b1a70 con 0x7f7574159420 2023-10-06T03:48:48.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.088+0000 7f7578b9b700 1 --2- 172.21.15.55:0/3956491393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7574159420 0x7f75740b5f40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f75680099b0 tx=0x7f7568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ebddfae8c7f078 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:48.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.088+0000 7f75737fe700 1 -- 172.21.15.55:0/3956491393 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568009ea0 con 0x7f7574159420 2023-10-06T03:48:48.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.088+0000 7f75737fe700 1 -- 172.21.15.55:0/3956491393 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756800b860 con 0x7f7574159420 2023-10-06T03:48:48.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.088+0000 7f75737fe700 1 -- 172.21.15.55:0/3956491393 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756800bc00 con 0x7f7574159420 2023-10-06T03:48:48.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.088+0000 7f7579b9d700 1 -- 172.21.15.55:0/3956491393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7574159420 msgr2=0x7f75740b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:48.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.088+0000 7f7579b9d700 1 --2- 172.21.15.55:0/3956491393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7574159420 0x7f75740b5f40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f75680099b0 tx=0x7f7568009250 comp rx=0 tx=0).stop 2023-10-06T03:48:48.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.089+0000 7f7579b9d700 1 -- 172.21.15.55:0/3956491393 shutdown_connections 2023-10-06T03:48:48.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.089+0000 7f7579b9d700 1 --2- 172.21.15.55:0/3956491393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7574159420 0x7f75740b5f40 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.089+0000 7f7579b9d700 1 --2- 172.21.15.55:0/3956491393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75741597f0 0x7f75740b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.089+0000 7f7579b9d700 1 -- 172.21.15.55:0/3956491393 >> 172.21.15.55:0/3956491393 conn(0x7f75740d0660 msgr2=0x7f75740d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:48.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.089+0000 7f7579b9d700 1 -- 172.21.15.55:0/3956491393 shutdown_connections 2023-10-06T03:48:48.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.089+0000 7f7579b9d700 1 -- 172.21.15.55:0/3956491393 wait complete. 2023-10-06T03:48:48.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.090+0000 7f7579b9d700 1 Processor -- start 2023-10-06T03:48:48.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.090+0000 7f7579b9d700 1 -- start start 2023-10-06T03:48:48.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.091+0000 7f7579b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7574159420 0x7f7574156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:48.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.091+0000 7f7579b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75741597f0 0x7f757416d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:48.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.091+0000 7f7579b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f757415d230 0x7f757416aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:48.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.091+0000 7f7579b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757416a570 con 0x7f75741597f0 2023-10-06T03:48:48.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.091+0000 7f7579b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757416a240 con 0x7f7574159420 2023-10-06T03:48:48.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.091+0000 7f7579b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7574156790 con 0x7f757415d230 2023-10-06T03:48:48.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.091+0000 7f7573fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75741597f0 0x7f757416d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:48.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.091+0000 7f7573fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75741597f0 0x7f757416d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57022/0 (socket says 172.21.15.55:57022) 2023-10-06T03:48:48.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.091+0000 7f7573fff700 1 -- 172.21.15.55:0/3179142475 learned_addr learned my addr 172.21.15.55:0/3179142475 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:48.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.091+0000 7f7578b9b700 1 --2- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7574159420 0x7f7574156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:48.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.092+0000 7f757939c700 1 --2- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f757415d230 0x7f757416aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:48.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.092+0000 7f7573fff700 1 -- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f757415d230 msgr2=0x7f757416aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:48.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.092+0000 7f7573fff700 1 --2- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f757415d230 0x7f757416aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.092+0000 7f7573fff700 1 -- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7574159420 msgr2=0x7f7574156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:48.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.092+0000 7f7573fff700 1 --2- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7574159420 0x7f7574156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.092+0000 7f7573fff700 1 -- 172.21.15.55:0/3179142475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75680094f0 con 0x7f75741597f0 2023-10-06T03:48:48.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.092+0000 7f7573fff700 1 --2- 172.21.15.55:0/3179142475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75741597f0 0x7f757416d730 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f756000ec80 tx=0x7f75600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:48.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.093+0000 7f7570ff9700 1 -- 172.21.15.55:0/3179142475 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756000c850 con 0x7f75741597f0 2023-10-06T03:48:48.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.093+0000 7f7570ff9700 1 -- 172.21.15.55:0/3179142475 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756000c9b0 con 0x7f75741597f0 2023-10-06T03:48:48.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.093+0000 7f7570ff9700 1 -- 172.21.15.55:0/3179142475 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756000cd50 con 0x7f75741597f0 2023-10-06T03:48:48.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.093+0000 7f7579b9d700 1 -- 172.21.15.55:0/3179142475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757416e030 con 0x7f75741597f0 2023-10-06T03:48:48.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.093+0000 7f7579b9d700 1 -- 172.21.15.55:0/3179142475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757416e5f0 con 0x7f75741597f0 2023-10-06T03:48:48.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.094+0000 7f7579b9d700 1 -- 172.21.15.55:0/3179142475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757416a6e0 con 0x7f75741597f0 2023-10-06T03:48:48.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.095+0000 7f7570ff9700 1 -- 172.21.15.55:0/3179142475 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 30) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f7560013680 con 0x7f75741597f0 2023-10-06T03:48:48.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.096+0000 7f7570ff9700 1 --2- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f75580670e0 0x7f7558069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:48.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.096+0000 7f7570ff9700 1 -- 172.21.15.55:0/3179142475 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f7560084ed0 con 0x7f75741597f0 2023-10-06T03:48:48.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.096+0000 7f7578b9b700 1 -- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f75580670e0 msgr2=0x7f7558069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:48.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.096+0000 7f7578b9b700 1 --2- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f75580670e0 0x7f7558069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:48.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.100+0000 7f7570ff9700 1 -- 172.21.15.55:0/3179142475 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7560051a30 con 0x7f75741597f0 2023-10-06T03:48:48.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.297+0000 7f7578b9b700 1 -- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f75580670e0 msgr2=0x7f7558069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:48.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.297+0000 7f7578b9b700 1 --2- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f75580670e0 0x7f7558069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:48.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.351+0000 7f7579b9d700 1 -- 172.21.15.55:0/3179142475 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f757404c3c0 con 0x7f75741597f0 2023-10-06T03:48:48.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.352+0000 7f7570ff9700 1 -- 172.21.15.55:0/3179142475 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f7560055050 con 0x7f75741597f0 2023-10-06T03:48:48.354 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:48.354 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4046797285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":4046797285},{"type":"v1","addr":"172.21.15.55:6835","nonce":4046797285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5645,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/675796170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":675796170},{"type":"v1","addr":"172.21.15.163:6836","nonce":675796170}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5648,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2543497851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2543497851},{"type":"v1","addr":"172.21.15.163:6834","nonce":2543497851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:48:22.429407+0000","modified":"2023-10-06T03:48:25.973506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6333":{"gid":6333,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3434145175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3434145175},{"type":"v1","addr":"172.21.15.55:6834","nonce":3434145175}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T03:48:48.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.354+0000 7f7579b9d700 1 -- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f75580670e0 msgr2=0x7f7558069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:48.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.354+0000 7f7579b9d700 1 --2- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f75580670e0 0x7f7558069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.354+0000 7f7579b9d700 1 -- 172.21.15.55:0/3179142475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75741597f0 msgr2=0x7f757416d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:48.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.354+0000 7f7579b9d700 1 --2- 172.21.15.55:0/3179142475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75741597f0 0x7f757416d730 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f756000ec80 tx=0x7f75600091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.354+0000 7f7579b9d700 1 -- 172.21.15.55:0/3179142475 shutdown_connections 2023-10-06T03:48:48.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.354+0000 7f7579b9d700 1 --2- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f75580670e0 0x7f7558069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.354+0000 7f7579b9d700 1 --2- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f757415d230 0x7f757416aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.354+0000 7f7579b9d700 1 --2- 172.21.15.55:0/3179142475 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7574159420 0x7f7574156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.354+0000 7f7579b9d700 1 --2- 172.21.15.55:0/3179142475 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75741597f0 0x7f757416d730 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.354+0000 7f7579b9d700 1 -- 172.21.15.55:0/3179142475 >> 172.21.15.55:0/3179142475 conn(0x7f75740d0660 msgr2=0x7f7574069210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:48.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.355+0000 7f7579b9d700 1 -- 172.21.15.55:0/3179142475 shutdown_connections 2023-10-06T03:48:48.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.355+0000 7f7579b9d700 1 -- 172.21.15.55:0/3179142475 wait complete. 2023-10-06T03:48:48.358 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 35 2023-10-06T03:48:48.374 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:48:48.481 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:48.479+0000 7fe1f5450180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:48:48.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.631+0000 7fc23d79d700 1 Processor -- start 2023-10-06T03:48:48.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.631+0000 7fc23d79d700 1 -- start start 2023-10-06T03:48:48.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.632+0000 7fc23d79d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc2381597f0 0x7fc2380b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:48.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.632+0000 7fc23d79d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc23815d230 0x7fc23815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:48.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.632+0000 7fc23d79d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc23815db70 con 0x7fc238159420 2023-10-06T03:48:48.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.632+0000 7fc23d79d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2380b11d0 con 0x7fc23815d230 2023-10-06T03:48:48.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.633+0000 7fc23d79d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2380b14b0 con 0x7fc2381597f0 2023-10-06T03:48:48.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.633+0000 7fc23cf9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc23815d230 0x7fc23815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:48.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.633+0000 7fc2377fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc2381597f0 0x7fc2380b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:48.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.633+0000 7fc23cf9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc23815d230 0x7fc23815d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57036/0 (socket says 172.21.15.55:57036) 2023-10-06T03:48:48.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.633+0000 7fc23cf9c700 1 -- 172.21.15.55:0/1475844253 learned_addr learned my addr 172.21.15.55:0/1475844253 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:48.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.633+0000 7fc23cf9c700 1 -- 172.21.15.55:0/1475844253 >> v2:172.21.15.163:3300/0 conn(0x7fc2381597f0 msgr2=0x7fc2380b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:48.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.633+0000 7fc23cf9c700 1 --2- 172.21.15.55:0/1475844253 >> v2:172.21.15.163:3300/0 conn(0x7fc2381597f0 0x7fc2380b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.633+0000 7fc23cf9c700 1 -- 172.21.15.55:0/1475844253 >> v1:172.21.15.163:6789/0 conn(0x7fc238159420 legacy=0x7fc2380b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:48.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.633+0000 7fc23cf9c700 1 -- 172.21.15.55:0/1475844253 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2380b1790 con 0x7fc23815d230 2023-10-06T03:48:48.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.633+0000 7fc2377fe700 1 --2- 172.21.15.55:0/1475844253 >> v2:172.21.15.163:3300/0 conn(0x7fc2381597f0 0x7fc2380b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:48.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.633+0000 7fc23cf9c700 1 --2- 172.21.15.55:0/1475844253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc23815d230 0x7fc23815d630 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc238089750 tx=0x7fc230009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cd41f5be17b84b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:48.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.634+0000 7fc236ffd700 1 -- 172.21.15.55:0/1475844253 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23000e070 con 0x7fc23815d230 2023-10-06T03:48:48.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.634+0000 7fc236ffd700 1 -- 172.21.15.55:0/1475844253 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23000ba20 con 0x7fc23815d230 2023-10-06T03:48:48.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.634+0000 7fc236ffd700 1 -- 172.21.15.55:0/1475844253 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2300092e0 con 0x7fc23815d230 2023-10-06T03:48:48.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.634+0000 7fc23d79d700 1 -- 172.21.15.55:0/1475844253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc23815d230 msgr2=0x7fc23815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:48.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.634+0000 7fc23d79d700 1 --2- 172.21.15.55:0/1475844253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc23815d230 0x7fc23815d630 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fc238089750 tx=0x7fc230009840 comp rx=0 tx=0).stop 2023-10-06T03:48:48.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.634+0000 7fc23d79d700 1 -- 172.21.15.55:0/1475844253 shutdown_connections 2023-10-06T03:48:48.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.634+0000 7fc23d79d700 1 --2- 172.21.15.55:0/1475844253 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc23815d230 0x7fc23815d630 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.634+0000 7fc23d79d700 1 --2- 172.21.15.55:0/1475844253 >> v2:172.21.15.163:3300/0 conn(0x7fc2381597f0 0x7fc2380b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.634+0000 7fc23d79d700 1 -- 172.21.15.55:0/1475844253 >> 172.21.15.55:0/1475844253 conn(0x7fc2380d0660 msgr2=0x7fc2380d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:48.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.635+0000 7fc23d79d700 1 -- 172.21.15.55:0/1475844253 shutdown_connections 2023-10-06T03:48:48.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.635+0000 7fc23d79d700 1 -- 172.21.15.55:0/1475844253 wait complete. 2023-10-06T03:48:48.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.636+0000 7fc23d79d700 1 Processor -- start 2023-10-06T03:48:48.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.636+0000 7fc23d79d700 1 -- start start 2023-10-06T03:48:48.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.637+0000 7fc23d79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc238159420 0x7fc23815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:48.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.637+0000 7fc23d79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2381597f0 0x7fc23816a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:48.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.637+0000 7fc23d79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc23815d230 0x7fc2380b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:48.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.637+0000 7fc23d79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23816a5a0 con 0x7fc2381597f0 2023-10-06T03:48:48.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.637+0000 7fc23d79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23815bf80 con 0x7fc238159420 2023-10-06T03:48:48.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.637+0000 7fc23d79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23815b370 con 0x7fc23815d230 2023-10-06T03:48:48.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.638+0000 7fc2377fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2381597f0 0x7fc23816a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:48.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.638+0000 7fc237fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc238159420 0x7fc23815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:48.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.638+0000 7fc237fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc238159420 0x7fc23815ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:41216/0 (socket says 172.21.15.55:41216) 2023-10-06T03:48:48.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.638+0000 7fc237fff700 1 -- 172.21.15.55:0/2165771362 learned_addr learned my addr 172.21.15.55:0/2165771362 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:48.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.638+0000 7fc23cf9c700 1 --2- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc23815d230 0x7fc2380b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:48.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.638+0000 7fc2377fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2381597f0 0x7fc23816a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57052/0 (socket says 172.21.15.55:57052) 2023-10-06T03:48:48.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.638+0000 7fc237fff700 1 -- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc23815d230 msgr2=0x7fc2380b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:48.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.638+0000 7fc237fff700 1 --2- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc23815d230 0x7fc2380b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.638+0000 7fc237fff700 1 -- 172.21.15.55:0/2165771362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2381597f0 msgr2=0x7fc23816a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:48.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.638+0000 7fc237fff700 1 --2- 172.21.15.55:0/2165771362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2381597f0 0x7fc23816a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.638+0000 7fc237fff700 1 -- 172.21.15.55:0/2165771362 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c009420 con 0x7fc238159420 2023-10-06T03:48:48.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.638+0000 7fc237fff700 1 --2- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc238159420 0x7fc23815ac70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc22800c5c0 tx=0x7fc22800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:48.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.639+0000 7fc21ffff700 1 -- 172.21.15.55:0/2165771362 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc228011590 con 0x7fc238159420 2023-10-06T03:48:48.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.639+0000 7fc21ffff700 1 -- 172.21.15.55:0/2165771362 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc228011710 con 0x7fc238159420 2023-10-06T03:48:48.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.639+0000 7fc21ffff700 1 -- 172.21.15.55:0/2165771362 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc228011ad0 con 0x7fc238159420 2023-10-06T03:48:48.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.639+0000 7fc23d79d700 1 -- 172.21.15.55:0/2165771362 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc230009ae0 con 0x7fc238159420 2023-10-06T03:48:48.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.639+0000 7fc23d79d700 1 -- 172.21.15.55:0/2165771362 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23815b930 con 0x7fc238159420 2023-10-06T03:48:48.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.640+0000 7fc23d79d700 1 -- 172.21.15.55:0/2165771362 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23815c0f0 con 0x7fc238159420 2023-10-06T03:48:48.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.641+0000 7fc21ffff700 1 -- 172.21.15.55:0/2165771362 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 30) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fc22801b3b0 con 0x7fc238159420 2023-10-06T03:48:48.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.642+0000 7fc21ffff700 1 --2- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc2240670e0 0x7fc224069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:48.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.642+0000 7fc21ffff700 1 -- 172.21.15.55:0/2165771362 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6708+0+0 (secure 0 0 0) 0x7fc228084160 con 0x7fc238159420 2023-10-06T03:48:48.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.642+0000 7fc2377fe700 1 -- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc2240670e0 msgr2=0x7fc224069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:48.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.642+0000 7fc2377fe700 1 --2- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc2240670e0 0x7fc224069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:48.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.646+0000 7fc21ffff700 1 -- 172.21.15.55:0/2165771362 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc228050cc0 con 0x7fc238159420 2023-10-06T03:48:48.654 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:48.652+0000 7fe1f5450180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:48:48.670 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:48.668+0000 7f4fc328e180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:48:48.750 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:48.748+0000 7fe1f5450180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:48:48.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.843+0000 7fc2377fe700 1 -- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc2240670e0 msgr2=0x7fc224069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:48.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.843+0000 7fc2377fe700 1 --2- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc2240670e0 0x7fc224069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:48.850 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:48.848+0000 7f4fc328e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:48:48.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.851+0000 7fc23d79d700 1 -- 172.21.15.55:0/2165771362 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc23804c3c0 con 0x7fc238159420 2023-10-06T03:48:48.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.852+0000 7fc21ffff700 1 -- 172.21.15.55:0/2165771362 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+17883 (secure 0 0 0) 0x7fc228017090 con 0x7fc238159420 2023-10-06T03:48:48.854 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:48.854 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":81,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:48:42.787202+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:48:20.608889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T03:48:21.410367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/85191":"2023-10-07T03:48:42.787170+0000","172.21.15.163:6835/85191":"2023-10-07T03:48:42.787170+0000","172.21.15.163:0/596976820":"2023-10-07T03:48:42.787170+0000","172.21.15.163:0/4293045398":"2023-10-07T03:48:42.787170+0000","172.21.15.163:0/1262983566":"2023-10-07T03:48:42.787170+0000","172.21.15.163:0/3262439175":"2023-10-07T03:48:42.787170+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:48:48.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.855+0000 7fc23d79d700 1 -- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc2240670e0 msgr2=0x7fc224069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:48.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.855+0000 7fc23d79d700 1 --2- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc2240670e0 0x7fc224069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.855+0000 7fc23d79d700 1 -- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc238159420 msgr2=0x7fc23815ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.855+0000 7fc23d79d700 1 --2- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc238159420 0x7fc23815ac70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc22800c5c0 tx=0x7fc22800c8d0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.855+0000 7fc23d79d700 1 -- 172.21.15.55:0/2165771362 shutdown_connections 2023-10-06T03:48:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.855+0000 7fc23d79d700 1 --2- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc2240670e0 0x7fc224069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.855+0000 7fc23d79d700 1 --2- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc23815d230 0x7fc2380b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.855+0000 7fc23d79d700 1 --2- 172.21.15.55:0/2165771362 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc238159420 0x7fc23815ac70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.855+0000 7fc23d79d700 1 --2- 172.21.15.55:0/2165771362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2381597f0 0x7fc23816a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:48.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.855+0000 7fc23d79d700 1 -- 172.21.15.55:0/2165771362 >> 172.21.15.55:0/2165771362 conn(0x7fc2380d0660 msgr2=0x7fc2381572c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:48.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.855+0000 7fc23d79d700 1 -- 172.21.15.55:0/2165771362 shutdown_connections 2023-10-06T03:48:48.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:48.855+0000 7fc23d79d700 1 -- 172.21.15.55:0/2165771362 wait complete. 2023-10-06T03:48:48.877 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:48:48.877 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:48:48.954 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:48.952+0000 7f4fc328e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:48:49.046 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:49.044+0000 7fe1f5450180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:48:49.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.065+0000 7f02635ff700 1 Processor -- start 2023-10-06T03:48:49.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.065+0000 7f02635ff700 1 -- start start 2023-10-06T03:48:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.066+0000 7f02635ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f025c0b1c20 0x7f025c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.066+0000 7f02635ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f025c157820 0x7f025c0abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.066+0000 7f02635ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c157c80 con 0x7f025c0b1850 2023-10-06T03:48:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.066+0000 7f02635ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c06fc30 con 0x7f025c157820 2023-10-06T03:48:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.066+0000 7f02635ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0cc260 con 0x7f025c0b1c20 2023-10-06T03:48:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.066+0000 7f026139b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f025c0b1850 0x7f025c0ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52542/0 (socket says 172.21.15.55:52542) 2023-10-06T03:48:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.066+0000 7f026139b700 1 -- 172.21.15.55:0/845578577 learned_addr learned my addr 172.21.15.55:0/845578577 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.066+0000 7f0260b9a700 1 --2- 172.21.15.55:0/845578577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f025c0b1c20 0x7f025c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.066+0000 7f0261b9c700 1 --2- 172.21.15.55:0/845578577 >> v2:172.21.15.163:3300/0 conn(0x7f025c157820 0x7f025c0abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.067+0000 7f0253fff700 1 -- 172.21.15.55:0/845578577 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871874821 0 0) 0x7f025c157c80 con 0x7f025c0b1850 2023-10-06T03:48:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.067+0000 7f0253fff700 1 -- 172.21.15.55:0/845578577 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0248003570 con 0x7f025c0b1850 2023-10-06T03:48:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.067+0000 7f0260b9a700 1 -- 172.21.15.55:0/845578577 >> v2:172.21.15.163:3300/0 conn(0x7f025c157820 msgr2=0x7f025c0abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:49.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.067+0000 7f0260b9a700 1 --2- 172.21.15.55:0/845578577 >> v2:172.21.15.163:3300/0 conn(0x7f025c157820 0x7f025c0abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.067+0000 7f0260b9a700 1 -- 172.21.15.55:0/845578577 >> v1:172.21.15.55:6789/0 conn(0x7f025c0b1850 legacy=0x7f025c0ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.067+0000 7f0260b9a700 1 -- 172.21.15.55:0/845578577 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c0cb300 con 0x7f025c0b1c20 2023-10-06T03:48:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.067+0000 7f0261b9c700 1 --2- 172.21.15.55:0/845578577 >> v2:172.21.15.163:3300/0 conn(0x7f025c157820 0x7f025c0abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.067+0000 7f0260b9a700 1 --2- 172.21.15.55:0/845578577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f025c0b1c20 0x7f025c1572e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f024c0098e0 tx=0x7f024c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9b69954df424161 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.068+0000 7f0253fff700 1 -- 172.21.15.55:0/845578577 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c011630 con 0x7f025c0b1c20 2023-10-06T03:48:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.068+0000 7f0253fff700 1 -- 172.21.15.55:0/845578577 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c011790 con 0x7f025c0b1c20 2023-10-06T03:48:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.068+0000 7f0253fff700 1 -- 172.21.15.55:0/845578577 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c011b30 con 0x7f025c0b1c20 2023-10-06T03:48:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.068+0000 7f02635ff700 1 -- 172.21.15.55:0/845578577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f025c0b1c20 msgr2=0x7f025c1572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.068+0000 7f02635ff700 1 --2- 172.21.15.55:0/845578577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f025c0b1c20 0x7f025c1572e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f024c0098e0 tx=0x7f024c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.068+0000 7f02635ff700 1 -- 172.21.15.55:0/845578577 shutdown_connections 2023-10-06T03:48:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.068+0000 7f02635ff700 1 --2- 172.21.15.55:0/845578577 >> v2:172.21.15.163:3300/0 conn(0x7f025c157820 0x7f025c0abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.068+0000 7f02635ff700 1 --2- 172.21.15.55:0/845578577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f025c0b1c20 0x7f025c1572e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.068+0000 7f02635ff700 1 -- 172.21.15.55:0/845578577 >> 172.21.15.55:0/845578577 conn(0x7f025c06a320 msgr2=0x7f025c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.069+0000 7f02635ff700 1 -- 172.21.15.55:0/845578577 shutdown_connections 2023-10-06T03:48:49.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.069+0000 7f02635ff700 1 -- 172.21.15.55:0/845578577 wait complete. 2023-10-06T03:48:49.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.070+0000 7f02635ff700 1 Processor -- start 2023-10-06T03:48:49.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.070+0000 7f02635ff700 1 -- start start 2023-10-06T03:48:49.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.071+0000 7f02635ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f025c0b1850 0x7f025c0ab260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:49.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.071+0000 7f02635ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f025c0b1c20 0x7f025c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:49.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.071+0000 7f02635ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f025c157820 0x7f025c16de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:49.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.071+0000 7f02635ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0b0b00 con 0x7f025c0b1c20 2023-10-06T03:48:49.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.071+0000 7f02635ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0ab7a0 con 0x7f025c0b1850 2023-10-06T03:48:49.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.071+0000 7f02635ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c0aff30 con 0x7f025c157820 2023-10-06T03:48:49.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.072+0000 7f0260b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f025c0b1c20 0x7f025c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:49.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.072+0000 7f026139b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f025c0b1850 0x7f025c0ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:49.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.072+0000 7f0260b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f025c0b1c20 0x7f025c15c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57064/0 (socket says 172.21.15.55:57064) 2023-10-06T03:48:49.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.072+0000 7f0260b9a700 1 -- 172.21.15.55:0/4188529376 learned_addr learned my addr 172.21.15.55:0/4188529376 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.072+0000 7f0261b9c700 1 --2- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f025c157820 0x7f025c16de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.072+0000 7f0260b9a700 1 -- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f025c157820 msgr2=0x7f025c16de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.072+0000 7f0260b9a700 1 --2- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f025c157820 0x7f025c16de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.072+0000 7f0260b9a700 1 -- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f025c0b1850 msgr2=0x7f025c0ab260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.072+0000 7f0260b9a700 1 --2- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f025c0b1850 0x7f025c0ab260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.072+0000 7f0260b9a700 1 -- 172.21.15.55:0/4188529376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0254009a20 con 0x7f025c0b1c20 2023-10-06T03:48:49.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.072+0000 7f026139b700 1 --2- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f025c0b1850 0x7f025c0ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:49.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.072+0000 7f0260b9a700 1 --2- 172.21.15.55:0/4188529376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f025c0b1c20 0x7f025c15c910 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f025c0c9ae0 tx=0x7f024c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:49.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.073+0000 7f02517fa700 1 -- 172.21.15.55:0/4188529376 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c011b10 con 0x7f025c0b1c20 2023-10-06T03:48:49.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.073+0000 7f02517fa700 1 -- 172.21.15.55:0/4188529376 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c000cc0 con 0x7f025c0b1c20 2023-10-06T03:48:49.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.073+0000 7f02517fa700 1 -- 172.21.15.55:0/4188529376 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c025620 con 0x7f025c0b1c20 2023-10-06T03:48:49.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.073+0000 7f02635ff700 1 -- 172.21.15.55:0/4188529376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024c009420 con 0x7f025c0b1c20 2023-10-06T03:48:49.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.073+0000 7f02635ff700 1 -- 172.21.15.55:0/4188529376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c0b04b0 con 0x7f025c0b1c20 2023-10-06T03:48:49.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.074+0000 7f02635ff700 1 -- 172.21.15.55:0/4188529376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c0b0c70 con 0x7f025c0b1c20 2023-10-06T03:48:49.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.075+0000 7f02517fa700 1 -- 172.21.15.55:0/4188529376 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 30) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f024c025a10 con 0x7f025c0b1c20 2023-10-06T03:48:49.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.076+0000 7f02517fa700 1 --2- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02400670e0 0x7f0240069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:49.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.076+0000 7f02517fa700 1 -- 172.21.15.55:0/4188529376 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f024c086410 con 0x7f025c0b1c20 2023-10-06T03:48:49.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.076+0000 7f026139b700 1 -- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02400670e0 msgr2=0x7f0240069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:49.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.076+0000 7f026139b700 1 --2- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02400670e0 0x7f0240069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:49.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.080+0000 7f02517fa700 1 -- 172.21.15.55:0/4188529376 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f024c053030 con 0x7f025c0b1c20 2023-10-06T03:48:49.144 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:49.142+0000 7fe1f5450180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:48:49.260 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:49.258+0000 7f4fc328e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:48:49.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.277+0000 7f026139b700 1 -- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02400670e0 msgr2=0x7f0240069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:49.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.277+0000 7f026139b700 1 --2- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02400670e0 0x7f0240069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:49.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.330+0000 7f02635ff700 1 -- 172.21.15.55:0/4188529376 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f025c04c3c0 con 0x7f025c0b1c20 2023-10-06T03:48:49.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.330+0000 7f02517fa700 1 -- 172.21.15.55:0/4188529376 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4601 (secure 0 0 0) 0x7f024c056650 con 0x7f025c0b1c20 2023-10-06T03:48:49.332 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:49.332 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4046797285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":4046797285},{"type":"v1","addr":"172.21.15.55:6835","nonce":4046797285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5645,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/675796170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":675796170},{"type":"v1","addr":"172.21.15.163:6836","nonce":675796170}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5648,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2543497851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2543497851},{"type":"v1","addr":"172.21.15.163:6834","nonce":2543497851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:48:22.429407+0000","modified":"2023-10-06T03:48:25.973506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6333":{"gid":6333,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3434145175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3434145175},{"type":"v1","addr":"172.21.15.55:6834","nonce":3434145175}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T03:48:49.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.332+0000 7f02635ff700 1 -- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02400670e0 msgr2=0x7f0240069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:49.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.332+0000 7f02635ff700 1 --2- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02400670e0 0x7f0240069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.332+0000 7f02635ff700 1 -- 172.21.15.55:0/4188529376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f025c0b1c20 msgr2=0x7f025c15c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:49.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.332+0000 7f02635ff700 1 --2- 172.21.15.55:0/4188529376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f025c0b1c20 0x7f025c15c910 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f025c0c9ae0 tx=0x7f024c017000 comp rx=0 tx=0).stop 2023-10-06T03:48:49.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.333+0000 7f02635ff700 1 -- 172.21.15.55:0/4188529376 shutdown_connections 2023-10-06T03:48:49.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.333+0000 7f02635ff700 1 --2- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02400670e0 0x7f0240069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.333+0000 7f02635ff700 1 --2- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f025c157820 0x7f025c16de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.333+0000 7f02635ff700 1 --2- 172.21.15.55:0/4188529376 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f025c0b1850 0x7f025c0ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.333+0000 7f02635ff700 1 --2- 172.21.15.55:0/4188529376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f025c0b1c20 0x7f025c15c910 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.333+0000 7f02635ff700 1 -- 172.21.15.55:0/4188529376 >> 172.21.15.55:0/4188529376 conn(0x7f025c06a320 msgr2=0x7f025c06de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:49.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.333+0000 7f02635ff700 1 -- 172.21.15.55:0/4188529376 shutdown_connections 2023-10-06T03:48:49.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.333+0000 7f02635ff700 1 -- 172.21.15.55:0/4188529376 wait complete. 2023-10-06T03:48:49.337 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 35 2023-10-06T03:48:49.352 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:48:49.361 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:49.359+0000 7f4fc328e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:48:49.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.546+0000 7ff50e5ff700 1 Processor -- start 2023-10-06T03:48:49.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.546+0000 7ff50e5ff700 1 -- start start 2023-10-06T03:48:49.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.547+0000 7ff50e5ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080cbf30 0x7ff5080ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:49.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.547+0000 7ff50e5ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff5080b1420 0x7ff5080c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:49.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.548+0000 7ff50e5ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5080cc300 con 0x7ff5080ccf90 2023-10-06T03:48:49.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.548+0000 7ff50e5ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff508156bf0 con 0x7ff5080b1420 2023-10-06T03:48:49.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.548+0000 7ff50e5ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080ae950 con 0x7ff5080cbf30 2023-10-06T03:48:49.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.548+0000 7ff50cb9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff5080b1420 0x7ff5080c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:49.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.548+0000 7ff50cb9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff5080b1420 0x7ff5080c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57074/0 (socket says 172.21.15.55:57074) 2023-10-06T03:48:49.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.548+0000 7ff50cb9c700 1 -- 172.21.15.55:0/1763944728 learned_addr learned my addr 172.21.15.55:0/1763944728 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:49.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.548+0000 7ff507fff700 1 --2- 172.21.15.55:0/1763944728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080cbf30 0x7ff5080ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:49.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.548+0000 7ff506ffd700 1 -- 172.21.15.55:0/1763944728 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310598783 0 0) 0x7ff5080cc300 con 0x7ff5080ccf90 2023-10-06T03:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.548+0000 7ff50cb9c700 1 -- 172.21.15.55:0/1763944728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080cbf30 msgr2=0x7ff5080ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.548+0000 7ff50cb9c700 1 --2- 172.21.15.55:0/1763944728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080cbf30 0x7ff5080ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.548+0000 7ff50cb9c700 1 -- 172.21.15.55:0/1763944728 >> v1:172.21.15.163:6789/0 conn(0x7ff5080ccf90 legacy=0x7ff5080b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.548+0000 7ff50cb9c700 1 -- 172.21.15.55:0/1763944728 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5080ad370 con 0x7ff5080b1420 2023-10-06T03:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.549+0000 7ff50cb9c700 1 --2- 172.21.15.55:0/1763944728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5080b1420 0x7ff5080c9740 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff50000d1d0 tx=0x7ff50000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ab141dd0c13a9a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.549+0000 7ff506ffd700 1 -- 172.21.15.55:0/1763944728 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500010070 con 0x7ff5080b1420 2023-10-06T03:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.549+0000 7ff506ffd700 1 -- 172.21.15.55:0/1763944728 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50000de00 con 0x7ff5080b1420 2023-10-06T03:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.549+0000 7ff506ffd700 1 -- 172.21.15.55:0/1763944728 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50000b320 con 0x7ff5080b1420 2023-10-06T03:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.549+0000 7ff50e5ff700 1 -- 172.21.15.55:0/1763944728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5080b1420 msgr2=0x7ff5080c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.549+0000 7ff50e5ff700 1 --2- 172.21.15.55:0/1763944728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5080b1420 0x7ff5080c9740 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff50000d1d0 tx=0x7ff50000ba50 comp rx=0 tx=0).stop 2023-10-06T03:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.550+0000 7ff50e5ff700 1 -- 172.21.15.55:0/1763944728 shutdown_connections 2023-10-06T03:48:49.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.550+0000 7ff50e5ff700 1 --2- 172.21.15.55:0/1763944728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5080b1420 0x7ff5080c9740 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.550+0000 7ff50e5ff700 1 --2- 172.21.15.55:0/1763944728 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080cbf30 0x7ff5080ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.550+0000 7ff50e5ff700 1 -- 172.21.15.55:0/1763944728 >> 172.21.15.55:0/1763944728 conn(0x7ff50806a320 msgr2=0x7ff5080cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:49.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.550+0000 7ff50e5ff700 1 -- 172.21.15.55:0/1763944728 shutdown_connections 2023-10-06T03:48:49.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.550+0000 7ff50e5ff700 1 -- 172.21.15.55:0/1763944728 wait complete. 2023-10-06T03:48:49.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.552+0000 7ff50e5ff700 1 Processor -- start 2023-10-06T03:48:49.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.552+0000 7ff50e5ff700 1 -- start start 2023-10-06T03:48:49.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.553+0000 7ff50e5ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5080b1420 0x7ff50814baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:49.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.553+0000 7ff50e5ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5080cbf30 0x7ff508151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:49.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff50e5ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080ccf90 0x7ff50806ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:49.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff50e5ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5081522e0 con 0x7ff5080cbf30 2023-10-06T03:48:49.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff50e5ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50814c080 con 0x7ff5080b1420 2023-10-06T03:48:49.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff50e5ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50814c360 con 0x7ff5080ccf90 2023-10-06T03:48:49.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff5077fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5080cbf30 0x7ff508151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:49.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff5077fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5080cbf30 0x7ff508151da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57080/0 (socket says 172.21.15.55:57080) 2023-10-06T03:48:49.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff5077fe700 1 -- 172.21.15.55:0/4200898373 learned_addr learned my addr 172.21.15.55:0/4200898373 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:49.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff50cb9c700 1 --2- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080ccf90 0x7ff50806ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:49.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff507fff700 1 --2- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5080b1420 0x7ff50814baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:49.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff5077fe700 1 -- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080ccf90 msgr2=0x7ff50806ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:49.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff5077fe700 1 --2- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080ccf90 0x7ff50806ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff5077fe700 1 -- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5080b1420 msgr2=0x7ff50814baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:49.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff5077fe700 1 --2- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5080b1420 0x7ff50814baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff5077fe700 1 -- 172.21.15.55:0/4200898373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50000bcf0 con 0x7ff5080cbf30 2023-10-06T03:48:49.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.554+0000 7ff5077fe700 1 --2- 172.21.15.55:0/4200898373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5080cbf30 0x7ff508151da0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0018d0 tx=0x7ff4fc001be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:49.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.555+0000 7ff4effff700 1 -- 172.21.15.55:0/4200898373 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc010260 con 0x7ff5080cbf30 2023-10-06T03:48:49.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.555+0000 7ff4effff700 1 -- 172.21.15.55:0/4200898373 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc0103e0 con 0x7ff5080cbf30 2023-10-06T03:48:49.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.555+0000 7ff50e5ff700 1 -- 172.21.15.55:0/4200898373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50814c640 con 0x7ff5080cbf30 2023-10-06T03:48:49.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.555+0000 7ff4effff700 1 -- 172.21.15.55:0/4200898373 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc00a820 con 0x7ff5080cbf30 2023-10-06T03:48:49.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.555+0000 7ff50e5ff700 1 -- 172.21.15.55:0/4200898373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50806f1a0 con 0x7ff5080cbf30 2023-10-06T03:48:49.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.556+0000 7ff50e5ff700 1 -- 172.21.15.55:0/4200898373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50804c3c0 con 0x7ff5080cbf30 2023-10-06T03:48:49.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.557+0000 7ff4effff700 1 -- 172.21.15.55:0/4200898373 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 30) v1 ==== 84899+0+0 (secure 0 0 0) 0x7ff4fc00ac10 con 0x7ff5080cbf30 2023-10-06T03:48:49.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.558+0000 7ff4effff700 1 --2- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff4f40670e0 0x7ff4f4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:49.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.558+0000 7ff4effff700 1 -- 172.21.15.55:0/4200898373 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6708+0+0 (secure 0 0 0) 0x7ff4fc088a20 con 0x7ff5080cbf30 2023-10-06T03:48:49.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.558+0000 7ff507fff700 1 -- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff4f40670e0 msgr2=0x7ff4f4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:49.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.558+0000 7ff507fff700 1 --2- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff4f40670e0 0x7ff4f4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:49.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.562+0000 7ff4effff700 1 -- 172.21.15.55:0/4200898373 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4fc055470 con 0x7ff5080cbf30 2023-10-06T03:48:49.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.759+0000 7ff507fff700 1 -- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff4f40670e0 msgr2=0x7ff4f4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:49.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.759+0000 7ff507fff700 1 --2- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff4f40670e0 0x7ff4f4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:49.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.811+0000 7ff50e5ff700 1 -- 172.21.15.55:0/4200898373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5080ad650 con 0x7ff5080cbf30 2023-10-06T03:48:49.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.812+0000 7ff4effff700 1 -- 172.21.15.55:0/4200898373 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4601 (secure 0 0 0) 0x7ff4fc058a90 con 0x7ff5080cbf30 2023-10-06T03:48:49.813 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:49.814 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4046797285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":4046797285},{"type":"v1","addr":"172.21.15.55:6835","nonce":4046797285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5645,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/675796170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":675796170},{"type":"v1","addr":"172.21.15.163:6836","nonce":675796170}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5648,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2543497851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2543497851},{"type":"v1","addr":"172.21.15.163:6834","nonce":2543497851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:48:22.429407+0000","modified":"2023-10-06T03:48:25.973506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6333},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6333":{"gid":6333,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3434145175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3434145175},{"type":"v1","addr":"172.21.15.55:6834","nonce":3434145175}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-06T03:48:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.814+0000 7ff50e5ff700 1 -- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff4f40670e0 msgr2=0x7ff4f4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.814+0000 7ff50e5ff700 1 --2- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff4f40670e0 0x7ff4f4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.814+0000 7ff50e5ff700 1 -- 172.21.15.55:0/4200898373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5080cbf30 msgr2=0x7ff508151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.814+0000 7ff50e5ff700 1 --2- 172.21.15.55:0/4200898373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5080cbf30 0x7ff508151da0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0018d0 tx=0x7ff4fc001be0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.814+0000 7ff50e5ff700 1 -- 172.21.15.55:0/4200898373 shutdown_connections 2023-10-06T03:48:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.814+0000 7ff50e5ff700 1 --2- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff4f40670e0 0x7ff4f4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.814+0000 7ff50e5ff700 1 --2- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5080ccf90 0x7ff50806ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.814+0000 7ff50e5ff700 1 --2- 172.21.15.55:0/4200898373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5080b1420 0x7ff50814baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.814+0000 7ff50e5ff700 1 --2- 172.21.15.55:0/4200898373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5080cbf30 0x7ff508151da0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.814+0000 7ff50e5ff700 1 -- 172.21.15.55:0/4200898373 >> 172.21.15.55:0/4200898373 conn(0x7ff50806a320 msgr2=0x7ff5080cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.814+0000 7ff50e5ff700 1 -- 172.21.15.55:0/4200898373 shutdown_connections 2023-10-06T03:48:49.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:49.814+0000 7ff50e5ff700 1 -- 172.21.15.55:0/4200898373 wait complete. 2023-10-06T03:48:49.817 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 35 2023-10-06T03:48:49.834 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:48:49.845 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:49.843+0000 7fe1f5450180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:48:50.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.063+0000 7f156d19d700 1 Processor -- start 2023-10-06T03:48:50.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.064+0000 7f156d19d700 1 -- start start 2023-10-06T03:48:50.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.065+0000 7f156d19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1568142ea0 0x7f15680b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:50.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.065+0000 7f156d19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15680ad870 0x7f156806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:50.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.065+0000 7f156d19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15680b40f0 con 0x7f15680b3b70 2023-10-06T03:48:50.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.065+0000 7f156d19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1568069640 con 0x7f1568142ea0 2023-10-06T03:48:50.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.065+0000 7f156d19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1568069920 con 0x7f15680ad870 2023-10-06T03:48:50.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.065+0000 7f1567fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1568142ea0 0x7f15680b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:50.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.065+0000 7f1567fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1568142ea0 0x7f15680b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57088/0 (socket says 172.21.15.55:57088) 2023-10-06T03:48:50.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.065+0000 7f1567fff700 1 -- 172.21.15.55:0/2830187275 learned_addr learned my addr 172.21.15.55:0/2830187275 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:50.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.065+0000 7f15677fe700 1 --2- 172.21.15.55:0/2830187275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15680ad870 0x7f156806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:50.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.066+0000 7f1567fff700 1 -- 172.21.15.55:0/2830187275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15680ad870 msgr2=0x7f156806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:50.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.066+0000 7f1567fff700 1 --2- 172.21.15.55:0/2830187275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15680ad870 0x7f156806b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.066+0000 7f1567fff700 1 -- 172.21.15.55:0/2830187275 >> v1:172.21.15.163:6789/0 conn(0x7f15680b3b70 legacy=0x7f156806cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:50.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.066+0000 7f1567fff700 1 -- 172.21.15.55:0/2830187275 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1568069c00 con 0x7f1568142ea0 2023-10-06T03:48:50.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.066+0000 7f1567fff700 1 --2- 172.21.15.55:0/2830187275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1568142ea0 0x7f15680b3630 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f15580099b0 tx=0x7f1558009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7610ebb0cc645 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:50.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.066+0000 7f1566ffd700 1 -- 172.21.15.55:0/2830187275 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558009ea0 con 0x7f1568142ea0 2023-10-06T03:48:50.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.066+0000 7f1566ffd700 1 -- 172.21.15.55:0/2830187275 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155800b860 con 0x7f1568142ea0 2023-10-06T03:48:50.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.066+0000 7f1566ffd700 1 -- 172.21.15.55:0/2830187275 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155800bc00 con 0x7f1568142ea0 2023-10-06T03:48:50.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.067+0000 7f156d19d700 1 -- 172.21.15.55:0/2830187275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1568142ea0 msgr2=0x7f15680b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:50.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.067+0000 7f156d19d700 1 --2- 172.21.15.55:0/2830187275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1568142ea0 0x7f15680b3630 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f15580099b0 tx=0x7f1558009250 comp rx=0 tx=0).stop 2023-10-06T03:48:50.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.067+0000 7f156d19d700 1 -- 172.21.15.55:0/2830187275 shutdown_connections 2023-10-06T03:48:50.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.067+0000 7f156d19d700 1 --2- 172.21.15.55:0/2830187275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1568142ea0 0x7f15680b3630 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.067+0000 7f156d19d700 1 --2- 172.21.15.55:0/2830187275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f15680ad870 0x7f156806b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.067+0000 7f156d19d700 1 -- 172.21.15.55:0/2830187275 >> 172.21.15.55:0/2830187275 conn(0x7f15680ce430 msgr2=0x7f15680ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:50.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.067+0000 7f156d19d700 1 -- 172.21.15.55:0/2830187275 shutdown_connections 2023-10-06T03:48:50.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.067+0000 7f156d19d700 1 -- 172.21.15.55:0/2830187275 wait complete. 2023-10-06T03:48:50.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.068+0000 7f156d19d700 1 Processor -- start 2023-10-06T03:48:50.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.069+0000 7f156d19d700 1 -- start start 2023-10-06T03:48:50.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f156d19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15680b3b70 0x7f15681590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:50.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f156d19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1568142ea0 0x7f1568141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:50.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f156d19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f156815c600 0x7f15680aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:50.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f156d19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156815ca70 con 0x7f156815c600 2023-10-06T03:48:50.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f156d19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156815cd50 con 0x7f15680b3b70 2023-10-06T03:48:50.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f156d19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156815d030 con 0x7f1568142ea0 2023-10-06T03:48:50.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f156c99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f156815c600 0x7f15680aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:50.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f156c99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f156815c600 0x7f15680aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57100/0 (socket says 172.21.15.55:57100) 2023-10-06T03:48:50.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f1567fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15680b3b70 0x7f15681590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:50.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f156c99c700 1 -- 172.21.15.55:0/1142122314 learned_addr learned my addr 172.21.15.55:0/1142122314 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:50.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f15677fe700 1 --2- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1568142ea0 0x7f1568141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:50.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f156c99c700 1 -- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1568142ea0 msgr2=0x7f1568141160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:50.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f156c99c700 1 --2- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1568142ea0 0x7f1568141160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f156c99c700 1 -- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15680b3b70 msgr2=0x7f15681590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:50.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f156c99c700 1 --2- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15680b3b70 0x7f15681590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.070+0000 7f156c99c700 1 -- 172.21.15.55:0/1142122314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15580094f0 con 0x7f156815c600 2023-10-06T03:48:50.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.071+0000 7f156c99c700 1 --2- 172.21.15.55:0/1142122314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f156815c600 0x7f15680aeec0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f156000c8e0 tx=0x7f1560001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:50.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.071+0000 7f154ffff700 1 -- 172.21.15.55:0/1142122314 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560012630 con 0x7f156815c600 2023-10-06T03:48:50.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.071+0000 7f154ffff700 1 -- 172.21.15.55:0/1142122314 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1560012790 con 0x7f156815c600 2023-10-06T03:48:50.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.071+0000 7f156d19d700 1 -- 172.21.15.55:0/1142122314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156815d310 con 0x7f156815c600 2023-10-06T03:48:50.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.071+0000 7f154ffff700 1 -- 172.21.15.55:0/1142122314 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560012b50 con 0x7f156815c600 2023-10-06T03:48:50.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.071+0000 7f156d19d700 1 -- 172.21.15.55:0/1142122314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15681595e0 con 0x7f156815c600 2023-10-06T03:48:50.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.072+0000 7f156d19d700 1 -- 172.21.15.55:0/1142122314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156815d830 con 0x7f156815c600 2023-10-06T03:48:50.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.073+0000 7f154ffff700 1 -- 172.21.15.55:0/1142122314 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 30) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f156001a3a0 con 0x7f156815c600 2023-10-06T03:48:50.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.074+0000 7f154ffff700 1 --2- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f15540670e0 0x7f1554069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:50.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.074+0000 7f154ffff700 1 -- 172.21.15.55:0/1142122314 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f1560084ea0 con 0x7f156815c600 2023-10-06T03:48:50.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.074+0000 7f1567fff700 1 -- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f15540670e0 msgr2=0x7f1554069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:50.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.075+0000 7f1567fff700 1 --2- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f15540670e0 0x7f1554069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:50.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.079+0000 7f154ffff700 1 -- 172.21.15.55:0/1142122314 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1560051a00 con 0x7f156815c600 2023-10-06T03:48:50.120 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:50.119+0000 7f4fc328e180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:48:50.134 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:50.132+0000 7fe1f5450180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:48:50.227 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:50.225+0000 7fe1f5450180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:48:50.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.276+0000 7f1567fff700 1 -- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f15540670e0 msgr2=0x7f1554069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:50.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.276+0000 7f1567fff700 1 --2- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f15540670e0 0x7f1554069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:50.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.287+0000 7f156d19d700 1 -- 172.21.15.55:0/1142122314 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f156804c3c0 con 0x7f156815c600 2023-10-06T03:48:50.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.288+0000 7f154ffff700 1 -- 172.21.15.55:0/1142122314 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+17883 (secure 0 0 0) 0x7f1560055020 con 0x7f156815c600 2023-10-06T03:48:50.290 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:50.290 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":81,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:48:42.787202+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:48:20.608889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-10-06T03:48:21.410367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/85191":"2023-10-07T03:48:42.787170+0000","172.21.15.163:6835/85191":"2023-10-07T03:48:42.787170+0000","172.21.15.163:0/596976820":"2023-10-07T03:48:42.787170+0000","172.21.15.163:0/4293045398":"2023-10-07T03:48:42.787170+0000","172.21.15.163:0/1262983566":"2023-10-07T03:48:42.787170+0000","172.21.15.163:0/3262439175":"2023-10-07T03:48:42.787170+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:48:50.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.290+0000 7f156d19d700 1 -- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f15540670e0 msgr2=0x7f1554069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:50.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.290+0000 7f156d19d700 1 --2- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f15540670e0 0x7f1554069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.290+0000 7f156d19d700 1 -- 172.21.15.55:0/1142122314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f156815c600 msgr2=0x7f15680aeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:50.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.290+0000 7f156d19d700 1 --2- 172.21.15.55:0/1142122314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f156815c600 0x7f15680aeec0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f156000c8e0 tx=0x7f1560001710 comp rx=0 tx=0).stop 2023-10-06T03:48:50.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.290+0000 7f156d19d700 1 -- 172.21.15.55:0/1142122314 shutdown_connections 2023-10-06T03:48:50.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.290+0000 7f156d19d700 1 --2- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f15540670e0 0x7f1554069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.290+0000 7f156d19d700 1 --2- 172.21.15.55:0/1142122314 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f156815c600 0x7f15680aeec0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.290+0000 7f156d19d700 1 --2- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f15680b3b70 0x7f15681590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.290+0000 7f156d19d700 1 --2- 172.21.15.55:0/1142122314 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1568142ea0 0x7f1568141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.290+0000 7f156d19d700 1 -- 172.21.15.55:0/1142122314 >> 172.21.15.55:0/1142122314 conn(0x7f15680ce430 msgr2=0x7f15680cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:50.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.291+0000 7f156d19d700 1 -- 172.21.15.55:0/1142122314 shutdown_connections 2023-10-06T03:48:50.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.291+0000 7f156d19d700 1 -- 172.21.15.55:0/1142122314 wait complete. 2023-10-06T03:48:50.311 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:48:50.381 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:50.379+0000 7fe1f5450180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:48:50.417 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:50.416+0000 7f4fc328e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:48:50.485 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:50.483+0000 7fe1f5450180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:48:50.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.500+0000 7fde2939d700 1 Processor -- start 2023-10-06T03:48:50.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.500+0000 7fde2939d700 1 -- start start 2023-10-06T03:48:50.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.501+0000 7fde2939d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fde24159420 0x7fde240b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:50.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.501+0000 7fde2939d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fde241597f0 0x7fde240b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:50.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.501+0000 7fde2939d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde2415d230 0x7fde2415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:50.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.501+0000 7fde2939d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde2415dbb0 con 0x7fde241597f0 2023-10-06T03:48:50.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.501+0000 7fde2939d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde240b11d0 con 0x7fde24159420 2023-10-06T03:48:50.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.501+0000 7fde2939d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde240b14b0 con 0x7fde2415d230 2023-10-06T03:48:50.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.502+0000 7fde237fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fde241597f0 0x7fde240b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:50.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.502+0000 7fde237fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fde241597f0 0x7fde240b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57108/0 (socket says 172.21.15.55:57108) 2023-10-06T03:48:50.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.502+0000 7fde237fe700 1 -- 172.21.15.55:0/922398513 learned_addr learned my addr 172.21.15.55:0/922398513 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:50.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.502+0000 7fde28b9c700 1 --2- 172.21.15.55:0/922398513 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde2415d230 0x7fde2415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:50.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.502+0000 7fde23fff700 1 --2- 172.21.15.55:0/922398513 >> v2:172.21.15.163:3300/0 conn(0x7fde24159420 0x7fde240b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:50.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.502+0000 7fde237fe700 1 -- 172.21.15.55:0/922398513 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde2415d230 msgr2=0x7fde2415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:50.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.502+0000 7fde237fe700 1 --2- 172.21.15.55:0/922398513 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde2415d230 0x7fde2415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.502+0000 7fde237fe700 1 -- 172.21.15.55:0/922398513 >> v2:172.21.15.163:3300/0 conn(0x7fde24159420 msgr2=0x7fde240b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:50.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.502+0000 7fde237fe700 1 --2- 172.21.15.55:0/922398513 >> v2:172.21.15.163:3300/0 conn(0x7fde24159420 0x7fde240b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.502+0000 7fde237fe700 1 -- 172.21.15.55:0/922398513 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde240b1790 con 0x7fde241597f0 2023-10-06T03:48:50.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.503+0000 7fde237fe700 1 --2- 172.21.15.55:0/922398513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde241597f0 0x7fde240b2270 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fde240b2f20 tx=0x7fde18009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5750d9fc1cec5701 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:50.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.503+0000 7fde22ffd700 1 -- 172.21.15.55:0/922398513 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1800b640 con 0x7fde241597f0 2023-10-06T03:48:50.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.503+0000 7fde22ffd700 1 -- 172.21.15.55:0/922398513 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1800b7a0 con 0x7fde241597f0 2023-10-06T03:48:50.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.503+0000 7fde22ffd700 1 -- 172.21.15.55:0/922398513 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1800bb60 con 0x7fde241597f0 2023-10-06T03:48:50.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.504+0000 7fde2939d700 1 -- 172.21.15.55:0/922398513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde241597f0 msgr2=0x7fde240b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:50.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.504+0000 7fde2939d700 1 --2- 172.21.15.55:0/922398513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde241597f0 0x7fde240b2270 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fde240b2f20 tx=0x7fde18009250 comp rx=0 tx=0).stop 2023-10-06T03:48:50.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.504+0000 7fde2939d700 1 -- 172.21.15.55:0/922398513 shutdown_connections 2023-10-06T03:48:50.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.504+0000 7fde2939d700 1 --2- 172.21.15.55:0/922398513 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde2415d230 0x7fde2415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.504+0000 7fde2939d700 1 --2- 172.21.15.55:0/922398513 >> v2:172.21.15.163:3300/0 conn(0x7fde24159420 0x7fde240b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.504+0000 7fde2939d700 1 --2- 172.21.15.55:0/922398513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde241597f0 0x7fde240b2270 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.504+0000 7fde2939d700 1 -- 172.21.15.55:0/922398513 >> 172.21.15.55:0/922398513 conn(0x7fde240d0660 msgr2=0x7fde240d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:50.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.504+0000 7fde2939d700 1 -- 172.21.15.55:0/922398513 shutdown_connections 2023-10-06T03:48:50.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.504+0000 7fde2939d700 1 -- 172.21.15.55:0/922398513 wait complete. 2023-10-06T03:48:50.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.505+0000 7fde2939d700 1 Processor -- start 2023-10-06T03:48:50.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.506+0000 7fde2939d700 1 -- start start 2023-10-06T03:48:50.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.506+0000 7fde2939d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde24159420 0x7fde240cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:50.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.506+0000 7fde2939d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde241597f0 0x7fde240b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:50.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.506+0000 7fde2939d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde2415d230 0x7fde240b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:50.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.506+0000 7fde2939d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2415a7e0 con 0x7fde24159420 2023-10-06T03:48:50.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.506+0000 7fde2939d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde240cc150 con 0x7fde241597f0 2023-10-06T03:48:50.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.506+0000 7fde2939d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2415a1d0 con 0x7fde2415d230 2023-10-06T03:48:50.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.507+0000 7fde23fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde24159420 0x7fde240cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:50.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.507+0000 7fde23fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde24159420 0x7fde240cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57124/0 (socket says 172.21.15.55:57124) 2023-10-06T03:48:50.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.507+0000 7fde23fff700 1 -- 172.21.15.55:0/2655214246 learned_addr learned my addr 172.21.15.55:0/2655214246 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:50.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.507+0000 7fde237fe700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde241597f0 0x7fde240b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:50.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.507+0000 7fde28b9c700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde2415d230 0x7fde240b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:50.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.507+0000 7fde23fff700 1 -- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde2415d230 msgr2=0x7fde240b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:50.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.507+0000 7fde23fff700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde2415d230 0x7fde240b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.507+0000 7fde23fff700 1 -- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde241597f0 msgr2=0x7fde240b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:50.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.507+0000 7fde23fff700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde241597f0 0x7fde240b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.507+0000 7fde23fff700 1 -- 172.21.15.55:0/2655214246 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde180094f0 con 0x7fde24159420 2023-10-06T03:48:50.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.507+0000 7fde28b9c700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde2415d230 0x7fde240b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:50.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.508+0000 7fde23fff700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde24159420 0x7fde240cbc10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fde1400ec80 tx=0x7fde140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:50.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.508+0000 7fde0bfff700 1 -- 172.21.15.55:0/2655214246 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1400c850 con 0x7fde24159420 2023-10-06T03:48:50.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.508+0000 7fde0bfff700 1 -- 172.21.15.55:0/2655214246 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1400c9b0 con 0x7fde24159420 2023-10-06T03:48:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.508+0000 7fde0bfff700 1 -- 172.21.15.55:0/2655214246 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1400cd50 con 0x7fde24159420 2023-10-06T03:48:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.508+0000 7fde2939d700 1 -- 172.21.15.55:0/2655214246 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde241552c0 con 0x7fde24159420 2023-10-06T03:48:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.508+0000 7fde2939d700 1 -- 172.21.15.55:0/2655214246 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde240b56b0 con 0x7fde24159420 2023-10-06T03:48:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.509+0000 7fde2939d700 1 -- 172.21.15.55:0/2655214246 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde240b58a0 con 0x7fde24159420 2023-10-06T03:48:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.510+0000 7fde0bfff700 1 -- 172.21.15.55:0/2655214246 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 30) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fde14013680 con 0x7fde24159420 2023-10-06T03:48:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.511+0000 7fde0bfff700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fde100670e0 0x7fde10069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.511+0000 7fde0bfff700 1 -- 172.21.15.55:0/2655214246 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6708+0+0 (secure 0 0 0) 0x7fde14084f80 con 0x7fde24159420 2023-10-06T03:48:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.512+0000 7fde237fe700 1 -- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fde100670e0 msgr2=0x7fde10069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:50.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.512+0000 7fde237fe700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fde100670e0 0x7fde10069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:48:50.516 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:50.515+0000 7f4fc328e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:48:50.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.516+0000 7fde0bfff700 1 -- 172.21.15.55:0/2655214246 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde14051ae0 con 0x7fde24159420 2023-10-06T03:48:50.676 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:50.674+0000 7fe1f5450180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:48:50.679 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:50.678+0000 7f4fc328e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:48:50.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.713+0000 7fde237fe700 1 -- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fde100670e0 msgr2=0x7fde10069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:48:50.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.713+0000 7fde237fe700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fde100670e0 0x7fde10069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:48:50.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.765+0000 7fde2939d700 1 -- 172.21.15.55:0/2655214246 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fde2404c3c0 con 0x7fde24159420 2023-10-06T03:48:50.767 INFO:tasks.ceph.mon.a.smithi055.stderr:2023-10-06T03:48:50.766+0000 7fe524ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:48:50.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.771+0000 7fde0bfff700 1 -- 172.21.15.55:0/2655214246 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7fde14055100 con 0x7fde24159420 2023-10-06T03:48:50.773 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:50.772+0000 7fe1f5450180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:48:50.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.773+0000 7fde2939d700 1 -- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fde100670e0 msgr2=0x7fde10069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:48:50.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.773+0000 7fde2939d700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fde100670e0 0x7fde10069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.773+0000 7fde2939d700 1 -- 172.21.15.55:0/2655214246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde24159420 msgr2=0x7fde240cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:50.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.773+0000 7fde2939d700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde24159420 0x7fde240cbc10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fde1400ec80 tx=0x7fde140091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.773+0000 7fde2939d700 1 -- 172.21.15.55:0/2655214246 shutdown_connections 2023-10-06T03:48:50.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.773+0000 7fde2939d700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fde100670e0 0x7fde10069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.773+0000 7fde2939d700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde2415d230 0x7fde240b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.773+0000 7fde2939d700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde24159420 0x7fde240cbc10 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.773+0000 7fde2939d700 1 --2- 172.21.15.55:0/2655214246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde241597f0 0x7fde240b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:50.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.773+0000 7fde2939d700 1 -- 172.21.15.55:0/2655214246 >> 172.21.15.55:0/2655214246 conn(0x7fde240d0660 msgr2=0x7fde240ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:50.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.774+0000 7fde2939d700 1 -- 172.21.15.55:0/2655214246 shutdown_connections 2023-10-06T03:48:50.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:50.774+0000 7fde2939d700 1 -- 172.21.15.55:0/2655214246 wait complete. 2023-10-06T03:48:50.777 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:48:50.777 INFO:tasks.ceph.mds.a.smithi055.stderr: -2226> 2023-10-06T03:48:27.906+0000 7f39441f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:48:50.791 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:50.789+0000 7f4fc328e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:48:50.798 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:48:50.823 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T03:48:50.874 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:50.872+0000 7fe1f5450180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:48:50.969 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:50.967+0000 7fe1f5450180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:48:51.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.001+0000 7f4f5d99d700 1 Processor -- start 2023-10-06T03:48:51.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.001+0000 7f4f5d99d700 1 -- start start 2023-10-06T03:48:51.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.002+0000 7f4f5d99d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4f580ccc00 0x7f4f58069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:51.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.002+0000 7f4f5d99d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f580c9f30 con 0x7f4f58159a60 2023-10-06T03:48:51.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.002+0000 7f4f5d99d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f580ca470 con 0x7f4f5806a020 2023-10-06T03:48:51.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.002+0000 7f4f5d99d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f5806dfc0 con 0x7f4f580ccc00 2023-10-06T03:48:51.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.002+0000 7f4f57fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4f580ccc00 0x7f4f58069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:51.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.003+0000 7f4f57fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4f580ccc00 0x7f4f58069ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:41282/0 (socket says 172.21.15.55:41282) 2023-10-06T03:48:51.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.003+0000 7f4f57fff700 1 -- 172.21.15.55:0/3616053753 learned_addr learned my addr 172.21.15.55:0/3616053753 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:51.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.003+0000 7f4f5c99b700 1 --1- 172.21.15.55:0/3616053753 >> v1:172.21.15.55:6789/0 conn(0x7f4f58159a60 0x7f4f580c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52564/0 (socket says 172.21.15.55:52564) 2023-10-06T03:48:51.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.003+0000 7f4f5d19c700 1 --1- 172.21.15.55:0/3616053753 >> v1:172.21.15.163:6789/0 conn(0x7f4f5806a020 0x7f4f580cb230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:34278/0 (socket says 172.21.15.55:34278) 2023-10-06T03:48:51.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.003+0000 7f4f57fff700 1 -- 172.21.15.55:0/3616053753 >> v1:172.21.15.163:6789/0 conn(0x7f4f5806a020 legacy=0x7f4f580cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:51.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.003+0000 7f4f57fff700 1 -- 172.21.15.55:0/3616053753 >> v1:172.21.15.55:6789/0 conn(0x7f4f58159a60 legacy=0x7f4f580c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:51.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.003+0000 7f4f577fe700 1 -- 172.21.15.55:0/3616053753 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 54795132 0 0) 0x7f4f580c9f30 con 0x7f4f58159a60 2023-10-06T03:48:51.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.003+0000 7f4f57fff700 1 -- 172.21.15.55:0/3616053753 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f580ca750 con 0x7f4f580ccc00 2023-10-06T03:48:51.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.003+0000 7f4f57fff700 1 --2- 172.21.15.55:0/3616053753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f580ccc00 0x7f4f58069ae0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4f440099b0 tx=0x7f4f44009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52c91ba0d746bb0e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:51.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.004+0000 7f4f577fe700 1 -- 172.21.15.55:0/3616053753 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400b8f0 con 0x7f4f580ccc00 2023-10-06T03:48:51.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.004+0000 7f4f577fe700 1 -- 172.21.15.55:0/3616053753 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4400ba50 con 0x7f4f580ccc00 2023-10-06T03:48:51.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.004+0000 7f4f577fe700 1 -- 172.21.15.55:0/3616053753 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400bdf0 con 0x7f4f580ccc00 2023-10-06T03:48:51.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.004+0000 7f4f5d99d700 1 -- 172.21.15.55:0/3616053753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f580ccc00 msgr2=0x7f4f58069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:51.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.004+0000 7f4f5d99d700 1 --2- 172.21.15.55:0/3616053753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f580ccc00 0x7f4f58069ae0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4f440099b0 tx=0x7f4f44009250 comp rx=0 tx=0).stop 2023-10-06T03:48:51.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.005+0000 7f4f5d99d700 1 -- 172.21.15.55:0/3616053753 shutdown_connections 2023-10-06T03:48:51.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.005+0000 7f4f5d99d700 1 --2- 172.21.15.55:0/3616053753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f580ccc00 0x7f4f58069ae0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:51.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.005+0000 7f4f5d99d700 1 -- 172.21.15.55:0/3616053753 >> 172.21.15.55:0/3616053753 conn(0x7f4f5806baa0 msgr2=0x7f4f5806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:51.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.005+0000 7f4f5d99d700 1 -- 172.21.15.55:0/3616053753 shutdown_connections 2023-10-06T03:48:51.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.005+0000 7f4f5d99d700 1 -- 172.21.15.55:0/3616053753 wait complete. 2023-10-06T03:48:51.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.006+0000 7f4f5d99d700 1 Processor -- start 2023-10-06T03:48:51.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.006+0000 7f4f5d99d700 1 -- start start 2023-10-06T03:48:51.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.007+0000 7f4f5d99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f5806a020 0x7f4f580c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:51.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.007+0000 7f4f5d99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f580ccc00 0x7f4f580c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:51.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.007+0000 7f4f5d99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f58159a60 0x7f4f580cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:51.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.007+0000 7f4f5d99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f580c6b00 con 0x7f4f580ccc00 2023-10-06T03:48:51.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.007+0000 7f4f5d99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f580c7440 con 0x7f4f58159a60 2023-10-06T03:48:51.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.007+0000 7f4f5d99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f580c7720 con 0x7f4f5806a020 2023-10-06T03:48:51.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.008+0000 7f4f57fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f580ccc00 0x7f4f580c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:51.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.008+0000 7f4f57fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f580ccc00 0x7f4f580c65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57140/0 (socket says 172.21.15.55:57140) 2023-10-06T03:48:51.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.008+0000 7f4f5c99b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f5806a020 0x7f4f580c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:51.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.008+0000 7f4f57fff700 1 -- 172.21.15.55:0/2335824012 learned_addr learned my addr 172.21.15.55:0/2335824012 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:51.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.008+0000 7f4f5d19c700 1 --2- 172.21.15.55:0/2335824012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f58159a60 0x7f4f580cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:51.014 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:51.008+0000 7f4fc328e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:48:51.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.008+0000 7f4f57fff700 1 -- 172.21.15.55:0/2335824012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f5806a020 msgr2=0x7f4f580c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:51.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.008+0000 7f4f57fff700 1 --2- 172.21.15.55:0/2335824012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f5806a020 0x7f4f580c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:51.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.008+0000 7f4f57fff700 1 -- 172.21.15.55:0/2335824012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f58159a60 msgr2=0x7f4f580cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.008+0000 7f4f57fff700 1 --2- 172.21.15.55:0/2335824012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f58159a60 0x7f4f580cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.008+0000 7f4f57fff700 1 -- 172.21.15.55:0/2335824012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f440094f0 con 0x7f4f580ccc00 2023-10-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.008+0000 7f4f5d19c700 1 --2- 172.21.15.55:0/2335824012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f58159a60 0x7f4f580cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.009+0000 7f4f57fff700 1 --2- 172.21.15.55:0/2335824012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f580ccc00 0x7f4f580c65c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4f44006f50 tx=0x7f4f44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.009+0000 7f4f54ff9700 1 -- 172.21.15.55:0/2335824012 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4401f070 con 0x7f4f580ccc00 2023-10-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.009+0000 7f4f54ff9700 1 -- 172.21.15.55:0/2335824012 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f44011bd0 con 0x7f4f580ccc00 2023-10-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.009+0000 7f4f54ff9700 1 -- 172.21.15.55:0/2335824012 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f44025420 con 0x7f4f580ccc00 2023-10-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.009+0000 7f4f5d99d700 1 -- 172.21.15.55:0/2335824012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f580c7a00 con 0x7f4f580ccc00 2023-10-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.009+0000 7f4f5d99d700 1 -- 172.21.15.55:0/2335824012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f580b2780 con 0x7f4f580ccc00 2023-10-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.011+0000 7f4f5d99d700 1 -- 172.21.15.55:0/2335824012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f580b29c0 con 0x7f4f580ccc00 2023-10-06T03:48:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.012+0000 7f4f54ff9700 1 -- 172.21.15.55:0/2335824012 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 31) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f4f44023070 con 0x7f4f580ccc00 2023-10-06T03:48:51.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.013+0000 7f4f54ff9700 1 -- 172.21.15.55:0/2335824012 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7095+0+0 (secure 0 0 0) 0x7f4f44085800 con 0x7f4f580ccc00 2023-10-06T03:48:51.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.017+0000 7f4f54ff9700 1 -- 172.21.15.55:0/2335824012 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f580b29c0 con 0x7f4f580ccc00 2023-10-06T03:48:51.055 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:51.051+0000 7fe17484b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:51.055 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:51.051+0000 7fe17484b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:51.056 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:51.051+0000 7fe17484b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:51.056 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:51.051+0000 7fe17484b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:51.056 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:51.051+0000 7fe17484b700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:51.056 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:51.053+0000 7fe171044700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:51.056 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:51.053+0000 7fe171044700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:51.056 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:51.053+0000 7fe171044700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:51.056 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:51.053+0000 7fe171044700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:51.057 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:48:51.053+0000 7fe171044700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:48:51.115 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:51.113+0000 7f4fc328e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:48:51.212 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:51.210+0000 7f4fc328e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:48:51.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.273+0000 7f4f5d99d700 1 -- 172.21.15.55:0/2335824012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4f580b29c0 con 0x7f4f580ccc00 2023-10-06T03:48:51.309 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:48:51.308+0000 7f4fc328e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:48:51.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.772+0000 7f4f54ff9700 1 -- 172.21.15.55:0/2335824012 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4f580b29c0 con 0x7f4f580ccc00 2023-10-06T03:48:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.775+0000 7f4f5d99d700 1 -- 172.21.15.55:0/2335824012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f580ccc00 msgr2=0x7f4f580c65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.775+0000 7f4f5d99d700 1 --2- 172.21.15.55:0/2335824012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f580ccc00 0x7f4f580c65c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4f44006f50 tx=0x7f4f44017000 comp rx=0 tx=0).stop 2023-10-06T03:48:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.775+0000 7f4f5d99d700 1 -- 172.21.15.55:0/2335824012 shutdown_connections 2023-10-06T03:48:51.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.775+0000 7f4f5d99d700 1 --2- 172.21.15.55:0/2335824012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f58159a60 0x7f4f580cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.775+0000 7f4f5d99d700 1 --2- 172.21.15.55:0/2335824012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f5806a020 0x7f4f580c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.775+0000 7f4f5d99d700 1 --2- 172.21.15.55:0/2335824012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f580ccc00 0x7f4f580c65c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.775+0000 7f4f5d99d700 1 -- 172.21.15.55:0/2335824012 >> 172.21.15.55:0/2335824012 conn(0x7f4f5806baa0 msgr2=0x7f4f58154360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.775+0000 7f4f5d99d700 1 -- 172.21.15.55:0/2335824012 shutdown_connections 2023-10-06T03:48:51.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:51.775+0000 7f4f5d99d700 1 -- 172.21.15.55:0/2335824012 wait complete. 2023-10-06T03:48:51.795 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:48:52.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.034+0000 7fa9c279d700 1 Processor -- start 2023-10-06T03:48:52.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.034+0000 7fa9c279d700 1 -- start start 2023-10-06T03:48:52.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.035+0000 7fa9c279d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa9bc0b0760 0x7fa9bc0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:52.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.035+0000 7fa9c279d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9bc0b0570 con 0x7fa9bc0ace60 2023-10-06T03:48:52.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.036+0000 7fa9c279d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9bc0af4d0 con 0x7fa9bc156350 2023-10-06T03:48:52.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.036+0000 7fa9c279d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9bc0b1140 con 0x7fa9bc0b0760 2023-10-06T03:48:52.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.036+0000 7fa9c1f9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa9bc0ace60 0x7fa9bc0ad260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52572/0 (socket says 172.21.15.55:52572) 2023-10-06T03:48:52.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.036+0000 7fa9c1f9c700 1 -- 172.21.15.55:0/3071908927 learned_addr learned my addr 172.21.15.55:0/3071908927 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:52.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.036+0000 7fa9c0f9a700 1 --2- 172.21.15.55:0/3071908927 >> v2:172.21.15.163:3300/0 conn(0x7fa9bc0b0760 0x7fa9bc0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:52.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.036+0000 7fa9b3fff700 1 -- 172.21.15.55:0/3071908927 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298386368 0 0) 0x7fa9bc0b0570 con 0x7fa9bc0ace60 2023-10-06T03:48:52.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.036+0000 7fa9b3fff700 1 -- 172.21.15.55:0/3071908927 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9a8003570 con 0x7fa9bc0ace60 2023-10-06T03:48:52.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.036+0000 7fa9b3fff700 1 -- 172.21.15.55:0/3071908927 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661154909 0 0) 0x7fa9bc0af4d0 con 0x7fa9bc156350 2023-10-06T03:48:52.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.036+0000 7fa9b3fff700 1 -- 172.21.15.55:0/3071908927 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9bc0b0570 con 0x7fa9bc156350 2023-10-06T03:48:52.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.036+0000 7fa9c0f9a700 1 -- 172.21.15.55:0/3071908927 >> v1:172.21.15.163:6789/0 conn(0x7fa9bc156350 legacy=0x7fa9bc157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:52.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.036+0000 7fa9c0f9a700 1 -- 172.21.15.55:0/3071908927 >> v1:172.21.15.55:6789/0 conn(0x7fa9bc0ace60 legacy=0x7fa9bc0ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:52.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.036+0000 7fa9c0f9a700 1 -- 172.21.15.55:0/3071908927 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc0af7b0 con 0x7fa9bc0b0760 2023-10-06T03:48:52.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.037+0000 7fa9c0f9a700 1 --2- 172.21.15.55:0/3071908927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9bc0b0760 0x7fa9bc0b0c00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0098e0 tx=0x7fa9ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=492d7cec6e147386 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:52.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.037+0000 7fa9b3fff700 1 -- 172.21.15.55:0/3071908927 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac011630 con 0x7fa9bc0b0760 2023-10-06T03:48:52.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.037+0000 7fa9b3fff700 1 -- 172.21.15.55:0/3071908927 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac011790 con 0x7fa9bc0b0760 2023-10-06T03:48:52.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.037+0000 7fa9b3fff700 1 -- 172.21.15.55:0/3071908927 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac011b10 con 0x7fa9bc0b0760 2023-10-06T03:48:52.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.038+0000 7fa9c279d700 1 -- 172.21.15.55:0/3071908927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9bc0b0760 msgr2=0x7fa9bc0b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:52.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.038+0000 7fa9c279d700 1 --2- 172.21.15.55:0/3071908927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9bc0b0760 0x7fa9bc0b0c00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0098e0 tx=0x7fa9ac0091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:52.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.038+0000 7fa9c279d700 1 -- 172.21.15.55:0/3071908927 shutdown_connections 2023-10-06T03:48:52.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.038+0000 7fa9c279d700 1 --2- 172.21.15.55:0/3071908927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9bc0b0760 0x7fa9bc0b0c00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:52.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.038+0000 7fa9c279d700 1 -- 172.21.15.55:0/3071908927 >> 172.21.15.55:0/3071908927 conn(0x7fa9bc068e20 msgr2=0x7fa9bc0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:52.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.038+0000 7fa9c279d700 1 -- 172.21.15.55:0/3071908927 shutdown_connections 2023-10-06T03:48:52.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.038+0000 7fa9c279d700 1 -- 172.21.15.55:0/3071908927 wait complete. 2023-10-06T03:48:52.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.039+0000 7fa9c279d700 1 Processor -- start 2023-10-06T03:48:52.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.040+0000 7fa9c279d700 1 -- start start 2023-10-06T03:48:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c279d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9bc0ace60 0x7fa9bc0ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c279d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9bc0b0760 0x7fa9bc14fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c279d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa9bc156350 0x7fa9bc0aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c279d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0cbf50 con 0x7fa9bc0b0760 2023-10-06T03:48:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c279d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0cb990 con 0x7fa9bc0ace60 2023-10-06T03:48:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c279d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9bc0cbc70 con 0x7fa9bc156350 2023-10-06T03:48:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c0f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9bc0b0760 0x7fa9bc14fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c0f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9bc0b0760 0x7fa9bc14fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57144/0 (socket says 172.21.15.55:57144) 2023-10-06T03:48:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c0f9a700 1 -- 172.21.15.55:0/2685206158 learned_addr learned my addr 172.21.15.55:0/2685206158 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c179b700 1 --2- 172.21.15.55:0/2685206158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9bc0ace60 0x7fa9bc0ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:52.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c1f9c700 1 --2- 172.21.15.55:0/2685206158 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa9bc156350 0x7fa9bc0aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:52.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c0f9a700 1 -- 172.21.15.55:0/2685206158 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa9bc156350 msgr2=0x7fa9bc0aaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:52.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c0f9a700 1 --2- 172.21.15.55:0/2685206158 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa9bc156350 0x7fa9bc0aaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:52.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c0f9a700 1 -- 172.21.15.55:0/2685206158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9bc0ace60 msgr2=0x7fa9bc0ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:52.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c0f9a700 1 --2- 172.21.15.55:0/2685206158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9bc0ace60 0x7fa9bc0ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:52.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.041+0000 7fa9c0f9a700 1 -- 172.21.15.55:0/2685206158 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac009420 con 0x7fa9bc0b0760 2023-10-06T03:48:52.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.042+0000 7fa9c0f9a700 1 --2- 172.21.15.55:0/2685206158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9bc0b0760 0x7fa9bc14fb90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009ed0 tx=0x7fa9ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:52.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.042+0000 7fa9b17fa700 1 -- 172.21.15.55:0/2685206158 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac011b10 con 0x7fa9bc0b0760 2023-10-06T03:48:52.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.042+0000 7fa9b17fa700 1 -- 172.21.15.55:0/2685206158 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac000c90 con 0x7fa9bc0b0760 2023-10-06T03:48:52.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.042+0000 7fa9c279d700 1 -- 172.21.15.55:0/2685206158 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9bc06d620 con 0x7fa9bc0b0760 2023-10-06T03:48:52.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.042+0000 7fa9b17fa700 1 -- 172.21.15.55:0/2685206158 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac01a680 con 0x7fa9bc0b0760 2023-10-06T03:48:52.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.042+0000 7fa9c279d700 1 -- 172.21.15.55:0/2685206158 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9bc0cb580 con 0x7fa9bc0b0760 2023-10-06T03:48:52.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.043+0000 7fa9c279d700 1 -- 172.21.15.55:0/2685206158 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9bc04c3c0 con 0x7fa9bc0b0760 2023-10-06T03:48:52.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.044+0000 7fa9b17fa700 1 -- 172.21.15.55:0/2685206158 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 32) v1 ==== 84578+0+0 (secure 0 0 0) 0x7fa9ac01aa30 con 0x7fa9bc0b0760 2023-10-06T03:48:52.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.045+0000 7fa9b17fa700 1 --2- 172.21.15.55:0/2685206158 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa9a0066d80 0x7fa9a0069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:52.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.045+0000 7fa9b17fa700 1 -- 172.21.15.55:0/2685206158 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7095+0+0 (secure 0 0 0) 0x7fa9ac086170 con 0x7fa9bc0b0760 2023-10-06T03:48:52.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.045+0000 7fa9c179b700 1 --2- 172.21.15.55:0/2685206158 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa9a0066d80 0x7fa9a0069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:52.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.046+0000 7fa9c179b700 1 --2- 172.21.15.55:0/2685206158 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa9a0066d80 0x7fa9a0069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8007fd0 tx=0x7fa9b8007d50 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:52.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.049+0000 7fa9b17fa700 1 -- 172.21.15.55:0/2685206158 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa9ac053120 con 0x7fa9bc0b0760 2023-10-06T03:48:52.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.250+0000 7fa9c279d700 1 -- 172.21.15.55:0/2685206158 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa9bc0afa90 con 0x7fa9bc0b0760 2023-10-06T03:48:52.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.995+0000 7fa9b17fa700 1 -- 172.21.15.55:0/2685206158 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa9ac01ace0 con 0x7fa9bc0b0760 2023-10-06T03:48:52.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.995+0000 7fa9b17fa700 1 -- 172.21.15.55:0/2685206158 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v84) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa9ac069630 con 0x7fa9bc0b0760 2023-10-06T03:48:53.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.998+0000 7fa9c279d700 1 -- 172.21.15.55:0/2685206158 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa9a0066d80 msgr2=0x7fa9a0069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:53.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.998+0000 7fa9c279d700 1 --2- 172.21.15.55:0/2685206158 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa9a0066d80 0x7fa9a0069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8007fd0 tx=0x7fa9b8007d50 comp rx=0 tx=0).stop 2023-10-06T03:48:53.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.998+0000 7fa9c279d700 1 -- 172.21.15.55:0/2685206158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9bc0b0760 msgr2=0x7fa9bc14fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.998+0000 7fa9c279d700 1 --2- 172.21.15.55:0/2685206158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9bc0b0760 0x7fa9bc14fb90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009ed0 tx=0x7fa9ac017000 comp rx=0 tx=0).stop 2023-10-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.999+0000 7fa9c279d700 1 -- 172.21.15.55:0/2685206158 shutdown_connections 2023-10-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.999+0000 7fa9c279d700 1 --2- 172.21.15.55:0/2685206158 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa9a0066d80 0x7fa9a0069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.999+0000 7fa9c279d700 1 --2- 172.21.15.55:0/2685206158 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa9bc156350 0x7fa9bc0aaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.999+0000 7fa9c279d700 1 --2- 172.21.15.55:0/2685206158 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa9bc0ace60 0x7fa9bc0ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.999+0000 7fa9c279d700 1 --2- 172.21.15.55:0/2685206158 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa9bc0b0760 0x7fa9bc14fb90 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.999+0000 7fa9c279d700 1 -- 172.21.15.55:0/2685206158 >> 172.21.15.55:0/2685206158 conn(0x7fa9bc068e20 msgr2=0x7fa9bc0abd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.999+0000 7fa9c279d700 1 -- 172.21.15.55:0/2685206158 shutdown_connections 2023-10-06T03:48:53.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:52.999+0000 7fa9c279d700 1 -- 172.21.15.55:0/2685206158 wait complete. 2023-10-06T03:48:53.002 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:48:53.021 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:48:53.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.214+0000 7fc91bfff700 1 Processor -- start 2023-10-06T03:48:53.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.214+0000 7fc91bfff700 1 -- start start 2023-10-06T03:48:53.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.215+0000 7fc91bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc91c0b0c00 0x7fc91c0ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:53.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.215+0000 7fc91bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc91c0b0570 con 0x7fc91c156350 2023-10-06T03:48:53.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.215+0000 7fc91bfff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc91c0af4d0 con 0x7fc91c0b0760 2023-10-06T03:48:53.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.215+0000 7fc91bfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc91c0af7b0 con 0x7fc91c0b0c00 2023-10-06T03:48:53.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.215+0000 7fc91affd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc91c156350 0x7fc91c157370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43162/0 (socket says 172.21.15.55:43162) 2023-10-06T03:48:53.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.215+0000 7fc91affd700 1 -- 172.21.15.55:0/2278355806 learned_addr learned my addr 172.21.15.55:0/2278355806 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.215+0000 7fc91b7fe700 1 --2- 172.21.15.55:0/2278355806 >> v2:172.21.15.163:3300/0 conn(0x7fc91c0b0c00 0x7fc91c0ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87916010 0 0) 0x7fc91c0b0570 con 0x7fc91c156350 2023-10-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc904003570 con 0x7fc91c156350 2023-10-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735523549 0 0) 0x7fc91c0af4d0 con 0x7fc91c0b0760 2023-10-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc91c0b0570 con 0x7fc91c0b0760 2023-10-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 543876458 0 0) 0x7fc904003570 con 0x7fc91c156350 2023-10-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 >> v2:172.21.15.163:3300/0 conn(0x7fc91c0b0c00 msgr2=0x7fc91c0ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:53.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 --2- 172.21.15.55:0/2278355806 >> v2:172.21.15.163:3300/0 conn(0x7fc91c0b0c00 0x7fc91c0ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 >> v1:172.21.15.163:6789/0 conn(0x7fc91c0b0760 legacy=0x7fc91c0ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c0afa90 con 0x7fc91c156350 2023-10-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fc90c001470 con 0x7fc91c156350 2023-10-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 >> v1:172.21.15.55:6789/0 conn(0x7fc91c156350 legacy=0x7fc91c157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 --2- 172.21.15.55:0/2278355806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc904003c40 0x7fc904004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 --2- 172.21.15.55:0/2278355806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc904004810 0x7fc904004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 --2- 172.21.15.55:0/2278355806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc904005210 0x7fc90400dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0af7b0 con 0x7fc904005210 2023-10-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90400e3c0 con 0x7fc904004810 2023-10-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90400e580 con 0x7fc904003c40 2023-10-06T03:48:53.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.216+0000 7fc91b7fe700 1 --2- 172.21.15.55:0/2278355806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc904005210 0x7fc90400dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.217+0000 7fc91affd700 1 --2- 172.21.15.55:0/2278355806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc904003c40 0x7fc904004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.217+0000 7fc91a7fc700 1 --2- 172.21.15.55:0/2278355806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc904004810 0x7fc904004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.217+0000 7fc91b7fe700 1 -- 172.21.15.55:0/2278355806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc904003c40 msgr2=0x7fc904004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.217+0000 7fc91b7fe700 1 --2- 172.21.15.55:0/2278355806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc904003c40 0x7fc904004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.217+0000 7fc91b7fe700 1 -- 172.21.15.55:0/2278355806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc904004810 msgr2=0x7fc904004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.217+0000 7fc91b7fe700 1 --2- 172.21.15.55:0/2278355806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc904004810 0x7fc904004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.217+0000 7fc91b7fe700 1 -- 172.21.15.55:0/2278355806 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc904003570 con 0x7fc904005210 2023-10-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.217+0000 7fc91b7fe700 1 -- 172.21.15.55:0/2278355806 reap_dead start 2023-10-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.217+0000 7fc91b7fe700 1 --2- 172.21.15.55:0/2278355806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc904005210 0x7fc90400dc70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc9140094d0 tx=0x7fc91400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6494e201c2b1c859 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.217+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91400ea90 con 0x7fc904005210 2023-10-06T03:48:53.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.217+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91400f040 con 0x7fc904005210 2023-10-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.217+0000 7fc919ffb700 1 -- 172.21.15.55:0/2278355806 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91400ee60 con 0x7fc904005210 2023-10-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.218+0000 7fc91bfff700 1 -- 172.21.15.55:0/2278355806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc904005210 msgr2=0x7fc90400dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.218+0000 7fc91bfff700 1 --2- 172.21.15.55:0/2278355806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc904005210 0x7fc90400dc70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc9140094d0 tx=0x7fc91400b040 comp rx=0 tx=0).stop 2023-10-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.218+0000 7fc91bfff700 1 -- 172.21.15.55:0/2278355806 shutdown_connections 2023-10-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.218+0000 7fc91bfff700 1 --2- 172.21.15.55:0/2278355806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc904005210 0x7fc90400dc70 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.218+0000 7fc91bfff700 1 -- 172.21.15.55:0/2278355806 >> 172.21.15.55:0/2278355806 conn(0x7fc91c068e20 msgr2=0x7fc91c0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.218+0000 7fc91bfff700 1 -- 172.21.15.55:0/2278355806 shutdown_connections 2023-10-06T03:48:53.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.218+0000 7fc91bfff700 1 -- 172.21.15.55:0/2278355806 wait complete. 2023-10-06T03:48:53.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.219+0000 7fc91bfff700 1 Processor -- start 2023-10-06T03:48:53.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.220+0000 7fc91bfff700 1 -- start start 2023-10-06T03:48:53.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.220+0000 7fc91bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc904005210 0x7fc91c1661c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:53.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.220+0000 7fc91bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc91c0c1480 0x7fc91c162b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:53.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.220+0000 7fc91bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc91c0c3520 0x7fc91c0c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:53.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.220+0000 7fc91bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c166700 con 0x7fc91c0c1480 2023-10-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.220+0000 7fc91bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0c3f00 con 0x7fc91c0c3520 2023-10-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.220+0000 7fc91bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0c1990 con 0x7fc904005210 2023-10-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.221+0000 7fc91a7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc91c0c1480 0x7fc91c162b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.221+0000 7fc91affd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc904005210 0x7fc91c1661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.221+0000 7fc91affd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc904005210 0x7fc91c1661c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:51600/0 (socket says 172.21.15.55:51600) 2023-10-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.221+0000 7fc91affd700 1 -- 172.21.15.55:0/1241927085 learned_addr learned my addr 172.21.15.55:0/1241927085 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.221+0000 7fc91a7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc91c0c1480 0x7fc91c162b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41202/0 (socket says 172.21.15.55:41202) 2023-10-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.221+0000 7fc91b7fe700 1 --2- 172.21.15.55:0/1241927085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc91c0c3520 0x7fc91c0c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.221+0000 7fc91a7fc700 1 -- 172.21.15.55:0/1241927085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc904005210 msgr2=0x7fc91c1661c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.221+0000 7fc91a7fc700 1 --2- 172.21.15.55:0/1241927085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc904005210 0x7fc91c1661c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.221+0000 7fc91a7fc700 1 -- 172.21.15.55:0/1241927085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc91c0c3520 msgr2=0x7fc91c0c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:53.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.221+0000 7fc91a7fc700 1 --2- 172.21.15.55:0/1241927085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc91c0c3520 0x7fc91c0c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.221+0000 7fc91a7fc700 1 -- 172.21.15.55:0/1241927085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c0c1c70 con 0x7fc91c0c1480 2023-10-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.222+0000 7fc91a7fc700 1 --2- 172.21.15.55:0/1241927085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc91c0c1480 0x7fc91c162b30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc910002ab0 tx=0x7fc910007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.222+0000 7fc902ffd700 1 -- 172.21.15.55:0/1241927085 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc910006a30 con 0x7fc91c0c1480 2023-10-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.222+0000 7fc902ffd700 1 -- 172.21.15.55:0/1241927085 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc910006b90 con 0x7fc91c0c1480 2023-10-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.222+0000 7fc902ffd700 1 -- 172.21.15.55:0/1241927085 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc910007050 con 0x7fc91c0c1480 2023-10-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.222+0000 7fc91bfff700 1 -- 172.21.15.55:0/1241927085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91c0c1f50 con 0x7fc91c0c1480 2023-10-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.222+0000 7fc91bfff700 1 -- 172.21.15.55:0/1241927085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91c0c2510 con 0x7fc91c0c1480 2023-10-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.223+0000 7fc91bfff700 1 -- 172.21.15.55:0/1241927085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91c0c2730 con 0x7fc91c0c1480 2023-10-06T03:48:53.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.224+0000 7fc902ffd700 1 -- 172.21.15.55:0/1241927085 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc9100073c0 con 0x7fc91c0c1480 2023-10-06T03:48:53.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.226+0000 7fc902ffd700 1 --2- 172.21.15.55:0/1241927085 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc908066f30 0x7fc9080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:53.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.226+0000 7fc902ffd700 1 -- 172.21.15.55:0/1241927085 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6657+0+0 (secure 0 0 0) 0x7fc910084b00 con 0x7fc91c0c1480 2023-10-06T03:48:53.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.226+0000 7fc91affd700 1 --2- 172.21.15.55:0/1241927085 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc908066f30 0x7fc9080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:53.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.227+0000 7fc91affd700 1 --2- 172.21.15.55:0/1241927085 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc908066f30 0x7fc9080693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc90c001120 tx=0x7fc90c001d40 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:53.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.230+0000 7fc902ffd700 1 -- 172.21.15.55:0/1241927085 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9100519b0 con 0x7fc91c0c1480 2023-10-06T03:48:53.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.431+0000 7fc91bfff700 1 -- 172.21.15.55:0/1241927085 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc91c04c3c0 con 0x7fc91c0c1480 2023-10-06T03:48:53.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:53.997+0000 7fc902ffd700 1 -- 172.21.15.55:0/1241927085 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v85) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc910054fd0 con 0x7fc91c0c1480 2023-10-06T03:48:54.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.002+0000 7fc91bfff700 1 -- 172.21.15.55:0/1241927085 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc908066f30 msgr2=0x7fc9080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:54.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.002+0000 7fc91bfff700 1 --2- 172.21.15.55:0/1241927085 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc908066f30 0x7fc9080693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc90c001120 tx=0x7fc90c001d40 comp rx=0 tx=0).stop 2023-10-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.002+0000 7fc91bfff700 1 -- 172.21.15.55:0/1241927085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc91c0c1480 msgr2=0x7fc91c162b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.002+0000 7fc91bfff700 1 --2- 172.21.15.55:0/1241927085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc91c0c1480 0x7fc91c162b30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc910002ab0 tx=0x7fc910007d30 comp rx=0 tx=0).stop 2023-10-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.002+0000 7fc91bfff700 1 -- 172.21.15.55:0/1241927085 shutdown_connections 2023-10-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.002+0000 7fc91bfff700 1 --2- 172.21.15.55:0/1241927085 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc908066f30 0x7fc9080693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.002+0000 7fc91bfff700 1 --2- 172.21.15.55:0/1241927085 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc91c0c3520 0x7fc91c0c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.002+0000 7fc91bfff700 1 --2- 172.21.15.55:0/1241927085 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc904005210 0x7fc91c1661c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.002+0000 7fc91bfff700 1 --2- 172.21.15.55:0/1241927085 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc91c0c1480 0x7fc91c162b30 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.002+0000 7fc91bfff700 1 -- 172.21.15.55:0/1241927085 >> 172.21.15.55:0/1241927085 conn(0x7fc91c068e20 msgr2=0x7fc91c0707c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.002+0000 7fc91bfff700 1 -- 172.21.15.55:0/1241927085 shutdown_connections 2023-10-06T03:48:54.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.002+0000 7fc91bfff700 1 -- 172.21.15.55:0/1241927085 wait complete. 2023-10-06T03:48:54.005 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2023-10-06T03:48:54.026 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:48:54.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.221+0000 7fe87c9fe700 1 Processor -- start 2023-10-06T03:48:54.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.221+0000 7fe87c9fe700 1 -- start start 2023-10-06T03:48:54.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.223+0000 7fe87c9fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8781407e0 0x7fe87806a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:54.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.223+0000 7fe87c9fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe878140bb0 0x7fe87806a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:54.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.223+0000 7fe87c9fe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8780c6820 con 0x7fe87806d4f0 2023-10-06T03:48:54.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.223+0000 7fe87c9fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8780c6de0 con 0x7fe8781407e0 2023-10-06T03:48:54.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.223+0000 7fe87c9fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe878141270 con 0x7fe878140bb0 2023-10-06T03:48:54.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.223+0000 7fe876d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8781407e0 0x7fe87806a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:54.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.223+0000 7fe876d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8781407e0 0x7fe87806a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41218/0 (socket says 172.21.15.55:41218) 2023-10-06T03:48:54.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.223+0000 7fe876d9d700 1 -- 172.21.15.55:0/82870983 learned_addr learned my addr 172.21.15.55:0/82870983 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:54.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.223+0000 7fe87659c700 1 --2- 172.21.15.55:0/82870983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe878140bb0 0x7fe87806a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.223+0000 7fe876d9d700 1 -- 172.21.15.55:0/82870983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe878140bb0 msgr2=0x7fe87806a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.223+0000 7fe876d9d700 1 --2- 172.21.15.55:0/82870983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe878140bb0 0x7fe87806a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.224+0000 7fe876d9d700 1 -- 172.21.15.55:0/82870983 >> v1:172.21.15.163:6789/0 conn(0x7fe87806d4f0 legacy=0x7fe87815b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.224+0000 7fe876d9d700 1 -- 172.21.15.55:0/82870983 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87806acf0 con 0x7fe8781407e0 2023-10-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.224+0000 7fe87659c700 1 --2- 172.21.15.55:0/82870983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe878140bb0 0x7fe87806a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.224+0000 7fe876d9d700 1 --2- 172.21.15.55:0/82870983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8781407e0 0x7fe87806a180 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe8680099b0 tx=0x7fe868009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e4e250dffd0d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.224+0000 7fe875d9b700 1 -- 172.21.15.55:0/82870983 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86800b860 con 0x7fe8781407e0 2023-10-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.224+0000 7fe875d9b700 1 -- 172.21.15.55:0/82870983 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86800b9c0 con 0x7fe8781407e0 2023-10-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.224+0000 7fe875d9b700 1 -- 172.21.15.55:0/82870983 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86800bd60 con 0x7fe8781407e0 2023-10-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.225+0000 7fe87c9fe700 1 -- 172.21.15.55:0/82870983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8781407e0 msgr2=0x7fe87806a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:54.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.225+0000 7fe87c9fe700 1 --2- 172.21.15.55:0/82870983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8781407e0 0x7fe87806a180 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe8680099b0 tx=0x7fe868009250 comp rx=0 tx=0).stop 2023-10-06T03:48:54.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.225+0000 7fe87c9fe700 1 -- 172.21.15.55:0/82870983 shutdown_connections 2023-10-06T03:48:54.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.225+0000 7fe87c9fe700 1 --2- 172.21.15.55:0/82870983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8781407e0 0x7fe87806a180 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.225+0000 7fe87c9fe700 1 --2- 172.21.15.55:0/82870983 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe878140bb0 0x7fe87806a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.225+0000 7fe87c9fe700 1 -- 172.21.15.55:0/82870983 >> 172.21.15.55:0/82870983 conn(0x7fe8780cc1e0 msgr2=0x7fe8780cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:54.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.225+0000 7fe87c9fe700 1 -- 172.21.15.55:0/82870983 shutdown_connections 2023-10-06T03:48:54.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.225+0000 7fe87c9fe700 1 -- 172.21.15.55:0/82870983 wait complete. 2023-10-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.226+0000 7fe87c9fe700 1 Processor -- start 2023-10-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.226+0000 7fe87c9fe700 1 -- start start 2023-10-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.227+0000 7fe87c9fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87806d4f0 0x7fe87815c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.227+0000 7fe87c9fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8781407e0 0x7fe8780b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.227+0000 7fe87c9fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe878140bb0 0x7fe87813ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.227+0000 7fe87c9fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8780afb50 con 0x7fe87806d4f0 2023-10-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.227+0000 7fe87c9fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87815f790 con 0x7fe8781407e0 2023-10-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.227+0000 7fe87c9fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8780b0390 con 0x7fe878140bb0 2023-10-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.228+0000 7fe876d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87806d4f0 0x7fe87815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:54.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.228+0000 7fe876d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87806d4f0 0x7fe87815c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41226/0 (socket says 172.21.15.55:41226) 2023-10-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.228+0000 7fe876d9d700 1 -- 172.21.15.55:0/117529211 learned_addr learned my addr 172.21.15.55:0/117529211 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.228+0000 7fe87659c700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8781407e0 0x7fe8780b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.228+0000 7fe87759e700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe878140bb0 0x7fe87813ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.228+0000 7fe876d9d700 1 -- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe878140bb0 msgr2=0x7fe87813ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.228+0000 7fe876d9d700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe878140bb0 0x7fe87813ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.228+0000 7fe876d9d700 1 -- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8781407e0 msgr2=0x7fe8780b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.228+0000 7fe876d9d700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8781407e0 0x7fe8780b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.228+0000 7fe876d9d700 1 -- 172.21.15.55:0/117529211 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8680094f0 con 0x7fe87806d4f0 2023-10-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.228+0000 7fe87759e700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe878140bb0 0x7fe87813ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:54.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.228+0000 7fe876d9d700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87806d4f0 0x7fe87815c440 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe868009290 tx=0x7fe868017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:54.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.229+0000 7fe85effd700 1 -- 172.21.15.55:0/117529211 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868024440 con 0x7fe87806d4f0 2023-10-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.229+0000 7fe85effd700 1 -- 172.21.15.55:0/117529211 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8680245a0 con 0x7fe87806d4f0 2023-10-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.229+0000 7fe85effd700 1 -- 172.21.15.55:0/117529211 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868024960 con 0x7fe87806d4f0 2023-10-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.229+0000 7fe87c9fe700 1 -- 172.21.15.55:0/117529211 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87815f4b0 con 0x7fe87806d4f0 2023-10-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.229+0000 7fe87c9fe700 1 -- 172.21.15.55:0/117529211 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8780b00a0 con 0x7fe87806d4f0 2023-10-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.230+0000 7fe87c9fe700 1 -- 172.21.15.55:0/117529211 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe87806c450 con 0x7fe87806d4f0 2023-10-06T03:48:54.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.231+0000 7fe85effd700 1 -- 172.21.15.55:0/117529211 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe86801a420 con 0x7fe87806d4f0 2023-10-06T03:48:54.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.232+0000 7fe85effd700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe864066f30 0x7fe8640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:54.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.232+0000 7fe85effd700 1 -- 172.21.15.55:0/117529211 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6243+0+0 (secure 0 0 0) 0x7fe8680866a0 con 0x7fe87806d4f0 2023-10-06T03:48:54.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.232+0000 7fe87659c700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe864066f30 0x7fe8640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:54.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.233+0000 7fe87659c700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe864066f30 0x7fe8640693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe87815c3a0 tx=0x7fe86c008040 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:54.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.236+0000 7fe85effd700 1 -- 172.21.15.55:0/117529211 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe868053770 con 0x7fe87806d4f0 2023-10-06T03:48:54.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.446+0000 7fe87c9fe700 1 -- 172.21.15.55:0/117529211 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe87806afd0 con 0x7fe87806d4f0 2023-10-06T03:48:54.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.450+0000 7fe85effd700 1 -- 172.21.15.55:0/117529211 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe868056d90 con 0x7fe87806d4f0 2023-10-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.453+0000 7fe87c9fe700 1 -- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe864066f30 msgr2=0x7fe8640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.453+0000 7fe87c9fe700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe864066f30 0x7fe8640693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe87815c3a0 tx=0x7fe86c008040 comp rx=0 tx=0).stop 2023-10-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.453+0000 7fe87c9fe700 1 -- 172.21.15.55:0/117529211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87806d4f0 msgr2=0x7fe87815c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.453+0000 7fe87c9fe700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87806d4f0 0x7fe87815c440 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe868009290 tx=0x7fe868017000 comp rx=0 tx=0).stop 2023-10-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.453+0000 7fe87c9fe700 1 -- 172.21.15.55:0/117529211 shutdown_connections 2023-10-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.453+0000 7fe87c9fe700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe864066f30 0x7fe8640693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.453+0000 7fe87c9fe700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe878140bb0 0x7fe87813ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.453+0000 7fe87c9fe700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe87806d4f0 0x7fe87815c440 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.453+0000 7fe87c9fe700 1 --2- 172.21.15.55:0/117529211 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8781407e0 0x7fe8780b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.453+0000 7fe87c9fe700 1 -- 172.21.15.55:0/117529211 >> 172.21.15.55:0/117529211 conn(0x7fe8780cc1e0 msgr2=0x7fe87806a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:54.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.453+0000 7fe87c9fe700 1 -- 172.21.15.55:0/117529211 shutdown_connections 2023-10-06T03:48:54.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.453+0000 7fe87c9fe700 1 -- 172.21.15.55:0/117529211 wait complete. 2023-10-06T03:48:54.472 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T03:48:54.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.659+0000 7fd9e939d700 1 Processor -- start 2023-10-06T03:48:54.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.659+0000 7fd9e939d700 1 -- start start 2023-10-06T03:48:54.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.660+0000 7fd9e939d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9e40b4e70 0x7fd9e40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:54.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.660+0000 7fd9e939d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e40b1b40 0x7fd9e415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:54.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.660+0000 7fd9e939d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e40cfb00 con 0x7fd9e40b16d0 2023-10-06T03:48:54.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.660+0000 7fd9e939d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e40b5c40 con 0x7fd9e40b4e70 2023-10-06T03:48:54.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.660+0000 7fd9e939d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e40b5f20 con 0x7fd9e40b1b40 2023-10-06T03:48:54.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.661+0000 7fd9e3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9e40b4e70 0x7fd9e40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:54.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.661+0000 7fd9e3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9e40b4e70 0x7fd9e40ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41228/0 (socket says 172.21.15.55:41228) 2023-10-06T03:48:54.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.661+0000 7fd9e3fff700 1 -- 172.21.15.55:0/2013626808 learned_addr learned my addr 172.21.15.55:0/2013626808 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:54.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.661+0000 7fd9e8b9c700 1 --2- 172.21.15.55:0/2013626808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e40b1b40 0x7fd9e415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:54.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.661+0000 7fd9e3fff700 1 -- 172.21.15.55:0/2013626808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e40b1b40 msgr2=0x7fd9e415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:54.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.661+0000 7fd9e3fff700 1 --2- 172.21.15.55:0/2013626808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e40b1b40 0x7fd9e415d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.661+0000 7fd9e3fff700 1 -- 172.21.15.55:0/2013626808 >> v1:172.21.15.163:6789/0 conn(0x7fd9e40b16d0 legacy=0x7fd9e415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:54.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.661+0000 7fd9e3fff700 1 -- 172.21.15.55:0/2013626808 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e40b6200 con 0x7fd9e40b4e70 2023-10-06T03:48:54.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.661+0000 7fd9e8b9c700 1 --2- 172.21.15.55:0/2013626808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e40b1b40 0x7fd9e415d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:54.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.661+0000 7fd9e3fff700 1 --2- 172.21.15.55:0/2013626808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e40b4e70 0x7fd9e40ccff0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40099b0 tx=0x7fd9d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf068cdf923173a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:54.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.662+0000 7fd9e2ffd700 1 -- 172.21.15.55:0/2013626808 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d400b860 con 0x7fd9e40b4e70 2023-10-06T03:48:54.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.662+0000 7fd9e2ffd700 1 -- 172.21.15.55:0/2013626808 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d400b9c0 con 0x7fd9e40b4e70 2023-10-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.662+0000 7fd9e2ffd700 1 -- 172.21.15.55:0/2013626808 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d400bd60 con 0x7fd9e40b4e70 2023-10-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.662+0000 7fd9e939d700 1 -- 172.21.15.55:0/2013626808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e40b4e70 msgr2=0x7fd9e40ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.662+0000 7fd9e939d700 1 --2- 172.21.15.55:0/2013626808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e40b4e70 0x7fd9e40ccff0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40099b0 tx=0x7fd9d4009250 comp rx=0 tx=0).stop 2023-10-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.662+0000 7fd9e939d700 1 -- 172.21.15.55:0/2013626808 shutdown_connections 2023-10-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.662+0000 7fd9e939d700 1 --2- 172.21.15.55:0/2013626808 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e40b1b40 0x7fd9e415d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.662+0000 7fd9e939d700 1 --2- 172.21.15.55:0/2013626808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e40b4e70 0x7fd9e40ccff0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.662+0000 7fd9e939d700 1 -- 172.21.15.55:0/2013626808 >> 172.21.15.55:0/2013626808 conn(0x7fd9e40d10c0 msgr2=0x7fd9e40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:54.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.662+0000 7fd9e939d700 1 -- 172.21.15.55:0/2013626808 shutdown_connections 2023-10-06T03:48:54.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.662+0000 7fd9e939d700 1 -- 172.21.15.55:0/2013626808 wait complete. 2023-10-06T03:48:54.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.664+0000 7fd9e939d700 1 Processor -- start 2023-10-06T03:48:54.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.664+0000 7fd9e939d700 1 -- start start 2023-10-06T03:48:54.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.665+0000 7fd9e939d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e40b16d0 0x7fd9e4147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:54.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.665+0000 7fd9e939d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e40b1b40 0x7fd9e40a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:54.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.665+0000 7fd9e939d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e40b4e70 0x7fd9e40a7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:54.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.665+0000 7fd9e939d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e40a7980 con 0x7fd9e40b16d0 2023-10-06T03:48:54.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.665+0000 7fd9e939d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e40ca9b0 con 0x7fd9e40b4e70 2023-10-06T03:48:54.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.665+0000 7fd9e939d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e40c9de0 con 0x7fd9e40b1b40 2023-10-06T03:48:54.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.665+0000 7fd9e3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e40b16d0 0x7fd9e4147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.665+0000 7fd9e3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e40b16d0 0x7fd9e4147180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41244/0 (socket says 172.21.15.55:41244) 2023-10-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.665+0000 7fd9e3fff700 1 -- 172.21.15.55:0/3137305679 learned_addr learned my addr 172.21.15.55:0/3137305679 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.665+0000 7fd9e8b9c700 1 --2- 172.21.15.55:0/3137305679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e40b4e70 0x7fd9e40a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.665+0000 7fd9e37fe700 1 --2- 172.21.15.55:0/3137305679 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e40b1b40 0x7fd9e40a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.666+0000 7fd9e3fff700 1 -- 172.21.15.55:0/3137305679 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e40b1b40 msgr2=0x7fd9e40a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:54.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.666+0000 7fd9e3fff700 1 --2- 172.21.15.55:0/3137305679 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e40b1b40 0x7fd9e40a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.666+0000 7fd9e3fff700 1 -- 172.21.15.55:0/3137305679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e40b4e70 msgr2=0x7fd9e40a7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.666+0000 7fd9e3fff700 1 --2- 172.21.15.55:0/3137305679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e40b4e70 0x7fd9e40a7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.666+0000 7fd9e3fff700 1 -- 172.21.15.55:0/3137305679 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d40094f0 con 0x7fd9e40b16d0 2023-10-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.666+0000 7fd9e3fff700 1 --2- 172.21.15.55:0/3137305679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e40b16d0 0x7fd9e4147180 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009290 tx=0x7fd9d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.666+0000 7fd9cbfff700 1 -- 172.21.15.55:0/3137305679 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4011e00 con 0x7fd9e40b16d0 2023-10-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.666+0000 7fd9cbfff700 1 -- 172.21.15.55:0/3137305679 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d401a450 con 0x7fd9e40b16d0 2023-10-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.666+0000 7fd9cbfff700 1 -- 172.21.15.55:0/3137305679 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d401a810 con 0x7fd9e40b16d0 2023-10-06T03:48:54.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.666+0000 7fd9e939d700 1 -- 172.21.15.55:0/3137305679 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e40ca080 con 0x7fd9e40b16d0 2023-10-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.666+0000 7fd9e939d700 1 -- 172.21.15.55:0/3137305679 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e40ca640 con 0x7fd9e40b16d0 2023-10-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.667+0000 7fd9e939d700 1 -- 172.21.15.55:0/3137305679 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e4088990 con 0x7fd9e40b16d0 2023-10-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.668+0000 7fd9cbfff700 1 -- 172.21.15.55:0/3137305679 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9d401abc0 con 0x7fd9e40b16d0 2023-10-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.669+0000 7fd9cbfff700 1 --2- 172.21.15.55:0/3137305679 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9d0066f60 0x7fd9d0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.669+0000 7fd9cbfff700 1 -- 172.21.15.55:0/3137305679 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6243+0+0 (secure 0 0 0) 0x7fd9d4085bf0 con 0x7fd9e40b16d0 2023-10-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.669+0000 7fd9e37fe700 1 --2- 172.21.15.55:0/3137305679 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9d0066f60 0x7fd9d0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:54.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.670+0000 7fd9e37fe700 1 --2- 172.21.15.55:0/3137305679 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9d0066f60 0x7fd9d0069410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd9e41581a0 tx=0x7fd9d8007ec0 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:54.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.673+0000 7fd9cbfff700 1 -- 172.21.15.55:0/3137305679 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9d4052c40 con 0x7fd9e40b16d0 2023-10-06T03:48:54.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.874+0000 7fd9e939d700 1 -- 172.21.15.55:0/3137305679 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fd9e404bb80 con 0x7fd9e40b16d0 2023-10-06T03:48:54.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.879+0000 7fd9cbfff700 1 -- 172.21.15.55:0/3137305679 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v26) v1 ==== 115+0+0 (secure 0 0 0) 0x7fd9d4056260 con 0x7fd9e40b16d0 2023-10-06T03:48:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.882+0000 7fd9e939d700 1 -- 172.21.15.55:0/3137305679 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9d0066f60 msgr2=0x7fd9d0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.882+0000 7fd9e939d700 1 --2- 172.21.15.55:0/3137305679 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9d0066f60 0x7fd9d0069410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd9e41581a0 tx=0x7fd9d8007ec0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.882+0000 7fd9e939d700 1 -- 172.21.15.55:0/3137305679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e40b16d0 msgr2=0x7fd9e4147180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.882+0000 7fd9e939d700 1 --2- 172.21.15.55:0/3137305679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e40b16d0 0x7fd9e4147180 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009290 tx=0x7fd9d4017000 comp rx=0 tx=0).stop 2023-10-06T03:48:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.882+0000 7fd9e939d700 1 -- 172.21.15.55:0/3137305679 shutdown_connections 2023-10-06T03:48:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.882+0000 7fd9e939d700 1 --2- 172.21.15.55:0/3137305679 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9d0066f60 0x7fd9d0069410 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.882+0000 7fd9e939d700 1 --2- 172.21.15.55:0/3137305679 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e40b4e70 0x7fd9e40a7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.882+0000 7fd9e939d700 1 --2- 172.21.15.55:0/3137305679 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e40b16d0 0x7fd9e4147180 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.882+0000 7fd9e939d700 1 --2- 172.21.15.55:0/3137305679 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e40b1b40 0x7fd9e40a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:54.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.882+0000 7fd9e939d700 1 -- 172.21.15.55:0/3137305679 >> 172.21.15.55:0/3137305679 conn(0x7fd9e40d10c0 msgr2=0x7fd9e40b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:54.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.882+0000 7fd9e939d700 1 -- 172.21.15.55:0/3137305679 shutdown_connections 2023-10-06T03:48:54.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:54.882+0000 7fd9e939d700 1 -- 172.21.15.55:0/3137305679 wait complete. 2023-10-06T03:48:54.901 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T03:48:55.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.092+0000 7f44aeb9d700 1 Processor -- start 2023-10-06T03:48:55.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.092+0000 7f44aeb9d700 1 -- start start 2023-10-06T03:48:55.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.093+0000 7f44aeb9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f44a81407e0 0x7f44a806a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:55.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.093+0000 7f44aeb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a8140bb0 0x7f44a806a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:55.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.093+0000 7f44aeb9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a80c6820 con 0x7f44a806d4f0 2023-10-06T03:48:55.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.093+0000 7f44aeb9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a80c6de0 con 0x7f44a81407e0 2023-10-06T03:48:55.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.093+0000 7f44aeb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a8141270 con 0x7f44a8140bb0 2023-10-06T03:48:55.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.093+0000 7f44adb9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f44a81407e0 0x7f44a806a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:55.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.093+0000 7f44adb9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f44a81407e0 0x7f44a806a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41250/0 (socket says 172.21.15.55:41250) 2023-10-06T03:48:55.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.093+0000 7f44adb9b700 1 -- 172.21.15.55:0/1826788353 learned_addr learned my addr 172.21.15.55:0/1826788353 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:55.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.094+0000 7f44adb9b700 1 -- 172.21.15.55:0/1826788353 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a8140bb0 msgr2=0x7f44a806a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:55.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.094+0000 7f44ad39a700 1 --2- 172.21.15.55:0/1826788353 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a8140bb0 0x7f44a806a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:55.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.094+0000 7f44adb9b700 1 --2- 172.21.15.55:0/1826788353 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a8140bb0 0x7f44a806a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:55.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.094+0000 7f44adb9b700 1 -- 172.21.15.55:0/1826788353 >> v1:172.21.15.163:6789/0 conn(0x7f44a806d4f0 legacy=0x7f44a815b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:55.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.094+0000 7f44adb9b700 1 -- 172.21.15.55:0/1826788353 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a806acf0 con 0x7f44a81407e0 2023-10-06T03:48:55.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.094+0000 7f44ad39a700 1 --2- 172.21.15.55:0/1826788353 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a8140bb0 0x7f44a806a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:55.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.094+0000 7f44adb9b700 1 --2- 172.21.15.55:0/1826788353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a81407e0 0x7f44a806a180 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f44a40099b0 tx=0x7f44a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f07887a53058afc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:55.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.094+0000 7f44acb99700 1 -- 172.21.15.55:0/1826788353 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a400b820 con 0x7f44a81407e0 2023-10-06T03:48:55.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.094+0000 7f44acb99700 1 -- 172.21.15.55:0/1826788353 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a400b980 con 0x7f44a81407e0 2023-10-06T03:48:55.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.094+0000 7f44acb99700 1 -- 172.21.15.55:0/1826788353 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a400bd20 con 0x7f44a81407e0 2023-10-06T03:48:55.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.095+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1826788353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a81407e0 msgr2=0x7f44a806a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:55.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.095+0000 7f44aeb9d700 1 --2- 172.21.15.55:0/1826788353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a81407e0 0x7f44a806a180 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f44a40099b0 tx=0x7f44a4009250 comp rx=0 tx=0).stop 2023-10-06T03:48:55.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.095+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1826788353 shutdown_connections 2023-10-06T03:48:55.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.095+0000 7f44aeb9d700 1 --2- 172.21.15.55:0/1826788353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a81407e0 0x7f44a806a180 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:55.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.095+0000 7f44aeb9d700 1 --2- 172.21.15.55:0/1826788353 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a8140bb0 0x7f44a806a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:55.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.095+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1826788353 >> 172.21.15.55:0/1826788353 conn(0x7f44a80cc1e0 msgr2=0x7f44a80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:55.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.095+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1826788353 shutdown_connections 2023-10-06T03:48:55.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.095+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1826788353 wait complete. 2023-10-06T03:48:55.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.096+0000 7f44aeb9d700 1 Processor -- start 2023-10-06T03:48:55.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.097+0000 7f44aeb9d700 1 -- start start 2023-10-06T03:48:55.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.097+0000 7f44aeb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44a806d4f0 0x7f44a80a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:55.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.097+0000 7f44aeb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a81407e0 0x7f44a8167d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:55.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44aeb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a8140bb0 0x7f44a80b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:55.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44aeb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a8168d90 con 0x7f44a81407e0 2023-10-06T03:48:55.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44aeb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a8157c10 con 0x7f44a806d4f0 2023-10-06T03:48:55.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44aeb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a8169130 con 0x7f44a8140bb0 2023-10-06T03:48:55.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44ad39a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a81407e0 0x7f44a8167d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:55.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44ad39a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a81407e0 0x7f44a8167d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41258/0 (socket says 172.21.15.55:41258) 2023-10-06T03:48:55.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44ad39a700 1 -- 172.21.15.55:0/1158542629 learned_addr learned my addr 172.21.15.55:0/1158542629 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:55.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44adb9b700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44a806d4f0 0x7f44a80a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:55.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44ae39c700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a8140bb0 0x7f44a80b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:55.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44ad39a700 1 -- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a8140bb0 msgr2=0x7f44a80b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:55.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44ad39a700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a8140bb0 0x7f44a80b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:55.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44ad39a700 1 -- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44a806d4f0 msgr2=0x7f44a80a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:55.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44ad39a700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44a806d4f0 0x7f44a80a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:55.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44ad39a700 1 -- 172.21.15.55:0/1158542629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a40094f0 con 0x7f44a81407e0 2023-10-06T03:48:55.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44ae39c700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a8140bb0 0x7f44a80b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:55.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.098+0000 7f44ad39a700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a81407e0 0x7f44a8167d40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f449800c800 tx=0x7f4498008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:55.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.099+0000 7f449dffb700 1 -- 172.21.15.55:0/1158542629 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498011600 con 0x7f44a81407e0 2023-10-06T03:48:55.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.099+0000 7f449dffb700 1 -- 172.21.15.55:0/1158542629 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4498011760 con 0x7f44a81407e0 2023-10-06T03:48:55.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.099+0000 7f449dffb700 1 -- 172.21.15.55:0/1158542629 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4498011ae0 con 0x7f44a81407e0 2023-10-06T03:48:55.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.099+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1158542629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a8168280 con 0x7f44a81407e0 2023-10-06T03:48:55.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.099+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1158542629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a81687d0 con 0x7f44a81407e0 2023-10-06T03:48:55.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.100+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1158542629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a81692a0 con 0x7f44a81407e0 2023-10-06T03:48:55.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.101+0000 7f449dffb700 1 -- 172.21.15.55:0/1158542629 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4498010350 con 0x7f44a81407e0 2023-10-06T03:48:55.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.101+0000 7f449dffb700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f448c066f30 0x7f448c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:55.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.102+0000 7f449dffb700 1 -- 172.21.15.55:0/1158542629 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f44980846a0 con 0x7f44a81407e0 2023-10-06T03:48:55.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.102+0000 7f44adb9b700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f448c066f30 0x7f448c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:55.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.103+0000 7f44adb9b700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f448c066f30 0x7f448c0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f44a40099b0 tx=0x7f44a4016000 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:55.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.106+0000 7f449dffb700 1 -- 172.21.15.55:0/1158542629 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4498055180 con 0x7f44a81407e0 2023-10-06T03:48:55.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.307+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1158542629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f44a806afd0 con 0x7f44a81407e0 2023-10-06T03:48:55.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.311+0000 7f449dffb700 1 -- 172.21.15.55:0/1158542629 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v27) v1 ==== 120+0+0 (secure 0 0 0) 0x7f4498054d10 con 0x7f44a81407e0 2023-10-06T03:48:55.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.314+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f448c066f30 msgr2=0x7f448c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:55.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.314+0000 7f44aeb9d700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f448c066f30 0x7f448c0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f44a40099b0 tx=0x7f44a4016000 comp rx=0 tx=0).stop 2023-10-06T03:48:55.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.314+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1158542629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a81407e0 msgr2=0x7f44a8167d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:55.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.314+0000 7f44aeb9d700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a81407e0 0x7f44a8167d40 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f449800c800 tx=0x7f4498008040 comp rx=0 tx=0).stop 2023-10-06T03:48:55.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.314+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1158542629 shutdown_connections 2023-10-06T03:48:55.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.314+0000 7f44aeb9d700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f448c066f30 0x7f448c0693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:55.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.314+0000 7f44aeb9d700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44a8140bb0 0x7f44a80b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:55.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.314+0000 7f44aeb9d700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44a806d4f0 0x7f44a80a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:55.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.314+0000 7f44aeb9d700 1 --2- 172.21.15.55:0/1158542629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44a81407e0 0x7f44a8167d40 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:55.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.314+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1158542629 >> 172.21.15.55:0/1158542629 conn(0x7f44a80cc1e0 msgr2=0x7f44a80cc5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:55.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.314+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1158542629 shutdown_connections 2023-10-06T03:48:55.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.314+0000 7f44aeb9d700 1 -- 172.21.15.55:0/1158542629 wait complete. 2023-10-06T03:48:55.333 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-06T03:48:55.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.518+0000 7fb9f25ff700 1 Processor -- start 2023-10-06T03:48:55.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.518+0000 7fb9f25ff700 1 -- start start 2023-10-06T03:48:55.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.520+0000 7fb9f25ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9ec0cbf30 0x7fb9ec0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:55.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.520+0000 7fb9f25ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb9ec0b1420 0x7fb9ec0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:55.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.520+0000 7fb9f25ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9ec0cc300 con 0x7fb9ec0ccf90 2023-10-06T03:48:55.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.520+0000 7fb9f25ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9ec156bf0 con 0x7fb9ec0b1420 2023-10-06T03:48:55.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.520+0000 7fb9f25ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0ae950 con 0x7fb9ec0cbf30 2023-10-06T03:48:55.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.520+0000 7fb9eb7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb9ec0ccf90 0x7fb9ec0b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43178/0 (socket says 172.21.15.55:43178) 2023-10-06T03:48:55.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.520+0000 7fb9eb7fe700 1 -- 172.21.15.55:0/3371704654 learned_addr learned my addr 172.21.15.55:0/3371704654 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:55.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.520+0000 7fb9ebfff700 1 --2- 172.21.15.55:0/3371704654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9ec0cbf30 0x7fb9ec0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:55.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.520+0000 7fb9f0b9c700 1 --2- 172.21.15.55:0/3371704654 >> v2:172.21.15.163:3300/0 conn(0x7fb9ec0b1420 0x7fb9ec0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:55.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.520+0000 7fb9eaffd700 1 -- 172.21.15.55:0/3371704654 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 931696152 0 0) 0x7fb9ec0cc300 con 0x7fb9ec0ccf90 2023-10-06T03:48:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.521+0000 7fb9eaffd700 1 -- 172.21.15.55:0/3371704654 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9d4003570 con 0x7fb9ec0ccf90 2023-10-06T03:48:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.521+0000 7fb9f0b9c700 1 -- 172.21.15.55:0/3371704654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9ec0cbf30 msgr2=0x7fb9ec0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.521+0000 7fb9f0b9c700 1 --2- 172.21.15.55:0/3371704654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9ec0cbf30 0x7fb9ec0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.521+0000 7fb9f0b9c700 1 -- 172.21.15.55:0/3371704654 >> v1:172.21.15.55:6789/0 conn(0x7fb9ec0ccf90 legacy=0x7fb9ec0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.521+0000 7fb9f0b9c700 1 -- 172.21.15.55:0/3371704654 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec0ad370 con 0x7fb9ec0b1420 2023-10-06T03:48:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.521+0000 7fb9f0b9c700 1 --2- 172.21.15.55:0/3371704654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9ec0b1420 0x7fb9ec0c9740 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009e90 tx=0x7fb9e4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2683f054a61f60bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.521+0000 7fb9eaffd700 1 -- 172.21.15.55:0/3371704654 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e400e070 con 0x7fb9ec0b1420 2023-10-06T03:48:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.521+0000 7fb9eaffd700 1 -- 172.21.15.55:0/3371704654 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e400bb40 con 0x7fb9ec0b1420 2023-10-06T03:48:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.521+0000 7fb9eaffd700 1 -- 172.21.15.55:0/3371704654 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4012430 con 0x7fb9ec0b1420 2023-10-06T03:48:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.522+0000 7fb9f25ff700 1 -- 172.21.15.55:0/3371704654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9ec0b1420 msgr2=0x7fb9ec0c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:55.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.522+0000 7fb9f25ff700 1 --2- 172.21.15.55:0/3371704654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9ec0b1420 0x7fb9ec0c9740 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009e90 tx=0x7fb9e4009780 comp rx=0 tx=0).stop 2023-10-06T03:48:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.522+0000 7fb9f25ff700 1 -- 172.21.15.55:0/3371704654 shutdown_connections 2023-10-06T03:48:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.522+0000 7fb9f25ff700 1 --2- 172.21.15.55:0/3371704654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9ec0b1420 0x7fb9ec0c9740 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.522+0000 7fb9f25ff700 1 --2- 172.21.15.55:0/3371704654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9ec0cbf30 0x7fb9ec0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.522+0000 7fb9f25ff700 1 -- 172.21.15.55:0/3371704654 >> 172.21.15.55:0/3371704654 conn(0x7fb9ec06a320 msgr2=0x7fb9ec0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.522+0000 7fb9f25ff700 1 -- 172.21.15.55:0/3371704654 shutdown_connections 2023-10-06T03:48:55.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.522+0000 7fb9f25ff700 1 -- 172.21.15.55:0/3371704654 wait complete. 2023-10-06T03:48:55.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.523+0000 7fb9f25ff700 1 Processor -- start 2023-10-06T03:48:55.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.524+0000 7fb9f25ff700 1 -- start start 2023-10-06T03:48:55.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.524+0000 7fb9f25ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9ec0b1420 0x7fb9ec0c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:55.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.524+0000 7fb9f25ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9ec0cbf30 0x7fb9ec16ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:55.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.524+0000 7fb9f25ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9ec0ccf90 0x7fb9ec16d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:55.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.524+0000 7fb9f25ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0c7180 con 0x7fb9ec0cbf30 2023-10-06T03:48:55.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.524+0000 7fb9f25ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0c7460 con 0x7fb9ec0ccf90 2023-10-06T03:48:55.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.524+0000 7fb9f25ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9ec0c7740 con 0x7fb9ec0b1420 2023-10-06T03:48:55.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.525+0000 7fb9eb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9ec0cbf30 0x7fb9ec16ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:55.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.525+0000 7fb9eb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9ec0cbf30 0x7fb9ec16ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41264/0 (socket says 172.21.15.55:41264) 2023-10-06T03:48:55.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.525+0000 7fb9eb7fe700 1 -- 172.21.15.55:0/2454414421 learned_addr learned my addr 172.21.15.55:0/2454414421 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:55.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.525+0000 7fb9ebfff700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9ec0b1420 0x7fb9ec0c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:55.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.525+0000 7fb9f0b9c700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9ec0ccf90 0x7fb9ec16d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:55.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.525+0000 7fb9eb7fe700 1 -- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9ec0b1420 msgr2=0x7fb9ec0c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:55.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.525+0000 7fb9eb7fe700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9ec0b1420 0x7fb9ec0c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:55.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.525+0000 7fb9eb7fe700 1 -- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9ec0ccf90 msgr2=0x7fb9ec16d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:55.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.525+0000 7fb9eb7fe700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9ec0ccf90 0x7fb9ec16d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:55.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.525+0000 7fb9eb7fe700 1 -- 172.21.15.55:0/2454414421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e4009a20 con 0x7fb9ec0cbf30 2023-10-06T03:48:55.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.525+0000 7fb9ebfff700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9ec0b1420 0x7fb9ec0c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:55.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.525+0000 7fb9eb7fe700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9ec0cbf30 0x7fb9ec16ce10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000c590 tx=0x7fb9e0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:55.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.526+0000 7fb9d3fff700 1 -- 172.21.15.55:0/2454414421 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0005b30 con 0x7fb9ec0cbf30 2023-10-06T03:48:55.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.526+0000 7fb9d3fff700 1 -- 172.21.15.55:0/2454414421 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e0005cb0 con 0x7fb9ec0cbf30 2023-10-06T03:48:55.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.526+0000 7fb9d3fff700 1 -- 172.21.15.55:0/2454414421 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0007320 con 0x7fb9ec0cbf30 2023-10-06T03:48:55.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.526+0000 7fb9f25ff700 1 -- 172.21.15.55:0/2454414421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec0c7a20 con 0x7fb9ec0cbf30 2023-10-06T03:48:55.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.526+0000 7fb9f25ff700 1 -- 172.21.15.55:0/2454414421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ec0c6e70 con 0x7fb9ec0cbf30 2023-10-06T03:48:55.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.527+0000 7fb9f25ff700 1 -- 172.21.15.55:0/2454414421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ec0c7f40 con 0x7fb9ec0cbf30 2023-10-06T03:48:55.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.528+0000 7fb9d3fff700 1 -- 172.21.15.55:0/2454414421 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb9e001b3b0 con 0x7fb9ec0cbf30 2023-10-06T03:48:55.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.529+0000 7fb9d3fff700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb9d8066f30 0x7fb9d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:55.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.529+0000 7fb9d3fff700 1 -- 172.21.15.55:0/2454414421 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6243+0+0 (secure 0 0 0) 0x7fb9e0083d10 con 0x7fb9ec0cbf30 2023-10-06T03:48:55.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.529+0000 7fb9ebfff700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb9d8066f30 0x7fb9d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:55.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.530+0000 7fb9ebfff700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb9d8066f30 0x7fb9d80693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009d80 tx=0x7fb9dc008040 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:55.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.534+0000 7fb9d3fff700 1 -- 172.21.15.55:0/2454414421 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9ec0c7f40 con 0x7fb9ec0cbf30 2023-10-06T03:48:55.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:55.813+0000 7fb9f25ff700 1 -- 172.21.15.55:0/2454414421 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fb9ec0c7f40 con 0x7fb9ec0cbf30 2023-10-06T03:48:56.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.008+0000 7fb9d3fff700 1 -- 172.21.15.55:0/2454414421 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v573) v1 ==== 156+0+0 (secure 0 0 0) 0x7fb9ec0c7f40 con 0x7fb9ec0cbf30 2023-10-06T03:48:56.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.010+0000 7fb9f25ff700 1 -- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb9d8066f30 msgr2=0x7fb9d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:56.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.010+0000 7fb9f25ff700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb9d8066f30 0x7fb9d80693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009d80 tx=0x7fb9dc008040 comp rx=0 tx=0).stop 2023-10-06T03:48:56.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.010+0000 7fb9f25ff700 1 -- 172.21.15.55:0/2454414421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9ec0cbf30 msgr2=0x7fb9ec16ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.010+0000 7fb9f25ff700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9ec0cbf30 0x7fb9ec16ce10 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000c590 tx=0x7fb9e0007e00 comp rx=0 tx=0).stop 2023-10-06T03:48:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.010+0000 7fb9f25ff700 1 -- 172.21.15.55:0/2454414421 shutdown_connections 2023-10-06T03:48:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.010+0000 7fb9f25ff700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb9d8066f30 0x7fb9d80693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.010+0000 7fb9f25ff700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9ec0ccf90 0x7fb9ec16d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.010+0000 7fb9f25ff700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9ec0b1420 0x7fb9ec0c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.011+0000 7fb9f25ff700 1 --2- 172.21.15.55:0/2454414421 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9ec0cbf30 0x7fb9ec16ce10 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.011+0000 7fb9f25ff700 1 -- 172.21.15.55:0/2454414421 >> 172.21.15.55:0/2454414421 conn(0x7fb9ec06a320 msgr2=0x7fb9ec158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.011+0000 7fb9f25ff700 1 -- 172.21.15.55:0/2454414421 shutdown_connections 2023-10-06T03:48:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.011+0000 7fb9f25ff700 1 -- 172.21.15.55:0/2454414421 wait complete. 2023-10-06T03:48:56.031 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-10-06T03:48:56.031 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-10-06T03:48:56.031 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T03:48:56.031 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-06T03:48:56.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.218+0000 7fb37579d700 1 Processor -- start 2023-10-06T03:48:56.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.218+0000 7fb37579d700 1 -- start start 2023-10-06T03:48:56.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.219+0000 7fb37579d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb370159420 0x7fb3700b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:56.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.219+0000 7fb37579d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb37015d230 0x7fb37015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:56.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.219+0000 7fb37579d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3700b2980 con 0x7fb3701597f0 2023-10-06T03:48:56.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.219+0000 7fb37579d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb37015db70 con 0x7fb370159420 2023-10-06T03:48:56.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.219+0000 7fb37579d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3700b11d0 con 0x7fb37015d230 2023-10-06T03:48:56.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.219+0000 7fb36ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb370159420 0x7fb3700b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:56.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.220+0000 7fb36ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb370159420 0x7fb3700b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41266/0 (socket says 172.21.15.55:41266) 2023-10-06T03:48:56.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.220+0000 7fb36ffff700 1 -- 172.21.15.55:0/2183827212 learned_addr learned my addr 172.21.15.55:0/2183827212 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:56.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.220+0000 7fb374f9c700 1 --2- 172.21.15.55:0/2183827212 >> v2:172.21.15.163:3300/0 conn(0x7fb37015d230 0x7fb37015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:56.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.220+0000 7fb36ffff700 1 -- 172.21.15.55:0/2183827212 >> v2:172.21.15.163:3300/0 conn(0x7fb37015d230 msgr2=0x7fb37015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:56.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.220+0000 7fb36ffff700 1 --2- 172.21.15.55:0/2183827212 >> v2:172.21.15.163:3300/0 conn(0x7fb37015d230 0x7fb37015d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:56.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.220+0000 7fb36ffff700 1 -- 172.21.15.55:0/2183827212 >> v1:172.21.15.163:6789/0 conn(0x7fb3701597f0 legacy=0x7fb3700b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:56.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.220+0000 7fb36ffff700 1 -- 172.21.15.55:0/2183827212 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3700b14b0 con 0x7fb370159420 2023-10-06T03:48:56.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.220+0000 7fb374f9c700 1 --2- 172.21.15.55:0/2183827212 >> v2:172.21.15.163:3300/0 conn(0x7fb37015d230 0x7fb37015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:56.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.220+0000 7fb36ffff700 1 --2- 172.21.15.55:0/2183827212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb370159420 0x7fb3700b5f40 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb37015d170 tx=0x7fb360009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c83114f4fbf0e61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:56.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.220+0000 7fb36effd700 1 -- 172.21.15.55:0/2183827212 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36000b820 con 0x7fb370159420 2023-10-06T03:48:56.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.221+0000 7fb36effd700 1 -- 172.21.15.55:0/2183827212 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36000b980 con 0x7fb370159420 2023-10-06T03:48:56.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.221+0000 7fb36effd700 1 -- 172.21.15.55:0/2183827212 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36000bd20 con 0x7fb370159420 2023-10-06T03:48:56.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.221+0000 7fb37579d700 1 -- 172.21.15.55:0/2183827212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb370159420 msgr2=0x7fb3700b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:56.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.221+0000 7fb37579d700 1 --2- 172.21.15.55:0/2183827212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb370159420 0x7fb3700b5f40 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb37015d170 tx=0x7fb360009250 comp rx=0 tx=0).stop 2023-10-06T03:48:56.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.221+0000 7fb37579d700 1 -- 172.21.15.55:0/2183827212 shutdown_connections 2023-10-06T03:48:56.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.221+0000 7fb37579d700 1 --2- 172.21.15.55:0/2183827212 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb370159420 0x7fb3700b5f40 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:56.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.221+0000 7fb37579d700 1 --2- 172.21.15.55:0/2183827212 >> v2:172.21.15.163:3300/0 conn(0x7fb37015d230 0x7fb37015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:56.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.221+0000 7fb37579d700 1 -- 172.21.15.55:0/2183827212 >> 172.21.15.55:0/2183827212 conn(0x7fb3700d0660 msgr2=0x7fb3700d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:56.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.221+0000 7fb37579d700 1 -- 172.21.15.55:0/2183827212 shutdown_connections 2023-10-06T03:48:56.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.221+0000 7fb37579d700 1 -- 172.21.15.55:0/2183827212 wait complete. 2023-10-06T03:48:56.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.223+0000 7fb37579d700 1 Processor -- start 2023-10-06T03:48:56.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.223+0000 7fb37579d700 1 -- start start 2023-10-06T03:48:56.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.224+0000 7fb37579d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb370159420 0x7fb3700cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:56.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.224+0000 7fb37579d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3701597f0 0x7fb37015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:56.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.224+0000 7fb37579d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb37015d230 0x7fb370169d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:56.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.224+0000 7fb37579d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37015b1b0 con 0x7fb370159420 2023-10-06T03:48:56.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.224+0000 7fb37579d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37015b450 con 0x7fb3701597f0 2023-10-06T03:48:56.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.224+0000 7fb37579d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37015b730 con 0x7fb37015d230 2023-10-06T03:48:56.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.224+0000 7fb36ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb370159420 0x7fb3700cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:56.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.224+0000 7fb36ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb370159420 0x7fb3700cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41270/0 (socket says 172.21.15.55:41270) 2023-10-06T03:48:56.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.224+0000 7fb36ffff700 1 -- 172.21.15.55:0/2133286514 learned_addr learned my addr 172.21.15.55:0/2133286514 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:56.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.224+0000 7fb374f9c700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb37015d230 0x7fb370169d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:56.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.224+0000 7fb36f7fe700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3701597f0 0x7fb37015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:56.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.225+0000 7fb36ffff700 1 -- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb37015d230 msgr2=0x7fb370169d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:56.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.225+0000 7fb36ffff700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb37015d230 0x7fb370169d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:56.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.225+0000 7fb36ffff700 1 -- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3701597f0 msgr2=0x7fb37015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:56.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.225+0000 7fb36ffff700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3701597f0 0x7fb37015ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:56.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.225+0000 7fb36ffff700 1 -- 172.21.15.55:0/2133286514 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3600094f0 con 0x7fb370159420 2023-10-06T03:48:56.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.225+0000 7fb374f9c700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb37015d230 0x7fb370169d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:56.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.225+0000 7fb36ffff700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb370159420 0x7fb3700cba40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb360009290 tx=0x7fb360016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:56.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.225+0000 7fb357fff700 1 -- 172.21.15.55:0/2133286514 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360011e20 con 0x7fb370159420 2023-10-06T03:48:56.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.225+0000 7fb357fff700 1 -- 172.21.15.55:0/2133286514 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb360020070 con 0x7fb370159420 2023-10-06T03:48:56.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.225+0000 7fb37579d700 1 -- 172.21.15.55:0/2133286514 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb37015ba10 con 0x7fb370159420 2023-10-06T03:48:56.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.225+0000 7fb357fff700 1 -- 172.21.15.55:0/2133286514 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36001b6a0 con 0x7fb370159420 2023-10-06T03:48:56.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.225+0000 7fb37579d700 1 -- 172.21.15.55:0/2133286514 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37015bfd0 con 0x7fb370159420 2023-10-06T03:48:56.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.226+0000 7fb37579d700 1 -- 172.21.15.55:0/2133286514 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb37004c3c0 con 0x7fb370159420 2023-10-06T03:48:56.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.227+0000 7fb357fff700 1 -- 172.21.15.55:0/2133286514 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb360022030 con 0x7fb370159420 2023-10-06T03:48:56.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.228+0000 7fb357fff700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb35c066f30 0x7fb35c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:56.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.228+0000 7fb357fff700 1 -- 172.21.15.55:0/2133286514 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6243+0+0 (secure 0 0 0) 0x7fb3600867d0 con 0x7fb370159420 2023-10-06T03:48:56.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.228+0000 7fb36f7fe700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb35c066f30 0x7fb35c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:56.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.229+0000 7fb36f7fe700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb35c066f30 0x7fb35c0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb37016ae80 tx=0x7fb364009040 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:56.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.232+0000 7fb357fff700 1 -- 172.21.15.55:0/2133286514 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb360053830 con 0x7fb370159420 2023-10-06T03:48:56.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:56.515+0000 7fb37579d700 1 -- 172.21.15.55:0/2133286514 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7fb37004c3c0 con 0x7fb370159420 2023-10-06T03:48:57.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.015+0000 7fb357fff700 1 -- 172.21.15.55:0/2133286514 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v574) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb360056e50 con 0x7fb370159420 2023-10-06T03:48:57.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.018+0000 7fb37579d700 1 -- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb35c066f30 msgr2=0x7fb35c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:57.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.018+0000 7fb37579d700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb35c066f30 0x7fb35c0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb37016ae80 tx=0x7fb364009040 comp rx=0 tx=0).stop 2023-10-06T03:48:57.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.018+0000 7fb37579d700 1 -- 172.21.15.55:0/2133286514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb370159420 msgr2=0x7fb3700cba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:57.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.018+0000 7fb37579d700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb370159420 0x7fb3700cba40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb360009290 tx=0x7fb360016000 comp rx=0 tx=0).stop 2023-10-06T03:48:57.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.018+0000 7fb37579d700 1 -- 172.21.15.55:0/2133286514 shutdown_connections 2023-10-06T03:48:57.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.018+0000 7fb37579d700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb35c066f30 0x7fb35c0693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.018+0000 7fb37579d700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb37015d230 0x7fb370169d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.018+0000 7fb37579d700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb370159420 0x7fb3700cba40 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.018+0000 7fb37579d700 1 --2- 172.21.15.55:0/2133286514 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3701597f0 0x7fb37015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.018+0000 7fb37579d700 1 -- 172.21.15.55:0/2133286514 >> 172.21.15.55:0/2133286514 conn(0x7fb3700d0660 msgr2=0x7fb3700ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:57.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.018+0000 7fb37579d700 1 -- 172.21.15.55:0/2133286514 shutdown_connections 2023-10-06T03:48:57.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.018+0000 7fb37579d700 1 -- 172.21.15.55:0/2133286514 wait complete. 2023-10-06T03:48:57.039 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:48:57.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.229+0000 7f6c4359e700 1 Processor -- start 2023-10-06T03:48:57.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.230+0000 7f6c4359e700 1 -- start start 2023-10-06T03:48:57.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.231+0000 7f6c4359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6c440c9db0 0x7f6c4406a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:57.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.231+0000 7f6c4359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6c44157130 0x7f6c4406a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.231+0000 7f6c4359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c4406e150 con 0x7f6c44153900 2023-10-06T03:48:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.231+0000 7f6c4359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c440c7610 con 0x7f6c440c9db0 2023-10-06T03:48:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.231+0000 7f6c4359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c440c7800 con 0x7f6c44157130 2023-10-06T03:48:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.231+0000 7f6c41d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6c440c9db0 0x7f6c4406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.231+0000 7f6c41d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6c440c9db0 0x7f6c4406a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41282/0 (socket says 172.21.15.55:41282) 2023-10-06T03:48:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.231+0000 7f6c41d9b700 1 -- 172.21.15.55:0/2496689312 learned_addr learned my addr 172.21.15.55:0/2496689312 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.231+0000 7f6c4259c700 1 --1- 172.21.15.55:0/2496689312 >> v1:172.21.15.55:6789/0 conn(0x7f6c44153900 0x7f6c440c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43186/0 (socket says 172.21.15.55:43186) 2023-10-06T03:48:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.231+0000 7f6c42d9d700 1 --2- 172.21.15.55:0/2496689312 >> v2:172.21.15.163:3300/0 conn(0x7f6c44157130 0x7f6c4406a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.232+0000 7f6c41d9b700 1 -- 172.21.15.55:0/2496689312 >> v2:172.21.15.163:3300/0 conn(0x7f6c44157130 msgr2=0x7f6c4406a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:57.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.232+0000 7f6c41d9b700 1 --2- 172.21.15.55:0/2496689312 >> v2:172.21.15.163:3300/0 conn(0x7f6c44157130 0x7f6c4406a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.232+0000 7f6c41d9b700 1 -- 172.21.15.55:0/2496689312 >> v1:172.21.15.55:6789/0 conn(0x7f6c44153900 legacy=0x7f6c440c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:57.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.232+0000 7f6c41d9b700 1 -- 172.21.15.55:0/2496689312 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c440c7ae0 con 0x7f6c440c9db0 2023-10-06T03:48:57.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.232+0000 7f6c4159a700 1 -- 172.21.15.55:0/2496689312 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174708532 0 0) 0x7f6c4406e150 con 0x7f6c44153900 2023-10-06T03:48:57.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.232+0000 7f6c41d9b700 1 --2- 172.21.15.55:0/2496689312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c440c9db0 0x7f6c4406a000 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6c440cd2d0 tx=0x7f6c38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d17a31041485cd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:57.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.232+0000 7f6c4159a700 1 -- 172.21.15.55:0/2496689312 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800b7e0 con 0x7f6c440c9db0 2023-10-06T03:48:57.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.232+0000 7f6c4159a700 1 -- 172.21.15.55:0/2496689312 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3800b940 con 0x7f6c440c9db0 2023-10-06T03:48:57.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.232+0000 7f6c4159a700 1 -- 172.21.15.55:0/2496689312 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800bd00 con 0x7f6c440c9db0 2023-10-06T03:48:57.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.233+0000 7f6c4359e700 1 -- 172.21.15.55:0/2496689312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c440c9db0 msgr2=0x7f6c4406a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:57.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.233+0000 7f6c4359e700 1 --2- 172.21.15.55:0/2496689312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c440c9db0 0x7f6c4406a000 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6c440cd2d0 tx=0x7f6c38009250 comp rx=0 tx=0).stop 2023-10-06T03:48:57.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.233+0000 7f6c4359e700 1 -- 172.21.15.55:0/2496689312 shutdown_connections 2023-10-06T03:48:57.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.233+0000 7f6c4359e700 1 --2- 172.21.15.55:0/2496689312 >> v2:172.21.15.163:3300/0 conn(0x7f6c44157130 0x7f6c4406a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.233+0000 7f6c4359e700 1 --2- 172.21.15.55:0/2496689312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c440c9db0 0x7f6c4406a000 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.233+0000 7f6c4359e700 1 -- 172.21.15.55:0/2496689312 >> 172.21.15.55:0/2496689312 conn(0x7f6c440cc1e0 msgr2=0x7f6c440cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:57.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.233+0000 7f6c4359e700 1 -- 172.21.15.55:0/2496689312 shutdown_connections 2023-10-06T03:48:57.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.233+0000 7f6c4359e700 1 -- 172.21.15.55:0/2496689312 wait complete. 2023-10-06T03:48:57.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.234+0000 7f6c4359e700 1 Processor -- start 2023-10-06T03:48:57.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.234+0000 7f6c4359e700 1 -- start start 2023-10-06T03:48:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.235+0000 7f6c4359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c440c9db0 0x7f6c4414be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.235+0000 7f6c4359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c44153900 0x7f6c441585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.235+0000 7f6c4359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c44157130 0x7f6c44158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.235+0000 7f6c4359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c440abd10 con 0x7f6c44153900 2023-10-06T03:48:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.235+0000 7f6c4359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c44151280 con 0x7f6c44157130 2023-10-06T03:48:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.235+0000 7f6c4359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c44151560 con 0x7f6c440c9db0 2023-10-06T03:48:57.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.236+0000 7f6c41d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c44153900 0x7f6c441585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:57.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.236+0000 7f6c41d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c44153900 0x7f6c441585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41284/0 (socket says 172.21.15.55:41284) 2023-10-06T03:48:57.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.236+0000 7f6c41d9b700 1 -- 172.21.15.55:0/3609009502 learned_addr learned my addr 172.21.15.55:0/3609009502 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:57.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.236+0000 7f6c41d9b700 1 -- 172.21.15.55:0/3609009502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c440c9db0 msgr2=0x7f6c4414be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:57.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.236+0000 7f6c41d9b700 1 --2- 172.21.15.55:0/3609009502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c440c9db0 0x7f6c4414be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.236+0000 7f6c41d9b700 1 -- 172.21.15.55:0/3609009502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c44157130 msgr2=0x7f6c44158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:57.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.236+0000 7f6c42d9d700 1 --2- 172.21.15.55:0/3609009502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c44157130 0x7f6c44158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:57.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.236+0000 7f6c41d9b700 1 --2- 172.21.15.55:0/3609009502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c44157130 0x7f6c44158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.236+0000 7f6c41d9b700 1 -- 172.21.15.55:0/3609009502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c380094f0 con 0x7f6c44153900 2023-10-06T03:48:57.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.236+0000 7f6c42d9d700 1 --2- 172.21.15.55:0/3609009502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c44157130 0x7f6c44158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:57.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.236+0000 7f6c41d9b700 1 --2- 172.21.15.55:0/3609009502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c44153900 0x7f6c441585d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6c380090b0 tx=0x7f6c38018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:57.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.237+0000 7f6c327fc700 1 -- 172.21.15.55:0/3609009502 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38010ea0 con 0x7f6c44153900 2023-10-06T03:48:57.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.237+0000 7f6c327fc700 1 -- 172.21.15.55:0/3609009502 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c38018420 con 0x7f6c44153900 2023-10-06T03:48:57.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.237+0000 7f6c4359e700 1 -- 172.21.15.55:0/3609009502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c44151840 con 0x7f6c44153900 2023-10-06T03:48:57.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.237+0000 7f6c327fc700 1 -- 172.21.15.55:0/3609009502 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c380187e0 con 0x7f6c44153900 2023-10-06T03:48:57.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.237+0000 7f6c4359e700 1 -- 172.21.15.55:0/3609009502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c441507d0 con 0x7f6c44153900 2023-10-06T03:48:57.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.238+0000 7f6c4359e700 1 -- 172.21.15.55:0/3609009502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c440abae0 con 0x7f6c44153900 2023-10-06T03:48:57.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.239+0000 7f6c327fc700 1 -- 172.21.15.55:0/3609009502 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c38018c10 con 0x7f6c44153900 2023-10-06T03:48:57.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.240+0000 7f6c327fc700 1 --2- 172.21.15.55:0/3609009502 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6c24066f30 0x7f6c240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:57.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.240+0000 7f6c327fc700 1 -- 172.21.15.55:0/3609009502 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f6c38014070 con 0x7f6c44153900 2023-10-06T03:48:57.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.240+0000 7f6c4259c700 1 --2- 172.21.15.55:0/3609009502 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6c24066f30 0x7f6c240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:57.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.241+0000 7f6c4259c700 1 --2- 172.21.15.55:0/3609009502 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6c24066f30 0x7f6c240693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6c340011e0 tx=0x7f6c340079b0 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:57.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.244+0000 7f6c327fc700 1 -- 172.21.15.55:0/3609009502 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c38052e40 con 0x7f6c44153900 2023-10-06T03:48:57.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.447+0000 7f6c4359e700 1 -- 172.21.15.55:0/3609009502 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6c440abe50 con 0x7f6c44153900 2023-10-06T03:48:57.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.452+0000 7f6c327fc700 1 -- 172.21.15.55:0/3609009502 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6c440abe50 con 0x7f6c44153900 2023-10-06T03:48:57.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.454+0000 7f6c4359e700 1 -- 172.21.15.55:0/3609009502 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6c24066f30 msgr2=0x7f6c240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:57.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.454+0000 7f6c4359e700 1 --2- 172.21.15.55:0/3609009502 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6c24066f30 0x7f6c240693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6c340011e0 tx=0x7f6c340079b0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.454+0000 7f6c4359e700 1 -- 172.21.15.55:0/3609009502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c44153900 msgr2=0x7f6c441585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:57.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.454+0000 7f6c4359e700 1 --2- 172.21.15.55:0/3609009502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c44153900 0x7f6c441585d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6c380090b0 tx=0x7f6c38018000 comp rx=0 tx=0).stop 2023-10-06T03:48:57.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.454+0000 7f6c4359e700 1 -- 172.21.15.55:0/3609009502 shutdown_connections 2023-10-06T03:48:57.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.454+0000 7f6c4359e700 1 --2- 172.21.15.55:0/3609009502 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6c24066f30 0x7f6c240693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.454+0000 7f6c4359e700 1 --2- 172.21.15.55:0/3609009502 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c44157130 0x7f6c44158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.454+0000 7f6c4359e700 1 --2- 172.21.15.55:0/3609009502 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c440c9db0 0x7f6c4414be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.454+0000 7f6c4359e700 1 --2- 172.21.15.55:0/3609009502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c44153900 0x7f6c441585d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.454+0000 7f6c4359e700 1 -- 172.21.15.55:0/3609009502 >> 172.21.15.55:0/3609009502 conn(0x7f6c440cc1e0 msgr2=0x7f6c4406e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:57.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.454+0000 7f6c4359e700 1 -- 172.21.15.55:0/3609009502 shutdown_connections 2023-10-06T03:48:57.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.454+0000 7f6c4359e700 1 -- 172.21.15.55:0/3609009502 wait complete. 2023-10-06T03:48:57.473 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:48:57.474 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:48:57.556 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:48:57.643 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:48:57.643 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:48:57.643 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:48:57.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.829+0000 7f3b0d39d700 1 Processor -- start 2023-10-06T03:48:57.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.829+0000 7f3b0d39d700 1 -- start start 2023-10-06T03:48:57.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.830+0000 7f3b0d39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b081595a0 0x7f3b080b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:57.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.830+0000 7f3b0d39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3b080c8830 0x7f3b08159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:57.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.830+0000 7f3b0d39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b080b1410 con 0x7f3b080c6c60 2023-10-06T03:48:57.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.830+0000 7f3b0d39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b080b2b90 con 0x7f3b080c8830 2023-10-06T03:48:57.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.830+0000 7f3b0d39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b08063230 con 0x7f3b081595a0 2023-10-06T03:48:57.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.831+0000 7f3b07fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b081595a0 0x7f3b080b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:57.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.831+0000 7f3b0cb9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3b080c8830 0x7f3b08159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:57.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.831+0000 7f3b07fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b081595a0 0x7f3b080b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:51700/0 (socket says 172.21.15.55:51700) 2023-10-06T03:48:57.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.831+0000 7f3b07fff700 1 -- 172.21.15.55:0/1036428087 learned_addr learned my addr 172.21.15.55:0/1036428087 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.831+0000 7f3b07fff700 1 -- 172.21.15.55:0/1036428087 >> v2:172.21.15.163:3300/0 conn(0x7f3b080c8830 msgr2=0x7f3b08159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.831+0000 7f3b07fff700 1 --2- 172.21.15.55:0/1036428087 >> v2:172.21.15.163:3300/0 conn(0x7f3b080c8830 0x7f3b08159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.831+0000 7f3b07fff700 1 -- 172.21.15.55:0/1036428087 >> v1:172.21.15.163:6789/0 conn(0x7f3b080c6c60 legacy=0x7f3b080c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.831+0000 7f3b07fff700 1 -- 172.21.15.55:0/1036428087 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b080634f0 con 0x7f3b081595a0 2023-10-06T03:48:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.831+0000 7f3b06ffd700 1 -- 172.21.15.55:0/1036428087 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 674306272 0 0) 0x7f3b080b1410 con 0x7f3b080c6c60 2023-10-06T03:48:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.832+0000 7f3b07fff700 1 --2- 172.21.15.55:0/1036428087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b081595a0 0x7f3b080b18a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3af80099b0 tx=0x7f3af8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdcfb66ac8669de2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.832+0000 7f3b06ffd700 1 -- 172.21.15.55:0/1036428087 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8009ea0 con 0x7f3b081595a0 2023-10-06T03:48:57.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.832+0000 7f3b06ffd700 1 -- 172.21.15.55:0/1036428087 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af800b860 con 0x7f3b081595a0 2023-10-06T03:48:57.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.832+0000 7f3b06ffd700 1 -- 172.21.15.55:0/1036428087 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af800bc00 con 0x7f3b081595a0 2023-10-06T03:48:57.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.832+0000 7f3b0d39d700 1 -- 172.21.15.55:0/1036428087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b081595a0 msgr2=0x7f3b080b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:57.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.832+0000 7f3b0d39d700 1 --2- 172.21.15.55:0/1036428087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b081595a0 0x7f3b080b18a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3af80099b0 tx=0x7f3af8009250 comp rx=0 tx=0).stop 2023-10-06T03:48:57.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.833+0000 7f3b0d39d700 1 -- 172.21.15.55:0/1036428087 shutdown_connections 2023-10-06T03:48:57.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.833+0000 7f3b0d39d700 1 --2- 172.21.15.55:0/1036428087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b081595a0 0x7f3b080b18a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.833+0000 7f3b0d39d700 1 --2- 172.21.15.55:0/1036428087 >> v2:172.21.15.163:3300/0 conn(0x7f3b080c8830 0x7f3b08159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.833+0000 7f3b0d39d700 1 -- 172.21.15.55:0/1036428087 >> 172.21.15.55:0/1036428087 conn(0x7f3b0806ddb0 msgr2=0x7f3b0806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:57.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.833+0000 7f3b0d39d700 1 -- 172.21.15.55:0/1036428087 shutdown_connections 2023-10-06T03:48:57.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.833+0000 7f3b0d39d700 1 -- 172.21.15.55:0/1036428087 wait complete. 2023-10-06T03:48:57.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.834+0000 7f3b0d39d700 1 Processor -- start 2023-10-06T03:48:57.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.834+0000 7f3b0d39d700 1 -- start start 2023-10-06T03:48:57.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.835+0000 7f3b0d39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b080c6c60 0x7f3b0816d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:57.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.835+0000 7f3b0d39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b080c8830 0x7f3b08145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:57.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.835+0000 7f3b0d39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b081595a0 0x7f3b0816a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:57.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.835+0000 7f3b0d39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b08169e70 con 0x7f3b080c6c60 2023-10-06T03:48:57.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.835+0000 7f3b0d39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0816d8e0 con 0x7f3b081595a0 2023-10-06T03:48:57.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.835+0000 7f3b0d39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0816dbc0 con 0x7f3b080c8830 2023-10-06T03:48:57.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.835+0000 7f3b07fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b080c6c60 0x7f3b0816d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:57.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.835+0000 7f3b07fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b080c6c60 0x7f3b0816d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41290/0 (socket says 172.21.15.55:41290) 2023-10-06T03:48:57.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.835+0000 7f3b07fff700 1 -- 172.21.15.55:0/4064036013 learned_addr learned my addr 172.21.15.55:0/4064036013 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:57.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.836+0000 7f3b0cb9c700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b081595a0 0x7f3b0816a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:57.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.836+0000 7f3b077fe700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b080c8830 0x7f3b08145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:57.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.836+0000 7f3b07fff700 1 -- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b080c8830 msgr2=0x7f3b08145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:57.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.836+0000 7f3b07fff700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b080c8830 0x7f3b08145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.836+0000 7f3b07fff700 1 -- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b081595a0 msgr2=0x7f3b0816a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:57.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.836+0000 7f3b07fff700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b081595a0 0x7f3b0816a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:57.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.836+0000 7f3b07fff700 1 -- 172.21.15.55:0/4064036013 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af80094f0 con 0x7f3b080c6c60 2023-10-06T03:48:57.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.836+0000 7f3b077fe700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b080c8830 0x7f3b08145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:57.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.836+0000 7f3b0cb9c700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b081595a0 0x7f3b0816a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:57.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.836+0000 7f3b07fff700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b080c6c60 0x7f3b0816d090 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f3af8006fd0 tx=0x7f3af8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:57.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.836+0000 7f3aeffff700 1 -- 172.21.15.55:0/4064036013 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af800bbe0 con 0x7f3b080c6c60 2023-10-06T03:48:57.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.836+0000 7f3aeffff700 1 -- 172.21.15.55:0/4064036013 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af801a430 con 0x7f3b080c6c60 2023-10-06T03:48:57.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.836+0000 7f3aeffff700 1 -- 172.21.15.55:0/4064036013 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af801a7d0 con 0x7f3b080c6c60 2023-10-06T03:48:57.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.836+0000 7f3b0d39d700 1 -- 172.21.15.55:0/4064036013 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0816dea0 con 0x7f3b080c6c60 2023-10-06T03:48:57.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.837+0000 7f3b0d39d700 1 -- 172.21.15.55:0/4064036013 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b08169a90 con 0x7f3b080c6c60 2023-10-06T03:48:57.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.838+0000 7f3b0d39d700 1 -- 172.21.15.55:0/4064036013 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0804c3c0 con 0x7f3b080c6c60 2023-10-06T03:48:57.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.838+0000 7f3aeffff700 1 -- 172.21.15.55:0/4064036013 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3af801ab40 con 0x7f3b080c6c60 2023-10-06T03:48:57.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.839+0000 7f3aeffff700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3af4066f30 0x7f3af40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:57.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.839+0000 7f3aeffff700 1 -- 172.21.15.55:0/4064036013 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f3af8086400 con 0x7f3b080c6c60 2023-10-06T03:48:57.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.840+0000 7f3b077fe700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3af4066f30 0x7f3af40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:57.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.840+0000 7f3b077fe700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3af4066f30 0x7f3af40693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0011e0 tx=0x7f3afc009040 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:57.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:57.844+0000 7f3aeffff700 1 -- 172.21.15.55:0/4064036013 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3af8053450 con 0x7f3b080c6c60 2023-10-06T03:48:58.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.089+0000 7f3b0d39d700 1 -- 172.21.15.55:0/4064036013 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b080637d0 con 0x7f3b080c6c60 2023-10-06T03:48:58.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.090+0000 7f3aeffff700 1 -- 172.21.15.55:0/4064036013 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f3af8015de0 con 0x7f3b080c6c60 2023-10-06T03:48:58.092 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:58.092 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/4046797285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":4046797285},{"type":"v1","addr":"172.21.15.55:6835","nonce":4046797285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5645,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/675796170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":675796170},{"type":"v1","addr":"172.21.15.163:6836","nonce":675796170}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5648,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2543497851","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2543497851},{"type":"v1","addr":"172.21.15.163:6834","nonce":2543497851}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":6693,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1276172285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1276172285},{"type":"v1","addr":"172.21.15.55:6834","nonce":1276172285}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[]} 2023-10-06T03:48:58.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.092+0000 7f3b0d39d700 1 -- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3af4066f30 msgr2=0x7f3af40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:58.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.092+0000 7f3b0d39d700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3af4066f30 0x7f3af40693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3afc0011e0 tx=0x7f3afc009040 comp rx=0 tx=0).stop 2023-10-06T03:48:58.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.092+0000 7f3b0d39d700 1 -- 172.21.15.55:0/4064036013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b080c6c60 msgr2=0x7f3b0816d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:58.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.092+0000 7f3b0d39d700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b080c6c60 0x7f3b0816d090 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f3af8006fd0 tx=0x7f3af8017000 comp rx=0 tx=0).stop 2023-10-06T03:48:58.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.092+0000 7f3b0d39d700 1 -- 172.21.15.55:0/4064036013 shutdown_connections 2023-10-06T03:48:58.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.092+0000 7f3b0d39d700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3af4066f30 0x7f3af40693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:58.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.092+0000 7f3b0d39d700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b081595a0 0x7f3b0816a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:58.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.092+0000 7f3b0d39d700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b080c6c60 0x7f3b0816d090 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:58.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.092+0000 7f3b0d39d700 1 --2- 172.21.15.55:0/4064036013 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b080c8830 0x7f3b08145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:58.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.092+0000 7f3b0d39d700 1 -- 172.21.15.55:0/4064036013 >> 172.21.15.55:0/4064036013 conn(0x7f3b0806ddb0 msgr2=0x7f3b080c8110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:58.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.092+0000 7f3b0d39d700 1 -- 172.21.15.55:0/4064036013 shutdown_connections 2023-10-06T03:48:58.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.092+0000 7f3b0d39d700 1 -- 172.21.15.55:0/4064036013 wait complete. 2023-10-06T03:48:58.095 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 37 2023-10-06T03:48:58.111 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:48:58.111 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:48:58.112 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:48:58.112 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:48:58.112 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:48:58.112 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:48:58.112 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:48:58.112 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:48:58.113 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:48:58.113 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:48:58.113 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:48:58.113 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:48:58.113 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:48:58.113 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:48:58.114 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:48:58.114 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:48:58.127 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:48:58.128 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:48:58.128 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:48:58.129 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:48:58.130 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:48:58.130 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:48:58.131 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:48:58.131 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:48:58.172 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:48:58.172 DEBUG:teuthology.parallel:result is None 2023-10-06T03:48:58.174 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:48:58.175 DEBUG:teuthology.parallel:result is None 2023-10-06T03:48:58.175 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:48:58.175 DEBUG:teuthology.parallel:result is None 2023-10-06T03:48:58.177 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:48:58.177 DEBUG:teuthology.parallel:result is None 2023-10-06T03:48:58.178 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:48:58.311 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T03:48:58.312 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2023-10-06T03:48:58.314 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-10-06T03:48:58.314 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T03:48:58.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.372+0000 7f944639d700 1 Processor -- start 2023-10-06T03:48:58.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.372+0000 7f944639d700 1 -- start start 2023-10-06T03:48:58.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.373+0000 7f944639d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9440142ea0 0x7f94400b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:58.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.373+0000 7f944639d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944006c0f0 0x7f944006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:58.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.373+0000 7f944639d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94400b40f0 con 0x7f94400ad870 2023-10-06T03:48:58.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.373+0000 7f944639d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9440069640 con 0x7f9440142ea0 2023-10-06T03:48:58.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.373+0000 7f944639d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9440069920 con 0x7f944006c0f0 2023-10-06T03:48:58.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.374+0000 7f9444b9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f94400ad870 0x7f944006b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43216/0 (socket says 172.21.15.55:43216) 2023-10-06T03:48:58.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.374+0000 7f9444b9a700 1 -- 172.21.15.55:0/1475899768 learned_addr learned my addr 172.21.15.55:0/1475899768 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:58.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.374+0000 7f944539b700 1 --2- 172.21.15.55:0/1475899768 >> v2:172.21.15.163:3300/0 conn(0x7f9440142ea0 0x7f94400b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:58.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.374+0000 7f9445b9c700 1 --2- 172.21.15.55:0/1475899768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944006c0f0 0x7f944006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:58.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.374+0000 7f9437fff700 1 -- 172.21.15.55:0/1475899768 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643956596 0 0) 0x7f94400b40f0 con 0x7f94400ad870 2023-10-06T03:48:58.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.374+0000 7f9437fff700 1 -- 172.21.15.55:0/1475899768 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f942c003570 con 0x7f94400ad870 2023-10-06T03:48:58.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.374+0000 7f9445b9c700 1 -- 172.21.15.55:0/1475899768 >> v2:172.21.15.163:3300/0 conn(0x7f9440142ea0 msgr2=0x7f94400b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:58.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.374+0000 7f9445b9c700 1 --2- 172.21.15.55:0/1475899768 >> v2:172.21.15.163:3300/0 conn(0x7f9440142ea0 0x7f94400b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:58.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.374+0000 7f9445b9c700 1 -- 172.21.15.55:0/1475899768 >> v1:172.21.15.55:6789/0 conn(0x7f94400ad870 legacy=0x7f944006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:58.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.374+0000 7f9445b9c700 1 -- 172.21.15.55:0/1475899768 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9440069c00 con 0x7f944006c0f0 2023-10-06T03:48:58.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.374+0000 7f944539b700 1 --2- 172.21.15.55:0/1475899768 >> v2:172.21.15.163:3300/0 conn(0x7f9440142ea0 0x7f94400b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:58.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.374+0000 7f9445b9c700 1 --2- 172.21.15.55:0/1475899768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944006c0f0 0x7f944006a120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9438009e60 tx=0x7f9438009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcd6b590f4fc7dce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:58.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.375+0000 7f9437fff700 1 -- 172.21.15.55:0/1475899768 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943800e070 con 0x7f944006c0f0 2023-10-06T03:48:58.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.375+0000 7f9437fff700 1 -- 172.21.15.55:0/1475899768 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943800bab0 con 0x7f944006c0f0 2023-10-06T03:48:58.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.375+0000 7f9437fff700 1 -- 172.21.15.55:0/1475899768 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943800be70 con 0x7f944006c0f0 2023-10-06T03:48:58.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.375+0000 7f944639d700 1 -- 172.21.15.55:0/1475899768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944006c0f0 msgr2=0x7f944006a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:58.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.375+0000 7f944639d700 1 --2- 172.21.15.55:0/1475899768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944006c0f0 0x7f944006a120 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9438009e60 tx=0x7f9438009780 comp rx=0 tx=0).stop 2023-10-06T03:48:58.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.375+0000 7f944639d700 1 -- 172.21.15.55:0/1475899768 shutdown_connections 2023-10-06T03:48:58.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.375+0000 7f944639d700 1 --2- 172.21.15.55:0/1475899768 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944006c0f0 0x7f944006a120 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:58.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.375+0000 7f944639d700 1 --2- 172.21.15.55:0/1475899768 >> v2:172.21.15.163:3300/0 conn(0x7f9440142ea0 0x7f94400b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:58.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.375+0000 7f944639d700 1 -- 172.21.15.55:0/1475899768 >> 172.21.15.55:0/1475899768 conn(0x7f94400ce430 msgr2=0x7f94400ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:58.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.375+0000 7f944639d700 1 -- 172.21.15.55:0/1475899768 shutdown_connections 2023-10-06T03:48:58.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.375+0000 7f944639d700 1 -- 172.21.15.55:0/1475899768 wait complete. 2023-10-06T03:48:58.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.377+0000 7f944639d700 1 Processor -- start 2023-10-06T03:48:58.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.377+0000 7f944639d700 1 -- start start 2023-10-06T03:48:58.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.378+0000 7f944639d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944006c0f0 0x7f944015d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:58.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.378+0000 7f944639d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9440142ea0 0x7f944015a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:58.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.378+0000 7f944639d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f944016aad0 0x7f94400aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:58.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.378+0000 7f944639d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944015c5c0 con 0x7f944016aad0 2023-10-06T03:48:58.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.378+0000 7f944639d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944015c8a0 con 0x7f9440142ea0 2023-10-06T03:48:58.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.378+0000 7f944639d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944015cb80 con 0x7f944006c0f0 2023-10-06T03:48:58.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.378+0000 7f9445b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f944016aad0 0x7f94400aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:58.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.378+0000 7f9445b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f944016aad0 0x7f94400aca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41326/0 (socket says 172.21.15.55:41326) 2023-10-06T03:48:58.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.378+0000 7f9445b9c700 1 -- 172.21.15.55:0/1206572784 learned_addr learned my addr 172.21.15.55:0/1206572784 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:58.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.378+0000 7f9444b9a700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9440142ea0 0x7f944015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:58.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.378+0000 7f944539b700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944006c0f0 0x7f944015d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:58.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.379+0000 7f9445b9c700 1 -- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944006c0f0 msgr2=0x7f944015d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:58.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.379+0000 7f9445b9c700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944006c0f0 0x7f944015d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:58.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.379+0000 7f9445b9c700 1 -- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9440142ea0 msgr2=0x7f944015a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:58.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.379+0000 7f9445b9c700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9440142ea0 0x7f944015a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:58.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.379+0000 7f9445b9c700 1 -- 172.21.15.55:0/1206572784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c009420 con 0x7f944016aad0 2023-10-06T03:48:58.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.379+0000 7f9444b9a700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9440142ea0 0x7f944015a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:48:58.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.379+0000 7f9445b9c700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f944016aad0 0x7f94400aca30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9438013000 tx=0x7f9438019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:58.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.379+0000 7f94357fa700 1 -- 172.21.15.55:0/1206572784 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438012e10 con 0x7f944016aad0 2023-10-06T03:48:58.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.379+0000 7f94357fa700 1 -- 172.21.15.55:0/1206572784 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9438027470 con 0x7f944016aad0 2023-10-06T03:48:58.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.379+0000 7f94357fa700 1 -- 172.21.15.55:0/1206572784 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438027830 con 0x7f944016aad0 2023-10-06T03:48:58.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.379+0000 7f944639d700 1 -- 172.21.15.55:0/1206572784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9438009a20 con 0x7f944016aad0 2023-10-06T03:48:58.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.379+0000 7f944639d700 1 -- 172.21.15.55:0/1206572784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944015d140 con 0x7f944016aad0 2023-10-06T03:48:58.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.380+0000 7f944639d700 1 -- 172.21.15.55:0/1206572784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f944015d4b0 con 0x7f944016aad0 2023-10-06T03:48:58.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.381+0000 7f94357fa700 1 -- 172.21.15.55:0/1206572784 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9438024070 con 0x7f944016aad0 2023-10-06T03:48:58.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.382+0000 7f94357fa700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9424066f30 0x7f94240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:58.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.382+0000 7f94357fa700 1 -- 172.21.15.55:0/1206572784 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f9438087150 con 0x7f944016aad0 2023-10-06T03:48:58.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.383+0000 7f944539b700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9424066f30 0x7f94240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:58.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.383+0000 7f944539b700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9424066f30 0x7f94240693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f944015f040 tx=0x7f943c0091a0 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:58.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.386+0000 7f94357fa700 1 -- 172.21.15.55:0/1206572784 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94380541a0 con 0x7f944016aad0 2023-10-06T03:48:58.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:58.603+0000 7f944639d700 1 -- 172.21.15.55:0/1206572784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f944004c3c0 con 0x7f944016aad0 2023-10-06T03:48:59.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.028+0000 7f94357fa700 1 -- 172.21.15.55:0/1206572784 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v87) v1 ==== 97+0+0 (secure 0 0 0) 0x7f94380577c0 con 0x7f944016aad0 2023-10-06T03:48:59.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.031+0000 7f944639d700 1 -- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9424066f30 msgr2=0x7f94240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:59.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.031+0000 7f944639d700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9424066f30 0x7f94240693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f944015f040 tx=0x7f943c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.031+0000 7f944639d700 1 -- 172.21.15.55:0/1206572784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f944016aad0 msgr2=0x7f94400aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:59.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.031+0000 7f944639d700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f944016aad0 0x7f94400aca30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9438013000 tx=0x7f9438019000 comp rx=0 tx=0).stop 2023-10-06T03:48:59.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.032+0000 7f944639d700 1 -- 172.21.15.55:0/1206572784 shutdown_connections 2023-10-06T03:48:59.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.032+0000 7f944639d700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9424066f30 0x7f94240693e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.032+0000 7f944639d700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f944016aad0 0x7f94400aca30 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.032+0000 7f944639d700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f944006c0f0 0x7f944015d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.032+0000 7f944639d700 1 --2- 172.21.15.55:0/1206572784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9440142ea0 0x7f944015a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.032+0000 7f944639d700 1 -- 172.21.15.55:0/1206572784 >> 172.21.15.55:0/1206572784 conn(0x7f94400ce430 msgr2=0x7f94400cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:59.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.032+0000 7f944639d700 1 -- 172.21.15.55:0/1206572784 shutdown_connections 2023-10-06T03:48:59.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.032+0000 7f944639d700 1 -- 172.21.15.55:0/1206572784 wait complete. 2023-10-06T03:48:59.035 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2023-10-06T03:48:59.054 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:48:59.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.240+0000 7f932a39d700 1 Processor -- start 2023-10-06T03:48:59.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.240+0000 7f932a39d700 1 -- start start 2023-10-06T03:48:59.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.241+0000 7f932a39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f93240c6c60 0x7f93240c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:59.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.241+0000 7f932a39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93240b1410 con 0x7f932406e8c0 2023-10-06T03:48:59.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.242+0000 7f932a39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9324063230 con 0x7f93241595a0 2023-10-06T03:48:59.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.242+0000 7f932a39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93240b2b90 con 0x7f93240c6c60 2023-10-06T03:48:59.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.242+0000 7f9328b9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f93240c6c60 0x7f93240c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:59.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.242+0000 7f9329b9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f932406e8c0 0x7f9324159970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43228/0 (socket says 172.21.15.55:43228) 2023-10-06T03:48:59.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.242+0000 7f9329b9c700 1 -- 172.21.15.55:0/3305017251 learned_addr learned my addr 172.21.15.55:0/3305017251 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:48:59.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.242+0000 7f9328b9a700 1 -- 172.21.15.55:0/3305017251 >> v1:172.21.15.163:6789/0 conn(0x7f93241595a0 legacy=0x7f93240b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:59.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.242+0000 7f9328b9a700 1 -- 172.21.15.55:0/3305017251 >> v1:172.21.15.55:6789/0 conn(0x7f932406e8c0 legacy=0x7f9324159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:59.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.242+0000 7f9328b9a700 1 -- 172.21.15.55:0/3305017251 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93240634f0 con 0x7f93240c6c60 2023-10-06T03:48:59.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.242+0000 7f9328b9a700 1 --2- 172.21.15.55:0/3305017251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240c6c60 0x7f93240c8120 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f93140098e0 tx=0x7f93140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe729d8b33c3c3f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:59.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.243+0000 7f931bfff700 1 -- 172.21.15.55:0/3305017251 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9314009e80 con 0x7f93240c6c60 2023-10-06T03:48:59.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.243+0000 7f931bfff700 1 -- 172.21.15.55:0/3305017251 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931400b820 con 0x7f93240c6c60 2023-10-06T03:48:59.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.243+0000 7f931bfff700 1 -- 172.21.15.55:0/3305017251 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931400bba0 con 0x7f93240c6c60 2023-10-06T03:48:59.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.243+0000 7f932a39d700 1 -- 172.21.15.55:0/3305017251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240c6c60 msgr2=0x7f93240c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:59.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.243+0000 7f932a39d700 1 --2- 172.21.15.55:0/3305017251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240c6c60 0x7f93240c8120 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f93140098e0 tx=0x7f93140091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.243+0000 7f932a39d700 1 -- 172.21.15.55:0/3305017251 shutdown_connections 2023-10-06T03:48:59.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.243+0000 7f932a39d700 1 --2- 172.21.15.55:0/3305017251 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240c6c60 0x7f93240c8120 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.243+0000 7f932a39d700 1 -- 172.21.15.55:0/3305017251 >> 172.21.15.55:0/3305017251 conn(0x7f932406ddb0 msgr2=0x7f932406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:59.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.243+0000 7f932a39d700 1 -- 172.21.15.55:0/3305017251 shutdown_connections 2023-10-06T03:48:59.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.243+0000 7f932a39d700 1 -- 172.21.15.55:0/3305017251 wait complete. 2023-10-06T03:48:59.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.245+0000 7f932a39d700 1 Processor -- start 2023-10-06T03:48:59.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.245+0000 7f932a39d700 1 -- start start 2023-10-06T03:48:59.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.246+0000 7f932a39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f932406e8c0 0x7f9324151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:59.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.246+0000 7f932a39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240c6c60 0x7f932413a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:59.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.246+0000 7f932a39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93241595a0 0x7f932415a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:59.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.246+0000 7f932a39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932415b2b0 con 0x7f93240c6c60 2023-10-06T03:48:59.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.246+0000 7f932a39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932415b590 con 0x7f93241595a0 2023-10-06T03:48:59.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.246+0000 7f932a39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932415b870 con 0x7f932406e8c0 2023-10-06T03:48:59.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.246+0000 7f9328b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240c6c60 0x7f932413a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:59.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.246+0000 7f9328b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240c6c60 0x7f932413a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41338/0 (socket says 172.21.15.55:41338) 2023-10-06T03:48:59.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.246+0000 7f9328b9a700 1 -- 172.21.15.55:0/1625978904 learned_addr learned my addr 172.21.15.55:0/1625978904 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:59.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.246+0000 7f9329b9c700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93241595a0 0x7f932415a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:59.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.246+0000 7f932939b700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f932406e8c0 0x7f9324151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:59.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.247+0000 7f9328b9a700 1 -- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f932406e8c0 msgr2=0x7f9324151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:59.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.247+0000 7f9328b9a700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f932406e8c0 0x7f9324151f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.247+0000 7f9328b9a700 1 -- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93241595a0 msgr2=0x7f932415a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:59.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.247+0000 7f9328b9a700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93241595a0 0x7f932415a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.247+0000 7f9328b9a700 1 -- 172.21.15.55:0/1625978904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9314009420 con 0x7f93240c6c60 2023-10-06T03:48:59.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.247+0000 7f932939b700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f932406e8c0 0x7f9324151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:48:59.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.247+0000 7f9328b9a700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240c6c60 0x7f932413a070 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f93140030c0 tx=0x7f9314016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:59.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.247+0000 7f93197fa700 1 -- 172.21.15.55:0/1625978904 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931401b470 con 0x7f93240c6c60 2023-10-06T03:48:59.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.247+0000 7f93197fa700 1 -- 172.21.15.55:0/1625978904 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931401b5f0 con 0x7f93240c6c60 2023-10-06T03:48:59.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.247+0000 7f93197fa700 1 -- 172.21.15.55:0/1625978904 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931401b9b0 con 0x7f93240c6c60 2023-10-06T03:48:59.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.247+0000 7f932a39d700 1 -- 172.21.15.55:0/1625978904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932415bb50 con 0x7f93240c6c60 2023-10-06T03:48:59.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.247+0000 7f932a39d700 1 -- 172.21.15.55:0/1625978904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932413a5b0 con 0x7f93240c6c60 2023-10-06T03:48:59.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.248+0000 7f932a39d700 1 -- 172.21.15.55:0/1625978904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932415c070 con 0x7f93240c6c60 2023-10-06T03:48:59.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.249+0000 7f93197fa700 1 -- 172.21.15.55:0/1625978904 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f931401c450 con 0x7f93240c6c60 2023-10-06T03:48:59.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.250+0000 7f93197fa700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9308066f30 0x7f93080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:59.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.250+0000 7f93197fa700 1 -- 172.21.15.55:0/1625978904 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f93140862f0 con 0x7f93240c6c60 2023-10-06T03:48:59.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.251+0000 7f932939b700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9308066f30 0x7f93080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:59.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.251+0000 7f932939b700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9308066f30 0x7f93080693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9320006fd0 tx=0x7f9320008040 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:59.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.254+0000 7f93197fa700 1 -- 172.21.15.55:0/1625978904 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f932415c070 con 0x7f93240c6c60 2023-10-06T03:48:59.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.533+0000 7f932a39d700 1 -- 172.21.15.55:0/1625978904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f932415c070 con 0x7f93240c6c60 2023-10-06T03:48:59.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.533+0000 7f93197fa700 1 -- 172.21.15.55:0/1625978904 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f932415c070 con 0x7f93240c6c60 2023-10-06T03:48:59.535 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:59.535 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:48:59.535 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2023-10-06T03:48:59.535 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.535 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2023-10-06T03:48:59.535 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9li2RaGRAAgRjtbw3h40vylEPjolzlpw==", 2023-10-06T03:48:59.536 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.536 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:59.536 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:48:59.536 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:48:59.536 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.536 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.536 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.536 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.536 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2023-10-06T03:48:59.536 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lXZu7LxAAu3QAtdDxtD/AuuqGG12sDw==", 2023-10-06T03:48:59.536 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.537 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:59.537 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:48:59.537 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:48:59.537 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.537 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.537 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.537 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.537 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2023-10-06T03:48:59.537 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lS22fJBAAZi6CfyLyJAFgIkaarad1mA==", 2023-10-06T03:48:59.537 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.537 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:59.538 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:48:59.538 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:48:59.538 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.538 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.538 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.538 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.538 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2023-10-06T03:48:59.538 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9ll0RIOhAAGtGdPv4Y2bge7xIv3eu2NQ==", 2023-10-06T03:48:59.538 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.538 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:59.538 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:48:59.539 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:48:59.539 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.539 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.539 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.539 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.539 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2023-10-06T03:48:59.539 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDHfx9lmTeaDRAAuycc9i7fPj8xrDkuZCAGVg==", 2023-10-06T03:48:59.539 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.539 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:59.539 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:59.539 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.539 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.540 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.540 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.540 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2023-10-06T03:48:59.540 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDKfx9lxOdXEhAA9S8tdOvhnmBeEbutqnUsZg==", 2023-10-06T03:48:59.540 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.540 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:59.541 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:59.541 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.541 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.541 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.541 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.541 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2023-10-06T03:48:59.541 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDNfx9l76I+GRAAjaj0gM7An9BCyGmfQgaDFw==", 2023-10-06T03:48:59.541 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.541 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:59.541 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:59.541 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.541 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.542 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.542 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.542 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2023-10-06T03:48:59.542 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDQfx9lsPK+HxAAzQCV0EuBenqOkUpiQW9uog==", 2023-10-06T03:48:59.542 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.542 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:59.542 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:59.542 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.542 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.542 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.542 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.542 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2023-10-06T03:48:59.543 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDWfx9lSHNrMxAAIn7awDmljO2cRtKyDalWnw==", 2023-10-06T03:48:59.543 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.543 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:59.543 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:59.543 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.543 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.543 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.543 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.543 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2023-10-06T03:48:59.543 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDZfx9lFoWuOBAAhzQOt3YChvuv0MhrspOjPw==", 2023-10-06T03:48:59.543 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.544 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:59.544 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:59.544 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.544 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.544 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.544 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.544 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2023-10-06T03:48:59.544 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDcfx9lB8iSOhAAwamSgGIXZ8owFE4FdbOOBw==", 2023-10-06T03:48:59.544 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.544 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:59.544 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:59.544 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.545 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.545 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.545 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.545 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2023-10-06T03:48:59.545 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDgfx9lUb3mABAAJ75UO2UQg2r+l+tDYVn0Dg==", 2023-10-06T03:48:59.545 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.545 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:48:59.545 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:48:59.545 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.545 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.545 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.545 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.546 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2023-10-06T03:48:59.546 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lfNvCBxAA1ZJq4XsWTZHVY0u0NdOqIw==", 2023-10-06T03:48:59.546 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.546 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:59.546 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2023-10-06T03:48:59.546 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:48:59.546 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.546 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.546 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.546 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2023-10-06T03:48:59.546 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lnvkoFhAAafxsOmItxmwUr1GJrGYJ6g==", 2023-10-06T03:48:59.547 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.547 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:59.547 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:48:59.547 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:48:59.547 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:48:59.547 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.547 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.547 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.547 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2023-10-06T03:48:59.547 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9l+9b4DhAAc5XHdikWbXsEblSE3FGScQ==", 2023-10-06T03:48:59.547 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.547 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:59.548 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:48:59.548 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:48:59.548 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:48:59.548 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.548 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.548 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.548 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2023-10-06T03:48:59.548 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9luSXrHBAAg6Ufa1Ep/EOgsQtuOPQgYg==", 2023-10-06T03:48:59.548 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.548 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:48:59.548 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:48:59.548 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:48:59.548 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:48:59.549 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.549 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.549 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.549 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2023-10-06T03:48:59.549 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC+fx9lEb50FxAAt0Oq28gsXtQvol7TV8trgg==", 2023-10-06T03:48:59.549 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.549 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:48:59.549 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:48:59.549 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:48:59.549 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.549 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.549 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.550 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.550 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2023-10-06T03:48:59.550 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lQAANEhAAsNiWPqCNEyUS3IELXSC1Cw==", 2023-10-06T03:48:59.550 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.550 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:48:59.550 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T03:48:59.550 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.550 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.550 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:48:59.550 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:48:59.550 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2023-10-06T03:48:59.550 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lCwU2ABAAMijxXd8hwgk3XzcdP4NZyg==", 2023-10-06T03:48:59.551 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:48:59.551 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:48:59.551 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T03:48:59.551 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:48:59.551 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.551 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:48:59.551 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2023-10-06T03:48:59.551 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:48:59.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.535+0000 7f932a39d700 1 -- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9308066f30 msgr2=0x7f93080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:59.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.535+0000 7f932a39d700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9308066f30 0x7f93080693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9320006fd0 tx=0x7f9320008040 comp rx=0 tx=0).stop 2023-10-06T03:48:59.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.535+0000 7f932a39d700 1 -- 172.21.15.55:0/1625978904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240c6c60 msgr2=0x7f932413a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:59.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.535+0000 7f932a39d700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240c6c60 0x7f932413a070 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f93140030c0 tx=0x7f9314016000 comp rx=0 tx=0).stop 2023-10-06T03:48:59.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.535+0000 7f932a39d700 1 -- 172.21.15.55:0/1625978904 shutdown_connections 2023-10-06T03:48:59.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.535+0000 7f932a39d700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9308066f30 0x7f93080693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.535+0000 7f932a39d700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f93241595a0 0x7f932415a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.535+0000 7f932a39d700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f932406e8c0 0x7f9324151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.535+0000 7f932a39d700 1 --2- 172.21.15.55:0/1625978904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f93240c6c60 0x7f932413a070 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.535+0000 7f932a39d700 1 -- 172.21.15.55:0/1625978904 >> 172.21.15.55:0/1625978904 conn(0x7f932406ddb0 msgr2=0x7f93241552d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:59.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.535+0000 7f932a39d700 1 -- 172.21.15.55:0/1625978904 shutdown_connections 2023-10-06T03:48:59.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.535+0000 7f932a39d700 1 -- 172.21.15.55:0/1625978904 wait complete. 2023-10-06T03:48:59.556 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:48:59.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.742+0000 7f87abfff700 1 Processor -- start 2023-10-06T03:48:59.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.742+0000 7f87abfff700 1 -- start start 2023-10-06T03:48:59.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.743+0000 7f87abfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f87ac150180 0x7f87ac157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:59.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.743+0000 7f87abfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f87ac0c7410 0x7f87ac069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:59.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.743+0000 7f87abfff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ac06e2a0 con 0x7f87ac150550 2023-10-06T03:48:59.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.743+0000 7f87abfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ac06b2e0 con 0x7f87ac0c7410 2023-10-06T03:48:59.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.743+0000 7f87abfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ac0c78e0 con 0x7f87ac150180 2023-10-06T03:48:59.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.744+0000 7f87ab7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f87ac0c7410 0x7f87ac069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:59.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.744+0000 7f87aaffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f87ac150180 0x7f87ac157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:59.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.744+0000 7f87ab7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f87ac0c7410 0x7f87ac069f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41342/0 (socket says 172.21.15.55:41342) 2023-10-06T03:48:59.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.744+0000 7f87ab7fe700 1 -- 172.21.15.55:0/2793248892 learned_addr learned my addr 172.21.15.55:0/2793248892 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:59.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.744+0000 7f87ab7fe700 1 -- 172.21.15.55:0/2793248892 >> v2:172.21.15.163:3300/0 conn(0x7f87ac150180 msgr2=0x7f87ac157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:48:59.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.744+0000 7f87ab7fe700 1 --2- 172.21.15.55:0/2793248892 >> v2:172.21.15.163:3300/0 conn(0x7f87ac150180 0x7f87ac157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.744+0000 7f87ab7fe700 1 -- 172.21.15.55:0/2793248892 >> v1:172.21.15.163:6789/0 conn(0x7f87ac150550 legacy=0x7f87ac0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:59.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.744+0000 7f87ab7fe700 1 -- 172.21.15.55:0/2793248892 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac0c7bc0 con 0x7f87ac0c7410 2023-10-06T03:48:59.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.744+0000 7f87ab7fe700 1 --2- 172.21.15.55:0/2793248892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ac0c7410 0x7f87ac069f50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f87ac06fd20 tx=0x7f87a400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6baccc5a906eb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:59.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.745+0000 7f87a9ffb700 1 -- 172.21.15.55:0/2793248892 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4010070 con 0x7f87ac0c7410 2023-10-06T03:48:59.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.745+0000 7f87a9ffb700 1 -- 172.21.15.55:0/2793248892 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a400dbd0 con 0x7f87ac0c7410 2023-10-06T03:48:59.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.745+0000 7f87a9ffb700 1 -- 172.21.15.55:0/2793248892 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4016400 con 0x7f87ac0c7410 2023-10-06T03:48:59.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.745+0000 7f87abfff700 1 -- 172.21.15.55:0/2793248892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ac0c7410 msgr2=0x7f87ac069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:59.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.745+0000 7f87abfff700 1 --2- 172.21.15.55:0/2793248892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ac0c7410 0x7f87ac069f50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f87ac06fd20 tx=0x7f87a400ba50 comp rx=0 tx=0).stop 2023-10-06T03:48:59.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.745+0000 7f87abfff700 1 -- 172.21.15.55:0/2793248892 shutdown_connections 2023-10-06T03:48:59.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.745+0000 7f87abfff700 1 --2- 172.21.15.55:0/2793248892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ac0c7410 0x7f87ac069f50 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.745+0000 7f87abfff700 1 --2- 172.21.15.55:0/2793248892 >> v2:172.21.15.163:3300/0 conn(0x7f87ac150180 0x7f87ac157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.745+0000 7f87abfff700 1 -- 172.21.15.55:0/2793248892 >> 172.21.15.55:0/2793248892 conn(0x7f87ac0cc1f0 msgr2=0x7f87ac0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:59.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.745+0000 7f87abfff700 1 -- 172.21.15.55:0/2793248892 shutdown_connections 2023-10-06T03:48:59.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.745+0000 7f87abfff700 1 -- 172.21.15.55:0/2793248892 wait complete. 2023-10-06T03:48:59.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.747+0000 7f87abfff700 1 Processor -- start 2023-10-06T03:48:59.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.747+0000 7f87abfff700 1 -- start start 2023-10-06T03:48:59.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.748+0000 7f87abfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ac0c7410 0x7f87ac0ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:59.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.748+0000 7f87abfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87ac150180 0x7f87ac14ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:59.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.748+0000 7f87abfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ac150550 0x7f87ac14a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:59.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.748+0000 7f87abfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac0c5950 con 0x7f87ac150550 2023-10-06T03:48:59.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.748+0000 7f87abfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac14a960 con 0x7f87ac0c7410 2023-10-06T03:48:59.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.748+0000 7f87abfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac0ae6d0 con 0x7f87ac150180 2023-10-06T03:48:59.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.748+0000 7f87ab7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ac150550 0x7f87ac14a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:59.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.748+0000 7f87ab7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ac150550 0x7f87ac14a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41346/0 (socket says 172.21.15.55:41346) 2023-10-06T03:48:59.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.748+0000 7f87ab7fe700 1 -- 172.21.15.55:0/927938562 learned_addr learned my addr 172.21.15.55:0/927938562 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:48:59.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.748+0000 7f87aaffd700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ac0c7410 0x7f87ac0ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:59.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.748+0000 7f87aa7fc700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87ac150180 0x7f87ac14ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:59.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.749+0000 7f87ab7fe700 1 -- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87ac150180 msgr2=0x7f87ac14ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:59.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.749+0000 7f87ab7fe700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87ac150180 0x7f87ac14ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.749+0000 7f87ab7fe700 1 -- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ac0c7410 msgr2=0x7f87ac0ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:59.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.749+0000 7f87ab7fe700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ac0c7410 0x7f87ac0ae0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.749+0000 7f87ab7fe700 1 -- 172.21.15.55:0/927938562 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a400bcf0 con 0x7f87ac150550 2023-10-06T03:48:59.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.749+0000 7f87aa7fc700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87ac150180 0x7f87ac14ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:48:59.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.749+0000 7f87ab7fe700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ac150550 0x7f87ac14a420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f87a400ba90 tx=0x7f87a401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:59.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.749+0000 7f8792ffd700 1 -- 172.21.15.55:0/927938562 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4016400 con 0x7f87ac150550 2023-10-06T03:48:59.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.749+0000 7f8792ffd700 1 -- 172.21.15.55:0/927938562 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a40032f0 con 0x7f87ac150550 2023-10-06T03:48:59.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.749+0000 7f87abfff700 1 -- 172.21.15.55:0/927938562 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ac0c4e70 con 0x7f87ac150550 2023-10-06T03:48:59.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.749+0000 7f8792ffd700 1 -- 172.21.15.55:0/927938562 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a401f6b0 con 0x7f87ac150550 2023-10-06T03:48:59.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.749+0000 7f87abfff700 1 -- 172.21.15.55:0/927938562 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ac0c5430 con 0x7f87ac150550 2023-10-06T03:48:59.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.750+0000 7f87abfff700 1 -- 172.21.15.55:0/927938562 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ac0c4ca0 con 0x7f87ac150550 2023-10-06T03:48:59.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.751+0000 7f8792ffd700 1 -- 172.21.15.55:0/927938562 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f87a4003040 con 0x7f87ac150550 2023-10-06T03:48:59.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.752+0000 7f8792ffd700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8798066f30 0x7f87980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:48:59.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.753+0000 7f8792ffd700 1 -- 172.21.15.55:0/927938562 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f87a4086750 con 0x7f87ac150550 2023-10-06T03:48:59.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.753+0000 7f87aaffd700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8798066f30 0x7f87980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:48:59.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.754+0000 7f87aaffd700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8798066f30 0x7f87980693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f87ac06a0a0 tx=0x7f879c0091a0 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:48:59.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.757+0000 7f8792ffd700 1 -- 172.21.15.55:0/927938562 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87a4053990 con 0x7f87ac150550 2023-10-06T03:48:59.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.962+0000 7f87abfff700 1 -- 172.21.15.55:0/927938562 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87ac04c3c0 con 0x7f87ac150550 2023-10-06T03:48:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.963+0000 7f8792ffd700 1 -- 172.21.15.55:0/927938562 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+14294 (secure 0 0 0) 0x7f87a4056fb0 con 0x7f87ac150550 2023-10-06T03:48:59.964 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:48:59.965 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":89,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:48:59.041068+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:48:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.965+0000 7f87abfff700 1 -- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8798066f30 msgr2=0x7f87980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.965+0000 7f87abfff700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8798066f30 0x7f87980693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f87ac06a0a0 tx=0x7f879c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.965+0000 7f87abfff700 1 -- 172.21.15.55:0/927938562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ac150550 msgr2=0x7f87ac14a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:48:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.965+0000 7f87abfff700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ac150550 0x7f87ac14a420 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f87a400ba90 tx=0x7f87a401c000 comp rx=0 tx=0).stop 2023-10-06T03:48:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.966+0000 7f87abfff700 1 -- 172.21.15.55:0/927938562 shutdown_connections 2023-10-06T03:48:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.966+0000 7f87abfff700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8798066f30 0x7f87980693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.966+0000 7f87abfff700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ac150550 0x7f87ac14a420 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.966+0000 7f87abfff700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ac0c7410 0x7f87ac0ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.966+0000 7f87abfff700 1 --2- 172.21.15.55:0/927938562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87ac150180 0x7f87ac14ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:48:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.966+0000 7f87abfff700 1 -- 172.21.15.55:0/927938562 >> 172.21.15.55:0/927938562 conn(0x7f87ac0cc1f0 msgr2=0x7f87ac0696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:48:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.966+0000 7f87abfff700 1 -- 172.21.15.55:0/927938562 shutdown_connections 2023-10-06T03:48:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:48:59.966+0000 7f87abfff700 1 -- 172.21.15.55:0/927938562 wait complete. 2023-10-06T03:48:59.985 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T03:48:59.986 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:49:00.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.180+0000 7fbd26b9d700 1 Processor -- start 2023-10-06T03:49:00.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.180+0000 7fbd26b9d700 1 -- start start 2023-10-06T03:49:00.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.182+0000 7fbd26b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd2006bfa0 0x7fbd20068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:00.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.182+0000 7fbd26b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbd200688d0 0x7fbd200caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:00.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.182+0000 7fbd26b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd200ca920 con 0x7fbd20159a70 2023-10-06T03:49:00.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.182+0000 7fbd26b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd20068d50 con 0x7fbd200688d0 2023-10-06T03:49:00.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.182+0000 7fbd26b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2006af00 con 0x7fbd2006bfa0 2023-10-06T03:49:00.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.182+0000 7fbd25b9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbd20159a70 0x7fbd200c82f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43244/0 (socket says 172.21.15.55:43244) 2023-10-06T03:49:00.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.182+0000 7fbd25b9b700 1 -- 172.21.15.55:0/3887596378 learned_addr learned my addr 172.21.15.55:0/3887596378 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:00.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.182+0000 7fbd2639c700 1 --2- 172.21.15.55:0/3887596378 >> v2:172.21.15.55:3300/0 conn(0x7fbd200688d0 0x7fbd200caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:00.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.182+0000 7fbd2539a700 1 --2- 172.21.15.55:0/3887596378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd2006bfa0 0x7fbd20068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:00.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.183+0000 7fbd24b99700 1 -- 172.21.15.55:0/3887596378 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204396094 0 0) 0x7fbd200ca920 con 0x7fbd20159a70 2023-10-06T03:49:00.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.183+0000 7fbd24b99700 1 -- 172.21.15.55:0/3887596378 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd0c003570 con 0x7fbd20159a70 2023-10-06T03:49:00.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.183+0000 7fbd2639c700 1 -- 172.21.15.55:0/3887596378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd2006bfa0 msgr2=0x7fbd20068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:00.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.183+0000 7fbd2639c700 1 --2- 172.21.15.55:0/3887596378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd2006bfa0 0x7fbd20068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.183+0000 7fbd2639c700 1 -- 172.21.15.55:0/3887596378 >> v1:172.21.15.55:6789/0 conn(0x7fbd20159a70 legacy=0x7fbd200c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:00.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.183+0000 7fbd2639c700 1 -- 172.21.15.55:0/3887596378 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2006b1e0 con 0x7fbd200688d0 2023-10-06T03:49:00.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.183+0000 7fbd2639c700 1 --2- 172.21.15.55:0/3887596378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd200688d0 0x7fbd200caec0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009e60 tx=0x7fbd18009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94437d41649ea70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:00.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.183+0000 7fbd24b99700 1 -- 172.21.15.55:0/3887596378 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1800e070 con 0x7fbd200688d0 2023-10-06T03:49:00.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.183+0000 7fbd24b99700 1 -- 172.21.15.55:0/3887596378 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1800bab0 con 0x7fbd200688d0 2023-10-06T03:49:00.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.183+0000 7fbd24b99700 1 -- 172.21.15.55:0/3887596378 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1800be70 con 0x7fbd200688d0 2023-10-06T03:49:00.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.184+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3887596378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd200688d0 msgr2=0x7fbd200caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:00.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.184+0000 7fbd26b9d700 1 --2- 172.21.15.55:0/3887596378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd200688d0 0x7fbd200caec0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009e60 tx=0x7fbd18009780 comp rx=0 tx=0).stop 2023-10-06T03:49:00.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.184+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3887596378 shutdown_connections 2023-10-06T03:49:00.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.184+0000 7fbd26b9d700 1 --2- 172.21.15.55:0/3887596378 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd200688d0 0x7fbd200caec0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.184+0000 7fbd26b9d700 1 --2- 172.21.15.55:0/3887596378 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd2006bfa0 0x7fbd20068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.184+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3887596378 >> 172.21.15.55:0/3887596378 conn(0x7fbd200cddc0 msgr2=0x7fbd200c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:00.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.184+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3887596378 shutdown_connections 2023-10-06T03:49:00.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.184+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3887596378 wait complete. 2023-10-06T03:49:00.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.185+0000 7fbd26b9d700 1 Processor -- start 2023-10-06T03:49:00.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.186+0000 7fbd26b9d700 1 -- start start 2023-10-06T03:49:00.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.186+0000 7fbd26b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd200688d0 0x7fbd20155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:00.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.186+0000 7fbd26b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd2006bfa0 0x7fbd20152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:00.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.186+0000 7fbd26b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd20159a70 0x7fbd20157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:00.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.186+0000 7fbd26b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd200c6630 con 0x7fbd200688d0 2023-10-06T03:49:00.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.186+0000 7fbd26b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd200b1290 con 0x7fbd2006bfa0 2023-10-06T03:49:00.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.186+0000 7fbd26b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd200c6070 con 0x7fbd20159a70 2023-10-06T03:49:00.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.187+0000 7fbd25b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd200688d0 0x7fbd20155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:00.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.187+0000 7fbd25b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd200688d0 0x7fbd20155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41364/0 (socket says 172.21.15.55:41364) 2023-10-06T03:49:00.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.187+0000 7fbd25b9b700 1 -- 172.21.15.55:0/3068441364 learned_addr learned my addr 172.21.15.55:0/3068441364 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:00.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.187+0000 7fbd2539a700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd2006bfa0 0x7fbd20152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:00.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.187+0000 7fbd2639c700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd20159a70 0x7fbd20157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:00.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.187+0000 7fbd25b9b700 1 -- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd20159a70 msgr2=0x7fbd20157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:00.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.187+0000 7fbd25b9b700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd20159a70 0x7fbd20157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.187+0000 7fbd25b9b700 1 -- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd2006bfa0 msgr2=0x7fbd20152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:00.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.187+0000 7fbd25b9b700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd2006bfa0 0x7fbd20152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.187+0000 7fbd25b9b700 1 -- 172.21.15.55:0/3068441364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd18009a20 con 0x7fbd200688d0 2023-10-06T03:49:00.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.187+0000 7fbd2639c700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd20159a70 0x7fbd20157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:00.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.187+0000 7fbd25b9b700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd200688d0 0x7fbd20155dc0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00c7c0 tx=0x7fbd1c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:00.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.188+0000 7fbd15ffb700 1 -- 172.21.15.55:0/3068441364 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c005b30 con 0x7fbd200688d0 2023-10-06T03:49:00.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.188+0000 7fbd15ffb700 1 -- 172.21.15.55:0/3068441364 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1c005cb0 con 0x7fbd200688d0 2023-10-06T03:49:00.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.188+0000 7fbd15ffb700 1 -- 172.21.15.55:0/3068441364 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c007320 con 0x7fbd200688d0 2023-10-06T03:49:00.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.188+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3068441364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd200c6350 con 0x7fbd200688d0 2023-10-06T03:49:00.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.188+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3068441364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd20153430 con 0x7fbd200688d0 2023-10-06T03:49:00.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.189+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3068441364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd200b1060 con 0x7fbd200688d0 2023-10-06T03:49:00.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.190+0000 7fbd15ffb700 1 -- 172.21.15.55:0/3068441364 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd1c01b3b0 con 0x7fbd200688d0 2023-10-06T03:49:00.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.190+0000 7fbd15ffb700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd04066f30 0x7fbd040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:00.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.190+0000 7fbd15ffb700 1 -- 172.21.15.55:0/3068441364 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fbd1c083a50 con 0x7fbd200688d0 2023-10-06T03:49:00.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.191+0000 7fbd2539a700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd04066f30 0x7fbd040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:00.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.192+0000 7fbd2539a700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd04066f30 0x7fbd040693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbd2016d130 tx=0x7fbd10009120 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:00.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.195+0000 7fbd15ffb700 1 -- 172.21.15.55:0/3068441364 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd200b1060 con 0x7fbd200688d0 2023-10-06T03:49:00.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.395+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3068441364 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbd200b1060 con 0x7fbd200688d0 2023-10-06T03:49:00.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.396+0000 7fbd15ffb700 1 -- 172.21.15.55:0/3068441364 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v89) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbd200b1060 con 0x7fbd200688d0 2023-10-06T03:49:00.398 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:49:00.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.398+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd04066f30 msgr2=0x7fbd040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:00.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.398+0000 7fbd26b9d700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd04066f30 0x7fbd040693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbd2016d130 tx=0x7fbd10009120 comp rx=0 tx=0).stop 2023-10-06T03:49:00.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.398+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3068441364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd200688d0 msgr2=0x7fbd20155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:00.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.398+0000 7fbd26b9d700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd200688d0 0x7fbd20155dc0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00c7c0 tx=0x7fbd1c007e00 comp rx=0 tx=0).stop 2023-10-06T03:49:00.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.398+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3068441364 shutdown_connections 2023-10-06T03:49:00.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.398+0000 7fbd26b9d700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd04066f30 0x7fbd040693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.398+0000 7fbd26b9d700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd20159a70 0x7fbd20157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.398+0000 7fbd26b9d700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd200688d0 0x7fbd20155dc0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.398+0000 7fbd26b9d700 1 --2- 172.21.15.55:0/3068441364 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd2006bfa0 0x7fbd20152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.398+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3068441364 >> 172.21.15.55:0/3068441364 conn(0x7fbd200cddc0 msgr2=0x7fbd200c2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:00.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.398+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3068441364 shutdown_connections 2023-10-06T03:49:00.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.398+0000 7fbd26b9d700 1 -- 172.21.15.55:0/3068441364 wait complete. 2023-10-06T03:49:00.417 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:49:00.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.607+0000 7f1d1cb9d700 1 Processor -- start 2023-10-06T03:49:00.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.607+0000 7f1d1cb9d700 1 -- start start 2023-10-06T03:49:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.608+0000 7f1d1cb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d18159a60 0x7f1d180c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.608+0000 7f1d1cb9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1d180ccc00 0x7f1d18069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.608+0000 7f1d1cb9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1d1806a020 0x7f1d180cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.608+0000 7f1d1cb9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1806dfc0 con 0x7f1d1806a020 2023-10-06T03:49:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.608+0000 7f1d1cb9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d180c9cb0 con 0x7f1d180ccc00 2023-10-06T03:49:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.608+0000 7f1d1cb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d180c9f30 con 0x7f1d18159a60 2023-10-06T03:49:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.609+0000 7f1d17fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1d1806a020 0x7f1d180cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.609+0000 7f1d17fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1d1806a020 0x7f1d180cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41374/0 (socket says 172.21.15.55:41374) 2023-10-06T03:49:00.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.609+0000 7f1d17fff700 1 -- 172.21.15.55:0/1087796081 learned_addr learned my addr 172.21.15.55:0/1087796081 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.609+0000 7f1d16ffd700 1 --2- 172.21.15.55:0/1087796081 >> v2:172.21.15.163:3300/0 conn(0x7f1d180ccc00 0x7f1d18069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.609+0000 7f1d177fe700 1 --2- 172.21.15.55:0/1087796081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d18159a60 0x7f1d180c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.609+0000 7f1d17fff700 1 -- 172.21.15.55:0/1087796081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d18159a60 msgr2=0x7f1d180c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.609+0000 7f1d17fff700 1 --2- 172.21.15.55:0/1087796081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d18159a60 0x7f1d180c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.609+0000 7f1d17fff700 1 -- 172.21.15.55:0/1087796081 >> v2:172.21.15.163:3300/0 conn(0x7f1d180ccc00 msgr2=0x7f1d18069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.609+0000 7f1d17fff700 1 --2- 172.21.15.55:0/1087796081 >> v2:172.21.15.163:3300/0 conn(0x7f1d180ccc00 0x7f1d18069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.609+0000 7f1d17fff700 1 -- 172.21.15.55:0/1087796081 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d180ca470 con 0x7f1d1806a020 2023-10-06T03:49:00.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.610+0000 7f1d17fff700 1 --2- 172.21.15.55:0/1087796081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d1806a020 0x7f1d180cb230 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1d1806cd90 tx=0x7f1d10009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0e7d92f275f5a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:00.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.610+0000 7f1d167fc700 1 -- 172.21.15.55:0/1087796081 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1000e070 con 0x7f1d1806a020 2023-10-06T03:49:00.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.610+0000 7f1d167fc700 1 -- 172.21.15.55:0/1087796081 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1000ba00 con 0x7f1d1806a020 2023-10-06T03:49:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.610+0000 7f1d167fc700 1 -- 172.21.15.55:0/1087796081 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d100092e0 con 0x7f1d1806a020 2023-10-06T03:49:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.610+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/1087796081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d1806a020 msgr2=0x7f1d180cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.610+0000 7f1d1cb9d700 1 --2- 172.21.15.55:0/1087796081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d1806a020 0x7f1d180cb230 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1d1806cd90 tx=0x7f1d10009840 comp rx=0 tx=0).stop 2023-10-06T03:49:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.610+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/1087796081 shutdown_connections 2023-10-06T03:49:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.610+0000 7f1d1cb9d700 1 --2- 172.21.15.55:0/1087796081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d1806a020 0x7f1d180cb230 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.610+0000 7f1d1cb9d700 1 --2- 172.21.15.55:0/1087796081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d18159a60 0x7f1d180c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.610+0000 7f1d1cb9d700 1 --2- 172.21.15.55:0/1087796081 >> v2:172.21.15.163:3300/0 conn(0x7f1d180ccc00 0x7f1d18069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.610+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/1087796081 >> 172.21.15.55:0/1087796081 conn(0x7f1d1806baa0 msgr2=0x7f1d1806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.611+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/1087796081 shutdown_connections 2023-10-06T03:49:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.611+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/1087796081 wait complete. 2023-10-06T03:49:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.612+0000 7f1d1cb9d700 1 Processor -- start 2023-10-06T03:49:00.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.612+0000 7f1d1cb9d700 1 -- start start 2023-10-06T03:49:00.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.613+0000 7f1d1cb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d1806a020 0x7f1d180b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:00.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.613+0000 7f1d1cb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d180ccc00 0x7f1d18155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:00.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.613+0000 7f1d1cb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d18159a60 0x7f1d180a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:00.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.613+0000 7f1d1cb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d180b1430 con 0x7f1d180ccc00 2023-10-06T03:49:00.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.613+0000 7f1d1cb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1816a1b0 con 0x7f1d1806a020 2023-10-06T03:49:00.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.613+0000 7f1d1cb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1816a490 con 0x7f1d18159a60 2023-10-06T03:49:00.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.613+0000 7f1d16ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d180ccc00 0x7f1d18155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:00.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.613+0000 7f1d16ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d180ccc00 0x7f1d18155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41386/0 (socket says 172.21.15.55:41386) 2023-10-06T03:49:00.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.613+0000 7f1d16ffd700 1 -- 172.21.15.55:0/120640975 learned_addr learned my addr 172.21.15.55:0/120640975 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:00.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.613+0000 7f1d177fe700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d1806a020 0x7f1d180b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:00.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.613+0000 7f1d17fff700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d18159a60 0x7f1d180a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:00.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.614+0000 7f1d16ffd700 1 -- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d18159a60 msgr2=0x7f1d180a9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:00.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.614+0000 7f1d16ffd700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d18159a60 0x7f1d180a9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.614+0000 7f1d16ffd700 1 -- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d1806a020 msgr2=0x7f1d180b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:00.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.614+0000 7f1d16ffd700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d1806a020 0x7f1d180b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.614+0000 7f1d16ffd700 1 -- 172.21.15.55:0/120640975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d10009ae0 con 0x7f1d180ccc00 2023-10-06T03:49:00.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.614+0000 7f1d17fff700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d18159a60 0x7f1d180a9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:00.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.614+0000 7f1d16ffd700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d180ccc00 0x7f1d18155450 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00ec80 tx=0x7f1d0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:00.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.614+0000 7f1cff7fe700 1 -- 172.21.15.55:0/120640975 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c00c850 con 0x7f1d180ccc00 2023-10-06T03:49:00.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.614+0000 7f1cff7fe700 1 -- 172.21.15.55:0/120640975 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c00c9b0 con 0x7f1d180ccc00 2023-10-06T03:49:00.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.614+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/120640975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1816a770 con 0x7f1d180ccc00 2023-10-06T03:49:00.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.614+0000 7f1cff7fe700 1 -- 172.21.15.55:0/120640975 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c00cd50 con 0x7f1d180ccc00 2023-10-06T03:49:00.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.614+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/120640975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d180b2de0 con 0x7f1d180ccc00 2023-10-06T03:49:00.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.615+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/120640975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1806eba0 con 0x7f1d180ccc00 2023-10-06T03:49:00.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.616+0000 7f1cff7fe700 1 -- 172.21.15.55:0/120640975 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d0c013680 con 0x7f1d180ccc00 2023-10-06T03:49:00.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.617+0000 7f1cff7fe700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1d04066f30 0x7f1d040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:00.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.617+0000 7f1cff7fe700 1 -- 172.21.15.55:0/120640975 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f1d0c084800 con 0x7f1d180ccc00 2023-10-06T03:49:00.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.618+0000 7f1d177fe700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1d04066f30 0x7f1d040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:00.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.618+0000 7f1d177fe700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1d04066f30 0x7f1d040693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1d180c83f0 tx=0x7f1d080091a0 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:00.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.621+0000 7f1cff7fe700 1 -- 172.21.15.55:0/120640975 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d0c051a40 con 0x7f1d180ccc00 2023-10-06T03:49:00.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.825+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/120640975 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d180ca750 con 0x7f1d180ccc00 2023-10-06T03:49:00.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.826+0000 7f1cff7fe700 1 -- 172.21.15.55:0/120640975 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d0c055060 con 0x7f1d180ccc00 2023-10-06T03:49:00.827 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:49:00.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.828+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1d04066f30 msgr2=0x7f1d040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:00.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.828+0000 7f1d1cb9d700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1d04066f30 0x7f1d040693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1d180c83f0 tx=0x7f1d080091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.828+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/120640975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d180ccc00 msgr2=0x7f1d18155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.828+0000 7f1d1cb9d700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d180ccc00 0x7f1d18155450 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c00ec80 tx=0x7f1d0c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.828+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/120640975 shutdown_connections 2023-10-06T03:49:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.828+0000 7f1d1cb9d700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1d04066f30 0x7f1d040693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.828+0000 7f1d1cb9d700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d18159a60 0x7f1d180a9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.828+0000 7f1d1cb9d700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d1806a020 0x7f1d180b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.828+0000 7f1d1cb9d700 1 --2- 172.21.15.55:0/120640975 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d180ccc00 0x7f1d18155450 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.828+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/120640975 >> 172.21.15.55:0/120640975 conn(0x7f1d1806baa0 msgr2=0x7f1d18158c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.828+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/120640975 shutdown_connections 2023-10-06T03:49:00.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:00.828+0000 7f1d1cb9d700 1 -- 172.21.15.55:0/120640975 wait complete. 2023-10-06T03:49:00.847 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:49:01.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.031+0000 7f50ccb9d700 1 Processor -- start 2023-10-06T03:49:01.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.031+0000 7f50ccb9d700 1 -- start start 2023-10-06T03:49:01.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.033+0000 7f50ccb9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f50c8156310 0x7f50c80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.033+0000 7f50ccb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50c80ae710 0x7f50c80c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.033+0000 7f50ccb9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c80ca860 con 0x7f50c81566e0 2023-10-06T03:49:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.033+0000 7f50ccb9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c80cab40 con 0x7f50c8156310 2023-10-06T03:49:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.033+0000 7f50ccb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80cae20 con 0x7f50c80ae710 2023-10-06T03:49:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.033+0000 7f50c77fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f50c8156310 0x7f50c80c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.033+0000 7f50c7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50c80ae710 0x7f50c80c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:01.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.033+0000 7f50c77fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f50c8156310 0x7f50c80c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:60694/0 (socket says 172.21.15.55:60694) 2023-10-06T03:49:01.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.033+0000 7f50c77fe700 1 -- 172.21.15.55:0/1758625312 learned_addr learned my addr 172.21.15.55:0/1758625312 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:01.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.034+0000 7f50c77fe700 1 -- 172.21.15.55:0/1758625312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50c80ae710 msgr2=0x7f50c80c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:01.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.034+0000 7f50c77fe700 1 --2- 172.21.15.55:0/1758625312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50c80ae710 0x7f50c80c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:01.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.034+0000 7f50c77fe700 1 -- 172.21.15.55:0/1758625312 >> v1:172.21.15.163:6789/0 conn(0x7f50c81566e0 legacy=0x7f50c80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:01.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.034+0000 7f50c77fe700 1 -- 172.21.15.55:0/1758625312 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c80cb100 con 0x7f50c8156310 2023-10-06T03:49:01.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.034+0000 7f50c7fff700 1 --2- 172.21.15.55:0/1758625312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50c80ae710 0x7f50c80c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:01.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.034+0000 7f50c77fe700 1 --2- 172.21.15.55:0/1758625312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c8156310 0x7f50c80c9dc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f50b80099b0 tx=0x7f50b8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96a0f340e702f9d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:01.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.035+0000 7f50c67fc700 1 -- 172.21.15.55:0/1758625312 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8009ea0 con 0x7f50c8156310 2023-10-06T03:49:01.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.035+0000 7f50c67fc700 1 -- 172.21.15.55:0/1758625312 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b800b860 con 0x7f50c8156310 2023-10-06T03:49:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.035+0000 7f50c67fc700 1 -- 172.21.15.55:0/1758625312 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b800bc00 con 0x7f50c8156310 2023-10-06T03:49:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.035+0000 7f50ccb9d700 1 -- 172.21.15.55:0/1758625312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c8156310 msgr2=0x7f50c80c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.035+0000 7f50ccb9d700 1 --2- 172.21.15.55:0/1758625312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c8156310 0x7f50c80c9dc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f50b80099b0 tx=0x7f50b8009250 comp rx=0 tx=0).stop 2023-10-06T03:49:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.036+0000 7f50ccb9d700 1 -- 172.21.15.55:0/1758625312 shutdown_connections 2023-10-06T03:49:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.036+0000 7f50ccb9d700 1 --2- 172.21.15.55:0/1758625312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50c80ae710 0x7f50c80c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.036+0000 7f50ccb9d700 1 --2- 172.21.15.55:0/1758625312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c8156310 0x7f50c80c9dc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.036+0000 7f50ccb9d700 1 -- 172.21.15.55:0/1758625312 >> 172.21.15.55:0/1758625312 conn(0x7f50c80cc1f0 msgr2=0x7f50c80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.036+0000 7f50ccb9d700 1 -- 172.21.15.55:0/1758625312 shutdown_connections 2023-10-06T03:49:01.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.036+0000 7f50ccb9d700 1 -- 172.21.15.55:0/1758625312 wait complete. 2023-10-06T03:49:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.037+0000 7f50ccb9d700 1 Processor -- start 2023-10-06T03:49:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.037+0000 7f50ccb9d700 1 -- start start 2023-10-06T03:49:01.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.038+0000 7f50ccb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50c80ae710 0x7f50c80c1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.038+0000 7f50ccb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c8156310 0x7f50c814d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.038+0000 7f50ccb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50c81566e0 0x7f50c814dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.038+0000 7f50ccb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c8152e00 con 0x7f50c81566e0 2023-10-06T03:49:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.038+0000 7f50ccb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80c23f0 con 0x7f50c8156310 2023-10-06T03:49:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.038+0000 7f50ccb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c80c26d0 con 0x7f50c80ae710 2023-10-06T03:49:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.039+0000 7f50c7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50c81566e0 0x7f50c814dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:01.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.039+0000 7f50c77fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50c80ae710 0x7f50c80c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.039+0000 7f50c7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50c81566e0 0x7f50c814dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41396/0 (socket says 172.21.15.55:41396) 2023-10-06T03:49:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.039+0000 7f50c7fff700 1 -- 172.21.15.55:0/2110740619 learned_addr learned my addr 172.21.15.55:0/2110740619 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.039+0000 7f50c6ffd700 1 --2- 172.21.15.55:0/2110740619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c8156310 0x7f50c814d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.039+0000 7f50c7fff700 1 -- 172.21.15.55:0/2110740619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50c80ae710 msgr2=0x7f50c80c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.039+0000 7f50c7fff700 1 --2- 172.21.15.55:0/2110740619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50c80ae710 0x7f50c80c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.039+0000 7f50c7fff700 1 -- 172.21.15.55:0/2110740619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c8156310 msgr2=0x7f50c814d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.039+0000 7f50c7fff700 1 --2- 172.21.15.55:0/2110740619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c8156310 0x7f50c814d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:01.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.039+0000 7f50c7fff700 1 -- 172.21.15.55:0/2110740619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c0009a20 con 0x7f50c81566e0 2023-10-06T03:49:01.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.039+0000 7f50c7fff700 1 --2- 172.21.15.55:0/2110740619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50c81566e0 0x7f50c814dad0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f50c000ef90 tx=0x7f50c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:01.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.040+0000 7f50af7fe700 1 -- 172.21.15.55:0/2110740619 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c000cbd0 con 0x7f50c81566e0 2023-10-06T03:49:01.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.040+0000 7f50af7fe700 1 -- 172.21.15.55:0/2110740619 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c000f040 con 0x7f50c81566e0 2023-10-06T03:49:01.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.040+0000 7f50af7fe700 1 -- 172.21.15.55:0/2110740619 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0014480 con 0x7f50c81566e0 2023-10-06T03:49:01.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.040+0000 7f50ccb9d700 1 -- 172.21.15.55:0/2110740619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b80094f0 con 0x7f50c81566e0 2023-10-06T03:49:01.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.040+0000 7f50ccb9d700 1 -- 172.21.15.55:0/2110740619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c8148420 con 0x7f50c81566e0 2023-10-06T03:49:01.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.041+0000 7f50ccb9d700 1 -- 172.21.15.55:0/2110740619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c81528f0 con 0x7f50c81566e0 2023-10-06T03:49:01.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.042+0000 7f50af7fe700 1 -- 172.21.15.55:0/2110740619 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50c00148b0 con 0x7f50c81566e0 2023-10-06T03:49:01.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.042+0000 7f50af7fe700 1 --2- 172.21.15.55:0/2110740619 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f50b4066f30 0x7f50b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:01.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.042+0000 7f50af7fe700 1 -- 172.21.15.55:0/2110740619 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f50c00852e0 con 0x7f50c81566e0 2023-10-06T03:49:01.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.043+0000 7f50c77fe700 1 --2- 172.21.15.55:0/2110740619 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f50b4066f30 0x7f50b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:01.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.043+0000 7f50c77fe700 1 --2- 172.21.15.55:0/2110740619 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f50b4066f30 0x7f50b40693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f50b80099b0 tx=0x7f50b8017000 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:01.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.047+0000 7f50af7fe700 1 -- 172.21.15.55:0/2110740619 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50c81528f0 con 0x7f50c81566e0 2023-10-06T03:49:01.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.250+0000 7f50ccb9d700 1 -- 172.21.15.55:0/2110740619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f50c81528f0 con 0x7f50c81566e0 2023-10-06T03:49:01.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.251+0000 7f50af7fe700 1 -- 172.21.15.55:0/2110740619 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v89) v1 ==== 84+0+126 (secure 0 0 0) 0x7f50c81528f0 con 0x7f50c81566e0 2023-10-06T03:49:01.252 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:01.252 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:49:01.253 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:01.253 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2023-10-06T03:49:01.253 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": "rbd" 2023-10-06T03:49:01.253 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:01.253 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:01.253 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 2, 2023-10-06T03:49:01.253 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2023-10-06T03:49:01.253 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:01.253 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:49:01.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.253+0000 7f50ccb9d700 1 -- 172.21.15.55:0/2110740619 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f50b4066f30 msgr2=0x7f50b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:01.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.253+0000 7f50ccb9d700 1 --2- 172.21.15.55:0/2110740619 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f50b4066f30 0x7f50b40693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f50b80099b0 tx=0x7f50b8017000 comp rx=0 tx=0).stop 2023-10-06T03:49:01.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.253+0000 7f50ccb9d700 1 -- 172.21.15.55:0/2110740619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50c81566e0 msgr2=0x7f50c814dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:01.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.253+0000 7f50ccb9d700 1 --2- 172.21.15.55:0/2110740619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50c81566e0 0x7f50c814dad0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f50c000ef90 tx=0x7f50c000b040 comp rx=0 tx=0).stop 2023-10-06T03:49:01.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.253+0000 7f50ccb9d700 1 -- 172.21.15.55:0/2110740619 shutdown_connections 2023-10-06T03:49:01.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.253+0000 7f50ccb9d700 1 --2- 172.21.15.55:0/2110740619 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f50b4066f30 0x7f50b40693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:01.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.253+0000 7f50ccb9d700 1 --2- 172.21.15.55:0/2110740619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f50c81566e0 0x7f50c814dad0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:01.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.253+0000 7f50ccb9d700 1 --2- 172.21.15.55:0/2110740619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f50c80ae710 0x7f50c80c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:01.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.253+0000 7f50ccb9d700 1 --2- 172.21.15.55:0/2110740619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50c8156310 0x7f50c814d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:01.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.253+0000 7f50ccb9d700 1 -- 172.21.15.55:0/2110740619 >> 172.21.15.55:0/2110740619 conn(0x7f50c80cc1f0 msgr2=0x7f50c8150c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:01.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.253+0000 7f50ccb9d700 1 -- 172.21.15.55:0/2110740619 shutdown_connections 2023-10-06T03:49:01.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.253+0000 7f50ccb9d700 1 -- 172.21.15.55:0/2110740619 wait complete. 2023-10-06T03:49:01.271 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:49:01.272 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:49:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.463+0000 7fcb8f59d700 1 Processor -- start 2023-10-06T03:49:01.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.463+0000 7fcb8f59d700 1 -- start start 2023-10-06T03:49:01.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.464+0000 7fcb8f59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcb880b0790 0x7fcb880cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:01.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.464+0000 7fcb8f59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcb881591e0 0x7fcb88159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:01.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.464+0000 7fcb8f59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb880cfa00 con 0x7fcb880b03c0 2023-10-06T03:49:01.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.464+0000 7fcb8f59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb880cb580 con 0x7fcb881591e0 2023-10-06T03:49:01.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.464+0000 7fcb8f59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb880cf160 con 0x7fcb880b0790 2023-10-06T03:49:01.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.464+0000 7fcb8ed9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcb881591e0 0x7fcb88159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:01.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.464+0000 7fcb8ed9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcb881591e0 0x7fcb88159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41402/0 (socket says 172.21.15.55:41402) 2023-10-06T03:49:01.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.464+0000 7fcb8ed9c700 1 -- 172.21.15.55:0/159156289 learned_addr learned my addr 172.21.15.55:0/159156289 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:01.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.465+0000 7fcb8dd9a700 1 --2- 172.21.15.55:0/159156289 >> v2:172.21.15.163:3300/0 conn(0x7fcb880b0790 0x7fcb880cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:01.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.465+0000 7fcb8ed9c700 1 -- 172.21.15.55:0/159156289 >> v2:172.21.15.163:3300/0 conn(0x7fcb880b0790 msgr2=0x7fcb880cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:01.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.465+0000 7fcb8ed9c700 1 --2- 172.21.15.55:0/159156289 >> v2:172.21.15.163:3300/0 conn(0x7fcb880b0790 0x7fcb880cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:01.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.465+0000 7fcb8ed9c700 1 -- 172.21.15.55:0/159156289 >> v1:172.21.15.163:6789/0 conn(0x7fcb880b03c0 legacy=0x7fcb880b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:01.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.465+0000 7fcb8ed9c700 1 -- 172.21.15.55:0/159156289 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb880cf440 con 0x7fcb881591e0 2023-10-06T03:49:01.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.465+0000 7fcb8ed9c700 1 --2- 172.21.15.55:0/159156289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb881591e0 0x7fcb88159600 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009f20 tx=0x7fcb80009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82486636353d7eca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:01.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.465+0000 7fcb8d599700 1 -- 172.21.15.55:0/159156289 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8000e070 con 0x7fcb881591e0 2023-10-06T03:49:01.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.466+0000 7fcb8d599700 1 -- 172.21.15.55:0/159156289 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8000ba40 con 0x7fcb881591e0 2023-10-06T03:49:01.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.466+0000 7fcb8d599700 1 -- 172.21.15.55:0/159156289 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb800092e0 con 0x7fcb881591e0 2023-10-06T03:49:01.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.466+0000 7fcb8f59d700 1 -- 172.21.15.55:0/159156289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb881591e0 msgr2=0x7fcb88159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:01.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.466+0000 7fcb8f59d700 1 --2- 172.21.15.55:0/159156289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb881591e0 0x7fcb88159600 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009f20 tx=0x7fcb80009840 comp rx=0 tx=0).stop 2023-10-06T03:49:01.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.466+0000 7fcb8f59d700 1 -- 172.21.15.55:0/159156289 shutdown_connections 2023-10-06T03:49:01.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.466+0000 7fcb8f59d700 1 --2- 172.21.15.55:0/159156289 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb881591e0 0x7fcb88159600 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:01.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.466+0000 7fcb8f59d700 1 --2- 172.21.15.55:0/159156289 >> v2:172.21.15.163:3300/0 conn(0x7fcb880b0790 0x7fcb880cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:01.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.466+0000 7fcb8f59d700 1 -- 172.21.15.55:0/159156289 >> 172.21.15.55:0/159156289 conn(0x7fcb88063230 msgr2=0x7fcb88063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:01.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.466+0000 7fcb8f59d700 1 -- 172.21.15.55:0/159156289 shutdown_connections 2023-10-06T03:49:01.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.466+0000 7fcb8f59d700 1 -- 172.21.15.55:0/159156289 wait complete. 2023-10-06T03:49:01.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.468+0000 7fcb8f59d700 1 Processor -- start 2023-10-06T03:49:01.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.468+0000 7fcb8f59d700 1 -- start start 2023-10-06T03:49:01.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8f59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb880b03c0 0x7fcb8815a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:01.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8f59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb880b0790 0x7fcb88158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:01.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8f59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb881591e0 0x7fcb880c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:01.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8f59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880c52a0 con 0x7fcb880b03c0 2023-10-06T03:49:01.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8f59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8815cad0 con 0x7fcb880b0790 2023-10-06T03:49:01.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8f59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8815cd70 con 0x7fcb881591e0 2023-10-06T03:49:01.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8e59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb880b03c0 0x7fcb8815a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:01.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8e59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb880b03c0 0x7fcb8815a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41406/0 (socket says 172.21.15.55:41406) 2023-10-06T03:49:01.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8e59b700 1 -- 172.21.15.55:0/3211731219 learned_addr learned my addr 172.21.15.55:0/3211731219 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:01.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8dd9a700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb880b0790 0x7fcb88158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:01.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8ed9c700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb881591e0 0x7fcb880c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:01.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8e59b700 1 -- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb881591e0 msgr2=0x7fcb880c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:01.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8e59b700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb881591e0 0x7fcb880c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:01.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8e59b700 1 -- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb880b0790 msgr2=0x7fcb88158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:01.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8e59b700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb880b0790 0x7fcb88158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:01.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8e59b700 1 -- 172.21.15.55:0/3211731219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb80009ae0 con 0x7fcb880b03c0 2023-10-06T03:49:01.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.469+0000 7fcb8ed9c700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb881591e0 0x7fcb880c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:01.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.470+0000 7fcb8e59b700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb880b03c0 0x7fcb8815a510 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400c4e0 tx=0x7fcb8400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:01.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.470+0000 7fcb7e7fc700 1 -- 172.21.15.55:0/3211731219 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84011590 con 0x7fcb880b03c0 2023-10-06T03:49:01.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.470+0000 7fcb7e7fc700 1 -- 172.21.15.55:0/3211731219 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb84011710 con 0x7fcb880b03c0 2023-10-06T03:49:01.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.470+0000 7fcb7e7fc700 1 -- 172.21.15.55:0/3211731219 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84011ad0 con 0x7fcb880b03c0 2023-10-06T03:49:01.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.470+0000 7fcb8f59d700 1 -- 172.21.15.55:0/3211731219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8815d050 con 0x7fcb880b03c0 2023-10-06T03:49:01.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.470+0000 7fcb8f59d700 1 -- 172.21.15.55:0/3211731219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8815d330 con 0x7fcb880b03c0 2023-10-06T03:49:01.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.471+0000 7fcb8f59d700 1 -- 172.21.15.55:0/3211731219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8804c3c0 con 0x7fcb880b03c0 2023-10-06T03:49:01.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.472+0000 7fcb7e7fc700 1 -- 172.21.15.55:0/3211731219 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb84001020 con 0x7fcb880b03c0 2023-10-06T03:49:01.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.472+0000 7fcb7e7fc700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcb6c066f30 0x7fcb6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:01.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.473+0000 7fcb7e7fc700 1 -- 172.21.15.55:0/3211731219 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fcb840839b0 con 0x7fcb880b03c0 2023-10-06T03:49:01.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.473+0000 7fcb8dd9a700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcb6c066f30 0x7fcb6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:01.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.474+0000 7fcb8dd9a700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcb6c066f30 0x7fcb6c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb880b0d30 tx=0x7fcb78008040 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:01.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.477+0000 7fcb7e7fc700 1 -- 172.21.15.55:0/3211731219 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb84050bf0 con 0x7fcb880b03c0 2023-10-06T03:49:01.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:01.680+0000 7fcb8f59d700 1 -- 172.21.15.55:0/3211731219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcb880cad50 con 0x7fcb880b03c0 2023-10-06T03:49:02.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.041+0000 7fcb7e7fc700 1 -- 172.21.15.55:0/3211731219 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v90) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcb84017070 con 0x7fcb880b03c0 2023-10-06T03:49:02.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.045+0000 7fcb8f59d700 1 -- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcb6c066f30 msgr2=0x7fcb6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:02.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.045+0000 7fcb8f59d700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcb6c066f30 0x7fcb6c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb880b0d30 tx=0x7fcb78008040 comp rx=0 tx=0).stop 2023-10-06T03:49:02.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.045+0000 7fcb8f59d700 1 -- 172.21.15.55:0/3211731219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb880b03c0 msgr2=0x7fcb8815a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:02.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.045+0000 7fcb8f59d700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb880b03c0 0x7fcb8815a510 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400c4e0 tx=0x7fcb8400c8a0 comp rx=0 tx=0).stop 2023-10-06T03:49:02.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.045+0000 7fcb8f59d700 1 -- 172.21.15.55:0/3211731219 shutdown_connections 2023-10-06T03:49:02.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.045+0000 7fcb8f59d700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcb6c066f30 0x7fcb6c0693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:02.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.045+0000 7fcb8f59d700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb881591e0 0x7fcb880c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:02.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.045+0000 7fcb8f59d700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb880b03c0 0x7fcb8815a510 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:02.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.045+0000 7fcb8f59d700 1 --2- 172.21.15.55:0/3211731219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb880b0790 0x7fcb88158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:02.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.045+0000 7fcb8f59d700 1 -- 172.21.15.55:0/3211731219 >> 172.21.15.55:0/3211731219 conn(0x7fcb88063230 msgr2=0x7fcb880d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:02.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.045+0000 7fcb8f59d700 1 -- 172.21.15.55:0/3211731219 shutdown_connections 2023-10-06T03:49:02.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.045+0000 7fcb8f59d700 1 -- 172.21.15.55:0/3211731219 wait complete. 2023-10-06T03:49:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2023-10-06T03:49:02.067 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:49:02.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.260+0000 7ff7b699d700 1 Processor -- start 2023-10-06T03:49:02.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.260+0000 7ff7b699d700 1 -- start start 2023-10-06T03:49:02.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.262+0000 7ff7b699d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff7b00caee0 0x7ff7b00ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:02.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.262+0000 7ff7b699d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b0156780 0x7ff7b00ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:02.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.262+0000 7ff7b699d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b00cbe50 con 0x7ff7b01563b0 2023-10-06T03:49:02.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.262+0000 7ff7b699d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b00cb370 con 0x7ff7b00caee0 2023-10-06T03:49:02.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.262+0000 7ff7b699d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b00cb650 con 0x7ff7b0156780 2023-10-06T03:49:02.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.262+0000 7ff7b519a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff7b00caee0 0x7ff7b00ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:02.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.262+0000 7ff7b519a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff7b00caee0 0x7ff7b00ae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41408/0 (socket says 172.21.15.55:41408) 2023-10-06T03:49:02.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.262+0000 7ff7b519a700 1 -- 172.21.15.55:0/420572751 learned_addr learned my addr 172.21.15.55:0/420572751 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:02.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.262+0000 7ff7b619c700 1 --2- 172.21.15.55:0/420572751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b0156780 0x7ff7b00ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:02.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.262+0000 7ff7b519a700 1 -- 172.21.15.55:0/420572751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b0156780 msgr2=0x7ff7b00ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:02.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.262+0000 7ff7b519a700 1 --2- 172.21.15.55:0/420572751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b0156780 0x7ff7b00ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:02.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.262+0000 7ff7b519a700 1 -- 172.21.15.55:0/420572751 >> v1:172.21.15.163:6789/0 conn(0x7ff7b01563b0 legacy=0x7ff7b00ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:02.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.263+0000 7ff7b519a700 1 -- 172.21.15.55:0/420572751 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b00c6860 con 0x7ff7b00caee0 2023-10-06T03:49:02.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.263+0000 7ff7b619c700 1 --2- 172.21.15.55:0/420572751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b0156780 0x7ff7b00ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:02.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.263+0000 7ff7b519a700 1 --2- 172.21.15.55:0/420572751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b00caee0 0x7ff7b00ae2b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00099b0 tx=0x7ff7a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33994cac5cba8b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:02.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.263+0000 7ff7b4999700 1 -- 172.21.15.55:0/420572751 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000b860 con 0x7ff7b00caee0 2023-10-06T03:49:02.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.263+0000 7ff7b4999700 1 -- 172.21.15.55:0/420572751 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a000b9c0 con 0x7ff7b00caee0 2023-10-06T03:49:02.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.263+0000 7ff7b4999700 1 -- 172.21.15.55:0/420572751 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000bd60 con 0x7ff7b00caee0 2023-10-06T03:49:02.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.264+0000 7ff7b699d700 1 -- 172.21.15.55:0/420572751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b00caee0 msgr2=0x7ff7b00ae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:02.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.264+0000 7ff7b699d700 1 --2- 172.21.15.55:0/420572751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b00caee0 0x7ff7b00ae2b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00099b0 tx=0x7ff7a0009250 comp rx=0 tx=0).stop 2023-10-06T03:49:02.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.264+0000 7ff7b699d700 1 -- 172.21.15.55:0/420572751 shutdown_connections 2023-10-06T03:49:02.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.264+0000 7ff7b699d700 1 --2- 172.21.15.55:0/420572751 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b0156780 0x7ff7b00ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:02.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.264+0000 7ff7b699d700 1 --2- 172.21.15.55:0/420572751 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b00caee0 0x7ff7b00ae2b0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:02.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.264+0000 7ff7b699d700 1 -- 172.21.15.55:0/420572751 >> 172.21.15.55:0/420572751 conn(0x7ff7b0068e20 msgr2=0x7ff7b00cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:02.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.264+0000 7ff7b699d700 1 -- 172.21.15.55:0/420572751 shutdown_connections 2023-10-06T03:49:02.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.264+0000 7ff7b699d700 1 -- 172.21.15.55:0/420572751 wait complete. 2023-10-06T03:49:02.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.265+0000 7ff7b699d700 1 Processor -- start 2023-10-06T03:49:02.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.265+0000 7ff7b699d700 1 -- start start 2023-10-06T03:49:02.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.266+0000 7ff7b699d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b00caee0 0x7ff7b016d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:02.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.266+0000 7ff7b699d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b01563b0 0x7ff7b014db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:02.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.266+0000 7ff7b699d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b0156780 0x7ff7b014e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:02.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.266+0000 7ff7b699d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b014b350 con 0x7ff7b00caee0 2023-10-06T03:49:02.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.266+0000 7ff7b699d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b014bed0 con 0x7ff7b0156780 2023-10-06T03:49:02.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.266+0000 7ff7b699d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b014b5a0 con 0x7ff7b01563b0 2023-10-06T03:49:02.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.267+0000 7ff7b599b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b00caee0 0x7ff7b016d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:02.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.267+0000 7ff7b599b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b00caee0 0x7ff7b016d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41420/0 (socket says 172.21.15.55:41420) 2023-10-06T03:49:02.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.267+0000 7ff7b599b700 1 -- 172.21.15.55:0/2679586881 learned_addr learned my addr 172.21.15.55:0/2679586881 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:02.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.267+0000 7ff7b519a700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b01563b0 0x7ff7b014db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:02.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.267+0000 7ff7b619c700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b0156780 0x7ff7b014e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:02.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.267+0000 7ff7b599b700 1 -- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b01563b0 msgr2=0x7ff7b014db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:02.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.267+0000 7ff7b599b700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b01563b0 0x7ff7b014db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:02.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.267+0000 7ff7b599b700 1 -- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b0156780 msgr2=0x7ff7b014e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:02.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.267+0000 7ff7b599b700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b0156780 0x7ff7b014e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:02.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.267+0000 7ff7b599b700 1 -- 172.21.15.55:0/2679586881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a00094f0 con 0x7ff7b00caee0 2023-10-06T03:49:02.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.267+0000 7ff7b619c700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b0156780 0x7ff7b014e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:02.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.267+0000 7ff7b599b700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b00caee0 0x7ff7b016d720 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00c5c0 tx=0x7ff7ac00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:02.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.268+0000 7ff7a5ffb700 1 -- 172.21.15.55:0/2679586881 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac011590 con 0x7ff7b00caee0 2023-10-06T03:49:02.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.268+0000 7ff7a5ffb700 1 -- 172.21.15.55:0/2679586881 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac011710 con 0x7ff7b00caee0 2023-10-06T03:49:02.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.268+0000 7ff7b699d700 1 -- 172.21.15.55:0/2679586881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b014b880 con 0x7ff7b00caee0 2023-10-06T03:49:02.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.268+0000 7ff7a5ffb700 1 -- 172.21.15.55:0/2679586881 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac011ad0 con 0x7ff7b00caee0 2023-10-06T03:49:02.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.268+0000 7ff7b699d700 1 -- 172.21.15.55:0/2679586881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b016dc60 con 0x7ff7b00caee0 2023-10-06T03:49:02.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.269+0000 7ff7b699d700 1 -- 172.21.15.55:0/2679586881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b004c3c0 con 0x7ff7b00caee0 2023-10-06T03:49:02.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.270+0000 7ff7a5ffb700 1 -- 172.21.15.55:0/2679586881 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7ac01b3b0 con 0x7ff7b00caee0 2023-10-06T03:49:02.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.271+0000 7ff7a5ffb700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff794066f30 0x7ff7940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:02.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.271+0000 7ff7a5ffb700 1 -- 172.21.15.55:0/2679586881 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5968+0+0 (secure 0 0 0) 0x7ff7ac084080 con 0x7ff7b00caee0 2023-10-06T03:49:02.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.272+0000 7ff7b519a700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff794066f30 0x7ff7940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:02.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.272+0000 7ff7b519a700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff794066f30 0x7ff7940693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00099b0 tx=0x7ff7a0017000 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:02.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.275+0000 7ff7a5ffb700 1 -- 172.21.15.55:0/2679586881 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7ac0511f0 con 0x7ff7b00caee0 2023-10-06T03:49:02.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:02.479+0000 7ff7b699d700 1 -- 172.21.15.55:0/2679586881 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff7b00c6b40 con 0x7ff7b00caee0 2023-10-06T03:49:03.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.051+0000 7ff7a5ffb700 1 -- 172.21.15.55:0/2679586881 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v91) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff7ac017020 con 0x7ff7b00caee0 2023-10-06T03:49:03.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.059+0000 7ff7b699d700 1 -- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff794066f30 msgr2=0x7ff7940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:03.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.059+0000 7ff7b699d700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff794066f30 0x7ff7940693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff7a00099b0 tx=0x7ff7a0017000 comp rx=0 tx=0).stop 2023-10-06T03:49:03.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.059+0000 7ff7b699d700 1 -- 172.21.15.55:0/2679586881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b00caee0 msgr2=0x7ff7b016d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:03.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.059+0000 7ff7b699d700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b00caee0 0x7ff7b016d720 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00c5c0 tx=0x7ff7ac00c980 comp rx=0 tx=0).stop 2023-10-06T03:49:03.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.060+0000 7ff7b699d700 1 -- 172.21.15.55:0/2679586881 shutdown_connections 2023-10-06T03:49:03.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.060+0000 7ff7b699d700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff794066f30 0x7ff7940693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:03.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.060+0000 7ff7b699d700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b0156780 0x7ff7b014e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:03.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.060+0000 7ff7b699d700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b00caee0 0x7ff7b016d720 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:03.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.060+0000 7ff7b699d700 1 --2- 172.21.15.55:0/2679586881 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b01563b0 0x7ff7b014db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:03.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.060+0000 7ff7b699d700 1 -- 172.21.15.55:0/2679586881 >> 172.21.15.55:0/2679586881 conn(0x7ff7b0068e20 msgr2=0x7ff7b00c80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:03.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.062+0000 7ff7b699d700 1 -- 172.21.15.55:0/2679586881 shutdown_connections 2023-10-06T03:49:03.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.066+0000 7ff7b699d700 1 -- 172.21.15.55:0/2679586881 wait complete. 2023-10-06T03:49:03.075 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2023-10-06T03:49:03.094 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:49:03.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.285+0000 7faf5bfff700 1 Processor -- start 2023-10-06T03:49:03.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.285+0000 7faf5bfff700 1 -- start start 2023-10-06T03:49:03.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.286+0000 7faf5bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faf5c0cb1a0 0x7faf5c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:03.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.286+0000 7faf5bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faf5c0ae0d0 0x7faf5c0ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:03.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.286+0000 7faf5bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5c06ebd0 con 0x7faf5c15a8a0 2023-10-06T03:49:03.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.286+0000 7faf5bfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c15b2e0 con 0x7faf5c0ae0d0 2023-10-06T03:49:03.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.286+0000 7faf5bfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c0aea10 con 0x7faf5c0cb1a0 2023-10-06T03:49:03.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.287+0000 7faf5a7fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7faf5c15a8a0 0x7faf5c06f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57418/0 (socket says 172.21.15.55:57418) 2023-10-06T03:49:03.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.287+0000 7faf5a7fc700 1 -- 172.21.15.55:0/1090222312 learned_addr learned my addr 172.21.15.55:0/1090222312 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:03.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.287+0000 7faf5b7fe700 1 --2- 172.21.15.55:0/1090222312 >> v2:172.21.15.55:3300/0 conn(0x7faf5c0ae0d0 0x7faf5c0ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:03.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.287+0000 7faf5affd700 1 --2- 172.21.15.55:0/1090222312 >> v2:172.21.15.163:3300/0 conn(0x7faf5c0cb1a0 0x7faf5c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:03.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.287+0000 7faf59ffb700 1 -- 172.21.15.55:0/1090222312 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870209408 0 0) 0x7faf5c06ebd0 con 0x7faf5c15a8a0 2023-10-06T03:49:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.287+0000 7faf59ffb700 1 -- 172.21.15.55:0/1090222312 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf44003570 con 0x7faf5c15a8a0 2023-10-06T03:49:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.287+0000 7faf5b7fe700 1 -- 172.21.15.55:0/1090222312 >> v2:172.21.15.163:3300/0 conn(0x7faf5c0cb1a0 msgr2=0x7faf5c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.287+0000 7faf5b7fe700 1 --2- 172.21.15.55:0/1090222312 >> v2:172.21.15.163:3300/0 conn(0x7faf5c0cb1a0 0x7faf5c15a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.287+0000 7faf5b7fe700 1 -- 172.21.15.55:0/1090222312 >> v1:172.21.15.55:6789/0 conn(0x7faf5c15a8a0 legacy=0x7faf5c06f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.287+0000 7faf5b7fe700 1 -- 172.21.15.55:0/1090222312 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c15ad20 con 0x7faf5c0ae0d0 2023-10-06T03:49:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.288+0000 7faf5b7fe700 1 --2- 172.21.15.55:0/1090222312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf5c0ae0d0 0x7faf5c0ae4d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0cce10 tx=0x7faf54009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2764481233deb6e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.288+0000 7faf59ffb700 1 -- 172.21.15.55:0/1090222312 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5400e070 con 0x7faf5c0ae0d0 2023-10-06T03:49:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.288+0000 7faf59ffb700 1 -- 172.21.15.55:0/1090222312 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5400bb40 con 0x7faf5c0ae0d0 2023-10-06T03:49:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.288+0000 7faf59ffb700 1 -- 172.21.15.55:0/1090222312 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54012430 con 0x7faf5c0ae0d0 2023-10-06T03:49:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.288+0000 7faf5bfff700 1 -- 172.21.15.55:0/1090222312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf5c0ae0d0 msgr2=0x7faf5c0ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:03.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.288+0000 7faf5bfff700 1 --2- 172.21.15.55:0/1090222312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf5c0ae0d0 0x7faf5c0ae4d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0cce10 tx=0x7faf54009780 comp rx=0 tx=0).stop 2023-10-06T03:49:03.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.289+0000 7faf5bfff700 1 -- 172.21.15.55:0/1090222312 shutdown_connections 2023-10-06T03:49:03.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.289+0000 7faf5bfff700 1 --2- 172.21.15.55:0/1090222312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf5c0ae0d0 0x7faf5c0ae4d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:03.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.289+0000 7faf5bfff700 1 --2- 172.21.15.55:0/1090222312 >> v2:172.21.15.163:3300/0 conn(0x7faf5c0cb1a0 0x7faf5c15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:03.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.289+0000 7faf5bfff700 1 -- 172.21.15.55:0/1090222312 >> 172.21.15.55:0/1090222312 conn(0x7faf5c0cc1e0 msgr2=0x7faf5c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:03.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.289+0000 7faf5bfff700 1 -- 172.21.15.55:0/1090222312 shutdown_connections 2023-10-06T03:49:03.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.289+0000 7faf5bfff700 1 -- 172.21.15.55:0/1090222312 wait complete. 2023-10-06T03:49:03.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.290+0000 7faf5bfff700 1 Processor -- start 2023-10-06T03:49:03.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.290+0000 7faf5bfff700 1 -- start start 2023-10-06T03:49:03.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.291+0000 7faf5bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf5c0ae0d0 0x7faf5c0c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:03.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.291+0000 7faf5bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf5c0cb1a0 0x7faf5c0c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:03.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.291+0000 7faf5bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf5c0c7e40 0x7faf5c0c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:03.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.291+0000 7faf5bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0c4090 con 0x7faf5c0cb1a0 2023-10-06T03:49:03.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.291+0000 7faf5bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0c5970 con 0x7faf5c0c7e40 2023-10-06T03:49:03.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.291+0000 7faf5bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0c6de0 con 0x7faf5c0ae0d0 2023-10-06T03:49:03.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.291+0000 7faf5a7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf5c0cb1a0 0x7faf5c0c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:03.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.292+0000 7faf5a7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf5c0cb1a0 0x7faf5c0c3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49900/0 (socket says 172.21.15.55:49900) 2023-10-06T03:49:03.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.292+0000 7faf5a7fc700 1 -- 172.21.15.55:0/3365898694 learned_addr learned my addr 172.21.15.55:0/3365898694 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:03.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.292+0000 7faf5b7fe700 1 --2- 172.21.15.55:0/3365898694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf5c0c7e40 0x7faf5c0c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:03.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.292+0000 7faf5affd700 1 --2- 172.21.15.55:0/3365898694 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf5c0ae0d0 0x7faf5c0c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:03.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.292+0000 7faf5a7fc700 1 -- 172.21.15.55:0/3365898694 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf5c0ae0d0 msgr2=0x7faf5c0c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:03.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.292+0000 7faf5a7fc700 1 --2- 172.21.15.55:0/3365898694 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf5c0ae0d0 0x7faf5c0c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:03.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.292+0000 7faf5a7fc700 1 -- 172.21.15.55:0/3365898694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf5c0c7e40 msgr2=0x7faf5c0c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:03.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.292+0000 7faf5a7fc700 1 --2- 172.21.15.55:0/3365898694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf5c0c7e40 0x7faf5c0c37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:03.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.292+0000 7faf5a7fc700 1 -- 172.21.15.55:0/3365898694 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf54009a20 con 0x7faf5c0cb1a0 2023-10-06T03:49:03.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.292+0000 7faf5a7fc700 1 --2- 172.21.15.55:0/3365898694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf5c0cb1a0 0x7faf5c0c3290 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7faf5000c590 tx=0x7faf50007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:03.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.292+0000 7faf42ffd700 1 -- 172.21.15.55:0/3365898694 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50005b30 con 0x7faf5c0cb1a0 2023-10-06T03:49:03.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.292+0000 7faf42ffd700 1 -- 172.21.15.55:0/3365898694 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf50005cb0 con 0x7faf5c0cb1a0 2023-10-06T03:49:03.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.292+0000 7faf42ffd700 1 -- 172.21.15.55:0/3365898694 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50007320 con 0x7faf5c0cb1a0 2023-10-06T03:49:03.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.293+0000 7faf5bfff700 1 -- 172.21.15.55:0/3365898694 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c0c7080 con 0x7faf5c0cb1a0 2023-10-06T03:49:03.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.293+0000 7faf5bfff700 1 -- 172.21.15.55:0/3365898694 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5c0c7640 con 0x7faf5c0cb1a0 2023-10-06T03:49:03.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.294+0000 7faf5bfff700 1 -- 172.21.15.55:0/3365898694 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5c06c450 con 0x7faf5c0cb1a0 2023-10-06T03:49:03.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.295+0000 7faf42ffd700 1 -- 172.21.15.55:0/3365898694 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faf5001b3b0 con 0x7faf5c0cb1a0 2023-10-06T03:49:03.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.296+0000 7faf42ffd700 1 --2- 172.21.15.55:0/3365898694 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7faf48066f30 0x7faf480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:03.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.296+0000 7faf42ffd700 1 -- 172.21.15.55:0/3365898694 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6350+0+0 (secure 0 0 0) 0x7faf50084080 con 0x7faf5c0cb1a0 2023-10-06T03:49:03.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.297+0000 7faf5affd700 1 --2- 172.21.15.55:0/3365898694 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7faf48066f30 0x7faf480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:03.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.297+0000 7faf5affd700 1 --2- 172.21.15.55:0/3365898694 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7faf48066f30 0x7faf480693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0c1b10 tx=0x7faf4c0091a0 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:03.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.299+0000 7faf42ffd700 1 -- 172.21.15.55:0/3365898694 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf50051050 con 0x7faf5c0cb1a0 2023-10-06T03:49:03.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:03.546+0000 7faf5bfff700 1 -- 172.21.15.55:0/3365898694 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7faf5c15b070 con 0x7faf5c0cb1a0 2023-10-06T03:49:04.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.062+0000 7faf42ffd700 1 -- 172.21.15.55:0/3365898694 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v40) v1 ==== 174+0+0 (secure 0 0 0) 0x7faf500170c0 con 0x7faf5c0cb1a0 2023-10-06T03:49:04.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.077+0000 7faf5bfff700 1 -- 172.21.15.55:0/3365898694 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7faf48066f30 msgr2=0x7faf480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:04.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.077+0000 7faf5bfff700 1 --2- 172.21.15.55:0/3365898694 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7faf48066f30 0x7faf480693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0c1b10 tx=0x7faf4c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.077+0000 7faf5bfff700 1 -- 172.21.15.55:0/3365898694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf5c0cb1a0 msgr2=0x7faf5c0c3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:04.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.077+0000 7faf5bfff700 1 --2- 172.21.15.55:0/3365898694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf5c0cb1a0 0x7faf5c0c3290 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7faf5000c590 tx=0x7faf50007e00 comp rx=0 tx=0).stop 2023-10-06T03:49:04.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.078+0000 7faf5bfff700 1 -- 172.21.15.55:0/3365898694 shutdown_connections 2023-10-06T03:49:04.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.078+0000 7faf5bfff700 1 --2- 172.21.15.55:0/3365898694 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7faf48066f30 0x7faf480693e0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0c1b10 tx=0x7faf4c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.078+0000 7faf5bfff700 1 --2- 172.21.15.55:0/3365898694 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf5c0c7e40 0x7faf5c0c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.078+0000 7faf5bfff700 1 --2- 172.21.15.55:0/3365898694 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf5c0ae0d0 0x7faf5c0c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.078+0000 7faf5bfff700 1 --2- 172.21.15.55:0/3365898694 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf5c0cb1a0 0x7faf5c0c3290 secure :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7faf5000c590 tx=0x7faf50007e00 comp rx=0 tx=0).stop 2023-10-06T03:49:04.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.078+0000 7faf5bfff700 1 -- 172.21.15.55:0/3365898694 >> 172.21.15.55:0/3365898694 conn(0x7faf5c0cc1e0 msgr2=0x7faf5c0c2370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:04.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.082+0000 7faf5bfff700 1 -- 172.21.15.55:0/3365898694 shutdown_connections 2023-10-06T03:49:04.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.082+0000 7faf5bfff700 1 -- 172.21.15.55:0/3365898694 wait complete. 2023-10-06T03:49:04.087 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 11 and data pool 12 2023-10-06T03:49:04.105 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:49:04.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.292+0000 7f8a5f59e700 1 Processor -- start 2023-10-06T03:49:04.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.292+0000 7f8a5f59e700 1 -- start start 2023-10-06T03:49:04.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.294+0000 7f8a5f59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8a60157270 0x7f8a60068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:04.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.294+0000 7f8a5f59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a600688d0 0x7f8a600b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:04.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.294+0000 7f8a5f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a600b0a00 0x7f8a6006a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:04.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.294+0000 7f8a5f59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a60068d50 con 0x7f8a600688d0 2023-10-06T03:49:04.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.294+0000 7f8a5f59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6015a980 con 0x7f8a60157270 2023-10-06T03:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.294+0000 7f8a5f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6015a050 con 0x7f8a600b0a00 2023-10-06T03:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.294+0000 7f8a5dd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a600688d0 0x7f8a600b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.294+0000 7f8a5dd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a600688d0 0x7f8a600b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49908/0 (socket says 172.21.15.55:49908) 2023-10-06T03:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.294+0000 7f8a5dd9b700 1 -- 172.21.15.55:0/1613593 learned_addr learned my addr 172.21.15.55:0/1613593 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.294+0000 7f8a5ed9d700 1 --2- 172.21.15.55:0/1613593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a600b0a00 0x7f8a6006a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.294+0000 7f8a5e59c700 1 --2- 172.21.15.55:0/1613593 >> v2:172.21.15.163:3300/0 conn(0x7f8a60157270 0x7f8a60068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.295+0000 7f8a5dd9b700 1 -- 172.21.15.55:0/1613593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a600b0a00 msgr2=0x7f8a6006a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.295+0000 7f8a5dd9b700 1 --2- 172.21.15.55:0/1613593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a600b0a00 0x7f8a6006a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.295+0000 7f8a5dd9b700 1 -- 172.21.15.55:0/1613593 >> v2:172.21.15.163:3300/0 conn(0x7f8a60157270 msgr2=0x7f8a60068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.295+0000 7f8a5dd9b700 1 --2- 172.21.15.55:0/1613593 >> v2:172.21.15.163:3300/0 conn(0x7f8a60157270 0x7f8a60068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.295+0000 7f8a5dd9b700 1 -- 172.21.15.55:0/1613593 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6015a330 con 0x7f8a600688d0 2023-10-06T03:49:04.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.295+0000 7f8a5dd9b700 1 --2- 172.21.15.55:0/1613593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a600688d0 0x7f8a600b04c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8a600b3360 tx=0x7f8a54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c70bec84a621d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:04.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.295+0000 7f8a5d59a700 1 -- 172.21.15.55:0/1613593 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400b640 con 0x7f8a600688d0 2023-10-06T03:49:04.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.295+0000 7f8a5d59a700 1 -- 172.21.15.55:0/1613593 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5400b7a0 con 0x7f8a600688d0 2023-10-06T03:49:04.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.295+0000 7f8a5d59a700 1 -- 172.21.15.55:0/1613593 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400bb60 con 0x7f8a600688d0 2023-10-06T03:49:04.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.296+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1613593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a600688d0 msgr2=0x7f8a600b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:04.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.296+0000 7f8a5f59e700 1 --2- 172.21.15.55:0/1613593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a600688d0 0x7f8a600b04c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8a600b3360 tx=0x7f8a54009250 comp rx=0 tx=0).stop 2023-10-06T03:49:04.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.296+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1613593 shutdown_connections 2023-10-06T03:49:04.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.296+0000 7f8a5f59e700 1 --2- 172.21.15.55:0/1613593 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a600b0a00 0x7f8a6006a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.296+0000 7f8a5f59e700 1 --2- 172.21.15.55:0/1613593 >> v2:172.21.15.163:3300/0 conn(0x7f8a60157270 0x7f8a60068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.296+0000 7f8a5f59e700 1 --2- 172.21.15.55:0/1613593 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a600688d0 0x7f8a600b04c0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.296+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1613593 >> 172.21.15.55:0/1613593 conn(0x7f8a600cd310 msgr2=0x7f8a600cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:04.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.296+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1613593 shutdown_connections 2023-10-06T03:49:04.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.296+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1613593 wait complete. 2023-10-06T03:49:04.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.297+0000 7f8a5f59e700 1 Processor -- start 2023-10-06T03:49:04.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.297+0000 7f8a5f59e700 1 -- start start 2023-10-06T03:49:04.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.298+0000 7f8a5f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a600688d0 0x7f8a6016dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:04.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.298+0000 7f8a5f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a600b0a00 0x7f8a60151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:04.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.298+0000 7f8a5f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a60157270 0x7f8a60169b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:04.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.298+0000 7f8a5f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6016d840 con 0x7f8a600688d0 2023-10-06T03:49:04.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.298+0000 7f8a5f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6016aa10 con 0x7f8a600b0a00 2023-10-06T03:49:04.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.298+0000 7f8a5f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a600cc8a0 con 0x7f8a60157270 2023-10-06T03:49:04.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.299+0000 7f8a5e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a600688d0 0x7f8a6016dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:04.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.299+0000 7f8a5e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a600688d0 0x7f8a6016dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49916/0 (socket says 172.21.15.55:49916) 2023-10-06T03:49:04.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.299+0000 7f8a5e59c700 1 -- 172.21.15.55:0/1509791245 learned_addr learned my addr 172.21.15.55:0/1509791245 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:04.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.299+0000 7f8a5ed9d700 1 --2- 172.21.15.55:0/1509791245 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a60157270 0x7f8a60169b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:04.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.299+0000 7f8a5dd9b700 1 --2- 172.21.15.55:0/1509791245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a600b0a00 0x7f8a60151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:04.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.299+0000 7f8a5e59c700 1 -- 172.21.15.55:0/1509791245 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a60157270 msgr2=0x7f8a60169b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:04.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.299+0000 7f8a5e59c700 1 --2- 172.21.15.55:0/1509791245 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a60157270 0x7f8a60169b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.299+0000 7f8a5e59c700 1 -- 172.21.15.55:0/1509791245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a600b0a00 msgr2=0x7f8a60151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:04.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.299+0000 7f8a5e59c700 1 --2- 172.21.15.55:0/1509791245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a600b0a00 0x7f8a60151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.299+0000 7f8a5e59c700 1 -- 172.21.15.55:0/1509791245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a540094f0 con 0x7f8a600688d0 2023-10-06T03:49:04.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.299+0000 7f8a5e59c700 1 --2- 172.21.15.55:0/1509791245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a600688d0 0x7f8a6016dcd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000e930 tx=0x7f8a500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:04.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.299+0000 7f8a4e7fc700 1 -- 172.21.15.55:0/1509791245 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000eed0 con 0x7f8a600688d0 2023-10-06T03:49:04.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.300+0000 7f8a4e7fc700 1 -- 172.21.15.55:0/1509791245 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5000c850 con 0x7f8a600688d0 2023-10-06T03:49:04.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.300+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1509791245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6016a730 con 0x7f8a600688d0 2023-10-06T03:49:04.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.300+0000 7f8a4e7fc700 1 -- 172.21.15.55:0/1509791245 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000cbf0 con 0x7f8a600688d0 2023-10-06T03:49:04.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.300+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1509791245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6016d4d0 con 0x7f8a600688d0 2023-10-06T03:49:04.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.301+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1509791245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6004c3c0 con 0x7f8a600688d0 2023-10-06T03:49:04.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.302+0000 7f8a4e7fc700 1 -- 172.21.15.55:0/1509791245 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a50013470 con 0x7f8a600688d0 2023-10-06T03:49:04.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.303+0000 7f8a4e7fc700 1 --2- 172.21.15.55:0/1509791245 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a40066f30 0x7f8a400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:04.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.303+0000 7f8a4e7fc700 1 -- 172.21.15.55:0/1509791245 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f8a50084af0 con 0x7f8a600688d0 2023-10-06T03:49:04.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.303+0000 7f8a5dd9b700 1 --2- 172.21.15.55:0/1509791245 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a40066f30 0x7f8a400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:04.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.304+0000 7f8a5dd9b700 1 --2- 172.21.15.55:0/1509791245 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a40066f30 0x7f8a400693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8a600b3360 tx=0x7f8a54017000 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:04.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.307+0000 7f8a4e7fc700 1 -- 172.21.15.55:0/1509791245 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a500519d0 con 0x7f8a600688d0 2023-10-06T03:49:04.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.511+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1509791245 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a6015a610 con 0x7f8a600688d0 2023-10-06T03:49:04.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.512+0000 7f8a4e7fc700 1 -- 172.21.15.55:0/1509791245 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17525 (secure 0 0 0) 0x7f8a50054ff0 con 0x7f8a600688d0 2023-10-06T03:49:04.513 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:04.514 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":92,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:49:03.547288+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:49:01.721643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T03:49:02.520120+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:49:04.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.514+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1509791245 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a40066f30 msgr2=0x7f8a400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:04.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.514+0000 7f8a5f59e700 1 --2- 172.21.15.55:0/1509791245 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a40066f30 0x7f8a400693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8a600b3360 tx=0x7f8a54017000 comp rx=0 tx=0).stop 2023-10-06T03:49:04.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.514+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1509791245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a600688d0 msgr2=0x7f8a6016dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:04.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.514+0000 7f8a5f59e700 1 --2- 172.21.15.55:0/1509791245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a600688d0 0x7f8a6016dcd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000e930 tx=0x7f8a500091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.514+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1509791245 shutdown_connections 2023-10-06T03:49:04.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.514+0000 7f8a5f59e700 1 --2- 172.21.15.55:0/1509791245 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a40066f30 0x7f8a400693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.514+0000 7f8a5f59e700 1 --2- 172.21.15.55:0/1509791245 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a60157270 0x7f8a60169b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.514+0000 7f8a5f59e700 1 --2- 172.21.15.55:0/1509791245 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a600688d0 0x7f8a6016dcd0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.514+0000 7f8a5f59e700 1 --2- 172.21.15.55:0/1509791245 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a600b0a00 0x7f8a60151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.514+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1509791245 >> 172.21.15.55:0/1509791245 conn(0x7f8a600cd310 msgr2=0x7f8a60159650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:04.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.514+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1509791245 shutdown_connections 2023-10-06T03:49:04.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.514+0000 7f8a5f59e700 1 -- 172.21.15.55:0/1509791245 wait complete. 2023-10-06T03:49:04.533 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:49:04.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.719+0000 7f33b539d700 1 Processor -- start 2023-10-06T03:49:04.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.719+0000 7f33b539d700 1 -- start start 2023-10-06T03:49:04.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.721+0000 7f33b539d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f33b01566e0 0x7f33b00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:04.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.721+0000 7f33b539d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f33b00cb900 0x7f33b00ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:04.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.721+0000 7f33b539d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b00ca860 con 0x7f33b0156310 2023-10-06T03:49:04.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.721+0000 7f33b539d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b00cab40 con 0x7f33b01566e0 2023-10-06T03:49:04.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.721+0000 7f33b539d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b00cae20 con 0x7f33b00cb900 2023-10-06T03:49:04.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.721+0000 7f33af7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f33b01566e0 0x7f33b00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:04.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.721+0000 7f33af7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f33b01566e0 0x7f33b00ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49930/0 (socket says 172.21.15.55:49930) 2023-10-06T03:49:04.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.721+0000 7f33af7fe700 1 -- 172.21.15.55:0/1078219502 learned_addr learned my addr 172.21.15.55:0/1078219502 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:04.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.721+0000 7f33b4b9c700 1 --2- 172.21.15.55:0/1078219502 >> v2:172.21.15.163:3300/0 conn(0x7f33b00cb900 0x7f33b00ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:04.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.722+0000 7f33af7fe700 1 -- 172.21.15.55:0/1078219502 >> v2:172.21.15.163:3300/0 conn(0x7f33b00cb900 msgr2=0x7f33b00ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:04.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.722+0000 7f33af7fe700 1 --2- 172.21.15.55:0/1078219502 >> v2:172.21.15.163:3300/0 conn(0x7f33b00cb900 0x7f33b00ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.722+0000 7f33af7fe700 1 -- 172.21.15.55:0/1078219502 >> v1:172.21.15.163:6789/0 conn(0x7f33b0156310 legacy=0x7f33b00c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:04.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.722+0000 7f33aeffd700 1 -- 172.21.15.55:0/1078219502 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599258799 0 0) 0x7f33b00ca860 con 0x7f33b0156310 2023-10-06T03:49:04.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.722+0000 7f33af7fe700 1 -- 172.21.15.55:0/1078219502 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b00cb100 con 0x7f33b01566e0 2023-10-06T03:49:04.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.722+0000 7f33af7fe700 1 --2- 172.21.15.55:0/1078219502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b01566e0 0x7f33b00ae000 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f33b00addc0 tx=0x7f33a400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b1497029d442b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:04.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.722+0000 7f33aeffd700 1 -- 172.21.15.55:0/1078219502 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a400dba0 con 0x7f33b01566e0 2023-10-06T03:49:04.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.722+0000 7f33aeffd700 1 -- 172.21.15.55:0/1078219502 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a400dd00 con 0x7f33b01566e0 2023-10-06T03:49:04.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.722+0000 7f33aeffd700 1 -- 172.21.15.55:0/1078219502 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a40196d0 con 0x7f33b01566e0 2023-10-06T03:49:04.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.723+0000 7f33b539d700 1 -- 172.21.15.55:0/1078219502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b01566e0 msgr2=0x7f33b00ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:04.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.723+0000 7f33b539d700 1 --2- 172.21.15.55:0/1078219502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b01566e0 0x7f33b00ae000 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f33b00addc0 tx=0x7f33a400b4f0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.723+0000 7f33b539d700 1 -- 172.21.15.55:0/1078219502 shutdown_connections 2023-10-06T03:49:04.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.723+0000 7f33b539d700 1 --2- 172.21.15.55:0/1078219502 >> v2:172.21.15.163:3300/0 conn(0x7f33b00cb900 0x7f33b00ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.723+0000 7f33b539d700 1 --2- 172.21.15.55:0/1078219502 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b01566e0 0x7f33b00ae000 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.723+0000 7f33b539d700 1 -- 172.21.15.55:0/1078219502 >> 172.21.15.55:0/1078219502 conn(0x7f33b00cc1f0 msgr2=0x7f33b00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:04.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.723+0000 7f33b539d700 1 -- 172.21.15.55:0/1078219502 shutdown_connections 2023-10-06T03:49:04.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.723+0000 7f33b539d700 1 -- 172.21.15.55:0/1078219502 wait complete. 2023-10-06T03:49:04.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.724+0000 7f33b539d700 1 Processor -- start 2023-10-06T03:49:04.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.725+0000 7f33b539d700 1 -- start start 2023-10-06T03:49:04.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.725+0000 7f33b539d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b00cb900 0x7f33b014d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:04.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33b539d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33b0156310 0x7f33b00c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:04.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33b539d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33b01566e0 0x7f33b0146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:04.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33b539d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b016de80 con 0x7f33b00cb900 2023-10-06T03:49:04.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33b539d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b0151520 con 0x7f33b01566e0 2023-10-06T03:49:04.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33b539d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b016d590 con 0x7f33b0156310 2023-10-06T03:49:04.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33affff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b00cb900 0x7f33b014d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:04.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33affff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b00cb900 0x7f33b014d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49946/0 (socket says 172.21.15.55:49946) 2023-10-06T03:49:04.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33affff700 1 -- 172.21.15.55:0/727847478 learned_addr learned my addr 172.21.15.55:0/727847478 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:04.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33af7fe700 1 --2- 172.21.15.55:0/727847478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33b0156310 0x7f33b00c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:04.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33b4b9c700 1 --2- 172.21.15.55:0/727847478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33b01566e0 0x7f33b0146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:04.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33affff700 1 -- 172.21.15.55:0/727847478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33b0156310 msgr2=0x7f33b00c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:04.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33affff700 1 --2- 172.21.15.55:0/727847478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33b0156310 0x7f33b00c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33affff700 1 -- 172.21.15.55:0/727847478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33b01566e0 msgr2=0x7f33b0146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:04.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33affff700 1 --2- 172.21.15.55:0/727847478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33b01566e0 0x7f33b0146610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.726+0000 7f33affff700 1 -- 172.21.15.55:0/727847478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a400b790 con 0x7f33b00cb900 2023-10-06T03:49:04.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.727+0000 7f33affff700 1 --2- 172.21.15.55:0/727847478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b00cb900 0x7f33b014d780 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c7e0 tx=0x7f33a0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:04.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.727+0000 7f3397fff700 1 -- 172.21.15.55:0/727847478 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a00105c0 con 0x7f33b00cb900 2023-10-06T03:49:04.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.727+0000 7f3397fff700 1 -- 172.21.15.55:0/727847478 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a0010740 con 0x7f33b00cb900 2023-10-06T03:49:04.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.727+0000 7f3397fff700 1 -- 172.21.15.55:0/727847478 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0010b00 con 0x7f33b00cb900 2023-10-06T03:49:04.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.727+0000 7f33b539d700 1 -- 172.21.15.55:0/727847478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b016e310 con 0x7f33b00cb900 2023-10-06T03:49:04.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.727+0000 7f33b539d700 1 -- 172.21.15.55:0/727847478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b016db10 con 0x7f33b00cb900 2023-10-06T03:49:04.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.728+0000 7f33b539d700 1 -- 172.21.15.55:0/727847478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b004c3c0 con 0x7f33b00cb900 2023-10-06T03:49:04.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.729+0000 7f3397fff700 1 -- 172.21.15.55:0/727847478 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f33a001b3b0 con 0x7f33b00cb900 2023-10-06T03:49:04.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.730+0000 7f3397fff700 1 --2- 172.21.15.55:0/727847478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f339c066f30 0x7f339c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:04.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.730+0000 7f3397fff700 1 -- 172.21.15.55:0/727847478 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f33a0083e70 con 0x7f33b00cb900 2023-10-06T03:49:04.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.730+0000 7f33af7fe700 1 --2- 172.21.15.55:0/727847478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f339c066f30 0x7f339c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:04.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.731+0000 7f33af7fe700 1 --2- 172.21.15.55:0/727847478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f339c066f30 0x7f339c0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f33b0156b50 tx=0x7f33a401b000 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:04.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.734+0000 7f3397fff700 1 -- 172.21.15.55:0/727847478 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33a0050d50 con 0x7f33b00cb900 2023-10-06T03:49:04.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.936+0000 7f33b539d700 1 -- 172.21.15.55:0/727847478 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33b00cd0b0 con 0x7f33b00cb900 2023-10-06T03:49:04.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.937+0000 7f3397fff700 1 -- 172.21.15.55:0/727847478 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+17525 (secure 0 0 0) 0x7f33a0017070 con 0x7f33b00cb900 2023-10-06T03:49:04.939 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:04.940 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":92,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:49:03.547288+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:49:01.721643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T03:49:02.520120+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:49:04.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.940+0000 7f33b539d700 1 -- 172.21.15.55:0/727847478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f339c066f30 msgr2=0x7f339c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:04.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.940+0000 7f33b539d700 1 --2- 172.21.15.55:0/727847478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f339c066f30 0x7f339c0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f33b0156b50 tx=0x7f33a401b000 comp rx=0 tx=0).stop 2023-10-06T03:49:04.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.940+0000 7f33b539d700 1 -- 172.21.15.55:0/727847478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b00cb900 msgr2=0x7f33b014d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.940+0000 7f33b539d700 1 --2- 172.21.15.55:0/727847478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b00cb900 0x7f33b014d780 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c7e0 tx=0x7f33a0007df0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.940+0000 7f33b539d700 1 -- 172.21.15.55:0/727847478 shutdown_connections 2023-10-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.940+0000 7f33b539d700 1 --2- 172.21.15.55:0/727847478 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f339c066f30 0x7f339c0693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.940+0000 7f33b539d700 1 --2- 172.21.15.55:0/727847478 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33b01566e0 0x7f33b0146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.940+0000 7f33b539d700 1 --2- 172.21.15.55:0/727847478 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33b00cb900 0x7f33b014d780 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.940+0000 7f33b539d700 1 --2- 172.21.15.55:0/727847478 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33b0156310 0x7f33b00c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.940+0000 7f33b539d700 1 -- 172.21.15.55:0/727847478 >> 172.21.15.55:0/727847478 conn(0x7f33b00cc1f0 msgr2=0x7f33b006ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.940+0000 7f33b539d700 1 -- 172.21.15.55:0/727847478 shutdown_connections 2023-10-06T03:49:04.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:04.940+0000 7f33b539d700 1 -- 172.21.15.55:0/727847478 wait complete. 2023-10-06T03:49:04.959 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:49:05.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.154+0000 7fbe0b59e700 1 Processor -- start 2023-10-06T03:49:05.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.154+0000 7fbe0b59e700 1 -- start start 2023-10-06T03:49:05.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.156+0000 7fbe0b59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbe0c159a70 0x7fbe0c0c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:05.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.156+0000 7fbe0b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbe0c068aa0 0x7fbe0c0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:05.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.156+0000 7fbe0b59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe0c0ca920 con 0x7fbe0c06bfa0 2023-10-06T03:49:05.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.156+0000 7fbe0b59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe0c06af00 con 0x7fbe0c068aa0 2023-10-06T03:49:05.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.156+0000 7fbe0b59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe0c06b1e0 con 0x7fbe0c159a70 2023-10-06T03:49:05.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.156+0000 7fbe0ad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbe0c068aa0 0x7fbe0c0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:05.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.156+0000 7fbe0ad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbe0c068aa0 0x7fbe0c0c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49960/0 (socket says 172.21.15.55:49960) 2023-10-06T03:49:05.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.156+0000 7fbe0ad9d700 1 -- 172.21.15.55:0/96409388 learned_addr learned my addr 172.21.15.55:0/96409388 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:05.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.156+0000 7fbe0a59c700 1 --2- 172.21.15.55:0/96409388 >> v2:172.21.15.163:3300/0 conn(0x7fbe0c159a70 0x7fbe0c0c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:05.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.156+0000 7fbe0ad9d700 1 -- 172.21.15.55:0/96409388 >> v2:172.21.15.163:3300/0 conn(0x7fbe0c159a70 msgr2=0x7fbe0c0c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:05.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.156+0000 7fbe0ad9d700 1 --2- 172.21.15.55:0/96409388 >> v2:172.21.15.163:3300/0 conn(0x7fbe0c159a70 0x7fbe0c0c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:05.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.156+0000 7fbe0ad9d700 1 -- 172.21.15.55:0/96409388 >> v1:172.21.15.163:6789/0 conn(0x7fbe0c06bfa0 legacy=0x7fbe0c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:05.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.156+0000 7fbe0ad9d700 1 -- 172.21.15.55:0/96409388 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c06b4c0 con 0x7fbe0c068aa0 2023-10-06T03:49:05.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.157+0000 7fbe0ad9d700 1 --2- 172.21.15.55:0/96409388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c068aa0 0x7fbe0c0c8830 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0640b0 tx=0x7fbe04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbde25df7bb4409 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:05.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.157+0000 7fbe0959a700 1 -- 172.21.15.55:0/96409388 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0400e070 con 0x7fbe0c068aa0 2023-10-06T03:49:05.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.157+0000 7fbe0959a700 1 -- 172.21.15.55:0/96409388 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0400ba10 con 0x7fbe0c068aa0 2023-10-06T03:49:05.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.157+0000 7fbe0959a700 1 -- 172.21.15.55:0/96409388 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe040092e0 con 0x7fbe0c068aa0 2023-10-06T03:49:05.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.157+0000 7fbe0b59e700 1 -- 172.21.15.55:0/96409388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c068aa0 msgr2=0x7fbe0c0c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:05.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.157+0000 7fbe0b59e700 1 --2- 172.21.15.55:0/96409388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c068aa0 0x7fbe0c0c8830 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c0640b0 tx=0x7fbe04009840 comp rx=0 tx=0).stop 2023-10-06T03:49:05.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.158+0000 7fbe0b59e700 1 -- 172.21.15.55:0/96409388 shutdown_connections 2023-10-06T03:49:05.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.158+0000 7fbe0b59e700 1 --2- 172.21.15.55:0/96409388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c068aa0 0x7fbe0c0c8830 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:05.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.158+0000 7fbe0b59e700 1 --2- 172.21.15.55:0/96409388 >> v2:172.21.15.163:3300/0 conn(0x7fbe0c159a70 0x7fbe0c0c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:05.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.158+0000 7fbe0b59e700 1 -- 172.21.15.55:0/96409388 >> 172.21.15.55:0/96409388 conn(0x7fbe0c0cddc0 msgr2=0x7fbe0c0c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:05.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.158+0000 7fbe0b59e700 1 -- 172.21.15.55:0/96409388 shutdown_connections 2023-10-06T03:49:05.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.158+0000 7fbe0b59e700 1 -- 172.21.15.55:0/96409388 wait complete. 2023-10-06T03:49:05.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.159+0000 7fbe0b59e700 1 Processor -- start 2023-10-06T03:49:05.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.159+0000 7fbe0b59e700 1 -- start start 2023-10-06T03:49:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.160+0000 7fbe0b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe0c068aa0 0x7fbe0c0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.160+0000 7fbe0b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06bfa0 0x7fbe0c0c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.160+0000 7fbe0b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe0c159a70 0x7fbe0c13faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.160+0000 7fbe0b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c1517b0 con 0x7fbe0c06bfa0 2023-10-06T03:49:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.160+0000 7fbe0b59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c140030 con 0x7fbe0c068aa0 2023-10-06T03:49:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.160+0000 7fbe0b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0c0c4a30 con 0x7fbe0c159a70 2023-10-06T03:49:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.160+0000 7fbe09d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06bfa0 0x7fbe0c0c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.161+0000 7fbe09d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06bfa0 0x7fbe0c0c43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49976/0 (socket says 172.21.15.55:49976) 2023-10-06T03:49:05.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.161+0000 7fbe09d9b700 1 -- 172.21.15.55:0/2515036916 learned_addr learned my addr 172.21.15.55:0/2515036916 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.161+0000 7fbe0a59c700 1 --2- 172.21.15.55:0/2515036916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe0c068aa0 0x7fbe0c0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.161+0000 7fbe0ad9d700 1 --2- 172.21.15.55:0/2515036916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe0c159a70 0x7fbe0c13faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.161+0000 7fbe09d9b700 1 -- 172.21.15.55:0/2515036916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe0c159a70 msgr2=0x7fbe0c13faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.161+0000 7fbe09d9b700 1 --2- 172.21.15.55:0/2515036916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe0c159a70 0x7fbe0c13faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.161+0000 7fbe09d9b700 1 -- 172.21.15.55:0/2515036916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe0c068aa0 msgr2=0x7fbe0c0ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.161+0000 7fbe09d9b700 1 --2- 172.21.15.55:0/2515036916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe0c068aa0 0x7fbe0c0ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.161+0000 7fbe09d9b700 1 -- 172.21.15.55:0/2515036916 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe04009ae0 con 0x7fbe0c06bfa0 2023-10-06T03:49:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.161+0000 7fbe09d9b700 1 --2- 172.21.15.55:0/2515036916 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06bfa0 0x7fbe0c0c43a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000b5e0 tx=0x7fbe000090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.161+0000 7fbdfa7fc700 1 -- 172.21.15.55:0/2515036916 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000f510 con 0x7fbe0c06bfa0 2023-10-06T03:49:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.161+0000 7fbdfa7fc700 1 -- 172.21.15.55:0/2515036916 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0000f670 con 0x7fbe0c06bfa0 2023-10-06T03:49:05.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.161+0000 7fbe0b59e700 1 -- 172.21.15.55:0/2515036916 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0c0c6ba0 con 0x7fbe0c06bfa0 2023-10-06T03:49:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.161+0000 7fbdfa7fc700 1 -- 172.21.15.55:0/2515036916 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000fa30 con 0x7fbe0c06bfa0 2023-10-06T03:49:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.162+0000 7fbe0b59e700 1 -- 172.21.15.55:0/2515036916 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0c0c7160 con 0x7fbe0c06bfa0 2023-10-06T03:49:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.162+0000 7fbe0b59e700 1 -- 172.21.15.55:0/2515036916 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0c0b0120 con 0x7fbe0c06bfa0 2023-10-06T03:49:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.163+0000 7fbdfa7fc700 1 -- 172.21.15.55:0/2515036916 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbe0001a400 con 0x7fbe0c06bfa0 2023-10-06T03:49:05.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.164+0000 7fbdfa7fc700 1 --2- 172.21.15.55:0/2515036916 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbdec066f30 0x7fbdec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:05.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.165+0000 7fbdfa7fc700 1 -- 172.21.15.55:0/2515036916 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fbe00084350 con 0x7fbe0c06bfa0 2023-10-06T03:49:05.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.165+0000 7fbe0a59c700 1 --2- 172.21.15.55:0/2515036916 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbdec066f30 0x7fbdec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:05.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.166+0000 7fbe0a59c700 1 --2- 172.21.15.55:0/2515036916 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbdec066f30 0x7fbdec0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc006fd0 tx=0x7fbdfc008040 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:05.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.168+0000 7fbdfa7fc700 1 -- 172.21.15.55:0/2515036916 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbe0c0b0120 con 0x7fbe0c06bfa0 2023-10-06T03:49:05.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:05.415+0000 7fbe0b59e700 1 -- 172.21.15.55:0/2515036916 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbe0c0b0120 con 0x7fbe0c06bfa0 2023-10-06T03:49:06.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.074+0000 7fbdfa7fc700 1 -- 172.21.15.55:0/2515036916 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbe0c0b0120 con 0x7fbe0c06bfa0 2023-10-06T03:49:06.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.076+0000 7fbe0b59e700 1 -- 172.21.15.55:0/2515036916 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbdec066f30 msgr2=0x7fbdec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:06.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.076+0000 7fbe0b59e700 1 --2- 172.21.15.55:0/2515036916 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbdec066f30 0x7fbdec0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc006fd0 tx=0x7fbdfc008040 comp rx=0 tx=0).stop 2023-10-06T03:49:06.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.076+0000 7fbe0b59e700 1 -- 172.21.15.55:0/2515036916 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06bfa0 msgr2=0x7fbe0c0c43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:06.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.076+0000 7fbe0b59e700 1 --2- 172.21.15.55:0/2515036916 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06bfa0 0x7fbe0c0c43a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000b5e0 tx=0x7fbe000090f0 comp rx=0 tx=0).stop 2023-10-06T03:49:06.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.077+0000 7fbe0b59e700 1 -- 172.21.15.55:0/2515036916 shutdown_connections 2023-10-06T03:49:06.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.077+0000 7fbe0b59e700 1 --2- 172.21.15.55:0/2515036916 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbdec066f30 0x7fbdec0693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:06.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.077+0000 7fbe0b59e700 1 --2- 172.21.15.55:0/2515036916 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbe0c159a70 0x7fbe0c13faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:06.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.077+0000 7fbe0b59e700 1 --2- 172.21.15.55:0/2515036916 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbe0c068aa0 0x7fbe0c0ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:06.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.077+0000 7fbe0b59e700 1 --2- 172.21.15.55:0/2515036916 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbe0c06bfa0 0x7fbe0c0c43a0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:06.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.077+0000 7fbe0b59e700 1 -- 172.21.15.55:0/2515036916 >> 172.21.15.55:0/2515036916 conn(0x7fbe0c0cddc0 msgr2=0x7fbe0c0c8710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.077+0000 7fbe0b59e700 1 -- 172.21.15.55:0/2515036916 shutdown_connections 2023-10-06T03:49:06.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.077+0000 7fbe0b59e700 1 -- 172.21.15.55:0/2515036916 wait complete. 2023-10-06T03:49:06.113 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:49:06.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.303+0000 7f5649dff700 1 Processor -- start 2023-10-06T03:49:06.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.303+0000 7f5649dff700 1 -- start start 2023-10-06T03:49:06.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.304+0000 7f5649dff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f56440c83f0 0x7f56440c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:06.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.304+0000 7f5649dff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f56440ccee0 0x7f56441561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:06.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.305+0000 7f5649dff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564415b0d0 con 0x7f56440c87c0 2023-10-06T03:49:06.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.305+0000 7f5649dff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56440c7a60 con 0x7f56440c83f0 2023-10-06T03:49:06.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.305+0000 7f5649dff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f564415a4c0 con 0x7f56440ccee0 2023-10-06T03:49:06.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.305+0000 7f56437fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f56440c83f0 0x7f56440c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:06.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.305+0000 7f56437fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f56440c83f0 0x7f56440c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49990/0 (socket says 172.21.15.55:49990) 2023-10-06T03:49:06.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.305+0000 7f56437fe700 1 -- 172.21.15.55:0/3351050872 learned_addr learned my addr 172.21.15.55:0/3351050872 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:06.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.305+0000 7f5642ffd700 1 --2- 172.21.15.55:0/3351050872 >> v2:172.21.15.163:3300/0 conn(0x7f56440ccee0 0x7f56441561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:06.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.305+0000 7f56437fe700 1 -- 172.21.15.55:0/3351050872 >> v2:172.21.15.163:3300/0 conn(0x7f56440ccee0 msgr2=0x7f56441561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:06.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.305+0000 7f56437fe700 1 --2- 172.21.15.55:0/3351050872 >> v2:172.21.15.163:3300/0 conn(0x7f56440ccee0 0x7f56441561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:06.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.305+0000 7f56437fe700 1 -- 172.21.15.55:0/3351050872 >> v1:172.21.15.163:6789/0 conn(0x7f56440c87c0 legacy=0x7f564406eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:06.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.305+0000 7f56437fe700 1 -- 172.21.15.55:0/3351050872 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564415a7a0 con 0x7f56440c83f0 2023-10-06T03:49:06.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.306+0000 7f56437fe700 1 --2- 172.21.15.55:0/3351050872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56440c83f0 0x7f56440c7520 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f56440c7eb0 tx=0x7f563400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8772680eb288268 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:06.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.306+0000 7f56427fc700 1 -- 172.21.15.55:0/3351050872 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563400dba0 con 0x7f56440c83f0 2023-10-06T03:49:06.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.306+0000 7f56427fc700 1 -- 172.21.15.55:0/3351050872 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563400dd00 con 0x7f56440c83f0 2023-10-06T03:49:06.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.306+0000 7f56427fc700 1 -- 172.21.15.55:0/3351050872 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56340196d0 con 0x7f56440c83f0 2023-10-06T03:49:06.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.306+0000 7f5649dff700 1 -- 172.21.15.55:0/3351050872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56440c83f0 msgr2=0x7f56440c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.306+0000 7f5649dff700 1 --2- 172.21.15.55:0/3351050872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56440c83f0 0x7f56440c7520 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f56440c7eb0 tx=0x7f563400b4f0 comp rx=0 tx=0).stop 2023-10-06T03:49:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.307+0000 7f5649dff700 1 -- 172.21.15.55:0/3351050872 shutdown_connections 2023-10-06T03:49:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.307+0000 7f5649dff700 1 --2- 172.21.15.55:0/3351050872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56440c83f0 0x7f56440c7520 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.307+0000 7f5649dff700 1 --2- 172.21.15.55:0/3351050872 >> v2:172.21.15.163:3300/0 conn(0x7f56440ccee0 0x7f56441561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.307+0000 7f5649dff700 1 -- 172.21.15.55:0/3351050872 >> 172.21.15.55:0/3351050872 conn(0x7f5644068e10 msgr2=0x7f56440cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.307+0000 7f5649dff700 1 -- 172.21.15.55:0/3351050872 shutdown_connections 2023-10-06T03:49:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.307+0000 7f5649dff700 1 -- 172.21.15.55:0/3351050872 wait complete. 2023-10-06T03:49:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.308+0000 7f5649dff700 1 Processor -- start 2023-10-06T03:49:06.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.308+0000 7f5649dff700 1 -- start start 2023-10-06T03:49:06.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.309+0000 7f5649dff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56440c83f0 0x7f564414d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:06.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.309+0000 7f5649dff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56440c87c0 0x7f56440c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:06.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.309+0000 7f5649dff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56440ccee0 0x7f5644147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:06.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.309+0000 7f5649dff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56440c3890 con 0x7f56440c87c0 2023-10-06T03:49:06.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.309+0000 7f5649dff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56440c3dc0 con 0x7f56440c83f0 2023-10-06T03:49:06.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.309+0000 7f5649dff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56440c64e0 con 0x7f56440ccee0 2023-10-06T03:49:06.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.309+0000 7f5642ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56440c87c0 0x7f56440c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:06.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.309+0000 7f5642ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56440c87c0 0x7f56440c5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50006/0 (socket says 172.21.15.55:50006) 2023-10-06T03:49:06.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.309+0000 7f5642ffd700 1 -- 172.21.15.55:0/1208369909 learned_addr learned my addr 172.21.15.55:0/1208369909 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:06.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.309+0000 7f56437fe700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56440c83f0 0x7f564414d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:06.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.310+0000 7f5643fff700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56440ccee0 0x7f5644147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:06.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.310+0000 7f5642ffd700 1 -- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56440ccee0 msgr2=0x7f5644147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:06.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.310+0000 7f5642ffd700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56440ccee0 0x7f5644147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:06.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.310+0000 7f5642ffd700 1 -- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56440c83f0 msgr2=0x7f564414d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:06.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.310+0000 7f5642ffd700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56440c83f0 0x7f564414d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:06.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.310+0000 7f5642ffd700 1 -- 172.21.15.55:0/1208369909 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563400b790 con 0x7f56440c87c0 2023-10-06T03:49:06.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.310+0000 7f56437fe700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56440c83f0 0x7f564414d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:06.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.310+0000 7f5642ffd700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56440c87c0 0x7f56440c5fa0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f563800ec80 tx=0x7f56380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:06.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.310+0000 7f562b7fe700 1 -- 172.21.15.55:0/1208369909 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563800c9d0 con 0x7f56440c87c0 2023-10-06T03:49:06.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.310+0000 7f562b7fe700 1 -- 172.21.15.55:0/1208369909 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563800cb30 con 0x7f56440c87c0 2023-10-06T03:49:06.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.310+0000 7f5649dff700 1 -- 172.21.15.55:0/1208369909 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56440c3ae0 con 0x7f56440c87c0 2023-10-06T03:49:06.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.310+0000 7f562b7fe700 1 -- 172.21.15.55:0/1208369909 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638013470 con 0x7f56440c87c0 2023-10-06T03:49:06.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.310+0000 7f5649dff700 1 -- 172.21.15.55:0/1208369909 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56440b0720 con 0x7f56440c87c0 2023-10-06T03:49:06.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.312+0000 7f5649dff700 1 -- 172.21.15.55:0/1208369909 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564404c3c0 con 0x7f56440c87c0 2023-10-06T03:49:06.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.313+0000 7f562b7fe700 1 -- 172.21.15.55:0/1208369909 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5638013820 con 0x7f56440c87c0 2023-10-06T03:49:06.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.313+0000 7f562b7fe700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5630066f30 0x7f56300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:06.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.313+0000 7f562b7fe700 1 -- 172.21.15.55:0/1208369909 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f5638084ad0 con 0x7f56440c87c0 2023-10-06T03:49:06.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.314+0000 7f56437fe700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5630066f30 0x7f56300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:06.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.314+0000 7f56437fe700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5630066f30 0x7f56300693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f56440c7eb0 tx=0x7f563401b000 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:06.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.318+0000 7f562b7fe700 1 -- 172.21.15.55:0/1208369909 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5638051a50 con 0x7f56440c87c0 2023-10-06T03:49:06.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:06.568+0000 7f5649dff700 1 -- 172.21.15.55:0/1208369909 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f56440cbd90 con 0x7f56440c87c0 2023-10-06T03:49:07.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.082+0000 7f562b7fe700 1 -- 172.21.15.55:0/1208369909 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5638055070 con 0x7f56440c87c0 2023-10-06T03:49:07.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.085+0000 7f5649dff700 1 -- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5630066f30 msgr2=0x7f56300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:07.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.085+0000 7f5649dff700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5630066f30 0x7f56300693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f56440c7eb0 tx=0x7f563401b000 comp rx=0 tx=0).stop 2023-10-06T03:49:07.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.085+0000 7f5649dff700 1 -- 172.21.15.55:0/1208369909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56440c87c0 msgr2=0x7f56440c5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:07.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.085+0000 7f5649dff700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56440c87c0 0x7f56440c5fa0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f563800ec80 tx=0x7f56380091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.085+0000 7f5649dff700 1 -- 172.21.15.55:0/1208369909 shutdown_connections 2023-10-06T03:49:07.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.085+0000 7f5649dff700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5630066f30 0x7f56300693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.085+0000 7f5649dff700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f56440ccee0 0x7f5644147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.085+0000 7f5649dff700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f56440c83f0 0x7f564414d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.085+0000 7f5649dff700 1 --2- 172.21.15.55:0/1208369909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f56440c87c0 0x7f56440c5fa0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.085+0000 7f5649dff700 1 -- 172.21.15.55:0/1208369909 >> 172.21.15.55:0/1208369909 conn(0x7f5644068e10 msgr2=0x7f564414f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:07.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.085+0000 7f5649dff700 1 -- 172.21.15.55:0/1208369909 shutdown_connections 2023-10-06T03:49:07.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.085+0000 7f5649dff700 1 -- 172.21.15.55:0/1208369909 wait complete. 2023-10-06T03:49:07.105 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:49:07.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.292+0000 7f36add9d700 1 Processor -- start 2023-10-06T03:49:07.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.292+0000 7f36add9d700 1 -- start start 2023-10-06T03:49:07.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.293+0000 7f36add9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36a8156310 0x7f36a80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:07.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.293+0000 7f36add9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f36a81566e0 0x7f36a80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:07.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.293+0000 7f36add9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a80ca860 con 0x7f36a80ca300 2023-10-06T03:49:07.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.293+0000 7f36add9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a80cab40 con 0x7f36a81566e0 2023-10-06T03:49:07.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.293+0000 7f36add9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a80cae20 con 0x7f36a8156310 2023-10-06T03:49:07.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.294+0000 7f36acd9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36a8156310 0x7f36a80c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:07.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.294+0000 7f36a7fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f36a81566e0 0x7f36a80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:07.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.294+0000 7f36acd9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36a8156310 0x7f36a80c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:33076/0 (socket says 172.21.15.55:33076) 2023-10-06T03:49:07.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.294+0000 7f36ad59c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f36a80ca300 0x7f36a80c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:38486/0 (socket says 172.21.15.55:38486) 2023-10-06T03:49:07.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.294+0000 7f36a7fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f36a81566e0 0x7f36a80ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:60952/0 (socket says 172.21.15.55:60952) 2023-10-06T03:49:07.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.294+0000 7f36acd9b700 1 -- 172.21.15.55:0/802020182 learned_addr learned my addr 172.21.15.55:0/802020182 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:07.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.294+0000 7f36a77fe700 1 -- 172.21.15.55:0/802020182 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718375313 0 0) 0x7f36a80ca860 con 0x7f36a80ca300 2023-10-06T03:49:07.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.294+0000 7f36a77fe700 1 -- 172.21.15.55:0/802020182 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3698003570 con 0x7f36a80ca300 2023-10-06T03:49:07.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.294+0000 7f36a7fff700 1 -- 172.21.15.55:0/802020182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36a8156310 msgr2=0x7f36a80c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:07.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.294+0000 7f36a7fff700 1 --2- 172.21.15.55:0/802020182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36a8156310 0x7f36a80c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.294+0000 7f36a7fff700 1 -- 172.21.15.55:0/802020182 >> v1:172.21.15.163:6789/0 conn(0x7f36a80ca300 legacy=0x7f36a80c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:07.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.294+0000 7f36a7fff700 1 -- 172.21.15.55:0/802020182 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a80cb100 con 0x7f36a81566e0 2023-10-06T03:49:07.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.294+0000 7f36acd9b700 1 --2- 172.21.15.55:0/802020182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36a8156310 0x7f36a80c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:07.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.295+0000 7f36a7fff700 1 --2- 172.21.15.55:0/802020182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a81566e0 0x7f36a80ae000 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f36940098e0 tx=0x7f36940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75951201454fb9ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:07.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.295+0000 7f36a77fe700 1 -- 172.21.15.55:0/802020182 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694011630 con 0x7f36a81566e0 2023-10-06T03:49:07.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.295+0000 7f36a77fe700 1 -- 172.21.15.55:0/802020182 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3694011790 con 0x7f36a81566e0 2023-10-06T03:49:07.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.295+0000 7f36a77fe700 1 -- 172.21.15.55:0/802020182 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694011b30 con 0x7f36a81566e0 2023-10-06T03:49:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.295+0000 7f36add9d700 1 -- 172.21.15.55:0/802020182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a81566e0 msgr2=0x7f36a80ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.295+0000 7f36add9d700 1 --2- 172.21.15.55:0/802020182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a81566e0 0x7f36a80ae000 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f36940098e0 tx=0x7f36940091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.295+0000 7f36add9d700 1 -- 172.21.15.55:0/802020182 shutdown_connections 2023-10-06T03:49:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.295+0000 7f36add9d700 1 --2- 172.21.15.55:0/802020182 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36a8156310 0x7f36a80c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.295+0000 7f36add9d700 1 --2- 172.21.15.55:0/802020182 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a81566e0 0x7f36a80ae000 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.295+0000 7f36add9d700 1 -- 172.21.15.55:0/802020182 >> 172.21.15.55:0/802020182 conn(0x7f36a80cc1f0 msgr2=0x7f36a80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.296+0000 7f36add9d700 1 -- 172.21.15.55:0/802020182 shutdown_connections 2023-10-06T03:49:07.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.296+0000 7f36add9d700 1 -- 172.21.15.55:0/802020182 wait complete. 2023-10-06T03:49:07.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.297+0000 7f36add9d700 1 Processor -- start 2023-10-06T03:49:07.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.297+0000 7f36add9d700 1 -- start start 2023-10-06T03:49:07.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.298+0000 7f36add9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a80ca300 0x7f36a8153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:07.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.298+0000 7f36add9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36a8156310 0x7f36a814c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:07.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.298+0000 7f36add9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36a81566e0 0x7f36a8149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:07.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.298+0000 7f36add9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a816d100 con 0x7f36a8156310 2023-10-06T03:49:07.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.298+0000 7f36add9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a8154700 con 0x7f36a80ca300 2023-10-06T03:49:07.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.298+0000 7f36add9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a8154140 con 0x7f36a81566e0 2023-10-06T03:49:07.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.298+0000 7f36a7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36a8156310 0x7f36a814c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:07.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.298+0000 7f36a7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36a8156310 0x7f36a814c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50016/0 (socket says 172.21.15.55:50016) 2023-10-06T03:49:07.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.298+0000 7f36a7fff700 1 -- 172.21.15.55:0/438430263 learned_addr learned my addr 172.21.15.55:0/438430263 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:07.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.298+0000 7f36acd9b700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a80ca300 0x7f36a8153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.299+0000 7f36ad59c700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36a81566e0 0x7f36a8149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.299+0000 7f36a7fff700 1 -- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36a81566e0 msgr2=0x7f36a8149410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.299+0000 7f36a7fff700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36a81566e0 0x7f36a8149410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.299+0000 7f36a7fff700 1 -- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a80ca300 msgr2=0x7f36a8153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.299+0000 7f36a7fff700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a80ca300 0x7f36a8153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.299+0000 7f36a7fff700 1 -- 172.21.15.55:0/438430263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c00b790 con 0x7f36a8156310 2023-10-06T03:49:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.299+0000 7f36acd9b700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a80ca300 0x7f36a8153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.299+0000 7f36a7fff700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36a8156310 0x7f36a814c2b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3694013000 tx=0x7f3694017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.299+0000 7f36a4ff9700 1 -- 172.21.15.55:0/438430263 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694011b10 con 0x7f36a8156310 2023-10-06T03:49:07.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.299+0000 7f36a4ff9700 1 -- 172.21.15.55:0/438430263 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36940253f0 con 0x7f36a8156310 2023-10-06T03:49:07.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.299+0000 7f36add9d700 1 -- 172.21.15.55:0/438430263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3694009420 con 0x7f36a8156310 2023-10-06T03:49:07.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.299+0000 7f36a4ff9700 1 -- 172.21.15.55:0/438430263 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36940257b0 con 0x7f36a8156310 2023-10-06T03:49:07.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.299+0000 7f36add9d700 1 -- 172.21.15.55:0/438430263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a816d3a0 con 0x7f36a8156310 2023-10-06T03:49:07.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.300+0000 7f36add9d700 1 -- 172.21.15.55:0/438430263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a804c3c0 con 0x7f36a8156310 2023-10-06T03:49:07.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.301+0000 7f36a4ff9700 1 -- 172.21.15.55:0/438430263 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3694025ba0 con 0x7f36a8156310 2023-10-06T03:49:07.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.302+0000 7f36a4ff9700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f368c066f30 0x7f368c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:07.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.302+0000 7f36a4ff9700 1 -- 172.21.15.55:0/438430263 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f36940861c0 con 0x7f36a8156310 2023-10-06T03:49:07.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.303+0000 7f36acd9b700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f368c066f30 0x7f368c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:07.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.304+0000 7f36acd9b700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f368c066f30 0x7f368c0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f369c00b240 tx=0x7f369c00b4f0 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:07.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.306+0000 7f36a4ff9700 1 -- 172.21.15.55:0/438430263 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36940530a0 con 0x7f36a8156310 2023-10-06T03:49:07.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.551+0000 7f36add9d700 1 -- 172.21.15.55:0/438430263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36a80cd0b0 con 0x7f36a8156310 2023-10-06T03:49:07.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.552+0000 7f36a4ff9700 1 -- 172.21.15.55:0/438430263 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f36940566c0 con 0x7f36a8156310 2023-10-06T03:49:07.555 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:07.555 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/4087100648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4087100648},{"type":"v1","addr":"172.21.15.163:6834","nonce":4087100648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2017152000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2017152000},{"type":"v1","addr":"172.21.15.163:6836","nonce":2017152000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1226557536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1226557536},{"type":"v1","addr":"172.21.15.55:6834","nonce":1226557536}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:49:03.547622+0000","modified":"2023-10-06T03:49:07.074089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6804":{"gid":6804,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/3960749421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3960749421},{"type":"v1","addr":"172.21.15.55:6835","nonce":3960749421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T03:49:07.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.554+0000 7f36add9d700 1 -- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f368c066f30 msgr2=0x7f368c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:07.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.554+0000 7f36add9d700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f368c066f30 0x7f368c0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f369c00b240 tx=0x7f369c00b4f0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.554+0000 7f36add9d700 1 -- 172.21.15.55:0/438430263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36a8156310 msgr2=0x7f36a814c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:07.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.554+0000 7f36add9d700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36a8156310 0x7f36a814c2b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3694013000 tx=0x7f3694017000 comp rx=0 tx=0).stop 2023-10-06T03:49:07.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.554+0000 7f36add9d700 1 -- 172.21.15.55:0/438430263 shutdown_connections 2023-10-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.554+0000 7f36add9d700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f368c066f30 0x7f368c0693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.554+0000 7f36add9d700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f36a81566e0 0x7f36a8149410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.554+0000 7f36add9d700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f36a80ca300 0x7f36a8153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.554+0000 7f36add9d700 1 --2- 172.21.15.55:0/438430263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f36a8156310 0x7f36a814c2b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.554+0000 7f36add9d700 1 -- 172.21.15.55:0/438430263 >> 172.21.15.55:0/438430263 conn(0x7f36a80cc1f0 msgr2=0x7f36a806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.554+0000 7f36add9d700 1 -- 172.21.15.55:0/438430263 shutdown_connections 2023-10-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.554+0000 7f36add9d700 1 -- 172.21.15.55:0/438430263 wait complete. 2023-10-06T03:49:07.557 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 44 2023-10-06T03:49:07.573 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:49:07.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.760+0000 7fc35999d700 1 Processor -- start 2023-10-06T03:49:07.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.760+0000 7fc35999d700 1 -- start start 2023-10-06T03:49:07.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.761+0000 7fc35999d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc354156310 0x7fc3540c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:07.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.761+0000 7fc35999d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3541566e0 0x7fc3540ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:07.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.761+0000 7fc35999d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3540ca860 con 0x7fc3540ca300 2023-10-06T03:49:07.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.761+0000 7fc35999d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3540cab40 con 0x7fc354156310 2023-10-06T03:49:07.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.761+0000 7fc35999d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540cae20 con 0x7fc3541566e0 2023-10-06T03:49:07.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.762+0000 7fc35899b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc354156310 0x7fc3540c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:07.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.762+0000 7fc35899b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc354156310 0x7fc3540c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50026/0 (socket says 172.21.15.55:50026) 2023-10-06T03:49:07.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.762+0000 7fc35899b700 1 -- 172.21.15.55:0/2238170747 learned_addr learned my addr 172.21.15.55:0/2238170747 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:07.765 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.762+0000 7fc35919c700 1 --1- 172.21.15.55:0/2238170747 >> v1:172.21.15.55:6789/0 conn(0x7fc3540ca300 0x7fc3540c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57424/0 (socket says 172.21.15.55:57424) 2023-10-06T03:49:07.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.762+0000 7fc353fff700 1 --2- 172.21.15.55:0/2238170747 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3541566e0 0x7fc3540ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:07.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.762+0000 7fc35899b700 1 -- 172.21.15.55:0/2238170747 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3541566e0 msgr2=0x7fc3540ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:07.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.762+0000 7fc35899b700 1 --2- 172.21.15.55:0/2238170747 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3541566e0 0x7fc3540ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.762+0000 7fc35899b700 1 -- 172.21.15.55:0/2238170747 >> v1:172.21.15.55:6789/0 conn(0x7fc3540ca300 legacy=0x7fc3540c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:07.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.762+0000 7fc35899b700 1 -- 172.21.15.55:0/2238170747 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3540cb100 con 0x7fc354156310 2023-10-06T03:49:07.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.762+0000 7fc3537fe700 1 -- 172.21.15.55:0/2238170747 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140309284 0 0) 0x7fc3540ca860 con 0x7fc3540ca300 2023-10-06T03:49:07.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.763+0000 7fc35899b700 1 --2- 172.21.15.55:0/2238170747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc354156310 0x7fc3540c9dc0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc3480099b0 tx=0x7fc348009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bd91d42c6f6e68 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:07.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.763+0000 7fc3537fe700 1 -- 172.21.15.55:0/2238170747 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348009ea0 con 0x7fc354156310 2023-10-06T03:49:07.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.763+0000 7fc3537fe700 1 -- 172.21.15.55:0/2238170747 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34800b860 con 0x7fc354156310 2023-10-06T03:49:07.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.763+0000 7fc3537fe700 1 -- 172.21.15.55:0/2238170747 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34800bc00 con 0x7fc354156310 2023-10-06T03:49:07.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.763+0000 7fc35999d700 1 -- 172.21.15.55:0/2238170747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc354156310 msgr2=0x7fc3540c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:07.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.763+0000 7fc35999d700 1 --2- 172.21.15.55:0/2238170747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc354156310 0x7fc3540c9dc0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fc3480099b0 tx=0x7fc348009250 comp rx=0 tx=0).stop 2023-10-06T03:49:07.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.763+0000 7fc35999d700 1 -- 172.21.15.55:0/2238170747 shutdown_connections 2023-10-06T03:49:07.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.763+0000 7fc35999d700 1 --2- 172.21.15.55:0/2238170747 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc354156310 0x7fc3540c9dc0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.763+0000 7fc35999d700 1 --2- 172.21.15.55:0/2238170747 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3541566e0 0x7fc3540ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.763+0000 7fc35999d700 1 -- 172.21.15.55:0/2238170747 >> 172.21.15.55:0/2238170747 conn(0x7fc3540cc1f0 msgr2=0x7fc3540cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:07.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.763+0000 7fc35999d700 1 -- 172.21.15.55:0/2238170747 shutdown_connections 2023-10-06T03:49:07.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.764+0000 7fc35999d700 1 -- 172.21.15.55:0/2238170747 wait complete. 2023-10-06T03:49:07.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.765+0000 7fc35999d700 1 Processor -- start 2023-10-06T03:49:07.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.765+0000 7fc35999d700 1 -- start start 2023-10-06T03:49:07.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.766+0000 7fc35999d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3540ca300 0x7fc3541467f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:07.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.766+0000 7fc35999d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc354156310 0x7fc35416b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:07.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.766+0000 7fc35999d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3541566e0 0x7fc35416be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:07.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.766+0000 7fc35999d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35414a930 con 0x7fc354156310 2023-10-06T03:49:07.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.766+0000 7fc35999d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540c32a0 con 0x7fc3541566e0 2023-10-06T03:49:07.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.766+0000 7fc35999d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc35414abd0 con 0x7fc3540ca300 2023-10-06T03:49:07.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.766+0000 7fc353fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc354156310 0x7fc35416b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:07.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.766+0000 7fc353fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc354156310 0x7fc35416b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50040/0 (socket says 172.21.15.55:50040) 2023-10-06T03:49:07.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.766+0000 7fc353fff700 1 -- 172.21.15.55:0/172557544 learned_addr learned my addr 172.21.15.55:0/172557544 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:07.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.767+0000 7fc35919c700 1 --2- 172.21.15.55:0/172557544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3541566e0 0x7fc35416be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:07.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.767+0000 7fc35899b700 1 --2- 172.21.15.55:0/172557544 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3540ca300 0x7fc3541467f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:07.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.767+0000 7fc353fff700 1 -- 172.21.15.55:0/172557544 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3540ca300 msgr2=0x7fc3541467f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:07.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.767+0000 7fc353fff700 1 --2- 172.21.15.55:0/172557544 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3540ca300 0x7fc3541467f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.767+0000 7fc353fff700 1 -- 172.21.15.55:0/172557544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3541566e0 msgr2=0x7fc35416be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:07.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.767+0000 7fc353fff700 1 --2- 172.21.15.55:0/172557544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3541566e0 0x7fc35416be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.767+0000 7fc353fff700 1 -- 172.21.15.55:0/172557544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3480094f0 con 0x7fc354156310 2023-10-06T03:49:07.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.767+0000 7fc353fff700 1 --2- 172.21.15.55:0/172557544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc354156310 0x7fc35416b920 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc34000ec50 tx=0x7fc3400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:07.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.767+0000 7fc350ff9700 1 -- 172.21.15.55:0/172557544 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34000ca60 con 0x7fc354156310 2023-10-06T03:49:07.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.767+0000 7fc350ff9700 1 -- 172.21.15.55:0/172557544 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34000cbc0 con 0x7fc354156310 2023-10-06T03:49:07.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.767+0000 7fc350ff9700 1 -- 172.21.15.55:0/172557544 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc340013470 con 0x7fc354156310 2023-10-06T03:49:07.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.767+0000 7fc35999d700 1 -- 172.21.15.55:0/172557544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35414aeb0 con 0x7fc354156310 2023-10-06T03:49:07.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.767+0000 7fc35999d700 1 -- 172.21.15.55:0/172557544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc354147460 con 0x7fc354156310 2023-10-06T03:49:07.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.768+0000 7fc35999d700 1 -- 172.21.15.55:0/172557544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc35404c3c0 con 0x7fc354156310 2023-10-06T03:49:07.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.769+0000 7fc350ff9700 1 -- 172.21.15.55:0/172557544 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc340013820 con 0x7fc354156310 2023-10-06T03:49:07.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.770+0000 7fc350ff9700 1 --2- 172.21.15.55:0/172557544 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc338066f30 0x7fc3380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:07.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.770+0000 7fc350ff9700 1 -- 172.21.15.55:0/172557544 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fc340084a50 con 0x7fc354156310 2023-10-06T03:49:07.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.770+0000 7fc35899b700 1 --2- 172.21.15.55:0/172557544 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc338066f30 0x7fc3380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:07.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.771+0000 7fc35899b700 1 --2- 172.21.15.55:0/172557544 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc338066f30 0x7fc3380693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc3480099b0 tx=0x7fc348017000 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:07.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.774+0000 7fc350ff9700 1 -- 172.21.15.55:0/172557544 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc340051930 con 0x7fc354156310 2023-10-06T03:49:07.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.975+0000 7fc35999d700 1 -- 172.21.15.55:0/172557544 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3540cd0b0 con 0x7fc354156310 2023-10-06T03:49:07.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.976+0000 7fc350ff9700 1 -- 172.21.15.55:0/172557544 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+17512 (secure 0 0 0) 0x7fc340054f50 con 0x7fc354156310 2023-10-06T03:49:07.978 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:07.978 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":93,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:49:05.060164+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:49:01.721643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T03:49:02.520120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:49:07.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.978+0000 7fc35999d700 1 -- 172.21.15.55:0/172557544 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc338066f30 msgr2=0x7fc3380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:07.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.978+0000 7fc35999d700 1 --2- 172.21.15.55:0/172557544 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc338066f30 0x7fc3380693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc3480099b0 tx=0x7fc348017000 comp rx=0 tx=0).stop 2023-10-06T03:49:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.979+0000 7fc35999d700 1 -- 172.21.15.55:0/172557544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc354156310 msgr2=0x7fc35416b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.979+0000 7fc35999d700 1 --2- 172.21.15.55:0/172557544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc354156310 0x7fc35416b920 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc34000ec50 tx=0x7fc3400091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.979+0000 7fc35999d700 1 -- 172.21.15.55:0/172557544 shutdown_connections 2023-10-06T03:49:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.979+0000 7fc35999d700 1 --2- 172.21.15.55:0/172557544 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc338066f30 0x7fc3380693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.979+0000 7fc35999d700 1 --2- 172.21.15.55:0/172557544 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3541566e0 0x7fc35416be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.979+0000 7fc35999d700 1 --2- 172.21.15.55:0/172557544 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3540ca300 0x7fc3541467f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.979+0000 7fc35999d700 1 --2- 172.21.15.55:0/172557544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc354156310 0x7fc35416b920 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.979+0000 7fc35999d700 1 -- 172.21.15.55:0/172557544 >> 172.21.15.55:0/172557544 conn(0x7fc3540cc1f0 msgr2=0x7fc35406ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.979+0000 7fc35999d700 1 -- 172.21.15.55:0/172557544 shutdown_connections 2023-10-06T03:49:07.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:07.979+0000 7fc35999d700 1 -- 172.21.15.55:0/172557544 wait complete. 2023-10-06T03:49:07.998 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:49:07.998 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:49:08.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.185+0000 7f63de79d700 1 Processor -- start 2023-10-06T03:49:08.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.185+0000 7f63de79d700 1 -- start start 2023-10-06T03:49:08.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.186+0000 7f63de79d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f63d81595a0 0x7f63d80b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:08.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.186+0000 7f63de79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d80c6c60 0x7f63d80c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:08.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.186+0000 7f63de79d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f63d80b1de0 0x7f63d80c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:08.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.186+0000 7f63de79d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d80b2b90 con 0x7f63d81595a0 2023-10-06T03:49:08.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.186+0000 7f63de79d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d80b1410 con 0x7f63d80b1de0 2023-10-06T03:49:08.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.186+0000 7f63de79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8063230 con 0x7f63d80c6c60 2023-10-06T03:49:08.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.187+0000 7f63dd79b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f63d81595a0 0x7f63d80b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:08.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.187+0000 7f63dd79b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f63d81595a0 0x7f63d80b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50052/0 (socket says 172.21.15.55:50052) 2023-10-06T03:49:08.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.187+0000 7f63dd79b700 1 -- 172.21.15.55:0/1442367727 learned_addr learned my addr 172.21.15.55:0/1442367727 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:08.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.187+0000 7f63ddf9c700 1 --2- 172.21.15.55:0/1442367727 >> v2:172.21.15.163:3300/0 conn(0x7f63d80b1de0 0x7f63d80c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:08.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.187+0000 7f63dcf9a700 1 --2- 172.21.15.55:0/1442367727 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d80c6c60 0x7f63d80c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:08.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.187+0000 7f63dd79b700 1 -- 172.21.15.55:0/1442367727 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d80c6c60 msgr2=0x7f63d80c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:08.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.187+0000 7f63dd79b700 1 --2- 172.21.15.55:0/1442367727 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d80c6c60 0x7f63d80c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.187+0000 7f63dd79b700 1 -- 172.21.15.55:0/1442367727 >> v2:172.21.15.163:3300/0 conn(0x7f63d80b1de0 msgr2=0x7f63d80c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:08.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.187+0000 7f63dd79b700 1 --2- 172.21.15.55:0/1442367727 >> v2:172.21.15.163:3300/0 conn(0x7f63d80b1de0 0x7f63d80c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.187+0000 7f63dd79b700 1 -- 172.21.15.55:0/1442367727 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d80634f0 con 0x7f63d81595a0 2023-10-06T03:49:08.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.187+0000 7f63dcf9a700 1 --2- 172.21.15.55:0/1442367727 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d80c6c60 0x7f63d80c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:08.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.188+0000 7f63dd79b700 1 --2- 172.21.15.55:0/1442367727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d81595a0 0x7f63d80b18a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f63d815d3c0 tx=0x7f63d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2cddeb3a88178a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:08.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.188+0000 7f63cffff700 1 -- 172.21.15.55:0/1442367727 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d4009e70 con 0x7f63d81595a0 2023-10-06T03:49:08.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.188+0000 7f63cffff700 1 -- 172.21.15.55:0/1442367727 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d400b860 con 0x7f63d81595a0 2023-10-06T03:49:08.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.188+0000 7f63cffff700 1 -- 172.21.15.55:0/1442367727 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d400bc00 con 0x7f63d81595a0 2023-10-06T03:49:08.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.188+0000 7f63de79d700 1 -- 172.21.15.55:0/1442367727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d81595a0 msgr2=0x7f63d80b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:08.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.188+0000 7f63de79d700 1 --2- 172.21.15.55:0/1442367727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d81595a0 0x7f63d80b18a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f63d815d3c0 tx=0x7f63d4009250 comp rx=0 tx=0).stop 2023-10-06T03:49:08.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.189+0000 7f63de79d700 1 -- 172.21.15.55:0/1442367727 shutdown_connections 2023-10-06T03:49:08.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.189+0000 7f63de79d700 1 --2- 172.21.15.55:0/1442367727 >> v2:172.21.15.163:3300/0 conn(0x7f63d80b1de0 0x7f63d80c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.189+0000 7f63de79d700 1 --2- 172.21.15.55:0/1442367727 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d81595a0 0x7f63d80b18a0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.189+0000 7f63de79d700 1 --2- 172.21.15.55:0/1442367727 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d80c6c60 0x7f63d80c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.189+0000 7f63de79d700 1 -- 172.21.15.55:0/1442367727 >> 172.21.15.55:0/1442367727 conn(0x7f63d806ddb0 msgr2=0x7f63d806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:08.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.189+0000 7f63de79d700 1 -- 172.21.15.55:0/1442367727 shutdown_connections 2023-10-06T03:49:08.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.189+0000 7f63de79d700 1 -- 172.21.15.55:0/1442367727 wait complete. 2023-10-06T03:49:08.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.190+0000 7f63de79d700 1 Processor -- start 2023-10-06T03:49:08.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.190+0000 7f63de79d700 1 -- start start 2023-10-06T03:49:08.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.191+0000 7f63de79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d80b1de0 0x7f63d814ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:08.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.191+0000 7f63de79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d80c6c60 0x7f63d814eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:08.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.191+0000 7f63de79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d81595a0 0x7f63d814efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:08.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.191+0000 7f63de79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8151c50 con 0x7f63d80c6c60 2023-10-06T03:49:08.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.191+0000 7f63de79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d8150da0 con 0x7f63d81595a0 2023-10-06T03:49:08.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.191+0000 7f63de79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80c8810 con 0x7f63d80b1de0 2023-10-06T03:49:08.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.191+0000 7f63dcf9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d80c6c60 0x7f63d814eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:08.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.191+0000 7f63dcf9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d80c6c60 0x7f63d814eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50066/0 (socket says 172.21.15.55:50066) 2023-10-06T03:49:08.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.191+0000 7f63dcf9a700 1 -- 172.21.15.55:0/106636441 learned_addr learned my addr 172.21.15.55:0/106636441 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:08.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.191+0000 7f63ddf9c700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d81595a0 0x7f63d814efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:08.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.191+0000 7f63dd79b700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d80b1de0 0x7f63d814ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:08.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.192+0000 7f63dcf9a700 1 -- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d80b1de0 msgr2=0x7f63d814ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:08.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.192+0000 7f63dcf9a700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d80b1de0 0x7f63d814ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.192+0000 7f63dcf9a700 1 -- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d81595a0 msgr2=0x7f63d814efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:08.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.192+0000 7f63dcf9a700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d81595a0 0x7f63d814efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.192+0000 7f63dcf9a700 1 -- 172.21.15.55:0/106636441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d40094f0 con 0x7f63d80c6c60 2023-10-06T03:49:08.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.192+0000 7f63ddf9c700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d81595a0 0x7f63d814efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:08.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.192+0000 7f63dcf9a700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d80c6c60 0x7f63d814eaa0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f63c800e8d0 tx=0x7f63c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:08.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.192+0000 7f63cd7fa700 1 -- 172.21.15.55:0/106636441 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c800ee70 con 0x7f63d80c6c60 2023-10-06T03:49:08.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.192+0000 7f63cd7fa700 1 -- 172.21.15.55:0/106636441 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c800c5e0 con 0x7f63d80c6c60 2023-10-06T03:49:08.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.192+0000 7f63de79d700 1 -- 172.21.15.55:0/106636441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d8151040 con 0x7f63d80c6c60 2023-10-06T03:49:08.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.192+0000 7f63cd7fa700 1 -- 172.21.15.55:0/106636441 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c800c980 con 0x7f63d80c6c60 2023-10-06T03:49:08.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.192+0000 7f63de79d700 1 -- 172.21.15.55:0/106636441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d8151600 con 0x7f63d80c6c60 2023-10-06T03:49:08.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.194+0000 7f63de79d700 1 -- 172.21.15.55:0/106636441 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d80ccec0 con 0x7f63d80c6c60 2023-10-06T03:49:08.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.194+0000 7f63cd7fa700 1 -- 172.21.15.55:0/106636441 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f63c8013410 con 0x7f63d80c6c60 2023-10-06T03:49:08.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.195+0000 7f63cd7fa700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f63bc066f30 0x7f63bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:08.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.195+0000 7f63cd7fa700 1 -- 172.21.15.55:0/106636441 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f63c8084610 con 0x7f63d80c6c60 2023-10-06T03:49:08.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.196+0000 7f63dd79b700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f63bc066f30 0x7f63bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:08.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.196+0000 7f63dd79b700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f63bc066f30 0x7f63bc0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f63d815d3c0 tx=0x7f63d4017000 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:08.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.200+0000 7f63cd7fa700 1 -- 172.21.15.55:0/106636441 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f63c8051590 con 0x7f63d80c6c60 2023-10-06T03:49:08.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.398+0000 7f63de79d700 1 -- 172.21.15.55:0/106636441 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63d8151d90 con 0x7f63bc066f30 2023-10-06T03:49:08.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.410+0000 7f63cd7fa700 1 -- 172.21.15.55:0/106636441 <== mgr.5828 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311313 (secure 0 0 0) 0x7f63d8151d90 con 0x7f63bc066f30 2023-10-06T03:49:08.415 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:08.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.414+0000 7f63de79d700 1 -- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f63bc066f30 msgr2=0x7f63bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:08.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.414+0000 7f63de79d700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f63bc066f30 0x7f63bc0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f63d815d3c0 tx=0x7f63d4017000 comp rx=0 tx=0).stop 2023-10-06T03:49:08.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.414+0000 7f63de79d700 1 -- 172.21.15.55:0/106636441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d80c6c60 msgr2=0x7f63d814eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:08.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.414+0000 7f63de79d700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d80c6c60 0x7f63d814eaa0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f63c800e8d0 tx=0x7f63c8008040 comp rx=0 tx=0).stop 2023-10-06T03:49:08.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.414+0000 7f63de79d700 1 -- 172.21.15.55:0/106636441 shutdown_connections 2023-10-06T03:49:08.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.414+0000 7f63de79d700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f63bc066f30 0x7f63bc0693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.414+0000 7f63de79d700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f63d81595a0 0x7f63d814efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.414+0000 7f63de79d700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f63d80b1de0 0x7f63d814ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.414+0000 7f63de79d700 1 --2- 172.21.15.55:0/106636441 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f63d80c6c60 0x7f63d814eaa0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.414+0000 7f63de79d700 1 -- 172.21.15.55:0/106636441 >> 172.21.15.55:0/106636441 conn(0x7f63d806ddb0 msgr2=0x7f63d8153ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:08.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.414+0000 7f63de79d700 1 -- 172.21.15.55:0/106636441 shutdown_connections 2023-10-06T03:49:08.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.414+0000 7f63de79d700 1 -- 172.21.15.55:0/106636441 wait complete. 2023-10-06T03:49:08.419 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:49:08.434 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-06T03:49:07.000487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":323,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2449780,"kb_used_data":10036,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752524940,"statfs":{"total":773094113280,"available":770585538560,"internally_reserved":0,"allocated":10276864,"data_stored":5756121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.958805"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068124+0000","last_change":"2023-10-06T03:49:04.065554+0000","last_active":"2023-10-06T03:49:04.068124+0000","last_peered":"2023-10-06T03:49:04.068124+0000","last_clean":"2023-10-06T03:49:04.068124+0000","last_became_active":"2023-10-06T03:49:04.065210+0000","last_became_peered":"2023-10-06T03:49:04.065210+0000","last_unstale":"2023-10-06T03:49:04.068124+0000","last_undegraded":"2023-10-06T03:49:04.068124+0000","last_fullsized":"2023-10-06T03:49:04.068124+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:57.778267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.078692+0000","last_change":"2023-10-06T03:49:04.076365+0000","last_active":"2023-10-06T03:49:04.078692+0000","last_peered":"2023-10-06T03:49:04.078692+0000","last_clean":"2023-10-06T03:49:04.078692+0000","last_became_active":"2023-10-06T03:49:04.076117+0000","last_became_peered":"2023-10-06T03:49:04.076117+0000","last_unstale":"2023-10-06T03:49:04.078692+0000","last_undegraded":"2023-10-06T03:49:04.078692+0000","last_fullsized":"2023-10-06T03:49:04.078692+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:24.383164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.080109+0000","last_change":"2023-10-06T03:49:04.066770+0000","last_active":"2023-10-06T03:49:04.080109+0000","last_peered":"2023-10-06T03:49:04.080109+0000","last_clean":"2023-10-06T03:49:04.080109+0000","last_became_active":"2023-10-06T03:49:04.066524+0000","last_became_peered":"2023-10-06T03:49:04.066524+0000","last_unstale":"2023-10-06T03:49:04.080109+0000","last_undegraded":"2023-10-06T03:49:04.080109+0000","last_fullsized":"2023-10-06T03:49:04.080109+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:18.788347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068129+0000","last_change":"2023-10-06T03:49:04.064296+0000","last_active":"2023-10-06T03:49:04.068129+0000","last_peered":"2023-10-06T03:49:04.068129+0000","last_clean":"2023-10-06T03:49:04.068129+0000","last_became_active":"2023-10-06T03:49:04.064047+0000","last_became_peered":"2023-10-06T03:49:04.064047+0000","last_unstale":"2023-10-06T03:49:04.068129+0000","last_undegraded":"2023-10-06T03:49:04.068129+0000","last_fullsized":"2023-10-06T03:49:04.068129+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:24:55.747936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.076623+0000","last_change":"2023-10-06T03:49:04.072399+0000","last_active":"2023-10-06T03:49:04.076623+0000","last_peered":"2023-10-06T03:49:04.076623+0000","last_clean":"2023-10-06T03:49:04.076623+0000","last_became_active":"2023-10-06T03:49:04.071660+0000","last_became_peered":"2023-10-06T03:49:04.071660+0000","last_unstale":"2023-10-06T03:49:04.076623+0000","last_undegraded":"2023-10-06T03:49:04.076623+0000","last_fullsized":"2023-10-06T03:49:04.076623+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:08.261321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068850+0000","last_change":"2023-10-06T03:49:04.066198+0000","last_active":"2023-10-06T03:49:04.068850+0000","last_peered":"2023-10-06T03:49:04.068850+0000","last_clean":"2023-10-06T03:49:04.068850+0000","last_became_active":"2023-10-06T03:49:04.065564+0000","last_became_peered":"2023-10-06T03:49:04.065564+0000","last_unstale":"2023-10-06T03:49:04.068850+0000","last_undegraded":"2023-10-06T03:49:04.068850+0000","last_fullsized":"2023-10-06T03:49:04.068850+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:30.261703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.071301+0000","last_change":"2023-10-06T03:49:04.068089+0000","last_active":"2023-10-06T03:49:04.071301+0000","last_peered":"2023-10-06T03:49:04.071301+0000","last_clean":"2023-10-06T03:49:04.071301+0000","last_became_active":"2023-10-06T03:49:04.067696+0000","last_became_peered":"2023-10-06T03:49:04.067696+0000","last_unstale":"2023-10-06T03:49:04.071301+0000","last_undegraded":"2023-10-06T03:49:04.071301+0000","last_fullsized":"2023-10-06T03:49:04.071301+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:51.305962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068241+0000","last_change":"2023-10-06T03:49:04.060653+0000","last_active":"2023-10-06T03:49:04.068241+0000","last_peered":"2023-10-06T03:49:04.068241+0000","last_clean":"2023-10-06T03:49:04.068241+0000","last_became_active":"2023-10-06T03:49:04.060322+0000","last_became_peered":"2023-10-06T03:49:04.060322+0000","last_unstale":"2023-10-06T03:49:04.068241+0000","last_undegraded":"2023-10-06T03:49:04.068241+0000","last_fullsized":"2023-10-06T03:49:04.068241+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:25.386744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.073113+0000","last_change":"2023-10-06T03:49:04.069230+0000","last_active":"2023-10-06T03:49:04.073113+0000","last_peered":"2023-10-06T03:49:04.073113+0000","last_clean":"2023-10-06T03:49:04.073113+0000","last_became_active":"2023-10-06T03:49:04.068964+0000","last_became_peered":"2023-10-06T03:49:04.068964+0000","last_unstale":"2023-10-06T03:49:04.073113+0000","last_undegraded":"2023-10-06T03:49:04.073113+0000","last_fullsized":"2023-10-06T03:49:04.073113+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:05.297394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.071218+0000","last_change":"2023-10-06T03:49:04.067370+0000","last_active":"2023-10-06T03:49:04.071218+0000","last_peered":"2023-10-06T03:49:04.071218+0000","last_clean":"2023-10-06T03:49:04.071218+0000","last_became_active":"2023-10-06T03:49:04.067115+0000","last_became_peered":"2023-10-06T03:49:04.067115+0000","last_unstale":"2023-10-06T03:49:04.071218+0000","last_undegraded":"2023-10-06T03:49:04.071218+0000","last_fullsized":"2023-10-06T03:49:04.071218+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:34.308700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.078147+0000","last_change":"2023-10-06T03:49:04.068374+0000","last_active":"2023-10-06T03:49:04.078147+0000","last_peered":"2023-10-06T03:49:04.078147+0000","last_clean":"2023-10-06T03:49:04.078147+0000","last_became_active":"2023-10-06T03:49:04.068115+0000","last_became_peered":"2023-10-06T03:49:04.068115+0000","last_unstale":"2023-10-06T03:49:04.078147+0000","last_undegraded":"2023-10-06T03:49:04.078147+0000","last_fullsized":"2023-10-06T03:49:04.078147+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:22.193560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.081875+0000","last_change":"2023-10-06T03:49:04.076378+0000","last_active":"2023-10-06T03:49:04.081875+0000","last_peered":"2023-10-06T03:49:04.081875+0000","last_clean":"2023-10-06T03:49:04.081875+0000","last_became_active":"2023-10-06T03:49:04.076097+0000","last_became_peered":"2023-10-06T03:49:04.076097+0000","last_unstale":"2023-10-06T03:49:04.081875+0000","last_undegraded":"2023-10-06T03:49:04.081875+0000","last_fullsized":"2023-10-06T03:49:04.081875+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:35.759692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.076602+0000","last_change":"2023-10-06T03:49:04.072653+0000","last_active":"2023-10-06T03:49:04.076602+0000","last_peered":"2023-10-06T03:49:04.076602+0000","last_clean":"2023-10-06T03:49:04.076602+0000","last_became_active":"2023-10-06T03:49:04.072101+0000","last_became_peered":"2023-10-06T03:49:04.072101+0000","last_unstale":"2023-10-06T03:49:04.076602+0000","last_undegraded":"2023-10-06T03:49:04.076602+0000","last_fullsized":"2023-10-06T03:49:04.076602+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:06.692485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068869+0000","last_change":"2023-10-06T03:49:04.066366+0000","last_active":"2023-10-06T03:49:04.068869+0000","last_peered":"2023-10-06T03:49:04.068869+0000","last_clean":"2023-10-06T03:49:04.068869+0000","last_became_active":"2023-10-06T03:49:04.065947+0000","last_became_peered":"2023-10-06T03:49:04.065947+0000","last_unstale":"2023-10-06T03:49:04.068869+0000","last_undegraded":"2023-10-06T03:49:04.068869+0000","last_fullsized":"2023-10-06T03:49:04.068869+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:32.178184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.078650+0000","last_change":"2023-10-06T03:49:04.075319+0000","last_active":"2023-10-06T03:49:04.078650+0000","last_peered":"2023-10-06T03:49:04.078650+0000","last_clean":"2023-10-06T03:49:04.078650+0000","last_became_active":"2023-10-06T03:49:04.075038+0000","last_became_peered":"2023-10-06T03:49:04.075038+0000","last_unstale":"2023-10-06T03:49:04.078650+0000","last_undegraded":"2023-10-06T03:49:04.078650+0000","last_fullsized":"2023-10-06T03:49:04.078650+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:22.938029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068249+0000","last_change":"2023-10-06T03:49:04.061999+0000","last_active":"2023-10-06T03:49:04.068249+0000","last_peered":"2023-10-06T03:49:04.068249+0000","last_clean":"2023-10-06T03:49:04.068249+0000","last_became_active":"2023-10-06T03:49:04.061737+0000","last_became_peered":"2023-10-06T03:49:04.061737+0000","last_unstale":"2023-10-06T03:49:04.068249+0000","last_undegraded":"2023-10-06T03:49:04.068249+0000","last_fullsized":"2023-10-06T03:49:04.068249+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:04.485978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.076700+0000","last_change":"2023-10-06T03:49:04.074276+0000","last_active":"2023-10-06T03:49:04.076700+0000","last_peered":"2023-10-06T03:49:04.076700+0000","last_clean":"2023-10-06T03:49:04.076700+0000","last_became_active":"2023-10-06T03:49:04.074022+0000","last_became_peered":"2023-10-06T03:49:04.074022+0000","last_unstale":"2023-10-06T03:49:04.076700+0000","last_undegraded":"2023-10-06T03:49:04.076700+0000","last_fullsized":"2023-10-06T03:49:04.076700+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:58.686045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.079832+0000","last_change":"2023-10-06T03:49:04.064954+0000","last_active":"2023-10-06T03:49:04.079832+0000","last_peered":"2023-10-06T03:49:04.079832+0000","last_clean":"2023-10-06T03:49:04.079832+0000","last_became_active":"2023-10-06T03:49:04.064631+0000","last_became_peered":"2023-10-06T03:49:04.064631+0000","last_unstale":"2023-10-06T03:49:04.079832+0000","last_undegraded":"2023-10-06T03:49:04.079832+0000","last_fullsized":"2023-10-06T03:49:04.079832+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:16.640709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068465+0000","last_change":"2023-10-06T03:49:04.062640+0000","last_active":"2023-10-06T03:49:04.068465+0000","last_peered":"2023-10-06T03:49:04.068465+0000","last_clean":"2023-10-06T03:49:04.068465+0000","last_became_active":"2023-10-06T03:49:04.062382+0000","last_became_peered":"2023-10-06T03:49:04.062382+0000","last_unstale":"2023-10-06T03:49:04.068465+0000","last_undegraded":"2023-10-06T03:49:04.068465+0000","last_fullsized":"2023-10-06T03:49:04.068465+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:57.778267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.088169+0000","last_change":"2023-10-06T03:49:04.080997+0000","last_active":"2023-10-06T03:49:04.088169+0000","last_peered":"2023-10-06T03:49:04.088169+0000","last_clean":"2023-10-06T03:49:04.088169+0000","last_became_active":"2023-10-06T03:49:04.080670+0000","last_became_peered":"2023-10-06T03:49:04.080670+0000","last_unstale":"2023-10-06T03:49:04.088169+0000","last_undegraded":"2023-10-06T03:49:04.088169+0000","last_fullsized":"2023-10-06T03:49:04.088169+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:39.795313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.082121+0000","last_change":"2023-10-06T03:49:04.081071+0000","last_active":"2023-10-06T03:49:04.082121+0000","last_peered":"2023-10-06T03:49:04.082121+0000","last_clean":"2023-10-06T03:49:04.082121+0000","last_became_active":"2023-10-06T03:49:04.080774+0000","last_became_peered":"2023-10-06T03:49:04.080774+0000","last_unstale":"2023-10-06T03:49:04.082121+0000","last_undegraded":"2023-10-06T03:49:04.082121+0000","last_fullsized":"2023-10-06T03:49:04.082121+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:31.705749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.085732+0000","last_change":"2023-10-06T03:49:04.077572+0000","last_active":"2023-10-06T03:49:04.085732+0000","last_peered":"2023-10-06T03:49:04.085732+0000","last_clean":"2023-10-06T03:49:04.085732+0000","last_became_active":"2023-10-06T03:49:04.076972+0000","last_became_peered":"2023-10-06T03:49:04.076972+0000","last_unstale":"2023-10-06T03:49:04.085732+0000","last_undegraded":"2023-10-06T03:49:04.085732+0000","last_fullsized":"2023-10-06T03:49:04.085732+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:43.895946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.073382+0000","last_change":"2023-10-06T03:49:04.070565+0000","last_active":"2023-10-06T03:49:04.073382+0000","last_peered":"2023-10-06T03:49:04.073382+0000","last_clean":"2023-10-06T03:49:04.073382+0000","last_became_active":"2023-10-06T03:49:04.070277+0000","last_became_peered":"2023-10-06T03:49:04.070277+0000","last_unstale":"2023-10-06T03:49:04.073382+0000","last_undegraded":"2023-10-06T03:49:04.073382+0000","last_fullsized":"2023-10-06T03:49:04.073382+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:35.482312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.081189+0000","last_change":"2023-10-06T03:49:04.071898+0000","last_active":"2023-10-06T03:49:04.081189+0000","last_peered":"2023-10-06T03:49:04.081189+0000","last_clean":"2023-10-06T03:49:04.081189+0000","last_became_active":"2023-10-06T03:49:04.071614+0000","last_became_peered":"2023-10-06T03:49:04.071614+0000","last_unstale":"2023-10-06T03:49:04.081189+0000","last_undegraded":"2023-10-06T03:49:04.081189+0000","last_fullsized":"2023-10-06T03:49:04.081189+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:58.678773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.078354+0000","last_change":"2023-10-06T03:49:04.069270+0000","last_active":"2023-10-06T03:49:04.078354+0000","last_peered":"2023-10-06T03:49:04.078354+0000","last_clean":"2023-10-06T03:49:04.078354+0000","last_became_active":"2023-10-06T03:49:04.069012+0000","last_became_peered":"2023-10-06T03:49:04.069012+0000","last_unstale":"2023-10-06T03:49:04.078354+0000","last_undegraded":"2023-10-06T03:49:04.078354+0000","last_fullsized":"2023-10-06T03:49:04.078354+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:27.394068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.082109+0000","last_change":"2023-10-06T03:49:04.078356+0000","last_active":"2023-10-06T03:49:04.082109+0000","last_peered":"2023-10-06T03:49:04.082109+0000","last_clean":"2023-10-06T03:49:04.082109+0000","last_became_active":"2023-10-06T03:49:04.077998+0000","last_became_peered":"2023-10-06T03:49:04.077998+0000","last_unstale":"2023-10-06T03:49:04.082109+0000","last_undegraded":"2023-10-06T03:49:04.082109+0000","last_fullsized":"2023-10-06T03:49:04.082109+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:18.788347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068187+0000","last_change":"2023-10-06T03:49:04.061399+0000","last_active":"2023-10-06T03:49:04.068187+0000","last_peered":"2023-10-06T03:49:04.068187+0000","last_clean":"2023-10-06T03:49:04.068187+0000","last_became_active":"2023-10-06T03:49:04.061055+0000","last_became_peered":"2023-10-06T03:49:04.061055+0000","last_unstale":"2023-10-06T03:49:04.068187+0000","last_undegraded":"2023-10-06T03:49:04.068187+0000","last_fullsized":"2023-10-06T03:49:04.068187+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:16.695070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068658+0000","last_change":"2023-10-06T03:49:04.064314+0000","last_active":"2023-10-06T03:49:04.068658+0000","last_peered":"2023-10-06T03:49:04.068658+0000","last_clean":"2023-10-06T03:49:04.068658+0000","last_became_active":"2023-10-06T03:49:04.064046+0000","last_became_peered":"2023-10-06T03:49:04.064046+0000","last_unstale":"2023-10-06T03:49:04.068658+0000","last_undegraded":"2023-10-06T03:49:04.068658+0000","last_fullsized":"2023-10-06T03:49:04.068658+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:24.059522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.067705+0000","last_change":"2023-10-06T03:49:04.066832+0000","last_active":"2023-10-06T03:49:04.067705+0000","last_peered":"2023-10-06T03:49:04.067705+0000","last_clean":"2023-10-06T03:49:04.067705+0000","last_became_active":"2023-10-06T03:49:04.066453+0000","last_became_peered":"2023-10-06T03:49:04.066453+0000","last_unstale":"2023-10-06T03:49:04.067705+0000","last_undegraded":"2023-10-06T03:49:04.067705+0000","last_fullsized":"2023-10-06T03:49:04.067705+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:24.450111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068214+0000","last_change":"2023-10-06T03:49:04.066022+0000","last_active":"2023-10-06T03:49:04.068214+0000","last_peered":"2023-10-06T03:49:04.068214+0000","last_clean":"2023-10-06T03:49:04.068214+0000","last_became_active":"2023-10-06T03:49:04.065244+0000","last_became_peered":"2023-10-06T03:49:04.065244+0000","last_unstale":"2023-10-06T03:49:04.068214+0000","last_undegraded":"2023-10-06T03:49:04.068214+0000","last_fullsized":"2023-10-06T03:49:04.068214+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:32.178184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.080828+0000","last_change":"2023-10-06T03:49:04.060812+0000","last_active":"2023-10-06T03:49:04.080828+0000","last_peered":"2023-10-06T03:49:04.080828+0000","last_clean":"2023-10-06T03:49:04.080828+0000","last_became_active":"2023-10-06T03:49:04.060548+0000","last_became_peered":"2023-10-06T03:49:04.060548+0000","last_unstale":"2023-10-06T03:49:04.080828+0000","last_undegraded":"2023-10-06T03:49:04.080828+0000","last_fullsized":"2023-10-06T03:49:04.080828+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:24:55.747936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068276+0000","last_change":"2023-10-06T03:49:04.064819+0000","last_active":"2023-10-06T03:49:04.068276+0000","last_peered":"2023-10-06T03:49:04.068276+0000","last_clean":"2023-10-06T03:49:04.068276+0000","last_became_active":"2023-10-06T03:49:04.064569+0000","last_became_peered":"2023-10-06T03:49:04.064569+0000","last_unstale":"2023-10-06T03:49:04.068276+0000","last_undegraded":"2023-10-06T03:49:04.068276+0000","last_fullsized":"2023-10-06T03:49:04.068276+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:05.297394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.079909+0000","last_change":"2023-10-06T03:49:04.064992+0000","last_active":"2023-10-06T03:49:04.079909+0000","last_peered":"2023-10-06T03:49:04.079909+0000","last_clean":"2023-10-06T03:49:04.079909+0000","last_became_active":"2023-10-06T03:49:04.064713+0000","last_became_peered":"2023-10-06T03:49:04.064713+0000","last_unstale":"2023-10-06T03:49:04.079909+0000","last_undegraded":"2023-10-06T03:49:04.079909+0000","last_fullsized":"2023-10-06T03:49:04.079909+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:04.277911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"92'4","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.089818+0000","last_change":"2023-10-06T03:49:03.073209+0000","last_active":"2023-10-06T03:49:04.089818+0000","last_peered":"2023-10-06T03:49:04.089818+0000","last_clean":"2023-10-06T03:49:04.089818+0000","last_became_active":"2023-10-06T03:49:03.072812+0000","last_became_peered":"2023-10-06T03:49:03.072812+0000","last_unstale":"2023-10-06T03:49:04.089818+0000","last_undegraded":"2023-10-06T03:49:04.089818+0000","last_fullsized":"2023-10-06T03:49:04.089818+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:53.076107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.078407+0000","last_change":"2023-10-06T03:49:04.072899+0000","last_active":"2023-10-06T03:49:04.078407+0000","last_peered":"2023-10-06T03:49:04.078407+0000","last_clean":"2023-10-06T03:49:04.078407+0000","last_became_active":"2023-10-06T03:49:04.072617+0000","last_became_peered":"2023-10-06T03:49:04.072617+0000","last_unstale":"2023-10-06T03:49:04.078407+0000","last_undegraded":"2023-10-06T03:49:04.078407+0000","last_fullsized":"2023-10-06T03:49:04.078407+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:50.496559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.082278+0000","last_change":"2023-10-06T03:49:03.062672+0000","last_active":"2023-10-06T03:49:04.082278+0000","last_peered":"2023-10-06T03:49:04.082278+0000","last_clean":"2023-10-06T03:49:04.082278+0000","last_became_active":"2023-10-06T03:49:03.062414+0000","last_became_peered":"2023-10-06T03:49:03.062414+0000","last_unstale":"2023-10-06T03:49:04.082278+0000","last_undegraded":"2023-10-06T03:49:04.082278+0000","last_fullsized":"2023-10-06T03:49:04.082278+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:46:47.404382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.073144+0000","last_change":"2023-10-06T03:49:04.070292+0000","last_active":"2023-10-06T03:49:04.073144+0000","last_peered":"2023-10-06T03:49:04.073144+0000","last_clean":"2023-10-06T03:49:04.073144+0000","last_became_active":"2023-10-06T03:49:04.070052+0000","last_became_peered":"2023-10-06T03:49:04.070052+0000","last_unstale":"2023-10-06T03:49:04.073144+0000","last_undegraded":"2023-10-06T03:49:04.073144+0000","last_fullsized":"2023-10-06T03:49:04.073144+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:04.485978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.085898+0000","last_change":"2023-10-06T03:49:03.068734+0000","last_active":"2023-10-06T03:49:04.085898+0000","last_peered":"2023-10-06T03:49:04.085898+0000","last_clean":"2023-10-06T03:49:04.085898+0000","last_became_active":"2023-10-06T03:49:03.068038+0000","last_became_peered":"2023-10-06T03:49:03.068038+0000","last_unstale":"2023-10-06T03:49:04.085898+0000","last_undegraded":"2023-10-06T03:49:04.085898+0000","last_fullsized":"2023-10-06T03:49:04.085898+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:08.871796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.082141+0000","last_change":"2023-10-06T03:49:04.080850+0000","last_active":"2023-10-06T03:49:04.082141+0000","last_peered":"2023-10-06T03:49:04.082141+0000","last_clean":"2023-10-06T03:49:04.082141+0000","last_became_active":"2023-10-06T03:49:04.080550+0000","last_became_peered":"2023-10-06T03:49:04.080550+0000","last_unstale":"2023-10-06T03:49:04.082141+0000","last_undegraded":"2023-10-06T03:49:04.082141+0000","last_fullsized":"2023-10-06T03:49:04.082141+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:13.885949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.082423+0000","last_change":"2023-10-06T03:49:03.073041+0000","last_active":"2023-10-06T03:49:04.082423+0000","last_peered":"2023-10-06T03:49:04.082423+0000","last_clean":"2023-10-06T03:49:04.082423+0000","last_became_active":"2023-10-06T03:49:03.072744+0000","last_became_peered":"2023-10-06T03:49:03.072744+0000","last_unstale":"2023-10-06T03:49:04.082423+0000","last_undegraded":"2023-10-06T03:49:04.082423+0000","last_fullsized":"2023-10-06T03:49:04.082423+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:42.642374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.079791+0000","last_change":"2023-10-06T03:49:04.076114+0000","last_active":"2023-10-06T03:49:04.079791+0000","last_peered":"2023-10-06T03:49:04.079791+0000","last_clean":"2023-10-06T03:49:04.079791+0000","last_became_active":"2023-10-06T03:49:04.075683+0000","last_became_peered":"2023-10-06T03:49:04.075683+0000","last_unstale":"2023-10-06T03:49:04.079791+0000","last_undegraded":"2023-10-06T03:49:04.079791+0000","last_fullsized":"2023-10-06T03:49:04.079791+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:29.832334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.084368+0000","last_change":"2023-10-06T03:49:03.068940+0000","last_active":"2023-10-06T03:49:04.084368+0000","last_peered":"2023-10-06T03:49:04.084368+0000","last_clean":"2023-10-06T03:49:04.084368+0000","last_became_active":"2023-10-06T03:49:03.068686+0000","last_became_peered":"2023-10-06T03:49:03.068686+0000","last_unstale":"2023-10-06T03:49:04.084368+0000","last_undegraded":"2023-10-06T03:49:04.084368+0000","last_fullsized":"2023-10-06T03:49:04.084368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:01.527205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.073446+0000","last_change":"2023-10-06T03:49:04.067693+0000","last_active":"2023-10-06T03:49:04.073446+0000","last_peered":"2023-10-06T03:49:04.073446+0000","last_clean":"2023-10-06T03:49:04.073446+0000","last_became_active":"2023-10-06T03:49:04.067419+0000","last_became_peered":"2023-10-06T03:49:04.067419+0000","last_unstale":"2023-10-06T03:49:04.073446+0000","last_undegraded":"2023-10-06T03:49:04.073446+0000","last_fullsized":"2023-10-06T03:49:04.073446+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:09.342388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.081729+0000","last_change":"2023-10-06T03:49:03.069610+0000","last_active":"2023-10-06T03:49:04.081729+0000","last_peered":"2023-10-06T03:49:04.081729+0000","last_clean":"2023-10-06T03:49:04.081729+0000","last_became_active":"2023-10-06T03:49:03.069160+0000","last_became_peered":"2023-10-06T03:49:03.069160+0000","last_unstale":"2023-10-06T03:49:04.081729+0000","last_undegraded":"2023-10-06T03:49:04.081729+0000","last_fullsized":"2023-10-06T03:49:04.081729+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:19.852270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.073393+0000","last_change":"2023-10-06T03:49:04.070720+0000","last_active":"2023-10-06T03:49:04.073393+0000","last_peered":"2023-10-06T03:49:04.073393+0000","last_clean":"2023-10-06T03:49:04.073393+0000","last_became_active":"2023-10-06T03:49:04.070458+0000","last_became_peered":"2023-10-06T03:49:04.070458+0000","last_unstale":"2023-10-06T03:49:04.073393+0000","last_undegraded":"2023-10-06T03:49:04.073393+0000","last_fullsized":"2023-10-06T03:49:04.073393+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:57.778267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.054993+0000","last_change":"2023-10-06T03:49:03.073187+0000","last_active":"2023-10-06T03:49:04.054993+0000","last_peered":"2023-10-06T03:49:04.054993+0000","last_clean":"2023-10-06T03:49:04.054993+0000","last_became_active":"2023-10-06T03:49:03.072757+0000","last_became_peered":"2023-10-06T03:49:03.072757+0000","last_unstale":"2023-10-06T03:49:04.054993+0000","last_undegraded":"2023-10-06T03:49:04.054993+0000","last_fullsized":"2023-10-06T03:49:04.054993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:15.686739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.071562+0000","last_change":"2023-10-06T03:49:04.069365+0000","last_active":"2023-10-06T03:49:04.071562+0000","last_peered":"2023-10-06T03:49:04.071562+0000","last_clean":"2023-10-06T03:49:04.071562+0000","last_became_active":"2023-10-06T03:49:04.069092+0000","last_became_peered":"2023-10-06T03:49:04.069092+0000","last_unstale":"2023-10-06T03:49:04.071562+0000","last_undegraded":"2023-10-06T03:49:04.071562+0000","last_fullsized":"2023-10-06T03:49:04.071562+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:21.957625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.058262+0000","last_change":"2023-10-06T03:49:03.065475+0000","last_active":"2023-10-06T03:49:04.058262+0000","last_peered":"2023-10-06T03:49:04.058262+0000","last_clean":"2023-10-06T03:49:04.058262+0000","last_became_active":"2023-10-06T03:49:03.065157+0000","last_became_peered":"2023-10-06T03:49:03.065157+0000","last_unstale":"2023-10-06T03:49:04.058262+0000","last_undegraded":"2023-10-06T03:49:04.058262+0000","last_fullsized":"2023-10-06T03:49:04.058262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:53.030591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.081209+0000","last_change":"2023-10-06T03:49:04.075427+0000","last_active":"2023-10-06T03:49:04.081209+0000","last_peered":"2023-10-06T03:49:04.081209+0000","last_clean":"2023-10-06T03:49:04.081209+0000","last_became_active":"2023-10-06T03:49:04.075161+0000","last_became_peered":"2023-10-06T03:49:04.075161+0000","last_unstale":"2023-10-06T03:49:04.081209+0000","last_undegraded":"2023-10-06T03:49:04.081209+0000","last_fullsized":"2023-10-06T03:49:04.081209+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:08.202087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.051552+0000","last_change":"2023-10-06T03:49:03.064156+0000","last_active":"2023-10-06T03:49:04.051552+0000","last_peered":"2023-10-06T03:49:04.051552+0000","last_clean":"2023-10-06T03:49:04.051552+0000","last_became_active":"2023-10-06T03:49:03.063871+0000","last_became_peered":"2023-10-06T03:49:03.063871+0000","last_unstale":"2023-10-06T03:49:04.051552+0000","last_undegraded":"2023-10-06T03:49:04.051552+0000","last_fullsized":"2023-10-06T03:49:04.051552+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:04.748459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.070945+0000","last_change":"2023-10-06T03:49:04.068921+0000","last_active":"2023-10-06T03:49:04.070945+0000","last_peered":"2023-10-06T03:49:04.070945+0000","last_clean":"2023-10-06T03:49:04.070945+0000","last_became_active":"2023-10-06T03:49:04.068675+0000","last_became_peered":"2023-10-06T03:49:04.068675+0000","last_unstale":"2023-10-06T03:49:04.070945+0000","last_undegraded":"2023-10-06T03:49:04.070945+0000","last_fullsized":"2023-10-06T03:49:04.070945+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:03.359619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.058199+0000","last_change":"2023-10-06T03:49:03.073229+0000","last_active":"2023-10-06T03:49:04.058199+0000","last_peered":"2023-10-06T03:49:04.058199+0000","last_clean":"2023-10-06T03:49:04.058199+0000","last_became_active":"2023-10-06T03:49:03.072966+0000","last_became_peered":"2023-10-06T03:49:03.072966+0000","last_unstale":"2023-10-06T03:49:04.058199+0000","last_undegraded":"2023-10-06T03:49:04.058199+0000","last_fullsized":"2023-10-06T03:49:04.058199+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:42.656195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068909+0000","last_change":"2023-10-06T03:49:04.066381+0000","last_active":"2023-10-06T03:49:04.068909+0000","last_peered":"2023-10-06T03:49:04.068909+0000","last_clean":"2023-10-06T03:49:04.068909+0000","last_became_active":"2023-10-06T03:49:04.065557+0000","last_became_peered":"2023-10-06T03:49:04.065557+0000","last_unstale":"2023-10-06T03:49:04.068909+0000","last_undegraded":"2023-10-06T03:49:04.068909+0000","last_fullsized":"2023-10-06T03:49:04.068909+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:05.297394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"92'3","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.089527+0000","last_change":"2023-10-06T03:49:03.064278+0000","last_active":"2023-10-06T03:49:04.089527+0000","last_peered":"2023-10-06T03:49:04.089527+0000","last_clean":"2023-10-06T03:49:04.089527+0000","last_became_active":"2023-10-06T03:49:03.064016+0000","last_became_peered":"2023-10-06T03:49:03.064016+0000","last_unstale":"2023-10-06T03:49:04.089527+0000","last_undegraded":"2023-10-06T03:49:04.089527+0000","last_fullsized":"2023-10-06T03:49:04.089527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:53.314723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.080369+0000","last_change":"2023-10-06T03:49:04.070264+0000","last_active":"2023-10-06T03:49:04.080369+0000","last_peered":"2023-10-06T03:49:04.080369+0000","last_clean":"2023-10-06T03:49:04.080369+0000","last_became_active":"2023-10-06T03:49:04.069987+0000","last_became_peered":"2023-10-06T03:49:04.069987+0000","last_unstale":"2023-10-06T03:49:04.080369+0000","last_undegraded":"2023-10-06T03:49:04.080369+0000","last_fullsized":"2023-10-06T03:49:04.080369+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:35.759692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"92'6","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.093851+0000","last_change":"2023-10-06T03:49:03.067403+0000","last_active":"2023-10-06T03:49:04.093851+0000","last_peered":"2023-10-06T03:49:04.093851+0000","last_clean":"2023-10-06T03:49:04.093851+0000","last_became_active":"2023-10-06T03:49:03.066554+0000","last_became_peered":"2023-10-06T03:49:03.066554+0000","last_unstale":"2023-10-06T03:49:04.093851+0000","last_undegraded":"2023-10-06T03:49:04.093851+0000","last_fullsized":"2023-10-06T03:49:04.093851+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:00.290117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.072968+0000","last_change":"2023-10-06T03:49:04.071242+0000","last_active":"2023-10-06T03:49:04.072968+0000","last_peered":"2023-10-06T03:49:04.072968+0000","last_clean":"2023-10-06T03:49:04.072968+0000","last_became_active":"2023-10-06T03:49:04.070939+0000","last_became_peered":"2023-10-06T03:49:04.070939+0000","last_unstale":"2023-10-06T03:49:04.072968+0000","last_undegraded":"2023-10-06T03:49:04.072968+0000","last_fullsized":"2023-10-06T03:49:04.072968+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:55.621702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.080379+0000","last_change":"2023-10-06T03:49:03.067259+0000","last_active":"2023-10-06T03:49:04.080379+0000","last_peered":"2023-10-06T03:49:04.080379+0000","last_clean":"2023-10-06T03:49:04.080379+0000","last_became_active":"2023-10-06T03:49:03.066955+0000","last_became_peered":"2023-10-06T03:49:03.066955+0000","last_unstale":"2023-10-06T03:49:04.080379+0000","last_undegraded":"2023-10-06T03:49:04.080379+0000","last_fullsized":"2023-10-06T03:49:04.080379+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:14.631791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.071182+0000","last_change":"2023-10-06T03:49:04.062093+0000","last_active":"2023-10-06T03:49:04.071182+0000","last_peered":"2023-10-06T03:49:04.071182+0000","last_clean":"2023-10-06T03:49:04.071182+0000","last_became_active":"2023-10-06T03:49:04.061839+0000","last_became_peered":"2023-10-06T03:49:04.061839+0000","last_unstale":"2023-10-06T03:49:04.071182+0000","last_undegraded":"2023-10-06T03:49:04.071182+0000","last_fullsized":"2023-10-06T03:49:04.071182+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:20.860601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.050086+0000","last_change":"2023-10-06T03:49:03.065805+0000","last_active":"2023-10-06T03:49:04.050086+0000","last_peered":"2023-10-06T03:49:04.050086+0000","last_clean":"2023-10-06T03:49:04.050086+0000","last_became_active":"2023-10-06T03:49:03.065268+0000","last_became_peered":"2023-10-06T03:49:03.065268+0000","last_unstale":"2023-10-06T03:49:04.050086+0000","last_undegraded":"2023-10-06T03:49:04.050086+0000","last_fullsized":"2023-10-06T03:49:04.050086+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:04.768736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.071264+0000","last_change":"2023-10-06T03:49:04.066584+0000","last_active":"2023-10-06T03:49:04.071264+0000","last_peered":"2023-10-06T03:49:04.071264+0000","last_clean":"2023-10-06T03:49:04.071264+0000","last_became_active":"2023-10-06T03:49:04.064757+0000","last_became_peered":"2023-10-06T03:49:04.064757+0000","last_unstale":"2023-10-06T03:49:04.071264+0000","last_undegraded":"2023-10-06T03:49:04.071264+0000","last_fullsized":"2023-10-06T03:49:04.071264+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:29.113361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.058608+0000","last_change":"2023-10-06T03:49:03.067598+0000","last_active":"2023-10-06T03:49:04.058608+0000","last_peered":"2023-10-06T03:49:04.058608+0000","last_clean":"2023-10-06T03:49:04.058608+0000","last_became_active":"2023-10-06T03:49:03.067311+0000","last_became_peered":"2023-10-06T03:49:03.067311+0000","last_unstale":"2023-10-06T03:49:04.058608+0000","last_undegraded":"2023-10-06T03:49:04.058608+0000","last_fullsized":"2023-10-06T03:49:04.058608+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:15.830816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.071543+0000","last_change":"2023-10-06T03:49:04.068771+0000","last_active":"2023-10-06T03:49:04.071543+0000","last_peered":"2023-10-06T03:49:04.071543+0000","last_clean":"2023-10-06T03:49:04.071543+0000","last_became_active":"2023-10-06T03:49:04.067211+0000","last_became_peered":"2023-10-06T03:49:04.067211+0000","last_unstale":"2023-10-06T03:49:04.071543+0000","last_undegraded":"2023-10-06T03:49:04.071543+0000","last_fullsized":"2023-10-06T03:49:04.071543+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:55.621702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"92'8","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.092723+0000","last_change":"2023-10-06T03:49:03.064219+0000","last_active":"2023-10-06T03:49:04.092723+0000","last_peered":"2023-10-06T03:49:04.092723+0000","last_clean":"2023-10-06T03:49:04.092723+0000","last_became_active":"2023-10-06T03:49:03.064008+0000","last_became_peered":"2023-10-06T03:49:03.064008+0000","last_unstale":"2023-10-06T03:49:04.092723+0000","last_undegraded":"2023-10-06T03:49:04.092723+0000","last_fullsized":"2023-10-06T03:49:04.092723+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:29.253372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.066273+0000","last_change":"2023-10-06T03:49:04.062944+0000","last_active":"2023-10-06T03:49:04.066273+0000","last_peered":"2023-10-06T03:49:04.066273+0000","last_clean":"2023-10-06T03:49:04.066273+0000","last_became_active":"2023-10-06T03:49:04.062702+0000","last_became_peered":"2023-10-06T03:49:04.062702+0000","last_unstale":"2023-10-06T03:49:04.066273+0000","last_undegraded":"2023-10-06T03:49:04.066273+0000","last_fullsized":"2023-10-06T03:49:04.066273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:43.848842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.082002+0000","last_change":"2023-10-06T03:49:03.072128+0000","last_active":"2023-10-06T03:49:04.082002+0000","last_peered":"2023-10-06T03:49:04.082002+0000","last_clean":"2023-10-06T03:49:04.082002+0000","last_became_active":"2023-10-06T03:49:03.071862+0000","last_became_peered":"2023-10-06T03:49:03.071862+0000","last_unstale":"2023-10-06T03:49:04.082002+0000","last_undegraded":"2023-10-06T03:49:04.082002+0000","last_fullsized":"2023-10-06T03:49:04.082002+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:31.564281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.060136+0000","last_change":"2023-10-06T03:49:04.056091+0000","last_active":"2023-10-06T03:49:04.060136+0000","last_peered":"2023-10-06T03:49:04.060136+0000","last_clean":"2023-10-06T03:49:04.060136+0000","last_became_active":"2023-10-06T03:49:04.055826+0000","last_became_peered":"2023-10-06T03:49:04.055826+0000","last_unstale":"2023-10-06T03:49:04.060136+0000","last_undegraded":"2023-10-06T03:49:04.060136+0000","last_fullsized":"2023-10-06T03:49:04.060136+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:13:32.044507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.058394+0000","last_change":"2023-10-06T03:49:03.074885+0000","last_active":"2023-10-06T03:49:04.058394+0000","last_peered":"2023-10-06T03:49:04.058394+0000","last_clean":"2023-10-06T03:49:04.058394+0000","last_became_active":"2023-10-06T03:49:03.074601+0000","last_became_peered":"2023-10-06T03:49:03.074601+0000","last_unstale":"2023-10-06T03:49:04.058394+0000","last_undegraded":"2023-10-06T03:49:04.058394+0000","last_fullsized":"2023-10-06T03:49:04.058394+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:31.169854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.073426+0000","last_change":"2023-10-06T03:49:04.071137+0000","last_active":"2023-10-06T03:49:04.073426+0000","last_peered":"2023-10-06T03:49:04.073426+0000","last_clean":"2023-10-06T03:49:04.073426+0000","last_became_active":"2023-10-06T03:49:04.070841+0000","last_became_peered":"2023-10-06T03:49:04.070841+0000","last_unstale":"2023-10-06T03:49:04.073426+0000","last_undegraded":"2023-10-06T03:49:04.073426+0000","last_fullsized":"2023-10-06T03:49:04.073426+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:17.032274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.050517+0000","last_change":"2023-10-06T03:49:03.068891+0000","last_active":"2023-10-06T03:49:04.050517+0000","last_peered":"2023-10-06T03:49:04.050517+0000","last_clean":"2023-10-06T03:49:04.050517+0000","last_became_active":"2023-10-06T03:49:03.068472+0000","last_became_peered":"2023-10-06T03:49:03.068472+0000","last_unstale":"2023-10-06T03:49:04.050517+0000","last_undegraded":"2023-10-06T03:49:04.050517+0000","last_fullsized":"2023-10-06T03:49:04.050517+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:04.795631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.081659+0000","last_change":"2023-10-06T03:49:04.074799+0000","last_active":"2023-10-06T03:49:04.081659+0000","last_peered":"2023-10-06T03:49:04.081659+0000","last_clean":"2023-10-06T03:49:04.081659+0000","last_became_active":"2023-10-06T03:49:04.074542+0000","last_became_peered":"2023-10-06T03:49:04.074542+0000","last_unstale":"2023-10-06T03:49:04.081659+0000","last_undegraded":"2023-10-06T03:49:04.081659+0000","last_fullsized":"2023-10-06T03:49:04.081659+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:26:40.677461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.050387+0000","last_change":"2023-10-06T03:49:03.064430+0000","last_active":"2023-10-06T03:49:04.050387+0000","last_peered":"2023-10-06T03:49:04.050387+0000","last_clean":"2023-10-06T03:49:04.050387+0000","last_became_active":"2023-10-06T03:49:03.064176+0000","last_became_peered":"2023-10-06T03:49:03.064176+0000","last_unstale":"2023-10-06T03:49:04.050387+0000","last_undegraded":"2023-10-06T03:49:04.050387+0000","last_fullsized":"2023-10-06T03:49:04.050387+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:47.823447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.060004+0000","last_change":"2023-10-06T03:49:04.058021+0000","last_active":"2023-10-06T03:49:04.060004+0000","last_peered":"2023-10-06T03:49:04.060004+0000","last_clean":"2023-10-06T03:49:04.060004+0000","last_became_active":"2023-10-06T03:49:04.057618+0000","last_became_peered":"2023-10-06T03:49:04.057618+0000","last_unstale":"2023-10-06T03:49:04.060004+0000","last_undegraded":"2023-10-06T03:49:04.060004+0000","last_fullsized":"2023-10-06T03:49:04.060004+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:36.224255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.049844+0000","last_change":"2023-10-06T03:49:03.072712+0000","last_active":"2023-10-06T03:49:04.049844+0000","last_peered":"2023-10-06T03:49:04.049844+0000","last_clean":"2023-10-06T03:49:04.049844+0000","last_became_active":"2023-10-06T03:49:03.071252+0000","last_became_peered":"2023-10-06T03:49:03.071252+0000","last_unstale":"2023-10-06T03:49:04.049844+0000","last_undegraded":"2023-10-06T03:49:04.049844+0000","last_fullsized":"2023-10-06T03:49:04.049844+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:34.223560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.073282+0000","last_change":"2023-10-06T03:49:04.069566+0000","last_active":"2023-10-06T03:49:04.073282+0000","last_peered":"2023-10-06T03:49:04.073282+0000","last_clean":"2023-10-06T03:49:04.073282+0000","last_became_active":"2023-10-06T03:49:04.068487+0000","last_became_peered":"2023-10-06T03:49:04.068487+0000","last_unstale":"2023-10-06T03:49:04.073282+0000","last_undegraded":"2023-10-06T03:49:04.073282+0000","last_fullsized":"2023-10-06T03:49:04.073282+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:30.733029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.058485+0000","last_change":"2023-10-06T03:49:03.074109+0000","last_active":"2023-10-06T03:49:04.058485+0000","last_peered":"2023-10-06T03:49:04.058485+0000","last_clean":"2023-10-06T03:49:04.058485+0000","last_became_active":"2023-10-06T03:49:03.073835+0000","last_became_peered":"2023-10-06T03:49:03.073835+0000","last_unstale":"2023-10-06T03:49:04.058485+0000","last_undegraded":"2023-10-06T03:49:04.058485+0000","last_fullsized":"2023-10-06T03:49:04.058485+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:50.590201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.080080+0000","last_change":"2023-10-06T03:49:04.069720+0000","last_active":"2023-10-06T03:49:04.080080+0000","last_peered":"2023-10-06T03:49:04.080080+0000","last_clean":"2023-10-06T03:49:04.080080+0000","last_became_active":"2023-10-06T03:49:04.069348+0000","last_became_peered":"2023-10-06T03:49:04.069348+0000","last_unstale":"2023-10-06T03:49:04.080080+0000","last_undegraded":"2023-10-06T03:49:04.080080+0000","last_fullsized":"2023-10-06T03:49:04.080080+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:34.308700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.049722+0000","last_change":"2023-10-06T03:49:03.071620+0000","last_active":"2023-10-06T03:49:04.049722+0000","last_peered":"2023-10-06T03:49:04.049722+0000","last_clean":"2023-10-06T03:49:04.049722+0000","last_became_active":"2023-10-06T03:49:03.071357+0000","last_became_peered":"2023-10-06T03:49:03.071357+0000","last_unstale":"2023-10-06T03:49:04.049722+0000","last_undegraded":"2023-10-06T03:49:04.049722+0000","last_fullsized":"2023-10-06T03:49:04.049722+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:42.913488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.080714+0000","last_change":"2023-10-06T03:49:04.062075+0000","last_active":"2023-10-06T03:49:04.080714+0000","last_peered":"2023-10-06T03:49:04.080714+0000","last_clean":"2023-10-06T03:49:04.080714+0000","last_became_active":"2023-10-06T03:49:04.061801+0000","last_became_peered":"2023-10-06T03:49:04.061801+0000","last_unstale":"2023-10-06T03:49:04.080714+0000","last_undegraded":"2023-10-06T03:49:04.080714+0000","last_fullsized":"2023-10-06T03:49:04.080714+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:03.359619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.058012+0000","last_change":"2023-10-06T03:49:03.064243+0000","last_active":"2023-10-06T03:49:04.058012+0000","last_peered":"2023-10-06T03:49:04.058012+0000","last_clean":"2023-10-06T03:49:04.058012+0000","last_became_active":"2023-10-06T03:49:03.063940+0000","last_became_peered":"2023-10-06T03:49:03.063940+0000","last_unstale":"2023-10-06T03:49:04.058012+0000","last_undegraded":"2023-10-06T03:49:04.058012+0000","last_fullsized":"2023-10-06T03:49:04.058012+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:42.656195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.069295+0000","last_change":"2023-10-06T03:49:04.067326+0000","last_active":"2023-10-06T03:49:04.069295+0000","last_peered":"2023-10-06T03:49:04.069295+0000","last_clean":"2023-10-06T03:49:04.069295+0000","last_became_active":"2023-10-06T03:49:04.067031+0000","last_became_peered":"2023-10-06T03:49:04.067031+0000","last_unstale":"2023-10-06T03:49:04.069295+0000","last_undegraded":"2023-10-06T03:49:04.069295+0000","last_fullsized":"2023-10-06T03:49:04.069295+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:35.482312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.083400+0000","last_change":"2023-10-06T03:49:03.067587+0000","last_active":"2023-10-06T03:49:04.083400+0000","last_peered":"2023-10-06T03:49:04.083400+0000","last_clean":"2023-10-06T03:49:04.083400+0000","last_became_active":"2023-10-06T03:49:03.067320+0000","last_became_peered":"2023-10-06T03:49:03.067320+0000","last_unstale":"2023-10-06T03:49:04.083400+0000","last_undegraded":"2023-10-06T03:49:04.083400+0000","last_fullsized":"2023-10-06T03:49:04.083400+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:18.628463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.056178+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:49:04.056178+0000","last_peered":"2023-10-06T03:49:04.056178+0000","last_clean":"2023-10-06T03:49:04.056178+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:49:04.056178+0000","last_undegraded":"2023-10-06T03:49:04.056178+0000","last_fullsized":"2023-10-06T03:49:04.056178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.076670+0000","last_change":"2023-10-06T03:49:04.073612+0000","last_active":"2023-10-06T03:49:04.076670+0000","last_peered":"2023-10-06T03:49:04.076670+0000","last_clean":"2023-10-06T03:49:04.076670+0000","last_became_active":"2023-10-06T03:49:04.073360+0000","last_became_peered":"2023-10-06T03:49:04.073360+0000","last_unstale":"2023-10-06T03:49:04.076670+0000","last_undegraded":"2023-10-06T03:49:04.076670+0000","last_fullsized":"2023-10-06T03:49:04.076670+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:20.130519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.086921+0000","last_change":"2023-10-06T03:49:03.071724+0000","last_active":"2023-10-06T03:49:04.086921+0000","last_peered":"2023-10-06T03:49:04.086921+0000","last_clean":"2023-10-06T03:49:04.086921+0000","last_became_active":"2023-10-06T03:49:03.071368+0000","last_became_peered":"2023-10-06T03:49:03.071368+0000","last_unstale":"2023-10-06T03:49:04.086921+0000","last_undegraded":"2023-10-06T03:49:04.086921+0000","last_fullsized":"2023-10-06T03:49:04.086921+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:06.794520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.048988+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:49:04.048988+0000","last_peered":"2023-10-06T03:49:04.048988+0000","last_clean":"2023-10-06T03:49:04.048988+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:49:04.048988+0000","last_undegraded":"2023-10-06T03:49:04.048988+0000","last_fullsized":"2023-10-06T03:49:04.048988+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.078674+0000","last_change":"2023-10-06T03:49:04.069234+0000","last_active":"2023-10-06T03:49:04.078674+0000","last_peered":"2023-10-06T03:49:04.078674+0000","last_clean":"2023-10-06T03:49:04.078674+0000","last_became_active":"2023-10-06T03:49:04.068911+0000","last_became_peered":"2023-10-06T03:49:04.068911+0000","last_unstale":"2023-10-06T03:49:04.078674+0000","last_undegraded":"2023-10-06T03:49:04.078674+0000","last_fullsized":"2023-10-06T03:49:04.078674+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:04.079813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.048714+0000","last_change":"2023-10-06T03:49:03.074317+0000","last_active":"2023-10-06T03:49:04.048714+0000","last_peered":"2023-10-06T03:49:04.048714+0000","last_clean":"2023-10-06T03:49:04.048714+0000","last_became_active":"2023-10-06T03:49:03.074062+0000","last_became_peered":"2023-10-06T03:49:03.074062+0000","last_unstale":"2023-10-06T03:49:04.048714+0000","last_undegraded":"2023-10-06T03:49:04.048714+0000","last_fullsized":"2023-10-06T03:49:04.048714+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:08.871796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":77,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.055322+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:49:04.055322+0000","last_peered":"2023-10-06T03:49:04.055322+0000","last_clean":"2023-10-06T03:49:04.055322+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:49:04.055322+0000","last_undegraded":"2023-10-06T03:49:04.055322+0000","last_fullsized":"2023-10-06T03:49:04.055322+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"83'183","reported_seq":462,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.051349+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:49:04.051349+0000","last_peered":"2023-10-06T03:49:04.051349+0000","last_clean":"2023-10-06T03:49:04.051349+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:49:04.051349+0000","last_undegraded":"2023-10-06T03:49:04.051349+0000","last_fullsized":"2023-10-06T03:49:04.051349+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.067266+0000","last_change":"2023-10-06T03:49:04.061483+0000","last_active":"2023-10-06T03:49:04.067266+0000","last_peered":"2023-10-06T03:49:04.067266+0000","last_clean":"2023-10-06T03:49:04.067266+0000","last_became_active":"2023-10-06T03:49:04.061177+0000","last_became_peered":"2023-10-06T03:49:04.061177+0000","last_unstale":"2023-10-06T03:49:04.067266+0000","last_undegraded":"2023-10-06T03:49:04.067266+0000","last_fullsized":"2023-10-06T03:49:04.067266+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:28.005151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.048706+0000","last_change":"2023-10-06T03:49:03.070278+0000","last_active":"2023-10-06T03:49:04.048706+0000","last_peered":"2023-10-06T03:49:04.048706+0000","last_clean":"2023-10-06T03:49:04.048706+0000","last_became_active":"2023-10-06T03:49:03.070008+0000","last_became_peered":"2023-10-06T03:49:03.070008+0000","last_unstale":"2023-10-06T03:49:04.048706+0000","last_undegraded":"2023-10-06T03:49:04.048706+0000","last_fullsized":"2023-10-06T03:49:04.048706+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:04.768736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":79,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.050174+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:49:04.050174+0000","last_peered":"2023-10-06T03:49:04.050174+0000","last_clean":"2023-10-06T03:49:04.050174+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:49:04.050174+0000","last_undegraded":"2023-10-06T03:49:04.050174+0000","last_fullsized":"2023-10-06T03:49:04.050174+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068165+0000","last_change":"2023-10-06T03:49:04.060748+0000","last_active":"2023-10-06T03:49:04.068165+0000","last_peered":"2023-10-06T03:49:04.068165+0000","last_clean":"2023-10-06T03:49:04.068165+0000","last_became_active":"2023-10-06T03:49:04.060450+0000","last_became_peered":"2023-10-06T03:49:04.060450+0000","last_unstale":"2023-10-06T03:49:04.068165+0000","last_undegraded":"2023-10-06T03:49:04.068165+0000","last_fullsized":"2023-10-06T03:49:04.068165+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:17.032274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.048640+0000","last_change":"2023-10-06T03:49:03.076714+0000","last_active":"2023-10-06T03:49:04.048640+0000","last_peered":"2023-10-06T03:49:04.048640+0000","last_clean":"2023-10-06T03:49:04.048640+0000","last_became_active":"2023-10-06T03:49:03.076454+0000","last_became_peered":"2023-10-06T03:49:03.076454+0000","last_unstale":"2023-10-06T03:49:04.048640+0000","last_undegraded":"2023-10-06T03:49:04.048640+0000","last_fullsized":"2023-10-06T03:49:04.048640+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:04.795631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.051444+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:49:04.051444+0000","last_peered":"2023-10-06T03:49:04.051444+0000","last_clean":"2023-10-06T03:49:04.051444+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:49:04.051444+0000","last_undegraded":"2023-10-06T03:49:04.051444+0000","last_fullsized":"2023-10-06T03:49:04.051444+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.078364+0000","last_change":"2023-10-06T03:49:04.070912+0000","last_active":"2023-10-06T03:49:04.078364+0000","last_peered":"2023-10-06T03:49:04.078364+0000","last_clean":"2023-10-06T03:49:04.078364+0000","last_became_active":"2023-10-06T03:49:04.070649+0000","last_became_peered":"2023-10-06T03:49:04.070649+0000","last_unstale":"2023-10-06T03:49:04.078364+0000","last_undegraded":"2023-10-06T03:49:04.078364+0000","last_fullsized":"2023-10-06T03:49:04.078364+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:16.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.088565+0000","last_change":"2023-10-06T03:49:03.070399+0000","last_active":"2023-10-06T03:49:04.088565+0000","last_peered":"2023-10-06T03:49:04.088565+0000","last_clean":"2023-10-06T03:49:04.088565+0000","last_became_active":"2023-10-06T03:49:03.070133+0000","last_became_peered":"2023-10-06T03:49:03.070133+0000","last_unstale":"2023-10-06T03:49:04.088565+0000","last_undegraded":"2023-10-06T03:49:04.088565+0000","last_fullsized":"2023-10-06T03:49:04.088565+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:00.278339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.055837+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:49:04.055837+0000","last_peered":"2023-10-06T03:49:04.055837+0000","last_clean":"2023-10-06T03:49:04.055837+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:49:04.055837+0000","last_undegraded":"2023-10-06T03:49:04.055837+0000","last_fullsized":"2023-10-06T03:49:04.055837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068938+0000","last_change":"2023-10-06T03:49:04.064787+0000","last_active":"2023-10-06T03:49:04.068938+0000","last_peered":"2023-10-06T03:49:04.068938+0000","last_clean":"2023-10-06T03:49:04.068938+0000","last_became_active":"2023-10-06T03:49:04.064527+0000","last_became_peered":"2023-10-06T03:49:04.064527+0000","last_unstale":"2023-10-06T03:49:04.068938+0000","last_undegraded":"2023-10-06T03:49:04.068938+0000","last_fullsized":"2023-10-06T03:49:04.068938+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:16.695070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.054522+0000","last_change":"2023-10-06T03:49:03.068973+0000","last_active":"2023-10-06T03:49:04.054522+0000","last_peered":"2023-10-06T03:49:04.054522+0000","last_clean":"2023-10-06T03:49:04.054522+0000","last_became_active":"2023-10-06T03:49:03.068629+0000","last_became_peered":"2023-10-06T03:49:03.068629+0000","last_unstale":"2023-10-06T03:49:04.054522+0000","last_undegraded":"2023-10-06T03:49:04.054522+0000","last_fullsized":"2023-10-06T03:49:04.054522+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:19.852270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.050502+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:49:04.050502+0000","last_peered":"2023-10-06T03:49:04.050502+0000","last_clean":"2023-10-06T03:49:04.050502+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:49:04.050502+0000","last_undegraded":"2023-10-06T03:49:04.050502+0000","last_fullsized":"2023-10-06T03:49:04.050502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.085758+0000","last_change":"2023-10-06T03:49:04.077775+0000","last_active":"2023-10-06T03:49:04.085758+0000","last_peered":"2023-10-06T03:49:04.085758+0000","last_clean":"2023-10-06T03:49:04.085758+0000","last_became_active":"2023-10-06T03:49:04.077310+0000","last_became_peered":"2023-10-06T03:49:04.077310+0000","last_unstale":"2023-10-06T03:49:04.085758+0000","last_undegraded":"2023-10-06T03:49:04.085758+0000","last_fullsized":"2023-10-06T03:49:04.085758+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:51.420823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.088398+0000","last_change":"2023-10-06T03:49:03.065837+0000","last_active":"2023-10-06T03:49:04.088398+0000","last_peered":"2023-10-06T03:49:04.088398+0000","last_clean":"2023-10-06T03:49:04.088398+0000","last_became_active":"2023-10-06T03:49:03.065590+0000","last_became_peered":"2023-10-06T03:49:03.065590+0000","last_unstale":"2023-10-06T03:49:04.088398+0000","last_undegraded":"2023-10-06T03:49:04.088398+0000","last_fullsized":"2023-10-06T03:49:04.088398+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:57.330077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.050829+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:49:04.050829+0000","last_peered":"2023-10-06T03:49:04.050829+0000","last_clean":"2023-10-06T03:49:04.050829+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:49:04.050829+0000","last_undegraded":"2023-10-06T03:49:04.050829+0000","last_fullsized":"2023-10-06T03:49:04.050829+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575038,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311252,"kb_used_data":1676,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060588,"statfs":{"total":96636764160,"available":96318042112,"internally_reserved":0,"allocated":1716224,"data_stored":1156852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.439,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.81899999999999995,"5min":1.0029999999999999,"15min":1.6100000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.42599999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94399999999999995,"5min":1.02,"15min":1.5649999999999999},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.41499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.97599999999999998,"5min":1.038,"15min":1.5469999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.747,"15min":0.76700000000000002},"min":{"1min":0.53100000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.98799999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68600000000000005,"15min":0.69799999999999995},"min":{"1min":0.437,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93500000000000005,"5min":0.97499999999999998,"15min":1.5900000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72499999999999998,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92200000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.749}]},{"osd":3,"last update":"Fri Oct 6 03:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71599999999999997,"15min":0.72499999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.39500000000000002},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.6279999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.501,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.876,"5min":0.98099999999999998,"15min":1.0409999999999999},"last":0.70799999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.42299999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78600000000000003,"5min":0.96499999999999997,"15min":1.0049999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.624},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.39400000000000002},"max":{"1min":0.85199999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55100000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.436,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":0.88400000000000001,"5min":1.018,"15min":1.069},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.50800000000000001,"5min":0.48299999999999998,"15min":0.436},"max":{"1min":0.94299999999999995,"5min":1.105,"15min":1.105},"last":0.68999999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.03,"5min":1.181,"15min":1.181},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.49399999999999999,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.90600000000000003,"5min":1.02,"15min":1.0209999999999999},"last":0.65700000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575039,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306600,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065240,"statfs":{"total":96636764160,"available":96322805760,"internally_reserved":0,"allocated":1146880,"data_stored":576009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.42199999999999999,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.88,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.39500000000000002},"max":{"1min":0.94799999999999995,"5min":0.98599999999999999,"15min":1.0449999999999999},"last":0.70599999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.47499999999999998,"5min":0.36499999999999999,"15min":0.33800000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.3180000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.35899999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.61499999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.751,"15min":0.751},"min":{"1min":0.48199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.98699999999999999,"5min":0.995,"15min":3.3319999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":3.5139999999999998},"last":0.72199999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.58499999999999996,"5min":0.48299999999999998,"15min":0.44600000000000001},"max":{"1min":0.97399999999999998,"5min":1.01,"15min":1.024},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76100000000000001,"15min":0.77400000000000002},"min":{"1min":0.49299999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.98799999999999999,"5min":0.99099999999999999,"15min":3.3300000000000001},"last":0.77800000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.45200000000000001,"5min":0.376,"15min":0.34799999999999998},"max":{"1min":0.91900000000000004,"5min":0.92600000000000005,"15min":1.0680000000000001},"last":0.44},{"interface":"front","average":{"1min":0.621,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36499999999999999},"max":{"1min":0.90700000000000003,"5min":1.0169999999999999,"15min":1.2929999999999999},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67000000000000004,"15min":0.66700000000000004},"min":{"1min":0.438,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.3340000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.42899999999999999,"15min":0.41199999999999998},"max":{"1min":1.0049999999999999,"5min":1.0269999999999999,"15min":1.2370000000000001},"last":0.503}]},{"osd":7,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68600000000000005,"15min":0.69899999999999995},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.619},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.47099999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.417},"last":0.66600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307164,"kb_used_data":1684,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064676,"statfs":{"total":96636764160,"available":96322228224,"internally_reserved":0,"allocated":1724416,"data_stored":1157920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58399999999999996,"5min":0.73599999999999999,"15min":0.77900000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.38100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.624,"5min":1.1319999999999999,"15min":1.2889999999999999},"last":0.47399999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.49099999999999999,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.32000000000000001,"15min":0.26100000000000001},"max":{"1min":0.70799999999999996,"5min":0.72699999999999998,"15min":0.999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.41399999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70199999999999996,"5min":0.77300000000000002,"15min":0.94099999999999995},"last":0.44900000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79600000000000004,"5min":0.86699999999999999,"15min":0.93999999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81499999999999995,"5min":0.90400000000000003,"15min":1.254},"last":0.68000000000000005}]},{"osd":4,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76300000000000001,"15min":0.76200000000000001},"min":{"1min":0.53100000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.3600000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.54800000000000004,"5min":0.439,"15min":0.439},"max":{"1min":0.97099999999999997,"5min":1.2310000000000001,"15min":1.476},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79500000000000004,"15min":0.79200000000000004},"min":{"1min":0.54600000000000004,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":0.95899999999999996,"5min":1.294,"15min":1.601},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80700000000000005,"15min":0.80500000000000005},"min":{"1min":0.622,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.165,"5min":1.224,"15min":1.5189999999999999},"last":0.72199999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79900000000000004,"15min":0.80900000000000005},"min":{"1min":0.54400000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1339999999999999,"5min":1.26,"15min":1.26},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.52800000000000002,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":1.0109999999999999,"5min":1.143,"15min":1.399},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84199999999999997,"15min":0.83699999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1519999999999999,"5min":1.272,"15min":1.272},"last":0.747},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83599999999999997,"15min":0.83199999999999996},"min":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.51200000000000001},"max":{"1min":1.1950000000000001,"5min":1.2010000000000001,"15min":1.556},"last":0.74099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575040,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306584,"kb_used_data":1104,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065256,"statfs":{"total":96636764160,"available":96322822144,"internally_reserved":0,"allocated":1130496,"data_stored":573495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46899999999999997,"15min":0.48599999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":0.64800000000000002,"5min":0.81000000000000005,"15min":2.3220000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.499,"15min":0.52200000000000002},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.29599999999999999},"max":{"1min":0.67900000000000005,"5min":0.78500000000000003,"15min":2.1440000000000001},"last":0.438}]},{"osd":2,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52800000000000002,"15min":0.55500000000000005},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":2.4329999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54200000000000004,"15min":0.55700000000000005},"min":{"1min":0.40500000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83399999999999996,"5min":0.84699999999999998,"15min":2.1819999999999999},"last":0.54400000000000004}]},{"osd":3,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50700000000000001,"15min":0.52500000000000002},"min":{"1min":0.35099999999999998,"5min":0.30499999999999999,"15min":0.28399999999999997},"max":{"1min":0.75700000000000001,"5min":0.81699999999999995,"15min":2.2709999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.60099999999999998,"15min":0.61899999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84599999999999997,"5min":1.0580000000000001,"15min":2.4700000000000002},"last":0.46800000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.82299999999999995,"15min":0.83499999999999996},"min":{"1min":0.57099999999999995,"5min":0.54200000000000004,"15min":0.49399999999999999},"max":{"1min":1.1910000000000001,"5min":1.2589999999999999,"15min":1.393},"last":0.752},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80400000000000005,"15min":0.82999999999999996},"min":{"1min":0.55700000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.256,"5min":1.306,"15min":2.2320000000000002},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79800000000000004,"15min":0.82299999999999995},"min":{"1min":0.58099999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.091,"5min":1.2869999999999999,"15min":2.3660000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.77700000000000002,"15min":0.80700000000000005},"min":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.6559999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78200000000000003,"15min":0.80900000000000005},"min":{"1min":0.58499999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1080000000000001,"5min":1.137,"15min":1.726},"last":0.621},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":0.84899999999999998},"min":{"1min":0.54700000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.167,"5min":1.167,"15min":2.4009999999999998},"last":0.73499999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81299999999999994,"15min":0.82299999999999995},"min":{"1min":0.63,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80000000000000004,"15min":0.83599999999999997},"min":{"1min":0.53400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.1519999999999999,"5min":1.2450000000000001,"15min":1.7649999999999999},"last":0.59599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575039,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310692,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061148,"statfs":{"total":96636764160,"available":96318615552,"internally_reserved":0,"allocated":1142784,"data_stored":574209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.34300000000000003,"5min":0.29699999999999999,"15min":0.28000000000000003},"max":{"1min":0.56200000000000006,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.38500000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63700000000000001,"5min":0.83599999999999997,"15min":0.89300000000000002},"last":0.41399999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.34100000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66100000000000003,"5min":0.73799999999999999,"15min":0.90000000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54700000000000004,"15min":0.55600000000000005},"min":{"1min":0.39200000000000002,"5min":0.372,"15min":0.34200000000000003},"max":{"1min":0.84199999999999997,"5min":0.85099999999999998,"15min":1.556},"last":0.439}]},{"osd":3,"last update":"Fri Oct 6 03:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.439,"5min":0.41399999999999998,"15min":0.38800000000000001},"max":{"1min":0.73199999999999998,"5min":1.78,"15min":1.78},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70199999999999996,"5min":0.86599999999999999,"15min":1.1910000000000001},"last":0.627}]},{"osd":4,"last update":"Fri Oct 6 03:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.59399999999999997,"5min":0.51400000000000001,"15min":0.439},"max":{"1min":1.0169999999999999,"5min":1.0649999999999999,"15min":1.5620000000000001},"last":0.746},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82899999999999996,"15min":0.83699999999999997},"min":{"1min":0.58799999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0620000000000001,"5min":1.1279999999999999,"15min":1.2270000000000001},"last":0.53100000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81999999999999995,"15min":0.83399999999999996},"min":{"1min":0.66500000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.97199999999999998,"5min":1.169,"15min":1.3},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.85199999999999998},"min":{"1min":0.56399999999999995,"5min":0.55500000000000005,"15min":0.51700000000000002},"max":{"1min":1.0980000000000001,"5min":1.855,"15min":1.855},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.55300000000000005,"5min":0.54200000000000004,"15min":0.53600000000000003},"max":{"1min":1.056,"5min":1.907,"15min":1.907},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.81299999999999994,"15min":0.81899999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.51100000000000001},"max":{"1min":1.0169999999999999,"5min":1.1080000000000001,"15min":1.4750000000000001},"last":0.86899999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.84399999999999997,"15min":0.84199999999999997},"min":{"1min":0.55900000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.113,"5min":1.4379999999999999,"15min":1.4990000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.86399999999999999,"15min":0.86799999999999999},"min":{"1min":0.59699999999999998,"5min":0.53100000000000003,"15min":0.497},"max":{"1min":1.123,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.63800000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575039,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306592,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065248,"statfs":{"total":96636764160,"available":96322813952,"internally_reserved":0,"allocated":1138688,"data_stored":575027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.32700000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.75700000000000001,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":0.61199999999999999,"5min":1.04,"15min":1.1699999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47699999999999998,"15min":0.47799999999999998},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.63900000000000001,"5min":0.68799999999999994,"15min":0.79500000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.83899999999999997},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.34599999999999997},"max":{"1min":0.69899999999999995,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.38700000000000001},"max":{"1min":0.68600000000000005,"5min":0.81999999999999995,"15min":0.97899999999999998},"last":0.48899999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76800000000000002,"15min":0.78700000000000003},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1,"5min":1.073,"15min":1.4099999999999999},"last":0.629},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.622,"5min":0.57399999999999995,"15min":0.51300000000000001},"max":{"1min":0.96899999999999997,"5min":1.0469999999999999,"15min":1.377},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.79000000000000004},"min":{"1min":0.58999999999999997,"5min":0.51900000000000002,"15min":0.47099999999999997},"max":{"1min":1.012,"5min":1.05,"15min":1.256},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.75600000000000001,"15min":0.78700000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.48099999999999998},"max":{"1min":0.93100000000000005,"5min":1.1100000000000001,"15min":1.306},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79300000000000004,"15min":0.81399999999999995},"min":{"1min":0.56999999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"max":{"1min":0.98999999999999999,"5min":1.1399999999999999,"15min":1.323},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81799999999999995,"15min":0.84999999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.49299999999999999},"max":{"1min":1.05,"5min":1.1040000000000001,"15min":1.6020000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79100000000000004,"15min":0.81399999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86799999999999999,"5min":1.2030000000000001,"15min":1.365},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78900000000000003,"15min":0.80800000000000005},"min":{"1min":0.55800000000000005,"5min":0.54600000000000004,"15min":0.46300000000000002},"max":{"1min":1.0289999999999999,"5min":1.093,"15min":1.2330000000000001},"last":0.69099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575039,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302512,"kb_used_data":1128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069328,"statfs":{"total":96636764160,"available":96326991872,"internally_reserved":0,"allocated":1155072,"data_stored":576113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69399999999999995,"15min":0.71999999999999997},"min":{"1min":0.51000000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.94799999999999995,"5min":1.0109999999999999,"15min":1.3120000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67500000000000004,"15min":0.69699999999999995},"min":{"1min":0.44700000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.84999999999999998,"5min":0.95299999999999996,"15min":1.01},"last":0.63800000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.70499999999999996,"15min":0.72599999999999998},"min":{"1min":0.42999999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0600000000000001,"5min":1.081,"15min":1.081},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.51500000000000001,"5min":0.49299999999999999,"15min":0.41299999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.3380000000000001},"last":0.747}]},{"osd":2,"last update":"Fri Oct 6 03:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71099999999999997,"15min":0.73699999999999999},"min":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.377},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73799999999999999,"15min":0.745},"min":{"1min":0.52000000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92900000000000005,"5min":1.022,"15min":1.3089999999999999},"last":0.69099999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.46300000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82999999999999996,"5min":1.0049999999999999,"15min":1.0409999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.54300000000000004,"5min":0.47099999999999997,"15min":0.46500000000000002},"max":{"1min":1.1040000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.085},"last":0.622},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63300000000000001,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":0.93500000000000005,"15min":1.1439999999999999},"last":0.82299999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.49399999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1839999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.38100000000000001},"max":{"1min":1.016,"5min":1.052,"15min":1.361},"last":0.78200000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.57099999999999995,"5min":0.46500000000000002,"15min":0.39900000000000002},"max":{"1min":0.997,"5min":0.997,"15min":1.1950000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.53600000000000003,"5min":0.51300000000000001,"15min":0.41299999999999998},"max":{"1min":1.214,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.83699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575039,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298384,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073456,"statfs":{"total":96636764160,"available":96331218944,"internally_reserved":0,"allocated":1122304,"data_stored":566496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69299999999999995,"15min":0.70699999999999996},"min":{"1min":0.39200000000000002,"5min":0.35999999999999999,"15min":0.35299999999999998},"max":{"1min":0.88500000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69699999999999995,"15min":0.71099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.32600000000000001},"max":{"1min":0.90600000000000003,"5min":0.96899999999999997,"15min":1},"last":0.63400000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:48:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69999999999999996,"15min":0.71399999999999997},"min":{"1min":0.499,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97599999999999998,"5min":0.998,"15min":1.0429999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70499999999999996,"15min":0.72199999999999998},"min":{"1min":0.58199999999999996,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94499999999999995,"5min":1.012,"15min":1.012},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:48:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.55000000000000004,"5min":0.502,"15min":0.40200000000000002},"max":{"1min":0.92100000000000004,"5min":1.046,"15min":1.046},"last":0.751},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.75,"15min":0.77300000000000002},"min":{"1min":0.51200000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94199999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.88300000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:48:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.73199999999999998},"min":{"1min":0.495,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.6419999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.747,"15min":0.75},"min":{"1min":0.60999999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":1.117,"15min":1.117},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:48:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.628,"15min":0.61499999999999999},"min":{"1min":0.441,"5min":0.40400000000000003,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.95999999999999996,"15min":1.024},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.626,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":1.036,"5min":1.044,"15min":1.175},"last":0.71899999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:48:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.42899999999999999,"5min":0.40200000000000002,"15min":0.34999999999999998},"max":{"1min":0.93300000000000005,"5min":1.591,"15min":1.591},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70099999999999996,"15min":0.69399999999999995},"min":{"1min":0.496,"5min":0.44600000000000001,"15min":0.40600000000000003},"max":{"1min":1.0169999999999999,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.86099999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:48:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67800000000000005,"15min":0.67900000000000005},"min":{"1min":0.52800000000000002,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.2589999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71399999999999997,"15min":0.71899999999999997},"min":{"1min":0.52600000000000002,"5min":0.498,"15min":0.435},"max":{"1min":0.97799999999999998,"5min":1.278,"15min":1.278},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:49:08.446 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:49:08.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.658+0000 7fec57fff700 1 Processor -- start 2023-10-06T03:49:08.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.659+0000 7fec57fff700 1 -- start start 2023-10-06T03:49:08.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.660+0000 7fec57fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec58156310 0x7fec580c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.660+0000 7fec57fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fec581566e0 0x7fec580ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.660+0000 7fec57fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec580ca860 con 0x7fec580ca300 2023-10-06T03:49:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.660+0000 7fec57fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec580cab40 con 0x7fec581566e0 2023-10-06T03:49:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.660+0000 7fec57fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec580cae20 con 0x7fec58156310 2023-10-06T03:49:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.660+0000 7fec567fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fec581566e0 0x7fec580ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.660+0000 7fec567fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fec581566e0 0x7fec580ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50080/0 (socket says 172.21.15.55:50080) 2023-10-06T03:49:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.660+0000 7fec567fc700 1 -- 172.21.15.55:0/769471487 learned_addr learned my addr 172.21.15.55:0/769471487 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.661+0000 7fec56ffd700 1 --2- 172.21.15.55:0/769471487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec58156310 0x7fec580c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.661+0000 7fec567fc700 1 -- 172.21.15.55:0/769471487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec58156310 msgr2=0x7fec580c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.661+0000 7fec567fc700 1 --2- 172.21.15.55:0/769471487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec58156310 0x7fec580c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.661+0000 7fec567fc700 1 -- 172.21.15.55:0/769471487 >> v1:172.21.15.163:6789/0 conn(0x7fec580ca300 legacy=0x7fec580c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:08.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.661+0000 7fec567fc700 1 -- 172.21.15.55:0/769471487 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec580cb100 con 0x7fec581566e0 2023-10-06T03:49:08.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.661+0000 7fec567fc700 1 --2- 172.21.15.55:0/769471487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec581566e0 0x7fec580ae000 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0099b0 tx=0x7fec4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a8b75db69dec5f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:08.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.661+0000 7fec55ffb700 1 -- 172.21.15.55:0/769471487 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c00b820 con 0x7fec581566e0 2023-10-06T03:49:08.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.661+0000 7fec55ffb700 1 -- 172.21.15.55:0/769471487 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4c00b980 con 0x7fec581566e0 2023-10-06T03:49:08.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.661+0000 7fec55ffb700 1 -- 172.21.15.55:0/769471487 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c00bd20 con 0x7fec581566e0 2023-10-06T03:49:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.662+0000 7fec57fff700 1 -- 172.21.15.55:0/769471487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec581566e0 msgr2=0x7fec580ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.662+0000 7fec57fff700 1 --2- 172.21.15.55:0/769471487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec581566e0 0x7fec580ae000 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0099b0 tx=0x7fec4c009250 comp rx=0 tx=0).stop 2023-10-06T03:49:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.662+0000 7fec57fff700 1 -- 172.21.15.55:0/769471487 shutdown_connections 2023-10-06T03:49:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.662+0000 7fec57fff700 1 --2- 172.21.15.55:0/769471487 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec58156310 0x7fec580c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.662+0000 7fec57fff700 1 --2- 172.21.15.55:0/769471487 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec581566e0 0x7fec580ae000 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.662+0000 7fec57fff700 1 -- 172.21.15.55:0/769471487 >> 172.21.15.55:0/769471487 conn(0x7fec580cc1f0 msgr2=0x7fec580cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.662+0000 7fec57fff700 1 -- 172.21.15.55:0/769471487 shutdown_connections 2023-10-06T03:49:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.662+0000 7fec57fff700 1 -- 172.21.15.55:0/769471487 wait complete. 2023-10-06T03:49:08.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.663+0000 7fec57fff700 1 Processor -- start 2023-10-06T03:49:08.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.663+0000 7fec57fff700 1 -- start start 2023-10-06T03:49:08.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.664+0000 7fec57fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec580ca300 0x7fec581470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.664+0000 7fec57fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec58156310 0x7fec580b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.664+0000 7fec57fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec581566e0 0x7fec58141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.664+0000 7fec57fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec58146c50 con 0x7fec58156310 2023-10-06T03:49:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.664+0000 7fec57fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec58146040 con 0x7fec580ca300 2023-10-06T03:49:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.664+0000 7fec57fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec58146320 con 0x7fec581566e0 2023-10-06T03:49:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.665+0000 7fec567fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec58156310 0x7fec580b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.665+0000 7fec567fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec58156310 0x7fec580b1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50084/0 (socket says 172.21.15.55:50084) 2023-10-06T03:49:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.665+0000 7fec567fc700 1 -- 172.21.15.55:0/1291751680 learned_addr learned my addr 172.21.15.55:0/1291751680 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.665+0000 7fec56ffd700 1 --2- 172.21.15.55:0/1291751680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec580ca300 0x7fec581470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.665+0000 7fec577fe700 1 --2- 172.21.15.55:0/1291751680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec581566e0 0x7fec58141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:08.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.665+0000 7fec567fc700 1 -- 172.21.15.55:0/1291751680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec581566e0 msgr2=0x7fec58141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.665+0000 7fec567fc700 1 --2- 172.21.15.55:0/1291751680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec581566e0 0x7fec58141b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.665+0000 7fec567fc700 1 -- 172.21.15.55:0/1291751680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec580ca300 msgr2=0x7fec581470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.665+0000 7fec567fc700 1 --2- 172.21.15.55:0/1291751680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec580ca300 0x7fec581470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.665+0000 7fec567fc700 1 -- 172.21.15.55:0/1291751680 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c0094f0 con 0x7fec58156310 2023-10-06T03:49:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.665+0000 7fec567fc700 1 --2- 172.21.15.55:0/1291751680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec58156310 0x7fec580b1570 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009290 tx=0x7fec4c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.666+0000 7fec3effd700 1 -- 172.21.15.55:0/1291751680 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c011b40 con 0x7fec58156310 2023-10-06T03:49:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.666+0000 7fec3effd700 1 -- 172.21.15.55:0/1291751680 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4c020070 con 0x7fec58156310 2023-10-06T03:49:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.666+0000 7fec3effd700 1 -- 172.21.15.55:0/1291751680 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c01b450 con 0x7fec58156310 2023-10-06T03:49:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.666+0000 7fec57fff700 1 -- 172.21.15.55:0/1291751680 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec58146600 con 0x7fec58156310 2023-10-06T03:49:08.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.666+0000 7fec57fff700 1 -- 172.21.15.55:0/1291751680 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec58147620 con 0x7fec58156310 2023-10-06T03:49:08.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.667+0000 7fec57fff700 1 -- 172.21.15.55:0/1291751680 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec58146e00 con 0x7fec58156310 2023-10-06T03:49:08.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.668+0000 7fec3effd700 1 -- 172.21.15.55:0/1291751680 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec4c022030 con 0x7fec58156310 2023-10-06T03:49:08.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.669+0000 7fec3effd700 1 --2- 172.21.15.55:0/1291751680 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fec44066f30 0x7fec440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:08.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.669+0000 7fec3effd700 1 -- 172.21.15.55:0/1291751680 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fec4c0865f0 con 0x7fec58156310 2023-10-06T03:49:08.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.669+0000 7fec56ffd700 1 --2- 172.21.15.55:0/1291751680 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fec44066f30 0x7fec440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:08.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.670+0000 7fec56ffd700 1 --2- 172.21.15.55:0/1291751680 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fec44066f30 0x7fec440693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fec48006fd0 tx=0x7fec48008040 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:08.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.673+0000 7fec3effd700 1 -- 172.21.15.55:0/1291751680 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec58146e00 con 0x7fec58156310 2023-10-06T03:49:08.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.869+0000 7fec57fff700 1 -- 172.21.15.55:0/1291751680 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec5804c3c0 con 0x7fec44066f30 2023-10-06T03:49:08.880 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.879+0000 7fec3effd700 1 -- 172.21.15.55:0/1291751680 <== mgr.5828 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311313 (secure 0 0 0) 0x7fec5804c3c0 con 0x7fec44066f30 2023-10-06T03:49:08.884 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:08.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.883+0000 7fec57fff700 1 -- 172.21.15.55:0/1291751680 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fec44066f30 msgr2=0x7fec440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:08.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.883+0000 7fec57fff700 1 --2- 172.21.15.55:0/1291751680 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fec44066f30 0x7fec440693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fec48006fd0 tx=0x7fec48008040 comp rx=0 tx=0).stop 2023-10-06T03:49:08.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.883+0000 7fec57fff700 1 -- 172.21.15.55:0/1291751680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec58156310 msgr2=0x7fec580b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:08.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.883+0000 7fec57fff700 1 --2- 172.21.15.55:0/1291751680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec58156310 0x7fec580b1570 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009290 tx=0x7fec4c016000 comp rx=0 tx=0).stop 2023-10-06T03:49:08.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.883+0000 7fec57fff700 1 -- 172.21.15.55:0/1291751680 shutdown_connections 2023-10-06T03:49:08.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.883+0000 7fec57fff700 1 --2- 172.21.15.55:0/1291751680 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fec44066f30 0x7fec440693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.883+0000 7fec57fff700 1 --2- 172.21.15.55:0/1291751680 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec581566e0 0x7fec58141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.883+0000 7fec57fff700 1 --2- 172.21.15.55:0/1291751680 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec580ca300 0x7fec581470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.883+0000 7fec57fff700 1 --2- 172.21.15.55:0/1291751680 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec58156310 0x7fec580b1570 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:08.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.883+0000 7fec57fff700 1 -- 172.21.15.55:0/1291751680 >> 172.21.15.55:0/1291751680 conn(0x7fec580cc1f0 msgr2=0x7fec58152060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:08.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.883+0000 7fec57fff700 1 -- 172.21.15.55:0/1291751680 shutdown_connections 2023-10-06T03:49:08.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:08.883+0000 7fec57fff700 1 -- 172.21.15.55:0/1291751680 wait complete. 2023-10-06T03:49:08.887 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:49:08.902 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-06T03:49:07.000487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":323,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2449780,"kb_used_data":10036,"kb_used_omap":0,"kb_used_meta":2439680,"kb_avail":752524940,"statfs":{"total":773094113280,"available":770585538560,"internally_reserved":0,"allocated":10276864,"data_stored":5756121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2498232320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.958805"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068124+0000","last_change":"2023-10-06T03:49:04.065554+0000","last_active":"2023-10-06T03:49:04.068124+0000","last_peered":"2023-10-06T03:49:04.068124+0000","last_clean":"2023-10-06T03:49:04.068124+0000","last_became_active":"2023-10-06T03:49:04.065210+0000","last_became_peered":"2023-10-06T03:49:04.065210+0000","last_unstale":"2023-10-06T03:49:04.068124+0000","last_undegraded":"2023-10-06T03:49:04.068124+0000","last_fullsized":"2023-10-06T03:49:04.068124+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:57.778267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.078692+0000","last_change":"2023-10-06T03:49:04.076365+0000","last_active":"2023-10-06T03:49:04.078692+0000","last_peered":"2023-10-06T03:49:04.078692+0000","last_clean":"2023-10-06T03:49:04.078692+0000","last_became_active":"2023-10-06T03:49:04.076117+0000","last_became_peered":"2023-10-06T03:49:04.076117+0000","last_unstale":"2023-10-06T03:49:04.078692+0000","last_undegraded":"2023-10-06T03:49:04.078692+0000","last_fullsized":"2023-10-06T03:49:04.078692+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:24.383164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.080109+0000","last_change":"2023-10-06T03:49:04.066770+0000","last_active":"2023-10-06T03:49:04.080109+0000","last_peered":"2023-10-06T03:49:04.080109+0000","last_clean":"2023-10-06T03:49:04.080109+0000","last_became_active":"2023-10-06T03:49:04.066524+0000","last_became_peered":"2023-10-06T03:49:04.066524+0000","last_unstale":"2023-10-06T03:49:04.080109+0000","last_undegraded":"2023-10-06T03:49:04.080109+0000","last_fullsized":"2023-10-06T03:49:04.080109+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:18.788347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068129+0000","last_change":"2023-10-06T03:49:04.064296+0000","last_active":"2023-10-06T03:49:04.068129+0000","last_peered":"2023-10-06T03:49:04.068129+0000","last_clean":"2023-10-06T03:49:04.068129+0000","last_became_active":"2023-10-06T03:49:04.064047+0000","last_became_peered":"2023-10-06T03:49:04.064047+0000","last_unstale":"2023-10-06T03:49:04.068129+0000","last_undegraded":"2023-10-06T03:49:04.068129+0000","last_fullsized":"2023-10-06T03:49:04.068129+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:24:55.747936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.076623+0000","last_change":"2023-10-06T03:49:04.072399+0000","last_active":"2023-10-06T03:49:04.076623+0000","last_peered":"2023-10-06T03:49:04.076623+0000","last_clean":"2023-10-06T03:49:04.076623+0000","last_became_active":"2023-10-06T03:49:04.071660+0000","last_became_peered":"2023-10-06T03:49:04.071660+0000","last_unstale":"2023-10-06T03:49:04.076623+0000","last_undegraded":"2023-10-06T03:49:04.076623+0000","last_fullsized":"2023-10-06T03:49:04.076623+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:13:08.261321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068850+0000","last_change":"2023-10-06T03:49:04.066198+0000","last_active":"2023-10-06T03:49:04.068850+0000","last_peered":"2023-10-06T03:49:04.068850+0000","last_clean":"2023-10-06T03:49:04.068850+0000","last_became_active":"2023-10-06T03:49:04.065564+0000","last_became_peered":"2023-10-06T03:49:04.065564+0000","last_unstale":"2023-10-06T03:49:04.068850+0000","last_undegraded":"2023-10-06T03:49:04.068850+0000","last_fullsized":"2023-10-06T03:49:04.068850+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:30.261703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.071301+0000","last_change":"2023-10-06T03:49:04.068089+0000","last_active":"2023-10-06T03:49:04.071301+0000","last_peered":"2023-10-06T03:49:04.071301+0000","last_clean":"2023-10-06T03:49:04.071301+0000","last_became_active":"2023-10-06T03:49:04.067696+0000","last_became_peered":"2023-10-06T03:49:04.067696+0000","last_unstale":"2023-10-06T03:49:04.071301+0000","last_undegraded":"2023-10-06T03:49:04.071301+0000","last_fullsized":"2023-10-06T03:49:04.071301+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:47:51.305962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068241+0000","last_change":"2023-10-06T03:49:04.060653+0000","last_active":"2023-10-06T03:49:04.068241+0000","last_peered":"2023-10-06T03:49:04.068241+0000","last_clean":"2023-10-06T03:49:04.068241+0000","last_became_active":"2023-10-06T03:49:04.060322+0000","last_became_peered":"2023-10-06T03:49:04.060322+0000","last_unstale":"2023-10-06T03:49:04.068241+0000","last_undegraded":"2023-10-06T03:49:04.068241+0000","last_fullsized":"2023-10-06T03:49:04.068241+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:25.386744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.073113+0000","last_change":"2023-10-06T03:49:04.069230+0000","last_active":"2023-10-06T03:49:04.073113+0000","last_peered":"2023-10-06T03:49:04.073113+0000","last_clean":"2023-10-06T03:49:04.073113+0000","last_became_active":"2023-10-06T03:49:04.068964+0000","last_became_peered":"2023-10-06T03:49:04.068964+0000","last_unstale":"2023-10-06T03:49:04.073113+0000","last_undegraded":"2023-10-06T03:49:04.073113+0000","last_fullsized":"2023-10-06T03:49:04.073113+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:05.297394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.071218+0000","last_change":"2023-10-06T03:49:04.067370+0000","last_active":"2023-10-06T03:49:04.071218+0000","last_peered":"2023-10-06T03:49:04.071218+0000","last_clean":"2023-10-06T03:49:04.071218+0000","last_became_active":"2023-10-06T03:49:04.067115+0000","last_became_peered":"2023-10-06T03:49:04.067115+0000","last_unstale":"2023-10-06T03:49:04.071218+0000","last_undegraded":"2023-10-06T03:49:04.071218+0000","last_fullsized":"2023-10-06T03:49:04.071218+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:34.308700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.078147+0000","last_change":"2023-10-06T03:49:04.068374+0000","last_active":"2023-10-06T03:49:04.078147+0000","last_peered":"2023-10-06T03:49:04.078147+0000","last_clean":"2023-10-06T03:49:04.078147+0000","last_became_active":"2023-10-06T03:49:04.068115+0000","last_became_peered":"2023-10-06T03:49:04.068115+0000","last_unstale":"2023-10-06T03:49:04.078147+0000","last_undegraded":"2023-10-06T03:49:04.078147+0000","last_fullsized":"2023-10-06T03:49:04.078147+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:22.193560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.081875+0000","last_change":"2023-10-06T03:49:04.076378+0000","last_active":"2023-10-06T03:49:04.081875+0000","last_peered":"2023-10-06T03:49:04.081875+0000","last_clean":"2023-10-06T03:49:04.081875+0000","last_became_active":"2023-10-06T03:49:04.076097+0000","last_became_peered":"2023-10-06T03:49:04.076097+0000","last_unstale":"2023-10-06T03:49:04.081875+0000","last_undegraded":"2023-10-06T03:49:04.081875+0000","last_fullsized":"2023-10-06T03:49:04.081875+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:35.759692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.076602+0000","last_change":"2023-10-06T03:49:04.072653+0000","last_active":"2023-10-06T03:49:04.076602+0000","last_peered":"2023-10-06T03:49:04.076602+0000","last_clean":"2023-10-06T03:49:04.076602+0000","last_became_active":"2023-10-06T03:49:04.072101+0000","last_became_peered":"2023-10-06T03:49:04.072101+0000","last_unstale":"2023-10-06T03:49:04.076602+0000","last_undegraded":"2023-10-06T03:49:04.076602+0000","last_fullsized":"2023-10-06T03:49:04.076602+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:41:06.692485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068869+0000","last_change":"2023-10-06T03:49:04.066366+0000","last_active":"2023-10-06T03:49:04.068869+0000","last_peered":"2023-10-06T03:49:04.068869+0000","last_clean":"2023-10-06T03:49:04.068869+0000","last_became_active":"2023-10-06T03:49:04.065947+0000","last_became_peered":"2023-10-06T03:49:04.065947+0000","last_unstale":"2023-10-06T03:49:04.068869+0000","last_undegraded":"2023-10-06T03:49:04.068869+0000","last_fullsized":"2023-10-06T03:49:04.068869+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:32.178184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.078650+0000","last_change":"2023-10-06T03:49:04.075319+0000","last_active":"2023-10-06T03:49:04.078650+0000","last_peered":"2023-10-06T03:49:04.078650+0000","last_clean":"2023-10-06T03:49:04.078650+0000","last_became_active":"2023-10-06T03:49:04.075038+0000","last_became_peered":"2023-10-06T03:49:04.075038+0000","last_unstale":"2023-10-06T03:49:04.078650+0000","last_undegraded":"2023-10-06T03:49:04.078650+0000","last_fullsized":"2023-10-06T03:49:04.078650+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:22.938029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068249+0000","last_change":"2023-10-06T03:49:04.061999+0000","last_active":"2023-10-06T03:49:04.068249+0000","last_peered":"2023-10-06T03:49:04.068249+0000","last_clean":"2023-10-06T03:49:04.068249+0000","last_became_active":"2023-10-06T03:49:04.061737+0000","last_became_peered":"2023-10-06T03:49:04.061737+0000","last_unstale":"2023-10-06T03:49:04.068249+0000","last_undegraded":"2023-10-06T03:49:04.068249+0000","last_fullsized":"2023-10-06T03:49:04.068249+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:04.485978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.076700+0000","last_change":"2023-10-06T03:49:04.074276+0000","last_active":"2023-10-06T03:49:04.076700+0000","last_peered":"2023-10-06T03:49:04.076700+0000","last_clean":"2023-10-06T03:49:04.076700+0000","last_became_active":"2023-10-06T03:49:04.074022+0000","last_became_peered":"2023-10-06T03:49:04.074022+0000","last_unstale":"2023-10-06T03:49:04.076700+0000","last_undegraded":"2023-10-06T03:49:04.076700+0000","last_fullsized":"2023-10-06T03:49:04.076700+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:58.686045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.079832+0000","last_change":"2023-10-06T03:49:04.064954+0000","last_active":"2023-10-06T03:49:04.079832+0000","last_peered":"2023-10-06T03:49:04.079832+0000","last_clean":"2023-10-06T03:49:04.079832+0000","last_became_active":"2023-10-06T03:49:04.064631+0000","last_became_peered":"2023-10-06T03:49:04.064631+0000","last_unstale":"2023-10-06T03:49:04.079832+0000","last_undegraded":"2023-10-06T03:49:04.079832+0000","last_fullsized":"2023-10-06T03:49:04.079832+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:01:16.640709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068465+0000","last_change":"2023-10-06T03:49:04.062640+0000","last_active":"2023-10-06T03:49:04.068465+0000","last_peered":"2023-10-06T03:49:04.068465+0000","last_clean":"2023-10-06T03:49:04.068465+0000","last_became_active":"2023-10-06T03:49:04.062382+0000","last_became_peered":"2023-10-06T03:49:04.062382+0000","last_unstale":"2023-10-06T03:49:04.068465+0000","last_undegraded":"2023-10-06T03:49:04.068465+0000","last_fullsized":"2023-10-06T03:49:04.068465+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:57.778267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.088169+0000","last_change":"2023-10-06T03:49:04.080997+0000","last_active":"2023-10-06T03:49:04.088169+0000","last_peered":"2023-10-06T03:49:04.088169+0000","last_clean":"2023-10-06T03:49:04.088169+0000","last_became_active":"2023-10-06T03:49:04.080670+0000","last_became_peered":"2023-10-06T03:49:04.080670+0000","last_unstale":"2023-10-06T03:49:04.088169+0000","last_undegraded":"2023-10-06T03:49:04.088169+0000","last_fullsized":"2023-10-06T03:49:04.088169+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:39.795313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.082121+0000","last_change":"2023-10-06T03:49:04.081071+0000","last_active":"2023-10-06T03:49:04.082121+0000","last_peered":"2023-10-06T03:49:04.082121+0000","last_clean":"2023-10-06T03:49:04.082121+0000","last_became_active":"2023-10-06T03:49:04.080774+0000","last_became_peered":"2023-10-06T03:49:04.080774+0000","last_unstale":"2023-10-06T03:49:04.082121+0000","last_undegraded":"2023-10-06T03:49:04.082121+0000","last_fullsized":"2023-10-06T03:49:04.082121+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:31.705749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.085732+0000","last_change":"2023-10-06T03:49:04.077572+0000","last_active":"2023-10-06T03:49:04.085732+0000","last_peered":"2023-10-06T03:49:04.085732+0000","last_clean":"2023-10-06T03:49:04.085732+0000","last_became_active":"2023-10-06T03:49:04.076972+0000","last_became_peered":"2023-10-06T03:49:04.076972+0000","last_unstale":"2023-10-06T03:49:04.085732+0000","last_undegraded":"2023-10-06T03:49:04.085732+0000","last_fullsized":"2023-10-06T03:49:04.085732+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:59:43.895946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.073382+0000","last_change":"2023-10-06T03:49:04.070565+0000","last_active":"2023-10-06T03:49:04.073382+0000","last_peered":"2023-10-06T03:49:04.073382+0000","last_clean":"2023-10-06T03:49:04.073382+0000","last_became_active":"2023-10-06T03:49:04.070277+0000","last_became_peered":"2023-10-06T03:49:04.070277+0000","last_unstale":"2023-10-06T03:49:04.073382+0000","last_undegraded":"2023-10-06T03:49:04.073382+0000","last_fullsized":"2023-10-06T03:49:04.073382+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:35.482312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.081189+0000","last_change":"2023-10-06T03:49:04.071898+0000","last_active":"2023-10-06T03:49:04.081189+0000","last_peered":"2023-10-06T03:49:04.081189+0000","last_clean":"2023-10-06T03:49:04.081189+0000","last_became_active":"2023-10-06T03:49:04.071614+0000","last_became_peered":"2023-10-06T03:49:04.071614+0000","last_unstale":"2023-10-06T03:49:04.081189+0000","last_undegraded":"2023-10-06T03:49:04.081189+0000","last_fullsized":"2023-10-06T03:49:04.081189+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:58.678773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.078354+0000","last_change":"2023-10-06T03:49:04.069270+0000","last_active":"2023-10-06T03:49:04.078354+0000","last_peered":"2023-10-06T03:49:04.078354+0000","last_clean":"2023-10-06T03:49:04.078354+0000","last_became_active":"2023-10-06T03:49:04.069012+0000","last_became_peered":"2023-10-06T03:49:04.069012+0000","last_unstale":"2023-10-06T03:49:04.078354+0000","last_undegraded":"2023-10-06T03:49:04.078354+0000","last_fullsized":"2023-10-06T03:49:04.078354+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:58:27.394068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.082109+0000","last_change":"2023-10-06T03:49:04.078356+0000","last_active":"2023-10-06T03:49:04.082109+0000","last_peered":"2023-10-06T03:49:04.082109+0000","last_clean":"2023-10-06T03:49:04.082109+0000","last_became_active":"2023-10-06T03:49:04.077998+0000","last_became_peered":"2023-10-06T03:49:04.077998+0000","last_unstale":"2023-10-06T03:49:04.082109+0000","last_undegraded":"2023-10-06T03:49:04.082109+0000","last_fullsized":"2023-10-06T03:49:04.082109+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:45:18.788347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068187+0000","last_change":"2023-10-06T03:49:04.061399+0000","last_active":"2023-10-06T03:49:04.068187+0000","last_peered":"2023-10-06T03:49:04.068187+0000","last_clean":"2023-10-06T03:49:04.068187+0000","last_became_active":"2023-10-06T03:49:04.061055+0000","last_became_peered":"2023-10-06T03:49:04.061055+0000","last_unstale":"2023-10-06T03:49:04.068187+0000","last_undegraded":"2023-10-06T03:49:04.068187+0000","last_fullsized":"2023-10-06T03:49:04.068187+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:16.695070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068658+0000","last_change":"2023-10-06T03:49:04.064314+0000","last_active":"2023-10-06T03:49:04.068658+0000","last_peered":"2023-10-06T03:49:04.068658+0000","last_clean":"2023-10-06T03:49:04.068658+0000","last_became_active":"2023-10-06T03:49:04.064046+0000","last_became_peered":"2023-10-06T03:49:04.064046+0000","last_unstale":"2023-10-06T03:49:04.068658+0000","last_undegraded":"2023-10-06T03:49:04.068658+0000","last_fullsized":"2023-10-06T03:49:04.068658+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:26:24.059522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.067705+0000","last_change":"2023-10-06T03:49:04.066832+0000","last_active":"2023-10-06T03:49:04.067705+0000","last_peered":"2023-10-06T03:49:04.067705+0000","last_clean":"2023-10-06T03:49:04.067705+0000","last_became_active":"2023-10-06T03:49:04.066453+0000","last_became_peered":"2023-10-06T03:49:04.066453+0000","last_unstale":"2023-10-06T03:49:04.067705+0000","last_undegraded":"2023-10-06T03:49:04.067705+0000","last_fullsized":"2023-10-06T03:49:04.067705+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:53:24.450111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068214+0000","last_change":"2023-10-06T03:49:04.066022+0000","last_active":"2023-10-06T03:49:04.068214+0000","last_peered":"2023-10-06T03:49:04.068214+0000","last_clean":"2023-10-06T03:49:04.068214+0000","last_became_active":"2023-10-06T03:49:04.065244+0000","last_became_peered":"2023-10-06T03:49:04.065244+0000","last_unstale":"2023-10-06T03:49:04.068214+0000","last_undegraded":"2023-10-06T03:49:04.068214+0000","last_fullsized":"2023-10-06T03:49:04.068214+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:32.178184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.080828+0000","last_change":"2023-10-06T03:49:04.060812+0000","last_active":"2023-10-06T03:49:04.080828+0000","last_peered":"2023-10-06T03:49:04.080828+0000","last_clean":"2023-10-06T03:49:04.080828+0000","last_became_active":"2023-10-06T03:49:04.060548+0000","last_became_peered":"2023-10-06T03:49:04.060548+0000","last_unstale":"2023-10-06T03:49:04.080828+0000","last_undegraded":"2023-10-06T03:49:04.080828+0000","last_fullsized":"2023-10-06T03:49:04.080828+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:24:55.747936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068276+0000","last_change":"2023-10-06T03:49:04.064819+0000","last_active":"2023-10-06T03:49:04.068276+0000","last_peered":"2023-10-06T03:49:04.068276+0000","last_clean":"2023-10-06T03:49:04.068276+0000","last_became_active":"2023-10-06T03:49:04.064569+0000","last_became_peered":"2023-10-06T03:49:04.064569+0000","last_unstale":"2023-10-06T03:49:04.068276+0000","last_undegraded":"2023-10-06T03:49:04.068276+0000","last_fullsized":"2023-10-06T03:49:04.068276+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:05.297394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.079909+0000","last_change":"2023-10-06T03:49:04.064992+0000","last_active":"2023-10-06T03:49:04.079909+0000","last_peered":"2023-10-06T03:49:04.079909+0000","last_clean":"2023-10-06T03:49:04.079909+0000","last_became_active":"2023-10-06T03:49:04.064713+0000","last_became_peered":"2023-10-06T03:49:04.064713+0000","last_unstale":"2023-10-06T03:49:04.079909+0000","last_undegraded":"2023-10-06T03:49:04.079909+0000","last_fullsized":"2023-10-06T03:49:04.079909+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:04.277911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"92'4","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.089818+0000","last_change":"2023-10-06T03:49:03.073209+0000","last_active":"2023-10-06T03:49:04.089818+0000","last_peered":"2023-10-06T03:49:04.089818+0000","last_clean":"2023-10-06T03:49:04.089818+0000","last_became_active":"2023-10-06T03:49:03.072812+0000","last_became_peered":"2023-10-06T03:49:03.072812+0000","last_unstale":"2023-10-06T03:49:04.089818+0000","last_undegraded":"2023-10-06T03:49:04.089818+0000","last_fullsized":"2023-10-06T03:49:04.089818+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:53.076107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.078407+0000","last_change":"2023-10-06T03:49:04.072899+0000","last_active":"2023-10-06T03:49:04.078407+0000","last_peered":"2023-10-06T03:49:04.078407+0000","last_clean":"2023-10-06T03:49:04.078407+0000","last_became_active":"2023-10-06T03:49:04.072617+0000","last_became_peered":"2023-10-06T03:49:04.072617+0000","last_unstale":"2023-10-06T03:49:04.078407+0000","last_undegraded":"2023-10-06T03:49:04.078407+0000","last_fullsized":"2023-10-06T03:49:04.078407+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:50.496559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.082278+0000","last_change":"2023-10-06T03:49:03.062672+0000","last_active":"2023-10-06T03:49:04.082278+0000","last_peered":"2023-10-06T03:49:04.082278+0000","last_clean":"2023-10-06T03:49:04.082278+0000","last_became_active":"2023-10-06T03:49:03.062414+0000","last_became_peered":"2023-10-06T03:49:03.062414+0000","last_unstale":"2023-10-06T03:49:04.082278+0000","last_undegraded":"2023-10-06T03:49:04.082278+0000","last_fullsized":"2023-10-06T03:49:04.082278+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:46:47.404382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.073144+0000","last_change":"2023-10-06T03:49:04.070292+0000","last_active":"2023-10-06T03:49:04.073144+0000","last_peered":"2023-10-06T03:49:04.073144+0000","last_clean":"2023-10-06T03:49:04.073144+0000","last_became_active":"2023-10-06T03:49:04.070052+0000","last_became_peered":"2023-10-06T03:49:04.070052+0000","last_unstale":"2023-10-06T03:49:04.073144+0000","last_undegraded":"2023-10-06T03:49:04.073144+0000","last_fullsized":"2023-10-06T03:49:04.073144+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:04.485978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.085898+0000","last_change":"2023-10-06T03:49:03.068734+0000","last_active":"2023-10-06T03:49:04.085898+0000","last_peered":"2023-10-06T03:49:04.085898+0000","last_clean":"2023-10-06T03:49:04.085898+0000","last_became_active":"2023-10-06T03:49:03.068038+0000","last_became_peered":"2023-10-06T03:49:03.068038+0000","last_unstale":"2023-10-06T03:49:04.085898+0000","last_undegraded":"2023-10-06T03:49:04.085898+0000","last_fullsized":"2023-10-06T03:49:04.085898+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:08.871796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.082141+0000","last_change":"2023-10-06T03:49:04.080850+0000","last_active":"2023-10-06T03:49:04.082141+0000","last_peered":"2023-10-06T03:49:04.082141+0000","last_clean":"2023-10-06T03:49:04.082141+0000","last_became_active":"2023-10-06T03:49:04.080550+0000","last_became_peered":"2023-10-06T03:49:04.080550+0000","last_unstale":"2023-10-06T03:49:04.082141+0000","last_undegraded":"2023-10-06T03:49:04.082141+0000","last_fullsized":"2023-10-06T03:49:04.082141+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:53:13.885949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.082423+0000","last_change":"2023-10-06T03:49:03.073041+0000","last_active":"2023-10-06T03:49:04.082423+0000","last_peered":"2023-10-06T03:49:04.082423+0000","last_clean":"2023-10-06T03:49:04.082423+0000","last_became_active":"2023-10-06T03:49:03.072744+0000","last_became_peered":"2023-10-06T03:49:03.072744+0000","last_unstale":"2023-10-06T03:49:04.082423+0000","last_undegraded":"2023-10-06T03:49:04.082423+0000","last_fullsized":"2023-10-06T03:49:04.082423+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:09:42.642374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.079791+0000","last_change":"2023-10-06T03:49:04.076114+0000","last_active":"2023-10-06T03:49:04.079791+0000","last_peered":"2023-10-06T03:49:04.079791+0000","last_clean":"2023-10-06T03:49:04.079791+0000","last_became_active":"2023-10-06T03:49:04.075683+0000","last_became_peered":"2023-10-06T03:49:04.075683+0000","last_unstale":"2023-10-06T03:49:04.079791+0000","last_undegraded":"2023-10-06T03:49:04.079791+0000","last_fullsized":"2023-10-06T03:49:04.079791+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:22:29.832334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.084368+0000","last_change":"2023-10-06T03:49:03.068940+0000","last_active":"2023-10-06T03:49:04.084368+0000","last_peered":"2023-10-06T03:49:04.084368+0000","last_clean":"2023-10-06T03:49:04.084368+0000","last_became_active":"2023-10-06T03:49:03.068686+0000","last_became_peered":"2023-10-06T03:49:03.068686+0000","last_unstale":"2023-10-06T03:49:04.084368+0000","last_undegraded":"2023-10-06T03:49:04.084368+0000","last_fullsized":"2023-10-06T03:49:04.084368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:01.527205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.073446+0000","last_change":"2023-10-06T03:49:04.067693+0000","last_active":"2023-10-06T03:49:04.073446+0000","last_peered":"2023-10-06T03:49:04.073446+0000","last_clean":"2023-10-06T03:49:04.073446+0000","last_became_active":"2023-10-06T03:49:04.067419+0000","last_became_peered":"2023-10-06T03:49:04.067419+0000","last_unstale":"2023-10-06T03:49:04.073446+0000","last_undegraded":"2023-10-06T03:49:04.073446+0000","last_fullsized":"2023-10-06T03:49:04.073446+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:09.342388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.081729+0000","last_change":"2023-10-06T03:49:03.069610+0000","last_active":"2023-10-06T03:49:04.081729+0000","last_peered":"2023-10-06T03:49:04.081729+0000","last_clean":"2023-10-06T03:49:04.081729+0000","last_became_active":"2023-10-06T03:49:03.069160+0000","last_became_peered":"2023-10-06T03:49:03.069160+0000","last_unstale":"2023-10-06T03:49:04.081729+0000","last_undegraded":"2023-10-06T03:49:04.081729+0000","last_fullsized":"2023-10-06T03:49:04.081729+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:19.852270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.073393+0000","last_change":"2023-10-06T03:49:04.070720+0000","last_active":"2023-10-06T03:49:04.073393+0000","last_peered":"2023-10-06T03:49:04.073393+0000","last_clean":"2023-10-06T03:49:04.073393+0000","last_became_active":"2023-10-06T03:49:04.070458+0000","last_became_peered":"2023-10-06T03:49:04.070458+0000","last_unstale":"2023-10-06T03:49:04.073393+0000","last_undegraded":"2023-10-06T03:49:04.073393+0000","last_fullsized":"2023-10-06T03:49:04.073393+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:07:57.778267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.054993+0000","last_change":"2023-10-06T03:49:03.073187+0000","last_active":"2023-10-06T03:49:04.054993+0000","last_peered":"2023-10-06T03:49:04.054993+0000","last_clean":"2023-10-06T03:49:04.054993+0000","last_became_active":"2023-10-06T03:49:03.072757+0000","last_became_peered":"2023-10-06T03:49:03.072757+0000","last_unstale":"2023-10-06T03:49:04.054993+0000","last_undegraded":"2023-10-06T03:49:04.054993+0000","last_fullsized":"2023-10-06T03:49:04.054993+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:15.686739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.071562+0000","last_change":"2023-10-06T03:49:04.069365+0000","last_active":"2023-10-06T03:49:04.071562+0000","last_peered":"2023-10-06T03:49:04.071562+0000","last_clean":"2023-10-06T03:49:04.071562+0000","last_became_active":"2023-10-06T03:49:04.069092+0000","last_became_peered":"2023-10-06T03:49:04.069092+0000","last_unstale":"2023-10-06T03:49:04.071562+0000","last_undegraded":"2023-10-06T03:49:04.071562+0000","last_fullsized":"2023-10-06T03:49:04.071562+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:21.957625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.058262+0000","last_change":"2023-10-06T03:49:03.065475+0000","last_active":"2023-10-06T03:49:04.058262+0000","last_peered":"2023-10-06T03:49:04.058262+0000","last_clean":"2023-10-06T03:49:04.058262+0000","last_became_active":"2023-10-06T03:49:03.065157+0000","last_became_peered":"2023-10-06T03:49:03.065157+0000","last_unstale":"2023-10-06T03:49:04.058262+0000","last_undegraded":"2023-10-06T03:49:04.058262+0000","last_fullsized":"2023-10-06T03:49:04.058262+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:27:53.030591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.081209+0000","last_change":"2023-10-06T03:49:04.075427+0000","last_active":"2023-10-06T03:49:04.081209+0000","last_peered":"2023-10-06T03:49:04.081209+0000","last_clean":"2023-10-06T03:49:04.081209+0000","last_became_active":"2023-10-06T03:49:04.075161+0000","last_became_peered":"2023-10-06T03:49:04.075161+0000","last_unstale":"2023-10-06T03:49:04.081209+0000","last_undegraded":"2023-10-06T03:49:04.081209+0000","last_fullsized":"2023-10-06T03:49:04.081209+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:08.202087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.051552+0000","last_change":"2023-10-06T03:49:03.064156+0000","last_active":"2023-10-06T03:49:04.051552+0000","last_peered":"2023-10-06T03:49:04.051552+0000","last_clean":"2023-10-06T03:49:04.051552+0000","last_became_active":"2023-10-06T03:49:03.063871+0000","last_became_peered":"2023-10-06T03:49:03.063871+0000","last_unstale":"2023-10-06T03:49:04.051552+0000","last_undegraded":"2023-10-06T03:49:04.051552+0000","last_fullsized":"2023-10-06T03:49:04.051552+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:04.748459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.070945+0000","last_change":"2023-10-06T03:49:04.068921+0000","last_active":"2023-10-06T03:49:04.070945+0000","last_peered":"2023-10-06T03:49:04.070945+0000","last_clean":"2023-10-06T03:49:04.070945+0000","last_became_active":"2023-10-06T03:49:04.068675+0000","last_became_peered":"2023-10-06T03:49:04.068675+0000","last_unstale":"2023-10-06T03:49:04.070945+0000","last_undegraded":"2023-10-06T03:49:04.070945+0000","last_fullsized":"2023-10-06T03:49:04.070945+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:03.359619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.058199+0000","last_change":"2023-10-06T03:49:03.073229+0000","last_active":"2023-10-06T03:49:04.058199+0000","last_peered":"2023-10-06T03:49:04.058199+0000","last_clean":"2023-10-06T03:49:04.058199+0000","last_became_active":"2023-10-06T03:49:03.072966+0000","last_became_peered":"2023-10-06T03:49:03.072966+0000","last_unstale":"2023-10-06T03:49:04.058199+0000","last_undegraded":"2023-10-06T03:49:04.058199+0000","last_fullsized":"2023-10-06T03:49:04.058199+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:42.656195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068909+0000","last_change":"2023-10-06T03:49:04.066381+0000","last_active":"2023-10-06T03:49:04.068909+0000","last_peered":"2023-10-06T03:49:04.068909+0000","last_clean":"2023-10-06T03:49:04.068909+0000","last_became_active":"2023-10-06T03:49:04.065557+0000","last_became_peered":"2023-10-06T03:49:04.065557+0000","last_unstale":"2023-10-06T03:49:04.068909+0000","last_undegraded":"2023-10-06T03:49:04.068909+0000","last_fullsized":"2023-10-06T03:49:04.068909+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:05.297394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"92'3","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.089527+0000","last_change":"2023-10-06T03:49:03.064278+0000","last_active":"2023-10-06T03:49:04.089527+0000","last_peered":"2023-10-06T03:49:04.089527+0000","last_clean":"2023-10-06T03:49:04.089527+0000","last_became_active":"2023-10-06T03:49:03.064016+0000","last_became_peered":"2023-10-06T03:49:03.064016+0000","last_unstale":"2023-10-06T03:49:04.089527+0000","last_undegraded":"2023-10-06T03:49:04.089527+0000","last_fullsized":"2023-10-06T03:49:04.089527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:53.314723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.080369+0000","last_change":"2023-10-06T03:49:04.070264+0000","last_active":"2023-10-06T03:49:04.080369+0000","last_peered":"2023-10-06T03:49:04.080369+0000","last_clean":"2023-10-06T03:49:04.080369+0000","last_became_active":"2023-10-06T03:49:04.069987+0000","last_became_peered":"2023-10-06T03:49:04.069987+0000","last_unstale":"2023-10-06T03:49:04.080369+0000","last_undegraded":"2023-10-06T03:49:04.080369+0000","last_fullsized":"2023-10-06T03:49:04.080369+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:35.759692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"92'6","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.093851+0000","last_change":"2023-10-06T03:49:03.067403+0000","last_active":"2023-10-06T03:49:04.093851+0000","last_peered":"2023-10-06T03:49:04.093851+0000","last_clean":"2023-10-06T03:49:04.093851+0000","last_became_active":"2023-10-06T03:49:03.066554+0000","last_became_peered":"2023-10-06T03:49:03.066554+0000","last_unstale":"2023-10-06T03:49:04.093851+0000","last_undegraded":"2023-10-06T03:49:04.093851+0000","last_fullsized":"2023-10-06T03:49:04.093851+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:14:00.290117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.072968+0000","last_change":"2023-10-06T03:49:04.071242+0000","last_active":"2023-10-06T03:49:04.072968+0000","last_peered":"2023-10-06T03:49:04.072968+0000","last_clean":"2023-10-06T03:49:04.072968+0000","last_became_active":"2023-10-06T03:49:04.070939+0000","last_became_peered":"2023-10-06T03:49:04.070939+0000","last_unstale":"2023-10-06T03:49:04.072968+0000","last_undegraded":"2023-10-06T03:49:04.072968+0000","last_fullsized":"2023-10-06T03:49:04.072968+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:55.621702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.080379+0000","last_change":"2023-10-06T03:49:03.067259+0000","last_active":"2023-10-06T03:49:04.080379+0000","last_peered":"2023-10-06T03:49:04.080379+0000","last_clean":"2023-10-06T03:49:04.080379+0000","last_became_active":"2023-10-06T03:49:03.066955+0000","last_became_peered":"2023-10-06T03:49:03.066955+0000","last_unstale":"2023-10-06T03:49:04.080379+0000","last_undegraded":"2023-10-06T03:49:04.080379+0000","last_fullsized":"2023-10-06T03:49:04.080379+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:23:14.631791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.071182+0000","last_change":"2023-10-06T03:49:04.062093+0000","last_active":"2023-10-06T03:49:04.071182+0000","last_peered":"2023-10-06T03:49:04.071182+0000","last_clean":"2023-10-06T03:49:04.071182+0000","last_became_active":"2023-10-06T03:49:04.061839+0000","last_became_peered":"2023-10-06T03:49:04.061839+0000","last_unstale":"2023-10-06T03:49:04.071182+0000","last_undegraded":"2023-10-06T03:49:04.071182+0000","last_fullsized":"2023-10-06T03:49:04.071182+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:20.860601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.050086+0000","last_change":"2023-10-06T03:49:03.065805+0000","last_active":"2023-10-06T03:49:04.050086+0000","last_peered":"2023-10-06T03:49:04.050086+0000","last_clean":"2023-10-06T03:49:04.050086+0000","last_became_active":"2023-10-06T03:49:03.065268+0000","last_became_peered":"2023-10-06T03:49:03.065268+0000","last_unstale":"2023-10-06T03:49:04.050086+0000","last_undegraded":"2023-10-06T03:49:04.050086+0000","last_fullsized":"2023-10-06T03:49:04.050086+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:04.768736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.071264+0000","last_change":"2023-10-06T03:49:04.066584+0000","last_active":"2023-10-06T03:49:04.071264+0000","last_peered":"2023-10-06T03:49:04.071264+0000","last_clean":"2023-10-06T03:49:04.071264+0000","last_became_active":"2023-10-06T03:49:04.064757+0000","last_became_peered":"2023-10-06T03:49:04.064757+0000","last_unstale":"2023-10-06T03:49:04.071264+0000","last_undegraded":"2023-10-06T03:49:04.071264+0000","last_fullsized":"2023-10-06T03:49:04.071264+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:05:29.113361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.058608+0000","last_change":"2023-10-06T03:49:03.067598+0000","last_active":"2023-10-06T03:49:04.058608+0000","last_peered":"2023-10-06T03:49:04.058608+0000","last_clean":"2023-10-06T03:49:04.058608+0000","last_became_active":"2023-10-06T03:49:03.067311+0000","last_became_peered":"2023-10-06T03:49:03.067311+0000","last_unstale":"2023-10-06T03:49:04.058608+0000","last_undegraded":"2023-10-06T03:49:04.058608+0000","last_fullsized":"2023-10-06T03:49:04.058608+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:11:15.830816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.071543+0000","last_change":"2023-10-06T03:49:04.068771+0000","last_active":"2023-10-06T03:49:04.071543+0000","last_peered":"2023-10-06T03:49:04.071543+0000","last_clean":"2023-10-06T03:49:04.071543+0000","last_became_active":"2023-10-06T03:49:04.067211+0000","last_became_peered":"2023-10-06T03:49:04.067211+0000","last_unstale":"2023-10-06T03:49:04.071543+0000","last_undegraded":"2023-10-06T03:49:04.071543+0000","last_fullsized":"2023-10-06T03:49:04.071543+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:55.621702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"92'8","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.092723+0000","last_change":"2023-10-06T03:49:03.064219+0000","last_active":"2023-10-06T03:49:04.092723+0000","last_peered":"2023-10-06T03:49:04.092723+0000","last_clean":"2023-10-06T03:49:04.092723+0000","last_became_active":"2023-10-06T03:49:03.064008+0000","last_became_peered":"2023-10-06T03:49:03.064008+0000","last_unstale":"2023-10-06T03:49:04.092723+0000","last_undegraded":"2023-10-06T03:49:04.092723+0000","last_fullsized":"2023-10-06T03:49:04.092723+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:00:29.253372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.066273+0000","last_change":"2023-10-06T03:49:04.062944+0000","last_active":"2023-10-06T03:49:04.066273+0000","last_peered":"2023-10-06T03:49:04.066273+0000","last_clean":"2023-10-06T03:49:04.066273+0000","last_became_active":"2023-10-06T03:49:04.062702+0000","last_became_peered":"2023-10-06T03:49:04.062702+0000","last_unstale":"2023-10-06T03:49:04.066273+0000","last_undegraded":"2023-10-06T03:49:04.066273+0000","last_fullsized":"2023-10-06T03:49:04.066273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:43.848842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.082002+0000","last_change":"2023-10-06T03:49:03.072128+0000","last_active":"2023-10-06T03:49:04.082002+0000","last_peered":"2023-10-06T03:49:04.082002+0000","last_clean":"2023-10-06T03:49:04.082002+0000","last_became_active":"2023-10-06T03:49:03.071862+0000","last_became_peered":"2023-10-06T03:49:03.071862+0000","last_unstale":"2023-10-06T03:49:04.082002+0000","last_undegraded":"2023-10-06T03:49:04.082002+0000","last_fullsized":"2023-10-06T03:49:04.082002+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:31.564281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.060136+0000","last_change":"2023-10-06T03:49:04.056091+0000","last_active":"2023-10-06T03:49:04.060136+0000","last_peered":"2023-10-06T03:49:04.060136+0000","last_clean":"2023-10-06T03:49:04.060136+0000","last_became_active":"2023-10-06T03:49:04.055826+0000","last_became_peered":"2023-10-06T03:49:04.055826+0000","last_unstale":"2023-10-06T03:49:04.060136+0000","last_undegraded":"2023-10-06T03:49:04.060136+0000","last_fullsized":"2023-10-06T03:49:04.060136+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:13:32.044507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.058394+0000","last_change":"2023-10-06T03:49:03.074885+0000","last_active":"2023-10-06T03:49:04.058394+0000","last_peered":"2023-10-06T03:49:04.058394+0000","last_clean":"2023-10-06T03:49:04.058394+0000","last_became_active":"2023-10-06T03:49:03.074601+0000","last_became_peered":"2023-10-06T03:49:03.074601+0000","last_unstale":"2023-10-06T03:49:04.058394+0000","last_undegraded":"2023-10-06T03:49:04.058394+0000","last_fullsized":"2023-10-06T03:49:04.058394+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:31.169854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.073426+0000","last_change":"2023-10-06T03:49:04.071137+0000","last_active":"2023-10-06T03:49:04.073426+0000","last_peered":"2023-10-06T03:49:04.073426+0000","last_clean":"2023-10-06T03:49:04.073426+0000","last_became_active":"2023-10-06T03:49:04.070841+0000","last_became_peered":"2023-10-06T03:49:04.070841+0000","last_unstale":"2023-10-06T03:49:04.073426+0000","last_undegraded":"2023-10-06T03:49:04.073426+0000","last_fullsized":"2023-10-06T03:49:04.073426+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:17.032274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.050517+0000","last_change":"2023-10-06T03:49:03.068891+0000","last_active":"2023-10-06T03:49:04.050517+0000","last_peered":"2023-10-06T03:49:04.050517+0000","last_clean":"2023-10-06T03:49:04.050517+0000","last_became_active":"2023-10-06T03:49:03.068472+0000","last_became_peered":"2023-10-06T03:49:03.068472+0000","last_unstale":"2023-10-06T03:49:04.050517+0000","last_undegraded":"2023-10-06T03:49:04.050517+0000","last_fullsized":"2023-10-06T03:49:04.050517+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:04.795631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.081659+0000","last_change":"2023-10-06T03:49:04.074799+0000","last_active":"2023-10-06T03:49:04.081659+0000","last_peered":"2023-10-06T03:49:04.081659+0000","last_clean":"2023-10-06T03:49:04.081659+0000","last_became_active":"2023-10-06T03:49:04.074542+0000","last_became_peered":"2023-10-06T03:49:04.074542+0000","last_unstale":"2023-10-06T03:49:04.081659+0000","last_undegraded":"2023-10-06T03:49:04.081659+0000","last_fullsized":"2023-10-06T03:49:04.081659+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:26:40.677461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.050387+0000","last_change":"2023-10-06T03:49:03.064430+0000","last_active":"2023-10-06T03:49:04.050387+0000","last_peered":"2023-10-06T03:49:04.050387+0000","last_clean":"2023-10-06T03:49:04.050387+0000","last_became_active":"2023-10-06T03:49:03.064176+0000","last_became_peered":"2023-10-06T03:49:03.064176+0000","last_unstale":"2023-10-06T03:49:04.050387+0000","last_undegraded":"2023-10-06T03:49:04.050387+0000","last_fullsized":"2023-10-06T03:49:04.050387+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:47.823447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.060004+0000","last_change":"2023-10-06T03:49:04.058021+0000","last_active":"2023-10-06T03:49:04.060004+0000","last_peered":"2023-10-06T03:49:04.060004+0000","last_clean":"2023-10-06T03:49:04.060004+0000","last_became_active":"2023-10-06T03:49:04.057618+0000","last_became_peered":"2023-10-06T03:49:04.057618+0000","last_unstale":"2023-10-06T03:49:04.060004+0000","last_undegraded":"2023-10-06T03:49:04.060004+0000","last_fullsized":"2023-10-06T03:49:04.060004+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:36.224255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.049844+0000","last_change":"2023-10-06T03:49:03.072712+0000","last_active":"2023-10-06T03:49:04.049844+0000","last_peered":"2023-10-06T03:49:04.049844+0000","last_clean":"2023-10-06T03:49:04.049844+0000","last_became_active":"2023-10-06T03:49:03.071252+0000","last_became_peered":"2023-10-06T03:49:03.071252+0000","last_unstale":"2023-10-06T03:49:04.049844+0000","last_undegraded":"2023-10-06T03:49:04.049844+0000","last_fullsized":"2023-10-06T03:49:04.049844+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:34.223560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.073282+0000","last_change":"2023-10-06T03:49:04.069566+0000","last_active":"2023-10-06T03:49:04.073282+0000","last_peered":"2023-10-06T03:49:04.073282+0000","last_clean":"2023-10-06T03:49:04.073282+0000","last_became_active":"2023-10-06T03:49:04.068487+0000","last_became_peered":"2023-10-06T03:49:04.068487+0000","last_unstale":"2023-10-06T03:49:04.073282+0000","last_undegraded":"2023-10-06T03:49:04.073282+0000","last_fullsized":"2023-10-06T03:49:04.073282+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:59:30.733029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.058485+0000","last_change":"2023-10-06T03:49:03.074109+0000","last_active":"2023-10-06T03:49:04.058485+0000","last_peered":"2023-10-06T03:49:04.058485+0000","last_clean":"2023-10-06T03:49:04.058485+0000","last_became_active":"2023-10-06T03:49:03.073835+0000","last_became_peered":"2023-10-06T03:49:03.073835+0000","last_unstale":"2023-10-06T03:49:04.058485+0000","last_undegraded":"2023-10-06T03:49:04.058485+0000","last_fullsized":"2023-10-06T03:49:04.058485+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:50.590201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.080080+0000","last_change":"2023-10-06T03:49:04.069720+0000","last_active":"2023-10-06T03:49:04.080080+0000","last_peered":"2023-10-06T03:49:04.080080+0000","last_clean":"2023-10-06T03:49:04.080080+0000","last_became_active":"2023-10-06T03:49:04.069348+0000","last_became_peered":"2023-10-06T03:49:04.069348+0000","last_unstale":"2023-10-06T03:49:04.080080+0000","last_undegraded":"2023-10-06T03:49:04.080080+0000","last_fullsized":"2023-10-06T03:49:04.080080+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:32:34.308700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.049722+0000","last_change":"2023-10-06T03:49:03.071620+0000","last_active":"2023-10-06T03:49:04.049722+0000","last_peered":"2023-10-06T03:49:04.049722+0000","last_clean":"2023-10-06T03:49:04.049722+0000","last_became_active":"2023-10-06T03:49:03.071357+0000","last_became_peered":"2023-10-06T03:49:03.071357+0000","last_unstale":"2023-10-06T03:49:04.049722+0000","last_undegraded":"2023-10-06T03:49:04.049722+0000","last_fullsized":"2023-10-06T03:49:04.049722+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:42.913488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.080714+0000","last_change":"2023-10-06T03:49:04.062075+0000","last_active":"2023-10-06T03:49:04.080714+0000","last_peered":"2023-10-06T03:49:04.080714+0000","last_clean":"2023-10-06T03:49:04.080714+0000","last_became_active":"2023-10-06T03:49:04.061801+0000","last_became_peered":"2023-10-06T03:49:04.061801+0000","last_unstale":"2023-10-06T03:49:04.080714+0000","last_undegraded":"2023-10-06T03:49:04.080714+0000","last_fullsized":"2023-10-06T03:49:04.080714+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:03.359619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.058012+0000","last_change":"2023-10-06T03:49:03.064243+0000","last_active":"2023-10-06T03:49:04.058012+0000","last_peered":"2023-10-06T03:49:04.058012+0000","last_clean":"2023-10-06T03:49:04.058012+0000","last_became_active":"2023-10-06T03:49:03.063940+0000","last_became_peered":"2023-10-06T03:49:03.063940+0000","last_unstale":"2023-10-06T03:49:04.058012+0000","last_undegraded":"2023-10-06T03:49:04.058012+0000","last_fullsized":"2023-10-06T03:49:04.058012+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:36:42.656195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.069295+0000","last_change":"2023-10-06T03:49:04.067326+0000","last_active":"2023-10-06T03:49:04.069295+0000","last_peered":"2023-10-06T03:49:04.069295+0000","last_clean":"2023-10-06T03:49:04.069295+0000","last_became_active":"2023-10-06T03:49:04.067031+0000","last_became_peered":"2023-10-06T03:49:04.067031+0000","last_unstale":"2023-10-06T03:49:04.069295+0000","last_undegraded":"2023-10-06T03:49:04.069295+0000","last_fullsized":"2023-10-06T03:49:04.069295+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:35.482312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.083400+0000","last_change":"2023-10-06T03:49:03.067587+0000","last_active":"2023-10-06T03:49:04.083400+0000","last_peered":"2023-10-06T03:49:04.083400+0000","last_clean":"2023-10-06T03:49:04.083400+0000","last_became_active":"2023-10-06T03:49:03.067320+0000","last_became_peered":"2023-10-06T03:49:03.067320+0000","last_unstale":"2023-10-06T03:49:04.083400+0000","last_undegraded":"2023-10-06T03:49:04.083400+0000","last_fullsized":"2023-10-06T03:49:04.083400+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:18.628463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.056178+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:49:04.056178+0000","last_peered":"2023-10-06T03:49:04.056178+0000","last_clean":"2023-10-06T03:49:04.056178+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:49:04.056178+0000","last_undegraded":"2023-10-06T03:49:04.056178+0000","last_fullsized":"2023-10-06T03:49:04.056178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.076670+0000","last_change":"2023-10-06T03:49:04.073612+0000","last_active":"2023-10-06T03:49:04.076670+0000","last_peered":"2023-10-06T03:49:04.076670+0000","last_clean":"2023-10-06T03:49:04.076670+0000","last_became_active":"2023-10-06T03:49:04.073360+0000","last_became_peered":"2023-10-06T03:49:04.073360+0000","last_unstale":"2023-10-06T03:49:04.076670+0000","last_undegraded":"2023-10-06T03:49:04.076670+0000","last_fullsized":"2023-10-06T03:49:04.076670+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:38:20.130519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.086921+0000","last_change":"2023-10-06T03:49:03.071724+0000","last_active":"2023-10-06T03:49:04.086921+0000","last_peered":"2023-10-06T03:49:04.086921+0000","last_clean":"2023-10-06T03:49:04.086921+0000","last_became_active":"2023-10-06T03:49:03.071368+0000","last_became_peered":"2023-10-06T03:49:03.071368+0000","last_unstale":"2023-10-06T03:49:04.086921+0000","last_undegraded":"2023-10-06T03:49:04.086921+0000","last_fullsized":"2023-10-06T03:49:04.086921+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:26:06.794520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.048988+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:49:04.048988+0000","last_peered":"2023-10-06T03:49:04.048988+0000","last_clean":"2023-10-06T03:49:04.048988+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:49:04.048988+0000","last_undegraded":"2023-10-06T03:49:04.048988+0000","last_fullsized":"2023-10-06T03:49:04.048988+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.078674+0000","last_change":"2023-10-06T03:49:04.069234+0000","last_active":"2023-10-06T03:49:04.078674+0000","last_peered":"2023-10-06T03:49:04.078674+0000","last_clean":"2023-10-06T03:49:04.078674+0000","last_became_active":"2023-10-06T03:49:04.068911+0000","last_became_peered":"2023-10-06T03:49:04.068911+0000","last_unstale":"2023-10-06T03:49:04.078674+0000","last_undegraded":"2023-10-06T03:49:04.078674+0000","last_fullsized":"2023-10-06T03:49:04.078674+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:04.079813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.048714+0000","last_change":"2023-10-06T03:49:03.074317+0000","last_active":"2023-10-06T03:49:04.048714+0000","last_peered":"2023-10-06T03:49:04.048714+0000","last_clean":"2023-10-06T03:49:04.048714+0000","last_became_active":"2023-10-06T03:49:03.074062+0000","last_became_peered":"2023-10-06T03:49:03.074062+0000","last_unstale":"2023-10-06T03:49:04.048714+0000","last_undegraded":"2023-10-06T03:49:04.048714+0000","last_fullsized":"2023-10-06T03:49:04.048714+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:08.871796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":77,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.055322+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:49:04.055322+0000","last_peered":"2023-10-06T03:49:04.055322+0000","last_clean":"2023-10-06T03:49:04.055322+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:49:04.055322+0000","last_undegraded":"2023-10-06T03:49:04.055322+0000","last_fullsized":"2023-10-06T03:49:04.055322+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"83'183","reported_seq":462,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.051349+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:49:04.051349+0000","last_peered":"2023-10-06T03:49:04.051349+0000","last_clean":"2023-10-06T03:49:04.051349+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:49:04.051349+0000","last_undegraded":"2023-10-06T03:49:04.051349+0000","last_fullsized":"2023-10-06T03:49:04.051349+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.067266+0000","last_change":"2023-10-06T03:49:04.061483+0000","last_active":"2023-10-06T03:49:04.067266+0000","last_peered":"2023-10-06T03:49:04.067266+0000","last_clean":"2023-10-06T03:49:04.067266+0000","last_became_active":"2023-10-06T03:49:04.061177+0000","last_became_peered":"2023-10-06T03:49:04.061177+0000","last_unstale":"2023-10-06T03:49:04.067266+0000","last_undegraded":"2023-10-06T03:49:04.067266+0000","last_fullsized":"2023-10-06T03:49:04.067266+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:28.005151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.048706+0000","last_change":"2023-10-06T03:49:03.070278+0000","last_active":"2023-10-06T03:49:04.048706+0000","last_peered":"2023-10-06T03:49:04.048706+0000","last_clean":"2023-10-06T03:49:04.048706+0000","last_became_active":"2023-10-06T03:49:03.070008+0000","last_became_peered":"2023-10-06T03:49:03.070008+0000","last_unstale":"2023-10-06T03:49:04.048706+0000","last_undegraded":"2023-10-06T03:49:04.048706+0000","last_fullsized":"2023-10-06T03:49:04.048706+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:04.768736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":79,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.050174+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:49:04.050174+0000","last_peered":"2023-10-06T03:49:04.050174+0000","last_clean":"2023-10-06T03:49:04.050174+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:49:04.050174+0000","last_undegraded":"2023-10-06T03:49:04.050174+0000","last_fullsized":"2023-10-06T03:49:04.050174+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068165+0000","last_change":"2023-10-06T03:49:04.060748+0000","last_active":"2023-10-06T03:49:04.068165+0000","last_peered":"2023-10-06T03:49:04.068165+0000","last_clean":"2023-10-06T03:49:04.068165+0000","last_became_active":"2023-10-06T03:49:04.060450+0000","last_became_peered":"2023-10-06T03:49:04.060450+0000","last_unstale":"2023-10-06T03:49:04.068165+0000","last_undegraded":"2023-10-06T03:49:04.068165+0000","last_fullsized":"2023-10-06T03:49:04.068165+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:03:17.032274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.048640+0000","last_change":"2023-10-06T03:49:03.076714+0000","last_active":"2023-10-06T03:49:04.048640+0000","last_peered":"2023-10-06T03:49:04.048640+0000","last_clean":"2023-10-06T03:49:04.048640+0000","last_became_active":"2023-10-06T03:49:03.076454+0000","last_became_peered":"2023-10-06T03:49:03.076454+0000","last_unstale":"2023-10-06T03:49:04.048640+0000","last_undegraded":"2023-10-06T03:49:04.048640+0000","last_fullsized":"2023-10-06T03:49:04.048640+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:05:04.795631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.051444+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:49:04.051444+0000","last_peered":"2023-10-06T03:49:04.051444+0000","last_clean":"2023-10-06T03:49:04.051444+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:49:04.051444+0000","last_undegraded":"2023-10-06T03:49:04.051444+0000","last_fullsized":"2023-10-06T03:49:04.051444+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.078364+0000","last_change":"2023-10-06T03:49:04.070912+0000","last_active":"2023-10-06T03:49:04.078364+0000","last_peered":"2023-10-06T03:49:04.078364+0000","last_clean":"2023-10-06T03:49:04.078364+0000","last_became_active":"2023-10-06T03:49:04.070649+0000","last_became_peered":"2023-10-06T03:49:04.070649+0000","last_unstale":"2023-10-06T03:49:04.078364+0000","last_undegraded":"2023-10-06T03:49:04.078364+0000","last_fullsized":"2023-10-06T03:49:04.078364+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:25:16.117858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"92'2","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.088565+0000","last_change":"2023-10-06T03:49:03.070399+0000","last_active":"2023-10-06T03:49:04.088565+0000","last_peered":"2023-10-06T03:49:04.088565+0000","last_clean":"2023-10-06T03:49:04.088565+0000","last_became_active":"2023-10-06T03:49:03.070133+0000","last_became_peered":"2023-10-06T03:49:03.070133+0000","last_unstale":"2023-10-06T03:49:04.088565+0000","last_undegraded":"2023-10-06T03:49:04.088565+0000","last_fullsized":"2023-10-06T03:49:04.088565+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:19:00.278339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.055837+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:49:04.055837+0000","last_peered":"2023-10-06T03:49:04.055837+0000","last_clean":"2023-10-06T03:49:04.055837+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:49:04.055837+0000","last_undegraded":"2023-10-06T03:49:04.055837+0000","last_fullsized":"2023-10-06T03:49:04.055837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.068938+0000","last_change":"2023-10-06T03:49:04.064787+0000","last_active":"2023-10-06T03:49:04.068938+0000","last_peered":"2023-10-06T03:49:04.068938+0000","last_clean":"2023-10-06T03:49:04.068938+0000","last_became_active":"2023-10-06T03:49:04.064527+0000","last_became_peered":"2023-10-06T03:49:04.064527+0000","last_unstale":"2023-10-06T03:49:04.068938+0000","last_undegraded":"2023-10-06T03:49:04.068938+0000","last_fullsized":"2023-10-06T03:49:04.068938+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:30:16.695070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.054522+0000","last_change":"2023-10-06T03:49:03.068973+0000","last_active":"2023-10-06T03:49:04.054522+0000","last_peered":"2023-10-06T03:49:04.054522+0000","last_clean":"2023-10-06T03:49:04.054522+0000","last_became_active":"2023-10-06T03:49:03.068629+0000","last_became_peered":"2023-10-06T03:49:03.068629+0000","last_unstale":"2023-10-06T03:49:04.054522+0000","last_undegraded":"2023-10-06T03:49:04.054522+0000","last_fullsized":"2023-10-06T03:49:04.054522+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:19.852270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.050502+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:49:04.050502+0000","last_peered":"2023-10-06T03:49:04.050502+0000","last_clean":"2023-10-06T03:49:04.050502+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:49:04.050502+0000","last_undegraded":"2023-10-06T03:49:04.050502+0000","last_fullsized":"2023-10-06T03:49:04.050502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.085758+0000","last_change":"2023-10-06T03:49:04.077775+0000","last_active":"2023-10-06T03:49:04.085758+0000","last_peered":"2023-10-06T03:49:04.085758+0000","last_clean":"2023-10-06T03:49:04.085758+0000","last_became_active":"2023-10-06T03:49:04.077310+0000","last_became_peered":"2023-10-06T03:49:04.077310+0000","last_unstale":"2023-10-06T03:49:04.085758+0000","last_undegraded":"2023-10-06T03:49:04.085758+0000","last_fullsized":"2023-10-06T03:49:04.085758+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:03.040822+0000","last_clean_scrub_stamp":"2023-10-06T03:49:03.040822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:51.420823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"92'1","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.088398+0000","last_change":"2023-10-06T03:49:03.065837+0000","last_active":"2023-10-06T03:49:04.088398+0000","last_peered":"2023-10-06T03:49:04.088398+0000","last_clean":"2023-10-06T03:49:04.088398+0000","last_became_active":"2023-10-06T03:49:03.065590+0000","last_became_peered":"2023-10-06T03:49:03.065590+0000","last_unstale":"2023-10-06T03:49:04.088398+0000","last_undegraded":"2023-10-06T03:49:04.088398+0000","last_fullsized":"2023-10-06T03:49:04.088398+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:02.032491+0000","last_clean_scrub_stamp":"2023-10-06T03:49:02.032491+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:57.330077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-10-06T03:49:04.050829+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:49:04.050829+0000","last_peered":"2023-10-06T03:49:04.050829+0000","last_clean":"2023-10-06T03:49:04.050829+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:49:04.050829+0000","last_undegraded":"2023-10-06T03:49:04.050829+0000","last_fullsized":"2023-10-06T03:49:04.050829+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575038,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311252,"kb_used_data":1676,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060588,"statfs":{"total":96636764160,"available":96318042112,"internally_reserved":0,"allocated":1716224,"data_stored":1156852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.439,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.81899999999999995,"5min":1.0029999999999999,"15min":1.6100000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.42599999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94399999999999995,"5min":1.02,"15min":1.5649999999999999},"last":0.52500000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.41499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.97599999999999998,"5min":1.038,"15min":1.5469999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.747,"15min":0.76700000000000002},"min":{"1min":0.53100000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.98799999999999999,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.68600000000000005,"15min":0.69799999999999995},"min":{"1min":0.437,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.93500000000000005,"5min":0.97499999999999998,"15min":1.5900000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72499999999999998,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92200000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.749}]},{"osd":3,"last update":"Fri Oct 6 03:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.71599999999999997,"15min":0.72499999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.39500000000000002},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.6279999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.501,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.876,"5min":0.98099999999999998,"15min":1.0409999999999999},"last":0.70799999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.59099999999999997},"min":{"1min":0.42299999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.78600000000000003,"5min":0.96499999999999997,"15min":1.0049999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.624},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.39400000000000002},"max":{"1min":0.85199999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55100000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.436,"5min":0.41199999999999998,"15min":0.36799999999999999},"max":{"1min":0.88400000000000001,"5min":1.018,"15min":1.069},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.50800000000000001,"5min":0.48299999999999998,"15min":0.436},"max":{"1min":0.94299999999999995,"5min":1.105,"15min":1.105},"last":0.68999999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:48:44 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.55500000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.03,"5min":1.181,"15min":1.181},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.49399999999999999,"5min":0.48999999999999999,"15min":0.434},"max":{"1min":0.90600000000000003,"5min":1.02,"15min":1.0209999999999999},"last":0.65700000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575039,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306600,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065240,"statfs":{"total":96636764160,"available":96322805760,"internally_reserved":0,"allocated":1146880,"data_stored":576009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69199999999999995,"15min":0.69899999999999995},"min":{"1min":0.42199999999999999,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.88,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72499999999999998,"15min":0.72099999999999997},"min":{"1min":0.437,"5min":0.41299999999999998,"15min":0.39500000000000002},"max":{"1min":0.94799999999999995,"5min":0.98599999999999999,"15min":1.0449999999999999},"last":0.70599999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.47499999999999998,"5min":0.36499999999999999,"15min":0.33800000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.3180000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.71599999999999997,"15min":0.71399999999999997},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.35899999999999999},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.61499999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.751,"15min":0.751},"min":{"1min":0.48199999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.98699999999999999,"5min":0.995,"15min":3.3319999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.47599999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":3.5139999999999998},"last":0.72199999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.753,"15min":0.75},"min":{"1min":0.58499999999999996,"5min":0.48299999999999998,"15min":0.44600000000000001},"max":{"1min":0.97399999999999998,"5min":1.01,"15min":1.024},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76100000000000001,"15min":0.77400000000000002},"min":{"1min":0.49299999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.98799999999999999,"5min":0.99099999999999999,"15min":3.3300000000000001},"last":0.77800000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57999999999999996,"15min":0.58099999999999996},"min":{"1min":0.45200000000000001,"5min":0.376,"15min":0.34799999999999998},"max":{"1min":0.91900000000000004,"5min":0.92600000000000005,"15min":1.0680000000000001},"last":0.44},{"interface":"front","average":{"1min":0.621,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36499999999999999},"max":{"1min":0.90700000000000003,"5min":1.0169999999999999,"15min":1.2929999999999999},"last":0.54400000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67000000000000004,"15min":0.66700000000000004},"min":{"1min":0.438,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.006,"5min":1.006,"15min":1.3340000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.48999999999999999,"5min":0.42899999999999999,"15min":0.41199999999999998},"max":{"1min":1.0049999999999999,"5min":1.0269999999999999,"15min":1.2370000000000001},"last":0.503}]},{"osd":7,"last update":"Fri Oct 6 03:48:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68600000000000005,"15min":0.69899999999999995},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.619},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.70499999999999996,"15min":0.70999999999999996},"min":{"1min":0.47099999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.417},"last":0.66600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307164,"kb_used_data":1684,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064676,"statfs":{"total":96636764160,"available":96322228224,"internally_reserved":0,"allocated":1724416,"data_stored":1157920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.48199999999999998,"15min":0.48499999999999999},"min":{"1min":0.34699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58399999999999996,"5min":0.73599999999999999,"15min":0.77900000000000003},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.56299999999999994,"15min":0.56699999999999995},"min":{"1min":0.38100000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.624,"5min":1.1319999999999999,"15min":1.2889999999999999},"last":0.47399999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.49099999999999999,"15min":0.49399999999999999},"min":{"1min":0.34899999999999998,"5min":0.32000000000000001,"15min":0.26100000000000001},"max":{"1min":0.70799999999999996,"5min":0.72699999999999998,"15min":0.999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.41399999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.70199999999999996,"5min":0.77300000000000002,"15min":0.94099999999999995},"last":0.44900000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.79600000000000004,"5min":0.86699999999999999,"15min":0.93999999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81499999999999995,"5min":0.90400000000000003,"15min":1.254},"last":0.68000000000000005}]},{"osd":4,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76300000000000001,"15min":0.76200000000000001},"min":{"1min":0.53100000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.3600000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.54800000000000004,"5min":0.439,"15min":0.439},"max":{"1min":0.97099999999999997,"5min":1.2310000000000001,"15min":1.476},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79500000000000004,"15min":0.79200000000000004},"min":{"1min":0.54600000000000004,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":0.95899999999999996,"5min":1.294,"15min":1.601},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80700000000000005,"15min":0.80500000000000005},"min":{"1min":0.622,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.165,"5min":1.224,"15min":1.5189999999999999},"last":0.72199999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79900000000000004,"15min":0.80900000000000005},"min":{"1min":0.54400000000000004,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1339999999999999,"5min":1.26,"15min":1.26},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.52800000000000002,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":1.0109999999999999,"5min":1.143,"15min":1.399},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84199999999999997,"15min":0.83699999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.1519999999999999,"5min":1.272,"15min":1.272},"last":0.747},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83599999999999997,"15min":0.83199999999999996},"min":{"1min":0.54000000000000004,"5min":0.53700000000000003,"15min":0.51200000000000001},"max":{"1min":1.1950000000000001,"5min":1.2010000000000001,"15min":1.556},"last":0.74099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575040,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306584,"kb_used_data":1104,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065256,"statfs":{"total":96636764160,"available":96322822144,"internally_reserved":0,"allocated":1130496,"data_stored":573495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46899999999999997,"15min":0.48599999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26900000000000002},"max":{"1min":0.64800000000000002,"5min":0.81000000000000005,"15min":2.3220000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.499,"15min":0.52200000000000002},"min":{"1min":0.33700000000000002,"5min":0.33600000000000002,"15min":0.29599999999999999},"max":{"1min":0.67900000000000005,"5min":0.78500000000000003,"15min":2.1440000000000001},"last":0.438}]},{"osd":2,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.52800000000000002,"15min":0.55500000000000005},"min":{"1min":0.33800000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":2.4329999999999998},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54200000000000004,"15min":0.55700000000000005},"min":{"1min":0.40500000000000003,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83399999999999996,"5min":0.84699999999999998,"15min":2.1819999999999999},"last":0.54400000000000004}]},{"osd":3,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.50700000000000001,"15min":0.52500000000000002},"min":{"1min":0.35099999999999998,"5min":0.30499999999999999,"15min":0.28399999999999997},"max":{"1min":0.75700000000000001,"5min":0.81699999999999995,"15min":2.2709999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.60099999999999998,"15min":0.61899999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84599999999999997,"5min":1.0580000000000001,"15min":2.4700000000000002},"last":0.46800000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.82299999999999995,"15min":0.83499999999999996},"min":{"1min":0.57099999999999995,"5min":0.54200000000000004,"15min":0.49399999999999999},"max":{"1min":1.1910000000000001,"5min":1.2589999999999999,"15min":1.393},"last":0.752},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80400000000000005,"15min":0.82999999999999996},"min":{"1min":0.55700000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.256,"5min":1.306,"15min":2.2320000000000002},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79800000000000004,"15min":0.82299999999999995},"min":{"1min":0.58099999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.091,"5min":1.2869999999999999,"15min":2.3660000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.77700000000000002,"15min":0.80700000000000005},"min":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.6559999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.78200000000000003,"15min":0.80900000000000005},"min":{"1min":0.58499999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1080000000000001,"5min":1.137,"15min":1.726},"last":0.621},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81399999999999995,"15min":0.84899999999999998},"min":{"1min":0.54700000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.167,"5min":1.167,"15min":2.4009999999999998},"last":0.73499999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:48:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81299999999999994,"15min":0.82299999999999995},"min":{"1min":0.63,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.302,"5min":1.302,"15min":1.302},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80000000000000004,"15min":0.83599999999999997},"min":{"1min":0.53400000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.1519999999999999,"5min":1.2450000000000001,"15min":1.7649999999999999},"last":0.59599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575039,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310692,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061148,"statfs":{"total":96636764160,"available":96318615552,"internally_reserved":0,"allocated":1142784,"data_stored":574209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.46999999999999997,"15min":0.46899999999999997},"min":{"1min":0.34300000000000003,"5min":0.29699999999999999,"15min":0.28000000000000003},"max":{"1min":0.56200000000000006,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.38500000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63700000000000001,"5min":0.83599999999999997,"15min":0.89300000000000002},"last":0.41399999999999998}]},{"osd":1,"last update":"Fri Oct 6 03:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.48599999999999999,"15min":0.48699999999999999},"min":{"1min":0.34100000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66100000000000003,"5min":0.73799999999999999,"15min":0.90000000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54700000000000004,"15min":0.55600000000000005},"min":{"1min":0.39200000000000002,"5min":0.372,"15min":0.34200000000000003},"max":{"1min":0.84199999999999997,"5min":0.85099999999999998,"15min":1.556},"last":0.439}]},{"osd":3,"last update":"Fri Oct 6 03:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.439,"5min":0.41399999999999998,"15min":0.38800000000000001},"max":{"1min":0.73199999999999998,"5min":1.78,"15min":1.78},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.70199999999999996,"5min":0.86599999999999999,"15min":1.1910000000000001},"last":0.627}]},{"osd":4,"last update":"Fri Oct 6 03:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.59399999999999997,"5min":0.51400000000000001,"15min":0.439},"max":{"1min":1.0169999999999999,"5min":1.0649999999999999,"15min":1.5620000000000001},"last":0.746},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82899999999999996,"15min":0.83699999999999997},"min":{"1min":0.58799999999999997,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0620000000000001,"5min":1.1279999999999999,"15min":1.2270000000000001},"last":0.53100000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.81999999999999995,"15min":0.83399999999999996},"min":{"1min":0.66500000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.97199999999999998,"5min":1.169,"15min":1.3},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.85199999999999998},"min":{"1min":0.56399999999999995,"5min":0.55500000000000005,"15min":0.51700000000000002},"max":{"1min":1.0980000000000001,"5min":1.855,"15min":1.855},"last":0.68000000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82599999999999996,"15min":0.83799999999999997},"min":{"1min":0.55300000000000005,"5min":0.54200000000000004,"15min":0.53600000000000003},"max":{"1min":1.056,"5min":1.907,"15min":1.907},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.81299999999999994,"15min":0.81899999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.51100000000000001},"max":{"1min":1.0169999999999999,"5min":1.1080000000000001,"15min":1.4750000000000001},"last":0.86899999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:48:43 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.84399999999999997,"15min":0.84199999999999997},"min":{"1min":0.55900000000000005,"5min":0.495,"15min":0.495},"max":{"1min":1.113,"5min":1.4379999999999999,"15min":1.4990000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.86399999999999999,"15min":0.86799999999999999},"min":{"1min":0.59699999999999998,"5min":0.53100000000000003,"15min":0.497},"max":{"1min":1.123,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.63800000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575039,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306592,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065248,"statfs":{"total":96636764160,"available":96322813952,"internally_reserved":0,"allocated":1138688,"data_stored":575027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.32700000000000001,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.75700000000000001,"5min":0.83999999999999997,"15min":0.95699999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.35099999999999998,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":0.61199999999999999,"5min":1.04,"15min":1.1699999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47699999999999998,"15min":0.47799999999999998},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.63900000000000001,"5min":0.68799999999999994,"15min":0.79500000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53500000000000003,"15min":0.54000000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.83899999999999997},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.34599999999999997},"max":{"1min":0.69899999999999995,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59099999999999997,"15min":0.60199999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.38700000000000001},"max":{"1min":0.68600000000000005,"5min":0.81999999999999995,"15min":0.97899999999999998},"last":0.48899999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76800000000000002,"15min":0.78700000000000003},"min":{"1min":0.48699999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1,"5min":1.073,"15min":1.4099999999999999},"last":0.629},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79800000000000004,"15min":0.82799999999999996},"min":{"1min":0.622,"5min":0.57399999999999995,"15min":0.51300000000000001},"max":{"1min":0.96899999999999997,"5min":1.0469999999999999,"15min":1.377},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.79000000000000004},"min":{"1min":0.58999999999999997,"5min":0.51900000000000002,"15min":0.47099999999999997},"max":{"1min":1.012,"5min":1.05,"15min":1.256},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.75600000000000001,"15min":0.78700000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.48099999999999998},"max":{"1min":0.93100000000000005,"5min":1.1100000000000001,"15min":1.306},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79300000000000004,"15min":0.81399999999999995},"min":{"1min":0.56999999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"max":{"1min":0.98999999999999999,"5min":1.1399999999999999,"15min":1.323},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81799999999999995,"15min":0.84999999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.49299999999999999},"max":{"1min":1.05,"5min":1.1040000000000001,"15min":1.6020000000000001},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:48:28 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79100000000000004,"15min":0.81399999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.86799999999999999,"5min":1.2030000000000001,"15min":1.365},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78900000000000003,"15min":0.80800000000000005},"min":{"1min":0.55800000000000005,"5min":0.54600000000000004,"15min":0.46300000000000002},"max":{"1min":1.0289999999999999,"5min":1.093,"15min":1.2330000000000001},"last":0.69099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575039,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302512,"kb_used_data":1128,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069328,"statfs":{"total":96636764160,"available":96326991872,"internally_reserved":0,"allocated":1155072,"data_stored":576113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69399999999999995,"15min":0.71999999999999997},"min":{"1min":0.51000000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.94799999999999995,"5min":1.0109999999999999,"15min":1.3120000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67500000000000004,"15min":0.69699999999999995},"min":{"1min":0.44700000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.84999999999999998,"5min":0.95299999999999996,"15min":1.01},"last":0.63800000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.70499999999999996,"15min":0.72599999999999998},"min":{"1min":0.42999999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.0600000000000001,"5min":1.081,"15min":1.081},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71599999999999997,"15min":0.73599999999999999},"min":{"1min":0.51500000000000001,"5min":0.49299999999999999,"15min":0.41299999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.3380000000000001},"last":0.747}]},{"osd":2,"last update":"Fri Oct 6 03:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71099999999999997,"15min":0.73699999999999999},"min":{"1min":0.47599999999999998,"5min":0.44400000000000001,"15min":0.377},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1759999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73799999999999999,"15min":0.745},"min":{"1min":0.52000000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.92900000000000005,"5min":1.022,"15min":1.3089999999999999},"last":0.69099999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.46300000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82999999999999996,"5min":1.0049999999999999,"15min":1.0409999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.54300000000000004,"5min":0.47099999999999997,"15min":0.46500000000000002},"max":{"1min":1.1040000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.45000000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.085},"last":0.622},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63300000000000001,"15min":0.63200000000000001},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.35099999999999998},"max":{"1min":0.85599999999999998,"5min":0.93500000000000005,"15min":1.1439999999999999},"last":0.82299999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65100000000000002,"15min":0.64700000000000002},"min":{"1min":0.49399999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1839999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.38100000000000001},"max":{"1min":1.016,"5min":1.052,"15min":1.361},"last":0.78200000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.68600000000000005,"15min":0.68799999999999994},"min":{"1min":0.57099999999999995,"5min":0.46500000000000002,"15min":0.39900000000000002},"max":{"1min":0.997,"5min":0.997,"15min":1.1950000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72399999999999998,"15min":0.71899999999999997},"min":{"1min":0.53600000000000003,"5min":0.51300000000000001,"15min":0.41299999999999998},"max":{"1min":1.214,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.83699999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575039,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298384,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073456,"statfs":{"total":96636764160,"available":96331218944,"internally_reserved":0,"allocated":1122304,"data_stored":566496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:48:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69299999999999995,"15min":0.70699999999999996},"min":{"1min":0.39200000000000002,"5min":0.35999999999999999,"15min":0.35299999999999998},"max":{"1min":0.88500000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69699999999999995,"15min":0.71099999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.32600000000000001},"max":{"1min":0.90600000000000003,"5min":0.96899999999999997,"15min":1},"last":0.63400000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:48:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69999999999999996,"15min":0.71399999999999997},"min":{"1min":0.499,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.97599999999999998,"5min":0.998,"15min":1.0429999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70499999999999996,"15min":0.72199999999999998},"min":{"1min":0.58199999999999996,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.94499999999999995,"5min":1.012,"15min":1.012},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:48:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.55000000000000004,"5min":0.502,"15min":0.40200000000000002},"max":{"1min":0.92100000000000004,"5min":1.046,"15min":1.046},"last":0.751},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.75,"15min":0.77300000000000002},"min":{"1min":0.51200000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94199999999999995,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.88300000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:48:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.73199999999999998},"min":{"1min":0.495,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.6419999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.747,"15min":0.75},"min":{"1min":0.60999999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89000000000000001,"5min":1.117,"15min":1.117},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:48:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.628,"15min":0.61499999999999999},"min":{"1min":0.441,"5min":0.40400000000000003,"15min":0.38300000000000001},"max":{"1min":0.879,"5min":0.95999999999999996,"15min":1.024},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.626,"15min":0.63100000000000001},"min":{"1min":0.41099999999999998,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":1.036,"5min":1.044,"15min":1.175},"last":0.71899999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:48:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.42899999999999999,"5min":0.40200000000000002,"15min":0.34999999999999998},"max":{"1min":0.93300000000000005,"5min":1.591,"15min":1.591},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.70099999999999996,"15min":0.69399999999999995},"min":{"1min":0.496,"5min":0.44600000000000001,"15min":0.40600000000000003},"max":{"1min":1.0169999999999999,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.86099999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:48:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67800000000000005,"15min":0.67900000000000005},"min":{"1min":0.52800000000000002,"5min":0.44600000000000001,"15min":0.40699999999999997},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.2589999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71399999999999997,"15min":0.71899999999999997},"min":{"1min":0.52600000000000002,"5min":0.498,"15min":0.435},"max":{"1min":0.97799999999999998,"5min":1.278,"15min":1.278},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:49:08.909 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:49:08.909 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:49:09.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.134+0000 7f802659d700 1 Processor -- start 2023-10-06T03:49:09.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.135+0000 7f802659d700 1 -- start start 2023-10-06T03:49:09.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.136+0000 7f802659d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f802006e630 0x7f80200abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:09.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.136+0000 7f802659d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f802015df30 0x7f80200ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:09.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.136+0000 7f802659d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8020063870 con 0x7f802006ea00 2023-10-06T03:49:09.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.136+0000 7f802659d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80200acda0 con 0x7f802015df30 2023-10-06T03:49:09.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.136+0000 7f802659d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80200acac0 con 0x7f802006e630 2023-10-06T03:49:09.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.136+0000 7f8024d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f802015df30 0x7f80200ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:09.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.136+0000 7f8024d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f802015df30 0x7f80200ac490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50086/0 (socket says 172.21.15.55:50086) 2023-10-06T03:49:09.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.136+0000 7f8024d9a700 1 -- 172.21.15.55:0/2348439241 learned_addr learned my addr 172.21.15.55:0/2348439241 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.136+0000 7f802559b700 1 --2- 172.21.15.55:0/2348439241 >> v2:172.21.15.163:3300/0 conn(0x7f802006e630 0x7f80200abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.137+0000 7f8024d9a700 1 -- 172.21.15.55:0/2348439241 >> v2:172.21.15.163:3300/0 conn(0x7f802006e630 msgr2=0x7f80200abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.137+0000 7f8024d9a700 1 --2- 172.21.15.55:0/2348439241 >> v2:172.21.15.163:3300/0 conn(0x7f802006e630 0x7f80200abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.137+0000 7f8024d9a700 1 -- 172.21.15.55:0/2348439241 >> v1:172.21.15.163:6789/0 conn(0x7f802006ea00 legacy=0x7f8020159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.137+0000 7f8024d9a700 1 -- 172.21.15.55:0/2348439241 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802015d450 con 0x7f802015df30 2023-10-06T03:49:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.137+0000 7f802559b700 1 --2- 172.21.15.55:0/2348439241 >> v2:172.21.15.163:3300/0 conn(0x7f802006e630 0x7f80200abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.137+0000 7f8024d9a700 1 --2- 172.21.15.55:0/2348439241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802015df30 0x7f80200ac490 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f80200645c0 tx=0x7f8010009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875f0ffa46a9e2aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.137+0000 7f8017fff700 1 -- 172.21.15.55:0/2348439241 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801000b820 con 0x7f802015df30 2023-10-06T03:49:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.137+0000 7f8017fff700 1 -- 172.21.15.55:0/2348439241 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801000b980 con 0x7f802015df30 2023-10-06T03:49:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.137+0000 7f8017fff700 1 -- 172.21.15.55:0/2348439241 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801000bd20 con 0x7f802015df30 2023-10-06T03:49:09.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.138+0000 7f802659d700 1 -- 172.21.15.55:0/2348439241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802015df30 msgr2=0x7f80200ac490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.138+0000 7f802659d700 1 --2- 172.21.15.55:0/2348439241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802015df30 0x7f80200ac490 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f80200645c0 tx=0x7f8010009250 comp rx=0 tx=0).stop 2023-10-06T03:49:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.138+0000 7f802659d700 1 -- 172.21.15.55:0/2348439241 shutdown_connections 2023-10-06T03:49:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.138+0000 7f802659d700 1 --2- 172.21.15.55:0/2348439241 >> v2:172.21.15.163:3300/0 conn(0x7f802006e630 0x7f80200abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.138+0000 7f802659d700 1 --2- 172.21.15.55:0/2348439241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802015df30 0x7f80200ac490 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.138+0000 7f802659d700 1 -- 172.21.15.55:0/2348439241 >> 172.21.15.55:0/2348439241 conn(0x7f80200c9f50 msgr2=0x7f802006dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.138+0000 7f802659d700 1 -- 172.21.15.55:0/2348439241 shutdown_connections 2023-10-06T03:49:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.138+0000 7f802659d700 1 -- 172.21.15.55:0/2348439241 wait complete. 2023-10-06T03:49:09.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.139+0000 7f802659d700 1 Processor -- start 2023-10-06T03:49:09.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.139+0000 7f802659d700 1 -- start start 2023-10-06T03:49:09.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.140+0000 7f802659d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f802006e630 0x7f802015f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:09.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.140+0000 7f802659d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802006ea00 0x7f8020154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:09.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.140+0000 7f802659d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802015df30 0x7f8020142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:09.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.140+0000 7f802659d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80200c4fa0 con 0x7f802015df30 2023-10-06T03:49:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.140+0000 7f802659d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80200c4ce0 con 0x7f802006e630 2023-10-06T03:49:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.140+0000 7f802659d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802015f730 con 0x7f802006ea00 2023-10-06T03:49:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.141+0000 7f8025d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802015df30 0x7f8020142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.141+0000 7f802559b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f802006e630 0x7f802015f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.141+0000 7f8025d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802015df30 0x7f8020142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50096/0 (socket says 172.21.15.55:50096) 2023-10-06T03:49:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.141+0000 7f8025d9c700 1 -- 172.21.15.55:0/3138113703 learned_addr learned my addr 172.21.15.55:0/3138113703 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.141+0000 7f802559b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f802006e630 0x7f802015f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:32778/0 (socket says 172.21.15.55:32778) 2023-10-06T03:49:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.141+0000 7f8024d9a700 1 --2- 172.21.15.55:0/3138113703 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802006ea00 0x7f8020154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.141+0000 7f8025d9c700 1 -- 172.21.15.55:0/3138113703 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802006ea00 msgr2=0x7f8020154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.141+0000 7f8025d9c700 1 --2- 172.21.15.55:0/3138113703 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802006ea00 0x7f8020154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.141+0000 7f8025d9c700 1 -- 172.21.15.55:0/3138113703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f802006e630 msgr2=0x7f802015f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.141+0000 7f8025d9c700 1 --2- 172.21.15.55:0/3138113703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f802006e630 0x7f802015f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.141+0000 7f8025d9c700 1 -- 172.21.15.55:0/3138113703 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80100094f0 con 0x7f802015df30 2023-10-06T03:49:09.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.141+0000 7f8025d9c700 1 --2- 172.21.15.55:0/3138113703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802015df30 0x7f8020142050 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f801800c930 tx=0x7f80180026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:09.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.142+0000 7f80157fa700 1 -- 172.21.15.55:0/3138113703 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018012630 con 0x7f802015df30 2023-10-06T03:49:09.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.142+0000 7f80157fa700 1 -- 172.21.15.55:0/3138113703 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80180127d0 con 0x7f802015df30 2023-10-06T03:49:09.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.142+0000 7f80157fa700 1 -- 172.21.15.55:0/3138113703 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8018012b90 con 0x7f802015df30 2023-10-06T03:49:09.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.142+0000 7f802659d700 1 -- 172.21.15.55:0/3138113703 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8020160230 con 0x7f802015df30 2023-10-06T03:49:09.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.142+0000 7f802659d700 1 -- 172.21.15.55:0/3138113703 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802015fc70 con 0x7f802015df30 2023-10-06T03:49:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.143+0000 7f802659d700 1 -- 172.21.15.55:0/3138113703 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80200c5110 con 0x7f802015df30 2023-10-06T03:49:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.144+0000 7f80157fa700 1 -- 172.21.15.55:0/3138113703 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f801801a3b0 con 0x7f802015df30 2023-10-06T03:49:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.145+0000 7f80157fa700 1 --2- 172.21.15.55:0/3138113703 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8004066f30 0x7f80040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.145+0000 7f80157fa700 1 -- 172.21.15.55:0/3138113703 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f8018084510 con 0x7f802015df30 2023-10-06T03:49:09.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.145+0000 7f802559b700 1 --2- 172.21.15.55:0/3138113703 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8004066f30 0x7f80040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:09.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.146+0000 7f802559b700 1 --2- 172.21.15.55:0/3138113703 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8004066f30 0x7f80040693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f801c006fd0 tx=0x7f801c008040 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:09.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.149+0000 7f80157fa700 1 -- 172.21.15.55:0/3138113703 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80200c5110 con 0x7f802015df30 2023-10-06T03:49:09.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.431+0000 7f802659d700 1 -- 172.21.15.55:0/3138113703 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f80200c50e0 con 0x7f802015df30 2023-10-06T03:49:09.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.435+0000 7f80157fa700 1 -- 172.21.15.55:0/3138113703 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f8018054bf0 con 0x7f802015df30 2023-10-06T03:49:09.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.438+0000 7f802659d700 1 -- 172.21.15.55:0/3138113703 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8004066f30 msgr2=0x7f80040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:09.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.438+0000 7f802659d700 1 --2- 172.21.15.55:0/3138113703 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8004066f30 0x7f80040693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f801c006fd0 tx=0x7f801c008040 comp rx=0 tx=0).stop 2023-10-06T03:49:09.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.438+0000 7f802659d700 1 -- 172.21.15.55:0/3138113703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802015df30 msgr2=0x7f8020142050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:09.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.438+0000 7f802659d700 1 --2- 172.21.15.55:0/3138113703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802015df30 0x7f8020142050 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f801800c930 tx=0x7f80180026e0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.438+0000 7f802659d700 1 -- 172.21.15.55:0/3138113703 shutdown_connections 2023-10-06T03:49:09.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.438+0000 7f802659d700 1 --2- 172.21.15.55:0/3138113703 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8004066f30 0x7f80040693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.438+0000 7f802659d700 1 --2- 172.21.15.55:0/3138113703 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f802015df30 0x7f8020142050 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.438+0000 7f802659d700 1 --2- 172.21.15.55:0/3138113703 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f802006e630 0x7f802015f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.438+0000 7f802659d700 1 --2- 172.21.15.55:0/3138113703 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f802006ea00 0x7f8020154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.438+0000 7f802659d700 1 -- 172.21.15.55:0/3138113703 >> 172.21.15.55:0/3138113703 conn(0x7f80200c9f50 msgr2=0x7f80200c3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:09.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.438+0000 7f802659d700 1 -- 172.21.15.55:0/3138113703 shutdown_connections 2023-10-06T03:49:09.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.438+0000 7f802659d700 1 -- 172.21.15.55:0/3138113703 wait complete. 2023-10-06T03:49:09.441 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2023-10-06T03:49:09.457 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:49:09.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.643+0000 7fd07a99d700 1 Processor -- start 2023-10-06T03:49:09.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.643+0000 7fd07a99d700 1 -- start start 2023-10-06T03:49:09.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.644+0000 7fd07a99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd0741597f0 0x7fd0740b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:09.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.644+0000 7fd07a99d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0740b11d0 con 0x7fd07415d230 2023-10-06T03:49:09.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.644+0000 7fd07a99d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0740b14b0 con 0x7fd074159420 2023-10-06T03:49:09.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.644+0000 7fd07a99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0740b1790 con 0x7fd0741597f0 2023-10-06T03:49:09.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.645+0000 7fd07919a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd0741597f0 0x7fd0740b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:09.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.645+0000 7fd07919a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd0741597f0 0x7fd0740b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50098/0 (socket says 172.21.15.55:50098) 2023-10-06T03:49:09.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.645+0000 7fd07919a700 1 -- 172.21.15.55:0/4129545016 learned_addr learned my addr 172.21.15.55:0/4129545016 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:09.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.645+0000 7fd07a19c700 1 --1- 172.21.15.55:0/4129545016 >> v1:172.21.15.55:6789/0 conn(0x7fd07415d230 0x7fd07415d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57428/0 (socket says 172.21.15.55:57428) 2023-10-06T03:49:09.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.645+0000 7fd07919a700 1 -- 172.21.15.55:0/4129545016 >> v1:172.21.15.163:6789/0 conn(0x7fd074159420 legacy=0x7fd0740b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:09.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.645+0000 7fd07919a700 1 -- 172.21.15.55:0/4129545016 >> v1:172.21.15.55:6789/0 conn(0x7fd07415d230 legacy=0x7fd07415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:09.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.645+0000 7fd07919a700 1 -- 172.21.15.55:0/4129545016 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0740b1a70 con 0x7fd0741597f0 2023-10-06T03:49:09.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.645+0000 7fd078999700 1 -- 172.21.15.55:0/4129545016 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474717352 0 0) 0x7fd0740b11d0 con 0x7fd07415d230 2023-10-06T03:49:09.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.645+0000 7fd07919a700 1 --2- 172.21.15.55:0/4129545016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0741597f0 0x7fd0740b2270 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd0640099b0 tx=0x7fd064009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e2bfa6fdcc5a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:09.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.646+0000 7fd078999700 1 -- 172.21.15.55:0/4129545016 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06400b860 con 0x7fd0741597f0 2023-10-06T03:49:09.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.646+0000 7fd078999700 1 -- 172.21.15.55:0/4129545016 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06400b9c0 con 0x7fd0741597f0 2023-10-06T03:49:09.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.646+0000 7fd078999700 1 -- 172.21.15.55:0/4129545016 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06400bd60 con 0x7fd0741597f0 2023-10-06T03:49:09.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.646+0000 7fd07a99d700 1 -- 172.21.15.55:0/4129545016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0741597f0 msgr2=0x7fd0740b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:09.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.646+0000 7fd07a99d700 1 --2- 172.21.15.55:0/4129545016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0741597f0 0x7fd0740b2270 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd0640099b0 tx=0x7fd064009250 comp rx=0 tx=0).stop 2023-10-06T03:49:09.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.646+0000 7fd07a99d700 1 -- 172.21.15.55:0/4129545016 shutdown_connections 2023-10-06T03:49:09.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.646+0000 7fd07a99d700 1 --2- 172.21.15.55:0/4129545016 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0741597f0 0x7fd0740b2270 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.646+0000 7fd07a99d700 1 -- 172.21.15.55:0/4129545016 >> 172.21.15.55:0/4129545016 conn(0x7fd0740d0660 msgr2=0x7fd0740d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:09.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.646+0000 7fd07a99d700 1 -- 172.21.15.55:0/4129545016 shutdown_connections 2023-10-06T03:49:09.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.646+0000 7fd07a99d700 1 -- 172.21.15.55:0/4129545016 wait complete. 2023-10-06T03:49:09.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.648+0000 7fd07a99d700 1 Processor -- start 2023-10-06T03:49:09.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.648+0000 7fd07a99d700 1 -- start start 2023-10-06T03:49:09.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07a99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd074159420 0x7fd0740c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:09.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07a99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0741597f0 0x7fd07415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:09.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07a99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd07415d230 0x7fd0740cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:09.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07a99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07415b140 con 0x7fd074159420 2023-10-06T03:49:09.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07a99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07415bf40 con 0x7fd0741597f0 2023-10-06T03:49:09.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07a99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07415b330 con 0x7fd07415d230 2023-10-06T03:49:09.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07999b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd074159420 0x7fd0740c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:09.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07999b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd074159420 0x7fd0740c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50104/0 (socket says 172.21.15.55:50104) 2023-10-06T03:49:09.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07999b700 1 -- 172.21.15.55:0/2194545971 learned_addr learned my addr 172.21.15.55:0/2194545971 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:09.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07999b700 1 -- 172.21.15.55:0/2194545971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd07415d230 msgr2=0x7fd0740cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07999b700 1 --2- 172.21.15.55:0/2194545971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd07415d230 0x7fd0740cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07999b700 1 -- 172.21.15.55:0/2194545971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0741597f0 msgr2=0x7fd07415ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07919a700 1 --2- 172.21.15.55:0/2194545971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0741597f0 0x7fd07415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07999b700 1 --2- 172.21.15.55:0/2194545971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0741597f0 0x7fd07415ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07999b700 1 -- 172.21.15.55:0/2194545971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0640094f0 con 0x7fd074159420 2023-10-06T03:49:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.649+0000 7fd07919a700 1 --2- 172.21.15.55:0/2194545971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0741597f0 0x7fd07415ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.650+0000 7fd07999b700 1 --2- 172.21.15.55:0/2194545971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd074159420 0x7fd0740c6dd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd07000c590 tx=0x7fd07000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.650+0000 7fd069ffb700 1 -- 172.21.15.55:0/2194545971 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd070011590 con 0x7fd074159420 2023-10-06T03:49:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.650+0000 7fd069ffb700 1 -- 172.21.15.55:0/2194545971 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0700116f0 con 0x7fd074159420 2023-10-06T03:49:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.650+0000 7fd069ffb700 1 -- 172.21.15.55:0/2194545971 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd070011ab0 con 0x7fd074159420 2023-10-06T03:49:09.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.650+0000 7fd07a99d700 1 -- 172.21.15.55:0/2194545971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07415b610 con 0x7fd074159420 2023-10-06T03:49:09.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.650+0000 7fd07a99d700 1 -- 172.21.15.55:0/2194545971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07415bbd0 con 0x7fd074159420 2023-10-06T03:49:09.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.651+0000 7fd07a99d700 1 -- 172.21.15.55:0/2194545971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07404c3c0 con 0x7fd074159420 2023-10-06T03:49:09.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.652+0000 7fd069ffb700 1 -- 172.21.15.55:0/2194545971 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd07001b3b0 con 0x7fd074159420 2023-10-06T03:49:09.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.653+0000 7fd069ffb700 1 --2- 172.21.15.55:0/2194545971 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd058066f30 0x7fd0580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:09.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.653+0000 7fd069ffb700 1 -- 172.21.15.55:0/2194545971 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fd070083e70 con 0x7fd074159420 2023-10-06T03:49:09.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.653+0000 7fd07919a700 1 --2- 172.21.15.55:0/2194545971 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd058066f30 0x7fd0580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:09.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.654+0000 7fd07919a700 1 --2- 172.21.15.55:0/2194545971 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd058066f30 0x7fd0580693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd0640099b0 tx=0x7fd064017000 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:09.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.657+0000 7fd069ffb700 1 -- 172.21.15.55:0/2194545971 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd070050d50 con 0x7fd074159420 2023-10-06T03:49:09.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.902+0000 7fd07a99d700 1 -- 172.21.15.55:0/2194545971 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0740b1d50 con 0x7fd074159420 2023-10-06T03:49:09.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.903+0000 7fd069ffb700 1 -- 172.21.15.55:0/2194545971 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fd070017020 con 0x7fd074159420 2023-10-06T03:49:09.904 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:09.904 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/4087100648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4087100648},{"type":"v1","addr":"172.21.15.163:6834","nonce":4087100648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2017152000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2017152000},{"type":"v1","addr":"172.21.15.163:6836","nonce":2017152000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1226557536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1226557536},{"type":"v1","addr":"172.21.15.55:6834","nonce":1226557536}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:49:03.547622+0000","modified":"2023-10-06T03:49:07.074089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6804":{"gid":6804,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/3960749421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3960749421},{"type":"v1","addr":"172.21.15.55:6835","nonce":3960749421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T03:49:09.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.905+0000 7fd07a99d700 1 -- 172.21.15.55:0/2194545971 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd058066f30 msgr2=0x7fd0580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:09.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.905+0000 7fd07a99d700 1 --2- 172.21.15.55:0/2194545971 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd058066f30 0x7fd0580693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd0640099b0 tx=0x7fd064017000 comp rx=0 tx=0).stop 2023-10-06T03:49:09.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.905+0000 7fd07a99d700 1 -- 172.21.15.55:0/2194545971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd074159420 msgr2=0x7fd0740c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:09.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.905+0000 7fd07a99d700 1 --2- 172.21.15.55:0/2194545971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd074159420 0x7fd0740c6dd0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd07000c590 tx=0x7fd07000c950 comp rx=0 tx=0).stop 2023-10-06T03:49:09.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.905+0000 7fd07a99d700 1 -- 172.21.15.55:0/2194545971 shutdown_connections 2023-10-06T03:49:09.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.905+0000 7fd07a99d700 1 --2- 172.21.15.55:0/2194545971 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd058066f30 0x7fd0580693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.905+0000 7fd07a99d700 1 --2- 172.21.15.55:0/2194545971 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd07415d230 0x7fd0740cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.905+0000 7fd07a99d700 1 --2- 172.21.15.55:0/2194545971 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd074159420 0x7fd0740c6dd0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.905+0000 7fd07a99d700 1 --2- 172.21.15.55:0/2194545971 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd0741597f0 0x7fd07415ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:09.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.905+0000 7fd07a99d700 1 -- 172.21.15.55:0/2194545971 >> 172.21.15.55:0/2194545971 conn(0x7fd0740d0660 msgr2=0x7fd074156a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:09.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.905+0000 7fd07a99d700 1 -- 172.21.15.55:0/2194545971 shutdown_connections 2023-10-06T03:49:09.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:09.905+0000 7fd07a99d700 1 -- 172.21.15.55:0/2194545971 wait complete. 2023-10-06T03:49:09.908 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 44 2023-10-06T03:49:10.033 INFO:tasks.ceph.mds.c.smithi055.stderr:2023-10-06T03:49:10.032+0000 7fb5401f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:49:10.327 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:49:03.547622+0000', 'modified': '2023-10-06T03:49:07.074089+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6804}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6804': {'gid': 6804, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6835/3960749421', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6833', 'nonce': 3960749421}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 3960749421}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:49:10.328 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:49:10.328 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T03:49:10.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.518+0000 7fa5ba59d700 1 Processor -- start 2023-10-06T03:49:10.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.518+0000 7fa5ba59d700 1 -- start start 2023-10-06T03:49:10.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.519+0000 7fa5ba59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa5b4156310 0x7fa5b40c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.519+0000 7fa5ba59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5b41566e0 0x7fa5b40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.519+0000 7fa5ba59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa5b40ca300 0x7fa5b40ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.519+0000 7fa5ba59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b40ca860 con 0x7fa5b40ca300 2023-10-06T03:49:10.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.519+0000 7fa5ba59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b40cab40 con 0x7fa5b4156310 2023-10-06T03:49:10.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.519+0000 7fa5ba59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40cae20 con 0x7fa5b41566e0 2023-10-06T03:49:10.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.520+0000 7fa5b9d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa5b40ca300 0x7fa5b40ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.520+0000 7fa5b9d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa5b40ca300 0x7fa5b40ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50110/0 (socket says 172.21.15.55:50110) 2023-10-06T03:49:10.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.520+0000 7fa5b9d9c700 1 -- 172.21.15.55:0/166629681 learned_addr learned my addr 172.21.15.55:0/166629681 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:10.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.520+0000 7fa5b8d9a700 1 --2- 172.21.15.55:0/166629681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5b41566e0 0x7fa5b40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.520+0000 7fa5b959b700 1 --2- 172.21.15.55:0/166629681 >> v2:172.21.15.163:3300/0 conn(0x7fa5b4156310 0x7fa5b40c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.520+0000 7fa5b9d9c700 1 -- 172.21.15.55:0/166629681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5b41566e0 msgr2=0x7fa5b40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:10.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.520+0000 7fa5b9d9c700 1 --2- 172.21.15.55:0/166629681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5b41566e0 0x7fa5b40ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.520+0000 7fa5b9d9c700 1 -- 172.21.15.55:0/166629681 >> v2:172.21.15.163:3300/0 conn(0x7fa5b4156310 msgr2=0x7fa5b40c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:10.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.520+0000 7fa5b9d9c700 1 --2- 172.21.15.55:0/166629681 >> v2:172.21.15.163:3300/0 conn(0x7fa5b4156310 0x7fa5b40c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.520+0000 7fa5b9d9c700 1 -- 172.21.15.55:0/166629681 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b40cb100 con 0x7fa5b40ca300 2023-10-06T03:49:10.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.521+0000 7fa5b9d9c700 1 --2- 172.21.15.55:0/166629681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5b40ca300 0x7fa5b40ae540 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40cce20 tx=0x7fa5ac00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c4b6bf6c634355 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:10.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.521+0000 7fa5abfff700 1 -- 172.21.15.55:0/166629681 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac010070 con 0x7fa5b40ca300 2023-10-06T03:49:10.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.521+0000 7fa5abfff700 1 -- 172.21.15.55:0/166629681 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac00dd30 con 0x7fa5b40ca300 2023-10-06T03:49:10.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.521+0000 7fa5abfff700 1 -- 172.21.15.55:0/166629681 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00dea0 con 0x7fa5b40ca300 2023-10-06T03:49:10.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.521+0000 7fa5ba59d700 1 -- 172.21.15.55:0/166629681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5b40ca300 msgr2=0x7fa5b40ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:10.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.521+0000 7fa5ba59d700 1 --2- 172.21.15.55:0/166629681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5b40ca300 0x7fa5b40ae540 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40cce20 tx=0x7fa5ac00ba50 comp rx=0 tx=0).stop 2023-10-06T03:49:10.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.521+0000 7fa5ba59d700 1 -- 172.21.15.55:0/166629681 shutdown_connections 2023-10-06T03:49:10.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.521+0000 7fa5ba59d700 1 --2- 172.21.15.55:0/166629681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5b40ca300 0x7fa5b40ae540 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.521+0000 7fa5ba59d700 1 --2- 172.21.15.55:0/166629681 >> v2:172.21.15.163:3300/0 conn(0x7fa5b4156310 0x7fa5b40c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.521+0000 7fa5ba59d700 1 --2- 172.21.15.55:0/166629681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5b41566e0 0x7fa5b40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.521+0000 7fa5ba59d700 1 -- 172.21.15.55:0/166629681 >> 172.21.15.55:0/166629681 conn(0x7fa5b40cc1f0 msgr2=0x7fa5b40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:10.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.521+0000 7fa5ba59d700 1 -- 172.21.15.55:0/166629681 shutdown_connections 2023-10-06T03:49:10.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.522+0000 7fa5ba59d700 1 -- 172.21.15.55:0/166629681 wait complete. 2023-10-06T03:49:10.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.523+0000 7fa5ba59d700 1 Processor -- start 2023-10-06T03:49:10.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.523+0000 7fa5ba59d700 1 -- start start 2023-10-06T03:49:10.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.524+0000 7fa5ba59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5b40ca300 0x7fa5b414e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.524+0000 7fa5ba59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5b4156310 0x7fa5b415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.524+0000 7fa5ba59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5b41566e0 0x7fa5b40c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.524+0000 7fa5ba59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b40af2b0 con 0x7fa5b41566e0 2023-10-06T03:49:10.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.524+0000 7fa5ba59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b414ee40 con 0x7fa5b4156310 2023-10-06T03:49:10.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.524+0000 7fa5ba59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b415e8f0 con 0x7fa5b40ca300 2023-10-06T03:49:10.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.524+0000 7fa5b9d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5b41566e0 0x7fa5b40c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.524+0000 7fa5b9d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5b41566e0 0x7fa5b40c24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50122/0 (socket says 172.21.15.55:50122) 2023-10-06T03:49:10.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.524+0000 7fa5b9d9c700 1 -- 172.21.15.55:0/45599672 learned_addr learned my addr 172.21.15.55:0/45599672 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:10.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.524+0000 7fa5b8d9a700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5b4156310 0x7fa5b415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.524+0000 7fa5b959b700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5b40ca300 0x7fa5b414e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.525+0000 7fa5b9d9c700 1 -- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5b40ca300 msgr2=0x7fa5b414e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:10.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.525+0000 7fa5b9d9c700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5b40ca300 0x7fa5b414e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.525+0000 7fa5b9d9c700 1 -- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5b4156310 msgr2=0x7fa5b415bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:10.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.525+0000 7fa5b9d9c700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5b4156310 0x7fa5b415bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.525+0000 7fa5b9d9c700 1 -- 172.21.15.55:0/45599672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac00bcf0 con 0x7fa5b41566e0 2023-10-06T03:49:10.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.525+0000 7fa5b8d9a700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5b4156310 0x7fa5b415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:10.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.525+0000 7fa5b9d9c700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5b41566e0 0x7fa5b40c24f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00d760 tx=0x7fa5ac01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:10.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.525+0000 7fa5a97fa700 1 -- 172.21.15.55:0/45599672 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00ab70 con 0x7fa5b41566e0 2023-10-06T03:49:10.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.525+0000 7fa5a97fa700 1 -- 172.21.15.55:0/45599672 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac00acd0 con 0x7fa5b41566e0 2023-10-06T03:49:10.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.525+0000 7fa5ba59d700 1 -- 172.21.15.55:0/45599672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b415ebd0 con 0x7fa5b41566e0 2023-10-06T03:49:10.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.525+0000 7fa5a97fa700 1 -- 172.21.15.55:0/45599672 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac01d680 con 0x7fa5b41566e0 2023-10-06T03:49:10.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.525+0000 7fa5ba59d700 1 -- 172.21.15.55:0/45599672 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b40af500 con 0x7fa5b41566e0 2023-10-06T03:49:10.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.527+0000 7fa5a97fa700 1 -- 172.21.15.55:0/45599672 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa5ac003040 con 0x7fa5b41566e0 2023-10-06T03:49:10.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.528+0000 7fa5a97fa700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa598066f30 0x7fa5980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.528+0000 7fa5a97fa700 1 -- 172.21.15.55:0/45599672 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fa5ac085e90 con 0x7fa5b41566e0 2023-10-06T03:49:10.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.529+0000 7fa5b959b700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa598066f30 0x7fa5980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.530+0000 7fa5b959b700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa598066f30 0x7fa5980693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009d40 tx=0x7fa5b0008040 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:10.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.530+0000 7fa5c9fd2b80 1 Processor -- start 2023-10-06T03:49:10.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.530+0000 7fa5c9fd2b80 1 -- start start 2023-10-06T03:49:10.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.531+0000 7fa5c9fd2b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5c1ee50 0x55ccd5bf9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.531+0000 7fa5c9fd2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5bff2c0 0x55ccd5bfa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.531+0000 7fa5c9fd2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c00a10 0x55ccd5bfa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.531+0000 7fa5c9fd2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd5b54a70 con 0x55ccd5bff2c0 2023-10-06T03:49:10.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.531+0000 7fa5c9fd2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd5b52270 con 0x55ccd5c00a10 2023-10-06T03:49:10.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.531+0000 7fa5c9fd2b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ccd5b4e060 con 0x55ccd5c1ee50 2023-10-06T03:49:10.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.531+0000 7fa5b8d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5bff2c0 0x55ccd5bfa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.531+0000 7fa5b8d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5bff2c0 0x55ccd5bfa200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50134/0 (socket says 172.21.15.55:50134) 2023-10-06T03:49:10.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.531+0000 7fa5b8d9a700 1 -- 172.21.15.55:0/2847983287 learned_addr learned my addr 172.21.15.55:0/2847983287 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:10.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.531+0000 7fa5b959b700 1 --2- 172.21.15.55:0/2847983287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5c1ee50 0x55ccd5bf9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.531+0000 7fa5b9d9c700 1 --2- 172.21.15.55:0/2847983287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c00a10 0x55ccd5bfa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.532+0000 7fa5b8d9a700 1 -- 172.21.15.55:0/2847983287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5c1ee50 msgr2=0x55ccd5bf9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:10.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.532+0000 7fa5b8d9a700 1 --2- 172.21.15.55:0/2847983287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5c1ee50 0x55ccd5bf9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.532+0000 7fa5b8d9a700 1 -- 172.21.15.55:0/2847983287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c00a10 msgr2=0x55ccd5bfa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:10.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.532+0000 7fa5b8d9a700 1 --2- 172.21.15.55:0/2847983287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c00a10 0x55ccd5bfa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.532+0000 7fa5b8d9a700 1 -- 172.21.15.55:0/2847983287 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccd59e8270 con 0x55ccd5bff2c0 2023-10-06T03:49:10.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.532+0000 7fa5b8d9a700 1 --2- 172.21.15.55:0/2847983287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5bff2c0 0x55ccd5bfa200 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009ce0 tx=0x7fa5a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2f6748679166cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:10.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.532+0000 7fa59e7fc700 1 -- 172.21.15.55:0/2847983287 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4011880 con 0x55ccd5bff2c0 2023-10-06T03:49:10.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.532+0000 7fa59e7fc700 1 -- 172.21.15.55:0/2847983287 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a40119e0 con 0x55ccd5bff2c0 2023-10-06T03:49:10.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.532+0000 7fa59e7fc700 1 -- 172.21.15.55:0/2847983287 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4011da0 con 0x55ccd5bff2c0 2023-10-06T03:49:10.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.533+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/2847983287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5bff2c0 msgr2=0x55ccd5bfa200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:10.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.533+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/2847983287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5bff2c0 0x55ccd5bfa200 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009ce0 tx=0x7fa5a4008040 comp rx=0 tx=0).stop 2023-10-06T03:49:10.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.533+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/2847983287 shutdown_connections 2023-10-06T03:49:10.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.533+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/2847983287 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c00a10 0x55ccd5bfa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.533+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/2847983287 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5c1ee50 0x55ccd5bf9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.533+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/2847983287 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5bff2c0 0x55ccd5bfa200 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.533+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/2847983287 >> 172.21.15.55:0/2847983287 conn(0x55ccd5bfee80 msgr2=0x55ccd5bff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:10.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.533+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/2847983287 shutdown_connections 2023-10-06T03:49:10.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.533+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/2847983287 wait complete. 2023-10-06T03:49:10.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.534+0000 7fa5c9fd2b80 1 -- start start 2023-10-06T03:49:10.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.534+0000 7fa5c9fd2b80 1 Processor -- start 2023-10-06T03:49:10.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.535+0000 7fa5c9fd2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5bff2c0 0x55ccd5c1f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.535+0000 7fa5c9fd2b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5c00a10 0x55ccd5c2a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.535+0000 7fa5c9fd2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c1ee50 0x55ccd5c43b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.535+0000 7fa5c9fd2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd5af9890 con 0x55ccd5bff2c0 2023-10-06T03:49:10.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.535+0000 7fa5c9fd2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd5b14ea0 con 0x55ccd5c1ee50 2023-10-06T03:49:10.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.535+0000 7fa5c9fd2b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ccd59d0e30 con 0x55ccd5c00a10 2023-10-06T03:49:10.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.535+0000 7fa5c9fd2b80 10 client.0 _ll_register_callbacks cb 0x55ccd5bc1020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:49:10.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.535+0000 7fa5b959b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5bff2c0 0x55ccd5c1f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.535+0000 7fa5b959b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5bff2c0 0x55ccd5c1f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50146/0 (socket says 172.21.15.55:50146) 2023-10-06T03:49:10.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.535+0000 7fa5b959b700 1 -- 172.21.15.55:0/1960413812 learned_addr learned my addr 172.21.15.55:0/1960413812 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:10.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.535+0000 7fa5b9d9c700 1 --2- 172.21.15.55:0/1960413812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c1ee50 0x55ccd5c43b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.535+0000 7fa5b8d9a700 1 --2- 172.21.15.55:0/1960413812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5c00a10 0x55ccd5c2a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.536+0000 7fa5b959b700 1 -- 172.21.15.55:0/1960413812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5c00a10 msgr2=0x55ccd5c2a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:10.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.536+0000 7fa5b959b700 1 --2- 172.21.15.55:0/1960413812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5c00a10 0x55ccd5c2a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.536+0000 7fa5b959b700 1 -- 172.21.15.55:0/1960413812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c1ee50 msgr2=0x55ccd5c43b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:10.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.536+0000 7fa5b959b700 1 --2- 172.21.15.55:0/1960413812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c1ee50 0x55ccd5c43b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.536+0000 7fa5b959b700 1 -- 172.21.15.55:0/1960413812 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a4009910 con 0x55ccd5bff2c0 2023-10-06T03:49:10.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.536+0000 7fa5b959b700 1 --2- 172.21.15.55:0/1960413812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5bff2c0 0x55ccd5c1f340 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0015970 tx=0x7fa5b0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24182acecd0befc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:10.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.536+0000 7fa5937fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:49:10.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.536+0000 7fa5937fe700 1 -- 172.21.15.55:0/1960413812 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0014560 con 0x55ccd5bff2c0 2023-10-06T03:49:10.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.536+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1960413812 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ccd5bf2940 con 0x55ccd5bff2c0 2023-10-06T03:49:10.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.536+0000 7fa5937fe700 1 -- 172.21.15.55:0/1960413812 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fa5b00146c0 con 0x55ccd5bff2c0 2023-10-06T03:49:10.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.536+0000 7fa5937fe700 1 -- 172.21.15.55:0/1960413812 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5b0014a90 con 0x55ccd5bff2c0 2023-10-06T03:49:10.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.536+0000 7fa5c9fd2b80 10 client.6939 fetch_fsmap learned FSMap version 44 2023-10-06T03:49:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.537+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1960413812 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55ccd598afe0 con 0x55ccd5bff2c0 2023-10-06T03:49:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.537+0000 7fa5937fe700 1 -- 172.21.15.55:0/1960413812 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ccd5bf2940 con 0x55ccd5bff2c0 2023-10-06T03:49:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.537+0000 7fa5937fe700 1 -- 172.21.15.55:0/1960413812 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 44) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa5b0028020 con 0x55ccd5bff2c0 2023-10-06T03:49:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.537+0000 7fa5c9fd2b80 10 client.6939 fetch_fsmap finished waiting for FSMap version 44 2023-10-06T03:49:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.537+0000 7fa5c9fd2b80 10 client.6939 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T03:49:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.537+0000 7fa5c9fd2b80 20 client.6939 populate_metadata read hostname 'smithi055' 2023-10-06T03:49:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.537+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/1960413812 >> [v2:172.21.15.55:6833/3960749421,v1:172.21.15.55:6835/3960749421] conn(0x55ccd5c2c810 0x55ccd5c2cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.537+0000 7fa5c9fd2b80 4 client.6939 mds_command: new command op to 6804 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:49:10.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.537+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1960413812 --> [v2:172.21.15.55:6833/3960749421,v1:172.21.15.55:6835/3960749421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccd5bcf870 con 0x55ccd5c2c810 2023-10-06T03:49:10.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.537+0000 7fa5b8d9a700 1 --2- 172.21.15.55:0/1960413812 >> [v2:172.21.15.55:6833/3960749421,v1:172.21.15.55:6835/3960749421] conn(0x55ccd5c2c810 0x55ccd5c2cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.538+0000 7fa5b8d9a700 1 --2- 172.21.15.55:0/1960413812 >> [v2:172.21.15.55:6833/3960749421,v1:172.21.15.55:6835/3960749421] conn(0x55ccd5c2c810 0x55ccd5c2cbe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3907dcb582376ac3 server_cookie=3c5dc8cef0deba0e in_seq=0 out_seq=0 2023-10-06T03:49:10.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.538+0000 7fa5937fe700 10 client.6939 ms_handle_connect on v2:172.21.15.55:6833/3960749421 2023-10-06T03:49:10.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.538+0000 7fa5937fe700 1 -- 172.21.15.55:0/1960413812 <== mds.0 v2:172.21.15.55:6833/3960749421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55ccd5bcf870 con 0x55ccd5c2c810 2023-10-06T03:49:10.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.538+0000 7fa5937fe700 10 client.6939 handle_command_reply: tid=0 2023-10-06T03:49:10.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.538+0000 7fa5c9fd2b80 1 client.6939 shutdown 2023-10-06T03:49:10.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.539+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1960413812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5bff2c0 msgr2=0x55ccd5c1f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:10.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.539+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/1960413812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5bff2c0 0x55ccd5c1f340 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0015970 tx=0x7fa5b0018000 comp rx=0 tx=0).stop 2023-10-06T03:49:10.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.539+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1960413812 shutdown_connections 2023-10-06T03:49:10.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.539+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/1960413812 >> [v2:172.21.15.55:6833/3960749421,v1:172.21.15.55:6835/3960749421] conn(0x55ccd5c2c810 0x55ccd5c2cbe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.539+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/1960413812 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c1ee50 0x55ccd5c43b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.539+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/1960413812 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5bff2c0 0x55ccd5c1f340 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.539+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/1960413812 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5c00a10 0x55ccd5c2a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.539+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1960413812 >> 172.21.15.55:0/1960413812 conn(0x55ccd5bfee80 msgr2=0x55ccd5bfdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:10.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.539+0000 7fa5937fe700 0 client.6939 ms_handle_reset on v2:172.21.15.55:6833/3960749421 2023-10-06T03:49:10.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.539+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1960413812 shutdown_connections 2023-10-06T03:49:10.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.539+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1960413812 wait complete. 2023-10-06T03:49:10.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.539+0000 7fa5c9fd2b80 20 client.6939 trim_cache size 0 max 16384 2023-10-06T03:49:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.557+0000 7fa5c9fd2b80 1 Processor -- start 2023-10-06T03:49:10.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.557+0000 7fa5c9fd2b80 1 -- start start 2023-10-06T03:49:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.557+0000 7fa5c9fd2b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5bff2c0 0x55ccd5bfde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.557+0000 7fa5c9fd2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c00a10 0x55ccd5c1d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.557+0000 7fa5c9fd2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5c1ee50 0x55ccd5c1d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.557+0000 7fa5c9fd2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd5bf44a0 con 0x55ccd5c1ee50 2023-10-06T03:49:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.557+0000 7fa5c9fd2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd5c43bd0 con 0x55ccd5c00a10 2023-10-06T03:49:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.557+0000 7fa5c9fd2b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ccd5c1fb70 con 0x55ccd5bff2c0 2023-10-06T03:49:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.558+0000 7fa5b9d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5c1ee50 0x55ccd5c1d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.558+0000 7fa5b9d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5c1ee50 0x55ccd5c1d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50160/0 (socket says 172.21.15.55:50160) 2023-10-06T03:49:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.558+0000 7fa5b9d9c700 1 -- 172.21.15.55:0/3308504912 learned_addr learned my addr 172.21.15.55:0/3308504912 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:10.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.558+0000 7fa5b959b700 1 --2- 172.21.15.55:0/3308504912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5bff2c0 0x55ccd5bfde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.558+0000 7fa5b9d9c700 1 -- 172.21.15.55:0/3308504912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5bff2c0 msgr2=0x55ccd5bfde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.558+0000 7fa5b8d9a700 1 --2- 172.21.15.55:0/3308504912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c00a10 0x55ccd5c1d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.558+0000 7fa5b9d9c700 1 --2- 172.21.15.55:0/3308504912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5bff2c0 0x55ccd5bfde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.558+0000 7fa5b9d9c700 1 -- 172.21.15.55:0/3308504912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c00a10 msgr2=0x55ccd5c1d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.558+0000 7fa5b9d9c700 1 --2- 172.21.15.55:0/3308504912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c00a10 0x55ccd5c1d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.558+0000 7fa5b9d9c700 1 -- 172.21.15.55:0/3308504912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccd5b1c780 con 0x55ccd5c1ee50 2023-10-06T03:49:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.558+0000 7fa5b959b700 1 --2- 172.21.15.55:0/3308504912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5bff2c0 0x55ccd5bfde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.558+0000 7fa5b8d9a700 1 --2- 172.21.15.55:0/3308504912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c00a10 0x55ccd5c1d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.558+0000 7fa5b9d9c700 1 --2- 172.21.15.55:0/3308504912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5c1ee50 0x55ccd5c1d590 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0840b0 tx=0x7fa5ac00dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5044244212bebf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.559+0000 7fa59effd700 1 -- 172.21.15.55:0/3308504912 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac010040 con 0x55ccd5c1ee50 2023-10-06T03:49:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.559+0000 7fa59effd700 1 -- 172.21.15.55:0/3308504912 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac05d9d0 con 0x55ccd5c1ee50 2023-10-06T03:49:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.559+0000 7fa59effd700 1 -- 172.21.15.55:0/3308504912 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00ae40 con 0x55ccd5c1ee50 2023-10-06T03:49:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.559+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/3308504912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5c1ee50 msgr2=0x55ccd5c1d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.559+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/3308504912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5c1ee50 0x55ccd5c1d590 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0840b0 tx=0x7fa5ac00dfd0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.559+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/3308504912 shutdown_connections 2023-10-06T03:49:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.559+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/3308504912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5c1ee50 0x55ccd5c1d590 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.559+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/3308504912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5bff2c0 0x55ccd5bfde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.559+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/3308504912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c00a10 0x55ccd5c1d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.559+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/3308504912 >> 172.21.15.55:0/3308504912 conn(0x55ccd5bfee80 msgr2=0x55ccd5c51320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.559+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/3308504912 shutdown_connections 2023-10-06T03:49:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.559+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/3308504912 wait complete. 2023-10-06T03:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.560+0000 7fa5c9fd2b80 1 -- start start 2023-10-06T03:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.560+0000 7fa5c9fd2b80 1 Processor -- start 2023-10-06T03:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.561+0000 7fa5c9fd2b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5bff2c0 0x55ccd5bff690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.561+0000 7fa5c9fd2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5c00a10 0x55ccd5bffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.561+0000 7fa5c9fd2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c1ee50 0x55ccd5bf9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.561+0000 7fa5c9fd2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd5caa8a0 con 0x55ccd5c00a10 2023-10-06T03:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.561+0000 7fa5c9fd2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccd5bf4fa0 con 0x55ccd5c1ee50 2023-10-06T03:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.561+0000 7fa5c9fd2b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ccd5ca9990 con 0x55ccd5bff2c0 2023-10-06T03:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.561+0000 7fa5c9fd2b80 10 client.0 _ll_register_callbacks cb 0x55ccd5bc1020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.561+0000 7fa5b8d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5c00a10 0x55ccd5bffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.561+0000 7fa5b8d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5c00a10 0x55ccd5bffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50162/0 (socket says 172.21.15.55:50162) 2023-10-06T03:49:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.561+0000 7fa5b8d9a700 1 -- 172.21.15.55:0/1511724359 learned_addr learned my addr 172.21.15.55:0/1511724359 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.562+0000 7fa5b9d9c700 1 --2- 172.21.15.55:0/1511724359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c1ee50 0x55ccd5bf9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.562+0000 7fa5b959b700 1 --2- 172.21.15.55:0/1511724359 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5bff2c0 0x55ccd5bff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.562+0000 7fa5b8d9a700 1 -- 172.21.15.55:0/1511724359 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5bff2c0 msgr2=0x55ccd5bff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.562+0000 7fa5b8d9a700 1 --2- 172.21.15.55:0/1511724359 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5bff2c0 0x55ccd5bff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.562+0000 7fa5b8d9a700 1 -- 172.21.15.55:0/1511724359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c1ee50 msgr2=0x55ccd5bf9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.562+0000 7fa5b8d9a700 1 --2- 172.21.15.55:0/1511724359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c1ee50 0x55ccd5bf9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.562+0000 7fa5b8d9a700 1 -- 172.21.15.55:0/1511724359 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b40af500 con 0x55ccd5c00a10 2023-10-06T03:49:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.562+0000 7fa5b8d9a700 1 --2- 172.21.15.55:0/1511724359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5c00a10 0x55ccd5bffbd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x55ccd5c2bae0 tx=0x7fa5a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedb9dad6d2f23cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.562+0000 7fa5937fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:49:10.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.562+0000 7fa5937fe700 1 -- 172.21.15.55:0/1511724359 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4023040 con 0x55ccd5c00a10 2023-10-06T03:49:10.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.562+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1511724359 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5b4151490 con 0x55ccd5c00a10 2023-10-06T03:49:10.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.562+0000 7fa5937fe700 1 -- 172.21.15.55:0/1511724359 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fa5a4015900 con 0x55ccd5c00a10 2023-10-06T03:49:10.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.563+0000 7fa5937fe700 1 -- 172.21.15.55:0/1511724359 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5a4015cd0 con 0x55ccd5c00a10 2023-10-06T03:49:10.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.563+0000 7fa5c9fd2b80 10 client.6945 fetch_fsmap learned FSMap version 44 2023-10-06T03:49:10.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.563+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1511724359 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55ccd5b03470 con 0x55ccd5c00a10 2023-10-06T03:49:10.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.563+0000 7fa5937fe700 1 -- 172.21.15.55:0/1511724359 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b4151490 con 0x55ccd5c00a10 2023-10-06T03:49:10.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.563+0000 7fa5937fe700 1 -- 172.21.15.55:0/1511724359 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 44) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fa5a402b4a0 con 0x55ccd5c00a10 2023-10-06T03:49:10.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.563+0000 7fa5c9fd2b80 10 client.6945 fetch_fsmap finished waiting for FSMap version 44 2023-10-06T03:49:10.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.563+0000 7fa5c9fd2b80 10 client.6945 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T03:49:10.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.563+0000 7fa5c9fd2b80 20 client.6945 populate_metadata read hostname 'smithi055' 2023-10-06T03:49:10.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.563+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/1511724359 >> [v2:172.21.15.55:6833/3960749421,v1:172.21.15.55:6835/3960749421] conn(0x55ccd5bfa080 0x55ccd5bfa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.563+0000 7fa5c9fd2b80 4 client.6945 mds_command: new command op to 6804 tid=0[{"prefix": "status"}] 2023-10-06T03:49:10.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.563+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1511724359 --> [v2:172.21.15.55:6833/3960749421,v1:172.21.15.55:6835/3960749421] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fa5a40119e0 con 0x55ccd5bfa080 2023-10-06T03:49:10.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.563+0000 7fa5b959b700 1 --2- 172.21.15.55:0/1511724359 >> [v2:172.21.15.55:6833/3960749421,v1:172.21.15.55:6835/3960749421] conn(0x55ccd5bfa080 0x55ccd5bfa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.564+0000 7fa5937fe700 10 client.6945 ms_handle_connect on v2:172.21.15.55:6833/3960749421 2023-10-06T03:49:10.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.564+0000 7fa5b959b700 1 --2- 172.21.15.55:0/1511724359 >> [v2:172.21.15.55:6833/3960749421,v1:172.21.15.55:6835/3960749421] conn(0x55ccd5bfa080 0x55ccd5bfa450 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5eadc3c449966c8 server_cookie=225eec08823c900 in_seq=0 out_seq=0 2023-10-06T03:49:10.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.564+0000 7fa5937fe700 1 -- 172.21.15.55:0/1511724359 <== mds.0 v2:172.21.15.55:6833/3960749421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x7fa5a40119e0 con 0x55ccd5bfa080 2023-10-06T03:49:10.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.564+0000 7fa5937fe700 10 client.6945 handle_command_reply: tid=0 2023-10-06T03:49:10.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.564+0000 7fa5c9fd2b80 1 client.6945 shutdown 2023-10-06T03:49:10.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.564+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1511724359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5c00a10 msgr2=0x55ccd5bffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.564+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/1511724359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5c00a10 0x55ccd5bffbd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x55ccd5c2bae0 tx=0x7fa5a4008040 comp rx=0 tx=0).stop 2023-10-06T03:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.564+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1511724359 shutdown_connections 2023-10-06T03:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.564+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/1511724359 >> [v2:172.21.15.55:6833/3960749421,v1:172.21.15.55:6835/3960749421] conn(0x55ccd5bfa080 0x55ccd5bfa450 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.564+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/1511724359 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55ccd5c1ee50 0x55ccd5bf9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.564+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/1511724359 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55ccd5bff2c0 0x55ccd5bff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.564+0000 7fa5c9fd2b80 1 --2- 172.21.15.55:0/1511724359 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55ccd5c00a10 0x55ccd5bffbd0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.564+0000 7fa5937fe700 0 client.6945 ms_handle_reset on v2:172.21.15.55:6833/3960749421 2023-10-06T03:49:10.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.564+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1511724359 >> 172.21.15.55:0/1511724359 conn(0x55ccd5bfee80 msgr2=0x55ccd5c2cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:10.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.565+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1511724359 shutdown_connections 2023-10-06T03:49:10.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.565+0000 7fa5c9fd2b80 1 -- 172.21.15.55:0/1511724359 wait complete. 2023-10-06T03:49:10.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.565+0000 7fa5c9fd2b80 20 client.6945 trim_cache size 0 max 16384 2023-10-06T03:49:10.575 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:49:10.575 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "8132c8ba-ee1c-41a2-b781-913925034ab6", 2023-10-06T03:49:10.575 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2023-10-06T03:49:10.575 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 6804, 2023-10-06T03:49:10.575 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2023-10-06T03:49:10.576 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2023-10-06T03:49:10.576 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2023-10-06T03:49:10.576 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 6.5026183189999998, 2023-10-06T03:49:10.576 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 44, 2023-10-06T03:49:10.576 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 93, 2023-10-06T03:49:10.576 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 93, 2023-10-06T03:49:10.576 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 12.247654928999999 2023-10-06T03:49:10.577 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:49:10.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.565+0000 7fa5ba59d700 1 -- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa598066f30 msgr2=0x7fa5980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:10.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.565+0000 7fa5ba59d700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa598066f30 0x7fa5980693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009d40 tx=0x7fa5b0008040 comp rx=0 tx=0).stop 2023-10-06T03:49:10.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.565+0000 7fa5ba59d700 1 -- 172.21.15.55:0/45599672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5b41566e0 msgr2=0x7fa5b40c24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:10.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.565+0000 7fa5ba59d700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5b41566e0 0x7fa5b40c24f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00d760 tx=0x7fa5ac01a000 comp rx=0 tx=0).stop 2023-10-06T03:49:10.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.565+0000 7fa5ba59d700 1 -- 172.21.15.55:0/45599672 shutdown_connections 2023-10-06T03:49:10.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.565+0000 7fa5ba59d700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa598066f30 0x7fa5980693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.565+0000 7fa5ba59d700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5b41566e0 0x7fa5b40c24f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.565+0000 7fa5ba59d700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5b40ca300 0x7fa5b414e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.565+0000 7fa5ba59d700 1 --2- 172.21.15.55:0/45599672 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5b4156310 0x7fa5b415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.565+0000 7fa5ba59d700 1 -- 172.21.15.55:0/45599672 >> 172.21.15.55:0/45599672 conn(0x7fa5b40cc1f0 msgr2=0x7fa5b406ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:10.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.566+0000 7fa5ba59d700 1 -- 172.21.15.55:0/45599672 shutdown_connections 2023-10-06T03:49:10.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.566+0000 7fa5ba59d700 1 -- 172.21.15.55:0/45599672 wait complete. 2023-10-06T03:49:10.584 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:49:10.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.770+0000 7f4ce339d700 1 Processor -- start 2023-10-06T03:49:10.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.770+0000 7f4ce339d700 1 -- start start 2023-10-06T03:49:10.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.771+0000 7f4ce339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cdc156310 0x7f4cdc0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.771+0000 7f4ce339d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4cdc1566e0 0x7f4cdc0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.771+0000 7f4ce339d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cdc0ca860 con 0x7f4cdc0ca300 2023-10-06T03:49:10.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.771+0000 7f4ce339d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cdc0cab40 con 0x7f4cdc1566e0 2023-10-06T03:49:10.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.771+0000 7f4ce339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc0cae20 con 0x7f4cdc156310 2023-10-06T03:49:10.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.772+0000 7f4ce1b9a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4cdc1566e0 0x7f4cdc0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.772+0000 7f4ce239b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cdc156310 0x7f4cdc0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.772+0000 7f4ce239b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cdc156310 0x7f4cdc0c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:33226/0 (socket says 172.21.15.55:33226) 2023-10-06T03:49:10.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.772+0000 7f4ce1b9a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4cdc1566e0 0x7f4cdc0ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:32852/0 (socket says 172.21.15.55:32852) 2023-10-06T03:49:10.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.772+0000 7f4ce239b700 1 -- 172.21.15.55:0/2304531104 learned_addr learned my addr 172.21.15.55:0/2304531104 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:10.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.772+0000 7f4ce239b700 1 -- 172.21.15.55:0/2304531104 >> v2:172.21.15.163:3300/0 conn(0x7f4cdc1566e0 msgr2=0x7f4cdc0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:10.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.772+0000 7f4ce239b700 1 --2- 172.21.15.55:0/2304531104 >> v2:172.21.15.163:3300/0 conn(0x7f4cdc1566e0 0x7f4cdc0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.772+0000 7f4ce239b700 1 -- 172.21.15.55:0/2304531104 >> v1:172.21.15.163:6789/0 conn(0x7f4cdc0ca300 legacy=0x7f4cdc0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:10.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.772+0000 7f4ce239b700 1 -- 172.21.15.55:0/2304531104 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cdc0cb100 con 0x7f4cdc156310 2023-10-06T03:49:10.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.772+0000 7f4ce239b700 1 --2- 172.21.15.55:0/2304531104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cdc156310 0x7f4cdc0c9dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80099b0 tx=0x7f4cd8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4d332a0b7f77a27 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:10.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.773+0000 7f4ce1399700 1 -- 172.21.15.55:0/2304531104 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8009ea0 con 0x7f4cdc156310 2023-10-06T03:49:10.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.773+0000 7f4ce1399700 1 -- 172.21.15.55:0/2304531104 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd800b860 con 0x7f4cdc156310 2023-10-06T03:49:10.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.773+0000 7f4ce1399700 1 -- 172.21.15.55:0/2304531104 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd800bc00 con 0x7f4cdc156310 2023-10-06T03:49:10.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.773+0000 7f4ce339d700 1 -- 172.21.15.55:0/2304531104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cdc156310 msgr2=0x7f4cdc0c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:10.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.773+0000 7f4ce339d700 1 --2- 172.21.15.55:0/2304531104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cdc156310 0x7f4cdc0c9dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80099b0 tx=0x7f4cd8009250 comp rx=0 tx=0).stop 2023-10-06T03:49:10.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.773+0000 7f4ce339d700 1 -- 172.21.15.55:0/2304531104 shutdown_connections 2023-10-06T03:49:10.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.773+0000 7f4ce339d700 1 --2- 172.21.15.55:0/2304531104 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cdc156310 0x7f4cdc0c9dc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.773+0000 7f4ce339d700 1 --2- 172.21.15.55:0/2304531104 >> v2:172.21.15.163:3300/0 conn(0x7f4cdc1566e0 0x7f4cdc0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.773+0000 7f4ce339d700 1 -- 172.21.15.55:0/2304531104 >> 172.21.15.55:0/2304531104 conn(0x7f4cdc0cc1f0 msgr2=0x7f4cdc0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:10.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.773+0000 7f4ce339d700 1 -- 172.21.15.55:0/2304531104 shutdown_connections 2023-10-06T03:49:10.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.773+0000 7f4ce339d700 1 -- 172.21.15.55:0/2304531104 wait complete. 2023-10-06T03:49:10.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.775+0000 7f4ce339d700 1 Processor -- start 2023-10-06T03:49:10.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.775+0000 7f4ce339d700 1 -- start start 2023-10-06T03:49:10.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.776+0000 7f4ce339d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cdc0ca300 0x7f4cdc16a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.776+0000 7f4ce339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cdc156310 0x7f4cdc16e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.776+0000 7f4ce339d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cdc1566e0 0x7f4cdc14fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.776+0000 7f4ce339d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc15d030 con 0x7f4cdc1566e0 2023-10-06T03:49:10.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.776+0000 7f4ce339d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc16a830 con 0x7f4cdc0ca300 2023-10-06T03:49:10.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.776+0000 7f4ce339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc16ab10 con 0x7f4cdc156310 2023-10-06T03:49:10.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.776+0000 7f4ce2b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cdc1566e0 0x7f4cdc14fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.776+0000 7f4ce239b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cdc0ca300 0x7f4cdc16a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.776+0000 7f4ce1b9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cdc156310 0x7f4cdc16e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.776+0000 7f4ce2b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cdc1566e0 0x7f4cdc14fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50166/0 (socket says 172.21.15.55:50166) 2023-10-06T03:49:10.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.776+0000 7f4ce2b9c700 1 -- 172.21.15.55:0/2795063895 learned_addr learned my addr 172.21.15.55:0/2795063895 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:10.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.777+0000 7f4ce2b9c700 1 -- 172.21.15.55:0/2795063895 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cdc156310 msgr2=0x7f4cdc16e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:10.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.777+0000 7f4ce2b9c700 1 --2- 172.21.15.55:0/2795063895 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cdc156310 0x7f4cdc16e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.777+0000 7f4ce2b9c700 1 -- 172.21.15.55:0/2795063895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cdc0ca300 msgr2=0x7f4cdc16a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:10.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.777+0000 7f4ce2b9c700 1 --2- 172.21.15.55:0/2795063895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cdc0ca300 0x7f4cdc16a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:10.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.777+0000 7f4ce2b9c700 1 -- 172.21.15.55:0/2795063895 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd80094f0 con 0x7f4cdc1566e0 2023-10-06T03:49:10.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.777+0000 7f4ce2b9c700 1 --2- 172.21.15.55:0/2795063895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cdc1566e0 0x7f4cdc14fab0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400c840 tx=0x7f4cd4001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:10.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.777+0000 7f4cd27fc700 1 -- 172.21.15.55:0/2795063895 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4012630 con 0x7f4cdc1566e0 2023-10-06T03:49:10.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.777+0000 7f4cd27fc700 1 -- 172.21.15.55:0/2795063895 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd40127b0 con 0x7f4cdc1566e0 2023-10-06T03:49:10.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.777+0000 7f4ce339d700 1 -- 172.21.15.55:0/2795063895 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cdc16adf0 con 0x7f4cdc1566e0 2023-10-06T03:49:10.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.777+0000 7f4cd27fc700 1 -- 172.21.15.55:0/2795063895 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4012b70 con 0x7f4cdc1566e0 2023-10-06T03:49:10.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.777+0000 7f4ce339d700 1 -- 172.21.15.55:0/2795063895 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cdc16e740 con 0x7f4cdc1566e0 2023-10-06T03:49:10.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.778+0000 7f4ce339d700 1 -- 172.21.15.55:0/2795063895 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cdc15d1a0 con 0x7f4cdc1566e0 2023-10-06T03:49:10.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.779+0000 7f4cd27fc700 1 -- 172.21.15.55:0/2795063895 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4cd401a3a0 con 0x7f4cdc1566e0 2023-10-06T03:49:10.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.780+0000 7f4cd27fc700 1 --2- 172.21.15.55:0/2795063895 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4cc0066f30 0x7f4cc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:10.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.780+0000 7f4cd27fc700 1 -- 172.21.15.55:0/2795063895 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f4cd40849f0 con 0x7f4cdc1566e0 2023-10-06T03:49:10.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.781+0000 7f4ce239b700 1 --2- 172.21.15.55:0/2795063895 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4cc0066f30 0x7f4cc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:10.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.782+0000 7f4ce239b700 1 --2- 172.21.15.55:0/2795063895 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4cc0066f30 0x7f4cc00693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80099b0 tx=0x7f4cd8017000 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:10.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:10.785+0000 7f4cd27fc700 1 -- 172.21.15.55:0/2795063895 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4cdc15d1a0 con 0x7f4cdc1566e0 2023-10-06T03:49:11.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:11.035+0000 7f4ce339d700 1 -- 172.21.15.55:0/2795063895 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4cdc15d170 con 0x7f4cdc1566e0 2023-10-06T03:49:11.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:11.035+0000 7f4cd27fc700 1 -- 172.21.15.55:0/2795063895 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4cd40518d0 con 0x7f4cdc1566e0 2023-10-06T03:49:11.037 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:49:11.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:11.038+0000 7f4ce339d700 1 -- 172.21.15.55:0/2795063895 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4cc0066f30 msgr2=0x7f4cc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:11.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:11.038+0000 7f4ce339d700 1 --2- 172.21.15.55:0/2795063895 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4cc0066f30 0x7f4cc00693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80099b0 tx=0x7f4cd8017000 comp rx=0 tx=0).stop 2023-10-06T03:49:11.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:11.038+0000 7f4ce339d700 1 -- 172.21.15.55:0/2795063895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cdc1566e0 msgr2=0x7f4cdc14fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:11.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:11.038+0000 7f4ce339d700 1 --2- 172.21.15.55:0/2795063895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cdc1566e0 0x7f4cdc14fab0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400c840 tx=0x7f4cd4001670 comp rx=0 tx=0).stop 2023-10-06T03:49:11.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:11.038+0000 7f4ce339d700 1 -- 172.21.15.55:0/2795063895 shutdown_connections 2023-10-06T03:49:11.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:11.038+0000 7f4ce339d700 1 --2- 172.21.15.55:0/2795063895 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4cc0066f30 0x7f4cc00693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:11.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:11.038+0000 7f4ce339d700 1 --2- 172.21.15.55:0/2795063895 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4cdc1566e0 0x7f4cdc14fab0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:11.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:11.038+0000 7f4ce339d700 1 --2- 172.21.15.55:0/2795063895 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4cdc0ca300 0x7f4cdc16a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:11.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:11.038+0000 7f4ce339d700 1 --2- 172.21.15.55:0/2795063895 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4cdc156310 0x7f4cdc16e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:11.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:11.038+0000 7f4ce339d700 1 -- 172.21.15.55:0/2795063895 >> 172.21.15.55:0/2795063895 conn(0x7f4cdc0cc1f0 msgr2=0x7f4cdc06a780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:11.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:11.038+0000 7f4ce339d700 1 -- 172.21.15.55:0/2795063895 shutdown_connections 2023-10-06T03:49:11.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:11.038+0000 7f4ce339d700 1 -- 172.21.15.55:0/2795063895 wait complete. 2023-10-06T03:49:11.057 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:49:11.057 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:49:11.057 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2023-10-06T03:49:11.057 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:49:11.058 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:49:11.058 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:49:11.058 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:49:11.058 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:49:11.058 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:49:11.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:11.058 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2023-10-06T03:49:11.096 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:49:11.097 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:49:11.097 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:49:11.097 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:49:11.097 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T03:49:11.097 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:49:11.097 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:49:11.097 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:49:11.097 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2023-10-06T03:49:11.097 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:49:11.098 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:49:11.098 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2023-10-06T03:49:11.098 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:49:11.098 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:49:11.098 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2023-10-06T03:49:11.098 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T03:49:11.098 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 41290sec preferred_lft 41290sec 2023-10-06T03:49:11.098 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link 2023-10-06T03:49:11.098 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:49:11.099 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:49:11.099 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:49:11.099 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2023-10-06T03:49:11.099 INFO:teuthology.orchestra.run.smithi055.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:49:11.099 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff 2023-10-06T03:49:11.099 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:49:11.099 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:49:11.099 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::1099:8eff:fe6d:102/64 scope link 2023-10-06T03:49:11.099 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:49:11.100 INFO:teuthology.orchestra.run.smithi055.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:49:11.100 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether f2:5d:53:d2:00:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:49:11.100 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f05d:53ff:fed2:a/64 scope link 2023-10-06T03:49:11.100 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:49:11.100 INFO:teuthology.orchestra.run.smithi055.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:49:11.100 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:49:11.100 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::e037:33ff:feb7:6c8f/64 scope link 2023-10-06T03:49:11.100 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:49:11.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:11.101 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:49:11.198 INFO:teuthology.orchestra.run.smithi055.stdout:1 2023-10-06T03:49:11.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:11.200 DEBUG:teuthology.orchestra.run.smithi055:> route 2023-10-06T03:49:11.239 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2023-10-06T03:49:11.239 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:49:11.240 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T03:49:11.240 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T03:49:11.240 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:49:11.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:11.240 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:49:11.240 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:49:11.240 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T03:49:11.240 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T03:49:11.240 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T03:49:11.241 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:49:11.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:11.438 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T03:49:11.475 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:49:11.475 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:49:11.475 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:49:11.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:11.476 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2023-10-06T03:49:11.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:49:11.578 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:49:11.615 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:49:11.615 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:49:11.615 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:49:11.691 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T03:49:11.691 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:49:11.692 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:49:11.823 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:49:11.824 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:49:11.824 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:49:11.840 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:49:11.840 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:49:11.840 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:49:11.840 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:49:11.840 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:49:11.861 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:49:11.861 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:49:11.861 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:49:11.862 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:49:11.862 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:49:11.862 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T03:49:11.862 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T03:49:11.862 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T03:49:11.868 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:49:11.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:11.869 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:49:11.931 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:49:12.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.122+0000 7fcecdf9d700 1 Processor -- start 2023-10-06T03:49:12.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.122+0000 7fcecdf9d700 1 -- start start 2023-10-06T03:49:12.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.123+0000 7fcecdf9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcec8156310 0x7fcec80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:12.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.123+0000 7fcecdf9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcec81566e0 0x7fcec80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:12.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.123+0000 7fcecdf9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec80ca860 con 0x7fcec80ca300 2023-10-06T03:49:12.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.123+0000 7fcecdf9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec80cab40 con 0x7fcec8156310 2023-10-06T03:49:12.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.123+0000 7fcecdf9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80cae20 con 0x7fcec81566e0 2023-10-06T03:49:12.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.124+0000 7fceccf9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcec8156310 0x7fcec80c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:12.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.124+0000 7fcec7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcec81566e0 0x7fcec80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:12.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.124+0000 7fceccf9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcec8156310 0x7fcec80c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:32866/0 (socket says 172.21.15.55:32866) 2023-10-06T03:49:12.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.124+0000 7fceccf9b700 1 -- 172.21.15.55:0/3851289883 learned_addr learned my addr 172.21.15.55:0/3851289883 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:12.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.124+0000 7fcec7fff700 1 -- 172.21.15.55:0/3851289883 >> v2:172.21.15.163:3300/0 conn(0x7fcec8156310 msgr2=0x7fcec80c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:12.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.124+0000 7fcec7fff700 1 --2- 172.21.15.55:0/3851289883 >> v2:172.21.15.163:3300/0 conn(0x7fcec8156310 0x7fcec80c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:12.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.124+0000 7fcec7fff700 1 -- 172.21.15.55:0/3851289883 >> v1:172.21.15.163:6789/0 conn(0x7fcec80ca300 legacy=0x7fcec80c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:12.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.124+0000 7fcec7fff700 1 -- 172.21.15.55:0/3851289883 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec80cb100 con 0x7fcec81566e0 2023-10-06T03:49:12.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.124+0000 7fceccf9b700 1 --2- 172.21.15.55:0/3851289883 >> v2:172.21.15.163:3300/0 conn(0x7fcec8156310 0x7fcec80c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:12.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.125+0000 7fcec7fff700 1 --2- 172.21.15.55:0/3851289883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcec81566e0 0x7fcec80ae000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fceb400bb90 tx=0x7fceb400b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cce691adb7354751 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:12.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.125+0000 7fcec77fe700 1 -- 172.21.15.55:0/3851289883 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb400db40 con 0x7fcec81566e0 2023-10-06T03:49:12.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.125+0000 7fcec77fe700 1 -- 172.21.15.55:0/3851289883 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb400dca0 con 0x7fcec81566e0 2023-10-06T03:49:12.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.125+0000 7fcec77fe700 1 -- 172.21.15.55:0/3851289883 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb40196b0 con 0x7fcec81566e0 2023-10-06T03:49:12.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.125+0000 7fcecdf9d700 1 -- 172.21.15.55:0/3851289883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcec81566e0 msgr2=0x7fcec80ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:12.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.125+0000 7fcecdf9d700 1 --2- 172.21.15.55:0/3851289883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcec81566e0 0x7fcec80ae000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fceb400bb90 tx=0x7fceb400b430 comp rx=0 tx=0).stop 2023-10-06T03:49:12.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.125+0000 7fcecdf9d700 1 -- 172.21.15.55:0/3851289883 shutdown_connections 2023-10-06T03:49:12.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.125+0000 7fcecdf9d700 1 --2- 172.21.15.55:0/3851289883 >> v2:172.21.15.163:3300/0 conn(0x7fcec8156310 0x7fcec80c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:12.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.125+0000 7fcecdf9d700 1 --2- 172.21.15.55:0/3851289883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcec81566e0 0x7fcec80ae000 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:12.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.125+0000 7fcecdf9d700 1 -- 172.21.15.55:0/3851289883 >> 172.21.15.55:0/3851289883 conn(0x7fcec80cc1f0 msgr2=0x7fcec80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:12.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.126+0000 7fcecdf9d700 1 -- 172.21.15.55:0/3851289883 shutdown_connections 2023-10-06T03:49:12.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.126+0000 7fcecdf9d700 1 -- 172.21.15.55:0/3851289883 wait complete. 2023-10-06T03:49:12.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.127+0000 7fcecdf9d700 1 Processor -- start 2023-10-06T03:49:12.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.127+0000 7fcecdf9d700 1 -- start start 2023-10-06T03:49:12.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.128+0000 7fcecdf9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcec80ca300 0x7fcec80c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:12.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.128+0000 7fcecdf9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcec8156310 0x7fcec8143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:12.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.128+0000 7fcecdf9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcec81566e0 0x7fcec8143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:12.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.128+0000 7fcecdf9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80aef90 con 0x7fcec80ca300 2023-10-06T03:49:12.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.128+0000 7fcecdf9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec8169e80 con 0x7fcec81566e0 2023-10-06T03:49:12.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.128+0000 7fcecdf9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec816a160 con 0x7fcec8156310 2023-10-06T03:49:12.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.128+0000 7fceccf9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcec80ca300 0x7fcec80c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:12.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.128+0000 7fceccf9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcec80ca300 0x7fcec80c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50170/0 (socket says 172.21.15.55:50170) 2023-10-06T03:49:12.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.128+0000 7fceccf9b700 1 -- 172.21.15.55:0/1859386368 learned_addr learned my addr 172.21.15.55:0/1859386368 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:12.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.128+0000 7fcec7fff700 1 --2- 172.21.15.55:0/1859386368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcec8156310 0x7fcec8143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:12.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.129+0000 7fcecd79c700 1 --2- 172.21.15.55:0/1859386368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcec81566e0 0x7fcec8143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:12.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.129+0000 7fceccf9b700 1 -- 172.21.15.55:0/1859386368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcec8156310 msgr2=0x7fcec8143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:12.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.129+0000 7fceccf9b700 1 --2- 172.21.15.55:0/1859386368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcec8156310 0x7fcec8143810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:12.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.129+0000 7fceccf9b700 1 -- 172.21.15.55:0/1859386368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcec81566e0 msgr2=0x7fcec8143d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:12.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.129+0000 7fceccf9b700 1 --2- 172.21.15.55:0/1859386368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcec81566e0 0x7fcec8143d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:12.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.129+0000 7fceccf9b700 1 -- 172.21.15.55:0/1859386368 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc0094f0 con 0x7fcec80ca300 2023-10-06T03:49:12.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.129+0000 7fceccf9b700 1 --2- 172.21.15.55:0/1859386368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcec80ca300 0x7fcec80c9960 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00ec50 tx=0x7fcebc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:12.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.129+0000 7fcec4ff9700 1 -- 172.21.15.55:0/1859386368 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00c850 con 0x7fcec80ca300 2023-10-06T03:49:12.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.129+0000 7fcec4ff9700 1 -- 172.21.15.55:0/1859386368 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc00c9b0 con 0x7fcec80ca300 2023-10-06T03:49:12.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.129+0000 7fcec4ff9700 1 -- 172.21.15.55:0/1859386368 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00cd50 con 0x7fcec80ca300 2023-10-06T03:49:12.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.130+0000 7fcecdf9d700 1 -- 172.21.15.55:0/1859386368 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb400b6d0 con 0x7fcec80ca300 2023-10-06T03:49:12.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.130+0000 7fcecdf9d700 1 -- 172.21.15.55:0/1859386368 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec816a720 con 0x7fcec80ca300 2023-10-06T03:49:12.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.131+0000 7fcecdf9d700 1 -- 172.21.15.55:0/1859386368 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec804c3c0 con 0x7fcec80ca300 2023-10-06T03:49:12.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.132+0000 7fcec4ff9700 1 -- 172.21.15.55:0/1859386368 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcebc013680 con 0x7fcec80ca300 2023-10-06T03:49:12.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.132+0000 7fcec4ff9700 1 --2- 172.21.15.55:0/1859386368 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fceac066f30 0x7fceac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:12.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.132+0000 7fcec4ff9700 1 -- 172.21.15.55:0/1859386368 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fcebc084f40 con 0x7fcec80ca300 2023-10-06T03:49:12.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.133+0000 7fcec7fff700 1 --2- 172.21.15.55:0/1859386368 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fceac066f30 0x7fceac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:12.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.134+0000 7fcec7fff700 1 --2- 172.21.15.55:0/1859386368 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fceac066f30 0x7fceac0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fceb400bb90 tx=0x7fceb401b000 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:12.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.137+0000 7fcec4ff9700 1 -- 172.21.15.55:0/1859386368 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcebc051e20 con 0x7fcec80ca300 2023-10-06T03:49:12.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.367+0000 7fcecdf9d700 1 -- 172.21.15.55:0/1859386368 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fcec816a2d0 con 0x7fceac066f30 2023-10-06T03:49:12.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.369+0000 7fcec4ff9700 1 -- 172.21.15.55:0/1859386368 <== mgr.5828 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fcec816a2d0 con 0x7fceac066f30 2023-10-06T03:49:12.370 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:49:12.370 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:12.370 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephfs" 2023-10-06T03:49:12.370 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:12.371 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:49:12.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.371+0000 7fcecdf9d700 1 -- 172.21.15.55:0/1859386368 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fceac066f30 msgr2=0x7fceac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:12.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.371+0000 7fcecdf9d700 1 --2- 172.21.15.55:0/1859386368 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fceac066f30 0x7fceac0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fceb400bb90 tx=0x7fceb401b000 comp rx=0 tx=0).stop 2023-10-06T03:49:12.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.371+0000 7fcecdf9d700 1 -- 172.21.15.55:0/1859386368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcec80ca300 msgr2=0x7fcec80c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:12.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.371+0000 7fcecdf9d700 1 --2- 172.21.15.55:0/1859386368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcec80ca300 0x7fcec80c9960 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00ec50 tx=0x7fcebc009250 comp rx=0 tx=0).stop 2023-10-06T03:49:12.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.371+0000 7fcecdf9d700 1 -- 172.21.15.55:0/1859386368 shutdown_connections 2023-10-06T03:49:12.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.371+0000 7fcecdf9d700 1 --2- 172.21.15.55:0/1859386368 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fceac066f30 0x7fceac0693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:12.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.371+0000 7fcecdf9d700 1 --2- 172.21.15.55:0/1859386368 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcec81566e0 0x7fcec8143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:12.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.371+0000 7fcecdf9d700 1 --2- 172.21.15.55:0/1859386368 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcec80ca300 0x7fcec80c9960 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:12.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.371+0000 7fcecdf9d700 1 --2- 172.21.15.55:0/1859386368 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcec8156310 0x7fcec8143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:12.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.371+0000 7fcecdf9d700 1 -- 172.21.15.55:0/1859386368 >> 172.21.15.55:0/1859386368 conn(0x7fcec80cc1f0 msgr2=0x7fcec8151de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:12.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.371+0000 7fcecdf9d700 1 -- 172.21.15.55:0/1859386368 shutdown_connections 2023-10-06T03:49:12.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.371+0000 7fcecdf9d700 1 -- 172.21.15.55:0/1859386368 wait complete. 2023-10-06T03:49:12.390 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T03:49:12.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.577+0000 7fca43fff700 1 Processor -- start 2023-10-06T03:49:12.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.577+0000 7fca43fff700 1 -- start start 2023-10-06T03:49:12.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.579+0000 7fca43fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca4406f060 0x7fca440ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:12.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.579+0000 7fca43fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fca4406f4b0 0x7fca440ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:12.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.579+0000 7fca43fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca4406ebd0 con 0x7fca440cb1a0 2023-10-06T03:49:12.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.579+0000 7fca43fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4415b2e0 con 0x7fca4406f4b0 2023-10-06T03:49:12.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.579+0000 7fca43fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4415aa70 con 0x7fca4406f060 2023-10-06T03:49:12.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.579+0000 7fca427fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca4406f060 0x7fca440ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.579+0000 7fca437fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fca4406f4b0 0x7fca440ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.579+0000 7fca437fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fca4406f4b0 0x7fca440ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:32872/0 (socket says 172.21.15.55:32872) 2023-10-06T03:49:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.579+0000 7fca427fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca4406f060 0x7fca440ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:33262/0 (socket says 172.21.15.55:33262) 2023-10-06T03:49:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.579+0000 7fca437fe700 1 -- 172.21.15.55:0/1562275864 learned_addr learned my addr 172.21.15.55:0/1562275864 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.580+0000 7fca437fe700 1 -- 172.21.15.55:0/1562275864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca4406f060 msgr2=0x7fca440ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.580+0000 7fca437fe700 1 --2- 172.21.15.55:0/1562275864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca4406f060 0x7fca440ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.580+0000 7fca437fe700 1 -- 172.21.15.55:0/1562275864 >> v1:172.21.15.163:6789/0 conn(0x7fca440cb1a0 legacy=0x7fca4415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.580+0000 7fca437fe700 1 -- 172.21.15.55:0/1562275864 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4415ad20 con 0x7fca4406f4b0 2023-10-06T03:49:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.580+0000 7fca41ffb700 1 -- 172.21.15.55:0/1562275864 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42028597 0 0) 0x7fca4406ebd0 con 0x7fca440cb1a0 2023-10-06T03:49:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.580+0000 7fca437fe700 1 --2- 172.21.15.55:0/1562275864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca4406f4b0 0x7fca440ae610 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009f20 tx=0x7fca3c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a023439256036f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.580+0000 7fca41ffb700 1 -- 172.21.15.55:0/1562275864 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c00e070 con 0x7fca4406f4b0 2023-10-06T03:49:12.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.580+0000 7fca41ffb700 1 -- 172.21.15.55:0/1562275864 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3c00ba60 con 0x7fca4406f4b0 2023-10-06T03:49:12.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.580+0000 7fca41ffb700 1 -- 172.21.15.55:0/1562275864 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3c0092e0 con 0x7fca4406f4b0 2023-10-06T03:49:12.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.581+0000 7fca43fff700 1 -- 172.21.15.55:0/1562275864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca4406f4b0 msgr2=0x7fca440ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:12.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.581+0000 7fca43fff700 1 --2- 172.21.15.55:0/1562275864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca4406f4b0 0x7fca440ae610 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009f20 tx=0x7fca3c009840 comp rx=0 tx=0).stop 2023-10-06T03:49:12.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.581+0000 7fca43fff700 1 -- 172.21.15.55:0/1562275864 shutdown_connections 2023-10-06T03:49:12.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.581+0000 7fca43fff700 1 --2- 172.21.15.55:0/1562275864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca4406f4b0 0x7fca440ae610 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:12.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.581+0000 7fca43fff700 1 --2- 172.21.15.55:0/1562275864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca4406f060 0x7fca440ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:12.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.581+0000 7fca43fff700 1 -- 172.21.15.55:0/1562275864 >> 172.21.15.55:0/1562275864 conn(0x7fca440cc1e0 msgr2=0x7fca440cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:12.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.581+0000 7fca43fff700 1 -- 172.21.15.55:0/1562275864 shutdown_connections 2023-10-06T03:49:12.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.581+0000 7fca43fff700 1 -- 172.21.15.55:0/1562275864 wait complete. 2023-10-06T03:49:12.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.582+0000 7fca43fff700 1 Processor -- start 2023-10-06T03:49:12.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.583+0000 7fca43fff700 1 -- start start 2023-10-06T03:49:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.583+0000 7fca43fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca4406f060 0x7fca440c0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.583+0000 7fca43fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca4406f4b0 0x7fca440c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.583+0000 7fca43fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca440cb1a0 0x7fca44152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.583+0000 7fca43fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca441572f0 con 0x7fca4406f060 2023-10-06T03:49:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.583+0000 7fca43fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca440c18d0 con 0x7fca440cb1a0 2023-10-06T03:49:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.583+0000 7fca43fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca440c1bb0 con 0x7fca4406f4b0 2023-10-06T03:49:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.584+0000 7fca42ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca4406f060 0x7fca440c0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.584+0000 7fca42ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca4406f060 0x7fca440c0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50178/0 (socket says 172.21.15.55:50178) 2023-10-06T03:49:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.584+0000 7fca42ffd700 1 -- 172.21.15.55:0/566045273 learned_addr learned my addr 172.21.15.55:0/566045273 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.584+0000 7fca437fe700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca440cb1a0 0x7fca44152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.584+0000 7fca427fc700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca4406f4b0 0x7fca440c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.584+0000 7fca42ffd700 1 -- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca4406f4b0 msgr2=0x7fca440c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.584+0000 7fca42ffd700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca4406f4b0 0x7fca440c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.584+0000 7fca42ffd700 1 -- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca440cb1a0 msgr2=0x7fca44152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.584+0000 7fca42ffd700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca440cb1a0 0x7fca44152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.584+0000 7fca42ffd700 1 -- 172.21.15.55:0/566045273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c009ae0 con 0x7fca4406f060 2023-10-06T03:49:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.584+0000 7fca437fe700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca440cb1a0 0x7fca44152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.585+0000 7fca42ffd700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca4406f060 0x7fca440c0f80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fca3400c530 tx=0x7fca3400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.585+0000 7fca2affd700 1 -- 172.21.15.55:0/566045273 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34011590 con 0x7fca4406f060 2023-10-06T03:49:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.585+0000 7fca2affd700 1 -- 172.21.15.55:0/566045273 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca340116f0 con 0x7fca4406f060 2023-10-06T03:49:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.585+0000 7fca43fff700 1 -- 172.21.15.55:0/566045273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca440c1e90 con 0x7fca4406f060 2023-10-06T03:49:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.585+0000 7fca2affd700 1 -- 172.21.15.55:0/566045273 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34011ab0 con 0x7fca4406f060 2023-10-06T03:49:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.585+0000 7fca43fff700 1 -- 172.21.15.55:0/566045273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca440c14c0 con 0x7fca4406f060 2023-10-06T03:49:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.586+0000 7fca43fff700 1 -- 172.21.15.55:0/566045273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4406c450 con 0x7fca4406f060 2023-10-06T03:49:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.587+0000 7fca2affd700 1 -- 172.21.15.55:0/566045273 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 33) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca3401b3b0 con 0x7fca4406f060 2023-10-06T03:49:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.588+0000 7fca2affd700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fca30066f30 0x7fca300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.588+0000 7fca2affd700 1 -- 172.21.15.55:0/566045273 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fca34083e30 con 0x7fca4406f060 2023-10-06T03:49:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.588+0000 7fca427fc700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fca30066f30 0x7fca300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:12.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.589+0000 7fca427fc700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fca30066f30 0x7fca300693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca38009d20 tx=0x7fca38008040 comp rx=0 tx=0).ready entity=mgr.5828 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:12.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.592+0000 7fca2affd700 1 -- 172.21.15.55:0/566045273 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca34050d10 con 0x7fca4406f060 2023-10-06T03:49:12.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:12.848+0000 7fca43fff700 1 -- 172.21.15.55:0/566045273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fca4415b000 con 0x7fca4406f060 2023-10-06T03:49:13.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.739+0000 7fca2affd700 1 -- 172.21.15.55:0/566045273 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 34) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fca340510e0 con 0x7fca4406f060 2023-10-06T03:49:13.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.741+0000 7fca2affd700 1 -- 172.21.15.55:0/566045273 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v34) v1 ==== 92+0+0 (secure 0 0 0) 0x7fca340547a0 con 0x7fca4406f060 2023-10-06T03:49:13.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.744+0000 7fca43fff700 1 -- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fca30066f30 msgr2=0x7fca300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:13.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.744+0000 7fca43fff700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fca30066f30 0x7fca300693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fca38009d20 tx=0x7fca38008040 comp rx=0 tx=0).stop 2023-10-06T03:49:13.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.744+0000 7fca43fff700 1 -- 172.21.15.55:0/566045273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca4406f060 msgr2=0x7fca440c0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:13.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.744+0000 7fca43fff700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca4406f060 0x7fca440c0f80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fca3400c530 tx=0x7fca3400c8f0 comp rx=0 tx=0).stop 2023-10-06T03:49:13.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.744+0000 7fca43fff700 1 -- 172.21.15.55:0/566045273 shutdown_connections 2023-10-06T03:49:13.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.744+0000 7fca43fff700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fca30066f30 0x7fca300693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:13.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.744+0000 7fca43fff700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca440cb1a0 0x7fca44152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:13.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.744+0000 7fca43fff700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca4406f060 0x7fca440c0f80 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:13.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.744+0000 7fca43fff700 1 --2- 172.21.15.55:0/566045273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca4406f4b0 0x7fca440c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:13.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.744+0000 7fca43fff700 1 -- 172.21.15.55:0/566045273 >> 172.21.15.55:0/566045273 conn(0x7fca440cc1e0 msgr2=0x7fca44153ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:13.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.744+0000 7fca43fff700 1 -- 172.21.15.55:0/566045273 shutdown_connections 2023-10-06T03:49:13.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.744+0000 7fca43fff700 1 -- 172.21.15.55:0/566045273 wait complete. 2023-10-06T03:49:13.766 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T03:49:13.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.964+0000 7f133a39d700 1 Processor -- start 2023-10-06T03:49:13.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.965+0000 7f133a39d700 1 -- start start 2023-10-06T03:49:13.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.966+0000 7f133a39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f13340b4e70 0x7f13340ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:13.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.966+0000 7f133a39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f13340b16d0 0x7f13340b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:13.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.966+0000 7f133a39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13340b2080 con 0x7f133415d240 2023-10-06T03:49:13.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.966+0000 7f133a39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13340cfb00 con 0x7f13340b16d0 2023-10-06T03:49:13.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.966+0000 7f133a39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13340b5c40 con 0x7f13340b4e70 2023-10-06T03:49:13.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.966+0000 7f1339b9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f133415d240 0x7f133415d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47476/0 (socket says 172.21.15.55:47476) 2023-10-06T03:49:13.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.966+0000 7f1338b9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f13340b16d0 0x7f13340b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:13.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.966+0000 7f1339b9c700 1 -- 172.21.15.55:0/2487093099 learned_addr learned my addr 172.21.15.55:0/2487093099 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:13.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.966+0000 7f133939b700 1 --2- 172.21.15.55:0/2487093099 >> v2:172.21.15.163:3300/0 conn(0x7f13340b4e70 0x7f13340ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:13.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.967+0000 7f132bfff700 1 -- 172.21.15.55:0/2487093099 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611430373 0 0) 0x7f13340b2080 con 0x7f133415d240 2023-10-06T03:49:13.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.967+0000 7f132bfff700 1 -- 172.21.15.55:0/2487093099 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1320003570 con 0x7f133415d240 2023-10-06T03:49:13.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.967+0000 7f1338b9a700 1 -- 172.21.15.55:0/2487093099 >> v2:172.21.15.163:3300/0 conn(0x7f13340b4e70 msgr2=0x7f13340ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:13.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.967+0000 7f1338b9a700 1 --2- 172.21.15.55:0/2487093099 >> v2:172.21.15.163:3300/0 conn(0x7f13340b4e70 0x7f13340ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:13.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.967+0000 7f1338b9a700 1 -- 172.21.15.55:0/2487093099 >> v1:172.21.15.55:6789/0 conn(0x7f133415d240 legacy=0x7f133415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:13.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.967+0000 7f1338b9a700 1 -- 172.21.15.55:0/2487093099 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13340b5f20 con 0x7f13340b16d0 2023-10-06T03:49:13.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.967+0000 7f1338b9a700 1 --2- 172.21.15.55:0/2487093099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13340b16d0 0x7f13340b1b40 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1334064910 tx=0x7f13240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e687371cd44d4256 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:13.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.967+0000 7f132bfff700 1 -- 172.21.15.55:0/2487093099 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324011630 con 0x7f13340b16d0 2023-10-06T03:49:13.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.967+0000 7f132bfff700 1 -- 172.21.15.55:0/2487093099 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13240117b0 con 0x7f13340b16d0 2023-10-06T03:49:13.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.967+0000 7f132bfff700 1 -- 172.21.15.55:0/2487093099 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324011b70 con 0x7f13340b16d0 2023-10-06T03:49:13.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.968+0000 7f133a39d700 1 -- 172.21.15.55:0/2487093099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13340b16d0 msgr2=0x7f13340b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:13.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.968+0000 7f133a39d700 1 --2- 172.21.15.55:0/2487093099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13340b16d0 0x7f13340b1b40 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1334064910 tx=0x7f13240091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:13.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.968+0000 7f133a39d700 1 -- 172.21.15.55:0/2487093099 shutdown_connections 2023-10-06T03:49:13.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.968+0000 7f133a39d700 1 --2- 172.21.15.55:0/2487093099 >> v2:172.21.15.163:3300/0 conn(0x7f13340b4e70 0x7f13340ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:13.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.968+0000 7f133a39d700 1 --2- 172.21.15.55:0/2487093099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f13340b16d0 0x7f13340b1b40 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:13.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.968+0000 7f133a39d700 1 -- 172.21.15.55:0/2487093099 >> 172.21.15.55:0/2487093099 conn(0x7f13340d10c0 msgr2=0x7f13340affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:13.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.968+0000 7f133a39d700 1 -- 172.21.15.55:0/2487093099 shutdown_connections 2023-10-06T03:49:13.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.968+0000 7f133a39d700 1 -- 172.21.15.55:0/2487093099 wait complete. 2023-10-06T03:49:13.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.970+0000 7f133a39d700 1 Processor -- start 2023-10-06T03:49:13.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.970+0000 7f133a39d700 1 -- start start 2023-10-06T03:49:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.971+0000 7f133a39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13340b16d0 0x7f13340c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.971+0000 7f133a39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13340b4e70 0x7f13340cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.971+0000 7f133a39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f133415d240 0x7f13340cbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.971+0000 7f133a39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13340ca230 con 0x7f133415d240 2023-10-06T03:49:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.971+0000 7f133a39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13340c5b80 con 0x7f13340b16d0 2023-10-06T03:49:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.971+0000 7f133a39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13340cc170 con 0x7f13340b4e70 2023-10-06T03:49:13.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.971+0000 7f1339b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f133415d240 0x7f13340cbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:13.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.971+0000 7f1339b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f133415d240 0x7f13340cbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41914/0 (socket says 172.21.15.55:41914) 2023-10-06T03:49:13.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.971+0000 7f1339b9c700 1 -- 172.21.15.55:0/2286766952 learned_addr learned my addr 172.21.15.55:0/2286766952 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:13.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.971+0000 7f1338b9a700 1 --2- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13340b4e70 0x7f13340cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:13.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.971+0000 7f133939b700 1 --2- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13340b16d0 0x7f13340c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:13.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.972+0000 7f1339b9c700 1 -- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13340b4e70 msgr2=0x7f13340cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:13.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.972+0000 7f1339b9c700 1 --2- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13340b4e70 0x7f13340cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:13.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.972+0000 7f1339b9c700 1 -- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13340b16d0 msgr2=0x7f13340c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:13.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.972+0000 7f1339b9c700 1 --2- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13340b16d0 0x7f13340c95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:13.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.972+0000 7f1339b9c700 1 -- 172.21.15.55:0/2286766952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1324009420 con 0x7f133415d240 2023-10-06T03:49:13.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.972+0000 7f1339b9c700 1 --2- 172.21.15.55:0/2286766952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f133415d240 0x7f13340cbc30 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f132c001460 tx=0x7f132c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:13.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.972+0000 7f13297fa700 1 -- 172.21.15.55:0/2286766952 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c00a870 con 0x7f133415d240 2023-10-06T03:49:13.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.972+0000 7f13297fa700 1 -- 172.21.15.55:0/2286766952 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c00a9d0 con 0x7f133415d240 2023-10-06T03:49:13.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.972+0000 7f13297fa700 1 -- 172.21.15.55:0/2286766952 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c00ad50 con 0x7f133415d240 2023-10-06T03:49:13.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.972+0000 7f133a39d700 1 -- 172.21.15.55:0/2286766952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13340c9f50 con 0x7f133415d240 2023-10-06T03:49:13.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.972+0000 7f133a39d700 1 -- 172.21.15.55:0/2286766952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13340c5810 con 0x7f133415d240 2023-10-06T03:49:13.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.973+0000 7f133a39d700 1 -- 172.21.15.55:0/2286766952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1334088990 con 0x7f133415d240 2023-10-06T03:49:13.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.974+0000 7f13297fa700 1 -- 172.21.15.55:0/2286766952 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 34) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f132c007300 con 0x7f133415d240 2023-10-06T03:49:13.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.975+0000 7f13297fa700 1 --2- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1318066f80 0x7f1318069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:13.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.975+0000 7f13297fa700 1 -- 172.21.15.55:0/2286766952 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f132c084b00 con 0x7f133415d240 2023-10-06T03:49:13.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.975+0000 7f133939b700 1 -- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1318066f80 msgr2=0x7f1318069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:13.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.975+0000 7f133939b700 1 --2- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1318066f80 0x7f1318069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:13.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:13.979+0000 7f13297fa700 1 -- 172.21.15.55:0/2286766952 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f132c051b00 con 0x7f133415d240 2023-10-06T03:49:14.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.176+0000 7f133939b700 1 -- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1318066f80 msgr2=0x7f1318069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:14.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.176+0000 7f133939b700 1 --2- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1318066f80 0x7f1318069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:14.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.185+0000 7f133a39d700 1 -- 172.21.15.55:0/2286766952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f133404bb80 con 0x7f133415d240 2023-10-06T03:49:14.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.191+0000 7f13297fa700 1 -- 172.21.15.55:0/2286766952 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f132c055120 con 0x7f133415d240 2023-10-06T03:49:14.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.193+0000 7f133a39d700 1 -- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1318066f80 msgr2=0x7f1318069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:14.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.193+0000 7f133a39d700 1 --2- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1318066f80 0x7f1318069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.193+0000 7f133a39d700 1 -- 172.21.15.55:0/2286766952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f133415d240 msgr2=0x7f13340cbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:14.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.193+0000 7f133a39d700 1 --2- 172.21.15.55:0/2286766952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f133415d240 0x7f13340cbc30 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f132c001460 tx=0x7f132c001a40 comp rx=0 tx=0).stop 2023-10-06T03:49:14.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.193+0000 7f133a39d700 1 -- 172.21.15.55:0/2286766952 shutdown_connections 2023-10-06T03:49:14.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.193+0000 7f133a39d700 1 --2- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1318066f80 0x7f1318069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.193+0000 7f133a39d700 1 --2- 172.21.15.55:0/2286766952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f133415d240 0x7f13340cbc30 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.193+0000 7f133a39d700 1 --2- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f13340b16d0 0x7f13340c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.193+0000 7f133a39d700 1 --2- 172.21.15.55:0/2286766952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f13340b4e70 0x7f13340cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.193+0000 7f133a39d700 1 -- 172.21.15.55:0/2286766952 >> 172.21.15.55:0/2286766952 conn(0x7f13340d10c0 msgr2=0x7f1334158ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:14.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.194+0000 7f133a39d700 1 -- 172.21.15.55:0/2286766952 shutdown_connections 2023-10-06T03:49:14.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.194+0000 7f133a39d700 1 -- 172.21.15.55:0/2286766952 wait complete. 2023-10-06T03:49:14.214 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T03:49:14.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.406+0000 7faff2b9d700 1 Processor -- start 2023-10-06T03:49:14.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.406+0000 7faff2b9d700 1 -- start start 2023-10-06T03:49:14.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.407+0000 7faff2b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafec0b4e70 0x7fafec0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:14.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.408+0000 7faff2b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafec0cfb00 con 0x7fafec0b16d0 2023-10-06T03:49:14.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.408+0000 7faff2b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafec15e060 con 0x7fafec0b1b40 2023-10-06T03:49:14.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.408+0000 7faff2b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafec0b5c40 con 0x7fafec0b4e70 2023-10-06T03:49:14.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.408+0000 7faff139a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fafec0b16d0 0x7fafec15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47486/0 (socket says 172.21.15.55:47486) 2023-10-06T03:49:14.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.408+0000 7faff139a700 1 -- 172.21.15.55:0/1611666299 learned_addr learned my addr 172.21.15.55:0/1611666299 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:14.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.408+0000 7faff1b9b700 1 --2- 172.21.15.55:0/1611666299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafec0b4e70 0x7fafec0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:14.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.408+0000 7faff0b99700 1 -- 172.21.15.55:0/1611666299 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942805554 0 0) 0x7fafec0cfb00 con 0x7fafec0b16d0 2023-10-06T03:49:14.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.408+0000 7faff0b99700 1 -- 172.21.15.55:0/1611666299 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafd8003570 con 0x7fafec0b16d0 2023-10-06T03:49:14.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.408+0000 7faff0b99700 1 -- 172.21.15.55:0/1611666299 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2666317378 0 0) 0x7fafec15e060 con 0x7fafec0b1b40 2023-10-06T03:49:14.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.408+0000 7faff0b99700 1 -- 172.21.15.55:0/1611666299 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafec0cfb00 con 0x7fafec0b1b40 2023-10-06T03:49:14.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.408+0000 7faff1b9b700 1 -- 172.21.15.55:0/1611666299 >> v1:172.21.15.163:6789/0 conn(0x7fafec0b1b40 legacy=0x7fafec15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:14.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.408+0000 7faff1b9b700 1 -- 172.21.15.55:0/1611666299 >> v1:172.21.15.55:6789/0 conn(0x7fafec0b16d0 legacy=0x7fafec15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:14.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.409+0000 7faff1b9b700 1 -- 172.21.15.55:0/1611666299 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec0b5f20 con 0x7fafec0b4e70 2023-10-06T03:49:14.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.409+0000 7faff1b9b700 1 --2- 172.21.15.55:0/1611666299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafec0b4e70 0x7fafec0ccff0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009910 tx=0x7fafe80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d43f41b6b8ed0e1a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:14.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.409+0000 7faff0b99700 1 -- 172.21.15.55:0/1611666299 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8011630 con 0x7fafec0b4e70 2023-10-06T03:49:14.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.409+0000 7faff0b99700 1 -- 172.21.15.55:0/1611666299 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe8011790 con 0x7fafec0b4e70 2023-10-06T03:49:14.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.409+0000 7faff0b99700 1 -- 172.21.15.55:0/1611666299 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8011b30 con 0x7fafec0b4e70 2023-10-06T03:49:14.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.410+0000 7faff2b9d700 1 -- 172.21.15.55:0/1611666299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafec0b4e70 msgr2=0x7fafec0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:14.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.410+0000 7faff2b9d700 1 --2- 172.21.15.55:0/1611666299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafec0b4e70 0x7fafec0ccff0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009910 tx=0x7fafe80091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.410+0000 7faff2b9d700 1 -- 172.21.15.55:0/1611666299 shutdown_connections 2023-10-06T03:49:14.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.410+0000 7faff2b9d700 1 --2- 172.21.15.55:0/1611666299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafec0b4e70 0x7fafec0ccff0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.410+0000 7faff2b9d700 1 -- 172.21.15.55:0/1611666299 >> 172.21.15.55:0/1611666299 conn(0x7fafec0d10c0 msgr2=0x7fafec0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:14.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.410+0000 7faff2b9d700 1 -- 172.21.15.55:0/1611666299 shutdown_connections 2023-10-06T03:49:14.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.410+0000 7faff2b9d700 1 -- 172.21.15.55:0/1611666299 wait complete. 2023-10-06T03:49:14.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.411+0000 7faff2b9d700 1 Processor -- start 2023-10-06T03:49:14.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.411+0000 7faff2b9d700 1 -- start start 2023-10-06T03:49:14.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.412+0000 7faff2b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafec0b16d0 0x7fafec15cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:14.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.412+0000 7faff2b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafec0b1b40 0x7fafec148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:14.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.412+0000 7faff2b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafec0b4e70 0x7fafec1493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:14.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.412+0000 7faff2b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec16a730 con 0x7fafec0b4e70 2023-10-06T03:49:14.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.412+0000 7faff2b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec1498e0 con 0x7fafec0b16d0 2023-10-06T03:49:14.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.412+0000 7faff2b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafec169e40 con 0x7fafec0b1b40 2023-10-06T03:49:14.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.413+0000 7faff1b9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafec0b16d0 0x7fafec15cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.413+0000 7faff239c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafec0b4e70 0x7fafec1493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.413+0000 7faff139a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafec0b1b40 0x7fafec148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.413+0000 7faff1b9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafec0b16d0 0x7fafec15cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:47874/0 (socket says 172.21.15.55:47874) 2023-10-06T03:49:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.413+0000 7faff1b9b700 1 -- 172.21.15.55:0/1798591661 learned_addr learned my addr 172.21.15.55:0/1798591661 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.413+0000 7faff1b9b700 1 -- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafec0b1b40 msgr2=0x7fafec148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.413+0000 7faff1b9b700 1 --2- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafec0b1b40 0x7fafec148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.413+0000 7faff1b9b700 1 -- 172.21.15.55:0/1798591661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafec0b4e70 msgr2=0x7fafec1493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.413+0000 7faff1b9b700 1 --2- 172.21.15.55:0/1798591661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafec0b4e70 0x7fafec1493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.413+0000 7faff1b9b700 1 -- 172.21.15.55:0/1798591661 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe8009420 con 0x7fafec0b16d0 2023-10-06T03:49:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.413+0000 7faff139a700 1 --2- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafec0b1b40 0x7fafec148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:14.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.414+0000 7faff1b9b700 1 --2- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafec0b16d0 0x7fafec15cb90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fafec070dd0 tx=0x7fafe8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:14.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.414+0000 7fafe1ffb700 1 -- 172.21.15.55:0/1798591661 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe801f070 con 0x7fafec0b16d0 2023-10-06T03:49:14.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.414+0000 7fafe1ffb700 1 -- 172.21.15.55:0/1798591661 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe8002f70 con 0x7fafec0b16d0 2023-10-06T03:49:14.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.414+0000 7fafe1ffb700 1 -- 172.21.15.55:0/1798591661 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8011b30 con 0x7fafec0b16d0 2023-10-06T03:49:14.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.414+0000 7faff2b9d700 1 -- 172.21.15.55:0/1798591661 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafec16a0e0 con 0x7fafec0b16d0 2023-10-06T03:49:14.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.414+0000 7faff2b9d700 1 -- 172.21.15.55:0/1798591661 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafec14cda0 con 0x7fafec0b16d0 2023-10-06T03:49:14.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.415+0000 7faff2b9d700 1 -- 172.21.15.55:0/1798591661 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafec088990 con 0x7fafec0b16d0 2023-10-06T03:49:14.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.416+0000 7fafe1ffb700 1 -- 172.21.15.55:0/1798591661 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 34) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fafe80193b0 con 0x7fafec0b16d0 2023-10-06T03:49:14.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.417+0000 7fafe1ffb700 1 --2- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fafd0066f80 0x7fafd0069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:14.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.417+0000 7fafe1ffb700 1 -- 172.21.15.55:0/1798591661 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fafe8085b00 con 0x7fafec0b16d0 2023-10-06T03:49:14.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.417+0000 7faff139a700 1 -- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fafd0066f80 msgr2=0x7fafd0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:14.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.417+0000 7faff139a700 1 --2- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fafd0066f80 0x7fafd0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:14.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.421+0000 7fafe1ffb700 1 -- 172.21.15.55:0/1798591661 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fafe8087e60 con 0x7fafec0b16d0 2023-10-06T03:49:14.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.618+0000 7faff139a700 1 -- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fafd0066f80 msgr2=0x7fafd0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:14.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.618+0000 7faff139a700 1 --2- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fafd0066f80 0x7fafd0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:14.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.626+0000 7faff2b9d700 1 -- 172.21.15.55:0/1798591661 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fafec04bb80 con 0x7fafec0b16d0 2023-10-06T03:49:14.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.633+0000 7fafe1ffb700 1 -- 172.21.15.55:0/1798591661 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7fafe8028020 con 0x7fafec0b16d0 2023-10-06T03:49:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.635+0000 7faff2b9d700 1 -- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fafd0066f80 msgr2=0x7fafd0069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.635+0000 7faff2b9d700 1 --2- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fafd0066f80 0x7fafd0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.635+0000 7faff2b9d700 1 -- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafec0b16d0 msgr2=0x7fafec15cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.635+0000 7faff2b9d700 1 --2- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafec0b16d0 0x7fafec15cb90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fafec070dd0 tx=0x7fafe8017000 comp rx=0 tx=0).stop 2023-10-06T03:49:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.635+0000 7faff2b9d700 1 -- 172.21.15.55:0/1798591661 shutdown_connections 2023-10-06T03:49:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.635+0000 7faff2b9d700 1 --2- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fafd0066f80 0x7fafd0069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.635+0000 7faff2b9d700 1 --2- 172.21.15.55:0/1798591661 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fafec0b4e70 0x7fafec1493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.635+0000 7faff2b9d700 1 --2- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fafec0b16d0 0x7fafec15cb90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.635+0000 7faff2b9d700 1 --2- 172.21.15.55:0/1798591661 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fafec0b1b40 0x7fafec148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.635+0000 7faff2b9d700 1 -- 172.21.15.55:0/1798591661 >> 172.21.15.55:0/1798591661 conn(0x7fafec0d10c0 msgr2=0x7fafec15da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:14.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.635+0000 7faff2b9d700 1 -- 172.21.15.55:0/1798591661 shutdown_connections 2023-10-06T03:49:14.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.635+0000 7faff2b9d700 1 -- 172.21.15.55:0/1798591661 wait complete. 2023-10-06T03:49:14.655 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-10-06T03:49:14.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.848+0000 7f786699d700 1 Processor -- start 2023-10-06T03:49:14.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.848+0000 7f786699d700 1 -- start start 2023-10-06T03:49:14.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.849+0000 7f786699d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f78600afa40 0x7f78601647e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:14.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.849+0000 7f786699d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78600afe60 con 0x7f78600ac0e0 2023-10-06T03:49:14.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.849+0000 7f786699d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786006e510 con 0x7f78601597d0 2023-10-06T03:49:14.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.849+0000 7f786699d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786015e450 con 0x7f78600afa40 2023-10-06T03:49:14.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.849+0000 7f786519a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f78600ac0e0 0x7f786006f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47496/0 (socket says 172.21.15.55:47496) 2023-10-06T03:49:14.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.849+0000 7f786519a700 1 -- 172.21.15.55:0/1790424609 learned_addr learned my addr 172.21.15.55:0/1790424609 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:14.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.850+0000 7f786619c700 1 --2- 172.21.15.55:0/1790424609 >> v2:172.21.15.55:3300/0 conn(0x7f78600afa40 0x7f78601647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.850+0000 7f7864999700 1 -- 172.21.15.55:0/1790424609 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485863958 0 0) 0x7f78600afe60 con 0x7f78600ac0e0 2023-10-06T03:49:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.850+0000 7f7864999700 1 -- 172.21.15.55:0/1790424609 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f784c003570 con 0x7f78600ac0e0 2023-10-06T03:49:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.850+0000 7f786619c700 1 -- 172.21.15.55:0/1790424609 >> v1:172.21.15.163:6789/0 conn(0x7f78601597d0 legacy=0x7f786006e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.850+0000 7f786619c700 1 -- 172.21.15.55:0/1790424609 >> v1:172.21.15.55:6789/0 conn(0x7f78600ac0e0 legacy=0x7f786006f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.850+0000 7f786619c700 1 -- 172.21.15.55:0/1790424609 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7860069180 con 0x7f78600afa40 2023-10-06T03:49:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.850+0000 7f786619c700 1 --2- 172.21.15.55:0/1790424609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78600afa40 0x7f78601647e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7858009e60 tx=0x7f7858009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7f078cfb97314 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.851+0000 7f7864999700 1 -- 172.21.15.55:0/1790424609 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785800e070 con 0x7f78600afa40 2023-10-06T03:49:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.851+0000 7f7864999700 1 -- 172.21.15.55:0/1790424609 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785800bb60 con 0x7f78600afa40 2023-10-06T03:49:14.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.851+0000 7f7864999700 1 -- 172.21.15.55:0/1790424609 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858012430 con 0x7f78600afa40 2023-10-06T03:49:14.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.851+0000 7f786699d700 1 -- 172.21.15.55:0/1790424609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78600afa40 msgr2=0x7f78601647e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:14.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.851+0000 7f786699d700 1 --2- 172.21.15.55:0/1790424609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78600afa40 0x7f78601647e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7858009e60 tx=0x7f7858009780 comp rx=0 tx=0).stop 2023-10-06T03:49:14.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.851+0000 7f786699d700 1 -- 172.21.15.55:0/1790424609 shutdown_connections 2023-10-06T03:49:14.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.851+0000 7f786699d700 1 --2- 172.21.15.55:0/1790424609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f78600afa40 0x7f78601647e0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.851+0000 7f786699d700 1 -- 172.21.15.55:0/1790424609 >> 172.21.15.55:0/1790424609 conn(0x7f786006adf0 msgr2=0x7f78600ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:14.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.851+0000 7f786699d700 1 -- 172.21.15.55:0/1790424609 shutdown_connections 2023-10-06T03:49:14.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.851+0000 7f786699d700 1 -- 172.21.15.55:0/1790424609 wait complete. 2023-10-06T03:49:14.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.854+0000 7f786699d700 1 Processor -- start 2023-10-06T03:49:14.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.854+0000 7f786699d700 1 -- start start 2023-10-06T03:49:14.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786699d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78600afa40 0x7f7860155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:14.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786699d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78601597d0 0x7f78601563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:14.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786699d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7860150d40 0x7f786015c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:14.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786699d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78601696c0 con 0x7f7860150d40 2023-10-06T03:49:14.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786699d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786015da70 con 0x7f78601597d0 2023-10-06T03:49:14.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786699d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7860152f80 con 0x7f78600afa40 2023-10-06T03:49:14.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786619c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7860150d40 0x7f786015c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:14.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786619c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7860150d40 0x7f786015c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41954/0 (socket says 172.21.15.55:41954) 2023-10-06T03:49:14.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786619c700 1 -- 172.21.15.55:0/2609797276 learned_addr learned my addr 172.21.15.55:0/2609797276 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:14.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786519a700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78601597d0 0x7f78601563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:14.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786599b700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78600afa40 0x7f7860155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:14.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786619c700 1 -- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78600afa40 msgr2=0x7f7860155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:14.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786619c700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f78600afa40 0x7f7860155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786619c700 1 -- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78601597d0 msgr2=0x7f78601563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:14.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786619c700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f78601597d0 0x7f78601563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:14.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.856+0000 7f786619c700 1 -- 172.21.15.55:0/2609797276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7858009a20 con 0x7f7860150d40 2023-10-06T03:49:14.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.857+0000 7f786619c700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7860150d40 0x7f786015c6c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f786006dad0 tx=0x7f7858019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:14.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.857+0000 7f7855ffb700 1 -- 172.21.15.55:0/2609797276 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858027470 con 0x7f7860150d40 2023-10-06T03:49:14.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.857+0000 7f7855ffb700 1 -- 172.21.15.55:0/2609797276 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78580275f0 con 0x7f7860150d40 2023-10-06T03:49:14.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.857+0000 7f7855ffb700 1 -- 172.21.15.55:0/2609797276 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78580279b0 con 0x7f7860150d40 2023-10-06T03:49:14.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.857+0000 7f786699d700 1 -- 172.21.15.55:0/2609797276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786015d4b0 con 0x7f7860150d40 2023-10-06T03:49:14.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.857+0000 7f786699d700 1 -- 172.21.15.55:0/2609797276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7860152c10 con 0x7f7860150d40 2023-10-06T03:49:14.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.858+0000 7f786699d700 1 -- 172.21.15.55:0/2609797276 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7860152a70 con 0x7f7860150d40 2023-10-06T03:49:14.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.859+0000 7f7855ffb700 1 -- 172.21.15.55:0/2609797276 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 34) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f7858024070 con 0x7f7860150d40 2023-10-06T03:49:14.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.860+0000 7f7855ffb700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 0x7f7844069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:14.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.860+0000 7f7855ffb700 1 -- 172.21.15.55:0/2609797276 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f7858087280 con 0x7f7860150d40 2023-10-06T03:49:14.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.860+0000 7f786599b700 1 -- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 msgr2=0x7f7844069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:14.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.860+0000 7f786599b700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 0x7f7844069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:14.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:14.864+0000 7f7855ffb700 1 -- 172.21.15.55:0/2609797276 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7860152a70 con 0x7f7860150d40 2023-10-06T03:49:15.037 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:15.035+0000 7f30c86e9180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:49:15.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:15.061+0000 7f786599b700 1 -- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 msgr2=0x7f7844069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:15.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:15.061+0000 7f786599b700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 0x7f7844069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:15.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:15.090+0000 7f786699d700 1 -- 172.21.15.55:0/2609797276 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f78601530c0 con 0x7f7844066f80 2023-10-06T03:49:15.109 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:15.107+0000 7f77f3e85180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:49:15.203 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:15.202+0000 7f30c86e9180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:49:15.280 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:15.279+0000 7f77f3e85180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:49:15.289 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:15.288+0000 7f30c86e9180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:49:15.370 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:15.368+0000 7f77f3e85180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:49:15.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:15.462+0000 7f786599b700 1 -- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 msgr2=0x7f7844069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:15.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:15.462+0000 7f786599b700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 0x7f7844069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T03:49:15.515 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:15.514+0000 7f30c86e9180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:49:15.602 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:15.600+0000 7f77f3e85180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:49:15.693 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:15.692+0000 7f30c86e9180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:49:15.787 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:15.786+0000 7f30c86e9180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:49:15.788 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:15.787+0000 7f77f3e85180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:49:15.815 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:49:15.819 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:49:15.887 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:15.886+0000 7f77f3e85180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:49:15.956 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:15.955+0000 7f30c86e9180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:49:16.055 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:16.054+0000 7f77f3e85180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:49:16.057 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:16.056+0000 7f30c86e9180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:49:16.160 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:16.159+0000 7f77f3e85180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:49:16.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:16.263+0000 7f786599b700 1 -- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 msgr2=0x7f7844069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:16.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:16.263+0000 7f786599b700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 0x7f7844069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T03:49:16.542 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:16.541+0000 7f30c86e9180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:49:16.637 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:16.636+0000 7f30c86e9180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:49:16.665 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:16.663+0000 7f77f3e85180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:49:16.727 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:16.726+0000 7f30c86e9180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:49:16.763 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:16.762+0000 7f77f3e85180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:49:16.857 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:16.856+0000 7f77f3e85180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:49:17.352 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:17.351+0000 7f30c86e9180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:49:17.508 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:17.507+0000 7f77f3e85180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:49:17.526 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:17.525+0000 7f30c86e9180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:49:17.623 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:17.622+0000 7f30c86e9180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:49:17.687 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:17.686+0000 7f77f3e85180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:49:17.787 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:17.786+0000 7f77f3e85180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:49:17.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:17.865+0000 7f786599b700 1 -- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 msgr2=0x7f7844069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:17.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:17.865+0000 7f786599b700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 0x7f7844069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T03:49:17.921 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:17.919+0000 7f30c86e9180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:49:18.017 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:18.016+0000 7f30c86e9180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:49:18.089 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:18.088+0000 7f77f3e85180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:49:18.190 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:18.188+0000 7f77f3e85180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:49:18.724 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:18.722+0000 7f30c86e9180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:49:18.930 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:18.928+0000 7f77f3e85180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:49:19.012 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.010+0000 7f30c86e9180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:49:19.107 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.105+0000 7f30c86e9180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:49:19.223 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:19.222+0000 7f77f3e85180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:49:19.264 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.262+0000 7f30c86e9180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:49:19.320 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:19.318+0000 7f77f3e85180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:49:19.369 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.367+0000 7f30c86e9180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:49:19.481 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:19.479+0000 7f77f3e85180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:49:19.559 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.557+0000 7f30c86e9180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:49:19.589 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:19.588+0000 7f77f3e85180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:49:19.653 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.651+0000 7f30c86e9180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:49:19.746 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.744+0000 7f30c86e9180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:49:19.788 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:19.786+0000 7f77f3e85180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:49:19.840 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.838+0000 7f30c86e9180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:49:19.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:19.852+0000 7f7855ffb700 1 -- 172.21.15.55:0/2609797276 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 35) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f7858003040 con 0x7f7860150d40 2023-10-06T03:49:19.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:19.853+0000 7f7855ffb700 1 -- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 msgr2=0x7f7844069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:19.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:19.853+0000 7f7855ffb700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 0x7f7844069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:19.890 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:19.889+0000 7f77f3e85180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:49:19.987 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.985+0000 7f303da37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:19.987 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.985+0000 7f303da37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:19.987 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.985+0000 7f303da37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:19.987 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.985+0000 7f303da37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:19.987 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.985+0000 7f303da37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:19.988 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.985+0000 7f303aa31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:19.988 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.985+0000 7f303aa31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:19.988 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.985+0000 7f303aa31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:19.988 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.985+0000 7f303aa31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:19.988 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:19.985+0000 7f303aa31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:19.997 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:19.996+0000 7f77f3e85180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:49:20.096 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:20.095+0000 7f77f3e85180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:49:20.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:20.855+0000 7f7855ffb700 1 -- 172.21.15.55:0/2609797276 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mgrmap(e 36) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f785806a670 con 0x7f7860150d40 2023-10-06T03:49:20.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:20.856+0000 7f7855ffb700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 0x7f78440693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:20.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:20.856+0000 7f7855ffb700 1 -- 172.21.15.55:0/2609797276 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f78601530c0 con 0x7f7844066f80 2023-10-06T03:49:20.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:20.857+0000 7f786599b700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 0x7f78440693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:20.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:20.861+0000 7f786599b700 1 --2- 172.21.15.55:0/2609797276 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7844066f80 0x7f78440693b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f785c0012a0 tx=0x7f785c007ec0 comp rx=0 tx=0).ready entity=mgr.6715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:20.872 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:20.871+0000 7f3044a45700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-06T03:49:20.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:20.871+0000 7f7855ffb700 1 -- 172.21.15.55:0/2609797276 <== mgr.6715 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f78601530c0 con 0x7f7844066f80 2023-10-06T03:49:20.875 INFO:teuthology.orchestra.run.smithi055.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-06T03:49:20.884 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T03:49:20.885 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T03:49:21.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.078+0000 7fe51a59d700 1 Processor -- start 2023-10-06T03:49:21.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.078+0000 7fe51a59d700 1 -- start start 2023-10-06T03:49:21.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.079+0000 7fe51a59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe5140ccc00 0x7fe514069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:21.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.079+0000 7fe51a59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5140c9f30 con 0x7fe514159a60 2023-10-06T03:49:21.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.079+0000 7fe51a59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5140ca470 con 0x7fe51406a020 2023-10-06T03:49:21.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.079+0000 7fe51a59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51406dfc0 con 0x7fe5140ccc00 2023-10-06T03:49:21.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.080+0000 7fe51959b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe514159a60 0x7fe5140c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47510/0 (socket says 172.21.15.55:47510) 2023-10-06T03:49:21.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.080+0000 7fe51959b700 1 -- 172.21.15.55:0/252369034 learned_addr learned my addr 172.21.15.55:0/252369034 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:21.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.080+0000 7fe518d9a700 1 --2- 172.21.15.55:0/252369034 >> v2:172.21.15.163:3300/0 conn(0x7fe5140ccc00 0x7fe514069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:21.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.080+0000 7fe50bfff700 1 -- 172.21.15.55:0/252369034 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694553153 0 0) 0x7fe5140c9f30 con 0x7fe514159a60 2023-10-06T03:49:21.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.080+0000 7fe50bfff700 1 -- 172.21.15.55:0/252369034 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe500003570 con 0x7fe514159a60 2023-10-06T03:49:21.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.080+0000 7fe518d9a700 1 -- 172.21.15.55:0/252369034 >> v1:172.21.15.163:6789/0 conn(0x7fe51406a020 legacy=0x7fe5140cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:21.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.080+0000 7fe518d9a700 1 -- 172.21.15.55:0/252369034 >> v1:172.21.15.55:6789/0 conn(0x7fe514159a60 legacy=0x7fe5140c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:21.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.080+0000 7fe518d9a700 1 -- 172.21.15.55:0/252369034 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5140ca750 con 0x7fe5140ccc00 2023-10-06T03:49:21.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.081+0000 7fe518d9a700 1 --2- 172.21.15.55:0/252369034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5140ccc00 0x7fe514069ae0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe5040098e0 tx=0x7fe5040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5e3878b82255f2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:21.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.081+0000 7fe50bfff700 1 -- 172.21.15.55:0/252369034 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50400b6d0 con 0x7fe5140ccc00 2023-10-06T03:49:21.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.081+0000 7fe50bfff700 1 -- 172.21.15.55:0/252369034 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50400b830 con 0x7fe5140ccc00 2023-10-06T03:49:21.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.081+0000 7fe50bfff700 1 -- 172.21.15.55:0/252369034 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50400bbd0 con 0x7fe5140ccc00 2023-10-06T03:49:21.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.082+0000 7fe51a59d700 1 -- 172.21.15.55:0/252369034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5140ccc00 msgr2=0x7fe514069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:21.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.082+0000 7fe51a59d700 1 --2- 172.21.15.55:0/252369034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5140ccc00 0x7fe514069ae0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe5040098e0 tx=0x7fe5040091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:21.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.082+0000 7fe51a59d700 1 -- 172.21.15.55:0/252369034 shutdown_connections 2023-10-06T03:49:21.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.082+0000 7fe51a59d700 1 --2- 172.21.15.55:0/252369034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5140ccc00 0x7fe514069ae0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:21.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.082+0000 7fe51a59d700 1 -- 172.21.15.55:0/252369034 >> 172.21.15.55:0/252369034 conn(0x7fe51406baa0 msgr2=0x7fe51406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:21.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.082+0000 7fe51a59d700 1 -- 172.21.15.55:0/252369034 shutdown_connections 2023-10-06T03:49:21.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.082+0000 7fe51a59d700 1 -- 172.21.15.55:0/252369034 wait complete. 2023-10-06T03:49:21.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.083+0000 7fe51a59d700 1 Processor -- start 2023-10-06T03:49:21.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.084+0000 7fe51a59d700 1 -- start start 2023-10-06T03:49:21.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.084+0000 7fe51a59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe51406a020 0x7fe5141452c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:21.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.084+0000 7fe51a59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5140ccc00 0x7fe514155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:21.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.084+0000 7fe51a59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe514159a60 0x7fe5141562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:21.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.085+0000 7fe51a59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe514145c20 con 0x7fe51406a020 2023-10-06T03:49:21.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.085+0000 7fe51a59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140c5740 con 0x7fe5140ccc00 2023-10-06T03:49:21.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.085+0000 7fe51a59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140c8570 con 0x7fe514159a60 2023-10-06T03:49:21.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.085+0000 7fe51959b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe51406a020 0x7fe5141452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:21.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.085+0000 7fe51959b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe51406a020 0x7fe5141452c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41956/0 (socket says 172.21.15.55:41956) 2023-10-06T03:49:21.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.085+0000 7fe51959b700 1 -- 172.21.15.55:0/3422694847 learned_addr learned my addr 172.21.15.55:0/3422694847 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:21.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.085+0000 7fe518d9a700 1 --2- 172.21.15.55:0/3422694847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5140ccc00 0x7fe514155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:21.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.085+0000 7fe519d9c700 1 --2- 172.21.15.55:0/3422694847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe514159a60 0x7fe5141562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:21.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.085+0000 7fe51959b700 1 -- 172.21.15.55:0/3422694847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe514159a60 msgr2=0x7fe5141562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:21.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.085+0000 7fe51959b700 1 --2- 172.21.15.55:0/3422694847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe514159a60 0x7fe5141562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:21.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.085+0000 7fe51959b700 1 -- 172.21.15.55:0/3422694847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5140ccc00 msgr2=0x7fe514155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:21.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.085+0000 7fe51959b700 1 --2- 172.21.15.55:0/3422694847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5140ccc00 0x7fe514155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:21.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.085+0000 7fe51959b700 1 -- 172.21.15.55:0/3422694847 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe504009420 con 0x7fe51406a020 2023-10-06T03:49:21.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.085+0000 7fe51959b700 1 --2- 172.21.15.55:0/3422694847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe51406a020 0x7fe5141452c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe51000c590 tx=0x7fe510007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:21.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.086+0000 7fe5097fa700 1 -- 172.21.15.55:0/3422694847 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510005b30 con 0x7fe51406a020 2023-10-06T03:49:21.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.086+0000 7fe5097fa700 1 -- 172.21.15.55:0/3422694847 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe510005cb0 con 0x7fe51406a020 2023-10-06T03:49:21.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.086+0000 7fe5097fa700 1 -- 172.21.15.55:0/3422694847 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe510007320 con 0x7fe51406a020 2023-10-06T03:49:21.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.086+0000 7fe51a59d700 1 -- 172.21.15.55:0/3422694847 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5140c8850 con 0x7fe51406a020 2023-10-06T03:49:21.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.086+0000 7fe51a59d700 1 -- 172.21.15.55:0/3422694847 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5140c5c70 con 0x7fe51406a020 2023-10-06T03:49:21.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.087+0000 7fe51a59d700 1 -- 172.21.15.55:0/3422694847 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51406eba0 con 0x7fe51406a020 2023-10-06T03:49:21.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.088+0000 7fe5097fa700 1 -- 172.21.15.55:0/3422694847 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 36) v1 ==== 84595+0+0 (secure 0 0 0) 0x7fe51001b3b0 con 0x7fe51406a020 2023-10-06T03:49:21.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.089+0000 7fe5097fa700 1 --2- 172.21.15.55:0/3422694847 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe4f8066dd0 0x7fe4f8069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:21.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.089+0000 7fe5097fa700 1 -- 172.21.15.55:0/3422694847 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6708+0+0 (secure 0 0 0) 0x7fe510083cc0 con 0x7fe51406a020 2023-10-06T03:49:21.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.089+0000 7fe518d9a700 1 --2- 172.21.15.55:0/3422694847 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe4f8066dd0 0x7fe4f8069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:21.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.090+0000 7fe518d9a700 1 --2- 172.21.15.55:0/3422694847 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe4f8066dd0 0x7fe4f8069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe5040093f0 tx=0x7fe504017000 comp rx=0 tx=0).ready entity=mgr.6715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:21.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.093+0000 7fe5097fa700 1 -- 172.21.15.55:0/3422694847 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe51008a080 con 0x7fe51406a020 2023-10-06T03:49:21.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.355+0000 7fe51a59d700 1 -- 172.21.15.55:0/3422694847 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fe5140caa30 con 0x7fe51406a020 2023-10-06T03:49:21.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.907+0000 7fe5097fa700 1 -- 172.21.15.55:0/3422694847 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fe5100510f0 con 0x7fe51406a020 2023-10-06T03:49:21.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.909+0000 7fe5097fa700 1 -- 172.21.15.55:0/3422694847 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v37) v1 ==== 93+0+0 (secure 0 0 0) 0x7fe5100547b0 con 0x7fe51406a020 2023-10-06T03:49:21.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.911+0000 7fe51a59d700 1 -- 172.21.15.55:0/3422694847 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe4f8066dd0 msgr2=0x7fe4f8069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:21.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.911+0000 7fe51a59d700 1 --2- 172.21.15.55:0/3422694847 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe4f8066dd0 0x7fe4f8069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe5040093f0 tx=0x7fe504017000 comp rx=0 tx=0).stop 2023-10-06T03:49:21.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.911+0000 7fe51a59d700 1 -- 172.21.15.55:0/3422694847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe51406a020 msgr2=0x7fe5141452c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:21.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.911+0000 7fe51a59d700 1 --2- 172.21.15.55:0/3422694847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe51406a020 0x7fe5141452c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe51000c590 tx=0x7fe510007e00 comp rx=0 tx=0).stop 2023-10-06T03:49:21.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.912+0000 7fe51a59d700 1 -- 172.21.15.55:0/3422694847 shutdown_connections 2023-10-06T03:49:21.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.912+0000 7fe51a59d700 1 --2- 172.21.15.55:0/3422694847 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe4f8066dd0 0x7fe4f8069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:21.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.912+0000 7fe51a59d700 1 --2- 172.21.15.55:0/3422694847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe514159a60 0x7fe5141562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:21.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.912+0000 7fe51a59d700 1 --2- 172.21.15.55:0/3422694847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe51406a020 0x7fe5141452c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:21.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.912+0000 7fe51a59d700 1 --2- 172.21.15.55:0/3422694847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5140ccc00 0x7fe514155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:21.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.912+0000 7fe51a59d700 1 -- 172.21.15.55:0/3422694847 >> 172.21.15.55:0/3422694847 conn(0x7fe51406baa0 msgr2=0x7fe514159e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:21.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.912+0000 7fe51a59d700 1 -- 172.21.15.55:0/3422694847 shutdown_connections 2023-10-06T03:49:21.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:21.912+0000 7fe51a59d700 1 -- 172.21.15.55:0/3422694847 wait complete. 2023-10-06T03:49:21.938 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:49:21.939 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:49:22.034 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:49:22.113 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:49:22.113 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:49:22.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:49:22.113 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:49:22.194 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T03:49:22.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:22.195 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:49:22.232 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:49:22.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.460+0000 7f91c7fff700 1 Processor -- start 2023-10-06T03:49:22.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.460+0000 7f91c7fff700 1 -- start start 2023-10-06T03:49:22.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.461+0000 7f91c7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91c8159cc0 0x7f91c80b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:22.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.461+0000 7f91c7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f91c80cf6c0 0x7f91c8163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:22.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.461+0000 7f91c7fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c80b5b40 con 0x7f91c80b2920 2023-10-06T03:49:22.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.461+0000 7f91c7fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c80ce260 con 0x7f91c80cf6c0 2023-10-06T03:49:22.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.461+0000 7f91c7fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c80b5e20 con 0x7f91c8159cc0 2023-10-06T03:49:22.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.462+0000 7f91c77fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f91c80cf6c0 0x7f91c8163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:22.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.462+0000 7f91c67fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f91c80b2920 0x7f91c815a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:47530/0 (socket says 172.21.15.55:47530) 2023-10-06T03:49:22.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.462+0000 7f91c67fc700 1 -- 172.21.15.55:0/755076081 learned_addr learned my addr 172.21.15.55:0/755076081 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:22.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.462+0000 7f91c6ffd700 1 --2- 172.21.15.55:0/755076081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91c8159cc0 0x7f91c80b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:22.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.462+0000 7f91c77fe700 1 -- 172.21.15.55:0/755076081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91c8159cc0 msgr2=0x7f91c80b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:22.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.462+0000 7f91c77fe700 1 --2- 172.21.15.55:0/755076081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91c8159cc0 0x7f91c80b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.462+0000 7f91c77fe700 1 -- 172.21.15.55:0/755076081 >> v1:172.21.15.55:6789/0 conn(0x7f91c80b2920 legacy=0x7f91c815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:22.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.462+0000 7f91c77fe700 1 -- 172.21.15.55:0/755076081 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c80b6100 con 0x7f91c80cf6c0 2023-10-06T03:49:22.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.462+0000 7f91c5ffb700 1 -- 172.21.15.55:0/755076081 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427294289 0 0) 0x7f91c80b5b40 con 0x7f91c80b2920 2023-10-06T03:49:22.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.463+0000 7f91c77fe700 1 --2- 172.21.15.55:0/755076081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91c80cf6c0 0x7f91c8163280 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009e60 tx=0x7f91c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06277617f90bc83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:22.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.463+0000 7f91c5ffb700 1 -- 172.21.15.55:0/755076081 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c000e070 con 0x7f91c80cf6c0 2023-10-06T03:49:22.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.463+0000 7f91c5ffb700 1 -- 172.21.15.55:0/755076081 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c000bb60 con 0x7f91c80cf6c0 2023-10-06T03:49:22.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.463+0000 7f91c5ffb700 1 -- 172.21.15.55:0/755076081 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0012430 con 0x7f91c80cf6c0 2023-10-06T03:49:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.463+0000 7f91c7fff700 1 -- 172.21.15.55:0/755076081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91c80cf6c0 msgr2=0x7f91c8163280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.463+0000 7f91c7fff700 1 --2- 172.21.15.55:0/755076081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91c80cf6c0 0x7f91c8163280 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f91c0009e60 tx=0x7f91c0009780 comp rx=0 tx=0).stop 2023-10-06T03:49:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.463+0000 7f91c7fff700 1 -- 172.21.15.55:0/755076081 shutdown_connections 2023-10-06T03:49:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.463+0000 7f91c7fff700 1 --2- 172.21.15.55:0/755076081 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91c80cf6c0 0x7f91c8163280 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.463+0000 7f91c7fff700 1 --2- 172.21.15.55:0/755076081 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91c8159cc0 0x7f91c80b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.463+0000 7f91c7fff700 1 -- 172.21.15.55:0/755076081 >> 172.21.15.55:0/755076081 conn(0x7f91c80d0560 msgr2=0x7f91c80d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.464+0000 7f91c7fff700 1 -- 172.21.15.55:0/755076081 shutdown_connections 2023-10-06T03:49:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.464+0000 7f91c7fff700 1 -- 172.21.15.55:0/755076081 wait complete. 2023-10-06T03:49:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.465+0000 7f91c7fff700 1 Processor -- start 2023-10-06T03:49:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.465+0000 7f91c7fff700 1 -- start start 2023-10-06T03:49:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.466+0000 7f91c7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91c80b2920 0x7f91c815f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:22.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.466+0000 7f91c7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91c80cf6c0 0x7f91c815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:22.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.466+0000 7f91c7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91c8159cc0 0x7f91c8151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:22.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.466+0000 7f91c7fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c8155150 con 0x7f91c80cf6c0 2023-10-06T03:49:22.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.466+0000 7f91c7fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c8154430 con 0x7f91c8159cc0 2023-10-06T03:49:22.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.466+0000 7f91c7fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c8154710 con 0x7f91c80b2920 2023-10-06T03:49:22.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.466+0000 7f91c67fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91c80cf6c0 0x7f91c815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.466+0000 7f91c67fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91c80cf6c0 0x7f91c815b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41980/0 (socket says 172.21.15.55:41980) 2023-10-06T03:49:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.466+0000 7f91c67fc700 1 -- 172.21.15.55:0/1525062755 learned_addr learned my addr 172.21.15.55:0/1525062755 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.466+0000 7f91c6ffd700 1 --2- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91c80b2920 0x7f91c815f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.466+0000 7f91c77fe700 1 --2- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91c8159cc0 0x7f91c8151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.467+0000 7f91c67fc700 1 -- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91c80b2920 msgr2=0x7f91c815f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.467+0000 7f91c67fc700 1 --2- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91c80b2920 0x7f91c815f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.467+0000 7f91c67fc700 1 -- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91c8159cc0 msgr2=0x7f91c8151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.467+0000 7f91c67fc700 1 --2- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91c8159cc0 0x7f91c8151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.467+0000 7f91c67fc700 1 -- 172.21.15.55:0/1525062755 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c0009a20 con 0x7f91c80cf6c0 2023-10-06T03:49:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.467+0000 7f91c67fc700 1 --2- 172.21.15.55:0/1525062755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91c80cf6c0 0x7f91c815b2b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c5b0 tx=0x7f91bc001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.467+0000 7f91aeffd700 1 -- 172.21.15.55:0/1525062755 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc005b30 con 0x7f91c80cf6c0 2023-10-06T03:49:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.467+0000 7f91aeffd700 1 -- 172.21.15.55:0/1525062755 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc005c90 con 0x7f91c80cf6c0 2023-10-06T03:49:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.467+0000 7f91c7fff700 1 -- 172.21.15.55:0/1525062755 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c8154de0 con 0x7f91c80cf6c0 2023-10-06T03:49:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.467+0000 7f91aeffd700 1 -- 172.21.15.55:0/1525062755 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc007300 con 0x7f91c80cf6c0 2023-10-06T03:49:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.468+0000 7f91c7fff700 1 -- 172.21.15.55:0/1525062755 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c815b7f0 con 0x7f91c80cf6c0 2023-10-06T03:49:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.469+0000 7f91c7fff700 1 -- 172.21.15.55:0/1525062755 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c8154c40 con 0x7f91c80cf6c0 2023-10-06T03:49:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.470+0000 7f91aeffd700 1 -- 172.21.15.55:0/1525062755 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f91bc0076b0 con 0x7f91c80cf6c0 2023-10-06T03:49:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.471+0000 7f91aeffd700 1 --2- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f91b40670e0 0x7f91b4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.471+0000 7f91aeffd700 1 -- 172.21.15.55:0/1525062755 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f91bc084090 con 0x7f91c80cf6c0 2023-10-06T03:49:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.471+0000 7f91c6ffd700 1 -- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f91b40670e0 msgr2=0x7f91b4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.471+0000 7f91c6ffd700 1 --2- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f91b40670e0 0x7f91b4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:22.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.474+0000 7f91aeffd700 1 -- 172.21.15.55:0/1525062755 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91c8154c40 con 0x7f91c80cf6c0 2023-10-06T03:49:22.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.672+0000 7f91c6ffd700 1 -- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f91b40670e0 msgr2=0x7f91b4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:22.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.672+0000 7f91c6ffd700 1 --2- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f91b40670e0 0x7f91b4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:22.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.722+0000 7f91c7fff700 1 -- 172.21.15.55:0/1525062755 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91c8155300 con 0x7f91c80cf6c0 2023-10-06T03:49:22.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.722+0000 7f91aeffd700 1 -- 172.21.15.55:0/1525062755 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f91c8155300 con 0x7f91c80cf6c0 2023-10-06T03:49:22.724 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:22.724 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/4087100648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4087100648},{"type":"v1","addr":"172.21.15.163:6834","nonce":4087100648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2017152000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2017152000},{"type":"v1","addr":"172.21.15.163:6836","nonce":2017152000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1226557536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1226557536},{"type":"v1","addr":"172.21.15.55:6834","nonce":1226557536}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:49:03.547622+0000","modified":"2023-10-06T03:49:07.074089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6804":{"gid":6804,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/3960749421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3960749421},{"type":"v1","addr":"172.21.15.55:6835","nonce":3960749421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T03:49:22.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.725+0000 7f91c7fff700 1 -- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f91b40670e0 msgr2=0x7f91b4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:22.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.725+0000 7f91c7fff700 1 --2- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f91b40670e0 0x7f91b4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.725+0000 7f91c7fff700 1 -- 172.21.15.55:0/1525062755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91c80cf6c0 msgr2=0x7f91c815b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:22.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.725+0000 7f91c7fff700 1 --2- 172.21.15.55:0/1525062755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91c80cf6c0 0x7f91c815b2b0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c5b0 tx=0x7f91bc001070 comp rx=0 tx=0).stop 2023-10-06T03:49:22.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.725+0000 7f91c7fff700 1 -- 172.21.15.55:0/1525062755 shutdown_connections 2023-10-06T03:49:22.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.725+0000 7f91c7fff700 1 --2- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f91b40670e0 0x7f91b4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.725+0000 7f91c7fff700 1 --2- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91c8159cc0 0x7f91c8151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.725+0000 7f91c7fff700 1 --2- 172.21.15.55:0/1525062755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91c80b2920 0x7f91c815f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.725+0000 7f91c7fff700 1 --2- 172.21.15.55:0/1525062755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91c80cf6c0 0x7f91c815b2b0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.725+0000 7f91c7fff700 1 -- 172.21.15.55:0/1525062755 >> 172.21.15.55:0/1525062755 conn(0x7f91c80d0560 msgr2=0x7f91c8152f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:22.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.725+0000 7f91c7fff700 1 -- 172.21.15.55:0/1525062755 shutdown_connections 2023-10-06T03:49:22.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.725+0000 7f91c7fff700 1 -- 172.21.15.55:0/1525062755 wait complete. 2023-10-06T03:49:22.728 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 44 2023-10-06T03:49:22.745 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:49:22.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.935+0000 7f3eef59d700 1 Processor -- start 2023-10-06T03:49:22.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.935+0000 7f3eef59d700 1 -- start start 2023-10-06T03:49:22.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.937+0000 7f3eef59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3ee80b1e50 0x7f3ee8159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:22.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.937+0000 7f3eef59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ee80ceaa0 0x7f3ee80b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:22.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.937+0000 7f3eef59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee80b6570 con 0x7f3ee81593b0 2023-10-06T03:49:22.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.937+0000 7f3eef59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee80cfe20 con 0x7f3ee80b1e50 2023-10-06T03:49:22.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.937+0000 7f3eef59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80cf210 con 0x7f3ee80ceaa0 2023-10-06T03:49:22.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.937+0000 7f3eedd9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3ee80b1e50 0x7f3ee8159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:22.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.937+0000 7f3eedd9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3ee80b1e50 0x7f3ee8159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41986/0 (socket says 172.21.15.55:41986) 2023-10-06T03:49:22.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.937+0000 7f3eedd9a700 1 -- 172.21.15.55:0/973519490 learned_addr learned my addr 172.21.15.55:0/973519490 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:22.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.937+0000 7f3eeed9c700 1 --2- 172.21.15.55:0/973519490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ee80ceaa0 0x7f3ee80b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:22.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.938+0000 7f3eedd9a700 1 -- 172.21.15.55:0/973519490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ee80ceaa0 msgr2=0x7f3ee80b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:22.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.938+0000 7f3eedd9a700 1 --2- 172.21.15.55:0/973519490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ee80ceaa0 0x7f3ee80b6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.938+0000 7f3eed599700 1 -- 172.21.15.55:0/973519490 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570932736 0 0) 0x7f3ee4001110 con 0x7f3ee81593b0 2023-10-06T03:49:22.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.938+0000 7f3eedd9a700 1 -- 172.21.15.55:0/973519490 >> v1:172.21.15.163:6789/0 conn(0x7f3ee81593b0 legacy=0x7f3ee80b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:22.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.938+0000 7f3eedd9a700 1 -- 172.21.15.55:0/973519490 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee80cf4f0 con 0x7f3ee80b1e50 2023-10-06T03:49:22.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.938+0000 7f3eedd9a700 1 --2- 172.21.15.55:0/973519490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee80b1e50 0x7f3ee8159780 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80099b0 tx=0x7f3ed8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be41e3e136ab1408 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:22.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.938+0000 7f3eed599700 1 -- 172.21.15.55:0/973519490 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8009ea0 con 0x7f3ee80b1e50 2023-10-06T03:49:22.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.938+0000 7f3eed599700 1 -- 172.21.15.55:0/973519490 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed800b860 con 0x7f3ee80b1e50 2023-10-06T03:49:22.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.938+0000 7f3eed599700 1 -- 172.21.15.55:0/973519490 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed800bc00 con 0x7f3ee80b1e50 2023-10-06T03:49:22.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.939+0000 7f3eef59d700 1 -- 172.21.15.55:0/973519490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee80b1e50 msgr2=0x7f3ee8159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:22.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.939+0000 7f3eef59d700 1 --2- 172.21.15.55:0/973519490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee80b1e50 0x7f3ee8159780 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80099b0 tx=0x7f3ed8009250 comp rx=0 tx=0).stop 2023-10-06T03:49:22.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.939+0000 7f3eef59d700 1 -- 172.21.15.55:0/973519490 shutdown_connections 2023-10-06T03:49:22.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.939+0000 7f3eef59d700 1 --2- 172.21.15.55:0/973519490 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ee80ceaa0 0x7f3ee80b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.939+0000 7f3eef59d700 1 --2- 172.21.15.55:0/973519490 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee80b1e50 0x7f3ee8159780 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.939+0000 7f3eef59d700 1 -- 172.21.15.55:0/973519490 >> 172.21.15.55:0/973519490 conn(0x7f3ee80d1130 msgr2=0x7f3ee80b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:22.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.939+0000 7f3eef59d700 1 -- 172.21.15.55:0/973519490 shutdown_connections 2023-10-06T03:49:22.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.939+0000 7f3eef59d700 1 -- 172.21.15.55:0/973519490 wait complete. 2023-10-06T03:49:22.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.941+0000 7f3eef59d700 1 Processor -- start 2023-10-06T03:49:22.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.941+0000 7f3eef59d700 1 -- start start 2023-10-06T03:49:22.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eef59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ee80b1e50 0x7f3ee80696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:22.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eef59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee80ceaa0 0x7f3ee80cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:22.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eef59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ee81593b0 0x7f3ee8169ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:22.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eef59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80c7cf0 con 0x7f3ee80ceaa0 2023-10-06T03:49:22.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eef59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80c70e0 con 0x7f3ee81593b0 2023-10-06T03:49:22.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eef59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee80c73c0 con 0x7f3ee80b1e50 2023-10-06T03:49:22.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eedd9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee80ceaa0 0x7f3ee80cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:22.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eedd9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee80ceaa0 0x7f3ee80cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42002/0 (socket says 172.21.15.55:42002) 2023-10-06T03:49:22.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eedd9a700 1 -- 172.21.15.55:0/796635904 learned_addr learned my addr 172.21.15.55:0/796635904 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:22.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eee59b700 1 --2- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ee80b1e50 0x7f3ee80696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:22.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eeed9c700 1 --2- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ee81593b0 0x7f3ee8169ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:22.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eedd9a700 1 -- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ee80b1e50 msgr2=0x7f3ee80696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:22.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eedd9a700 1 --2- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ee80b1e50 0x7f3ee80696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eedd9a700 1 -- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ee81593b0 msgr2=0x7f3ee8169ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:22.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eedd9a700 1 --2- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ee81593b0 0x7f3ee8169ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:22.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.942+0000 7f3eedd9a700 1 -- 172.21.15.55:0/796635904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed80094f0 con 0x7f3ee80ceaa0 2023-10-06T03:49:22.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.943+0000 7f3eedd9a700 1 --2- 172.21.15.55:0/796635904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee80ceaa0 0x7f3ee80cbaf0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3ee806fe00 tx=0x7f3ed8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:22.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.943+0000 7f3ede7fc700 1 -- 172.21.15.55:0/796635904 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8000d10 con 0x7f3ee80ceaa0 2023-10-06T03:49:22.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.943+0000 7f3ede7fc700 1 -- 172.21.15.55:0/796635904 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed800bbe0 con 0x7f3ee80ceaa0 2023-10-06T03:49:22.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.943+0000 7f3eef59d700 1 -- 172.21.15.55:0/796635904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee80c76a0 con 0x7f3ee80ceaa0 2023-10-06T03:49:22.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.943+0000 7f3ede7fc700 1 -- 172.21.15.55:0/796635904 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8011670 con 0x7f3ee80ceaa0 2023-10-06T03:49:22.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.943+0000 7f3eef59d700 1 -- 172.21.15.55:0/796635904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee8069bf0 con 0x7f3ee80ceaa0 2023-10-06T03:49:22.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.944+0000 7f3eef59d700 1 -- 172.21.15.55:0/796635904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee8069e70 con 0x7f3ee80ceaa0 2023-10-06T03:49:22.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.945+0000 7f3ede7fc700 1 -- 172.21.15.55:0/796635904 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f3ed8011aa0 con 0x7f3ee80ceaa0 2023-10-06T03:49:22.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.946+0000 7f3ede7fc700 1 --2- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3ecc0670e0 0x7f3ecc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:22.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.946+0000 7f3ede7fc700 1 -- 172.21.15.55:0/796635904 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f3ed80861e0 con 0x7f3ee80ceaa0 2023-10-06T03:49:22.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.946+0000 7f3eee59b700 1 -- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3ecc0670e0 msgr2=0x7f3ecc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:22.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.946+0000 7f3eee59b700 1 --2- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3ecc0670e0 0x7f3ecc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:22.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:22.950+0000 7f3ede7fc700 1 -- 172.21.15.55:0/796635904 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ed8052d40 con 0x7f3ee80ceaa0 2023-10-06T03:49:23.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.147+0000 7f3eee59b700 1 -- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3ecc0670e0 msgr2=0x7f3ecc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:23.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.147+0000 7f3eee59b700 1 --2- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3ecc0670e0 0x7f3ecc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:23.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.157+0000 7f3eef59d700 1 -- 172.21.15.55:0/796635904 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ee804c3c0 con 0x7f3ee80ceaa0 2023-10-06T03:49:23.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.158+0000 7f3ede7fc700 1 -- 172.21.15.55:0/796635904 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+17885 (secure 0 0 0) 0x7f3ed8056360 con 0x7f3ee80ceaa0 2023-10-06T03:49:23.160 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:23.160 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":94,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:49:19.842571+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:49:01.721643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T03:49:02.520120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3249461740":"2023-10-07T03:49:19.842520+0000","172.21.15.163:0/4224423400":"2023-10-07T03:49:19.842520+0000","172.21.15.163:6837/85191":"2023-10-07T03:49:19.842520+0000","172.21.15.163:6835/85191":"2023-10-07T03:49:19.842520+0000","172.21.15.163:0/1889998714":"2023-10-07T03:49:19.842520+0000","172.21.15.163:0/4148366955":"2023-10-07T03:49:19.842520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:49:23.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.160+0000 7f3eef59d700 1 -- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3ecc0670e0 msgr2=0x7f3ecc069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:23.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.160+0000 7f3eef59d700 1 --2- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3ecc0670e0 0x7f3ecc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.160+0000 7f3eef59d700 1 -- 172.21.15.55:0/796635904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee80ceaa0 msgr2=0x7f3ee80cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:23.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.160+0000 7f3eef59d700 1 --2- 172.21.15.55:0/796635904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee80ceaa0 0x7f3ee80cbaf0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3ee806fe00 tx=0x7f3ed8017000 comp rx=0 tx=0).stop 2023-10-06T03:49:23.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.161+0000 7f3eef59d700 1 -- 172.21.15.55:0/796635904 shutdown_connections 2023-10-06T03:49:23.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.161+0000 7f3eef59d700 1 --2- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3ecc0670e0 0x7f3ecc069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.161+0000 7f3eef59d700 1 --2- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3ee81593b0 0x7f3ee8169ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.161+0000 7f3eef59d700 1 --2- 172.21.15.55:0/796635904 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3ee80b1e50 0x7f3ee80696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.161+0000 7f3eef59d700 1 --2- 172.21.15.55:0/796635904 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3ee80ceaa0 0x7f3ee80cbaf0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.161+0000 7f3eef59d700 1 -- 172.21.15.55:0/796635904 >> 172.21.15.55:0/796635904 conn(0x7f3ee80d1130 msgr2=0x7f3ee80b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:23.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.161+0000 7f3eef59d700 1 -- 172.21.15.55:0/796635904 shutdown_connections 2023-10-06T03:49:23.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.161+0000 7f3eef59d700 1 -- 172.21.15.55:0/796635904 wait complete. 2023-10-06T03:49:23.181 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:49:01.721643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:49:02.520120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:49:23.181 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:49:23.201 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:23.199+0000 7f7dd8257180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:49:23.267 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:23.266+0000 7fd239592180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:49:23.364 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:23.362+0000 7f7dd8257180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:49:23.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.374+0000 7f4afeb9d700 1 Processor -- start 2023-10-06T03:49:23.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.374+0000 7f4afeb9d700 1 -- start start 2023-10-06T03:49:23.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.375+0000 7f4afeb9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4af8153900 0x7f4af80c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:23.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.376+0000 7f4afeb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4af806a710 0x7f4af806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:23.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.376+0000 7f4afeb9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af806e150 con 0x7f4af80c9db0 2023-10-06T03:49:23.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.376+0000 7f4afeb9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af80c7800 con 0x7f4af8153900 2023-10-06T03:49:23.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.376+0000 7f4afeb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af80c7ae0 con 0x7f4af806a710 2023-10-06T03:49:23.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.376+0000 7f4afdb9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4af8153900 0x7f4af80c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:23.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.376+0000 7f4afe39c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4af806a710 0x7f4af806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:23.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.376+0000 7f4afd39a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f4af80c9db0 0x7f4af806a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:34998/0 (socket says 172.21.15.55:34998) 2023-10-06T03:49:23.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.376+0000 7f4afd39a700 1 -- 172.21.15.55:0/3266319827 learned_addr learned my addr 172.21.15.55:0/3266319827 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:23.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.376+0000 7f4afcb99700 1 -- 172.21.15.55:0/3266319827 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3499941582 0 0) 0x7f4af806e150 con 0x7f4af80c9db0 2023-10-06T03:49:23.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.376+0000 7f4afcb99700 1 -- 172.21.15.55:0/3266319827 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ae4003570 con 0x7f4af80c9db0 2023-10-06T03:49:23.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.376+0000 7f4afdb9b700 1 -- 172.21.15.55:0/3266319827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4af806a710 msgr2=0x7f4af806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:23.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.376+0000 7f4afdb9b700 1 --2- 172.21.15.55:0/3266319827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4af806a710 0x7f4af806e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.376+0000 7f4afdb9b700 1 -- 172.21.15.55:0/3266319827 >> v1:172.21.15.163:6789/0 conn(0x7f4af80c9db0 legacy=0x7f4af806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:23.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.376+0000 7f4afdb9b700 1 -- 172.21.15.55:0/3266319827 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af80c7dc0 con 0x7f4af8153900 2023-10-06T03:49:23.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.377+0000 7f4afe39c700 1 --2- 172.21.15.55:0/3266319827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4af806a710 0x7f4af806e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:23.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.377+0000 7f4afdb9b700 1 --2- 172.21.15.55:0/3266319827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4af8153900 0x7f4af80c6f00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4af400aa90 tx=0x7f4af400a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de066bfa771fda24 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:23.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.377+0000 7f4afcb99700 1 -- 172.21.15.55:0/3266319827 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af400c970 con 0x7f4af8153900 2023-10-06T03:49:23.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.377+0000 7f4afcb99700 1 -- 172.21.15.55:0/3266319827 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af400cad0 con 0x7f4af8153900 2023-10-06T03:49:23.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.377+0000 7f4afcb99700 1 -- 172.21.15.55:0/3266319827 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af400ce70 con 0x7f4af8153900 2023-10-06T03:49:23.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.378+0000 7f4afeb9d700 1 -- 172.21.15.55:0/3266319827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4af8153900 msgr2=0x7f4af80c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:23.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.378+0000 7f4afeb9d700 1 --2- 172.21.15.55:0/3266319827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4af8153900 0x7f4af80c6f00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4af400aa90 tx=0x7f4af400a330 comp rx=0 tx=0).stop 2023-10-06T03:49:23.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.378+0000 7f4afeb9d700 1 -- 172.21.15.55:0/3266319827 shutdown_connections 2023-10-06T03:49:23.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.378+0000 7f4afeb9d700 1 --2- 172.21.15.55:0/3266319827 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4af806a710 0x7f4af806e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.378+0000 7f4afeb9d700 1 --2- 172.21.15.55:0/3266319827 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4af8153900 0x7f4af80c6f00 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.378+0000 7f4afeb9d700 1 -- 172.21.15.55:0/3266319827 >> 172.21.15.55:0/3266319827 conn(0x7f4af80cc1e0 msgr2=0x7f4af80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:23.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.378+0000 7f4afeb9d700 1 -- 172.21.15.55:0/3266319827 shutdown_connections 2023-10-06T03:49:23.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.378+0000 7f4afeb9d700 1 -- 172.21.15.55:0/3266319827 wait complete. 2023-10-06T03:49:23.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.379+0000 7f4afeb9d700 1 Processor -- start 2023-10-06T03:49:23.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.379+0000 7f4afeb9d700 1 -- start start 2023-10-06T03:49:23.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.380+0000 7f4afeb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4af806a710 0x7f4af816a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:23.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.380+0000 7f4afeb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4af80c9db0 0x7f4af8144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:23.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.380+0000 7f4afeb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4af8153900 0x7f4af80aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:23.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.380+0000 7f4afeb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af80b03c0 con 0x7f4af80c9db0 2023-10-06T03:49:23.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.380+0000 7f4afeb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af80af7b0 con 0x7f4af806a710 2023-10-06T03:49:23.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.380+0000 7f4afeb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af80afa90 con 0x7f4af8153900 2023-10-06T03:49:23.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.381+0000 7f4afd39a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4af80c9db0 0x7f4af8144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:23.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.381+0000 7f4afe39c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4af8153900 0x7f4af80aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:23.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.381+0000 7f4afdb9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4af806a710 0x7f4af816a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:23.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.381+0000 7f4afe39c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4af8153900 0x7f4af80aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:38158/0 (socket says 172.21.15.55:38158) 2023-10-06T03:49:23.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.381+0000 7f4afdb9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4af806a710 0x7f4af816a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:53558/0 (socket says 172.21.15.55:53558) 2023-10-06T03:49:23.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.381+0000 7f4afdb9b700 1 -- 172.21.15.55:0/2587924664 learned_addr learned my addr 172.21.15.55:0/2587924664 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:23.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.381+0000 7f4afd39a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4af80c9db0 0x7f4af8144fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48898/0 (socket says 172.21.15.55:48898) 2023-10-06T03:49:23.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.381+0000 7f4afdb9b700 1 -- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4af8153900 msgr2=0x7f4af80aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:23.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.381+0000 7f4afdb9b700 1 --2- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4af8153900 0x7f4af80aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.381+0000 7f4afdb9b700 1 -- 172.21.15.55:0/2587924664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4af80c9db0 msgr2=0x7f4af8144fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:23.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.381+0000 7f4afdb9b700 1 --2- 172.21.15.55:0/2587924664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4af80c9db0 0x7f4af8144fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.381+0000 7f4afdb9b700 1 -- 172.21.15.55:0/2587924664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af0009a20 con 0x7f4af806a710 2023-10-06T03:49:23.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.381+0000 7f4afdb9b700 1 --2- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4af806a710 0x7f4af816a140 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4af806ce80 tx=0x7f4af401a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:23.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.382+0000 7f4aedffb700 1 -- 172.21.15.55:0/2587924664 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4021430 con 0x7f4af806a710 2023-10-06T03:49:23.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.382+0000 7f4aedffb700 1 -- 172.21.15.55:0/2587924664 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af4021590 con 0x7f4af806a710 2023-10-06T03:49:23.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.382+0000 7f4afeb9d700 1 -- 172.21.15.55:0/2587924664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af400a5d0 con 0x7f4af806a710 2023-10-06T03:49:23.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.382+0000 7f4aedffb700 1 -- 172.21.15.55:0/2587924664 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4021950 con 0x7f4af806a710 2023-10-06T03:49:23.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.382+0000 7f4afeb9d700 1 -- 172.21.15.55:0/2587924664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af80b0050 con 0x7f4af806a710 2023-10-06T03:49:23.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.383+0000 7f4afeb9d700 1 -- 172.21.15.55:0/2587924664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af806c450 con 0x7f4af806a710 2023-10-06T03:49:23.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.384+0000 7f4aedffb700 1 -- 172.21.15.55:0/2587924664 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f4af40103f0 con 0x7f4af806a710 2023-10-06T03:49:23.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.385+0000 7f4aedffb700 1 --2- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4adc0670e0 0x7f4adc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:23.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.385+0000 7f4aedffb700 1 -- 172.21.15.55:0/2587924664 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f4af40863e0 con 0x7f4af806a710 2023-10-06T03:49:23.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.385+0000 7f4afd39a700 1 -- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4adc0670e0 msgr2=0x7f4adc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:23.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.385+0000 7f4afd39a700 1 --2- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4adc0670e0 0x7f4adc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:23.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.389+0000 7f4aedffb700 1 -- 172.21.15.55:0/2587924664 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4af4052f40 con 0x7f4af806a710 2023-10-06T03:49:23.441 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:23.440+0000 7fd239592180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:49:23.452 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:23.450+0000 7f7dd8257180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:49:23.531 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:23.530+0000 7fd239592180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:49:23.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.586+0000 7f4afd39a700 1 -- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4adc0670e0 msgr2=0x7f4adc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:23.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.586+0000 7f4afd39a700 1 --2- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4adc0670e0 0x7f4adc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:23.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.591+0000 7f4afeb9d700 1 -- 172.21.15.55:0/2587924664 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4af806dd20 con 0x7f4af806a710 2023-10-06T03:49:23.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.592+0000 7f4aedffb700 1 -- 172.21.15.55:0/2587924664 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v94) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4af4056560 con 0x7f4af806a710 2023-10-06T03:49:23.593 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:49:23.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.594+0000 7f4afeb9d700 1 -- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4adc0670e0 msgr2=0x7f4adc069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:23.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.594+0000 7f4afeb9d700 1 --2- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4adc0670e0 0x7f4adc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.594+0000 7f4afeb9d700 1 -- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4af806a710 msgr2=0x7f4af816a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:23.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.594+0000 7f4afeb9d700 1 --2- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4af806a710 0x7f4af816a140 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4af806ce80 tx=0x7f4af401a000 comp rx=0 tx=0).stop 2023-10-06T03:49:23.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.594+0000 7f4afeb9d700 1 -- 172.21.15.55:0/2587924664 shutdown_connections 2023-10-06T03:49:23.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.594+0000 7f4afeb9d700 1 --2- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4adc0670e0 0x7f4adc069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.594+0000 7f4afeb9d700 1 --2- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4af8153900 0x7f4af80aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.594+0000 7f4afeb9d700 1 --2- 172.21.15.55:0/2587924664 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4af806a710 0x7f4af816a140 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.594+0000 7f4afeb9d700 1 --2- 172.21.15.55:0/2587924664 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4af80c9db0 0x7f4af8144fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.594+0000 7f4afeb9d700 1 -- 172.21.15.55:0/2587924664 >> 172.21.15.55:0/2587924664 conn(0x7f4af80cc1e0 msgr2=0x7f4af80c1360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:23.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.594+0000 7f4afeb9d700 1 -- 172.21.15.55:0/2587924664 shutdown_connections 2023-10-06T03:49:23.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.594+0000 7f4afeb9d700 1 -- 172.21.15.55:0/2587924664 wait complete. 2023-10-06T03:49:23.615 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:49:23.676 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:23.674+0000 7f7dd8257180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:49:23.771 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:23.769+0000 7fd239592180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:49:23.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.809+0000 7f87f199d700 1 Processor -- start 2023-10-06T03:49:23.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.809+0000 7f87f199d700 1 -- start start 2023-10-06T03:49:23.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.811+0000 7f87f199d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f87ec0b16d0 0x7f87ec0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:23.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.811+0000 7f87f199d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec0b2080 con 0x7f87ec0b4e70 2023-10-06T03:49:23.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.811+0000 7f87f199d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec0cfb00 con 0x7f87ec0b33b0 2023-10-06T03:49:23.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.811+0000 7f87f199d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ec0b5c40 con 0x7f87ec0b16d0 2023-10-06T03:49:23.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.811+0000 7f87f099b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f87ec0b4e70 0x7f87ec0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42768/0 (socket says 172.21.15.55:42768) 2023-10-06T03:49:23.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.811+0000 7f87f099b700 1 -- 172.21.15.55:0/1547089060 learned_addr learned my addr 172.21.15.55:0/1547089060 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:23.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.811+0000 7f87ebfff700 1 --2- 172.21.15.55:0/1547089060 >> v2:172.21.15.163:3300/0 conn(0x7f87ec0b16d0 0x7f87ec0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.811+0000 7f87eb7fe700 1 -- 172.21.15.55:0/1547089060 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755118592 0 0) 0x7f87ec0b2080 con 0x7f87ec0b4e70 2023-10-06T03:49:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.811+0000 7f87eb7fe700 1 -- 172.21.15.55:0/1547089060 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87dc003570 con 0x7f87ec0b4e70 2023-10-06T03:49:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.812+0000 7f87ebfff700 1 -- 172.21.15.55:0/1547089060 >> v1:172.21.15.163:6789/0 conn(0x7f87ec0b33b0 legacy=0x7f87ec0b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.812+0000 7f87eb7fe700 1 -- 172.21.15.55:0/1547089060 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773976768 0 0) 0x7f87ec0cfb00 con 0x7f87ec0b33b0 2023-10-06T03:49:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.812+0000 7f87ebfff700 1 -- 172.21.15.55:0/1547089060 >> v1:172.21.15.55:6789/0 conn(0x7f87ec0b4e70 legacy=0x7f87ec0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.812+0000 7f87ebfff700 1 -- 172.21.15.55:0/1547089060 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec0b5f20 con 0x7f87ec0b16d0 2023-10-06T03:49:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.812+0000 7f87ebfff700 1 --2- 172.21.15.55:0/1547089060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0b16d0 0x7f87ec0b1b40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f87d80098e0 tx=0x7f87d80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80591615b00fb1e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:23.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.812+0000 7f87eb7fe700 1 -- 172.21.15.55:0/1547089060 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8011630 con 0x7f87ec0b16d0 2023-10-06T03:49:23.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.812+0000 7f87eb7fe700 1 -- 172.21.15.55:0/1547089060 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d8011790 con 0x7f87ec0b16d0 2023-10-06T03:49:23.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.812+0000 7f87eb7fe700 1 -- 172.21.15.55:0/1547089060 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8011b30 con 0x7f87ec0b16d0 2023-10-06T03:49:23.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.813+0000 7f87f199d700 1 -- 172.21.15.55:0/1547089060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0b16d0 msgr2=0x7f87ec0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:23.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.813+0000 7f87f199d700 1 --2- 172.21.15.55:0/1547089060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0b16d0 0x7f87ec0b1b40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f87d80098e0 tx=0x7f87d80091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.813+0000 7f87f199d700 1 -- 172.21.15.55:0/1547089060 shutdown_connections 2023-10-06T03:49:23.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.813+0000 7f87f199d700 1 --2- 172.21.15.55:0/1547089060 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0b16d0 0x7f87ec0b1b40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.813+0000 7f87f199d700 1 -- 172.21.15.55:0/1547089060 >> 172.21.15.55:0/1547089060 conn(0x7f87ec0d10c0 msgr2=0x7f87ec0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:23.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.813+0000 7f87f199d700 1 -- 172.21.15.55:0/1547089060 shutdown_connections 2023-10-06T03:49:23.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.813+0000 7f87f199d700 1 -- 172.21.15.55:0/1547089060 wait complete. 2023-10-06T03:49:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.815+0000 7f87f199d700 1 Processor -- start 2023-10-06T03:49:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.815+0000 7f87f199d700 1 -- start start 2023-10-06T03:49:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.816+0000 7f87f199d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ec0b16d0 0x7f87ec0c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.816+0000 7f87f199d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0b33b0 0x7f87ec0c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.816+0000 7f87f199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87ec0b4e70 0x7f87ec16a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.816+0000 7f87f199d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec156a40 con 0x7f87ec0b16d0 2023-10-06T03:49:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.816+0000 7f87f199d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec15a8c0 con 0x7f87ec0b33b0 2023-10-06T03:49:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.816+0000 7f87f199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec0c5c90 con 0x7f87ec0b4e70 2023-10-06T03:49:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.816+0000 7f87f099b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ec0b16d0 0x7f87ec0c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.816+0000 7f87f099b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ec0b16d0 0x7f87ec0c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48914/0 (socket says 172.21.15.55:48914) 2023-10-06T03:49:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.816+0000 7f87ebfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0b33b0 0x7f87ec0c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:23.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.816+0000 7f87f099b700 1 -- 172.21.15.55:0/240688170 learned_addr learned my addr 172.21.15.55:0/240688170 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:23.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.816+0000 7f87f119c700 1 --2- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87ec0b4e70 0x7f87ec16a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:23.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.817+0000 7f87f099b700 1 -- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87ec0b4e70 msgr2=0x7f87ec16a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:23.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.817+0000 7f87f099b700 1 --2- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87ec0b4e70 0x7f87ec16a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.817+0000 7f87f099b700 1 -- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0b33b0 msgr2=0x7f87ec0c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:23.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.817+0000 7f87f099b700 1 --2- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0b33b0 0x7f87ec0c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:23.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.817+0000 7f87f099b700 1 -- 172.21.15.55:0/240688170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d8009420 con 0x7f87ec0b16d0 2023-10-06T03:49:23.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.817+0000 7f87ebfff700 1 --2- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0b33b0 0x7f87ec0c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:23.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.817+0000 7f87f099b700 1 --2- 172.21.15.55:0/240688170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ec0b16d0 0x7f87ec0c7d60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f87e000c7e0 tx=0x7f87e0007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:23.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.817+0000 7f87e8ff9700 1 -- 172.21.15.55:0/240688170 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0007360 con 0x7f87ec0b16d0 2023-10-06T03:49:23.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.817+0000 7f87e8ff9700 1 -- 172.21.15.55:0/240688170 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e00074c0 con 0x7f87ec0b16d0 2023-10-06T03:49:23.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.817+0000 7f87e8ff9700 1 -- 172.21.15.55:0/240688170 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e0004830 con 0x7f87ec0b16d0 2023-10-06T03:49:23.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.817+0000 7f87f199d700 1 -- 172.21.15.55:0/240688170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec0c5f70 con 0x7f87ec0b16d0 2023-10-06T03:49:23.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.817+0000 7f87f199d700 1 -- 172.21.15.55:0/240688170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec0c6530 con 0x7f87ec0b16d0 2023-10-06T03:49:23.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.818+0000 7f87f199d700 1 -- 172.21.15.55:0/240688170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec088990 con 0x7f87ec0b16d0 2023-10-06T03:49:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.819+0000 7f87e8ff9700 1 -- 172.21.15.55:0/240688170 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f87e0004ba0 con 0x7f87ec0b16d0 2023-10-06T03:49:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.820+0000 7f87e8ff9700 1 --2- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f87d00670e0 0x7f87d0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.820+0000 7f87e8ff9700 1 -- 172.21.15.55:0/240688170 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f87e0084b10 con 0x7f87ec0b16d0 2023-10-06T03:49:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.820+0000 7f87ebfff700 1 -- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f87d00670e0 msgr2=0x7f87d0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.820+0000 7f87ebfff700 1 --2- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f87d00670e0 0x7f87d0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:23.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:23.828+0000 7f87e8ff9700 1 -- 172.21.15.55:0/240688170 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87e00516f0 con 0x7f87ec0b16d0 2023-10-06T03:49:23.854 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:23.851+0000 7f7dd8257180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:49:23.948 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:23.946+0000 7f7dd8257180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:49:23.960 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:23.959+0000 7fd239592180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:49:24.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.021+0000 7f87ebfff700 1 -- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f87d00670e0 msgr2=0x7f87d0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:24.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.021+0000 7f87ebfff700 1 --2- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f87d00670e0 0x7f87d0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:24.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.029+0000 7f87f199d700 1 -- 172.21.15.55:0/240688170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87ec0b5780 con 0x7f87ec0b16d0 2023-10-06T03:49:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.030+0000 7f87e8ff9700 1 -- 172.21.15.55:0/240688170 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87e0054d10 con 0x7f87ec0b16d0 2023-10-06T03:49:24.031 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:49:24.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.032+0000 7f87f199d700 1 -- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f87d00670e0 msgr2=0x7f87d0069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:24.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.032+0000 7f87f199d700 1 --2- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f87d00670e0 0x7f87d0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.032+0000 7f87f199d700 1 -- 172.21.15.55:0/240688170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ec0b16d0 msgr2=0x7f87ec0c7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:24.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.032+0000 7f87f199d700 1 --2- 172.21.15.55:0/240688170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ec0b16d0 0x7f87ec0c7d60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f87e000c7e0 tx=0x7f87e0007d50 comp rx=0 tx=0).stop 2023-10-06T03:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.032+0000 7f87f199d700 1 -- 172.21.15.55:0/240688170 shutdown_connections 2023-10-06T03:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.032+0000 7f87f199d700 1 --2- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f87d00670e0 0x7f87d0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.032+0000 7f87f199d700 1 --2- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f87ec0b4e70 0x7f87ec16a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.032+0000 7f87f199d700 1 --2- 172.21.15.55:0/240688170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f87ec0b16d0 0x7f87ec0c7d60 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.032+0000 7f87f199d700 1 --2- 172.21.15.55:0/240688170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f87ec0b33b0 0x7f87ec0c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.032+0000 7f87f199d700 1 -- 172.21.15.55:0/240688170 >> 172.21.15.55:0/240688170 conn(0x7f87ec0d10c0 msgr2=0x7f87ec0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.032+0000 7f87f199d700 1 -- 172.21.15.55:0/240688170 shutdown_connections 2023-10-06T03:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.032+0000 7f87f199d700 1 -- 172.21.15.55:0/240688170 wait complete. 2023-10-06T03:49:24.053 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:49:24.059 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:24.057+0000 7fd239592180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:49:24.110 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:24.108+0000 7f7dd8257180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:49:24.210 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:24.208+0000 7f7dd8257180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:49:24.227 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:24.226+0000 7fd239592180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:49:24.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.247+0000 7fda0c9ff700 1 Processor -- start 2023-10-06T03:49:24.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.247+0000 7fda0c9ff700 1 -- start start 2023-10-06T03:49:24.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.249+0000 7fda0c9ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fda0806a9c0 0x7fda0806b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:24.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.249+0000 7fda0c9ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda080ac450 con 0x7fda0806b160 2023-10-06T03:49:24.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.249+0000 7fda0c9ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda08069d00 con 0x7fda080637d0 2023-10-06T03:49:24.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.249+0000 7fda0c9ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda080693d0 con 0x7fda0806a9c0 2023-10-06T03:49:24.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.249+0000 7fda0659c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fda0806b160 0x7fda08068cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42776/0 (socket says 172.21.15.55:42776) 2023-10-06T03:49:24.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.249+0000 7fda0659c700 1 -- 172.21.15.55:0/4034663140 learned_addr learned my addr 172.21.15.55:0/4034663140 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:24.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.249+0000 7fda06d9d700 1 --2- 172.21.15.55:0/4034663140 >> v2:172.21.15.163:3300/0 conn(0x7fda0806a9c0 0x7fda0806b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:24.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.249+0000 7fda0559a700 1 -- 172.21.15.55:0/4034663140 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344608808 0 0) 0x7fda080ac450 con 0x7fda0806b160 2023-10-06T03:49:24.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.250+0000 7fda0559a700 1 -- 172.21.15.55:0/4034663140 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9f0003570 con 0x7fda0806b160 2023-10-06T03:49:24.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.250+0000 7fda0559a700 1 -- 172.21.15.55:0/4034663140 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183958397 0 0) 0x7fda08069d00 con 0x7fda080637d0 2023-10-06T03:49:24.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.250+0000 7fda0559a700 1 -- 172.21.15.55:0/4034663140 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda080ac450 con 0x7fda080637d0 2023-10-06T03:49:24.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.250+0000 7fda06d9d700 1 -- 172.21.15.55:0/4034663140 >> v1:172.21.15.163:6789/0 conn(0x7fda080637d0 legacy=0x7fda0806a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:24.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.250+0000 7fda06d9d700 1 -- 172.21.15.55:0/4034663140 >> v1:172.21.15.55:6789/0 conn(0x7fda0806b160 legacy=0x7fda08068cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:24.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.250+0000 7fda06d9d700 1 -- 172.21.15.55:0/4034663140 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda08068830 con 0x7fda0806a9c0 2023-10-06T03:49:24.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.250+0000 7fda06d9d700 1 --2- 172.21.15.55:0/4034663140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0806a9c0 0x7fda0806b530 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fda00009e60 tx=0x7fda00009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=235b98e897b9d446 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:24.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.250+0000 7fda0559a700 1 -- 172.21.15.55:0/4034663140 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0000e070 con 0x7fda0806a9c0 2023-10-06T03:49:24.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.250+0000 7fda0559a700 1 -- 172.21.15.55:0/4034663140 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0000bab0 con 0x7fda0806a9c0 2023-10-06T03:49:24.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.250+0000 7fda0559a700 1 -- 172.21.15.55:0/4034663140 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0000be50 con 0x7fda0806a9c0 2023-10-06T03:49:24.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.251+0000 7fda0c9ff700 1 -- 172.21.15.55:0/4034663140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0806a9c0 msgr2=0x7fda0806b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:24.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.251+0000 7fda0c9ff700 1 --2- 172.21.15.55:0/4034663140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0806a9c0 0x7fda0806b530 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fda00009e60 tx=0x7fda00009780 comp rx=0 tx=0).stop 2023-10-06T03:49:24.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.251+0000 7fda0c9ff700 1 -- 172.21.15.55:0/4034663140 shutdown_connections 2023-10-06T03:49:24.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.251+0000 7fda0c9ff700 1 --2- 172.21.15.55:0/4034663140 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0806a9c0 0x7fda0806b530 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.251+0000 7fda0c9ff700 1 -- 172.21.15.55:0/4034663140 >> 172.21.15.55:0/4034663140 conn(0x7fda0806ded0 msgr2=0x7fda0806e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:24.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.251+0000 7fda0c9ff700 1 -- 172.21.15.55:0/4034663140 shutdown_connections 2023-10-06T03:49:24.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.251+0000 7fda0c9ff700 1 -- 172.21.15.55:0/4034663140 wait complete. 2023-10-06T03:49:24.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.253+0000 7fda0c9ff700 1 Processor -- start 2023-10-06T03:49:24.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.253+0000 7fda0c9ff700 1 -- start start 2023-10-06T03:49:24.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.254+0000 7fda0c9ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda080637d0 0x7fda081455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:24.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.254+0000 7fda0c9ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0806a9c0 0x7fda081582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:24.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.254+0000 7fda0c9ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda0806b160 0x7fda08158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:24.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.254+0000 7fda0c9ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda08157e30 con 0x7fda0806b160 2023-10-06T03:49:24.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.254+0000 7fda0c9ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda08155f40 con 0x7fda0806a9c0 2023-10-06T03:49:24.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.254+0000 7fda0c9ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda08158ec0 con 0x7fda080637d0 2023-10-06T03:49:24.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.254+0000 7fda06d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda0806b160 0x7fda08158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:24.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.254+0000 7fda0659c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda080637d0 0x7fda081455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:24.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.254+0000 7fda06d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda0806b160 0x7fda08158820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48922/0 (socket says 172.21.15.55:48922) 2023-10-06T03:49:24.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.254+0000 7fda06d9d700 1 -- 172.21.15.55:0/1248350474 learned_addr learned my addr 172.21.15.55:0/1248350474 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:24.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.254+0000 7fda05d9b700 1 --2- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0806a9c0 0x7fda081582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:24.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.255+0000 7fda0659c700 1 -- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0806a9c0 msgr2=0x7fda081582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:24.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.255+0000 7fda0659c700 1 --2- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0806a9c0 0x7fda081582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.255+0000 7fda0659c700 1 -- 172.21.15.55:0/1248350474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda0806b160 msgr2=0x7fda08158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:24.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.255+0000 7fda0659c700 1 --2- 172.21.15.55:0/1248350474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda0806b160 0x7fda08158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.255+0000 7fda0659c700 1 -- 172.21.15.55:0/1248350474 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda00009a20 con 0x7fda080637d0 2023-10-06T03:49:24.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.255+0000 7fda06d9d700 1 --2- 172.21.15.55:0/1248350474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda0806b160 0x7fda08158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:24.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.255+0000 7fda0659c700 1 --2- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda080637d0 0x7fda081455a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800c580 tx=0x7fd9f8007d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:24.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.255+0000 7fd9f67fc700 1 -- 172.21.15.55:0/1248350474 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8005b30 con 0x7fda080637d0 2023-10-06T03:49:24.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.255+0000 7fd9f67fc700 1 -- 172.21.15.55:0/1248350474 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8005c90 con 0x7fda080637d0 2023-10-06T03:49:24.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.255+0000 7fda0c9ff700 1 -- 172.21.15.55:0/1248350474 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda081591a0 con 0x7fda080637d0 2023-10-06T03:49:24.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.255+0000 7fd9f67fc700 1 -- 172.21.15.55:0/1248350474 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8007320 con 0x7fda080637d0 2023-10-06T03:49:24.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.255+0000 7fda0c9ff700 1 -- 172.21.15.55:0/1248350474 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda08159760 con 0x7fda080637d0 2023-10-06T03:49:24.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.256+0000 7fda0c9ff700 1 -- 172.21.15.55:0/1248350474 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda08155d10 con 0x7fda080637d0 2023-10-06T03:49:24.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.257+0000 7fd9f67fc700 1 -- 172.21.15.55:0/1248350474 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fd9f801b3b0 con 0x7fda080637d0 2023-10-06T03:49:24.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.258+0000 7fd9f67fc700 1 --2- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9e8067110 0x7fd9e80695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:24.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.259+0000 7fd9f67fc700 1 -- 172.21.15.55:0/1248350474 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6708+0+0 (secure 0 0 0) 0x7fd9f80840c0 con 0x7fda080637d0 2023-10-06T03:49:24.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.259+0000 7fda05d9b700 1 -- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9e8067110 msgr2=0x7fd9e80695c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:24.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.259+0000 7fda05d9b700 1 --2- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9e8067110 0x7fd9e80695c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:24.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.263+0000 7fd9f67fc700 1 -- 172.21.15.55:0/1248350474 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda08155d10 con 0x7fda080637d0 2023-10-06T03:49:24.335 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:24.334+0000 7fd239592180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:49:24.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.460+0000 7fda05d9b700 1 -- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9e8067110 msgr2=0x7fd9e80695c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:24.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.460+0000 7fda05d9b700 1 --2- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9e8067110 0x7fd9e80695c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:24.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.465+0000 7fda0c9ff700 1 -- 172.21.15.55:0/1248350474 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda08155d10 con 0x7fda080637d0 2023-10-06T03:49:24.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.466+0000 7fd9f67fc700 1 -- 172.21.15.55:0/1248350474 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9f8017020 con 0x7fda080637d0 2023-10-06T03:49:24.468 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T03:49:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.468+0000 7fda0c9ff700 1 -- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9e8067110 msgr2=0x7fd9e80695c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.468+0000 7fda0c9ff700 1 --2- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9e8067110 0x7fd9e80695c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.468+0000 7fda0c9ff700 1 -- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda080637d0 msgr2=0x7fda081455a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.468+0000 7fda0c9ff700 1 --2- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda080637d0 0x7fda081455a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800c580 tx=0x7fd9f8007d70 comp rx=0 tx=0).stop 2023-10-06T03:49:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.468+0000 7fda0c9ff700 1 -- 172.21.15.55:0/1248350474 shutdown_connections 2023-10-06T03:49:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.468+0000 7fda0c9ff700 1 --2- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9e8067110 0x7fd9e80695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.468+0000 7fda0c9ff700 1 --2- 172.21.15.55:0/1248350474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda0806b160 0x7fda08158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.468+0000 7fda0c9ff700 1 --2- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda080637d0 0x7fda081455a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.468+0000 7fda0c9ff700 1 --2- 172.21.15.55:0/1248350474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda0806a9c0 0x7fda081582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.468+0000 7fda0c9ff700 1 -- 172.21.15.55:0/1248350474 >> 172.21.15.55:0/1248350474 conn(0x7fda0806ded0 msgr2=0x7fda080ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:24.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.469+0000 7fda0c9ff700 1 -- 172.21.15.55:0/1248350474 shutdown_connections 2023-10-06T03:49:24.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.469+0000 7fda0c9ff700 1 -- 172.21.15.55:0/1248350474 wait complete. 2023-10-06T03:49:24.488 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:49:24.694 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:24.692+0000 7f7dd8257180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:49:24.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.722+0000 7f54e3fff700 1 Processor -- start 2023-10-06T03:49:24.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.722+0000 7f54e3fff700 1 -- start start 2023-10-06T03:49:24.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.723+0000 7f54e3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40b1b40 0x7f54e40b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:24.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.723+0000 7f54e3fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e40cfb00 con 0x7f54e40b16d0 2023-10-06T03:49:24.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.723+0000 7f54e3fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e40b5c40 con 0x7f54e40b4e70 2023-10-06T03:49:24.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.723+0000 7f54e3fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40b5f20 con 0x7f54e40b1b40 2023-10-06T03:49:24.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.724+0000 7f54e2ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f54e40b4e70 0x7f54e40ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:35030/0 (socket says 172.21.15.55:35030) 2023-10-06T03:49:24.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.724+0000 7f54e2ffd700 1 -- 172.21.15.55:0/4134915399 learned_addr learned my addr 172.21.15.55:0/4134915399 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:24.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.724+0000 7f54e27fc700 1 --1- 172.21.15.55:0/4134915399 >> v1:172.21.15.55:6789/0 conn(0x7f54e40b16d0 0x7f54e40b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42784/0 (socket says 172.21.15.55:42784) 2023-10-06T03:49:24.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.724+0000 7f54e37fe700 1 --2- 172.21.15.55:0/4134915399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40b1b40 0x7f54e40b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:24.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.724+0000 7f54e1ffb700 1 -- 172.21.15.55:0/4134915399 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808367209 0 0) 0x7f54e40b5c40 con 0x7f54e40b4e70 2023-10-06T03:49:24.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.724+0000 7f54e1ffb700 1 -- 172.21.15.55:0/4134915399 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54cc003570 con 0x7f54e40b4e70 2023-10-06T03:49:24.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.724+0000 7f54e1ffb700 1 -- 172.21.15.55:0/4134915399 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409686355 0 0) 0x7f54e40cfb00 con 0x7f54e40b16d0 2023-10-06T03:49:24.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.724+0000 7f54e37fe700 1 -- 172.21.15.55:0/4134915399 >> v1:172.21.15.163:6789/0 conn(0x7f54e40b4e70 legacy=0x7f54e40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:24.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.724+0000 7f54e37fe700 1 -- 172.21.15.55:0/4134915399 >> v1:172.21.15.55:6789/0 conn(0x7f54e40b16d0 legacy=0x7f54e40b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:24.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.724+0000 7f54e37fe700 1 -- 172.21.15.55:0/4134915399 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40b6200 con 0x7f54e40b1b40 2023-10-06T03:49:24.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.724+0000 7f54e37fe700 1 --2- 172.21.15.55:0/4134915399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40b1b40 0x7f54e40b3ac0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009e90 tx=0x7f54dc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd40df7b671f9b84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:24.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.725+0000 7f54e1ffb700 1 -- 172.21.15.55:0/4134915399 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc00e070 con 0x7f54e40b1b40 2023-10-06T03:49:24.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.725+0000 7f54e1ffb700 1 -- 172.21.15.55:0/4134915399 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc00bc30 con 0x7f54e40b1b40 2023-10-06T03:49:24.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.725+0000 7f54e1ffb700 1 -- 172.21.15.55:0/4134915399 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc012430 con 0x7f54e40b1b40 2023-10-06T03:49:24.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.725+0000 7f54e3fff700 1 -- 172.21.15.55:0/4134915399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40b1b40 msgr2=0x7f54e40b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:24.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.725+0000 7f54e3fff700 1 --2- 172.21.15.55:0/4134915399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40b1b40 0x7f54e40b3ac0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009e90 tx=0x7f54dc009780 comp rx=0 tx=0).stop 2023-10-06T03:49:24.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.725+0000 7f54e3fff700 1 -- 172.21.15.55:0/4134915399 shutdown_connections 2023-10-06T03:49:24.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.725+0000 7f54e3fff700 1 --2- 172.21.15.55:0/4134915399 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40b1b40 0x7f54e40b3ac0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.725+0000 7f54e3fff700 1 -- 172.21.15.55:0/4134915399 >> 172.21.15.55:0/4134915399 conn(0x7f54e40d10c0 msgr2=0x7f54e40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:24.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.726+0000 7f54e3fff700 1 -- 172.21.15.55:0/4134915399 shutdown_connections 2023-10-06T03:49:24.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.726+0000 7f54e3fff700 1 -- 172.21.15.55:0/4134915399 wait complete. 2023-10-06T03:49:24.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.727+0000 7f54e3fff700 1 Processor -- start 2023-10-06T03:49:24.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.727+0000 7f54e3fff700 1 -- start start 2023-10-06T03:49:24.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.728+0000 7f54e3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e40b16d0 0x7f54e40c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:24.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.728+0000 7f54e3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40b1b40 0x7f54e415e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:24.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.728+0000 7f54e3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54e40b4e70 0x7f54e414bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:24.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.728+0000 7f54e3fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e416ab70 con 0x7f54e40b4e70 2023-10-06T03:49:24.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.728+0000 7f54e3fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e416a890 con 0x7f54e40b16d0 2023-10-06T03:49:24.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.728+0000 7f54e3fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40c6500 con 0x7f54e40b1b40 2023-10-06T03:49:24.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.728+0000 7f54e37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54e40b4e70 0x7f54e414bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:24.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.729+0000 7f54e2ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e40b16d0 0x7f54e40c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:24.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.729+0000 7f54e37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54e40b4e70 0x7f54e414bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48938/0 (socket says 172.21.15.55:48938) 2023-10-06T03:49:24.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.729+0000 7f54e37fe700 1 -- 172.21.15.55:0/618857724 learned_addr learned my addr 172.21.15.55:0/618857724 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:24.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.729+0000 7f54e27fc700 1 --2- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40b1b40 0x7f54e415e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:24.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.729+0000 7f54e37fe700 1 -- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40b1b40 msgr2=0x7f54e415e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:24.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.729+0000 7f54e37fe700 1 --2- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40b1b40 0x7f54e415e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.729+0000 7f54e37fe700 1 -- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e40b16d0 msgr2=0x7f54e40c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:24.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.729+0000 7f54e37fe700 1 --2- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e40b16d0 0x7f54e40c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.729+0000 7f54e37fe700 1 -- 172.21.15.55:0/618857724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc009a20 con 0x7f54e40b4e70 2023-10-06T03:49:24.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.729+0000 7f54e37fe700 1 --2- 172.21.15.55:0/618857724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54e40b4e70 0x7f54e414bc50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f54e4070dd0 tx=0x7f54dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:24.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.729+0000 7f54caffd700 1 -- 172.21.15.55:0/618857724 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc007000 con 0x7f54e40b4e70 2023-10-06T03:49:24.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.730+0000 7f54caffd700 1 -- 172.21.15.55:0/618857724 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc0263f0 con 0x7f54e40b4e70 2023-10-06T03:49:24.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.730+0000 7f54caffd700 1 -- 172.21.15.55:0/618857724 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0267d0 con 0x7f54e40b4e70 2023-10-06T03:49:24.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.730+0000 7f54e3fff700 1 -- 172.21.15.55:0/618857724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e40c67e0 con 0x7f54e40b4e70 2023-10-06T03:49:24.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.730+0000 7f54e3fff700 1 -- 172.21.15.55:0/618857724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e415ed20 con 0x7f54e40b4e70 2023-10-06T03:49:24.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.731+0000 7f54e3fff700 1 -- 172.21.15.55:0/618857724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e40c6360 con 0x7f54e40b4e70 2023-10-06T03:49:24.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.732+0000 7f54caffd700 1 -- 172.21.15.55:0/618857724 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f54dc026c00 con 0x7f54e40b4e70 2023-10-06T03:49:24.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.733+0000 7f54caffd700 1 --2- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f54d00670e0 0x7f54d0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:24.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.733+0000 7f54caffd700 1 -- 172.21.15.55:0/618857724 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f54dc086980 con 0x7f54e40b4e70 2023-10-06T03:49:24.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.733+0000 7f54e2ffd700 1 -- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f54d00670e0 msgr2=0x7f54d0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:24.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.733+0000 7f54e2ffd700 1 --2- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f54d00670e0 0x7f54d0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:24.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.737+0000 7f54caffd700 1 -- 172.21.15.55:0/618857724 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54e40c6360 con 0x7f54e40b4e70 2023-10-06T03:49:24.790 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:24.787+0000 7f7dd8257180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:49:24.845 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:24.842+0000 7fd239592180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:49:24.879 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:24.877+0000 7f7dd8257180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:49:24.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.934+0000 7f54e2ffd700 1 -- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f54d00670e0 msgr2=0x7f54d0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:24.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.934+0000 7f54e2ffd700 1 --2- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f54d00670e0 0x7f54d0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:24.943 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:24.941+0000 7fd239592180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:49:24.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.943+0000 7f54e3fff700 1 -- 172.21.15.55:0/618857724 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f54e40c6360 con 0x7f54e40b4e70 2023-10-06T03:49:24.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.944+0000 7f54caffd700 1 -- 172.21.15.55:0/618857724 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f54e40c6360 con 0x7f54e40b4e70 2023-10-06T03:49:24.946 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T03:49:24.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.946+0000 7f54e3fff700 1 -- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f54d00670e0 msgr2=0x7f54d0069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:24.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.946+0000 7f54e3fff700 1 --2- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f54d00670e0 0x7f54d0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.946+0000 7f54e3fff700 1 -- 172.21.15.55:0/618857724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54e40b4e70 msgr2=0x7f54e414bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:24.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.946+0000 7f54e3fff700 1 --2- 172.21.15.55:0/618857724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54e40b4e70 0x7f54e414bc50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f54e4070dd0 tx=0x7f54dc019000 comp rx=0 tx=0).stop 2023-10-06T03:49:24.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.946+0000 7f54e3fff700 1 -- 172.21.15.55:0/618857724 shutdown_connections 2023-10-06T03:49:24.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.946+0000 7f54e3fff700 1 --2- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f54d00670e0 0x7f54d0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.946+0000 7f54e3fff700 1 --2- 172.21.15.55:0/618857724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54e40b4e70 0x7f54e414bc50 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.946+0000 7f54e3fff700 1 --2- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f54e40b16d0 0x7f54e40c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.946+0000 7f54e3fff700 1 --2- 172.21.15.55:0/618857724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54e40b1b40 0x7f54e415e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:24.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.946+0000 7f54e3fff700 1 -- 172.21.15.55:0/618857724 >> 172.21.15.55:0/618857724 conn(0x7f54e40d10c0 msgr2=0x7f54e415c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:24.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.946+0000 7f54e3fff700 1 -- 172.21.15.55:0/618857724 shutdown_connections 2023-10-06T03:49:24.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:24.946+0000 7f54e3fff700 1 -- 172.21.15.55:0/618857724 wait complete. 2023-10-06T03:49:24.967 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:49:25.040 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:25.038+0000 7fd239592180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:49:25.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.154+0000 7f7047fff700 1 Processor -- start 2023-10-06T03:49:25.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.154+0000 7f7047fff700 1 -- start start 2023-10-06T03:49:25.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.155+0000 7f7047fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7048159cc0 0x7f70480b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:25.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.155+0000 7f7047fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f70480cf6c0 0x7f7048163280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:25.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.155+0000 7f7047fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70480b5b40 con 0x7f70480b2920 2023-10-06T03:49:25.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.155+0000 7f7047fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70480ce260 con 0x7f70480cf6c0 2023-10-06T03:49:25.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.155+0000 7f7047fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70480b5e20 con 0x7f7048159cc0 2023-10-06T03:49:25.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.156+0000 7f70467fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f70480b2920 0x7f704815a090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42796/0 (socket says 172.21.15.55:42796) 2023-10-06T03:49:25.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.156+0000 7f70467fc700 1 -- 172.21.15.55:0/2795388721 learned_addr learned my addr 172.21.15.55:0/2795388721 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:25.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.156+0000 7f7046ffd700 1 --2- 172.21.15.55:0/2795388721 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7048159cc0 0x7f70480b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:25.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.156+0000 7f70477fe700 1 --2- 172.21.15.55:0/2795388721 >> v2:172.21.15.163:3300/0 conn(0x7f70480cf6c0 0x7f7048163280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:25.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.156+0000 7f7045ffb700 1 -- 172.21.15.55:0/2795388721 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134227243 0 0) 0x7f70480b5b40 con 0x7f70480b2920 2023-10-06T03:49:25.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.156+0000 7f7045ffb700 1 -- 172.21.15.55:0/2795388721 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7030003570 con 0x7f70480b2920 2023-10-06T03:49:25.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.156+0000 7f7046ffd700 1 -- 172.21.15.55:0/2795388721 >> v2:172.21.15.163:3300/0 conn(0x7f70480cf6c0 msgr2=0x7f7048163280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:25.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.156+0000 7f7046ffd700 1 --2- 172.21.15.55:0/2795388721 >> v2:172.21.15.163:3300/0 conn(0x7f70480cf6c0 0x7f7048163280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.156+0000 7f7046ffd700 1 -- 172.21.15.55:0/2795388721 >> v1:172.21.15.55:6789/0 conn(0x7f70480b2920 legacy=0x7f704815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:25.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.156+0000 7f7046ffd700 1 -- 172.21.15.55:0/2795388721 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70480b6100 con 0x7f7048159cc0 2023-10-06T03:49:25.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.156+0000 7f70477fe700 1 --2- 172.21.15.55:0/2795388721 >> v2:172.21.15.163:3300/0 conn(0x7f70480cf6c0 0x7f7048163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:25.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.157+0000 7f7046ffd700 1 --2- 172.21.15.55:0/2795388721 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7048159cc0 0x7f70480b5530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f70380098d0 tx=0x7f7038009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4f49be07a7cbdcb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:25.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.157+0000 7f7045ffb700 1 -- 172.21.15.55:0/2795388721 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038005c10 con 0x7f7048159cc0 2023-10-06T03:49:25.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.157+0000 7f7045ffb700 1 -- 172.21.15.55:0/2795388721 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7038005d70 con 0x7f7048159cc0 2023-10-06T03:49:25.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.157+0000 7f7045ffb700 1 -- 172.21.15.55:0/2795388721 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038006280 con 0x7f7048159cc0 2023-10-06T03:49:25.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.158+0000 7f7047fff700 1 -- 172.21.15.55:0/2795388721 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7048159cc0 msgr2=0x7f70480b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:25.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.158+0000 7f7047fff700 1 --2- 172.21.15.55:0/2795388721 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7048159cc0 0x7f70480b5530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f70380098d0 tx=0x7f7038009120 comp rx=0 tx=0).stop 2023-10-06T03:49:25.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.158+0000 7f7047fff700 1 -- 172.21.15.55:0/2795388721 shutdown_connections 2023-10-06T03:49:25.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.158+0000 7f7047fff700 1 --2- 172.21.15.55:0/2795388721 >> v2:172.21.15.163:3300/0 conn(0x7f70480cf6c0 0x7f7048163280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.158+0000 7f7047fff700 1 --2- 172.21.15.55:0/2795388721 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7048159cc0 0x7f70480b5530 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.158+0000 7f7047fff700 1 -- 172.21.15.55:0/2795388721 >> 172.21.15.55:0/2795388721 conn(0x7f70480d0560 msgr2=0x7f70480d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:25.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.158+0000 7f7047fff700 1 -- 172.21.15.55:0/2795388721 shutdown_connections 2023-10-06T03:49:25.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.158+0000 7f7047fff700 1 -- 172.21.15.55:0/2795388721 wait complete. 2023-10-06T03:49:25.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.159+0000 7f7047fff700 1 Processor -- start 2023-10-06T03:49:25.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.159+0000 7f7047fff700 1 -- start start 2023-10-06T03:49:25.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.160+0000 7f7047fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70480b2920 0x7f704815a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:25.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.160+0000 7f7047fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70480cf6c0 0x7f7048169f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:25.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.160+0000 7f7047fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7048159cc0 0x7f704814fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:25.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.160+0000 7f7047fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048156550 con 0x7f70480b2920 2023-10-06T03:49:25.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.160+0000 7f7047fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704816a480 con 0x7f70480cf6c0 2023-10-06T03:49:25.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.160+0000 7f7047fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7048150180 con 0x7f7048159cc0 2023-10-06T03:49:25.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.161+0000 7f7046ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70480b2920 0x7f704815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:25.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.161+0000 7f7046ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70480b2920 0x7f704815a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48942/0 (socket says 172.21.15.55:48942) 2023-10-06T03:49:25.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.161+0000 7f7046ffd700 1 -- 172.21.15.55:0/4133371044 learned_addr learned my addr 172.21.15.55:0/4133371044 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:25.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.161+0000 7f70467fc700 1 --2- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70480cf6c0 0x7f7048169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:25.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.161+0000 7f70477fe700 1 --2- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7048159cc0 0x7f704814fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:25.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.161+0000 7f7046ffd700 1 -- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7048159cc0 msgr2=0x7f704814fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:25.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.161+0000 7f7046ffd700 1 --2- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7048159cc0 0x7f704814fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.161+0000 7f7046ffd700 1 -- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70480cf6c0 msgr2=0x7f7048169f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:25.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.161+0000 7f7046ffd700 1 --2- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70480cf6c0 0x7f7048169f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.161+0000 7f7046ffd700 1 -- 172.21.15.55:0/4133371044 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040009a20 con 0x7f70480b2920 2023-10-06T03:49:25.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.161+0000 7f7046ffd700 1 --2- 172.21.15.55:0/4133371044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70480b2920 0x7f704815a090 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7038009ec0 tx=0x7f7038016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:25.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.162+0000 7f702effd700 1 -- 172.21.15.55:0/4133371044 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703801d730 con 0x7f70480b2920 2023-10-06T03:49:25.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.162+0000 7f702effd700 1 -- 172.21.15.55:0/4133371044 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703801d890 con 0x7f70480b2920 2023-10-06T03:49:25.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.162+0000 7f7047fff700 1 -- 172.21.15.55:0/4133371044 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70380093c0 con 0x7f70480b2920 2023-10-06T03:49:25.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.162+0000 7f702effd700 1 -- 172.21.15.55:0/4133371044 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703801dc50 con 0x7f70480b2920 2023-10-06T03:49:25.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.162+0000 7f7047fff700 1 -- 172.21.15.55:0/4133371044 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7048154780 con 0x7f70480b2920 2023-10-06T03:49:25.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.163+0000 7f7047fff700 1 -- 172.21.15.55:0/4133371044 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704804c3c0 con 0x7f70480b2920 2023-10-06T03:49:25.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.164+0000 7f702effd700 1 -- 172.21.15.55:0/4133371044 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f703801b480 con 0x7f70480b2920 2023-10-06T03:49:25.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.165+0000 7f702effd700 1 --2- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70340670e0 0x7f7034069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:25.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.165+0000 7f702effd700 1 -- 172.21.15.55:0/4133371044 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f7038086880 con 0x7f70480b2920 2023-10-06T03:49:25.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.165+0000 7f70467fc700 1 -- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70340670e0 msgr2=0x7f7034069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:25.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.165+0000 7f70467fc700 1 --2- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70340670e0 0x7f7034069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:25.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.169+0000 7f702effd700 1 -- 172.21.15.55:0/4133371044 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70380534a0 con 0x7f70480b2920 2023-10-06T03:49:25.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.366+0000 7f70467fc700 1 -- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70340670e0 msgr2=0x7f7034069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:25.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.366+0000 7f70467fc700 1 --2- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70340670e0 0x7f7034069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:25.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.420+0000 7f7047fff700 1 -- 172.21.15.55:0/4133371044 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70480b63e0 con 0x7f70480b2920 2023-10-06T03:49:25.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.421+0000 7f702effd700 1 -- 172.21.15.55:0/4133371044 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f7038056ac0 con 0x7f70480b2920 2023-10-06T03:49:25.424 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:25.424 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/4087100648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4087100648},{"type":"v1","addr":"172.21.15.163:6834","nonce":4087100648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2017152000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2017152000},{"type":"v1","addr":"172.21.15.163:6836","nonce":2017152000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1226557536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1226557536},{"type":"v1","addr":"172.21.15.55:6834","nonce":1226557536}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:49:03.547622+0000","modified":"2023-10-06T03:49:07.074089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6804":{"gid":6804,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/3960749421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3960749421},{"type":"v1","addr":"172.21.15.55:6835","nonce":3960749421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T03:49:25.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.425+0000 7f7047fff700 1 -- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70340670e0 msgr2=0x7f7034069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:25.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.425+0000 7f7047fff700 1 --2- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70340670e0 0x7f7034069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.425+0000 7f7047fff700 1 -- 172.21.15.55:0/4133371044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70480b2920 msgr2=0x7f704815a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:25.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.425+0000 7f7047fff700 1 --2- 172.21.15.55:0/4133371044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70480b2920 0x7f704815a090 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f7038009ec0 tx=0x7f7038016000 comp rx=0 tx=0).stop 2023-10-06T03:49:25.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.425+0000 7f7047fff700 1 -- 172.21.15.55:0/4133371044 shutdown_connections 2023-10-06T03:49:25.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.425+0000 7f7047fff700 1 --2- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f70340670e0 0x7f7034069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.425+0000 7f7047fff700 1 --2- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7048159cc0 0x7f704814fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.425+0000 7f7047fff700 1 --2- 172.21.15.55:0/4133371044 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f70480b2920 0x7f704815a090 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.425+0000 7f7047fff700 1 --2- 172.21.15.55:0/4133371044 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f70480cf6c0 0x7f7048169f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.425+0000 7f7047fff700 1 -- 172.21.15.55:0/4133371044 >> 172.21.15.55:0/4133371044 conn(0x7f70480d0560 msgr2=0x7f70480b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:25.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.425+0000 7f7047fff700 1 -- 172.21.15.55:0/4133371044 shutdown_connections 2023-10-06T03:49:25.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.425+0000 7f7047fff700 1 -- 172.21.15.55:0/4133371044 wait complete. 2023-10-06T03:49:25.429 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 44 2023-10-06T03:49:25.453 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:49:25.507 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:25.505+0000 7f7dd8257180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:49:25.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.667+0000 7f1ddf59e700 1 Processor -- start 2023-10-06T03:49:25.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.668+0000 7f1ddf59e700 1 -- start start 2023-10-06T03:49:25.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.669+0000 7f1ddf59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1de0159420 0x7f1de00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:25.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.669+0000 7f1ddf59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1de01597f0 0x7f1de00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:25.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.669+0000 7f1ddf59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1de00b11d0 con 0x7f1de015d230 2023-10-06T03:49:25.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.669+0000 7f1ddf59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de00b14b0 con 0x7f1de01597f0 2023-10-06T03:49:25.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.669+0000 7f1ddf59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de00b1790 con 0x7f1de0159420 2023-10-06T03:49:25.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.669+0000 7f1dddd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1de01597f0 0x7f1de00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:25.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.669+0000 7f1dddd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1de01597f0 0x7f1de00b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48952/0 (socket says 172.21.15.55:48952) 2023-10-06T03:49:25.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.669+0000 7f1dded9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1de015d230 0x7f1de015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42804/0 (socket says 172.21.15.55:42804) 2023-10-06T03:49:25.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.669+0000 7f1dddd9b700 1 -- 172.21.15.55:0/2324570266 learned_addr learned my addr 172.21.15.55:0/2324570266 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:25.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.669+0000 7f1dde59c700 1 --2- 172.21.15.55:0/2324570266 >> v2:172.21.15.163:3300/0 conn(0x7f1de0159420 0x7f1de00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:25.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.670+0000 7f1dddd9b700 1 -- 172.21.15.55:0/2324570266 >> v2:172.21.15.163:3300/0 conn(0x7f1de0159420 msgr2=0x7f1de00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:25.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.670+0000 7f1dddd9b700 1 --2- 172.21.15.55:0/2324570266 >> v2:172.21.15.163:3300/0 conn(0x7f1de0159420 0x7f1de00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.670+0000 7f1dddd9b700 1 -- 172.21.15.55:0/2324570266 >> v1:172.21.15.55:6789/0 conn(0x7f1de015d230 legacy=0x7f1de015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:25.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.670+0000 7f1dddd9b700 1 -- 172.21.15.55:0/2324570266 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de00b1a70 con 0x7f1de01597f0 2023-10-06T03:49:25.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.670+0000 7f1ddd59a700 1 -- 172.21.15.55:0/2324570266 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040133464 0 0) 0x7f1de00b11d0 con 0x7f1de015d230 2023-10-06T03:49:25.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.670+0000 7f1dddd9b700 1 --2- 172.21.15.55:0/2324570266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1de01597f0 0x7f1de00b2270 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1de00b2f20 tx=0x7f1dd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23a94bbff0df65a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:25.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.670+0000 7f1ddd59a700 1 -- 172.21.15.55:0/2324570266 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4009e70 con 0x7f1de01597f0 2023-10-06T03:49:25.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.670+0000 7f1ddd59a700 1 -- 172.21.15.55:0/2324570266 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd400b860 con 0x7f1de01597f0 2023-10-06T03:49:25.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.670+0000 7f1ddd59a700 1 -- 172.21.15.55:0/2324570266 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd400bc00 con 0x7f1de01597f0 2023-10-06T03:49:25.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.671+0000 7f1ddf59e700 1 -- 172.21.15.55:0/2324570266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1de01597f0 msgr2=0x7f1de00b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:25.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.671+0000 7f1ddf59e700 1 --2- 172.21.15.55:0/2324570266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1de01597f0 0x7f1de00b2270 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f1de00b2f20 tx=0x7f1dd4009250 comp rx=0 tx=0).stop 2023-10-06T03:49:25.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.671+0000 7f1ddf59e700 1 -- 172.21.15.55:0/2324570266 shutdown_connections 2023-10-06T03:49:25.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.671+0000 7f1ddf59e700 1 --2- 172.21.15.55:0/2324570266 >> v2:172.21.15.163:3300/0 conn(0x7f1de0159420 0x7f1de00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.671+0000 7f1ddf59e700 1 --2- 172.21.15.55:0/2324570266 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1de01597f0 0x7f1de00b2270 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.671+0000 7f1ddf59e700 1 -- 172.21.15.55:0/2324570266 >> 172.21.15.55:0/2324570266 conn(0x7f1de00d0660 msgr2=0x7f1de00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:25.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.671+0000 7f1ddf59e700 1 -- 172.21.15.55:0/2324570266 shutdown_connections 2023-10-06T03:49:25.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.671+0000 7f1ddf59e700 1 -- 172.21.15.55:0/2324570266 wait complete. 2023-10-06T03:49:25.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.673+0000 7f1ddf59e700 1 Processor -- start 2023-10-06T03:49:25.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.673+0000 7f1ddf59e700 1 -- start start 2023-10-06T03:49:25.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.674+0000 7f1ddf59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1de0159420 0x7f1de01534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:25.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.674+0000 7f1ddf59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1de01597f0 0x7f1de00b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:25.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.674+0000 7f1ddf59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1de015d230 0x7f1de016cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:25.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.674+0000 7f1ddf59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de00c6ce0 con 0x7f1de015d230 2023-10-06T03:49:25.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.674+0000 7f1ddf59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de00b5440 con 0x7f1de0159420 2023-10-06T03:49:25.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.674+0000 7f1ddf59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de00b4830 con 0x7f1de01597f0 2023-10-06T03:49:25.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.674+0000 7f1dded9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1de015d230 0x7f1de016cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:25.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.674+0000 7f1dded9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1de015d230 0x7f1de016cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48966/0 (socket says 172.21.15.55:48966) 2023-10-06T03:49:25.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.674+0000 7f1dded9d700 1 -- 172.21.15.55:0/1061427184 learned_addr learned my addr 172.21.15.55:0/1061427184 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:25.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.674+0000 7f1dddd9b700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1de01597f0 0x7f1de00b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:25.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.674+0000 7f1dde59c700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1de0159420 0x7f1de01534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:25.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.675+0000 7f1dded9d700 1 -- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1de01597f0 msgr2=0x7f1de00b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:25.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.675+0000 7f1dded9d700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1de01597f0 0x7f1de00b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.675+0000 7f1dded9d700 1 -- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1de0159420 msgr2=0x7f1de01534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:25.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.675+0000 7f1dded9d700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1de0159420 0x7f1de01534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.675+0000 7f1dded9d700 1 -- 172.21.15.55:0/1061427184 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd40094f0 con 0x7f1de015d230 2023-10-06T03:49:25.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.675+0000 7f1dde59c700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1de0159420 0x7f1de01534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:25.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.675+0000 7f1dded9d700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1de015d230 0x7f1de016cfd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1dd800ccc0 tx=0x7f1dd80019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:25.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.675+0000 7f1dce7fc700 1 -- 172.21.15.55:0/1061427184 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8005df0 con 0x7f1de015d230 2023-10-06T03:49:25.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.675+0000 7f1dce7fc700 1 -- 172.21.15.55:0/1061427184 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd800a670 con 0x7f1de015d230 2023-10-06T03:49:25.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.675+0000 7f1ddf59e700 1 -- 172.21.15.55:0/1061427184 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de00b4b10 con 0x7f1de015d230 2023-10-06T03:49:25.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.675+0000 7f1dce7fc700 1 -- 172.21.15.55:0/1061427184 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd800aa30 con 0x7f1de015d230 2023-10-06T03:49:25.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.675+0000 7f1ddf59e700 1 -- 172.21.15.55:0/1061427184 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de00b50d0 con 0x7f1de015d230 2023-10-06T03:49:25.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.676+0000 7f1ddf59e700 1 -- 172.21.15.55:0/1061427184 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de00b57d0 con 0x7f1de015d230 2023-10-06T03:49:25.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.678+0000 7f1dce7fc700 1 -- 172.21.15.55:0/1061427184 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f1dd80070f0 con 0x7f1de015d230 2023-10-06T03:49:25.683 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:25.680+0000 7f7dd8257180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:49:25.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.679+0000 7f1dce7fc700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1dc00671b0 0x7f1dc0069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:25.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.680+0000 7f1dce7fc700 1 -- 172.21.15.55:0/1061427184 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f1dd8085590 con 0x7f1de015d230 2023-10-06T03:49:25.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.680+0000 7f1dde59c700 1 -- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1dc00671b0 msgr2=0x7f1dc0069660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:25.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.680+0000 7f1dde59c700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1dc00671b0 0x7f1dc0069660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:25.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.682+0000 7f1dce7fc700 1 -- 172.21.15.55:0/1061427184 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1dd80521f0 con 0x7f1de015d230 2023-10-06T03:49:25.712 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:25.709+0000 7fd239592180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:49:25.780 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:25.778+0000 7f7dd8257180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:49:25.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.881+0000 7f1dde59c700 1 -- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1dc00671b0 msgr2=0x7f1dc0069660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:25.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.881+0000 7f1dde59c700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1dc00671b0 0x7f1dc0069660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:25.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.888+0000 7f1ddf59e700 1 -- 172.21.15.55:0/1061427184 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1de00b57d0 con 0x7f1de015d230 2023-10-06T03:49:25.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.889+0000 7f1dce7fc700 1 -- 172.21.15.55:0/1061427184 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+17885 (secure 0 0 0) 0x7f1de00b57d0 con 0x7f1de015d230 2023-10-06T03:49:25.892 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:25.892 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":94,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:49:19.842571+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:49:01.721643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T03:49:02.520120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3249461740":"2023-10-07T03:49:19.842520+0000","172.21.15.163:0/4224423400":"2023-10-07T03:49:19.842520+0000","172.21.15.163:6837/85191":"2023-10-07T03:49:19.842520+0000","172.21.15.163:6835/85191":"2023-10-07T03:49:19.842520+0000","172.21.15.163:0/1889998714":"2023-10-07T03:49:19.842520+0000","172.21.15.163:0/4148366955":"2023-10-07T03:49:19.842520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:49:25.893 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:25.891+0000 7fd239592180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:49:25.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.892+0000 7f1ddf59e700 1 -- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1dc00671b0 msgr2=0x7f1dc0069660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:25.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.892+0000 7f1ddf59e700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1dc00671b0 0x7f1dc0069660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.892+0000 7f1ddf59e700 1 -- 172.21.15.55:0/1061427184 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1de015d230 msgr2=0x7f1de016cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:25.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.892+0000 7f1ddf59e700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1de015d230 0x7f1de016cfd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1dd800ccc0 tx=0x7f1dd80019f0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.892+0000 7f1ddf59e700 1 -- 172.21.15.55:0/1061427184 shutdown_connections 2023-10-06T03:49:25.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.892+0000 7f1ddf59e700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1dc00671b0 0x7f1dc0069660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.892+0000 7f1ddf59e700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1de015d230 0x7f1de016cfd0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.892+0000 7f1ddf59e700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1de0159420 0x7f1de01534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.892+0000 7f1ddf59e700 1 --2- 172.21.15.55:0/1061427184 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1de01597f0 0x7f1de00b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:25.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.892+0000 7f1ddf59e700 1 -- 172.21.15.55:0/1061427184 >> 172.21.15.55:0/1061427184 conn(0x7f1de00d0660 msgr2=0x7f1de0153b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:25.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.892+0000 7f1ddf59e700 1 -- 172.21.15.55:0/1061427184 shutdown_connections 2023-10-06T03:49:25.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:25.892+0000 7f1ddf59e700 1 -- 172.21.15.55:0/1061427184 wait complete. 2023-10-06T03:49:25.913 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:49:25.913 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:49:25.998 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:25.995+0000 7fd239592180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:49:26.076 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:26.074+0000 7f7dd8257180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:49:26.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.103+0000 7f38fa79d700 1 Processor -- start 2023-10-06T03:49:26.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.103+0000 7f38fa79d700 1 -- start start 2023-10-06T03:49:26.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.105+0000 7f38fa79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38f4159420 0x7f38f40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:26.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.105+0000 7f38fa79d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f38f415d230 0x7f38f415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:26.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.105+0000 7f38fa79d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f415dbd0 con 0x7f38f41597f0 2023-10-06T03:49:26.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.105+0000 7f38fa79d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f40b2980 con 0x7f38f415d230 2023-10-06T03:49:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.105+0000 7f38fa79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40b11d0 con 0x7f38f4159420 2023-10-06T03:49:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.105+0000 7f38f9f9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f38f415d230 0x7f38f415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.105+0000 7f38f9f9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f38f415d230 0x7f38f415d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48982/0 (socket says 172.21.15.55:48982) 2023-10-06T03:49:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.105+0000 7f38f9f9c700 1 -- 172.21.15.55:0/1103024856 learned_addr learned my addr 172.21.15.55:0/1103024856 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.105+0000 7f38f979b700 1 --2- 172.21.15.55:0/1103024856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38f4159420 0x7f38f40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.105+0000 7f38f9f9c700 1 -- 172.21.15.55:0/1103024856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38f4159420 msgr2=0x7f38f40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:26.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.105+0000 7f38f9f9c700 1 --2- 172.21.15.55:0/1103024856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38f4159420 0x7f38f40b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.105+0000 7f38f9f9c700 1 -- 172.21.15.55:0/1103024856 >> v1:172.21.15.55:6789/0 conn(0x7f38f41597f0 legacy=0x7f38f40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.105+0000 7f38f9f9c700 1 -- 172.21.15.55:0/1103024856 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f40b14b0 con 0x7f38f415d230 2023-10-06T03:49:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.106+0000 7f38f9f9c700 1 --2- 172.21.15.55:0/1103024856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38f415d230 0x7f38f415d690 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009f20 tx=0x7f38ec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a927af51b4321cc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.106+0000 7f38ebfff700 1 -- 172.21.15.55:0/1103024856 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec00e070 con 0x7f38f415d230 2023-10-06T03:49:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.106+0000 7f38ebfff700 1 -- 172.21.15.55:0/1103024856 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec00b9d0 con 0x7f38f415d230 2023-10-06T03:49:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.106+0000 7f38ebfff700 1 -- 172.21.15.55:0/1103024856 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec0092e0 con 0x7f38f415d230 2023-10-06T03:49:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.107+0000 7f38fa79d700 1 -- 172.21.15.55:0/1103024856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38f415d230 msgr2=0x7f38f415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:26.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.107+0000 7f38fa79d700 1 --2- 172.21.15.55:0/1103024856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38f415d230 0x7f38f415d690 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f38ec009f20 tx=0x7f38ec009840 comp rx=0 tx=0).stop 2023-10-06T03:49:26.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.107+0000 7f38fa79d700 1 -- 172.21.15.55:0/1103024856 shutdown_connections 2023-10-06T03:49:26.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.107+0000 7f38fa79d700 1 --2- 172.21.15.55:0/1103024856 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38f415d230 0x7f38f415d690 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.107+0000 7f38fa79d700 1 --2- 172.21.15.55:0/1103024856 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38f4159420 0x7f38f40b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.107+0000 7f38fa79d700 1 -- 172.21.15.55:0/1103024856 >> 172.21.15.55:0/1103024856 conn(0x7f38f40d0660 msgr2=0x7f38f40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:26.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.107+0000 7f38fa79d700 1 -- 172.21.15.55:0/1103024856 shutdown_connections 2023-10-06T03:49:26.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.107+0000 7f38fa79d700 1 -- 172.21.15.55:0/1103024856 wait complete. 2023-10-06T03:49:26.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.108+0000 7f38fa79d700 1 Processor -- start 2023-10-06T03:49:26.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.109+0000 7f38fa79d700 1 -- start start 2023-10-06T03:49:26.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.109+0000 7f38fa79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38f4159420 0x7f38f40ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:26.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38fa79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38f41597f0 0x7f38f40cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:26.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38fa79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f415d230 0x7f38f416a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:26.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38fa79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40caf70 con 0x7f38f41597f0 2023-10-06T03:49:26.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38fa79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40cb250 con 0x7f38f415d230 2023-10-06T03:49:26.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38fa79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40cb530 con 0x7f38f4159420 2023-10-06T03:49:26.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38f8f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38f41597f0 0x7f38f40cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:26.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38f8f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38f41597f0 0x7f38f40cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48986/0 (socket says 172.21.15.55:48986) 2023-10-06T03:49:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38f979b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38f4159420 0x7f38f40ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38f8f9a700 1 -- 172.21.15.55:0/324177495 learned_addr learned my addr 172.21.15.55:0/324177495 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38f9f9c700 1 --2- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f415d230 0x7f38f416a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38f979b700 1 -- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f415d230 msgr2=0x7f38f416a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:26.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38f979b700 1 --2- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f415d230 0x7f38f416a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38f979b700 1 -- 172.21.15.55:0/324177495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38f41597f0 msgr2=0x7f38f40cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:26.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38f979b700 1 --2- 172.21.15.55:0/324177495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38f41597f0 0x7f38f40cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38f979b700 1 -- 172.21.15.55:0/324177495 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec009ae0 con 0x7f38f4159420 2023-10-06T03:49:26.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.110+0000 7f38f8f9a700 1 --2- 172.21.15.55:0/324177495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38f41597f0 0x7f38f40cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:26.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.111+0000 7f38f979b700 1 --2- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38f4159420 0x7f38f40ca990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f38f000ea50 tx=0x7f38f00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:26.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.111+0000 7f38e97fa700 1 -- 172.21.15.55:0/324177495 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f000ee90 con 0x7f38f4159420 2023-10-06T03:49:26.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.111+0000 7f38e97fa700 1 -- 172.21.15.55:0/324177495 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f000c7f0 con 0x7f38f4159420 2023-10-06T03:49:26.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.111+0000 7f38fa79d700 1 -- 172.21.15.55:0/324177495 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f40cb810 con 0x7f38f4159420 2023-10-06T03:49:26.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.111+0000 7f38e97fa700 1 -- 172.21.15.55:0/324177495 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f000cb90 con 0x7f38f4159420 2023-10-06T03:49:26.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.111+0000 7f38fa79d700 1 -- 172.21.15.55:0/324177495 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f4155920 con 0x7f38f4159420 2023-10-06T03:49:26.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.112+0000 7f38fa79d700 1 -- 172.21.15.55:0/324177495 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f404c3c0 con 0x7f38f4159420 2023-10-06T03:49:26.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.113+0000 7f38e97fa700 1 -- 172.21.15.55:0/324177495 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f38f0013470 con 0x7f38f4159420 2023-10-06T03:49:26.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.114+0000 7f38e97fa700 1 --2- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f38d80670e0 0x7f38d8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:26.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.114+0000 7f38e97fa700 1 -- 172.21.15.55:0/324177495 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f38f0084e70 con 0x7f38f4159420 2023-10-06T03:49:26.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.114+0000 7f38f8f9a700 1 -- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f38d80670e0 msgr2=0x7f38d8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:26.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.114+0000 7f38f8f9a700 1 --2- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f38d80670e0 0x7f38d8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:26.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.123+0000 7f38e97fa700 1 -- 172.21.15.55:0/324177495 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38f0088030 con 0x7f38f4159420 2023-10-06T03:49:26.178 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:26.176+0000 7f7dd8257180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:49:26.304 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:26.301+0000 7fd239592180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:49:26.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.315+0000 7f38f8f9a700 1 -- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f38d80670e0 msgr2=0x7f38d8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:26.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.315+0000 7f38f8f9a700 1 --2- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f38d80670e0 0x7f38d8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:26.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.374+0000 7f38fa79d700 1 -- 172.21.15.55:0/324177495 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38f40b1790 con 0x7f38f4159420 2023-10-06T03:49:26.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.375+0000 7f38e97fa700 1 -- 172.21.15.55:0/324177495 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f38f00519d0 con 0x7f38f4159420 2023-10-06T03:49:26.378 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:26.378 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/4087100648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4087100648},{"type":"v1","addr":"172.21.15.163:6834","nonce":4087100648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2017152000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2017152000},{"type":"v1","addr":"172.21.15.163:6836","nonce":2017152000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1226557536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1226557536},{"type":"v1","addr":"172.21.15.55:6834","nonce":1226557536}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:49:03.547622+0000","modified":"2023-10-06T03:49:07.074089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6804":{"gid":6804,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/3960749421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3960749421},{"type":"v1","addr":"172.21.15.55:6835","nonce":3960749421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T03:49:26.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.378+0000 7f38fa79d700 1 -- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f38d80670e0 msgr2=0x7f38d8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.378+0000 7f38fa79d700 1 --2- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f38d80670e0 0x7f38d8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.378+0000 7f38fa79d700 1 -- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38f4159420 msgr2=0x7f38f40ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.378+0000 7f38fa79d700 1 --2- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38f4159420 0x7f38f40ca990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f38f000ea50 tx=0x7f38f00091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.378+0000 7f38fa79d700 1 -- 172.21.15.55:0/324177495 shutdown_connections 2023-10-06T03:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.378+0000 7f38fa79d700 1 --2- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f38d80670e0 0x7f38d8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.378+0000 7f38fa79d700 1 --2- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38f415d230 0x7f38f416a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.378+0000 7f38fa79d700 1 --2- 172.21.15.55:0/324177495 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38f4159420 0x7f38f40ca990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.378+0000 7f38fa79d700 1 --2- 172.21.15.55:0/324177495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38f41597f0 0x7f38f40cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.378+0000 7f38fa79d700 1 -- 172.21.15.55:0/324177495 >> 172.21.15.55:0/324177495 conn(0x7f38f40d0660 msgr2=0x7f38f41564f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:26.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.378+0000 7f38fa79d700 1 -- 172.21.15.55:0/324177495 shutdown_connections 2023-10-06T03:49:26.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.378+0000 7f38fa79d700 1 -- 172.21.15.55:0/324177495 wait complete. 2023-10-06T03:49:26.382 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 44 2023-10-06T03:49:26.399 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:49:26.408 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:26.406+0000 7fd239592180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:49:26.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.591+0000 7f9fdd7ff700 1 Processor -- start 2023-10-06T03:49:26.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.591+0000 7f9fdd7ff700 1 -- start start 2023-10-06T03:49:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.593+0000 7f9fdd7ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd8068550 0x7f9fd8068920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.593+0000 7f9fdd7ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9fd815e010 0x7f9fd806f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.593+0000 7f9fdd7ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9fd806f640 0x7f9fd806a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.593+0000 7f9fdd7ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd80b3430 con 0x7f9fd806f640 2023-10-06T03:49:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.593+0000 7f9fdd7ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd806ec70 con 0x7f9fd815e010 2023-10-06T03:49:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.593+0000 7f9fdd7ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80b2e70 con 0x7f9fd8068550 2023-10-06T03:49:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.593+0000 7f9fd77fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9fd806f640 0x7f9fd806a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.593+0000 7f9fd67fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9fd815e010 0x7f9fd806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:26.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.593+0000 7f9fd77fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9fd806f640 0x7f9fd806a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49000/0 (socket says 172.21.15.55:49000) 2023-10-06T03:49:26.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.593+0000 7f9fd77fe700 1 -- 172.21.15.55:0/2507499613 learned_addr learned my addr 172.21.15.55:0/2507499613 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:26.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.593+0000 7f9fd6ffd700 1 --2- 172.21.15.55:0/2507499613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd8068550 0x7f9fd8068920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:26.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.594+0000 7f9fd77fe700 1 -- 172.21.15.55:0/2507499613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd8068550 msgr2=0x7f9fd8068920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:26.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.594+0000 7f9fd77fe700 1 --2- 172.21.15.55:0/2507499613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd8068550 0x7f9fd8068920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.594+0000 7f9fd77fe700 1 -- 172.21.15.55:0/2507499613 >> v2:172.21.15.163:3300/0 conn(0x7f9fd815e010 msgr2=0x7f9fd806f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:26.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.594+0000 7f9fd77fe700 1 --2- 172.21.15.55:0/2507499613 >> v2:172.21.15.163:3300/0 conn(0x7f9fd815e010 0x7f9fd806f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.594+0000 7f9fd77fe700 1 -- 172.21.15.55:0/2507499613 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd80b3150 con 0x7f9fd806f640 2023-10-06T03:49:26.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.594+0000 7f9fd77fe700 1 --2- 172.21.15.55:0/2507499613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd806f640 0x7f9fd806a7e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9fd806b9a0 tx=0x7f9fd0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102240f531247805 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:26.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.594+0000 7f9fd5ffb700 1 -- 172.21.15.55:0/2507499613 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd000e070 con 0x7f9fd806f640 2023-10-06T03:49:26.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.594+0000 7f9fd5ffb700 1 -- 172.21.15.55:0/2507499613 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd000b970 con 0x7f9fd806f640 2023-10-06T03:49:26.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.594+0000 7f9fd5ffb700 1 -- 172.21.15.55:0/2507499613 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd00092e0 con 0x7f9fd806f640 2023-10-06T03:49:26.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.595+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/2507499613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd806f640 msgr2=0x7f9fd806a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:26.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.595+0000 7f9fdd7ff700 1 --2- 172.21.15.55:0/2507499613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd806f640 0x7f9fd806a7e0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9fd806b9a0 tx=0x7f9fd0009840 comp rx=0 tx=0).stop 2023-10-06T03:49:26.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.595+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/2507499613 shutdown_connections 2023-10-06T03:49:26.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.595+0000 7f9fdd7ff700 1 --2- 172.21.15.55:0/2507499613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd806f640 0x7f9fd806a7e0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.595+0000 7f9fdd7ff700 1 --2- 172.21.15.55:0/2507499613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd8068550 0x7f9fd8068920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.595+0000 7f9fdd7ff700 1 --2- 172.21.15.55:0/2507499613 >> v2:172.21.15.163:3300/0 conn(0x7f9fd815e010 0x7f9fd806f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.595+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/2507499613 >> 172.21.15.55:0/2507499613 conn(0x7f9fd80cd480 msgr2=0x7f9fd80cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:26.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.595+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/2507499613 shutdown_connections 2023-10-06T03:49:26.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.595+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/2507499613 wait complete. 2023-10-06T03:49:26.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.596+0000 7f9fdd7ff700 1 Processor -- start 2023-10-06T03:49:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.597+0000 7f9fdd7ff700 1 -- start start 2023-10-06T03:49:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.597+0000 7f9fdd7ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd8068550 0x7f9fd80ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.597+0000 7f9fdd7ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd806f640 0x7f9fd8158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fdd7ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd815e010 0x7f9fd8149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fdd7ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80ab790 con 0x7f9fd8068550 2023-10-06T03:49:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fdd7ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80aba70 con 0x7f9fd815e010 2023-10-06T03:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fdd7ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd80abd50 con 0x7f9fd806f640 2023-10-06T03:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fd6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd8068550 0x7f9fd80ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fd6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd8068550 0x7f9fd80ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49016/0 (socket says 172.21.15.55:49016) 2023-10-06T03:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fd6ffd700 1 -- 172.21.15.55:0/1652819022 learned_addr learned my addr 172.21.15.55:0/1652819022 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fd77fe700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd815e010 0x7f9fd8149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fd67fc700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd806f640 0x7f9fd8158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fd6ffd700 1 -- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd806f640 msgr2=0x7f9fd8158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fd6ffd700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd806f640 0x7f9fd8158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fd6ffd700 1 -- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd815e010 msgr2=0x7f9fd8149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fd6ffd700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd815e010 0x7f9fd8149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fd6ffd700 1 -- 172.21.15.55:0/1652819022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd0009ae0 con 0x7f9fd8068550 2023-10-06T03:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fd67fc700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd806f640 0x7f9fd8158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.598+0000 7f9fd77fe700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd815e010 0x7f9fd8149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.599+0000 7f9fd6ffd700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd8068550 0x7f9fd80ad8a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800ec80 tx=0x7f9fc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.599+0000 7f9fbeffd700 1 -- 172.21.15.55:0/1652819022 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc800c850 con 0x7f9fd8068550 2023-10-06T03:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.599+0000 7f9fbeffd700 1 -- 172.21.15.55:0/1652819022 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc800c9b0 con 0x7f9fd8068550 2023-10-06T03:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.599+0000 7f9fbeffd700 1 -- 172.21.15.55:0/1652819022 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc800cd50 con 0x7f9fd8068550 2023-10-06T03:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.599+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/1652819022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd80ac030 con 0x7f9fd8068550 2023-10-06T03:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.599+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/1652819022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd80af200 con 0x7f9fd8068550 2023-10-06T03:49:26.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.600+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/1652819022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd814b6a0 con 0x7f9fd8068550 2023-10-06T03:49:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.601+0000 7f9fbeffd700 1 -- 172.21.15.55:0/1652819022 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f9fc8013680 con 0x7f9fd8068550 2023-10-06T03:49:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.603+0000 7f9fbeffd700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9fc40670e0 0x7f9fc4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.603+0000 7f9fbeffd700 1 -- 172.21.15.55:0/1652819022 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f9fc8084f00 con 0x7f9fd8068550 2023-10-06T03:49:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.603+0000 7f9fd67fc700 1 -- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9fc40670e0 msgr2=0x7f9fc4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.603+0000 7f9fd67fc700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9fc40670e0 0x7f9fc4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:26.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.607+0000 7f9fbeffd700 1 -- 172.21.15.55:0/1652819022 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fc8051a60 con 0x7f9fd8068550 2023-10-06T03:49:26.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.804+0000 7f9fd67fc700 1 -- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9fc40670e0 msgr2=0x7f9fc4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:26.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.804+0000 7f9fd67fc700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9fc40670e0 0x7f9fc4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:26.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.854+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/1652819022 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fd804c3c0 con 0x7f9fd8068550 2023-10-06T03:49:26.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.855+0000 7f9fbeffd700 1 -- 172.21.15.55:0/1652819022 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f9fc8055080 con 0x7f9fd8068550 2023-10-06T03:49:26.858 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:26.858 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/4087100648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4087100648},{"type":"v1","addr":"172.21.15.163:6834","nonce":4087100648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2017152000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2017152000},{"type":"v1","addr":"172.21.15.163:6836","nonce":2017152000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1226557536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1226557536},{"type":"v1","addr":"172.21.15.55:6834","nonce":1226557536}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:49:03.547622+0000","modified":"2023-10-06T03:49:07.074089+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6804},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6804":{"gid":6804,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/3960749421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3960749421},{"type":"v1","addr":"172.21.15.55:6835","nonce":3960749421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-06T03:49:26.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.857+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9fc40670e0 msgr2=0x7f9fc4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:26.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.857+0000 7f9fdd7ff700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9fc40670e0 0x7f9fc4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.857+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/1652819022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd8068550 msgr2=0x7f9fd80ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:26.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.857+0000 7f9fdd7ff700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd8068550 0x7f9fd80ad8a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800ec80 tx=0x7f9fc80091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.857+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/1652819022 shutdown_connections 2023-10-06T03:49:26.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.857+0000 7f9fdd7ff700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9fc40670e0 0x7f9fc4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.857+0000 7f9fdd7ff700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9fd815e010 0x7f9fd8149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.857+0000 7f9fdd7ff700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9fd8068550 0x7f9fd80ad8a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.857+0000 7f9fdd7ff700 1 --2- 172.21.15.55:0/1652819022 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9fd806f640 0x7f9fd8158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:26.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.857+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/1652819022 >> 172.21.15.55:0/1652819022 conn(0x7f9fd80cd480 msgr2=0x7f9fd8151360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:26.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.857+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/1652819022 shutdown_connections 2023-10-06T03:49:26.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:26.857+0000 7f9fdd7ff700 1 -- 172.21.15.55:0/1652819022 wait complete. 2023-10-06T03:49:26.864 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 44 2023-10-06T03:49:26.878 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:49:26.885 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:26.883+0000 7f7dd8257180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:49:27.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.106+0000 7f286f59d700 1 Processor -- start 2023-10-06T03:49:27.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.107+0000 7f286f59d700 1 -- start start 2023-10-06T03:49:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.108+0000 7f286f59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f286806a0a0 0x7f286815b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.108+0000 7f286f59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f286806dd20 0x7f286815b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.108+0000 7f286f59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f286806ae70 con 0x7f286806d950 2023-10-06T03:49:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.108+0000 7f286f59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28680c8a60 con 0x7f286806dd20 2023-10-06T03:49:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.108+0000 7f286f59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286806a540 con 0x7f286806a0a0 2023-10-06T03:49:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.108+0000 7f286ed9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f286806dd20 0x7f286815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.108+0000 7f286ed9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f286806dd20 0x7f286815b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49024/0 (socket says 172.21.15.55:49024) 2023-10-06T03:49:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.108+0000 7f286ed9c700 1 -- 172.21.15.55:0/2070747108 learned_addr learned my addr 172.21.15.55:0/2070747108 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:27.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.108+0000 7f286dd9a700 1 --2- 172.21.15.55:0/2070747108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f286806a0a0 0x7f286815b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.109+0000 7f286ed9c700 1 -- 172.21.15.55:0/2070747108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f286806a0a0 msgr2=0x7f286815b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.109+0000 7f286ed9c700 1 --2- 172.21.15.55:0/2070747108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f286806a0a0 0x7f286815b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.109+0000 7f286ed9c700 1 -- 172.21.15.55:0/2070747108 >> v1:172.21.15.163:6789/0 conn(0x7f286806d950 legacy=0x7f28680c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.109+0000 7f286ed9c700 1 -- 172.21.15.55:0/2070747108 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286806a820 con 0x7f286806dd20 2023-10-06T03:49:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.109+0000 7f286ed9c700 1 --2- 172.21.15.55:0/2070747108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f286806dd20 0x7f286815b950 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f2860009f20 tx=0x7f2860009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b6754d8ec74919 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.109+0000 7f286d599700 1 -- 172.21.15.55:0/2070747108 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286000e070 con 0x7f286806dd20 2023-10-06T03:49:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.109+0000 7f286d599700 1 -- 172.21.15.55:0/2070747108 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286000b9d0 con 0x7f286806dd20 2023-10-06T03:49:27.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.109+0000 7f286d599700 1 -- 172.21.15.55:0/2070747108 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28600092e0 con 0x7f286806dd20 2023-10-06T03:49:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.110+0000 7f286f59d700 1 -- 172.21.15.55:0/2070747108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f286806dd20 msgr2=0x7f286815b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.110+0000 7f286f59d700 1 --2- 172.21.15.55:0/2070747108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f286806dd20 0x7f286815b950 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f2860009f20 tx=0x7f2860009840 comp rx=0 tx=0).stop 2023-10-06T03:49:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.110+0000 7f286f59d700 1 -- 172.21.15.55:0/2070747108 shutdown_connections 2023-10-06T03:49:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.110+0000 7f286f59d700 1 --2- 172.21.15.55:0/2070747108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f286806dd20 0x7f286815b950 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.110+0000 7f286f59d700 1 --2- 172.21.15.55:0/2070747108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f286806a0a0 0x7f286815b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.110+0000 7f286f59d700 1 -- 172.21.15.55:0/2070747108 >> 172.21.15.55:0/2070747108 conn(0x7f28680cc1f0 msgr2=0x7f28680cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.110+0000 7f286f59d700 1 -- 172.21.15.55:0/2070747108 shutdown_connections 2023-10-06T03:49:27.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.110+0000 7f286f59d700 1 -- 172.21.15.55:0/2070747108 wait complete. 2023-10-06T03:49:27.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.111+0000 7f286f59d700 1 Processor -- start 2023-10-06T03:49:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.112+0000 7f286f59d700 1 -- start start 2023-10-06T03:49:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.112+0000 7f286f59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f286806a0a0 0x7f2868142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286f59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f286806d950 0x7f28680c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286f59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f286806dd20 0x7f2868150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286f59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286814d0b0 con 0x7f286806dd20 2023-10-06T03:49:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286f59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28680c44a0 con 0x7f286806d950 2023-10-06T03:49:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286f59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286815acc0 con 0x7f286806a0a0 2023-10-06T03:49:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286ed9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f286806dd20 0x7f2868150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286ed9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f286806dd20 0x7f2868150ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49030/0 (socket says 172.21.15.55:49030) 2023-10-06T03:49:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286ed9c700 1 -- 172.21.15.55:0/3542612506 learned_addr learned my addr 172.21.15.55:0/3542612506 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:27.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286e59b700 1 --2- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f286806a0a0 0x7f2868142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:27.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286dd9a700 1 --2- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f286806d950 0x7f28680c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:27.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286ed9c700 1 -- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f286806a0a0 msgr2=0x7f2868142c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:27.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286ed9c700 1 --2- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f286806a0a0 0x7f2868142c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286ed9c700 1 -- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f286806d950 msgr2=0x7f28680c3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:27.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286ed9c700 1 --2- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f286806d950 0x7f28680c3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.113+0000 7f286ed9c700 1 -- 172.21.15.55:0/3542612506 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2860009ae0 con 0x7f286806dd20 2023-10-06T03:49:27.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.114+0000 7f286ed9c700 1 --2- 172.21.15.55:0/3542612506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f286806dd20 0x7f2868150ba0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f286000b610 tx=0x7f2860018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:27.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.114+0000 7f285e7fc700 1 -- 172.21.15.55:0/3542612506 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860008ad0 con 0x7f286806dd20 2023-10-06T03:49:27.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.114+0000 7f285e7fc700 1 -- 172.21.15.55:0/3542612506 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2860008c30 con 0x7f286806dd20 2023-10-06T03:49:27.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.114+0000 7f285e7fc700 1 -- 172.21.15.55:0/3542612506 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286001c410 con 0x7f286806dd20 2023-10-06T03:49:27.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.114+0000 7f286f59d700 1 -- 172.21.15.55:0/3542612506 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28680c4710 con 0x7f286806dd20 2023-10-06T03:49:27.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.114+0000 7f286f59d700 1 -- 172.21.15.55:0/3542612506 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2868143150 con 0x7f286806dd20 2023-10-06T03:49:27.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.115+0000 7f286f59d700 1 -- 172.21.15.55:0/3542612506 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28680c4c30 con 0x7f286806dd20 2023-10-06T03:49:27.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.117+0000 7f285e7fc700 1 -- 172.21.15.55:0/3542612506 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f2860003040 con 0x7f286806dd20 2023-10-06T03:49:27.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.117+0000 7f285e7fc700 1 --2- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f284c0670e0 0x7f284c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:27.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.118+0000 7f285e7fc700 1 -- 172.21.15.55:0/3542612506 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f2860086870 con 0x7f286806dd20 2023-10-06T03:49:27.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.118+0000 7f286e59b700 1 -- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f284c0670e0 msgr2=0x7f284c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:27.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.118+0000 7f286e59b700 1 --2- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f284c0670e0 0x7f284c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:27.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.122+0000 7f285e7fc700 1 -- 172.21.15.55:0/3542612506 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28680c4c30 con 0x7f286806dd20 2023-10-06T03:49:27.166 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:27.164+0000 7fd239592180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:49:27.173 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:27.172+0000 7f7dd8257180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:49:27.265 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:27.263+0000 7f7dd8257180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:49:27.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.319+0000 7f286e59b700 1 -- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f284c0670e0 msgr2=0x7f284c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:27.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.319+0000 7f286e59b700 1 --2- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f284c0670e0 0x7f284c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:27.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.328+0000 7f286f59d700 1 -- 172.21.15.55:0/3542612506 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28680c4c30 con 0x7f286806dd20 2023-10-06T03:49:27.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.329+0000 7f285e7fc700 1 -- 172.21.15.55:0/3542612506 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+17885 (secure 0 0 0) 0x7f28680c4c30 con 0x7f286806dd20 2023-10-06T03:49:27.332 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:27.332 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":94,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:49:19.842571+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:49:01.721643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-10-06T03:49:02.520120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3249461740":"2023-10-07T03:49:19.842520+0000","172.21.15.163:0/4224423400":"2023-10-07T03:49:19.842520+0000","172.21.15.163:6837/85191":"2023-10-07T03:49:19.842520+0000","172.21.15.163:6835/85191":"2023-10-07T03:49:19.842520+0000","172.21.15.163:0/1889998714":"2023-10-07T03:49:19.842520+0000","172.21.15.163:0/4148366955":"2023-10-07T03:49:19.842520+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:49:27.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.332+0000 7f286f59d700 1 -- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f284c0670e0 msgr2=0x7f284c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:27.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.332+0000 7f286f59d700 1 --2- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f284c0670e0 0x7f284c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.332+0000 7f286f59d700 1 -- 172.21.15.55:0/3542612506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f286806dd20 msgr2=0x7f2868150ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.332+0000 7f286f59d700 1 --2- 172.21.15.55:0/3542612506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f286806dd20 0x7f2868150ba0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f286000b610 tx=0x7f2860018000 comp rx=0 tx=0).stop 2023-10-06T03:49:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.332+0000 7f286f59d700 1 -- 172.21.15.55:0/3542612506 shutdown_connections 2023-10-06T03:49:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.332+0000 7f286f59d700 1 --2- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f284c0670e0 0x7f284c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.332+0000 7f286f59d700 1 --2- 172.21.15.55:0/3542612506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f286806dd20 0x7f2868150ba0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.332+0000 7f286f59d700 1 --2- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f286806a0a0 0x7f2868142c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.332+0000 7f286f59d700 1 --2- 172.21.15.55:0/3542612506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f286806d950 0x7f28680c3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.332+0000 7f286f59d700 1 -- 172.21.15.55:0/3542612506 >> 172.21.15.55:0/3542612506 conn(0x7f28680cc1f0 msgr2=0x7f28681573a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.332+0000 7f286f59d700 1 -- 172.21.15.55:0/3542612506 shutdown_connections 2023-10-06T03:49:27.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.332+0000 7f286f59d700 1 -- 172.21.15.55:0/3542612506 wait complete. 2023-10-06T03:49:27.354 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:49:27.418 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:27.416+0000 7f7dd8257180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:49:27.462 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:27.459+0000 7fd239592180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:49:27.524 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:27.522+0000 7f7dd8257180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:49:27.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.543+0000 7f18ce19d700 1 Processor -- start 2023-10-06T03:49:27.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.543+0000 7f18ce19d700 1 -- start start 2023-10-06T03:49:27.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.544+0000 7f18ce19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18c806c0f0 0x7f18c81642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:27.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.544+0000 7f18ce19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c80b40f0 con 0x7f18c8142ea0 2023-10-06T03:49:27.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.544+0000 7f18ce19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c8069640 con 0x7f18c80ad870 2023-10-06T03:49:27.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.544+0000 7f18ce19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c8069920 con 0x7f18c806c0f0 2023-10-06T03:49:27.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.544+0000 7f18cd19b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f18c8142ea0 0x7f18c80b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42822/0 (socket says 172.21.15.55:42822) 2023-10-06T03:49:27.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.544+0000 7f18cd19b700 1 -- 172.21.15.55:0/1317431952 learned_addr learned my addr 172.21.15.55:0/1317431952 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:27.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.545+0000 7f18cd99c700 1 --2- 172.21.15.55:0/1317431952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18c806c0f0 0x7f18c81642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:27.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.545+0000 7f18bffff700 1 -- 172.21.15.55:0/1317431952 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445569719 0 0) 0x7f18c80b40f0 con 0x7f18c8142ea0 2023-10-06T03:49:27.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.545+0000 7f18bffff700 1 -- 172.21.15.55:0/1317431952 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18b4003570 con 0x7f18c8142ea0 2023-10-06T03:49:27.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.545+0000 7f18bffff700 1 -- 172.21.15.55:0/1317431952 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489254397 0 0) 0x7f18c8069640 con 0x7f18c80ad870 2023-10-06T03:49:27.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.545+0000 7f18bffff700 1 -- 172.21.15.55:0/1317431952 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18c80b40f0 con 0x7f18c80ad870 2023-10-06T03:49:27.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.545+0000 7f18cd99c700 1 -- 172.21.15.55:0/1317431952 >> v1:172.21.15.163:6789/0 conn(0x7f18c80ad870 legacy=0x7f18c806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:27.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.545+0000 7f18cd99c700 1 -- 172.21.15.55:0/1317431952 >> v1:172.21.15.55:6789/0 conn(0x7f18c8142ea0 legacy=0x7f18c80b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:27.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.545+0000 7f18cd99c700 1 -- 172.21.15.55:0/1317431952 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c8069c00 con 0x7f18c806c0f0 2023-10-06T03:49:27.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.545+0000 7f18cd99c700 1 --2- 172.21.15.55:0/1317431952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18c806c0f0 0x7f18c81642e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009e90 tx=0x7f18c0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c696543254a3567 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:27.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.546+0000 7f18bffff700 1 -- 172.21.15.55:0/1317431952 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c000e070 con 0x7f18c806c0f0 2023-10-06T03:49:27.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.546+0000 7f18bffff700 1 -- 172.21.15.55:0/1317431952 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c000bb80 con 0x7f18c806c0f0 2023-10-06T03:49:27.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.546+0000 7f18bffff700 1 -- 172.21.15.55:0/1317431952 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0012430 con 0x7f18c806c0f0 2023-10-06T03:49:27.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.546+0000 7f18ce19d700 1 -- 172.21.15.55:0/1317431952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18c806c0f0 msgr2=0x7f18c81642e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:27.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.546+0000 7f18ce19d700 1 --2- 172.21.15.55:0/1317431952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18c806c0f0 0x7f18c81642e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009e90 tx=0x7f18c0009780 comp rx=0 tx=0).stop 2023-10-06T03:49:27.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.546+0000 7f18ce19d700 1 -- 172.21.15.55:0/1317431952 shutdown_connections 2023-10-06T03:49:27.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.546+0000 7f18ce19d700 1 --2- 172.21.15.55:0/1317431952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18c806c0f0 0x7f18c81642e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.546+0000 7f18ce19d700 1 -- 172.21.15.55:0/1317431952 >> 172.21.15.55:0/1317431952 conn(0x7f18c80ce430 msgr2=0x7f18c80ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:27.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.546+0000 7f18ce19d700 1 -- 172.21.15.55:0/1317431952 shutdown_connections 2023-10-06T03:49:27.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.547+0000 7f18ce19d700 1 -- 172.21.15.55:0/1317431952 wait complete. 2023-10-06T03:49:27.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.548+0000 7f18ce19d700 1 Processor -- start 2023-10-06T03:49:27.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.548+0000 7f18ce19d700 1 -- start start 2023-10-06T03:49:27.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.549+0000 7f18ce19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18c806c0f0 0x7f18c815cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:27.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.549+0000 7f18ce19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18c8142ea0 0x7f18c80ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:27.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.549+0000 7f18ce19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18c80acda0 0x7f18c80c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:27.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.549+0000 7f18ce19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c815a170 con 0x7f18c80acda0 2023-10-06T03:49:27.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.549+0000 7f18ce19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80ad1e0 con 0x7f18c806c0f0 2023-10-06T03:49:27.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.549+0000 7f18ce19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c815a3c0 con 0x7f18c8142ea0 2023-10-06T03:49:27.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.549+0000 7f18cd99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18c80acda0 0x7f18c80c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:27.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.549+0000 7f18cd99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18c80acda0 0x7f18c80c93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49038/0 (socket says 172.21.15.55:49038) 2023-10-06T03:49:27.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.549+0000 7f18cd99c700 1 -- 172.21.15.55:0/1681404411 learned_addr learned my addr 172.21.15.55:0/1681404411 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:27.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.549+0000 7f18cd19b700 1 --2- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18c806c0f0 0x7f18c815cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:27.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.550+0000 7f18cc99a700 1 --2- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18c8142ea0 0x7f18c80ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:27.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.550+0000 7f18cd99c700 1 -- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18c8142ea0 msgr2=0x7f18c80ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:27.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.550+0000 7f18cd99c700 1 --2- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18c8142ea0 0x7f18c80ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.550+0000 7f18cd99c700 1 -- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18c806c0f0 msgr2=0x7f18c815cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:27.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.550+0000 7f18cd99c700 1 --2- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18c806c0f0 0x7f18c815cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.550+0000 7f18cd99c700 1 -- 172.21.15.55:0/1681404411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c0009a20 con 0x7f18c80acda0 2023-10-06T03:49:27.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.550+0000 7f18cd99c700 1 --2- 172.21.15.55:0/1681404411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18c80acda0 0x7f18c80c93b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f18c0013000 tx=0x7f18c0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:27.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.550+0000 7f18bd7fa700 1 -- 172.21.15.55:0/1681404411 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c001b420 con 0x7f18c80acda0 2023-10-06T03:49:27.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.550+0000 7f18bd7fa700 1 -- 172.21.15.55:0/1681404411 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c001b5a0 con 0x7f18c80acda0 2023-10-06T03:49:27.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.550+0000 7f18bd7fa700 1 -- 172.21.15.55:0/1681404411 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c001b960 con 0x7f18c80acda0 2023-10-06T03:49:27.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.551+0000 7f18ce19d700 1 -- 172.21.15.55:0/1681404411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c815a6a0 con 0x7f18c80acda0 2023-10-06T03:49:27.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.551+0000 7f18ce19d700 1 -- 172.21.15.55:0/1681404411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c815ac60 con 0x7f18c80acda0 2023-10-06T03:49:27.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.552+0000 7f18ce19d700 1 -- 172.21.15.55:0/1681404411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c815ae50 con 0x7f18c80acda0 2023-10-06T03:49:27.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.552+0000 7f18bd7fa700 1 -- 172.21.15.55:0/1681404411 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f18c001bd50 con 0x7f18c80acda0 2023-10-06T03:49:27.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.553+0000 7f18bd7fa700 1 --2- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f18ac0670e0 0x7f18ac069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:27.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.553+0000 7f18bd7fa700 1 -- 172.21.15.55:0/1681404411 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f18c0086720 con 0x7f18c80acda0 2023-10-06T03:49:27.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.554+0000 7f18cd19b700 1 -- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f18ac0670e0 msgr2=0x7f18ac069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:27.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.554+0000 7f18cd19b700 1 --2- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f18ac0670e0 0x7f18ac069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:27.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.558+0000 7f18bd7fa700 1 -- 172.21.15.55:0/1681404411 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18c0053220 con 0x7f18c80acda0 2023-10-06T03:49:27.562 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:27.559+0000 7fd239592180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:49:27.716 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:27.714+0000 7f7dd8257180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:49:27.724 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:27.721+0000 7fd239592180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:49:27.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.755+0000 7f18cd19b700 1 -- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f18ac0670e0 msgr2=0x7f18ac069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:27.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.755+0000 7f18cd19b700 1 --2- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f18ac0670e0 0x7f18ac069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:27.809 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:27.807+0000 7f7dd8257180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:49:27.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.808+0000 7f18ce19d700 1 -- 172.21.15.55:0/1681404411 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f18c804c3c0 con 0x7f18c80acda0 2023-10-06T03:49:27.810 INFO:tasks.ceph.mon.a.smithi055.stderr:2023-10-06T03:49:27.808+0000 7fe524ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:49:27.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.814+0000 7f18bd7fa700 1 -- 172.21.15.55:0/1681404411 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) v1 ==== 161+0+0 (secure 0 0 0) 0x7f18c0056840 con 0x7f18c80acda0 2023-10-06T03:49:27.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.817+0000 7f18ce19d700 1 -- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f18ac0670e0 msgr2=0x7f18ac069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:27.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.817+0000 7f18ce19d700 1 --2- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f18ac0670e0 0x7f18ac069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.817+0000 7f18ce19d700 1 -- 172.21.15.55:0/1681404411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18c80acda0 msgr2=0x7f18c80c93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.817+0000 7f18ce19d700 1 --2- 172.21.15.55:0/1681404411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18c80acda0 0x7f18c80c93b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f18c0013000 tx=0x7f18c0019000 comp rx=0 tx=0).stop 2023-10-06T03:49:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.817+0000 7f18ce19d700 1 -- 172.21.15.55:0/1681404411 shutdown_connections 2023-10-06T03:49:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.817+0000 7f18ce19d700 1 --2- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f18ac0670e0 0x7f18ac069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.817+0000 7f18ce19d700 1 --2- 172.21.15.55:0/1681404411 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18c80acda0 0x7f18c80c93b0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.817+0000 7f18ce19d700 1 --2- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18c806c0f0 0x7f18c815cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.817+0000 7f18ce19d700 1 --2- 172.21.15.55:0/1681404411 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18c8142ea0 0x7f18c80ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:27.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.817+0000 7f18ce19d700 1 -- 172.21.15.55:0/1681404411 >> 172.21.15.55:0/1681404411 conn(0x7f18c80ce430 msgr2=0x7f18c80adff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:27.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.818+0000 7f18ce19d700 1 -- 172.21.15.55:0/1681404411 shutdown_connections 2023-10-06T03:49:27.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:27.818+0000 7f18ce19d700 1 -- 172.21.15.55:0/1681404411 wait complete. 2023-10-06T03:49:27.822 INFO:tasks.ceph.mds.c.smithi055.stderr: -1156> 2023-10-06T03:49:10.032+0000 7fb5401f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:49:27.823 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:49:27.835 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:27.833+0000 7fd239592180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:49:27.838 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:49:27.859 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-10-06T03:49:27.906 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:27.904+0000 7f7dd8257180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:49:28.002 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:28.000+0000 7f7dd8257180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:49:28.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.033+0000 7fca5f59d700 1 Processor -- start 2023-10-06T03:49:28.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.034+0000 7fca5f59d700 1 -- start start 2023-10-06T03:49:28.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.035+0000 7fca5f59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fca580b4e70 0x7fca580ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:28.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.035+0000 7fca5f59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca580b1b40 0x7fca5815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:28.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.035+0000 7fca5f59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca580cfb00 con 0x7fca580b16d0 2023-10-06T03:49:28.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.035+0000 7fca5f59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca580b5c40 con 0x7fca580b4e70 2023-10-06T03:49:28.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.035+0000 7fca5f59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca580b5f20 con 0x7fca580b1b40 2023-10-06T03:49:28.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.036+0000 7fca5e59b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fca580b4e70 0x7fca580ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:28.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.036+0000 7fca5ed9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca580b1b40 0x7fca5815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:28.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.036+0000 7fca5dd9a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fca580b16d0 0x7fca5815d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:35056/0 (socket says 172.21.15.55:35056) 2023-10-06T03:49:28.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.036+0000 7fca5e59b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fca580b4e70 0x7fca580ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:53728/0 (socket says 172.21.15.55:53728) 2023-10-06T03:49:28.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.036+0000 7fca5ed9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca580b1b40 0x7fca5815d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:38348/0 (socket says 172.21.15.55:38348) 2023-10-06T03:49:28.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.036+0000 7fca5dd9a700 1 -- 172.21.15.55:0/1960686511 learned_addr learned my addr 172.21.15.55:0/1960686511 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.036+0000 7fca5d599700 1 -- 172.21.15.55:0/1960686511 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055480934 0 0) 0x7fca580cfb00 con 0x7fca580b16d0 2023-10-06T03:49:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.036+0000 7fca5d599700 1 -- 172.21.15.55:0/1960686511 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca44003570 con 0x7fca580b16d0 2023-10-06T03:49:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.036+0000 7fca5ed9c700 1 -- 172.21.15.55:0/1960686511 >> v2:172.21.15.163:3300/0 conn(0x7fca580b4e70 msgr2=0x7fca580ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.036+0000 7fca5ed9c700 1 --2- 172.21.15.55:0/1960686511 >> v2:172.21.15.163:3300/0 conn(0x7fca580b4e70 0x7fca580ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.036+0000 7fca5ed9c700 1 -- 172.21.15.55:0/1960686511 >> v1:172.21.15.163:6789/0 conn(0x7fca580b16d0 legacy=0x7fca5815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.036+0000 7fca5ed9c700 1 -- 172.21.15.55:0/1960686511 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca580b6200 con 0x7fca580b1b40 2023-10-06T03:49:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.036+0000 7fca5e59b700 1 --2- 172.21.15.55:0/1960686511 >> v2:172.21.15.163:3300/0 conn(0x7fca580b4e70 0x7fca580ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.037+0000 7fca5ed9c700 1 --2- 172.21.15.55:0/1960686511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca580b1b40 0x7fca5815d950 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fca50009e60 tx=0x7fca50009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=991b51cc3cc3b335 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.037+0000 7fca5d599700 1 -- 172.21.15.55:0/1960686511 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5000e070 con 0x7fca580b1b40 2023-10-06T03:49:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.037+0000 7fca5d599700 1 -- 172.21.15.55:0/1960686511 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5000bab0 con 0x7fca580b1b40 2023-10-06T03:49:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.037+0000 7fca5d599700 1 -- 172.21.15.55:0/1960686511 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5000be30 con 0x7fca580b1b40 2023-10-06T03:49:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.037+0000 7fca5f59d700 1 -- 172.21.15.55:0/1960686511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca580b1b40 msgr2=0x7fca5815d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.037+0000 7fca5f59d700 1 --2- 172.21.15.55:0/1960686511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca580b1b40 0x7fca5815d950 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fca50009e60 tx=0x7fca50009780 comp rx=0 tx=0).stop 2023-10-06T03:49:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.037+0000 7fca5f59d700 1 -- 172.21.15.55:0/1960686511 shutdown_connections 2023-10-06T03:49:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.037+0000 7fca5f59d700 1 --2- 172.21.15.55:0/1960686511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca580b1b40 0x7fca5815d950 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.037+0000 7fca5f59d700 1 --2- 172.21.15.55:0/1960686511 >> v2:172.21.15.163:3300/0 conn(0x7fca580b4e70 0x7fca580ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.037+0000 7fca5f59d700 1 -- 172.21.15.55:0/1960686511 >> 172.21.15.55:0/1960686511 conn(0x7fca580d10c0 msgr2=0x7fca580affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.038+0000 7fca5f59d700 1 -- 172.21.15.55:0/1960686511 shutdown_connections 2023-10-06T03:49:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.038+0000 7fca5f59d700 1 -- 172.21.15.55:0/1960686511 wait complete. 2023-10-06T03:49:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.039+0000 7fca5f59d700 1 Processor -- start 2023-10-06T03:49:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.039+0000 7fca5f59d700 1 -- start start 2023-10-06T03:49:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.040+0000 7fca5f59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca580b16d0 0x7fca58145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.040+0000 7fca5f59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca580b1b40 0x7fca580ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.040+0000 7fca5f59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca580b4e70 0x7fca580cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.040+0000 7fca5f59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca580c5760 con 0x7fca580b16d0 2023-10-06T03:49:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.040+0000 7fca5f59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca580c8660 con 0x7fca580b4e70 2023-10-06T03:49:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.040+0000 7fca5f59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca580c8940 con 0x7fca580b1b40 2023-10-06T03:49:28.045 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:28.040+0000 7fd239592180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:49:28.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.040+0000 7fca5e59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca580b16d0 0x7fca58145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:28.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.040+0000 7fca5e59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca580b16d0 0x7fca58145c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49056/0 (socket says 172.21.15.55:49056) 2023-10-06T03:49:28.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.040+0000 7fca5e59b700 1 -- 172.21.15.55:0/1503606452 learned_addr learned my addr 172.21.15.55:0/1503606452 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:28.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.040+0000 7fca5ed9c700 1 --2- 172.21.15.55:0/1503606452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca580b4e70 0x7fca580cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:28.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.040+0000 7fca5dd9a700 1 --2- 172.21.15.55:0/1503606452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca580b1b40 0x7fca580ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:28.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.041+0000 7fca5e59b700 1 -- 172.21.15.55:0/1503606452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca580b1b40 msgr2=0x7fca580ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:28.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.041+0000 7fca5e59b700 1 --2- 172.21.15.55:0/1503606452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca580b1b40 0x7fca580ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:28.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.041+0000 7fca5e59b700 1 -- 172.21.15.55:0/1503606452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca580b4e70 msgr2=0x7fca580cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:28.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.041+0000 7fca5e59b700 1 --2- 172.21.15.55:0/1503606452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca580b4e70 0x7fca580cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:28.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.041+0000 7fca5e59b700 1 -- 172.21.15.55:0/1503606452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca54009420 con 0x7fca580b16d0 2023-10-06T03:49:28.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.041+0000 7fca5e59b700 1 --2- 172.21.15.55:0/1503606452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca580b16d0 0x7fca58145c10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fca5400ec80 tx=0x7fca540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:28.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.041+0000 7fca4e7fc700 1 -- 172.21.15.55:0/1503606452 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5400c8e0 con 0x7fca580b16d0 2023-10-06T03:49:28.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.041+0000 7fca4e7fc700 1 -- 172.21.15.55:0/1503606452 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5400ca40 con 0x7fca580b16d0 2023-10-06T03:49:28.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.041+0000 7fca4e7fc700 1 -- 172.21.15.55:0/1503606452 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5400cdc0 con 0x7fca580b16d0 2023-10-06T03:49:28.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.041+0000 7fca5f59d700 1 -- 172.21.15.55:0/1503606452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca50009a20 con 0x7fca580b16d0 2023-10-06T03:49:28.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.042+0000 7fca5f59d700 1 -- 172.21.15.55:0/1503606452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca580c8f00 con 0x7fca580b16d0 2023-10-06T03:49:28.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.043+0000 7fca5f59d700 1 -- 172.21.15.55:0/1503606452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca58088990 con 0x7fca580b16d0 2023-10-06T03:49:28.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.044+0000 7fca4e7fc700 1 -- 172.21.15.55:0/1503606452 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 38) v1 ==== 84451+0+0 (secure 0 0 0) 0x7fca54013680 con 0x7fca580b16d0 2023-10-06T03:49:28.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.045+0000 7fca4e7fc700 1 -- 172.21.15.55:0/1503606452 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7095+0+0 (secure 0 0 0) 0x7fca54084960 con 0x7fca580b16d0 2023-10-06T03:49:28.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.050+0000 7fca4e7fc700 1 -- 172.21.15.55:0/1503606452 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca54051930 con 0x7fca580b16d0 2023-10-06T03:49:28.089 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:28.086+0000 7f7d52581700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:28.089 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:28.086+0000 7f7d52581700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:28.089 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:28.086+0000 7f7d52581700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:28.089 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:28.086+0000 7f7d52581700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:28.089 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:28.086+0000 7f7d575cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:28.089 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:28.086+0000 7f7d575cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:28.090 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:28.086+0000 7f7d575cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:28.090 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:28.086+0000 7f7d575cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:28.090 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:28.086+0000 7f7d575cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:28.090 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:28.087+0000 7f7d52581700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:49:28.152 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:28.149+0000 7fd239592180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:49:28.247 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:28.245+0000 7fd239592180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:49:28.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.308+0000 7fca5f59d700 1 -- 172.21.15.55:0/1503606452 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fca5804bb80 con 0x7fca580b16d0 2023-10-06T03:49:28.346 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:28.343+0000 7fd239592180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:49:28.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.814+0000 7fca4e7fc700 1 -- 172.21.15.55:0/1503606452 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) v1 ==== 104+0+0 (secure 0 0 0) 0x7fca54054f50 con 0x7fca580b16d0 2023-10-06T03:49:28.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.817+0000 7fca5f59d700 1 -- 172.21.15.55:0/1503606452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca580b16d0 msgr2=0x7fca58145c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:28.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.817+0000 7fca5f59d700 1 --2- 172.21.15.55:0/1503606452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca580b16d0 0x7fca58145c10 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fca5400ec80 tx=0x7fca540091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:28.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.817+0000 7fca5f59d700 1 -- 172.21.15.55:0/1503606452 shutdown_connections 2023-10-06T03:49:28.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.817+0000 7fca5f59d700 1 --2- 172.21.15.55:0/1503606452 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fca580b4e70 0x7fca580cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:28.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.817+0000 7fca5f59d700 1 --2- 172.21.15.55:0/1503606452 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fca580b16d0 0x7fca58145c10 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.817+0000 7fca5f59d700 1 --2- 172.21.15.55:0/1503606452 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fca580b1b40 0x7fca580ca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.817+0000 7fca5f59d700 1 -- 172.21.15.55:0/1503606452 >> 172.21.15.55:0/1503606452 conn(0x7fca580d10c0 msgr2=0x7fca580b5450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.817+0000 7fca5f59d700 1 -- 172.21.15.55:0/1503606452 shutdown_connections 2023-10-06T03:49:28.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:28.817+0000 7fca5f59d700 1 -- 172.21.15.55:0/1503606452 wait complete. 2023-10-06T03:49:28.840 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:49:29.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.078+0000 7fab9b59d700 1 Processor -- start 2023-10-06T03:49:29.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.078+0000 7fab9b59d700 1 -- start start 2023-10-06T03:49:29.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.080+0000 7fab9b59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fab940cb560 0x7fab940cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:29.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.080+0000 7fab9b59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fab94069ec0 0x7fab9415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:29.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.080+0000 7fab9b59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab94159db0 con 0x7fab940cc970 2023-10-06T03:49:29.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.080+0000 7fab9b59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9406aa30 con 0x7fab94069ec0 2023-10-06T03:49:29.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.080+0000 7fab9b59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab940cce10 con 0x7fab940cb560 2023-10-06T03:49:29.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.080+0000 7fab9ad9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fab94069ec0 0x7fab9415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:29.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.080+0000 7fab9ad9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fab94069ec0 0x7fab9415ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49066/0 (socket says 172.21.15.55:49066) 2023-10-06T03:49:29.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.080+0000 7fab9ad9c700 1 -- 172.21.15.55:0/3982631021 learned_addr learned my addr 172.21.15.55:0/3982631021 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:29.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.080+0000 7fab9ad9c700 1 -- 172.21.15.55:0/3982631021 >> v2:172.21.15.163:3300/0 conn(0x7fab940cb560 msgr2=0x7fab940cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:29.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.080+0000 7fab9ad9c700 1 --2- 172.21.15.55:0/3982631021 >> v2:172.21.15.163:3300/0 conn(0x7fab940cb560 0x7fab940cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:29.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.080+0000 7fab9ad9c700 1 -- 172.21.15.55:0/3982631021 >> v1:172.21.15.163:6789/0 conn(0x7fab940cc970 legacy=0x7fab940697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:29.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.080+0000 7fab9ad9c700 1 -- 172.21.15.55:0/3982631021 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9406a470 con 0x7fab94069ec0 2023-10-06T03:49:29.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.081+0000 7fab9ad9c700 1 --2- 172.21.15.55:0/3982631021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94069ec0 0x7fab9415ba60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009f20 tx=0x7fab8c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b0e371318591c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:29.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.081+0000 7fab99599700 1 -- 172.21.15.55:0/3982631021 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c00e070 con 0x7fab94069ec0 2023-10-06T03:49:29.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.081+0000 7fab99599700 1 -- 172.21.15.55:0/3982631021 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c00ba00 con 0x7fab94069ec0 2023-10-06T03:49:29.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.081+0000 7fab99599700 1 -- 172.21.15.55:0/3982631021 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0092e0 con 0x7fab94069ec0 2023-10-06T03:49:29.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.081+0000 7fab9b59d700 1 -- 172.21.15.55:0/3982631021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94069ec0 msgr2=0x7fab9415ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:29.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.081+0000 7fab9b59d700 1 --2- 172.21.15.55:0/3982631021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94069ec0 0x7fab9415ba60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fab8c009f20 tx=0x7fab8c009840 comp rx=0 tx=0).stop 2023-10-06T03:49:29.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.082+0000 7fab9b59d700 1 -- 172.21.15.55:0/3982631021 shutdown_connections 2023-10-06T03:49:29.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.082+0000 7fab9b59d700 1 --2- 172.21.15.55:0/3982631021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94069ec0 0x7fab9415ba60 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:29.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.082+0000 7fab9b59d700 1 --2- 172.21.15.55:0/3982631021 >> v2:172.21.15.163:3300/0 conn(0x7fab940cb560 0x7fab940cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:29.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.082+0000 7fab9b59d700 1 -- 172.21.15.55:0/3982631021 >> 172.21.15.55:0/3982631021 conn(0x7fab9406baf0 msgr2=0x7fab9406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:29.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.082+0000 7fab9b59d700 1 -- 172.21.15.55:0/3982631021 shutdown_connections 2023-10-06T03:49:29.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.082+0000 7fab9b59d700 1 -- 172.21.15.55:0/3982631021 wait complete. 2023-10-06T03:49:29.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.083+0000 7fab9b59d700 1 Processor -- start 2023-10-06T03:49:29.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.083+0000 7fab9b59d700 1 -- start start 2023-10-06T03:49:29.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.084+0000 7fab9b59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94069ec0 0x7fab940c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:29.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.084+0000 7fab9b59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab940cb560 0x7fab9414c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:29.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.084+0000 7fab9b59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab940cc970 0x7fab9416a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:29.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.084+0000 7fab9b59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9414c700 con 0x7fab94069ec0 2023-10-06T03:49:29.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.084+0000 7fab9b59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab940c6680 con 0x7fab940cc970 2023-10-06T03:49:29.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.084+0000 7fab9b59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9416a8c0 con 0x7fab940cb560 2023-10-06T03:49:29.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.084+0000 7fab9a59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94069ec0 0x7fab940c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:29.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.084+0000 7fab9a59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94069ec0 0x7fab940c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49074/0 (socket says 172.21.15.55:49074) 2023-10-06T03:49:29.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.084+0000 7fab9a59b700 1 -- 172.21.15.55:0/1587796329 learned_addr learned my addr 172.21.15.55:0/1587796329 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:29.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.084+0000 7fab9ad9c700 1 --2- 172.21.15.55:0/1587796329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab940cc970 0x7fab9416a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:29.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.084+0000 7fab99d9a700 1 --2- 172.21.15.55:0/1587796329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab940cb560 0x7fab9414c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:29.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.085+0000 7fab9a59b700 1 -- 172.21.15.55:0/1587796329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab940cb560 msgr2=0x7fab9414c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:29.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.085+0000 7fab9a59b700 1 --2- 172.21.15.55:0/1587796329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab940cb560 0x7fab9414c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:29.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.085+0000 7fab9a59b700 1 -- 172.21.15.55:0/1587796329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab940cc970 msgr2=0x7fab9416a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:29.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.085+0000 7fab9a59b700 1 --2- 172.21.15.55:0/1587796329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab940cc970 0x7fab9416a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:29.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.085+0000 7fab9a59b700 1 -- 172.21.15.55:0/1587796329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c009ae0 con 0x7fab94069ec0 2023-10-06T03:49:29.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.085+0000 7fab9a59b700 1 --2- 172.21.15.55:0/1587796329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94069ec0 0x7fab940c6b10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fab9000b840 tx=0x7fab90009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:29.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.085+0000 7fab8a7fc700 1 -- 172.21.15.55:0/1587796329 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9000f510 con 0x7fab94069ec0 2023-10-06T03:49:29.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.085+0000 7fab8a7fc700 1 -- 172.21.15.55:0/1587796329 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9000f670 con 0x7fab94069ec0 2023-10-06T03:49:29.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.085+0000 7fab8a7fc700 1 -- 172.21.15.55:0/1587796329 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9000fa30 con 0x7fab94069ec0 2023-10-06T03:49:29.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.085+0000 7fab9b59d700 1 -- 172.21.15.55:0/1587796329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab940c5a70 con 0x7fab94069ec0 2023-10-06T03:49:29.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.085+0000 7fab9b59d700 1 -- 172.21.15.55:0/1587796329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab940c6030 con 0x7fab94069ec0 2023-10-06T03:49:29.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.086+0000 7fab9b59d700 1 -- 172.21.15.55:0/1587796329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab940c6960 con 0x7fab94069ec0 2023-10-06T03:49:29.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.087+0000 7fab8a7fc700 1 -- 172.21.15.55:0/1587796329 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 39) v1 ==== 84578+0+0 (secure 0 0 0) 0x7fab9001a400 con 0x7fab94069ec0 2023-10-06T03:49:29.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.088+0000 7fab8a7fc700 1 --2- 172.21.15.55:0/1587796329 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fab78066d80 0x7fab78069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:29.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.088+0000 7fab8a7fc700 1 -- 172.21.15.55:0/1587796329 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7095+0+0 (secure 0 0 0) 0x7fab900843f0 con 0x7fab94069ec0 2023-10-06T03:49:29.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.089+0000 7fab99d9a700 1 --2- 172.21.15.55:0/1587796329 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fab78066d80 0x7fab78069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:29.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.089+0000 7fab99d9a700 1 --2- 172.21.15.55:0/1587796329 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fab78066d80 0x7fab78069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fab84006fd0 tx=0x7fab84008040 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:29.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.092+0000 7fab8a7fc700 1 -- 172.21.15.55:0/1587796329 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab90051350 con 0x7fab94069ec0 2023-10-06T03:49:29.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:29.293+0000 7fab9b59d700 1 -- 172.21.15.55:0/1587796329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fab9404c3c0 con 0x7fab94069ec0 2023-10-06T03:49:30.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.032+0000 7fab8a7fc700 1 -- 172.21.15.55:0/1587796329 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fab9000fda0 con 0x7fab94069ec0 2023-10-06T03:49:30.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.032+0000 7fab8a7fc700 1 -- 172.21.15.55:0/1587796329 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) v1 ==== 181+0+0 (secure 0 0 0) 0x7fab90054de0 con 0x7fab94069ec0 2023-10-06T03:49:30.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.036+0000 7fab9b59d700 1 -- 172.21.15.55:0/1587796329 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fab78066d80 msgr2=0x7fab78069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:30.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.036+0000 7fab9b59d700 1 --2- 172.21.15.55:0/1587796329 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fab78066d80 0x7fab78069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fab84006fd0 tx=0x7fab84008040 comp rx=0 tx=0).stop 2023-10-06T03:49:30.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.036+0000 7fab9b59d700 1 -- 172.21.15.55:0/1587796329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94069ec0 msgr2=0x7fab940c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:30.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.036+0000 7fab9b59d700 1 --2- 172.21.15.55:0/1587796329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94069ec0 0x7fab940c6b10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fab9000b840 tx=0x7fab90009250 comp rx=0 tx=0).stop 2023-10-06T03:49:30.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.036+0000 7fab9b59d700 1 -- 172.21.15.55:0/1587796329 shutdown_connections 2023-10-06T03:49:30.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.036+0000 7fab9b59d700 1 --2- 172.21.15.55:0/1587796329 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fab78066d80 0x7fab78069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:30.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.036+0000 7fab9b59d700 1 --2- 172.21.15.55:0/1587796329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fab940cc970 0x7fab9416a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:30.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.036+0000 7fab9b59d700 1 --2- 172.21.15.55:0/1587796329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fab94069ec0 0x7fab940c6b10 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:30.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.036+0000 7fab9b59d700 1 --2- 172.21.15.55:0/1587796329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fab940cb560 0x7fab9414c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:30.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.036+0000 7fab9b59d700 1 -- 172.21.15.55:0/1587796329 >> 172.21.15.55:0/1587796329 conn(0x7fab9406baf0 msgr2=0x7fab940c4680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:30.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.036+0000 7fab9b59d700 1 -- 172.21.15.55:0/1587796329 shutdown_connections 2023-10-06T03:49:30.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.036+0000 7fab9b59d700 1 -- 172.21.15.55:0/1587796329 wait complete. 2023-10-06T03:49:30.040 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:49:30.059 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:49:30.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.253+0000 7f9eebfff700 1 Processor -- start 2023-10-06T03:49:30.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.253+0000 7f9eebfff700 1 -- start start 2023-10-06T03:49:30.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9eebfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9eec15b050 0x7f9eec0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:30.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9eebfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eec0cc8d0 0x7f9eec15b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:30.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9eebfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eec157a50 con 0x7f9eec15b960 2023-10-06T03:49:30.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9eebfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eec1591b0 con 0x7f9eec15b050 2023-10-06T03:49:30.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9eebfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0ca9e0 con 0x7f9eec0cc8d0 2023-10-06T03:49:30.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9eeb7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9eec15b960 0x7f9eec157f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42838/0 (socket says 172.21.15.55:42838) 2023-10-06T03:49:30.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9eeb7fe700 1 -- 172.21.15.55:0/1364689819 learned_addr learned my addr 172.21.15.55:0/1364689819 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:30.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9eea7fc700 1 --2- 172.21.15.55:0/1364689819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eec0cc8d0 0x7f9eec15b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:30.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9eeaffd700 1 --2- 172.21.15.55:0/1364689819 >> v2:172.21.15.163:3300/0 conn(0x7f9eec15b050 0x7f9eec0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:30.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9ee9ffb700 1 -- 172.21.15.55:0/1364689819 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904775816 0 0) 0x7f9eec157a50 con 0x7f9eec15b960 2023-10-06T03:49:30.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9ee9ffb700 1 -- 172.21.15.55:0/1364689819 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed4003570 con 0x7f9eec15b960 2023-10-06T03:49:30.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9eea7fc700 1 -- 172.21.15.55:0/1364689819 >> v2:172.21.15.163:3300/0 conn(0x7f9eec15b050 msgr2=0x7f9eec0ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:30.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9eea7fc700 1 --2- 172.21.15.55:0/1364689819 >> v2:172.21.15.163:3300/0 conn(0x7f9eec15b050 0x7f9eec0ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:30.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9eea7fc700 1 -- 172.21.15.55:0/1364689819 >> v1:172.21.15.55:6789/0 conn(0x7f9eec15b960 legacy=0x7f9eec157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:30.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.254+0000 7f9eea7fc700 1 -- 172.21.15.55:0/1364689819 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec0cacc0 con 0x7f9eec0cc8d0 2023-10-06T03:49:30.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.255+0000 7f9eea7fc700 1 --2- 172.21.15.55:0/1364689819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eec0cc8d0 0x7f9eec15b420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00098e0 tx=0x7f9ee00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8570dd8353fc6549 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:30.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.255+0000 7f9ee9ffb700 1 -- 172.21.15.55:0/1364689819 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0011630 con 0x7f9eec0cc8d0 2023-10-06T03:49:30.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.255+0000 7f9ee9ffb700 1 -- 172.21.15.55:0/1364689819 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee0011790 con 0x7f9eec0cc8d0 2023-10-06T03:49:30.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.255+0000 7f9ee9ffb700 1 -- 172.21.15.55:0/1364689819 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0011b30 con 0x7f9eec0cc8d0 2023-10-06T03:49:30.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.255+0000 7f9eebfff700 1 -- 172.21.15.55:0/1364689819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eec0cc8d0 msgr2=0x7f9eec15b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:30.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.255+0000 7f9eebfff700 1 --2- 172.21.15.55:0/1364689819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eec0cc8d0 0x7f9eec15b420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00098e0 tx=0x7f9ee00091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:30.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.256+0000 7f9eebfff700 1 -- 172.21.15.55:0/1364689819 shutdown_connections 2023-10-06T03:49:30.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.256+0000 7f9eebfff700 1 --2- 172.21.15.55:0/1364689819 >> v2:172.21.15.163:3300/0 conn(0x7f9eec15b050 0x7f9eec0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:30.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.256+0000 7f9eebfff700 1 --2- 172.21.15.55:0/1364689819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eec0cc8d0 0x7f9eec15b420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:30.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.256+0000 7f9eebfff700 1 -- 172.21.15.55:0/1364689819 >> 172.21.15.55:0/1364689819 conn(0x7f9eec068e20 msgr2=0x7f9eec0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:30.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.256+0000 7f9eebfff700 1 -- 172.21.15.55:0/1364689819 shutdown_connections 2023-10-06T03:49:30.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.256+0000 7f9eebfff700 1 -- 172.21.15.55:0/1364689819 wait complete. 2023-10-06T03:49:30.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.257+0000 7f9eebfff700 1 Processor -- start 2023-10-06T03:49:30.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.257+0000 7f9eebfff700 1 -- start start 2023-10-06T03:49:30.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.258+0000 7f9eebfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eec0cc8d0 0x7f9eec0af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:30.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.258+0000 7f9eebfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eec15b050 0x7f9eec168fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:30.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.258+0000 7f9eebfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9eec15b960 0x7f9eec0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:30.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.258+0000 7f9eebfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0c7280 con 0x7f9eec15b960 2023-10-06T03:49:30.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.258+0000 7f9eebfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec151520 con 0x7f9eec0cc8d0 2023-10-06T03:49:30.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.258+0000 7f9eebfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0c7ac0 con 0x7f9eec15b050 2023-10-06T03:49:30.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.259+0000 7f9eeb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9eec15b960 0x7f9eec0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:30.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.259+0000 7f9eeb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9eec15b960 0x7f9eec0694a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49088/0 (socket says 172.21.15.55:49088) 2023-10-06T03:49:30.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.259+0000 7f9eeb7fe700 1 -- 172.21.15.55:0/3920711062 learned_addr learned my addr 172.21.15.55:0/3920711062 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:30.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.259+0000 7f9eeaffd700 1 --2- 172.21.15.55:0/3920711062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eec0cc8d0 0x7f9eec0af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:30.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.259+0000 7f9eea7fc700 1 --2- 172.21.15.55:0/3920711062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eec15b050 0x7f9eec168fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:30.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.259+0000 7f9eeb7fe700 1 -- 172.21.15.55:0/3920711062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eec15b050 msgr2=0x7f9eec168fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:30.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.259+0000 7f9eeb7fe700 1 --2- 172.21.15.55:0/3920711062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eec15b050 0x7f9eec168fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:30.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.259+0000 7f9eeb7fe700 1 -- 172.21.15.55:0/3920711062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eec0cc8d0 msgr2=0x7f9eec0af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:30.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.259+0000 7f9eeb7fe700 1 --2- 172.21.15.55:0/3920711062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eec0cc8d0 0x7f9eec0af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:30.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.259+0000 7f9eeb7fe700 1 -- 172.21.15.55:0/3920711062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee0009420 con 0x7f9eec15b960 2023-10-06T03:49:30.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.259+0000 7f9eeb7fe700 1 --2- 172.21.15.55:0/3920711062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9eec15b960 0x7f9eec0694a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4001460 tx=0x7f9ee4001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:30.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.260+0000 7f9ed2ffd700 1 -- 172.21.15.55:0/3920711062 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee400a8e0 con 0x7f9eec15b960 2023-10-06T03:49:30.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.260+0000 7f9eebfff700 1 -- 172.21.15.55:0/3920711062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eec0af730 con 0x7f9eec15b960 2023-10-06T03:49:30.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.260+0000 7f9ed2ffd700 1 -- 172.21.15.55:0/3920711062 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee400aa40 con 0x7f9eec15b960 2023-10-06T03:49:30.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.260+0000 7f9ed2ffd700 1 -- 172.21.15.55:0/3920711062 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee400adc0 con 0x7f9eec15b960 2023-10-06T03:49:30.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.260+0000 7f9eebfff700 1 -- 172.21.15.55:0/3920711062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eec0c7750 con 0x7f9eec15b960 2023-10-06T03:49:30.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.261+0000 7f9eebfff700 1 -- 172.21.15.55:0/3920711062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eec0c7c30 con 0x7f9eec15b960 2023-10-06T03:49:30.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.262+0000 7f9ed2ffd700 1 -- 172.21.15.55:0/3920711062 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ee4007300 con 0x7f9eec15b960 2023-10-06T03:49:30.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.262+0000 7f9ed2ffd700 1 --2- 172.21.15.55:0/3920711062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9ed8066f30 0x7f9ed80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:30.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.263+0000 7f9ed2ffd700 1 -- 172.21.15.55:0/3920711062 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6657+0+0 (secure 0 0 0) 0x7f9ee4084e90 con 0x7f9eec15b960 2023-10-06T03:49:30.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.263+0000 7f9eeaffd700 1 --2- 172.21.15.55:0/3920711062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9ed8066f30 0x7f9ed80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:30.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.264+0000 7f9eeaffd700 1 --2- 172.21.15.55:0/3920711062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9ed8066f30 0x7f9ed80693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009db0 tx=0x7f9edc0091a0 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:30.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.267+0000 7f9ed2ffd700 1 -- 172.21.15.55:0/3920711062 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ee4051c10 con 0x7f9eec15b960 2023-10-06T03:49:30.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:30.468+0000 7f9eebfff700 1 -- 172.21.15.55:0/3920711062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9eec04c3c0 con 0x7f9eec15b960 2023-10-06T03:49:31.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.035+0000 7f9ed2ffd700 1 -- 172.21.15.55:0/3920711062 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9ee4086020 con 0x7f9eec15b960 2023-10-06T03:49:31.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.039+0000 7f9eebfff700 1 -- 172.21.15.55:0/3920711062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9ed8066f30 msgr2=0x7f9ed80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:31.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.039+0000 7f9eebfff700 1 --2- 172.21.15.55:0/3920711062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9ed8066f30 0x7f9ed80693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009db0 tx=0x7f9edc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.039+0000 7f9eebfff700 1 -- 172.21.15.55:0/3920711062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9eec15b960 msgr2=0x7f9eec0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:31.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.039+0000 7f9eebfff700 1 --2- 172.21.15.55:0/3920711062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9eec15b960 0x7f9eec0694a0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4001460 tx=0x7f9ee4001ae0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.039+0000 7f9eebfff700 1 -- 172.21.15.55:0/3920711062 shutdown_connections 2023-10-06T03:49:31.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.039+0000 7f9eebfff700 1 --2- 172.21.15.55:0/3920711062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9ed8066f30 0x7f9ed80693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.039+0000 7f9eebfff700 1 --2- 172.21.15.55:0/3920711062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9eec15b960 0x7f9eec0694a0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.039+0000 7f9eebfff700 1 --2- 172.21.15.55:0/3920711062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9eec0cc8d0 0x7f9eec0af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.039+0000 7f9eebfff700 1 --2- 172.21.15.55:0/3920711062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9eec15b050 0x7f9eec168fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.039+0000 7f9eebfff700 1 -- 172.21.15.55:0/3920711062 >> 172.21.15.55:0/3920711062 conn(0x7f9eec068e20 msgr2=0x7f9eec06ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:31.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.039+0000 7f9eebfff700 1 -- 172.21.15.55:0/3920711062 shutdown_connections 2023-10-06T03:49:31.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.039+0000 7f9eebfff700 1 -- 172.21.15.55:0/3920711062 wait complete. 2023-10-06T03:49:31.042 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2023-10-06T03:49:31.061 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:49:31.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.253+0000 7f8d61d9d700 1 Processor -- start 2023-10-06T03:49:31.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.254+0000 7f8d61d9d700 1 -- start start 2023-10-06T03:49:31.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.255+0000 7f8d61d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8d5c06f060 0x7f8d5c0ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.255+0000 7f8d61d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d5c06f4b0 0x7f8d5c0ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.255+0000 7f8d61d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5c06ebd0 con 0x7f8d5c0cb1a0 2023-10-06T03:49:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.255+0000 7f8d61d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d5c15b2e0 con 0x7f8d5c06f060 2023-10-06T03:49:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.255+0000 7f8d61d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c15aa70 con 0x7f8d5c06f4b0 2023-10-06T03:49:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.255+0000 7f8d60d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8d5c0cb1a0 0x7f8d5c15a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42854/0 (socket says 172.21.15.55:42854) 2023-10-06T03:49:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.255+0000 7f8d60d9b700 1 -- 172.21.15.55:0/296038748 learned_addr learned my addr 172.21.15.55:0/296038748 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.255+0000 7f8d5bfff700 1 --2- 172.21.15.55:0/296038748 >> v2:172.21.15.55:3300/0 conn(0x7f8d5c06f060 0x7f8d5c0ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.255+0000 7f8d5b7fe700 1 -- 172.21.15.55:0/296038748 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426196485 0 0) 0x7f8d5c06ebd0 con 0x7f8d5c0cb1a0 2023-10-06T03:49:31.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.256+0000 7f8d6159c700 1 --2- 172.21.15.55:0/296038748 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d5c06f4b0 0x7f8d5c0ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.256+0000 7f8d5b7fe700 1 -- 172.21.15.55:0/296038748 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d4c003570 con 0x7f8d5c0cb1a0 2023-10-06T03:49:31.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.256+0000 7f8d5bfff700 1 -- 172.21.15.55:0/296038748 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d5c06f4b0 msgr2=0x7f8d5c0ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:31.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.256+0000 7f8d5bfff700 1 --2- 172.21.15.55:0/296038748 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d5c06f4b0 0x7f8d5c0ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.256+0000 7f8d5bfff700 1 -- 172.21.15.55:0/296038748 >> v1:172.21.15.55:6789/0 conn(0x7f8d5c0cb1a0 legacy=0x7f8d5c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:31.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.256+0000 7f8d5bfff700 1 -- 172.21.15.55:0/296038748 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c15ad20 con 0x7f8d5c06f060 2023-10-06T03:49:31.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.256+0000 7f8d5bfff700 1 --2- 172.21.15.55:0/296038748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d5c06f060 0x7f8d5c0ae0d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8d480098e0 tx=0x7f8d480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d297015da55ca607 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:31.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.256+0000 7f8d5b7fe700 1 -- 172.21.15.55:0/296038748 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4800b6d0 con 0x7f8d5c06f060 2023-10-06T03:49:31.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.256+0000 7f8d5b7fe700 1 -- 172.21.15.55:0/296038748 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4800b830 con 0x7f8d5c06f060 2023-10-06T03:49:31.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.256+0000 7f8d5b7fe700 1 -- 172.21.15.55:0/296038748 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4800bbd0 con 0x7f8d5c06f060 2023-10-06T03:49:31.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.257+0000 7f8d61d9d700 1 -- 172.21.15.55:0/296038748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d5c06f060 msgr2=0x7f8d5c0ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:31.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.257+0000 7f8d61d9d700 1 --2- 172.21.15.55:0/296038748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d5c06f060 0x7f8d5c0ae0d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8d480098e0 tx=0x7f8d480091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.257+0000 7f8d61d9d700 1 -- 172.21.15.55:0/296038748 shutdown_connections 2023-10-06T03:49:31.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.257+0000 7f8d61d9d700 1 --2- 172.21.15.55:0/296038748 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d5c06f4b0 0x7f8d5c0ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.257+0000 7f8d61d9d700 1 --2- 172.21.15.55:0/296038748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d5c06f060 0x7f8d5c0ae0d0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.257+0000 7f8d61d9d700 1 -- 172.21.15.55:0/296038748 >> 172.21.15.55:0/296038748 conn(0x7f8d5c0cc1e0 msgr2=0x7f8d5c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:31.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.257+0000 7f8d61d9d700 1 -- 172.21.15.55:0/296038748 shutdown_connections 2023-10-06T03:49:31.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.257+0000 7f8d61d9d700 1 -- 172.21.15.55:0/296038748 wait complete. 2023-10-06T03:49:31.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.258+0000 7f8d61d9d700 1 Processor -- start 2023-10-06T03:49:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.259+0000 7f8d61d9d700 1 -- start start 2023-10-06T03:49:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.259+0000 7f8d61d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d5c06f060 0x7f8d5c0c4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.259+0000 7f8d61d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d5c06f4b0 0x7f8d5c16abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.259+0000 7f8d61d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d5c0cb1a0 0x7f8d5c153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d61d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c16cdf0 con 0x7f8d5c06f4b0 2023-10-06T03:49:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d61d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0c4f20 con 0x7f8d5c0cb1a0 2023-10-06T03:49:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d61d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0c5200 con 0x7f8d5c06f060 2023-10-06T03:49:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d5bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d5c06f4b0 0x7f8d5c16abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d5bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d5c06f4b0 0x7f8d5c16abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49098/0 (socket says 172.21.15.55:49098) 2023-10-06T03:49:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d5bfff700 1 -- 172.21.15.55:0/1737634098 learned_addr learned my addr 172.21.15.55:0/1737634098 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d60d9b700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d5c06f060 0x7f8d5c0c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d6159c700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d5c0cb1a0 0x7f8d5c153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d5bfff700 1 -- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d5c06f060 msgr2=0x7f8d5c0c4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d5bfff700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d5c06f060 0x7f8d5c0c4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d5bfff700 1 -- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d5c0cb1a0 msgr2=0x7f8d5c153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d5bfff700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d5c0cb1a0 0x7f8d5c153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d5bfff700 1 -- 172.21.15.55:0/1737634098 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48009420 con 0x7f8d5c06f4b0 2023-10-06T03:49:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d6159c700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d5c0cb1a0 0x7f8d5c153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.260+0000 7f8d5bfff700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d5c06f4b0 0x7f8d5c16abd0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c06ce80 tx=0x7f8d48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.261+0000 7f8d58ff9700 1 -- 172.21.15.55:0/1737634098 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4801be20 con 0x7f8d5c06f4b0 2023-10-06T03:49:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.261+0000 7f8d58ff9700 1 -- 172.21.15.55:0/1737634098 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d48011420 con 0x7f8d5c06f4b0 2023-10-06T03:49:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.261+0000 7f8d61d9d700 1 -- 172.21.15.55:0/1737634098 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5c0c0f80 con 0x7f8d5c06f4b0 2023-10-06T03:49:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.261+0000 7f8d58ff9700 1 -- 172.21.15.55:0/1737634098 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d480117e0 con 0x7f8d5c06f4b0 2023-10-06T03:49:31.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.261+0000 7f8d61d9d700 1 -- 172.21.15.55:0/1737634098 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5c0c1540 con 0x7f8d5c06f4b0 2023-10-06T03:49:31.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.262+0000 7f8d61d9d700 1 -- 172.21.15.55:0/1737634098 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5c0c1730 con 0x7f8d5c06f4b0 2023-10-06T03:49:31.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.263+0000 7f8d58ff9700 1 -- 172.21.15.55:0/1737634098 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8d48011b90 con 0x7f8d5c06f4b0 2023-10-06T03:49:31.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.264+0000 7f8d58ff9700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8d40066f30 0x7f8d400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.264+0000 7f8d58ff9700 1 -- 172.21.15.55:0/1737634098 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f8d48085fa0 con 0x7f8d5c06f4b0 2023-10-06T03:49:31.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.264+0000 7f8d60d9b700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8d40066f30 0x7f8d400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.265+0000 7f8d60d9b700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8d40066f30 0x7f8d400693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8d500058d0 tx=0x7f8d50009040 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:31.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.269+0000 7f8d58ff9700 1 -- 172.21.15.55:0/1737634098 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d48053000 con 0x7f8d5c06f4b0 2023-10-06T03:49:31.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.473+0000 7f8d61d9d700 1 -- 172.21.15.55:0/1737634098 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8d5c06c480 con 0x7f8d5c06f4b0 2023-10-06T03:49:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.478+0000 7f8d58ff9700 1 -- 172.21.15.55:0/1737634098 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8d48056620 con 0x7f8d5c06f4b0 2023-10-06T03:49:31.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.480+0000 7f8d61d9d700 1 -- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8d40066f30 msgr2=0x7f8d400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:31.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.480+0000 7f8d61d9d700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8d40066f30 0x7f8d400693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8d500058d0 tx=0x7f8d50009040 comp rx=0 tx=0).stop 2023-10-06T03:49:31.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.480+0000 7f8d61d9d700 1 -- 172.21.15.55:0/1737634098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d5c06f4b0 msgr2=0x7f8d5c16abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:31.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.480+0000 7f8d61d9d700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d5c06f4b0 0x7f8d5c16abd0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c06ce80 tx=0x7f8d48017000 comp rx=0 tx=0).stop 2023-10-06T03:49:31.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.480+0000 7f8d61d9d700 1 -- 172.21.15.55:0/1737634098 shutdown_connections 2023-10-06T03:49:31.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.480+0000 7f8d61d9d700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8d40066f30 0x7f8d400693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.480+0000 7f8d61d9d700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8d5c0cb1a0 0x7f8d5c153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.480+0000 7f8d61d9d700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8d5c06f060 0x7f8d5c0c4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.480+0000 7f8d61d9d700 1 --2- 172.21.15.55:0/1737634098 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8d5c06f4b0 0x7f8d5c16abd0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.480+0000 7f8d61d9d700 1 -- 172.21.15.55:0/1737634098 >> 172.21.15.55:0/1737634098 conn(0x7f8d5c0cc1e0 msgr2=0x7f8d5c0c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:31.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.480+0000 7f8d61d9d700 1 -- 172.21.15.55:0/1737634098 shutdown_connections 2023-10-06T03:49:31.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.480+0000 7f8d61d9d700 1 -- 172.21.15.55:0/1737634098 wait complete. 2023-10-06T03:49:31.499 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T03:49:31.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.687+0000 7fec65f9d700 1 Processor -- start 2023-10-06T03:49:31.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.687+0000 7fec65f9d700 1 -- start start 2023-10-06T03:49:31.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.688+0000 7fec65f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fec6015a3b0 0x7fec6015a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.688+0000 7fec65f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec600aef00 0x7fec6015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.688+0000 7fec65f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec600aea70 con 0x7fec6006e470 2023-10-06T03:49:31.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.688+0000 7fec65f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec600ae480 con 0x7fec6015a3b0 2023-10-06T03:49:31.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.688+0000 7fec65f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6015acc0 con 0x7fec600aef00 2023-10-06T03:49:31.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.689+0000 7fec6579c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fec6006e470 0x7fec600c7280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42868/0 (socket says 172.21.15.55:42868) 2023-10-06T03:49:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.689+0000 7fec6579c700 1 -- 172.21.15.55:0/546877137 learned_addr learned my addr 172.21.15.55:0/546877137 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.689+0000 7fec5ffff700 1 --2- 172.21.15.55:0/546877137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec600aef00 0x7fec6015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.689+0000 7fec64f9b700 1 --2- 172.21.15.55:0/546877137 >> v2:172.21.15.163:3300/0 conn(0x7fec6015a3b0 0x7fec6015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.689+0000 7fec5f7fe700 1 -- 172.21.15.55:0/546877137 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837672819 0 0) 0x7fec600aea70 con 0x7fec6006e470 2023-10-06T03:49:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.689+0000 7fec5f7fe700 1 -- 172.21.15.55:0/546877137 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec50003570 con 0x7fec6006e470 2023-10-06T03:49:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.689+0000 7fec5f7fe700 1 -- 172.21.15.55:0/546877137 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2863027999 0 0) 0x7fec50003570 con 0x7fec6006e470 2023-10-06T03:49:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.689+0000 7fec5f7fe700 1 -- 172.21.15.55:0/546877137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec600aef00 msgr2=0x7fec6015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.689+0000 7fec5f7fe700 1 --2- 172.21.15.55:0/546877137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec600aef00 0x7fec6015d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.689+0000 7fec5f7fe700 1 -- 172.21.15.55:0/546877137 >> v2:172.21.15.163:3300/0 conn(0x7fec6015a3b0 msgr2=0x7fec6015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.689+0000 7fec5f7fe700 1 --2- 172.21.15.55:0/546877137 >> v2:172.21.15.163:3300/0 conn(0x7fec6015a3b0 0x7fec6015a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec5f7fe700 1 -- 172.21.15.55:0/546877137 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6015dd30 con 0x7fec6006e470 2023-10-06T03:49:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec5f7fe700 1 -- 172.21.15.55:0/546877137 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fec580017f0 con 0x7fec6006e470 2023-10-06T03:49:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec5f7fe700 1 -- 172.21.15.55:0/546877137 >> v1:172.21.15.55:6789/0 conn(0x7fec6006e470 legacy=0x7fec600c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec5f7fe700 1 --2- 172.21.15.55:0/546877137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec50003a60 0x7fec50003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec5f7fe700 1 --2- 172.21.15.55:0/546877137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec600aef00 0x7fec50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec5f7fe700 1 --2- 172.21.15.55:0/546877137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec50004d50 0x7fec500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec5f7fe700 1 -- 172.21.15.55:0/546877137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec600ae480 con 0x7fec50003a60 2023-10-06T03:49:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec5f7fe700 1 -- 172.21.15.55:0/546877137 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6015acc0 con 0x7fec50004d50 2023-10-06T03:49:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec5f7fe700 1 -- 172.21.15.55:0/546877137 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec5000ba60 con 0x7fec600aef00 2023-10-06T03:49:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec64f9b700 1 --2- 172.21.15.55:0/546877137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec50003a60 0x7fec50003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec5ffff700 1 --2- 172.21.15.55:0/546877137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec600aef00 0x7fec50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec6579c700 1 --2- 172.21.15.55:0/546877137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec50004d50 0x7fec500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec64f9b700 1 -- 172.21.15.55:0/546877137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec600aef00 msgr2=0x7fec50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec64f9b700 1 --2- 172.21.15.55:0/546877137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec600aef00 0x7fec50004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec64f9b700 1 -- 172.21.15.55:0/546877137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec50004d50 msgr2=0x7fec500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec64f9b700 1 --2- 172.21.15.55:0/546877137 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec50004d50 0x7fec500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec64f9b700 1 -- 172.21.15.55:0/546877137 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec50003570 con 0x7fec50003a60 2023-10-06T03:49:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.690+0000 7fec5ffff700 1 --2- 172.21.15.55:0/546877137 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec600aef00 0x7fec50004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.691+0000 7fec64f9b700 1 --2- 172.21.15.55:0/546877137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec50003a60 0x7fec50003e70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fec54009be0 tx=0x7fec540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6639859afdc86 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.691+0000 7fec5f7fe700 1 -- 172.21.15.55:0/546877137 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5400e6b0 con 0x7fec50003a60 2023-10-06T03:49:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.691+0000 7fec5f7fe700 1 -- 172.21.15.55:0/546877137 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5400e810 con 0x7fec50003a60 2023-10-06T03:49:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.691+0000 7fec5f7fe700 1 -- 172.21.15.55:0/546877137 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5400ebd0 con 0x7fec50003a60 2023-10-06T03:49:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.691+0000 7fec65f9d700 1 -- 172.21.15.55:0/546877137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec50003a60 msgr2=0x7fec50003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.691+0000 7fec65f9d700 1 --2- 172.21.15.55:0/546877137 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec50003a60 0x7fec50003e70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fec54009be0 tx=0x7fec540091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.691+0000 7fec6579c700 1 -- 172.21.15.55:0/546877137 reap_dead start 2023-10-06T03:49:31.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.691+0000 7fec65f9d700 1 -- 172.21.15.55:0/546877137 shutdown_connections 2023-10-06T03:49:31.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.692+0000 7fec65f9d700 1 -- 172.21.15.55:0/546877137 >> 172.21.15.55:0/546877137 conn(0x7fec600c9f70 msgr2=0x7fec6006dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:31.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.692+0000 7fec65f9d700 1 -- 172.21.15.55:0/546877137 shutdown_connections 2023-10-06T03:49:31.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.692+0000 7fec65f9d700 1 -- 172.21.15.55:0/546877137 wait complete. 2023-10-06T03:49:31.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.693+0000 7fec65f9d700 1 Processor -- start 2023-10-06T03:49:31.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.693+0000 7fec65f9d700 1 -- start start 2023-10-06T03:49:31.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.694+0000 7fec65f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec50003a60 0x7fec601518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.694+0000 7fec65f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec600c4f70 0x7fec600cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.694+0000 7fec65f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec6016d6f0 0x7fec600cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.694+0000 7fec65f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec600c56e0 con 0x7fec50003a60 2023-10-06T03:49:31.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.694+0000 7fec65f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6016d260 con 0x7fec6016d6f0 2023-10-06T03:49:31.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.694+0000 7fec65f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec600c5400 con 0x7fec600c4f70 2023-10-06T03:49:31.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.694+0000 7fec64f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec50003a60 0x7fec601518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.694+0000 7fec64f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec50003a60 0x7fec601518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49120/0 (socket says 172.21.15.55:49120) 2023-10-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.694+0000 7fec64f9b700 1 -- 172.21.15.55:0/3849947887 learned_addr learned my addr 172.21.15.55:0/3849947887 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.695+0000 7fec6579c700 1 --2- 172.21.15.55:0/3849947887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec6016d6f0 0x7fec600cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.695+0000 7fec5ffff700 1 --2- 172.21.15.55:0/3849947887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec600c4f70 0x7fec600cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.695+0000 7fec64f9b700 1 -- 172.21.15.55:0/3849947887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec600c4f70 msgr2=0x7fec600cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.695+0000 7fec64f9b700 1 --2- 172.21.15.55:0/3849947887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec600c4f70 0x7fec600cb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.695+0000 7fec64f9b700 1 -- 172.21.15.55:0/3849947887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec6016d6f0 msgr2=0x7fec600cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.695+0000 7fec64f9b700 1 --2- 172.21.15.55:0/3849947887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec6016d6f0 0x7fec600cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.695+0000 7fec64f9b700 1 -- 172.21.15.55:0/3849947887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6016cc10 con 0x7fec50003a60 2023-10-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.695+0000 7fec64f9b700 1 --2- 172.21.15.55:0/3849947887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec50003a60 0x7fec601518d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fec54009700 tx=0x7fec54019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:31.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.695+0000 7fec5cff9700 1 -- 172.21.15.55:0/3849947887 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5400e6b0 con 0x7fec50003a60 2023-10-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.695+0000 7fec5cff9700 1 -- 172.21.15.55:0/3849947887 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec54021040 con 0x7fec50003a60 2023-10-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.695+0000 7fec65f9d700 1 -- 172.21.15.55:0/3849947887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6016cef0 con 0x7fec50003a60 2023-10-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.695+0000 7fec5cff9700 1 -- 172.21.15.55:0/3849947887 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54025640 con 0x7fec50003a60 2023-10-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.696+0000 7fec65f9d700 1 -- 172.21.15.55:0/3849947887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec601527a0 con 0x7fec50003a60 2023-10-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.697+0000 7fec65f9d700 1 -- 172.21.15.55:0/3849947887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6016ca70 con 0x7fec50003a60 2023-10-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.697+0000 7fec5cff9700 1 -- 172.21.15.55:0/3849947887 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec54025a70 con 0x7fec50003a60 2023-10-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.698+0000 7fec5cff9700 1 --2- 172.21.15.55:0/3849947887 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fec44066f30 0x7fec440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.698+0000 7fec5cff9700 1 -- 172.21.15.55:0/3849947887 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6243+0+0 (secure 0 0 0) 0x7fec54086750 con 0x7fec50003a60 2023-10-06T03:49:31.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.699+0000 7fec5ffff700 1 --2- 172.21.15.55:0/3849947887 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fec44066f30 0x7fec440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:31.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.699+0000 7fec5ffff700 1 --2- 172.21.15.55:0/3849947887 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fec44066f30 0x7fec440693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fec60153050 tx=0x7fec4c008040 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:31.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.702+0000 7fec5cff9700 1 -- 172.21.15.55:0/3849947887 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec6016ca70 con 0x7fec50003a60 2023-10-06T03:49:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.904+0000 7fec65f9d700 1 -- 172.21.15.55:0/3849947887 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fec6016d540 con 0x7fec50003a60 2023-10-06T03:49:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.909+0000 7fec5cff9700 1 -- 172.21.15.55:0/3849947887 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v32) v1 ==== 115+0+0 (secure 0 0 0) 0x7fec540537a0 con 0x7fec50003a60 2023-10-06T03:49:31.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.912+0000 7fec65f9d700 1 -- 172.21.15.55:0/3849947887 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fec44066f30 msgr2=0x7fec440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.912+0000 7fec65f9d700 1 --2- 172.21.15.55:0/3849947887 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fec44066f30 0x7fec440693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fec60153050 tx=0x7fec4c008040 comp rx=0 tx=0).stop 2023-10-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.912+0000 7fec65f9d700 1 -- 172.21.15.55:0/3849947887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec50003a60 msgr2=0x7fec601518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.912+0000 7fec65f9d700 1 --2- 172.21.15.55:0/3849947887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec50003a60 0x7fec601518d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fec54009700 tx=0x7fec54019000 comp rx=0 tx=0).stop 2023-10-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.912+0000 7fec65f9d700 1 -- 172.21.15.55:0/3849947887 shutdown_connections 2023-10-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.912+0000 7fec65f9d700 1 --2- 172.21.15.55:0/3849947887 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fec44066f30 0x7fec440693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.912+0000 7fec65f9d700 1 --2- 172.21.15.55:0/3849947887 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec6016d6f0 0x7fec600cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.912+0000 7fec65f9d700 1 --2- 172.21.15.55:0/3849947887 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec50003a60 0x7fec601518d0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.912+0000 7fec65f9d700 1 --2- 172.21.15.55:0/3849947887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec600c4f70 0x7fec600cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.912+0000 7fec65f9d700 1 -- 172.21.15.55:0/3849947887 >> 172.21.15.55:0/3849947887 conn(0x7fec600c9f70 msgr2=0x7fec60153580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.912+0000 7fec65f9d700 1 -- 172.21.15.55:0/3849947887 shutdown_connections 2023-10-06T03:49:31.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:31.912+0000 7fec65f9d700 1 -- 172.21.15.55:0/3849947887 wait complete. 2023-10-06T03:49:31.931 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T03:49:32.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.119+0000 7f298d39d700 1 Processor -- start 2023-10-06T03:49:32.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.120+0000 7f298d39d700 1 -- start start 2023-10-06T03:49:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.121+0000 7f298d39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29880cb1a0 0x7f298815a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.121+0000 7f298d39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29880ae0d0 0x7f29880ae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.121+0000 7f298d39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298806ebd0 con 0x7f298815a8a0 2023-10-06T03:49:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.121+0000 7f298d39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298815b2e0 con 0x7f29880ae0d0 2023-10-06T03:49:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.121+0000 7f298d39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29880aea50 con 0x7f29880cb1a0 2023-10-06T03:49:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.121+0000 7f298cb9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29880ae0d0 0x7f29880ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.121+0000 7f298cb9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29880ae0d0 0x7f29880ae510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49132/0 (socket says 172.21.15.55:49132) 2023-10-06T03:49:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.121+0000 7f298cb9c700 1 -- 172.21.15.55:0/2982756300 learned_addr learned my addr 172.21.15.55:0/2982756300 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.121+0000 7f29877fe700 1 --1- 172.21.15.55:0/2982756300 >> v1:172.21.15.55:6789/0 conn(0x7f298815a8a0 0x7f298806f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42884/0 (socket says 172.21.15.55:42884) 2023-10-06T03:49:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.121+0000 7f2987fff700 1 --2- 172.21.15.55:0/2982756300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29880cb1a0 0x7f298815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.122+0000 7f298cb9c700 1 -- 172.21.15.55:0/2982756300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29880cb1a0 msgr2=0x7f298815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.122+0000 7f298cb9c700 1 --2- 172.21.15.55:0/2982756300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29880cb1a0 0x7f298815a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.122+0000 7f298cb9c700 1 -- 172.21.15.55:0/2982756300 >> v1:172.21.15.55:6789/0 conn(0x7f298815a8a0 legacy=0x7f298806f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.122+0000 7f298cb9c700 1 -- 172.21.15.55:0/2982756300 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298815ad20 con 0x7f29880ae0d0 2023-10-06T03:49:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.122+0000 7f298cb9c700 1 --2- 172.21.15.55:0/2982756300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29880ae0d0 0x7f29880ae510 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f298000d1a0 tx=0x7f298000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf42ba5ceeaa4742 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.122+0000 7f2986ffd700 1 -- 172.21.15.55:0/2982756300 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980010070 con 0x7f29880ae0d0 2023-10-06T03:49:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.122+0000 7f2986ffd700 1 -- 172.21.15.55:0/2982756300 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298000b0d0 con 0x7f29880ae0d0 2023-10-06T03:49:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.122+0000 7f2986ffd700 1 -- 172.21.15.55:0/2982756300 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298000b490 con 0x7f29880ae0d0 2023-10-06T03:49:32.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.123+0000 7f298d39d700 1 -- 172.21.15.55:0/2982756300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29880ae0d0 msgr2=0x7f29880ae510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:32.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.123+0000 7f298d39d700 1 --2- 172.21.15.55:0/2982756300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29880ae0d0 0x7f29880ae510 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f298000d1a0 tx=0x7f298000ba50 comp rx=0 tx=0).stop 2023-10-06T03:49:32.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.123+0000 7f298d39d700 1 -- 172.21.15.55:0/2982756300 shutdown_connections 2023-10-06T03:49:32.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.123+0000 7f298d39d700 1 --2- 172.21.15.55:0/2982756300 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29880ae0d0 0x7f29880ae510 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:32.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.123+0000 7f298d39d700 1 --2- 172.21.15.55:0/2982756300 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29880cb1a0 0x7f298815a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.123+0000 7f298d39d700 1 -- 172.21.15.55:0/2982756300 >> 172.21.15.55:0/2982756300 conn(0x7f29880cc1e0 msgr2=0x7f29880cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.123+0000 7f298d39d700 1 -- 172.21.15.55:0/2982756300 shutdown_connections 2023-10-06T03:49:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.123+0000 7f298d39d700 1 -- 172.21.15.55:0/2982756300 wait complete. 2023-10-06T03:49:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.124+0000 7f298d39d700 1 Processor -- start 2023-10-06T03:49:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.124+0000 7f298d39d700 1 -- start start 2023-10-06T03:49:32.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.125+0000 7f298d39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29880ae0d0 0x7f29880c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:32.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.125+0000 7f298d39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29880cb1a0 0x7f29880ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:32.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.125+0000 7f298d39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29880af960 0x7f29881579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:32.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.125+0000 7f298d39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29880c5520 con 0x7f29880ae0d0 2023-10-06T03:49:32.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.125+0000 7f298d39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29880ade50 con 0x7f29880af960 2023-10-06T03:49:32.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.125+0000 7f298d39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2988157550 con 0x7f29880cb1a0 2023-10-06T03:49:32.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.126+0000 7f2987fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29880ae0d0 0x7f29880c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:32.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.126+0000 7f2987fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29880ae0d0 0x7f29880c4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49134/0 (socket says 172.21.15.55:49134) 2023-10-06T03:49:32.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.126+0000 7f2987fff700 1 -- 172.21.15.55:0/1673118781 learned_addr learned my addr 172.21.15.55:0/1673118781 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:32.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.126+0000 7f2987fff700 1 -- 172.21.15.55:0/1673118781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29880cb1a0 msgr2=0x7f29880ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:32.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.126+0000 7f2987fff700 1 --2- 172.21.15.55:0/1673118781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29880cb1a0 0x7f29880ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:32.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.126+0000 7f2987fff700 1 -- 172.21.15.55:0/1673118781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29880af960 msgr2=0x7f29881579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:32.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.126+0000 7f2987fff700 1 --2- 172.21.15.55:0/1673118781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29880af960 0x7f29881579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:32.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.126+0000 7f2987fff700 1 -- 172.21.15.55:0/1673118781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298000bcf0 con 0x7f29880ae0d0 2023-10-06T03:49:32.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.126+0000 7f2987fff700 1 --2- 172.21.15.55:0/1673118781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29880ae0d0 0x7f29880c4bb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f297800d890 tx=0x7f2978008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:32.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.126+0000 7f296ffff700 1 -- 172.21.15.55:0/1673118781 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297800b600 con 0x7f29880ae0d0 2023-10-06T03:49:32.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.126+0000 7f296ffff700 1 -- 172.21.15.55:0/1673118781 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297800b760 con 0x7f29880ae0d0 2023-10-06T03:49:32.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.126+0000 7f296ffff700 1 -- 172.21.15.55:0/1673118781 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297800bae0 con 0x7f29880ae0d0 2023-10-06T03:49:32.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.127+0000 7f298d39d700 1 -- 172.21.15.55:0/1673118781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29880c5240 con 0x7f29880ae0d0 2023-10-06T03:49:32.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.127+0000 7f298d39d700 1 -- 172.21.15.55:0/1673118781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29881571e0 con 0x7f29880ae0d0 2023-10-06T03:49:32.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.128+0000 7f298d39d700 1 -- 172.21.15.55:0/1673118781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2988156d60 con 0x7f29880ae0d0 2023-10-06T03:49:32.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.128+0000 7f296ffff700 1 -- 172.21.15.55:0/1673118781 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f297801e460 con 0x7f29880ae0d0 2023-10-06T03:49:32.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.129+0000 7f296ffff700 1 --2- 172.21.15.55:0/1673118781 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2974066f30 0x7f29740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:32.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.129+0000 7f296ffff700 1 -- 172.21.15.55:0/1673118781 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f2978084ed0 con 0x7f29880ae0d0 2023-10-06T03:49:32.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.130+0000 7f29877fe700 1 --2- 172.21.15.55:0/1673118781 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2974066f30 0x7f29740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:32.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.130+0000 7f29877fe700 1 --2- 172.21.15.55:0/1673118781 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2974066f30 0x7f29740693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29880c4b00 tx=0x7f297c007b30 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:32.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.133+0000 7f296ffff700 1 -- 172.21.15.55:0/1673118781 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2988156d60 con 0x7f29880ae0d0 2023-10-06T03:49:32.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.339+0000 7f298d39d700 1 -- 172.21.15.55:0/1673118781 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f2988156d60 con 0x7f29880ae0d0 2023-10-06T03:49:32.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.343+0000 7f296ffff700 1 -- 172.21.15.55:0/1673118781 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v33) v1 ==== 120+0+0 (secure 0 0 0) 0x7f2988156d60 con 0x7f29880ae0d0 2023-10-06T03:49:32.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.346+0000 7f298d39d700 1 -- 172.21.15.55:0/1673118781 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2974066f30 msgr2=0x7f29740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.346+0000 7f298d39d700 1 --2- 172.21.15.55:0/1673118781 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2974066f30 0x7f29740693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29880c4b00 tx=0x7f297c007b30 comp rx=0 tx=0).stop 2023-10-06T03:49:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.346+0000 7f298d39d700 1 -- 172.21.15.55:0/1673118781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29880ae0d0 msgr2=0x7f29880c4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.346+0000 7f298d39d700 1 --2- 172.21.15.55:0/1673118781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29880ae0d0 0x7f29880c4bb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f297800d890 tx=0x7f2978008040 comp rx=0 tx=0).stop 2023-10-06T03:49:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.346+0000 7f298d39d700 1 -- 172.21.15.55:0/1673118781 shutdown_connections 2023-10-06T03:49:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.346+0000 7f298d39d700 1 --2- 172.21.15.55:0/1673118781 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2974066f30 0x7f29740693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.346+0000 7f298d39d700 1 --2- 172.21.15.55:0/1673118781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29880af960 0x7f29881579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.346+0000 7f298d39d700 1 --2- 172.21.15.55:0/1673118781 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29880ae0d0 0x7f29880c4bb0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.346+0000 7f298d39d700 1 --2- 172.21.15.55:0/1673118781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29880cb1a0 0x7f29880ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.346+0000 7f298d39d700 1 -- 172.21.15.55:0/1673118781 >> 172.21.15.55:0/1673118781 conn(0x7f29880cc1e0 msgr2=0x7f298806ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.346+0000 7f298d39d700 1 -- 172.21.15.55:0/1673118781 shutdown_connections 2023-10-06T03:49:32.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.346+0000 7f298d39d700 1 -- 172.21.15.55:0/1673118781 wait complete. 2023-10-06T03:49:32.365 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-06T03:49:32.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.550+0000 7f735199d700 1 Processor -- start 2023-10-06T03:49:32.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.550+0000 7f735199d700 1 -- start start 2023-10-06T03:49:32.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.551+0000 7f735199d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f734c157770 0x7f734c150180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:32.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.551+0000 7f735199d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f734c0c7150 0x7f734c06ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:32.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.551+0000 7f735199d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734c06e2a0 con 0x7f734c0c6d00 2023-10-06T03:49:32.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.551+0000 7f735199d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734c157b40 con 0x7f734c0c7150 2023-10-06T03:49:32.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.551+0000 7f735199d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734c06b2e0 con 0x7f734c157770 2023-10-06T03:49:32.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.551+0000 7f735119c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f734c0c7150 0x7f734c06ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:32.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.551+0000 7f735119c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f734c0c7150 0x7f734c06ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49142/0 (socket says 172.21.15.55:49142) 2023-10-06T03:49:32.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.551+0000 7f735119c700 1 -- 172.21.15.55:0/2201994805 learned_addr learned my addr 172.21.15.55:0/2201994805 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:32.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.552+0000 7f735099b700 1 --2- 172.21.15.55:0/2201994805 >> v2:172.21.15.163:3300/0 conn(0x7f734c157770 0x7f734c150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.552+0000 7f735119c700 1 -- 172.21.15.55:0/2201994805 >> v2:172.21.15.163:3300/0 conn(0x7f734c157770 msgr2=0x7f734c150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.552+0000 7f735119c700 1 --2- 172.21.15.55:0/2201994805 >> v2:172.21.15.163:3300/0 conn(0x7f734c157770 0x7f734c150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.552+0000 7f735119c700 1 -- 172.21.15.55:0/2201994805 >> v1:172.21.15.163:6789/0 conn(0x7f734c0c6d00 legacy=0x7f734c06e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.552+0000 7f735119c700 1 -- 172.21.15.55:0/2201994805 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c0c78e0 con 0x7f734c0c7150 2023-10-06T03:49:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.552+0000 7f735119c700 1 --2- 172.21.15.55:0/2201994805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f734c0c7150 0x7f734c06ee40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f734c06fd20 tx=0x7f7344009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337ca0a7283c9d62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.552+0000 7f734b7fe700 1 -- 172.21.15.55:0/2201994805 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734400e070 con 0x7f734c0c7150 2023-10-06T03:49:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.552+0000 7f734b7fe700 1 -- 172.21.15.55:0/2201994805 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734400ba20 con 0x7f734c0c7150 2023-10-06T03:49:32.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.552+0000 7f734b7fe700 1 -- 172.21.15.55:0/2201994805 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73440092e0 con 0x7f734c0c7150 2023-10-06T03:49:32.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.553+0000 7f735199d700 1 -- 172.21.15.55:0/2201994805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f734c0c7150 msgr2=0x7f734c06ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:32.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.553+0000 7f735199d700 1 --2- 172.21.15.55:0/2201994805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f734c0c7150 0x7f734c06ee40 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f734c06fd20 tx=0x7f7344009840 comp rx=0 tx=0).stop 2023-10-06T03:49:32.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.553+0000 7f735199d700 1 -- 172.21.15.55:0/2201994805 shutdown_connections 2023-10-06T03:49:32.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.553+0000 7f735199d700 1 --2- 172.21.15.55:0/2201994805 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f734c0c7150 0x7f734c06ee40 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:32.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.553+0000 7f735199d700 1 --2- 172.21.15.55:0/2201994805 >> v2:172.21.15.163:3300/0 conn(0x7f734c157770 0x7f734c150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:32.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.553+0000 7f735199d700 1 -- 172.21.15.55:0/2201994805 >> 172.21.15.55:0/2201994805 conn(0x7f734c0cc1f0 msgr2=0x7f734c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:32.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.553+0000 7f735199d700 1 -- 172.21.15.55:0/2201994805 shutdown_connections 2023-10-06T03:49:32.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.553+0000 7f735199d700 1 -- 172.21.15.55:0/2201994805 wait complete. 2023-10-06T03:49:32.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.555+0000 7f735199d700 1 Processor -- start 2023-10-06T03:49:32.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.555+0000 7f735199d700 1 -- start start 2023-10-06T03:49:32.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c0c6d00 0x7f734c16a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:32.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735199d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c0c7150 0x7f734c16cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:32.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735199d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f734c157770 0x7f734c16d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:32.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735199d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0b1dc0 con 0x7f734c157770 2023-10-06T03:49:32.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735199d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c14f190 con 0x7f734c0c7150 2023-10-06T03:49:32.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0af130 con 0x7f734c0c6d00 2023-10-06T03:49:32.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735119c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f734c157770 0x7f734c16d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:32.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735119c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f734c157770 0x7f734c16d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49152/0 (socket says 172.21.15.55:49152) 2023-10-06T03:49:32.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735119c700 1 -- 172.21.15.55:0/1797440382 learned_addr learned my addr 172.21.15.55:0/1797440382 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:32.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735099b700 1 --2- 172.21.15.55:0/1797440382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c0c6d00 0x7f734c16a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:32.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f734bfff700 1 --2- 172.21.15.55:0/1797440382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c0c7150 0x7f734c16cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:32.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735119c700 1 -- 172.21.15.55:0/1797440382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c0c6d00 msgr2=0x7f734c16a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:32.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735119c700 1 --2- 172.21.15.55:0/1797440382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c0c6d00 0x7f734c16a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:32.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735119c700 1 -- 172.21.15.55:0/1797440382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c0c7150 msgr2=0x7f734c16cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:32.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735119c700 1 --2- 172.21.15.55:0/1797440382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c0c7150 0x7f734c16cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:32.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.556+0000 7f735119c700 1 -- 172.21.15.55:0/1797440382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7344009ae0 con 0x7f734c157770 2023-10-06T03:49:32.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.557+0000 7f735119c700 1 --2- 172.21.15.55:0/1797440382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f734c157770 0x7f734c16d190 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f73440072f0 tx=0x7f7344018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:32.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.557+0000 7f7348ff9700 1 -- 172.21.15.55:0/1797440382 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344008b00 con 0x7f734c157770 2023-10-06T03:49:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.557+0000 7f7348ff9700 1 -- 172.21.15.55:0/1797440382 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7344008c60 con 0x7f734c157770 2023-10-06T03:49:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.557+0000 7f7348ff9700 1 -- 172.21.15.55:0/1797440382 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344026460 con 0x7f734c157770 2023-10-06T03:49:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.557+0000 7f735199d700 1 -- 172.21.15.55:0/1797440382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734c0af410 con 0x7f734c157770 2023-10-06T03:49:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.557+0000 7f735199d700 1 -- 172.21.15.55:0/1797440382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734c0cb6c0 con 0x7f734c157770 2023-10-06T03:49:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.558+0000 7f735199d700 1 -- 172.21.15.55:0/1797440382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734c0aef90 con 0x7f734c157770 2023-10-06T03:49:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.559+0000 7f7348ff9700 1 -- 172.21.15.55:0/1797440382 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7344003040 con 0x7f734c157770 2023-10-06T03:49:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.560+0000 7f7348ff9700 1 --2- 172.21.15.55:0/1797440382 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7330066f30 0x7f73300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.560+0000 7f7348ff9700 1 -- 172.21.15.55:0/1797440382 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f7344086690 con 0x7f734c157770 2023-10-06T03:49:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.560+0000 7f735099b700 1 --2- 172.21.15.55:0/1797440382 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7330066f30 0x7f73300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.561+0000 7f735099b700 1 --2- 172.21.15.55:0/1797440382 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7330066f30 0x7f73300693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7340009d10 tx=0x7f73400091a0 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:32.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.564+0000 7f7348ff9700 1 -- 172.21.15.55:0/1797440382 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f734c0aef90 con 0x7f734c157770 2023-10-06T03:49:32.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:32.853+0000 7f735199d700 1 -- 172.21.15.55:0/1797440382 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f734c0aef90 con 0x7f734c157770 2023-10-06T03:49:33.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.039+0000 7f7348ff9700 1 -- 172.21.15.55:0/1797440382 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v605) v1 ==== 166+0+0 (secure 0 0 0) 0x7f734c0aef90 con 0x7f734c157770 2023-10-06T03:49:33.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.042+0000 7f735199d700 1 -- 172.21.15.55:0/1797440382 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7330066f30 msgr2=0x7f73300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.042+0000 7f735199d700 1 --2- 172.21.15.55:0/1797440382 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7330066f30 0x7f73300693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7340009d10 tx=0x7f73400091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.042+0000 7f735199d700 1 -- 172.21.15.55:0/1797440382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f734c157770 msgr2=0x7f734c16d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.042+0000 7f735199d700 1 --2- 172.21.15.55:0/1797440382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f734c157770 0x7f734c16d190 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f73440072f0 tx=0x7f7344018000 comp rx=0 tx=0).stop 2023-10-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.042+0000 7f735199d700 1 -- 172.21.15.55:0/1797440382 shutdown_connections 2023-10-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.042+0000 7f735199d700 1 --2- 172.21.15.55:0/1797440382 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7330066f30 0x7f73300693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.042+0000 7f735199d700 1 --2- 172.21.15.55:0/1797440382 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f734c157770 0x7f734c16d190 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.042+0000 7f735199d700 1 --2- 172.21.15.55:0/1797440382 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f734c0c6d00 0x7f734c16a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.042+0000 7f735199d700 1 --2- 172.21.15.55:0/1797440382 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f734c0c7150 0x7f734c16cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.042+0000 7f735199d700 1 -- 172.21.15.55:0/1797440382 >> 172.21.15.55:0/1797440382 conn(0x7f734c0cc1f0 msgr2=0x7f734c0adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:33.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.042+0000 7f735199d700 1 -- 172.21.15.55:0/1797440382 shutdown_connections 2023-10-06T03:49:33.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.042+0000 7f735199d700 1 -- 172.21.15.55:0/1797440382 wait complete. 2023-10-06T03:49:33.061 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-10-06T03:49:33.062 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-10-06T03:49:33.062 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T03:49:33.062 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-06T03:49:33.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.247+0000 7fedef39d700 1 Processor -- start 2023-10-06T03:49:33.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.247+0000 7fedef39d700 1 -- start start 2023-10-06T03:49:33.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.248+0000 7fedef39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede8157270 0x7fede8068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:33.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.248+0000 7fedef39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede8068d40 con 0x7fede80b0bd0 2023-10-06T03:49:33.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.248+0000 7fedef39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede815a980 con 0x7fede80688d0 2023-10-06T03:49:33.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.248+0000 7fedef39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede815a050 con 0x7fede8157270 2023-10-06T03:49:33.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.249+0000 7fedeeb9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fede80b0bd0 0x7fede815d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:41054/0 (socket says 172.21.15.55:41054) 2023-10-06T03:49:33.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.249+0000 7fedeeb9c700 1 -- 172.21.15.55:0/809315850 learned_addr learned my addr 172.21.15.55:0/809315850 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:33.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.249+0000 7fedee39b700 1 --2- 172.21.15.55:0/809315850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede8157270 0x7fede8068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:33.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.249+0000 7feded399700 1 -- 172.21.15.55:0/809315850 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637562132 0 0) 0x7fede8068d40 con 0x7fede80b0bd0 2023-10-06T03:49:33.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.249+0000 7feded399700 1 -- 172.21.15.55:0/809315850 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedd4003570 con 0x7fede80b0bd0 2023-10-06T03:49:33.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.249+0000 7feded399700 1 -- 172.21.15.55:0/809315850 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326270913 0 0) 0x7fede815a980 con 0x7fede80688d0 2023-10-06T03:49:33.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.249+0000 7feded399700 1 -- 172.21.15.55:0/809315850 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fede8068d40 con 0x7fede80688d0 2023-10-06T03:49:33.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.249+0000 7fedee39b700 1 -- 172.21.15.55:0/809315850 >> v1:172.21.15.163:6789/0 conn(0x7fede80688d0 legacy=0x7fede80b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:33.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.249+0000 7fedee39b700 1 -- 172.21.15.55:0/809315850 >> v1:172.21.15.55:6789/0 conn(0x7fede80b0bd0 legacy=0x7fede815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:33.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.249+0000 7fedee39b700 1 -- 172.21.15.55:0/809315850 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede815a330 con 0x7fede8157270 2023-10-06T03:49:33.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.250+0000 7fedee39b700 1 --2- 172.21.15.55:0/809315850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede8157270 0x7fede8068390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fede4009910 tx=0x7fede40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef442ddc8d30decd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:33.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.250+0000 7feded399700 1 -- 172.21.15.55:0/809315850 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede4011630 con 0x7fede8157270 2023-10-06T03:49:33.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.250+0000 7feded399700 1 -- 172.21.15.55:0/809315850 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede4011790 con 0x7fede8157270 2023-10-06T03:49:33.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.250+0000 7feded399700 1 -- 172.21.15.55:0/809315850 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede4011b30 con 0x7fede8157270 2023-10-06T03:49:33.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.251+0000 7fedef39d700 1 -- 172.21.15.55:0/809315850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede8157270 msgr2=0x7fede8068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:33.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.251+0000 7fedef39d700 1 --2- 172.21.15.55:0/809315850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede8157270 0x7fede8068390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fede4009910 tx=0x7fede40091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:33.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.251+0000 7fedef39d700 1 -- 172.21.15.55:0/809315850 shutdown_connections 2023-10-06T03:49:33.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.251+0000 7fedef39d700 1 --2- 172.21.15.55:0/809315850 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede8157270 0x7fede8068390 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:33.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.251+0000 7fedef39d700 1 -- 172.21.15.55:0/809315850 >> 172.21.15.55:0/809315850 conn(0x7fede80cd310 msgr2=0x7fede80cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:33.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.251+0000 7fedef39d700 1 -- 172.21.15.55:0/809315850 shutdown_connections 2023-10-06T03:49:33.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.251+0000 7fedef39d700 1 -- 172.21.15.55:0/809315850 wait complete. 2023-10-06T03:49:33.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.253+0000 7fedef39d700 1 Processor -- start 2023-10-06T03:49:33.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.253+0000 7fedef39d700 1 -- start start 2023-10-06T03:49:33.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fedef39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fede80688d0 0x7fede80cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:33.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fedef39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fede80b0bd0 0x7fede8155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:33.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fedef39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede8157270 0x7fede814fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:33.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fedef39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede8154790 con 0x7fede80b0bd0 2023-10-06T03:49:33.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fedef39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede8153b80 con 0x7fede80688d0 2023-10-06T03:49:33.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fedef39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede8153e60 con 0x7fede8157270 2023-10-06T03:49:33.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fededb9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fede80b0bd0 0x7fede8155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:33.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fededb9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fede80b0bd0 0x7fede8155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46626/0 (socket says 172.21.15.55:46626) 2023-10-06T03:49:33.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fededb9a700 1 -- 172.21.15.55:0/2856112914 learned_addr learned my addr 172.21.15.55:0/2856112914 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:33.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fedee39b700 1 --2- 172.21.15.55:0/2856112914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fede80688d0 0x7fede80cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:33.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fedeeb9c700 1 --2- 172.21.15.55:0/2856112914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede8157270 0x7fede814fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:33.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fededb9a700 1 -- 172.21.15.55:0/2856112914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede8157270 msgr2=0x7fede814fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:33.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fededb9a700 1 --2- 172.21.15.55:0/2856112914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede8157270 0x7fede814fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:33.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fededb9a700 1 -- 172.21.15.55:0/2856112914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fede80688d0 msgr2=0x7fede80cc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:33.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fededb9a700 1 --2- 172.21.15.55:0/2856112914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fede80688d0 0x7fede80cc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:33.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.254+0000 7fededb9a700 1 -- 172.21.15.55:0/2856112914 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede4009420 con 0x7fede80b0bd0 2023-10-06T03:49:33.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.255+0000 7fededb9a700 1 --2- 172.21.15.55:0/2856112914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fede80b0bd0 0x7fede8155180 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fedd800c790 tx=0x7fedd8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:33.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.255+0000 7fedde7fc700 1 -- 172.21.15.55:0/2856112914 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8010510 con 0x7fede80b0bd0 2023-10-06T03:49:33.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.255+0000 7fedde7fc700 1 -- 172.21.15.55:0/2856112914 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd8010670 con 0x7fede80b0bd0 2023-10-06T03:49:33.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.255+0000 7fedde7fc700 1 -- 172.21.15.55:0/2856112914 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8010a10 con 0x7fede80b0bd0 2023-10-06T03:49:33.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.255+0000 7fedef39d700 1 -- 172.21.15.55:0/2856112914 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede8154140 con 0x7fede80b0bd0 2023-10-06T03:49:33.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.255+0000 7fedef39d700 1 -- 172.21.15.55:0/2856112914 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede80ccaa0 con 0x7fede80b0bd0 2023-10-06T03:49:33.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.256+0000 7fedef39d700 1 -- 172.21.15.55:0/2856112914 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede804c3c0 con 0x7fede80b0bd0 2023-10-06T03:49:33.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.257+0000 7fedde7fc700 1 -- 172.21.15.55:0/2856112914 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fedd801b400 con 0x7fede80b0bd0 2023-10-06T03:49:33.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.258+0000 7fedde7fc700 1 --2- 172.21.15.55:0/2856112914 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fedcc067000 0x7fedcc0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:33.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.258+0000 7fedde7fc700 1 -- 172.21.15.55:0/2856112914 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6243+0+0 (secure 0 0 0) 0x7fedd80844f0 con 0x7fede80b0bd0 2023-10-06T03:49:33.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.259+0000 7fedee39b700 1 --2- 172.21.15.55:0/2856112914 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fedcc067000 0x7fedcc0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:33.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.260+0000 7fedee39b700 1 --2- 172.21.15.55:0/2856112914 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fedcc067000 0x7fedcc0694b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fede40091e0 tx=0x7fede4017000 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:33.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.263+0000 7fedde7fc700 1 -- 172.21.15.55:0/2856112914 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedd8051520 con 0x7fede80b0bd0 2023-10-06T03:49:33.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:33.548+0000 7fedef39d700 1 -- 172.21.15.55:0/2856112914 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7fede815a610 con 0x7fede80b0bd0 2023-10-06T03:49:34.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.045+0000 7fedde7fc700 1 -- 172.21.15.55:0/2856112914 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v606) v1 ==== 186+0+0 (secure 0 0 0) 0x7fedd805a020 con 0x7fede80b0bd0 2023-10-06T03:49:34.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.048+0000 7fedef39d700 1 -- 172.21.15.55:0/2856112914 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fedcc067000 msgr2=0x7fedcc0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:34.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.048+0000 7fedef39d700 1 --2- 172.21.15.55:0/2856112914 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fedcc067000 0x7fedcc0694b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fede40091e0 tx=0x7fede4017000 comp rx=0 tx=0).stop 2023-10-06T03:49:34.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.048+0000 7fedef39d700 1 -- 172.21.15.55:0/2856112914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fede80b0bd0 msgr2=0x7fede8155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:34.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.048+0000 7fedef39d700 1 --2- 172.21.15.55:0/2856112914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fede80b0bd0 0x7fede8155180 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fedd800c790 tx=0x7fedd8007e10 comp rx=0 tx=0).stop 2023-10-06T03:49:34.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.048+0000 7fedef39d700 1 -- 172.21.15.55:0/2856112914 shutdown_connections 2023-10-06T03:49:34.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.048+0000 7fedef39d700 1 --2- 172.21.15.55:0/2856112914 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fedcc067000 0x7fedcc0694b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.048+0000 7fedef39d700 1 --2- 172.21.15.55:0/2856112914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fede8157270 0x7fede814fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.048+0000 7fedef39d700 1 --2- 172.21.15.55:0/2856112914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fede80688d0 0x7fede80cc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.048+0000 7fedef39d700 1 --2- 172.21.15.55:0/2856112914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fede80b0bd0 0x7fede8155180 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.048+0000 7fedef39d700 1 -- 172.21.15.55:0/2856112914 >> 172.21.15.55:0/2856112914 conn(0x7fede80cd310 msgr2=0x7fede80b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:34.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.048+0000 7fedef39d700 1 -- 172.21.15.55:0/2856112914 shutdown_connections 2023-10-06T03:49:34.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.048+0000 7fedef39d700 1 -- 172.21.15.55:0/2856112914 wait complete. 2023-10-06T03:49:34.069 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:49:34.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.261+0000 7f4c71f9d700 1 Processor -- start 2023-10-06T03:49:34.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.261+0000 7f4c71f9d700 1 -- start start 2023-10-06T03:49:34.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.262+0000 7f4c71f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4c6c0b4e70 0x7f4c6c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:34.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.262+0000 7f4c71f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c6c0b16d0 0x7f4c6c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:34.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.262+0000 7f4c71f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4c6c15d240 0x7f4c6c15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:34.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.262+0000 7f4c71f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6c0b2080 con 0x7f4c6c0b4e70 2023-10-06T03:49:34.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.262+0000 7f4c71f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6c0cfb00 con 0x7f4c6c15d240 2023-10-06T03:49:34.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.262+0000 7f4c71f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0b5c40 con 0x7f4c6c0b16d0 2023-10-06T03:49:34.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.263+0000 7f4c70f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4c6c0b4e70 0x7f4c6c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:34.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.263+0000 7f4c70f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4c6c0b4e70 0x7f4c6c0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46632/0 (socket says 172.21.15.55:46632) 2023-10-06T03:49:34.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.263+0000 7f4c70f9b700 1 -- 172.21.15.55:0/684980880 learned_addr learned my addr 172.21.15.55:0/684980880 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:34.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.263+0000 7f4c6bfff700 1 --2- 172.21.15.55:0/684980880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c6c0b16d0 0x7f4c6c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:34.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.263+0000 7f4c7179c700 1 --2- 172.21.15.55:0/684980880 >> v2:172.21.15.163:3300/0 conn(0x7f4c6c15d240 0x7f4c6c15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:34.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.263+0000 7f4c70f9b700 1 -- 172.21.15.55:0/684980880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c6c0b16d0 msgr2=0x7f4c6c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:34.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.263+0000 7f4c70f9b700 1 --2- 172.21.15.55:0/684980880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c6c0b16d0 0x7f4c6c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.263+0000 7f4c70f9b700 1 -- 172.21.15.55:0/684980880 >> v2:172.21.15.163:3300/0 conn(0x7f4c6c15d240 msgr2=0x7f4c6c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:34.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.263+0000 7f4c70f9b700 1 --2- 172.21.15.55:0/684980880 >> v2:172.21.15.163:3300/0 conn(0x7f4c6c15d240 0x7f4c6c15d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.263+0000 7f4c70f9b700 1 -- 172.21.15.55:0/684980880 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c0b5f20 con 0x7f4c6c0b4e70 2023-10-06T03:49:34.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.263+0000 7f4c6bfff700 1 --2- 172.21.15.55:0/684980880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c6c0b16d0 0x7f4c6c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:34.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.263+0000 7f4c7179c700 1 --2- 172.21.15.55:0/684980880 >> v2:172.21.15.163:3300/0 conn(0x7f4c6c15d240 0x7f4c6c15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:34.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.263+0000 7f4c70f9b700 1 --2- 172.21.15.55:0/684980880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c6c0b4e70 0x7f4c6c0ccff0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c15d180 tx=0x7f4c60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ef55a50e0488b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:34.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.263+0000 7f4c6b7fe700 1 -- 172.21.15.55:0/684980880 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6000b860 con 0x7f4c6c0b4e70 2023-10-06T03:49:34.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.264+0000 7f4c6b7fe700 1 -- 172.21.15.55:0/684980880 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6000b9c0 con 0x7f4c6c0b4e70 2023-10-06T03:49:34.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.264+0000 7f4c6b7fe700 1 -- 172.21.15.55:0/684980880 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6000bd60 con 0x7f4c6c0b4e70 2023-10-06T03:49:34.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.264+0000 7f4c71f9d700 1 -- 172.21.15.55:0/684980880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c6c0b4e70 msgr2=0x7f4c6c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:34.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.264+0000 7f4c71f9d700 1 --2- 172.21.15.55:0/684980880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c6c0b4e70 0x7f4c6c0ccff0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c15d180 tx=0x7f4c60009250 comp rx=0 tx=0).stop 2023-10-06T03:49:34.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.264+0000 7f4c71f9d700 1 -- 172.21.15.55:0/684980880 shutdown_connections 2023-10-06T03:49:34.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.264+0000 7f4c71f9d700 1 --2- 172.21.15.55:0/684980880 >> v2:172.21.15.163:3300/0 conn(0x7f4c6c15d240 0x7f4c6c15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.264+0000 7f4c71f9d700 1 --2- 172.21.15.55:0/684980880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c6c0b4e70 0x7f4c6c0ccff0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.264+0000 7f4c71f9d700 1 --2- 172.21.15.55:0/684980880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c6c0b16d0 0x7f4c6c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.264+0000 7f4c71f9d700 1 -- 172.21.15.55:0/684980880 >> 172.21.15.55:0/684980880 conn(0x7f4c6c0d10c0 msgr2=0x7f4c6c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:34.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.264+0000 7f4c71f9d700 1 -- 172.21.15.55:0/684980880 shutdown_connections 2023-10-06T03:49:34.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.264+0000 7f4c71f9d700 1 -- 172.21.15.55:0/684980880 wait complete. 2023-10-06T03:49:34.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.266+0000 7f4c71f9d700 1 Processor -- start 2023-10-06T03:49:34.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.266+0000 7f4c71f9d700 1 -- start start 2023-10-06T03:49:34.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.267+0000 7f4c71f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c6c0b16d0 0x7f4c6c14c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:34.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.267+0000 7f4c71f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c6c0b4e70 0x7f4c6c068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:34.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.267+0000 7f4c71f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c6c15d240 0x7f4c6c0688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:34.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.267+0000 7f4c71f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c16b070 con 0x7f4c6c0b16d0 2023-10-06T03:49:34.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.267+0000 7f4c71f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0c8130 con 0x7f4c6c15d240 2023-10-06T03:49:34.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.267+0000 7f4c71f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0cab70 con 0x7f4c6c0b4e70 2023-10-06T03:49:34.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.267+0000 7f4c70f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c6c0b16d0 0x7f4c6c14c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:34.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.267+0000 7f4c70f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c6c0b16d0 0x7f4c6c14c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46642/0 (socket says 172.21.15.55:46642) 2023-10-06T03:49:34.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.267+0000 7f4c70f9b700 1 -- 172.21.15.55:0/2187484288 learned_addr learned my addr 172.21.15.55:0/2187484288 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:34.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.267+0000 7f4c70f9b700 1 -- 172.21.15.55:0/2187484288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c6c0b4e70 msgr2=0x7f4c6c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:34.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.267+0000 7f4c70f9b700 1 --2- 172.21.15.55:0/2187484288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c6c0b4e70 0x7f4c6c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.267+0000 7f4c70f9b700 1 -- 172.21.15.55:0/2187484288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c6c15d240 msgr2=0x7f4c6c0688d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:34.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.267+0000 7f4c70f9b700 1 --2- 172.21.15.55:0/2187484288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c6c15d240 0x7f4c6c0688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.267+0000 7f4c70f9b700 1 -- 172.21.15.55:0/2187484288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c600094f0 con 0x7f4c6c0b16d0 2023-10-06T03:49:34.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.268+0000 7f4c70f9b700 1 --2- 172.21.15.55:0/2187484288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c6c0b16d0 0x7f4c6c14c1e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c070dd0 tx=0x7f4c60017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:34.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.268+0000 7f4c68ff9700 1 -- 172.21.15.55:0/2187484288 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60024440 con 0x7f4c6c0b16d0 2023-10-06T03:49:34.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.268+0000 7f4c68ff9700 1 -- 172.21.15.55:0/2187484288 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c600245a0 con 0x7f4c6c0b16d0 2023-10-06T03:49:34.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.268+0000 7f4c68ff9700 1 -- 172.21.15.55:0/2187484288 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60024960 con 0x7f4c6c0b16d0 2023-10-06T03:49:34.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.268+0000 7f4c71f9d700 1 -- 172.21.15.55:0/2187484288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6c0c7800 con 0x7f4c6c0b16d0 2023-10-06T03:49:34.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.268+0000 7f4c71f9d700 1 -- 172.21.15.55:0/2187484288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6c0c7dc0 con 0x7f4c6c0b16d0 2023-10-06T03:49:34.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.269+0000 7f4c71f9d700 1 -- 172.21.15.55:0/2187484288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6c0ce140 con 0x7f4c6c0b16d0 2023-10-06T03:49:34.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.270+0000 7f4c68ff9700 1 -- 172.21.15.55:0/2187484288 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4c6001a420 con 0x7f4c6c0b16d0 2023-10-06T03:49:34.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.271+0000 7f4c68ff9700 1 --2- 172.21.15.55:0/2187484288 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4c50066f30 0x7f4c500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:34.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.271+0000 7f4c68ff9700 1 -- 172.21.15.55:0/2187484288 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f4c60086700 con 0x7f4c6c0b16d0 2023-10-06T03:49:34.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.271+0000 7f4c6bfff700 1 --2- 172.21.15.55:0/2187484288 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4c50066f30 0x7f4c500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:34.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.272+0000 7f4c6bfff700 1 --2- 172.21.15.55:0/2187484288 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4c50066f30 0x7f4c500693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0af5b0 tx=0x7f4c58008040 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:34.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.275+0000 7f4c68ff9700 1 -- 172.21.15.55:0/2187484288 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c600536b0 con 0x7f4c6c0b16d0 2023-10-06T03:49:34.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.480+0000 7f4c71f9d700 1 -- 172.21.15.55:0/2187484288 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4c6c04bb80 con 0x7f4c6c0b16d0 2023-10-06T03:49:34.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.485+0000 7f4c68ff9700 1 -- 172.21.15.55:0/2187484288 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4c60056cd0 con 0x7f4c6c0b16d0 2023-10-06T03:49:34.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.487+0000 7f4c71f9d700 1 -- 172.21.15.55:0/2187484288 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4c50066f30 msgr2=0x7f4c500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:34.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.487+0000 7f4c71f9d700 1 --2- 172.21.15.55:0/2187484288 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4c50066f30 0x7f4c500693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0af5b0 tx=0x7f4c58008040 comp rx=0 tx=0).stop 2023-10-06T03:49:34.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.487+0000 7f4c71f9d700 1 -- 172.21.15.55:0/2187484288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c6c0b16d0 msgr2=0x7f4c6c14c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:34.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.487+0000 7f4c71f9d700 1 --2- 172.21.15.55:0/2187484288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c6c0b16d0 0x7f4c6c14c1e0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c070dd0 tx=0x7f4c60017000 comp rx=0 tx=0).stop 2023-10-06T03:49:34.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.487+0000 7f4c71f9d700 1 -- 172.21.15.55:0/2187484288 shutdown_connections 2023-10-06T03:49:34.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.487+0000 7f4c71f9d700 1 --2- 172.21.15.55:0/2187484288 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4c50066f30 0x7f4c500693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.487+0000 7f4c71f9d700 1 --2- 172.21.15.55:0/2187484288 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4c6c15d240 0x7f4c6c0688d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.487+0000 7f4c71f9d700 1 --2- 172.21.15.55:0/2187484288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4c6c0b16d0 0x7f4c6c14c1e0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.487+0000 7f4c71f9d700 1 --2- 172.21.15.55:0/2187484288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4c6c0b4e70 0x7f4c6c068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.487+0000 7f4c71f9d700 1 -- 172.21.15.55:0/2187484288 >> 172.21.15.55:0/2187484288 conn(0x7f4c6c0d10c0 msgr2=0x7f4c6c158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:34.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.487+0000 7f4c71f9d700 1 -- 172.21.15.55:0/2187484288 shutdown_connections 2023-10-06T03:49:34.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.487+0000 7f4c71f9d700 1 -- 172.21.15.55:0/2187484288 wait complete. 2023-10-06T03:49:34.506 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:49:34.506 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:49:34.591 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:49:34.674 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:49:34.674 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:49:34.674 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:49:34.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.860+0000 7f62fb19d700 1 Processor -- start 2023-10-06T03:49:34.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.860+0000 7f62fb19d700 1 -- start start 2023-10-06T03:49:34.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.861+0000 7f62fb19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f62f4156310 0x7f62f40c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:34.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.861+0000 7f62fb19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62f40ca860 con 0x7f62f41566e0 2023-10-06T03:49:34.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.861+0000 7f62fb19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62f40cab40 con 0x7f62f40ae710 2023-10-06T03:49:34.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.861+0000 7f62fb19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f40cae20 con 0x7f62f4156310 2023-10-06T03:49:34.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.861+0000 7f62fa19b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f62f4156310 0x7f62f40c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:34.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.861+0000 7f62fa19b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f62f4156310 0x7f62f40c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46654/0 (socket says 172.21.15.55:46654) 2023-10-06T03:49:34.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.861+0000 7f62fa19b700 1 -- 172.21.15.55:0/1118887019 learned_addr learned my addr 172.21.15.55:0/1118887019 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:34.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.862+0000 7f62fa19b700 1 -- 172.21.15.55:0/1118887019 >> v1:172.21.15.163:6789/0 conn(0x7f62f40ae710 legacy=0x7f62f40c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:34.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.862+0000 7f62fa19b700 1 -- 172.21.15.55:0/1118887019 >> v1:172.21.15.55:6789/0 conn(0x7f62f41566e0 legacy=0x7f62f40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:34.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.862+0000 7f62fa19b700 1 -- 172.21.15.55:0/1118887019 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f40cb100 con 0x7f62f4156310 2023-10-06T03:49:34.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.862+0000 7f62f9199700 1 -- 172.21.15.55:0/1118887019 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695092122 0 0) 0x7f62f40ca860 con 0x7f62f41566e0 2023-10-06T03:49:34.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.862+0000 7f62fa19b700 1 --2- 172.21.15.55:0/1118887019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f4156310 0x7f62f40c9dc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f62f00099b0 tx=0x7f62f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1f9b5c01a3af34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:34.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.862+0000 7f62f9199700 1 -- 172.21.15.55:0/1118887019 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f000b820 con 0x7f62f4156310 2023-10-06T03:49:34.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.862+0000 7f62f9199700 1 -- 172.21.15.55:0/1118887019 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f000b980 con 0x7f62f4156310 2023-10-06T03:49:34.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.862+0000 7f62f9199700 1 -- 172.21.15.55:0/1118887019 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f000bd20 con 0x7f62f4156310 2023-10-06T03:49:34.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.863+0000 7f62fb19d700 1 -- 172.21.15.55:0/1118887019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f4156310 msgr2=0x7f62f40c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:34.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.863+0000 7f62fb19d700 1 --2- 172.21.15.55:0/1118887019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f4156310 0x7f62f40c9dc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f62f00099b0 tx=0x7f62f0009250 comp rx=0 tx=0).stop 2023-10-06T03:49:34.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.863+0000 7f62fb19d700 1 -- 172.21.15.55:0/1118887019 shutdown_connections 2023-10-06T03:49:34.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.863+0000 7f62fb19d700 1 --2- 172.21.15.55:0/1118887019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f4156310 0x7f62f40c9dc0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.863+0000 7f62fb19d700 1 -- 172.21.15.55:0/1118887019 >> 172.21.15.55:0/1118887019 conn(0x7f62f40cc1f0 msgr2=0x7f62f40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:34.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.863+0000 7f62fb19d700 1 -- 172.21.15.55:0/1118887019 shutdown_connections 2023-10-06T03:49:34.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.863+0000 7f62fb19d700 1 -- 172.21.15.55:0/1118887019 wait complete. 2023-10-06T03:49:34.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.864+0000 7f62fb19d700 1 Processor -- start 2023-10-06T03:49:34.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.865+0000 7f62fb19d700 1 -- start start 2023-10-06T03:49:34.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.865+0000 7f62fb19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62f40ae710 0x7f62f416a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:34.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.865+0000 7f62fb19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62f4156310 0x7f62f41506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:34.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.865+0000 7f62fb19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f41566e0 0x7f62f4069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.866+0000 7f62fb19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f41547b0 con 0x7f62f41566e0 2023-10-06T03:49:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.866+0000 7f62fb19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f416adc0 con 0x7f62f40ae710 2023-10-06T03:49:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.866+0000 7f62fb19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f41538c0 con 0x7f62f4156310 2023-10-06T03:49:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.866+0000 7f62fa99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f41566e0 0x7f62f4069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.866+0000 7f62fa99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f41566e0 0x7f62f4069f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46656/0 (socket says 172.21.15.55:46656) 2023-10-06T03:49:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.866+0000 7f62fa99c700 1 -- 172.21.15.55:0/337703480 learned_addr learned my addr 172.21.15.55:0/337703480 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.866+0000 7f62fa19b700 1 --2- 172.21.15.55:0/337703480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62f40ae710 0x7f62f416a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.866+0000 7f62fa99c700 1 -- 172.21.15.55:0/337703480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62f4156310 msgr2=0x7f62f41506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.866+0000 7f62fa99c700 1 --2- 172.21.15.55:0/337703480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62f4156310 0x7f62f41506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.866+0000 7f62fa99c700 1 -- 172.21.15.55:0/337703480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62f40ae710 msgr2=0x7f62f416a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:34.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.866+0000 7f62fa99c700 1 --2- 172.21.15.55:0/337703480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62f40ae710 0x7f62f416a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:34.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.866+0000 7f62fa99c700 1 -- 172.21.15.55:0/337703480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f00094f0 con 0x7f62f41566e0 2023-10-06T03:49:34.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.866+0000 7f62fa99c700 1 --2- 172.21.15.55:0/337703480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f41566e0 0x7f62f4069f50 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f62ec00c890 tx=0x7f62ec0026a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:34.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.867+0000 7f62ea7fc700 1 -- 172.21.15.55:0/337703480 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec012630 con 0x7f62f41566e0 2023-10-06T03:49:34.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.867+0000 7f62ea7fc700 1 -- 172.21.15.55:0/337703480 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ec012790 con 0x7f62f41566e0 2023-10-06T03:49:34.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.867+0000 7f62ea7fc700 1 -- 172.21.15.55:0/337703480 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec012b50 con 0x7f62f41566e0 2023-10-06T03:49:34.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.867+0000 7f62fb19d700 1 -- 172.21.15.55:0/337703480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f4153ba0 con 0x7f62f41566e0 2023-10-06T03:49:34.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.867+0000 7f62fb19d700 1 -- 172.21.15.55:0/337703480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f4154160 con 0x7f62f41566e0 2023-10-06T03:49:34.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.868+0000 7f62fb19d700 1 -- 172.21.15.55:0/337703480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f4154920 con 0x7f62f41566e0 2023-10-06T03:49:34.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.869+0000 7f62ea7fc700 1 -- 172.21.15.55:0/337703480 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62ec00a420 con 0x7f62f41566e0 2023-10-06T03:49:34.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.870+0000 7f62ea7fc700 1 --2- 172.21.15.55:0/337703480 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f62d8066fb0 0x7f62d8069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:34.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.870+0000 7f62ea7fc700 1 -- 172.21.15.55:0/337703480 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f62ec084bd0 con 0x7f62f41566e0 2023-10-06T03:49:34.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.870+0000 7f62fa19b700 1 --2- 172.21.15.55:0/337703480 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f62d8066fb0 0x7f62d8069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:34.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.871+0000 7f62fa19b700 1 --2- 172.21.15.55:0/337703480 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f62d8066fb0 0x7f62d8069460 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f62f00099b0 tx=0x7f62f0016000 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:34.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:34.874+0000 7f62ea7fc700 1 -- 172.21.15.55:0/337703480 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62ec051b80 con 0x7f62f41566e0 2023-10-06T03:49:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:35.120+0000 7f62fb19d700 1 -- 172.21.15.55:0/337703480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62f404c3c0 con 0x7f62f41566e0 2023-10-06T03:49:35.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:35.121+0000 7f62ea7fc700 1 -- 172.21.15.55:0/337703480 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f62ec016de0 con 0x7f62f41566e0 2023-10-06T03:49:35.123 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:35.123 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5957,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/4087100648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":4087100648},{"type":"v1","addr":"172.21.15.163:6834","nonce":4087100648}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":6556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2017152000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2017152000},{"type":"v1","addr":"172.21.15.163:6836","nonce":2017152000}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":6559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1226557536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1226557536},{"type":"v1","addr":"172.21.15.55:6834","nonce":1226557536}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":39},{"gid":7131,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/3160359502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3160359502},{"type":"v1","addr":"172.21.15.55:6835","nonce":3160359502}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":46}],"filesystems":[]} 2023-10-06T03:49:35.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:35.123+0000 7f62fb19d700 1 -- 172.21.15.55:0/337703480 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f62d8066fb0 msgr2=0x7f62d8069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:35.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:35.123+0000 7f62fb19d700 1 --2- 172.21.15.55:0/337703480 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f62d8066fb0 0x7f62d8069460 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f62f00099b0 tx=0x7f62f0016000 comp rx=0 tx=0).stop 2023-10-06T03:49:35.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:35.123+0000 7f62fb19d700 1 -- 172.21.15.55:0/337703480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f41566e0 msgr2=0x7f62f4069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:35.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:35.123+0000 7f62fb19d700 1 --2- 172.21.15.55:0/337703480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f41566e0 0x7f62f4069f50 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f62ec00c890 tx=0x7f62ec0026a0 comp rx=0 tx=0).stop 2023-10-06T03:49:35.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:35.123+0000 7f62fb19d700 1 -- 172.21.15.55:0/337703480 shutdown_connections 2023-10-06T03:49:35.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:35.123+0000 7f62fb19d700 1 --2- 172.21.15.55:0/337703480 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f62d8066fb0 0x7f62d8069460 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:35.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:35.123+0000 7f62fb19d700 1 --2- 172.21.15.55:0/337703480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f62f41566e0 0x7f62f4069f50 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:35.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:35.123+0000 7f62fb19d700 1 --2- 172.21.15.55:0/337703480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f62f40ae710 0x7f62f416a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:35.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:35.123+0000 7f62fb19d700 1 --2- 172.21.15.55:0/337703480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f62f4156310 0x7f62f41506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:35.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:35.123+0000 7f62fb19d700 1 -- 172.21.15.55:0/337703480 >> 172.21.15.55:0/337703480 conn(0x7f62f40cc1f0 msgr2=0x7f62f40c98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:35.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:35.123+0000 7f62fb19d700 1 -- 172.21.15.55:0/337703480 shutdown_connections 2023-10-06T03:49:35.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:35.123+0000 7f62fb19d700 1 -- 172.21.15.55:0/337703480 wait complete. 2023-10-06T03:49:35.128 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 46 2023-10-06T03:49:35.144 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:49:35.144 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:49:35.145 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:49:35.145 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:49:35.145 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:49:35.145 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:49:35.146 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:49:35.146 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:49:35.146 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:49:35.146 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:49:35.146 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:49:35.146 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:49:35.147 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:49:35.147 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:49:35.147 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:49:35.147 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:49:35.161 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:49:35.161 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:49:35.164 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:49:35.164 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:49:35.165 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:49:35.166 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:49:35.206 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:49:35.206 DEBUG:teuthology.parallel:result is None 2023-10-06T03:49:35.210 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:49:35.211 DEBUG:teuthology.parallel:result is None 2023-10-06T03:49:35.214 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:49:35.214 DEBUG:teuthology.parallel:result is None 2023-10-06T03:49:35.336 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-10-06T03:49:35.351 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T03:49:35.351 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2023-10-06T03:49:35.367 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-06T03:49:35.367 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-10-06T03:49:35.367 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-10-06T03:49:41.161 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:49:41.161 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:49:41.167 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:49:41.167 DEBUG:teuthology.parallel:result is None 2023-10-06T03:49:41.167 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:49:41.310 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T03:49:41.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.364+0000 7f01d239d700 1 Processor -- start 2023-10-06T03:49:41.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.365+0000 7f01d239d700 1 -- start start 2023-10-06T03:49:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.366+0000 7f01d239d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc1566e0 0x7f01cc0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.366+0000 7f01d239d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01cc0cb900 0x7f01cc0ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.366+0000 7f01d239d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01cc0ca860 con 0x7f01cc156310 2023-10-06T03:49:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.366+0000 7f01d239d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01cc0cab40 con 0x7f01cc0cb900 2023-10-06T03:49:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.366+0000 7f01d239d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc0cae20 con 0x7f01cc1566e0 2023-10-06T03:49:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.366+0000 7f01d1b9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01cc0cb900 0x7f01cc0ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.366+0000 7f01d1b9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01cc0cb900 0x7f01cc0ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46714/0 (socket says 172.21.15.55:46714) 2023-10-06T03:49:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.366+0000 7f01d1b9c700 1 -- 172.21.15.55:0/2940637857 learned_addr learned my addr 172.21.15.55:0/2940637857 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:41.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.366+0000 7f01d0b9a700 1 --2- 172.21.15.55:0/2940637857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc1566e0 0x7f01cc0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:41.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.367+0000 7f01d1b9c700 1 -- 172.21.15.55:0/2940637857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc1566e0 msgr2=0x7f01cc0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:41.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.367+0000 7f01d1b9c700 1 --2- 172.21.15.55:0/2940637857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc1566e0 0x7f01cc0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:41.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.367+0000 7f01d1b9c700 1 -- 172.21.15.55:0/2940637857 >> v1:172.21.15.163:6789/0 conn(0x7f01cc156310 legacy=0x7f01cc0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:41.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.367+0000 7f01d1b9c700 1 -- 172.21.15.55:0/2940637857 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc0cb100 con 0x7f01cc0cb900 2023-10-06T03:49:41.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.367+0000 7f01c3fff700 1 -- 172.21.15.55:0/2940637857 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608345108 0 0) 0x7f01cc0ca860 con 0x7f01cc156310 2023-10-06T03:49:41.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.367+0000 7f01d1b9c700 1 --2- 172.21.15.55:0/2940637857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01cc0cb900 0x7f01cc0ae540 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f01c400d1a0 tx=0x7f01c400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8586fcfaac7ba46b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:41.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.367+0000 7f01c3fff700 1 -- 172.21.15.55:0/2940637857 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4010070 con 0x7f01cc0cb900 2023-10-06T03:49:41.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.367+0000 7f01c3fff700 1 -- 172.21.15.55:0/2940637857 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c400de10 con 0x7f01cc0cb900 2023-10-06T03:49:41.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.367+0000 7f01c3fff700 1 -- 172.21.15.55:0/2940637857 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c400b320 con 0x7f01cc0cb900 2023-10-06T03:49:41.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.368+0000 7f01d239d700 1 -- 172.21.15.55:0/2940637857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01cc0cb900 msgr2=0x7f01cc0ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:41.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.368+0000 7f01d239d700 1 --2- 172.21.15.55:0/2940637857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01cc0cb900 0x7f01cc0ae540 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f01c400d1a0 tx=0x7f01c400ba50 comp rx=0 tx=0).stop 2023-10-06T03:49:41.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.368+0000 7f01d239d700 1 -- 172.21.15.55:0/2940637857 shutdown_connections 2023-10-06T03:49:41.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.368+0000 7f01d239d700 1 --2- 172.21.15.55:0/2940637857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01cc0cb900 0x7f01cc0ae540 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:41.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.368+0000 7f01d239d700 1 --2- 172.21.15.55:0/2940637857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc1566e0 0x7f01cc0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:41.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.368+0000 7f01d239d700 1 -- 172.21.15.55:0/2940637857 >> 172.21.15.55:0/2940637857 conn(0x7f01cc0cc1f0 msgr2=0x7f01cc0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:41.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.368+0000 7f01d239d700 1 -- 172.21.15.55:0/2940637857 shutdown_connections 2023-10-06T03:49:41.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.368+0000 7f01d239d700 1 -- 172.21.15.55:0/2940637857 wait complete. 2023-10-06T03:49:41.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.369+0000 7f01d239d700 1 Processor -- start 2023-10-06T03:49:41.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.370+0000 7f01d239d700 1 -- start start 2023-10-06T03:49:41.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.370+0000 7f01d239d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc0cb900 0x7f01cc0c3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:41.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.370+0000 7f01d239d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01cc156310 0x7f01cc14cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d239d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01cc1566e0 0x7f01cc14fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d239d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc0b0920 con 0x7f01cc1566e0 2023-10-06T03:49:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d239d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc14f550 con 0x7f01cc156310 2023-10-06T03:49:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d239d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc0b0b10 con 0x7f01cc0cb900 2023-10-06T03:49:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d1b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01cc1566e0 0x7f01cc14fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d1b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01cc1566e0 0x7f01cc14fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46722/0 (socket says 172.21.15.55:46722) 2023-10-06T03:49:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d1b9c700 1 -- 172.21.15.55:0/1903070806 learned_addr learned my addr 172.21.15.55:0/1903070806 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:41.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d139b700 1 --2- 172.21.15.55:0/1903070806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc0cb900 0x7f01cc0c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:41.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d0b9a700 1 --2- 172.21.15.55:0/1903070806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01cc156310 0x7f01cc14cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:41.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d1b9c700 1 -- 172.21.15.55:0/1903070806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc0cb900 msgr2=0x7f01cc0c3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:41.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d1b9c700 1 --2- 172.21.15.55:0/1903070806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc0cb900 0x7f01cc0c3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:41.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d1b9c700 1 -- 172.21.15.55:0/1903070806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01cc156310 msgr2=0x7f01cc14cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:41.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d1b9c700 1 --2- 172.21.15.55:0/1903070806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01cc156310 0x7f01cc14cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:41.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d1b9c700 1 -- 172.21.15.55:0/1903070806 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c400bcf0 con 0x7f01cc1566e0 2023-10-06T03:49:41.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.371+0000 7f01d1b9c700 1 --2- 172.21.15.55:0/1903070806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01cc1566e0 0x7f01cc14fc30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f01cc06fe40 tx=0x7f01c401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:41.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.372+0000 7f01c17fa700 1 -- 172.21.15.55:0/1903070806 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c400ac90 con 0x7f01cc1566e0 2023-10-06T03:49:41.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.372+0000 7f01c17fa700 1 -- 172.21.15.55:0/1903070806 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c400adf0 con 0x7f01cc1566e0 2023-10-06T03:49:41.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.372+0000 7f01d239d700 1 -- 172.21.15.55:0/1903070806 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01cc0b0df0 con 0x7f01cc1566e0 2023-10-06T03:49:41.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.372+0000 7f01c17fa700 1 -- 172.21.15.55:0/1903070806 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c401d680 con 0x7f01cc1566e0 2023-10-06T03:49:41.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.372+0000 7f01d239d700 1 -- 172.21.15.55:0/1903070806 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01cc0c39a0 con 0x7f01cc1566e0 2023-10-06T03:49:41.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.373+0000 7f01d239d700 1 -- 172.21.15.55:0/1903070806 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01cc0b1310 con 0x7f01cc1566e0 2023-10-06T03:49:41.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.374+0000 7f01c17fa700 1 -- 172.21.15.55:0/1903070806 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01c4003040 con 0x7f01cc1566e0 2023-10-06T03:49:41.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.375+0000 7f01c17fa700 1 --2- 172.21.15.55:0/1903070806 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01b0066f30 0x7f01b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:41.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.375+0000 7f01c17fa700 1 -- 172.21.15.55:0/1903070806 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f01c4085eb0 con 0x7f01cc1566e0 2023-10-06T03:49:41.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.375+0000 7f01d139b700 1 --2- 172.21.15.55:0/1903070806 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01b0066f30 0x7f01b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:41.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.376+0000 7f01d139b700 1 --2- 172.21.15.55:0/1903070806 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01b0066f30 0x7f01b00693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f01c80011e0 tx=0x7f01c8009040 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:41.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.379+0000 7f01c17fa700 1 -- 172.21.15.55:0/1903070806 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01cc0b1310 con 0x7f01cc1566e0 2023-10-06T03:49:41.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:41.597+0000 7f01d239d700 1 -- 172.21.15.55:0/1903070806 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f01cc0b1310 con 0x7f01cc1566e0 2023-10-06T03:49:42.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-06T03:49:42.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-10-06T03:49:42.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-10-06T03:49:42.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.732+0000 7f01c17fa700 1 -- 172.21.15.55:0/1903070806 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v103) v1 ==== 97+0+0 (secure 0 0 0) 0x7f01cc0b1310 con 0x7f01cc1566e0 2023-10-06T03:49:42.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.735+0000 7f01d239d700 1 -- 172.21.15.55:0/1903070806 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01b0066f30 msgr2=0x7f01b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:42.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.735+0000 7f01d239d700 1 --2- 172.21.15.55:0/1903070806 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01b0066f30 0x7f01b00693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f01c80011e0 tx=0x7f01c8009040 comp rx=0 tx=0).stop 2023-10-06T03:49:42.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.735+0000 7f01d239d700 1 -- 172.21.15.55:0/1903070806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01cc1566e0 msgr2=0x7f01cc14fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:42.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.735+0000 7f01d239d700 1 --2- 172.21.15.55:0/1903070806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01cc1566e0 0x7f01cc14fc30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f01cc06fe40 tx=0x7f01c401a000 comp rx=0 tx=0).stop 2023-10-06T03:49:42.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.735+0000 7f01d239d700 1 -- 172.21.15.55:0/1903070806 shutdown_connections 2023-10-06T03:49:42.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.735+0000 7f01d239d700 1 --2- 172.21.15.55:0/1903070806 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01b0066f30 0x7f01b00693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:42.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.735+0000 7f01d239d700 1 --2- 172.21.15.55:0/1903070806 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01cc1566e0 0x7f01cc14fc30 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:42.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.735+0000 7f01d239d700 1 --2- 172.21.15.55:0/1903070806 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01cc0cb900 0x7f01cc0c3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:42.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.735+0000 7f01d239d700 1 --2- 172.21.15.55:0/1903070806 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01cc156310 0x7f01cc14cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:42.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.735+0000 7f01d239d700 1 -- 172.21.15.55:0/1903070806 >> 172.21.15.55:0/1903070806 conn(0x7f01cc0cc1f0 msgr2=0x7f01cc1518c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:42.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.735+0000 7f01d239d700 1 -- 172.21.15.55:0/1903070806 shutdown_connections 2023-10-06T03:49:42.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.735+0000 7f01d239d700 1 -- 172.21.15.55:0/1903070806 wait complete. 2023-10-06T03:49:42.738 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2023-10-06T03:49:42.758 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:49:42.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.948+0000 7fe1ce3ff700 1 Processor -- start 2023-10-06T03:49:42.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.948+0000 7fe1ce3ff700 1 -- start start 2023-10-06T03:49:42.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.949+0000 7fe1ce3ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1c80b1420 0x7fe1c80ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:42.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.949+0000 7fe1ce3ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c80ccf90 0x7fe1c80c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:42.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.949+0000 7fe1ce3ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c80cc300 con 0x7fe1c80cbf30 2023-10-06T03:49:42.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.949+0000 7fe1ce3ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c8156bf0 con 0x7fe1c80b1420 2023-10-06T03:49:42.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.949+0000 7fe1ce3ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80ae950 con 0x7fe1c80ccf90 2023-10-06T03:49:42.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.950+0000 7fe1c77fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1c80b1420 0x7fe1c80ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:42.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.950+0000 7fe1c77fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe1c80b1420 0x7fe1c80ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46738/0 (socket says 172.21.15.55:46738) 2023-10-06T03:49:42.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.950+0000 7fe1c77fe700 1 -- 172.21.15.55:0/45704965 learned_addr learned my addr 172.21.15.55:0/45704965 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:42.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.950+0000 7fe1cc99c700 1 --2- 172.21.15.55:0/45704965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c80ccf90 0x7fe1c80c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:42.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.950+0000 7fe1c77fe700 1 -- 172.21.15.55:0/45704965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c80ccf90 msgr2=0x7fe1c80c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:42.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.950+0000 7fe1c77fe700 1 --2- 172.21.15.55:0/45704965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c80ccf90 0x7fe1c80c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:42.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.950+0000 7fe1c77fe700 1 -- 172.21.15.55:0/45704965 >> v1:172.21.15.163:6789/0 conn(0x7fe1c80cbf30 legacy=0x7fe1c80b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:42.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.950+0000 7fe1c77fe700 1 -- 172.21.15.55:0/45704965 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80ad370 con 0x7fe1c80b1420 2023-10-06T03:49:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.950+0000 7fe1c77fe700 1 --2- 172.21.15.55:0/45704965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c80b1420 0x7fe1c80ae410 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00bc80 tx=0x7fe1bc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62cac9ee542bf8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.951+0000 7fe1c6ffd700 1 -- 172.21.15.55:0/45704965 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00dbb0 con 0x7fe1c80b1420 2023-10-06T03:49:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.951+0000 7fe1c6ffd700 1 -- 172.21.15.55:0/45704965 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1bc00dd10 con 0x7fe1c80b1420 2023-10-06T03:49:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.951+0000 7fe1c6ffd700 1 -- 172.21.15.55:0/45704965 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc0196b0 con 0x7fe1c80b1420 2023-10-06T03:49:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.951+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/45704965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c80b1420 msgr2=0x7fe1c80ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:42.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.951+0000 7fe1ce3ff700 1 --2- 172.21.15.55:0/45704965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c80b1420 0x7fe1c80ae410 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00bc80 tx=0x7fe1bc00b4f0 comp rx=0 tx=0).stop 2023-10-06T03:49:42.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.951+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/45704965 shutdown_connections 2023-10-06T03:49:42.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.951+0000 7fe1ce3ff700 1 --2- 172.21.15.55:0/45704965 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c80ccf90 0x7fe1c80c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:42.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.951+0000 7fe1ce3ff700 1 --2- 172.21.15.55:0/45704965 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c80b1420 0x7fe1c80ae410 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:42.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.951+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/45704965 >> 172.21.15.55:0/45704965 conn(0x7fe1c806a320 msgr2=0x7fe1c80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:42.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.951+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/45704965 shutdown_connections 2023-10-06T03:49:42.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.951+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/45704965 wait complete. 2023-10-06T03:49:42.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.953+0000 7fe1ce3ff700 1 Processor -- start 2023-10-06T03:49:42.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.953+0000 7fe1ce3ff700 1 -- start start 2023-10-06T03:49:42.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.954+0000 7fe1ce3ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c80b1420 0x7fe1c8155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:42.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.954+0000 7fe1ce3ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c80cbf30 0x7fe1c816e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:42.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.954+0000 7fe1ce3ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c80ccf90 0x7fe1c80c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:42.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.954+0000 7fe1ce3ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80c1ad0 con 0x7fe1c80b1420 2023-10-06T03:49:42.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.954+0000 7fe1ce3ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80c1db0 con 0x7fe1c80ccf90 2023-10-06T03:49:42.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.954+0000 7fe1ce3ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80c2090 con 0x7fe1c80cbf30 2023-10-06T03:49:42.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.954+0000 7fe1c7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c80b1420 0x7fe1c8155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:42.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.954+0000 7fe1c7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c80b1420 0x7fe1c8155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46740/0 (socket says 172.21.15.55:46740) 2023-10-06T03:49:42.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.954+0000 7fe1c7fff700 1 -- 172.21.15.55:0/2204845317 learned_addr learned my addr 172.21.15.55:0/2204845317 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.954+0000 7fe1c77fe700 1 --2- 172.21.15.55:0/2204845317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c80cbf30 0x7fe1c816e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.954+0000 7fe1cc99c700 1 --2- 172.21.15.55:0/2204845317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c80ccf90 0x7fe1c80c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.954+0000 7fe1c7fff700 1 -- 172.21.15.55:0/2204845317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c80cbf30 msgr2=0x7fe1c816e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.955+0000 7fe1c7fff700 1 --2- 172.21.15.55:0/2204845317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c80cbf30 0x7fe1c816e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.955+0000 7fe1c7fff700 1 -- 172.21.15.55:0/2204845317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c80ccf90 msgr2=0x7fe1c80c2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:42.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.955+0000 7fe1c7fff700 1 --2- 172.21.15.55:0/2204845317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c80ccf90 0x7fe1c80c2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:42.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.955+0000 7fe1c7fff700 1 -- 172.21.15.55:0/2204845317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc00b790 con 0x7fe1c80b1420 2023-10-06T03:49:42.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.955+0000 7fe1c7fff700 1 --2- 172.21.15.55:0/2204845317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c80b1420 0x7fe1c8155640 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800c590 tx=0x7fe1b800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:42.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.955+0000 7fe1affff700 1 -- 172.21.15.55:0/2204845317 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8011590 con 0x7fe1c80b1420 2023-10-06T03:49:42.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.955+0000 7fe1affff700 1 -- 172.21.15.55:0/2204845317 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b8011710 con 0x7fe1c80b1420 2023-10-06T03:49:42.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.955+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/2204845317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c80c2370 con 0x7fe1c80b1420 2023-10-06T03:49:42.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.955+0000 7fe1affff700 1 -- 172.21.15.55:0/2204845317 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8011ad0 con 0x7fe1c80b1420 2023-10-06T03:49:42.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.955+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/2204845317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c8155b80 con 0x7fe1c80b1420 2023-10-06T03:49:42.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.956+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/2204845317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c80c2890 con 0x7fe1c80b1420 2023-10-06T03:49:42.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.957+0000 7fe1affff700 1 -- 172.21.15.55:0/2204845317 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1b801b3b0 con 0x7fe1c80b1420 2023-10-06T03:49:42.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.958+0000 7fe1affff700 1 --2- 172.21.15.55:0/2204845317 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe1b4066f30 0x7fe1b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:42.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.958+0000 7fe1affff700 1 -- 172.21.15.55:0/2204845317 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fe1b8083a60 con 0x7fe1c80b1420 2023-10-06T03:49:42.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.958+0000 7fe1c77fe700 1 --2- 172.21.15.55:0/2204845317 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe1b4066f30 0x7fe1b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:42.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.959+0000 7fe1c77fe700 1 --2- 172.21.15.55:0/2204845317 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe1b4066f30 0x7fe1b40693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00bc80 tx=0x7fe1bc019000 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:42.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:42.962+0000 7fe1affff700 1 -- 172.21.15.55:0/2204845317 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1b8050ca0 con 0x7fe1c80b1420 2023-10-06T03:49:43.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.243+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/2204845317 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe1c8155d70 con 0x7fe1c80b1420 2023-10-06T03:49:43.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.243+0000 7fe1affff700 1 -- 172.21.15.55:0/2204845317 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe1b8017070 con 0x7fe1c80b1420 2023-10-06T03:49:43.245 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:43.245 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:49:43.245 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2023-10-06T03:49:43.245 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.245 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2023-10-06T03:49:43.246 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9li2RaGRAAgRjtbw3h40vylEPjolzlpw==", 2023-10-06T03:49:43.246 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.246 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:49:43.246 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:49:43.246 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:49:43.247 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.247 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.247 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.247 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.247 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2023-10-06T03:49:43.247 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lXZu7LxAAu3QAtdDxtD/AuuqGG12sDw==", 2023-10-06T03:49:43.247 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.247 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:49:43.248 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:49:43.248 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:49:43.248 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.248 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.248 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.248 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.248 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2023-10-06T03:49:43.248 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lS22fJBAAZi6CfyLyJAFgIkaarad1mA==", 2023-10-06T03:49:43.248 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.248 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:49:43.249 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:49:43.249 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:49:43.249 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.249 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.249 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.249 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.249 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2023-10-06T03:49:43.249 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9ll0RIOhAAGtGdPv4Y2bge7xIv3eu2NQ==", 2023-10-06T03:49:43.249 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.250 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:49:43.250 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:49:43.250 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:49:43.250 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.250 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.250 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.250 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.250 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2023-10-06T03:49:43.250 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDHfx9lmTeaDRAAuycc9i7fPj8xrDkuZCAGVg==", 2023-10-06T03:49:43.250 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.251 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:49:43.251 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:49:43.251 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.251 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.251 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.251 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.251 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2023-10-06T03:49:43.251 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDKfx9lxOdXEhAA9S8tdOvhnmBeEbutqnUsZg==", 2023-10-06T03:49:43.251 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.251 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:49:43.251 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:49:43.251 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.252 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.252 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.252 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.252 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2023-10-06T03:49:43.252 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDNfx9l76I+GRAAjaj0gM7An9BCyGmfQgaDFw==", 2023-10-06T03:49:43.252 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.252 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:49:43.252 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:49:43.252 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.252 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.252 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.253 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.253 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2023-10-06T03:49:43.253 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDQfx9lsPK+HxAAzQCV0EuBenqOkUpiQW9uog==", 2023-10-06T03:49:43.253 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.253 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:49:43.253 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:49:43.253 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.253 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.254 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.254 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.254 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2023-10-06T03:49:43.254 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDWfx9lSHNrMxAAIn7awDmljO2cRtKyDalWnw==", 2023-10-06T03:49:43.254 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.254 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:49:43.254 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:49:43.255 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.255 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.255 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.255 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.255 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2023-10-06T03:49:43.255 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDZfx9lFoWuOBAAhzQOt3YChvuv0MhrspOjPw==", 2023-10-06T03:49:43.255 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.255 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:49:43.255 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:49:43.255 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.255 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.256 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.256 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.256 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2023-10-06T03:49:43.256 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDcfx9lB8iSOhAAwamSgGIXZ8owFE4FdbOOBw==", 2023-10-06T03:49:43.256 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.256 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:49:43.256 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:49:43.256 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.256 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.257 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.257 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.257 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2023-10-06T03:49:43.257 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDgfx9lUb3mABAAJ75UO2UQg2r+l+tDYVn0Dg==", 2023-10-06T03:49:43.257 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.257 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:49:43.257 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:49:43.257 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.257 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.257 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.257 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.258 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2023-10-06T03:49:43.258 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lfNvCBxAA1ZJq4XsWTZHVY0u0NdOqIw==", 2023-10-06T03:49:43.258 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.258 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:49:43.258 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2023-10-06T03:49:43.258 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:49:43.258 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.259 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.259 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.259 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2023-10-06T03:49:43.259 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lnvkoFhAAafxsOmItxmwUr1GJrGYJ6g==", 2023-10-06T03:49:43.259 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.259 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:49:43.259 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:49:43.259 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:49:43.259 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:49:43.259 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.259 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.260 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.260 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2023-10-06T03:49:43.260 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9l+9b4DhAAc5XHdikWbXsEblSE3FGScQ==", 2023-10-06T03:49:43.260 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.260 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:49:43.260 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:49:43.260 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:49:43.260 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:49:43.260 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.261 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.261 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.261 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2023-10-06T03:49:43.261 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9luSXrHBAAg6Ufa1Ep/EOgsQtuOPQgYg==", 2023-10-06T03:49:43.261 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.261 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:49:43.261 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:49:43.261 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:49:43.261 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:49:43.261 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.261 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.262 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.262 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2023-10-06T03:49:43.262 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC+fx9lEb50FxAAt0Oq28gsXtQvol7TV8trgg==", 2023-10-06T03:49:43.262 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.262 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:49:43.262 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:49:43.262 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:49:43.262 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.262 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.263 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.263 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.263 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2023-10-06T03:49:43.263 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lQAANEhAAsNiWPqCNEyUS3IELXSC1Cw==", 2023-10-06T03:49:43.263 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.263 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:49:43.263 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T03:49:43.263 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.263 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.263 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:43.263 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:43.263 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2023-10-06T03:49:43.264 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lCwU2ABAAMijxXd8hwgk3XzcdP4NZyg==", 2023-10-06T03:49:43.264 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:49:43.264 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:49:43.264 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T03:49:43.264 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:49:43.264 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.264 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:43.264 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2023-10-06T03:49:43.265 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:49:43.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.245+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/2204845317 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe1b4066f30 msgr2=0x7fe1b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.245+0000 7fe1ce3ff700 1 --2- 172.21.15.55:0/2204845317 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe1b4066f30 0x7fe1b40693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00bc80 tx=0x7fe1bc019000 comp rx=0 tx=0).stop 2023-10-06T03:49:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.245+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/2204845317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c80b1420 msgr2=0x7fe1c8155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.245+0000 7fe1ce3ff700 1 --2- 172.21.15.55:0/2204845317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c80b1420 0x7fe1c8155640 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800c590 tx=0x7fe1b800c8a0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.245+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/2204845317 shutdown_connections 2023-10-06T03:49:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.245+0000 7fe1ce3ff700 1 --2- 172.21.15.55:0/2204845317 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe1b4066f30 0x7fe1b40693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.245+0000 7fe1ce3ff700 1 --2- 172.21.15.55:0/2204845317 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe1c80ccf90 0x7fe1c80c2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.245+0000 7fe1ce3ff700 1 --2- 172.21.15.55:0/2204845317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe1c80b1420 0x7fe1c8155640 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.245+0000 7fe1ce3ff700 1 --2- 172.21.15.55:0/2204845317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe1c80cbf30 0x7fe1c816e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.245+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/2204845317 >> 172.21.15.55:0/2204845317 conn(0x7fe1c806a320 msgr2=0x7fe1c80ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:43.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.245+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/2204845317 shutdown_connections 2023-10-06T03:49:43.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.246+0000 7fe1ce3ff700 1 -- 172.21.15.55:0/2204845317 wait complete. 2023-10-06T03:49:43.268 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:49:43.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.454+0000 7fea6dd9d700 1 Processor -- start 2023-10-06T03:49:43.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.454+0000 7fea6dd9d700 1 -- start start 2023-10-06T03:49:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.455+0000 7fea6dd9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fea68142ea0 0x7fea680b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.455+0000 7fea6dd9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea680ad870 0x7fea6806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.455+0000 7fea6dd9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea680b40f0 con 0x7fea680b3b70 2023-10-06T03:49:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.455+0000 7fea6dd9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea68069640 con 0x7fea68142ea0 2023-10-06T03:49:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.455+0000 7fea6dd9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea68069920 con 0x7fea680ad870 2023-10-06T03:49:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.455+0000 7fea6cd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fea68142ea0 0x7fea680b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:43.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.456+0000 7fea6cd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fea68142ea0 0x7fea680b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55718/0 (socket says 172.21.15.55:55718) 2023-10-06T03:49:43.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.456+0000 7fea6cd9b700 1 -- 172.21.15.55:0/567805479 learned_addr learned my addr 172.21.15.55:0/567805479 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:43.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.456+0000 7fea67fff700 1 --2- 172.21.15.55:0/567805479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea680ad870 0x7fea6806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.456+0000 7fea6cd9b700 1 -- 172.21.15.55:0/567805479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea680ad870 msgr2=0x7fea6806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.456+0000 7fea6cd9b700 1 --2- 172.21.15.55:0/567805479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea680ad870 0x7fea6806b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.456+0000 7fea6cd9b700 1 -- 172.21.15.55:0/567805479 >> v1:172.21.15.163:6789/0 conn(0x7fea680b3b70 legacy=0x7fea6806cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.456+0000 7fea6cd9b700 1 -- 172.21.15.55:0/567805479 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea68069c00 con 0x7fea68142ea0 2023-10-06T03:49:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.456+0000 7fea67fff700 1 --2- 172.21.15.55:0/567805479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea680ad870 0x7fea6806b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:43.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.456+0000 7fea6cd9b700 1 --2- 172.21.15.55:0/567805479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea68142ea0 0x7fea680b3630 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0099b0 tx=0x7fea5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d122f59c57cd7bc0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.456+0000 7fea677fe700 1 -- 172.21.15.55:0/567805479 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c00b860 con 0x7fea68142ea0 2023-10-06T03:49:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.457+0000 7fea677fe700 1 -- 172.21.15.55:0/567805479 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5c00b9c0 con 0x7fea68142ea0 2023-10-06T03:49:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.457+0000 7fea677fe700 1 -- 172.21.15.55:0/567805479 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5c00bd60 con 0x7fea68142ea0 2023-10-06T03:49:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.457+0000 7fea6dd9d700 1 -- 172.21.15.55:0/567805479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea68142ea0 msgr2=0x7fea680b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.457+0000 7fea6dd9d700 1 --2- 172.21.15.55:0/567805479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea68142ea0 0x7fea680b3630 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0099b0 tx=0x7fea5c009250 comp rx=0 tx=0).stop 2023-10-06T03:49:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.457+0000 7fea6dd9d700 1 -- 172.21.15.55:0/567805479 shutdown_connections 2023-10-06T03:49:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.457+0000 7fea6dd9d700 1 --2- 172.21.15.55:0/567805479 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea68142ea0 0x7fea680b3630 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.457+0000 7fea6dd9d700 1 --2- 172.21.15.55:0/567805479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea680ad870 0x7fea6806b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.457+0000 7fea6dd9d700 1 -- 172.21.15.55:0/567805479 >> 172.21.15.55:0/567805479 conn(0x7fea680ce430 msgr2=0x7fea680ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.457+0000 7fea6dd9d700 1 -- 172.21.15.55:0/567805479 shutdown_connections 2023-10-06T03:49:43.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.457+0000 7fea6dd9d700 1 -- 172.21.15.55:0/567805479 wait complete. 2023-10-06T03:49:43.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.459+0000 7fea6dd9d700 1 Processor -- start 2023-10-06T03:49:43.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.459+0000 7fea6dd9d700 1 -- start start 2023-10-06T03:49:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea6dd9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea680b3b70 0x7fea6815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea6dd9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea68142ea0 0x7fea6813f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea6dd9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea681401b0 0x7fea68159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea6dd9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea680c7940 con 0x7fea68142ea0 2023-10-06T03:49:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea6dd9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6813ff30 con 0x7fea681401b0 2023-10-06T03:49:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea6dd9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea680c7be0 con 0x7fea680b3b70 2023-10-06T03:49:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea67fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea68142ea0 0x7fea6813f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea67fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea68142ea0 0x7fea6813f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55732/0 (socket says 172.21.15.55:55732) 2023-10-06T03:49:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea67fff700 1 -- 172.21.15.55:0/270311422 learned_addr learned my addr 172.21.15.55:0/270311422 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea6cd9b700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea680b3b70 0x7fea6815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:43.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea6d59c700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea681401b0 0x7fea68159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:43.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea67fff700 1 -- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea680b3b70 msgr2=0x7fea6815ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:43.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea67fff700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea680b3b70 0x7fea6815ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea67fff700 1 -- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea681401b0 msgr2=0x7fea68159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:43.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea67fff700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea681401b0 0x7fea68159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea67fff700 1 -- 172.21.15.55:0/270311422 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c0094f0 con 0x7fea68142ea0 2023-10-06T03:49:43.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea6d59c700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea681401b0 0x7fea68159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:43.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea6cd9b700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea680b3b70 0x7fea6815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:43.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.460+0000 7fea67fff700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea68142ea0 0x7fea6813f9f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fea5400e8d0 tx=0x7fea54008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:43.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.461+0000 7fea64ff9700 1 -- 172.21.15.55:0/270311422 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5400ee70 con 0x7fea68142ea0 2023-10-06T03:49:43.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.461+0000 7fea64ff9700 1 -- 172.21.15.55:0/270311422 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5400c5e0 con 0x7fea68142ea0 2023-10-06T03:49:43.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.461+0000 7fea6dd9d700 1 -- 172.21.15.55:0/270311422 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea680c7ec0 con 0x7fea68142ea0 2023-10-06T03:49:43.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.461+0000 7fea64ff9700 1 -- 172.21.15.55:0/270311422 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5400c980 con 0x7fea68142ea0 2023-10-06T03:49:43.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.461+0000 7fea6dd9d700 1 -- 172.21.15.55:0/270311422 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6815bfd0 con 0x7fea68142ea0 2023-10-06T03:49:43.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.462+0000 7fea6dd9d700 1 -- 172.21.15.55:0/270311422 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6815c1f0 con 0x7fea68142ea0 2023-10-06T03:49:43.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.463+0000 7fea64ff9700 1 -- 172.21.15.55:0/270311422 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea54013410 con 0x7fea68142ea0 2023-10-06T03:49:43.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.463+0000 7fea64ff9700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fea4c066f30 0x7fea4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:43.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.464+0000 7fea64ff9700 1 -- 172.21.15.55:0/270311422 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fea54084350 con 0x7fea68142ea0 2023-10-06T03:49:43.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.464+0000 7fea6cd9b700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fea4c066f30 0x7fea4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:43.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.465+0000 7fea6cd9b700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fea4c066f30 0x7fea4c0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0099b0 tx=0x7fea5c017000 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:43.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.468+0000 7fea64ff9700 1 -- 172.21.15.55:0/270311422 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea54051590 con 0x7fea68142ea0 2023-10-06T03:49:43.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.672+0000 7fea6dd9d700 1 -- 172.21.15.55:0/270311422 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea6804c3c0 con 0x7fea68142ea0 2023-10-06T03:49:43.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.673+0000 7fea64ff9700 1 -- 172.21.15.55:0/270311422 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+14295 (secure 0 0 0) 0x7fea54054bb0 con 0x7fea68142ea0 2023-10-06T03:49:43.674 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:43.675 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":103,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:49:42.726001+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:49:43.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.675+0000 7fea6dd9d700 1 -- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fea4c066f30 msgr2=0x7fea4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:43.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.675+0000 7fea6dd9d700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fea4c066f30 0x7fea4c0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0099b0 tx=0x7fea5c017000 comp rx=0 tx=0).stop 2023-10-06T03:49:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.675+0000 7fea6dd9d700 1 -- 172.21.15.55:0/270311422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea68142ea0 msgr2=0x7fea6813f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.675+0000 7fea6dd9d700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea68142ea0 0x7fea6813f9f0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fea5400e8d0 tx=0x7fea54008040 comp rx=0 tx=0).stop 2023-10-06T03:49:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.675+0000 7fea6dd9d700 1 -- 172.21.15.55:0/270311422 shutdown_connections 2023-10-06T03:49:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.675+0000 7fea6dd9d700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fea4c066f30 0x7fea4c0693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.675+0000 7fea6dd9d700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea681401b0 0x7fea68159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.675+0000 7fea6dd9d700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea680b3b70 0x7fea6815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.675+0000 7fea6dd9d700 1 --2- 172.21.15.55:0/270311422 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea68142ea0 0x7fea6813f9f0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.675+0000 7fea6dd9d700 1 -- 172.21.15.55:0/270311422 >> 172.21.15.55:0/270311422 conn(0x7fea680ce430 msgr2=0x7fea68141ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:43.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.675+0000 7fea6dd9d700 1 -- 172.21.15.55:0/270311422 shutdown_connections 2023-10-06T03:49:43.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.675+0000 7fea6dd9d700 1 -- 172.21.15.55:0/270311422 wait complete. 2023-10-06T03:49:43.694 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T03:49:43.695 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:49:43.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.883+0000 7f225759e700 1 Processor -- start 2023-10-06T03:49:43.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.883+0000 7f225759e700 1 -- start start 2023-10-06T03:49:43.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.885+0000 7f225759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f225806f060 0x7f22580ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:43.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.885+0000 7f225759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f225806ebd0 con 0x7f225806f4b0 2023-10-06T03:49:43.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.885+0000 7f225759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f225815aa70 con 0x7f22580cb1a0 2023-10-06T03:49:43.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.885+0000 7f225759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f225815b2e0 con 0x7f225806f060 2023-10-06T03:49:43.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.885+0000 7f2255d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f225806f060 0x7f22580ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:43.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.885+0000 7f2255d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f225806f060 0x7f22580ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55740/0 (socket says 172.21.15.55:55740) 2023-10-06T03:49:43.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.885+0000 7f2255d9b700 1 -- 172.21.15.55:0/3851301981 learned_addr learned my addr 172.21.15.55:0/3851301981 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:43.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.885+0000 7f2256d9d700 1 --1- 172.21.15.55:0/3851301981 >> v1:172.21.15.55:6789/0 conn(0x7f225806f4b0 0x7f22580c9db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51808/0 (socket says 172.21.15.55:51808) 2023-10-06T03:49:43.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.886+0000 7f2255d9b700 1 -- 172.21.15.55:0/3851301981 >> v1:172.21.15.163:6789/0 conn(0x7f22580cb1a0 legacy=0x7f225815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:43.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.886+0000 7f2255d9b700 1 -- 172.21.15.55:0/3851301981 >> v1:172.21.15.55:6789/0 conn(0x7f225806f4b0 legacy=0x7f22580c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:43.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.886+0000 7f2255d9b700 1 -- 172.21.15.55:0/3851301981 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225815ad20 con 0x7f225806f060 2023-10-06T03:49:43.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.886+0000 7f225559a700 1 -- 172.21.15.55:0/3851301981 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351835521 0 0) 0x7f225806ebd0 con 0x7f225806f4b0 2023-10-06T03:49:43.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.886+0000 7f2255d9b700 1 --2- 172.21.15.55:0/3851301981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f225806f060 0x7f22580ae0d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f224c0099b0 tx=0x7f224c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c7238c94c1e37f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:43.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.886+0000 7f225559a700 1 -- 172.21.15.55:0/3851301981 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c00b8f0 con 0x7f225806f060 2023-10-06T03:49:43.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.886+0000 7f225559a700 1 -- 172.21.15.55:0/3851301981 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224c00ba50 con 0x7f225806f060 2023-10-06T03:49:43.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.886+0000 7f225559a700 1 -- 172.21.15.55:0/3851301981 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c00bdf0 con 0x7f225806f060 2023-10-06T03:49:43.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.887+0000 7f225759e700 1 -- 172.21.15.55:0/3851301981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f225806f060 msgr2=0x7f22580ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:43.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.887+0000 7f225759e700 1 --2- 172.21.15.55:0/3851301981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f225806f060 0x7f22580ae0d0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f224c0099b0 tx=0x7f224c009250 comp rx=0 tx=0).stop 2023-10-06T03:49:43.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.887+0000 7f225759e700 1 -- 172.21.15.55:0/3851301981 shutdown_connections 2023-10-06T03:49:43.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.887+0000 7f225759e700 1 --2- 172.21.15.55:0/3851301981 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f225806f060 0x7f22580ae0d0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.887+0000 7f225759e700 1 -- 172.21.15.55:0/3851301981 >> 172.21.15.55:0/3851301981 conn(0x7f22580cc1e0 msgr2=0x7f22580cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:43.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.887+0000 7f225759e700 1 -- 172.21.15.55:0/3851301981 shutdown_connections 2023-10-06T03:49:43.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.887+0000 7f225759e700 1 -- 172.21.15.55:0/3851301981 wait complete. 2023-10-06T03:49:43.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.888+0000 7f225759e700 1 Processor -- start 2023-10-06T03:49:43.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.889+0000 7f225759e700 1 -- start start 2023-10-06T03:49:43.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.889+0000 7f225759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f225806f060 0x7f22580c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:43.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.889+0000 7f225759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f225806f4b0 0x7f225815c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:43.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.889+0000 7f225759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22580cb1a0 0x7f225815c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:43.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.889+0000 7f225759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2258157b80 con 0x7f225806f4b0 2023-10-06T03:49:43.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.889+0000 7f225759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22580c3ce0 con 0x7f22580cb1a0 2023-10-06T03:49:43.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.889+0000 7f225759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22580c3fc0 con 0x7f225806f060 2023-10-06T03:49:43.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.890+0000 7f2255d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f225806f4b0 0x7f225815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:43.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.890+0000 7f2255d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f225806f4b0 0x7f225815c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55752/0 (socket says 172.21.15.55:55752) 2023-10-06T03:49:43.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.890+0000 7f2255d9b700 1 -- 172.21.15.55:0/3533470087 learned_addr learned my addr 172.21.15.55:0/3533470087 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:43.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.890+0000 7f2256d9d700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22580cb1a0 0x7f225815c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:43.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.890+0000 7f2255d9b700 1 -- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f225806f060 msgr2=0x7f22580c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:43.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.890+0000 7f225659c700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f225806f060 0x7f22580c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:43.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.890+0000 7f2255d9b700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f225806f060 0x7f22580c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.890+0000 7f2255d9b700 1 -- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22580cb1a0 msgr2=0x7f225815c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:43.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.890+0000 7f2255d9b700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22580cb1a0 0x7f225815c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:43.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.890+0000 7f2255d9b700 1 -- 172.21.15.55:0/3533470087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c0094f0 con 0x7f225806f4b0 2023-10-06T03:49:43.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.890+0000 7f2256d9d700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22580cb1a0 0x7f225815c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:43.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.890+0000 7f225659c700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f225806f060 0x7f22580c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:43.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.891+0000 7f2255d9b700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f225806f4b0 0x7f225815c000 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f224c006f50 tx=0x7f224c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:43.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.891+0000 7f22467fc700 1 -- 172.21.15.55:0/3533470087 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c01f070 con 0x7f225806f4b0 2023-10-06T03:49:43.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.891+0000 7f22467fc700 1 -- 172.21.15.55:0/3533470087 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224c011c80 con 0x7f225806f4b0 2023-10-06T03:49:43.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.891+0000 7f22467fc700 1 -- 172.21.15.55:0/3533470087 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c025420 con 0x7f225806f4b0 2023-10-06T03:49:43.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.891+0000 7f225759e700 1 -- 172.21.15.55:0/3533470087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22580c42a0 con 0x7f225806f4b0 2023-10-06T03:49:43.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.891+0000 7f225759e700 1 -- 172.21.15.55:0/3533470087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22580c2b60 con 0x7f225806f4b0 2023-10-06T03:49:43.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.892+0000 7f225759e700 1 -- 172.21.15.55:0/3533470087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2258157950 con 0x7f225806f4b0 2023-10-06T03:49:43.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.893+0000 7f22467fc700 1 -- 172.21.15.55:0/3533470087 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f224c023070 con 0x7f225806f4b0 2023-10-06T03:49:43.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.894+0000 7f22467fc700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2238066f30 0x7f22380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:43.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.894+0000 7f22467fc700 1 -- 172.21.15.55:0/3533470087 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f224c0857a0 con 0x7f225806f4b0 2023-10-06T03:49:43.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.894+0000 7f225659c700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2238066f30 0x7f22380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:43.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.895+0000 7f225659c700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2238066f30 0x7f22380693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f22480017a0 tx=0x7f2248009040 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:43.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:43.898+0000 7f22467fc700 1 -- 172.21.15.55:0/3533470087 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f224c052a90 con 0x7f225806f4b0 2023-10-06T03:49:44.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.098+0000 7f225759e700 1 -- 172.21.15.55:0/3533470087 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2258157cc0 con 0x7f225806f4b0 2023-10-06T03:49:44.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.099+0000 7f22467fc700 1 -- 172.21.15.55:0/3533470087 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v103) v1 ==== 92+0+10 (secure 0 0 0) 0x7f224c02a020 con 0x7f225806f4b0 2023-10-06T03:49:44.100 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:49:44.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.101+0000 7f225759e700 1 -- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2238066f30 msgr2=0x7f22380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.101+0000 7f225759e700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2238066f30 0x7f22380693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f22480017a0 tx=0x7f2248009040 comp rx=0 tx=0).stop 2023-10-06T03:49:44.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.101+0000 7f225759e700 1 -- 172.21.15.55:0/3533470087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f225806f4b0 msgr2=0x7f225815c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.101+0000 7f225759e700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f225806f4b0 0x7f225815c000 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f224c006f50 tx=0x7f224c017000 comp rx=0 tx=0).stop 2023-10-06T03:49:44.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.101+0000 7f225759e700 1 -- 172.21.15.55:0/3533470087 shutdown_connections 2023-10-06T03:49:44.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.101+0000 7f225759e700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2238066f30 0x7f22380693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.101+0000 7f225759e700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22580cb1a0 0x7f225815c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.101+0000 7f225759e700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f225806f060 0x7f22580c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.101+0000 7f225759e700 1 --2- 172.21.15.55:0/3533470087 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f225806f4b0 0x7f225815c000 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.101+0000 7f225759e700 1 -- 172.21.15.55:0/3533470087 >> 172.21.15.55:0/3533470087 conn(0x7f22580cc1e0 msgr2=0x7f22580af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:44.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.101+0000 7f225759e700 1 -- 172.21.15.55:0/3533470087 shutdown_connections 2023-10-06T03:49:44.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.101+0000 7f225759e700 1 -- 172.21.15.55:0/3533470087 wait complete. 2023-10-06T03:49:44.122 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:49:44.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.310+0000 7fc69979d700 1 Processor -- start 2023-10-06T03:49:44.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.310+0000 7fc69979d700 1 -- start start 2023-10-06T03:49:44.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.311+0000 7fc69979d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6940b4e70 0x7fc6940ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:44.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.311+0000 7fc69979d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc6940b1b40 0x7fc69415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:44.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.311+0000 7fc69979d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6940cfb00 con 0x7fc6940b16d0 2023-10-06T03:49:44.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.311+0000 7fc69979d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6940b5c40 con 0x7fc6940b1b40 2023-10-06T03:49:44.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.311+0000 7fc69979d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6940b5f20 con 0x7fc6940b4e70 2023-10-06T03:49:44.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.312+0000 7fc6937fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc6940b16d0 0x7fc69415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51824/0 (socket says 172.21.15.55:51824) 2023-10-06T03:49:44.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.312+0000 7fc6937fe700 1 -- 172.21.15.55:0/2408900133 learned_addr learned my addr 172.21.15.55:0/2408900133 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:44.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.312+0000 7fc693fff700 1 --2- 172.21.15.55:0/2408900133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6940b4e70 0x7fc6940ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:44.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.312+0000 7fc698f9c700 1 --2- 172.21.15.55:0/2408900133 >> v2:172.21.15.163:3300/0 conn(0x7fc6940b1b40 0x7fc69415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:44.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.312+0000 7fc692ffd700 1 -- 172.21.15.55:0/2408900133 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314554506 0 0) 0x7fc6940cfb00 con 0x7fc6940b16d0 2023-10-06T03:49:44.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.312+0000 7fc692ffd700 1 -- 172.21.15.55:0/2408900133 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc67c003570 con 0x7fc6940b16d0 2023-10-06T03:49:44.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.312+0000 7fc693fff700 1 -- 172.21.15.55:0/2408900133 >> v2:172.21.15.163:3300/0 conn(0x7fc6940b1b40 msgr2=0x7fc69415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.312+0000 7fc693fff700 1 --2- 172.21.15.55:0/2408900133 >> v2:172.21.15.163:3300/0 conn(0x7fc6940b1b40 0x7fc69415d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.312+0000 7fc692ffd700 1 -- 172.21.15.55:0/2408900133 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2217769178 0 0) 0x7fc67c003570 con 0x7fc6940b16d0 2023-10-06T03:49:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.312+0000 7fc693fff700 1 -- 172.21.15.55:0/2408900133 >> v1:172.21.15.55:6789/0 conn(0x7fc6940b16d0 legacy=0x7fc69415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.312+0000 7fc693fff700 1 -- 172.21.15.55:0/2408900133 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6940b6200 con 0x7fc6940b4e70 2023-10-06T03:49:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.312+0000 7fc698f9c700 1 --2- 172.21.15.55:0/2408900133 >> v2:172.21.15.163:3300/0 conn(0x7fc6940b1b40 0x7fc69415d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.313+0000 7fc693fff700 1 --2- 172.21.15.55:0/2408900133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6940b4e70 0x7fc6940ccff0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc6840098e0 tx=0x7fc6840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1cf446e55f41a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.313+0000 7fc692ffd700 1 -- 172.21.15.55:0/2408900133 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688001470 con 0x7fc6940b4e70 2023-10-06T03:49:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.313+0000 7fc692ffd700 1 -- 172.21.15.55:0/2408900133 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc684011420 con 0x7fc6940b4e70 2023-10-06T03:49:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.313+0000 7fc692ffd700 1 -- 172.21.15.55:0/2408900133 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6840117c0 con 0x7fc6940b4e70 2023-10-06T03:49:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.313+0000 7fc69979d700 1 -- 172.21.15.55:0/2408900133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6940b4e70 msgr2=0x7fc6940ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.314+0000 7fc69979d700 1 --2- 172.21.15.55:0/2408900133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6940b4e70 0x7fc6940ccff0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc6840098e0 tx=0x7fc6840091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.314+0000 7fc69979d700 1 -- 172.21.15.55:0/2408900133 shutdown_connections 2023-10-06T03:49:44.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.314+0000 7fc69979d700 1 --2- 172.21.15.55:0/2408900133 >> v2:172.21.15.163:3300/0 conn(0x7fc6940b1b40 0x7fc69415d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.314+0000 7fc69979d700 1 --2- 172.21.15.55:0/2408900133 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6940b4e70 0x7fc6940ccff0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.314+0000 7fc69979d700 1 -- 172.21.15.55:0/2408900133 >> 172.21.15.55:0/2408900133 conn(0x7fc6940d10c0 msgr2=0x7fc6940affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:44.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.314+0000 7fc69979d700 1 -- 172.21.15.55:0/2408900133 shutdown_connections 2023-10-06T03:49:44.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.314+0000 7fc69979d700 1 -- 172.21.15.55:0/2408900133 wait complete. 2023-10-06T03:49:44.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.315+0000 7fc69979d700 1 Processor -- start 2023-10-06T03:49:44.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.315+0000 7fc69979d700 1 -- start start 2023-10-06T03:49:44.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.316+0000 7fc69979d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6940b16d0 0x7fc6940b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:44.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.316+0000 7fc69979d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6940b1b40 0x7fc694156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:44.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.316+0000 7fc69979d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6940b4e70 0x7fc694157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:44.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.316+0000 7fc69979d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69416aea0 con 0x7fc6940b4e70 2023-10-06T03:49:44.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.316+0000 7fc69979d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69416e910 con 0x7fc6940b1b40 2023-10-06T03:49:44.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.316+0000 7fc69979d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69416a850 con 0x7fc6940b16d0 2023-10-06T03:49:44.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.316+0000 7fc698f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6940b4e70 0x7fc694157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:44.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.316+0000 7fc698f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6940b4e70 0x7fc694157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55760/0 (socket says 172.21.15.55:55760) 2023-10-06T03:49:44.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.316+0000 7fc698f9c700 1 -- 172.21.15.55:0/991955553 learned_addr learned my addr 172.21.15.55:0/991955553 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:44.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.316+0000 7fc6937fe700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6940b1b40 0x7fc694156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:44.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.317+0000 7fc693fff700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6940b16d0 0x7fc6940b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:44.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.317+0000 7fc698f9c700 1 -- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6940b16d0 msgr2=0x7fc6940b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.317+0000 7fc698f9c700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6940b16d0 0x7fc6940b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.317+0000 7fc698f9c700 1 -- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6940b1b40 msgr2=0x7fc694156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.317+0000 7fc698f9c700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6940b1b40 0x7fc694156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.317+0000 7fc698f9c700 1 -- 172.21.15.55:0/991955553 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c009a20 con 0x7fc6940b4e70 2023-10-06T03:49:44.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.317+0000 7fc693fff700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6940b16d0 0x7fc6940b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:44.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.317+0000 7fc698f9c700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6940b4e70 0x7fc694157370 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009fc0 tx=0x7fc68c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:44.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.317+0000 7fc67bfff700 1 -- 172.21.15.55:0/991955553 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c00cc80 con 0x7fc6940b4e70 2023-10-06T03:49:44.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.317+0000 7fc67bfff700 1 -- 172.21.15.55:0/991955553 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c00f040 con 0x7fc6940b4e70 2023-10-06T03:49:44.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.317+0000 7fc69979d700 1 -- 172.21.15.55:0/991955553 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc684009420 con 0x7fc6940b4e70 2023-10-06T03:49:44.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.317+0000 7fc67bfff700 1 -- 172.21.15.55:0/991955553 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c014480 con 0x7fc6940b4e70 2023-10-06T03:49:44.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.317+0000 7fc69979d700 1 -- 172.21.15.55:0/991955553 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69416a500 con 0x7fc6940b4e70 2023-10-06T03:49:44.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.318+0000 7fc69979d700 1 -- 172.21.15.55:0/991955553 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6940ce1c0 con 0x7fc6940b4e70 2023-10-06T03:49:44.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.319+0000 7fc67bfff700 1 -- 172.21.15.55:0/991955553 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc68c003020 con 0x7fc6940b4e70 2023-10-06T03:49:44.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.320+0000 7fc67bfff700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc680066f30 0x7fc6800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:44.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.320+0000 7fc67bfff700 1 -- 172.21.15.55:0/991955553 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fc68c084f60 con 0x7fc6940b4e70 2023-10-06T03:49:44.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.320+0000 7fc693fff700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc680066f30 0x7fc6800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:44.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.321+0000 7fc693fff700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc680066f30 0x7fc6800693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc6840030c0 tx=0x7fc684016000 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:44.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.324+0000 7fc67bfff700 1 -- 172.21.15.55:0/991955553 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc68c0521a0 con 0x7fc6940b4e70 2023-10-06T03:49:44.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.526+0000 7fc69979d700 1 -- 172.21.15.55:0/991955553 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc69416afe0 con 0x7fc6940b4e70 2023-10-06T03:49:44.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.526+0000 7fc67bfff700 1 -- 172.21.15.55:0/991955553 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc68c018020 con 0x7fc6940b4e70 2023-10-06T03:49:44.528 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:49:44.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.529+0000 7fc69979d700 1 -- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc680066f30 msgr2=0x7fc6800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.529+0000 7fc69979d700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc680066f30 0x7fc6800693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc6840030c0 tx=0x7fc684016000 comp rx=0 tx=0).stop 2023-10-06T03:49:44.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.529+0000 7fc69979d700 1 -- 172.21.15.55:0/991955553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6940b4e70 msgr2=0x7fc694157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.529+0000 7fc69979d700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6940b4e70 0x7fc694157370 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009fc0 tx=0x7fc68c00b040 comp rx=0 tx=0).stop 2023-10-06T03:49:44.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.529+0000 7fc69979d700 1 -- 172.21.15.55:0/991955553 shutdown_connections 2023-10-06T03:49:44.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.529+0000 7fc69979d700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc680066f30 0x7fc6800693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.529+0000 7fc69979d700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6940b4e70 0x7fc694157370 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.529+0000 7fc69979d700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6940b16d0 0x7fc6940b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.529+0000 7fc69979d700 1 --2- 172.21.15.55:0/991955553 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6940b1b40 0x7fc694156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.529+0000 7fc69979d700 1 -- 172.21.15.55:0/991955553 >> 172.21.15.55:0/991955553 conn(0x7fc6940d10c0 msgr2=0x7fc6940b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:44.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.529+0000 7fc69979d700 1 -- 172.21.15.55:0/991955553 shutdown_connections 2023-10-06T03:49:44.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.529+0000 7fc69979d700 1 -- 172.21.15.55:0/991955553 wait complete. 2023-10-06T03:49:44.548 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:49:44.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.733+0000 7f9ce7fff700 1 Processor -- start 2023-10-06T03:49:44.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.733+0000 7f9ce7fff700 1 -- start start 2023-10-06T03:49:44.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.735+0000 7f9ce7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9ce8156310 0x7f9ce80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:44.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.735+0000 7f9ce7fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce80ca860 con 0x7f9ce80ae710 2023-10-06T03:49:44.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.735+0000 7f9ce7fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce80cab40 con 0x7f9ce81566e0 2023-10-06T03:49:44.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.735+0000 7f9ce7fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce80cae20 con 0x7f9ce8156310 2023-10-06T03:49:44.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.735+0000 7f9ce6ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9ce8156310 0x7f9ce80c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:44.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.735+0000 7f9ce6ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9ce8156310 0x7f9ce80c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55770/0 (socket says 172.21.15.55:55770) 2023-10-06T03:49:44.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.735+0000 7f9ce6ffd700 1 -- 172.21.15.55:0/1022598960 learned_addr learned my addr 172.21.15.55:0/1022598960 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:44.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.736+0000 7f9ce6ffd700 1 -- 172.21.15.55:0/1022598960 >> v1:172.21.15.163:6789/0 conn(0x7f9ce81566e0 legacy=0x7f9ce80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.736+0000 7f9ce6ffd700 1 -- 172.21.15.55:0/1022598960 >> v1:172.21.15.55:6789/0 conn(0x7f9ce80ae710 legacy=0x7f9ce80c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.736+0000 7f9ce6ffd700 1 -- 172.21.15.55:0/1022598960 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce80cb100 con 0x7f9ce8156310 2023-10-06T03:49:44.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.736+0000 7f9ce6ffd700 1 --2- 172.21.15.55:0/1022598960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8156310 0x7f9ce80c9dc0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800bc50 tx=0x7f9cd800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c2f38fb7003259 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:44.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.736+0000 7f9ce5ffb700 1 -- 172.21.15.55:0/1022598960 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd800be60 con 0x7f9ce8156310 2023-10-06T03:49:44.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.737+0000 7f9ce5ffb700 1 -- 172.21.15.55:0/1022598960 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd800de00 con 0x7f9ce8156310 2023-10-06T03:49:44.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.737+0000 7f9ce5ffb700 1 -- 172.21.15.55:0/1022598960 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd80196b0 con 0x7f9ce8156310 2023-10-06T03:49:44.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.737+0000 7f9ce7fff700 1 -- 172.21.15.55:0/1022598960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8156310 msgr2=0x7f9ce80c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.737+0000 7f9ce7fff700 1 --2- 172.21.15.55:0/1022598960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8156310 0x7f9ce80c9dc0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800bc50 tx=0x7f9cd800b4f0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.737+0000 7f9ce7fff700 1 -- 172.21.15.55:0/1022598960 shutdown_connections 2023-10-06T03:49:44.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.737+0000 7f9ce7fff700 1 --2- 172.21.15.55:0/1022598960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce8156310 0x7f9ce80c9dc0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.737+0000 7f9ce7fff700 1 -- 172.21.15.55:0/1022598960 >> 172.21.15.55:0/1022598960 conn(0x7f9ce80cc1f0 msgr2=0x7f9ce80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:44.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.737+0000 7f9ce7fff700 1 -- 172.21.15.55:0/1022598960 shutdown_connections 2023-10-06T03:49:44.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.737+0000 7f9ce7fff700 1 -- 172.21.15.55:0/1022598960 wait complete. 2023-10-06T03:49:44.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.739+0000 7f9ce7fff700 1 Processor -- start 2023-10-06T03:49:44.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.739+0000 7f9ce7fff700 1 -- start start 2023-10-06T03:49:44.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce80ae710 0x7f9ce816ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:44.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ce8156310 0x7f9ce814df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:44.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ce81566e0 0x7f9ce80c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:44.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce7fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce8152a80 con 0x7f9ce80ae710 2023-10-06T03:49:44.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce7fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce816b070 con 0x7f9ce8156310 2023-10-06T03:49:44.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce7fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce8151e70 con 0x7f9ce81566e0 2023-10-06T03:49:44.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce80ae710 0x7f9ce816ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:44.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce80ae710 0x7f9ce816ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55782/0 (socket says 172.21.15.55:55782) 2023-10-06T03:49:44.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce6ffd700 1 -- 172.21.15.55:0/768517066 learned_addr learned my addr 172.21.15.55:0/768517066 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:44.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce67fc700 1 --2- 172.21.15.55:0/768517066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ce8156310 0x7f9ce814df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:44.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce77fe700 1 --2- 172.21.15.55:0/768517066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ce81566e0 0x7f9ce80c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:44.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce6ffd700 1 -- 172.21.15.55:0/768517066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ce81566e0 msgr2=0x7f9ce80c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce6ffd700 1 --2- 172.21.15.55:0/768517066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ce81566e0 0x7f9ce80c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce6ffd700 1 -- 172.21.15.55:0/768517066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ce8156310 msgr2=0x7f9ce814df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce6ffd700 1 --2- 172.21.15.55:0/768517066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ce8156310 0x7f9ce814df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.741+0000 7f9ce6ffd700 1 -- 172.21.15.55:0/768517066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd800b790 con 0x7f9ce80ae710 2023-10-06T03:49:44.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.742+0000 7f9ce6ffd700 1 --2- 172.21.15.55:0/768517066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce80ae710 0x7f9ce816ab30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9ce806fe40 tx=0x7f9cd801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:44.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.742+0000 7f9cceffd700 1 -- 172.21.15.55:0/768517066 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8010850 con 0x7f9ce80ae710 2023-10-06T03:49:44.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.742+0000 7f9cceffd700 1 -- 172.21.15.55:0/768517066 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd80109b0 con 0x7f9ce80ae710 2023-10-06T03:49:44.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.742+0000 7f9ce7fff700 1 -- 172.21.15.55:0/768517066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce8152150 con 0x7f9ce80ae710 2023-10-06T03:49:44.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.742+0000 7f9cceffd700 1 -- 172.21.15.55:0/768517066 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8010d70 con 0x7f9ce80ae710 2023-10-06T03:49:44.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.742+0000 7f9ce7fff700 1 -- 172.21.15.55:0/768517066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce8152710 con 0x7f9ce80ae710 2023-10-06T03:49:44.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.743+0000 7f9ce7fff700 1 -- 172.21.15.55:0/768517066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce8159190 con 0x7f9ce80ae710 2023-10-06T03:49:44.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.744+0000 7f9cceffd700 1 -- 172.21.15.55:0/768517066 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9cd8023600 con 0x7f9ce80ae710 2023-10-06T03:49:44.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.745+0000 7f9cceffd700 1 --2- 172.21.15.55:0/768517066 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cd4066f30 0x7f9cd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:44.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.745+0000 7f9cceffd700 1 -- 172.21.15.55:0/768517066 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f9cd8014070 con 0x7f9ce80ae710 2023-10-06T03:49:44.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.745+0000 7f9ce67fc700 1 --2- 172.21.15.55:0/768517066 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cd4066f30 0x7f9cd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:44.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.746+0000 7f9ce67fc700 1 --2- 172.21.15.55:0/768517066 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cd4066f30 0x7f9cd40693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80c2a50 tx=0x7f9cdc007ec0 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:44.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.752+0000 7f9cceffd700 1 -- 172.21.15.55:0/768517066 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cd80537a0 con 0x7f9ce80ae710 2023-10-06T03:49:44.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.955+0000 7f9ce7fff700 1 -- 172.21.15.55:0/768517066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9ce804c3c0 con 0x7f9ce80ae710 2023-10-06T03:49:44.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.956+0000 7f9cceffd700 1 -- 172.21.15.55:0/768517066 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) v1 ==== 84+0+126 (secure 0 0 0) 0x7f9cd800b5f0 con 0x7f9ce80ae710 2023-10-06T03:49:44.957 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:44.957 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:49:44.958 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:44.958 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2023-10-06T03:49:44.958 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": "rbd" 2023-10-06T03:49:44.958 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:49:44.958 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:44.958 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 2, 2023-10-06T03:49:44.958 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2023-10-06T03:49:44.958 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:44.958 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:49:44.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.958+0000 7f9ce7fff700 1 -- 172.21.15.55:0/768517066 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cd4066f30 msgr2=0x7f9cd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.958+0000 7f9ce7fff700 1 --2- 172.21.15.55:0/768517066 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cd4066f30 0x7f9cd40693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80c2a50 tx=0x7f9cdc007ec0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.958+0000 7f9ce7fff700 1 -- 172.21.15.55:0/768517066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce80ae710 msgr2=0x7f9ce816ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:44.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.958+0000 7f9ce7fff700 1 --2- 172.21.15.55:0/768517066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce80ae710 0x7f9ce816ab30 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f9ce806fe40 tx=0x7f9cd801b000 comp rx=0 tx=0).stop 2023-10-06T03:49:44.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.958+0000 7f9ce7fff700 1 -- 172.21.15.55:0/768517066 shutdown_connections 2023-10-06T03:49:44.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.958+0000 7f9ce7fff700 1 --2- 172.21.15.55:0/768517066 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cd4066f30 0x7f9cd40693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.958+0000 7f9ce7fff700 1 --2- 172.21.15.55:0/768517066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9ce81566e0 0x7f9ce80c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.958+0000 7f9ce7fff700 1 --2- 172.21.15.55:0/768517066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9ce80ae710 0x7f9ce816ab30 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.958+0000 7f9ce7fff700 1 --2- 172.21.15.55:0/768517066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9ce8156310 0x7f9ce814df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:44.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.958+0000 7f9ce7fff700 1 -- 172.21.15.55:0/768517066 >> 172.21.15.55:0/768517066 conn(0x7f9ce80cc1f0 msgr2=0x7f9ce814fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:44.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.958+0000 7f9ce7fff700 1 -- 172.21.15.55:0/768517066 shutdown_connections 2023-10-06T03:49:44.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:44.958+0000 7f9ce7fff700 1 -- 172.21.15.55:0/768517066 wait complete. 2023-10-06T03:49:44.977 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:49:44.977 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:49:45.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.166+0000 7fcb7499d700 1 Processor -- start 2023-10-06T03:49:45.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.166+0000 7fcb7499d700 1 -- start start 2023-10-06T03:49:45.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.168+0000 7fcb7499d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcb70159dc0 0x7fcb70160a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:45.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.168+0000 7fcb7499d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7015a480 con 0x7fcb70153480 2023-10-06T03:49:45.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.168+0000 7fcb7499d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb700afb40 con 0x7fcb70153850 2023-10-06T03:49:45.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.168+0000 7fcb7499d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb700afe20 con 0x7fcb70159dc0 2023-10-06T03:49:45.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.168+0000 7fcb6f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcb70153480 0x7fcb700ae750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51856/0 (socket says 172.21.15.55:51856) 2023-10-06T03:49:45.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.168+0000 7fcb6f7fe700 1 -- 172.21.15.55:0/761840166 learned_addr learned my addr 172.21.15.55:0/761840166 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:45.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.168+0000 7fcb6ffff700 1 --2- 172.21.15.55:0/761840166 >> v2:172.21.15.55:3300/0 conn(0x7fcb70159dc0 0x7fcb70160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:45.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.168+0000 7fcb6e7fc700 1 -- 172.21.15.55:0/761840166 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791800132 0 0) 0x7fcb7015a480 con 0x7fcb70153480 2023-10-06T03:49:45.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.169+0000 7fcb6e7fc700 1 -- 172.21.15.55:0/761840166 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb58003570 con 0x7fcb70153480 2023-10-06T03:49:45.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.169+0000 7fcb6ffff700 1 -- 172.21.15.55:0/761840166 >> v1:172.21.15.163:6789/0 conn(0x7fcb70153850 legacy=0x7fcb701596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:45.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.169+0000 7fcb6e7fc700 1 -- 172.21.15.55:0/761840166 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794180881 0 0) 0x7fcb700afb40 con 0x7fcb70153850 2023-10-06T03:49:45.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.169+0000 7fcb6ffff700 1 -- 172.21.15.55:0/761840166 >> v1:172.21.15.55:6789/0 conn(0x7fcb70153480 legacy=0x7fcb700ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:45.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.169+0000 7fcb6ffff700 1 -- 172.21.15.55:0/761840166 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb700b0100 con 0x7fcb70159dc0 2023-10-06T03:49:45.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.169+0000 7fcb6ffff700 1 --2- 172.21.15.55:0/761840166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb70159dc0 0x7fcb70160a50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009e60 tx=0x7fcb68009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461b10159d0ef539 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:45.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.169+0000 7fcb6e7fc700 1 -- 172.21.15.55:0/761840166 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6800bb60 con 0x7fcb70159dc0 2023-10-06T03:49:45.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.169+0000 7fcb6e7fc700 1 -- 172.21.15.55:0/761840166 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6800bcc0 con 0x7fcb70159dc0 2023-10-06T03:49:45.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.169+0000 7fcb6e7fc700 1 -- 172.21.15.55:0/761840166 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68011460 con 0x7fcb70159dc0 2023-10-06T03:49:45.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.170+0000 7fcb7499d700 1 -- 172.21.15.55:0/761840166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb70159dc0 msgr2=0x7fcb70160a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:45.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.170+0000 7fcb7499d700 1 --2- 172.21.15.55:0/761840166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb70159dc0 0x7fcb70160a50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009e60 tx=0x7fcb68009780 comp rx=0 tx=0).stop 2023-10-06T03:49:45.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.170+0000 7fcb7499d700 1 -- 172.21.15.55:0/761840166 shutdown_connections 2023-10-06T03:49:45.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.170+0000 7fcb7499d700 1 --2- 172.21.15.55:0/761840166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb70159dc0 0x7fcb70160a50 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:45.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.170+0000 7fcb7499d700 1 -- 172.21.15.55:0/761840166 >> 172.21.15.55:0/761840166 conn(0x7fcb70068e20 msgr2=0x7fcb700cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:45.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.170+0000 7fcb7499d700 1 -- 172.21.15.55:0/761840166 shutdown_connections 2023-10-06T03:49:45.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.170+0000 7fcb7499d700 1 -- 172.21.15.55:0/761840166 wait complete. 2023-10-06T03:49:45.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.171+0000 7fcb7499d700 1 Processor -- start 2023-10-06T03:49:45.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.172+0000 7fcb7499d700 1 -- start start 2023-10-06T03:49:45.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.172+0000 7fcb7499d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb70153480 0x7fcb700a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:45.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.172+0000 7fcb7499d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb70153850 0x7fcb70155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:45.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.172+0000 7fcb7499d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb70159dc0 0x7fcb70165070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:45.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.172+0000 7fcb7499d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700a69a0 con 0x7fcb70153850 2023-10-06T03:49:45.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.172+0000 7fcb7499d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb70165b60 con 0x7fcb70159dc0 2023-10-06T03:49:45.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.172+0000 7fcb7499d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb70165e40 con 0x7fcb70153480 2023-10-06T03:49:45.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.173+0000 7fcb6effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb70153850 0x7fcb70155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:45.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.173+0000 7fcb6effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb70153850 0x7fcb70155ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55798/0 (socket says 172.21.15.55:55798) 2023-10-06T03:49:45.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.173+0000 7fcb6effd700 1 -- 172.21.15.55:0/1149965430 learned_addr learned my addr 172.21.15.55:0/1149965430 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:45.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.173+0000 7fcb6f7fe700 1 --2- 172.21.15.55:0/1149965430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb70153480 0x7fcb700a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:45.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.173+0000 7fcb6ffff700 1 --2- 172.21.15.55:0/1149965430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb70159dc0 0x7fcb70165070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:45.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.173+0000 7fcb6effd700 1 -- 172.21.15.55:0/1149965430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb70153480 msgr2=0x7fcb700a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:45.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.173+0000 7fcb6effd700 1 --2- 172.21.15.55:0/1149965430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb70153480 0x7fcb700a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:45.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.173+0000 7fcb6effd700 1 -- 172.21.15.55:0/1149965430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb70159dc0 msgr2=0x7fcb70165070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:45.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.173+0000 7fcb6effd700 1 --2- 172.21.15.55:0/1149965430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb70159dc0 0x7fcb70165070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:45.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.173+0000 7fcb6effd700 1 -- 172.21.15.55:0/1149965430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb68009a20 con 0x7fcb70153850 2023-10-06T03:49:45.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.174+0000 7fcb6effd700 1 --2- 172.21.15.55:0/1149965430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb70153850 0x7fcb70155ef0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400c7e0 tx=0x7fcb64007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:45.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.174+0000 7fcb577fe700 1 -- 172.21.15.55:0/1149965430 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb640105c0 con 0x7fcb70153850 2023-10-06T03:49:45.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.174+0000 7fcb577fe700 1 -- 172.21.15.55:0/1149965430 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb64010740 con 0x7fcb70153850 2023-10-06T03:49:45.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.174+0000 7fcb577fe700 1 -- 172.21.15.55:0/1149965430 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64010b00 con 0x7fcb70153850 2023-10-06T03:49:45.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.174+0000 7fcb7499d700 1 -- 172.21.15.55:0/1149965430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb700a6630 con 0x7fcb70153850 2023-10-06T03:49:45.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.174+0000 7fcb7499d700 1 -- 172.21.15.55:0/1149965430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb70150960 con 0x7fcb70153850 2023-10-06T03:49:45.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.175+0000 7fcb7499d700 1 -- 172.21.15.55:0/1149965430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb70150b80 con 0x7fcb70153850 2023-10-06T03:49:45.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.176+0000 7fcb577fe700 1 -- 172.21.15.55:0/1149965430 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb6401b3b0 con 0x7fcb70153850 2023-10-06T03:49:45.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.177+0000 7fcb577fe700 1 --2- 172.21.15.55:0/1149965430 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcb5c066f60 0x7fcb5c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:45.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.177+0000 7fcb577fe700 1 -- 172.21.15.55:0/1149965430 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fcb64083b10 con 0x7fcb70153850 2023-10-06T03:49:45.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.177+0000 7fcb6f7fe700 1 --2- 172.21.15.55:0/1149965430 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcb5c066f60 0x7fcb5c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:45.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.178+0000 7fcb6f7fe700 1 --2- 172.21.15.55:0/1149965430 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcb5c066f60 0x7fcb5c069410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb701697d0 tx=0x7fcb60009040 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:45.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.181+0000 7fcb577fe700 1 -- 172.21.15.55:0/1149965430 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb64050d50 con 0x7fcb70153850 2023-10-06T03:49:45.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.382+0000 7fcb7499d700 1 -- 172.21.15.55:0/1149965430 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcb700a6490 con 0x7fcb70153850 2023-10-06T03:49:45.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.755+0000 7fcb577fe700 1 -- 172.21.15.55:0/1149965430 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v104) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcb700a6490 con 0x7fcb70153850 2023-10-06T03:49:45.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.758+0000 7fcb7499d700 1 -- 172.21.15.55:0/1149965430 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcb5c066f60 msgr2=0x7fcb5c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:45.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.758+0000 7fcb7499d700 1 --2- 172.21.15.55:0/1149965430 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcb5c066f60 0x7fcb5c069410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb701697d0 tx=0x7fcb60009040 comp rx=0 tx=0).stop 2023-10-06T03:49:45.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.758+0000 7fcb7499d700 1 -- 172.21.15.55:0/1149965430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb70153850 msgr2=0x7fcb70155ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:45.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.758+0000 7fcb7499d700 1 --2- 172.21.15.55:0/1149965430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb70153850 0x7fcb70155ef0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400c7e0 tx=0x7fcb64007df0 comp rx=0 tx=0).stop 2023-10-06T03:49:45.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.759+0000 7fcb7499d700 1 -- 172.21.15.55:0/1149965430 shutdown_connections 2023-10-06T03:49:45.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.759+0000 7fcb7499d700 1 --2- 172.21.15.55:0/1149965430 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcb5c066f60 0x7fcb5c069410 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:45.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.759+0000 7fcb7499d700 1 --2- 172.21.15.55:0/1149965430 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb70159dc0 0x7fcb70165070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:45.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.759+0000 7fcb7499d700 1 --2- 172.21.15.55:0/1149965430 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb70153480 0x7fcb700a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:45.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.759+0000 7fcb7499d700 1 --2- 172.21.15.55:0/1149965430 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb70153850 0x7fcb70155ef0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:45.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.759+0000 7fcb7499d700 1 -- 172.21.15.55:0/1149965430 >> 172.21.15.55:0/1149965430 conn(0x7fcb70068e20 msgr2=0x7fcb700704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:45.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.759+0000 7fcb7499d700 1 -- 172.21.15.55:0/1149965430 shutdown_connections 2023-10-06T03:49:45.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.759+0000 7fcb7499d700 1 -- 172.21.15.55:0/1149965430 wait complete. 2023-10-06T03:49:45.764 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2023-10-06T03:49:45.782 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:49:45.924 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:49:45.929 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:49:45.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.975+0000 7f769ebff700 1 Processor -- start 2023-10-06T03:49:45.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.975+0000 7f769ebff700 1 -- start start 2023-10-06T03:49:45.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.977+0000 7f769ebff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7698159950 0x7f76980cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:45.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.977+0000 7f769ebff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76980af320 0x7f7698069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:45.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.977+0000 7f769ebff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76980ce380 con 0x7f7698069aa0 2023-10-06T03:49:45.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.977+0000 7f769ebff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7698069f20 con 0x7f7698159950 2023-10-06T03:49:45.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.977+0000 7f769ebff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76980cec20 con 0x7f76980af320 2023-10-06T03:49:45.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.978+0000 7f769c99b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7698159950 0x7f76980cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:45.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.978+0000 7f7697fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76980af320 0x7f7698069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:45.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.978+0000 7f769c99b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7698159950 0x7f76980cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55810/0 (socket says 172.21.15.55:55810) 2023-10-06T03:49:45.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.978+0000 7f769c99b700 1 -- 172.21.15.55:0/1746498317 learned_addr learned my addr 172.21.15.55:0/1746498317 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:45.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.978+0000 7f769c99b700 1 -- 172.21.15.55:0/1746498317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76980af320 msgr2=0x7f7698069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:45.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.978+0000 7f769c99b700 1 --2- 172.21.15.55:0/1746498317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76980af320 0x7f7698069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:45.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.978+0000 7f769c99b700 1 -- 172.21.15.55:0/1746498317 >> v1:172.21.15.55:6789/0 conn(0x7f7698069aa0 legacy=0x7f769815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:45.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.978+0000 7f769c99b700 1 -- 172.21.15.55:0/1746498317 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76980afa90 con 0x7f7698159950 2023-10-06T03:49:45.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.978+0000 7f76977fe700 1 -- 172.21.15.55:0/1746498317 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220456336 0 0) 0x7f76980ce380 con 0x7f7698069aa0 2023-10-06T03:49:45.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.979+0000 7f769c99b700 1 --2- 172.21.15.55:0/1746498317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7698159950 0x7f76980cbb90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f768c0099b0 tx=0x7f768c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93526429bf14d4cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:45.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.979+0000 7f76977fe700 1 -- 172.21.15.55:0/1746498317 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c009ea0 con 0x7f7698159950 2023-10-06T03:49:45.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.979+0000 7f76977fe700 1 -- 172.21.15.55:0/1746498317 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c00b860 con 0x7f7698159950 2023-10-06T03:49:45.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.979+0000 7f76977fe700 1 -- 172.21.15.55:0/1746498317 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c00bc00 con 0x7f7698159950 2023-10-06T03:49:45.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.980+0000 7f769ebff700 1 -- 172.21.15.55:0/1746498317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7698159950 msgr2=0x7f76980cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:45.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.980+0000 7f769ebff700 1 --2- 172.21.15.55:0/1746498317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7698159950 0x7f76980cbb90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f768c0099b0 tx=0x7f768c009250 comp rx=0 tx=0).stop 2023-10-06T03:49:45.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.980+0000 7f769ebff700 1 -- 172.21.15.55:0/1746498317 shutdown_connections 2023-10-06T03:49:45.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.980+0000 7f769ebff700 1 --2- 172.21.15.55:0/1746498317 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7698159950 0x7f76980cbb90 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:45.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.980+0000 7f769ebff700 1 --2- 172.21.15.55:0/1746498317 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f76980af320 0x7f7698069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:45.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.980+0000 7f769ebff700 1 -- 172.21.15.55:0/1746498317 >> 172.21.15.55:0/1746498317 conn(0x7f7698068e00 msgr2=0x7f76980d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:45.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.980+0000 7f769ebff700 1 -- 172.21.15.55:0/1746498317 shutdown_connections 2023-10-06T03:49:45.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.980+0000 7f769ebff700 1 -- 172.21.15.55:0/1746498317 wait complete. 2023-10-06T03:49:45.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.981+0000 7f769ebff700 1 Processor -- start 2023-10-06T03:49:45.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.982+0000 7f769ebff700 1 -- start start 2023-10-06T03:49:45.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.982+0000 7f769ebff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7698069aa0 0x7f76980b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:45.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.982+0000 7f769ebff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76980af320 0x7f76980c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:45.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.982+0000 7f769ebff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7698159950 0x7f7698169ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:45.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.982+0000 7f769ebff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769816aa00 con 0x7f76980af320 2023-10-06T03:49:45.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.983+0000 7f769ebff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769816ace0 con 0x7f7698069aa0 2023-10-06T03:49:45.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.983+0000 7f769ebff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769815d670 con 0x7f7698159950 2023-10-06T03:49:45.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.983+0000 7f7697fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76980af320 0x7f76980c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:45.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.983+0000 7f769c99b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7698069aa0 0x7f76980b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:45.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.983+0000 7f7697fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76980af320 0x7f76980c8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55814/0 (socket says 172.21.15.55:55814) 2023-10-06T03:49:45.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.983+0000 7f7697fff700 1 -- 172.21.15.55:0/3506096415 learned_addr learned my addr 172.21.15.55:0/3506096415 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:45.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.983+0000 7f769d19c700 1 --2- 172.21.15.55:0/3506096415 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7698159950 0x7f7698169ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:45.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.983+0000 7f769c99b700 1 -- 172.21.15.55:0/3506096415 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7698159950 msgr2=0x7f7698169ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:45.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.983+0000 7f769c99b700 1 --2- 172.21.15.55:0/3506096415 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7698159950 0x7f7698169ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:45.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.983+0000 7f769c99b700 1 -- 172.21.15.55:0/3506096415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76980af320 msgr2=0x7f76980c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:45.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.983+0000 7f769c99b700 1 --2- 172.21.15.55:0/3506096415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76980af320 0x7f76980c8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:45.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.983+0000 7f769c99b700 1 -- 172.21.15.55:0/3506096415 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768c0094f0 con 0x7f7698069aa0 2023-10-06T03:49:45.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.983+0000 7f7697fff700 1 --2- 172.21.15.55:0/3506096415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76980af320 0x7f76980c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:45.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.984+0000 7f769c99b700 1 --2- 172.21.15.55:0/3506096415 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7698069aa0 0x7f76980b1af0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f76980d0760 tx=0x7f768c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:45.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.984+0000 7f7694ff9700 1 -- 172.21.15.55:0/3506096415 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c000d10 con 0x7f7698069aa0 2023-10-06T03:49:45.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.984+0000 7f7694ff9700 1 -- 172.21.15.55:0/3506096415 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c00b9d0 con 0x7f7698069aa0 2023-10-06T03:49:45.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.984+0000 7f7694ff9700 1 -- 172.21.15.55:0/3506096415 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c025420 con 0x7f7698069aa0 2023-10-06T03:49:45.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.984+0000 7f769ebff700 1 -- 172.21.15.55:0/3506096415 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769815d950 con 0x7f7698069aa0 2023-10-06T03:49:45.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.984+0000 7f769ebff700 1 -- 172.21.15.55:0/3506096415 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76980b2030 con 0x7f7698069aa0 2023-10-06T03:49:45.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.985+0000 7f769ebff700 1 -- 172.21.15.55:0/3506096415 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769815d4d0 con 0x7f7698069aa0 2023-10-06T03:49:45.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.986+0000 7f7694ff9700 1 -- 172.21.15.55:0/3506096415 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f768c023070 con 0x7f7698069aa0 2023-10-06T03:49:45.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.987+0000 7f7694ff9700 1 --2- 172.21.15.55:0/3506096415 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f767c066f30 0x7f767c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:45.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.987+0000 7f7694ff9700 1 -- 172.21.15.55:0/3506096415 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5968+0+0 (secure 0 0 0) 0x7f768c085690 con 0x7f7698069aa0 2023-10-06T03:49:45.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.988+0000 7f7697fff700 1 --2- 172.21.15.55:0/3506096415 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f767c066f30 0x7f767c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:45.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.988+0000 7f7697fff700 1 --2- 172.21.15.55:0/3506096415 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f767c066f30 0x7f767c0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7684009fd0 tx=0x7f76840091a0 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:45.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:45.993+0000 7f7694ff9700 1 -- 172.21.15.55:0/3506096415 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f769815d4d0 con 0x7f7698069aa0 2023-10-06T03:49:46.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:46.198+0000 7f769ebff700 1 -- 172.21.15.55:0/3506096415 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f769815d4d0 con 0x7f7698069aa0 2023-10-06T03:49:46.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:46.772+0000 7f7694ff9700 1 -- 172.21.15.55:0/3506096415 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v105) v1 ==== 170+0+0 (secure 0 0 0) 0x7f769815d4d0 con 0x7f7698069aa0 2023-10-06T03:49:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:46.786+0000 7f769ebff700 1 -- 172.21.15.55:0/3506096415 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f767c066f30 msgr2=0x7f767c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:46.786+0000 7f769ebff700 1 --2- 172.21.15.55:0/3506096415 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f767c066f30 0x7f767c0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7684009fd0 tx=0x7f76840091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:46.786+0000 7f769ebff700 1 -- 172.21.15.55:0/3506096415 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7698069aa0 msgr2=0x7f76980b1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:46.786+0000 7f769ebff700 1 --2- 172.21.15.55:0/3506096415 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7698069aa0 0x7f76980b1af0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f76980d0760 tx=0x7f768c017000 comp rx=0 tx=0).stop 2023-10-06T03:49:46.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:46.787+0000 7f769ebff700 1 -- 172.21.15.55:0/3506096415 shutdown_connections 2023-10-06T03:49:46.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:46.787+0000 7f769ebff700 1 --2- 172.21.15.55:0/3506096415 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f767c066f30 0x7f767c0693e0 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7684009fd0 tx=0x7f76840091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:46.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:46.787+0000 7f769ebff700 1 --2- 172.21.15.55:0/3506096415 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7698159950 0x7f7698169ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:46.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:46.787+0000 7f769ebff700 1 --2- 172.21.15.55:0/3506096415 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7698069aa0 0x7f76980b1af0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:46.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:46.787+0000 7f769ebff700 1 --2- 172.21.15.55:0/3506096415 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f76980af320 0x7f76980c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:46.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:46.787+0000 7f769ebff700 1 -- 172.21.15.55:0/3506096415 >> 172.21.15.55:0/3506096415 conn(0x7f7698068e00 msgr2=0x7f7698158660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:46.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:46.790+0000 7f769ebff700 1 -- 172.21.15.55:0/3506096415 shutdown_connections 2023-10-06T03:49:46.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:46.790+0000 7f769ebff700 1 -- 172.21.15.55:0/3506096415 wait complete. 2023-10-06T03:49:46.793 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2023-10-06T03:49:46.811 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:49:47.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.001+0000 7fa6b679d700 1 Processor -- start 2023-10-06T03:49:47.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.001+0000 7fa6b679d700 1 -- start start 2023-10-06T03:49:47.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.002+0000 7fa6b679d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6b00c9db0 0x7fa6b006a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:47.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.002+0000 7fa6b679d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6b0157130 0x7fa6b006a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:47.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.002+0000 7fa6b679d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b006e150 con 0x7fa6b0153900 2023-10-06T03:49:47.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.002+0000 7fa6b679d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b00c7610 con 0x7fa6b0157130 2023-10-06T03:49:47.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.002+0000 7fa6b679d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b00c7800 con 0x7fa6b00c9db0 2023-10-06T03:49:47.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.003+0000 7fa6b5f9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6b0157130 0x7fa6b006a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:47.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.003+0000 7fa6b5f9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6b0157130 0x7fa6b006a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55826/0 (socket says 172.21.15.55:55826) 2023-10-06T03:49:47.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.003+0000 7fa6b5f9c700 1 -- 172.21.15.55:0/2426907342 learned_addr learned my addr 172.21.15.55:0/2426907342 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:47.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.003+0000 7fa6b4f9a700 1 --2- 172.21.15.55:0/2426907342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6b00c9db0 0x7fa6b006a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:47.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.003+0000 7fa6b5f9c700 1 -- 172.21.15.55:0/2426907342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6b00c9db0 msgr2=0x7fa6b006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:47.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.003+0000 7fa6b5f9c700 1 --2- 172.21.15.55:0/2426907342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6b00c9db0 0x7fa6b006a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:47.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.003+0000 7fa6b5f9c700 1 -- 172.21.15.55:0/2426907342 >> v1:172.21.15.163:6789/0 conn(0x7fa6b0153900 legacy=0x7fa6b00c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:47.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.003+0000 7fa6b5f9c700 1 -- 172.21.15.55:0/2426907342 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b00c7ae0 con 0x7fa6b0157130 2023-10-06T03:49:47.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.004+0000 7fa6b5f9c700 1 --2- 172.21.15.55:0/2426907342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6b0157130 0x7fa6b006a540 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009f20 tx=0x7fa6a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae95191442f13c0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:47.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.004+0000 7fa6a7fff700 1 -- 172.21.15.55:0/2426907342 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800e070 con 0x7fa6b0157130 2023-10-06T03:49:47.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.004+0000 7fa6a7fff700 1 -- 172.21.15.55:0/2426907342 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a800b9d0 con 0x7fa6b0157130 2023-10-06T03:49:47.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.004+0000 7fa6a7fff700 1 -- 172.21.15.55:0/2426907342 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a80092e0 con 0x7fa6b0157130 2023-10-06T03:49:47.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.004+0000 7fa6b679d700 1 -- 172.21.15.55:0/2426907342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6b0157130 msgr2=0x7fa6b006a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:47.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.004+0000 7fa6b679d700 1 --2- 172.21.15.55:0/2426907342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6b0157130 0x7fa6b006a540 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009f20 tx=0x7fa6a8009840 comp rx=0 tx=0).stop 2023-10-06T03:49:47.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.004+0000 7fa6b679d700 1 -- 172.21.15.55:0/2426907342 shutdown_connections 2023-10-06T03:49:47.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.005+0000 7fa6b679d700 1 --2- 172.21.15.55:0/2426907342 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6b0157130 0x7fa6b006a540 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:47.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.005+0000 7fa6b679d700 1 --2- 172.21.15.55:0/2426907342 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6b00c9db0 0x7fa6b006a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:47.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.005+0000 7fa6b679d700 1 -- 172.21.15.55:0/2426907342 >> 172.21.15.55:0/2426907342 conn(0x7fa6b00cc1e0 msgr2=0x7fa6b00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:47.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.005+0000 7fa6b679d700 1 -- 172.21.15.55:0/2426907342 shutdown_connections 2023-10-06T03:49:47.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.005+0000 7fa6b679d700 1 -- 172.21.15.55:0/2426907342 wait complete. 2023-10-06T03:49:47.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.006+0000 7fa6b679d700 1 Processor -- start 2023-10-06T03:49:47.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.006+0000 7fa6b679d700 1 -- start start 2023-10-06T03:49:47.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.007+0000 7fa6b679d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6b00c9db0 0x7fa6b0169320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:47.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.007+0000 7fa6b679d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6b0153900 0x7fa6b00aa290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:47.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.007+0000 7fa6b679d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6b0157130 0x7fa6b01506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:47.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.007+0000 7fa6b679d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b014ced0 con 0x7fa6b0153900 2023-10-06T03:49:47.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.007+0000 7fa6b679d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b014d770 con 0x7fa6b00c9db0 2023-10-06T03:49:47.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.007+0000 7fa6b679d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b00aab30 con 0x7fa6b0157130 2023-10-06T03:49:47.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.008+0000 7fa6b4f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6b0153900 0x7fa6b00aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:47.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.008+0000 7fa6b4f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6b0153900 0x7fa6b00aa290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55838/0 (socket says 172.21.15.55:55838) 2023-10-06T03:49:47.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.008+0000 7fa6b4f9a700 1 -- 172.21.15.55:0/2119815656 learned_addr learned my addr 172.21.15.55:0/2119815656 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:47.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.008+0000 7fa6b579b700 1 --2- 172.21.15.55:0/2119815656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6b00c9db0 0x7fa6b0169320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:47.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.008+0000 7fa6b5f9c700 1 --2- 172.21.15.55:0/2119815656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6b0157130 0x7fa6b01506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:47.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.008+0000 7fa6b4f9a700 1 -- 172.21.15.55:0/2119815656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6b0157130 msgr2=0x7fa6b01506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:47.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.008+0000 7fa6b4f9a700 1 --2- 172.21.15.55:0/2119815656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6b0157130 0x7fa6b01506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:47.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.008+0000 7fa6b4f9a700 1 -- 172.21.15.55:0/2119815656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6b00c9db0 msgr2=0x7fa6b0169320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:47.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.008+0000 7fa6b4f9a700 1 --2- 172.21.15.55:0/2119815656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6b00c9db0 0x7fa6b0169320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:47.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.008+0000 7fa6b4f9a700 1 -- 172.21.15.55:0/2119815656 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a8009ae0 con 0x7fa6b0153900 2023-10-06T03:49:47.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.008+0000 7fa6b4f9a700 1 --2- 172.21.15.55:0/2119815656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6b0153900 0x7fa6b00aa290 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000ec80 tx=0x7fa6a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:47.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.008+0000 7fa6a57fa700 1 -- 172.21.15.55:0/2119815656 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a000c850 con 0x7fa6b0153900 2023-10-06T03:49:47.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.009+0000 7fa6a57fa700 1 -- 172.21.15.55:0/2119815656 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a000c9b0 con 0x7fa6b0153900 2023-10-06T03:49:47.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.009+0000 7fa6b679d700 1 -- 172.21.15.55:0/2119815656 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b014d120 con 0x7fa6b0153900 2023-10-06T03:49:47.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.009+0000 7fa6a57fa700 1 -- 172.21.15.55:0/2119815656 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a000cd50 con 0x7fa6b0153900 2023-10-06T03:49:47.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.009+0000 7fa6b679d700 1 -- 172.21.15.55:0/2119815656 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b0169860 con 0x7fa6b0153900 2023-10-06T03:49:47.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.010+0000 7fa6b679d700 1 -- 172.21.15.55:0/2119815656 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b006c450 con 0x7fa6b0153900 2023-10-06T03:49:47.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.011+0000 7fa6a57fa700 1 -- 172.21.15.55:0/2119815656 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa6a0013680 con 0x7fa6b0153900 2023-10-06T03:49:47.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.012+0000 7fa6a57fa700 1 --2- 172.21.15.55:0/2119815656 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa694066f30 0x7fa6940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:47.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.012+0000 7fa6a57fa700 1 -- 172.21.15.55:0/2119815656 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6350+0+0 (secure 0 0 0) 0x7fa6a00852b0 con 0x7fa6b0153900 2023-10-06T03:49:47.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.012+0000 7fa6b579b700 1 --2- 172.21.15.55:0/2119815656 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa694066f30 0x7fa6940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:47.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.013+0000 7fa6b579b700 1 --2- 172.21.15.55:0/2119815656 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa694066f30 0x7fa6940693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac003a80 tx=0x7fa6ac007ec0 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:47.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.016+0000 7fa6a57fa700 1 -- 172.21.15.55:0/2119815656 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6a0052280 con 0x7fa6b0153900 2023-10-06T03:49:47.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.263+0000 7fa6b679d700 1 -- 172.21.15.55:0/2119815656 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa6b00c7dc0 con 0x7fa6b0153900 2023-10-06T03:49:47.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.778+0000 7fa6a57fa700 1 -- 172.21.15.55:0/2119815656 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v49) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa6a00558a0 con 0x7fa6b0153900 2023-10-06T03:49:47.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.787+0000 7fa6b679d700 1 -- 172.21.15.55:0/2119815656 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa694066f30 msgr2=0x7fa6940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:47.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.787+0000 7fa6b679d700 1 --2- 172.21.15.55:0/2119815656 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa694066f30 0x7fa6940693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac003a80 tx=0x7fa6ac007ec0 comp rx=0 tx=0).stop 2023-10-06T03:49:47.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.787+0000 7fa6b679d700 1 -- 172.21.15.55:0/2119815656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6b0153900 msgr2=0x7fa6b00aa290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:47.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.788+0000 7fa6b679d700 1 --2- 172.21.15.55:0/2119815656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6b0153900 0x7fa6b00aa290 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000ec80 tx=0x7fa6a00091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:47.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.790+0000 7fa6b679d700 1 -- 172.21.15.55:0/2119815656 shutdown_connections 2023-10-06T03:49:47.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.790+0000 7fa6b679d700 1 --2- 172.21.15.55:0/2119815656 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa694066f30 0x7fa6940693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:47.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.790+0000 7fa6b679d700 1 --2- 172.21.15.55:0/2119815656 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6b0157130 0x7fa6b01506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:47.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.790+0000 7fa6b679d700 1 --2- 172.21.15.55:0/2119815656 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6b00c9db0 0x7fa6b0169320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:47.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.790+0000 7fa6b679d700 1 --2- 172.21.15.55:0/2119815656 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6b0153900 0x7fa6b00aa290 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:47.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.790+0000 7fa6b679d700 1 -- 172.21.15.55:0/2119815656 >> 172.21.15.55:0/2119815656 conn(0x7fa6b00cc1e0 msgr2=0x7fa6b00c2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:47.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.790+0000 7fa6b679d700 1 -- 172.21.15.55:0/2119815656 shutdown_connections 2023-10-06T03:49:47.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:47.790+0000 7fa6b679d700 1 -- 172.21.15.55:0/2119815656 wait complete. 2023-10-06T03:49:47.795 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 13 and data pool 14 2023-10-06T03:49:47.819 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:49:48.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.005+0000 7f050259d700 1 Processor -- start 2023-10-06T03:49:48.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.005+0000 7f050259d700 1 -- start start 2023-10-06T03:49:48.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.006+0000 7f050259d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f04fc159420 0x7f04fc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.006+0000 7f050259d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04fc0b2980 con 0x7f04fc1597f0 2023-10-06T03:49:48.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.006+0000 7f050259d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04fc0b11d0 con 0x7f04fc15d230 2023-10-06T03:49:48.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.006+0000 7f050259d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04fc0b14b0 con 0x7f04fc159420 2023-10-06T03:49:48.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.007+0000 7f050159b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f04fc159420 0x7f04fc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.007+0000 7f050159b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f04fc159420 0x7f04fc0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55854/0 (socket says 172.21.15.55:55854) 2023-10-06T03:49:48.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.007+0000 7f050159b700 1 -- 172.21.15.55:0/1011776640 learned_addr learned my addr 172.21.15.55:0/1011776640 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:48.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.007+0000 7f050159b700 1 -- 172.21.15.55:0/1011776640 >> v1:172.21.15.163:6789/0 conn(0x7f04fc15d230 legacy=0x7f04fc15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.007+0000 7f050159b700 1 -- 172.21.15.55:0/1011776640 >> v1:172.21.15.55:6789/0 conn(0x7f04fc1597f0 legacy=0x7f04fc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.007+0000 7f050159b700 1 -- 172.21.15.55:0/1011776640 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc0b1790 con 0x7f04fc159420 2023-10-06T03:49:48.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.007+0000 7f050159b700 1 --2- 172.21.15.55:0/1011776640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04fc159420 0x7f04fc0b5f40 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f04f80099b0 tx=0x7f04f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c7d44bb96a0e2a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:48.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.008+0000 7f04f3fff700 1 -- 172.21.15.55:0/1011776640 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f800b860 con 0x7f04fc159420 2023-10-06T03:49:48.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.008+0000 7f04f3fff700 1 -- 172.21.15.55:0/1011776640 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f800b9c0 con 0x7f04fc159420 2023-10-06T03:49:48.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.008+0000 7f04f3fff700 1 -- 172.21.15.55:0/1011776640 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f800bd60 con 0x7f04fc159420 2023-10-06T03:49:48.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.008+0000 7f050259d700 1 -- 172.21.15.55:0/1011776640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04fc159420 msgr2=0x7f04fc0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.008+0000 7f050259d700 1 --2- 172.21.15.55:0/1011776640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04fc159420 0x7f04fc0b5f40 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f04f80099b0 tx=0x7f04f8009250 comp rx=0 tx=0).stop 2023-10-06T03:49:48.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.008+0000 7f050259d700 1 -- 172.21.15.55:0/1011776640 shutdown_connections 2023-10-06T03:49:48.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.008+0000 7f050259d700 1 --2- 172.21.15.55:0/1011776640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04fc159420 0x7f04fc0b5f40 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.008+0000 7f050259d700 1 -- 172.21.15.55:0/1011776640 >> 172.21.15.55:0/1011776640 conn(0x7f04fc0d0660 msgr2=0x7f04fc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:48.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.008+0000 7f050259d700 1 -- 172.21.15.55:0/1011776640 shutdown_connections 2023-10-06T03:49:48.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.008+0000 7f050259d700 1 -- 172.21.15.55:0/1011776640 wait complete. 2023-10-06T03:49:48.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.010+0000 7f050259d700 1 Processor -- start 2023-10-06T03:49:48.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.010+0000 7f050259d700 1 -- start start 2023-10-06T03:49:48.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.011+0000 7f050259d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04fc159420 0x7f04fc157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.011+0000 7f050259d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04fc1597f0 0x7f04fc0ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.011+0000 7f050259d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04fc15d230 0x7f04fc069760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.011+0000 7f050259d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc0cb650 con 0x7f04fc159420 2023-10-06T03:49:48.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.011+0000 7f050259d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc0caa40 con 0x7f04fc1597f0 2023-10-06T03:49:48.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.011+0000 7f050259d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc0cad20 con 0x7f04fc15d230 2023-10-06T03:49:48.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.011+0000 7f050159b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04fc159420 0x7f04fc157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.011+0000 7f050159b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04fc159420 0x7f04fc157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55866/0 (socket says 172.21.15.55:55866) 2023-10-06T03:49:48.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.011+0000 7f050159b700 1 -- 172.21.15.55:0/598249298 learned_addr learned my addr 172.21.15.55:0/598249298 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:48.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.011+0000 7f0500d9a700 1 --2- 172.21.15.55:0/598249298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04fc1597f0 0x7f04fc0ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.011+0000 7f0501d9c700 1 --2- 172.21.15.55:0/598249298 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04fc15d230 0x7f04fc069760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.012+0000 7f050159b700 1 -- 172.21.15.55:0/598249298 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04fc15d230 msgr2=0x7f04fc069760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.012+0000 7f050159b700 1 --2- 172.21.15.55:0/598249298 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04fc15d230 0x7f04fc069760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.012+0000 7f050159b700 1 -- 172.21.15.55:0/598249298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04fc1597f0 msgr2=0x7f04fc0ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.012+0000 7f050159b700 1 --2- 172.21.15.55:0/598249298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04fc1597f0 0x7f04fc0ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.012+0000 7f050159b700 1 -- 172.21.15.55:0/598249298 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f80094f0 con 0x7f04fc159420 2023-10-06T03:49:48.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.012+0000 7f050159b700 1 --2- 172.21.15.55:0/598249298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04fc159420 0x7f04fc157350 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f04fc089340 tx=0x7f04f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:48.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.012+0000 7f04f17fa700 1 -- 172.21.15.55:0/598249298 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f8024440 con 0x7f04fc159420 2023-10-06T03:49:48.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.012+0000 7f04f17fa700 1 -- 172.21.15.55:0/598249298 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04f80245a0 con 0x7f04fc159420 2023-10-06T03:49:48.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.012+0000 7f050259d700 1 -- 172.21.15.55:0/598249298 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04fc0cb000 con 0x7f04fc159420 2023-10-06T03:49:48.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.012+0000 7f04f17fa700 1 -- 172.21.15.55:0/598249298 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04f8024940 con 0x7f04fc159420 2023-10-06T03:49:48.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.013+0000 7f050259d700 1 -- 172.21.15.55:0/598249298 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04fc0ca6c0 con 0x7f04fc159420 2023-10-06T03:49:48.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.014+0000 7f050259d700 1 -- 172.21.15.55:0/598249298 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04fc04c3c0 con 0x7f04fc159420 2023-10-06T03:49:48.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.015+0000 7f04f17fa700 1 -- 172.21.15.55:0/598249298 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f04f801a420 con 0x7f04fc159420 2023-10-06T03:49:48.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.015+0000 7f04f17fa700 1 --2- 172.21.15.55:0/598249298 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f04e0066f30 0x7f04e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.015+0000 7f04f17fa700 1 -- 172.21.15.55:0/598249298 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f04f8086710 con 0x7f04fc159420 2023-10-06T03:49:48.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.016+0000 7f0500d9a700 1 --2- 172.21.15.55:0/598249298 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f04e0066f30 0x7f04e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.016+0000 7f0500d9a700 1 --2- 172.21.15.55:0/598249298 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f04e0066f30 0x7f04e00693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0c7ea0 tx=0x7f04ec007ec0 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:48.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.020+0000 7f04f17fa700 1 -- 172.21.15.55:0/598249298 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04f80536a0 con 0x7f04fc159420 2023-10-06T03:49:48.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.222+0000 7f050259d700 1 -- 172.21.15.55:0/598249298 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04fc0b1a70 con 0x7f04fc159420 2023-10-06T03:49:48.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.223+0000 7f04f17fa700 1 -- 172.21.15.55:0/598249298 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+17536 (secure 0 0 0) 0x7f04f8056cc0 con 0x7f04fc159420 2023-10-06T03:49:48.225 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:48.225 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":106,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:49:47.264684+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:49:45.423436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T03:49:46.240142+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:49:48.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.225+0000 7f050259d700 1 -- 172.21.15.55:0/598249298 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f04e0066f30 msgr2=0x7f04e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.225+0000 7f050259d700 1 --2- 172.21.15.55:0/598249298 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f04e0066f30 0x7f04e00693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0c7ea0 tx=0x7f04ec007ec0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.225+0000 7f050259d700 1 -- 172.21.15.55:0/598249298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04fc159420 msgr2=0x7f04fc157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.225+0000 7f050259d700 1 --2- 172.21.15.55:0/598249298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04fc159420 0x7f04fc157350 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f04fc089340 tx=0x7f04f8017000 comp rx=0 tx=0).stop 2023-10-06T03:49:48.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.225+0000 7f050259d700 1 -- 172.21.15.55:0/598249298 shutdown_connections 2023-10-06T03:49:48.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.225+0000 7f050259d700 1 --2- 172.21.15.55:0/598249298 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f04e0066f30 0x7f04e00693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.225+0000 7f050259d700 1 --2- 172.21.15.55:0/598249298 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04fc15d230 0x7f04fc069760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.225+0000 7f050259d700 1 --2- 172.21.15.55:0/598249298 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04fc159420 0x7f04fc157350 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.225+0000 7f050259d700 1 --2- 172.21.15.55:0/598249298 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04fc1597f0 0x7f04fc0ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.225+0000 7f050259d700 1 -- 172.21.15.55:0/598249298 >> 172.21.15.55:0/598249298 conn(0x7f04fc0d0660 msgr2=0x7f04fc0b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:48.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.225+0000 7f050259d700 1 -- 172.21.15.55:0/598249298 shutdown_connections 2023-10-06T03:49:48.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.226+0000 7f050259d700 1 -- 172.21.15.55:0/598249298 wait complete. 2023-10-06T03:49:48.245 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:49:48.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.431+0000 7ff63d19d700 1 Processor -- start 2023-10-06T03:49:48.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.431+0000 7ff63d19d700 1 -- start start 2023-10-06T03:49:48.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.432+0000 7ff63d19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff638159420 0x7ff6380b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.432+0000 7ff63d19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff6381597f0 0x7ff6380b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.432+0000 7ff63d19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6380b11d0 con 0x7ff63815d230 2023-10-06T03:49:48.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.432+0000 7ff63d19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6380b14b0 con 0x7ff638159420 2023-10-06T03:49:48.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.432+0000 7ff63d19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6380b1790 con 0x7ff6381597f0 2023-10-06T03:49:48.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.433+0000 7ff637fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff638159420 0x7ff6380b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.433+0000 7ff637fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff638159420 0x7ff6380b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55870/0 (socket says 172.21.15.55:55870) 2023-10-06T03:49:48.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.433+0000 7ff637fff700 1 -- 172.21.15.55:0/3889245728 learned_addr learned my addr 172.21.15.55:0/3889245728 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:48.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.433+0000 7ff6377fe700 1 --2- 172.21.15.55:0/3889245728 >> v2:172.21.15.163:3300/0 conn(0x7ff6381597f0 0x7ff6380b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.433+0000 7ff637fff700 1 -- 172.21.15.55:0/3889245728 >> v2:172.21.15.163:3300/0 conn(0x7ff6381597f0 msgr2=0x7ff6380b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:48.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.433+0000 7ff637fff700 1 --2- 172.21.15.55:0/3889245728 >> v2:172.21.15.163:3300/0 conn(0x7ff6381597f0 0x7ff6380b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.433+0000 7ff637fff700 1 -- 172.21.15.55:0/3889245728 >> v1:172.21.15.163:6789/0 conn(0x7ff63815d230 legacy=0x7ff63815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:48.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.433+0000 7ff637fff700 1 -- 172.21.15.55:0/3889245728 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6380b1a70 con 0x7ff638159420 2023-10-06T03:49:48.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.433+0000 7ff6377fe700 1 --2- 172.21.15.55:0/3889245728 >> v2:172.21.15.163:3300/0 conn(0x7ff6381597f0 0x7ff6380b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:48.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.434+0000 7ff637fff700 1 --2- 172.21.15.55:0/3889245728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff638159420 0x7ff6380b5f40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff638089750 tx=0x7ff628009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c9d9f4eee4c780 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:48.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.434+0000 7ff636ffd700 1 -- 172.21.15.55:0/3889245728 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62800b860 con 0x7ff638159420 2023-10-06T03:49:48.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.434+0000 7ff636ffd700 1 -- 172.21.15.55:0/3889245728 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62800b9c0 con 0x7ff638159420 2023-10-06T03:49:48.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.434+0000 7ff636ffd700 1 -- 172.21.15.55:0/3889245728 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62800bd60 con 0x7ff638159420 2023-10-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.434+0000 7ff63d19d700 1 -- 172.21.15.55:0/3889245728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff638159420 msgr2=0x7ff6380b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.434+0000 7ff63d19d700 1 --2- 172.21.15.55:0/3889245728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff638159420 0x7ff6380b5f40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff638089750 tx=0x7ff628009250 comp rx=0 tx=0).stop 2023-10-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.435+0000 7ff63d19d700 1 -- 172.21.15.55:0/3889245728 shutdown_connections 2023-10-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.435+0000 7ff63d19d700 1 --2- 172.21.15.55:0/3889245728 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff638159420 0x7ff6380b5f40 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.435+0000 7ff63d19d700 1 --2- 172.21.15.55:0/3889245728 >> v2:172.21.15.163:3300/0 conn(0x7ff6381597f0 0x7ff6380b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.435+0000 7ff63d19d700 1 -- 172.21.15.55:0/3889245728 >> 172.21.15.55:0/3889245728 conn(0x7ff6380d0660 msgr2=0x7ff6380d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.435+0000 7ff63d19d700 1 -- 172.21.15.55:0/3889245728 shutdown_connections 2023-10-06T03:49:48.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.435+0000 7ff63d19d700 1 -- 172.21.15.55:0/3889245728 wait complete. 2023-10-06T03:49:48.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.436+0000 7ff63d19d700 1 Processor -- start 2023-10-06T03:49:48.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.436+0000 7ff63d19d700 1 -- start start 2023-10-06T03:49:48.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.437+0000 7ff63d19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff638159420 0x7ff638149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.437+0000 7ff63d19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6381597f0 0x7ff6380c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.437+0000 7ff63d19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff63815d230 0x7ff638068ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.437+0000 7ff63d19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6380c67d0 con 0x7ff6381597f0 2023-10-06T03:49:48.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.437+0000 7ff63d19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6380c5940 con 0x7ff63815d230 2023-10-06T03:49:48.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.437+0000 7ff63d19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6380c7780 con 0x7ff638159420 2023-10-06T03:49:48.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.437+0000 7ff6377fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6381597f0 0x7ff6380c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.437+0000 7ff6377fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6381597f0 0x7ff6380c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55872/0 (socket says 172.21.15.55:55872) 2023-10-06T03:49:48.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.437+0000 7ff6377fe700 1 -- 172.21.15.55:0/2291400982 learned_addr learned my addr 172.21.15.55:0/2291400982 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:48.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff63c99c700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff63815d230 0x7ff638068ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff6377fe700 1 -- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff638159420 msgr2=0x7ff638149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff637fff700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff638159420 0x7ff638149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff6377fe700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff638159420 0x7ff638149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff6377fe700 1 -- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff63815d230 msgr2=0x7ff638068ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff6377fe700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff63815d230 0x7ff638068ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff6377fe700 1 -- 172.21.15.55:0/2291400982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280094f0 con 0x7ff6381597f0 2023-10-06T03:49:48.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff63c99c700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff63815d230 0x7ff638068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:48.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff637fff700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff638159420 0x7ff638149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:48.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff6377fe700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6381597f0 0x7ff6380c5400 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00e8a0 tx=0x7ff62c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:48.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff61ffff700 1 -- 172.21.15.55:0/2291400982 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c00c430 con 0x7ff6381597f0 2023-10-06T03:49:48.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff61ffff700 1 -- 172.21.15.55:0/2291400982 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62c00c590 con 0x7ff6381597f0 2023-10-06T03:49:48.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff63d19d700 1 -- 172.21.15.55:0/2291400982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6380c5b50 con 0x7ff6381597f0 2023-10-06T03:49:48.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff61ffff700 1 -- 172.21.15.55:0/2291400982 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c00c950 con 0x7ff6381597f0 2023-10-06T03:49:48.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.438+0000 7ff63d19d700 1 -- 172.21.15.55:0/2291400982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6380c6110 con 0x7ff6381597f0 2023-10-06T03:49:48.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.439+0000 7ff63d19d700 1 -- 172.21.15.55:0/2291400982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63804c3c0 con 0x7ff6381597f0 2023-10-06T03:49:48.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.440+0000 7ff61ffff700 1 -- 172.21.15.55:0/2291400982 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff62c00ccc0 con 0x7ff6381597f0 2023-10-06T03:49:48.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.441+0000 7ff61ffff700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff624066f30 0x7ff6240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.441+0000 7ff61ffff700 1 -- 172.21.15.55:0/2291400982 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6450+0+0 (secure 0 0 0) 0x7ff62c084640 con 0x7ff6381597f0 2023-10-06T03:49:48.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.442+0000 7ff637fff700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff624066f30 0x7ff6240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.443+0000 7ff637fff700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff624066f30 0x7ff6240693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff638089750 tx=0x7ff628017000 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:48.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.445+0000 7ff61ffff700 1 -- 172.21.15.55:0/2291400982 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff62c054fb0 con 0x7ff6381597f0 2023-10-06T03:49:48.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.648+0000 7ff63d19d700 1 -- 172.21.15.55:0/2291400982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6380b1d50 con 0x7ff6381597f0 2023-10-06T03:49:48.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.649+0000 7ff61ffff700 1 -- 172.21.15.55:0/2291400982 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+17536 (secure 0 0 0) 0x7ff62c05a070 con 0x7ff6381597f0 2023-10-06T03:49:48.651 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:48.651 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":106,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:49:47.264684+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:49:45.423436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T03:49:46.240142+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:49:48.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.652+0000 7ff63d19d700 1 -- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff624066f30 msgr2=0x7ff6240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.652+0000 7ff63d19d700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff624066f30 0x7ff6240693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff638089750 tx=0x7ff628017000 comp rx=0 tx=0).stop 2023-10-06T03:49:48.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.652+0000 7ff63d19d700 1 -- 172.21.15.55:0/2291400982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6381597f0 msgr2=0x7ff6380c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.652+0000 7ff63d19d700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6381597f0 0x7ff6380c5400 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00e8a0 tx=0x7ff62c008040 comp rx=0 tx=0).stop 2023-10-06T03:49:48.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.652+0000 7ff63d19d700 1 -- 172.21.15.55:0/2291400982 shutdown_connections 2023-10-06T03:49:48.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.652+0000 7ff63d19d700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff624066f30 0x7ff6240693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.652+0000 7ff63d19d700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff63815d230 0x7ff638068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.652+0000 7ff63d19d700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff638159420 0x7ff638149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.652+0000 7ff63d19d700 1 --2- 172.21.15.55:0/2291400982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6381597f0 0x7ff6380c5400 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.652+0000 7ff63d19d700 1 -- 172.21.15.55:0/2291400982 >> 172.21.15.55:0/2291400982 conn(0x7ff6380d0660 msgr2=0x7ff638069c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:48.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.652+0000 7ff63d19d700 1 -- 172.21.15.55:0/2291400982 shutdown_connections 2023-10-06T03:49:48.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.652+0000 7ff63d19d700 1 -- 172.21.15.55:0/2291400982 wait complete. 2023-10-06T03:49:48.672 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:49:48.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.862+0000 7f7a7539d700 1 Processor -- start 2023-10-06T03:49:48.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.862+0000 7f7a7539d700 1 -- start start 2023-10-06T03:49:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.864+0000 7f7a7539d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7a7006f060 0x7f7a700ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.864+0000 7f7a7539d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a7006f4b0 0x7f7a700ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.864+0000 7f7a7539d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a7006ebd0 con 0x7f7a700cb1a0 2023-10-06T03:49:48.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.864+0000 7f7a7539d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a7015b2e0 con 0x7f7a7006f060 2023-10-06T03:49:48.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.864+0000 7f7a7539d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7015aa70 con 0x7f7a7006f4b0 2023-10-06T03:49:48.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.864+0000 7f7a6ffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7a700cb1a0 0x7f7a7015a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51884/0 (socket says 172.21.15.55:51884) 2023-10-06T03:49:48.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.864+0000 7f7a6ffff700 1 -- 172.21.15.55:0/3578269836 learned_addr learned my addr 172.21.15.55:0/3578269836 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:48.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.864+0000 7f7a6f7fe700 1 --2- 172.21.15.55:0/3578269836 >> v2:172.21.15.163:3300/0 conn(0x7f7a7006f060 0x7f7a700ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.864+0000 7f7a74b9c700 1 --2- 172.21.15.55:0/3578269836 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a7006f4b0 0x7f7a700ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.864+0000 7f7a6effd700 1 -- 172.21.15.55:0/3578269836 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477011823 0 0) 0x7f7a7006ebd0 con 0x7f7a700cb1a0 2023-10-06T03:49:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.864+0000 7f7a6effd700 1 -- 172.21.15.55:0/3578269836 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a58003570 con 0x7f7a700cb1a0 2023-10-06T03:49:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.865+0000 7f7a6effd700 1 -- 172.21.15.55:0/3578269836 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2602226093 0 0) 0x7f7a58003570 con 0x7f7a700cb1a0 2023-10-06T03:49:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.865+0000 7f7a6f7fe700 1 -- 172.21.15.55:0/3578269836 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a7006f4b0 msgr2=0x7f7a700ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.865+0000 7f7a6f7fe700 1 --2- 172.21.15.55:0/3578269836 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a7006f4b0 0x7f7a700ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.865+0000 7f7a6f7fe700 1 -- 172.21.15.55:0/3578269836 >> v1:172.21.15.55:6789/0 conn(0x7f7a700cb1a0 legacy=0x7f7a7015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.865+0000 7f7a6f7fe700 1 -- 172.21.15.55:0/3578269836 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7015ad20 con 0x7f7a7006f060 2023-10-06T03:49:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.865+0000 7f7a74b9c700 1 --2- 172.21.15.55:0/3578269836 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a7006f4b0 0x7f7a700ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.865+0000 7f7a6f7fe700 1 --2- 172.21.15.55:0/3578269836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a7006f060 0x7f7a700ae0d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7a640098e0 tx=0x7f7a640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65d5990dbd11d71a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.865+0000 7f7a6effd700 1 -- 172.21.15.55:0/3578269836 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60001470 con 0x7f7a7006f060 2023-10-06T03:49:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.865+0000 7f7a6effd700 1 -- 172.21.15.55:0/3578269836 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6400f4a0 con 0x7f7a7006f060 2023-10-06T03:49:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.865+0000 7f7a6effd700 1 -- 172.21.15.55:0/3578269836 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6400f840 con 0x7f7a7006f060 2023-10-06T03:49:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.866+0000 7f7a7539d700 1 -- 172.21.15.55:0/3578269836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a7006f060 msgr2=0x7f7a700ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.866+0000 7f7a7539d700 1 --2- 172.21.15.55:0/3578269836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a7006f060 0x7f7a700ae0d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f7a640098e0 tx=0x7f7a640091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.866+0000 7f7a7539d700 1 -- 172.21.15.55:0/3578269836 shutdown_connections 2023-10-06T03:49:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.866+0000 7f7a7539d700 1 --2- 172.21.15.55:0/3578269836 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a7006f4b0 0x7f7a700ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.866+0000 7f7a7539d700 1 --2- 172.21.15.55:0/3578269836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a7006f060 0x7f7a700ae0d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.866+0000 7f7a7539d700 1 -- 172.21.15.55:0/3578269836 >> 172.21.15.55:0/3578269836 conn(0x7f7a700cc1e0 msgr2=0x7f7a700cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:48.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.866+0000 7f7a7539d700 1 -- 172.21.15.55:0/3578269836 shutdown_connections 2023-10-06T03:49:48.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.866+0000 7f7a7539d700 1 -- 172.21.15.55:0/3578269836 wait complete. 2023-10-06T03:49:48.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.867+0000 7f7a7539d700 1 Processor -- start 2023-10-06T03:49:48.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.868+0000 7f7a7539d700 1 -- start start 2023-10-06T03:49:48.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.868+0000 7f7a7539d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a7006f060 0x7f7a70150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.868+0000 7f7a7539d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a7006f4b0 0x7f7a7014fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.868+0000 7f7a7539d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a700cb1a0 0x7f7a700c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.868+0000 7f7a7539d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a70153440 con 0x7f7a7006f4b0 2023-10-06T03:49:48.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.868+0000 7f7a7539d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a70150150 con 0x7f7a700cb1a0 2023-10-06T03:49:48.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.868+0000 7f7a7539d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700b1750 con 0x7f7a7006f060 2023-10-06T03:49:48.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.869+0000 7f7a6f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a7006f4b0 0x7f7a7014fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.869+0000 7f7a6f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a7006f4b0 0x7f7a7014fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55874/0 (socket says 172.21.15.55:55874) 2023-10-06T03:49:48.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.869+0000 7f7a6f7fe700 1 -- 172.21.15.55:0/2142048108 learned_addr learned my addr 172.21.15.55:0/2142048108 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:48.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.869+0000 7f7a6ffff700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a7006f060 0x7f7a70150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.869+0000 7f7a74b9c700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a700cb1a0 0x7f7a700c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.869+0000 7f7a6f7fe700 1 -- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a7006f060 msgr2=0x7f7a70150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.869+0000 7f7a6f7fe700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a7006f060 0x7f7a70150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.869+0000 7f7a6f7fe700 1 -- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a700cb1a0 msgr2=0x7f7a700c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:48.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.869+0000 7f7a6f7fe700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a700cb1a0 0x7f7a700c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:48.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.869+0000 7f7a6f7fe700 1 -- 172.21.15.55:0/2142048108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a68009a20 con 0x7f7a7006f4b0 2023-10-06T03:49:48.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.869+0000 7f7a6ffff700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a7006f060 0x7f7a70150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:48.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.869+0000 7f7a74b9c700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a700cb1a0 0x7f7a700c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:48.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.869+0000 7f7a6f7fe700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a7006f4b0 0x7f7a7014fc10 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7a640093f0 tx=0x7f7a64018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:48.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.870+0000 7f7a57fff700 1 -- 172.21.15.55:0/2142048108 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6401e070 con 0x7f7a7006f4b0 2023-10-06T03:49:48.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.870+0000 7f7a57fff700 1 -- 172.21.15.55:0/2142048108 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a64000c90 con 0x7f7a7006f4b0 2023-10-06T03:49:48.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.870+0000 7f7a57fff700 1 -- 172.21.15.55:0/2142048108 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64023460 con 0x7f7a7006f4b0 2023-10-06T03:49:48.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.870+0000 7f7a7539d700 1 -- 172.21.15.55:0/2142048108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a64009420 con 0x7f7a7006f4b0 2023-10-06T03:49:48.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.870+0000 7f7a7539d700 1 -- 172.21.15.55:0/2142048108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a700b1d10 con 0x7f7a7006f4b0 2023-10-06T03:49:48.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.871+0000 7f7a7539d700 1 -- 172.21.15.55:0/2142048108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a701535f0 con 0x7f7a7006f4b0 2023-10-06T03:49:48.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.872+0000 7f7a57fff700 1 -- 172.21.15.55:0/2142048108 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a64018420 con 0x7f7a7006f4b0 2023-10-06T03:49:48.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.872+0000 7f7a57fff700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a5c066f30 0x7f7a5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:48.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.872+0000 7f7a57fff700 1 -- 172.21.15.55:0/2142048108 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f7a64014070 con 0x7f7a7006f4b0 2023-10-06T03:49:48.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.873+0000 7f7a6ffff700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a5c066f30 0x7f7a5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:48.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.874+0000 7f7a6ffff700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a5c066f30 0x7f7a5c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7a600011e0 tx=0x7f7a600018a0 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:48.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:48.877+0000 7f7a57fff700 1 -- 172.21.15.55:0/2142048108 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a701535f0 con 0x7f7a7006f4b0 2023-10-06T03:49:49.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:49.126+0000 7f7a7539d700 1 -- 172.21.15.55:0/2142048108 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7a70153580 con 0x7f7a7006f4b0 2023-10-06T03:49:49.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:49.792+0000 7f7a57fff700 1 -- 172.21.15.55:0/2142048108 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7a64052660 con 0x7f7a7006f4b0 2023-10-06T03:49:49.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:49.795+0000 7f7a7539d700 1 -- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a5c066f30 msgr2=0x7f7a5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:49.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:49.795+0000 7f7a7539d700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a5c066f30 0x7f7a5c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7a600011e0 tx=0x7f7a600018a0 comp rx=0 tx=0).stop 2023-10-06T03:49:49.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:49.795+0000 7f7a7539d700 1 -- 172.21.15.55:0/2142048108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a7006f4b0 msgr2=0x7f7a7014fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:49.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:49.795+0000 7f7a7539d700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a7006f4b0 0x7f7a7014fc10 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f7a640093f0 tx=0x7f7a64018000 comp rx=0 tx=0).stop 2023-10-06T03:49:49.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:49.795+0000 7f7a7539d700 1 -- 172.21.15.55:0/2142048108 shutdown_connections 2023-10-06T03:49:49.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:49.795+0000 7f7a7539d700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a5c066f30 0x7f7a5c0693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:49.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:49.795+0000 7f7a7539d700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a700cb1a0 0x7f7a700c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:49.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:49.795+0000 7f7a7539d700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a7006f060 0x7f7a70150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:49.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:49.795+0000 7f7a7539d700 1 --2- 172.21.15.55:0/2142048108 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a7006f4b0 0x7f7a7014fc10 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:49.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:49.795+0000 7f7a7539d700 1 -- 172.21.15.55:0/2142048108 >> 172.21.15.55:0/2142048108 conn(0x7f7a700cc1e0 msgr2=0x7f7a700c1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:49.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:49.795+0000 7f7a7539d700 1 -- 172.21.15.55:0/2142048108 shutdown_connections 2023-10-06T03:49:49.798 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:49.796+0000 7f7a7539d700 1 -- 172.21.15.55:0/2142048108 wait complete. 2023-10-06T03:49:49.816 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:49:50.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.009+0000 7f2b1bfff700 1 Processor -- start 2023-10-06T03:49:50.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.009+0000 7f2b1bfff700 1 -- start start 2023-10-06T03:49:50.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.011+0000 7f2b1bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2b1c1407e0 0x7f2b1c06a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:50.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.011+0000 7f2b1bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b1c140bb0 0x7f2b1c06a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:50.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.011+0000 7f2b1bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1c0c6820 con 0x7f2b1c06d4f0 2023-10-06T03:49:50.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.011+0000 7f2b1bfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b1c0c6de0 con 0x7f2b1c1407e0 2023-10-06T03:49:50.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.011+0000 7f2b1bfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b1c141270 con 0x7f2b1c140bb0 2023-10-06T03:49:50.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.011+0000 7f2b1affd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2b1c1407e0 0x7f2b1c06a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:50.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.011+0000 7f2b1affd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2b1c1407e0 0x7f2b1c06a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55890/0 (socket says 172.21.15.55:55890) 2023-10-06T03:49:50.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.011+0000 7f2b1b7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2b1c06d4f0 0x7f2b1c06d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51890/0 (socket says 172.21.15.55:51890) 2023-10-06T03:49:50.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.011+0000 7f2b1affd700 1 -- 172.21.15.55:0/1254737284 learned_addr learned my addr 172.21.15.55:0/1254737284 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:50.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.011+0000 7f2b1a7fc700 1 --2- 172.21.15.55:0/1254737284 >> v2:172.21.15.163:3300/0 conn(0x7f2b1c140bb0 0x7f2b1c06a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:50.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.011+0000 7f2b19ffb700 1 -- 172.21.15.55:0/1254737284 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648969001 0 0) 0x7f2b1c0c6820 con 0x7f2b1c06d4f0 2023-10-06T03:49:50.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.011+0000 7f2b19ffb700 1 -- 172.21.15.55:0/1254737284 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b04003570 con 0x7f2b1c06d4f0 2023-10-06T03:49:50.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.012+0000 7f2b1a7fc700 1 -- 172.21.15.55:0/1254737284 >> v2:172.21.15.55:3300/0 conn(0x7f2b1c1407e0 msgr2=0x7f2b1c06a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:50.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.012+0000 7f2b1a7fc700 1 --2- 172.21.15.55:0/1254737284 >> v2:172.21.15.55:3300/0 conn(0x7f2b1c1407e0 0x7f2b1c06a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:50.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.012+0000 7f2b1a7fc700 1 -- 172.21.15.55:0/1254737284 >> v1:172.21.15.55:6789/0 conn(0x7f2b1c06d4f0 legacy=0x7f2b1c06d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:50.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.012+0000 7f2b1a7fc700 1 -- 172.21.15.55:0/1254737284 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c06acf0 con 0x7f2b1c140bb0 2023-10-06T03:49:50.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.012+0000 7f2b1a7fc700 1 --2- 172.21.15.55:0/1254737284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b1c140bb0 0x7f2b1c06a6c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c0c3210 tx=0x7f2b100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4dc026cdcd94137 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:50.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.012+0000 7f2b19ffb700 1 -- 172.21.15.55:0/1254737284 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10011630 con 0x7f2b1c140bb0 2023-10-06T03:49:50.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.012+0000 7f2b19ffb700 1 -- 172.21.15.55:0/1254737284 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b100117b0 con 0x7f2b1c140bb0 2023-10-06T03:49:50.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.012+0000 7f2b19ffb700 1 -- 172.21.15.55:0/1254737284 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10011b70 con 0x7f2b1c140bb0 2023-10-06T03:49:50.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.013+0000 7f2b1bfff700 1 -- 172.21.15.55:0/1254737284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b1c140bb0 msgr2=0x7f2b1c06a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:50.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.013+0000 7f2b1bfff700 1 --2- 172.21.15.55:0/1254737284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b1c140bb0 0x7f2b1c06a6c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c0c3210 tx=0x7f2b100091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:50.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.013+0000 7f2b1bfff700 1 -- 172.21.15.55:0/1254737284 shutdown_connections 2023-10-06T03:49:50.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.013+0000 7f2b1bfff700 1 --2- 172.21.15.55:0/1254737284 >> v2:172.21.15.55:3300/0 conn(0x7f2b1c1407e0 0x7f2b1c06a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:50.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.013+0000 7f2b1bfff700 1 --2- 172.21.15.55:0/1254737284 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b1c140bb0 0x7f2b1c06a6c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:50.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.013+0000 7f2b1bfff700 1 -- 172.21.15.55:0/1254737284 >> 172.21.15.55:0/1254737284 conn(0x7f2b1c0cc1e0 msgr2=0x7f2b1c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:50.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.013+0000 7f2b1bfff700 1 -- 172.21.15.55:0/1254737284 shutdown_connections 2023-10-06T03:49:50.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.013+0000 7f2b1bfff700 1 -- 172.21.15.55:0/1254737284 wait complete. 2023-10-06T03:49:50.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.014+0000 7f2b1bfff700 1 Processor -- start 2023-10-06T03:49:50.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.014+0000 7f2b1bfff700 1 -- start start 2023-10-06T03:49:50.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.015+0000 7f2b1bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b1c06d4f0 0x7f2b1c0c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:50.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.015+0000 7f2b1bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b1c1407e0 0x7f2b1c15cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:50.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.015+0000 7f2b1bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b1c140bb0 0x7f2b1c168630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:50.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.015+0000 7f2b1bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c15d2d0 con 0x7f2b1c140bb0 2023-10-06T03:49:50.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.015+0000 7f2b1bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c0c5570 con 0x7f2b1c1407e0 2023-10-06T03:49:50.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.015+0000 7f2b1bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c168b70 con 0x7f2b1c06d4f0 2023-10-06T03:49:50.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.016+0000 7f2b1b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b1c140bb0 0x7f2b1c168630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:50.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.016+0000 7f2b1b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b1c140bb0 0x7f2b1c168630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55894/0 (socket says 172.21.15.55:55894) 2023-10-06T03:49:50.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.016+0000 7f2b1b7fe700 1 -- 172.21.15.55:0/3034096073 learned_addr learned my addr 172.21.15.55:0/3034096073 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:50.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.016+0000 7f2b1affd700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b1c06d4f0 0x7f2b1c0c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:50.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.016+0000 7f2b1a7fc700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b1c1407e0 0x7f2b1c15cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:50.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.016+0000 7f2b1b7fe700 1 -- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b1c06d4f0 msgr2=0x7f2b1c0c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:50.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.016+0000 7f2b1b7fe700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b1c06d4f0 0x7f2b1c0c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:50.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.016+0000 7f2b1b7fe700 1 -- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b1c1407e0 msgr2=0x7f2b1c15cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:50.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.016+0000 7f2b1b7fe700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b1c1407e0 0x7f2b1c15cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:50.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.016+0000 7f2b1b7fe700 1 -- 172.21.15.55:0/3034096073 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b10009420 con 0x7f2b1c140bb0 2023-10-06T03:49:50.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.016+0000 7f2b1affd700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b1c06d4f0 0x7f2b1c0c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:50.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.016+0000 7f2b1b7fe700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b1c140bb0 0x7f2b1c168630 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400cd60 tx=0x7f2b14001b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:50.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.017+0000 7f2b02ffd700 1 -- 172.21.15.55:0/3034096073 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14005df0 con 0x7f2b1c140bb0 2023-10-06T03:49:50.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.017+0000 7f2b02ffd700 1 -- 172.21.15.55:0/3034096073 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1400a920 con 0x7f2b1c140bb0 2023-10-06T03:49:50.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.017+0000 7f2b02ffd700 1 -- 172.21.15.55:0/3034096073 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1400aca0 con 0x7f2b1c140bb0 2023-10-06T03:49:50.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.017+0000 7f2b1bfff700 1 -- 172.21.15.55:0/3034096073 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1c0c4960 con 0x7f2b1c140bb0 2023-10-06T03:49:50.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.017+0000 7f2b1bfff700 1 -- 172.21.15.55:0/3034096073 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1c0c4f20 con 0x7f2b1c140bb0 2023-10-06T03:49:50.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.018+0000 7f2b1bfff700 1 -- 172.21.15.55:0/3034096073 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b1c06a560 con 0x7f2b1c140bb0 2023-10-06T03:49:50.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.019+0000 7f2b02ffd700 1 -- 172.21.15.55:0/3034096073 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b14007300 con 0x7f2b1c140bb0 2023-10-06T03:49:50.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.020+0000 7f2b02ffd700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b08066f30 0x7f2b080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:50.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.020+0000 7f2b02ffd700 1 -- 172.21.15.55:0/3034096073 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f2b14084a00 con 0x7f2b1c140bb0 2023-10-06T03:49:50.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.020+0000 7f2b1affd700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b08066f30 0x7f2b080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:50.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.021+0000 7f2b1affd700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b08066f30 0x7f2b080693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c006d40 tx=0x7f2b0c008040 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:50.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.024+0000 7f2b02ffd700 1 -- 172.21.15.55:0/3034096073 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b14051960 con 0x7f2b1c140bb0 2023-10-06T03:49:50.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.271+0000 7f2b1bfff700 1 -- 172.21.15.55:0/3034096073 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2b1c06b040 con 0x7f2b1c140bb0 2023-10-06T03:49:50.799 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.798+0000 7f2b02ffd700 1 -- 172.21.15.55:0/3034096073 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2b14054f80 con 0x7f2b1c140bb0 2023-10-06T03:49:50.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.800+0000 7f2b1bfff700 1 -- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b08066f30 msgr2=0x7f2b080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:50.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.800+0000 7f2b1bfff700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b08066f30 0x7f2b080693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c006d40 tx=0x7f2b0c008040 comp rx=0 tx=0).stop 2023-10-06T03:49:50.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.800+0000 7f2b1bfff700 1 -- 172.21.15.55:0/3034096073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b1c140bb0 msgr2=0x7f2b1c168630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:50.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.800+0000 7f2b1bfff700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b1c140bb0 0x7f2b1c168630 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400cd60 tx=0x7f2b14001b60 comp rx=0 tx=0).stop 2023-10-06T03:49:50.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.800+0000 7f2b1bfff700 1 -- 172.21.15.55:0/3034096073 shutdown_connections 2023-10-06T03:49:50.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.800+0000 7f2b1bfff700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b08066f30 0x7f2b080693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:50.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.800+0000 7f2b1bfff700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b1c140bb0 0x7f2b1c168630 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:50.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.800+0000 7f2b1bfff700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b1c06d4f0 0x7f2b1c0c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:50.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.800+0000 7f2b1bfff700 1 --2- 172.21.15.55:0/3034096073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b1c1407e0 0x7f2b1c15cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:50.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.800+0000 7f2b1bfff700 1 -- 172.21.15.55:0/3034096073 >> 172.21.15.55:0/3034096073 conn(0x7f2b1c0cc1e0 msgr2=0x7f2b1c0c1920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:50.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.801+0000 7f2b1bfff700 1 -- 172.21.15.55:0/3034096073 shutdown_connections 2023-10-06T03:49:50.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:50.801+0000 7f2b1bfff700 1 -- 172.21.15.55:0/3034096073 wait complete. 2023-10-06T03:49:50.820 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:49:51.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.006+0000 7fb50e79d700 1 Processor -- start 2023-10-06T03:49:51.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.006+0000 7fb50e79d700 1 -- start start 2023-10-06T03:49:51.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.008+0000 7fb50e79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb50806a0a0 0x7fb50815b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.008+0000 7fb50e79d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb50806dd20 0x7fb50815b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.008+0000 7fb50e79d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50806ae70 con 0x7fb50806d950 2023-10-06T03:49:51.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.008+0000 7fb50e79d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5080c8a60 con 0x7fb50806dd20 2023-10-06T03:49:51.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.008+0000 7fb50e79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50806a540 con 0x7fb50806a0a0 2023-10-06T03:49:51.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.008+0000 7fb50df9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb50806dd20 0x7fb50815b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.008+0000 7fb50df9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb50806dd20 0x7fb50815b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55902/0 (socket says 172.21.15.55:55902) 2023-10-06T03:49:51.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.008+0000 7fb50df9c700 1 -- 172.21.15.55:0/716313163 learned_addr learned my addr 172.21.15.55:0/716313163 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:51.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.008+0000 7fb50cf9a700 1 --2- 172.21.15.55:0/716313163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb50806a0a0 0x7fb50815b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.008+0000 7fb50df9c700 1 -- 172.21.15.55:0/716313163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb50806a0a0 msgr2=0x7fb50815b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:51.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.008+0000 7fb50df9c700 1 --2- 172.21.15.55:0/716313163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb50806a0a0 0x7fb50815b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.008+0000 7fb50df9c700 1 -- 172.21.15.55:0/716313163 >> v1:172.21.15.163:6789/0 conn(0x7fb50806d950 legacy=0x7fb5080c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.008+0000 7fb50df9c700 1 -- 172.21.15.55:0/716313163 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50806a820 con 0x7fb50806dd20 2023-10-06T03:49:51.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.009+0000 7fb50df9c700 1 --2- 172.21.15.55:0/716313163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50806dd20 0x7fb50815b950 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb500009f20 tx=0x7fb500009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4992811d3ca35c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:51.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.009+0000 7fb4fffff700 1 -- 172.21.15.55:0/716313163 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50000e070 con 0x7fb50806dd20 2023-10-06T03:49:51.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.009+0000 7fb4fffff700 1 -- 172.21.15.55:0/716313163 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50000ba80 con 0x7fb50806dd20 2023-10-06T03:49:51.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.009+0000 7fb4fffff700 1 -- 172.21.15.55:0/716313163 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5000092e0 con 0x7fb50806dd20 2023-10-06T03:49:51.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.009+0000 7fb50e79d700 1 -- 172.21.15.55:0/716313163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50806dd20 msgr2=0x7fb50815b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.009+0000 7fb50e79d700 1 --2- 172.21.15.55:0/716313163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50806dd20 0x7fb50815b950 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb500009f20 tx=0x7fb500009840 comp rx=0 tx=0).stop 2023-10-06T03:49:51.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.010+0000 7fb50e79d700 1 -- 172.21.15.55:0/716313163 shutdown_connections 2023-10-06T03:49:51.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.010+0000 7fb50e79d700 1 --2- 172.21.15.55:0/716313163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50806dd20 0x7fb50815b950 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.010+0000 7fb50e79d700 1 --2- 172.21.15.55:0/716313163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb50806a0a0 0x7fb50815b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.010+0000 7fb50e79d700 1 -- 172.21.15.55:0/716313163 >> 172.21.15.55:0/716313163 conn(0x7fb5080cc1f0 msgr2=0x7fb5080cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:51.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.010+0000 7fb50e79d700 1 -- 172.21.15.55:0/716313163 shutdown_connections 2023-10-06T03:49:51.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.010+0000 7fb50e79d700 1 -- 172.21.15.55:0/716313163 wait complete. 2023-10-06T03:49:51.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.011+0000 7fb50e79d700 1 Processor -- start 2023-10-06T03:49:51.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.011+0000 7fb50e79d700 1 -- start start 2023-10-06T03:49:51.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.012+0000 7fb50e79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb50806a0a0 0x7fb508151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.012+0000 7fb50e79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb50806d950 0x7fb50815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.012+0000 7fb50e79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50806dd20 0x7fb5080c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.012+0000 7fb50e79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5081584f0 con 0x7fb50806dd20 2023-10-06T03:49:51.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.012+0000 7fb50e79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080c5780 con 0x7fb50806d950 2023-10-06T03:49:51.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.012+0000 7fb50e79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080c5a60 con 0x7fb50806a0a0 2023-10-06T03:49:51.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.013+0000 7fb50df9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50806dd20 0x7fb5080c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.013+0000 7fb50d79b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb50806a0a0 0x7fb508151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.013+0000 7fb50df9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50806dd20 0x7fb5080c4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55914/0 (socket says 172.21.15.55:55914) 2023-10-06T03:49:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.013+0000 7fb50df9c700 1 -- 172.21.15.55:0/2119734993 learned_addr learned my addr 172.21.15.55:0/2119734993 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.013+0000 7fb50cf9a700 1 --2- 172.21.15.55:0/2119734993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb50806d950 0x7fb50815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.013+0000 7fb50df9c700 1 -- 172.21.15.55:0/2119734993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb50806a0a0 msgr2=0x7fb508151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.013+0000 7fb50df9c700 1 --2- 172.21.15.55:0/2119734993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb50806a0a0 0x7fb508151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.013+0000 7fb50df9c700 1 -- 172.21.15.55:0/2119734993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb50806d950 msgr2=0x7fb50815a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.013+0000 7fb50df9c700 1 --2- 172.21.15.55:0/2119734993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb50806d950 0x7fb50815a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.013+0000 7fb50df9c700 1 -- 172.21.15.55:0/2119734993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb500009ae0 con 0x7fb50806dd20 2023-10-06T03:49:51.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.013+0000 7fb50df9c700 1 --2- 172.21.15.55:0/2119734993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50806dd20 0x7fb5080c4a10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb50000b6c0 tx=0x7fb500018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:51.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.014+0000 7fb4fd7fa700 1 -- 172.21.15.55:0/2119734993 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5000032f0 con 0x7fb50806dd20 2023-10-06T03:49:51.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.014+0000 7fb4fd7fa700 1 -- 172.21.15.55:0/2119734993 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb500008d20 con 0x7fb50806dd20 2023-10-06T03:49:51.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.014+0000 7fb50e79d700 1 -- 172.21.15.55:0/2119734993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5080c5d40 con 0x7fb50806dd20 2023-10-06T03:49:51.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.014+0000 7fb4fd7fa700 1 -- 172.21.15.55:0/2119734993 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500026670 con 0x7fb50806dd20 2023-10-06T03:49:51.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.014+0000 7fb50e79d700 1 -- 172.21.15.55:0/2119734993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb508152150 con 0x7fb50806dd20 2023-10-06T03:49:51.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.015+0000 7fb50e79d700 1 -- 172.21.15.55:0/2119734993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50804c3c0 con 0x7fb50806dd20 2023-10-06T03:49:51.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.016+0000 7fb4fd7fa700 1 -- 172.21.15.55:0/2119734993 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb500003040 con 0x7fb50806dd20 2023-10-06T03:49:51.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.017+0000 7fb4fd7fa700 1 --2- 172.21.15.55:0/2119734993 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb4ec066f30 0x7fb4ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.017+0000 7fb4fd7fa700 1 -- 172.21.15.55:0/2119734993 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fb500086890 con 0x7fb50806dd20 2023-10-06T03:49:51.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.017+0000 7fb50d79b700 1 --2- 172.21.15.55:0/2119734993 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb4ec066f30 0x7fb4ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.018+0000 7fb50d79b700 1 --2- 172.21.15.55:0/2119734993 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb4ec066f30 0x7fb4ec0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb504003a80 tx=0x7fb504007ec0 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:51.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.021+0000 7fb4fd7fa700 1 -- 172.21.15.55:0/2119734993 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5000538c0 con 0x7fb50806dd20 2023-10-06T03:49:51.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.266+0000 7fb50e79d700 1 -- 172.21.15.55:0/2119734993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb508156b10 con 0x7fb50806dd20 2023-10-06T03:49:51.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.267+0000 7fb4fd7fa700 1 -- 172.21.15.55:0/2119734993 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fb500021090 con 0x7fb50806dd20 2023-10-06T03:49:51.268 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:51.269 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2272331626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2272331626},{"type":"v1","addr":"172.21.15.163:6834","nonce":2272331626}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2110964237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2110964237},{"type":"v1","addr":"172.21.15.163:6836","nonce":2110964237}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2942349047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2942349047},{"type":"v1","addr":"172.21.15.55:6833","nonce":2942349047}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:49:47.265041+0000","modified":"2023-10-06T03:49:50.792326+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7254":{"gid":7254,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/2226955335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2226955335},{"type":"v1","addr":"172.21.15.55:6835","nonce":2226955335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T03:49:51.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.269+0000 7fb50e79d700 1 -- 172.21.15.55:0/2119734993 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb4ec066f30 msgr2=0x7fb4ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.269+0000 7fb50e79d700 1 --2- 172.21.15.55:0/2119734993 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb4ec066f30 0x7fb4ec0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb504003a80 tx=0x7fb504007ec0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.269+0000 7fb50e79d700 1 -- 172.21.15.55:0/2119734993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50806dd20 msgr2=0x7fb5080c4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.269+0000 7fb50e79d700 1 --2- 172.21.15.55:0/2119734993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50806dd20 0x7fb5080c4a10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb50000b6c0 tx=0x7fb500018000 comp rx=0 tx=0).stop 2023-10-06T03:49:51.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.269+0000 7fb50e79d700 1 -- 172.21.15.55:0/2119734993 shutdown_connections 2023-10-06T03:49:51.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.269+0000 7fb50e79d700 1 --2- 172.21.15.55:0/2119734993 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb4ec066f30 0x7fb4ec0693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.269+0000 7fb50e79d700 1 --2- 172.21.15.55:0/2119734993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb50806dd20 0x7fb5080c4a10 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.269+0000 7fb50e79d700 1 --2- 172.21.15.55:0/2119734993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb50806a0a0 0x7fb508151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.269+0000 7fb50e79d700 1 --2- 172.21.15.55:0/2119734993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb50806d950 0x7fb50815a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.269+0000 7fb50e79d700 1 -- 172.21.15.55:0/2119734993 >> 172.21.15.55:0/2119734993 conn(0x7fb5080cc1f0 msgr2=0x7fb5080c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:51.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.269+0000 7fb50e79d700 1 -- 172.21.15.55:0/2119734993 shutdown_connections 2023-10-06T03:49:51.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.269+0000 7fb50e79d700 1 -- 172.21.15.55:0/2119734993 wait complete. 2023-10-06T03:49:51.272 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 53 2023-10-06T03:49:51.288 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:49:51.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.474+0000 7ffaec99d700 1 Processor -- start 2023-10-06T03:49:51.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.474+0000 7ffaec99d700 1 -- start start 2023-10-06T03:49:51.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.475+0000 7ffaec99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffae80c6c60 0x7ffae80c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.475+0000 7ffaec99d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae80b1410 con 0x7ffae806e8c0 2023-10-06T03:49:51.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.475+0000 7ffaec99d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae8063230 con 0x7ffae81595a0 2023-10-06T03:49:51.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.475+0000 7ffaec99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae80b2b90 con 0x7ffae80c6c60 2023-10-06T03:49:51.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.475+0000 7ffae6ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffae80c6c60 0x7ffae80c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.475+0000 7ffae6ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffae80c6c60 0x7ffae80c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55918/0 (socket says 172.21.15.55:55918) 2023-10-06T03:49:51.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.475+0000 7ffae7fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ffae806e8c0 0x7ffae8159970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51892/0 (socket says 172.21.15.55:51892) 2023-10-06T03:49:51.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.475+0000 7ffae6ffd700 1 -- 172.21.15.55:0/605522371 learned_addr learned my addr 172.21.15.55:0/605522371 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.476+0000 7ffae6ffd700 1 -- 172.21.15.55:0/605522371 >> v1:172.21.15.163:6789/0 conn(0x7ffae81595a0 legacy=0x7ffae80b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.476+0000 7ffae6ffd700 1 -- 172.21.15.55:0/605522371 >> v1:172.21.15.55:6789/0 conn(0x7ffae806e8c0 legacy=0x7ffae8159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.476+0000 7ffae6ffd700 1 -- 172.21.15.55:0/605522371 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae80634f0 con 0x7ffae80c6c60 2023-10-06T03:49:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.476+0000 7ffae6ffd700 1 --2- 172.21.15.55:0/605522371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae80c6c60 0x7ffae80c8120 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0099b0 tx=0x7ffadc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5764057671b697fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.476+0000 7ffae67fc700 1 -- 172.21.15.55:0/605522371 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc00b760 con 0x7ffae80c6c60 2023-10-06T03:49:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.477+0000 7ffae67fc700 1 -- 172.21.15.55:0/605522371 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffadc00b8c0 con 0x7ffae80c6c60 2023-10-06T03:49:51.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.477+0000 7ffae67fc700 1 -- 172.21.15.55:0/605522371 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc00bc80 con 0x7ffae80c6c60 2023-10-06T03:49:51.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.477+0000 7ffaec99d700 1 -- 172.21.15.55:0/605522371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae80c6c60 msgr2=0x7ffae80c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.477+0000 7ffaec99d700 1 --2- 172.21.15.55:0/605522371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae80c6c60 0x7ffae80c8120 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0099b0 tx=0x7ffadc009250 comp rx=0 tx=0).stop 2023-10-06T03:49:51.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.477+0000 7ffaec99d700 1 -- 172.21.15.55:0/605522371 shutdown_connections 2023-10-06T03:49:51.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.477+0000 7ffaec99d700 1 --2- 172.21.15.55:0/605522371 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae80c6c60 0x7ffae80c8120 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.477+0000 7ffaec99d700 1 -- 172.21.15.55:0/605522371 >> 172.21.15.55:0/605522371 conn(0x7ffae806ddb0 msgr2=0x7ffae806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:51.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.477+0000 7ffaec99d700 1 -- 172.21.15.55:0/605522371 shutdown_connections 2023-10-06T03:49:51.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.477+0000 7ffaec99d700 1 -- 172.21.15.55:0/605522371 wait complete. 2023-10-06T03:49:51.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.479+0000 7ffaec99d700 1 Processor -- start 2023-10-06T03:49:51.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.479+0000 7ffaec99d700 1 -- start start 2023-10-06T03:49:51.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffaec99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae806e8c0 0x7ffae80c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffaec99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffae80c6c60 0x7ffae814fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffaec99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffae81595a0 0x7ffae816ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffaec99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae816a530 con 0x7ffae806e8c0 2023-10-06T03:49:51.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffaec99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae815b060 con 0x7ffae80c6c60 2023-10-06T03:49:51.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffaec99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae815b340 con 0x7ffae81595a0 2023-10-06T03:49:51.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffae77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae806e8c0 0x7ffae80c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffae77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae806e8c0 0x7ffae80c60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55924/0 (socket says 172.21.15.55:55924) 2023-10-06T03:49:51.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffae77fe700 1 -- 172.21.15.55:0/1853197912 learned_addr learned my addr 172.21.15.55:0/1853197912 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:51.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffae77fe700 1 -- 172.21.15.55:0/1853197912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffae81595a0 msgr2=0x7ffae816ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffae77fe700 1 --2- 172.21.15.55:0/1853197912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffae81595a0 0x7ffae816ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffae77fe700 1 -- 172.21.15.55:0/1853197912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffae80c6c60 msgr2=0x7ffae814fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffae77fe700 1 --2- 172.21.15.55:0/1853197912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffae80c6c60 0x7ffae814fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffae77fe700 1 -- 172.21.15.55:0/1853197912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc0094f0 con 0x7ffae806e8c0 2023-10-06T03:49:51.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.480+0000 7ffae77fe700 1 --2- 172.21.15.55:0/1853197912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae806e8c0 0x7ffae80c60b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ffad800c4e0 tx=0x7ffad800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:51.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.481+0000 7ffacf7fe700 1 -- 172.21.15.55:0/1853197912 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8011590 con 0x7ffae806e8c0 2023-10-06T03:49:51.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.481+0000 7ffacf7fe700 1 -- 172.21.15.55:0/1853197912 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad8011710 con 0x7ffae806e8c0 2023-10-06T03:49:51.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.481+0000 7ffacf7fe700 1 -- 172.21.15.55:0/1853197912 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8011ad0 con 0x7ffae806e8c0 2023-10-06T03:49:51.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.481+0000 7ffaec99d700 1 -- 172.21.15.55:0/1853197912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae815b620 con 0x7ffae806e8c0 2023-10-06T03:49:51.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.481+0000 7ffaec99d700 1 -- 172.21.15.55:0/1853197912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae80c65f0 con 0x7ffae806e8c0 2023-10-06T03:49:51.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.482+0000 7ffaec99d700 1 -- 172.21.15.55:0/1853197912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae816aa30 con 0x7ffae806e8c0 2023-10-06T03:49:51.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.483+0000 7ffacf7fe700 1 -- 172.21.15.55:0/1853197912 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffad8001020 con 0x7ffae806e8c0 2023-10-06T03:49:51.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.484+0000 7ffacf7fe700 1 --2- 172.21.15.55:0/1853197912 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffad4066f30 0x7ffad40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.484+0000 7ffacf7fe700 1 -- 172.21.15.55:0/1853197912 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7ffad8083d10 con 0x7ffae806e8c0 2023-10-06T03:49:51.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.484+0000 7ffae6ffd700 1 --2- 172.21.15.55:0/1853197912 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffad4066f30 0x7ffad40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.485+0000 7ffae6ffd700 1 --2- 172.21.15.55:0/1853197912 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffad4066f30 0x7ffad40693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0099b0 tx=0x7ffadc016000 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:51.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.488+0000 7ffacf7fe700 1 -- 172.21.15.55:0/1853197912 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffae816aa30 con 0x7ffae806e8c0 2023-10-06T03:49:51.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.694+0000 7ffaec99d700 1 -- 172.21.15.55:0/1853197912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffae816aa30 con 0x7ffae806e8c0 2023-10-06T03:49:51.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.695+0000 7ffacf7fe700 1 -- 172.21.15.55:0/1853197912 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+17523 (secure 0 0 0) 0x7ffad8017070 con 0x7ffae806e8c0 2023-10-06T03:49:51.697 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:51.697 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":107,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:49:48.775953+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:49:45.423436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T03:49:46.240142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:49:51.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.697+0000 7ffaec99d700 1 -- 172.21.15.55:0/1853197912 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffad4066f30 msgr2=0x7ffad40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.697+0000 7ffaec99d700 1 --2- 172.21.15.55:0/1853197912 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffad4066f30 0x7ffad40693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0099b0 tx=0x7ffadc016000 comp rx=0 tx=0).stop 2023-10-06T03:49:51.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.697+0000 7ffaec99d700 1 -- 172.21.15.55:0/1853197912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae806e8c0 msgr2=0x7ffae80c60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.697+0000 7ffaec99d700 1 --2- 172.21.15.55:0/1853197912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae806e8c0 0x7ffae80c60b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ffad800c4e0 tx=0x7ffad800c8a0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.698+0000 7ffaec99d700 1 -- 172.21.15.55:0/1853197912 shutdown_connections 2023-10-06T03:49:51.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.698+0000 7ffaec99d700 1 --2- 172.21.15.55:0/1853197912 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffad4066f30 0x7ffad40693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.698+0000 7ffaec99d700 1 --2- 172.21.15.55:0/1853197912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffae81595a0 0x7ffae816ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.698+0000 7ffaec99d700 1 --2- 172.21.15.55:0/1853197912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffae806e8c0 0x7ffae80c60b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.698+0000 7ffaec99d700 1 --2- 172.21.15.55:0/1853197912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffae80c6c60 0x7ffae814fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.698+0000 7ffaec99d700 1 -- 172.21.15.55:0/1853197912 >> 172.21.15.55:0/1853197912 conn(0x7ffae806ddb0 msgr2=0x7ffae8158740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:51.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.698+0000 7ffaec99d700 1 -- 172.21.15.55:0/1853197912 shutdown_connections 2023-10-06T03:49:51.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.698+0000 7ffaec99d700 1 -- 172.21.15.55:0/1853197912 wait complete. 2023-10-06T03:49:51.718 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:49:51.718 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:49:51.736 INFO:tasks.ceph.mds.a.smithi055.stderr:2023-10-06T03:49:51.734+0000 7f702e7f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:49:51.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.904+0000 7fd72d39d700 1 Processor -- start 2023-10-06T03:49:51.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.904+0000 7fd72d39d700 1 -- start start 2023-10-06T03:49:51.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.905+0000 7fd72d39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd7281597f0 0x7fd7280b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.905+0000 7fd72d39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd72815d230 0x7fd72815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.905+0000 7fd72d39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd72815dbb0 con 0x7fd728159420 2023-10-06T03:49:51.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.905+0000 7fd72d39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7280b11d0 con 0x7fd7281597f0 2023-10-06T03:49:51.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.905+0000 7fd72d39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280b14b0 con 0x7fd72815d230 2023-10-06T03:49:51.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.906+0000 7fd7277fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd7281597f0 0x7fd7280b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.906+0000 7fd7277fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd7281597f0 0x7fd7280b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55936/0 (socket says 172.21.15.55:55936) 2023-10-06T03:49:51.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.906+0000 7fd7277fe700 1 -- 172.21.15.55:0/1649042711 learned_addr learned my addr 172.21.15.55:0/1649042711 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:51.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.906+0000 7fd727fff700 1 --1- 172.21.15.55:0/1649042711 >> v1:172.21.15.55:6789/0 conn(0x7fd728159420 0x7fd7280b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51900/0 (socket says 172.21.15.55:51900) 2023-10-06T03:49:51.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.906+0000 7fd72cb9c700 1 --2- 172.21.15.55:0/1649042711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd72815d230 0x7fd72815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.906+0000 7fd7277fe700 1 -- 172.21.15.55:0/1649042711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd72815d230 msgr2=0x7fd72815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:51.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.906+0000 7fd7277fe700 1 --2- 172.21.15.55:0/1649042711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd72815d230 0x7fd72815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.906+0000 7fd726ffd700 1 -- 172.21.15.55:0/1649042711 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599438341 0 0) 0x7fd72815dbb0 con 0x7fd728159420 2023-10-06T03:49:51.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.906+0000 7fd7277fe700 1 -- 172.21.15.55:0/1649042711 >> v1:172.21.15.55:6789/0 conn(0x7fd728159420 legacy=0x7fd7280b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.906+0000 7fd7277fe700 1 -- 172.21.15.55:0/1649042711 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7280b1790 con 0x7fd7281597f0 2023-10-06T03:49:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.907+0000 7fd7277fe700 1 --2- 172.21.15.55:0/1649042711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281597f0 0x7fd7280b2270 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0099b0 tx=0x7fd71c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3081014e0a837b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.907+0000 7fd726ffd700 1 -- 172.21.15.55:0/1649042711 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c009ea0 con 0x7fd7281597f0 2023-10-06T03:49:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.907+0000 7fd726ffd700 1 -- 172.21.15.55:0/1649042711 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71c00b860 con 0x7fd7281597f0 2023-10-06T03:49:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.907+0000 7fd726ffd700 1 -- 172.21.15.55:0/1649042711 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c00bc00 con 0x7fd7281597f0 2023-10-06T03:49:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.907+0000 7fd72d39d700 1 -- 172.21.15.55:0/1649042711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281597f0 msgr2=0x7fd7280b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.907+0000 7fd72d39d700 1 --2- 172.21.15.55:0/1649042711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281597f0 0x7fd7280b2270 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0099b0 tx=0x7fd71c009250 comp rx=0 tx=0).stop 2023-10-06T03:49:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.907+0000 7fd72d39d700 1 -- 172.21.15.55:0/1649042711 shutdown_connections 2023-10-06T03:49:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.907+0000 7fd72d39d700 1 --2- 172.21.15.55:0/1649042711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd72815d230 0x7fd72815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.908+0000 7fd72d39d700 1 --2- 172.21.15.55:0/1649042711 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281597f0 0x7fd7280b2270 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.908+0000 7fd72d39d700 1 -- 172.21.15.55:0/1649042711 >> 172.21.15.55:0/1649042711 conn(0x7fd7280d0660 msgr2=0x7fd7280d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:51.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.908+0000 7fd72d39d700 1 -- 172.21.15.55:0/1649042711 shutdown_connections 2023-10-06T03:49:51.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.908+0000 7fd72d39d700 1 -- 172.21.15.55:0/1649042711 wait complete. 2023-10-06T03:49:51.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.909+0000 7fd72d39d700 1 Processor -- start 2023-10-06T03:49:51.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.909+0000 7fd72d39d700 1 -- start start 2023-10-06T03:49:51.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.910+0000 7fd72d39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd728159420 0x7fd7281575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.910+0000 7fd72d39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281597f0 0x7fd72815b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.910+0000 7fd72d39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd72815d230 0x7fd728153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.910+0000 7fd72d39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7280c5530 con 0x7fd7281597f0 2023-10-06T03:49:51.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.910+0000 7fd72d39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72815cba0 con 0x7fd728159420 2023-10-06T03:49:51.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.910+0000 7fd72d39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72815c300 con 0x7fd72815d230 2023-10-06T03:49:51.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.910+0000 7fd7277fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281597f0 0x7fd72815b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.910+0000 7fd7277fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281597f0 0x7fd72815b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55952/0 (socket says 172.21.15.55:55952) 2023-10-06T03:49:51.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.910+0000 7fd7277fe700 1 -- 172.21.15.55:0/3632919777 learned_addr learned my addr 172.21.15.55:0/3632919777 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:51.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.910+0000 7fd72cb9c700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd72815d230 0x7fd728153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.910+0000 7fd727fff700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd728159420 0x7fd7281575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.911+0000 7fd7277fe700 1 -- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd72815d230 msgr2=0x7fd728153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.911+0000 7fd7277fe700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd72815d230 0x7fd728153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.911+0000 7fd7277fe700 1 -- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd728159420 msgr2=0x7fd7281575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:51.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.911+0000 7fd7277fe700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd728159420 0x7fd7281575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:51.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.911+0000 7fd7277fe700 1 -- 172.21.15.55:0/3632919777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c0094f0 con 0x7fd7281597f0 2023-10-06T03:49:51.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.911+0000 7fd727fff700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd728159420 0x7fd7281575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:51.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.911+0000 7fd7277fe700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281597f0 0x7fd72815b880 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd728089340 tx=0x7fd71c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:51.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.911+0000 7fd70ffff700 1 -- 172.21.15.55:0/3632919777 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c00bbe0 con 0x7fd7281597f0 2023-10-06T03:49:51.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.911+0000 7fd70ffff700 1 -- 172.21.15.55:0/3632919777 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71c011420 con 0x7fd7281597f0 2023-10-06T03:49:51.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.911+0000 7fd70ffff700 1 -- 172.21.15.55:0/3632919777 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c0117e0 con 0x7fd7281597f0 2023-10-06T03:49:51.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.911+0000 7fd72d39d700 1 -- 172.21.15.55:0/3632919777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72815c5e0 con 0x7fd7281597f0 2023-10-06T03:49:51.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.911+0000 7fd72d39d700 1 -- 172.21.15.55:0/3632919777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd728159e50 con 0x7fd7281597f0 2023-10-06T03:49:51.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.912+0000 7fd72d39d700 1 -- 172.21.15.55:0/3632919777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72804c3c0 con 0x7fd7281597f0 2023-10-06T03:49:51.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.913+0000 7fd70ffff700 1 -- 172.21.15.55:0/3632919777 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd71c011c10 con 0x7fd7281597f0 2023-10-06T03:49:51.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.914+0000 7fd70ffff700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd714066f30 0x7fd7140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:51.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.914+0000 7fd70ffff700 1 -- 172.21.15.55:0/3632919777 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fd71c0862b0 con 0x7fd7281597f0 2023-10-06T03:49:51.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.915+0000 7fd727fff700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd714066f30 0x7fd7140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:51.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.915+0000 7fd727fff700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd714066f30 0x7fd7140693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7180011e0 tx=0x7fd718007c90 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:51.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:51.918+0000 7fd70ffff700 1 -- 172.21.15.55:0/3632919777 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd71c053190 con 0x7fd7281597f0 2023-10-06T03:49:52.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.119+0000 7fd72d39d700 1 -- 172.21.15.55:0/3632919777 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7280c5670 con 0x7fd714066f30 2023-10-06T03:49:52.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.131+0000 7fd70ffff700 1 -- 172.21.15.55:0/3632919777 <== mgr.6799 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311569 (secure 0 0 0) 0x7fd7280c5670 con 0x7fd714066f30 2023-10-06T03:49:52.137 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:52.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.135+0000 7fd72d39d700 1 -- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd714066f30 msgr2=0x7fd7140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:52.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.135+0000 7fd72d39d700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd714066f30 0x7fd7140693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7180011e0 tx=0x7fd718007c90 comp rx=0 tx=0).stop 2023-10-06T03:49:52.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.135+0000 7fd72d39d700 1 -- 172.21.15.55:0/3632919777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281597f0 msgr2=0x7fd72815b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.135+0000 7fd72d39d700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281597f0 0x7fd72815b880 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd728089340 tx=0x7fd71c017000 comp rx=0 tx=0).stop 2023-10-06T03:49:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.135+0000 7fd72d39d700 1 -- 172.21.15.55:0/3632919777 shutdown_connections 2023-10-06T03:49:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.135+0000 7fd72d39d700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd714066f30 0x7fd7140693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.135+0000 7fd72d39d700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd72815d230 0x7fd728153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.135+0000 7fd72d39d700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd728159420 0x7fd7281575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.135+0000 7fd72d39d700 1 --2- 172.21.15.55:0/3632919777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd7281597f0 0x7fd72815b880 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.135+0000 7fd72d39d700 1 -- 172.21.15.55:0/3632919777 >> 172.21.15.55:0/3632919777 conn(0x7fd7280d0660 msgr2=0x7fd728156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.136+0000 7fd72d39d700 1 -- 172.21.15.55:0/3632919777 shutdown_connections 2023-10-06T03:49:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.136+0000 7fd72d39d700 1 -- 172.21.15.55:0/3632919777 wait complete. 2023-10-06T03:49:52.140 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:49:52.155 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-06T03:49:52.029724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":363,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":243,"ondisk_log_size":243,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2455224,"kb_used_data":11384,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752519496,"statfs":{"total":773094113280,"available":770579963904,"internally_reserved":0,"allocated":11657216,"data_stored":6490636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001648"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.793544+0000","last_change":"2023-10-06T03:49:47.789270+0000","last_active":"2023-10-06T03:49:47.793544+0000","last_peered":"2023-10-06T03:49:47.793544+0000","last_clean":"2023-10-06T03:49:47.793544+0000","last_became_active":"2023-10-06T03:49:47.788600+0000","last_became_peered":"2023-10-06T03:49:47.788600+0000","last_unstale":"2023-10-06T03:49:47.793544+0000","last_undegraded":"2023-10-06T03:49:47.793544+0000","last_fullsized":"2023-10-06T03:49:47.793544+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:31.915354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786503+0000","last_change":"2023-10-06T03:49:47.784606+0000","last_active":"2023-10-06T03:49:47.786503+0000","last_peered":"2023-10-06T03:49:47.786503+0000","last_clean":"2023-10-06T03:49:47.786503+0000","last_became_active":"2023-10-06T03:49:47.784319+0000","last_became_peered":"2023-10-06T03:49:47.784319+0000","last_unstale":"2023-10-06T03:49:47.786503+0000","last_undegraded":"2023-10-06T03:49:47.786503+0000","last_fullsized":"2023-10-06T03:49:47.786503+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:32.783620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.780370+0000","last_change":"2023-10-06T03:49:47.775923+0000","last_active":"2023-10-06T03:49:47.780370+0000","last_peered":"2023-10-06T03:49:47.780370+0000","last_clean":"2023-10-06T03:49:47.780370+0000","last_became_active":"2023-10-06T03:49:47.775579+0000","last_became_peered":"2023-10-06T03:49:47.775579+0000","last_unstale":"2023-10-06T03:49:47.780370+0000","last_undegraded":"2023-10-06T03:49:47.780370+0000","last_fullsized":"2023-10-06T03:49:47.780370+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:16.265534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.807530+0000","last_change":"2023-10-06T03:49:47.798390+0000","last_active":"2023-10-06T03:49:47.807530+0000","last_peered":"2023-10-06T03:49:47.807530+0000","last_clean":"2023-10-06T03:49:47.807530+0000","last_became_active":"2023-10-06T03:49:47.798003+0000","last_became_peered":"2023-10-06T03:49:47.798003+0000","last_unstale":"2023-10-06T03:49:47.807530+0000","last_undegraded":"2023-10-06T03:49:47.807530+0000","last_fullsized":"2023-10-06T03:49:47.807530+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:10.207499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.783305+0000","last_change":"2023-10-06T03:49:47.774228+0000","last_active":"2023-10-06T03:49:47.783305+0000","last_peered":"2023-10-06T03:49:47.783305+0000","last_clean":"2023-10-06T03:49:47.783305+0000","last_became_active":"2023-10-06T03:49:47.773969+0000","last_became_peered":"2023-10-06T03:49:47.773969+0000","last_unstale":"2023-10-06T03:49:47.783305+0000","last_undegraded":"2023-10-06T03:49:47.783305+0000","last_fullsized":"2023-10-06T03:49:47.783305+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:14.763921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.802944+0000","last_change":"2023-10-06T03:49:47.801538+0000","last_active":"2023-10-06T03:49:47.802944+0000","last_peered":"2023-10-06T03:49:47.802944+0000","last_clean":"2023-10-06T03:49:47.802944+0000","last_became_active":"2023-10-06T03:49:47.799566+0000","last_became_peered":"2023-10-06T03:49:47.799566+0000","last_unstale":"2023-10-06T03:49:47.802944+0000","last_undegraded":"2023-10-06T03:49:47.802944+0000","last_fullsized":"2023-10-06T03:49:47.802944+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:13.718257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.788796+0000","last_change":"2023-10-06T03:49:47.787934+0000","last_active":"2023-10-06T03:49:47.788796+0000","last_peered":"2023-10-06T03:49:47.788796+0000","last_clean":"2023-10-06T03:49:47.788796+0000","last_became_active":"2023-10-06T03:49:47.787683+0000","last_became_peered":"2023-10-06T03:49:47.787683+0000","last_unstale":"2023-10-06T03:49:47.788796+0000","last_undegraded":"2023-10-06T03:49:47.788796+0000","last_fullsized":"2023-10-06T03:49:47.788796+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:33.647871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.792627+0000","last_change":"2023-10-06T03:49:47.782656+0000","last_active":"2023-10-06T03:49:47.792627+0000","last_peered":"2023-10-06T03:49:47.792627+0000","last_clean":"2023-10-06T03:49:47.792627+0000","last_became_active":"2023-10-06T03:49:47.782063+0000","last_became_peered":"2023-10-06T03:49:47.782063+0000","last_unstale":"2023-10-06T03:49:47.792627+0000","last_undegraded":"2023-10-06T03:49:47.792627+0000","last_fullsized":"2023-10-06T03:49:47.792627+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:51.936358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.785464+0000","last_change":"2023-10-06T03:49:47.780506+0000","last_active":"2023-10-06T03:49:47.785464+0000","last_peered":"2023-10-06T03:49:47.785464+0000","last_clean":"2023-10-06T03:49:47.785464+0000","last_became_active":"2023-10-06T03:49:47.779406+0000","last_became_peered":"2023-10-06T03:49:47.779406+0000","last_unstale":"2023-10-06T03:49:47.785464+0000","last_undegraded":"2023-10-06T03:49:47.785464+0000","last_fullsized":"2023-10-06T03:49:47.785464+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:20.642945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798257+0000","last_change":"2023-10-06T03:49:47.796864+0000","last_active":"2023-10-06T03:49:47.798257+0000","last_peered":"2023-10-06T03:49:47.798257+0000","last_clean":"2023-10-06T03:49:47.798257+0000","last_became_active":"2023-10-06T03:49:47.796553+0000","last_became_peered":"2023-10-06T03:49:47.796553+0000","last_unstale":"2023-10-06T03:49:47.798257+0000","last_undegraded":"2023-10-06T03:49:47.798257+0000","last_fullsized":"2023-10-06T03:49:47.798257+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:47.044717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.787316+0000","last_change":"2023-10-06T03:49:47.785231+0000","last_active":"2023-10-06T03:49:47.787316+0000","last_peered":"2023-10-06T03:49:47.787316+0000","last_clean":"2023-10-06T03:49:47.787316+0000","last_became_active":"2023-10-06T03:49:47.784929+0000","last_became_peered":"2023-10-06T03:49:47.784929+0000","last_unstale":"2023-10-06T03:49:47.787316+0000","last_undegraded":"2023-10-06T03:49:47.787316+0000","last_fullsized":"2023-10-06T03:49:47.787316+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:57.703376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.787202+0000","last_change":"2023-10-06T03:49:47.784567+0000","last_active":"2023-10-06T03:49:47.787202+0000","last_peered":"2023-10-06T03:49:47.787202+0000","last_clean":"2023-10-06T03:49:47.787202+0000","last_became_active":"2023-10-06T03:49:47.784303+0000","last_became_peered":"2023-10-06T03:49:47.784303+0000","last_unstale":"2023-10-06T03:49:47.787202+0000","last_undegraded":"2023-10-06T03:49:47.787202+0000","last_fullsized":"2023-10-06T03:49:47.787202+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:22.428494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.802307+0000","last_change":"2023-10-06T03:49:47.797712+0000","last_active":"2023-10-06T03:49:47.802307+0000","last_peered":"2023-10-06T03:49:47.802307+0000","last_clean":"2023-10-06T03:49:47.802307+0000","last_became_active":"2023-10-06T03:49:47.796202+0000","last_became_peered":"2023-10-06T03:49:47.796202+0000","last_unstale":"2023-10-06T03:49:47.802307+0000","last_undegraded":"2023-10-06T03:49:47.802307+0000","last_fullsized":"2023-10-06T03:49:47.802307+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:01.617354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798736+0000","last_change":"2023-10-06T03:49:47.796997+0000","last_active":"2023-10-06T03:49:47.798736+0000","last_peered":"2023-10-06T03:49:47.798736+0000","last_clean":"2023-10-06T03:49:47.798736+0000","last_became_active":"2023-10-06T03:49:47.796690+0000","last_became_peered":"2023-10-06T03:49:47.796690+0000","last_unstale":"2023-10-06T03:49:47.798736+0000","last_undegraded":"2023-10-06T03:49:47.798736+0000","last_fullsized":"2023-10-06T03:49:47.798736+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:38.769554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.797970+0000","last_change":"2023-10-06T03:49:47.792060+0000","last_active":"2023-10-06T03:49:47.797970+0000","last_peered":"2023-10-06T03:49:47.797970+0000","last_clean":"2023-10-06T03:49:47.797970+0000","last_became_active":"2023-10-06T03:49:47.790622+0000","last_became_peered":"2023-10-06T03:49:47.790622+0000","last_unstale":"2023-10-06T03:49:47.797970+0000","last_undegraded":"2023-10-06T03:49:47.797970+0000","last_fullsized":"2023-10-06T03:49:47.797970+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:13.345647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.787285+0000","last_change":"2023-10-06T03:49:47.784537+0000","last_active":"2023-10-06T03:49:47.787285+0000","last_peered":"2023-10-06T03:49:47.787285+0000","last_clean":"2023-10-06T03:49:47.787285+0000","last_became_active":"2023-10-06T03:49:47.784113+0000","last_became_peered":"2023-10-06T03:49:47.784113+0000","last_unstale":"2023-10-06T03:49:47.787285+0000","last_undegraded":"2023-10-06T03:49:47.787285+0000","last_fullsized":"2023-10-06T03:49:47.787285+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:34.769673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.787772+0000","last_change":"2023-10-06T03:49:47.782168+0000","last_active":"2023-10-06T03:49:47.787772+0000","last_peered":"2023-10-06T03:49:47.787772+0000","last_clean":"2023-10-06T03:49:47.787772+0000","last_became_active":"2023-10-06T03:49:47.781891+0000","last_became_peered":"2023-10-06T03:49:47.781891+0000","last_unstale":"2023-10-06T03:49:47.787772+0000","last_undegraded":"2023-10-06T03:49:47.787772+0000","last_fullsized":"2023-10-06T03:49:47.787772+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:26.405246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.782594+0000","last_change":"2023-10-06T03:49:47.781056+0000","last_active":"2023-10-06T03:49:47.782594+0000","last_peered":"2023-10-06T03:49:47.782594+0000","last_clean":"2023-10-06T03:49:47.782594+0000","last_became_active":"2023-10-06T03:49:47.780746+0000","last_became_peered":"2023-10-06T03:49:47.780746+0000","last_unstale":"2023-10-06T03:49:47.782594+0000","last_undegraded":"2023-10-06T03:49:47.782594+0000","last_fullsized":"2023-10-06T03:49:47.782594+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:54.767724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798740+0000","last_change":"2023-10-06T03:49:47.797817+0000","last_active":"2023-10-06T03:49:47.798740+0000","last_peered":"2023-10-06T03:49:47.798740+0000","last_clean":"2023-10-06T03:49:47.798740+0000","last_became_active":"2023-10-06T03:49:47.797493+0000","last_became_peered":"2023-10-06T03:49:47.797493+0000","last_unstale":"2023-10-06T03:49:47.798740+0000","last_undegraded":"2023-10-06T03:49:47.798740+0000","last_fullsized":"2023-10-06T03:49:47.798740+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:52.359312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798176+0000","last_change":"2023-10-06T03:49:47.792520+0000","last_active":"2023-10-06T03:49:47.798176+0000","last_peered":"2023-10-06T03:49:47.798176+0000","last_clean":"2023-10-06T03:49:47.798176+0000","last_became_active":"2023-10-06T03:49:47.792251+0000","last_became_peered":"2023-10-06T03:49:47.792251+0000","last_unstale":"2023-10-06T03:49:47.798176+0000","last_undegraded":"2023-10-06T03:49:47.798176+0000","last_fullsized":"2023-10-06T03:49:47.798176+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:52.097059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.785638+0000","last_change":"2023-10-06T03:49:47.781705+0000","last_active":"2023-10-06T03:49:47.785638+0000","last_peered":"2023-10-06T03:49:47.785638+0000","last_clean":"2023-10-06T03:49:47.785638+0000","last_became_active":"2023-10-06T03:49:47.781364+0000","last_became_peered":"2023-10-06T03:49:47.781364+0000","last_unstale":"2023-10-06T03:49:47.785638+0000","last_undegraded":"2023-10-06T03:49:47.785638+0000","last_fullsized":"2023-10-06T03:49:47.785638+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:43.972937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798756+0000","last_change":"2023-10-06T03:49:47.797699+0000","last_active":"2023-10-06T03:49:47.798756+0000","last_peered":"2023-10-06T03:49:47.798756+0000","last_clean":"2023-10-06T03:49:47.798756+0000","last_became_active":"2023-10-06T03:49:47.797436+0000","last_became_peered":"2023-10-06T03:49:47.797436+0000","last_unstale":"2023-10-06T03:49:47.798756+0000","last_undegraded":"2023-10-06T03:49:47.798756+0000","last_fullsized":"2023-10-06T03:49:47.798756+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:54.050273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.793129+0000","last_change":"2023-10-06T03:49:47.791743+0000","last_active":"2023-10-06T03:49:47.793129+0000","last_peered":"2023-10-06T03:49:47.793129+0000","last_clean":"2023-10-06T03:49:47.793129+0000","last_became_active":"2023-10-06T03:49:47.791403+0000","last_became_peered":"2023-10-06T03:49:47.791403+0000","last_unstale":"2023-10-06T03:49:47.793129+0000","last_undegraded":"2023-10-06T03:49:47.793129+0000","last_fullsized":"2023-10-06T03:49:47.793129+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:39.878197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.781565+0000","last_change":"2023-10-06T03:49:47.777398+0000","last_active":"2023-10-06T03:49:47.781565+0000","last_peered":"2023-10-06T03:49:47.781565+0000","last_clean":"2023-10-06T03:49:47.781565+0000","last_became_active":"2023-10-06T03:49:47.776542+0000","last_became_peered":"2023-10-06T03:49:47.776542+0000","last_unstale":"2023-10-06T03:49:47.781565+0000","last_undegraded":"2023-10-06T03:49:47.781565+0000","last_fullsized":"2023-10-06T03:49:47.781565+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:09.479114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786542+0000","last_change":"2023-10-06T03:49:47.783949+0000","last_active":"2023-10-06T03:49:47.786542+0000","last_peered":"2023-10-06T03:49:47.786542+0000","last_clean":"2023-10-06T03:49:47.786542+0000","last_became_active":"2023-10-06T03:49:47.783637+0000","last_became_peered":"2023-10-06T03:49:47.783637+0000","last_unstale":"2023-10-06T03:49:47.786542+0000","last_undegraded":"2023-10-06T03:49:47.786542+0000","last_fullsized":"2023-10-06T03:49:47.786542+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:20.377785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798113+0000","last_change":"2023-10-06T03:49:47.796630+0000","last_active":"2023-10-06T03:49:47.798113+0000","last_peered":"2023-10-06T03:49:47.798113+0000","last_clean":"2023-10-06T03:49:47.798113+0000","last_became_active":"2023-10-06T03:49:47.796346+0000","last_became_peered":"2023-10-06T03:49:47.796346+0000","last_unstale":"2023-10-06T03:49:47.798113+0000","last_undegraded":"2023-10-06T03:49:47.798113+0000","last_fullsized":"2023-10-06T03:49:47.798113+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:02.836245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.775346+0000","last_change":"2023-10-06T03:49:47.772613+0000","last_active":"2023-10-06T03:49:47.775346+0000","last_peered":"2023-10-06T03:49:47.775346+0000","last_clean":"2023-10-06T03:49:47.775346+0000","last_became_active":"2023-10-06T03:49:47.772333+0000","last_became_peered":"2023-10-06T03:49:47.772333+0000","last_unstale":"2023-10-06T03:49:47.775346+0000","last_undegraded":"2023-10-06T03:49:47.775346+0000","last_fullsized":"2023-10-06T03:49:47.775346+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:33.444850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798192+0000","last_change":"2023-10-06T03:49:47.792782+0000","last_active":"2023-10-06T03:49:47.798192+0000","last_peered":"2023-10-06T03:49:47.798192+0000","last_clean":"2023-10-06T03:49:47.798192+0000","last_became_active":"2023-10-06T03:49:47.792039+0000","last_became_peered":"2023-10-06T03:49:47.792039+0000","last_unstale":"2023-10-06T03:49:47.798192+0000","last_undegraded":"2023-10-06T03:49:47.798192+0000","last_fullsized":"2023-10-06T03:49:47.798192+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:40.274992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786322+0000","last_change":"2023-10-06T03:49:47.783823+0000","last_active":"2023-10-06T03:49:47.786322+0000","last_peered":"2023-10-06T03:49:47.786322+0000","last_clean":"2023-10-06T03:49:47.786322+0000","last_became_active":"2023-10-06T03:49:47.782859+0000","last_became_peered":"2023-10-06T03:49:47.782859+0000","last_unstale":"2023-10-06T03:49:47.786322+0000","last_undegraded":"2023-10-06T03:49:47.786322+0000","last_fullsized":"2023-10-06T03:49:47.786322+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:05.450712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.785507+0000","last_change":"2023-10-06T03:49:47.779301+0000","last_active":"2023-10-06T03:49:47.785507+0000","last_peered":"2023-10-06T03:49:47.785507+0000","last_clean":"2023-10-06T03:49:47.785507+0000","last_became_active":"2023-10-06T03:49:47.779030+0000","last_became_peered":"2023-10-06T03:49:47.779030+0000","last_unstale":"2023-10-06T03:49:47.785507+0000","last_undegraded":"2023-10-06T03:49:47.785507+0000","last_fullsized":"2023-10-06T03:49:47.785507+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:07.974494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.793426+0000","last_change":"2023-10-06T03:49:47.791604+0000","last_active":"2023-10-06T03:49:47.793426+0000","last_peered":"2023-10-06T03:49:47.793426+0000","last_clean":"2023-10-06T03:49:47.793426+0000","last_became_active":"2023-10-06T03:49:47.791319+0000","last_became_peered":"2023-10-06T03:49:47.791319+0000","last_unstale":"2023-10-06T03:49:47.793426+0000","last_undegraded":"2023-10-06T03:49:47.793426+0000","last_fullsized":"2023-10-06T03:49:47.793426+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:40.274992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.787082+0000","last_change":"2023-10-06T03:49:47.784378+0000","last_active":"2023-10-06T03:49:47.787082+0000","last_peered":"2023-10-06T03:49:47.787082+0000","last_clean":"2023-10-06T03:49:47.787082+0000","last_became_active":"2023-10-06T03:49:47.783711+0000","last_became_peered":"2023-10-06T03:49:47.783711+0000","last_unstale":"2023-10-06T03:49:47.787082+0000","last_undegraded":"2023-10-06T03:49:47.787082+0000","last_fullsized":"2023-10-06T03:49:47.787082+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:27.472375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.796322+0000","last_change":"2023-10-06T03:49:47.792410+0000","last_active":"2023-10-06T03:49:47.796322+0000","last_peered":"2023-10-06T03:49:47.796322+0000","last_clean":"2023-10-06T03:49:47.796322+0000","last_became_active":"2023-10-06T03:49:47.792161+0000","last_became_peered":"2023-10-06T03:49:47.792161+0000","last_unstale":"2023-10-06T03:49:47.796322+0000","last_undegraded":"2023-10-06T03:49:47.796322+0000","last_fullsized":"2023-10-06T03:49:47.796322+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:45.707331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"106'4","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.806941+0000","last_change":"2023-10-06T03:49:46.774436+0000","last_active":"2023-10-06T03:49:47.806941+0000","last_peered":"2023-10-06T03:49:47.806941+0000","last_clean":"2023-10-06T03:49:47.806941+0000","last_became_active":"2023-10-06T03:49:46.774149+0000","last_became_peered":"2023-10-06T03:49:46.774149+0000","last_unstale":"2023-10-06T03:49:47.806941+0000","last_undegraded":"2023-10-06T03:49:47.806941+0000","last_fullsized":"2023-10-06T03:49:47.806941+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:00.438204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786971+0000","last_change":"2023-10-06T03:49:47.780333+0000","last_active":"2023-10-06T03:49:47.786971+0000","last_peered":"2023-10-06T03:49:47.786971+0000","last_clean":"2023-10-06T03:49:47.786971+0000","last_became_active":"2023-10-06T03:49:47.780060+0000","last_became_peered":"2023-10-06T03:49:47.780060+0000","last_unstale":"2023-10-06T03:49:47.786971+0000","last_undegraded":"2023-10-06T03:49:47.786971+0000","last_fullsized":"2023-10-06T03:49:47.786971+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:05.183707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.801983+0000","last_change":"2023-10-06T03:49:46.778045+0000","last_active":"2023-10-06T03:49:47.801983+0000","last_peered":"2023-10-06T03:49:47.801983+0000","last_clean":"2023-10-06T03:49:47.801983+0000","last_became_active":"2023-10-06T03:49:46.777746+0000","last_became_peered":"2023-10-06T03:49:46.777746+0000","last_unstale":"2023-10-06T03:49:47.801983+0000","last_undegraded":"2023-10-06T03:49:47.801983+0000","last_fullsized":"2023-10-06T03:49:47.801983+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:09.526031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798353+0000","last_change":"2023-10-06T03:49:47.797355+0000","last_active":"2023-10-06T03:49:47.798353+0000","last_peered":"2023-10-06T03:49:47.798353+0000","last_clean":"2023-10-06T03:49:47.798353+0000","last_became_active":"2023-10-06T03:49:47.796981+0000","last_became_peered":"2023-10-06T03:49:47.796981+0000","last_unstale":"2023-10-06T03:49:47.798353+0000","last_undegraded":"2023-10-06T03:49:47.798353+0000","last_fullsized":"2023-10-06T03:49:47.798353+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:24.179034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798891+0000","last_change":"2023-10-06T03:49:46.782511+0000","last_active":"2023-10-06T03:49:47.798891+0000","last_peered":"2023-10-06T03:49:47.798891+0000","last_clean":"2023-10-06T03:49:47.798891+0000","last_became_active":"2023-10-06T03:49:46.782113+0000","last_became_peered":"2023-10-06T03:49:46.782113+0000","last_unstale":"2023-10-06T03:49:47.798891+0000","last_undegraded":"2023-10-06T03:49:47.798891+0000","last_fullsized":"2023-10-06T03:49:47.798891+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:00.967847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.787750+0000","last_change":"2023-10-06T03:49:47.781651+0000","last_active":"2023-10-06T03:49:47.787750+0000","last_peered":"2023-10-06T03:49:47.787750+0000","last_clean":"2023-10-06T03:49:47.787750+0000","last_became_active":"2023-10-06T03:49:47.781282+0000","last_became_peered":"2023-10-06T03:49:47.781282+0000","last_unstale":"2023-10-06T03:49:47.787750+0000","last_undegraded":"2023-10-06T03:49:47.787750+0000","last_fullsized":"2023-10-06T03:49:47.787750+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:09.275210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.799431+0000","last_change":"2023-10-06T03:49:46.773253+0000","last_active":"2023-10-06T03:49:47.799431+0000","last_peered":"2023-10-06T03:49:47.799431+0000","last_clean":"2023-10-06T03:49:47.799431+0000","last_became_active":"2023-10-06T03:49:46.772968+0000","last_became_peered":"2023-10-06T03:49:46.772968+0000","last_unstale":"2023-10-06T03:49:47.799431+0000","last_undegraded":"2023-10-06T03:49:47.799431+0000","last_fullsized":"2023-10-06T03:49:47.799431+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:57.942952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.797541+0000","last_change":"2023-10-06T03:49:47.789938+0000","last_active":"2023-10-06T03:49:47.797541+0000","last_peered":"2023-10-06T03:49:47.797541+0000","last_clean":"2023-10-06T03:49:47.797541+0000","last_became_active":"2023-10-06T03:49:47.789662+0000","last_became_peered":"2023-10-06T03:49:47.789662+0000","last_unstale":"2023-10-06T03:49:47.797541+0000","last_undegraded":"2023-10-06T03:49:47.797541+0000","last_fullsized":"2023-10-06T03:49:47.797541+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:17.548015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.802032+0000","last_change":"2023-10-06T03:49:46.781773+0000","last_active":"2023-10-06T03:49:47.802032+0000","last_peered":"2023-10-06T03:49:47.802032+0000","last_clean":"2023-10-06T03:49:47.802032+0000","last_became_active":"2023-10-06T03:49:46.781494+0000","last_became_peered":"2023-10-06T03:49:46.781494+0000","last_unstale":"2023-10-06T03:49:47.802032+0000","last_undegraded":"2023-10-06T03:49:47.802032+0000","last_fullsized":"2023-10-06T03:49:47.802032+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:03.287036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.779426+0000","last_change":"2023-10-06T03:49:47.774508+0000","last_active":"2023-10-06T03:49:47.779426+0000","last_peered":"2023-10-06T03:49:47.779426+0000","last_clean":"2023-10-06T03:49:47.779426+0000","last_became_active":"2023-10-06T03:49:47.774208+0000","last_became_peered":"2023-10-06T03:49:47.774208+0000","last_unstale":"2023-10-06T03:49:47.779426+0000","last_undegraded":"2023-10-06T03:49:47.779426+0000","last_fullsized":"2023-10-06T03:49:47.779426+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:11.424246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.805404+0000","last_change":"2023-10-06T03:49:46.785965+0000","last_active":"2023-10-06T03:49:47.805404+0000","last_peered":"2023-10-06T03:49:47.805404+0000","last_clean":"2023-10-06T03:49:47.805404+0000","last_became_active":"2023-10-06T03:49:46.785664+0000","last_became_peered":"2023-10-06T03:49:46.785664+0000","last_unstale":"2023-10-06T03:49:47.805404+0000","last_undegraded":"2023-10-06T03:49:47.805404+0000","last_fullsized":"2023-10-06T03:49:47.805404+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:29.978688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.801623+0000","last_change":"2023-10-06T03:49:47.799980+0000","last_active":"2023-10-06T03:49:47.801623+0000","last_peered":"2023-10-06T03:49:47.801623+0000","last_clean":"2023-10-06T03:49:47.801623+0000","last_became_active":"2023-10-06T03:49:47.799209+0000","last_became_peered":"2023-10-06T03:49:47.799209+0000","last_unstale":"2023-10-06T03:49:47.801623+0000","last_undegraded":"2023-10-06T03:49:47.801623+0000","last_fullsized":"2023-10-06T03:49:47.801623+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:34.061339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.766952+0000","last_change":"2023-10-06T03:49:46.784477+0000","last_active":"2023-10-06T03:49:47.766952+0000","last_peered":"2023-10-06T03:49:47.766952+0000","last_clean":"2023-10-06T03:49:47.766952+0000","last_became_active":"2023-10-06T03:49:46.784160+0000","last_became_peered":"2023-10-06T03:49:46.784160+0000","last_unstale":"2023-10-06T03:49:47.766952+0000","last_undegraded":"2023-10-06T03:49:47.766952+0000","last_fullsized":"2023-10-06T03:49:47.766952+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:16.540025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.797869+0000","last_change":"2023-10-06T03:49:47.790618+0000","last_active":"2023-10-06T03:49:47.797869+0000","last_peered":"2023-10-06T03:49:47.797869+0000","last_clean":"2023-10-06T03:49:47.797869+0000","last_became_active":"2023-10-06T03:49:47.790315+0000","last_became_peered":"2023-10-06T03:49:47.790315+0000","last_unstale":"2023-10-06T03:49:47.797869+0000","last_undegraded":"2023-10-06T03:49:47.797869+0000","last_fullsized":"2023-10-06T03:49:47.797869+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:20.377785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.772550+0000","last_change":"2023-10-06T03:49:46.786656+0000","last_active":"2023-10-06T03:49:47.772550+0000","last_peered":"2023-10-06T03:49:47.772550+0000","last_clean":"2023-10-06T03:49:47.772550+0000","last_became_active":"2023-10-06T03:49:46.786328+0000","last_became_peered":"2023-10-06T03:49:46.786328+0000","last_unstale":"2023-10-06T03:49:47.772550+0000","last_undegraded":"2023-10-06T03:49:47.772550+0000","last_fullsized":"2023-10-06T03:49:47.772550+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:03.287036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.785001+0000","last_change":"2023-10-06T03:49:47.773723+0000","last_active":"2023-10-06T03:49:47.785001+0000","last_peered":"2023-10-06T03:49:47.785001+0000","last_clean":"2023-10-06T03:49:47.785001+0000","last_became_active":"2023-10-06T03:49:47.773447+0000","last_became_peered":"2023-10-06T03:49:47.773447+0000","last_unstale":"2023-10-06T03:49:47.785001+0000","last_undegraded":"2023-10-06T03:49:47.785001+0000","last_fullsized":"2023-10-06T03:49:47.785001+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.494933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.774350+0000","last_change":"2023-10-06T03:49:46.783682+0000","last_active":"2023-10-06T03:49:47.774350+0000","last_peered":"2023-10-06T03:49:47.774350+0000","last_clean":"2023-10-06T03:49:47.774350+0000","last_became_active":"2023-10-06T03:49:46.783286+0000","last_became_peered":"2023-10-06T03:49:46.783286+0000","last_unstale":"2023-10-06T03:49:47.774350+0000","last_undegraded":"2023-10-06T03:49:47.774350+0000","last_fullsized":"2023-10-06T03:49:47.774350+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:10.416256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.788310+0000","last_change":"2023-10-06T03:49:47.787196+0000","last_active":"2023-10-06T03:49:47.788310+0000","last_peered":"2023-10-06T03:49:47.788310+0000","last_clean":"2023-10-06T03:49:47.788310+0000","last_became_active":"2023-10-06T03:49:47.786862+0000","last_became_peered":"2023-10-06T03:49:47.786862+0000","last_unstale":"2023-10-06T03:49:47.788310+0000","last_undegraded":"2023-10-06T03:49:47.788310+0000","last_fullsized":"2023-10-06T03:49:47.788310+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:35:02.011614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.767813+0000","last_change":"2023-10-06T03:49:46.788480+0000","last_active":"2023-10-06T03:49:47.767813+0000","last_peered":"2023-10-06T03:49:47.767813+0000","last_clean":"2023-10-06T03:49:47.767813+0000","last_became_active":"2023-10-06T03:49:46.788221+0000","last_became_peered":"2023-10-06T03:49:46.788221+0000","last_unstale":"2023-10-06T03:49:47.767813+0000","last_undegraded":"2023-10-06T03:49:47.767813+0000","last_fullsized":"2023-10-06T03:49:47.767813+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:00.609364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.781611+0000","last_change":"2023-10-06T03:49:47.778229+0000","last_active":"2023-10-06T03:49:47.781611+0000","last_peered":"2023-10-06T03:49:47.781611+0000","last_clean":"2023-10-06T03:49:47.781611+0000","last_became_active":"2023-10-06T03:49:47.777970+0000","last_became_peered":"2023-10-06T03:49:47.777970+0000","last_unstale":"2023-10-06T03:49:47.781611+0000","last_undegraded":"2023-10-06T03:49:47.781611+0000","last_fullsized":"2023-10-06T03:49:47.781611+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:08.886392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"106'3","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.807141+0000","last_change":"2023-10-06T03:49:46.787883+0000","last_active":"2023-10-06T03:49:47.807141+0000","last_peered":"2023-10-06T03:49:47.807141+0000","last_clean":"2023-10-06T03:49:47.807141+0000","last_became_active":"2023-10-06T03:49:46.787373+0000","last_became_peered":"2023-10-06T03:49:46.787373+0000","last_unstale":"2023-10-06T03:49:47.807141+0000","last_undegraded":"2023-10-06T03:49:47.807141+0000","last_fullsized":"2023-10-06T03:49:47.807141+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:00.967847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.797581+0000","last_change":"2023-10-06T03:49:47.790616+0000","last_active":"2023-10-06T03:49:47.797581+0000","last_peered":"2023-10-06T03:49:47.797581+0000","last_clean":"2023-10-06T03:49:47.797581+0000","last_became_active":"2023-10-06T03:49:47.790327+0000","last_became_peered":"2023-10-06T03:49:47.790327+0000","last_unstale":"2023-10-06T03:49:47.797581+0000","last_undegraded":"2023-10-06T03:49:47.797581+0000","last_fullsized":"2023-10-06T03:49:47.797581+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:32.783620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"106'6","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.809894+0000","last_change":"2023-10-06T03:49:46.780674+0000","last_active":"2023-10-06T03:49:47.809894+0000","last_peered":"2023-10-06T03:49:47.809894+0000","last_clean":"2023-10-06T03:49:47.809894+0000","last_became_active":"2023-10-06T03:49:46.780331+0000","last_became_peered":"2023-10-06T03:49:46.780331+0000","last_unstale":"2023-10-06T03:49:47.809894+0000","last_undegraded":"2023-10-06T03:49:47.809894+0000","last_fullsized":"2023-10-06T03:49:47.809894+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:51.351322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.792605+0000","last_change":"2023-10-06T03:49:47.782444+0000","last_active":"2023-10-06T03:49:47.792605+0000","last_peered":"2023-10-06T03:49:47.792605+0000","last_clean":"2023-10-06T03:49:47.792605+0000","last_became_active":"2023-10-06T03:49:47.781698+0000","last_became_peered":"2023-10-06T03:49:47.781698+0000","last_unstale":"2023-10-06T03:49:47.792605+0000","last_undegraded":"2023-10-06T03:49:47.792605+0000","last_fullsized":"2023-10-06T03:49:47.792605+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:16.509044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.801772+0000","last_change":"2023-10-06T03:49:46.786444+0000","last_active":"2023-10-06T03:49:47.801772+0000","last_peered":"2023-10-06T03:49:47.801772+0000","last_clean":"2023-10-06T03:49:47.801772+0000","last_became_active":"2023-10-06T03:49:46.786240+0000","last_became_peered":"2023-10-06T03:49:46.786240+0000","last_unstale":"2023-10-06T03:49:47.801772+0000","last_undegraded":"2023-10-06T03:49:47.801772+0000","last_fullsized":"2023-10-06T03:49:47.801772+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:31.233223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.778517+0000","last_change":"2023-10-06T03:49:47.772427+0000","last_active":"2023-10-06T03:49:47.778517+0000","last_peered":"2023-10-06T03:49:47.778517+0000","last_clean":"2023-10-06T03:49:47.778517+0000","last_became_active":"2023-10-06T03:49:47.772110+0000","last_became_peered":"2023-10-06T03:49:47.772110+0000","last_unstale":"2023-10-06T03:49:47.778517+0000","last_undegraded":"2023-10-06T03:49:47.778517+0000","last_fullsized":"2023-10-06T03:49:47.778517+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:38.218445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.771191+0000","last_change":"2023-10-06T03:49:46.791774+0000","last_active":"2023-10-06T03:49:47.771191+0000","last_peered":"2023-10-06T03:49:47.771191+0000","last_clean":"2023-10-06T03:49:47.771191+0000","last_became_active":"2023-10-06T03:49:46.790979+0000","last_became_peered":"2023-10-06T03:49:46.790979+0000","last_unstale":"2023-10-06T03:49:47.771191+0000","last_undegraded":"2023-10-06T03:49:47.771191+0000","last_fullsized":"2023-10-06T03:49:47.771191+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:53.042283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798325+0000","last_change":"2023-10-06T03:49:47.796924+0000","last_active":"2023-10-06T03:49:47.798325+0000","last_peered":"2023-10-06T03:49:47.798325+0000","last_clean":"2023-10-06T03:49:47.798325+0000","last_became_active":"2023-10-06T03:49:47.796523+0000","last_became_peered":"2023-10-06T03:49:47.796523+0000","last_unstale":"2023-10-06T03:49:47.798325+0000","last_undegraded":"2023-10-06T03:49:47.798325+0000","last_fullsized":"2023-10-06T03:49:47.798325+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:40.976185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.788194+0000","last_change":"2023-10-06T03:49:46.780647+0000","last_active":"2023-10-06T03:49:47.788194+0000","last_peered":"2023-10-06T03:49:47.788194+0000","last_clean":"2023-10-06T03:49:47.788194+0000","last_became_active":"2023-10-06T03:49:46.780391+0000","last_became_peered":"2023-10-06T03:49:46.780391+0000","last_unstale":"2023-10-06T03:49:47.788194+0000","last_undegraded":"2023-10-06T03:49:47.788194+0000","last_fullsized":"2023-10-06T03:49:47.788194+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:01.394824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.785782+0000","last_change":"2023-10-06T03:49:47.783551+0000","last_active":"2023-10-06T03:49:47.785782+0000","last_peered":"2023-10-06T03:49:47.785782+0000","last_clean":"2023-10-06T03:49:47.785782+0000","last_became_active":"2023-10-06T03:49:47.783250+0000","last_became_peered":"2023-10-06T03:49:47.783250+0000","last_unstale":"2023-10-06T03:49:47.785782+0000","last_undegraded":"2023-10-06T03:49:47.785782+0000","last_fullsized":"2023-10-06T03:49:47.785782+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:06.680989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"106'8","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.808583+0000","last_change":"2023-10-06T03:49:46.784798+0000","last_active":"2023-10-06T03:49:47.808583+0000","last_peered":"2023-10-06T03:49:47.808583+0000","last_clean":"2023-10-06T03:49:47.808583+0000","last_became_active":"2023-10-06T03:49:46.784323+0000","last_became_peered":"2023-10-06T03:49:46.784323+0000","last_unstale":"2023-10-06T03:49:47.808583+0000","last_undegraded":"2023-10-06T03:49:47.808583+0000","last_fullsized":"2023-10-06T03:49:47.808583+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:01.103063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.780278+0000","last_change":"2023-10-06T03:49:47.776050+0000","last_active":"2023-10-06T03:49:47.780278+0000","last_peered":"2023-10-06T03:49:47.780278+0000","last_clean":"2023-10-06T03:49:47.780278+0000","last_became_active":"2023-10-06T03:49:47.775755+0000","last_became_peered":"2023-10-06T03:49:47.775755+0000","last_unstale":"2023-10-06T03:49:47.780278+0000","last_undegraded":"2023-10-06T03:49:47.780278+0000","last_fullsized":"2023-10-06T03:49:47.780278+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:50.192073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.797814+0000","last_change":"2023-10-06T03:49:46.780843+0000","last_active":"2023-10-06T03:49:47.797814+0000","last_peered":"2023-10-06T03:49:47.797814+0000","last_clean":"2023-10-06T03:49:47.797814+0000","last_became_active":"2023-10-06T03:49:46.780292+0000","last_became_peered":"2023-10-06T03:49:46.780292+0000","last_unstale":"2023-10-06T03:49:47.797814+0000","last_undegraded":"2023-10-06T03:49:47.797814+0000","last_fullsized":"2023-10-06T03:49:47.797814+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:09.526031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.794442+0000","last_change":"2023-10-06T03:49:47.791544+0000","last_active":"2023-10-06T03:49:47.794442+0000","last_peered":"2023-10-06T03:49:47.794442+0000","last_clean":"2023-10-06T03:49:47.794442+0000","last_became_active":"2023-10-06T03:49:47.791269+0000","last_became_peered":"2023-10-06T03:49:47.791269+0000","last_unstale":"2023-10-06T03:49:47.794442+0000","last_undegraded":"2023-10-06T03:49:47.794442+0000","last_fullsized":"2023-10-06T03:49:47.794442+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:41.003218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.770682+0000","last_change":"2023-10-06T03:49:46.778038+0000","last_active":"2023-10-06T03:49:47.770682+0000","last_peered":"2023-10-06T03:49:47.770682+0000","last_clean":"2023-10-06T03:49:47.770682+0000","last_became_active":"2023-10-06T03:49:46.777711+0000","last_became_peered":"2023-10-06T03:49:46.777711+0000","last_unstale":"2023-10-06T03:49:47.770682+0000","last_undegraded":"2023-10-06T03:49:47.770682+0000","last_fullsized":"2023-10-06T03:49:47.770682+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:08.471124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.781486+0000","last_change":"2023-10-06T03:49:47.777345+0000","last_active":"2023-10-06T03:49:47.781486+0000","last_peered":"2023-10-06T03:49:47.781486+0000","last_clean":"2023-10-06T03:49:47.781486+0000","last_became_active":"2023-10-06T03:49:47.777054+0000","last_became_peered":"2023-10-06T03:49:47.777054+0000","last_unstale":"2023-10-06T03:49:47.781486+0000","last_undegraded":"2023-10-06T03:49:47.781486+0000","last_fullsized":"2023-10-06T03:49:47.781486+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:25.358460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.770123+0000","last_change":"2023-10-06T03:49:46.787882+0000","last_active":"2023-10-06T03:49:47.770123+0000","last_peered":"2023-10-06T03:49:47.770123+0000","last_clean":"2023-10-06T03:49:47.770123+0000","last_became_active":"2023-10-06T03:49:46.787230+0000","last_became_peered":"2023-10-06T03:49:46.787230+0000","last_unstale":"2023-10-06T03:49:47.770123+0000","last_undegraded":"2023-10-06T03:49:47.770123+0000","last_fullsized":"2023-10-06T03:49:47.770123+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:31.233223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786610+0000","last_change":"2023-10-06T03:49:47.784850+0000","last_active":"2023-10-06T03:49:47.786610+0000","last_peered":"2023-10-06T03:49:47.786610+0000","last_clean":"2023-10-06T03:49:47.786610+0000","last_became_active":"2023-10-06T03:49:47.784466+0000","last_became_peered":"2023-10-06T03:49:47.784466+0000","last_unstale":"2023-10-06T03:49:47.786610+0000","last_undegraded":"2023-10-06T03:49:47.786610+0000","last_fullsized":"2023-10-06T03:49:47.786610+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:27.622375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.767290+0000","last_change":"2023-10-06T03:49:46.782877+0000","last_active":"2023-10-06T03:49:47.767290+0000","last_peered":"2023-10-06T03:49:47.767290+0000","last_clean":"2023-10-06T03:49:47.767290+0000","last_became_active":"2023-10-06T03:49:46.781926+0000","last_became_peered":"2023-10-06T03:49:46.781926+0000","last_unstale":"2023-10-06T03:49:47.767290+0000","last_undegraded":"2023-10-06T03:49:47.767290+0000","last_fullsized":"2023-10-06T03:49:47.767290+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:34.375175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.779389+0000","last_change":"2023-10-06T03:49:47.776450+0000","last_active":"2023-10-06T03:49:47.779389+0000","last_peered":"2023-10-06T03:49:47.779389+0000","last_clean":"2023-10-06T03:49:47.779389+0000","last_became_active":"2023-10-06T03:49:47.776169+0000","last_became_peered":"2023-10-06T03:49:47.776169+0000","last_unstale":"2023-10-06T03:49:47.779389+0000","last_undegraded":"2023-10-06T03:49:47.779389+0000","last_fullsized":"2023-10-06T03:49:47.779389+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:27.190523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.771912+0000","last_change":"2023-10-06T03:49:46.792516+0000","last_active":"2023-10-06T03:49:47.771912+0000","last_peered":"2023-10-06T03:49:47.771912+0000","last_clean":"2023-10-06T03:49:47.771912+0000","last_became_active":"2023-10-06T03:49:46.792012+0000","last_became_peered":"2023-10-06T03:49:46.792012+0000","last_unstale":"2023-10-06T03:49:47.771912+0000","last_undegraded":"2023-10-06T03:49:47.771912+0000","last_fullsized":"2023-10-06T03:49:47.771912+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:06.966504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798833+0000","last_change":"2023-10-06T03:49:47.796697+0000","last_active":"2023-10-06T03:49:47.798833+0000","last_peered":"2023-10-06T03:49:47.798833+0000","last_clean":"2023-10-06T03:49:47.798833+0000","last_became_active":"2023-10-06T03:49:47.796440+0000","last_became_peered":"2023-10-06T03:49:47.796440+0000","last_unstale":"2023-10-06T03:49:47.798833+0000","last_undegraded":"2023-10-06T03:49:47.798833+0000","last_fullsized":"2023-10-06T03:49:47.798833+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:19.438975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.772942+0000","last_change":"2023-10-06T03:49:46.784342+0000","last_active":"2023-10-06T03:49:47.772942+0000","last_peered":"2023-10-06T03:49:47.772942+0000","last_clean":"2023-10-06T03:49:47.772942+0000","last_became_active":"2023-10-06T03:49:46.784137+0000","last_became_peered":"2023-10-06T03:49:46.784137+0000","last_unstale":"2023-10-06T03:49:47.772942+0000","last_undegraded":"2023-10-06T03:49:47.772942+0000","last_fullsized":"2023-10-06T03:49:47.772942+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:38.870207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.800176+0000","last_change":"2023-10-06T03:49:47.799422+0000","last_active":"2023-10-06T03:49:47.800176+0000","last_peered":"2023-10-06T03:49:47.800176+0000","last_clean":"2023-10-06T03:49:47.800176+0000","last_became_active":"2023-10-06T03:49:47.799039+0000","last_became_peered":"2023-10-06T03:49:47.799039+0000","last_unstale":"2023-10-06T03:49:47.800176+0000","last_undegraded":"2023-10-06T03:49:47.800176+0000","last_fullsized":"2023-10-06T03:49:47.800176+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:37.233664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.767473+0000","last_change":"2023-10-06T03:49:46.783252+0000","last_active":"2023-10-06T03:49:47.767473+0000","last_peered":"2023-10-06T03:49:47.767473+0000","last_clean":"2023-10-06T03:49:47.767473+0000","last_became_active":"2023-10-06T03:49:46.782981+0000","last_became_peered":"2023-10-06T03:49:46.782981+0000","last_unstale":"2023-10-06T03:49:47.767473+0000","last_undegraded":"2023-10-06T03:49:47.767473+0000","last_fullsized":"2023-10-06T03:49:47.767473+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:36.224710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786921+0000","last_change":"2023-10-06T03:49:47.782977+0000","last_active":"2023-10-06T03:49:47.786921+0000","last_peered":"2023-10-06T03:49:47.786921+0000","last_clean":"2023-10-06T03:49:47.786921+0000","last_became_active":"2023-10-06T03:49:47.782598+0000","last_became_peered":"2023-10-06T03:49:47.782598+0000","last_unstale":"2023-10-06T03:49:47.786921+0000","last_undegraded":"2023-10-06T03:49:47.786921+0000","last_fullsized":"2023-10-06T03:49:47.786921+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:28.965158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.772669+0000","last_change":"2023-10-06T03:49:46.785971+0000","last_active":"2023-10-06T03:49:47.772669+0000","last_peered":"2023-10-06T03:49:47.772669+0000","last_clean":"2023-10-06T03:49:47.772669+0000","last_became_active":"2023-10-06T03:49:46.785686+0000","last_became_peered":"2023-10-06T03:49:46.785686+0000","last_unstale":"2023-10-06T03:49:47.772669+0000","last_undegraded":"2023-10-06T03:49:47.772669+0000","last_fullsized":"2023-10-06T03:49:47.772669+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:51.089069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.795334+0000","last_change":"2023-10-06T03:49:47.794256+0000","last_active":"2023-10-06T03:49:47.795334+0000","last_peered":"2023-10-06T03:49:47.795334+0000","last_clean":"2023-10-06T03:49:47.795334+0000","last_became_active":"2023-10-06T03:49:47.793837+0000","last_became_peered":"2023-10-06T03:49:47.793837+0000","last_unstale":"2023-10-06T03:49:47.795334+0000","last_undegraded":"2023-10-06T03:49:47.795334+0000","last_fullsized":"2023-10-06T03:49:47.795334+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:16.509044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.772566+0000","last_change":"2023-10-06T03:49:46.781425+0000","last_active":"2023-10-06T03:49:47.772566+0000","last_peered":"2023-10-06T03:49:47.772566+0000","last_clean":"2023-10-06T03:49:47.772566+0000","last_became_active":"2023-10-06T03:49:46.781050+0000","last_became_peered":"2023-10-06T03:49:46.781050+0000","last_unstale":"2023-10-06T03:49:47.772566+0000","last_undegraded":"2023-10-06T03:49:47.772566+0000","last_fullsized":"2023-10-06T03:49:47.772566+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:01.394824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.769873+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:49:47.769873+0000","last_peered":"2023-10-06T03:49:47.769873+0000","last_clean":"2023-10-06T03:49:47.769873+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:49:47.769873+0000","last_undegraded":"2023-10-06T03:49:47.769873+0000","last_fullsized":"2023-10-06T03:49:47.769873+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.781247+0000","last_change":"2023-10-06T03:49:47.777001+0000","last_active":"2023-10-06T03:49:47.781247+0000","last_peered":"2023-10-06T03:49:47.781247+0000","last_clean":"2023-10-06T03:49:47.781247+0000","last_became_active":"2023-10-06T03:49:47.776722+0000","last_became_peered":"2023-10-06T03:49:47.776722+0000","last_unstale":"2023-10-06T03:49:47.781247+0000","last_undegraded":"2023-10-06T03:49:47.781247+0000","last_fullsized":"2023-10-06T03:49:47.781247+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:38.223180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.767735+0000","last_change":"2023-10-06T03:49:46.782221+0000","last_active":"2023-10-06T03:49:47.767735+0000","last_peered":"2023-10-06T03:49:47.767735+0000","last_clean":"2023-10-06T03:49:47.767735+0000","last_became_active":"2023-10-06T03:49:46.781917+0000","last_became_peered":"2023-10-06T03:49:46.781917+0000","last_unstale":"2023-10-06T03:49:47.767735+0000","last_undegraded":"2023-10-06T03:49:47.767735+0000","last_fullsized":"2023-10-06T03:49:47.767735+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:40.482430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.765432+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:49:47.765432+0000","last_peered":"2023-10-06T03:49:47.765432+0000","last_clean":"2023-10-06T03:49:47.765432+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:49:47.765432+0000","last_undegraded":"2023-10-06T03:49:47.765432+0000","last_fullsized":"2023-10-06T03:49:47.765432+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.791017+0000","last_change":"2023-10-06T03:49:47.788586+0000","last_active":"2023-10-06T03:49:47.791017+0000","last_peered":"2023-10-06T03:49:47.791017+0000","last_clean":"2023-10-06T03:49:47.791017+0000","last_became_active":"2023-10-06T03:49:47.788322+0000","last_became_peered":"2023-10-06T03:49:47.788322+0000","last_unstale":"2023-10-06T03:49:47.791017+0000","last_undegraded":"2023-10-06T03:49:47.791017+0000","last_fullsized":"2023-10-06T03:49:47.791017+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:17.408086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.794515+0000","last_change":"2023-10-06T03:49:46.775600+0000","last_active":"2023-10-06T03:49:47.794515+0000","last_peered":"2023-10-06T03:49:47.794515+0000","last_clean":"2023-10-06T03:49:47.794515+0000","last_became_active":"2023-10-06T03:49:46.774748+0000","last_became_peered":"2023-10-06T03:49:46.774748+0000","last_unstale":"2023-10-06T03:49:47.794515+0000","last_undegraded":"2023-10-06T03:49:47.794515+0000","last_fullsized":"2023-10-06T03:49:47.794515+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:14.411208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786086+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:49:47.786086+0000","last_peered":"2023-10-06T03:49:47.786086+0000","last_clean":"2023-10-06T03:49:47.786086+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:49:47.786086+0000","last_undegraded":"2023-10-06T03:49:47.786086+0000","last_fullsized":"2023-10-06T03:49:47.786086+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"96'205","reported_seq":551,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.772773+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:49:47.772773+0000","last_peered":"2023-10-06T03:49:47.772773+0000","last_clean":"2023-10-06T03:49:47.772773+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:49:47.772773+0000","last_undegraded":"2023-10-06T03:49:47.772773+0000","last_fullsized":"2023-10-06T03:49:47.772773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.780359+0000","last_change":"2023-10-06T03:49:47.776643+0000","last_active":"2023-10-06T03:49:47.780359+0000","last_peered":"2023-10-06T03:49:47.780359+0000","last_clean":"2023-10-06T03:49:47.780359+0000","last_became_active":"2023-10-06T03:49:47.776248+0000","last_became_peered":"2023-10-06T03:49:47.776248+0000","last_unstale":"2023-10-06T03:49:47.780359+0000","last_undegraded":"2023-10-06T03:49:47.780359+0000","last_fullsized":"2023-10-06T03:49:47.780359+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:48.485492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.797380+0000","last_change":"2023-10-06T03:49:46.780848+0000","last_active":"2023-10-06T03:49:47.797380+0000","last_peered":"2023-10-06T03:49:47.797380+0000","last_clean":"2023-10-06T03:49:47.797380+0000","last_became_active":"2023-10-06T03:49:46.780339+0000","last_became_peered":"2023-10-06T03:49:46.780339+0000","last_unstale":"2023-10-06T03:49:47.797380+0000","last_undegraded":"2023-10-06T03:49:47.797380+0000","last_fullsized":"2023-10-06T03:49:47.797380+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:59.899446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.772031+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:49:47.772031+0000","last_peered":"2023-10-06T03:49:47.772031+0000","last_clean":"2023-10-06T03:49:47.772031+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:49:47.772031+0000","last_undegraded":"2023-10-06T03:49:47.772031+0000","last_fullsized":"2023-10-06T03:49:47.772031+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.788814+0000","last_change":"2023-10-06T03:49:47.788071+0000","last_active":"2023-10-06T03:49:47.788814+0000","last_peered":"2023-10-06T03:49:47.788814+0000","last_clean":"2023-10-06T03:49:47.788814+0000","last_became_active":"2023-10-06T03:49:47.787748+0000","last_became_peered":"2023-10-06T03:49:47.787748+0000","last_unstale":"2023-10-06T03:49:47.788814+0000","last_undegraded":"2023-10-06T03:49:47.788814+0000","last_fullsized":"2023-10-06T03:49:47.788814+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:09.919684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.785417+0000","last_change":"2023-10-06T03:49:46.789229+0000","last_active":"2023-10-06T03:49:47.785417+0000","last_peered":"2023-10-06T03:49:47.785417+0000","last_clean":"2023-10-06T03:49:47.785417+0000","last_became_active":"2023-10-06T03:49:46.788886+0000","last_became_peered":"2023-10-06T03:49:46.788886+0000","last_unstale":"2023-10-06T03:49:47.785417+0000","last_undegraded":"2023-10-06T03:49:47.785417+0000","last_fullsized":"2023-10-06T03:49:47.785417+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:45.152283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":90,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.772864+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:49:47.772864+0000","last_peered":"2023-10-06T03:49:47.772864+0000","last_clean":"2023-10-06T03:49:47.772864+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:49:47.772864+0000","last_undegraded":"2023-10-06T03:49:47.772864+0000","last_fullsized":"2023-10-06T03:49:47.772864+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.783138+0000","last_change":"2023-10-06T03:49:47.777651+0000","last_active":"2023-10-06T03:49:47.783138+0000","last_peered":"2023-10-06T03:49:47.783138+0000","last_clean":"2023-10-06T03:49:47.783138+0000","last_became_active":"2023-10-06T03:49:47.775150+0000","last_became_peered":"2023-10-06T03:49:47.775150+0000","last_unstale":"2023-10-06T03:49:47.783138+0000","last_undegraded":"2023-10-06T03:49:47.783138+0000","last_fullsized":"2023-10-06T03:49:47.783138+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:48.866890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.806301+0000","last_change":"2023-10-06T03:49:46.777556+0000","last_active":"2023-10-06T03:49:47.806301+0000","last_peered":"2023-10-06T03:49:47.806301+0000","last_clean":"2023-10-06T03:49:47.806301+0000","last_became_active":"2023-10-06T03:49:46.777193+0000","last_became_peered":"2023-10-06T03:49:46.777193+0000","last_unstale":"2023-10-06T03:49:47.806301+0000","last_undegraded":"2023-10-06T03:49:47.806301+0000","last_fullsized":"2023-10-06T03:49:47.806301+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:59.899446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.773892+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:49:47.773892+0000","last_peered":"2023-10-06T03:49:47.773892+0000","last_clean":"2023-10-06T03:49:47.773892+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:49:47.773892+0000","last_undegraded":"2023-10-06T03:49:47.773892+0000","last_fullsized":"2023-10-06T03:49:47.773892+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786456+0000","last_change":"2023-10-06T03:49:47.783536+0000","last_active":"2023-10-06T03:49:47.786456+0000","last_peered":"2023-10-06T03:49:47.786456+0000","last_clean":"2023-10-06T03:49:47.786456+0000","last_became_active":"2023-10-06T03:49:47.783261+0000","last_became_peered":"2023-10-06T03:49:47.783261+0000","last_unstale":"2023-10-06T03:49:47.786456+0000","last_undegraded":"2023-10-06T03:49:47.786456+0000","last_fullsized":"2023-10-06T03:49:47.786456+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:10.207499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.767192+0000","last_change":"2023-10-06T03:49:46.786749+0000","last_active":"2023-10-06T03:49:47.767192+0000","last_peered":"2023-10-06T03:49:47.767192+0000","last_clean":"2023-10-06T03:49:47.767192+0000","last_became_active":"2023-10-06T03:49:46.786351+0000","last_became_peered":"2023-10-06T03:49:46.786351+0000","last_unstale":"2023-10-06T03:49:47.767192+0000","last_undegraded":"2023-10-06T03:49:47.767192+0000","last_fullsized":"2023-10-06T03:49:47.767192+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:47.190140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.767133+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:49:47.767133+0000","last_peered":"2023-10-06T03:49:47.767133+0000","last_clean":"2023-10-06T03:49:47.767133+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:49:47.767133+0000","last_undegraded":"2023-10-06T03:49:47.767133+0000","last_fullsized":"2023-10-06T03:49:47.767133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.787304+0000","last_change":"2023-10-06T03:49:47.785019+0000","last_active":"2023-10-06T03:49:47.787304+0000","last_peered":"2023-10-06T03:49:47.787304+0000","last_clean":"2023-10-06T03:49:47.787304+0000","last_became_active":"2023-10-06T03:49:47.784599+0000","last_became_peered":"2023-10-06T03:49:47.784599+0000","last_unstale":"2023-10-06T03:49:47.787304+0000","last_undegraded":"2023-10-06T03:49:47.787304+0000","last_fullsized":"2023-10-06T03:49:47.787304+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:39.982048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.804948+0000","last_change":"2023-10-06T03:49:46.775487+0000","last_active":"2023-10-06T03:49:47.804948+0000","last_peered":"2023-10-06T03:49:47.804948+0000","last_clean":"2023-10-06T03:49:47.804948+0000","last_became_active":"2023-10-06T03:49:46.775203+0000","last_became_peered":"2023-10-06T03:49:46.775203+0000","last_unstale":"2023-10-06T03:49:47.804948+0000","last_undegraded":"2023-10-06T03:49:47.804948+0000","last_fullsized":"2023-10-06T03:49:47.804948+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:47.477502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.767704+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:49:47.767704+0000","last_peered":"2023-10-06T03:49:47.767704+0000","last_clean":"2023-10-06T03:49:47.767704+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:49:47.767704+0000","last_undegraded":"2023-10-06T03:49:47.767704+0000","last_fullsized":"2023-10-06T03:49:47.767704+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575045,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311432,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060408,"statfs":{"total":96636764160,"available":96317857792,"internally_reserved":0,"allocated":1900544,"data_stored":1253096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.66800000000000004,"15min":0.69699999999999995},"min":{"1min":0.377,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.91300000000000003,"5min":1.0029999999999999,"15min":1.6100000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.66500000000000004,"15min":0.70399999999999996},"min":{"1min":0.46000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88700000000000001,"5min":1.02,"15min":1.5649999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.70599999999999996,"15min":0.749},"min":{"1min":0.41199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.95499999999999996,"5min":1.038,"15min":1.5469999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.72999999999999998,"15min":0.76300000000000001},"min":{"1min":0.49199999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96199999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.67100000000000004,"15min":0.69699999999999995},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82599999999999996,"5min":0.97499999999999998,"15min":1.5900000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.73699999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91900000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.70499999999999996,"15min":0.72399999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39500000000000002},"max":{"1min":0.81799999999999995,"5min":1.016,"15min":1.6279999999999999},"last":0.63},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72999999999999998,"15min":0.74199999999999999},"min":{"1min":0.52900000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.0409999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.0049999999999999},"last":0.754},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63,"15min":0.622},"min":{"1min":0.45700000000000002,"5min":0.41299999999999998,"15min":0.39400000000000002},"max":{"1min":0.90600000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.57799999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.069},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.69199999999999995,"15min":0.68000000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.436},"max":{"1min":1.0640000000000001,"5min":1.105,"15min":1.105},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.746,"15min":0.72599999999999998},"min":{"1min":0.47499999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.1220000000000001,"5min":1.181,"15min":1.181},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71699999999999997,"15min":0.70399999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.73899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575046,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306772,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065068,"statfs":{"total":96636764160,"available":96322629632,"internally_reserved":0,"allocated":1323008,"data_stored":665762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68700000000000006,"15min":0.70099999999999996},"min":{"1min":0.39400000000000002,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.47399999999999998,"5min":0.41299999999999998,"15min":0.39500000000000002},"max":{"1min":0.84999999999999998,"5min":0.98599999999999999,"15min":1.0069999999999999},"last":0.67300000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.41599999999999998,"5min":0.36499999999999999,"15min":0.33800000000000002},"max":{"1min":0.83199999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71199999999999997,"15min":0.71499999999999997},"min":{"1min":0.52500000000000002,"5min":0.441,"15min":0.35899999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73699999999999999,"15min":0.74299999999999999},"min":{"1min":0.54900000000000004,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.71699999999999997,"15min":0.73399999999999999},"min":{"1min":0.42099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78500000000000003,"5min":1.0209999999999999,"15min":1.0409999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.44600000000000001},"max":{"1min":0.83299999999999996,"5min":1.01,"15min":1.024},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.56999999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.097,"5min":1.097,"15min":1.1639999999999999},"last":0.624}]},{"osd":4,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.34799999999999998},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63700000000000001,"15min":0.63200000000000001},"min":{"1min":0.47299999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.67900000000000005,"15min":0.66400000000000003},"min":{"1min":0.59999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0489999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.503,"5min":0.45900000000000002,"15min":0.41199999999999998},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.753},{"interface":"front","average":{"1min":0.754,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.54000000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.77300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575045,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307336,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064504,"statfs":{"total":96636764160,"available":96322052096,"internally_reserved":0,"allocated":1900544,"data_stored":1254088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47199999999999998,"15min":0.48599999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.024,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47999999999999998,"15min":0.49299999999999999},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.309},"max":{"1min":0.65600000000000003,"5min":0.72699999999999998,"15min":0.84799999999999998},"last":0.375},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54600000000000004,"15min":0.55500000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.626,"5min":0.77300000000000002,"15min":0.94099999999999995},"last":0.47399999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59299999999999997,"15min":0.60299999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73099999999999998,"5min":0.86699999999999999,"15min":0.91700000000000004},"last":0.622},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.53000000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.32500000000000001},"max":{"1min":0.74299999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.498}]},{"osd":4,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.75600000000000001,"15min":0.76100000000000001},"min":{"1min":0.496,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0289999999999999,"5min":1.0389999999999999,"15min":1.1699999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.78400000000000003,"15min":0.79900000000000004},"min":{"1min":0.52500000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.98899999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.77800000000000002,"15min":0.78900000000000003},"min":{"1min":0.53800000000000003,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":1.0089999999999999,"5min":1.294,"15min":1.294},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.79200000000000004,"15min":0.80400000000000005},"min":{"1min":0.55000000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.008,"5min":1.224,"15min":1.224},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78600000000000003,"15min":0.80600000000000005},"min":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.077,"5min":1.26,"15min":1.26},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.79800000000000004,"15min":0.81799999999999995},"min":{"1min":0.59599999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0720000000000001,"5min":1.143,"15min":1.224},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81899999999999995,"15min":0.83699999999999997},"min":{"1min":0.59299999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.038,"5min":1.272,"15min":1.272},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.59999999999999998,"5min":0.53700000000000003,"15min":0.51200000000000001},"max":{"1min":1.0009999999999999,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.64700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575047,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306772,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065068,"statfs":{"total":96636764160,"available":96322629632,"internally_reserved":0,"allocated":1323008,"data_stored":664886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.47799999999999998},"min":{"1min":0.33800000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.72599999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.495,"5min":0.496,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.70999999999999996,"5min":0.78500000000000003,"15min":0.83599999999999997},"last":0.58699999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.54600000000000004},"min":{"1min":0.40000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.69499999999999995,"5min":0.80100000000000005,"15min":1.1299999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52900000000000003,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80100000000000005,"5min":0.83399999999999996,"15min":0.89400000000000002},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.28399999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.93200000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59599999999999997,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.378,"15min":0.378},"max":{"1min":0.91800000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.80000000000000004,"15min":0.82999999999999996},"min":{"1min":0.56200000000000006,"5min":0.54200000000000004,"15min":0.49399999999999999},"max":{"1min":1.0189999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.79100000000000004,"15min":0.82299999999999995},"min":{"1min":0.56200000000000006,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1279999999999999,"5min":1.306,"15min":1.3400000000000001},"last":0.96499999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.58299999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0629999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.749,"5min":0.76800000000000002,"15min":0.80500000000000005},"min":{"1min":0.53500000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98599999999999999,"5min":1.1499999999999999,"15min":1.6559999999999999},"last":1.194}]},{"osd":6,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.75800000000000001,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98399999999999999,"5min":1.137,"15min":1.726},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.79700000000000004,"15min":0.83699999999999997},"min":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.98899999999999999,"5min":1.167,"15min":1.843},"last":1.0580000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80300000000000005,"15min":0.82199999999999995},"min":{"1min":0.59799999999999998,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.1060000000000001,"5min":1.302,"15min":1.302},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.78700000000000003,"15min":0.83199999999999996},"min":{"1min":0.59599999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.0569999999999999,"5min":1.2450000000000001,"15min":1.7649999999999999},"last":0.875}]}]},{"osd":2,"up_from":13,"seq":55834575046,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310844,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060996,"statfs":{"total":96636764160,"available":96318459904,"internally_reserved":0,"allocated":1298432,"data_stored":662672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46899999999999997,"15min":0.47199999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.28000000000000003},"max":{"1min":0.66700000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51400000000000001,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.89300000000000002},"last":0.435}]},{"osd":1,"last update":"Fri Oct 6 03:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.621,"5min":0.73799999999999999,"15min":0.90000000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.55300000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.94899999999999995},"last":0.46600000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57999999999999996,"15min":0.58899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79000000000000004,"5min":1.78,"15min":1.78},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.39100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.96499999999999997},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80800000000000005,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.51400000000000001,"15min":0.439},"max":{"1min":1,"5min":1.0649999999999999,"15min":1.5620000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82699999999999996,"15min":0.84199999999999997},"min":{"1min":0.53100000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.05,"5min":1.1279999999999999,"15min":1.2270000000000001},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81899999999999995,"15min":0.83899999999999997},"min":{"1min":0.54900000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.999,"5min":1.169,"15min":1.3},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.85399999999999998},"min":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.51700000000000002},"max":{"1min":0.98599999999999999,"5min":1.855,"15min":1.855},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81899999999999995,"15min":0.83799999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":1.907,"15min":1.907},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.52600000000000002},"max":{"1min":0.98299999999999998,"5min":1.1080000000000001,"15min":1.4750000000000001},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.83599999999999997,"15min":0.84699999999999998},"min":{"1min":0.59499999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.022,"5min":1.4379999999999999,"15min":1.4990000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.85299999999999998,"15min":0.871},"min":{"1min":0.54800000000000004,"5min":0.53100000000000003,"15min":0.497},"max":{"1min":1.0860000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.64300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575046,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306752,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065088,"statfs":{"total":96636764160,"available":96322650112,"internally_reserved":0,"allocated":1302528,"data_stored":663650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45600000000000002,"15min":0.45900000000000002},"min":{"1min":0.30599999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60399999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48099999999999998,"15min":0.498},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":0.79200000000000004,"5min":1.04,"15min":1.1699999999999999},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47699999999999998,"15min":0.48199999999999998},"min":{"1min":0.32200000000000001,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.65800000000000003,"5min":0.68799999999999994,"15min":0.79500000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.40600000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.64300000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.82099999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80700000000000005,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58199999999999996,"15min":0.60299999999999998},"min":{"1min":0.47699999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73299999999999998,"5min":0.81999999999999995,"15min":0.97899999999999998},"last":0.74199999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89900000000000002,"5min":1.028,"15min":1.4099999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78300000000000003,"15min":0.82499999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.92700000000000005,"5min":1.0469999999999999,"15min":1.377},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75,"15min":0.78700000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97399999999999998,"5min":1.05,"15min":1.256},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74399999999999999,"15min":0.78200000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.49099999999999999},"max":{"1min":0.88900000000000001,"5min":0.99199999999999999,"15min":1.2030000000000001},"last":0.90200000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77400000000000002,"15min":0.81000000000000005},"min":{"1min":0.56999999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"max":{"1min":0.85199999999999998,"5min":1.089,"15min":1.323},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.80500000000000005,"15min":0.84499999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.073,"5min":1.1040000000000001,"15min":1.2849999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.77300000000000002,"15min":0.81000000000000005},"min":{"1min":0.55900000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.94799999999999995,"5min":1.0700000000000001,"15min":1.2470000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.77500000000000002,"15min":0.80400000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.89000000000000001,"5min":1.093,"15min":1.2330000000000001},"last":0.878}]}]},{"osd":4,"up_from":13,"seq":55834575046,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1310720,"data_stored":663810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:49:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67800000000000005,"15min":0.71699999999999997},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.316},"max":{"1min":0.81499999999999995,"5min":0.96999999999999997,"15min":1.022},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.49299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":1.01},"last":0.63100000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:49:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68500000000000005,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.35899999999999999,"15min":0.33200000000000002},"max":{"1min":0.89700000000000002,"5min":1.081,"15min":1.081},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70899999999999996,"15min":0.73399999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41299999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.044},"last":0.75}]},{"osd":2,"last update":"Fri Oct 6 03:49:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69799999999999995,"15min":0.73499999999999999},"min":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.377},"max":{"1min":0.96399999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72699999999999998,"15min":0.74399999999999999},"min":{"1min":0.51200000000000001,"5min":0.48799999999999999,"15min":0.48499999999999999},"max":{"1min":0.93799999999999994,"5min":1.0109999999999999,"15min":1.0660000000000001},"last":0.70399999999999996}]},{"osd":3,"last update":"Fri Oct 6 03:49:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.72699999999999998},"min":{"1min":0.46200000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0409999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73999999999999999,"15min":0.76400000000000001},"min":{"1min":0.51200000000000001,"5min":0.47099999999999997,"15min":0.46500000000000002},"max":{"1min":0.97599999999999998,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:49:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.627,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.98499999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.629,"15min":0.63},"min":{"1min":0.44,"5min":0.41499999999999998,"15min":0.35099999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.96799999999999997},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Oct 6 03:49:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65700000000000003,"15min":0.64300000000000002},"min":{"1min":0.434,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.093,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.68899999999999995},"min":{"1min":0.46500000000000002,"5min":0.42599999999999999,"15min":0.38100000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:49:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.39900000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.133},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72499999999999998,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.41299999999999998},"max":{"1min":1.1870000000000001,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.61899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575046,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302652,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069188,"statfs":{"total":96636764160,"available":96326848512,"internally_reserved":0,"allocated":1298432,"data_stored":662672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68700000000000006,"15min":0.70999999999999996},"min":{"1min":0.443,"5min":0.35999999999999999,"15min":0.35299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.48899999999999999,"5min":0.42399999999999999,"15min":0.32600000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70099999999999996,"15min":0.71899999999999997},"min":{"1min":0.54800000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.0429999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.48599999999999999,"5min":0.44,"15min":0.32400000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81799999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.54100000000000004,"5min":0.502,"15min":0.40200000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.749,"15min":0.77400000000000002},"min":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.50700000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.76600000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70899999999999996,"15min":0.73099999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95099999999999996,"5min":1.091,"15min":1.6419999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.748,"15min":0.751},"min":{"1min":0.50600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.371,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.52300000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.47499999999999998,"5min":0.40200000000000002,"15min":0.34999999999999998},"max":{"1min":0.92400000000000004,"5min":1.591,"15min":1.591},"last":0.498},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.71699999999999997,"15min":0.69499999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.196,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.78300000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68300000000000005,"15min":0.67600000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.94899999999999995,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72499999999999998,"15min":0.71799999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.056,"5min":1.278,"15min":1.278},"last":0.55100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:49:52.162 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:49:52.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.381+0000 7f726299d700 1 Processor -- start 2023-10-06T03:49:52.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.381+0000 7f726299d700 1 -- start start 2023-10-06T03:49:52.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.382+0000 7f726299d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f725c1595a0 0x7f725c0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:52.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.382+0000 7f726299d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f725c0c6c60 0x7f725c0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:52.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.382+0000 7f726299d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725c0b1410 con 0x7f725c0b1de0 2023-10-06T03:49:52.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.382+0000 7f726299d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725c0b2b90 con 0x7f725c1595a0 2023-10-06T03:49:52.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.382+0000 7f726299d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725c063230 con 0x7f725c0c6c60 2023-10-06T03:49:52.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.382+0000 7f726199b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f725c1595a0 0x7f725c0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:52.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.383+0000 7f726219c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f725c0b1de0 0x7f725c15d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51908/0 (socket says 172.21.15.55:51908) 2023-10-06T03:49:52.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.383+0000 7f726219c700 1 -- 172.21.15.55:0/457427757 learned_addr learned my addr 172.21.15.55:0/457427757 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:52.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.383+0000 7f726119a700 1 --2- 172.21.15.55:0/457427757 >> v2:172.21.15.163:3300/0 conn(0x7f725c0c6c60 0x7f725c0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:52.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.383+0000 7f726199b700 1 -- 172.21.15.55:0/457427757 >> v2:172.21.15.163:3300/0 conn(0x7f725c0c6c60 msgr2=0x7f725c0c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:52.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.383+0000 7f726199b700 1 --2- 172.21.15.55:0/457427757 >> v2:172.21.15.163:3300/0 conn(0x7f725c0c6c60 0x7f725c0c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.383+0000 7f726199b700 1 -- 172.21.15.55:0/457427757 >> v1:172.21.15.55:6789/0 conn(0x7f725c0b1de0 legacy=0x7f725c15d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:52.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.383+0000 7f726199b700 1 -- 172.21.15.55:0/457427757 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c0634f0 con 0x7f725c1595a0 2023-10-06T03:49:52.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.383+0000 7f7260999700 1 -- 172.21.15.55:0/457427757 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752641536 0 0) 0x7f725c0b1410 con 0x7f725c0b1de0 2023-10-06T03:49:52.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.383+0000 7f726199b700 1 --2- 172.21.15.55:0/457427757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f725c1595a0 0x7f725c0b18a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f725c0b07d0 tx=0x7f72580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae247be50220574 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:52.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.383+0000 7f7260999700 1 -- 172.21.15.55:0/457427757 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725800b6b0 con 0x7f725c1595a0 2023-10-06T03:49:52.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.384+0000 7f7260999700 1 -- 172.21.15.55:0/457427757 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725800b810 con 0x7f725c1595a0 2023-10-06T03:49:52.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.384+0000 7f7260999700 1 -- 172.21.15.55:0/457427757 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725800bbb0 con 0x7f725c1595a0 2023-10-06T03:49:52.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.384+0000 7f726299d700 1 -- 172.21.15.55:0/457427757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f725c1595a0 msgr2=0x7f725c0b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:52.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.384+0000 7f726299d700 1 --2- 172.21.15.55:0/457427757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f725c1595a0 0x7f725c0b18a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f725c0b07d0 tx=0x7f72580091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.384+0000 7f726299d700 1 -- 172.21.15.55:0/457427757 shutdown_connections 2023-10-06T03:49:52.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.384+0000 7f726299d700 1 --2- 172.21.15.55:0/457427757 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f725c1595a0 0x7f725c0b18a0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.384+0000 7f726299d700 1 --2- 172.21.15.55:0/457427757 >> v2:172.21.15.163:3300/0 conn(0x7f725c0c6c60 0x7f725c0c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.384+0000 7f726299d700 1 -- 172.21.15.55:0/457427757 >> 172.21.15.55:0/457427757 conn(0x7f725c06ddb0 msgr2=0x7f725c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:52.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.384+0000 7f726299d700 1 -- 172.21.15.55:0/457427757 shutdown_connections 2023-10-06T03:49:52.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.384+0000 7f726299d700 1 -- 172.21.15.55:0/457427757 wait complete. 2023-10-06T03:49:52.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.386+0000 7f726299d700 1 Processor -- start 2023-10-06T03:49:52.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.386+0000 7f726299d700 1 -- start start 2023-10-06T03:49:52.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726299d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f725c0b1de0 0x7f725c0c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:52.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726299d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f725c0c6c60 0x7f725c15a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:52.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726299d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f725c1595a0 0x7f725c169d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:52.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726299d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c0c3100 con 0x7f725c0b1de0 2023-10-06T03:49:52.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726299d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f725c16a290 con 0x7f725c0c6c60 2023-10-06T03:49:52.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726299d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725c15bcb0 con 0x7f725c1595a0 2023-10-06T03:49:52.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726199b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f725c0b1de0 0x7f725c0c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:52.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726199b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f725c0b1de0 0x7f725c0c56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55968/0 (socket says 172.21.15.55:55968) 2023-10-06T03:49:52.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726199b700 1 -- 172.21.15.55:0/386179820 learned_addr learned my addr 172.21.15.55:0/386179820 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:52.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726119a700 1 --2- 172.21.15.55:0/386179820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f725c0c6c60 0x7f725c15a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:52.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726199b700 1 -- 172.21.15.55:0/386179820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f725c1595a0 msgr2=0x7f725c169d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:52.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726199b700 1 --2- 172.21.15.55:0/386179820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f725c1595a0 0x7f725c169d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726199b700 1 -- 172.21.15.55:0/386179820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f725c0c6c60 msgr2=0x7f725c15a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:52.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726199b700 1 --2- 172.21.15.55:0/386179820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f725c0c6c60 0x7f725c15a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.387+0000 7f726199b700 1 -- 172.21.15.55:0/386179820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7258009420 con 0x7f725c0b1de0 2023-10-06T03:49:52.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.388+0000 7f726199b700 1 --2- 172.21.15.55:0/386179820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f725c0b1de0 0x7f725c0c56e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f725c064340 tx=0x7f7258017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:52.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.388+0000 7f7251ffb700 1 -- 172.21.15.55:0/386179820 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258011420 con 0x7f725c0b1de0 2023-10-06T03:49:52.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.388+0000 7f7251ffb700 1 -- 172.21.15.55:0/386179820 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7258011580 con 0x7f725c0b1de0 2023-10-06T03:49:52.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.388+0000 7f7251ffb700 1 -- 172.21.15.55:0/386179820 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258011940 con 0x7f725c0b1de0 2023-10-06T03:49:52.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.388+0000 7f726299d700 1 -- 172.21.15.55:0/386179820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725c0c5d50 con 0x7f725c0b1de0 2023-10-06T03:49:52.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.388+0000 7f726299d700 1 -- 172.21.15.55:0/386179820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725c0c6310 con 0x7f725c0b1de0 2023-10-06T03:49:52.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.389+0000 7f726299d700 1 -- 172.21.15.55:0/386179820 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725c04c3c0 con 0x7f725c0b1de0 2023-10-06T03:49:52.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.390+0000 7f7251ffb700 1 -- 172.21.15.55:0/386179820 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7258011d30 con 0x7f725c0b1de0 2023-10-06T03:49:52.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.391+0000 7f7251ffb700 1 --2- 172.21.15.55:0/386179820 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7240066f30 0x7f72400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:52.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.391+0000 7f7251ffb700 1 -- 172.21.15.55:0/386179820 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f7258085c40 con 0x7f725c0b1de0 2023-10-06T03:49:52.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.392+0000 7f726119a700 1 --2- 172.21.15.55:0/386179820 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7240066f30 0x7f72400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:52.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.392+0000 7f726119a700 1 --2- 172.21.15.55:0/386179820 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7240066f30 0x7f72400693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f725c15b1e0 tx=0x7f724c008040 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:52.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.395+0000 7f7251ffb700 1 -- 172.21.15.55:0/386179820 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7258052bc0 con 0x7f725c0b1de0 2023-10-06T03:49:52.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.592+0000 7f726299d700 1 -- 172.21.15.55:0/386179820 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f725c0637d0 con 0x7f7240066f30 2023-10-06T03:49:52.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.601+0000 7f7251ffb700 1 -- 172.21.15.55:0/386179820 <== mgr.6799 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311569 (secure 0 0 0) 0x7f725c0637d0 con 0x7f7240066f30 2023-10-06T03:49:52.607 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:52.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.606+0000 7f726299d700 1 -- 172.21.15.55:0/386179820 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7240066f30 msgr2=0x7f72400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:52.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.606+0000 7f726299d700 1 --2- 172.21.15.55:0/386179820 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7240066f30 0x7f72400693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f725c15b1e0 tx=0x7f724c008040 comp rx=0 tx=0).stop 2023-10-06T03:49:52.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.606+0000 7f726299d700 1 -- 172.21.15.55:0/386179820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f725c0b1de0 msgr2=0x7f725c0c56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:52.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.606+0000 7f726299d700 1 --2- 172.21.15.55:0/386179820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f725c0b1de0 0x7f725c0c56e0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f725c064340 tx=0x7f7258017000 comp rx=0 tx=0).stop 2023-10-06T03:49:52.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.606+0000 7f726299d700 1 -- 172.21.15.55:0/386179820 shutdown_connections 2023-10-06T03:49:52.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.606+0000 7f726299d700 1 --2- 172.21.15.55:0/386179820 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7240066f30 0x7f72400693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.606+0000 7f726299d700 1 --2- 172.21.15.55:0/386179820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f725c1595a0 0x7f725c169d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.606+0000 7f726299d700 1 --2- 172.21.15.55:0/386179820 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f725c0b1de0 0x7f725c0c56e0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.606+0000 7f726299d700 1 --2- 172.21.15.55:0/386179820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f725c0c6c60 0x7f725c15a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.606+0000 7f726299d700 1 -- 172.21.15.55:0/386179820 >> 172.21.15.55:0/386179820 conn(0x7f725c06ddb0 msgr2=0x7f725c153f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:52.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.606+0000 7f726299d700 1 -- 172.21.15.55:0/386179820 shutdown_connections 2023-10-06T03:49:52.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.606+0000 7f726299d700 1 -- 172.21.15.55:0/386179820 wait complete. 2023-10-06T03:49:52.611 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:49:52.625 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-10-06T03:49:52.029724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":363,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":243,"ondisk_log_size":243,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2455224,"kb_used_data":11384,"kb_used_omap":0,"kb_used_meta":2443776,"kb_avail":752519496,"statfs":{"total":773094113280,"available":770579963904,"internally_reserved":0,"allocated":11657216,"data_stored":6490636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2502426624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001648"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.793544+0000","last_change":"2023-10-06T03:49:47.789270+0000","last_active":"2023-10-06T03:49:47.793544+0000","last_peered":"2023-10-06T03:49:47.793544+0000","last_clean":"2023-10-06T03:49:47.793544+0000","last_became_active":"2023-10-06T03:49:47.788600+0000","last_became_peered":"2023-10-06T03:49:47.788600+0000","last_unstale":"2023-10-06T03:49:47.793544+0000","last_undegraded":"2023-10-06T03:49:47.793544+0000","last_fullsized":"2023-10-06T03:49:47.793544+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:31.915354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786503+0000","last_change":"2023-10-06T03:49:47.784606+0000","last_active":"2023-10-06T03:49:47.786503+0000","last_peered":"2023-10-06T03:49:47.786503+0000","last_clean":"2023-10-06T03:49:47.786503+0000","last_became_active":"2023-10-06T03:49:47.784319+0000","last_became_peered":"2023-10-06T03:49:47.784319+0000","last_unstale":"2023-10-06T03:49:47.786503+0000","last_undegraded":"2023-10-06T03:49:47.786503+0000","last_fullsized":"2023-10-06T03:49:47.786503+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:32.783620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.780370+0000","last_change":"2023-10-06T03:49:47.775923+0000","last_active":"2023-10-06T03:49:47.780370+0000","last_peered":"2023-10-06T03:49:47.780370+0000","last_clean":"2023-10-06T03:49:47.780370+0000","last_became_active":"2023-10-06T03:49:47.775579+0000","last_became_peered":"2023-10-06T03:49:47.775579+0000","last_unstale":"2023-10-06T03:49:47.780370+0000","last_undegraded":"2023-10-06T03:49:47.780370+0000","last_fullsized":"2023-10-06T03:49:47.780370+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:55:16.265534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.807530+0000","last_change":"2023-10-06T03:49:47.798390+0000","last_active":"2023-10-06T03:49:47.807530+0000","last_peered":"2023-10-06T03:49:47.807530+0000","last_clean":"2023-10-06T03:49:47.807530+0000","last_became_active":"2023-10-06T03:49:47.798003+0000","last_became_peered":"2023-10-06T03:49:47.798003+0000","last_unstale":"2023-10-06T03:49:47.807530+0000","last_undegraded":"2023-10-06T03:49:47.807530+0000","last_fullsized":"2023-10-06T03:49:47.807530+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:10.207499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.783305+0000","last_change":"2023-10-06T03:49:47.774228+0000","last_active":"2023-10-06T03:49:47.783305+0000","last_peered":"2023-10-06T03:49:47.783305+0000","last_clean":"2023-10-06T03:49:47.783305+0000","last_became_active":"2023-10-06T03:49:47.773969+0000","last_became_peered":"2023-10-06T03:49:47.773969+0000","last_unstale":"2023-10-06T03:49:47.783305+0000","last_undegraded":"2023-10-06T03:49:47.783305+0000","last_fullsized":"2023-10-06T03:49:47.783305+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:04:14.763921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.802944+0000","last_change":"2023-10-06T03:49:47.801538+0000","last_active":"2023-10-06T03:49:47.802944+0000","last_peered":"2023-10-06T03:49:47.802944+0000","last_clean":"2023-10-06T03:49:47.802944+0000","last_became_active":"2023-10-06T03:49:47.799566+0000","last_became_peered":"2023-10-06T03:49:47.799566+0000","last_unstale":"2023-10-06T03:49:47.802944+0000","last_undegraded":"2023-10-06T03:49:47.802944+0000","last_fullsized":"2023-10-06T03:49:47.802944+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:13.718257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.788796+0000","last_change":"2023-10-06T03:49:47.787934+0000","last_active":"2023-10-06T03:49:47.788796+0000","last_peered":"2023-10-06T03:49:47.788796+0000","last_clean":"2023-10-06T03:49:47.788796+0000","last_became_active":"2023-10-06T03:49:47.787683+0000","last_became_peered":"2023-10-06T03:49:47.787683+0000","last_unstale":"2023-10-06T03:49:47.788796+0000","last_undegraded":"2023-10-06T03:49:47.788796+0000","last_fullsized":"2023-10-06T03:49:47.788796+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:07:33.647871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.792627+0000","last_change":"2023-10-06T03:49:47.782656+0000","last_active":"2023-10-06T03:49:47.792627+0000","last_peered":"2023-10-06T03:49:47.792627+0000","last_clean":"2023-10-06T03:49:47.792627+0000","last_became_active":"2023-10-06T03:49:47.782063+0000","last_became_peered":"2023-10-06T03:49:47.782063+0000","last_unstale":"2023-10-06T03:49:47.792627+0000","last_undegraded":"2023-10-06T03:49:47.792627+0000","last_fullsized":"2023-10-06T03:49:47.792627+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:51.936358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.785464+0000","last_change":"2023-10-06T03:49:47.780506+0000","last_active":"2023-10-06T03:49:47.785464+0000","last_peered":"2023-10-06T03:49:47.785464+0000","last_clean":"2023-10-06T03:49:47.785464+0000","last_became_active":"2023-10-06T03:49:47.779406+0000","last_became_peered":"2023-10-06T03:49:47.779406+0000","last_unstale":"2023-10-06T03:49:47.785464+0000","last_undegraded":"2023-10-06T03:49:47.785464+0000","last_fullsized":"2023-10-06T03:49:47.785464+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:22:20.642945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798257+0000","last_change":"2023-10-06T03:49:47.796864+0000","last_active":"2023-10-06T03:49:47.798257+0000","last_peered":"2023-10-06T03:49:47.798257+0000","last_clean":"2023-10-06T03:49:47.798257+0000","last_became_active":"2023-10-06T03:49:47.796553+0000","last_became_peered":"2023-10-06T03:49:47.796553+0000","last_unstale":"2023-10-06T03:49:47.798257+0000","last_undegraded":"2023-10-06T03:49:47.798257+0000","last_fullsized":"2023-10-06T03:49:47.798257+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:26:47.044717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.787316+0000","last_change":"2023-10-06T03:49:47.785231+0000","last_active":"2023-10-06T03:49:47.787316+0000","last_peered":"2023-10-06T03:49:47.787316+0000","last_clean":"2023-10-06T03:49:47.787316+0000","last_became_active":"2023-10-06T03:49:47.784929+0000","last_became_peered":"2023-10-06T03:49:47.784929+0000","last_unstale":"2023-10-06T03:49:47.787316+0000","last_undegraded":"2023-10-06T03:49:47.787316+0000","last_fullsized":"2023-10-06T03:49:47.787316+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:57.703376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.787202+0000","last_change":"2023-10-06T03:49:47.784567+0000","last_active":"2023-10-06T03:49:47.787202+0000","last_peered":"2023-10-06T03:49:47.787202+0000","last_clean":"2023-10-06T03:49:47.787202+0000","last_became_active":"2023-10-06T03:49:47.784303+0000","last_became_peered":"2023-10-06T03:49:47.784303+0000","last_unstale":"2023-10-06T03:49:47.787202+0000","last_undegraded":"2023-10-06T03:49:47.787202+0000","last_fullsized":"2023-10-06T03:49:47.787202+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:22.428494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.802307+0000","last_change":"2023-10-06T03:49:47.797712+0000","last_active":"2023-10-06T03:49:47.802307+0000","last_peered":"2023-10-06T03:49:47.802307+0000","last_clean":"2023-10-06T03:49:47.802307+0000","last_became_active":"2023-10-06T03:49:47.796202+0000","last_became_peered":"2023-10-06T03:49:47.796202+0000","last_unstale":"2023-10-06T03:49:47.802307+0000","last_undegraded":"2023-10-06T03:49:47.802307+0000","last_fullsized":"2023-10-06T03:49:47.802307+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:01.617354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798736+0000","last_change":"2023-10-06T03:49:47.796997+0000","last_active":"2023-10-06T03:49:47.798736+0000","last_peered":"2023-10-06T03:49:47.798736+0000","last_clean":"2023-10-06T03:49:47.798736+0000","last_became_active":"2023-10-06T03:49:47.796690+0000","last_became_peered":"2023-10-06T03:49:47.796690+0000","last_unstale":"2023-10-06T03:49:47.798736+0000","last_undegraded":"2023-10-06T03:49:47.798736+0000","last_fullsized":"2023-10-06T03:49:47.798736+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:24:38.769554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.797970+0000","last_change":"2023-10-06T03:49:47.792060+0000","last_active":"2023-10-06T03:49:47.797970+0000","last_peered":"2023-10-06T03:49:47.797970+0000","last_clean":"2023-10-06T03:49:47.797970+0000","last_became_active":"2023-10-06T03:49:47.790622+0000","last_became_peered":"2023-10-06T03:49:47.790622+0000","last_unstale":"2023-10-06T03:49:47.797970+0000","last_undegraded":"2023-10-06T03:49:47.797970+0000","last_fullsized":"2023-10-06T03:49:47.797970+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:53:13.345647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.787285+0000","last_change":"2023-10-06T03:49:47.784537+0000","last_active":"2023-10-06T03:49:47.787285+0000","last_peered":"2023-10-06T03:49:47.787285+0000","last_clean":"2023-10-06T03:49:47.787285+0000","last_became_active":"2023-10-06T03:49:47.784113+0000","last_became_peered":"2023-10-06T03:49:47.784113+0000","last_unstale":"2023-10-06T03:49:47.787285+0000","last_undegraded":"2023-10-06T03:49:47.787285+0000","last_fullsized":"2023-10-06T03:49:47.787285+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:34.769673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.787772+0000","last_change":"2023-10-06T03:49:47.782168+0000","last_active":"2023-10-06T03:49:47.787772+0000","last_peered":"2023-10-06T03:49:47.787772+0000","last_clean":"2023-10-06T03:49:47.787772+0000","last_became_active":"2023-10-06T03:49:47.781891+0000","last_became_peered":"2023-10-06T03:49:47.781891+0000","last_unstale":"2023-10-06T03:49:47.787772+0000","last_undegraded":"2023-10-06T03:49:47.787772+0000","last_fullsized":"2023-10-06T03:49:47.787772+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:48:26.405246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.782594+0000","last_change":"2023-10-06T03:49:47.781056+0000","last_active":"2023-10-06T03:49:47.782594+0000","last_peered":"2023-10-06T03:49:47.782594+0000","last_clean":"2023-10-06T03:49:47.782594+0000","last_became_active":"2023-10-06T03:49:47.780746+0000","last_became_peered":"2023-10-06T03:49:47.780746+0000","last_unstale":"2023-10-06T03:49:47.782594+0000","last_undegraded":"2023-10-06T03:49:47.782594+0000","last_fullsized":"2023-10-06T03:49:47.782594+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:54.767724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798740+0000","last_change":"2023-10-06T03:49:47.797817+0000","last_active":"2023-10-06T03:49:47.798740+0000","last_peered":"2023-10-06T03:49:47.798740+0000","last_clean":"2023-10-06T03:49:47.798740+0000","last_became_active":"2023-10-06T03:49:47.797493+0000","last_became_peered":"2023-10-06T03:49:47.797493+0000","last_unstale":"2023-10-06T03:49:47.798740+0000","last_undegraded":"2023-10-06T03:49:47.798740+0000","last_fullsized":"2023-10-06T03:49:47.798740+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:52.359312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798176+0000","last_change":"2023-10-06T03:49:47.792520+0000","last_active":"2023-10-06T03:49:47.798176+0000","last_peered":"2023-10-06T03:49:47.798176+0000","last_clean":"2023-10-06T03:49:47.798176+0000","last_became_active":"2023-10-06T03:49:47.792251+0000","last_became_peered":"2023-10-06T03:49:47.792251+0000","last_unstale":"2023-10-06T03:49:47.798176+0000","last_undegraded":"2023-10-06T03:49:47.798176+0000","last_fullsized":"2023-10-06T03:49:47.798176+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:52.097059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.785638+0000","last_change":"2023-10-06T03:49:47.781705+0000","last_active":"2023-10-06T03:49:47.785638+0000","last_peered":"2023-10-06T03:49:47.785638+0000","last_clean":"2023-10-06T03:49:47.785638+0000","last_became_active":"2023-10-06T03:49:47.781364+0000","last_became_peered":"2023-10-06T03:49:47.781364+0000","last_unstale":"2023-10-06T03:49:47.785638+0000","last_undegraded":"2023-10-06T03:49:47.785638+0000","last_fullsized":"2023-10-06T03:49:47.785638+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:45:43.972937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798756+0000","last_change":"2023-10-06T03:49:47.797699+0000","last_active":"2023-10-06T03:49:47.798756+0000","last_peered":"2023-10-06T03:49:47.798756+0000","last_clean":"2023-10-06T03:49:47.798756+0000","last_became_active":"2023-10-06T03:49:47.797436+0000","last_became_peered":"2023-10-06T03:49:47.797436+0000","last_unstale":"2023-10-06T03:49:47.798756+0000","last_undegraded":"2023-10-06T03:49:47.798756+0000","last_fullsized":"2023-10-06T03:49:47.798756+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:54.050273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.793129+0000","last_change":"2023-10-06T03:49:47.791743+0000","last_active":"2023-10-06T03:49:47.793129+0000","last_peered":"2023-10-06T03:49:47.793129+0000","last_clean":"2023-10-06T03:49:47.793129+0000","last_became_active":"2023-10-06T03:49:47.791403+0000","last_became_peered":"2023-10-06T03:49:47.791403+0000","last_unstale":"2023-10-06T03:49:47.793129+0000","last_undegraded":"2023-10-06T03:49:47.793129+0000","last_fullsized":"2023-10-06T03:49:47.793129+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:39.878197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.781565+0000","last_change":"2023-10-06T03:49:47.777398+0000","last_active":"2023-10-06T03:49:47.781565+0000","last_peered":"2023-10-06T03:49:47.781565+0000","last_clean":"2023-10-06T03:49:47.781565+0000","last_became_active":"2023-10-06T03:49:47.776542+0000","last_became_peered":"2023-10-06T03:49:47.776542+0000","last_unstale":"2023-10-06T03:49:47.781565+0000","last_undegraded":"2023-10-06T03:49:47.781565+0000","last_fullsized":"2023-10-06T03:49:47.781565+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:09.479114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786542+0000","last_change":"2023-10-06T03:49:47.783949+0000","last_active":"2023-10-06T03:49:47.786542+0000","last_peered":"2023-10-06T03:49:47.786542+0000","last_clean":"2023-10-06T03:49:47.786542+0000","last_became_active":"2023-10-06T03:49:47.783637+0000","last_became_peered":"2023-10-06T03:49:47.783637+0000","last_unstale":"2023-10-06T03:49:47.786542+0000","last_undegraded":"2023-10-06T03:49:47.786542+0000","last_fullsized":"2023-10-06T03:49:47.786542+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:20.377785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798113+0000","last_change":"2023-10-06T03:49:47.796630+0000","last_active":"2023-10-06T03:49:47.798113+0000","last_peered":"2023-10-06T03:49:47.798113+0000","last_clean":"2023-10-06T03:49:47.798113+0000","last_became_active":"2023-10-06T03:49:47.796346+0000","last_became_peered":"2023-10-06T03:49:47.796346+0000","last_unstale":"2023-10-06T03:49:47.798113+0000","last_undegraded":"2023-10-06T03:49:47.798113+0000","last_fullsized":"2023-10-06T03:49:47.798113+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:02.836245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.775346+0000","last_change":"2023-10-06T03:49:47.772613+0000","last_active":"2023-10-06T03:49:47.775346+0000","last_peered":"2023-10-06T03:49:47.775346+0000","last_clean":"2023-10-06T03:49:47.775346+0000","last_became_active":"2023-10-06T03:49:47.772333+0000","last_became_peered":"2023-10-06T03:49:47.772333+0000","last_unstale":"2023-10-06T03:49:47.775346+0000","last_undegraded":"2023-10-06T03:49:47.775346+0000","last_fullsized":"2023-10-06T03:49:47.775346+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:14:33.444850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798192+0000","last_change":"2023-10-06T03:49:47.792782+0000","last_active":"2023-10-06T03:49:47.798192+0000","last_peered":"2023-10-06T03:49:47.798192+0000","last_clean":"2023-10-06T03:49:47.798192+0000","last_became_active":"2023-10-06T03:49:47.792039+0000","last_became_peered":"2023-10-06T03:49:47.792039+0000","last_unstale":"2023-10-06T03:49:47.798192+0000","last_undegraded":"2023-10-06T03:49:47.798192+0000","last_fullsized":"2023-10-06T03:49:47.798192+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:40.274992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786322+0000","last_change":"2023-10-06T03:49:47.783823+0000","last_active":"2023-10-06T03:49:47.786322+0000","last_peered":"2023-10-06T03:49:47.786322+0000","last_clean":"2023-10-06T03:49:47.786322+0000","last_became_active":"2023-10-06T03:49:47.782859+0000","last_became_peered":"2023-10-06T03:49:47.782859+0000","last_unstale":"2023-10-06T03:49:47.786322+0000","last_undegraded":"2023-10-06T03:49:47.786322+0000","last_fullsized":"2023-10-06T03:49:47.786322+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:05.450712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.785507+0000","last_change":"2023-10-06T03:49:47.779301+0000","last_active":"2023-10-06T03:49:47.785507+0000","last_peered":"2023-10-06T03:49:47.785507+0000","last_clean":"2023-10-06T03:49:47.785507+0000","last_became_active":"2023-10-06T03:49:47.779030+0000","last_became_peered":"2023-10-06T03:49:47.779030+0000","last_unstale":"2023-10-06T03:49:47.785507+0000","last_undegraded":"2023-10-06T03:49:47.785507+0000","last_fullsized":"2023-10-06T03:49:47.785507+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:07.974494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.793426+0000","last_change":"2023-10-06T03:49:47.791604+0000","last_active":"2023-10-06T03:49:47.793426+0000","last_peered":"2023-10-06T03:49:47.793426+0000","last_clean":"2023-10-06T03:49:47.793426+0000","last_became_active":"2023-10-06T03:49:47.791319+0000","last_became_peered":"2023-10-06T03:49:47.791319+0000","last_unstale":"2023-10-06T03:49:47.793426+0000","last_undegraded":"2023-10-06T03:49:47.793426+0000","last_fullsized":"2023-10-06T03:49:47.793426+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:37:40.274992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.787082+0000","last_change":"2023-10-06T03:49:47.784378+0000","last_active":"2023-10-06T03:49:47.787082+0000","last_peered":"2023-10-06T03:49:47.787082+0000","last_clean":"2023-10-06T03:49:47.787082+0000","last_became_active":"2023-10-06T03:49:47.783711+0000","last_became_peered":"2023-10-06T03:49:47.783711+0000","last_unstale":"2023-10-06T03:49:47.787082+0000","last_undegraded":"2023-10-06T03:49:47.787082+0000","last_fullsized":"2023-10-06T03:49:47.787082+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:30:27.472375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.796322+0000","last_change":"2023-10-06T03:49:47.792410+0000","last_active":"2023-10-06T03:49:47.796322+0000","last_peered":"2023-10-06T03:49:47.796322+0000","last_clean":"2023-10-06T03:49:47.796322+0000","last_became_active":"2023-10-06T03:49:47.792161+0000","last_became_peered":"2023-10-06T03:49:47.792161+0000","last_unstale":"2023-10-06T03:49:47.796322+0000","last_undegraded":"2023-10-06T03:49:47.796322+0000","last_fullsized":"2023-10-06T03:49:47.796322+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:45.707331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"106'4","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.806941+0000","last_change":"2023-10-06T03:49:46.774436+0000","last_active":"2023-10-06T03:49:47.806941+0000","last_peered":"2023-10-06T03:49:47.806941+0000","last_clean":"2023-10-06T03:49:47.806941+0000","last_became_active":"2023-10-06T03:49:46.774149+0000","last_became_peered":"2023-10-06T03:49:46.774149+0000","last_unstale":"2023-10-06T03:49:47.806941+0000","last_undegraded":"2023-10-06T03:49:47.806941+0000","last_fullsized":"2023-10-06T03:49:47.806941+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:30:00.438204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786971+0000","last_change":"2023-10-06T03:49:47.780333+0000","last_active":"2023-10-06T03:49:47.786971+0000","last_peered":"2023-10-06T03:49:47.786971+0000","last_clean":"2023-10-06T03:49:47.786971+0000","last_became_active":"2023-10-06T03:49:47.780060+0000","last_became_peered":"2023-10-06T03:49:47.780060+0000","last_unstale":"2023-10-06T03:49:47.786971+0000","last_undegraded":"2023-10-06T03:49:47.786971+0000","last_fullsized":"2023-10-06T03:49:47.786971+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:33:05.183707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.801983+0000","last_change":"2023-10-06T03:49:46.778045+0000","last_active":"2023-10-06T03:49:47.801983+0000","last_peered":"2023-10-06T03:49:47.801983+0000","last_clean":"2023-10-06T03:49:47.801983+0000","last_became_active":"2023-10-06T03:49:46.777746+0000","last_became_peered":"2023-10-06T03:49:46.777746+0000","last_unstale":"2023-10-06T03:49:47.801983+0000","last_undegraded":"2023-10-06T03:49:47.801983+0000","last_fullsized":"2023-10-06T03:49:47.801983+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:09.526031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798353+0000","last_change":"2023-10-06T03:49:47.797355+0000","last_active":"2023-10-06T03:49:47.798353+0000","last_peered":"2023-10-06T03:49:47.798353+0000","last_clean":"2023-10-06T03:49:47.798353+0000","last_became_active":"2023-10-06T03:49:47.796981+0000","last_became_peered":"2023-10-06T03:49:47.796981+0000","last_unstale":"2023-10-06T03:49:47.798353+0000","last_undegraded":"2023-10-06T03:49:47.798353+0000","last_fullsized":"2023-10-06T03:49:47.798353+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:24.179034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798891+0000","last_change":"2023-10-06T03:49:46.782511+0000","last_active":"2023-10-06T03:49:47.798891+0000","last_peered":"2023-10-06T03:49:47.798891+0000","last_clean":"2023-10-06T03:49:47.798891+0000","last_became_active":"2023-10-06T03:49:46.782113+0000","last_became_peered":"2023-10-06T03:49:46.782113+0000","last_unstale":"2023-10-06T03:49:47.798891+0000","last_undegraded":"2023-10-06T03:49:47.798891+0000","last_fullsized":"2023-10-06T03:49:47.798891+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:00.967847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.787750+0000","last_change":"2023-10-06T03:49:47.781651+0000","last_active":"2023-10-06T03:49:47.787750+0000","last_peered":"2023-10-06T03:49:47.787750+0000","last_clean":"2023-10-06T03:49:47.787750+0000","last_became_active":"2023-10-06T03:49:47.781282+0000","last_became_peered":"2023-10-06T03:49:47.781282+0000","last_unstale":"2023-10-06T03:49:47.787750+0000","last_undegraded":"2023-10-06T03:49:47.787750+0000","last_fullsized":"2023-10-06T03:49:47.787750+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:38:09.275210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.799431+0000","last_change":"2023-10-06T03:49:46.773253+0000","last_active":"2023-10-06T03:49:47.799431+0000","last_peered":"2023-10-06T03:49:47.799431+0000","last_clean":"2023-10-06T03:49:47.799431+0000","last_became_active":"2023-10-06T03:49:46.772968+0000","last_became_peered":"2023-10-06T03:49:46.772968+0000","last_unstale":"2023-10-06T03:49:47.799431+0000","last_undegraded":"2023-10-06T03:49:47.799431+0000","last_fullsized":"2023-10-06T03:49:47.799431+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:57.942952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.797541+0000","last_change":"2023-10-06T03:49:47.789938+0000","last_active":"2023-10-06T03:49:47.797541+0000","last_peered":"2023-10-06T03:49:47.797541+0000","last_clean":"2023-10-06T03:49:47.797541+0000","last_became_active":"2023-10-06T03:49:47.789662+0000","last_became_peered":"2023-10-06T03:49:47.789662+0000","last_unstale":"2023-10-06T03:49:47.797541+0000","last_undegraded":"2023-10-06T03:49:47.797541+0000","last_fullsized":"2023-10-06T03:49:47.797541+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:17.548015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.802032+0000","last_change":"2023-10-06T03:49:46.781773+0000","last_active":"2023-10-06T03:49:47.802032+0000","last_peered":"2023-10-06T03:49:47.802032+0000","last_clean":"2023-10-06T03:49:47.802032+0000","last_became_active":"2023-10-06T03:49:46.781494+0000","last_became_peered":"2023-10-06T03:49:46.781494+0000","last_unstale":"2023-10-06T03:49:47.802032+0000","last_undegraded":"2023-10-06T03:49:47.802032+0000","last_fullsized":"2023-10-06T03:49:47.802032+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:03.287036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.779426+0000","last_change":"2023-10-06T03:49:47.774508+0000","last_active":"2023-10-06T03:49:47.779426+0000","last_peered":"2023-10-06T03:49:47.779426+0000","last_clean":"2023-10-06T03:49:47.779426+0000","last_became_active":"2023-10-06T03:49:47.774208+0000","last_became_peered":"2023-10-06T03:49:47.774208+0000","last_unstale":"2023-10-06T03:49:47.779426+0000","last_undegraded":"2023-10-06T03:49:47.779426+0000","last_fullsized":"2023-10-06T03:49:47.779426+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:11.424246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.805404+0000","last_change":"2023-10-06T03:49:46.785965+0000","last_active":"2023-10-06T03:49:47.805404+0000","last_peered":"2023-10-06T03:49:47.805404+0000","last_clean":"2023-10-06T03:49:47.805404+0000","last_became_active":"2023-10-06T03:49:46.785664+0000","last_became_peered":"2023-10-06T03:49:46.785664+0000","last_unstale":"2023-10-06T03:49:47.805404+0000","last_undegraded":"2023-10-06T03:49:47.805404+0000","last_fullsized":"2023-10-06T03:49:47.805404+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:29.978688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.801623+0000","last_change":"2023-10-06T03:49:47.799980+0000","last_active":"2023-10-06T03:49:47.801623+0000","last_peered":"2023-10-06T03:49:47.801623+0000","last_clean":"2023-10-06T03:49:47.801623+0000","last_became_active":"2023-10-06T03:49:47.799209+0000","last_became_peered":"2023-10-06T03:49:47.799209+0000","last_unstale":"2023-10-06T03:49:47.801623+0000","last_undegraded":"2023-10-06T03:49:47.801623+0000","last_fullsized":"2023-10-06T03:49:47.801623+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:46:34.061339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.766952+0000","last_change":"2023-10-06T03:49:46.784477+0000","last_active":"2023-10-06T03:49:47.766952+0000","last_peered":"2023-10-06T03:49:47.766952+0000","last_clean":"2023-10-06T03:49:47.766952+0000","last_became_active":"2023-10-06T03:49:46.784160+0000","last_became_peered":"2023-10-06T03:49:46.784160+0000","last_unstale":"2023-10-06T03:49:47.766952+0000","last_undegraded":"2023-10-06T03:49:47.766952+0000","last_fullsized":"2023-10-06T03:49:47.766952+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:16.540025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.797869+0000","last_change":"2023-10-06T03:49:47.790618+0000","last_active":"2023-10-06T03:49:47.797869+0000","last_peered":"2023-10-06T03:49:47.797869+0000","last_clean":"2023-10-06T03:49:47.797869+0000","last_became_active":"2023-10-06T03:49:47.790315+0000","last_became_peered":"2023-10-06T03:49:47.790315+0000","last_unstale":"2023-10-06T03:49:47.797869+0000","last_undegraded":"2023-10-06T03:49:47.797869+0000","last_fullsized":"2023-10-06T03:49:47.797869+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:20.377785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.772550+0000","last_change":"2023-10-06T03:49:46.786656+0000","last_active":"2023-10-06T03:49:47.772550+0000","last_peered":"2023-10-06T03:49:47.772550+0000","last_clean":"2023-10-06T03:49:47.772550+0000","last_became_active":"2023-10-06T03:49:46.786328+0000","last_became_peered":"2023-10-06T03:49:46.786328+0000","last_unstale":"2023-10-06T03:49:47.772550+0000","last_undegraded":"2023-10-06T03:49:47.772550+0000","last_fullsized":"2023-10-06T03:49:47.772550+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:41:03.287036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.785001+0000","last_change":"2023-10-06T03:49:47.773723+0000","last_active":"2023-10-06T03:49:47.785001+0000","last_peered":"2023-10-06T03:49:47.785001+0000","last_clean":"2023-10-06T03:49:47.785001+0000","last_became_active":"2023-10-06T03:49:47.773447+0000","last_became_peered":"2023-10-06T03:49:47.773447+0000","last_unstale":"2023-10-06T03:49:47.785001+0000","last_undegraded":"2023-10-06T03:49:47.785001+0000","last_fullsized":"2023-10-06T03:49:47.785001+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:33:37.494933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.774350+0000","last_change":"2023-10-06T03:49:46.783682+0000","last_active":"2023-10-06T03:49:47.774350+0000","last_peered":"2023-10-06T03:49:47.774350+0000","last_clean":"2023-10-06T03:49:47.774350+0000","last_became_active":"2023-10-06T03:49:46.783286+0000","last_became_peered":"2023-10-06T03:49:46.783286+0000","last_unstale":"2023-10-06T03:49:47.774350+0000","last_undegraded":"2023-10-06T03:49:47.774350+0000","last_fullsized":"2023-10-06T03:49:47.774350+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:10.416256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.788310+0000","last_change":"2023-10-06T03:49:47.787196+0000","last_active":"2023-10-06T03:49:47.788310+0000","last_peered":"2023-10-06T03:49:47.788310+0000","last_clean":"2023-10-06T03:49:47.788310+0000","last_became_active":"2023-10-06T03:49:47.786862+0000","last_became_peered":"2023-10-06T03:49:47.786862+0000","last_unstale":"2023-10-06T03:49:47.788310+0000","last_undegraded":"2023-10-06T03:49:47.788310+0000","last_fullsized":"2023-10-06T03:49:47.788310+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:35:02.011614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.767813+0000","last_change":"2023-10-06T03:49:46.788480+0000","last_active":"2023-10-06T03:49:47.767813+0000","last_peered":"2023-10-06T03:49:47.767813+0000","last_clean":"2023-10-06T03:49:47.767813+0000","last_became_active":"2023-10-06T03:49:46.788221+0000","last_became_peered":"2023-10-06T03:49:46.788221+0000","last_unstale":"2023-10-06T03:49:47.767813+0000","last_undegraded":"2023-10-06T03:49:47.767813+0000","last_fullsized":"2023-10-06T03:49:47.767813+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:00.609364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.781611+0000","last_change":"2023-10-06T03:49:47.778229+0000","last_active":"2023-10-06T03:49:47.781611+0000","last_peered":"2023-10-06T03:49:47.781611+0000","last_clean":"2023-10-06T03:49:47.781611+0000","last_became_active":"2023-10-06T03:49:47.777970+0000","last_became_peered":"2023-10-06T03:49:47.777970+0000","last_unstale":"2023-10-06T03:49:47.781611+0000","last_undegraded":"2023-10-06T03:49:47.781611+0000","last_fullsized":"2023-10-06T03:49:47.781611+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:08.886392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"106'3","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.807141+0000","last_change":"2023-10-06T03:49:46.787883+0000","last_active":"2023-10-06T03:49:47.807141+0000","last_peered":"2023-10-06T03:49:47.807141+0000","last_clean":"2023-10-06T03:49:47.807141+0000","last_became_active":"2023-10-06T03:49:46.787373+0000","last_became_peered":"2023-10-06T03:49:46.787373+0000","last_unstale":"2023-10-06T03:49:47.807141+0000","last_undegraded":"2023-10-06T03:49:47.807141+0000","last_fullsized":"2023-10-06T03:49:47.807141+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:09:00.967847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.797581+0000","last_change":"2023-10-06T03:49:47.790616+0000","last_active":"2023-10-06T03:49:47.797581+0000","last_peered":"2023-10-06T03:49:47.797581+0000","last_clean":"2023-10-06T03:49:47.797581+0000","last_became_active":"2023-10-06T03:49:47.790327+0000","last_became_peered":"2023-10-06T03:49:47.790327+0000","last_unstale":"2023-10-06T03:49:47.797581+0000","last_undegraded":"2023-10-06T03:49:47.797581+0000","last_fullsized":"2023-10-06T03:49:47.797581+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T03:57:32.783620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"106'6","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.809894+0000","last_change":"2023-10-06T03:49:46.780674+0000","last_active":"2023-10-06T03:49:47.809894+0000","last_peered":"2023-10-06T03:49:47.809894+0000","last_clean":"2023-10-06T03:49:47.809894+0000","last_became_active":"2023-10-06T03:49:46.780331+0000","last_became_peered":"2023-10-06T03:49:46.780331+0000","last_unstale":"2023-10-06T03:49:47.809894+0000","last_undegraded":"2023-10-06T03:49:47.809894+0000","last_fullsized":"2023-10-06T03:49:47.809894+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:29:51.351322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.792605+0000","last_change":"2023-10-06T03:49:47.782444+0000","last_active":"2023-10-06T03:49:47.792605+0000","last_peered":"2023-10-06T03:49:47.792605+0000","last_clean":"2023-10-06T03:49:47.792605+0000","last_became_active":"2023-10-06T03:49:47.781698+0000","last_became_peered":"2023-10-06T03:49:47.781698+0000","last_unstale":"2023-10-06T03:49:47.792605+0000","last_undegraded":"2023-10-06T03:49:47.792605+0000","last_fullsized":"2023-10-06T03:49:47.792605+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:16.509044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.801772+0000","last_change":"2023-10-06T03:49:46.786444+0000","last_active":"2023-10-06T03:49:47.801772+0000","last_peered":"2023-10-06T03:49:47.801772+0000","last_clean":"2023-10-06T03:49:47.801772+0000","last_became_active":"2023-10-06T03:49:46.786240+0000","last_became_peered":"2023-10-06T03:49:46.786240+0000","last_unstale":"2023-10-06T03:49:47.801772+0000","last_undegraded":"2023-10-06T03:49:47.801772+0000","last_fullsized":"2023-10-06T03:49:47.801772+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:31.233223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.778517+0000","last_change":"2023-10-06T03:49:47.772427+0000","last_active":"2023-10-06T03:49:47.778517+0000","last_peered":"2023-10-06T03:49:47.778517+0000","last_clean":"2023-10-06T03:49:47.778517+0000","last_became_active":"2023-10-06T03:49:47.772110+0000","last_became_peered":"2023-10-06T03:49:47.772110+0000","last_unstale":"2023-10-06T03:49:47.778517+0000","last_undegraded":"2023-10-06T03:49:47.778517+0000","last_fullsized":"2023-10-06T03:49:47.778517+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:44:38.218445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.771191+0000","last_change":"2023-10-06T03:49:46.791774+0000","last_active":"2023-10-06T03:49:47.771191+0000","last_peered":"2023-10-06T03:49:47.771191+0000","last_clean":"2023-10-06T03:49:47.771191+0000","last_became_active":"2023-10-06T03:49:46.790979+0000","last_became_peered":"2023-10-06T03:49:46.790979+0000","last_unstale":"2023-10-06T03:49:47.771191+0000","last_undegraded":"2023-10-06T03:49:47.771191+0000","last_fullsized":"2023-10-06T03:49:47.771191+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:18:53.042283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798325+0000","last_change":"2023-10-06T03:49:47.796924+0000","last_active":"2023-10-06T03:49:47.798325+0000","last_peered":"2023-10-06T03:49:47.798325+0000","last_clean":"2023-10-06T03:49:47.798325+0000","last_became_active":"2023-10-06T03:49:47.796523+0000","last_became_peered":"2023-10-06T03:49:47.796523+0000","last_unstale":"2023-10-06T03:49:47.798325+0000","last_undegraded":"2023-10-06T03:49:47.798325+0000","last_fullsized":"2023-10-06T03:49:47.798325+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:40.976185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.788194+0000","last_change":"2023-10-06T03:49:46.780647+0000","last_active":"2023-10-06T03:49:47.788194+0000","last_peered":"2023-10-06T03:49:47.788194+0000","last_clean":"2023-10-06T03:49:47.788194+0000","last_became_active":"2023-10-06T03:49:46.780391+0000","last_became_peered":"2023-10-06T03:49:46.780391+0000","last_unstale":"2023-10-06T03:49:47.788194+0000","last_undegraded":"2023-10-06T03:49:47.788194+0000","last_fullsized":"2023-10-06T03:49:47.788194+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:01.394824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.785782+0000","last_change":"2023-10-06T03:49:47.783551+0000","last_active":"2023-10-06T03:49:47.785782+0000","last_peered":"2023-10-06T03:49:47.785782+0000","last_clean":"2023-10-06T03:49:47.785782+0000","last_became_active":"2023-10-06T03:49:47.783250+0000","last_became_peered":"2023-10-06T03:49:47.783250+0000","last_unstale":"2023-10-06T03:49:47.785782+0000","last_undegraded":"2023-10-06T03:49:47.785782+0000","last_fullsized":"2023-10-06T03:49:47.785782+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:06.680989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"106'8","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.808583+0000","last_change":"2023-10-06T03:49:46.784798+0000","last_active":"2023-10-06T03:49:47.808583+0000","last_peered":"2023-10-06T03:49:47.808583+0000","last_clean":"2023-10-06T03:49:47.808583+0000","last_became_active":"2023-10-06T03:49:46.784323+0000","last_became_peered":"2023-10-06T03:49:46.784323+0000","last_unstale":"2023-10-06T03:49:47.808583+0000","last_undegraded":"2023-10-06T03:49:47.808583+0000","last_fullsized":"2023-10-06T03:49:47.808583+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:01.103063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.780278+0000","last_change":"2023-10-06T03:49:47.776050+0000","last_active":"2023-10-06T03:49:47.780278+0000","last_peered":"2023-10-06T03:49:47.780278+0000","last_clean":"2023-10-06T03:49:47.780278+0000","last_became_active":"2023-10-06T03:49:47.775755+0000","last_became_peered":"2023-10-06T03:49:47.775755+0000","last_unstale":"2023-10-06T03:49:47.780278+0000","last_undegraded":"2023-10-06T03:49:47.780278+0000","last_fullsized":"2023-10-06T03:49:47.780278+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:34:50.192073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.797814+0000","last_change":"2023-10-06T03:49:46.780843+0000","last_active":"2023-10-06T03:49:47.797814+0000","last_peered":"2023-10-06T03:49:47.797814+0000","last_clean":"2023-10-06T03:49:47.797814+0000","last_became_active":"2023-10-06T03:49:46.780292+0000","last_became_peered":"2023-10-06T03:49:46.780292+0000","last_unstale":"2023-10-06T03:49:47.797814+0000","last_undegraded":"2023-10-06T03:49:47.797814+0000","last_fullsized":"2023-10-06T03:49:47.797814+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:12:09.526031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.794442+0000","last_change":"2023-10-06T03:49:47.791544+0000","last_active":"2023-10-06T03:49:47.794442+0000","last_peered":"2023-10-06T03:49:47.794442+0000","last_clean":"2023-10-06T03:49:47.794442+0000","last_became_active":"2023-10-06T03:49:47.791269+0000","last_became_peered":"2023-10-06T03:49:47.791269+0000","last_unstale":"2023-10-06T03:49:47.794442+0000","last_undegraded":"2023-10-06T03:49:47.794442+0000","last_fullsized":"2023-10-06T03:49:47.794442+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:46:41.003218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.770682+0000","last_change":"2023-10-06T03:49:46.778038+0000","last_active":"2023-10-06T03:49:47.770682+0000","last_peered":"2023-10-06T03:49:47.770682+0000","last_clean":"2023-10-06T03:49:47.770682+0000","last_became_active":"2023-10-06T03:49:46.777711+0000","last_became_peered":"2023-10-06T03:49:46.777711+0000","last_unstale":"2023-10-06T03:49:47.770682+0000","last_undegraded":"2023-10-06T03:49:47.770682+0000","last_fullsized":"2023-10-06T03:49:47.770682+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:57:08.471124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.781486+0000","last_change":"2023-10-06T03:49:47.777345+0000","last_active":"2023-10-06T03:49:47.781486+0000","last_peered":"2023-10-06T03:49:47.781486+0000","last_clean":"2023-10-06T03:49:47.781486+0000","last_became_active":"2023-10-06T03:49:47.777054+0000","last_became_peered":"2023-10-06T03:49:47.777054+0000","last_unstale":"2023-10-06T03:49:47.781486+0000","last_undegraded":"2023-10-06T03:49:47.781486+0000","last_fullsized":"2023-10-06T03:49:47.781486+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:25.358460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.770123+0000","last_change":"2023-10-06T03:49:46.787882+0000","last_active":"2023-10-06T03:49:47.770123+0000","last_peered":"2023-10-06T03:49:47.770123+0000","last_clean":"2023-10-06T03:49:47.770123+0000","last_became_active":"2023-10-06T03:49:46.787230+0000","last_became_peered":"2023-10-06T03:49:46.787230+0000","last_unstale":"2023-10-06T03:49:47.770123+0000","last_undegraded":"2023-10-06T03:49:47.770123+0000","last_fullsized":"2023-10-06T03:49:47.770123+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:49:31.233223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786610+0000","last_change":"2023-10-06T03:49:47.784850+0000","last_active":"2023-10-06T03:49:47.786610+0000","last_peered":"2023-10-06T03:49:47.786610+0000","last_clean":"2023-10-06T03:49:47.786610+0000","last_became_active":"2023-10-06T03:49:47.784466+0000","last_became_peered":"2023-10-06T03:49:47.784466+0000","last_unstale":"2023-10-06T03:49:47.786610+0000","last_undegraded":"2023-10-06T03:49:47.786610+0000","last_fullsized":"2023-10-06T03:49:47.786610+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:27.622375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.767290+0000","last_change":"2023-10-06T03:49:46.782877+0000","last_active":"2023-10-06T03:49:47.767290+0000","last_peered":"2023-10-06T03:49:47.767290+0000","last_clean":"2023-10-06T03:49:47.767290+0000","last_became_active":"2023-10-06T03:49:46.781926+0000","last_became_peered":"2023-10-06T03:49:46.781926+0000","last_unstale":"2023-10-06T03:49:47.767290+0000","last_undegraded":"2023-10-06T03:49:47.767290+0000","last_fullsized":"2023-10-06T03:49:47.767290+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:05:34.375175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.779389+0000","last_change":"2023-10-06T03:49:47.776450+0000","last_active":"2023-10-06T03:49:47.779389+0000","last_peered":"2023-10-06T03:49:47.779389+0000","last_clean":"2023-10-06T03:49:47.779389+0000","last_became_active":"2023-10-06T03:49:47.776169+0000","last_became_peered":"2023-10-06T03:49:47.776169+0000","last_unstale":"2023-10-06T03:49:47.779389+0000","last_undegraded":"2023-10-06T03:49:47.779389+0000","last_fullsized":"2023-10-06T03:49:47.779389+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:27.190523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.771912+0000","last_change":"2023-10-06T03:49:46.792516+0000","last_active":"2023-10-06T03:49:47.771912+0000","last_peered":"2023-10-06T03:49:47.771912+0000","last_clean":"2023-10-06T03:49:47.771912+0000","last_became_active":"2023-10-06T03:49:46.792012+0000","last_became_peered":"2023-10-06T03:49:46.792012+0000","last_unstale":"2023-10-06T03:49:47.771912+0000","last_undegraded":"2023-10-06T03:49:47.771912+0000","last_fullsized":"2023-10-06T03:49:47.771912+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:05:06.966504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.798833+0000","last_change":"2023-10-06T03:49:47.796697+0000","last_active":"2023-10-06T03:49:47.798833+0000","last_peered":"2023-10-06T03:49:47.798833+0000","last_clean":"2023-10-06T03:49:47.798833+0000","last_became_active":"2023-10-06T03:49:47.796440+0000","last_became_peered":"2023-10-06T03:49:47.796440+0000","last_unstale":"2023-10-06T03:49:47.798833+0000","last_undegraded":"2023-10-06T03:49:47.798833+0000","last_fullsized":"2023-10-06T03:49:47.798833+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:19.438975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.772942+0000","last_change":"2023-10-06T03:49:46.784342+0000","last_active":"2023-10-06T03:49:47.772942+0000","last_peered":"2023-10-06T03:49:47.772942+0000","last_clean":"2023-10-06T03:49:47.772942+0000","last_became_active":"2023-10-06T03:49:46.784137+0000","last_became_peered":"2023-10-06T03:49:46.784137+0000","last_unstale":"2023-10-06T03:49:47.772942+0000","last_undegraded":"2023-10-06T03:49:47.772942+0000","last_fullsized":"2023-10-06T03:49:47.772942+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:38.870207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.800176+0000","last_change":"2023-10-06T03:49:47.799422+0000","last_active":"2023-10-06T03:49:47.800176+0000","last_peered":"2023-10-06T03:49:47.800176+0000","last_clean":"2023-10-06T03:49:47.800176+0000","last_became_active":"2023-10-06T03:49:47.799039+0000","last_became_peered":"2023-10-06T03:49:47.799039+0000","last_unstale":"2023-10-06T03:49:47.800176+0000","last_undegraded":"2023-10-06T03:49:47.800176+0000","last_fullsized":"2023-10-06T03:49:47.800176+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:37.233664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.767473+0000","last_change":"2023-10-06T03:49:46.783252+0000","last_active":"2023-10-06T03:49:47.767473+0000","last_peered":"2023-10-06T03:49:47.767473+0000","last_clean":"2023-10-06T03:49:47.767473+0000","last_became_active":"2023-10-06T03:49:46.782981+0000","last_became_peered":"2023-10-06T03:49:46.782981+0000","last_unstale":"2023-10-06T03:49:47.767473+0000","last_undegraded":"2023-10-06T03:49:47.767473+0000","last_fullsized":"2023-10-06T03:49:47.767473+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:35:36.224710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786921+0000","last_change":"2023-10-06T03:49:47.782977+0000","last_active":"2023-10-06T03:49:47.786921+0000","last_peered":"2023-10-06T03:49:47.786921+0000","last_clean":"2023-10-06T03:49:47.786921+0000","last_became_active":"2023-10-06T03:49:47.782598+0000","last_became_peered":"2023-10-06T03:49:47.782598+0000","last_unstale":"2023-10-06T03:49:47.786921+0000","last_undegraded":"2023-10-06T03:49:47.786921+0000","last_fullsized":"2023-10-06T03:49:47.786921+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:26:28.965158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.772669+0000","last_change":"2023-10-06T03:49:46.785971+0000","last_active":"2023-10-06T03:49:47.772669+0000","last_peered":"2023-10-06T03:49:47.772669+0000","last_clean":"2023-10-06T03:49:47.772669+0000","last_became_active":"2023-10-06T03:49:46.785686+0000","last_became_peered":"2023-10-06T03:49:46.785686+0000","last_unstale":"2023-10-06T03:49:47.772669+0000","last_undegraded":"2023-10-06T03:49:47.772669+0000","last_fullsized":"2023-10-06T03:49:47.772669+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:51.089069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.795334+0000","last_change":"2023-10-06T03:49:47.794256+0000","last_active":"2023-10-06T03:49:47.795334+0000","last_peered":"2023-10-06T03:49:47.795334+0000","last_clean":"2023-10-06T03:49:47.795334+0000","last_became_active":"2023-10-06T03:49:47.793837+0000","last_became_peered":"2023-10-06T03:49:47.793837+0000","last_unstale":"2023-10-06T03:49:47.795334+0000","last_undegraded":"2023-10-06T03:49:47.795334+0000","last_fullsized":"2023-10-06T03:49:47.795334+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:16.509044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.772566+0000","last_change":"2023-10-06T03:49:46.781425+0000","last_active":"2023-10-06T03:49:47.772566+0000","last_peered":"2023-10-06T03:49:47.772566+0000","last_clean":"2023-10-06T03:49:47.772566+0000","last_became_active":"2023-10-06T03:49:46.781050+0000","last_became_peered":"2023-10-06T03:49:46.781050+0000","last_unstale":"2023-10-06T03:49:47.772566+0000","last_undegraded":"2023-10-06T03:49:47.772566+0000","last_fullsized":"2023-10-06T03:49:47.772566+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:01.394824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.769873+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:49:47.769873+0000","last_peered":"2023-10-06T03:49:47.769873+0000","last_clean":"2023-10-06T03:49:47.769873+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:49:47.769873+0000","last_undegraded":"2023-10-06T03:49:47.769873+0000","last_fullsized":"2023-10-06T03:49:47.769873+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.781247+0000","last_change":"2023-10-06T03:49:47.777001+0000","last_active":"2023-10-06T03:49:47.781247+0000","last_peered":"2023-10-06T03:49:47.781247+0000","last_clean":"2023-10-06T03:49:47.781247+0000","last_became_active":"2023-10-06T03:49:47.776722+0000","last_became_peered":"2023-10-06T03:49:47.776722+0000","last_unstale":"2023-10-06T03:49:47.781247+0000","last_undegraded":"2023-10-06T03:49:47.781247+0000","last_fullsized":"2023-10-06T03:49:47.781247+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:38.223180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.767735+0000","last_change":"2023-10-06T03:49:46.782221+0000","last_active":"2023-10-06T03:49:47.767735+0000","last_peered":"2023-10-06T03:49:47.767735+0000","last_clean":"2023-10-06T03:49:47.767735+0000","last_became_active":"2023-10-06T03:49:46.781917+0000","last_became_peered":"2023-10-06T03:49:46.781917+0000","last_unstale":"2023-10-06T03:49:47.767735+0000","last_undegraded":"2023-10-06T03:49:47.767735+0000","last_fullsized":"2023-10-06T03:49:47.767735+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:40.482430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.765432+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:49:47.765432+0000","last_peered":"2023-10-06T03:49:47.765432+0000","last_clean":"2023-10-06T03:49:47.765432+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:49:47.765432+0000","last_undegraded":"2023-10-06T03:49:47.765432+0000","last_fullsized":"2023-10-06T03:49:47.765432+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.791017+0000","last_change":"2023-10-06T03:49:47.788586+0000","last_active":"2023-10-06T03:49:47.791017+0000","last_peered":"2023-10-06T03:49:47.791017+0000","last_clean":"2023-10-06T03:49:47.791017+0000","last_became_active":"2023-10-06T03:49:47.788322+0000","last_became_peered":"2023-10-06T03:49:47.788322+0000","last_unstale":"2023-10-06T03:49:47.791017+0000","last_undegraded":"2023-10-06T03:49:47.791017+0000","last_fullsized":"2023-10-06T03:49:47.791017+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:17.408086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.794515+0000","last_change":"2023-10-06T03:49:46.775600+0000","last_active":"2023-10-06T03:49:47.794515+0000","last_peered":"2023-10-06T03:49:47.794515+0000","last_clean":"2023-10-06T03:49:47.794515+0000","last_became_active":"2023-10-06T03:49:46.774748+0000","last_became_peered":"2023-10-06T03:49:46.774748+0000","last_unstale":"2023-10-06T03:49:47.794515+0000","last_undegraded":"2023-10-06T03:49:47.794515+0000","last_fullsized":"2023-10-06T03:49:47.794515+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:14.411208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":89,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786086+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:49:47.786086+0000","last_peered":"2023-10-06T03:49:47.786086+0000","last_clean":"2023-10-06T03:49:47.786086+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:49:47.786086+0000","last_undegraded":"2023-10-06T03:49:47.786086+0000","last_fullsized":"2023-10-06T03:49:47.786086+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"96'205","reported_seq":551,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.772773+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:49:47.772773+0000","last_peered":"2023-10-06T03:49:47.772773+0000","last_clean":"2023-10-06T03:49:47.772773+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:49:47.772773+0000","last_undegraded":"2023-10-06T03:49:47.772773+0000","last_fullsized":"2023-10-06T03:49:47.772773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.780359+0000","last_change":"2023-10-06T03:49:47.776643+0000","last_active":"2023-10-06T03:49:47.780359+0000","last_peered":"2023-10-06T03:49:47.780359+0000","last_clean":"2023-10-06T03:49:47.780359+0000","last_became_active":"2023-10-06T03:49:47.776248+0000","last_became_peered":"2023-10-06T03:49:47.776248+0000","last_unstale":"2023-10-06T03:49:47.780359+0000","last_undegraded":"2023-10-06T03:49:47.780359+0000","last_fullsized":"2023-10-06T03:49:47.780359+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:48.485492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.797380+0000","last_change":"2023-10-06T03:49:46.780848+0000","last_active":"2023-10-06T03:49:47.797380+0000","last_peered":"2023-10-06T03:49:47.797380+0000","last_clean":"2023-10-06T03:49:47.797380+0000","last_became_active":"2023-10-06T03:49:46.780339+0000","last_became_peered":"2023-10-06T03:49:46.780339+0000","last_unstale":"2023-10-06T03:49:47.797380+0000","last_undegraded":"2023-10-06T03:49:47.797380+0000","last_fullsized":"2023-10-06T03:49:47.797380+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:59.899446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.772031+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:49:47.772031+0000","last_peered":"2023-10-06T03:49:47.772031+0000","last_clean":"2023-10-06T03:49:47.772031+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:49:47.772031+0000","last_undegraded":"2023-10-06T03:49:47.772031+0000","last_fullsized":"2023-10-06T03:49:47.772031+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.788814+0000","last_change":"2023-10-06T03:49:47.788071+0000","last_active":"2023-10-06T03:49:47.788814+0000","last_peered":"2023-10-06T03:49:47.788814+0000","last_clean":"2023-10-06T03:49:47.788814+0000","last_became_active":"2023-10-06T03:49:47.787748+0000","last_became_peered":"2023-10-06T03:49:47.787748+0000","last_unstale":"2023-10-06T03:49:47.788814+0000","last_undegraded":"2023-10-06T03:49:47.788814+0000","last_fullsized":"2023-10-06T03:49:47.788814+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:32:09.919684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.785417+0000","last_change":"2023-10-06T03:49:46.789229+0000","last_active":"2023-10-06T03:49:47.785417+0000","last_peered":"2023-10-06T03:49:47.785417+0000","last_clean":"2023-10-06T03:49:47.785417+0000","last_became_active":"2023-10-06T03:49:46.788886+0000","last_became_peered":"2023-10-06T03:49:46.788886+0000","last_unstale":"2023-10-06T03:49:47.785417+0000","last_undegraded":"2023-10-06T03:49:47.785417+0000","last_fullsized":"2023-10-06T03:49:47.785417+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:02:45.152283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":90,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.772864+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:49:47.772864+0000","last_peered":"2023-10-06T03:49:47.772864+0000","last_clean":"2023-10-06T03:49:47.772864+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:49:47.772864+0000","last_undegraded":"2023-10-06T03:49:47.772864+0000","last_fullsized":"2023-10-06T03:49:47.772864+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.783138+0000","last_change":"2023-10-06T03:49:47.777651+0000","last_active":"2023-10-06T03:49:47.783138+0000","last_peered":"2023-10-06T03:49:47.783138+0000","last_clean":"2023-10-06T03:49:47.783138+0000","last_became_active":"2023-10-06T03:49:47.775150+0000","last_became_peered":"2023-10-06T03:49:47.775150+0000","last_unstale":"2023-10-06T03:49:47.783138+0000","last_undegraded":"2023-10-06T03:49:47.783138+0000","last_fullsized":"2023-10-06T03:49:47.783138+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:19:48.866890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"106'1","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.806301+0000","last_change":"2023-10-06T03:49:46.777556+0000","last_active":"2023-10-06T03:49:47.806301+0000","last_peered":"2023-10-06T03:49:47.806301+0000","last_clean":"2023-10-06T03:49:47.806301+0000","last_became_active":"2023-10-06T03:49:46.777193+0000","last_became_peered":"2023-10-06T03:49:46.777193+0000","last_unstale":"2023-10-06T03:49:47.806301+0000","last_undegraded":"2023-10-06T03:49:47.806301+0000","last_fullsized":"2023-10-06T03:49:47.806301+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:59.899446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.773892+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:49:47.773892+0000","last_peered":"2023-10-06T03:49:47.773892+0000","last_clean":"2023-10-06T03:49:47.773892+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:49:47.773892+0000","last_undegraded":"2023-10-06T03:49:47.773892+0000","last_fullsized":"2023-10-06T03:49:47.773892+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.786456+0000","last_change":"2023-10-06T03:49:47.783536+0000","last_active":"2023-10-06T03:49:47.786456+0000","last_peered":"2023-10-06T03:49:47.786456+0000","last_clean":"2023-10-06T03:49:47.786456+0000","last_became_active":"2023-10-06T03:49:47.783261+0000","last_became_peered":"2023-10-06T03:49:47.783261+0000","last_unstale":"2023-10-06T03:49:47.786456+0000","last_undegraded":"2023-10-06T03:49:47.786456+0000","last_fullsized":"2023-10-06T03:49:47.786456+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:30:10.207499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.767192+0000","last_change":"2023-10-06T03:49:46.786749+0000","last_active":"2023-10-06T03:49:47.767192+0000","last_peered":"2023-10-06T03:49:47.767192+0000","last_clean":"2023-10-06T03:49:47.767192+0000","last_became_active":"2023-10-06T03:49:46.786351+0000","last_became_peered":"2023-10-06T03:49:46.786351+0000","last_unstale":"2023-10-06T03:49:47.767192+0000","last_undegraded":"2023-10-06T03:49:47.767192+0000","last_fullsized":"2023-10-06T03:49:47.767192+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:16:47.190140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.767133+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:49:47.767133+0000","last_peered":"2023-10-06T03:49:47.767133+0000","last_clean":"2023-10-06T03:49:47.767133+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:49:47.767133+0000","last_undegraded":"2023-10-06T03:49:47.767133+0000","last_fullsized":"2023-10-06T03:49:47.767133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.787304+0000","last_change":"2023-10-06T03:49:47.785019+0000","last_active":"2023-10-06T03:49:47.787304+0000","last_peered":"2023-10-06T03:49:47.787304+0000","last_clean":"2023-10-06T03:49:47.787304+0000","last_became_active":"2023-10-06T03:49:47.784599+0000","last_became_peered":"2023-10-06T03:49:47.784599+0000","last_unstale":"2023-10-06T03:49:47.787304+0000","last_undegraded":"2023-10-06T03:49:47.787304+0000","last_fullsized":"2023-10-06T03:49:47.787304+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:46.752975+0000","last_clean_scrub_stamp":"2023-10-06T03:49:46.752975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:39.982048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"106'2","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.804948+0000","last_change":"2023-10-06T03:49:46.775487+0000","last_active":"2023-10-06T03:49:47.804948+0000","last_peered":"2023-10-06T03:49:47.804948+0000","last_clean":"2023-10-06T03:49:47.804948+0000","last_became_active":"2023-10-06T03:49:46.775203+0000","last_became_peered":"2023-10-06T03:49:46.775203+0000","last_unstale":"2023-10-06T03:49:47.804948+0000","last_undegraded":"2023-10-06T03:49:47.804948+0000","last_fullsized":"2023-10-06T03:49:47.804948+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:49:45.744985+0000","last_clean_scrub_stamp":"2023-10-06T03:49:45.744985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:09:47.477502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-10-06T03:49:47.767704+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:49:47.767704+0000","last_peered":"2023-10-06T03:49:47.767704+0000","last_clean":"2023-10-06T03:49:47.767704+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:49:47.767704+0000","last_undegraded":"2023-10-06T03:49:47.767704+0000","last_fullsized":"2023-10-06T03:49:47.767704+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575045,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311432,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060408,"statfs":{"total":96636764160,"available":96317857792,"internally_reserved":0,"allocated":1900544,"data_stored":1253096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.66800000000000004,"15min":0.69699999999999995},"min":{"1min":0.377,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.91300000000000003,"5min":1.0029999999999999,"15min":1.6100000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.66500000000000004,"15min":0.70399999999999996},"min":{"1min":0.46000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.88700000000000001,"5min":1.02,"15min":1.5649999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.70599999999999996,"15min":0.749},"min":{"1min":0.41199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.95499999999999996,"5min":1.038,"15min":1.5469999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.72999999999999998,"15min":0.76300000000000001},"min":{"1min":0.49199999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.96199999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.67100000000000004,"15min":0.69699999999999995},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82599999999999996,"5min":0.97499999999999998,"15min":1.5900000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.73699999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91900000000000004,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.70499999999999996,"15min":0.72399999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39500000000000002},"max":{"1min":0.81799999999999995,"5min":1.016,"15min":1.6279999999999999},"last":0.63},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72999999999999998,"15min":0.74199999999999999},"min":{"1min":0.52900000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.0409999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.60999999999999999,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.0049999999999999},"last":0.754},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63,"15min":0.622},"min":{"1min":0.45700000000000002,"5min":0.41299999999999998,"15min":0.39400000000000002},"max":{"1min":0.90600000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.57799999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.65300000000000002,"15min":0.63900000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.069},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.69199999999999995,"15min":0.68000000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.436},"max":{"1min":1.0640000000000001,"5min":1.105,"15min":1.105},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.746,"15min":0.72599999999999998},"min":{"1min":0.47499999999999998,"5min":0.436,"15min":0.436},"max":{"1min":1.1220000000000001,"5min":1.181,"15min":1.181},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71699999999999997,"15min":0.70399999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.73899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575046,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306772,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065068,"statfs":{"total":96636764160,"available":96322629632,"internally_reserved":0,"allocated":1323008,"data_stored":665762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68700000000000006,"15min":0.70099999999999996},"min":{"1min":0.39400000000000002,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.47399999999999998,"5min":0.41299999999999998,"15min":0.39500000000000002},"max":{"1min":0.84999999999999998,"5min":0.98599999999999999,"15min":1.0069999999999999},"last":0.67300000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.41599999999999998,"5min":0.36499999999999999,"15min":0.33800000000000002},"max":{"1min":0.83199999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71199999999999997,"15min":0.71499999999999997},"min":{"1min":0.52500000000000002,"5min":0.441,"15min":0.35899999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73699999999999999,"15min":0.74299999999999999},"min":{"1min":0.54900000000000004,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.71699999999999997,"15min":0.73399999999999999},"min":{"1min":0.42099999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78500000000000003,"5min":1.0209999999999999,"15min":1.0409999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.44600000000000001},"max":{"1min":0.83299999999999996,"5min":1.01,"15min":1.024},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.56999999999999995,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.097,"5min":1.097,"15min":1.1639999999999999},"last":0.624}]},{"osd":4,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.58499999999999996,"15min":0.57599999999999996},"min":{"1min":0.44,"5min":0.38200000000000001,"15min":0.34799999999999998},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63700000000000001,"15min":0.63200000000000001},"min":{"1min":0.47299999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.67900000000000005,"15min":0.66400000000000003},"min":{"1min":0.59999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0489999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71999999999999997,"15min":0.71299999999999997},"min":{"1min":0.503,"5min":0.45900000000000002,"15min":0.41199999999999998},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.753},{"interface":"front","average":{"1min":0.754,"5min":0.70699999999999996,"15min":0.70599999999999996},"min":{"1min":0.54000000000000004,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.77300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575045,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307336,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064504,"statfs":{"total":96636764160,"available":96322052096,"internally_reserved":0,"allocated":1900544,"data_stored":1254088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47199999999999998,"15min":0.48599999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.024,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47999999999999998,"15min":0.49299999999999999},"min":{"1min":0.34300000000000003,"5min":0.32000000000000001,"15min":0.309},"max":{"1min":0.65600000000000003,"5min":0.72699999999999998,"15min":0.84799999999999998},"last":0.375},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54600000000000004,"15min":0.55500000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.626,"5min":0.77300000000000002,"15min":0.94099999999999995},"last":0.47399999999999998}]},{"osd":3,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59299999999999997,"15min":0.60299999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.73099999999999998,"5min":0.86699999999999999,"15min":0.91700000000000004},"last":0.622},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.53000000000000003,"15min":0.53500000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.32500000000000001},"max":{"1min":0.74299999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.498}]},{"osd":4,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.75600000000000001,"15min":0.76100000000000001},"min":{"1min":0.496,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0289999999999999,"5min":1.0389999999999999,"15min":1.1699999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.78400000000000003,"15min":0.79900000000000004},"min":{"1min":0.52500000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.98899999999999999,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.77800000000000002,"15min":0.78900000000000003},"min":{"1min":0.53800000000000003,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":1.0089999999999999,"5min":1.294,"15min":1.294},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.79200000000000004,"15min":0.80400000000000005},"min":{"1min":0.55000000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.008,"5min":1.224,"15min":1.224},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78600000000000003,"15min":0.80600000000000005},"min":{"1min":0.53100000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.077,"5min":1.26,"15min":1.26},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.79800000000000004,"15min":0.81799999999999995},"min":{"1min":0.59599999999999997,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0720000000000001,"5min":1.143,"15min":1.224},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.81899999999999995,"15min":0.83699999999999997},"min":{"1min":0.59299999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.038,"5min":1.272,"15min":1.272},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.59999999999999998,"5min":0.53700000000000003,"15min":0.51200000000000001},"max":{"1min":1.0009999999999999,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.64700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575047,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306772,"kb_used_data":1292,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065068,"statfs":{"total":96636764160,"available":96322629632,"internally_reserved":0,"allocated":1323008,"data_stored":664886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.47799999999999998},"min":{"1min":0.33800000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.72599999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.495,"5min":0.496,"15min":0.51800000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.70999999999999996,"5min":0.78500000000000003,"15min":0.83599999999999997},"last":0.58699999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.54600000000000004},"min":{"1min":0.40000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.69499999999999995,"5min":0.80100000000000005,"15min":1.1299999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.52900000000000003,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.80100000000000005,"5min":0.83399999999999996,"15min":0.89400000000000002},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.318,"5min":0.30499999999999999,"15min":0.28399999999999997},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.93200000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59599999999999997,"15min":0.61099999999999999},"min":{"1min":0.432,"5min":0.378,"15min":0.378},"max":{"1min":0.91800000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.80000000000000004,"15min":0.82999999999999996},"min":{"1min":0.56200000000000006,"5min":0.54200000000000004,"15min":0.49399999999999999},"max":{"1min":1.0189999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.79100000000000004,"15min":0.82299999999999995},"min":{"1min":0.56200000000000006,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1279999999999999,"5min":1.306,"15min":1.3400000000000001},"last":0.96499999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.78500000000000003,"15min":0.81299999999999994},"min":{"1min":0.58299999999999996,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0629999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.749,"5min":0.76800000000000002,"15min":0.80500000000000005},"min":{"1min":0.53500000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98599999999999999,"5min":1.1499999999999999,"15min":1.6559999999999999},"last":1.194}]},{"osd":6,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.75800000000000001,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.98399999999999999,"5min":1.137,"15min":1.726},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.79700000000000004,"15min":0.83699999999999997},"min":{"1min":0.56899999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.98899999999999999,"5min":1.167,"15min":1.843},"last":1.0580000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:49:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80300000000000005,"15min":0.82199999999999995},"min":{"1min":0.59799999999999998,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.1060000000000001,"5min":1.302,"15min":1.302},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.78700000000000003,"15min":0.83199999999999996},"min":{"1min":0.59599999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.0569999999999999,"5min":1.2450000000000001,"15min":1.7649999999999999},"last":0.875}]}]},{"osd":2,"up_from":13,"seq":55834575046,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310844,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060996,"statfs":{"total":96636764160,"available":96318459904,"internally_reserved":0,"allocated":1298432,"data_stored":662672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46899999999999997,"15min":0.47199999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.28000000000000003},"max":{"1min":0.66700000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.51400000000000001,"15min":0.52600000000000002},"min":{"1min":0.32400000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.89300000000000002},"last":0.435}]},{"osd":1,"last update":"Fri Oct 6 03:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.621,"5min":0.73799999999999999,"15min":0.90000000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.55300000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.94899999999999995},"last":0.46600000000000003}]},{"osd":3,"last update":"Fri Oct 6 03:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57999999999999996,"15min":0.58899999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79000000000000004,"5min":1.78,"15min":1.78},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55700000000000005,"15min":0.57199999999999995},"min":{"1min":0.39100000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.96499999999999997},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80800000000000005,"15min":0.82699999999999996},"min":{"1min":0.58399999999999996,"5min":0.51400000000000001,"15min":0.439},"max":{"1min":1,"5min":1.0649999999999999,"15min":1.5620000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82699999999999996,"15min":0.84199999999999997},"min":{"1min":0.53100000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.05,"5min":1.1279999999999999,"15min":1.2270000000000001},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Oct 6 03:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81899999999999995,"15min":0.83899999999999997},"min":{"1min":0.54900000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.999,"5min":1.169,"15min":1.3},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82599999999999996,"15min":0.85399999999999998},"min":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.51700000000000002},"max":{"1min":0.98599999999999999,"5min":1.855,"15min":1.855},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.81899999999999995,"15min":0.83799999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.026,"5min":1.907,"15min":1.907},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.56999999999999995,"5min":0.54400000000000004,"15min":0.52600000000000002},"max":{"1min":0.98299999999999998,"5min":1.1080000000000001,"15min":1.4750000000000001},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:49:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.83599999999999997,"15min":0.84699999999999998},"min":{"1min":0.59499999999999997,"5min":0.495,"15min":0.495},"max":{"1min":1.022,"5min":1.4379999999999999,"15min":1.4990000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.85299999999999998,"15min":0.871},"min":{"1min":0.54800000000000004,"5min":0.53100000000000003,"15min":0.497},"max":{"1min":1.0860000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.64300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575046,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306752,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065088,"statfs":{"total":96636764160,"available":96322650112,"internally_reserved":0,"allocated":1302528,"data_stored":663650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45600000000000002,"15min":0.45900000000000002},"min":{"1min":0.30599999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.60399999999999998,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48099999999999998,"15min":0.498},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":0.79200000000000004,"5min":1.04,"15min":1.1699999999999999},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47699999999999998,"15min":0.48199999999999998},"min":{"1min":0.32200000000000001,"5min":0.314,"15min":0.29299999999999998},"max":{"1min":0.65800000000000003,"5min":0.68799999999999994,"15min":0.79500000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.53200000000000003,"15min":0.53800000000000003},"min":{"1min":0.40600000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.64300000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.82099999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.41199999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80700000000000005,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.58199999999999996,"15min":0.60299999999999998},"min":{"1min":0.47699999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.73299999999999998,"5min":0.81999999999999995,"15min":0.97899999999999998},"last":0.74199999999999999}]},{"osd":4,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89900000000000002,"5min":1.028,"15min":1.4099999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78300000000000003,"15min":0.82499999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.92700000000000005,"5min":1.0469999999999999,"15min":1.377},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75,"15min":0.78700000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97399999999999998,"5min":1.05,"15min":1.256},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74399999999999999,"15min":0.78200000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.49099999999999999},"max":{"1min":0.88900000000000001,"5min":0.99199999999999999,"15min":1.2030000000000001},"last":0.90200000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77400000000000002,"15min":0.81000000000000005},"min":{"1min":0.56999999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"max":{"1min":0.85199999999999998,"5min":1.089,"15min":1.323},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.751,"5min":0.80500000000000005,"15min":0.84499999999999997},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.073,"5min":1.1040000000000001,"15min":1.2849999999999999},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:49:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.77300000000000002,"15min":0.81000000000000005},"min":{"1min":0.55900000000000005,"5min":0.498,"15min":0.498},"max":{"1min":0.94799999999999995,"5min":1.0700000000000001,"15min":1.2470000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.77500000000000002,"15min":0.80400000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.89000000000000001,"5min":1.093,"15min":1.2330000000000001},"last":0.878}]}]},{"osd":4,"up_from":13,"seq":55834575046,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1310720,"data_stored":663810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:49:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67800000000000005,"15min":0.71699999999999997},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.316},"max":{"1min":0.81499999999999995,"5min":0.96999999999999997,"15min":1.022},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.49299999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":1.01},"last":0.63100000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:49:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.68500000000000005,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.35899999999999999,"15min":0.33200000000000002},"max":{"1min":0.89700000000000002,"5min":1.081,"15min":1.081},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70899999999999996,"15min":0.73399999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41299999999999998},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.044},"last":0.75}]},{"osd":2,"last update":"Fri Oct 6 03:49:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69799999999999995,"15min":0.73499999999999999},"min":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.377},"max":{"1min":0.96399999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.72699999999999998,"15min":0.74399999999999999},"min":{"1min":0.51200000000000001,"5min":0.48799999999999999,"15min":0.48499999999999999},"max":{"1min":0.93799999999999994,"5min":1.0109999999999999,"15min":1.0660000000000001},"last":0.70399999999999996}]},{"osd":3,"last update":"Fri Oct 6 03:49:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.72699999999999998},"min":{"1min":0.46200000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0409999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.73999999999999999,"15min":0.76400000000000001},"min":{"1min":0.51200000000000001,"5min":0.47099999999999997,"15min":0.46500000000000002},"max":{"1min":0.97599999999999998,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:49:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.627,"15min":0.622},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.98499999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.629,"15min":0.63},"min":{"1min":0.44,"5min":0.41499999999999998,"15min":0.35099999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.96799999999999997},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Oct 6 03:49:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65700000000000003,"15min":0.64300000000000002},"min":{"1min":0.434,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.093,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70099999999999996,"15min":0.68899999999999995},"min":{"1min":0.46500000000000002,"5min":0.42599999999999999,"15min":0.38100000000000001},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Oct 6 03:49:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.39900000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.133},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72499999999999998,"15min":0.71599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.41299999999999998},"max":{"1min":1.1870000000000001,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.61899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575046,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302652,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069188,"statfs":{"total":96636764160,"available":96326848512,"internally_reserved":0,"allocated":1298432,"data_stored":662672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68700000000000006,"15min":0.70999999999999996},"min":{"1min":0.443,"5min":0.35999999999999999,"15min":0.35299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.48899999999999999,"5min":0.42399999999999999,"15min":0.32600000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70099999999999996,"15min":0.71899999999999997},"min":{"1min":0.54800000000000004,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.0429999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70599999999999996,"15min":0.72799999999999998},"min":{"1min":0.48599999999999999,"5min":0.44,"15min":0.32400000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.81799999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.754,"15min":0.77500000000000002},"min":{"1min":0.54100000000000004,"5min":0.502,"15min":0.40200000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.749,"15min":0.77400000000000002},"min":{"1min":0.52800000000000002,"5min":0.51200000000000001,"15min":0.50700000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.76600000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70899999999999996,"15min":0.73099999999999998},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.95099999999999996,"5min":1.091,"15min":1.6419999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.748,"15min":0.751},"min":{"1min":0.50600000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.64200000000000002,"15min":0.61599999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63500000000000001,"15min":0.628},"min":{"1min":0.371,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.52300000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.66600000000000004,"15min":0.64600000000000002},"min":{"1min":0.47499999999999998,"5min":0.40200000000000002,"15min":0.34999999999999998},"max":{"1min":0.92400000000000004,"5min":1.591,"15min":1.591},"last":0.498},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.71699999999999997,"15min":0.69499999999999995},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.196,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.78300000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:49:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68300000000000005,"15min":0.67600000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.94899999999999995,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.72499999999999998,"15min":0.71799999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.056,"5min":1.278,"15min":1.278},"last":0.55100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:49:52.632 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:49:52.632 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:49:52.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.858+0000 7f856279d700 1 Processor -- start 2023-10-06T03:49:52.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.858+0000 7f856279d700 1 -- start start 2023-10-06T03:49:52.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.860+0000 7f856279d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c06af90 0x7f855c0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:52.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.860+0000 7f856279d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f855c06b360 0x7f855c0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.860+0000 7f856279d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855c0c9860 con 0x7f855c15e020 2023-10-06T03:49:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.860+0000 7f856279d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f855c0c8cd0 con 0x7f855c06b360 2023-10-06T03:49:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.860+0000 7f856279d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855c0cd200 con 0x7f855c06af90 2023-10-06T03:49:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.860+0000 7f8560f9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f855c06b360 0x7f855c0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.860+0000 7f8560f9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f855c06b360 0x7f855c0ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55980/0 (socket says 172.21.15.55:55980) 2023-10-06T03:49:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.860+0000 7f8560f9a700 1 -- 172.21.15.55:0/2927677767 learned_addr learned my addr 172.21.15.55:0/2927677767 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.860+0000 7f856179b700 1 --2- 172.21.15.55:0/2927677767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c06af90 0x7f855c0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.860+0000 7f8560f9a700 1 -- 172.21.15.55:0/2927677767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c06af90 msgr2=0x7f855c0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.860+0000 7f8560f9a700 1 --2- 172.21.15.55:0/2927677767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c06af90 0x7f855c0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.860+0000 7f8560f9a700 1 -- 172.21.15.55:0/2927677767 >> v1:172.21.15.163:6789/0 conn(0x7f855c15e020 legacy=0x7f855c06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.860+0000 7f8560f9a700 1 -- 172.21.15.55:0/2927677767 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855c15e4c0 con 0x7f855c06b360 2023-10-06T03:49:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.860+0000 7f856179b700 1 --2- 172.21.15.55:0/2927677767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c06af90 0x7f855c0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.861+0000 7f8560f9a700 1 --2- 172.21.15.55:0/2927677767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f855c06b360 0x7f855c0ccb00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f854c0099b0 tx=0x7f854c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907aeeb42bb17bc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.861+0000 7f8553fff700 1 -- 172.21.15.55:0/2927677767 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c00b8b0 con 0x7f855c06b360 2023-10-06T03:49:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.861+0000 7f8553fff700 1 -- 172.21.15.55:0/2927677767 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854c00ba10 con 0x7f855c06b360 2023-10-06T03:49:52.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.861+0000 7f8553fff700 1 -- 172.21.15.55:0/2927677767 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c00bdb0 con 0x7f855c06b360 2023-10-06T03:49:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.861+0000 7f856279d700 1 -- 172.21.15.55:0/2927677767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f855c06b360 msgr2=0x7f855c0ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.861+0000 7f856279d700 1 --2- 172.21.15.55:0/2927677767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f855c06b360 0x7f855c0ccb00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f854c0099b0 tx=0x7f854c009250 comp rx=0 tx=0).stop 2023-10-06T03:49:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.862+0000 7f856279d700 1 -- 172.21.15.55:0/2927677767 shutdown_connections 2023-10-06T03:49:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.862+0000 7f856279d700 1 --2- 172.21.15.55:0/2927677767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c06af90 0x7f855c0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.862+0000 7f856279d700 1 --2- 172.21.15.55:0/2927677767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f855c06b360 0x7f855c0ccb00 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.862+0000 7f856279d700 1 -- 172.21.15.55:0/2927677767 >> 172.21.15.55:0/2927677767 conn(0x7f855c06baa0 msgr2=0x7f855c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:52.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.862+0000 7f856279d700 1 -- 172.21.15.55:0/2927677767 shutdown_connections 2023-10-06T03:49:52.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.862+0000 7f856279d700 1 -- 172.21.15.55:0/2927677767 wait complete. 2023-10-06T03:49:52.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.863+0000 7f856279d700 1 Processor -- start 2023-10-06T03:49:52.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.863+0000 7f856279d700 1 -- start start 2023-10-06T03:49:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.864+0000 7f856279d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f855c06af90 0x7f855c154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.864+0000 7f856279d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f855c06b360 0x7f855c0c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.864+0000 7f856279d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c15e020 0x7f855c0c3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.864+0000 7f856279d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c0ae5b0 con 0x7f855c06af90 2023-10-06T03:49:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.864+0000 7f856279d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c0ae280 con 0x7f855c06b360 2023-10-06T03:49:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.864+0000 7f856279d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855c0c7720 con 0x7f855c15e020 2023-10-06T03:49:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.864+0000 7f856179b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f855c06af90 0x7f855c154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.864+0000 7f856179b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f855c06af90 0x7f855c154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:55986/0 (socket says 172.21.15.55:55986) 2023-10-06T03:49:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.864+0000 7f856179b700 1 -- 172.21.15.55:0/3981669851 learned_addr learned my addr 172.21.15.55:0/3981669851 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:52.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.865+0000 7f8561f9c700 1 --2- 172.21.15.55:0/3981669851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c15e020 0x7f855c0c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.865+0000 7f8560f9a700 1 --2- 172.21.15.55:0/3981669851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f855c06b360 0x7f855c0c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.865+0000 7f856179b700 1 -- 172.21.15.55:0/3981669851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c15e020 msgr2=0x7f855c0c3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.865+0000 7f856179b700 1 --2- 172.21.15.55:0/3981669851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c15e020 0x7f855c0c3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.865+0000 7f856179b700 1 -- 172.21.15.55:0/3981669851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f855c06b360 msgr2=0x7f855c0c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.865+0000 7f856179b700 1 --2- 172.21.15.55:0/3981669851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f855c06b360 0x7f855c0c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.865+0000 7f856179b700 1 -- 172.21.15.55:0/3981669851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c0094f0 con 0x7f855c06af90 2023-10-06T03:49:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.865+0000 7f856179b700 1 --2- 172.21.15.55:0/3981669851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f855c06af90 0x7f855c154210 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f855800c830 tx=0x7f8558008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.865+0000 7f85517fa700 1 -- 172.21.15.55:0/3981669851 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558011600 con 0x7f855c06af90 2023-10-06T03:49:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.865+0000 7f85517fa700 1 -- 172.21.15.55:0/3981669851 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8558011760 con 0x7f855c06af90 2023-10-06T03:49:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.865+0000 7f85517fa700 1 -- 172.21.15.55:0/3981669851 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558011b20 con 0x7f855c06af90 2023-10-06T03:49:52.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.865+0000 7f856279d700 1 -- 172.21.15.55:0/3981669851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855c0c6e80 con 0x7f855c06af90 2023-10-06T03:49:52.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.865+0000 7f856279d700 1 -- 172.21.15.55:0/3981669851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855c0c7440 con 0x7f855c06af90 2023-10-06T03:49:52.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.866+0000 7f856279d700 1 -- 172.21.15.55:0/3981669851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855c04c3c0 con 0x7f855c06af90 2023-10-06T03:49:52.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.867+0000 7f85517fa700 1 -- 172.21.15.55:0/3981669851 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f855800a3e0 con 0x7f855c06af90 2023-10-06T03:49:52.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.868+0000 7f85517fa700 1 --2- 172.21.15.55:0/3981669851 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8540066f30 0x7f85400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:52.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.868+0000 7f85517fa700 1 -- 172.21.15.55:0/3981669851 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f8558084dd0 con 0x7f855c06af90 2023-10-06T03:49:52.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.869+0000 7f8560f9a700 1 --2- 172.21.15.55:0/3981669851 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8540066f30 0x7f85400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:52.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.869+0000 7f8560f9a700 1 --2- 172.21.15.55:0/3981669851 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8540066f30 0x7f85400693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f854c0099b0 tx=0x7f854c016000 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:52.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:52.873+0000 7f85517fa700 1 -- 172.21.15.55:0/3981669851 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8558051cb0 con 0x7f855c06af90 2023-10-06T03:49:53.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.155+0000 7f856279d700 1 -- 172.21.15.55:0/3981669851 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f855c0cbb90 con 0x7f855c06af90 2023-10-06T03:49:53.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.160+0000 7f85517fa700 1 -- 172.21.15.55:0/3981669851 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f85580150e0 con 0x7f855c06af90 2023-10-06T03:49:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.163+0000 7f856279d700 1 -- 172.21.15.55:0/3981669851 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8540066f30 msgr2=0x7f85400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.163+0000 7f856279d700 1 --2- 172.21.15.55:0/3981669851 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8540066f30 0x7f85400693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f854c0099b0 tx=0x7f854c016000 comp rx=0 tx=0).stop 2023-10-06T03:49:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.163+0000 7f856279d700 1 -- 172.21.15.55:0/3981669851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f855c06af90 msgr2=0x7f855c154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.163+0000 7f856279d700 1 --2- 172.21.15.55:0/3981669851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f855c06af90 0x7f855c154210 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f855800c830 tx=0x7f8558008040 comp rx=0 tx=0).stop 2023-10-06T03:49:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.163+0000 7f856279d700 1 -- 172.21.15.55:0/3981669851 shutdown_connections 2023-10-06T03:49:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.163+0000 7f856279d700 1 --2- 172.21.15.55:0/3981669851 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8540066f30 0x7f85400693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.163+0000 7f856279d700 1 --2- 172.21.15.55:0/3981669851 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f855c15e020 0x7f855c0c3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.163+0000 7f856279d700 1 --2- 172.21.15.55:0/3981669851 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f855c06af90 0x7f855c154210 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.163+0000 7f856279d700 1 --2- 172.21.15.55:0/3981669851 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f855c06b360 0x7f855c0c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.163+0000 7f856279d700 1 -- 172.21.15.55:0/3981669851 >> 172.21.15.55:0/3981669851 conn(0x7f855c06baa0 msgr2=0x7f855c0cb220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:53.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.163+0000 7f856279d700 1 -- 172.21.15.55:0/3981669851 shutdown_connections 2023-10-06T03:49:53.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.163+0000 7f856279d700 1 -- 172.21.15.55:0/3981669851 wait complete. 2023-10-06T03:49:53.166 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2023-10-06T03:49:53.181 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:49:53.369 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.368+0000 7f4ee599d700 1 Processor -- start 2023-10-06T03:49:53.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.368+0000 7f4ee599d700 1 -- start start 2023-10-06T03:49:53.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.369+0000 7f4ee599d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4ee0159420 0x7f4ee00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:53.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.369+0000 7f4ee599d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee00b2980 con 0x7f4ee01597f0 2023-10-06T03:49:53.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.369+0000 7f4ee599d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee00b11d0 con 0x7f4ee015d230 2023-10-06T03:49:53.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.369+0000 7f4ee599d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee00b14b0 con 0x7f4ee0159420 2023-10-06T03:49:53.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.370+0000 7f4ee499b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4ee0159420 0x7f4ee00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:53.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.370+0000 7f4edffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4ee01597f0 0x7f4ee00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39554/0 (socket says 172.21.15.55:39554) 2023-10-06T03:49:53.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.370+0000 7f4edffff700 1 -- 172.21.15.55:0/2977375445 learned_addr learned my addr 172.21.15.55:0/2977375445 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:53.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.370+0000 7f4ee499b700 1 -- 172.21.15.55:0/2977375445 >> v1:172.21.15.163:6789/0 conn(0x7f4ee015d230 legacy=0x7f4ee015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:53.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.370+0000 7f4ee499b700 1 -- 172.21.15.55:0/2977375445 >> v1:172.21.15.55:6789/0 conn(0x7f4ee01597f0 legacy=0x7f4ee00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:53.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.370+0000 7f4ee499b700 1 -- 172.21.15.55:0/2977375445 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee00b1790 con 0x7f4ee0159420 2023-10-06T03:49:53.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.370+0000 7f4edf7fe700 1 -- 172.21.15.55:0/2977375445 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476654062 0 0) 0x7f4ee00b2980 con 0x7f4ee01597f0 2023-10-06T03:49:53.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.370+0000 7f4ee499b700 1 --2- 172.21.15.55:0/2977375445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee0159420 0x7f4ee00b5f40 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40098e0 tx=0x7f4ed40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a100e223fe12074 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:53.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.371+0000 7f4edf7fe700 1 -- 172.21.15.55:0/2977375445 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4009e80 con 0x7f4ee0159420 2023-10-06T03:49:53.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.371+0000 7f4edf7fe700 1 -- 172.21.15.55:0/2977375445 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed400b860 con 0x7f4ee0159420 2023-10-06T03:49:53.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.371+0000 7f4edf7fe700 1 -- 172.21.15.55:0/2977375445 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400bbe0 con 0x7f4ee0159420 2023-10-06T03:49:53.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.371+0000 7f4ee599d700 1 -- 172.21.15.55:0/2977375445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee0159420 msgr2=0x7f4ee00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:53.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.371+0000 7f4ee599d700 1 --2- 172.21.15.55:0/2977375445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee0159420 0x7f4ee00b5f40 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40098e0 tx=0x7f4ed40091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:53.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.371+0000 7f4ee599d700 1 -- 172.21.15.55:0/2977375445 shutdown_connections 2023-10-06T03:49:53.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.371+0000 7f4ee599d700 1 --2- 172.21.15.55:0/2977375445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee0159420 0x7f4ee00b5f40 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:53.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.371+0000 7f4ee599d700 1 -- 172.21.15.55:0/2977375445 >> 172.21.15.55:0/2977375445 conn(0x7f4ee00d0660 msgr2=0x7f4ee00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:53.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.371+0000 7f4ee599d700 1 -- 172.21.15.55:0/2977375445 shutdown_connections 2023-10-06T03:49:53.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.371+0000 7f4ee599d700 1 -- 172.21.15.55:0/2977375445 wait complete. 2023-10-06T03:49:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.373+0000 7f4ee599d700 1 Processor -- start 2023-10-06T03:49:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.373+0000 7f4ee599d700 1 -- start start 2023-10-06T03:49:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.374+0000 7f4ee599d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee0159420 0x7f4ee01552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.374+0000 7f4ee599d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee01597f0 0x7f4ee00ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.374+0000 7f4ee599d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee015d230 0x7f4ee00cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.374+0000 7f4ee599d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee0155810 con 0x7f4ee0159420 2023-10-06T03:49:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.374+0000 7f4ee599d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee00cae80 con 0x7f4ee015d230 2023-10-06T03:49:53.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.374+0000 7f4ee599d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee00cb160 con 0x7f4ee01597f0 2023-10-06T03:49:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.374+0000 7f4ee499b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee0159420 0x7f4ee01552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.374+0000 7f4ee499b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee0159420 0x7f4ee01552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51566/0 (socket says 172.21.15.55:51566) 2023-10-06T03:49:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.374+0000 7f4ee499b700 1 -- 172.21.15.55:0/1892305692 learned_addr learned my addr 172.21.15.55:0/1892305692 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.375+0000 7f4edffff700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee01597f0 0x7f4ee00ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.375+0000 7f4ee499b700 1 -- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee01597f0 msgr2=0x7f4ee00ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.375+0000 7f4ee499b700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee01597f0 0x7f4ee00ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.375+0000 7f4ee499b700 1 -- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee015d230 msgr2=0x7f4ee00cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.375+0000 7f4ee519c700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee015d230 0x7f4ee00cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.375+0000 7f4ee499b700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee015d230 0x7f4ee00cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:53.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.375+0000 7f4ee499b700 1 -- 172.21.15.55:0/1892305692 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed4009420 con 0x7f4ee0159420 2023-10-06T03:49:53.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.375+0000 7f4edffff700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee01597f0 0x7f4ee00ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:53.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.375+0000 7f4ee519c700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee015d230 0x7f4ee00cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:53.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.375+0000 7f4ee499b700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee0159420 0x7f4ee01552d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0089340 tx=0x7f4ed4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:53.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.375+0000 7f4edcff9700 1 -- 172.21.15.55:0/1892305692 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4011420 con 0x7f4ee0159420 2023-10-06T03:49:53.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.375+0000 7f4edcff9700 1 -- 172.21.15.55:0/1892305692 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed4011580 con 0x7f4ee0159420 2023-10-06T03:49:53.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.375+0000 7f4edcff9700 1 -- 172.21.15.55:0/1892305692 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4011940 con 0x7f4ee0159420 2023-10-06T03:49:53.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.375+0000 7f4ee599d700 1 -- 172.21.15.55:0/1892305692 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee00cb440 con 0x7f4ee0159420 2023-10-06T03:49:53.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.376+0000 7f4ee599d700 1 -- 172.21.15.55:0/1892305692 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee00b4dc0 con 0x7f4ee0159420 2023-10-06T03:49:53.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.376+0000 7f4ee599d700 1 -- 172.21.15.55:0/1892305692 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee00b4fb0 con 0x7f4ee0159420 2023-10-06T03:49:53.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.377+0000 7f4edcff9700 1 -- 172.21.15.55:0/1892305692 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ed4011cf0 con 0x7f4ee0159420 2023-10-06T03:49:53.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.378+0000 7f4edcff9700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4ec4066f30 0x7f4ec40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:53.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.378+0000 7f4edcff9700 1 -- 172.21.15.55:0/1892305692 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f4ed4086100 con 0x7f4ee0159420 2023-10-06T03:49:53.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.379+0000 7f4edffff700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4ec4066f30 0x7f4ec40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:53.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.379+0000 7f4edffff700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4ec4066f30 0x7f4ec40693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4ee006a080 tx=0x7f4ecc009040 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:53.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.382+0000 7f4edcff9700 1 -- 172.21.15.55:0/1892305692 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ee00b4fb0 con 0x7f4ee0159420 2023-10-06T03:49:53.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.630+0000 7f4ee599d700 1 -- 172.21.15.55:0/1892305692 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ee00b4fb0 con 0x7f4ee0159420 2023-10-06T03:49:53.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.630+0000 7f4edcff9700 1 -- 172.21.15.55:0/1892305692 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f4ee00b4fb0 con 0x7f4ee0159420 2023-10-06T03:49:53.632 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:49:53.632 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2272331626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2272331626},{"type":"v1","addr":"172.21.15.163:6834","nonce":2272331626}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2110964237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2110964237},{"type":"v1","addr":"172.21.15.163:6836","nonce":2110964237}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2942349047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2942349047},{"type":"v1","addr":"172.21.15.55:6833","nonce":2942349047}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:49:47.265041+0000","modified":"2023-10-06T03:49:50.792326+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7254":{"gid":7254,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/2226955335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2226955335},{"type":"v1","addr":"172.21.15.55:6835","nonce":2226955335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T03:49:53.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.633+0000 7f4ee599d700 1 -- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4ec4066f30 msgr2=0x7f4ec40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:53.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.633+0000 7f4ee599d700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4ec4066f30 0x7f4ec40693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4ee006a080 tx=0x7f4ecc009040 comp rx=0 tx=0).stop 2023-10-06T03:49:53.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.633+0000 7f4ee599d700 1 -- 172.21.15.55:0/1892305692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee0159420 msgr2=0x7f4ee01552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:53.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.633+0000 7f4ee599d700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee0159420 0x7f4ee01552d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0089340 tx=0x7f4ed4017000 comp rx=0 tx=0).stop 2023-10-06T03:49:53.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.633+0000 7f4ee599d700 1 -- 172.21.15.55:0/1892305692 shutdown_connections 2023-10-06T03:49:53.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.633+0000 7f4ee599d700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4ec4066f30 0x7f4ec40693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:53.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.633+0000 7f4ee599d700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ee015d230 0x7f4ee00cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:53.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.633+0000 7f4ee599d700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ee0159420 0x7f4ee01552d0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:53.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.633+0000 7f4ee599d700 1 --2- 172.21.15.55:0/1892305692 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ee01597f0 0x7f4ee00ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:53.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.633+0000 7f4ee599d700 1 -- 172.21.15.55:0/1892305692 >> 172.21.15.55:0/1892305692 conn(0x7f4ee00d0660 msgr2=0x7f4ee015d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:53.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.633+0000 7f4ee599d700 1 -- 172.21.15.55:0/1892305692 shutdown_connections 2023-10-06T03:49:53.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:53.633+0000 7f4ee599d700 1 -- 172.21.15.55:0/1892305692 wait complete. 2023-10-06T03:49:53.636 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 53 2023-10-06T03:49:54.055 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:49:47.265041+0000', 'modified': '2023-10-06T03:49:50.792326+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7254}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7254': {'gid': 7254, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6835/2226955335', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6834', 'nonce': 2226955335}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 2226955335}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:49:54.055 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:49:54.055 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T03:49:54.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.244+0000 7f3800d9d700 1 Processor -- start 2023-10-06T03:49:54.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.245+0000 7f3800d9d700 1 -- start start 2023-10-06T03:49:54.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.246+0000 7f3800d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f37fc0c6a70 0x7f37fc06acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.246+0000 7f3800d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37fc0696c0 0x7f37fc0c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.246+0000 7f3800d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37fc06a840 con 0x7f37fc0ca660 2023-10-06T03:49:54.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.246+0000 7f3800d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37fc06b210 con 0x7f37fc0c6a70 2023-10-06T03:49:54.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.246+0000 7f3800d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc069c30 con 0x7f37fc0696c0 2023-10-06T03:49:54.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.246+0000 7f37fbfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f37fc0ca660 0x7f37fc06d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39558/0 (socket says 172.21.15.55:39558) 2023-10-06T03:49:54.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.246+0000 7f37fbfff700 1 -- 172.21.15.55:0/4041500781 learned_addr learned my addr 172.21.15.55:0/4041500781 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:54.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.247+0000 7f37fb7fe700 1 --2- 172.21.15.55:0/4041500781 >> v2:172.21.15.163:3300/0 conn(0x7f37fc0c6a70 0x7f37fc06acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.247+0000 7f37faffd700 1 --2- 172.21.15.55:0/4041500781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37fc0696c0 0x7f37fc0c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.247+0000 7f37fa7fc700 1 -- 172.21.15.55:0/4041500781 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4286594364 0 0) 0x7f37fc06a840 con 0x7f37fc0ca660 2023-10-06T03:49:54.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.247+0000 7f37fa7fc700 1 -- 172.21.15.55:0/4041500781 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e4003570 con 0x7f37fc0ca660 2023-10-06T03:49:54.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.247+0000 7f37fb7fe700 1 -- 172.21.15.55:0/4041500781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37fc0696c0 msgr2=0x7f37fc0c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:54.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.247+0000 7f37fb7fe700 1 --2- 172.21.15.55:0/4041500781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37fc0696c0 0x7f37fc0c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.247+0000 7f37fb7fe700 1 -- 172.21.15.55:0/4041500781 >> v1:172.21.15.55:6789/0 conn(0x7f37fc0ca660 legacy=0x7f37fc06d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.247+0000 7f37fb7fe700 1 -- 172.21.15.55:0/4041500781 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc069f10 con 0x7f37fc0c6a70 2023-10-06T03:49:54.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.247+0000 7f37faffd700 1 --2- 172.21.15.55:0/4041500781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37fc0696c0 0x7f37fc0c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:54.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.247+0000 7f37fb7fe700 1 --2- 172.21.15.55:0/4041500781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc0c6a70 0x7f37fc06acd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0098e0 tx=0x7f37ec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4925e9438b86ce7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:54.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.248+0000 7f37fa7fc700 1 -- 172.21.15.55:0/4041500781 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec011630 con 0x7f37fc0c6a70 2023-10-06T03:49:54.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.248+0000 7f37fa7fc700 1 -- 172.21.15.55:0/4041500781 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec011790 con 0x7f37fc0c6a70 2023-10-06T03:49:54.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.248+0000 7f37fa7fc700 1 -- 172.21.15.55:0/4041500781 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec011b30 con 0x7f37fc0c6a70 2023-10-06T03:49:54.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.248+0000 7f3800d9d700 1 -- 172.21.15.55:0/4041500781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc0c6a70 msgr2=0x7f37fc06acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.248+0000 7f3800d9d700 1 --2- 172.21.15.55:0/4041500781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc0c6a70 0x7f37fc06acd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f37ec0098e0 tx=0x7f37ec0091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.248+0000 7f3800d9d700 1 -- 172.21.15.55:0/4041500781 shutdown_connections 2023-10-06T03:49:54.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.248+0000 7f3800d9d700 1 --2- 172.21.15.55:0/4041500781 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc0c6a70 0x7f37fc06acd0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.248+0000 7f3800d9d700 1 --2- 172.21.15.55:0/4041500781 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37fc0696c0 0x7f37fc0c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.248+0000 7f3800d9d700 1 -- 172.21.15.55:0/4041500781 >> 172.21.15.55:0/4041500781 conn(0x7f37fc0cc1f0 msgr2=0x7f37fc0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:54.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.248+0000 7f3800d9d700 1 -- 172.21.15.55:0/4041500781 shutdown_connections 2023-10-06T03:49:54.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.248+0000 7f3800d9d700 1 -- 172.21.15.55:0/4041500781 wait complete. 2023-10-06T03:49:54.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.250+0000 7f3800d9d700 1 Processor -- start 2023-10-06T03:49:54.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.250+0000 7f3800d9d700 1 -- start start 2023-10-06T03:49:54.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.251+0000 7f3800d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37fc0696c0 0x7f37fc14c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.251+0000 7f3800d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37fc0c6a70 0x7f37fc159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.251+0000 7f3800d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc0ca660 0x7f37fc14d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.251+0000 7f3800d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc0aef90 con 0x7f37fc0c6a70 2023-10-06T03:49:54.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.251+0000 7f3800d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc0c52e0 con 0x7f37fc0ca660 2023-10-06T03:49:54.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.251+0000 7f3800d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc14c7c0 con 0x7f37fc0696c0 2023-10-06T03:49:54.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.251+0000 7f37faffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37fc0c6a70 0x7f37fc159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.251+0000 7f37faffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37fc0c6a70 0x7f37fc159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51568/0 (socket says 172.21.15.55:51568) 2023-10-06T03:49:54.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.251+0000 7f37faffd700 1 -- 172.21.15.55:0/3199205542 learned_addr learned my addr 172.21.15.55:0/3199205542 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:54.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.251+0000 7f37fb7fe700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37fc0696c0 0x7f37fc14c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.251+0000 7f37faffd700 1 -- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37fc0696c0 msgr2=0x7f37fc14c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.251+0000 7f37fbfff700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc0ca660 0x7f37fc14d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.251+0000 7f37faffd700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37fc0696c0 0x7f37fc14c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.252+0000 7f37faffd700 1 -- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc0ca660 msgr2=0x7f37fc14d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.252+0000 7f37faffd700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc0ca660 0x7f37fc14d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.252+0000 7f37faffd700 1 -- 172.21.15.55:0/3199205542 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f0009420 con 0x7f37fc0c6a70 2023-10-06T03:49:54.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.252+0000 7f37fb7fe700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37fc0696c0 0x7f37fc14c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:49:54.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.252+0000 7f37faffd700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37fc0c6a70 0x7f37fc159370 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f37f000ec80 tx=0x7f37f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:54.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.252+0000 7f37e37fe700 1 -- 172.21.15.55:0/3199205542 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f000c9d0 con 0x7f37fc0c6a70 2023-10-06T03:49:54.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.252+0000 7f37e37fe700 1 -- 172.21.15.55:0/3199205542 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f000cb30 con 0x7f37fc0c6a70 2023-10-06T03:49:54.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.252+0000 7f37e37fe700 1 -- 172.21.15.55:0/3199205542 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f000ceb0 con 0x7f37fc0c6a70 2023-10-06T03:49:54.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.252+0000 7f3800d9d700 1 -- 172.21.15.55:0/3199205542 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ec009420 con 0x7f37fc0c6a70 2023-10-06T03:49:54.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.252+0000 7f3800d9d700 1 -- 172.21.15.55:0/3199205542 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37fc0c49b0 con 0x7f37fc0c6a70 2023-10-06T03:49:54.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.255+0000 7f37e37fe700 1 -- 172.21.15.55:0/3199205542 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f37f0013680 con 0x7f37fc0c6a70 2023-10-06T03:49:54.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.256+0000 7f37e37fe700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f37e8066f30 0x7f37e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.256+0000 7f37e37fe700 1 -- 172.21.15.55:0/3199205542 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f37f0084d10 con 0x7f37fc0c6a70 2023-10-06T03:49:54.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.256+0000 7f37fb7fe700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f37e8066f30 0x7f37e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.257+0000 7f37fb7fe700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f37e8066f30 0x7f37e80693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f37ec003080 tx=0x7f37ec017000 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:54.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.258+0000 7f38106d4b80 1 Processor -- start 2023-10-06T03:49:54.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.258+0000 7f38106d4b80 1 -- start start 2023-10-06T03:49:54.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.259+0000 7f38106d4b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 0x55c1e10b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.259+0000 7f38106d4b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10be2c0 0x55c1e10b9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.259+0000 7f38106d4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10bfa10 0x55c1e10b9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.259+0000 7f38106d4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e1013a70 con 0x55c1e10bfa10 2023-10-06T03:49:54.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.259+0000 7f38106d4b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e1011270 con 0x55c1e10be2c0 2023-10-06T03:49:54.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.259+0000 7f38106d4b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1e100d060 con 0x55c1e10dde50 2023-10-06T03:49:54.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.259+0000 7f37fbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10bfa10 0x55c1e10b9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.259+0000 7f37fbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10bfa10 0x55c1e10b9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51574/0 (socket says 172.21.15.55:51574) 2023-10-06T03:49:54.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.259+0000 7f37fbfff700 1 -- 172.21.15.55:0/1803986706 learned_addr learned my addr 172.21.15.55:0/1803986706 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:54.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.259+0000 7f37fb7fe700 1 --2- 172.21.15.55:0/1803986706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 0x55c1e10b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.259+0000 7f37faffd700 1 --2- 172.21.15.55:0/1803986706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10be2c0 0x55c1e10b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.260+0000 7f37fbfff700 1 -- 172.21.15.55:0/1803986706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 msgr2=0x55c1e10b8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:54.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.260+0000 7f37fbfff700 1 --2- 172.21.15.55:0/1803986706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 0x55c1e10b8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.260+0000 7f37fbfff700 1 -- 172.21.15.55:0/1803986706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10be2c0 msgr2=0x55c1e10b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:54.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.260+0000 7f37fbfff700 1 --2- 172.21.15.55:0/1803986706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10be2c0 0x55c1e10b9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.260+0000 7f37fbfff700 1 -- 172.21.15.55:0/1803986706 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1e0ea7270 con 0x55c1e10bfa10 2023-10-06T03:49:54.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.260+0000 7f37fbfff700 1 --2- 172.21.15.55:0/1803986706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10bfa10 0x55c1e10b9740 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f37f4001f60 tx=0x7f37f4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76ff37caff8d705 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.260+0000 7f37e0ff9700 1 -- 172.21.15.55:0/1803986706 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f400b070 con 0x55c1e10bfa10 2023-10-06T03:49:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.260+0000 7f37e0ff9700 1 -- 172.21.15.55:0/1803986706 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f400d9c0 con 0x55c1e10bfa10 2023-10-06T03:49:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.260+0000 7f37e0ff9700 1 -- 172.21.15.55:0/1803986706 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f400dd40 con 0x55c1e10bfa10 2023-10-06T03:49:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.261+0000 7f38106d4b80 1 -- 172.21.15.55:0/1803986706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10bfa10 msgr2=0x55c1e10b9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.261+0000 7f38106d4b80 1 --2- 172.21.15.55:0/1803986706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10bfa10 0x55c1e10b9740 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f37f4001f60 tx=0x7f37f4009040 comp rx=0 tx=0).stop 2023-10-06T03:49:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.261+0000 7f38106d4b80 1 -- 172.21.15.55:0/1803986706 shutdown_connections 2023-10-06T03:49:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.261+0000 7f38106d4b80 1 --2- 172.21.15.55:0/1803986706 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10bfa10 0x55c1e10b9740 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.261+0000 7f38106d4b80 1 --2- 172.21.15.55:0/1803986706 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 0x55c1e10b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.261+0000 7f38106d4b80 1 --2- 172.21.15.55:0/1803986706 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10be2c0 0x55c1e10b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.261+0000 7f38106d4b80 1 -- 172.21.15.55:0/1803986706 >> 172.21.15.55:0/1803986706 conn(0x55c1e10bde80 msgr2=0x55c1e10be6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:54.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.261+0000 7f38106d4b80 1 -- 172.21.15.55:0/1803986706 shutdown_connections 2023-10-06T03:49:54.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.261+0000 7f38106d4b80 1 -- 172.21.15.55:0/1803986706 wait complete. 2023-10-06T03:49:54.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.262+0000 7f38106d4b80 1 -- start start 2023-10-06T03:49:54.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.262+0000 7f38106d4b80 1 Processor -- start 2023-10-06T03:49:54.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.263+0000 7f38106d4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10be2c0 0x55c1e1116170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.263+0000 7f38106d4b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 0x55c1e1105c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.263+0000 7f38106d4b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 0x55c1e10b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.263+0000 7f38106d4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e0fb8890 con 0x55c1e10be2c0 2023-10-06T03:49:54.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.263+0000 7f38106d4b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e0fd3ea0 con 0x55c1e10bfa10 2023-10-06T03:49:54.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.263+0000 7f38106d4b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1e0e8fe30 con 0x55c1e10dde50 2023-10-06T03:49:54.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.263+0000 7f38106d4b80 10 client.0 _ll_register_callbacks cb 0x55c1e1080020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:49:54.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.263+0000 7f37fb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10be2c0 0x55c1e1116170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.263+0000 7f37fb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10be2c0 0x55c1e1116170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51590/0 (socket says 172.21.15.55:51590) 2023-10-06T03:49:54.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.263+0000 7f37fb7fe700 1 -- 172.21.15.55:0/1383435111 learned_addr learned my addr 172.21.15.55:0/1383435111 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.263+0000 7f37faffd700 1 --2- 172.21.15.55:0/1383435111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 0x55c1e1105c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.263+0000 7f37fbfff700 1 --2- 172.21.15.55:0/1383435111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 0x55c1e10b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.264+0000 7f37fb7fe700 1 -- 172.21.15.55:0/1383435111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 msgr2=0x55c1e10b90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.264+0000 7f37fb7fe700 1 --2- 172.21.15.55:0/1383435111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 0x55c1e10b90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.264+0000 7f37fb7fe700 1 -- 172.21.15.55:0/1383435111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 msgr2=0x55c1e1105c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.264+0000 7f37fb7fe700 1 --2- 172.21.15.55:0/1383435111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 0x55c1e1105c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.264+0000 7f37fb7fe700 1 -- 172.21.15.55:0/1383435111 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e4003570 con 0x55c1e10be2c0 2023-10-06T03:49:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.264+0000 7f37faffd700 1 --2- 172.21.15.55:0/1383435111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 0x55c1e1105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.264+0000 7f37fb7fe700 1 --2- 172.21.15.55:0/1383435111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10be2c0 0x55c1e1116170 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f37ec013000 tx=0x7f37ec01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949bbe8a278815de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.264+0000 7f37d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:49:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.264+0000 7f37d9ffb700 1 -- 172.21.15.55:0/1383435111 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec01a710 con 0x55c1e10be2c0 2023-10-06T03:49:54.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.264+0000 7f37d9ffb700 1 -- 172.21.15.55:0/1383435111 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f37ec01a870 con 0x55c1e10be2c0 2023-10-06T03:49:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.264+0000 7f38106d4b80 1 -- 172.21.15.55:0/1383435111 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1e10b1940 con 0x55c1e10be2c0 2023-10-06T03:49:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.264+0000 7f37d9ffb700 1 -- 172.21.15.55:0/1383435111 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37ec01ac40 con 0x55c1e10be2c0 2023-10-06T03:49:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.264+0000 7f38106d4b80 10 client.7410 fetch_fsmap learned FSMap version 53 2023-10-06T03:49:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.265+0000 7f38106d4b80 1 -- 172.21.15.55:0/1383435111 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55c1e0e49fe0 con 0x55c1e10be2c0 2023-10-06T03:49:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.265+0000 7f37d9ffb700 1 -- 172.21.15.55:0/1383435111 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1e10b1940 con 0x55c1e10be2c0 2023-10-06T03:49:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.265+0000 7f37d9ffb700 1 -- 172.21.15.55:0/1383435111 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 53) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f37ec033420 con 0x55c1e10be2c0 2023-10-06T03:49:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.265+0000 7f38106d4b80 10 client.7410 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T03:49:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.265+0000 7f38106d4b80 10 client.7410 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:49:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.265+0000 7f38106d4b80 20 client.7410 populate_metadata read hostname 'smithi055' 2023-10-06T03:49:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.265+0000 7f38106d4b80 1 --2- 172.21.15.55:0/1383435111 >> [v2:172.21.15.55:6834/2226955335,v1:172.21.15.55:6835/2226955335] conn(0x55c1e10b95f0 0x55c1e1103f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.265+0000 7f38106d4b80 4 client.7410 mds_command: new command op to 7254 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:49:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.265+0000 7f38106d4b80 1 -- 172.21.15.55:0/1383435111 --> [v2:172.21.15.55:6834/2226955335,v1:172.21.15.55:6835/2226955335] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1e108e870 con 0x55c1e10b95f0 2023-10-06T03:49:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.265+0000 7f37faffd700 1 --2- 172.21.15.55:0/1383435111 >> [v2:172.21.15.55:6834/2226955335,v1:172.21.15.55:6835/2226955335] conn(0x55c1e10b95f0 0x55c1e1103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.266+0000 7f37d9ffb700 10 client.7410 ms_handle_connect on v2:172.21.15.55:6834/2226955335 2023-10-06T03:49:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.266+0000 7f37faffd700 1 --2- 172.21.15.55:0/1383435111 >> [v2:172.21.15.55:6834/2226955335,v1:172.21.15.55:6835/2226955335] conn(0x55c1e10b95f0 0x55c1e1103f60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=854ff635741ce041 server_cookie=65fd2d46095b9918 in_seq=0 out_seq=0 2023-10-06T03:49:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.266+0000 7f37d9ffb700 1 -- 172.21.15.55:0/1383435111 <== mds.0 v2:172.21.15.55:6834/2226955335 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55c1e108e870 con 0x55c1e10b95f0 2023-10-06T03:49:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.266+0000 7f37d9ffb700 10 client.7410 handle_command_reply: tid=0 2023-10-06T03:49:54.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.266+0000 7f38106d4b80 1 client.7410 shutdown 2023-10-06T03:49:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.267+0000 7f38106d4b80 1 -- 172.21.15.55:0/1383435111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10be2c0 msgr2=0x55c1e1116170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.267+0000 7f38106d4b80 1 --2- 172.21.15.55:0/1383435111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10be2c0 0x55c1e1116170 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f37ec013000 tx=0x7f37ec01e040 comp rx=0 tx=0).stop 2023-10-06T03:49:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.267+0000 7f38106d4b80 1 -- 172.21.15.55:0/1383435111 shutdown_connections 2023-10-06T03:49:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.267+0000 7f38106d4b80 1 --2- 172.21.15.55:0/1383435111 >> [v2:172.21.15.55:6834/2226955335,v1:172.21.15.55:6835/2226955335] conn(0x55c1e10b95f0 0x55c1e1103f60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.267+0000 7f38106d4b80 1 --2- 172.21.15.55:0/1383435111 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 0x55c1e10b90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.267+0000 7f38106d4b80 1 --2- 172.21.15.55:0/1383435111 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10be2c0 0x55c1e1116170 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.267+0000 7f38106d4b80 1 --2- 172.21.15.55:0/1383435111 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 0x55c1e1105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.267+0000 7f37d9ffb700 0 client.7410 ms_handle_reset on v2:172.21.15.55:6834/2226955335 2023-10-06T03:49:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.267+0000 7f38106d4b80 1 -- 172.21.15.55:0/1383435111 >> 172.21.15.55:0/1383435111 conn(0x55c1e10bde80 msgr2=0x55c1e10bed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.267+0000 7f38106d4b80 1 -- 172.21.15.55:0/1383435111 shutdown_connections 2023-10-06T03:49:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.267+0000 7f38106d4b80 1 -- 172.21.15.55:0/1383435111 wait complete. 2023-10-06T03:49:54.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.267+0000 7f38106d4b80 20 client.7410 trim_cache size 0 max 16384 2023-10-06T03:49:54.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.285+0000 7f38106d4b80 1 Processor -- start 2023-10-06T03:49:54.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.285+0000 7f38106d4b80 1 -- start start 2023-10-06T03:49:54.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.285+0000 7f38106d4b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10be2c0 0x55c1e10b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.285+0000 7f38106d4b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 0x55c1e10b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.285+0000 7f38106d4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10dde50 0x55c1e10eb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.285+0000 7f38106d4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e10bdcb0 con 0x55c1e10dde50 2023-10-06T03:49:54.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.285+0000 7f38106d4b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e10df0c0 con 0x55c1e10bfa10 2023-10-06T03:49:54.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.285+0000 7f38106d4b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1e10b91e0 con 0x55c1e10be2c0 2023-10-06T03:49:54.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.286+0000 7f37fbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10dde50 0x55c1e10eb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.286+0000 7f37fbfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10dde50 0x55c1e10eb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51596/0 (socket says 172.21.15.55:51596) 2023-10-06T03:49:54.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.286+0000 7f37fbfff700 1 -- 172.21.15.55:0/4077631877 learned_addr learned my addr 172.21.15.55:0/4077631877 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:54.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.286+0000 7f37fbfff700 1 -- 172.21.15.55:0/4077631877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10be2c0 msgr2=0x55c1e10b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:54.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.286+0000 7f37fbfff700 1 --2- 172.21.15.55:0/4077631877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10be2c0 0x55c1e10b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.286+0000 7f37fbfff700 1 -- 172.21.15.55:0/4077631877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 msgr2=0x55c1e10b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:54.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.286+0000 7f37fbfff700 1 --2- 172.21.15.55:0/4077631877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 0x55c1e10b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.286+0000 7f37fbfff700 1 -- 172.21.15.55:0/4077631877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1e0fdb780 con 0x55c1e10dde50 2023-10-06T03:49:54.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.286+0000 7f37fbfff700 1 --2- 172.21.15.55:0/4077631877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10dde50 0x55c1e10eb1f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f37f400d430 tx=0x7f37f4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48bd3a25dba6cec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:54.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.287+0000 7f37e17fa700 1 -- 172.21.15.55:0/4077631877 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f4001aa0 con 0x55c1e10dde50 2023-10-06T03:49:54.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.287+0000 7f37e17fa700 1 -- 172.21.15.55:0/4077631877 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f4001640 con 0x55c1e10dde50 2023-10-06T03:49:54.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.287+0000 7f37e17fa700 1 -- 172.21.15.55:0/4077631877 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f40138d0 con 0x55c1e10dde50 2023-10-06T03:49:54.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.287+0000 7f38106d4b80 1 -- 172.21.15.55:0/4077631877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10dde50 msgr2=0x55c1e10eb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.287+0000 7f38106d4b80 1 --2- 172.21.15.55:0/4077631877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10dde50 0x55c1e10eb1f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f37f400d430 tx=0x7f37f4018000 comp rx=0 tx=0).stop 2023-10-06T03:49:54.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.287+0000 7f38106d4b80 1 -- 172.21.15.55:0/4077631877 shutdown_connections 2023-10-06T03:49:54.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.287+0000 7f38106d4b80 1 --2- 172.21.15.55:0/4077631877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10dde50 0x55c1e10eb1f0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.287+0000 7f38106d4b80 1 --2- 172.21.15.55:0/4077631877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10be2c0 0x55c1e10b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.287+0000 7f38106d4b80 1 --2- 172.21.15.55:0/4077631877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 0x55c1e10b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.287+0000 7f38106d4b80 1 -- 172.21.15.55:0/4077631877 >> 172.21.15.55:0/4077631877 conn(0x55c1e10bde80 msgr2=0x55c1e10df260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:54.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.287+0000 7f38106d4b80 1 -- 172.21.15.55:0/4077631877 shutdown_connections 2023-10-06T03:49:54.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.287+0000 7f38106d4b80 1 -- 172.21.15.55:0/4077631877 wait complete. 2023-10-06T03:49:54.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.288+0000 7f38106d4b80 1 -- start start 2023-10-06T03:49:54.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.288+0000 7f38106d4b80 1 Processor -- start 2023-10-06T03:49:54.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.289+0000 7f38106d4b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10be2c0 0x55c1e1168e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.289+0000 7f38106d4b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 0x55c1e10eb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.289+0000 7f38106d4b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 0x55c1e10eb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.289+0000 7f38106d4b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e1124850 con 0x55c1e10be2c0 2023-10-06T03:49:54.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.289+0000 7f38106d4b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1e111c620 con 0x55c1e10bfa10 2023-10-06T03:49:54.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.289+0000 7f38106d4b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1e1169340 con 0x55c1e10dde50 2023-10-06T03:49:54.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.289+0000 7f38106d4b80 10 client.0 _ll_register_callbacks cb 0x55c1e1080020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:49:54.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.289+0000 7f37fb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10be2c0 0x55c1e1168e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.289+0000 7f37fb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10be2c0 0x55c1e1168e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51598/0 (socket says 172.21.15.55:51598) 2023-10-06T03:49:54.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.289+0000 7f37fb7fe700 1 -- 172.21.15.55:0/2461143469 learned_addr learned my addr 172.21.15.55:0/2461143469 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:54.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.289+0000 7f37faffd700 1 --2- 172.21.15.55:0/2461143469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 0x55c1e10eb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.289+0000 7f37fbfff700 1 --2- 172.21.15.55:0/2461143469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 0x55c1e10eb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.290+0000 7f37fb7fe700 1 -- 172.21.15.55:0/2461143469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 msgr2=0x55c1e10eb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:54.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.290+0000 7f37fb7fe700 1 --2- 172.21.15.55:0/2461143469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 0x55c1e10eb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.290+0000 7f37fb7fe700 1 -- 172.21.15.55:0/2461143469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 msgr2=0x55c1e10eb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:54.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.290+0000 7f37fb7fe700 1 --2- 172.21.15.55:0/2461143469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 0x55c1e10eb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.290+0000 7f37fb7fe700 1 -- 172.21.15.55:0/2461143469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1e0ea7270 con 0x55c1e10be2c0 2023-10-06T03:49:54.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.290+0000 7f37fb7fe700 1 --2- 172.21.15.55:0/2461143469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10be2c0 0x55c1e1168e00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f37ec006d80 tx=0x7f37ec00b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcc648300877867 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:54.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.290+0000 7f37d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:49:54.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.290+0000 7f37d9ffb700 1 -- 172.21.15.55:0/2461143469 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec011b30 con 0x55c1e10be2c0 2023-10-06T03:49:54.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.290+0000 7f37d9ffb700 1 -- 172.21.15.55:0/2461143469 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f37ec011c90 con 0x55c1e10be2c0 2023-10-06T03:49:54.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.290+0000 7f38106d4b80 1 -- 172.21.15.55:0/2461143469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37fc0c3cf0 con 0x55c1e10be2c0 2023-10-06T03:49:54.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.290+0000 7f37d9ffb700 1 -- 172.21.15.55:0/2461143469 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37ec01ae50 con 0x55c1e10be2c0 2023-10-06T03:49:54.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.290+0000 7f38106d4b80 10 client.7416 fetch_fsmap learned FSMap version 53 2023-10-06T03:49:54.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.290+0000 7f38106d4b80 1 -- 172.21.15.55:0/2461143469 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55c1e0fc2470 con 0x55c1e10be2c0 2023-10-06T03:49:54.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.291+0000 7f37d9ffb700 1 -- 172.21.15.55:0/2461143469 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc0c3cf0 con 0x55c1e10be2c0 2023-10-06T03:49:54.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.291+0000 7f37d9ffb700 1 -- 172.21.15.55:0/2461143469 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 53) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f37ec033bc0 con 0x55c1e10be2c0 2023-10-06T03:49:54.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.291+0000 7f38106d4b80 10 client.7416 fetch_fsmap finished waiting for FSMap version 53 2023-10-06T03:49:54.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.291+0000 7f38106d4b80 10 client.7416 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T03:49:54.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.291+0000 7f38106d4b80 20 client.7416 populate_metadata read hostname 'smithi055' 2023-10-06T03:49:54.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.291+0000 7f38106d4b80 1 --2- 172.21.15.55:0/2461143469 >> [v2:172.21.15.55:6834/2226955335,v1:172.21.15.55:6835/2226955335] conn(0x55c1e10bcde0 0x55c1e10bd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.291+0000 7f38106d4b80 4 client.7416 mds_command: new command op to 7254 tid=0[{"prefix": "status"}] 2023-10-06T03:49:54.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.291+0000 7f38106d4b80 1 -- 172.21.15.55:0/2461143469 --> [v2:172.21.15.55:6834/2226955335,v1:172.21.15.55:6835/2226955335] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f37f400d9c0 con 0x55c1e10bcde0 2023-10-06T03:49:54.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.291+0000 7f37faffd700 1 --2- 172.21.15.55:0/2461143469 >> [v2:172.21.15.55:6834/2226955335,v1:172.21.15.55:6835/2226955335] conn(0x55c1e10bcde0 0x55c1e10bd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f37d9ffb700 10 client.7416 ms_handle_connect on v2:172.21.15.55:6834/2226955335 2023-10-06T03:49:54.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f37faffd700 1 --2- 172.21.15.55:0/2461143469 >> [v2:172.21.15.55:6834/2226955335,v1:172.21.15.55:6835/2226955335] conn(0x55c1e10bcde0 0x55c1e10bd1b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcab87f9902daf1c server_cookie=69ebd4184bb54bd4 in_seq=0 out_seq=0 2023-10-06T03:49:54.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f37d9ffb700 1 -- 172.21.15.55:0/2461143469 <== mds.0 v2:172.21.15.55:6834/2226955335 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f37f400d9c0 con 0x55c1e10bcde0 2023-10-06T03:49:54.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f37d9ffb700 10 client.7416 handle_command_reply: tid=0 2023-10-06T03:49:54.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f38106d4b80 1 client.7416 shutdown 2023-10-06T03:49:54.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f38106d4b80 1 -- 172.21.15.55:0/2461143469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10be2c0 msgr2=0x55c1e1168e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f38106d4b80 1 --2- 172.21.15.55:0/2461143469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10be2c0 0x55c1e1168e00 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f37ec006d80 tx=0x7f37ec00b050 comp rx=0 tx=0).stop 2023-10-06T03:49:54.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f38106d4b80 1 -- 172.21.15.55:0/2461143469 shutdown_connections 2023-10-06T03:49:54.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f38106d4b80 1 --2- 172.21.15.55:0/2461143469 >> [v2:172.21.15.55:6834/2226955335,v1:172.21.15.55:6835/2226955335] conn(0x55c1e10bcde0 0x55c1e10bd1b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f38106d4b80 1 --2- 172.21.15.55:0/2461143469 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55c1e10dde50 0x55c1e10eb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f38106d4b80 1 --2- 172.21.15.55:0/2461143469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55c1e10be2c0 0x55c1e1168e00 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f38106d4b80 1 --2- 172.21.15.55:0/2461143469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55c1e10bfa10 0x55c1e10eb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f37d9ffb700 0 client.7416 ms_handle_reset on v2:172.21.15.55:6834/2226955335 2023-10-06T03:49:54.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f38106d4b80 1 -- 172.21.15.55:0/2461143469 >> 172.21.15.55:0/2461143469 conn(0x55c1e10bde80 msgr2=0x55c1e1112310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:54.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f38106d4b80 1 -- 172.21.15.55:0/2461143469 shutdown_connections 2023-10-06T03:49:54.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.292+0000 7f38106d4b80 1 -- 172.21.15.55:0/2461143469 wait complete. 2023-10-06T03:49:54.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.293+0000 7f38106d4b80 20 client.7416 trim_cache size 0 max 16384 2023-10-06T03:49:54.299 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:49:54.299 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "8132c8ba-ee1c-41a2-b781-913925034ab6", 2023-10-06T03:49:54.299 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2023-10-06T03:49:54.299 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 7254, 2023-10-06T03:49:54.299 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2023-10-06T03:49:54.300 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2023-10-06T03:49:54.300 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2023-10-06T03:49:54.300 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 6.5298244700000003, 2023-10-06T03:49:54.300 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 53, 2023-10-06T03:49:54.300 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 107, 2023-10-06T03:49:54.300 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 107, 2023-10-06T03:49:54.300 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 12.979908623 2023-10-06T03:49:54.300 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:49:54.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.293+0000 7f3800d9d700 1 -- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f37e8066f30 msgr2=0x7f37e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.293+0000 7f3800d9d700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f37e8066f30 0x7f37e80693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f37ec003080 tx=0x7f37ec017000 comp rx=0 tx=0).stop 2023-10-06T03:49:54.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.293+0000 7f3800d9d700 1 -- 172.21.15.55:0/3199205542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37fc0c6a70 msgr2=0x7f37fc159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.293+0000 7f3800d9d700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37fc0c6a70 0x7f37fc159370 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f37f000ec80 tx=0x7f37f00091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.293+0000 7f3800d9d700 1 -- 172.21.15.55:0/3199205542 shutdown_connections 2023-10-06T03:49:54.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.293+0000 7f3800d9d700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f37e8066f30 0x7f37e80693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.293+0000 7f3800d9d700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f37fc0ca660 0x7f37fc14d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.293+0000 7f3800d9d700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f37fc0696c0 0x7f37fc14c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.293+0000 7f3800d9d700 1 --2- 172.21.15.55:0/3199205542 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f37fc0c6a70 0x7f37fc159370 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.293+0000 7f3800d9d700 1 -- 172.21.15.55:0/3199205542 >> 172.21.15.55:0/3199205542 conn(0x7f37fc0cc1f0 msgr2=0x7f37fc0c1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:54.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.293+0000 7f3800d9d700 1 -- 172.21.15.55:0/3199205542 shutdown_connections 2023-10-06T03:49:54.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.293+0000 7f3800d9d700 1 -- 172.21.15.55:0/3199205542 wait complete. 2023-10-06T03:49:54.311 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:49:54.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.497+0000 7f1c5e99d700 1 Processor -- start 2023-10-06T03:49:54.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.497+0000 7f1c5e99d700 1 -- start start 2023-10-06T03:49:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.498+0000 7f1c5e99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c58159420 0x7f1c580b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.498+0000 7f1c5e99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c581597f0 0x7f1c580b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.498+0000 7f1c5e99d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1c5815d230 0x7f1c5815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.498+0000 7f1c5e99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5815dbd0 con 0x7f1c58159420 2023-10-06T03:49:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.498+0000 7f1c5e99d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c580b11d0 con 0x7f1c5815d230 2023-10-06T03:49:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.498+0000 7f1c5e99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c580b14b0 con 0x7f1c581597f0 2023-10-06T03:49:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.499+0000 7f1c5d99b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c58159420 0x7f1c580b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.499+0000 7f1c5d99b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c58159420 0x7f1c580b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51614/0 (socket says 172.21.15.55:51614) 2023-10-06T03:49:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.499+0000 7f1c5d99b700 1 -- 172.21.15.55:0/502735586 learned_addr learned my addr 172.21.15.55:0/502735586 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.499+0000 7f1c5d19a700 1 --2- 172.21.15.55:0/502735586 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c581597f0 0x7f1c580b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.499+0000 7f1c5e19c700 1 --2- 172.21.15.55:0/502735586 >> v2:172.21.15.163:3300/0 conn(0x7f1c5815d230 0x7f1c5815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.499+0000 7f1c5d99b700 1 -- 172.21.15.55:0/502735586 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c581597f0 msgr2=0x7f1c580b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:54.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.499+0000 7f1c5d99b700 1 --2- 172.21.15.55:0/502735586 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c581597f0 0x7f1c580b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.499+0000 7f1c5d99b700 1 -- 172.21.15.55:0/502735586 >> v2:172.21.15.163:3300/0 conn(0x7f1c5815d230 msgr2=0x7f1c5815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:54.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.499+0000 7f1c5d99b700 1 --2- 172.21.15.55:0/502735586 >> v2:172.21.15.163:3300/0 conn(0x7f1c5815d230 0x7f1c5815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.499+0000 7f1c5d99b700 1 -- 172.21.15.55:0/502735586 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c580b1790 con 0x7f1c58159420 2023-10-06T03:49:54.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.499+0000 7f1c5d99b700 1 --2- 172.21.15.55:0/502735586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c58159420 0x7f1c580b5f40 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1c580d1720 tx=0x7f1c54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744a9cfc4be85605 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:54.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.500+0000 7f1c5c999700 1 -- 172.21.15.55:0/502735586 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400b860 con 0x7f1c58159420 2023-10-06T03:49:54.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.500+0000 7f1c5c999700 1 -- 172.21.15.55:0/502735586 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5400b9c0 con 0x7f1c58159420 2023-10-06T03:49:54.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.500+0000 7f1c5c999700 1 -- 172.21.15.55:0/502735586 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400bd60 con 0x7f1c58159420 2023-10-06T03:49:54.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.500+0000 7f1c5e99d700 1 -- 172.21.15.55:0/502735586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c58159420 msgr2=0x7f1c580b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.500+0000 7f1c5e99d700 1 --2- 172.21.15.55:0/502735586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c58159420 0x7f1c580b5f40 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f1c580d1720 tx=0x7f1c54009250 comp rx=0 tx=0).stop 2023-10-06T03:49:54.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.500+0000 7f1c5e99d700 1 -- 172.21.15.55:0/502735586 shutdown_connections 2023-10-06T03:49:54.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.500+0000 7f1c5e99d700 1 --2- 172.21.15.55:0/502735586 >> v2:172.21.15.163:3300/0 conn(0x7f1c5815d230 0x7f1c5815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.500+0000 7f1c5e99d700 1 --2- 172.21.15.55:0/502735586 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c58159420 0x7f1c580b5f40 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.500+0000 7f1c5e99d700 1 --2- 172.21.15.55:0/502735586 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c581597f0 0x7f1c580b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.500+0000 7f1c5e99d700 1 -- 172.21.15.55:0/502735586 >> 172.21.15.55:0/502735586 conn(0x7f1c580d0660 msgr2=0x7f1c580d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:54.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.500+0000 7f1c5e99d700 1 -- 172.21.15.55:0/502735586 shutdown_connections 2023-10-06T03:49:54.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.500+0000 7f1c5e99d700 1 -- 172.21.15.55:0/502735586 wait complete. 2023-10-06T03:49:54.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.502+0000 7f1c5e99d700 1 Processor -- start 2023-10-06T03:49:54.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.502+0000 7f1c5e99d700 1 -- start start 2023-10-06T03:49:54.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.503+0000 7f1c5e99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c58159420 0x7f1c5815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.503+0000 7f1c5e99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c581597f0 0x7f1c5815c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.503+0000 7f1c5e99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c5815d230 0x7f1c580c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.503+0000 7f1c5e99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5815b840 con 0x7f1c581597f0 2023-10-06T03:49:54.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.503+0000 7f1c5e99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5815ac30 con 0x7f1c5815d230 2023-10-06T03:49:54.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.503+0000 7f1c5e99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5815af10 con 0x7f1c58159420 2023-10-06T03:49:54.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.503+0000 7f1c5d19a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c581597f0 0x7f1c5815c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.503+0000 7f1c5d19a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c581597f0 0x7f1c5815c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51628/0 (socket says 172.21.15.55:51628) 2023-10-06T03:49:54.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.503+0000 7f1c5d19a700 1 -- 172.21.15.55:0/1846759332 learned_addr learned my addr 172.21.15.55:0/1846759332 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:54.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.503+0000 7f1c5d99b700 1 --2- 172.21.15.55:0/1846759332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c58159420 0x7f1c5815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.503+0000 7f1c5e19c700 1 --2- 172.21.15.55:0/1846759332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c5815d230 0x7f1c580c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.503+0000 7f1c5d19a700 1 -- 172.21.15.55:0/1846759332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c58159420 msgr2=0x7f1c5815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.504+0000 7f1c5d19a700 1 --2- 172.21.15.55:0/1846759332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c58159420 0x7f1c5815bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.504+0000 7f1c5d19a700 1 -- 172.21.15.55:0/1846759332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c5815d230 msgr2=0x7f1c580c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.504+0000 7f1c5d19a700 1 --2- 172.21.15.55:0/1846759332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c5815d230 0x7f1c580c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.504+0000 7f1c5d19a700 1 -- 172.21.15.55:0/1846759332 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c540094f0 con 0x7f1c581597f0 2023-10-06T03:49:54.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.504+0000 7f1c5d19a700 1 --2- 172.21.15.55:0/1846759332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c581597f0 0x7f1c5815c210 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800d890 tx=0x7f1c48008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:54.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.504+0000 7f1c4dffb700 1 -- 172.21.15.55:0/1846759332 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c480125f0 con 0x7f1c581597f0 2023-10-06T03:49:54.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.504+0000 7f1c4dffb700 1 -- 172.21.15.55:0/1846759332 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c48012770 con 0x7f1c581597f0 2023-10-06T03:49:54.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.504+0000 7f1c4dffb700 1 -- 172.21.15.55:0/1846759332 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c48012b30 con 0x7f1c581597f0 2023-10-06T03:49:54.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.504+0000 7f1c5e99d700 1 -- 172.21.15.55:0/1846759332 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5815b1f0 con 0x7f1c581597f0 2023-10-06T03:49:54.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.504+0000 7f1c5e99d700 1 -- 172.21.15.55:0/1846759332 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c580ca450 con 0x7f1c581597f0 2023-10-06T03:49:54.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.505+0000 7f1c5e99d700 1 -- 172.21.15.55:0/1846759332 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5804c3c0 con 0x7f1c581597f0 2023-10-06T03:49:54.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.506+0000 7f1c4dffb700 1 -- 172.21.15.55:0/1846759332 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c4801e460 con 0x7f1c581597f0 2023-10-06T03:49:54.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.507+0000 7f1c4dffb700 1 --2- 172.21.15.55:0/1846759332 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c3c066f30 0x7f1c3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:54.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.507+0000 7f1c4dffb700 1 -- 172.21.15.55:0/1846759332 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f1c48084fe0 con 0x7f1c581597f0 2023-10-06T03:49:54.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.508+0000 7f1c5d99b700 1 --2- 172.21.15.55:0/1846759332 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c3c066f30 0x7f1c3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:54.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.508+0000 7f1c5d99b700 1 --2- 172.21.15.55:0/1846759332 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c3c066f30 0x7f1c3c0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c580d1720 tx=0x7f1c54017000 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:54.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.512+0000 7f1c4dffb700 1 -- 172.21.15.55:0/1846759332 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c48051ee0 con 0x7f1c581597f0 2023-10-06T03:49:54.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.761+0000 7f1c5e99d700 1 -- 172.21.15.55:0/1846759332 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1c580b1a70 con 0x7f1c581597f0 2023-10-06T03:49:54.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.762+0000 7f1c4dffb700 1 -- 172.21.15.55:0/1846759332 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1c48055500 con 0x7f1c581597f0 2023-10-06T03:49:54.763 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:49:54.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.764+0000 7f1c5e99d700 1 -- 172.21.15.55:0/1846759332 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c3c066f30 msgr2=0x7f1c3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.764+0000 7f1c5e99d700 1 --2- 172.21.15.55:0/1846759332 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c3c066f30 0x7f1c3c0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c580d1720 tx=0x7f1c54017000 comp rx=0 tx=0).stop 2023-10-06T03:49:54.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.764+0000 7f1c5e99d700 1 -- 172.21.15.55:0/1846759332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c581597f0 msgr2=0x7f1c5815c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:54.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.764+0000 7f1c5e99d700 1 --2- 172.21.15.55:0/1846759332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c581597f0 0x7f1c5815c210 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1c4800d890 tx=0x7f1c48008040 comp rx=0 tx=0).stop 2023-10-06T03:49:54.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.764+0000 7f1c5e99d700 1 -- 172.21.15.55:0/1846759332 shutdown_connections 2023-10-06T03:49:54.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.764+0000 7f1c5e99d700 1 --2- 172.21.15.55:0/1846759332 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c3c066f30 0x7f1c3c0693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.764+0000 7f1c5e99d700 1 --2- 172.21.15.55:0/1846759332 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c5815d230 0x7f1c580c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.764+0000 7f1c5e99d700 1 --2- 172.21.15.55:0/1846759332 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c58159420 0x7f1c5815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.764+0000 7f1c5e99d700 1 --2- 172.21.15.55:0/1846759332 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c581597f0 0x7f1c5815c210 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:54.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.764+0000 7f1c5e99d700 1 -- 172.21.15.55:0/1846759332 >> 172.21.15.55:0/1846759332 conn(0x7f1c580d0660 msgr2=0x7f1c580b5520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:54.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.764+0000 7f1c5e99d700 1 -- 172.21.15.55:0/1846759332 shutdown_connections 2023-10-06T03:49:54.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:54.764+0000 7f1c5e99d700 1 -- 172.21.15.55:0/1846759332 wait complete. 2023-10-06T03:49:54.783 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:49:54.783 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:49:54.784 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2023-10-06T03:49:54.784 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:49:54.784 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:49:54.784 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:49:54.784 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:49:54.784 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:49:54.784 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:49:54.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:54.784 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2023-10-06T03:49:54.822 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:49:54.822 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:49:54.823 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:49:54.823 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:49:54.823 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T03:49:54.823 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:49:54.823 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:49:54.823 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:49:54.823 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2023-10-06T03:49:54.823 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:49:54.823 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:49:54.823 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2023-10-06T03:49:54.823 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:49:54.824 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:49:54.824 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2023-10-06T03:49:54.824 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T03:49:54.824 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 41247sec preferred_lft 41247sec 2023-10-06T03:49:54.824 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link 2023-10-06T03:49:54.824 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:49:54.824 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:49:54.824 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:49:54.824 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2023-10-06T03:49:54.824 INFO:teuthology.orchestra.run.smithi055.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:49:54.824 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff 2023-10-06T03:49:54.825 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:49:54.825 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:49:54.825 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::1099:8eff:fe6d:102/64 scope link 2023-10-06T03:49:54.825 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:49:54.825 INFO:teuthology.orchestra.run.smithi055.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:49:54.825 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether f2:5d:53:d2:00:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:49:54.825 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f05d:53ff:fed2:a/64 scope link 2023-10-06T03:49:54.825 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:49:54.825 INFO:teuthology.orchestra.run.smithi055.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:49:54.825 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:49:54.826 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::e037:33ff:feb7:6c8f/64 scope link 2023-10-06T03:49:54.826 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:49:54.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:54.826 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:49:54.922 INFO:teuthology.orchestra.run.smithi055.stdout:1 2023-10-06T03:49:54.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:54.925 DEBUG:teuthology.orchestra.run.smithi055:> route 2023-10-06T03:49:54.963 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2023-10-06T03:49:54.963 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:49:54.963 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T03:49:54.963 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T03:49:54.963 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:49:54.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:54.964 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:49:54.965 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:49:54.965 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T03:49:54.965 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T03:49:54.965 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T03:49:54.965 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:49:55.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:55.163 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T03:49:55.199 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:49:55.199 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:49:55.200 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:49:55.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:55.200 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2023-10-06T03:49:55.302 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:49:55.302 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:49:55.340 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:49:55.340 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:49:55.341 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:49:55.417 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T03:49:55.417 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:49:55.418 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:49:55.550 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:49:55.551 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:49:55.551 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:49:55.567 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:49:55.567 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:49:55.567 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:49:55.567 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:49:55.568 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:49:55.589 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:49:55.589 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:49:55.589 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:49:55.589 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:49:55.589 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:49:55.590 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T03:49:55.590 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T03:49:55.590 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T03:49:55.592 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:49:55.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:49:55.592 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:49:55.655 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:49:55.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.848+0000 7fb3cf59d700 1 Processor -- start 2023-10-06T03:49:55.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.848+0000 7fb3cf59d700 1 -- start start 2023-10-06T03:49:55.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.849+0000 7fb3cf59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb3c81597f0 0x7fb3c80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:55.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.849+0000 7fb3cf59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c815d230 0x7fb3c815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:55.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.849+0000 7fb3cf59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c815dbb0 con 0x7fb3c8159420 2023-10-06T03:49:55.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.849+0000 7fb3cf59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3c80b11d0 con 0x7fb3c81597f0 2023-10-06T03:49:55.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.849+0000 7fb3cf59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c80b14b0 con 0x7fb3c815d230 2023-10-06T03:49:55.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.850+0000 7fb3cdd9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb3c81597f0 0x7fb3c80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:55.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.850+0000 7fb3ce59b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb3c8159420 0x7fb3c80b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39564/0 (socket says 172.21.15.55:39564) 2023-10-06T03:49:55.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.850+0000 7fb3ce59b700 1 -- 172.21.15.55:0/615584773 learned_addr learned my addr 172.21.15.55:0/615584773 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:55.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.850+0000 7fb3ced9c700 1 --2- 172.21.15.55:0/615584773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c815d230 0x7fb3c815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:55.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.850+0000 7fb3cdd9a700 1 -- 172.21.15.55:0/615584773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c815d230 msgr2=0x7fb3c815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:55.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.850+0000 7fb3cdd9a700 1 --2- 172.21.15.55:0/615584773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c815d230 0x7fb3c815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:55.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.850+0000 7fb3cdd9a700 1 -- 172.21.15.55:0/615584773 >> v1:172.21.15.55:6789/0 conn(0x7fb3c8159420 legacy=0x7fb3c80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:55.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.850+0000 7fb3cdd9a700 1 -- 172.21.15.55:0/615584773 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c80b1790 con 0x7fb3c81597f0 2023-10-06T03:49:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.850+0000 7fb3cd599700 1 -- 172.21.15.55:0/615584773 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045598456 0 0) 0x7fb3c815dbb0 con 0x7fb3c8159420 2023-10-06T03:49:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.850+0000 7fb3cdd9a700 1 --2- 172.21.15.55:0/615584773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3c81597f0 0x7fb3c80b2270 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80098e0 tx=0x7fb3b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da79b6c06fb6c5bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.851+0000 7fb3cd599700 1 -- 172.21.15.55:0/615584773 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8009e80 con 0x7fb3c81597f0 2023-10-06T03:49:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.851+0000 7fb3cd599700 1 -- 172.21.15.55:0/615584773 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b800b860 con 0x7fb3c81597f0 2023-10-06T03:49:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.851+0000 7fb3cd599700 1 -- 172.21.15.55:0/615584773 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b800bbe0 con 0x7fb3c81597f0 2023-10-06T03:49:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.851+0000 7fb3cf59d700 1 -- 172.21.15.55:0/615584773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3c81597f0 msgr2=0x7fb3c80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.851+0000 7fb3cf59d700 1 --2- 172.21.15.55:0/615584773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3c81597f0 0x7fb3c80b2270 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80098e0 tx=0x7fb3b80091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.851+0000 7fb3cf59d700 1 -- 172.21.15.55:0/615584773 shutdown_connections 2023-10-06T03:49:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.851+0000 7fb3cf59d700 1 --2- 172.21.15.55:0/615584773 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c815d230 0x7fb3c815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.851+0000 7fb3cf59d700 1 --2- 172.21.15.55:0/615584773 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3c81597f0 0x7fb3c80b2270 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.851+0000 7fb3cf59d700 1 -- 172.21.15.55:0/615584773 >> 172.21.15.55:0/615584773 conn(0x7fb3c80d0660 msgr2=0x7fb3c80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:55.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.851+0000 7fb3cf59d700 1 -- 172.21.15.55:0/615584773 shutdown_connections 2023-10-06T03:49:55.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.851+0000 7fb3cf59d700 1 -- 172.21.15.55:0/615584773 wait complete. 2023-10-06T03:49:55.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.853+0000 7fb3cf59d700 1 Processor -- start 2023-10-06T03:49:55.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.853+0000 7fb3cf59d700 1 -- start start 2023-10-06T03:49:55.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.854+0000 7fb3cf59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c8159420 0x7fb3c80c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:55.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.854+0000 7fb3cf59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3c81597f0 0x7fb3c80c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:55.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.854+0000 7fb3cf59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3c815d230 0x7fb3c8154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:55.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.854+0000 7fb3cf59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c80c7070 con 0x7fb3c81597f0 2023-10-06T03:49:55.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.854+0000 7fb3cf59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c815ac70 con 0x7fb3c815d230 2023-10-06T03:49:55.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.854+0000 7fb3cf59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c815b230 con 0x7fb3c8159420 2023-10-06T03:49:55.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.854+0000 7fb3cdd9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3c81597f0 0x7fb3c80c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:55.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.854+0000 7fb3cdd9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3c81597f0 0x7fb3c80c6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51636/0 (socket says 172.21.15.55:51636) 2023-10-06T03:49:55.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.854+0000 7fb3cdd9a700 1 -- 172.21.15.55:0/4263648552 learned_addr learned my addr 172.21.15.55:0/4263648552 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:55.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.854+0000 7fb3ce59b700 1 --2- 172.21.15.55:0/4263648552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c8159420 0x7fb3c80c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:55.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.854+0000 7fb3ced9c700 1 --2- 172.21.15.55:0/4263648552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3c815d230 0x7fb3c8154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:55.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.855+0000 7fb3cdd9a700 1 -- 172.21.15.55:0/4263648552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c8159420 msgr2=0x7fb3c80c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:55.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.855+0000 7fb3cdd9a700 1 --2- 172.21.15.55:0/4263648552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c8159420 0x7fb3c80c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:55.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.855+0000 7fb3cdd9a700 1 -- 172.21.15.55:0/4263648552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3c815d230 msgr2=0x7fb3c8154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:55.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.855+0000 7fb3cdd9a700 1 --2- 172.21.15.55:0/4263648552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3c815d230 0x7fb3c8154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:55.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.855+0000 7fb3cdd9a700 1 -- 172.21.15.55:0/4263648552 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b8009420 con 0x7fb3c81597f0 2023-10-06T03:49:55.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.855+0000 7fb3cdd9a700 1 --2- 172.21.15.55:0/4263648552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3c81597f0 0x7fb3c80c6b30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8013000 tx=0x7fb3b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:55.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.855+0000 7fb3be7fc700 1 -- 172.21.15.55:0/4263648552 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b800bbe0 con 0x7fb3c81597f0 2023-10-06T03:49:55.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.855+0000 7fb3be7fc700 1 -- 172.21.15.55:0/4263648552 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b8000d10 con 0x7fb3c81597f0 2023-10-06T03:49:55.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.855+0000 7fb3be7fc700 1 -- 172.21.15.55:0/4263648552 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8011690 con 0x7fb3c81597f0 2023-10-06T03:49:55.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.855+0000 7fb3cf59d700 1 -- 172.21.15.55:0/4263648552 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c80c7280 con 0x7fb3c81597f0 2023-10-06T03:49:55.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.855+0000 7fb3cf59d700 1 -- 172.21.15.55:0/4263648552 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c815aec0 con 0x7fb3c81597f0 2023-10-06T03:49:55.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.856+0000 7fb3cf59d700 1 -- 172.21.15.55:0/4263648552 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c815b3a0 con 0x7fb3c81597f0 2023-10-06T03:49:55.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.857+0000 7fb3be7fc700 1 -- 172.21.15.55:0/4263648552 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3b8011a40 con 0x7fb3c81597f0 2023-10-06T03:49:55.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.858+0000 7fb3be7fc700 1 --2- 172.21.15.55:0/4263648552 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb3ac066f30 0x7fb3ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:55.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.858+0000 7fb3be7fc700 1 -- 172.21.15.55:0/4263648552 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fb3b8086130 con 0x7fb3c81597f0 2023-10-06T03:49:55.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.859+0000 7fb3ce59b700 1 --2- 172.21.15.55:0/4263648552 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb3ac066f30 0x7fb3ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:55.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.859+0000 7fb3ce59b700 1 --2- 172.21.15.55:0/4263648552 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb3ac066f30 0x7fb3ac0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4007fd0 tx=0x7fb3c4007d30 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:55.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:55.863+0000 7fb3be7fc700 1 -- 172.21.15.55:0/4263648552 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3b80530b0 con 0x7fb3c81597f0 2023-10-06T03:49:56.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.092+0000 7fb3cf59d700 1 -- 172.21.15.55:0/4263648552 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3c804c3c0 con 0x7fb3ac066f30 2023-10-06T03:49:56.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.095+0000 7fb3be7fc700 1 -- 172.21.15.55:0/4263648552 <== mgr.6799 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb3c804c3c0 con 0x7fb3ac066f30 2023-10-06T03:49:56.096 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:49:56.096 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:49:56.096 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephfs" 2023-10-06T03:49:56.096 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:49:56.096 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:49:56.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.097+0000 7fb3cf59d700 1 -- 172.21.15.55:0/4263648552 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb3ac066f30 msgr2=0x7fb3ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:56.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.097+0000 7fb3cf59d700 1 --2- 172.21.15.55:0/4263648552 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb3ac066f30 0x7fb3ac0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4007fd0 tx=0x7fb3c4007d30 comp rx=0 tx=0).stop 2023-10-06T03:49:56.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.097+0000 7fb3cf59d700 1 -- 172.21.15.55:0/4263648552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3c81597f0 msgr2=0x7fb3c80c6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:56.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.097+0000 7fb3cf59d700 1 --2- 172.21.15.55:0/4263648552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3c81597f0 0x7fb3c80c6b30 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8013000 tx=0x7fb3b8017000 comp rx=0 tx=0).stop 2023-10-06T03:49:56.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.097+0000 7fb3cf59d700 1 -- 172.21.15.55:0/4263648552 shutdown_connections 2023-10-06T03:49:56.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.097+0000 7fb3cf59d700 1 --2- 172.21.15.55:0/4263648552 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb3ac066f30 0x7fb3ac0693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:56.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.097+0000 7fb3cf59d700 1 --2- 172.21.15.55:0/4263648552 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb3c815d230 0x7fb3c8154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:56.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.097+0000 7fb3cf59d700 1 --2- 172.21.15.55:0/4263648552 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb3c8159420 0x7fb3c80c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:56.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.097+0000 7fb3cf59d700 1 --2- 172.21.15.55:0/4263648552 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb3c81597f0 0x7fb3c80c6b30 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:56.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.097+0000 7fb3cf59d700 1 -- 172.21.15.55:0/4263648552 >> 172.21.15.55:0/4263648552 conn(0x7fb3c80d0660 msgr2=0x7fb3c80b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:56.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.097+0000 7fb3cf59d700 1 -- 172.21.15.55:0/4263648552 shutdown_connections 2023-10-06T03:49:56.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.097+0000 7fb3cf59d700 1 -- 172.21.15.55:0/4263648552 wait complete. 2023-10-06T03:49:56.116 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T03:49:56.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.303+0000 7fdc25b9d700 1 Processor -- start 2023-10-06T03:49:56.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.303+0000 7fdc25b9d700 1 -- start start 2023-10-06T03:49:56.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.304+0000 7fdc25b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2006f060 0x7fdc200ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:56.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.304+0000 7fdc25b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2006ebd0 con 0x7fdc2006f4b0 2023-10-06T03:49:56.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.304+0000 7fdc25b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2015aa70 con 0x7fdc200cb1a0 2023-10-06T03:49:56.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.304+0000 7fdc25b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2015b2e0 con 0x7fdc2006f060 2023-10-06T03:49:56.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.305+0000 7fdc2539c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fdc2006f4b0 0x7fdc200c9db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39572/0 (socket says 172.21.15.55:39572) 2023-10-06T03:49:56.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.305+0000 7fdc2539c700 1 -- 172.21.15.55:0/435232077 learned_addr learned my addr 172.21.15.55:0/435232077 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:49:56.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.305+0000 7fdc1ffff700 1 --2- 172.21.15.55:0/435232077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2006f060 0x7fdc200ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:56.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.305+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469776749 0 0) 0x7fdc2006ebd0 con 0x7fdc2006f4b0 2023-10-06T03:49:56.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.305+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc10003570 con 0x7fdc2006f4b0 2023-10-06T03:49:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.305+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719862762 0 0) 0x7fdc2015aa70 con 0x7fdc200cb1a0 2023-10-06T03:49:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc2006ebd0 con 0x7fdc200cb1a0 2023-10-06T03:49:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 619257555 0 0) 0x7fdc10003570 con 0x7fdc2006f4b0 2023-10-06T03:49:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2006f060 msgr2=0x7fdc200ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1f7fe700 1 --2- 172.21.15.55:0/435232077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2006f060 0x7fdc200ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:56.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 >> v1:172.21.15.163:6789/0 conn(0x7fdc200cb1a0 legacy=0x7fdc2015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2015ad20 con 0x7fdc2006f4b0 2023-10-06T03:49:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fdc180017f0 con 0x7fdc2006f4b0 2023-10-06T03:49:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 >> v1:172.21.15.55:6789/0 conn(0x7fdc2006f4b0 legacy=0x7fdc200c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1f7fe700 1 --2- 172.21.15.55:0/435232077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc10003c20 0x7fdc10004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1f7fe700 1 --2- 172.21.15.55:0/435232077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc10004810 0x7fdc10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1f7fe700 1 --2- 172.21.15.55:0/435232077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2006f060 0x7fdc10005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2015b2e0 con 0x7fdc10003c20 2023-10-06T03:49:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1000bbf0 con 0x7fdc10004810 2023-10-06T03:49:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1000bdb0 con 0x7fdc2006f060 2023-10-06T03:49:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc24b9b700 1 --2- 172.21.15.55:0/435232077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc10003c20 0x7fdc10004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:56.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc1ffff700 1 --2- 172.21.15.55:0/435232077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc10004810 0x7fdc10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:56.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc2539c700 1 --2- 172.21.15.55:0/435232077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2006f060 0x7fdc10005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:56.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc24b9b700 1 -- 172.21.15.55:0/435232077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2006f060 msgr2=0x7fdc10005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:56.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc24b9b700 1 --2- 172.21.15.55:0/435232077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2006f060 0x7fdc10005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:56.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc24b9b700 1 -- 172.21.15.55:0/435232077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc10004810 msgr2=0x7fdc10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:56.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc24b9b700 1 --2- 172.21.15.55:0/435232077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc10004810 0x7fdc10004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:56.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.306+0000 7fdc24b9b700 1 -- 172.21.15.55:0/435232077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc10003570 con 0x7fdc10003c20 2023-10-06T03:49:56.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.307+0000 7fdc24b9b700 1 --2- 172.21.15.55:0/435232077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc10003c20 0x7fdc10004010 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdc14001a10 tx=0x7fdc14007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7139e5575b19ce2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:56.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.307+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400c5d0 con 0x7fdc10003c20 2023-10-06T03:49:56.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.307+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1400c730 con 0x7fdc10003c20 2023-10-06T03:49:56.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.307+0000 7fdc1f7fe700 1 -- 172.21.15.55:0/435232077 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400cad0 con 0x7fdc10003c20 2023-10-06T03:49:56.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.307+0000 7fdc25b9d700 1 -- 172.21.15.55:0/435232077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc10003c20 msgr2=0x7fdc10004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:56.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.307+0000 7fdc25b9d700 1 --2- 172.21.15.55:0/435232077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc10003c20 0x7fdc10004010 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fdc14001a10 tx=0x7fdc14007d30 comp rx=0 tx=0).stop 2023-10-06T03:49:56.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.307+0000 7fdc2539c700 1 -- 172.21.15.55:0/435232077 reap_dead start 2023-10-06T03:49:56.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.308+0000 7fdc25b9d700 1 -- 172.21.15.55:0/435232077 shutdown_connections 2023-10-06T03:49:56.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.308+0000 7fdc25b9d700 1 -- 172.21.15.55:0/435232077 >> 172.21.15.55:0/435232077 conn(0x7fdc200cc1e0 msgr2=0x7fdc200cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:56.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.308+0000 7fdc25b9d700 1 -- 172.21.15.55:0/435232077 shutdown_connections 2023-10-06T03:49:56.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.308+0000 7fdc25b9d700 1 -- 172.21.15.55:0/435232077 wait complete. 2023-10-06T03:49:56.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.309+0000 7fdc25b9d700 1 Processor -- start 2023-10-06T03:49:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.309+0000 7fdc25b9d700 1 -- start start 2023-10-06T03:49:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.310+0000 7fdc25b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc10003c20 0x7fdc200afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.310+0000 7fdc25b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc201536a0 0x7fdc2014e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.310+0000 7fdc25b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc200c57b0 0x7fdc200c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.310+0000 7fdc25b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200c6150 con 0x7fdc201536a0 2023-10-06T03:49:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.310+0000 7fdc25b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc20153ea0 con 0x7fdc10003c20 2023-10-06T03:49:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.310+0000 7fdc25b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc20154180 con 0x7fdc200c57b0 2023-10-06T03:49:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.310+0000 7fdc1ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc201536a0 0x7fdc2014e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.310+0000 7fdc1ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc201536a0 0x7fdc2014e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51650/0 (socket says 172.21.15.55:51650) 2023-10-06T03:49:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.310+0000 7fdc1ffff700 1 -- 172.21.15.55:0/2609504647 learned_addr learned my addr 172.21.15.55:0/2609504647 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:56.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.311+0000 7fdc24b9b700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc10003c20 0x7fdc200afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.311+0000 7fdc2539c700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc200c57b0 0x7fdc200c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.311+0000 7fdc1ffff700 1 -- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc200c57b0 msgr2=0x7fdc200c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.311+0000 7fdc1ffff700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc200c57b0 0x7fdc200c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.311+0000 7fdc1ffff700 1 -- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc10003c20 msgr2=0x7fdc200afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.311+0000 7fdc1ffff700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc10003c20 0x7fdc200afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.311+0000 7fdc1ffff700 1 -- 172.21.15.55:0/2609504647 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc20154460 con 0x7fdc201536a0 2023-10-06T03:49:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.311+0000 7fdc24b9b700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc10003c20 0x7fdc200afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.311+0000 7fdc1ffff700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc201536a0 0x7fdc2014e500 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c011d40 tx=0x7fdc0c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.311+0000 7fdc1cff9700 1 -- 172.21.15.55:0/2609504647 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c00fac0 con 0x7fdc201536a0 2023-10-06T03:49:56.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.311+0000 7fdc1cff9700 1 -- 172.21.15.55:0/2609504647 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c00fc20 con 0x7fdc201536a0 2023-10-06T03:49:56.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.311+0000 7fdc25b9d700 1 -- 172.21.15.55:0/2609504647 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc20154740 con 0x7fdc201536a0 2023-10-06T03:49:56.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.311+0000 7fdc1cff9700 1 -- 172.21.15.55:0/2609504647 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c014440 con 0x7fdc201536a0 2023-10-06T03:49:56.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.311+0000 7fdc25b9d700 1 -- 172.21.15.55:0/2609504647 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc20153b10 con 0x7fdc201536a0 2023-10-06T03:49:56.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.312+0000 7fdc25b9d700 1 -- 172.21.15.55:0/2609504647 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2006c450 con 0x7fdc201536a0 2023-10-06T03:49:56.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.313+0000 7fdc1cff9700 1 -- 172.21.15.55:0/2609504647 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdc0c014a80 con 0x7fdc201536a0 2023-10-06T03:49:56.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.314+0000 7fdc1cff9700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fdc04066f30 0x7fdc040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:56.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.314+0000 7fdc1cff9700 1 -- 172.21.15.55:0/2609504647 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fdc0c018030 con 0x7fdc201536a0 2023-10-06T03:49:56.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.315+0000 7fdc24b9b700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fdc04066f30 0x7fdc040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:56.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.316+0000 7fdc24b9b700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fdc04066f30 0x7fdc040693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdc14008fd0 tx=0x7fdc14015000 comp rx=0 tx=0).ready entity=mgr.6799 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:56.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.319+0000 7fdc1cff9700 1 -- 172.21.15.55:0/2609504647 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc0c052500 con 0x7fdc201536a0 2023-10-06T03:49:56.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.578+0000 7fdc25b9d700 1 -- 172.21.15.55:0/2609504647 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fdc2015b000 con 0x7fdc201536a0 2023-10-06T03:49:56.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.825+0000 7fdc1cff9700 1 -- 172.21.15.55:0/2609504647 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 41) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fdc0c014d30 con 0x7fdc201536a0 2023-10-06T03:49:56.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.826+0000 7fdc1cff9700 1 -- 172.21.15.55:0/2609504647 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v41) v1 ==== 92+0+0 (secure 0 0 0) 0x7fdc0c052500 con 0x7fdc201536a0 2023-10-06T03:49:56.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.828+0000 7fdc25b9d700 1 -- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fdc04066f30 msgr2=0x7fdc040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:56.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.828+0000 7fdc25b9d700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fdc04066f30 0x7fdc040693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdc14008fd0 tx=0x7fdc14015000 comp rx=0 tx=0).stop 2023-10-06T03:49:56.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.828+0000 7fdc25b9d700 1 -- 172.21.15.55:0/2609504647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc201536a0 msgr2=0x7fdc2014e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:56.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.828+0000 7fdc25b9d700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc201536a0 0x7fdc2014e500 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c011d40 tx=0x7fdc0c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:56.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.828+0000 7fdc25b9d700 1 -- 172.21.15.55:0/2609504647 shutdown_connections 2023-10-06T03:49:56.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.828+0000 7fdc25b9d700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fdc04066f30 0x7fdc040693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:56.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.828+0000 7fdc25b9d700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc200c57b0 0x7fdc200c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:56.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.828+0000 7fdc25b9d700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc10003c20 0x7fdc200afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:56.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.828+0000 7fdc25b9d700 1 --2- 172.21.15.55:0/2609504647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc201536a0 0x7fdc2014e500 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:56.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.828+0000 7fdc25b9d700 1 -- 172.21.15.55:0/2609504647 >> 172.21.15.55:0/2609504647 conn(0x7fdc200cc1e0 msgr2=0x7fdc200c7250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:56.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.829+0000 7fdc25b9d700 1 -- 172.21.15.55:0/2609504647 shutdown_connections 2023-10-06T03:49:56.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:56.829+0000 7fdc25b9d700 1 -- 172.21.15.55:0/2609504647 wait complete. 2023-10-06T03:49:56.848 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T03:49:57.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.041+0000 7f120d39d700 1 Processor -- start 2023-10-06T03:49:57.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.041+0000 7f120d39d700 1 -- start start 2023-10-06T03:49:57.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.042+0000 7f120d39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12080b4e70 0x7f12080ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.042+0000 7f120d39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f12080b1b40 0x7f120815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.042+0000 7f120d39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12080cfb00 con 0x7f12080b16d0 2023-10-06T03:49:57.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.042+0000 7f120d39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12080b5c40 con 0x7f12080b1b40 2023-10-06T03:49:57.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.042+0000 7f120d39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12080b5f20 con 0x7f12080b4e70 2023-10-06T03:49:57.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.042+0000 7f120cb9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f12080b1b40 0x7f120815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.042+0000 7f120cb9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f12080b1b40 0x7f120815d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51668/0 (socket says 172.21.15.55:51668) 2023-10-06T03:49:57.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.042+0000 7f120cb9c700 1 -- 172.21.15.55:0/1568793312 learned_addr learned my addr 172.21.15.55:0/1568793312 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:57.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.042+0000 7f1207fff700 1 --2- 172.21.15.55:0/1568793312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12080b4e70 0x7f12080ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.043+0000 7f1206ffd700 1 -- 172.21.15.55:0/1568793312 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867555899 0 0) 0x7f12080cfb00 con 0x7f12080b16d0 2023-10-06T03:49:57.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.043+0000 7f120cb9c700 1 -- 172.21.15.55:0/1568793312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12080b4e70 msgr2=0x7f12080ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:57.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.043+0000 7f120cb9c700 1 --2- 172.21.15.55:0/1568793312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12080b4e70 0x7f12080ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.043+0000 7f120cb9c700 1 -- 172.21.15.55:0/1568793312 >> v1:172.21.15.163:6789/0 conn(0x7f12080b16d0 legacy=0x7f120815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:57.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.043+0000 7f120cb9c700 1 -- 172.21.15.55:0/1568793312 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12080b6200 con 0x7f12080b1b40 2023-10-06T03:49:57.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.043+0000 7f120cb9c700 1 --2- 172.21.15.55:0/1568793312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12080b1b40 0x7f120815d950 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1200009f20 tx=0x7f1200009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13949b60b4a3396 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:57.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.043+0000 7f1206ffd700 1 -- 172.21.15.55:0/1568793312 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120000e070 con 0x7f12080b1b40 2023-10-06T03:49:57.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.043+0000 7f1206ffd700 1 -- 172.21.15.55:0/1568793312 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120000b9d0 con 0x7f12080b1b40 2023-10-06T03:49:57.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.043+0000 7f1206ffd700 1 -- 172.21.15.55:0/1568793312 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12000092e0 con 0x7f12080b1b40 2023-10-06T03:49:57.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.044+0000 7f120d39d700 1 -- 172.21.15.55:0/1568793312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12080b1b40 msgr2=0x7f120815d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:57.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.044+0000 7f120d39d700 1 --2- 172.21.15.55:0/1568793312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12080b1b40 0x7f120815d950 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1200009f20 tx=0x7f1200009840 comp rx=0 tx=0).stop 2023-10-06T03:49:57.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.044+0000 7f120d39d700 1 -- 172.21.15.55:0/1568793312 shutdown_connections 2023-10-06T03:49:57.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.044+0000 7f120d39d700 1 --2- 172.21.15.55:0/1568793312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12080b1b40 0x7f120815d950 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.044+0000 7f120d39d700 1 --2- 172.21.15.55:0/1568793312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12080b4e70 0x7f12080ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.044+0000 7f120d39d700 1 -- 172.21.15.55:0/1568793312 >> 172.21.15.55:0/1568793312 conn(0x7f12080d10c0 msgr2=0x7f12080affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:57.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.044+0000 7f120d39d700 1 -- 172.21.15.55:0/1568793312 shutdown_connections 2023-10-06T03:49:57.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.044+0000 7f120d39d700 1 -- 172.21.15.55:0/1568793312 wait complete. 2023-10-06T03:49:57.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.046+0000 7f120d39d700 1 Processor -- start 2023-10-06T03:49:57.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.046+0000 7f120d39d700 1 -- start start 2023-10-06T03:49:57.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f120d39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12080b16d0 0x7f12080af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f120d39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12080b1b40 0x7f12080c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f120d39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12080b4e70 0x7f12080ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f120d39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12080af5a0 con 0x7f12080b1b40 2023-10-06T03:49:57.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f120d39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12080c9e10 con 0x7f12080b16d0 2023-10-06T03:49:57.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f120d39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12080ca0f0 con 0x7f12080b4e70 2023-10-06T03:49:57.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f12077fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12080b1b40 0x7f12080c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f12077fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12080b1b40 0x7f12080c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51676/0 (socket says 172.21.15.55:51676) 2023-10-06T03:49:57.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f12077fe700 1 -- 172.21.15.55:0/2610412335 learned_addr learned my addr 172.21.15.55:0/2610412335 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:57.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f1207fff700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12080b16d0 0x7f12080af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f120cb9c700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12080b4e70 0x7f12080ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f1207fff700 1 -- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12080b4e70 msgr2=0x7f12080ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:57.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f1207fff700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12080b4e70 0x7f12080ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f1207fff700 1 -- 172.21.15.55:0/2610412335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12080b1b40 msgr2=0x7f12080c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:57.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f1207fff700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12080b1b40 0x7f12080c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f1207fff700 1 -- 172.21.15.55:0/2610412335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1200009ae0 con 0x7f12080b16d0 2023-10-06T03:49:57.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.047+0000 7f12077fe700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12080b1b40 0x7f12080c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:57.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.048+0000 7f1207fff700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12080b16d0 0x7f12080af060 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f11f800e900 tx=0x7f11f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:57.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.048+0000 7f11effff700 1 -- 172.21.15.55:0/2610412335 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8013680 con 0x7f12080b16d0 2023-10-06T03:49:57.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.048+0000 7f11effff700 1 -- 172.21.15.55:0/2610412335 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f80137e0 con 0x7f12080b16d0 2023-10-06T03:49:57.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.048+0000 7f120d39d700 1 -- 172.21.15.55:0/2610412335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12080ca3d0 con 0x7f12080b16d0 2023-10-06T03:49:57.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.048+0000 7f11effff700 1 -- 172.21.15.55:0/2610412335 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f8013b80 con 0x7f12080b16d0 2023-10-06T03:49:57.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.048+0000 7f120d39d700 1 -- 172.21.15.55:0/2610412335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12080c7ea0 con 0x7f12080b16d0 2023-10-06T03:49:57.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.049+0000 7f120d39d700 1 -- 172.21.15.55:0/2610412335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12080b1510 con 0x7f12080b16d0 2023-10-06T03:49:57.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.050+0000 7f11effff700 1 -- 172.21.15.55:0/2610412335 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 41) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f11f80123d0 con 0x7f12080b16d0 2023-10-06T03:49:57.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.051+0000 7f11effff700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f4066f80 0x7f11f4069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.051+0000 7f11effff700 1 -- 172.21.15.55:0/2610412335 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f11f8084820 con 0x7f12080b16d0 2023-10-06T03:49:57.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.051+0000 7f12077fe700 1 -- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f4066f80 msgr2=0x7f11f4069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:57.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.051+0000 7f12077fe700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f4066f80 0x7f11f4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:57.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.055+0000 7f11effff700 1 -- 172.21.15.55:0/2610412335 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11f80517a0 con 0x7f12080b16d0 2023-10-06T03:49:57.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.252+0000 7f12077fe700 1 -- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f4066f80 msgr2=0x7f11f4069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:57.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.252+0000 7f12077fe700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f4066f80 0x7f11f4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:57.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.259+0000 7f120d39d700 1 -- 172.21.15.55:0/2610412335 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f12080b5a20 con 0x7f12080b16d0 2023-10-06T03:49:57.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.266+0000 7f11effff700 1 -- 172.21.15.55:0/2610412335 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f11f8054dc0 con 0x7f12080b16d0 2023-10-06T03:49:57.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.268+0000 7f120d39d700 1 -- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f4066f80 msgr2=0x7f11f4069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:57.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.268+0000 7f120d39d700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f4066f80 0x7f11f4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.268+0000 7f120d39d700 1 -- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12080b16d0 msgr2=0x7f12080af060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:57.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.268+0000 7f120d39d700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12080b16d0 0x7f12080af060 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f11f800e900 tx=0x7f11f80091a0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.268+0000 7f120d39d700 1 -- 172.21.15.55:0/2610412335 shutdown_connections 2023-10-06T03:49:57.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.268+0000 7f120d39d700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f11f4066f80 0x7f11f4069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.268+0000 7f120d39d700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f12080b4e70 0x7f12080ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.268+0000 7f120d39d700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f12080b16d0 0x7f12080af060 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.268+0000 7f120d39d700 1 --2- 172.21.15.55:0/2610412335 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f12080b1b40 0x7f12080c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.268+0000 7f120d39d700 1 -- 172.21.15.55:0/2610412335 >> 172.21.15.55:0/2610412335 conn(0x7f12080d10c0 msgr2=0x7f12080b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:57.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.268+0000 7f120d39d700 1 -- 172.21.15.55:0/2610412335 shutdown_connections 2023-10-06T03:49:57.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.268+0000 7f120d39d700 1 -- 172.21.15.55:0/2610412335 wait complete. 2023-10-06T03:49:57.288 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T03:49:57.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.478+0000 7fe2bc99d700 1 Processor -- start 2023-10-06T03:49:57.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.478+0000 7fe2bc99d700 1 -- start start 2023-10-06T03:49:57.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.479+0000 7fe2bc99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2b80b16d0 0x7fe2b80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.479+0000 7fe2bc99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe2b80b33b0 0x7fe2b80b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.479+0000 7fe2bc99d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b80b2080 con 0x7fe2b80b4e70 2023-10-06T03:49:57.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.479+0000 7fe2bc99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b80cfb00 con 0x7fe2b80b33b0 2023-10-06T03:49:57.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.479+0000 7fe2bc99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80b5c40 con 0x7fe2b80b16d0 2023-10-06T03:49:57.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.480+0000 7fe2b7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe2b80b33b0 0x7fe2b80b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.480+0000 7fe2b7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe2b80b33b0 0x7fe2b80b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51690/0 (socket says 172.21.15.55:51690) 2023-10-06T03:49:57.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.480+0000 7fe2b7fff700 1 -- 172.21.15.55:0/1050521897 learned_addr learned my addr 172.21.15.55:0/1050521897 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:57.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.480+0000 7fe2b6ffd700 1 --2- 172.21.15.55:0/1050521897 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2b80b16d0 0x7fe2b80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.480+0000 7fe2b7fff700 1 -- 172.21.15.55:0/1050521897 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2b80b16d0 msgr2=0x7fe2b80b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:57.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.480+0000 7fe2b7fff700 1 --2- 172.21.15.55:0/1050521897 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2b80b16d0 0x7fe2b80b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.480+0000 7fe2b7fff700 1 -- 172.21.15.55:0/1050521897 >> v1:172.21.15.163:6789/0 conn(0x7fe2b80b4e70 legacy=0x7fe2b80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:57.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.480+0000 7fe2b7fff700 1 -- 172.21.15.55:0/1050521897 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b80b5f20 con 0x7fe2b80b33b0 2023-10-06T03:49:57.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.480+0000 7fe2b67fc700 1 -- 172.21.15.55:0/1050521897 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 815869880 0 0) 0x7fe2b80b2080 con 0x7fe2b80b4e70 2023-10-06T03:49:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.481+0000 7fe2b7fff700 1 --2- 172.21.15.55:0/1050521897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2b80b33b0 0x7fe2b80b3830 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009f20 tx=0x7fe2b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9623456772b86bf7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.481+0000 7fe2b67fc700 1 -- 172.21.15.55:0/1050521897 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000e070 con 0x7fe2b80b33b0 2023-10-06T03:49:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.481+0000 7fe2b67fc700 1 -- 172.21.15.55:0/1050521897 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b000b9d0 con 0x7fe2b80b33b0 2023-10-06T03:49:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.481+0000 7fe2b67fc700 1 -- 172.21.15.55:0/1050521897 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00092e0 con 0x7fe2b80b33b0 2023-10-06T03:49:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.482+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1050521897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2b80b33b0 msgr2=0x7fe2b80b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.482+0000 7fe2bc99d700 1 --2- 172.21.15.55:0/1050521897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2b80b33b0 0x7fe2b80b3830 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009f20 tx=0x7fe2b0009840 comp rx=0 tx=0).stop 2023-10-06T03:49:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.482+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1050521897 shutdown_connections 2023-10-06T03:49:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.482+0000 7fe2bc99d700 1 --2- 172.21.15.55:0/1050521897 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2b80b33b0 0x7fe2b80b3830 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.482+0000 7fe2bc99d700 1 --2- 172.21.15.55:0/1050521897 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2b80b16d0 0x7fe2b80b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.482+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1050521897 >> 172.21.15.55:0/1050521897 conn(0x7fe2b80d10c0 msgr2=0x7fe2b80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:57.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.482+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1050521897 shutdown_connections 2023-10-06T03:49:57.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.482+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1050521897 wait complete. 2023-10-06T03:49:57.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.483+0000 7fe2bc99d700 1 Processor -- start 2023-10-06T03:49:57.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.483+0000 7fe2bc99d700 1 -- start start 2023-10-06T03:49:57.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.484+0000 7fe2bc99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2b80b16d0 0x7fe2b80ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.484+0000 7fe2bc99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2b80b33b0 0x7fe2b8068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.484+0000 7fe2bc99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2b80b4e70 0x7fe2b8068f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.484+0000 7fe2bc99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80c6660 con 0x7fe2b80b4e70 2023-10-06T03:49:57.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.484+0000 7fe2bc99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b816dfa0 con 0x7fe2b80b16d0 2023-10-06T03:49:57.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.484+0000 7fe2bc99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80c5d30 con 0x7fe2b80b33b0 2023-10-06T03:49:57.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.485+0000 7fe2b7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2b80b4e70 0x7fe2b8068f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.485+0000 7fe2b77fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2b80b16d0 0x7fe2b80ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.485+0000 7fe2b7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2b80b4e70 0x7fe2b8068f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51700/0 (socket says 172.21.15.55:51700) 2023-10-06T03:49:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.485+0000 7fe2b7fff700 1 -- 172.21.15.55:0/1931810337 learned_addr learned my addr 172.21.15.55:0/1931810337 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.485+0000 7fe2b6ffd700 1 --2- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2b80b33b0 0x7fe2b8068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.485+0000 7fe2b7fff700 1 -- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2b80b33b0 msgr2=0x7fe2b8068a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.485+0000 7fe2b7fff700 1 --2- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2b80b33b0 0x7fe2b8068a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.485+0000 7fe2b7fff700 1 -- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2b80b16d0 msgr2=0x7fe2b80ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.485+0000 7fe2b7fff700 1 --2- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2b80b16d0 0x7fe2b80ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.485+0000 7fe2b7fff700 1 -- 172.21.15.55:0/1931810337 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b0009ae0 con 0x7fe2b80b4e70 2023-10-06T03:49:57.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.485+0000 7fe2b6ffd700 1 --2- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2b80b33b0 0x7fe2b8068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:57.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.485+0000 7fe2b7fff700 1 --2- 172.21.15.55:0/1931810337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2b80b4e70 0x7fe2b8068f40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000b610 tx=0x7fe2b0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:57.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.486+0000 7fe29f7fe700 1 -- 172.21.15.55:0/1931810337 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00070a0 con 0x7fe2b80b4e70 2023-10-06T03:49:57.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.486+0000 7fe29f7fe700 1 -- 172.21.15.55:0/1931810337 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b0007b40 con 0x7fe2b80b4e70 2023-10-06T03:49:57.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.486+0000 7fe29f7fe700 1 -- 172.21.15.55:0/1931810337 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b001d420 con 0x7fe2b80b4e70 2023-10-06T03:49:57.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.486+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1931810337 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b80c6010 con 0x7fe2b80b4e70 2023-10-06T03:49:57.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.486+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1931810337 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b80ca790 con 0x7fe2b80b4e70 2023-10-06T03:49:57.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.487+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1931810337 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b8088990 con 0x7fe2b80b4e70 2023-10-06T03:49:57.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.489+0000 7fe29f7fe700 1 -- 172.21.15.55:0/1931810337 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 41) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fe2b0003050 con 0x7fe2b80b4e70 2023-10-06T03:49:57.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.490+0000 7fe29f7fe700 1 --2- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe2a4066f80 0x7fe2a4069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.490+0000 7fe29f7fe700 1 -- 172.21.15.55:0/1931810337 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fe2b0086490 con 0x7fe2b80b4e70 2023-10-06T03:49:57.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.490+0000 7fe2b77fe700 1 -- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe2a4066f80 msgr2=0x7fe2a4069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:57.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.490+0000 7fe2b77fe700 1 --2- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe2a4066f80 0x7fe2a4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:57.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.495+0000 7fe29f7fe700 1 -- 172.21.15.55:0/1931810337 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2b0053370 con 0x7fe2b80b4e70 2023-10-06T03:49:57.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.691+0000 7fe2b77fe700 1 -- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe2a4066f80 msgr2=0x7fe2a4069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:57.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.691+0000 7fe2b77fe700 1 --2- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe2a4066f80 0x7fe2a4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:57.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.701+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1931810337 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fe2b80c67a0 con 0x7fe2b80b4e70 2023-10-06T03:49:57.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.705+0000 7fe29f7fe700 1 -- 172.21.15.55:0/1931810337 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v36)=0 v36) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe2b0056990 con 0x7fe2b80b4e70 2023-10-06T03:49:57.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.707+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe2a4066f80 msgr2=0x7fe2a4069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:49:57.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.707+0000 7fe2bc99d700 1 --2- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe2a4066f80 0x7fe2a4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.707+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1931810337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2b80b4e70 msgr2=0x7fe2b8068f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:57.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.707+0000 7fe2bc99d700 1 --2- 172.21.15.55:0/1931810337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2b80b4e70 0x7fe2b8068f40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000b610 tx=0x7fe2b0018000 comp rx=0 tx=0).stop 2023-10-06T03:49:57.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.708+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1931810337 shutdown_connections 2023-10-06T03:49:57.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.708+0000 7fe2bc99d700 1 --2- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe2a4066f80 0x7fe2a4069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.708+0000 7fe2bc99d700 1 --2- 172.21.15.55:0/1931810337 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2b80b4e70 0x7fe2b8068f40 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.708+0000 7fe2bc99d700 1 --2- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2b80b16d0 0x7fe2b80ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.708+0000 7fe2bc99d700 1 --2- 172.21.15.55:0/1931810337 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2b80b33b0 0x7fe2b8068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.708+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1931810337 >> 172.21.15.55:0/1931810337 conn(0x7fe2b80d10c0 msgr2=0x7fe2b8158ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:57.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.708+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1931810337 shutdown_connections 2023-10-06T03:49:57.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.708+0000 7fe2bc99d700 1 -- 172.21.15.55:0/1931810337 wait complete. 2023-10-06T03:49:57.727 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /bad-path 1M 2023-10-06T03:49:57.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.922+0000 7f3d87fff700 1 Processor -- start 2023-10-06T03:49:57.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.923+0000 7f3d87fff700 1 -- start start 2023-10-06T03:49:57.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.924+0000 7f3d87fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3d88159450 0x7f3d8815d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.924+0000 7f3d87fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d88159820 0x7f3d8815d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.924+0000 7f3d87fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d880cfa80 con 0x7f3d880ce140 2023-10-06T03:49:57.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.924+0000 7f3d87fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d880b3670 con 0x7f3d88159450 2023-10-06T03:49:57.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.924+0000 7f3d87fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d880cee70 con 0x7f3d88159820 2023-10-06T03:49:57.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.924+0000 7f3d867fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d88159820 0x7f3d8815d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.924+0000 7f3d86ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3d88159450 0x7f3d8815d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.924+0000 7f3d867fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d88159820 0x7f3d8815d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:59422/0 (socket says 172.21.15.55:59422) 2023-10-06T03:49:57.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.924+0000 7f3d867fc700 1 -- 172.21.15.55:0/4025026437 learned_addr learned my addr 172.21.15.55:0/4025026437 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:57.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.925+0000 7f3d85ffb700 1 -- 172.21.15.55:0/4025026437 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372145920 0 0) 0x7f3d880cfa80 con 0x7f3d880ce140 2023-10-06T03:49:57.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.925+0000 7f3d85ffb700 1 -- 172.21.15.55:0/4025026437 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d70003570 con 0x7f3d880ce140 2023-10-06T03:49:57.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.925+0000 7f3d867fc700 1 -- 172.21.15.55:0/4025026437 >> v2:172.21.15.163:3300/0 conn(0x7f3d88159450 msgr2=0x7f3d8815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:49:57.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.925+0000 7f3d867fc700 1 --2- 172.21.15.55:0/4025026437 >> v2:172.21.15.163:3300/0 conn(0x7f3d88159450 0x7f3d8815d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.925+0000 7f3d867fc700 1 -- 172.21.15.55:0/4025026437 >> v1:172.21.15.163:6789/0 conn(0x7f3d880ce140 legacy=0x7f3d880ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:57.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.925+0000 7f3d867fc700 1 -- 172.21.15.55:0/4025026437 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d880cf150 con 0x7f3d88159820 2023-10-06T03:49:57.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.925+0000 7f3d86ffd700 1 --2- 172.21.15.55:0/4025026437 >> v2:172.21.15.163:3300/0 conn(0x7f3d88159450 0x7f3d8815d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:49:57.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.925+0000 7f3d867fc700 1 --2- 172.21.15.55:0/4025026437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d88159820 0x7f3d8815d780 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0099b0 tx=0x7f3d7c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac757856f68c0c3d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:57.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.925+0000 7f3d85ffb700 1 -- 172.21.15.55:0/4025026437 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c00b680 con 0x7f3d88159820 2023-10-06T03:49:57.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.925+0000 7f3d85ffb700 1 -- 172.21.15.55:0/4025026437 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7c00b7e0 con 0x7f3d88159820 2023-10-06T03:49:57.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.925+0000 7f3d85ffb700 1 -- 172.21.15.55:0/4025026437 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c00bb80 con 0x7f3d88159820 2023-10-06T03:49:57.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.926+0000 7f3d87fff700 1 -- 172.21.15.55:0/4025026437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d88159820 msgr2=0x7f3d8815d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:57.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.926+0000 7f3d87fff700 1 --2- 172.21.15.55:0/4025026437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d88159820 0x7f3d8815d780 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0099b0 tx=0x7f3d7c009250 comp rx=0 tx=0).stop 2023-10-06T03:49:57.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.926+0000 7f3d87fff700 1 -- 172.21.15.55:0/4025026437 shutdown_connections 2023-10-06T03:49:57.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.926+0000 7f3d87fff700 1 --2- 172.21.15.55:0/4025026437 >> v2:172.21.15.163:3300/0 conn(0x7f3d88159450 0x7f3d8815d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.926+0000 7f3d87fff700 1 --2- 172.21.15.55:0/4025026437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d88159820 0x7f3d8815d780 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.926+0000 7f3d87fff700 1 -- 172.21.15.55:0/4025026437 >> 172.21.15.55:0/4025026437 conn(0x7f3d880d0570 msgr2=0x7f3d880d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:49:57.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.926+0000 7f3d87fff700 1 -- 172.21.15.55:0/4025026437 shutdown_connections 2023-10-06T03:49:57.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.926+0000 7f3d87fff700 1 -- 172.21.15.55:0/4025026437 wait complete. 2023-10-06T03:49:57.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.927+0000 7f3d87fff700 1 Processor -- start 2023-10-06T03:49:57.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.928+0000 7f3d87fff700 1 -- start start 2023-10-06T03:49:57.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.928+0000 7f3d87fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d880ce140 0x7f3d88157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d87fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d88159450 0x7f3d88157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d87fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d88159820 0x7f3d88157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d87fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8816be10 con 0x7f3d880ce140 2023-10-06T03:49:57.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d87fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8816b270 con 0x7f3d88159450 2023-10-06T03:49:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d87fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8816b4e0 con 0x7f3d88159820 2023-10-06T03:49:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d86ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d880ce140 0x7f3d88157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d86ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d880ce140 0x7f3d88157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51712/0 (socket says 172.21.15.55:51712) 2023-10-06T03:49:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d86ffd700 1 -- 172.21.15.55:0/4205266328 learned_addr learned my addr 172.21.15.55:0/4205266328 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:49:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d867fc700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d88159450 0x7f3d88157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d877fe700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d88159820 0x7f3d88157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:49:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d86ffd700 1 -- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d88159820 msgr2=0x7f3d88157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d86ffd700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d88159820 0x7f3d88157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d86ffd700 1 -- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d88159450 msgr2=0x7f3d88157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:49:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d86ffd700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d88159450 0x7f3d88157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:49:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d86ffd700 1 -- 172.21.15.55:0/4205266328 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d78009420 con 0x7f3d880ce140 2023-10-06T03:49:57.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.929+0000 7f3d877fe700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d88159820 0x7f3d88157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:49:57.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.930+0000 7f3d86ffd700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d880ce140 0x7f3d88157540 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800ec80 tx=0x7f3d780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:49:57.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.930+0000 7f3d6effd700 1 -- 172.21.15.55:0/4205266328 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800c8f0 con 0x7f3d880ce140 2023-10-06T03:49:57.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.930+0000 7f3d6effd700 1 -- 172.21.15.55:0/4205266328 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7800ca50 con 0x7f3d880ce140 2023-10-06T03:49:57.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.930+0000 7f3d87fff700 1 -- 172.21.15.55:0/4205266328 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c0094f0 con 0x7f3d880ce140 2023-10-06T03:49:57.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.930+0000 7f3d6effd700 1 -- 172.21.15.55:0/4205266328 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800cdd0 con 0x7f3d880ce140 2023-10-06T03:49:57.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.930+0000 7f3d87fff700 1 -- 172.21.15.55:0/4205266328 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8816baa0 con 0x7f3d880ce140 2023-10-06T03:49:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.931+0000 7f3d87fff700 1 -- 172.21.15.55:0/4205266328 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8816c0f0 con 0x7f3d880ce140 2023-10-06T03:49:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.932+0000 7f3d6effd700 1 -- 172.21.15.55:0/4205266328 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 41) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f3d78013680 con 0x7f3d880ce140 2023-10-06T03:49:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.933+0000 7f3d6effd700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 0x7f3d74069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:49:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.933+0000 7f3d6effd700 1 -- 172.21.15.55:0/4205266328 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f3d78084bd0 con 0x7f3d880ce140 2023-10-06T03:49:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.933+0000 7f3d867fc700 1 -- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 msgr2=0x7f3d74069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:57.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.933+0000 7f3d867fc700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 0x7f3d74069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:49:57.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:57.937+0000 7f3d6effd700 1 -- 172.21.15.55:0/4205266328 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d78051ab0 con 0x7f3d880ce140 2023-10-06T03:49:58.127 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:58.126+0000 7fb8b12fc180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:49:58.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:58.134+0000 7f3d867fc700 1 -- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 msgr2=0x7f3d74069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:58.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:58.134+0000 7f3d867fc700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 0x7f3d74069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:49:58.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:58.169+0000 7f3d87fff700 1 -- 172.21.15.55:0/4205266328 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d8816bf50 con 0x7f3d74066f80 2023-10-06T03:49:58.184 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:58.183+0000 7f9ab2649180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:49:58.291 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:58.290+0000 7fb8b12fc180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:49:58.355 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:58.354+0000 7f9ab2649180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:49:58.377 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:58.376+0000 7fb8b12fc180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:49:58.444 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:58.443+0000 7f9ab2649180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:49:58.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:58.535+0000 7f3d867fc700 1 -- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 msgr2=0x7f3d74069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:58.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:58.535+0000 7f3d867fc700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 0x7f3d74069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T03:49:58.601 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:58.600+0000 7fb8b12fc180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:49:58.678 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:58.676+0000 7f9ab2649180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:49:58.777 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:58.776+0000 7fb8b12fc180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:49:58.866 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:58.862+0000 7f9ab2649180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:49:58.869 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:58.868+0000 7fb8b12fc180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:49:58.960 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:58.959+0000 7f9ab2649180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:49:59.033 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:59.032+0000 7fb8b12fc180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:49:59.128 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:59.127+0000 7f9ab2649180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:49:59.134 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:59.133+0000 7fb8b12fc180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:49:59.233 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:59.232+0000 7f9ab2649180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:49:59.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:59.336+0000 7f3d867fc700 1 -- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 msgr2=0x7f3d74069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:49:59.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:49:59.336+0000 7f3d867fc700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 0x7f3d74069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T03:49:59.616 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:59.615+0000 7fb8b12fc180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:49:59.710 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:59.709+0000 7fb8b12fc180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:49:59.738 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:59.737+0000 7f9ab2649180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:49:59.801 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:49:59.800+0000 7fb8b12fc180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:49:59.837 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:59.835+0000 7f9ab2649180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:49:59.931 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:49:59.930+0000 7f9ab2649180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:50:00.427 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:00.426+0000 7fb8b12fc180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:50:00.583 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:50:00.582+0000 7f9ab2649180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:50:00.599 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:00.598+0000 7fb8b12fc180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:50:00.698 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:00.696+0000 7fb8b12fc180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:50:00.764 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:50:00.762+0000 7f9ab2649180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:50:00.864 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:50:00.863+0000 7f9ab2649180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:50:00.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:00.938+0000 7f3d867fc700 1 -- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 msgr2=0x7f3d74069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:50:00.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:00.938+0000 7f3d867fc700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 0x7f3d74069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T03:50:00.994 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:00.993+0000 7fb8b12fc180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:50:01.092 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:01.091+0000 7fb8b12fc180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:50:01.167 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:50:01.165+0000 7f9ab2649180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:50:01.267 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:50:01.266+0000 7f9ab2649180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:50:01.799 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:01.798+0000 7fb8b12fc180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:50:02.005 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:50:02.003+0000 7f9ab2649180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:50:02.086 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:02.084+0000 7fb8b12fc180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:50:02.180 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:02.177+0000 7fb8b12fc180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:50:02.298 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:50:02.297+0000 7f9ab2649180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:50:02.332 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:02.330+0000 7fb8b12fc180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:50:02.395 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:50:02.393+0000 7f9ab2649180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:50:02.436 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:02.434+0000 7fb8b12fc180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:50:02.558 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:50:02.557+0000 7f9ab2649180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:50:02.628 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:02.626+0000 7fb8b12fc180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:50:02.667 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:50:02.666+0000 7f9ab2649180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:50:02.721 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:02.719+0000 7fb8b12fc180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:50:02.815 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:02.813+0000 7fb8b12fc180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:50:02.868 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:50:02.867+0000 7f9ab2649180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:50:02.909 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:02.907+0000 7fb8b12fc180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:50:02.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:02.920+0000 7f3d6effd700 1 -- 172.21.15.55:0/4205266328 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 42) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f3d78051e80 con 0x7f3d880ce140 2023-10-06T03:50:02.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:02.921+0000 7f3d6effd700 1 -- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 msgr2=0x7f3d74069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:50:02.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:02.921+0000 7f3d6effd700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 0x7f3d74069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:50:02.972 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:50:02.971+0000 7f9ab2649180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:50:03.045 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:03.043+0000 7fb823631700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:50:03.046 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:03.043+0000 7fb823631700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:50:03.046 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:03.043+0000 7fb823631700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:50:03.046 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:03.043+0000 7fb823631700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:50:03.046 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:03.043+0000 7fb823631700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:50:03.048 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:03.046+0000 7fb826637700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:50:03.048 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:03.046+0000 7fb826637700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:50:03.048 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:03.046+0000 7fb826637700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:50:03.048 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:03.046+0000 7fb826637700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:50:03.049 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:50:03.046+0000 7fb826637700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:50:03.075 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:50:03.074+0000 7f9ab2649180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:50:03.174 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:50:03.172+0000 7f9ab2649180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:50:03.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.925+0000 7f3d6effd700 1 -- 172.21.15.55:0/4205266328 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mgrmap(e 43) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f3d7805c280 con 0x7f3d880ce140 2023-10-06T03:50:03.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.925+0000 7f3d6effd700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 0x7f3d740693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:50:03.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.925+0000 7f3d6effd700 1 -- 172.21.15.55:0/4205266328 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d7400e8c0 con 0x7f3d74066f80 2023-10-06T03:50:03.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.929+0000 7f3d867fc700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 0x7f3d740693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:50:03.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.933+0000 7f3d867fc700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 0x7f3d740693b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0099b0 tx=0x7f3d7c017000 comp rx=0 tx=0).ready entity=mgr.6422 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:50:03.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.996+0000 7f3d6effd700 1 -- 172.21.15.55:0/4205266328 <== mgr.6422 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f3d7400e8c0 con 0x7f3d74066f80 2023-10-06T03:50:03.998 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path /bad-path 2023-10-06T03:50:04.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.999+0000 7f3d87fff700 1 -- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 msgr2=0x7f3d740693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:50:04.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.999+0000 7f3d87fff700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 0x7f3d740693b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0099b0 tx=0x7f3d7c017000 comp rx=0 tx=0).stop 2023-10-06T03:50:04.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.999+0000 7f3d87fff700 1 -- 172.21.15.55:0/4205266328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d880ce140 msgr2=0x7f3d88157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:50:04.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.999+0000 7f3d87fff700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d880ce140 0x7f3d88157540 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800ec80 tx=0x7f3d780091a0 comp rx=0 tx=0).stop 2023-10-06T03:50:04.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.999+0000 7f3d87fff700 1 -- 172.21.15.55:0/4205266328 shutdown_connections 2023-10-06T03:50:04.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.999+0000 7f3d87fff700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d74066f80 0x7f3d740693b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:50:04.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.999+0000 7f3d87fff700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d88159820 0x7f3d88157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:50:04.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.999+0000 7f3d87fff700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d880ce140 0x7f3d88157540 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:50:04.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.999+0000 7f3d87fff700 1 --2- 172.21.15.55:0/4205266328 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d88159450 0x7f3d88157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:50:04.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.999+0000 7f3d87fff700 1 -- 172.21.15.55:0/4205266328 >> 172.21.15.55:0/4205266328 conn(0x7f3d880d0570 msgr2=0x7f3d8815bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:50:04.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.999+0000 7f3d87fff700 1 -- 172.21.15.55:0/4205266328 shutdown_connections 2023-10-06T03:50:04.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:03.999+0000 7f3d87fff700 1 -- 172.21.15.55:0/4205266328 wait complete. 2023-10-06T03:50:04.021 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-10-06T03:50:04.021 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-10-06T03:50:04.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.216+0000 7fc9aa59d700 1 Processor -- start 2023-10-06T03:50:04.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.216+0000 7fc9aa59d700 1 -- start start 2023-10-06T03:50:04.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.217+0000 7fc9aa59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9a40ae180 0x7fc9a40acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:50:04.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.217+0000 7fc9aa59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a40aeb10 con 0x7fc9a406b290 2023-10-06T03:50:04.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.217+0000 7fc9aa59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a40af540 con 0x7fc9a40656e0 2023-10-06T03:50:04.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.217+0000 7fc9aa59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a40aedf0 con 0x7fc9a40ae180 2023-10-06T03:50:04.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.217+0000 7fc9a9d9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc9a406b290 0x7fc9a40b1110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34854/0 (socket says 172.21.15.55:34854) 2023-10-06T03:50:04.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.217+0000 7fc9a9d9c700 1 -- 172.21.15.55:0/3906598231 learned_addr learned my addr 172.21.15.55:0/3906598231 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:50:04.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.217+0000 7fc9a959b700 1 --2- 172.21.15.55:0/3906598231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9a40ae180 0x7fc9a40acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:50:04.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.217+0000 7fc99bfff700 1 -- 172.21.15.55:0/3906598231 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812725471 0 0) 0x7fc9a40aeb10 con 0x7fc9a406b290 2023-10-06T03:50:04.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.218+0000 7fc99bfff700 1 -- 172.21.15.55:0/3906598231 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc990003570 con 0x7fc9a406b290 2023-10-06T03:50:04.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.218+0000 7fc99bfff700 1 -- 172.21.15.55:0/3906598231 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731568733 0 0) 0x7fc9a40af540 con 0x7fc9a40656e0 2023-10-06T03:50:04.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.218+0000 7fc99bfff700 1 -- 172.21.15.55:0/3906598231 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9a40aeb10 con 0x7fc9a40656e0 2023-10-06T03:50:04.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.218+0000 7fc9a959b700 1 -- 172.21.15.55:0/3906598231 >> v1:172.21.15.163:6789/0 conn(0x7fc9a40656e0 legacy=0x7fc9a40ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:50:04.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.218+0000 7fc9a959b700 1 -- 172.21.15.55:0/3906598231 >> v1:172.21.15.55:6789/0 conn(0x7fc9a406b290 legacy=0x7fc9a40b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:50:04.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.218+0000 7fc9a959b700 1 -- 172.21.15.55:0/3906598231 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a40af820 con 0x7fc9a40ae180 2023-10-06T03:50:04.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.218+0000 7fc9a959b700 1 --2- 172.21.15.55:0/3906598231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9a40ae180 0x7fc9a40acb50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009900 tx=0x7fc9a0009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f1f5e292c41ed14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:50:04.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.218+0000 7fc99bfff700 1 -- 172.21.15.55:0/3906598231 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0005c10 con 0x7fc9a40ae180 2023-10-06T03:50:04.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.219+0000 7fc99bfff700 1 -- 172.21.15.55:0/3906598231 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a0005d70 con 0x7fc9a40ae180 2023-10-06T03:50:04.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.219+0000 7fc99bfff700 1 -- 172.21.15.55:0/3906598231 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0006260 con 0x7fc9a40ae180 2023-10-06T03:50:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.219+0000 7fc9aa59d700 1 -- 172.21.15.55:0/3906598231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9a40ae180 msgr2=0x7fc9a40acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:50:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.219+0000 7fc9aa59d700 1 --2- 172.21.15.55:0/3906598231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9a40ae180 0x7fc9a40acb50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009900 tx=0x7fc9a0009120 comp rx=0 tx=0).stop 2023-10-06T03:50:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.219+0000 7fc9aa59d700 1 -- 172.21.15.55:0/3906598231 shutdown_connections 2023-10-06T03:50:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.219+0000 7fc9aa59d700 1 --2- 172.21.15.55:0/3906598231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9a40ae180 0x7fc9a40acb50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:50:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.219+0000 7fc9aa59d700 1 -- 172.21.15.55:0/3906598231 >> 172.21.15.55:0/3906598231 conn(0x7fc9a406f6a0 msgr2=0x7fc9a406ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:50:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.219+0000 7fc9aa59d700 1 -- 172.21.15.55:0/3906598231 shutdown_connections 2023-10-06T03:50:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.219+0000 7fc9aa59d700 1 -- 172.21.15.55:0/3906598231 wait complete. 2023-10-06T03:50:04.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.221+0000 7fc9aa59d700 1 Processor -- start 2023-10-06T03:50:04.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.221+0000 7fc9aa59d700 1 -- start start 2023-10-06T03:50:04.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.221+0000 7fc9aa59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9a40656e0 0x7fc9a40cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:50:04.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9aa59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9a406b290 0x7fc9a415a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:50:04.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9aa59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9a40ae180 0x7fc9a41578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:50:04.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9aa59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a415b940 con 0x7fc9a406b290 2023-10-06T03:50:04.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9aa59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a415aa90 con 0x7fc9a40ae180 2023-10-06T03:50:04.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9aa59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a415ad30 con 0x7fc9a40656e0 2023-10-06T03:50:04.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9a8d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9a406b290 0x7fc9a415a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:50:04.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9a8d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9a406b290 0x7fc9a415a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53490/0 (socket says 172.21.15.55:53490) 2023-10-06T03:50:04.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9a8d9a700 1 -- 172.21.15.55:0/2096930318 learned_addr learned my addr 172.21.15.55:0/2096930318 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:50:04.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9a9d9c700 1 --2- 172.21.15.55:0/2096930318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9a40ae180 0x7fc9a41578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:50:04.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9a959b700 1 --2- 172.21.15.55:0/2096930318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9a40656e0 0x7fc9a40cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:50:04.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9a8d9a700 1 -- 172.21.15.55:0/2096930318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9a40656e0 msgr2=0x7fc9a40cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:50:04.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9a8d9a700 1 --2- 172.21.15.55:0/2096930318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9a40656e0 0x7fc9a40cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:50:04.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9a8d9a700 1 -- 172.21.15.55:0/2096930318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9a40ae180 msgr2=0x7fc9a41578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:50:04.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9a8d9a700 1 --2- 172.21.15.55:0/2096930318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9a40ae180 0x7fc9a41578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:50:04.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9a8d9a700 1 -- 172.21.15.55:0/2096930318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a00093c0 con 0x7fc9a406b290 2023-10-06T03:50:04.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.222+0000 7fc9a8d9a700 1 --2- 172.21.15.55:0/2096930318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9a406b290 0x7fc9a415a550 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc99400c7c0 tx=0x7fc994007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:50:04.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.223+0000 7fc9997fa700 1 -- 172.21.15.55:0/2096930318 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994010510 con 0x7fc9a406b290 2023-10-06T03:50:04.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.223+0000 7fc9997fa700 1 -- 172.21.15.55:0/2096930318 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc994010690 con 0x7fc9a406b290 2023-10-06T03:50:04.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.223+0000 7fc9aa59d700 1 -- 172.21.15.55:0/2096930318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a415b010 con 0x7fc9a406b290 2023-10-06T03:50:04.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.223+0000 7fc9997fa700 1 -- 172.21.15.55:0/2096930318 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994010a50 con 0x7fc9a406b290 2023-10-06T03:50:04.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.223+0000 7fc9aa59d700 1 -- 172.21.15.55:0/2096930318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a415b5d0 con 0x7fc9a406b290 2023-10-06T03:50:04.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.224+0000 7fc9aa59d700 1 -- 172.21.15.55:0/2096930318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a404c3c0 con 0x7fc9a406b290 2023-10-06T03:50:04.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.225+0000 7fc9997fa700 1 -- 172.21.15.55:0/2096930318 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 43) v1 ==== 84595+0+0 (secure 0 0 0) 0x7fc99401b400 con 0x7fc9a406b290 2023-10-06T03:50:04.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.225+0000 7fc9997fa700 1 --2- 172.21.15.55:0/2096930318 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc988066dd0 0x7fc988069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:50:04.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.225+0000 7fc9997fa700 1 -- 172.21.15.55:0/2096930318 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6708+0+0 (secure 0 0 0) 0x7fc994083e00 con 0x7fc9a406b290 2023-10-06T03:50:04.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.226+0000 7fc9a959b700 1 --2- 172.21.15.55:0/2096930318 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc988066dd0 0x7fc988069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:50:04.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.226+0000 7fc9a959b700 1 --2- 172.21.15.55:0/2096930318 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc988066dd0 0x7fc988069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009390 tx=0x7fc9a0016000 comp rx=0 tx=0).ready entity=mgr.6422 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:50:04.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.230+0000 7fc9997fa700 1 -- 172.21.15.55:0/2096930318 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc99404b5a0 con 0x7fc9a406b290 2023-10-06T03:50:04.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.460+0000 7fc9aa59d700 1 -- 172.21.15.55:0/2096930318 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc9a415ba80 con 0x7fc988066dd0 2023-10-06T03:50:04.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.688+0000 7fc9997fa700 1 -- 172.21.15.55:0/2096930318 <== mgr.6422 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7fc9a415ba80 con 0x7fc988066dd0 2023-10-06T03:50:04.689 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:50:04.689 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:50:03", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-06T03:50:04.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.690+0000 7fc9aa59d700 1 -- 172.21.15.55:0/2096930318 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc988066dd0 msgr2=0x7fc988069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:50:04.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.690+0000 7fc9aa59d700 1 --2- 172.21.15.55:0/2096930318 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc988066dd0 0x7fc988069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009390 tx=0x7fc9a0016000 comp rx=0 tx=0).stop 2023-10-06T03:50:04.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.690+0000 7fc9aa59d700 1 -- 172.21.15.55:0/2096930318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9a406b290 msgr2=0x7fc9a415a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:50:04.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.690+0000 7fc9aa59d700 1 --2- 172.21.15.55:0/2096930318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9a406b290 0x7fc9a415a550 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc99400c7c0 tx=0x7fc994007db0 comp rx=0 tx=0).stop 2023-10-06T03:50:04.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.690+0000 7fc9aa59d700 1 -- 172.21.15.55:0/2096930318 shutdown_connections 2023-10-06T03:50:04.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.690+0000 7fc9aa59d700 1 --2- 172.21.15.55:0/2096930318 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc988066dd0 0x7fc988069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:50:04.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.690+0000 7fc9aa59d700 1 --2- 172.21.15.55:0/2096930318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9a40ae180 0x7fc9a41578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:50:04.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.690+0000 7fc9aa59d700 1 --2- 172.21.15.55:0/2096930318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9a40656e0 0x7fc9a40cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:50:04.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.690+0000 7fc9aa59d700 1 --2- 172.21.15.55:0/2096930318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9a406b290 0x7fc9a415a550 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:50:04.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.690+0000 7fc9aa59d700 1 -- 172.21.15.55:0/2096930318 >> 172.21.15.55:0/2096930318 conn(0x7fc9a406f6a0 msgr2=0x7fc9a40ace00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:50:04.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.690+0000 7fc9aa59d700 1 -- 172.21.15.55:0/2096930318 shutdown_connections 2023-10-06T03:50:04.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:50:04.690+0000 7fc9aa59d700 1 -- 172.21.15.55:0/2096930318 wait complete. 2023-10-06T03:50:04.709 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:50:03", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-06T03:50:16.029 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:50:16.035 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:50:46.143 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:50:46.148 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:51:01.009 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:01.007+0000 7fb815e16700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:01.009 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:01.007+0000 7fb815e16700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:01.009 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:01.007+0000 7fb815e16700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:01.009 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:01.007+0000 7fb815e16700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:01.009 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:01.007+0000 7fb815e16700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:04.710 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-10-06T03:51:04.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.915+0000 7f1c8dd9d700 1 Processor -- start 2023-10-06T03:51:04.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.915+0000 7f1c8dd9d700 1 -- start start 2023-10-06T03:51:04.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8dd9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c88156320 0x7f1c880c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:04.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8dd9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c880cb8a0 0x7f1c88159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:04.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8dd9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1c880ca280 0x7f1c8815a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:04.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8dd9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c880cb410 con 0x7f1c88156320 2023-10-06T03:51:04.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8dd9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c8815ace0 con 0x7f1c880ca280 2023-10-06T03:51:04.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8dd9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c880ca800 con 0x7f1c880cb8a0 2023-10-06T03:51:04.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8cd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c88156320 0x7f1c880c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:04.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8cd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c88156320 0x7f1c880c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54172/0 (socket says 172.21.15.55:54172) 2023-10-06T03:51:04.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8cd9b700 1 -- 172.21.15.55:0/1177994083 learned_addr learned my addr 172.21.15.55:0/1177994083 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:04.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c87fff700 1 --2- 172.21.15.55:0/1177994083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c880cb8a0 0x7f1c88159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:04.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8d59c700 1 --2- 172.21.15.55:0/1177994083 >> v2:172.21.15.163:3300/0 conn(0x7f1c880ca280 0x7f1c8815a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:04.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8cd9b700 1 -- 172.21.15.55:0/1177994083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c880cb8a0 msgr2=0x7f1c88159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:04.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8cd9b700 1 --2- 172.21.15.55:0/1177994083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c880cb8a0 0x7f1c88159eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:04.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8cd9b700 1 -- 172.21.15.55:0/1177994083 >> v2:172.21.15.163:3300/0 conn(0x7f1c880ca280 msgr2=0x7f1c8815a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:04.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8cd9b700 1 --2- 172.21.15.55:0/1177994083 >> v2:172.21.15.163:3300/0 conn(0x7f1c880ca280 0x7f1c8815a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:04.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8cd9b700 1 -- 172.21.15.55:0/1177994083 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c880caae0 con 0x7f1c88156320 2023-10-06T03:51:04.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.917+0000 7f1c8d59c700 1 --2- 172.21.15.55:0/1177994083 >> v2:172.21.15.163:3300/0 conn(0x7f1c880ca280 0x7f1c8815a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:04.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.918+0000 7f1c8cd9b700 1 --2- 172.21.15.55:0/1177994083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c88156320 0x7f1c880c9d40 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1c8806f9e0 tx=0x7f1c7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c415e881645291e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:04.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.918+0000 7f1c877fe700 1 -- 172.21.15.55:0/1177994083 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c009e70 con 0x7f1c88156320 2023-10-06T03:51:04.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.918+0000 7f1c877fe700 1 -- 172.21.15.55:0/1177994083 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7c00b860 con 0x7f1c88156320 2023-10-06T03:51:04.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.918+0000 7f1c877fe700 1 -- 172.21.15.55:0/1177994083 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c00bc00 con 0x7f1c88156320 2023-10-06T03:51:04.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.918+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/1177994083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c88156320 msgr2=0x7f1c880c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:04.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.918+0000 7f1c8dd9d700 1 --2- 172.21.15.55:0/1177994083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c88156320 0x7f1c880c9d40 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1c8806f9e0 tx=0x7f1c7c009250 comp rx=0 tx=0).stop 2023-10-06T03:51:04.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.919+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/1177994083 shutdown_connections 2023-10-06T03:51:04.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.919+0000 7f1c8dd9d700 1 --2- 172.21.15.55:0/1177994083 >> v2:172.21.15.163:3300/0 conn(0x7f1c880ca280 0x7f1c8815a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:04.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.919+0000 7f1c8dd9d700 1 --2- 172.21.15.55:0/1177994083 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c88156320 0x7f1c880c9d40 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:04.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.919+0000 7f1c8dd9d700 1 --2- 172.21.15.55:0/1177994083 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c880cb8a0 0x7f1c88159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:04.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.919+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/1177994083 >> 172.21.15.55:0/1177994083 conn(0x7f1c880c8be0 msgr2=0x7f1c880c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:04.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.919+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/1177994083 shutdown_connections 2023-10-06T03:51:04.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.919+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/1177994083 wait complete. 2023-10-06T03:51:04.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.920+0000 7f1c8dd9d700 1 Processor -- start 2023-10-06T03:51:04.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.920+0000 7f1c8dd9d700 1 -- start start 2023-10-06T03:51:04.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.921+0000 7f1c8dd9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c880ca280 0x7f1c88149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:04.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.921+0000 7f1c8dd9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c880cb8a0 0x7f1c880b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:04.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.921+0000 7f1c8dd9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c88156320 0x7f1c880ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:04.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.921+0000 7f1c8dd9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8814b7f0 con 0x7f1c880cb8a0 2023-10-06T03:51:04.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.921+0000 7f1c8dd9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8814abe0 con 0x7f1c88156320 2023-10-06T03:51:04.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.921+0000 7f1c8dd9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8814aec0 con 0x7f1c880ca280 2023-10-06T03:51:04.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.921+0000 7f1c87fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c880cb8a0 0x7f1c880b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:04.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.921+0000 7f1c87fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c880cb8a0 0x7f1c880b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54174/0 (socket says 172.21.15.55:54174) 2023-10-06T03:51:04.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.921+0000 7f1c87fff700 1 -- 172.21.15.55:0/2400558312 learned_addr learned my addr 172.21.15.55:0/2400558312 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:04.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.922+0000 7f1c8d59c700 1 --2- 172.21.15.55:0/2400558312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c88156320 0x7f1c880ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:04.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.922+0000 7f1c8cd9b700 1 --2- 172.21.15.55:0/2400558312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c880ca280 0x7f1c88149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:04.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.922+0000 7f1c87fff700 1 -- 172.21.15.55:0/2400558312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c880ca280 msgr2=0x7f1c88149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:04.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.922+0000 7f1c87fff700 1 --2- 172.21.15.55:0/2400558312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c880ca280 0x7f1c88149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:04.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.922+0000 7f1c87fff700 1 -- 172.21.15.55:0/2400558312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c88156320 msgr2=0x7f1c880ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:04.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.922+0000 7f1c87fff700 1 --2- 172.21.15.55:0/2400558312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c88156320 0x7f1c880ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:04.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.922+0000 7f1c87fff700 1 -- 172.21.15.55:0/2400558312 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c0094f0 con 0x7f1c880cb8a0 2023-10-06T03:51:04.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.922+0000 7f1c87fff700 1 --2- 172.21.15.55:0/2400558312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c880cb8a0 0x7f1c880b10a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400ea70 tx=0x7f1c740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:04.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.922+0000 7f1c84ff9700 1 -- 172.21.15.55:0/2400558312 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7400c920 con 0x7f1c880cb8a0 2023-10-06T03:51:04.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.922+0000 7f1c84ff9700 1 -- 172.21.15.55:0/2400558312 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7400caa0 con 0x7f1c880cb8a0 2023-10-06T03:51:04.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.922+0000 7f1c84ff9700 1 -- 172.21.15.55:0/2400558312 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7400ce60 con 0x7f1c880cb8a0 2023-10-06T03:51:04.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.922+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/2400558312 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8814b1a0 con 0x7f1c880cb8a0 2023-10-06T03:51:04.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.923+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/2400558312 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8814a870 con 0x7f1c880cb8a0 2023-10-06T03:51:04.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.923+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/2400558312 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c8814b9a0 con 0x7f1c880cb8a0 2023-10-06T03:51:04.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.924+0000 7f1c84ff9700 1 -- 172.21.15.55:0/2400558312 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 45) v1 ==== 84973+0+0 (secure 0 0 0) 0x7f1c74013680 con 0x7f1c880cb8a0 2023-10-06T03:51:04.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.925+0000 7f1c84ff9700 1 --2- 172.21.15.55:0/2400558312 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c6c0671c0 0x7f1c6c069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:04.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.925+0000 7f1c84ff9700 1 -- 172.21.15.55:0/2400558312 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7f1c740851a0 con 0x7f1c880cb8a0 2023-10-06T03:51:04.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.926+0000 7f1c8cd9b700 1 --2- 172.21.15.55:0/2400558312 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c6c0671c0 0x7f1c6c069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:04.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.926+0000 7f1c8cd9b700 1 --2- 172.21.15.55:0/2400558312 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c6c0671c0 0x7f1c6c069670 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1c8806f9e0 tx=0x7f1c7c017000 comp rx=0 tx=0).ready entity=mgr.6422 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:04.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:04.930+0000 7f1c84ff9700 1 -- 172.21.15.55:0/2400558312 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c8814b9a0 con 0x7f1c880cb8a0 2023-10-06T03:51:05.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.160+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/2400558312 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1c8814b930 con 0x7f1c6c0671c0 2023-10-06T03:51:05.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.170+0000 7f1c84ff9700 1 -- 172.21.15.55:0/2400558312 <== mgr.6422 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f1c8814b930 con 0x7f1c6c0671c0 2023-10-06T03:51:05.172 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:05.172 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:50:03", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-06T03:51:05.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.172+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/2400558312 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c6c0671c0 msgr2=0x7f1c6c069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:05.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.172+0000 7f1c8dd9d700 1 --2- 172.21.15.55:0/2400558312 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c6c0671c0 0x7f1c6c069670 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1c8806f9e0 tx=0x7f1c7c017000 comp rx=0 tx=0).stop 2023-10-06T03:51:05.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.172+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/2400558312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c880cb8a0 msgr2=0x7f1c880b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:05.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.172+0000 7f1c8dd9d700 1 --2- 172.21.15.55:0/2400558312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c880cb8a0 0x7f1c880b10a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400ea70 tx=0x7f1c740091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.172+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/2400558312 shutdown_connections 2023-10-06T03:51:05.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.172+0000 7f1c8dd9d700 1 --2- 172.21.15.55:0/2400558312 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c6c0671c0 0x7f1c6c069670 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.172+0000 7f1c8dd9d700 1 --2- 172.21.15.55:0/2400558312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c88156320 0x7f1c880ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.172+0000 7f1c8dd9d700 1 --2- 172.21.15.55:0/2400558312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c880ca280 0x7f1c88149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.172+0000 7f1c8dd9d700 1 --2- 172.21.15.55:0/2400558312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c880cb8a0 0x7f1c880b10a0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.172+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/2400558312 >> 172.21.15.55:0/2400558312 conn(0x7f1c880c8be0 msgr2=0x7f1c8806e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:05.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.172+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/2400558312 shutdown_connections 2023-10-06T03:51:05.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.172+0000 7f1c8dd9d700 1 -- 172.21.15.55:0/2400558312 wait complete. 2023-10-06T03:51:05.192 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:50:03", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-06T03:51:05.192 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /bad-path 2023-10-06T03:51:05.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.382+0000 7f82b559d700 1 Processor -- start 2023-10-06T03:51:05.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.383+0000 7f82b559d700 1 -- start start 2023-10-06T03:51:05.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.384+0000 7f82b559d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82b00b0760 0x7f82b00b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:05.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.384+0000 7f82b559d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f82b00ace60 0x7f82b00ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:05.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.384+0000 7f82b559d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b00b1110 con 0x7f82b0156350 2023-10-06T03:51:05.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.384+0000 7f82b559d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b00b0570 con 0x7f82b00ace60 2023-10-06T03:51:05.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.384+0000 7f82b559d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b00af4d0 con 0x7f82b00b0760 2023-10-06T03:51:05.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.384+0000 7f82b4d9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f82b00ace60 0x7f82b00ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:05.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.384+0000 7f82af7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82b00b0760 0x7f82b00b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:05.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.384+0000 7f82af7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82b00b0760 0x7f82b00b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:52930/0 (socket says 172.21.15.55:52930) 2023-10-06T03:51:05.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.384+0000 7f82af7fe700 1 -- 172.21.15.55:0/3646585288 learned_addr learned my addr 172.21.15.55:0/3646585288 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:05.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.384+0000 7f82b4d9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f82b00ace60 0x7f82b00ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:59052/0 (socket says 172.21.15.55:59052) 2023-10-06T03:51:05.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.384+0000 7f82affff700 1 --1- 172.21.15.55:0/3646585288 >> v1:172.21.15.163:6789/0 conn(0x7f82b0156350 0x7f82b0157370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:58330/0 (socket says 172.21.15.55:58330) 2023-10-06T03:51:05.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.385+0000 7f82af7fe700 1 -- 172.21.15.55:0/3646585288 >> v2:172.21.15.163:3300/0 conn(0x7f82b00ace60 msgr2=0x7f82b00ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:05.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.385+0000 7f82af7fe700 1 --2- 172.21.15.55:0/3646585288 >> v2:172.21.15.163:3300/0 conn(0x7f82b00ace60 0x7f82b00ad2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.385+0000 7f82af7fe700 1 -- 172.21.15.55:0/3646585288 >> v1:172.21.15.163:6789/0 conn(0x7f82b0156350 legacy=0x7f82b0157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:05.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.385+0000 7f82af7fe700 1 -- 172.21.15.55:0/3646585288 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b00af7b0 con 0x7f82b00b0760 2023-10-06T03:51:05.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.385+0000 7f82b4d9c700 1 --2- 172.21.15.55:0/3646585288 >> v2:172.21.15.163:3300/0 conn(0x7f82b00ace60 0x7f82b00ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:05.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.385+0000 7f82af7fe700 1 --2- 172.21.15.55:0/3646585288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82b00b0760 0x7f82b00b0bd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f82a40099b0 tx=0x7f82a4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52608812b58a27f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:05.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.385+0000 7f82aeffd700 1 -- 172.21.15.55:0/3646585288 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4009ea0 con 0x7f82b00b0760 2023-10-06T03:51:05.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.386+0000 7f82aeffd700 1 -- 172.21.15.55:0/3646585288 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a400b860 con 0x7f82b00b0760 2023-10-06T03:51:05.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.386+0000 7f82aeffd700 1 -- 172.21.15.55:0/3646585288 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a400bc00 con 0x7f82b00b0760 2023-10-06T03:51:05.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.386+0000 7f82b559d700 1 -- 172.21.15.55:0/3646585288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82b00b0760 msgr2=0x7f82b00b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:05.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.386+0000 7f82b559d700 1 --2- 172.21.15.55:0/3646585288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82b00b0760 0x7f82b00b0bd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f82a40099b0 tx=0x7f82a4009250 comp rx=0 tx=0).stop 2023-10-06T03:51:05.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.386+0000 7f82b559d700 1 -- 172.21.15.55:0/3646585288 shutdown_connections 2023-10-06T03:51:05.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.386+0000 7f82b559d700 1 --2- 172.21.15.55:0/3646585288 >> v2:172.21.15.163:3300/0 conn(0x7f82b00ace60 0x7f82b00ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.386+0000 7f82b559d700 1 --2- 172.21.15.55:0/3646585288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82b00b0760 0x7f82b00b0bd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.386+0000 7f82b559d700 1 -- 172.21.15.55:0/3646585288 >> 172.21.15.55:0/3646585288 conn(0x7f82b0068e20 msgr2=0x7f82b00cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:05.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.386+0000 7f82b559d700 1 -- 172.21.15.55:0/3646585288 shutdown_connections 2023-10-06T03:51:05.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.386+0000 7f82b559d700 1 -- 172.21.15.55:0/3646585288 wait complete. 2023-10-06T03:51:05.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.388+0000 7f82b559d700 1 Processor -- start 2023-10-06T03:51:05.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.388+0000 7f82b559d700 1 -- start start 2023-10-06T03:51:05.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.389+0000 7f82b559d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82b00ace60 0x7f82b014f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:05.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.389+0000 7f82b559d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82b00b0760 0x7f82b00cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:05.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.389+0000 7f82b559d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b0156350 0x7f82b00cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:05.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.389+0000 7f82b559d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b00a5c30 con 0x7f82b0156350 2023-10-06T03:51:05.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.389+0000 7f82b559d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b00caa30 con 0x7f82b00ace60 2023-10-06T03:51:05.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.389+0000 7f82b559d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b00cad10 con 0x7f82b00b0760 2023-10-06T03:51:05.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.389+0000 7f82b4d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b0156350 0x7f82b00cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:05.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.389+0000 7f82b4d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b0156350 0x7f82b00cb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54176/0 (socket says 172.21.15.55:54176) 2023-10-06T03:51:05.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.389+0000 7f82b4d9c700 1 -- 172.21.15.55:0/3067651934 learned_addr learned my addr 172.21.15.55:0/3067651934 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:05.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.389+0000 7f82affff700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82b00ace60 0x7f82b014f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:05.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.389+0000 7f82af7fe700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82b00b0760 0x7f82b00cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:05.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.389+0000 7f82b4d9c700 1 -- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82b00b0760 msgr2=0x7f82b00cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:05.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.389+0000 7f82b4d9c700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82b00b0760 0x7f82b00cb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.389+0000 7f82b4d9c700 1 -- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82b00ace60 msgr2=0x7f82b014f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:05.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.390+0000 7f82b4d9c700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82b00ace60 0x7f82b014f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.390+0000 7f82b4d9c700 1 -- 172.21.15.55:0/3067651934 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a800ba40 con 0x7f82b0156350 2023-10-06T03:51:05.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.390+0000 7f82affff700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82b00ace60 0x7f82b014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:05.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.390+0000 7f82b4d9c700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b0156350 0x7f82b00cb5b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f82a800f4e0 tx=0x7f82a800d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:05.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.390+0000 7f8297fff700 1 -- 172.21.15.55:0/3067651934 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8010040 con 0x7f82b0156350 2023-10-06T03:51:05.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.390+0000 7f8297fff700 1 -- 172.21.15.55:0/3067651934 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a8007e30 con 0x7f82b0156350 2023-10-06T03:51:05.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.390+0000 7f82b559d700 1 -- 172.21.15.55:0/3067651934 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a40094f0 con 0x7f82b0156350 2023-10-06T03:51:05.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.390+0000 7f8297fff700 1 -- 172.21.15.55:0/3067651934 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800fe40 con 0x7f82b0156350 2023-10-06T03:51:05.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.390+0000 7f82b559d700 1 -- 172.21.15.55:0/3067651934 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b00a58c0 con 0x7f82b0156350 2023-10-06T03:51:05.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.391+0000 7f82b559d700 1 -- 172.21.15.55:0/3067651934 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b00a5da0 con 0x7f82b0156350 2023-10-06T03:51:05.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.392+0000 7f8297fff700 1 -- 172.21.15.55:0/3067651934 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 45) v1 ==== 84973+0+0 (secure 0 0 0) 0x7f82a8009760 con 0x7f82b0156350 2023-10-06T03:51:05.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.393+0000 7f8297fff700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f829c0671c0 0x7f829c069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:05.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.393+0000 7f8297fff700 1 -- 172.21.15.55:0/3067651934 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7f82a80857a0 con 0x7f82b0156350 2023-10-06T03:51:05.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.394+0000 7f82affff700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f829c0671c0 0x7f829c069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:05.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.394+0000 7f82affff700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f829c0671c0 0x7f829c069670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f82a000b240 tx=0x7f82a0001670 comp rx=0 tx=0).ready entity=mgr.6422 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:05.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.397+0000 7f8297fff700 1 -- 172.21.15.55:0/3067651934 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82a8052170 con 0x7f82b0156350 2023-10-06T03:51:05.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.626+0000 7f82b559d700 1 -- 172.21.15.55:0/3067651934 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7f82b004c3c0 con 0x7f829c0671c0 2023-10-06T03:51:05.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.699+0000 7f8297fff700 1 -- 172.21.15.55:0/3067651934 <== mgr.6422 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f82b004c3c0 con 0x7f829c0671c0 2023-10-06T03:51:05.701 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path /bad-path 2023-10-06T03:51:05.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.701+0000 7f82b559d700 1 -- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f829c0671c0 msgr2=0x7f829c069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:05.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.701+0000 7f82b559d700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f829c0671c0 0x7f829c069670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f82a000b240 tx=0x7f82a0001670 comp rx=0 tx=0).stop 2023-10-06T03:51:05.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.701+0000 7f82b559d700 1 -- 172.21.15.55:0/3067651934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b0156350 msgr2=0x7f82b00cb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:05.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.701+0000 7f82b559d700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b0156350 0x7f82b00cb5b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f82a800f4e0 tx=0x7f82a800d040 comp rx=0 tx=0).stop 2023-10-06T03:51:05.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.701+0000 7f82b559d700 1 -- 172.21.15.55:0/3067651934 shutdown_connections 2023-10-06T03:51:05.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.701+0000 7f82b559d700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f829c0671c0 0x7f829c069670 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.701+0000 7f82b559d700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82b0156350 0x7f82b00cb5b0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.701+0000 7f82b559d700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82b00ace60 0x7f82b014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.701+0000 7f82b559d700 1 --2- 172.21.15.55:0/3067651934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82b00b0760 0x7f82b00cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.701+0000 7f82b559d700 1 -- 172.21.15.55:0/3067651934 >> 172.21.15.55:0/3067651934 conn(0x7f82b0068e20 msgr2=0x7f82b0069610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:05.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.701+0000 7f82b559d700 1 -- 172.21.15.55:0/3067651934 shutdown_connections 2023-10-06T03:51:05.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.701+0000 7f82b559d700 1 -- 172.21.15.55:0/3067651934 wait complete. 2023-10-06T03:51:05.721 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-10-06T03:51:05.721 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T03:51:05.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.909+0000 7f991899d700 1 Processor -- start 2023-10-06T03:51:05.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.909+0000 7f991899d700 1 -- start start 2023-10-06T03:51:05.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.910+0000 7f991899d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99140cb1a0 0x7f991415a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:05.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.910+0000 7f991899d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f991415a8a0 0x7f991406f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:05.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.910+0000 7f991899d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f991406f5a0 0x7f99140ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:05.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.910+0000 7f991899d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991415b2e0 con 0x7f991415a8a0 2023-10-06T03:51:05.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.910+0000 7f991899d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991406ebd0 con 0x7f991406f5a0 2023-10-06T03:51:05.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.910+0000 7f991899d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991415ad20 con 0x7f99140cb1a0 2023-10-06T03:51:05.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.911+0000 7f9912ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f991415a8a0 0x7f991406f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:05.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.911+0000 7f9912ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f991415a8a0 0x7f991406f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54192/0 (socket says 172.21.15.55:54192) 2023-10-06T03:51:05.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.911+0000 7f99137fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99140cb1a0 0x7f991415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:05.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.911+0000 7f9912ffd700 1 -- 172.21.15.55:0/3271953175 learned_addr learned my addr 172.21.15.55:0/3271953175 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:05.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.911+0000 7f9913fff700 1 --2- 172.21.15.55:0/3271953175 >> v2:172.21.15.163:3300/0 conn(0x7f991406f5a0 0x7f99140ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:05.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.911+0000 7f9912ffd700 1 -- 172.21.15.55:0/3271953175 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99140cb1a0 msgr2=0x7f991415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:05.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.911+0000 7f9912ffd700 1 --2- 172.21.15.55:0/3271953175 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99140cb1a0 0x7f991415a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.911+0000 7f9912ffd700 1 -- 172.21.15.55:0/3271953175 >> v2:172.21.15.163:3300/0 conn(0x7f991406f5a0 msgr2=0x7f99140ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:05.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.911+0000 7f9912ffd700 1 --2- 172.21.15.55:0/3271953175 >> v2:172.21.15.163:3300/0 conn(0x7f991406f5a0 0x7f99140ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.911+0000 7f9912ffd700 1 -- 172.21.15.55:0/3271953175 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991415b000 con 0x7f991415a8a0 2023-10-06T03:51:05.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.912+0000 7f9912ffd700 1 --2- 172.21.15.55:0/3271953175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991415a8a0 0x7f991406f060 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f991406d580 tx=0x7f9908009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e319ba154fac4427 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:05.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.912+0000 7f99127fc700 1 -- 172.21.15.55:0/3271953175 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990800b6d0 con 0x7f991415a8a0 2023-10-06T03:51:05.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.912+0000 7f99127fc700 1 -- 172.21.15.55:0/3271953175 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990800b830 con 0x7f991415a8a0 2023-10-06T03:51:05.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.912+0000 7f99127fc700 1 -- 172.21.15.55:0/3271953175 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990800bbf0 con 0x7f991415a8a0 2023-10-06T03:51:05.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.912+0000 7f991899d700 1 -- 172.21.15.55:0/3271953175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991415a8a0 msgr2=0x7f991406f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:05.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.912+0000 7f991899d700 1 --2- 172.21.15.55:0/3271953175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991415a8a0 0x7f991406f060 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f991406d580 tx=0x7f9908009250 comp rx=0 tx=0).stop 2023-10-06T03:51:05.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.913+0000 7f991899d700 1 -- 172.21.15.55:0/3271953175 shutdown_connections 2023-10-06T03:51:05.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.913+0000 7f991899d700 1 --2- 172.21.15.55:0/3271953175 >> v2:172.21.15.163:3300/0 conn(0x7f991406f5a0 0x7f99140ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.913+0000 7f991899d700 1 --2- 172.21.15.55:0/3271953175 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f99140cb1a0 0x7f991415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.913+0000 7f991899d700 1 --2- 172.21.15.55:0/3271953175 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991415a8a0 0x7f991406f060 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.913+0000 7f991899d700 1 -- 172.21.15.55:0/3271953175 >> 172.21.15.55:0/3271953175 conn(0x7f99140cc1e0 msgr2=0x7f99140cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:05.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.913+0000 7f991899d700 1 -- 172.21.15.55:0/3271953175 shutdown_connections 2023-10-06T03:51:05.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.913+0000 7f991899d700 1 -- 172.21.15.55:0/3271953175 wait complete. 2023-10-06T03:51:05.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.914+0000 7f991899d700 1 Processor -- start 2023-10-06T03:51:05.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.914+0000 7f991899d700 1 -- start start 2023-10-06T03:51:05.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.915+0000 7f991899d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991406f5a0 0x7f991416af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:05.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.915+0000 7f991899d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99140cb1a0 0x7f9914159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:05.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.915+0000 7f991899d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f991414e400 0x7f9914159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:05.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.915+0000 7f991899d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991416b490 con 0x7f991406f5a0 2023-10-06T03:51:05.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.915+0000 7f991899d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991416aac0 con 0x7f99140cb1a0 2023-10-06T03:51:05.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.915+0000 7f991899d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991415a2c0 con 0x7f991414e400 2023-10-06T03:51:05.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.915+0000 7f99137fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991406f5a0 0x7f991416af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:05.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.915+0000 7f99137fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991406f5a0 0x7f991416af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54202/0 (socket says 172.21.15.55:54202) 2023-10-06T03:51:05.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.915+0000 7f9912ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99140cb1a0 0x7f9914159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:05.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.915+0000 7f99137fe700 1 -- 172.21.15.55:0/1580807760 learned_addr learned my addr 172.21.15.55:0/1580807760 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:05.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.915+0000 7f9913fff700 1 --2- 172.21.15.55:0/1580807760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f991414e400 0x7f9914159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:05.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.916+0000 7f99137fe700 1 -- 172.21.15.55:0/1580807760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f991414e400 msgr2=0x7f9914159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:05.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.916+0000 7f99137fe700 1 --2- 172.21.15.55:0/1580807760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f991414e400 0x7f9914159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.916+0000 7f99137fe700 1 -- 172.21.15.55:0/1580807760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99140cb1a0 msgr2=0x7f9914159720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:05.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.916+0000 7f99137fe700 1 --2- 172.21.15.55:0/1580807760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99140cb1a0 0x7f9914159720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:05.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.916+0000 7f99137fe700 1 -- 172.21.15.55:0/1580807760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99080094f0 con 0x7f991406f5a0 2023-10-06T03:51:05.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.916+0000 7f99137fe700 1 --2- 172.21.15.55:0/1580807760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991406f5a0 0x7f991416af50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f990400ec80 tx=0x7f99040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:05.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.916+0000 7f98fb7fe700 1 -- 172.21.15.55:0/1580807760 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400c7f0 con 0x7f991406f5a0 2023-10-06T03:51:05.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.916+0000 7f98fb7fe700 1 -- 172.21.15.55:0/1580807760 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990400c970 con 0x7f991406f5a0 2023-10-06T03:51:05.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.916+0000 7f98fb7fe700 1 -- 172.21.15.55:0/1580807760 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400cd30 con 0x7f991406f5a0 2023-10-06T03:51:05.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.916+0000 7f991899d700 1 -- 172.21.15.55:0/1580807760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991415a5a0 con 0x7f991406f5a0 2023-10-06T03:51:05.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.917+0000 7f991899d700 1 -- 172.21.15.55:0/1580807760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991416a420 con 0x7f991406f5a0 2023-10-06T03:51:05.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.918+0000 7f991899d700 1 -- 172.21.15.55:0/1580807760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991416a910 con 0x7f991406f5a0 2023-10-06T03:51:05.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.919+0000 7f98fb7fe700 1 -- 172.21.15.55:0/1580807760 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 45) v1 ==== 84973+0+0 (secure 0 0 0) 0x7f9904013680 con 0x7f991406f5a0 2023-10-06T03:51:05.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.919+0000 7f98fb7fe700 1 --2- 172.21.15.55:0/1580807760 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f99000671c0 0x7f9900069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:05.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.919+0000 7f98fb7fe700 1 -- 172.21.15.55:0/1580807760 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7f9904085050 con 0x7f991406f5a0 2023-10-06T03:51:05.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.920+0000 7f9912ffd700 1 --2- 172.21.15.55:0/1580807760 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f99000671c0 0x7f9900069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:05.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.921+0000 7f9912ffd700 1 --2- 172.21.15.55:0/1580807760 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f99000671c0 0x7f9900069670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f991406d580 tx=0x7f9908017000 comp rx=0 tx=0).ready entity=mgr.6422 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:05.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:05.924+0000 7f98fb7fe700 1 -- 172.21.15.55:0/1580807760 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9904051b00 con 0x7f991406f5a0 2023-10-06T03:51:06.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.183+0000 7f991899d700 1 -- 172.21.15.55:0/1580807760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f991406e860 con 0x7f991406f5a0 2023-10-06T03:51:06.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.891+0000 7f98fb7fe700 1 -- 172.21.15.55:0/1580807760 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 46) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f9904051ed0 con 0x7f991406f5a0 2023-10-06T03:51:06.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.892+0000 7f98fb7fe700 1 -- 172.21.15.55:0/1580807760 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v46) v1 ==== 93+0+0 (secure 0 0 0) 0x7f9904055590 con 0x7f991406f5a0 2023-10-06T03:51:06.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.895+0000 7f991899d700 1 -- 172.21.15.55:0/1580807760 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f99000671c0 msgr2=0x7f9900069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:06.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.895+0000 7f991899d700 1 --2- 172.21.15.55:0/1580807760 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f99000671c0 0x7f9900069670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f991406d580 tx=0x7f9908017000 comp rx=0 tx=0).stop 2023-10-06T03:51:06.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.895+0000 7f991899d700 1 -- 172.21.15.55:0/1580807760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991406f5a0 msgr2=0x7f991416af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:06.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.895+0000 7f991899d700 1 --2- 172.21.15.55:0/1580807760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991406f5a0 0x7f991416af50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f990400ec80 tx=0x7f99040091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:06.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.895+0000 7f991899d700 1 -- 172.21.15.55:0/1580807760 shutdown_connections 2023-10-06T03:51:06.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.895+0000 7f991899d700 1 --2- 172.21.15.55:0/1580807760 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f99000671c0 0x7f9900069670 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:06.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.895+0000 7f991899d700 1 --2- 172.21.15.55:0/1580807760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f991414e400 0x7f9914159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:06.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.895+0000 7f991899d700 1 --2- 172.21.15.55:0/1580807760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f991406f5a0 0x7f991416af50 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:06.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.895+0000 7f991899d700 1 --2- 172.21.15.55:0/1580807760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f99140cb1a0 0x7f9914159720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:06.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.895+0000 7f991899d700 1 -- 172.21.15.55:0/1580807760 >> 172.21.15.55:0/1580807760 conn(0x7f99140cc1e0 msgr2=0x7f99140aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:06.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.895+0000 7f991899d700 1 -- 172.21.15.55:0/1580807760 shutdown_connections 2023-10-06T03:51:06.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:06.895+0000 7f991899d700 1 -- 172.21.15.55:0/1580807760 wait complete. 2023-10-06T03:51:06.919 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:51:06.919 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:51:07.009 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:51:07.094 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:51:07.094 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:51:07.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:07.094 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:51:07.182 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T03:51:07.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:51:07.183 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:51:07.220 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:51:07.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.449+0000 7efda9f9d700 1 Processor -- start 2023-10-06T03:51:07.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.449+0000 7efda9f9d700 1 -- start start 2023-10-06T03:51:07.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.451+0000 7efda9f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efda41597f0 0x7efda40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:07.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.451+0000 7efda9f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efda415d230 0x7efda415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:07.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.451+0000 7efda9f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda415dbb0 con 0x7efda4159420 2023-10-06T03:51:07.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.451+0000 7efda9f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda40b11d0 con 0x7efda41597f0 2023-10-06T03:51:07.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.451+0000 7efda9f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda40b14b0 con 0x7efda415d230 2023-10-06T03:51:07.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.451+0000 7efda979c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efda415d230 0x7efda415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:07.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.451+0000 7efda8f9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7efda4159420 0x7efda40b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:58332/0 (socket says 172.21.15.55:58332) 2023-10-06T03:51:07.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.451+0000 7efda3fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efda41597f0 0x7efda40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:07.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.451+0000 7efda8f9b700 1 -- 172.21.15.55:0/1677298600 learned_addr learned my addr 172.21.15.55:0/1677298600 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:07.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.452+0000 7efda37fe700 1 -- 172.21.15.55:0/1677298600 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529525874 0 0) 0x7efda415dbb0 con 0x7efda4159420 2023-10-06T03:51:07.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.452+0000 7efda37fe700 1 -- 172.21.15.55:0/1677298600 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd94003570 con 0x7efda4159420 2023-10-06T03:51:07.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.452+0000 7efda979c700 1 -- 172.21.15.55:0/1677298600 >> v2:172.21.15.163:3300/0 conn(0x7efda41597f0 msgr2=0x7efda40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:07.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.452+0000 7efda979c700 1 --2- 172.21.15.55:0/1677298600 >> v2:172.21.15.163:3300/0 conn(0x7efda41597f0 0x7efda40b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:07.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.452+0000 7efda979c700 1 -- 172.21.15.55:0/1677298600 >> v1:172.21.15.163:6789/0 conn(0x7efda4159420 legacy=0x7efda40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:07.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.452+0000 7efda979c700 1 -- 172.21.15.55:0/1677298600 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda40b1790 con 0x7efda415d230 2023-10-06T03:51:07.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.452+0000 7efda979c700 1 --2- 172.21.15.55:0/1677298600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efda415d230 0x7efda415d670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009e60 tx=0x7efd9c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a36d161a132179 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:07.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.452+0000 7efda37fe700 1 -- 172.21.15.55:0/1677298600 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00e070 con 0x7efda415d230 2023-10-06T03:51:07.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.452+0000 7efda37fe700 1 -- 172.21.15.55:0/1677298600 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c00bab0 con 0x7efda415d230 2023-10-06T03:51:07.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.452+0000 7efda37fe700 1 -- 172.21.15.55:0/1677298600 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00be70 con 0x7efda415d230 2023-10-06T03:51:07.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.453+0000 7efda9f9d700 1 -- 172.21.15.55:0/1677298600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efda415d230 msgr2=0x7efda415d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:07.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.453+0000 7efda9f9d700 1 --2- 172.21.15.55:0/1677298600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efda415d230 0x7efda415d670 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009e60 tx=0x7efd9c009780 comp rx=0 tx=0).stop 2023-10-06T03:51:07.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.453+0000 7efda9f9d700 1 -- 172.21.15.55:0/1677298600 shutdown_connections 2023-10-06T03:51:07.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.453+0000 7efda9f9d700 1 --2- 172.21.15.55:0/1677298600 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efda415d230 0x7efda415d670 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:07.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.453+0000 7efda9f9d700 1 --2- 172.21.15.55:0/1677298600 >> v2:172.21.15.163:3300/0 conn(0x7efda41597f0 0x7efda40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:07.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.453+0000 7efda9f9d700 1 -- 172.21.15.55:0/1677298600 >> 172.21.15.55:0/1677298600 conn(0x7efda40d0660 msgr2=0x7efda40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:07.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.453+0000 7efda9f9d700 1 -- 172.21.15.55:0/1677298600 shutdown_connections 2023-10-06T03:51:07.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.453+0000 7efda9f9d700 1 -- 172.21.15.55:0/1677298600 wait complete. 2023-10-06T03:51:07.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.455+0000 7efda9f9d700 1 Processor -- start 2023-10-06T03:51:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.455+0000 7efda9f9d700 1 -- start start 2023-10-06T03:51:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.456+0000 7efda9f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efda4159420 0x7efda40c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.456+0000 7efda9f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efda41597f0 0x7efda4156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.456+0000 7efda9f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efda415d230 0x7efda415aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.456+0000 7efda9f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda415aa50 con 0x7efda415d230 2023-10-06T03:51:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.456+0000 7efda9f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda4159e40 con 0x7efda4159420 2023-10-06T03:51:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.456+0000 7efda9f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda415a120 con 0x7efda41597f0 2023-10-06T03:51:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.456+0000 7efda979c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efda415d230 0x7efda415aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.456+0000 7efda8f9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efda4159420 0x7efda40c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.456+0000 7efda3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efda41597f0 0x7efda4156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:07.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.456+0000 7efda979c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efda415d230 0x7efda415aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54234/0 (socket says 172.21.15.55:54234) 2023-10-06T03:51:07.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.456+0000 7efda8f9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efda4159420 0x7efda40c71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:59120/0 (socket says 172.21.15.55:59120) 2023-10-06T03:51:07.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.456+0000 7efda979c700 1 -- 172.21.15.55:0/1104289658 learned_addr learned my addr 172.21.15.55:0/1104289658 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:07.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.456+0000 7efda979c700 1 -- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efda41597f0 msgr2=0x7efda4156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:07.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.456+0000 7efda979c700 1 --2- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efda41597f0 0x7efda4156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:07.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.457+0000 7efda979c700 1 -- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efda4159420 msgr2=0x7efda40c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:07.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.457+0000 7efda979c700 1 --2- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efda4159420 0x7efda40c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:07.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.457+0000 7efda979c700 1 -- 172.21.15.55:0/1104289658 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c009a20 con 0x7efda415d230 2023-10-06T03:51:07.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.457+0000 7efda8f9b700 1 --2- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efda4159420 0x7efda40c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:07.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.457+0000 7efda979c700 1 --2- 172.21.15.55:0/1104289658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efda415d230 0x7efda415aee0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7efda415dd40 tx=0x7efd9c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:07.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.457+0000 7efda0ff9700 1 -- 172.21.15.55:0/1104289658 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c012e10 con 0x7efda415d230 2023-10-06T03:51:07.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.457+0000 7efda0ff9700 1 -- 172.21.15.55:0/1104289658 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c027470 con 0x7efda415d230 2023-10-06T03:51:07.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.457+0000 7efda0ff9700 1 -- 172.21.15.55:0/1104289658 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c027830 con 0x7efda415d230 2023-10-06T03:51:07.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.457+0000 7efda9f9d700 1 -- 172.21.15.55:0/1104289658 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda415a400 con 0x7efda415d230 2023-10-06T03:51:07.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.457+0000 7efda9f9d700 1 -- 172.21.15.55:0/1104289658 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda40c6640 con 0x7efda415d230 2023-10-06T03:51:07.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.458+0000 7efda9f9d700 1 -- 172.21.15.55:0/1104289658 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda415ac00 con 0x7efda415d230 2023-10-06T03:51:07.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.459+0000 7efda0ff9700 1 -- 172.21.15.55:0/1104289658 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 46) v1 ==== 84956+0+0 (secure 0 0 0) 0x7efd9c024070 con 0x7efda415d230 2023-10-06T03:51:07.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.460+0000 7efda0ff9700 1 --2- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efd88067170 0x7efd88069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:07.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.460+0000 7efda0ff9700 1 -- 172.21.15.55:0/1104289658 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7efd9c0876e0 con 0x7efda415d230 2023-10-06T03:51:07.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.460+0000 7efda8f9b700 1 -- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efd88067170 msgr2=0x7efd88069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:07.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.460+0000 7efda8f9b700 1 --2- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efd88067170 0x7efd88069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:07.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.464+0000 7efda0ff9700 1 -- 172.21.15.55:0/1104289658 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd9c054260 con 0x7efda415d230 2023-10-06T03:51:07.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.661+0000 7efda8f9b700 1 -- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efd88067170 msgr2=0x7efd88069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:07.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.661+0000 7efda8f9b700 1 --2- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efd88067170 0x7efd88069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:07.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.715+0000 7efda9f9d700 1 -- 172.21.15.55:0/1104289658 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efda404c3c0 con 0x7efda415d230 2023-10-06T03:51:07.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.716+0000 7efda0ff9700 1 -- 172.21.15.55:0/1104289658 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4605 (secure 0 0 0) 0x7efd9c057880 con 0x7efda415d230 2023-10-06T03:51:07.718 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:07.719 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2272331626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2272331626},{"type":"v1","addr":"172.21.15.163:6834","nonce":2272331626}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2110964237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2110964237},{"type":"v1","addr":"172.21.15.163:6836","nonce":2110964237}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2942349047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2942349047},{"type":"v1","addr":"172.21.15.55:6833","nonce":2942349047}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:49:47.265041+0000","modified":"2023-10-06T03:49:50.792326+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7254":{"gid":7254,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/2226955335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2226955335},{"type":"v1","addr":"172.21.15.55:6835","nonce":2226955335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T03:51:07.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.718+0000 7efda9f9d700 1 -- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efd88067170 msgr2=0x7efd88069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:07.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.718+0000 7efda9f9d700 1 --2- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efd88067170 0x7efd88069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:07.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.718+0000 7efda9f9d700 1 -- 172.21.15.55:0/1104289658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efda415d230 msgr2=0x7efda415aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:07.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.718+0000 7efda9f9d700 1 --2- 172.21.15.55:0/1104289658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efda415d230 0x7efda415aee0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7efda415dd40 tx=0x7efd9c019000 comp rx=0 tx=0).stop 2023-10-06T03:51:07.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.718+0000 7efda9f9d700 1 -- 172.21.15.55:0/1104289658 shutdown_connections 2023-10-06T03:51:07.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.718+0000 7efda9f9d700 1 --2- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efd88067170 0x7efd88069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:07.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.719+0000 7efda9f9d700 1 --2- 172.21.15.55:0/1104289658 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efda415d230 0x7efda415aee0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:07.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.719+0000 7efda9f9d700 1 --2- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efda4159420 0x7efda40c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:07.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.719+0000 7efda9f9d700 1 --2- 172.21.15.55:0/1104289658 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efda41597f0 0x7efda4156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:07.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.719+0000 7efda9f9d700 1 -- 172.21.15.55:0/1104289658 >> 172.21.15.55:0/1104289658 conn(0x7efda40d0660 msgr2=0x7efda40b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:07.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.719+0000 7efda9f9d700 1 -- 172.21.15.55:0/1104289658 shutdown_connections 2023-10-06T03:51:07.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.719+0000 7efda9f9d700 1 -- 172.21.15.55:0/1104289658 wait complete. 2023-10-06T03:51:07.723 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 53 2023-10-06T03:51:07.740 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:51:07.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.933+0000 7f7748b9d700 1 Processor -- start 2023-10-06T03:51:07.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.933+0000 7f7748b9d700 1 -- start start 2023-10-06T03:51:07.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.934+0000 7f7748b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7744156310 0x7f77440c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:07.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.934+0000 7f7748b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77440ca860 con 0x7f77441566e0 2023-10-06T03:51:07.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.934+0000 7f7748b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77440cab40 con 0x7f77440ae710 2023-10-06T03:51:07.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.934+0000 7f7748b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77440cae20 con 0x7f7744156310 2023-10-06T03:51:07.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.934+0000 7f77437fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7744156310 0x7f77440c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:07.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.934+0000 7f7742ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f77441566e0 0x7f77440ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51758/0 (socket says 172.21.15.55:51758) 2023-10-06T03:51:07.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.934+0000 7f7742ffd700 1 -- 172.21.15.55:0/3659662284 learned_addr learned my addr 172.21.15.55:0/3659662284 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:07.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.934+0000 7f77437fe700 1 --2- 172.21.15.55:0/3659662284 >> v2:172.21.15.55:3300/0 conn(0x7f7744156310 0x7f77440c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54248/0 (socket says 172.21.15.55:54248) 2023-10-06T03:51:07.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.935+0000 7f77437fe700 1 -- 172.21.15.55:0/3659662284 >> v1:172.21.15.163:6789/0 conn(0x7f77440ae710 legacy=0x7f77440c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:07.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.935+0000 7f77437fe700 1 -- 172.21.15.55:0/3659662284 >> v1:172.21.15.55:6789/0 conn(0x7f77441566e0 legacy=0x7f77440ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:07.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.935+0000 7f77427fc700 1 -- 172.21.15.55:0/3659662284 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524523810 0 0) 0x7f77440cab40 con 0x7f77440ae710 2023-10-06T03:51:07.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.935+0000 7f77437fe700 1 -- 172.21.15.55:0/3659662284 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77440cb100 con 0x7f7744156310 2023-10-06T03:51:07.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.935+0000 7f77437fe700 1 --2- 172.21.15.55:0/3659662284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7744156310 0x7f77440c9dc0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f77340098e0 tx=0x7f77340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92dbd889f35d9813 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:07.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.935+0000 7f77427fc700 1 -- 172.21.15.55:0/3659662284 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7734009e80 con 0x7f7744156310 2023-10-06T03:51:07.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.935+0000 7f77427fc700 1 -- 172.21.15.55:0/3659662284 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773400b760 con 0x7f7744156310 2023-10-06T03:51:07.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.935+0000 7f77427fc700 1 -- 172.21.15.55:0/3659662284 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773400bae0 con 0x7f7744156310 2023-10-06T03:51:07.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.936+0000 7f7748b9d700 1 -- 172.21.15.55:0/3659662284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7744156310 msgr2=0x7f77440c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:07.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.936+0000 7f7748b9d700 1 --2- 172.21.15.55:0/3659662284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7744156310 0x7f77440c9dc0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f77340098e0 tx=0x7f77340091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:07.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.936+0000 7f7748b9d700 1 -- 172.21.15.55:0/3659662284 shutdown_connections 2023-10-06T03:51:07.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.936+0000 7f7748b9d700 1 --2- 172.21.15.55:0/3659662284 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7744156310 0x7f77440c9dc0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:07.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.936+0000 7f7748b9d700 1 -- 172.21.15.55:0/3659662284 >> 172.21.15.55:0/3659662284 conn(0x7f77440cc1f0 msgr2=0x7f77440cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:07.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.936+0000 7f7748b9d700 1 -- 172.21.15.55:0/3659662284 shutdown_connections 2023-10-06T03:51:07.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.936+0000 7f7748b9d700 1 -- 172.21.15.55:0/3659662284 wait complete. 2023-10-06T03:51:07.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.938+0000 7f7748b9d700 1 Processor -- start 2023-10-06T03:51:07.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.938+0000 7f7748b9d700 1 -- start start 2023-10-06T03:51:07.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7748b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77440ae710 0x7f774415a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:07.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7748b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7744156310 0x7f7744150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:07.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7748b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77441566e0 0x7f77441514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:07.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7748b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7744152cb0 con 0x7f77441566e0 2023-10-06T03:51:07.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7748b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7744153250 con 0x7f77440ae710 2023-10-06T03:51:07.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7748b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774414b7e0 con 0x7f7744156310 2023-10-06T03:51:07.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7743fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77441566e0 0x7f77441514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:07.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7743fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77441566e0 0x7f77441514d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54252/0 (socket says 172.21.15.55:54252) 2023-10-06T03:51:07.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7743fff700 1 -- 172.21.15.55:0/1536783433 learned_addr learned my addr 172.21.15.55:0/1536783433 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:07.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f77437fe700 1 --2- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77440ae710 0x7f774415a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:07.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7742ffd700 1 --2- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7744156310 0x7f7744150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:07.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7743fff700 1 -- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7744156310 msgr2=0x7f7744150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:07.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7743fff700 1 --2- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7744156310 0x7f7744150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:07.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7743fff700 1 -- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77440ae710 msgr2=0x7f774415a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:07.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7743fff700 1 --2- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77440ae710 0x7f774415a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:07.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.939+0000 7f7743fff700 1 -- 172.21.15.55:0/1536783433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7734009420 con 0x7f77441566e0 2023-10-06T03:51:07.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.940+0000 7f7743fff700 1 --2- 172.21.15.55:0/1536783433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77441566e0 0x7f77441514d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f773c002230 tx=0x7f773c0016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:07.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.940+0000 7f772b7fe700 1 -- 172.21.15.55:0/1536783433 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c012630 con 0x7f77441566e0 2023-10-06T03:51:07.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.940+0000 7f772b7fe700 1 -- 172.21.15.55:0/1536783433 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773c0127b0 con 0x7f77441566e0 2023-10-06T03:51:07.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.940+0000 7f7748b9d700 1 -- 172.21.15.55:0/1536783433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774414b220 con 0x7f77441566e0 2023-10-06T03:51:07.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.940+0000 7f772b7fe700 1 -- 172.21.15.55:0/1536783433 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c012b70 con 0x7f77441566e0 2023-10-06T03:51:07.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.940+0000 7f7748b9d700 1 -- 172.21.15.55:0/1536783433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7744152ee0 con 0x7f77441566e0 2023-10-06T03:51:07.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.941+0000 7f7748b9d700 1 -- 172.21.15.55:0/1536783433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f774404c3c0 con 0x7f77441566e0 2023-10-06T03:51:07.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.942+0000 7f772b7fe700 1 -- 172.21.15.55:0/1536783433 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 46) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f773c003030 con 0x7f77441566e0 2023-10-06T03:51:07.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.943+0000 7f772b7fe700 1 --2- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7730067170 0x7f7730069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:07.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.943+0000 7f772b7fe700 1 -- 172.21.15.55:0/1536783433 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7f773c084c80 con 0x7f77441566e0 2023-10-06T03:51:07.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.943+0000 7f77437fe700 1 -- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7730067170 msgr2=0x7f7730069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:07.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.943+0000 7f77437fe700 1 --2- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7730067170 0x7f7730069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:07.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:07.947+0000 7f772b7fe700 1 -- 172.21.15.55:0/1536783433 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f773c051760 con 0x7f77441566e0 2023-10-06T03:51:08.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.144+0000 7f77437fe700 1 -- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7730067170 msgr2=0x7f7730069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:08.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.144+0000 7f77437fe700 1 --2- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7730067170 0x7f7730069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:08.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.153+0000 7f7748b9d700 1 -- 172.21.15.55:0/1536783433 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f774406ced0 con 0x7f77441566e0 2023-10-06T03:51:08.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.154+0000 7f772b7fe700 1 -- 172.21.15.55:0/1536783433 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+18088 (secure 0 0 0) 0x7f773c054d80 con 0x7f77441566e0 2023-10-06T03:51:08.156 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:08.157 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":110,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:51:04.865684+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:49:45.423436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T03:49:46.240142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.1","mappings":[{"from":1,"to":7}]},{"pgid":"14.2c","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]},{"pgid":"14.36","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/1033630463":"2023-10-07T03:50:02.911384+0000","172.21.15.163:0/347072733":"2023-10-07T03:50:02.911384+0000","172.21.15.163:6837/85191":"2023-10-07T03:50:02.911384+0000","172.21.15.163:6835/85191":"2023-10-07T03:50:02.911384+0000","172.21.15.163:0/1160979081":"2023-10-07T03:50:02.911384+0000","172.21.15.163:0/1786945833":"2023-10-07T03:50:02.911384+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:51:08.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.156+0000 7f7748b9d700 1 -- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7730067170 msgr2=0x7f7730069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:08.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.156+0000 7f7748b9d700 1 --2- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7730067170 0x7f7730069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.156+0000 7f7748b9d700 1 -- 172.21.15.55:0/1536783433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77441566e0 msgr2=0x7f77441514d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:08.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.156+0000 7f7748b9d700 1 --2- 172.21.15.55:0/1536783433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77441566e0 0x7f77441514d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f773c002230 tx=0x7f773c0016f0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.156+0000 7f7748b9d700 1 -- 172.21.15.55:0/1536783433 shutdown_connections 2023-10-06T03:51:08.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.156+0000 7f7748b9d700 1 --2- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7730067170 0x7f7730069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.156+0000 7f7748b9d700 1 --2- 172.21.15.55:0/1536783433 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f77441566e0 0x7f77441514d0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.156+0000 7f7748b9d700 1 --2- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f77440ae710 0x7f774415a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.156+0000 7f7748b9d700 1 --2- 172.21.15.55:0/1536783433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7744156310 0x7f7744150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.156+0000 7f7748b9d700 1 -- 172.21.15.55:0/1536783433 >> 172.21.15.55:0/1536783433 conn(0x7f77440cc1f0 msgr2=0x7f77440c2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:08.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.156+0000 7f7748b9d700 1 -- 172.21.15.55:0/1536783433 shutdown_connections 2023-10-06T03:51:08.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.156+0000 7f7748b9d700 1 -- 172.21.15.55:0/1536783433 wait complete. 2023-10-06T03:51:08.177 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:49:45.423436+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:49:46.240142+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:51:08.177 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:51:08.194 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:08.193+0000 7f42572c8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:51:08.267 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:08.264+0000 7f3612271180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:51:08.357 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:08.356+0000 7f42572c8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:51:08.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.368+0000 7f7be8d9d700 1 Processor -- start 2023-10-06T03:51:08.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.368+0000 7f7be8d9d700 1 -- start start 2023-10-06T03:51:08.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.369+0000 7f7be8d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7be40b4e70 0x7f7be40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:08.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.369+0000 7f7be8d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7be40b16d0 0x7f7be40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:08.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.369+0000 7f7be8d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be40b2080 con 0x7f7be415d240 2023-10-06T03:51:08.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.369+0000 7f7be8d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be40cfb00 con 0x7f7be40b4e70 2023-10-06T03:51:08.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.369+0000 7f7be8d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be40b5c40 con 0x7f7be40b16d0 2023-10-06T03:51:08.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.369+0000 7f7be37fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7be40b4e70 0x7f7be40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:08.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.369+0000 7f7be37fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7be40b4e70 0x7f7be40ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54256/0 (socket says 172.21.15.55:54256) 2023-10-06T03:51:08.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.369+0000 7f7be37fe700 1 -- 172.21.15.55:0/2003770635 learned_addr learned my addr 172.21.15.55:0/2003770635 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:08.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.370+0000 7f7be2ffd700 1 --2- 172.21.15.55:0/2003770635 >> v2:172.21.15.163:3300/0 conn(0x7f7be40b16d0 0x7f7be40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:08.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.370+0000 7f7be37fe700 1 -- 172.21.15.55:0/2003770635 >> v2:172.21.15.163:3300/0 conn(0x7f7be40b16d0 msgr2=0x7f7be40b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:08.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.370+0000 7f7be37fe700 1 --2- 172.21.15.55:0/2003770635 >> v2:172.21.15.163:3300/0 conn(0x7f7be40b16d0 0x7f7be40b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.370+0000 7f7be37fe700 1 -- 172.21.15.55:0/2003770635 >> v1:172.21.15.163:6789/0 conn(0x7f7be415d240 legacy=0x7f7be415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:08.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.370+0000 7f7be37fe700 1 -- 172.21.15.55:0/2003770635 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be40b5f20 con 0x7f7be40b4e70 2023-10-06T03:51:08.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.370+0000 7f7be27fc700 1 -- 172.21.15.55:0/2003770635 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504531596 0 0) 0x7f7be40b2080 con 0x7f7be415d240 2023-10-06T03:51:08.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.370+0000 7f7be37fe700 1 --2- 172.21.15.55:0/2003770635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7be40b4e70 0x7f7be40ccff0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7be4089830 tx=0x7f7bd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d372dff6ea2b9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:08.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.370+0000 7f7be27fc700 1 -- 172.21.15.55:0/2003770635 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4009e70 con 0x7f7be40b4e70 2023-10-06T03:51:08.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.371+0000 7f7be27fc700 1 -- 172.21.15.55:0/2003770635 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd400b860 con 0x7f7be40b4e70 2023-10-06T03:51:08.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.371+0000 7f7be27fc700 1 -- 172.21.15.55:0/2003770635 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400bc00 con 0x7f7be40b4e70 2023-10-06T03:51:08.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.371+0000 7f7be8d9d700 1 -- 172.21.15.55:0/2003770635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7be40b4e70 msgr2=0x7f7be40ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:08.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.371+0000 7f7be8d9d700 1 --2- 172.21.15.55:0/2003770635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7be40b4e70 0x7f7be40ccff0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7be4089830 tx=0x7f7bd4009250 comp rx=0 tx=0).stop 2023-10-06T03:51:08.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.371+0000 7f7be8d9d700 1 -- 172.21.15.55:0/2003770635 shutdown_connections 2023-10-06T03:51:08.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.371+0000 7f7be8d9d700 1 --2- 172.21.15.55:0/2003770635 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7be40b4e70 0x7f7be40ccff0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.371+0000 7f7be8d9d700 1 --2- 172.21.15.55:0/2003770635 >> v2:172.21.15.163:3300/0 conn(0x7f7be40b16d0 0x7f7be40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.371+0000 7f7be8d9d700 1 -- 172.21.15.55:0/2003770635 >> 172.21.15.55:0/2003770635 conn(0x7f7be40d10c0 msgr2=0x7f7be40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:08.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.371+0000 7f7be8d9d700 1 -- 172.21.15.55:0/2003770635 shutdown_connections 2023-10-06T03:51:08.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.371+0000 7f7be8d9d700 1 -- 172.21.15.55:0/2003770635 wait complete. 2023-10-06T03:51:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.373+0000 7f7be8d9d700 1 Processor -- start 2023-10-06T03:51:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.373+0000 7f7be8d9d700 1 -- start start 2023-10-06T03:51:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.374+0000 7f7be8d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be40b16d0 0x7f7be4145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.374+0000 7f7be8d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7be40b4e70 0x7f7be40c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.374+0000 7f7be8d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7be415d240 0x7f7be40c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.374+0000 7f7be8d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be41460d0 con 0x7f7be415d240 2023-10-06T03:51:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.374+0000 7f7be8d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be416af90 con 0x7f7be40b16d0 2023-10-06T03:51:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.374+0000 7f7be8d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be416b1e0 con 0x7f7be40b4e70 2023-10-06T03:51:08.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.374+0000 7f7be3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7be415d240 0x7f7be40c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:08.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.374+0000 7f7be37fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be40b16d0 0x7f7be4145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:08.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.374+0000 7f7be3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7be415d240 0x7f7be40c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54270/0 (socket says 172.21.15.55:54270) 2023-10-06T03:51:08.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.374+0000 7f7be3fff700 1 -- 172.21.15.55:0/3771278637 learned_addr learned my addr 172.21.15.55:0/3771278637 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:08.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.374+0000 7f7be2ffd700 1 --2- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7be40b4e70 0x7f7be40c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:08.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.375+0000 7f7be37fe700 1 -- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7be40b4e70 msgr2=0x7f7be40c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:08.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.375+0000 7f7be37fe700 1 --2- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7be40b4e70 0x7f7be40c65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.375+0000 7f7be37fe700 1 -- 172.21.15.55:0/3771278637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7be415d240 msgr2=0x7f7be40c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:08.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.375+0000 7f7be37fe700 1 --2- 172.21.15.55:0/3771278637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7be415d240 0x7f7be40c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.375+0000 7f7be37fe700 1 -- 172.21.15.55:0/3771278637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd40094f0 con 0x7f7be40b16d0 2023-10-06T03:51:08.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.375+0000 7f7be3fff700 1 --2- 172.21.15.55:0/3771278637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7be415d240 0x7f7be40c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:08.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.375+0000 7f7be37fe700 1 --2- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be40b16d0 0x7f7be4145b90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7be4071380 tx=0x7f7bd4017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:08.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.375+0000 7f7bcb7fe700 1 -- 172.21.15.55:0/3771278637 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd400bbe0 con 0x7f7be40b16d0 2023-10-06T03:51:08.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.375+0000 7f7bcb7fe700 1 -- 172.21.15.55:0/3771278637 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd4011420 con 0x7f7be40b16d0 2023-10-06T03:51:08.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.375+0000 7f7be8d9d700 1 -- 172.21.15.55:0/3771278637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be416b4c0 con 0x7f7be40b16d0 2023-10-06T03:51:08.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.375+0000 7f7bcb7fe700 1 -- 172.21.15.55:0/3771278637 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd40117c0 con 0x7f7be40b16d0 2023-10-06T03:51:08.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.375+0000 7f7be8d9d700 1 -- 172.21.15.55:0/3771278637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be416ba80 con 0x7f7be40b16d0 2023-10-06T03:51:08.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.376+0000 7f7be8d9d700 1 -- 172.21.15.55:0/3771278637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be4088990 con 0x7f7be40b16d0 2023-10-06T03:51:08.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.377+0000 7f7bcb7fe700 1 -- 172.21.15.55:0/3771278637 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 46) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f7bd4011b70 con 0x7f7be40b16d0 2023-10-06T03:51:08.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.378+0000 7f7bcb7fe700 1 --2- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7bd0067170 0x7f7bd0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:08.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.378+0000 7f7bcb7fe700 1 -- 172.21.15.55:0/3771278637 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7f7bd4085d90 con 0x7f7be40b16d0 2023-10-06T03:51:08.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.378+0000 7f7be2ffd700 1 -- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7bd0067170 msgr2=0x7f7bd0069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:08.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.378+0000 7f7be2ffd700 1 --2- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7bd0067170 0x7f7bd0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:08.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.383+0000 7f7bcb7fe700 1 -- 172.21.15.55:0/3771278637 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bd40528f0 con 0x7f7be40b16d0 2023-10-06T03:51:08.442 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:08.439+0000 7f3612271180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:51:08.445 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:08.444+0000 7f42572c8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:51:08.531 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:08.529+0000 7f3612271180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:51:08.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.580+0000 7f7be2ffd700 1 -- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7bd0067170 msgr2=0x7f7bd0069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:08.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.580+0000 7f7be2ffd700 1 --2- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7bd0067170 0x7f7bd0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:08.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.584+0000 7f7be8d9d700 1 -- 172.21.15.55:0/3771278637 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7be40af6e0 con 0x7f7be40b16d0 2023-10-06T03:51:08.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.585+0000 7f7bcb7fe700 1 -- 172.21.15.55:0/3771278637 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v110) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7bd4015020 con 0x7f7be40b16d0 2023-10-06T03:51:08.587 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:51:08.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.587+0000 7f7be8d9d700 1 -- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7bd0067170 msgr2=0x7f7bd0069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:08.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.587+0000 7f7be8d9d700 1 --2- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7bd0067170 0x7f7bd0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.587+0000 7f7be8d9d700 1 -- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be40b16d0 msgr2=0x7f7be4145b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.587+0000 7f7be8d9d700 1 --2- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be40b16d0 0x7f7be4145b90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7be4071380 tx=0x7f7bd4017000 comp rx=0 tx=0).stop 2023-10-06T03:51:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.587+0000 7f7be8d9d700 1 -- 172.21.15.55:0/3771278637 shutdown_connections 2023-10-06T03:51:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.587+0000 7f7be8d9d700 1 --2- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7bd0067170 0x7f7bd0069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.587+0000 7f7be8d9d700 1 --2- 172.21.15.55:0/3771278637 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7be415d240 0x7f7be40c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.587+0000 7f7be8d9d700 1 --2- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7be40b16d0 0x7f7be4145b90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.587+0000 7f7be8d9d700 1 --2- 172.21.15.55:0/3771278637 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7be40b4e70 0x7f7be40c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.587+0000 7f7be8d9d700 1 -- 172.21.15.55:0/3771278637 >> 172.21.15.55:0/3771278637 conn(0x7f7be40d10c0 msgr2=0x7f7be4069a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.587+0000 7f7be8d9d700 1 -- 172.21.15.55:0/3771278637 shutdown_connections 2023-10-06T03:51:08.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.588+0000 7f7be8d9d700 1 -- 172.21.15.55:0/3771278637 wait complete. 2023-10-06T03:51:08.609 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:51:08.670 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:08.669+0000 7f42572c8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:51:08.768 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:08.766+0000 7f3612271180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:51:08.801 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.798+0000 7f3099d9d700 1 Processor -- start 2023-10-06T03:51:08.801 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.799+0000 7f3099d9d700 1 -- start start 2023-10-06T03:51:08.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.800+0000 7f3099d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f30940b4e70 0x7f30940ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:08.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.800+0000 7f3099d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f30940b1b40 0x7f309415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:08.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.800+0000 7f3099d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30940cfb00 con 0x7f30940b16d0 2023-10-06T03:51:08.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.800+0000 7f3099d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30940b5c40 con 0x7f30940b4e70 2023-10-06T03:51:08.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.800+0000 7f3099d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30940b5f20 con 0x7f30940b1b40 2023-10-06T03:51:08.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.800+0000 7f3098d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f30940b4e70 0x7f30940ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:08.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.800+0000 7f3093fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f30940b16d0 0x7f309415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51760/0 (socket says 172.21.15.55:51760) 2023-10-06T03:51:08.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.800+0000 7f3093fff700 1 -- 172.21.15.55:0/81649614 learned_addr learned my addr 172.21.15.55:0/81649614 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:08.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.800+0000 7f309959c700 1 --2- 172.21.15.55:0/81649614 >> v2:172.21.15.163:3300/0 conn(0x7f30940b1b40 0x7f309415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:08.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.801+0000 7f30937fe700 1 -- 172.21.15.55:0/81649614 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718964095 0 0) 0x7f30940cfb00 con 0x7f30940b16d0 2023-10-06T03:51:08.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.801+0000 7f3098d9b700 1 -- 172.21.15.55:0/81649614 >> v2:172.21.15.163:3300/0 conn(0x7f30940b1b40 msgr2=0x7f309415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:08.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.801+0000 7f3098d9b700 1 --2- 172.21.15.55:0/81649614 >> v2:172.21.15.163:3300/0 conn(0x7f30940b1b40 0x7f309415d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.801+0000 7f3098d9b700 1 -- 172.21.15.55:0/81649614 >> v1:172.21.15.55:6789/0 conn(0x7f30940b16d0 legacy=0x7f309415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:08.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.801+0000 7f3098d9b700 1 -- 172.21.15.55:0/81649614 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30940b6200 con 0x7f30940b4e70 2023-10-06T03:51:08.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.801+0000 7f3098d9b700 1 --2- 172.21.15.55:0/81649614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30940b4e70 0x7f30940ccff0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3094071380 tx=0x7f30880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9766f2e972121ad7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:08.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.801+0000 7f30937fe700 1 -- 172.21.15.55:0/81649614 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088011630 con 0x7f30940b4e70 2023-10-06T03:51:08.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.801+0000 7f30937fe700 1 -- 172.21.15.55:0/81649614 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3088011790 con 0x7f30940b4e70 2023-10-06T03:51:08.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.801+0000 7f30937fe700 1 -- 172.21.15.55:0/81649614 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088011b30 con 0x7f30940b4e70 2023-10-06T03:51:08.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.802+0000 7f3099d9d700 1 -- 172.21.15.55:0/81649614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30940b4e70 msgr2=0x7f30940ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:08.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.802+0000 7f3099d9d700 1 --2- 172.21.15.55:0/81649614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30940b4e70 0x7f30940ccff0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3094071380 tx=0x7f30880091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.802+0000 7f3099d9d700 1 -- 172.21.15.55:0/81649614 shutdown_connections 2023-10-06T03:51:08.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.802+0000 7f3099d9d700 1 --2- 172.21.15.55:0/81649614 >> v2:172.21.15.163:3300/0 conn(0x7f30940b1b40 0x7f309415d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.802+0000 7f3099d9d700 1 --2- 172.21.15.55:0/81649614 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30940b4e70 0x7f30940ccff0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.802+0000 7f3099d9d700 1 -- 172.21.15.55:0/81649614 >> 172.21.15.55:0/81649614 conn(0x7f30940d10c0 msgr2=0x7f30940affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:08.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.802+0000 7f3099d9d700 1 -- 172.21.15.55:0/81649614 shutdown_connections 2023-10-06T03:51:08.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.802+0000 7f3099d9d700 1 -- 172.21.15.55:0/81649614 wait complete. 2023-10-06T03:51:08.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.804+0000 7f3099d9d700 1 Processor -- start 2023-10-06T03:51:08.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.804+0000 7f3099d9d700 1 -- start start 2023-10-06T03:51:08.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3099d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30940b16d0 0x7f309415d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:08.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3099d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30940b1b40 0x7f309415a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:08.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3099d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30940b4e70 0x7f309415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:08.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3099d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309415c850 con 0x7f30940b16d0 2023-10-06T03:51:08.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3099d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309415caf0 con 0x7f30940b1b40 2023-10-06T03:51:08.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3099d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309415cdd0 con 0x7f30940b4e70 2023-10-06T03:51:08.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3098d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30940b16d0 0x7f309415d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:08.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3098d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30940b16d0 0x7f309415d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54278/0 (socket says 172.21.15.55:54278) 2023-10-06T03:51:08.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3098d9b700 1 -- 172.21.15.55:0/772961329 learned_addr learned my addr 172.21.15.55:0/772961329 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:08.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3093fff700 1 --2- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30940b1b40 0x7f309415a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:08.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f309959c700 1 --2- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30940b4e70 0x7f309415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:08.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3098d9b700 1 -- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30940b4e70 msgr2=0x7f309415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:08.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3098d9b700 1 --2- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30940b4e70 0x7f309415b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3098d9b700 1 -- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30940b1b40 msgr2=0x7f309415a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:08.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3098d9b700 1 --2- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30940b1b40 0x7f309415a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:08.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.805+0000 7f3098d9b700 1 -- 172.21.15.55:0/772961329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3088009420 con 0x7f30940b16d0 2023-10-06T03:51:08.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.806+0000 7f3098d9b700 1 --2- 172.21.15.55:0/772961329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30940b16d0 0x7f309415d890 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3094070dd0 tx=0x7f3088017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:08.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.806+0000 7f3090ff9700 1 -- 172.21.15.55:0/772961329 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088000d10 con 0x7f30940b16d0 2023-10-06T03:51:08.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.806+0000 7f3090ff9700 1 -- 172.21.15.55:0/772961329 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3088011b10 con 0x7f30940b16d0 2023-10-06T03:51:08.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.806+0000 7f3090ff9700 1 -- 172.21.15.55:0/772961329 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088023660 con 0x7f30940b16d0 2023-10-06T03:51:08.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.806+0000 7f3099d9d700 1 -- 172.21.15.55:0/772961329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309415d0b0 con 0x7f30940b16d0 2023-10-06T03:51:08.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.806+0000 7f3099d9d700 1 -- 172.21.15.55:0/772961329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309415ad10 con 0x7f30940b16d0 2023-10-06T03:51:08.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.808+0000 7f307e7fc700 1 -- 172.21.15.55:0/772961329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3094088990 con 0x7f30940b16d0 2023-10-06T03:51:08.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.809+0000 7f3090ff9700 1 -- 172.21.15.55:0/772961329 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 46) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f30880193b0 con 0x7f30940b16d0 2023-10-06T03:51:08.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.809+0000 7f3090ff9700 1 --2- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3078067170 0x7f3078069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:08.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.809+0000 7f3090ff9700 1 -- 172.21.15.55:0/772961329 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7f3088086140 con 0x7f30940b16d0 2023-10-06T03:51:08.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.810+0000 7f3093fff700 1 -- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3078067170 msgr2=0x7f3078069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:08.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.810+0000 7f3093fff700 1 --2- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3078067170 0x7f3078069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:08.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:08.814+0000 7f3090ff9700 1 -- 172.21.15.55:0/772961329 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3088052ca0 con 0x7f30940b16d0 2023-10-06T03:51:08.847 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:08.846+0000 7f42572c8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:51:08.942 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:08.941+0000 7f42572c8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:51:08.958 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:08.955+0000 7f3612271180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:51:09.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.011+0000 7f3093fff700 1 -- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3078067170 msgr2=0x7f3078069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:09.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.011+0000 7f3093fff700 1 --2- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3078067170 0x7f3078069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:09.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.021+0000 7f307e7fc700 1 -- 172.21.15.55:0/772961329 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f309404bb80 con 0x7f30940b16d0 2023-10-06T03:51:09.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.021+0000 7f3090ff9700 1 -- 172.21.15.55:0/772961329 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3088015020 con 0x7f30940b16d0 2023-10-06T03:51:09.024 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:51:09.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.025+0000 7f307e7fc700 1 -- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3078067170 msgr2=0x7f3078069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:09.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.025+0000 7f307e7fc700 1 --2- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3078067170 0x7f3078069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.025+0000 7f307e7fc700 1 -- 172.21.15.55:0/772961329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30940b16d0 msgr2=0x7f309415d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:09.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.025+0000 7f307e7fc700 1 --2- 172.21.15.55:0/772961329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30940b16d0 0x7f309415d890 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3094070dd0 tx=0x7f3088017000 comp rx=0 tx=0).stop 2023-10-06T03:51:09.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.025+0000 7f307e7fc700 1 -- 172.21.15.55:0/772961329 shutdown_connections 2023-10-06T03:51:09.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.025+0000 7f307e7fc700 1 --2- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3078067170 0x7f3078069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.025+0000 7f307e7fc700 1 --2- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30940b4e70 0x7f309415b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.025+0000 7f307e7fc700 1 --2- 172.21.15.55:0/772961329 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30940b16d0 0x7f309415d890 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.025+0000 7f307e7fc700 1 --2- 172.21.15.55:0/772961329 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30940b1b40 0x7f309415a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.025+0000 7f307e7fc700 1 -- 172.21.15.55:0/772961329 >> 172.21.15.55:0/772961329 conn(0x7f30940d10c0 msgr2=0x7f30940b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:09.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.025+0000 7f307e7fc700 1 -- 172.21.15.55:0/772961329 shutdown_connections 2023-10-06T03:51:09.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.025+0000 7f307e7fc700 1 -- 172.21.15.55:0/772961329 wait complete. 2023-10-06T03:51:09.045 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:51:09.057 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:09.055+0000 7f3612271180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:51:09.104 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:09.103+0000 7f42572c8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:51:09.205 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:09.204+0000 7f42572c8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:51:09.226 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:09.224+0000 7f3612271180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:51:09.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.238+0000 7fc37199d700 1 Processor -- start 2023-10-06T03:51:09.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.238+0000 7fc37199d700 1 -- start start 2023-10-06T03:51:09.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.239+0000 7fc37199d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc36c0b16d0 0x7fc36c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:09.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.239+0000 7fc37199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc36c0b33b0 0x7fc36c0b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:09.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.239+0000 7fc37199d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36c0b2080 con 0x7fc36c0b4e70 2023-10-06T03:51:09.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.239+0000 7fc37199d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36c0cfb00 con 0x7fc36c0b16d0 2023-10-06T03:51:09.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.239+0000 7fc37199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c0b5c40 con 0x7fc36c0b33b0 2023-10-06T03:51:09.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.240+0000 7fc37099b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc36c0b4e70 0x7fc36c0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51762/0 (socket says 172.21.15.55:51762) 2023-10-06T03:51:09.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.240+0000 7fc37099b700 1 -- 172.21.15.55:0/203995641 learned_addr learned my addr 172.21.15.55:0/203995641 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:09.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.240+0000 7fc36bfff700 1 --2- 172.21.15.55:0/203995641 >> v2:172.21.15.163:3300/0 conn(0x7fc36c0b16d0 0x7fc36c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:09.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.240+0000 7fc37119c700 1 --2- 172.21.15.55:0/203995641 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc36c0b33b0 0x7fc36c0b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:09.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.240+0000 7fc36b7fe700 1 -- 172.21.15.55:0/203995641 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820692911 0 0) 0x7fc36c0b2080 con 0x7fc36c0b4e70 2023-10-06T03:51:09.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.240+0000 7fc36b7fe700 1 -- 172.21.15.55:0/203995641 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc35c003570 con 0x7fc36c0b4e70 2023-10-06T03:51:09.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.240+0000 7fc36bfff700 1 -- 172.21.15.55:0/203995641 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc36c0b33b0 msgr2=0x7fc36c0b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:09.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.240+0000 7fc36bfff700 1 --2- 172.21.15.55:0/203995641 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc36c0b33b0 0x7fc36c0b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.240+0000 7fc36bfff700 1 -- 172.21.15.55:0/203995641 >> v1:172.21.15.55:6789/0 conn(0x7fc36c0b4e70 legacy=0x7fc36c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.240+0000 7fc36bfff700 1 -- 172.21.15.55:0/203995641 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c0b5f20 con 0x7fc36c0b16d0 2023-10-06T03:51:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.240+0000 7fc37119c700 1 --2- 172.21.15.55:0/203995641 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc36c0b33b0 0x7fc36c0b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.241+0000 7fc36bfff700 1 --2- 172.21.15.55:0/203995641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc36c0b16d0 0x7fc36c0b1b40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc3580098e0 tx=0x7fc3580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24df0185c63e6a35 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.241+0000 7fc36b7fe700 1 -- 172.21.15.55:0/203995641 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358011630 con 0x7fc36c0b16d0 2023-10-06T03:51:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.241+0000 7fc36b7fe700 1 -- 172.21.15.55:0/203995641 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc358011790 con 0x7fc36c0b16d0 2023-10-06T03:51:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.241+0000 7fc36b7fe700 1 -- 172.21.15.55:0/203995641 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc358011b10 con 0x7fc36c0b16d0 2023-10-06T03:51:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.241+0000 7fc37199d700 1 -- 172.21.15.55:0/203995641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc36c0b16d0 msgr2=0x7fc36c0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.241+0000 7fc37199d700 1 --2- 172.21.15.55:0/203995641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc36c0b16d0 0x7fc36c0b1b40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc3580098e0 tx=0x7fc3580091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.242+0000 7fc37199d700 1 -- 172.21.15.55:0/203995641 shutdown_connections 2023-10-06T03:51:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.242+0000 7fc37199d700 1 --2- 172.21.15.55:0/203995641 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc36c0b33b0 0x7fc36c0b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.242+0000 7fc37199d700 1 --2- 172.21.15.55:0/203995641 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc36c0b16d0 0x7fc36c0b1b40 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.242+0000 7fc37199d700 1 -- 172.21.15.55:0/203995641 >> 172.21.15.55:0/203995641 conn(0x7fc36c0d10c0 msgr2=0x7fc36c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:09.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.242+0000 7fc37199d700 1 -- 172.21.15.55:0/203995641 shutdown_connections 2023-10-06T03:51:09.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.242+0000 7fc37199d700 1 -- 172.21.15.55:0/203995641 wait complete. 2023-10-06T03:51:09.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.243+0000 7fc37199d700 1 Processor -- start 2023-10-06T03:51:09.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.243+0000 7fc37199d700 1 -- start start 2023-10-06T03:51:09.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.244+0000 7fc37199d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc36c0b16d0 0x7fc36c0c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:09.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.244+0000 7fc37199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc36c0b33b0 0x7fc36c0ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:09.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.244+0000 7fc37199d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc36c0b4e70 0x7fc36c0cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:09.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.244+0000 7fc37199d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c0c8a60 con 0x7fc36c0b16d0 2023-10-06T03:51:09.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.244+0000 7fc37199d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c0c7e50 con 0x7fc36c0b4e70 2023-10-06T03:51:09.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.244+0000 7fc37199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c0c8130 con 0x7fc36c0b33b0 2023-10-06T03:51:09.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.245+0000 7fc37099b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc36c0b16d0 0x7fc36c0c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:09.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.245+0000 7fc37099b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc36c0b16d0 0x7fc36c0c74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54294/0 (socket says 172.21.15.55:54294) 2023-10-06T03:51:09.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.245+0000 7fc37099b700 1 -- 172.21.15.55:0/3147594318 learned_addr learned my addr 172.21.15.55:0/3147594318 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:09.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.245+0000 7fc36bfff700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc36c0b33b0 0x7fc36c0ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:09.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.245+0000 7fc37119c700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc36c0b4e70 0x7fc36c0cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:09.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.245+0000 7fc37099b700 1 -- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc36c0b33b0 msgr2=0x7fc36c0ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:09.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.245+0000 7fc37099b700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc36c0b33b0 0x7fc36c0ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.245+0000 7fc37099b700 1 -- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc36c0b4e70 msgr2=0x7fc36c0cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:09.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.245+0000 7fc37099b700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc36c0b4e70 0x7fc36c0cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.245+0000 7fc37099b700 1 -- 172.21.15.55:0/3147594318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc364009a20 con 0x7fc36c0b16d0 2023-10-06T03:51:09.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.245+0000 7fc36bfff700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc36c0b33b0 0x7fc36c0ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:09.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.245+0000 7fc37099b700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc36c0b16d0 0x7fc36c0c74e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fc3600011e0 tx=0x7fc360007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:09.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.246+0000 7fc368ff9700 1 -- 172.21.15.55:0/3147594318 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360005b30 con 0x7fc36c0b16d0 2023-10-06T03:51:09.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.246+0000 7fc368ff9700 1 -- 172.21.15.55:0/3147594318 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc360005c90 con 0x7fc36c0b16d0 2023-10-06T03:51:09.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.246+0000 7fc368ff9700 1 -- 172.21.15.55:0/3147594318 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc360007320 con 0x7fc36c0b16d0 2023-10-06T03:51:09.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.246+0000 7fc37199d700 1 -- 172.21.15.55:0/3147594318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc358009420 con 0x7fc36c0b16d0 2023-10-06T03:51:09.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.246+0000 7fc37199d700 1 -- 172.21.15.55:0/3147594318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c0c86f0 con 0x7fc36c0b16d0 2023-10-06T03:51:09.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.247+0000 7fc37199d700 1 -- 172.21.15.55:0/3147594318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36c088990 con 0x7fc36c0b16d0 2023-10-06T03:51:09.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.249+0000 7fc368ff9700 1 -- 172.21.15.55:0/3147594318 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 46) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fc360007690 con 0x7fc36c0b16d0 2023-10-06T03:51:09.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.250+0000 7fc368ff9700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc350067170 0x7fc350069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:09.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.250+0000 7fc368ff9700 1 -- 172.21.15.55:0/3147594318 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7fc360084200 con 0x7fc36c0b16d0 2023-10-06T03:51:09.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.250+0000 7fc36bfff700 1 -- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc350067170 msgr2=0x7fc350069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:09.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.250+0000 7fc36bfff700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc350067170 0x7fc350069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:09.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.254+0000 7fc368ff9700 1 -- 172.21.15.55:0/3147594318 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc360050d70 con 0x7fc36c0b16d0 2023-10-06T03:51:09.337 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:09.335+0000 7f3612271180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:51:09.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.451+0000 7fc36bfff700 1 -- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc350067170 msgr2=0x7fc350069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:09.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.451+0000 7fc36bfff700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc350067170 0x7fc350069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:09.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.455+0000 7fc37199d700 1 -- 172.21.15.55:0/3147594318 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc36c04bb80 con 0x7fc36c0b16d0 2023-10-06T03:51:09.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.456+0000 7fc368ff9700 1 -- 172.21.15.55:0/3147594318 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc360054390 con 0x7fc36c0b16d0 2023-10-06T03:51:09.458 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T03:51:09.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.458+0000 7fc37199d700 1 -- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc350067170 msgr2=0x7fc350069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:09.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.458+0000 7fc37199d700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc350067170 0x7fc350069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.458+0000 7fc37199d700 1 -- 172.21.15.55:0/3147594318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc36c0b16d0 msgr2=0x7fc36c0c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:09.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.458+0000 7fc37199d700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc36c0b16d0 0x7fc36c0c74e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fc3600011e0 tx=0x7fc360007df0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.458+0000 7fc37199d700 1 -- 172.21.15.55:0/3147594318 shutdown_connections 2023-10-06T03:51:09.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.458+0000 7fc37199d700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc350067170 0x7fc350069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.458+0000 7fc37199d700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc36c0b4e70 0x7fc36c0cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.458+0000 7fc37199d700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc36c0b16d0 0x7fc36c0c74e0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.458+0000 7fc37199d700 1 --2- 172.21.15.55:0/3147594318 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc36c0b33b0 0x7fc36c0ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.458+0000 7fc37199d700 1 -- 172.21.15.55:0/3147594318 >> 172.21.15.55:0/3147594318 conn(0x7fc36c0d10c0 msgr2=0x7fc36c069b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:09.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.458+0000 7fc37199d700 1 -- 172.21.15.55:0/3147594318 shutdown_connections 2023-10-06T03:51:09.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.458+0000 7fc37199d700 1 -- 172.21.15.55:0/3147594318 wait complete. 2023-10-06T03:51:09.479 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:51:09.694 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:09.693+0000 7f42572c8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:51:09.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.717+0000 7f09609fe700 1 Processor -- start 2023-10-06T03:51:09.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.717+0000 7f09609fe700 1 -- start start 2023-10-06T03:51:09.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.718+0000 7f09609fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f095c0b4e70 0x7f095c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:09.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.718+0000 7f09609fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f095c0b16d0 0x7f095c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:09.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.718+0000 7f09609fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f095c15d240 0x7f095c15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:09.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.718+0000 7f09609fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095c0b2080 con 0x7f095c15d240 2023-10-06T03:51:09.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.718+0000 7f09609fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095c0cfb00 con 0x7f095c0b4e70 2023-10-06T03:51:09.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.718+0000 7f09609fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095c0b5c40 con 0x7f095c0b16d0 2023-10-06T03:51:09.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.719+0000 7f095b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f095c15d240 0x7f095c15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:09.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.719+0000 7f095b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f095c15d240 0x7f095c15d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54296/0 (socket says 172.21.15.55:54296) 2023-10-06T03:51:09.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.719+0000 7f095b59e700 1 -- 172.21.15.55:0/1265345102 learned_addr learned my addr 172.21.15.55:0/1265345102 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:09.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.719+0000 7f095ad9d700 1 --2- 172.21.15.55:0/1265345102 >> v2:172.21.15.163:3300/0 conn(0x7f095c0b4e70 0x7f095c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:09.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.719+0000 7f095a59c700 1 --2- 172.21.15.55:0/1265345102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f095c0b16d0 0x7f095c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:09.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.719+0000 7f095b59e700 1 -- 172.21.15.55:0/1265345102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f095c0b16d0 msgr2=0x7f095c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:09.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.719+0000 7f095b59e700 1 --2- 172.21.15.55:0/1265345102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f095c0b16d0 0x7f095c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.719+0000 7f095b59e700 1 -- 172.21.15.55:0/1265345102 >> v2:172.21.15.163:3300/0 conn(0x7f095c0b4e70 msgr2=0x7f095c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:09.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.719+0000 7f095b59e700 1 --2- 172.21.15.55:0/1265345102 >> v2:172.21.15.163:3300/0 conn(0x7f095c0b4e70 0x7f095c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.719+0000 7f095b59e700 1 -- 172.21.15.55:0/1265345102 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c0b5f20 con 0x7f095c15d240 2023-10-06T03:51:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.719+0000 7f095b59e700 1 --2- 172.21.15.55:0/1265345102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f095c15d240 0x7f095c15d6c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f095c064910 tx=0x7f0954009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114b1c1b6f1885d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.720+0000 7f0959d9b700 1 -- 172.21.15.55:0/1265345102 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095400e070 con 0x7f095c15d240 2023-10-06T03:51:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.720+0000 7f0959d9b700 1 -- 172.21.15.55:0/1265345102 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095400b970 con 0x7f095c15d240 2023-10-06T03:51:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.720+0000 7f0959d9b700 1 -- 172.21.15.55:0/1265345102 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09540092e0 con 0x7f095c15d240 2023-10-06T03:51:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.720+0000 7f09609fe700 1 -- 172.21.15.55:0/1265345102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f095c15d240 msgr2=0x7f095c15d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.720+0000 7f09609fe700 1 --2- 172.21.15.55:0/1265345102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f095c15d240 0x7f095c15d6c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f095c064910 tx=0x7f0954009840 comp rx=0 tx=0).stop 2023-10-06T03:51:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.720+0000 7f09609fe700 1 -- 172.21.15.55:0/1265345102 shutdown_connections 2023-10-06T03:51:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.720+0000 7f09609fe700 1 --2- 172.21.15.55:0/1265345102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f095c15d240 0x7f095c15d6c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.720+0000 7f09609fe700 1 --2- 172.21.15.55:0/1265345102 >> v2:172.21.15.163:3300/0 conn(0x7f095c0b4e70 0x7f095c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.720+0000 7f09609fe700 1 --2- 172.21.15.55:0/1265345102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f095c0b16d0 0x7f095c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.720+0000 7f09609fe700 1 -- 172.21.15.55:0/1265345102 >> 172.21.15.55:0/1265345102 conn(0x7f095c0d10c0 msgr2=0x7f095c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:09.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.720+0000 7f09609fe700 1 -- 172.21.15.55:0/1265345102 shutdown_connections 2023-10-06T03:51:09.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.720+0000 7f09609fe700 1 -- 172.21.15.55:0/1265345102 wait complete. 2023-10-06T03:51:09.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.722+0000 7f09609fe700 1 Processor -- start 2023-10-06T03:51:09.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.722+0000 7f09609fe700 1 -- start start 2023-10-06T03:51:09.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.723+0000 7f09609fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f095c0b16d0 0x7f095c158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:09.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.723+0000 7f09609fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f095c0b4e70 0x7f095c159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:09.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.723+0000 7f09609fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f095c15d240 0x7f095c15e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:09.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.723+0000 7f09609fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c153230 con 0x7f095c15d240 2023-10-06T03:51:09.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.723+0000 7f09609fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c153510 con 0x7f095c0b4e70 2023-10-06T03:51:09.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.723+0000 7f09609fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095c1537f0 con 0x7f095c0b16d0 2023-10-06T03:51:09.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.723+0000 7f095b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f095c15d240 0x7f095c15e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:09.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.723+0000 7f095b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f095c15d240 0x7f095c15e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54298/0 (socket says 172.21.15.55:54298) 2023-10-06T03:51:09.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.723+0000 7f095ad9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f095c0b16d0 0x7f095c158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:09.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.723+0000 7f095b59e700 1 -- 172.21.15.55:0/2228845587 learned_addr learned my addr 172.21.15.55:0/2228845587 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:09.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.723+0000 7f095a59c700 1 --2- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f095c0b4e70 0x7f095c159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:09.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.724+0000 7f095b59e700 1 -- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f095c0b16d0 msgr2=0x7f095c158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:09.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.724+0000 7f095b59e700 1 --2- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f095c0b16d0 0x7f095c158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.724+0000 7f095b59e700 1 -- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f095c0b4e70 msgr2=0x7f095c159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:09.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.724+0000 7f095b59e700 1 --2- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f095c0b4e70 0x7f095c159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.724+0000 7f095b59e700 1 -- 172.21.15.55:0/2228845587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0954009ae0 con 0x7f095c15d240 2023-10-06T03:51:09.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.724+0000 7f095ad9d700 1 --2- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f095c0b16d0 0x7f095c158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:09.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.724+0000 7f095b59e700 1 --2- 172.21.15.55:0/2228845587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f095c15d240 0x7f095c15e680 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f095c070dd0 tx=0x7f0954018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:09.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.724+0000 7f0942ffd700 1 -- 172.21.15.55:0/2228845587 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0954008ad0 con 0x7f095c15d240 2023-10-06T03:51:09.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.724+0000 7f0942ffd700 1 -- 172.21.15.55:0/2228845587 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0954008c50 con 0x7f095c15d240 2023-10-06T03:51:09.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.724+0000 7f09609fe700 1 -- 172.21.15.55:0/2228845587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095c153ad0 con 0x7f095c15d240 2023-10-06T03:51:09.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.724+0000 7f0942ffd700 1 -- 172.21.15.55:0/2228845587 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0954025460 con 0x7f095c15d240 2023-10-06T03:51:09.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.724+0000 7f09609fe700 1 -- 172.21.15.55:0/2228845587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095c15ed80 con 0x7f095c15d240 2023-10-06T03:51:09.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.725+0000 7f09609fe700 1 -- 172.21.15.55:0/2228845587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095c154020 con 0x7f095c15d240 2023-10-06T03:51:09.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.726+0000 7f0942ffd700 1 -- 172.21.15.55:0/2228845587 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 46) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f0954003040 con 0x7f095c15d240 2023-10-06T03:51:09.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.727+0000 7f0942ffd700 1 --2- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0948067170 0x7f0948069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:09.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.727+0000 7f0942ffd700 1 -- 172.21.15.55:0/2228845587 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7f0954086bb0 con 0x7f095c15d240 2023-10-06T03:51:09.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.727+0000 7f095ad9d700 1 -- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0948067170 msgr2=0x7f0948069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:09.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.727+0000 7f095ad9d700 1 --2- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0948067170 0x7f0948069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:09.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.732+0000 7f0942ffd700 1 -- 172.21.15.55:0/2228845587 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0954053630 con 0x7f095c15d240 2023-10-06T03:51:09.790 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:09.789+0000 7f42572c8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:51:09.855 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:09.853+0000 7f3612271180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:51:09.880 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:09.879+0000 7f42572c8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:51:09.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.928+0000 7f095ad9d700 1 -- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0948067170 msgr2=0x7f0948069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:09.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.928+0000 7f095ad9d700 1 --2- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0948067170 0x7f0948069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:09.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.935+0000 7f09609fe700 1 -- 172.21.15.55:0/2228845587 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f095c04bb80 con 0x7f095c15d240 2023-10-06T03:51:09.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.936+0000 7f0942ffd700 1 -- 172.21.15.55:0/2228845587 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0954056c50 con 0x7f095c15d240 2023-10-06T03:51:09.938 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T03:51:09.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.939+0000 7f09609fe700 1 -- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0948067170 msgr2=0x7f0948069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:09.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.939+0000 7f09609fe700 1 --2- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0948067170 0x7f0948069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.939+0000 7f09609fe700 1 -- 172.21.15.55:0/2228845587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f095c15d240 msgr2=0x7f095c15e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:09.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.939+0000 7f09609fe700 1 --2- 172.21.15.55:0/2228845587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f095c15d240 0x7f095c15e680 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f095c070dd0 tx=0x7f0954018000 comp rx=0 tx=0).stop 2023-10-06T03:51:09.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.939+0000 7f09609fe700 1 -- 172.21.15.55:0/2228845587 shutdown_connections 2023-10-06T03:51:09.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.939+0000 7f09609fe700 1 --2- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0948067170 0x7f0948069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.939+0000 7f09609fe700 1 --2- 172.21.15.55:0/2228845587 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f095c15d240 0x7f095c15e680 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.939+0000 7f09609fe700 1 --2- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f095c0b16d0 0x7f095c158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.939+0000 7f09609fe700 1 --2- 172.21.15.55:0/2228845587 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f095c0b4e70 0x7f095c159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:09.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.939+0000 7f09609fe700 1 -- 172.21.15.55:0/2228845587 >> 172.21.15.55:0/2228845587 conn(0x7f095c0d10c0 msgr2=0x7f095c0ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:09.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.939+0000 7f09609fe700 1 -- 172.21.15.55:0/2228845587 shutdown_connections 2023-10-06T03:51:09.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:09.939+0000 7f09609fe700 1 -- 172.21.15.55:0/2228845587 wait complete. 2023-10-06T03:51:09.956 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:09.954+0000 7f3612271180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:51:09.959 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:51:10.053 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:10.051+0000 7f3612271180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:51:10.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.145+0000 7fc662f9d700 1 Processor -- start 2023-10-06T03:51:10.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.145+0000 7fc662f9d700 1 -- start start 2023-10-06T03:51:10.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.146+0000 7fc662f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc65c159420 0x7fc65c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:10.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.146+0000 7fc662f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc65c1597f0 0x7fc65c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:10.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.146+0000 7fc662f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc65c15d230 0x7fc65c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:10.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.146+0000 7fc662f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65c15dbd0 con 0x7fc65c15d230 2023-10-06T03:51:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.146+0000 7fc662f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65c0b11d0 con 0x7fc65c1597f0 2023-10-06T03:51:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.146+0000 7fc662f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c0b14b0 con 0x7fc65c159420 2023-10-06T03:51:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.147+0000 7fc66279c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc65c15d230 0x7fc65c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.147+0000 7fc66279c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc65c15d230 0x7fc65c15d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54310/0 (socket says 172.21.15.55:54310) 2023-10-06T03:51:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.147+0000 7fc66279c700 1 -- 172.21.15.55:0/3800174043 learned_addr learned my addr 172.21.15.55:0/3800174043 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.147+0000 7fc661f9b700 1 --2- 172.21.15.55:0/3800174043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc65c159420 0x7fc65c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:10.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.147+0000 7fc66179a700 1 --2- 172.21.15.55:0/3800174043 >> v2:172.21.15.163:3300/0 conn(0x7fc65c1597f0 0x7fc65c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.147+0000 7fc66279c700 1 -- 172.21.15.55:0/3800174043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc65c159420 msgr2=0x7fc65c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.147+0000 7fc66279c700 1 --2- 172.21.15.55:0/3800174043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc65c159420 0x7fc65c0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.147+0000 7fc66279c700 1 -- 172.21.15.55:0/3800174043 >> v2:172.21.15.163:3300/0 conn(0x7fc65c1597f0 msgr2=0x7fc65c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.147+0000 7fc66279c700 1 --2- 172.21.15.55:0/3800174043 >> v2:172.21.15.163:3300/0 conn(0x7fc65c1597f0 0x7fc65c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.147+0000 7fc66279c700 1 -- 172.21.15.55:0/3800174043 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c0b1790 con 0x7fc65c15d230 2023-10-06T03:51:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.148+0000 7fc66279c700 1 --2- 172.21.15.55:0/3800174043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc65c15d230 0x7fc65c15d690 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0d1770 tx=0x7fc654009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b6848e0c9a5657 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.148+0000 7fc660f99700 1 -- 172.21.15.55:0/3800174043 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65400e070 con 0x7fc65c15d230 2023-10-06T03:51:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.148+0000 7fc660f99700 1 -- 172.21.15.55:0/3800174043 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65400b970 con 0x7fc65c15d230 2023-10-06T03:51:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.148+0000 7fc660f99700 1 -- 172.21.15.55:0/3800174043 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6540092e0 con 0x7fc65c15d230 2023-10-06T03:51:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.148+0000 7fc662f9d700 1 -- 172.21.15.55:0/3800174043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc65c15d230 msgr2=0x7fc65c15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:10.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.148+0000 7fc662f9d700 1 --2- 172.21.15.55:0/3800174043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc65c15d230 0x7fc65c15d690 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0d1770 tx=0x7fc654009840 comp rx=0 tx=0).stop 2023-10-06T03:51:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.148+0000 7fc662f9d700 1 -- 172.21.15.55:0/3800174043 shutdown_connections 2023-10-06T03:51:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.148+0000 7fc662f9d700 1 --2- 172.21.15.55:0/3800174043 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc65c15d230 0x7fc65c15d690 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.148+0000 7fc662f9d700 1 --2- 172.21.15.55:0/3800174043 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc65c159420 0x7fc65c0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.148+0000 7fc662f9d700 1 --2- 172.21.15.55:0/3800174043 >> v2:172.21.15.163:3300/0 conn(0x7fc65c1597f0 0x7fc65c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.148+0000 7fc662f9d700 1 -- 172.21.15.55:0/3800174043 >> 172.21.15.55:0/3800174043 conn(0x7fc65c0d0660 msgr2=0x7fc65c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.149+0000 7fc662f9d700 1 -- 172.21.15.55:0/3800174043 shutdown_connections 2023-10-06T03:51:10.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.149+0000 7fc662f9d700 1 -- 172.21.15.55:0/3800174043 wait complete. 2023-10-06T03:51:10.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.150+0000 7fc662f9d700 1 Processor -- start 2023-10-06T03:51:10.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.150+0000 7fc662f9d700 1 -- start start 2023-10-06T03:51:10.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.151+0000 7fc662f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc65c159420 0x7fc65c155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:10.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.151+0000 7fc662f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc65c1597f0 0x7fc65c0c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.151+0000 7fc662f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc65c15d230 0x7fc65c0b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.151+0000 7fc662f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c155e90 con 0x7fc65c15d230 2023-10-06T03:51:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.151+0000 7fc662f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c0ca640 con 0x7fc65c1597f0 2023-10-06T03:51:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.151+0000 7fc662f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c0c6f40 con 0x7fc65c159420 2023-10-06T03:51:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.151+0000 7fc661f9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc65c159420 0x7fc65c155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.151+0000 7fc66179a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc65c1597f0 0x7fc65c0c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.151+0000 7fc66279c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc65c15d230 0x7fc65c0b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.151+0000 7fc66179a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc65c1597f0 0x7fc65c0c6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:59210/0 (socket says 172.21.15.55:59210) 2023-10-06T03:51:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.151+0000 7fc66179a700 1 -- 172.21.15.55:0/243469842 learned_addr learned my addr 172.21.15.55:0/243469842 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.151+0000 7fc661f9b700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc65c159420 0x7fc65c155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:53056/0 (socket says 172.21.15.55:53056) 2023-10-06T03:51:10.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.152+0000 7fc661f9b700 1 -- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc65c1597f0 msgr2=0x7fc65c0c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.152+0000 7fc661f9b700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc65c1597f0 0x7fc65c0c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.152+0000 7fc661f9b700 1 -- 172.21.15.55:0/243469842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc65c15d230 msgr2=0x7fc65c0b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.152+0000 7fc661f9b700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc65c15d230 0x7fc65c0b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.152+0000 7fc661f9b700 1 -- 172.21.15.55:0/243469842 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc654009ae0 con 0x7fc65c159420 2023-10-06T03:51:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.152+0000 7fc66179a700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc65c1597f0 0x7fc65c0c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.152+0000 7fc66279c700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc65c15d230 0x7fc65c0b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.152+0000 7fc661f9b700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc65c159420 0x7fc65c155950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc65800ea00 tx=0x7fc6580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.152+0000 7fc651ffb700 1 -- 172.21.15.55:0/243469842 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658013680 con 0x7fc65c159420 2023-10-06T03:51:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.152+0000 7fc651ffb700 1 -- 172.21.15.55:0/243469842 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc658013800 con 0x7fc65c159420 2023-10-06T03:51:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.153+0000 7fc662f9d700 1 -- 172.21.15.55:0/243469842 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65c0c7220 con 0x7fc65c159420 2023-10-06T03:51:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.153+0000 7fc651ffb700 1 -- 172.21.15.55:0/243469842 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658013bc0 con 0x7fc65c159420 2023-10-06T03:51:10.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.153+0000 7fc662f9d700 1 -- 172.21.15.55:0/243469842 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65c0c77e0 con 0x7fc65c159420 2023-10-06T03:51:10.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.154+0000 7fc662f9d700 1 -- 172.21.15.55:0/243469842 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65c0d01f0 con 0x7fc65c159420 2023-10-06T03:51:10.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.155+0000 7fc651ffb700 1 -- 172.21.15.55:0/243469842 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 46) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fc65800c490 con 0x7fc65c159420 2023-10-06T03:51:10.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.155+0000 7fc651ffb700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc640067170 0x7fc640069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:10.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.155+0000 7fc651ffb700 1 -- 172.21.15.55:0/243469842 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7fc658084d30 con 0x7fc65c159420 2023-10-06T03:51:10.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.156+0000 7fc66179a700 1 -- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc640067170 msgr2=0x7fc640069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:10.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.156+0000 7fc66179a700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc640067170 0x7fc640069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:10.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.160+0000 7fc651ffb700 1 -- 172.21.15.55:0/243469842 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc658051810 con 0x7fc65c159420 2023-10-06T03:51:10.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.357+0000 7fc66179a700 1 -- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc640067170 msgr2=0x7fc640069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:10.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.357+0000 7fc66179a700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc640067170 0x7fc640069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:10.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.407+0000 7fc662f9d700 1 -- 172.21.15.55:0/243469842 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc65c0ca780 con 0x7fc65c159420 2023-10-06T03:51:10.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.408+0000 7fc651ffb700 1 -- 172.21.15.55:0/243469842 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fc658020090 con 0x7fc65c159420 2023-10-06T03:51:10.411 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:10.411 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2272331626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2272331626},{"type":"v1","addr":"172.21.15.163:6834","nonce":2272331626}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2110964237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2110964237},{"type":"v1","addr":"172.21.15.163:6836","nonce":2110964237}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2942349047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2942349047},{"type":"v1","addr":"172.21.15.55:6833","nonce":2942349047}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:49:47.265041+0000","modified":"2023-10-06T03:49:50.792326+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7254":{"gid":7254,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/2226955335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2226955335},{"type":"v1","addr":"172.21.15.55:6835","nonce":2226955335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T03:51:10.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.411+0000 7fc662f9d700 1 -- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc640067170 msgr2=0x7fc640069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:10.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.411+0000 7fc662f9d700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc640067170 0x7fc640069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.411+0000 7fc662f9d700 1 -- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc65c159420 msgr2=0x7fc65c155950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:10.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.411+0000 7fc662f9d700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc65c159420 0x7fc65c155950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc65800ea00 tx=0x7fc6580091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.411+0000 7fc662f9d700 1 -- 172.21.15.55:0/243469842 shutdown_connections 2023-10-06T03:51:10.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.411+0000 7fc662f9d700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc640067170 0x7fc640069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.411+0000 7fc662f9d700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc65c15d230 0x7fc65c0b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.411+0000 7fc662f9d700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc65c159420 0x7fc65c155950 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.411+0000 7fc662f9d700 1 --2- 172.21.15.55:0/243469842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc65c1597f0 0x7fc65c0c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.411+0000 7fc662f9d700 1 -- 172.21.15.55:0/243469842 >> 172.21.15.55:0/243469842 conn(0x7fc65c0d0660 msgr2=0x7fc65c157a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:10.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.411+0000 7fc662f9d700 1 -- 172.21.15.55:0/243469842 shutdown_connections 2023-10-06T03:51:10.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.411+0000 7fc662f9d700 1 -- 172.21.15.55:0/243469842 wait complete. 2023-10-06T03:51:10.415 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 53 2023-10-06T03:51:10.431 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:51:10.506 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:10.505+0000 7f42572c8180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:51:10.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.648+0000 7fcd6f59d700 1 Processor -- start 2023-10-06T03:51:10.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.648+0000 7fcd6f59d700 1 -- start start 2023-10-06T03:51:10.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.651+0000 7fcd6f59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6815de70 0x7fcd6806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:10.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.651+0000 7fcd6f59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6806e8c0 con 0x7fcd680696c0 2023-10-06T03:51:10.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.651+0000 7fcd6f59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd680ca740 con 0x7fcd6815daa0 2023-10-06T03:51:10.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.651+0000 7fcd6f59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6806e5e0 con 0x7fcd6815de70 2023-10-06T03:51:10.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.651+0000 7fcd6e59b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fcd6815daa0 0x7fcd680c7110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:58356/0 (socket says 172.21.15.55:58356) 2023-10-06T03:51:10.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.651+0000 7fcd6e59b700 1 -- 172.21.15.55:0/434581112 learned_addr learned my addr 172.21.15.55:0/434581112 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:10.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.651+0000 7fcd6dd9a700 1 --2- 172.21.15.55:0/434581112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6815de70 0x7fcd6806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:10.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.652+0000 7fcd6d599700 1 -- 172.21.15.55:0/434581112 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221159265 0 0) 0x7fcd680ca740 con 0x7fcd6815daa0 2023-10-06T03:51:10.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.652+0000 7fcd6d599700 1 -- 172.21.15.55:0/434581112 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd50003570 con 0x7fcd6815daa0 2023-10-06T03:51:10.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.652+0000 7fcd6dd9a700 1 -- 172.21.15.55:0/434581112 >> v1:172.21.15.163:6789/0 conn(0x7fcd6815daa0 legacy=0x7fcd680c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:10.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.652+0000 7fcd6dd9a700 1 -- 172.21.15.55:0/434581112 >> v1:172.21.15.55:6789/0 conn(0x7fcd680696c0 legacy=0x7fcd6806ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:10.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.652+0000 7fcd6dd9a700 1 -- 172.21.15.55:0/434581112 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680ca0f0 con 0x7fcd6815de70 2023-10-06T03:51:10.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.652+0000 7fcd6dd9a700 1 --2- 172.21.15.55:0/434581112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6815de70 0x7fcd6806e000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009e90 tx=0x7fcd5c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca7644c38b634b2d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:10.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.652+0000 7fcd6d599700 1 -- 172.21.15.55:0/434581112 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c00e070 con 0x7fcd6815de70 2023-10-06T03:51:10.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.653+0000 7fcd6d599700 1 -- 172.21.15.55:0/434581112 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c00bc30 con 0x7fcd6815de70 2023-10-06T03:51:10.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.653+0000 7fcd6f59d700 1 -- 172.21.15.55:0/434581112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6815de70 msgr2=0x7fcd6806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:10.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.653+0000 7fcd6f59d700 1 --2- 172.21.15.55:0/434581112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6815de70 0x7fcd6806e000 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009e90 tx=0x7fcd5c009780 comp rx=0 tx=0).stop 2023-10-06T03:51:10.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.653+0000 7fcd6f59d700 1 -- 172.21.15.55:0/434581112 shutdown_connections 2023-10-06T03:51:10.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.653+0000 7fcd6f59d700 1 --2- 172.21.15.55:0/434581112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6815de70 0x7fcd6806e000 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.653+0000 7fcd6f59d700 1 -- 172.21.15.55:0/434581112 >> 172.21.15.55:0/434581112 conn(0x7fcd680cc1f0 msgr2=0x7fcd680cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:10.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.653+0000 7fcd6f59d700 1 -- 172.21.15.55:0/434581112 shutdown_connections 2023-10-06T03:51:10.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.653+0000 7fcd6f59d700 1 -- 172.21.15.55:0/434581112 wait complete. 2023-10-06T03:51:10.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.655+0000 7fcd6f59d700 1 Processor -- start 2023-10-06T03:51:10.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.655+0000 7fcd6f59d700 1 -- start start 2023-10-06T03:51:10.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.656+0000 7fcd6f59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd680696c0 0x7fcd680c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:10.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.656+0000 7fcd6f59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd6815daa0 0x7fcd680c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:10.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.656+0000 7fcd6f59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6815de70 0x7fcd68147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:10.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.656+0000 7fcd6f59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680c61e0 con 0x7fcd680696c0 2023-10-06T03:51:10.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.656+0000 7fcd6f59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680c4c90 con 0x7fcd6815daa0 2023-10-06T03:51:10.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.656+0000 7fcd6f59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680c4ee0 con 0x7fcd6815de70 2023-10-06T03:51:10.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.656+0000 7fcd6e59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd680696c0 0x7fcd680c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:10.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.656+0000 7fcd6e59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd680696c0 0x7fcd680c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54336/0 (socket says 172.21.15.55:54336) 2023-10-06T03:51:10.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.656+0000 7fcd6e59b700 1 -- 172.21.15.55:0/1757835608 learned_addr learned my addr 172.21.15.55:0/1757835608 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:10.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.656+0000 7fcd6dd9a700 1 --2- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd6815daa0 0x7fcd680c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:10.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.656+0000 7fcd6ed9c700 1 --2- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6815de70 0x7fcd68147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:10.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.657+0000 7fcd6e59b700 1 -- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6815de70 msgr2=0x7fcd68147610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:10.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.657+0000 7fcd6e59b700 1 --2- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6815de70 0x7fcd68147610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.657+0000 7fcd6e59b700 1 -- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd6815daa0 msgr2=0x7fcd680c3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:10.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.657+0000 7fcd6e59b700 1 --2- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd6815daa0 0x7fcd680c3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.657+0000 7fcd6e59b700 1 -- 172.21.15.55:0/1757835608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c009a20 con 0x7fcd680696c0 2023-10-06T03:51:10.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.657+0000 7fcd6e59b700 1 --2- 172.21.15.55:0/1757835608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd680696c0 0x7fcd680c5ca0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000c7c0 tx=0x7fcd60007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:10.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.657+0000 7fcd667fc700 1 -- 172.21.15.55:0/1757835608 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60005b30 con 0x7fcd680696c0 2023-10-06T03:51:10.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.657+0000 7fcd667fc700 1 -- 172.21.15.55:0/1757835608 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd60005c90 con 0x7fcd680696c0 2023-10-06T03:51:10.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.657+0000 7fcd6f59d700 1 -- 172.21.15.55:0/1757835608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd680c51c0 con 0x7fcd680696c0 2023-10-06T03:51:10.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.657+0000 7fcd667fc700 1 -- 172.21.15.55:0/1757835608 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60007320 con 0x7fcd680696c0 2023-10-06T03:51:10.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.657+0000 7fcd6f59d700 1 -- 172.21.15.55:0/1757835608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd680c5800 con 0x7fcd680696c0 2023-10-06T03:51:10.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.659+0000 7fcd6f59d700 1 -- 172.21.15.55:0/1757835608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6804c3c0 con 0x7fcd680696c0 2023-10-06T03:51:10.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.659+0000 7fcd667fc700 1 -- 172.21.15.55:0/1757835608 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 46) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fcd6001b3b0 con 0x7fcd680696c0 2023-10-06T03:51:10.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.660+0000 7fcd667fc700 1 --2- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd54067170 0x7fcd54069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:10.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.660+0000 7fcd667fc700 1 -- 172.21.15.55:0/1757835608 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7fcd60084270 con 0x7fcd680696c0 2023-10-06T03:51:10.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.661+0000 7fcd6dd9a700 1 -- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd54067170 msgr2=0x7fcd54069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:10.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.661+0000 7fcd6dd9a700 1 --2- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd54067170 0x7fcd54069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:10.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.664+0000 7fcd667fc700 1 -- 172.21.15.55:0/1757835608 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd60050d50 con 0x7fcd680696c0 2023-10-06T03:51:10.679 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:10.678+0000 7f42572c8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:51:10.717 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:10.716+0000 7f3612271180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:51:10.777 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:10.775+0000 7f42572c8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:51:10.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.862+0000 7fcd6dd9a700 1 -- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd54067170 msgr2=0x7fcd54069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:10.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.862+0000 7fcd6dd9a700 1 --2- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd54067170 0x7fcd54069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:10.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.871+0000 7fcd6f59d700 1 -- 172.21.15.55:0/1757835608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd680ca3d0 con 0x7fcd680696c0 2023-10-06T03:51:10.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.872+0000 7fcd667fc700 1 -- 172.21.15.55:0/1757835608 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+18088 (secure 0 0 0) 0x7fcd60054370 con 0x7fcd680696c0 2023-10-06T03:51:10.874 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:10.874 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":110,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:51:04.865684+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:49:45.423436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T03:49:46.240142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.1","mappings":[{"from":1,"to":7}]},{"pgid":"14.2c","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]},{"pgid":"14.36","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/1033630463":"2023-10-07T03:50:02.911384+0000","172.21.15.163:0/347072733":"2023-10-07T03:50:02.911384+0000","172.21.15.163:6837/85191":"2023-10-07T03:50:02.911384+0000","172.21.15.163:6835/85191":"2023-10-07T03:50:02.911384+0000","172.21.15.163:0/1160979081":"2023-10-07T03:50:02.911384+0000","172.21.15.163:0/1786945833":"2023-10-07T03:50:02.911384+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:51:10.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.875+0000 7fcd6f59d700 1 -- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd54067170 msgr2=0x7fcd54069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:10.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.875+0000 7fcd6f59d700 1 --2- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd54067170 0x7fcd54069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.875+0000 7fcd6f59d700 1 -- 172.21.15.55:0/1757835608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd680696c0 msgr2=0x7fcd680c5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:10.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.875+0000 7fcd6f59d700 1 --2- 172.21.15.55:0/1757835608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd680696c0 0x7fcd680c5ca0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000c7c0 tx=0x7fcd60007e00 comp rx=0 tx=0).stop 2023-10-06T03:51:10.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.875+0000 7fcd6f59d700 1 -- 172.21.15.55:0/1757835608 shutdown_connections 2023-10-06T03:51:10.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.875+0000 7fcd6f59d700 1 --2- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd54067170 0x7fcd54069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.875+0000 7fcd6f59d700 1 --2- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6815de70 0x7fcd68147610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.875+0000 7fcd6f59d700 1 --2- 172.21.15.55:0/1757835608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd680696c0 0x7fcd680c5ca0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.875+0000 7fcd6f59d700 1 --2- 172.21.15.55:0/1757835608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd6815daa0 0x7fcd680c3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:10.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.875+0000 7fcd6f59d700 1 -- 172.21.15.55:0/1757835608 >> 172.21.15.55:0/1757835608 conn(0x7fcd680cc1f0 msgr2=0x7fcd680c23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:10.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.875+0000 7fcd6f59d700 1 -- 172.21.15.55:0/1757835608 shutdown_connections 2023-10-06T03:51:10.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:10.875+0000 7fcd6f59d700 1 -- 172.21.15.55:0/1757835608 wait complete. 2023-10-06T03:51:10.895 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:51:10.895 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:51:10.900 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:10.899+0000 7f3612271180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:51:11.004 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:11.003+0000 7f3612271180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:51:11.074 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:11.073+0000 7f42572c8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:51:11.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.085+0000 7f457f59e700 1 Processor -- start 2023-10-06T03:51:11.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.085+0000 7f457f59e700 1 -- start start 2023-10-06T03:51:11.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.087+0000 7f457f59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4580159420 0x7f45800b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:11.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.087+0000 7f457f59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45800b2980 con 0x7f458015d230 2023-10-06T03:51:11.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.087+0000 7f457f59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45800b11d0 con 0x7f45801597f0 2023-10-06T03:51:11.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.087+0000 7f457f59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45800b14b0 con 0x7f4580159420 2023-10-06T03:51:11.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.087+0000 7f457ed9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f458015d230 0x7f458015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51778/0 (socket says 172.21.15.55:51778) 2023-10-06T03:51:11.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.087+0000 7f457ed9d700 1 -- 172.21.15.55:0/3752795230 learned_addr learned my addr 172.21.15.55:0/3752795230 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:11.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.087+0000 7f457e59c700 1 --2- 172.21.15.55:0/3752795230 >> v2:172.21.15.163:3300/0 conn(0x7f4580159420 0x7f45800b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:11.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.087+0000 7f457d59a700 1 -- 172.21.15.55:0/3752795230 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746812975 0 0) 0x7f45800b2980 con 0x7f458015d230 2023-10-06T03:51:11.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.087+0000 7f457d59a700 1 -- 172.21.15.55:0/3752795230 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4568003570 con 0x7f458015d230 2023-10-06T03:51:11.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.088+0000 7f457d59a700 1 -- 172.21.15.55:0/3752795230 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554488583 0 0) 0x7f45800b11d0 con 0x7f45801597f0 2023-10-06T03:51:11.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.088+0000 7f457d59a700 1 -- 172.21.15.55:0/3752795230 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45800b2980 con 0x7f45801597f0 2023-10-06T03:51:11.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.088+0000 7f457e59c700 1 -- 172.21.15.55:0/3752795230 >> v1:172.21.15.163:6789/0 conn(0x7f45801597f0 legacy=0x7f45800b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:11.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.088+0000 7f457e59c700 1 -- 172.21.15.55:0/3752795230 >> v1:172.21.15.55:6789/0 conn(0x7f458015d230 legacy=0x7f458015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:11.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.088+0000 7f457e59c700 1 -- 172.21.15.55:0/3752795230 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45800b1790 con 0x7f4580159420 2023-10-06T03:51:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.088+0000 7f457e59c700 1 --2- 172.21.15.55:0/3752795230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4580159420 0x7f45800b5f40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f45700098e0 tx=0x7f45700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=831c321853cb14f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.088+0000 7f457d59a700 1 -- 172.21.15.55:0/3752795230 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570011630 con 0x7f4580159420 2023-10-06T03:51:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.088+0000 7f457d59a700 1 -- 172.21.15.55:0/3752795230 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4570011790 con 0x7f4580159420 2023-10-06T03:51:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.088+0000 7f457d59a700 1 -- 172.21.15.55:0/3752795230 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4570011b30 con 0x7f4580159420 2023-10-06T03:51:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.089+0000 7f457f59e700 1 -- 172.21.15.55:0/3752795230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4580159420 msgr2=0x7f45800b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.089+0000 7f457f59e700 1 --2- 172.21.15.55:0/3752795230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4580159420 0x7f45800b5f40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f45700098e0 tx=0x7f45700091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.089+0000 7f457f59e700 1 -- 172.21.15.55:0/3752795230 shutdown_connections 2023-10-06T03:51:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.089+0000 7f457f59e700 1 --2- 172.21.15.55:0/3752795230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4580159420 0x7f45800b5f40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.089+0000 7f457f59e700 1 -- 172.21.15.55:0/3752795230 >> 172.21.15.55:0/3752795230 conn(0x7f45800d0660 msgr2=0x7f45800d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:11.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.089+0000 7f457f59e700 1 -- 172.21.15.55:0/3752795230 shutdown_connections 2023-10-06T03:51:11.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.089+0000 7f457f59e700 1 -- 172.21.15.55:0/3752795230 wait complete. 2023-10-06T03:51:11.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.091+0000 7f457f59e700 1 Processor -- start 2023-10-06T03:51:11.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.091+0000 7f457f59e700 1 -- start start 2023-10-06T03:51:11.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4580159420 0x7f4580154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:11.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45801597f0 0x7f45800b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:11.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458015d230 0x7f45800b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:11.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4580154650 con 0x7f45801597f0 2023-10-06T03:51:11.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458015aaf0 con 0x7f458015d230 2023-10-06T03:51:11.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458015add0 con 0x7f4580159420 2023-10-06T03:51:11.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457e59c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4580159420 0x7f4580154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:11.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457dd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45801597f0 0x7f45800b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:11.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457e59c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4580159420 0x7f4580154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:53096/0 (socket says 172.21.15.55:53096) 2023-10-06T03:51:11.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457e59c700 1 -- 172.21.15.55:0/4023767033 learned_addr learned my addr 172.21.15.55:0/4023767033 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:11.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457ed9d700 1 --2- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458015d230 0x7f45800b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:11.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457dd9b700 1 -- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4580159420 msgr2=0x7f4580154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:11.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457dd9b700 1 --2- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4580159420 0x7f4580154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457dd9b700 1 -- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458015d230 msgr2=0x7f45800b5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:11.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.092+0000 7f457dd9b700 1 --2- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458015d230 0x7f45800b5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.093+0000 7f457dd9b700 1 -- 172.21.15.55:0/4023767033 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4570009420 con 0x7f45801597f0 2023-10-06T03:51:11.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.093+0000 7f457dd9b700 1 --2- 172.21.15.55:0/4023767033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45801597f0 0x7f45800b58e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4580159c70 tx=0x7f4574007d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:11.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.093+0000 7f456e7fc700 1 -- 172.21.15.55:0/4023767033 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574011590 con 0x7f45801597f0 2023-10-06T03:51:11.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.093+0000 7f456e7fc700 1 -- 172.21.15.55:0/4023767033 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45740116f0 con 0x7f45801597f0 2023-10-06T03:51:11.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.093+0000 7f457f59e700 1 -- 172.21.15.55:0/4023767033 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458015b0b0 con 0x7f45801597f0 2023-10-06T03:51:11.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.093+0000 7f456e7fc700 1 -- 172.21.15.55:0/4023767033 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574011ab0 con 0x7f45801597f0 2023-10-06T03:51:11.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.093+0000 7f457f59e700 1 -- 172.21.15.55:0/4023767033 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4580159d50 con 0x7f45801597f0 2023-10-06T03:51:11.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.094+0000 7f457f59e700 1 -- 172.21.15.55:0/4023767033 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458004c3c0 con 0x7f45801597f0 2023-10-06T03:51:11.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.095+0000 7f456e7fc700 1 -- 172.21.15.55:0/4023767033 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 46) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f457401c460 con 0x7f45801597f0 2023-10-06T03:51:11.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.096+0000 7f456e7fc700 1 --2- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4560067170 0x7f4560069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:11.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.096+0000 7f456e7fc700 1 -- 172.21.15.55:0/4023767033 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7f45740848a0 con 0x7f45801597f0 2023-10-06T03:51:11.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.096+0000 7f457e59c700 1 -- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4560067170 msgr2=0x7f4560069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:11.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.096+0000 7f457e59c700 1 --2- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4560067170 0x7f4560069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:11.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.101+0000 7f456e7fc700 1 -- 172.21.15.55:0/4023767033 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4574051420 con 0x7f45801597f0 2023-10-06T03:51:11.170 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:11.169+0000 7f42572c8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:51:11.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.297+0000 7f457e59c700 1 -- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4560067170 msgr2=0x7f4560069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:11.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.297+0000 7f457e59c700 1 --2- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4560067170 0x7f4560069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:11.314 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:11.310+0000 7f3612271180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:51:11.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.349+0000 7f457f59e700 1 -- 172.21.15.55:0/4023767033 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4580169b30 con 0x7f45801597f0 2023-10-06T03:51:11.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.350+0000 7f456e7fc700 1 -- 172.21.15.55:0/4023767033 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f4574054a40 con 0x7f45801597f0 2023-10-06T03:51:11.352 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:11.352 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2272331626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2272331626},{"type":"v1","addr":"172.21.15.163:6834","nonce":2272331626}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2110964237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2110964237},{"type":"v1","addr":"172.21.15.163:6836","nonce":2110964237}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2942349047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2942349047},{"type":"v1","addr":"172.21.15.55:6833","nonce":2942349047}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:49:47.265041+0000","modified":"2023-10-06T03:49:50.792326+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7254":{"gid":7254,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/2226955335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2226955335},{"type":"v1","addr":"172.21.15.55:6835","nonce":2226955335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T03:51:11.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.353+0000 7f457f59e700 1 -- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4560067170 msgr2=0x7f4560069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:11.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.353+0000 7f457f59e700 1 --2- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4560067170 0x7f4560069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.353+0000 7f457f59e700 1 -- 172.21.15.55:0/4023767033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45801597f0 msgr2=0x7f45800b58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:11.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.353+0000 7f457f59e700 1 --2- 172.21.15.55:0/4023767033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45801597f0 0x7f45800b58e0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4580159c70 tx=0x7f4574007d50 comp rx=0 tx=0).stop 2023-10-06T03:51:11.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.353+0000 7f457f59e700 1 -- 172.21.15.55:0/4023767033 shutdown_connections 2023-10-06T03:51:11.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.353+0000 7f457f59e700 1 --2- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4560067170 0x7f4560069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.353+0000 7f457f59e700 1 --2- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f458015d230 0x7f45800b5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.353+0000 7f457f59e700 1 --2- 172.21.15.55:0/4023767033 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4580159420 0x7f4580154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.353+0000 7f457f59e700 1 --2- 172.21.15.55:0/4023767033 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f45801597f0 0x7f45800b58e0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.353+0000 7f457f59e700 1 -- 172.21.15.55:0/4023767033 >> 172.21.15.55:0/4023767033 conn(0x7f45800d0660 msgr2=0x7f4580069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:11.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.353+0000 7f457f59e700 1 -- 172.21.15.55:0/4023767033 shutdown_connections 2023-10-06T03:51:11.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.353+0000 7f457f59e700 1 -- 172.21.15.55:0/4023767033 wait complete. 2023-10-06T03:51:11.357 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 53 2023-10-06T03:51:11.374 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:51:11.419 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:11.415+0000 7f3612271180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:51:11.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.602+0000 7fda0e3ff700 1 Processor -- start 2023-10-06T03:51:11.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.602+0000 7fda0e3ff700 1 -- start start 2023-10-06T03:51:11.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.603+0000 7fda0e3ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fda08159890 0x7fda0815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:11.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.603+0000 7fda0e3ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fda08069560 0x7fda080699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:11.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.603+0000 7fda0e3ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda08069ee0 con 0x7fda08159c60 2023-10-06T03:51:11.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.603+0000 7fda0e3ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda0815a160 con 0x7fda08159890 2023-10-06T03:51:11.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.603+0000 7fda0e3ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda080d13a0 con 0x7fda08069560 2023-10-06T03:51:11.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.604+0000 7fda07fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fda08159890 0x7fda0815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:11.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.604+0000 7fda07fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fda08159890 0x7fda0815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54356/0 (socket says 172.21.15.55:54356) 2023-10-06T03:51:11.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.604+0000 7fda077fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fda08159c60 0x7fda0815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:51780/0 (socket says 172.21.15.55:51780) 2023-10-06T03:51:11.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.604+0000 7fda07fff700 1 -- 172.21.15.55:0/891804949 learned_addr learned my addr 172.21.15.55:0/891804949 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:11.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.604+0000 7fda0c99c700 1 --2- 172.21.15.55:0/891804949 >> v2:172.21.15.163:3300/0 conn(0x7fda08069560 0x7fda080699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:11.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.604+0000 7fda07fff700 1 -- 172.21.15.55:0/891804949 >> v2:172.21.15.163:3300/0 conn(0x7fda08069560 msgr2=0x7fda080699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:11.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.604+0000 7fda07fff700 1 --2- 172.21.15.55:0/891804949 >> v2:172.21.15.163:3300/0 conn(0x7fda08069560 0x7fda080699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.604+0000 7fda07fff700 1 -- 172.21.15.55:0/891804949 >> v1:172.21.15.55:6789/0 conn(0x7fda08159c60 legacy=0x7fda0815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:11.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.604+0000 7fda07fff700 1 -- 172.21.15.55:0/891804949 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0815e140 con 0x7fda08159890 2023-10-06T03:51:11.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.604+0000 7fda07fff700 1 --2- 172.21.15.55:0/891804949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda08159890 0x7fda0815abd0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fda0806a890 tx=0x7fd9f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7369f51d15107003 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:11.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.605+0000 7fda06ffd700 1 -- 172.21.15.55:0/891804949 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8009e70 con 0x7fda08159890 2023-10-06T03:51:11.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.605+0000 7fda06ffd700 1 -- 172.21.15.55:0/891804949 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f800b860 con 0x7fda08159890 2023-10-06T03:51:11.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.605+0000 7fda06ffd700 1 -- 172.21.15.55:0/891804949 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f800bc00 con 0x7fda08159890 2023-10-06T03:51:11.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.605+0000 7fda0e3ff700 1 -- 172.21.15.55:0/891804949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda08159890 msgr2=0x7fda0815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:11.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.605+0000 7fda0e3ff700 1 --2- 172.21.15.55:0/891804949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda08159890 0x7fda0815abd0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fda0806a890 tx=0x7fd9f8009250 comp rx=0 tx=0).stop 2023-10-06T03:51:11.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.605+0000 7fda0e3ff700 1 -- 172.21.15.55:0/891804949 shutdown_connections 2023-10-06T03:51:11.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.605+0000 7fda0e3ff700 1 --2- 172.21.15.55:0/891804949 >> v2:172.21.15.163:3300/0 conn(0x7fda08069560 0x7fda080699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.605+0000 7fda0e3ff700 1 --2- 172.21.15.55:0/891804949 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda08159890 0x7fda0815abd0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.605+0000 7fda0e3ff700 1 -- 172.21.15.55:0/891804949 >> 172.21.15.55:0/891804949 conn(0x7fda08068e20 msgr2=0x7fda080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:11.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.606+0000 7fda0e3ff700 1 -- 172.21.15.55:0/891804949 shutdown_connections 2023-10-06T03:51:11.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.606+0000 7fda0e3ff700 1 -- 172.21.15.55:0/891804949 wait complete. 2023-10-06T03:51:11.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.607+0000 7fda0e3ff700 1 Processor -- start 2023-10-06T03:51:11.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.607+0000 7fda0e3ff700 1 -- start start 2023-10-06T03:51:11.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.608+0000 7fda0e3ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda08069560 0x7fda080c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:11.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.608+0000 7fda0e3ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda08159890 0x7fda080b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:11.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.608+0000 7fda0e3ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda08159c60 0x7fda0816d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:11.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.608+0000 7fda0e3ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda080b64c0 con 0x7fda08159c60 2023-10-06T03:51:11.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.608+0000 7fda0e3ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda080b4430 con 0x7fda08159890 2023-10-06T03:51:11.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.608+0000 7fda0e3ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda080b4710 con 0x7fda08069560 2023-10-06T03:51:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.608+0000 7fda0c99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda08159c60 0x7fda0816d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.608+0000 7fda077fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda08159890 0x7fda080b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.608+0000 7fda0c99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda08159c60 0x7fda0816d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54368/0 (socket says 172.21.15.55:54368) 2023-10-06T03:51:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.608+0000 7fda0c99c700 1 -- 172.21.15.55:0/527842180 learned_addr learned my addr 172.21.15.55:0/527842180 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.608+0000 7fda07fff700 1 --2- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda08069560 0x7fda080c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.609+0000 7fda0c99c700 1 -- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda08069560 msgr2=0x7fda080c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.609+0000 7fda0c99c700 1 --2- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda08069560 0x7fda080c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.609+0000 7fda0c99c700 1 -- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda08159890 msgr2=0x7fda080b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.609+0000 7fda0c99c700 1 --2- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda08159890 0x7fda080b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.609+0000 7fda0c99c700 1 -- 172.21.15.55:0/527842180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f80094f0 con 0x7fda08159c60 2023-10-06T03:51:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.609+0000 7fda0c99c700 1 --2- 172.21.15.55:0/527842180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda08159c60 0x7fda0816d3d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fda00009e90 tx=0x7fda0000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:11.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.609+0000 7fd9effff700 1 -- 172.21.15.55:0/527842180 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0000cb70 con 0x7fda08159c60 2023-10-06T03:51:11.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.609+0000 7fd9effff700 1 -- 172.21.15.55:0/527842180 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0000f040 con 0x7fda08159c60 2023-10-06T03:51:11.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.609+0000 7fd9effff700 1 -- 172.21.15.55:0/527842180 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00014480 con 0x7fda08159c60 2023-10-06T03:51:11.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.610+0000 7fda0e3ff700 1 -- 172.21.15.55:0/527842180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda080b49f0 con 0x7fda08159c60 2023-10-06T03:51:11.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.610+0000 7fda0e3ff700 1 -- 172.21.15.55:0/527842180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda080b61b0 con 0x7fda08159c60 2023-10-06T03:51:11.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.611+0000 7fda0e3ff700 1 -- 172.21.15.55:0/527842180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0804c3c0 con 0x7fda08159c60 2023-10-06T03:51:11.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.612+0000 7fd9effff700 1 -- 172.21.15.55:0/527842180 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 46) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fda00014830 con 0x7fda08159c60 2023-10-06T03:51:11.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.612+0000 7fd9effff700 1 --2- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9f4067170 0x7fd9f4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:11.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.613+0000 7fd9effff700 1 -- 172.21.15.55:0/527842180 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7fda00085b50 con 0x7fda08159c60 2023-10-06T03:51:11.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.613+0000 7fda07fff700 1 -- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9f4067170 msgr2=0x7fd9f4069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:11.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.613+0000 7fda07fff700 1 --2- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9f4067170 0x7fd9f4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:11.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.617+0000 7fd9effff700 1 -- 172.21.15.55:0/527842180 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fda00052520 con 0x7fda08159c60 2023-10-06T03:51:11.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.814+0000 7fda07fff700 1 -- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9f4067170 msgr2=0x7fd9f4069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:11.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.814+0000 7fda07fff700 1 --2- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9f4067170 0x7fd9f4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:11.875 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:11.874+0000 7f42572c8180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:51:11.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.921+0000 7fda0e3ff700 1 -- 172.21.15.55:0/527842180 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda0806a5b0 con 0x7fda08159c60 2023-10-06T03:51:11.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.922+0000 7fd9effff700 1 -- 172.21.15.55:0/527842180 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fda0005b070 con 0x7fda08159c60 2023-10-06T03:51:11.924 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:11.924 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2272331626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2272331626},{"type":"v1","addr":"172.21.15.163:6834","nonce":2272331626}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2110964237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2110964237},{"type":"v1","addr":"172.21.15.163:6836","nonce":2110964237}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2942349047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2942349047},{"type":"v1","addr":"172.21.15.55:6833","nonce":2942349047}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:49:47.265041+0000","modified":"2023-10-06T03:49:50.792326+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7254":{"gid":7254,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/2226955335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":2226955335},{"type":"v1","addr":"172.21.15.55:6835","nonce":2226955335}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-06T03:51:11.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.925+0000 7fda0e3ff700 1 -- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9f4067170 msgr2=0x7fd9f4069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:11.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.925+0000 7fda0e3ff700 1 --2- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9f4067170 0x7fd9f4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.925+0000 7fda0e3ff700 1 -- 172.21.15.55:0/527842180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda08159c60 msgr2=0x7fda0816d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:11.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.925+0000 7fda0e3ff700 1 --2- 172.21.15.55:0/527842180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda08159c60 0x7fda0816d3d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fda00009e90 tx=0x7fda0000b040 comp rx=0 tx=0).stop 2023-10-06T03:51:11.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.925+0000 7fda0e3ff700 1 -- 172.21.15.55:0/527842180 shutdown_connections 2023-10-06T03:51:11.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.925+0000 7fda0e3ff700 1 --2- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9f4067170 0x7fd9f4069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.925+0000 7fda0e3ff700 1 --2- 172.21.15.55:0/527842180 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fda08159c60 0x7fda0816d3d0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.925+0000 7fda0e3ff700 1 --2- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fda08069560 0x7fda080c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.925+0000 7fda0e3ff700 1 --2- 172.21.15.55:0/527842180 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fda08159890 0x7fda080b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:11.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.925+0000 7fda0e3ff700 1 -- 172.21.15.55:0/527842180 >> 172.21.15.55:0/527842180 conn(0x7fda08068e20 msgr2=0x7fda081534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:11.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.925+0000 7fda0e3ff700 1 -- 172.21.15.55:0/527842180 shutdown_connections 2023-10-06T03:51:11.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:11.925+0000 7fda0e3ff700 1 -- 172.21.15.55:0/527842180 wait complete. 2023-10-06T03:51:11.928 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 53 2023-10-06T03:51:11.944 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:51:12.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.132+0000 7f6daed9d700 1 Processor -- start 2023-10-06T03:51:12.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.132+0000 7f6daed9d700 1 -- start start 2023-10-06T03:51:12.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.133+0000 7f6daed9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6da81597f0 0x7f6da80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:12.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.134+0000 7f6daed9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6da815d230 0x7f6da815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:12.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.134+0000 7f6daed9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da815dbd0 con 0x7f6da8159420 2023-10-06T03:51:12.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.134+0000 7f6daed9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da80b11d0 con 0x7f6da815d230 2023-10-06T03:51:12.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.134+0000 7f6daed9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da80b14b0 con 0x7f6da81597f0 2023-10-06T03:51:12.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.134+0000 7f6dae59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6da815d230 0x7f6da815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:12.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.134+0000 7f6dae59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6da815d230 0x7f6da815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54376/0 (socket says 172.21.15.55:54376) 2023-10-06T03:51:12.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.134+0000 7f6dadd9b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f6da8159420 0x7f6da80b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:58368/0 (socket says 172.21.15.55:58368) 2023-10-06T03:51:12.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.134+0000 7f6dae59c700 1 -- 172.21.15.55:0/807613970 learned_addr learned my addr 172.21.15.55:0/807613970 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:12.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.134+0000 7f6dad59a700 1 --2- 172.21.15.55:0/807613970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6da81597f0 0x7f6da80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:12.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.134+0000 7f6dae59c700 1 -- 172.21.15.55:0/807613970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6da81597f0 msgr2=0x7f6da80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:12.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.134+0000 7f6dae59c700 1 --2- 172.21.15.55:0/807613970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6da81597f0 0x7f6da80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.134+0000 7f6dae59c700 1 -- 172.21.15.55:0/807613970 >> v1:172.21.15.163:6789/0 conn(0x7f6da8159420 legacy=0x7f6da80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:12.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.134+0000 7f6dae59c700 1 -- 172.21.15.55:0/807613970 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da80b1790 con 0x7f6da815d230 2023-10-06T03:51:12.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.134+0000 7f6dacd99700 1 -- 172.21.15.55:0/807613970 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650032700 0 0) 0x7f6da4001110 con 0x7f6da8159420 2023-10-06T03:51:12.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.135+0000 7f6dae59c700 1 --2- 172.21.15.55:0/807613970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da815d230 0x7f6da815d690 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009f20 tx=0x7f6da0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369cbb21fd13c9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:12.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.135+0000 7f6dacd99700 1 -- 172.21.15.55:0/807613970 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da000e070 con 0x7f6da815d230 2023-10-06T03:51:12.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.135+0000 7f6dacd99700 1 -- 172.21.15.55:0/807613970 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da000b9d0 con 0x7f6da815d230 2023-10-06T03:51:12.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.135+0000 7f6dacd99700 1 -- 172.21.15.55:0/807613970 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da00092e0 con 0x7f6da815d230 2023-10-06T03:51:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.135+0000 7f6daed9d700 1 -- 172.21.15.55:0/807613970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da815d230 msgr2=0x7f6da815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.135+0000 7f6daed9d700 1 --2- 172.21.15.55:0/807613970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da815d230 0x7f6da815d690 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009f20 tx=0x7f6da0009840 comp rx=0 tx=0).stop 2023-10-06T03:51:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.136+0000 7f6daed9d700 1 -- 172.21.15.55:0/807613970 shutdown_connections 2023-10-06T03:51:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.136+0000 7f6daed9d700 1 --2- 172.21.15.55:0/807613970 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da815d230 0x7f6da815d690 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.136+0000 7f6daed9d700 1 --2- 172.21.15.55:0/807613970 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6da81597f0 0x7f6da80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.136+0000 7f6daed9d700 1 -- 172.21.15.55:0/807613970 >> 172.21.15.55:0/807613970 conn(0x7f6da80d0660 msgr2=0x7f6da80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.136+0000 7f6daed9d700 1 -- 172.21.15.55:0/807613970 shutdown_connections 2023-10-06T03:51:12.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.136+0000 7f6daed9d700 1 -- 172.21.15.55:0/807613970 wait complete. 2023-10-06T03:51:12.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.137+0000 7f6daed9d700 1 Processor -- start 2023-10-06T03:51:12.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.137+0000 7f6daed9d700 1 -- start start 2023-10-06T03:51:12.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.138+0000 7f6daed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da8159420 0x7f6da81550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:12.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.138+0000 7f6daed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6da81597f0 0x7f6da815ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:12.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.138+0000 7f6daed9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6da815d230 0x7f6da815f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:12.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.138+0000 7f6daed9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8169a40 con 0x7f6da8159420 2023-10-06T03:51:12.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.138+0000 7f6daed9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da815c2b0 con 0x7f6da81597f0 2023-10-06T03:51:12.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.138+0000 7f6daed9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da815c590 con 0x7f6da815d230 2023-10-06T03:51:12.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.139+0000 7f6dadd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da8159420 0x7f6da81550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:12.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.139+0000 7f6dadd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da8159420 0x7f6da81550a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54382/0 (socket says 172.21.15.55:54382) 2023-10-06T03:51:12.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.139+0000 7f6dadd9b700 1 -- 172.21.15.55:0/3124603057 learned_addr learned my addr 172.21.15.55:0/3124603057 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:12.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.139+0000 7f6dad59a700 1 --2- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6da81597f0 0x7f6da815ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:12.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.139+0000 7f6dae59c700 1 --2- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6da815d230 0x7f6da815f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:12.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.139+0000 7f6dadd9b700 1 -- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6da815d230 msgr2=0x7f6da815f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:12.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.139+0000 7f6dadd9b700 1 --2- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6da815d230 0x7f6da815f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.139+0000 7f6dadd9b700 1 -- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6da81597f0 msgr2=0x7f6da815ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:12.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.139+0000 7f6dadd9b700 1 --2- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6da81597f0 0x7f6da815ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.139+0000 7f6dadd9b700 1 -- 172.21.15.55:0/3124603057 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da0009ae0 con 0x7f6da8159420 2023-10-06T03:51:12.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.139+0000 7f6dadd9b700 1 --2- 172.21.15.55:0/3124603057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da8159420 0x7f6da81550a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6da400c810 tx=0x7f6da4007c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:12.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.140+0000 7f6d9dffb700 1 -- 172.21.15.55:0/3124603057 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4005a70 con 0x7f6da8159420 2023-10-06T03:51:12.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.140+0000 7f6d9dffb700 1 -- 172.21.15.55:0/3124603057 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da4005bd0 con 0x7f6da8159420 2023-10-06T03:51:12.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.140+0000 7f6d9dffb700 1 -- 172.21.15.55:0/3124603057 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da40070d0 con 0x7f6da8159420 2023-10-06T03:51:12.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.140+0000 7f6daed9d700 1 -- 172.21.15.55:0/3124603057 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da815c870 con 0x7f6da8159420 2023-10-06T03:51:12.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.140+0000 7f6daed9d700 1 -- 172.21.15.55:0/3124603057 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da815bf50 con 0x7f6da8159420 2023-10-06T03:51:12.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.141+0000 7f6daed9d700 1 -- 172.21.15.55:0/3124603057 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da81555e0 con 0x7f6da8159420 2023-10-06T03:51:12.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.142+0000 7f6d9dffb700 1 -- 172.21.15.55:0/3124603057 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 46) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f6da40074c0 con 0x7f6da8159420 2023-10-06T03:51:12.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.143+0000 7f6d9dffb700 1 --2- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6d8c067170 0x7f6d8c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:12.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.143+0000 7f6d9dffb700 1 -- 172.21.15.55:0/3124603057 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7f6da4084170 con 0x7f6da8159420 2023-10-06T03:51:12.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.143+0000 7f6dad59a700 1 -- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6d8c067170 msgr2=0x7f6d8c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:12.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.143+0000 7f6dad59a700 1 --2- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6d8c067170 0x7f6d8c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:12.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.147+0000 7f6d9dffb700 1 -- 172.21.15.55:0/3124603057 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6da4050c50 con 0x7f6da8159420 2023-10-06T03:51:12.164 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:12.163+0000 7f42572c8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:51:12.220 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:12.219+0000 7f3612271180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:51:12.255 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:12.254+0000 7f42572c8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:51:12.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.344+0000 7f6dad59a700 1 -- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6d8c067170 msgr2=0x7f6d8c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:12.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.344+0000 7f6dad59a700 1 --2- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6d8c067170 0x7f6d8c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:12.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.357+0000 7f6daed9d700 1 -- 172.21.15.55:0/3124603057 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6da815dbd0 con 0x7f6da8159420 2023-10-06T03:51:12.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.358+0000 7f6d9dffb700 1 -- 172.21.15.55:0/3124603057 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+18088 (secure 0 0 0) 0x7f6da815dbd0 con 0x7f6da8159420 2023-10-06T03:51:12.360 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:12.361 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":110,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:51:04.865684+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:49:45.423436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-10-06T03:49:46.240142+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.1","mappings":[{"from":1,"to":7}]},{"pgid":"14.2c","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":2}]},{"pgid":"14.36","mappings":[{"from":1,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/1033630463":"2023-10-07T03:50:02.911384+0000","172.21.15.163:0/347072733":"2023-10-07T03:50:02.911384+0000","172.21.15.163:6837/85191":"2023-10-07T03:50:02.911384+0000","172.21.15.163:6835/85191":"2023-10-07T03:50:02.911384+0000","172.21.15.163:0/1160979081":"2023-10-07T03:50:02.911384+0000","172.21.15.163:0/1786945833":"2023-10-07T03:50:02.911384+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:51:12.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.361+0000 7f6daed9d700 1 -- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6d8c067170 msgr2=0x7f6d8c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:12.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.361+0000 7f6daed9d700 1 --2- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6d8c067170 0x7f6d8c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.361+0000 7f6daed9d700 1 -- 172.21.15.55:0/3124603057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da8159420 msgr2=0x7f6da81550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:12.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.361+0000 7f6daed9d700 1 --2- 172.21.15.55:0/3124603057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da8159420 0x7f6da81550a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6da400c810 tx=0x7f6da4007c50 comp rx=0 tx=0).stop 2023-10-06T03:51:12.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.361+0000 7f6daed9d700 1 -- 172.21.15.55:0/3124603057 shutdown_connections 2023-10-06T03:51:12.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.361+0000 7f6daed9d700 1 --2- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6d8c067170 0x7f6d8c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.361+0000 7f6daed9d700 1 --2- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6da815d230 0x7f6da815f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.361+0000 7f6daed9d700 1 --2- 172.21.15.55:0/3124603057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6da8159420 0x7f6da81550a0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.361+0000 7f6daed9d700 1 --2- 172.21.15.55:0/3124603057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6da81597f0 0x7f6da815ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.361+0000 7f6daed9d700 1 -- 172.21.15.55:0/3124603057 >> 172.21.15.55:0/3124603057 conn(0x7f6da80d0660 msgr2=0x7f6da80d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:12.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.361+0000 7f6daed9d700 1 -- 172.21.15.55:0/3124603057 shutdown_connections 2023-10-06T03:51:12.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.361+0000 7f6daed9d700 1 -- 172.21.15.55:0/3124603057 wait complete. 2023-10-06T03:51:12.382 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:51:12.410 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:12.409+0000 7f42572c8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:51:12.514 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:12.513+0000 7f42572c8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:51:12.518 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:12.517+0000 7f3612271180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:51:12.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.576+0000 7f43be39d700 1 Processor -- start 2023-10-06T03:51:12.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.576+0000 7f43be39d700 1 -- start start 2023-10-06T03:51:12.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.577+0000 7f43be39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f43b8159420 0x7f43b80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:12.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.577+0000 7f43be39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f43b81597f0 0x7f43b80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:12.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.577+0000 7f43be39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b80b11d0 con 0x7f43b815d230 2023-10-06T03:51:12.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.577+0000 7f43be39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b80b14b0 con 0x7f43b8159420 2023-10-06T03:51:12.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.577+0000 7f43be39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b80b1790 con 0x7f43b81597f0 2023-10-06T03:51:12.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.577+0000 7f43bd39b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f43b8159420 0x7f43b80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:12.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.577+0000 7f43bd39b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f43b8159420 0x7f43b80b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54390/0 (socket says 172.21.15.55:54390) 2023-10-06T03:51:12.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.577+0000 7f43bd39b700 1 -- 172.21.15.55:0/3158336074 learned_addr learned my addr 172.21.15.55:0/3158336074 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:12.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.578+0000 7f43bcb9a700 1 --2- 172.21.15.55:0/3158336074 >> v2:172.21.15.163:3300/0 conn(0x7f43b81597f0 0x7f43b80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:12.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.578+0000 7f43bd39b700 1 -- 172.21.15.55:0/3158336074 >> v2:172.21.15.163:3300/0 conn(0x7f43b81597f0 msgr2=0x7f43b80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:12.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.578+0000 7f43bd39b700 1 --2- 172.21.15.55:0/3158336074 >> v2:172.21.15.163:3300/0 conn(0x7f43b81597f0 0x7f43b80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.578+0000 7f43bd39b700 1 -- 172.21.15.55:0/3158336074 >> v1:172.21.15.163:6789/0 conn(0x7f43b815d230 legacy=0x7f43b815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:12.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.578+0000 7f43bd39b700 1 -- 172.21.15.55:0/3158336074 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b80b1a70 con 0x7f43b8159420 2023-10-06T03:51:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.578+0000 7f43affff700 1 -- 172.21.15.55:0/3158336074 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144956682 0 0) 0x7f43b80b11d0 con 0x7f43b815d230 2023-10-06T03:51:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.578+0000 7f43bd39b700 1 --2- 172.21.15.55:0/3158336074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43b8159420 0x7f43b80b5f40 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f43b8089750 tx=0x7f43b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87589765754deb0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.578+0000 7f43affff700 1 -- 172.21.15.55:0/3158336074 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b4009e70 con 0x7f43b8159420 2023-10-06T03:51:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.579+0000 7f43affff700 1 -- 172.21.15.55:0/3158336074 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b400b860 con 0x7f43b8159420 2023-10-06T03:51:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.579+0000 7f43affff700 1 -- 172.21.15.55:0/3158336074 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b400bc00 con 0x7f43b8159420 2023-10-06T03:51:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.579+0000 7f43be39d700 1 -- 172.21.15.55:0/3158336074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43b8159420 msgr2=0x7f43b80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:12.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.579+0000 7f43be39d700 1 --2- 172.21.15.55:0/3158336074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43b8159420 0x7f43b80b5f40 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f43b8089750 tx=0x7f43b4009250 comp rx=0 tx=0).stop 2023-10-06T03:51:12.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.579+0000 7f43be39d700 1 -- 172.21.15.55:0/3158336074 shutdown_connections 2023-10-06T03:51:12.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.579+0000 7f43be39d700 1 --2- 172.21.15.55:0/3158336074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43b8159420 0x7f43b80b5f40 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.579+0000 7f43be39d700 1 --2- 172.21.15.55:0/3158336074 >> v2:172.21.15.163:3300/0 conn(0x7f43b81597f0 0x7f43b80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.579+0000 7f43be39d700 1 -- 172.21.15.55:0/3158336074 >> 172.21.15.55:0/3158336074 conn(0x7f43b80d0660 msgr2=0x7f43b80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:12.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.579+0000 7f43be39d700 1 -- 172.21.15.55:0/3158336074 shutdown_connections 2023-10-06T03:51:12.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.579+0000 7f43be39d700 1 -- 172.21.15.55:0/3158336074 wait complete. 2023-10-06T03:51:12.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.581+0000 7f43be39d700 1 Processor -- start 2023-10-06T03:51:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.581+0000 7f43be39d700 1 -- start start 2023-10-06T03:51:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.582+0000 7f43be39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b8159420 0x7f43b8156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.582+0000 7f43be39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43b81597f0 0x7f43b80ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.582+0000 7f43be39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43b815d230 0x7f43b80c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.582+0000 7f43be39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80c7bd0 con 0x7f43b81597f0 2023-10-06T03:51:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.582+0000 7f43be39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b816d580 con 0x7f43b815d230 2023-10-06T03:51:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.582+0000 7f43be39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80c6870 con 0x7f43b8159420 2023-10-06T03:51:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.582+0000 7f43bcb9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43b81597f0 0x7f43b80ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.582+0000 7f43bcb9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43b81597f0 0x7f43b80ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54406/0 (socket says 172.21.15.55:54406) 2023-10-06T03:51:12.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.582+0000 7f43bcb9a700 1 -- 172.21.15.55:0/3185379163 learned_addr learned my addr 172.21.15.55:0/3185379163 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.582+0000 7f43bd39b700 1 --2- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b8159420 0x7f43b8156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.582+0000 7f43bdb9c700 1 --2- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43b815d230 0x7f43b80c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.583+0000 7f43bcb9a700 1 -- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b8159420 msgr2=0x7f43b8156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.583+0000 7f43bcb9a700 1 --2- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b8159420 0x7f43b8156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.583+0000 7f43bcb9a700 1 -- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43b815d230 msgr2=0x7f43b80c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.583+0000 7f43bcb9a700 1 --2- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43b815d230 0x7f43b80c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.583+0000 7f43bcb9a700 1 -- 172.21.15.55:0/3185379163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b40094f0 con 0x7f43b81597f0 2023-10-06T03:51:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.583+0000 7f43bcb9a700 1 --2- 172.21.15.55:0/3185379163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43b81597f0 0x7f43b80ca120 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f43a800ec50 tx=0x7f43a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:12.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.583+0000 7f43ad7fa700 1 -- 172.21.15.55:0/3185379163 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800c850 con 0x7f43b81597f0 2023-10-06T03:51:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.583+0000 7f43ad7fa700 1 -- 172.21.15.55:0/3185379163 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a800c9b0 con 0x7f43b81597f0 2023-10-06T03:51:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.583+0000 7f43ad7fa700 1 -- 172.21.15.55:0/3185379163 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800cd70 con 0x7f43b81597f0 2023-10-06T03:51:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.583+0000 7f43be39d700 1 -- 172.21.15.55:0/3185379163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b80c6b50 con 0x7f43b81597f0 2023-10-06T03:51:12.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.583+0000 7f43be39d700 1 -- 172.21.15.55:0/3185379163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b80c7860 con 0x7f43b81597f0 2023-10-06T03:51:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.585+0000 7f43be39d700 1 -- 172.21.15.55:0/3185379163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b80c7d80 con 0x7f43b81597f0 2023-10-06T03:51:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.586+0000 7f43ad7fa700 1 -- 172.21.15.55:0/3185379163 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 46) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f43a8013680 con 0x7f43b81597f0 2023-10-06T03:51:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.587+0000 7f43ad7fa700 1 --2- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f439c067170 0x7f439c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.587+0000 7f43ad7fa700 1 -- 172.21.15.55:0/3185379163 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6824+0+0 (secure 0 0 0) 0x7f43a8084f80 con 0x7f43b81597f0 2023-10-06T03:51:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.587+0000 7f43bd39b700 1 -- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f439c067170 msgr2=0x7f439c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:12.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.587+0000 7f43bd39b700 1 --2- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f439c067170 0x7f439c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:12.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.591+0000 7f43ad7fa700 1 -- 172.21.15.55:0/3185379163 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43b80c7d80 con 0x7f43b81597f0 2023-10-06T03:51:12.619 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:12.617+0000 7f3612271180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:51:12.705 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:12.704+0000 7f42572c8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:51:12.783 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:12.782+0000 7f3612271180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:51:12.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.788+0000 7f43bd39b700 1 -- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f439c067170 msgr2=0x7f439c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:12.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.788+0000 7f43bd39b700 1 --2- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f439c067170 0x7f439c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:12.799 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:12.798+0000 7f42572c8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:51:12.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.840+0000 7f43be39d700 1 -- 172.21.15.55:0/3185379163 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f43b804c3c0 con 0x7f43b81597f0 2023-10-06T03:51:12.842 INFO:tasks.ceph.mon.a.smithi055.stderr:2023-10-06T03:51:12.841+0000 7fe524ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:51:12.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.846+0000 7f43ad7fa700 1 -- 172.21.15.55:0/3185379163 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7f43a8051a60 con 0x7f43b81597f0 2023-10-06T03:51:12.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.850+0000 7f43be39d700 1 -- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f439c067170 msgr2=0x7f439c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:12.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.850+0000 7f43be39d700 1 --2- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f439c067170 0x7f439c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.850+0000 7f43be39d700 1 -- 172.21.15.55:0/3185379163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43b81597f0 msgr2=0x7f43b80ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:12.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.850+0000 7f43be39d700 1 --2- 172.21.15.55:0/3185379163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43b81597f0 0x7f43b80ca120 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f43a800ec50 tx=0x7f43a80091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.850+0000 7f43be39d700 1 -- 172.21.15.55:0/3185379163 shutdown_connections 2023-10-06T03:51:12.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.850+0000 7f43be39d700 1 --2- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f439c067170 0x7f439c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.850+0000 7f43be39d700 1 --2- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43b815d230 0x7f43b80c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.852 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.850+0000 7f43be39d700 1 --2- 172.21.15.55:0/3185379163 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43b8159420 0x7f43b8156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.850+0000 7f43be39d700 1 --2- 172.21.15.55:0/3185379163 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43b81597f0 0x7f43b80ca120 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:12.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.850+0000 7f43be39d700 1 -- 172.21.15.55:0/3185379163 >> 172.21.15.55:0/3185379163 conn(0x7f43b80d0660 msgr2=0x7f43b80cfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:12.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.850+0000 7f43be39d700 1 -- 172.21.15.55:0/3185379163 shutdown_connections 2023-10-06T03:51:12.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:12.850+0000 7f43be39d700 1 -- 172.21.15.55:0/3185379163 wait complete. 2023-10-06T03:51:12.853 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:51:12.859 INFO:tasks.ceph.mds.a.smithi055.stderr: -5341> 2023-10-06T03:49:51.734+0000 7f702e7f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:51:12.872 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:51:12.893 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:12.892+0000 7f42572c8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:51:12.896 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:12.895+0000 7f3612271180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:51:12.907 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T03:51:12.989 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:12.988+0000 7f42572c8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:51:13.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.072+0000 7ff7d3fff700 1 Processor -- start 2023-10-06T03:51:13.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.072+0000 7ff7d3fff700 1 -- start start 2023-10-06T03:51:13.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.073+0000 7ff7d3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7d4157a50 0x7ff7d40c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:13.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.073+0000 7ff7d3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff7d40cb590 0x7ff7d40c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:13.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.074+0000 7ff7d3fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d40cd240 con 0x7ff7d4157680 2023-10-06T03:51:13.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.074+0000 7ff7d3fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d40cc700 con 0x7ff7d40cb590 2023-10-06T03:51:13.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.074+0000 7ff7d3fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40c9a20 con 0x7ff7d4157a50 2023-10-06T03:51:13.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.074+0000 7ff7d2ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7ff7d4157680 0x7ff7d40cae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:53306/0 (socket says 172.21.15.55:53306) 2023-10-06T03:51:13.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.074+0000 7ff7d2ffd700 1 -- 172.21.15.55:0/2564899891 learned_addr learned my addr 172.21.15.55:0/2564899891 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:13.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.074+0000 7ff7d27fc700 1 --2- 172.21.15.55:0/2564899891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7d4157a50 0x7ff7d40c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:13.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.074+0000 7ff7d37fe700 1 --2- 172.21.15.55:0/2564899891 >> v2:172.21.15.55:3300/0 conn(0x7ff7d40cb590 0x7ff7d40c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:13.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.074+0000 7ff7d37fe700 1 -- 172.21.15.55:0/2564899891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7d4157a50 msgr2=0x7ff7d40c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:13.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.074+0000 7ff7d37fe700 1 --2- 172.21.15.55:0/2564899891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7d4157a50 0x7ff7d40c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:13.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.074+0000 7ff7d1ffb700 1 -- 172.21.15.55:0/2564899891 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1013296248 0 0) 0x7ff7d40cd240 con 0x7ff7d4157680 2023-10-06T03:51:13.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.074+0000 7ff7d37fe700 1 -- 172.21.15.55:0/2564899891 >> v1:172.21.15.163:6789/0 conn(0x7ff7d4157680 legacy=0x7ff7d40cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:13.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.074+0000 7ff7d37fe700 1 -- 172.21.15.55:0/2564899891 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d40c9750 con 0x7ff7d40cb590 2023-10-06T03:51:13.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.075+0000 7ff7d37fe700 1 --2- 172.21.15.55:0/2564899891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40cb590 0x7ff7d40c9210 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009e60 tx=0x7ff7cc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c57aeafefbde3c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:13.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.075+0000 7ff7d1ffb700 1 -- 172.21.15.55:0/2564899891 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc00e070 con 0x7ff7d40cb590 2023-10-06T03:51:13.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.075+0000 7ff7d1ffb700 1 -- 172.21.15.55:0/2564899891 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc00bb60 con 0x7ff7d40cb590 2023-10-06T03:51:13.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.075+0000 7ff7d1ffb700 1 -- 172.21.15.55:0/2564899891 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc012430 con 0x7ff7d40cb590 2023-10-06T03:51:13.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.075+0000 7ff7d3fff700 1 -- 172.21.15.55:0/2564899891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40cb590 msgr2=0x7ff7d40c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:13.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.075+0000 7ff7d3fff700 1 --2- 172.21.15.55:0/2564899891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40cb590 0x7ff7d40c9210 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc009e60 tx=0x7ff7cc009780 comp rx=0 tx=0).stop 2023-10-06T03:51:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.076+0000 7ff7d3fff700 1 -- 172.21.15.55:0/2564899891 shutdown_connections 2023-10-06T03:51:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.076+0000 7ff7d3fff700 1 --2- 172.21.15.55:0/2564899891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40cb590 0x7ff7d40c9210 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.076+0000 7ff7d3fff700 1 --2- 172.21.15.55:0/2564899891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7d4157a50 0x7ff7d40c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.076+0000 7ff7d3fff700 1 -- 172.21.15.55:0/2564899891 >> 172.21.15.55:0/2564899891 conn(0x7ff7d406baa0 msgr2=0x7ff7d406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.076+0000 7ff7d3fff700 1 -- 172.21.15.55:0/2564899891 shutdown_connections 2023-10-06T03:51:13.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.076+0000 7ff7d3fff700 1 -- 172.21.15.55:0/2564899891 wait complete. 2023-10-06T03:51:13.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:13.076+0000 7f41d5689700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:13.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:13.076+0000 7f41d5689700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:13.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:13.076+0000 7f41d5689700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:13.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:13.076+0000 7f41d5689700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:13.082 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:13.076+0000 7f41d5689700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:13.082 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:13.076+0000 7f41d1e82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:13.082 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:13.076+0000 7f41d1e82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:13.082 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:13.076+0000 7f41d1e82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:13.082 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:13.076+0000 7f41d1e82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:13.082 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:13.076+0000 7f41d1e82700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:13.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.077+0000 7ff7d3fff700 1 Processor -- start 2023-10-06T03:51:13.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.078+0000 7ff7d3fff700 1 -- start start 2023-10-06T03:51:13.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.078+0000 7ff7d3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40cb590 0x7ff7d415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:13.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.078+0000 7ff7d3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7d4157680 0x7ff7d414c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:13.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7d4157a50 0x7ff7d40b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:13.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d3fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d4159960 con 0x7ff7d40cb590 2023-10-06T03:51:13.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d3fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d4158d50 con 0x7ff7d4157680 2023-10-06T03:51:13.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d3fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d4159030 con 0x7ff7d4157a50 2023-10-06T03:51:13.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40cb590 0x7ff7d415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:13.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40cb590 0x7ff7d415a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41080/0 (socket says 172.21.15.55:41080) 2023-10-06T03:51:13.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d2ffd700 1 -- 172.21.15.55:0/2679680241 learned_addr learned my addr 172.21.15.55:0/2679680241 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:13.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d27fc700 1 --2- 172.21.15.55:0/2679680241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7d4157680 0x7ff7d414c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:13.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d37fe700 1 --2- 172.21.15.55:0/2679680241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7d4157a50 0x7ff7d40b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:13.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d2ffd700 1 -- 172.21.15.55:0/2679680241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7d4157a50 msgr2=0x7ff7d40b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:13.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d2ffd700 1 --2- 172.21.15.55:0/2679680241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7d4157a50 0x7ff7d40b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:13.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d2ffd700 1 -- 172.21.15.55:0/2679680241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7d4157680 msgr2=0x7ff7d414c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:13.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d2ffd700 1 --2- 172.21.15.55:0/2679680241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7d4157680 0x7ff7d414c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:13.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d2ffd700 1 -- 172.21.15.55:0/2679680241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc009a20 con 0x7ff7d40cb590 2023-10-06T03:51:13.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d37fe700 1 --2- 172.21.15.55:0/2679680241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7d4157a50 0x7ff7d40b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:13.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d27fc700 1 --2- 172.21.15.55:0/2679680241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7d4157680 0x7ff7d414c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:13.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.079+0000 7ff7d2ffd700 1 --2- 172.21.15.55:0/2679680241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40cb590 0x7ff7d415a6b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400c810 tx=0x7ff7c4007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:13.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.080+0000 7ff7baffd700 1 -- 172.21.15.55:0/2679680241 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4005c00 con 0x7ff7d40cb590 2023-10-06T03:51:13.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.080+0000 7ff7baffd700 1 -- 172.21.15.55:0/2679680241 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c4005d60 con 0x7ff7d40cb590 2023-10-06T03:51:13.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.080+0000 7ff7d3fff700 1 -- 172.21.15.55:0/2679680241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d4159310 con 0x7ff7d40cb590 2023-10-06T03:51:13.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.080+0000 7ff7baffd700 1 -- 172.21.15.55:0/2679680241 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4007340 con 0x7ff7d40cb590 2023-10-06T03:51:13.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.080+0000 7ff7d3fff700 1 -- 172.21.15.55:0/2679680241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d415abf0 con 0x7ff7d40cb590 2023-10-06T03:51:13.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.081+0000 7ff7d3fff700 1 -- 172.21.15.55:0/2679680241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d406eba0 con 0x7ff7d40cb590 2023-10-06T03:51:13.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.082+0000 7ff7baffd700 1 -- 172.21.15.55:0/2679680241 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 47) v1 ==== 84451+0+0 (secure 0 0 0) 0x7ff7c401b400 con 0x7ff7d40cb590 2023-10-06T03:51:13.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.083+0000 7ff7baffd700 1 -- 172.21.15.55:0/2679680241 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7254+0+0 (secure 0 0 0) 0x7ff7c40841f0 con 0x7ff7d40cb590 2023-10-06T03:51:13.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.087+0000 7ff7baffd700 1 -- 172.21.15.55:0/2679680241 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7c40511a0 con 0x7ff7d40cb590 2023-10-06T03:51:13.112 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:13.110+0000 7f3612271180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:51:13.211 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:13.210+0000 7f3612271180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:51:13.307 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:13.306+0000 7f3612271180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:51:13.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.335+0000 7ff7d3fff700 1 -- 172.21.15.55:0/2679680241 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff7d41597b0 con 0x7ff7d40cb590 2023-10-06T03:51:13.405 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:13.404+0000 7f3612271180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:51:13.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.905+0000 7ff7baffd700 1 -- 172.21.15.55:0/2679680241 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff7c40547c0 con 0x7ff7d40cb590 2023-10-06T03:51:13.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.908+0000 7ff7d3fff700 1 -- 172.21.15.55:0/2679680241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40cb590 msgr2=0x7ff7d415a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:13.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.908+0000 7ff7d3fff700 1 --2- 172.21.15.55:0/2679680241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40cb590 0x7ff7d415a6b0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400c810 tx=0x7ff7c4007d90 comp rx=0 tx=0).stop 2023-10-06T03:51:13.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.908+0000 7ff7d3fff700 1 -- 172.21.15.55:0/2679680241 shutdown_connections 2023-10-06T03:51:13.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.908+0000 7ff7d3fff700 1 --2- 172.21.15.55:0/2679680241 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7d4157a50 0x7ff7d40b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:13.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.908+0000 7ff7d3fff700 1 --2- 172.21.15.55:0/2679680241 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7d40cb590 0x7ff7d415a6b0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:13.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.908+0000 7ff7d3fff700 1 --2- 172.21.15.55:0/2679680241 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7d4157680 0x7ff7d414c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:13.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.908+0000 7ff7d3fff700 1 -- 172.21.15.55:0/2679680241 >> 172.21.15.55:0/2679680241 conn(0x7ff7d406baa0 msgr2=0x7ff7d4154fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:13.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.908+0000 7ff7d3fff700 1 -- 172.21.15.55:0/2679680241 shutdown_connections 2023-10-06T03:51:13.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:13.908+0000 7ff7d3fff700 1 -- 172.21.15.55:0/2679680241 wait complete. 2023-10-06T03:51:13.928 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:51:14.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.163+0000 7f1f3e59d700 1 Processor -- start 2023-10-06T03:51:14.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.163+0000 7f1f3e59d700 1 -- start start 2023-10-06T03:51:14.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.165+0000 7f1f3e59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1f380cc8d0 0x7f1f3815b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:14.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.165+0000 7f1f3e59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1f3815b960 0x7f1f380ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:14.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.165+0000 7f1f3e59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f38157a50 con 0x7f1f3815b050 2023-10-06T03:51:14.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.165+0000 7f1f3e59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f38158f10 con 0x7f1f3815b960 2023-10-06T03:51:14.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.165+0000 7f1f3e59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f381591b0 con 0x7f1f380cc8d0 2023-10-06T03:51:14.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.165+0000 7f1f3dd9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1f3815b960 0x7f1f380ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:14.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.165+0000 7f1f3dd9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1f3815b960 0x7f1f380ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41092/0 (socket says 172.21.15.55:41092) 2023-10-06T03:51:14.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.165+0000 7f1f3dd9c700 1 -- 172.21.15.55:0/2528943138 learned_addr learned my addr 172.21.15.55:0/2528943138 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:14.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.165+0000 7f1f3cd9a700 1 --2- 172.21.15.55:0/2528943138 >> v2:172.21.15.163:3300/0 conn(0x7f1f380cc8d0 0x7f1f3815b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:14.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.165+0000 7f1f3dd9c700 1 -- 172.21.15.55:0/2528943138 >> v2:172.21.15.163:3300/0 conn(0x7f1f380cc8d0 msgr2=0x7f1f3815b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:14.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.165+0000 7f1f3dd9c700 1 --2- 172.21.15.55:0/2528943138 >> v2:172.21.15.163:3300/0 conn(0x7f1f380cc8d0 0x7f1f3815b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:14.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.165+0000 7f1f3dd9c700 1 -- 172.21.15.55:0/2528943138 >> v1:172.21.15.163:6789/0 conn(0x7f1f3815b050 legacy=0x7f1f380ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:14.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.166+0000 7f1f3dd9c700 1 -- 172.21.15.55:0/2528943138 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f380ca9e0 con 0x7f1f3815b960 2023-10-06T03:51:14.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.166+0000 7f1f3dd9c700 1 --2- 172.21.15.55:0/2528943138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f3815b960 0x7f1f380ada30 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1f3806fdd0 tx=0x7f1f30009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648eff51bdfe3173 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:14.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.166+0000 7f1f2ffff700 1 -- 172.21.15.55:0/2528943138 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3000e070 con 0x7f1f3815b960 2023-10-06T03:51:14.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.166+0000 7f1f2ffff700 1 -- 172.21.15.55:0/2528943138 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3000ba20 con 0x7f1f3815b960 2023-10-06T03:51:14.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.166+0000 7f1f2ffff700 1 -- 172.21.15.55:0/2528943138 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f300092e0 con 0x7f1f3815b960 2023-10-06T03:51:14.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.167+0000 7f1f3e59d700 1 -- 172.21.15.55:0/2528943138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f3815b960 msgr2=0x7f1f380ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:14.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.167+0000 7f1f3e59d700 1 --2- 172.21.15.55:0/2528943138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f3815b960 0x7f1f380ada30 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1f3806fdd0 tx=0x7f1f30009840 comp rx=0 tx=0).stop 2023-10-06T03:51:14.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.167+0000 7f1f3e59d700 1 -- 172.21.15.55:0/2528943138 shutdown_connections 2023-10-06T03:51:14.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.167+0000 7f1f3e59d700 1 --2- 172.21.15.55:0/2528943138 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f3815b960 0x7f1f380ada30 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:14.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.167+0000 7f1f3e59d700 1 --2- 172.21.15.55:0/2528943138 >> v2:172.21.15.163:3300/0 conn(0x7f1f380cc8d0 0x7f1f3815b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:14.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.167+0000 7f1f3e59d700 1 -- 172.21.15.55:0/2528943138 >> 172.21.15.55:0/2528943138 conn(0x7f1f38068e20 msgr2=0x7f1f380cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:14.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.167+0000 7f1f3e59d700 1 -- 172.21.15.55:0/2528943138 shutdown_connections 2023-10-06T03:51:14.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.167+0000 7f1f3e59d700 1 -- 172.21.15.55:0/2528943138 wait complete. 2023-10-06T03:51:14.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.168+0000 7f1f3e59d700 1 Processor -- start 2023-10-06T03:51:14.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.168+0000 7f1f3e59d700 1 -- start start 2023-10-06T03:51:14.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.169+0000 7f1f3e59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f380cc8d0 0x7f1f380b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:14.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.169+0000 7f1f3e59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f3815b050 0x7f1f3814f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:14.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.169+0000 7f1f3e59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f3815b960 0x7f1f3816ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:14.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.169+0000 7f1f3e59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380b1220 con 0x7f1f3815b050 2023-10-06T03:51:14.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.169+0000 7f1f3e59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f38152e70 con 0x7f1f3815b960 2023-10-06T03:51:14.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.169+0000 7f1f3e59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380b0c60 con 0x7f1f380cc8d0 2023-10-06T03:51:14.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.170+0000 7f1f3cd9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f3815b050 0x7f1f3814f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:14.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.170+0000 7f1f3cd9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f3815b050 0x7f1f3814f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41108/0 (socket says 172.21.15.55:41108) 2023-10-06T03:51:14.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.170+0000 7f1f3cd9a700 1 -- 172.21.15.55:0/193698159 learned_addr learned my addr 172.21.15.55:0/193698159 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:14.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.170+0000 7f1f3d59b700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f380cc8d0 0x7f1f380b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:14.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.170+0000 7f1f3dd9c700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f3815b960 0x7f1f3816ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:14.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.170+0000 7f1f3cd9a700 1 -- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f380cc8d0 msgr2=0x7f1f380b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:14.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.170+0000 7f1f3cd9a700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f380cc8d0 0x7f1f380b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:14.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.170+0000 7f1f3cd9a700 1 -- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f3815b960 msgr2=0x7f1f3816ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:14.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.170+0000 7f1f3cd9a700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f3815b960 0x7f1f3816ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:14.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.170+0000 7f1f3cd9a700 1 -- 172.21.15.55:0/193698159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f30009ae0 con 0x7f1f3815b050 2023-10-06T03:51:14.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.170+0000 7f1f3dd9c700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f3815b960 0x7f1f3816ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:51:14.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.170+0000 7f1f3d59b700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f380cc8d0 0x7f1f380b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:51:14.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.170+0000 7f1f3cd9a700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f3815b050 0x7f1f3814f200 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800e930 tx=0x7f1f280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:14.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.171+0000 7f1f2d7fa700 1 -- 172.21.15.55:0/193698159 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800eed0 con 0x7f1f3815b050 2023-10-06T03:51:14.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.171+0000 7f1f2d7fa700 1 -- 172.21.15.55:0/193698159 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2800c850 con 0x7f1f3815b050 2023-10-06T03:51:14.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.171+0000 7f1f2d7fa700 1 -- 172.21.15.55:0/193698159 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800cbf0 con 0x7f1f3815b050 2023-10-06T03:51:14.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.171+0000 7f1f3e59d700 1 -- 172.21.15.55:0/193698159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f380b0f40 con 0x7f1f3815b050 2023-10-06T03:51:14.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.171+0000 7f1f3e59d700 1 -- 172.21.15.55:0/193698159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3814f740 con 0x7f1f3815b050 2023-10-06T03:51:14.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.172+0000 7f1f3e59d700 1 -- 172.21.15.55:0/193698159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f38152fe0 con 0x7f1f3815b050 2023-10-06T03:51:14.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.173+0000 7f1f2d7fa700 1 -- 172.21.15.55:0/193698159 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 48) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f1f28013470 con 0x7f1f3815b050 2023-10-06T03:51:14.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.174+0000 7f1f2d7fa700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1f1c066d80 0x7f1f1c069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:14.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.174+0000 7f1f2d7fa700 1 -- 172.21.15.55:0/193698159 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 7254+0+0 (secure 0 0 0) 0x7f1f28084a20 con 0x7f1f3815b050 2023-10-06T03:51:14.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.174+0000 7f1f3d59b700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1f1c066d80 0x7f1f1c069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:14.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.175+0000 7f1f3d59b700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1f1c066d80 0x7f1f1c069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1f340017a0 tx=0x7f1f34009040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:14.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.179+0000 7f1f2d7fa700 1 -- 172.21.15.55:0/193698159 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f28051930 con 0x7f1f3815b050 2023-10-06T03:51:14.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:14.386+0000 7f1f3e59d700 1 -- 172.21.15.55:0/193698159 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1f38152cc0 con 0x7f1f3815b050 2023-10-06T03:51:15.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.009+0000 7f1f2d7fa700 1 -- 172.21.15.55:0/193698159 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v113) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1f28054f50 con 0x7f1f3815b050 2023-10-06T03:51:15.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.014+0000 7f1f3e59d700 1 -- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1f1c066d80 msgr2=0x7f1f1c069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:15.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.014+0000 7f1f3e59d700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1f1c066d80 0x7f1f1c069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1f340017a0 tx=0x7f1f34009040 comp rx=0 tx=0).stop 2023-10-06T03:51:15.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.014+0000 7f1f3e59d700 1 -- 172.21.15.55:0/193698159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f3815b050 msgr2=0x7f1f3814f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:15.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.014+0000 7f1f3e59d700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f3815b050 0x7f1f3814f200 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800e930 tx=0x7f1f280091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:15.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.014+0000 7f1f3e59d700 1 -- 172.21.15.55:0/193698159 shutdown_connections 2023-10-06T03:51:15.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.014+0000 7f1f3e59d700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1f1c066d80 0x7f1f1c069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:15.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.014+0000 7f1f3e59d700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f3815b960 0x7f1f3816ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:15.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.014+0000 7f1f3e59d700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f380cc8d0 0x7f1f380b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:15.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.014+0000 7f1f3e59d700 1 --2- 172.21.15.55:0/193698159 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f3815b050 0x7f1f3814f200 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:15.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.014+0000 7f1f3e59d700 1 -- 172.21.15.55:0/193698159 >> 172.21.15.55:0/193698159 conn(0x7f1f38068e20 msgr2=0x7f1f38152150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:15.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.014+0000 7f1f3e59d700 1 -- 172.21.15.55:0/193698159 shutdown_connections 2023-10-06T03:51:15.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.014+0000 7f1f3e59d700 1 -- 172.21.15.55:0/193698159 wait complete. 2023-10-06T03:51:15.017 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:51:15.035 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:51:15.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.228+0000 7f4abaf9d700 1 Processor -- start 2023-10-06T03:51:15.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.228+0000 7f4abaf9d700 1 -- start start 2023-10-06T03:51:15.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.229+0000 7f4abaf9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4ab40b0790 0x7f4ab40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:15.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.229+0000 7f4abaf9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab41591e0 0x7f4ab4159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:15.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.229+0000 7f4abaf9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab40cfa00 con 0x7f4ab40b03c0 2023-10-06T03:51:15.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.229+0000 7f4abaf9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab40cb580 con 0x7f4ab40b0790 2023-10-06T03:51:15.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.229+0000 7f4abaf9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab40cf160 con 0x7f4ab41591e0 2023-10-06T03:51:15.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.229+0000 7f4ab979a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4ab40b0790 0x7f4ab40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:15.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.230+0000 7f4ab979a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4ab40b0790 0x7f4ab40cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41116/0 (socket says 172.21.15.55:41116) 2023-10-06T03:51:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.230+0000 7f4ab979a700 1 -- 172.21.15.55:0/3665757610 learned_addr learned my addr 172.21.15.55:0/3665757610 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.230+0000 7f4ab9f9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4ab40b03c0 0x7f4ab40b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57312/0 (socket says 172.21.15.55:57312) 2023-10-06T03:51:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.230+0000 7f4aba79c700 1 --2- 172.21.15.55:0/3665757610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab41591e0 0x7f4ab4159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.230+0000 7f4ab979a700 1 -- 172.21.15.55:0/3665757610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab41591e0 msgr2=0x7f4ab4159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.230+0000 7f4ab979a700 1 --2- 172.21.15.55:0/3665757610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab41591e0 0x7f4ab4159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:15.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.230+0000 7f4ab979a700 1 -- 172.21.15.55:0/3665757610 >> v1:172.21.15.55:6789/0 conn(0x7f4ab40b03c0 legacy=0x7f4ab40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:15.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.230+0000 7f4ab979a700 1 -- 172.21.15.55:0/3665757610 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab40cf440 con 0x7f4ab40b0790 2023-10-06T03:51:15.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.230+0000 7f4ab979a700 1 --2- 172.21.15.55:0/3665757610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab40b0790 0x7f4ab40cea70 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40099e0 tx=0x7f4aa4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3cf7ad485dbd38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:15.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.230+0000 7f4ab8f99700 1 -- 172.21.15.55:0/3665757610 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa400b820 con 0x7f4ab40b0790 2023-10-06T03:51:15.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.231+0000 7f4ab8f99700 1 -- 172.21.15.55:0/3665757610 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa400b980 con 0x7f4ab40b0790 2023-10-06T03:51:15.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.231+0000 7f4ab8f99700 1 -- 172.21.15.55:0/3665757610 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa400bd20 con 0x7f4ab40b0790 2023-10-06T03:51:15.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.231+0000 7f4abaf9d700 1 -- 172.21.15.55:0/3665757610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab40b0790 msgr2=0x7f4ab40cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:15.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.231+0000 7f4abaf9d700 1 --2- 172.21.15.55:0/3665757610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab40b0790 0x7f4ab40cea70 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40099e0 tx=0x7f4aa4009250 comp rx=0 tx=0).stop 2023-10-06T03:51:15.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.231+0000 7f4abaf9d700 1 -- 172.21.15.55:0/3665757610 shutdown_connections 2023-10-06T03:51:15.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.231+0000 7f4abaf9d700 1 --2- 172.21.15.55:0/3665757610 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab41591e0 0x7f4ab4159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:15.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.231+0000 7f4abaf9d700 1 --2- 172.21.15.55:0/3665757610 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab40b0790 0x7f4ab40cea70 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:15.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.231+0000 7f4abaf9d700 1 -- 172.21.15.55:0/3665757610 >> 172.21.15.55:0/3665757610 conn(0x7f4ab4063230 msgr2=0x7f4ab4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:15.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.231+0000 7f4abaf9d700 1 -- 172.21.15.55:0/3665757610 shutdown_connections 2023-10-06T03:51:15.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.231+0000 7f4abaf9d700 1 -- 172.21.15.55:0/3665757610 wait complete. 2023-10-06T03:51:15.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.233+0000 7f4abaf9d700 1 Processor -- start 2023-10-06T03:51:15.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.233+0000 7f4abaf9d700 1 -- start start 2023-10-06T03:51:15.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.234+0000 7f4abaf9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab40b03c0 0x7f4ab40ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.234+0000 7f4abaf9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab40b0790 0x7f4ab416d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.234+0000 7f4abaf9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ab41591e0 0x7f4ab416d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.234+0000 7f4abaf9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab40c9f30 con 0x7f4ab40b0790 2023-10-06T03:51:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.234+0000 7f4abaf9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab40c9320 con 0x7f4ab41591e0 2023-10-06T03:51:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.234+0000 7f4abaf9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab40c9600 con 0x7f4ab40b03c0 2023-10-06T03:51:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.234+0000 7f4ab979a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab40b0790 0x7f4ab416d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.234+0000 7f4ab9f9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab40b03c0 0x7f4ab40ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:15.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.234+0000 7f4ab979a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab40b0790 0x7f4ab416d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41130/0 (socket says 172.21.15.55:41130) 2023-10-06T03:51:15.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.234+0000 7f4ab979a700 1 -- 172.21.15.55:0/1463051440 learned_addr learned my addr 172.21.15.55:0/1463051440 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:15.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.234+0000 7f4aba79c700 1 --2- 172.21.15.55:0/1463051440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ab41591e0 0x7f4ab416d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:15.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.235+0000 7f4ab979a700 1 -- 172.21.15.55:0/1463051440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab40b03c0 msgr2=0x7f4ab40ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:15.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.235+0000 7f4ab979a700 1 --2- 172.21.15.55:0/1463051440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab40b03c0 0x7f4ab40ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:15.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.235+0000 7f4ab979a700 1 -- 172.21.15.55:0/1463051440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ab41591e0 msgr2=0x7f4ab416d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:15.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.235+0000 7f4ab979a700 1 --2- 172.21.15.55:0/1463051440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ab41591e0 0x7f4ab416d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:15.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.235+0000 7f4ab979a700 1 -- 172.21.15.55:0/1463051440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa40094f0 con 0x7f4ab40b0790 2023-10-06T03:51:15.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.235+0000 7f4ab979a700 1 --2- 172.21.15.55:0/1463051440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab40b0790 0x7f4ab416d2d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009290 tx=0x7f4aa4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.235+0000 7f4aa9ffb700 1 -- 172.21.15.55:0/1463051440 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4011d10 con 0x7f4ab40b0790 2023-10-06T03:51:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.235+0000 7f4aa9ffb700 1 -- 172.21.15.55:0/1463051440 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa4011e90 con 0x7f4ab40b0790 2023-10-06T03:51:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.235+0000 7f4aa9ffb700 1 -- 172.21.15.55:0/1463051440 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa40266a0 con 0x7f4ab40b0790 2023-10-06T03:51:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.235+0000 7f4abaf9d700 1 -- 172.21.15.55:0/1463051440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab40c98e0 con 0x7f4ab40b0790 2023-10-06T03:51:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.236+0000 7f4abaf9d700 1 -- 172.21.15.55:0/1463051440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab416cf80 con 0x7f4ab40b0790 2023-10-06T03:51:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.237+0000 7f4abaf9d700 1 -- 172.21.15.55:0/1463051440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab404c3c0 con 0x7f4ab40b0790 2023-10-06T03:51:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.237+0000 7f4aa9ffb700 1 -- 172.21.15.55:0/1463051440 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 48) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f4aa401a420 con 0x7f4ab40b0790 2023-10-06T03:51:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.238+0000 7f4aa9ffb700 1 --2- 172.21.15.55:0/1463051440 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4a98066d80 0x7f4a98069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.238+0000 7f4aa9ffb700 1 -- 172.21.15.55:0/1463051440 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6816+0+0 (secure 0 0 0) 0x7f4aa40861f0 con 0x7f4ab40b0790 2023-10-06T03:51:15.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.239+0000 7f4ab9f9b700 1 --2- 172.21.15.55:0/1463051440 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4a98066d80 0x7f4a98069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:15.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.239+0000 7f4ab9f9b700 1 --2- 172.21.15.55:0/1463051440 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4a98066d80 0x7f4a98069230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0004fd0 tx=0x7f4ab0007eb0 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:15.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.243+0000 7f4aa9ffb700 1 -- 172.21.15.55:0/1463051440 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4aa4053290 con 0x7f4ab40b0790 2023-10-06T03:51:15.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.445+0000 7f4abaf9d700 1 -- 172.21.15.55:0/1463051440 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4ab416ade0 con 0x7f4ab40b0790 2023-10-06T03:51:15.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:15.917+0000 7f4aa9ffb700 1 -- 172.21.15.55:0/1463051440 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4aa4053660 con 0x7f4ab40b0790 2023-10-06T03:51:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.014+0000 7f4aa9ffb700 1 -- 172.21.15.55:0/1463051440 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v114) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4aa4053290 con 0x7f4ab40b0790 2023-10-06T03:51:16.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.018+0000 7f4abaf9d700 1 -- 172.21.15.55:0/1463051440 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4a98066d80 msgr2=0x7f4a98069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.018+0000 7f4abaf9d700 1 --2- 172.21.15.55:0/1463051440 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4a98066d80 0x7f4a98069230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0004fd0 tx=0x7f4ab0007eb0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.018+0000 7f4abaf9d700 1 -- 172.21.15.55:0/1463051440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab40b0790 msgr2=0x7f4ab416d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.018+0000 7f4abaf9d700 1 --2- 172.21.15.55:0/1463051440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab40b0790 0x7f4ab416d2d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4009290 tx=0x7f4aa4017000 comp rx=0 tx=0).stop 2023-10-06T03:51:16.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.018+0000 7f4abaf9d700 1 -- 172.21.15.55:0/1463051440 shutdown_connections 2023-10-06T03:51:16.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.018+0000 7f4abaf9d700 1 --2- 172.21.15.55:0/1463051440 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4a98066d80 0x7f4a98069230 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.018+0000 7f4abaf9d700 1 --2- 172.21.15.55:0/1463051440 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4ab41591e0 0x7f4ab416d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.018+0000 7f4abaf9d700 1 --2- 172.21.15.55:0/1463051440 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4ab40b03c0 0x7f4ab40ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.018+0000 7f4abaf9d700 1 --2- 172.21.15.55:0/1463051440 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4ab40b0790 0x7f4ab416d2d0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.018+0000 7f4abaf9d700 1 -- 172.21.15.55:0/1463051440 >> 172.21.15.55:0/1463051440 conn(0x7f4ab4063230 msgr2=0x7f4ab4068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:16.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.019+0000 7f4abaf9d700 1 -- 172.21.15.55:0/1463051440 shutdown_connections 2023-10-06T03:51:16.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.019+0000 7f4abaf9d700 1 -- 172.21.15.55:0/1463051440 wait complete. 2023-10-06T03:51:16.023 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2023-10-06T03:51:16.040 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:51:16.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.241+0000 7fc92a59d700 1 Processor -- start 2023-10-06T03:51:16.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.241+0000 7fc92a59d700 1 -- start start 2023-10-06T03:51:16.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.243+0000 7fc92a59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc924153900 0x7fc9240c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:16.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.243+0000 7fc92a59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc92406a710 0x7fc92406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:16.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.243+0000 7fc92a59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92406e150 con 0x7fc9240c9db0 2023-10-06T03:51:16.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.243+0000 7fc92a59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9240c7800 con 0x7fc924153900 2023-10-06T03:51:16.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.243+0000 7fc92a59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9240c7ae0 con 0x7fc92406a710 2023-10-06T03:51:16.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.243+0000 7fc92959b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc924153900 0x7fc9240c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:16.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.243+0000 7fc92959b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc924153900 0x7fc9240c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41132/0 (socket says 172.21.15.55:41132) 2023-10-06T03:51:16.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.243+0000 7fc92959b700 1 -- 172.21.15.55:0/142283135 learned_addr learned my addr 172.21.15.55:0/142283135 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:16.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.243+0000 7fc928d9a700 1 --1- 172.21.15.55:0/142283135 >> v1:172.21.15.55:6789/0 conn(0x7fc9240c9db0 0x7fc92406a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57326/0 (socket says 172.21.15.55:57326) 2023-10-06T03:51:16.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.243+0000 7fc929d9c700 1 --2- 172.21.15.55:0/142283135 >> v2:172.21.15.163:3300/0 conn(0x7fc92406a710 0x7fc92406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:16.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.244+0000 7fc92959b700 1 -- 172.21.15.55:0/142283135 >> v2:172.21.15.163:3300/0 conn(0x7fc92406a710 msgr2=0x7fc92406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:16.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.244+0000 7fc92959b700 1 --2- 172.21.15.55:0/142283135 >> v2:172.21.15.163:3300/0 conn(0x7fc92406a710 0x7fc92406e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.244+0000 7fc92959b700 1 -- 172.21.15.55:0/142283135 >> v1:172.21.15.55:6789/0 conn(0x7fc9240c9db0 legacy=0x7fc92406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.244+0000 7fc92959b700 1 -- 172.21.15.55:0/142283135 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9240c7dc0 con 0x7fc924153900 2023-10-06T03:51:16.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.244+0000 7fc91bfff700 1 -- 172.21.15.55:0/142283135 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481306361 0 0) 0x7fc92406e150 con 0x7fc9240c9db0 2023-10-06T03:51:16.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.244+0000 7fc929d9c700 1 --2- 172.21.15.55:0/142283135 >> v2:172.21.15.163:3300/0 conn(0x7fc92406a710 0x7fc92406e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:51:16.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.244+0000 7fc92959b700 1 --2- 172.21.15.55:0/142283135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc924153900 0x7fc9240c6f00 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc9240cd2d0 tx=0x7fc920009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a4cf2c97aa0d67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:16.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.244+0000 7fc91bfff700 1 -- 172.21.15.55:0/142283135 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92000b860 con 0x7fc924153900 2023-10-06T03:51:16.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.244+0000 7fc91bfff700 1 -- 172.21.15.55:0/142283135 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92000b9c0 con 0x7fc924153900 2023-10-06T03:51:16.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.244+0000 7fc91bfff700 1 -- 172.21.15.55:0/142283135 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92000bd60 con 0x7fc924153900 2023-10-06T03:51:16.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.245+0000 7fc92a59d700 1 -- 172.21.15.55:0/142283135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc924153900 msgr2=0x7fc9240c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.245+0000 7fc92a59d700 1 --2- 172.21.15.55:0/142283135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc924153900 0x7fc9240c6f00 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc9240cd2d0 tx=0x7fc920009250 comp rx=0 tx=0).stop 2023-10-06T03:51:16.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.245+0000 7fc92a59d700 1 -- 172.21.15.55:0/142283135 shutdown_connections 2023-10-06T03:51:16.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.245+0000 7fc92a59d700 1 --2- 172.21.15.55:0/142283135 >> v2:172.21.15.163:3300/0 conn(0x7fc92406a710 0x7fc92406e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.245+0000 7fc92a59d700 1 --2- 172.21.15.55:0/142283135 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc924153900 0x7fc9240c6f00 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.245+0000 7fc92a59d700 1 -- 172.21.15.55:0/142283135 >> 172.21.15.55:0/142283135 conn(0x7fc9240cc1e0 msgr2=0x7fc9240cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:16.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.245+0000 7fc92a59d700 1 -- 172.21.15.55:0/142283135 shutdown_connections 2023-10-06T03:51:16.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.245+0000 7fc92a59d700 1 -- 172.21.15.55:0/142283135 wait complete. 2023-10-06T03:51:16.249 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:51:16.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.246+0000 7fc92a59d700 1 Processor -- start 2023-10-06T03:51:16.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.246+0000 7fc92a59d700 1 -- start start 2023-10-06T03:51:16.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.247+0000 7fc92a59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc92406a710 0x7fc92413e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:16.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.247+0000 7fc92a59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9240c9db0 0x7fc924158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:16.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.247+0000 7fc92a59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc924153900 0x7fc9240aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:16.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.247+0000 7fc92a59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9240ad1d0 con 0x7fc9240c9db0 2023-10-06T03:51:16.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.247+0000 7fc92a59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92413ef20 con 0x7fc92406a710 2023-10-06T03:51:16.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.247+0000 7fc92a59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9240ac5c0 con 0x7fc924153900 2023-10-06T03:51:16.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.248+0000 7fc928d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9240c9db0 0x7fc924158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:16.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.248+0000 7fc928d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9240c9db0 0x7fc924158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41138/0 (socket says 172.21.15.55:41138) 2023-10-06T03:51:16.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.248+0000 7fc928d9a700 1 -- 172.21.15.55:0/519100959 learned_addr learned my addr 172.21.15.55:0/519100959 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:16.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.248+0000 7fc92959b700 1 --2- 172.21.15.55:0/519100959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc92406a710 0x7fc92413e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:16.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.248+0000 7fc929d9c700 1 --2- 172.21.15.55:0/519100959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc924153900 0x7fc9240aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:16.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.248+0000 7fc928d9a700 1 -- 172.21.15.55:0/519100959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc924153900 msgr2=0x7fc9240aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.248+0000 7fc928d9a700 1 --2- 172.21.15.55:0/519100959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc924153900 0x7fc9240aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.248+0000 7fc928d9a700 1 -- 172.21.15.55:0/519100959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc92406a710 msgr2=0x7fc92413e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.248+0000 7fc928d9a700 1 --2- 172.21.15.55:0/519100959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc92406a710 0x7fc92413e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.248+0000 7fc928d9a700 1 -- 172.21.15.55:0/519100959 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9200094f0 con 0x7fc9240c9db0 2023-10-06T03:51:16.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.248+0000 7fc928d9a700 1 --2- 172.21.15.55:0/519100959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9240c9db0 0x7fc924158620 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc91400c7e0 tx=0x7fc914007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:16.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.249+0000 7fc9197fa700 1 -- 172.21.15.55:0/519100959 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914005b40 con 0x7fc9240c9db0 2023-10-06T03:51:16.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.249+0000 7fc9197fa700 1 -- 172.21.15.55:0/519100959 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc914005cc0 con 0x7fc9240c9db0 2023-10-06T03:51:16.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.249+0000 7fc9197fa700 1 -- 172.21.15.55:0/519100959 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914007320 con 0x7fc9240c9db0 2023-10-06T03:51:16.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.249+0000 7fc92a59d700 1 -- 172.21.15.55:0/519100959 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9240ac8a0 con 0x7fc9240c9db0 2023-10-06T03:51:16.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.249+0000 7fc92a59d700 1 -- 172.21.15.55:0/519100959 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9240ace60 con 0x7fc9240c9db0 2023-10-06T03:51:16.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.250+0000 7fc92a59d700 1 -- 172.21.15.55:0/519100959 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9240ad380 con 0x7fc9240c9db0 2023-10-06T03:51:16.255 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:51:16.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.251+0000 7fc9197fa700 1 -- 172.21.15.55:0/519100959 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc91401b3b0 con 0x7fc9240c9db0 2023-10-06T03:51:16.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.252+0000 7fc9197fa700 1 --2- 172.21.15.55:0/519100959 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc908066f30 0x7fc9080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:16.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.252+0000 7fc9197fa700 1 -- 172.21.15.55:0/519100959 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6286+0+0 (secure 0 0 0) 0x7fc914083c90 con 0x7fc9240c9db0 2023-10-06T03:51:16.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.252+0000 7fc92959b700 1 --2- 172.21.15.55:0/519100959 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc908066f30 0x7fc9080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:16.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.253+0000 7fc92959b700 1 --2- 172.21.15.55:0/519100959 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc908066f30 0x7fc9080693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc9240cd2d0 tx=0x7fc920018000 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:16.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.256+0000 7fc9197fa700 1 -- 172.21.15.55:0/519100959 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9240ad380 con 0x7fc9240c9db0 2023-10-06T03:51:16.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.463+0000 7fc92a59d700 1 -- 172.21.15.55:0/519100959 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc9240ad380 con 0x7fc9240c9db0 2023-10-06T03:51:16.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.467+0000 7fc9197fa700 1 -- 172.21.15.55:0/519100959 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc9240ad380 con 0x7fc9240c9db0 2023-10-06T03:51:16.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.469+0000 7fc92a59d700 1 -- 172.21.15.55:0/519100959 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc908066f30 msgr2=0x7fc9080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.469+0000 7fc92a59d700 1 --2- 172.21.15.55:0/519100959 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc908066f30 0x7fc9080693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc9240cd2d0 tx=0x7fc920018000 comp rx=0 tx=0).stop 2023-10-06T03:51:16.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.469+0000 7fc92a59d700 1 -- 172.21.15.55:0/519100959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9240c9db0 msgr2=0x7fc924158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.469+0000 7fc92a59d700 1 --2- 172.21.15.55:0/519100959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9240c9db0 0x7fc924158620 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc91400c7e0 tx=0x7fc914007de0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.469+0000 7fc92a59d700 1 -- 172.21.15.55:0/519100959 shutdown_connections 2023-10-06T03:51:16.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.469+0000 7fc92a59d700 1 --2- 172.21.15.55:0/519100959 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc908066f30 0x7fc9080693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.469+0000 7fc92a59d700 1 --2- 172.21.15.55:0/519100959 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc924153900 0x7fc9240aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.469+0000 7fc92a59d700 1 --2- 172.21.15.55:0/519100959 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc92406a710 0x7fc92413e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.469+0000 7fc92a59d700 1 --2- 172.21.15.55:0/519100959 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9240c9db0 0x7fc924158620 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.470+0000 7fc92a59d700 1 -- 172.21.15.55:0/519100959 >> 172.21.15.55:0/519100959 conn(0x7fc9240cc1e0 msgr2=0x7fc92406e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:16.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.470+0000 7fc92a59d700 1 -- 172.21.15.55:0/519100959 shutdown_connections 2023-10-06T03:51:16.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.470+0000 7fc92a59d700 1 -- 172.21.15.55:0/519100959 wait complete. 2023-10-06T03:51:16.488 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T03:51:16.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.675+0000 7f0ab099d700 1 Processor -- start 2023-10-06T03:51:16.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.676+0000 7f0ab099d700 1 -- start start 2023-10-06T03:51:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.677+0000 7f0ab099d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0aac156490 0x7f0aac0ad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.677+0000 7f0ab099d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aac0afc80 con 0x7f0aac156860 2023-10-06T03:51:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.677+0000 7f0ab099d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aac0af090 con 0x7f0aac06af00 2023-10-06T03:51:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.677+0000 7f0ab099d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aac0af350 con 0x7f0aac156490 2023-10-06T03:51:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.677+0000 7f0aaaffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0aac156860 0x7f0aac06a7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57340/0 (socket says 172.21.15.55:57340) 2023-10-06T03:51:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.677+0000 7f0aaaffd700 1 -- 172.21.15.55:0/4045694943 learned_addr learned my addr 172.21.15.55:0/4045694943 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:16.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aab7fe700 1 --2- 172.21.15.55:0/4045694943 >> v2:172.21.15.163:3300/0 conn(0x7f0aac156490 0x7f0aac0ad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:16.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416925382 0 0) 0x7f0aac0afc80 con 0x7f0aac156860 2023-10-06T03:51:16.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a94003570 con 0x7f0aac156860 2023-10-06T03:51:16.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457977440 0 0) 0x7f0aac0af090 con 0x7f0aac06af00 2023-10-06T03:51:16.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aac0afc80 con 0x7f0aac06af00 2023-10-06T03:51:16.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 737605540 0 0) 0x7f0a94003570 con 0x7f0aac156860 2023-10-06T03:51:16.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 >> v2:172.21.15.163:3300/0 conn(0x7f0aac156490 msgr2=0x7f0aac0ad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:16.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 --2- 172.21.15.55:0/4045694943 >> v2:172.21.15.163:3300/0 conn(0x7f0aac156490 0x7f0aac0ad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 >> v1:172.21.15.163:6789/0 conn(0x7f0aac06af00 legacy=0x7f0aac160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac0af630 con 0x7f0aac156860 2023-10-06T03:51:16.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f0aa0001470 con 0x7f0aac156860 2023-10-06T03:51:16.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 >> v1:172.21.15.55:6789/0 conn(0x7f0aac156860 legacy=0x7f0aac06a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 --2- 172.21.15.55:0/4045694943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a94003bf0 0x7f0a94003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 --2- 172.21.15.55:0/4045694943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a94004810 0x7f0a94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 --2- 172.21.15.55:0/4045694943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a940051c0 0x7f0a9400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac0af350 con 0x7f0a94003bf0 2023-10-06T03:51:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9400c0b0 con 0x7f0a940051c0 2023-10-06T03:51:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9400c270 con 0x7f0a94004810 2023-10-06T03:51:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.678+0000 7f0aab7fe700 1 --2- 172.21.15.55:0/4045694943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a94003bf0 0x7f0a94003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.679+0000 7f0aaaffd700 1 --2- 172.21.15.55:0/4045694943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a94004810 0x7f0a94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.679+0000 7f0aabfff700 1 --2- 172.21.15.55:0/4045694943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a940051c0 0x7f0a9400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.679+0000 7f0aab7fe700 1 -- 172.21.15.55:0/4045694943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a94004810 msgr2=0x7f0a94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.679+0000 7f0aab7fe700 1 --2- 172.21.15.55:0/4045694943 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a94004810 0x7f0a94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.679+0000 7f0aab7fe700 1 -- 172.21.15.55:0/4045694943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a940051c0 msgr2=0x7f0a9400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.679+0000 7f0aab7fe700 1 --2- 172.21.15.55:0/4045694943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a940051c0 0x7f0a9400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.679+0000 7f0aab7fe700 1 -- 172.21.15.55:0/4045694943 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a94003570 con 0x7f0a94003bf0 2023-10-06T03:51:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.679+0000 7f0aabfff700 1 --2- 172.21.15.55:0/4045694943 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a940051c0 0x7f0a9400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.679+0000 7f0aabfff700 1 -- 172.21.15.55:0/4045694943 reap_dead start 2023-10-06T03:51:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.679+0000 7f0aab7fe700 1 --2- 172.21.15.55:0/4045694943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a94003bf0 0x7f0a94003fe0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009c00 tx=0x7f0a9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda05db7a053b303 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.679+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c00e710 con 0x7f0a94003bf0 2023-10-06T03:51:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.679+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c00e870 con 0x7f0a94003bf0 2023-10-06T03:51:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.679+0000 7f0aaa7fc700 1 -- 172.21.15.55:0/4045694943 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c00ec30 con 0x7f0a94003bf0 2023-10-06T03:51:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.680+0000 7f0ab099d700 1 -- 172.21.15.55:0/4045694943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a94003bf0 msgr2=0x7f0a94003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.680+0000 7f0ab099d700 1 --2- 172.21.15.55:0/4045694943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a94003bf0 0x7f0a94003fe0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009c00 tx=0x7f0a9c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.680+0000 7f0ab099d700 1 -- 172.21.15.55:0/4045694943 shutdown_connections 2023-10-06T03:51:16.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.680+0000 7f0ab099d700 1 --2- 172.21.15.55:0/4045694943 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a94003bf0 0x7f0a94003fe0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.680+0000 7f0ab099d700 1 -- 172.21.15.55:0/4045694943 >> 172.21.15.55:0/4045694943 conn(0x7f0aac068390 msgr2=0x7f0aac068790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:16.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.680+0000 7f0ab099d700 1 -- 172.21.15.55:0/4045694943 shutdown_connections 2023-10-06T03:51:16.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.680+0000 7f0ab099d700 1 -- 172.21.15.55:0/4045694943 wait complete. 2023-10-06T03:51:16.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.682+0000 7f0ab099d700 1 Processor -- start 2023-10-06T03:51:16.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.682+0000 7f0ab099d700 1 -- start start 2023-10-06T03:51:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.682+0000 7f0ab099d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a94003bf0 0x7f0aac16b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0ab099d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac0c7400 0x7f0aac0c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0ab099d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aac154800 0x7f0aac154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0ab099d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac13e7c0 con 0x7f0a94003bf0 2023-10-06T03:51:16.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0ab099d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac0c6640 con 0x7f0aac154800 2023-10-06T03:51:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0ab099d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac0c6920 con 0x7f0aac0c7400 2023-10-06T03:51:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0aab7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a94003bf0 0x7f0aac16b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0aab7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a94003bf0 0x7f0aac16b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41162/0 (socket says 172.21.15.55:41162) 2023-10-06T03:51:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0aab7fe700 1 -- 172.21.15.55:0/998843524 learned_addr learned my addr 172.21.15.55:0/998843524 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0aaaffd700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac0c7400 0x7f0aac0c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0aab7fe700 1 -- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac0c7400 msgr2=0x7f0aac0c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0aabfff700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aac154800 0x7f0aac154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0aab7fe700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac0c7400 0x7f0aac0c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0aab7fe700 1 -- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aac154800 msgr2=0x7f0aac154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0aab7fe700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aac154800 0x7f0aac154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0aab7fe700 1 -- 172.21.15.55:0/998843524 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac0c6c00 con 0x7f0a94003bf0 2023-10-06T03:51:16.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0aaaffd700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac0c7400 0x7f0aac0c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:51:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.683+0000 7f0aab7fe700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a94003bf0 0x7f0aac16b7a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400c080 tx=0x7f0a9c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.684+0000 7f0a937fe700 1 -- 172.21.15.55:0/998843524 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c022070 con 0x7f0a94003bf0 2023-10-06T03:51:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.684+0000 7f0a937fe700 1 -- 172.21.15.55:0/998843524 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c00e9e0 con 0x7f0a94003bf0 2023-10-06T03:51:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.684+0000 7f0a937fe700 1 -- 172.21.15.55:0/998843524 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c0276a0 con 0x7f0a94003bf0 2023-10-06T03:51:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.684+0000 7f0ab099d700 1 -- 172.21.15.55:0/998843524 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aac0c6ee0 con 0x7f0a94003bf0 2023-10-06T03:51:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.684+0000 7f0ab099d700 1 -- 172.21.15.55:0/998843524 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aac0c7db0 con 0x7f0a94003bf0 2023-10-06T03:51:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.685+0000 7f0ab099d700 1 -- 172.21.15.55:0/998843524 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aac0c7250 con 0x7f0a94003bf0 2023-10-06T03:51:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.686+0000 7f0a937fe700 1 -- 172.21.15.55:0/998843524 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a9c01c450 con 0x7f0a94003bf0 2023-10-06T03:51:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.687+0000 7f0a937fe700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0a98066f30 0x7f0a980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:16.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.687+0000 7f0a937fe700 1 -- 172.21.15.55:0/998843524 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6286+0+0 (secure 0 0 0) 0x7f0a9c086370 con 0x7f0a94003bf0 2023-10-06T03:51:16.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.687+0000 7f0aaaffd700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0a98066f30 0x7f0a980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:16.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.688+0000 7f0aaaffd700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0a98066f30 0x7f0a980693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0001ca0 tx=0x7f0aa0009040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:16.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.691+0000 7f0a937fe700 1 -- 172.21.15.55:0/998843524 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a9c053300 con 0x7f0a94003bf0 2023-10-06T03:51:16.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.892+0000 7f0ab099d700 1 -- 172.21.15.55:0/998843524 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f0aac0c8210 con 0x7f0a94003bf0 2023-10-06T03:51:16.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.897+0000 7f0a937fe700 1 -- 172.21.15.55:0/998843524 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v38) v1 ==== 115+0+0 (secure 0 0 0) 0x7f0a9c056920 con 0x7f0a94003bf0 2023-10-06T03:51:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.899+0000 7f0ab099d700 1 -- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0a98066f30 msgr2=0x7f0a980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.899+0000 7f0ab099d700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0a98066f30 0x7f0a980693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0001ca0 tx=0x7f0aa0009040 comp rx=0 tx=0).stop 2023-10-06T03:51:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.899+0000 7f0ab099d700 1 -- 172.21.15.55:0/998843524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a94003bf0 msgr2=0x7f0aac16b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.899+0000 7f0ab099d700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a94003bf0 0x7f0aac16b7a0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400c080 tx=0x7f0a9c01a000 comp rx=0 tx=0).stop 2023-10-06T03:51:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.899+0000 7f0ab099d700 1 -- 172.21.15.55:0/998843524 shutdown_connections 2023-10-06T03:51:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.899+0000 7f0ab099d700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0a98066f30 0x7f0a980693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.899+0000 7f0ab099d700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aac154800 0x7f0aac154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.899+0000 7f0ab099d700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a94003bf0 0x7f0aac16b7a0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.899+0000 7f0ab099d700 1 --2- 172.21.15.55:0/998843524 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac0c7400 0x7f0aac0c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.899+0000 7f0ab099d700 1 -- 172.21.15.55:0/998843524 >> 172.21.15.55:0/998843524 conn(0x7f0aac068390 msgr2=0x7f0aac0697e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.899+0000 7f0ab099d700 1 -- 172.21.15.55:0/998843524 shutdown_connections 2023-10-06T03:51:16.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:16.899+0000 7f0ab099d700 1 -- 172.21.15.55:0/998843524 wait complete. 2023-10-06T03:51:16.918 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T03:51:17.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.106+0000 7f1627fff700 1 Processor -- start 2023-10-06T03:51:17.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.106+0000 7f1627fff700 1 -- start start 2023-10-06T03:51:17.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.107+0000 7f1627fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1628159a60 0x7f16280c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:17.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.107+0000 7f1627fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f16280ccc00 0x7f1628069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:17.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.107+0000 7f1627fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16280c9cb0 con 0x7f162806a020 2023-10-06T03:51:17.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.107+0000 7f1627fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162806dfc0 con 0x7f16280ccc00 2023-10-06T03:51:17.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.107+0000 7f1627fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16280c9f30 con 0x7f1628159a60 2023-10-06T03:51:17.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.108+0000 7f16267fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f16280ccc00 0x7f1628069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:17.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.108+0000 7f16267fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f16280ccc00 0x7f1628069ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41170/0 (socket says 172.21.15.55:41170) 2023-10-06T03:51:17.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.108+0000 7f16267fc700 1 -- 172.21.15.55:0/1761436709 learned_addr learned my addr 172.21.15.55:0/1761436709 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:17.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.108+0000 7f1626ffd700 1 --2- 172.21.15.55:0/1761436709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1628159a60 0x7f16280c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:17.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.108+0000 7f16267fc700 1 -- 172.21.15.55:0/1761436709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1628159a60 msgr2=0x7f16280c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:17.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.108+0000 7f16267fc700 1 --2- 172.21.15.55:0/1761436709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1628159a60 0x7f16280c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.108+0000 7f16267fc700 1 -- 172.21.15.55:0/1761436709 >> v1:172.21.15.55:6789/0 conn(0x7f162806a020 legacy=0x7f16280cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:17.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.108+0000 7f16267fc700 1 -- 172.21.15.55:0/1761436709 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16280ca470 con 0x7f16280ccc00 2023-10-06T03:51:17.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.108+0000 7f1625ffb700 1 -- 172.21.15.55:0/1761436709 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313377591 0 0) 0x7f16280c9cb0 con 0x7f162806a020 2023-10-06T03:51:17.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.108+0000 7f16267fc700 1 --2- 172.21.15.55:0/1761436709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16280ccc00 0x7f1628069ae0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f161c0099b0 tx=0x7f161c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef4338408f6728 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:17.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.109+0000 7f1625ffb700 1 -- 172.21.15.55:0/1761436709 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c009ea0 con 0x7f16280ccc00 2023-10-06T03:51:17.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.109+0000 7f1625ffb700 1 -- 172.21.15.55:0/1761436709 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c00b8f0 con 0x7f16280ccc00 2023-10-06T03:51:17.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.109+0000 7f1625ffb700 1 -- 172.21.15.55:0/1761436709 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00bc90 con 0x7f16280ccc00 2023-10-06T03:51:17.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.109+0000 7f1627fff700 1 -- 172.21.15.55:0/1761436709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16280ccc00 msgr2=0x7f1628069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:17.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.109+0000 7f1627fff700 1 --2- 172.21.15.55:0/1761436709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16280ccc00 0x7f1628069ae0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f161c0099b0 tx=0x7f161c009250 comp rx=0 tx=0).stop 2023-10-06T03:51:17.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.109+0000 7f1627fff700 1 -- 172.21.15.55:0/1761436709 shutdown_connections 2023-10-06T03:51:17.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.109+0000 7f1627fff700 1 --2- 172.21.15.55:0/1761436709 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1628159a60 0x7f16280c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.109+0000 7f1627fff700 1 --2- 172.21.15.55:0/1761436709 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16280ccc00 0x7f1628069ae0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.109+0000 7f1627fff700 1 -- 172.21.15.55:0/1761436709 >> 172.21.15.55:0/1761436709 conn(0x7f162806baa0 msgr2=0x7f162806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:17.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.109+0000 7f1627fff700 1 -- 172.21.15.55:0/1761436709 shutdown_connections 2023-10-06T03:51:17.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.109+0000 7f1627fff700 1 -- 172.21.15.55:0/1761436709 wait complete. 2023-10-06T03:51:17.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.111+0000 7f1627fff700 1 Processor -- start 2023-10-06T03:51:17.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.111+0000 7f1627fff700 1 -- start start 2023-10-06T03:51:17.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.112+0000 7f1627fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f162806a020 0x7f16280b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.112+0000 7f1627fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16280ccc00 0x7f16280b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.112+0000 7f1627fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1628159a60 0x7f162814f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.112+0000 7f1627fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16280b2720 con 0x7f16280ccc00 2023-10-06T03:51:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.112+0000 7f1627fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16280b4080 con 0x7f162806a020 2023-10-06T03:51:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.112+0000 7f1627fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16280b1b10 con 0x7f1628159a60 2023-10-06T03:51:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.112+0000 7f16267fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16280ccc00 0x7f16280b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.112+0000 7f16267fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16280ccc00 0x7f16280b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41180/0 (socket says 172.21.15.55:41180) 2023-10-06T03:51:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.112+0000 7f16267fc700 1 -- 172.21.15.55:0/1542849840 learned_addr learned my addr 172.21.15.55:0/1542849840 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.112+0000 7f1626ffd700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f162806a020 0x7f16280b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.112+0000 7f16277fe700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1628159a60 0x7f162814f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:17.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.113+0000 7f16267fc700 1 -- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1628159a60 msgr2=0x7f162814f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.113+0000 7f16267fc700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1628159a60 0x7f162814f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.113+0000 7f16267fc700 1 -- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f162806a020 msgr2=0x7f16280b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.113+0000 7f16267fc700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f162806a020 0x7f16280b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.113+0000 7f16267fc700 1 -- 172.21.15.55:0/1542849840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c0094f0 con 0x7f16280ccc00 2023-10-06T03:51:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.113+0000 7f1626ffd700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f162806a020 0x7f16280b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.113+0000 7f16267fc700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16280ccc00 0x7f16280b3b40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f161c006fd0 tx=0x7f161c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.113+0000 7f160effd700 1 -- 172.21.15.55:0/1542849840 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00bb30 con 0x7f16280ccc00 2023-10-06T03:51:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.113+0000 7f160effd700 1 -- 172.21.15.55:0/1542849840 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c026430 con 0x7f16280ccc00 2023-10-06T03:51:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.113+0000 7f160effd700 1 -- 172.21.15.55:0/1542849840 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00bc90 con 0x7f16280ccc00 2023-10-06T03:51:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.113+0000 7f1627fff700 1 -- 172.21.15.55:0/1542849840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16280b1df0 con 0x7f16280ccc00 2023-10-06T03:51:17.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.113+0000 7f1627fff700 1 -- 172.21.15.55:0/1542849840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16280b23b0 con 0x7f16280ccc00 2023-10-06T03:51:17.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.114+0000 7f1627fff700 1 -- 172.21.15.55:0/1542849840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162806eba0 con 0x7f16280ccc00 2023-10-06T03:51:17.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.115+0000 7f160effd700 1 -- 172.21.15.55:0/1542849840 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f161c023030 con 0x7f16280ccc00 2023-10-06T03:51:17.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.116+0000 7f160effd700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1614066f30 0x7f16140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:17.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.116+0000 7f160effd700 1 -- 172.21.15.55:0/1542849840 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6286+0+0 (secure 0 0 0) 0x7f161c085fe0 con 0x7f16280ccc00 2023-10-06T03:51:17.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.117+0000 7f1626ffd700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1614066f30 0x7f16140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:17.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.117+0000 7f1626ffd700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1614066f30 0x7f16140693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1618009ce0 tx=0x7f1618008040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:17.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.120+0000 7f160effd700 1 -- 172.21.15.55:0/1542849840 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f161c053030 con 0x7f16280ccc00 2023-10-06T03:51:17.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.326+0000 7f1627fff700 1 -- 172.21.15.55:0/1542849840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f16280ca750 con 0x7f16280ccc00 2023-10-06T03:51:17.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.332+0000 7f160effd700 1 -- 172.21.15.55:0/1542849840 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v39) v1 ==== 120+0+0 (secure 0 0 0) 0x7f161c056650 con 0x7f16280ccc00 2023-10-06T03:51:17.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.334+0000 7f1627fff700 1 -- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1614066f30 msgr2=0x7f16140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:17.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.334+0000 7f1627fff700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1614066f30 0x7f16140693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1618009ce0 tx=0x7f1618008040 comp rx=0 tx=0).stop 2023-10-06T03:51:17.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.334+0000 7f1627fff700 1 -- 172.21.15.55:0/1542849840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16280ccc00 msgr2=0x7f16280b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:17.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.334+0000 7f1627fff700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16280ccc00 0x7f16280b3b40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f161c006fd0 tx=0x7f161c017000 comp rx=0 tx=0).stop 2023-10-06T03:51:17.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.334+0000 7f1627fff700 1 -- 172.21.15.55:0/1542849840 shutdown_connections 2023-10-06T03:51:17.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.334+0000 7f1627fff700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1614066f30 0x7f16140693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.334+0000 7f1627fff700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1628159a60 0x7f162814f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.334+0000 7f1627fff700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f162806a020 0x7f16280b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.334+0000 7f1627fff700 1 --2- 172.21.15.55:0/1542849840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f16280ccc00 0x7f16280b3b40 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.334+0000 7f1627fff700 1 -- 172.21.15.55:0/1542849840 >> 172.21.15.55:0/1542849840 conn(0x7f162806baa0 msgr2=0x7f16281578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:17.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.334+0000 7f1627fff700 1 -- 172.21.15.55:0/1542849840 shutdown_connections 2023-10-06T03:51:17.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.334+0000 7f1627fff700 1 -- 172.21.15.55:0/1542849840 wait complete. 2023-10-06T03:51:17.353 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-06T03:51:17.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.540+0000 7fef5339d700 1 Processor -- start 2023-10-06T03:51:17.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.540+0000 7fef5339d700 1 -- start start 2023-10-06T03:51:17.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.542+0000 7fef5339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef4c150550 0x7fef4c157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:17.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.542+0000 7fef5339d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fef4c0c7410 0x7fef4c069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:17.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.542+0000 7fef5339d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef4c06e2a0 con 0x7fef4c150180 2023-10-06T03:51:17.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.542+0000 7fef5339d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4c06b2e0 con 0x7fef4c0c7410 2023-10-06T03:51:17.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.542+0000 7fef5339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c0c78e0 con 0x7fef4c150550 2023-10-06T03:51:17.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.542+0000 7fef51b9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef4c150550 0x7fef4c157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:17.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.542+0000 7fef51b9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef4c150550 0x7fef4c157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:58830/0 (socket says 172.21.15.55:58830) 2023-10-06T03:51:17.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.542+0000 7fef51b9a700 1 -- 172.21.15.55:0/4151953438 learned_addr learned my addr 172.21.15.55:0/4151953438 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:17.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.542+0000 7fef52b9c700 1 --2- 172.21.15.55:0/4151953438 >> v2:172.21.15.163:3300/0 conn(0x7fef4c0c7410 0x7fef4c069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:17.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.543+0000 7fef51b9a700 1 -- 172.21.15.55:0/4151953438 >> v2:172.21.15.163:3300/0 conn(0x7fef4c0c7410 msgr2=0x7fef4c069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:17.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.543+0000 7fef51b9a700 1 --2- 172.21.15.55:0/4151953438 >> v2:172.21.15.163:3300/0 conn(0x7fef4c0c7410 0x7fef4c069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.543+0000 7fef51399700 1 -- 172.21.15.55:0/4151953438 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055337260 0 0) 0x7fef4c06e2a0 con 0x7fef4c150180 2023-10-06T03:51:17.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.543+0000 7fef51b9a700 1 -- 172.21.15.55:0/4151953438 >> v1:172.21.15.163:6789/0 conn(0x7fef4c150180 legacy=0x7fef4c0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:17.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.543+0000 7fef51b9a700 1 -- 172.21.15.55:0/4151953438 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c0c7bc0 con 0x7fef4c150550 2023-10-06T03:51:17.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.543+0000 7fef51b9a700 1 --2- 172.21.15.55:0/4151953438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef4c150550 0x7fef4c157770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0099e0 tx=0x7fef3c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1c5b0f18cb48471 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:17.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.543+0000 7fef51399700 1 -- 172.21.15.55:0/4151953438 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00b820 con 0x7fef4c150550 2023-10-06T03:51:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.544+0000 7fef51399700 1 -- 172.21.15.55:0/4151953438 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c00b980 con 0x7fef4c150550 2023-10-06T03:51:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.544+0000 7fef51399700 1 -- 172.21.15.55:0/4151953438 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00bd20 con 0x7fef4c150550 2023-10-06T03:51:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.544+0000 7fef5339d700 1 -- 172.21.15.55:0/4151953438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef4c150550 msgr2=0x7fef4c157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.544+0000 7fef5339d700 1 --2- 172.21.15.55:0/4151953438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef4c150550 0x7fef4c157770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0099e0 tx=0x7fef3c009250 comp rx=0 tx=0).stop 2023-10-06T03:51:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.544+0000 7fef5339d700 1 -- 172.21.15.55:0/4151953438 shutdown_connections 2023-10-06T03:51:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.544+0000 7fef5339d700 1 --2- 172.21.15.55:0/4151953438 >> v2:172.21.15.163:3300/0 conn(0x7fef4c0c7410 0x7fef4c069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.544+0000 7fef5339d700 1 --2- 172.21.15.55:0/4151953438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef4c150550 0x7fef4c157770 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.544+0000 7fef5339d700 1 -- 172.21.15.55:0/4151953438 >> 172.21.15.55:0/4151953438 conn(0x7fef4c0cc1f0 msgr2=0x7fef4c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.544+0000 7fef5339d700 1 -- 172.21.15.55:0/4151953438 shutdown_connections 2023-10-06T03:51:17.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.544+0000 7fef5339d700 1 -- 172.21.15.55:0/4151953438 wait complete. 2023-10-06T03:51:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.546+0000 7fef5339d700 1 Processor -- start 2023-10-06T03:51:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.546+0000 7fef5339d700 1 -- start start 2023-10-06T03:51:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef5339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef4c0c7410 0x7fef4c148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef5339d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef4c150180 0x7fef4c0ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef5339d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef4c150550 0x7fef4c14a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef5339d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c0ae950 con 0x7fef4c150180 2023-10-06T03:51:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef5339d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c0c4db0 con 0x7fef4c150550 2023-10-06T03:51:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef5339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4c0c4a80 con 0x7fef4c0c7410 2023-10-06T03:51:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef51b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef4c150180 0x7fef4c0ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef51b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef4c150180 0x7fef4c0ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41194/0 (socket says 172.21.15.55:41194) 2023-10-06T03:51:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef51b9a700 1 -- 172.21.15.55:0/411247525 learned_addr learned my addr 172.21.15.55:0/411247525 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef52b9c700 1 --2- 172.21.15.55:0/411247525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef4c150550 0x7fef4c14a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef5239b700 1 --2- 172.21.15.55:0/411247525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef4c0c7410 0x7fef4c148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef51b9a700 1 -- 172.21.15.55:0/411247525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef4c0c7410 msgr2=0x7fef4c148f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef51b9a700 1 --2- 172.21.15.55:0/411247525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef4c0c7410 0x7fef4c148f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef51b9a700 1 -- 172.21.15.55:0/411247525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef4c150550 msgr2=0x7fef4c14a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef51b9a700 1 --2- 172.21.15.55:0/411247525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef4c150550 0x7fef4c14a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.547+0000 7fef51b9a700 1 -- 172.21.15.55:0/411247525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3c0094f0 con 0x7fef4c150180 2023-10-06T03:51:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.548+0000 7fef51b9a700 1 --2- 172.21.15.55:0/411247525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef4c150180 0x7fef4c0ad6d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0c7a20 tx=0x7fef3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:17.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.548+0000 7fef427fc700 1 -- 172.21.15.55:0/411247525 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c026450 con 0x7fef4c150180 2023-10-06T03:51:17.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.548+0000 7fef427fc700 1 -- 172.21.15.55:0/411247525 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c0265b0 con 0x7fef4c150180 2023-10-06T03:51:17.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.548+0000 7fef427fc700 1 -- 172.21.15.55:0/411247525 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c026970 con 0x7fef4c150180 2023-10-06T03:51:17.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.548+0000 7fef5339d700 1 -- 172.21.15.55:0/411247525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4c0ae0b0 con 0x7fef4c150180 2023-10-06T03:51:17.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.548+0000 7fef5339d700 1 -- 172.21.15.55:0/411247525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef4c0ae670 con 0x7fef4c150180 2023-10-06T03:51:17.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.549+0000 7fef5339d700 1 -- 172.21.15.55:0/411247525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4c04c3c0 con 0x7fef4c150180 2023-10-06T03:51:17.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.550+0000 7fef427fc700 1 -- 172.21.15.55:0/411247525 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef3c01a420 con 0x7fef4c150180 2023-10-06T03:51:17.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.551+0000 7fef427fc700 1 --2- 172.21.15.55:0/411247525 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fef30066f30 0x7fef300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:17.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.551+0000 7fef427fc700 1 -- 172.21.15.55:0/411247525 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6286+0+0 (secure 0 0 0) 0x7fef3c086740 con 0x7fef4c150180 2023-10-06T03:51:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.551+0000 7fef5239b700 1 --2- 172.21.15.55:0/411247525 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fef30066f30 0x7fef300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:17.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.552+0000 7fef5239b700 1 --2- 172.21.15.55:0/411247525 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fef30066f30 0x7fef300693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef480011e0 tx=0x7fef48009040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:17.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.555+0000 7fef427fc700 1 -- 172.21.15.55:0/411247525 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef3c053750 con 0x7fef4c150180 2023-10-06T03:51:17.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.842+0000 7fef5339d700 1 -- 172.21.15.55:0/411247525 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7fef4c0c4ef0 con 0x7fef4c150180 2023-10-06T03:51:17.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.934+0000 7fef427fc700 1 -- 172.21.15.55:0/411247525 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v673) v1 ==== 183+0+0 (secure 0 0 0) 0x7fef3c056d70 con 0x7fef4c150180 2023-10-06T03:51:17.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.936+0000 7fef5339d700 1 -- 172.21.15.55:0/411247525 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fef30066f30 msgr2=0x7fef300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:17.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.936+0000 7fef5339d700 1 --2- 172.21.15.55:0/411247525 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fef30066f30 0x7fef300693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fef480011e0 tx=0x7fef48009040 comp rx=0 tx=0).stop 2023-10-06T03:51:17.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.936+0000 7fef5339d700 1 -- 172.21.15.55:0/411247525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef4c150180 msgr2=0x7fef4c0ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:17.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.936+0000 7fef5339d700 1 --2- 172.21.15.55:0/411247525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef4c150180 0x7fef4c0ad6d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0c7a20 tx=0x7fef3c017000 comp rx=0 tx=0).stop 2023-10-06T03:51:17.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.936+0000 7fef5339d700 1 -- 172.21.15.55:0/411247525 shutdown_connections 2023-10-06T03:51:17.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.936+0000 7fef5339d700 1 --2- 172.21.15.55:0/411247525 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fef30066f30 0x7fef300693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.936+0000 7fef5339d700 1 --2- 172.21.15.55:0/411247525 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fef4c150550 0x7fef4c14a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.936+0000 7fef5339d700 1 --2- 172.21.15.55:0/411247525 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fef4c0c7410 0x7fef4c148f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.936+0000 7fef5339d700 1 --2- 172.21.15.55:0/411247525 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fef4c150180 0x7fef4c0ad6d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:17.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.936+0000 7fef5339d700 1 -- 172.21.15.55:0/411247525 >> 172.21.15.55:0/411247525 conn(0x7fef4c0cc1f0 msgr2=0x7fef4c158e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:17.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.936+0000 7fef5339d700 1 -- 172.21.15.55:0/411247525 shutdown_connections 2023-10-06T03:51:17.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:17.936+0000 7fef5339d700 1 -- 172.21.15.55:0/411247525 wait complete. 2023-10-06T03:51:17.956 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-10-06T03:51:17.957 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-10-06T03:51:17.957 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T03:51:17.957 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-06T03:51:18.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.143+0000 7fa88b59e700 1 Processor -- start 2023-10-06T03:51:18.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.143+0000 7fa88b59e700 1 -- start start 2023-10-06T03:51:18.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.145+0000 7fa88b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c1566e0 0x7fa88c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:18.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.145+0000 7fa88b59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88c0ca860 con 0x7fa88c0cb900 2023-10-06T03:51:18.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.145+0000 7fa88b59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88c0cab40 con 0x7fa88c156310 2023-10-06T03:51:18.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.145+0000 7fa88b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0cae20 con 0x7fa88c1566e0 2023-10-06T03:51:18.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.145+0000 7fa88ad9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa88c0cb900 0x7fa88c0c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57354/0 (socket says 172.21.15.55:57354) 2023-10-06T03:51:18.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.145+0000 7fa88ad9d700 1 -- 172.21.15.55:0/1620287883 learned_addr learned my addr 172.21.15.55:0/1620287883 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:18.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.145+0000 7fa889d9b700 1 --2- 172.21.15.55:0/1620287883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c1566e0 0x7fa88c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:18.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.146+0000 7fa88959a700 1 -- 172.21.15.55:0/1620287883 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600817061 0 0) 0x7fa88c0ca860 con 0x7fa88c0cb900 2023-10-06T03:51:18.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.146+0000 7fa88959a700 1 -- 172.21.15.55:0/1620287883 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa874003570 con 0x7fa88c0cb900 2023-10-06T03:51:18.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.146+0000 7fa88959a700 1 -- 172.21.15.55:0/1620287883 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646613246 0 0) 0x7fa88c0cab40 con 0x7fa88c156310 2023-10-06T03:51:18.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.146+0000 7fa88959a700 1 -- 172.21.15.55:0/1620287883 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa88c0ca860 con 0x7fa88c156310 2023-10-06T03:51:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.146+0000 7fa889d9b700 1 -- 172.21.15.55:0/1620287883 >> v1:172.21.15.163:6789/0 conn(0x7fa88c156310 legacy=0x7fa88c0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.146+0000 7fa889d9b700 1 -- 172.21.15.55:0/1620287883 >> v1:172.21.15.55:6789/0 conn(0x7fa88c0cb900 legacy=0x7fa88c0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.146+0000 7fa889d9b700 1 -- 172.21.15.55:0/1620287883 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c0cb100 con 0x7fa88c1566e0 2023-10-06T03:51:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.146+0000 7fa889d9b700 1 --2- 172.21.15.55:0/1620287883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c1566e0 0x7fa88c0ae000 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa880009910 tx=0x7fa8800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b05e429d37585a7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.146+0000 7fa88959a700 1 -- 172.21.15.55:0/1620287883 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880011630 con 0x7fa88c1566e0 2023-10-06T03:51:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.146+0000 7fa88959a700 1 -- 172.21.15.55:0/1620287883 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa880011790 con 0x7fa88c1566e0 2023-10-06T03:51:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.147+0000 7fa88959a700 1 -- 172.21.15.55:0/1620287883 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880011b30 con 0x7fa88c1566e0 2023-10-06T03:51:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.147+0000 7fa88b59e700 1 -- 172.21.15.55:0/1620287883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c1566e0 msgr2=0x7fa88c0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.147+0000 7fa88b59e700 1 --2- 172.21.15.55:0/1620287883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c1566e0 0x7fa88c0ae000 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa880009910 tx=0x7fa8800091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.147+0000 7fa88b59e700 1 -- 172.21.15.55:0/1620287883 shutdown_connections 2023-10-06T03:51:18.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.147+0000 7fa88b59e700 1 --2- 172.21.15.55:0/1620287883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c1566e0 0x7fa88c0ae000 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.147+0000 7fa88b59e700 1 -- 172.21.15.55:0/1620287883 >> 172.21.15.55:0/1620287883 conn(0x7fa88c0cc1f0 msgr2=0x7fa88c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.147+0000 7fa88b59e700 1 -- 172.21.15.55:0/1620287883 shutdown_connections 2023-10-06T03:51:18.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.147+0000 7fa88b59e700 1 -- 172.21.15.55:0/1620287883 wait complete. 2023-10-06T03:51:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.149+0000 7fa88b59e700 1 Processor -- start 2023-10-06T03:51:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.149+0000 7fa88b59e700 1 -- start start 2023-10-06T03:51:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.150+0000 7fa88b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa88c0cb900 0x7fa88c169d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.150+0000 7fa88b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c156310 0x7fa88c16aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.150+0000 7fa88b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c1566e0 0x7fa88c151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.150+0000 7fa88b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c16afb0 con 0x7fa88c1566e0 2023-10-06T03:51:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.150+0000 7fa88b59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0c1490 con 0x7fa88c0cb900 2023-10-06T03:51:18.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.150+0000 7fa88b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0c1770 con 0x7fa88c156310 2023-10-06T03:51:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.150+0000 7fa88ad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c1566e0 0x7fa88c151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.150+0000 7fa88ad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c1566e0 0x7fa88c151bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41206/0 (socket says 172.21.15.55:41206) 2023-10-06T03:51:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.150+0000 7fa88ad9d700 1 -- 172.21.15.55:0/2324598232 learned_addr learned my addr 172.21.15.55:0/2324598232 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.150+0000 7fa889d9b700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c156310 0x7fa88c16aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.150+0000 7fa88a59c700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa88c0cb900 0x7fa88c169d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.151+0000 7fa88ad9d700 1 -- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c156310 msgr2=0x7fa88c16aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.151+0000 7fa88ad9d700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c156310 0x7fa88c16aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.151+0000 7fa88ad9d700 1 -- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa88c0cb900 msgr2=0x7fa88c169d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.151+0000 7fa88ad9d700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa88c0cb900 0x7fa88c169d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.151+0000 7fa88ad9d700 1 -- 172.21.15.55:0/2324598232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa880009420 con 0x7fa88c1566e0 2023-10-06T03:51:18.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.151+0000 7fa88a59c700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa88c0cb900 0x7fa88c169d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:18.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.151+0000 7fa88ad9d700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c1566e0 0x7fa88c151bc0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fa88400cc60 tx=0x7fa884001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:18.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.151+0000 7fa87a7fc700 1 -- 172.21.15.55:0/2324598232 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88400a960 con 0x7fa88c1566e0 2023-10-06T03:51:18.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.151+0000 7fa87a7fc700 1 -- 172.21.15.55:0/2324598232 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88400aae0 con 0x7fa88c1566e0 2023-10-06T03:51:18.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.151+0000 7fa87a7fc700 1 -- 172.21.15.55:0/2324598232 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88400aea0 con 0x7fa88c1566e0 2023-10-06T03:51:18.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.151+0000 7fa88b59e700 1 -- 172.21.15.55:0/2324598232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88c0c1a50 con 0x7fa88c1566e0 2023-10-06T03:51:18.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.151+0000 7fa88b59e700 1 -- 172.21.15.55:0/2324598232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88c152100 con 0x7fa88c1566e0 2023-10-06T03:51:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.152+0000 7fa88b59e700 1 -- 172.21.15.55:0/2324598232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88c153760 con 0x7fa88c1566e0 2023-10-06T03:51:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.153+0000 7fa87a7fc700 1 -- 172.21.15.55:0/2324598232 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa884007300 con 0x7fa88c1566e0 2023-10-06T03:51:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.154+0000 7fa87a7fc700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa86c066f30 0x7fa86c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.154+0000 7fa87a7fc700 1 -- 172.21.15.55:0/2324598232 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6286+0+0 (secure 0 0 0) 0x7fa884084bd0 con 0x7fa88c1566e0 2023-10-06T03:51:18.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.155+0000 7fa88a59c700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa86c066f30 0x7fa86c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:18.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.155+0000 7fa88a59c700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa86c066f30 0x7fa86c0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa87c003990 tx=0x7fa87c009040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:18.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.158+0000 7fa87a7fc700 1 -- 172.21.15.55:0/2324598232 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa88c153760 con 0x7fa88c1566e0 2023-10-06T03:51:18.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.443+0000 7fa88b59e700 1 -- 172.21.15.55:0/2324598232 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7fa88c153760 con 0x7fa88c1566e0 2023-10-06T03:51:18.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.940+0000 7fa87a7fc700 1 -- 172.21.15.55:0/2324598232 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v674) v1 ==== 151+0+0 (secure 0 0 0) 0x7fa88c153760 con 0x7fa88c1566e0 2023-10-06T03:51:18.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.942+0000 7fa88b59e700 1 -- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa86c066f30 msgr2=0x7fa86c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:18.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.942+0000 7fa88b59e700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa86c066f30 0x7fa86c0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa87c003990 tx=0x7fa87c009040 comp rx=0 tx=0).stop 2023-10-06T03:51:18.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.942+0000 7fa88b59e700 1 -- 172.21.15.55:0/2324598232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c1566e0 msgr2=0x7fa88c151bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:18.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.942+0000 7fa88b59e700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c1566e0 0x7fa88c151bc0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fa88400cc60 tx=0x7fa884001ae0 comp rx=0 tx=0).stop 2023-10-06T03:51:18.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.942+0000 7fa88b59e700 1 -- 172.21.15.55:0/2324598232 shutdown_connections 2023-10-06T03:51:18.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.942+0000 7fa88b59e700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa86c066f30 0x7fa86c0693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:18.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.942+0000 7fa88b59e700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c1566e0 0x7fa88c151bc0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:18.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.942+0000 7fa88b59e700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa88c0cb900 0x7fa88c169d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:18.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.942+0000 7fa88b59e700 1 --2- 172.21.15.55:0/2324598232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c156310 0x7fa88c16aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:18.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.942+0000 7fa88b59e700 1 -- 172.21.15.55:0/2324598232 >> 172.21.15.55:0/2324598232 conn(0x7fa88c0cc1f0 msgr2=0x7fa88c154920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:18.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.943+0000 7fa88b59e700 1 -- 172.21.15.55:0/2324598232 shutdown_connections 2023-10-06T03:51:18.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:18.943+0000 7fa88b59e700 1 -- 172.21.15.55:0/2324598232 wait complete. 2023-10-06T03:51:18.963 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:51:19.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.153+0000 7f2c12f9d700 1 Processor -- start 2023-10-06T03:51:19.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.154+0000 7f2c12f9d700 1 -- start start 2023-10-06T03:51:19.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.155+0000 7f2c12f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c0c06a710 0x7f2c0c160e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:19.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.155+0000 7f2c12f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c0c06e150 con 0x7f2c0c153900 2023-10-06T03:51:19.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.155+0000 7f2c12f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c0c0c7800 con 0x7f2c0c0c9db0 2023-10-06T03:51:19.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.155+0000 7f2c12f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0c7610 con 0x7f2c0c06a710 2023-10-06T03:51:19.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.155+0000 7f2c11f9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2c0c153900 0x7f2c0c0c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57370/0 (socket says 172.21.15.55:57370) 2023-10-06T03:51:19.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.155+0000 7f2c11f9b700 1 -- 172.21.15.55:0/3292206159 learned_addr learned my addr 172.21.15.55:0/3292206159 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:19.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.155+0000 7f2c1279c700 1 --2- 172.21.15.55:0/3292206159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c0c06a710 0x7f2c0c160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:19.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.155+0000 7f2c10f99700 1 -- 172.21.15.55:0/3292206159 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267484183 0 0) 0x7f2c0c06e150 con 0x7f2c0c153900 2023-10-06T03:51:19.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.156+0000 7f2c10f99700 1 -- 172.21.15.55:0/3292206159 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bf8004680 con 0x7f2c0c153900 2023-10-06T03:51:19.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.156+0000 7f2c10f99700 1 -- 172.21.15.55:0/3292206159 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512040635 0 0) 0x7f2c0c0c7800 con 0x7f2c0c0c9db0 2023-10-06T03:51:19.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.156+0000 7f2c10f99700 1 -- 172.21.15.55:0/3292206159 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c0c06e150 con 0x7f2c0c0c9db0 2023-10-06T03:51:19.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.156+0000 7f2c1279c700 1 -- 172.21.15.55:0/3292206159 >> v1:172.21.15.163:6789/0 conn(0x7f2c0c0c9db0 legacy=0x7f2c0c06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:19.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.156+0000 7f2c10f99700 1 -- 172.21.15.55:0/3292206159 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 255489234 0 0) 0x7f2bf8004680 con 0x7f2c0c153900 2023-10-06T03:51:19.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.156+0000 7f2c1279c700 1 -- 172.21.15.55:0/3292206159 >> v1:172.21.15.55:6789/0 conn(0x7f2c0c153900 legacy=0x7f2c0c0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:19.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.156+0000 7f2c1279c700 1 -- 172.21.15.55:0/3292206159 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c0c7ae0 con 0x7f2c0c06a710 2023-10-06T03:51:19.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.156+0000 7f2c1279c700 1 --2- 172.21.15.55:0/3292206159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c0c06a710 0x7f2c0c160e70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2c04009e90 tx=0x7f2c04009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f664c35649198961 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:19.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.156+0000 7f2c10f99700 1 -- 172.21.15.55:0/3292206159 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0400e070 con 0x7f2c0c06a710 2023-10-06T03:51:19.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.157+0000 7f2c10f99700 1 -- 172.21.15.55:0/3292206159 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0400b870 con 0x7f2c0c06a710 2023-10-06T03:51:19.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.157+0000 7f2c10f99700 1 -- 172.21.15.55:0/3292206159 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08001470 con 0x7f2c0c06a710 2023-10-06T03:51:19.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.157+0000 7f2c12f9d700 1 -- 172.21.15.55:0/3292206159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c0c06a710 msgr2=0x7f2c0c160e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:19.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.157+0000 7f2c12f9d700 1 --2- 172.21.15.55:0/3292206159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c0c06a710 0x7f2c0c160e70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2c04009e90 tx=0x7f2c04009780 comp rx=0 tx=0).stop 2023-10-06T03:51:19.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.157+0000 7f2c12f9d700 1 -- 172.21.15.55:0/3292206159 shutdown_connections 2023-10-06T03:51:19.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.157+0000 7f2c12f9d700 1 --2- 172.21.15.55:0/3292206159 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c0c06a710 0x7f2c0c160e70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.157+0000 7f2c12f9d700 1 -- 172.21.15.55:0/3292206159 >> 172.21.15.55:0/3292206159 conn(0x7f2c0c0cc1e0 msgr2=0x7f2c0c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:19.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.157+0000 7f2c12f9d700 1 -- 172.21.15.55:0/3292206159 shutdown_connections 2023-10-06T03:51:19.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.157+0000 7f2c12f9d700 1 -- 172.21.15.55:0/3292206159 wait complete. 2023-10-06T03:51:19.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.159+0000 7f2c12f9d700 1 Processor -- start 2023-10-06T03:51:19.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.159+0000 7f2c12f9d700 1 -- start start 2023-10-06T03:51:19.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.160+0000 7f2c12f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c0c06a710 0x7f2c0c16b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:19.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.160+0000 7f2c12f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c0c0c9db0 0x7f2c0c150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:19.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.160+0000 7f2c12f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c0c153900 0x7f2c0c1520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:19.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.160+0000 7f2c12f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0c33a0 con 0x7f2c0c153900 2023-10-06T03:51:19.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.160+0000 7f2c12f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c16b700 con 0x7f2c0c06a710 2023-10-06T03:51:19.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.160+0000 7f2c12f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0c0ac720 con 0x7f2c0c0c9db0 2023-10-06T03:51:19.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.160+0000 7f2c1279c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c0c153900 0x7f2c0c1520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:19.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.160+0000 7f2c1279c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c0c153900 0x7f2c0c1520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41216/0 (socket says 172.21.15.55:41216) 2023-10-06T03:51:19.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.160+0000 7f2c1279c700 1 -- 172.21.15.55:0/709206842 learned_addr learned my addr 172.21.15.55:0/709206842 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:19.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.160+0000 7f2c1179a700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c0c0c9db0 0x7f2c0c150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:19.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.160+0000 7f2c11f9b700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c0c06a710 0x7f2c0c16b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:19.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.161+0000 7f2c1279c700 1 -- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c0c0c9db0 msgr2=0x7f2c0c150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:19.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.161+0000 7f2c1279c700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c0c0c9db0 0x7f2c0c150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.161+0000 7f2c1279c700 1 -- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c0c06a710 msgr2=0x7f2c0c16b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:19.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.161+0000 7f2c1279c700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c0c06a710 0x7f2c0c16b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.161+0000 7f2c1279c700 1 -- 172.21.15.55:0/709206842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c04009a20 con 0x7f2c0c153900 2023-10-06T03:51:19.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.161+0000 7f2c1179a700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c0c0c9db0 0x7f2c0c150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:19.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.161+0000 7f2c1279c700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c0c153900 0x7f2c0c1520f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2c04009de0 tx=0x7f2c04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:19.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.161+0000 7f2c01ffb700 1 -- 172.21.15.55:0/709206842 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04023070 con 0x7f2c0c153900 2023-10-06T03:51:19.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.161+0000 7f2c01ffb700 1 -- 172.21.15.55:0/709206842 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c04012d80 con 0x7f2c0c153900 2023-10-06T03:51:19.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.161+0000 7f2c01ffb700 1 -- 172.21.15.55:0/709206842 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0401e6a0 con 0x7f2c0c153900 2023-10-06T03:51:19.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.161+0000 7f2c12f9d700 1 -- 172.21.15.55:0/709206842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0c0abb80 con 0x7f2c0c153900 2023-10-06T03:51:19.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.161+0000 7f2c12f9d700 1 -- 172.21.15.55:0/709206842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0c0ac0d0 con 0x7f2c0c153900 2023-10-06T03:51:19.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.162+0000 7f2c12f9d700 1 -- 172.21.15.55:0/709206842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c0c0ac8d0 con 0x7f2c0c153900 2023-10-06T03:51:19.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.163+0000 7f2c01ffb700 1 -- 172.21.15.55:0/709206842 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c0401ead0 con 0x7f2c0c153900 2023-10-06T03:51:19.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.164+0000 7f2c01ffb700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2bf0066f30 0x7f2bf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:19.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.164+0000 7f2c01ffb700 1 -- 172.21.15.55:0/709206842 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6286+0+0 (secure 0 0 0) 0x7f2c040863b0 con 0x7f2c0c153900 2023-10-06T03:51:19.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.164+0000 7f2c11f9b700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2bf0066f30 0x7f2bf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:19.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.165+0000 7f2c11f9b700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2bf0066f30 0x7f2bf00693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c167be0 tx=0x7f2c080015e0 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:19.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.168+0000 7f2c01ffb700 1 -- 172.21.15.55:0/709206842 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c0c0ac8d0 con 0x7f2c0c153900 2023-10-06T03:51:19.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.370+0000 7f2c12f9d700 1 -- 172.21.15.55:0/709206842 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2c0c0ac890 con 0x7f2c0c153900 2023-10-06T03:51:19.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.374+0000 7f2c01ffb700 1 -- 172.21.15.55:0/709206842 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2c04053340 con 0x7f2c0c153900 2023-10-06T03:51:19.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.376+0000 7f2c12f9d700 1 -- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2bf0066f30 msgr2=0x7f2bf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:19.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.376+0000 7f2c12f9d700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2bf0066f30 0x7f2bf00693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c167be0 tx=0x7f2c080015e0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.376+0000 7f2c12f9d700 1 -- 172.21.15.55:0/709206842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c0c153900 msgr2=0x7f2c0c1520f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:19.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.376+0000 7f2c12f9d700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c0c153900 0x7f2c0c1520f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2c04009de0 tx=0x7f2c04017000 comp rx=0 tx=0).stop 2023-10-06T03:51:19.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.376+0000 7f2c12f9d700 1 -- 172.21.15.55:0/709206842 shutdown_connections 2023-10-06T03:51:19.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.376+0000 7f2c12f9d700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2bf0066f30 0x7f2bf00693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.376+0000 7f2c12f9d700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2c0c153900 0x7f2c0c1520f0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.376+0000 7f2c12f9d700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2c0c06a710 0x7f2c0c16b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.376+0000 7f2c12f9d700 1 --2- 172.21.15.55:0/709206842 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2c0c0c9db0 0x7f2c0c150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.376+0000 7f2c12f9d700 1 -- 172.21.15.55:0/709206842 >> 172.21.15.55:0/709206842 conn(0x7f2c0c0cc1e0 msgr2=0x7f2c0c0c18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:19.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.376+0000 7f2c12f9d700 1 -- 172.21.15.55:0/709206842 shutdown_connections 2023-10-06T03:51:19.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.376+0000 7f2c12f9d700 1 -- 172.21.15.55:0/709206842 wait complete. 2023-10-06T03:51:19.395 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:51:19.396 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:51:19.473 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:51:19.589 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:51:19.589 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:51:19.589 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:51:19.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.775+0000 7fc29d39d700 1 Processor -- start 2023-10-06T03:51:19.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.775+0000 7fc29d39d700 1 -- start start 2023-10-06T03:51:19.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.777+0000 7fc29d39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc29806d950 0x7fc2980c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:19.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.777+0000 7fc29d39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc29806dd20 0x7fc2980c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:19.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.777+0000 7fc29d39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc29806ae70 con 0x7fc29806a0a0 2023-10-06T03:51:19.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.777+0000 7fc29d39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc29806a540 con 0x7fc29806d950 2023-10-06T03:51:19.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.777+0000 7fc29d39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29806a820 con 0x7fc29806dd20 2023-10-06T03:51:19.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.777+0000 7fc2977fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc29806a0a0 0x7fc2980c6c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57374/0 (socket says 172.21.15.55:57374) 2023-10-06T03:51:19.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.777+0000 7fc2977fe700 1 -- 172.21.15.55:0/4198292571 learned_addr learned my addr 172.21.15.55:0/4198292571 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:19.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.777+0000 7fc297fff700 1 --2- 172.21.15.55:0/4198292571 >> v2:172.21.15.163:3300/0 conn(0x7fc29806d950 0x7fc2980c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:19.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.777+0000 7fc29cb9c700 1 --2- 172.21.15.55:0/4198292571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc29806dd20 0x7fc2980c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:19.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.777+0000 7fc296ffd700 1 -- 172.21.15.55:0/4198292571 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059510038 0 0) 0x7fc29806ae70 con 0x7fc29806a0a0 2023-10-06T03:51:19.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.777+0000 7fc296ffd700 1 -- 172.21.15.55:0/4198292571 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc280003570 con 0x7fc29806a0a0 2023-10-06T03:51:19.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc296ffd700 1 -- 172.21.15.55:0/4198292571 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2123925037 0 0) 0x7fc280003570 con 0x7fc29806a0a0 2023-10-06T03:51:19.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc296ffd700 1 -- 172.21.15.55:0/4198292571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc29806dd20 msgr2=0x7fc2980c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:19.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc296ffd700 1 --2- 172.21.15.55:0/4198292571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc29806dd20 0x7fc2980c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc296ffd700 1 -- 172.21.15.55:0/4198292571 >> v2:172.21.15.163:3300/0 conn(0x7fc29806d950 msgr2=0x7fc2980c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:19.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc296ffd700 1 --2- 172.21.15.55:0/4198292571 >> v2:172.21.15.163:3300/0 conn(0x7fc29806d950 0x7fc2980c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc296ffd700 1 -- 172.21.15.55:0/4198292571 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29806ab00 con 0x7fc29806a0a0 2023-10-06T03:51:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc296ffd700 1 -- 172.21.15.55:0/4198292571 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fc28c001470 con 0x7fc29806a0a0 2023-10-06T03:51:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc296ffd700 1 -- 172.21.15.55:0/4198292571 >> v1:172.21.15.55:6789/0 conn(0x7fc29806a0a0 legacy=0x7fc2980c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc296ffd700 1 --2- 172.21.15.55:0/4198292571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc29806dd20 0x7fc280003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc296ffd700 1 --2- 172.21.15.55:0/4198292571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2800040b0 0x7fc280004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc296ffd700 1 --2- 172.21.15.55:0/4198292571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc280004d50 0x7fc2800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc296ffd700 1 -- 172.21.15.55:0/4198292571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29806a540 con 0x7fc2800040b0 2023-10-06T03:51:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc296ffd700 1 -- 172.21.15.55:0/4198292571 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29806a820 con 0x7fc280004d50 2023-10-06T03:51:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc296ffd700 1 -- 172.21.15.55:0/4198292571 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28000bb60 con 0x7fc29806dd20 2023-10-06T03:51:19.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc2977fe700 1 --2- 172.21.15.55:0/4198292571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2800040b0 0x7fc280004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:19.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc297fff700 1 --2- 172.21.15.55:0/4198292571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc29806dd20 0x7fc280003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:19.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.778+0000 7fc29cb9c700 1 --2- 172.21.15.55:0/4198292571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc280004d50 0x7fc2800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:19.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.779+0000 7fc2977fe700 1 -- 172.21.15.55:0/4198292571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc29806dd20 msgr2=0x7fc280003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:19.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.779+0000 7fc2977fe700 1 --2- 172.21.15.55:0/4198292571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc29806dd20 0x7fc280003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.779+0000 7fc2977fe700 1 -- 172.21.15.55:0/4198292571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc280004d50 msgr2=0x7fc2800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:19.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.779+0000 7fc2977fe700 1 --2- 172.21.15.55:0/4198292571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc280004d50 0x7fc2800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.779+0000 7fc2977fe700 1 -- 172.21.15.55:0/4198292571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc280003570 con 0x7fc2800040b0 2023-10-06T03:51:19.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.779+0000 7fc2977fe700 1 --2- 172.21.15.55:0/4198292571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2800040b0 0x7fc280004810 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00c2a0 tx=0x7fc28c001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6a9498aadcc33f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:19.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.779+0000 7fc296ffd700 1 -- 172.21.15.55:0/4198292571 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c00cb70 con 0x7fc2800040b0 2023-10-06T03:51:19.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.779+0000 7fc296ffd700 1 -- 172.21.15.55:0/4198292571 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c00ccd0 con 0x7fc2800040b0 2023-10-06T03:51:19.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.779+0000 7fc296ffd700 1 -- 172.21.15.55:0/4198292571 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c005b70 con 0x7fc2800040b0 2023-10-06T03:51:19.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.779+0000 7fc29d39d700 1 -- 172.21.15.55:0/4198292571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2800040b0 msgr2=0x7fc280004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:19.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.779+0000 7fc29d39d700 1 --2- 172.21.15.55:0/4198292571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2800040b0 0x7fc280004810 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00c2a0 tx=0x7fc28c001dc0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.780+0000 7fc29cb9c700 1 -- 172.21.15.55:0/4198292571 reap_dead start 2023-10-06T03:51:19.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.780+0000 7fc29d39d700 1 -- 172.21.15.55:0/4198292571 shutdown_connections 2023-10-06T03:51:19.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.780+0000 7fc29d39d700 1 -- 172.21.15.55:0/4198292571 >> 172.21.15.55:0/4198292571 conn(0x7fc2980cc1f0 msgr2=0x7fc2980cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:19.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.780+0000 7fc29d39d700 1 -- 172.21.15.55:0/4198292571 shutdown_connections 2023-10-06T03:51:19.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.780+0000 7fc29d39d700 1 -- 172.21.15.55:0/4198292571 wait complete. 2023-10-06T03:51:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.781+0000 7fc29d39d700 1 Processor -- start 2023-10-06T03:51:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.781+0000 7fc29d39d700 1 -- start start 2023-10-06T03:51:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.782+0000 7fc29d39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2800040b0 0x7fc298150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.782+0000 7fc29d39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2980afea0 0x7fc2980b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:19.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.782+0000 7fc29d39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2980c4580 0x7fc2980c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:19.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.782+0000 7fc29d39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2980c4f60 con 0x7fc2800040b0 2023-10-06T03:51:19.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.782+0000 7fc29d39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2980afa10 con 0x7fc2980afea0 2023-10-06T03:51:19.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.782+0000 7fc29d39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc298150fe0 con 0x7fc2980c4580 2023-10-06T03:51:19.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc297fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2800040b0 0x7fc298150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:19.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc297fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2800040b0 0x7fc298150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41238/0 (socket says 172.21.15.55:41238) 2023-10-06T03:51:19.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc297fff700 1 -- 172.21.15.55:0/1625748472 learned_addr learned my addr 172.21.15.55:0/1625748472 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:19.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc29cb9c700 1 --2- 172.21.15.55:0/1625748472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2980c4580 0x7fc2980c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:19.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc2977fe700 1 --2- 172.21.15.55:0/1625748472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2980afea0 0x7fc2980b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:19.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc297fff700 1 -- 172.21.15.55:0/1625748472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2980c4580 msgr2=0x7fc2980c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:19.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc297fff700 1 --2- 172.21.15.55:0/1625748472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2980c4580 0x7fc2980c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc297fff700 1 -- 172.21.15.55:0/1625748472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2980afea0 msgr2=0x7fc2980b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:19.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc297fff700 1 --2- 172.21.15.55:0/1625748472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2980afea0 0x7fc2980b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:19.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc297fff700 1 -- 172.21.15.55:0/1625748472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2980cabd0 con 0x7fc2800040b0 2023-10-06T03:51:19.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc297fff700 1 --2- 172.21.15.55:0/1625748472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2800040b0 0x7fc298150aa0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc288011d40 tx=0x7fc2880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:19.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc27ffff700 1 -- 172.21.15.55:0/1625748472 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28800fa40 con 0x7fc2800040b0 2023-10-06T03:51:19.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc27ffff700 1 -- 172.21.15.55:0/1625748472 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28800fba0 con 0x7fc2800040b0 2023-10-06T03:51:19.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc27ffff700 1 -- 172.21.15.55:0/1625748472 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc288014440 con 0x7fc2800040b0 2023-10-06T03:51:19.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.783+0000 7fc29d39d700 1 -- 172.21.15.55:0/1625748472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2980af3c0 con 0x7fc2800040b0 2023-10-06T03:51:19.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.784+0000 7fc29d39d700 1 -- 172.21.15.55:0/1625748472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29815c040 con 0x7fc2800040b0 2023-10-06T03:51:19.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.785+0000 7fc29d39d700 1 -- 172.21.15.55:0/1625748472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2980afcf0 con 0x7fc2800040b0 2023-10-06T03:51:19.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.786+0000 7fc27ffff700 1 -- 172.21.15.55:0/1625748472 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc288014870 con 0x7fc2800040b0 2023-10-06T03:51:19.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.786+0000 7fc27ffff700 1 --2- 172.21.15.55:0/1625748472 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc284066f30 0x7fc2840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:19.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.787+0000 7fc27ffff700 1 -- 172.21.15.55:0/1625748472 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6286+0+0 (secure 0 0 0) 0x7fc288018030 con 0x7fc2800040b0 2023-10-06T03:51:19.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.787+0000 7fc2977fe700 1 --2- 172.21.15.55:0/1625748472 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc284066f30 0x7fc2840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:19.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.788+0000 7fc2977fe700 1 --2- 172.21.15.55:0/1625748472 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc284066f30 0x7fc2840693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc28c015040 tx=0x7fc28c012000 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:19.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:19.791+0000 7fc27ffff700 1 -- 172.21.15.55:0/1625748472 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2880522b0 con 0x7fc2800040b0 2023-10-06T03:51:20.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.040+0000 7fc29d39d700 1 -- 172.21.15.55:0/1625748472 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2980af220 con 0x7fc2800040b0 2023-10-06T03:51:20.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.041+0000 7fc27ffff700 1 -- 172.21.15.55:0/1625748472 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fc288022de0 con 0x7fc2800040b0 2023-10-06T03:51:20.042 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:20.043 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6305,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2272331626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2272331626},{"type":"v1","addr":"172.21.15.163:6834","nonce":2272331626}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":6979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/2110964237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2110964237},{"type":"v1","addr":"172.21.15.163:6836","nonce":2110964237}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7236,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6833/2942349047","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2942349047},{"type":"v1","addr":"172.21.15.55:6833","nonce":2942349047}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47},{"gid":7629,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/195076290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6834","nonce":195076290},{"type":"v1","addr":"172.21.15.55:6835","nonce":195076290}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[]} 2023-10-06T03:51:20.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.043+0000 7fc29d39d700 1 -- 172.21.15.55:0/1625748472 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc284066f30 msgr2=0x7fc2840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:20.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.043+0000 7fc29d39d700 1 --2- 172.21.15.55:0/1625748472 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc284066f30 0x7fc2840693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc28c015040 tx=0x7fc28c012000 comp rx=0 tx=0).stop 2023-10-06T03:51:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.043+0000 7fc29d39d700 1 -- 172.21.15.55:0/1625748472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2800040b0 msgr2=0x7fc298150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.043+0000 7fc29d39d700 1 --2- 172.21.15.55:0/1625748472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2800040b0 0x7fc298150aa0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fc288011d40 tx=0x7fc2880091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.043+0000 7fc29d39d700 1 -- 172.21.15.55:0/1625748472 shutdown_connections 2023-10-06T03:51:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.043+0000 7fc29d39d700 1 --2- 172.21.15.55:0/1625748472 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc284066f30 0x7fc2840693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.043+0000 7fc29d39d700 1 --2- 172.21.15.55:0/1625748472 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2980c4580 0x7fc2980c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.043+0000 7fc29d39d700 1 --2- 172.21.15.55:0/1625748472 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2800040b0 0x7fc298150aa0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.043+0000 7fc29d39d700 1 --2- 172.21.15.55:0/1625748472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2980afea0 0x7fc2980b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.043+0000 7fc29d39d700 1 -- 172.21.15.55:0/1625748472 >> 172.21.15.55:0/1625748472 conn(0x7fc2980cc1f0 msgr2=0x7fc2981552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.043+0000 7fc29d39d700 1 -- 172.21.15.55:0/1625748472 shutdown_connections 2023-10-06T03:51:20.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.043+0000 7fc29d39d700 1 -- 172.21.15.55:0/1625748472 wait complete. 2023-10-06T03:51:20.046 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 55 2023-10-06T03:51:20.062 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:51:20.063 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:51:20.063 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:51:20.063 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:51:20.063 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:51:20.064 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:51:20.064 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:51:20.064 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:51:20.064 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:51:20.064 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:51:20.064 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:51:20.064 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:51:20.065 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:51:20.065 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:51:20.065 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:51:20.065 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:51:20.078 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:51:20.078 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:51:20.079 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:51:20.079 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:51:20.081 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:51:20.081 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:51:20.082 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:51:20.082 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:51:20.125 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:51:20.125 DEBUG:teuthology.parallel:result is None 2023-10-06T03:51:20.126 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:51:20.126 DEBUG:teuthology.parallel:result is None 2023-10-06T03:51:20.128 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:51:20.128 DEBUG:teuthology.parallel:result is None 2023-10-06T03:51:20.129 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:51:20.129 DEBUG:teuthology.parallel:result is None 2023-10-06T03:51:20.129 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:51:20.264 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2023-10-06T03:51:20.265 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T03:51:20.268 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T03:51:20.268 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-10-06T03:51:20.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.325+0000 7fc3d359e700 1 Processor -- start 2023-10-06T03:51:20.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.325+0000 7fc3d359e700 1 -- start start 2023-10-06T03:51:20.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.326+0000 7fc3d359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc3d41597f0 0x7fc3d40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:20.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.326+0000 7fc3d359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d40b11d0 con 0x7fc3d4159420 2023-10-06T03:51:20.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.326+0000 7fc3d359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d40b14b0 con 0x7fc3d415d230 2023-10-06T03:51:20.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.326+0000 7fc3d359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d40b1790 con 0x7fc3d41597f0 2023-10-06T03:51:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.327+0000 7fc3d259c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc3d4159420 0x7fc3d40b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57410/0 (socket says 172.21.15.55:57410) 2023-10-06T03:51:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.327+0000 7fc3d1d9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc3d41597f0 0x7fc3d40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.327+0000 7fc3d259c700 1 -- 172.21.15.55:0/2865034168 learned_addr learned my addr 172.21.15.55:0/2865034168 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.327+0000 7fc3d1d9b700 1 -- 172.21.15.55:0/2865034168 >> v1:172.21.15.163:6789/0 conn(0x7fc3d415d230 legacy=0x7fc3d415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.327+0000 7fc3d1d9b700 1 -- 172.21.15.55:0/2865034168 >> v1:172.21.15.55:6789/0 conn(0x7fc3d4159420 legacy=0x7fc3d40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.327+0000 7fc3d1d9b700 1 -- 172.21.15.55:0/2865034168 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d40b1a70 con 0x7fc3d41597f0 2023-10-06T03:51:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.327+0000 7fc3d1d9b700 1 --2- 172.21.15.55:0/2865034168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3d41597f0 0x7fc3d40b2270 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80098e0 tx=0x7fc3c80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b45fb6b11e4c855 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.328+0000 7fc3d159a700 1 -- 172.21.15.55:0/2865034168 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8009e80 con 0x7fc3d41597f0 2023-10-06T03:51:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.328+0000 7fc3d159a700 1 -- 172.21.15.55:0/2865034168 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c800b860 con 0x7fc3d41597f0 2023-10-06T03:51:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.328+0000 7fc3d159a700 1 -- 172.21.15.55:0/2865034168 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c800bbe0 con 0x7fc3d41597f0 2023-10-06T03:51:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.328+0000 7fc3d359e700 1 -- 172.21.15.55:0/2865034168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3d41597f0 msgr2=0x7fc3d40b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.328+0000 7fc3d359e700 1 --2- 172.21.15.55:0/2865034168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3d41597f0 0x7fc3d40b2270 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80098e0 tx=0x7fc3c80091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.328+0000 7fc3d359e700 1 -- 172.21.15.55:0/2865034168 shutdown_connections 2023-10-06T03:51:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.328+0000 7fc3d359e700 1 --2- 172.21.15.55:0/2865034168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3d41597f0 0x7fc3d40b2270 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.328+0000 7fc3d359e700 1 -- 172.21.15.55:0/2865034168 >> 172.21.15.55:0/2865034168 conn(0x7fc3d40d0660 msgr2=0x7fc3d40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.328+0000 7fc3d359e700 1 -- 172.21.15.55:0/2865034168 shutdown_connections 2023-10-06T03:51:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.328+0000 7fc3d359e700 1 -- 172.21.15.55:0/2865034168 wait complete. 2023-10-06T03:51:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.330+0000 7fc3d359e700 1 Processor -- start 2023-10-06T03:51:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.330+0000 7fc3d359e700 1 -- start start 2023-10-06T03:51:20.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.331+0000 7fc3d359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3d4159420 0x7fc3d41431e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:20.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.331+0000 7fc3d359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3d41597f0 0x7fc3d4157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:20.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.331+0000 7fc3d359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3d415d230 0x7fc3d4158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:20.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.331+0000 7fc3d359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d416c900 con 0x7fc3d41597f0 2023-10-06T03:51:20.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.331+0000 7fc3d359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d416bcf0 con 0x7fc3d4159420 2023-10-06T03:51:20.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.331+0000 7fc3d359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d416bfd0 con 0x7fc3d415d230 2023-10-06T03:51:20.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.331+0000 7fc3d1d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3d41597f0 0x7fc3d4157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:20.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.331+0000 7fc3d1d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3d41597f0 0x7fc3d4157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41282/0 (socket says 172.21.15.55:41282) 2023-10-06T03:51:20.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.331+0000 7fc3d1d9b700 1 -- 172.21.15.55:0/2791214236 learned_addr learned my addr 172.21.15.55:0/2791214236 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:20.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.331+0000 7fc3d2d9d700 1 --2- 172.21.15.55:0/2791214236 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3d415d230 0x7fc3d4158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:20.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.331+0000 7fc3d259c700 1 --2- 172.21.15.55:0/2791214236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3d4159420 0x7fc3d41431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:20.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.332+0000 7fc3d1d9b700 1 -- 172.21.15.55:0/2791214236 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3d415d230 msgr2=0x7fc3d4158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:20.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.332+0000 7fc3d1d9b700 1 --2- 172.21.15.55:0/2791214236 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3d415d230 0x7fc3d4158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:20.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.332+0000 7fc3d1d9b700 1 -- 172.21.15.55:0/2791214236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3d4159420 msgr2=0x7fc3d41431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:20.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.332+0000 7fc3d1d9b700 1 --2- 172.21.15.55:0/2791214236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3d4159420 0x7fc3d41431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:20.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.332+0000 7fc3d1d9b700 1 -- 172.21.15.55:0/2791214236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c8009420 con 0x7fc3d41597f0 2023-10-06T03:51:20.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.332+0000 7fc3d1d9b700 1 --2- 172.21.15.55:0/2791214236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3d41597f0 0x7fc3d4157e00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8013000 tx=0x7fc3c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:20.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.332+0000 7fc3c27fc700 1 -- 172.21.15.55:0/2791214236 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c800b9d0 con 0x7fc3d41597f0 2023-10-06T03:51:20.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.332+0000 7fc3c27fc700 1 -- 172.21.15.55:0/2791214236 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c800bb30 con 0x7fc3d41597f0 2023-10-06T03:51:20.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.332+0000 7fc3d359e700 1 -- 172.21.15.55:0/2791214236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d416c2b0 con 0x7fc3d41597f0 2023-10-06T03:51:20.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.332+0000 7fc3c27fc700 1 -- 172.21.15.55:0/2791214236 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8025650 con 0x7fc3d41597f0 2023-10-06T03:51:20.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.332+0000 7fc3d359e700 1 -- 172.21.15.55:0/2791214236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d416cd90 con 0x7fc3d41597f0 2023-10-06T03:51:20.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.334+0000 7fc3d359e700 1 -- 172.21.15.55:0/2791214236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d416cf80 con 0x7fc3d41597f0 2023-10-06T03:51:20.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.334+0000 7fc3c27fc700 1 -- 172.21.15.55:0/2791214236 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc3c8023070 con 0x7fc3d41597f0 2023-10-06T03:51:20.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.336+0000 7fc3c27fc700 1 --2- 172.21.15.55:0/2791214236 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc3b4066f30 0x7fc3b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:20.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.336+0000 7fc3c27fc700 1 -- 172.21.15.55:0/2791214236 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6286+0+0 (secure 0 0 0) 0x7fc3c8085d00 con 0x7fc3d41597f0 2023-10-06T03:51:20.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.337+0000 7fc3d259c700 1 --2- 172.21.15.55:0/2791214236 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc3b4066f30 0x7fc3b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:20.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.337+0000 7fc3d259c700 1 --2- 172.21.15.55:0/2791214236 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc3b4066f30 0x7fc3b40693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4007130 tx=0x7fc3c4007eb0 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:20.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.340+0000 7fc3c27fc700 1 -- 172.21.15.55:0/2791214236 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3c8052c90 con 0x7fc3d41597f0 2023-10-06T03:51:20.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:20.558+0000 7fc3d359e700 1 -- 172.21.15.55:0/2791214236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc3d404c3c0 con 0x7fc3d41597f0 2023-10-06T03:51:21.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.776+0000 7fc3c27fc700 1 -- 172.21.15.55:0/2791214236 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v119) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc3c80562b0 con 0x7fc3d41597f0 2023-10-06T03:51:21.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.779+0000 7fc3d359e700 1 -- 172.21.15.55:0/2791214236 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc3b4066f30 msgr2=0x7fc3b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:21.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.779+0000 7fc3d359e700 1 --2- 172.21.15.55:0/2791214236 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc3b4066f30 0x7fc3b40693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4007130 tx=0x7fc3c4007eb0 comp rx=0 tx=0).stop 2023-10-06T03:51:21.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.779+0000 7fc3d359e700 1 -- 172.21.15.55:0/2791214236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3d41597f0 msgr2=0x7fc3d4157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:21.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.779+0000 7fc3d359e700 1 --2- 172.21.15.55:0/2791214236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3d41597f0 0x7fc3d4157e00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8013000 tx=0x7fc3c8017000 comp rx=0 tx=0).stop 2023-10-06T03:51:21.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.779+0000 7fc3d359e700 1 -- 172.21.15.55:0/2791214236 shutdown_connections 2023-10-06T03:51:21.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.779+0000 7fc3d359e700 1 --2- 172.21.15.55:0/2791214236 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc3b4066f30 0x7fc3b40693e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:21.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.779+0000 7fc3d359e700 1 --2- 172.21.15.55:0/2791214236 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3d415d230 0x7fc3d4158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:21.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.779+0000 7fc3d359e700 1 --2- 172.21.15.55:0/2791214236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3d4159420 0x7fc3d41431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:21.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.779+0000 7fc3d359e700 1 --2- 172.21.15.55:0/2791214236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3d41597f0 0x7fc3d4157e00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:21.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.779+0000 7fc3d359e700 1 -- 172.21.15.55:0/2791214236 >> 172.21.15.55:0/2791214236 conn(0x7fc3d40d0660 msgr2=0x7fc3d4068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:21.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.779+0000 7fc3d359e700 1 -- 172.21.15.55:0/2791214236 shutdown_connections 2023-10-06T03:51:21.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.779+0000 7fc3d359e700 1 -- 172.21.15.55:0/2791214236 wait complete. 2023-10-06T03:51:21.782 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2023-10-06T03:51:21.801 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:51:21.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.992+0000 7f23d0f9d700 1 Processor -- start 2023-10-06T03:51:21.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.992+0000 7f23d0f9d700 1 -- start start 2023-10-06T03:51:21.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.994+0000 7f23d0f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23cc1566e0 0x7f23cc0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:21.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.994+0000 7f23d0f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23cc0cb900 0x7f23cc0ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:21.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.994+0000 7f23d0f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23cc0ca860 con 0x7f23cc156310 2023-10-06T03:51:21.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.994+0000 7f23d0f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23cc0cab40 con 0x7f23cc1566e0 2023-10-06T03:51:21.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.994+0000 7f23d0f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0cae20 con 0x7f23cc0cb900 2023-10-06T03:51:21.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.994+0000 7f23caffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23cc1566e0 0x7f23cc0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:21.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.994+0000 7f23cbfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23cc0cb900 0x7f23cc0ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:21.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.994+0000 7f23cbfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23cc0cb900 0x7f23cc0ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:58940/0 (socket says 172.21.15.55:58940) 2023-10-06T03:51:21.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.994+0000 7f23caffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f23cc1566e0 0x7f23cc0ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:43356/0 (socket says 172.21.15.55:43356) 2023-10-06T03:51:21.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.995+0000 7f23cbfff700 1 -- 172.21.15.55:0/1119157157 learned_addr learned my addr 172.21.15.55:0/1119157157 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:21.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.994+0000 7f23cb7fe700 1 --1- 172.21.15.55:0/1119157157 >> v1:172.21.15.163:6789/0 conn(0x7f23cc156310 0x7f23cc0c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:53386/0 (socket says 172.21.15.55:53386) 2023-10-06T03:51:21.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.995+0000 7f23cbfff700 1 -- 172.21.15.55:0/1119157157 >> v2:172.21.15.163:3300/0 conn(0x7f23cc1566e0 msgr2=0x7f23cc0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:21.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.995+0000 7f23cbfff700 1 --2- 172.21.15.55:0/1119157157 >> v2:172.21.15.163:3300/0 conn(0x7f23cc1566e0 0x7f23cc0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:21.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.995+0000 7f23cbfff700 1 -- 172.21.15.55:0/1119157157 >> v1:172.21.15.163:6789/0 conn(0x7f23cc156310 legacy=0x7f23cc0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:21.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.995+0000 7f23cbfff700 1 -- 172.21.15.55:0/1119157157 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc0cb100 con 0x7f23cc0cb900 2023-10-06T03:51:21.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.995+0000 7f23caffd700 1 --2- 172.21.15.55:0/1119157157 >> v2:172.21.15.163:3300/0 conn(0x7f23cc1566e0 0x7f23cc0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:21.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.995+0000 7f23cbfff700 1 --2- 172.21.15.55:0/1119157157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23cc0cb900 0x7f23cc0ae540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009f20 tx=0x7f23c4009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f3969a247a90eee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:21.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.996+0000 7f23ca7fc700 1 -- 172.21.15.55:0/1119157157 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c400e070 con 0x7f23cc0cb900 2023-10-06T03:51:21.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.996+0000 7f23ca7fc700 1 -- 172.21.15.55:0/1119157157 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c400b9d0 con 0x7f23cc0cb900 2023-10-06T03:51:21.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.996+0000 7f23ca7fc700 1 -- 172.21.15.55:0/1119157157 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40092e0 con 0x7f23cc0cb900 2023-10-06T03:51:21.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.996+0000 7f23d0f9d700 1 -- 172.21.15.55:0/1119157157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23cc0cb900 msgr2=0x7f23cc0ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:21.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.996+0000 7f23d0f9d700 1 --2- 172.21.15.55:0/1119157157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23cc0cb900 0x7f23cc0ae540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009f20 tx=0x7f23c4009840 comp rx=0 tx=0).stop 2023-10-06T03:51:21.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.996+0000 7f23d0f9d700 1 -- 172.21.15.55:0/1119157157 shutdown_connections 2023-10-06T03:51:21.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.996+0000 7f23d0f9d700 1 --2- 172.21.15.55:0/1119157157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23cc0cb900 0x7f23cc0ae540 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:21.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.996+0000 7f23d0f9d700 1 --2- 172.21.15.55:0/1119157157 >> v2:172.21.15.163:3300/0 conn(0x7f23cc1566e0 0x7f23cc0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.996+0000 7f23d0f9d700 1 -- 172.21.15.55:0/1119157157 >> 172.21.15.55:0/1119157157 conn(0x7f23cc0cc1f0 msgr2=0x7f23cc0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:22.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.996+0000 7f23d0f9d700 1 -- 172.21.15.55:0/1119157157 shutdown_connections 2023-10-06T03:51:22.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.996+0000 7f23d0f9d700 1 -- 172.21.15.55:0/1119157157 wait complete. 2023-10-06T03:51:22.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.998+0000 7f23d0f9d700 1 Processor -- start 2023-10-06T03:51:22.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.998+0000 7f23d0f9d700 1 -- start start 2023-10-06T03:51:22.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.999+0000 7f23d0f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23cc0cb900 0x7f23cc16a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.999+0000 7f23d0f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23cc156310 0x7f23cc169f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.999+0000 7f23d0f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23cc1566e0 0x7f23cc1670e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.999+0000 7f23d0f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc16a4d0 con 0x7f23cc156310 2023-10-06T03:51:22.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.999+0000 7f23d0f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc167620 con 0x7f23cc0cb900 2023-10-06T03:51:22.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.999+0000 7f23d0f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc16ac50 con 0x7f23cc1566e0 2023-10-06T03:51:22.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.999+0000 7f23caffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23cc156310 0x7f23cc169f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.999+0000 7f23caffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23cc156310 0x7f23cc169f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41286/0 (socket says 172.21.15.55:41286) 2023-10-06T03:51:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:21.999+0000 7f23caffd700 1 -- 172.21.15.55:0/939040857 learned_addr learned my addr 172.21.15.55:0/939040857 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23cb7fe700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23cc0cb900 0x7f23cc16a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23cbfff700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23cc1566e0 0x7f23cc1670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23caffd700 1 -- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23cc1566e0 msgr2=0x7f23cc1670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23caffd700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23cc1566e0 0x7f23cc1670e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23caffd700 1 -- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23cc0cb900 msgr2=0x7f23cc16a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23caffd700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23cc0cb900 0x7f23cc16a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23caffd700 1 -- 172.21.15.55:0/939040857 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c000b6d0 con 0x7f23cc156310 2023-10-06T03:51:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23cbfff700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23cc1566e0 0x7f23cc1670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:51:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23cb7fe700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23cc0cb900 0x7f23cc16a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:51:22.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23caffd700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23cc156310 0x7f23cc169f90 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f23c000ed70 tx=0x7f23c000b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:22.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23b37fe700 1 -- 172.21.15.55:0/939040857 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c000dde0 con 0x7f23cc156310 2023-10-06T03:51:22.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23b37fe700 1 -- 172.21.15.55:0/939040857 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c0010440 con 0x7f23cc156310 2023-10-06T03:51:22.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23b37fe700 1 -- 172.21.15.55:0/939040857 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c00107e0 con 0x7f23cc156310 2023-10-06T03:51:22.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23d0f9d700 1 -- 172.21.15.55:0/939040857 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c4009ae0 con 0x7f23cc156310 2023-10-06T03:51:22.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.000+0000 7f23d0f9d700 1 -- 172.21.15.55:0/939040857 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc16b210 con 0x7f23cc156310 2023-10-06T03:51:22.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.001+0000 7f23d0f9d700 1 -- 172.21.15.55:0/939040857 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc04c3c0 con 0x7f23cc156310 2023-10-06T03:51:22.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.002+0000 7f23b37fe700 1 -- 172.21.15.55:0/939040857 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f23c0010b50 con 0x7f23cc156310 2023-10-06T03:51:22.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.003+0000 7f23b37fe700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f23b8066f30 0x7f23b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.004+0000 7f23b37fe700 1 -- 172.21.15.55:0/939040857 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f23c0084e60 con 0x7f23cc156310 2023-10-06T03:51:22.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.004+0000 7f23cb7fe700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f23b8066f30 0x7f23b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.005+0000 7f23cb7fe700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f23b8066f30 0x7f23b80693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f23bc001820 tx=0x7f23bc009040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:22.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.008+0000 7f23b37fe700 1 -- 172.21.15.55:0/939040857 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23c00520a0 con 0x7f23cc156310 2023-10-06T03:51:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.288+0000 7f23d0f9d700 1 -- 172.21.15.55:0/939040857 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f23cc0cd0b0 con 0x7f23cc156310 2023-10-06T03:51:22.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.288+0000 7f23b37fe700 1 -- 172.21.15.55:0/939040857 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f23c00556c0 con 0x7f23cc156310 2023-10-06T03:51:22.290 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:22.290 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:51:22.290 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2023-10-06T03:51:22.290 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.290 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2023-10-06T03:51:22.290 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9li2RaGRAAgRjtbw3h40vylEPjolzlpw==", 2023-10-06T03:51:22.291 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.291 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:51:22.291 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:51:22.291 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:51:22.291 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.291 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.291 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.291 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.291 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2023-10-06T03:51:22.291 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lXZu7LxAAu3QAtdDxtD/AuuqGG12sDw==", 2023-10-06T03:51:22.291 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.291 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:51:22.292 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:51:22.292 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:51:22.292 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.292 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.292 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.292 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.292 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2023-10-06T03:51:22.292 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lS22fJBAAZi6CfyLyJAFgIkaarad1mA==", 2023-10-06T03:51:22.292 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.292 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:51:22.292 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:51:22.292 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:51:22.293 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.293 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.293 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.293 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.293 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2023-10-06T03:51:22.293 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9ll0RIOhAAGtGdPv4Y2bge7xIv3eu2NQ==", 2023-10-06T03:51:22.293 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.294 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:51:22.294 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:51:22.294 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:51:22.294 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.294 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.294 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.294 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.294 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2023-10-06T03:51:22.294 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDHfx9lmTeaDRAAuycc9i7fPj8xrDkuZCAGVg==", 2023-10-06T03:51:22.295 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.295 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:51:22.295 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:51:22.295 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.295 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.295 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.295 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.295 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2023-10-06T03:51:22.295 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDKfx9lxOdXEhAA9S8tdOvhnmBeEbutqnUsZg==", 2023-10-06T03:51:22.295 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.296 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:51:22.296 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:51:22.296 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.296 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.296 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.296 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.296 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2023-10-06T03:51:22.296 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDNfx9l76I+GRAAjaj0gM7An9BCyGmfQgaDFw==", 2023-10-06T03:51:22.296 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.296 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:51:22.296 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:51:22.296 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.297 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.297 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.297 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.297 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2023-10-06T03:51:22.297 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDQfx9lsPK+HxAAzQCV0EuBenqOkUpiQW9uog==", 2023-10-06T03:51:22.297 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.297 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:51:22.297 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:51:22.297 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.297 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.297 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.297 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.297 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2023-10-06T03:51:22.298 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDWfx9lSHNrMxAAIn7awDmljO2cRtKyDalWnw==", 2023-10-06T03:51:22.298 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.298 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:51:22.298 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:51:22.298 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.298 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.298 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.298 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.298 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2023-10-06T03:51:22.298 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDZfx9lFoWuOBAAhzQOt3YChvuv0MhrspOjPw==", 2023-10-06T03:51:22.298 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.299 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:51:22.299 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:51:22.299 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.299 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.299 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.299 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.299 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2023-10-06T03:51:22.299 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDcfx9lB8iSOhAAwamSgGIXZ8owFE4FdbOOBw==", 2023-10-06T03:51:22.299 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.299 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:51:22.299 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:51:22.299 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.300 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.300 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.300 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.300 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2023-10-06T03:51:22.300 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDgfx9lUb3mABAAJ75UO2UQg2r+l+tDYVn0Dg==", 2023-10-06T03:51:22.300 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.300 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:51:22.300 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:51:22.300 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.300 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.300 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.300 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.301 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2023-10-06T03:51:22.301 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lfNvCBxAA1ZJq4XsWTZHVY0u0NdOqIw==", 2023-10-06T03:51:22.301 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.301 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:51:22.301 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2023-10-06T03:51:22.301 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:51:22.301 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.301 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.301 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.301 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2023-10-06T03:51:22.301 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lnvkoFhAAafxsOmItxmwUr1GJrGYJ6g==", 2023-10-06T03:51:22.301 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.302 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:51:22.302 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:51:22.302 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:51:22.302 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:51:22.302 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.302 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.302 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.302 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2023-10-06T03:51:22.302 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9l+9b4DhAAc5XHdikWbXsEblSE3FGScQ==", 2023-10-06T03:51:22.302 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.302 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:51:22.302 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:51:22.302 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:51:22.303 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:51:22.303 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.303 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.303 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.303 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2023-10-06T03:51:22.303 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9luSXrHBAAg6Ufa1Ep/EOgsQtuOPQgYg==", 2023-10-06T03:51:22.303 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.303 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:51:22.303 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:51:22.303 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:51:22.303 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:51:22.303 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.304 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.304 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.304 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2023-10-06T03:51:22.304 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC+fx9lEb50FxAAt0Oq28gsXtQvol7TV8trgg==", 2023-10-06T03:51:22.304 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.304 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:51:22.304 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:51:22.304 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:51:22.304 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.304 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.304 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.304 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.305 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2023-10-06T03:51:22.305 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lQAANEhAAsNiWPqCNEyUS3IELXSC1Cw==", 2023-10-06T03:51:22.305 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.305 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:51:22.305 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T03:51:22.305 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.305 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.305 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:22.305 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:22.305 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2023-10-06T03:51:22.305 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lCwU2ABAAMijxXd8hwgk3XzcdP4NZyg==", 2023-10-06T03:51:22.305 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:51:22.306 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:51:22.306 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T03:51:22.306 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:51:22.306 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.306 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:22.306 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2023-10-06T03:51:22.306 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:51:22.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.290+0000 7f23d0f9d700 1 -- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f23b8066f30 msgr2=0x7f23b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:22.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.290+0000 7f23d0f9d700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f23b8066f30 0x7f23b80693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f23bc001820 tx=0x7f23bc009040 comp rx=0 tx=0).stop 2023-10-06T03:51:22.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.290+0000 7f23d0f9d700 1 -- 172.21.15.55:0/939040857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23cc156310 msgr2=0x7f23cc169f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:22.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.290+0000 7f23d0f9d700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23cc156310 0x7f23cc169f90 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f23c000ed70 tx=0x7f23c000b430 comp rx=0 tx=0).stop 2023-10-06T03:51:22.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.291+0000 7f23d0f9d700 1 -- 172.21.15.55:0/939040857 shutdown_connections 2023-10-06T03:51:22.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.291+0000 7f23d0f9d700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f23b8066f30 0x7f23b80693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.291+0000 7f23d0f9d700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f23cc1566e0 0x7f23cc1670e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.291+0000 7f23d0f9d700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f23cc0cb900 0x7f23cc16a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.291+0000 7f23d0f9d700 1 --2- 172.21.15.55:0/939040857 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f23cc156310 0x7f23cc169f90 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.291+0000 7f23d0f9d700 1 -- 172.21.15.55:0/939040857 >> 172.21.15.55:0/939040857 conn(0x7f23cc0cc1f0 msgr2=0x7f23cc0addb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:22.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.291+0000 7f23d0f9d700 1 -- 172.21.15.55:0/939040857 shutdown_connections 2023-10-06T03:51:22.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.291+0000 7f23d0f9d700 1 -- 172.21.15.55:0/939040857 wait complete. 2023-10-06T03:51:22.310 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:51:22.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.494+0000 7f9f5759e700 1 Processor -- start 2023-10-06T03:51:22.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.495+0000 7f9f5759e700 1 -- start start 2023-10-06T03:51:22.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.496+0000 7f9f5759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f5806d950 0x7f9f580c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.496+0000 7f9f5759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9f5806a0a0 0x7f9f580c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.496+0000 7f9f5759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5806ae70 con 0x7f9f5806dd20 2023-10-06T03:51:22.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.496+0000 7f9f5759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5806a540 con 0x7f9f5806a0a0 2023-10-06T03:51:22.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.496+0000 7f9f5759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5806a820 con 0x7f9f5806d950 2023-10-06T03:51:22.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.496+0000 7f9f56d9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9f5806dd20 0x7f9f580c7150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57420/0 (socket says 172.21.15.55:57420) 2023-10-06T03:51:22.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.496+0000 7f9f56d9d700 1 -- 172.21.15.55:0/2599906216 learned_addr learned my addr 172.21.15.55:0/2599906216 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.496+0000 7f9f55d9b700 1 --2- 172.21.15.55:0/2599906216 >> v2:172.21.15.163:3300/0 conn(0x7f9f5806a0a0 0x7f9f580c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.496+0000 7f9f5659c700 1 --2- 172.21.15.55:0/2599906216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f5806d950 0x7f9f580c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 -- 172.21.15.55:0/2599906216 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100294880 0 0) 0x7f9f5806ae70 con 0x7f9f5806dd20 2023-10-06T03:51:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 -- 172.21.15.55:0/2599906216 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f40003570 con 0x7f9f5806dd20 2023-10-06T03:51:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 -- 172.21.15.55:0/2599906216 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 715676630 0 0) 0x7f9f40003570 con 0x7f9f5806dd20 2023-10-06T03:51:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 -- 172.21.15.55:0/2599906216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f5806d950 msgr2=0x7f9f580c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 --2- 172.21.15.55:0/2599906216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f5806d950 0x7f9f580c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 -- 172.21.15.55:0/2599906216 >> v2:172.21.15.163:3300/0 conn(0x7f9f5806a0a0 msgr2=0x7f9f580c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 --2- 172.21.15.55:0/2599906216 >> v2:172.21.15.163:3300/0 conn(0x7f9f5806a0a0 0x7f9f580c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 -- 172.21.15.55:0/2599906216 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5806ab00 con 0x7f9f5806dd20 2023-10-06T03:51:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 -- 172.21.15.55:0/2599906216 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f9f500017f0 con 0x7f9f5806dd20 2023-10-06T03:51:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 -- 172.21.15.55:0/2599906216 >> v1:172.21.15.55:6789/0 conn(0x7f9f5806dd20 legacy=0x7f9f580c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 --2- 172.21.15.55:0/2599906216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f5806d950 0x7f9f40003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 --2- 172.21.15.55:0/2599906216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f40003fe0 0x7f9f40004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 --2- 172.21.15.55:0/2599906216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f40004d50 0x7f9f400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 -- 172.21.15.55:0/2599906216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5806a540 con 0x7f9f40003fe0 2023-10-06T03:51:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 -- 172.21.15.55:0/2599906216 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5806a820 con 0x7f9f40004d50 2023-10-06T03:51:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5559a700 1 -- 172.21.15.55:0/2599906216 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4000ba90 con 0x7f9f5806d950 2023-10-06T03:51:22.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f55d9b700 1 --2- 172.21.15.55:0/2599906216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f40003fe0 0x7f9f40004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.497+0000 7f9f5659c700 1 --2- 172.21.15.55:0/2599906216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f5806d950 0x7f9f40003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.498+0000 7f9f56d9d700 1 --2- 172.21.15.55:0/2599906216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f40004d50 0x7f9f400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.498+0000 7f9f55d9b700 1 -- 172.21.15.55:0/2599906216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f5806d950 msgr2=0x7f9f40003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.498+0000 7f9f55d9b700 1 --2- 172.21.15.55:0/2599906216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f5806d950 0x7f9f40003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.498+0000 7f9f55d9b700 1 -- 172.21.15.55:0/2599906216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f40004d50 msgr2=0x7f9f400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.498+0000 7f9f55d9b700 1 --2- 172.21.15.55:0/2599906216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f40004d50 0x7f9f400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.498+0000 7f9f55d9b700 1 -- 172.21.15.55:0/2599906216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f40003570 con 0x7f9f40003fe0 2023-10-06T03:51:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.498+0000 7f9f55d9b700 1 --2- 172.21.15.55:0/2599906216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f40003fe0 0x7f9f40004810 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9f5806a470 tx=0x7f9f4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b17bbe65a0497c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.498+0000 7f9f5559a700 1 -- 172.21.15.55:0/2599906216 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c00e6b0 con 0x7f9f40003fe0 2023-10-06T03:51:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.498+0000 7f9f5559a700 1 -- 172.21.15.55:0/2599906216 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c00e810 con 0x7f9f40003fe0 2023-10-06T03:51:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.498+0000 7f9f5559a700 1 -- 172.21.15.55:0/2599906216 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c00ebd0 con 0x7f9f40003fe0 2023-10-06T03:51:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.499+0000 7f9f5759e700 1 -- 172.21.15.55:0/2599906216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f40003fe0 msgr2=0x7f9f40004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:22.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.499+0000 7f9f5759e700 1 --2- 172.21.15.55:0/2599906216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f40003fe0 0x7f9f40004810 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9f5806a470 tx=0x7f9f4c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.499+0000 7f9f56d9d700 1 -- 172.21.15.55:0/2599906216 reap_dead start 2023-10-06T03:51:22.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.499+0000 7f9f5759e700 1 -- 172.21.15.55:0/2599906216 shutdown_connections 2023-10-06T03:51:22.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.499+0000 7f9f5759e700 1 -- 172.21.15.55:0/2599906216 >> 172.21.15.55:0/2599906216 conn(0x7f9f580cc1f0 msgr2=0x7f9f580cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:22.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.499+0000 7f9f5759e700 1 -- 172.21.15.55:0/2599906216 shutdown_connections 2023-10-06T03:51:22.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.499+0000 7f9f5759e700 1 -- 172.21.15.55:0/2599906216 wait complete. 2023-10-06T03:51:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.500+0000 7f9f5759e700 1 Processor -- start 2023-10-06T03:51:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.501+0000 7f9f5759e700 1 -- start start 2023-10-06T03:51:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.501+0000 7f9f5759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f40003fe0 0x7f9f5813fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.501+0000 7f9f5759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f580c0f90 0x7f9f580c1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.501+0000 7f9f5759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f580c5250 0x7f9f580c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.501+0000 7f9f5759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f580c1940 con 0x7f9f580c5250 2023-10-06T03:51:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.501+0000 7f9f5759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5816bc10 con 0x7f9f40003fe0 2023-10-06T03:51:22.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.501+0000 7f9f5759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f580c6010 con 0x7f9f580c0f90 2023-10-06T03:51:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.502+0000 7f9f56d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f580c5250 0x7f9f580c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.502+0000 7f9f56d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f580c5250 0x7f9f580c56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41302/0 (socket says 172.21.15.55:41302) 2023-10-06T03:51:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.502+0000 7f9f56d9d700 1 -- 172.21.15.55:0/1564038968 learned_addr learned my addr 172.21.15.55:0/1564038968 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.502+0000 7f9f5659c700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f40003fe0 0x7f9f5813fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.502+0000 7f9f55d9b700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f580c0f90 0x7f9f580c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.502+0000 7f9f56d9d700 1 -- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f580c0f90 msgr2=0x7f9f580c1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.502+0000 7f9f56d9d700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f580c0f90 0x7f9f580c1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.502+0000 7f9f56d9d700 1 -- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f40003fe0 msgr2=0x7f9f5813fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.502+0000 7f9f56d9d700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f40003fe0 0x7f9f5813fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.502+0000 7f9f56d9d700 1 -- 172.21.15.55:0/1564038968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5816b2e0 con 0x7f9f580c5250 2023-10-06T03:51:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.502+0000 7f9f5659c700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f40003fe0 0x7f9f5813fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:22.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.502+0000 7f9f55d9b700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f580c0f90 0x7f9f580c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:22.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.502+0000 7f9f56d9d700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f580c5250 0x7f9f580c56c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f9f5806e3f0 tx=0x7f9f50002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:22.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.503+0000 7f9f467fc700 1 -- 172.21.15.55:0/1564038968 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50000ff0 con 0x7f9f580c5250 2023-10-06T03:51:22.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.503+0000 7f9f467fc700 1 -- 172.21.15.55:0/1564038968 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5000ad20 con 0x7f9f580c5250 2023-10-06T03:51:22.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.503+0000 7f9f467fc700 1 -- 172.21.15.55:0/1564038968 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50013640 con 0x7f9f580c5250 2023-10-06T03:51:22.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.503+0000 7f9f5759e700 1 -- 172.21.15.55:0/1564038968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5816b5c0 con 0x7f9f580c5250 2023-10-06T03:51:22.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.503+0000 7f9f5759e700 1 -- 172.21.15.55:0/1564038968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5816c0a0 con 0x7f9f580c5250 2023-10-06T03:51:22.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.504+0000 7f9f5759e700 1 -- 172.21.15.55:0/1564038968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5804c3c0 con 0x7f9f580c5250 2023-10-06T03:51:22.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.505+0000 7f9f467fc700 1 -- 172.21.15.55:0/1564038968 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f50013a30 con 0x7f9f580c5250 2023-10-06T03:51:22.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.505+0000 7f9f467fc700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9f38066f30 0x7f9f380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.505+0000 7f9f467fc700 1 -- 172.21.15.55:0/1564038968 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f9f50084cd0 con 0x7f9f580c5250 2023-10-06T03:51:22.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.506+0000 7f9f5659c700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9f38066f30 0x7f9f380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.507+0000 7f9f5659c700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9f38066f30 0x7f9f380693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9f48006dc0 tx=0x7f9f48008040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:22.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.510+0000 7f9f467fc700 1 -- 172.21.15.55:0/1564038968 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f50051f10 con 0x7f9f580c5250 2023-10-06T03:51:22.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.710+0000 7f9f5759e700 1 -- 172.21.15.55:0/1564038968 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f580c9000 con 0x7f9f580c5250 2023-10-06T03:51:22.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.711+0000 7f9f467fc700 1 -- 172.21.15.55:0/1564038968 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+14303 (secure 0 0 0) 0x7f9f50055530 con 0x7f9f580c5250 2023-10-06T03:51:22.713 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:22.713 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":119,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:51:21.772120+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:51:22.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.713+0000 7f9f5759e700 1 -- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9f38066f30 msgr2=0x7f9f380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:22.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.714+0000 7f9f5759e700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9f38066f30 0x7f9f380693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9f48006dc0 tx=0x7f9f48008040 comp rx=0 tx=0).stop 2023-10-06T03:51:22.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.714+0000 7f9f5759e700 1 -- 172.21.15.55:0/1564038968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f580c5250 msgr2=0x7f9f580c56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:22.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.714+0000 7f9f5759e700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f580c5250 0x7f9f580c56c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f9f5806e3f0 tx=0x7f9f50002210 comp rx=0 tx=0).stop 2023-10-06T03:51:22.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.714+0000 7f9f5759e700 1 -- 172.21.15.55:0/1564038968 shutdown_connections 2023-10-06T03:51:22.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.714+0000 7f9f5759e700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9f38066f30 0x7f9f380693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.714+0000 7f9f5759e700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f580c5250 0x7f9f580c56c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.714+0000 7f9f5759e700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f40003fe0 0x7f9f5813fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.714+0000 7f9f5759e700 1 --2- 172.21.15.55:0/1564038968 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f580c0f90 0x7f9f580c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.714+0000 7f9f5759e700 1 -- 172.21.15.55:0/1564038968 >> 172.21.15.55:0/1564038968 conn(0x7f9f580cc1f0 msgr2=0x7f9f580b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:22.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.714+0000 7f9f5759e700 1 -- 172.21.15.55:0/1564038968 shutdown_connections 2023-10-06T03:51:22.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.714+0000 7f9f5759e700 1 -- 172.21.15.55:0/1564038968 wait complete. 2023-10-06T03:51:22.733 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T03:51:22.733 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:51:22.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.922+0000 7f7ab45ff700 1 Processor -- start 2023-10-06T03:51:22.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.923+0000 7f7ab45ff700 1 -- start start 2023-10-06T03:51:22.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.924+0000 7f7ab45ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aac155eb0 0x7f7aac06e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.924+0000 7f7ab45ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7aac0c70b0 0x7f7aac069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.924+0000 7f7ab45ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7aac069cb0 0x7f7aac159a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.924+0000 7f7ab45ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aac06a130 con 0x7f7aac0c70b0 2023-10-06T03:51:22.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.924+0000 7f7ab45ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aac06eb90 con 0x7f7aac069cb0 2023-10-06T03:51:22.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.924+0000 7f7ab45ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac06ee70 con 0x7f7aac155eb0 2023-10-06T03:51:22.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.924+0000 7f7ab1b9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7aac0c70b0 0x7f7aac069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.924+0000 7f7ab1b9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7aac0c70b0 0x7f7aac069770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41314/0 (socket says 172.21.15.55:41314) 2023-10-06T03:51:22.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.924+0000 7f7ab1b9a700 1 -- 172.21.15.55:0/2002308976 learned_addr learned my addr 172.21.15.55:0/2002308976 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:22.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.924+0000 7f7ab2b9c700 1 --2- 172.21.15.55:0/2002308976 >> v2:172.21.15.163:3300/0 conn(0x7f7aac069cb0 0x7f7aac159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.924+0000 7f7ab239b700 1 --2- 172.21.15.55:0/2002308976 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aac155eb0 0x7f7aac06e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.925+0000 7f7ab1b9a700 1 -- 172.21.15.55:0/2002308976 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aac155eb0 msgr2=0x7f7aac06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:22.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.925+0000 7f7ab1b9a700 1 --2- 172.21.15.55:0/2002308976 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aac155eb0 0x7f7aac06e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.925+0000 7f7ab1b9a700 1 -- 172.21.15.55:0/2002308976 >> v2:172.21.15.163:3300/0 conn(0x7f7aac069cb0 msgr2=0x7f7aac159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:22.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.925+0000 7f7ab1b9a700 1 --2- 172.21.15.55:0/2002308976 >> v2:172.21.15.163:3300/0 conn(0x7f7aac069cb0 0x7f7aac159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.925+0000 7f7ab1b9a700 1 -- 172.21.15.55:0/2002308976 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac06f150 con 0x7f7aac0c70b0 2023-10-06T03:51:22.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.925+0000 7f7ab1b9a700 1 --2- 172.21.15.55:0/2002308976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aac0c70b0 0x7f7aac069770 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0c7740 tx=0x7f7a9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad933bec844c3f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:22.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.925+0000 7f7ab1399700 1 -- 172.21.15.55:0/2002308976 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00b640 con 0x7f7aac0c70b0 2023-10-06T03:51:22.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.925+0000 7f7ab1399700 1 -- 172.21.15.55:0/2002308976 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c00b7a0 con 0x7f7aac0c70b0 2023-10-06T03:51:22.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.925+0000 7f7ab1399700 1 -- 172.21.15.55:0/2002308976 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00bb60 con 0x7f7aac0c70b0 2023-10-06T03:51:22.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.926+0000 7f7ab45ff700 1 -- 172.21.15.55:0/2002308976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aac0c70b0 msgr2=0x7f7aac069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:22.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.926+0000 7f7ab45ff700 1 --2- 172.21.15.55:0/2002308976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aac0c70b0 0x7f7aac069770 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0c7740 tx=0x7f7a9c009250 comp rx=0 tx=0).stop 2023-10-06T03:51:22.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.926+0000 7f7ab45ff700 1 -- 172.21.15.55:0/2002308976 shutdown_connections 2023-10-06T03:51:22.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.926+0000 7f7ab45ff700 1 --2- 172.21.15.55:0/2002308976 >> v2:172.21.15.163:3300/0 conn(0x7f7aac069cb0 0x7f7aac159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.926+0000 7f7ab45ff700 1 --2- 172.21.15.55:0/2002308976 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aac155eb0 0x7f7aac06e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.926+0000 7f7ab45ff700 1 --2- 172.21.15.55:0/2002308976 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aac0c70b0 0x7f7aac069770 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.926+0000 7f7ab45ff700 1 -- 172.21.15.55:0/2002308976 >> 172.21.15.55:0/2002308976 conn(0x7f7aac068e10 msgr2=0x7f7aac0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:22.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.926+0000 7f7ab45ff700 1 -- 172.21.15.55:0/2002308976 shutdown_connections 2023-10-06T03:51:22.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.926+0000 7f7ab45ff700 1 -- 172.21.15.55:0/2002308976 wait complete. 2023-10-06T03:51:22.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.927+0000 7f7ab45ff700 1 Processor -- start 2023-10-06T03:51:22.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.928+0000 7f7ab45ff700 1 -- start start 2023-10-06T03:51:22.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.928+0000 7f7ab45ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aac069cb0 0x7f7aac154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab45ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aac0c70b0 0x7f7aac0c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab45ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aac155eb0 0x7f7aac153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab45ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac155450 con 0x7f7aac155eb0 2023-10-06T03:51:22.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab45ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac0b0cc0 con 0x7f7aac0c70b0 2023-10-06T03:51:22.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab45ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac0c3b20 con 0x7f7aac069cb0 2023-10-06T03:51:22.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab2b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aac155eb0 0x7f7aac153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab2b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aac155eb0 0x7f7aac153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41320/0 (socket says 172.21.15.55:41320) 2023-10-06T03:51:22.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab2b9c700 1 -- 172.21.15.55:0/834155306 learned_addr learned my addr 172.21.15.55:0/834155306 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:22.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab1b9a700 1 --2- 172.21.15.55:0/834155306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aac0c70b0 0x7f7aac0c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab239b700 1 --2- 172.21.15.55:0/834155306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aac069cb0 0x7f7aac154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab2b9c700 1 -- 172.21.15.55:0/834155306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aac069cb0 msgr2=0x7f7aac154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:22.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab2b9c700 1 --2- 172.21.15.55:0/834155306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aac069cb0 0x7f7aac154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab2b9c700 1 -- 172.21.15.55:0/834155306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aac0c70b0 msgr2=0x7f7aac0c35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:22.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab2b9c700 1 --2- 172.21.15.55:0/834155306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aac0c70b0 0x7f7aac0c35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:22.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.929+0000 7f7ab2b9c700 1 -- 172.21.15.55:0/834155306 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c0094f0 con 0x7f7aac155eb0 2023-10-06T03:51:22.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.930+0000 7f7ab2b9c700 1 --2- 172.21.15.55:0/834155306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aac155eb0 0x7f7aac153460 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009ec0 tx=0x7f7aa400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:22.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.930+0000 7f7aa27fc700 1 -- 172.21.15.55:0/834155306 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa400cc20 con 0x7f7aac155eb0 2023-10-06T03:51:22.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.930+0000 7f7aa27fc700 1 -- 172.21.15.55:0/834155306 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa400f040 con 0x7f7aac155eb0 2023-10-06T03:51:22.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.930+0000 7f7ab45ff700 1 -- 172.21.15.55:0/834155306 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aac1539a0 con 0x7f7aac155eb0 2023-10-06T03:51:22.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.930+0000 7f7aa27fc700 1 -- 172.21.15.55:0/834155306 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4014480 con 0x7f7aac155eb0 2023-10-06T03:51:22.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.930+0000 7f7ab45ff700 1 -- 172.21.15.55:0/834155306 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aac0b03b0 con 0x7f7aac155eb0 2023-10-06T03:51:22.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.931+0000 7f7ab45ff700 1 -- 172.21.15.55:0/834155306 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aac0b0e50 con 0x7f7aac155eb0 2023-10-06T03:51:22.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.932+0000 7f7aa27fc700 1 -- 172.21.15.55:0/834155306 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7aa4014830 con 0x7f7aac155eb0 2023-10-06T03:51:22.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.933+0000 7f7aa27fc700 1 --2- 172.21.15.55:0/834155306 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a90066f30 0x7f7a900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:22.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.933+0000 7f7aa27fc700 1 -- 172.21.15.55:0/834155306 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f7aa4084f80 con 0x7f7aac155eb0 2023-10-06T03:51:22.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.933+0000 7f7ab239b700 1 --2- 172.21.15.55:0/834155306 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a90066f30 0x7f7a900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:22.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.934+0000 7f7ab239b700 1 --2- 172.21.15.55:0/834155306 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a90066f30 0x7f7a900693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009c10 tx=0x7f7aa8008040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:22.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:22.937+0000 7f7aa27fc700 1 -- 172.21.15.55:0/834155306 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7aa4055c50 con 0x7f7aac155eb0 2023-10-06T03:51:23.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.140+0000 7f7ab45ff700 1 -- 172.21.15.55:0/834155306 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7aac06f430 con 0x7f7aac155eb0 2023-10-06T03:51:23.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.141+0000 7f7aa27fc700 1 -- 172.21.15.55:0/834155306 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v119) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7aa4018020 con 0x7f7aac155eb0 2023-10-06T03:51:23.142 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:51:23.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.143+0000 7f7ab45ff700 1 -- 172.21.15.55:0/834155306 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a90066f30 msgr2=0x7f7a900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.143+0000 7f7ab45ff700 1 --2- 172.21.15.55:0/834155306 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a90066f30 0x7f7a900693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009c10 tx=0x7f7aa8008040 comp rx=0 tx=0).stop 2023-10-06T03:51:23.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.143+0000 7f7ab45ff700 1 -- 172.21.15.55:0/834155306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aac155eb0 msgr2=0x7f7aac153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.143+0000 7f7ab45ff700 1 --2- 172.21.15.55:0/834155306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aac155eb0 0x7f7aac153460 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009ec0 tx=0x7f7aa400b040 comp rx=0 tx=0).stop 2023-10-06T03:51:23.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.143+0000 7f7ab45ff700 1 -- 172.21.15.55:0/834155306 shutdown_connections 2023-10-06T03:51:23.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.143+0000 7f7ab45ff700 1 --2- 172.21.15.55:0/834155306 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a90066f30 0x7f7a900693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.143+0000 7f7ab45ff700 1 --2- 172.21.15.55:0/834155306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aac155eb0 0x7f7aac153460 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.143+0000 7f7ab45ff700 1 --2- 172.21.15.55:0/834155306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aac069cb0 0x7f7aac154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.143+0000 7f7ab45ff700 1 --2- 172.21.15.55:0/834155306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aac0c70b0 0x7f7aac0c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.143+0000 7f7ab45ff700 1 -- 172.21.15.55:0/834155306 >> 172.21.15.55:0/834155306 conn(0x7f7aac068e10 msgr2=0x7f7aac0af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:23.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.143+0000 7f7ab45ff700 1 -- 172.21.15.55:0/834155306 shutdown_connections 2023-10-06T03:51:23.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.143+0000 7f7ab45ff700 1 -- 172.21.15.55:0/834155306 wait complete. 2023-10-06T03:51:23.162 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:51:23.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.350+0000 7f033719d700 1 Processor -- start 2023-10-06T03:51:23.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.350+0000 7f033719d700 1 -- start start 2023-10-06T03:51:23.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.352+0000 7f033719d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0330159a70 0x7f03300c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:23.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.352+0000 7f033719d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03300ca920 con 0x7f033006bfa0 2023-10-06T03:51:23.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.352+0000 7f033719d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f033006af00 con 0x7f0330068aa0 2023-10-06T03:51:23.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.352+0000 7f033719d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033006b1e0 con 0x7f0330159a70 2023-10-06T03:51:23.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.352+0000 7f033599a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f033006bfa0 0x7f0330068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38114/0 (socket says 172.21.15.55:38114) 2023-10-06T03:51:23.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.352+0000 7f033599a700 1 -- 172.21.15.55:0/2181773594 learned_addr learned my addr 172.21.15.55:0/2181773594 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:23.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.352+0000 7f033619b700 1 --2- 172.21.15.55:0/2181773594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0330159a70 0x7f03300c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:23.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.352+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172025593 0 0) 0x7f03300ca920 con 0x7f033006bfa0 2023-10-06T03:51:23.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.352+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f031c003570 con 0x7f033006bfa0 2023-10-06T03:51:23.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402942263 0 0) 0x7f033006af00 con 0x7f0330068aa0 2023-10-06T03:51:23.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03300ca920 con 0x7f0330068aa0 2023-10-06T03:51:23.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3628557779 0 0) 0x7f031c003570 con 0x7f033006bfa0 2023-10-06T03:51:23.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0330159a70 msgr2=0x7f03300c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:23.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 --2- 172.21.15.55:0/2181773594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0330159a70 0x7f03300c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 >> v1:172.21.15.163:6789/0 conn(0x7f0330068aa0 legacy=0x7f03300caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033006b4c0 con 0x7f033006bfa0 2023-10-06T03:51:23.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f03200015f0 con 0x7f033006bfa0 2023-10-06T03:51:23.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 >> v1:172.21.15.55:6789/0 conn(0x7f033006bfa0 legacy=0x7f0330068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 --2- 172.21.15.55:0/2181773594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0330159a70 0x7f031c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:23.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 --2- 172.21.15.55:0/2181773594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f031c0041d0 0x7f031c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:23.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 --2- 172.21.15.55:0/2181773594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f031c004d50 0x7f031c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:23.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033006b1e0 con 0x7f031c0041d0 2023-10-06T03:51:23.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031c00bbf0 con 0x7f031c004d50 2023-10-06T03:51:23.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031c00bdb0 con 0x7f0330159a70 2023-10-06T03:51:23.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f033599a700 1 --2- 172.21.15.55:0/2181773594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f031c0041d0 0x7f031c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:23.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f033699c700 1 --2- 172.21.15.55:0/2181773594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f031c004d50 0x7f031c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:23.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.353+0000 7f033619b700 1 --2- 172.21.15.55:0/2181773594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0330159a70 0x7f031c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:23.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.354+0000 7f033599a700 1 -- 172.21.15.55:0/2181773594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0330159a70 msgr2=0x7f031c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:23.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.354+0000 7f033599a700 1 --2- 172.21.15.55:0/2181773594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0330159a70 0x7f031c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.354+0000 7f033599a700 1 -- 172.21.15.55:0/2181773594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f031c004d50 msgr2=0x7f031c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:23.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.354+0000 7f033599a700 1 --2- 172.21.15.55:0/2181773594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f031c004d50 0x7f031c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.354+0000 7f033599a700 1 -- 172.21.15.55:0/2181773594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c003570 con 0x7f031c0041d0 2023-10-06T03:51:23.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.354+0000 7f033619b700 1 --2- 172.21.15.55:0/2181773594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0330159a70 0x7f031c003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:51:23.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.354+0000 7f033599a700 1 --2- 172.21.15.55:0/2181773594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f031c0041d0 0x7f031c004810 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f032000c210 tx=0x7f0320007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f8e6251fbb89e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:23.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.354+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032000cb00 con 0x7f031c0041d0 2023-10-06T03:51:23.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.354+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032000cc60 con 0x7f031c0041d0 2023-10-06T03:51:23.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.354+0000 7f0335199700 1 -- 172.21.15.55:0/2181773594 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03200070d0 con 0x7f031c0041d0 2023-10-06T03:51:23.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.354+0000 7f033719d700 1 -- 172.21.15.55:0/2181773594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f031c0041d0 msgr2=0x7f031c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.354+0000 7f033719d700 1 --2- 172.21.15.55:0/2181773594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f031c0041d0 0x7f031c004810 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f032000c210 tx=0x7f0320007810 comp rx=0 tx=0).stop 2023-10-06T03:51:23.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.355+0000 7f033699c700 1 -- 172.21.15.55:0/2181773594 reap_dead start 2023-10-06T03:51:23.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.355+0000 7f033719d700 1 -- 172.21.15.55:0/2181773594 shutdown_connections 2023-10-06T03:51:23.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.355+0000 7f033719d700 1 -- 172.21.15.55:0/2181773594 >> 172.21.15.55:0/2181773594 conn(0x7f03300cddc0 msgr2=0x7f03300c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:23.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.355+0000 7f033719d700 1 -- 172.21.15.55:0/2181773594 shutdown_connections 2023-10-06T03:51:23.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.355+0000 7f033719d700 1 -- 172.21.15.55:0/2181773594 wait complete. 2023-10-06T03:51:23.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.356+0000 7f033719d700 1 Processor -- start 2023-10-06T03:51:23.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.356+0000 7f033719d700 1 -- start start 2023-10-06T03:51:23.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.357+0000 7f033719d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f031c0041d0 0x7f0330149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:23.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.357+0000 7f033719d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f033015d730 0x7f033015b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:23.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.357+0000 7f033719d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033015b620 0x7f03300cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:23.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.357+0000 7f033719d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0330169e50 con 0x7f033015d730 2023-10-06T03:51:23.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.357+0000 7f033719d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033015e530 con 0x7f033015b620 2023-10-06T03:51:23.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.357+0000 7f033719d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0330157eb0 con 0x7f031c0041d0 2023-10-06T03:51:23.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.357+0000 7f033599a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f033015d730 0x7f033015b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:23.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.358+0000 7f033599a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f033015d730 0x7f033015b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:36990/0 (socket says 172.21.15.55:36990) 2023-10-06T03:51:23.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.358+0000 7f033599a700 1 -- 172.21.15.55:0/1970002028 learned_addr learned my addr 172.21.15.55:0/1970002028 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:23.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.358+0000 7f033619b700 1 --2- 172.21.15.55:0/1970002028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f031c0041d0 0x7f0330149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:23.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.358+0000 7f033699c700 1 --2- 172.21.15.55:0/1970002028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033015b620 0x7f03300cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:23.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.358+0000 7f033599a700 1 -- 172.21.15.55:0/1970002028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f031c0041d0 msgr2=0x7f0330149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.358+0000 7f033599a700 1 --2- 172.21.15.55:0/1970002028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f031c0041d0 0x7f0330149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.358+0000 7f033599a700 1 -- 172.21.15.55:0/1970002028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033015b620 msgr2=0x7f03300cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.358+0000 7f033599a700 1 --2- 172.21.15.55:0/1970002028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033015b620 0x7f03300cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.358+0000 7f033599a700 1 -- 172.21.15.55:0/1970002028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033015dc90 con 0x7f033015d730 2023-10-06T03:51:23.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.358+0000 7f033599a700 1 --2- 172.21.15.55:0/1970002028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f033015d730 0x7f033015b0e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f031c0057f0 tx=0x7f0320015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:23.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.358+0000 7f03267fc700 1 -- 172.21.15.55:0/1970002028 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03200017d0 con 0x7f033015d730 2023-10-06T03:51:23.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.358+0000 7f03267fc700 1 -- 172.21.15.55:0/1970002028 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0320001ac0 con 0x7f033015d730 2023-10-06T03:51:23.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.358+0000 7f03267fc700 1 -- 172.21.15.55:0/1970002028 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0320001e80 con 0x7f033015d730 2023-10-06T03:51:23.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.358+0000 7f033719d700 1 -- 172.21.15.55:0/1970002028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033015df70 con 0x7f033015d730 2023-10-06T03:51:23.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.359+0000 7f033719d700 1 -- 172.21.15.55:0/1970002028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033014a4c0 con 0x7f033015d730 2023-10-06T03:51:23.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.360+0000 7f033719d700 1 -- 172.21.15.55:0/1970002028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0330157c80 con 0x7f033015d730 2023-10-06T03:51:23.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.360+0000 7f03267fc700 1 -- 172.21.15.55:0/1970002028 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0320019070 con 0x7f033015d730 2023-10-06T03:51:23.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.361+0000 7f03267fc700 1 --2- 172.21.15.55:0/1970002028 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0314066f30 0x7f03140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:23.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.361+0000 7f03267fc700 1 -- 172.21.15.55:0/1970002028 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f0320085550 con 0x7f033015d730 2023-10-06T03:51:23.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.362+0000 7f033619b700 1 --2- 172.21.15.55:0/1970002028 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0314066f30 0x7f03140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:23.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.363+0000 7f033619b700 1 --2- 172.21.15.55:0/1970002028 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0314066f30 0x7f03140693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f032c009d40 tx=0x7f032c008040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:23.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.366+0000 7f03267fc700 1 -- 172.21.15.55:0/1970002028 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0320052830 con 0x7f033015d730 2023-10-06T03:51:23.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.566+0000 7f033719d700 1 -- 172.21.15.55:0/1970002028 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0330063d10 con 0x7f033015d730 2023-10-06T03:51:23.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.567+0000 7f03267fc700 1 -- 172.21.15.55:0/1970002028 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0320007a40 con 0x7f033015d730 2023-10-06T03:51:23.568 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:51:23.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.569+0000 7f033719d700 1 -- 172.21.15.55:0/1970002028 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0314066f30 msgr2=0x7f03140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.569+0000 7f033719d700 1 --2- 172.21.15.55:0/1970002028 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0314066f30 0x7f03140693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f032c009d40 tx=0x7f032c008040 comp rx=0 tx=0).stop 2023-10-06T03:51:23.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.569+0000 7f033719d700 1 -- 172.21.15.55:0/1970002028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f033015d730 msgr2=0x7f033015b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.569+0000 7f033719d700 1 --2- 172.21.15.55:0/1970002028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f033015d730 0x7f033015b0e0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f031c0057f0 tx=0x7f0320015000 comp rx=0 tx=0).stop 2023-10-06T03:51:23.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.569+0000 7f033719d700 1 -- 172.21.15.55:0/1970002028 shutdown_connections 2023-10-06T03:51:23.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.569+0000 7f033719d700 1 --2- 172.21.15.55:0/1970002028 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0314066f30 0x7f03140693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.569+0000 7f033719d700 1 --2- 172.21.15.55:0/1970002028 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f033015b620 0x7f03300cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.569+0000 7f033719d700 1 --2- 172.21.15.55:0/1970002028 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f031c0041d0 0x7f0330149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.569+0000 7f033719d700 1 --2- 172.21.15.55:0/1970002028 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f033015d730 0x7f033015b0e0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.569+0000 7f033719d700 1 -- 172.21.15.55:0/1970002028 >> 172.21.15.55:0/1970002028 conn(0x7f03300cddc0 msgr2=0x7f0330068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:23.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.569+0000 7f033719d700 1 -- 172.21.15.55:0/1970002028 shutdown_connections 2023-10-06T03:51:23.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.569+0000 7f033719d700 1 -- 172.21.15.55:0/1970002028 wait complete. 2023-10-06T03:51:23.588 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:51:23.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.774+0000 7f8a9bfff700 1 Processor -- start 2023-10-06T03:51:23.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.775+0000 7f8a9bfff700 1 -- start start 2023-10-06T03:51:23.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.776+0000 7f8a9bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8a9c150550 0x7f8a9c157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:23.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.776+0000 7f8a9bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a9c0c7410 0x7f8a9c069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:23.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.776+0000 7f8a9bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a9c06e2a0 con 0x7f8a9c150180 2023-10-06T03:51:23.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.776+0000 7f8a9bfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9c06b2e0 con 0x7f8a9c0c7410 2023-10-06T03:51:23.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.776+0000 7f8a9bfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9c0c78e0 con 0x7f8a9c150550 2023-10-06T03:51:23.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.776+0000 7f8a9b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8a9c0c7410 0x7f8a9c069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:23.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.776+0000 7f8a9affd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8a9c150180 0x7f8a9c0c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38122/0 (socket says 172.21.15.55:38122) 2023-10-06T03:51:23.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.776+0000 7f8a9affd700 1 -- 172.21.15.55:0/1610197431 learned_addr learned my addr 172.21.15.55:0/1610197431 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:23.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.777+0000 7f8a9a7fc700 1 --2- 172.21.15.55:0/1610197431 >> v2:172.21.15.163:3300/0 conn(0x7f8a9c150550 0x7f8a9c157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:23.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.777+0000 7f8a9b7fe700 1 -- 172.21.15.55:0/1610197431 >> v2:172.21.15.163:3300/0 conn(0x7f8a9c150550 msgr2=0x7f8a9c157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:23.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.777+0000 7f8a9b7fe700 1 --2- 172.21.15.55:0/1610197431 >> v2:172.21.15.163:3300/0 conn(0x7f8a9c150550 0x7f8a9c157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.777+0000 7f8a9b7fe700 1 -- 172.21.15.55:0/1610197431 >> v1:172.21.15.55:6789/0 conn(0x7f8a9c150180 legacy=0x7f8a9c0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.777+0000 7f8a9b7fe700 1 -- 172.21.15.55:0/1610197431 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c0c7bc0 con 0x7f8a9c0c7410 2023-10-06T03:51:23.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.777+0000 7f8a99ffb700 1 -- 172.21.15.55:0/1610197431 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034130795 0 0) 0x7f8a9c06e2a0 con 0x7f8a9c150180 2023-10-06T03:51:23.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.777+0000 7f8a9b7fe700 1 --2- 172.21.15.55:0/1610197431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a9c0c7410 0x7f8a9c069f50 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c06d1e0 tx=0x7f8a94009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69ab685f425311 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:23.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.777+0000 7f8a99ffb700 1 -- 172.21.15.55:0/1610197431 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9400e070 con 0x7f8a9c0c7410 2023-10-06T03:51:23.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.777+0000 7f8a99ffb700 1 -- 172.21.15.55:0/1610197431 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9400bc30 con 0x7f8a9c0c7410 2023-10-06T03:51:23.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.777+0000 7f8a99ffb700 1 -- 172.21.15.55:0/1610197431 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a94012430 con 0x7f8a9c0c7410 2023-10-06T03:51:23.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.778+0000 7f8a9bfff700 1 -- 172.21.15.55:0/1610197431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a9c0c7410 msgr2=0x7f8a9c069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.778+0000 7f8a9bfff700 1 --2- 172.21.15.55:0/1610197431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a9c0c7410 0x7f8a9c069f50 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c06d1e0 tx=0x7f8a94009780 comp rx=0 tx=0).stop 2023-10-06T03:51:23.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.778+0000 7f8a9bfff700 1 -- 172.21.15.55:0/1610197431 shutdown_connections 2023-10-06T03:51:23.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.778+0000 7f8a9bfff700 1 --2- 172.21.15.55:0/1610197431 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a9c0c7410 0x7f8a9c069f50 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.778+0000 7f8a9bfff700 1 --2- 172.21.15.55:0/1610197431 >> v2:172.21.15.163:3300/0 conn(0x7f8a9c150550 0x7f8a9c157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.778+0000 7f8a9bfff700 1 -- 172.21.15.55:0/1610197431 >> 172.21.15.55:0/1610197431 conn(0x7f8a9c0cc1f0 msgr2=0x7f8a9c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:23.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.778+0000 7f8a9bfff700 1 -- 172.21.15.55:0/1610197431 shutdown_connections 2023-10-06T03:51:23.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.778+0000 7f8a9bfff700 1 -- 172.21.15.55:0/1610197431 wait complete. 2023-10-06T03:51:23.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.780+0000 7f8a9bfff700 1 Processor -- start 2023-10-06T03:51:23.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.780+0000 7f8a9bfff700 1 -- start start 2023-10-06T03:51:23.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a9c0c7410 0x7f8a9c0c2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:23.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a9c150180 0x7f8a9c069c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:23.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a9c150550 0x7f8a9c06a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:23.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c14c9e0 con 0x7f8a9c0c7410 2023-10-06T03:51:23.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c14f0a0 con 0x7f8a9c150180 2023-10-06T03:51:23.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c0b1e10 con 0x7f8a9c150550 2023-10-06T03:51:23.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a9c0c7410 0x7f8a9c0c2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:23.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a9c0c7410 0x7f8a9c0c2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37020/0 (socket says 172.21.15.55:37020) 2023-10-06T03:51:23.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9affd700 1 -- 172.21.15.55:0/585557170 learned_addr learned my addr 172.21.15.55:0/585557170 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:23.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9b7fe700 1 --2- 172.21.15.55:0/585557170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a9c150550 0x7f8a9c06a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:23.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9a7fc700 1 --2- 172.21.15.55:0/585557170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a9c150180 0x7f8a9c069c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:23.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9affd700 1 -- 172.21.15.55:0/585557170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a9c150550 msgr2=0x7f8a9c06a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9affd700 1 --2- 172.21.15.55:0/585557170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a9c150550 0x7f8a9c06a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9affd700 1 -- 172.21.15.55:0/585557170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a9c150180 msgr2=0x7f8a9c069c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9affd700 1 --2- 172.21.15.55:0/585557170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a9c150180 0x7f8a9c069c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.781+0000 7f8a9affd700 1 -- 172.21.15.55:0/585557170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a94009a20 con 0x7f8a9c0c7410 2023-10-06T03:51:23.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.782+0000 7f8a9affd700 1 --2- 172.21.15.55:0/585557170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a9c0c7410 0x7f8a9c0c2150 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00c810 tx=0x7f8a8c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:23.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.782+0000 7f8a82ffd700 1 -- 172.21.15.55:0/585557170 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c005b30 con 0x7f8a9c0c7410 2023-10-06T03:51:23.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.782+0000 7f8a82ffd700 1 -- 172.21.15.55:0/585557170 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8c005c90 con 0x7f8a9c0c7410 2023-10-06T03:51:23.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.782+0000 7f8a9bfff700 1 -- 172.21.15.55:0/585557170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9c0b1290 con 0x7f8a9c0c7410 2023-10-06T03:51:23.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.782+0000 7f8a82ffd700 1 -- 172.21.15.55:0/585557170 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c007300 con 0x7f8a9c0c7410 2023-10-06T03:51:23.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.782+0000 7f8a9bfff700 1 -- 172.21.15.55:0/585557170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9c0b1b30 con 0x7f8a9c0c7410 2023-10-06T03:51:23.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.783+0000 7f8a9bfff700 1 -- 172.21.15.55:0/585557170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9c14f7e0 con 0x7f8a9c0c7410 2023-10-06T03:51:23.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.784+0000 7f8a82ffd700 1 -- 172.21.15.55:0/585557170 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a8c01b400 con 0x7f8a9c0c7410 2023-10-06T03:51:23.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.785+0000 7f8a82ffd700 1 --2- 172.21.15.55:0/585557170 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a88066f30 0x7f8a880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:23.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.785+0000 7f8a82ffd700 1 -- 172.21.15.55:0/585557170 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f8a8c084080 con 0x7f8a9c0c7410 2023-10-06T03:51:23.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.785+0000 7f8a9a7fc700 1 --2- 172.21.15.55:0/585557170 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a88066f30 0x7f8a880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:23.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.786+0000 7f8a9a7fc700 1 --2- 172.21.15.55:0/585557170 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a88066f30 0x7f8a880693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a90006fd0 tx=0x7f8a90008040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:23.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.789+0000 7f8a82ffd700 1 -- 172.21.15.55:0/585557170 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a8c051340 con 0x7f8a9c0c7410 2023-10-06T03:51:23.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.994+0000 7f8a9bfff700 1 -- 172.21.15.55:0/585557170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8a9c04c3c0 con 0x7f8a9c0c7410 2023-10-06T03:51:23.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.995+0000 7f8a82ffd700 1 -- 172.21.15.55:0/585557170 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v119) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8a8c054960 con 0x7f8a9c0c7410 2023-10-06T03:51:23.997 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:23.997 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:51:23.997 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:23.997 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2023-10-06T03:51:23.997 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": "rbd" 2023-10-06T03:51:23.997 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:51:23.997 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:23.997 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 2, 2023-10-06T03:51:23.997 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2023-10-06T03:51:23.997 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:23.998 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:51:23.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.997+0000 7f8a9bfff700 1 -- 172.21.15.55:0/585557170 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a88066f30 msgr2=0x7f8a880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.997+0000 7f8a9bfff700 1 --2- 172.21.15.55:0/585557170 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a88066f30 0x7f8a880693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8a90006fd0 tx=0x7f8a90008040 comp rx=0 tx=0).stop 2023-10-06T03:51:23.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.997+0000 7f8a9bfff700 1 -- 172.21.15.55:0/585557170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a9c0c7410 msgr2=0x7f8a9c0c2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:23.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.997+0000 7f8a9bfff700 1 --2- 172.21.15.55:0/585557170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a9c0c7410 0x7f8a9c0c2150 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00c810 tx=0x7f8a8c007df0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.997+0000 7f8a9bfff700 1 -- 172.21.15.55:0/585557170 shutdown_connections 2023-10-06T03:51:23.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.997+0000 7f8a9bfff700 1 --2- 172.21.15.55:0/585557170 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8a88066f30 0x7f8a880693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.997+0000 7f8a9bfff700 1 --2- 172.21.15.55:0/585557170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8a9c150550 0x7f8a9c06a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:23.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.997+0000 7f8a9bfff700 1 --2- 172.21.15.55:0/585557170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8a9c0c7410 0x7f8a9c0c2150 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:24.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.997+0000 7f8a9bfff700 1 --2- 172.21.15.55:0/585557170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8a9c150180 0x7f8a9c069c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:24.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.997+0000 7f8a9bfff700 1 -- 172.21.15.55:0/585557170 >> 172.21.15.55:0/585557170 conn(0x7f8a9c0cc1f0 msgr2=0x7f8a9c1562e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:24.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.997+0000 7f8a9bfff700 1 -- 172.21.15.55:0/585557170 shutdown_connections 2023-10-06T03:51:24.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:23.997+0000 7f8a9bfff700 1 -- 172.21.15.55:0/585557170 wait complete. 2023-10-06T03:51:24.016 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:51:24.017 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:51:24.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.206+0000 7fd93359e700 1 Processor -- start 2023-10-06T03:51:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.206+0000 7fd93359e700 1 -- start start 2023-10-06T03:51:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.207+0000 7fd93359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd9340685e0 0x7fd9340ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.207+0000 7fd93359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9340cb360 0x7fd9340ca680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.207+0000 7fd93359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd934068fa0 0x7fd9340cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.207+0000 7fd93359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd93406d890 con 0x7fd934068fa0 2023-10-06T03:51:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.207+0000 7fd93359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9340cb860 con 0x7fd9340685e0 2023-10-06T03:51:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.207+0000 7fd93359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340cbb40 con 0x7fd9340cb360 2023-10-06T03:51:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.208+0000 7fd932d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd934068fa0 0x7fd9340cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.208+0000 7fd932d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd934068fa0 0x7fd9340cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37024/0 (socket says 172.21.15.55:37024) 2023-10-06T03:51:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.208+0000 7fd932d9d700 1 -- 172.21.15.55:0/2388944914 learned_addr learned my addr 172.21.15.55:0/2388944914 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.208+0000 7fd931d9b700 1 --2- 172.21.15.55:0/2388944914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9340cb360 0x7fd9340ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.208+0000 7fd93259c700 1 --2- 172.21.15.55:0/2388944914 >> v2:172.21.15.163:3300/0 conn(0x7fd9340685e0 0x7fd9340ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.208+0000 7fd932d9d700 1 -- 172.21.15.55:0/2388944914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9340cb360 msgr2=0x7fd9340ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.208+0000 7fd932d9d700 1 --2- 172.21.15.55:0/2388944914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9340cb360 0x7fd9340ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.208+0000 7fd932d9d700 1 -- 172.21.15.55:0/2388944914 >> v2:172.21.15.163:3300/0 conn(0x7fd9340685e0 msgr2=0x7fd9340ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.208+0000 7fd932d9d700 1 --2- 172.21.15.55:0/2388944914 >> v2:172.21.15.163:3300/0 conn(0x7fd9340685e0 0x7fd9340ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.208+0000 7fd932d9d700 1 -- 172.21.15.55:0/2388944914 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9340cbe20 con 0x7fd934068fa0 2023-10-06T03:51:24.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.208+0000 7fd931d9b700 1 --2- 172.21.15.55:0/2388944914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9340cb360 0x7fd9340ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.209+0000 7fd932d9d700 1 --2- 172.21.15.55:0/2388944914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd934068fa0 0x7fd9340cabc0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd934065be0 tx=0x7fd92c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5689af997b5932 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.209+0000 7fd93159a700 1 -- 172.21.15.55:0/2388944914 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c00e070 con 0x7fd934068fa0 2023-10-06T03:51:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.209+0000 7fd93159a700 1 -- 172.21.15.55:0/2388944914 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92c00ba50 con 0x7fd934068fa0 2023-10-06T03:51:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.209+0000 7fd93159a700 1 -- 172.21.15.55:0/2388944914 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c0092e0 con 0x7fd934068fa0 2023-10-06T03:51:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.209+0000 7fd93359e700 1 -- 172.21.15.55:0/2388944914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd934068fa0 msgr2=0x7fd9340cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.209+0000 7fd93359e700 1 --2- 172.21.15.55:0/2388944914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd934068fa0 0x7fd9340cabc0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd934065be0 tx=0x7fd92c009840 comp rx=0 tx=0).stop 2023-10-06T03:51:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.209+0000 7fd93359e700 1 -- 172.21.15.55:0/2388944914 shutdown_connections 2023-10-06T03:51:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.209+0000 7fd93359e700 1 --2- 172.21.15.55:0/2388944914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd934068fa0 0x7fd9340cabc0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.209+0000 7fd93359e700 1 --2- 172.21.15.55:0/2388944914 >> v2:172.21.15.163:3300/0 conn(0x7fd9340685e0 0x7fd9340ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.209+0000 7fd93359e700 1 --2- 172.21.15.55:0/2388944914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9340cb360 0x7fd9340ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.209+0000 7fd93359e700 1 -- 172.21.15.55:0/2388944914 >> 172.21.15.55:0/2388944914 conn(0x7fd93406a380 msgr2=0x7fd9340cc570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.209+0000 7fd93359e700 1 -- 172.21.15.55:0/2388944914 shutdown_connections 2023-10-06T03:51:24.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.210+0000 7fd93359e700 1 -- 172.21.15.55:0/2388944914 wait complete. 2023-10-06T03:51:24.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.211+0000 7fd93359e700 1 Processor -- start 2023-10-06T03:51:24.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.211+0000 7fd93359e700 1 -- start start 2023-10-06T03:51:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.212+0000 7fd93359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9340685e0 0x7fd9340b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.212+0000 7fd93359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd934068fa0 0x7fd934157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.212+0000 7fd93359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9340cb360 0x7fd934154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.212+0000 7fd93359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340b0840 con 0x7fd934068fa0 2023-10-06T03:51:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.212+0000 7fd93359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9341552a0 con 0x7fd9340cb360 2023-10-06T03:51:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.212+0000 7fd93359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9340b0a50 con 0x7fd9340685e0 2023-10-06T03:51:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.212+0000 7fd931d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd934068fa0 0x7fd934157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.212+0000 7fd931d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd934068fa0 0x7fd934157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37034/0 (socket says 172.21.15.55:37034) 2023-10-06T03:51:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.212+0000 7fd931d9b700 1 -- 172.21.15.55:0/1869774142 learned_addr learned my addr 172.21.15.55:0/1869774142 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.213+0000 7fd932d9d700 1 --2- 172.21.15.55:0/1869774142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9340cb360 0x7fd934154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.213+0000 7fd931d9b700 1 -- 172.21.15.55:0/1869774142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9340685e0 msgr2=0x7fd9340b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.213+0000 7fd931d9b700 1 --2- 172.21.15.55:0/1869774142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9340685e0 0x7fd9340b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.213+0000 7fd931d9b700 1 -- 172.21.15.55:0/1869774142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9340cb360 msgr2=0x7fd934154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.213+0000 7fd931d9b700 1 --2- 172.21.15.55:0/1869774142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9340cb360 0x7fd934154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.213+0000 7fd931d9b700 1 -- 172.21.15.55:0/1869774142 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c009ae0 con 0x7fd934068fa0 2023-10-06T03:51:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.213+0000 7fd931d9b700 1 --2- 172.21.15.55:0/1869774142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd934068fa0 0x7fd934157f80 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd92800d890 tx=0x7fd928008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.213+0000 7fd9227fc700 1 -- 172.21.15.55:0/1869774142 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9280125f0 con 0x7fd934068fa0 2023-10-06T03:51:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.213+0000 7fd9227fc700 1 -- 172.21.15.55:0/1869774142 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd928012750 con 0x7fd934068fa0 2023-10-06T03:51:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.213+0000 7fd9227fc700 1 -- 172.21.15.55:0/1869774142 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd928012af0 con 0x7fd934068fa0 2023-10-06T03:51:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.213+0000 7fd93359e700 1 -- 172.21.15.55:0/1869774142 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9340b0d30 con 0x7fd934068fa0 2023-10-06T03:51:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.213+0000 7fd93359e700 1 -- 172.21.15.55:0/1869774142 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9340b12f0 con 0x7fd934068fa0 2023-10-06T03:51:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.214+0000 7fd93359e700 1 -- 172.21.15.55:0/1869774142 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93406a780 con 0x7fd934068fa0 2023-10-06T03:51:24.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.215+0000 7fd9227fc700 1 -- 172.21.15.55:0/1869774142 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd928014430 con 0x7fd934068fa0 2023-10-06T03:51:24.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.216+0000 7fd9227fc700 1 --2- 172.21.15.55:0/1869774142 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd914066f30 0x7fd9140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:24.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.216+0000 7fd9227fc700 1 -- 172.21.15.55:0/1869774142 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fd928084a70 con 0x7fd934068fa0 2023-10-06T03:51:24.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.216+0000 7fd93259c700 1 --2- 172.21.15.55:0/1869774142 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd914066f30 0x7fd9140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:24.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.217+0000 7fd93259c700 1 --2- 172.21.15.55:0/1869774142 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd914066f30 0x7fd9140693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd924009a60 tx=0x7fd924008040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:24.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.220+0000 7fd9227fc700 1 -- 172.21.15.55:0/1869774142 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd928051cb0 con 0x7fd934068fa0 2023-10-06T03:51:24.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.425+0000 7fd93359e700 1 -- 172.21.15.55:0/1869774142 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd9340cc100 con 0x7fd934068fa0 2023-10-06T03:51:24.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.977+0000 7fd9227fc700 1 -- 172.21.15.55:0/1869774142 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v120) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd9280552d0 con 0x7fd934068fa0 2023-10-06T03:51:24.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.981+0000 7fd93359e700 1 -- 172.21.15.55:0/1869774142 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd914066f30 msgr2=0x7fd9140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:24.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.981+0000 7fd93359e700 1 --2- 172.21.15.55:0/1869774142 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd914066f30 0x7fd9140693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd924009a60 tx=0x7fd924008040 comp rx=0 tx=0).stop 2023-10-06T03:51:24.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.981+0000 7fd93359e700 1 -- 172.21.15.55:0/1869774142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd934068fa0 msgr2=0x7fd934157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:24.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.981+0000 7fd93359e700 1 --2- 172.21.15.55:0/1869774142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd934068fa0 0x7fd934157f80 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd92800d890 tx=0x7fd928008040 comp rx=0 tx=0).stop 2023-10-06T03:51:24.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.981+0000 7fd93359e700 1 -- 172.21.15.55:0/1869774142 shutdown_connections 2023-10-06T03:51:24.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.981+0000 7fd93359e700 1 --2- 172.21.15.55:0/1869774142 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd914066f30 0x7fd9140693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:24.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.981+0000 7fd93359e700 1 --2- 172.21.15.55:0/1869774142 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9340cb360 0x7fd934154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:24.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.981+0000 7fd93359e700 1 --2- 172.21.15.55:0/1869774142 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9340685e0 0x7fd9340b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:24.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.981+0000 7fd93359e700 1 --2- 172.21.15.55:0/1869774142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd934068fa0 0x7fd934157f80 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:24.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.981+0000 7fd93359e700 1 -- 172.21.15.55:0/1869774142 >> 172.21.15.55:0/1869774142 conn(0x7fd93406a380 msgr2=0x7fd93406ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:24.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.982+0000 7fd93359e700 1 -- 172.21.15.55:0/1869774142 shutdown_connections 2023-10-06T03:51:24.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:24.982+0000 7fd93359e700 1 -- 172.21.15.55:0/1869774142 wait complete. 2023-10-06T03:51:24.987 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2023-10-06T03:51:25.005 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:51:25.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.199+0000 7ff07979d700 1 Processor -- start 2023-10-06T03:51:25.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.199+0000 7ff07979d700 1 -- start start 2023-10-06T03:51:25.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.200+0000 7ff07979d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff074068aa0 0x7ff07406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:25.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.200+0000 7ff07979d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07415ad70 con 0x7ff0741577c0 2023-10-06T03:51:25.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.200+0000 7ff07979d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07415a160 con 0x7ff0741573f0 2023-10-06T03:51:25.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.200+0000 7ff07979d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0740c8610 con 0x7ff074068aa0 2023-10-06T03:51:25.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.201+0000 7ff0737fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff0741577c0 0x7ff074068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38132/0 (socket says 172.21.15.55:38132) 2023-10-06T03:51:25.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.201+0000 7ff0737fe700 1 -- 172.21.15.55:0/3454692817 learned_addr learned my addr 172.21.15.55:0/3454692817 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:25.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.201+0000 7ff078f9c700 1 --2- 172.21.15.55:0/3454692817 >> v2:172.21.15.163:3300/0 conn(0x7ff074068aa0 0x7ff07406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:25.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.201+0000 7ff072ffd700 1 -- 172.21.15.55:0/3454692817 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1829737961 0 0) 0x7ff07415ad70 con 0x7ff0741577c0 2023-10-06T03:51:25.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.201+0000 7ff072ffd700 1 -- 172.21.15.55:0/3454692817 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff05c003570 con 0x7ff0741577c0 2023-10-06T03:51:25.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.201+0000 7ff072ffd700 1 -- 172.21.15.55:0/3454692817 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4287703684 0 0) 0x7ff07415a160 con 0x7ff0741573f0 2023-10-06T03:51:25.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.201+0000 7ff072ffd700 1 -- 172.21.15.55:0/3454692817 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff07415ad70 con 0x7ff0741573f0 2023-10-06T03:51:25.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.201+0000 7ff078f9c700 1 -- 172.21.15.55:0/3454692817 >> v1:172.21.15.163:6789/0 conn(0x7ff0741573f0 legacy=0x7ff07406c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:25.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.201+0000 7ff078f9c700 1 -- 172.21.15.55:0/3454692817 >> v1:172.21.15.55:6789/0 conn(0x7ff0741577c0 legacy=0x7ff074068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.201+0000 7ff078f9c700 1 -- 172.21.15.55:0/3454692817 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07415a440 con 0x7ff074068aa0 2023-10-06T03:51:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.202+0000 7ff078f9c700 1 --2- 172.21.15.55:0/3454692817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff074068aa0 0x7ff07406bcb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009e60 tx=0x7ff06c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cabb6dcd036104b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.202+0000 7ff072ffd700 1 -- 172.21.15.55:0/3454692817 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c00e070 con 0x7ff074068aa0 2023-10-06T03:51:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.202+0000 7ff072ffd700 1 -- 172.21.15.55:0/3454692817 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c00bab0 con 0x7ff074068aa0 2023-10-06T03:51:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.202+0000 7ff072ffd700 1 -- 172.21.15.55:0/3454692817 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c00be70 con 0x7ff074068aa0 2023-10-06T03:51:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.202+0000 7ff07979d700 1 -- 172.21.15.55:0/3454692817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff074068aa0 msgr2=0x7ff07406bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.202+0000 7ff07979d700 1 --2- 172.21.15.55:0/3454692817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff074068aa0 0x7ff07406bcb0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009e60 tx=0x7ff06c009780 comp rx=0 tx=0).stop 2023-10-06T03:51:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.203+0000 7ff07979d700 1 -- 172.21.15.55:0/3454692817 shutdown_connections 2023-10-06T03:51:25.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.203+0000 7ff07979d700 1 --2- 172.21.15.55:0/3454692817 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff074068aa0 0x7ff07406bcb0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:25.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.203+0000 7ff07979d700 1 -- 172.21.15.55:0/3454692817 >> 172.21.15.55:0/3454692817 conn(0x7ff0740cddc0 msgr2=0x7ff0740c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:25.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.203+0000 7ff07979d700 1 -- 172.21.15.55:0/3454692817 shutdown_connections 2023-10-06T03:51:25.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.203+0000 7ff07979d700 1 -- 172.21.15.55:0/3454692817 wait complete. 2023-10-06T03:51:25.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.204+0000 7ff07979d700 1 Processor -- start 2023-10-06T03:51:25.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.204+0000 7ff07979d700 1 -- start start 2023-10-06T03:51:25.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.205+0000 7ff07979d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff074068aa0 0x7ff074154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:25.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.205+0000 7ff07979d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0741573f0 0x7ff0740ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:25.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.205+0000 7ff07979d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0741577c0 0x7ff0741590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:25.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.205+0000 7ff07979d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740b2240 con 0x7ff0741573f0 2023-10-06T03:51:25.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.205+0000 7ff07979d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff074154990 con 0x7ff074068aa0 2023-10-06T03:51:25.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.205+0000 7ff07979d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740b1630 con 0x7ff0741577c0 2023-10-06T03:51:25.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.205+0000 7ff0737fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0741573f0 0x7ff0740ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:25.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.206+0000 7ff078f9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0741577c0 0x7ff0741590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:25.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.206+0000 7ff073fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff074068aa0 0x7ff074154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:25.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.206+0000 7ff0737fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0741573f0 0x7ff0740ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37036/0 (socket says 172.21.15.55:37036) 2023-10-06T03:51:25.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.206+0000 7ff0737fe700 1 -- 172.21.15.55:0/2210425075 learned_addr learned my addr 172.21.15.55:0/2210425075 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:25.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.206+0000 7ff0737fe700 1 -- 172.21.15.55:0/2210425075 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0741577c0 msgr2=0x7ff0741590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:25.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.206+0000 7ff0737fe700 1 --2- 172.21.15.55:0/2210425075 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0741577c0 0x7ff0741590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:25.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.206+0000 7ff0737fe700 1 -- 172.21.15.55:0/2210425075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff074068aa0 msgr2=0x7ff074154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:25.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.206+0000 7ff0737fe700 1 --2- 172.21.15.55:0/2210425075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff074068aa0 0x7ff074154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:25.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.206+0000 7ff0737fe700 1 -- 172.21.15.55:0/2210425075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c009a20 con 0x7ff0741573f0 2023-10-06T03:51:25.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.206+0000 7ff073fff700 1 --2- 172.21.15.55:0/2210425075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff074068aa0 0x7ff074154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:25.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.206+0000 7ff0737fe700 1 --2- 172.21.15.55:0/2210425075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0741573f0 0x7ff0740ad6e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff06800c790 tx=0x7ff068007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:25.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.207+0000 7ff05bfff700 1 -- 172.21.15.55:0/2210425075 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068005b30 con 0x7ff0741573f0 2023-10-06T03:51:25.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.207+0000 7ff05bfff700 1 -- 172.21.15.55:0/2210425075 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff068005c90 con 0x7ff0741573f0 2023-10-06T03:51:25.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.207+0000 7ff07979d700 1 -- 172.21.15.55:0/2210425075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0740b1910 con 0x7ff0741573f0 2023-10-06T03:51:25.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.207+0000 7ff05bfff700 1 -- 172.21.15.55:0/2210425075 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068007300 con 0x7ff0741573f0 2023-10-06T03:51:25.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.207+0000 7ff07979d700 1 -- 172.21.15.55:0/2210425075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0740b1ed0 con 0x7ff0741573f0 2023-10-06T03:51:25.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.208+0000 7ff07979d700 1 -- 172.21.15.55:0/2210425075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff07404c3c0 con 0x7ff0741573f0 2023-10-06T03:51:25.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.209+0000 7ff05bfff700 1 -- 172.21.15.55:0/2210425075 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff06801b3b0 con 0x7ff0741573f0 2023-10-06T03:51:25.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.210+0000 7ff05bfff700 1 --2- 172.21.15.55:0/2210425075 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff060066f30 0x7ff0600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:25.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.210+0000 7ff05bfff700 1 -- 172.21.15.55:0/2210425075 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5968+0+0 (secure 0 0 0) 0x7ff068083a40 con 0x7ff0741573f0 2023-10-06T03:51:25.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.210+0000 7ff073fff700 1 --2- 172.21.15.55:0/2210425075 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff060066f30 0x7ff0600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:25.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.211+0000 7ff073fff700 1 --2- 172.21.15.55:0/2210425075 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff060066f30 0x7ff0600693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff064007fd0 tx=0x7ff064007d50 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:25.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.214+0000 7ff05bfff700 1 -- 172.21.15.55:0/2210425075 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff068050b30 con 0x7ff0741573f0 2023-10-06T03:51:25.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.420+0000 7ff07979d700 1 -- 172.21.15.55:0/2210425075 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff07415a720 con 0x7ff0741573f0 2023-10-06T03:51:25.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.989+0000 7ff05bfff700 1 -- 172.21.15.55:0/2210425075 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v121) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff068017020 con 0x7ff0741573f0 2023-10-06T03:51:25.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.995+0000 7ff07979d700 1 -- 172.21.15.55:0/2210425075 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff060066f30 msgr2=0x7ff0600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:25.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.995+0000 7ff07979d700 1 --2- 172.21.15.55:0/2210425075 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff060066f30 0x7ff0600693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff064007fd0 tx=0x7ff064007d50 comp rx=0 tx=0).stop 2023-10-06T03:51:25.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.995+0000 7ff07979d700 1 -- 172.21.15.55:0/2210425075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0741573f0 msgr2=0x7ff0740ad6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:25.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.995+0000 7ff07979d700 1 --2- 172.21.15.55:0/2210425075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0741573f0 0x7ff0740ad6e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7ff06800c790 tx=0x7ff068007e80 comp rx=0 tx=0).stop 2023-10-06T03:51:25.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.996+0000 7ff07979d700 1 -- 172.21.15.55:0/2210425075 shutdown_connections 2023-10-06T03:51:25.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.996+0000 7ff07979d700 1 --2- 172.21.15.55:0/2210425075 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff060066f30 0x7ff0600693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:25.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.996+0000 7ff07979d700 1 --2- 172.21.15.55:0/2210425075 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff0741577c0 0x7ff0741590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:25.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.996+0000 7ff07979d700 1 --2- 172.21.15.55:0/2210425075 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff074068aa0 0x7ff074154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:25.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.996+0000 7ff07979d700 1 --2- 172.21.15.55:0/2210425075 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff0741573f0 0x7ff0740ad6e0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:25.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.996+0000 7ff07979d700 1 -- 172.21.15.55:0/2210425075 >> 172.21.15.55:0/2210425075 conn(0x7ff0740cddc0 msgr2=0x7ff074156200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:26.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:25.999+0000 7ff07979d700 1 -- 172.21.15.55:0/2210425075 shutdown_connections 2023-10-06T03:51:26.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.000+0000 7ff07979d700 1 -- 172.21.15.55:0/2210425075 wait complete. 2023-10-06T03:51:26.004 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2023-10-06T03:51:26.027 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:51:26.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.216+0000 7fe66759e700 1 Processor -- start 2023-10-06T03:51:26.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.216+0000 7fe66759e700 1 -- start start 2023-10-06T03:51:26.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.217+0000 7fe66759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe6680b4e70 0x7fe6680ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:26.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.217+0000 7fe66759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6680b1b40 0x7fe66815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:26.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.217+0000 7fe66759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6680cfb00 con 0x7fe6680b16d0 2023-10-06T03:51:26.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.217+0000 7fe66759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6680b5c40 con 0x7fe6680b4e70 2023-10-06T03:51:26.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.217+0000 7fe66759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6680b5f20 con 0x7fe6680b1b40 2023-10-06T03:51:26.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.218+0000 7fe666d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6680b1b40 0x7fe66815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:26.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.218+0000 7fe66659c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe6680b4e70 0x7fe6680ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:26.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.218+0000 7fe666d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6680b1b40 0x7fe66815d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:35860/0 (socket says 172.21.15.55:35860) 2023-10-06T03:51:26.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.218+0000 7fe666d9d700 1 -- 172.21.15.55:0/302584313 learned_addr learned my addr 172.21.15.55:0/302584313 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:26.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.218+0000 7fe666d9d700 1 -- 172.21.15.55:0/302584313 >> v2:172.21.15.163:3300/0 conn(0x7fe6680b4e70 msgr2=0x7fe6680ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:26.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.218+0000 7fe666d9d700 1 --2- 172.21.15.55:0/302584313 >> v2:172.21.15.163:3300/0 conn(0x7fe6680b4e70 0x7fe6680ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:26.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.219+0000 7fe666d9d700 1 -- 172.21.15.55:0/302584313 >> v1:172.21.15.163:6789/0 conn(0x7fe6680b16d0 legacy=0x7fe66815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:26.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.219+0000 7fe666d9d700 1 -- 172.21.15.55:0/302584313 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6680b6200 con 0x7fe6680b1b40 2023-10-06T03:51:26.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.219+0000 7fe66659c700 1 --2- 172.21.15.55:0/302584313 >> v2:172.21.15.163:3300/0 conn(0x7fe6680b4e70 0x7fe6680ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:26.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.219+0000 7fe666d9d700 1 --2- 172.21.15.55:0/302584313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6680b1b40 0x7fe66815d950 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe660009f20 tx=0x7fe660009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32da1a18ce670db0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:26.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.219+0000 7fe66559a700 1 -- 172.21.15.55:0/302584313 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66000e070 con 0x7fe6680b1b40 2023-10-06T03:51:26.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.219+0000 7fe66559a700 1 -- 172.21.15.55:0/302584313 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66000b9d0 con 0x7fe6680b1b40 2023-10-06T03:51:26.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.219+0000 7fe66559a700 1 -- 172.21.15.55:0/302584313 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6600092e0 con 0x7fe6680b1b40 2023-10-06T03:51:26.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.220+0000 7fe66759e700 1 -- 172.21.15.55:0/302584313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6680b1b40 msgr2=0x7fe66815d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:26.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.220+0000 7fe66759e700 1 --2- 172.21.15.55:0/302584313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6680b1b40 0x7fe66815d950 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe660009f20 tx=0x7fe660009840 comp rx=0 tx=0).stop 2023-10-06T03:51:26.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.220+0000 7fe66759e700 1 -- 172.21.15.55:0/302584313 shutdown_connections 2023-10-06T03:51:26.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.220+0000 7fe66759e700 1 --2- 172.21.15.55:0/302584313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6680b1b40 0x7fe66815d950 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:26.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.220+0000 7fe66759e700 1 --2- 172.21.15.55:0/302584313 >> v2:172.21.15.163:3300/0 conn(0x7fe6680b4e70 0x7fe6680ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:26.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.220+0000 7fe66759e700 1 -- 172.21.15.55:0/302584313 >> 172.21.15.55:0/302584313 conn(0x7fe6680d10c0 msgr2=0x7fe6680affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:26.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.220+0000 7fe66759e700 1 -- 172.21.15.55:0/302584313 shutdown_connections 2023-10-06T03:51:26.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.220+0000 7fe66759e700 1 -- 172.21.15.55:0/302584313 wait complete. 2023-10-06T03:51:26.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.221+0000 7fe66759e700 1 Processor -- start 2023-10-06T03:51:26.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.221+0000 7fe66759e700 1 -- start start 2023-10-06T03:51:26.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.222+0000 7fe66759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6680b16d0 0x7fe66814c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:26.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.222+0000 7fe66759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6680b1b40 0x7fe668158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:26.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.222+0000 7fe66759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6680b4e70 0x7fe668159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:26.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.222+0000 7fe66759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6680cab60 con 0x7fe6680b4e70 2023-10-06T03:51:26.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.222+0000 7fe66759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66814c680 con 0x7fe6680b16d0 2023-10-06T03:51:26.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.222+0000 7fe66759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6680c7d70 con 0x7fe6680b1b40 2023-10-06T03:51:26.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.223+0000 7fe666d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6680b4e70 0x7fe668159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:26.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.223+0000 7fe666d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6680b4e70 0x7fe668159130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37040/0 (socket says 172.21.15.55:37040) 2023-10-06T03:51:26.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.223+0000 7fe666d9d700 1 -- 172.21.15.55:0/1630959210 learned_addr learned my addr 172.21.15.55:0/1630959210 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:26.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.223+0000 7fe665d9b700 1 --2- 172.21.15.55:0/1630959210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6680b1b40 0x7fe668158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:26.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.223+0000 7fe66659c700 1 --2- 172.21.15.55:0/1630959210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6680b16d0 0x7fe66814c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:26.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.223+0000 7fe666d9d700 1 -- 172.21.15.55:0/1630959210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6680b1b40 msgr2=0x7fe668158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:26.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.223+0000 7fe666d9d700 1 --2- 172.21.15.55:0/1630959210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6680b1b40 0x7fe668158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:26.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.223+0000 7fe666d9d700 1 -- 172.21.15.55:0/1630959210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6680b16d0 msgr2=0x7fe66814c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:26.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.223+0000 7fe666d9d700 1 --2- 172.21.15.55:0/1630959210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6680b16d0 0x7fe66814c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:26.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.223+0000 7fe666d9d700 1 -- 172.21.15.55:0/1630959210 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe658009420 con 0x7fe6680b4e70 2023-10-06T03:51:26.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.223+0000 7fe666d9d700 1 --2- 172.21.15.55:0/1630959210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6680b4e70 0x7fe668159130 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fe660009430 tx=0x7fe660018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:26.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.224+0000 7fe6567fc700 1 -- 172.21.15.55:0/1630959210 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660008b00 con 0x7fe6680b4e70 2023-10-06T03:51:26.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.224+0000 7fe6567fc700 1 -- 172.21.15.55:0/1630959210 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe660008c60 con 0x7fe6680b4e70 2023-10-06T03:51:26.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.224+0000 7fe6567fc700 1 -- 172.21.15.55:0/1630959210 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe660026460 con 0x7fe6680b4e70 2023-10-06T03:51:26.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.224+0000 7fe66759e700 1 -- 172.21.15.55:0/1630959210 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe660009ae0 con 0x7fe6680b4e70 2023-10-06T03:51:26.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.224+0000 7fe66759e700 1 -- 172.21.15.55:0/1630959210 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6680c7720 con 0x7fe6680b4e70 2023-10-06T03:51:26.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.225+0000 7fe66759e700 1 -- 172.21.15.55:0/1630959210 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6680c7580 con 0x7fe6680b4e70 2023-10-06T03:51:26.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.226+0000 7fe6567fc700 1 -- 172.21.15.55:0/1630959210 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe660003040 con 0x7fe6680b4e70 2023-10-06T03:51:26.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.228+0000 7fe6567fc700 1 --2- 172.21.15.55:0/1630959210 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe648066f30 0x7fe6480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:26.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.228+0000 7fe6567fc700 1 -- 172.21.15.55:0/1630959210 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6350+0+0 (secure 0 0 0) 0x7fe6600867f0 con 0x7fe6680b4e70 2023-10-06T03:51:26.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.229+0000 7fe66659c700 1 --2- 172.21.15.55:0/1630959210 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe648066f30 0x7fe6480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:26.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.229+0000 7fe66659c700 1 --2- 172.21.15.55:0/1630959210 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe648066f30 0x7fe6480693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe658009f40 tx=0x7fe6580091a0 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:26.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.232+0000 7fe6567fc700 1 -- 172.21.15.55:0/1630959210 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6600537c0 con 0x7fe6680b4e70 2023-10-06T03:51:26.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.478+0000 7fe66759e700 1 -- 172.21.15.55:0/1630959210 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe6680c7580 con 0x7fe6680b4e70 2023-10-06T03:51:26.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:26.991+0000 7fe6567fc700 1 -- 172.21.15.55:0/1630959210 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v57) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe6680c7580 con 0x7fe6680b4e70 2023-10-06T03:51:27.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.002+0000 7fe66759e700 1 -- 172.21.15.55:0/1630959210 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe648066f30 msgr2=0x7fe6480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:27.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.002+0000 7fe66759e700 1 --2- 172.21.15.55:0/1630959210 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe648066f30 0x7fe6480693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe658009f40 tx=0x7fe6580091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.002+0000 7fe66759e700 1 -- 172.21.15.55:0/1630959210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6680b4e70 msgr2=0x7fe668159130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:27.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.002+0000 7fe66759e700 1 --2- 172.21.15.55:0/1630959210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6680b4e70 0x7fe668159130 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fe660009430 tx=0x7fe660018000 comp rx=0 tx=0).stop 2023-10-06T03:51:27.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.003+0000 7fe66759e700 1 -- 172.21.15.55:0/1630959210 shutdown_connections 2023-10-06T03:51:27.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.003+0000 7fe66759e700 1 --2- 172.21.15.55:0/1630959210 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe648066f30 0x7fe6480693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.003+0000 7fe66759e700 1 --2- 172.21.15.55:0/1630959210 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6680b4e70 0x7fe668159130 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.003+0000 7fe66759e700 1 --2- 172.21.15.55:0/1630959210 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6680b16d0 0x7fe66814c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.003+0000 7fe66759e700 1 --2- 172.21.15.55:0/1630959210 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe6680b1b40 0x7fe668158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.003+0000 7fe66759e700 1 -- 172.21.15.55:0/1630959210 >> 172.21.15.55:0/1630959210 conn(0x7fe6680d10c0 msgr2=0x7fe6680b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:27.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.007+0000 7fe66759e700 1 -- 172.21.15.55:0/1630959210 shutdown_connections 2023-10-06T03:51:27.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.008+0000 7fe66759e700 1 -- 172.21.15.55:0/1630959210 wait complete. 2023-10-06T03:51:27.018 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 15 and data pool 16 2023-10-06T03:51:27.039 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:51:27.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.226+0000 7f1eeed9d700 1 Processor -- start 2023-10-06T03:51:27.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.226+0000 7f1eeed9d700 1 -- start start 2023-10-06T03:51:27.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.227+0000 7f1eeed9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1ee81597f0 0x7f1ee80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:27.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.227+0000 7f1eeed9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1ee815d230 0x7f1ee815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:27.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.227+0000 7f1eeed9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee815db70 con 0x7f1ee8159420 2023-10-06T03:51:27.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.227+0000 7f1eeed9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee80b11d0 con 0x7f1ee81597f0 2023-10-06T03:51:27.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.227+0000 7f1eeed9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee80b14b0 con 0x7f1ee815d230 2023-10-06T03:51:27.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.228+0000 7f1eed59a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1ee81597f0 0x7f1ee80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:27.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.228+0000 7f1eedd9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1ee8159420 0x7f1ee80b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38140/0 (socket says 172.21.15.55:38140) 2023-10-06T03:51:27.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.228+0000 7f1eedd9b700 1 -- 172.21.15.55:0/2270904283 learned_addr learned my addr 172.21.15.55:0/2270904283 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:27.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.228+0000 7f1eee59c700 1 --2- 172.21.15.55:0/2270904283 >> v2:172.21.15.163:3300/0 conn(0x7f1ee815d230 0x7f1ee815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:27.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.228+0000 7f1eed59a700 1 -- 172.21.15.55:0/2270904283 >> v2:172.21.15.163:3300/0 conn(0x7f1ee815d230 msgr2=0x7f1ee815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:27.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.228+0000 7f1eed59a700 1 --2- 172.21.15.55:0/2270904283 >> v2:172.21.15.163:3300/0 conn(0x7f1ee815d230 0x7f1ee815d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.228+0000 7f1eed59a700 1 -- 172.21.15.55:0/2270904283 >> v1:172.21.15.55:6789/0 conn(0x7f1ee8159420 legacy=0x7f1ee80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:27.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.228+0000 7f1eed59a700 1 -- 172.21.15.55:0/2270904283 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee80b1790 con 0x7f1ee81597f0 2023-10-06T03:51:27.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.228+0000 7f1eed59a700 1 --2- 172.21.15.55:0/2270904283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ee81597f0 0x7f1ee80b2270 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1ee815d170 tx=0x7f1ed80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becb40b128de58b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:27.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.228+0000 7f1eecd99700 1 -- 172.21.15.55:0/2270904283 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8011630 con 0x7f1ee81597f0 2023-10-06T03:51:27.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.229+0000 7f1eecd99700 1 -- 172.21.15.55:0/2270904283 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed8011790 con 0x7f1ee81597f0 2023-10-06T03:51:27.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.229+0000 7f1eecd99700 1 -- 172.21.15.55:0/2270904283 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8011b30 con 0x7f1ee81597f0 2023-10-06T03:51:27.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.229+0000 7f1eeed9d700 1 -- 172.21.15.55:0/2270904283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ee81597f0 msgr2=0x7f1ee80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:27.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.229+0000 7f1eeed9d700 1 --2- 172.21.15.55:0/2270904283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ee81597f0 0x7f1ee80b2270 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1ee815d170 tx=0x7f1ed80091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.229+0000 7f1eeed9d700 1 -- 172.21.15.55:0/2270904283 shutdown_connections 2023-10-06T03:51:27.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.229+0000 7f1eeed9d700 1 --2- 172.21.15.55:0/2270904283 >> v2:172.21.15.163:3300/0 conn(0x7f1ee815d230 0x7f1ee815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.229+0000 7f1eeed9d700 1 --2- 172.21.15.55:0/2270904283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ee81597f0 0x7f1ee80b2270 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.229+0000 7f1eeed9d700 1 -- 172.21.15.55:0/2270904283 >> 172.21.15.55:0/2270904283 conn(0x7f1ee80d0660 msgr2=0x7f1ee80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:27.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.229+0000 7f1eeed9d700 1 -- 172.21.15.55:0/2270904283 shutdown_connections 2023-10-06T03:51:27.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.229+0000 7f1eeed9d700 1 -- 172.21.15.55:0/2270904283 wait complete. 2023-10-06T03:51:27.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.231+0000 7f1eeed9d700 1 Processor -- start 2023-10-06T03:51:27.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.231+0000 7f1eeed9d700 1 -- start start 2023-10-06T03:51:27.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eeed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ee8159420 0x7f1ee81431b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:27.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eeed9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ee81597f0 0x7f1ee8155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:27.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eeed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ee815d230 0x7f1ee8156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:27.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eeed9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee815b8e0 con 0x7f1ee8159420 2023-10-06T03:51:27.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eeed9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee815ad10 con 0x7f1ee815d230 2023-10-06T03:51:27.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eeed9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee815bd70 con 0x7f1ee81597f0 2023-10-06T03:51:27.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eedd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ee8159420 0x7f1ee81431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:27.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eedd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ee8159420 0x7f1ee81431b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37056/0 (socket says 172.21.15.55:37056) 2023-10-06T03:51:27.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eedd9b700 1 -- 172.21.15.55:0/1973720603 learned_addr learned my addr 172.21.15.55:0/1973720603 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:27.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eee59c700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ee815d230 0x7f1ee8156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:27.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eed59a700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ee81597f0 0x7f1ee8155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:27.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eedd9b700 1 -- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ee81597f0 msgr2=0x7f1ee8155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:27.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eedd9b700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ee81597f0 0x7f1ee8155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eedd9b700 1 -- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ee815d230 msgr2=0x7f1ee8156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:27.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eedd9b700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ee815d230 0x7f1ee8156310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eedd9b700 1 -- 172.21.15.55:0/1973720603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed8009420 con 0x7f1ee8159420 2023-10-06T03:51:27.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.232+0000 7f1eee59c700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ee815d230 0x7f1ee8156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:27.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.233+0000 7f1eedd9b700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ee8159420 0x7f1ee81431b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400c7c0 tx=0x7f1ee4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:27.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.233+0000 7f1eddffb700 1 -- 172.21.15.55:0/1973720603 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4005b30 con 0x7f1ee8159420 2023-10-06T03:51:27.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.233+0000 7f1eddffb700 1 -- 172.21.15.55:0/1973720603 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee4005cb0 con 0x7f1ee8159420 2023-10-06T03:51:27.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.233+0000 7f1eddffb700 1 -- 172.21.15.55:0/1973720603 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4004a80 con 0x7f1ee8159420 2023-10-06T03:51:27.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.233+0000 7f1eeed9d700 1 -- 172.21.15.55:0/1973720603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee815afb0 con 0x7f1ee8159420 2023-10-06T03:51:27.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.233+0000 7f1eeed9d700 1 -- 172.21.15.55:0/1973720603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee815b570 con 0x7f1ee8159420 2023-10-06T03:51:27.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.234+0000 7f1eeed9d700 1 -- 172.21.15.55:0/1973720603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee815bbc0 con 0x7f1ee8159420 2023-10-06T03:51:27.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.235+0000 7f1eddffb700 1 -- 172.21.15.55:0/1973720603 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ee40070f0 con 0x7f1ee8159420 2023-10-06T03:51:27.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.236+0000 7f1eddffb700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1ecc066f30 0x7f1ecc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:27.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.236+0000 7f1eddffb700 1 -- 172.21.15.55:0/1973720603 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f1ee4083d40 con 0x7f1ee8159420 2023-10-06T03:51:27.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.236+0000 7f1eed59a700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1ecc066f30 0x7f1ecc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:27.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.237+0000 7f1eed59a700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1ecc066f30 0x7f1ecc0693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8159c70 tx=0x7f1ed8017000 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:27.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.240+0000 7f1eddffb700 1 -- 172.21.15.55:0/1973720603 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ee4050ca0 con 0x7f1ee8159420 2023-10-06T03:51:27.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.445+0000 7f1eeed9d700 1 -- 172.21.15.55:0/1973720603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ee804c3c0 con 0x7f1ee8159420 2023-10-06T03:51:27.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.446+0000 7f1eddffb700 1 -- 172.21.15.55:0/1973720603 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17536 (secure 0 0 0) 0x7f1ee4018030 con 0x7f1ee8159420 2023-10-06T03:51:27.583 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:27.583 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":122,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:51:26.480025+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:51:24.466543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T03:51:25.462282+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:51:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.448+0000 7f1eeed9d700 1 -- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1ecc066f30 msgr2=0x7f1ecc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.448+0000 7f1eeed9d700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1ecc066f30 0x7f1ecc0693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8159c70 tx=0x7f1ed8017000 comp rx=0 tx=0).stop 2023-10-06T03:51:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.448+0000 7f1eeed9d700 1 -- 172.21.15.55:0/1973720603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ee8159420 msgr2=0x7f1ee81431b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.448+0000 7f1eeed9d700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ee8159420 0x7f1ee81431b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400c7c0 tx=0x7f1ee4007f70 comp rx=0 tx=0).stop 2023-10-06T03:51:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.449+0000 7f1eeed9d700 1 -- 172.21.15.55:0/1973720603 shutdown_connections 2023-10-06T03:51:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.449+0000 7f1eeed9d700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1ecc066f30 0x7f1ecc0693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.449+0000 7f1eeed9d700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ee815d230 0x7f1ee8156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.449+0000 7f1eeed9d700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ee8159420 0x7f1ee81431b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.449+0000 7f1eeed9d700 1 --2- 172.21.15.55:0/1973720603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ee81597f0 0x7f1ee8155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.449+0000 7f1eeed9d700 1 -- 172.21.15.55:0/1973720603 >> 172.21.15.55:0/1973720603 conn(0x7f1ee80d0660 msgr2=0x7f1ee8155450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.449+0000 7f1eeed9d700 1 -- 172.21.15.55:0/1973720603 shutdown_connections 2023-10-06T03:51:27.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.449+0000 7f1eeed9d700 1 -- 172.21.15.55:0/1973720603 wait complete. 2023-10-06T03:51:27.587 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:51:27.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.773+0000 7fe0dc99d700 1 Processor -- start 2023-10-06T03:51:27.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.773+0000 7fe0dc99d700 1 -- start start 2023-10-06T03:51:27.799 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.774+0000 7fe0dc99d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe0d8159420 0x7fe0d80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:27.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.774+0000 7fe0dc99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0d81597f0 0x7fe0d80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:27.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.774+0000 7fe0dc99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe0d815d230 0x7fe0d815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:27.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.774+0000 7fe0dc99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d815dbd0 con 0x7fe0d815d230 2023-10-06T03:51:27.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.774+0000 7fe0dc99d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d80b11d0 con 0x7fe0d8159420 2023-10-06T03:51:27.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.774+0000 7fe0dc99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d80b14b0 con 0x7fe0d81597f0 2023-10-06T03:51:27.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.775+0000 7fe0d7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe0d815d230 0x7fe0d815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:27.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.775+0000 7fe0d77fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe0d8159420 0x7fe0d80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:27.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.775+0000 7fe0d6ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0d81597f0 0x7fe0d80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:27.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.775+0000 7fe0d77fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe0d8159420 0x7fe0d80b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:47570/0 (socket says 172.21.15.55:47570) 2023-10-06T03:51:27.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.775+0000 7fe0d77fe700 1 -- 172.21.15.55:0/3836542993 learned_addr learned my addr 172.21.15.55:0/3836542993 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:27.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.775+0000 7fe0d6ffd700 1 -- 172.21.15.55:0/3836542993 >> v2:172.21.15.163:3300/0 conn(0x7fe0d8159420 msgr2=0x7fe0d80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:27.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.775+0000 7fe0d6ffd700 1 --2- 172.21.15.55:0/3836542993 >> v2:172.21.15.163:3300/0 conn(0x7fe0d8159420 0x7fe0d80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.775+0000 7fe0d6ffd700 1 -- 172.21.15.55:0/3836542993 >> v2:172.21.15.55:3300/0 conn(0x7fe0d815d230 msgr2=0x7fe0d815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:27.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.775+0000 7fe0d6ffd700 1 --2- 172.21.15.55:0/3836542993 >> v2:172.21.15.55:3300/0 conn(0x7fe0d815d230 0x7fe0d815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.775+0000 7fe0d6ffd700 1 -- 172.21.15.55:0/3836542993 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d80b1790 con 0x7fe0d81597f0 2023-10-06T03:51:27.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.775+0000 7fe0d7fff700 1 --2- 172.21.15.55:0/3836542993 >> v2:172.21.15.55:3300/0 conn(0x7fe0d815d230 0x7fe0d815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:27.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.775+0000 7fe0d77fe700 1 --2- 172.21.15.55:0/3836542993 >> v2:172.21.15.163:3300/0 conn(0x7fe0d8159420 0x7fe0d80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:27.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.776+0000 7fe0d6ffd700 1 --2- 172.21.15.55:0/3836542993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0d81597f0 0x7fe0d80b2270 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80b2f20 tx=0x7fe0cc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc40babeca492aa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:27.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.776+0000 7fe0d67fc700 1 -- 172.21.15.55:0/3836542993 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc011630 con 0x7fe0d81597f0 2023-10-06T03:51:27.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.776+0000 7fe0d67fc700 1 -- 172.21.15.55:0/3836542993 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0cc0117b0 con 0x7fe0d81597f0 2023-10-06T03:51:27.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.776+0000 7fe0d67fc700 1 -- 172.21.15.55:0/3836542993 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc011b70 con 0x7fe0d81597f0 2023-10-06T03:51:27.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.776+0000 7fe0dc99d700 1 -- 172.21.15.55:0/3836542993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0d81597f0 msgr2=0x7fe0d80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:27.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.776+0000 7fe0dc99d700 1 --2- 172.21.15.55:0/3836542993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0d81597f0 0x7fe0d80b2270 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80b2f20 tx=0x7fe0cc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.777+0000 7fe0dc99d700 1 -- 172.21.15.55:0/3836542993 shutdown_connections 2023-10-06T03:51:27.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.777+0000 7fe0dc99d700 1 --2- 172.21.15.55:0/3836542993 >> v2:172.21.15.55:3300/0 conn(0x7fe0d815d230 0x7fe0d815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.777+0000 7fe0dc99d700 1 --2- 172.21.15.55:0/3836542993 >> v2:172.21.15.163:3300/0 conn(0x7fe0d8159420 0x7fe0d80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.777+0000 7fe0dc99d700 1 --2- 172.21.15.55:0/3836542993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0d81597f0 0x7fe0d80b2270 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.777+0000 7fe0dc99d700 1 -- 172.21.15.55:0/3836542993 >> 172.21.15.55:0/3836542993 conn(0x7fe0d80d0660 msgr2=0x7fe0d80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:27.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.777+0000 7fe0dc99d700 1 -- 172.21.15.55:0/3836542993 shutdown_connections 2023-10-06T03:51:27.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.777+0000 7fe0dc99d700 1 -- 172.21.15.55:0/3836542993 wait complete. 2023-10-06T03:51:27.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.778+0000 7fe0dc99d700 1 Processor -- start 2023-10-06T03:51:27.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.778+0000 7fe0dc99d700 1 -- start start 2023-10-06T03:51:27.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.779+0000 7fe0dc99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d8159420 0x7fe0d814d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:27.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.779+0000 7fe0dc99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0d81597f0 0x7fe0d8155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:27.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.779+0000 7fe0dc99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0d815d230 0x7fe0d81559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:27.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.779+0000 7fe0dc99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d814e6d0 con 0x7fe0d8159420 2023-10-06T03:51:27.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.779+0000 7fe0dc99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d814dac0 con 0x7fe0d815d230 2023-10-06T03:51:27.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.779+0000 7fe0dc99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d814dda0 con 0x7fe0d81597f0 2023-10-06T03:51:27.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.779+0000 7fe0d77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d8159420 0x7fe0d814d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:27.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.779+0000 7fe0d77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d8159420 0x7fe0d814d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37082/0 (socket says 172.21.15.55:37082) 2023-10-06T03:51:27.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.779+0000 7fe0d77fe700 1 -- 172.21.15.55:0/683846463 learned_addr learned my addr 172.21.15.55:0/683846463 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:27.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.779+0000 7fe0d6ffd700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0d81597f0 0x7fe0d8155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.779+0000 7fe0d7fff700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0d815d230 0x7fe0d81559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.780+0000 7fe0d77fe700 1 -- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0d81597f0 msgr2=0x7fe0d8155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.780+0000 7fe0d77fe700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0d81597f0 0x7fe0d8155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.780+0000 7fe0d77fe700 1 -- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0d815d230 msgr2=0x7fe0d81559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.780+0000 7fe0d77fe700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0d815d230 0x7fe0d81559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.780+0000 7fe0d77fe700 1 -- 172.21.15.55:0/683846463 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c80094f0 con 0x7fe0d8159420 2023-10-06T03:51:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.780+0000 7fe0d7fff700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0d815d230 0x7fe0d81559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.780+0000 7fe0d77fe700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d8159420 0x7fe0d814d2d0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800ec50 tx=0x7fe0c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.780+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/683846463 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80097d0 con 0x7fe0d8159420 2023-10-06T03:51:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.780+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/683846463 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c800c920 con 0x7fe0d8159420 2023-10-06T03:51:27.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.780+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/683846463 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c800cca0 con 0x7fe0d8159420 2023-10-06T03:51:27.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.780+0000 7fe0dc99d700 1 -- 172.21.15.55:0/683846463 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d0009a20 con 0x7fe0d8159420 2023-10-06T03:51:27.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.780+0000 7fe0dc99d700 1 -- 172.21.15.55:0/683846463 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0cc009420 con 0x7fe0d8159420 2023-10-06T03:51:27.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.781+0000 7fe0dc99d700 1 -- 172.21.15.55:0/683846463 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d804c3c0 con 0x7fe0d8159420 2023-10-06T03:51:27.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.782+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/683846463 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe0c8013680 con 0x7fe0d8159420 2023-10-06T03:51:27.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.783+0000 7fe0bf7fe700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe0c4066f30 0x7fe0c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:27.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.783+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/683846463 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fe0c8084f90 con 0x7fe0d8159420 2023-10-06T03:51:27.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.784+0000 7fe0d6ffd700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe0c4066f30 0x7fe0c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:27.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.784+0000 7fe0d6ffd700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe0c4066f30 0x7fe0c40693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80b2f20 tx=0x7fe0cc017000 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:27.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.787+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/683846463 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0c8051e70 con 0x7fe0d8159420 2023-10-06T03:51:27.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.992+0000 7fe0dc99d700 1 -- 172.21.15.55:0/683846463 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0d80b1a70 con 0x7fe0d8159420 2023-10-06T03:51:28.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.993+0000 7fe0bf7fe700 1 -- 172.21.15.55:0/683846463 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17523 (secure 0 0 0) 0x7fe0c8017090 con 0x7fe0d8159420 2023-10-06T03:51:28.006 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:28.006 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":123,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:51:27.985532+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:51:24.466543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T03:51:25.462282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:51:28.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.995+0000 7fe0dc99d700 1 -- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe0c4066f30 msgr2=0x7fe0c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.995+0000 7fe0dc99d700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe0c4066f30 0x7fe0c40693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80b2f20 tx=0x7fe0cc017000 comp rx=0 tx=0).stop 2023-10-06T03:51:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.995+0000 7fe0dc99d700 1 -- 172.21.15.55:0/683846463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d8159420 msgr2=0x7fe0d814d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.995+0000 7fe0dc99d700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d8159420 0x7fe0d814d2d0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800ec50 tx=0x7fe0c8009250 comp rx=0 tx=0).stop 2023-10-06T03:51:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.995+0000 7fe0dc99d700 1 -- 172.21.15.55:0/683846463 shutdown_connections 2023-10-06T03:51:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.995+0000 7fe0dc99d700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe0c4066f30 0x7fe0c40693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.995+0000 7fe0dc99d700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0d815d230 0x7fe0d81559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.995+0000 7fe0dc99d700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0d8159420 0x7fe0d814d2d0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.995+0000 7fe0dc99d700 1 --2- 172.21.15.55:0/683846463 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0d81597f0 0x7fe0d8155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.995+0000 7fe0dc99d700 1 -- 172.21.15.55:0/683846463 >> 172.21.15.55:0/683846463 conn(0x7fe0d80d0660 msgr2=0x7fe0d81569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.995+0000 7fe0dc99d700 1 -- 172.21.15.55:0/683846463 shutdown_connections 2023-10-06T03:51:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:27.996+0000 7fe0dc99d700 1 -- 172.21.15.55:0/683846463 wait complete. 2023-10-06T03:51:28.016 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:51:28.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.205+0000 7feb2579d700 1 Processor -- start 2023-10-06T03:51:28.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.205+0000 7feb2579d700 1 -- start start 2023-10-06T03:51:28.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.206+0000 7feb2579d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feb201407e0 0x7feb2006a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:28.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.206+0000 7feb2579d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb20140bb0 0x7feb2006a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:28.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.206+0000 7feb2579d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb200c6820 con 0x7feb2006d4f0 2023-10-06T03:51:28.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.206+0000 7feb2579d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb200c6de0 con 0x7feb201407e0 2023-10-06T03:51:28.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.206+0000 7feb2579d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb20141270 con 0x7feb20140bb0 2023-10-06T03:51:28.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.207+0000 7feb1ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feb201407e0 0x7feb2006a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:28.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.207+0000 7feb1f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb20140bb0 0x7feb2006a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:28.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.207+0000 7feb24f9c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7feb2006d4f0 0x7feb2015b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:50104/0 (socket says 172.21.15.55:50104) 2023-10-06T03:51:28.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.207+0000 7feb1f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb20140bb0 0x7feb2006a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:35904/0 (socket says 172.21.15.55:35904) 2023-10-06T03:51:28.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.207+0000 7feb24f9c700 1 -- 172.21.15.55:0/3041928611 learned_addr learned my addr 172.21.15.55:0/3041928611 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:28.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.207+0000 7feb1ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feb201407e0 0x7feb2006a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:47582/0 (socket says 172.21.15.55:47582) 2023-10-06T03:51:28.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.207+0000 7feb1f7fe700 1 -- 172.21.15.55:0/3041928611 >> v2:172.21.15.163:3300/0 conn(0x7feb201407e0 msgr2=0x7feb2006a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:28.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.207+0000 7feb1effd700 1 -- 172.21.15.55:0/3041928611 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156884968 0 0) 0x7feb200c6820 con 0x7feb2006d4f0 2023-10-06T03:51:28.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.207+0000 7feb1f7fe700 1 --2- 172.21.15.55:0/3041928611 >> v2:172.21.15.163:3300/0 conn(0x7feb201407e0 0x7feb2006a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:28.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.207+0000 7feb1f7fe700 1 -- 172.21.15.55:0/3041928611 >> v1:172.21.15.163:6789/0 conn(0x7feb2006d4f0 legacy=0x7feb2015b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:28.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.208+0000 7feb1f7fe700 1 -- 172.21.15.55:0/3041928611 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2006acf0 con 0x7feb20140bb0 2023-10-06T03:51:28.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.208+0000 7feb1f7fe700 1 --2- 172.21.15.55:0/3041928611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb20140bb0 0x7feb2006a6c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feb140098e0 tx=0x7feb140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d0ab554401be4f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:28.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.208+0000 7feb1effd700 1 -- 172.21.15.55:0/3041928611 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1400b640 con 0x7feb20140bb0 2023-10-06T03:51:28.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.208+0000 7feb1effd700 1 -- 172.21.15.55:0/3041928611 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1400b7a0 con 0x7feb20140bb0 2023-10-06T03:51:28.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.208+0000 7feb1effd700 1 -- 172.21.15.55:0/3041928611 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1400bb40 con 0x7feb20140bb0 2023-10-06T03:51:28.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.209+0000 7feb2579d700 1 -- 172.21.15.55:0/3041928611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb20140bb0 msgr2=0x7feb2006a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:28.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.209+0000 7feb2579d700 1 --2- 172.21.15.55:0/3041928611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb20140bb0 0x7feb2006a6c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feb140098e0 tx=0x7feb140091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:28.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.209+0000 7feb2579d700 1 -- 172.21.15.55:0/3041928611 shutdown_connections 2023-10-06T03:51:28.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.209+0000 7feb2579d700 1 --2- 172.21.15.55:0/3041928611 >> v2:172.21.15.163:3300/0 conn(0x7feb201407e0 0x7feb2006a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:28.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.209+0000 7feb2579d700 1 --2- 172.21.15.55:0/3041928611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb20140bb0 0x7feb2006a6c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:28.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.209+0000 7feb2579d700 1 -- 172.21.15.55:0/3041928611 >> 172.21.15.55:0/3041928611 conn(0x7feb200cc1e0 msgr2=0x7feb200cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:28.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.209+0000 7feb2579d700 1 -- 172.21.15.55:0/3041928611 shutdown_connections 2023-10-06T03:51:28.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.209+0000 7feb2579d700 1 -- 172.21.15.55:0/3041928611 wait complete. 2023-10-06T03:51:28.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.210+0000 7feb2579d700 1 Processor -- start 2023-10-06T03:51:28.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.210+0000 7feb2579d700 1 -- start start 2023-10-06T03:51:28.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.211+0000 7feb2579d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb2006d4f0 0x7feb200cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:28.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.211+0000 7feb2579d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb201407e0 0x7feb2013f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:28.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.211+0000 7feb2579d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb20140bb0 0x7feb2016dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:28.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.211+0000 7feb2579d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2013fa90 con 0x7feb201407e0 2023-10-06T03:51:28.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.211+0000 7feb2579d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb200cbe40 con 0x7feb2006d4f0 2023-10-06T03:51:28.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.211+0000 7feb2579d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2013f860 con 0x7feb20140bb0 2023-10-06T03:51:28.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.212+0000 7feb1f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb201407e0 0x7feb2013f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:28.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.212+0000 7feb1f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb201407e0 0x7feb2013f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37084/0 (socket says 172.21.15.55:37084) 2023-10-06T03:51:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.212+0000 7feb1f7fe700 1 -- 172.21.15.55:0/1878758423 learned_addr learned my addr 172.21.15.55:0/1878758423 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.212+0000 7feb24f9c700 1 --2- 172.21.15.55:0/1878758423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb20140bb0 0x7feb2016dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.212+0000 7feb1ffff700 1 --2- 172.21.15.55:0/1878758423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb2006d4f0 0x7feb200cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.212+0000 7feb1f7fe700 1 -- 172.21.15.55:0/1878758423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb20140bb0 msgr2=0x7feb2016dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.212+0000 7feb1f7fe700 1 --2- 172.21.15.55:0/1878758423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb20140bb0 0x7feb2016dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.212+0000 7feb1f7fe700 1 -- 172.21.15.55:0/1878758423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb2006d4f0 msgr2=0x7feb200cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.212+0000 7feb1f7fe700 1 --2- 172.21.15.55:0/1878758423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb2006d4f0 0x7feb200cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.212+0000 7feb1f7fe700 1 -- 172.21.15.55:0/1878758423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb14009420 con 0x7feb201407e0 2023-10-06T03:51:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.212+0000 7feb1f7fe700 1 --2- 172.21.15.55:0/1878758423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb201407e0 0x7feb2013f320 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7feb14013000 tx=0x7feb14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:28.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.213+0000 7feb07fff700 1 -- 172.21.15.55:0/1878758423 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1400bb20 con 0x7feb201407e0 2023-10-06T03:51:28.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.213+0000 7feb07fff700 1 -- 172.21.15.55:0/1878758423 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb14000c90 con 0x7feb201407e0 2023-10-06T03:51:28.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.213+0000 7feb07fff700 1 -- 172.21.15.55:0/1878758423 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb14011650 con 0x7feb201407e0 2023-10-06T03:51:28.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.213+0000 7feb2579d700 1 -- 172.21.15.55:0/1878758423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb200cb0a0 con 0x7feb201407e0 2023-10-06T03:51:28.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.213+0000 7feb2579d700 1 -- 172.21.15.55:0/1878758423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2013d0b0 con 0x7feb201407e0 2023-10-06T03:51:28.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.214+0000 7feb2579d700 1 -- 172.21.15.55:0/1878758423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb200b1f10 con 0x7feb201407e0 2023-10-06T03:51:28.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.215+0000 7feb07fff700 1 -- 172.21.15.55:0/1878758423 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feb14011a40 con 0x7feb201407e0 2023-10-06T03:51:28.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.216+0000 7feb07fff700 1 --2- 172.21.15.55:0/1878758423 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7feb0c066f30 0x7feb0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:28.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.216+0000 7feb07fff700 1 -- 172.21.15.55:0/1878758423 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7feb14085c50 con 0x7feb201407e0 2023-10-06T03:51:28.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.216+0000 7feb1ffff700 1 --2- 172.21.15.55:0/1878758423 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7feb0c066f30 0x7feb0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:28.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.217+0000 7feb1ffff700 1 --2- 172.21.15.55:0/1878758423 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7feb0c066f30 0x7feb0c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feb10009db0 tx=0x7feb100091a0 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:28.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.220+0000 7feb07fff700 1 -- 172.21.15.55:0/1878758423 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb14052b30 con 0x7feb201407e0 2023-10-06T03:51:28.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:28.469+0000 7feb2579d700 1 -- 172.21.15.55:0/1878758423 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feb2006c480 con 0x7feb201407e0 2023-10-06T03:51:29.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.017+0000 7feb07fff700 1 -- 172.21.15.55:0/1878758423 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) v1 ==== 118+0+0 (secure 0 0 0) 0x7feb140280b0 con 0x7feb201407e0 2023-10-06T03:51:29.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.020+0000 7feb2579d700 1 -- 172.21.15.55:0/1878758423 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7feb0c066f30 msgr2=0x7feb0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:29.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.020+0000 7feb2579d700 1 --2- 172.21.15.55:0/1878758423 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7feb0c066f30 0x7feb0c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7feb10009db0 tx=0x7feb100091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:29.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.020+0000 7feb2579d700 1 -- 172.21.15.55:0/1878758423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb201407e0 msgr2=0x7feb2013f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:29.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.020+0000 7feb2579d700 1 --2- 172.21.15.55:0/1878758423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb201407e0 0x7feb2013f320 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7feb14013000 tx=0x7feb14017000 comp rx=0 tx=0).stop 2023-10-06T03:51:29.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.020+0000 7feb2579d700 1 -- 172.21.15.55:0/1878758423 shutdown_connections 2023-10-06T03:51:29.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.020+0000 7feb2579d700 1 --2- 172.21.15.55:0/1878758423 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7feb0c066f30 0x7feb0c0693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:29.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.020+0000 7feb2579d700 1 --2- 172.21.15.55:0/1878758423 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feb20140bb0 0x7feb2016dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:29.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.020+0000 7feb2579d700 1 --2- 172.21.15.55:0/1878758423 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feb2006d4f0 0x7feb200cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:29.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.020+0000 7feb2579d700 1 --2- 172.21.15.55:0/1878758423 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feb201407e0 0x7feb2013f320 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:29.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.020+0000 7feb2579d700 1 -- 172.21.15.55:0/1878758423 >> 172.21.15.55:0/1878758423 conn(0x7feb200cc1e0 msgr2=0x7feb20069770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:29.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.021+0000 7feb2579d700 1 -- 172.21.15.55:0/1878758423 shutdown_connections 2023-10-06T03:51:29.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.021+0000 7feb2579d700 1 -- 172.21.15.55:0/1878758423 wait complete. 2023-10-06T03:51:29.047 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:51:29.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.240+0000 7f5f3e59d700 1 Processor -- start 2023-10-06T03:51:29.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.241+0000 7f5f3e59d700 1 -- start start 2023-10-06T03:51:29.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.242+0000 7f5f3e59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f3815a3b0 0x7f5f3815a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:29.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.242+0000 7f5f3e59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f380aea70 con 0x7f5f380aef00 2023-10-06T03:51:29.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.242+0000 7f5f3e59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3815dad0 con 0x7f5f3806e470 2023-10-06T03:51:29.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.242+0000 7f5f3e59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380ae480 con 0x7f5f3815a3b0 2023-10-06T03:51:29.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.242+0000 7f5f3cd9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5f380aef00 0x7f5f3815d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38148/0 (socket says 172.21.15.55:38148) 2023-10-06T03:51:29.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.242+0000 7f5f3cd9a700 1 -- 172.21.15.55:0/3280821289 learned_addr learned my addr 172.21.15.55:0/3280821289 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:29.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.243+0000 7f5f3d59b700 1 --2- 172.21.15.55:0/3280821289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f3815a3b0 0x7f5f3815a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:29.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.243+0000 7f5f2ffff700 1 -- 172.21.15.55:0/3280821289 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558964069 0 0) 0x7f5f380aea70 con 0x7f5f380aef00 2023-10-06T03:51:29.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.243+0000 7f5f2ffff700 1 -- 172.21.15.55:0/3280821289 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f24003570 con 0x7f5f380aef00 2023-10-06T03:51:29.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.243+0000 7f5f3d59b700 1 -- 172.21.15.55:0/3280821289 >> v1:172.21.15.163:6789/0 conn(0x7f5f3806e470 legacy=0x7f5f380c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.243+0000 7f5f2ffff700 1 -- 172.21.15.55:0/3280821289 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526120135 0 0) 0x7f5f3815dad0 con 0x7f5f3806e470 2023-10-06T03:51:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.243+0000 7f5f3d59b700 1 -- 172.21.15.55:0/3280821289 >> v1:172.21.15.55:6789/0 conn(0x7f5f380aef00 legacy=0x7f5f3815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.243+0000 7f5f3d59b700 1 -- 172.21.15.55:0/3280821289 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3815acc0 con 0x7f5f3815a3b0 2023-10-06T03:51:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.243+0000 7f5f3d59b700 1 --2- 172.21.15.55:0/3280821289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f3815a3b0 0x7f5f3815a780 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009910 tx=0x7f5f340091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8b729ee9f6a36c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.244+0000 7f5f2ffff700 1 -- 172.21.15.55:0/3280821289 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3400b680 con 0x7f5f3815a3b0 2023-10-06T03:51:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.244+0000 7f5f2ffff700 1 -- 172.21.15.55:0/3280821289 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3400b7e0 con 0x7f5f3815a3b0 2023-10-06T03:51:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.244+0000 7f5f2ffff700 1 -- 172.21.15.55:0/3280821289 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3400bba0 con 0x7f5f3815a3b0 2023-10-06T03:51:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.244+0000 7f5f3e59d700 1 -- 172.21.15.55:0/3280821289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f3815a3b0 msgr2=0x7f5f3815a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.244+0000 7f5f3e59d700 1 --2- 172.21.15.55:0/3280821289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f3815a3b0 0x7f5f3815a780 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009910 tx=0x7f5f340091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.244+0000 7f5f3e59d700 1 -- 172.21.15.55:0/3280821289 shutdown_connections 2023-10-06T03:51:29.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.244+0000 7f5f3e59d700 1 --2- 172.21.15.55:0/3280821289 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f3815a3b0 0x7f5f3815a780 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:29.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.244+0000 7f5f3e59d700 1 -- 172.21.15.55:0/3280821289 >> 172.21.15.55:0/3280821289 conn(0x7f5f380c9f70 msgr2=0x7f5f3806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:29.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.244+0000 7f5f3e59d700 1 -- 172.21.15.55:0/3280821289 shutdown_connections 2023-10-06T03:51:29.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.244+0000 7f5f3e59d700 1 -- 172.21.15.55:0/3280821289 wait complete. 2023-10-06T03:51:29.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.246+0000 7f5f3e59d700 1 Processor -- start 2023-10-06T03:51:29.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.246+0000 7f5f3e59d700 1 -- start start 2023-10-06T03:51:29.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.247+0000 7f5f3e59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f3806e470 0x7f5f38152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:29.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.247+0000 7f5f3e59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f380aef00 0x7f5f380c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:29.369 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.247+0000 7f5f3e59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f3815a3b0 0x7f5f380b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:29.369 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.247+0000 7f5f3e59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f381698e0 con 0x7f5f3815a3b0 2023-10-06T03:51:29.369 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.247+0000 7f5f3e59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380b07b0 con 0x7f5f3806e470 2023-10-06T03:51:29.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.247+0000 7f5f3e59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380b0a90 con 0x7f5f380aef00 2023-10-06T03:51:29.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.247+0000 7f5f3dd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f3815a3b0 0x7f5f380b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:29.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.247+0000 7f5f3dd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f3815a3b0 0x7f5f380b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37088/0 (socket says 172.21.15.55:37088) 2023-10-06T03:51:29.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.247+0000 7f5f3dd9c700 1 -- 172.21.15.55:0/2301899236 learned_addr learned my addr 172.21.15.55:0/2301899236 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:29.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.247+0000 7f5f3cd9a700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f380aef00 0x7f5f380c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:29.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.247+0000 7f5f3d59b700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f3806e470 0x7f5f38152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:29.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.248+0000 7f5f3dd9c700 1 -- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f380aef00 msgr2=0x7f5f380c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:29.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.248+0000 7f5f3dd9c700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f380aef00 0x7f5f380c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:29.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.248+0000 7f5f3dd9c700 1 -- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f3806e470 msgr2=0x7f5f38152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:29.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.248+0000 7f5f3dd9c700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f3806e470 0x7f5f38152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:29.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.248+0000 7f5f3dd9c700 1 -- 172.21.15.55:0/2301899236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f34009420 con 0x7f5f3815a3b0 2023-10-06T03:51:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.248+0000 7f5f3d59b700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f3806e470 0x7f5f38152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:51:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.248+0000 7f5f3dd9c700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f3815a3b0 0x7f5f380b1db0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5f30001560 tx=0x7f5f30001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.248+0000 7f5f2d7fa700 1 -- 172.21.15.55:0/2301899236 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3000a780 con 0x7f5f3815a3b0 2023-10-06T03:51:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.248+0000 7f5f2d7fa700 1 -- 172.21.15.55:0/2301899236 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3000a8e0 con 0x7f5f3815a3b0 2023-10-06T03:51:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.248+0000 7f5f3e59d700 1 -- 172.21.15.55:0/2301899236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f380b0d70 con 0x7f5f3815a3b0 2023-10-06T03:51:29.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.248+0000 7f5f2d7fa700 1 -- 172.21.15.55:0/2301899236 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3000aca0 con 0x7f5f3815a3b0 2023-10-06T03:51:29.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.248+0000 7f5f3e59d700 1 -- 172.21.15.55:0/2301899236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f381529c0 con 0x7f5f3815a3b0 2023-10-06T03:51:29.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.249+0000 7f5f3e59d700 1 -- 172.21.15.55:0/2301899236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f380b1290 con 0x7f5f3815a3b0 2023-10-06T03:51:29.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.250+0000 7f5f2d7fa700 1 -- 172.21.15.55:0/2301899236 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f30012630 con 0x7f5f3815a3b0 2023-10-06T03:51:29.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.251+0000 7f5f2d7fa700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5f1c066f30 0x7f5f1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:29.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.251+0000 7f5f2d7fa700 1 -- 172.21.15.55:0/2301899236 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f5f30084be0 con 0x7f5f3815a3b0 2023-10-06T03:51:29.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.252+0000 7f5f3d59b700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5f1c066f30 0x7f5f1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:29.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.252+0000 7f5f3d59b700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5f1c066f30 0x7f5f1c0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009860 tx=0x7f5f34017000 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:29.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.255+0000 7f5f2d7fa700 1 -- 172.21.15.55:0/2301899236 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f380b1290 con 0x7f5f3815a3b0 2023-10-06T03:51:29.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:29.504+0000 7f5f3e59d700 1 -- 172.21.15.55:0/2301899236 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5f380b1290 con 0x7f5f3815a3b0 2023-10-06T03:51:30.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.027+0000 7f5f2d7fa700 1 -- 172.21.15.55:0/2301899236 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v61) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5f380b1290 con 0x7f5f3815a3b0 2023-10-06T03:51:30.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.030+0000 7f5f3e59d700 1 -- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5f1c066f30 msgr2=0x7f5f1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:30.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.030+0000 7f5f3e59d700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5f1c066f30 0x7f5f1c0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009860 tx=0x7f5f34017000 comp rx=0 tx=0).stop 2023-10-06T03:51:30.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.030+0000 7f5f3e59d700 1 -- 172.21.15.55:0/2301899236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f3815a3b0 msgr2=0x7f5f380b1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:30.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.030+0000 7f5f3e59d700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f3815a3b0 0x7f5f380b1db0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5f30001560 tx=0x7f5f30001770 comp rx=0 tx=0).stop 2023-10-06T03:51:30.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.030+0000 7f5f3e59d700 1 -- 172.21.15.55:0/2301899236 shutdown_connections 2023-10-06T03:51:30.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.030+0000 7f5f3e59d700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5f1c066f30 0x7f5f1c0693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.030+0000 7f5f3e59d700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f3815a3b0 0x7f5f380b1db0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.030+0000 7f5f3e59d700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f3806e470 0x7f5f38152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.030+0000 7f5f3e59d700 1 --2- 172.21.15.55:0/2301899236 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f380aef00 0x7f5f380c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.030+0000 7f5f3e59d700 1 -- 172.21.15.55:0/2301899236 >> 172.21.15.55:0/2301899236 conn(0x7f5f380c9f70 msgr2=0x7f5f3806e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:30.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.030+0000 7f5f3e59d700 1 -- 172.21.15.55:0/2301899236 shutdown_connections 2023-10-06T03:51:30.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.030+0000 7f5f3e59d700 1 -- 172.21.15.55:0/2301899236 wait complete. 2023-10-06T03:51:30.201 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:51:30.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.392+0000 7f9e48f9d700 1 Processor -- start 2023-10-06T03:51:30.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.392+0000 7f9e48f9d700 1 -- start start 2023-10-06T03:51:30.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.394+0000 7f9e48f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e44156310 0x7f9e440c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:30.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.394+0000 7f9e48f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9e441566e0 0x7f9e440ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:30.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.394+0000 7f9e48f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e440ca860 con 0x7f9e440ca300 2023-10-06T03:51:30.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.394+0000 7f9e48f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e440cab40 con 0x7f9e441566e0 2023-10-06T03:51:30.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.394+0000 7f9e48f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440cae20 con 0x7f9e44156310 2023-10-06T03:51:30.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.394+0000 7f9e42ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9e441566e0 0x7f9e440ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:30.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.394+0000 7f9e43fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9e440ca300 0x7f9e440c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38156/0 (socket says 172.21.15.55:38156) 2023-10-06T03:51:30.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.394+0000 7f9e43fff700 1 -- 172.21.15.55:0/1906059187 learned_addr learned my addr 172.21.15.55:0/1906059187 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:30.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.394+0000 7f9e437fe700 1 --2- 172.21.15.55:0/1906059187 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e44156310 0x7f9e440c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:30.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.394+0000 7f9e42ffd700 1 -- 172.21.15.55:0/1906059187 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e44156310 msgr2=0x7f9e440c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:30.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.395+0000 7f9e42ffd700 1 --2- 172.21.15.55:0/1906059187 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e44156310 0x7f9e440c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.395+0000 7f9e42ffd700 1 -- 172.21.15.55:0/1906059187 >> v1:172.21.15.55:6789/0 conn(0x7f9e440ca300 legacy=0x7f9e440c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:30.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.395+0000 7f9e42ffd700 1 -- 172.21.15.55:0/1906059187 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e440cb100 con 0x7f9e441566e0 2023-10-06T03:51:30.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.395+0000 7f9e427fc700 1 -- 172.21.15.55:0/1906059187 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731372160 0 0) 0x7f9e440ca860 con 0x7f9e440ca300 2023-10-06T03:51:30.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.395+0000 7f9e42ffd700 1 --2- 172.21.15.55:0/1906059187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e441566e0 0x7f9e440ae000 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f9e380098e0 tx=0x7f9e380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1337f8f5b293ecd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:30.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.395+0000 7f9e427fc700 1 -- 172.21.15.55:0/1906059187 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38009e80 con 0x7f9e441566e0 2023-10-06T03:51:30.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.395+0000 7f9e427fc700 1 -- 172.21.15.55:0/1906059187 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3800b860 con 0x7f9e441566e0 2023-10-06T03:51:30.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.395+0000 7f9e427fc700 1 -- 172.21.15.55:0/1906059187 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3800bbe0 con 0x7f9e441566e0 2023-10-06T03:51:30.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.396+0000 7f9e48f9d700 1 -- 172.21.15.55:0/1906059187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e441566e0 msgr2=0x7f9e440ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:30.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.396+0000 7f9e48f9d700 1 --2- 172.21.15.55:0/1906059187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e441566e0 0x7f9e440ae000 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f9e380098e0 tx=0x7f9e380091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.396+0000 7f9e48f9d700 1 -- 172.21.15.55:0/1906059187 shutdown_connections 2023-10-06T03:51:30.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.396+0000 7f9e48f9d700 1 --2- 172.21.15.55:0/1906059187 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e44156310 0x7f9e440c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.396+0000 7f9e48f9d700 1 --2- 172.21.15.55:0/1906059187 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e441566e0 0x7f9e440ae000 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.396+0000 7f9e48f9d700 1 -- 172.21.15.55:0/1906059187 >> 172.21.15.55:0/1906059187 conn(0x7f9e440cc1f0 msgr2=0x7f9e440cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:30.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.396+0000 7f9e48f9d700 1 -- 172.21.15.55:0/1906059187 shutdown_connections 2023-10-06T03:51:30.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.396+0000 7f9e48f9d700 1 -- 172.21.15.55:0/1906059187 wait complete. 2023-10-06T03:51:30.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.397+0000 7f9e48f9d700 1 Processor -- start 2023-10-06T03:51:30.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.397+0000 7f9e48f9d700 1 -- start start 2023-10-06T03:51:30.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.398+0000 7f9e48f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e440ca300 0x7f9e4416a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:30.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.398+0000 7f9e48f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e44156310 0x7f9e440af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:30.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.398+0000 7f9e48f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e441566e0 0x7f9e440b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:30.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.398+0000 7f9e48f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4416a2f0 con 0x7f9e440ca300 2023-10-06T03:51:30.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.398+0000 7f9e48f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e44169fc0 con 0x7f9e44156310 2023-10-06T03:51:30.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.398+0000 7f9e48f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440b1840 con 0x7f9e441566e0 2023-10-06T03:51:30.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.399+0000 7f9e437fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e440ca300 0x7f9e4416a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:30.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.399+0000 7f9e437fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e440ca300 0x7f9e4416a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37118/0 (socket says 172.21.15.55:37118) 2023-10-06T03:51:30.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.399+0000 7f9e437fe700 1 -- 172.21.15.55:0/2588731040 learned_addr learned my addr 172.21.15.55:0/2588731040 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:30.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.399+0000 7f9e43fff700 1 --2- 172.21.15.55:0/2588731040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e441566e0 0x7f9e440b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:30.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.399+0000 7f9e42ffd700 1 --2- 172.21.15.55:0/2588731040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e44156310 0x7f9e440af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:30.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.399+0000 7f9e437fe700 1 -- 172.21.15.55:0/2588731040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e441566e0 msgr2=0x7f9e440b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:30.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.399+0000 7f9e437fe700 1 --2- 172.21.15.55:0/2588731040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e441566e0 0x7f9e440b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.399+0000 7f9e437fe700 1 -- 172.21.15.55:0/2588731040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e44156310 msgr2=0x7f9e440af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:30.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.399+0000 7f9e437fe700 1 --2- 172.21.15.55:0/2588731040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e44156310 0x7f9e440af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.399+0000 7f9e437fe700 1 -- 172.21.15.55:0/2588731040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e38009420 con 0x7f9e440ca300 2023-10-06T03:51:30.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.399+0000 7f9e437fe700 1 --2- 172.21.15.55:0/2588731040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e440ca300 0x7f9e4416a780 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400d890 tx=0x7f9e34008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:30.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.399+0000 7f9e2b7fe700 1 -- 172.21.15.55:0/2588731040 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3400b600 con 0x7f9e440ca300 2023-10-06T03:51:30.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.399+0000 7f9e2b7fe700 1 -- 172.21.15.55:0/2588731040 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3400b760 con 0x7f9e440ca300 2023-10-06T03:51:30.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.399+0000 7f9e2b7fe700 1 -- 172.21.15.55:0/2588731040 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3400bae0 con 0x7f9e440ca300 2023-10-06T03:51:30.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.400+0000 7f9e48f9d700 1 -- 172.21.15.55:0/2588731040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e440b1b20 con 0x7f9e440ca300 2023-10-06T03:51:30.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.400+0000 7f9e48f9d700 1 -- 172.21.15.55:0/2588731040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4416acc0 con 0x7f9e440ca300 2023-10-06T03:51:30.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.401+0000 7f9e48f9d700 1 -- 172.21.15.55:0/2588731040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4416a5d0 con 0x7f9e440ca300 2023-10-06T03:51:30.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.402+0000 7f9e2b7fe700 1 -- 172.21.15.55:0/2588731040 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e3401e460 con 0x7f9e440ca300 2023-10-06T03:51:30.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.402+0000 7f9e2b7fe700 1 --2- 172.21.15.55:0/2588731040 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9e30066f30 0x7f9e300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:30.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.402+0000 7f9e2b7fe700 1 -- 172.21.15.55:0/2588731040 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f9e34084f70 con 0x7f9e440ca300 2023-10-06T03:51:30.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.403+0000 7f9e42ffd700 1 --2- 172.21.15.55:0/2588731040 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9e30066f30 0x7f9e300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:30.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.404+0000 7f9e42ffd700 1 --2- 172.21.15.55:0/2588731040 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9e30066f30 0x7f9e300693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e380098e0 tx=0x7f9e38017000 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:30.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.407+0000 7f9e2b7fe700 1 -- 172.21.15.55:0/2588731040 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e34051e50 con 0x7f9e440ca300 2023-10-06T03:51:30.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.655+0000 7f9e48f9d700 1 -- 172.21.15.55:0/2588731040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e4404c3c0 con 0x7f9e440ca300 2023-10-06T03:51:30.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.655+0000 7f9e2b7fe700 1 -- 172.21.15.55:0/2588731040 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f9e34055470 con 0x7f9e440ca300 2023-10-06T03:51:30.657 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:30.657 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1295635544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":1295635544},{"type":"v1","addr":"172.21.15.55:6835","nonce":1295635544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7737,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2115790576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2115790576},{"type":"v1","addr":"172.21.15.163:6834","nonce":2115790576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1021807391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1021807391},{"type":"v1","addr":"172.21.15.163:6836","nonce":1021807391}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:51:26.480340+0000","modified":"2023-10-06T03:51:30.017376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7743":{"gid":7743,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/4240717455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":4240717455},{"type":"v1","addr":"172.21.15.55:6834","nonce":4240717455}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T03:51:30.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.657+0000 7f9e48f9d700 1 -- 172.21.15.55:0/2588731040 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9e30066f30 msgr2=0x7f9e300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:30.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.657+0000 7f9e48f9d700 1 --2- 172.21.15.55:0/2588731040 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9e30066f30 0x7f9e300693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9e380098e0 tx=0x7f9e38017000 comp rx=0 tx=0).stop 2023-10-06T03:51:30.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.657+0000 7f9e48f9d700 1 -- 172.21.15.55:0/2588731040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e440ca300 msgr2=0x7f9e4416a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:30.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.657+0000 7f9e48f9d700 1 --2- 172.21.15.55:0/2588731040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e440ca300 0x7f9e4416a780 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400d890 tx=0x7f9e34008040 comp rx=0 tx=0).stop 2023-10-06T03:51:30.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.657+0000 7f9e48f9d700 1 -- 172.21.15.55:0/2588731040 shutdown_connections 2023-10-06T03:51:30.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.657+0000 7f9e48f9d700 1 --2- 172.21.15.55:0/2588731040 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9e30066f30 0x7f9e300693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.657+0000 7f9e48f9d700 1 --2- 172.21.15.55:0/2588731040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9e441566e0 0x7f9e440b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.657+0000 7f9e48f9d700 1 --2- 172.21.15.55:0/2588731040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9e440ca300 0x7f9e4416a780 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.657+0000 7f9e48f9d700 1 --2- 172.21.15.55:0/2588731040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9e44156310 0x7f9e440af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.657+0000 7f9e48f9d700 1 -- 172.21.15.55:0/2588731040 >> 172.21.15.55:0/2588731040 conn(0x7f9e440cc1f0 msgr2=0x7f9e44069fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:30.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.658+0000 7f9e48f9d700 1 -- 172.21.15.55:0/2588731040 shutdown_connections 2023-10-06T03:51:30.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.658+0000 7f9e48f9d700 1 -- 172.21.15.55:0/2588731040 wait complete. 2023-10-06T03:51:30.662 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 61 2023-10-06T03:51:30.677 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:51:30.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.863+0000 7fbc3a99d700 1 Processor -- start 2023-10-06T03:51:30.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.863+0000 7fbc3a99d700 1 -- start start 2023-10-06T03:51:30.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.864+0000 7fbc3a99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc3406a660 0x7fbc34157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:30.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.864+0000 7fbc3a99d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3406e2a0 con 0x7fbc34150550 2023-10-06T03:51:30.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.864+0000 7fbc3a99d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3406b2e0 con 0x7fbc34150180 2023-10-06T03:51:30.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.864+0000 7fbc3a99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc340c78e0 con 0x7fbc3406a660 2023-10-06T03:51:30.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.864+0000 7fbc3919a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbc34150550 0x7fbc34069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38172/0 (socket says 172.21.15.55:38172) 2023-10-06T03:51:30.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.864+0000 7fbc3919a700 1 -- 172.21.15.55:0/2827585344 learned_addr learned my addr 172.21.15.55:0/2827585344 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:30.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 -- 172.21.15.55:0/2827585344 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761710567 0 0) 0x7fbc3406e2a0 con 0x7fbc34150550 2023-10-06T03:51:30.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc3a19c700 1 --2- 172.21.15.55:0/2827585344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc3406a660 0x7fbc34157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:30.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 -- 172.21.15.55:0/2827585344 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc20003570 con 0x7fbc34150550 2023-10-06T03:51:30.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 -- 172.21.15.55:0/2827585344 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1514830987 0 0) 0x7fbc20003570 con 0x7fbc34150550 2023-10-06T03:51:30.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 -- 172.21.15.55:0/2827585344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc3406a660 msgr2=0x7fbc34157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:30.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 --2- 172.21.15.55:0/2827585344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc3406a660 0x7fbc34157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 -- 172.21.15.55:0/2827585344 >> v1:172.21.15.163:6789/0 conn(0x7fbc34150180 legacy=0x7fbc340c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:30.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 -- 172.21.15.55:0/2827585344 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc340c7bc0 con 0x7fbc34150550 2023-10-06T03:51:30.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 -- 172.21.15.55:0/2827585344 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fbc24001470 con 0x7fbc34150550 2023-10-06T03:51:30.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 -- 172.21.15.55:0/2827585344 >> v1:172.21.15.55:6789/0 conn(0x7fbc34150550 legacy=0x7fbc34069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:30.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 --2- 172.21.15.55:0/2827585344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc20003a90 0x7fbc20003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:30.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 --2- 172.21.15.55:0/2827585344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20004810 0x7fbc20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:30.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 --2- 172.21.15.55:0/2827585344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc3406a660 0x7fbc200051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:30.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 -- 172.21.15.55:0/2827585344 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc340c78e0 con 0x7fbc20004810 2023-10-06T03:51:30.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 -- 172.21.15.55:0/2827585344 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2000bb60 con 0x7fbc20003a90 2023-10-06T03:51:30.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.865+0000 7fbc38999700 1 -- 172.21.15.55:0/2827585344 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2000bd20 con 0x7fbc3406a660 2023-10-06T03:51:30.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.866+0000 7fbc3919a700 1 --2- 172.21.15.55:0/2827585344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20004810 0x7fbc20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:30.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.866+0000 7fbc3999b700 1 --2- 172.21.15.55:0/2827585344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc20003a90 0x7fbc20003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:30.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.866+0000 7fbc3a19c700 1 --2- 172.21.15.55:0/2827585344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc3406a660 0x7fbc200051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:30.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.866+0000 7fbc3919a700 1 -- 172.21.15.55:0/2827585344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc3406a660 msgr2=0x7fbc200051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:30.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.866+0000 7fbc3919a700 1 --2- 172.21.15.55:0/2827585344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc3406a660 0x7fbc200051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.866+0000 7fbc3919a700 1 -- 172.21.15.55:0/2827585344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc20003a90 msgr2=0x7fbc20003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:30.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.866+0000 7fbc3919a700 1 --2- 172.21.15.55:0/2827585344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc20003a90 0x7fbc20003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.866+0000 7fbc3919a700 1 -- 172.21.15.55:0/2827585344 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc20003570 con 0x7fbc20004810 2023-10-06T03:51:30.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.866+0000 7fbc3a19c700 1 --2- 172.21.15.55:0/2827585344 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc3406a660 0x7fbc200051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:30.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.866+0000 7fbc3999b700 1 --2- 172.21.15.55:0/2827585344 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc20003a90 0x7fbc20003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:30.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.867+0000 7fbc3919a700 1 --2- 172.21.15.55:0/2827585344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20004810 0x7fbc20004c80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400c3c0 tx=0x7fbc24001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b0f00ce5ff235b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:30.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.867+0000 7fbc38999700 1 -- 172.21.15.55:0/2827585344 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2400cc90 con 0x7fbc20004810 2023-10-06T03:51:30.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.867+0000 7fbc38999700 1 -- 172.21.15.55:0/2827585344 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2400cdf0 con 0x7fbc20004810 2023-10-06T03:51:30.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.867+0000 7fbc38999700 1 -- 172.21.15.55:0/2827585344 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24005b50 con 0x7fbc20004810 2023-10-06T03:51:30.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.867+0000 7fbc3a99d700 1 -- 172.21.15.55:0/2827585344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20004810 msgr2=0x7fbc20004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:30.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.867+0000 7fbc3a99d700 1 --2- 172.21.15.55:0/2827585344 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc20004810 0x7fbc20004c80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400c3c0 tx=0x7fbc24001e50 comp rx=0 tx=0).stop 2023-10-06T03:51:30.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.867+0000 7fbc3a19c700 1 -- 172.21.15.55:0/2827585344 reap_dead start 2023-10-06T03:51:30.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.867+0000 7fbc3a99d700 1 -- 172.21.15.55:0/2827585344 shutdown_connections 2023-10-06T03:51:30.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.867+0000 7fbc3a99d700 1 -- 172.21.15.55:0/2827585344 >> 172.21.15.55:0/2827585344 conn(0x7fbc340cc1f0 msgr2=0x7fbc340cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:30.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.868+0000 7fbc3a99d700 1 -- 172.21.15.55:0/2827585344 shutdown_connections 2023-10-06T03:51:30.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.868+0000 7fbc3a99d700 1 -- 172.21.15.55:0/2827585344 wait complete. 2023-10-06T03:51:30.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.869+0000 7fbc3a99d700 1 Processor -- start 2023-10-06T03:51:30.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.869+0000 7fbc3a99d700 1 -- start start 2023-10-06T03:51:30.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.870+0000 7fbc3a99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc20004810 0x7fbc340af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:30.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.870+0000 7fbc3a99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc3414aa50 0x7fbc3416cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:30.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.870+0000 7fbc3a99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc3416d420 0x7fbc3416d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:30.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.870+0000 7fbc3a99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc340ae840 con 0x7fbc3414aa50 2023-10-06T03:51:30.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.870+0000 7fbc3a99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3416dd90 con 0x7fbc3416d420 2023-10-06T03:51:30.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.870+0000 7fbc3a99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc340ade40 con 0x7fbc20004810 2023-10-06T03:51:30.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.870+0000 7fbc3919a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc3414aa50 0x7fbc3416cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:30.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.870+0000 7fbc3919a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc3414aa50 0x7fbc3416cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37134/0 (socket says 172.21.15.55:37134) 2023-10-06T03:51:30.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.870+0000 7fbc3919a700 1 -- 172.21.15.55:0/4279663077 learned_addr learned my addr 172.21.15.55:0/4279663077 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:30.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.870+0000 7fbc3a19c700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc3416d420 0x7fbc3416d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:30.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.870+0000 7fbc3999b700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc20004810 0x7fbc340af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:30.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.871+0000 7fbc3919a700 1 -- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc20004810 msgr2=0x7fbc340af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:30.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.871+0000 7fbc3919a700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc20004810 0x7fbc340af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.871+0000 7fbc3919a700 1 -- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc3416d420 msgr2=0x7fbc3416d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:30.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.871+0000 7fbc3919a700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc3416d420 0x7fbc3416d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:30.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.871+0000 7fbc3919a700 1 -- 172.21.15.55:0/4279663077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc340ad230 con 0x7fbc3414aa50 2023-10-06T03:51:30.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.871+0000 7fbc3999b700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc20004810 0x7fbc340af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:30.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.871+0000 7fbc3919a700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc3414aa50 0x7fbc3416cee0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fbc24001ed0 tx=0x7fbc24015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:30.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.871+0000 7fbc29ffb700 1 -- 172.21.15.55:0/4279663077 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2400cc90 con 0x7fbc3414aa50 2023-10-06T03:51:30.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.871+0000 7fbc29ffb700 1 -- 172.21.15.55:0/4279663077 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2400c9e0 con 0x7fbc3414aa50 2023-10-06T03:51:30.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.871+0000 7fbc29ffb700 1 -- 172.21.15.55:0/4279663077 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24007530 con 0x7fbc3414aa50 2023-10-06T03:51:30.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.871+0000 7fbc3a99d700 1 -- 172.21.15.55:0/4279663077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc340ad510 con 0x7fbc3414aa50 2023-10-06T03:51:30.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.871+0000 7fbc3a99d700 1 -- 172.21.15.55:0/4279663077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc340adad0 con 0x7fbc3414aa50 2023-10-06T03:51:30.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.872+0000 7fbc3a99d700 1 -- 172.21.15.55:0/4279663077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc340adff0 con 0x7fbc3414aa50 2023-10-06T03:51:30.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.873+0000 7fbc29ffb700 1 -- 172.21.15.55:0/4279663077 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc24019070 con 0x7fbc3414aa50 2023-10-06T03:51:30.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.874+0000 7fbc29ffb700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbc18066f60 0x7fbc18069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:30.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.874+0000 7fbc29ffb700 1 -- 172.21.15.55:0/4279663077 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fbc24085cc0 con 0x7fbc3414aa50 2023-10-06T03:51:30.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.874+0000 7fbc3999b700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbc18066f60 0x7fbc18069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:30.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.875+0000 7fbc3999b700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbc18066f60 0x7fbc18069410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbc30001370 tx=0x7fbc30009040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:30.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:30.879+0000 7fbc29ffb700 1 -- 172.21.15.55:0/4279663077 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc340adff0 con 0x7fbc3414aa50 2023-10-06T03:51:30.959 INFO:tasks.ceph.mds.c.smithi055.stderr:2023-10-06T03:51:30.958+0000 7fbe1e7f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:51:31.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.084+0000 7fbc3a99d700 1 -- 172.21.15.55:0/4279663077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc340adfb0 con 0x7fbc3414aa50 2023-10-06T03:51:31.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.085+0000 7fbc29ffb700 1 -- 172.21.15.55:0/4279663077 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17523 (secure 0 0 0) 0x7fbc2408c080 con 0x7fbc3414aa50 2023-10-06T03:51:31.109 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:31.109 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":123,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:51:27.985532+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:51:24.466543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T03:51:25.462282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:51:31.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.088+0000 7fbc3a99d700 1 -- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbc18066f60 msgr2=0x7fbc18069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:31.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.088+0000 7fbc3a99d700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbc18066f60 0x7fbc18069410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbc30001370 tx=0x7fbc30009040 comp rx=0 tx=0).stop 2023-10-06T03:51:31.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.088+0000 7fbc3a99d700 1 -- 172.21.15.55:0/4279663077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc3414aa50 msgr2=0x7fbc3416cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:31.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.088+0000 7fbc3a99d700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc3414aa50 0x7fbc3416cee0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fbc24001ed0 tx=0x7fbc24015000 comp rx=0 tx=0).stop 2023-10-06T03:51:31.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.088+0000 7fbc3a99d700 1 -- 172.21.15.55:0/4279663077 shutdown_connections 2023-10-06T03:51:31.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.088+0000 7fbc3a99d700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbc18066f60 0x7fbc18069410 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.088+0000 7fbc3a99d700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc3416d420 0x7fbc3416d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.088+0000 7fbc3a99d700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc20004810 0x7fbc340af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.088+0000 7fbc3a99d700 1 --2- 172.21.15.55:0/4279663077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc3414aa50 0x7fbc3416cee0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.088+0000 7fbc3a99d700 1 -- 172.21.15.55:0/4279663077 >> 172.21.15.55:0/4279663077 conn(0x7fbc340cc1f0 msgr2=0x7fbc34157770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:31.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.088+0000 7fbc3a99d700 1 -- 172.21.15.55:0/4279663077 shutdown_connections 2023-10-06T03:51:31.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.088+0000 7fbc3a99d700 1 -- 172.21.15.55:0/4279663077 wait complete. 2023-10-06T03:51:31.112 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:51:31.112 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:51:31.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.297+0000 7f5537fff700 1 Processor -- start 2023-10-06T03:51:31.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.297+0000 7f5537fff700 1 -- start start 2023-10-06T03:51:31.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.298+0000 7f5537fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5538159420 0x7f55380b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.298+0000 7f5537fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f55381597f0 0x7f55380b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.298+0000 7f5537fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f553815d230 0x7f553815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.298+0000 7f5537fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f553815dbd0 con 0x7f553815d230 2023-10-06T03:51:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.298+0000 7f5537fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55380b11d0 con 0x7f55381597f0 2023-10-06T03:51:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.298+0000 7f5537fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55380b14b0 con 0x7f5538159420 2023-10-06T03:51:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.299+0000 7f55377fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f553815d230 0x7f553815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:31.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.299+0000 7f55377fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f553815d230 0x7f553815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37144/0 (socket says 172.21.15.55:37144) 2023-10-06T03:51:31.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.299+0000 7f55377fe700 1 -- 172.21.15.55:0/1297867057 learned_addr learned my addr 172.21.15.55:0/1297867057 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:31.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.299+0000 7f55367fc700 1 --2- 172.21.15.55:0/1297867057 >> v2:172.21.15.163:3300/0 conn(0x7f55381597f0 0x7f55380b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:31.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.299+0000 7f5536ffd700 1 --2- 172.21.15.55:0/1297867057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5538159420 0x7f55380b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:31.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.299+0000 7f55377fe700 1 -- 172.21.15.55:0/1297867057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5538159420 msgr2=0x7f55380b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:31.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.299+0000 7f55377fe700 1 --2- 172.21.15.55:0/1297867057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5538159420 0x7f55380b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.299+0000 7f55377fe700 1 -- 172.21.15.55:0/1297867057 >> v2:172.21.15.163:3300/0 conn(0x7f55381597f0 msgr2=0x7f55380b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:31.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.299+0000 7f55377fe700 1 --2- 172.21.15.55:0/1297867057 >> v2:172.21.15.163:3300/0 conn(0x7f55381597f0 0x7f55380b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.299+0000 7f55377fe700 1 -- 172.21.15.55:0/1297867057 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55380b1790 con 0x7f553815d230 2023-10-06T03:51:31.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.299+0000 7f55377fe700 1 --2- 172.21.15.55:0/1297867057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553815d230 0x7f553815d690 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f55380d1770 tx=0x7f5530009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb374cf56aaaca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:31.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.300+0000 7f5535ffb700 1 -- 172.21.15.55:0/1297867057 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553000e070 con 0x7f553815d230 2023-10-06T03:51:31.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.300+0000 7f5535ffb700 1 -- 172.21.15.55:0/1297867057 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553000ba50 con 0x7f553815d230 2023-10-06T03:51:31.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.300+0000 7f5535ffb700 1 -- 172.21.15.55:0/1297867057 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55300092e0 con 0x7f553815d230 2023-10-06T03:51:31.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.300+0000 7f5537fff700 1 -- 172.21.15.55:0/1297867057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553815d230 msgr2=0x7f553815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:31.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.300+0000 7f5537fff700 1 --2- 172.21.15.55:0/1297867057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553815d230 0x7f553815d690 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f55380d1770 tx=0x7f5530009840 comp rx=0 tx=0).stop 2023-10-06T03:51:31.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.300+0000 7f5537fff700 1 -- 172.21.15.55:0/1297867057 shutdown_connections 2023-10-06T03:51:31.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.300+0000 7f5537fff700 1 --2- 172.21.15.55:0/1297867057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553815d230 0x7f553815d690 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.300+0000 7f5537fff700 1 --2- 172.21.15.55:0/1297867057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5538159420 0x7f55380b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.300+0000 7f5537fff700 1 --2- 172.21.15.55:0/1297867057 >> v2:172.21.15.163:3300/0 conn(0x7f55381597f0 0x7f55380b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.300+0000 7f5537fff700 1 -- 172.21.15.55:0/1297867057 >> 172.21.15.55:0/1297867057 conn(0x7f55380d0660 msgr2=0x7f55380d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:31.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.300+0000 7f5537fff700 1 -- 172.21.15.55:0/1297867057 shutdown_connections 2023-10-06T03:51:31.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.300+0000 7f5537fff700 1 -- 172.21.15.55:0/1297867057 wait complete. 2023-10-06T03:51:31.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.302+0000 7f5537fff700 1 Processor -- start 2023-10-06T03:51:31.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.302+0000 7f5537fff700 1 -- start start 2023-10-06T03:51:31.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.303+0000 7f5537fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5538159420 0x7f5538155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:31.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.303+0000 7f5537fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55381597f0 0x7f55380c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:31.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.303+0000 7f5537fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553815d230 0x7f55380b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:31.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.303+0000 7f5537fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5538155e90 con 0x7f553815d230 2023-10-06T03:51:31.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.303+0000 7f5537fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380ca640 con 0x7f55381597f0 2023-10-06T03:51:31.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.303+0000 7f5537fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55380c6f40 con 0x7f5538159420 2023-10-06T03:51:31.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.303+0000 7f55377fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553815d230 0x7f55380b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:31.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.303+0000 7f55377fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553815d230 0x7f55380b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37152/0 (socket says 172.21.15.55:37152) 2023-10-06T03:51:31.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.303+0000 7f55377fe700 1 -- 172.21.15.55:0/2083825059 learned_addr learned my addr 172.21.15.55:0/2083825059 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:31.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.303+0000 7f55367fc700 1 --2- 172.21.15.55:0/2083825059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55381597f0 0x7f55380c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:31.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.303+0000 7f5536ffd700 1 --2- 172.21.15.55:0/2083825059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5538159420 0x7f5538155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:31.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.304+0000 7f55377fe700 1 -- 172.21.15.55:0/2083825059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5538159420 msgr2=0x7f5538155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:31.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.304+0000 7f55377fe700 1 --2- 172.21.15.55:0/2083825059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5538159420 0x7f5538155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.304+0000 7f55377fe700 1 -- 172.21.15.55:0/2083825059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55381597f0 msgr2=0x7f55380c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:31.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.304+0000 7f55377fe700 1 --2- 172.21.15.55:0/2083825059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55381597f0 0x7f55380c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.304+0000 7f55377fe700 1 -- 172.21.15.55:0/2083825059 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5530009ae0 con 0x7f553815d230 2023-10-06T03:51:31.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.304+0000 7f55377fe700 1 --2- 172.21.15.55:0/2083825059 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553815d230 0x7f55380b2170 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5530009fd0 tx=0x7f5530018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:31.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.304+0000 7f551effd700 1 -- 172.21.15.55:0/2083825059 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55300032f0 con 0x7f553815d230 2023-10-06T03:51:31.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.304+0000 7f551effd700 1 -- 172.21.15.55:0/2083825059 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5530008e70 con 0x7f553815d230 2023-10-06T03:51:31.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.304+0000 7f551effd700 1 -- 172.21.15.55:0/2083825059 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553001c640 con 0x7f553815d230 2023-10-06T03:51:31.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.304+0000 7f5537fff700 1 -- 172.21.15.55:0/2083825059 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55380c7220 con 0x7f553815d230 2023-10-06T03:51:31.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.304+0000 7f5537fff700 1 -- 172.21.15.55:0/2083825059 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55380c77e0 con 0x7f553815d230 2023-10-06T03:51:31.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.305+0000 7f5537fff700 1 -- 172.21.15.55:0/2083825059 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55380d01f0 con 0x7f553815d230 2023-10-06T03:51:31.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.306+0000 7f551effd700 1 -- 172.21.15.55:0/2083825059 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5530003040 con 0x7f553815d230 2023-10-06T03:51:31.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.307+0000 7f551effd700 1 --2- 172.21.15.55:0/2083825059 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5524066f30 0x7f55240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:31.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.307+0000 7f551effd700 1 -- 172.21.15.55:0/2083825059 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f55300861a0 con 0x7f553815d230 2023-10-06T03:51:31.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.308+0000 7f5536ffd700 1 --2- 172.21.15.55:0/2083825059 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5524066f30 0x7f55240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:31.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.308+0000 7f5536ffd700 1 --2- 172.21.15.55:0/2083825059 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5524066f30 0x7f55240693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5528006fd0 tx=0x7f5528008040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:31.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.311+0000 7f551effd700 1 -- 172.21.15.55:0/2083825059 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5530053080 con 0x7f553815d230 2023-10-06T03:51:31.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.513+0000 7f5537fff700 1 -- 172.21.15.55:0/2083825059 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55380ca7d0 con 0x7f5524066f30 2023-10-06T03:51:31.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.525+0000 7f551effd700 1 -- 172.21.15.55:0/2083825059 <== mgr.7228 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311773 (secure 0 0 0) 0x7f55380ca7d0 con 0x7f5524066f30 2023-10-06T03:51:31.537 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:31.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.529+0000 7f5537fff700 1 -- 172.21.15.55:0/2083825059 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5524066f30 msgr2=0x7f55240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:31.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.529+0000 7f5537fff700 1 --2- 172.21.15.55:0/2083825059 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5524066f30 0x7f55240693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5528006fd0 tx=0x7f5528008040 comp rx=0 tx=0).stop 2023-10-06T03:51:31.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.529+0000 7f5537fff700 1 -- 172.21.15.55:0/2083825059 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553815d230 msgr2=0x7f55380b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:31.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.529+0000 7f5537fff700 1 --2- 172.21.15.55:0/2083825059 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553815d230 0x7f55380b2170 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f5530009fd0 tx=0x7f5530018000 comp rx=0 tx=0).stop 2023-10-06T03:51:31.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.529+0000 7f5537fff700 1 -- 172.21.15.55:0/2083825059 shutdown_connections 2023-10-06T03:51:31.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.529+0000 7f5537fff700 1 --2- 172.21.15.55:0/2083825059 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5524066f30 0x7f55240693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.529+0000 7f5537fff700 1 --2- 172.21.15.55:0/2083825059 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553815d230 0x7f55380b2170 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.529+0000 7f5537fff700 1 --2- 172.21.15.55:0/2083825059 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5538159420 0x7f5538155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.529+0000 7f5537fff700 1 --2- 172.21.15.55:0/2083825059 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55381597f0 0x7f55380c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.529+0000 7f5537fff700 1 -- 172.21.15.55:0/2083825059 >> 172.21.15.55:0/2083825059 conn(0x7f55380d0660 msgr2=0x7f5538157a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:31.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.529+0000 7f5537fff700 1 -- 172.21.15.55:0/2083825059 shutdown_connections 2023-10-06T03:51:31.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.529+0000 7f5537fff700 1 -- 172.21.15.55:0/2083825059 wait complete. 2023-10-06T03:51:31.541 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:51:31.548 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T03:51:31.019177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":403,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":265,"ondisk_log_size":265,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2460956,"kb_used_data":13020,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752513764,"statfs":{"total":773094113280,"available":770574094336,"internally_reserved":0,"allocated":13332480,"data_stored":7401087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001735"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013294+0000","last_change":"2023-10-06T03:51:27.006190+0000","last_active":"2023-10-06T03:51:27.013294+0000","last_peered":"2023-10-06T03:51:27.013294+0000","last_clean":"2023-10-06T03:51:27.013294+0000","last_became_active":"2023-10-06T03:51:27.005921+0000","last_became_peered":"2023-10-06T03:51:27.005921+0000","last_unstale":"2023-10-06T03:51:27.013294+0000","last_undegraded":"2023-10-06T03:51:27.013294+0000","last_fullsized":"2023-10-06T03:51:27.013294+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:02.608098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.003571+0000","last_change":"2023-10-06T03:51:26.996209+0000","last_active":"2023-10-06T03:51:27.003571+0000","last_peered":"2023-10-06T03:51:27.003571+0000","last_clean":"2023-10-06T03:51:27.003571+0000","last_became_active":"2023-10-06T03:51:26.995937+0000","last_became_peered":"2023-10-06T03:51:26.995937+0000","last_unstale":"2023-10-06T03:51:27.003571+0000","last_undegraded":"2023-10-06T03:51:27.003571+0000","last_fullsized":"2023-10-06T03:51:27.003571+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:48.741807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013536+0000","last_change":"2023-10-06T03:51:27.006464+0000","last_active":"2023-10-06T03:51:27.013536+0000","last_peered":"2023-10-06T03:51:27.013536+0000","last_clean":"2023-10-06T03:51:27.013536+0000","last_became_active":"2023-10-06T03:51:27.006169+0000","last_became_peered":"2023-10-06T03:51:27.006169+0000","last_unstale":"2023-10-06T03:51:27.013536+0000","last_undegraded":"2023-10-06T03:51:27.013536+0000","last_fullsized":"2023-10-06T03:51:27.013536+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:00.627533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.020768+0000","last_change":"2023-10-06T03:51:27.016234+0000","last_active":"2023-10-06T03:51:27.020768+0000","last_peered":"2023-10-06T03:51:27.020768+0000","last_clean":"2023-10-06T03:51:27.020768+0000","last_became_active":"2023-10-06T03:51:27.015956+0000","last_became_peered":"2023-10-06T03:51:27.015956+0000","last_unstale":"2023-10-06T03:51:27.020768+0000","last_undegraded":"2023-10-06T03:51:27.020768+0000","last_fullsized":"2023-10-06T03:51:27.020768+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:15:26.862706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.018120+0000","last_change":"2023-10-06T03:51:27.018120+0000","last_active":"2023-10-06T03:51:27.018120+0000","last_peered":"2023-10-06T03:51:27.018120+0000","last_clean":"2023-10-06T03:51:27.018120+0000","last_became_active":"2023-10-06T03:51:27.017690+0000","last_became_peered":"2023-10-06T03:51:27.017690+0000","last_unstale":"2023-10-06T03:51:27.018120+0000","last_undegraded":"2023-10-06T03:51:27.018120+0000","last_fullsized":"2023-10-06T03:51:27.018120+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:12.907241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013715+0000","last_change":"2023-10-06T03:51:27.011182+0000","last_active":"2023-10-06T03:51:27.013715+0000","last_peered":"2023-10-06T03:51:27.013715+0000","last_clean":"2023-10-06T03:51:27.013715+0000","last_became_active":"2023-10-06T03:51:27.010891+0000","last_became_peered":"2023-10-06T03:51:27.010891+0000","last_unstale":"2023-10-06T03:51:27.013715+0000","last_undegraded":"2023-10-06T03:51:27.013715+0000","last_fullsized":"2023-10-06T03:51:27.013715+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:27.792100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013325+0000","last_change":"2023-10-06T03:51:27.010154+0000","last_active":"2023-10-06T03:51:27.013325+0000","last_peered":"2023-10-06T03:51:27.013325+0000","last_clean":"2023-10-06T03:51:27.013325+0000","last_became_active":"2023-10-06T03:51:27.009783+0000","last_became_peered":"2023-10-06T03:51:27.009783+0000","last_unstale":"2023-10-06T03:51:27.013325+0000","last_undegraded":"2023-10-06T03:51:27.013325+0000","last_fullsized":"2023-10-06T03:51:27.013325+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:34.306331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.020128+0000","last_change":"2023-10-06T03:51:27.020128+0000","last_active":"2023-10-06T03:51:27.020128+0000","last_peered":"2023-10-06T03:51:27.020128+0000","last_clean":"2023-10-06T03:51:27.020128+0000","last_became_active":"2023-10-06T03:51:27.019749+0000","last_became_peered":"2023-10-06T03:51:27.019749+0000","last_unstale":"2023-10-06T03:51:27.020128+0000","last_undegraded":"2023-10-06T03:51:27.020128+0000","last_fullsized":"2023-10-06T03:51:27.020128+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:56.744594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013291+0000","last_change":"2023-10-06T03:51:27.001564+0000","last_active":"2023-10-06T03:51:27.013291+0000","last_peered":"2023-10-06T03:51:27.013291+0000","last_clean":"2023-10-06T03:51:27.013291+0000","last_became_active":"2023-10-06T03:51:27.001307+0000","last_became_peered":"2023-10-06T03:51:27.001307+0000","last_unstale":"2023-10-06T03:51:27.013291+0000","last_undegraded":"2023-10-06T03:51:27.013291+0000","last_fullsized":"2023-10-06T03:51:27.013291+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:32.773546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.007075+0000","last_change":"2023-10-06T03:51:27.007075+0000","last_active":"2023-10-06T03:51:27.007075+0000","last_peered":"2023-10-06T03:51:27.007075+0000","last_clean":"2023-10-06T03:51:27.007075+0000","last_became_active":"2023-10-06T03:51:27.006753+0000","last_became_peered":"2023-10-06T03:51:27.006753+0000","last_unstale":"2023-10-06T03:51:27.007075+0000","last_undegraded":"2023-10-06T03:51:27.007075+0000","last_fullsized":"2023-10-06T03:51:27.007075+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:26.398839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.022844+0000","last_change":"2023-10-06T03:51:27.017654+0000","last_active":"2023-10-06T03:51:27.022844+0000","last_peered":"2023-10-06T03:51:27.022844+0000","last_clean":"2023-10-06T03:51:27.022844+0000","last_became_active":"2023-10-06T03:51:27.017368+0000","last_became_peered":"2023-10-06T03:51:27.017368+0000","last_unstale":"2023-10-06T03:51:27.022844+0000","last_undegraded":"2023-10-06T03:51:27.022844+0000","last_fullsized":"2023-10-06T03:51:27.022844+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:34.306331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.012351+0000","last_change":"2023-10-06T03:51:27.006545+0000","last_active":"2023-10-06T03:51:27.012351+0000","last_peered":"2023-10-06T03:51:27.012351+0000","last_clean":"2023-10-06T03:51:27.012351+0000","last_became_active":"2023-10-06T03:51:27.006286+0000","last_became_peered":"2023-10-06T03:51:27.006286+0000","last_unstale":"2023-10-06T03:51:27.012351+0000","last_undegraded":"2023-10-06T03:51:27.012351+0000","last_fullsized":"2023-10-06T03:51:27.012351+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:41.424643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.012817+0000","last_change":"2023-10-06T03:51:27.010450+0000","last_active":"2023-10-06T03:51:27.012817+0000","last_peered":"2023-10-06T03:51:27.012817+0000","last_clean":"2023-10-06T03:51:27.012817+0000","last_became_active":"2023-10-06T03:51:27.010197+0000","last_became_peered":"2023-10-06T03:51:27.010197+0000","last_unstale":"2023-10-06T03:51:27.012817+0000","last_undegraded":"2023-10-06T03:51:27.012817+0000","last_fullsized":"2023-10-06T03:51:27.012817+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:03.629521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.006748+0000","last_change":"2023-10-06T03:51:27.006748+0000","last_active":"2023-10-06T03:51:27.006748+0000","last_peered":"2023-10-06T03:51:27.006748+0000","last_clean":"2023-10-06T03:51:27.006748+0000","last_became_active":"2023-10-06T03:51:27.006490+0000","last_became_peered":"2023-10-06T03:51:27.006490+0000","last_unstale":"2023-10-06T03:51:27.006748+0000","last_undegraded":"2023-10-06T03:51:27.006748+0000","last_fullsized":"2023-10-06T03:51:27.006748+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:15:26.862706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.007133+0000","last_change":"2023-10-06T03:51:27.007133+0000","last_active":"2023-10-06T03:51:27.007133+0000","last_peered":"2023-10-06T03:51:27.007133+0000","last_clean":"2023-10-06T03:51:27.007133+0000","last_became_active":"2023-10-06T03:51:27.006798+0000","last_became_peered":"2023-10-06T03:51:27.006798+0000","last_unstale":"2023-10-06T03:51:27.007133+0000","last_undegraded":"2023-10-06T03:51:27.007133+0000","last_fullsized":"2023-10-06T03:51:27.007133+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:42.614149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.005273+0000","last_change":"2023-10-06T03:51:27.005273+0000","last_active":"2023-10-06T03:51:27.005273+0000","last_peered":"2023-10-06T03:51:27.005273+0000","last_clean":"2023-10-06T03:51:27.005273+0000","last_became_active":"2023-10-06T03:51:27.004992+0000","last_became_peered":"2023-10-06T03:51:27.004992+0000","last_unstale":"2023-10-06T03:51:27.005273+0000","last_undegraded":"2023-10-06T03:51:27.005273+0000","last_fullsized":"2023-10-06T03:51:27.005273+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:57.631308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013663+0000","last_change":"2023-10-06T03:51:27.006819+0000","last_active":"2023-10-06T03:51:27.013663+0000","last_peered":"2023-10-06T03:51:27.013663+0000","last_clean":"2023-10-06T03:51:27.013663+0000","last_became_active":"2023-10-06T03:51:27.006540+0000","last_became_peered":"2023-10-06T03:51:27.006540+0000","last_unstale":"2023-10-06T03:51:27.013663+0000","last_undegraded":"2023-10-06T03:51:27.013663+0000","last_fullsized":"2023-10-06T03:51:27.013663+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:32.787881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013932+0000","last_change":"2023-10-06T03:51:27.011726+0000","last_active":"2023-10-06T03:51:27.013932+0000","last_peered":"2023-10-06T03:51:27.013932+0000","last_clean":"2023-10-06T03:51:27.013932+0000","last_became_active":"2023-10-06T03:51:27.011279+0000","last_became_peered":"2023-10-06T03:51:27.011279+0000","last_unstale":"2023-10-06T03:51:27.013932+0000","last_undegraded":"2023-10-06T03:51:27.013932+0000","last_fullsized":"2023-10-06T03:51:27.013932+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:57.199095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.011302+0000","last_change":"2023-10-06T03:51:27.000180+0000","last_active":"2023-10-06T03:51:27.011302+0000","last_peered":"2023-10-06T03:51:27.011302+0000","last_clean":"2023-10-06T03:51:27.011302+0000","last_became_active":"2023-10-06T03:51:26.999804+0000","last_became_peered":"2023-10-06T03:51:26.999804+0000","last_unstale":"2023-10-06T03:51:27.011302+0000","last_undegraded":"2023-10-06T03:51:27.011302+0000","last_fullsized":"2023-10-06T03:51:27.011302+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:59.533661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.006117+0000","last_change":"2023-10-06T03:51:26.998584+0000","last_active":"2023-10-06T03:51:27.006117+0000","last_peered":"2023-10-06T03:51:27.006117+0000","last_clean":"2023-10-06T03:51:27.006117+0000","last_became_active":"2023-10-06T03:51:26.998320+0000","last_became_peered":"2023-10-06T03:51:26.998320+0000","last_unstale":"2023-10-06T03:51:27.006117+0000","last_undegraded":"2023-10-06T03:51:27.006117+0000","last_fullsized":"2023-10-06T03:51:27.006117+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:27.956599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.005270+0000","last_change":"2023-10-06T03:51:27.005270+0000","last_active":"2023-10-06T03:51:27.005270+0000","last_peered":"2023-10-06T03:51:27.005270+0000","last_clean":"2023-10-06T03:51:27.005270+0000","last_became_active":"2023-10-06T03:51:27.005021+0000","last_became_peered":"2023-10-06T03:51:27.005021+0000","last_unstale":"2023-10-06T03:51:27.005270+0000","last_undegraded":"2023-10-06T03:51:27.005270+0000","last_fullsized":"2023-10-06T03:51:27.005270+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:30.255876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.014371+0000","last_change":"2023-10-06T03:51:27.014371+0000","last_active":"2023-10-06T03:51:27.014371+0000","last_peered":"2023-10-06T03:51:27.014371+0000","last_clean":"2023-10-06T03:51:27.014371+0000","last_became_active":"2023-10-06T03:51:27.014112+0000","last_became_peered":"2023-10-06T03:51:27.014112+0000","last_unstale":"2023-10-06T03:51:27.014371+0000","last_undegraded":"2023-10-06T03:51:27.014371+0000","last_fullsized":"2023-10-06T03:51:27.014371+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:44.298593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013264+0000","last_change":"2023-10-06T03:51:27.005697+0000","last_active":"2023-10-06T03:51:27.013264+0000","last_peered":"2023-10-06T03:51:27.013264+0000","last_clean":"2023-10-06T03:51:27.013264+0000","last_became_active":"2023-10-06T03:51:27.005405+0000","last_became_peered":"2023-10-06T03:51:27.005405+0000","last_unstale":"2023-10-06T03:51:27.013264+0000","last_undegraded":"2023-10-06T03:51:27.013264+0000","last_fullsized":"2023-10-06T03:51:27.013264+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:15.743634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.018302+0000","last_change":"2023-10-06T03:51:27.018302+0000","last_active":"2023-10-06T03:51:27.018302+0000","last_peered":"2023-10-06T03:51:27.018302+0000","last_clean":"2023-10-06T03:51:27.018302+0000","last_became_active":"2023-10-06T03:51:27.018042+0000","last_became_peered":"2023-10-06T03:51:27.018042+0000","last_unstale":"2023-10-06T03:51:27.018302+0000","last_undegraded":"2023-10-06T03:51:27.018302+0000","last_fullsized":"2023-10-06T03:51:27.018302+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:46.761766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.012409+0000","last_change":"2023-10-06T03:51:27.008627+0000","last_active":"2023-10-06T03:51:27.012409+0000","last_peered":"2023-10-06T03:51:27.012409+0000","last_clean":"2023-10-06T03:51:27.012409+0000","last_became_active":"2023-10-06T03:51:27.007558+0000","last_became_peered":"2023-10-06T03:51:27.007558+0000","last_unstale":"2023-10-06T03:51:27.012409+0000","last_undegraded":"2023-10-06T03:51:27.012409+0000","last_fullsized":"2023-10-06T03:51:27.012409+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:22.331588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.015368+0000","last_change":"2023-10-06T03:51:27.012298+0000","last_active":"2023-10-06T03:51:27.015368+0000","last_peered":"2023-10-06T03:51:27.015368+0000","last_clean":"2023-10-06T03:51:27.015368+0000","last_became_active":"2023-10-06T03:51:27.012033+0000","last_became_peered":"2023-10-06T03:51:27.012033+0000","last_unstale":"2023-10-06T03:51:27.015368+0000","last_undegraded":"2023-10-06T03:51:27.015368+0000","last_fullsized":"2023-10-06T03:51:27.015368+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:27.590794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.011171+0000","last_change":"2023-10-06T03:51:27.004430+0000","last_active":"2023-10-06T03:51:27.011171+0000","last_peered":"2023-10-06T03:51:27.011171+0000","last_clean":"2023-10-06T03:51:27.011171+0000","last_became_active":"2023-10-06T03:51:27.003746+0000","last_became_peered":"2023-10-06T03:51:27.003746+0000","last_unstale":"2023-10-06T03:51:27.011171+0000","last_undegraded":"2023-10-06T03:51:27.011171+0000","last_fullsized":"2023-10-06T03:51:27.011171+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:32.773546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.015080+0000","last_change":"2023-10-06T03:51:27.008006+0000","last_active":"2023-10-06T03:51:27.015080+0000","last_peered":"2023-10-06T03:51:27.015080+0000","last_clean":"2023-10-06T03:51:27.015080+0000","last_became_active":"2023-10-06T03:51:27.007558+0000","last_became_peered":"2023-10-06T03:51:27.007558+0000","last_unstale":"2023-10-06T03:51:27.015080+0000","last_undegraded":"2023-10-06T03:51:27.015080+0000","last_fullsized":"2023-10-06T03:51:27.015080+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:46.761766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.004263+0000","last_change":"2023-10-06T03:51:27.004263+0000","last_active":"2023-10-06T03:51:27.004263+0000","last_peered":"2023-10-06T03:51:27.004263+0000","last_clean":"2023-10-06T03:51:27.004263+0000","last_became_active":"2023-10-06T03:51:27.003843+0000","last_became_peered":"2023-10-06T03:51:27.003843+0000","last_unstale":"2023-10-06T03:51:27.004263+0000","last_undegraded":"2023-10-06T03:51:27.004263+0000","last_fullsized":"2023-10-06T03:51:27.004263+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:24.850221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.012317+0000","last_change":"2023-10-06T03:51:27.007144+0000","last_active":"2023-10-06T03:51:27.012317+0000","last_peered":"2023-10-06T03:51:27.012317+0000","last_clean":"2023-10-06T03:51:27.012317+0000","last_became_active":"2023-10-06T03:51:27.006892+0000","last_became_peered":"2023-10-06T03:51:27.006892+0000","last_unstale":"2023-10-06T03:51:27.012317+0000","last_undegraded":"2023-10-06T03:51:27.012317+0000","last_fullsized":"2023-10-06T03:51:27.012317+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:05.693451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.012645+0000","last_change":"2023-10-06T03:51:26.998372+0000","last_active":"2023-10-06T03:51:27.012645+0000","last_peered":"2023-10-06T03:51:27.012645+0000","last_clean":"2023-10-06T03:51:27.012645+0000","last_became_active":"2023-10-06T03:51:26.998106+0000","last_became_peered":"2023-10-06T03:51:26.998106+0000","last_unstale":"2023-10-06T03:51:27.012645+0000","last_undegraded":"2023-10-06T03:51:27.012645+0000","last_fullsized":"2023-10-06T03:51:27.012645+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:58.329261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.009277+0000","last_change":"2023-10-06T03:51:27.009277+0000","last_active":"2023-10-06T03:51:27.009277+0000","last_peered":"2023-10-06T03:51:27.009277+0000","last_clean":"2023-10-06T03:51:27.009277+0000","last_became_active":"2023-10-06T03:51:27.009013+0000","last_became_peered":"2023-10-06T03:51:27.009013+0000","last_unstale":"2023-10-06T03:51:27.009277+0000","last_undegraded":"2023-10-06T03:51:27.009277+0000","last_fullsized":"2023-10-06T03:51:27.009277+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:35:04.371568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.999127+0000","last_change":"2023-10-06T03:51:26.996011+0000","last_active":"2023-10-06T03:51:26.999127+0000","last_peered":"2023-10-06T03:51:26.999127+0000","last_clean":"2023-10-06T03:51:26.999127+0000","last_became_active":"2023-10-06T03:51:26.995494+0000","last_became_peered":"2023-10-06T03:51:26.995494+0000","last_unstale":"2023-10-06T03:51:26.999127+0000","last_undegraded":"2023-10-06T03:51:26.999127+0000","last_fullsized":"2023-10-06T03:51:26.999127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:12.935675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"122'4","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.022421+0000","last_change":"2023-10-06T03:51:26.008165+0000","last_active":"2023-10-06T03:51:27.022421+0000","last_peered":"2023-10-06T03:51:27.022421+0000","last_clean":"2023-10-06T03:51:27.022421+0000","last_became_active":"2023-10-06T03:51:26.007909+0000","last_became_peered":"2023-10-06T03:51:26.007909+0000","last_unstale":"2023-10-06T03:51:27.022421+0000","last_undegraded":"2023-10-06T03:51:27.022421+0000","last_fullsized":"2023-10-06T03:51:27.022421+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:13.022473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.024552+0000","last_change":"2023-10-06T03:51:27.024552+0000","last_active":"2023-10-06T03:51:27.024552+0000","last_peered":"2023-10-06T03:51:27.024552+0000","last_clean":"2023-10-06T03:51:27.024552+0000","last_became_active":"2023-10-06T03:51:27.024281+0000","last_became_peered":"2023-10-06T03:51:27.024281+0000","last_unstale":"2023-10-06T03:51:27.024552+0000","last_undegraded":"2023-10-06T03:51:27.024552+0000","last_fullsized":"2023-10-06T03:51:27.024552+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:03.655845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"122'1","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.002201+0000","last_change":"2023-10-06T03:51:26.009666+0000","last_active":"2023-10-06T03:51:27.002201+0000","last_peered":"2023-10-06T03:51:27.002201+0000","last_clean":"2023-10-06T03:51:27.002201+0000","last_became_active":"2023-10-06T03:51:26.009402+0000","last_became_peered":"2023-10-06T03:51:26.009402+0000","last_unstale":"2023-10-06T03:51:27.002201+0000","last_undegraded":"2023-10-06T03:51:27.002201+0000","last_fullsized":"2023-10-06T03:51:27.002201+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:49.724213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.023068+0000","last_change":"2023-10-06T03:51:27.017913+0000","last_active":"2023-10-06T03:51:27.023068+0000","last_peered":"2023-10-06T03:51:27.023068+0000","last_clean":"2023-10-06T03:51:27.023068+0000","last_became_active":"2023-10-06T03:51:27.017671+0000","last_became_peered":"2023-10-06T03:51:27.017671+0000","last_unstale":"2023-10-06T03:51:27.023068+0000","last_undegraded":"2023-10-06T03:51:27.023068+0000","last_fullsized":"2023-10-06T03:51:27.023068+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:47.161538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"122'2","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.022882+0000","last_change":"2023-10-06T03:51:26.005937+0000","last_active":"2023-10-06T03:51:27.022882+0000","last_peered":"2023-10-06T03:51:27.022882+0000","last_clean":"2023-10-06T03:51:27.022882+0000","last_became_active":"2023-10-06T03:51:26.005629+0000","last_became_peered":"2023-10-06T03:51:26.005629+0000","last_unstale":"2023-10-06T03:51:27.022882+0000","last_undegraded":"2023-10-06T03:51:27.022882+0000","last_fullsized":"2023-10-06T03:51:27.022882+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:53.125506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.017681+0000","last_change":"2023-10-06T03:51:27.012937+0000","last_active":"2023-10-06T03:51:27.017681+0000","last_peered":"2023-10-06T03:51:27.017681+0000","last_clean":"2023-10-06T03:51:27.017681+0000","last_became_active":"2023-10-06T03:51:27.012633+0000","last_became_peered":"2023-10-06T03:51:27.012633+0000","last_unstale":"2023-10-06T03:51:27.017681+0000","last_undegraded":"2023-10-06T03:51:27.017681+0000","last_fullsized":"2023-10-06T03:51:27.017681+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:57.199095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"122'2","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.023461+0000","last_change":"2023-10-06T03:51:26.002695+0000","last_active":"2023-10-06T03:51:27.023461+0000","last_peered":"2023-10-06T03:51:27.023461+0000","last_clean":"2023-10-06T03:51:27.023461+0000","last_became_active":"2023-10-06T03:51:26.002404+0000","last_became_peered":"2023-10-06T03:51:26.002404+0000","last_unstale":"2023-10-06T03:51:27.023461+0000","last_undegraded":"2023-10-06T03:51:27.023461+0000","last_fullsized":"2023-10-06T03:51:27.023461+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:45.196305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.022170+0000","last_change":"2023-10-06T03:51:27.020654+0000","last_active":"2023-10-06T03:51:27.022170+0000","last_peered":"2023-10-06T03:51:27.022170+0000","last_clean":"2023-10-06T03:51:27.022170+0000","last_became_active":"2023-10-06T03:51:27.020239+0000","last_became_peered":"2023-10-06T03:51:27.020239+0000","last_unstale":"2023-10-06T03:51:27.022170+0000","last_undegraded":"2023-10-06T03:51:27.022170+0000","last_fullsized":"2023-10-06T03:51:27.022170+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:57.631308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"122'1","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.993736+0000","last_change":"2023-10-06T03:51:26.010740+0000","last_active":"2023-10-06T03:51:26.993736+0000","last_peered":"2023-10-06T03:51:26.993736+0000","last_clean":"2023-10-06T03:51:26.993736+0000","last_became_active":"2023-10-06T03:51:26.010468+0000","last_became_peered":"2023-10-06T03:51:26.010468+0000","last_unstale":"2023-10-06T03:51:26.993736+0000","last_undegraded":"2023-10-06T03:51:26.993736+0000","last_fullsized":"2023-10-06T03:51:26.993736+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:11.896681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.011328+0000","last_change":"2023-10-06T03:51:27.000611+0000","last_active":"2023-10-06T03:51:27.011328+0000","last_peered":"2023-10-06T03:51:27.011328+0000","last_clean":"2023-10-06T03:51:27.011328+0000","last_became_active":"2023-10-06T03:51:27.000321+0000","last_became_peered":"2023-10-06T03:51:27.000321+0000","last_unstale":"2023-10-06T03:51:27.011328+0000","last_undegraded":"2023-10-06T03:51:27.011328+0000","last_fullsized":"2023-10-06T03:51:27.011328+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:11.602376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"122'1","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.020512+0000","last_change":"2023-10-06T03:51:26.002221+0000","last_active":"2023-10-06T03:51:27.020512+0000","last_peered":"2023-10-06T03:51:27.020512+0000","last_clean":"2023-10-06T03:51:27.020512+0000","last_became_active":"2023-10-06T03:51:26.000862+0000","last_became_peered":"2023-10-06T03:51:26.000862+0000","last_unstale":"2023-10-06T03:51:27.020512+0000","last_undegraded":"2023-10-06T03:51:27.020512+0000","last_fullsized":"2023-10-06T03:51:27.020512+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:26.361323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.020513+0000","last_change":"2023-10-06T03:51:27.013845+0000","last_active":"2023-10-06T03:51:27.020513+0000","last_peered":"2023-10-06T03:51:27.020513+0000","last_clean":"2023-10-06T03:51:27.020513+0000","last_became_active":"2023-10-06T03:51:27.013561+0000","last_became_peered":"2023-10-06T03:51:27.013561+0000","last_unstale":"2023-10-06T03:51:27.020513+0000","last_undegraded":"2023-10-06T03:51:27.020513+0000","last_fullsized":"2023-10-06T03:51:27.020513+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:14.033033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.993169+0000","last_change":"2023-10-06T03:51:26.001908+0000","last_active":"2023-10-06T03:51:26.993169+0000","last_peered":"2023-10-06T03:51:26.993169+0000","last_clean":"2023-10-06T03:51:26.993169+0000","last_became_active":"2023-10-06T03:51:26.001569+0000","last_became_peered":"2023-10-06T03:51:26.001569+0000","last_unstale":"2023-10-06T03:51:26.993169+0000","last_undegraded":"2023-10-06T03:51:26.993169+0000","last_fullsized":"2023-10-06T03:51:26.993169+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:39.038453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.012964+0000","last_change":"2023-10-06T03:51:27.009537+0000","last_active":"2023-10-06T03:51:27.012964+0000","last_peered":"2023-10-06T03:51:27.012964+0000","last_clean":"2023-10-06T03:51:27.012964+0000","last_became_active":"2023-10-06T03:51:27.009059+0000","last_became_peered":"2023-10-06T03:51:27.009059+0000","last_unstale":"2023-10-06T03:51:27.012964+0000","last_undegraded":"2023-10-06T03:51:27.012964+0000","last_fullsized":"2023-10-06T03:51:27.012964+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:44.010040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.991960+0000","last_change":"2023-10-06T03:51:26.005255+0000","last_active":"2023-10-06T03:51:26.991960+0000","last_peered":"2023-10-06T03:51:26.991960+0000","last_clean":"2023-10-06T03:51:26.991960+0000","last_became_active":"2023-10-06T03:51:26.004972+0000","last_became_peered":"2023-10-06T03:51:26.004972+0000","last_unstale":"2023-10-06T03:51:26.991960+0000","last_undegraded":"2023-10-06T03:51:26.991960+0000","last_fullsized":"2023-10-06T03:51:26.991960+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:56.188535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.999108+0000","last_change":"2023-10-06T03:51:26.995751+0000","last_active":"2023-10-06T03:51:26.999108+0000","last_peered":"2023-10-06T03:51:26.999108+0000","last_clean":"2023-10-06T03:51:26.999108+0000","last_became_active":"2023-10-06T03:51:26.995120+0000","last_became_peered":"2023-10-06T03:51:26.995120+0000","last_unstale":"2023-10-06T03:51:26.999108+0000","last_undegraded":"2023-10-06T03:51:26.999108+0000","last_fullsized":"2023-10-06T03:51:26.999108+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:30.207842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.992013+0000","last_change":"2023-10-06T03:51:26.005874+0000","last_active":"2023-10-06T03:51:26.992013+0000","last_peered":"2023-10-06T03:51:26.992013+0000","last_clean":"2023-10-06T03:51:26.992013+0000","last_became_active":"2023-10-06T03:51:26.005545+0000","last_became_peered":"2023-10-06T03:51:26.005545+0000","last_unstale":"2023-10-06T03:51:26.992013+0000","last_undegraded":"2023-10-06T03:51:26.992013+0000","last_fullsized":"2023-10-06T03:51:26.992013+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:27.433759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.017351+0000","last_change":"2023-10-06T03:51:27.017351+0000","last_active":"2023-10-06T03:51:27.017351+0000","last_peered":"2023-10-06T03:51:27.017351+0000","last_clean":"2023-10-06T03:51:27.017351+0000","last_became_active":"2023-10-06T03:51:27.017086+0000","last_became_peered":"2023-10-06T03:51:27.017086+0000","last_unstale":"2023-10-06T03:51:27.017351+0000","last_undegraded":"2023-10-06T03:51:27.017351+0000","last_fullsized":"2023-10-06T03:51:27.017351+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:48.578436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.988665+0000","last_change":"2023-10-06T03:51:26.005721+0000","last_active":"2023-10-06T03:51:26.988665+0000","last_peered":"2023-10-06T03:51:26.988665+0000","last_clean":"2023-10-06T03:51:26.988665+0000","last_became_active":"2023-10-06T03:51:26.005442+0000","last_became_peered":"2023-10-06T03:51:26.005442+0000","last_unstale":"2023-10-06T03:51:26.988665+0000","last_undegraded":"2023-10-06T03:51:26.988665+0000","last_fullsized":"2023-10-06T03:51:26.988665+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:38.387346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.021443+0000","last_change":"2023-10-06T03:51:27.021443+0000","last_active":"2023-10-06T03:51:27.021443+0000","last_peered":"2023-10-06T03:51:27.021443+0000","last_clean":"2023-10-06T03:51:27.021443+0000","last_became_active":"2023-10-06T03:51:27.021215+0000","last_became_peered":"2023-10-06T03:51:27.021215+0000","last_unstale":"2023-10-06T03:51:27.021443+0000","last_undegraded":"2023-10-06T03:51:27.021443+0000","last_fullsized":"2023-10-06T03:51:27.021443+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:04.260304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"122'3","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.015095+0000","last_change":"2023-10-06T03:51:26.009548+0000","last_active":"2023-10-06T03:51:27.015095+0000","last_peered":"2023-10-06T03:51:27.015095+0000","last_clean":"2023-10-06T03:51:27.015095+0000","last_became_active":"2023-10-06T03:51:26.009288+0000","last_became_peered":"2023-10-06T03:51:26.009288+0000","last_unstale":"2023-10-06T03:51:27.015095+0000","last_undegraded":"2023-10-06T03:51:27.015095+0000","last_fullsized":"2023-10-06T03:51:27.015095+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:47.731247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.010763+0000","last_change":"2023-10-06T03:51:27.010763+0000","last_active":"2023-10-06T03:51:27.010763+0000","last_peered":"2023-10-06T03:51:27.010763+0000","last_clean":"2023-10-06T03:51:27.010763+0000","last_became_active":"2023-10-06T03:51:27.007859+0000","last_became_peered":"2023-10-06T03:51:27.007859+0000","last_unstale":"2023-10-06T03:51:27.010763+0000","last_undegraded":"2023-10-06T03:51:27.010763+0000","last_fullsized":"2023-10-06T03:51:27.010763+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:04:04.078671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"122'6","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.026061+0000","last_change":"2023-10-06T03:51:25.993345+0000","last_active":"2023-10-06T03:51:27.026061+0000","last_peered":"2023-10-06T03:51:27.026061+0000","last_clean":"2023-10-06T03:51:27.026061+0000","last_became_active":"2023-10-06T03:51:25.993067+0000","last_became_peered":"2023-10-06T03:51:25.993067+0000","last_unstale":"2023-10-06T03:51:27.026061+0000","last_undegraded":"2023-10-06T03:51:27.026061+0000","last_fullsized":"2023-10-06T03:51:27.026061+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:53.901794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.018395+0000","last_change":"2023-10-06T03:51:27.013055+0000","last_active":"2023-10-06T03:51:27.018395+0000","last_peered":"2023-10-06T03:51:27.018395+0000","last_clean":"2023-10-06T03:51:27.018395+0000","last_became_active":"2023-10-06T03:51:27.012794+0000","last_became_peered":"2023-10-06T03:51:27.012794+0000","last_unstale":"2023-10-06T03:51:27.018395+0000","last_undegraded":"2023-10-06T03:51:27.018395+0000","last_fullsized":"2023-10-06T03:51:27.018395+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:46.206866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"122'2","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.026060+0000","last_change":"2023-10-06T03:51:26.012192+0000","last_active":"2023-10-06T03:51:27.026060+0000","last_peered":"2023-10-06T03:51:27.026060+0000","last_clean":"2023-10-06T03:51:27.026060+0000","last_became_active":"2023-10-06T03:51:26.011853+0000","last_became_peered":"2023-10-06T03:51:26.011853+0000","last_unstale":"2023-10-06T03:51:27.026060+0000","last_undegraded":"2023-10-06T03:51:27.026060+0000","last_fullsized":"2023-10-06T03:51:27.026060+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:57.318701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013780+0000","last_change":"2023-10-06T03:51:27.003308+0000","last_active":"2023-10-06T03:51:27.013780+0000","last_peered":"2023-10-06T03:51:27.013780+0000","last_clean":"2023-10-06T03:51:27.013780+0000","last_became_active":"2023-10-06T03:51:27.003059+0000","last_became_peered":"2023-10-06T03:51:27.003059+0000","last_unstale":"2023-10-06T03:51:27.013780+0000","last_undegraded":"2023-10-06T03:51:27.013780+0000","last_fullsized":"2023-10-06T03:51:27.013780+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:01.933888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.991947+0000","last_change":"2023-10-06T03:51:26.007819+0000","last_active":"2023-10-06T03:51:26.991947+0000","last_peered":"2023-10-06T03:51:26.991947+0000","last_clean":"2023-10-06T03:51:26.991947+0000","last_became_active":"2023-10-06T03:51:26.007539+0000","last_became_peered":"2023-10-06T03:51:26.007539+0000","last_unstale":"2023-10-06T03:51:26.991947+0000","last_undegraded":"2023-10-06T03:51:26.991947+0000","last_fullsized":"2023-10-06T03:51:26.991947+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:00.923327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.015236+0000","last_change":"2023-10-06T03:51:27.015236+0000","last_active":"2023-10-06T03:51:27.015236+0000","last_peered":"2023-10-06T03:51:27.015236+0000","last_clean":"2023-10-06T03:51:27.015236+0000","last_became_active":"2023-10-06T03:51:27.014972+0000","last_became_peered":"2023-10-06T03:51:27.014972+0000","last_unstale":"2023-10-06T03:51:27.015236+0000","last_undegraded":"2023-10-06T03:51:27.015236+0000","last_fullsized":"2023-10-06T03:51:27.015236+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:03.633065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.008292+0000","last_change":"2023-10-06T03:51:26.007085+0000","last_active":"2023-10-06T03:51:26.008292+0000","last_peered":"2023-10-06T03:51:26.008292+0000","last_clean":"2023-10-06T03:51:26.008292+0000","last_became_active":"2023-10-06T03:51:26.006801+0000","last_became_peered":"2023-10-06T03:51:26.006801+0000","last_unstale":"2023-10-06T03:51:26.008292+0000","last_undegraded":"2023-10-06T03:51:26.008292+0000","last_fullsized":"2023-10-06T03:51:26.008292+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:37.732660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013644+0000","last_change":"2023-10-06T03:51:27.006753+0000","last_active":"2023-10-06T03:51:27.013644+0000","last_peered":"2023-10-06T03:51:27.013644+0000","last_clean":"2023-10-06T03:51:27.013644+0000","last_became_active":"2023-10-06T03:51:27.006454+0000","last_became_peered":"2023-10-06T03:51:27.006454+0000","last_unstale":"2023-10-06T03:51:27.013644+0000","last_undegraded":"2023-10-06T03:51:27.013644+0000","last_fullsized":"2023-10-06T03:51:27.013644+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:21.042231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"122'8","reported_seq":18,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.023713+0000","last_change":"2023-10-06T03:51:26.001111+0000","last_active":"2023-10-06T03:51:27.023713+0000","last_peered":"2023-10-06T03:51:27.023713+0000","last_clean":"2023-10-06T03:51:27.023713+0000","last_became_active":"2023-10-06T03:51:26.000829+0000","last_became_peered":"2023-10-06T03:51:26.000829+0000","last_unstale":"2023-10-06T03:51:27.023713+0000","last_undegraded":"2023-10-06T03:51:27.023713+0000","last_fullsized":"2023-10-06T03:51:27.023713+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:51.309439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013332+0000","last_change":"2023-10-06T03:51:27.006156+0000","last_active":"2023-10-06T03:51:27.013332+0000","last_peered":"2023-10-06T03:51:27.013332+0000","last_clean":"2023-10-06T03:51:27.013332+0000","last_became_active":"2023-10-06T03:51:27.005882+0000","last_became_peered":"2023-10-06T03:51:27.005882+0000","last_unstale":"2023-10-06T03:51:27.013332+0000","last_undegraded":"2023-10-06T03:51:27.013332+0000","last_fullsized":"2023-10-06T03:51:27.013332+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:48.236512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"122'2","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.025955+0000","last_change":"2023-10-06T03:51:26.010112+0000","last_active":"2023-10-06T03:51:27.025955+0000","last_peered":"2023-10-06T03:51:27.025955+0000","last_clean":"2023-10-06T03:51:27.025955+0000","last_became_active":"2023-10-06T03:51:26.009797+0000","last_became_peered":"2023-10-06T03:51:26.009797+0000","last_unstale":"2023-10-06T03:51:27.025955+0000","last_undegraded":"2023-10-06T03:51:27.025955+0000","last_fullsized":"2023-10-06T03:51:27.025955+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:18.194158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.015315+0000","last_change":"2023-10-06T03:51:27.009587+0000","last_active":"2023-10-06T03:51:27.015315+0000","last_peered":"2023-10-06T03:51:27.015315+0000","last_clean":"2023-10-06T03:51:27.015315+0000","last_became_active":"2023-10-06T03:51:27.009315+0000","last_became_peered":"2023-10-06T03:51:27.009315+0000","last_unstale":"2023-10-06T03:51:27.015315+0000","last_undegraded":"2023-10-06T03:51:27.015315+0000","last_fullsized":"2023-10-06T03:51:27.015315+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:27.069885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.988842+0000","last_change":"2023-10-06T03:51:26.005001+0000","last_active":"2023-10-06T03:51:26.988842+0000","last_peered":"2023-10-06T03:51:26.988842+0000","last_clean":"2023-10-06T03:51:26.988842+0000","last_became_active":"2023-10-06T03:51:26.004721+0000","last_became_peered":"2023-10-06T03:51:26.004721+0000","last_unstale":"2023-10-06T03:51:26.988842+0000","last_undegraded":"2023-10-06T03:51:26.988842+0000","last_fullsized":"2023-10-06T03:51:26.988842+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:25.341432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.001829+0000","last_change":"2023-10-06T03:51:26.999323+0000","last_active":"2023-10-06T03:51:27.001829+0000","last_peered":"2023-10-06T03:51:27.001829+0000","last_clean":"2023-10-06T03:51:27.001829+0000","last_became_active":"2023-10-06T03:51:26.999068+0000","last_became_peered":"2023-10-06T03:51:26.999068+0000","last_unstale":"2023-10-06T03:51:27.001829+0000","last_undegraded":"2023-10-06T03:51:27.001829+0000","last_fullsized":"2023-10-06T03:51:27.001829+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:20.239138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.993830+0000","last_change":"2023-10-06T03:51:26.003558+0000","last_active":"2023-10-06T03:51:26.993830+0000","last_peered":"2023-10-06T03:51:26.993830+0000","last_clean":"2023-10-06T03:51:26.993830+0000","last_became_active":"2023-10-06T03:51:26.003266+0000","last_became_peered":"2023-10-06T03:51:26.003266+0000","last_unstale":"2023-10-06T03:51:26.993830+0000","last_undegraded":"2023-10-06T03:51:26.993830+0000","last_fullsized":"2023-10-06T03:51:26.993830+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:35:03.361008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.017661+0000","last_change":"2023-10-06T03:51:27.011649+0000","last_active":"2023-10-06T03:51:27.017661+0000","last_peered":"2023-10-06T03:51:27.017661+0000","last_clean":"2023-10-06T03:51:27.017661+0000","last_became_active":"2023-10-06T03:51:27.011353+0000","last_became_peered":"2023-10-06T03:51:27.011353+0000","last_unstale":"2023-10-06T03:51:27.017661+0000","last_undegraded":"2023-10-06T03:51:27.017661+0000","last_fullsized":"2023-10-06T03:51:27.017661+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:31:21.112082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.008342+0000","last_change":"2023-10-06T03:51:26.005561+0000","last_active":"2023-10-06T03:51:26.008342+0000","last_peered":"2023-10-06T03:51:26.008342+0000","last_clean":"2023-10-06T03:51:26.008342+0000","last_became_active":"2023-10-06T03:51:26.005298+0000","last_became_peered":"2023-10-06T03:51:26.005298+0000","last_unstale":"2023-10-06T03:51:26.008342+0000","last_undegraded":"2023-10-06T03:51:26.008342+0000","last_fullsized":"2023-10-06T03:51:26.008342+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:35.317997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.020831+0000","last_change":"2023-10-06T03:51:27.020831+0000","last_active":"2023-10-06T03:51:27.020831+0000","last_peered":"2023-10-06T03:51:27.020831+0000","last_clean":"2023-10-06T03:51:27.020831+0000","last_became_active":"2023-10-06T03:51:27.020577+0000","last_became_peered":"2023-10-06T03:51:27.020577+0000","last_unstale":"2023-10-06T03:51:27.020831+0000","last_undegraded":"2023-10-06T03:51:27.020831+0000","last_fullsized":"2023-10-06T03:51:27.020831+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:54.912355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.991247+0000","last_change":"2023-10-06T03:51:26.004369+0000","last_active":"2023-10-06T03:51:26.991247+0000","last_peered":"2023-10-06T03:51:26.991247+0000","last_clean":"2023-10-06T03:51:26.991247+0000","last_became_active":"2023-10-06T03:51:26.004058+0000","last_became_peered":"2023-10-06T03:51:26.004058+0000","last_unstale":"2023-10-06T03:51:26.991247+0000","last_undegraded":"2023-10-06T03:51:26.991247+0000","last_fullsized":"2023-10-06T03:51:26.991247+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:19.103441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.024001+0000","last_change":"2023-10-06T03:51:27.024001+0000","last_active":"2023-10-06T03:51:27.024001+0000","last_peered":"2023-10-06T03:51:27.024001+0000","last_clean":"2023-10-06T03:51:27.024001+0000","last_became_active":"2023-10-06T03:51:27.023601+0000","last_became_peered":"2023-10-06T03:51:27.023601+0000","last_unstale":"2023-10-06T03:51:27.024001+0000","last_undegraded":"2023-10-06T03:51:27.024001+0000","last_fullsized":"2023-10-06T03:51:27.024001+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:04.511512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.013273+0000","last_change":"2023-10-06T03:51:26.007697+0000","last_active":"2023-10-06T03:51:26.013273+0000","last_peered":"2023-10-06T03:51:26.013273+0000","last_clean":"2023-10-06T03:51:26.013273+0000","last_became_active":"2023-10-06T03:51:26.007431+0000","last_became_peered":"2023-10-06T03:51:26.007431+0000","last_unstale":"2023-10-06T03:51:26.013273+0000","last_undegraded":"2023-10-06T03:51:26.013273+0000","last_fullsized":"2023-10-06T03:51:26.013273+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:47.825844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.007817+0000","last_change":"2023-10-06T03:51:27.007817+0000","last_active":"2023-10-06T03:51:27.007817+0000","last_peered":"2023-10-06T03:51:27.007817+0000","last_clean":"2023-10-06T03:51:27.007817+0000","last_became_active":"2023-10-06T03:51:27.007475+0000","last_became_peered":"2023-10-06T03:51:27.007475+0000","last_unstale":"2023-10-06T03:51:27.007817+0000","last_undegraded":"2023-10-06T03:51:27.007817+0000","last_fullsized":"2023-10-06T03:51:27.007817+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:03.629521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.993212+0000","last_change":"2023-10-06T03:51:26.002917+0000","last_active":"2023-10-06T03:51:26.993212+0000","last_peered":"2023-10-06T03:51:26.993212+0000","last_clean":"2023-10-06T03:51:26.993212+0000","last_became_active":"2023-10-06T03:51:26.002615+0000","last_became_peered":"2023-10-06T03:51:26.002615+0000","last_unstale":"2023-10-06T03:51:26.993212+0000","last_undegraded":"2023-10-06T03:51:26.993212+0000","last_fullsized":"2023-10-06T03:51:26.993212+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:29.245315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.015455+0000","last_change":"2023-10-06T03:51:27.005445+0000","last_active":"2023-10-06T03:51:27.015455+0000","last_peered":"2023-10-06T03:51:27.015455+0000","last_clean":"2023-10-06T03:51:27.015455+0000","last_became_active":"2023-10-06T03:51:27.005077+0000","last_became_peered":"2023-10-06T03:51:27.005077+0000","last_unstale":"2023-10-06T03:51:27.015455+0000","last_undegraded":"2023-10-06T03:51:27.015455+0000","last_fullsized":"2023-10-06T03:51:27.015455+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:01.933888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.008344+0000","last_change":"2023-10-06T03:51:26.007078+0000","last_active":"2023-10-06T03:51:26.008344+0000","last_peered":"2023-10-06T03:51:26.008344+0000","last_clean":"2023-10-06T03:51:26.008344+0000","last_became_active":"2023-10-06T03:51:26.006779+0000","last_became_peered":"2023-10-06T03:51:26.006779+0000","last_unstale":"2023-10-06T03:51:26.008344+0000","last_undegraded":"2023-10-06T03:51:26.008344+0000","last_fullsized":"2023-10-06T03:51:26.008344+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:16.164706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013728+0000","last_change":"2023-10-06T03:51:27.007390+0000","last_active":"2023-10-06T03:51:27.013728+0000","last_peered":"2023-10-06T03:51:27.013728+0000","last_clean":"2023-10-06T03:51:27.013728+0000","last_became_active":"2023-10-06T03:51:27.007148+0000","last_became_peered":"2023-10-06T03:51:27.007148+0000","last_unstale":"2023-10-06T03:51:27.013728+0000","last_undegraded":"2023-10-06T03:51:27.013728+0000","last_fullsized":"2023-10-06T03:51:27.013728+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:48:11.540041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"122'2","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.018670+0000","last_change":"2023-10-06T03:51:26.005705+0000","last_active":"2023-10-06T03:51:27.018670+0000","last_peered":"2023-10-06T03:51:27.018670+0000","last_clean":"2023-10-06T03:51:27.018670+0000","last_became_active":"2023-10-06T03:51:26.005458+0000","last_became_peered":"2023-10-06T03:51:26.005458+0000","last_unstale":"2023-10-06T03:51:27.018670+0000","last_undegraded":"2023-10-06T03:51:27.018670+0000","last_fullsized":"2023-10-06T03:51:27.018670+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:17.449763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.991983+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:51:26.991983+0000","last_peered":"2023-10-06T03:51:26.991983+0000","last_clean":"2023-10-06T03:51:26.991983+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:51:26.991983+0000","last_undegraded":"2023-10-06T03:51:26.991983+0000","last_fullsized":"2023-10-06T03:51:26.991983+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013684+0000","last_change":"2023-10-06T03:51:27.008355+0000","last_active":"2023-10-06T03:51:27.013684+0000","last_peered":"2023-10-06T03:51:27.013684+0000","last_clean":"2023-10-06T03:51:27.013684+0000","last_became_active":"2023-10-06T03:51:27.008068+0000","last_became_peered":"2023-10-06T03:51:27.008068+0000","last_unstale":"2023-10-06T03:51:27.013684+0000","last_undegraded":"2023-10-06T03:51:27.013684+0000","last_fullsized":"2023-10-06T03:51:27.013684+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:50.589706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.013358+0000","last_change":"2023-10-06T03:51:26.010495+0000","last_active":"2023-10-06T03:51:26.013358+0000","last_peered":"2023-10-06T03:51:26.013358+0000","last_clean":"2023-10-06T03:51:26.013358+0000","last_became_active":"2023-10-06T03:51:26.010157+0000","last_became_peered":"2023-10-06T03:51:26.010157+0000","last_unstale":"2023-10-06T03:51:26.013358+0000","last_undegraded":"2023-10-06T03:51:26.013358+0000","last_fullsized":"2023-10-06T03:51:26.013358+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:49.281675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":102,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.994500+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:51:26.994500+0000","last_peered":"2023-10-06T03:51:26.994500+0000","last_clean":"2023-10-06T03:51:26.994500+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:51:26.994500+0000","last_undegraded":"2023-10-06T03:51:26.994500+0000","last_fullsized":"2023-10-06T03:51:26.994500+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.012581+0000","last_change":"2023-10-06T03:51:26.998480+0000","last_active":"2023-10-06T03:51:27.012581+0000","last_peered":"2023-10-06T03:51:27.012581+0000","last_clean":"2023-10-06T03:51:27.012581+0000","last_became_active":"2023-10-06T03:51:26.998219+0000","last_became_peered":"2023-10-06T03:51:26.998219+0000","last_unstale":"2023-10-06T03:51:27.012581+0000","last_undegraded":"2023-10-06T03:51:27.012581+0000","last_fullsized":"2023-10-06T03:51:27.012581+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:46.206866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"122'1","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.993971+0000","last_change":"2023-10-06T03:51:26.008613+0000","last_active":"2023-10-06T03:51:26.993971+0000","last_peered":"2023-10-06T03:51:26.993971+0000","last_clean":"2023-10-06T03:51:26.993971+0000","last_became_active":"2023-10-06T03:51:26.008346+0000","last_became_peered":"2023-10-06T03:51:26.008346+0000","last_unstale":"2023-10-06T03:51:26.993971+0000","last_undegraded":"2023-10-06T03:51:26.993971+0000","last_fullsized":"2023-10-06T03:51:26.993971+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:45.751206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:25.995771+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:51:25.995771+0000","last_peered":"2023-10-06T03:51:25.995771+0000","last_clean":"2023-10-06T03:51:25.995771+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:51:25.995771+0000","last_undegraded":"2023-10-06T03:51:25.995771+0000","last_fullsized":"2023-10-06T03:51:25.995771+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"112'227","reported_seq":641,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.002689+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:51:26.002689+0000","last_peered":"2023-10-06T03:51:26.002689+0000","last_clean":"2023-10-06T03:51:26.002689+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:51:26.002689+0000","last_undegraded":"2023-10-06T03:51:26.002689+0000","last_fullsized":"2023-10-06T03:51:26.002689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.011674+0000","last_change":"2023-10-06T03:51:27.011674+0000","last_active":"2023-10-06T03:51:27.011674+0000","last_peered":"2023-10-06T03:51:27.011674+0000","last_clean":"2023-10-06T03:51:27.011674+0000","last_became_active":"2023-10-06T03:51:27.010409+0000","last_became_peered":"2023-10-06T03:51:27.010409+0000","last_unstale":"2023-10-06T03:51:27.011674+0000","last_undegraded":"2023-10-06T03:51:27.011674+0000","last_fullsized":"2023-10-06T03:51:27.011674+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:51.302796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.002924+0000","last_change":"2023-10-06T03:51:26.003657+0000","last_active":"2023-10-06T03:51:27.002924+0000","last_peered":"2023-10-06T03:51:27.002924+0000","last_clean":"2023-10-06T03:51:27.002924+0000","last_became_active":"2023-10-06T03:51:26.003384+0000","last_became_peered":"2023-10-06T03:51:26.003384+0000","last_unstale":"2023-10-06T03:51:27.002924+0000","last_undegraded":"2023-10-06T03:51:27.002924+0000","last_fullsized":"2023-10-06T03:51:27.002924+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:55.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.001579+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:51:26.001579+0000","last_peered":"2023-10-06T03:51:26.001579+0000","last_clean":"2023-10-06T03:51:26.001579+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:51:26.001579+0000","last_undegraded":"2023-10-06T03:51:26.001579+0000","last_fullsized":"2023-10-06T03:51:26.001579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.005080+0000","last_change":"2023-10-06T03:51:27.001983+0000","last_active":"2023-10-06T03:51:27.005080+0000","last_peered":"2023-10-06T03:51:27.005080+0000","last_clean":"2023-10-06T03:51:27.005080+0000","last_became_active":"2023-10-06T03:51:27.001713+0000","last_became_peered":"2023-10-06T03:51:27.001713+0000","last_unstale":"2023-10-06T03:51:27.005080+0000","last_undegraded":"2023-10-06T03:51:27.005080+0000","last_fullsized":"2023-10-06T03:51:27.005080+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:58.329261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"122'2","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.023863+0000","last_change":"2023-10-06T03:51:26.011254+0000","last_active":"2023-10-06T03:51:27.023863+0000","last_peered":"2023-10-06T03:51:27.023863+0000","last_clean":"2023-10-06T03:51:27.023863+0000","last_became_active":"2023-10-06T03:51:26.010949+0000","last_became_peered":"2023-10-06T03:51:26.010949+0000","last_unstale":"2023-10-06T03:51:27.023863+0000","last_undegraded":"2023-10-06T03:51:27.023863+0000","last_fullsized":"2023-10-06T03:51:27.023863+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:24.241375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.002780+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:51:26.002780+0000","last_peered":"2023-10-06T03:51:26.002780+0000","last_clean":"2023-10-06T03:51:26.002780+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:51:26.002780+0000","last_undegraded":"2023-10-06T03:51:26.002780+0000","last_fullsized":"2023-10-06T03:51:26.002780+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.015583+0000","last_change":"2023-10-06T03:51:27.015583+0000","last_active":"2023-10-06T03:51:27.015583+0000","last_peered":"2023-10-06T03:51:27.015583+0000","last_clean":"2023-10-06T03:51:27.015583+0000","last_became_active":"2023-10-06T03:51:27.015326+0000","last_became_peered":"2023-10-06T03:51:27.015326+0000","last_unstale":"2023-10-06T03:51:27.015583+0000","last_undegraded":"2023-10-06T03:51:27.015583+0000","last_fullsized":"2023-10-06T03:51:27.015583+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:23.512840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"122'1","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.023279+0000","last_change":"2023-10-06T03:51:26.002515+0000","last_active":"2023-10-06T03:51:27.023279+0000","last_peered":"2023-10-06T03:51:27.023279+0000","last_clean":"2023-10-06T03:51:27.023279+0000","last_became_active":"2023-10-06T03:51:26.002112+0000","last_became_peered":"2023-10-06T03:51:26.002112+0000","last_unstale":"2023-10-06T03:51:27.023279+0000","last_undegraded":"2023-10-06T03:51:27.023279+0000","last_fullsized":"2023-10-06T03:51:27.023279+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:02.645285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.003078+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:51:27.003078+0000","last_peered":"2023-10-06T03:51:27.003078+0000","last_clean":"2023-10-06T03:51:27.003078+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:51:27.003078+0000","last_undegraded":"2023-10-06T03:51:27.003078+0000","last_fullsized":"2023-10-06T03:51:27.003078+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.023737+0000","last_change":"2023-10-06T03:51:27.023737+0000","last_active":"2023-10-06T03:51:27.023737+0000","last_peered":"2023-10-06T03:51:27.023737+0000","last_clean":"2023-10-06T03:51:27.023737+0000","last_became_active":"2023-10-06T03:51:27.023460+0000","last_became_peered":"2023-10-06T03:51:27.023460+0000","last_unstale":"2023-10-06T03:51:27.023737+0000","last_undegraded":"2023-10-06T03:51:27.023737+0000","last_fullsized":"2023-10-06T03:51:27.023737+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:11:09.366396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.013499+0000","last_change":"2023-10-06T03:51:26.009737+0000","last_active":"2023-10-06T03:51:26.013499+0000","last_peered":"2023-10-06T03:51:26.013499+0000","last_clean":"2023-10-06T03:51:26.013499+0000","last_became_active":"2023-10-06T03:51:26.009211+0000","last_became_peered":"2023-10-06T03:51:26.009211+0000","last_unstale":"2023-10-06T03:51:26.013499+0000","last_undegraded":"2023-10-06T03:51:26.013499+0000","last_fullsized":"2023-10-06T03:51:26.013499+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:07:02.657802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":116,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.992803+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:51:26.992803+0000","last_peered":"2023-10-06T03:51:26.992803+0000","last_clean":"2023-10-06T03:51:26.992803+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:51:26.992803+0000","last_undegraded":"2023-10-06T03:51:26.992803+0000","last_fullsized":"2023-10-06T03:51:26.992803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.020360+0000","last_change":"2023-10-06T03:51:27.017224+0000","last_active":"2023-10-06T03:51:27.020360+0000","last_peered":"2023-10-06T03:51:27.020360+0000","last_clean":"2023-10-06T03:51:27.020360+0000","last_became_active":"2023-10-06T03:51:27.016928+0000","last_became_peered":"2023-10-06T03:51:27.016928+0000","last_unstale":"2023-10-06T03:51:27.020360+0000","last_undegraded":"2023-10-06T03:51:27.020360+0000","last_fullsized":"2023-10-06T03:51:27.020360+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:42.614149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.993044+0000","last_change":"2023-10-06T03:51:25.998061+0000","last_active":"2023-10-06T03:51:26.993044+0000","last_peered":"2023-10-06T03:51:26.993044+0000","last_clean":"2023-10-06T03:51:26.993044+0000","last_became_active":"2023-10-06T03:51:25.997343+0000","last_became_peered":"2023-10-06T03:51:25.997343+0000","last_unstale":"2023-10-06T03:51:26.993044+0000","last_undegraded":"2023-10-06T03:51:26.993044+0000","last_fullsized":"2023-10-06T03:51:26.993044+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:23.839660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.992686+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:51:26.992686+0000","last_peered":"2023-10-06T03:51:26.992686+0000","last_clean":"2023-10-06T03:51:26.992686+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:51:26.992686+0000","last_undegraded":"2023-10-06T03:51:26.992686+0000","last_fullsized":"2023-10-06T03:51:26.992686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575063,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311632,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060208,"statfs":{"total":96636764160,"available":96317652992,"internally_reserved":0,"allocated":2105344,"data_stored":1364369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.69599999999999995},"min":{"1min":0.48399999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.89400000000000002,"5min":0.998,"15min":1.0409999999999999},"last":0.62},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65600000000000003,"15min":0.70199999999999996},"min":{"1min":0.48899999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83699999999999997,"5min":1.02,"15min":1.02},"last":0.57899999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.70099999999999996,"15min":0.75},"min":{"1min":0.55000000000000004,"5min":0.318,"15min":0.318},"max":{"1min":0.96199999999999997,"5min":1.038,"15min":1.0640000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73099999999999998,"15min":0.76600000000000001},"min":{"1min":0.56699999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95699999999999996,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.64500000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66400000000000003,"15min":0.69599999999999995},"min":{"1min":0.55000000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":0.97499999999999998,"15min":1.044},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71299999999999997,"15min":0.73999999999999999},"min":{"1min":0.58199999999999996,"5min":0.44,"15min":0.44},"max":{"1min":0.85999999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.68899999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69899999999999995,"15min":0.72499999999999998},"min":{"1min":0.52200000000000002,"5min":0.41799999999999998,"15min":0.39500000000000002},"max":{"1min":0.81599999999999995,"5min":1.0089999999999999,"15min":1.016},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72599999999999998,"15min":0.74299999999999999},"min":{"1min":0.54000000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92000000000000004,"5min":0.98499999999999999,"15min":1.0409999999999999},"last":0.79200000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61699999999999999,"15min":0.59399999999999997},"min":{"1min":0.42499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90600000000000003,"5min":0.99199999999999999,"15min":1.0049999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63700000000000001,"15min":0.622},"min":{"1min":0.442,"5min":0.41299999999999998,"15min":0.39400000000000002},"max":{"1min":0.94699999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.54700000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66300000000000003,"15min":0.64000000000000001},"min":{"1min":0.48299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92300000000000004,"5min":1.022,"15min":1.069},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69299999999999995,"15min":0.68000000000000005},"min":{"1min":0.501,"5min":0.44500000000000001,"15min":0.436},"max":{"1min":0.997,"5min":1.105,"15min":1.105},"last":0.502}]},{"osd":6,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.748,"15min":0.72699999999999998},"min":{"1min":0.47799999999999998,"5min":0.442,"15min":0.436},"max":{"1min":1.034,"5min":1.181,"15min":1.181},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.503,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90900000000000003,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.70299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575064,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306984,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064856,"statfs":{"total":96636764160,"available":96322412544,"internally_reserved":0,"allocated":1540096,"data_stored":783964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.70099999999999996},"min":{"1min":0.45200000000000001,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.80300000000000005,"5min":1.288,"15min":1.288},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69799999999999995,"15min":0.72399999999999998},"min":{"1min":0.57299999999999995,"5min":0.41299999999999998,"15min":0.40600000000000003},"max":{"1min":0.88300000000000001,"5min":0.96799999999999997,"15min":1.0069999999999999},"last":0.77200000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.72699999999999998},"min":{"1min":0.5,"5min":0.36499999999999999,"15min":0.33800000000000002},"max":{"1min":0.88500000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69399999999999995,"15min":0.71599999999999997},"min":{"1min":0.46400000000000002,"5min":0.441,"15min":0.35899999999999999},"max":{"1min":0.85199999999999998,"5min":1.427,"15min":1.427},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.52500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89500000000000002,"5min":1.179,"15min":1.179},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70799999999999996,"15min":0.73699999999999999},"min":{"1min":0.47799999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92500000000000004,"5min":1.0209999999999999,"15min":1.0409999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72199999999999998,"15min":0.749},"min":{"1min":0.51800000000000002,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":0.81499999999999995,"5min":1.01,"15min":1.024},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72899999999999998,"15min":0.75800000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86599999999999999,"5min":1.097,"15min":1.1639999999999999},"last":0.82899999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.78900000000000003,"5min":1.462,"15min":1.462},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.627,"15min":0.63},"min":{"1min":0.377,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93100000000000005,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68999999999999995,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":1.0109999999999999,"15min":1.0489999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41199999999999998},"max":{"1min":0.91000000000000003,"5min":1.508,"15min":1.508},"last":0.62}]},{"osd":7,"last update":"Fri Oct 6 03:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.41499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.98399999999999999,"5min":1.21,"15min":1.21},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.40000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.95499999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.55400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575063,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307536,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064304,"statfs":{"total":96636764160,"available":96321847296,"internally_reserved":0,"allocated":2105344,"data_stored":1365221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46500000000000002,"15min":0.48699999999999999},"min":{"1min":0.33100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68300000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.438},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54800000000000004,"15min":0.56799999999999995},"min":{"1min":0.36399999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73699999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46600000000000003,"15min":0.49199999999999999},"min":{"1min":0.32700000000000001,"5min":0.32000000000000001,"15min":0.309},"max":{"1min":0.58599999999999997,"5min":0.72699999999999998,"15min":0.84799999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.94099999999999995},"last":0.499}]},{"osd":3,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.45800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.77400000000000002,"5min":0.86699999999999999,"15min":0.91700000000000004},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.53700000000000003},"min":{"1min":0.41299999999999998,"5min":0.34200000000000003,"15min":0.32500000000000001},"max":{"1min":0.74199999999999999,"5min":0.81499999999999995,"15min":0.90400000000000003},"last":0.59799999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.50800000000000001,"5min":0.46700000000000003,"15min":0.41499999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.1699999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.76600000000000001,"15min":0.79900000000000004},"min":{"1min":0.58699999999999997,"5min":0.51400000000000001,"15min":0.439},"max":{"1min":0.996,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77200000000000002,"15min":0.78700000000000003},"min":{"1min":0.58499999999999996,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":0.94999999999999996,"5min":1.294,"15min":1.294},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.77900000000000003,"15min":0.80000000000000004},"min":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.45100000000000001},"max":{"1min":1.0029999999999999,"5min":1.224,"15min":1.224},"last":0.71899999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78200000000000003,"15min":0.80300000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.117,"5min":1.26,"15min":1.26},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78500000000000003,"15min":0.81599999999999995},"min":{"1min":0.55200000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0880000000000001,"5min":1.143,"15min":1.224},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.81799999999999995,"15min":0.83699999999999997},"min":{"1min":0.53100000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0629999999999999,"5min":1.1519999999999999,"15min":1.272},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81399999999999995,"15min":0.83099999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.51200000000000001},"max":{"1min":1.079,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.68999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575065,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306976,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064864,"statfs":{"total":96636764160,"available":96322420736,"internally_reserved":0,"allocated":1531904,"data_stored":782500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.505,"15min":0.51800000000000002},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.68999999999999995,"5min":0.70999999999999996,"15min":0.83599999999999997},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.54500000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.745,"5min":0.80100000000000005,"15min":1.1299999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.31900000000000001},"max":{"1min":0.73999999999999999,"5min":0.83399999999999996,"15min":0.89400000000000002},"last":0.497}]},{"osd":3,"last update":"Fri Oct 6 03:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.318,"15min":0.30499999999999999},"max":{"1min":0.83499999999999996,"5min":0.83699999999999997,"15min":0.93200000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.79800000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.54000000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.80600000000000005,"15min":0.82999999999999996},"min":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.49399999999999999},"max":{"1min":1.05,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.80200000000000005,"15min":0.82499999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.45500000000000002},"max":{"1min":1.1240000000000001,"5min":1.306,"15min":1.3400000000000001},"last":0.96099999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79300000000000004,"15min":0.81499999999999995},"min":{"1min":0.55100000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1459999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.042},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.6559999999999999},"last":0.77700000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76600000000000001,"15min":0.80600000000000005},"min":{"1min":0.56100000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.726},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.80900000000000005,"15min":0.84099999999999997},"min":{"1min":0.52900000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.212,"5min":1.212,"15min":1.843},"last":1.01}]},{"osd":7,"last update":"Fri Oct 6 03:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.80800000000000005,"15min":0.82499999999999996},"min":{"1min":0.502,"5min":0.496,"15min":0.496},"max":{"1min":1.1419999999999999,"5min":1.302,"15min":1.302},"last":0.996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.80300000000000005,"15min":0.83199999999999996},"min":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.495},"max":{"1min":1.085,"5min":1.2450000000000001,"15min":1.7649999999999999},"last":0.89100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575064,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311048,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060792,"statfs":{"total":96636764160,"available":96318251008,"internally_reserved":0,"allocated":1507328,"data_stored":773875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.45500000000000002,"15min":0.47399999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.622,"5min":0.74199999999999999,"15min":0.80100000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.51900000000000002,"15min":0.53100000000000003},"min":{"1min":0.379,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":0.76500000000000001,"5min":0.86699999999999999,"15min":0.89300000000000002},"last":0.36799999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.28699999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.90000000000000002},"last":0.311},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83099999999999996,"5min":0.89700000000000002,"15min":0.94899999999999995},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.374,"5min":0.37,"15min":0.37},"max":{"1min":0.78300000000000003,"5min":1.78,"15min":1.78},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56799999999999995,"15min":0.57699999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.33300000000000002},"max":{"1min":0.81599999999999995,"5min":0.84699999999999998,"15min":0.96499999999999997},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.57999999999999996,"5min":0.51400000000000001,"15min":0.51200000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.5620000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.82699999999999996,"15min":0.84399999999999997},"min":{"1min":0.61199999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.2270000000000001},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Oct 6 03:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.82599999999999996,"15min":0.84199999999999997},"min":{"1min":0.59399999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.135,"5min":1.135,"15min":1.3},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82199999999999995,"15min":0.85399999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0449999999999999,"5min":1.855,"15min":1.855},"last":0.68300000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.53600000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99299999999999999,"5min":1.907,"15min":1.907},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.81100000000000005,"15min":0.82799999999999996},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0680000000000001,"5min":1.1080000000000001,"15min":1.4750000000000001},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.82399999999999995,"15min":0.84999999999999998},"min":{"1min":0.54700000000000004,"5min":0.495,"15min":0.495},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.4990000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83699999999999997,"15min":0.871},"min":{"1min":0.56899999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1140000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.57399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575064,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306956,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064884,"statfs":{"total":96636764160,"available":96322441216,"internally_reserved":0,"allocated":1511424,"data_stored":775373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:50:29 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44600000000000001,"15min":0.45900000000000002},"min":{"1min":0.34899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68700000000000006,"5min":0.75700000000000001,"15min":0.83999999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47199999999999998,"15min":0.497},"min":{"1min":0.374,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":0.67900000000000005,"5min":1.04,"15min":1.1699999999999999},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:50:29 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53400000000000003,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:50:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80000000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57899999999999996,"15min":0.60099999999999998},"min":{"1min":0.48099999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.87},"last":0.58699999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:50:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73799999999999999,"15min":0.77900000000000003},"min":{"1min":0.54200000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.4099999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77000000000000002,"15min":0.82399999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.98699999999999999,"5min":1.0469999999999999,"15min":1.377},"last":0.94199999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:50:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.78800000000000003},"min":{"1min":0.52500000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88600000000000001,"5min":1.012,"15min":1.256},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.745,"15min":0.78100000000000003},"min":{"1min":0.56200000000000006,"5min":0.53500000000000003,"15min":0.49099999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.2030000000000001},"last":0.79300000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:50:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":0.81100000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.04,"5min":1.0580000000000001,"15min":1.323},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79100000000000004,"15min":0.84299999999999997},"min":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0509999999999999,"5min":1.075,"15min":1.2849999999999999},"last":0.96799999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:50:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.76000000000000001,"15min":0.80800000000000005},"min":{"1min":0.60399999999999998,"5min":0.498,"15min":0.498},"max":{"1min":0.95699999999999996,"5min":0.98999999999999999,"15min":1.2470000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76900000000000002,"15min":0.80200000000000005},"min":{"1min":0.53700000000000003,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.95599999999999996,"5min":1.093,"15min":1.2330000000000001},"last":0.77800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575064,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306972,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064868,"statfs":{"total":96636764160,"available":96322424832,"internally_reserved":0,"allocated":1527808,"data_stored":781944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.69499999999999995,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.316},"max":{"1min":0.996,"5min":0.996,"15min":1.0189999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67900000000000005,"15min":0.70299999999999996},"min":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.316},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.01},"last":0.80300000000000005}]},{"osd":1,"last update":"Fri Oct 6 03:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69199999999999995,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.42999999999999999,"15min":0.33200000000000002},"max":{"1min":1.004,"5min":1.081,"15min":1.081},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.72099999999999997,"15min":0.73899999999999999},"min":{"1min":0.45400000000000001,"5min":0.42799999999999999,"15min":0.41299999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.68700000000000006}]},{"osd":2,"last update":"Fri Oct 6 03:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71299999999999997,"15min":0.73799999999999999},"min":{"1min":0.55500000000000005,"5min":0.44400000000000001,"15min":0.377},"max":{"1min":0.95499999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.874},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73799999999999999,"15min":0.748},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.022},"last":0.84299999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70899999999999996,"15min":0.72799999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97399999999999998,"5min":1.0189999999999999,"15min":1.0409999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73799999999999999,"15min":0.76500000000000001},"min":{"1min":0.53500000000000003,"5min":0.47099999999999997,"15min":0.46500000000000002},"max":{"1min":0.90800000000000003,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.90000000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.64200000000000002,"15min":0.627},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.044,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.502}]},{"osd":6,"last update":"Fri Oct 6 03:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65500000000000003,"15min":0.64100000000000001},"min":{"1min":0.437,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88100000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70899999999999996,"15min":0.68799999999999994},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38100000000000001},"max":{"1min":1.0760000000000001,"5min":1.083,"15min":1.083},"last":0.53700000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69599999999999995,"15min":0.67900000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93700000000000006,"5min":1.046,"15min":1.133},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72699999999999998,"15min":0.71399999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.41299999999999998},"max":{"1min":1.016,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.77900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575064,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1503232,"data_stored":773841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68500000000000005,"15min":0.70799999999999996},"min":{"1min":0.50600000000000001,"5min":0.35999999999999999,"15min":0.35299999999999998},"max":{"1min":0.878,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70999999999999996,"15min":0.72099999999999997},"min":{"1min":0.495,"5min":0.42399999999999999,"15min":0.32600000000000001},"max":{"1min":0.98999999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.58799999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.75700000000000001,"15min":0.73699999999999999},"min":{"1min":0.56100000000000005,"5min":0.45900000000000002,"15min":0.40000000000000002},"max":{"1min":5.8239999999999998,"5min":5.8239999999999998,"15min":5.8239999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.76000000000000001,"15min":0.746},"min":{"1min":0.501,"5min":0.44,"15min":0.32400000000000001},"max":{"1min":5.8680000000000003,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.48999999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.75700000000000001,"15min":0.77300000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.40200000000000002},"max":{"1min":0.96799999999999997,"5min":1.155,"15min":1.155},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75900000000000001,"15min":0.77600000000000002},"min":{"1min":0.58599999999999997,"5min":0.51200000000000001,"15min":0.50700000000000001},"max":{"1min":1,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.749}]},{"osd":3,"last update":"Fri Oct 6 03:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71399999999999997,"15min":0.73199999999999998},"min":{"1min":0.52100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89900000000000002,"5min":1.091,"15min":1.6419999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.59899999999999998,"5min":0.503,"15min":0.47799999999999998},"max":{"1min":0.94299999999999995,"5min":1.238,"15min":1.238},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64400000000000002,"15min":0.61699999999999999},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":1.04,"15min":1.04},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.64100000000000001,"15min":0.626},"min":{"1min":0.441,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":0.92700000000000005,"5min":1.147,"15min":1.147},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.66600000000000004,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.40200000000000002,"15min":0.34999999999999998},"max":{"1min":0.79400000000000004,"5min":1.591,"15min":1.591},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71499999999999997,"15min":0.69199999999999995},"min":{"1min":0.46999999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83099999999999996,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.86899999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.68100000000000005,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.81399999999999995,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72599999999999998,"15min":0.71599999999999997},"min":{"1min":0.47499999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89400000000000002,"5min":1.278,"15min":1.278},"last":0.91500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:51:31.697 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:51:31.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.889+0000 7f4bbcb9d700 1 Processor -- start 2023-10-06T03:51:31.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.889+0000 7f4bbcb9d700 1 -- start start 2023-10-06T03:51:31.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.890+0000 7f4bbcb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bb81566e0 0x7f4bb80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:31.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.890+0000 7f4bbcb9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4bb80cb900 0x7f4bb80ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:31.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.890+0000 7f4bbcb9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb80ca860 con 0x7f4bb8156310 2023-10-06T03:51:31.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.890+0000 7f4bbcb9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb80cab40 con 0x7f4bb80cb900 2023-10-06T03:51:31.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.890+0000 7f4bbcb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80cae20 con 0x7f4bb81566e0 2023-10-06T03:51:31.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.891+0000 7f4bb7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4bb80cb900 0x7f4bb80ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:31.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.891+0000 7f4bb7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4bb80cb900 0x7f4bb80ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37166/0 (socket says 172.21.15.55:37166) 2023-10-06T03:51:31.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.891+0000 7f4bb7fff700 1 -- 172.21.15.55:0/2821581639 learned_addr learned my addr 172.21.15.55:0/2821581639 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:31.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.891+0000 7f4bb6ffd700 1 --2- 172.21.15.55:0/2821581639 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bb81566e0 0x7f4bb80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:31.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.891+0000 7f4bb7fff700 1 -- 172.21.15.55:0/2821581639 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bb81566e0 msgr2=0x7f4bb80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:31.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.891+0000 7f4bb7fff700 1 --2- 172.21.15.55:0/2821581639 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bb81566e0 0x7f4bb80ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.891+0000 7f4bb7fff700 1 -- 172.21.15.55:0/2821581639 >> v1:172.21.15.163:6789/0 conn(0x7f4bb8156310 legacy=0x7f4bb80c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:31.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.891+0000 7f4bb7fff700 1 -- 172.21.15.55:0/2821581639 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb80cb100 con 0x7f4bb80cb900 2023-10-06T03:51:31.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.891+0000 7f4bb7fff700 1 --2- 172.21.15.55:0/2821581639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bb80cb900 0x7f4bb80ae540 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009f20 tx=0x7f4bb0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558cc2d2c263a858 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.892+0000 7f4bb67fc700 1 -- 172.21.15.55:0/2821581639 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000e070 con 0x7f4bb80cb900 2023-10-06T03:51:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.892+0000 7f4bb67fc700 1 -- 172.21.15.55:0/2821581639 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb000b9d0 con 0x7f4bb80cb900 2023-10-06T03:51:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.892+0000 7f4bb67fc700 1 -- 172.21.15.55:0/2821581639 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb00092e0 con 0x7f4bb80cb900 2023-10-06T03:51:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.892+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/2821581639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bb80cb900 msgr2=0x7f4bb80ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.892+0000 7f4bbcb9d700 1 --2- 172.21.15.55:0/2821581639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bb80cb900 0x7f4bb80ae540 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009f20 tx=0x7f4bb0009840 comp rx=0 tx=0).stop 2023-10-06T03:51:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.892+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/2821581639 shutdown_connections 2023-10-06T03:51:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.892+0000 7f4bbcb9d700 1 --2- 172.21.15.55:0/2821581639 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bb80cb900 0x7f4bb80ae540 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.892+0000 7f4bbcb9d700 1 --2- 172.21.15.55:0/2821581639 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bb81566e0 0x7f4bb80ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.892+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/2821581639 >> 172.21.15.55:0/2821581639 conn(0x7f4bb80cc1f0 msgr2=0x7f4bb80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.892+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/2821581639 shutdown_connections 2023-10-06T03:51:31.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.893+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/2821581639 wait complete. 2023-10-06T03:51:31.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.894+0000 7f4bbcb9d700 1 Processor -- start 2023-10-06T03:51:31.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.894+0000 7f4bbcb9d700 1 -- start start 2023-10-06T03:51:31.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.895+0000 7f4bbcb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bb80cb900 0x7f4bb80c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:31.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.895+0000 7f4bbcb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bb8156310 0x7f4bb80c2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.895+0000 7f4bbcb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bb81566e0 0x7f4bb8154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.895+0000 7f4bbcb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80c2fa0 con 0x7f4bb81566e0 2023-10-06T03:51:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.895+0000 7f4bbcb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80ca100 con 0x7f4bb80cb900 2023-10-06T03:51:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.895+0000 7f4bbcb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80ca3e0 con 0x7f4bb8156310 2023-10-06T03:51:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.895+0000 7f4bb7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bb81566e0 0x7f4bb8154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.895+0000 7f4bb7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bb81566e0 0x7f4bb8154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37174/0 (socket says 172.21.15.55:37174) 2023-10-06T03:51:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.895+0000 7f4bb7fff700 1 -- 172.21.15.55:0/1986367169 learned_addr learned my addr 172.21.15.55:0/1986367169 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.895+0000 7f4bb7fff700 1 -- 172.21.15.55:0/1986367169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bb8156310 msgr2=0x7f4bb80c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.895+0000 7f4bb7fff700 1 --2- 172.21.15.55:0/1986367169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bb8156310 0x7f4bb80c2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.896+0000 7f4bb7fff700 1 -- 172.21.15.55:0/1986367169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bb80cb900 msgr2=0x7f4bb80c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.896+0000 7f4bb77fe700 1 --2- 172.21.15.55:0/1986367169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bb80cb900 0x7f4bb80c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.896+0000 7f4bb7fff700 1 --2- 172.21.15.55:0/1986367169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bb80cb900 0x7f4bb80c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.896+0000 7f4bb7fff700 1 -- 172.21.15.55:0/1986367169 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb0009ae0 con 0x7f4bb81566e0 2023-10-06T03:51:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.896+0000 7f4bb77fe700 1 --2- 172.21.15.55:0/1986367169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bb80cb900 0x7f4bb80c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:51:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.896+0000 7f4bb7fff700 1 --2- 172.21.15.55:0/1986367169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bb81566e0 0x7f4bb8154aa0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000b610 tx=0x7f4bb0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.896+0000 7f4b9f7fe700 1 -- 172.21.15.55:0/1986367169 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0008c30 con 0x7f4bb81566e0 2023-10-06T03:51:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.896+0000 7f4b9f7fe700 1 -- 172.21.15.55:0/1986367169 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb0008d90 con 0x7f4bb81566e0 2023-10-06T03:51:31.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.896+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/1986367169 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb80af130 con 0x7f4bb81566e0 2023-10-06T03:51:31.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.896+0000 7f4b9f7fe700 1 -- 172.21.15.55:0/1986367169 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb001d650 con 0x7f4bb81566e0 2023-10-06T03:51:31.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.896+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/1986367169 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb8147cf0 con 0x7f4bb81566e0 2023-10-06T03:51:31.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.897+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/1986367169 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb80aef90 con 0x7f4bb81566e0 2023-10-06T03:51:31.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.898+0000 7f4b9f7fe700 1 -- 172.21.15.55:0/1986367169 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4bb0003040 con 0x7f4bb81566e0 2023-10-06T03:51:31.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.899+0000 7f4b9f7fe700 1 --2- 172.21.15.55:0/1986367169 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4ba4066f30 0x7f4ba40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:31.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.899+0000 7f4b9f7fe700 1 -- 172.21.15.55:0/1986367169 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f4bb0086480 con 0x7f4bb81566e0 2023-10-06T03:51:31.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.900+0000 7f4bb77fe700 1 --2- 172.21.15.55:0/1986367169 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4ba4066f30 0x7f4ba40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:31.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.901+0000 7f4bb77fe700 1 --2- 172.21.15.55:0/1986367169 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4ba4066f30 0x7f4ba40693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8001840 tx=0x7f4ba8009040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:31.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:31.904+0000 7f4b9f7fe700 1 -- 172.21.15.55:0/1986367169 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bb80aef90 con 0x7f4bb81566e0 2023-10-06T03:51:32.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.101+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/1986367169 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4bb80aef90 con 0x7f4ba4066f30 2023-10-06T03:51:32.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.111+0000 7f4b9f7fe700 1 -- 172.21.15.55:0/1986367169 <== mgr.7228 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311773 (secure 0 0 0) 0x7f4bb80aef90 con 0x7f4ba4066f30 2023-10-06T03:51:32.117 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:32.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.116+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/1986367169 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4ba4066f30 msgr2=0x7f4ba40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:32.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.116+0000 7f4bbcb9d700 1 --2- 172.21.15.55:0/1986367169 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4ba4066f30 0x7f4ba40693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8001840 tx=0x7f4ba8009040 comp rx=0 tx=0).stop 2023-10-06T03:51:32.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.116+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/1986367169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bb81566e0 msgr2=0x7f4bb8154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:32.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.116+0000 7f4bbcb9d700 1 --2- 172.21.15.55:0/1986367169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bb81566e0 0x7f4bb8154aa0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f4bb000b610 tx=0x7f4bb0018000 comp rx=0 tx=0).stop 2023-10-06T03:51:32.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.116+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/1986367169 shutdown_connections 2023-10-06T03:51:32.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.116+0000 7f4bbcb9d700 1 --2- 172.21.15.55:0/1986367169 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4ba4066f30 0x7f4ba40693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:32.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.116+0000 7f4bbcb9d700 1 --2- 172.21.15.55:0/1986367169 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bb81566e0 0x7f4bb8154aa0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:32.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.116+0000 7f4bbcb9d700 1 --2- 172.21.15.55:0/1986367169 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bb80cb900 0x7f4bb80c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:32.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.116+0000 7f4bbcb9d700 1 --2- 172.21.15.55:0/1986367169 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bb8156310 0x7f4bb80c2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:32.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.116+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/1986367169 >> 172.21.15.55:0/1986367169 conn(0x7f4bb80cc1f0 msgr2=0x7f4bb806ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:32.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.116+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/1986367169 shutdown_connections 2023-10-06T03:51:32.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.116+0000 7f4bbcb9d700 1 -- 172.21.15.55:0/1986367169 wait complete. 2023-10-06T03:51:32.236 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:51:32.236 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T03:51:31.019177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":403,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":265,"ondisk_log_size":265,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2460956,"kb_used_data":13020,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752513764,"statfs":{"total":773094113280,"available":770574094336,"internally_reserved":0,"allocated":13332480,"data_stored":7401087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001735"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013294+0000","last_change":"2023-10-06T03:51:27.006190+0000","last_active":"2023-10-06T03:51:27.013294+0000","last_peered":"2023-10-06T03:51:27.013294+0000","last_clean":"2023-10-06T03:51:27.013294+0000","last_became_active":"2023-10-06T03:51:27.005921+0000","last_became_peered":"2023-10-06T03:51:27.005921+0000","last_unstale":"2023-10-06T03:51:27.013294+0000","last_undegraded":"2023-10-06T03:51:27.013294+0000","last_fullsized":"2023-10-06T03:51:27.013294+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:25:02.608098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.003571+0000","last_change":"2023-10-06T03:51:26.996209+0000","last_active":"2023-10-06T03:51:27.003571+0000","last_peered":"2023-10-06T03:51:27.003571+0000","last_clean":"2023-10-06T03:51:27.003571+0000","last_became_active":"2023-10-06T03:51:26.995937+0000","last_became_peered":"2023-10-06T03:51:26.995937+0000","last_unstale":"2023-10-06T03:51:27.003571+0000","last_undegraded":"2023-10-06T03:51:27.003571+0000","last_fullsized":"2023-10-06T03:51:27.003571+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:48.741807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013536+0000","last_change":"2023-10-06T03:51:27.006464+0000","last_active":"2023-10-06T03:51:27.013536+0000","last_peered":"2023-10-06T03:51:27.013536+0000","last_clean":"2023-10-06T03:51:27.013536+0000","last_became_active":"2023-10-06T03:51:27.006169+0000","last_became_peered":"2023-10-06T03:51:27.006169+0000","last_unstale":"2023-10-06T03:51:27.013536+0000","last_undegraded":"2023-10-06T03:51:27.013536+0000","last_fullsized":"2023-10-06T03:51:27.013536+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:00.627533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.020768+0000","last_change":"2023-10-06T03:51:27.016234+0000","last_active":"2023-10-06T03:51:27.020768+0000","last_peered":"2023-10-06T03:51:27.020768+0000","last_clean":"2023-10-06T03:51:27.020768+0000","last_became_active":"2023-10-06T03:51:27.015956+0000","last_became_peered":"2023-10-06T03:51:27.015956+0000","last_unstale":"2023-10-06T03:51:27.020768+0000","last_undegraded":"2023-10-06T03:51:27.020768+0000","last_fullsized":"2023-10-06T03:51:27.020768+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:15:26.862706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.018120+0000","last_change":"2023-10-06T03:51:27.018120+0000","last_active":"2023-10-06T03:51:27.018120+0000","last_peered":"2023-10-06T03:51:27.018120+0000","last_clean":"2023-10-06T03:51:27.018120+0000","last_became_active":"2023-10-06T03:51:27.017690+0000","last_became_peered":"2023-10-06T03:51:27.017690+0000","last_unstale":"2023-10-06T03:51:27.018120+0000","last_undegraded":"2023-10-06T03:51:27.018120+0000","last_fullsized":"2023-10-06T03:51:27.018120+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:12.907241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013715+0000","last_change":"2023-10-06T03:51:27.011182+0000","last_active":"2023-10-06T03:51:27.013715+0000","last_peered":"2023-10-06T03:51:27.013715+0000","last_clean":"2023-10-06T03:51:27.013715+0000","last_became_active":"2023-10-06T03:51:27.010891+0000","last_became_peered":"2023-10-06T03:51:27.010891+0000","last_unstale":"2023-10-06T03:51:27.013715+0000","last_undegraded":"2023-10-06T03:51:27.013715+0000","last_fullsized":"2023-10-06T03:51:27.013715+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:27.792100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013325+0000","last_change":"2023-10-06T03:51:27.010154+0000","last_active":"2023-10-06T03:51:27.013325+0000","last_peered":"2023-10-06T03:51:27.013325+0000","last_clean":"2023-10-06T03:51:27.013325+0000","last_became_active":"2023-10-06T03:51:27.009783+0000","last_became_peered":"2023-10-06T03:51:27.009783+0000","last_unstale":"2023-10-06T03:51:27.013325+0000","last_undegraded":"2023-10-06T03:51:27.013325+0000","last_fullsized":"2023-10-06T03:51:27.013325+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:34.306331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.020128+0000","last_change":"2023-10-06T03:51:27.020128+0000","last_active":"2023-10-06T03:51:27.020128+0000","last_peered":"2023-10-06T03:51:27.020128+0000","last_clean":"2023-10-06T03:51:27.020128+0000","last_became_active":"2023-10-06T03:51:27.019749+0000","last_became_peered":"2023-10-06T03:51:27.019749+0000","last_unstale":"2023-10-06T03:51:27.020128+0000","last_undegraded":"2023-10-06T03:51:27.020128+0000","last_fullsized":"2023-10-06T03:51:27.020128+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:56.744594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013291+0000","last_change":"2023-10-06T03:51:27.001564+0000","last_active":"2023-10-06T03:51:27.013291+0000","last_peered":"2023-10-06T03:51:27.013291+0000","last_clean":"2023-10-06T03:51:27.013291+0000","last_became_active":"2023-10-06T03:51:27.001307+0000","last_became_peered":"2023-10-06T03:51:27.001307+0000","last_unstale":"2023-10-06T03:51:27.013291+0000","last_undegraded":"2023-10-06T03:51:27.013291+0000","last_fullsized":"2023-10-06T03:51:27.013291+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:32.773546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.007075+0000","last_change":"2023-10-06T03:51:27.007075+0000","last_active":"2023-10-06T03:51:27.007075+0000","last_peered":"2023-10-06T03:51:27.007075+0000","last_clean":"2023-10-06T03:51:27.007075+0000","last_became_active":"2023-10-06T03:51:27.006753+0000","last_became_peered":"2023-10-06T03:51:27.006753+0000","last_unstale":"2023-10-06T03:51:27.007075+0000","last_undegraded":"2023-10-06T03:51:27.007075+0000","last_fullsized":"2023-10-06T03:51:27.007075+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:40:26.398839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.022844+0000","last_change":"2023-10-06T03:51:27.017654+0000","last_active":"2023-10-06T03:51:27.022844+0000","last_peered":"2023-10-06T03:51:27.022844+0000","last_clean":"2023-10-06T03:51:27.022844+0000","last_became_active":"2023-10-06T03:51:27.017368+0000","last_became_peered":"2023-10-06T03:51:27.017368+0000","last_unstale":"2023-10-06T03:51:27.022844+0000","last_undegraded":"2023-10-06T03:51:27.022844+0000","last_fullsized":"2023-10-06T03:51:27.022844+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:34.306331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.012351+0000","last_change":"2023-10-06T03:51:27.006545+0000","last_active":"2023-10-06T03:51:27.012351+0000","last_peered":"2023-10-06T03:51:27.012351+0000","last_clean":"2023-10-06T03:51:27.012351+0000","last_became_active":"2023-10-06T03:51:27.006286+0000","last_became_peered":"2023-10-06T03:51:27.006286+0000","last_unstale":"2023-10-06T03:51:27.012351+0000","last_undegraded":"2023-10-06T03:51:27.012351+0000","last_fullsized":"2023-10-06T03:51:27.012351+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:41.424643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.012817+0000","last_change":"2023-10-06T03:51:27.010450+0000","last_active":"2023-10-06T03:51:27.012817+0000","last_peered":"2023-10-06T03:51:27.012817+0000","last_clean":"2023-10-06T03:51:27.012817+0000","last_became_active":"2023-10-06T03:51:27.010197+0000","last_became_peered":"2023-10-06T03:51:27.010197+0000","last_unstale":"2023-10-06T03:51:27.012817+0000","last_undegraded":"2023-10-06T03:51:27.012817+0000","last_fullsized":"2023-10-06T03:51:27.012817+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:03.629521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.006748+0000","last_change":"2023-10-06T03:51:27.006748+0000","last_active":"2023-10-06T03:51:27.006748+0000","last_peered":"2023-10-06T03:51:27.006748+0000","last_clean":"2023-10-06T03:51:27.006748+0000","last_became_active":"2023-10-06T03:51:27.006490+0000","last_became_peered":"2023-10-06T03:51:27.006490+0000","last_unstale":"2023-10-06T03:51:27.006748+0000","last_undegraded":"2023-10-06T03:51:27.006748+0000","last_fullsized":"2023-10-06T03:51:27.006748+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:15:26.862706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.007133+0000","last_change":"2023-10-06T03:51:27.007133+0000","last_active":"2023-10-06T03:51:27.007133+0000","last_peered":"2023-10-06T03:51:27.007133+0000","last_clean":"2023-10-06T03:51:27.007133+0000","last_became_active":"2023-10-06T03:51:27.006798+0000","last_became_peered":"2023-10-06T03:51:27.006798+0000","last_unstale":"2023-10-06T03:51:27.007133+0000","last_undegraded":"2023-10-06T03:51:27.007133+0000","last_fullsized":"2023-10-06T03:51:27.007133+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:42.614149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.005273+0000","last_change":"2023-10-06T03:51:27.005273+0000","last_active":"2023-10-06T03:51:27.005273+0000","last_peered":"2023-10-06T03:51:27.005273+0000","last_clean":"2023-10-06T03:51:27.005273+0000","last_became_active":"2023-10-06T03:51:27.004992+0000","last_became_peered":"2023-10-06T03:51:27.004992+0000","last_unstale":"2023-10-06T03:51:27.005273+0000","last_undegraded":"2023-10-06T03:51:27.005273+0000","last_fullsized":"2023-10-06T03:51:27.005273+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:57.631308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013663+0000","last_change":"2023-10-06T03:51:27.006819+0000","last_active":"2023-10-06T03:51:27.013663+0000","last_peered":"2023-10-06T03:51:27.013663+0000","last_clean":"2023-10-06T03:51:27.013663+0000","last_became_active":"2023-10-06T03:51:27.006540+0000","last_became_peered":"2023-10-06T03:51:27.006540+0000","last_unstale":"2023-10-06T03:51:27.013663+0000","last_undegraded":"2023-10-06T03:51:27.013663+0000","last_fullsized":"2023-10-06T03:51:27.013663+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:45:32.787881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013932+0000","last_change":"2023-10-06T03:51:27.011726+0000","last_active":"2023-10-06T03:51:27.013932+0000","last_peered":"2023-10-06T03:51:27.013932+0000","last_clean":"2023-10-06T03:51:27.013932+0000","last_became_active":"2023-10-06T03:51:27.011279+0000","last_became_peered":"2023-10-06T03:51:27.011279+0000","last_unstale":"2023-10-06T03:51:27.013932+0000","last_undegraded":"2023-10-06T03:51:27.013932+0000","last_fullsized":"2023-10-06T03:51:27.013932+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:57.199095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.011302+0000","last_change":"2023-10-06T03:51:27.000180+0000","last_active":"2023-10-06T03:51:27.011302+0000","last_peered":"2023-10-06T03:51:27.011302+0000","last_clean":"2023-10-06T03:51:27.011302+0000","last_became_active":"2023-10-06T03:51:26.999804+0000","last_became_peered":"2023-10-06T03:51:26.999804+0000","last_unstale":"2023-10-06T03:51:27.011302+0000","last_undegraded":"2023-10-06T03:51:27.011302+0000","last_fullsized":"2023-10-06T03:51:27.011302+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:59.533661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.006117+0000","last_change":"2023-10-06T03:51:26.998584+0000","last_active":"2023-10-06T03:51:27.006117+0000","last_peered":"2023-10-06T03:51:27.006117+0000","last_clean":"2023-10-06T03:51:27.006117+0000","last_became_active":"2023-10-06T03:51:26.998320+0000","last_became_peered":"2023-10-06T03:51:26.998320+0000","last_unstale":"2023-10-06T03:51:27.006117+0000","last_undegraded":"2023-10-06T03:51:27.006117+0000","last_fullsized":"2023-10-06T03:51:27.006117+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:05:27.956599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.005270+0000","last_change":"2023-10-06T03:51:27.005270+0000","last_active":"2023-10-06T03:51:27.005270+0000","last_peered":"2023-10-06T03:51:27.005270+0000","last_clean":"2023-10-06T03:51:27.005270+0000","last_became_active":"2023-10-06T03:51:27.005021+0000","last_became_peered":"2023-10-06T03:51:27.005021+0000","last_unstale":"2023-10-06T03:51:27.005270+0000","last_undegraded":"2023-10-06T03:51:27.005270+0000","last_fullsized":"2023-10-06T03:51:27.005270+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:30.255876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.014371+0000","last_change":"2023-10-06T03:51:27.014371+0000","last_active":"2023-10-06T03:51:27.014371+0000","last_peered":"2023-10-06T03:51:27.014371+0000","last_clean":"2023-10-06T03:51:27.014371+0000","last_became_active":"2023-10-06T03:51:27.014112+0000","last_became_peered":"2023-10-06T03:51:27.014112+0000","last_unstale":"2023-10-06T03:51:27.014371+0000","last_undegraded":"2023-10-06T03:51:27.014371+0000","last_fullsized":"2023-10-06T03:51:27.014371+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:09:44.298593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013264+0000","last_change":"2023-10-06T03:51:27.005697+0000","last_active":"2023-10-06T03:51:27.013264+0000","last_peered":"2023-10-06T03:51:27.013264+0000","last_clean":"2023-10-06T03:51:27.013264+0000","last_became_active":"2023-10-06T03:51:27.005405+0000","last_became_peered":"2023-10-06T03:51:27.005405+0000","last_unstale":"2023-10-06T03:51:27.013264+0000","last_undegraded":"2023-10-06T03:51:27.013264+0000","last_fullsized":"2023-10-06T03:51:27.013264+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:15.743634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.018302+0000","last_change":"2023-10-06T03:51:27.018302+0000","last_active":"2023-10-06T03:51:27.018302+0000","last_peered":"2023-10-06T03:51:27.018302+0000","last_clean":"2023-10-06T03:51:27.018302+0000","last_became_active":"2023-10-06T03:51:27.018042+0000","last_became_peered":"2023-10-06T03:51:27.018042+0000","last_unstale":"2023-10-06T03:51:27.018302+0000","last_undegraded":"2023-10-06T03:51:27.018302+0000","last_fullsized":"2023-10-06T03:51:27.018302+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:46.761766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.012409+0000","last_change":"2023-10-06T03:51:27.008627+0000","last_active":"2023-10-06T03:51:27.012409+0000","last_peered":"2023-10-06T03:51:27.012409+0000","last_clean":"2023-10-06T03:51:27.012409+0000","last_became_active":"2023-10-06T03:51:27.007558+0000","last_became_peered":"2023-10-06T03:51:27.007558+0000","last_unstale":"2023-10-06T03:51:27.012409+0000","last_undegraded":"2023-10-06T03:51:27.012409+0000","last_fullsized":"2023-10-06T03:51:27.012409+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:22.331588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.015368+0000","last_change":"2023-10-06T03:51:27.012298+0000","last_active":"2023-10-06T03:51:27.015368+0000","last_peered":"2023-10-06T03:51:27.015368+0000","last_clean":"2023-10-06T03:51:27.015368+0000","last_became_active":"2023-10-06T03:51:27.012033+0000","last_became_peered":"2023-10-06T03:51:27.012033+0000","last_unstale":"2023-10-06T03:51:27.015368+0000","last_undegraded":"2023-10-06T03:51:27.015368+0000","last_fullsized":"2023-10-06T03:51:27.015368+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:27.590794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.011171+0000","last_change":"2023-10-06T03:51:27.004430+0000","last_active":"2023-10-06T03:51:27.011171+0000","last_peered":"2023-10-06T03:51:27.011171+0000","last_clean":"2023-10-06T03:51:27.011171+0000","last_became_active":"2023-10-06T03:51:27.003746+0000","last_became_peered":"2023-10-06T03:51:27.003746+0000","last_unstale":"2023-10-06T03:51:27.011171+0000","last_undegraded":"2023-10-06T03:51:27.011171+0000","last_fullsized":"2023-10-06T03:51:27.011171+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:32.773546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.015080+0000","last_change":"2023-10-06T03:51:27.008006+0000","last_active":"2023-10-06T03:51:27.015080+0000","last_peered":"2023-10-06T03:51:27.015080+0000","last_clean":"2023-10-06T03:51:27.015080+0000","last_became_active":"2023-10-06T03:51:27.007558+0000","last_became_peered":"2023-10-06T03:51:27.007558+0000","last_unstale":"2023-10-06T03:51:27.015080+0000","last_undegraded":"2023-10-06T03:51:27.015080+0000","last_fullsized":"2023-10-06T03:51:27.015080+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:46.761766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.004263+0000","last_change":"2023-10-06T03:51:27.004263+0000","last_active":"2023-10-06T03:51:27.004263+0000","last_peered":"2023-10-06T03:51:27.004263+0000","last_clean":"2023-10-06T03:51:27.004263+0000","last_became_active":"2023-10-06T03:51:27.003843+0000","last_became_peered":"2023-10-06T03:51:27.003843+0000","last_unstale":"2023-10-06T03:51:27.004263+0000","last_undegraded":"2023-10-06T03:51:27.004263+0000","last_fullsized":"2023-10-06T03:51:27.004263+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:24.850221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.012317+0000","last_change":"2023-10-06T03:51:27.007144+0000","last_active":"2023-10-06T03:51:27.012317+0000","last_peered":"2023-10-06T03:51:27.012317+0000","last_clean":"2023-10-06T03:51:27.012317+0000","last_became_active":"2023-10-06T03:51:27.006892+0000","last_became_peered":"2023-10-06T03:51:27.006892+0000","last_unstale":"2023-10-06T03:51:27.012317+0000","last_undegraded":"2023-10-06T03:51:27.012317+0000","last_fullsized":"2023-10-06T03:51:27.012317+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:05.693451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.012645+0000","last_change":"2023-10-06T03:51:26.998372+0000","last_active":"2023-10-06T03:51:27.012645+0000","last_peered":"2023-10-06T03:51:27.012645+0000","last_clean":"2023-10-06T03:51:27.012645+0000","last_became_active":"2023-10-06T03:51:26.998106+0000","last_became_peered":"2023-10-06T03:51:26.998106+0000","last_unstale":"2023-10-06T03:51:27.012645+0000","last_undegraded":"2023-10-06T03:51:27.012645+0000","last_fullsized":"2023-10-06T03:51:27.012645+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:58.329261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.009277+0000","last_change":"2023-10-06T03:51:27.009277+0000","last_active":"2023-10-06T03:51:27.009277+0000","last_peered":"2023-10-06T03:51:27.009277+0000","last_clean":"2023-10-06T03:51:27.009277+0000","last_became_active":"2023-10-06T03:51:27.009013+0000","last_became_peered":"2023-10-06T03:51:27.009013+0000","last_unstale":"2023-10-06T03:51:27.009277+0000","last_undegraded":"2023-10-06T03:51:27.009277+0000","last_fullsized":"2023-10-06T03:51:27.009277+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:35:04.371568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.999127+0000","last_change":"2023-10-06T03:51:26.996011+0000","last_active":"2023-10-06T03:51:26.999127+0000","last_peered":"2023-10-06T03:51:26.999127+0000","last_clean":"2023-10-06T03:51:26.999127+0000","last_became_active":"2023-10-06T03:51:26.995494+0000","last_became_peered":"2023-10-06T03:51:26.995494+0000","last_unstale":"2023-10-06T03:51:26.999127+0000","last_undegraded":"2023-10-06T03:51:26.999127+0000","last_fullsized":"2023-10-06T03:51:26.999127+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:21:12.935675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"122'4","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.022421+0000","last_change":"2023-10-06T03:51:26.008165+0000","last_active":"2023-10-06T03:51:27.022421+0000","last_peered":"2023-10-06T03:51:27.022421+0000","last_clean":"2023-10-06T03:51:27.022421+0000","last_became_active":"2023-10-06T03:51:26.007909+0000","last_became_peered":"2023-10-06T03:51:26.007909+0000","last_unstale":"2023-10-06T03:51:27.022421+0000","last_undegraded":"2023-10-06T03:51:27.022421+0000","last_fullsized":"2023-10-06T03:51:27.022421+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:13.022473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.024552+0000","last_change":"2023-10-06T03:51:27.024552+0000","last_active":"2023-10-06T03:51:27.024552+0000","last_peered":"2023-10-06T03:51:27.024552+0000","last_clean":"2023-10-06T03:51:27.024552+0000","last_became_active":"2023-10-06T03:51:27.024281+0000","last_became_peered":"2023-10-06T03:51:27.024281+0000","last_unstale":"2023-10-06T03:51:27.024552+0000","last_undegraded":"2023-10-06T03:51:27.024552+0000","last_fullsized":"2023-10-06T03:51:27.024552+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:03.655845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"122'1","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.002201+0000","last_change":"2023-10-06T03:51:26.009666+0000","last_active":"2023-10-06T03:51:27.002201+0000","last_peered":"2023-10-06T03:51:27.002201+0000","last_clean":"2023-10-06T03:51:27.002201+0000","last_became_active":"2023-10-06T03:51:26.009402+0000","last_became_peered":"2023-10-06T03:51:26.009402+0000","last_unstale":"2023-10-06T03:51:27.002201+0000","last_undegraded":"2023-10-06T03:51:27.002201+0000","last_fullsized":"2023-10-06T03:51:27.002201+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:59:49.724213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.023068+0000","last_change":"2023-10-06T03:51:27.017913+0000","last_active":"2023-10-06T03:51:27.023068+0000","last_peered":"2023-10-06T03:51:27.023068+0000","last_clean":"2023-10-06T03:51:27.023068+0000","last_became_active":"2023-10-06T03:51:27.017671+0000","last_became_peered":"2023-10-06T03:51:27.017671+0000","last_unstale":"2023-10-06T03:51:27.023068+0000","last_undegraded":"2023-10-06T03:51:27.023068+0000","last_fullsized":"2023-10-06T03:51:27.023068+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:47.161538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"122'2","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.022882+0000","last_change":"2023-10-06T03:51:26.005937+0000","last_active":"2023-10-06T03:51:27.022882+0000","last_peered":"2023-10-06T03:51:27.022882+0000","last_clean":"2023-10-06T03:51:27.022882+0000","last_became_active":"2023-10-06T03:51:26.005629+0000","last_became_peered":"2023-10-06T03:51:26.005629+0000","last_unstale":"2023-10-06T03:51:27.022882+0000","last_undegraded":"2023-10-06T03:51:27.022882+0000","last_fullsized":"2023-10-06T03:51:27.022882+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:53.125506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.017681+0000","last_change":"2023-10-06T03:51:27.012937+0000","last_active":"2023-10-06T03:51:27.017681+0000","last_peered":"2023-10-06T03:51:27.017681+0000","last_clean":"2023-10-06T03:51:27.017681+0000","last_became_active":"2023-10-06T03:51:27.012633+0000","last_became_peered":"2023-10-06T03:51:27.012633+0000","last_unstale":"2023-10-06T03:51:27.017681+0000","last_undegraded":"2023-10-06T03:51:27.017681+0000","last_fullsized":"2023-10-06T03:51:27.017681+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:57.199095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"122'2","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.023461+0000","last_change":"2023-10-06T03:51:26.002695+0000","last_active":"2023-10-06T03:51:27.023461+0000","last_peered":"2023-10-06T03:51:27.023461+0000","last_clean":"2023-10-06T03:51:27.023461+0000","last_became_active":"2023-10-06T03:51:26.002404+0000","last_became_peered":"2023-10-06T03:51:26.002404+0000","last_unstale":"2023-10-06T03:51:27.023461+0000","last_undegraded":"2023-10-06T03:51:27.023461+0000","last_fullsized":"2023-10-06T03:51:27.023461+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:45.196305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.022170+0000","last_change":"2023-10-06T03:51:27.020654+0000","last_active":"2023-10-06T03:51:27.022170+0000","last_peered":"2023-10-06T03:51:27.022170+0000","last_clean":"2023-10-06T03:51:27.022170+0000","last_became_active":"2023-10-06T03:51:27.020239+0000","last_became_peered":"2023-10-06T03:51:27.020239+0000","last_unstale":"2023-10-06T03:51:27.022170+0000","last_undegraded":"2023-10-06T03:51:27.022170+0000","last_fullsized":"2023-10-06T03:51:27.022170+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:57.631308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"122'1","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.993736+0000","last_change":"2023-10-06T03:51:26.010740+0000","last_active":"2023-10-06T03:51:26.993736+0000","last_peered":"2023-10-06T03:51:26.993736+0000","last_clean":"2023-10-06T03:51:26.993736+0000","last_became_active":"2023-10-06T03:51:26.010468+0000","last_became_peered":"2023-10-06T03:51:26.010468+0000","last_unstale":"2023-10-06T03:51:26.993736+0000","last_undegraded":"2023-10-06T03:51:26.993736+0000","last_fullsized":"2023-10-06T03:51:26.993736+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:11.896681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.011328+0000","last_change":"2023-10-06T03:51:27.000611+0000","last_active":"2023-10-06T03:51:27.011328+0000","last_peered":"2023-10-06T03:51:27.011328+0000","last_clean":"2023-10-06T03:51:27.011328+0000","last_became_active":"2023-10-06T03:51:27.000321+0000","last_became_peered":"2023-10-06T03:51:27.000321+0000","last_unstale":"2023-10-06T03:51:27.011328+0000","last_undegraded":"2023-10-06T03:51:27.011328+0000","last_fullsized":"2023-10-06T03:51:27.011328+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:11.602376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"122'1","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.020512+0000","last_change":"2023-10-06T03:51:26.002221+0000","last_active":"2023-10-06T03:51:27.020512+0000","last_peered":"2023-10-06T03:51:27.020512+0000","last_clean":"2023-10-06T03:51:27.020512+0000","last_became_active":"2023-10-06T03:51:26.000862+0000","last_became_peered":"2023-10-06T03:51:26.000862+0000","last_unstale":"2023-10-06T03:51:27.020512+0000","last_undegraded":"2023-10-06T03:51:27.020512+0000","last_fullsized":"2023-10-06T03:51:27.020512+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:26.361323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.020513+0000","last_change":"2023-10-06T03:51:27.013845+0000","last_active":"2023-10-06T03:51:27.020513+0000","last_peered":"2023-10-06T03:51:27.020513+0000","last_clean":"2023-10-06T03:51:27.020513+0000","last_became_active":"2023-10-06T03:51:27.013561+0000","last_became_peered":"2023-10-06T03:51:27.013561+0000","last_unstale":"2023-10-06T03:51:27.020513+0000","last_undegraded":"2023-10-06T03:51:27.020513+0000","last_fullsized":"2023-10-06T03:51:27.020513+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:07:14.033033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.993169+0000","last_change":"2023-10-06T03:51:26.001908+0000","last_active":"2023-10-06T03:51:26.993169+0000","last_peered":"2023-10-06T03:51:26.993169+0000","last_clean":"2023-10-06T03:51:26.993169+0000","last_became_active":"2023-10-06T03:51:26.001569+0000","last_became_peered":"2023-10-06T03:51:26.001569+0000","last_unstale":"2023-10-06T03:51:26.993169+0000","last_undegraded":"2023-10-06T03:51:26.993169+0000","last_fullsized":"2023-10-06T03:51:26.993169+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:39.038453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.012964+0000","last_change":"2023-10-06T03:51:27.009537+0000","last_active":"2023-10-06T03:51:27.012964+0000","last_peered":"2023-10-06T03:51:27.012964+0000","last_clean":"2023-10-06T03:51:27.012964+0000","last_became_active":"2023-10-06T03:51:27.009059+0000","last_became_peered":"2023-10-06T03:51:27.009059+0000","last_unstale":"2023-10-06T03:51:27.012964+0000","last_undegraded":"2023-10-06T03:51:27.012964+0000","last_fullsized":"2023-10-06T03:51:27.012964+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:44.010040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.991960+0000","last_change":"2023-10-06T03:51:26.005255+0000","last_active":"2023-10-06T03:51:26.991960+0000","last_peered":"2023-10-06T03:51:26.991960+0000","last_clean":"2023-10-06T03:51:26.991960+0000","last_became_active":"2023-10-06T03:51:26.004972+0000","last_became_peered":"2023-10-06T03:51:26.004972+0000","last_unstale":"2023-10-06T03:51:26.991960+0000","last_undegraded":"2023-10-06T03:51:26.991960+0000","last_fullsized":"2023-10-06T03:51:26.991960+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:25:56.188535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.999108+0000","last_change":"2023-10-06T03:51:26.995751+0000","last_active":"2023-10-06T03:51:26.999108+0000","last_peered":"2023-10-06T03:51:26.999108+0000","last_clean":"2023-10-06T03:51:26.999108+0000","last_became_active":"2023-10-06T03:51:26.995120+0000","last_became_peered":"2023-10-06T03:51:26.995120+0000","last_unstale":"2023-10-06T03:51:26.999108+0000","last_undegraded":"2023-10-06T03:51:26.999108+0000","last_fullsized":"2023-10-06T03:51:26.999108+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:30.207842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.992013+0000","last_change":"2023-10-06T03:51:26.005874+0000","last_active":"2023-10-06T03:51:26.992013+0000","last_peered":"2023-10-06T03:51:26.992013+0000","last_clean":"2023-10-06T03:51:26.992013+0000","last_became_active":"2023-10-06T03:51:26.005545+0000","last_became_peered":"2023-10-06T03:51:26.005545+0000","last_unstale":"2023-10-06T03:51:26.992013+0000","last_undegraded":"2023-10-06T03:51:26.992013+0000","last_fullsized":"2023-10-06T03:51:26.992013+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:27.433759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.017351+0000","last_change":"2023-10-06T03:51:27.017351+0000","last_active":"2023-10-06T03:51:27.017351+0000","last_peered":"2023-10-06T03:51:27.017351+0000","last_clean":"2023-10-06T03:51:27.017351+0000","last_became_active":"2023-10-06T03:51:27.017086+0000","last_became_peered":"2023-10-06T03:51:27.017086+0000","last_unstale":"2023-10-06T03:51:27.017351+0000","last_undegraded":"2023-10-06T03:51:27.017351+0000","last_fullsized":"2023-10-06T03:51:27.017351+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:22:48.578436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.988665+0000","last_change":"2023-10-06T03:51:26.005721+0000","last_active":"2023-10-06T03:51:26.988665+0000","last_peered":"2023-10-06T03:51:26.988665+0000","last_clean":"2023-10-06T03:51:26.988665+0000","last_became_active":"2023-10-06T03:51:26.005442+0000","last_became_peered":"2023-10-06T03:51:26.005442+0000","last_unstale":"2023-10-06T03:51:26.988665+0000","last_undegraded":"2023-10-06T03:51:26.988665+0000","last_fullsized":"2023-10-06T03:51:26.988665+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:16:38.387346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.021443+0000","last_change":"2023-10-06T03:51:27.021443+0000","last_active":"2023-10-06T03:51:27.021443+0000","last_peered":"2023-10-06T03:51:27.021443+0000","last_clean":"2023-10-06T03:51:27.021443+0000","last_became_active":"2023-10-06T03:51:27.021215+0000","last_became_peered":"2023-10-06T03:51:27.021215+0000","last_unstale":"2023-10-06T03:51:27.021443+0000","last_undegraded":"2023-10-06T03:51:27.021443+0000","last_fullsized":"2023-10-06T03:51:27.021443+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:04.260304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"122'3","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.015095+0000","last_change":"2023-10-06T03:51:26.009548+0000","last_active":"2023-10-06T03:51:27.015095+0000","last_peered":"2023-10-06T03:51:27.015095+0000","last_clean":"2023-10-06T03:51:27.015095+0000","last_became_active":"2023-10-06T03:51:26.009288+0000","last_became_peered":"2023-10-06T03:51:26.009288+0000","last_unstale":"2023-10-06T03:51:27.015095+0000","last_undegraded":"2023-10-06T03:51:27.015095+0000","last_fullsized":"2023-10-06T03:51:27.015095+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:47.731247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.010763+0000","last_change":"2023-10-06T03:51:27.010763+0000","last_active":"2023-10-06T03:51:27.010763+0000","last_peered":"2023-10-06T03:51:27.010763+0000","last_clean":"2023-10-06T03:51:27.010763+0000","last_became_active":"2023-10-06T03:51:27.007859+0000","last_became_peered":"2023-10-06T03:51:27.007859+0000","last_unstale":"2023-10-06T03:51:27.010763+0000","last_undegraded":"2023-10-06T03:51:27.010763+0000","last_fullsized":"2023-10-06T03:51:27.010763+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:04:04.078671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"122'6","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.026061+0000","last_change":"2023-10-06T03:51:25.993345+0000","last_active":"2023-10-06T03:51:27.026061+0000","last_peered":"2023-10-06T03:51:27.026061+0000","last_clean":"2023-10-06T03:51:27.026061+0000","last_became_active":"2023-10-06T03:51:25.993067+0000","last_became_peered":"2023-10-06T03:51:25.993067+0000","last_unstale":"2023-10-06T03:51:27.026061+0000","last_undegraded":"2023-10-06T03:51:27.026061+0000","last_fullsized":"2023-10-06T03:51:27.026061+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:53.901794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.018395+0000","last_change":"2023-10-06T03:51:27.013055+0000","last_active":"2023-10-06T03:51:27.018395+0000","last_peered":"2023-10-06T03:51:27.018395+0000","last_clean":"2023-10-06T03:51:27.018395+0000","last_became_active":"2023-10-06T03:51:27.012794+0000","last_became_peered":"2023-10-06T03:51:27.012794+0000","last_unstale":"2023-10-06T03:51:27.018395+0000","last_undegraded":"2023-10-06T03:51:27.018395+0000","last_fullsized":"2023-10-06T03:51:27.018395+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:46.206866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"122'2","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.026060+0000","last_change":"2023-10-06T03:51:26.012192+0000","last_active":"2023-10-06T03:51:27.026060+0000","last_peered":"2023-10-06T03:51:27.026060+0000","last_clean":"2023-10-06T03:51:27.026060+0000","last_became_active":"2023-10-06T03:51:26.011853+0000","last_became_peered":"2023-10-06T03:51:26.011853+0000","last_unstale":"2023-10-06T03:51:27.026060+0000","last_undegraded":"2023-10-06T03:51:27.026060+0000","last_fullsized":"2023-10-06T03:51:27.026060+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:57.318701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013780+0000","last_change":"2023-10-06T03:51:27.003308+0000","last_active":"2023-10-06T03:51:27.013780+0000","last_peered":"2023-10-06T03:51:27.013780+0000","last_clean":"2023-10-06T03:51:27.013780+0000","last_became_active":"2023-10-06T03:51:27.003059+0000","last_became_peered":"2023-10-06T03:51:27.003059+0000","last_unstale":"2023-10-06T03:51:27.013780+0000","last_undegraded":"2023-10-06T03:51:27.013780+0000","last_fullsized":"2023-10-06T03:51:27.013780+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:01.933888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.991947+0000","last_change":"2023-10-06T03:51:26.007819+0000","last_active":"2023-10-06T03:51:26.991947+0000","last_peered":"2023-10-06T03:51:26.991947+0000","last_clean":"2023-10-06T03:51:26.991947+0000","last_became_active":"2023-10-06T03:51:26.007539+0000","last_became_peered":"2023-10-06T03:51:26.007539+0000","last_unstale":"2023-10-06T03:51:26.991947+0000","last_undegraded":"2023-10-06T03:51:26.991947+0000","last_fullsized":"2023-10-06T03:51:26.991947+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:00.923327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.015236+0000","last_change":"2023-10-06T03:51:27.015236+0000","last_active":"2023-10-06T03:51:27.015236+0000","last_peered":"2023-10-06T03:51:27.015236+0000","last_clean":"2023-10-06T03:51:27.015236+0000","last_became_active":"2023-10-06T03:51:27.014972+0000","last_became_peered":"2023-10-06T03:51:27.014972+0000","last_unstale":"2023-10-06T03:51:27.015236+0000","last_undegraded":"2023-10-06T03:51:27.015236+0000","last_fullsized":"2023-10-06T03:51:27.015236+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:15:03.633065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.008292+0000","last_change":"2023-10-06T03:51:26.007085+0000","last_active":"2023-10-06T03:51:26.008292+0000","last_peered":"2023-10-06T03:51:26.008292+0000","last_clean":"2023-10-06T03:51:26.008292+0000","last_became_active":"2023-10-06T03:51:26.006801+0000","last_became_peered":"2023-10-06T03:51:26.006801+0000","last_unstale":"2023-10-06T03:51:26.008292+0000","last_undegraded":"2023-10-06T03:51:26.008292+0000","last_fullsized":"2023-10-06T03:51:26.008292+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:37.732660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013644+0000","last_change":"2023-10-06T03:51:27.006753+0000","last_active":"2023-10-06T03:51:27.013644+0000","last_peered":"2023-10-06T03:51:27.013644+0000","last_clean":"2023-10-06T03:51:27.013644+0000","last_became_active":"2023-10-06T03:51:27.006454+0000","last_became_peered":"2023-10-06T03:51:27.006454+0000","last_unstale":"2023-10-06T03:51:27.013644+0000","last_undegraded":"2023-10-06T03:51:27.013644+0000","last_fullsized":"2023-10-06T03:51:27.013644+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:21.042231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"122'8","reported_seq":18,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.023713+0000","last_change":"2023-10-06T03:51:26.001111+0000","last_active":"2023-10-06T03:51:27.023713+0000","last_peered":"2023-10-06T03:51:27.023713+0000","last_clean":"2023-10-06T03:51:27.023713+0000","last_became_active":"2023-10-06T03:51:26.000829+0000","last_became_peered":"2023-10-06T03:51:26.000829+0000","last_unstale":"2023-10-06T03:51:27.023713+0000","last_undegraded":"2023-10-06T03:51:27.023713+0000","last_fullsized":"2023-10-06T03:51:27.023713+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:51.309439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013332+0000","last_change":"2023-10-06T03:51:27.006156+0000","last_active":"2023-10-06T03:51:27.013332+0000","last_peered":"2023-10-06T03:51:27.013332+0000","last_clean":"2023-10-06T03:51:27.013332+0000","last_became_active":"2023-10-06T03:51:27.005882+0000","last_became_peered":"2023-10-06T03:51:27.005882+0000","last_unstale":"2023-10-06T03:51:27.013332+0000","last_undegraded":"2023-10-06T03:51:27.013332+0000","last_fullsized":"2023-10-06T03:51:27.013332+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:45:48.236512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"122'2","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.025955+0000","last_change":"2023-10-06T03:51:26.010112+0000","last_active":"2023-10-06T03:51:27.025955+0000","last_peered":"2023-10-06T03:51:27.025955+0000","last_clean":"2023-10-06T03:51:27.025955+0000","last_became_active":"2023-10-06T03:51:26.009797+0000","last_became_peered":"2023-10-06T03:51:26.009797+0000","last_unstale":"2023-10-06T03:51:27.025955+0000","last_undegraded":"2023-10-06T03:51:27.025955+0000","last_fullsized":"2023-10-06T03:51:27.025955+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:20:18.194158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.015315+0000","last_change":"2023-10-06T03:51:27.009587+0000","last_active":"2023-10-06T03:51:27.015315+0000","last_peered":"2023-10-06T03:51:27.015315+0000","last_clean":"2023-10-06T03:51:27.015315+0000","last_became_active":"2023-10-06T03:51:27.009315+0000","last_became_peered":"2023-10-06T03:51:27.009315+0000","last_unstale":"2023-10-06T03:51:27.015315+0000","last_undegraded":"2023-10-06T03:51:27.015315+0000","last_fullsized":"2023-10-06T03:51:27.015315+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:41:27.069885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.988842+0000","last_change":"2023-10-06T03:51:26.005001+0000","last_active":"2023-10-06T03:51:26.988842+0000","last_peered":"2023-10-06T03:51:26.988842+0000","last_clean":"2023-10-06T03:51:26.988842+0000","last_became_active":"2023-10-06T03:51:26.004721+0000","last_became_peered":"2023-10-06T03:51:26.004721+0000","last_unstale":"2023-10-06T03:51:26.988842+0000","last_undegraded":"2023-10-06T03:51:26.988842+0000","last_fullsized":"2023-10-06T03:51:26.988842+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:25.341432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.001829+0000","last_change":"2023-10-06T03:51:26.999323+0000","last_active":"2023-10-06T03:51:27.001829+0000","last_peered":"2023-10-06T03:51:27.001829+0000","last_clean":"2023-10-06T03:51:27.001829+0000","last_became_active":"2023-10-06T03:51:26.999068+0000","last_became_peered":"2023-10-06T03:51:26.999068+0000","last_unstale":"2023-10-06T03:51:27.001829+0000","last_undegraded":"2023-10-06T03:51:27.001829+0000","last_fullsized":"2023-10-06T03:51:27.001829+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:52:20.239138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.993830+0000","last_change":"2023-10-06T03:51:26.003558+0000","last_active":"2023-10-06T03:51:26.993830+0000","last_peered":"2023-10-06T03:51:26.993830+0000","last_clean":"2023-10-06T03:51:26.993830+0000","last_became_active":"2023-10-06T03:51:26.003266+0000","last_became_peered":"2023-10-06T03:51:26.003266+0000","last_unstale":"2023-10-06T03:51:26.993830+0000","last_undegraded":"2023-10-06T03:51:26.993830+0000","last_fullsized":"2023-10-06T03:51:26.993830+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:35:03.361008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.017661+0000","last_change":"2023-10-06T03:51:27.011649+0000","last_active":"2023-10-06T03:51:27.017661+0000","last_peered":"2023-10-06T03:51:27.017661+0000","last_clean":"2023-10-06T03:51:27.017661+0000","last_became_active":"2023-10-06T03:51:27.011353+0000","last_became_peered":"2023-10-06T03:51:27.011353+0000","last_unstale":"2023-10-06T03:51:27.017661+0000","last_undegraded":"2023-10-06T03:51:27.017661+0000","last_fullsized":"2023-10-06T03:51:27.017661+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:31:21.112082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.008342+0000","last_change":"2023-10-06T03:51:26.005561+0000","last_active":"2023-10-06T03:51:26.008342+0000","last_peered":"2023-10-06T03:51:26.008342+0000","last_clean":"2023-10-06T03:51:26.008342+0000","last_became_active":"2023-10-06T03:51:26.005298+0000","last_became_peered":"2023-10-06T03:51:26.005298+0000","last_unstale":"2023-10-06T03:51:26.008342+0000","last_undegraded":"2023-10-06T03:51:26.008342+0000","last_fullsized":"2023-10-06T03:51:26.008342+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:47:35.317997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.020831+0000","last_change":"2023-10-06T03:51:27.020831+0000","last_active":"2023-10-06T03:51:27.020831+0000","last_peered":"2023-10-06T03:51:27.020831+0000","last_clean":"2023-10-06T03:51:27.020831+0000","last_became_active":"2023-10-06T03:51:27.020577+0000","last_became_peered":"2023-10-06T03:51:27.020577+0000","last_unstale":"2023-10-06T03:51:27.020831+0000","last_undegraded":"2023-10-06T03:51:27.020831+0000","last_fullsized":"2023-10-06T03:51:27.020831+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:54.912355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.991247+0000","last_change":"2023-10-06T03:51:26.004369+0000","last_active":"2023-10-06T03:51:26.991247+0000","last_peered":"2023-10-06T03:51:26.991247+0000","last_clean":"2023-10-06T03:51:26.991247+0000","last_became_active":"2023-10-06T03:51:26.004058+0000","last_became_peered":"2023-10-06T03:51:26.004058+0000","last_unstale":"2023-10-06T03:51:26.991247+0000","last_undegraded":"2023-10-06T03:51:26.991247+0000","last_fullsized":"2023-10-06T03:51:26.991247+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:10:19.103441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.024001+0000","last_change":"2023-10-06T03:51:27.024001+0000","last_active":"2023-10-06T03:51:27.024001+0000","last_peered":"2023-10-06T03:51:27.024001+0000","last_clean":"2023-10-06T03:51:27.024001+0000","last_became_active":"2023-10-06T03:51:27.023601+0000","last_became_peered":"2023-10-06T03:51:27.023601+0000","last_unstale":"2023-10-06T03:51:27.024001+0000","last_undegraded":"2023-10-06T03:51:27.024001+0000","last_fullsized":"2023-10-06T03:51:27.024001+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:48:04.511512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.013273+0000","last_change":"2023-10-06T03:51:26.007697+0000","last_active":"2023-10-06T03:51:26.013273+0000","last_peered":"2023-10-06T03:51:26.013273+0000","last_clean":"2023-10-06T03:51:26.013273+0000","last_became_active":"2023-10-06T03:51:26.007431+0000","last_became_peered":"2023-10-06T03:51:26.007431+0000","last_unstale":"2023-10-06T03:51:26.013273+0000","last_undegraded":"2023-10-06T03:51:26.013273+0000","last_fullsized":"2023-10-06T03:51:26.013273+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:47.825844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.007817+0000","last_change":"2023-10-06T03:51:27.007817+0000","last_active":"2023-10-06T03:51:27.007817+0000","last_peered":"2023-10-06T03:51:27.007817+0000","last_clean":"2023-10-06T03:51:27.007817+0000","last_became_active":"2023-10-06T03:51:27.007475+0000","last_became_peered":"2023-10-06T03:51:27.007475+0000","last_unstale":"2023-10-06T03:51:27.007817+0000","last_undegraded":"2023-10-06T03:51:27.007817+0000","last_fullsized":"2023-10-06T03:51:27.007817+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:12:03.629521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.993212+0000","last_change":"2023-10-06T03:51:26.002917+0000","last_active":"2023-10-06T03:51:26.993212+0000","last_peered":"2023-10-06T03:51:26.993212+0000","last_clean":"2023-10-06T03:51:26.993212+0000","last_became_active":"2023-10-06T03:51:26.002615+0000","last_became_peered":"2023-10-06T03:51:26.002615+0000","last_unstale":"2023-10-06T03:51:26.993212+0000","last_undegraded":"2023-10-06T03:51:26.993212+0000","last_fullsized":"2023-10-06T03:51:26.993212+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:29.245315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.015455+0000","last_change":"2023-10-06T03:51:27.005445+0000","last_active":"2023-10-06T03:51:27.015455+0000","last_peered":"2023-10-06T03:51:27.015455+0000","last_clean":"2023-10-06T03:51:27.015455+0000","last_became_active":"2023-10-06T03:51:27.005077+0000","last_became_peered":"2023-10-06T03:51:27.005077+0000","last_unstale":"2023-10-06T03:51:27.015455+0000","last_undegraded":"2023-10-06T03:51:27.015455+0000","last_fullsized":"2023-10-06T03:51:27.015455+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:45:01.933888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.008344+0000","last_change":"2023-10-06T03:51:26.007078+0000","last_active":"2023-10-06T03:51:26.008344+0000","last_peered":"2023-10-06T03:51:26.008344+0000","last_clean":"2023-10-06T03:51:26.008344+0000","last_became_active":"2023-10-06T03:51:26.006779+0000","last_became_peered":"2023-10-06T03:51:26.006779+0000","last_unstale":"2023-10-06T03:51:26.008344+0000","last_undegraded":"2023-10-06T03:51:26.008344+0000","last_fullsized":"2023-10-06T03:51:26.008344+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:16.164706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013728+0000","last_change":"2023-10-06T03:51:27.007390+0000","last_active":"2023-10-06T03:51:27.013728+0000","last_peered":"2023-10-06T03:51:27.013728+0000","last_clean":"2023-10-06T03:51:27.013728+0000","last_became_active":"2023-10-06T03:51:27.007148+0000","last_became_peered":"2023-10-06T03:51:27.007148+0000","last_unstale":"2023-10-06T03:51:27.013728+0000","last_undegraded":"2023-10-06T03:51:27.013728+0000","last_fullsized":"2023-10-06T03:51:27.013728+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:48:11.540041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"122'2","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.018670+0000","last_change":"2023-10-06T03:51:26.005705+0000","last_active":"2023-10-06T03:51:27.018670+0000","last_peered":"2023-10-06T03:51:27.018670+0000","last_clean":"2023-10-06T03:51:27.018670+0000","last_became_active":"2023-10-06T03:51:26.005458+0000","last_became_peered":"2023-10-06T03:51:26.005458+0000","last_unstale":"2023-10-06T03:51:27.018670+0000","last_undegraded":"2023-10-06T03:51:27.018670+0000","last_fullsized":"2023-10-06T03:51:27.018670+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:17.449763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":109,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.991983+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:51:26.991983+0000","last_peered":"2023-10-06T03:51:26.991983+0000","last_clean":"2023-10-06T03:51:26.991983+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:51:26.991983+0000","last_undegraded":"2023-10-06T03:51:26.991983+0000","last_fullsized":"2023-10-06T03:51:26.991983+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.013684+0000","last_change":"2023-10-06T03:51:27.008355+0000","last_active":"2023-10-06T03:51:27.013684+0000","last_peered":"2023-10-06T03:51:27.013684+0000","last_clean":"2023-10-06T03:51:27.013684+0000","last_became_active":"2023-10-06T03:51:27.008068+0000","last_became_peered":"2023-10-06T03:51:27.008068+0000","last_unstale":"2023-10-06T03:51:27.013684+0000","last_undegraded":"2023-10-06T03:51:27.013684+0000","last_fullsized":"2023-10-06T03:51:27.013684+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:10:50.589706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.013358+0000","last_change":"2023-10-06T03:51:26.010495+0000","last_active":"2023-10-06T03:51:26.013358+0000","last_peered":"2023-10-06T03:51:26.013358+0000","last_clean":"2023-10-06T03:51:26.013358+0000","last_became_active":"2023-10-06T03:51:26.010157+0000","last_became_peered":"2023-10-06T03:51:26.010157+0000","last_unstale":"2023-10-06T03:51:26.013358+0000","last_undegraded":"2023-10-06T03:51:26.013358+0000","last_fullsized":"2023-10-06T03:51:26.013358+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:49.281675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":102,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.994500+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:51:26.994500+0000","last_peered":"2023-10-06T03:51:26.994500+0000","last_clean":"2023-10-06T03:51:26.994500+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:51:26.994500+0000","last_undegraded":"2023-10-06T03:51:26.994500+0000","last_fullsized":"2023-10-06T03:51:26.994500+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.012581+0000","last_change":"2023-10-06T03:51:26.998480+0000","last_active":"2023-10-06T03:51:27.012581+0000","last_peered":"2023-10-06T03:51:27.012581+0000","last_clean":"2023-10-06T03:51:27.012581+0000","last_became_active":"2023-10-06T03:51:26.998219+0000","last_became_peered":"2023-10-06T03:51:26.998219+0000","last_unstale":"2023-10-06T03:51:27.012581+0000","last_undegraded":"2023-10-06T03:51:27.012581+0000","last_fullsized":"2023-10-06T03:51:27.012581+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:46.206866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"122'1","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.993971+0000","last_change":"2023-10-06T03:51:26.008613+0000","last_active":"2023-10-06T03:51:26.993971+0000","last_peered":"2023-10-06T03:51:26.993971+0000","last_clean":"2023-10-06T03:51:26.993971+0000","last_became_active":"2023-10-06T03:51:26.008346+0000","last_became_peered":"2023-10-06T03:51:26.008346+0000","last_unstale":"2023-10-06T03:51:26.993971+0000","last_undegraded":"2023-10-06T03:51:26.993971+0000","last_fullsized":"2023-10-06T03:51:26.993971+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:43:45.751206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:25.995771+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:51:25.995771+0000","last_peered":"2023-10-06T03:51:25.995771+0000","last_clean":"2023-10-06T03:51:25.995771+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:51:25.995771+0000","last_undegraded":"2023-10-06T03:51:25.995771+0000","last_fullsized":"2023-10-06T03:51:25.995771+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"112'227","reported_seq":641,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.002689+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:51:26.002689+0000","last_peered":"2023-10-06T03:51:26.002689+0000","last_clean":"2023-10-06T03:51:26.002689+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:51:26.002689+0000","last_undegraded":"2023-10-06T03:51:26.002689+0000","last_fullsized":"2023-10-06T03:51:26.002689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.011674+0000","last_change":"2023-10-06T03:51:27.011674+0000","last_active":"2023-10-06T03:51:27.011674+0000","last_peered":"2023-10-06T03:51:27.011674+0000","last_clean":"2023-10-06T03:51:27.011674+0000","last_became_active":"2023-10-06T03:51:27.010409+0000","last_became_peered":"2023-10-06T03:51:27.010409+0000","last_unstale":"2023-10-06T03:51:27.011674+0000","last_undegraded":"2023-10-06T03:51:27.011674+0000","last_fullsized":"2023-10-06T03:51:27.011674+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:44:51.302796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.002924+0000","last_change":"2023-10-06T03:51:26.003657+0000","last_active":"2023-10-06T03:51:27.002924+0000","last_peered":"2023-10-06T03:51:27.002924+0000","last_clean":"2023-10-06T03:51:27.002924+0000","last_became_active":"2023-10-06T03:51:26.003384+0000","last_became_peered":"2023-10-06T03:51:26.003384+0000","last_unstale":"2023-10-06T03:51:27.002924+0000","last_undegraded":"2023-10-06T03:51:27.002924+0000","last_fullsized":"2023-10-06T03:51:27.002924+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:55.734034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.001579+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:51:26.001579+0000","last_peered":"2023-10-06T03:51:26.001579+0000","last_clean":"2023-10-06T03:51:26.001579+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:51:26.001579+0000","last_undegraded":"2023-10-06T03:51:26.001579+0000","last_fullsized":"2023-10-06T03:51:26.001579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.005080+0000","last_change":"2023-10-06T03:51:27.001983+0000","last_active":"2023-10-06T03:51:27.005080+0000","last_peered":"2023-10-06T03:51:27.005080+0000","last_clean":"2023-10-06T03:51:27.005080+0000","last_became_active":"2023-10-06T03:51:27.001713+0000","last_became_peered":"2023-10-06T03:51:27.001713+0000","last_unstale":"2023-10-06T03:51:27.005080+0000","last_undegraded":"2023-10-06T03:51:27.005080+0000","last_fullsized":"2023-10-06T03:51:27.005080+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:58:58.329261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"122'2","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.023863+0000","last_change":"2023-10-06T03:51:26.011254+0000","last_active":"2023-10-06T03:51:27.023863+0000","last_peered":"2023-10-06T03:51:27.023863+0000","last_clean":"2023-10-06T03:51:27.023863+0000","last_became_active":"2023-10-06T03:51:26.010949+0000","last_became_peered":"2023-10-06T03:51:26.010949+0000","last_unstale":"2023-10-06T03:51:27.023863+0000","last_undegraded":"2023-10-06T03:51:27.023863+0000","last_fullsized":"2023-10-06T03:51:27.023863+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:24.241375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.002780+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:51:26.002780+0000","last_peered":"2023-10-06T03:51:26.002780+0000","last_clean":"2023-10-06T03:51:26.002780+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:51:26.002780+0000","last_undegraded":"2023-10-06T03:51:26.002780+0000","last_fullsized":"2023-10-06T03:51:26.002780+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.015583+0000","last_change":"2023-10-06T03:51:27.015583+0000","last_active":"2023-10-06T03:51:27.015583+0000","last_peered":"2023-10-06T03:51:27.015583+0000","last_clean":"2023-10-06T03:51:27.015583+0000","last_became_active":"2023-10-06T03:51:27.015326+0000","last_became_peered":"2023-10-06T03:51:27.015326+0000","last_unstale":"2023-10-06T03:51:27.015583+0000","last_undegraded":"2023-10-06T03:51:27.015583+0000","last_fullsized":"2023-10-06T03:51:27.015583+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:17:23.512840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"122'1","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.023279+0000","last_change":"2023-10-06T03:51:26.002515+0000","last_active":"2023-10-06T03:51:27.023279+0000","last_peered":"2023-10-06T03:51:27.023279+0000","last_clean":"2023-10-06T03:51:27.023279+0000","last_became_active":"2023-10-06T03:51:26.002112+0000","last_became_peered":"2023-10-06T03:51:26.002112+0000","last_unstale":"2023-10-06T03:51:27.023279+0000","last_undegraded":"2023-10-06T03:51:27.023279+0000","last_fullsized":"2023-10-06T03:51:27.023279+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:15:02.645285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.003078+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:51:27.003078+0000","last_peered":"2023-10-06T03:51:27.003078+0000","last_clean":"2023-10-06T03:51:27.003078+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:51:27.003078+0000","last_undegraded":"2023-10-06T03:51:27.003078+0000","last_fullsized":"2023-10-06T03:51:27.003078+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":8,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.023737+0000","last_change":"2023-10-06T03:51:27.023737+0000","last_active":"2023-10-06T03:51:27.023737+0000","last_peered":"2023-10-06T03:51:27.023737+0000","last_clean":"2023-10-06T03:51:27.023737+0000","last_became_active":"2023-10-06T03:51:27.023460+0000","last_became_peered":"2023-10-06T03:51:27.023460+0000","last_unstale":"2023-10-06T03:51:27.023737+0000","last_undegraded":"2023-10-06T03:51:27.023737+0000","last_fullsized":"2023-10-06T03:51:27.023737+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:11:09.366396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.013499+0000","last_change":"2023-10-06T03:51:26.009737+0000","last_active":"2023-10-06T03:51:26.013499+0000","last_peered":"2023-10-06T03:51:26.013499+0000","last_clean":"2023-10-06T03:51:26.013499+0000","last_became_active":"2023-10-06T03:51:26.009211+0000","last_became_peered":"2023-10-06T03:51:26.009211+0000","last_unstale":"2023-10-06T03:51:26.013499+0000","last_undegraded":"2023-10-06T03:51:26.013499+0000","last_fullsized":"2023-10-06T03:51:26.013499+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:07:02.657802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":116,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.992803+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:51:26.992803+0000","last_peered":"2023-10-06T03:51:26.992803+0000","last_clean":"2023-10-06T03:51:26.992803+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:51:26.992803+0000","last_undegraded":"2023-10-06T03:51:26.992803+0000","last_fullsized":"2023-10-06T03:51:26.992803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:27.020360+0000","last_change":"2023-10-06T03:51:27.017224+0000","last_active":"2023-10-06T03:51:27.020360+0000","last_peered":"2023-10-06T03:51:27.020360+0000","last_clean":"2023-10-06T03:51:27.020360+0000","last_became_active":"2023-10-06T03:51:27.016928+0000","last_became_peered":"2023-10-06T03:51:27.016928+0000","last_unstale":"2023-10-06T03:51:27.020360+0000","last_undegraded":"2023-10-06T03:51:27.020360+0000","last_fullsized":"2023-10-06T03:51:27.020360+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:25.976013+0000","last_clean_scrub_stamp":"2023-10-06T03:51:25.976013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:42.614149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.993044+0000","last_change":"2023-10-06T03:51:25.998061+0000","last_active":"2023-10-06T03:51:26.993044+0000","last_peered":"2023-10-06T03:51:26.993044+0000","last_clean":"2023-10-06T03:51:26.993044+0000","last_became_active":"2023-10-06T03:51:25.997343+0000","last_became_peered":"2023-10-06T03:51:25.997343+0000","last_unstale":"2023-10-06T03:51:26.993044+0000","last_undegraded":"2023-10-06T03:51:26.993044+0000","last_fullsized":"2023-10-06T03:51:26.993044+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:51:24.965453+0000","last_clean_scrub_stamp":"2023-10-06T03:51:24.965453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:23.839660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":116,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-06T03:51:26.992686+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:51:26.992686+0000","last_peered":"2023-10-06T03:51:26.992686+0000","last_clean":"2023-10-06T03:51:26.992686+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:51:26.992686+0000","last_undegraded":"2023-10-06T03:51:26.992686+0000","last_fullsized":"2023-10-06T03:51:26.992686+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575063,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311632,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060208,"statfs":{"total":96636764160,"available":96317652992,"internally_reserved":0,"allocated":2105344,"data_stored":1364369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.66200000000000003,"15min":0.69599999999999995},"min":{"1min":0.48399999999999999,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.89400000000000002,"5min":0.998,"15min":1.0409999999999999},"last":0.62},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65600000000000003,"15min":0.70199999999999996},"min":{"1min":0.48899999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83699999999999997,"5min":1.02,"15min":1.02},"last":0.57899999999999996}]},{"osd":1,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.70099999999999996,"15min":0.75},"min":{"1min":0.55000000000000004,"5min":0.318,"15min":0.318},"max":{"1min":0.96199999999999997,"5min":1.038,"15min":1.0640000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73099999999999998,"15min":0.76600000000000001},"min":{"1min":0.56699999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95699999999999996,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.64500000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66400000000000003,"15min":0.69599999999999995},"min":{"1min":0.55000000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84999999999999998,"5min":0.97499999999999998,"15min":1.044},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71299999999999997,"15min":0.73999999999999999},"min":{"1min":0.58199999999999996,"5min":0.44,"15min":0.44},"max":{"1min":0.85999999999999999,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.68899999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69899999999999995,"15min":0.72499999999999998},"min":{"1min":0.52200000000000002,"5min":0.41799999999999998,"15min":0.39500000000000002},"max":{"1min":0.81599999999999995,"5min":1.0089999999999999,"15min":1.016},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72599999999999998,"15min":0.74299999999999999},"min":{"1min":0.54000000000000004,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92000000000000004,"5min":0.98499999999999999,"15min":1.0409999999999999},"last":0.79200000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61699999999999999,"15min":0.59399999999999997},"min":{"1min":0.42499999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90600000000000003,"5min":0.99199999999999999,"15min":1.0049999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63700000000000001,"15min":0.622},"min":{"1min":0.442,"5min":0.41299999999999998,"15min":0.39400000000000002},"max":{"1min":0.94699999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.54700000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66300000000000003,"15min":0.64000000000000001},"min":{"1min":0.48299999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92300000000000004,"5min":1.022,"15min":1.069},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69299999999999995,"15min":0.68000000000000005},"min":{"1min":0.501,"5min":0.44500000000000001,"15min":0.436},"max":{"1min":0.997,"5min":1.105,"15min":1.105},"last":0.502}]},{"osd":6,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.748,"15min":0.72699999999999998},"min":{"1min":0.47799999999999998,"5min":0.442,"15min":0.436},"max":{"1min":1.034,"5min":1.181,"15min":1.181},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.70299999999999996},"min":{"1min":0.503,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90900000000000003,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.70299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575064,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306984,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064856,"statfs":{"total":96636764160,"available":96322412544,"internally_reserved":0,"allocated":1540096,"data_stored":783964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.67000000000000004,"15min":0.70099999999999996},"min":{"1min":0.45200000000000001,"5min":0.32600000000000001,"15min":0.313},"max":{"1min":0.80300000000000005,"5min":1.288,"15min":1.288},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69799999999999995,"15min":0.72399999999999998},"min":{"1min":0.57299999999999995,"5min":0.41299999999999998,"15min":0.40600000000000003},"max":{"1min":0.88300000000000001,"5min":0.96799999999999997,"15min":1.0069999999999999},"last":0.77200000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68500000000000005,"15min":0.72699999999999998},"min":{"1min":0.5,"5min":0.36499999999999999,"15min":0.33800000000000002},"max":{"1min":0.88500000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.69399999999999995,"15min":0.71599999999999997},"min":{"1min":0.46400000000000002,"5min":0.441,"15min":0.35899999999999999},"max":{"1min":0.85199999999999998,"5min":1.427,"15min":1.427},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.52500000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89500000000000002,"5min":1.179,"15min":1.179},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70799999999999996,"15min":0.73699999999999999},"min":{"1min":0.47799999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92500000000000004,"5min":1.0209999999999999,"15min":1.0409999999999999},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Oct 6 03:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72199999999999998,"15min":0.749},"min":{"1min":0.51800000000000002,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":0.81499999999999995,"5min":1.01,"15min":1.024},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72899999999999998,"15min":0.75800000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86599999999999999,"5min":1.097,"15min":1.1639999999999999},"last":0.82899999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.78900000000000003,"5min":1.462,"15min":1.462},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.627,"15min":0.63},"min":{"1min":0.377,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93100000000000005,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68999999999999995,"15min":0.66600000000000004},"min":{"1min":0.45000000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92000000000000004,"5min":1.0109999999999999,"15min":1.0489999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41199999999999998},"max":{"1min":0.91000000000000003,"5min":1.508,"15min":1.508},"last":0.62}]},{"osd":7,"last update":"Fri Oct 6 03:50:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70299999999999996,"15min":0.69299999999999995},"min":{"1min":0.41499999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.98399999999999999,"5min":1.21,"15min":1.21},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.40000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.95499999999999996,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.55400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575063,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307536,"kb_used_data":2056,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064304,"statfs":{"total":96636764160,"available":96321847296,"internally_reserved":0,"allocated":2105344,"data_stored":1365221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46500000000000002,"15min":0.48699999999999999},"min":{"1min":0.33100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.68300000000000005,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.438},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54800000000000004,"15min":0.56799999999999995},"min":{"1min":0.36399999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73699999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.51800000000000002}]},{"osd":2,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46600000000000003,"15min":0.49199999999999999},"min":{"1min":0.32700000000000001,"5min":0.32000000000000001,"15min":0.309},"max":{"1min":0.58599999999999997,"5min":0.72699999999999998,"15min":0.84799999999999998},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.94099999999999995},"last":0.499}]},{"osd":3,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58299999999999996,"15min":0.60199999999999998},"min":{"1min":0.45800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.77400000000000002,"5min":0.86699999999999999,"15min":0.91700000000000004},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52400000000000002,"15min":0.53700000000000003},"min":{"1min":0.41299999999999998,"5min":0.34200000000000003,"15min":0.32500000000000001},"max":{"1min":0.74199999999999999,"5min":0.81499999999999995,"15min":0.90400000000000003},"last":0.59799999999999998}]},{"osd":4,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.50800000000000001,"5min":0.46700000000000003,"15min":0.41499999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.1699999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.76600000000000001,"15min":0.79900000000000004},"min":{"1min":0.58699999999999997,"5min":0.51400000000000001,"15min":0.439},"max":{"1min":0.996,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77200000000000002,"15min":0.78700000000000003},"min":{"1min":0.58499999999999996,"5min":0.48199999999999998,"15min":0.45400000000000001},"max":{"1min":0.94999999999999996,"5min":1.294,"15min":1.294},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.77900000000000003,"15min":0.80000000000000004},"min":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.45100000000000001},"max":{"1min":1.0029999999999999,"5min":1.224,"15min":1.224},"last":0.71899999999999997}]},{"osd":6,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78200000000000003,"15min":0.80300000000000005},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.117,"5min":1.26,"15min":1.26},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78500000000000003,"15min":0.81599999999999995},"min":{"1min":0.55200000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0880000000000001,"5min":1.143,"15min":1.224},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri Oct 6 03:50:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.81799999999999995,"15min":0.83699999999999997},"min":{"1min":0.53100000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0629999999999999,"5min":1.1519999999999999,"15min":1.272},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81399999999999995,"15min":0.83099999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.51200000000000001},"max":{"1min":1.079,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.68999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575065,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306976,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064864,"statfs":{"total":96636764160,"available":96322420736,"internally_reserved":0,"allocated":1531904,"data_stored":782500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.505,"15min":0.51800000000000002},"min":{"1min":0.34200000000000003,"5min":0.309,"15min":0.309},"max":{"1min":0.68999999999999995,"5min":0.70999999999999996,"15min":0.83599999999999997},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Oct 6 03:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.54500000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.745,"5min":0.80100000000000005,"15min":1.1299999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53000000000000003,"15min":0.55100000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.31900000000000001},"max":{"1min":0.73999999999999999,"5min":0.83399999999999996,"15min":0.89400000000000002},"last":0.497}]},{"osd":3,"last update":"Fri Oct 6 03:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52400000000000002,"15min":0.52300000000000002},"min":{"1min":0.35499999999999998,"5min":0.318,"15min":0.30499999999999999},"max":{"1min":0.83499999999999996,"5min":0.83699999999999997,"15min":0.93200000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.378,"15min":0.378},"max":{"1min":0.79800000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.54000000000000004}]},{"osd":4,"last update":"Fri Oct 6 03:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.80600000000000005,"15min":0.82999999999999996},"min":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.49399999999999999},"max":{"1min":1.05,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.80200000000000005,"15min":0.82499999999999996},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.45500000000000002},"max":{"1min":1.1240000000000001,"5min":1.306,"15min":1.3400000000000001},"last":0.96099999999999997}]},{"osd":5,"last update":"Fri Oct 6 03:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79300000000000004,"15min":0.81499999999999995},"min":{"1min":0.55100000000000005,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.1459999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.042},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.78200000000000003,"15min":0.80700000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.194,"5min":1.194,"15min":1.6559999999999999},"last":0.77700000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.76600000000000001,"15min":0.80600000000000005},"min":{"1min":0.56100000000000005,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.726},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.80900000000000005,"15min":0.84099999999999997},"min":{"1min":0.52900000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.212,"5min":1.212,"15min":1.843},"last":1.01}]},{"osd":7,"last update":"Fri Oct 6 03:50:47 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.80800000000000005,"15min":0.82499999999999996},"min":{"1min":0.502,"5min":0.496,"15min":0.496},"max":{"1min":1.1419999999999999,"5min":1.302,"15min":1.302},"last":0.996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.80300000000000005,"15min":0.83199999999999996},"min":{"1min":0.58999999999999997,"5min":0.53100000000000003,"15min":0.495},"max":{"1min":1.085,"5min":1.2450000000000001,"15min":1.7649999999999999},"last":0.89100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575064,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311048,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060792,"statfs":{"total":96636764160,"available":96318251008,"internally_reserved":0,"allocated":1507328,"data_stored":773875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.45500000000000002,"15min":0.47399999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.622,"5min":0.74199999999999999,"15min":0.80100000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.51900000000000002,"15min":0.53100000000000003},"min":{"1min":0.379,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":0.76500000000000001,"5min":0.86699999999999999,"15min":0.89300000000000002},"last":0.36799999999999999}]},{"osd":1,"last update":"Fri Oct 6 03:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.28699999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.90000000000000002},"last":0.311},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54200000000000004,"15min":0.55600000000000005},"min":{"1min":0.38400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83099999999999996,"5min":0.89700000000000002,"15min":0.94899999999999995},"last":0.44600000000000001}]},{"osd":3,"last update":"Fri Oct 6 03:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57699999999999996,"15min":0.59099999999999997},"min":{"1min":0.374,"5min":0.37,"15min":0.37},"max":{"1min":0.78300000000000003,"5min":1.78,"15min":1.78},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56799999999999995,"15min":0.57699999999999996},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.33300000000000002},"max":{"1min":0.81599999999999995,"5min":0.84699999999999998,"15min":0.96499999999999997},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.80600000000000005,"15min":0.82599999999999996},"min":{"1min":0.57999999999999996,"5min":0.51400000000000001,"15min":0.51200000000000001},"max":{"1min":1.111,"5min":1.111,"15min":1.5620000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.82699999999999996,"15min":0.84399999999999997},"min":{"1min":0.61199999999999999,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.2270000000000001},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Oct 6 03:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.82599999999999996,"15min":0.84199999999999997},"min":{"1min":0.59399999999999997,"5min":0.503,"15min":0.503},"max":{"1min":1.135,"5min":1.135,"15min":1.3},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.82199999999999995,"15min":0.85399999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0449999999999999,"5min":1.855,"15min":1.855},"last":0.68300000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.81499999999999995,"15min":0.83799999999999997},"min":{"1min":0.53600000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.99299999999999999,"5min":1.907,"15min":1.907},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.81100000000000005,"15min":0.82799999999999996},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0680000000000001,"5min":1.1080000000000001,"15min":1.4750000000000001},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:50:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.82399999999999995,"15min":0.84999999999999998},"min":{"1min":0.54700000000000004,"5min":0.495,"15min":0.495},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.4990000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83699999999999997,"15min":0.871},"min":{"1min":0.56899999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1140000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.57399999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575064,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306956,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064884,"statfs":{"total":96636764160,"available":96322441216,"internally_reserved":0,"allocated":1511424,"data_stored":775373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:50:29 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44600000000000001,"15min":0.45900000000000002},"min":{"1min":0.34899999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.68700000000000006,"5min":0.75700000000000001,"15min":0.83999999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47199999999999998,"15min":0.497},"min":{"1min":0.374,"5min":0.317,"15min":0.29799999999999999},"max":{"1min":0.67900000000000005,"5min":1.04,"15min":1.1699999999999999},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Oct 6 03:50:29 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.47999999999999998,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.53400000000000003,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:50:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55800000000000005,"15min":0.56999999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80000000000000004,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.57899999999999996,"15min":0.60099999999999998},"min":{"1min":0.48099999999999998,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.87},"last":0.58699999999999997}]},{"osd":4,"last update":"Fri Oct 6 03:50:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73799999999999999,"15min":0.77900000000000003},"min":{"1min":0.54200000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.4099999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77000000000000002,"15min":0.82399999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.98699999999999999,"5min":1.0469999999999999,"15min":1.377},"last":0.94199999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:50:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.78800000000000003},"min":{"1min":0.52500000000000002,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.88600000000000001,"5min":1.012,"15min":1.256},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.745,"15min":0.78100000000000003},"min":{"1min":0.56200000000000006,"5min":0.53500000000000003,"15min":0.49099999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.2030000000000001},"last":0.79300000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:50:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.76500000000000001,"15min":0.81100000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.04,"5min":1.0580000000000001,"15min":1.323},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79100000000000004,"15min":0.84299999999999997},"min":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0509999999999999,"5min":1.075,"15min":1.2849999999999999},"last":0.96799999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:50:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.76000000000000001,"15min":0.80800000000000005},"min":{"1min":0.60399999999999998,"5min":0.498,"15min":0.498},"max":{"1min":0.95699999999999996,"5min":0.98999999999999999,"15min":1.2470000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76900000000000002,"15min":0.80200000000000005},"min":{"1min":0.53700000000000003,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":0.95599999999999996,"5min":1.093,"15min":1.2330000000000001},"last":0.77800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575064,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306972,"kb_used_data":1492,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064868,"statfs":{"total":96636764160,"available":96322424832,"internally_reserved":0,"allocated":1527808,"data_stored":781944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.69499999999999995,"15min":0.72099999999999997},"min":{"1min":0.48899999999999999,"5min":0.433,"15min":0.316},"max":{"1min":0.996,"5min":0.996,"15min":1.0189999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67900000000000005,"15min":0.70299999999999996},"min":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.316},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.01},"last":0.80300000000000005}]},{"osd":1,"last update":"Fri Oct 6 03:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69199999999999995,"15min":0.72499999999999998},"min":{"1min":0.48199999999999998,"5min":0.42999999999999999,"15min":0.33200000000000002},"max":{"1min":1.004,"5min":1.081,"15min":1.081},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.72099999999999997,"15min":0.73899999999999999},"min":{"1min":0.45400000000000001,"5min":0.42799999999999999,"15min":0.41299999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.68700000000000006}]},{"osd":2,"last update":"Fri Oct 6 03:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71299999999999997,"15min":0.73799999999999999},"min":{"1min":0.55500000000000005,"5min":0.44400000000000001,"15min":0.377},"max":{"1min":0.95499999999999996,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.874},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73799999999999999,"15min":0.748},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.022},"last":0.84299999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70899999999999996,"15min":0.72799999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97399999999999998,"5min":1.0189999999999999,"15min":1.0409999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73799999999999999,"15min":0.76500000000000001},"min":{"1min":0.53500000000000003,"5min":0.47099999999999997,"15min":0.46500000000000002},"max":{"1min":0.90800000000000003,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.90000000000000002}]},{"osd":5,"last update":"Fri Oct 6 03:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.64200000000000002,"15min":0.627},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.044,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.502}]},{"osd":6,"last update":"Fri Oct 6 03:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65500000000000003,"15min":0.64100000000000001},"min":{"1min":0.437,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88100000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70899999999999996,"15min":0.68799999999999994},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.38100000000000001},"max":{"1min":1.0760000000000001,"5min":1.083,"15min":1.083},"last":0.53700000000000003}]},{"osd":7,"last update":"Fri Oct 6 03:50:50 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69599999999999995,"15min":0.67900000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93700000000000006,"5min":1.046,"15min":1.133},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72699999999999998,"15min":0.71399999999999997},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.41299999999999998},"max":{"1min":1.016,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.77900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575064,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1503232,"data_stored":773841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68500000000000005,"15min":0.70799999999999996},"min":{"1min":0.50600000000000001,"5min":0.35999999999999999,"15min":0.35299999999999998},"max":{"1min":0.878,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70999999999999996,"15min":0.72099999999999997},"min":{"1min":0.495,"5min":0.42399999999999999,"15min":0.32600000000000001},"max":{"1min":0.98999999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.58799999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.75700000000000001,"15min":0.73699999999999999},"min":{"1min":0.56100000000000005,"5min":0.45900000000000002,"15min":0.40000000000000002},"max":{"1min":5.8239999999999998,"5min":5.8239999999999998,"15min":5.8239999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.76000000000000001,"15min":0.746},"min":{"1min":0.501,"5min":0.44,"15min":0.32400000000000001},"max":{"1min":5.8680000000000003,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.48999999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.75700000000000001,"15min":0.77300000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.40200000000000002},"max":{"1min":0.96799999999999997,"5min":1.155,"15min":1.155},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75900000000000001,"15min":0.77600000000000002},"min":{"1min":0.58599999999999997,"5min":0.51200000000000001,"15min":0.50700000000000001},"max":{"1min":1,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.749}]},{"osd":3,"last update":"Fri Oct 6 03:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71399999999999997,"15min":0.73199999999999998},"min":{"1min":0.52100000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89900000000000002,"5min":1.091,"15min":1.6419999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.59899999999999998,"5min":0.503,"15min":0.47799999999999998},"max":{"1min":0.94299999999999995,"5min":1.238,"15min":1.238},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri Oct 6 03:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.64400000000000002,"15min":0.61699999999999999},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":1.04,"15min":1.04},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.64100000000000001,"15min":0.626},"min":{"1min":0.441,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":0.92700000000000005,"5min":1.147,"15min":1.147},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Oct 6 03:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.66600000000000004,"15min":0.64400000000000002},"min":{"1min":0.46899999999999997,"5min":0.40200000000000002,"15min":0.34999999999999998},"max":{"1min":0.79400000000000004,"5min":1.591,"15min":1.591},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71499999999999997,"15min":0.69199999999999995},"min":{"1min":0.46999999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83099999999999996,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.86899999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:50:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.68100000000000005,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.81399999999999995,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.72599999999999998,"15min":0.71599999999999997},"min":{"1min":0.47499999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89400000000000002,"5min":1.278,"15min":1.278},"last":0.91500000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:51:32.392 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:51:32.407 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:51:32.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.601+0000 7f7a5759e700 1 Processor -- start 2023-10-06T03:51:32.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.601+0000 7f7a5759e700 1 -- start start 2023-10-06T03:51:32.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.602+0000 7f7a5759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7a5806e630 0x7f7a580abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:32.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.602+0000 7f7a5759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7a58159840 0x7f7a580ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:32.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.602+0000 7f7a5759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a58063870 con 0x7f7a5815df30 2023-10-06T03:51:32.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.602+0000 7f7a5759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a580acda0 con 0x7f7a58159840 2023-10-06T03:51:32.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.602+0000 7f7a5759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a580acac0 con 0x7f7a5806e630 2023-10-06T03:51:32.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.603+0000 7f7a56d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7a58159840 0x7f7a580ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:32.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.603+0000 7f7a56d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7a58159840 0x7f7a580ac490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37178/0 (socket says 172.21.15.55:37178) 2023-10-06T03:51:32.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.603+0000 7f7a56d9d700 1 -- 172.21.15.55:0/1875142174 learned_addr learned my addr 172.21.15.55:0/1875142174 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:32.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.603+0000 7f7a5659c700 1 --2- 172.21.15.55:0/1875142174 >> v2:172.21.15.163:3300/0 conn(0x7f7a5806e630 0x7f7a580abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:32.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.603+0000 7f7a56d9d700 1 -- 172.21.15.55:0/1875142174 >> v2:172.21.15.163:3300/0 conn(0x7f7a5806e630 msgr2=0x7f7a580abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:32.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.603+0000 7f7a56d9d700 1 --2- 172.21.15.55:0/1875142174 >> v2:172.21.15.163:3300/0 conn(0x7f7a5806e630 0x7f7a580abf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:32.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.603+0000 7f7a56d9d700 1 -- 172.21.15.55:0/1875142174 >> v1:172.21.15.163:6789/0 conn(0x7f7a5815df30 legacy=0x7f7a58159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:32.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.603+0000 7f7a56d9d700 1 -- 172.21.15.55:0/1875142174 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5815d450 con 0x7f7a58159840 2023-10-06T03:51:32.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.603+0000 7f7a56d9d700 1 --2- 172.21.15.55:0/1875142174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a58159840 0x7f7a580ac490 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f7a580645c0 tx=0x7f7a50009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebe711456ec189 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:32.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.604+0000 7f7a5559a700 1 -- 172.21.15.55:0/1875142174 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000e070 con 0x7f7a58159840 2023-10-06T03:51:32.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.604+0000 7f7a5559a700 1 -- 172.21.15.55:0/1875142174 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5000ba20 con 0x7f7a58159840 2023-10-06T03:51:32.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.604+0000 7f7a5559a700 1 -- 172.21.15.55:0/1875142174 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a500092e0 con 0x7f7a58159840 2023-10-06T03:51:32.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.604+0000 7f7a5759e700 1 -- 172.21.15.55:0/1875142174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a58159840 msgr2=0x7f7a580ac490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:32.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.604+0000 7f7a5759e700 1 --2- 172.21.15.55:0/1875142174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a58159840 0x7f7a580ac490 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f7a580645c0 tx=0x7f7a50009840 comp rx=0 tx=0).stop 2023-10-06T03:51:32.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.604+0000 7f7a5759e700 1 -- 172.21.15.55:0/1875142174 shutdown_connections 2023-10-06T03:51:32.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.604+0000 7f7a5759e700 1 --2- 172.21.15.55:0/1875142174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a58159840 0x7f7a580ac490 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:32.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.604+0000 7f7a5759e700 1 --2- 172.21.15.55:0/1875142174 >> v2:172.21.15.163:3300/0 conn(0x7f7a5806e630 0x7f7a580abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:32.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.604+0000 7f7a5759e700 1 -- 172.21.15.55:0/1875142174 >> 172.21.15.55:0/1875142174 conn(0x7f7a580c9f50 msgr2=0x7f7a5806dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:32.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.604+0000 7f7a5759e700 1 -- 172.21.15.55:0/1875142174 shutdown_connections 2023-10-06T03:51:32.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.604+0000 7f7a5759e700 1 -- 172.21.15.55:0/1875142174 wait complete. 2023-10-06T03:51:32.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.606+0000 7f7a5759e700 1 Processor -- start 2023-10-06T03:51:32.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.606+0000 7f7a5759e700 1 -- start start 2023-10-06T03:51:32.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.607+0000 7f7a5759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a5806e630 0x7f7a58150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:32.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.607+0000 7f7a5759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a58159840 0x7f7a58160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:32.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.607+0000 7f7a5759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a5815df30 0x7f7a58150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:32.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.607+0000 7f7a5759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58152650 con 0x7f7a58159840 2023-10-06T03:51:32.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.607+0000 7f7a5759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58152b10 con 0x7f7a5815df30 2023-10-06T03:51:32.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.607+0000 7f7a5759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58152830 con 0x7f7a5806e630 2023-10-06T03:51:32.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.607+0000 7f7a55d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a58159840 0x7f7a58160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:32.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.607+0000 7f7a55d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a58159840 0x7f7a58160070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:37184/0 (socket says 172.21.15.55:37184) 2023-10-06T03:51:32.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.607+0000 7f7a55d9b700 1 -- 172.21.15.55:0/456738056 learned_addr learned my addr 172.21.15.55:0/456738056 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:32.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.607+0000 7f7a5659c700 1 --2- 172.21.15.55:0/456738056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a5806e630 0x7f7a58150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:32.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.607+0000 7f7a56d9d700 1 --2- 172.21.15.55:0/456738056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a5815df30 0x7f7a58150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:32.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.608+0000 7f7a55d9b700 1 -- 172.21.15.55:0/456738056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a5806e630 msgr2=0x7f7a58150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:32.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.608+0000 7f7a55d9b700 1 --2- 172.21.15.55:0/456738056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a5806e630 0x7f7a58150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:32.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.608+0000 7f7a55d9b700 1 -- 172.21.15.55:0/456738056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a5815df30 msgr2=0x7f7a58150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:32.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.608+0000 7f7a55d9b700 1 --2- 172.21.15.55:0/456738056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a5815df30 0x7f7a58150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:32.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.608+0000 7f7a55d9b700 1 -- 172.21.15.55:0/456738056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a50009ae0 con 0x7f7a58159840 2023-10-06T03:51:32.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.608+0000 7f7a55d9b700 1 --2- 172.21.15.55:0/456738056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a58159840 0x7f7a58160070 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00c590 tx=0x7f7a4c007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:32.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.608+0000 7f7a467fc700 1 -- 172.21.15.55:0/456738056 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c011590 con 0x7f7a58159840 2023-10-06T03:51:32.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.608+0000 7f7a467fc700 1 -- 172.21.15.55:0/456738056 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c011710 con 0x7f7a58159840 2023-10-06T03:51:32.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.608+0000 7f7a5759e700 1 -- 172.21.15.55:0/456738056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5815ef30 con 0x7f7a58159840 2023-10-06T03:51:32.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.608+0000 7f7a467fc700 1 -- 172.21.15.55:0/456738056 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c011ad0 con 0x7f7a58159840 2023-10-06T03:51:32.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.608+0000 7f7a5759e700 1 -- 172.21.15.55:0/456738056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5815f4f0 con 0x7f7a58159840 2023-10-06T03:51:32.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.609+0000 7f7a5759e700 1 -- 172.21.15.55:0/456738056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5804c3c0 con 0x7f7a58159840 2023-10-06T03:51:32.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.610+0000 7f7a467fc700 1 -- 172.21.15.55:0/456738056 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a4c01b3b0 con 0x7f7a58159840 2023-10-06T03:51:32.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.611+0000 7f7a467fc700 1 --2- 172.21.15.55:0/456738056 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a38066f30 0x7f7a380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:32.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.611+0000 7f7a467fc700 1 -- 172.21.15.55:0/456738056 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f7a4c083c00 con 0x7f7a58159840 2023-10-06T03:51:32.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.611+0000 7f7a5659c700 1 --2- 172.21.15.55:0/456738056 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a38066f30 0x7f7a380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:32.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.612+0000 7f7a5659c700 1 --2- 172.21.15.55:0/456738056 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a38066f30 0x7f7a380693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009db0 tx=0x7f7a480091a0 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:32.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.615+0000 7f7a467fc700 1 -- 172.21.15.55:0/456738056 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a4c050ae0 con 0x7f7a58159840 2023-10-06T03:51:32.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.900+0000 7f7a5759e700 1 -- 172.21.15.55:0/456738056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7a580c9100 con 0x7f7a58159840 2023-10-06T03:51:32.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.905+0000 7f7a467fc700 1 -- 172.21.15.55:0/456738056 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f7a4c017070 con 0x7f7a58159840 2023-10-06T03:51:32.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.908+0000 7f7a5759e700 1 -- 172.21.15.55:0/456738056 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a38066f30 msgr2=0x7f7a380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:32.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.908+0000 7f7a5759e700 1 --2- 172.21.15.55:0/456738056 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a38066f30 0x7f7a380693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009db0 tx=0x7f7a480091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:32.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.908+0000 7f7a5759e700 1 -- 172.21.15.55:0/456738056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a58159840 msgr2=0x7f7a58160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:32.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.908+0000 7f7a5759e700 1 --2- 172.21.15.55:0/456738056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a58159840 0x7f7a58160070 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00c590 tx=0x7f7a4c007f50 comp rx=0 tx=0).stop 2023-10-06T03:51:32.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.908+0000 7f7a5759e700 1 -- 172.21.15.55:0/456738056 shutdown_connections 2023-10-06T03:51:32.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.908+0000 7f7a5759e700 1 --2- 172.21.15.55:0/456738056 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a38066f30 0x7f7a380693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:32.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.908+0000 7f7a5759e700 1 --2- 172.21.15.55:0/456738056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a5815df30 0x7f7a58150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:32.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.908+0000 7f7a5759e700 1 --2- 172.21.15.55:0/456738056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a5806e630 0x7f7a58150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:32.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.908+0000 7f7a5759e700 1 --2- 172.21.15.55:0/456738056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a58159840 0x7f7a58160070 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:32.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.908+0000 7f7a5759e700 1 -- 172.21.15.55:0/456738056 >> 172.21.15.55:0/456738056 conn(0x7f7a580c9f50 msgr2=0x7f7a580cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:32.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.908+0000 7f7a5759e700 1 -- 172.21.15.55:0/456738056 shutdown_connections 2023-10-06T03:51:32.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:32.908+0000 7f7a5759e700 1 -- 172.21.15.55:0/456738056 wait complete. 2023-10-06T03:51:32.939 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2023-10-06T03:51:32.939 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:51:33.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.125+0000 7fe8d4f9d700 1 Processor -- start 2023-10-06T03:51:33.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.125+0000 7fe8d4f9d700 1 -- start start 2023-10-06T03:51:33.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.126+0000 7fe8d4f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe8d01566e0 0x7fe8d00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:33.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.126+0000 7fe8d4f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d00ca860 con 0x7fe8d00cb900 2023-10-06T03:51:33.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.126+0000 7fe8d4f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d00cab40 con 0x7fe8d0156310 2023-10-06T03:51:33.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.126+0000 7fe8d4f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d00cae20 con 0x7fe8d01566e0 2023-10-06T03:51:33.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.127+0000 7fe8cffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe8d00cb900 0x7fe8d00c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59018/0 (socket says 172.21.15.55:59018) 2023-10-06T03:51:33.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.127+0000 7fe8cffff700 1 -- 172.21.15.55:0/249202877 learned_addr learned my addr 172.21.15.55:0/249202877 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:33.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.127+0000 7fe8ceffd700 1 --2- 172.21.15.55:0/249202877 >> v2:172.21.15.163:3300/0 conn(0x7fe8d01566e0 0x7fe8d00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:33.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.127+0000 7fe8ce7fc700 1 -- 172.21.15.55:0/249202877 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131635952 0 0) 0x7fe8d00ca860 con 0x7fe8d00cb900 2023-10-06T03:51:33.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.127+0000 7fe8ce7fc700 1 -- 172.21.15.55:0/249202877 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8b8003570 con 0x7fe8d00cb900 2023-10-06T03:51:33.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.127+0000 7fe8ceffd700 1 -- 172.21.15.55:0/249202877 >> v1:172.21.15.163:6789/0 conn(0x7fe8d0156310 legacy=0x7fe8d00c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:33.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.127+0000 7fe8ceffd700 1 -- 172.21.15.55:0/249202877 >> v1:172.21.15.55:6789/0 conn(0x7fe8d00cb900 legacy=0x7fe8d00c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:33.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.128+0000 7fe8ceffd700 1 -- 172.21.15.55:0/249202877 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d00cb100 con 0x7fe8d01566e0 2023-10-06T03:51:33.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.128+0000 7fe8ceffd700 1 --2- 172.21.15.55:0/249202877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d01566e0 0x7fe8d00ae000 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40098e0 tx=0x7fe8c40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=684e3f9bb12ca8af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:33.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.128+0000 7fe8ce7fc700 1 -- 172.21.15.55:0/249202877 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c400b640 con 0x7fe8d01566e0 2023-10-06T03:51:33.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.128+0000 7fe8ce7fc700 1 -- 172.21.15.55:0/249202877 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c400b7a0 con 0x7fe8d01566e0 2023-10-06T03:51:33.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.128+0000 7fe8ce7fc700 1 -- 172.21.15.55:0/249202877 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c400bb40 con 0x7fe8d01566e0 2023-10-06T03:51:33.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.129+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/249202877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d01566e0 msgr2=0x7fe8d00ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:33.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.129+0000 7fe8d4f9d700 1 --2- 172.21.15.55:0/249202877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d01566e0 0x7fe8d00ae000 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40098e0 tx=0x7fe8c40091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.129+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/249202877 shutdown_connections 2023-10-06T03:51:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.129+0000 7fe8d4f9d700 1 --2- 172.21.15.55:0/249202877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d01566e0 0x7fe8d00ae000 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.129+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/249202877 >> 172.21.15.55:0/249202877 conn(0x7fe8d00cc1f0 msgr2=0x7fe8d00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.129+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/249202877 shutdown_connections 2023-10-06T03:51:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.129+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/249202877 wait complete. 2023-10-06T03:51:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.130+0000 7fe8d4f9d700 1 Processor -- start 2023-10-06T03:51:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.130+0000 7fe8d4f9d700 1 -- start start 2023-10-06T03:51:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.131+0000 7fe8d4f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8d00cb900 0x7fe8d0150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.131+0000 7fe8d4f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d0156310 0x7fe8d016a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.131+0000 7fe8d4f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8d01566e0 0x7fe8d00af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:33.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.131+0000 7fe8d4f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d01512a0 con 0x7fe8d01566e0 2023-10-06T03:51:33.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.131+0000 7fe8d4f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d015e270 con 0x7fe8d0156310 2023-10-06T03:51:33.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.131+0000 7fe8d4f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d016aba0 con 0x7fe8d00cb900 2023-10-06T03:51:33.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.132+0000 7fe8cffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8d01566e0 0x7fe8d00af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:33.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.132+0000 7fe8cffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8d01566e0 0x7fe8d00af330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53634/0 (socket says 172.21.15.55:53634) 2023-10-06T03:51:33.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.132+0000 7fe8cffff700 1 -- 172.21.15.55:0/1789786760 learned_addr learned my addr 172.21.15.55:0/1789786760 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:33.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.132+0000 7fe8ceffd700 1 --2- 172.21.15.55:0/1789786760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d0156310 0x7fe8d016a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:33.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.132+0000 7fe8cf7fe700 1 --2- 172.21.15.55:0/1789786760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8d00cb900 0x7fe8d0150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:33.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.132+0000 7fe8cffff700 1 -- 172.21.15.55:0/1789786760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8d00cb900 msgr2=0x7fe8d0150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:33.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.132+0000 7fe8cffff700 1 --2- 172.21.15.55:0/1789786760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8d00cb900 0x7fe8d0150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:33.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.132+0000 7fe8cffff700 1 -- 172.21.15.55:0/1789786760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d0156310 msgr2=0x7fe8d016a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:33.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.132+0000 7fe8cffff700 1 --2- 172.21.15.55:0/1789786760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d0156310 0x7fe8d016a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:33.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.132+0000 7fe8cffff700 1 -- 172.21.15.55:0/1789786760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c4009420 con 0x7fe8d01566e0 2023-10-06T03:51:33.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.132+0000 7fe8cffff700 1 --2- 172.21.15.55:0/1789786760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8d01566e0 0x7fe8d00af330 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8001460 tx=0x7fe8c8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:33.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.132+0000 7fe8b77fe700 1 -- 172.21.15.55:0/1789786760 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c800a870 con 0x7fe8d01566e0 2023-10-06T03:51:33.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.133+0000 7fe8b77fe700 1 -- 172.21.15.55:0/1789786760 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c800a9d0 con 0x7fe8d01566e0 2023-10-06T03:51:33.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.133+0000 7fe8b77fe700 1 -- 172.21.15.55:0/1789786760 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c800ad50 con 0x7fe8d01566e0 2023-10-06T03:51:33.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.133+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/1789786760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d015c1b0 con 0x7fe8d01566e0 2023-10-06T03:51:33.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.133+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/1789786760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d015dc20 con 0x7fe8d01566e0 2023-10-06T03:51:33.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.134+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/1789786760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d015e400 con 0x7fe8d01566e0 2023-10-06T03:51:33.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.135+0000 7fe8b77fe700 1 -- 172.21.15.55:0/1789786760 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8c8007300 con 0x7fe8d01566e0 2023-10-06T03:51:33.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.135+0000 7fe8b77fe700 1 --2- 172.21.15.55:0/1789786760 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe8bc066f30 0x7fe8bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:33.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.135+0000 7fe8b77fe700 1 -- 172.21.15.55:0/1789786760 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fe8c8084ac0 con 0x7fe8d01566e0 2023-10-06T03:51:33.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.136+0000 7fe8cf7fe700 1 --2- 172.21.15.55:0/1789786760 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe8bc066f30 0x7fe8bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:33.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.137+0000 7fe8cf7fe700 1 --2- 172.21.15.55:0/1789786760 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe8bc066f30 0x7fe8bc0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00b1080 tx=0x7fe8c0009040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:33.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.139+0000 7fe8b77fe700 1 -- 172.21.15.55:0/1789786760 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8c8051ac0 con 0x7fe8d01566e0 2023-10-06T03:51:33.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.384+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/1789786760 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8d004c3c0 con 0x7fe8d01566e0 2023-10-06T03:51:33.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.385+0000 7fe8b77fe700 1 -- 172.21.15.55:0/1789786760 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fe8c80550e0 con 0x7fe8d01566e0 2023-10-06T03:51:33.401 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:33.402 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1295635544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":1295635544},{"type":"v1","addr":"172.21.15.55:6835","nonce":1295635544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7737,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2115790576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2115790576},{"type":"v1","addr":"172.21.15.163:6834","nonce":2115790576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1021807391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1021807391},{"type":"v1","addr":"172.21.15.163:6836","nonce":1021807391}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:51:26.480340+0000","modified":"2023-10-06T03:51:30.017376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7743":{"gid":7743,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/4240717455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":4240717455},{"type":"v1","addr":"172.21.15.55:6834","nonce":4240717455}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T03:51:33.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.387+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/1789786760 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe8bc066f30 msgr2=0x7fe8bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:33.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.387+0000 7fe8d4f9d700 1 --2- 172.21.15.55:0/1789786760 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe8bc066f30 0x7fe8bc0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00b1080 tx=0x7fe8c0009040 comp rx=0 tx=0).stop 2023-10-06T03:51:33.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.387+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/1789786760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8d01566e0 msgr2=0x7fe8d00af330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:33.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.387+0000 7fe8d4f9d700 1 --2- 172.21.15.55:0/1789786760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8d01566e0 0x7fe8d00af330 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8001460 tx=0x7fe8c8001a40 comp rx=0 tx=0).stop 2023-10-06T03:51:33.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.387+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/1789786760 shutdown_connections 2023-10-06T03:51:33.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.387+0000 7fe8d4f9d700 1 --2- 172.21.15.55:0/1789786760 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe8bc066f30 0x7fe8bc0693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:33.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.387+0000 7fe8d4f9d700 1 --2- 172.21.15.55:0/1789786760 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8d01566e0 0x7fe8d00af330 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:33.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.387+0000 7fe8d4f9d700 1 --2- 172.21.15.55:0/1789786760 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8d00cb900 0x7fe8d0150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:33.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.387+0000 7fe8d4f9d700 1 --2- 172.21.15.55:0/1789786760 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8d0156310 0x7fe8d016a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:33.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.387+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/1789786760 >> 172.21.15.55:0/1789786760 conn(0x7fe8d00cc1f0 msgr2=0x7fe8d006ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:33.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.387+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/1789786760 shutdown_connections 2023-10-06T03:51:33.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:33.387+0000 7fe8d4f9d700 1 -- 172.21.15.55:0/1789786760 wait complete. 2023-10-06T03:51:33.404 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 61 2023-10-06T03:51:33.808 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:51:26.480340+0000', 'modified': '2023-10-06T03:51:30.017376+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7743}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7743': {'gid': 7743, 'name': 'c', 'rank': 0, 'incarnation': 57, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6834/4240717455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6832', 'nonce': 4240717455}, {'type': 'v1', 'addr': '172.21.15.55:6834', 'nonce': 4240717455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:51:33.824 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:51:33.824 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T03:51:34.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.016+0000 7f1d7979d700 1 Processor -- start 2023-10-06T03:51:34.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.016+0000 7f1d7979d700 1 -- start start 2023-10-06T03:51:34.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.017+0000 7f1d7979d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d74142ea0 0x7f1d740b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.017+0000 7f1d7979d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1d7406c0f0 0x7f1d7406a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.017+0000 7f1d7979d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d740b40f0 con 0x7f1d740ad870 2023-10-06T03:51:34.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.017+0000 7f1d7979d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d74069640 con 0x7f1d7406c0f0 2023-10-06T03:51:34.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.017+0000 7f1d7979d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d74069920 con 0x7f1d74142ea0 2023-10-06T03:51:34.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.018+0000 7f1d737fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1d740ad870 0x7f1d7406b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59032/0 (socket says 172.21.15.55:59032) 2023-10-06T03:51:34.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.018+0000 7f1d737fe700 1 -- 172.21.15.55:0/2404389588 learned_addr learned my addr 172.21.15.55:0/2404389588 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:34.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.018+0000 7f1d73fff700 1 --2- 172.21.15.55:0/2404389588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d74142ea0 0x7f1d740b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.018+0000 7f1d78f9c700 1 --2- 172.21.15.55:0/2404389588 >> v2:172.21.15.163:3300/0 conn(0x7f1d7406c0f0 0x7f1d7406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.018+0000 7f1d72ffd700 1 -- 172.21.15.55:0/2404389588 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803402776 0 0) 0x7f1d740b40f0 con 0x7f1d740ad870 2023-10-06T03:51:34.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.018+0000 7f1d72ffd700 1 -- 172.21.15.55:0/2404389588 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d5c003570 con 0x7f1d740ad870 2023-10-06T03:51:34.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.018+0000 7f1d78f9c700 1 -- 172.21.15.55:0/2404389588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d74142ea0 msgr2=0x7f1d740b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:34.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.018+0000 7f1d78f9c700 1 --2- 172.21.15.55:0/2404389588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d74142ea0 0x7f1d740b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.018+0000 7f1d78f9c700 1 -- 172.21.15.55:0/2404389588 >> v1:172.21.15.55:6789/0 conn(0x7f1d740ad870 legacy=0x7f1d7406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.018+0000 7f1d78f9c700 1 -- 172.21.15.55:0/2404389588 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d74069c00 con 0x7f1d7406c0f0 2023-10-06T03:51:34.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.018+0000 7f1d73fff700 1 --2- 172.21.15.55:0/2404389588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d74142ea0 0x7f1d740b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:34.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.018+0000 7f1d78f9c700 1 --2- 172.21.15.55:0/2404389588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7406c0f0 0x7f1d7406a120 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009e60 tx=0x7f1d6c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70f1bbed590c9740 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:34.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.019+0000 7f1d72ffd700 1 -- 172.21.15.55:0/2404389588 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c00e070 con 0x7f1d7406c0f0 2023-10-06T03:51:34.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.019+0000 7f1d72ffd700 1 -- 172.21.15.55:0/2404389588 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c00bab0 con 0x7f1d7406c0f0 2023-10-06T03:51:34.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.019+0000 7f1d72ffd700 1 -- 172.21.15.55:0/2404389588 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c00be70 con 0x7f1d7406c0f0 2023-10-06T03:51:34.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.019+0000 7f1d7979d700 1 -- 172.21.15.55:0/2404389588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7406c0f0 msgr2=0x7f1d7406a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.019+0000 7f1d7979d700 1 --2- 172.21.15.55:0/2404389588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7406c0f0 0x7f1d7406a120 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009e60 tx=0x7f1d6c009780 comp rx=0 tx=0).stop 2023-10-06T03:51:34.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.019+0000 7f1d7979d700 1 -- 172.21.15.55:0/2404389588 shutdown_connections 2023-10-06T03:51:34.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.019+0000 7f1d7979d700 1 --2- 172.21.15.55:0/2404389588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7406c0f0 0x7f1d7406a120 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.019+0000 7f1d7979d700 1 --2- 172.21.15.55:0/2404389588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d74142ea0 0x7f1d740b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.019+0000 7f1d7979d700 1 -- 172.21.15.55:0/2404389588 >> 172.21.15.55:0/2404389588 conn(0x7f1d740ce430 msgr2=0x7f1d740ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:34.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.020+0000 7f1d7979d700 1 -- 172.21.15.55:0/2404389588 shutdown_connections 2023-10-06T03:51:34.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.020+0000 7f1d7979d700 1 -- 172.21.15.55:0/2404389588 wait complete. 2023-10-06T03:51:34.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.021+0000 7f1d7979d700 1 Processor -- start 2023-10-06T03:51:34.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.021+0000 7f1d7979d700 1 -- start start 2023-10-06T03:51:34.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.022+0000 7f1d7979d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d7406c0f0 0x7f1d7416aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.022+0000 7f1d7979d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d74142ea0 0x7f1d7416a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.022+0000 7f1d7979d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7415eac0 0x7f1d74158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.022+0000 7f1d7979d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740c85b0 con 0x7f1d7406c0f0 2023-10-06T03:51:34.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.022+0000 7f1d7979d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d740c9130 con 0x7f1d7415eac0 2023-10-06T03:51:34.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.022+0000 7f1d7979d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7416b1e0 con 0x7f1d74142ea0 2023-10-06T03:51:34.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.022+0000 7f1d73fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d7406c0f0 0x7f1d7416aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.022+0000 7f1d73fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d7406c0f0 0x7f1d7416aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53646/0 (socket says 172.21.15.55:53646) 2023-10-06T03:51:34.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.022+0000 7f1d73fff700 1 -- 172.21.15.55:0/2172103803 learned_addr learned my addr 172.21.15.55:0/2172103803 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:34.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.023+0000 7f1d78f9c700 1 --2- 172.21.15.55:0/2172103803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7415eac0 0x7f1d74158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.023+0000 7f1d737fe700 1 --2- 172.21.15.55:0/2172103803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d74142ea0 0x7f1d7416a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.023+0000 7f1d73fff700 1 -- 172.21.15.55:0/2172103803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d74142ea0 msgr2=0x7f1d7416a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.023+0000 7f1d73fff700 1 --2- 172.21.15.55:0/2172103803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d74142ea0 0x7f1d7416a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.023+0000 7f1d73fff700 1 -- 172.21.15.55:0/2172103803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7415eac0 msgr2=0x7f1d74158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.023+0000 7f1d73fff700 1 --2- 172.21.15.55:0/2172103803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7415eac0 0x7f1d74158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.023+0000 7f1d73fff700 1 -- 172.21.15.55:0/2172103803 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d64009420 con 0x7f1d7406c0f0 2023-10-06T03:51:34.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.023+0000 7f1d73fff700 1 --2- 172.21.15.55:0/2172103803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d7406c0f0 0x7f1d7416aca0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400ece0 tx=0x7f1d640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:34.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.023+0000 7f1d5bfff700 1 -- 172.21.15.55:0/2172103803 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6400c850 con 0x7f1d7406c0f0 2023-10-06T03:51:34.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.023+0000 7f1d5bfff700 1 -- 172.21.15.55:0/2172103803 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6400c9b0 con 0x7f1d7406c0f0 2023-10-06T03:51:34.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.023+0000 7f1d5bfff700 1 -- 172.21.15.55:0/2172103803 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6400cd30 con 0x7f1d7406c0f0 2023-10-06T03:51:34.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.024+0000 7f1d7979d700 1 -- 172.21.15.55:0/2172103803 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6c009a20 con 0x7f1d7406c0f0 2023-10-06T03:51:34.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.024+0000 7f1d7979d700 1 -- 172.21.15.55:0/2172103803 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d740c8ae0 con 0x7f1d7406c0f0 2023-10-06T03:51:34.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.026+0000 7f1d5bfff700 1 -- 172.21.15.55:0/2172103803 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d64013680 con 0x7f1d7406c0f0 2023-10-06T03:51:34.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.027+0000 7f1d5bfff700 1 --2- 172.21.15.55:0/2172103803 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1d60066f30 0x7f1d600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.027+0000 7f1d5bfff700 1 -- 172.21.15.55:0/2172103803 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f1d64084af0 con 0x7f1d7406c0f0 2023-10-06T03:51:34.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.027+0000 7f1d737fe700 1 --2- 172.21.15.55:0/2172103803 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1d60066f30 0x7f1d600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.028+0000 7f1d737fe700 1 --2- 172.21.15.55:0/2172103803 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1d60066f30 0x7f1d600693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d740cce80 tx=0x7f1d68009040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.028+0000 7f1d891f2b80 1 Processor -- start 2023-10-06T03:51:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.028+0000 7f1d891f2b80 1 -- start start 2023-10-06T03:51:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.029+0000 7f1d891f2b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439a3e50 0x55894397ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.029+0000 7f1d891f2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439842c0 0x55894397f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.029+0000 7f1d891f2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894397f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.029+0000 7f1d891f2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589438d9a70 con 0x558943985a10 2023-10-06T03:51:34.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.029+0000 7f1d891f2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589438d7270 con 0x5589439842c0 2023-10-06T03:51:34.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.029+0000 7f1d891f2b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589438d3060 con 0x5589439a3e50 2023-10-06T03:51:34.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.029+0000 7f1d78f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894397f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.029+0000 7f1d78f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894397f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53658/0 (socket says 172.21.15.55:53658) 2023-10-06T03:51:34.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.029+0000 7f1d78f9c700 1 -- 172.21.15.55:0/51127007 learned_addr learned my addr 172.21.15.55:0/51127007 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:34.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.030+0000 7f1d737fe700 1 --2- 172.21.15.55:0/51127007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439842c0 0x55894397f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.030+0000 7f1d73fff700 1 --2- 172.21.15.55:0/51127007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439a3e50 0x55894397ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.030+0000 7f1d78f9c700 1 -- 172.21.15.55:0/51127007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439a3e50 msgr2=0x55894397ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:34.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.030+0000 7f1d78f9c700 1 --2- 172.21.15.55:0/51127007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439a3e50 0x55894397ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.030+0000 7f1d78f9c700 1 -- 172.21.15.55:0/51127007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439842c0 msgr2=0x55894397f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:34.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.030+0000 7f1d78f9c700 1 --2- 172.21.15.55:0/51127007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439842c0 0x55894397f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.030+0000 7f1d78f9c700 1 -- 172.21.15.55:0/51127007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55894376d270 con 0x558943985a10 2023-10-06T03:51:34.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.030+0000 7f1d78f9c700 1 --2- 172.21.15.55:0/51127007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894397f740 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1d7406f0d0 tx=0x7f1d6c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824e3eb034b48014 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:34.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.030+0000 7f1d597fa700 1 -- 172.21.15.55:0/51127007 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c00bc20 con 0x558943985a10 2023-10-06T03:51:34.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.030+0000 7f1d597fa700 1 -- 172.21.15.55:0/51127007 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c00bda0 con 0x558943985a10 2023-10-06T03:51:34.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.030+0000 7f1d597fa700 1 -- 172.21.15.55:0/51127007 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c012890 con 0x558943985a10 2023-10-06T03:51:34.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.031+0000 7f1d891f2b80 1 -- 172.21.15.55:0/51127007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 msgr2=0x55894397f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.031+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/51127007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894397f740 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f1d7406f0d0 tx=0x7f1d6c019000 comp rx=0 tx=0).stop 2023-10-06T03:51:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.031+0000 7f1d891f2b80 1 -- 172.21.15.55:0/51127007 shutdown_connections 2023-10-06T03:51:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.031+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/51127007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894397f740 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.031+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/51127007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439a3e50 0x55894397ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.031+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/51127007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439842c0 0x55894397f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.031+0000 7f1d891f2b80 1 -- 172.21.15.55:0/51127007 >> 172.21.15.55:0/51127007 conn(0x558943983e80 msgr2=0x5589439846b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:34.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.031+0000 7f1d891f2b80 1 -- 172.21.15.55:0/51127007 shutdown_connections 2023-10-06T03:51:34.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.031+0000 7f1d891f2b80 1 -- 172.21.15.55:0/51127007 wait complete. 2023-10-06T03:51:34.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.032+0000 7f1d891f2b80 1 -- start start 2023-10-06T03:51:34.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.032+0000 7f1d891f2b80 1 Processor -- start 2023-10-06T03:51:34.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.033+0000 7f1d891f2b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439842c0 0x5589439a2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.033+0000 7f1d891f2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894399c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.033+0000 7f1d891f2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439a3e50 0x55894399cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.033+0000 7f1d891f2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55894387e890 con 0x558943985a10 2023-10-06T03:51:34.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.033+0000 7f1d891f2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558943899ea0 con 0x5589439a3e50 2023-10-06T03:51:34.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.033+0000 7f1d891f2b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x558943755e30 con 0x5589439842c0 2023-10-06T03:51:34.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.033+0000 7f1d891f2b80 10 client.0 _ll_register_callbacks cb 0x558943946020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:51:34.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.033+0000 7f1d737fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894399c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.033+0000 7f1d737fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894399c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53674/0 (socket says 172.21.15.55:53674) 2023-10-06T03:51:34.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.033+0000 7f1d737fe700 1 -- 172.21.15.55:0/2448652449 learned_addr learned my addr 172.21.15.55:0/2448652449 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:34.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.034+0000 7f1d737fe700 1 -- 172.21.15.55:0/2448652449 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439842c0 msgr2=0x5589439a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:34.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.034+0000 7f1d737fe700 1 --2- 172.21.15.55:0/2448652449 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439842c0 0x5589439a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.034+0000 7f1d737fe700 1 -- 172.21.15.55:0/2448652449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439a3e50 msgr2=0x55894399cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:34.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.034+0000 7f1d78f9c700 1 --2- 172.21.15.55:0/2448652449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439a3e50 0x55894399cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.034+0000 7f1d737fe700 1 --2- 172.21.15.55:0/2448652449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439a3e50 0x55894399cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.034+0000 7f1d737fe700 1 -- 172.21.15.55:0/2448652449 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d74069c00 con 0x558943985a10 2023-10-06T03:51:34.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.034+0000 7f1d78f9c700 1 --2- 172.21.15.55:0/2448652449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439a3e50 0x55894399cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:51:34.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.034+0000 7f1d737fe700 1 --2- 172.21.15.55:0/2448652449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894399c5c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f1d680058d0 tx=0x7f1d68015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee929130f6add204 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:34.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.034+0000 7f1d527fc700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:51:34.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.034+0000 7f1d527fc700 1 -- 172.21.15.55:0/2448652449 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68021400 con 0x558943985a10 2023-10-06T03:51:34.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.034+0000 7f1d527fc700 1 -- 172.21.15.55:0/2448652449 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f1d6801c040 con 0x558943985a10 2023-10-06T03:51:34.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.034+0000 7f1d891f2b80 1 -- 172.21.15.55:0/2448652449 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558943977940 con 0x558943985a10 2023-10-06T03:51:34.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.035+0000 7f1d527fc700 1 -- 172.21.15.55:0/2448652449 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d68021770 con 0x558943985a10 2023-10-06T03:51:34.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.035+0000 7f1d891f2b80 10 client.7872 fetch_fsmap learned FSMap version 61 2023-10-06T03:51:34.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.035+0000 7f1d891f2b80 1 -- 172.21.15.55:0/2448652449 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55894370ffe0 con 0x558943985a10 2023-10-06T03:51:34.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.035+0000 7f1d527fc700 1 -- 172.21.15.55:0/2448652449 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558943977940 con 0x558943985a10 2023-10-06T03:51:34.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.035+0000 7f1d527fc700 1 -- 172.21.15.55:0/2448652449 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 61) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1d680218d0 con 0x558943985a10 2023-10-06T03:51:34.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.035+0000 7f1d891f2b80 10 client.7872 fetch_fsmap finished waiting for FSMap version 61 2023-10-06T03:51:34.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.035+0000 7f1d891f2b80 10 client.7872 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T03:51:34.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.035+0000 7f1d891f2b80 20 client.7872 populate_metadata read hostname 'smithi055' 2023-10-06T03:51:34.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.035+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/2448652449 >> [v2:172.21.15.55:6832/4240717455,v1:172.21.15.55:6834/4240717455] conn(0x5589439ae2f0 0x5589439ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.035+0000 7f1d891f2b80 4 client.7872 mds_command: new command op to 7743 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:51:34.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.035+0000 7f1d891f2b80 1 -- 172.21.15.55:0/2448652449 --> [v2:172.21.15.55:6832/4240717455,v1:172.21.15.55:6834/4240717455] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558943954870 con 0x5589439ae2f0 2023-10-06T03:51:34.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.036+0000 7f1d73fff700 1 --2- 172.21.15.55:0/2448652449 >> [v2:172.21.15.55:6832/4240717455,v1:172.21.15.55:6834/4240717455] conn(0x5589439ae2f0 0x5589439ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.036+0000 7f1d73fff700 1 --2- 172.21.15.55:0/2448652449 >> [v2:172.21.15.55:6832/4240717455,v1:172.21.15.55:6834/4240717455] conn(0x5589439ae2f0 0x5589439ae6c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e376d3318e852a1 server_cookie=5055f6bab61c1a69 in_seq=0 out_seq=0 2023-10-06T03:51:34.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.036+0000 7f1d527fc700 10 client.7872 ms_handle_connect on v2:172.21.15.55:6832/4240717455 2023-10-06T03:51:34.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.037+0000 7f1d527fc700 1 -- 172.21.15.55:0/2448652449 <== mds.0 v2:172.21.15.55:6832/4240717455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x558943954870 con 0x5589439ae2f0 2023-10-06T03:51:34.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.037+0000 7f1d527fc700 10 client.7872 handle_command_reply: tid=0 2023-10-06T03:51:34.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.037+0000 7f1d891f2b80 1 client.7872 shutdown 2023-10-06T03:51:34.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.037+0000 7f1d891f2b80 1 -- 172.21.15.55:0/2448652449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 msgr2=0x55894399c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.037+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/2448652449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894399c5c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f1d680058d0 tx=0x7f1d68015000 comp rx=0 tx=0).stop 2023-10-06T03:51:34.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.037+0000 7f1d891f2b80 1 -- 172.21.15.55:0/2448652449 shutdown_connections 2023-10-06T03:51:34.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.037+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/2448652449 >> [v2:172.21.15.55:6832/4240717455,v1:172.21.15.55:6834/4240717455] conn(0x5589439ae2f0 0x5589439ae6c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.037+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/2448652449 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439a3e50 0x55894399cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.037+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/2448652449 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439842c0 0x5589439a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.037+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/2448652449 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894399c5c0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.037+0000 7f1d891f2b80 1 -- 172.21.15.55:0/2448652449 >> 172.21.15.55:0/2448652449 conn(0x558943983e80 msgr2=0x558943984d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:34.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.037+0000 7f1d527fc700 0 client.7872 ms_handle_reset on v2:172.21.15.55:6832/4240717455 2023-10-06T03:51:34.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.037+0000 7f1d891f2b80 1 -- 172.21.15.55:0/2448652449 shutdown_connections 2023-10-06T03:51:34.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.037+0000 7f1d891f2b80 1 -- 172.21.15.55:0/2448652449 wait complete. 2023-10-06T03:51:34.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.038+0000 7f1d891f2b80 20 client.7872 trim_cache size 0 max 16384 2023-10-06T03:51:34.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.056+0000 7f1d891f2b80 1 Processor -- start 2023-10-06T03:51:34.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.056+0000 7f1d891f2b80 1 -- start start 2023-10-06T03:51:34.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d891f2b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439842c0 0x55894397fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d891f2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558943985a10 0x558943980d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d891f2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5589439a3e50 0x558943998500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d891f2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589439e35e0 con 0x5589439a3e50 2023-10-06T03:51:34.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d891f2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589439af960 con 0x558943985a10 2023-10-06T03:51:34.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d891f2b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589439622e0 con 0x5589439842c0 2023-10-06T03:51:34.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d78f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5589439a3e50 0x558943998500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d78f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5589439a3e50 0x558943998500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53682/0 (socket says 172.21.15.55:53682) 2023-10-06T03:51:34.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d78f9c700 1 -- 172.21.15.55:0/1343164248 learned_addr learned my addr 172.21.15.55:0/1343164248 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:34.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d73fff700 1 --2- 172.21.15.55:0/1343164248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439842c0 0x55894397fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d737fe700 1 --2- 172.21.15.55:0/1343164248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558943985a10 0x558943980d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d78f9c700 1 -- 172.21.15.55:0/1343164248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439842c0 msgr2=0x55894397fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:34.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d78f9c700 1 --2- 172.21.15.55:0/1343164248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439842c0 0x55894397fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d78f9c700 1 -- 172.21.15.55:0/1343164248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558943985a10 msgr2=0x558943980d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:34.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d78f9c700 1 --2- 172.21.15.55:0/1343164248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558943985a10 0x558943980d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d78f9c700 1 -- 172.21.15.55:0/1343164248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589438a1780 con 0x5589439a3e50 2023-10-06T03:51:34.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d73fff700 1 --2- 172.21.15.55:0/1343164248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439842c0 0x55894397fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:34.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.057+0000 7f1d737fe700 1 --2- 172.21.15.55:0/1343164248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558943985a10 0x558943980d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:34.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.058+0000 7f1d78f9c700 1 --2- 172.21.15.55:0/1343164248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5589439a3e50 0x558943998500 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c000f80 tx=0x7f1d6c021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6542ef5405d2da65 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:34.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.058+0000 7f1d59ffb700 1 -- 172.21.15.55:0/1343164248 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c02d070 con 0x5589439a3e50 2023-10-06T03:51:34.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.058+0000 7f1d59ffb700 1 -- 172.21.15.55:0/1343164248 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c012c40 con 0x5589439a3e50 2023-10-06T03:51:34.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.058+0000 7f1d59ffb700 1 -- 172.21.15.55:0/1343164248 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c012430 con 0x5589439a3e50 2023-10-06T03:51:34.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.058+0000 7f1d891f2b80 1 -- 172.21.15.55:0/1343164248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5589439a3e50 msgr2=0x558943998500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.059+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/1343164248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5589439a3e50 0x558943998500 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c000f80 tx=0x7f1d6c021040 comp rx=0 tx=0).stop 2023-10-06T03:51:34.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.059+0000 7f1d891f2b80 1 -- 172.21.15.55:0/1343164248 shutdown_connections 2023-10-06T03:51:34.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.059+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/1343164248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5589439a3e50 0x558943998500 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.059+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/1343164248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439842c0 0x55894397fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.059+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/1343164248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558943985a10 0x558943980d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.059+0000 7f1d891f2b80 1 -- 172.21.15.55:0/1343164248 >> 172.21.15.55:0/1343164248 conn(0x558943983e80 msgr2=0x558943979400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:34.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.059+0000 7f1d891f2b80 1 -- 172.21.15.55:0/1343164248 shutdown_connections 2023-10-06T03:51:34.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.059+0000 7f1d891f2b80 1 -- 172.21.15.55:0/1343164248 wait complete. 2023-10-06T03:51:34.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.059+0000 7f1d891f2b80 1 -- start start 2023-10-06T03:51:34.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.059+0000 7f1d891f2b80 1 Processor -- start 2023-10-06T03:51:34.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.060+0000 7f1d891f2b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439842c0 0x5589439797e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.060+0000 7f1d891f2b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894397ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.060+0000 7f1d891f2b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439a3e50 0x55894397f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.060+0000 7f1d891f2b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589439c9260 con 0x558943985a10 2023-10-06T03:51:34.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.060+0000 7f1d891f2b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558943a2fcf0 con 0x5589439842c0 2023-10-06T03:51:34.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.060+0000 7f1d891f2b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x558943981080 con 0x5589439a3e50 2023-10-06T03:51:34.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.060+0000 7f1d891f2b80 10 client.0 _ll_register_callbacks cb 0x5589439a1e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:51:34.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.061+0000 7f1d737fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894397ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.061+0000 7f1d737fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894397ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53690/0 (socket says 172.21.15.55:53690) 2023-10-06T03:51:34.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.061+0000 7f1d737fe700 1 -- 172.21.15.55:0/175346892 learned_addr learned my addr 172.21.15.55:0/175346892 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:34.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.061+0000 7f1d73fff700 1 --2- 172.21.15.55:0/175346892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439842c0 0x5589439797e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.061+0000 7f1d78f9c700 1 --2- 172.21.15.55:0/175346892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439a3e50 0x55894397f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.061+0000 7f1d737fe700 1 -- 172.21.15.55:0/175346892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439a3e50 msgr2=0x55894397f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:34.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.061+0000 7f1d737fe700 1 --2- 172.21.15.55:0/175346892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439a3e50 0x55894397f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.061+0000 7f1d737fe700 1 -- 172.21.15.55:0/175346892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439842c0 msgr2=0x5589439797e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:34.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.061+0000 7f1d737fe700 1 --2- 172.21.15.55:0/175346892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439842c0 0x5589439797e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.061+0000 7f1d737fe700 1 -- 172.21.15.55:0/175346892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55894376d270 con 0x558943985a10 2023-10-06T03:51:34.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.061+0000 7f1d737fe700 1 --2- 172.21.15.55:0/175346892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894397ead0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f1d680058d0 tx=0x7f1d68019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c27e30c4cda251 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:34.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.061+0000 7f1d527fc700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:51:34.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.062+0000 7f1d527fc700 1 -- 172.21.15.55:0/175346892 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68014c10 con 0x558943985a10 2023-10-06T03:51:34.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.062+0000 7f1d527fc700 1 -- 172.21.15.55:0/175346892 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f1d6803bda0 con 0x558943985a10 2023-10-06T03:51:34.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.062+0000 7f1d891f2b80 1 -- 172.21.15.55:0/175346892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d740c9d30 con 0x558943985a10 2023-10-06T03:51:34.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.062+0000 7f1d527fc700 1 -- 172.21.15.55:0/175346892 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d68029560 con 0x558943985a10 2023-10-06T03:51:34.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.062+0000 7f1d891f2b80 10 client.7878 fetch_fsmap learned FSMap version 61 2023-10-06T03:51:34.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.062+0000 7f1d891f2b80 1 -- 172.21.15.55:0/175346892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x558943888470 con 0x558943985a10 2023-10-06T03:51:34.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.062+0000 7f1d527fc700 1 -- 172.21.15.55:0/175346892 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d740c9d30 con 0x558943985a10 2023-10-06T03:51:34.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.062+0000 7f1d527fc700 1 -- 172.21.15.55:0/175346892 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 61) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1d680296c0 con 0x558943985a10 2023-10-06T03:51:34.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.062+0000 7f1d891f2b80 10 client.7878 fetch_fsmap finished waiting for FSMap version 61 2023-10-06T03:51:34.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.062+0000 7f1d891f2b80 10 client.7878 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T03:51:34.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.062+0000 7f1d891f2b80 20 client.7878 populate_metadata read hostname 'smithi055' 2023-10-06T03:51:34.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.062+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/175346892 >> [v2:172.21.15.55:6832/4240717455,v1:172.21.15.55:6834/4240717455] conn(0x558943a2df10 0x558943a2e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.062+0000 7f1d891f2b80 4 client.7878 mds_command: new command op to 7743 tid=0[{"prefix": "status"}] 2023-10-06T03:51:34.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.062+0000 7f1d891f2b80 1 -- 172.21.15.55:0/175346892 --> [v2:172.21.15.55:6832/4240717455,v1:172.21.15.55:6834/4240717455] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f1d6c00bda0 con 0x558943a2df10 2023-10-06T03:51:34.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d73fff700 1 --2- 172.21.15.55:0/175346892 >> [v2:172.21.15.55:6832/4240717455,v1:172.21.15.55:6834/4240717455] conn(0x558943a2df10 0x558943a2e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d73fff700 1 --2- 172.21.15.55:0/175346892 >> [v2:172.21.15.55:6832/4240717455,v1:172.21.15.55:6834/4240717455] conn(0x558943a2df10 0x558943a2e2e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2cac8b413557854 server_cookie=e8180c3165349b04 in_seq=0 out_seq=0 2023-10-06T03:51:34.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d527fc700 10 client.7878 ms_handle_connect on v2:172.21.15.55:6832/4240717455 2023-10-06T03:51:34.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d527fc700 1 -- 172.21.15.55:0/175346892 <== mds.0 v2:172.21.15.55:6832/4240717455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x7f1d6c00bda0 con 0x558943a2df10 2023-10-06T03:51:34.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d527fc700 10 client.7878 handle_command_reply: tid=0 2023-10-06T03:51:34.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d891f2b80 1 client.7878 shutdown 2023-10-06T03:51:34.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d891f2b80 1 -- 172.21.15.55:0/175346892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 msgr2=0x55894397ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/175346892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894397ead0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f1d680058d0 tx=0x7f1d68019000 comp rx=0 tx=0).stop 2023-10-06T03:51:34.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d891f2b80 1 -- 172.21.15.55:0/175346892 shutdown_connections 2023-10-06T03:51:34.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/175346892 >> [v2:172.21.15.55:6832/4240717455,v1:172.21.15.55:6834/4240717455] conn(0x558943a2df10 0x558943a2e2e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/175346892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5589439a3e50 0x55894397f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/175346892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5589439842c0 0x5589439797e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d891f2b80 1 --2- 172.21.15.55:0/175346892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558943985a10 0x55894397ead0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d527fc700 0 client.7878 ms_handle_reset on v2:172.21.15.55:6832/4240717455 2023-10-06T03:51:34.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.063+0000 7f1d891f2b80 1 -- 172.21.15.55:0/175346892 >> 172.21.15.55:0/175346892 conn(0x558943983e80 msgr2=0x5589439cdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:34.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.064+0000 7f1d891f2b80 1 -- 172.21.15.55:0/175346892 shutdown_connections 2023-10-06T03:51:34.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.064+0000 7f1d891f2b80 1 -- 172.21.15.55:0/175346892 wait complete. 2023-10-06T03:51:34.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.064+0000 7f1d891f2b80 20 client.7878 trim_cache size 0 max 16384 2023-10-06T03:51:34.290 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:51:34.291 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "8132c8ba-ee1c-41a2-b781-913925034ab6", 2023-10-06T03:51:34.291 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2023-10-06T03:51:34.291 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 7743, 2023-10-06T03:51:34.291 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2023-10-06T03:51:34.291 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2023-10-06T03:51:34.291 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2023-10-06T03:51:34.291 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 7.0780782779999996, 2023-10-06T03:51:34.291 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 61, 2023-10-06T03:51:34.291 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 123, 2023-10-06T03:51:34.291 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 123, 2023-10-06T03:51:34.291 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 13.791323633999999 2023-10-06T03:51:34.291 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:51:34.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.064+0000 7f1d7979d700 1 -- 172.21.15.55:0/2172103803 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1d60066f30 msgr2=0x7f1d600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.064+0000 7f1d7979d700 1 --2- 172.21.15.55:0/2172103803 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1d60066f30 0x7f1d600693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1d740cce80 tx=0x7f1d68009040 comp rx=0 tx=0).stop 2023-10-06T03:51:34.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.064+0000 7f1d7979d700 1 -- 172.21.15.55:0/2172103803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d7406c0f0 msgr2=0x7f1d7416aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.064+0000 7f1d7979d700 1 --2- 172.21.15.55:0/2172103803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d7406c0f0 0x7f1d7416aca0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400ece0 tx=0x7f1d640091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.064+0000 7f1d7979d700 1 -- 172.21.15.55:0/2172103803 shutdown_connections 2023-10-06T03:51:34.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.064+0000 7f1d7979d700 1 --2- 172.21.15.55:0/2172103803 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1d60066f30 0x7f1d600693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.064+0000 7f1d7979d700 1 --2- 172.21.15.55:0/2172103803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d7415eac0 0x7f1d74158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.064+0000 7f1d7979d700 1 --2- 172.21.15.55:0/2172103803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d7406c0f0 0x7f1d7416aca0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.064+0000 7f1d7979d700 1 --2- 172.21.15.55:0/2172103803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d74142ea0 0x7f1d7416a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.064+0000 7f1d7979d700 1 -- 172.21.15.55:0/2172103803 >> 172.21.15.55:0/2172103803 conn(0x7f1d740ce430 msgr2=0x7f1d74141f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:34.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.065+0000 7f1d7979d700 1 -- 172.21.15.55:0/2172103803 shutdown_connections 2023-10-06T03:51:34.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.065+0000 7f1d7979d700 1 -- 172.21.15.55:0/2172103803 wait complete. 2023-10-06T03:51:34.295 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:51:34.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.480+0000 7ff2fb19d700 1 Processor -- start 2023-10-06T03:51:34.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.481+0000 7ff2fb19d700 1 -- start start 2023-10-06T03:51:34.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.482+0000 7ff2fb19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff2f40c8830 0x7ff2f4164800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.482+0000 7ff2fb19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f40b1410 con 0x7ff2f40c6c60 2023-10-06T03:51:34.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.482+0000 7ff2fb19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f4063230 con 0x7ff2f41595a0 2023-10-06T03:51:34.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.482+0000 7ff2fb19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f40b2b90 con 0x7ff2f40c8830 2023-10-06T03:51:34.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.482+0000 7ff2f999a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff2f40c6c60 0x7ff2f40c8120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59046/0 (socket says 172.21.15.55:59046) 2023-10-06T03:51:34.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.482+0000 7ff2f999a700 1 -- 172.21.15.55:0/1802464105 learned_addr learned my addr 172.21.15.55:0/1802464105 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:34.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.482+0000 7ff2fa99c700 1 --2- 172.21.15.55:0/1802464105 >> v2:172.21.15.163:3300/0 conn(0x7ff2f40c8830 0x7ff2f4164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.483+0000 7ff2f9199700 1 -- 172.21.15.55:0/1802464105 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737857006 0 0) 0x7ff2f40b1410 con 0x7ff2f40c6c60 2023-10-06T03:51:34.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.483+0000 7ff2f9199700 1 -- 172.21.15.55:0/1802464105 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e0003570 con 0x7ff2f40c6c60 2023-10-06T03:51:34.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.483+0000 7ff2fa99c700 1 -- 172.21.15.55:0/1802464105 >> v1:172.21.15.163:6789/0 conn(0x7ff2f41595a0 legacy=0x7ff2f40b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.483+0000 7ff2fa99c700 1 -- 172.21.15.55:0/1802464105 >> v1:172.21.15.55:6789/0 conn(0x7ff2f40c6c60 legacy=0x7ff2f40c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.483+0000 7ff2fa99c700 1 -- 172.21.15.55:0/1802464105 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f40634f0 con 0x7ff2f40c8830 2023-10-06T03:51:34.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.483+0000 7ff2fa99c700 1 --2- 172.21.15.55:0/1802464105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2f40c8830 0x7ff2f4164800 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009e60 tx=0x7ff2ec009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=effe9d9d2a44aeb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:34.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.484+0000 7ff2f9199700 1 -- 172.21.15.55:0/1802464105 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec00e070 con 0x7ff2f40c8830 2023-10-06T03:51:34.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.484+0000 7ff2f9199700 1 -- 172.21.15.55:0/1802464105 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec00bb60 con 0x7ff2f40c8830 2023-10-06T03:51:34.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.484+0000 7ff2f9199700 1 -- 172.21.15.55:0/1802464105 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec012430 con 0x7ff2f40c8830 2023-10-06T03:51:34.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.484+0000 7ff2fb19d700 1 -- 172.21.15.55:0/1802464105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2f40c8830 msgr2=0x7ff2f4164800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.484+0000 7ff2fb19d700 1 --2- 172.21.15.55:0/1802464105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2f40c8830 0x7ff2f4164800 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009e60 tx=0x7ff2ec009780 comp rx=0 tx=0).stop 2023-10-06T03:51:34.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.484+0000 7ff2fb19d700 1 -- 172.21.15.55:0/1802464105 shutdown_connections 2023-10-06T03:51:34.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.484+0000 7ff2fb19d700 1 --2- 172.21.15.55:0/1802464105 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2f40c8830 0x7ff2f4164800 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.484+0000 7ff2fb19d700 1 -- 172.21.15.55:0/1802464105 >> 172.21.15.55:0/1802464105 conn(0x7ff2f406ddb0 msgr2=0x7ff2f406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:34.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.484+0000 7ff2fb19d700 1 -- 172.21.15.55:0/1802464105 shutdown_connections 2023-10-06T03:51:34.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.484+0000 7ff2fb19d700 1 -- 172.21.15.55:0/1802464105 wait complete. 2023-10-06T03:51:34.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.486+0000 7ff2fb19d700 1 Processor -- start 2023-10-06T03:51:34.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.486+0000 7ff2fb19d700 1 -- start start 2023-10-06T03:51:34.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2fb19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2f40c6c60 0x7ff2f413d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2fb19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2f40c8830 0x7ff2f414d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2fb19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2f41595a0 0x7ff2f414b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2fb19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f414cd20 con 0x7ff2f40c8830 2023-10-06T03:51:34.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2fb19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f414c110 con 0x7ff2f40c6c60 2023-10-06T03:51:34.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2fb19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f414c3f0 con 0x7ff2f41595a0 2023-10-06T03:51:34.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2f999a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2f40c8830 0x7ff2f414d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2f999a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2f40c8830 0x7ff2f414d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53704/0 (socket says 172.21.15.55:53704) 2023-10-06T03:51:34.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2f999a700 1 -- 172.21.15.55:0/2678887024 learned_addr learned my addr 172.21.15.55:0/2678887024 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:34.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2fa99c700 1 --2- 172.21.15.55:0/2678887024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2f41595a0 0x7ff2f414b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2fa19b700 1 --2- 172.21.15.55:0/2678887024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2f40c6c60 0x7ff2f413d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2f999a700 1 -- 172.21.15.55:0/2678887024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2f41595a0 msgr2=0x7ff2f414b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2f999a700 1 --2- 172.21.15.55:0/2678887024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2f41595a0 0x7ff2f414b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2f999a700 1 -- 172.21.15.55:0/2678887024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2f40c6c60 msgr2=0x7ff2f413d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2f999a700 1 --2- 172.21.15.55:0/2678887024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2f40c6c60 0x7ff2f413d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.487+0000 7ff2f999a700 1 -- 172.21.15.55:0/2678887024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec009a20 con 0x7ff2f40c8830 2023-10-06T03:51:34.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.488+0000 7ff2f999a700 1 --2- 172.21.15.55:0/2678887024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2f40c8830 0x7ff2f414d1b0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400c7c0 tx=0x7ff2e4007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:34.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.488+0000 7ff2ea7fc700 1 -- 172.21.15.55:0/2678887024 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4005c00 con 0x7ff2f40c8830 2023-10-06T03:51:34.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.488+0000 7ff2ea7fc700 1 -- 172.21.15.55:0/2678887024 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e4005d60 con 0x7ff2f40c8830 2023-10-06T03:51:34.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.488+0000 7ff2ea7fc700 1 -- 172.21.15.55:0/2678887024 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4007320 con 0x7ff2f40c8830 2023-10-06T03:51:34.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.488+0000 7ff2fb19d700 1 -- 172.21.15.55:0/2678887024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f414c6d0 con 0x7ff2f40c8830 2023-10-06T03:51:34.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.488+0000 7ff2fb19d700 1 -- 172.21.15.55:0/2678887024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f413d9d0 con 0x7ff2f40c8830 2023-10-06T03:51:34.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.489+0000 7ff2fb19d700 1 -- 172.21.15.55:0/2678887024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f404c3c0 con 0x7ff2f40c8830 2023-10-06T03:51:34.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.490+0000 7ff2ea7fc700 1 -- 172.21.15.55:0/2678887024 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff2e4007690 con 0x7ff2f40c8830 2023-10-06T03:51:34.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.491+0000 7ff2ea7fc700 1 --2- 172.21.15.55:0/2678887024 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff2d8066f30 0x7ff2d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:34.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.491+0000 7ff2ea7fc700 1 -- 172.21.15.55:0/2678887024 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7ff2e4084410 con 0x7ff2f40c8830 2023-10-06T03:51:34.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.492+0000 7ff2fa19b700 1 --2- 172.21.15.55:0/2678887024 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff2d8066f30 0x7ff2d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:34.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.493+0000 7ff2fa19b700 1 --2- 172.21.15.55:0/2678887024 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff2d8066f30 0x7ff2d80693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00037f0 tx=0x7ff2f0009040 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:34.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.496+0000 7ff2ea7fc700 1 -- 172.21.15.55:0/2678887024 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2e4051370 con 0x7ff2f40c8830 2023-10-06T03:51:34.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.742+0000 7ff2fb19d700 1 -- 172.21.15.55:0/2678887024 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff2f414ce60 con 0x7ff2f40c8830 2023-10-06T03:51:34.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.742+0000 7ff2ea7fc700 1 -- 172.21.15.55:0/2678887024 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff2e4054990 con 0x7ff2f40c8830 2023-10-06T03:51:34.744 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:51:34.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.744+0000 7ff2fb19d700 1 -- 172.21.15.55:0/2678887024 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff2d8066f30 msgr2=0x7ff2d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.744+0000 7ff2fb19d700 1 --2- 172.21.15.55:0/2678887024 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff2d8066f30 0x7ff2d80693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00037f0 tx=0x7ff2f0009040 comp rx=0 tx=0).stop 2023-10-06T03:51:34.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.744+0000 7ff2fb19d700 1 -- 172.21.15.55:0/2678887024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2f40c8830 msgr2=0x7ff2f414d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:34.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.744+0000 7ff2fb19d700 1 --2- 172.21.15.55:0/2678887024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2f40c8830 0x7ff2f414d1b0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400c7c0 tx=0x7ff2e4007dd0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.744+0000 7ff2fb19d700 1 -- 172.21.15.55:0/2678887024 shutdown_connections 2023-10-06T03:51:34.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.744+0000 7ff2fb19d700 1 --2- 172.21.15.55:0/2678887024 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff2d8066f30 0x7ff2d80693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.744+0000 7ff2fb19d700 1 --2- 172.21.15.55:0/2678887024 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff2f41595a0 0x7ff2f414b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.744+0000 7ff2fb19d700 1 --2- 172.21.15.55:0/2678887024 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff2f40c6c60 0x7ff2f413d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.744+0000 7ff2fb19d700 1 --2- 172.21.15.55:0/2678887024 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff2f40c8830 0x7ff2f414d1b0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:34.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.744+0000 7ff2fb19d700 1 -- 172.21.15.55:0/2678887024 >> 172.21.15.55:0/2678887024 conn(0x7ff2f406ddb0 msgr2=0x7ff2f4154d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:34.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.745+0000 7ff2fb19d700 1 -- 172.21.15.55:0/2678887024 shutdown_connections 2023-10-06T03:51:34.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:34.745+0000 7ff2fb19d700 1 -- 172.21.15.55:0/2678887024 wait complete. 2023-10-06T03:51:34.763 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:51:34.764 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:51:34.764 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2023-10-06T03:51:34.764 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:51:34.764 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:51:34.764 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:51:34.764 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:51:34.764 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:51:34.764 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:51:34.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:51:34.764 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2023-10-06T03:51:34.802 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:51:34.804 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:51:34.804 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:51:34.804 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:51:34.804 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T03:51:34.804 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:51:34.804 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:51:34.804 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:51:34.804 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2023-10-06T03:51:34.804 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:51:34.804 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:51:34.805 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2023-10-06T03:51:34.805 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:51:34.805 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:51:34.805 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2023-10-06T03:51:34.805 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T03:51:34.805 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 41147sec preferred_lft 41147sec 2023-10-06T03:51:34.805 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link 2023-10-06T03:51:34.805 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:51:34.805 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:51:34.805 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:51:34.806 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2023-10-06T03:51:34.806 INFO:teuthology.orchestra.run.smithi055.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:51:34.806 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff 2023-10-06T03:51:34.806 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:51:34.806 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:51:34.806 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::1099:8eff:fe6d:102/64 scope link 2023-10-06T03:51:34.806 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:51:34.806 INFO:teuthology.orchestra.run.smithi055.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:51:34.806 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether f2:5d:53:d2:00:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:51:34.806 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f05d:53ff:fed2:a/64 scope link 2023-10-06T03:51:34.807 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:51:34.807 INFO:teuthology.orchestra.run.smithi055.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:51:34.807 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:51:34.807 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::e037:33ff:feb7:6c8f/64 scope link 2023-10-06T03:51:34.807 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:51:34.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:51:34.807 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:51:34.902 INFO:teuthology.orchestra.run.smithi055.stdout:1 2023-10-06T03:51:34.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:51:34.905 DEBUG:teuthology.orchestra.run.smithi055:> route 2023-10-06T03:51:34.944 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2023-10-06T03:51:34.944 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:51:34.944 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T03:51:34.944 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T03:51:34.944 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:51:34.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:51:34.945 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:51:34.945 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:51:34.945 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T03:51:34.945 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T03:51:34.945 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T03:51:34.945 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:51:35.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:51:35.270 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T03:51:35.309 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:51:35.331 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:51:35.331 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:51:35.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:51:35.332 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2023-10-06T03:51:35.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:51:35.427 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:51:35.464 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:51:35.478 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:51:35.478 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:51:35.540 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T03:51:35.541 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:51:35.541 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:51:35.672 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:51:35.686 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:51:35.686 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:51:35.688 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:51:35.688 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:51:35.689 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:51:35.689 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:51:35.689 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:51:35.710 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:51:35.721 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:51:35.721 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:51:35.721 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:51:35.721 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:51:35.721 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T03:51:35.722 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T03:51:35.722 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T03:51:35.723 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:51:35.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:51:35.723 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:51:35.786 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:51:36.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.165+0000 7fe38539d700 1 Processor -- start 2023-10-06T03:51:36.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.166+0000 7fe38539d700 1 -- start start 2023-10-06T03:51:36.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.167+0000 7fe38539d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe38006a0a0 0x7fe38015b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:36.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.167+0000 7fe38539d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe38006dd20 0x7fe38015b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:36.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.167+0000 7fe38539d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38006ae70 con 0x7fe38006d950 2023-10-06T03:51:36.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.167+0000 7fe38539d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3800c8a60 con 0x7fe38006a0a0 2023-10-06T03:51:36.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.167+0000 7fe38539d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38006a540 con 0x7fe38006dd20 2023-10-06T03:51:36.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.167+0000 7fe37f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe38006a0a0 0x7fe38015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:36.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.167+0000 7fe384b9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe38006dd20 0x7fe38015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:36.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.167+0000 7fe37f7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe38006a0a0 0x7fe38015b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:41454/0 (socket says 172.21.15.55:41454) 2023-10-06T03:51:36.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.167+0000 7fe384b9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe38006dd20 0x7fe38015b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:58636/0 (socket says 172.21.15.55:58636) 2023-10-06T03:51:36.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.167+0000 7fe37f7fe700 1 -- 172.21.15.55:0/3975622547 learned_addr learned my addr 172.21.15.55:0/3975622547 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:36.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.168+0000 7fe37f7fe700 1 -- 172.21.15.55:0/3975622547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe38006dd20 msgr2=0x7fe38015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:36.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.168+0000 7fe37f7fe700 1 --2- 172.21.15.55:0/3975622547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe38006dd20 0x7fe38015b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:36.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.168+0000 7fe37f7fe700 1 -- 172.21.15.55:0/3975622547 >> v1:172.21.15.163:6789/0 conn(0x7fe38006d950 legacy=0x7fe3800c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:36.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.168+0000 7fe37f7fe700 1 -- 172.21.15.55:0/3975622547 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38006a820 con 0x7fe38006a0a0 2023-10-06T03:51:36.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.168+0000 7fe384b9c700 1 --2- 172.21.15.55:0/3975622547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe38006dd20 0x7fe38015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:36.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.168+0000 7fe37f7fe700 1 --2- 172.21.15.55:0/3975622547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe38006a0a0 0x7fe38015b410 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe3740099b0 tx=0x7fe374009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27c8bde13931b225 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:36.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.168+0000 7fe37effd700 1 -- 172.21.15.55:0/3975622547 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374009ea0 con 0x7fe38006a0a0 2023-10-06T03:51:36.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.168+0000 7fe37effd700 1 -- 172.21.15.55:0/3975622547 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37400b860 con 0x7fe38006a0a0 2023-10-06T03:51:36.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.168+0000 7fe37effd700 1 -- 172.21.15.55:0/3975622547 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400bc00 con 0x7fe38006a0a0 2023-10-06T03:51:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.169+0000 7fe38539d700 1 -- 172.21.15.55:0/3975622547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe38006a0a0 msgr2=0x7fe38015b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.169+0000 7fe38539d700 1 --2- 172.21.15.55:0/3975622547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe38006a0a0 0x7fe38015b410 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe3740099b0 tx=0x7fe374009250 comp rx=0 tx=0).stop 2023-10-06T03:51:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.169+0000 7fe38539d700 1 -- 172.21.15.55:0/3975622547 shutdown_connections 2023-10-06T03:51:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.169+0000 7fe38539d700 1 --2- 172.21.15.55:0/3975622547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe38006dd20 0x7fe38015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.169+0000 7fe38539d700 1 --2- 172.21.15.55:0/3975622547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe38006a0a0 0x7fe38015b410 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.169+0000 7fe38539d700 1 -- 172.21.15.55:0/3975622547 >> 172.21.15.55:0/3975622547 conn(0x7fe3800cc1f0 msgr2=0x7fe3800cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.169+0000 7fe38539d700 1 -- 172.21.15.55:0/3975622547 shutdown_connections 2023-10-06T03:51:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.169+0000 7fe38539d700 1 -- 172.21.15.55:0/3975622547 wait complete. 2023-10-06T03:51:36.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.170+0000 7fe38539d700 1 Processor -- start 2023-10-06T03:51:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.171+0000 7fe38539d700 1 -- start start 2023-10-06T03:51:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.171+0000 7fe38539d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe38006a0a0 0x7fe38014d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.171+0000 7fe38539d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe38006d950 0x7fe380151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe38539d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe38006dd20 0x7fe3800c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe38539d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3800b0f70 con 0x7fe38006d950 2023-10-06T03:51:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe38539d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3800b00b0 con 0x7fe38006dd20 2023-10-06T03:51:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe38539d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe380152ed0 con 0x7fe38006a0a0 2023-10-06T03:51:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe37f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe38006d950 0x7fe380151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe37f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe38006d950 0x7fe380151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53706/0 (socket says 172.21.15.55:53706) 2023-10-06T03:51:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe37f7fe700 1 -- 172.21.15.55:0/2614977530 learned_addr learned my addr 172.21.15.55:0/2614977530 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:36.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe37ffff700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe38006a0a0 0x7fe38014d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:36.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe384b9c700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe38006dd20 0x7fe3800c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:36.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe37f7fe700 1 -- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe38006a0a0 msgr2=0x7fe38014d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:36.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe37f7fe700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe38006a0a0 0x7fe38014d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:36.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe37f7fe700 1 -- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe38006dd20 msgr2=0x7fe3800c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:36.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe37f7fe700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe38006dd20 0x7fe3800c47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:36.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe37f7fe700 1 -- 172.21.15.55:0/2614977530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe378009a20 con 0x7fe38006d950 2023-10-06T03:51:36.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.172+0000 7fe384b9c700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe38006dd20 0x7fe3800c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:51:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.173+0000 7fe37f7fe700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe38006d950 0x7fe380151fe0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe374006fd0 tx=0x7fe374017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.173+0000 7fe367fff700 1 -- 172.21.15.55:0/2614977530 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400bbe0 con 0x7fe38006d950 2023-10-06T03:51:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.173+0000 7fe367fff700 1 -- 172.21.15.55:0/2614977530 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe374011420 con 0x7fe38006d950 2023-10-06T03:51:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.173+0000 7fe367fff700 1 -- 172.21.15.55:0/2614977530 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3740117c0 con 0x7fe38006d950 2023-10-06T03:51:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.173+0000 7fe38539d700 1 -- 172.21.15.55:0/2614977530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3740094f0 con 0x7fe38006d950 2023-10-06T03:51:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.173+0000 7fe38539d700 1 -- 172.21.15.55:0/2614977530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3800b05d0 con 0x7fe38006d950 2023-10-06T03:51:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.174+0000 7fe38539d700 1 -- 172.21.15.55:0/2614977530 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3800b10b0 con 0x7fe38006d950 2023-10-06T03:51:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.175+0000 7fe367fff700 1 -- 172.21.15.55:0/2614977530 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe374011bf0 con 0x7fe38006d950 2023-10-06T03:51:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.176+0000 7fe367fff700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe36c066f30 0x7fe36c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:36.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.176+0000 7fe367fff700 1 -- 172.21.15.55:0/2614977530 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fe3740860d0 con 0x7fe38006d950 2023-10-06T03:51:36.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.176+0000 7fe37ffff700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe36c066f30 0x7fe36c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:36.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.177+0000 7fe37ffff700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe36c066f30 0x7fe36c0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe370007fd0 tx=0x7fe370007df0 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:36.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.180+0000 7fe367fff700 1 -- 172.21.15.55:0/2614977530 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3800b10b0 con 0x7fe38006d950 2023-10-06T03:51:36.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.413+0000 7fe38539d700 1 -- 172.21.15.55:0/2614977530 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3801585c0 con 0x7fe36c066f30 2023-10-06T03:51:36.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.415+0000 7fe367fff700 1 -- 172.21.15.55:0/2614977530 <== mgr.7228 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe3801585c0 con 0x7fe36c066f30 2023-10-06T03:51:36.420 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:51:36.421 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:51:36.421 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephfs" 2023-10-06T03:51:36.421 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:51:36.421 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:51:36.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.417+0000 7fe38539d700 1 -- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe36c066f30 msgr2=0x7fe36c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:36.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.417+0000 7fe38539d700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe36c066f30 0x7fe36c0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe370007fd0 tx=0x7fe370007df0 comp rx=0 tx=0).stop 2023-10-06T03:51:36.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.417+0000 7fe38539d700 1 -- 172.21.15.55:0/2614977530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe38006d950 msgr2=0x7fe380151fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:36.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.417+0000 7fe38539d700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe38006d950 0x7fe380151fe0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe374006fd0 tx=0x7fe374017000 comp rx=0 tx=0).stop 2023-10-06T03:51:36.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.417+0000 7fe38539d700 1 -- 172.21.15.55:0/2614977530 shutdown_connections 2023-10-06T03:51:36.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.417+0000 7fe38539d700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe36c066f30 0x7fe36c0693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:36.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.417+0000 7fe38539d700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe38006dd20 0x7fe3800c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:36.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.417+0000 7fe38539d700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe38006a0a0 0x7fe38014d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:36.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.417+0000 7fe38539d700 1 --2- 172.21.15.55:0/2614977530 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe38006d950 0x7fe380151fe0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:36.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.417+0000 7fe38539d700 1 -- 172.21.15.55:0/2614977530 >> 172.21.15.55:0/2614977530 conn(0x7fe3800cc1f0 msgr2=0x7fe3800b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:36.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.418+0000 7fe38539d700 1 -- 172.21.15.55:0/2614977530 shutdown_connections 2023-10-06T03:51:36.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.418+0000 7fe38539d700 1 -- 172.21.15.55:0/2614977530 wait complete. 2023-10-06T03:51:36.436 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T03:51:36.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.624+0000 7fde7f59e700 1 Processor -- start 2023-10-06T03:51:36.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.624+0000 7fde7f59e700 1 -- start start 2023-10-06T03:51:36.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.626+0000 7fde7f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde800cb1a0 0x7fde8015a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:36.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.626+0000 7fde7f59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8006ebd0 con 0x7fde800ae0d0 2023-10-06T03:51:36.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.626+0000 7fde7f59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8015ad20 con 0x7fde8015a8a0 2023-10-06T03:51:36.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.626+0000 7fde7f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8015b2e0 con 0x7fde800cb1a0 2023-10-06T03:51:36.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.626+0000 7fde7ed9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fde800ae0d0 0x7fde800c9db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59058/0 (socket says 172.21.15.55:59058) 2023-10-06T03:51:36.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.626+0000 7fde7ed9d700 1 -- 172.21.15.55:0/3689534961 learned_addr learned my addr 172.21.15.55:0/3689534961 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:36.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.626+0000 7fde7e59c700 1 --2- 172.21.15.55:0/3689534961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde800cb1a0 0x7fde8015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:36.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.626+0000 7fde7d59a700 1 -- 172.21.15.55:0/3689534961 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3652590074 0 0) 0x7fde8006ebd0 con 0x7fde800ae0d0 2023-10-06T03:51:36.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.626+0000 7fde7d59a700 1 -- 172.21.15.55:0/3689534961 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde68003570 con 0x7fde800ae0d0 2023-10-06T03:51:36.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.627+0000 7fde7d59a700 1 -- 172.21.15.55:0/3689534961 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 581348148 0 0) 0x7fde8015ad20 con 0x7fde8015a8a0 2023-10-06T03:51:36.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.627+0000 7fde7d59a700 1 -- 172.21.15.55:0/3689534961 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde8006ebd0 con 0x7fde8015a8a0 2023-10-06T03:51:36.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.627+0000 7fde7d59a700 1 -- 172.21.15.55:0/3689534961 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2940570123 0 0) 0x7fde68003570 con 0x7fde800ae0d0 2023-10-06T03:51:36.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.627+0000 7fde7e59c700 1 -- 172.21.15.55:0/3689534961 >> v1:172.21.15.163:6789/0 conn(0x7fde8015a8a0 legacy=0x7fde8006f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:36.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.627+0000 7fde7e59c700 1 -- 172.21.15.55:0/3689534961 >> v1:172.21.15.55:6789/0 conn(0x7fde800ae0d0 legacy=0x7fde800c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:36.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.627+0000 7fde7e59c700 1 -- 172.21.15.55:0/3689534961 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8015b000 con 0x7fde800cb1a0 2023-10-06T03:51:36.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.627+0000 7fde7e59c700 1 --2- 172.21.15.55:0/3689534961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde800cb1a0 0x7fde8015a360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fde70009910 tx=0x7fde700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7261b4b70fe4f61 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:36.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.627+0000 7fde7d59a700 1 -- 172.21.15.55:0/3689534961 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde780017f0 con 0x7fde800cb1a0 2023-10-06T03:51:36.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.627+0000 7fde7d59a700 1 -- 172.21.15.55:0/3689534961 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde70011420 con 0x7fde800cb1a0 2023-10-06T03:51:36.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.627+0000 7fde7d59a700 1 -- 172.21.15.55:0/3689534961 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde700117c0 con 0x7fde800cb1a0 2023-10-06T03:51:36.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.628+0000 7fde7f59e700 1 -- 172.21.15.55:0/3689534961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde800cb1a0 msgr2=0x7fde8015a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:36.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.628+0000 7fde7f59e700 1 --2- 172.21.15.55:0/3689534961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde800cb1a0 0x7fde8015a360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fde70009910 tx=0x7fde700091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:36.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.628+0000 7fde7f59e700 1 -- 172.21.15.55:0/3689534961 shutdown_connections 2023-10-06T03:51:36.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.628+0000 7fde7f59e700 1 --2- 172.21.15.55:0/3689534961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde800cb1a0 0x7fde8015a360 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:36.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.628+0000 7fde7f59e700 1 -- 172.21.15.55:0/3689534961 >> 172.21.15.55:0/3689534961 conn(0x7fde800cc1e0 msgr2=0x7fde800cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.628+0000 7fde7f59e700 1 -- 172.21.15.55:0/3689534961 shutdown_connections 2023-10-06T03:51:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.628+0000 7fde7f59e700 1 -- 172.21.15.55:0/3689534961 wait complete. 2023-10-06T03:51:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.629+0000 7fde7f59e700 1 Processor -- start 2023-10-06T03:51:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.630+0000 7fde7f59e700 1 -- start start 2023-10-06T03:51:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.630+0000 7fde7f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde800ae0d0 0x7fde8014a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.630+0000 7fde7f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde800cb1a0 0x7fde800c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.630+0000 7fde7f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde801542e0 0x7fde800b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.630+0000 7fde7f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde800c3ed0 con 0x7fde801542e0 2023-10-06T03:51:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.630+0000 7fde7f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde800c3330 con 0x7fde800ae0d0 2023-10-06T03:51:36.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.630+0000 7fde7f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde800c35a0 con 0x7fde800cb1a0 2023-10-06T03:51:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.631+0000 7fde7ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde801542e0 0x7fde800b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.631+0000 7fde7ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde801542e0 0x7fde800b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53714/0 (socket says 172.21.15.55:53714) 2023-10-06T03:51:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.631+0000 7fde7ed9d700 1 -- 172.21.15.55:0/3531721136 learned_addr learned my addr 172.21.15.55:0/3531721136 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.631+0000 7fde7e59c700 1 --2- 172.21.15.55:0/3531721136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde800ae0d0 0x7fde8014a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.631+0000 7fde7dd9b700 1 --2- 172.21.15.55:0/3531721136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde800cb1a0 0x7fde800c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.631+0000 7fde7ed9d700 1 -- 172.21.15.55:0/3531721136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde800cb1a0 msgr2=0x7fde800c4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.631+0000 7fde7ed9d700 1 --2- 172.21.15.55:0/3531721136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde800cb1a0 0x7fde800c4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.631+0000 7fde7ed9d700 1 -- 172.21.15.55:0/3531721136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde800ae0d0 msgr2=0x7fde8014a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.631+0000 7fde7ed9d700 1 --2- 172.21.15.55:0/3531721136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde800ae0d0 0x7fde8014a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.631+0000 7fde7ed9d700 1 -- 172.21.15.55:0/3531721136 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde70009420 con 0x7fde801542e0 2023-10-06T03:51:36.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.632+0000 7fde7ed9d700 1 --2- 172.21.15.55:0/3531721136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde801542e0 0x7fde800b15b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fde7800cc90 tx=0x7fde78007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:36.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.632+0000 7fde6e7fc700 1 -- 172.21.15.55:0/3531721136 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7800ce40 con 0x7fde801542e0 2023-10-06T03:51:36.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.632+0000 7fde6e7fc700 1 -- 172.21.15.55:0/3531721136 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde78004dc0 con 0x7fde801542e0 2023-10-06T03:51:36.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.632+0000 7fde6e7fc700 1 -- 172.21.15.55:0/3531721136 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7800ab30 con 0x7fde801542e0 2023-10-06T03:51:36.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.632+0000 7fde7f59e700 1 -- 172.21.15.55:0/3531721136 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde800c3880 con 0x7fde801542e0 2023-10-06T03:51:36.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.632+0000 7fde7f59e700 1 -- 172.21.15.55:0/3531721136 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8014a560 con 0x7fde801542e0 2023-10-06T03:51:36.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.633+0000 7fde7f59e700 1 -- 172.21.15.55:0/3531721136 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8006c450 con 0x7fde801542e0 2023-10-06T03:51:36.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.634+0000 7fde6e7fc700 1 -- 172.21.15.55:0/3531721136 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 49) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde78007150 con 0x7fde801542e0 2023-10-06T03:51:36.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.635+0000 7fde6e7fc700 1 --2- 172.21.15.55:0/3531721136 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fde60066f30 0x7fde600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:36.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.635+0000 7fde6e7fc700 1 -- 172.21.15.55:0/3531721136 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fde78084470 con 0x7fde801542e0 2023-10-06T03:51:36.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.636+0000 7fde7e59c700 1 --2- 172.21.15.55:0/3531721136 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fde60066f30 0x7fde600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:36.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.636+0000 7fde7e59c700 1 --2- 172.21.15.55:0/3531721136 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fde60066f30 0x7fde600693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fde700030c0 tx=0x7fde70016000 comp rx=0 tx=0).ready entity=mgr.7228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:36.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.639+0000 7fde6e7fc700 1 -- 172.21.15.55:0/3531721136 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde780513d0 con 0x7fde801542e0 2023-10-06T03:51:36.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:36.898+0000 7fde7f59e700 1 -- 172.21.15.55:0/3531721136 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fde800c4010 con 0x7fde801542e0 2023-10-06T03:51:37.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:37.771+0000 7fde6e7fc700 1 -- 172.21.15.55:0/3531721136 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 50) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fde780517a0 con 0x7fde801542e0 2023-10-06T03:51:37.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:37.773+0000 7fde6e7fc700 1 -- 172.21.15.55:0/3531721136 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v50) v1 ==== 92+0+0 (secure 0 0 0) 0x7fde780840c0 con 0x7fde801542e0 2023-10-06T03:51:37.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:37.776+0000 7fde7f59e700 1 -- 172.21.15.55:0/3531721136 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fde60066f30 msgr2=0x7fde600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:37.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:37.776+0000 7fde7f59e700 1 --2- 172.21.15.55:0/3531721136 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fde60066f30 0x7fde600693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fde700030c0 tx=0x7fde70016000 comp rx=0 tx=0).stop 2023-10-06T03:51:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:37.776+0000 7fde7f59e700 1 -- 172.21.15.55:0/3531721136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde801542e0 msgr2=0x7fde800b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:37.776+0000 7fde7f59e700 1 --2- 172.21.15.55:0/3531721136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde801542e0 0x7fde800b15b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fde7800cc90 tx=0x7fde78007d20 comp rx=0 tx=0).stop 2023-10-06T03:51:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:37.776+0000 7fde7f59e700 1 -- 172.21.15.55:0/3531721136 shutdown_connections 2023-10-06T03:51:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:37.776+0000 7fde7f59e700 1 --2- 172.21.15.55:0/3531721136 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fde60066f30 0x7fde600693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:37.776+0000 7fde7f59e700 1 --2- 172.21.15.55:0/3531721136 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fde801542e0 0x7fde800b15b0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:37.776+0000 7fde7f59e700 1 --2- 172.21.15.55:0/3531721136 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fde800ae0d0 0x7fde8014a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:37.776+0000 7fde7f59e700 1 --2- 172.21.15.55:0/3531721136 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fde800cb1a0 0x7fde800c4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:37.776+0000 7fde7f59e700 1 -- 172.21.15.55:0/3531721136 >> 172.21.15.55:0/3531721136 conn(0x7fde800cc1e0 msgr2=0x7fde80154960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:37.776+0000 7fde7f59e700 1 -- 172.21.15.55:0/3531721136 shutdown_connections 2023-10-06T03:51:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:37.776+0000 7fde7f59e700 1 -- 172.21.15.55:0/3531721136 wait complete. 2023-10-06T03:51:37.915 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T03:51:38.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.108+0000 7f146099d700 1 Processor -- start 2023-10-06T03:51:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.108+0000 7f146099d700 1 -- start start 2023-10-06T03:51:38.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.109+0000 7f146099d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f145c0cb230 0x7f145c0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:38.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.109+0000 7f146099d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145c0c9f30 con 0x7f145c159a60 2023-10-06T03:51:38.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.109+0000 7f146099d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f145c0ca470 con 0x7f145c0ccc00 2023-10-06T03:51:38.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.109+0000 7f146099d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145c0cbbd0 con 0x7f145c0cb230 2023-10-06T03:51:38.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.109+0000 7f145b7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f145c159a60 0x7f145c0c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:59082/0 (socket says 172.21.15.55:59082) 2023-10-06T03:51:38.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.109+0000 7f145b7fe700 1 -- 172.21.15.55:0/2005001648 learned_addr learned my addr 172.21.15.55:0/2005001648 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:51:38.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.109+0000 7f145bfff700 1 --2- 172.21.15.55:0/2005001648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f145c0cb230 0x7f145c0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:38.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.110+0000 7f145a7fc700 1 -- 172.21.15.55:0/2005001648 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3432472159 0 0) 0x7f145c0c9f30 con 0x7f145c159a60 2023-10-06T03:51:38.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.110+0000 7f145a7fc700 1 -- 172.21.15.55:0/2005001648 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1444003570 con 0x7f145c159a60 2023-10-06T03:51:38.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.110+0000 7f145a7fc700 1 -- 172.21.15.55:0/2005001648 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299808512 0 0) 0x7f145c0ca470 con 0x7f145c0ccc00 2023-10-06T03:51:38.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.110+0000 7f145a7fc700 1 -- 172.21.15.55:0/2005001648 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f145c0c9f30 con 0x7f145c0ccc00 2023-10-06T03:51:38.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.110+0000 7f145bfff700 1 -- 172.21.15.55:0/2005001648 >> v1:172.21.15.163:6789/0 conn(0x7f145c0ccc00 legacy=0x7f145c069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:38.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.110+0000 7f145bfff700 1 -- 172.21.15.55:0/2005001648 >> v1:172.21.15.55:6789/0 conn(0x7f145c159a60 legacy=0x7f145c0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:38.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.110+0000 7f145bfff700 1 -- 172.21.15.55:0/2005001648 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c0ca750 con 0x7f145c0cb230 2023-10-06T03:51:38.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.110+0000 7f145bfff700 1 --2- 172.21.15.55:0/2005001648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f145c0cb230 0x7f145c0cb690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1454009e90 tx=0x7f1454009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3590e9f7239dc0a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:38.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.111+0000 7f145a7fc700 1 -- 172.21.15.55:0/2005001648 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145400e070 con 0x7f145c0cb230 2023-10-06T03:51:38.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.111+0000 7f145a7fc700 1 -- 172.21.15.55:0/2005001648 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145400bc10 con 0x7f145c0cb230 2023-10-06T03:51:38.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.111+0000 7f145a7fc700 1 -- 172.21.15.55:0/2005001648 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454012430 con 0x7f145c0cb230 2023-10-06T03:51:38.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.111+0000 7f146099d700 1 -- 172.21.15.55:0/2005001648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f145c0cb230 msgr2=0x7f145c0cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:38.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.111+0000 7f146099d700 1 --2- 172.21.15.55:0/2005001648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f145c0cb230 0x7f145c0cb690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1454009e90 tx=0x7f1454009780 comp rx=0 tx=0).stop 2023-10-06T03:51:38.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.111+0000 7f146099d700 1 -- 172.21.15.55:0/2005001648 shutdown_connections 2023-10-06T03:51:38.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.111+0000 7f146099d700 1 --2- 172.21.15.55:0/2005001648 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f145c0cb230 0x7f145c0cb690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.111+0000 7f146099d700 1 -- 172.21.15.55:0/2005001648 >> 172.21.15.55:0/2005001648 conn(0x7f145c06baa0 msgr2=0x7f145c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:38.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.112+0000 7f146099d700 1 -- 172.21.15.55:0/2005001648 shutdown_connections 2023-10-06T03:51:38.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.112+0000 7f146099d700 1 -- 172.21.15.55:0/2005001648 wait complete. 2023-10-06T03:51:38.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.113+0000 7f146099d700 1 Processor -- start 2023-10-06T03:51:38.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.113+0000 7f146099d700 1 -- start start 2023-10-06T03:51:38.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.114+0000 7f146099d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145c0cb230 0x7f145c0c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:38.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.114+0000 7f146099d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f145c0ccc00 0x7f145c0c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:38.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.114+0000 7f146099d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f145c159a60 0x7f145c0c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:38.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.114+0000 7f146099d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c0b1b00 con 0x7f145c159a60 2023-10-06T03:51:38.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.114+0000 7f146099d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c1562d0 con 0x7f145c0cb230 2023-10-06T03:51:38.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.114+0000 7f146099d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145c0b1d10 con 0x7f145c0ccc00 2023-10-06T03:51:38.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.114+0000 7f145bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f145c159a60 0x7f145c0c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:38.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.114+0000 7f145affd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f145c0ccc00 0x7f145c0c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:38.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.114+0000 7f145b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145c0cb230 0x7f145c0c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:38.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.114+0000 7f145affd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f145c0ccc00 0x7f145c0c5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:58684/0 (socket says 172.21.15.55:58684) 2023-10-06T03:51:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.114+0000 7f145affd700 1 -- 172.21.15.55:0/3565119409 learned_addr learned my addr 172.21.15.55:0/3565119409 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.115+0000 7f145affd700 1 -- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145c0cb230 msgr2=0x7f145c0c5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.115+0000 7f145affd700 1 --2- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145c0cb230 0x7f145c0c5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.115+0000 7f145affd700 1 -- 172.21.15.55:0/3565119409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f145c159a60 msgr2=0x7f145c0c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.115+0000 7f145affd700 1 --2- 172.21.15.55:0/3565119409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f145c159a60 0x7f145c0c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.115+0000 7f145affd700 1 -- 172.21.15.55:0/3565119409 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1454009a20 con 0x7f145c0ccc00 2023-10-06T03:51:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.115+0000 7f145affd700 1 --2- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f145c0ccc00 0x7f145c0c5c80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f145000c790 tx=0x7f1450007e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.115+0000 7f14437fe700 1 -- 172.21.15.55:0/3565119409 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450010510 con 0x7f145c0ccc00 2023-10-06T03:51:38.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.115+0000 7f14437fe700 1 -- 172.21.15.55:0/3565119409 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1450010670 con 0x7f145c0ccc00 2023-10-06T03:51:38.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.115+0000 7f146099d700 1 -- 172.21.15.55:0/3565119409 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145c0b1ff0 con 0x7f145c0ccc00 2023-10-06T03:51:38.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.115+0000 7f14437fe700 1 -- 172.21.15.55:0/3565119409 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450010a10 con 0x7f145c0ccc00 2023-10-06T03:51:38.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.116+0000 7f146099d700 1 -- 172.21.15.55:0/3565119409 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145c0c7d20 con 0x7f145c0ccc00 2023-10-06T03:51:38.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.117+0000 7f146099d700 1 -- 172.21.15.55:0/3565119409 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f145c0b2510 con 0x7f145c0ccc00 2023-10-06T03:51:38.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.117+0000 7f14437fe700 1 -- 172.21.15.55:0/3565119409 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 50) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f145001b3b0 con 0x7f145c0ccc00 2023-10-06T03:51:38.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.118+0000 7f14437fe700 1 --2- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1448066f80 0x7f1448069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:38.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.118+0000 7f14437fe700 1 -- 172.21.15.55:0/3565119409 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f14500841c0 con 0x7f145c0ccc00 2023-10-06T03:51:38.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.118+0000 7f145b7fe700 1 -- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1448066f80 msgr2=0x7f1448069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:38.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.118+0000 7f145b7fe700 1 --2- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1448066f80 0x7f1448069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:38.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.122+0000 7f14437fe700 1 -- 172.21.15.55:0/3565119409 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f145c0b2510 con 0x7f145c0ccc00 2023-10-06T03:51:38.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.319+0000 7f145b7fe700 1 -- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1448066f80 msgr2=0x7f1448069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:38.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.319+0000 7f145b7fe700 1 --2- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1448066f80 0x7f1448069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:38.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.332+0000 7f146099d700 1 -- 172.21.15.55:0/3565119409 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f145c0b2510 con 0x7f145c0ccc00 2023-10-06T03:51:38.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.337+0000 7f14437fe700 1 -- 172.21.15.55:0/3565119409 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v41)=0 v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7f145c0b2510 con 0x7f145c0ccc00 2023-10-06T03:51:38.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.340+0000 7f146099d700 1 -- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1448066f80 msgr2=0x7f1448069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:38.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.340+0000 7f146099d700 1 --2- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1448066f80 0x7f1448069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.340+0000 7f146099d700 1 -- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f145c0ccc00 msgr2=0x7f145c0c5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:38.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.340+0000 7f146099d700 1 --2- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f145c0ccc00 0x7f145c0c5c80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f145000c790 tx=0x7f1450007e10 comp rx=0 tx=0).stop 2023-10-06T03:51:38.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.340+0000 7f146099d700 1 -- 172.21.15.55:0/3565119409 shutdown_connections 2023-10-06T03:51:38.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.340+0000 7f146099d700 1 --2- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1448066f80 0x7f1448069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.340+0000 7f146099d700 1 --2- 172.21.15.55:0/3565119409 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f145c159a60 0x7f145c0c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.340+0000 7f146099d700 1 --2- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f145c0cb230 0x7f145c0c5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.340+0000 7f146099d700 1 --2- 172.21.15.55:0/3565119409 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f145c0ccc00 0x7f145c0c5c80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.340+0000 7f146099d700 1 -- 172.21.15.55:0/3565119409 >> 172.21.15.55:0/3565119409 conn(0x7f145c06baa0 msgr2=0x7f145c06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:38.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.340+0000 7f146099d700 1 -- 172.21.15.55:0/3565119409 shutdown_connections 2023-10-06T03:51:38.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.340+0000 7f146099d700 1 -- 172.21.15.55:0/3565119409 wait complete. 2023-10-06T03:51:38.360 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T03:51:38.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.565+0000 7f031759e700 1 Processor -- start 2023-10-06T03:51:38.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.565+0000 7f031759e700 1 -- start start 2023-10-06T03:51:38.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.566+0000 7f031759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f03180b4e70 0x7f03180ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:38.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.567+0000 7f031759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03180b16d0 0x7f03180b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:38.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.567+0000 7f031759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03180b2080 con 0x7f031815d240 2023-10-06T03:51:38.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.567+0000 7f031759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03180cfb00 con 0x7f03180b4e70 2023-10-06T03:51:38.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.567+0000 7f031759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03180b5c40 con 0x7f03180b16d0 2023-10-06T03:51:38.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.567+0000 7f031659c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f03180b4e70 0x7f03180ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:38.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.567+0000 7f0315d9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03180b16d0 0x7f03180b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:38.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.567+0000 7f0315d9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03180b16d0 0x7f03180b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:58692/0 (socket says 172.21.15.55:58692) 2023-10-06T03:51:38.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.567+0000 7f031659c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f03180b4e70 0x7f03180ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:41486/0 (socket says 172.21.15.55:41486) 2023-10-06T03:51:38.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.567+0000 7f0315d9b700 1 -- 172.21.15.55:0/2373676433 learned_addr learned my addr 172.21.15.55:0/2373676433 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:38.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.567+0000 7f0316d9d700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f031815d240 0x7f031815d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:33516/0 (socket says 172.21.15.55:33516) 2023-10-06T03:51:38.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.568+0000 7f031659c700 1 -- 172.21.15.55:0/2373676433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03180b16d0 msgr2=0x7f03180b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:38.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.568+0000 7f031659c700 1 --2- 172.21.15.55:0/2373676433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03180b16d0 0x7f03180b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.568+0000 7f031659c700 1 -- 172.21.15.55:0/2373676433 >> v1:172.21.15.163:6789/0 conn(0x7f031815d240 legacy=0x7f031815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:38.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.568+0000 7f031659c700 1 -- 172.21.15.55:0/2373676433 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03180b5f20 con 0x7f03180b4e70 2023-10-06T03:51:38.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.568+0000 7f0315d9b700 1 --2- 172.21.15.55:0/2373676433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03180b16d0 0x7f03180b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:51:38.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.568+0000 7f031559a700 1 -- 172.21.15.55:0/2373676433 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124164791 0 0) 0x7f03180b2080 con 0x7f031815d240 2023-10-06T03:51:38.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.568+0000 7f031659c700 1 --2- 172.21.15.55:0/2373676433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03180b4e70 0x7f03180ccff0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f03080098e0 tx=0x7f03080091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcafd48a84991e45 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:38.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.568+0000 7f031559a700 1 -- 172.21.15.55:0/2373676433 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030800b640 con 0x7f03180b4e70 2023-10-06T03:51:38.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.568+0000 7f031559a700 1 -- 172.21.15.55:0/2373676433 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030800b7a0 con 0x7f03180b4e70 2023-10-06T03:51:38.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.568+0000 7f031559a700 1 -- 172.21.15.55:0/2373676433 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030800bb40 con 0x7f03180b4e70 2023-10-06T03:51:38.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.569+0000 7f031759e700 1 -- 172.21.15.55:0/2373676433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03180b4e70 msgr2=0x7f03180ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:38.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.569+0000 7f031759e700 1 --2- 172.21.15.55:0/2373676433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03180b4e70 0x7f03180ccff0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f03080098e0 tx=0x7f03080091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.569+0000 7f031759e700 1 -- 172.21.15.55:0/2373676433 shutdown_connections 2023-10-06T03:51:38.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.569+0000 7f031759e700 1 --2- 172.21.15.55:0/2373676433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03180b4e70 0x7f03180ccff0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.569+0000 7f031759e700 1 --2- 172.21.15.55:0/2373676433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03180b16d0 0x7f03180b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.569+0000 7f031759e700 1 -- 172.21.15.55:0/2373676433 >> 172.21.15.55:0/2373676433 conn(0x7f03180d10c0 msgr2=0x7f03180affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:38.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.569+0000 7f031759e700 1 -- 172.21.15.55:0/2373676433 shutdown_connections 2023-10-06T03:51:38.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.569+0000 7f031759e700 1 -- 172.21.15.55:0/2373676433 wait complete. 2023-10-06T03:51:38.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.571+0000 7f031759e700 1 Processor -- start 2023-10-06T03:51:38.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.571+0000 7f031759e700 1 -- start start 2023-10-06T03:51:38.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f031759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03180b16d0 0x7f03181576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:38.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f031759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03180b4e70 0x7f03180cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:38.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f031759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f031815d240 0x7f03180cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:38.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f031759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03180cc430 con 0x7f03180b4e70 2023-10-06T03:51:38.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f031759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03180c6770 con 0x7f031815d240 2023-10-06T03:51:38.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f031759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03180c9e90 con 0x7f03180b16d0 2023-10-06T03:51:38.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f0315d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03180b4e70 0x7f03180cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:38.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f0315d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03180b4e70 0x7f03180cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53766/0 (socket says 172.21.15.55:53766) 2023-10-06T03:51:38.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f0315d9b700 1 -- 172.21.15.55:0/1405495732 learned_addr learned my addr 172.21.15.55:0/1405495732 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:38.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f031659c700 1 --2- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03180b16d0 0x7f03181576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:38.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f0316d9d700 1 --2- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f031815d240 0x7f03180cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:38.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f0315d9b700 1 -- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03180b16d0 msgr2=0x7f03181576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:38.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f0315d9b700 1 --2- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03180b16d0 0x7f03181576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f0315d9b700 1 -- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f031815d240 msgr2=0x7f03180cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:38.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.572+0000 7f0315d9b700 1 --2- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f031815d240 0x7f03180cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.573+0000 7f0315d9b700 1 -- 172.21.15.55:0/1405495732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c0094f0 con 0x7f03180b4e70 2023-10-06T03:51:38.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.573+0000 7f0315d9b700 1 --2- 172.21.15.55:0/1405495732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03180b4e70 0x7f03180cb9b0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f030c00ec50 tx=0x7f030c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:38.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.573+0000 7f03067fc700 1 -- 172.21.15.55:0/1405495732 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00c850 con 0x7f03180b4e70 2023-10-06T03:51:38.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.573+0000 7f03067fc700 1 -- 172.21.15.55:0/1405495732 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c00c9b0 con 0x7f03180b4e70 2023-10-06T03:51:38.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.573+0000 7f03067fc700 1 -- 172.21.15.55:0/1405495732 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c00cd50 con 0x7f03180b4e70 2023-10-06T03:51:38.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.573+0000 7f031759e700 1 -- 172.21.15.55:0/1405495732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0308009420 con 0x7f03180b4e70 2023-10-06T03:51:38.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.573+0000 7f031759e700 1 -- 172.21.15.55:0/1405495732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03180ca220 con 0x7f03180b4e70 2023-10-06T03:51:38.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.575+0000 7f031759e700 1 -- 172.21.15.55:0/1405495732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03180ca570 con 0x7f03180b4e70 2023-10-06T03:51:38.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.575+0000 7f03067fc700 1 -- 172.21.15.55:0/1405495732 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 50) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f030c013680 con 0x7f03180b4e70 2023-10-06T03:51:38.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.576+0000 7f03067fc700 1 --2- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02f8066f80 0x7f02f8069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:38.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.576+0000 7f03067fc700 1 -- 172.21.15.55:0/1405495732 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f030c084ce0 con 0x7f03180b4e70 2023-10-06T03:51:38.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.577+0000 7f031659c700 1 -- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02f8066f80 msgr2=0x7f02f8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:38.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.577+0000 7f031659c700 1 --2- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02f8066f80 0x7f02f8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:38.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.580+0000 7f03067fc700 1 -- 172.21.15.55:0/1405495732 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f030c051bc0 con 0x7f03180b4e70 2023-10-06T03:51:38.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.778+0000 7f031659c700 1 -- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02f8066f80 msgr2=0x7f02f8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:38.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.778+0000 7f031659c700 1 --2- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02f8066f80 0x7f02f8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:38.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.874+0000 7f031759e700 1 -- 172.21.15.55:0/1405495732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f03180889e0 con 0x7f03180b4e70 2023-10-06T03:51:38.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.880+0000 7f03067fc700 1 -- 172.21.15.55:0/1405495732 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v42)=0 v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7f030c0551e0 con 0x7f03180b4e70 2023-10-06T03:51:38.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.882+0000 7f031759e700 1 -- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02f8066f80 msgr2=0x7f02f8069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:38.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.882+0000 7f031759e700 1 --2- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02f8066f80 0x7f02f8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.882+0000 7f031759e700 1 -- 172.21.15.55:0/1405495732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03180b4e70 msgr2=0x7f03180cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:38.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.882+0000 7f031759e700 1 --2- 172.21.15.55:0/1405495732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03180b4e70 0x7f03180cb9b0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f030c00ec50 tx=0x7f030c009250 comp rx=0 tx=0).stop 2023-10-06T03:51:38.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.882+0000 7f031759e700 1 -- 172.21.15.55:0/1405495732 shutdown_connections 2023-10-06T03:51:38.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.882+0000 7f031759e700 1 --2- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f02f8066f80 0x7f02f8069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.882+0000 7f031759e700 1 --2- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f031815d240 0x7f03180cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.882+0000 7f031759e700 1 --2- 172.21.15.55:0/1405495732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03180b16d0 0x7f03181576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.882+0000 7f031759e700 1 --2- 172.21.15.55:0/1405495732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03180b4e70 0x7f03180cb9b0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:38.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.882+0000 7f031759e700 1 -- 172.21.15.55:0/1405495732 >> 172.21.15.55:0/1405495732 conn(0x7f03180d10c0 msgr2=0x7f031815c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:38.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.883+0000 7f031759e700 1 -- 172.21.15.55:0/1405495732 shutdown_connections 2023-10-06T03:51:38.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:38.883+0000 7f031759e700 1 -- 172.21.15.55:0/1405495732 wait complete. 2023-10-06T03:51:38.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:38.902 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T03:51:38.947 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-10-06T03:51:39.065 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:39.063+0000 7f6efb94a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:51:39.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.183+0000 7f114b59d700 1 Processor -- start 2023-10-06T03:51:39.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.183+0000 7f114b59d700 1 -- start start 2023-10-06T03:51:39.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.184+0000 7f114b59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f11440b03c0 0x7f11440cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:39.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.184+0000 7f114b59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f11440b0790 0x7f11440b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:39.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.184+0000 7f114b59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11440cfa00 con 0x7f11441591e0 2023-10-06T03:51:39.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.184+0000 7f114b59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11440cb580 con 0x7f11440b0790 2023-10-06T03:51:39.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.184+0000 7f114b59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11440b2de0 con 0x7f11440b03c0 2023-10-06T03:51:39.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.185+0000 7f1149d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f11440b0790 0x7f11440b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:39.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.185+0000 7f1149d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f11440b0790 0x7f11440b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53774/0 (socket says 172.21.15.55:53774) 2023-10-06T03:51:39.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.185+0000 7f1149d9a700 1 -- 172.21.15.55:0/2223267541 learned_addr learned my addr 172.21.15.55:0/2223267541 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.185+0000 7f114a59b700 1 --2- 172.21.15.55:0/2223267541 >> v2:172.21.15.163:3300/0 conn(0x7f11440b03c0 0x7f11440cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.185+0000 7f1149d9a700 1 -- 172.21.15.55:0/2223267541 >> v2:172.21.15.163:3300/0 conn(0x7f11440b03c0 msgr2=0x7f11440cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.185+0000 7f1149d9a700 1 --2- 172.21.15.55:0/2223267541 >> v2:172.21.15.163:3300/0 conn(0x7f11440b03c0 0x7f11440cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.185+0000 7f1149d9a700 1 -- 172.21.15.55:0/2223267541 >> v1:172.21.15.163:6789/0 conn(0x7f11441591e0 legacy=0x7f1144159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.185+0000 7f1149d9a700 1 -- 172.21.15.55:0/2223267541 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11440cf160 con 0x7f11440b0790 2023-10-06T03:51:39.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.185+0000 7f1149d9a700 1 --2- 172.21.15.55:0/2223267541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11440b0790 0x7f11440b28a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f11340099b0 tx=0x7f1134009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c2247790cb0741 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:39.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.186+0000 7f1149599700 1 -- 172.21.15.55:0/2223267541 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134009ea0 con 0x7f11440b0790 2023-10-06T03:51:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.186+0000 7f1149599700 1 -- 172.21.15.55:0/2223267541 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113400b820 con 0x7f11440b0790 2023-10-06T03:51:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.186+0000 7f1149599700 1 -- 172.21.15.55:0/2223267541 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113400bbe0 con 0x7f11440b0790 2023-10-06T03:51:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.186+0000 7f114b59d700 1 -- 172.21.15.55:0/2223267541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11440b0790 msgr2=0x7f11440b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.186+0000 7f114b59d700 1 --2- 172.21.15.55:0/2223267541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11440b0790 0x7f11440b28a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f11340099b0 tx=0x7f1134009250 comp rx=0 tx=0).stop 2023-10-06T03:51:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.186+0000 7f114b59d700 1 -- 172.21.15.55:0/2223267541 shutdown_connections 2023-10-06T03:51:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.186+0000 7f114b59d700 1 --2- 172.21.15.55:0/2223267541 >> v2:172.21.15.163:3300/0 conn(0x7f11440b03c0 0x7f11440cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.186+0000 7f114b59d700 1 --2- 172.21.15.55:0/2223267541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11440b0790 0x7f11440b28a0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.186+0000 7f114b59d700 1 -- 172.21.15.55:0/2223267541 >> 172.21.15.55:0/2223267541 conn(0x7f1144063230 msgr2=0x7f1144063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.186+0000 7f114b59d700 1 -- 172.21.15.55:0/2223267541 shutdown_connections 2023-10-06T03:51:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.186+0000 7f114b59d700 1 -- 172.21.15.55:0/2223267541 wait complete. 2023-10-06T03:51:39.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.188+0000 7f114b59d700 1 Processor -- start 2023-10-06T03:51:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.188+0000 7f114b59d700 1 -- start start 2023-10-06T03:51:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.189+0000 7f114b59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11440b03c0 0x7f11440d0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.189+0000 7f114b59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11440b0790 0x7f11440d0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.189+0000 7f114b59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11441591e0 0x7f11441571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.189+0000 7f114b59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11440c7630 con 0x7f11440b0790 2023-10-06T03:51:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.189+0000 7f114b59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11440c6370 con 0x7f11440b03c0 2023-10-06T03:51:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.189+0000 7f114b59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11440c6650 con 0x7f11441591e0 2023-10-06T03:51:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.189+0000 7f1149d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11440b0790 0x7f11440d0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.189+0000 7f1149d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11440b0790 0x7f11440d0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:53786/0 (socket says 172.21.15.55:53786) 2023-10-06T03:51:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.189+0000 7f1149d9a700 1 -- 172.21.15.55:0/2811022631 learned_addr learned my addr 172.21.15.55:0/2811022631 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.189+0000 7f114a59b700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11440b03c0 0x7f11440d0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.189+0000 7f114ad9c700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11441591e0 0x7f11441571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:39.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.189+0000 7f1149d9a700 1 -- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11441591e0 msgr2=0x7f11441571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:39.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.189+0000 7f1149d9a700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11441591e0 0x7f11441571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:39.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.189+0000 7f1149d9a700 1 -- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11440b03c0 msgr2=0x7f11440d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:39.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.190+0000 7f1149d9a700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11440b03c0 0x7f11440d0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:39.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.190+0000 7f1149d9a700 1 -- 172.21.15.55:0/2811022631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11340094f0 con 0x7f11440b0790 2023-10-06T03:51:39.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.190+0000 7f1149d9a700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11440b0790 0x7f11440d0a60 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f11440cfe40 tx=0x7f1134017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:39.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.190+0000 7f113a7fc700 1 -- 172.21.15.55:0/2811022631 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113401f070 con 0x7f11440b0790 2023-10-06T03:51:39.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.190+0000 7f113a7fc700 1 -- 172.21.15.55:0/2811022631 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1134000bc0 con 0x7f11440b0790 2023-10-06T03:51:39.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.190+0000 7f113a7fc700 1 -- 172.21.15.55:0/2811022631 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113400bbc0 con 0x7f11440b0790 2023-10-06T03:51:39.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.190+0000 7f114b59d700 1 -- 172.21.15.55:0/2811022631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11440c6930 con 0x7f11440b0790 2023-10-06T03:51:39.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.190+0000 7f114b59d700 1 -- 172.21.15.55:0/2811022631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114414dae0 con 0x7f11440b0790 2023-10-06T03:51:39.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.191+0000 7f114b59d700 1 -- 172.21.15.55:0/2811022631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11440c77a0 con 0x7f11440b0790 2023-10-06T03:51:39.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.192+0000 7f113a7fc700 1 -- 172.21.15.55:0/2811022631 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 50) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f1134023070 con 0x7f11440b0790 2023-10-06T03:51:39.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.193+0000 7f113a7fc700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 0x7f1128069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:39.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.193+0000 7f113a7fc700 1 -- 172.21.15.55:0/2811022631 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f1134085c10 con 0x7f11440b0790 2023-10-06T03:51:39.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.193+0000 7f114a59b700 1 -- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 msgr2=0x7f1128069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:39.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.193+0000 7f114a59b700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 0x7f1128069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:51:39.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.197+0000 7f113a7fc700 1 -- 172.21.15.55:0/2811022631 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1134052b90 con 0x7f11440b0790 2023-10-06T03:51:39.206 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:39.204+0000 7fc91b6c4180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:51:39.231 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:39.229+0000 7f6efb94a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:51:39.317 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:39.315+0000 7f6efb94a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:51:39.425 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:39.378+0000 7fc91b6c4180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:51:39.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.394+0000 7f114a59b700 1 -- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 msgr2=0x7f1128069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:39.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.394+0000 7f114a59b700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 0x7f1128069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:51:39.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.426+0000 7f114b59d700 1 -- 172.21.15.55:0/2811022631 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f114404c3c0 con 0x7f1128066f80 2023-10-06T03:51:39.469 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:39.468+0000 7fc91b6c4180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:51:39.563 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:39.543+0000 7f6efb94a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:51:39.702 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:39.701+0000 7fc91b6c4180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:51:39.721 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:39.719+0000 7f6efb94a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:51:39.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.795+0000 7f114a59b700 1 -- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 msgr2=0x7f1128069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:39.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:39.795+0000 7f114a59b700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 0x7f1128069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T03:51:39.814 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:39.813+0000 7f6efb94a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:51:39.888 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:39.887+0000 7fc91b6c4180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:51:39.980 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:39.978+0000 7f6efb94a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:51:40.080 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:39.984+0000 7fc91b6c4180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:51:40.081 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:40.079+0000 7f6efb94a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:51:40.152 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:40.151+0000 7fc91b6c4180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:51:40.257 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:40.256+0000 7fc91b6c4180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:51:40.570 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:40.568+0000 7f6efb94a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:51:40.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:40.596+0000 7f114a59b700 1 -- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 msgr2=0x7f1128069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:40.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:40.596+0000 7f114a59b700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 0x7f1128069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T03:51:40.664 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:40.662+0000 7f6efb94a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:51:40.754 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:40.752+0000 7f6efb94a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:51:40.763 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:40.762+0000 7fc91b6c4180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:51:40.862 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:40.861+0000 7fc91b6c4180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:51:40.957 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:40.956+0000 7fc91b6c4180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:51:41.379 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:41.377+0000 7f6efb94a180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:51:41.552 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:41.550+0000 7f6efb94a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:51:41.614 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:41.612+0000 7fc91b6c4180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:51:41.678 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:41.648+0000 7f6efb94a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:51:41.795 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:41.794+0000 7fc91b6c4180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:51:41.896 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:41.895+0000 7fc91b6c4180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:51:41.947 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:41.945+0000 7f6efb94a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:51:42.046 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:42.044+0000 7f6efb94a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:51:42.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:42.198+0000 7f114a59b700 1 -- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 msgr2=0x7f1128069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:51:42.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:42.198+0000 7f114a59b700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 0x7f1128069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T03:51:42.201 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:42.199+0000 7fc91b6c4180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:51:42.302 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:42.301+0000 7fc91b6c4180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:51:42.749 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:42.747+0000 7f6efb94a180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:51:43.039 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:43.037+0000 7f6efb94a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:51:43.044 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:43.043+0000 7fc91b6c4180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:51:43.132 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:43.130+0000 7f6efb94a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:51:43.287 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:43.285+0000 7f6efb94a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:51:43.338 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:43.336+0000 7fc91b6c4180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:51:43.392 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:43.390+0000 7f6efb94a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:51:43.434 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:43.432+0000 7fc91b6c4180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:51:43.583 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:43.581+0000 7f6efb94a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:51:43.597 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:43.596+0000 7fc91b6c4180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:51:43.676 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:43.674+0000 7f6efb94a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:51:43.706 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:43.705+0000 7fc91b6c4180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:51:43.768 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:43.766+0000 7f6efb94a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:51:43.861 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:43.859+0000 7f6efb94a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:51:43.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:43.874+0000 7f113a7fc700 1 -- 172.21.15.55:0/2811022631 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 51) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f1134052f60 con 0x7f11440b0790 2023-10-06T03:51:43.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:43.874+0000 7f113a7fc700 1 -- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 msgr2=0x7f1128069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:51:43.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:43.874+0000 7f113a7fc700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 0x7f1128069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:43.910 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:43.908+0000 7fc91b6c4180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:51:44.002 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:44.000+0000 7f6e70c77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:44.003 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:44.000+0000 7f6e70c77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:44.003 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:44.000+0000 7f6e70c77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:44.003 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:44.000+0000 7f6e70c77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:44.003 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:44.000+0000 7f6e70c77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:44.003 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:44.001+0000 7f6e6dc71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:44.003 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:44.001+0000 7f6e6dc71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:44.004 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:44.001+0000 7f6e6dc71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:44.004 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:44.001+0000 7f6e6dc71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:44.004 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:51:44.001+0000 7f6e6dc71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:51:44.020 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:44.019+0000 7fc91b6c4180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:51:44.117 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:44.115+0000 7fc91b6c4180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:51:44.215 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:51:44.214+0000 7fc91b6c4180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:51:44.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.877+0000 7f113a7fc700 1 -- 172.21.15.55:0/2811022631 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mgrmap(e 52) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f113405d360 con 0x7f11440b0790 2023-10-06T03:51:44.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.877+0000 7f113a7fc700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 0x7f1128069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:44.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.877+0000 7f113a7fc700 1 -- 172.21.15.55:0/2811022631 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f114404c3c0 con 0x7f1128066f80 2023-10-06T03:51:44.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.880+0000 7f114a59b700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 0x7f1128069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:44.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.884+0000 7f114a59b700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 0x7f1128069430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f11400097c0 tx=0x7f11400091a0 comp rx=0 tx=0).ready entity=mgr.6803 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:44.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.950+0000 7f113a7fc700 1 -- 172.21.15.55:0/2811022631 <== mgr.6803 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f114404c3c0 con 0x7f1128066f80 2023-10-06T03:51:44.952 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path snap_test_dir1 2023-10-06T03:51:44.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.953+0000 7f114b59d700 1 -- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 msgr2=0x7f1128069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:44.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.953+0000 7f114b59d700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 0x7f1128069430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f11400097c0 tx=0x7f11400091a0 comp rx=0 tx=0).stop 2023-10-06T03:51:44.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.953+0000 7f114b59d700 1 -- 172.21.15.55:0/2811022631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11440b0790 msgr2=0x7f11440d0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:44.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.953+0000 7f114b59d700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11440b0790 0x7f11440d0a60 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f11440cfe40 tx=0x7f1134017000 comp rx=0 tx=0).stop 2023-10-06T03:51:44.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.953+0000 7f114b59d700 1 -- 172.21.15.55:0/2811022631 shutdown_connections 2023-10-06T03:51:44.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.953+0000 7f114b59d700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f1128066f80 0x7f1128069430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:44.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.953+0000 7f114b59d700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11441591e0 0x7f11441571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:44.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.953+0000 7f114b59d700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11440b03c0 0x7f11440d0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:44.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.953+0000 7f114b59d700 1 --2- 172.21.15.55:0/2811022631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11440b0790 0x7f11440d0a60 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:44.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.953+0000 7f114b59d700 1 -- 172.21.15.55:0/2811022631 >> 172.21.15.55:0/2811022631 conn(0x7f1144063230 msgr2=0x7f11440cba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:44.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.953+0000 7f114b59d700 1 -- 172.21.15.55:0/2811022631 shutdown_connections 2023-10-06T03:51:44.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:44.953+0000 7f114b59d700 1 -- 172.21.15.55:0/2811022631 wait complete. 2023-10-06T03:51:44.974 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-06T03:51:44.975 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-06-03_52_00 in ~16s... 2023-10-06T03:51:44.975 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-10-06T03:51:44.975 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-10-06T03:51:45.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.165+0000 7f020099d700 1 Processor -- start 2023-10-06T03:51:45.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.166+0000 7f020099d700 1 -- start start 2023-10-06T03:51:45.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.167+0000 7f020099d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01fc156350 0x7f01fc157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:45.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.167+0000 7f020099d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01fc0b0c00 0x7f01fc15a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:45.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.167+0000 7f020099d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01fc0b0570 con 0x7f01fc0b0760 2023-10-06T03:51:45.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.167+0000 7f020099d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01fc1578b0 con 0x7f01fc156350 2023-10-06T03:51:45.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.167+0000 7f020099d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0af4d0 con 0x7f01fc0b0c00 2023-10-06T03:51:45.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.167+0000 7f01fb7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01fc156350 0x7f01fc157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:45.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.167+0000 7f01fb7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01fc156350 0x7f01fc157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35060/0 (socket says 172.21.15.55:35060) 2023-10-06T03:51:45.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.167+0000 7f01fb7fe700 1 -- 172.21.15.55:0/153369457 learned_addr learned my addr 172.21.15.55:0/153369457 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:45.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.168+0000 7f01fbfff700 1 --2- 172.21.15.55:0/153369457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01fc0b0c00 0x7f01fc15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:45.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.168+0000 7f01fb7fe700 1 -- 172.21.15.55:0/153369457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01fc0b0c00 msgr2=0x7f01fc15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:51:45.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.168+0000 7f01fb7fe700 1 --2- 172.21.15.55:0/153369457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01fc0b0c00 0x7f01fc15a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:45.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.168+0000 7f01fb7fe700 1 -- 172.21.15.55:0/153369457 >> v1:172.21.15.163:6789/0 conn(0x7f01fc0b0760 legacy=0x7f01fc159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:45.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.168+0000 7f01fb7fe700 1 -- 172.21.15.55:0/153369457 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc0af7b0 con 0x7f01fc156350 2023-10-06T03:51:45.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.168+0000 7f01fbfff700 1 --2- 172.21.15.55:0/153369457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01fc0b0c00 0x7f01fc15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:51:45.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.168+0000 7f01fb7fe700 1 --2- 172.21.15.55:0/153369457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01fc156350 0x7f01fc157370 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0099b0 tx=0x7f01ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7156f39641ca2982 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:45.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.168+0000 7f01fa7fc700 1 -- 172.21.15.55:0/153369457 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00b860 con 0x7f01fc156350 2023-10-06T03:51:45.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.168+0000 7f01fa7fc700 1 -- 172.21.15.55:0/153369457 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec00b9c0 con 0x7f01fc156350 2023-10-06T03:51:45.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.168+0000 7f01fa7fc700 1 -- 172.21.15.55:0/153369457 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00bd60 con 0x7f01fc156350 2023-10-06T03:51:45.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.169+0000 7f020099d700 1 -- 172.21.15.55:0/153369457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01fc156350 msgr2=0x7f01fc157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:45.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.169+0000 7f020099d700 1 --2- 172.21.15.55:0/153369457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01fc156350 0x7f01fc157370 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0099b0 tx=0x7f01ec009250 comp rx=0 tx=0).stop 2023-10-06T03:51:45.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.169+0000 7f020099d700 1 -- 172.21.15.55:0/153369457 shutdown_connections 2023-10-06T03:51:45.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.169+0000 7f020099d700 1 --2- 172.21.15.55:0/153369457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01fc0b0c00 0x7f01fc15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:45.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.169+0000 7f020099d700 1 --2- 172.21.15.55:0/153369457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01fc156350 0x7f01fc157370 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:45.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.169+0000 7f020099d700 1 -- 172.21.15.55:0/153369457 >> 172.21.15.55:0/153369457 conn(0x7f01fc068e20 msgr2=0x7f01fc0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:45.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.169+0000 7f020099d700 1 -- 172.21.15.55:0/153369457 shutdown_connections 2023-10-06T03:51:45.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.169+0000 7f020099d700 1 -- 172.21.15.55:0/153369457 wait complete. 2023-10-06T03:51:45.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.171+0000 7f020099d700 1 Processor -- start 2023-10-06T03:51:45.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.171+0000 7f020099d700 1 -- start start 2023-10-06T03:51:45.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f020099d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01fc0b0760 0x7f01fc141360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:45.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f020099d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01fc0b0c00 0x7f01fc0aacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:45.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f020099d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01fc156350 0x7f01fc16ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:45.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f020099d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc1418a0 con 0x7f01fc0b0c00 2023-10-06T03:51:45.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f020099d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0cbc70 con 0x7f01fc0b0760 2023-10-06T03:51:45.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f020099d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0cb060 con 0x7f01fc156350 2023-10-06T03:51:45.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f01faffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01fc0b0c00 0x7f01fc0aacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:45.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f01fb7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01fc0b0760 0x7f01fc141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:45.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f01faffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01fc0b0c00 0x7f01fc0aacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:35070/0 (socket says 172.21.15.55:35070) 2023-10-06T03:51:45.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f01faffd700 1 -- 172.21.15.55:0/3612336021 learned_addr learned my addr 172.21.15.55:0/3612336021 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:51:45.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f01fbfff700 1 --2- 172.21.15.55:0/3612336021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01fc156350 0x7f01fc16ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:45.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f01faffd700 1 -- 172.21.15.55:0/3612336021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01fc156350 msgr2=0x7f01fc16ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:45.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f01faffd700 1 --2- 172.21.15.55:0/3612336021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01fc156350 0x7f01fc16ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:45.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f01faffd700 1 -- 172.21.15.55:0/3612336021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01fc0b0760 msgr2=0x7f01fc141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:45.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f01faffd700 1 --2- 172.21.15.55:0/3612336021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01fc0b0760 0x7f01fc141360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:45.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.172+0000 7f01faffd700 1 -- 172.21.15.55:0/3612336021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0094f0 con 0x7f01fc0b0c00 2023-10-06T03:51:45.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.173+0000 7f01faffd700 1 --2- 172.21.15.55:0/3612336021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01fc0b0c00 0x7f01fc0aacc0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f01f000c510 tx=0x7f01f000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:45.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.173+0000 7f01e37fe700 1 -- 172.21.15.55:0/3612336021 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0011590 con 0x7f01fc0b0c00 2023-10-06T03:51:45.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.173+0000 7f01e37fe700 1 -- 172.21.15.55:0/3612336021 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f0011710 con 0x7f01fc0b0c00 2023-10-06T03:51:45.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.173+0000 7f01e37fe700 1 -- 172.21.15.55:0/3612336021 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0011ad0 con 0x7f01fc0b0c00 2023-10-06T03:51:45.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.173+0000 7f020099d700 1 -- 172.21.15.55:0/3612336021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01fc0cb340 con 0x7f01fc0b0c00 2023-10-06T03:51:45.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.173+0000 7f020099d700 1 -- 172.21.15.55:0/3612336021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01fc0cb900 con 0x7f01fc0b0c00 2023-10-06T03:51:45.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.174+0000 7f020099d700 1 -- 172.21.15.55:0/3612336021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01fc141b20 con 0x7f01fc0b0c00 2023-10-06T03:51:45.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.175+0000 7f01e37fe700 1 -- 172.21.15.55:0/3612336021 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 52) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f01f001b3b0 con 0x7f01fc0b0c00 2023-10-06T03:51:45.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.176+0000 7f01e37fe700 1 --2- 172.21.15.55:0/3612336021 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01e8066dd0 0x7f01e8069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:51:45.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.176+0000 7f01e37fe700 1 -- 172.21.15.55:0/3612336021 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f01f0083a30 con 0x7f01fc0b0c00 2023-10-06T03:51:45.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.176+0000 7f01fb7fe700 1 --2- 172.21.15.55:0/3612336021 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01e8066dd0 0x7f01e8069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:51:45.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.177+0000 7f01fb7fe700 1 --2- 172.21.15.55:0/3612336021 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01e8066dd0 0x7f01e8069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0099b0 tx=0x7f01ec017000 comp rx=0 tx=0).ready entity=mgr.6803 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:51:45.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.180+0000 7f01e37fe700 1 -- 172.21.15.55:0/3612336021 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01fc141b20 con 0x7f01fc0b0c00 2023-10-06T03:51:45.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.409+0000 7f020099d700 1 -- 172.21.15.55:0/3612336021 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01fc141b20 con 0x7f01e8066dd0 2023-10-06T03:51:45.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.419+0000 7f01e37fe700 1 -- 172.21.15.55:0/3612336021 <== mgr.6803 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f01fc141b20 con 0x7f01e8066dd0 2023-10-06T03:51:45.425 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:51:45.425 INFO:teuthology.orchestra.run.smithi055.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-10-06T03:51:45.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.421+0000 7f020099d700 1 -- 172.21.15.55:0/3612336021 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01e8066dd0 msgr2=0x7f01e8069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:45.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.421+0000 7f020099d700 1 --2- 172.21.15.55:0/3612336021 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01e8066dd0 0x7f01e8069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f01ec0099b0 tx=0x7f01ec017000 comp rx=0 tx=0).stop 2023-10-06T03:51:45.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.421+0000 7f020099d700 1 -- 172.21.15.55:0/3612336021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01fc0b0c00 msgr2=0x7f01fc0aacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:51:45.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.421+0000 7f020099d700 1 --2- 172.21.15.55:0/3612336021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01fc0b0c00 0x7f01fc0aacc0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f01f000c510 tx=0x7f01f000c820 comp rx=0 tx=0).stop 2023-10-06T03:51:45.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.421+0000 7f020099d700 1 -- 172.21.15.55:0/3612336021 shutdown_connections 2023-10-06T03:51:45.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.421+0000 7f020099d700 1 --2- 172.21.15.55:0/3612336021 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f01e8066dd0 0x7f01e8069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:45.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.421+0000 7f020099d700 1 --2- 172.21.15.55:0/3612336021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01fc156350 0x7f01fc16ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:45.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.421+0000 7f020099d700 1 --2- 172.21.15.55:0/3612336021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01fc0b0760 0x7f01fc141360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:45.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.421+0000 7f020099d700 1 --2- 172.21.15.55:0/3612336021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01fc0b0c00 0x7f01fc0aacc0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:51:45.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.421+0000 7f020099d700 1 -- 172.21.15.55:0/3612336021 >> 172.21.15.55:0/3612336021 conn(0x7f01fc068e20 msgr2=0x7f01fc152f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:51:45.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.422+0000 7f020099d700 1 -- 172.21.15.55:0/3612336021 shutdown_connections 2023-10-06T03:51:45.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:51:45.422+0000 7f020099d700 1 -- 172.21.15.55:0/3612336021 wait complete. 2023-10-06T03:51:45.441 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-10-06T03:51:45.441 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{}]} 2023-10-06T03:51:45.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:45.441 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:51:45.480 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:51:45.502 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:51:45.502 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:51:46.353 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:51:46.361 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:51:46.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:46.503 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:51:46.544 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:51:46.677 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:51:46.677 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:51:47.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:47.780 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:51:47.869 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:51:48.036 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:51:48.037 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:51:49.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:49.038 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:51:49.082 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:51:49.083 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:51:49.083 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:51:50.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:50.213 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:51:50.256 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:51:50.256 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:51:50.256 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:51:51.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:51.257 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:51:51.301 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:51:51.301 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:51:51.301 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:51:52.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:52.302 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:51:52.346 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:51:52.383 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:51:52.383 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:51:53.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:53.384 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:51:53.428 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:51:53.445 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:51:53.445 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:51:54.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:54.447 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:51:54.490 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:51:54.490 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:51:54.490 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:51:55.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:55.494 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:51:55.538 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:51:55.542 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:51:55.542 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:51:56.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:56.543 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:51:56.586 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:51:56.587 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:51:56.587 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:51:57.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:57.588 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:51:57.631 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:51:57.632 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:51:57.632 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:51:58.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:58.632 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:51:58.676 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:51:58.676 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:51:58.676 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:51:59.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:51:59.677 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:51:59.720 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:51:59.720 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:51:59.720 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:52:00.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:52:00.722 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:52:00.766 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T03:52:00.767 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T03:52:00.767 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:52:00.964 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:00.963+0000 7f6e60456700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:00.964 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:00.963+0000 7f6e60456700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:00.965 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:00.963+0000 7f6e60456700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:00.965 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:00.963+0000 7f6e60456700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:00.965 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:00.963+0000 7f6e60456700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:01.768 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:52:01.843 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:52:01.885 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_52_00_UTC 2023-10-06T03:52:01.915 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-03_52_00_UTC'} 2023-10-06T03:52:01.915 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-06-03_52_00_UTC'} 2023-10-06T03:52:01.916 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T03:52:01.916 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-06-03_52_00_UTC'] 2023-10-06T03:52:01.916 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696564304.9749978, now = 1696564321.9162211, timo = 16 2023-10-06T03:52:02.917 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-06T03:52:03.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.174+0000 7f463759d700 1 Processor -- start 2023-10-06T03:52:03.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.174+0000 7f463759d700 1 -- start start 2023-10-06T03:52:03.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.175+0000 7f463759d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4630153480 0x7f46300ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:03.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.175+0000 7f463759d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4630153850 0x7f46301596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:03.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.175+0000 7f463759d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f463015a050 con 0x7f4630159bf0 2023-10-06T03:52:03.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.175+0000 7f463759d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46300aec90 con 0x7f4630153850 2023-10-06T03:52:03.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.175+0000 7f463759d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463015a480 con 0x7f4630153480 2023-10-06T03:52:03.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.176+0000 7f4635d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4630153850 0x7f46301596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:03.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.176+0000 7f4635d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4630153850 0x7f46301596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38420/0 (socket says 172.21.15.55:38420) 2023-10-06T03:52:03.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.176+0000 7f4635d9a700 1 -- 172.21.15.55:0/4279589500 learned_addr learned my addr 172.21.15.55:0/4279589500 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:03.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.176+0000 7f4636d9c700 1 --1- 172.21.15.55:0/4279589500 >> v1:172.21.15.55:6789/0 conn(0x7f4630159bf0 0x7f46301560b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49528/0 (socket says 172.21.15.55:49528) 2023-10-06T03:52:03.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.176+0000 7f463659b700 1 --2- 172.21.15.55:0/4279589500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4630153480 0x7f46300ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:03.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.176+0000 7f4635d9a700 1 -- 172.21.15.55:0/4279589500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4630153480 msgr2=0x7f46300ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:03.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.176+0000 7f4635d9a700 1 --2- 172.21.15.55:0/4279589500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4630153480 0x7f46300ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:03.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.176+0000 7f4635d9a700 1 -- 172.21.15.55:0/4279589500 >> v1:172.21.15.55:6789/0 conn(0x7f4630159bf0 legacy=0x7f46301560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:03.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.176+0000 7f4635d9a700 1 -- 172.21.15.55:0/4279589500 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46300afb40 con 0x7f4630153850 2023-10-06T03:52:03.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.176+0000 7f4635d9a700 1 --2- 172.21.15.55:0/4279589500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4630153850 0x7f46301596b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f46200099b0 tx=0x7f4620009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92f66a0e93b6c65 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:03.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.177+0000 7f4635599700 1 -- 172.21.15.55:0/4279589500 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000b860 con 0x7f4630153850 2023-10-06T03:52:03.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.177+0000 7f4635599700 1 -- 172.21.15.55:0/4279589500 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462000b9c0 con 0x7f4630153850 2023-10-06T03:52:03.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.177+0000 7f4635599700 1 -- 172.21.15.55:0/4279589500 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462000bd60 con 0x7f4630153850 2023-10-06T03:52:03.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.177+0000 7f463759d700 1 -- 172.21.15.55:0/4279589500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4630153850 msgr2=0x7f46301596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:03.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.177+0000 7f463759d700 1 --2- 172.21.15.55:0/4279589500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4630153850 0x7f46301596b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f46200099b0 tx=0x7f4620009250 comp rx=0 tx=0).stop 2023-10-06T03:52:03.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.177+0000 7f463759d700 1 -- 172.21.15.55:0/4279589500 shutdown_connections 2023-10-06T03:52:03.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.177+0000 7f463759d700 1 --2- 172.21.15.55:0/4279589500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4630153480 0x7f46300ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:03.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.177+0000 7f463759d700 1 --2- 172.21.15.55:0/4279589500 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4630153850 0x7f46301596b0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:03.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.177+0000 7f463759d700 1 -- 172.21.15.55:0/4279589500 >> 172.21.15.55:0/4279589500 conn(0x7f4630068e20 msgr2=0x7f46300cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:03.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.177+0000 7f463759d700 1 -- 172.21.15.55:0/4279589500 shutdown_connections 2023-10-06T03:52:03.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.177+0000 7f463759d700 1 -- 172.21.15.55:0/4279589500 wait complete. 2023-10-06T03:52:03.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.179+0000 7f463759d700 1 Processor -- start 2023-10-06T03:52:03.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.179+0000 7f463759d700 1 -- start start 2023-10-06T03:52:03.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f463759d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4630153480 0x7f463015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:03.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f463759d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4630159bf0 0x7f46300abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:03.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f463759d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4630152170 0x7f4630157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:03.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f463759d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463015ef30 con 0x7f4630153480 2023-10-06T03:52:03.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f463759d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46300ac310 con 0x7f4630159bf0 2023-10-06T03:52:03.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f463759d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46300ad600 con 0x7f4630152170 2023-10-06T03:52:03.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f463659b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4630153480 0x7f463015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:03.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f463659b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4630153480 0x7f463015e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38436/0 (socket says 172.21.15.55:38436) 2023-10-06T03:52:03.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f463659b700 1 -- 172.21.15.55:0/4220102963 learned_addr learned my addr 172.21.15.55:0/4220102963 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:03.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f4636d9c700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4630152170 0x7f4630157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:03.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f463659b700 1 -- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4630152170 msgr2=0x7f4630157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:03.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f4635d9a700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4630159bf0 0x7f46300abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:03.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f463659b700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4630152170 0x7f4630157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:03.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f463659b700 1 -- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4630159bf0 msgr2=0x7f46300abdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:03.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f463659b700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4630159bf0 0x7f46300abdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:03.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f463659b700 1 -- 172.21.15.55:0/4220102963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46200094f0 con 0x7f4630153480 2023-10-06T03:52:03.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f4636d9c700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4630152170 0x7f4630157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:03.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.180+0000 7f4635d9a700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4630159bf0 0x7f46300abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:03.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.181+0000 7f463659b700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4630153480 0x7f463015e9f0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f462c00d890 tx=0x7f462c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:03.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.181+0000 7f46267fc700 1 -- 172.21.15.55:0/4220102963 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c00b600 con 0x7f4630153480 2023-10-06T03:52:03.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.181+0000 7f46267fc700 1 -- 172.21.15.55:0/4220102963 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c00b760 con 0x7f4630153480 2023-10-06T03:52:03.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.181+0000 7f46267fc700 1 -- 172.21.15.55:0/4220102963 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c00bae0 con 0x7f4630153480 2023-10-06T03:52:03.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.181+0000 7f463759d700 1 -- 172.21.15.55:0/4220102963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46300ac560 con 0x7f4630153480 2023-10-06T03:52:03.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.181+0000 7f463759d700 1 -- 172.21.15.55:0/4220102963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46300acb20 con 0x7f4630153480 2023-10-06T03:52:03.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.182+0000 7f463759d700 1 -- 172.21.15.55:0/4220102963 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46300ad450 con 0x7f4630153480 2023-10-06T03:52:03.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.183+0000 7f46267fc700 1 -- 172.21.15.55:0/4220102963 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 54) v1 ==== 84973+0+0 (secure 0 0 0) 0x7f462c01e460 con 0x7f4630153480 2023-10-06T03:52:03.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.184+0000 7f46267fc700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f46140671c0 0x7f4614069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:03.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.184+0000 7f46267fc700 1 -- 172.21.15.55:0/4220102963 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f462c085390 con 0x7f4630153480 2023-10-06T03:52:03.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.184+0000 7f4635d9a700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f46140671c0 0x7f4614069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:03.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.185+0000 7f4635d9a700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f46140671c0 0x7f4614069670 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f46200099b0 tx=0x7f4620017000 comp rx=0 tx=0).ready entity=mgr.6803 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:03.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.188+0000 7f46267fc700 1 -- 172.21.15.55:0/4220102963 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f462c051f80 con 0x7f4630153480 2023-10-06T03:52:03.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.416+0000 7f463759d700 1 -- 172.21.15.55:0/4220102963 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f46300afe20 con 0x7f46140671c0 2023-10-06T03:52:03.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.491+0000 7f46267fc700 1 -- 172.21.15.55:0/4220102963 <== mgr.6803 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f46300afe20 con 0x7f46140671c0 2023-10-06T03:52:03.543 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path snap_test_dir1 2023-10-06T03:52:03.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.494+0000 7f463759d700 1 -- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f46140671c0 msgr2=0x7f4614069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:03.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.494+0000 7f463759d700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f46140671c0 0x7f4614069670 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f46200099b0 tx=0x7f4620017000 comp rx=0 tx=0).stop 2023-10-06T03:52:03.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.494+0000 7f463759d700 1 -- 172.21.15.55:0/4220102963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4630153480 msgr2=0x7f463015e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:03.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.494+0000 7f463759d700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4630153480 0x7f463015e9f0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f462c00d890 tx=0x7f462c008040 comp rx=0 tx=0).stop 2023-10-06T03:52:03.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.494+0000 7f463759d700 1 -- 172.21.15.55:0/4220102963 shutdown_connections 2023-10-06T03:52:03.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.494+0000 7f463759d700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f46140671c0 0x7f4614069670 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:03.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.494+0000 7f463759d700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4630152170 0x7f4630157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:03.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.494+0000 7f463759d700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4630153480 0x7f463015e9f0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:03.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.494+0000 7f463759d700 1 --2- 172.21.15.55:0/4220102963 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4630159bf0 0x7f46300abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:03.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.494+0000 7f463759d700 1 -- 172.21.15.55:0/4220102963 >> 172.21.15.55:0/4220102963 conn(0x7f4630068e20 msgr2=0x7f463006e740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:03.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.494+0000 7f463759d700 1 -- 172.21.15.55:0/4220102963 shutdown_connections 2023-10-06T03:52:03.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.494+0000 7f463759d700 1 -- 172.21.15.55:0/4220102963 wait complete. 2023-10-06T03:52:03.545 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-06T03:52:03.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:52:03.545 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T03:52:03.582 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_52_00_UTC 2023-10-06T03:52:03.583 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-03_52_00_UTC 2023-10-06T03:52:03.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:52:03.584 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-03_52_00_UTC) 2023-10-06T03:52:03.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:52:03.668 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-06T03:52:03.747 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T03:52:03.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.978+0000 7efea6d9d700 1 Processor -- start 2023-10-06T03:52:04.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.978+0000 7efea6d9d700 1 -- start start 2023-10-06T03:52:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.979+0000 7efea6d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea015a3b0 0x7efea015a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:04.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.979+0000 7efea6d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efea00aef00 0x7efea015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.979+0000 7efea6d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea00aea70 con 0x7efea006e470 2023-10-06T03:52:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.979+0000 7efea6d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea00ae480 con 0x7efea00aef00 2023-10-06T03:52:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.979+0000 7efea6d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea015acc0 con 0x7efea015a3b0 2023-10-06T03:52:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.979+0000 7efea659c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7efea006e470 0x7efea00c7280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49538/0 (socket says 172.21.15.55:49538) 2023-10-06T03:52:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.979+0000 7efea659c700 1 -- 172.21.15.55:0/547862363 learned_addr learned my addr 172.21.15.55:0/547862363 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.980+0000 7efea5d9b700 1 --2- 172.21.15.55:0/547862363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea015a3b0 0x7efea015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.980+0000 7efea559a700 1 --2- 172.21.15.55:0/547862363 >> v2:172.21.15.163:3300/0 conn(0x7efea00aef00 0x7efea015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.980+0000 7efea4d99700 1 -- 172.21.15.55:0/547862363 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737470815 0 0) 0x7efea00aea70 con 0x7efea006e470 2023-10-06T03:52:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.980+0000 7efea4d99700 1 -- 172.21.15.55:0/547862363 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe8c003570 con 0x7efea006e470 2023-10-06T03:52:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.980+0000 7efea4d99700 1 -- 172.21.15.55:0/547862363 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2933181972 0 0) 0x7efe8c003570 con 0x7efea006e470 2023-10-06T03:52:04.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.980+0000 7efea4d99700 1 -- 172.21.15.55:0/547862363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea015a3b0 msgr2=0x7efea015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:04.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.980+0000 7efea4d99700 1 --2- 172.21.15.55:0/547862363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea015a3b0 0x7efea015a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:04.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.980+0000 7efea4d99700 1 -- 172.21.15.55:0/547862363 >> v2:172.21.15.163:3300/0 conn(0x7efea00aef00 msgr2=0x7efea015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:04.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.980+0000 7efea4d99700 1 --2- 172.21.15.55:0/547862363 >> v2:172.21.15.163:3300/0 conn(0x7efea00aef00 0x7efea015d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:04.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.980+0000 7efea4d99700 1 -- 172.21.15.55:0/547862363 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea015dd30 con 0x7efea006e470 2023-10-06T03:52:04.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.980+0000 7efea4d99700 1 -- 172.21.15.55:0/547862363 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7efe980017f0 con 0x7efea006e470 2023-10-06T03:52:04.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.980+0000 7efea4d99700 1 -- 172.21.15.55:0/547862363 >> v1:172.21.15.55:6789/0 conn(0x7efea006e470 legacy=0x7efea00c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:04.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.980+0000 7efea4d99700 1 --2- 172.21.15.55:0/547862363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe8c003a60 0x7efe8c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:04.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea4d99700 1 --2- 172.21.15.55:0/547862363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea015a3b0 0x7efe8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:04.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea4d99700 1 --2- 172.21.15.55:0/547862363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe8c004d50 0x7efe8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:04.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea4d99700 1 -- 172.21.15.55:0/547862363 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea00ae480 con 0x7efe8c003a60 2023-10-06T03:52:04.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea4d99700 1 -- 172.21.15.55:0/547862363 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea015acc0 con 0x7efe8c004d50 2023-10-06T03:52:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea4d99700 1 -- 172.21.15.55:0/547862363 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c00ba60 con 0x7efea015a3b0 2023-10-06T03:52:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea559a700 1 --2- 172.21.15.55:0/547862363 >> v2:172.21.15.163:3300/0 conn(0x7efea00aef00 0x7efea015d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:52:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea5d9b700 1 --2- 172.21.15.55:0/547862363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe8c003a60 0x7efe8c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea659c700 1 --2- 172.21.15.55:0/547862363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe8c004d50 0x7efe8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea559a700 1 --2- 172.21.15.55:0/547862363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea015a3b0 0x7efe8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea5d9b700 1 -- 172.21.15.55:0/547862363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea015a3b0 msgr2=0x7efe8c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea5d9b700 1 --2- 172.21.15.55:0/547862363 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea015a3b0 0x7efe8c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea5d9b700 1 -- 172.21.15.55:0/547862363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe8c004d50 msgr2=0x7efe8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea5d9b700 1 --2- 172.21.15.55:0/547862363 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe8c004d50 0x7efe8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea5d9b700 1 -- 172.21.15.55:0/547862363 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c003570 con 0x7efe8c003a60 2023-10-06T03:52:04.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.981+0000 7efea5d9b700 1 --2- 172.21.15.55:0/547862363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe8c003a60 0x7efe8c003e70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009c00 tx=0x7efe9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ca9198b5101bb4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:04.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.982+0000 7efea4d99700 1 -- 172.21.15.55:0/547862363 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00e6b0 con 0x7efe8c003a60 2023-10-06T03:52:04.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.982+0000 7efea4d99700 1 -- 172.21.15.55:0/547862363 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c00e810 con 0x7efe8c003a60 2023-10-06T03:52:04.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.982+0000 7efea4d99700 1 -- 172.21.15.55:0/547862363 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00ebd0 con 0x7efe8c003a60 2023-10-06T03:52:04.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.982+0000 7efea6d9d700 1 -- 172.21.15.55:0/547862363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe8c003a60 msgr2=0x7efe8c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:04.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.982+0000 7efea6d9d700 1 --2- 172.21.15.55:0/547862363 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efe8c003a60 0x7efe8c003e70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009c00 tx=0x7efe9c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:04.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.982+0000 7efea659c700 1 -- 172.21.15.55:0/547862363 reap_dead start 2023-10-06T03:52:04.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.982+0000 7efea6d9d700 1 -- 172.21.15.55:0/547862363 shutdown_connections 2023-10-06T03:52:04.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.982+0000 7efea6d9d700 1 -- 172.21.15.55:0/547862363 >> 172.21.15.55:0/547862363 conn(0x7efea00c9f70 msgr2=0x7efea006dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:04.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.982+0000 7efea6d9d700 1 -- 172.21.15.55:0/547862363 shutdown_connections 2023-10-06T03:52:04.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.982+0000 7efea6d9d700 1 -- 172.21.15.55:0/547862363 wait complete. 2023-10-06T03:52:04.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.984+0000 7efea6d9d700 1 Processor -- start 2023-10-06T03:52:04.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.984+0000 7efea6d9d700 1 -- start start 2023-10-06T03:52:04.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea6d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe8c003a60 0x7efea0151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:04.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea0157830 0x7efea00b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:04.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea6d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea00c27c0 0x7efea00c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:04.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea6d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea01699f0 con 0x7efea0157830 2023-10-06T03:52:04.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea6d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea016a7f0 con 0x7efe8c003a60 2023-10-06T03:52:04.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea6d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea00c3150 con 0x7efea00c27c0 2023-10-06T03:52:04.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea559a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea0157830 0x7efea00b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:04.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea559a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea0157830 0x7efea00b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38460/0 (socket says 172.21.15.55:38460) 2023-10-06T03:52:04.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea559a700 1 -- 172.21.15.55:0/2048220193 learned_addr learned my addr 172.21.15.55:0/2048220193 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:04.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea5d9b700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe8c003a60 0x7efea0151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:04.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea659c700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea00c27c0 0x7efea00c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:04.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea559a700 1 -- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea00c27c0 msgr2=0x7efea00c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:04.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea559a700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea00c27c0 0x7efea00c2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:04.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea559a700 1 -- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe8c003a60 msgr2=0x7efea0151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:04.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea559a700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe8c003a60 0x7efea0151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:04.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea559a700 1 -- 172.21.15.55:0/2048220193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea0169be0 con 0x7efea0157830 2023-10-06T03:52:04.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.985+0000 7efea5d9b700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe8c003a60 0x7efea0151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:04.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.986+0000 7efea559a700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea0157830 0x7efea00b13c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7efe90011a20 tx=0x7efe90008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:04.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.986+0000 7efe95ffb700 1 -- 172.21.15.55:0/2048220193 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9000f8e0 con 0x7efea0157830 2023-10-06T03:52:04.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.986+0000 7efe95ffb700 1 -- 172.21.15.55:0/2048220193 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9000fa40 con 0x7efea0157830 2023-10-06T03:52:04.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.986+0000 7efe95ffb700 1 -- 172.21.15.55:0/2048220193 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9000fdc0 con 0x7efea0157830 2023-10-06T03:52:04.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.986+0000 7efea6d9d700 1 -- 172.21.15.55:0/2048220193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea0169ec0 con 0x7efea0157830 2023-10-06T03:52:04.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.986+0000 7efea6d9d700 1 -- 172.21.15.55:0/2048220193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea016a480 con 0x7efea0157830 2023-10-06T03:52:04.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.987+0000 7efea6d9d700 1 -- 172.21.15.55:0/2048220193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea006cfa0 con 0x7efea0157830 2023-10-06T03:52:04.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.988+0000 7efe95ffb700 1 -- 172.21.15.55:0/2048220193 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 54) v1 ==== 84973+0+0 (secure 0 0 0) 0x7efe90015670 con 0x7efea0157830 2023-10-06T03:52:04.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.989+0000 7efe95ffb700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efe840671c0 0x7efe84069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:04.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.989+0000 7efe95ffb700 1 -- 172.21.15.55:0/2048220193 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6708+0+0 (secure 0 0 0) 0x7efe900856d0 con 0x7efea0157830 2023-10-06T03:52:04.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.989+0000 7efea5d9b700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efe840671c0 0x7efe84069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:04.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.990+0000 7efea5d9b700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efe840671c0 0x7efe84069670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0091e0 tx=0x7efe9c01a000 comp rx=0 tx=0).ready entity=mgr.6803 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:04.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:03.993+0000 7efe95ffb700 1 -- 172.21.15.55:0/2048220193 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe90052220 con 0x7efea0157830 2023-10-06T03:52:04.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.248+0000 7efea6d9d700 1 -- 172.21.15.55:0/2048220193 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7efea015e010 con 0x7efea0157830 2023-10-06T03:52:04.834 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.832+0000 7efe95ffb700 1 -- 172.21.15.55:0/2048220193 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 55) v1 ==== 84956+0+0 (secure 0 0 0) 0x7efe9005b0c0 con 0x7efea0157830 2023-10-06T03:52:05.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.834+0000 7efe95ffb700 1 -- 172.21.15.55:0/2048220193 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v55) v1 ==== 93+0+0 (secure 0 0 0) 0x7efe90052220 con 0x7efea0157830 2023-10-06T03:52:05.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.837+0000 7efea6d9d700 1 -- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efe840671c0 msgr2=0x7efe84069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:05.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.837+0000 7efea6d9d700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efe840671c0 0x7efe84069670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe9c0091e0 tx=0x7efe9c01a000 comp rx=0 tx=0).stop 2023-10-06T03:52:05.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.837+0000 7efea6d9d700 1 -- 172.21.15.55:0/2048220193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea0157830 msgr2=0x7efea00b13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:05.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.837+0000 7efea6d9d700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea0157830 0x7efea00b13c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7efe90011a20 tx=0x7efe90008040 comp rx=0 tx=0).stop 2023-10-06T03:52:05.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.837+0000 7efea6d9d700 1 -- 172.21.15.55:0/2048220193 shutdown_connections 2023-10-06T03:52:05.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.837+0000 7efea6d9d700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7efe840671c0 0x7efe84069670 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:05.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.837+0000 7efea6d9d700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efea00c27c0 0x7efea00c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:05.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.837+0000 7efea6d9d700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efe8c003a60 0x7efea0151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:05.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.837+0000 7efea6d9d700 1 --2- 172.21.15.55:0/2048220193 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efea0157830 0x7efea00b13c0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:05.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.837+0000 7efea6d9d700 1 -- 172.21.15.55:0/2048220193 >> 172.21.15.55:0/2048220193 conn(0x7efea00c9f70 msgr2=0x7efea0153570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:05.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.837+0000 7efea6d9d700 1 -- 172.21.15.55:0/2048220193 shutdown_connections 2023-10-06T03:52:05.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:04.837+0000 7efea6d9d700 1 -- 172.21.15.55:0/2048220193 wait complete. 2023-10-06T03:52:05.636 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:52:05.637 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:52:05.720 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:52:06.118 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:06.117+0000 7f1bf9efe180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:52:06.259 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:06.163+0000 7f882846d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:52:06.260 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:52:06.260 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T03:52:06.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:52:06.260 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T03:52:06.292 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:06.291+0000 7f1bf9efe180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:52:06.337 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:06.336+0000 7f882846d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:52:06.346 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T03:52:06.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:52:06.411 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T03:52:06.412 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:06.378+0000 7f1bf9efe180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:52:06.427 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:06.426+0000 7f882846d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:52:06.450 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:52:06.604 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:06.602+0000 7f1bf9efe180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:52:06.665 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:06.661+0000 7f882846d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:52:06.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.679+0000 7fb77359d700 1 Processor -- start 2023-10-06T03:52:06.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.679+0000 7fb77359d700 1 -- start start 2023-10-06T03:52:06.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.680+0000 7fb77359d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76c1597f0 0x7fb76c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:06.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.680+0000 7fb77359d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb76c15d230 0x7fb76c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:06.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.680+0000 7fb77359d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb76c15dbd0 con 0x7fb76c159420 2023-10-06T03:52:06.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.680+0000 7fb77359d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76c0b11d0 con 0x7fb76c15d230 2023-10-06T03:52:06.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.681+0000 7fb77359d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c0b14b0 con 0x7fb76c1597f0 2023-10-06T03:52:06.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.681+0000 7fb772d9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb76c15d230 0x7fb76c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:06.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.681+0000 7fb771d9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76c1597f0 0x7fb76c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:06.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.681+0000 7fb772d9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb76c15d230 0x7fb76c15d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:52530/0 (socket says 172.21.15.55:52530) 2023-10-06T03:52:06.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.681+0000 7fb771d9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76c1597f0 0x7fb76c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:44564/0 (socket says 172.21.15.55:44564) 2023-10-06T03:52:06.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.681+0000 7fb772d9c700 1 -- 172.21.15.55:0/2438714568 learned_addr learned my addr 172.21.15.55:0/2438714568 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:06.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.681+0000 7fb77259b700 1 --1- 172.21.15.55:0/2438714568 >> v1:172.21.15.163:6789/0 conn(0x7fb76c159420 0x7fb76c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:39246/0 (socket says 172.21.15.55:39246) 2023-10-06T03:52:06.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.681+0000 7fb772d9c700 1 -- 172.21.15.55:0/2438714568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76c1597f0 msgr2=0x7fb76c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:06.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.681+0000 7fb772d9c700 1 --2- 172.21.15.55:0/2438714568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76c1597f0 0x7fb76c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:06.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.681+0000 7fb772d9c700 1 -- 172.21.15.55:0/2438714568 >> v1:172.21.15.163:6789/0 conn(0x7fb76c159420 legacy=0x7fb76c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:06.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.681+0000 7fb772d9c700 1 -- 172.21.15.55:0/2438714568 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c0b1790 con 0x7fb76c15d230 2023-10-06T03:52:06.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.682+0000 7fb772d9c700 1 --2- 172.21.15.55:0/2438714568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb76c15d230 0x7fb76c15d690 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb764009f20 tx=0x7fb764009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8bf5469af8423df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:06.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.682+0000 7fb771599700 1 -- 172.21.15.55:0/2438714568 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76400e070 con 0x7fb76c15d230 2023-10-06T03:52:06.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.682+0000 7fb771599700 1 -- 172.21.15.55:0/2438714568 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76400b9d0 con 0x7fb76c15d230 2023-10-06T03:52:06.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.682+0000 7fb771599700 1 -- 172.21.15.55:0/2438714568 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7640092e0 con 0x7fb76c15d230 2023-10-06T03:52:06.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.683+0000 7fb77359d700 1 -- 172.21.15.55:0/2438714568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb76c15d230 msgr2=0x7fb76c15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:06.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.683+0000 7fb77359d700 1 --2- 172.21.15.55:0/2438714568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb76c15d230 0x7fb76c15d690 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb764009f20 tx=0x7fb764009840 comp rx=0 tx=0).stop 2023-10-06T03:52:06.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.683+0000 7fb77359d700 1 -- 172.21.15.55:0/2438714568 shutdown_connections 2023-10-06T03:52:06.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.683+0000 7fb77359d700 1 --2- 172.21.15.55:0/2438714568 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb76c15d230 0x7fb76c15d690 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:06.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.683+0000 7fb77359d700 1 --2- 172.21.15.55:0/2438714568 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76c1597f0 0x7fb76c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:06.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.683+0000 7fb77359d700 1 -- 172.21.15.55:0/2438714568 >> 172.21.15.55:0/2438714568 conn(0x7fb76c0d0660 msgr2=0x7fb76c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:06.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.683+0000 7fb77359d700 1 -- 172.21.15.55:0/2438714568 shutdown_connections 2023-10-06T03:52:06.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.683+0000 7fb77359d700 1 -- 172.21.15.55:0/2438714568 wait complete. 2023-10-06T03:52:06.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.684+0000 7fb77359d700 1 Processor -- start 2023-10-06T03:52:06.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.684+0000 7fb77359d700 1 -- start start 2023-10-06T03:52:06.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.685+0000 7fb77359d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb76c159420 0x7fb76c15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:06.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.685+0000 7fb77359d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb76c1597f0 0x7fb76c15c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:06.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.685+0000 7fb77359d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76c15d230 0x7fb76c157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:06.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.685+0000 7fb77359d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c0c95c0 con 0x7fb76c159420 2023-10-06T03:52:06.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.685+0000 7fb77359d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c0c9750 con 0x7fb76c1597f0 2023-10-06T03:52:06.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.685+0000 7fb77359d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76c15ac90 con 0x7fb76c15d230 2023-10-06T03:52:06.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.686+0000 7fb771d9a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb76c1597f0 0x7fb76c15c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:06.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.686+0000 7fb771d9a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb76c1597f0 0x7fb76c15c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:52532/0 (socket says 172.21.15.55:52532) 2023-10-06T03:52:06.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.686+0000 7fb77259b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb76c159420 0x7fb76c15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:06.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.686+0000 7fb771d9a700 1 -- 172.21.15.55:0/2582493164 learned_addr learned my addr 172.21.15.55:0/2582493164 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:06.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.686+0000 7fb772d9c700 1 --2- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76c15d230 0x7fb76c157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:06.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.686+0000 7fb771d9a700 1 -- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76c15d230 msgr2=0x7fb76c157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:06.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.686+0000 7fb771d9a700 1 --2- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76c15d230 0x7fb76c157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:06.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.686+0000 7fb771d9a700 1 -- 172.21.15.55:0/2582493164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb76c159420 msgr2=0x7fb76c15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:06.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.686+0000 7fb771d9a700 1 --2- 172.21.15.55:0/2582493164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb76c159420 0x7fb76c15a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:06.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.686+0000 7fb771d9a700 1 -- 172.21.15.55:0/2582493164 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb764009ae0 con 0x7fb76c1597f0 2023-10-06T03:52:06.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.686+0000 7fb771d9a700 1 --2- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb76c1597f0 0x7fb76c15c870 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00ec80 tx=0x7fb75c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:06.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.687+0000 7fb7627fc700 1 -- 172.21.15.55:0/2582493164 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c00c850 con 0x7fb76c1597f0 2023-10-06T03:52:06.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.687+0000 7fb7627fc700 1 -- 172.21.15.55:0/2582493164 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75c00c9b0 con 0x7fb76c1597f0 2023-10-06T03:52:06.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.687+0000 7fb7627fc700 1 -- 172.21.15.55:0/2582493164 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c00cd50 con 0x7fb76c1597f0 2023-10-06T03:52:06.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.687+0000 7fb77359d700 1 -- 172.21.15.55:0/2582493164 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76c15af70 con 0x7fb76c1597f0 2023-10-06T03:52:06.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.687+0000 7fb77359d700 1 -- 172.21.15.55:0/2582493164 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76c15b530 con 0x7fb76c1597f0 2023-10-06T03:52:06.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.688+0000 7fb77359d700 1 -- 172.21.15.55:0/2582493164 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb76c0ccf00 con 0x7fb76c1597f0 2023-10-06T03:52:06.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.689+0000 7fb7627fc700 1 -- 172.21.15.55:0/2582493164 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 55) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fb75c013680 con 0x7fb76c1597f0 2023-10-06T03:52:06.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.690+0000 7fb7627fc700 1 --2- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb750067170 0x7fb750069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:06.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.690+0000 7fb7627fc700 1 -- 172.21.15.55:0/2582493164 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6708+0+0 (secure 0 0 0) 0x7fb75c084f40 con 0x7fb76c1597f0 2023-10-06T03:52:06.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.690+0000 7fb77259b700 1 -- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb750067170 msgr2=0x7fb750069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:06.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.690+0000 7fb77259b700 1 --2- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb750067170 0x7fb750069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:52:06.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.698+0000 7fb7627fc700 1 -- 172.21.15.55:0/2582493164 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb75c051a90 con 0x7fb76c1597f0 2023-10-06T03:52:06.786 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:06.785+0000 7f1bf9efe180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:52:06.849 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:06.847+0000 7f882846d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:52:06.890 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:06.877+0000 7f1bf9efe180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:52:06.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.891+0000 7fb77259b700 1 -- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb750067170 msgr2=0x7fb750069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:06.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.891+0000 7fb77259b700 1 --2- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb750067170 0x7fb750069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:52:06.945 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:06.944+0000 7f882846d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:52:06.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.952+0000 7fb77359d700 1 -- 172.21.15.55:0/2582493164 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb76c04c3c0 con 0x7fb76c1597f0 2023-10-06T03:52:06.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.953+0000 7fb7627fc700 1 -- 172.21.15.55:0/2582493164 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fb75c0550b0 con 0x7fb76c1597f0 2023-10-06T03:52:06.960 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:06.961 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1295635544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":1295635544},{"type":"v1","addr":"172.21.15.55:6835","nonce":1295635544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7737,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2115790576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2115790576},{"type":"v1","addr":"172.21.15.163:6834","nonce":2115790576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1021807391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1021807391},{"type":"v1","addr":"172.21.15.163:6836","nonce":1021807391}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:51:26.480340+0000","modified":"2023-10-06T03:51:30.017376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7743":{"gid":7743,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/4240717455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":4240717455},{"type":"v1","addr":"172.21.15.55:6834","nonce":4240717455}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T03:52:06.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.956+0000 7fb77359d700 1 -- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb750067170 msgr2=0x7fb750069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:52:06.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.956+0000 7fb77359d700 1 --2- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb750067170 0x7fb750069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:06.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.956+0000 7fb77359d700 1 -- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb76c1597f0 msgr2=0x7fb76c15c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:06.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.956+0000 7fb77359d700 1 --2- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb76c1597f0 0x7fb76c15c870 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00ec80 tx=0x7fb75c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:06.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.956+0000 7fb77359d700 1 -- 172.21.15.55:0/2582493164 shutdown_connections 2023-10-06T03:52:06.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.956+0000 7fb77359d700 1 --2- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb750067170 0x7fb750069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:06.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.956+0000 7fb77359d700 1 --2- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb76c15d230 0x7fb76c157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:06.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.956+0000 7fb77359d700 1 --2- 172.21.15.55:0/2582493164 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb76c159420 0x7fb76c15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:06.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.956+0000 7fb77359d700 1 --2- 172.21.15.55:0/2582493164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb76c1597f0 0x7fb76c15c870 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:06.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.956+0000 7fb77359d700 1 -- 172.21.15.55:0/2582493164 >> 172.21.15.55:0/2582493164 conn(0x7fb76c0d0660 msgr2=0x7fb76c0b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:06.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.957+0000 7fb77359d700 1 -- 172.21.15.55:0/2582493164 shutdown_connections 2023-10-06T03:52:06.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:06.957+0000 7fb77359d700 1 -- 172.21.15.55:0/2582493164 wait complete. 2023-10-06T03:52:06.964 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 61 2023-10-06T03:52:06.983 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:52:07.043 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:07.042+0000 7f1bf9efe180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:52:07.125 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:07.123+0000 7f882846d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:52:07.144 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:07.143+0000 7f1bf9efe180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:52:07.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.228+0000 7f8131f9d700 1 Processor -- start 2023-10-06T03:52:07.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.228+0000 7f8131f9d700 1 -- start start 2023-10-06T03:52:07.267 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:07.230+0000 7f882846d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:52:07.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.230+0000 7f8131f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f812c159420 0x7f812c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:07.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.230+0000 7f8131f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f812c15d230 0x7f812c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:07.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.230+0000 7f8131f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812c0b2980 con 0x7f812c1597f0 2023-10-06T03:52:07.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.230+0000 7f8131f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812c15dbb0 con 0x7f812c159420 2023-10-06T03:52:07.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.230+0000 7f8131f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0b11d0 con 0x7f812c15d230 2023-10-06T03:52:07.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.230+0000 7f8130f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f812c159420 0x7f812c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:07.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.230+0000 7f8130f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f812c159420 0x7f812c0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38504/0 (socket says 172.21.15.55:38504) 2023-10-06T03:52:07.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.230+0000 7f8130f9b700 1 -- 172.21.15.55:0/1147281985 learned_addr learned my addr 172.21.15.55:0/1147281985 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:07.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.230+0000 7f813179c700 1 --2- 172.21.15.55:0/1147281985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f812c15d230 0x7f812c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:07.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.231+0000 7f8130f9b700 1 -- 172.21.15.55:0/1147281985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f812c15d230 msgr2=0x7f812c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:07.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.231+0000 7f8130f9b700 1 --2- 172.21.15.55:0/1147281985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f812c15d230 0x7f812c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.231+0000 7f8130f9b700 1 -- 172.21.15.55:0/1147281985 >> v1:172.21.15.163:6789/0 conn(0x7f812c1597f0 legacy=0x7f812c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:07.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.231+0000 7f8130f9b700 1 -- 172.21.15.55:0/1147281985 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c0b14b0 con 0x7f812c159420 2023-10-06T03:52:07.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.231+0000 7f8130f9b700 1 --2- 172.21.15.55:0/1147281985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f812c159420 0x7f812c0b5f40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f81200099b0 tx=0x7f8120009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3640046a733a94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:07.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.231+0000 7f812b7fe700 1 -- 172.21.15.55:0/1147281985 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120009ea0 con 0x7f812c159420 2023-10-06T03:52:07.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.231+0000 7f812b7fe700 1 -- 172.21.15.55:0/1147281985 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812000b860 con 0x7f812c159420 2023-10-06T03:52:07.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.231+0000 7f812b7fe700 1 -- 172.21.15.55:0/1147281985 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812000bc00 con 0x7f812c159420 2023-10-06T03:52:07.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.232+0000 7f8131f9d700 1 -- 172.21.15.55:0/1147281985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f812c159420 msgr2=0x7f812c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:07.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.232+0000 7f8131f9d700 1 --2- 172.21.15.55:0/1147281985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f812c159420 0x7f812c0b5f40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f81200099b0 tx=0x7f8120009250 comp rx=0 tx=0).stop 2023-10-06T03:52:07.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.232+0000 7f8131f9d700 1 -- 172.21.15.55:0/1147281985 shutdown_connections 2023-10-06T03:52:07.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.232+0000 7f8131f9d700 1 --2- 172.21.15.55:0/1147281985 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f812c15d230 0x7f812c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.232+0000 7f8131f9d700 1 --2- 172.21.15.55:0/1147281985 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f812c159420 0x7f812c0b5f40 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.232+0000 7f8131f9d700 1 -- 172.21.15.55:0/1147281985 >> 172.21.15.55:0/1147281985 conn(0x7f812c0d0660 msgr2=0x7f812c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:07.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.232+0000 7f8131f9d700 1 -- 172.21.15.55:0/1147281985 shutdown_connections 2023-10-06T03:52:07.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.232+0000 7f8131f9d700 1 -- 172.21.15.55:0/1147281985 wait complete. 2023-10-06T03:52:07.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.234+0000 7f8131f9d700 1 Processor -- start 2023-10-06T03:52:07.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.234+0000 7f8131f9d700 1 -- start start 2023-10-06T03:52:07.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f8131f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f812c159420 0x7f812c0c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:07.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f8131f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f812c1597f0 0x7f812c0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:07.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f8131f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f812c15d230 0x7f812c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:07.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f8131f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0c9020 con 0x7f812c15d230 2023-10-06T03:52:07.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f8131f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0c8650 con 0x7f812c159420 2023-10-06T03:52:07.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f8131f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0cbd00 con 0x7f812c1597f0 2023-10-06T03:52:07.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f813179c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f812c15d230 0x7f812c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:07.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f812bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f812c1597f0 0x7f812c0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:07.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f8130f9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f812c159420 0x7f812c0c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:07.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f813179c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f812c15d230 0x7f812c0cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38520/0 (socket says 172.21.15.55:38520) 2023-10-06T03:52:07.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f812bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f812c1597f0 0x7f812c0cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:44594/0 (socket says 172.21.15.55:44594) 2023-10-06T03:52:07.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f813179c700 1 -- 172.21.15.55:0/1316028961 learned_addr learned my addr 172.21.15.55:0/1316028961 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:07.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f813179c700 1 -- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f812c1597f0 msgr2=0x7f812c0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:07.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f813179c700 1 --2- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f812c1597f0 0x7f812c0cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.235+0000 7f813179c700 1 -- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f812c159420 msgr2=0x7f812c0c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:07.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.236+0000 7f813179c700 1 --2- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f812c159420 0x7f812c0c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.236+0000 7f813179c700 1 -- 172.21.15.55:0/1316028961 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81200094f0 con 0x7f812c15d230 2023-10-06T03:52:07.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.236+0000 7f812bfff700 1 --2- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f812c1597f0 0x7f812c0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:52:07.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.236+0000 7f8130f9b700 1 --2- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f812c159420 0x7f812c0c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:52:07.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.236+0000 7f813179c700 1 --2- 172.21.15.55:0/1316028961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f812c15d230 0x7f812c0cb7c0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f8124009e20 tx=0x7f812400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:07.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.236+0000 7f8128ff9700 1 -- 172.21.15.55:0/1316028961 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400caa0 con 0x7f812c15d230 2023-10-06T03:52:07.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.236+0000 7f8128ff9700 1 -- 172.21.15.55:0/1316028961 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812400f040 con 0x7f812c15d230 2023-10-06T03:52:07.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.236+0000 7f8128ff9700 1 -- 172.21.15.55:0/1316028961 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400ce50 con 0x7f812c15d230 2023-10-06T03:52:07.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.236+0000 7f8131f9d700 1 -- 172.21.15.55:0/1316028961 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c0c7a40 con 0x7f812c15d230 2023-10-06T03:52:07.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.236+0000 7f8131f9d700 1 -- 172.21.15.55:0/1316028961 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c0c8000 con 0x7f812c15d230 2023-10-06T03:52:07.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.237+0000 7f8131f9d700 1 -- 172.21.15.55:0/1316028961 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812c0c8800 con 0x7f812c15d230 2023-10-06T03:52:07.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.238+0000 7f8128ff9700 1 -- 172.21.15.55:0/1316028961 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 55) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f8124014690 con 0x7f812c15d230 2023-10-06T03:52:07.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.239+0000 7f8128ff9700 1 --2- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8110067170 0x7f8110069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:07.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.239+0000 7f8128ff9700 1 -- 172.21.15.55:0/1316028961 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f81240858e0 con 0x7f812c15d230 2023-10-06T03:52:07.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.240+0000 7f8130f9b700 1 -- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8110067170 msgr2=0x7f8110069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:07.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.240+0000 7f8130f9b700 1 --2- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8110067170 0x7f8110069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:52:07.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.244+0000 7f8128ff9700 1 -- 172.21.15.55:0/1316028961 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f812c0c8800 con 0x7f812c15d230 2023-10-06T03:52:07.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.441+0000 7f8130f9b700 1 -- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8110067170 msgr2=0x7f8110069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:07.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.441+0000 7f8130f9b700 1 --2- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8110067170 0x7f8110069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:52:07.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.449+0000 7f8131f9d700 1 -- 172.21.15.55:0/1316028961 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f812c0c8800 con 0x7f812c15d230 2023-10-06T03:52:07.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.451+0000 7f8128ff9700 1 -- 172.21.15.55:0/1316028961 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+17895 (secure 0 0 0) 0x7f812c0c8800 con 0x7f812c15d230 2023-10-06T03:52:07.493 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:07.494 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":124,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:51:43.863704+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:51:24.466543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T03:51:25.462282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/85191":"2023-10-07T03:51:43.863644+0000","172.21.15.163:6835/85191":"2023-10-07T03:51:43.863644+0000","172.21.15.163:0/2003915970":"2023-10-07T03:51:43.863644+0000","172.21.15.163:0/357062062":"2023-10-07T03:51:43.863644+0000","172.21.15.163:0/2409430869":"2023-10-07T03:51:43.863644+0000","172.21.15.163:0/3972526625":"2023-10-07T03:51:43.863644+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:52:07.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.453+0000 7f8131f9d700 1 -- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8110067170 msgr2=0x7f8110069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:52:07.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.453+0000 7f8131f9d700 1 --2- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8110067170 0x7f8110069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.453+0000 7f8131f9d700 1 -- 172.21.15.55:0/1316028961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f812c15d230 msgr2=0x7f812c0cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:07.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.453+0000 7f8131f9d700 1 --2- 172.21.15.55:0/1316028961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f812c15d230 0x7f812c0cb7c0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f8124009e20 tx=0x7f812400b040 comp rx=0 tx=0).stop 2023-10-06T03:52:07.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.453+0000 7f8131f9d700 1 -- 172.21.15.55:0/1316028961 shutdown_connections 2023-10-06T03:52:07.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.453+0000 7f8131f9d700 1 --2- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8110067170 0x7f8110069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.453+0000 7f8131f9d700 1 --2- 172.21.15.55:0/1316028961 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f812c15d230 0x7f812c0cb7c0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.453+0000 7f8131f9d700 1 --2- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f812c159420 0x7f812c0c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.453+0000 7f8131f9d700 1 --2- 172.21.15.55:0/1316028961 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f812c1597f0 0x7f812c0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.453+0000 7f8131f9d700 1 -- 172.21.15.55:0/1316028961 >> 172.21.15.55:0/1316028961 conn(0x7f812c0d0660 msgr2=0x7f812c069a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:07.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.453+0000 7f8131f9d700 1 -- 172.21.15.55:0/1316028961 shutdown_connections 2023-10-06T03:52:07.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.453+0000 7f8131f9d700 1 -- 172.21.15.55:0/1316028961 wait complete. 2023-10-06T03:52:07.497 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:51:24.466543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:51:25.462282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T03:52:07.508 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:52:07.625 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:07.624+0000 7f1bf9efe180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:52:07.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.698+0000 7f24189fe700 1 Processor -- start 2023-10-06T03:52:07.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.698+0000 7f24189fe700 1 -- start start 2023-10-06T03:52:07.735 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:07.718+0000 7f1bf9efe180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:52:07.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.700+0000 7f24189fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f24140b1b40 0x7f24140b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:07.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.700+0000 7f24189fe700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24140cfb00 con 0x7f24140b4e70 2023-10-06T03:52:07.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.700+0000 7f24189fe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24140b5c40 con 0x7f24140b16d0 2023-10-06T03:52:07.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.700+0000 7f24189fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24140b5f20 con 0x7f24140b1b40 2023-10-06T03:52:07.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.700+0000 7f2412d9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f24140b4e70 0x7f24140ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49554/0 (socket says 172.21.15.55:49554) 2023-10-06T03:52:07.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.700+0000 7f2412d9d700 1 -- 172.21.15.55:0/668871283 learned_addr learned my addr 172.21.15.55:0/668871283 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:07.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.700+0000 7f241359e700 1 --2- 172.21.15.55:0/668871283 >> v2:172.21.15.163:3300/0 conn(0x7f24140b1b40 0x7f24140b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:07.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.701+0000 7f2411d9b700 1 -- 172.21.15.55:0/668871283 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3002076929 0 0) 0x7f24140cfb00 con 0x7f24140b4e70 2023-10-06T03:52:07.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.701+0000 7f2411d9b700 1 -- 172.21.15.55:0/668871283 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23fc003570 con 0x7f24140b4e70 2023-10-06T03:52:07.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.701+0000 7f2411d9b700 1 -- 172.21.15.55:0/668871283 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846281388 0 0) 0x7f24140b5c40 con 0x7f24140b16d0 2023-10-06T03:52:07.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.701+0000 7f2411d9b700 1 -- 172.21.15.55:0/668871283 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24140cfb00 con 0x7f24140b16d0 2023-10-06T03:52:07.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.701+0000 7f241359e700 1 -- 172.21.15.55:0/668871283 >> v1:172.21.15.163:6789/0 conn(0x7f24140b16d0 legacy=0x7f24140b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:07.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.701+0000 7f241359e700 1 -- 172.21.15.55:0/668871283 >> v1:172.21.15.55:6789/0 conn(0x7f24140b4e70 legacy=0x7f24140ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:07.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.701+0000 7f241359e700 1 -- 172.21.15.55:0/668871283 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24140b6200 con 0x7f24140b1b40 2023-10-06T03:52:07.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.701+0000 7f241359e700 1 --2- 172.21.15.55:0/668871283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24140b1b40 0x7f24140b3ac0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f240c009e60 tx=0x7f240c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=523d69333296dcb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:07.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.701+0000 7f2411d9b700 1 -- 172.21.15.55:0/668871283 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c00e070 con 0x7f24140b1b40 2023-10-06T03:52:07.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.702+0000 7f2411d9b700 1 -- 172.21.15.55:0/668871283 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240c00bab0 con 0x7f24140b1b40 2023-10-06T03:52:07.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.702+0000 7f2411d9b700 1 -- 172.21.15.55:0/668871283 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c00be50 con 0x7f24140b1b40 2023-10-06T03:52:07.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.702+0000 7f24189fe700 1 -- 172.21.15.55:0/668871283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24140b1b40 msgr2=0x7f24140b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:07.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.702+0000 7f24189fe700 1 --2- 172.21.15.55:0/668871283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24140b1b40 0x7f24140b3ac0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f240c009e60 tx=0x7f240c009780 comp rx=0 tx=0).stop 2023-10-06T03:52:07.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.702+0000 7f24189fe700 1 -- 172.21.15.55:0/668871283 shutdown_connections 2023-10-06T03:52:07.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.702+0000 7f24189fe700 1 --2- 172.21.15.55:0/668871283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24140b1b40 0x7f24140b3ac0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.702+0000 7f24189fe700 1 -- 172.21.15.55:0/668871283 >> 172.21.15.55:0/668871283 conn(0x7f24140d10c0 msgr2=0x7f24140affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:07.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.702+0000 7f24189fe700 1 -- 172.21.15.55:0/668871283 shutdown_connections 2023-10-06T03:52:07.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.702+0000 7f24189fe700 1 -- 172.21.15.55:0/668871283 wait complete. 2023-10-06T03:52:07.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.704+0000 7f24189fe700 1 Processor -- start 2023-10-06T03:52:07.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.704+0000 7f24189fe700 1 -- start start 2023-10-06T03:52:07.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.705+0000 7f24189fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24140b16d0 0x7f241416b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:07.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.705+0000 7f24189fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24140b1b40 0x7f241416d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:07.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.705+0000 7f24189fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24140b4e70 0x7f24140c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:07.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.705+0000 7f24189fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241416a690 con 0x7f24140b4e70 2023-10-06T03:52:07.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.705+0000 7f24189fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241416a970 con 0x7f24140b16d0 2023-10-06T03:52:07.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.705+0000 7f24189fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241416ac50 con 0x7f24140b1b40 2023-10-06T03:52:07.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.705+0000 7f241359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24140b4e70 0x7f24140c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:07.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.705+0000 7f241359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24140b4e70 0x7f24140c74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38522/0 (socket says 172.21.15.55:38522) 2023-10-06T03:52:07.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.705+0000 7f241359e700 1 -- 172.21.15.55:0/3312740347 learned_addr learned my addr 172.21.15.55:0/3312740347 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:07.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.705+0000 7f2412d9d700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24140b16d0 0x7f241416b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:07.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.705+0000 7f241259c700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24140b1b40 0x7f241416d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:07.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.705+0000 7f241359e700 1 -- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24140b1b40 msgr2=0x7f241416d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:07.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.705+0000 7f241359e700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24140b1b40 0x7f241416d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.706+0000 7f241359e700 1 -- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24140b16d0 msgr2=0x7f241416b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:07.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.706+0000 7f241359e700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24140b16d0 0x7f241416b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.706+0000 7f241359e700 1 -- 172.21.15.55:0/3312740347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c009a20 con 0x7f24140b4e70 2023-10-06T03:52:07.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.706+0000 7f241259c700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24140b1b40 0x7f241416d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:07.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.706+0000 7f241359e700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24140b4e70 0x7f24140c74e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f2414070dd0 tx=0x7f240c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:07.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.706+0000 7f23faffd700 1 -- 172.21.15.55:0/3312740347 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c012db0 con 0x7f24140b4e70 2023-10-06T03:52:07.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.706+0000 7f23faffd700 1 -- 172.21.15.55:0/3312740347 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240c0263f0 con 0x7f24140b4e70 2023-10-06T03:52:07.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.706+0000 7f23faffd700 1 -- 172.21.15.55:0/3312740347 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c026790 con 0x7f24140b4e70 2023-10-06T03:52:07.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.706+0000 7f24189fe700 1 -- 172.21.15.55:0/3312740347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241416af30 con 0x7f24140b4e70 2023-10-06T03:52:07.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.706+0000 7f24189fe700 1 -- 172.21.15.55:0/3312740347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241416a2b0 con 0x7f24140b4e70 2023-10-06T03:52:07.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.707+0000 7f24189fe700 1 -- 172.21.15.55:0/3312740347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241416b2a0 con 0x7f24140b4e70 2023-10-06T03:52:07.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.708+0000 7f23faffd700 1 -- 172.21.15.55:0/3312740347 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 55) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f240c026bc0 con 0x7f24140b4e70 2023-10-06T03:52:07.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.709+0000 7f23faffd700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2400067170 0x7f2400069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:07.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.710+0000 7f23faffd700 1 -- 172.21.15.55:0/3312740347 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f240c086d50 con 0x7f24140b4e70 2023-10-06T03:52:07.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.710+0000 7f2412d9d700 1 -- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2400067170 msgr2=0x7f2400069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:07.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.710+0000 7f2412d9d700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2400067170 0x7f2400069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:52:07.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.714+0000 7f23faffd700 1 -- 172.21.15.55:0/3312740347 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f240c053790 con 0x7f24140b4e70 2023-10-06T03:52:07.745 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:07.744+0000 7f882846d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:52:07.809 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:07.808+0000 7f1bf9efe180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:52:07.843 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:07.842+0000 7f882846d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:52:07.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.911+0000 7f2412d9d700 1 -- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2400067170 msgr2=0x7f2400069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:07.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.911+0000 7f2412d9d700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2400067170 0x7f2400069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:52:07.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.915+0000 7f24189fe700 1 -- 172.21.15.55:0/3312740347 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f241415d180 con 0x7f24140b4e70 2023-10-06T03:52:07.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.915+0000 7f23faffd700 1 -- 172.21.15.55:0/3312740347 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v124) v1 ==== 92+0+10 (secure 0 0 0) 0x7f240c056db0 con 0x7f24140b4e70 2023-10-06T03:52:07.917 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:52:07.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.917+0000 7f24189fe700 1 -- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2400067170 msgr2=0x7f2400069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:52:07.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.917+0000 7f24189fe700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2400067170 0x7f2400069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.917+0000 7f24189fe700 1 -- 172.21.15.55:0/3312740347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24140b4e70 msgr2=0x7f24140c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.917+0000 7f24189fe700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24140b4e70 0x7f24140c74e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f2414070dd0 tx=0x7f240c019000 comp rx=0 tx=0).stop 2023-10-06T03:52:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.918+0000 7f24189fe700 1 -- 172.21.15.55:0/3312740347 shutdown_connections 2023-10-06T03:52:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.918+0000 7f24189fe700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2400067170 0x7f2400069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.918+0000 7f24189fe700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24140b4e70 0x7f24140c74e0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.918+0000 7f24189fe700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24140b16d0 0x7f241416b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.918+0000 7f24189fe700 1 --2- 172.21.15.55:0/3312740347 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24140b1b40 0x7f241416d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:07.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.918+0000 7f24189fe700 1 -- 172.21.15.55:0/3312740347 >> 172.21.15.55:0/3312740347 conn(0x7f24140d10c0 msgr2=0x7f24140ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:07.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.918+0000 7f24189fe700 1 -- 172.21.15.55:0/3312740347 shutdown_connections 2023-10-06T03:52:07.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:07.918+0000 7f24189fe700 1 -- 172.21.15.55:0/3312740347 wait complete. 2023-10-06T03:52:07.937 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:52:07.968 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:07.938+0000 7f882846d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:52:08.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.163+0000 7f294fbff700 1 Processor -- start 2023-10-06T03:52:08.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.163+0000 7f294fbff700 1 -- start start 2023-10-06T03:52:08.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.164+0000 7f294fbff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f294806bcd0 0x7f29480caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:08.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.164+0000 7f294fbff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294815d7e0 con 0x7f294815df50 2023-10-06T03:52:08.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.164+0000 7f294fbff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294815dac0 con 0x7f294806c0a0 2023-10-06T03:52:08.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.164+0000 7f294fbff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294815aa30 con 0x7f294806bcd0 2023-10-06T03:52:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.165+0000 7f294e19c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f294815df50 0x7f29480c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49566/0 (socket says 172.21.15.55:49566) 2023-10-06T03:52:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.165+0000 7f294e19c700 1 -- 172.21.15.55:0/1924935839 learned_addr learned my addr 172.21.15.55:0/1924935839 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.165+0000 7f294d99b700 1 --2- 172.21.15.55:0/1924935839 >> v2:172.21.15.163:3300/0 conn(0x7f294806bcd0 0x7f29480caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.165+0000 7f294c999700 1 -- 172.21.15.55:0/1924935839 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511089599 0 0) 0x7f294815d7e0 con 0x7f294815df50 2023-10-06T03:52:08.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.165+0000 7f294c999700 1 -- 172.21.15.55:0/1924935839 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2934003570 con 0x7f294815df50 2023-10-06T03:52:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.165+0000 7f294d99b700 1 -- 172.21.15.55:0/1924935839 >> v1:172.21.15.163:6789/0 conn(0x7f294806c0a0 legacy=0x7f294815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.165+0000 7f294d99b700 1 -- 172.21.15.55:0/1924935839 >> v1:172.21.15.55:6789/0 conn(0x7f294815df50 legacy=0x7f29480c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.165+0000 7f294d99b700 1 -- 172.21.15.55:0/1924935839 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294815ad10 con 0x7f294806bcd0 2023-10-06T03:52:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.166+0000 7f294d99b700 1 --2- 172.21.15.55:0/1924935839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f294806bcd0 0x7f29480caa90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f29440098e0 tx=0x7f29440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d5a2f12422b139 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.166+0000 7f294c999700 1 -- 172.21.15.55:0/1924935839 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294400b640 con 0x7f294806bcd0 2023-10-06T03:52:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.166+0000 7f294c999700 1 -- 172.21.15.55:0/1924935839 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294400b7a0 con 0x7f294806bcd0 2023-10-06T03:52:08.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.166+0000 7f294c999700 1 -- 172.21.15.55:0/1924935839 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294400bb40 con 0x7f294806bcd0 2023-10-06T03:52:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.166+0000 7f294fbff700 1 -- 172.21.15.55:0/1924935839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f294806bcd0 msgr2=0x7f29480caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.166+0000 7f294fbff700 1 --2- 172.21.15.55:0/1924935839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f294806bcd0 0x7f29480caa90 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f29440098e0 tx=0x7f29440091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.167+0000 7f294fbff700 1 -- 172.21.15.55:0/1924935839 shutdown_connections 2023-10-06T03:52:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.167+0000 7f294fbff700 1 --2- 172.21.15.55:0/1924935839 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f294806bcd0 0x7f29480caa90 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.167+0000 7f294fbff700 1 -- 172.21.15.55:0/1924935839 >> 172.21.15.55:0/1924935839 conn(0x7f29480ce600 msgr2=0x7f29480cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:08.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.167+0000 7f294fbff700 1 -- 172.21.15.55:0/1924935839 shutdown_connections 2023-10-06T03:52:08.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.167+0000 7f294fbff700 1 -- 172.21.15.55:0/1924935839 wait complete. 2023-10-06T03:52:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.168+0000 7f294fbff700 1 Processor -- start 2023-10-06T03:52:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.168+0000 7f294fbff700 1 -- start start 2023-10-06T03:52:08.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.169+0000 7f294fbff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f294806bcd0 0x7f29481523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:08.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.169+0000 7f294fbff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f294806c0a0 0x7f294816a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:08.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.169+0000 7f294fbff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f294815df50 0x7f29481574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:08.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.169+0000 7f294fbff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294816acc0 con 0x7f294815df50 2023-10-06T03:52:08.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.169+0000 7f294fbff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294816afa0 con 0x7f294806bcd0 2023-10-06T03:52:08.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.169+0000 7f294fbff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294816b280 con 0x7f294806c0a0 2023-10-06T03:52:08.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.169+0000 7f294e19c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f294815df50 0x7f29481574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:08.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.169+0000 7f294d99b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f294806bcd0 0x7f29481523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:08.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.170+0000 7f294d99b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f294806bcd0 0x7f29481523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:52572/0 (socket says 172.21.15.55:52572) 2023-10-06T03:52:08.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.170+0000 7f294d99b700 1 -- 172.21.15.55:0/2789000743 learned_addr learned my addr 172.21.15.55:0/2789000743 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:08.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.170+0000 7f294d19a700 1 --2- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f294806c0a0 0x7f294816a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:08.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.170+0000 7f294d99b700 1 -- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f294806c0a0 msgr2=0x7f294816a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:08.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.170+0000 7f294d99b700 1 --2- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f294806c0a0 0x7f294816a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.170+0000 7f294d99b700 1 -- 172.21.15.55:0/2789000743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f294815df50 msgr2=0x7f29481574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:08.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.170+0000 7f294d99b700 1 --2- 172.21.15.55:0/2789000743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f294815df50 0x7f29481574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.170+0000 7f294d99b700 1 -- 172.21.15.55:0/2789000743 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2944009420 con 0x7f294806bcd0 2023-10-06T03:52:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.170+0000 7f294d99b700 1 --2- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f294806bcd0 0x7f29481523e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f29440093f0 tx=0x7f2944017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.171+0000 7f293dffb700 1 -- 172.21.15.55:0/2789000743 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944006af0 con 0x7f294806bcd0 2023-10-06T03:52:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.171+0000 7f293dffb700 1 -- 172.21.15.55:0/2789000743 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2944000bc0 con 0x7f294806bcd0 2023-10-06T03:52:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.171+0000 7f293dffb700 1 -- 172.21.15.55:0/2789000743 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294400bb40 con 0x7f294806bcd0 2023-10-06T03:52:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.171+0000 7f294fbff700 1 -- 172.21.15.55:0/2789000743 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29480b2aa0 con 0x7f294806bcd0 2023-10-06T03:52:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.171+0000 7f294fbff700 1 -- 172.21.15.55:0/2789000743 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2948152920 con 0x7f294806bcd0 2023-10-06T03:52:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.172+0000 7f294fbff700 1 -- 172.21.15.55:0/2789000743 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29480b2fc0 con 0x7f294806bcd0 2023-10-06T03:52:08.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.173+0000 7f293dffb700 1 -- 172.21.15.55:0/2789000743 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 55) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f2944023070 con 0x7f294806bcd0 2023-10-06T03:52:08.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.173+0000 7f293dffb700 1 --2- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f292c067170 0x7f292c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:08.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.174+0000 7f293dffb700 1 -- 172.21.15.55:0/2789000743 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f2944085f10 con 0x7f294806bcd0 2023-10-06T03:52:08.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.174+0000 7f294d19a700 1 -- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f292c067170 msgr2=0x7f292c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:08.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.174+0000 7f294d19a700 1 --2- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f292c067170 0x7f292c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:52:08.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.178+0000 7f293dffb700 1 -- 172.21.15.55:0/2789000743 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2944052990 con 0x7f294806bcd0 2023-10-06T03:52:08.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.375+0000 7f294d19a700 1 -- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f292c067170 msgr2=0x7f292c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:08.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.375+0000 7f294d19a700 1 --2- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f292c067170 0x7f292c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:52:08.432 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:08.430+0000 7f1bf9efe180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:52:08.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.381+0000 7f294fbff700 1 -- 172.21.15.55:0/2789000743 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f294804c3c0 con 0x7f294806bcd0 2023-10-06T03:52:08.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.382+0000 7f293dffb700 1 -- 172.21.15.55:0/2789000743 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29440150a0 con 0x7f294806bcd0 2023-10-06T03:52:08.433 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:52:08.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.384+0000 7f294fbff700 1 -- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f292c067170 msgr2=0x7f292c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:52:08.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.384+0000 7f294fbff700 1 --2- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f292c067170 0x7f292c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.384+0000 7f294fbff700 1 -- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f294806bcd0 msgr2=0x7f29481523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:08.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.384+0000 7f294fbff700 1 --2- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f294806bcd0 0x7f29481523e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f29440093f0 tx=0x7f2944017000 comp rx=0 tx=0).stop 2023-10-06T03:52:08.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.385+0000 7f294fbff700 1 -- 172.21.15.55:0/2789000743 shutdown_connections 2023-10-06T03:52:08.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.385+0000 7f294fbff700 1 --2- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f292c067170 0x7f292c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.385+0000 7f294fbff700 1 --2- 172.21.15.55:0/2789000743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f294815df50 0x7f29481574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.385+0000 7f294fbff700 1 --2- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f294806bcd0 0x7f29481523e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.385+0000 7f294fbff700 1 --2- 172.21.15.55:0/2789000743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f294806c0a0 0x7f294816a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.385+0000 7f294fbff700 1 -- 172.21.15.55:0/2789000743 >> 172.21.15.55:0/2789000743 conn(0x7f29480ce600 msgr2=0x7f294815c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:08.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.385+0000 7f294fbff700 1 -- 172.21.15.55:0/2789000743 shutdown_connections 2023-10-06T03:52:08.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.385+0000 7f294fbff700 1 -- 172.21.15.55:0/2789000743 wait complete. 2023-10-06T03:52:08.435 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T03:52:08.597 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:08.595+0000 7f882846d180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:52:08.604 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:08.603+0000 7f1bf9efe180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:52:08.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.660+0000 7f517f59e700 1 Processor -- start 2023-10-06T03:52:08.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.660+0000 7f517f59e700 1 -- start start 2023-10-06T03:52:08.914 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:08.699+0000 7f1bf9efe180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:52:08.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.661+0000 7f517f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5180159a60 0x7f51800c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:08.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.661+0000 7f517f59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f51800ccc00 0x7f5180069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:08.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.661+0000 7f517f59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f518006a020 0x7f51800cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:08.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.661+0000 7f517f59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518006dfc0 con 0x7f518006a020 2023-10-06T03:52:08.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.661+0000 7f517f59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51800c9cb0 con 0x7f51800ccc00 2023-10-06T03:52:08.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.661+0000 7f517f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51800c9f30 con 0x7f5180159a60 2023-10-06T03:52:08.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.662+0000 7f517ed9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f518006a020 0x7f51800cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:08.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.662+0000 7f517ed9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f518006a020 0x7f51800cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38544/0 (socket says 172.21.15.55:38544) 2023-10-06T03:52:08.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.662+0000 7f517ed9d700 1 -- 172.21.15.55:0/1792632722 learned_addr learned my addr 172.21.15.55:0/1792632722 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:08.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.662+0000 7f517dd9b700 1 --2- 172.21.15.55:0/1792632722 >> v2:172.21.15.163:3300/0 conn(0x7f51800ccc00 0x7f5180069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:08.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.662+0000 7f517e59c700 1 --2- 172.21.15.55:0/1792632722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5180159a60 0x7f51800c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:08.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.662+0000 7f517ed9d700 1 -- 172.21.15.55:0/1792632722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5180159a60 msgr2=0x7f51800c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:08.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.662+0000 7f517ed9d700 1 --2- 172.21.15.55:0/1792632722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5180159a60 0x7f51800c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.662+0000 7f517ed9d700 1 -- 172.21.15.55:0/1792632722 >> v2:172.21.15.163:3300/0 conn(0x7f51800ccc00 msgr2=0x7f5180069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:08.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.662+0000 7f517ed9d700 1 --2- 172.21.15.55:0/1792632722 >> v2:172.21.15.163:3300/0 conn(0x7f51800ccc00 0x7f5180069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.662+0000 7f517ed9d700 1 -- 172.21.15.55:0/1792632722 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800ca470 con 0x7f518006a020 2023-10-06T03:52:08.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.663+0000 7f517ed9d700 1 --2- 172.21.15.55:0/1792632722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f518006a020 0x7f51800cb230 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f518006cd90 tx=0x7f5178009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d843c8c13367a94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:08.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.663+0000 7f517d59a700 1 -- 172.21.15.55:0/1792632722 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517800e070 con 0x7f518006a020 2023-10-06T03:52:08.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.663+0000 7f517d59a700 1 -- 172.21.15.55:0/1792632722 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517800ba00 con 0x7f518006a020 2023-10-06T03:52:08.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.663+0000 7f517d59a700 1 -- 172.21.15.55:0/1792632722 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51780092e0 con 0x7f518006a020 2023-10-06T03:52:08.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.664+0000 7f517f59e700 1 -- 172.21.15.55:0/1792632722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f518006a020 msgr2=0x7f51800cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:08.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.664+0000 7f517f59e700 1 --2- 172.21.15.55:0/1792632722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f518006a020 0x7f51800cb230 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f518006cd90 tx=0x7f5178009840 comp rx=0 tx=0).stop 2023-10-06T03:52:08.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.664+0000 7f517f59e700 1 -- 172.21.15.55:0/1792632722 shutdown_connections 2023-10-06T03:52:08.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.664+0000 7f517f59e700 1 --2- 172.21.15.55:0/1792632722 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f518006a020 0x7f51800cb230 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.664+0000 7f517f59e700 1 --2- 172.21.15.55:0/1792632722 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5180159a60 0x7f51800c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.664+0000 7f517f59e700 1 --2- 172.21.15.55:0/1792632722 >> v2:172.21.15.163:3300/0 conn(0x7f51800ccc00 0x7f5180069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.664+0000 7f517f59e700 1 -- 172.21.15.55:0/1792632722 >> 172.21.15.55:0/1792632722 conn(0x7f518006baa0 msgr2=0x7f518006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:08.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.664+0000 7f517f59e700 1 -- 172.21.15.55:0/1792632722 shutdown_connections 2023-10-06T03:52:08.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.664+0000 7f517f59e700 1 -- 172.21.15.55:0/1792632722 wait complete. 2023-10-06T03:52:08.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.665+0000 7f517f59e700 1 Processor -- start 2023-10-06T03:52:08.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.665+0000 7f517f59e700 1 -- start start 2023-10-06T03:52:08.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.666+0000 7f517f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f518006a020 0x7f518014d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:08.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.666+0000 7f517f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51800ccc00 0x7f518015c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:08.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.666+0000 7f517f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5180159a60 0x7f518015c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:08.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.666+0000 7f517f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51800b3ae0 con 0x7f5180159a60 2023-10-06T03:52:08.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.666+0000 7f517f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51800b3dc0 con 0x7f51800ccc00 2023-10-06T03:52:08.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.666+0000 7f517f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51800b40a0 con 0x7f518006a020 2023-10-06T03:52:08.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.667+0000 7f517ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5180159a60 0x7f518015c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:08.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.667+0000 7f517ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5180159a60 0x7f518015c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38556/0 (socket says 172.21.15.55:38556) 2023-10-06T03:52:08.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.667+0000 7f517ed9d700 1 -- 172.21.15.55:0/4287704631 learned_addr learned my addr 172.21.15.55:0/4287704631 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:08.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.667+0000 7f517e59c700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f518006a020 0x7f518014d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:08.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.667+0000 7f517dd9b700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51800ccc00 0x7f518015c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:08.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.667+0000 7f517ed9d700 1 -- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f518006a020 msgr2=0x7f518014d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:08.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.667+0000 7f517ed9d700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f518006a020 0x7f518014d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.667+0000 7f517ed9d700 1 -- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51800ccc00 msgr2=0x7f518015c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:08.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.667+0000 7f517ed9d700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51800ccc00 0x7f518015c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.667+0000 7f517ed9d700 1 -- 172.21.15.55:0/4287704631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5178009ae0 con 0x7f5180159a60 2023-10-06T03:52:08.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.667+0000 7f517dd9b700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51800ccc00 0x7f518015c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:08.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.667+0000 7f517ed9d700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5180159a60 0x7f518015c610 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f517800c010 tx=0x7f5178018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:08.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.668+0000 7f516e7fc700 1 -- 172.21.15.55:0/4287704631 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178008b80 con 0x7f5180159a60 2023-10-06T03:52:08.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.668+0000 7f516e7fc700 1 -- 172.21.15.55:0/4287704631 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5178008d00 con 0x7f5180159a60 2023-10-06T03:52:08.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.668+0000 7f516e7fc700 1 -- 172.21.15.55:0/4287704631 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517801d670 con 0x7f5180159a60 2023-10-06T03:52:08.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.668+0000 7f517f59e700 1 -- 172.21.15.55:0/4287704631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518015d3b0 con 0x7f5180159a60 2023-10-06T03:52:08.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.668+0000 7f517f59e700 1 -- 172.21.15.55:0/4287704631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51800b0c70 con 0x7f5180159a60 2023-10-06T03:52:08.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.669+0000 7f517f59e700 1 -- 172.21.15.55:0/4287704631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518006eba0 con 0x7f5180159a60 2023-10-06T03:52:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.670+0000 7f516e7fc700 1 -- 172.21.15.55:0/4287704631 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 55) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f5178003040 con 0x7f5180159a60 2023-10-06T03:52:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.672+0000 7f516e7fc700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f51600671a0 0x7f5160069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.672+0000 7f516e7fc700 1 -- 172.21.15.55:0/4287704631 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f51780868b0 con 0x7f5180159a60 2023-10-06T03:52:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.672+0000 7f517e59c700 1 -- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f51600671a0 msgr2=0x7f5160069650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.672+0000 7f517e59c700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f51600671a0 0x7f5160069650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:52:08.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.675+0000 7f516e7fc700 1 -- 172.21.15.55:0/4287704631 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51780532f0 con 0x7f5180159a60 2023-10-06T03:52:08.940 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:08.779+0000 7f882846d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:52:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.873+0000 7f517e59c700 1 -- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f51600671a0 msgr2=0x7f5160069650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.873+0000 7f517e59c700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f51600671a0 0x7f5160069650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:52:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.879+0000 7f517f59e700 1 -- 172.21.15.55:0/4287704631 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51800ca750 con 0x7f5180159a60 2023-10-06T03:52:08.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.879+0000 7f516e7fc700 1 -- 172.21.15.55:0/4287704631 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5178009940 con 0x7f5180159a60 2023-10-06T03:52:08.942 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T03:52:08.942 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:08.879+0000 7f882846d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:52:08.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.882+0000 7f517f59e700 1 -- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f51600671a0 msgr2=0x7f5160069650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:52:08.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.882+0000 7f517f59e700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f51600671a0 0x7f5160069650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.882+0000 7f517f59e700 1 -- 172.21.15.55:0/4287704631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5180159a60 msgr2=0x7f518015c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:08.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.882+0000 7f517f59e700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5180159a60 0x7f518015c610 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f517800c010 tx=0x7f5178018000 comp rx=0 tx=0).stop 2023-10-06T03:52:08.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.882+0000 7f517f59e700 1 -- 172.21.15.55:0/4287704631 shutdown_connections 2023-10-06T03:52:08.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.882+0000 7f517f59e700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f51600671a0 0x7f5160069650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.882+0000 7f517f59e700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5180159a60 0x7f518015c610 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.882+0000 7f517f59e700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f518006a020 0x7f518014d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.882+0000 7f517f59e700 1 --2- 172.21.15.55:0/4287704631 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51800ccc00 0x7f518015c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:08.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.882+0000 7f517f59e700 1 -- 172.21.15.55:0/4287704631 >> 172.21.15.55:0/4287704631 conn(0x7f518006baa0 msgr2=0x7f51800cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:08.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.882+0000 7f517f59e700 1 -- 172.21.15.55:0/4287704631 shutdown_connections 2023-10-06T03:52:08.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:08.882+0000 7f517f59e700 1 -- 172.21.15.55:0/4287704631 wait complete. 2023-10-06T03:52:08.945 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T03:52:08.993 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:08.992+0000 7f1bf9efe180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:52:09.092 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:09.091+0000 7f1bf9efe180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:52:09.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.140+0000 7fcd80b9d700 1 Processor -- start 2023-10-06T03:52:09.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.140+0000 7fcd80b9d700 1 -- start start 2023-10-06T03:52:09.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.141+0000 7fcd80b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcd7c0cb1a0 0x7fcd7c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:09.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.141+0000 7fcd80b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd7c15a8a0 0x7fcd7c06f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:09.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.141+0000 7fcd80b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7c06ebd0 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.141+0000 7fcd80b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c15b2e0 con 0x7fcd7c15a8a0 2023-10-06T03:52:09.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.141+0000 7fcd80b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c15ad20 con 0x7fcd7c0cb1a0 2023-10-06T03:52:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.142+0000 7fcd7affd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcd7c15a8a0 0x7fcd7c06f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.142+0000 7fcd7bfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcd7c06f5a0 0x7fcd7c0ae0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49582/0 (socket says 172.21.15.55:49582) 2023-10-06T03:52:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.142+0000 7fcd7bfff700 1 -- 172.21.15.55:0/2201016347 learned_addr learned my addr 172.21.15.55:0/2201016347 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:09.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.142+0000 7fcd7b7fe700 1 --2- 172.21.15.55:0/2201016347 >> v2:172.21.15.163:3300/0 conn(0x7fcd7c0cb1a0 0x7fcd7c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:09.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.142+0000 7fcd7a7fc700 1 -- 172.21.15.55:0/2201016347 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1006564036 0 0) 0x7fcd7c06ebd0 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.142+0000 7fcd7affd700 1 -- 172.21.15.55:0/2201016347 >> v2:172.21.15.163:3300/0 conn(0x7fcd7c0cb1a0 msgr2=0x7fcd7c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.142+0000 7fcd7affd700 1 --2- 172.21.15.55:0/2201016347 >> v2:172.21.15.163:3300/0 conn(0x7fcd7c0cb1a0 0x7fcd7c15a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.142+0000 7fcd7affd700 1 -- 172.21.15.55:0/2201016347 >> v1:172.21.15.55:6789/0 conn(0x7fcd7c06f5a0 legacy=0x7fcd7c0ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.142+0000 7fcd7affd700 1 -- 172.21.15.55:0/2201016347 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c15b000 con 0x7fcd7c15a8a0 2023-10-06T03:52:09.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.143+0000 7fcd7affd700 1 --2- 172.21.15.55:0/2201016347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd7c15a8a0 0x7fcd7c06f060 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0cce10 tx=0x7fcd700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505c849b129afed5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:09.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.143+0000 7fcd7a7fc700 1 -- 172.21.15.55:0/2201016347 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7000b690 con 0x7fcd7c15a8a0 2023-10-06T03:52:09.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.143+0000 7fcd7a7fc700 1 -- 172.21.15.55:0/2201016347 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7000b7f0 con 0x7fcd7c15a8a0 2023-10-06T03:52:09.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.143+0000 7fcd7a7fc700 1 -- 172.21.15.55:0/2201016347 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7000bb90 con 0x7fcd7c15a8a0 2023-10-06T03:52:09.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.143+0000 7fcd80b9d700 1 -- 172.21.15.55:0/2201016347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd7c15a8a0 msgr2=0x7fcd7c06f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:09.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.143+0000 7fcd80b9d700 1 --2- 172.21.15.55:0/2201016347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd7c15a8a0 0x7fcd7c06f060 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0cce10 tx=0x7fcd700091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.144+0000 7fcd80b9d700 1 -- 172.21.15.55:0/2201016347 shutdown_connections 2023-10-06T03:52:09.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.144+0000 7fcd80b9d700 1 --2- 172.21.15.55:0/2201016347 >> v2:172.21.15.163:3300/0 conn(0x7fcd7c0cb1a0 0x7fcd7c15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.144+0000 7fcd80b9d700 1 --2- 172.21.15.55:0/2201016347 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd7c15a8a0 0x7fcd7c06f060 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.144+0000 7fcd80b9d700 1 -- 172.21.15.55:0/2201016347 >> 172.21.15.55:0/2201016347 conn(0x7fcd7c0cc1e0 msgr2=0x7fcd7c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:09.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.144+0000 7fcd80b9d700 1 -- 172.21.15.55:0/2201016347 shutdown_connections 2023-10-06T03:52:09.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.144+0000 7fcd80b9d700 1 -- 172.21.15.55:0/2201016347 wait complete. 2023-10-06T03:52:09.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.145+0000 7fcd80b9d700 1 Processor -- start 2023-10-06T03:52:09.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.145+0000 7fcd80b9d700 1 -- start start 2023-10-06T03:52:09.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.146+0000 7fcd80b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd7c06f5a0 0x7fcd7c152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:09.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.146+0000 7fcd80b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c0cb1a0 0x7fcd7c0c0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:09.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.146+0000 7fcd80b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c0c26c0 0x7fcd7c16a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:09.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.146+0000 7fcd80b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c153390 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.146+0000 7fcd80b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c0c1880 con 0x7fcd7c0cb1a0 2023-10-06T03:52:09.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.146+0000 7fcd80b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c0c1b60 con 0x7fcd7c0c26c0 2023-10-06T03:52:09.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.147+0000 7fcd7b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd7c06f5a0 0x7fcd7c152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:09.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.147+0000 7fcd7b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd7c06f5a0 0x7fcd7c152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38586/0 (socket says 172.21.15.55:38586) 2023-10-06T03:52:09.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.147+0000 7fcd7b7fe700 1 -- 172.21.15.55:0/1642082154 learned_addr learned my addr 172.21.15.55:0/1642082154 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:09.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.147+0000 7fcd7affd700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c0cb1a0 0x7fcd7c0c0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:09.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.147+0000 7fcd7bfff700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c0c26c0 0x7fcd7c16a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:09.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.147+0000 7fcd7b7fe700 1 -- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c0c26c0 msgr2=0x7fcd7c16a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:09.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.147+0000 7fcd7b7fe700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c0c26c0 0x7fcd7c16a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.147+0000 7fcd7b7fe700 1 -- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c0cb1a0 msgr2=0x7fcd7c0c0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:09.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.147+0000 7fcd7b7fe700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c0cb1a0 0x7fcd7c0c0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.147+0000 7fcd7b7fe700 1 -- 172.21.15.55:0/1642082154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd70009420 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.147+0000 7fcd7affd700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c0cb1a0 0x7fcd7c0c0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:52:09.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.147+0000 7fcd7b7fe700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd7c06f5a0 0x7fcd7c152e50 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00ec80 tx=0x7fcd6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:09.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.148+0000 7fcd637fe700 1 -- 172.21.15.55:0/1642082154 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00c850 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.148+0000 7fcd637fe700 1 -- 172.21.15.55:0/1642082154 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c00c9b0 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.148+0000 7fcd637fe700 1 -- 172.21.15.55:0/1642082154 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00cd50 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.148+0000 7fcd80b9d700 1 -- 172.21.15.55:0/1642082154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c0c1e40 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.148+0000 7fcd80b9d700 1 -- 172.21.15.55:0/1642082154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c16a140 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.149+0000 7fcd80b9d700 1 -- 172.21.15.55:0/1642082154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7c06c450 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.152+0000 7fcd637fe700 1 -- 172.21.15.55:0/1642082154 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 55) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fcd6c013680 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.152+0000 7fcd637fe700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd68067170 0x7fcd68069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:09.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.153+0000 7fcd637fe700 1 -- 172.21.15.55:0/1642082154 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6708+0+0 (secure 0 0 0) 0x7fcd6c084f80 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.153+0000 7fcd7affd700 1 -- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd68067170 msgr2=0x7fcd68069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:09.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.153+0000 7fcd7affd700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd68067170 0x7fcd68069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:52:09.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.157+0000 7fcd637fe700 1 -- 172.21.15.55:0/1642082154 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd6c051ad0 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.229 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:09.192+0000 7f882846d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:52:09.294 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:09.292+0000 7f882846d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:52:09.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.354+0000 7fcd7affd700 1 -- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd68067170 msgr2=0x7fcd68069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:09.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.354+0000 7fcd7affd700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd68067170 0x7fcd68069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:52:09.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.360+0000 7fcd80b9d700 1 -- 172.21.15.55:0/1642082154 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd7c06e860 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.360+0000 7fcd637fe700 1 -- 172.21.15.55:0/1642082154 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd6c0170a0 con 0x7fcd7c06f5a0 2023-10-06T03:52:09.375 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T03:52:09.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.362+0000 7fcd80b9d700 1 -- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd68067170 msgr2=0x7fcd68069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:52:09.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.362+0000 7fcd80b9d700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd68067170 0x7fcd68069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.362+0000 7fcd80b9d700 1 -- 172.21.15.55:0/1642082154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd7c06f5a0 msgr2=0x7fcd7c152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:09.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.362+0000 7fcd80b9d700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd7c06f5a0 0x7fcd7c152e50 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00ec80 tx=0x7fcd6c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.362+0000 7fcd80b9d700 1 -- 172.21.15.55:0/1642082154 shutdown_connections 2023-10-06T03:52:09.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.362+0000 7fcd80b9d700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcd68067170 0x7fcd68069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.362+0000 7fcd80b9d700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd7c0c26c0 0x7fcd7c16a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.362+0000 7fcd80b9d700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd7c06f5a0 0x7fcd7c152e50 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.362+0000 7fcd80b9d700 1 --2- 172.21.15.55:0/1642082154 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd7c0cb1a0 0x7fcd7c0c0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.362+0000 7fcd80b9d700 1 -- 172.21.15.55:0/1642082154 >> 172.21.15.55:0/1642082154 conn(0x7fcd7c0cc1e0 msgr2=0x7fcd7c155e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:09.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.363+0000 7fcd80b9d700 1 -- 172.21.15.55:0/1642082154 shutdown_connections 2023-10-06T03:52:09.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.363+0000 7fcd80b9d700 1 -- 172.21.15.55:0/1642082154 wait complete. 2023-10-06T03:52:09.382 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:52:09.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.569+0000 7f73c999d700 1 Processor -- start 2023-10-06T03:52:09.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.570+0000 7f73c999d700 1 -- start start 2023-10-06T03:52:09.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.571+0000 7f73c999d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f73c4142ea0 0x7f73c40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:09.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.571+0000 7f73c999d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f73c406c0f0 0x7f73c406a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:09.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.571+0000 7f73c999d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c40b40f0 con 0x7f73c40ad870 2023-10-06T03:52:09.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.571+0000 7f73c999d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c4069640 con 0x7f73c406c0f0 2023-10-06T03:52:09.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.571+0000 7f73c999d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c4069920 con 0x7f73c4142ea0 2023-10-06T03:52:09.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.571+0000 7f73c919c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f73c406c0f0 0x7f73c406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:09.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.571+0000 7f73c3fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f73c40ad870 0x7f73c406b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49590/0 (socket says 172.21.15.55:49590) 2023-10-06T03:52:09.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.571+0000 7f73c3fff700 1 -- 172.21.15.55:0/3597504762 learned_addr learned my addr 172.21.15.55:0/3597504762 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:09.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.571+0000 7f73c899b700 1 --2- 172.21.15.55:0/3597504762 >> v2:172.21.15.163:3300/0 conn(0x7f73c4142ea0 0x7f73c40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:09.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.572+0000 7f73c919c700 1 -- 172.21.15.55:0/3597504762 >> v2:172.21.15.163:3300/0 conn(0x7f73c4142ea0 msgr2=0x7f73c40b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:09.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.572+0000 7f73c919c700 1 --2- 172.21.15.55:0/3597504762 >> v2:172.21.15.163:3300/0 conn(0x7f73c4142ea0 0x7f73c40b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.572+0000 7f73c919c700 1 -- 172.21.15.55:0/3597504762 >> v1:172.21.15.55:6789/0 conn(0x7f73c40ad870 legacy=0x7f73c406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:09.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.572+0000 7f73c919c700 1 -- 172.21.15.55:0/3597504762 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c4069c00 con 0x7f73c406c0f0 2023-10-06T03:52:09.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.572+0000 7f73c919c700 1 --2- 172.21.15.55:0/3597504762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73c406c0f0 0x7f73c406a120 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f73c4071240 tx=0x7f73bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd735272c69f8b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:09.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.572+0000 7f73c37fe700 1 -- 172.21.15.55:0/3597504762 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc00e070 con 0x7f73c406c0f0 2023-10-06T03:52:09.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.572+0000 7f73c37fe700 1 -- 172.21.15.55:0/3597504762 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73bc00bb60 con 0x7f73c406c0f0 2023-10-06T03:52:09.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.572+0000 7f73c37fe700 1 -- 172.21.15.55:0/3597504762 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc012430 con 0x7f73c406c0f0 2023-10-06T03:52:09.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.573+0000 7f73c999d700 1 -- 172.21.15.55:0/3597504762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73c406c0f0 msgr2=0x7f73c406a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:09.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.573+0000 7f73c999d700 1 --2- 172.21.15.55:0/3597504762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73c406c0f0 0x7f73c406a120 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f73c4071240 tx=0x7f73bc009780 comp rx=0 tx=0).stop 2023-10-06T03:52:09.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.573+0000 7f73c999d700 1 -- 172.21.15.55:0/3597504762 shutdown_connections 2023-10-06T03:52:09.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.573+0000 7f73c999d700 1 --2- 172.21.15.55:0/3597504762 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73c406c0f0 0x7f73c406a120 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.573+0000 7f73c999d700 1 --2- 172.21.15.55:0/3597504762 >> v2:172.21.15.163:3300/0 conn(0x7f73c4142ea0 0x7f73c40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.573+0000 7f73c999d700 1 -- 172.21.15.55:0/3597504762 >> 172.21.15.55:0/3597504762 conn(0x7f73c40ce430 msgr2=0x7f73c40ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:09.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.573+0000 7f73c999d700 1 -- 172.21.15.55:0/3597504762 shutdown_connections 2023-10-06T03:52:09.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.573+0000 7f73c999d700 1 -- 172.21.15.55:0/3597504762 wait complete. 2023-10-06T03:52:09.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.575+0000 7f73c999d700 1 Processor -- start 2023-10-06T03:52:09.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.575+0000 7f73c999d700 1 -- start start 2023-10-06T03:52:09.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c999d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73c406c0f0 0x7f73c4159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:09.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c999d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73c4142ea0 0x7f73c4159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:09.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c999d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73c415c6e0 0x7f73c415cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:09.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c999d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c415d0a0 con 0x7f73c406c0f0 2023-10-06T03:52:09.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c999d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c4140210 con 0x7f73c415c6e0 2023-10-06T03:52:09.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c999d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c416d210 con 0x7f73c4142ea0 2023-10-06T03:52:09.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c899b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73c406c0f0 0x7f73c4159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:09.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c899b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73c406c0f0 0x7f73c4159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38604/0 (socket says 172.21.15.55:38604) 2023-10-06T03:52:09.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c899b700 1 -- 172.21.15.55:0/3269243982 learned_addr learned my addr 172.21.15.55:0/3269243982 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:09.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c919c700 1 --2- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73c415c6e0 0x7f73c415cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:09.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c3fff700 1 --2- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73c4142ea0 0x7f73c4159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:09.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c899b700 1 -- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73c4142ea0 msgr2=0x7f73c4159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:09.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c899b700 1 --2- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73c4142ea0 0x7f73c4159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c899b700 1 -- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73c415c6e0 msgr2=0x7f73c415cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:09.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c899b700 1 --2- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73c415c6e0 0x7f73c415cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:09.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.576+0000 7f73c899b700 1 -- 172.21.15.55:0/3269243982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc009a20 con 0x7f73c406c0f0 2023-10-06T03:52:09.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.577+0000 7f73c899b700 1 --2- 172.21.15.55:0/3269243982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73c406c0f0 0x7f73c4159070 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f73b800e930 tx=0x7f73b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:09.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.577+0000 7f73c0ff9700 1 -- 172.21.15.55:0/3269243982 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8013680 con 0x7f73c406c0f0 2023-10-06T03:52:09.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.577+0000 7f73c0ff9700 1 -- 172.21.15.55:0/3269243982 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b80137e0 con 0x7f73c406c0f0 2023-10-06T03:52:09.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.577+0000 7f73c999d700 1 -- 172.21.15.55:0/3269243982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c416d4f0 con 0x7f73c406c0f0 2023-10-06T03:52:09.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.577+0000 7f73c0ff9700 1 -- 172.21.15.55:0/3269243982 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8013b80 con 0x7f73c406c0f0 2023-10-06T03:52:09.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.577+0000 7f73c999d700 1 -- 172.21.15.55:0/3269243982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c416dab0 con 0x7f73c406c0f0 2023-10-06T03:52:09.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.578+0000 7f73c999d700 1 -- 172.21.15.55:0/3269243982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c406c9c0 con 0x7f73c406c0f0 2023-10-06T03:52:09.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.579+0000 7f73c0ff9700 1 -- 172.21.15.55:0/3269243982 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 55) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f73b800c3e0 con 0x7f73c406c0f0 2023-10-06T03:52:09.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.580+0000 7f73c0ff9700 1 --2- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f73a8067170 0x7f73a8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:09.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.580+0000 7f73c0ff9700 1 -- 172.21.15.55:0/3269243982 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f73b8084ea0 con 0x7f73c406c0f0 2023-10-06T03:52:09.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.580+0000 7f73c3fff700 1 -- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f73a8067170 msgr2=0x7f73a8069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:09.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.580+0000 7f73c3fff700 1 --2- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f73a8067170 0x7f73a8069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:52:09.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.588+0000 7f73c0ff9700 1 -- 172.21.15.55:0/3269243982 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73b80519f0 con 0x7f73c406c0f0 2023-10-06T03:52:09.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.781+0000 7f73c3fff700 1 -- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f73a8067170 msgr2=0x7f73a8069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:09.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.781+0000 7f73c3fff700 1 --2- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f73a8067170 0x7f73a8069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:52:09.834 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:09.791+0000 7f1bf9efe180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:52:09.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.839+0000 7f73c999d700 1 -- 172.21.15.55:0/3269243982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73c404c3c0 con 0x7f73c406c0f0 2023-10-06T03:52:09.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.840+0000 7f73c0ff9700 1 -- 172.21.15.55:0/3269243982 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f73b8021090 con 0x7f73c406c0f0 2023-10-06T03:52:09.850 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:09.851 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1295635544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":1295635544},{"type":"v1","addr":"172.21.15.55:6835","nonce":1295635544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7737,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2115790576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2115790576},{"type":"v1","addr":"172.21.15.163:6834","nonce":2115790576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1021807391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1021807391},{"type":"v1","addr":"172.21.15.163:6836","nonce":1021807391}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:51:26.480340+0000","modified":"2023-10-06T03:51:30.017376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7743":{"gid":7743,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/4240717455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":4240717455},{"type":"v1","addr":"172.21.15.55:6834","nonce":4240717455}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T03:52:09.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.842+0000 7f73c999d700 1 -- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f73a8067170 msgr2=0x7f73a8069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:52:10.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.842+0000 7f73c999d700 1 --2- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f73a8067170 0x7f73a8069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:10.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.842+0000 7f73c999d700 1 -- 172.21.15.55:0/3269243982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73c406c0f0 msgr2=0x7f73c4159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:10.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.842+0000 7f73c999d700 1 --2- 172.21.15.55:0/3269243982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73c406c0f0 0x7f73c4159070 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f73b800e930 tx=0x7f73b80091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.842+0000 7f73c999d700 1 -- 172.21.15.55:0/3269243982 shutdown_connections 2023-10-06T03:52:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.842+0000 7f73c999d700 1 --2- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f73a8067170 0x7f73a8069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.842+0000 7f73c999d700 1 --2- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f73c415c6e0 0x7f73c415cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.842+0000 7f73c999d700 1 --2- 172.21.15.55:0/3269243982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f73c406c0f0 0x7f73c4159070 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.842+0000 7f73c999d700 1 --2- 172.21.15.55:0/3269243982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f73c4142ea0 0x7f73c4159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.842+0000 7f73c999d700 1 -- 172.21.15.55:0/3269243982 >> 172.21.15.55:0/3269243982 conn(0x7f73c40ce430 msgr2=0x7f73c4141ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.843+0000 7f73c999d700 1 -- 172.21.15.55:0/3269243982 shutdown_connections 2023-10-06T03:52:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:09.843+0000 7f73c999d700 1 -- 172.21.15.55:0/3269243982 wait complete. 2023-10-06T03:52:10.060 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 61 2023-10-06T03:52:10.062 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:52:10.062 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:10.040+0000 7f882846d180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:52:10.080 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.079+0000 7f1bf9efe180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:52:10.173 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.172+0000 7f1bf9efe180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:52:11.169 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.327+0000 7f1bf9efe180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:52:11.176 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.431+0000 7f1bf9efe180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:52:11.176 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.621+0000 7f1bf9efe180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:52:11.176 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.715+0000 7f1bf9efe180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:52:11.185 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.808+0000 7f1bf9efe180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:52:11.185 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.901+0000 7f1bf9efe180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:52:11.185 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.995+0000 7f1b74281700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:11.185 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.995+0000 7f1b74281700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:11.185 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.995+0000 7f1b74281700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:11.185 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.995+0000 7f1b74281700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:11.185 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.995+0000 7f1b74281700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:11.185 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.996+0000 7f1b77287700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:11.185 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.996+0000 7f1b77287700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:11.186 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.996+0000 7f1b77287700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:11.186 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.996+0000 7f1b77287700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:11.186 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:10.996+0000 7f1b77287700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:11.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.256+0000 7fd00519d700 1 Processor -- start 2023-10-06T03:52:11.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.256+0000 7fd00519d700 1 -- start start 2023-10-06T03:52:11.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.257+0000 7fd00519d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd000150180 0x7fd000157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:11.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.257+0000 7fd00519d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd000150550 0x7fd0000c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:11.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.257+0000 7fd00519d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00006e2a0 con 0x7fd0000c7240 2023-10-06T03:52:11.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.258+0000 7fd00519d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00006b2e0 con 0x7fd000150550 2023-10-06T03:52:11.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.258+0000 7fd00519d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0000c7680 con 0x7fd000150180 2023-10-06T03:52:11.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.258+0000 7fcfff7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd000150550 0x7fd0000c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:11.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.258+0000 7fcfff7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd000150550 0x7fd0000c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38620/0 (socket says 172.21.15.55:38620) 2023-10-06T03:52:11.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.258+0000 7fcfff7fe700 1 -- 172.21.15.55:0/3978819721 learned_addr learned my addr 172.21.15.55:0/3978819721 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:11.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.258+0000 7fcffffff700 1 --2- 172.21.15.55:0/3978819721 >> v2:172.21.15.163:3300/0 conn(0x7fd000150180 0x7fd000157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:11.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.258+0000 7fcfff7fe700 1 -- 172.21.15.55:0/3978819721 >> v2:172.21.15.163:3300/0 conn(0x7fd000150180 msgr2=0x7fd000157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:11.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.258+0000 7fcfff7fe700 1 --2- 172.21.15.55:0/3978819721 >> v2:172.21.15.163:3300/0 conn(0x7fd000150180 0x7fd000157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.258+0000 7fcfff7fe700 1 -- 172.21.15.55:0/3978819721 >> v1:172.21.15.163:6789/0 conn(0x7fd0000c7240 legacy=0x7fd000069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:11.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.258+0000 7fcfff7fe700 1 -- 172.21.15.55:0/3978819721 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0000c78e0 con 0x7fd000150550 2023-10-06T03:52:11.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.258+0000 7fcffeffd700 1 -- 172.21.15.55:0/3978819721 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213754141 0 0) 0x7fd00006e2a0 con 0x7fd0000c7240 2023-10-06T03:52:11.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.259+0000 7fcfff7fe700 1 --2- 172.21.15.55:0/3978819721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd000150550 0x7fd0000c6d00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fd00006fd20 tx=0x7fcff400b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a5f20bf4d6537 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:11.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.259+0000 7fcffeffd700 1 -- 172.21.15.55:0/3978819721 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff400da60 con 0x7fd000150550 2023-10-06T03:52:11.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.259+0000 7fcffeffd700 1 -- 172.21.15.55:0/3978819721 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff400dbc0 con 0x7fd000150550 2023-10-06T03:52:11.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.259+0000 7fcffeffd700 1 -- 172.21.15.55:0/3978819721 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff4017420 con 0x7fd000150550 2023-10-06T03:52:11.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.259+0000 7fd00519d700 1 -- 172.21.15.55:0/3978819721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd000150550 msgr2=0x7fd0000c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:11.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.259+0000 7fd00519d700 1 --2- 172.21.15.55:0/3978819721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd000150550 0x7fd0000c6d00 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fd00006fd20 tx=0x7fcff400b4e0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.260+0000 7fd00519d700 1 -- 172.21.15.55:0/3978819721 shutdown_connections 2023-10-06T03:52:11.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.260+0000 7fd00519d700 1 --2- 172.21.15.55:0/3978819721 >> v2:172.21.15.163:3300/0 conn(0x7fd000150180 0x7fd000157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.260+0000 7fd00519d700 1 --2- 172.21.15.55:0/3978819721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd000150550 0x7fd0000c6d00 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.260+0000 7fd00519d700 1 -- 172.21.15.55:0/3978819721 >> 172.21.15.55:0/3978819721 conn(0x7fd0000cc1f0 msgr2=0x7fd0000cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:11.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.260+0000 7fd00519d700 1 -- 172.21.15.55:0/3978819721 shutdown_connections 2023-10-06T03:52:11.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.260+0000 7fd00519d700 1 -- 172.21.15.55:0/3978819721 wait complete. 2023-10-06T03:52:11.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.261+0000 7fd00519d700 1 Processor -- start 2023-10-06T03:52:11.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.261+0000 7fd00519d700 1 -- start start 2023-10-06T03:52:11.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.262+0000 7fd00519d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0000c7240 0x7fd000148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:11.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.262+0000 7fd00519d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000150180 0x7fd000154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:11.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.262+0000 7fd00519d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd000150550 0x7fd000155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:11.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.262+0000 7fd00519d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00016a7c0 con 0x7fd0000c7240 2023-10-06T03:52:11.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.262+0000 7fd00519d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd000169bb0 con 0x7fd000150180 2023-10-06T03:52:11.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.262+0000 7fd00519d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd000169e90 con 0x7fd000150550 2023-10-06T03:52:11.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.262+0000 7fcffffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0000c7240 0x7fd000148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:11.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.263+0000 7fcffffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0000c7240 0x7fd000148c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38626/0 (socket says 172.21.15.55:38626) 2023-10-06T03:52:11.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.263+0000 7fcffffff700 1 -- 172.21.15.55:0/2571213609 learned_addr learned my addr 172.21.15.55:0/2571213609 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:11.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.263+0000 7fcfff7fe700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000150180 0x7fd000154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:11.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.263+0000 7fd00499c700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd000150550 0x7fd000155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:11.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.263+0000 7fcffffff700 1 -- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd000150550 msgr2=0x7fd000155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:11.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.263+0000 7fcffffff700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd000150550 0x7fd000155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.263+0000 7fcffffff700 1 -- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000150180 msgr2=0x7fd000154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:11.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.263+0000 7fcffffff700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000150180 0x7fd000154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.263+0000 7fcffffff700 1 -- 172.21.15.55:0/2571213609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff400b780 con 0x7fd0000c7240 2023-10-06T03:52:11.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.263+0000 7fcfff7fe700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000150180 0x7fd000154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:52:11.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.263+0000 7fd00499c700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd000150550 0x7fd000155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:11.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.263+0000 7fcffffff700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0000c7240 0x7fd000148c80 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fcff000ec80 tx=0x7fcff00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:11.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.263+0000 7fcfe7fff700 1 -- 172.21.15.55:0/2571213609 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff000c850 con 0x7fd0000c7240 2023-10-06T03:52:11.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.264+0000 7fcfe7fff700 1 -- 172.21.15.55:0/2571213609 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff000c9b0 con 0x7fd0000c7240 2023-10-06T03:52:11.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.264+0000 7fcfe7fff700 1 -- 172.21.15.55:0/2571213609 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff000cd50 con 0x7fd0000c7240 2023-10-06T03:52:11.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.264+0000 7fd00519d700 1 -- 172.21.15.55:0/2571213609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00016a170 con 0x7fd0000c7240 2023-10-06T03:52:11.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.264+0000 7fd00519d700 1 -- 172.21.15.55:0/2571213609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0001491c0 con 0x7fd0000c7240 2023-10-06T03:52:11.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.265+0000 7fd00519d700 1 -- 172.21.15.55:0/2571213609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00004c3c0 con 0x7fd0000c7240 2023-10-06T03:52:11.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.266+0000 7fcfe7fff700 1 -- 172.21.15.55:0/2571213609 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 55) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fcff0013680 con 0x7fd0000c7240 2023-10-06T03:52:11.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.267+0000 7fcfe7fff700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcfec067170 0x7fcfec069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:11.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.267+0000 7fcfe7fff700 1 -- 172.21.15.55:0/2571213609 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6708+0+0 (secure 0 0 0) 0x7fcff0084f40 con 0x7fd0000c7240 2023-10-06T03:52:11.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.267+0000 7fcfff7fe700 1 -- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcfec067170 msgr2=0x7fcfec069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:11.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.267+0000 7fcfff7fe700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcfec067170 0x7fcfec069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:52:11.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.271+0000 7fcfe7fff700 1 -- 172.21.15.55:0/2571213609 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcff0051b30 con 0x7fd0000c7240 2023-10-06T03:52:11.202 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:10.340+0000 7f882846d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:52:11.646 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:10.436+0000 7f882846d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:52:11.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.468+0000 7fcfff7fe700 1 -- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcfec067170 msgr2=0x7fcfec069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:11.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.468+0000 7fcfff7fe700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcfec067170 0x7fcfec069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:52:11.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.476+0000 7fd00519d700 1 -- 172.21.15.55:0/2571213609 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd00006ada0 con 0x7fd0000c7240 2023-10-06T03:52:11.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.477+0000 7fcfe7fff700 1 -- 172.21.15.55:0/2571213609 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+17895 (secure 0 0 0) 0x7fcff0055150 con 0x7fd0000c7240 2023-10-06T03:52:11.651 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:11.651 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":124,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:51:43.863704+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:51:24.466543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T03:51:25.462282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/85191":"2023-10-07T03:51:43.863644+0000","172.21.15.163:6835/85191":"2023-10-07T03:51:43.863644+0000","172.21.15.163:0/2003915970":"2023-10-07T03:51:43.863644+0000","172.21.15.163:0/357062062":"2023-10-07T03:51:43.863644+0000","172.21.15.163:0/2409430869":"2023-10-07T03:51:43.863644+0000","172.21.15.163:0/3972526625":"2023-10-07T03:51:43.863644+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:52:11.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.480+0000 7fd00519d700 1 -- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcfec067170 msgr2=0x7fcfec069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:52:11.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.480+0000 7fd00519d700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcfec067170 0x7fcfec069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.480+0000 7fd00519d700 1 -- 172.21.15.55:0/2571213609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0000c7240 msgr2=0x7fd000148c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:11.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.480+0000 7fd00519d700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0000c7240 0x7fd000148c80 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fcff000ec80 tx=0x7fcff00091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.480+0000 7fd00519d700 1 -- 172.21.15.55:0/2571213609 shutdown_connections 2023-10-06T03:52:11.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.480+0000 7fd00519d700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcfec067170 0x7fcfec069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.480+0000 7fd00519d700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd000150550 0x7fd000155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.480+0000 7fd00519d700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd0000c7240 0x7fd000148c80 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.480+0000 7fd00519d700 1 --2- 172.21.15.55:0/2571213609 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd000150180 0x7fd000154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.480+0000 7fd00519d700 1 -- 172.21.15.55:0/2571213609 >> 172.21.15.55:0/2571213609 conn(0x7fd0000cc1f0 msgr2=0x7fd0000cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:11.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.480+0000 7fd00519d700 1 -- 172.21.15.55:0/2571213609 shutdown_connections 2023-10-06T03:52:11.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:10.480+0000 7fd00519d700 1 -- 172.21.15.55:0/2571213609 wait complete. 2023-10-06T03:52:11.654 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T03:52:11.654 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:52:11.655 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:10.599+0000 7f882846d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:52:11.656 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:10.707+0000 7f882846d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:52:11.656 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:10.903+0000 7f882846d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:52:11.656 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:11.014+0000 7f882846d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:52:11.656 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:11.113+0000 7f882846d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:52:11.767 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:11.210+0000 7f882846d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:52:11.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.961+0000 7f006c99d700 1 Processor -- start 2023-10-06T03:52:11.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.961+0000 7f006c99d700 1 -- start start 2023-10-06T03:52:11.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.962+0000 7f006c99d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f00680ae710 0x7f00680c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:11.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.962+0000 7f006c99d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00680ca860 con 0x7f0068156310 2023-10-06T03:52:11.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.962+0000 7f006c99d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00680cab40 con 0x7f00681566e0 2023-10-06T03:52:11.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.962+0000 7f006c99d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00680cae20 con 0x7f00680ae710 2023-10-06T03:52:11.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.963+0000 7f00677fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0068156310 0x7f00680c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49594/0 (socket says 172.21.15.55:49594) 2023-10-06T03:52:11.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.963+0000 7f00677fe700 1 -- 172.21.15.55:0/1657478883 learned_addr learned my addr 172.21.15.55:0/1657478883 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:11.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.963+0000 7f0067fff700 1 --2- 172.21.15.55:0/1657478883 >> v2:172.21.15.163:3300/0 conn(0x7f00680ae710 0x7f00680c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:11.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.963+0000 7f00667fc700 1 -- 172.21.15.55:0/1657478883 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898393143 0 0) 0x7f00680ca860 con 0x7f0068156310 2023-10-06T03:52:11.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.963+0000 7f00667fc700 1 -- 172.21.15.55:0/1657478883 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0050003570 con 0x7f0068156310 2023-10-06T03:52:11.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.963+0000 7f00667fc700 1 -- 172.21.15.55:0/1657478883 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1452171758 0 0) 0x7f00680cab40 con 0x7f00681566e0 2023-10-06T03:52:11.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.963+0000 7f00667fc700 1 -- 172.21.15.55:0/1657478883 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00680ca860 con 0x7f00681566e0 2023-10-06T03:52:11.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.963+0000 7f0067fff700 1 -- 172.21.15.55:0/1657478883 >> v1:172.21.15.163:6789/0 conn(0x7f00681566e0 legacy=0x7f00680ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:11.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.963+0000 7f0067fff700 1 -- 172.21.15.55:0/1657478883 >> v1:172.21.15.55:6789/0 conn(0x7f0068156310 legacy=0x7f00680c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:11.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.963+0000 7f0067fff700 1 -- 172.21.15.55:0/1657478883 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00680cb100 con 0x7f00680ae710 2023-10-06T03:52:11.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.964+0000 7f0067fff700 1 --2- 172.21.15.55:0/1657478883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00680ae710 0x7f00680c6830 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0060009e60 tx=0x7f0060009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=657e238062a27359 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:11.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.964+0000 7f00667fc700 1 -- 172.21.15.55:0/1657478883 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006000e070 con 0x7f00680ae710 2023-10-06T03:52:11.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.964+0000 7f00667fc700 1 -- 172.21.15.55:0/1657478883 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006000bab0 con 0x7f00680ae710 2023-10-06T03:52:11.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.964+0000 7f00667fc700 1 -- 172.21.15.55:0/1657478883 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006000be70 con 0x7f00680ae710 2023-10-06T03:52:11.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.964+0000 7f006c99d700 1 -- 172.21.15.55:0/1657478883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00680ae710 msgr2=0x7f00680c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:11.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.964+0000 7f006c99d700 1 --2- 172.21.15.55:0/1657478883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00680ae710 0x7f00680c6830 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0060009e60 tx=0x7f0060009780 comp rx=0 tx=0).stop 2023-10-06T03:52:11.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.965+0000 7f006c99d700 1 -- 172.21.15.55:0/1657478883 shutdown_connections 2023-10-06T03:52:11.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.965+0000 7f006c99d700 1 --2- 172.21.15.55:0/1657478883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00680ae710 0x7f00680c6830 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.965+0000 7f006c99d700 1 -- 172.21.15.55:0/1657478883 >> 172.21.15.55:0/1657478883 conn(0x7f00680cc1f0 msgr2=0x7f00680cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:11.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.965+0000 7f006c99d700 1 -- 172.21.15.55:0/1657478883 shutdown_connections 2023-10-06T03:52:11.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.965+0000 7f006c99d700 1 -- 172.21.15.55:0/1657478883 wait complete. 2023-10-06T03:52:11.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.966+0000 7f006c99d700 1 Processor -- start 2023-10-06T03:52:11.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.966+0000 7f006c99d700 1 -- start start 2023-10-06T03:52:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.967+0000 7f006c99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00680ae710 0x7f00680af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.967+0000 7f006c99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0068156310 0x7f00680b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.967+0000 7f006c99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00681566e0 0x7f006813fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.967+0000 7f006c99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00680b1b90 con 0x7f0068156310 2023-10-06T03:52:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.967+0000 7f006c99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0068140290 con 0x7f00681566e0 2023-10-06T03:52:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.967+0000 7f006c99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00680b1da0 con 0x7f00680ae710 2023-10-06T03:52:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.967+0000 7f0066ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0068156310 0x7f00680b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.967+0000 7f0066ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0068156310 0x7f00680b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38632/0 (socket says 172.21.15.55:38632) 2023-10-06T03:52:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.967+0000 7f0066ffd700 1 -- 172.21.15.55:0/4004252912 learned_addr learned my addr 172.21.15.55:0/4004252912 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.967+0000 7f0067fff700 1 --2- 172.21.15.55:0/4004252912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00681566e0 0x7f006813fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:11.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.967+0000 7f00677fe700 1 --2- 172.21.15.55:0/4004252912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00680ae710 0x7f00680af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.968+0000 7f0066ffd700 1 -- 172.21.15.55:0/4004252912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00680ae710 msgr2=0x7f00680af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.968+0000 7f0066ffd700 1 --2- 172.21.15.55:0/4004252912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00680ae710 0x7f00680af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.968+0000 7f0066ffd700 1 -- 172.21.15.55:0/4004252912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00681566e0 msgr2=0x7f006813fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.968+0000 7f0066ffd700 1 --2- 172.21.15.55:0/4004252912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00681566e0 0x7f006813fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.968+0000 7f0066ffd700 1 -- 172.21.15.55:0/4004252912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0060009a20 con 0x7f0068156310 2023-10-06T03:52:11.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.968+0000 7f0066ffd700 1 --2- 172.21.15.55:0/4004252912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0068156310 0x7f00680b1650 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f005c00c7c0 tx=0x7f005c007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.968+0000 7f004f7fe700 1 -- 172.21.15.55:0/4004252912 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c010510 con 0x7f0068156310 2023-10-06T03:52:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.968+0000 7f004f7fe700 1 -- 172.21.15.55:0/4004252912 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c0106b0 con 0x7f0068156310 2023-10-06T03:52:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.968+0000 7f004f7fe700 1 -- 172.21.15.55:0/4004252912 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c010a70 con 0x7f0068156310 2023-10-06T03:52:11.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.968+0000 7f006c99d700 1 -- 172.21.15.55:0/4004252912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00680c76b0 con 0x7f0068156310 2023-10-06T03:52:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.969+0000 7f006c99d700 1 -- 172.21.15.55:0/4004252912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00680c2b80 con 0x7f0068156310 2023-10-06T03:52:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.969+0000 7f006c99d700 1 -- 172.21.15.55:0/4004252912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006806ce30 con 0x7f0068156310 2023-10-06T03:52:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.970+0000 7f004f7fe700 1 -- 172.21.15.55:0/4004252912 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 57) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f005c012440 con 0x7f0068156310 2023-10-06T03:52:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.971+0000 7f004f7fe700 1 --2- 172.21.15.55:0/4004252912 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0054066d80 0x7f0054069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:11.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.971+0000 7f004f7fe700 1 -- 172.21.15.55:0/4004252912 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7108+0+0 (secure 0 0 0) 0x7f005c083db0 con 0x7f0068156310 2023-10-06T03:52:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.972+0000 7f00677fe700 1 --2- 172.21.15.55:0/4004252912 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0054066d80 0x7f0054069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:11.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.973+0000 7f00677fe700 1 --2- 172.21.15.55:0/4004252912 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0054066d80 0x7f0054069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0058007fd0 tx=0x7f0058009040 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:11.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:11.976+0000 7f004f7fe700 1 -- 172.21.15.55:0/4004252912 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f005c04b5a0 con 0x7f0068156310 2023-10-06T03:52:12.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.223+0000 7f006c99d700 1 -- 172.21.15.55:0/4004252912 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0068155a80 con 0x7f0068156310 2023-10-06T03:52:12.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.224+0000 7f004f7fe700 1 -- 172.21.15.55:0/4004252912 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f005c017070 con 0x7f0068156310 2023-10-06T03:52:12.239 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:12.239 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1295635544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":1295635544},{"type":"v1","addr":"172.21.15.55:6835","nonce":1295635544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7737,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2115790576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2115790576},{"type":"v1","addr":"172.21.15.163:6834","nonce":2115790576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1021807391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1021807391},{"type":"v1","addr":"172.21.15.163:6836","nonce":1021807391}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:51:26.480340+0000","modified":"2023-10-06T03:51:30.017376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7743":{"gid":7743,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/4240717455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":4240717455},{"type":"v1","addr":"172.21.15.55:6834","nonce":4240717455}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T03:52:12.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.226+0000 7f006c99d700 1 -- 172.21.15.55:0/4004252912 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0054066d80 msgr2=0x7f0054069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:12.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.226+0000 7f006c99d700 1 --2- 172.21.15.55:0/4004252912 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0054066d80 0x7f0054069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0058007fd0 tx=0x7f0058009040 comp rx=0 tx=0).stop 2023-10-06T03:52:12.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.226+0000 7f006c99d700 1 -- 172.21.15.55:0/4004252912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0068156310 msgr2=0x7f00680b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:12.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.226+0000 7f006c99d700 1 --2- 172.21.15.55:0/4004252912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0068156310 0x7f00680b1650 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f005c00c7c0 tx=0x7f005c007de0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.226+0000 7f006c99d700 1 -- 172.21.15.55:0/4004252912 shutdown_connections 2023-10-06T03:52:12.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.226+0000 7f006c99d700 1 --2- 172.21.15.55:0/4004252912 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0054066d80 0x7f0054069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.226+0000 7f006c99d700 1 --2- 172.21.15.55:0/4004252912 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f00681566e0 0x7f006813fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.226+0000 7f006c99d700 1 --2- 172.21.15.55:0/4004252912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f00680ae710 0x7f00680af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.227+0000 7f006c99d700 1 --2- 172.21.15.55:0/4004252912 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0068156310 0x7f00680b1650 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.227+0000 7f006c99d700 1 -- 172.21.15.55:0/4004252912 >> 172.21.15.55:0/4004252912 conn(0x7f00680cc1f0 msgr2=0x7f0068153ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:12.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.227+0000 7f006c99d700 1 -- 172.21.15.55:0/4004252912 shutdown_connections 2023-10-06T03:52:12.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.227+0000 7f006c99d700 1 -- 172.21.15.55:0/4004252912 wait complete. 2023-10-06T03:52:12.242 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 61 2023-10-06T03:52:12.246 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:52:12.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.438+0000 7fd559f9d700 1 Processor -- start 2023-10-06T03:52:12.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.439+0000 7fd559f9d700 1 -- start start 2023-10-06T03:52:12.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.440+0000 7fd559f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd554150550 0x7fd554157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:12.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.440+0000 7fd559f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5540c7410 0x7fd554069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:12.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.440+0000 7fd559f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55406e2a0 con 0x7fd554150180 2023-10-06T03:52:12.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.440+0000 7fd559f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55406b2e0 con 0x7fd554150550 2023-10-06T03:52:12.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.440+0000 7fd559f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540c78e0 con 0x7fd5540c7410 2023-10-06T03:52:12.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.440+0000 7fd553fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd554150550 0x7fd554157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:12.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.440+0000 7fd553fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd554150550 0x7fd554157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38646/0 (socket says 172.21.15.55:38646) 2023-10-06T03:52:12.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.440+0000 7fd553fff700 1 -- 172.21.15.55:0/2561987665 learned_addr learned my addr 172.21.15.55:0/2561987665 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:12.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.440+0000 7fd55979c700 1 --2- 172.21.15.55:0/2561987665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5540c7410 0x7fd554069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:12.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.441+0000 7fd553fff700 1 -- 172.21.15.55:0/2561987665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5540c7410 msgr2=0x7fd554069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:12.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.441+0000 7fd553fff700 1 --2- 172.21.15.55:0/2561987665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5540c7410 0x7fd554069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.441+0000 7fd553fff700 1 -- 172.21.15.55:0/2561987665 >> v1:172.21.15.163:6789/0 conn(0x7fd554150180 legacy=0x7fd5540c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:12.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.441+0000 7fd553fff700 1 -- 172.21.15.55:0/2561987665 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5540c7bc0 con 0x7fd554150550 2023-10-06T03:52:12.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.441+0000 7fd553fff700 1 --2- 172.21.15.55:0/2561987665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd554150550 0x7fd554157770 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fd5400099b0 tx=0x7fd540009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba4e8ef00e05f57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:12.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.441+0000 7fd5537fe700 1 -- 172.21.15.55:0/2561987665 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54000b860 con 0x7fd554150550 2023-10-06T03:52:12.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.441+0000 7fd5537fe700 1 -- 172.21.15.55:0/2561987665 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54000b9c0 con 0x7fd554150550 2023-10-06T03:52:12.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.441+0000 7fd5537fe700 1 -- 172.21.15.55:0/2561987665 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54000bd60 con 0x7fd554150550 2023-10-06T03:52:12.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.442+0000 7fd559f9d700 1 -- 172.21.15.55:0/2561987665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd554150550 msgr2=0x7fd554157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:12.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.442+0000 7fd559f9d700 1 --2- 172.21.15.55:0/2561987665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd554150550 0x7fd554157770 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fd5400099b0 tx=0x7fd540009250 comp rx=0 tx=0).stop 2023-10-06T03:52:12.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.442+0000 7fd559f9d700 1 -- 172.21.15.55:0/2561987665 shutdown_connections 2023-10-06T03:52:12.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.442+0000 7fd559f9d700 1 --2- 172.21.15.55:0/2561987665 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd5540c7410 0x7fd554069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.442+0000 7fd559f9d700 1 --2- 172.21.15.55:0/2561987665 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd554150550 0x7fd554157770 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.442+0000 7fd559f9d700 1 -- 172.21.15.55:0/2561987665 >> 172.21.15.55:0/2561987665 conn(0x7fd5540cc1f0 msgr2=0x7fd5540cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:12.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.442+0000 7fd559f9d700 1 -- 172.21.15.55:0/2561987665 shutdown_connections 2023-10-06T03:52:12.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.442+0000 7fd559f9d700 1 -- 172.21.15.55:0/2561987665 wait complete. 2023-10-06T03:52:12.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.443+0000 7fd559f9d700 1 Processor -- start 2023-10-06T03:52:12.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.444+0000 7fd559f9d700 1 -- start start 2023-10-06T03:52:12.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.444+0000 7fd559f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540c7410 0x7fd5540b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:12.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.444+0000 7fd559f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd554150180 0x7fd55414a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:12.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.444+0000 7fd559f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd554150550 0x7fd55415dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:12.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.444+0000 7fd559f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540c2170 con 0x7fd5540c7410 2023-10-06T03:52:12.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.444+0000 7fd559f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540b1e10 con 0x7fd554150180 2023-10-06T03:52:12.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.444+0000 7fd559f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540c2db0 con 0x7fd554150550 2023-10-06T03:52:12.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.445+0000 7fd558f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540c7410 0x7fd5540b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:12.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.445+0000 7fd558f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540c7410 0x7fd5540b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38662/0 (socket says 172.21.15.55:38662) 2023-10-06T03:52:12.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.445+0000 7fd558f9b700 1 -- 172.21.15.55:0/793496594 learned_addr learned my addr 172.21.15.55:0/793496594 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:12.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.445+0000 7fd55979c700 1 --2- 172.21.15.55:0/793496594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd554150550 0x7fd55415dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:12.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.445+0000 7fd553fff700 1 --2- 172.21.15.55:0/793496594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd554150180 0x7fd55414a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:12.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.445+0000 7fd558f9b700 1 -- 172.21.15.55:0/793496594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd554150550 msgr2=0x7fd55415dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:12.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.445+0000 7fd558f9b700 1 --2- 172.21.15.55:0/793496594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd554150550 0x7fd55415dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.445+0000 7fd558f9b700 1 -- 172.21.15.55:0/793496594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd554150180 msgr2=0x7fd55414a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:12.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.445+0000 7fd558f9b700 1 --2- 172.21.15.55:0/793496594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd554150180 0x7fd55414a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.445+0000 7fd558f9b700 1 -- 172.21.15.55:0/793496594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400094f0 con 0x7fd5540c7410 2023-10-06T03:52:12.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.445+0000 7fd558f9b700 1 --2- 172.21.15.55:0/793496594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540c7410 0x7fd5540b18d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd54800c590 tx=0x7fd54800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:12.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.446+0000 7fd550ff9700 1 -- 172.21.15.55:0/793496594 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548011590 con 0x7fd5540c7410 2023-10-06T03:52:12.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.446+0000 7fd550ff9700 1 -- 172.21.15.55:0/793496594 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5480116f0 con 0x7fd5540c7410 2023-10-06T03:52:12.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.446+0000 7fd550ff9700 1 -- 172.21.15.55:0/793496594 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548011ab0 con 0x7fd5540c7410 2023-10-06T03:52:12.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.446+0000 7fd559f9d700 1 -- 172.21.15.55:0/793496594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5540c1b20 con 0x7fd5540c7410 2023-10-06T03:52:12.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.446+0000 7fd559f9d700 1 -- 172.21.15.55:0/793496594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55414ae80 con 0x7fd5540c7410 2023-10-06T03:52:12.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.447+0000 7fd559f9d700 1 -- 172.21.15.55:0/793496594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55404c3c0 con 0x7fd5540c7410 2023-10-06T03:52:12.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.448+0000 7fd550ff9700 1 -- 172.21.15.55:0/793496594 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 57) v1 ==== 84578+0+0 (secure 0 0 0) 0x7fd54801b3b0 con 0x7fd5540c7410 2023-10-06T03:52:12.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.448+0000 7fd550ff9700 1 --2- 172.21.15.55:0/793496594 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd538066d80 0x7fd538069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:12.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.449+0000 7fd550ff9700 1 -- 172.21.15.55:0/793496594 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7108+0+0 (secure 0 0 0) 0x7fd548083c90 con 0x7fd5540c7410 2023-10-06T03:52:12.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.449+0000 7fd553fff700 1 --2- 172.21.15.55:0/793496594 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd538066d80 0x7fd538069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:12.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.450+0000 7fd553fff700 1 --2- 172.21.15.55:0/793496594 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd538066d80 0x7fd538069230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd5400099b0 tx=0x7fd540017000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:12.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.453+0000 7fd550ff9700 1 -- 172.21.15.55:0/793496594 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd548050c30 con 0x7fd5540c7410 2023-10-06T03:52:12.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.700+0000 7fd559f9d700 1 -- 172.21.15.55:0/793496594 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5540ab520 con 0x7fd5540c7410 2023-10-06T03:52:12.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.700+0000 7fd550ff9700 1 -- 172.21.15.55:0/793496594 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fd548017070 con 0x7fd5540c7410 2023-10-06T03:52:12.705 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:12.705 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1295635544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":1295635544},{"type":"v1","addr":"172.21.15.55:6835","nonce":1295635544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7737,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2115790576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2115790576},{"type":"v1","addr":"172.21.15.163:6834","nonce":2115790576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1021807391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1021807391},{"type":"v1","addr":"172.21.15.163:6836","nonce":1021807391}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:51:26.480340+0000","modified":"2023-10-06T03:51:30.017376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7743":{"gid":7743,"name":"c","rank":0,"incarnation":57,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/4240717455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":4240717455},{"type":"v1","addr":"172.21.15.55:6834","nonce":4240717455}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-06T03:52:12.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.703+0000 7fd559f9d700 1 -- 172.21.15.55:0/793496594 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd538066d80 msgr2=0x7fd538069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:12.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.703+0000 7fd559f9d700 1 --2- 172.21.15.55:0/793496594 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd538066d80 0x7fd538069230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd5400099b0 tx=0x7fd540017000 comp rx=0 tx=0).stop 2023-10-06T03:52:12.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.703+0000 7fd559f9d700 1 -- 172.21.15.55:0/793496594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540c7410 msgr2=0x7fd5540b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:12.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.703+0000 7fd559f9d700 1 --2- 172.21.15.55:0/793496594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540c7410 0x7fd5540b18d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd54800c590 tx=0x7fd54800c950 comp rx=0 tx=0).stop 2023-10-06T03:52:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.703+0000 7fd559f9d700 1 -- 172.21.15.55:0/793496594 shutdown_connections 2023-10-06T03:52:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.703+0000 7fd559f9d700 1 --2- 172.21.15.55:0/793496594 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd538066d80 0x7fd538069230 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.703+0000 7fd559f9d700 1 --2- 172.21.15.55:0/793496594 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd554150550 0x7fd55415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.703+0000 7fd559f9d700 1 --2- 172.21.15.55:0/793496594 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540c7410 0x7fd5540b18d0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.703+0000 7fd559f9d700 1 --2- 172.21.15.55:0/793496594 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd554150180 0x7fd55414a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.703+0000 7fd559f9d700 1 -- 172.21.15.55:0/793496594 >> 172.21.15.55:0/793496594 conn(0x7fd5540cc1f0 msgr2=0x7fd5540c8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.703+0000 7fd559f9d700 1 -- 172.21.15.55:0/793496594 shutdown_connections 2023-10-06T03:52:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.703+0000 7fd559f9d700 1 -- 172.21.15.55:0/793496594 wait complete. 2023-10-06T03:52:12.707 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 61 2023-10-06T03:52:12.722 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:52:12.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.916+0000 7ff438f9d700 1 Processor -- start 2023-10-06T03:52:12.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.917+0000 7ff438f9d700 1 -- start start 2023-10-06T03:52:12.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.918+0000 7ff438f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff434142ea0 0x7ff4340b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:12.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.918+0000 7ff438f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff4340ad870 0x7ff43406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:12.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.918+0000 7ff438f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4340b40f0 con 0x7ff4340b3b70 2023-10-06T03:52:12.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.918+0000 7ff438f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff434069640 con 0x7ff4340ad870 2023-10-06T03:52:12.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.918+0000 7ff438f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff434069920 con 0x7ff434142ea0 2023-10-06T03:52:12.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.918+0000 7ff432ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff4340ad870 0x7ff43406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:12.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.918+0000 7ff432ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff4340ad870 0x7ff43406b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38676/0 (socket says 172.21.15.55:38676) 2023-10-06T03:52:12.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.918+0000 7ff432ffd700 1 -- 172.21.15.55:0/1052686748 learned_addr learned my addr 172.21.15.55:0/1052686748 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:12.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.918+0000 7ff4337fe700 1 --2- 172.21.15.55:0/1052686748 >> v2:172.21.15.163:3300/0 conn(0x7ff434142ea0 0x7ff4340b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:12.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.919+0000 7ff432ffd700 1 -- 172.21.15.55:0/1052686748 >> v2:172.21.15.163:3300/0 conn(0x7ff434142ea0 msgr2=0x7ff4340b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:12.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.919+0000 7ff432ffd700 1 --2- 172.21.15.55:0/1052686748 >> v2:172.21.15.163:3300/0 conn(0x7ff434142ea0 0x7ff4340b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.919+0000 7ff432ffd700 1 -- 172.21.15.55:0/1052686748 >> v1:172.21.15.163:6789/0 conn(0x7ff4340b3b70 legacy=0x7ff43406cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:12.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.919+0000 7ff432ffd700 1 -- 172.21.15.55:0/1052686748 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff434069c00 con 0x7ff4340ad870 2023-10-06T03:52:12.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.919+0000 7ff432ffd700 1 --2- 172.21.15.55:0/1052686748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4340ad870 0x7ff43406b9e0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff434071240 tx=0x7ff428009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a830a807842908e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:12.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.919+0000 7ff4327fc700 1 -- 172.21.15.55:0/1052686748 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff428009e70 con 0x7ff4340ad870 2023-10-06T03:52:12.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.919+0000 7ff4327fc700 1 -- 172.21.15.55:0/1052686748 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42800b860 con 0x7ff4340ad870 2023-10-06T03:52:12.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.919+0000 7ff4327fc700 1 -- 172.21.15.55:0/1052686748 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42800bc00 con 0x7ff4340ad870 2023-10-06T03:52:12.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.920+0000 7ff438f9d700 1 -- 172.21.15.55:0/1052686748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4340ad870 msgr2=0x7ff43406b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:12.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.920+0000 7ff438f9d700 1 --2- 172.21.15.55:0/1052686748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4340ad870 0x7ff43406b9e0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff434071240 tx=0x7ff428009250 comp rx=0 tx=0).stop 2023-10-06T03:52:12.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.920+0000 7ff438f9d700 1 -- 172.21.15.55:0/1052686748 shutdown_connections 2023-10-06T03:52:12.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.920+0000 7ff438f9d700 1 --2- 172.21.15.55:0/1052686748 >> v2:172.21.15.163:3300/0 conn(0x7ff434142ea0 0x7ff4340b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.920+0000 7ff438f9d700 1 --2- 172.21.15.55:0/1052686748 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4340ad870 0x7ff43406b9e0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.920+0000 7ff438f9d700 1 -- 172.21.15.55:0/1052686748 >> 172.21.15.55:0/1052686748 conn(0x7ff4340ce430 msgr2=0x7ff4340ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:12.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.920+0000 7ff438f9d700 1 -- 172.21.15.55:0/1052686748 shutdown_connections 2023-10-06T03:52:12.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.920+0000 7ff438f9d700 1 -- 172.21.15.55:0/1052686748 wait complete. 2023-10-06T03:52:12.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.921+0000 7ff438f9d700 1 Processor -- start 2023-10-06T03:52:12.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.922+0000 7ff438f9d700 1 -- start start 2023-10-06T03:52:12.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.922+0000 7ff438f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4340b3b70 0x7ff43416dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:12.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.922+0000 7ff438f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434142ea0 0x7ff4340c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:12.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.922+0000 7ff438f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4340c7ab0 0x7ff4340c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:12.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.922+0000 7ff438f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43415edb0 con 0x7ff4340b3b70 2023-10-06T03:52:12.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.922+0000 7ff438f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340c8490 con 0x7ff4340c7ab0 2023-10-06T03:52:12.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.922+0000 7ff438f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4340c98f0 con 0x7ff434142ea0 2023-10-06T03:52:12.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.923+0000 7ff4337fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4340b3b70 0x7ff43416dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:12.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.923+0000 7ff4337fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4340b3b70 0x7ff43416dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38692/0 (socket says 172.21.15.55:38692) 2023-10-06T03:52:12.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.923+0000 7ff4337fe700 1 -- 172.21.15.55:0/2997365188 learned_addr learned my addr 172.21.15.55:0/2997365188 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:12.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.923+0000 7ff432ffd700 1 --2- 172.21.15.55:0/2997365188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434142ea0 0x7ff4340c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:12.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.923+0000 7ff433fff700 1 --2- 172.21.15.55:0/2997365188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4340c7ab0 0x7ff4340c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:12.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.923+0000 7ff4337fe700 1 -- 172.21.15.55:0/2997365188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434142ea0 msgr2=0x7ff4340c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:12.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.923+0000 7ff4337fe700 1 --2- 172.21.15.55:0/2997365188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434142ea0 0x7ff4340c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.923+0000 7ff4337fe700 1 -- 172.21.15.55:0/2997365188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4340c7ab0 msgr2=0x7ff4340c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:12.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.923+0000 7ff4337fe700 1 --2- 172.21.15.55:0/2997365188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4340c7ab0 0x7ff4340c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:12.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.923+0000 7ff4337fe700 1 -- 172.21.15.55:0/2997365188 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4280094f0 con 0x7ff4340b3b70 2023-10-06T03:52:12.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.923+0000 7ff4337fe700 1 --2- 172.21.15.55:0/2997365188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4340b3b70 0x7ff43416dab0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff42400e930 tx=0x7ff4240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:12.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.924+0000 7ff41b7fe700 1 -- 172.21.15.55:0/2997365188 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42400eed0 con 0x7ff4340b3b70 2023-10-06T03:52:13.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.924+0000 7ff41b7fe700 1 -- 172.21.15.55:0/2997365188 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42400c850 con 0x7ff4340b3b70 2023-10-06T03:52:13.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.924+0000 7ff438f9d700 1 -- 172.21.15.55:0/2997365188 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43416dff0 con 0x7ff4340b3b70 2023-10-06T03:52:13.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.924+0000 7ff41b7fe700 1 -- 172.21.15.55:0/2997365188 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42400cbf0 con 0x7ff4340b3b70 2023-10-06T03:52:13.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.924+0000 7ff438f9d700 1 -- 172.21.15.55:0/2997365188 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43415ead0 con 0x7ff4340b3b70 2023-10-06T03:52:13.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.925+0000 7ff438f9d700 1 -- 172.21.15.55:0/2997365188 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43404c3c0 con 0x7ff4340b3b70 2023-10-06T03:52:13.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.926+0000 7ff41b7fe700 1 -- 172.21.15.55:0/2997365188 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 57) v1 ==== 84578+0+0 (secure 0 0 0) 0x7ff424013470 con 0x7ff4340b3b70 2023-10-06T03:52:13.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.926+0000 7ff41b7fe700 1 --2- 172.21.15.55:0/2997365188 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff420066d80 0x7ff420069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:13.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.927+0000 7ff41b7fe700 1 -- 172.21.15.55:0/2997365188 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7108+0+0 (secure 0 0 0) 0x7ff424084ac0 con 0x7ff4340b3b70 2023-10-06T03:52:13.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.927+0000 7ff432ffd700 1 --2- 172.21.15.55:0/2997365188 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff420066d80 0x7ff420069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:13.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.928+0000 7ff432ffd700 1 --2- 172.21.15.55:0/2997365188 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff420066d80 0x7ff420069230 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff4340b12a0 tx=0x7ff428017000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:13.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:12.931+0000 7ff41b7fe700 1 -- 172.21.15.55:0/2997365188 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff424051990 con 0x7ff4340b3b70 2023-10-06T03:52:13.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.027+0000 7ff41b7fe700 1 -- 172.21.15.55:0/2997365188 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7ff424051d60 con 0x7ff4340b3b70 2023-10-06T03:52:13.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.137+0000 7ff438f9d700 1 -- 172.21.15.55:0/2997365188 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff434143270 con 0x7ff4340b3b70 2023-10-06T03:52:13.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.138+0000 7ff41b7fe700 1 -- 172.21.15.55:0/2997365188 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+18510 (secure 0 0 0) 0x7ff424055420 con 0x7ff4340b3b70 2023-10-06T03:52:13.147 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:13.148 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":125,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:52:10.905015+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:51:24.466543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-10-06T03:51:25.462282+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":125,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3544067284":"2023-10-07T03:52:10.904948+0000","172.21.15.163:0/3968893879":"2023-10-07T03:52:10.904948+0000","172.21.15.163:0/2623252410":"2023-10-07T03:52:10.904948+0000","172.21.15.163:0/3805627016":"2023-10-07T03:52:10.904948+0000","172.21.15.163:0/378507895":"2023-10-07T03:52:10.904948+0000","172.21.15.163:0/3247547835":"2023-10-07T03:52:10.904948+0000","172.21.15.163:0/2638749782":"2023-10-07T03:52:10.904948+0000","172.21.15.163:6837/85191":"2023-10-07T03:51:43.863644+0000","172.21.15.163:6835/85191":"2023-10-07T03:51:43.863644+0000","172.21.15.163:0/2003915970":"2023-10-07T03:51:43.863644+0000","172.21.15.163:0/357062062":"2023-10-07T03:51:43.863644+0000","172.21.15.163:0/1980310619":"2023-10-07T03:52:10.904948+0000","172.21.15.163:0/2409430869":"2023-10-07T03:51:43.863644+0000","172.21.15.163:0/3972526625":"2023-10-07T03:51:43.863644+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":16,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:52:13.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.141+0000 7ff438f9d700 1 -- 172.21.15.55:0/2997365188 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff420066d80 msgr2=0x7ff420069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:13.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.141+0000 7ff438f9d700 1 --2- 172.21.15.55:0/2997365188 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff420066d80 0x7ff420069230 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff4340b12a0 tx=0x7ff428017000 comp rx=0 tx=0).stop 2023-10-06T03:52:13.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.141+0000 7ff438f9d700 1 -- 172.21.15.55:0/2997365188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4340b3b70 msgr2=0x7ff43416dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:13.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.141+0000 7ff438f9d700 1 --2- 172.21.15.55:0/2997365188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4340b3b70 0x7ff43416dab0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ff42400e930 tx=0x7ff4240091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.141+0000 7ff438f9d700 1 -- 172.21.15.55:0/2997365188 shutdown_connections 2023-10-06T03:52:13.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.141+0000 7ff438f9d700 1 --2- 172.21.15.55:0/2997365188 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff420066d80 0x7ff420069230 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.141+0000 7ff438f9d700 1 --2- 172.21.15.55:0/2997365188 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff4340c7ab0 0x7ff4340c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.141+0000 7ff438f9d700 1 --2- 172.21.15.55:0/2997365188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4340b3b70 0x7ff43416dab0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.141+0000 7ff438f9d700 1 --2- 172.21.15.55:0/2997365188 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff434142ea0 0x7ff4340c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.141+0000 7ff438f9d700 1 -- 172.21.15.55:0/2997365188 >> 172.21.15.55:0/2997365188 conn(0x7ff4340ce430 msgr2=0x7ff434141e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:13.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.141+0000 7ff438f9d700 1 -- 172.21.15.55:0/2997365188 shutdown_connections 2023-10-06T03:52:13.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.141+0000 7ff438f9d700 1 -- 172.21.15.55:0/2997365188 wait complete. 2023-10-06T03:52:13.160 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T03:52:13.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.355+0000 7fd1e09fe700 1 Processor -- start 2023-10-06T03:52:13.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.355+0000 7fd1e09fe700 1 -- start start 2023-10-06T03:52:13.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.356+0000 7fd1e09fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd1dc159420 0x7fd1dc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:13.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.356+0000 7fd1e09fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc1597f0 0x7fd1dc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:13.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.356+0000 7fd1e09fe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1dc0b11d0 con 0x7fd1dc15d230 2023-10-06T03:52:13.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.356+0000 7fd1e09fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1dc0b14b0 con 0x7fd1dc159420 2023-10-06T03:52:13.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.356+0000 7fd1e09fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0b1790 con 0x7fd1dc1597f0 2023-10-06T03:52:13.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.357+0000 7fd1dad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd1dc159420 0x7fd1dc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:13.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.357+0000 7fd1dad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd1dc159420 0x7fd1dc0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51218/0 (socket says 172.21.15.55:51218) 2023-10-06T03:52:13.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.357+0000 7fd1dad9d700 1 -- 172.21.15.55:0/79177117 learned_addr learned my addr 172.21.15.55:0/79177117 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:13.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.357+0000 7fd1da59c700 1 --2- 172.21.15.55:0/79177117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc1597f0 0x7fd1dc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:13.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.357+0000 7fd1dad9d700 1 -- 172.21.15.55:0/79177117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc1597f0 msgr2=0x7fd1dc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:13.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.357+0000 7fd1dad9d700 1 --2- 172.21.15.55:0/79177117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc1597f0 0x7fd1dc0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.357+0000 7fd1dad9d700 1 -- 172.21.15.55:0/79177117 >> v1:172.21.15.163:6789/0 conn(0x7fd1dc15d230 legacy=0x7fd1dc0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:13.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.357+0000 7fd1dad9d700 1 -- 172.21.15.55:0/79177117 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc0b1a70 con 0x7fd1dc159420 2023-10-06T03:52:13.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.357+0000 7fd1dad9d700 1 --2- 172.21.15.55:0/79177117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1dc159420 0x7fd1dc0b5f40 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0099b0 tx=0x7fd1cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7933fa40a24ae9c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:13.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.358+0000 7fd1d9d9b700 1 -- 172.21.15.55:0/79177117 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc00b860 con 0x7fd1dc159420 2023-10-06T03:52:13.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.358+0000 7fd1d9d9b700 1 -- 172.21.15.55:0/79177117 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc00b9c0 con 0x7fd1dc159420 2023-10-06T03:52:13.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.358+0000 7fd1d9d9b700 1 -- 172.21.15.55:0/79177117 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc00bd60 con 0x7fd1dc159420 2023-10-06T03:52:13.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.358+0000 7fd1e09fe700 1 -- 172.21.15.55:0/79177117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1dc159420 msgr2=0x7fd1dc0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:13.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.358+0000 7fd1e09fe700 1 --2- 172.21.15.55:0/79177117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1dc159420 0x7fd1dc0b5f40 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0099b0 tx=0x7fd1cc009250 comp rx=0 tx=0).stop 2023-10-06T03:52:13.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.358+0000 7fd1e09fe700 1 -- 172.21.15.55:0/79177117 shutdown_connections 2023-10-06T03:52:13.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.358+0000 7fd1e09fe700 1 --2- 172.21.15.55:0/79177117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1dc159420 0x7fd1dc0b5f40 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.358+0000 7fd1e09fe700 1 --2- 172.21.15.55:0/79177117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc1597f0 0x7fd1dc0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.358+0000 7fd1e09fe700 1 -- 172.21.15.55:0/79177117 >> 172.21.15.55:0/79177117 conn(0x7fd1dc0d0660 msgr2=0x7fd1dc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:13.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.358+0000 7fd1e09fe700 1 -- 172.21.15.55:0/79177117 shutdown_connections 2023-10-06T03:52:13.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.358+0000 7fd1e09fe700 1 -- 172.21.15.55:0/79177117 wait complete. 2023-10-06T03:52:13.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.360+0000 7fd1e09fe700 1 Processor -- start 2023-10-06T03:52:13.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.360+0000 7fd1e09fe700 1 -- start start 2023-10-06T03:52:13.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1e09fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc159420 0x7fd1dc16ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:13.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1e09fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc1597f0 0x7fd1dc0c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:13.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1e09fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1dc15d230 0x7fd1dc15d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:13.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1e09fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0c5ec0 con 0x7fd1dc15d230 2023-10-06T03:52:13.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1e09fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc16d6e0 con 0x7fd1dc159420 2023-10-06T03:52:13.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1e09fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc16d9c0 con 0x7fd1dc1597f0 2023-10-06T03:52:13.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1db59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1dc15d230 0x7fd1dc15d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:13.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1db59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1dc15d230 0x7fd1dc15d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51232/0 (socket says 172.21.15.55:51232) 2023-10-06T03:52:13.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1db59e700 1 -- 172.21.15.55:0/3628389027 learned_addr learned my addr 172.21.15.55:0/3628389027 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:13.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1dad9d700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc159420 0x7fd1dc16ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:13.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1da59c700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc1597f0 0x7fd1dc0c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:13.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1db59e700 1 -- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc1597f0 msgr2=0x7fd1dc0c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:13.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1db59e700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc1597f0 0x7fd1dc0c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1db59e700 1 -- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc159420 msgr2=0x7fd1dc16ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:13.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1db59e700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc159420 0x7fd1dc16ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1db59e700 1 -- 172.21.15.55:0/3628389027 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc0094f0 con 0x7fd1dc15d230 2023-10-06T03:52:13.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.361+0000 7fd1dad9d700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc159420 0x7fd1dc16ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:13.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.362+0000 7fd1db59e700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1dc15d230 0x7fd1dc15d8e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400c840 tx=0x7fd1d4001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:13.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.362+0000 7fd1c2ffd700 1 -- 172.21.15.55:0/3628389027 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4012630 con 0x7fd1dc15d230 2023-10-06T03:52:13.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.362+0000 7fd1c2ffd700 1 -- 172.21.15.55:0/3628389027 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d40127b0 con 0x7fd1dc15d230 2023-10-06T03:52:13.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.362+0000 7fd1c2ffd700 1 -- 172.21.15.55:0/3628389027 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4012b70 con 0x7fd1dc15d230 2023-10-06T03:52:13.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.362+0000 7fd1e09fe700 1 -- 172.21.15.55:0/3628389027 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc16dca0 con 0x7fd1dc15d230 2023-10-06T03:52:13.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.362+0000 7fd1e09fe700 1 -- 172.21.15.55:0/3628389027 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1dc16d3d0 con 0x7fd1dc15d230 2023-10-06T03:52:13.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.363+0000 7fd1e09fe700 1 -- 172.21.15.55:0/3628389027 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1dc04c3c0 con 0x7fd1dc15d230 2023-10-06T03:52:13.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.364+0000 7fd1c2ffd700 1 -- 172.21.15.55:0/3628389027 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd1d4003030 con 0x7fd1dc15d230 2023-10-06T03:52:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.365+0000 7fd1c2ffd700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd1c8067050 0x7fd1c8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.365+0000 7fd1c2ffd700 1 -- 172.21.15.55:0/3628389027 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 7108+0+0 (secure 0 0 0) 0x7fd1d4084c60 con 0x7fd1dc15d230 2023-10-06T03:52:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.365+0000 7fd1dad9d700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd1c8067050 0x7fd1c8069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.366+0000 7fd1dad9d700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd1c8067050 0x7fd1c8069500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0099b0 tx=0x7fd1cc017000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.369+0000 7fd1c2ffd700 1 -- 172.21.15.55:0/3628389027 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1d40517a0 con 0x7fd1dc15d230 2023-10-06T03:52:13.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.618+0000 7fd1e09fe700 1 -- 172.21.15.55:0/3628389027 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd1dc0b1d80 con 0x7fd1dc15d230 2023-10-06T03:52:13.635 INFO:tasks.ceph.mon.a.smithi055.stderr:2023-10-06T03:52:13.619+0000 7fe524ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T03:52:13.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.625+0000 7fd1c2ffd700 1 -- 172.21.15.55:0/3628389027 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd1d4054dc0 con 0x7fd1dc15d230 2023-10-06T03:52:13.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.627+0000 7fd1e09fe700 1 -- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd1c8067050 msgr2=0x7fd1c8069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:13.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.627+0000 7fd1e09fe700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd1c8067050 0x7fd1c8069500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0099b0 tx=0x7fd1cc017000 comp rx=0 tx=0).stop 2023-10-06T03:52:13.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.627+0000 7fd1e09fe700 1 -- 172.21.15.55:0/3628389027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1dc15d230 msgr2=0x7fd1dc15d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:13.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.627+0000 7fd1e09fe700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1dc15d230 0x7fd1dc15d8e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400c840 tx=0x7fd1d4001670 comp rx=0 tx=0).stop 2023-10-06T03:52:13.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.628+0000 7fd1e09fe700 1 -- 172.21.15.55:0/3628389027 shutdown_connections 2023-10-06T03:52:13.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.628+0000 7fd1e09fe700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd1c8067050 0x7fd1c8069500 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.628+0000 7fd1e09fe700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1dc15d230 0x7fd1dc15d8e0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.628+0000 7fd1e09fe700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1dc159420 0x7fd1dc16ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.628+0000 7fd1e09fe700 1 --2- 172.21.15.55:0/3628389027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1dc1597f0 0x7fd1dc0c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.628+0000 7fd1e09fe700 1 -- 172.21.15.55:0/3628389027 >> 172.21.15.55:0/3628389027 conn(0x7fd1dc0d0660 msgr2=0x7fd1dc1562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:13.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.628+0000 7fd1e09fe700 1 -- 172.21.15.55:0/3628389027 shutdown_connections 2023-10-06T03:52:13.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.628+0000 7fd1e09fe700 1 -- 172.21.15.55:0/3628389027 wait complete. 2023-10-06T03:52:13.643 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T03:52:13.643 INFO:tasks.ceph.mds.c.smithi055.stderr: -5901> 2023-10-06T03:51:30.958+0000 7fbe1e7f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:52:13.647 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T03:52:13.682 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-10-06T03:52:13.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.837+0000 7f08b3fff700 1 Processor -- start 2023-10-06T03:52:13.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.838+0000 7f08b3fff700 1 -- start start 2023-10-06T03:52:13.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.839+0000 7f08b3fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f08b406f060 0x7f08b40ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:13.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.839+0000 7f08b3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f08b406f4b0 0x7f08b40ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.839+0000 7f08b3fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b406ebd0 con 0x7f08b40cb1a0 2023-10-06T03:52:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.839+0000 7f08b3fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b415b2e0 con 0x7f08b406f4b0 2023-10-06T03:52:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.839+0000 7f08b3fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b415aa70 con 0x7f08b406f060 2023-10-06T03:52:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.839+0000 7f08b37fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f08b406f4b0 0x7f08b40ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.839+0000 7f08b2ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f08b40cb1a0 0x7f08b415a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39236/0 (socket says 172.21.15.55:39236) 2023-10-06T03:52:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.839+0000 7f08b2ffd700 1 -- 172.21.15.55:0/2753391795 learned_addr learned my addr 172.21.15.55:0/2753391795 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.839+0000 7f08b27fc700 1 --2- 172.21.15.55:0/2753391795 >> v2:172.21.15.163:3300/0 conn(0x7f08b406f060 0x7f08b40ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.840+0000 7f08b37fe700 1 -- 172.21.15.55:0/2753391795 >> v2:172.21.15.163:3300/0 conn(0x7f08b406f060 msgr2=0x7f08b40ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.840+0000 7f08b37fe700 1 --2- 172.21.15.55:0/2753391795 >> v2:172.21.15.163:3300/0 conn(0x7f08b406f060 0x7f08b40ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.840+0000 7f08b37fe700 1 -- 172.21.15.55:0/2753391795 >> v1:172.21.15.55:6789/0 conn(0x7f08b40cb1a0 legacy=0x7f08b415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.840+0000 7f08b37fe700 1 -- 172.21.15.55:0/2753391795 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b415ad20 con 0x7f08b406f4b0 2023-10-06T03:52:13.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.840+0000 7f08b1ffb700 1 -- 172.21.15.55:0/2753391795 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230329882 0 0) 0x7f08b406ebd0 con 0x7f08b40cb1a0 2023-10-06T03:52:13.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.840+0000 7f08b37fe700 1 --2- 172.21.15.55:0/2753391795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08b406f4b0 0x7f08b40ae610 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f08b4070d30 tx=0x7f08ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2156fe672780ffd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:13.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.840+0000 7f08b1ffb700 1 -- 172.21.15.55:0/2753391795 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac00e070 con 0x7f08b406f4b0 2023-10-06T03:52:13.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.840+0000 7f08b1ffb700 1 -- 172.21.15.55:0/2753391795 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ac00bbf0 con 0x7f08b406f4b0 2023-10-06T03:52:13.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.840+0000 7f08b1ffb700 1 -- 172.21.15.55:0/2753391795 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac012430 con 0x7f08b406f4b0 2023-10-06T03:52:13.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.841+0000 7f08b3fff700 1 -- 172.21.15.55:0/2753391795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08b406f4b0 msgr2=0x7f08b40ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:13.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.841+0000 7f08b3fff700 1 --2- 172.21.15.55:0/2753391795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08b406f4b0 0x7f08b40ae610 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f08b4070d30 tx=0x7f08ac009780 comp rx=0 tx=0).stop 2023-10-06T03:52:13.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.841+0000 7f08b3fff700 1 -- 172.21.15.55:0/2753391795 shutdown_connections 2023-10-06T03:52:13.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.841+0000 7f08b3fff700 1 --2- 172.21.15.55:0/2753391795 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08b406f4b0 0x7f08b40ae610 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.841+0000 7f08b3fff700 1 --2- 172.21.15.55:0/2753391795 >> v2:172.21.15.163:3300/0 conn(0x7f08b406f060 0x7f08b40ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.841+0000 7f08b3fff700 1 -- 172.21.15.55:0/2753391795 >> 172.21.15.55:0/2753391795 conn(0x7f08b40cc1e0 msgr2=0x7f08b40cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:13.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.841+0000 7f08b3fff700 1 -- 172.21.15.55:0/2753391795 shutdown_connections 2023-10-06T03:52:13.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.841+0000 7f08b3fff700 1 -- 172.21.15.55:0/2753391795 wait complete. 2023-10-06T03:52:13.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.842+0000 7f08b3fff700 1 Processor -- start 2023-10-06T03:52:13.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.842+0000 7f08b3fff700 1 -- start start 2023-10-06T03:52:13.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.843+0000 7f08b3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b406f060 0x7f08b4168570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:13.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.843+0000 7f08b3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08b406f4b0 0x7f08b4167e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:13.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.843+0000 7f08b3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08b40cb1a0 0x7f08b4165000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:13.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.843+0000 7f08b3fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b41683d0 con 0x7f08b40cb1a0 2023-10-06T03:52:13.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.843+0000 7f08b3fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b4169810 con 0x7f08b406f060 2023-10-06T03:52:13.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.843+0000 7f08b3fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b4168c00 con 0x7f08b406f4b0 2023-10-06T03:52:13.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.844+0000 7f08b37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08b40cb1a0 0x7f08b4165000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:13.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.844+0000 7f08b37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08b40cb1a0 0x7f08b4165000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51266/0 (socket says 172.21.15.55:51266) 2023-10-06T03:52:13.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.844+0000 7f08b37fe700 1 -- 172.21.15.55:0/2316012444 learned_addr learned my addr 172.21.15.55:0/2316012444 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:13.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.844+0000 7f08b2ffd700 1 --2- 172.21.15.55:0/2316012444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b406f060 0x7f08b4168570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:13.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.844+0000 7f08b27fc700 1 --2- 172.21.15.55:0/2316012444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08b406f4b0 0x7f08b4167e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:13.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.844+0000 7f08b37fe700 1 -- 172.21.15.55:0/2316012444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08b406f4b0 msgr2=0x7f08b4167e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:13.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.844+0000 7f08b37fe700 1 --2- 172.21.15.55:0/2316012444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08b406f4b0 0x7f08b4167e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.844+0000 7f08b37fe700 1 -- 172.21.15.55:0/2316012444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b406f060 msgr2=0x7f08b4168570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:13.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.844+0000 7f08b37fe700 1 --2- 172.21.15.55:0/2316012444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b406f060 0x7f08b4168570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:13.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.844+0000 7f08b37fe700 1 -- 172.21.15.55:0/2316012444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac009a20 con 0x7f08b40cb1a0 2023-10-06T03:52:13.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.844+0000 7f08b37fe700 1 --2- 172.21.15.55:0/2316012444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08b40cb1a0 0x7f08b4165000 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f08b406ce80 tx=0x7f08ac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:13.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.845+0000 7f089affd700 1 -- 172.21.15.55:0/2316012444 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac027470 con 0x7f08b40cb1a0 2023-10-06T03:52:13.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.845+0000 7f089affd700 1 -- 172.21.15.55:0/2316012444 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ac0275f0 con 0x7f08b40cb1a0 2023-10-06T03:52:13.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.845+0000 7f08b3fff700 1 -- 172.21.15.55:0/2316012444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b4168ee0 con 0x7f08b40cb1a0 2023-10-06T03:52:13.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.845+0000 7f089affd700 1 -- 172.21.15.55:0/2316012444 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0279b0 con 0x7f08b40cb1a0 2023-10-06T03:52:13.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.845+0000 7f08b3fff700 1 -- 172.21.15.55:0/2316012444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b41694a0 con 0x7f08b40cb1a0 2023-10-06T03:52:13.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.846+0000 7f08b3fff700 1 -- 172.21.15.55:0/2316012444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b41699a0 con 0x7f08b40cb1a0 2023-10-06T03:52:13.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.847+0000 7f089affd700 1 -- 172.21.15.55:0/2316012444 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f08ac024070 con 0x7f08b40cb1a0 2023-10-06T03:52:13.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.848+0000 7f089affd700 1 --2- 172.21.15.55:0/2316012444 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f08a0067050 0x7f08a0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:13.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.848+0000 7f089affd700 1 -- 172.21.15.55:0/2316012444 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 7166+0+0 (secure 0 0 0) 0x7f08ac0873f0 con 0x7f08b40cb1a0 2023-10-06T03:52:13.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.849+0000 7f08b2ffd700 1 --2- 172.21.15.55:0/2316012444 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f08a0067050 0x7f08a0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:13.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.849+0000 7f08b2ffd700 1 --2- 172.21.15.55:0/2316012444 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f08a0067050 0x7f08a0069500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f08b415e540 tx=0x7f08a4007f50 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:13.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:13.852+0000 7f089affd700 1 -- 172.21.15.55:0/2316012444 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08ac053fa0 con 0x7f08b40cb1a0 2023-10-06T03:52:14.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.104+0000 7f08b3fff700 1 -- 172.21.15.55:0/2316012444 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f08b415b000 con 0x7f08b40cb1a0 2023-10-06T03:52:14.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.625+0000 7f089affd700 1 -- 172.21.15.55:0/2316012444 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v63) v1 ==== 104+0+0 (secure 0 0 0) 0x7f08ac0575c0 con 0x7f08b40cb1a0 2023-10-06T03:52:14.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.628+0000 7f08b3fff700 1 -- 172.21.15.55:0/2316012444 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f08a0067050 msgr2=0x7f08a0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:14.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.628+0000 7f08b3fff700 1 --2- 172.21.15.55:0/2316012444 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f08a0067050 0x7f08a0069500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f08b415e540 tx=0x7f08a4007f50 comp rx=0 tx=0).stop 2023-10-06T03:52:14.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.628+0000 7f08b3fff700 1 -- 172.21.15.55:0/2316012444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08b40cb1a0 msgr2=0x7f08b4165000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:14.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.628+0000 7f08b3fff700 1 --2- 172.21.15.55:0/2316012444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08b40cb1a0 0x7f08b4165000 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f08b406ce80 tx=0x7f08ac019000 comp rx=0 tx=0).stop 2023-10-06T03:52:14.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.628+0000 7f08b3fff700 1 -- 172.21.15.55:0/2316012444 shutdown_connections 2023-10-06T03:52:14.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.628+0000 7f08b3fff700 1 --2- 172.21.15.55:0/2316012444 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f08a0067050 0x7f08a0069500 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:14.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.628+0000 7f08b3fff700 1 --2- 172.21.15.55:0/2316012444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08b40cb1a0 0x7f08b4165000 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:14.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.628+0000 7f08b3fff700 1 --2- 172.21.15.55:0/2316012444 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f08b406f060 0x7f08b4168570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:14.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.628+0000 7f08b3fff700 1 --2- 172.21.15.55:0/2316012444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08b406f4b0 0x7f08b4167e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:14.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.628+0000 7f08b3fff700 1 -- 172.21.15.55:0/2316012444 >> 172.21.15.55:0/2316012444 conn(0x7f08b40cc1e0 msgr2=0x7f08b4155040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:14.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.629+0000 7f08b3fff700 1 -- 172.21.15.55:0/2316012444 shutdown_connections 2023-10-06T03:52:14.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.629+0000 7f08b3fff700 1 -- 172.21.15.55:0/2316012444 wait complete. 2023-10-06T03:52:14.650 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T03:52:14.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.882+0000 7faf4279d700 1 Processor -- start 2023-10-06T03:52:14.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.883+0000 7faf4279d700 1 -- start start 2023-10-06T03:52:14.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf4279d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faf3c0ae180 0x7faf3c0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf4279d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf3c0656e0 0x7faf3c0ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf4279d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faf3c06b290 0x7faf3c1594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf4279d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3c0aedf0 con 0x7faf3c06b290 2023-10-06T03:52:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf4279d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3c0aeb10 con 0x7faf3c0ae180 2023-10-06T03:52:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf4279d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0af540 con 0x7faf3c0656e0 2023-10-06T03:52:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf41f9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faf3c06b290 0x7faf3c1594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf41f9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faf3c06b290 0x7faf3c1594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51280/0 (socket says 172.21.15.55:51280) 2023-10-06T03:52:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf41f9c700 1 -- 172.21.15.55:0/3344035221 learned_addr learned my addr 172.21.15.55:0/3344035221 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:14.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf4179b700 1 --2- 172.21.15.55:0/3344035221 >> v2:172.21.15.163:3300/0 conn(0x7faf3c0ae180 0x7faf3c0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:14.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf40f9a700 1 --2- 172.21.15.55:0/3344035221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf3c0656e0 0x7faf3c0ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:14.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf41f9c700 1 -- 172.21.15.55:0/3344035221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf3c0656e0 msgr2=0x7faf3c0ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:14.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf41f9c700 1 --2- 172.21.15.55:0/3344035221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf3c0656e0 0x7faf3c0ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:14.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf41f9c700 1 -- 172.21.15.55:0/3344035221 >> v2:172.21.15.163:3300/0 conn(0x7faf3c0ae180 msgr2=0x7faf3c0acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:14.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf41f9c700 1 --2- 172.21.15.55:0/3344035221 >> v2:172.21.15.163:3300/0 conn(0x7faf3c0ae180 0x7faf3c0acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:14.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf41f9c700 1 -- 172.21.15.55:0/3344035221 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0af820 con 0x7faf3c06b290 2023-10-06T03:52:14.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf4179b700 1 --2- 172.21.15.55:0/3344035221 >> v2:172.21.15.163:3300/0 conn(0x7faf3c0ae180 0x7faf3c0acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:14.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.884+0000 7faf40f9a700 1 --2- 172.21.15.55:0/3344035221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf3c0656e0 0x7faf3c0ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:14.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.885+0000 7faf41f9c700 1 --2- 172.21.15.55:0/3344035221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf3c06b290 0x7faf3c1594b0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7faf3c067820 tx=0x7faf34009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a0d282ce36480a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:14.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.885+0000 7faf33fff700 1 -- 172.21.15.55:0/3344035221 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3400e070 con 0x7faf3c06b290 2023-10-06T03:52:14.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.885+0000 7faf33fff700 1 -- 172.21.15.55:0/3344035221 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3400ba50 con 0x7faf3c06b290 2023-10-06T03:52:14.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.885+0000 7faf33fff700 1 -- 172.21.15.55:0/3344035221 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf340092e0 con 0x7faf3c06b290 2023-10-06T03:52:14.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.885+0000 7faf4279d700 1 -- 172.21.15.55:0/3344035221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf3c06b290 msgr2=0x7faf3c1594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:14.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.885+0000 7faf4279d700 1 --2- 172.21.15.55:0/3344035221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf3c06b290 0x7faf3c1594b0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7faf3c067820 tx=0x7faf34009840 comp rx=0 tx=0).stop 2023-10-06T03:52:14.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.886+0000 7faf4279d700 1 -- 172.21.15.55:0/3344035221 shutdown_connections 2023-10-06T03:52:14.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.886+0000 7faf4279d700 1 --2- 172.21.15.55:0/3344035221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf3c06b290 0x7faf3c1594b0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:14.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.886+0000 7faf4279d700 1 --2- 172.21.15.55:0/3344035221 >> v2:172.21.15.163:3300/0 conn(0x7faf3c0ae180 0x7faf3c0acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:14.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.886+0000 7faf4279d700 1 --2- 172.21.15.55:0/3344035221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf3c0656e0 0x7faf3c0ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:14.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.886+0000 7faf4279d700 1 -- 172.21.15.55:0/3344035221 >> 172.21.15.55:0/3344035221 conn(0x7faf3c06f6a0 msgr2=0x7faf3c06ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:14.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.886+0000 7faf4279d700 1 -- 172.21.15.55:0/3344035221 shutdown_connections 2023-10-06T03:52:14.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.886+0000 7faf4279d700 1 -- 172.21.15.55:0/3344035221 wait complete. 2023-10-06T03:52:14.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.887+0000 7faf4279d700 1 Processor -- start 2023-10-06T03:52:14.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.887+0000 7faf4279d700 1 -- start start 2023-10-06T03:52:14.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.888+0000 7faf4279d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3c0656e0 0x7faf3c1555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:14.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.888+0000 7faf4279d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf3c06b290 0x7faf3c16a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:14.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.888+0000 7faf4279d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf3c0ae180 0x7faf3c098f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:14.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.888+0000 7faf4279d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c15a3f0 con 0x7faf3c0ae180 2023-10-06T03:52:14.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.888+0000 7faf4279d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c15ac90 con 0x7faf3c0656e0 2023-10-06T03:52:14.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.888+0000 7faf4279d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c15d580 con 0x7faf3c06b290 2023-10-06T03:52:14.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.888+0000 7faf41f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf3c0ae180 0x7faf3c098f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.888+0000 7faf41f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf3c0ae180 0x7faf3c098f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51294/0 (socket says 172.21.15.55:51294) 2023-10-06T03:52:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.888+0000 7faf41f9c700 1 -- 172.21.15.55:0/84161868 learned_addr learned my addr 172.21.15.55:0/84161868 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.889+0000 7faf4179b700 1 --2- 172.21.15.55:0/84161868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3c0656e0 0x7faf3c1555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.889+0000 7faf40f9a700 1 --2- 172.21.15.55:0/84161868 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf3c06b290 0x7faf3c16a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.889+0000 7faf41f9c700 1 -- 172.21.15.55:0/84161868 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf3c06b290 msgr2=0x7faf3c16a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.889+0000 7faf41f9c700 1 --2- 172.21.15.55:0/84161868 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf3c06b290 0x7faf3c16a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.889+0000 7faf41f9c700 1 -- 172.21.15.55:0/84161868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3c0656e0 msgr2=0x7faf3c1555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:14.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.889+0000 7faf41f9c700 1 --2- 172.21.15.55:0/84161868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3c0656e0 0x7faf3c1555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.889+0000 7faf41f9c700 1 -- 172.21.15.55:0/84161868 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf34009ae0 con 0x7faf3c0ae180 2023-10-06T03:52:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.889+0000 7faf41f9c700 1 --2- 172.21.15.55:0/84161868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf3c0ae180 0x7faf3c098f70 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0678d0 tx=0x7faf34018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.889+0000 7faf317fa700 1 -- 172.21.15.55:0/84161868 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf340094e0 con 0x7faf3c0ae180 2023-10-06T03:52:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.889+0000 7faf317fa700 1 -- 172.21.15.55:0/84161868 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf340032f0 con 0x7faf3c0ae180 2023-10-06T03:52:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.889+0000 7faf317fa700 1 -- 172.21.15.55:0/84161868 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34026460 con 0x7faf3c0ae180 2023-10-06T03:52:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.890+0000 7faf4279d700 1 -- 172.21.15.55:0/84161868 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c15a640 con 0x7faf3c0ae180 2023-10-06T03:52:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.890+0000 7faf4279d700 1 -- 172.21.15.55:0/84161868 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c155ae0 con 0x7faf3c0ae180 2023-10-06T03:52:14.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.891+0000 7faf4279d700 1 -- 172.21.15.55:0/84161868 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c15ae40 con 0x7faf3c0ae180 2023-10-06T03:52:14.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.891+0000 7faf317fa700 1 -- 172.21.15.55:0/84161868 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7faf34003040 con 0x7faf3c0ae180 2023-10-06T03:52:14.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.892+0000 7faf317fa700 1 --2- 172.21.15.55:0/84161868 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7faf20067050 0x7faf20069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:14.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.892+0000 7faf317fa700 1 -- 172.21.15.55:0/84161868 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 7166+0+0 (secure 0 0 0) 0x7faf34086d00 con 0x7faf3c0ae180 2023-10-06T03:52:14.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.893+0000 7faf4179b700 1 --2- 172.21.15.55:0/84161868 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7faf20067050 0x7faf20069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:14.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.893+0000 7faf4179b700 1 --2- 172.21.15.55:0/84161868 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7faf20067050 0x7faf20069500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faf3c16c8c0 tx=0x7faf38008040 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:14.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:14.897+0000 7faf317fa700 1 -- 172.21.15.55:0/84161868 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faf3c15ae40 con 0x7faf3c0ae180 2023-10-06T03:52:15.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:15.101+0000 7faf4279d700 1 -- 172.21.15.55:0/84161868 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faf3c15ae00 con 0x7faf3c0ae180 2023-10-06T03:52:15.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:15.768+0000 7faf317fa700 1 -- 172.21.15.55:0/84161868 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v127) v1 ==== 181+0+0 (secure 0 0 0) 0x7faf34053700 con 0x7faf3c0ae180 2023-10-06T03:52:15.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:15.773+0000 7faf4279d700 1 -- 172.21.15.55:0/84161868 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7faf20067050 msgr2=0x7faf20069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:15.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:15.773+0000 7faf4279d700 1 --2- 172.21.15.55:0/84161868 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7faf20067050 0x7faf20069500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7faf3c16c8c0 tx=0x7faf38008040 comp rx=0 tx=0).stop 2023-10-06T03:52:15.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:15.773+0000 7faf4279d700 1 -- 172.21.15.55:0/84161868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf3c0ae180 msgr2=0x7faf3c098f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:15.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:15.773+0000 7faf4279d700 1 --2- 172.21.15.55:0/84161868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf3c0ae180 0x7faf3c098f70 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0678d0 tx=0x7faf34018000 comp rx=0 tx=0).stop 2023-10-06T03:52:15.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:15.773+0000 7faf4279d700 1 -- 172.21.15.55:0/84161868 shutdown_connections 2023-10-06T03:52:15.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:15.773+0000 7faf4279d700 1 --2- 172.21.15.55:0/84161868 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7faf20067050 0x7faf20069500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:15.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:15.773+0000 7faf4279d700 1 --2- 172.21.15.55:0/84161868 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faf3c0ae180 0x7faf3c098f70 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:15.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:15.773+0000 7faf4279d700 1 --2- 172.21.15.55:0/84161868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faf3c0656e0 0x7faf3c1555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:15.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:15.773+0000 7faf4279d700 1 --2- 172.21.15.55:0/84161868 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faf3c06b290 0x7faf3c16a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:15.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:15.773+0000 7faf4279d700 1 -- 172.21.15.55:0/84161868 >> 172.21.15.55:0/84161868 conn(0x7faf3c06f6a0 msgr2=0x7faf3c0b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:15.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:15.773+0000 7faf4279d700 1 -- 172.21.15.55:0/84161868 shutdown_connections 2023-10-06T03:52:15.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:15.773+0000 7faf4279d700 1 -- 172.21.15.55:0/84161868 wait complete. 2023-10-06T03:52:15.791 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2023-10-06T03:52:15.796 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T03:52:16.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.000+0000 7fa053fff700 1 Processor -- start 2023-10-06T03:52:16.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.000+0000 7fa053fff700 1 -- start start 2023-10-06T03:52:16.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.001+0000 7fa053fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa054153480 0x7fa0540ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:16.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.001+0000 7fa053fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa054159dc0 0x7fa054157280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:16.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.001+0000 7fa053fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0540aec90 con 0x7fa054153850 2023-10-06T03:52:16.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.001+0000 7fa053fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa05415a480 con 0x7fa054153480 2023-10-06T03:52:16.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.001+0000 7fa053fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540afb40 con 0x7fa054159dc0 2023-10-06T03:52:16.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.001+0000 7fa052ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa054153480 0x7fa0540ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:16.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.002+0000 7fa052ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa054153480 0x7fa0540ae750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51300/0 (socket says 172.21.15.55:51300) 2023-10-06T03:52:16.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.002+0000 7fa052ffd700 1 -- 172.21.15.55:0/672589055 learned_addr learned my addr 172.21.15.55:0/672589055 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:16.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.002+0000 7fa0537fe700 1 --2- 172.21.15.55:0/672589055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa054159dc0 0x7fa054157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:16.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.002+0000 7fa052ffd700 1 -- 172.21.15.55:0/672589055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa054159dc0 msgr2=0x7fa054157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:16.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.002+0000 7fa052ffd700 1 --2- 172.21.15.55:0/672589055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa054159dc0 0x7fa054157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:16.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.002+0000 7fa052ffd700 1 -- 172.21.15.55:0/672589055 >> v1:172.21.15.163:6789/0 conn(0x7fa054153850 legacy=0x7fa0541596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:16.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.002+0000 7fa052ffd700 1 -- 172.21.15.55:0/672589055 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0540afe20 con 0x7fa054153480 2023-10-06T03:52:16.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.002+0000 7fa0537fe700 1 --2- 172.21.15.55:0/672589055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa054159dc0 0x7fa054157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:16.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.002+0000 7fa052ffd700 1 --2- 172.21.15.55:0/672589055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa054153480 0x7fa0540ae750 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fa0440099b0 tx=0x7fa044009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f159efa449a43cc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:16.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.003+0000 7fa051ffb700 1 -- 172.21.15.55:0/672589055 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04400b860 con 0x7fa054153480 2023-10-06T03:52:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.003+0000 7fa051ffb700 1 -- 172.21.15.55:0/672589055 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04400b9c0 con 0x7fa054153480 2023-10-06T03:52:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.003+0000 7fa051ffb700 1 -- 172.21.15.55:0/672589055 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04400bd60 con 0x7fa054153480 2023-10-06T03:52:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.003+0000 7fa053fff700 1 -- 172.21.15.55:0/672589055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa054153480 msgr2=0x7fa0540ae750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.003+0000 7fa053fff700 1 --2- 172.21.15.55:0/672589055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa054153480 0x7fa0540ae750 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fa0440099b0 tx=0x7fa044009250 comp rx=0 tx=0).stop 2023-10-06T03:52:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.003+0000 7fa053fff700 1 -- 172.21.15.55:0/672589055 shutdown_connections 2023-10-06T03:52:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.003+0000 7fa053fff700 1 --2- 172.21.15.55:0/672589055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa054159dc0 0x7fa054157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.003+0000 7fa053fff700 1 --2- 172.21.15.55:0/672589055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa054153480 0x7fa0540ae750 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:16.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.003+0000 7fa053fff700 1 -- 172.21.15.55:0/672589055 >> 172.21.15.55:0/672589055 conn(0x7fa054068e20 msgr2=0x7fa0540cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:16.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.003+0000 7fa053fff700 1 -- 172.21.15.55:0/672589055 shutdown_connections 2023-10-06T03:52:16.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.003+0000 7fa053fff700 1 -- 172.21.15.55:0/672589055 wait complete. 2023-10-06T03:52:16.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.005+0000 7fa053fff700 1 Processor -- start 2023-10-06T03:52:16.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.005+0000 7fa053fff700 1 -- start start 2023-10-06T03:52:16.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa053fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa054153480 0x7fa054144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:16.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa053fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa054153850 0x7fa0541561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:16.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa053fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa054159dc0 0x7fa0541566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:16.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa053fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0541451b0 con 0x7fa054153480 2023-10-06T03:52:16.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa053fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540c8830 con 0x7fa054159dc0 2023-10-06T03:52:16.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa053fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540c8b10 con 0x7fa054153850 2023-10-06T03:52:16.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa052ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa054153480 0x7fa054144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:16.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa052ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa054153480 0x7fa054144c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51302/0 (socket says 172.21.15.55:51302) 2023-10-06T03:52:16.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa052ffd700 1 -- 172.21.15.55:0/2355611930 learned_addr learned my addr 172.21.15.55:0/2355611930 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:16.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa052ffd700 1 -- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa054153850 msgr2=0x7fa0541561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:16.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa0527fc700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa054153850 0x7fa0541561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:16.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa0537fe700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa054159dc0 0x7fa0541566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:16.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa052ffd700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa054153850 0x7fa0541561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa052ffd700 1 -- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa054159dc0 msgr2=0x7fa0541566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa052ffd700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa054159dc0 0x7fa0541566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa052ffd700 1 -- 172.21.15.55:0/2355611930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0440094f0 con 0x7fa054153480 2023-10-06T03:52:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa0537fe700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa054159dc0 0x7fa0541566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.006+0000 7fa0527fc700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa054153850 0x7fa0541561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:16.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.007+0000 7fa052ffd700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa054153480 0x7fa054144c70 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fa044009290 tx=0x7fa044017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:16.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.007+0000 7fa03affd700 1 -- 172.21.15.55:0/2355611930 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044024440 con 0x7fa054153480 2023-10-06T03:52:16.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.007+0000 7fa03affd700 1 -- 172.21.15.55:0/2355611930 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0440245a0 con 0x7fa054153480 2023-10-06T03:52:16.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.007+0000 7fa053fff700 1 -- 172.21.15.55:0/2355611930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0540c8df0 con 0x7fa054153480 2023-10-06T03:52:16.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.007+0000 7fa03affd700 1 -- 172.21.15.55:0/2355611930 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044024960 con 0x7fa054153480 2023-10-06T03:52:16.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.007+0000 7fa053fff700 1 -- 172.21.15.55:0/2355611930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0540c93b0 con 0x7fa054153480 2023-10-06T03:52:16.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.008+0000 7fa053fff700 1 -- 172.21.15.55:0/2355611930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa05404c3c0 con 0x7fa054153480 2023-10-06T03:52:16.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.009+0000 7fa03affd700 1 -- 172.21.15.55:0/2355611930 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fa04401a420 con 0x7fa054153480 2023-10-06T03:52:16.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.010+0000 7fa03affd700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa040067050 0x7fa040069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:16.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.010+0000 7fa03affd700 1 -- 172.21.15.55:0/2355611930 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6728+0+0 (secure 0 0 0) 0x7fa044086ab0 con 0x7fa054153480 2023-10-06T03:52:16.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.010+0000 7fa0527fc700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa040067050 0x7fa040069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:16.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.011+0000 7fa0527fc700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa040067050 0x7fa040069500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa0540cd120 tx=0x7fa048009040 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:16.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.014+0000 7fa03affd700 1 -- 172.21.15.55:0/2355611930 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa044053770 con 0x7fa054153480 2023-10-06T03:52:16.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:16.216+0000 7fa053fff700 1 -- 172.21.15.55:0/2355611930 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa0540b0100 con 0x7fa054153480 2023-10-06T03:52:16.481 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:52:16.616 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:52:17.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.041+0000 7fa03affd700 1 -- 172.21.15.55:0/2355611930 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v128) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa044056d90 con 0x7fa054153480 2023-10-06T03:52:17.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.045+0000 7fa053fff700 1 -- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa040067050 msgr2=0x7fa040069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:17.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.045+0000 7fa053fff700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa040067050 0x7fa040069500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fa0540cd120 tx=0x7fa048009040 comp rx=0 tx=0).stop 2023-10-06T03:52:17.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.045+0000 7fa053fff700 1 -- 172.21.15.55:0/2355611930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa054153480 msgr2=0x7fa054144c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:17.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.045+0000 7fa053fff700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa054153480 0x7fa054144c70 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fa044009290 tx=0x7fa044017000 comp rx=0 tx=0).stop 2023-10-06T03:52:17.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.045+0000 7fa053fff700 1 -- 172.21.15.55:0/2355611930 shutdown_connections 2023-10-06T03:52:17.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.045+0000 7fa053fff700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa040067050 0x7fa040069500 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.045+0000 7fa053fff700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa054159dc0 0x7fa0541566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.045+0000 7fa053fff700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa054153480 0x7fa054144c70 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.045+0000 7fa053fff700 1 --2- 172.21.15.55:0/2355611930 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa054153850 0x7fa0541561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.045+0000 7fa053fff700 1 -- 172.21.15.55:0/2355611930 >> 172.21.15.55:0/2355611930 conn(0x7fa054068e20 msgr2=0x7fa054158740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:17.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.045+0000 7fa053fff700 1 -- 172.21.15.55:0/2355611930 shutdown_connections 2023-10-06T03:52:17.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.045+0000 7fa053fff700 1 -- 172.21.15.55:0/2355611930 wait complete. 2023-10-06T03:52:17.061 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2023-10-06T03:52:17.067 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T03:52:17.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.255+0000 7fcc6bfff700 1 Processor -- start 2023-10-06T03:52:17.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.255+0000 7fcc6bfff700 1 -- start start 2023-10-06T03:52:17.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.256+0000 7fcc6bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcc6c0b4e70 0x7fcc6c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:17.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.256+0000 7fcc6bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcc6c0b1b40 0x7fcc6c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:17.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.256+0000 7fcc6bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6c0cfb00 con 0x7fcc6c0b16d0 2023-10-06T03:52:17.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.256+0000 7fcc6bfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6c0b5c40 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.256+0000 7fcc6bfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6c0b5f20 con 0x7fcc6c0b1b40 2023-10-06T03:52:17.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.256+0000 7fcc6affd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcc6c0b4e70 0x7fcc6c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:17.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.257+0000 7fcc6affd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcc6c0b4e70 0x7fcc6c0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51318/0 (socket says 172.21.15.55:51318) 2023-10-06T03:52:17.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.257+0000 7fcc6affd700 1 -- 172.21.15.55:0/228445666 learned_addr learned my addr 172.21.15.55:0/228445666 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:17.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.257+0000 7fcc6a7fc700 1 --1- 172.21.15.55:0/228445666 >> v1:172.21.15.55:6789/0 conn(0x7fcc6c0b16d0 0x7fcc6c15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39248/0 (socket says 172.21.15.55:39248) 2023-10-06T03:52:17.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.257+0000 7fcc6b7fe700 1 --2- 172.21.15.55:0/228445666 >> v2:172.21.15.163:3300/0 conn(0x7fcc6c0b1b40 0x7fcc6c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:17.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.257+0000 7fcc6affd700 1 -- 172.21.15.55:0/228445666 >> v2:172.21.15.163:3300/0 conn(0x7fcc6c0b1b40 msgr2=0x7fcc6c15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:17.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.257+0000 7fcc6affd700 1 --2- 172.21.15.55:0/228445666 >> v2:172.21.15.163:3300/0 conn(0x7fcc6c0b1b40 0x7fcc6c15d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.257+0000 7fcc6affd700 1 -- 172.21.15.55:0/228445666 >> v1:172.21.15.55:6789/0 conn(0x7fcc6c0b16d0 legacy=0x7fcc6c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:17.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.257+0000 7fcc6affd700 1 -- 172.21.15.55:0/228445666 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c0b6200 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.257+0000 7fcc6b7fe700 1 --2- 172.21.15.55:0/228445666 >> v2:172.21.15.163:3300/0 conn(0x7fcc6c0b1b40 0x7fcc6c15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:17.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.257+0000 7fcc6affd700 1 --2- 172.21.15.55:0/228445666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc6c0b4e70 0x7fcc6c0ccff0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c071380 tx=0x7fcc5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7566201b32223d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:17.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.257+0000 7fcc69ffb700 1 -- 172.21.15.55:0/228445666 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c00b860 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.258+0000 7fcc69ffb700 1 -- 172.21.15.55:0/228445666 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c00b9c0 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.258+0000 7fcc69ffb700 1 -- 172.21.15.55:0/228445666 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c00bd60 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.258+0000 7fcc6bfff700 1 -- 172.21.15.55:0/228445666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc6c0b4e70 msgr2=0x7fcc6c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:17.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.258+0000 7fcc6bfff700 1 --2- 172.21.15.55:0/228445666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc6c0b4e70 0x7fcc6c0ccff0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c071380 tx=0x7fcc5c009250 comp rx=0 tx=0).stop 2023-10-06T03:52:17.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.258+0000 7fcc6bfff700 1 -- 172.21.15.55:0/228445666 shutdown_connections 2023-10-06T03:52:17.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.258+0000 7fcc6bfff700 1 --2- 172.21.15.55:0/228445666 >> v2:172.21.15.163:3300/0 conn(0x7fcc6c0b1b40 0x7fcc6c15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.258+0000 7fcc6bfff700 1 --2- 172.21.15.55:0/228445666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc6c0b4e70 0x7fcc6c0ccff0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.258+0000 7fcc6bfff700 1 -- 172.21.15.55:0/228445666 >> 172.21.15.55:0/228445666 conn(0x7fcc6c0d10c0 msgr2=0x7fcc6c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.258+0000 7fcc6bfff700 1 -- 172.21.15.55:0/228445666 shutdown_connections 2023-10-06T03:52:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.258+0000 7fcc6bfff700 1 -- 172.21.15.55:0/228445666 wait complete. 2023-10-06T03:52:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.260+0000 7fcc6bfff700 1 Processor -- start 2023-10-06T03:52:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.260+0000 7fcc6bfff700 1 -- start start 2023-10-06T03:52:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.261+0000 7fcc6bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc6c0b16d0 0x7fcc6c0aec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:17.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.261+0000 7fcc6bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc6c0b1b40 0x7fcc6c0c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:17.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.261+0000 7fcc6bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc6c0b4e70 0x7fcc6c169e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:17.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.261+0000 7fcc6bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c0c9a20 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.261+0000 7fcc6bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c0c6490 con 0x7fcc6c0b1b40 2023-10-06T03:52:17.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.261+0000 7fcc6bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c0c6770 con 0x7fcc6c0b16d0 2023-10-06T03:52:17.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.261+0000 7fcc6b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc6c0b4e70 0x7fcc6c169e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:17.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.261+0000 7fcc6b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc6c0b4e70 0x7fcc6c169e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51332/0 (socket says 172.21.15.55:51332) 2023-10-06T03:52:17.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.261+0000 7fcc6b7fe700 1 -- 172.21.15.55:0/2527587601 learned_addr learned my addr 172.21.15.55:0/2527587601 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:17.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.261+0000 7fcc6a7fc700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc6c0b1b40 0x7fcc6c0c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:17.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.261+0000 7fcc6affd700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc6c0b16d0 0x7fcc6c0aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:17.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.262+0000 7fcc6b7fe700 1 -- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc6c0b16d0 msgr2=0x7fcc6c0aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:17.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.262+0000 7fcc6b7fe700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc6c0b16d0 0x7fcc6c0aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.262+0000 7fcc6b7fe700 1 -- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc6c0b1b40 msgr2=0x7fcc6c0c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:17.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.262+0000 7fcc6b7fe700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc6c0b1b40 0x7fcc6c0c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.262+0000 7fcc6b7fe700 1 -- 172.21.15.55:0/2527587601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c0094f0 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.262+0000 7fcc6affd700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc6c0b16d0 0x7fcc6c0aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:17.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.262+0000 7fcc6b7fe700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc6c0b4e70 0x7fcc6c169e40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400d180 tx=0x7fcc64009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:17.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.262+0000 7fcc52ffd700 1 -- 172.21.15.55:0/2527587601 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6400dd70 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.262+0000 7fcc52ffd700 1 -- 172.21.15.55:0/2527587601 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6400f040 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.262+0000 7fcc52ffd700 1 -- 172.21.15.55:0/2527587601 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64015690 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.262+0000 7fcc6bfff700 1 -- 172.21.15.55:0/2527587601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6c0c6a50 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.262+0000 7fcc6bfff700 1 -- 172.21.15.55:0/2527587601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6c0af1b0 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.263+0000 7fcc6bfff700 1 -- 172.21.15.55:0/2527587601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6c0b2a20 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.264+0000 7fcc52ffd700 1 -- 172.21.15.55:0/2527587601 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fcc6401e3b0 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.265+0000 7fcc52ffd700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcc58067050 0x7fcc58069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:17.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.265+0000 7fcc52ffd700 1 -- 172.21.15.55:0/2527587601 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6286+0+0 (secure 0 0 0) 0x7fcc64084fc0 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.266+0000 7fcc6affd700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcc58067050 0x7fcc58069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:17.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.267+0000 7fcc6affd700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcc58067050 0x7fcc58069500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c071380 tx=0x7fcc5c017000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:17.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.269+0000 7fcc52ffd700 1 -- 172.21.15.55:0/2527587601 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc64051d30 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.473+0000 7fcc6bfff700 1 -- 172.21.15.55:0/2527587601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcc6c04bb80 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.477+0000 7fcc52ffd700 1 -- 172.21.15.55:0/2527587601 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcc64055350 con 0x7fcc6c0b4e70 2023-10-06T03:52:17.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.479+0000 7fcc6bfff700 1 -- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcc58067050 msgr2=0x7fcc58069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:17.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.479+0000 7fcc6bfff700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcc58067050 0x7fcc58069500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c071380 tx=0x7fcc5c017000 comp rx=0 tx=0).stop 2023-10-06T03:52:17.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.479+0000 7fcc6bfff700 1 -- 172.21.15.55:0/2527587601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc6c0b4e70 msgr2=0x7fcc6c169e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:17.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.479+0000 7fcc6bfff700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc6c0b4e70 0x7fcc6c169e40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400d180 tx=0x7fcc64009a50 comp rx=0 tx=0).stop 2023-10-06T03:52:17.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.479+0000 7fcc6bfff700 1 -- 172.21.15.55:0/2527587601 shutdown_connections 2023-10-06T03:52:17.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.479+0000 7fcc6bfff700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcc58067050 0x7fcc58069500 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.479+0000 7fcc6bfff700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcc6c0b4e70 0x7fcc6c169e40 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.479+0000 7fcc6bfff700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcc6c0b16d0 0x7fcc6c0aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.479+0000 7fcc6bfff700 1 --2- 172.21.15.55:0/2527587601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcc6c0b1b40 0x7fcc6c0c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.479+0000 7fcc6bfff700 1 -- 172.21.15.55:0/2527587601 >> 172.21.15.55:0/2527587601 conn(0x7fcc6c0d10c0 msgr2=0x7fcc6c15ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:17.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.479+0000 7fcc6bfff700 1 -- 172.21.15.55:0/2527587601 shutdown_connections 2023-10-06T03:52:17.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.479+0000 7fcc6bfff700 1 -- 172.21.15.55:0/2527587601 wait complete. 2023-10-06T03:52:17.503 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T03:52:17.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.690+0000 7f5bb899d700 1 Processor -- start 2023-10-06T03:52:17.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.690+0000 7f5bb899d700 1 -- start start 2023-10-06T03:52:17.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.691+0000 7f5bb899d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5bb40cb1a0 0x7f5bb415a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:17.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.691+0000 7f5bb899d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb406ebd0 con 0x7f5bb415a8a0 2023-10-06T03:52:17.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.692+0000 7f5bb899d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bb415ad20 con 0x7f5bb40ae0d0 2023-10-06T03:52:17.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.692+0000 7f5bb899d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb415b2e0 con 0x7f5bb40cb1a0 2023-10-06T03:52:17.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.692+0000 7f5bb2ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5bb415a8a0 0x7f5bb406f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39256/0 (socket says 172.21.15.55:39256) 2023-10-06T03:52:17.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.692+0000 7f5bb2ffd700 1 -- 172.21.15.55:0/3866039118 learned_addr learned my addr 172.21.15.55:0/3866039118 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:17.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.692+0000 7f5bb37fe700 1 --2- 172.21.15.55:0/3866039118 >> v2:172.21.15.163:3300/0 conn(0x7f5bb40cb1a0 0x7f5bb415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:17.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.692+0000 7f5bb27fc700 1 -- 172.21.15.55:0/3866039118 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456294056 0 0) 0x7f5bb406ebd0 con 0x7f5bb415a8a0 2023-10-06T03:52:17.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.692+0000 7f5bb27fc700 1 -- 172.21.15.55:0/3866039118 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b9c003570 con 0x7f5bb415a8a0 2023-10-06T03:52:17.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.692+0000 7f5bb27fc700 1 -- 172.21.15.55:0/3866039118 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2203243658 0 0) 0x7f5b9c003570 con 0x7f5bb415a8a0 2023-10-06T03:52:17.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.692+0000 7f5bb27fc700 1 -- 172.21.15.55:0/3866039118 >> v2:172.21.15.163:3300/0 conn(0x7f5bb40cb1a0 msgr2=0x7f5bb415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:17.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.692+0000 7f5bb27fc700 1 --2- 172.21.15.55:0/3866039118 >> v2:172.21.15.163:3300/0 conn(0x7f5bb40cb1a0 0x7f5bb415a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.692+0000 7f5bb27fc700 1 -- 172.21.15.55:0/3866039118 >> v1:172.21.15.163:6789/0 conn(0x7f5bb40ae0d0 legacy=0x7f5bb40ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:17.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.692+0000 7f5bb27fc700 1 -- 172.21.15.55:0/3866039118 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb415b000 con 0x7f5bb415a8a0 2023-10-06T03:52:17.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.692+0000 7f5bb27fc700 1 -- 172.21.15.55:0/3866039118 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f5ba8001470 con 0x7f5bb415a8a0 2023-10-06T03:52:17.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.692+0000 7f5bb37fe700 1 --2- 172.21.15.55:0/3866039118 >> v2:172.21.15.163:3300/0 conn(0x7f5bb40cb1a0 0x7f5bb415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:52:17.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb27fc700 1 -- 172.21.15.55:0/3866039118 >> v1:172.21.15.55:6789/0 conn(0x7f5bb415a8a0 legacy=0x7f5bb406f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:17.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb27fc700 1 --2- 172.21.15.55:0/3866039118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b9c003a60 0x7f5b9c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:17.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb27fc700 1 --2- 172.21.15.55:0/3866039118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b9c004810 0x7f5b9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:17.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb27fc700 1 --2- 172.21.15.55:0/3866039118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9c0051c0 0x7f5b9c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:17.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb27fc700 1 -- 172.21.15.55:0/3866039118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb415b2e0 con 0x7f5b9c004810 2023-10-06T03:52:17.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb27fc700 1 -- 172.21.15.55:0/3866039118 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c00c0d0 con 0x7f5b9c003a60 2023-10-06T03:52:17.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb27fc700 1 -- 172.21.15.55:0/3866039118 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c00c290 con 0x7f5b9c0051c0 2023-10-06T03:52:17.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb2ffd700 1 --2- 172.21.15.55:0/3866039118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b9c004810 0x7f5b9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:17.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb37fe700 1 --2- 172.21.15.55:0/3866039118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b9c003a60 0x7f5b9c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:17.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb3fff700 1 --2- 172.21.15.55:0/3866039118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9c0051c0 0x7f5b9c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:17.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb2ffd700 1 -- 172.21.15.55:0/3866039118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9c0051c0 msgr2=0x7f5b9c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:17.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb2ffd700 1 --2- 172.21.15.55:0/3866039118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9c0051c0 0x7f5b9c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb2ffd700 1 -- 172.21.15.55:0/3866039118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b9c003a60 msgr2=0x7f5b9c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:17.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb2ffd700 1 --2- 172.21.15.55:0/3866039118 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5b9c003a60 0x7f5b9c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb2ffd700 1 -- 172.21.15.55:0/3866039118 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c003570 con 0x7f5b9c004810 2023-10-06T03:52:17.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb3fff700 1 --2- 172.21.15.55:0/3866039118 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9c0051c0 0x7f5b9c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:17.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.693+0000 7f5bb3fff700 1 -- 172.21.15.55:0/3866039118 reap_dead start 2023-10-06T03:52:17.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.694+0000 7f5bb2ffd700 1 --2- 172.21.15.55:0/3866039118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b9c004810 0x7f5b9c004c80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800c390 tx=0x7f5bb40c8be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7986a5a307b00f04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:17.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.694+0000 7f5bb27fc700 1 -- 172.21.15.55:0/3866039118 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba800cc60 con 0x7f5b9c004810 2023-10-06T03:52:17.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.694+0000 7f5bb27fc700 1 -- 172.21.15.55:0/3866039118 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba800cdc0 con 0x7f5b9c004810 2023-10-06T03:52:17.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.694+0000 7f5bb27fc700 1 -- 172.21.15.55:0/3866039118 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8005b70 con 0x7f5b9c004810 2023-10-06T03:52:17.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.694+0000 7f5bb899d700 1 -- 172.21.15.55:0/3866039118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b9c004810 msgr2=0x7f5b9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:17.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.694+0000 7f5bb899d700 1 --2- 172.21.15.55:0/3866039118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b9c004810 0x7f5b9c004c80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800c390 tx=0x7f5bb40c8be0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.694+0000 7f5bb899d700 1 -- 172.21.15.55:0/3866039118 shutdown_connections 2023-10-06T03:52:17.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.694+0000 7f5bb899d700 1 --2- 172.21.15.55:0/3866039118 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b9c004810 0x7f5b9c004c80 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.694+0000 7f5bb899d700 1 -- 172.21.15.55:0/3866039118 >> 172.21.15.55:0/3866039118 conn(0x7f5bb40cc1e0 msgr2=0x7f5bb40cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:17.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.695+0000 7f5bb899d700 1 -- 172.21.15.55:0/3866039118 shutdown_connections 2023-10-06T03:52:17.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.695+0000 7f5bb899d700 1 -- 172.21.15.55:0/3866039118 wait complete. 2023-10-06T03:52:17.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.696+0000 7f5bb899d700 1 Processor -- start 2023-10-06T03:52:17.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.696+0000 7f5bb899d700 1 -- start start 2023-10-06T03:52:17.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.697+0000 7f5bb899d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9c004810 0x7f5bb41598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:17.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.697+0000 7f5bb899d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bb413b9b0 0x7f5bb4157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:17.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.697+0000 7f5bb899d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb41582a0 0x7f5bb406a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:17.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.697+0000 7f5bb899d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb4159430 con 0x7f5bb413b9b0 2023-10-06T03:52:17.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.697+0000 7f5bb899d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb40c21e0 con 0x7f5bb41582a0 2023-10-06T03:52:17.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.697+0000 7f5bb899d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb4158820 con 0x7f5b9c004810 2023-10-06T03:52:17.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.697+0000 7f5bb2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bb413b9b0 0x7f5bb4157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:17.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.697+0000 7f5bb2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bb413b9b0 0x7f5bb4157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51346/0 (socket says 172.21.15.55:51346) 2023-10-06T03:52:17.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.697+0000 7f5bb2ffd700 1 -- 172.21.15.55:0/2331640749 learned_addr learned my addr 172.21.15.55:0/2331640749 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:17.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.697+0000 7f5bb37fe700 1 --2- 172.21.15.55:0/2331640749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9c004810 0x7f5bb41598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:17.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.697+0000 7f5bb3fff700 1 --2- 172.21.15.55:0/2331640749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb41582a0 0x7f5bb406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:17.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.698+0000 7f5bb2ffd700 1 -- 172.21.15.55:0/2331640749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9c004810 msgr2=0x7f5bb41598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:17.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.698+0000 7f5bb2ffd700 1 --2- 172.21.15.55:0/2331640749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9c004810 0x7f5bb41598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.698+0000 7f5bb2ffd700 1 -- 172.21.15.55:0/2331640749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb41582a0 msgr2=0x7f5bb406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:17.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.698+0000 7f5bb2ffd700 1 --2- 172.21.15.55:0/2331640749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb41582a0 0x7f5bb406a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:17.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.698+0000 7f5bb2ffd700 1 -- 172.21.15.55:0/2331640749 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb4158b00 con 0x7f5bb413b9b0 2023-10-06T03:52:17.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.698+0000 7f5bb2ffd700 1 --2- 172.21.15.55:0/2331640749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bb413b9b0 0x7f5bb4157d60 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00c0a0 tx=0x7f5ba8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:17.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.698+0000 7f5b9b7fe700 1 -- 172.21.15.55:0/2331640749 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba800cc60 con 0x7f5bb413b9b0 2023-10-06T03:52:17.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.698+0000 7f5b9b7fe700 1 -- 172.21.15.55:0/2331640749 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba8001640 con 0x7f5bb413b9b0 2023-10-06T03:52:17.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.698+0000 7f5bb899d700 1 -- 172.21.15.55:0/2331640749 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb4158de0 con 0x7f5bb413b9b0 2023-10-06T03:52:17.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.698+0000 7f5b9b7fe700 1 -- 172.21.15.55:0/2331640749 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8007300 con 0x7f5bb413b9b0 2023-10-06T03:52:17.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.699+0000 7f5bb899d700 1 -- 172.21.15.55:0/2331640749 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb413be20 con 0x7f5bb413b9b0 2023-10-06T03:52:17.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.699+0000 7f5bb899d700 1 -- 172.21.15.55:0/2331640749 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bb406c450 con 0x7f5bb413b9b0 2023-10-06T03:52:17.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.700+0000 7f5b9b7fe700 1 -- 172.21.15.55:0/2331640749 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f5ba80076b0 con 0x7f5bb413b9b0 2023-10-06T03:52:17.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.702+0000 7f5b9b7fe700 1 --2- 172.21.15.55:0/2331640749 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5ba0067050 0x7f5ba0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:17.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.702+0000 7f5b9b7fe700 1 -- 172.21.15.55:0/2331640749 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6286+0+0 (secure 0 0 0) 0x7f5ba80853b0 con 0x7f5bb413b9b0 2023-10-06T03:52:17.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.702+0000 7f5bb37fe700 1 --2- 172.21.15.55:0/2331640749 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5ba0067050 0x7f5ba0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:17.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.703+0000 7f5bb37fe700 1 --2- 172.21.15.55:0/2331640749 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5ba0067050 0x7f5ba0069500 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4006f00 tx=0x7f5ba40091a0 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:17.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.706+0000 7f5b9b7fe700 1 -- 172.21.15.55:0/2331640749 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ba80522d0 con 0x7f5bb413b9b0 2023-10-06T03:52:17.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.908+0000 7f5bb899d700 1 -- 172.21.15.55:0/2331640749 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f5bb406e860 con 0x7f5bb413b9b0 2023-10-06T03:52:17.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.917+0000 7f5b9b7fe700 1 -- 172.21.15.55:0/2331640749 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v44) v1 ==== 115+0+0 (secure 0 0 0) 0x7f5ba80558f0 con 0x7f5bb413b9b0 2023-10-06T03:52:17.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.920+0000 7f5bb899d700 1 -- 172.21.15.55:0/2331640749 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5ba0067050 msgr2=0x7f5ba0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:18.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.920+0000 7f5bb899d700 1 --2- 172.21.15.55:0/2331640749 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5ba0067050 0x7f5ba0069500 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4006f00 tx=0x7f5ba40091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.920+0000 7f5bb899d700 1 -- 172.21.15.55:0/2331640749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bb413b9b0 msgr2=0x7f5bb4157d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:18.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.920+0000 7f5bb899d700 1 --2- 172.21.15.55:0/2331640749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bb413b9b0 0x7f5bb4157d60 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00c0a0 tx=0x7f5ba8015000 comp rx=0 tx=0).stop 2023-10-06T03:52:18.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.920+0000 7f5bb899d700 1 -- 172.21.15.55:0/2331640749 shutdown_connections 2023-10-06T03:52:18.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.920+0000 7f5bb899d700 1 --2- 172.21.15.55:0/2331640749 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5ba0067050 0x7f5ba0069500 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.920+0000 7f5bb899d700 1 --2- 172.21.15.55:0/2331640749 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5bb41582a0 0x7f5bb406a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.920+0000 7f5bb899d700 1 --2- 172.21.15.55:0/2331640749 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b9c004810 0x7f5bb41598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.920+0000 7f5bb899d700 1 --2- 172.21.15.55:0/2331640749 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5bb413b9b0 0x7f5bb4157d60 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.920+0000 7f5bb899d700 1 -- 172.21.15.55:0/2331640749 >> 172.21.15.55:0/2331640749 conn(0x7f5bb40cc1e0 msgr2=0x7f5bb40ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:18.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.921+0000 7f5bb899d700 1 -- 172.21.15.55:0/2331640749 shutdown_connections 2023-10-06T03:52:18.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:17.921+0000 7f5bb899d700 1 -- 172.21.15.55:0/2331640749 wait complete. 2023-10-06T03:52:18.215 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T03:52:18.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.406+0000 7f543a99d700 1 Processor -- start 2023-10-06T03:52:18.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.406+0000 7f543a99d700 1 -- start start 2023-10-06T03:52:18.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.408+0000 7f543a99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f543415a3b0 0x7f543415a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:18.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.408+0000 7f543a99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f54340aef00 0x7f543415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:18.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.408+0000 7f543a99d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54340aea70 con 0x7f543406e470 2023-10-06T03:52:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.408+0000 7f543a99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54340ae480 con 0x7f54340aef00 2023-10-06T03:52:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.408+0000 7f543a99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543415acc0 con 0x7f543415a3b0 2023-10-06T03:52:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.408+0000 7f543919a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f54340aef00 0x7f543415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.408+0000 7f543a19c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f543406e470 0x7f54340c7280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39258/0 (socket says 172.21.15.55:39258) 2023-10-06T03:52:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.408+0000 7f543919a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f54340aef00 0x7f543415d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51356/0 (socket says 172.21.15.55:51356) 2023-10-06T03:52:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.408+0000 7f543a19c700 1 -- 172.21.15.55:0/1369847201 learned_addr learned my addr 172.21.15.55:0/1369847201 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.408+0000 7f543999b700 1 --2- 172.21.15.55:0/1369847201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f543415a3b0 0x7f543415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.408+0000 7f543919a700 1 -- 172.21.15.55:0/1369847201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f543415a3b0 msgr2=0x7f543415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:18.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.408+0000 7f543919a700 1 --2- 172.21.15.55:0/1369847201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f543415a3b0 0x7f543415a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.409+0000 7f543919a700 1 -- 172.21.15.55:0/1369847201 >> v1:172.21.15.55:6789/0 conn(0x7f543406e470 legacy=0x7f54340c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:18.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.409+0000 7f543919a700 1 -- 172.21.15.55:0/1369847201 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543415dd30 con 0x7f54340aef00 2023-10-06T03:52:18.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.409+0000 7f543999b700 1 --2- 172.21.15.55:0/1369847201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f543415a3b0 0x7f543415a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:18.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.409+0000 7f543919a700 1 --2- 172.21.15.55:0/1369847201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54340aef00 0x7f543415d3c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f54240098e0 tx=0x7f54240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f762b0d50322564 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:18.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.409+0000 7f5438999700 1 -- 172.21.15.55:0/1369847201 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424009e80 con 0x7f54340aef00 2023-10-06T03:52:18.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.409+0000 7f5438999700 1 -- 172.21.15.55:0/1369847201 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542400b760 con 0x7f54340aef00 2023-10-06T03:52:18.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.409+0000 7f5438999700 1 -- 172.21.15.55:0/1369847201 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400bae0 con 0x7f54340aef00 2023-10-06T03:52:18.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.410+0000 7f543a99d700 1 -- 172.21.15.55:0/1369847201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54340aef00 msgr2=0x7f543415d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.410+0000 7f543a99d700 1 --2- 172.21.15.55:0/1369847201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54340aef00 0x7f543415d3c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f54240098e0 tx=0x7f54240091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.410+0000 7f543a99d700 1 -- 172.21.15.55:0/1369847201 shutdown_connections 2023-10-06T03:52:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.410+0000 7f543a99d700 1 --2- 172.21.15.55:0/1369847201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f543415a3b0 0x7f543415a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.410+0000 7f543a99d700 1 --2- 172.21.15.55:0/1369847201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f54340aef00 0x7f543415d3c0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.410+0000 7f543a99d700 1 -- 172.21.15.55:0/1369847201 >> 172.21.15.55:0/1369847201 conn(0x7f54340c9f70 msgr2=0x7f543406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.410+0000 7f543a99d700 1 -- 172.21.15.55:0/1369847201 shutdown_connections 2023-10-06T03:52:18.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.410+0000 7f543a99d700 1 -- 172.21.15.55:0/1369847201 wait complete. 2023-10-06T03:52:18.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.411+0000 7f543a99d700 1 Processor -- start 2023-10-06T03:52:18.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.411+0000 7f543a99d700 1 -- start start 2023-10-06T03:52:18.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.412+0000 7f543a99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f543406e470 0x7f54340b10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:18.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.412+0000 7f543a99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54340aef00 0x7f543416ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:18.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.412+0000 7f543a99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f543415a3b0 0x7f5434156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:18.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.412+0000 7f543a99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5434152e80 con 0x7f543406e470 2023-10-06T03:52:18.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.412+0000 7f543a99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54340b1600 con 0x7f543415a3b0 2023-10-06T03:52:18.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.412+0000 7f543a99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543416b150 con 0x7f54340aef00 2023-10-06T03:52:18.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.413+0000 7f543999b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f543406e470 0x7f54340b10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:18.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.413+0000 7f543999b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f543406e470 0x7f54340b10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51372/0 (socket says 172.21.15.55:51372) 2023-10-06T03:52:18.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.413+0000 7f543999b700 1 -- 172.21.15.55:0/864906505 learned_addr learned my addr 172.21.15.55:0/864906505 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:18.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.413+0000 7f543919a700 1 --2- 172.21.15.55:0/864906505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54340aef00 0x7f543416ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:18.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.413+0000 7f543a19c700 1 --2- 172.21.15.55:0/864906505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f543415a3b0 0x7f5434156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:18.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.413+0000 7f543999b700 1 -- 172.21.15.55:0/864906505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54340aef00 msgr2=0x7f543416ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:18.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.413+0000 7f543999b700 1 --2- 172.21.15.55:0/864906505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54340aef00 0x7f543416ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.413+0000 7f543999b700 1 -- 172.21.15.55:0/864906505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f543415a3b0 msgr2=0x7f5434156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:18.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.413+0000 7f543999b700 1 --2- 172.21.15.55:0/864906505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f543415a3b0 0x7f5434156000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.413+0000 7f543999b700 1 -- 172.21.15.55:0/864906505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5424009420 con 0x7f543406e470 2023-10-06T03:52:18.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.413+0000 7f543999b700 1 --2- 172.21.15.55:0/864906505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f543406e470 0x7f54340b10c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f543000c880 tx=0x7f5430008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:18.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.414+0000 7f5429ffb700 1 -- 172.21.15.55:0/864906505 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430011600 con 0x7f543406e470 2023-10-06T03:52:18.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.414+0000 7f5429ffb700 1 -- 172.21.15.55:0/864906505 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5430011760 con 0x7f543406e470 2023-10-06T03:52:18.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.414+0000 7f543a99d700 1 -- 172.21.15.55:0/864906505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54341543d0 con 0x7f543406e470 2023-10-06T03:52:18.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.414+0000 7f5429ffb700 1 -- 172.21.15.55:0/864906505 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430011b00 con 0x7f543406e470 2023-10-06T03:52:18.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.414+0000 7f543a99d700 1 -- 172.21.15.55:0/864906505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5434154990 con 0x7f543406e470 2023-10-06T03:52:18.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.415+0000 7f543a99d700 1 -- 172.21.15.55:0/864906505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5434158410 con 0x7f543406e470 2023-10-06T03:52:18.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.416+0000 7f5429ffb700 1 -- 172.21.15.55:0/864906505 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f5430018350 con 0x7f543406e470 2023-10-06T03:52:18.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.416+0000 7f5429ffb700 1 --2- 172.21.15.55:0/864906505 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5418067050 0x7f5418069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:18.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.417+0000 7f5429ffb700 1 -- 172.21.15.55:0/864906505 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6286+0+0 (secure 0 0 0) 0x7f5430084850 con 0x7f543406e470 2023-10-06T03:52:18.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.417+0000 7f543919a700 1 --2- 172.21.15.55:0/864906505 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5418067050 0x7f5418069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:18.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.418+0000 7f543919a700 1 --2- 172.21.15.55:0/864906505 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5418067050 0x7f5418069500 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f54240093f0 tx=0x7f5424016000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:18.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.421+0000 7f5429ffb700 1 -- 172.21.15.55:0/864906505 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54300515c0 con 0x7f543406e470 2023-10-06T03:52:18.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.620+0000 7f543a99d700 1 -- 172.21.15.55:0/864906505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f54340c3600 con 0x7f543406e470 2023-10-06T03:52:18.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.625+0000 7f5429ffb700 1 -- 172.21.15.55:0/864906505 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v45) v1 ==== 120+0+0 (secure 0 0 0) 0x7f5430054be0 con 0x7f543406e470 2023-10-06T03:52:18.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.627+0000 7f543a99d700 1 -- 172.21.15.55:0/864906505 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5418067050 msgr2=0x7f5418069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:18.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.627+0000 7f543a99d700 1 --2- 172.21.15.55:0/864906505 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5418067050 0x7f5418069500 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f54240093f0 tx=0x7f5424016000 comp rx=0 tx=0).stop 2023-10-06T03:52:18.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.627+0000 7f543a99d700 1 -- 172.21.15.55:0/864906505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f543406e470 msgr2=0x7f54340b10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:18.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.627+0000 7f543a99d700 1 --2- 172.21.15.55:0/864906505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f543406e470 0x7f54340b10c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f543000c880 tx=0x7f5430008040 comp rx=0 tx=0).stop 2023-10-06T03:52:18.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.627+0000 7f543a99d700 1 -- 172.21.15.55:0/864906505 shutdown_connections 2023-10-06T03:52:18.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.627+0000 7f543a99d700 1 --2- 172.21.15.55:0/864906505 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f5418067050 0x7f5418069500 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.627+0000 7f543a99d700 1 --2- 172.21.15.55:0/864906505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f543415a3b0 0x7f5434156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.627+0000 7f543a99d700 1 --2- 172.21.15.55:0/864906505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f543406e470 0x7f54340b10c0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.627+0000 7f543a99d700 1 --2- 172.21.15.55:0/864906505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f54340aef00 0x7f543416ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.627+0000 7f543a99d700 1 -- 172.21.15.55:0/864906505 >> 172.21.15.55:0/864906505 conn(0x7f54340c9f70 msgr2=0x7f543415c120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:18.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.627+0000 7f543a99d700 1 -- 172.21.15.55:0/864906505 shutdown_connections 2023-10-06T03:52:18.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.627+0000 7f543a99d700 1 -- 172.21.15.55:0/864906505 wait complete. 2023-10-06T03:52:18.646 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-06T03:52:18.834 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.833+0000 7f38c5b9d700 1 Processor -- start 2023-10-06T03:52:18.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.833+0000 7f38c5b9d700 1 -- start start 2023-10-06T03:52:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.834+0000 7f38c5b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f38c01597f0 0x7f38c00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.834+0000 7f38c5b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c00b11d0 con 0x7f38c0159420 2023-10-06T03:52:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.834+0000 7f38c5b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c00b14b0 con 0x7f38c015d230 2023-10-06T03:52:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.834+0000 7f38c5b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c00b1790 con 0x7f38c01597f0 2023-10-06T03:52:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.834+0000 7f38c4b9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f38c0159420 0x7f38c00b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39264/0 (socket says 172.21.15.55:39264) 2023-10-06T03:52:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.834+0000 7f38c4b9b700 1 -- 172.21.15.55:0/3997463042 learned_addr learned my addr 172.21.15.55:0/3997463042 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.834+0000 7f38bffff700 1 --2- 172.21.15.55:0/3997463042 >> v2:172.21.15.55:3300/0 conn(0x7f38c01597f0 0x7f38c00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.835+0000 7f38bf7fe700 1 -- 172.21.15.55:0/3997463042 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177818232 0 0) 0x7f38c00b11d0 con 0x7f38c0159420 2023-10-06T03:52:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.835+0000 7f38bf7fe700 1 -- 172.21.15.55:0/3997463042 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38b0003570 con 0x7f38c0159420 2023-10-06T03:52:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.835+0000 7f38bffff700 1 -- 172.21.15.55:0/3997463042 >> v1:172.21.15.163:6789/0 conn(0x7f38c015d230 legacy=0x7f38c015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.835+0000 7f38bffff700 1 -- 172.21.15.55:0/3997463042 >> v1:172.21.15.55:6789/0 conn(0x7f38c0159420 legacy=0x7f38c00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.835+0000 7f38bffff700 1 -- 172.21.15.55:0/3997463042 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c00b1a70 con 0x7f38c01597f0 2023-10-06T03:52:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.835+0000 7f38bffff700 1 --2- 172.21.15.55:0/3997463042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38c01597f0 0x7f38c00b2270 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0098e0 tx=0x7f38ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df9e099682bd60c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.835+0000 7f38bf7fe700 1 -- 172.21.15.55:0/3997463042 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00b640 con 0x7f38c01597f0 2023-10-06T03:52:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.836+0000 7f38bf7fe700 1 -- 172.21.15.55:0/3997463042 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ac00b7a0 con 0x7f38c01597f0 2023-10-06T03:52:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.836+0000 7f38bf7fe700 1 -- 172.21.15.55:0/3997463042 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ac00bb40 con 0x7f38c01597f0 2023-10-06T03:52:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.836+0000 7f38c5b9d700 1 -- 172.21.15.55:0/3997463042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38c01597f0 msgr2=0x7f38c00b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.836+0000 7f38c5b9d700 1 --2- 172.21.15.55:0/3997463042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38c01597f0 0x7f38c00b2270 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0098e0 tx=0x7f38ac0091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.836+0000 7f38c5b9d700 1 -- 172.21.15.55:0/3997463042 shutdown_connections 2023-10-06T03:52:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.836+0000 7f38c5b9d700 1 --2- 172.21.15.55:0/3997463042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38c01597f0 0x7f38c00b2270 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.836+0000 7f38c5b9d700 1 -- 172.21.15.55:0/3997463042 >> 172.21.15.55:0/3997463042 conn(0x7f38c00d0660 msgr2=0x7f38c00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.836+0000 7f38c5b9d700 1 -- 172.21.15.55:0/3997463042 shutdown_connections 2023-10-06T03:52:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.836+0000 7f38c5b9d700 1 -- 172.21.15.55:0/3997463042 wait complete. 2023-10-06T03:52:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.838+0000 7f38c5b9d700 1 Processor -- start 2023-10-06T03:52:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.838+0000 7f38c5b9d700 1 -- start start 2023-10-06T03:52:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c5b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38c0159420 0x7f38c0153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c5b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38c01597f0 0x7f38c016a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c5b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38c015d230 0x7f38c0068ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c5b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c01546c0 con 0x7f38c015d230 2023-10-06T03:52:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c5b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c015bd10 con 0x7f38c01597f0 2023-10-06T03:52:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c5b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c016a610 con 0x7f38c0159420 2023-10-06T03:52:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c539c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38c015d230 0x7f38c0068ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:18.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c539c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38c015d230 0x7f38c0068ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51388/0 (socket says 172.21.15.55:51388) 2023-10-06T03:52:18.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c539c700 1 -- 172.21.15.55:0/975609920 learned_addr learned my addr 172.21.15.55:0/975609920 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:18.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c4b9b700 1 --2- 172.21.15.55:0/975609920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38c0159420 0x7f38c0153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:18.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38bffff700 1 --2- 172.21.15.55:0/975609920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38c01597f0 0x7f38c016a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:18.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c539c700 1 -- 172.21.15.55:0/975609920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38c0159420 msgr2=0x7f38c0153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:18.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c539c700 1 --2- 172.21.15.55:0/975609920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38c0159420 0x7f38c0153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c539c700 1 -- 172.21.15.55:0/975609920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38c01597f0 msgr2=0x7f38c016a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:18.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c539c700 1 --2- 172.21.15.55:0/975609920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38c01597f0 0x7f38c016a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:18.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.839+0000 7f38c539c700 1 -- 172.21.15.55:0/975609920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ac009420 con 0x7f38c015d230 2023-10-06T03:52:18.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.840+0000 7f38c539c700 1 --2- 172.21.15.55:0/975609920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38c015d230 0x7f38c0068ce0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f38b800c810 tx=0x7f38b8001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:18.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.840+0000 7f38bcff9700 1 -- 172.21.15.55:0/975609920 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8012630 con 0x7f38c015d230 2023-10-06T03:52:18.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.840+0000 7f38bcff9700 1 -- 172.21.15.55:0/975609920 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b80127b0 con 0x7f38c015d230 2023-10-06T03:52:18.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.840+0000 7f38bcff9700 1 -- 172.21.15.55:0/975609920 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b8012b70 con 0x7f38c015d230 2023-10-06T03:52:18.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.840+0000 7f38c5b9d700 1 -- 172.21.15.55:0/975609920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c015b100 con 0x7f38c015d230 2023-10-06T03:52:18.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.840+0000 7f38c5b9d700 1 -- 172.21.15.55:0/975609920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c015b6c0 con 0x7f38c015d230 2023-10-06T03:52:18.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.841+0000 7f38c5b9d700 1 -- 172.21.15.55:0/975609920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c004c3c0 con 0x7f38c015d230 2023-10-06T03:52:18.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.842+0000 7f38bcff9700 1 -- 172.21.15.55:0/975609920 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f38b8003030 con 0x7f38c015d230 2023-10-06T03:52:18.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.843+0000 7f38bcff9700 1 --2- 172.21.15.55:0/975609920 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f38a4067050 0x7f38a4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:18.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.843+0000 7f38bcff9700 1 -- 172.21.15.55:0/975609920 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6286+0+0 (secure 0 0 0) 0x7f38b8084920 con 0x7f38c015d230 2023-10-06T03:52:18.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.843+0000 7f38c4b9b700 1 --2- 172.21.15.55:0/975609920 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f38a4067050 0x7f38a4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:18.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.844+0000 7f38c4b9b700 1 --2- 172.21.15.55:0/975609920 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f38a4067050 0x7f38a4069500 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f38c00cda40 tx=0x7f38b4009040 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:18.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:18.847+0000 7f38bcff9700 1 -- 172.21.15.55:0/975609920 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38b80517a0 con 0x7f38c015d230 2023-10-06T03:52:19.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:19.131+0000 7f38c5b9d700 1 -- 172.21.15.55:0/975609920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f38c00b1d50 con 0x7f38c015d230 2023-10-06T03:52:20.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.051+0000 7f38bcff9700 1 -- 172.21.15.55:0/975609920 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v722) v1 ==== 148+0+0 (secure 0 0 0) 0x7f38b8054dc0 con 0x7f38c015d230 2023-10-06T03:52:20.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.055+0000 7f38c5b9d700 1 -- 172.21.15.55:0/975609920 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f38a4067050 msgr2=0x7f38a4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:20.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.055+0000 7f38c5b9d700 1 --2- 172.21.15.55:0/975609920 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f38a4067050 0x7f38a4069500 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f38c00cda40 tx=0x7f38b4009040 comp rx=0 tx=0).stop 2023-10-06T03:52:20.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.055+0000 7f38c5b9d700 1 -- 172.21.15.55:0/975609920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38c015d230 msgr2=0x7f38c0068ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.055+0000 7f38c5b9d700 1 --2- 172.21.15.55:0/975609920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38c015d230 0x7f38c0068ce0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f38b800c810 tx=0x7f38b8001650 comp rx=0 tx=0).stop 2023-10-06T03:52:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.055+0000 7f38c5b9d700 1 -- 172.21.15.55:0/975609920 shutdown_connections 2023-10-06T03:52:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.055+0000 7f38c5b9d700 1 --2- 172.21.15.55:0/975609920 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f38a4067050 0x7f38a4069500 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.055+0000 7f38c5b9d700 1 --2- 172.21.15.55:0/975609920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f38c015d230 0x7f38c0068ce0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.055+0000 7f38c5b9d700 1 --2- 172.21.15.55:0/975609920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f38c0159420 0x7f38c0153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.055+0000 7f38c5b9d700 1 --2- 172.21.15.55:0/975609920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f38c01597f0 0x7f38c016a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.055+0000 7f38c5b9d700 1 -- 172.21.15.55:0/975609920 >> 172.21.15.55:0/975609920 conn(0x7f38c00d0660 msgr2=0x7f38c00b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.055+0000 7f38c5b9d700 1 -- 172.21.15.55:0/975609920 shutdown_connections 2023-10-06T03:52:20.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.055+0000 7f38c5b9d700 1 -- 172.21.15.55:0/975609920 wait complete. 2023-10-06T03:52:20.077 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-10-06T03:52:20.115 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2023-10-06T03:52:20.116 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T03:52:20.116 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-06T03:52:20.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.306+0000 7f9cc8b9d700 1 Processor -- start 2023-10-06T03:52:20.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.307+0000 7f9cc8b9d700 1 -- start start 2023-10-06T03:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.308+0000 7f9cc8b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9cc4150180 0x7f9cc4157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.308+0000 7f9cc8b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9cc40c7410 0x7f9cc4069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.308+0000 7f9cc8b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc406e2a0 con 0x7f9cc4150550 2023-10-06T03:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.308+0000 7f9cc8b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc406b2e0 con 0x7f9cc40c7410 2023-10-06T03:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.308+0000 7f9cc8b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc40c78e0 con 0x7f9cc4150180 2023-10-06T03:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.308+0000 7f9cc3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9cc40c7410 0x7f9cc4069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.308+0000 7f9cc3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9cc40c7410 0x7f9cc4069f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51402/0 (socket says 172.21.15.55:51402) 2023-10-06T03:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.308+0000 7f9cc3fff700 1 -- 172.21.15.55:0/963267388 learned_addr learned my addr 172.21.15.55:0/963267388 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:20.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.308+0000 7f9cc37fe700 1 --2- 172.21.15.55:0/963267388 >> v2:172.21.15.163:3300/0 conn(0x7f9cc4150180 0x7f9cc4157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.309+0000 7f9cc3fff700 1 -- 172.21.15.55:0/963267388 >> v2:172.21.15.163:3300/0 conn(0x7f9cc4150180 msgr2=0x7f9cc4157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.309+0000 7f9cc3fff700 1 --2- 172.21.15.55:0/963267388 >> v2:172.21.15.163:3300/0 conn(0x7f9cc4150180 0x7f9cc4157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.309+0000 7f9cc3fff700 1 -- 172.21.15.55:0/963267388 >> v1:172.21.15.163:6789/0 conn(0x7f9cc4150550 legacy=0x7f9cc40c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.309+0000 7f9cc3fff700 1 -- 172.21.15.55:0/963267388 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc40c7bc0 con 0x7f9cc40c7410 2023-10-06T03:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.309+0000 7f9cc27fc700 1 -- 172.21.15.55:0/963267388 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9685359 0 0) 0x7f9cc406e2a0 con 0x7f9cc4150550 2023-10-06T03:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.309+0000 7f9cc3fff700 1 --2- 172.21.15.55:0/963267388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cc40c7410 0x7f9cc4069f50 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f9cc406fd20 tx=0x7f9cbc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155fe21e3e6dd34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.309+0000 7f9cc27fc700 1 -- 172.21.15.55:0/963267388 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc00e070 con 0x7f9cc40c7410 2023-10-06T03:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.309+0000 7f9cc27fc700 1 -- 172.21.15.55:0/963267388 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc00ba20 con 0x7f9cc40c7410 2023-10-06T03:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.309+0000 7f9cc27fc700 1 -- 172.21.15.55:0/963267388 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc0092e0 con 0x7f9cc40c7410 2023-10-06T03:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.310+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/963267388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cc40c7410 msgr2=0x7f9cc4069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:20.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.310+0000 7f9cc8b9d700 1 --2- 172.21.15.55:0/963267388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cc40c7410 0x7f9cc4069f50 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f9cc406fd20 tx=0x7f9cbc009840 comp rx=0 tx=0).stop 2023-10-06T03:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.310+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/963267388 shutdown_connections 2023-10-06T03:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.310+0000 7f9cc8b9d700 1 --2- 172.21.15.55:0/963267388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cc40c7410 0x7f9cc4069f50 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.310+0000 7f9cc8b9d700 1 --2- 172.21.15.55:0/963267388 >> v2:172.21.15.163:3300/0 conn(0x7f9cc4150180 0x7f9cc4157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.310+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/963267388 >> 172.21.15.55:0/963267388 conn(0x7f9cc40cc1f0 msgr2=0x7f9cc40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.310+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/963267388 shutdown_connections 2023-10-06T03:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.310+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/963267388 wait complete. 2023-10-06T03:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.311+0000 7f9cc8b9d700 1 Processor -- start 2023-10-06T03:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.312+0000 7f9cc8b9d700 1 -- start start 2023-10-06T03:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.312+0000 7f9cc8b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cc40c7410 0x7f9cc4149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.312+0000 7f9cc8b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cc4150180 0x7f9cc40c2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.312+0000 7f9cc8b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cc4150550 0x7f9cc4154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:20.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.312+0000 7f9cc8b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc40b0100 con 0x7f9cc40c7410 2023-10-06T03:52:20.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.312+0000 7f9cc8b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc40c3010 con 0x7f9cc4150550 2023-10-06T03:52:20.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.312+0000 7f9cc8b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc40afc70 con 0x7f9cc4150180 2023-10-06T03:52:20.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.313+0000 7f9cc37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cc40c7410 0x7f9cc4149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:20.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.313+0000 7f9cc37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cc40c7410 0x7f9cc4149bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51408/0 (socket says 172.21.15.55:51408) 2023-10-06T03:52:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.313+0000 7f9cc37fe700 1 -- 172.21.15.55:0/275869305 learned_addr learned my addr 172.21.15.55:0/275869305 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.313+0000 7f9cc37fe700 1 -- 172.21.15.55:0/275869305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cc4150180 msgr2=0x7f9cc40c2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.313+0000 7f9cc37fe700 1 --2- 172.21.15.55:0/275869305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cc4150180 0x7f9cc40c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.313+0000 7f9cc37fe700 1 -- 172.21.15.55:0/275869305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cc4150550 msgr2=0x7f9cc4154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.313+0000 7f9cc37fe700 1 --2- 172.21.15.55:0/275869305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cc4150550 0x7f9cc4154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.313+0000 7f9cc37fe700 1 -- 172.21.15.55:0/275869305 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc009ae0 con 0x7f9cc40c7410 2023-10-06T03:52:20.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.313+0000 7f9cc37fe700 1 --2- 172.21.15.55:0/275869305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cc40c7410 0x7f9cc4149bc0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400ec80 tx=0x7f9cb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:20.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.314+0000 7f9cab7fe700 1 -- 172.21.15.55:0/275869305 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb400c9d0 con 0x7f9cc40c7410 2023-10-06T03:52:20.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.314+0000 7f9cab7fe700 1 -- 172.21.15.55:0/275869305 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb400cb30 con 0x7f9cc40c7410 2023-10-06T03:52:20.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.314+0000 7f9cab7fe700 1 -- 172.21.15.55:0/275869305 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4013470 con 0x7f9cc40c7410 2023-10-06T03:52:20.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.314+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/275869305 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc40af340 con 0x7f9cc40c7410 2023-10-06T03:52:20.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.314+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/275869305 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc40af900 con 0x7f9cc40c7410 2023-10-06T03:52:20.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.315+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/275869305 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc404c3c0 con 0x7f9cc40c7410 2023-10-06T03:52:20.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.316+0000 7f9cab7fe700 1 -- 172.21.15.55:0/275869305 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f9cb4013820 con 0x7f9cc40c7410 2023-10-06T03:52:20.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.316+0000 7f9cab7fe700 1 --2- 172.21.15.55:0/275869305 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cb0067050 0x7f9cb0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:20.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.317+0000 7f9cab7fe700 1 -- 172.21.15.55:0/275869305 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6286+0+0 (secure 0 0 0) 0x7f9cb4084d70 con 0x7f9cc40c7410 2023-10-06T03:52:20.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.317+0000 7f9cc2ffd700 1 --2- 172.21.15.55:0/275869305 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cb0067050 0x7f9cb0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:20.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.318+0000 7f9cc2ffd700 1 --2- 172.21.15.55:0/275869305 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cb0067050 0x7f9cb0069500 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9cc416dce0 tx=0x7f9cb80079b0 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:20.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.321+0000 7f9cab7fe700 1 -- 172.21.15.55:0/275869305 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9cb4051bf0 con 0x7f9cc40c7410 2023-10-06T03:52:20.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:20.604+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/275869305 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f9cc406ada0 con 0x7f9cc40c7410 2023-10-06T03:52:21.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.053+0000 7f9cab7fe700 1 -- 172.21.15.55:0/275869305 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v723) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9cb4055210 con 0x7f9cc40c7410 2023-10-06T03:52:21.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.056+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/275869305 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cb0067050 msgr2=0x7f9cb0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:21.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.056+0000 7f9cc8b9d700 1 --2- 172.21.15.55:0/275869305 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cb0067050 0x7f9cb0069500 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9cc416dce0 tx=0x7f9cb80079b0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.056+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/275869305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cc40c7410 msgr2=0x7f9cc4149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:21.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.056+0000 7f9cc8b9d700 1 --2- 172.21.15.55:0/275869305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cc40c7410 0x7f9cc4149bc0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400ec80 tx=0x7f9cb40091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.056+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/275869305 shutdown_connections 2023-10-06T03:52:21.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.056+0000 7f9cc8b9d700 1 --2- 172.21.15.55:0/275869305 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cb0067050 0x7f9cb0069500 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.056+0000 7f9cc8b9d700 1 --2- 172.21.15.55:0/275869305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9cc4150550 0x7f9cc4154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.056+0000 7f9cc8b9d700 1 --2- 172.21.15.55:0/275869305 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9cc40c7410 0x7f9cc4149bc0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.056+0000 7f9cc8b9d700 1 --2- 172.21.15.55:0/275869305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9cc4150180 0x7f9cc40c2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.056+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/275869305 >> 172.21.15.55:0/275869305 conn(0x7f9cc40cc1f0 msgr2=0x7f9cc4157940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:21.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.057+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/275869305 shutdown_connections 2023-10-06T03:52:21.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.057+0000 7f9cc8b9d700 1 -- 172.21.15.55:0/275869305 wait complete. 2023-10-06T03:52:21.077 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T03:52:21.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.268+0000 7fb2ef59e700 1 Processor -- start 2023-10-06T03:52:21.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.268+0000 7fb2ef59e700 1 -- start start 2023-10-06T03:52:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.270+0000 7fb2ef59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb2f01407e0 0x7fb2f006a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.270+0000 7fb2ef59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb2f0140bb0 0x7fb2f006a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.270+0000 7fb2ef59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f00c6820 con 0x7fb2f006d4f0 2023-10-06T03:52:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.270+0000 7fb2ef59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f00c6de0 con 0x7fb2f0140bb0 2023-10-06T03:52:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.270+0000 7fb2ef59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f0141270 con 0x7fb2f01407e0 2023-10-06T03:52:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.270+0000 7fb2edd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb2f0140bb0 0x7fb2f006a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.270+0000 7fb2edd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb2f0140bb0 0x7fb2f006a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51412/0 (socket says 172.21.15.55:51412) 2023-10-06T03:52:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.270+0000 7fb2edd9b700 1 -- 172.21.15.55:0/2131126462 learned_addr learned my addr 172.21.15.55:0/2131126462 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:21.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.270+0000 7fb2ee59c700 1 --2- 172.21.15.55:0/2131126462 >> v2:172.21.15.163:3300/0 conn(0x7fb2f01407e0 0x7fb2f006a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:21.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.270+0000 7fb2edd9b700 1 -- 172.21.15.55:0/2131126462 >> v2:172.21.15.163:3300/0 conn(0x7fb2f01407e0 msgr2=0x7fb2f006a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:21.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.270+0000 7fb2edd9b700 1 --2- 172.21.15.55:0/2131126462 >> v2:172.21.15.163:3300/0 conn(0x7fb2f01407e0 0x7fb2f006a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.270+0000 7fb2edd9b700 1 -- 172.21.15.55:0/2131126462 >> v1:172.21.15.163:6789/0 conn(0x7fb2f006d4f0 legacy=0x7fb2f006d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:21.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.271+0000 7fb2edd9b700 1 -- 172.21.15.55:0/2131126462 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f006acf0 con 0x7fb2f0140bb0 2023-10-06T03:52:21.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.271+0000 7fb2edd9b700 1 --2- 172.21.15.55:0/2131126462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0140bb0 0x7fb2f006a6c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fb2f006f9a0 tx=0x7fb2e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d8901b1a59256a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:21.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.271+0000 7fb2ed59a700 1 -- 172.21.15.55:0/2131126462 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e400b860 con 0x7fb2f0140bb0 2023-10-06T03:52:21.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.271+0000 7fb2ed59a700 1 -- 172.21.15.55:0/2131126462 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e400b9c0 con 0x7fb2f0140bb0 2023-10-06T03:52:21.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.271+0000 7fb2ed59a700 1 -- 172.21.15.55:0/2131126462 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e400bd60 con 0x7fb2f0140bb0 2023-10-06T03:52:21.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.272+0000 7fb2ef59e700 1 -- 172.21.15.55:0/2131126462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0140bb0 msgr2=0x7fb2f006a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:21.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.272+0000 7fb2ef59e700 1 --2- 172.21.15.55:0/2131126462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0140bb0 0x7fb2f006a6c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fb2f006f9a0 tx=0x7fb2e4009250 comp rx=0 tx=0).stop 2023-10-06T03:52:21.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.272+0000 7fb2ef59e700 1 -- 172.21.15.55:0/2131126462 shutdown_connections 2023-10-06T03:52:21.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.272+0000 7fb2ef59e700 1 --2- 172.21.15.55:0/2131126462 >> v2:172.21.15.163:3300/0 conn(0x7fb2f01407e0 0x7fb2f006a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.272+0000 7fb2ef59e700 1 --2- 172.21.15.55:0/2131126462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f0140bb0 0x7fb2f006a6c0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.272+0000 7fb2ef59e700 1 -- 172.21.15.55:0/2131126462 >> 172.21.15.55:0/2131126462 conn(0x7fb2f00cc1e0 msgr2=0x7fb2f00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:21.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.272+0000 7fb2ef59e700 1 -- 172.21.15.55:0/2131126462 shutdown_connections 2023-10-06T03:52:21.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.272+0000 7fb2ef59e700 1 -- 172.21.15.55:0/2131126462 wait complete. 2023-10-06T03:52:21.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.273+0000 7fb2ef59e700 1 Processor -- start 2023-10-06T03:52:21.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.273+0000 7fb2ef59e700 1 -- start start 2023-10-06T03:52:21.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.274+0000 7fb2ef59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f006d4f0 0x7fb2f00c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:21.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.274+0000 7fb2ef59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2f01407e0 0x7fb2f013d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:21.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.274+0000 7fb2ef59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2f0140bb0 0x7fb2f00c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:21.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.274+0000 7fb2ef59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f015be70 con 0x7fb2f006d4f0 2023-10-06T03:52:21.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.274+0000 7fb2ef59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f00c5a00 con 0x7fb2f01407e0 2023-10-06T03:52:21.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.274+0000 7fb2ef59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f00c5ce0 con 0x7fb2f0140bb0 2023-10-06T03:52:21.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.275+0000 7fb2ee59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f006d4f0 0x7fb2f00c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:21.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.275+0000 7fb2ee59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f006d4f0 0x7fb2f00c3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51418/0 (socket says 172.21.15.55:51418) 2023-10-06T03:52:21.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.275+0000 7fb2ee59c700 1 -- 172.21.15.55:0/3023738274 learned_addr learned my addr 172.21.15.55:0/3023738274 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:21.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.275+0000 7fb2ee59c700 1 -- 172.21.15.55:0/3023738274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2f0140bb0 msgr2=0x7fb2f00c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:21.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.275+0000 7fb2ee59c700 1 --2- 172.21.15.55:0/3023738274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2f0140bb0 0x7fb2f00c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.275+0000 7fb2ee59c700 1 -- 172.21.15.55:0/3023738274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2f01407e0 msgr2=0x7fb2f013d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:21.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.275+0000 7fb2edd9b700 1 --2- 172.21.15.55:0/3023738274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2f01407e0 0x7fb2f013d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:21.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.275+0000 7fb2ee59c700 1 --2- 172.21.15.55:0/3023738274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2f01407e0 0x7fb2f013d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.275+0000 7fb2ee59c700 1 -- 172.21.15.55:0/3023738274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e40094f0 con 0x7fb2f006d4f0 2023-10-06T03:52:21.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.275+0000 7fb2edd9b700 1 --2- 172.21.15.55:0/3023738274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2f01407e0 0x7fb2f013d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:21.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.275+0000 7fb2ee59c700 1 --2- 172.21.15.55:0/3023738274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f006d4f0 0x7fb2f00c3290 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000d890 tx=0x7fb2e0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:21.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.275+0000 7fb2de7fc700 1 -- 172.21.15.55:0/3023738274 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00125f0 con 0x7fb2f006d4f0 2023-10-06T03:52:21.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.276+0000 7fb2de7fc700 1 -- 172.21.15.55:0/3023738274 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e0012770 con 0x7fb2f006d4f0 2023-10-06T03:52:21.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.276+0000 7fb2ef59e700 1 -- 172.21.15.55:0/3023738274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f00c5fc0 con 0x7fb2f006d4f0 2023-10-06T03:52:21.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.276+0000 7fb2de7fc700 1 -- 172.21.15.55:0/3023738274 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0012b30 con 0x7fb2f006d4f0 2023-10-06T03:52:21.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.276+0000 7fb2ef59e700 1 -- 172.21.15.55:0/3023738274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f016d9b0 con 0x7fb2f006d4f0 2023-10-06T03:52:21.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.277+0000 7fb2ef59e700 1 -- 172.21.15.55:0/3023738274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f016dba0 con 0x7fb2f006d4f0 2023-10-06T03:52:21.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.278+0000 7fb2de7fc700 1 -- 172.21.15.55:0/3023738274 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fb2e001e460 con 0x7fb2f006d4f0 2023-10-06T03:52:21.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.278+0000 7fb2de7fc700 1 --2- 172.21.15.55:0/3023738274 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb2d0067050 0x7fb2d0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:21.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.279+0000 7fb2de7fc700 1 -- 172.21.15.55:0/3023738274 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6286+0+0 (secure 0 0 0) 0x7fb2e0085060 con 0x7fb2f006d4f0 2023-10-06T03:52:21.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.279+0000 7fb2edd9b700 1 --2- 172.21.15.55:0/3023738274 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb2d0067050 0x7fb2d0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:21.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.280+0000 7fb2edd9b700 1 --2- 172.21.15.55:0/3023738274 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb2d0067050 0x7fb2d0069500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb2f006f9a0 tx=0x7fb2e4017000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:21.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.283+0000 7fb2de7fc700 1 -- 172.21.15.55:0/3023738274 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2e0051f80 con 0x7fb2f006d4f0 2023-10-06T03:52:21.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.484+0000 7fb2ef59e700 1 -- 172.21.15.55:0/3023738274 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb2f006afd0 con 0x7fb2f006d4f0 2023-10-06T03:52:21.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.489+0000 7fb2de7fc700 1 -- 172.21.15.55:0/3023738274 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb2e00555a0 con 0x7fb2f006d4f0 2023-10-06T03:52:21.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.491+0000 7fb2ef59e700 1 -- 172.21.15.55:0/3023738274 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb2d0067050 msgr2=0x7fb2d0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:21.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.491+0000 7fb2ef59e700 1 --2- 172.21.15.55:0/3023738274 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb2d0067050 0x7fb2d0069500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb2f006f9a0 tx=0x7fb2e4017000 comp rx=0 tx=0).stop 2023-10-06T03:52:21.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.491+0000 7fb2ef59e700 1 -- 172.21.15.55:0/3023738274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f006d4f0 msgr2=0x7fb2f00c3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:21.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.491+0000 7fb2ef59e700 1 --2- 172.21.15.55:0/3023738274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f006d4f0 0x7fb2f00c3290 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000d890 tx=0x7fb2e0008040 comp rx=0 tx=0).stop 2023-10-06T03:52:21.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.492+0000 7fb2ef59e700 1 -- 172.21.15.55:0/3023738274 shutdown_connections 2023-10-06T03:52:21.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.492+0000 7fb2ef59e700 1 --2- 172.21.15.55:0/3023738274 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb2d0067050 0x7fb2d0069500 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.492+0000 7fb2ef59e700 1 --2- 172.21.15.55:0/3023738274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2f0140bb0 0x7fb2f00c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.492+0000 7fb2ef59e700 1 --2- 172.21.15.55:0/3023738274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2f006d4f0 0x7fb2f00c3290 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.492+0000 7fb2ef59e700 1 --2- 172.21.15.55:0/3023738274 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2f01407e0 0x7fb2f013d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:21.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.492+0000 7fb2ef59e700 1 -- 172.21.15.55:0/3023738274 >> 172.21.15.55:0/3023738274 conn(0x7fb2f00cc1e0 msgr2=0x7fb2f00af920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:21.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.492+0000 7fb2ef59e700 1 -- 172.21.15.55:0/3023738274 shutdown_connections 2023-10-06T03:52:21.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:21.492+0000 7fb2ef59e700 1 -- 172.21.15.55:0/3023738274 wait complete. 2023-10-06T03:52:21.511 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T03:52:21.624 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:52:21.707 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T03:52:22.125 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T03:52:22.146 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T03:52:22.146 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:52:22.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.334+0000 7fe2d359e700 1 Processor -- start 2023-10-06T03:52:22.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.334+0000 7fe2d359e700 1 -- start start 2023-10-06T03:52:22.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.335+0000 7fe2d359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2d4156310 0x7fe2d40c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:22.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.335+0000 7fe2d359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe2d40ae710 0x7fe2d40c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:22.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.335+0000 7fe2d359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d40ca860 con 0x7fe2d41566e0 2023-10-06T03:52:22.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.335+0000 7fe2d359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d40cab40 con 0x7fe2d40ae710 2023-10-06T03:52:22.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.335+0000 7fe2d359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d40cae20 con 0x7fe2d4156310 2023-10-06T03:52:22.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.336+0000 7fe2d2d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe2d40ae710 0x7fe2d40c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:22.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.336+0000 7fe2d1d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe2d41566e0 0x7fe2d40ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:39268/0 (socket says 172.21.15.55:39268) 2023-10-06T03:52:22.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.336+0000 7fe2d2d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe2d40ae710 0x7fe2d40c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51424/0 (socket says 172.21.15.55:51424) 2023-10-06T03:52:22.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.336+0000 7fe2d1d9b700 1 -- 172.21.15.55:0/3768154348 learned_addr learned my addr 172.21.15.55:0/3768154348 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:22.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.336+0000 7fe2d259c700 1 --2- 172.21.15.55:0/3768154348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2d4156310 0x7fe2d40c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:22.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.336+0000 7fe2d2d9d700 1 -- 172.21.15.55:0/3768154348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2d4156310 msgr2=0x7fe2d40c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:22.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.336+0000 7fe2d2d9d700 1 --2- 172.21.15.55:0/3768154348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2d4156310 0x7fe2d40c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:22.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.336+0000 7fe2d2d9d700 1 -- 172.21.15.55:0/3768154348 >> v1:172.21.15.55:6789/0 conn(0x7fe2d41566e0 legacy=0x7fe2d40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:22.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.336+0000 7fe2d2d9d700 1 -- 172.21.15.55:0/3768154348 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d40cb100 con 0x7fe2d40ae710 2023-10-06T03:52:22.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.336+0000 7fe2d159a700 1 -- 172.21.15.55:0/3768154348 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252023286 0 0) 0x7fe2d40ca860 con 0x7fe2d41566e0 2023-10-06T03:52:22.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.336+0000 7fe2d2d9d700 1 --2- 172.21.15.55:0/3768154348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d40ae710 0x7fe2d40c6830 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00d0d0 tx=0x7fe2cc00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ef7aef17259621 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:22.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.337+0000 7fe2d159a700 1 -- 172.21.15.55:0/3768154348 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc010070 con 0x7fe2d40ae710 2023-10-06T03:52:22.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.337+0000 7fe2d159a700 1 -- 172.21.15.55:0/3768154348 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2cc00ddf0 con 0x7fe2d40ae710 2023-10-06T03:52:22.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.337+0000 7fe2d159a700 1 -- 172.21.15.55:0/3768154348 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc00b320 con 0x7fe2d40ae710 2023-10-06T03:52:22.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.337+0000 7fe2d359e700 1 -- 172.21.15.55:0/3768154348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d40ae710 msgr2=0x7fe2d40c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:22.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.337+0000 7fe2d359e700 1 --2- 172.21.15.55:0/3768154348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d40ae710 0x7fe2d40c6830 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00d0d0 tx=0x7fe2cc00b990 comp rx=0 tx=0).stop 2023-10-06T03:52:22.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.337+0000 7fe2d359e700 1 -- 172.21.15.55:0/3768154348 shutdown_connections 2023-10-06T03:52:22.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.337+0000 7fe2d359e700 1 --2- 172.21.15.55:0/3768154348 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d40ae710 0x7fe2d40c6830 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:22.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.337+0000 7fe2d359e700 1 --2- 172.21.15.55:0/3768154348 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2d4156310 0x7fe2d40c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:22.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.337+0000 7fe2d359e700 1 -- 172.21.15.55:0/3768154348 >> 172.21.15.55:0/3768154348 conn(0x7fe2d40cc1f0 msgr2=0x7fe2d40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:22.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.337+0000 7fe2d359e700 1 -- 172.21.15.55:0/3768154348 shutdown_connections 2023-10-06T03:52:22.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.337+0000 7fe2d359e700 1 -- 172.21.15.55:0/3768154348 wait complete. 2023-10-06T03:52:22.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.339+0000 7fe2d359e700 1 Processor -- start 2023-10-06T03:52:22.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.339+0000 7fe2d359e700 1 -- start start 2023-10-06T03:52:22.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2d40ae710 0x7fe2d40b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:22.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d4156310 0x7fe2d415b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:22.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2d41566e0 0x7fe2d415b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:22.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d4150160 con 0x7fe2d4156310 2023-10-06T03:52:22.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d406af90 con 0x7fe2d40ae710 2023-10-06T03:52:22.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d41698d0 con 0x7fe2d41566e0 2023-10-06T03:52:22.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d1d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d4156310 0x7fe2d415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:22.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d1d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d4156310 0x7fe2d415b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:51428/0 (socket says 172.21.15.55:51428) 2023-10-06T03:52:22.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d1d9b700 1 -- 172.21.15.55:0/2483162796 learned_addr learned my addr 172.21.15.55:0/2483162796 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:22.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d259c700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2d40ae710 0x7fe2d40b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:22.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d1d9b700 1 -- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2d41566e0 msgr2=0x7fe2d415b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:22.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d2d9d700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2d41566e0 0x7fe2d415b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:22.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d1d9b700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2d41566e0 0x7fe2d415b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:22.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d1d9b700 1 -- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2d40ae710 msgr2=0x7fe2d40b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:22.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d1d9b700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2d40ae710 0x7fe2d40b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:22.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d1d9b700 1 -- 172.21.15.55:0/2483162796 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc00bc30 con 0x7fe2d4156310 2023-10-06T03:52:22.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.340+0000 7fe2d259c700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2d40ae710 0x7fe2d40b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:22.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.341+0000 7fe2d1d9b700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d4156310 0x7fe2d415b300 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800c840 tx=0x7fe2c8007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:22.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.341+0000 7fe2c27fc700 1 -- 172.21.15.55:0/2483162796 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8005c00 con 0x7fe2d4156310 2023-10-06T03:52:22.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.341+0000 7fe2c27fc700 1 -- 172.21.15.55:0/2483162796 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c8005d60 con 0x7fe2d4156310 2023-10-06T03:52:22.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.341+0000 7fe2d359e700 1 -- 172.21.15.55:0/2483162796 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d415be70 con 0x7fe2d4156310 2023-10-06T03:52:22.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.341+0000 7fe2c27fc700 1 -- 172.21.15.55:0/2483162796 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80072e0 con 0x7fe2d4156310 2023-10-06T03:52:22.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.341+0000 7fe2d359e700 1 -- 172.21.15.55:0/2483162796 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d415c430 con 0x7fe2d4156310 2023-10-06T03:52:22.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.342+0000 7fe2d359e700 1 -- 172.21.15.55:0/2483162796 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d404c3c0 con 0x7fe2d4156310 2023-10-06T03:52:22.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.343+0000 7fe2c27fc700 1 -- 172.21.15.55:0/2483162796 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fe2c801b3b0 con 0x7fe2d4156310 2023-10-06T03:52:22.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.344+0000 7fe2c27fc700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe2b4067050 0x7fe2b4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:22.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.344+0000 7fe2c27fc700 1 -- 172.21.15.55:0/2483162796 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6286+0+0 (secure 0 0 0) 0x7fe2c8083d60 con 0x7fe2d4156310 2023-10-06T03:52:22.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.344+0000 7fe2d259c700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe2b4067050 0x7fe2b4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:22.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.345+0000 7fe2d259c700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe2b4067050 0x7fe2b4069500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4006fd0 tx=0x7fe2c4008040 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:22.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.348+0000 7fe2c27fc700 1 -- 172.21.15.55:0/2483162796 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2c804b5a0 con 0x7fe2d4156310 2023-10-06T03:52:22.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.595+0000 7fe2d359e700 1 -- 172.21.15.55:0/2483162796 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2d40cd0b0 con 0x7fe2d4156310 2023-10-06T03:52:22.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.596+0000 7fe2c27fc700 1 -- 172.21.15.55:0/2483162796 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fe2c8017090 con 0x7fe2d4156310 2023-10-06T03:52:22.791 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:22.791 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":63,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/1295635544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":1295635544},{"type":"v1","addr":"172.21.15.55:6835","nonce":1295635544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7737,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2115790576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2115790576},{"type":"v1","addr":"172.21.15.163:6834","nonce":2115790576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":7740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6836/1021807391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1021807391},{"type":"v1","addr":"172.21.15.163:6836","nonce":1021807391}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56},{"gid":8124,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3900643421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3900643421},{"type":"v1","addr":"172.21.15.55:6834","nonce":3900643421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[]} 2023-10-06T03:52:22.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.598+0000 7fe2d359e700 1 -- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe2b4067050 msgr2=0x7fe2b4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:22.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.598+0000 7fe2d359e700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe2b4067050 0x7fe2b4069500 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4006fd0 tx=0x7fe2c4008040 comp rx=0 tx=0).stop 2023-10-06T03:52:22.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.598+0000 7fe2d359e700 1 -- 172.21.15.55:0/2483162796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d4156310 msgr2=0x7fe2d415b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:22.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.598+0000 7fe2d359e700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d4156310 0x7fe2d415b300 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800c840 tx=0x7fe2c8007db0 comp rx=0 tx=0).stop 2023-10-06T03:52:22.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.598+0000 7fe2d359e700 1 -- 172.21.15.55:0/2483162796 shutdown_connections 2023-10-06T03:52:22.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.598+0000 7fe2d359e700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe2b4067050 0x7fe2b4069500 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:22.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.598+0000 7fe2d359e700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2d41566e0 0x7fe2d415b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:22.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.598+0000 7fe2d359e700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2d40ae710 0x7fe2d40b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:22.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.598+0000 7fe2d359e700 1 --2- 172.21.15.55:0/2483162796 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2d4156310 0x7fe2d415b300 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:22.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.598+0000 7fe2d359e700 1 -- 172.21.15.55:0/2483162796 >> 172.21.15.55:0/2483162796 conn(0x7fe2d40cc1f0 msgr2=0x7fe2d40c12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:22.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.598+0000 7fe2d359e700 1 -- 172.21.15.55:0/2483162796 shutdown_connections 2023-10-06T03:52:22.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:22.598+0000 7fe2d359e700 1 -- 172.21.15.55:0/2483162796 wait complete. 2023-10-06T03:52:22.794 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 63 2023-10-06T03:52:22.795 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T03:52:22.795 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T03:52:22.796 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T03:52:22.796 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:52:22.796 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T03:52:22.796 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T03:52:22.797 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T03:52:22.797 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:52:22.797 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T03:52:22.797 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T03:52:22.797 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T03:52:22.797 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:52:22.798 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T03:52:22.799 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T03:52:22.799 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T03:52:22.808 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T03:52:22.811 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T03:52:22.811 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T03:52:22.812 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T03:52:22.812 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T03:52:22.814 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T03:52:22.814 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T03:52:22.815 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T03:52:22.815 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T03:52:22.857 INFO:tasks.ceph.mds.c:Started 2023-10-06T03:52:23.121 DEBUG:teuthology.parallel:result is None 2023-10-06T03:52:23.122 INFO:tasks.ceph.mds.b:Started 2023-10-06T03:52:23.124 INFO:tasks.ceph.mds.a:Started 2023-10-06T03:52:23.124 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T03:52:23.125 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-10-06T03:52:23.125 DEBUG:teuthology.parallel:result is None 2023-10-06T03:52:23.125 DEBUG:teuthology.parallel:result is None 2023-10-06T03:52:23.126 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T03:52:23.126 INFO:tasks.ceph.mds.a.smithi055.stderr:2023-10-06T03:52:22.991+0000 7fa5c65fd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-06T03:52:23.127 INFO:tasks.ceph.mds.d:Started 2023-10-06T03:52:23.248 DEBUG:teuthology.parallel:result is None 2023-10-06T03:52:23.249 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T03:52:23.255 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2023-10-06T03:52:23.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.437+0000 7f0caf59d700 1 Processor -- start 2023-10-06T03:52:23.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.437+0000 7f0caf59d700 1 -- start start 2023-10-06T03:52:23.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.438+0000 7f0caf59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ca8150550 0x7f0ca8157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:23.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.438+0000 7f0caf59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca806e2a0 con 0x7f0ca8150180 2023-10-06T03:52:23.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.438+0000 7f0caf59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca806b2e0 con 0x7f0ca80c7410 2023-10-06T03:52:23.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.438+0000 7f0caf59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca80c78e0 con 0x7f0ca8150550 2023-10-06T03:52:23.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.439+0000 7f0cae59b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0ca8150180 0x7f0ca80c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49124/0 (socket says 172.21.15.55:49124) 2023-10-06T03:52:23.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.439+0000 7f0cae59b700 1 -- 172.21.15.55:0/4205942004 learned_addr learned my addr 172.21.15.55:0/4205942004 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:23.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.439+0000 7f0cadd9a700 1 --2- 172.21.15.55:0/4205942004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ca8150550 0x7f0ca8157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:23.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.439+0000 7f0cad599700 1 -- 172.21.15.55:0/4205942004 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562942988 0 0) 0x7f0ca806e2a0 con 0x7f0ca8150180 2023-10-06T03:52:23.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.439+0000 7f0cad599700 1 -- 172.21.15.55:0/4205942004 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c94003570 con 0x7f0ca8150180 2023-10-06T03:52:23.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.439+0000 7f0cadd9a700 1 -- 172.21.15.55:0/4205942004 >> v1:172.21.15.163:6789/0 conn(0x7f0ca80c7410 legacy=0x7f0ca8069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:23.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.439+0000 7f0cadd9a700 1 -- 172.21.15.55:0/4205942004 >> v1:172.21.15.55:6789/0 conn(0x7f0ca8150180 legacy=0x7f0ca80c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:23.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.439+0000 7f0cadd9a700 1 -- 172.21.15.55:0/4205942004 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca80c7bc0 con 0x7f0ca8150550 2023-10-06T03:52:23.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.439+0000 7f0cadd9a700 1 --2- 172.21.15.55:0/4205942004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ca8150550 0x7f0ca8157770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009940 tx=0x7f0c980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d8a07720bb6d316 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:23.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.440+0000 7f0cad599700 1 -- 172.21.15.55:0/4205942004 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800b670 con 0x7f0ca8150550 2023-10-06T03:52:23.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.440+0000 7f0cad599700 1 -- 172.21.15.55:0/4205942004 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9800b7d0 con 0x7f0ca8150550 2023-10-06T03:52:23.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.440+0000 7f0cad599700 1 -- 172.21.15.55:0/4205942004 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800bb90 con 0x7f0ca8150550 2023-10-06T03:52:23.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.440+0000 7f0caf59d700 1 -- 172.21.15.55:0/4205942004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ca8150550 msgr2=0x7f0ca8157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:23.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.440+0000 7f0caf59d700 1 --2- 172.21.15.55:0/4205942004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ca8150550 0x7f0ca8157770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009940 tx=0x7f0c980091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:23.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.440+0000 7f0caf59d700 1 -- 172.21.15.55:0/4205942004 shutdown_connections 2023-10-06T03:52:23.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.440+0000 7f0caf59d700 1 --2- 172.21.15.55:0/4205942004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ca8150550 0x7f0ca8157770 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:23.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.440+0000 7f0caf59d700 1 -- 172.21.15.55:0/4205942004 >> 172.21.15.55:0/4205942004 conn(0x7f0ca80cc1f0 msgr2=0x7f0ca80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:23.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.440+0000 7f0caf59d700 1 -- 172.21.15.55:0/4205942004 shutdown_connections 2023-10-06T03:52:23.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.440+0000 7f0caf59d700 1 -- 172.21.15.55:0/4205942004 wait complete. 2023-10-06T03:52:23.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.442+0000 7f0caf59d700 1 Processor -- start 2023-10-06T03:52:23.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.442+0000 7f0caf59d700 1 -- start start 2023-10-06T03:52:23.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.443+0000 7f0caf59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca80c7410 0x7f0ca814b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:23.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.443+0000 7f0caf59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ca8150180 0x7f0ca80ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:23.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.443+0000 7f0caf59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ca8150550 0x7f0ca80c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:23.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.443+0000 7f0caf59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca80ad550 con 0x7f0ca80c7410 2023-10-06T03:52:23.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.443+0000 7f0caf59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca814b580 con 0x7f0ca8150180 2023-10-06T03:52:23.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.443+0000 7f0caf59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca80ad7c0 con 0x7f0ca8150550 2023-10-06T03:52:23.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.443+0000 7f0cae59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca80c7410 0x7f0ca814b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.443+0000 7f0cae59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca80c7410 0x7f0ca814b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54454/0 (socket says 172.21.15.55:54454) 2023-10-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.443+0000 7f0cae59b700 1 -- 172.21.15.55:0/1668179217 learned_addr learned my addr 172.21.15.55:0/1668179217 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.443+0000 7f0cadd9a700 1 --2- 172.21.15.55:0/1668179217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ca8150180 0x7f0ca80ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.443+0000 7f0caed9c700 1 --2- 172.21.15.55:0/1668179217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ca8150550 0x7f0ca80c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.444+0000 7f0cae59b700 1 -- 172.21.15.55:0/1668179217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ca8150550 msgr2=0x7f0ca80c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.444+0000 7f0cae59b700 1 --2- 172.21.15.55:0/1668179217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ca8150550 0x7f0ca80c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.444+0000 7f0cae59b700 1 -- 172.21.15.55:0/1668179217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ca8150180 msgr2=0x7f0ca80ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.444+0000 7f0cae59b700 1 --2- 172.21.15.55:0/1668179217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ca8150180 0x7f0ca80ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.444+0000 7f0cae59b700 1 -- 172.21.15.55:0/1668179217 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c98009420 con 0x7f0ca80c7410 2023-10-06T03:52:23.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.444+0000 7f0cae59b700 1 --2- 172.21.15.55:0/1668179217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca80c7410 0x7f0ca814b040 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400c7c0 tx=0x7f0ca4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:23.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.444+0000 7f0c9e7fc700 1 -- 172.21.15.55:0/1668179217 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4005cd0 con 0x7f0ca80c7410 2023-10-06T03:52:23.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.444+0000 7f0c9e7fc700 1 -- 172.21.15.55:0/1668179217 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca4005e30 con 0x7f0ca80c7410 2023-10-06T03:52:23.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.444+0000 7f0c9e7fc700 1 -- 172.21.15.55:0/1668179217 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4007320 con 0x7f0ca80c7410 2023-10-06T03:52:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.444+0000 7f0caf59d700 1 -- 172.21.15.55:0/1668179217 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca80adaa0 con 0x7f0ca80c7410 2023-10-06T03:52:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.444+0000 7f0caf59d700 1 -- 172.21.15.55:0/1668179217 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca80c42b0 con 0x7f0ca80c7410 2023-10-06T03:52:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.445+0000 7f0caf59d700 1 -- 172.21.15.55:0/1668179217 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca80adfc0 con 0x7f0ca80c7410 2023-10-06T03:52:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.446+0000 7f0c9e7fc700 1 -- 172.21.15.55:0/1668179217 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f0ca401b3b0 con 0x7f0ca80c7410 2023-10-06T03:52:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.447+0000 7f0c9e7fc700 1 --2- 172.21.15.55:0/1668179217 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0c8c067080 0x7f0c8c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.447+0000 7f0c9e7fc700 1 -- 172.21.15.55:0/1668179217 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6286+0+0 (secure 0 0 0) 0x7f0ca4083ff0 con 0x7f0ca80c7410 2023-10-06T03:52:23.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.448+0000 7f0cadd9a700 1 --2- 172.21.15.55:0/1668179217 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0c8c067080 0x7f0c8c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:23.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.448+0000 7f0cadd9a700 1 --2- 172.21.15.55:0/1668179217 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0c8c067080 0x7f0c8c069530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009860 tx=0x7f0c98017000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:23.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.452+0000 7f0c9e7fc700 1 -- 172.21.15.55:0/1668179217 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ca80adfc0 con 0x7f0ca80c7410 2023-10-06T03:52:23.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:23.672+0000 7f0caf59d700 1 -- 172.21.15.55:0/1668179217 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0ca80adfc0 con 0x7f0ca80c7410 2023-10-06T03:52:25.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.070+0000 7f0c9e7fc700 1 -- 172.21.15.55:0/1668179217 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v133) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0ca80adfc0 con 0x7f0ca80c7410 2023-10-06T03:52:25.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.074+0000 7f0caf59d700 1 -- 172.21.15.55:0/1668179217 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0c8c067080 msgr2=0x7f0c8c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:25.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.074+0000 7f0caf59d700 1 --2- 172.21.15.55:0/1668179217 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0c8c067080 0x7f0c8c069530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009860 tx=0x7f0c98017000 comp rx=0 tx=0).stop 2023-10-06T03:52:25.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.074+0000 7f0caf59d700 1 -- 172.21.15.55:0/1668179217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca80c7410 msgr2=0x7f0ca814b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:25.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.074+0000 7f0caf59d700 1 --2- 172.21.15.55:0/1668179217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca80c7410 0x7f0ca814b040 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400c7c0 tx=0x7f0ca4007e00 comp rx=0 tx=0).stop 2023-10-06T03:52:25.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.074+0000 7f0caf59d700 1 -- 172.21.15.55:0/1668179217 shutdown_connections 2023-10-06T03:52:25.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.074+0000 7f0caf59d700 1 --2- 172.21.15.55:0/1668179217 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0c8c067080 0x7f0c8c069530 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.074+0000 7f0caf59d700 1 --2- 172.21.15.55:0/1668179217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0ca8150550 0x7f0ca80c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.074+0000 7f0caf59d700 1 --2- 172.21.15.55:0/1668179217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0ca80c7410 0x7f0ca814b040 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.074+0000 7f0caf59d700 1 --2- 172.21.15.55:0/1668179217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0ca8150180 0x7f0ca80ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.074+0000 7f0caf59d700 1 -- 172.21.15.55:0/1668179217 >> 172.21.15.55:0/1668179217 conn(0x7f0ca80cc1f0 msgr2=0x7f0ca8158b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:25.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.074+0000 7f0caf59d700 1 -- 172.21.15.55:0/1668179217 shutdown_connections 2023-10-06T03:52:25.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.074+0000 7f0caf59d700 1 -- 172.21.15.55:0/1668179217 wait complete. 2023-10-06T03:52:25.113 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2023-10-06T03:52:25.115 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T03:52:25.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.310+0000 7fc529fff700 1 Processor -- start 2023-10-06T03:52:25.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.311+0000 7fc529fff700 1 -- start start 2023-10-06T03:52:25.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.312+0000 7fc529fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc5240ccc40 0x7fc524155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.312+0000 7fc529fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5240ad930 con 0x7fc524156150 2023-10-06T03:52:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.312+0000 7fc529fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5240acd20 con 0x7fc5240cd010 2023-10-06T03:52:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.312+0000 7fc529fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5240ad000 con 0x7fc5240ccc40 2023-10-06T03:52:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.312+0000 7fc522ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc524156150 0x7fc5240ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49130/0 (socket says 172.21.15.55:49130) 2023-10-06T03:52:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.312+0000 7fc522ffd700 1 -- 172.21.15.55:0/3733058784 learned_addr learned my addr 172.21.15.55:0/3733058784 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5237fe700 1 --2- 172.21.15.55:0/3733058784 >> v2:172.21.15.163:3300/0 conn(0x7fc5240ccc40 0x7fc524155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 -- 172.21.15.55:0/3733058784 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2391669291 0 0) 0x7fc5240ad930 con 0x7fc524156150 2023-10-06T03:52:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 -- 172.21.15.55:0/3733058784 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc50c003570 con 0x7fc524156150 2023-10-06T03:52:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 -- 172.21.15.55:0/3733058784 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2113261182 0 0) 0x7fc50c003570 con 0x7fc524156150 2023-10-06T03:52:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 -- 172.21.15.55:0/3733058784 >> v2:172.21.15.163:3300/0 conn(0x7fc5240ccc40 msgr2=0x7fc524155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 --2- 172.21.15.55:0/3733058784 >> v2:172.21.15.163:3300/0 conn(0x7fc5240ccc40 0x7fc524155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 -- 172.21.15.55:0/3733058784 >> v1:172.21.15.163:6789/0 conn(0x7fc5240cd010 legacy=0x7fc5240b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 -- 172.21.15.55:0/3733058784 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5240ad2e0 con 0x7fc524156150 2023-10-06T03:52:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 -- 172.21.15.55:0/3733058784 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fc518001470 con 0x7fc524156150 2023-10-06T03:52:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 -- 172.21.15.55:0/3733058784 >> v1:172.21.15.55:6789/0 conn(0x7fc524156150 legacy=0x7fc5240ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 --2- 172.21.15.55:0/3733058784 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc50c003ab0 0x7fc50c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 --2- 172.21.15.55:0/3733058784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc50c004810 0x7fc50c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 --2- 172.21.15.55:0/3733058784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc50c005210 0x7fc50c00dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 -- 172.21.15.55:0/3733058784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240ad000 con 0x7fc50c005210 2023-10-06T03:52:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 -- 172.21.15.55:0/3733058784 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c00e3c0 con 0x7fc50c004810 2023-10-06T03:52:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.313+0000 7fc5227fc700 1 -- 172.21.15.55:0/3733058784 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c00e580 con 0x7fc50c003ab0 2023-10-06T03:52:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.314+0000 7fc523fff700 1 --2- 172.21.15.55:0/3733058784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc50c005210 0x7fc50c00dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.314+0000 7fc5237fe700 1 --2- 172.21.15.55:0/3733058784 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc50c003ab0 0x7fc50c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.314+0000 7fc522ffd700 1 --2- 172.21.15.55:0/3733058784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc50c004810 0x7fc50c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.314+0000 7fc523fff700 1 -- 172.21.15.55:0/3733058784 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc50c003ab0 msgr2=0x7fc50c003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:25.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.314+0000 7fc523fff700 1 --2- 172.21.15.55:0/3733058784 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc50c003ab0 0x7fc50c003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.314+0000 7fc523fff700 1 -- 172.21.15.55:0/3733058784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc50c004810 msgr2=0x7fc50c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:25.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.314+0000 7fc523fff700 1 --2- 172.21.15.55:0/3733058784 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc50c004810 0x7fc50c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.314+0000 7fc523fff700 1 -- 172.21.15.55:0/3733058784 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c003570 con 0x7fc50c005210 2023-10-06T03:52:25.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.314+0000 7fc523fff700 1 -- 172.21.15.55:0/3733058784 reap_dead start 2023-10-06T03:52:25.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.314+0000 7fc523fff700 1 --2- 172.21.15.55:0/3733058784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc50c005210 0x7fc50c00dc70 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fc51c001fc0 tx=0x7fc51c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b57c022ceb2a30 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:25.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.314+0000 7fc5227fc700 1 -- 172.21.15.55:0/3733058784 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c001950 con 0x7fc50c005210 2023-10-06T03:52:25.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.314+0000 7fc5227fc700 1 -- 172.21.15.55:0/3733058784 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c00c8b0 con 0x7fc50c005210 2023-10-06T03:52:25.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.315+0000 7fc5227fc700 1 -- 172.21.15.55:0/3733058784 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c00cc90 con 0x7fc50c005210 2023-10-06T03:52:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.315+0000 7fc529fff700 1 -- 172.21.15.55:0/3733058784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc50c005210 msgr2=0x7fc50c00dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.315+0000 7fc529fff700 1 --2- 172.21.15.55:0/3733058784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc50c005210 0x7fc50c00dc70 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fc51c001fc0 tx=0x7fc51c001640 comp rx=0 tx=0).stop 2023-10-06T03:52:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.315+0000 7fc529fff700 1 -- 172.21.15.55:0/3733058784 shutdown_connections 2023-10-06T03:52:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.315+0000 7fc529fff700 1 --2- 172.21.15.55:0/3733058784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc50c005210 0x7fc50c00dc70 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.315+0000 7fc529fff700 1 -- 172.21.15.55:0/3733058784 >> 172.21.15.55:0/3733058784 conn(0x7fc52406a320 msgr2=0x7fc5240cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.315+0000 7fc529fff700 1 -- 172.21.15.55:0/3733058784 shutdown_connections 2023-10-06T03:52:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.315+0000 7fc529fff700 1 -- 172.21.15.55:0/3733058784 wait complete. 2023-10-06T03:52:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.317+0000 7fc529fff700 1 Processor -- start 2023-10-06T03:52:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.317+0000 7fc529fff700 1 -- start start 2023-10-06T03:52:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc529fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc50c005210 0x7fc524169bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc529fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc52416a100 0x7fc52415e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc529fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc52415e7c0 0x7fc5240c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc529fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52415ec30 con 0x7fc52415e7c0 2023-10-06T03:52:25.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc529fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52416b450 con 0x7fc52416a100 2023-10-06T03:52:25.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc529fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52416a6f0 con 0x7fc50c005210 2023-10-06T03:52:25.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc523fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc52415e7c0 0x7fc5240c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc523fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc52415e7c0 0x7fc5240c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54466/0 (socket says 172.21.15.55:54466) 2023-10-06T03:52:25.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc523fff700 1 -- 172.21.15.55:0/2078939611 learned_addr learned my addr 172.21.15.55:0/2078939611 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:25.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc523fff700 1 -- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc50c005210 msgr2=0x7fc524169bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:25.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc5237fe700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc50c005210 0x7fc524169bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc522ffd700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc52416a100 0x7fc52415e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc523fff700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc50c005210 0x7fc524169bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc523fff700 1 -- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc52416a100 msgr2=0x7fc52415e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc523fff700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc52416a100 0x7fc52415e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc523fff700 1 -- 172.21.15.55:0/2078939611 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52416a9d0 con 0x7fc52415e7c0 2023-10-06T03:52:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc522ffd700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc52416a100 0x7fc52415e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.318+0000 7fc523fff700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc52415e7c0 0x7fc5240c9030 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc52406d2a0 tx=0x7fc51c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.319+0000 7fc50b7fe700 1 -- 172.21.15.55:0/2078939611 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c001950 con 0x7fc52415e7c0 2023-10-06T03:52:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.319+0000 7fc50b7fe700 1 -- 172.21.15.55:0/2078939611 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51c00c6d0 con 0x7fc52415e7c0 2023-10-06T03:52:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.319+0000 7fc50b7fe700 1 -- 172.21.15.55:0/2078939611 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51c01e850 con 0x7fc52415e7c0 2023-10-06T03:52:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.319+0000 7fc529fff700 1 -- 172.21.15.55:0/2078939611 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52416acb0 con 0x7fc52415e7c0 2023-10-06T03:52:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.319+0000 7fc529fff700 1 -- 172.21.15.55:0/2078939611 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc524152780 con 0x7fc52415e7c0 2023-10-06T03:52:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.320+0000 7fc529fff700 1 -- 172.21.15.55:0/2078939611 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52404c3c0 con 0x7fc52415e7c0 2023-10-06T03:52:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.321+0000 7fc50b7fe700 1 -- 172.21.15.55:0/2078939611 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fc51c01ec40 con 0x7fc52415e7c0 2023-10-06T03:52:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.322+0000 7fc50b7fe700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc510067050 0x7fc510069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.322+0000 7fc50b7fe700 1 -- 172.21.15.55:0/2078939611 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fc51c085ab0 con 0x7fc52415e7c0 2023-10-06T03:52:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.323+0000 7fc5237fe700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc510067050 0x7fc510069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.323+0000 7fc5237fe700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc510067050 0x7fc510069500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc514006f00 tx=0x7fc514008040 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:25.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.327+0000 7fc50b7fe700 1 -- 172.21.15.55:0/2078939611 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc51c052be0 con 0x7fc52415e7c0 2023-10-06T03:52:25.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.604+0000 7fc529fff700 1 -- 172.21.15.55:0/2078939611 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc5240ad5c0 con 0x7fc52415e7c0 2023-10-06T03:52:25.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.604+0000 7fc50b7fe700 1 -- 172.21.15.55:0/2078939611 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc51c017020 con 0x7fc52415e7c0 2023-10-06T03:52:25.606 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:25.606 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:52:25.606 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2023-10-06T03:52:25.606 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.606 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2023-10-06T03:52:25.606 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9li2RaGRAAgRjtbw3h40vylEPjolzlpw==", 2023-10-06T03:52:25.606 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.607 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:52:25.607 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:52:25.607 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:52:25.607 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.607 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.607 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.607 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.607 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2023-10-06T03:52:25.607 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lXZu7LxAAu3QAtdDxtD/AuuqGG12sDw==", 2023-10-06T03:52:25.607 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.607 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:52:25.607 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:52:25.608 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:52:25.608 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.608 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.608 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.608 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.608 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2023-10-06T03:52:25.608 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lS22fJBAAZi6CfyLyJAFgIkaarad1mA==", 2023-10-06T03:52:25.608 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.608 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:52:25.608 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:52:25.608 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:52:25.608 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.609 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.609 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.609 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.609 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2023-10-06T03:52:25.609 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9ll0RIOhAAGtGdPv4Y2bge7xIv3eu2NQ==", 2023-10-06T03:52:25.609 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.609 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:52:25.609 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:52:25.609 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:52:25.609 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.609 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.609 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.610 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.610 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2023-10-06T03:52:25.610 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDHfx9lmTeaDRAAuycc9i7fPj8xrDkuZCAGVg==", 2023-10-06T03:52:25.610 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.610 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:52:25.610 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:52:25.610 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.610 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.610 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.610 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.610 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2023-10-06T03:52:25.611 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDKfx9lxOdXEhAA9S8tdOvhnmBeEbutqnUsZg==", 2023-10-06T03:52:25.611 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.611 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:52:25.611 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:52:25.611 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.611 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.611 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.611 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.611 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2023-10-06T03:52:25.611 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDNfx9l76I+GRAAjaj0gM7An9BCyGmfQgaDFw==", 2023-10-06T03:52:25.611 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.611 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:52:25.612 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:52:25.612 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.612 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.612 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.612 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.612 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2023-10-06T03:52:25.612 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDQfx9lsPK+HxAAzQCV0EuBenqOkUpiQW9uog==", 2023-10-06T03:52:25.612 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.612 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:52:25.612 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:52:25.612 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.612 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.612 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.613 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.613 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2023-10-06T03:52:25.613 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDWfx9lSHNrMxAAIn7awDmljO2cRtKyDalWnw==", 2023-10-06T03:52:25.613 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.613 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:52:25.613 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:52:25.613 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.613 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.613 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.613 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.613 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2023-10-06T03:52:25.614 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDZfx9lFoWuOBAAhzQOt3YChvuv0MhrspOjPw==", 2023-10-06T03:52:25.614 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.615 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:52:25.615 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:52:25.615 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.615 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.632 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.632 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.632 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2023-10-06T03:52:25.632 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDcfx9lB8iSOhAAwamSgGIXZ8owFE4FdbOOBw==", 2023-10-06T03:52:25.632 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.632 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:52:25.632 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:52:25.633 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.633 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.633 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.633 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.633 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2023-10-06T03:52:25.633 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDgfx9lUb3mABAAJ75UO2UQg2r+l+tDYVn0Dg==", 2023-10-06T03:52:25.633 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.633 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T03:52:25.633 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T03:52:25.633 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.634 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.634 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.634 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.634 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2023-10-06T03:52:25.634 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lfNvCBxAA1ZJq4XsWTZHVY0u0NdOqIw==", 2023-10-06T03:52:25.634 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.634 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:52:25.634 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2023-10-06T03:52:25.634 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T03:52:25.634 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.634 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.635 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.635 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2023-10-06T03:52:25.635 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lnvkoFhAAafxsOmItxmwUr1GJrGYJ6g==", 2023-10-06T03:52:25.635 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.635 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:52:25.635 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:52:25.635 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:52:25.635 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:52:25.635 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.635 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.635 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.636 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2023-10-06T03:52:25.636 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9l+9b4DhAAc5XHdikWbXsEblSE3FGScQ==", 2023-10-06T03:52:25.636 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.636 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:52:25.636 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:52:25.636 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:52:25.636 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:52:25.636 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.636 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.636 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.636 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2023-10-06T03:52:25.636 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9luSXrHBAAg6Ufa1Ep/EOgsQtuOPQgYg==", 2023-10-06T03:52:25.636 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.637 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T03:52:25.637 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T03:52:25.637 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T03:52:25.637 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T03:52:25.637 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.637 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.637 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.637 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2023-10-06T03:52:25.637 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC+fx9lEb50FxAAt0Oq28gsXtQvol7TV8trgg==", 2023-10-06T03:52:25.637 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.637 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:52:25.638 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T03:52:25.638 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T03:52:25.638 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.638 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.638 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.638 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.638 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2023-10-06T03:52:25.638 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lQAANEhAAsNiWPqCNEyUS3IELXSC1Cw==", 2023-10-06T03:52:25.638 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.638 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:52:25.638 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T03:52:25.638 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.639 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.639 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:25.639 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:25.639 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2023-10-06T03:52:25.639 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lCwU2ABAAMijxXd8hwgk3XzcdP4NZyg==", 2023-10-06T03:52:25.639 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T03:52:25.639 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T03:52:25.639 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T03:52:25.639 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T03:52:25.639 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.639 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:25.640 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2023-10-06T03:52:25.640 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:52:25.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.607+0000 7fc529fff700 1 -- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc510067050 msgr2=0x7fc510069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:25.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.607+0000 7fc529fff700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc510067050 0x7fc510069500 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc514006f00 tx=0x7fc514008040 comp rx=0 tx=0).stop 2023-10-06T03:52:25.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.607+0000 7fc529fff700 1 -- 172.21.15.55:0/2078939611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc52415e7c0 msgr2=0x7fc5240c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:25.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.607+0000 7fc529fff700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc52415e7c0 0x7fc5240c9030 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fc52406d2a0 tx=0x7fc51c015000 comp rx=0 tx=0).stop 2023-10-06T03:52:25.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.607+0000 7fc529fff700 1 -- 172.21.15.55:0/2078939611 shutdown_connections 2023-10-06T03:52:25.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.607+0000 7fc529fff700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fc510067050 0x7fc510069500 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.607+0000 7fc529fff700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc52415e7c0 0x7fc5240c9030 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.607+0000 7fc529fff700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc50c005210 0x7fc524169bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.607+0000 7fc529fff700 1 --2- 172.21.15.55:0/2078939611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc52416a100 0x7fc52415e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.607+0000 7fc529fff700 1 -- 172.21.15.55:0/2078939611 >> 172.21.15.55:0/2078939611 conn(0x7fc52406a320 msgr2=0x7fc5240abfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:25.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.607+0000 7fc529fff700 1 -- 172.21.15.55:0/2078939611 shutdown_connections 2023-10-06T03:52:25.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.607+0000 7fc529fff700 1 -- 172.21.15.55:0/2078939611 wait complete. 2023-10-06T03:52:25.643 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:52:25.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.830+0000 7fed17fff700 1 Processor -- start 2023-10-06T03:52:25.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.831+0000 7fed17fff700 1 -- start start 2023-10-06T03:52:25.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.832+0000 7fed17fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fed18157270 0x7fed18068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.832+0000 7fed17fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed1815a980 con 0x7fed180688d0 2023-10-06T03:52:25.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.832+0000 7fed17fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed1815a050 con 0x7fed180b0bd0 2023-10-06T03:52:25.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.832+0000 7fed17fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed18068d50 con 0x7fed18157270 2023-10-06T03:52:25.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.832+0000 7fed167fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fed180688d0 0x7fed180b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49140/0 (socket says 172.21.15.55:49140) 2023-10-06T03:52:25.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.832+0000 7fed167fc700 1 -- 172.21.15.55:0/2532810149 learned_addr learned my addr 172.21.15.55:0/2532810149 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:25.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed16ffd700 1 --2- 172.21.15.55:0/2532810149 >> v2:172.21.15.163:3300/0 conn(0x7fed18157270 0x7fed18068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 -- 172.21.15.55:0/2532810149 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312869379 0 0) 0x7fed1815a980 con 0x7fed180688d0 2023-10-06T03:52:25.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 -- 172.21.15.55:0/2532810149 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed00003570 con 0x7fed180688d0 2023-10-06T03:52:25.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 -- 172.21.15.55:0/2532810149 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1399085876 0 0) 0x7fed00003570 con 0x7fed180688d0 2023-10-06T03:52:25.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 -- 172.21.15.55:0/2532810149 >> v2:172.21.15.163:3300/0 conn(0x7fed18157270 msgr2=0x7fed18068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:25.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 --2- 172.21.15.55:0/2532810149 >> v2:172.21.15.163:3300/0 conn(0x7fed18157270 0x7fed18068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 -- 172.21.15.55:0/2532810149 >> v1:172.21.15.163:6789/0 conn(0x7fed180b0bd0 legacy=0x7fed1815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:25.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 -- 172.21.15.55:0/2532810149 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1815a330 con 0x7fed180688d0 2023-10-06T03:52:25.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 -- 172.21.15.55:0/2532810149 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fed0c001470 con 0x7fed180688d0 2023-10-06T03:52:25.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 -- 172.21.15.55:0/2532810149 >> v1:172.21.15.55:6789/0 conn(0x7fed180688d0 legacy=0x7fed180b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:25.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 --2- 172.21.15.55:0/2532810149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed00003a60 0x7fed00003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 --2- 172.21.15.55:0/2532810149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed00004810 0x7fed00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 --2- 172.21.15.55:0/2532810149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed000051c0 0x7fed0000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 -- 172.21.15.55:0/2532810149 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed18068d50 con 0x7fed00004810 2023-10-06T03:52:25.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 -- 172.21.15.55:0/2532810149 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0000c050 con 0x7fed000051c0 2023-10-06T03:52:25.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed15ffb700 1 -- 172.21.15.55:0/2532810149 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0000c210 con 0x7fed00003a60 2023-10-06T03:52:25.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.833+0000 7fed167fc700 1 --2- 172.21.15.55:0/2532810149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed00004810 0x7fed00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.834+0000 7fed177fe700 1 --2- 172.21.15.55:0/2532810149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed000051c0 0x7fed0000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.834+0000 7fed16ffd700 1 --2- 172.21.15.55:0/2532810149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed00003a60 0x7fed00003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.834+0000 7fed167fc700 1 -- 172.21.15.55:0/2532810149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed00003a60 msgr2=0x7fed00003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:25.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.834+0000 7fed167fc700 1 --2- 172.21.15.55:0/2532810149 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed00003a60 0x7fed00003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.834+0000 7fed167fc700 1 -- 172.21.15.55:0/2532810149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed000051c0 msgr2=0x7fed0000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:25.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.834+0000 7fed167fc700 1 --2- 172.21.15.55:0/2532810149 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed000051c0 0x7fed0000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.834+0000 7fed167fc700 1 -- 172.21.15.55:0/2532810149 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed00003570 con 0x7fed00004810 2023-10-06T03:52:25.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.834+0000 7fed177fe700 1 -- 172.21.15.55:0/2532810149 reap_dead start 2023-10-06T03:52:25.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.834+0000 7fed167fc700 1 --2- 172.21.15.55:0/2532810149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed00004810 0x7fed00004c80 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00c2d0 tx=0x7fed0c001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12b1b07454d6997 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:25.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.834+0000 7fed15ffb700 1 -- 172.21.15.55:0/2532810149 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c00cbc0 con 0x7fed00004810 2023-10-06T03:52:25.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.834+0000 7fed15ffb700 1 -- 172.21.15.55:0/2532810149 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c00cd20 con 0x7fed00004810 2023-10-06T03:52:25.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.834+0000 7fed15ffb700 1 -- 172.21.15.55:0/2532810149 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c005b70 con 0x7fed00004810 2023-10-06T03:52:25.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.835+0000 7fed17fff700 1 -- 172.21.15.55:0/2532810149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed00004810 msgr2=0x7fed00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:25.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.835+0000 7fed17fff700 1 --2- 172.21.15.55:0/2532810149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed00004810 0x7fed00004c80 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00c2d0 tx=0x7fed0c001df0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.835+0000 7fed17fff700 1 -- 172.21.15.55:0/2532810149 shutdown_connections 2023-10-06T03:52:25.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.835+0000 7fed17fff700 1 --2- 172.21.15.55:0/2532810149 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed00004810 0x7fed00004c80 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.835+0000 7fed17fff700 1 -- 172.21.15.55:0/2532810149 >> 172.21.15.55:0/2532810149 conn(0x7fed180cd310 msgr2=0x7fed180cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:25.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.835+0000 7fed17fff700 1 -- 172.21.15.55:0/2532810149 shutdown_connections 2023-10-06T03:52:25.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.835+0000 7fed17fff700 1 -- 172.21.15.55:0/2532810149 wait complete. 2023-10-06T03:52:25.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.837+0000 7fed17fff700 1 Processor -- start 2023-10-06T03:52:25.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.837+0000 7fed17fff700 1 -- start start 2023-10-06T03:52:25.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.838+0000 7fed17fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed00004810 0x7fed180af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.838+0000 7fed17fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed1816e320 0x7fed1816e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.838+0000 7fed17fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed1815e700 0x7fed1815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.838+0000 7fed17fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed18154770 con 0x7fed00004810 2023-10-06T03:52:25.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.838+0000 7fed17fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180cae00 con 0x7fed1815e700 2023-10-06T03:52:25.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.838+0000 7fed17fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1816ee10 con 0x7fed1816e320 2023-10-06T03:52:25.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.838+0000 7fed16ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed00004810 0x7fed180af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.839+0000 7fed16ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed00004810 0x7fed180af640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54488/0 (socket says 172.21.15.55:54488) 2023-10-06T03:52:25.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.839+0000 7fed16ffd700 1 -- 172.21.15.55:0/1394044473 learned_addr learned my addr 172.21.15.55:0/1394044473 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:25.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.839+0000 7fed177fe700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed1815e700 0x7fed1815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.839+0000 7fed167fc700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed1816e320 0x7fed1816e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.839+0000 7fed16ffd700 1 -- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed1816e320 msgr2=0x7fed1816e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:25.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.839+0000 7fed16ffd700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed1816e320 0x7fed1816e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.839+0000 7fed16ffd700 1 -- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed1815e700 msgr2=0x7fed1815eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:25.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.839+0000 7fed16ffd700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed1815e700 0x7fed1815eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:25.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.839+0000 7fed16ffd700 1 -- 172.21.15.55:0/1394044473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1816f0f0 con 0x7fed00004810 2023-10-06T03:52:25.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.839+0000 7fed167fc700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed1816e320 0x7fed1816e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:25.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.839+0000 7fed16ffd700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed00004810 0x7fed180af640 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fed08011700 tx=0x7fed08008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:25.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.840+0000 7fecfeffd700 1 -- 172.21.15.55:0/1394044473 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed08015670 con 0x7fed00004810 2023-10-06T03:52:25.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.840+0000 7fecfeffd700 1 -- 172.21.15.55:0/1394044473 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed080157d0 con 0x7fed00004810 2023-10-06T03:52:25.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.840+0000 7fecfeffd700 1 -- 172.21.15.55:0/1394044473 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed08015b50 con 0x7fed00004810 2023-10-06T03:52:25.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.840+0000 7fed17fff700 1 -- 172.21.15.55:0/1394044473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed180caa90 con 0x7fed00004810 2023-10-06T03:52:25.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.840+0000 7fed17fff700 1 -- 172.21.15.55:0/1394044473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed180adb30 con 0x7fed00004810 2023-10-06T03:52:25.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.841+0000 7fed17fff700 1 -- 172.21.15.55:0/1394044473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1816f330 con 0x7fed00004810 2023-10-06T03:52:25.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.842+0000 7fecfeffd700 1 -- 172.21.15.55:0/1394044473 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fed08013410 con 0x7fed00004810 2023-10-06T03:52:25.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.843+0000 7fecfeffd700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fed04067050 0x7fed04069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:25.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.843+0000 7fecfeffd700 1 -- 172.21.15.55:0/1394044473 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fed080849b0 con 0x7fed00004810 2023-10-06T03:52:25.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.843+0000 7fed167fc700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fed04067050 0x7fed04069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:25.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.844+0000 7fed167fc700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fed04067050 0x7fed04069500 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fed0c008fd0 tx=0x7fed0c015000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:25.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:25.847+0000 7fecfeffd700 1 -- 172.21.15.55:0/1394044473 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed1816f330 con 0x7fed00004810 2023-10-06T03:52:26.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.049+0000 7fed17fff700 1 -- 172.21.15.55:0/1394044473 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed180ca8f0 con 0x7fed00004810 2023-10-06T03:52:26.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.050+0000 7fecfeffd700 1 -- 172.21.15.55:0/1394044473 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+14303 (secure 0 0 0) 0x7fed0806be60 con 0x7fed00004810 2023-10-06T03:52:26.061 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:26.061 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":133,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:52:25.065651+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:52:26.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.052+0000 7fed17fff700 1 -- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fed04067050 msgr2=0x7fed04069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.052+0000 7fed17fff700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fed04067050 0x7fed04069500 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fed0c008fd0 tx=0x7fed0c015000 comp rx=0 tx=0).stop 2023-10-06T03:52:26.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.052+0000 7fed17fff700 1 -- 172.21.15.55:0/1394044473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed00004810 msgr2=0x7fed180af640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.052+0000 7fed17fff700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed00004810 0x7fed180af640 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fed08011700 tx=0x7fed08008040 comp rx=0 tx=0).stop 2023-10-06T03:52:26.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.053+0000 7fed17fff700 1 -- 172.21.15.55:0/1394044473 shutdown_connections 2023-10-06T03:52:26.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.053+0000 7fed17fff700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fed04067050 0x7fed04069500 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.053+0000 7fed17fff700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fed1815e700 0x7fed1815eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.053+0000 7fed17fff700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fed00004810 0x7fed180af640 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.053+0000 7fed17fff700 1 --2- 172.21.15.55:0/1394044473 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fed1816e320 0x7fed1816e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.053+0000 7fed17fff700 1 -- 172.21.15.55:0/1394044473 >> 172.21.15.55:0/1394044473 conn(0x7fed180cd310 msgr2=0x7fed18159490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:26.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.053+0000 7fed17fff700 1 -- 172.21.15.55:0/1394044473 shutdown_connections 2023-10-06T03:52:26.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.053+0000 7fed17fff700 1 -- 172.21.15.55:0/1394044473 wait complete. 2023-10-06T03:52:26.072 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T03:52:26.072 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T03:52:26.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.261+0000 7fe60f39d700 1 Processor -- start 2023-10-06T03:52:26.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.261+0000 7fe60f39d700 1 -- start start 2023-10-06T03:52:26.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.263+0000 7fe60f39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe6080cb1a0 0x7fe60815a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:26.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.263+0000 7fe60f39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60806ebd0 con 0x7fe60815a8a0 2023-10-06T03:52:26.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.263+0000 7fe60f39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60815ad20 con 0x7fe6080ae0d0 2023-10-06T03:52:26.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.263+0000 7fe60f39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60815b2e0 con 0x7fe6080cb1a0 2023-10-06T03:52:26.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.263+0000 7fe60db9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe60815a8a0 0x7fe60806f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49144/0 (socket says 172.21.15.55:49144) 2023-10-06T03:52:26.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.263+0000 7fe60db9a700 1 -- 172.21.15.55:0/2356395010 learned_addr learned my addr 172.21.15.55:0/2356395010 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:26.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.263+0000 7fe60e39b700 1 --2- 172.21.15.55:0/2356395010 >> v2:172.21.15.163:3300/0 conn(0x7fe6080cb1a0 0x7fe60815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:26.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.263+0000 7fe60d399700 1 -- 172.21.15.55:0/2356395010 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230812798 0 0) 0x7fe60806ebd0 con 0x7fe60815a8a0 2023-10-06T03:52:26.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.263+0000 7fe60d399700 1 -- 172.21.15.55:0/2356395010 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5f4003570 con 0x7fe60815a8a0 2023-10-06T03:52:26.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.264+0000 7fe60d399700 1 -- 172.21.15.55:0/2356395010 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944419502 0 0) 0x7fe60815ad20 con 0x7fe6080ae0d0 2023-10-06T03:52:26.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.264+0000 7fe60d399700 1 -- 172.21.15.55:0/2356395010 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe60806ebd0 con 0x7fe6080ae0d0 2023-10-06T03:52:26.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.264+0000 7fe60d399700 1 -- 172.21.15.55:0/2356395010 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4253281594 0 0) 0x7fe5f4003570 con 0x7fe60815a8a0 2023-10-06T03:52:26.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.264+0000 7fe60e39b700 1 -- 172.21.15.55:0/2356395010 >> v1:172.21.15.163:6789/0 conn(0x7fe6080ae0d0 legacy=0x7fe6080ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.264+0000 7fe60e39b700 1 -- 172.21.15.55:0/2356395010 >> v1:172.21.15.55:6789/0 conn(0x7fe60815a8a0 legacy=0x7fe60806f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.264+0000 7fe60e39b700 1 -- 172.21.15.55:0/2356395010 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60815b000 con 0x7fe6080cb1a0 2023-10-06T03:52:26.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.264+0000 7fe60e39b700 1 --2- 172.21.15.55:0/2356395010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6080cb1a0 0x7fe60815a360 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe6040098e0 tx=0x7fe6040091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e7df51104d14d50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:26.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.265+0000 7fe60d399700 1 -- 172.21.15.55:0/2356395010 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8001470 con 0x7fe6080cb1a0 2023-10-06T03:52:26.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.265+0000 7fe60d399700 1 -- 172.21.15.55:0/2356395010 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe604011420 con 0x7fe6080cb1a0 2023-10-06T03:52:26.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.265+0000 7fe60d399700 1 -- 172.21.15.55:0/2356395010 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6040117c0 con 0x7fe6080cb1a0 2023-10-06T03:52:26.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.265+0000 7fe60f39d700 1 -- 172.21.15.55:0/2356395010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6080cb1a0 msgr2=0x7fe60815a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.265+0000 7fe60f39d700 1 --2- 172.21.15.55:0/2356395010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6080cb1a0 0x7fe60815a360 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe6040098e0 tx=0x7fe6040091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.265+0000 7fe60f39d700 1 -- 172.21.15.55:0/2356395010 shutdown_connections 2023-10-06T03:52:26.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.265+0000 7fe60f39d700 1 --2- 172.21.15.55:0/2356395010 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6080cb1a0 0x7fe60815a360 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.265+0000 7fe60f39d700 1 -- 172.21.15.55:0/2356395010 >> 172.21.15.55:0/2356395010 conn(0x7fe6080cc1e0 msgr2=0x7fe6080cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:26.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.265+0000 7fe60f39d700 1 -- 172.21.15.55:0/2356395010 shutdown_connections 2023-10-06T03:52:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.265+0000 7fe60f39d700 1 -- 172.21.15.55:0/2356395010 wait complete. 2023-10-06T03:52:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.267+0000 7fe60f39d700 1 Processor -- start 2023-10-06T03:52:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.267+0000 7fe60f39d700 1 -- start start 2023-10-06T03:52:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.268+0000 7fe60f39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6080ae0d0 0x7fe60814e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.268+0000 7fe60f39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6080cb1a0 0x7fe60814f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.268+0000 7fe60f39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe608152b40 0x7fe60814f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.268+0000 7fe60f39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe608151e10 con 0x7fe6080cb1a0 2023-10-06T03:52:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.268+0000 7fe60f39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6081526b0 con 0x7fe6080ae0d0 2023-10-06T03:52:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.268+0000 7fe60f39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080afc90 con 0x7fe608152b40 2023-10-06T03:52:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.268+0000 7fe60db9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6080cb1a0 0x7fe60814f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.268+0000 7fe60db9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6080cb1a0 0x7fe60814f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54496/0 (socket says 172.21.15.55:54496) 2023-10-06T03:52:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.268+0000 7fe60db9a700 1 -- 172.21.15.55:0/2611740861 learned_addr learned my addr 172.21.15.55:0/2611740861 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:26.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.268+0000 7fe60db9a700 1 -- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe608152b40 msgr2=0x7fe60814f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.268+0000 7fe60eb9c700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe608152b40 0x7fe60814f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:26.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.268+0000 7fe60e39b700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6080ae0d0 0x7fe60814e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:26.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.269+0000 7fe60db9a700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe608152b40 0x7fe60814f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.269+0000 7fe60db9a700 1 -- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6080ae0d0 msgr2=0x7fe60814e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.269+0000 7fe60db9a700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6080ae0d0 0x7fe60814e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.269+0000 7fe60db9a700 1 -- 172.21.15.55:0/2611740861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe604009420 con 0x7fe6080cb1a0 2023-10-06T03:52:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.269+0000 7fe60eb9c700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe608152b40 0x7fe60814f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.269+0000 7fe60e39b700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6080ae0d0 0x7fe60814e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.269+0000 7fe60db9a700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6080cb1a0 0x7fe60814f060 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800c810 tx=0x7fe5f8001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.269+0000 7fe5fe7fc700 1 -- 172.21.15.55:0/2611740861 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8007360 con 0x7fe6080cb1a0 2023-10-06T03:52:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.269+0000 7fe5fe7fc700 1 -- 172.21.15.55:0/2611740861 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f8007500 con 0x7fe6080cb1a0 2023-10-06T03:52:26.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.269+0000 7fe5fe7fc700 1 -- 172.21.15.55:0/2611740861 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8004a80 con 0x7fe6080cb1a0 2023-10-06T03:52:26.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.269+0000 7fe60f39d700 1 -- 172.21.15.55:0/2611740861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe608152060 con 0x7fe6080cb1a0 2023-10-06T03:52:26.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.269+0000 7fe60f39d700 1 -- 172.21.15.55:0/2611740861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60814e830 con 0x7fe6080cb1a0 2023-10-06T03:52:26.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.270+0000 7fe60f39d700 1 -- 172.21.15.55:0/2611740861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe608152860 con 0x7fe6080cb1a0 2023-10-06T03:52:26.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.271+0000 7fe5fe7fc700 1 -- 172.21.15.55:0/2611740861 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fe5f801c3b0 con 0x7fe6080cb1a0 2023-10-06T03:52:26.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.272+0000 7fe5fe7fc700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe5ec067050 0x7fe5ec069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:26.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.272+0000 7fe5fe7fc700 1 -- 172.21.15.55:0/2611740861 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fe5f8083dd0 con 0x7fe6080cb1a0 2023-10-06T03:52:26.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.272+0000 7fe60e39b700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe5ec067050 0x7fe5ec069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:26.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.273+0000 7fe60e39b700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe5ec067050 0x7fe5ec069500 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe604003080 tx=0x7fe604016000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:26.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.276+0000 7fe5fe7fc700 1 -- 172.21.15.55:0/2611740861 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5f804c5a0 con 0x7fe6080cb1a0 2023-10-06T03:52:26.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.476+0000 7fe60f39d700 1 -- 172.21.15.55:0/2611740861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe60806e860 con 0x7fe6080cb1a0 2023-10-06T03:52:26.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.476+0000 7fe5fe7fc700 1 -- 172.21.15.55:0/2611740861 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v133) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe5f80180a0 con 0x7fe6080cb1a0 2023-10-06T03:52:26.532 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T03:52:26.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.478+0000 7fe60f39d700 1 -- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe5ec067050 msgr2=0x7fe5ec069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.478+0000 7fe60f39d700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe5ec067050 0x7fe5ec069500 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe604003080 tx=0x7fe604016000 comp rx=0 tx=0).stop 2023-10-06T03:52:26.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.478+0000 7fe60f39d700 1 -- 172.21.15.55:0/2611740861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6080cb1a0 msgr2=0x7fe60814f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.478+0000 7fe60f39d700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6080cb1a0 0x7fe60814f060 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800c810 tx=0x7fe5f8001a10 comp rx=0 tx=0).stop 2023-10-06T03:52:26.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.479+0000 7fe60f39d700 1 -- 172.21.15.55:0/2611740861 shutdown_connections 2023-10-06T03:52:26.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.479+0000 7fe60f39d700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe5ec067050 0x7fe5ec069500 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.479+0000 7fe60f39d700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe608152b40 0x7fe60814f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.479+0000 7fe60f39d700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe6080ae0d0 0x7fe60814e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.479+0000 7fe60f39d700 1 --2- 172.21.15.55:0/2611740861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe6080cb1a0 0x7fe60814f060 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.479+0000 7fe60f39d700 1 -- 172.21.15.55:0/2611740861 >> 172.21.15.55:0/2611740861 conn(0x7fe6080cc1e0 msgr2=0x7fe6080c7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:26.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.479+0000 7fe60f39d700 1 -- 172.21.15.55:0/2611740861 shutdown_connections 2023-10-06T03:52:26.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.479+0000 7fe60f39d700 1 -- 172.21.15.55:0/2611740861 wait complete. 2023-10-06T03:52:26.538 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T03:52:26.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.728+0000 7fd55b59d700 1 Processor -- start 2023-10-06T03:52:26.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.728+0000 7fd55b59d700 1 -- start start 2023-10-06T03:52:26.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.729+0000 7fd55b59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd5540b4e70 0x7fd5540ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:26.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.729+0000 7fd55b59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd5540b16d0 0x7fd5540b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:26.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.730+0000 7fd55b59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5540b2080 con 0x7fd55415d240 2023-10-06T03:52:26.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.730+0000 7fd55b59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5540cfb00 con 0x7fd5540b4e70 2023-10-06T03:52:26.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.730+0000 7fd55b59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5540b5c40 con 0x7fd5540b16d0 2023-10-06T03:52:26.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.730+0000 7fd55a59b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd5540b4e70 0x7fd5540ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:26.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.730+0000 7fd55a59b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd5540b4e70 0x7fd5540ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54498/0 (socket says 172.21.15.55:54498) 2023-10-06T03:52:26.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.730+0000 7fd55a59b700 1 -- 172.21.15.55:0/2226376724 learned_addr learned my addr 172.21.15.55:0/2226376724 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:26.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.730+0000 7fd559d9a700 1 --2- 172.21.15.55:0/2226376724 >> v2:172.21.15.163:3300/0 conn(0x7fd5540b16d0 0x7fd5540b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:26.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.730+0000 7fd55a59b700 1 -- 172.21.15.55:0/2226376724 >> v2:172.21.15.163:3300/0 conn(0x7fd5540b16d0 msgr2=0x7fd5540b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:26.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.730+0000 7fd55a59b700 1 --2- 172.21.15.55:0/2226376724 >> v2:172.21.15.163:3300/0 conn(0x7fd5540b16d0 0x7fd5540b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.730+0000 7fd55a59b700 1 -- 172.21.15.55:0/2226376724 >> v1:172.21.15.163:6789/0 conn(0x7fd55415d240 legacy=0x7fd55415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.730+0000 7fd55a59b700 1 -- 172.21.15.55:0/2226376724 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5540b5f20 con 0x7fd5540b4e70 2023-10-06T03:52:26.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.730+0000 7fd559d9a700 1 --2- 172.21.15.55:0/2226376724 >> v2:172.21.15.163:3300/0 conn(0x7fd5540b16d0 0x7fd5540b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:26.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.731+0000 7fd55a59b700 1 --2- 172.21.15.55:0/2226376724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540b4e70 0x7fd5540ccff0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fd554089830 tx=0x7fd550009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e66ebdcd474a27f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.731+0000 7fd559599700 1 -- 172.21.15.55:0/2226376724 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000b860 con 0x7fd5540b4e70 2023-10-06T03:52:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.731+0000 7fd559599700 1 -- 172.21.15.55:0/2226376724 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55000b9c0 con 0x7fd5540b4e70 2023-10-06T03:52:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.731+0000 7fd559599700 1 -- 172.21.15.55:0/2226376724 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55000bd60 con 0x7fd5540b4e70 2023-10-06T03:52:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.731+0000 7fd55b59d700 1 -- 172.21.15.55:0/2226376724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540b4e70 msgr2=0x7fd5540ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.731+0000 7fd55b59d700 1 --2- 172.21.15.55:0/2226376724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540b4e70 0x7fd5540ccff0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fd554089830 tx=0x7fd550009250 comp rx=0 tx=0).stop 2023-10-06T03:52:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.732+0000 7fd55b59d700 1 -- 172.21.15.55:0/2226376724 shutdown_connections 2023-10-06T03:52:26.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.732+0000 7fd55b59d700 1 --2- 172.21.15.55:0/2226376724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540b4e70 0x7fd5540ccff0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.732+0000 7fd55b59d700 1 --2- 172.21.15.55:0/2226376724 >> v2:172.21.15.163:3300/0 conn(0x7fd5540b16d0 0x7fd5540b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.732+0000 7fd55b59d700 1 -- 172.21.15.55:0/2226376724 >> 172.21.15.55:0/2226376724 conn(0x7fd5540d10c0 msgr2=0x7fd5540affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.732+0000 7fd55b59d700 1 -- 172.21.15.55:0/2226376724 shutdown_connections 2023-10-06T03:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.732+0000 7fd55b59d700 1 -- 172.21.15.55:0/2226376724 wait complete. 2023-10-06T03:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.733+0000 7fd55b59d700 1 Processor -- start 2023-10-06T03:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.733+0000 7fd55b59d700 1 -- start start 2023-10-06T03:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.734+0000 7fd55b59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5540b16d0 0x7fd554156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.734+0000 7fd55b59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540b4e70 0x7fd5540c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.734+0000 7fd55b59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd55415d240 0x7fd5540cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.734+0000 7fd55b59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540cb2f0 con 0x7fd5540b4e70 2023-10-06T03:52:26.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.734+0000 7fd55b59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540ca9c0 con 0x7fd5540b16d0 2023-10-06T03:52:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.734+0000 7fd55b59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540caca0 con 0x7fd55415d240 2023-10-06T03:52:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.734+0000 7fd559d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540b4e70 0x7fd5540c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.734+0000 7fd559d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540b4e70 0x7fd5540c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54504/0 (socket says 172.21.15.55:54504) 2023-10-06T03:52:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.734+0000 7fd559d9a700 1 -- 172.21.15.55:0/1189456892 learned_addr learned my addr 172.21.15.55:0/1189456892 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.735+0000 7fd55a59b700 1 --2- 172.21.15.55:0/1189456892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5540b16d0 0x7fd554156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.735+0000 7fd55ad9c700 1 --2- 172.21.15.55:0/1189456892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd55415d240 0x7fd5540cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.735+0000 7fd559d9a700 1 -- 172.21.15.55:0/1189456892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd55415d240 msgr2=0x7fd5540cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.735+0000 7fd559d9a700 1 --2- 172.21.15.55:0/1189456892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd55415d240 0x7fd5540cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.735+0000 7fd559d9a700 1 -- 172.21.15.55:0/1189456892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5540b16d0 msgr2=0x7fd554156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.735+0000 7fd559d9a700 1 --2- 172.21.15.55:0/1189456892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5540b16d0 0x7fd554156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.735+0000 7fd559d9a700 1 -- 172.21.15.55:0/1189456892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5500094f0 con 0x7fd5540b4e70 2023-10-06T03:52:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.735+0000 7fd559d9a700 1 --2- 172.21.15.55:0/1189456892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540b4e70 0x7fd5540c5390 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fd54400e8d0 tx=0x7fd544008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.735+0000 7fd54a7fc700 1 -- 172.21.15.55:0/1189456892 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544013410 con 0x7fd5540b4e70 2023-10-06T03:52:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.735+0000 7fd54a7fc700 1 -- 172.21.15.55:0/1189456892 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd544013570 con 0x7fd5540b4e70 2023-10-06T03:52:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.735+0000 7fd55b59d700 1 -- 172.21.15.55:0/1189456892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5540caf80 con 0x7fd5540b4e70 2023-10-06T03:52:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.735+0000 7fd54a7fc700 1 -- 172.21.15.55:0/1189456892 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544013910 con 0x7fd5540b4e70 2023-10-06T03:52:26.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.735+0000 7fd55b59d700 1 -- 172.21.15.55:0/1189456892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5540ca690 con 0x7fd5540b4e70 2023-10-06T03:52:26.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.736+0000 7fd55b59d700 1 -- 172.21.15.55:0/1189456892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5540cb460 con 0x7fd5540b4e70 2023-10-06T03:52:26.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.737+0000 7fd54a7fc700 1 -- 172.21.15.55:0/1189456892 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fd544013d00 con 0x7fd5540b4e70 2023-10-06T03:52:26.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.738+0000 7fd54a7fc700 1 --2- 172.21.15.55:0/1189456892 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd538067050 0x7fd538069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:26.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.738+0000 7fd54a7fc700 1 -- 172.21.15.55:0/1189456892 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fd544084550 con 0x7fd5540b4e70 2023-10-06T03:52:26.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.739+0000 7fd55a59b700 1 --2- 172.21.15.55:0/1189456892 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd538067050 0x7fd538069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:26.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.740+0000 7fd55a59b700 1 --2- 172.21.15.55:0/1189456892 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd538067050 0x7fd538069500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd554089830 tx=0x7fd550017000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:26.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.742+0000 7fd54a7fc700 1 -- 172.21.15.55:0/1189456892 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd544051570 con 0x7fd5540b4e70 2023-10-06T03:52:26.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.942+0000 7fd55b59d700 1 -- 172.21.15.55:0/1189456892 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd55404bb80 con 0x7fd5540b4e70 2023-10-06T03:52:26.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.943+0000 7fd54a7fc700 1 -- 172.21.15.55:0/1189456892 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd544054b90 con 0x7fd5540b4e70 2023-10-06T03:52:26.959 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T03:52:26.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.945+0000 7fd55b59d700 1 -- 172.21.15.55:0/1189456892 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd538067050 msgr2=0x7fd538069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.945+0000 7fd55b59d700 1 --2- 172.21.15.55:0/1189456892 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd538067050 0x7fd538069500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd554089830 tx=0x7fd550017000 comp rx=0 tx=0).stop 2023-10-06T03:52:26.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.945+0000 7fd55b59d700 1 -- 172.21.15.55:0/1189456892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540b4e70 msgr2=0x7fd5540c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:26.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.945+0000 7fd55b59d700 1 --2- 172.21.15.55:0/1189456892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540b4e70 0x7fd5540c5390 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fd54400e8d0 tx=0x7fd544008040 comp rx=0 tx=0).stop 2023-10-06T03:52:26.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.945+0000 7fd55b59d700 1 -- 172.21.15.55:0/1189456892 shutdown_connections 2023-10-06T03:52:26.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.945+0000 7fd55b59d700 1 --2- 172.21.15.55:0/1189456892 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd538067050 0x7fd538069500 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.945+0000 7fd55b59d700 1 --2- 172.21.15.55:0/1189456892 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd55415d240 0x7fd5540cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.945+0000 7fd55b59d700 1 --2- 172.21.15.55:0/1189456892 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd5540b16d0 0x7fd554156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.945+0000 7fd55b59d700 1 --2- 172.21.15.55:0/1189456892 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd5540b4e70 0x7fd5540c5390 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:26.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.945+0000 7fd55b59d700 1 -- 172.21.15.55:0/1189456892 >> 172.21.15.55:0/1189456892 conn(0x7fd5540d10c0 msgr2=0x7fd554158fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:26.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.945+0000 7fd55b59d700 1 -- 172.21.15.55:0/1189456892 shutdown_connections 2023-10-06T03:52:26.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:26.945+0000 7fd55b59d700 1 -- 172.21.15.55:0/1189456892 wait complete. 2023-10-06T03:52:26.964 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T03:52:27.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.151+0000 7f75b359d700 1 Processor -- start 2023-10-06T03:52:27.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.151+0000 7f75b359d700 1 -- start start 2023-10-06T03:52:27.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.152+0000 7f75b359d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f75ac1597f0 0x7f75ac0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:27.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.152+0000 7f75b359d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75ac15d230 0x7f75ac15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:27.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.152+0000 7f75b359d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ac15dbb0 con 0x7f75ac159420 2023-10-06T03:52:27.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.152+0000 7f75b359d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75ac0b11d0 con 0x7f75ac1597f0 2023-10-06T03:52:27.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.152+0000 7f75b359d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac0b14b0 con 0x7f75ac15d230 2023-10-06T03:52:27.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.152+0000 7f75b1d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f75ac1597f0 0x7f75ac0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:27.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.153+0000 7f75b1d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f75ac1597f0 0x7f75ac0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54508/0 (socket says 172.21.15.55:54508) 2023-10-06T03:52:27.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.153+0000 7f75b1d9a700 1 -- 172.21.15.55:0/283450450 learned_addr learned my addr 172.21.15.55:0/283450450 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:27.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.153+0000 7f75b2d9c700 1 --2- 172.21.15.55:0/283450450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75ac15d230 0x7f75ac15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:27.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.153+0000 7f75b1d9a700 1 -- 172.21.15.55:0/283450450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75ac15d230 msgr2=0x7f75ac15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:27.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.153+0000 7f75b1d9a700 1 --2- 172.21.15.55:0/283450450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75ac15d230 0x7f75ac15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:27.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.153+0000 7f75b1d9a700 1 -- 172.21.15.55:0/283450450 >> v1:172.21.15.163:6789/0 conn(0x7f75ac159420 legacy=0x7f75ac0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:27.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.153+0000 7f75b1d9a700 1 -- 172.21.15.55:0/283450450 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ac0b1790 con 0x7f75ac1597f0 2023-10-06T03:52:27.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.153+0000 7f75b1d9a700 1 --2- 172.21.15.55:0/283450450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ac1597f0 0x7f75ac0b2270 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f759c0099b0 tx=0x7f759c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155fed1fc41242d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:27.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.153+0000 7f75b1599700 1 -- 172.21.15.55:0/283450450 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c00b860 con 0x7f75ac1597f0 2023-10-06T03:52:27.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.154+0000 7f75b1599700 1 -- 172.21.15.55:0/283450450 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c00b9c0 con 0x7f75ac1597f0 2023-10-06T03:52:27.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.154+0000 7f75b1599700 1 -- 172.21.15.55:0/283450450 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c00bd60 con 0x7f75ac1597f0 2023-10-06T03:52:27.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.154+0000 7f75b359d700 1 -- 172.21.15.55:0/283450450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ac1597f0 msgr2=0x7f75ac0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:27.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.154+0000 7f75b359d700 1 --2- 172.21.15.55:0/283450450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ac1597f0 0x7f75ac0b2270 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f759c0099b0 tx=0x7f759c009250 comp rx=0 tx=0).stop 2023-10-06T03:52:27.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.154+0000 7f75b359d700 1 -- 172.21.15.55:0/283450450 shutdown_connections 2023-10-06T03:52:27.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.154+0000 7f75b359d700 1 --2- 172.21.15.55:0/283450450 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75ac15d230 0x7f75ac15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:27.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.154+0000 7f75b359d700 1 --2- 172.21.15.55:0/283450450 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ac1597f0 0x7f75ac0b2270 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:27.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.154+0000 7f75b359d700 1 -- 172.21.15.55:0/283450450 >> 172.21.15.55:0/283450450 conn(0x7f75ac0d0660 msgr2=0x7f75ac0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:27.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.154+0000 7f75b359d700 1 -- 172.21.15.55:0/283450450 shutdown_connections 2023-10-06T03:52:27.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.154+0000 7f75b359d700 1 -- 172.21.15.55:0/283450450 wait complete. 2023-10-06T03:52:27.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.156+0000 7f75b359d700 1 Processor -- start 2023-10-06T03:52:27.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.156+0000 7f75b359d700 1 -- start start 2023-10-06T03:52:27.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b359d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75ac159420 0x7f75ac0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:27.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b359d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75ac1597f0 0x7f75ac154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:27.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b359d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ac15d230 0x7f75ac155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:27.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b359d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac0c9120 con 0x7f75ac15d230 2023-10-06T03:52:27.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b359d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac0c5950 con 0x7f75ac159420 2023-10-06T03:52:27.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b359d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac0c5c30 con 0x7f75ac1597f0 2023-10-06T03:52:27.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b2d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ac15d230 0x7f75ac155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:27.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b2d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ac15d230 0x7f75ac155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54512/0 (socket says 172.21.15.55:54512) 2023-10-06T03:52:27.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b2d9c700 1 -- 172.21.15.55:0/664302388 learned_addr learned my addr 172.21.15.55:0/664302388 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:27.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b1d9a700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75ac1597f0 0x7f75ac154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:27.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b259b700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75ac159420 0x7f75ac0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:27.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b2d9c700 1 -- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75ac1597f0 msgr2=0x7f75ac154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:27.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b2d9c700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75ac1597f0 0x7f75ac154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:27.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b2d9c700 1 -- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75ac159420 msgr2=0x7f75ac0c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:27.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b2d9c700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75ac159420 0x7f75ac0c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:27.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b2d9c700 1 -- 172.21.15.55:0/664302388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c0094f0 con 0x7f75ac15d230 2023-10-06T03:52:27.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.157+0000 7f75b1d9a700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75ac1597f0 0x7f75ac154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:27.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.158+0000 7f75b2d9c700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ac15d230 0x7f75ac155cf0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f75a4007f20 tx=0x7f75a4009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:27.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.158+0000 7f75a27fc700 1 -- 172.21.15.55:0/664302388 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a400eca0 con 0x7f75ac15d230 2023-10-06T03:52:27.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.158+0000 7f75a27fc700 1 -- 172.21.15.55:0/664302388 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a400ee20 con 0x7f75ac15d230 2023-10-06T03:52:27.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.158+0000 7f75b359d700 1 -- 172.21.15.55:0/664302388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ac0c5f10 con 0x7f75ac15d230 2023-10-06T03:52:27.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.158+0000 7f75a27fc700 1 -- 172.21.15.55:0/664302388 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4014670 con 0x7f75ac15d230 2023-10-06T03:52:27.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.158+0000 7f75b359d700 1 -- 172.21.15.55:0/664302388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ac0c7ab0 con 0x7f75ac15d230 2023-10-06T03:52:27.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.159+0000 7f75b359d700 1 -- 172.21.15.55:0/664302388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ac0c9290 con 0x7f75ac15d230 2023-10-06T03:52:27.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.160+0000 7f75a27fc700 1 -- 172.21.15.55:0/664302388 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f75a40149e0 con 0x7f75ac15d230 2023-10-06T03:52:27.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.161+0000 7f75a27fc700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7590067050 0x7f7590069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:27.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.161+0000 7f75a27fc700 1 -- 172.21.15.55:0/664302388 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f75a4084dc0 con 0x7f75ac15d230 2023-10-06T03:52:27.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.161+0000 7f75b259b700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7590067050 0x7f7590069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:27.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.162+0000 7f75b259b700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7590067050 0x7f7590069500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f75a8003860 tx=0x7f75a8009040 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:27.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.165+0000 7f75a27fc700 1 -- 172.21.15.55:0/664302388 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75a4051ef0 con 0x7f75ac15d230 2023-10-06T03:52:27.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.372+0000 7f75b359d700 1 -- 172.21.15.55:0/664302388 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f75ac04c3c0 con 0x7f75ac15d230 2023-10-06T03:52:27.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.372+0000 7f75a27fc700 1 -- 172.21.15.55:0/664302388 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v133) v1 ==== 84+0+126 (secure 0 0 0) 0x7f75a4055510 con 0x7f75ac15d230 2023-10-06T03:52:27.421 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:27.421 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:52:27.421 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:27.421 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2023-10-06T03:52:27.421 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": "rbd" 2023-10-06T03:52:27.421 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T03:52:27.421 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:27.421 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 2, 2023-10-06T03:52:27.422 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2023-10-06T03:52:27.422 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:27.422 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:52:27.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.374+0000 7f75b359d700 1 -- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7590067050 msgr2=0x7f7590069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:27.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.374+0000 7f75b359d700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7590067050 0x7f7590069500 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f75a8003860 tx=0x7f75a8009040 comp rx=0 tx=0).stop 2023-10-06T03:52:27.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.375+0000 7f75b359d700 1 -- 172.21.15.55:0/664302388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ac15d230 msgr2=0x7f75ac155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:27.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.375+0000 7f75b359d700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ac15d230 0x7f75ac155cf0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f75a4007f20 tx=0x7f75a4009860 comp rx=0 tx=0).stop 2023-10-06T03:52:27.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.375+0000 7f75b359d700 1 -- 172.21.15.55:0/664302388 shutdown_connections 2023-10-06T03:52:27.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.375+0000 7f75b359d700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7590067050 0x7f7590069500 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:27.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.375+0000 7f75b359d700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75ac15d230 0x7f75ac155cf0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:27.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.375+0000 7f75b359d700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75ac159420 0x7f75ac0c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:27.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.375+0000 7f75b359d700 1 --2- 172.21.15.55:0/664302388 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75ac1597f0 0x7f75ac154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:27.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.375+0000 7f75b359d700 1 -- 172.21.15.55:0/664302388 >> 172.21.15.55:0/664302388 conn(0x7f75ac0d0660 msgr2=0x7f75ac0b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:27.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.375+0000 7f75b359d700 1 -- 172.21.15.55:0/664302388 shutdown_connections 2023-10-06T03:52:27.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.375+0000 7f75b359d700 1 -- 172.21.15.55:0/664302388 wait complete. 2023-10-06T03:52:27.425 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T03:52:27.727 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T03:52:27.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.918+0000 7f8edf59e700 1 Processor -- start 2023-10-06T03:52:27.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.918+0000 7f8edf59e700 1 -- start start 2023-10-06T03:52:27.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.919+0000 7f8edf59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ee00b0790 0x7f8ee00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:27.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.919+0000 7f8edf59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8ee01591e0 0x7f8ee0159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:27.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.919+0000 7f8edf59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee00cfa00 con 0x7f8ee00b03c0 2023-10-06T03:52:27.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.919+0000 7f8edf59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee00cb580 con 0x7f8ee00b0790 2023-10-06T03:52:27.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.919+0000 7f8edf59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee00cf160 con 0x7f8ee01591e0 2023-10-06T03:52:27.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.919+0000 7f8eddd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ee00b0790 0x7f8ee00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:27.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.919+0000 7f8eddd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8ee00b0790 0x7f8ee00cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54528/0 (socket says 172.21.15.55:54528) 2023-10-06T03:52:27.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.919+0000 7f8eddd9b700 1 -- 172.21.15.55:0/3415506823 learned_addr learned my addr 172.21.15.55:0/3415506823 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:27.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.920+0000 7f8eddd9b700 1 -- 172.21.15.55:0/3415506823 >> v2:172.21.15.163:3300/0 conn(0x7f8ee01591e0 msgr2=0x7f8ee0159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:27.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.920+0000 7f8eded9d700 1 --2- 172.21.15.55:0/3415506823 >> v2:172.21.15.163:3300/0 conn(0x7f8ee01591e0 0x7f8ee0159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:27.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.920+0000 7f8eddd9b700 1 --2- 172.21.15.55:0/3415506823 >> v2:172.21.15.163:3300/0 conn(0x7f8ee01591e0 0x7f8ee0159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:27.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.920+0000 7f8eddd9b700 1 -- 172.21.15.55:0/3415506823 >> v1:172.21.15.163:6789/0 conn(0x7f8ee00b03c0 legacy=0x7f8ee00b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:27.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.920+0000 7f8eddd9b700 1 -- 172.21.15.55:0/3415506823 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee00cf440 con 0x7f8ee00b0790 2023-10-06T03:52:27.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.920+0000 7f8eded9d700 1 --2- 172.21.15.55:0/3415506823 >> v2:172.21.15.163:3300/0 conn(0x7f8ee01591e0 0x7f8ee0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:27.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.920+0000 7f8eddd9b700 1 --2- 172.21.15.55:0/3415506823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ee00b0790 0x7f8ee00cea70 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40099b0 tx=0x7f8ed4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f46cdc385d57e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:27.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.920+0000 7f8edd59a700 1 -- 172.21.15.55:0/3415506823 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed400b7e0 con 0x7f8ee00b0790 2023-10-06T03:52:27.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.920+0000 7f8edd59a700 1 -- 172.21.15.55:0/3415506823 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed400b940 con 0x7f8ee00b0790 2023-10-06T03:52:27.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.920+0000 7f8edd59a700 1 -- 172.21.15.55:0/3415506823 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed400bd00 con 0x7f8ee00b0790 2023-10-06T03:52:27.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.921+0000 7f8edf59e700 1 -- 172.21.15.55:0/3415506823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ee00b0790 msgr2=0x7f8ee00cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:27.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.921+0000 7f8edf59e700 1 --2- 172.21.15.55:0/3415506823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ee00b0790 0x7f8ee00cea70 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40099b0 tx=0x7f8ed4009250 comp rx=0 tx=0).stop 2023-10-06T03:52:27.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.921+0000 7f8edf59e700 1 -- 172.21.15.55:0/3415506823 shutdown_connections 2023-10-06T03:52:27.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.921+0000 7f8edf59e700 1 --2- 172.21.15.55:0/3415506823 >> v2:172.21.15.163:3300/0 conn(0x7f8ee01591e0 0x7f8ee0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:27.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.921+0000 7f8edf59e700 1 --2- 172.21.15.55:0/3415506823 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ee00b0790 0x7f8ee00cea70 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:27.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.921+0000 7f8edf59e700 1 -- 172.21.15.55:0/3415506823 >> 172.21.15.55:0/3415506823 conn(0x7f8ee0063230 msgr2=0x7f8ee0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:27.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.921+0000 7f8edf59e700 1 -- 172.21.15.55:0/3415506823 shutdown_connections 2023-10-06T03:52:27.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.921+0000 7f8edf59e700 1 -- 172.21.15.55:0/3415506823 wait complete. 2023-10-06T03:52:27.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.922+0000 7f8edf59e700 1 Processor -- start 2023-10-06T03:52:27.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.923+0000 7f8edf59e700 1 -- start start 2023-10-06T03:52:27.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.923+0000 7f8edf59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ee00b03c0 0x7f8ee0156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:27.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.923+0000 7f8edf59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ee00b0790 0x7f8ee00c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:27.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.923+0000 7f8edf59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ee01591e0 0x7f8ee016a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:27.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.923+0000 7f8edf59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee0156280 con 0x7f8ee00b0790 2023-10-06T03:52:27.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.923+0000 7f8edf59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee00c7c30 con 0x7f8ee01591e0 2023-10-06T03:52:27.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.923+0000 7f8edf59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee00c7f10 con 0x7f8ee00b03c0 2023-10-06T03:52:27.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.924+0000 7f8eddd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ee00b0790 0x7f8ee00c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:27.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.924+0000 7f8eddd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ee00b0790 0x7f8ee00c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54540/0 (socket says 172.21.15.55:54540) 2023-10-06T03:52:27.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.924+0000 7f8eddd9b700 1 -- 172.21.15.55:0/407993847 learned_addr learned my addr 172.21.15.55:0/407993847 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:27.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.924+0000 7f8eded9d700 1 --2- 172.21.15.55:0/407993847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ee01591e0 0x7f8ee016a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:27.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.924+0000 7f8ede59c700 1 --2- 172.21.15.55:0/407993847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ee00b03c0 0x7f8ee0156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:27.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.924+0000 7f8eddd9b700 1 -- 172.21.15.55:0/407993847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ee00b03c0 msgr2=0x7f8ee0156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:27.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.924+0000 7f8eddd9b700 1 --2- 172.21.15.55:0/407993847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ee00b03c0 0x7f8ee0156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:27.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.924+0000 7f8eddd9b700 1 -- 172.21.15.55:0/407993847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ee01591e0 msgr2=0x7f8ee016a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:27.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.924+0000 7f8eddd9b700 1 --2- 172.21.15.55:0/407993847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ee01591e0 0x7f8ee016a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:27.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.924+0000 7f8eddd9b700 1 -- 172.21.15.55:0/407993847 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed40094f0 con 0x7f8ee00b0790 2023-10-06T03:52:27.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.924+0000 7f8eddd9b700 1 --2- 172.21.15.55:0/407993847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ee00b0790 0x7f8ee00c75a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40030a0 tx=0x7f8ed4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:27.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.925+0000 7f8ece7fc700 1 -- 172.21.15.55:0/407993847 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4006a90 con 0x7f8ee00b0790 2023-10-06T03:52:27.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.925+0000 7f8ece7fc700 1 -- 172.21.15.55:0/407993847 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed4020070 con 0x7f8ee00b0790 2023-10-06T03:52:27.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.925+0000 7f8ece7fc700 1 -- 172.21.15.55:0/407993847 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed4011db0 con 0x7f8ee00b0790 2023-10-06T03:52:27.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.925+0000 7f8edf59e700 1 -- 172.21.15.55:0/407993847 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee00c81f0 con 0x7f8ee00b0790 2023-10-06T03:52:27.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.925+0000 7f8edf59e700 1 -- 172.21.15.55:0/407993847 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee00ca4a0 con 0x7f8ee00b0790 2023-10-06T03:52:27.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.926+0000 7f8edf59e700 1 -- 172.21.15.55:0/407993847 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee004c3c0 con 0x7f8ee00b0790 2023-10-06T03:52:27.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.927+0000 7f8ece7fc700 1 -- 172.21.15.55:0/407993847 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f8ed4022030 con 0x7f8ee00b0790 2023-10-06T03:52:27.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.928+0000 7f8ece7fc700 1 --2- 172.21.15.55:0/407993847 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8ec0067050 0x7f8ec0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:27.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.928+0000 7f8ece7fc700 1 -- 172.21.15.55:0/407993847 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f8ed40863d0 con 0x7f8ee00b0790 2023-10-06T03:52:27.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.928+0000 7f8ede59c700 1 --2- 172.21.15.55:0/407993847 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8ec0067050 0x7f8ec0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:27.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.929+0000 7f8ede59c700 1 --2- 172.21.15.55:0/407993847 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8ec0067050 0x7f8ec0069500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009af0 tx=0x7f8ed0009060 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:27.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:27.932+0000 7f8ece7fc700 1 -- 172.21.15.55:0/407993847 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ed4053500 con 0x7f8ee00b0790 2023-10-06T03:52:28.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:28.136+0000 7f8edf59e700 1 -- 172.21.15.55:0/407993847 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8ee004c3c0 con 0x7f8ee00b0790 2023-10-06T03:52:29.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.098+0000 7f8ece7fc700 1 -- 172.21.15.55:0/407993847 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v134) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8ed4023e60 con 0x7f8ee00b0790 2023-10-06T03:52:29.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.102+0000 7f8edf59e700 1 -- 172.21.15.55:0/407993847 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8ec0067050 msgr2=0x7f8ec0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:29.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.102+0000 7f8edf59e700 1 --2- 172.21.15.55:0/407993847 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8ec0067050 0x7f8ec0069500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009af0 tx=0x7f8ed0009060 comp rx=0 tx=0).stop 2023-10-06T03:52:29.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.103+0000 7f8edf59e700 1 -- 172.21.15.55:0/407993847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ee00b0790 msgr2=0x7f8ee00c75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:29.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.103+0000 7f8edf59e700 1 --2- 172.21.15.55:0/407993847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ee00b0790 0x7f8ee00c75a0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40030a0 tx=0x7f8ed4016000 comp rx=0 tx=0).stop 2023-10-06T03:52:29.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.103+0000 7f8edf59e700 1 -- 172.21.15.55:0/407993847 shutdown_connections 2023-10-06T03:52:29.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.103+0000 7f8edf59e700 1 --2- 172.21.15.55:0/407993847 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f8ec0067050 0x7f8ec0069500 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:29.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.103+0000 7f8edf59e700 1 --2- 172.21.15.55:0/407993847 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8ee01591e0 0x7f8ee016a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:29.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.103+0000 7f8edf59e700 1 --2- 172.21.15.55:0/407993847 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8ee00b03c0 0x7f8ee0156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:29.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.103+0000 7f8edf59e700 1 --2- 172.21.15.55:0/407993847 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8ee00b0790 0x7f8ee00c75a0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:29.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.103+0000 7f8edf59e700 1 -- 172.21.15.55:0/407993847 >> 172.21.15.55:0/407993847 conn(0x7f8ee0063230 msgr2=0x7f8ee0068650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:29.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.103+0000 7f8edf59e700 1 -- 172.21.15.55:0/407993847 shutdown_connections 2023-10-06T03:52:29.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.104+0000 7f8edf59e700 1 -- 172.21.15.55:0/407993847 wait complete. 2023-10-06T03:52:29.357 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2023-10-06T03:52:29.358 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T03:52:29.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.562+0000 7fe75ffff700 1 Processor -- start 2023-10-06T03:52:29.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.562+0000 7fe75ffff700 1 -- start start 2023-10-06T03:52:29.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.563+0000 7fe75ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe7601595d0 0x7fe7600ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:29.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.563+0000 7fe75ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe7600cf480 0x7fe76015d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:29.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.563+0000 7fe75ffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7600b10f0 con 0x7fe76015d760 2023-10-06T03:52:29.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.563+0000 7fe75ffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7600cd530 con 0x7fe7600cf480 2023-10-06T03:52:29.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.563+0000 7fe75ffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76015db50 con 0x7fe7601595d0 2023-10-06T03:52:29.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.564+0000 7fe75e7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe7600cf480 0x7fe76015d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:29.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.564+0000 7fe75f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe76015d760 0x7fe7601599a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49160/0 (socket says 172.21.15.55:49160) 2023-10-06T03:52:29.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.564+0000 7fe75f7fe700 1 -- 172.21.15.55:0/2193180932 learned_addr learned my addr 172.21.15.55:0/2193180932 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:29.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.564+0000 7fe75effd700 1 --2- 172.21.15.55:0/2193180932 >> v2:172.21.15.163:3300/0 conn(0x7fe7601595d0 0x7fe7600ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:29.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.564+0000 7fe75e7fc700 1 -- 172.21.15.55:0/2193180932 >> v2:172.21.15.163:3300/0 conn(0x7fe7601595d0 msgr2=0x7fe7600ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:29.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.564+0000 7fe75e7fc700 1 --2- 172.21.15.55:0/2193180932 >> v2:172.21.15.163:3300/0 conn(0x7fe7601595d0 0x7fe7600ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:29.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.564+0000 7fe75e7fc700 1 -- 172.21.15.55:0/2193180932 >> v1:172.21.15.55:6789/0 conn(0x7fe76015d760 legacy=0x7fe7601599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:29.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.564+0000 7fe75e7fc700 1 -- 172.21.15.55:0/2193180932 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76015e000 con 0x7fe7600cf480 2023-10-06T03:52:29.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.564+0000 7fe75dffb700 1 -- 172.21.15.55:0/2193180932 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618007580 0 0) 0x7fe7600b10f0 con 0x7fe76015d760 2023-10-06T03:52:29.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.564+0000 7fe75e7fc700 1 --2- 172.21.15.55:0/2193180932 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7600cf480 0x7fe76015d220 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe76006a4d0 tx=0x7fe7540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2fff810ba19b13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:29.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.565+0000 7fe75dffb700 1 -- 172.21.15.55:0/2193180932 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754011630 con 0x7fe7600cf480 2023-10-06T03:52:29.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.565+0000 7fe75dffb700 1 -- 172.21.15.55:0/2193180932 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe754011790 con 0x7fe7600cf480 2023-10-06T03:52:29.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.565+0000 7fe75dffb700 1 -- 172.21.15.55:0/2193180932 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754011b30 con 0x7fe7600cf480 2023-10-06T03:52:29.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.565+0000 7fe75ffff700 1 -- 172.21.15.55:0/2193180932 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7600cf480 msgr2=0x7fe76015d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:29.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.565+0000 7fe75ffff700 1 --2- 172.21.15.55:0/2193180932 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7600cf480 0x7fe76015d220 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe76006a4d0 tx=0x7fe7540091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:29.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.565+0000 7fe75ffff700 1 -- 172.21.15.55:0/2193180932 shutdown_connections 2023-10-06T03:52:29.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.565+0000 7fe75ffff700 1 --2- 172.21.15.55:0/2193180932 >> v2:172.21.15.163:3300/0 conn(0x7fe7601595d0 0x7fe7600ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:29.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.565+0000 7fe75ffff700 1 --2- 172.21.15.55:0/2193180932 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe7600cf480 0x7fe76015d220 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:29.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.565+0000 7fe75ffff700 1 -- 172.21.15.55:0/2193180932 >> 172.21.15.55:0/2193180932 conn(0x7fe7600d05f0 msgr2=0x7fe7600d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:29.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.565+0000 7fe75ffff700 1 -- 172.21.15.55:0/2193180932 shutdown_connections 2023-10-06T03:52:29.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.565+0000 7fe75ffff700 1 -- 172.21.15.55:0/2193180932 wait complete. 2023-10-06T03:52:29.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.567+0000 7fe75ffff700 1 Processor -- start 2023-10-06T03:52:29.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.567+0000 7fe75ffff700 1 -- start start 2023-10-06T03:52:29.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.568+0000 7fe75ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7600cf480 0x7fe7600b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:29.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.568+0000 7fe75ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7601595d0 0x7fe760169700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:29.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.568+0000 7fe75ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe76015d760 0x7fe760157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:29.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.568+0000 7fe75ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76016a6b0 con 0x7fe76015d760 2023-10-06T03:52:29.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.568+0000 7fe75ffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76016aa50 con 0x7fe7601595d0 2023-10-06T03:52:29.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.568+0000 7fe75ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe760169e10 con 0x7fe7600cf480 2023-10-06T03:52:29.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.568+0000 7fe75f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe76015d760 0x7fe760157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:29.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.568+0000 7fe75f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe76015d760 0x7fe760157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54554/0 (socket says 172.21.15.55:54554) 2023-10-06T03:52:29.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.568+0000 7fe75f7fe700 1 -- 172.21.15.55:0/2576572437 learned_addr learned my addr 172.21.15.55:0/2576572437 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:29.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.568+0000 7fe75effd700 1 --2- 172.21.15.55:0/2576572437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7600cf480 0x7fe7600b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:29.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.568+0000 7fe75e7fc700 1 --2- 172.21.15.55:0/2576572437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7601595d0 0x7fe760169700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:29.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.569+0000 7fe75f7fe700 1 -- 172.21.15.55:0/2576572437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7600cf480 msgr2=0x7fe7600b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:29.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.569+0000 7fe75f7fe700 1 --2- 172.21.15.55:0/2576572437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7600cf480 0x7fe7600b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:29.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.569+0000 7fe75f7fe700 1 -- 172.21.15.55:0/2576572437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7601595d0 msgr2=0x7fe760169700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:29.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.569+0000 7fe75f7fe700 1 --2- 172.21.15.55:0/2576572437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7601595d0 0x7fe760169700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:29.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.569+0000 7fe75f7fe700 1 -- 172.21.15.55:0/2576572437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe754009420 con 0x7fe76015d760 2023-10-06T03:52:29.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.569+0000 7fe75f7fe700 1 --2- 172.21.15.55:0/2576572437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe76015d760 0x7fe760157d80 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe75800cc10 tx=0x7fe758001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:29.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.569+0000 7fe746ffd700 1 -- 172.21.15.55:0/2576572437 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75800a850 con 0x7fe76015d760 2023-10-06T03:52:29.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.569+0000 7fe746ffd700 1 -- 172.21.15.55:0/2576572437 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75800a9d0 con 0x7fe76015d760 2023-10-06T03:52:29.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.569+0000 7fe75ffff700 1 -- 172.21.15.55:0/2576572437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe76016a0f0 con 0x7fe76015d760 2023-10-06T03:52:29.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.569+0000 7fe746ffd700 1 -- 172.21.15.55:0/2576572437 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75800ad90 con 0x7fe76015d760 2023-10-06T03:52:29.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.569+0000 7fe75ffff700 1 -- 172.21.15.55:0/2576572437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7601582c0 con 0x7fe76015d760 2023-10-06T03:52:29.641 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.570+0000 7fe75ffff700 1 -- 172.21.15.55:0/2576572437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe76016abe0 con 0x7fe76015d760 2023-10-06T03:52:29.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.572+0000 7fe746ffd700 1 -- 172.21.15.55:0/2576572437 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fe758007300 con 0x7fe76015d760 2023-10-06T03:52:29.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.573+0000 7fe746ffd700 1 --2- 172.21.15.55:0/2576572437 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe74c067050 0x7fe74c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:29.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.573+0000 7fe746ffd700 1 -- 172.21.15.55:0/2576572437 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5968+0+0 (secure 0 0 0) 0x7fe758085490 con 0x7fe76015d760 2023-10-06T03:52:29.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.574+0000 7fe75effd700 1 --2- 172.21.15.55:0/2576572437 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe74c067050 0x7fe74c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:29.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.575+0000 7fe75effd700 1 --2- 172.21.15.55:0/2576572437 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe74c067050 0x7fe74c069500 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe750009da0 tx=0x7fe750009120 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:29.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.577+0000 7fe746ffd700 1 -- 172.21.15.55:0/2576572437 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe758052670 con 0x7fe76015d760 2023-10-06T03:52:29.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:29.778+0000 7fe75ffff700 1 -- 172.21.15.55:0/2576572437 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe76004c3c0 con 0x7fe76015d760 2023-10-06T03:52:30.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.103+0000 7fe746ffd700 1 -- 172.21.15.55:0/2576572437 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v135) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe758055c90 con 0x7fe76015d760 2023-10-06T03:52:30.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.114+0000 7fe75ffff700 1 -- 172.21.15.55:0/2576572437 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe74c067050 msgr2=0x7fe74c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:30.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.114+0000 7fe75ffff700 1 --2- 172.21.15.55:0/2576572437 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe74c067050 0x7fe74c069500 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe750009da0 tx=0x7fe750009120 comp rx=0 tx=0).stop 2023-10-06T03:52:30.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.114+0000 7fe75ffff700 1 -- 172.21.15.55:0/2576572437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe76015d760 msgr2=0x7fe760157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:30.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.114+0000 7fe75ffff700 1 --2- 172.21.15.55:0/2576572437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe76015d760 0x7fe760157d80 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fe75800cc10 tx=0x7fe758001970 comp rx=0 tx=0).stop 2023-10-06T03:52:30.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.115+0000 7fe75ffff700 1 -- 172.21.15.55:0/2576572437 shutdown_connections 2023-10-06T03:52:30.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.115+0000 7fe75ffff700 1 --2- 172.21.15.55:0/2576572437 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe74c067050 0x7fe74c069500 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:30.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.115+0000 7fe75ffff700 1 --2- 172.21.15.55:0/2576572437 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe76015d760 0x7fe760157d80 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:30.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.115+0000 7fe75ffff700 1 --2- 172.21.15.55:0/2576572437 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe7600cf480 0x7fe7600b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:30.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.115+0000 7fe75ffff700 1 --2- 172.21.15.55:0/2576572437 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe7601595d0 0x7fe760169700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:30.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.115+0000 7fe75ffff700 1 -- 172.21.15.55:0/2576572437 >> 172.21.15.55:0/2576572437 conn(0x7fe7600d05f0 msgr2=0x7fe760069570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:30.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.119+0000 7fe75ffff700 1 -- 172.21.15.55:0/2576572437 shutdown_connections 2023-10-06T03:52:30.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.119+0000 7fe75ffff700 1 -- 172.21.15.55:0/2576572437 wait complete. 2023-10-06T03:52:30.127 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2023-10-06T03:52:30.151 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T03:52:30.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.385+0000 7f7bd659d700 1 Processor -- start 2023-10-06T03:52:30.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.385+0000 7f7bd659d700 1 -- start start 2023-10-06T03:52:30.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.386+0000 7f7bd659d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bd01407e0 0x7f7bd006a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:30.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.386+0000 7f7bd659d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd00c6820 con 0x7f7bd0140bb0 2023-10-06T03:52:30.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.386+0000 7f7bd659d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd00c6de0 con 0x7f7bd006dc00 2023-10-06T03:52:30.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.386+0000 7f7bd659d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd0141270 con 0x7f7bd01407e0 2023-10-06T03:52:30.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.386+0000 7f7bd4d9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7bd0140bb0 0x7f7bd006d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49172/0 (socket says 172.21.15.55:49172) 2023-10-06T03:52:30.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.386+0000 7f7bd4d9a700 1 -- 172.21.15.55:0/3110711324 learned_addr learned my addr 172.21.15.55:0/3110711324 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:30.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bd559b700 1 --2- 172.21.15.55:0/3110711324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bd01407e0 0x7f7bd006a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:30.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3187200227 0 0) 0x7f7bd00c6820 con 0x7f7bd0140bb0 2023-10-06T03:52:30.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bbc003570 con 0x7f7bd0140bb0 2023-10-06T03:52:30.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042014910 0 0) 0x7f7bd00c6de0 con 0x7f7bd006dc00 2023-10-06T03:52:30.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bd00c6820 con 0x7f7bd006dc00 2023-10-06T03:52:30.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 545237593 0 0) 0x7f7bbc003570 con 0x7f7bd0140bb0 2023-10-06T03:52:30.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bd01407e0 msgr2=0x7f7bd006a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:30.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 --2- 172.21.15.55:0/3110711324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bd01407e0 0x7f7bd006a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:30.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 >> v1:172.21.15.163:6789/0 conn(0x7f7bd006dc00 legacy=0x7f7bd00c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:30.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd006acf0 con 0x7f7bd0140bb0 2023-10-06T03:52:30.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f7bc0001470 con 0x7f7bd0140bb0 2023-10-06T03:52:30.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 >> v1:172.21.15.55:6789/0 conn(0x7f7bd0140bb0 legacy=0x7f7bd006d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:30.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 --2- 172.21.15.55:0/3110711324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bd01407e0 0x7f7bbc003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:30.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 --2- 172.21.15.55:0/3110711324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bbc004180 0x7f7bbc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:30.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 --2- 172.21.15.55:0/3110711324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bbc004d50 0x7f7bbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:30.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd0141270 con 0x7f7bbc004180 2023-10-06T03:52:30.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc00bba0 con 0x7f7bbc004d50 2023-10-06T03:52:30.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.387+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc00bd60 con 0x7f7bd01407e0 2023-10-06T03:52:30.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.388+0000 7f7bd4d9a700 1 --2- 172.21.15.55:0/3110711324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bbc004180 0x7f7bbc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:30.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.388+0000 7f7bd559b700 1 --2- 172.21.15.55:0/3110711324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bd01407e0 0x7f7bbc003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:30.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.388+0000 7f7bd5d9c700 1 --2- 172.21.15.55:0/3110711324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bbc004d50 0x7f7bbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:30.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.388+0000 7f7bd559b700 1 -- 172.21.15.55:0/3110711324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bbc004d50 msgr2=0x7f7bbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:30.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.388+0000 7f7bd559b700 1 --2- 172.21.15.55:0/3110711324 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bbc004d50 0x7f7bbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:30.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.388+0000 7f7bd559b700 1 -- 172.21.15.55:0/3110711324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bbc004180 msgr2=0x7f7bbc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:30.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.388+0000 7f7bd559b700 1 --2- 172.21.15.55:0/3110711324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bbc004180 0x7f7bbc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:30.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.388+0000 7f7bd559b700 1 -- 172.21.15.55:0/3110711324 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc003570 con 0x7f7bd01407e0 2023-10-06T03:52:30.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.388+0000 7f7bd4d9a700 1 --2- 172.21.15.55:0/3110711324 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bbc004180 0x7f7bbc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:52:30.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.388+0000 7f7bd559b700 1 --2- 172.21.15.55:0/3110711324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bd01407e0 0x7f7bbc003c20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009c00 tx=0x7f7bcc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbef71b37a691a10 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:30.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.389+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc00e710 con 0x7f7bd01407e0 2023-10-06T03:52:30.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.389+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bcc00e870 con 0x7f7bd01407e0 2023-10-06T03:52:30.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.389+0000 7f7bc7fff700 1 -- 172.21.15.55:0/3110711324 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc00ec30 con 0x7f7bd01407e0 2023-10-06T03:52:30.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.389+0000 7f7bd659d700 1 -- 172.21.15.55:0/3110711324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bd01407e0 msgr2=0x7f7bbc003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:30.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.389+0000 7f7bd659d700 1 --2- 172.21.15.55:0/3110711324 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bd01407e0 0x7f7bbc003c20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009c00 tx=0x7f7bcc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:30.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.389+0000 7f7bd5d9c700 1 -- 172.21.15.55:0/3110711324 reap_dead start 2023-10-06T03:52:30.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.389+0000 7f7bd659d700 1 -- 172.21.15.55:0/3110711324 shutdown_connections 2023-10-06T03:52:30.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.389+0000 7f7bd659d700 1 -- 172.21.15.55:0/3110711324 >> 172.21.15.55:0/3110711324 conn(0x7f7bd00cc1e0 msgr2=0x7f7bd00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:30.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.389+0000 7f7bd659d700 1 -- 172.21.15.55:0/3110711324 shutdown_connections 2023-10-06T03:52:30.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.389+0000 7f7bd659d700 1 -- 172.21.15.55:0/3110711324 wait complete. 2023-10-06T03:52:30.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.391+0000 7f7bd659d700 1 Processor -- start 2023-10-06T03:52:30.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.391+0000 7f7bd659d700 1 -- start start 2023-10-06T03:52:30.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd659d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bd01407e0 0x7f7bd013b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:30.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd659d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bd00c3290 0x7f7bd013d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:30.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd659d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bd013d990 0x7f7bd013df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:30.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd659d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd015d180 con 0x7f7bd00c3290 2023-10-06T03:52:30.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd659d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd015c8e0 con 0x7f7bd013d990 2023-10-06T03:52:30.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd659d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd015cbc0 con 0x7f7bd01407e0 2023-10-06T03:52:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd4d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bd00c3290 0x7f7bd013d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd4d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bd00c3290 0x7f7bd013d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54578/0 (socket says 172.21.15.55:54578) 2023-10-06T03:52:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd4d9a700 1 -- 172.21.15.55:0/2743684877 learned_addr learned my addr 172.21.15.55:0/2743684877 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd559b700 1 --2- 172.21.15.55:0/2743684877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bd01407e0 0x7f7bd013b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd5d9c700 1 --2- 172.21.15.55:0/2743684877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bd013d990 0x7f7bd013df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:30.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd4d9a700 1 -- 172.21.15.55:0/2743684877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bd01407e0 msgr2=0x7f7bd013b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd4d9a700 1 --2- 172.21.15.55:0/2743684877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bd01407e0 0x7f7bd013b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd4d9a700 1 -- 172.21.15.55:0/2743684877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bd013d990 msgr2=0x7f7bd013df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd4d9a700 1 --2- 172.21.15.55:0/2743684877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bd013d990 0x7f7bd013df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.392+0000 7f7bd4d9a700 1 -- 172.21.15.55:0/2743684877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd015cea0 con 0x7f7bd00c3290 2023-10-06T03:52:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.393+0000 7f7bd4d9a700 1 --2- 172.21.15.55:0/2743684877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bd00c3290 0x7f7bd013d450 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0007410 tx=0x7f7bc0001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.393+0000 7f7bc57fa700 1 -- 172.21.15.55:0/2743684877 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0018030 con 0x7f7bd00c3290 2023-10-06T03:52:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.393+0000 7f7bc57fa700 1 -- 172.21.15.55:0/2743684877 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc000f870 con 0x7f7bd00c3290 2023-10-06T03:52:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.393+0000 7f7bd659d700 1 -- 172.21.15.55:0/2743684877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd013bba0 con 0x7f7bd00c3290 2023-10-06T03:52:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.393+0000 7f7bc57fa700 1 -- 172.21.15.55:0/2743684877 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc000fbf0 con 0x7f7bd00c3290 2023-10-06T03:52:30.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.393+0000 7f7bd659d700 1 -- 172.21.15.55:0/2743684877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd015c4f0 con 0x7f7bd00c3290 2023-10-06T03:52:30.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.394+0000 7f7bd659d700 1 -- 172.21.15.55:0/2743684877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd013ba00 con 0x7f7bd00c3290 2023-10-06T03:52:30.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.396+0000 7f7bc57fa700 1 -- 172.21.15.55:0/2743684877 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f7bc001b070 con 0x7f7bd00c3290 2023-10-06T03:52:30.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.397+0000 7f7bc57fa700 1 --2- 172.21.15.55:0/2743684877 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7bb4067050 0x7f7bb4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:30.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.397+0000 7f7bc57fa700 1 -- 172.21.15.55:0/2743684877 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6350+0+0 (secure 0 0 0) 0x7f7bc0019070 con 0x7f7bd00c3290 2023-10-06T03:52:30.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.397+0000 7f7bd559b700 1 --2- 172.21.15.55:0/2743684877 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7bb4067050 0x7f7bb4069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:30.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.398+0000 7f7bd559b700 1 --2- 172.21.15.55:0/2743684877 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7bb4067050 0x7f7bb4069500 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0091e0 tx=0x7f7bcc01a000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:30.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.401+0000 7f7bc57fa700 1 -- 172.21.15.55:0/2743684877 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7bc0052070 con 0x7f7bd00c3290 2023-10-06T03:52:30.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:30.647+0000 7f7bd659d700 1 -- 172.21.15.55:0/2743684877 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7bd013ba00 con 0x7f7bd00c3290 2023-10-06T03:52:31.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.128+0000 7f7bc57fa700 1 -- 172.21.15.55:0/2743684877 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v66) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7bd013ba00 con 0x7f7bd00c3290 2023-10-06T03:52:31.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.130+0000 7f7bd659d700 1 -- 172.21.15.55:0/2743684877 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7bb4067050 msgr2=0x7f7bb4069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:31.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.130+0000 7f7bd659d700 1 --2- 172.21.15.55:0/2743684877 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7bb4067050 0x7f7bb4069500 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0091e0 tx=0x7f7bcc01a000 comp rx=0 tx=0).stop 2023-10-06T03:52:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.130+0000 7f7bd659d700 1 -- 172.21.15.55:0/2743684877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bd00c3290 msgr2=0x7f7bd013d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.130+0000 7f7bd659d700 1 --2- 172.21.15.55:0/2743684877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bd00c3290 0x7f7bd013d450 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0007410 tx=0x7f7bc0001df0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.131+0000 7f7bd659d700 1 -- 172.21.15.55:0/2743684877 shutdown_connections 2023-10-06T03:52:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.131+0000 7f7bd659d700 1 --2- 172.21.15.55:0/2743684877 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f7bb4067050 0x7f7bb4069500 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.131+0000 7f7bd659d700 1 --2- 172.21.15.55:0/2743684877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7bd013d990 0x7f7bd013df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.131+0000 7f7bd659d700 1 --2- 172.21.15.55:0/2743684877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7bd01407e0 0x7f7bd013b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.131+0000 7f7bd659d700 1 --2- 172.21.15.55:0/2743684877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7bd00c3290 0x7f7bd013d450 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.131+0000 7f7bd659d700 1 -- 172.21.15.55:0/2743684877 >> 172.21.15.55:0/2743684877 conn(0x7f7bd00cc1e0 msgr2=0x7f7bd00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:31.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.131+0000 7f7bd659d700 1 -- 172.21.15.55:0/2743684877 shutdown_connections 2023-10-06T03:52:31.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.131+0000 7f7bd659d700 1 -- 172.21.15.55:0/2743684877 wait complete. 2023-10-06T03:52:31.134 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 17 and data pool 18 2023-10-06T03:52:31.155 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:52:31.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.344+0000 7f3d30f9d700 1 Processor -- start 2023-10-06T03:52:31.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.344+0000 7f3d30f9d700 1 -- start start 2023-10-06T03:52:31.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.345+0000 7f3d30f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3d2c156310 0x7f3d2c0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:31.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.345+0000 7f3d30f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3d2c0ae710 0x7f3d2c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:31.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.345+0000 7f3d30f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d2c0ca860 con 0x7f3d2c1566e0 2023-10-06T03:52:31.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.345+0000 7f3d30f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d2c0cab40 con 0x7f3d2c0ae710 2023-10-06T03:52:31.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.345+0000 7f3d30f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d2c0cae20 con 0x7f3d2c156310 2023-10-06T03:52:31.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.345+0000 7f3d2bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3d2c0ae710 0x7f3d2c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:31.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.346+0000 7f3d2bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3d2c0ae710 0x7f3d2c0c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54582/0 (socket says 172.21.15.55:54582) 2023-10-06T03:52:31.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.346+0000 7f3d2bfff700 1 -- 172.21.15.55:0/2632619125 learned_addr learned my addr 172.21.15.55:0/2632619125 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:31.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.346+0000 7f3d2bfff700 1 -- 172.21.15.55:0/2632619125 >> v2:172.21.15.163:3300/0 conn(0x7f3d2c156310 msgr2=0x7f3d2c0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:31.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.346+0000 7f3d2b7fe700 1 --2- 172.21.15.55:0/2632619125 >> v2:172.21.15.163:3300/0 conn(0x7f3d2c156310 0x7f3d2c0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:31.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.346+0000 7f3d2bfff700 1 --2- 172.21.15.55:0/2632619125 >> v2:172.21.15.163:3300/0 conn(0x7f3d2c156310 0x7f3d2c0c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.346+0000 7f3d2bfff700 1 -- 172.21.15.55:0/2632619125 >> v1:172.21.15.163:6789/0 conn(0x7f3d2c1566e0 legacy=0x7f3d2c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:31.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.346+0000 7f3d2bfff700 1 -- 172.21.15.55:0/2632619125 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c0cb100 con 0x7f3d2c0ae710 2023-10-06T03:52:31.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.346+0000 7f3d2b7fe700 1 --2- 172.21.15.55:0/2632619125 >> v2:172.21.15.163:3300/0 conn(0x7f3d2c156310 0x7f3d2c0c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:31.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.346+0000 7f3d2bfff700 1 --2- 172.21.15.55:0/2632619125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c0ae710 0x7f3d2c0c6830 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0cce20 tx=0x7f3d24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0bc902b7471187 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:31.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.347+0000 7f3d2a7fc700 1 -- 172.21.15.55:0/2632619125 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2400e070 con 0x7f3d2c0ae710 2023-10-06T03:52:31.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.347+0000 7f3d2a7fc700 1 -- 172.21.15.55:0/2632619125 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2400ba10 con 0x7f3d2c0ae710 2023-10-06T03:52:31.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.347+0000 7f3d2a7fc700 1 -- 172.21.15.55:0/2632619125 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d240092e0 con 0x7f3d2c0ae710 2023-10-06T03:52:31.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.347+0000 7f3d30f9d700 1 -- 172.21.15.55:0/2632619125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c0ae710 msgr2=0x7f3d2c0c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:31.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.347+0000 7f3d30f9d700 1 --2- 172.21.15.55:0/2632619125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c0ae710 0x7f3d2c0c6830 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0cce20 tx=0x7f3d24009840 comp rx=0 tx=0).stop 2023-10-06T03:52:31.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.347+0000 7f3d30f9d700 1 -- 172.21.15.55:0/2632619125 shutdown_connections 2023-10-06T03:52:31.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.347+0000 7f3d30f9d700 1 --2- 172.21.15.55:0/2632619125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c0ae710 0x7f3d2c0c6830 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.347+0000 7f3d30f9d700 1 --2- 172.21.15.55:0/2632619125 >> v2:172.21.15.163:3300/0 conn(0x7f3d2c156310 0x7f3d2c0c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.347+0000 7f3d30f9d700 1 -- 172.21.15.55:0/2632619125 >> 172.21.15.55:0/2632619125 conn(0x7f3d2c0cc1f0 msgr2=0x7f3d2c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:31.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.347+0000 7f3d30f9d700 1 -- 172.21.15.55:0/2632619125 shutdown_connections 2023-10-06T03:52:31.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.347+0000 7f3d30f9d700 1 -- 172.21.15.55:0/2632619125 wait complete. 2023-10-06T03:52:31.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.349+0000 7f3d30f9d700 1 Processor -- start 2023-10-06T03:52:31.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.349+0000 7f3d30f9d700 1 -- start start 2023-10-06T03:52:31.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.350+0000 7f3d30f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d2c0ae710 0x7f3d2c148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:31.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.350+0000 7f3d30f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d2c156310 0x7f3d2c149840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:31.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.350+0000 7f3d30f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1566e0 0x7f3d2c154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:31.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.350+0000 7f3d30f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c148dc0 con 0x7f3d2c1566e0 2023-10-06T03:52:31.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.350+0000 7f3d30f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c0afd00 con 0x7f3d2c156310 2023-10-06T03:52:31.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.350+0000 7f3d30f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c0af110 con 0x7f3d2c0ae710 2023-10-06T03:52:31.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.350+0000 7f3d2bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1566e0 0x7f3d2c154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:31.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.350+0000 7f3d2bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1566e0 0x7f3d2c154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54592/0 (socket says 172.21.15.55:54592) 2023-10-06T03:52:31.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.350+0000 7f3d2bfff700 1 -- 172.21.15.55:0/1302187352 learned_addr learned my addr 172.21.15.55:0/1302187352 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:31.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.350+0000 7f3d2b7fe700 1 --2- 172.21.15.55:0/1302187352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d2c0ae710 0x7f3d2c148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:31.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.350+0000 7f3d2affd700 1 --2- 172.21.15.55:0/1302187352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d2c156310 0x7f3d2c149840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:31.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.351+0000 7f3d2bfff700 1 -- 172.21.15.55:0/1302187352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d2c0ae710 msgr2=0x7f3d2c148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:31.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.351+0000 7f3d2bfff700 1 --2- 172.21.15.55:0/1302187352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d2c0ae710 0x7f3d2c148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.351+0000 7f3d2bfff700 1 -- 172.21.15.55:0/1302187352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d2c156310 msgr2=0x7f3d2c149840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:31.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.351+0000 7f3d2bfff700 1 --2- 172.21.15.55:0/1302187352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d2c156310 0x7f3d2c149840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.351+0000 7f3d2bfff700 1 -- 172.21.15.55:0/1302187352 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d24009ae0 con 0x7f3d2c1566e0 2023-10-06T03:52:31.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.351+0000 7f3d2bfff700 1 --2- 172.21.15.55:0/1302187352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1566e0 0x7f3d2c154d10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009df0 tx=0x7f3d24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:31.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.351+0000 7f3d137fe700 1 -- 172.21.15.55:0/1302187352 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24012e60 con 0x7f3d2c1566e0 2023-10-06T03:52:31.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.351+0000 7f3d137fe700 1 -- 172.21.15.55:0/1302187352 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d240092e0 con 0x7f3d2c1566e0 2023-10-06T03:52:31.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.351+0000 7f3d137fe700 1 -- 172.21.15.55:0/1302187352 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2401d650 con 0x7f3d2c1566e0 2023-10-06T03:52:31.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.351+0000 7f3d30f9d700 1 -- 172.21.15.55:0/1302187352 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2c0af3d0 con 0x7f3d2c1566e0 2023-10-06T03:52:31.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.351+0000 7f3d30f9d700 1 -- 172.21.15.55:0/1302187352 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2c0af990 con 0x7f3d2c1566e0 2023-10-06T03:52:31.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.352+0000 7f3d30f9d700 1 -- 172.21.15.55:0/1302187352 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2c0afeb0 con 0x7f3d2c1566e0 2023-10-06T03:52:31.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.353+0000 7f3d137fe700 1 -- 172.21.15.55:0/1302187352 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f3d2401d9c0 con 0x7f3d2c1566e0 2023-10-06T03:52:31.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.354+0000 7f3d137fe700 1 --2- 172.21.15.55:0/1302187352 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d18067050 0x7f3d18069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:31.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.354+0000 7f3d137fe700 1 -- 172.21.15.55:0/1302187352 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f3d24086420 con 0x7f3d2c1566e0 2023-10-06T03:52:31.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.354+0000 7f3d2b7fe700 1 --2- 172.21.15.55:0/1302187352 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d18067050 0x7f3d18069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:31.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.355+0000 7f3d2b7fe700 1 --2- 172.21.15.55:0/1302187352 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d18067050 0x7f3d18069500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c006fd0 tx=0x7f3d1c008040 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:31.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.358+0000 7f3d137fe700 1 -- 172.21.15.55:0/1302187352 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d2c0afeb0 con 0x7f3d2c1566e0 2023-10-06T03:52:31.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.561+0000 7f3d30f9d700 1 -- 172.21.15.55:0/1302187352 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d2c0afeb0 con 0x7f3d2c1566e0 2023-10-06T03:52:31.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.562+0000 7f3d137fe700 1 -- 172.21.15.55:0/1302187352 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+17536 (secure 0 0 0) 0x7f3d2c0afeb0 con 0x7f3d2c1566e0 2023-10-06T03:52:31.594 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:31.594 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":136,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:52:30.648367+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:52:28.177340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T03:52:29.819781+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:52:31.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.564+0000 7f3d30f9d700 1 -- 172.21.15.55:0/1302187352 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d18067050 msgr2=0x7f3d18069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.564+0000 7f3d30f9d700 1 --2- 172.21.15.55:0/1302187352 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d18067050 0x7f3d18069500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c006fd0 tx=0x7f3d1c008040 comp rx=0 tx=0).stop 2023-10-06T03:52:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.564+0000 7f3d30f9d700 1 -- 172.21.15.55:0/1302187352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1566e0 msgr2=0x7f3d2c154d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.564+0000 7f3d30f9d700 1 --2- 172.21.15.55:0/1302187352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1566e0 0x7f3d2c154d10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009df0 tx=0x7f3d24017000 comp rx=0 tx=0).stop 2023-10-06T03:52:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.565+0000 7f3d30f9d700 1 -- 172.21.15.55:0/1302187352 shutdown_connections 2023-10-06T03:52:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.565+0000 7f3d30f9d700 1 --2- 172.21.15.55:0/1302187352 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3d18067050 0x7f3d18069500 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.565+0000 7f3d30f9d700 1 --2- 172.21.15.55:0/1302187352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3d2c1566e0 0x7f3d2c154d10 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.565+0000 7f3d30f9d700 1 --2- 172.21.15.55:0/1302187352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3d2c0ae710 0x7f3d2c148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.565+0000 7f3d30f9d700 1 --2- 172.21.15.55:0/1302187352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3d2c156310 0x7f3d2c149840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.565+0000 7f3d30f9d700 1 -- 172.21.15.55:0/1302187352 >> 172.21.15.55:0/1302187352 conn(0x7f3d2c0cc1f0 msgr2=0x7f3d2c0c19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:31.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.565+0000 7f3d30f9d700 1 -- 172.21.15.55:0/1302187352 shutdown_connections 2023-10-06T03:52:31.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.565+0000 7f3d30f9d700 1 -- 172.21.15.55:0/1302187352 wait complete. 2023-10-06T03:52:31.599 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:52:31.801 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.799+0000 7f911f39d700 1 Processor -- start 2023-10-06T03:52:31.824 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.799+0000 7f911f39d700 1 -- start start 2023-10-06T03:52:31.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.801+0000 7f911f39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9118156310 0x7f91180c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:31.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.801+0000 7f911f39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f91181566e0 0x7f91180ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:31.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.801+0000 7f911f39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91180ca860 con 0x7f91180ca300 2023-10-06T03:52:31.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.801+0000 7f911f39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91180cab40 con 0x7f9118156310 2023-10-06T03:52:31.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.801+0000 7f911f39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91180cae20 con 0x7f91181566e0 2023-10-06T03:52:31.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.801+0000 7f911e39b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9118156310 0x7f91180c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:31.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.801+0000 7f911e39b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9118156310 0x7f91180c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54608/0 (socket says 172.21.15.55:54608) 2023-10-06T03:52:31.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.801+0000 7f911e39b700 1 -- 172.21.15.55:0/92211778 learned_addr learned my addr 172.21.15.55:0/92211778 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:31.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.801+0000 7f911db9a700 1 --2- 172.21.15.55:0/92211778 >> v2:172.21.15.163:3300/0 conn(0x7f91181566e0 0x7f91180ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:31.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.801+0000 7f911e39b700 1 -- 172.21.15.55:0/92211778 >> v2:172.21.15.163:3300/0 conn(0x7f91181566e0 msgr2=0x7f91180ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:31.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.801+0000 7f911e39b700 1 --2- 172.21.15.55:0/92211778 >> v2:172.21.15.163:3300/0 conn(0x7f91181566e0 0x7f91180ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.801+0000 7f911e39b700 1 -- 172.21.15.55:0/92211778 >> v1:172.21.15.55:6789/0 conn(0x7f91180ca300 legacy=0x7f91180c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:31.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.802+0000 7f911e39b700 1 -- 172.21.15.55:0/92211778 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91180cb100 con 0x7f9118156310 2023-10-06T03:52:31.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.802+0000 7f911e39b700 1 --2- 172.21.15.55:0/92211778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9118156310 0x7f91180c9dc0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f911806c000 tx=0x7f9114009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4913cee44299cb76 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:31.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.802+0000 7f911d399700 1 -- 172.21.15.55:0/92211778 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911400b860 con 0x7f9118156310 2023-10-06T03:52:31.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.802+0000 7f911d399700 1 -- 172.21.15.55:0/92211778 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911400b9c0 con 0x7f9118156310 2023-10-06T03:52:31.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.802+0000 7f911d399700 1 -- 172.21.15.55:0/92211778 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911400bd60 con 0x7f9118156310 2023-10-06T03:52:31.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.803+0000 7f911f39d700 1 -- 172.21.15.55:0/92211778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9118156310 msgr2=0x7f91180c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:31.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.803+0000 7f911f39d700 1 --2- 172.21.15.55:0/92211778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9118156310 0x7f91180c9dc0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f911806c000 tx=0x7f9114009250 comp rx=0 tx=0).stop 2023-10-06T03:52:31.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.803+0000 7f911f39d700 1 -- 172.21.15.55:0/92211778 shutdown_connections 2023-10-06T03:52:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.803+0000 7f911f39d700 1 --2- 172.21.15.55:0/92211778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9118156310 0x7f91180c9dc0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.803+0000 7f911f39d700 1 --2- 172.21.15.55:0/92211778 >> v2:172.21.15.163:3300/0 conn(0x7f91181566e0 0x7f91180ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.803+0000 7f911f39d700 1 -- 172.21.15.55:0/92211778 >> 172.21.15.55:0/92211778 conn(0x7f91180cc1f0 msgr2=0x7f91180cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.803+0000 7f911f39d700 1 -- 172.21.15.55:0/92211778 shutdown_connections 2023-10-06T03:52:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.803+0000 7f911f39d700 1 -- 172.21.15.55:0/92211778 wait complete. 2023-10-06T03:52:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.804+0000 7f911f39d700 1 Processor -- start 2023-10-06T03:52:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.804+0000 7f911f39d700 1 -- start start 2023-10-06T03:52:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.805+0000 7f911f39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91180ca300 0x7f91180c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.805+0000 7f911f39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9118156310 0x7f911816a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.805+0000 7f911f39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91181566e0 0x7f911816d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:31.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.805+0000 7f911f39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911816e2e0 con 0x7f9118156310 2023-10-06T03:52:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.805+0000 7f911f39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9118169d30 con 0x7f91180ca300 2023-10-06T03:52:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.805+0000 7f911f39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911816a290 con 0x7f91181566e0 2023-10-06T03:52:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.806+0000 7f911db9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9118156310 0x7f911816a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.806+0000 7f911db9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9118156310 0x7f911816a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54616/0 (socket says 172.21.15.55:54616) 2023-10-06T03:52:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.806+0000 7f911db9a700 1 -- 172.21.15.55:0/3682196222 learned_addr learned my addr 172.21.15.55:0/3682196222 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.806+0000 7f911e39b700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91180ca300 0x7f91180c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.806+0000 7f911eb9c700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91181566e0 0x7f911816d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:31.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.806+0000 7f911db9a700 1 -- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91181566e0 msgr2=0x7f911816d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.806+0000 7f911db9a700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91181566e0 0x7f911816d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.806+0000 7f911db9a700 1 -- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91180ca300 msgr2=0x7f91180c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.806+0000 7f911db9a700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91180ca300 0x7f91180c26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.806+0000 7f911db9a700 1 -- 172.21.15.55:0/3682196222 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91140094f0 con 0x7f9118156310 2023-10-06T03:52:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.806+0000 7f911eb9c700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91181566e0 0x7f911816d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.806+0000 7f911db9a700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9118156310 0x7f911816a720 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f910800d860 tx=0x7f9108008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.807+0000 7f910e7fc700 1 -- 172.21.15.55:0/3682196222 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108012630 con 0x7f9118156310 2023-10-06T03:52:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.807+0000 7f910e7fc700 1 -- 172.21.15.55:0/3682196222 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9108012790 con 0x7f9118156310 2023-10-06T03:52:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.807+0000 7f911f39d700 1 -- 172.21.15.55:0/3682196222 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911816dd20 con 0x7f9118156310 2023-10-06T03:52:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.807+0000 7f910e7fc700 1 -- 172.21.15.55:0/3682196222 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108012b30 con 0x7f9118156310 2023-10-06T03:52:31.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.807+0000 7f911f39d700 1 -- 172.21.15.55:0/3682196222 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9118169f20 con 0x7f9118156310 2023-10-06T03:52:31.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.808+0000 7f911f39d700 1 -- 172.21.15.55:0/3682196222 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911804c3c0 con 0x7f9118156310 2023-10-06T03:52:31.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.809+0000 7f910e7fc700 1 -- 172.21.15.55:0/3682196222 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f910801d400 con 0x7f9118156310 2023-10-06T03:52:31.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.810+0000 7f910e7fc700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f90fc067050 0x7f90fc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:31.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.810+0000 7f910e7fc700 1 -- 172.21.15.55:0/3682196222 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f9108084c90 con 0x7f9118156310 2023-10-06T03:52:31.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.810+0000 7f911e39b700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f90fc067050 0x7f90fc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:31.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.811+0000 7f911e39b700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f90fc067050 0x7f90fc069500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f911806c000 tx=0x7f9114017000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:31.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:31.814+0000 7f910e7fc700 1 -- 172.21.15.55:0/3682196222 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9108051b00 con 0x7f9118156310 2023-10-06T03:52:32.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.017+0000 7f911f39d700 1 -- 172.21.15.55:0/3682196222 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91180cd0b0 con 0x7f9118156310 2023-10-06T03:52:32.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.018+0000 7f910e7fc700 1 -- 172.21.15.55:0/3682196222 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+17523 (secure 0 0 0) 0x7f9108055120 con 0x7f9118156310 2023-10-06T03:52:32.020 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:32.020 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":137,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:52:31.768186+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:52:28.177340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T03:52:29.819781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:52:32.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.020+0000 7f911f39d700 1 -- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f90fc067050 msgr2=0x7f90fc069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:32.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.020+0000 7f911f39d700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f90fc067050 0x7f90fc069500 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f911806c000 tx=0x7f9114017000 comp rx=0 tx=0).stop 2023-10-06T03:52:32.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.020+0000 7f911f39d700 1 -- 172.21.15.55:0/3682196222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9118156310 msgr2=0x7f911816a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:32.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.020+0000 7f911f39d700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9118156310 0x7f911816a720 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f910800d860 tx=0x7f9108008040 comp rx=0 tx=0).stop 2023-10-06T03:52:32.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.020+0000 7f911f39d700 1 -- 172.21.15.55:0/3682196222 shutdown_connections 2023-10-06T03:52:32.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.020+0000 7f911f39d700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f90fc067050 0x7f90fc069500 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:32.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.020+0000 7f911f39d700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91181566e0 0x7f911816d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:32.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.020+0000 7f911f39d700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91180ca300 0x7f91180c26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:32.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.020+0000 7f911f39d700 1 --2- 172.21.15.55:0/3682196222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9118156310 0x7f911816a720 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:32.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.020+0000 7f911f39d700 1 -- 172.21.15.55:0/3682196222 >> 172.21.15.55:0/3682196222 conn(0x7f91180cc1f0 msgr2=0x7f91180aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:32.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.020+0000 7f911f39d700 1 -- 172.21.15.55:0/3682196222 shutdown_connections 2023-10-06T03:52:32.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.020+0000 7f911f39d700 1 -- 172.21.15.55:0/3682196222 wait complete. 2023-10-06T03:52:32.058 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T03:52:32.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.251+0000 7f6cb3fff700 1 Processor -- start 2023-10-06T03:52:32.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.251+0000 7f6cb3fff700 1 -- start start 2023-10-06T03:52:32.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.252+0000 7f6cb3fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6cb40aef00 0x7f6cb415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:32.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.252+0000 7f6cb3fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb40aea70 con 0x7f6cb406e470 2023-10-06T03:52:32.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.252+0000 7f6cb3fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb40ae480 con 0x7f6cb415a3b0 2023-10-06T03:52:32.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.252+0000 7f6cb3fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb415dd30 con 0x7f6cb40aef00 2023-10-06T03:52:32.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.253+0000 7f6cb37fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6cb406e470 0x7f6cb40c7280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:49198/0 (socket says 172.21.15.55:49198) 2023-10-06T03:52:32.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.253+0000 7f6cb37fe700 1 -- 172.21.15.55:0/1478459914 learned_addr learned my addr 172.21.15.55:0/1478459914 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:32.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.253+0000 7f6cb27fc700 1 --2- 172.21.15.55:0/1478459914 >> v2:172.21.15.163:3300/0 conn(0x7f6cb40aef00 0x7f6cb415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:32.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.253+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220795356 0 0) 0x7f6cb40aea70 con 0x7f6cb406e470 2023-10-06T03:52:32.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.253+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c9c003570 con 0x7f6cb406e470 2023-10-06T03:52:32.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.253+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2458275775 0 0) 0x7f6cb40ae480 con 0x7f6cb415a3b0 2023-10-06T03:52:32.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.253+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb40aea70 con 0x7f6cb415a3b0 2023-10-06T03:52:32.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.253+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1859105745 0 0) 0x7f6c9c003570 con 0x7f6cb406e470 2023-10-06T03:52:32.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.253+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 >> v2:172.21.15.163:3300/0 conn(0x7f6cb40aef00 msgr2=0x7f6cb415d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:32.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.253+0000 7f6cb1ffb700 1 --2- 172.21.15.55:0/1478459914 >> v2:172.21.15.163:3300/0 conn(0x7f6cb40aef00 0x7f6cb415d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:32.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.253+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 >> v1:172.21.15.163:6789/0 conn(0x7f6cb415a3b0 legacy=0x7f6cb415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:32.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb415e010 con 0x7f6cb406e470 2023-10-06T03:52:32.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f6cac0017f0 con 0x7f6cb406e470 2023-10-06T03:52:32.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 >> v1:172.21.15.55:6789/0 conn(0x7f6cb406e470 legacy=0x7f6cb40c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:32.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb1ffb700 1 --2- 172.21.15.55:0/1478459914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c9c003bf0 0x7f6c9c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:32.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb1ffb700 1 --2- 172.21.15.55:0/1478459914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c9c004810 0x7f6c9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:32.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb1ffb700 1 --2- 172.21.15.55:0/1478459914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c9c0051c0 0x7f6c9c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:32.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb415dd30 con 0x7f6c9c0051c0 2023-10-06T03:52:32.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c00c0b0 con 0x7f6c9c003bf0 2023-10-06T03:52:32.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c00c270 con 0x7f6c9c004810 2023-10-06T03:52:32.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb37fe700 1 --2- 172.21.15.55:0/1478459914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c9c0051c0 0x7f6c9c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:32.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb2ffd700 1 --2- 172.21.15.55:0/1478459914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c9c003bf0 0x7f6c9c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:32.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb27fc700 1 --2- 172.21.15.55:0/1478459914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c9c004810 0x7f6c9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:32.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb37fe700 1 -- 172.21.15.55:0/1478459914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c9c004810 msgr2=0x7f6c9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:32.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb37fe700 1 --2- 172.21.15.55:0/1478459914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c9c004810 0x7f6c9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:32.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb37fe700 1 -- 172.21.15.55:0/1478459914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c9c003bf0 msgr2=0x7f6c9c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:32.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb37fe700 1 --2- 172.21.15.55:0/1478459914 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c9c003bf0 0x7f6c9c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:32.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb37fe700 1 -- 172.21.15.55:0/1478459914 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c003570 con 0x7f6c9c0051c0 2023-10-06T03:52:32.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.254+0000 7f6cb27fc700 1 --2- 172.21.15.55:0/1478459914 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c9c004810 0x7f6c9c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:32.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.255+0000 7f6cb37fe700 1 -- 172.21.15.55:0/1478459914 reap_dead start 2023-10-06T03:52:32.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.255+0000 7f6cb37fe700 1 --2- 172.21.15.55:0/1478459914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c9c0051c0 0x7f6c9c00ba60 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00a8d0 tx=0x7f6cac002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc87b0c46ed668e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:32.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.255+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac0070d0 con 0x7f6c9c0051c0 2023-10-06T03:52:32.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.255+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac007270 con 0x7f6c9c0051c0 2023-10-06T03:52:32.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.255+0000 7f6cb1ffb700 1 -- 172.21.15.55:0/1478459914 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac007630 con 0x7f6c9c0051c0 2023-10-06T03:52:32.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.255+0000 7f6cb3fff700 1 -- 172.21.15.55:0/1478459914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c9c0051c0 msgr2=0x7f6c9c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:32.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.255+0000 7f6cb3fff700 1 --2- 172.21.15.55:0/1478459914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c9c0051c0 0x7f6c9c00ba60 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00a8d0 tx=0x7f6cac002190 comp rx=0 tx=0).stop 2023-10-06T03:52:32.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.256+0000 7f6cb3fff700 1 -- 172.21.15.55:0/1478459914 shutdown_connections 2023-10-06T03:52:32.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.256+0000 7f6cb3fff700 1 --2- 172.21.15.55:0/1478459914 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c9c0051c0 0x7f6c9c00ba60 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:32.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.256+0000 7f6cb3fff700 1 -- 172.21.15.55:0/1478459914 >> 172.21.15.55:0/1478459914 conn(0x7f6cb40c9f70 msgr2=0x7f6cb406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:32.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.256+0000 7f6cb3fff700 1 -- 172.21.15.55:0/1478459914 shutdown_connections 2023-10-06T03:52:32.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.256+0000 7f6cb3fff700 1 -- 172.21.15.55:0/1478459914 wait complete. 2023-10-06T03:52:32.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.257+0000 7f6cb3fff700 1 Processor -- start 2023-10-06T03:52:32.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.257+0000 7f6cb3fff700 1 -- start start 2023-10-06T03:52:32.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.258+0000 7f6cb3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c9c0051c0 0x7f6cb4153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:32.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.258+0000 7f6cb3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6cb40afd70 0x7f6cb4169500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:32.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.258+0000 7f6cb3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cb4169a40 0x7f6cb40b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:32.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.258+0000 7f6cb3fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40c4ab0 con 0x7f6c9c0051c0 2023-10-06T03:52:32.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.258+0000 7f6cb3fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40c5630 con 0x7f6cb40afd70 2023-10-06T03:52:32.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.258+0000 7f6cb3fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40c4d00 con 0x7f6cb4169a40 2023-10-06T03:52:32.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.258+0000 7f6cb2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c9c0051c0 0x7f6cb4153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:32.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.258+0000 7f6cb2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c9c0051c0 0x7f6cb4153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54642/0 (socket says 172.21.15.55:54642) 2023-10-06T03:52:32.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.258+0000 7f6cb2ffd700 1 -- 172.21.15.55:0/2489364072 learned_addr learned my addr 172.21.15.55:0/2489364072 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:32.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.259+0000 7f6cb37fe700 1 --2- 172.21.15.55:0/2489364072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cb4169a40 0x7f6cb40b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:32.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.259+0000 7f6cb27fc700 1 --2- 172.21.15.55:0/2489364072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6cb40afd70 0x7f6cb4169500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:32.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.259+0000 7f6cb2ffd700 1 -- 172.21.15.55:0/2489364072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cb4169a40 msgr2=0x7f6cb40b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:32.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.259+0000 7f6cb2ffd700 1 --2- 172.21.15.55:0/2489364072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cb4169a40 0x7f6cb40b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:32.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.259+0000 7f6cb2ffd700 1 -- 172.21.15.55:0/2489364072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6cb40afd70 msgr2=0x7f6cb4169500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:32.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.259+0000 7f6cb2ffd700 1 --2- 172.21.15.55:0/2489364072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6cb40afd70 0x7f6cb4169500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:32.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.259+0000 7f6cb2ffd700 1 -- 172.21.15.55:0/2489364072 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb40c4fe0 con 0x7f6c9c0051c0 2023-10-06T03:52:32.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.259+0000 7f6cb2ffd700 1 --2- 172.21.15.55:0/2489364072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c9c0051c0 0x7f6cb4153490 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4002b20 tx=0x7f6ca4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:32.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.259+0000 7f6c9affd700 1 -- 172.21.15.55:0/2489364072 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca400f9a0 con 0x7f6c9c0051c0 2023-10-06T03:52:32.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.259+0000 7f6c9affd700 1 -- 172.21.15.55:0/2489364072 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca400fb20 con 0x7f6c9c0051c0 2023-10-06T03:52:32.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.259+0000 7f6cb3fff700 1 -- 172.21.15.55:0/2489364072 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb40c52c0 con 0x7f6c9c0051c0 2023-10-06T03:52:32.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.259+0000 7f6c9affd700 1 -- 172.21.15.55:0/2489364072 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca400fee0 con 0x7f6c9c0051c0 2023-10-06T03:52:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.259+0000 7f6cb3fff700 1 -- 172.21.15.55:0/2489364072 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb40b2700 con 0x7f6c9c0051c0 2023-10-06T03:52:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.260+0000 7f6cb3fff700 1 -- 172.21.15.55:0/2489364072 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb406cfa0 con 0x7f6c9c0051c0 2023-10-06T03:52:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.261+0000 7f6c9affd700 1 -- 172.21.15.55:0/2489364072 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f6ca4013600 con 0x7f6c9c0051c0 2023-10-06T03:52:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.262+0000 7f6c9affd700 1 --2- 172.21.15.55:0/2489364072 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6ca0067050 0x7f6ca0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.262+0000 7f6c9affd700 1 -- 172.21.15.55:0/2489364072 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f6ca4084b90 con 0x7f6c9c0051c0 2023-10-06T03:52:32.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.262+0000 7f6cb27fc700 1 --2- 172.21.15.55:0/2489364072 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6ca0067050 0x7f6ca0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:32.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.263+0000 7f6cb27fc700 1 --2- 172.21.15.55:0/2489364072 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6ca0067050 0x7f6ca0069500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80030c0 tx=0x7f6ca8008040 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:32.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.266+0000 7f6c9affd700 1 -- 172.21.15.55:0/2489364072 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ca4051960 con 0x7f6c9c0051c0 2023-10-06T03:52:32.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:32.512+0000 7f6cb3fff700 1 -- 172.21.15.55:0/2489364072 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6cb40ae700 con 0x7f6c9c0051c0 2023-10-06T03:52:33.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.152+0000 7f6c9affd700 1 -- 172.21.15.55:0/2489364072 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v69) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6ca4054f80 con 0x7f6c9c0051c0 2023-10-06T03:52:33.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.155+0000 7f6cb3fff700 1 -- 172.21.15.55:0/2489364072 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6ca0067050 msgr2=0x7f6ca0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:33.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.155+0000 7f6cb3fff700 1 --2- 172.21.15.55:0/2489364072 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6ca0067050 0x7f6ca0069500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80030c0 tx=0x7f6ca8008040 comp rx=0 tx=0).stop 2023-10-06T03:52:33.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.155+0000 7f6cb3fff700 1 -- 172.21.15.55:0/2489364072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c9c0051c0 msgr2=0x7f6cb4153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:33.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.155+0000 7f6cb3fff700 1 --2- 172.21.15.55:0/2489364072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c9c0051c0 0x7f6cb4153490 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4002b20 tx=0x7f6ca4007d30 comp rx=0 tx=0).stop 2023-10-06T03:52:33.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.155+0000 7f6cb3fff700 1 -- 172.21.15.55:0/2489364072 shutdown_connections 2023-10-06T03:52:33.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.155+0000 7f6cb3fff700 1 --2- 172.21.15.55:0/2489364072 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6ca0067050 0x7f6ca0069500 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:33.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.155+0000 7f6cb3fff700 1 --2- 172.21.15.55:0/2489364072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6cb4169a40 0x7f6cb40b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:33.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.155+0000 7f6cb3fff700 1 --2- 172.21.15.55:0/2489364072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c9c0051c0 0x7f6cb4153490 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:33.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.155+0000 7f6cb3fff700 1 --2- 172.21.15.55:0/2489364072 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6cb40afd70 0x7f6cb4169500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:33.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.155+0000 7f6cb3fff700 1 -- 172.21.15.55:0/2489364072 >> 172.21.15.55:0/2489364072 conn(0x7f6cb40c9f70 msgr2=0x7f6cb415d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:33.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.155+0000 7f6cb3fff700 1 -- 172.21.15.55:0/2489364072 shutdown_connections 2023-10-06T03:52:33.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.155+0000 7f6cb3fff700 1 -- 172.21.15.55:0/2489364072 wait complete. 2023-10-06T03:52:33.176 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T03:52:33.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.369+0000 7fced099d700 1 Processor -- start 2023-10-06T03:52:33.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.369+0000 7fced099d700 1 -- start start 2023-10-06T03:52:33.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.370+0000 7fced099d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc0b4e70 0x7fcecc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:33.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.370+0000 7fced099d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcecc0cfb00 con 0x7fcecc0b16d0 2023-10-06T03:52:33.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.370+0000 7fced099d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcecc15e060 con 0x7fcecc0b1b40 2023-10-06T03:52:33.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.370+0000 7fced099d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc0b5c40 con 0x7fcecc0b4e70 2023-10-06T03:52:33.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.371+0000 7fcecaffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcecc0b16d0 0x7fcecc15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35224/0 (socket says 172.21.15.55:35224) 2023-10-06T03:52:33.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.371+0000 7fcecaffd700 1 -- 172.21.15.55:0/4176864298 learned_addr learned my addr 172.21.15.55:0/4176864298 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:33.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.371+0000 7fcecb7fe700 1 --2- 172.21.15.55:0/4176864298 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc0b4e70 0x7fcecc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:33.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.371+0000 7fceca7fc700 1 -- 172.21.15.55:0/4176864298 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611680297 0 0) 0x7fcecc0cfb00 con 0x7fcecc0b16d0 2023-10-06T03:52:33.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.371+0000 7fceca7fc700 1 -- 172.21.15.55:0/4176864298 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fceb4003570 con 0x7fcecc0b16d0 2023-10-06T03:52:33.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.371+0000 7fceca7fc700 1 -- 172.21.15.55:0/4176864298 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051556719 0 0) 0x7fcecc15e060 con 0x7fcecc0b1b40 2023-10-06T03:52:33.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.371+0000 7fceca7fc700 1 -- 172.21.15.55:0/4176864298 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcecc0cfb00 con 0x7fcecc0b1b40 2023-10-06T03:52:33.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.371+0000 7fceca7fc700 1 -- 172.21.15.55:0/4176864298 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3758301998 0 0) 0x7fceb4003570 con 0x7fcecc0b16d0 2023-10-06T03:52:33.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.371+0000 7fcecb7fe700 1 -- 172.21.15.55:0/4176864298 >> v1:172.21.15.163:6789/0 conn(0x7fcecc0b1b40 legacy=0x7fcecc15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:33.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.371+0000 7fcecb7fe700 1 -- 172.21.15.55:0/4176864298 >> v1:172.21.15.55:6789/0 conn(0x7fcecc0b16d0 legacy=0x7fcecc15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:33.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.371+0000 7fcecb7fe700 1 -- 172.21.15.55:0/4176864298 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc0b5f20 con 0x7fcecc0b4e70 2023-10-06T03:52:33.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.372+0000 7fcecb7fe700 1 --2- 172.21.15.55:0/4176864298 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc0b4e70 0x7fcecc0ccff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009910 tx=0x7fcebc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5ccba6150908c5b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:33.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.372+0000 7fceca7fc700 1 -- 172.21.15.55:0/4176864298 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec0001470 con 0x7fcecc0b4e70 2023-10-06T03:52:33.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.372+0000 7fceca7fc700 1 -- 172.21.15.55:0/4176864298 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc011420 con 0x7fcecc0b4e70 2023-10-06T03:52:33.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.372+0000 7fceca7fc700 1 -- 172.21.15.55:0/4176864298 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc0117c0 con 0x7fcecc0b4e70 2023-10-06T03:52:33.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.372+0000 7fced099d700 1 -- 172.21.15.55:0/4176864298 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc0b4e70 msgr2=0x7fcecc0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:33.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.372+0000 7fced099d700 1 --2- 172.21.15.55:0/4176864298 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc0b4e70 0x7fcecc0ccff0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009910 tx=0x7fcebc0091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:33.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.373+0000 7fced099d700 1 -- 172.21.15.55:0/4176864298 shutdown_connections 2023-10-06T03:52:33.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.373+0000 7fced099d700 1 --2- 172.21.15.55:0/4176864298 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc0b4e70 0x7fcecc0ccff0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:33.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.373+0000 7fced099d700 1 -- 172.21.15.55:0/4176864298 >> 172.21.15.55:0/4176864298 conn(0x7fcecc0d10c0 msgr2=0x7fcecc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:33.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.373+0000 7fced099d700 1 -- 172.21.15.55:0/4176864298 shutdown_connections 2023-10-06T03:52:33.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.373+0000 7fced099d700 1 -- 172.21.15.55:0/4176864298 wait complete. 2023-10-06T03:52:33.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.374+0000 7fced099d700 1 Processor -- start 2023-10-06T03:52:33.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.374+0000 7fced099d700 1 -- start start 2023-10-06T03:52:33.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.375+0000 7fced099d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcecc0b16d0 0x7fcecc157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:33.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.375+0000 7fced099d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcecc0b1b40 0x7fcecc16a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:33.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.375+0000 7fced099d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc0b4e70 0x7fcecc0cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:33.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.375+0000 7fced099d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc0ca4a0 con 0x7fcecc0b1b40 2023-10-06T03:52:33.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.375+0000 7fced099d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc0c9890 con 0x7fcecc0b16d0 2023-10-06T03:52:33.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.375+0000 7fced099d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc0c9b70 con 0x7fcecc0b4e70 2023-10-06T03:52:33.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.375+0000 7fcecaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcecc0b1b40 0x7fcecc16a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:33.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.375+0000 7fcecaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcecc0b1b40 0x7fcecc16a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49844/0 (socket says 172.21.15.55:49844) 2023-10-06T03:52:33.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.375+0000 7fcecaffd700 1 -- 172.21.15.55:0/3927407506 learned_addr learned my addr 172.21.15.55:0/3927407506 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:33.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.376+0000 7fcecb7fe700 1 --2- 172.21.15.55:0/3927407506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcecc0b16d0 0x7fcecc157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:33.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.376+0000 7fcecbfff700 1 --2- 172.21.15.55:0/3927407506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc0b4e70 0x7fcecc0cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:33.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.376+0000 7fcecaffd700 1 -- 172.21.15.55:0/3927407506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc0b4e70 msgr2=0x7fcecc0cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:33.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.376+0000 7fcecaffd700 1 --2- 172.21.15.55:0/3927407506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc0b4e70 0x7fcecc0cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:33.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.376+0000 7fcecaffd700 1 -- 172.21.15.55:0/3927407506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcecc0b16d0 msgr2=0x7fcecc157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:33.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.376+0000 7fcecaffd700 1 --2- 172.21.15.55:0/3927407506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcecc0b16d0 0x7fcecc157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:33.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.376+0000 7fcecaffd700 1 -- 172.21.15.55:0/3927407506 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc009420 con 0x7fcecc0b1b40 2023-10-06T03:52:33.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.376+0000 7fcecaffd700 1 --2- 172.21.15.55:0/3927407506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcecc0b1b40 0x7fcecc16a940 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fcec000cb70 tx=0x7fcec0001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:33.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.376+0000 7fceb37fe700 1 -- 172.21.15.55:0/3927407506 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec0007150 con 0x7fcecc0b1b40 2023-10-06T03:52:33.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.377+0000 7fceb37fe700 1 -- 172.21.15.55:0/3927407506 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec00072d0 con 0x7fcecc0b1b40 2023-10-06T03:52:33.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.377+0000 7fceb37fe700 1 -- 172.21.15.55:0/3927407506 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec0007690 con 0x7fcecc0b1b40 2023-10-06T03:52:33.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.377+0000 7fced099d700 1 -- 172.21.15.55:0/3927407506 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcecc0c9e50 con 0x7fcecc0b1b40 2023-10-06T03:52:33.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.377+0000 7fced099d700 1 -- 172.21.15.55:0/3927407506 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcecc16ae80 con 0x7fcecc0b1b40 2023-10-06T03:52:33.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.378+0000 7fced099d700 1 -- 172.21.15.55:0/3927407506 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcecc088990 con 0x7fcecc0b1b40 2023-10-06T03:52:33.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.379+0000 7fceb37fe700 1 -- 172.21.15.55:0/3927407506 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fcec0004a40 con 0x7fcecc0b1b40 2023-10-06T03:52:33.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.379+0000 7fceb37fe700 1 --2- 172.21.15.55:0/3927407506 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fceb8067050 0x7fceb8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:33.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.380+0000 7fceb37fe700 1 -- 172.21.15.55:0/3927407506 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fcec0084080 con 0x7fcecc0b1b40 2023-10-06T03:52:33.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.380+0000 7fcecb7fe700 1 --2- 172.21.15.55:0/3927407506 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fceb8067050 0x7fceb8069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:33.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.381+0000 7fcecb7fe700 1 --2- 172.21.15.55:0/3927407506 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fceb8067050 0x7fceb8069500 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0030c0 tx=0x7fcebc016000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:33.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.383+0000 7fceb37fe700 1 -- 172.21.15.55:0/3927407506 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcec0050f70 con 0x7fcecc0b1b40 2023-10-06T03:52:33.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:33.629+0000 7fced099d700 1 -- 172.21.15.55:0/3927407506 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcecc04bb80 con 0x7fcecc0b1b40 2023-10-06T03:52:34.083 INFO:tasks.ceph.mds.d.smithi163.stderr:2023-10-06T03:52:34.082+0000 7ffa9fbf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T03:52:34.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.169+0000 7fceb37fe700 1 -- 172.21.15.55:0/3927407506 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcec00180c0 con 0x7fcecc0b1b40 2023-10-06T03:52:34.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.171+0000 7fced099d700 1 -- 172.21.15.55:0/3927407506 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fceb8067050 msgr2=0x7fceb8069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:34.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.171+0000 7fced099d700 1 --2- 172.21.15.55:0/3927407506 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fceb8067050 0x7fceb8069500 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0030c0 tx=0x7fcebc016000 comp rx=0 tx=0).stop 2023-10-06T03:52:34.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.171+0000 7fced099d700 1 -- 172.21.15.55:0/3927407506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcecc0b1b40 msgr2=0x7fcecc16a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:34.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.171+0000 7fced099d700 1 --2- 172.21.15.55:0/3927407506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcecc0b1b40 0x7fcecc16a940 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fcec000cb70 tx=0x7fcec0001ab0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.172+0000 7fced099d700 1 -- 172.21.15.55:0/3927407506 shutdown_connections 2023-10-06T03:52:34.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.172+0000 7fced099d700 1 --2- 172.21.15.55:0/3927407506 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fceb8067050 0x7fceb8069500 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.172+0000 7fced099d700 1 --2- 172.21.15.55:0/3927407506 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcecc0b4e70 0x7fcecc0cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.172+0000 7fced099d700 1 --2- 172.21.15.55:0/3927407506 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcecc0b16d0 0x7fcecc157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.172+0000 7fced099d700 1 --2- 172.21.15.55:0/3927407506 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcecc0b1b40 0x7fcecc16a940 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.172+0000 7fced099d700 1 -- 172.21.15.55:0/3927407506 >> 172.21.15.55:0/3927407506 conn(0x7fcecc0d10c0 msgr2=0x7fcecc159c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:34.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.172+0000 7fced099d700 1 -- 172.21.15.55:0/3927407506 shutdown_connections 2023-10-06T03:52:34.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.172+0000 7fced099d700 1 -- 172.21.15.55:0/3927407506 wait complete. 2023-10-06T03:52:34.192 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:52:34.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.389+0000 7fbb3959d700 1 Processor -- start 2023-10-06T03:52:34.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.389+0000 7fbb3959d700 1 -- start start 2023-10-06T03:52:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.391+0000 7fbb3959d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbb341595a0 0x7fbb340b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.391+0000 7fbb3959d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb340c8830 0x7fbb34159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.391+0000 7fbb3959d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb340b1410 con 0x7fbb340c6c60 2023-10-06T03:52:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.391+0000 7fbb3959d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb340b2b90 con 0x7fbb340c8830 2023-10-06T03:52:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.391+0000 7fbb3959d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb34063230 con 0x7fbb341595a0 2023-10-06T03:52:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.391+0000 7fbb38d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb340c8830 0x7fbb34159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.391+0000 7fbb38d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb340c8830 0x7fbb34159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49856/0 (socket says 172.21.15.55:49856) 2023-10-06T03:52:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.391+0000 7fbb38d9c700 1 -- 172.21.15.55:0/361670265 learned_addr learned my addr 172.21.15.55:0/361670265 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.391+0000 7fbb38d9c700 1 -- 172.21.15.55:0/361670265 >> v2:172.21.15.163:3300/0 conn(0x7fbb341595a0 msgr2=0x7fbb340b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:34.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.391+0000 7fbb33fff700 1 --2- 172.21.15.55:0/361670265 >> v2:172.21.15.163:3300/0 conn(0x7fbb341595a0 0x7fbb340b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.391+0000 7fbb38d9c700 1 --2- 172.21.15.55:0/361670265 >> v2:172.21.15.163:3300/0 conn(0x7fbb341595a0 0x7fbb340b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.391+0000 7fbb38d9c700 1 -- 172.21.15.55:0/361670265 >> v1:172.21.15.163:6789/0 conn(0x7fbb340c6c60 legacy=0x7fbb340c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.391+0000 7fbb38d9c700 1 -- 172.21.15.55:0/361670265 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb340634f0 con 0x7fbb340c8830 2023-10-06T03:52:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.391+0000 7fbb33fff700 1 --2- 172.21.15.55:0/361670265 >> v2:172.21.15.163:3300/0 conn(0x7fbb341595a0 0x7fbb340b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.392+0000 7fbb38d9c700 1 --2- 172.21.15.55:0/361670265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb340c8830 0x7fbb34159970 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fbb340c8d10 tx=0x7fbb2c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcde9941c0cf310 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.392+0000 7fbb32ffd700 1 -- 172.21.15.55:0/361670265 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00e070 con 0x7fbb340c8830 2023-10-06T03:52:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.392+0000 7fbb32ffd700 1 -- 172.21.15.55:0/361670265 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c00ba10 con 0x7fbb340c8830 2023-10-06T03:52:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.392+0000 7fbb32ffd700 1 -- 172.21.15.55:0/361670265 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0092e0 con 0x7fbb340c8830 2023-10-06T03:52:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.392+0000 7fbb3959d700 1 -- 172.21.15.55:0/361670265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb340c8830 msgr2=0x7fbb34159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:34.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.392+0000 7fbb3959d700 1 --2- 172.21.15.55:0/361670265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb340c8830 0x7fbb34159970 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fbb340c8d10 tx=0x7fbb2c009840 comp rx=0 tx=0).stop 2023-10-06T03:52:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.393+0000 7fbb3959d700 1 -- 172.21.15.55:0/361670265 shutdown_connections 2023-10-06T03:52:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.393+0000 7fbb3959d700 1 --2- 172.21.15.55:0/361670265 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb340c8830 0x7fbb34159970 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.393+0000 7fbb3959d700 1 --2- 172.21.15.55:0/361670265 >> v2:172.21.15.163:3300/0 conn(0x7fbb341595a0 0x7fbb340b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.393+0000 7fbb3959d700 1 -- 172.21.15.55:0/361670265 >> 172.21.15.55:0/361670265 conn(0x7fbb3406ddb0 msgr2=0x7fbb3406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.393+0000 7fbb3959d700 1 -- 172.21.15.55:0/361670265 shutdown_connections 2023-10-06T03:52:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.393+0000 7fbb3959d700 1 -- 172.21.15.55:0/361670265 wait complete. 2023-10-06T03:52:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.394+0000 7fbb3959d700 1 Processor -- start 2023-10-06T03:52:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.394+0000 7fbb3959d700 1 -- start start 2023-10-06T03:52:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.395+0000 7fbb3959d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb340c6c60 0x7fbb340c4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.395+0000 7fbb3959d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb340c8830 0x7fbb340c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.395+0000 7fbb3959d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb341595a0 0x7fbb3415af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:34.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.395+0000 7fbb3959d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340c2320 con 0x7fbb340c8830 2023-10-06T03:52:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.395+0000 7fbb3959d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3415c3d0 con 0x7fbb341595a0 2023-10-06T03:52:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.395+0000 7fbb3959d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb340c19f0 con 0x7fbb340c6c60 2023-10-06T03:52:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.396+0000 7fbb337fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb340c8830 0x7fbb340c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.396+0000 7fbb337fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb340c8830 0x7fbb340c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49862/0 (socket says 172.21.15.55:49862) 2023-10-06T03:52:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.396+0000 7fbb337fe700 1 -- 172.21.15.55:0/2883362588 learned_addr learned my addr 172.21.15.55:0/2883362588 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.396+0000 7fbb33fff700 1 --2- 172.21.15.55:0/2883362588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb340c6c60 0x7fbb340c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.396+0000 7fbb38d9c700 1 --2- 172.21.15.55:0/2883362588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb341595a0 0x7fbb3415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.396+0000 7fbb337fe700 1 -- 172.21.15.55:0/2883362588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb340c6c60 msgr2=0x7fbb340c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.396+0000 7fbb337fe700 1 --2- 172.21.15.55:0/2883362588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb340c6c60 0x7fbb340c4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.396+0000 7fbb337fe700 1 -- 172.21.15.55:0/2883362588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb341595a0 msgr2=0x7fbb3415af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:34.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.396+0000 7fbb337fe700 1 --2- 172.21.15.55:0/2883362588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb341595a0 0x7fbb3415af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.396+0000 7fbb337fe700 1 -- 172.21.15.55:0/2883362588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c009ae0 con 0x7fbb340c8830 2023-10-06T03:52:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.396+0000 7fbb337fe700 1 --2- 172.21.15.55:0/2883362588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb340c8830 0x7fbb340c4690 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800c590 tx=0x7fbb28007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.396+0000 7fbb1bfff700 1 -- 172.21.15.55:0/2883362588 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb28011590 con 0x7fbb340c8830 2023-10-06T03:52:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.396+0000 7fbb1bfff700 1 -- 172.21.15.55:0/2883362588 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb280116f0 con 0x7fbb340c8830 2023-10-06T03:52:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.396+0000 7fbb1bfff700 1 -- 172.21.15.55:0/2883362588 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb28011ab0 con 0x7fbb340c8830 2023-10-06T03:52:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.397+0000 7fbb3959d700 1 -- 172.21.15.55:0/2883362588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb340c1cd0 con 0x7fbb340c8830 2023-10-06T03:52:34.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.397+0000 7fbb3959d700 1 -- 172.21.15.55:0/2883362588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3415b450 con 0x7fbb340c8830 2023-10-06T03:52:34.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.398+0000 7fbb3959d700 1 -- 172.21.15.55:0/2883362588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb341568f0 con 0x7fbb340c8830 2023-10-06T03:52:34.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.399+0000 7fbb1bfff700 1 -- 172.21.15.55:0/2883362588 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fbb2801c460 con 0x7fbb340c8830 2023-10-06T03:52:34.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.400+0000 7fbb1bfff700 1 --2- 172.21.15.55:0/2883362588 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbb20067050 0x7fbb20069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:34.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.400+0000 7fbb1bfff700 1 -- 172.21.15.55:0/2883362588 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fbb28084680 con 0x7fbb340c8830 2023-10-06T03:52:34.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.400+0000 7fbb33fff700 1 --2- 172.21.15.55:0/2883362588 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbb20067050 0x7fbb20069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:34.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.401+0000 7fbb33fff700 1 --2- 172.21.15.55:0/2883362588 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbb20067050 0x7fbb20069500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbb24006fd0 tx=0x7fbb24008040 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:34.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.404+0000 7fbb1bfff700 1 -- 172.21.15.55:0/2883362588 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb28051450 con 0x7fbb340c8830 2023-10-06T03:52:34.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.653+0000 7fbb3959d700 1 -- 172.21.15.55:0/2883362588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb3404c3c0 con 0x7fbb340c8830 2023-10-06T03:52:34.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.654+0000 7fbb1bfff700 1 -- 172.21.15.55:0/2883362588 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fbb28054a70 con 0x7fbb340c8830 2023-10-06T03:52:34.656 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:34.656 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2624033145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2624033145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2624033145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3851023466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3851023466},{"type":"v1","addr":"172.21.15.55:6834","nonce":3851023466}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/2633241022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2633241022},{"type":"v1","addr":"172.21.15.163:6833","nonce":2633241022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:52:30.648704+0000","modified":"2023-10-06T03:52:34.152554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7879":{"gid":7879,"name":"d","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6836/246684871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":246684871},{"type":"v1","addr":"172.21.15.163:6836","nonce":246684871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T03:52:34.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.656+0000 7fbb3959d700 1 -- 172.21.15.55:0/2883362588 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbb20067050 msgr2=0x7fbb20069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:34.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.656+0000 7fbb3959d700 1 --2- 172.21.15.55:0/2883362588 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbb20067050 0x7fbb20069500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbb24006fd0 tx=0x7fbb24008040 comp rx=0 tx=0).stop 2023-10-06T03:52:34.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.656+0000 7fbb3959d700 1 -- 172.21.15.55:0/2883362588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb340c8830 msgr2=0x7fbb340c4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:34.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.656+0000 7fbb3959d700 1 --2- 172.21.15.55:0/2883362588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb340c8830 0x7fbb340c4690 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800c590 tx=0x7fbb28007f30 comp rx=0 tx=0).stop 2023-10-06T03:52:34.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.656+0000 7fbb3959d700 1 -- 172.21.15.55:0/2883362588 shutdown_connections 2023-10-06T03:52:34.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.656+0000 7fbb3959d700 1 --2- 172.21.15.55:0/2883362588 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbb20067050 0x7fbb20069500 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.656+0000 7fbb3959d700 1 --2- 172.21.15.55:0/2883362588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb341595a0 0x7fbb3415af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.656+0000 7fbb3959d700 1 --2- 172.21.15.55:0/2883362588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb340c6c60 0x7fbb340c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.656+0000 7fbb3959d700 1 --2- 172.21.15.55:0/2883362588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb340c8830 0x7fbb340c4690 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.656+0000 7fbb3959d700 1 -- 172.21.15.55:0/2883362588 >> 172.21.15.55:0/2883362588 conn(0x7fbb3406ddb0 msgr2=0x7fbb341583f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:34.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.656+0000 7fbb3959d700 1 -- 172.21.15.55:0/2883362588 shutdown_connections 2023-10-06T03:52:34.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.656+0000 7fbb3959d700 1 -- 172.21.15.55:0/2883362588 wait complete. 2023-10-06T03:52:34.659 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 70 2023-10-06T03:52:34.675 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T03:52:34.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.873+0000 7fbd68b9d700 1 Processor -- start 2023-10-06T03:52:34.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.873+0000 7fbd68b9d700 1 -- start start 2023-10-06T03:52:34.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.874+0000 7fbd68b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd640ad870 0x7fbd6406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:34.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.874+0000 7fbd68b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd6406bf20 0x7fbd6406a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:34.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.874+0000 7fbd68b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd640b40f0 con 0x7fbd64142ea0 2023-10-06T03:52:34.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.874+0000 7fbd68b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6406c3a0 con 0x7fbd640ad870 2023-10-06T03:52:34.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.874+0000 7fbd68b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd64069640 con 0x7fbd6406bf20 2023-10-06T03:52:34.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.875+0000 7fbd63fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd6406bf20 0x7fbd6406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:34.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.875+0000 7fbd62ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd640ad870 0x7fbd6406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:34.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.875+0000 7fbd63fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd6406bf20 0x7fbd6406a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:39554/0 (socket says 172.21.15.55:39554) 2023-10-06T03:52:34.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.875+0000 7fbd62ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbd640ad870 0x7fbd6406b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:55306/0 (socket says 172.21.15.55:55306) 2023-10-06T03:52:34.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.875+0000 7fbd63fff700 1 -- 172.21.15.55:0/2899309794 learned_addr learned my addr 172.21.15.55:0/2899309794 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:34.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.875+0000 7fbd62ffd700 1 -- 172.21.15.55:0/2899309794 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd6406bf20 msgr2=0x7fbd6406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:34.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.875+0000 7fbd62ffd700 1 --2- 172.21.15.55:0/2899309794 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd6406bf20 0x7fbd6406a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.875+0000 7fbd62ffd700 1 -- 172.21.15.55:0/2899309794 >> v1:172.21.15.163:6789/0 conn(0x7fbd64142ea0 legacy=0x7fbd640b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:34.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.875+0000 7fbd62ffd700 1 -- 172.21.15.55:0/2899309794 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd64069920 con 0x7fbd640ad870 2023-10-06T03:52:34.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.875+0000 7fbd63fff700 1 --2- 172.21.15.55:0/2899309794 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd6406bf20 0x7fbd6406a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:52:34.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.876+0000 7fbd62ffd700 1 --2- 172.21.15.55:0/2899309794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd640ad870 0x7fbd6406b9e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbd580098e0 tx=0x7fbd580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caf62ff625b81fc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:34.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.876+0000 7fbd627fc700 1 -- 172.21.15.55:0/2899309794 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5800b640 con 0x7fbd640ad870 2023-10-06T03:52:34.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.876+0000 7fbd627fc700 1 -- 172.21.15.55:0/2899309794 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5800b7a0 con 0x7fbd640ad870 2023-10-06T03:52:34.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.876+0000 7fbd627fc700 1 -- 172.21.15.55:0/2899309794 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5800bb40 con 0x7fbd640ad870 2023-10-06T03:52:34.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.876+0000 7fbd68b9d700 1 -- 172.21.15.55:0/2899309794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd640ad870 msgr2=0x7fbd6406b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:34.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.876+0000 7fbd68b9d700 1 --2- 172.21.15.55:0/2899309794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd640ad870 0x7fbd6406b9e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbd580098e0 tx=0x7fbd580091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.876+0000 7fbd68b9d700 1 -- 172.21.15.55:0/2899309794 shutdown_connections 2023-10-06T03:52:34.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.876+0000 7fbd68b9d700 1 --2- 172.21.15.55:0/2899309794 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd6406bf20 0x7fbd6406a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.876+0000 7fbd68b9d700 1 --2- 172.21.15.55:0/2899309794 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd640ad870 0x7fbd6406b9e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.876+0000 7fbd68b9d700 1 -- 172.21.15.55:0/2899309794 >> 172.21.15.55:0/2899309794 conn(0x7fbd640ce430 msgr2=0x7fbd640ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:34.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.876+0000 7fbd68b9d700 1 -- 172.21.15.55:0/2899309794 shutdown_connections 2023-10-06T03:52:34.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.877+0000 7fbd68b9d700 1 -- 172.21.15.55:0/2899309794 wait complete. 2023-10-06T03:52:34.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.878+0000 7fbd68b9d700 1 Processor -- start 2023-10-06T03:52:34.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.878+0000 7fbd68b9d700 1 -- start start 2023-10-06T03:52:34.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.879+0000 7fbd68b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd6406bf20 0x7fbd6416a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:34.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.879+0000 7fbd68b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd64142ea0 0x7fbd640c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:34.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.879+0000 7fbd68b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6415da90 0x7fbd640cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:34.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.879+0000 7fbd68b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640c98e0 con 0x7fbd64142ea0 2023-10-06T03:52:34.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.879+0000 7fbd68b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6416ddf0 con 0x7fbd6415da90 2023-10-06T03:52:34.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.879+0000 7fbd68b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640c9ad0 con 0x7fbd6406bf20 2023-10-06T03:52:34.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.879+0000 7fbd62ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd64142ea0 0x7fbd640c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:34.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.879+0000 7fbd62ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd64142ea0 0x7fbd640c93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49866/0 (socket says 172.21.15.55:49866) 2023-10-06T03:52:34.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.879+0000 7fbd62ffd700 1 -- 172.21.15.55:0/3660397872 learned_addr learned my addr 172.21.15.55:0/3660397872 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:34.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.879+0000 7fbd63fff700 1 --2- 172.21.15.55:0/3660397872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6415da90 0x7fbd640cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:34.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.879+0000 7fbd637fe700 1 --2- 172.21.15.55:0/3660397872 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd6406bf20 0x7fbd6416a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:34.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.880+0000 7fbd62ffd700 1 -- 172.21.15.55:0/3660397872 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd6406bf20 msgr2=0x7fbd6416a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:34.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.880+0000 7fbd62ffd700 1 --2- 172.21.15.55:0/3660397872 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd6406bf20 0x7fbd6416a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.880+0000 7fbd62ffd700 1 -- 172.21.15.55:0/3660397872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6415da90 msgr2=0x7fbd640cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:34.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.880+0000 7fbd62ffd700 1 --2- 172.21.15.55:0/3660397872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6415da90 0x7fbd640cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:34.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.880+0000 7fbd62ffd700 1 -- 172.21.15.55:0/3660397872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c009ae0 con 0x7fbd64142ea0 2023-10-06T03:52:34.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.880+0000 7fbd62ffd700 1 --2- 172.21.15.55:0/3660397872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd64142ea0 0x7fbd640c93a0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fbd58013000 tx=0x7fbd58017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:34.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.880+0000 7fbd4b7fe700 1 -- 172.21.15.55:0/3660397872 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5801bc40 con 0x7fbd64142ea0 2023-10-06T03:52:34.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.880+0000 7fbd4b7fe700 1 -- 172.21.15.55:0/3660397872 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5801bde0 con 0x7fbd64142ea0 2023-10-06T03:52:34.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.880+0000 7fbd68b9d700 1 -- 172.21.15.55:0/3660397872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd58009420 con 0x7fbd64142ea0 2023-10-06T03:52:34.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.880+0000 7fbd4b7fe700 1 -- 172.21.15.55:0/3660397872 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd58011670 con 0x7fbd64142ea0 2023-10-06T03:52:34.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.881+0000 7fbd68b9d700 1 -- 172.21.15.55:0/3660397872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6416d7a0 con 0x7fbd64142ea0 2023-10-06T03:52:34.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.881+0000 7fbd68b9d700 1 -- 172.21.15.55:0/3660397872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6416df60 con 0x7fbd64142ea0 2023-10-06T03:52:34.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.882+0000 7fbd4b7fe700 1 -- 172.21.15.55:0/3660397872 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fbd580119e0 con 0x7fbd64142ea0 2023-10-06T03:52:34.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.883+0000 7fbd4b7fe700 1 --2- 172.21.15.55:0/3660397872 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd50067050 0x7fbd50069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:34.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.883+0000 7fbd4b7fe700 1 -- 172.21.15.55:0/3660397872 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fbd58085fe0 con 0x7fbd64142ea0 2023-10-06T03:52:34.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.884+0000 7fbd637fe700 1 --2- 172.21.15.55:0/3660397872 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd50067050 0x7fbd50069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:34.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.884+0000 7fbd637fe700 1 --2- 172.21.15.55:0/3660397872 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd50067050 0x7fbd50069500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbd6415bed0 tx=0x7fbd54009040 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:34.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:34.888+0000 7fbd4b7fe700 1 -- 172.21.15.55:0/3660397872 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd58052df0 con 0x7fbd64142ea0 2023-10-06T03:52:35.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.092+0000 7fbd68b9d700 1 -- 172.21.15.55:0/3660397872 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd6404c3c0 con 0x7fbd64142ea0 2023-10-06T03:52:35.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.093+0000 7fbd4b7fe700 1 -- 172.21.15.55:0/3660397872 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+17523 (secure 0 0 0) 0x7fbd58056410 con 0x7fbd64142ea0 2023-10-06T03:52:35.094 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:35.095 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":137,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T03:52:31.768186+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:52:28.177340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T03:52:29.819781+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T03:52:35.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.095+0000 7fbd68b9d700 1 -- 172.21.15.55:0/3660397872 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd50067050 msgr2=0x7fbd50069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.095+0000 7fbd68b9d700 1 --2- 172.21.15.55:0/3660397872 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd50067050 0x7fbd50069500 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbd6415bed0 tx=0x7fbd54009040 comp rx=0 tx=0).stop 2023-10-06T03:52:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.095+0000 7fbd68b9d700 1 -- 172.21.15.55:0/3660397872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd64142ea0 msgr2=0x7fbd640c93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.095+0000 7fbd68b9d700 1 --2- 172.21.15.55:0/3660397872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd64142ea0 0x7fbd640c93a0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fbd58013000 tx=0x7fbd58017000 comp rx=0 tx=0).stop 2023-10-06T03:52:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.095+0000 7fbd68b9d700 1 -- 172.21.15.55:0/3660397872 shutdown_connections 2023-10-06T03:52:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.095+0000 7fbd68b9d700 1 --2- 172.21.15.55:0/3660397872 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbd50067050 0x7fbd50069500 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.095+0000 7fbd68b9d700 1 --2- 172.21.15.55:0/3660397872 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbd6415da90 0x7fbd640cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.095+0000 7fbd68b9d700 1 --2- 172.21.15.55:0/3660397872 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbd6406bf20 0x7fbd6416a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.095+0000 7fbd68b9d700 1 --2- 172.21.15.55:0/3660397872 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbd64142ea0 0x7fbd640c93a0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.095+0000 7fbd68b9d700 1 -- 172.21.15.55:0/3660397872 >> 172.21.15.55:0/3660397872 conn(0x7fbd640ce430 msgr2=0x7fbd640b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.095+0000 7fbd68b9d700 1 -- 172.21.15.55:0/3660397872 shutdown_connections 2023-10-06T03:52:35.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.095+0000 7fbd68b9d700 1 -- 172.21.15.55:0/3660397872 wait complete. 2023-10-06T03:52:35.114 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T03:52:35.114 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:52:35.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.300+0000 7f484679d700 1 Processor -- start 2023-10-06T03:52:35.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.300+0000 7f484679d700 1 -- start start 2023-10-06T03:52:35.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f484679d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4840156310 0x7f48400c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:35.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f484679d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f48401566e0 0x7f48400ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:35.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f484679d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48400ca300 0x7f48400ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:35.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f484679d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48400ca860 con 0x7f4840156310 2023-10-06T03:52:35.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f484679d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48400cab40 con 0x7f48401566e0 2023-10-06T03:52:35.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f484679d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48400cae20 con 0x7f48400ca300 2023-10-06T03:52:35.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f484579b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4840156310 0x7f48400c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:35.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f484579b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4840156310 0x7f48400c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49882/0 (socket says 172.21.15.55:49882) 2023-10-06T03:52:35.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f484579b700 1 -- 172.21.15.55:0/3233367559 learned_addr learned my addr 172.21.15.55:0/3233367559 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:35.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f4844f9a700 1 --2- 172.21.15.55:0/3233367559 >> v2:172.21.15.163:3300/0 conn(0x7f48401566e0 0x7f48400ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:35.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f4845f9c700 1 --2- 172.21.15.55:0/3233367559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48400ca300 0x7f48400ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:35.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f484579b700 1 -- 172.21.15.55:0/3233367559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48400ca300 msgr2=0x7f48400ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:35.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f484579b700 1 --2- 172.21.15.55:0/3233367559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48400ca300 0x7f48400ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f484579b700 1 -- 172.21.15.55:0/3233367559 >> v2:172.21.15.163:3300/0 conn(0x7f48401566e0 msgr2=0x7f48400ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:35.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f484579b700 1 --2- 172.21.15.55:0/3233367559 >> v2:172.21.15.163:3300/0 conn(0x7f48401566e0 0x7f48400ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.302+0000 7f484579b700 1 -- 172.21.15.55:0/3233367559 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48400cb100 con 0x7f4840156310 2023-10-06T03:52:35.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.303+0000 7f484579b700 1 --2- 172.21.15.55:0/3233367559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4840156310 0x7f48400c9dc0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f484006c000 tx=0x7f483c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d89189a316da16 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:35.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.303+0000 7f4837fff700 1 -- 172.21.15.55:0/3233367559 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c00b860 con 0x7f4840156310 2023-10-06T03:52:35.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.303+0000 7f4837fff700 1 -- 172.21.15.55:0/3233367559 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c00b9c0 con 0x7f4840156310 2023-10-06T03:52:35.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.303+0000 7f4837fff700 1 -- 172.21.15.55:0/3233367559 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c00bd60 con 0x7f4840156310 2023-10-06T03:52:35.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.303+0000 7f484679d700 1 -- 172.21.15.55:0/3233367559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4840156310 msgr2=0x7f48400c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:35.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.303+0000 7f484679d700 1 --2- 172.21.15.55:0/3233367559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4840156310 0x7f48400c9dc0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f484006c000 tx=0x7f483c009250 comp rx=0 tx=0).stop 2023-10-06T03:52:35.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.304+0000 7f484679d700 1 -- 172.21.15.55:0/3233367559 shutdown_connections 2023-10-06T03:52:35.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.304+0000 7f484679d700 1 --2- 172.21.15.55:0/3233367559 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48400ca300 0x7f48400ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.304+0000 7f484679d700 1 --2- 172.21.15.55:0/3233367559 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4840156310 0x7f48400c9dc0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.304+0000 7f484679d700 1 --2- 172.21.15.55:0/3233367559 >> v2:172.21.15.163:3300/0 conn(0x7f48401566e0 0x7f48400ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.304+0000 7f484679d700 1 -- 172.21.15.55:0/3233367559 >> 172.21.15.55:0/3233367559 conn(0x7f48400cc1f0 msgr2=0x7f48400cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:35.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.304+0000 7f484679d700 1 -- 172.21.15.55:0/3233367559 shutdown_connections 2023-10-06T03:52:35.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.304+0000 7f484679d700 1 -- 172.21.15.55:0/3233367559 wait complete. 2023-10-06T03:52:35.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.305+0000 7f484679d700 1 Processor -- start 2023-10-06T03:52:35.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.305+0000 7f484679d700 1 -- start start 2023-10-06T03:52:35.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.306+0000 7f484679d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48400ca300 0x7f4840144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:35.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.306+0000 7f484679d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4840156310 0x7f484015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:35.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.306+0000 7f484679d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48401566e0 0x7f48400aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:35.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.306+0000 7f484679d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48400c4ad0 con 0x7f4840156310 2023-10-06T03:52:35.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.306+0000 7f484679d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4840165850 con 0x7f48400ca300 2023-10-06T03:52:35.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.306+0000 7f484679d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48401451e0 con 0x7f48401566e0 2023-10-06T03:52:35.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.306+0000 7f4844f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4840156310 0x7f484015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:35.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.307+0000 7f4844f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4840156310 0x7f484015a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49898/0 (socket says 172.21.15.55:49898) 2023-10-06T03:52:35.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.307+0000 7f4844f9a700 1 -- 172.21.15.55:0/3563816062 learned_addr learned my addr 172.21.15.55:0/3563816062 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:35.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.307+0000 7f484579b700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48400ca300 0x7f4840144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:35.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.307+0000 7f4845f9c700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48401566e0 0x7f48400aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:35.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.307+0000 7f4844f9a700 1 -- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48401566e0 msgr2=0x7f48400aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:35.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.307+0000 7f4844f9a700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48401566e0 0x7f48400aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.307+0000 7f4844f9a700 1 -- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48400ca300 msgr2=0x7f4840144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:35.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.307+0000 7f4844f9a700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48400ca300 0x7f4840144ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.307+0000 7f4844f9a700 1 -- 172.21.15.55:0/3563816062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c0094f0 con 0x7f4840156310 2023-10-06T03:52:35.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.307+0000 7f4845f9c700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48401566e0 0x7f48400aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:35.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.307+0000 7f4844f9a700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4840156310 0x7f484015a8f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f483000fdf0 tx=0x7f4830009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:35.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.307+0000 7f48357fa700 1 -- 172.21.15.55:0/3563816062 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830009e90 con 0x7f4840156310 2023-10-06T03:52:35.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.307+0000 7f48357fa700 1 -- 172.21.15.55:0/3563816062 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483000dd80 con 0x7f4840156310 2023-10-06T03:52:35.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.308+0000 7f48357fa700 1 -- 172.21.15.55:0/3563816062 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830011660 con 0x7f4840156310 2023-10-06T03:52:35.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.308+0000 7f484679d700 1 -- 172.21.15.55:0/3563816062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48400c4230 con 0x7f4840156310 2023-10-06T03:52:35.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.308+0000 7f484679d700 1 -- 172.21.15.55:0/3563816062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48400c47f0 con 0x7f4840156310 2023-10-06T03:52:35.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.309+0000 7f484679d700 1 -- 172.21.15.55:0/3563816062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484004c3c0 con 0x7f4840156310 2023-10-06T03:52:35.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.310+0000 7f48357fa700 1 -- 172.21.15.55:0/3563816062 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f4830011a10 con 0x7f4840156310 2023-10-06T03:52:35.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.310+0000 7f48357fa700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4824067050 0x7f4824069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:35.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.310+0000 7f48357fa700 1 -- 172.21.15.55:0/3563816062 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f48300876a0 con 0x7f4840156310 2023-10-06T03:52:35.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.311+0000 7f484579b700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4824067050 0x7f4824069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:35.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.311+0000 7f484579b700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4824067050 0x7f4824069500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f484006c000 tx=0x7f483c017000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:35.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.315+0000 7f48357fa700 1 -- 172.21.15.55:0/3563816062 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4830054360 con 0x7f4840156310 2023-10-06T03:52:35.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.513+0000 7f484679d700 1 -- 172.21.15.55:0/3563816062 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4840165680 con 0x7f4824067050 2023-10-06T03:52:35.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.525+0000 7f48357fa700 1 -- 172.21.15.55:0/3563816062 <== mgr.7690 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311999 (secure 0 0 0) 0x7f483c00bb30 con 0x7f4824067050 2023-10-06T03:52:35.556 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:35.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.529+0000 7f484679d700 1 -- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4824067050 msgr2=0x7f4824069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:35.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.529+0000 7f484679d700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4824067050 0x7f4824069500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f484006c000 tx=0x7f483c017000 comp rx=0 tx=0).stop 2023-10-06T03:52:35.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.529+0000 7f484679d700 1 -- 172.21.15.55:0/3563816062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4840156310 msgr2=0x7f484015a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:35.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.529+0000 7f484679d700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4840156310 0x7f484015a8f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f483000fdf0 tx=0x7f4830009580 comp rx=0 tx=0).stop 2023-10-06T03:52:35.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.529+0000 7f484679d700 1 -- 172.21.15.55:0/3563816062 shutdown_connections 2023-10-06T03:52:35.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.529+0000 7f484679d700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4824067050 0x7f4824069500 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.529+0000 7f484679d700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48401566e0 0x7f48400aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.529+0000 7f484679d700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48400ca300 0x7f4840144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.529+0000 7f484679d700 1 --2- 172.21.15.55:0/3563816062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4840156310 0x7f484015a8f0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.529+0000 7f484679d700 1 -- 172.21.15.55:0/3563816062 >> 172.21.15.55:0/3563816062 conn(0x7f48400cc1f0 msgr2=0x7f48400c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:35.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.529+0000 7f484679d700 1 -- 172.21.15.55:0/3563816062 shutdown_connections 2023-10-06T03:52:35.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.529+0000 7f484679d700 1 -- 172.21.15.55:0/3563816062 wait complete. 2023-10-06T03:52:35.567 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:52:35.567 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-06T03:52:34.933978+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":443,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2462336,"kb_used_data":14400,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752512384,"statfs":{"total":773094113280,"available":770572681216,"internally_reserved":0,"allocated":14745600,"data_stored":8166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.833090"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.785996+0000","last_change":"2023-10-06T03:52:30.780956+0000","last_active":"2023-10-06T03:52:30.785996+0000","last_peered":"2023-10-06T03:52:30.785996+0000","last_clean":"2023-10-06T03:52:30.785996+0000","last_became_active":"2023-10-06T03:52:30.780670+0000","last_became_peered":"2023-10-06T03:52:30.780670+0000","last_unstale":"2023-10-06T03:52:30.785996+0000","last_undegraded":"2023-10-06T03:52:30.785996+0000","last_fullsized":"2023-10-06T03:52:30.785996+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:07.137633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792648+0000","last_change":"2023-10-06T03:52:30.789178+0000","last_active":"2023-10-06T03:52:30.792648+0000","last_peered":"2023-10-06T03:52:30.792648+0000","last_clean":"2023-10-06T03:52:30.792648+0000","last_became_active":"2023-10-06T03:52:30.788531+0000","last_became_peered":"2023-10-06T03:52:30.788531+0000","last_unstale":"2023-10-06T03:52:30.792648+0000","last_undegraded":"2023-10-06T03:52:30.792648+0000","last_fullsized":"2023-10-06T03:52:30.792648+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:20.203153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792700+0000","last_change":"2023-10-06T03:52:30.789224+0000","last_active":"2023-10-06T03:52:30.792700+0000","last_peered":"2023-10-06T03:52:30.792700+0000","last_clean":"2023-10-06T03:52:30.792700+0000","last_became_active":"2023-10-06T03:52:30.788944+0000","last_became_peered":"2023-10-06T03:52:30.788944+0000","last_unstale":"2023-10-06T03:52:30.792700+0000","last_undegraded":"2023-10-06T03:52:30.792700+0000","last_fullsized":"2023-10-06T03:52:30.792700+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:53.099855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.793655+0000","last_change":"2023-10-06T03:52:30.793655+0000","last_active":"2023-10-06T03:52:30.793655+0000","last_peered":"2023-10-06T03:52:30.793655+0000","last_clean":"2023-10-06T03:52:30.793655+0000","last_became_active":"2023-10-06T03:52:30.793255+0000","last_became_peered":"2023-10-06T03:52:30.793255+0000","last_unstale":"2023-10-06T03:52:30.793655+0000","last_undegraded":"2023-10-06T03:52:30.793655+0000","last_fullsized":"2023-10-06T03:52:30.793655+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:27.485634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.784766+0000","last_change":"2023-10-06T03:52:30.779116+0000","last_active":"2023-10-06T03:52:30.784766+0000","last_peered":"2023-10-06T03:52:30.784766+0000","last_clean":"2023-10-06T03:52:30.784766+0000","last_became_active":"2023-10-06T03:52:30.777363+0000","last_became_peered":"2023-10-06T03:52:30.777363+0000","last_unstale":"2023-10-06T03:52:30.784766+0000","last_undegraded":"2023-10-06T03:52:30.784766+0000","last_fullsized":"2023-10-06T03:52:30.784766+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:28.909246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792579+0000","last_change":"2023-10-06T03:52:30.792579+0000","last_active":"2023-10-06T03:52:30.792579+0000","last_peered":"2023-10-06T03:52:30.792579+0000","last_clean":"2023-10-06T03:52:30.792579+0000","last_became_active":"2023-10-06T03:52:30.791930+0000","last_became_peered":"2023-10-06T03:52:30.791930+0000","last_unstale":"2023-10-06T03:52:30.792579+0000","last_undegraded":"2023-10-06T03:52:30.792579+0000","last_fullsized":"2023-10-06T03:52:30.792579+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:08:20.731175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.788109+0000","last_change":"2023-10-06T03:52:30.788109+0000","last_active":"2023-10-06T03:52:30.788109+0000","last_peered":"2023-10-06T03:52:30.788109+0000","last_clean":"2023-10-06T03:52:30.788109+0000","last_became_active":"2023-10-06T03:52:30.787543+0000","last_became_peered":"2023-10-06T03:52:30.787543+0000","last_unstale":"2023-10-06T03:52:30.788109+0000","last_undegraded":"2023-10-06T03:52:30.788109+0000","last_fullsized":"2023-10-06T03:52:30.788109+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:30.779261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791534+0000","last_change":"2023-10-06T03:52:30.785750+0000","last_active":"2023-10-06T03:52:30.791534+0000","last_peered":"2023-10-06T03:52:30.791534+0000","last_clean":"2023-10-06T03:52:30.791534+0000","last_became_active":"2023-10-06T03:52:30.785510+0000","last_became_peered":"2023-10-06T03:52:30.785510+0000","last_unstale":"2023-10-06T03:52:30.791534+0000","last_undegraded":"2023-10-06T03:52:30.791534+0000","last_fullsized":"2023-10-06T03:52:30.791534+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:38.497087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.798553+0000","last_change":"2023-10-06T03:52:30.798553+0000","last_active":"2023-10-06T03:52:30.798553+0000","last_peered":"2023-10-06T03:52:30.798553+0000","last_clean":"2023-10-06T03:52:30.798553+0000","last_became_active":"2023-10-06T03:52:30.798266+0000","last_became_peered":"2023-10-06T03:52:30.798266+0000","last_unstale":"2023-10-06T03:52:30.798553+0000","last_undegraded":"2023-10-06T03:52:30.798553+0000","last_fullsized":"2023-10-06T03:52:30.798553+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:15.592998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.781489+0000","last_change":"2023-10-06T03:52:30.781489+0000","last_active":"2023-10-06T03:52:30.781489+0000","last_peered":"2023-10-06T03:52:30.781489+0000","last_clean":"2023-10-06T03:52:30.781489+0000","last_became_active":"2023-10-06T03:52:30.781246+0000","last_became_peered":"2023-10-06T03:52:30.781246+0000","last_unstale":"2023-10-06T03:52:30.781489+0000","last_undegraded":"2023-10-06T03:52:30.781489+0000","last_fullsized":"2023-10-06T03:52:30.781489+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:18.017097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.788252+0000","last_change":"2023-10-06T03:52:30.786019+0000","last_active":"2023-10-06T03:52:30.788252+0000","last_peered":"2023-10-06T03:52:30.788252+0000","last_clean":"2023-10-06T03:52:30.788252+0000","last_became_active":"2023-10-06T03:52:30.785686+0000","last_became_peered":"2023-10-06T03:52:30.785686+0000","last_unstale":"2023-10-06T03:52:30.788252+0000","last_undegraded":"2023-10-06T03:52:30.788252+0000","last_fullsized":"2023-10-06T03:52:30.788252+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:31:28.545878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.786728+0000","last_change":"2023-10-06T03:52:30.786728+0000","last_active":"2023-10-06T03:52:30.786728+0000","last_peered":"2023-10-06T03:52:30.786728+0000","last_clean":"2023-10-06T03:52:30.786728+0000","last_became_active":"2023-10-06T03:52:30.785935+0000","last_became_peered":"2023-10-06T03:52:30.785935+0000","last_unstale":"2023-10-06T03:52:30.786728+0000","last_undegraded":"2023-10-06T03:52:30.786728+0000","last_fullsized":"2023-10-06T03:52:30.786728+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:29.816312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.785174+0000","last_change":"2023-10-06T03:52:30.785174+0000","last_active":"2023-10-06T03:52:30.785174+0000","last_peered":"2023-10-06T03:52:30.785174+0000","last_clean":"2023-10-06T03:52:30.785174+0000","last_became_active":"2023-10-06T03:52:30.784914+0000","last_became_peered":"2023-10-06T03:52:30.784914+0000","last_unstale":"2023-10-06T03:52:30.785174+0000","last_undegraded":"2023-10-06T03:52:30.785174+0000","last_fullsized":"2023-10-06T03:52:30.785174+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:05.986856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792881+0000","last_change":"2023-10-06T03:52:30.789284+0000","last_active":"2023-10-06T03:52:30.792881+0000","last_peered":"2023-10-06T03:52:30.792881+0000","last_clean":"2023-10-06T03:52:30.792881+0000","last_became_active":"2023-10-06T03:52:30.788703+0000","last_became_peered":"2023-10-06T03:52:30.788703+0000","last_unstale":"2023-10-06T03:52:30.792881+0000","last_undegraded":"2023-10-06T03:52:30.792881+0000","last_fullsized":"2023-10-06T03:52:30.792881+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:27.224627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.787993+0000","last_change":"2023-10-06T03:52:30.786907+0000","last_active":"2023-10-06T03:52:30.787993+0000","last_peered":"2023-10-06T03:52:30.787993+0000","last_clean":"2023-10-06T03:52:30.787993+0000","last_became_active":"2023-10-06T03:52:30.786654+0000","last_became_peered":"2023-10-06T03:52:30.786654+0000","last_unstale":"2023-10-06T03:52:30.787993+0000","last_undegraded":"2023-10-06T03:52:30.787993+0000","last_fullsized":"2023-10-06T03:52:30.787993+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:21.845674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.782101+0000","last_change":"2023-10-06T03:52:30.782101+0000","last_active":"2023-10-06T03:52:30.782101+0000","last_peered":"2023-10-06T03:52:30.782101+0000","last_clean":"2023-10-06T03:52:30.782101+0000","last_became_active":"2023-10-06T03:52:30.780786+0000","last_became_peered":"2023-10-06T03:52:30.780786+0000","last_unstale":"2023-10-06T03:52:30.782101+0000","last_undegraded":"2023-10-06T03:52:30.782101+0000","last_fullsized":"2023-10-06T03:52:30.782101+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:12.142020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.790989+0000","last_change":"2023-10-06T03:52:30.790989+0000","last_active":"2023-10-06T03:52:30.790989+0000","last_peered":"2023-10-06T03:52:30.790989+0000","last_clean":"2023-10-06T03:52:30.790989+0000","last_became_active":"2023-10-06T03:52:30.789650+0000","last_became_peered":"2023-10-06T03:52:30.789650+0000","last_unstale":"2023-10-06T03:52:30.790989+0000","last_undegraded":"2023-10-06T03:52:30.790989+0000","last_fullsized":"2023-10-06T03:52:30.790989+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:51.019536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792686+0000","last_change":"2023-10-06T03:52:30.789977+0000","last_active":"2023-10-06T03:52:30.792686+0000","last_peered":"2023-10-06T03:52:30.792686+0000","last_clean":"2023-10-06T03:52:30.792686+0000","last_became_active":"2023-10-06T03:52:30.789689+0000","last_became_peered":"2023-10-06T03:52:30.789689+0000","last_unstale":"2023-10-06T03:52:30.792686+0000","last_undegraded":"2023-10-06T03:52:30.792686+0000","last_fullsized":"2023-10-06T03:52:30.792686+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:50.603686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.793194+0000","last_change":"2023-10-06T03:52:30.789478+0000","last_active":"2023-10-06T03:52:30.793194+0000","last_peered":"2023-10-06T03:52:30.793194+0000","last_clean":"2023-10-06T03:52:30.793194+0000","last_became_active":"2023-10-06T03:52:30.789179+0000","last_became_peered":"2023-10-06T03:52:30.789179+0000","last_unstale":"2023-10-06T03:52:30.793194+0000","last_undegraded":"2023-10-06T03:52:30.793194+0000","last_fullsized":"2023-10-06T03:52:30.793194+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:53.475559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.782858+0000","last_change":"2023-10-06T03:52:30.782858+0000","last_active":"2023-10-06T03:52:30.782858+0000","last_peered":"2023-10-06T03:52:30.782858+0000","last_clean":"2023-10-06T03:52:30.782858+0000","last_became_active":"2023-10-06T03:52:30.782599+0000","last_became_peered":"2023-10-06T03:52:30.782599+0000","last_unstale":"2023-10-06T03:52:30.782858+0000","last_undegraded":"2023-10-06T03:52:30.782858+0000","last_fullsized":"2023-10-06T03:52:30.782858+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:51.424396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.796174+0000","last_change":"2023-10-06T03:52:30.796174+0000","last_active":"2023-10-06T03:52:30.796174+0000","last_peered":"2023-10-06T03:52:30.796174+0000","last_clean":"2023-10-06T03:52:30.796174+0000","last_became_active":"2023-10-06T03:52:30.795915+0000","last_became_peered":"2023-10-06T03:52:30.795915+0000","last_unstale":"2023-10-06T03:52:30.796174+0000","last_undegraded":"2023-10-06T03:52:30.796174+0000","last_fullsized":"2023-10-06T03:52:30.796174+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:19.902819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.782330+0000","last_change":"2023-10-06T03:52:30.780064+0000","last_active":"2023-10-06T03:52:30.782330+0000","last_peered":"2023-10-06T03:52:30.782330+0000","last_clean":"2023-10-06T03:52:30.782330+0000","last_became_active":"2023-10-06T03:52:30.779799+0000","last_became_peered":"2023-10-06T03:52:30.779799+0000","last_unstale":"2023-10-06T03:52:30.782330+0000","last_undegraded":"2023-10-06T03:52:30.782330+0000","last_fullsized":"2023-10-06T03:52:30.782330+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:02.370347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.799023+0000","last_change":"2023-10-06T03:52:30.799023+0000","last_active":"2023-10-06T03:52:30.799023+0000","last_peered":"2023-10-06T03:52:30.799023+0000","last_clean":"2023-10-06T03:52:30.799023+0000","last_became_active":"2023-10-06T03:52:30.798724+0000","last_became_peered":"2023-10-06T03:52:30.798724+0000","last_unstale":"2023-10-06T03:52:30.799023+0000","last_undegraded":"2023-10-06T03:52:30.799023+0000","last_fullsized":"2023-10-06T03:52:30.799023+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:49.834343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.789061+0000","last_change":"2023-10-06T03:52:30.789061+0000","last_active":"2023-10-06T03:52:30.789061+0000","last_peered":"2023-10-06T03:52:30.789061+0000","last_clean":"2023-10-06T03:52:30.789061+0000","last_became_active":"2023-10-06T03:52:30.788786+0000","last_became_peered":"2023-10-06T03:52:30.788786+0000","last_unstale":"2023-10-06T03:52:30.789061+0000","last_undegraded":"2023-10-06T03:52:30.789061+0000","last_fullsized":"2023-10-06T03:52:30.789061+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:38.872611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792438+0000","last_change":"2023-10-06T03:52:30.792438+0000","last_active":"2023-10-06T03:52:30.792438+0000","last_peered":"2023-10-06T03:52:30.792438+0000","last_clean":"2023-10-06T03:52:30.792438+0000","last_became_active":"2023-10-06T03:52:30.792144+0000","last_became_peered":"2023-10-06T03:52:30.792144+0000","last_unstale":"2023-10-06T03:52:30.792438+0000","last_undegraded":"2023-10-06T03:52:30.792438+0000","last_fullsized":"2023-10-06T03:52:30.792438+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:23.019719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.781661+0000","last_change":"2023-10-06T03:52:30.781661+0000","last_active":"2023-10-06T03:52:30.781661+0000","last_peered":"2023-10-06T03:52:30.781661+0000","last_clean":"2023-10-06T03:52:30.781661+0000","last_became_active":"2023-10-06T03:52:30.781399+0000","last_became_peered":"2023-10-06T03:52:30.781399+0000","last_unstale":"2023-10-06T03:52:30.781661+0000","last_undegraded":"2023-10-06T03:52:30.781661+0000","last_fullsized":"2023-10-06T03:52:30.781661+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:58.880990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.793214+0000","last_change":"2023-10-06T03:52:30.789509+0000","last_active":"2023-10-06T03:52:30.793214+0000","last_peered":"2023-10-06T03:52:30.793214+0000","last_clean":"2023-10-06T03:52:30.793214+0000","last_became_active":"2023-10-06T03:52:30.789222+0000","last_became_peered":"2023-10-06T03:52:30.789222+0000","last_unstale":"2023-10-06T03:52:30.793214+0000","last_undegraded":"2023-10-06T03:52:30.793214+0000","last_fullsized":"2023-10-06T03:52:30.793214+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:16.612133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.783357+0000","last_change":"2023-10-06T03:52:30.783357+0000","last_active":"2023-10-06T03:52:30.783357+0000","last_peered":"2023-10-06T03:52:30.783357+0000","last_clean":"2023-10-06T03:52:30.783357+0000","last_became_active":"2023-10-06T03:52:30.782815+0000","last_became_peered":"2023-10-06T03:52:30.782815+0000","last_unstale":"2023-10-06T03:52:30.783357+0000","last_undegraded":"2023-10-06T03:52:30.783357+0000","last_fullsized":"2023-10-06T03:52:30.783357+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:50.704636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.787322+0000","last_change":"2023-10-06T03:52:30.787322+0000","last_active":"2023-10-06T03:52:30.787322+0000","last_peered":"2023-10-06T03:52:30.787322+0000","last_clean":"2023-10-06T03:52:30.787322+0000","last_became_active":"2023-10-06T03:52:30.787081+0000","last_became_peered":"2023-10-06T03:52:30.787081+0000","last_unstale":"2023-10-06T03:52:30.787322+0000","last_undegraded":"2023-10-06T03:52:30.787322+0000","last_fullsized":"2023-10-06T03:52:30.787322+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:40.364445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791730+0000","last_change":"2023-10-06T03:52:30.786966+0000","last_active":"2023-10-06T03:52:30.791730+0000","last_peered":"2023-10-06T03:52:30.791730+0000","last_clean":"2023-10-06T03:52:30.791730+0000","last_became_active":"2023-10-06T03:52:30.786709+0000","last_became_peered":"2023-10-06T03:52:30.786709+0000","last_unstale":"2023-10-06T03:52:30.791730+0000","last_undegraded":"2023-10-06T03:52:30.791730+0000","last_fullsized":"2023-10-06T03:52:30.791730+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:40.199639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.788080+0000","last_change":"2023-10-06T03:52:30.788080+0000","last_active":"2023-10-06T03:52:30.788080+0000","last_peered":"2023-10-06T03:52:30.788080+0000","last_clean":"2023-10-06T03:52:30.788080+0000","last_became_active":"2023-10-06T03:52:30.787766+0000","last_became_peered":"2023-10-06T03:52:30.787766+0000","last_unstale":"2023-10-06T03:52:30.788080+0000","last_undegraded":"2023-10-06T03:52:30.788080+0000","last_fullsized":"2023-10-06T03:52:30.788080+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:54.235229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.788385+0000","last_change":"2023-10-06T03:52:30.788385+0000","last_active":"2023-10-06T03:52:30.788385+0000","last_peered":"2023-10-06T03:52:30.788385+0000","last_clean":"2023-10-06T03:52:30.788385+0000","last_became_active":"2023-10-06T03:52:30.788096+0000","last_became_peered":"2023-10-06T03:52:30.788096+0000","last_unstale":"2023-10-06T03:52:30.788385+0000","last_undegraded":"2023-10-06T03:52:30.788385+0000","last_fullsized":"2023-10-06T03:52:30.788385+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:06.430173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.782949+0000","last_change":"2023-10-06T03:52:30.782949+0000","last_active":"2023-10-06T03:52:30.782949+0000","last_peered":"2023-10-06T03:52:30.782949+0000","last_clean":"2023-10-06T03:52:30.782949+0000","last_became_active":"2023-10-06T03:52:30.782656+0000","last_became_peered":"2023-10-06T03:52:30.782656+0000","last_unstale":"2023-10-06T03:52:30.782949+0000","last_undegraded":"2023-10-06T03:52:30.782949+0000","last_fullsized":"2023-10-06T03:52:30.782949+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:49.834343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"136'4","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.140386+0000","last_change":"2023-10-06T03:52:30.122412+0000","last_active":"2023-10-06T03:52:31.140386+0000","last_peered":"2023-10-06T03:52:31.140386+0000","last_clean":"2023-10-06T03:52:31.140386+0000","last_became_active":"2023-10-06T03:52:30.122148+0000","last_became_peered":"2023-10-06T03:52:30.122148+0000","last_unstale":"2023-10-06T03:52:31.140386+0000","last_undegraded":"2023-10-06T03:52:31.140386+0000","last_fullsized":"2023-10-06T03:52:31.140386+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:49.189447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792867+0000","last_change":"2023-10-06T03:52:30.792867+0000","last_active":"2023-10-06T03:52:30.792867+0000","last_peered":"2023-10-06T03:52:30.792867+0000","last_clean":"2023-10-06T03:52:30.792867+0000","last_became_active":"2023-10-06T03:52:30.792401+0000","last_became_peered":"2023-10-06T03:52:30.792401+0000","last_unstale":"2023-10-06T03:52:30.792867+0000","last_undegraded":"2023-10-06T03:52:30.792867+0000","last_fullsized":"2023-10-06T03:52:30.792867+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:11.861935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"136'1","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.771658+0000","last_change":"2023-10-06T03:52:30.115824+0000","last_active":"2023-10-06T03:52:30.771658+0000","last_peered":"2023-10-06T03:52:30.771658+0000","last_clean":"2023-10-06T03:52:30.771658+0000","last_became_active":"2023-10-06T03:52:30.115527+0000","last_became_peered":"2023-10-06T03:52:30.115527+0000","last_unstale":"2023-10-06T03:52:30.771658+0000","last_undegraded":"2023-10-06T03:52:30.771658+0000","last_fullsized":"2023-10-06T03:52:30.771658+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:19.855379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.783882+0000","last_change":"2023-10-06T03:52:30.783882+0000","last_active":"2023-10-06T03:52:30.783882+0000","last_peered":"2023-10-06T03:52:30.783882+0000","last_clean":"2023-10-06T03:52:30.783882+0000","last_became_active":"2023-10-06T03:52:30.783597+0000","last_became_peered":"2023-10-06T03:52:30.783597+0000","last_unstale":"2023-10-06T03:52:30.783882+0000","last_undegraded":"2023-10-06T03:52:30.783882+0000","last_fullsized":"2023-10-06T03:52:30.783882+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:51.019536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"136'2","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.138473+0000","last_change":"2023-10-06T03:52:30.126541+0000","last_active":"2023-10-06T03:52:31.138473+0000","last_peered":"2023-10-06T03:52:31.138473+0000","last_clean":"2023-10-06T03:52:31.138473+0000","last_became_active":"2023-10-06T03:52:30.126207+0000","last_became_peered":"2023-10-06T03:52:30.126207+0000","last_unstale":"2023-10-06T03:52:31.138473+0000","last_undegraded":"2023-10-06T03:52:31.138473+0000","last_fullsized":"2023-10-06T03:52:31.138473+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:29.096899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.795521+0000","last_change":"2023-10-06T03:52:30.795521+0000","last_active":"2023-10-06T03:52:30.795521+0000","last_peered":"2023-10-06T03:52:30.795521+0000","last_clean":"2023-10-06T03:52:30.795521+0000","last_became_active":"2023-10-06T03:52:30.795269+0000","last_became_peered":"2023-10-06T03:52:30.795269+0000","last_unstale":"2023-10-06T03:52:30.795521+0000","last_undegraded":"2023-10-06T03:52:30.795521+0000","last_fullsized":"2023-10-06T03:52:30.795521+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:13.980241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"136'2","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.141212+0000","last_change":"2023-10-06T03:52:30.114186+0000","last_active":"2023-10-06T03:52:31.141212+0000","last_peered":"2023-10-06T03:52:31.141212+0000","last_clean":"2023-10-06T03:52:31.141212+0000","last_became_active":"2023-10-06T03:52:30.113925+0000","last_became_peered":"2023-10-06T03:52:30.113925+0000","last_unstale":"2023-10-06T03:52:31.141212+0000","last_undegraded":"2023-10-06T03:52:31.141212+0000","last_fullsized":"2023-10-06T03:52:31.141212+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:17.813294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.777518+0000","last_change":"2023-10-06T03:52:30.777518+0000","last_active":"2023-10-06T03:52:30.777518+0000","last_peered":"2023-10-06T03:52:30.777518+0000","last_clean":"2023-10-06T03:52:30.777518+0000","last_became_active":"2023-10-06T03:52:30.777243+0000","last_became_peered":"2023-10-06T03:52:30.777243+0000","last_unstale":"2023-10-06T03:52:30.777518+0000","last_undegraded":"2023-10-06T03:52:30.777518+0000","last_fullsized":"2023-10-06T03:52:30.777518+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:08.324906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.138812+0000","last_change":"2023-10-06T03:52:30.123795+0000","last_active":"2023-10-06T03:52:31.138812+0000","last_peered":"2023-10-06T03:52:31.138812+0000","last_clean":"2023-10-06T03:52:31.138812+0000","last_became_active":"2023-10-06T03:52:30.123344+0000","last_became_peered":"2023-10-06T03:52:30.123344+0000","last_unstale":"2023-10-06T03:52:31.138812+0000","last_undegraded":"2023-10-06T03:52:31.138812+0000","last_fullsized":"2023-10-06T03:52:31.138812+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:49.592921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.788538+0000","last_change":"2023-10-06T03:52:30.787672+0000","last_active":"2023-10-06T03:52:30.788538+0000","last_peered":"2023-10-06T03:52:30.788538+0000","last_clean":"2023-10-06T03:52:30.788538+0000","last_became_active":"2023-10-06T03:52:30.787408+0000","last_became_peered":"2023-10-06T03:52:30.787408+0000","last_unstale":"2023-10-06T03:52:30.788538+0000","last_undegraded":"2023-10-06T03:52:30.788538+0000","last_fullsized":"2023-10-06T03:52:30.788538+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:55.058210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"136'1","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.776781+0000","last_change":"2023-10-06T03:52:30.121391+0000","last_active":"2023-10-06T03:52:30.776781+0000","last_peered":"2023-10-06T03:52:30.776781+0000","last_clean":"2023-10-06T03:52:30.776781+0000","last_became_active":"2023-10-06T03:52:30.120531+0000","last_became_peered":"2023-10-06T03:52:30.120531+0000","last_unstale":"2023-10-06T03:52:30.776781+0000","last_undegraded":"2023-10-06T03:52:30.776781+0000","last_fullsized":"2023-10-06T03:52:30.776781+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:54.236766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791211+0000","last_change":"2023-10-06T03:52:30.781510+0000","last_active":"2023-10-06T03:52:30.791211+0000","last_peered":"2023-10-06T03:52:30.791211+0000","last_clean":"2023-10-06T03:52:30.791211+0000","last_became_active":"2023-10-06T03:52:30.781269+0000","last_became_peered":"2023-10-06T03:52:30.781269+0000","last_unstale":"2023-10-06T03:52:30.791211+0000","last_undegraded":"2023-10-06T03:52:30.791211+0000","last_fullsized":"2023-10-06T03:52:30.791211+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:43.869500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.124808+0000","last_change":"2023-10-06T03:52:30.119373+0000","last_active":"2023-10-06T03:52:30.124808+0000","last_peered":"2023-10-06T03:52:30.124808+0000","last_clean":"2023-10-06T03:52:30.124808+0000","last_became_active":"2023-10-06T03:52:30.119118+0000","last_became_peered":"2023-10-06T03:52:30.119118+0000","last_unstale":"2023-10-06T03:52:30.124808+0000","last_undegraded":"2023-10-06T03:52:30.124808+0000","last_fullsized":"2023-10-06T03:52:30.124808+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:52.555468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792187+0000","last_change":"2023-10-06T03:52:30.792187+0000","last_active":"2023-10-06T03:52:30.792187+0000","last_peered":"2023-10-06T03:52:30.792187+0000","last_clean":"2023-10-06T03:52:30.792187+0000","last_became_active":"2023-10-06T03:52:30.791904+0000","last_became_peered":"2023-10-06T03:52:30.791904+0000","last_unstale":"2023-10-06T03:52:30.792187+0000","last_undegraded":"2023-10-06T03:52:30.792187+0000","last_fullsized":"2023-10-06T03:52:30.792187+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:48.623123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.124912+0000","last_change":"2023-10-06T03:52:30.123936+0000","last_active":"2023-10-06T03:52:30.124912+0000","last_peered":"2023-10-06T03:52:30.124912+0000","last_clean":"2023-10-06T03:52:30.124912+0000","last_became_active":"2023-10-06T03:52:30.121761+0000","last_became_peered":"2023-10-06T03:52:30.121761+0000","last_unstale":"2023-10-06T03:52:30.124912+0000","last_undegraded":"2023-10-06T03:52:30.124912+0000","last_fullsized":"2023-10-06T03:52:30.124912+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:49.625382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.786434+0000","last_change":"2023-10-06T03:52:30.786434+0000","last_active":"2023-10-06T03:52:30.786434+0000","last_peered":"2023-10-06T03:52:30.786434+0000","last_clean":"2023-10-06T03:52:30.786434+0000","last_became_active":"2023-10-06T03:52:30.786137+0000","last_became_peered":"2023-10-06T03:52:30.786137+0000","last_unstale":"2023-10-06T03:52:30.786434+0000","last_undegraded":"2023-10-06T03:52:30.786434+0000","last_fullsized":"2023-10-06T03:52:30.786434+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:44.658960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"136'1","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.772169+0000","last_change":"2023-10-06T03:52:30.122297+0000","last_active":"2023-10-06T03:52:30.772169+0000","last_peered":"2023-10-06T03:52:30.772169+0000","last_clean":"2023-10-06T03:52:30.772169+0000","last_became_active":"2023-10-06T03:52:30.122053+0000","last_became_peered":"2023-10-06T03:52:30.122053+0000","last_unstale":"2023-10-06T03:52:30.772169+0000","last_undegraded":"2023-10-06T03:52:30.772169+0000","last_fullsized":"2023-10-06T03:52:30.772169+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:08.789519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.785896+0000","last_change":"2023-10-06T03:52:30.780142+0000","last_active":"2023-10-06T03:52:30.785896+0000","last_peered":"2023-10-06T03:52:30.785896+0000","last_clean":"2023-10-06T03:52:30.785896+0000","last_became_active":"2023-10-06T03:52:30.779316+0000","last_became_peered":"2023-10-06T03:52:30.779316+0000","last_unstale":"2023-10-06T03:52:30.785896+0000","last_undegraded":"2023-10-06T03:52:30.785896+0000","last_fullsized":"2023-10-06T03:52:30.785896+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:40.343603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"136'2","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.139391+0000","last_change":"2023-10-06T03:52:30.123022+0000","last_active":"2023-10-06T03:52:31.139391+0000","last_peered":"2023-10-06T03:52:31.139391+0000","last_clean":"2023-10-06T03:52:31.139391+0000","last_became_active":"2023-10-06T03:52:30.122727+0000","last_became_peered":"2023-10-06T03:52:30.122727+0000","last_unstale":"2023-10-06T03:52:31.139391+0000","last_undegraded":"2023-10-06T03:52:31.139391+0000","last_fullsized":"2023-10-06T03:52:31.139391+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:35.897602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.790414+0000","last_change":"2023-10-06T03:52:30.790414+0000","last_active":"2023-10-06T03:52:30.790414+0000","last_peered":"2023-10-06T03:52:30.790414+0000","last_clean":"2023-10-06T03:52:30.790414+0000","last_became_active":"2023-10-06T03:52:30.790072+0000","last_became_peered":"2023-10-06T03:52:30.790072+0000","last_unstale":"2023-10-06T03:52:30.790414+0000","last_undegraded":"2023-10-06T03:52:30.790414+0000","last_fullsized":"2023-10-06T03:52:30.790414+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:20.822712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"136'1","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.782695+0000","last_change":"2023-10-06T03:52:30.126842+0000","last_active":"2023-10-06T03:52:30.782695+0000","last_peered":"2023-10-06T03:52:30.782695+0000","last_clean":"2023-10-06T03:52:30.782695+0000","last_became_active":"2023-10-06T03:52:30.126484+0000","last_became_peered":"2023-10-06T03:52:30.126484+0000","last_unstale":"2023-10-06T03:52:30.782695+0000","last_undegraded":"2023-10-06T03:52:30.782695+0000","last_fullsized":"2023-10-06T03:52:30.782695+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:37.861846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791182+0000","last_change":"2023-10-06T03:52:30.791182+0000","last_active":"2023-10-06T03:52:30.791182+0000","last_peered":"2023-10-06T03:52:30.791182+0000","last_clean":"2023-10-06T03:52:30.791182+0000","last_became_active":"2023-10-06T03:52:30.790881+0000","last_became_peered":"2023-10-06T03:52:30.790881+0000","last_unstale":"2023-10-06T03:52:30.791182+0000","last_undegraded":"2023-10-06T03:52:30.791182+0000","last_fullsized":"2023-10-06T03:52:30.791182+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:05.986856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.131452+0000","last_change":"2023-10-06T03:52:30.126295+0000","last_active":"2023-10-06T03:52:30.131452+0000","last_peered":"2023-10-06T03:52:30.131452+0000","last_clean":"2023-10-06T03:52:30.131452+0000","last_became_active":"2023-10-06T03:52:30.126042+0000","last_became_peered":"2023-10-06T03:52:30.126042+0000","last_unstale":"2023-10-06T03:52:30.131452+0000","last_undegraded":"2023-10-06T03:52:30.131452+0000","last_fullsized":"2023-10-06T03:52:30.131452+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:39.364845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791424+0000","last_change":"2023-10-06T03:52:30.791424+0000","last_active":"2023-10-06T03:52:30.791424+0000","last_peered":"2023-10-06T03:52:30.791424+0000","last_clean":"2023-10-06T03:52:30.791424+0000","last_became_active":"2023-10-06T03:52:30.790495+0000","last_became_peered":"2023-10-06T03:52:30.790495+0000","last_unstale":"2023-10-06T03:52:30.791424+0000","last_undegraded":"2023-10-06T03:52:30.791424+0000","last_fullsized":"2023-10-06T03:52:30.791424+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:41.471661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"136'2","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.140692+0000","last_change":"2023-10-06T03:52:30.127220+0000","last_active":"2023-10-06T03:52:31.140692+0000","last_peered":"2023-10-06T03:52:31.140692+0000","last_clean":"2023-10-06T03:52:31.140692+0000","last_became_active":"2023-10-06T03:52:30.126966+0000","last_became_peered":"2023-10-06T03:52:30.126966+0000","last_unstale":"2023-10-06T03:52:31.140692+0000","last_undegraded":"2023-10-06T03:52:31.140692+0000","last_fullsized":"2023-10-06T03:52:31.140692+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:06.959275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.794648+0000","last_change":"2023-10-06T03:52:30.793815+0000","last_active":"2023-10-06T03:52:30.794648+0000","last_peered":"2023-10-06T03:52:30.794648+0000","last_clean":"2023-10-06T03:52:30.794648+0000","last_became_active":"2023-10-06T03:52:30.793596+0000","last_became_peered":"2023-10-06T03:52:30.793596+0000","last_unstale":"2023-10-06T03:52:30.794648+0000","last_undegraded":"2023-10-06T03:52:30.794648+0000","last_fullsized":"2023-10-06T03:52:30.794648+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:53.684006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.131594+0000","last_change":"2023-10-06T03:52:30.126930+0000","last_active":"2023-10-06T03:52:30.131594+0000","last_peered":"2023-10-06T03:52:30.131594+0000","last_clean":"2023-10-06T03:52:30.131594+0000","last_became_active":"2023-10-06T03:52:30.126448+0000","last_became_peered":"2023-10-06T03:52:30.126448+0000","last_unstale":"2023-10-06T03:52:30.131594+0000","last_undegraded":"2023-10-06T03:52:30.131594+0000","last_fullsized":"2023-10-06T03:52:30.131594+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:05.419407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792835+0000","last_change":"2023-10-06T03:52:30.788782+0000","last_active":"2023-10-06T03:52:30.792835+0000","last_peered":"2023-10-06T03:52:30.792835+0000","last_clean":"2023-10-06T03:52:30.792835+0000","last_became_active":"2023-10-06T03:52:30.788502+0000","last_became_peered":"2023-10-06T03:52:30.788502+0000","last_unstale":"2023-10-06T03:52:30.792835+0000","last_undegraded":"2023-10-06T03:52:30.792835+0000","last_fullsized":"2023-10-06T03:52:30.792835+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:08.176820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.775372+0000","last_change":"2023-10-06T03:52:30.123836+0000","last_active":"2023-10-06T03:52:30.775372+0000","last_peered":"2023-10-06T03:52:30.775372+0000","last_clean":"2023-10-06T03:52:30.775372+0000","last_became_active":"2023-10-06T03:52:30.123516+0000","last_became_peered":"2023-10-06T03:52:30.123516+0000","last_unstale":"2023-10-06T03:52:30.775372+0000","last_undegraded":"2023-10-06T03:52:30.775372+0000","last_fullsized":"2023-10-06T03:52:30.775372+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:47.339010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791009+0000","last_change":"2023-10-06T03:52:30.791009+0000","last_active":"2023-10-06T03:52:30.791009+0000","last_peered":"2023-10-06T03:52:30.791009+0000","last_clean":"2023-10-06T03:52:30.791009+0000","last_became_active":"2023-10-06T03:52:30.790739+0000","last_became_peered":"2023-10-06T03:52:30.790739+0000","last_unstale":"2023-10-06T03:52:30.791009+0000","last_undegraded":"2023-10-06T03:52:30.791009+0000","last_fullsized":"2023-10-06T03:52:30.791009+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:30.107665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.133884+0000","last_change":"2023-10-06T03:52:30.130812+0000","last_active":"2023-10-06T03:52:30.133884+0000","last_peered":"2023-10-06T03:52:30.133884+0000","last_clean":"2023-10-06T03:52:30.133884+0000","last_became_active":"2023-10-06T03:52:30.130450+0000","last_became_peered":"2023-10-06T03:52:30.130450+0000","last_unstale":"2023-10-06T03:52:30.133884+0000","last_undegraded":"2023-10-06T03:52:30.133884+0000","last_fullsized":"2023-10-06T03:52:30.133884+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:47.339010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.785094+0000","last_change":"2023-10-06T03:52:30.785094+0000","last_active":"2023-10-06T03:52:30.785094+0000","last_peered":"2023-10-06T03:52:30.785094+0000","last_clean":"2023-10-06T03:52:30.785094+0000","last_became_active":"2023-10-06T03:52:30.784835+0000","last_became_peered":"2023-10-06T03:52:30.784835+0000","last_unstale":"2023-10-06T03:52:30.785094+0000","last_undegraded":"2023-10-06T03:52:30.785094+0000","last_fullsized":"2023-10-06T03:52:30.785094+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:38.355926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"136'2","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.139253+0000","last_change":"2023-10-06T03:52:30.127133+0000","last_active":"2023-10-06T03:52:31.139253+0000","last_peered":"2023-10-06T03:52:31.139253+0000","last_clean":"2023-10-06T03:52:31.139253+0000","last_became_active":"2023-10-06T03:52:30.126818+0000","last_became_peered":"2023-10-06T03:52:30.126818+0000","last_unstale":"2023-10-06T03:52:31.139253+0000","last_undegraded":"2023-10-06T03:52:31.139253+0000","last_fullsized":"2023-10-06T03:52:31.139253+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:57.985312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.778510+0000","last_change":"2023-10-06T03:52:30.778510+0000","last_active":"2023-10-06T03:52:30.778510+0000","last_peered":"2023-10-06T03:52:30.778510+0000","last_clean":"2023-10-06T03:52:30.778510+0000","last_became_active":"2023-10-06T03:52:30.778050+0000","last_became_peered":"2023-10-06T03:52:30.778050+0000","last_unstale":"2023-10-06T03:52:30.778510+0000","last_undegraded":"2023-10-06T03:52:30.778510+0000","last_fullsized":"2023-10-06T03:52:30.778510+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:08:45.904493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.774512+0000","last_change":"2023-10-06T03:52:30.124624+0000","last_active":"2023-10-06T03:52:30.774512+0000","last_peered":"2023-10-06T03:52:30.774512+0000","last_clean":"2023-10-06T03:52:30.774512+0000","last_became_active":"2023-10-06T03:52:30.124329+0000","last_became_peered":"2023-10-06T03:52:30.124329+0000","last_unstale":"2023-10-06T03:52:30.774512+0000","last_undegraded":"2023-10-06T03:52:30.774512+0000","last_fullsized":"2023-10-06T03:52:30.774512+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:06.737045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792657+0000","last_change":"2023-10-06T03:52:30.792657+0000","last_active":"2023-10-06T03:52:30.792657+0000","last_peered":"2023-10-06T03:52:30.792657+0000","last_clean":"2023-10-06T03:52:30.792657+0000","last_became_active":"2023-10-06T03:52:30.791489+0000","last_became_peered":"2023-10-06T03:52:30.791489+0000","last_unstale":"2023-10-06T03:52:30.792657+0000","last_undegraded":"2023-10-06T03:52:30.792657+0000","last_fullsized":"2023-10-06T03:52:30.792657+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:23.327175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.774777+0000","last_change":"2023-10-06T03:52:30.125853+0000","last_active":"2023-10-06T03:52:30.774777+0000","last_peered":"2023-10-06T03:52:30.774777+0000","last_clean":"2023-10-06T03:52:30.774777+0000","last_became_active":"2023-10-06T03:52:30.125026+0000","last_became_peered":"2023-10-06T03:52:30.125026+0000","last_unstale":"2023-10-06T03:52:30.774777+0000","last_undegraded":"2023-10-06T03:52:30.774777+0000","last_fullsized":"2023-10-06T03:52:30.774777+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:14.292118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.789955+0000","last_change":"2023-10-06T03:52:30.789955+0000","last_active":"2023-10-06T03:52:30.789955+0000","last_peered":"2023-10-06T03:52:30.789955+0000","last_clean":"2023-10-06T03:52:30.789955+0000","last_became_active":"2023-10-06T03:52:30.789685+0000","last_became_peered":"2023-10-06T03:52:30.789685+0000","last_unstale":"2023-10-06T03:52:30.789955+0000","last_undegraded":"2023-10-06T03:52:30.789955+0000","last_fullsized":"2023-10-06T03:52:30.789955+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:26.405684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.128980+0000","last_change":"2023-10-06T03:52:30.126293+0000","last_active":"2023-10-06T03:52:30.128980+0000","last_peered":"2023-10-06T03:52:30.128980+0000","last_clean":"2023-10-06T03:52:30.128980+0000","last_became_active":"2023-10-06T03:52:30.126058+0000","last_became_peered":"2023-10-06T03:52:30.126058+0000","last_unstale":"2023-10-06T03:52:30.128980+0000","last_undegraded":"2023-10-06T03:52:30.128980+0000","last_fullsized":"2023-10-06T03:52:30.128980+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:13.116018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.787768+0000","last_change":"2023-10-06T03:52:30.787768+0000","last_active":"2023-10-06T03:52:30.787768+0000","last_peered":"2023-10-06T03:52:30.787768+0000","last_clean":"2023-10-06T03:52:30.787768+0000","last_became_active":"2023-10-06T03:52:30.787507+0000","last_became_peered":"2023-10-06T03:52:30.787507+0000","last_unstale":"2023-10-06T03:52:30.787768+0000","last_undegraded":"2023-10-06T03:52:30.787768+0000","last_fullsized":"2023-10-06T03:52:30.787768+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:57.047978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.773809+0000","last_change":"2023-10-06T03:52:30.123623+0000","last_active":"2023-10-06T03:52:30.773809+0000","last_peered":"2023-10-06T03:52:30.773809+0000","last_clean":"2023-10-06T03:52:30.773809+0000","last_became_active":"2023-10-06T03:52:30.123051+0000","last_became_peered":"2023-10-06T03:52:30.123051+0000","last_unstale":"2023-10-06T03:52:30.773809+0000","last_undegraded":"2023-10-06T03:52:30.773809+0000","last_fullsized":"2023-10-06T03:52:30.773809+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:39.188874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791661+0000","last_change":"2023-10-06T03:52:30.786538+0000","last_active":"2023-10-06T03:52:30.791661+0000","last_peered":"2023-10-06T03:52:30.791661+0000","last_clean":"2023-10-06T03:52:30.791661+0000","last_became_active":"2023-10-06T03:52:30.786295+0000","last_became_peered":"2023-10-06T03:52:30.786295+0000","last_unstale":"2023-10-06T03:52:30.791661+0000","last_undegraded":"2023-10-06T03:52:30.791661+0000","last_fullsized":"2023-10-06T03:52:30.791661+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:23.327175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.773034+0000","last_change":"2023-10-06T03:52:30.126345+0000","last_active":"2023-10-06T03:52:30.773034+0000","last_peered":"2023-10-06T03:52:30.773034+0000","last_clean":"2023-10-06T03:52:30.773034+0000","last_became_active":"2023-10-06T03:52:30.126070+0000","last_became_peered":"2023-10-06T03:52:30.126070+0000","last_unstale":"2023-10-06T03:52:30.773034+0000","last_undegraded":"2023-10-06T03:52:30.773034+0000","last_fullsized":"2023-10-06T03:52:30.773034+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:26.213861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.788953+0000","last_change":"2023-10-06T03:52:30.788953+0000","last_active":"2023-10-06T03:52:30.788953+0000","last_peered":"2023-10-06T03:52:30.788953+0000","last_clean":"2023-10-06T03:52:30.788953+0000","last_became_active":"2023-10-06T03:52:30.788498+0000","last_became_peered":"2023-10-06T03:52:30.788498+0000","last_unstale":"2023-10-06T03:52:30.788953+0000","last_undegraded":"2023-10-06T03:52:30.788953+0000","last_fullsized":"2023-10-06T03:52:30.788953+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:08.176820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.124204+0000","last_change":"2023-10-06T03:52:30.120855+0000","last_active":"2023-10-06T03:52:30.124204+0000","last_peered":"2023-10-06T03:52:30.124204+0000","last_clean":"2023-10-06T03:52:30.124204+0000","last_became_active":"2023-10-06T03:52:30.120503+0000","last_became_peered":"2023-10-06T03:52:30.120503+0000","last_unstale":"2023-10-06T03:52:30.124204+0000","last_undegraded":"2023-10-06T03:52:30.124204+0000","last_fullsized":"2023-10-06T03:52:30.124204+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:46.440943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.786623+0000","last_change":"2023-10-06T03:52:30.786623+0000","last_active":"2023-10-06T03:52:30.786623+0000","last_peered":"2023-10-06T03:52:30.786623+0000","last_clean":"2023-10-06T03:52:30.786623+0000","last_became_active":"2023-10-06T03:52:30.786384+0000","last_became_peered":"2023-10-06T03:52:30.786384+0000","last_unstale":"2023-10-06T03:52:30.786623+0000","last_undegraded":"2023-10-06T03:52:30.786623+0000","last_fullsized":"2023-10-06T03:52:30.786623+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:52.123264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.131936+0000","last_change":"2023-10-06T03:52:30.127346+0000","last_active":"2023-10-06T03:52:30.131936+0000","last_peered":"2023-10-06T03:52:30.131936+0000","last_clean":"2023-10-06T03:52:30.131936+0000","last_became_active":"2023-10-06T03:52:30.127068+0000","last_became_peered":"2023-10-06T03:52:30.127068+0000","last_unstale":"2023-10-06T03:52:30.131936+0000","last_undegraded":"2023-10-06T03:52:30.131936+0000","last_fullsized":"2023-10-06T03:52:30.131936+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:53.224463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792669+0000","last_change":"2023-10-06T03:52:30.789354+0000","last_active":"2023-10-06T03:52:30.792669+0000","last_peered":"2023-10-06T03:52:30.792669+0000","last_clean":"2023-10-06T03:52:30.792669+0000","last_became_active":"2023-10-06T03:52:30.788899+0000","last_became_peered":"2023-10-06T03:52:30.788899+0000","last_unstale":"2023-10-06T03:52:30.792669+0000","last_undegraded":"2023-10-06T03:52:30.792669+0000","last_fullsized":"2023-10-06T03:52:30.792669+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:13.980241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"136'3","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.139404+0000","last_change":"2023-10-06T03:52:30.120725+0000","last_active":"2023-10-06T03:52:31.139404+0000","last_peered":"2023-10-06T03:52:31.139404+0000","last_clean":"2023-10-06T03:52:31.139404+0000","last_became_active":"2023-10-06T03:52:30.120448+0000","last_became_peered":"2023-10-06T03:52:30.120448+0000","last_unstale":"2023-10-06T03:52:31.139404+0000","last_undegraded":"2023-10-06T03:52:31.139404+0000","last_fullsized":"2023-10-06T03:52:31.139404+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:42.123352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":123,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.772771+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:52:30.772771+0000","last_peered":"2023-10-06T03:52:30.772771+0000","last_clean":"2023-10-06T03:52:30.772771+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:52:30.772771+0000","last_undegraded":"2023-10-06T03:52:30.772771+0000","last_fullsized":"2023-10-06T03:52:30.772771+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791890+0000","last_change":"2023-10-06T03:52:30.791890+0000","last_active":"2023-10-06T03:52:30.791890+0000","last_peered":"2023-10-06T03:52:30.791890+0000","last_clean":"2023-10-06T03:52:30.791890+0000","last_became_active":"2023-10-06T03:52:30.791444+0000","last_became_peered":"2023-10-06T03:52:30.791444+0000","last_unstale":"2023-10-06T03:52:30.791890+0000","last_undegraded":"2023-10-06T03:52:30.791890+0000","last_fullsized":"2023-10-06T03:52:30.791890+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:48.349776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"136'6","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.143486+0000","last_change":"2023-10-06T03:52:30.122962+0000","last_active":"2023-10-06T03:52:31.143486+0000","last_peered":"2023-10-06T03:52:31.143486+0000","last_clean":"2023-10-06T03:52:31.143486+0000","last_became_active":"2023-10-06T03:52:30.122675+0000","last_became_peered":"2023-10-06T03:52:30.122675+0000","last_unstale":"2023-10-06T03:52:31.143486+0000","last_undegraded":"2023-10-06T03:52:31.143486+0000","last_fullsized":"2023-10-06T03:52:31.143486+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:22.008954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.781089+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:52:30.781089+0000","last_peered":"2023-10-06T03:52:30.781089+0000","last_clean":"2023-10-06T03:52:30.781089+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:52:30.781089+0000","last_undegraded":"2023-10-06T03:52:30.781089+0000","last_fullsized":"2023-10-06T03:52:30.781089+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.784795+0000","last_change":"2023-10-06T03:52:30.781226+0000","last_active":"2023-10-06T03:52:30.784795+0000","last_peered":"2023-10-06T03:52:30.784795+0000","last_clean":"2023-10-06T03:52:30.784795+0000","last_became_active":"2023-10-06T03:52:30.780960+0000","last_became_peered":"2023-10-06T03:52:30.780960+0000","last_unstale":"2023-10-06T03:52:30.784795+0000","last_undegraded":"2023-10-06T03:52:30.784795+0000","last_fullsized":"2023-10-06T03:52:30.784795+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:20.866145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.141241+0000","last_change":"2023-10-06T03:52:30.124669+0000","last_active":"2023-10-06T03:52:31.141241+0000","last_peered":"2023-10-06T03:52:31.141241+0000","last_clean":"2023-10-06T03:52:31.141241+0000","last_became_active":"2023-10-06T03:52:30.124349+0000","last_became_peered":"2023-10-06T03:52:30.124349+0000","last_unstale":"2023-10-06T03:52:31.141241+0000","last_undegraded":"2023-10-06T03:52:31.141241+0000","last_fullsized":"2023-10-06T03:52:31.141241+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:39.364845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":113,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.101559+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:52:30.101559+0000","last_peered":"2023-10-06T03:52:30.101559+0000","last_clean":"2023-10-06T03:52:30.101559+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:52:30.101559+0000","last_undegraded":"2023-10-06T03:52:30.101559+0000","last_fullsized":"2023-10-06T03:52:30.101559+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"125'249","reported_seq":731,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.118685+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:52:30.118685+0000","last_peered":"2023-10-06T03:52:30.118685+0000","last_clean":"2023-10-06T03:52:30.118685+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:52:30.118685+0000","last_undegraded":"2023-10-06T03:52:30.118685+0000","last_fullsized":"2023-10-06T03:52:30.118685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.785965+0000","last_change":"2023-10-06T03:52:30.785965+0000","last_active":"2023-10-06T03:52:30.785965+0000","last_peered":"2023-10-06T03:52:30.785965+0000","last_clean":"2023-10-06T03:52:30.785965+0000","last_became_active":"2023-10-06T03:52:30.785689+0000","last_became_peered":"2023-10-06T03:52:30.785689+0000","last_unstale":"2023-10-06T03:52:30.785965+0000","last_undegraded":"2023-10-06T03:52:30.785965+0000","last_fullsized":"2023-10-06T03:52:30.785965+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:36.796724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.776817+0000","last_change":"2023-10-06T03:52:30.120206+0000","last_active":"2023-10-06T03:52:30.776817+0000","last_peered":"2023-10-06T03:52:30.776817+0000","last_clean":"2023-10-06T03:52:30.776817+0000","last_became_active":"2023-10-06T03:52:30.119548+0000","last_became_peered":"2023-10-06T03:52:30.119548+0000","last_unstale":"2023-10-06T03:52:30.776817+0000","last_undegraded":"2023-10-06T03:52:30.776817+0000","last_fullsized":"2023-10-06T03:52:30.776817+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:43.648194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":114,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.117819+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:52:30.117819+0000","last_peered":"2023-10-06T03:52:30.117819+0000","last_clean":"2023-10-06T03:52:30.117819+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:52:30.117819+0000","last_undegraded":"2023-10-06T03:52:30.117819+0000","last_fullsized":"2023-10-06T03:52:30.117819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.796924+0000","last_change":"2023-10-06T03:52:30.796924+0000","last_active":"2023-10-06T03:52:30.796924+0000","last_peered":"2023-10-06T03:52:30.796924+0000","last_clean":"2023-10-06T03:52:30.796924+0000","last_became_active":"2023-10-06T03:52:30.796673+0000","last_became_peered":"2023-10-06T03:52:30.796673+0000","last_unstale":"2023-10-06T03:52:30.796924+0000","last_undegraded":"2023-10-06T03:52:30.796924+0000","last_fullsized":"2023-10-06T03:52:30.796924+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:03.223461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"136'8","reported_seq":18,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.142155+0000","last_change":"2023-10-06T03:52:30.124575+0000","last_active":"2023-10-06T03:52:31.142155+0000","last_peered":"2023-10-06T03:52:31.142155+0000","last_clean":"2023-10-06T03:52:31.142155+0000","last_became_active":"2023-10-06T03:52:30.124300+0000","last_became_peered":"2023-10-06T03:52:30.124300+0000","last_unstale":"2023-10-06T03:52:31.142155+0000","last_undegraded":"2023-10-06T03:52:31.142155+0000","last_fullsized":"2023-10-06T03:52:31.142155+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:53.224463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.118761+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:52:30.118761+0000","last_peered":"2023-10-06T03:52:30.118761+0000","last_clean":"2023-10-06T03:52:30.118761+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:52:30.118761+0000","last_undegraded":"2023-10-06T03:52:30.118761+0000","last_fullsized":"2023-10-06T03:52:30.118761+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.786351+0000","last_change":"2023-10-06T03:52:30.786351+0000","last_active":"2023-10-06T03:52:30.786351+0000","last_peered":"2023-10-06T03:52:30.786351+0000","last_clean":"2023-10-06T03:52:30.786351+0000","last_became_active":"2023-10-06T03:52:30.786101+0000","last_became_peered":"2023-10-06T03:52:30.786101+0000","last_unstale":"2023-10-06T03:52:30.786351+0000","last_undegraded":"2023-10-06T03:52:30.786351+0000","last_fullsized":"2023-10-06T03:52:30.786351+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:40.740425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.775271+0000","last_change":"2023-10-06T03:52:30.118651+0000","last_active":"2023-10-06T03:52:30.775271+0000","last_peered":"2023-10-06T03:52:30.775271+0000","last_clean":"2023-10-06T03:52:30.775271+0000","last_became_active":"2023-10-06T03:52:30.118385+0000","last_became_peered":"2023-10-06T03:52:30.118385+0000","last_unstale":"2023-10-06T03:52:30.775271+0000","last_undegraded":"2023-10-06T03:52:30.775271+0000","last_fullsized":"2023-10-06T03:52:30.775271+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:50.008770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.111356+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:52:30.111356+0000","last_peered":"2023-10-06T03:52:30.111356+0000","last_clean":"2023-10-06T03:52:30.111356+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:52:30.111356+0000","last_undegraded":"2023-10-06T03:52:30.111356+0000","last_fullsized":"2023-10-06T03:52:30.111356+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792384+0000","last_change":"2023-10-06T03:52:30.792384+0000","last_active":"2023-10-06T03:52:30.792384+0000","last_peered":"2023-10-06T03:52:30.792384+0000","last_clean":"2023-10-06T03:52:30.792384+0000","last_became_active":"2023-10-06T03:52:30.792112+0000","last_became_peered":"2023-10-06T03:52:30.792112+0000","last_unstale":"2023-10-06T03:52:30.792384+0000","last_undegraded":"2023-10-06T03:52:30.792384+0000","last_fullsized":"2023-10-06T03:52:30.792384+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:40.375611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.122758+0000","last_change":"2023-10-06T03:52:30.115911+0000","last_active":"2023-10-06T03:52:30.122758+0000","last_peered":"2023-10-06T03:52:30.122758+0000","last_clean":"2023-10-06T03:52:30.122758+0000","last_became_active":"2023-10-06T03:52:30.115634+0000","last_became_peered":"2023-10-06T03:52:30.115634+0000","last_unstale":"2023-10-06T03:52:30.122758+0000","last_undegraded":"2023-10-06T03:52:30.122758+0000","last_fullsized":"2023-10-06T03:52:30.122758+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:14.292118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.774608+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:52:30.774608+0000","last_peered":"2023-10-06T03:52:30.774608+0000","last_clean":"2023-10-06T03:52:30.774608+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:52:30.774608+0000","last_undegraded":"2023-10-06T03:52:30.774608+0000","last_fullsized":"2023-10-06T03:52:30.774608+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.793138+0000","last_change":"2023-10-06T03:52:30.788624+0000","last_active":"2023-10-06T03:52:30.793138+0000","last_peered":"2023-10-06T03:52:30.793138+0000","last_clean":"2023-10-06T03:52:30.793138+0000","last_became_active":"2023-10-06T03:52:30.788334+0000","last_became_peered":"2023-10-06T03:52:30.788334+0000","last_unstale":"2023-10-06T03:52:30.793138+0000","last_undegraded":"2023-10-06T03:52:30.793138+0000","last_fullsized":"2023-10-06T03:52:30.793138+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:06.430173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.774268+0000","last_change":"2023-10-06T03:52:30.125314+0000","last_active":"2023-10-06T03:52:30.774268+0000","last_peered":"2023-10-06T03:52:30.774268+0000","last_clean":"2023-10-06T03:52:30.774268+0000","last_became_active":"2023-10-06T03:52:30.125015+0000","last_became_peered":"2023-10-06T03:52:30.125015+0000","last_unstale":"2023-10-06T03:52:30.774268+0000","last_undegraded":"2023-10-06T03:52:30.774268+0000","last_fullsized":"2023-10-06T03:52:30.774268+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:33.395555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.774354+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:52:30.774354+0000","last_peered":"2023-10-06T03:52:30.774354+0000","last_clean":"2023-10-06T03:52:30.774354+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:52:30.774354+0000","last_undegraded":"2023-10-06T03:52:30.774354+0000","last_fullsized":"2023-10-06T03:52:30.774354+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575074,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311816,"kb_used_data":2240,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060024,"statfs":{"total":96636764160,"available":96317464576,"internally_reserved":0,"allocated":2293760,"data_stored":1462363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:51:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.69299999999999995},"min":{"1min":0.41899999999999998,"5min":0.377,"15min":0.27100000000000002},"max":{"1min":0.89200000000000002,"5min":0.998,"15min":1.0409999999999999},"last":0.63},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.45000000000000001,"5min":0.42599999999999999,"15min":0.36499999999999999},"max":{"1min":0.91700000000000004,"5min":1.02,"15min":1.02},"last":0.71499999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:51:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70699999999999996,"15min":0.74299999999999999},"min":{"1min":0.46899999999999997,"5min":0.41199999999999998,"15min":0.318},"max":{"1min":0.96299999999999997,"5min":1.038,"15min":1.0640000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73699999999999999,"15min":0.76200000000000001},"min":{"1min":0.57199999999999995,"5min":0.49199999999999999,"15min":0.45900000000000002},"max":{"1min":0.90200000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:51:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67800000000000005,"15min":0.69799999999999995},"min":{"1min":0.497,"5min":0.41199999999999998,"15min":0.39100000000000001},"max":{"1min":0.90500000000000003,"5min":0.97499999999999998,"15min":1.044},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71299999999999997,"15min":0.73699999999999999},"min":{"1min":0.47199999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.92700000000000005,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:51:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70699999999999996,"15min":0.72299999999999998},"min":{"1min":0.53300000000000003,"5min":0.41799999999999998,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":1.0089999999999999,"15min":1.016},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.74099999999999999},"min":{"1min":0.55000000000000004,"5min":0.501,"15min":0.45600000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":1.0409999999999999},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:51:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61099999999999999,"15min":0.59299999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63600000000000001,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.39400000000000002},"max":{"1min":0.93500000000000005,"5min":0.94699999999999995,"15min":0.97599999999999998},"last":0.60099999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:51:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.65800000000000003,"15min":0.63800000000000001},"min":{"1min":0.42699999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82299999999999995,"5min":1.022,"15min":1.069},"last":0.439},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68899999999999995,"15min":0.68000000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95699999999999996,"5min":1.0640000000000001,"15min":1.105},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:51:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.74199999999999999,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.436},"max":{"1min":1.0069999999999999,"5min":1.1220000000000001,"15min":1.181},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.47999999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.083,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575075,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307132,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064708,"statfs":{"total":96636764160,"available":96322260992,"internally_reserved":0,"allocated":1691648,"data_stored":872779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69199999999999995,"15min":0.70299999999999996},"min":{"1min":0.442,"5min":0.39400000000000002,"15min":0.313},"max":{"1min":0.93100000000000005,"5min":1.288,"15min":1.288},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93600000000000005,"5min":0.998,"15min":1.0069999999999999},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71199999999999997,"15min":0.72499999999999998},"min":{"1min":0.50700000000000001,"5min":0.41599999999999998,"15min":0.33800000000000002},"max":{"1min":0.88200000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.69899999999999995,"15min":0.71299999999999997},"min":{"1min":0.48499999999999999,"5min":0.46400000000000002,"15min":0.35899999999999999},"max":{"1min":0.84599999999999997,"5min":1.427,"15min":1.427},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71999999999999997,"15min":0.73699999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.34699999999999998},"max":{"1min":0.91100000000000003,"5min":1.179,"15min":1.179},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.71099999999999997,"15min":0.73299999999999998},"min":{"1min":0.505,"5min":0.42099999999999999,"15min":0.39600000000000002},"max":{"1min":0.91100000000000003,"5min":1.0209999999999999,"15min":1.0409999999999999},"last":0.55900000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72899999999999998,"15min":0.748},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":0.95399999999999996,"5min":1.024,"15min":1.024},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.73999999999999999,"15min":0.755},"min":{"1min":0.53000000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85999999999999999,"5min":1.097,"15min":1.1639999999999999},"last":0.51200000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":1.462,"15min":1.462},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64200000000000002,"15min":0.63700000000000001},"min":{"1min":0.41699999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95199999999999996,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.46700000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70699999999999996,"15min":0.67400000000000004},"min":{"1min":0.47899999999999998,"5min":0.438,"15min":0.40000000000000002},"max":{"1min":0.91600000000000004,"5min":1.0109999999999999,"15min":1.0489999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73599999999999999,"15min":0.71699999999999997},"min":{"1min":0.52000000000000002,"5min":0.42799999999999999,"15min":0.41199999999999998},"max":{"1min":0.93799999999999994,"5min":1.508,"15min":1.508},"last":0.64600000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73099999999999998,"15min":0.69699999999999995},"min":{"1min":0.46100000000000002,"5min":0.41499999999999998,"15min":0.41199999999999998},"max":{"1min":0.96299999999999997,"5min":1.21,"15min":1.21},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.71199999999999997},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.36099999999999999},"max":{"1min":0.92600000000000005,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.628}]}]},{"osd":1,"up_from":13,"seq":55834575074,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307708,"kb_used_data":2228,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064132,"statfs":{"total":96636764160,"available":96321671168,"internally_reserved":0,"allocated":2281472,"data_stored":1462593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:51:46 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.34899999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.73399999999999999,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.45400000000000001,"5min":0.36399999999999999,"15min":0.34599999999999997},"max":{"1min":0.70999999999999996,"5min":1.024,"15min":1.1319999999999999},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:51:46 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45900000000000002,"15min":0.48799999999999999},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.309},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.84799999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.40600000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78300000000000003,"5min":0.80500000000000005,"15min":0.94099999999999995},"last":0.40100000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:51:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.45300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.83099999999999996,"5min":0.86599999999999999,"15min":0.91700000000000004},"last":0.498},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.34200000000000003,"15min":0.32500000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.90400000000000003},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:51:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75,"15min":0.76000000000000001},"min":{"1min":0.56200000000000006,"5min":0.496,"15min":0.41499999999999998},"max":{"1min":1,"5min":1.044,"15min":1.1699999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.79700000000000004},"min":{"1min":0.55800000000000005,"5min":0.52500000000000002,"15min":0.439},"max":{"1min":1,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.94399999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:51:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.45400000000000001},"max":{"1min":0.95899999999999996,"5min":1.294,"15min":1.294},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.45100000000000001},"max":{"1min":1.0389999999999999,"5min":1.224,"15min":1.224},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:51:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.58999999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98599999999999999,"5min":1.26,"15min":1.26},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78800000000000003,"15min":0.81599999999999995},"min":{"1min":0.60599999999999998,"5min":0.52800000000000002,"15min":0.52100000000000002},"max":{"1min":1.0660000000000001,"5min":1.143,"15min":1.224},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:51:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.53900000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0369999999999999,"5min":1.1519999999999999,"15min":1.272},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.82499999999999996,"15min":0.83199999999999996},"min":{"1min":0.59299999999999997,"5min":0.52000000000000002,"15min":0.51200000000000001},"max":{"1min":1.0680000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.92900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575076,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307128,"kb_used_data":1648,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064712,"statfs":{"total":96636764160,"available":96322265088,"internally_reserved":0,"allocated":1687552,"data_stored":871269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47699999999999998,"15min":0.48299999999999998},"min":{"1min":0.30199999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70499999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.50700000000000001,"15min":0.51600000000000001},"min":{"1min":0.35099999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.69999999999999996,"5min":0.70999999999999996,"15min":0.83599999999999997},"last":0.40999999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52800000000000002,"15min":0.54100000000000004},"min":{"1min":0.38700000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73599999999999999,"5min":0.80100000000000005,"15min":1.1299999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53600000000000003,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.31900000000000001},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":0.89400000000000002},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52200000000000002,"15min":0.51900000000000002},"min":{"1min":0.33200000000000002,"5min":0.318,"15min":0.30499999999999999},"max":{"1min":0.80000000000000004,"5min":0.83699999999999997,"15min":0.93200000000000005},"last":0.432},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58899999999999997,"15min":0.60999999999999999},"min":{"1min":0.46600000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.78700000000000003,"5min":0.91800000000000004,"15min":1.0580000000000001},"last":0.497}]},{"osd":4,"last update":"Fri Oct 6 03:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80700000000000005,"15min":0.82499999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.49399999999999999},"max":{"1min":1.109,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.80300000000000005,"15min":0.81899999999999995},"min":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.45500000000000002},"max":{"1min":1.079,"5min":1.306,"15min":1.3400000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":0.81100000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.042,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78500000000000003,"15min":0.80400000000000005},"min":{"1min":0.54100000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0629999999999999,"5min":1.194,"15min":1.6559999999999999},"last":0.80500000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77900000000000003,"15min":0.80300000000000005},"min":{"1min":0.625,"5min":0.51500000000000001,"15min":0.48099999999999998},"max":{"1min":0.97299999999999998,"5min":1.1699999999999999,"15min":1.726},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81299999999999994,"15min":0.83699999999999997},"min":{"1min":0.57299999999999995,"5min":0.52900000000000003,"15min":0.52400000000000002},"max":{"1min":1.2010000000000001,"5min":1.212,"15min":1.843},"last":0.86299999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.81699999999999995,"15min":0.82399999999999995},"min":{"1min":0.61299999999999999,"5min":0.502,"15min":0.496},"max":{"1min":1.1799999999999999,"5min":1.302,"15min":1.302},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.495},"max":{"1min":1.004,"5min":1.2450000000000001,"15min":1.7649999999999999},"last":0.91100000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575075,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311220,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060620,"statfs":{"total":96636764160,"available":96318074880,"internally_reserved":0,"allocated":1683456,"data_stored":871247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45300000000000001,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54400000000000004,"5min":0.74199999999999999,"15min":0.80100000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.52200000000000002,"15min":0.53100000000000003},"min":{"1min":0.34100000000000003,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":0.72099999999999997,"5min":0.86699999999999999,"15min":0.89300000000000002},"last":0.51100000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48399999999999999,"15min":0.48899999999999999},"min":{"1min":0.311,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72799999999999998,"5min":0.81200000000000006,"15min":0.90000000000000002},"last":0.495},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54700000000000004,"15min":0.55400000000000005},"min":{"1min":0.42099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71399999999999997,"5min":0.89700000000000002,"15min":0.94899999999999995},"last":0.57499999999999996}]},{"osd":3,"last update":"Fri Oct 6 03:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55700000000000005,"15min":0.58899999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69799999999999995,"5min":0.79000000000000004,"15min":1.78},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.56000000000000005,"15min":0.57399999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.33300000000000002},"max":{"1min":0.65800000000000003,"5min":0.84699999999999998,"15min":0.96499999999999997},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.5620000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.82999999999999996,"15min":0.83699999999999997},"min":{"1min":0.62,"5min":0.53100000000000003,"15min":0.46500000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.83499999999999996,"15min":0.83799999999999997},"min":{"1min":0.623,"5min":0.54900000000000004,"15min":0.503},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81200000000000006,"15min":0.84599999999999997},"min":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.855},"last":0.82199999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.78800000000000003,"15min":0.82699999999999996},"min":{"1min":0.54800000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.907},"last":0.874},{"interface":"front","average":{"1min":0.745,"5min":0.80900000000000005,"15min":0.82199999999999995},"min":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0609999999999999,"5min":1.1080000000000001,"15min":1.4750000000000001},"last":0.80400000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.82299999999999995,"15min":0.84399999999999997},"min":{"1min":0.56799999999999995,"5min":0.54700000000000004,"15min":0.495},"max":{"1min":1.1559999999999999,"5min":1.2350000000000001,"15min":1.4990000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81999999999999995,"15min":0.86299999999999999},"min":{"1min":0.57399999999999995,"5min":0.54800000000000004,"15min":0.53100000000000003},"max":{"1min":1.194,"5min":1.194,"15min":1.9450000000000001},"last":0.747}]}]},{"osd":3,"up_from":13,"seq":55834575075,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307156,"kb_used_data":1676,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064684,"statfs":{"total":96636764160,"available":96322236416,"internally_reserved":0,"allocated":1716224,"data_stored":880828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27000000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.495},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29799999999999999},"max":{"1min":0.71599999999999997,"5min":0.79200000000000004,"15min":1.04},"last":0.33900000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.314},"max":{"1min":0.79200000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70299999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69299999999999995,"5min":0.80700000000000005,"15min":1.4199999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.59899999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.87},"last":0.46100000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.77100000000000002,"15min":0.77900000000000003},"min":{"1min":0.627,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.3340000000000001},"last":0.626},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.79900000000000004,"15min":0.82199999999999995},"min":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.2889999999999999},"last":0.70499999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.53300000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.256},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76600000000000001,"15min":0.77900000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.49099999999999999},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.2030000000000001},"last":0.66400000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.3029999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.81000000000000005,"15min":0.83999999999999997},"min":{"1min":0.61499999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0429999999999999,"5min":1.075,"15min":1.2849999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78200000000000003,"15min":0.80600000000000005},"min":{"1min":0.53800000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.222,"5min":1.222,"15min":1.2370000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.77700000000000002,"15min":0.79900000000000004},"min":{"1min":0.56599999999999995,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.2330000000000001},"last":0.745}]}]},{"osd":4,"up_from":13,"seq":55834575075,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307140,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064700,"statfs":{"total":96636764160,"available":96322252800,"internally_reserved":0,"allocated":1699840,"data_stored":873761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70199999999999996,"15min":0.71499999999999997},"min":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.316},"max":{"1min":0.98499999999999999,"5min":0.996,"15min":1.0189999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68000000000000005,"15min":0.69899999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.316},"max":{"1min":0.91200000000000003,"5min":0.96499999999999997,"15min":1.01},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Oct 6 03:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.71999999999999997},"min":{"1min":0.48599999999999999,"5min":0.42999999999999999,"15min":0.33200000000000002},"max":{"1min":0.92300000000000004,"5min":1.0600000000000001,"15min":1.081},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73499999999999999,"15min":0.73699999999999999},"min":{"1min":0.45500000000000002,"5min":0.42799999999999999,"15min":0.41299999999999998},"max":{"1min":1.02,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.72099999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71799999999999997,"15min":0.73199999999999998},"min":{"1min":0.54100000000000004,"5min":0.47599999999999998,"15min":0.377},"max":{"1min":1.004,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73699999999999999,"15min":0.746},"min":{"1min":0.46999999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94599999999999995,"5min":1.0169999999999999,"15min":1.022},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90600000000000003,"5min":1.0189999999999999,"15min":1.0409999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.46500000000000002},"max":{"1min":0.96799999999999997,"5min":1.1040000000000001,"15min":1.1850000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63600000000000001,"15min":0.625},"min":{"1min":0.40899999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.048,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.434,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85899999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.504}]},{"osd":6,"last update":"Fri Oct 6 03:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.64500000000000002},"min":{"1min":0.45800000000000002,"5min":0.434,"15min":0.42599999999999999},"max":{"1min":0.85399999999999998,"5min":1.1839999999999999,"15min":1.2310000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69899999999999995,"15min":0.68500000000000005},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.38100000000000001},"max":{"1min":0.94399999999999995,"5min":1.083,"15min":1.083},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68700000000000006,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90800000000000003,"5min":1.046,"15min":1.046},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.56699999999999995,"5min":0.47499999999999998,"15min":0.41299999999999998},"max":{"1min":0.92900000000000005,"5min":1.214,"15min":1.2889999999999999},"last":0.78500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575075,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303036,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068804,"statfs":{"total":96636764160,"available":96326455296,"internally_reserved":0,"allocated":1691648,"data_stored":871407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69799999999999995,"15min":0.70599999999999996},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.35299999999999998},"max":{"1min":0.94699999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.71199999999999997,"15min":0.71599999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.32600000000000001},"max":{"1min":0.872,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.80400000000000005}]},{"osd":1,"last update":"Fri Oct 6 03:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.77000000000000002,"15min":0.73099999999999998},"min":{"1min":0.50800000000000001,"5min":0.46600000000000003,"15min":0.40000000000000002},"max":{"1min":0.875,"5min":5.8239999999999998,"15min":5.8239999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.76800000000000002,"15min":0.73899999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.32400000000000001},"max":{"1min":0.90700000000000003,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.81699999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.49099999999999999,"5min":0.46400000000000002,"15min":0.40200000000000002},"max":{"1min":0.97899999999999998,"5min":1.155,"15min":1.155},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76000000000000001,"15min":0.77100000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.89500000000000002,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.875}]},{"osd":3,"last update":"Fri Oct 6 03:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.84499999999999997,"5min":1.091,"15min":1.6419999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.749,"15min":0.748},"min":{"1min":0.57999999999999996,"5min":0.50600000000000001,"15min":0.47799999999999998},"max":{"1min":0.92800000000000005,"5min":1.238,"15min":1.238},"last":0.83399999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65100000000000002,"15min":0.61699999999999999},"min":{"1min":0.372,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88200000000000001,"5min":1.04,"15min":1.04},"last":0.498},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":0.90200000000000002,"5min":1.147,"15min":1.147},"last":0.45900000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.65800000000000003,"15min":0.64200000000000002},"min":{"1min":0.47199999999999998,"5min":0.40200000000000002,"15min":0.34999999999999998},"max":{"1min":0.91100000000000003,"5min":1.0449999999999999,"15min":1.591},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71099999999999997,"15min":0.68899999999999995},"min":{"1min":0.45000000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97199999999999998,"5min":1.196,"15min":1.6140000000000001},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.46100000000000002,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.83999999999999997,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72299999999999998,"15min":0.71199999999999997},"min":{"1min":0.47099999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95399999999999996,"5min":1.0880000000000001,"15min":1.278},"last":0.48699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:52:35.646 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T03:52:35.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.833+0000 7fbc6359e700 1 Processor -- start 2023-10-06T03:52:35.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.833+0000 7fbc6359e700 1 -- start start 2023-10-06T03:52:35.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.834+0000 7fbc6359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbc641566e0 0x7fbc640ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:35.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.834+0000 7fbc6359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc640ca860 con 0x7fbc640cb900 2023-10-06T03:52:35.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.835+0000 7fbc6359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc640cab40 con 0x7fbc64156310 2023-10-06T03:52:35.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.835+0000 7fbc6359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc640cae20 con 0x7fbc641566e0 2023-10-06T03:52:35.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.835+0000 7fbc61d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbc641566e0 0x7fbc640ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:35.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.835+0000 7fbc62d9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbc640cb900 0x7fbc640c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35238/0 (socket says 172.21.15.55:35238) 2023-10-06T03:52:35.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.835+0000 7fbc62d9d700 1 -- 172.21.15.55:0/998196200 learned_addr learned my addr 172.21.15.55:0/998196200 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:35.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.835+0000 7fbc61d9b700 1 -- 172.21.15.55:0/998196200 >> v1:172.21.15.163:6789/0 conn(0x7fbc64156310 legacy=0x7fbc640c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:35.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.835+0000 7fbc61d9b700 1 -- 172.21.15.55:0/998196200 >> v1:172.21.15.55:6789/0 conn(0x7fbc640cb900 legacy=0x7fbc640c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:35.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.835+0000 7fbc61d9b700 1 -- 172.21.15.55:0/998196200 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc640cb100 con 0x7fbc641566e0 2023-10-06T03:52:35.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.835+0000 7fbc6159a700 1 -- 172.21.15.55:0/998196200 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2121036114 0 0) 0x7fbc640ca860 con 0x7fbc640cb900 2023-10-06T03:52:35.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.836+0000 7fbc61d9b700 1 --2- 172.21.15.55:0/998196200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc641566e0 0x7fbc640ae000 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fbc580098e0 tx=0x7fbc580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93081093a10a8ee server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:35.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.836+0000 7fbc6159a700 1 -- 172.21.15.55:0/998196200 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58009e80 con 0x7fbc641566e0 2023-10-06T03:52:35.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.836+0000 7fbc6159a700 1 -- 172.21.15.55:0/998196200 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5800b820 con 0x7fbc641566e0 2023-10-06T03:52:35.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.836+0000 7fbc6159a700 1 -- 172.21.15.55:0/998196200 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5800bba0 con 0x7fbc641566e0 2023-10-06T03:52:35.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.836+0000 7fbc6359e700 1 -- 172.21.15.55:0/998196200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc641566e0 msgr2=0x7fbc640ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:35.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.836+0000 7fbc6359e700 1 --2- 172.21.15.55:0/998196200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc641566e0 0x7fbc640ae000 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fbc580098e0 tx=0x7fbc580091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.836+0000 7fbc6359e700 1 -- 172.21.15.55:0/998196200 shutdown_connections 2023-10-06T03:52:35.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.836+0000 7fbc6359e700 1 --2- 172.21.15.55:0/998196200 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc641566e0 0x7fbc640ae000 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.836+0000 7fbc6359e700 1 -- 172.21.15.55:0/998196200 >> 172.21.15.55:0/998196200 conn(0x7fbc640cc1f0 msgr2=0x7fbc640cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:35.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.836+0000 7fbc6359e700 1 -- 172.21.15.55:0/998196200 shutdown_connections 2023-10-06T03:52:35.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.837+0000 7fbc6359e700 1 -- 172.21.15.55:0/998196200 wait complete. 2023-10-06T03:52:35.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.838+0000 7fbc6359e700 1 Processor -- start 2023-10-06T03:52:35.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.838+0000 7fbc6359e700 1 -- start start 2023-10-06T03:52:35.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.839+0000 7fbc6359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc640cb900 0x7fbc6416aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:35.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.839+0000 7fbc6359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc64156310 0x7fbc6415c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:35.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.839+0000 7fbc6359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc641566e0 0x7fbc6414f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:35.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.839+0000 7fbc6359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc64155420 con 0x7fbc640cb900 2023-10-06T03:52:35.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.839+0000 7fbc6359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6415cb90 con 0x7fbc641566e0 2023-10-06T03:52:35.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.839+0000 7fbc6359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6414e350 con 0x7fbc64156310 2023-10-06T03:52:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.839+0000 7fbc6259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc640cb900 0x7fbc6416aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.839+0000 7fbc6259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc640cb900 0x7fbc6416aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49906/0 (socket says 172.21.15.55:49906) 2023-10-06T03:52:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.839+0000 7fbc6259c700 1 -- 172.21.15.55:0/1677669840 learned_addr learned my addr 172.21.15.55:0/1677669840 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.840+0000 7fbc6259c700 1 -- 172.21.15.55:0/1677669840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc64156310 msgr2=0x7fbc6415c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.840+0000 7fbc6259c700 1 --2- 172.21.15.55:0/1677669840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc64156310 0x7fbc6415c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.840+0000 7fbc6259c700 1 -- 172.21.15.55:0/1677669840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc641566e0 msgr2=0x7fbc6414f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.840+0000 7fbc6259c700 1 --2- 172.21.15.55:0/1677669840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc641566e0 0x7fbc6414f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.840+0000 7fbc6259c700 1 -- 172.21.15.55:0/1677669840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc58009420 con 0x7fbc640cb900 2023-10-06T03:52:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.840+0000 7fbc6259c700 1 --2- 172.21.15.55:0/1677669840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc640cb900 0x7fbc6416aa70 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400b630 tx=0x7fbc540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.840+0000 7fbc527fc700 1 -- 172.21.15.55:0/1677669840 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5400f510 con 0x7fbc640cb900 2023-10-06T03:52:35.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.840+0000 7fbc527fc700 1 -- 172.21.15.55:0/1677669840 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5400f690 con 0x7fbc640cb900 2023-10-06T03:52:35.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.840+0000 7fbc527fc700 1 -- 172.21.15.55:0/1677669840 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5400fa50 con 0x7fbc640cb900 2023-10-06T03:52:35.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.840+0000 7fbc6359e700 1 -- 172.21.15.55:0/1677669840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6415c740 con 0x7fbc640cb900 2023-10-06T03:52:35.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.840+0000 7fbc6359e700 1 -- 172.21.15.55:0/1677669840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6415d0c0 con 0x7fbc640cb900 2023-10-06T03:52:35.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.841+0000 7fbc6359e700 1 -- 172.21.15.55:0/1677669840 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6415d2e0 con 0x7fbc640cb900 2023-10-06T03:52:35.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.842+0000 7fbc527fc700 1 -- 172.21.15.55:0/1677669840 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fbc5401a400 con 0x7fbc640cb900 2023-10-06T03:52:35.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.843+0000 7fbc527fc700 1 --2- 172.21.15.55:0/1677669840 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbc44067050 0x7fbc44069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:35.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.844+0000 7fbc527fc700 1 -- 172.21.15.55:0/1677669840 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fbc54084530 con 0x7fbc640cb900 2023-10-06T03:52:35.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.844+0000 7fbc61d9b700 1 --2- 172.21.15.55:0/1677669840 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbc44067050 0x7fbc44069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:35.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.845+0000 7fbc61d9b700 1 --2- 172.21.15.55:0/1677669840 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbc44067050 0x7fbc44069500 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbc580098e0 tx=0x7fbc58016000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:35.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:35.848+0000 7fbc527fc700 1 -- 172.21.15.55:0/1677669840 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc54051380 con 0x7fbc640cb900 2023-10-06T03:52:36.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.044+0000 7fbc6359e700 1 -- 172.21.15.55:0/1677669840 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbc6414e140 con 0x7fbc44067050 2023-10-06T03:52:36.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.054+0000 7fbc527fc700 1 -- 172.21.15.55:0/1677669840 <== mgr.7690 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311999 (secure 0 0 0) 0x7fbc5800ba20 con 0x7fbc44067050 2023-10-06T03:52:36.059 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:36.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.058+0000 7fbc6359e700 1 -- 172.21.15.55:0/1677669840 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbc44067050 msgr2=0x7fbc44069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:36.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.058+0000 7fbc6359e700 1 --2- 172.21.15.55:0/1677669840 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbc44067050 0x7fbc44069500 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbc580098e0 tx=0x7fbc58016000 comp rx=0 tx=0).stop 2023-10-06T03:52:36.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.058+0000 7fbc6359e700 1 -- 172.21.15.55:0/1677669840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc640cb900 msgr2=0x7fbc6416aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:36.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.058+0000 7fbc6359e700 1 --2- 172.21.15.55:0/1677669840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc640cb900 0x7fbc6416aa70 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400b630 tx=0x7fbc540091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:36.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.058+0000 7fbc6359e700 1 -- 172.21.15.55:0/1677669840 shutdown_connections 2023-10-06T03:52:36.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.058+0000 7fbc6359e700 1 --2- 172.21.15.55:0/1677669840 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fbc44067050 0x7fbc44069500 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:36.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.058+0000 7fbc6359e700 1 --2- 172.21.15.55:0/1677669840 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbc641566e0 0x7fbc6414f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:36.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.058+0000 7fbc6359e700 1 --2- 172.21.15.55:0/1677669840 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbc640cb900 0x7fbc6416aa70 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:36.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.058+0000 7fbc6359e700 1 --2- 172.21.15.55:0/1677669840 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbc64156310 0x7fbc6415c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:36.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.058+0000 7fbc6359e700 1 -- 172.21.15.55:0/1677669840 >> 172.21.15.55:0/1677669840 conn(0x7fbc640cc1f0 msgr2=0x7fbc64151c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:36.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.058+0000 7fbc6359e700 1 -- 172.21.15.55:0/1677669840 shutdown_connections 2023-10-06T03:52:36.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.058+0000 7fbc6359e700 1 -- 172.21.15.55:0/1677669840 wait complete. 2023-10-06T03:52:36.063 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T03:52:36.077 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-10-06T03:52:34.933978+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":443,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2462336,"kb_used_data":14400,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752512384,"statfs":{"total":773094113280,"available":770572681216,"internally_reserved":0,"allocated":14745600,"data_stored":8166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.833090"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.785996+0000","last_change":"2023-10-06T03:52:30.780956+0000","last_active":"2023-10-06T03:52:30.785996+0000","last_peered":"2023-10-06T03:52:30.785996+0000","last_clean":"2023-10-06T03:52:30.785996+0000","last_became_active":"2023-10-06T03:52:30.780670+0000","last_became_peered":"2023-10-06T03:52:30.780670+0000","last_unstale":"2023-10-06T03:52:30.785996+0000","last_undegraded":"2023-10-06T03:52:30.785996+0000","last_fullsized":"2023-10-06T03:52:30.785996+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:07.137633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792648+0000","last_change":"2023-10-06T03:52:30.789178+0000","last_active":"2023-10-06T03:52:30.792648+0000","last_peered":"2023-10-06T03:52:30.792648+0000","last_clean":"2023-10-06T03:52:30.792648+0000","last_became_active":"2023-10-06T03:52:30.788531+0000","last_became_peered":"2023-10-06T03:52:30.788531+0000","last_unstale":"2023-10-06T03:52:30.792648+0000","last_undegraded":"2023-10-06T03:52:30.792648+0000","last_fullsized":"2023-10-06T03:52:30.792648+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:33:20.203153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792700+0000","last_change":"2023-10-06T03:52:30.789224+0000","last_active":"2023-10-06T03:52:30.792700+0000","last_peered":"2023-10-06T03:52:30.792700+0000","last_clean":"2023-10-06T03:52:30.792700+0000","last_became_active":"2023-10-06T03:52:30.788944+0000","last_became_peered":"2023-10-06T03:52:30.788944+0000","last_unstale":"2023-10-06T03:52:30.792700+0000","last_undegraded":"2023-10-06T03:52:30.792700+0000","last_fullsized":"2023-10-06T03:52:30.792700+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:16:53.099855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.793655+0000","last_change":"2023-10-06T03:52:30.793655+0000","last_active":"2023-10-06T03:52:30.793655+0000","last_peered":"2023-10-06T03:52:30.793655+0000","last_clean":"2023-10-06T03:52:30.793655+0000","last_became_active":"2023-10-06T03:52:30.793255+0000","last_became_peered":"2023-10-06T03:52:30.793255+0000","last_unstale":"2023-10-06T03:52:30.793655+0000","last_undegraded":"2023-10-06T03:52:30.793655+0000","last_fullsized":"2023-10-06T03:52:30.793655+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:27.485634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.784766+0000","last_change":"2023-10-06T03:52:30.779116+0000","last_active":"2023-10-06T03:52:30.784766+0000","last_peered":"2023-10-06T03:52:30.784766+0000","last_clean":"2023-10-06T03:52:30.784766+0000","last_became_active":"2023-10-06T03:52:30.777363+0000","last_became_peered":"2023-10-06T03:52:30.777363+0000","last_unstale":"2023-10-06T03:52:30.784766+0000","last_undegraded":"2023-10-06T03:52:30.784766+0000","last_fullsized":"2023-10-06T03:52:30.784766+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:28.909246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792579+0000","last_change":"2023-10-06T03:52:30.792579+0000","last_active":"2023-10-06T03:52:30.792579+0000","last_peered":"2023-10-06T03:52:30.792579+0000","last_clean":"2023-10-06T03:52:30.792579+0000","last_became_active":"2023-10-06T03:52:30.791930+0000","last_became_peered":"2023-10-06T03:52:30.791930+0000","last_unstale":"2023-10-06T03:52:30.792579+0000","last_undegraded":"2023-10-06T03:52:30.792579+0000","last_fullsized":"2023-10-06T03:52:30.792579+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:08:20.731175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.788109+0000","last_change":"2023-10-06T03:52:30.788109+0000","last_active":"2023-10-06T03:52:30.788109+0000","last_peered":"2023-10-06T03:52:30.788109+0000","last_clean":"2023-10-06T03:52:30.788109+0000","last_became_active":"2023-10-06T03:52:30.787543+0000","last_became_peered":"2023-10-06T03:52:30.787543+0000","last_unstale":"2023-10-06T03:52:30.788109+0000","last_undegraded":"2023-10-06T03:52:30.788109+0000","last_fullsized":"2023-10-06T03:52:30.788109+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:41:30.779261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791534+0000","last_change":"2023-10-06T03:52:30.785750+0000","last_active":"2023-10-06T03:52:30.791534+0000","last_peered":"2023-10-06T03:52:30.791534+0000","last_clean":"2023-10-06T03:52:30.791534+0000","last_became_active":"2023-10-06T03:52:30.785510+0000","last_became_peered":"2023-10-06T03:52:30.785510+0000","last_unstale":"2023-10-06T03:52:30.791534+0000","last_undegraded":"2023-10-06T03:52:30.791534+0000","last_fullsized":"2023-10-06T03:52:30.791534+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:38.497087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.798553+0000","last_change":"2023-10-06T03:52:30.798553+0000","last_active":"2023-10-06T03:52:30.798553+0000","last_peered":"2023-10-06T03:52:30.798553+0000","last_clean":"2023-10-06T03:52:30.798553+0000","last_became_active":"2023-10-06T03:52:30.798266+0000","last_became_peered":"2023-10-06T03:52:30.798266+0000","last_unstale":"2023-10-06T03:52:30.798553+0000","last_undegraded":"2023-10-06T03:52:30.798553+0000","last_fullsized":"2023-10-06T03:52:30.798553+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:15.592998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.781489+0000","last_change":"2023-10-06T03:52:30.781489+0000","last_active":"2023-10-06T03:52:30.781489+0000","last_peered":"2023-10-06T03:52:30.781489+0000","last_clean":"2023-10-06T03:52:30.781489+0000","last_became_active":"2023-10-06T03:52:30.781246+0000","last_became_peered":"2023-10-06T03:52:30.781246+0000","last_unstale":"2023-10-06T03:52:30.781489+0000","last_undegraded":"2023-10-06T03:52:30.781489+0000","last_fullsized":"2023-10-06T03:52:30.781489+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:18.017097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.788252+0000","last_change":"2023-10-06T03:52:30.786019+0000","last_active":"2023-10-06T03:52:30.788252+0000","last_peered":"2023-10-06T03:52:30.788252+0000","last_clean":"2023-10-06T03:52:30.788252+0000","last_became_active":"2023-10-06T03:52:30.785686+0000","last_became_peered":"2023-10-06T03:52:30.785686+0000","last_unstale":"2023-10-06T03:52:30.788252+0000","last_undegraded":"2023-10-06T03:52:30.788252+0000","last_fullsized":"2023-10-06T03:52:30.788252+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:31:28.545878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.786728+0000","last_change":"2023-10-06T03:52:30.786728+0000","last_active":"2023-10-06T03:52:30.786728+0000","last_peered":"2023-10-06T03:52:30.786728+0000","last_clean":"2023-10-06T03:52:30.786728+0000","last_became_active":"2023-10-06T03:52:30.785935+0000","last_became_peered":"2023-10-06T03:52:30.785935+0000","last_unstale":"2023-10-06T03:52:30.786728+0000","last_undegraded":"2023-10-06T03:52:30.786728+0000","last_fullsized":"2023-10-06T03:52:30.786728+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:11:29.816312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.785174+0000","last_change":"2023-10-06T03:52:30.785174+0000","last_active":"2023-10-06T03:52:30.785174+0000","last_peered":"2023-10-06T03:52:30.785174+0000","last_clean":"2023-10-06T03:52:30.785174+0000","last_became_active":"2023-10-06T03:52:30.784914+0000","last_became_peered":"2023-10-06T03:52:30.784914+0000","last_unstale":"2023-10-06T03:52:30.785174+0000","last_undegraded":"2023-10-06T03:52:30.785174+0000","last_fullsized":"2023-10-06T03:52:30.785174+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:05.986856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792881+0000","last_change":"2023-10-06T03:52:30.789284+0000","last_active":"2023-10-06T03:52:30.792881+0000","last_peered":"2023-10-06T03:52:30.792881+0000","last_clean":"2023-10-06T03:52:30.792881+0000","last_became_active":"2023-10-06T03:52:30.788703+0000","last_became_peered":"2023-10-06T03:52:30.788703+0000","last_unstale":"2023-10-06T03:52:30.792881+0000","last_undegraded":"2023-10-06T03:52:30.792881+0000","last_fullsized":"2023-10-06T03:52:30.792881+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:27.224627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.787993+0000","last_change":"2023-10-06T03:52:30.786907+0000","last_active":"2023-10-06T03:52:30.787993+0000","last_peered":"2023-10-06T03:52:30.787993+0000","last_clean":"2023-10-06T03:52:30.787993+0000","last_became_active":"2023-10-06T03:52:30.786654+0000","last_became_peered":"2023-10-06T03:52:30.786654+0000","last_unstale":"2023-10-06T03:52:30.787993+0000","last_undegraded":"2023-10-06T03:52:30.787993+0000","last_fullsized":"2023-10-06T03:52:30.787993+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:21.845674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.782101+0000","last_change":"2023-10-06T03:52:30.782101+0000","last_active":"2023-10-06T03:52:30.782101+0000","last_peered":"2023-10-06T03:52:30.782101+0000","last_clean":"2023-10-06T03:52:30.782101+0000","last_became_active":"2023-10-06T03:52:30.780786+0000","last_became_peered":"2023-10-06T03:52:30.780786+0000","last_unstale":"2023-10-06T03:52:30.782101+0000","last_undegraded":"2023-10-06T03:52:30.782101+0000","last_fullsized":"2023-10-06T03:52:30.782101+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:39:12.142020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.790989+0000","last_change":"2023-10-06T03:52:30.790989+0000","last_active":"2023-10-06T03:52:30.790989+0000","last_peered":"2023-10-06T03:52:30.790989+0000","last_clean":"2023-10-06T03:52:30.790989+0000","last_became_active":"2023-10-06T03:52:30.789650+0000","last_became_peered":"2023-10-06T03:52:30.789650+0000","last_unstale":"2023-10-06T03:52:30.790989+0000","last_undegraded":"2023-10-06T03:52:30.790989+0000","last_fullsized":"2023-10-06T03:52:30.790989+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:51.019536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792686+0000","last_change":"2023-10-06T03:52:30.789977+0000","last_active":"2023-10-06T03:52:30.792686+0000","last_peered":"2023-10-06T03:52:30.792686+0000","last_clean":"2023-10-06T03:52:30.792686+0000","last_became_active":"2023-10-06T03:52:30.789689+0000","last_became_peered":"2023-10-06T03:52:30.789689+0000","last_unstale":"2023-10-06T03:52:30.792686+0000","last_undegraded":"2023-10-06T03:52:30.792686+0000","last_fullsized":"2023-10-06T03:52:30.792686+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:50.603686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.793194+0000","last_change":"2023-10-06T03:52:30.789478+0000","last_active":"2023-10-06T03:52:30.793194+0000","last_peered":"2023-10-06T03:52:30.793194+0000","last_clean":"2023-10-06T03:52:30.793194+0000","last_became_active":"2023-10-06T03:52:30.789179+0000","last_became_peered":"2023-10-06T03:52:30.789179+0000","last_unstale":"2023-10-06T03:52:30.793194+0000","last_undegraded":"2023-10-06T03:52:30.793194+0000","last_fullsized":"2023-10-06T03:52:30.793194+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:53.475559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.782858+0000","last_change":"2023-10-06T03:52:30.782858+0000","last_active":"2023-10-06T03:52:30.782858+0000","last_peered":"2023-10-06T03:52:30.782858+0000","last_clean":"2023-10-06T03:52:30.782858+0000","last_became_active":"2023-10-06T03:52:30.782599+0000","last_became_peered":"2023-10-06T03:52:30.782599+0000","last_unstale":"2023-10-06T03:52:30.782858+0000","last_undegraded":"2023-10-06T03:52:30.782858+0000","last_fullsized":"2023-10-06T03:52:30.782858+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:51.424396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.796174+0000","last_change":"2023-10-06T03:52:30.796174+0000","last_active":"2023-10-06T03:52:30.796174+0000","last_peered":"2023-10-06T03:52:30.796174+0000","last_clean":"2023-10-06T03:52:30.796174+0000","last_became_active":"2023-10-06T03:52:30.795915+0000","last_became_peered":"2023-10-06T03:52:30.795915+0000","last_unstale":"2023-10-06T03:52:30.796174+0000","last_undegraded":"2023-10-06T03:52:30.796174+0000","last_fullsized":"2023-10-06T03:52:30.796174+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:19.902819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.782330+0000","last_change":"2023-10-06T03:52:30.780064+0000","last_active":"2023-10-06T03:52:30.782330+0000","last_peered":"2023-10-06T03:52:30.782330+0000","last_clean":"2023-10-06T03:52:30.782330+0000","last_became_active":"2023-10-06T03:52:30.779799+0000","last_became_peered":"2023-10-06T03:52:30.779799+0000","last_unstale":"2023-10-06T03:52:30.782330+0000","last_undegraded":"2023-10-06T03:52:30.782330+0000","last_fullsized":"2023-10-06T03:52:30.782330+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:53:02.370347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.799023+0000","last_change":"2023-10-06T03:52:30.799023+0000","last_active":"2023-10-06T03:52:30.799023+0000","last_peered":"2023-10-06T03:52:30.799023+0000","last_clean":"2023-10-06T03:52:30.799023+0000","last_became_active":"2023-10-06T03:52:30.798724+0000","last_became_peered":"2023-10-06T03:52:30.798724+0000","last_unstale":"2023-10-06T03:52:30.799023+0000","last_undegraded":"2023-10-06T03:52:30.799023+0000","last_fullsized":"2023-10-06T03:52:30.799023+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:49.834343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.789061+0000","last_change":"2023-10-06T03:52:30.789061+0000","last_active":"2023-10-06T03:52:30.789061+0000","last_peered":"2023-10-06T03:52:30.789061+0000","last_clean":"2023-10-06T03:52:30.789061+0000","last_became_active":"2023-10-06T03:52:30.788786+0000","last_became_peered":"2023-10-06T03:52:30.788786+0000","last_unstale":"2023-10-06T03:52:30.789061+0000","last_undegraded":"2023-10-06T03:52:30.789061+0000","last_fullsized":"2023-10-06T03:52:30.789061+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:38.872611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792438+0000","last_change":"2023-10-06T03:52:30.792438+0000","last_active":"2023-10-06T03:52:30.792438+0000","last_peered":"2023-10-06T03:52:30.792438+0000","last_clean":"2023-10-06T03:52:30.792438+0000","last_became_active":"2023-10-06T03:52:30.792144+0000","last_became_peered":"2023-10-06T03:52:30.792144+0000","last_unstale":"2023-10-06T03:52:30.792438+0000","last_undegraded":"2023-10-06T03:52:30.792438+0000","last_fullsized":"2023-10-06T03:52:30.792438+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:23.019719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.781661+0000","last_change":"2023-10-06T03:52:30.781661+0000","last_active":"2023-10-06T03:52:30.781661+0000","last_peered":"2023-10-06T03:52:30.781661+0000","last_clean":"2023-10-06T03:52:30.781661+0000","last_became_active":"2023-10-06T03:52:30.781399+0000","last_became_peered":"2023-10-06T03:52:30.781399+0000","last_unstale":"2023-10-06T03:52:30.781661+0000","last_undegraded":"2023-10-06T03:52:30.781661+0000","last_fullsized":"2023-10-06T03:52:30.781661+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:58.880990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.793214+0000","last_change":"2023-10-06T03:52:30.789509+0000","last_active":"2023-10-06T03:52:30.793214+0000","last_peered":"2023-10-06T03:52:30.793214+0000","last_clean":"2023-10-06T03:52:30.793214+0000","last_became_active":"2023-10-06T03:52:30.789222+0000","last_became_peered":"2023-10-06T03:52:30.789222+0000","last_unstale":"2023-10-06T03:52:30.793214+0000","last_undegraded":"2023-10-06T03:52:30.793214+0000","last_fullsized":"2023-10-06T03:52:30.793214+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:16.612133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.783357+0000","last_change":"2023-10-06T03:52:30.783357+0000","last_active":"2023-10-06T03:52:30.783357+0000","last_peered":"2023-10-06T03:52:30.783357+0000","last_clean":"2023-10-06T03:52:30.783357+0000","last_became_active":"2023-10-06T03:52:30.782815+0000","last_became_peered":"2023-10-06T03:52:30.782815+0000","last_unstale":"2023-10-06T03:52:30.783357+0000","last_undegraded":"2023-10-06T03:52:30.783357+0000","last_fullsized":"2023-10-06T03:52:30.783357+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:17:50.704636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.787322+0000","last_change":"2023-10-06T03:52:30.787322+0000","last_active":"2023-10-06T03:52:30.787322+0000","last_peered":"2023-10-06T03:52:30.787322+0000","last_clean":"2023-10-06T03:52:30.787322+0000","last_became_active":"2023-10-06T03:52:30.787081+0000","last_became_peered":"2023-10-06T03:52:30.787081+0000","last_unstale":"2023-10-06T03:52:30.787322+0000","last_undegraded":"2023-10-06T03:52:30.787322+0000","last_fullsized":"2023-10-06T03:52:30.787322+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:40.364445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791730+0000","last_change":"2023-10-06T03:52:30.786966+0000","last_active":"2023-10-06T03:52:30.791730+0000","last_peered":"2023-10-06T03:52:30.791730+0000","last_clean":"2023-10-06T03:52:30.791730+0000","last_became_active":"2023-10-06T03:52:30.786709+0000","last_became_peered":"2023-10-06T03:52:30.786709+0000","last_unstale":"2023-10-06T03:52:30.791730+0000","last_undegraded":"2023-10-06T03:52:30.791730+0000","last_fullsized":"2023-10-06T03:52:30.791730+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:40.199639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.788080+0000","last_change":"2023-10-06T03:52:30.788080+0000","last_active":"2023-10-06T03:52:30.788080+0000","last_peered":"2023-10-06T03:52:30.788080+0000","last_clean":"2023-10-06T03:52:30.788080+0000","last_became_active":"2023-10-06T03:52:30.787766+0000","last_became_peered":"2023-10-06T03:52:30.787766+0000","last_unstale":"2023-10-06T03:52:30.788080+0000","last_undegraded":"2023-10-06T03:52:30.788080+0000","last_fullsized":"2023-10-06T03:52:30.788080+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:54.235229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.788385+0000","last_change":"2023-10-06T03:52:30.788385+0000","last_active":"2023-10-06T03:52:30.788385+0000","last_peered":"2023-10-06T03:52:30.788385+0000","last_clean":"2023-10-06T03:52:30.788385+0000","last_became_active":"2023-10-06T03:52:30.788096+0000","last_became_peered":"2023-10-06T03:52:30.788096+0000","last_unstale":"2023-10-06T03:52:30.788385+0000","last_undegraded":"2023-10-06T03:52:30.788385+0000","last_fullsized":"2023-10-06T03:52:30.788385+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:06.430173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.782949+0000","last_change":"2023-10-06T03:52:30.782949+0000","last_active":"2023-10-06T03:52:30.782949+0000","last_peered":"2023-10-06T03:52:30.782949+0000","last_clean":"2023-10-06T03:52:30.782949+0000","last_became_active":"2023-10-06T03:52:30.782656+0000","last_became_peered":"2023-10-06T03:52:30.782656+0000","last_unstale":"2023-10-06T03:52:30.782949+0000","last_undegraded":"2023-10-06T03:52:30.782949+0000","last_fullsized":"2023-10-06T03:52:30.782949+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:49.834343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"136'4","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.140386+0000","last_change":"2023-10-06T03:52:30.122412+0000","last_active":"2023-10-06T03:52:31.140386+0000","last_peered":"2023-10-06T03:52:31.140386+0000","last_clean":"2023-10-06T03:52:31.140386+0000","last_became_active":"2023-10-06T03:52:30.122148+0000","last_became_peered":"2023-10-06T03:52:30.122148+0000","last_unstale":"2023-10-06T03:52:31.140386+0000","last_undegraded":"2023-10-06T03:52:31.140386+0000","last_fullsized":"2023-10-06T03:52:31.140386+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:49:49.189447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792867+0000","last_change":"2023-10-06T03:52:30.792867+0000","last_active":"2023-10-06T03:52:30.792867+0000","last_peered":"2023-10-06T03:52:30.792867+0000","last_clean":"2023-10-06T03:52:30.792867+0000","last_became_active":"2023-10-06T03:52:30.792401+0000","last_became_peered":"2023-10-06T03:52:30.792401+0000","last_unstale":"2023-10-06T03:52:30.792867+0000","last_undegraded":"2023-10-06T03:52:30.792867+0000","last_fullsized":"2023-10-06T03:52:30.792867+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:40:11.861935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"136'1","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.771658+0000","last_change":"2023-10-06T03:52:30.115824+0000","last_active":"2023-10-06T03:52:30.771658+0000","last_peered":"2023-10-06T03:52:30.771658+0000","last_clean":"2023-10-06T03:52:30.771658+0000","last_became_active":"2023-10-06T03:52:30.115527+0000","last_became_peered":"2023-10-06T03:52:30.115527+0000","last_unstale":"2023-10-06T03:52:30.771658+0000","last_undegraded":"2023-10-06T03:52:30.771658+0000","last_fullsized":"2023-10-06T03:52:30.771658+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:19.855379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.783882+0000","last_change":"2023-10-06T03:52:30.783882+0000","last_active":"2023-10-06T03:52:30.783882+0000","last_peered":"2023-10-06T03:52:30.783882+0000","last_clean":"2023-10-06T03:52:30.783882+0000","last_became_active":"2023-10-06T03:52:30.783597+0000","last_became_peered":"2023-10-06T03:52:30.783597+0000","last_unstale":"2023-10-06T03:52:30.783882+0000","last_undegraded":"2023-10-06T03:52:30.783882+0000","last_fullsized":"2023-10-06T03:52:30.783882+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:51.019536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"136'2","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.138473+0000","last_change":"2023-10-06T03:52:30.126541+0000","last_active":"2023-10-06T03:52:31.138473+0000","last_peered":"2023-10-06T03:52:31.138473+0000","last_clean":"2023-10-06T03:52:31.138473+0000","last_became_active":"2023-10-06T03:52:30.126207+0000","last_became_peered":"2023-10-06T03:52:30.126207+0000","last_unstale":"2023-10-06T03:52:31.138473+0000","last_undegraded":"2023-10-06T03:52:31.138473+0000","last_fullsized":"2023-10-06T03:52:31.138473+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:29.096899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.795521+0000","last_change":"2023-10-06T03:52:30.795521+0000","last_active":"2023-10-06T03:52:30.795521+0000","last_peered":"2023-10-06T03:52:30.795521+0000","last_clean":"2023-10-06T03:52:30.795521+0000","last_became_active":"2023-10-06T03:52:30.795269+0000","last_became_peered":"2023-10-06T03:52:30.795269+0000","last_unstale":"2023-10-06T03:52:30.795521+0000","last_undegraded":"2023-10-06T03:52:30.795521+0000","last_fullsized":"2023-10-06T03:52:30.795521+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:13.980241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"136'2","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.141212+0000","last_change":"2023-10-06T03:52:30.114186+0000","last_active":"2023-10-06T03:52:31.141212+0000","last_peered":"2023-10-06T03:52:31.141212+0000","last_clean":"2023-10-06T03:52:31.141212+0000","last_became_active":"2023-10-06T03:52:30.113925+0000","last_became_peered":"2023-10-06T03:52:30.113925+0000","last_unstale":"2023-10-06T03:52:31.141212+0000","last_undegraded":"2023-10-06T03:52:31.141212+0000","last_fullsized":"2023-10-06T03:52:31.141212+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:30:17.813294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.777518+0000","last_change":"2023-10-06T03:52:30.777518+0000","last_active":"2023-10-06T03:52:30.777518+0000","last_peered":"2023-10-06T03:52:30.777518+0000","last_clean":"2023-10-06T03:52:30.777518+0000","last_became_active":"2023-10-06T03:52:30.777243+0000","last_became_peered":"2023-10-06T03:52:30.777243+0000","last_unstale":"2023-10-06T03:52:30.777518+0000","last_undegraded":"2023-10-06T03:52:30.777518+0000","last_fullsized":"2023-10-06T03:52:30.777518+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:08.324906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"136'1","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.138812+0000","last_change":"2023-10-06T03:52:30.123795+0000","last_active":"2023-10-06T03:52:31.138812+0000","last_peered":"2023-10-06T03:52:31.138812+0000","last_clean":"2023-10-06T03:52:31.138812+0000","last_became_active":"2023-10-06T03:52:30.123344+0000","last_became_peered":"2023-10-06T03:52:30.123344+0000","last_unstale":"2023-10-06T03:52:31.138812+0000","last_undegraded":"2023-10-06T03:52:31.138812+0000","last_fullsized":"2023-10-06T03:52:31.138812+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:39:49.592921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.788538+0000","last_change":"2023-10-06T03:52:30.787672+0000","last_active":"2023-10-06T03:52:30.788538+0000","last_peered":"2023-10-06T03:52:30.788538+0000","last_clean":"2023-10-06T03:52:30.788538+0000","last_became_active":"2023-10-06T03:52:30.787408+0000","last_became_peered":"2023-10-06T03:52:30.787408+0000","last_unstale":"2023-10-06T03:52:30.788538+0000","last_undegraded":"2023-10-06T03:52:30.788538+0000","last_fullsized":"2023-10-06T03:52:30.788538+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:55.058210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"136'1","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.776781+0000","last_change":"2023-10-06T03:52:30.121391+0000","last_active":"2023-10-06T03:52:30.776781+0000","last_peered":"2023-10-06T03:52:30.776781+0000","last_clean":"2023-10-06T03:52:30.776781+0000","last_became_active":"2023-10-06T03:52:30.120531+0000","last_became_peered":"2023-10-06T03:52:30.120531+0000","last_unstale":"2023-10-06T03:52:30.776781+0000","last_undegraded":"2023-10-06T03:52:30.776781+0000","last_fullsized":"2023-10-06T03:52:30.776781+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:54.236766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791211+0000","last_change":"2023-10-06T03:52:30.781510+0000","last_active":"2023-10-06T03:52:30.791211+0000","last_peered":"2023-10-06T03:52:30.791211+0000","last_clean":"2023-10-06T03:52:30.791211+0000","last_became_active":"2023-10-06T03:52:30.781269+0000","last_became_peered":"2023-10-06T03:52:30.781269+0000","last_unstale":"2023-10-06T03:52:30.791211+0000","last_undegraded":"2023-10-06T03:52:30.791211+0000","last_fullsized":"2023-10-06T03:52:30.791211+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:26:43.869500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.124808+0000","last_change":"2023-10-06T03:52:30.119373+0000","last_active":"2023-10-06T03:52:30.124808+0000","last_peered":"2023-10-06T03:52:30.124808+0000","last_clean":"2023-10-06T03:52:30.124808+0000","last_became_active":"2023-10-06T03:52:30.119118+0000","last_became_peered":"2023-10-06T03:52:30.119118+0000","last_unstale":"2023-10-06T03:52:30.124808+0000","last_undegraded":"2023-10-06T03:52:30.124808+0000","last_fullsized":"2023-10-06T03:52:30.124808+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:38:52.555468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792187+0000","last_change":"2023-10-06T03:52:30.792187+0000","last_active":"2023-10-06T03:52:30.792187+0000","last_peered":"2023-10-06T03:52:30.792187+0000","last_clean":"2023-10-06T03:52:30.792187+0000","last_became_active":"2023-10-06T03:52:30.791904+0000","last_became_peered":"2023-10-06T03:52:30.791904+0000","last_unstale":"2023-10-06T03:52:30.792187+0000","last_undegraded":"2023-10-06T03:52:30.792187+0000","last_fullsized":"2023-10-06T03:52:30.792187+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:56:48.623123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.124912+0000","last_change":"2023-10-06T03:52:30.123936+0000","last_active":"2023-10-06T03:52:30.124912+0000","last_peered":"2023-10-06T03:52:30.124912+0000","last_clean":"2023-10-06T03:52:30.124912+0000","last_became_active":"2023-10-06T03:52:30.121761+0000","last_became_peered":"2023-10-06T03:52:30.121761+0000","last_unstale":"2023-10-06T03:52:30.124912+0000","last_undegraded":"2023-10-06T03:52:30.124912+0000","last_fullsized":"2023-10-06T03:52:30.124912+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:49:49.625382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.786434+0000","last_change":"2023-10-06T03:52:30.786434+0000","last_active":"2023-10-06T03:52:30.786434+0000","last_peered":"2023-10-06T03:52:30.786434+0000","last_clean":"2023-10-06T03:52:30.786434+0000","last_became_active":"2023-10-06T03:52:30.786137+0000","last_became_peered":"2023-10-06T03:52:30.786137+0000","last_unstale":"2023-10-06T03:52:30.786434+0000","last_undegraded":"2023-10-06T03:52:30.786434+0000","last_fullsized":"2023-10-06T03:52:30.786434+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:44.658960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"136'1","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.772169+0000","last_change":"2023-10-06T03:52:30.122297+0000","last_active":"2023-10-06T03:52:30.772169+0000","last_peered":"2023-10-06T03:52:30.772169+0000","last_clean":"2023-10-06T03:52:30.772169+0000","last_became_active":"2023-10-06T03:52:30.122053+0000","last_became_peered":"2023-10-06T03:52:30.122053+0000","last_unstale":"2023-10-06T03:52:30.772169+0000","last_undegraded":"2023-10-06T03:52:30.772169+0000","last_fullsized":"2023-10-06T03:52:30.772169+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:08.789519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.785896+0000","last_change":"2023-10-06T03:52:30.780142+0000","last_active":"2023-10-06T03:52:30.785896+0000","last_peered":"2023-10-06T03:52:30.785896+0000","last_clean":"2023-10-06T03:52:30.785896+0000","last_became_active":"2023-10-06T03:52:30.779316+0000","last_became_peered":"2023-10-06T03:52:30.779316+0000","last_unstale":"2023-10-06T03:52:30.785896+0000","last_undegraded":"2023-10-06T03:52:30.785896+0000","last_fullsized":"2023-10-06T03:52:30.785896+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:14:40.343603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"136'2","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.139391+0000","last_change":"2023-10-06T03:52:30.123022+0000","last_active":"2023-10-06T03:52:31.139391+0000","last_peered":"2023-10-06T03:52:31.139391+0000","last_clean":"2023-10-06T03:52:31.139391+0000","last_became_active":"2023-10-06T03:52:30.122727+0000","last_became_peered":"2023-10-06T03:52:30.122727+0000","last_unstale":"2023-10-06T03:52:31.139391+0000","last_undegraded":"2023-10-06T03:52:31.139391+0000","last_fullsized":"2023-10-06T03:52:31.139391+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:46:35.897602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.790414+0000","last_change":"2023-10-06T03:52:30.790414+0000","last_active":"2023-10-06T03:52:30.790414+0000","last_peered":"2023-10-06T03:52:30.790414+0000","last_clean":"2023-10-06T03:52:30.790414+0000","last_became_active":"2023-10-06T03:52:30.790072+0000","last_became_peered":"2023-10-06T03:52:30.790072+0000","last_unstale":"2023-10-06T03:52:30.790414+0000","last_undegraded":"2023-10-06T03:52:30.790414+0000","last_fullsized":"2023-10-06T03:52:30.790414+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:50:20.822712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"136'1","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.782695+0000","last_change":"2023-10-06T03:52:30.126842+0000","last_active":"2023-10-06T03:52:30.782695+0000","last_peered":"2023-10-06T03:52:30.782695+0000","last_clean":"2023-10-06T03:52:30.782695+0000","last_became_active":"2023-10-06T03:52:30.126484+0000","last_became_peered":"2023-10-06T03:52:30.126484+0000","last_unstale":"2023-10-06T03:52:30.782695+0000","last_undegraded":"2023-10-06T03:52:30.782695+0000","last_fullsized":"2023-10-06T03:52:30.782695+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:37.861846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791182+0000","last_change":"2023-10-06T03:52:30.791182+0000","last_active":"2023-10-06T03:52:30.791182+0000","last_peered":"2023-10-06T03:52:30.791182+0000","last_clean":"2023-10-06T03:52:30.791182+0000","last_became_active":"2023-10-06T03:52:30.790881+0000","last_became_peered":"2023-10-06T03:52:30.790881+0000","last_unstale":"2023-10-06T03:52:30.791182+0000","last_undegraded":"2023-10-06T03:52:30.791182+0000","last_fullsized":"2023-10-06T03:52:30.791182+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:05.986856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.131452+0000","last_change":"2023-10-06T03:52:30.126295+0000","last_active":"2023-10-06T03:52:30.131452+0000","last_peered":"2023-10-06T03:52:30.131452+0000","last_clean":"2023-10-06T03:52:30.131452+0000","last_became_active":"2023-10-06T03:52:30.126042+0000","last_became_peered":"2023-10-06T03:52:30.126042+0000","last_unstale":"2023-10-06T03:52:30.131452+0000","last_undegraded":"2023-10-06T03:52:30.131452+0000","last_fullsized":"2023-10-06T03:52:30.131452+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:39.364845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791424+0000","last_change":"2023-10-06T03:52:30.791424+0000","last_active":"2023-10-06T03:52:30.791424+0000","last_peered":"2023-10-06T03:52:30.791424+0000","last_clean":"2023-10-06T03:52:30.791424+0000","last_became_active":"2023-10-06T03:52:30.790495+0000","last_became_peered":"2023-10-06T03:52:30.790495+0000","last_unstale":"2023-10-06T03:52:30.791424+0000","last_undegraded":"2023-10-06T03:52:30.791424+0000","last_fullsized":"2023-10-06T03:52:30.791424+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:41.471661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"136'2","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.140692+0000","last_change":"2023-10-06T03:52:30.127220+0000","last_active":"2023-10-06T03:52:31.140692+0000","last_peered":"2023-10-06T03:52:31.140692+0000","last_clean":"2023-10-06T03:52:31.140692+0000","last_became_active":"2023-10-06T03:52:30.126966+0000","last_became_peered":"2023-10-06T03:52:30.126966+0000","last_unstale":"2023-10-06T03:52:31.140692+0000","last_undegraded":"2023-10-06T03:52:31.140692+0000","last_fullsized":"2023-10-06T03:52:31.140692+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:49:06.959275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.794648+0000","last_change":"2023-10-06T03:52:30.793815+0000","last_active":"2023-10-06T03:52:30.794648+0000","last_peered":"2023-10-06T03:52:30.794648+0000","last_clean":"2023-10-06T03:52:30.794648+0000","last_became_active":"2023-10-06T03:52:30.793596+0000","last_became_peered":"2023-10-06T03:52:30.793596+0000","last_unstale":"2023-10-06T03:52:30.794648+0000","last_undegraded":"2023-10-06T03:52:30.794648+0000","last_fullsized":"2023-10-06T03:52:30.794648+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:46:53.684006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.131594+0000","last_change":"2023-10-06T03:52:30.126930+0000","last_active":"2023-10-06T03:52:30.131594+0000","last_peered":"2023-10-06T03:52:30.131594+0000","last_clean":"2023-10-06T03:52:30.131594+0000","last_became_active":"2023-10-06T03:52:30.126448+0000","last_became_peered":"2023-10-06T03:52:30.126448+0000","last_unstale":"2023-10-06T03:52:30.131594+0000","last_undegraded":"2023-10-06T03:52:30.131594+0000","last_fullsized":"2023-10-06T03:52:30.131594+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:05.419407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792835+0000","last_change":"2023-10-06T03:52:30.788782+0000","last_active":"2023-10-06T03:52:30.792835+0000","last_peered":"2023-10-06T03:52:30.792835+0000","last_clean":"2023-10-06T03:52:30.792835+0000","last_became_active":"2023-10-06T03:52:30.788502+0000","last_became_peered":"2023-10-06T03:52:30.788502+0000","last_unstale":"2023-10-06T03:52:30.792835+0000","last_undegraded":"2023-10-06T03:52:30.792835+0000","last_fullsized":"2023-10-06T03:52:30.792835+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:08.176820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.775372+0000","last_change":"2023-10-06T03:52:30.123836+0000","last_active":"2023-10-06T03:52:30.775372+0000","last_peered":"2023-10-06T03:52:30.775372+0000","last_clean":"2023-10-06T03:52:30.775372+0000","last_became_active":"2023-10-06T03:52:30.123516+0000","last_became_peered":"2023-10-06T03:52:30.123516+0000","last_unstale":"2023-10-06T03:52:30.775372+0000","last_undegraded":"2023-10-06T03:52:30.775372+0000","last_fullsized":"2023-10-06T03:52:30.775372+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:47.339010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791009+0000","last_change":"2023-10-06T03:52:30.791009+0000","last_active":"2023-10-06T03:52:30.791009+0000","last_peered":"2023-10-06T03:52:30.791009+0000","last_clean":"2023-10-06T03:52:30.791009+0000","last_became_active":"2023-10-06T03:52:30.790739+0000","last_became_peered":"2023-10-06T03:52:30.790739+0000","last_unstale":"2023-10-06T03:52:30.791009+0000","last_undegraded":"2023-10-06T03:52:30.791009+0000","last_fullsized":"2023-10-06T03:52:30.791009+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:23:30.107665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.133884+0000","last_change":"2023-10-06T03:52:30.130812+0000","last_active":"2023-10-06T03:52:30.133884+0000","last_peered":"2023-10-06T03:52:30.133884+0000","last_clean":"2023-10-06T03:52:30.133884+0000","last_became_active":"2023-10-06T03:52:30.130450+0000","last_became_peered":"2023-10-06T03:52:30.130450+0000","last_unstale":"2023-10-06T03:52:30.133884+0000","last_undegraded":"2023-10-06T03:52:30.133884+0000","last_fullsized":"2023-10-06T03:52:30.133884+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:47.339010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.785094+0000","last_change":"2023-10-06T03:52:30.785094+0000","last_active":"2023-10-06T03:52:30.785094+0000","last_peered":"2023-10-06T03:52:30.785094+0000","last_clean":"2023-10-06T03:52:30.785094+0000","last_became_active":"2023-10-06T03:52:30.784835+0000","last_became_peered":"2023-10-06T03:52:30.784835+0000","last_unstale":"2023-10-06T03:52:30.785094+0000","last_undegraded":"2023-10-06T03:52:30.785094+0000","last_fullsized":"2023-10-06T03:52:30.785094+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:10:38.355926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"136'2","reported_seq":11,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.139253+0000","last_change":"2023-10-06T03:52:30.127133+0000","last_active":"2023-10-06T03:52:31.139253+0000","last_peered":"2023-10-06T03:52:31.139253+0000","last_clean":"2023-10-06T03:52:31.139253+0000","last_became_active":"2023-10-06T03:52:30.126818+0000","last_became_peered":"2023-10-06T03:52:30.126818+0000","last_unstale":"2023-10-06T03:52:31.139253+0000","last_undegraded":"2023-10-06T03:52:31.139253+0000","last_fullsized":"2023-10-06T03:52:31.139253+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:57.985312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.778510+0000","last_change":"2023-10-06T03:52:30.778510+0000","last_active":"2023-10-06T03:52:30.778510+0000","last_peered":"2023-10-06T03:52:30.778510+0000","last_clean":"2023-10-06T03:52:30.778510+0000","last_became_active":"2023-10-06T03:52:30.778050+0000","last_became_peered":"2023-10-06T03:52:30.778050+0000","last_unstale":"2023-10-06T03:52:30.778510+0000","last_undegraded":"2023-10-06T03:52:30.778510+0000","last_fullsized":"2023-10-06T03:52:30.778510+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:08:45.904493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.774512+0000","last_change":"2023-10-06T03:52:30.124624+0000","last_active":"2023-10-06T03:52:30.774512+0000","last_peered":"2023-10-06T03:52:30.774512+0000","last_clean":"2023-10-06T03:52:30.774512+0000","last_became_active":"2023-10-06T03:52:30.124329+0000","last_became_peered":"2023-10-06T03:52:30.124329+0000","last_unstale":"2023-10-06T03:52:30.774512+0000","last_undegraded":"2023-10-06T03:52:30.774512+0000","last_fullsized":"2023-10-06T03:52:30.774512+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:43:06.737045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792657+0000","last_change":"2023-10-06T03:52:30.792657+0000","last_active":"2023-10-06T03:52:30.792657+0000","last_peered":"2023-10-06T03:52:30.792657+0000","last_clean":"2023-10-06T03:52:30.792657+0000","last_became_active":"2023-10-06T03:52:30.791489+0000","last_became_peered":"2023-10-06T03:52:30.791489+0000","last_unstale":"2023-10-06T03:52:30.792657+0000","last_undegraded":"2023-10-06T03:52:30.792657+0000","last_fullsized":"2023-10-06T03:52:30.792657+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:23.327175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.774777+0000","last_change":"2023-10-06T03:52:30.125853+0000","last_active":"2023-10-06T03:52:30.774777+0000","last_peered":"2023-10-06T03:52:30.774777+0000","last_clean":"2023-10-06T03:52:30.774777+0000","last_became_active":"2023-10-06T03:52:30.125026+0000","last_became_peered":"2023-10-06T03:52:30.125026+0000","last_unstale":"2023-10-06T03:52:30.774777+0000","last_undegraded":"2023-10-06T03:52:30.774777+0000","last_fullsized":"2023-10-06T03:52:30.774777+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:14.292118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.789955+0000","last_change":"2023-10-06T03:52:30.789955+0000","last_active":"2023-10-06T03:52:30.789955+0000","last_peered":"2023-10-06T03:52:30.789955+0000","last_clean":"2023-10-06T03:52:30.789955+0000","last_became_active":"2023-10-06T03:52:30.789685+0000","last_became_peered":"2023-10-06T03:52:30.789685+0000","last_unstale":"2023-10-06T03:52:30.789955+0000","last_undegraded":"2023-10-06T03:52:30.789955+0000","last_fullsized":"2023-10-06T03:52:30.789955+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:29:26.405684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.128980+0000","last_change":"2023-10-06T03:52:30.126293+0000","last_active":"2023-10-06T03:52:30.128980+0000","last_peered":"2023-10-06T03:52:30.128980+0000","last_clean":"2023-10-06T03:52:30.128980+0000","last_became_active":"2023-10-06T03:52:30.126058+0000","last_became_peered":"2023-10-06T03:52:30.126058+0000","last_unstale":"2023-10-06T03:52:30.128980+0000","last_undegraded":"2023-10-06T03:52:30.128980+0000","last_fullsized":"2023-10-06T03:52:30.128980+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:21:13.116018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.787768+0000","last_change":"2023-10-06T03:52:30.787768+0000","last_active":"2023-10-06T03:52:30.787768+0000","last_peered":"2023-10-06T03:52:30.787768+0000","last_clean":"2023-10-06T03:52:30.787768+0000","last_became_active":"2023-10-06T03:52:30.787507+0000","last_became_peered":"2023-10-06T03:52:30.787507+0000","last_unstale":"2023-10-06T03:52:30.787768+0000","last_undegraded":"2023-10-06T03:52:30.787768+0000","last_fullsized":"2023-10-06T03:52:30.787768+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:57.047978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.773809+0000","last_change":"2023-10-06T03:52:30.123623+0000","last_active":"2023-10-06T03:52:30.773809+0000","last_peered":"2023-10-06T03:52:30.773809+0000","last_clean":"2023-10-06T03:52:30.773809+0000","last_became_active":"2023-10-06T03:52:30.123051+0000","last_became_peered":"2023-10-06T03:52:30.123051+0000","last_unstale":"2023-10-06T03:52:30.773809+0000","last_undegraded":"2023-10-06T03:52:30.773809+0000","last_fullsized":"2023-10-06T03:52:30.773809+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:39.188874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791661+0000","last_change":"2023-10-06T03:52:30.786538+0000","last_active":"2023-10-06T03:52:30.791661+0000","last_peered":"2023-10-06T03:52:30.791661+0000","last_clean":"2023-10-06T03:52:30.791661+0000","last_became_active":"2023-10-06T03:52:30.786295+0000","last_became_peered":"2023-10-06T03:52:30.786295+0000","last_unstale":"2023-10-06T03:52:30.791661+0000","last_undegraded":"2023-10-06T03:52:30.791661+0000","last_fullsized":"2023-10-06T03:52:30.791661+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:56:23.327175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.773034+0000","last_change":"2023-10-06T03:52:30.126345+0000","last_active":"2023-10-06T03:52:30.773034+0000","last_peered":"2023-10-06T03:52:30.773034+0000","last_clean":"2023-10-06T03:52:30.773034+0000","last_became_active":"2023-10-06T03:52:30.126070+0000","last_became_peered":"2023-10-06T03:52:30.126070+0000","last_unstale":"2023-10-06T03:52:30.773034+0000","last_undegraded":"2023-10-06T03:52:30.773034+0000","last_fullsized":"2023-10-06T03:52:30.773034+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:42:26.213861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.788953+0000","last_change":"2023-10-06T03:52:30.788953+0000","last_active":"2023-10-06T03:52:30.788953+0000","last_peered":"2023-10-06T03:52:30.788953+0000","last_clean":"2023-10-06T03:52:30.788953+0000","last_became_active":"2023-10-06T03:52:30.788498+0000","last_became_peered":"2023-10-06T03:52:30.788498+0000","last_unstale":"2023-10-06T03:52:30.788953+0000","last_undegraded":"2023-10-06T03:52:30.788953+0000","last_fullsized":"2023-10-06T03:52:30.788953+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:44:08.176820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.124204+0000","last_change":"2023-10-06T03:52:30.120855+0000","last_active":"2023-10-06T03:52:30.124204+0000","last_peered":"2023-10-06T03:52:30.124204+0000","last_clean":"2023-10-06T03:52:30.124204+0000","last_became_active":"2023-10-06T03:52:30.120503+0000","last_became_peered":"2023-10-06T03:52:30.120503+0000","last_unstale":"2023-10-06T03:52:30.124204+0000","last_undegraded":"2023-10-06T03:52:30.124204+0000","last_fullsized":"2023-10-06T03:52:30.124204+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:46.440943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.786623+0000","last_change":"2023-10-06T03:52:30.786623+0000","last_active":"2023-10-06T03:52:30.786623+0000","last_peered":"2023-10-06T03:52:30.786623+0000","last_clean":"2023-10-06T03:52:30.786623+0000","last_became_active":"2023-10-06T03:52:30.786384+0000","last_became_peered":"2023-10-06T03:52:30.786384+0000","last_unstale":"2023-10-06T03:52:30.786623+0000","last_undegraded":"2023-10-06T03:52:30.786623+0000","last_fullsized":"2023-10-06T03:52:30.786623+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:30:52.123264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.131936+0000","last_change":"2023-10-06T03:52:30.127346+0000","last_active":"2023-10-06T03:52:30.131936+0000","last_peered":"2023-10-06T03:52:30.131936+0000","last_clean":"2023-10-06T03:52:30.131936+0000","last_became_active":"2023-10-06T03:52:30.127068+0000","last_became_peered":"2023-10-06T03:52:30.127068+0000","last_unstale":"2023-10-06T03:52:30.131936+0000","last_undegraded":"2023-10-06T03:52:30.131936+0000","last_fullsized":"2023-10-06T03:52:30.131936+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:53.224463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792669+0000","last_change":"2023-10-06T03:52:30.789354+0000","last_active":"2023-10-06T03:52:30.792669+0000","last_peered":"2023-10-06T03:52:30.792669+0000","last_clean":"2023-10-06T03:52:30.792669+0000","last_became_active":"2023-10-06T03:52:30.788899+0000","last_became_peered":"2023-10-06T03:52:30.788899+0000","last_unstale":"2023-10-06T03:52:30.792669+0000","last_undegraded":"2023-10-06T03:52:30.792669+0000","last_fullsized":"2023-10-06T03:52:30.792669+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:13.980241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"136'3","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.139404+0000","last_change":"2023-10-06T03:52:30.120725+0000","last_active":"2023-10-06T03:52:31.139404+0000","last_peered":"2023-10-06T03:52:31.139404+0000","last_clean":"2023-10-06T03:52:31.139404+0000","last_became_active":"2023-10-06T03:52:30.120448+0000","last_became_peered":"2023-10-06T03:52:30.120448+0000","last_unstale":"2023-10-06T03:52:31.139404+0000","last_undegraded":"2023-10-06T03:52:31.139404+0000","last_fullsized":"2023-10-06T03:52:31.139404+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:23:42.123352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":123,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.772771+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T03:52:30.772771+0000","last_peered":"2023-10-06T03:52:30.772771+0000","last_clean":"2023-10-06T03:52:30.772771+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T03:52:30.772771+0000","last_undegraded":"2023-10-06T03:52:30.772771+0000","last_fullsized":"2023-10-06T03:52:30.772771+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.791890+0000","last_change":"2023-10-06T03:52:30.791890+0000","last_active":"2023-10-06T03:52:30.791890+0000","last_peered":"2023-10-06T03:52:30.791890+0000","last_clean":"2023-10-06T03:52:30.791890+0000","last_became_active":"2023-10-06T03:52:30.791444+0000","last_became_peered":"2023-10-06T03:52:30.791444+0000","last_unstale":"2023-10-06T03:52:30.791890+0000","last_undegraded":"2023-10-06T03:52:30.791890+0000","last_fullsized":"2023-10-06T03:52:30.791890+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:24:48.349776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"136'6","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.143486+0000","last_change":"2023-10-06T03:52:30.122962+0000","last_active":"2023-10-06T03:52:31.143486+0000","last_peered":"2023-10-06T03:52:31.143486+0000","last_clean":"2023-10-06T03:52:31.143486+0000","last_became_active":"2023-10-06T03:52:30.122675+0000","last_became_peered":"2023-10-06T03:52:30.122675+0000","last_unstale":"2023-10-06T03:52:31.143486+0000","last_undegraded":"2023-10-06T03:52:31.143486+0000","last_fullsized":"2023-10-06T03:52:31.143486+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:22.008954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.781089+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T03:52:30.781089+0000","last_peered":"2023-10-06T03:52:30.781089+0000","last_clean":"2023-10-06T03:52:30.781089+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T03:52:30.781089+0000","last_undegraded":"2023-10-06T03:52:30.781089+0000","last_fullsized":"2023-10-06T03:52:30.781089+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.784795+0000","last_change":"2023-10-06T03:52:30.781226+0000","last_active":"2023-10-06T03:52:30.784795+0000","last_peered":"2023-10-06T03:52:30.784795+0000","last_clean":"2023-10-06T03:52:30.784795+0000","last_became_active":"2023-10-06T03:52:30.780960+0000","last_became_peered":"2023-10-06T03:52:30.780960+0000","last_unstale":"2023-10-06T03:52:30.784795+0000","last_undegraded":"2023-10-06T03:52:30.784795+0000","last_fullsized":"2023-10-06T03:52:30.784795+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:20.866145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"136'2","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.141241+0000","last_change":"2023-10-06T03:52:30.124669+0000","last_active":"2023-10-06T03:52:31.141241+0000","last_peered":"2023-10-06T03:52:31.141241+0000","last_clean":"2023-10-06T03:52:31.141241+0000","last_became_active":"2023-10-06T03:52:30.124349+0000","last_became_peered":"2023-10-06T03:52:30.124349+0000","last_unstale":"2023-10-06T03:52:31.141241+0000","last_undegraded":"2023-10-06T03:52:31.141241+0000","last_fullsized":"2023-10-06T03:52:31.141241+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:39.364845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":113,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.101559+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T03:52:30.101559+0000","last_peered":"2023-10-06T03:52:30.101559+0000","last_clean":"2023-10-06T03:52:30.101559+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T03:52:30.101559+0000","last_undegraded":"2023-10-06T03:52:30.101559+0000","last_fullsized":"2023-10-06T03:52:30.101559+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"125'249","reported_seq":731,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.118685+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T03:52:30.118685+0000","last_peered":"2023-10-06T03:52:30.118685+0000","last_clean":"2023-10-06T03:52:30.118685+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T03:52:30.118685+0000","last_undegraded":"2023-10-06T03:52:30.118685+0000","last_fullsized":"2023-10-06T03:52:30.118685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.785965+0000","last_change":"2023-10-06T03:52:30.785965+0000","last_active":"2023-10-06T03:52:30.785965+0000","last_peered":"2023-10-06T03:52:30.785965+0000","last_clean":"2023-10-06T03:52:30.785965+0000","last_became_active":"2023-10-06T03:52:30.785689+0000","last_became_peered":"2023-10-06T03:52:30.785689+0000","last_unstale":"2023-10-06T03:52:30.785965+0000","last_undegraded":"2023-10-06T03:52:30.785965+0000","last_fullsized":"2023-10-06T03:52:30.785965+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:19:36.796724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.776817+0000","last_change":"2023-10-06T03:52:30.120206+0000","last_active":"2023-10-06T03:52:30.776817+0000","last_peered":"2023-10-06T03:52:30.776817+0000","last_clean":"2023-10-06T03:52:30.776817+0000","last_became_active":"2023-10-06T03:52:30.119548+0000","last_became_peered":"2023-10-06T03:52:30.119548+0000","last_unstale":"2023-10-06T03:52:30.776817+0000","last_undegraded":"2023-10-06T03:52:30.776817+0000","last_fullsized":"2023-10-06T03:52:30.776817+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:01:43.648194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":114,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.117819+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T03:52:30.117819+0000","last_peered":"2023-10-06T03:52:30.117819+0000","last_clean":"2023-10-06T03:52:30.117819+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T03:52:30.117819+0000","last_undegraded":"2023-10-06T03:52:30.117819+0000","last_fullsized":"2023-10-06T03:52:30.117819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.796924+0000","last_change":"2023-10-06T03:52:30.796924+0000","last_active":"2023-10-06T03:52:30.796924+0000","last_peered":"2023-10-06T03:52:30.796924+0000","last_clean":"2023-10-06T03:52:30.796924+0000","last_became_active":"2023-10-06T03:52:30.796673+0000","last_became_peered":"2023-10-06T03:52:30.796673+0000","last_unstale":"2023-10-06T03:52:30.796924+0000","last_undegraded":"2023-10-06T03:52:30.796924+0000","last_fullsized":"2023-10-06T03:52:30.796924+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:59:03.223461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"136'8","reported_seq":18,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:31.142155+0000","last_change":"2023-10-06T03:52:30.124575+0000","last_active":"2023-10-06T03:52:31.142155+0000","last_peered":"2023-10-06T03:52:31.142155+0000","last_clean":"2023-10-06T03:52:31.142155+0000","last_became_active":"2023-10-06T03:52:30.124300+0000","last_became_peered":"2023-10-06T03:52:30.124300+0000","last_unstale":"2023-10-06T03:52:31.142155+0000","last_undegraded":"2023-10-06T03:52:31.142155+0000","last_fullsized":"2023-10-06T03:52:31.142155+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:53.224463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.118761+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T03:52:30.118761+0000","last_peered":"2023-10-06T03:52:30.118761+0000","last_clean":"2023-10-06T03:52:30.118761+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T03:52:30.118761+0000","last_undegraded":"2023-10-06T03:52:30.118761+0000","last_fullsized":"2023-10-06T03:52:30.118761+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.786351+0000","last_change":"2023-10-06T03:52:30.786351+0000","last_active":"2023-10-06T03:52:30.786351+0000","last_peered":"2023-10-06T03:52:30.786351+0000","last_clean":"2023-10-06T03:52:30.786351+0000","last_became_active":"2023-10-06T03:52:30.786101+0000","last_became_peered":"2023-10-06T03:52:30.786101+0000","last_unstale":"2023-10-06T03:52:30.786351+0000","last_undegraded":"2023-10-06T03:52:30.786351+0000","last_fullsized":"2023-10-06T03:52:30.786351+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:40.740425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.775271+0000","last_change":"2023-10-06T03:52:30.118651+0000","last_active":"2023-10-06T03:52:30.775271+0000","last_peered":"2023-10-06T03:52:30.775271+0000","last_clean":"2023-10-06T03:52:30.775271+0000","last_became_active":"2023-10-06T03:52:30.118385+0000","last_became_peered":"2023-10-06T03:52:30.118385+0000","last_unstale":"2023-10-06T03:52:30.775271+0000","last_undegraded":"2023-10-06T03:52:30.775271+0000","last_fullsized":"2023-10-06T03:52:30.775271+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:23:50.008770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":108,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.111356+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T03:52:30.111356+0000","last_peered":"2023-10-06T03:52:30.111356+0000","last_clean":"2023-10-06T03:52:30.111356+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T03:52:30.111356+0000","last_undegraded":"2023-10-06T03:52:30.111356+0000","last_fullsized":"2023-10-06T03:52:30.111356+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":8,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.792384+0000","last_change":"2023-10-06T03:52:30.792384+0000","last_active":"2023-10-06T03:52:30.792384+0000","last_peered":"2023-10-06T03:52:30.792384+0000","last_clean":"2023-10-06T03:52:30.792384+0000","last_became_active":"2023-10-06T03:52:30.792112+0000","last_became_peered":"2023-10-06T03:52:30.792112+0000","last_unstale":"2023-10-06T03:52:30.792384+0000","last_undegraded":"2023-10-06T03:52:30.792384+0000","last_fullsized":"2023-10-06T03:52:30.792384+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:40.375611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.122758+0000","last_change":"2023-10-06T03:52:30.115911+0000","last_active":"2023-10-06T03:52:30.122758+0000","last_peered":"2023-10-06T03:52:30.122758+0000","last_clean":"2023-10-06T03:52:30.122758+0000","last_became_active":"2023-10-06T03:52:30.115634+0000","last_became_peered":"2023-10-06T03:52:30.115634+0000","last_unstale":"2023-10-06T03:52:30.122758+0000","last_undegraded":"2023-10-06T03:52:30.122758+0000","last_fullsized":"2023-10-06T03:52:30.122758+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:14.292118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.774608+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T03:52:30.774608+0000","last_peered":"2023-10-06T03:52:30.774608+0000","last_clean":"2023-10-06T03:52:30.774608+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T03:52:30.774608+0000","last_undegraded":"2023-10-06T03:52:30.774608+0000","last_fullsized":"2023-10-06T03:52:30.774608+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.793138+0000","last_change":"2023-10-06T03:52:30.788624+0000","last_active":"2023-10-06T03:52:30.793138+0000","last_peered":"2023-10-06T03:52:30.793138+0000","last_clean":"2023-10-06T03:52:30.793138+0000","last_became_active":"2023-10-06T03:52:30.788334+0000","last_became_peered":"2023-10-06T03:52:30.788334+0000","last_unstale":"2023-10-06T03:52:30.793138+0000","last_undegraded":"2023-10-06T03:52:30.793138+0000","last_fullsized":"2023-10-06T03:52:30.793138+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:30.096499+0000","last_clean_scrub_stamp":"2023-10-06T03:52:30.096499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:06.430173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.774268+0000","last_change":"2023-10-06T03:52:30.125314+0000","last_active":"2023-10-06T03:52:30.774268+0000","last_peered":"2023-10-06T03:52:30.774268+0000","last_clean":"2023-10-06T03:52:30.774268+0000","last_became_active":"2023-10-06T03:52:30.125015+0000","last_became_peered":"2023-10-06T03:52:30.125015+0000","last_unstale":"2023-10-06T03:52:30.774268+0000","last_undegraded":"2023-10-06T03:52:30.774268+0000","last_fullsized":"2023-10-06T03:52:30.774268+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:52:29.085734+0000","last_clean_scrub_stamp":"2023-10-06T03:52:29.085734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:31:33.395555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":130,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-06T03:52:30.774354+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T03:52:30.774354+0000","last_peered":"2023-10-06T03:52:30.774354+0000","last_clean":"2023-10-06T03:52:30.774354+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T03:52:30.774354+0000","last_undegraded":"2023-10-06T03:52:30.774354+0000","last_fullsized":"2023-10-06T03:52:30.774354+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575074,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311816,"kb_used_data":2240,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060024,"statfs":{"total":96636764160,"available":96317464576,"internally_reserved":0,"allocated":2293760,"data_stored":1462363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:51:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.69299999999999995},"min":{"1min":0.41899999999999998,"5min":0.377,"15min":0.27100000000000002},"max":{"1min":0.89200000000000002,"5min":0.998,"15min":1.0409999999999999},"last":0.63},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.66900000000000004,"15min":0.70099999999999996},"min":{"1min":0.45000000000000001,"5min":0.42599999999999999,"15min":0.36499999999999999},"max":{"1min":0.91700000000000004,"5min":1.02,"15min":1.02},"last":0.71499999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:51:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70699999999999996,"15min":0.74299999999999999},"min":{"1min":0.46899999999999997,"5min":0.41199999999999998,"15min":0.318},"max":{"1min":0.96299999999999997,"5min":1.038,"15min":1.0640000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73699999999999999,"15min":0.76200000000000001},"min":{"1min":0.57199999999999995,"5min":0.49199999999999999,"15min":0.45900000000000002},"max":{"1min":0.90200000000000002,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:51:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67800000000000005,"15min":0.69799999999999995},"min":{"1min":0.497,"5min":0.41199999999999998,"15min":0.39100000000000001},"max":{"1min":0.90500000000000003,"5min":0.97499999999999998,"15min":1.044},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71299999999999997,"15min":0.73699999999999999},"min":{"1min":0.47199999999999998,"5min":0.44,"15min":0.44},"max":{"1min":0.92700000000000005,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Oct 6 03:51:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70699999999999996,"15min":0.72299999999999998},"min":{"1min":0.53300000000000003,"5min":0.41799999999999998,"15min":0.39500000000000002},"max":{"1min":0.89000000000000001,"5min":1.0089999999999999,"15min":1.016},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.74099999999999999},"min":{"1min":0.55000000000000004,"5min":0.501,"15min":0.45600000000000002},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":1.0409999999999999},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:51:50 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61099999999999999,"15min":0.59299999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63600000000000001,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.39400000000000002},"max":{"1min":0.93500000000000005,"5min":0.94699999999999995,"15min":0.97599999999999998},"last":0.60099999999999998}]},{"osd":5,"last update":"Fri Oct 6 03:51:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.65800000000000003,"15min":0.63800000000000001},"min":{"1min":0.42699999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.82299999999999995,"5min":1.022,"15min":1.069},"last":0.439},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68899999999999995,"15min":0.68000000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.95699999999999996,"5min":1.0640000000000001,"15min":1.105},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Oct 6 03:51:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.74199999999999999,"15min":0.72599999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.436},"max":{"1min":1.0069999999999999,"5min":1.1220000000000001,"15min":1.181},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.47999999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.083,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575075,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307132,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064708,"statfs":{"total":96636764160,"available":96322260992,"internally_reserved":0,"allocated":1691648,"data_stored":872779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.69199999999999995,"15min":0.70299999999999996},"min":{"1min":0.442,"5min":0.39400000000000002,"15min":0.313},"max":{"1min":0.93100000000000005,"5min":1.288,"15min":1.288},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69899999999999995,"15min":0.71699999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93600000000000005,"5min":0.998,"15min":1.0069999999999999},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Oct 6 03:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71199999999999997,"15min":0.72499999999999998},"min":{"1min":0.50700000000000001,"5min":0.41599999999999998,"15min":0.33800000000000002},"max":{"1min":0.88200000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.69899999999999995,"15min":0.71299999999999997},"min":{"1min":0.48499999999999999,"5min":0.46400000000000002,"15min":0.35899999999999999},"max":{"1min":0.84599999999999997,"5min":1.427,"15min":1.427},"last":0.61699999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71999999999999997,"15min":0.73699999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.34699999999999998},"max":{"1min":0.91100000000000003,"5min":1.179,"15min":1.179},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.71099999999999997,"15min":0.73299999999999998},"min":{"1min":0.505,"5min":0.42099999999999999,"15min":0.39600000000000002},"max":{"1min":0.91100000000000003,"5min":1.0209999999999999,"15min":1.0409999999999999},"last":0.55900000000000005}]},{"osd":3,"last update":"Fri Oct 6 03:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72899999999999998,"15min":0.748},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.45500000000000002},"max":{"1min":0.95399999999999996,"5min":1.024,"15min":1.024},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.73999999999999999,"15min":0.755},"min":{"1min":0.53000000000000003,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85999999999999999,"5min":1.097,"15min":1.1639999999999999},"last":0.51200000000000001}]},{"osd":4,"last update":"Fri Oct 6 03:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.58199999999999996},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":1.462,"15min":1.462},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64200000000000002,"15min":0.63700000000000001},"min":{"1min":0.41699999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95199999999999996,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.46700000000000003}]},{"osd":5,"last update":"Fri Oct 6 03:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70699999999999996,"15min":0.67400000000000004},"min":{"1min":0.47899999999999998,"5min":0.438,"15min":0.40000000000000002},"max":{"1min":0.91600000000000004,"5min":1.0109999999999999,"15min":1.0489999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73599999999999999,"15min":0.71699999999999997},"min":{"1min":0.52000000000000002,"5min":0.42799999999999999,"15min":0.41199999999999998},"max":{"1min":0.93799999999999994,"5min":1.508,"15min":1.508},"last":0.64600000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:52:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73099999999999998,"15min":0.69699999999999995},"min":{"1min":0.46100000000000002,"5min":0.41499999999999998,"15min":0.41199999999999998},"max":{"1min":0.96299999999999997,"5min":1.21,"15min":1.21},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73699999999999999,"15min":0.71199999999999997},"min":{"1min":0.435,"5min":0.40000000000000002,"15min":0.36099999999999999},"max":{"1min":0.92600000000000005,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.628}]}]},{"osd":1,"up_from":13,"seq":55834575074,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307708,"kb_used_data":2228,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064132,"statfs":{"total":96636764160,"available":96321671168,"internally_reserved":0,"allocated":2281472,"data_stored":1462593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:51:46 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.48699999999999999},"min":{"1min":0.34899999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.73399999999999999,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.45400000000000001,"5min":0.36399999999999999,"15min":0.34599999999999997},"max":{"1min":0.70999999999999996,"5min":1.024,"15min":1.1319999999999999},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:51:46 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45900000000000002,"15min":0.48799999999999999},"min":{"1min":0.34599999999999997,"5min":0.32700000000000001,"15min":0.309},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.84799999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.40600000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78300000000000003,"5min":0.80500000000000005,"15min":0.94099999999999995},"last":0.40100000000000002}]},{"osd":3,"last update":"Fri Oct 6 03:51:46 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.45300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.83099999999999996,"5min":0.86599999999999999,"15min":0.91700000000000004},"last":0.498},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53000000000000003,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.34200000000000003,"15min":0.32500000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.90400000000000003},"last":0.52300000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:51:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75,"15min":0.76000000000000001},"min":{"1min":0.56200000000000006,"5min":0.496,"15min":0.41499999999999998},"max":{"1min":1,"5min":1.044,"15min":1.1699999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.79700000000000004},"min":{"1min":0.55800000000000005,"5min":0.52500000000000002,"15min":0.439},"max":{"1min":1,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.94399999999999995}]},{"osd":5,"last update":"Fri Oct 6 03:51:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.58099999999999996,"5min":0.53800000000000003,"15min":0.45400000000000001},"max":{"1min":0.95899999999999996,"5min":1.294,"15min":1.294},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.45100000000000001},"max":{"1min":1.0389999999999999,"5min":1.224,"15min":1.224},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Oct 6 03:51:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.79800000000000004,"15min":0.80500000000000005},"min":{"1min":0.58999999999999997,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.98599999999999999,"5min":1.26,"15min":1.26},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78800000000000003,"15min":0.81599999999999995},"min":{"1min":0.60599999999999998,"5min":0.52800000000000002,"15min":0.52100000000000002},"max":{"1min":1.0660000000000001,"5min":1.143,"15min":1.224},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Oct 6 03:51:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.82999999999999996,"15min":0.83799999999999997},"min":{"1min":0.53900000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0369999999999999,"5min":1.1519999999999999,"15min":1.272},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.82499999999999996,"15min":0.83199999999999996},"min":{"1min":0.59299999999999997,"5min":0.52000000000000002,"15min":0.51200000000000001},"max":{"1min":1.0680000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.92900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575076,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307128,"kb_used_data":1648,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064712,"statfs":{"total":96636764160,"available":96322265088,"internally_reserved":0,"allocated":1687552,"data_stored":871269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 03:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47699999999999998,"15min":0.48299999999999998},"min":{"1min":0.30199999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.70499999999999996,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.50700000000000001,"15min":0.51600000000000001},"min":{"1min":0.35099999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.69999999999999996,"5min":0.70999999999999996,"15min":0.83599999999999997},"last":0.40999999999999998}]},{"osd":2,"last update":"Fri Oct 6 03:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52800000000000002,"15min":0.54100000000000004},"min":{"1min":0.38700000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73599999999999999,"5min":0.80100000000000005,"15min":1.1299999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53600000000000003,"15min":0.54600000000000004},"min":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.31900000000000001},"max":{"1min":0.76600000000000001,"5min":0.83399999999999996,"15min":0.89400000000000002},"last":0.46999999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52200000000000002,"15min":0.51900000000000002},"min":{"1min":0.33200000000000002,"5min":0.318,"15min":0.30499999999999999},"max":{"1min":0.80000000000000004,"5min":0.83699999999999997,"15min":0.93200000000000005},"last":0.432},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58899999999999997,"15min":0.60999999999999999},"min":{"1min":0.46600000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.78700000000000003,"5min":0.91800000000000004,"15min":1.0580000000000001},"last":0.497}]},{"osd":4,"last update":"Fri Oct 6 03:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80700000000000005,"15min":0.82499999999999996},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.49399999999999999},"max":{"1min":1.109,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.80300000000000005,"15min":0.81899999999999995},"min":{"1min":0.56000000000000005,"5min":0.51500000000000001,"15min":0.45500000000000002},"max":{"1min":1.079,"5min":1.306,"15min":1.3400000000000001},"last":0.88200000000000001}]},{"osd":5,"last update":"Fri Oct 6 03:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.79800000000000004,"15min":0.81100000000000005},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.042,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78500000000000003,"15min":0.80400000000000005},"min":{"1min":0.54100000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0629999999999999,"5min":1.194,"15min":1.6559999999999999},"last":0.80500000000000005}]},{"osd":6,"last update":"Fri Oct 6 03:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.77900000000000003,"15min":0.80300000000000005},"min":{"1min":0.625,"5min":0.51500000000000001,"15min":0.48099999999999998},"max":{"1min":0.97299999999999998,"5min":1.1699999999999999,"15min":1.726},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81299999999999994,"15min":0.83699999999999997},"min":{"1min":0.57299999999999995,"5min":0.52900000000000003,"15min":0.52400000000000002},"max":{"1min":1.2010000000000001,"5min":1.212,"15min":1.843},"last":0.86299999999999999}]},{"osd":7,"last update":"Fri Oct 6 03:51:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.81699999999999995,"15min":0.82399999999999995},"min":{"1min":0.61299999999999999,"5min":0.502,"15min":0.496},"max":{"1min":1.1799999999999999,"5min":1.302,"15min":1.302},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.495},"max":{"1min":1.004,"5min":1.2450000000000001,"15min":1.7649999999999999},"last":0.91100000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575075,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311220,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060620,"statfs":{"total":96636764160,"available":96318074880,"internally_reserved":0,"allocated":1683456,"data_stored":871247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.45300000000000001,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.54400000000000004,"5min":0.74199999999999999,"15min":0.80100000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.52200000000000002,"15min":0.53100000000000003},"min":{"1min":0.34100000000000003,"5min":0.32400000000000001,"15min":0.311},"max":{"1min":0.72099999999999997,"5min":0.86699999999999999,"15min":0.89300000000000002},"last":0.51100000000000001}]},{"osd":1,"last update":"Fri Oct 6 03:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48399999999999999,"15min":0.48899999999999999},"min":{"1min":0.311,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.72799999999999998,"5min":0.81200000000000006,"15min":0.90000000000000002},"last":0.495},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54700000000000004,"15min":0.55400000000000005},"min":{"1min":0.42099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71399999999999997,"5min":0.89700000000000002,"15min":0.94899999999999995},"last":0.57499999999999996}]},{"osd":3,"last update":"Fri Oct 6 03:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55700000000000005,"15min":0.58899999999999997},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.69799999999999995,"5min":0.79000000000000004,"15min":1.78},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.56000000000000005,"15min":0.57399999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.33300000000000002},"max":{"1min":0.65800000000000003,"5min":0.84699999999999998,"15min":0.96499999999999997},"last":0.53400000000000003}]},{"osd":4,"last update":"Fri Oct 6 03:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.82199999999999995},"min":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.51400000000000001},"max":{"1min":1.238,"5min":1.238,"15min":1.5620000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.82999999999999996,"15min":0.83699999999999997},"min":{"1min":0.62,"5min":0.53100000000000003,"15min":0.46500000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.83499999999999996,"15min":0.83799999999999997},"min":{"1min":0.623,"5min":0.54900000000000004,"15min":0.503},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81200000000000006,"15min":0.84599999999999997},"min":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.855},"last":0.82199999999999995}]},{"osd":6,"last update":"Fri Oct 6 03:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.78800000000000003,"15min":0.82699999999999996},"min":{"1min":0.54800000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.907},"last":0.874},{"interface":"front","average":{"1min":0.745,"5min":0.80900000000000005,"15min":0.82199999999999995},"min":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0609999999999999,"5min":1.1080000000000001,"15min":1.4750000000000001},"last":0.80400000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:51:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.82299999999999995,"15min":0.84399999999999997},"min":{"1min":0.56799999999999995,"5min":0.54700000000000004,"15min":0.495},"max":{"1min":1.1559999999999999,"5min":1.2350000000000001,"15min":1.4990000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81999999999999995,"15min":0.86299999999999999},"min":{"1min":0.57399999999999995,"5min":0.54800000000000004,"15min":0.53100000000000003},"max":{"1min":1.194,"5min":1.194,"15min":1.9450000000000001},"last":0.747}]}]},{"osd":3,"up_from":13,"seq":55834575075,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307156,"kb_used_data":1676,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064684,"statfs":{"total":96636764160,"available":96322236416,"internally_reserved":0,"allocated":1716224,"data_stored":880828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27000000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.47999999999999998,"15min":0.495},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.29799999999999999},"max":{"1min":0.71599999999999997,"5min":0.79200000000000004,"15min":1.04},"last":0.33900000000000002}]},{"osd":1,"last update":"Fri Oct 6 03:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.49099999999999999,"15min":0.48399999999999999},"min":{"1min":0.37,"5min":0.32200000000000001,"15min":0.314},"max":{"1min":0.79200000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54100000000000004,"15min":0.53900000000000003},"min":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70299999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Oct 6 03:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.56499999999999995},"min":{"1min":0.44700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69299999999999995,"5min":0.80700000000000005,"15min":1.4199999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58499999999999996,"15min":0.59899999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.87},"last":0.46100000000000002}]},{"osd":4,"last update":"Fri Oct 6 03:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.77100000000000002,"15min":0.77900000000000003},"min":{"1min":0.627,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.3340000000000001},"last":0.626},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.79900000000000004,"15min":0.82199999999999995},"min":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.2889999999999999},"last":0.70499999999999996}]},{"osd":5,"last update":"Fri Oct 6 03:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.76000000000000001,"15min":0.78600000000000003},"min":{"1min":0.53300000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.256},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76600000000000001,"15min":0.77900000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.49099999999999999},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.2030000000000001},"last":0.66400000000000003}]},{"osd":6,"last update":"Fri Oct 6 03:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.77400000000000002,"15min":0.80700000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.3029999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.81000000000000005,"15min":0.83999999999999997},"min":{"1min":0.61499999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0429999999999999,"5min":1.075,"15min":1.2849999999999999},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Oct 6 03:51:32 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78200000000000003,"15min":0.80600000000000005},"min":{"1min":0.53800000000000003,"5min":0.498,"15min":0.498},"max":{"1min":1.222,"5min":1.222,"15min":1.2370000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.77700000000000002,"15min":0.79900000000000004},"min":{"1min":0.56599999999999995,"5min":0.48399999999999999,"15min":0.46300000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.2330000000000001},"last":0.745}]}]},{"osd":4,"up_from":13,"seq":55834575075,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307140,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064700,"statfs":{"total":96636764160,"available":96322252800,"internally_reserved":0,"allocated":1699840,"data_stored":873761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70199999999999996,"15min":0.71499999999999997},"min":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.316},"max":{"1min":0.98499999999999999,"5min":0.996,"15min":1.0189999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68000000000000005,"15min":0.69899999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.316},"max":{"1min":0.91200000000000003,"5min":0.96499999999999997,"15min":1.01},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Oct 6 03:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68899999999999995,"15min":0.71999999999999997},"min":{"1min":0.48599999999999999,"5min":0.42999999999999999,"15min":0.33200000000000002},"max":{"1min":0.92300000000000004,"5min":1.0600000000000001,"15min":1.081},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73499999999999999,"15min":0.73699999999999999},"min":{"1min":0.45500000000000002,"5min":0.42799999999999999,"15min":0.41299999999999998},"max":{"1min":1.02,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.72099999999999997}]},{"osd":2,"last update":"Fri Oct 6 03:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71799999999999997,"15min":0.73199999999999998},"min":{"1min":0.54100000000000004,"5min":0.47599999999999998,"15min":0.377},"max":{"1min":1.004,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.73699999999999999,"15min":0.746},"min":{"1min":0.46999999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94599999999999995,"5min":1.0169999999999999,"15min":1.022},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Oct 6 03:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90600000000000003,"5min":1.0189999999999999,"15min":1.0409999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.53100000000000003,"5min":0.51200000000000001,"15min":0.46500000000000002},"max":{"1min":0.96799999999999997,"5min":1.1040000000000001,"15min":1.1850000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Fri Oct 6 03:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63600000000000001,"15min":0.625},"min":{"1min":0.40899999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.048,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.63500000000000001,"15min":0.627},"min":{"1min":0.434,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85899999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.504}]},{"osd":6,"last update":"Fri Oct 6 03:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.64500000000000002},"min":{"1min":0.45800000000000002,"5min":0.434,"15min":0.42599999999999999},"max":{"1min":0.85399999999999998,"5min":1.1839999999999999,"15min":1.2310000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.69899999999999995,"15min":0.68500000000000005},"min":{"1min":0.432,"5min":0.39900000000000002,"15min":0.38100000000000001},"max":{"1min":0.94399999999999995,"5min":1.083,"15min":1.083},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Oct 6 03:51:52 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68700000000000006,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90800000000000003,"5min":1.046,"15min":1.046},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.72299999999999998,"15min":0.71399999999999997},"min":{"1min":0.56699999999999995,"5min":0.47499999999999998,"15min":0.41299999999999998},"max":{"1min":0.92900000000000005,"5min":1.214,"15min":1.2889999999999999},"last":0.78500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575075,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303036,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068804,"statfs":{"total":96636764160,"available":96326455296,"internally_reserved":0,"allocated":1691648,"data_stored":871407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 03:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69799999999999995,"15min":0.70599999999999996},"min":{"1min":0.434,"5min":0.39200000000000002,"15min":0.35299999999999998},"max":{"1min":0.94699999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.71199999999999997,"15min":0.71599999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.32600000000000001},"max":{"1min":0.872,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.80400000000000005}]},{"osd":1,"last update":"Fri Oct 6 03:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.77000000000000002,"15min":0.73099999999999998},"min":{"1min":0.50800000000000001,"5min":0.46600000000000003,"15min":0.40000000000000002},"max":{"1min":0.875,"5min":5.8239999999999998,"15min":5.8239999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.76800000000000002,"15min":0.73899999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.32400000000000001},"max":{"1min":0.90700000000000003,"5min":5.8680000000000003,"15min":5.8680000000000003},"last":0.81699999999999995}]},{"osd":2,"last update":"Fri Oct 6 03:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.49099999999999999,"5min":0.46400000000000002,"15min":0.40200000000000002},"max":{"1min":0.97899999999999998,"5min":1.155,"15min":1.155},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76000000000000001,"15min":0.77100000000000002},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.89500000000000002,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.875}]},{"osd":3,"last update":"Fri Oct 6 03:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.48899999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.84499999999999997,"5min":1.091,"15min":1.6419999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.749,"15min":0.748},"min":{"1min":0.57999999999999996,"5min":0.50600000000000001,"15min":0.47799999999999998},"max":{"1min":0.92800000000000005,"5min":1.238,"15min":1.238},"last":0.83399999999999996}]},{"osd":4,"last update":"Fri Oct 6 03:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65100000000000002,"15min":0.61699999999999999},"min":{"1min":0.372,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88200000000000001,"5min":1.04,"15min":1.04},"last":0.498},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.39600000000000002,"5min":0.371,"15min":0.36799999999999999},"max":{"1min":0.90200000000000002,"5min":1.147,"15min":1.147},"last":0.45900000000000002}]},{"osd":6,"last update":"Fri Oct 6 03:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.65800000000000003,"15min":0.64200000000000002},"min":{"1min":0.47199999999999998,"5min":0.40200000000000002,"15min":0.34999999999999998},"max":{"1min":0.91100000000000003,"5min":1.0449999999999999,"15min":1.591},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71099999999999997,"15min":0.68899999999999995},"min":{"1min":0.45000000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97199999999999998,"5min":1.196,"15min":1.6140000000000001},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Oct 6 03:51:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.46100000000000002,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.83999999999999997,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72299999999999998,"15min":0.71199999999999997},"min":{"1min":0.47099999999999997,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95399999999999996,"5min":1.0880000000000001,"15min":1.278},"last":0.48699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T03:52:36.173 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T03:52:36.178 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T03:52:36.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.370+0000 7faddef9d700 1 Processor -- start 2023-10-06T03:52:36.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.370+0000 7faddef9d700 1 -- start start 2023-10-06T03:52:36.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.372+0000 7faddef9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadd80caee0 0x7fadd80ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:36.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.372+0000 7faddef9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fadd8156780 0x7fadd80ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:36.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.372+0000 7faddef9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd80cbe50 con 0x7fadd81563b0 2023-10-06T03:52:36.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.372+0000 7faddef9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd80cb370 con 0x7fadd8156780 2023-10-06T03:52:36.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.372+0000 7faddef9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80cb650 con 0x7fadd80caee0 2023-10-06T03:52:36.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.372+0000 7fadde79c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fadd8156780 0x7fadd80ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:36.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.372+0000 7fadde79c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fadd8156780 0x7fadd80ae7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49912/0 (socket says 172.21.15.55:49912) 2023-10-06T03:52:36.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.372+0000 7fadde79c700 1 -- 172.21.15.55:0/1844940713 learned_addr learned my addr 172.21.15.55:0/1844940713 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:36.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.372+0000 7faddd79a700 1 --2- 172.21.15.55:0/1844940713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadd80caee0 0x7fadd80ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:36.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.372+0000 7fadde79c700 1 -- 172.21.15.55:0/1844940713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadd80caee0 msgr2=0x7fadd80ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:36.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.372+0000 7fadde79c700 1 --2- 172.21.15.55:0/1844940713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadd80caee0 0x7fadd80ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:36.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.372+0000 7fadde79c700 1 -- 172.21.15.55:0/1844940713 >> v1:172.21.15.163:6789/0 conn(0x7fadd81563b0 legacy=0x7fadd80ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:36.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.372+0000 7fadde79c700 1 -- 172.21.15.55:0/1844940713 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd80c6860 con 0x7fadd8156780 2023-10-06T03:52:36.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.372+0000 7faddd79a700 1 --2- 172.21.15.55:0/1844940713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadd80caee0 0x7fadd80ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:36.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.373+0000 7fadde79c700 1 --2- 172.21.15.55:0/1844940713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd8156780 0x7fadd80ae7f0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fadd000d1a0 tx=0x7fadd000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80219cb63ceabd95 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:36.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.373+0000 7faddcf99700 1 -- 172.21.15.55:0/1844940713 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd0010070 con 0x7fadd8156780 2023-10-06T03:52:36.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.373+0000 7faddcf99700 1 -- 172.21.15.55:0/1844940713 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd000de80 con 0x7fadd8156780 2023-10-06T03:52:36.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.373+0000 7faddcf99700 1 -- 172.21.15.55:0/1844940713 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000b320 con 0x7fadd8156780 2023-10-06T03:52:36.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.373+0000 7faddef9d700 1 -- 172.21.15.55:0/1844940713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd8156780 msgr2=0x7fadd80ae7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:36.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.373+0000 7faddef9d700 1 --2- 172.21.15.55:0/1844940713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd8156780 0x7fadd80ae7f0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fadd000d1a0 tx=0x7fadd000ba50 comp rx=0 tx=0).stop 2023-10-06T03:52:36.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.374+0000 7faddef9d700 1 -- 172.21.15.55:0/1844940713 shutdown_connections 2023-10-06T03:52:36.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.374+0000 7faddef9d700 1 --2- 172.21.15.55:0/1844940713 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd8156780 0x7fadd80ae7f0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:36.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.374+0000 7faddef9d700 1 --2- 172.21.15.55:0/1844940713 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadd80caee0 0x7fadd80ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:36.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.374+0000 7faddef9d700 1 -- 172.21.15.55:0/1844940713 >> 172.21.15.55:0/1844940713 conn(0x7fadd8068e20 msgr2=0x7fadd80cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:36.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.374+0000 7faddef9d700 1 -- 172.21.15.55:0/1844940713 shutdown_connections 2023-10-06T03:52:36.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.374+0000 7faddef9d700 1 -- 172.21.15.55:0/1844940713 wait complete. 2023-10-06T03:52:36.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.375+0000 7faddef9d700 1 Processor -- start 2023-10-06T03:52:36.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.375+0000 7faddef9d700 1 -- start start 2023-10-06T03:52:36.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.376+0000 7faddef9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd80caee0 0x7fadd80b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:36.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.376+0000 7faddef9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadd81563b0 0x7fadd8149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:36.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.376+0000 7faddef9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadd8156780 0x7fadd80aae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:36.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.376+0000 7faddef9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80a9dd0 con 0x7fadd80caee0 2023-10-06T03:52:36.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.376+0000 7faddef9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80aa0b0 con 0x7fadd8156780 2023-10-06T03:52:36.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.376+0000 7faddef9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80aa390 con 0x7fadd81563b0 2023-10-06T03:52:36.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.376+0000 7fadddf9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd80caee0 0x7fadd80b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:36.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.377+0000 7fadddf9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd80caee0 0x7fadd80b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49922/0 (socket says 172.21.15.55:49922) 2023-10-06T03:52:36.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.377+0000 7faddd79a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadd81563b0 0x7fadd8149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:36.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.377+0000 7fadddf9b700 1 -- 172.21.15.55:0/689441263 learned_addr learned my addr 172.21.15.55:0/689441263 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:36.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.377+0000 7fadde79c700 1 --2- 172.21.15.55:0/689441263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadd8156780 0x7fadd80aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:36.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.377+0000 7fadddf9b700 1 -- 172.21.15.55:0/689441263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadd81563b0 msgr2=0x7fadd8149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:36.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.377+0000 7fadddf9b700 1 --2- 172.21.15.55:0/689441263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadd81563b0 0x7fadd8149d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:36.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.377+0000 7fadddf9b700 1 -- 172.21.15.55:0/689441263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadd8156780 msgr2=0x7fadd80aae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:36.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.377+0000 7fadddf9b700 1 --2- 172.21.15.55:0/689441263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadd8156780 0x7fadd80aae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.377+0000 7fadddf9b700 1 -- 172.21.15.55:0/689441263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd000bcf0 con 0x7fadd80caee0 2023-10-06T03:52:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.377+0000 7fadddf9b700 1 --2- 172.21.15.55:0/689441263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd80caee0 0x7fadd80b1890 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fadd400c5d0 tx=0x7fadd4007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.377+0000 7fadcdffb700 1 -- 172.21.15.55:0/689441263 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd4011590 con 0x7fadd80caee0 2023-10-06T03:52:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.378+0000 7fadcdffb700 1 -- 172.21.15.55:0/689441263 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd4011710 con 0x7fadd80caee0 2023-10-06T03:52:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.378+0000 7faddef9d700 1 -- 172.21.15.55:0/689441263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd80aa670 con 0x7fadd80caee0 2023-10-06T03:52:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.378+0000 7fadcdffb700 1 -- 172.21.15.55:0/689441263 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd4011ad0 con 0x7fadd80caee0 2023-10-06T03:52:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.378+0000 7faddef9d700 1 -- 172.21.15.55:0/689441263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd814a240 con 0x7fadd80caee0 2023-10-06T03:52:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.379+0000 7faddef9d700 1 -- 172.21.15.55:0/689441263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd80cbc90 con 0x7fadd80caee0 2023-10-06T03:52:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.380+0000 7fadcdffb700 1 -- 172.21.15.55:0/689441263 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fadd401c460 con 0x7fadd80caee0 2023-10-06T03:52:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.380+0000 7fadcdffb700 1 --2- 172.21.15.55:0/689441263 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fadbc067050 0x7fadbc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:36.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.380+0000 7fadcdffb700 1 -- 172.21.15.55:0/689441263 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fadd40846b0 con 0x7fadd80caee0 2023-10-06T03:52:36.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.381+0000 7faddd79a700 1 --2- 172.21.15.55:0/689441263 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fadbc067050 0x7fadbc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:36.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.381+0000 7faddd79a700 1 --2- 172.21.15.55:0/689441263 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fadbc067050 0x7fadbc069500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fadd80af420 tx=0x7fadc8006f90 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:36.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.384+0000 7fadcdffb700 1 -- 172.21.15.55:0/689441263 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fadd4051480 con 0x7fadd80caee0 2023-10-06T03:52:36.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.664+0000 7faddef9d700 1 -- 172.21.15.55:0/689441263 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fadd804c3c0 con 0x7fadd80caee0 2023-10-06T03:52:36.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.672+0000 7fadcdffb700 1 -- 172.21.15.55:0/689441263 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7fadd4054aa0 con 0x7fadd80caee0 2023-10-06T03:52:36.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.674+0000 7faddef9d700 1 -- 172.21.15.55:0/689441263 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fadbc067050 msgr2=0x7fadbc069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:36.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.674+0000 7faddef9d700 1 --2- 172.21.15.55:0/689441263 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fadbc067050 0x7fadbc069500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fadd80af420 tx=0x7fadc8006f90 comp rx=0 tx=0).stop 2023-10-06T03:52:36.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.674+0000 7faddef9d700 1 -- 172.21.15.55:0/689441263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd80caee0 msgr2=0x7fadd80b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:36.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.674+0000 7faddef9d700 1 --2- 172.21.15.55:0/689441263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd80caee0 0x7fadd80b1890 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fadd400c5d0 tx=0x7fadd4007f60 comp rx=0 tx=0).stop 2023-10-06T03:52:36.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.675+0000 7faddef9d700 1 -- 172.21.15.55:0/689441263 shutdown_connections 2023-10-06T03:52:36.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.675+0000 7faddef9d700 1 --2- 172.21.15.55:0/689441263 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fadbc067050 0x7fadbc069500 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:36.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.675+0000 7faddef9d700 1 --2- 172.21.15.55:0/689441263 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadd8156780 0x7fadd80aae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:36.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.675+0000 7faddef9d700 1 --2- 172.21.15.55:0/689441263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadd80caee0 0x7fadd80b1890 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:36.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.675+0000 7faddef9d700 1 --2- 172.21.15.55:0/689441263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadd81563b0 0x7fadd8149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:36.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.675+0000 7faddef9d700 1 -- 172.21.15.55:0/689441263 >> 172.21.15.55:0/689441263 conn(0x7fadd8068e20 msgr2=0x7fadd81554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:36.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.675+0000 7faddef9d700 1 -- 172.21.15.55:0/689441263 shutdown_connections 2023-10-06T03:52:36.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:36.675+0000 7faddef9d700 1 -- 172.21.15.55:0/689441263 wait complete. 2023-10-06T03:52:36.678 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2023-10-06T03:52:36.694 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T03:52:37.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.121+0000 7fdfdf59e700 1 Processor -- start 2023-10-06T03:52:37.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.122+0000 7fdfdf59e700 1 -- start start 2023-10-06T03:52:37.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.123+0000 7fdfdf59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdfe0150180 0x7fdfe0157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:37.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.123+0000 7fdfdf59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0150550 0x7fdfe00c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:37.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.123+0000 7fdfdf59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe006e2a0 con 0x7fdfe00c7240 2023-10-06T03:52:37.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.123+0000 7fdfdf59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe00c76c0 con 0x7fdfe0150180 2023-10-06T03:52:37.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.123+0000 7fdfdf59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe006b2e0 con 0x7fdfe0150550 2023-10-06T03:52:37.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.123+0000 7fdfde59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdfe0150180 0x7fdfe0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:37.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.123+0000 7fdfde59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdfe0150180 0x7fdfe0157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49936/0 (socket says 172.21.15.55:49936) 2023-10-06T03:52:37.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.123+0000 7fdfde59c700 1 -- 172.21.15.55:0/164973481 learned_addr learned my addr 172.21.15.55:0/164973481 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:37.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.124+0000 7fdfddd9b700 1 --2- 172.21.15.55:0/164973481 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0150550 0x7fdfe00c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:37.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.124+0000 7fdfde59c700 1 -- 172.21.15.55:0/164973481 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0150550 msgr2=0x7fdfe00c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:37.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.124+0000 7fdfde59c700 1 --2- 172.21.15.55:0/164973481 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0150550 0x7fdfe00c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:37.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.124+0000 7fdfde59c700 1 -- 172.21.15.55:0/164973481 >> v1:172.21.15.163:6789/0 conn(0x7fdfe00c7240 legacy=0x7fdfe0069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:37.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.124+0000 7fdfde59c700 1 -- 172.21.15.55:0/164973481 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe00c78e0 con 0x7fdfe0150180 2023-10-06T03:52:37.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.124+0000 7fdfddd9b700 1 --2- 172.21.15.55:0/164973481 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0150550 0x7fdfe00c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:37.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.124+0000 7fdfde59c700 1 --2- 172.21.15.55:0/164973481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe0150180 0x7fdfe0157770 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00099e0 tx=0x7fdfd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d5875661fc1e7d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:37.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.124+0000 7fdfdd59a700 1 -- 172.21.15.55:0/164973481 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd000b7e0 con 0x7fdfe0150180 2023-10-06T03:52:37.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.124+0000 7fdfdd59a700 1 -- 172.21.15.55:0/164973481 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd000b940 con 0x7fdfe0150180 2023-10-06T03:52:37.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.124+0000 7fdfdd59a700 1 -- 172.21.15.55:0/164973481 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd000bce0 con 0x7fdfe0150180 2023-10-06T03:52:37.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.125+0000 7fdfdf59e700 1 -- 172.21.15.55:0/164973481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe0150180 msgr2=0x7fdfe0157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:37.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.125+0000 7fdfdf59e700 1 --2- 172.21.15.55:0/164973481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe0150180 0x7fdfe0157770 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00099e0 tx=0x7fdfd0009250 comp rx=0 tx=0).stop 2023-10-06T03:52:37.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.125+0000 7fdfdf59e700 1 -- 172.21.15.55:0/164973481 shutdown_connections 2023-10-06T03:52:37.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.125+0000 7fdfdf59e700 1 --2- 172.21.15.55:0/164973481 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe0150180 0x7fdfe0157770 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:37.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.125+0000 7fdfdf59e700 1 --2- 172.21.15.55:0/164973481 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0150550 0x7fdfe00c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:37.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.125+0000 7fdfdf59e700 1 -- 172.21.15.55:0/164973481 >> 172.21.15.55:0/164973481 conn(0x7fdfe00cc1f0 msgr2=0x7fdfe00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:37.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.125+0000 7fdfdf59e700 1 -- 172.21.15.55:0/164973481 shutdown_connections 2023-10-06T03:52:37.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.125+0000 7fdfdf59e700 1 -- 172.21.15.55:0/164973481 wait complete. 2023-10-06T03:52:37.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.126+0000 7fdfdf59e700 1 Processor -- start 2023-10-06T03:52:37.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.126+0000 7fdfdf59e700 1 -- start start 2023-10-06T03:52:37.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.127+0000 7fdfdf59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfe00c7240 0x7fdfe00abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:37.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.127+0000 7fdfdf59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0150180 0x7fdfe00caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:37.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.127+0000 7fdfdf59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe0150550 0x7fdfe0143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:37.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.127+0000 7fdfdf59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe00c32a0 con 0x7fdfe0150550 2023-10-06T03:52:37.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.127+0000 7fdfdf59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe00ad1e0 con 0x7fdfe00c7240 2023-10-06T03:52:37.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.127+0000 7fdfdf59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe0144580 con 0x7fdfe0150180 2023-10-06T03:52:37.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.128+0000 7fdfded9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe0150550 0x7fdfe0143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:37.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.128+0000 7fdfde59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfe00c7240 0x7fdfe00abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:37.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.128+0000 7fdfddd9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0150180 0x7fdfe00caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:37.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.128+0000 7fdfded9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe0150550 0x7fdfe0143710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49952/0 (socket says 172.21.15.55:49952) 2023-10-06T03:52:37.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.128+0000 7fdfde59c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfe00c7240 0x7fdfe00abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:55354/0 (socket says 172.21.15.55:55354) 2023-10-06T03:52:37.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.128+0000 7fdfded9d700 1 -- 172.21.15.55:0/3240629865 learned_addr learned my addr 172.21.15.55:0/3240629865 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:37.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.128+0000 7fdfded9d700 1 -- 172.21.15.55:0/3240629865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0150180 msgr2=0x7fdfe00caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:37.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.128+0000 7fdfded9d700 1 --2- 172.21.15.55:0/3240629865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0150180 0x7fdfe00caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:37.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.128+0000 7fdfded9d700 1 -- 172.21.15.55:0/3240629865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfe00c7240 msgr2=0x7fdfe00abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:37.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.128+0000 7fdfded9d700 1 --2- 172.21.15.55:0/3240629865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfe00c7240 0x7fdfe00abab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:37.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.128+0000 7fdfded9d700 1 -- 172.21.15.55:0/3240629865 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd00094f0 con 0x7fdfe0150550 2023-10-06T03:52:37.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.128+0000 7fdfded9d700 1 --2- 172.21.15.55:0/3240629865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe0150550 0x7fdfe0143710 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800c930 tx=0x7fdfd80026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:37.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.129+0000 7fdfce7fc700 1 -- 172.21.15.55:0/3240629865 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8012630 con 0x7fdfe0150550 2023-10-06T03:52:37.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.129+0000 7fdfce7fc700 1 -- 172.21.15.55:0/3240629865 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd80127d0 con 0x7fdfe0150550 2023-10-06T03:52:37.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.129+0000 7fdfce7fc700 1 -- 172.21.15.55:0/3240629865 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8012b90 con 0x7fdfe0150550 2023-10-06T03:52:37.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.129+0000 7fdfdf59e700 1 -- 172.21.15.55:0/3240629865 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe00ac9c0 con 0x7fdfe0150550 2023-10-06T03:52:37.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.129+0000 7fdfdf59e700 1 -- 172.21.15.55:0/3240629865 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe00ac400 con 0x7fdfe0150550 2023-10-06T03:52:37.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.130+0000 7fdfdf59e700 1 -- 172.21.15.55:0/3240629865 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe004c3c0 con 0x7fdfe0150550 2023-10-06T03:52:37.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.131+0000 7fdfce7fc700 1 -- 172.21.15.55:0/3240629865 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fdfd801a3b0 con 0x7fdfe0150550 2023-10-06T03:52:37.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.132+0000 7fdfce7fc700 1 --2- 172.21.15.55:0/3240629865 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fdfc0067050 0x7fdfc0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:37.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.132+0000 7fdfce7fc700 1 -- 172.21.15.55:0/3240629865 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fdfd8084680 con 0x7fdfe0150550 2023-10-06T03:52:37.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.132+0000 7fdfde59c700 1 --2- 172.21.15.55:0/3240629865 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fdfc0067050 0x7fdfc0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:37.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.133+0000 7fdfde59c700 1 --2- 172.21.15.55:0/3240629865 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fdfc0067050 0x7fdfc0069500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00099e0 tx=0x7fdfd0016000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:37.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.136+0000 7fdfce7fc700 1 -- 172.21.15.55:0/3240629865 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdfd8054ee0 con 0x7fdfe0150550 2023-10-06T03:52:37.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.385+0000 7fdfdf59e700 1 -- 172.21.15.55:0/3240629865 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfe006ada0 con 0x7fdfe0150550 2023-10-06T03:52:37.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.385+0000 7fdfce7fc700 1 -- 172.21.15.55:0/3240629865 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fdfd8054a70 con 0x7fdfe0150550 2023-10-06T03:52:37.408 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:37.408 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2624033145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2624033145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2624033145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3851023466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3851023466},{"type":"v1","addr":"172.21.15.55:6834","nonce":3851023466}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/2633241022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2633241022},{"type":"v1","addr":"172.21.15.163:6833","nonce":2633241022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:52:30.648704+0000","modified":"2023-10-06T03:52:34.152554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7879":{"gid":7879,"name":"d","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6836/246684871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":246684871},{"type":"v1","addr":"172.21.15.163:6836","nonce":246684871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T03:52:37.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.388+0000 7fdfdf59e700 1 -- 172.21.15.55:0/3240629865 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fdfc0067050 msgr2=0x7fdfc0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:37.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.388+0000 7fdfdf59e700 1 --2- 172.21.15.55:0/3240629865 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fdfc0067050 0x7fdfc0069500 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00099e0 tx=0x7fdfd0016000 comp rx=0 tx=0).stop 2023-10-06T03:52:37.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.388+0000 7fdfdf59e700 1 -- 172.21.15.55:0/3240629865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe0150550 msgr2=0x7fdfe0143710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:37.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.388+0000 7fdfdf59e700 1 --2- 172.21.15.55:0/3240629865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe0150550 0x7fdfe0143710 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800c930 tx=0x7fdfd80026e0 comp rx=0 tx=0).stop 2023-10-06T03:52:37.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.388+0000 7fdfdf59e700 1 -- 172.21.15.55:0/3240629865 shutdown_connections 2023-10-06T03:52:37.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.388+0000 7fdfdf59e700 1 --2- 172.21.15.55:0/3240629865 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fdfc0067050 0x7fdfc0069500 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:37.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.388+0000 7fdfdf59e700 1 --2- 172.21.15.55:0/3240629865 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdfe0150550 0x7fdfe0143710 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:37.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.388+0000 7fdfdf59e700 1 --2- 172.21.15.55:0/3240629865 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdfe00c7240 0x7fdfe00abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:37.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.388+0000 7fdfdf59e700 1 --2- 172.21.15.55:0/3240629865 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdfe0150180 0x7fdfe00caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:37.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.388+0000 7fdfdf59e700 1 -- 172.21.15.55:0/3240629865 >> 172.21.15.55:0/3240629865 conn(0x7fdfe00cc1f0 msgr2=0x7fdfe00696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:37.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.388+0000 7fdfdf59e700 1 -- 172.21.15.55:0/3240629865 shutdown_connections 2023-10-06T03:52:37.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:37.388+0000 7fdfdf59e700 1 -- 172.21.15.55:0/3240629865 wait complete. 2023-10-06T03:52:37.411 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 70 2023-10-06T03:52:37.814 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T03:52:30.648704+0000', 'modified': '2023-10-06T03:52:34.152554+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7879}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7879': {'gid': 7879, 'name': 'd', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6836/246684871', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6834', 'nonce': 246684871}, {'type': 'v1', 'addr': '172.21.15.163:6836', 'nonce': 246684871}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T03:52:37.814 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T03:52:37.815 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T03:52:38.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.004+0000 7fae3d39d700 1 Processor -- start 2023-10-06T03:52:38.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.005+0000 7fae3d39d700 1 -- start start 2023-10-06T03:52:38.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.006+0000 7fae3d39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fae38157270 0x7fae38068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.006+0000 7fae3d39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380688d0 0x7fae380b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.006+0000 7fae3d39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3815a980 con 0x7fae380b0a00 2023-10-06T03:52:38.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.006+0000 7fae3d39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3815e310 con 0x7fae38157270 2023-10-06T03:52:38.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.006+0000 7fae3d39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae38068d20 con 0x7fae380688d0 2023-10-06T03:52:38.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.006+0000 7fae3cb9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fae380b0a00 0x7fae3806a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35246/0 (socket says 172.21.15.55:35246) 2023-10-06T03:52:38.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.006+0000 7fae3cb9c700 1 -- 172.21.15.55:0/2348761447 learned_addr learned my addr 172.21.15.55:0/2348761447 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:38.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.006+0000 7fae37fff700 1 --2- 172.21.15.55:0/2348761447 >> v2:172.21.15.163:3300/0 conn(0x7fae38157270 0x7fae38068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.006+0000 7fae377fe700 1 --2- 172.21.15.55:0/2348761447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380688d0 0x7fae380b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.007+0000 7fae36ffd700 1 -- 172.21.15.55:0/2348761447 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 168160035 0 0) 0x7fae3815a980 con 0x7fae380b0a00 2023-10-06T03:52:38.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.007+0000 7fae36ffd700 1 -- 172.21.15.55:0/2348761447 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae20003570 con 0x7fae380b0a00 2023-10-06T03:52:38.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.007+0000 7fae37fff700 1 -- 172.21.15.55:0/2348761447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380688d0 msgr2=0x7fae380b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:38.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.007+0000 7fae37fff700 1 --2- 172.21.15.55:0/2348761447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380688d0 0x7fae380b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.007+0000 7fae37fff700 1 -- 172.21.15.55:0/2348761447 >> v1:172.21.15.55:6789/0 conn(0x7fae380b0a00 legacy=0x7fae3806a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.007+0000 7fae37fff700 1 -- 172.21.15.55:0/2348761447 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3815a050 con 0x7fae38157270 2023-10-06T03:52:38.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.007+0000 7fae377fe700 1 --2- 172.21.15.55:0/2348761447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380688d0 0x7fae380b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:52:38.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.007+0000 7fae37fff700 1 --2- 172.21.15.55:0/2348761447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae38157270 0x7fae38068390 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fae280098e0 tx=0x7fae280091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc82e8cd9afbd62b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:38.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.008+0000 7fae36ffd700 1 -- 172.21.15.55:0/2348761447 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28011630 con 0x7fae38157270 2023-10-06T03:52:38.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.008+0000 7fae36ffd700 1 -- 172.21.15.55:0/2348761447 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae28011790 con 0x7fae38157270 2023-10-06T03:52:38.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.008+0000 7fae36ffd700 1 -- 172.21.15.55:0/2348761447 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28011b30 con 0x7fae38157270 2023-10-06T03:52:38.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.008+0000 7fae3d39d700 1 -- 172.21.15.55:0/2348761447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae38157270 msgr2=0x7fae38068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.008+0000 7fae3d39d700 1 --2- 172.21.15.55:0/2348761447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae38157270 0x7fae38068390 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fae280098e0 tx=0x7fae280091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.008+0000 7fae3d39d700 1 -- 172.21.15.55:0/2348761447 shutdown_connections 2023-10-06T03:52:38.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.008+0000 7fae3d39d700 1 --2- 172.21.15.55:0/2348761447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae38157270 0x7fae38068390 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.008+0000 7fae3d39d700 1 --2- 172.21.15.55:0/2348761447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380688d0 0x7fae380b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.008+0000 7fae3d39d700 1 -- 172.21.15.55:0/2348761447 >> 172.21.15.55:0/2348761447 conn(0x7fae380cd310 msgr2=0x7fae380cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:38.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.008+0000 7fae3d39d700 1 -- 172.21.15.55:0/2348761447 shutdown_connections 2023-10-06T03:52:38.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.008+0000 7fae3d39d700 1 -- 172.21.15.55:0/2348761447 wait complete. 2023-10-06T03:52:38.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.010+0000 7fae3d39d700 1 Processor -- start 2023-10-06T03:52:38.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.010+0000 7fae3d39d700 1 -- start start 2023-10-06T03:52:38.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.011+0000 7fae3d39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380688d0 0x7fae3814fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.011+0000 7fae3d39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae380b0a00 0x7fae38155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.011+0000 7fae3d39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae38157270 0x7fae381582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.011+0000 7fae3d39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae380adb50 con 0x7fae380b0a00 2023-10-06T03:52:38.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.011+0000 7fae3d39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae380ad820 con 0x7fae38157270 2023-10-06T03:52:38.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.011+0000 7fae3d39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae381562e0 con 0x7fae380688d0 2023-10-06T03:52:38.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.011+0000 7fae377fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae380b0a00 0x7fae38155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.011+0000 7fae377fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae380b0a00 0x7fae38155890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49954/0 (socket says 172.21.15.55:49954) 2023-10-06T03:52:38.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.011+0000 7fae377fe700 1 -- 172.21.15.55:0/877240235 learned_addr learned my addr 172.21.15.55:0/877240235 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:38.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.011+0000 7fae37fff700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380688d0 0x7fae3814fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.011+0000 7fae3cb9c700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae38157270 0x7fae381582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.011+0000 7fae377fe700 1 -- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380688d0 msgr2=0x7fae3814fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.011+0000 7fae377fe700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380688d0 0x7fae3814fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.011+0000 7fae377fe700 1 -- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae38157270 msgr2=0x7fae381582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.012+0000 7fae377fe700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae38157270 0x7fae381582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.012+0000 7fae377fe700 1 -- 172.21.15.55:0/877240235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c009420 con 0x7fae380b0a00 2023-10-06T03:52:38.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.012+0000 7fae3cb9c700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae38157270 0x7fae381582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:38.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.012+0000 7fae377fe700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae380b0a00 0x7fae38155890 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00ec80 tx=0x7fae2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:38.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.012+0000 7fae1ffff700 1 -- 172.21.15.55:0/877240235 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00c8f0 con 0x7fae380b0a00 2023-10-06T03:52:38.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.012+0000 7fae1ffff700 1 -- 172.21.15.55:0/877240235 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c00ca50 con 0x7fae380b0a00 2023-10-06T03:52:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.012+0000 7fae1ffff700 1 -- 172.21.15.55:0/877240235 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c00cdd0 con 0x7fae380b0a00 2023-10-06T03:52:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.012+0000 7fae3d39d700 1 -- 172.21.15.55:0/877240235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae28009420 con 0x7fae380b0a00 2023-10-06T03:52:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.012+0000 7fae3d39d700 1 -- 172.21.15.55:0/877240235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae381503a0 con 0x7fae380b0a00 2023-10-06T03:52:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.014+0000 7fae1ffff700 1 -- 172.21.15.55:0/877240235 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fae2c013680 con 0x7fae380b0a00 2023-10-06T03:52:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.015+0000 7fae1ffff700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fae24067050 0x7fae24069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.015+0000 7fae1ffff700 1 -- 172.21.15.55:0/877240235 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fae2c084db0 con 0x7fae380b0a00 2023-10-06T03:52:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.016+0000 7fae37fff700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fae24067050 0x7fae24069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.016+0000 7fae37fff700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fae24067050 0x7fae24069500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fae280091e0 tx=0x7fae28017000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.017+0000 7fae4cdc8b80 1 Processor -- start 2023-10-06T03:52:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.017+0000 7fae4cdc8b80 1 -- start start 2023-10-06T03:52:38.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.018+0000 7fae4cdc8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c642e50 0x55a07c61dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.018+0000 7fae4cdc8b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c6232c0 0x55a07c61e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.018+0000 7fae4cdc8b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 0x55a07c61e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.018+0000 7fae4cdc8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c578a70 con 0x55a07c642e50 2023-10-06T03:52:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.018+0000 7fae4cdc8b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c576270 con 0x55a07c624a10 2023-10-06T03:52:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.018+0000 7fae4cdc8b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a07c572060 con 0x55a07c6232c0 2023-10-06T03:52:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.019+0000 7fae37fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c642e50 0x55a07c61dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.019+0000 7fae37fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c642e50 0x55a07c61dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49968/0 (socket says 172.21.15.55:49968) 2023-10-06T03:52:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.019+0000 7fae37fff700 1 -- 172.21.15.55:0/2209990386 learned_addr learned my addr 172.21.15.55:0/2209990386 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.019+0000 7fae377fe700 1 --2- 172.21.15.55:0/2209990386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c6232c0 0x55a07c61e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.019+0000 7fae37fff700 1 -- 172.21.15.55:0/2209990386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c6232c0 msgr2=0x55a07c61e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:38.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.019+0000 7fae37fff700 1 --2- 172.21.15.55:0/2209990386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c6232c0 0x55a07c61e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.019+0000 7fae3cb9c700 1 --2- 172.21.15.55:0/2209990386 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 0x55a07c61e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.019+0000 7fae37fff700 1 -- 172.21.15.55:0/2209990386 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 msgr2=0x55a07c61e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:38.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.019+0000 7fae37fff700 1 --2- 172.21.15.55:0/2209990386 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 0x55a07c61e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.019+0000 7fae37fff700 1 -- 172.21.15.55:0/2209990386 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a07c40c270 con 0x55a07c642e50 2023-10-06T03:52:38.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.019+0000 7fae377fe700 1 --2- 172.21.15.55:0/2209990386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c6232c0 0x55a07c61e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:38.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.019+0000 7fae3cb9c700 1 --2- 172.21.15.55:0/2209990386 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 0x55a07c61e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:38.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.019+0000 7fae37fff700 1 --2- 172.21.15.55:0/2209990386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c642e50 0x55a07c61dcc0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fae28003080 tx=0x7fae28021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bd950e5a13d484 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:38.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.020+0000 7fae1d7fa700 1 -- 172.21.15.55:0/2209990386 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28021070 con 0x55a07c642e50 2023-10-06T03:52:38.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.020+0000 7fae1d7fa700 1 -- 172.21.15.55:0/2209990386 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2802ac30 con 0x55a07c642e50 2023-10-06T03:52:38.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.020+0000 7fae1d7fa700 1 -- 172.21.15.55:0/2209990386 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2802ade0 con 0x55a07c642e50 2023-10-06T03:52:38.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.020+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/2209990386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c642e50 msgr2=0x55a07c61dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.020+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/2209990386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c642e50 0x55a07c61dcc0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fae28003080 tx=0x7fae28021000 comp rx=0 tx=0).stop 2023-10-06T03:52:38.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.020+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/2209990386 shutdown_connections 2023-10-06T03:52:38.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.020+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/2209990386 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 0x55a07c61e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.020+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/2209990386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c642e50 0x55a07c61dcc0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.020+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/2209990386 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c6232c0 0x55a07c61e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.020+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/2209990386 >> 172.21.15.55:0/2209990386 conn(0x55a07c622e80 msgr2=0x55a07c6236b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:38.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.020+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/2209990386 shutdown_connections 2023-10-06T03:52:38.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.020+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/2209990386 wait complete. 2023-10-06T03:52:38.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.021+0000 7fae4cdc8b80 1 -- start start 2023-10-06T03:52:38.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.021+0000 7fae4cdc8b80 1 Processor -- start 2023-10-06T03:52:38.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.022+0000 7fae4cdc8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c6232c0 0x55a07c641a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.022+0000 7fae4cdc8b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 0x55a07c643220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.022+0000 7fae4cdc8b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c642e50 0x55a07c66c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.022+0000 7fae4cdc8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c51d890 con 0x55a07c6232c0 2023-10-06T03:52:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.022+0000 7fae4cdc8b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c538ea0 con 0x55a07c624a10 2023-10-06T03:52:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.022+0000 7fae4cdc8b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a07c3f4e30 con 0x55a07c642e50 2023-10-06T03:52:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.022+0000 7fae4cdc8b80 10 client.0 _ll_register_callbacks cb 0x55a07c5e5020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:52:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.023+0000 7fae37fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c6232c0 0x55a07c641a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.023+0000 7fae37fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c6232c0 0x55a07c641a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49970/0 (socket says 172.21.15.55:49970) 2023-10-06T03:52:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.023+0000 7fae37fff700 1 -- 172.21.15.55:0/2850518303 learned_addr learned my addr 172.21.15.55:0/2850518303 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.023+0000 7fae377fe700 1 --2- 172.21.15.55:0/2850518303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 0x55a07c643220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.023+0000 7fae3cb9c700 1 --2- 172.21.15.55:0/2850518303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c642e50 0x55a07c66c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.023+0000 7fae37fff700 1 -- 172.21.15.55:0/2850518303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c642e50 msgr2=0x55a07c66c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:38.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.023+0000 7fae37fff700 1 --2- 172.21.15.55:0/2850518303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c642e50 0x55a07c66c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.023+0000 7fae37fff700 1 -- 172.21.15.55:0/2850518303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 msgr2=0x55a07c643220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:38.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.023+0000 7fae37fff700 1 --2- 172.21.15.55:0/2850518303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 0x55a07c643220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.023+0000 7fae37fff700 1 -- 172.21.15.55:0/2850518303 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3815a050 con 0x55a07c6232c0 2023-10-06T03:52:38.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.023+0000 7fae37fff700 1 --2- 172.21.15.55:0/2850518303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c6232c0 0x55a07c641a10 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fae28034040 tx=0x7fae2802c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0f97cfed330830 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:38.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.023+0000 7fae167fc700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:52:38.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.023+0000 7fae167fc700 1 -- 172.21.15.55:0/2850518303 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28021040 con 0x55a07c6232c0 2023-10-06T03:52:38.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.023+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/2850518303 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a07c616940 con 0x55a07c6232c0 2023-10-06T03:52:38.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.024+0000 7fae167fc700 1 -- 172.21.15.55:0/2850518303 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fae2803c410 con 0x55a07c6232c0 2023-10-06T03:52:38.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.024+0000 7fae167fc700 1 -- 172.21.15.55:0/2850518303 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae2803c7e0 con 0x55a07c6232c0 2023-10-06T03:52:38.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.024+0000 7fae4cdc8b80 10 client.8361 fetch_fsmap learned FSMap version 70 2023-10-06T03:52:38.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.024+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/2850518303 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x55a07c3aefe0 con 0x55a07c6232c0 2023-10-06T03:52:38.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.024+0000 7fae167fc700 1 -- 172.21.15.55:0/2850518303 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a07c616940 con 0x55a07c6232c0 2023-10-06T03:52:38.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.024+0000 7fae167fc700 1 -- 172.21.15.55:0/2850518303 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 70) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fae2803c940 con 0x55a07c6232c0 2023-10-06T03:52:38.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.024+0000 7fae4cdc8b80 10 client.8361 fetch_fsmap finished waiting for FSMap version 70 2023-10-06T03:52:38.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.024+0000 7fae4cdc8b80 10 client.8361 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T03:52:38.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.024+0000 7fae4cdc8b80 20 client.8361 populate_metadata read hostname 'smithi055' 2023-10-06T03:52:38.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.024+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/2850518303 >> [v2:172.21.15.163:6834/246684871,v1:172.21.15.163:6836/246684871] conn(0x55a07c67b500 0x55a07c67b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.024+0000 7fae4cdc8b80 4 client.8361 mds_command: new command op to 7879 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T03:52:38.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.024+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/2850518303 --> [v2:172.21.15.163:6834/246684871,v1:172.21.15.163:6836/246684871] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a07c5f3870 con 0x55a07c67b500 2023-10-06T03:52:38.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.025+0000 7fae377fe700 1 --2- 172.21.15.55:0/2850518303 >> [v2:172.21.15.163:6834/246684871,v1:172.21.15.163:6836/246684871] conn(0x55a07c67b500 0x55a07c67b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.025+0000 7fae377fe700 1 --2- 172.21.15.55:0/2850518303 >> [v2:172.21.15.163:6834/246684871,v1:172.21.15.163:6836/246684871] conn(0x55a07c67b500 0x55a07c67b8d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e56c2c1eabe5552 server_cookie=4a4fd0bc52e3dd2a in_seq=0 out_seq=0 2023-10-06T03:52:38.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.025+0000 7fae167fc700 10 client.8361 ms_handle_connect on v2:172.21.15.163:6834/246684871 2023-10-06T03:52:38.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.026+0000 7fae167fc700 1 -- 172.21.15.55:0/2850518303 <== mds.0 v2:172.21.15.163:6834/246684871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a07c5f3870 con 0x55a07c67b500 2023-10-06T03:52:38.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.026+0000 7fae167fc700 10 client.8361 handle_command_reply: tid=0 2023-10-06T03:52:38.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.027+0000 7fae4cdc8b80 1 client.8361 shutdown 2023-10-06T03:52:38.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.027+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/2850518303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c6232c0 msgr2=0x55a07c641a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.027+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/2850518303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c6232c0 0x55a07c641a10 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fae28034040 tx=0x7fae2802c040 comp rx=0 tx=0).stop 2023-10-06T03:52:38.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.027+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/2850518303 shutdown_connections 2023-10-06T03:52:38.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.027+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/2850518303 >> [v2:172.21.15.163:6834/246684871,v1:172.21.15.163:6836/246684871] conn(0x55a07c67b500 0x55a07c67b8d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.027+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/2850518303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c642e50 0x55a07c66c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.027+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/2850518303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c6232c0 0x55a07c641a10 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.027+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/2850518303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 0x55a07c643220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.027+0000 7fae167fc700 0 client.8361 ms_handle_reset on v2:172.21.15.163:6834/246684871 2023-10-06T03:52:38.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.027+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/2850518303 >> 172.21.15.55:0/2850518303 conn(0x55a07c622e80 msgr2=0x55a07c623d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:38.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.027+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/2850518303 shutdown_connections 2023-10-06T03:52:38.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.027+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/2850518303 wait complete. 2023-10-06T03:52:38.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.027+0000 7fae4cdc8b80 20 client.8361 trim_cache size 0 max 16384 2023-10-06T03:52:38.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.044+0000 7fae4cdc8b80 1 Processor -- start 2023-10-06T03:52:38.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.045+0000 7fae4cdc8b80 1 -- start start 2023-10-06T03:52:38.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.045+0000 7fae4cdc8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c6232c0 0x55a07c64f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.045+0000 7fae4cdc8b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c624a10 0x55a07c623fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.045+0000 7fae4cdc8b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c642e50 0x55a07c6688c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.045+0000 7fae4cdc8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c643480 con 0x55a07c6232c0 2023-10-06T03:52:38.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.045+0000 7fae4cdc8b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c6402a0 con 0x55a07c642e50 2023-10-06T03:52:38.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.045+0000 7fae4cdc8b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a07c64b9b0 con 0x55a07c624a10 2023-10-06T03:52:38.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.046+0000 7fae37fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c6232c0 0x55a07c64f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.046+0000 7fae37fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c6232c0 0x55a07c64f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49986/0 (socket says 172.21.15.55:49986) 2023-10-06T03:52:38.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.046+0000 7fae37fff700 1 -- 172.21.15.55:0/1053611660 learned_addr learned my addr 172.21.15.55:0/1053611660 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:38.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.046+0000 7fae3cb9c700 1 --2- 172.21.15.55:0/1053611660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c642e50 0x55a07c6688c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.046+0000 7fae377fe700 1 --2- 172.21.15.55:0/1053611660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c624a10 0x55a07c623fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.046+0000 7fae37fff700 1 -- 172.21.15.55:0/1053611660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c624a10 msgr2=0x55a07c623fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.046+0000 7fae37fff700 1 --2- 172.21.15.55:0/1053611660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c624a10 0x55a07c623fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.046+0000 7fae37fff700 1 -- 172.21.15.55:0/1053611660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c642e50 msgr2=0x55a07c6688c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.046+0000 7fae37fff700 1 --2- 172.21.15.55:0/1053611660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c642e50 0x55a07c6688c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.046+0000 7fae37fff700 1 -- 172.21.15.55:0/1053611660 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a07c540780 con 0x55a07c6232c0 2023-10-06T03:52:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.046+0000 7fae377fe700 1 --2- 172.21.15.55:0/1053611660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c624a10 0x55a07c623fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.046+0000 7fae3cb9c700 1 --2- 172.21.15.55:0/1053611660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c642e50 0x55a07c6688c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.046+0000 7fae37fff700 1 --2- 172.21.15.55:0/1053611660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c6232c0 0x55a07c64f6c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fae2802ac00 tx=0x7fae2801c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c99597a9d52f0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.046+0000 7fae1dffb700 1 -- 172.21.15.55:0/1053611660 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2803cea0 con 0x55a07c6232c0 2023-10-06T03:52:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.047+0000 7fae1dffb700 1 -- 172.21.15.55:0/1053611660 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2801aac0 con 0x55a07c6232c0 2023-10-06T03:52:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.047+0000 7fae1dffb700 1 -- 172.21.15.55:0/1053611660 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae28021070 con 0x55a07c6232c0 2023-10-06T03:52:38.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.047+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/1053611660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c6232c0 msgr2=0x55a07c64f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.047+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/1053611660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c6232c0 0x55a07c64f6c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fae2802ac00 tx=0x7fae2801c070 comp rx=0 tx=0).stop 2023-10-06T03:52:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.047+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/1053611660 shutdown_connections 2023-10-06T03:52:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.047+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/1053611660 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c642e50 0x55a07c6688c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.047+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/1053611660 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c6232c0 0x55a07c64f6c0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.047+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/1053611660 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c624a10 0x55a07c623fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.047+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/1053611660 >> 172.21.15.55:0/1053611660 conn(0x55a07c622e80 msgr2=0x55a07c640810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.047+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/1053611660 shutdown_connections 2023-10-06T03:52:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.047+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/1053611660 wait complete. 2023-10-06T03:52:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.048+0000 7fae4cdc8b80 1 -- start start 2023-10-06T03:52:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.048+0000 7fae4cdc8b80 1 Processor -- start 2023-10-06T03:52:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.049+0000 7fae4cdc8b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c6232c0 0x55a07c650720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.049+0000 7fae4cdc8b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 0x55a07c668690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.049+0000 7fae4cdc8b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c642e50 0x55a07c668bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.049+0000 7fae4cdc8b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c6444f0 con 0x55a07c642e50 2023-10-06T03:52:38.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.049+0000 7fae4cdc8b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a07c63f2f0 con 0x55a07c624a10 2023-10-06T03:52:38.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.049+0000 7fae4cdc8b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a07c650c60 con 0x55a07c6232c0 2023-10-06T03:52:38.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.049+0000 7fae4cdc8b80 10 client.0 _ll_register_callbacks cb 0x55a07c5e5020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T03:52:38.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.049+0000 7fae3cb9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c642e50 0x55a07c668bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.049+0000 7fae3cb9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c642e50 0x55a07c668bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49994/0 (socket says 172.21.15.55:49994) 2023-10-06T03:52:38.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.049+0000 7fae3cb9c700 1 -- 172.21.15.55:0/3245030884 learned_addr learned my addr 172.21.15.55:0/3245030884 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:38.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.049+0000 7fae37fff700 1 --2- 172.21.15.55:0/3245030884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c6232c0 0x55a07c650720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.049+0000 7fae377fe700 1 --2- 172.21.15.55:0/3245030884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 0x55a07c668690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.050+0000 7fae3cb9c700 1 -- 172.21.15.55:0/3245030884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c6232c0 msgr2=0x55a07c650720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:38.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.050+0000 7fae3cb9c700 1 --2- 172.21.15.55:0/3245030884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c6232c0 0x55a07c650720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.050+0000 7fae3cb9c700 1 -- 172.21.15.55:0/3245030884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 msgr2=0x55a07c668690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:38.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.050+0000 7fae3cb9c700 1 --2- 172.21.15.55:0/3245030884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 0x55a07c668690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.050+0000 7fae3cb9c700 1 -- 172.21.15.55:0/3245030884 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a07c3aefe0 con 0x55a07c642e50 2023-10-06T03:52:38.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.050+0000 7fae3cb9c700 1 --2- 172.21.15.55:0/3245030884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c642e50 0x55a07c668bd0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fae30005c50 tx=0x7fae30009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0b057fd5016a93 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:38.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.050+0000 7fae167fc700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T03:52:38.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.050+0000 7fae167fc700 1 -- 172.21.15.55:0/3245030884 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30016690 con 0x55a07c642e50 2023-10-06T03:52:38.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.050+0000 7fae167fc700 1 -- 172.21.15.55:0/3245030884 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fae300167f0 con 0x55a07c642e50 2023-10-06T03:52:38.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.050+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/3245030884 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae380aa990 con 0x55a07c642e50 2023-10-06T03:52:38.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.050+0000 7fae167fc700 1 -- 172.21.15.55:0/3245030884 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae30016bc0 con 0x55a07c642e50 2023-10-06T03:52:38.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.050+0000 7fae4cdc8b80 10 client.8367 fetch_fsmap learned FSMap version 70 2023-10-06T03:52:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.050+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/3245030884 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x55a07c527470 con 0x55a07c642e50 2023-10-06T03:52:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.051+0000 7fae167fc700 1 -- 172.21.15.55:0/3245030884 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae380aa990 con 0x55a07c642e50 2023-10-06T03:52:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.051+0000 7fae167fc700 1 -- 172.21.15.55:0/3245030884 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 70) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fae30016d20 con 0x55a07c642e50 2023-10-06T03:52:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.051+0000 7fae4cdc8b80 10 client.8367 fetch_fsmap finished waiting for FSMap version 70 2023-10-06T03:52:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.051+0000 7fae4cdc8b80 10 client.8367 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T03:52:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.051+0000 7fae4cdc8b80 20 client.8367 populate_metadata read hostname 'smithi055' 2023-10-06T03:52:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.051+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/3245030884 >> [v2:172.21.15.163:6834/246684871,v1:172.21.15.163:6836/246684871] conn(0x55a07c669790 0x55a07c669b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.051+0000 7fae4cdc8b80 4 client.8367 mds_command: new command op to 7879 tid=0[{"prefix": "status"}] 2023-10-06T03:52:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.051+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/3245030884 --> [v2:172.21.15.163:6834/246684871,v1:172.21.15.163:6836/246684871] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fae2802ac30 con 0x55a07c669790 2023-10-06T03:52:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.051+0000 7fae37fff700 1 --2- 172.21.15.55:0/3245030884 >> [v2:172.21.15.163:6834/246684871,v1:172.21.15.163:6836/246684871] conn(0x55a07c669790 0x55a07c669b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.052+0000 7fae167fc700 10 client.8367 ms_handle_connect on v2:172.21.15.163:6834/246684871 2023-10-06T03:52:38.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.052+0000 7fae37fff700 1 --2- 172.21.15.55:0/3245030884 >> [v2:172.21.15.163:6834/246684871,v1:172.21.15.163:6836/246684871] conn(0x55a07c669790 0x55a07c669b60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7526a52223ade23d server_cookie=cde0180c2f187b22 in_seq=0 out_seq=0 2023-10-06T03:52:38.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.052+0000 7fae167fc700 1 -- 172.21.15.55:0/3245030884 <== mds.0 v2:172.21.15.163:6834/246684871 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x7fae2802ac30 con 0x55a07c669790 2023-10-06T03:52:38.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.052+0000 7fae167fc700 10 client.8367 handle_command_reply: tid=0 2023-10-06T03:52:38.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.052+0000 7fae4cdc8b80 1 client.8367 shutdown 2023-10-06T03:52:38.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.052+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/3245030884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c642e50 msgr2=0x55a07c668bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.052+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/3245030884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c642e50 0x55a07c668bd0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fae30005c50 tx=0x7fae30009040 comp rx=0 tx=0).stop 2023-10-06T03:52:38.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/3245030884 shutdown_connections 2023-10-06T03:52:38.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/3245030884 >> [v2:172.21.15.163:6834/246684871,v1:172.21.15.163:6836/246684871] conn(0x55a07c669790 0x55a07c669b60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/3245030884 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55a07c642e50 0x55a07c668bd0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/3245030884 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55a07c6232c0 0x55a07c650720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae4cdc8b80 1 --2- 172.21.15.55:0/3245030884 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55a07c624a10 0x55a07c668690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae167fc700 0 client.8367 ms_handle_reset on v2:172.21.15.163:6834/246684871 2023-10-06T03:52:38.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/3245030884 >> 172.21.15.55:0/3245030884 conn(0x55a07c622e80 msgr2=0x55a07c67b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:38.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/3245030884 shutdown_connections 2023-10-06T03:52:38.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae4cdc8b80 1 -- 172.21.15.55:0/3245030884 wait complete. 2023-10-06T03:52:38.206 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae4cdc8b80 20 client.8367 trim_cache size 0 max 16384 2023-10-06T03:52:38.206 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T03:52:38.207 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "8132c8ba-ee1c-41a2-b781-913925034ab6", 2023-10-06T03:52:38.207 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2023-10-06T03:52:38.207 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 7879, 2023-10-06T03:52:38.207 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2023-10-06T03:52:38.207 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2023-10-06T03:52:38.207 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2023-10-06T03:52:38.207 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 6.9243156189999997, 2023-10-06T03:52:38.207 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 70, 2023-10-06T03:52:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 137, 2023-10-06T03:52:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 137, 2023-10-06T03:52:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 14.794475630000001 2023-10-06T03:52:38.209 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T03:52:38.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae3d39d700 1 -- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fae24067050 msgr2=0x7fae24069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae3d39d700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fae24067050 0x7fae24069500 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fae280091e0 tx=0x7fae28017000 comp rx=0 tx=0).stop 2023-10-06T03:52:38.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae3d39d700 1 -- 172.21.15.55:0/877240235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae380b0a00 msgr2=0x7fae38155890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae3d39d700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae380b0a00 0x7fae38155890 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00ec80 tx=0x7fae2c0091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae3d39d700 1 -- 172.21.15.55:0/877240235 shutdown_connections 2023-10-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae3d39d700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fae24067050 0x7fae24069500 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae3d39d700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae38157270 0x7fae381582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae3d39d700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380688d0 0x7fae3814fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.053+0000 7fae3d39d700 1 --2- 172.21.15.55:0/877240235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae380b0a00 0x7fae38155890 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.054+0000 7fae3d39d700 1 -- 172.21.15.55:0/877240235 >> 172.21.15.55:0/877240235 conn(0x7fae380cd310 msgr2=0x7fae38159650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:38.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.054+0000 7fae3d39d700 1 -- 172.21.15.55:0/877240235 shutdown_connections 2023-10-06T03:52:38.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.054+0000 7fae3d39d700 1 -- 172.21.15.55:0/877240235 wait complete. 2023-10-06T03:52:38.213 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T03:52:38.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.399+0000 7fb71f59e700 1 Processor -- start 2023-10-06T03:52:38.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.399+0000 7fb71f59e700 1 -- start start 2023-10-06T03:52:38.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.400+0000 7fb71f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb720150550 0x7fb720157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.400+0000 7fb71f59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb7200c7410 0x7fb720069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.400+0000 7fb71f59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72006e2a0 con 0x7fb720150180 2023-10-06T03:52:38.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.400+0000 7fb71f59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72006b2e0 con 0x7fb7200c7410 2023-10-06T03:52:38.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.400+0000 7fb71f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200c78e0 con 0x7fb720150550 2023-10-06T03:52:38.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.401+0000 7fb71dd9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb720150550 0x7fb720157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.401+0000 7fb71ed9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb7200c7410 0x7fb720069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.401+0000 7fb71ed9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb7200c7410 0x7fb720069f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:55432/0 (socket says 172.21.15.55:55432) 2023-10-06T03:52:38.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.401+0000 7fb71ed9d700 1 -- 172.21.15.55:0/4285053747 learned_addr learned my addr 172.21.15.55:0/4285053747 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:38.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.401+0000 7fb71dd9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb720150550 0x7fb720157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:39700/0 (socket says 172.21.15.55:39700) 2023-10-06T03:52:38.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.401+0000 7fb71dd9b700 1 -- 172.21.15.55:0/4285053747 >> v2:172.21.15.163:3300/0 conn(0x7fb7200c7410 msgr2=0x7fb720069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:38.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.401+0000 7fb71dd9b700 1 --2- 172.21.15.55:0/4285053747 >> v2:172.21.15.163:3300/0 conn(0x7fb7200c7410 0x7fb720069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.401+0000 7fb71dd9b700 1 -- 172.21.15.55:0/4285053747 >> v1:172.21.15.163:6789/0 conn(0x7fb720150180 legacy=0x7fb7200c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.401+0000 7fb71dd9b700 1 -- 172.21.15.55:0/4285053747 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7200c7bc0 con 0x7fb720150550 2023-10-06T03:52:38.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.402+0000 7fb71dd9b700 1 --2- 172.21.15.55:0/4285053747 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb720150550 0x7fb720157770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb714009910 tx=0x7fb7140091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e1113b44e076cf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:38.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.402+0000 7fb71d59a700 1 -- 172.21.15.55:0/4285053747 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71400b640 con 0x7fb720150550 2023-10-06T03:52:38.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.402+0000 7fb71d59a700 1 -- 172.21.15.55:0/4285053747 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71400b7a0 con 0x7fb720150550 2023-10-06T03:52:38.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.402+0000 7fb71d59a700 1 -- 172.21.15.55:0/4285053747 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71400bb60 con 0x7fb720150550 2023-10-06T03:52:38.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.402+0000 7fb71f59e700 1 -- 172.21.15.55:0/4285053747 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb720150550 msgr2=0x7fb720157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.402+0000 7fb71f59e700 1 --2- 172.21.15.55:0/4285053747 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb720150550 0x7fb720157770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb714009910 tx=0x7fb7140091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.403+0000 7fb71f59e700 1 -- 172.21.15.55:0/4285053747 shutdown_connections 2023-10-06T03:52:38.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.403+0000 7fb71f59e700 1 --2- 172.21.15.55:0/4285053747 >> v2:172.21.15.163:3300/0 conn(0x7fb7200c7410 0x7fb720069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.403+0000 7fb71f59e700 1 --2- 172.21.15.55:0/4285053747 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb720150550 0x7fb720157770 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.403+0000 7fb71f59e700 1 -- 172.21.15.55:0/4285053747 >> 172.21.15.55:0/4285053747 conn(0x7fb7200cc1f0 msgr2=0x7fb7200cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:38.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.403+0000 7fb71f59e700 1 -- 172.21.15.55:0/4285053747 shutdown_connections 2023-10-06T03:52:38.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.403+0000 7fb71f59e700 1 -- 172.21.15.55:0/4285053747 wait complete. 2023-10-06T03:52:38.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.404+0000 7fb71f59e700 1 Processor -- start 2023-10-06T03:52:38.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.404+0000 7fb71f59e700 1 -- start start 2023-10-06T03:52:38.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.405+0000 7fb71f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7200c7410 0x7fb720149120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.405+0000 7fb71f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb720150180 0x7fb7200c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.405+0000 7fb71f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb720150550 0x7fb7200c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.405+0000 7fb71f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200ae870 con 0x7fb720150550 2023-10-06T03:52:38.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.405+0000 7fb71f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200adc60 con 0x7fb7200c7410 2023-10-06T03:52:38.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.405+0000 7fb71f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200adf40 con 0x7fb720150180 2023-10-06T03:52:38.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.405+0000 7fb71ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb720150550 0x7fb7200c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.405+0000 7fb71ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb720150550 0x7fb7200c42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49996/0 (socket says 172.21.15.55:49996) 2023-10-06T03:52:38.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.405+0000 7fb71ed9d700 1 -- 172.21.15.55:0/172071039 learned_addr learned my addr 172.21.15.55:0/172071039 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:38.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.406+0000 7fb71dd9b700 1 --2- 172.21.15.55:0/172071039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb720150180 0x7fb7200c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.406+0000 7fb71e59c700 1 --2- 172.21.15.55:0/172071039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7200c7410 0x7fb720149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.406+0000 7fb71ed9d700 1 -- 172.21.15.55:0/172071039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb720150180 msgr2=0x7fb7200c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.406+0000 7fb71ed9d700 1 --2- 172.21.15.55:0/172071039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb720150180 0x7fb7200c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.406+0000 7fb71ed9d700 1 -- 172.21.15.55:0/172071039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7200c7410 msgr2=0x7fb720149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.406+0000 7fb71ed9d700 1 --2- 172.21.15.55:0/172071039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7200c7410 0x7fb720149120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.406+0000 7fb71ed9d700 1 -- 172.21.15.55:0/172071039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb714009420 con 0x7fb720150550 2023-10-06T03:52:38.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.406+0000 7fb71ed9d700 1 --2- 172.21.15.55:0/172071039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb720150550 0x7fb7200c42b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb71800e140 tx=0x7fb71800d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:38.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.406+0000 7fb70e7fc700 1 -- 172.21.15.55:0/172071039 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718010040 con 0x7fb720150550 2023-10-06T03:52:38.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.407+0000 7fb70e7fc700 1 -- 172.21.15.55:0/172071039 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71800ec20 con 0x7fb720150550 2023-10-06T03:52:38.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.407+0000 7fb70e7fc700 1 -- 172.21.15.55:0/172071039 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718014400 con 0x7fb720150550 2023-10-06T03:52:38.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.407+0000 7fb71f59e700 1 -- 172.21.15.55:0/172071039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7200ae220 con 0x7fb720150550 2023-10-06T03:52:38.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.407+0000 7fb71f59e700 1 -- 172.21.15.55:0/172071039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb720149660 con 0x7fb720150550 2023-10-06T03:52:38.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.408+0000 7fb71f59e700 1 -- 172.21.15.55:0/172071039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72004c3c0 con 0x7fb720150550 2023-10-06T03:52:38.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.409+0000 7fb70e7fc700 1 -- 172.21.15.55:0/172071039 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fb718014830 con 0x7fb720150550 2023-10-06T03:52:38.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.410+0000 7fb70e7fc700 1 --2- 172.21.15.55:0/172071039 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb700067050 0x7fb700069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:38.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.410+0000 7fb70e7fc700 1 -- 172.21.15.55:0/172071039 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fb718085470 con 0x7fb720150550 2023-10-06T03:52:38.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.410+0000 7fb71e59c700 1 --2- 172.21.15.55:0/172071039 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb700067050 0x7fb700069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:38.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.411+0000 7fb71e59c700 1 --2- 172.21.15.55:0/172071039 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb700067050 0x7fb700069500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb710003a80 tx=0x7fb710007ec0 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:38.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.413+0000 7fb70e7fc700 1 -- 172.21.15.55:0/172071039 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7180521d0 con 0x7fb720150550 2023-10-06T03:52:38.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.659+0000 7fb71f59e700 1 -- 172.21.15.55:0/172071039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb72006ada0 con 0x7fb720150550 2023-10-06T03:52:38.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.660+0000 7fb70e7fc700 1 -- 172.21.15.55:0/172071039 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb7180557f0 con 0x7fb720150550 2023-10-06T03:52:38.662 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T03:52:38.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.662+0000 7fb71f59e700 1 -- 172.21.15.55:0/172071039 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb700067050 msgr2=0x7fb700069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.662+0000 7fb71f59e700 1 --2- 172.21.15.55:0/172071039 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb700067050 0x7fb700069500 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb710003a80 tx=0x7fb710007ec0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.662+0000 7fb71f59e700 1 -- 172.21.15.55:0/172071039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb720150550 msgr2=0x7fb7200c42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:38.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.662+0000 7fb71f59e700 1 --2- 172.21.15.55:0/172071039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb720150550 0x7fb7200c42b0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb71800e140 tx=0x7fb71800d040 comp rx=0 tx=0).stop 2023-10-06T03:52:38.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.662+0000 7fb71f59e700 1 -- 172.21.15.55:0/172071039 shutdown_connections 2023-10-06T03:52:38.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.662+0000 7fb71f59e700 1 --2- 172.21.15.55:0/172071039 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fb700067050 0x7fb700069500 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.662+0000 7fb71f59e700 1 --2- 172.21.15.55:0/172071039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb720150550 0x7fb7200c42b0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.662+0000 7fb71f59e700 1 --2- 172.21.15.55:0/172071039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7200c7410 0x7fb720149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.662+0000 7fb71f59e700 1 --2- 172.21.15.55:0/172071039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb720150180 0x7fb7200c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:38.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.662+0000 7fb71f59e700 1 -- 172.21.15.55:0/172071039 >> 172.21.15.55:0/172071039 conn(0x7fb7200cc1f0 msgr2=0x7fb720158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:38.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.662+0000 7fb71f59e700 1 -- 172.21.15.55:0/172071039 shutdown_connections 2023-10-06T03:52:38.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:38.662+0000 7fb71f59e700 1 -- 172.21.15.55:0/172071039 wait complete. 2023-10-06T03:52:38.682 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T03:52:38.685 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T03:52:38.686 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2023-10-06T03:52:38.686 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T03:52:38.686 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T03:52:38.686 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T03:52:38.686 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T03:52:38.686 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T03:52:38.686 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:52:38.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:52:38.687 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2023-10-06T03:52:38.725 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T03:52:38.725 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T03:52:38.726 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T03:52:38.726 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:52:38.726 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T03:52:38.807 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:52:38.807 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:52:38.807 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:52:38.807 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2023-10-06T03:52:38.807 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:52:38.807 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:52:38.808 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2023-10-06T03:52:38.808 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T03:52:38.808 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:52:38.808 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2023-10-06T03:52:38.808 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T03:52:38.808 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 41083sec preferred_lft 41083sec 2023-10-06T03:52:38.808 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link 2023-10-06T03:52:38.808 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:52:38.808 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T03:52:38.808 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T03:52:38.808 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2023-10-06T03:52:38.809 INFO:teuthology.orchestra.run.smithi055.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T03:52:38.809 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff 2023-10-06T03:52:38.809 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T03:52:38.809 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:52:38.809 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::1099:8eff:fe6d:102/64 scope link 2023-10-06T03:52:38.809 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:52:38.809 INFO:teuthology.orchestra.run.smithi055.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:52:38.809 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether f2:5d:53:d2:00:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T03:52:38.810 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f05d:53ff:fed2:a/64 scope link 2023-10-06T03:52:38.810 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:52:38.810 INFO:teuthology.orchestra.run.smithi055.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T03:52:38.810 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T03:52:38.810 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::e037:33ff:feb7:6c8f/64 scope link 2023-10-06T03:52:38.810 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T03:52:38.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:52:38.811 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T03:52:38.872 INFO:teuthology.orchestra.run.smithi055.stdout:1 2023-10-06T03:52:38.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:52:39.122 DEBUG:teuthology.orchestra.run.smithi055:> route 2023-10-06T03:52:39.162 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2023-10-06T03:52:39.162 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T03:52:39.162 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T03:52:39.162 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T03:52:39.162 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T03:52:39.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:52:39.164 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T03:52:39.164 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T03:52:39.164 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T03:52:39.164 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T03:52:39.164 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T03:52:39.164 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T03:52:39.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:52:39.435 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T03:52:39.474 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T03:52:39.475 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T03:52:39.475 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:52:39.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:52:39.476 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2023-10-06T03:52:39.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:52:39.583 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T03:52:39.621 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T03:52:39.621 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T03:52:39.621 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:52:39.697 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T03:52:39.753 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T03:52:39.754 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:52:39.847 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:52:39.894 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:52:39.895 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:52:39.896 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:52:39.897 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:52:39.897 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:52:39.897 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:52:39.898 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:52:39.898 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T03:52:39.898 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T03:52:39.898 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T03:52:39.898 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T03:52:39.898 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T03:52:39.899 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T03:52:39.899 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T03:52:39.899 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T03:52:39.899 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T03:52:39.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T03:52:39.899 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T03:52:39.963 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T03:52:40.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.233+0000 7f680539d700 1 Processor -- start 2023-10-06T03:52:40.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.233+0000 7f680539d700 1 -- start start 2023-10-06T03:52:40.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.234+0000 7f680539d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6800156310 0x7f68000c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:40.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.234+0000 7f680539d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f68000ae710 0x7f68000c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:40.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.234+0000 7f680539d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68000ca860 con 0x7f68001566e0 2023-10-06T03:52:40.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.234+0000 7f680539d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68000cab40 con 0x7f68000ae710 2023-10-06T03:52:40.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.234+0000 7f680539d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68000cae20 con 0x7f6800156310 2023-10-06T03:52:40.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.234+0000 7f6804b9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f68000ae710 0x7f68000c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:40.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.234+0000 7f67ff7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f68001566e0 0x7f68000ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35254/0 (socket says 172.21.15.55:35254) 2023-10-06T03:52:40.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.234+0000 7f6804b9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f68000ae710 0x7f68000c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:49998/0 (socket says 172.21.15.55:49998) 2023-10-06T03:52:40.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.234+0000 7f67ff7fe700 1 -- 172.21.15.55:0/4224919189 learned_addr learned my addr 172.21.15.55:0/4224919189 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:40.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.235+0000 7f67fffff700 1 --2- 172.21.15.55:0/4224919189 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6800156310 0x7f68000c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:40.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.235+0000 7f6804b9c700 1 -- 172.21.15.55:0/4224919189 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6800156310 msgr2=0x7f68000c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:40.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.235+0000 7f6804b9c700 1 --2- 172.21.15.55:0/4224919189 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6800156310 0x7f68000c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.235+0000 7f6804b9c700 1 -- 172.21.15.55:0/4224919189 >> v1:172.21.15.55:6789/0 conn(0x7f68001566e0 legacy=0x7f68000ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:40.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.235+0000 7f6804b9c700 1 -- 172.21.15.55:0/4224919189 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68000cb100 con 0x7f68000ae710 2023-10-06T03:52:40.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.235+0000 7f67feffd700 1 -- 172.21.15.55:0/4224919189 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072287917 0 0) 0x7f68000ca860 con 0x7f68001566e0 2023-10-06T03:52:40.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.235+0000 7f6804b9c700 1 --2- 172.21.15.55:0/4224919189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68000ae710 0x7f68000c6830 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009e60 tx=0x7f67f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b46ceabde3b886a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:40.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.235+0000 7f67feffd700 1 -- 172.21.15.55:0/4224919189 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800e070 con 0x7f68000ae710 2023-10-06T03:52:40.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.235+0000 7f67feffd700 1 -- 172.21.15.55:0/4224919189 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f800bc30 con 0x7f68000ae710 2023-10-06T03:52:40.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.235+0000 7f67feffd700 1 -- 172.21.15.55:0/4224919189 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f8012430 con 0x7f68000ae710 2023-10-06T03:52:40.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.236+0000 7f680539d700 1 -- 172.21.15.55:0/4224919189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68000ae710 msgr2=0x7f68000c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:40.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.236+0000 7f680539d700 1 --2- 172.21.15.55:0/4224919189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68000ae710 0x7f68000c6830 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009e60 tx=0x7f67f8009780 comp rx=0 tx=0).stop 2023-10-06T03:52:40.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.236+0000 7f680539d700 1 -- 172.21.15.55:0/4224919189 shutdown_connections 2023-10-06T03:52:40.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.236+0000 7f680539d700 1 --2- 172.21.15.55:0/4224919189 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68000ae710 0x7f68000c6830 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.236+0000 7f680539d700 1 --2- 172.21.15.55:0/4224919189 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6800156310 0x7f68000c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.236+0000 7f680539d700 1 -- 172.21.15.55:0/4224919189 >> 172.21.15.55:0/4224919189 conn(0x7f68000cc1f0 msgr2=0x7f68000cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:40.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.236+0000 7f680539d700 1 -- 172.21.15.55:0/4224919189 shutdown_connections 2023-10-06T03:52:40.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.236+0000 7f680539d700 1 -- 172.21.15.55:0/4224919189 wait complete. 2023-10-06T03:52:40.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.237+0000 7f680539d700 1 Processor -- start 2023-10-06T03:52:40.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.238+0000 7f680539d700 1 -- start start 2023-10-06T03:52:40.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f680539d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68000ae710 0x7f6800156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:40.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f680539d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6800156310 0x7f6800142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:40.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f680539d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68001566e0 0x7f68000c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f680539d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68001571f0 con 0x7f6800156310 2023-10-06T03:52:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f680539d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68001588e0 con 0x7f68000ae710 2023-10-06T03:52:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f680539d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6800158bc0 con 0x7f68001566e0 2023-10-06T03:52:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f67ff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6800156310 0x7f6800142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f67ff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6800156310 0x7f6800142d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50006/0 (socket says 172.21.15.55:50006) 2023-10-06T03:52:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f67ff7fe700 1 -- 172.21.15.55:0/2211698124 learned_addr learned my addr 172.21.15.55:0/2211698124 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f67fffff700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68000ae710 0x7f6800156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f67ff7fe700 1 -- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68001566e0 msgr2=0x7f68000c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f6804b9c700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68001566e0 0x7f68000c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:40.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f67ff7fe700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68001566e0 0x7f68000c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f67ff7fe700 1 -- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68000ae710 msgr2=0x7f6800156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f67ff7fe700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68000ae710 0x7f6800156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f67ff7fe700 1 -- 172.21.15.55:0/2211698124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f8009a20 con 0x7f6800156310 2023-10-06T03:52:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f6804b9c700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68001566e0 0x7f68000c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.239+0000 7f67fffff700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68000ae710 0x7f6800156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T03:52:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.240+0000 7f67ff7fe700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6800156310 0x7f6800142d10 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f67f400c810 tx=0x7f67f4007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.240+0000 7f67e7fff700 1 -- 172.21.15.55:0/2211698124 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4005c00 con 0x7f6800156310 2023-10-06T03:52:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.240+0000 7f67e7fff700 1 -- 172.21.15.55:0/2211698124 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f4005d60 con 0x7f6800156310 2023-10-06T03:52:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.240+0000 7f680539d700 1 -- 172.21.15.55:0/2211698124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6800158ea0 con 0x7f6800156310 2023-10-06T03:52:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.240+0000 7f67e7fff700 1 -- 172.21.15.55:0/2211698124 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4007320 con 0x7f6800156310 2023-10-06T03:52:40.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.240+0000 7f680539d700 1 -- 172.21.15.55:0/2211698124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6800143250 con 0x7f6800156310 2023-10-06T03:52:40.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.241+0000 7f680539d700 1 -- 172.21.15.55:0/2211698124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68001593a0 con 0x7f6800156310 2023-10-06T03:52:40.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.242+0000 7f67e7fff700 1 -- 172.21.15.55:0/2211698124 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7f67f401b400 con 0x7f6800156310 2023-10-06T03:52:40.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.243+0000 7f67e7fff700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f67ec067050 0x7f67ec069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:40.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.243+0000 7f67e7fff700 1 -- 172.21.15.55:0/2211698124 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f67f4084510 con 0x7f6800156310 2023-10-06T03:52:40.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.243+0000 7f67fffff700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f67ec067050 0x7f67ec069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:40.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.244+0000 7f67fffff700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f67ec067050 0x7f67ec069500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f67f0006fd0 tx=0x7f67f0008040 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:40.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.247+0000 7f67e7fff700 1 -- 172.21.15.55:0/2211698124 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67f4051360 con 0x7f6800156310 2023-10-06T03:52:40.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.477+0000 7f680539d700 1 -- 172.21.15.55:0/2211698124 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f680004c3c0 con 0x7f67ec067050 2023-10-06T03:52:40.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.480+0000 7f67e7fff700 1 -- 172.21.15.55:0/2211698124 <== mgr.7690 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f680004c3c0 con 0x7f67ec067050 2023-10-06T03:52:40.511 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T03:52:40.511 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T03:52:40.511 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephfs" 2023-10-06T03:52:40.512 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T03:52:40.512 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T03:52:40.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.481+0000 7f680539d700 1 -- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f67ec067050 msgr2=0x7f67ec069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:40.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.481+0000 7f680539d700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f67ec067050 0x7f67ec069500 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f67f0006fd0 tx=0x7f67f0008040 comp rx=0 tx=0).stop 2023-10-06T03:52:40.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.481+0000 7f680539d700 1 -- 172.21.15.55:0/2211698124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6800156310 msgr2=0x7f6800142d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:40.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.481+0000 7f680539d700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6800156310 0x7f6800142d10 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f67f400c810 tx=0x7f67f4007db0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.481+0000 7f680539d700 1 -- 172.21.15.55:0/2211698124 shutdown_connections 2023-10-06T03:52:40.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.481+0000 7f680539d700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f67ec067050 0x7f67ec069500 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.481+0000 7f680539d700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68001566e0 0x7f68000c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.481+0000 7f680539d700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68000ae710 0x7f6800156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.481+0000 7f680539d700 1 --2- 172.21.15.55:0/2211698124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6800156310 0x7f6800142d10 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.481+0000 7f680539d700 1 -- 172.21.15.55:0/2211698124 >> 172.21.15.55:0/2211698124 conn(0x7f68000cc1f0 msgr2=0x7f68000c9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:40.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.481+0000 7f680539d700 1 -- 172.21.15.55:0/2211698124 shutdown_connections 2023-10-06T03:52:40.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.482+0000 7f680539d700 1 -- 172.21.15.55:0/2211698124 wait complete. 2023-10-06T03:52:40.515 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T03:52:40.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.703+0000 7fefc199d700 1 Processor -- start 2023-10-06T03:52:40.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.703+0000 7fefc199d700 1 -- start start 2023-10-06T03:52:40.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.705+0000 7fefc199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefbc0cb1a0 0x7fefbc15a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:40.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.705+0000 7fefc199d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fefbc15a8a0 0x7fefbc06f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:40.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.705+0000 7fefc199d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefbc06ebd0 con 0x7fefbc06f5a0 2023-10-06T03:52:40.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.705+0000 7fefc199d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefbc15b2e0 con 0x7fefbc15a8a0 2023-10-06T03:52:40.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.705+0000 7fefc199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc15ad20 con 0x7fefbc0cb1a0 2023-10-06T03:52:40.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.705+0000 7fefbbfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fefbc15a8a0 0x7fefbc06f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:40.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.705+0000 7fefc119c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fefbc06f5a0 0x7fefbc0ae0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:39742/0 (socket says 172.21.15.55:39742) 2023-10-06T03:52:40.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.705+0000 7fefc119c700 1 -- 172.21.15.55:0/2315139917 learned_addr learned my addr 172.21.15.55:0/2315139917 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:40.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.705+0000 7fefc099b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefbc0cb1a0 0x7fefbc15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:40.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.706+0000 7fefbbfff700 1 -- 172.21.15.55:0/2315139917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefbc0cb1a0 msgr2=0x7fefbc15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:40.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.706+0000 7fefbbfff700 1 --2- 172.21.15.55:0/2315139917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefbc0cb1a0 0x7fefbc15a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.706+0000 7fefbbfff700 1 -- 172.21.15.55:0/2315139917 >> v1:172.21.15.163:6789/0 conn(0x7fefbc06f5a0 legacy=0x7fefbc0ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:40.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.706+0000 7fefbbfff700 1 -- 172.21.15.55:0/2315139917 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc15b000 con 0x7fefbc15a8a0 2023-10-06T03:52:40.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.706+0000 7fefbbfff700 1 --2- 172.21.15.55:0/2315139917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefbc15a8a0 0x7fefbc06f060 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fefa80098e0 tx=0x7fefa80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbf7886c006087cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:40.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.706+0000 7fefbb7fe700 1 -- 172.21.15.55:0/2315139917 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa800b6d0 con 0x7fefbc15a8a0 2023-10-06T03:52:40.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.706+0000 7fefbb7fe700 1 -- 172.21.15.55:0/2315139917 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa800b830 con 0x7fefbc15a8a0 2023-10-06T03:52:40.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.706+0000 7fefbb7fe700 1 -- 172.21.15.55:0/2315139917 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa800bbd0 con 0x7fefbc15a8a0 2023-10-06T03:52:40.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.707+0000 7fefc199d700 1 -- 172.21.15.55:0/2315139917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefbc15a8a0 msgr2=0x7fefbc06f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:40.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.707+0000 7fefc199d700 1 --2- 172.21.15.55:0/2315139917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefbc15a8a0 0x7fefbc06f060 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fefa80098e0 tx=0x7fefa80091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.707+0000 7fefc199d700 1 -- 172.21.15.55:0/2315139917 shutdown_connections 2023-10-06T03:52:40.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.707+0000 7fefc199d700 1 --2- 172.21.15.55:0/2315139917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefbc0cb1a0 0x7fefbc15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.707+0000 7fefc199d700 1 --2- 172.21.15.55:0/2315139917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefbc15a8a0 0x7fefbc06f060 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.707+0000 7fefc199d700 1 -- 172.21.15.55:0/2315139917 >> 172.21.15.55:0/2315139917 conn(0x7fefbc0cc1e0 msgr2=0x7fefbc0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:40.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.707+0000 7fefc199d700 1 -- 172.21.15.55:0/2315139917 shutdown_connections 2023-10-06T03:52:40.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.707+0000 7fefc199d700 1 -- 172.21.15.55:0/2315139917 wait complete. 2023-10-06T03:52:40.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.709+0000 7fefc199d700 1 Processor -- start 2023-10-06T03:52:40.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.709+0000 7fefc199d700 1 -- start start 2023-10-06T03:52:40.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc199d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefbc06f5a0 0x7fefbc16b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:40.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc199d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefbc0cb1a0 0x7fefbc14eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:40.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefbc14e0b0 0x7fefbc0c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:40.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc199d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc14d7c0 con 0x7fefbc06f5a0 2023-10-06T03:52:40.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc199d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc169f60 con 0x7fefbc0cb1a0 2023-10-06T03:52:40.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc14cc40 con 0x7fefbc14e0b0 2023-10-06T03:52:40.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc099b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefbc06f5a0 0x7fefbc16b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:40.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc099b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefbc06f5a0 0x7fefbc16b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50014/0 (socket says 172.21.15.55:50014) 2023-10-06T03:52:40.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc099b700 1 -- 172.21.15.55:0/763031576 learned_addr learned my addr 172.21.15.55:0/763031576 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:40.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc119c700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefbc14e0b0 0x7fefbc0c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:40.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefbbfff700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefbc0cb1a0 0x7fefbc14eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:40.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc099b700 1 -- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefbc14e0b0 msgr2=0x7fefbc0c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:40.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc099b700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefbc14e0b0 0x7fefbc0c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc099b700 1 -- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefbc0cb1a0 msgr2=0x7fefbc14eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:40.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc099b700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefbc0cb1a0 0x7fefbc14eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:40.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefc099b700 1 -- 172.21.15.55:0/763031576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa8009420 con 0x7fefbc06f5a0 2023-10-06T03:52:40.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.710+0000 7fefbbfff700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefbc0cb1a0 0x7fefbc14eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:40.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.711+0000 7fefc099b700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefbc06f5a0 0x7fefbc16b190 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fefb000ec80 tx=0x7fefb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:40.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.711+0000 7fefb8ff9700 1 -- 172.21.15.55:0/763031576 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb000c850 con 0x7fefbc06f5a0 2023-10-06T03:52:40.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.711+0000 7fefb8ff9700 1 -- 172.21.15.55:0/763031576 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb000c9b0 con 0x7fefbc06f5a0 2023-10-06T03:52:40.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.711+0000 7fefb8ff9700 1 -- 172.21.15.55:0/763031576 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb000cd50 con 0x7fefbc06f5a0 2023-10-06T03:52:40.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.711+0000 7fefc199d700 1 -- 172.21.15.55:0/763031576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefbc14cf20 con 0x7fefbc06f5a0 2023-10-06T03:52:40.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.711+0000 7fefc199d700 1 -- 172.21.15.55:0/763031576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc14d4e0 con 0x7fefbc06f5a0 2023-10-06T03:52:40.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.712+0000 7fefc199d700 1 -- 172.21.15.55:0/763031576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefbc06c450 con 0x7fefbc06f5a0 2023-10-06T03:52:40.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.713+0000 7fefb8ff9700 1 -- 172.21.15.55:0/763031576 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 58) v1 ==== 84842+0+0 (secure 0 0 0) 0x7fefb0013680 con 0x7fefbc06f5a0 2023-10-06T03:52:40.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.714+0000 7fefb8ff9700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fefa0067050 0x7fefa0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:40.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.714+0000 7fefb8ff9700 1 -- 172.21.15.55:0/763031576 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fefb0084cd0 con 0x7fefbc06f5a0 2023-10-06T03:52:40.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.715+0000 7fefbbfff700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fefa0067050 0x7fefa0069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:40.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.716+0000 7fefbbfff700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fefa0067050 0x7fefa0069500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fefa80093f0 tx=0x7fefa8017000 comp rx=0 tx=0).ready entity=mgr.7690 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:40.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.719+0000 7fefb8ff9700 1 -- 172.21.15.55:0/763031576 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fefb0051aa0 con 0x7fefbc06f5a0 2023-10-06T03:52:40.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:40.982+0000 7fefc199d700 1 -- 172.21.15.55:0/763031576 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fefbc06e860 con 0x7fefbc06f5a0 2023-10-06T03:52:41.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:41.815+0000 7fefb8ff9700 1 -- 172.21.15.55:0/763031576 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 59) v1 ==== 84859+0+0 (secure 0 0 0) 0x7fefb0084020 con 0x7fefbc06f5a0 2023-10-06T03:52:41.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:41.817+0000 7fefb8ff9700 1 -- 172.21.15.55:0/763031576 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v59) v1 ==== 92+0+0 (secure 0 0 0) 0x7fefb0051aa0 con 0x7fefbc06f5a0 2023-10-06T03:52:41.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:41.820+0000 7fefc199d700 1 -- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fefa0067050 msgr2=0x7fefa0069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:41.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:41.820+0000 7fefc199d700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fefa0067050 0x7fefa0069500 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fefa80093f0 tx=0x7fefa8017000 comp rx=0 tx=0).stop 2023-10-06T03:52:41.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:41.820+0000 7fefc199d700 1 -- 172.21.15.55:0/763031576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefbc06f5a0 msgr2=0x7fefbc16b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:41.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:41.820+0000 7fefc199d700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefbc06f5a0 0x7fefbc16b190 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fefb000ec80 tx=0x7fefb00091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:41.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:41.820+0000 7fefc199d700 1 -- 172.21.15.55:0/763031576 shutdown_connections 2023-10-06T03:52:41.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:41.820+0000 7fefc199d700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fefa0067050 0x7fefa0069500 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:41.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:41.820+0000 7fefc199d700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fefbc14e0b0 0x7fefbc0c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:41.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:41.820+0000 7fefc199d700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fefbc06f5a0 0x7fefbc16b190 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:41.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:41.820+0000 7fefc199d700 1 --2- 172.21.15.55:0/763031576 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fefbc0cb1a0 0x7fefbc14eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:41.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:41.820+0000 7fefc199d700 1 -- 172.21.15.55:0/763031576 >> 172.21.15.55:0/763031576 conn(0x7fefbc0cc1e0 msgr2=0x7fefbc14bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:41.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:41.820+0000 7fefc199d700 1 -- 172.21.15.55:0/763031576 shutdown_connections 2023-10-06T03:52:41.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:41.820+0000 7fefc199d700 1 -- 172.21.15.55:0/763031576 wait complete. 2023-10-06T03:52:41.927 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T03:52:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.119+0000 7f9d1759e700 1 Processor -- start 2023-10-06T03:52:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.119+0000 7f9d1759e700 1 -- start start 2023-10-06T03:52:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.120+0000 7f9d1759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9d180b4e70 0x7f9d180ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.120+0000 7f9d1759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9d180b16d0 0x7f9d180b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.120+0000 7f9d1759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d1815d240 0x7f9d1815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.120+0000 7f9d1759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d180b2080 con 0x7f9d180b4e70 2023-10-06T03:52:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.120+0000 7f9d1759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d180cfb00 con 0x7f9d180b16d0 2023-10-06T03:52:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.120+0000 7f9d1759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180b5c40 con 0x7f9d1815d240 2023-10-06T03:52:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.120+0000 7f9d1659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9d180b4e70 0x7f9d180ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.120+0000 7f9d1659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9d180b4e70 0x7f9d180ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50042/0 (socket says 172.21.15.55:50042) 2023-10-06T03:52:42.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.120+0000 7f9d1659c700 1 -- 172.21.15.55:0/3007604304 learned_addr learned my addr 172.21.15.55:0/3007604304 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:42.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.120+0000 7f9d15d9b700 1 --2- 172.21.15.55:0/3007604304 >> v2:172.21.15.163:3300/0 conn(0x7f9d180b16d0 0x7f9d180b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:42.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.120+0000 7f9d16d9d700 1 --2- 172.21.15.55:0/3007604304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d1815d240 0x7f9d1815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:42.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.121+0000 7f9d1659c700 1 -- 172.21.15.55:0/3007604304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d1815d240 msgr2=0x7f9d1815d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:42.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.121+0000 7f9d1659c700 1 --2- 172.21.15.55:0/3007604304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d1815d240 0x7f9d1815d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.121+0000 7f9d1659c700 1 -- 172.21.15.55:0/3007604304 >> v2:172.21.15.163:3300/0 conn(0x7f9d180b16d0 msgr2=0x7f9d180b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:42.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.121+0000 7f9d1659c700 1 --2- 172.21.15.55:0/3007604304 >> v2:172.21.15.163:3300/0 conn(0x7f9d180b16d0 0x7f9d180b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.121+0000 7f9d1659c700 1 -- 172.21.15.55:0/3007604304 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d180b5f20 con 0x7f9d180b4e70 2023-10-06T03:52:42.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.121+0000 7f9d1659c700 1 --2- 172.21.15.55:0/3007604304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d180b4e70 0x7f9d180ccff0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f9d18089830 tx=0x7f9d08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69136cd2504e9eba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:42.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.121+0000 7f9d1559a700 1 -- 172.21.15.55:0/3007604304 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800b640 con 0x7f9d180b4e70 2023-10-06T03:52:42.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.121+0000 7f9d1559a700 1 -- 172.21.15.55:0/3007604304 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0800b7a0 con 0x7f9d180b4e70 2023-10-06T03:52:42.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.121+0000 7f9d1559a700 1 -- 172.21.15.55:0/3007604304 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800bb60 con 0x7f9d180b4e70 2023-10-06T03:52:42.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.122+0000 7f9d1759e700 1 -- 172.21.15.55:0/3007604304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d180b4e70 msgr2=0x7f9d180ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:42.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.122+0000 7f9d1759e700 1 --2- 172.21.15.55:0/3007604304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d180b4e70 0x7f9d180ccff0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f9d18089830 tx=0x7f9d08009250 comp rx=0 tx=0).stop 2023-10-06T03:52:42.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.122+0000 7f9d1759e700 1 -- 172.21.15.55:0/3007604304 shutdown_connections 2023-10-06T03:52:42.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.122+0000 7f9d1759e700 1 --2- 172.21.15.55:0/3007604304 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d1815d240 0x7f9d1815d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.122+0000 7f9d1759e700 1 --2- 172.21.15.55:0/3007604304 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d180b4e70 0x7f9d180ccff0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.122+0000 7f9d1759e700 1 --2- 172.21.15.55:0/3007604304 >> v2:172.21.15.163:3300/0 conn(0x7f9d180b16d0 0x7f9d180b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.122+0000 7f9d1759e700 1 -- 172.21.15.55:0/3007604304 >> 172.21.15.55:0/3007604304 conn(0x7f9d180d10c0 msgr2=0x7f9d180affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:42.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.122+0000 7f9d1759e700 1 -- 172.21.15.55:0/3007604304 shutdown_connections 2023-10-06T03:52:42.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.122+0000 7f9d1759e700 1 -- 172.21.15.55:0/3007604304 wait complete. 2023-10-06T03:52:42.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.124+0000 7f9d1759e700 1 Processor -- start 2023-10-06T03:52:42.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.124+0000 7f9d1759e700 1 -- start start 2023-10-06T03:52:42.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.125+0000 7f9d1759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d180b16d0 0x7f9d180c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:42.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.125+0000 7f9d1759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d180b4e70 0x7f9d18145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:42.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.125+0000 7f9d1759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1815d240 0x7f9d180c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:42.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.125+0000 7f9d1759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180c7f20 con 0x7f9d1815d240 2023-10-06T03:52:42.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.125+0000 7f9d1759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180c7b60 con 0x7f9d180b16d0 2023-10-06T03:52:42.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.125+0000 7f9d1759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180c6940 con 0x7f9d180b4e70 2023-10-06T03:52:42.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.125+0000 7f9d16d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1815d240 0x7f9d180c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:42.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.125+0000 7f9d1659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d180b16d0 0x7f9d180c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:42.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.125+0000 7f9d1659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d180b16d0 0x7f9d180c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:55488/0 (socket says 172.21.15.55:55488) 2023-10-06T03:52:42.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.125+0000 7f9d16d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1815d240 0x7f9d180c7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50046/0 (socket says 172.21.15.55:50046) 2023-10-06T03:52:42.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.125+0000 7f9d16d9d700 1 -- 172.21.15.55:0/2440974602 learned_addr learned my addr 172.21.15.55:0/2440974602 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:42.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.125+0000 7f9d15d9b700 1 --2- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d180b4e70 0x7f9d18145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:42.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.126+0000 7f9d16d9d700 1 -- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d180b4e70 msgr2=0x7f9d18145eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:42.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.126+0000 7f9d16d9d700 1 --2- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d180b4e70 0x7f9d18145eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.126+0000 7f9d16d9d700 1 -- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d180b16d0 msgr2=0x7f9d180c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:42.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.126+0000 7f9d16d9d700 1 --2- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d180b16d0 0x7f9d180c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.126+0000 7f9d16d9d700 1 -- 172.21.15.55:0/2440974602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d080094f0 con 0x7f9d1815d240 2023-10-06T03:52:42.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.126+0000 7f9d16d9d700 1 --2- 172.21.15.55:0/2440974602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1815d240 0x7f9d180c7620 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009ec0 tx=0x7f9d1000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:42.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.126+0000 7f9d067fc700 1 -- 172.21.15.55:0/2440974602 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1000cb70 con 0x7f9d1815d240 2023-10-06T03:52:42.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.126+0000 7f9d067fc700 1 -- 172.21.15.55:0/2440974602 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1000f040 con 0x7f9d1815d240 2023-10-06T03:52:42.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.126+0000 7f9d1759e700 1 -- 172.21.15.55:0/2440974602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d180c6c20 con 0x7f9d1815d240 2023-10-06T03:52:42.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.126+0000 7f9d067fc700 1 -- 172.21.15.55:0/2440974602 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10014480 con 0x7f9d1815d240 2023-10-06T03:52:42.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.126+0000 7f9d1759e700 1 -- 172.21.15.55:0/2440974602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d180c71e0 con 0x7f9d1815d240 2023-10-06T03:52:42.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.127+0000 7f9d1759e700 1 -- 172.21.15.55:0/2440974602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d18088990 con 0x7f9d1815d240 2023-10-06T03:52:42.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.128+0000 7f9d067fc700 1 -- 172.21.15.55:0/2440974602 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 59) v1 ==== 84859+0+0 (secure 0 0 0) 0x7f9d10014830 con 0x7f9d1815d240 2023-10-06T03:52:42.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.130+0000 7f9d067fc700 1 --2- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cf80670a0 0x7f9cf8069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:42.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.130+0000 7f9d067fc700 1 -- 172.21.15.55:0/2440974602 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f9d10085410 con 0x7f9d1815d240 2023-10-06T03:52:42.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.130+0000 7f9d1659c700 1 -- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cf80670a0 msgr2=0x7f9cf8069550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:42.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.130+0000 7f9d1659c700 1 --2- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cf80670a0 0x7f9cf8069550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:52:42.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.134+0000 7f9d067fc700 1 -- 172.21.15.55:0/2440974602 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d100520d0 con 0x7f9d1815d240 2023-10-06T03:52:42.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.331+0000 7f9d1659c700 1 -- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cf80670a0 msgr2=0x7f9cf8069550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:42.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.331+0000 7f9d1659c700 1 --2- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cf80670a0 0x7f9cf8069550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:52:42.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.338+0000 7f9d1759e700 1 -- 172.21.15.55:0/2440974602 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f9d180c8060 con 0x7f9d1815d240 2023-10-06T03:52:42.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.346+0000 7f9d067fc700 1 -- 172.21.15.55:0/2440974602 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v47)=0 v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9d100556f0 con 0x7f9d1815d240 2023-10-06T03:52:42.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.348+0000 7f9d1759e700 1 -- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cf80670a0 msgr2=0x7f9cf8069550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:52:42.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.348+0000 7f9d1759e700 1 --2- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cf80670a0 0x7f9cf8069550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.349+0000 7f9d1759e700 1 -- 172.21.15.55:0/2440974602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1815d240 msgr2=0x7f9d180c7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:42.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.349+0000 7f9d1759e700 1 --2- 172.21.15.55:0/2440974602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1815d240 0x7f9d180c7620 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009ec0 tx=0x7f9d1000b040 comp rx=0 tx=0).stop 2023-10-06T03:52:42.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.349+0000 7f9d1759e700 1 -- 172.21.15.55:0/2440974602 shutdown_connections 2023-10-06T03:52:42.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.349+0000 7f9d1759e700 1 --2- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f9cf80670a0 0x7f9cf8069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.349+0000 7f9d1759e700 1 --2- 172.21.15.55:0/2440974602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d1815d240 0x7f9d180c7620 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.349+0000 7f9d1759e700 1 --2- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d180b16d0 0x7f9d180c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.349+0000 7f9d1759e700 1 --2- 172.21.15.55:0/2440974602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d180b4e70 0x7f9d18145eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.349+0000 7f9d1759e700 1 -- 172.21.15.55:0/2440974602 >> 172.21.15.55:0/2440974602 conn(0x7f9d180d10c0 msgr2=0x7f9d18068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:42.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.349+0000 7f9d1759e700 1 -- 172.21.15.55:0/2440974602 shutdown_connections 2023-10-06T03:52:42.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.349+0000 7f9d1759e700 1 -- 172.21.15.55:0/2440974602 wait complete. 2023-10-06T03:52:42.378 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T03:52:42.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.569+0000 7f43f03ff700 1 Processor -- start 2023-10-06T03:52:42.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.570+0000 7f43f03ff700 1 -- start start 2023-10-06T03:52:42.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.571+0000 7f43f03ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43e8159c60 0x7f43e815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:42.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.571+0000 7f43f03ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f43e815db10 0x7f43e8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:42.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.571+0000 7f43f03ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e815a160 con 0x7f43e8159890 2023-10-06T03:52:42.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.571+0000 7f43f03ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e80d13a0 con 0x7f43e815db10 2023-10-06T03:52:42.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.571+0000 7f43f03ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e815e140 con 0x7f43e8159c60 2023-10-06T03:52:42.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.571+0000 7f43ee99c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f43e815db10 0x7f43e8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:42.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.571+0000 7f43ed99a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43e8159c60 0x7f43e815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:42.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.571+0000 7f43ee19b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f43e8159890 0x7f43e815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:39770/0 (socket says 172.21.15.55:39770) 2023-10-06T03:52:42.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.571+0000 7f43ed99a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43e8159c60 0x7f43e815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:39788/0 (socket says 172.21.15.55:39788) 2023-10-06T03:52:42.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.571+0000 7f43ee99c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f43e815db10 0x7f43e8069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:55490/0 (socket says 172.21.15.55:55490) 2023-10-06T03:52:42.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.571+0000 7f43ee19b700 1 -- 172.21.15.55:0/1493334819 learned_addr learned my addr 172.21.15.55:0/1493334819 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:42.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.572+0000 7f43ed199700 1 -- 172.21.15.55:0/1493334819 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103525380 0 0) 0x7f43e815a160 con 0x7f43e8159890 2023-10-06T03:52:42.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.572+0000 7f43ed199700 1 -- 172.21.15.55:0/1493334819 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43d4003570 con 0x7f43e8159890 2023-10-06T03:52:42.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.572+0000 7f43ed99a700 1 -- 172.21.15.55:0/1493334819 >> v2:172.21.15.163:3300/0 conn(0x7f43e815db10 msgr2=0x7f43e8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:42.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.572+0000 7f43ed99a700 1 --2- 172.21.15.55:0/1493334819 >> v2:172.21.15.163:3300/0 conn(0x7f43e815db10 0x7f43e8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.572+0000 7f43ed99a700 1 -- 172.21.15.55:0/1493334819 >> v1:172.21.15.163:6789/0 conn(0x7f43e8159890 legacy=0x7f43e815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:42.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.572+0000 7f43ed99a700 1 -- 172.21.15.55:0/1493334819 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e815a3d0 con 0x7f43e8159c60 2023-10-06T03:52:42.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.572+0000 7f43ee99c700 1 --2- 172.21.15.55:0/1493334819 >> v2:172.21.15.163:3300/0 conn(0x7f43e815db10 0x7f43e8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:52:42.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.572+0000 7f43ed99a700 1 --2- 172.21.15.55:0/1493334819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43e8159c60 0x7f43e815d5d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f43d80098e0 tx=0x7f43d80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9d5e729ad23228c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:42.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.573+0000 7f43ed199700 1 -- 172.21.15.55:0/1493334819 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8011630 con 0x7f43e8159c60 2023-10-06T03:52:42.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.573+0000 7f43ed199700 1 -- 172.21.15.55:0/1493334819 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d8011790 con 0x7f43e8159c60 2023-10-06T03:52:42.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.573+0000 7f43ed199700 1 -- 172.21.15.55:0/1493334819 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d8011b10 con 0x7f43e8159c60 2023-10-06T03:52:42.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.573+0000 7f43f03ff700 1 -- 172.21.15.55:0/1493334819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43e8159c60 msgr2=0x7f43e815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:42.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.573+0000 7f43f03ff700 1 --2- 172.21.15.55:0/1493334819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43e8159c60 0x7f43e815d5d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f43d80098e0 tx=0x7f43d80091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.573+0000 7f43f03ff700 1 -- 172.21.15.55:0/1493334819 shutdown_connections 2023-10-06T03:52:42.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.573+0000 7f43f03ff700 1 --2- 172.21.15.55:0/1493334819 >> v2:172.21.15.163:3300/0 conn(0x7f43e815db10 0x7f43e8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.573+0000 7f43f03ff700 1 --2- 172.21.15.55:0/1493334819 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43e8159c60 0x7f43e815d5d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.573+0000 7f43f03ff700 1 -- 172.21.15.55:0/1493334819 >> 172.21.15.55:0/1493334819 conn(0x7f43e8068e20 msgr2=0x7f43e80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:42.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.573+0000 7f43f03ff700 1 -- 172.21.15.55:0/1493334819 shutdown_connections 2023-10-06T03:52:42.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.573+0000 7f43f03ff700 1 -- 172.21.15.55:0/1493334819 wait complete. 2023-10-06T03:52:42.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.575+0000 7f43f03ff700 1 Processor -- start 2023-10-06T03:52:42.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.575+0000 7f43f03ff700 1 -- start start 2023-10-06T03:52:42.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.576+0000 7f43f03ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e8159890 0x7f43e814d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.576+0000 7f43f03ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43e8159c60 0x7f43e80b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.576+0000 7f43f03ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43e815db10 0x7f43e8169bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.576+0000 7f43f03ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e814d920 con 0x7f43e8159890 2023-10-06T03:52:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.576+0000 7f43f03ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e80c8df0 con 0x7f43e8159c60 2023-10-06T03:52:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.576+0000 7f43f03ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e80b5bd0 con 0x7f43e815db10 2023-10-06T03:52:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.576+0000 7f43ee19b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e8159890 0x7f43e814d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.576+0000 7f43ee99c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43e815db10 0x7f43e8169bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:42.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.576+0000 7f43ed99a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43e8159c60 0x7f43e80b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.576+0000 7f43ee19b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e8159890 0x7f43e814d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50062/0 (socket says 172.21.15.55:50062) 2023-10-06T03:52:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.576+0000 7f43ee19b700 1 -- 172.21.15.55:0/4261675456 learned_addr learned my addr 172.21.15.55:0/4261675456 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.577+0000 7f43ee99c700 1 -- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43e8159c60 msgr2=0x7f43e80b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.577+0000 7f43ee99c700 1 --2- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43e8159c60 0x7f43e80b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.577+0000 7f43ee99c700 1 -- 172.21.15.55:0/4261675456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e8159890 msgr2=0x7f43e814d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.577+0000 7f43ee99c700 1 --2- 172.21.15.55:0/4261675456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e8159890 0x7f43e814d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.577+0000 7f43ee99c700 1 -- 172.21.15.55:0/4261675456 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e0009a20 con 0x7f43e815db10 2023-10-06T03:52:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.577+0000 7f43ee99c700 1 --2- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43e815db10 0x7f43e8169bc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009e70 tx=0x7f43e000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:42.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.577+0000 7f43de7fc700 1 -- 172.21.15.55:0/4261675456 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e000caa0 con 0x7f43e815db10 2023-10-06T03:52:42.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.577+0000 7f43de7fc700 1 -- 172.21.15.55:0/4261675456 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e000f040 con 0x7f43e815db10 2023-10-06T03:52:42.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.577+0000 7f43f03ff700 1 -- 172.21.15.55:0/4261675456 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d8009420 con 0x7f43e815db10 2023-10-06T03:52:42.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.577+0000 7f43de7fc700 1 -- 172.21.15.55:0/4261675456 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e000ce90 con 0x7f43e815db10 2023-10-06T03:52:42.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.577+0000 7f43f03ff700 1 -- 172.21.15.55:0/4261675456 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e80c8a80 con 0x7f43e815db10 2023-10-06T03:52:42.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.578+0000 7f43f03ff700 1 -- 172.21.15.55:0/4261675456 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e80c8f60 con 0x7f43e815db10 2023-10-06T03:52:42.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.579+0000 7f43de7fc700 1 -- 172.21.15.55:0/4261675456 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 59) v1 ==== 84859+0+0 (secure 0 0 0) 0x7f43e0003020 con 0x7f43e815db10 2023-10-06T03:52:42.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.580+0000 7f43de7fc700 1 --2- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f43cc0670a0 0x7f43cc069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:42.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.580+0000 7f43de7fc700 1 -- 172.21.15.55:0/4261675456 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f43e0085370 con 0x7f43e815db10 2023-10-06T03:52:42.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.580+0000 7f43ee19b700 1 -- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f43cc0670a0 msgr2=0x7f43cc069550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:42.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.580+0000 7f43ee19b700 1 --2- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f43cc0670a0 0x7f43cc069550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:52:42.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.585+0000 7f43de7fc700 1 -- 172.21.15.55:0/4261675456 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43e0052030 con 0x7f43e815db10 2023-10-06T03:52:42.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.781+0000 7f43ee19b700 1 -- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f43cc0670a0 msgr2=0x7f43cc069550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:42.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.781+0000 7f43ee19b700 1 --2- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f43cc0670a0 0x7f43cc069550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:52:42.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.789+0000 7f43f03ff700 1 -- 172.21.15.55:0/4261675456 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f43e804c3c0 con 0x7f43e815db10 2023-10-06T03:52:42.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.799+0000 7f43de7fc700 1 -- 172.21.15.55:0/4261675456 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v48)=0 v48) v1 ==== 133+0+0 (secure 0 0 0) 0x7f43e0055650 con 0x7f43e815db10 2023-10-06T03:52:42.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.801+0000 7f43f03ff700 1 -- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f43cc0670a0 msgr2=0x7f43cc069550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:52:42.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.801+0000 7f43f03ff700 1 --2- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f43cc0670a0 0x7f43cc069550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.801+0000 7f43f03ff700 1 -- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43e815db10 msgr2=0x7f43e8169bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:42.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.801+0000 7f43f03ff700 1 --2- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43e815db10 0x7f43e8169bc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009e70 tx=0x7f43e000b040 comp rx=0 tx=0).stop 2023-10-06T03:52:42.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.801+0000 7f43f03ff700 1 -- 172.21.15.55:0/4261675456 shutdown_connections 2023-10-06T03:52:42.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.801+0000 7f43f03ff700 1 --2- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f43cc0670a0 0x7f43cc069550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.801+0000 7f43f03ff700 1 --2- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f43e815db10 0x7f43e8169bc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.801+0000 7f43f03ff700 1 --2- 172.21.15.55:0/4261675456 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f43e8159890 0x7f43e814d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.801+0000 7f43f03ff700 1 --2- 172.21.15.55:0/4261675456 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f43e8159c60 0x7f43e80b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:42.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.801+0000 7f43f03ff700 1 -- 172.21.15.55:0/4261675456 >> 172.21.15.55:0/4261675456 conn(0x7f43e8068e20 msgr2=0x7f43e815b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:42.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.801+0000 7f43f03ff700 1 -- 172.21.15.55:0/4261675456 shutdown_connections 2023-10-06T03:52:42.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:42.801+0000 7f43f03ff700 1 -- 172.21.15.55:0/4261675456 wait complete. 2023-10-06T03:52:42.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T03:52:42.832 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T03:52:42.881 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1 1M 2023-10-06T03:52:43.107 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:43.106+0000 7f67c1693180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:52:43.323 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:43.270+0000 7f67c1693180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:52:43.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.115+0000 7f5764d9d700 1 Processor -- start 2023-10-06T03:52:43.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.115+0000 7f5764d9d700 1 -- start start 2023-10-06T03:52:43.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.116+0000 7f5764d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f57600b0790 0x7f57600cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:43.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.116+0000 7f5764d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f57601591e0 0x7f5760159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:43.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.117+0000 7f5764d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57600cfa00 con 0x7f57600b03c0 2023-10-06T03:52:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.117+0000 7f5764d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57600cb580 con 0x7f57601591e0 2023-10-06T03:52:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.117+0000 7f5764d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57600cf160 con 0x7f57600b0790 2023-10-06T03:52:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.117+0000 7f575ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f57601591e0 0x7f5760159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.117+0000 7f575ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f57601591e0 0x7f5760159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46122/0 (socket says 172.21.15.55:46122) 2023-10-06T03:52:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.117+0000 7f575ffff700 1 -- 172.21.15.55:0/257661983 learned_addr learned my addr 172.21.15.55:0/257661983 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.117+0000 7f575effd700 1 --2- 172.21.15.55:0/257661983 >> v2:172.21.15.163:3300/0 conn(0x7f57600b0790 0x7f57600cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.117+0000 7f575ffff700 1 -- 172.21.15.55:0/257661983 >> v2:172.21.15.163:3300/0 conn(0x7f57600b0790 msgr2=0x7f57600cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.117+0000 7f575ffff700 1 --2- 172.21.15.55:0/257661983 >> v2:172.21.15.163:3300/0 conn(0x7f57600b0790 0x7f57600cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.117+0000 7f575ffff700 1 -- 172.21.15.55:0/257661983 >> v1:172.21.15.163:6789/0 conn(0x7f57600b03c0 legacy=0x7f57600b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:43.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.117+0000 7f575ffff700 1 -- 172.21.15.55:0/257661983 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57600cf440 con 0x7f57601591e0 2023-10-06T03:52:43.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.117+0000 7f575e7fc700 1 -- 172.21.15.55:0/257661983 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309535320 0 0) 0x7f57600cfa00 con 0x7f57600b03c0 2023-10-06T03:52:43.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.118+0000 7f575ffff700 1 --2- 172.21.15.55:0/257661983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57601591e0 0x7f5760159600 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f5758009f20 tx=0x7f5758009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5264c56380b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:43.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.118+0000 7f575e7fc700 1 -- 172.21.15.55:0/257661983 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575800e070 con 0x7f57601591e0 2023-10-06T03:52:43.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.118+0000 7f575e7fc700 1 -- 172.21.15.55:0/257661983 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575800b990 con 0x7f57601591e0 2023-10-06T03:52:43.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.118+0000 7f575e7fc700 1 -- 172.21.15.55:0/257661983 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580092e0 con 0x7f57601591e0 2023-10-06T03:52:43.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.119+0000 7f5764d9d700 1 -- 172.21.15.55:0/257661983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57601591e0 msgr2=0x7f5760159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:43.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.119+0000 7f5764d9d700 1 --2- 172.21.15.55:0/257661983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57601591e0 0x7f5760159600 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f5758009f20 tx=0x7f5758009840 comp rx=0 tx=0).stop 2023-10-06T03:52:43.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.119+0000 7f5764d9d700 1 -- 172.21.15.55:0/257661983 shutdown_connections 2023-10-06T03:52:43.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.119+0000 7f5764d9d700 1 --2- 172.21.15.55:0/257661983 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57601591e0 0x7f5760159600 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:43.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.119+0000 7f5764d9d700 1 --2- 172.21.15.55:0/257661983 >> v2:172.21.15.163:3300/0 conn(0x7f57600b0790 0x7f57600cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:43.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.119+0000 7f5764d9d700 1 -- 172.21.15.55:0/257661983 >> 172.21.15.55:0/257661983 conn(0x7f5760063230 msgr2=0x7f5760063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:43.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.119+0000 7f5764d9d700 1 -- 172.21.15.55:0/257661983 shutdown_connections 2023-10-06T03:52:43.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.119+0000 7f5764d9d700 1 -- 172.21.15.55:0/257661983 wait complete. 2023-10-06T03:52:43.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.120+0000 7f5764d9d700 1 Processor -- start 2023-10-06T03:52:43.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.121+0000 7f5764d9d700 1 -- start start 2023-10-06T03:52:43.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f5764d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57600b03c0 0x7f576015a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:43.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f5764d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57600b0790 0x7f57600c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:43.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f5764d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57601591e0 0x7f5760157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:43.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f5764d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576016d9e0 con 0x7f57600b0790 2023-10-06T03:52:43.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f5764d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57600c5700 con 0x7f57600b03c0 2023-10-06T03:52:43.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f5764d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576015c780 con 0x7f57601591e0 2023-10-06T03:52:43.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f575effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57600b0790 0x7f57600c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:43.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f575effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57600b0790 0x7f57600c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46138/0 (socket says 172.21.15.55:46138) 2023-10-06T03:52:43.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f575effd700 1 -- 172.21.15.55:0/2306711356 learned_addr learned my addr 172.21.15.55:0/2306711356 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:43.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f575ffff700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57601591e0 0x7f5760157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:43.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f575f7fe700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57600b03c0 0x7f576015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:43.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f575effd700 1 -- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57601591e0 msgr2=0x7f5760157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:43.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f575effd700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57601591e0 0x7f5760157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:43.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f575effd700 1 -- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57600b03c0 msgr2=0x7f576015a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:43.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f575effd700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57600b03c0 0x7f576015a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:43.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f575effd700 1 -- 172.21.15.55:0/2306711356 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5758009ae0 con 0x7f57600b0790 2023-10-06T03:52:43.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.122+0000 7f575ffff700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57601591e0 0x7f5760157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T03:52:43.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.123+0000 7f575effd700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57600b0790 0x7f57600c8c40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f575400ec80 tx=0x7f57540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:43.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.123+0000 7f57477fe700 1 -- 172.21.15.55:0/2306711356 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575400c850 con 0x7f57600b0790 2023-10-06T03:52:43.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.123+0000 7f57477fe700 1 -- 172.21.15.55:0/2306711356 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575400c9b0 con 0x7f57600b0790 2023-10-06T03:52:43.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.123+0000 7f5764d9d700 1 -- 172.21.15.55:0/2306711356 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576016ce60 con 0x7f57600b0790 2023-10-06T03:52:43.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.123+0000 7f57477fe700 1 -- 172.21.15.55:0/2306711356 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575400cd50 con 0x7f57600b0790 2023-10-06T03:52:43.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.123+0000 7f5764d9d700 1 -- 172.21.15.55:0/2306711356 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576016d1f0 con 0x7f57600b0790 2023-10-06T03:52:43.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.124+0000 7f5764d9d700 1 -- 172.21.15.55:0/2306711356 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576016d820 con 0x7f57600b0790 2023-10-06T03:52:43.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.125+0000 7f57477fe700 1 -- 172.21.15.55:0/2306711356 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 59) v1 ==== 84859+0+0 (secure 0 0 0) 0x7f5754013680 con 0x7f57600b0790 2023-10-06T03:52:43.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.126+0000 7f57477fe700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 0x7f574c069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:43.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.126+0000 7f57477fe700 1 -- 172.21.15.55:0/2306711356 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f5754084cf0 con 0x7f57600b0790 2023-10-06T03:52:43.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.126+0000 7f575f7fe700 1 -- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 msgr2=0x7f574c069550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:43.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.126+0000 7f575f7fe700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 0x7f574c069550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T03:52:43.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.130+0000 7f57477fe700 1 -- 172.21.15.55:0/2306711356 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5754051ac0 con 0x7f57600b0790 2023-10-06T03:52:43.353 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:43.181+0000 7f060ccd9180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T03:52:43.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.327+0000 7f575f7fe700 1 -- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 msgr2=0x7f574c069550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:43.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.327+0000 7f575f7fe700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 0x7f574c069550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T03:52:43.355 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:43.353+0000 7f060ccd9180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T03:52:43.356 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:43.355+0000 7f67c1693180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:52:43.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.360+0000 7f5764d9d700 1 -- 172.21.15.55:0/2306711356 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f576004c3c0 con 0x7f574c0670a0 2023-10-06T03:52:43.444 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:43.443+0000 7f060ccd9180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T03:52:43.581 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:43.580+0000 7f67c1693180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:52:43.677 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:43.675+0000 7f060ccd9180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:52:43.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.728+0000 7f575f7fe700 1 -- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 msgr2=0x7f574c069550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:43.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:43.728+0000 7f575f7fe700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 0x7f574c069550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T03:52:43.773 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:43.756+0000 7f67c1693180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:52:43.849 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:43.848+0000 7f67c1693180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:52:43.913 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:43.859+0000 7f060ccd9180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T03:52:43.956 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:43.955+0000 7f060ccd9180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T03:52:44.010 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:44.009+0000 7f67c1693180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:52:44.110 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:44.109+0000 7f67c1693180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:52:44.125 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:44.121+0000 7f060ccd9180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T03:52:44.228 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:44.226+0000 7f060ccd9180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T03:52:44.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:44.528+0000 7f575f7fe700 1 -- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 msgr2=0x7f574c069550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:44.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:44.528+0000 7f575f7fe700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 0x7f574c069550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T03:52:44.593 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:44.591+0000 7f67c1693180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:52:44.685 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:44.684+0000 7f67c1693180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:52:44.732 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:44.729+0000 7f060ccd9180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T03:52:44.775 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:44.774+0000 7f67c1693180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:52:45.040 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:44.827+0000 7f060ccd9180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T03:52:45.040 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:44.921+0000 7f060ccd9180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T03:52:45.405 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:45.404+0000 7f67c1693180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:52:45.576 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:45.573+0000 7f060ccd9180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T03:52:45.916 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:45.581+0000 7f67c1693180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:52:45.916 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:45.677+0000 7f67c1693180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:52:45.916 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:45.753+0000 7f060ccd9180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T03:52:45.917 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:45.854+0000 7f060ccd9180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T03:52:45.974 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:45.973+0000 7f67c1693180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:52:46.070 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:46.069+0000 7f67c1693180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:52:46.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:46.130+0000 7f575f7fe700 1 -- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 msgr2=0x7f574c069550 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T03:52:46.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:46.130+0000 7f575f7fe700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 0x7f574c069550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T03:52:46.162 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:46.159+0000 7f060ccd9180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T03:52:46.262 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:46.259+0000 7f060ccd9180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T03:52:46.779 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:46.777+0000 7f67c1693180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:52:46.788 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:52:46.818 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:52:47.006 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:47.004+0000 7f060ccd9180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T03:52:47.179 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:47.075+0000 7f67c1693180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:52:47.180 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:47.169+0000 7f67c1693180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:52:47.299 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:47.296+0000 7f060ccd9180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T03:52:47.326 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:47.323+0000 7f67c1693180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:52:47.395 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:47.393+0000 7f060ccd9180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T03:52:47.514 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:47.429+0000 7f67c1693180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:52:47.556 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:47.553+0000 7f060ccd9180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T03:52:47.621 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:47.620+0000 7f67c1693180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:52:47.665 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:47.663+0000 7f060ccd9180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T03:52:47.713 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:47.712+0000 7f67c1693180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:52:47.807 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:47.806+0000 7f67c1693180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:52:47.863 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:47.861+0000 7f060ccd9180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T03:52:47.902 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:47.901+0000 7f67c1693180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:52:47.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:47.910+0000 7f57477fe700 1 -- 172.21.15.55:0/2306711356 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 60) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f5754051e90 con 0x7f57600b0790 2023-10-06T03:52:48.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:47.910+0000 7f57477fe700 1 -- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 msgr2=0x7f574c069550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T03:52:48.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:47.910+0000 7f57477fe700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 0x7f574c069550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:48.105 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:47.967+0000 7f060ccd9180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T03:52:48.106 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:48.068+0000 7f060ccd9180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T03:52:48.106 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:48.034+0000 7f67339f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:48.106 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:48.034+0000 7f67339f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:48.107 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:48.034+0000 7f67339f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:48.107 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:48.034+0000 7f67339f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:48.107 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:48.034+0000 7f67339f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:48.107 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:48.035+0000 7f67369f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:48.107 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:48.035+0000 7f67369f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:48.107 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:48.035+0000 7f67369f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:48.107 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:48.035+0000 7f67369f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:48.107 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:52:48.035+0000 7f67369f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:52:48.169 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T03:52:48.167+0000 7f060ccd9180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T03:52:48.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.915+0000 7f57477fe700 1 -- 172.21.15.55:0/2306711356 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mgrmap(e 61) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f5754082b60 con 0x7f57600b0790 2023-10-06T03:52:48.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.916+0000 7f57477fe700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 0x7f574c0694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:48.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.916+0000 7f57477fe700 1 -- 172.21.15.55:0/2306711356 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f576004c3c0 con 0x7f574c0670a0 2023-10-06T03:52:48.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.919+0000 7f575f7fe700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 0x7f574c0694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:48.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.926+0000 7f575f7fe700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 0x7f574c0694d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f57600cde20 tx=0x7f5750007ec0 comp rx=0 tx=0).ready entity=mgr.8026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:48.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.991+0000 7f57477fe700 1 -- 172.21.15.55:0/2306711356 <== mgr.8026 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f576004c3c0 con 0x7f574c0670a0 2023-10-06T03:52:49.043 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path /snap_test_dir1 2023-10-06T03:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.994+0000 7f5764d9d700 1 -- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 msgr2=0x7f574c0694d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.994+0000 7f5764d9d700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 0x7f574c0694d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f57600cde20 tx=0x7f5750007ec0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.994+0000 7f5764d9d700 1 -- 172.21.15.55:0/2306711356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57600b0790 msgr2=0x7f57600c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.994+0000 7f5764d9d700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57600b0790 0x7f57600c8c40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f575400ec80 tx=0x7f57540091a0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.994+0000 7f5764d9d700 1 -- 172.21.15.55:0/2306711356 shutdown_connections 2023-10-06T03:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.994+0000 7f5764d9d700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f574c0670a0 0x7f574c0694d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.994+0000 7f5764d9d700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f57601591e0 0x7f5760157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.994+0000 7f5764d9d700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f57600b03c0 0x7f576015a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.994+0000 7f5764d9d700 1 --2- 172.21.15.55:0/2306711356 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f57600b0790 0x7f57600c8c40 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.994+0000 7f5764d9d700 1 -- 172.21.15.55:0/2306711356 >> 172.21.15.55:0/2306711356 conn(0x7f5760063230 msgr2=0x7f57600ca0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.994+0000 7f5764d9d700 1 -- 172.21.15.55:0/2306711356 shutdown_connections 2023-10-06T03:52:49.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:48.994+0000 7f5764d9d700 1 -- 172.21.15.55:0/2306711356 wait complete. 2023-10-06T03:52:49.048 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-10-06T03:52:49.048 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs /snap_test_dir1 55n 2023-10-06T03:52:49.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.240+0000 7ff6bffff700 1 Processor -- start 2023-10-06T03:52:49.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.241+0000 7ff6bffff700 1 -- start start 2023-10-06T03:52:49.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.242+0000 7ff6bffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6c00ae9c0 0x7ff6c01611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:49.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.242+0000 7ff6bffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6c00cbe50 con 0x7ff6c01563b0 2023-10-06T03:52:49.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.242+0000 7ff6bffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6c00cb370 con 0x7ff6c00caee0 2023-10-06T03:52:49.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.242+0000 7ff6bffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00cb650 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.242+0000 7ff6beffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff6c01563b0 0x7ff6c00ca7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:41246/0 (socket says 172.21.15.55:41246) 2023-10-06T03:52:49.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.242+0000 7ff6beffd700 1 -- 172.21.15.55:0/3698762470 learned_addr learned my addr 172.21.15.55:0/3698762470 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T03:52:49.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.242+0000 7ff6bf7fe700 1 --2- 172.21.15.55:0/3698762470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6c00ae9c0 0x7ff6c01611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:49.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.243+0000 7ff6bdffb700 1 -- 172.21.15.55:0/3698762470 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 405795674 0 0) 0x7ff6c00cbe50 con 0x7ff6c01563b0 2023-10-06T03:52:49.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.243+0000 7ff6bdffb700 1 -- 172.21.15.55:0/3698762470 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6a8003570 con 0x7ff6c01563b0 2023-10-06T03:52:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.243+0000 7ff6bdffb700 1 -- 172.21.15.55:0/3698762470 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081573456 0 0) 0x7ff6c00cb370 con 0x7ff6c00caee0 2023-10-06T03:52:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.243+0000 7ff6bdffb700 1 -- 172.21.15.55:0/3698762470 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6c00cbe50 con 0x7ff6c00caee0 2023-10-06T03:52:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.243+0000 7ff6bdffb700 1 -- 172.21.15.55:0/3698762470 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4098541581 0 0) 0x7ff6a8003570 con 0x7ff6c01563b0 2023-10-06T03:52:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.243+0000 7ff6bf7fe700 1 -- 172.21.15.55:0/3698762470 >> v1:172.21.15.163:6789/0 conn(0x7ff6c00caee0 legacy=0x7ff6c00ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:49.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.243+0000 7ff6bf7fe700 1 -- 172.21.15.55:0/3698762470 >> v1:172.21.15.55:6789/0 conn(0x7ff6c01563b0 legacy=0x7ff6c00ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:49.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.243+0000 7ff6bf7fe700 1 -- 172.21.15.55:0/3698762470 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c00c6860 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.243+0000 7ff6bf7fe700 1 --2- 172.21.15.55:0/3698762470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6c00ae9c0 0x7ff6c01611c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009e90 tx=0x7ff6b8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acacdfbbe0789ea8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:49.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.244+0000 7ff6bdffb700 1 -- 172.21.15.55:0/3698762470 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b800e070 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.244+0000 7ff6bdffb700 1 -- 172.21.15.55:0/3698762470 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b800b870 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.244+0000 7ff6bdffb700 1 -- 172.21.15.55:0/3698762470 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0001470 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.244+0000 7ff6bffff700 1 -- 172.21.15.55:0/3698762470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6c00ae9c0 msgr2=0x7ff6c01611c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.244+0000 7ff6bffff700 1 --2- 172.21.15.55:0/3698762470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6c00ae9c0 0x7ff6c01611c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009e90 tx=0x7ff6b8009780 comp rx=0 tx=0).stop 2023-10-06T03:52:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.244+0000 7ff6bffff700 1 -- 172.21.15.55:0/3698762470 shutdown_connections 2023-10-06T03:52:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.244+0000 7ff6bffff700 1 --2- 172.21.15.55:0/3698762470 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6c00ae9c0 0x7ff6c01611c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.244+0000 7ff6bffff700 1 -- 172.21.15.55:0/3698762470 >> 172.21.15.55:0/3698762470 conn(0x7ff6c0068e20 msgr2=0x7ff6c00cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:49.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.244+0000 7ff6bffff700 1 -- 172.21.15.55:0/3698762470 shutdown_connections 2023-10-06T03:52:49.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.244+0000 7ff6bffff700 1 -- 172.21.15.55:0/3698762470 wait complete. 2023-10-06T03:52:49.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.246+0000 7ff6bffff700 1 Processor -- start 2023-10-06T03:52:49.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.246+0000 7ff6bffff700 1 -- start start 2023-10-06T03:52:49.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6bffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6c00ae9c0 0x7ff6c0146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:49.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6bffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6c00caee0 0x7ff6c00b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:49.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6bffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6c01563b0 0x7ff6c00c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:49.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6bffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00b13e0 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6bffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00b07d0 con 0x7ff6c00caee0 2023-10-06T03:52:49.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6bffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c00b0ab0 con 0x7ff6c01563b0 2023-10-06T03:52:49.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6beffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6c00ae9c0 0x7ff6c0146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:49.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6beffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6c00ae9c0 0x7ff6c0146c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46154/0 (socket says 172.21.15.55:46154) 2023-10-06T03:52:49.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6beffd700 1 -- 172.21.15.55:0/4282518074 learned_addr learned my addr 172.21.15.55:0/4282518074 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:49.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6bf7fe700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6c01563b0 0x7ff6c00c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:49.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6be7fc700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6c00caee0 0x7ff6c00b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:49.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6beffd700 1 -- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6c01563b0 msgr2=0x7ff6c00c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:49.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6beffd700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6c01563b0 0x7ff6c00c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6beffd700 1 -- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6c00caee0 msgr2=0x7ff6c00b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:49.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6beffd700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6c00caee0 0x7ff6c00b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6beffd700 1 -- 172.21.15.55:0/4282518074 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b8009a20 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.247+0000 7ff6be7fc700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6c00caee0 0x7ff6c00b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T03:52:49.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.248+0000 7ff6beffd700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6c00ae9c0 0x7ff6c0146c80 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000c7f0 tx=0x7ff6b0001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:49.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.248+0000 7ff6a6ffd700 1 -- 172.21.15.55:0/4282518074 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0007360 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.248+0000 7ff6a6ffd700 1 -- 172.21.15.55:0/4282518074 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6b00074e0 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.248+0000 7ff6bffff700 1 -- 172.21.15.55:0/4282518074 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c00b0d90 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.248+0000 7ff6a6ffd700 1 -- 172.21.15.55:0/4282518074 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b0004830 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.248+0000 7ff6bffff700 1 -- 172.21.15.55:0/4282518074 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c01471c0 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.249+0000 7ff6bffff700 1 -- 172.21.15.55:0/4282518074 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c00c8990 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.250+0000 7ff6a6ffd700 1 -- 172.21.15.55:0/4282518074 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 61) v1 ==== 84595+0+0 (secure 0 0 0) 0x7ff6b0004c00 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.251+0000 7ff6a6ffd700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff6ac066dd0 0x7ff6ac069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:49.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.251+0000 7ff6a6ffd700 1 -- 172.21.15.55:0/4282518074 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6794+0+0 (secure 0 0 0) 0x7ff6b0083e90 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.251+0000 7ff6be7fc700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff6ac066dd0 0x7ff6ac069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:49.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.252+0000 7ff6be7fc700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff6ac066dd0 0x7ff6ac069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00c1f40 tx=0x7ff6b4009040 comp rx=0 tx=0).ready entity=mgr.8026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:49.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.255+0000 7ff6a6ffd700 1 -- 172.21.15.55:0/4282518074 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6b0050f20 con 0x7ff6c00ae9c0 2023-10-06T03:52:49.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.482+0000 7ff6bffff700 1 -- 172.21.15.55:0/4282518074 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6c00b1520 con 0x7ff6ac066dd0 2023-10-06T03:52:49.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.544+0000 7ff6a6ffd700 1 -- 172.21.15.55:0/4282518074 <== mgr.8026 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7ff6b40035c0 con 0x7ff6ac066dd0 2023-10-06T03:52:49.712 INFO:teuthology.orchestra.run.smithi055.stdout:Retention added to path /snap_test_dir1 2023-10-06T03:52:49.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.546+0000 7ff6bffff700 1 -- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff6ac066dd0 msgr2=0x7ff6ac069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:49.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.546+0000 7ff6bffff700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff6ac066dd0 0x7ff6ac069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff6c00c1f40 tx=0x7ff6b4009040 comp rx=0 tx=0).stop 2023-10-06T03:52:49.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.546+0000 7ff6bffff700 1 -- 172.21.15.55:0/4282518074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6c00ae9c0 msgr2=0x7ff6c0146c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:49.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.546+0000 7ff6bffff700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6c00ae9c0 0x7ff6c0146c80 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7ff6b000c7f0 tx=0x7ff6b0001a10 comp rx=0 tx=0).stop 2023-10-06T03:52:49.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.546+0000 7ff6bffff700 1 -- 172.21.15.55:0/4282518074 shutdown_connections 2023-10-06T03:52:49.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.546+0000 7ff6bffff700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ff6ac066dd0 0x7ff6ac069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.546+0000 7ff6bffff700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff6c01563b0 0x7ff6c00c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.546+0000 7ff6bffff700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff6c00ae9c0 0x7ff6c0146c80 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.546+0000 7ff6bffff700 1 --2- 172.21.15.55:0/4282518074 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff6c00caee0 0x7ff6c00b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.546+0000 7ff6bffff700 1 -- 172.21.15.55:0/4282518074 >> 172.21.15.55:0/4282518074 conn(0x7ff6c0068e20 msgr2=0x7ff6c00c8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:49.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.546+0000 7ff6bffff700 1 -- 172.21.15.55:0/4282518074 shutdown_connections 2023-10-06T03:52:49.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.546+0000 7ff6bffff700 1 -- 172.21.15.55:0/4282518074 wait complete. 2023-10-06T03:52:49.714 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-10-06T03:52:49.715 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-10-06T03:52:49.715 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json /snap_test_dir1 2023-10-06T03:52:49.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.905+0000 7f3a7bfff700 1 Processor -- start 2023-10-06T03:52:49.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.905+0000 7f3a7bfff700 1 -- start start 2023-10-06T03:52:49.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.906+0000 7f3a7bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a7c15b050 0x7f3a7c0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:49.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.906+0000 7f3a7bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a7c15bb30 0x7f3a7c0ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:49.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.906+0000 7f3a7bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7c157a50 con 0x7f3a7c0cc8d0 2023-10-06T03:52:49.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.906+0000 7f3a7bfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a7c158f10 con 0x7f3a7c15bb30 2023-10-06T03:52:49.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.906+0000 7f3a7bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c1591b0 con 0x7f3a7c15b050 2023-10-06T03:52:49.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.906+0000 7f3a7b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a7c15bb30 0x7f3a7c0ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:49.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.906+0000 7f3a7b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3a7c15bb30 0x7f3a7c0ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46160/0 (socket says 172.21.15.55:46160) 2023-10-06T03:52:49.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.906+0000 7f3a7b7fe700 1 -- 172.21.15.55:0/1121253631 learned_addr learned my addr 172.21.15.55:0/1121253631 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:49.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.907+0000 7f3a7affd700 1 --2- 172.21.15.55:0/1121253631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a7c15b050 0x7f3a7c0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:49.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.907+0000 7f3a7b7fe700 1 -- 172.21.15.55:0/1121253631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a7c15b050 msgr2=0x7f3a7c0ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T03:52:49.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.907+0000 7f3a7b7fe700 1 --2- 172.21.15.55:0/1121253631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a7c15b050 0x7f3a7c0ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.907+0000 7f3a7b7fe700 1 -- 172.21.15.55:0/1121253631 >> v1:172.21.15.55:6789/0 conn(0x7f3a7c0cc8d0 legacy=0x7f3a7c15b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:49.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.907+0000 7f3a7b7fe700 1 -- 172.21.15.55:0/1121253631 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c0ca9e0 con 0x7f3a7c15bb30 2023-10-06T03:52:49.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.907+0000 7f3a7b7fe700 1 --2- 172.21.15.55:0/1121253631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7c15bb30 0x7f3a7c0ada30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009f20 tx=0x7f3a74009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d71a9b8acb51a55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:49.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.907+0000 7f3a79ffb700 1 -- 172.21.15.55:0/1121253631 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7400e070 con 0x7f3a7c15bb30 2023-10-06T03:52:49.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.907+0000 7f3a79ffb700 1 -- 172.21.15.55:0/1121253631 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7400ba80 con 0x7f3a7c15bb30 2023-10-06T03:52:49.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.907+0000 7f3a79ffb700 1 -- 172.21.15.55:0/1121253631 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a740092e0 con 0x7f3a7c15bb30 2023-10-06T03:52:49.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.908+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1121253631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7c15bb30 msgr2=0x7f3a7c0ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:49.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.908+0000 7f3a7bfff700 1 --2- 172.21.15.55:0/1121253631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7c15bb30 0x7f3a7c0ada30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009f20 tx=0x7f3a74009840 comp rx=0 tx=0).stop 2023-10-06T03:52:49.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.908+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1121253631 shutdown_connections 2023-10-06T03:52:49.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.908+0000 7f3a7bfff700 1 --2- 172.21.15.55:0/1121253631 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7c15bb30 0x7f3a7c0ada30 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.908+0000 7f3a7bfff700 1 --2- 172.21.15.55:0/1121253631 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a7c15b050 0x7f3a7c0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.908+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1121253631 >> 172.21.15.55:0/1121253631 conn(0x7f3a7c068e20 msgr2=0x7f3a7c0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:49.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.908+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1121253631 shutdown_connections 2023-10-06T03:52:49.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.908+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1121253631 wait complete. 2023-10-06T03:52:49.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.909+0000 7f3a7bfff700 1 Processor -- start 2023-10-06T03:52:49.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.910+0000 7f3a7bfff700 1 -- start start 2023-10-06T03:52:49.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.910+0000 7f3a7bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a7c0cc8d0 0x7f3a7c0c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:49.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.910+0000 7f3a7bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7c15b050 0x7f3a7c1568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:49.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.911+0000 7f3a7bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c15bb30 0x7f3a7c0c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:49.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.911+0000 7f3a7bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c156de0 con 0x7f3a7c15b050 2023-10-06T03:52:49.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.911+0000 7f3a7bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c156410 con 0x7f3a7c15bb30 2023-10-06T03:52:49.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.911+0000 7f3a7bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c155e00 con 0x7f3a7c0cc8d0 2023-10-06T03:52:49.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.911+0000 7f3a7a7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7c15b050 0x7f3a7c1568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:49.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.911+0000 7f3a7a7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7c15b050 0x7f3a7c1568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:46176/0 (socket says 172.21.15.55:46176) 2023-10-06T03:52:49.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.911+0000 7f3a7a7fc700 1 -- 172.21.15.55:0/1150259644 learned_addr learned my addr 172.21.15.55:0/1150259644 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T03:52:49.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.911+0000 7f3a7affd700 1 --2- 172.21.15.55:0/1150259644 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a7c0cc8d0 0x7f3a7c0c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:49.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.911+0000 7f3a7b7fe700 1 --2- 172.21.15.55:0/1150259644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c15bb30 0x7f3a7c0c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:49.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.911+0000 7f3a7a7fc700 1 -- 172.21.15.55:0/1150259644 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a7c0cc8d0 msgr2=0x7f3a7c0c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:49.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.911+0000 7f3a7a7fc700 1 --2- 172.21.15.55:0/1150259644 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a7c0cc8d0 0x7f3a7c0c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.911+0000 7f3a7a7fc700 1 -- 172.21.15.55:0/1150259644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c15bb30 msgr2=0x7f3a7c0c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:49.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.911+0000 7f3a7a7fc700 1 --2- 172.21.15.55:0/1150259644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c15bb30 0x7f3a7c0c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:49.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.911+0000 7f3a7a7fc700 1 -- 172.21.15.55:0/1150259644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a74009ae0 con 0x7f3a7c15b050 2023-10-06T03:52:49.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.912+0000 7f3a7a7fc700 1 --2- 172.21.15.55:0/1150259644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7c15b050 0x7f3a7c1568a0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000c4e0 tx=0x7f3a7000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:49.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.912+0000 7f3a62ffd700 1 -- 172.21.15.55:0/1150259644 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70011590 con 0x7f3a7c15b050 2023-10-06T03:52:49.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.912+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1150259644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c0c6720 con 0x7f3a7c15b050 2023-10-06T03:52:49.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.912+0000 7f3a62ffd700 1 -- 172.21.15.55:0/1150259644 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a70011710 con 0x7f3a7c15b050 2023-10-06T03:52:49.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.912+0000 7f3a62ffd700 1 -- 172.21.15.55:0/1150259644 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70011ad0 con 0x7f3a7c15b050 2023-10-06T03:52:49.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.912+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1150259644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c1560a0 con 0x7f3a7c15b050 2023-10-06T03:52:49.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.913+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1150259644 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c04c3c0 con 0x7f3a7c15b050 2023-10-06T03:52:49.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.914+0000 7f3a62ffd700 1 -- 172.21.15.55:0/1150259644 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 61) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f3a7001b3b0 con 0x7f3a7c15b050 2023-10-06T03:52:49.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.915+0000 7f3a62ffd700 1 --2- 172.21.15.55:0/1150259644 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a68066dd0 0x7f3a68069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T03:52:49.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.915+0000 7f3a62ffd700 1 -- 172.21.15.55:0/1150259644 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6794+0+0 (secure 0 0 0) 0x7f3a70083b80 con 0x7f3a7c15b050 2023-10-06T03:52:49.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.916+0000 7f3a7affd700 1 --2- 172.21.15.55:0/1150259644 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a68066dd0 0x7f3a68069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T03:52:49.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.916+0000 7f3a7affd700 1 --2- 172.21.15.55:0/1150259644 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a68066dd0 0x7f3a68069280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c006fd0 tx=0x7f3a6c008040 comp rx=0 tx=0).ready entity=mgr.8026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T03:52:49.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.920+0000 7f3a62ffd700 1 -- 172.21.15.55:0/1150259644 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a70050b80 con 0x7f3a7c15b050 2023-10-06T03:52:49.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:49.966+0000 7f3a62ffd700 1 -- 172.21.15.55:0/1150259644 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 62) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f3a70050f50 con 0x7f3a7c15b050 2023-10-06T03:52:50.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:50.151+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1150259644 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a7c0cad50 con 0x7f3a68066dd0 2023-10-06T03:52:50.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:50.162+0000 7f3a62ffd700 1 -- 172.21.15.55:0/1150259644 <== mgr.8026 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7f3a7c0cad50 con 0x7f3a68066dd0 2023-10-06T03:52:50.167 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T03:52:50.167 INFO:teuthology.orchestra.run.smithi055.stdout:{"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-10-06T03:52:50.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:50.163+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1150259644 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a68066dd0 msgr2=0x7f3a68069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:50.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:50.163+0000 7f3a7bfff700 1 --2- 172.21.15.55:0/1150259644 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a68066dd0 0x7f3a68069280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c006fd0 tx=0x7f3a6c008040 comp rx=0 tx=0).stop 2023-10-06T03:52:50.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:50.163+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1150259644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7c15b050 msgr2=0x7f3a7c1568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T03:52:50.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:50.163+0000 7f3a7bfff700 1 --2- 172.21.15.55:0/1150259644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7c15b050 0x7f3a7c1568a0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f3a7000c4e0 tx=0x7f3a7000c7f0 comp rx=0 tx=0).stop 2023-10-06T03:52:50.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:50.163+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1150259644 shutdown_connections 2023-10-06T03:52:50.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:50.163+0000 7f3a7bfff700 1 --2- 172.21.15.55:0/1150259644 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f3a68066dd0 0x7f3a68069280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:50.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:50.163+0000 7f3a7bfff700 1 --2- 172.21.15.55:0/1150259644 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3a7c15bb30 0x7f3a7c0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:50.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:50.163+0000 7f3a7bfff700 1 --2- 172.21.15.55:0/1150259644 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3a7c0cc8d0 0x7f3a7c0c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:50.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:50.163+0000 7f3a7bfff700 1 --2- 172.21.15.55:0/1150259644 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3a7c15b050 0x7f3a7c1568a0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T03:52:50.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:50.163+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1150259644 >> 172.21.15.55:0/1150259644 conn(0x7f3a7c068e20 msgr2=0x7f3a7c15a030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T03:52:50.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:50.163+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1150259644 shutdown_connections 2023-10-06T03:52:50.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T03:52:50.163+0000 7f3a7bfff700 1 -- 172.21.15.55:0/1150259644 wait complete. 2023-10-06T03:52:50.183 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-10-06T03:52:50.183 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1M'], 'retention': [{'n': 55}]} 2023-10-06T03:53:00.559 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:53:00.556+0000 7f67259d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:53:00.559 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:53:00.556+0000 7f67259d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:53:00.559 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:53:00.557+0000 7f67259d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:53:00.559 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:53:00.557+0000 7f67259d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:53:00.559 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T03:53:00.557+0000 7f67259d5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T03:53:16.919 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:53:16.924 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:53:47.025 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:53:47.030 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:54:17.134 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:54:17.140 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:54:47.250 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:54:47.332 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:55:17.430 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:55:17.436 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:55:47.544 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:55:47.563 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:56:17.667 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:56:17.673 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:56:47.771 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:56:47.777 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:57:17.903 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:57:17.912 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:57:48.037 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:57:48.043 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:58:18.150 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:58:18.156 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:58:48.274 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:58:48.280 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:59:18.388 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:59:18.394 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:59:48.498 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T03:59:48.503 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:00:18.603 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:00:18.608 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:00:48.716 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:00:48.722 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:01:18.847 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:01:18.854 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:01:48.960 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:01:48.965 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:02:19.064 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:02:19.069 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:02:49.187 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:02:49.193 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:03:19.325 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:03:19.331 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:03:49.432 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:03:49.438 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:04:19.547 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:04:19.553 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:04:49.648 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:04:49.653 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:05:19.804 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:05:19.810 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:05:49.916 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:05:49.922 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:06:20.030 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:06:20.036 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:06:50.135 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:06:50.141 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:07:20.264 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:07:20.270 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:07:50.394 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:07:50.399 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:08:20.508 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:08:20.513 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:08:50.621 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:08:50.627 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:09:20.727 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:09:20.733 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:09:50.829 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:09:50.835 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:10:20.960 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:10:20.966 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:10:51.073 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:10:51.078 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:11:21.179 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:11:21.184 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:11:51.308 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:11:51.314 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:12:21.415 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:12:21.421 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:12:51.552 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:12:51.558 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:13:21.673 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:13:21.680 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:13:51.786 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:13:51.793 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:14:21.899 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:14:21.905 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:14:52.014 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:14:52.020 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:15:22.169 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:15:22.174 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:15:52.271 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:15:52.277 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:16:22.377 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:16:22.382 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:16:52.490 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:16:52.496 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:17:22.604 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:17:22.609 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:17:52.729 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:17:52.734 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:18:22.838 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:18:22.845 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:18:53.015 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:18:53.021 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:19:23.129 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:19:23.135 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:19:53.234 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:19:53.240 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:20:23.340 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:20:23.345 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:20:53.452 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:20:53.458 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:21:23.565 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:21:23.571 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:21:53.678 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:21:53.684 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:22:23.817 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:22:23.823 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:22:53.996 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:22:54.002 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:23:24.112 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:23:24.117 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:23:54.274 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:23:54.280 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:24:24.378 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:24:24.384 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:24:54.508 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:24:54.514 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:25:24.629 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:25:24.635 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:25:54.768 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:25:54.773 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:26:24.880 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:26:24.886 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:26:54.994 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:26:54.999 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:27:25.099 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:27:25.105 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:27:55.295 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:27:55.301 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:28:25.404 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:28:25.476 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:28:55.663 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:28:55.669 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:29:25.826 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:29:25.832 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:29:55.931 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:29:55.936 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:30:26.111 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:30:26.117 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:30:56.336 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:30:56.341 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:31:26.478 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:31:26.484 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:31:56.650 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:31:56.656 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:32:26.838 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:32:26.843 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:32:57.026 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:32:57.032 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:33:27.206 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:33:27.211 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:33:57.319 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:33:57.325 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:34:27.516 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:34:27.522 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:34:57.762 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:34:57.768 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:35:27.986 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:35:27.992 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:35:58.091 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:35:58.097 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:36:28.227 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:36:28.232 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:36:58.347 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:36:58.353 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:37:28.469 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:37:28.475 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:37:58.615 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:37:58.621 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:38:28.780 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:38:28.785 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:38:58.943 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:38:58.948 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:39:29.047 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:39:29.053 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:39:59.152 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:39:59.158 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:40:29.349 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:40:29.355 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:40:59.555 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:40:59.560 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:41:29.668 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:41:29.673 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:41:59.839 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:41:59.845 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:42:30.028 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:42:30.033 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:43:00.224 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:43:00.230 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:43:30.354 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:43:30.360 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:44:00.493 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:44:00.499 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:44:30.592 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:44:30.598 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:45:00.699 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:45:00.705 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:45:30.806 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:45:30.812 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:46:00.905 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:46:00.911 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:46:31.058 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:46:31.064 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:47:01.197 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:47:01.203 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:47:31.377 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:47:31.383 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:48:01.505 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:48:01.511 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:48:16.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:16.185 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T04:48:16.235 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_54_00_UTC 2023-10-06T04:48:16.235 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_55_00_UTC 2023-10-06T04:48:16.235 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_56_00_UTC 2023-10-06T04:48:16.235 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_57_00_UTC 2023-10-06T04:48:16.235 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_58_00_UTC 2023-10-06T04:48:16.235 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_59_00_UTC 2023-10-06T04:48:16.236 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_00_00_UTC 2023-10-06T04:48:16.236 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_01_00_UTC 2023-10-06T04:48:16.236 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_02_00_UTC 2023-10-06T04:48:16.236 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_03_00_UTC 2023-10-06T04:48:16.236 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_04_00_UTC 2023-10-06T04:48:16.236 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_05_00_UTC 2023-10-06T04:48:16.236 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_06_00_UTC 2023-10-06T04:48:16.236 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_07_00_UTC 2023-10-06T04:48:16.236 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_08_00_UTC 2023-10-06T04:48:16.236 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_09_00_UTC 2023-10-06T04:48:16.236 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_10_00_UTC 2023-10-06T04:48:16.237 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_11_00_UTC 2023-10-06T04:48:16.237 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_12_00_UTC 2023-10-06T04:48:16.237 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_13_01_UTC 2023-10-06T04:48:16.237 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_14_00_UTC 2023-10-06T04:48:16.237 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_15_00_UTC 2023-10-06T04:48:16.237 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_16_00_UTC 2023-10-06T04:48:16.237 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_17_00_UTC 2023-10-06T04:48:16.237 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_18_00_UTC 2023-10-06T04:48:16.237 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_19_00_UTC 2023-10-06T04:48:16.237 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_20_00_UTC 2023-10-06T04:48:16.237 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_21_00_UTC 2023-10-06T04:48:16.237 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_22_00_UTC 2023-10-06T04:48:16.238 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_23_00_UTC 2023-10-06T04:48:16.238 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_24_00_UTC 2023-10-06T04:48:16.238 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_25_00_UTC 2023-10-06T04:48:16.238 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_26_00_UTC 2023-10-06T04:48:16.238 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_27_00_UTC 2023-10-06T04:48:16.238 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_28_00_UTC 2023-10-06T04:48:16.238 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_29_00_UTC 2023-10-06T04:48:16.238 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_30_00_UTC 2023-10-06T04:48:16.238 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_31_00_UTC 2023-10-06T04:48:16.238 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_32_00_UTC 2023-10-06T04:48:16.238 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_33_00_UTC 2023-10-06T04:48:16.239 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_34_00_UTC 2023-10-06T04:48:16.239 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_35_00_UTC 2023-10-06T04:48:16.239 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_36_00_UTC 2023-10-06T04:48:16.239 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_37_00_UTC 2023-10-06T04:48:16.239 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_38_00_UTC 2023-10-06T04:48:16.239 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_39_00_UTC 2023-10-06T04:48:16.239 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_40_00_UTC 2023-10-06T04:48:16.239 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_41_00_UTC 2023-10-06T04:48:16.239 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_42_00_UTC 2023-10-06T04:48:16.239 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_43_00_UTC 2023-10-06T04:48:16.240 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_44_00_UTC 2023-10-06T04:48:16.240 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_45_00_UTC 2023-10-06T04:48:16.240 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_46_00_UTC 2023-10-06T04:48:16.240 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_47_00_UTC 2023-10-06T04:48:16.240 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_48_00_UTC 2023-10-06T04:48:16.241 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-03_54_00_UTC', 'scheduled-2023-10-06-03_55_00_UTC', 'scheduled-2023-10-06-03_56_00_UTC', 'scheduled-2023-10-06-03_57_00_UTC', 'scheduled-2023-10-06-03_58_00_UTC', 'scheduled-2023-10-06-03_59_00_UTC', 'scheduled-2023-10-06-04_00_00_UTC', 'scheduled-2023-10-06-04_01_00_UTC', 'scheduled-2023-10-06-04_02_00_UTC', 'scheduled-2023-10-06-04_03_00_UTC', 'scheduled-2023-10-06-04_04_00_UTC', 'scheduled-2023-10-06-04_05_00_UTC', 'scheduled-2023-10-06-04_06_00_UTC', 'scheduled-2023-10-06-04_07_00_UTC', 'scheduled-2023-10-06-04_08_00_UTC', 'scheduled-2023-10-06-04_09_00_UTC', 'scheduled-2023-10-06-04_10_00_UTC', 'scheduled-2023-10-06-04_11_00_UTC', 'scheduled-2023-10-06-04_12_00_UTC', 'scheduled-2023-10-06-04_13_01_UTC', 'scheduled-2023-10-06-04_14_00_UTC', 'scheduled-2023-10-06-04_15_00_UTC', 'scheduled-2023-10-06-04_16_00_UTC', 'scheduled-2023-10-06-04_17_00_UTC', 'scheduled-2023-10-06-04_18_00_UTC', 'scheduled-2023-10-06-04_19_00_UTC', 'scheduled-2023-10-06-04_20_00_UTC', 'scheduled-2023-10-06-04_21_00_UTC', 'scheduled-2023-10-06-04_22_00_UTC', 'scheduled-2023-10-06-04_23_00_UTC', 'scheduled-2023-10-06-04_24_00_UTC', 'scheduled-2023-10-06-04_25_00_UTC', 'scheduled-2023-10-06-04_26_00_UTC', 'scheduled-2023-10-06-04_27_00_UTC', 'scheduled-2023-10-06-04_28_00_UTC', 'scheduled-2023-10-06-04_29_00_UTC', 'scheduled-2023-10-06-04_30_00_UTC', 'scheduled-2023-10-06-04_31_00_UTC', 'scheduled-2023-10-06-04_32_00_UTC', 'scheduled-2023-10-06-04_33_00_UTC', 'scheduled-2023-10-06-04_34_00_UTC', 'scheduled-2023-10-06-04_35_00_UTC', 'scheduled-2023-10-06-04_36_00_UTC', 'scheduled-2023-10-06-04_37_00_UTC', 'scheduled-2023-10-06-04_38_00_UTC', 'scheduled-2023-10-06-04_39_00_UTC', 'scheduled-2023-10-06-04_40_00_UTC', 'scheduled-2023-10-06-04_41_00_UTC', 'scheduled-2023-10-06-04_42_00_UTC', 'scheduled-2023-10-06-04_43_00_UTC', 'scheduled-2023-10-06-04_44_00_UTC', 'scheduled-2023-10-06-04_45_00_UTC', 'scheduled-2023-10-06-04_46_00_UTC', 'scheduled-2023-10-06-04_47_00_UTC', 'scheduled-2023-10-06-04_48_00_UTC'] 2023-10-06T04:48:16.241 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1 2023-10-06T04:48:16.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.436+0000 7f9741b9d700 1 Processor -- start 2023-10-06T04:48:16.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.436+0000 7f9741b9d700 1 -- start start 2023-10-06T04:48:16.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.438+0000 7f9741b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f973c0cc8d0 0x7f973c15b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:16.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.438+0000 7f9741b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f973c15b960 0x7f973c0ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:16.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.438+0000 7f9741b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c157a50 con 0x7f973c15b050 2023-10-06T04:48:16.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.438+0000 7f9741b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c158f10 con 0x7f973c15b960 2023-10-06T04:48:16.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.438+0000 7f9741b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c1591b0 con 0x7f973c0cc8d0 2023-10-06T04:48:16.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.438+0000 7f974139c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f973c15b960 0x7f973c0ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:16.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.438+0000 7f974139c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f973c15b960 0x7f973c0ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58824/0 (socket says 172.21.15.55:58824) 2023-10-06T04:48:16.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.438+0000 7f974139c700 1 -- 172.21.15.55:0/117370 learned_addr learned my addr 172.21.15.55:0/117370 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:16.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.438+0000 7f973bfff700 1 --2- 172.21.15.55:0/117370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f973c0cc8d0 0x7f973c15b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:16.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.438+0000 7f974139c700 1 -- 172.21.15.55:0/117370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f973c0cc8d0 msgr2=0x7f973c15b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:16.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.438+0000 7f974139c700 1 --2- 172.21.15.55:0/117370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f973c0cc8d0 0x7f973c15b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:16.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.438+0000 7f974139c700 1 -- 172.21.15.55:0/117370 >> v1:172.21.15.163:6789/0 conn(0x7f973c15b050 legacy=0x7f973c0ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:16.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.438+0000 7f974139c700 1 -- 172.21.15.55:0/117370 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c0ca9e0 con 0x7f973c15b960 2023-10-06T04:48:16.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.438+0000 7f973bfff700 1 --2- 172.21.15.55:0/117370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f973c0cc8d0 0x7f973c15b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:16.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.439+0000 7f974139c700 1 --2- 172.21.15.55:0/117370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f973c15b960 0x7f973c0ada30 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f9734009f20 tx=0x7f9734009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bead05a8a7ebeea1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:16.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.439+0000 7f973b7fe700 1 -- 172.21.15.55:0/117370 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973400e070 con 0x7f973c15b960 2023-10-06T04:48:16.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.439+0000 7f973b7fe700 1 -- 172.21.15.55:0/117370 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973400ba40 con 0x7f973c15b960 2023-10-06T04:48:16.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.439+0000 7f973b7fe700 1 -- 172.21.15.55:0/117370 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97340092e0 con 0x7f973c15b960 2023-10-06T04:48:16.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.439+0000 7f9741b9d700 1 -- 172.21.15.55:0/117370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f973c15b960 msgr2=0x7f973c0ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:16.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.439+0000 7f9741b9d700 1 --2- 172.21.15.55:0/117370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f973c15b960 0x7f973c0ada30 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f9734009f20 tx=0x7f9734009840 comp rx=0 tx=0).stop 2023-10-06T04:48:16.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.440+0000 7f9741b9d700 1 -- 172.21.15.55:0/117370 shutdown_connections 2023-10-06T04:48:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.440+0000 7f9741b9d700 1 --2- 172.21.15.55:0/117370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f973c15b960 0x7f973c0ada30 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.440+0000 7f9741b9d700 1 --2- 172.21.15.55:0/117370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f973c0cc8d0 0x7f973c15b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.440+0000 7f9741b9d700 1 -- 172.21.15.55:0/117370 >> 172.21.15.55:0/117370 conn(0x7f973c068e20 msgr2=0x7f973c0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.440+0000 7f9741b9d700 1 -- 172.21.15.55:0/117370 shutdown_connections 2023-10-06T04:48:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.440+0000 7f9741b9d700 1 -- 172.21.15.55:0/117370 wait complete. 2023-10-06T04:48:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.441+0000 7f9741b9d700 1 Processor -- start 2023-10-06T04:48:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.441+0000 7f9741b9d700 1 -- start start 2023-10-06T04:48:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.442+0000 7f9741b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f973c0cc8d0 0x7f973c153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.442+0000 7f9741b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f973c15b050 0x7f973c0c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.442+0000 7f9741b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f973c15b960 0x7f973c15d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.442+0000 7f9741b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0c88f0 con 0x7f973c15b050 2023-10-06T04:48:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.442+0000 7f9741b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c169ad0 con 0x7f973c0cc8d0 2023-10-06T04:48:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.442+0000 7f9741b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0c8050 con 0x7f973c15b960 2023-10-06T04:48:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.442+0000 7f973bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f973c15b050 0x7f973c0c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.442+0000 7f973bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f973c15b050 0x7f973c0c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58826/0 (socket says 172.21.15.55:58826) 2023-10-06T04:48:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.442+0000 7f973bfff700 1 -- 172.21.15.55:0/1490464565 learned_addr learned my addr 172.21.15.55:0/1490464565 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.442+0000 7f9740b9b700 1 --2- 172.21.15.55:0/1490464565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f973c0cc8d0 0x7f973c153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.443+0000 7f974139c700 1 --2- 172.21.15.55:0/1490464565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f973c15b960 0x7f973c15d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.443+0000 7f973bfff700 1 -- 172.21.15.55:0/1490464565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f973c15b960 msgr2=0x7f973c15d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.443+0000 7f973bfff700 1 --2- 172.21.15.55:0/1490464565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f973c15b960 0x7f973c15d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.443+0000 7f973bfff700 1 -- 172.21.15.55:0/1490464565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f973c0cc8d0 msgr2=0x7f973c153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.443+0000 7f973bfff700 1 --2- 172.21.15.55:0/1490464565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f973c0cc8d0 0x7f973c153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:16.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.443+0000 7f973bfff700 1 -- 172.21.15.55:0/1490464565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9734009ae0 con 0x7f973c15b050 2023-10-06T04:48:16.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.443+0000 7f973bfff700 1 --2- 172.21.15.55:0/1490464565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f973c15b050 0x7f973c0c76e0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f972800c850 tx=0x7f9728008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:16.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.443+0000 7f9738ff9700 1 -- 172.21.15.55:0/1490464565 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728011600 con 0x7f973c15b050 2023-10-06T04:48:16.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.443+0000 7f9738ff9700 1 -- 172.21.15.55:0/1490464565 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9728011760 con 0x7f973c15b050 2023-10-06T04:48:16.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.443+0000 7f9738ff9700 1 -- 172.21.15.55:0/1490464565 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728011b00 con 0x7f973c15b050 2023-10-06T04:48:16.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.443+0000 7f9741b9d700 1 -- 172.21.15.55:0/1490464565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c0c8330 con 0x7f973c15b050 2023-10-06T04:48:16.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.444+0000 7f9741b9d700 1 -- 172.21.15.55:0/1490464565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c1539b0 con 0x7f973c15b050 2023-10-06T04:48:16.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.444+0000 7f9741b9d700 1 -- 172.21.15.55:0/1490464565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c04c3c0 con 0x7f973c15b050 2023-10-06T04:48:16.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.445+0000 7f9738ff9700 1 -- 172.21.15.55:0/1490464565 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 64) v1 ==== 84871+0+0 (secure 0 0 0) 0x7f9728018350 con 0x7f973c15b050 2023-10-06T04:48:16.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.446+0000 7f9738ff9700 1 --2- 172.21.15.55:0/1490464565 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97200670a0 0x7f9720069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:16.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.446+0000 7f9738ff9700 1 -- 172.21.15.55:0/1490464565 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6882+0+0 (secure 0 0 0) 0x7f9728084c00 con 0x7f973c15b050 2023-10-06T04:48:16.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.447+0000 7f9740b9b700 1 --2- 172.21.15.55:0/1490464565 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97200670a0 0x7f9720069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:16.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.447+0000 7f9740b9b700 1 --2- 172.21.15.55:0/1490464565 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97200670a0 0x7f9720069550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9730009af0 tx=0x7f9730009060 comp rx=0 tx=0).ready entity=mgr.8026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:16.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.452+0000 7f9738ff9700 1 -- 172.21.15.55:0/1490464565 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9728051710 con 0x7f973c15b050 2023-10-06T04:48:16.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.683+0000 7f9741b9d700 1 -- 172.21.15.55:0/1490464565 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f973c06b7c0 con 0x7f97200670a0 2023-10-06T04:48:16.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.697+0000 7f9738ff9700 1 -- 172.21.15.55:0/1490464565 <== mgr.8026 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f973c06b7c0 con 0x7f97200670a0 2023-10-06T04:48:16.700 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:16.700 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:52:48", "first": "2023-10-06T03:53:00", "last": "2023-10-06T04:48:00", "last_pruned": "2023-10-06T04:48:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-06T04:48:16.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.699+0000 7f9741b9d700 1 -- 172.21.15.55:0/1490464565 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97200670a0 msgr2=0x7f9720069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:16.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.699+0000 7f9741b9d700 1 --2- 172.21.15.55:0/1490464565 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97200670a0 0x7f9720069550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9730009af0 tx=0x7f9730009060 comp rx=0 tx=0).stop 2023-10-06T04:48:16.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.699+0000 7f9741b9d700 1 -- 172.21.15.55:0/1490464565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f973c15b050 msgr2=0x7f973c0c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:16.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.699+0000 7f9741b9d700 1 --2- 172.21.15.55:0/1490464565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f973c15b050 0x7f973c0c76e0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f972800c850 tx=0x7f9728008040 comp rx=0 tx=0).stop 2023-10-06T04:48:16.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.699+0000 7f9741b9d700 1 -- 172.21.15.55:0/1490464565 shutdown_connections 2023-10-06T04:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.699+0000 7f9741b9d700 1 --2- 172.21.15.55:0/1490464565 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f97200670a0 0x7f9720069550 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.699+0000 7f9741b9d700 1 --2- 172.21.15.55:0/1490464565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f973c15b960 0x7f973c15d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.699+0000 7f9741b9d700 1 --2- 172.21.15.55:0/1490464565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f973c0cc8d0 0x7f973c153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.699+0000 7f9741b9d700 1 --2- 172.21.15.55:0/1490464565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f973c15b050 0x7f973c0c76e0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.699+0000 7f9741b9d700 1 -- 172.21.15.55:0/1490464565 >> 172.21.15.55:0/1490464565 conn(0x7f973c068e20 msgr2=0x7f973c06d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.699+0000 7f9741b9d700 1 -- 172.21.15.55:0/1490464565 shutdown_connections 2023-10-06T04:48:16.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.699+0000 7f9741b9d700 1 -- 172.21.15.55:0/1490464565 wait complete. 2023-10-06T04:48:16.720 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T03:52:48", "first": "2023-10-06T03:53:00", "last": "2023-10-06T04:48:00", "last_pruned": "2023-10-06T04:48:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-06T04:48:16.720 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1M', 'retention': {'n': 55}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T03:52:48', 'first': '2023-10-06T03:53:00', 'last': '2023-10-06T04:48:00', 'last_pruned': '2023-10-06T04:48:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-10-06T04:48:16.720 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-10-06T04:48:16.720 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-10-06T04:48:16.720 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1 2023-10-06T04:48:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.909+0000 7fa4a9b9d700 1 Processor -- start 2023-10-06T04:48:16.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.909+0000 7fa4a9b9d700 1 -- start start 2023-10-06T04:48:16.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.909+0000 7fa4a9b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4a415b050 0x7fa4a40ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.909+0000 7fa4a9b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa4a415bb30 0x7fa4a40ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.909+0000 7fa4a9b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a4157a50 con 0x7fa4a40cc8d0 2023-10-06T04:48:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.909+0000 7fa4a9b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a4158f10 con 0x7fa4a415b050 2023-10-06T04:48:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.909+0000 7fa4a9b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a41591b0 con 0x7fa4a415bb30 2023-10-06T04:48:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.910+0000 7fa4a8b9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4a415b050 0x7fa4a40ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.910+0000 7fa4a8b9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4a415b050 0x7fa4a40ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58842/0 (socket says 172.21.15.55:58842) 2023-10-06T04:48:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.910+0000 7fa4a8b9b700 1 -- 172.21.15.55:0/1888189617 learned_addr learned my addr 172.21.15.55:0/1888189617 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:16.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.910+0000 7fa4a939c700 1 --2- 172.21.15.55:0/1888189617 >> v2:172.21.15.163:3300/0 conn(0x7fa4a415bb30 0x7fa4a40ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.910+0000 7fa4a8b9b700 1 -- 172.21.15.55:0/1888189617 >> v2:172.21.15.163:3300/0 conn(0x7fa4a415bb30 msgr2=0x7fa4a40ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.910+0000 7fa4a8b9b700 1 --2- 172.21.15.55:0/1888189617 >> v2:172.21.15.163:3300/0 conn(0x7fa4a415bb30 0x7fa4a40ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.910+0000 7fa4a8b9b700 1 -- 172.21.15.55:0/1888189617 >> v1:172.21.15.163:6789/0 conn(0x7fa4a40cc8d0 legacy=0x7fa4a415b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.910+0000 7fa4a8b9b700 1 -- 172.21.15.55:0/1888189617 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a40ca9e0 con 0x7fa4a415b050 2023-10-06T04:48:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.910+0000 7fa4a8b9b700 1 --2- 172.21.15.55:0/1888189617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a415b050 0x7fa4a40ca100 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40645e0 tx=0x7fa498009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9bee558e0b6689 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.911+0000 7fa4a37fe700 1 -- 172.21.15.55:0/1888189617 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498009e70 con 0x7fa4a415b050 2023-10-06T04:48:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.911+0000 7fa4a37fe700 1 -- 172.21.15.55:0/1888189617 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49800b860 con 0x7fa4a415b050 2023-10-06T04:48:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.911+0000 7fa4a37fe700 1 -- 172.21.15.55:0/1888189617 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49800bc00 con 0x7fa4a415b050 2023-10-06T04:48:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.911+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/1888189617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a415b050 msgr2=0x7fa4a40ca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.911+0000 7fa4a9b9d700 1 --2- 172.21.15.55:0/1888189617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a415b050 0x7fa4a40ca100 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fa4a40645e0 tx=0x7fa498009250 comp rx=0 tx=0).stop 2023-10-06T04:48:16.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.911+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/1888189617 shutdown_connections 2023-10-06T04:48:16.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.911+0000 7fa4a9b9d700 1 --2- 172.21.15.55:0/1888189617 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a415b050 0x7fa4a40ca100 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:16.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.911+0000 7fa4a9b9d700 1 --2- 172.21.15.55:0/1888189617 >> v2:172.21.15.163:3300/0 conn(0x7fa4a415bb30 0x7fa4a40ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:16.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.911+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/1888189617 >> 172.21.15.55:0/1888189617 conn(0x7fa4a4068e20 msgr2=0x7fa4a40cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:16.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.911+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/1888189617 shutdown_connections 2023-10-06T04:48:16.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.912+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/1888189617 wait complete. 2023-10-06T04:48:16.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.913+0000 7fa4a9b9d700 1 Processor -- start 2023-10-06T04:48:16.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.913+0000 7fa4a9b9d700 1 -- start start 2023-10-06T04:48:16.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.914+0000 7fa4a9b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a40cc8d0 0x7fa4a406dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:16.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.914+0000 7fa4a9b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a415b050 0x7fa4a40b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:16.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.914+0000 7fa4a9b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4a415bb30 0x7fa4a40a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:16.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.914+0000 7fa4a9b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a4153d50 con 0x7fa4a40cc8d0 2023-10-06T04:48:16.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.914+0000 7fa4a9b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40b04d0 con 0x7fa4a415b050 2023-10-06T04:48:16.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.914+0000 7fa4a9b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a416dab0 con 0x7fa4a415bb30 2023-10-06T04:48:16.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.914+0000 7fa4a8b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a40cc8d0 0x7fa4a406dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:16.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.914+0000 7fa4a8b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a40cc8d0 0x7fa4a406dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58850/0 (socket says 172.21.15.55:58850) 2023-10-06T04:48:16.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.914+0000 7fa4a8b9b700 1 -- 172.21.15.55:0/2877900562 learned_addr learned my addr 172.21.15.55:0/2877900562 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:16.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.915+0000 7fa4a3fff700 1 --2- 172.21.15.55:0/2877900562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a415b050 0x7fa4a40b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:16.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.915+0000 7fa4a8b9b700 1 -- 172.21.15.55:0/2877900562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4a415bb30 msgr2=0x7fa4a40a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:16.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.915+0000 7fa4a8b9b700 1 --2- 172.21.15.55:0/2877900562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4a415bb30 0x7fa4a40a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:16.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.915+0000 7fa4a8b9b700 1 -- 172.21.15.55:0/2877900562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a415b050 msgr2=0x7fa4a40b0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:16.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.915+0000 7fa4a8b9b700 1 --2- 172.21.15.55:0/2877900562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a415b050 0x7fa4a40b0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:16.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.915+0000 7fa4a8b9b700 1 -- 172.21.15.55:0/2877900562 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4980094f0 con 0x7fa4a40cc8d0 2023-10-06T04:48:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.915+0000 7fa4a3fff700 1 --2- 172.21.15.55:0/2877900562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a415b050 0x7fa4a40b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.915+0000 7fa4a8b9b700 1 --2- 172.21.15.55:0/2877900562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a40cc8d0 0x7fa4a406dc40 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fa498006fd0 tx=0x7fa498017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.915+0000 7fa4a0ff9700 1 -- 172.21.15.55:0/2877900562 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498011420 con 0x7fa4a40cc8d0 2023-10-06T04:48:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.915+0000 7fa4a0ff9700 1 -- 172.21.15.55:0/2877900562 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4980115a0 con 0x7fa4a40cc8d0 2023-10-06T04:48:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.915+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/2877900562 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a40afe80 con 0x7fa4a40cc8d0 2023-10-06T04:48:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.915+0000 7fa4a0ff9700 1 -- 172.21.15.55:0/2877900562 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498011960 con 0x7fa4a40cc8d0 2023-10-06T04:48:16.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.915+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/2877900562 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a406e180 con 0x7fa4a40cc8d0 2023-10-06T04:48:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.916+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/2877900562 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a40b07b0 con 0x7fa4a40cc8d0 2023-10-06T04:48:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.917+0000 7fa4a0ff9700 1 -- 172.21.15.55:0/2877900562 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 64) v1 ==== 84871+0+0 (secure 0 0 0) 0x7fa4980253f0 con 0x7fa4a40cc8d0 2023-10-06T04:48:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.918+0000 7fa4a0ff9700 1 --2- 172.21.15.55:0/2877900562 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa4880670a0 0x7fa488069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.918+0000 7fa4a0ff9700 1 -- 172.21.15.55:0/2877900562 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6838+0+0 (secure 0 0 0) 0x7fa498086560 con 0x7fa4a40cc8d0 2023-10-06T04:48:16.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.918+0000 7fa4a3fff700 1 --2- 172.21.15.55:0/2877900562 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa4880670a0 0x7fa488069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:16.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.919+0000 7fa4a3fff700 1 --2- 172.21.15.55:0/2877900562 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa4880670a0 0x7fa488069550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4147a80 tx=0x7fa490009040 comp rx=0 tx=0).ready entity=mgr.8026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:16.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:16.924+0000 7fa4a0ff9700 1 -- 172.21.15.55:0/2877900562 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa4980531b0 con 0x7fa4a40cc8d0 2023-10-06T04:48:17.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:17.154+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/2877900562 --> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4a404c3c0 con 0x7fa4880670a0 2023-10-06T04:48:17.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:17.232+0000 7fa4a0ff9700 1 -- 172.21.15.55:0/2877900562 <== mgr.8026 v2:172.21.15.163:6835/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7fa4a404c3c0 con 0x7fa4880670a0 2023-10-06T04:48:17.235 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path /snap_test_dir1 2023-10-06T04:48:17.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:17.235+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/2877900562 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa4880670a0 msgr2=0x7fa488069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:17.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:17.235+0000 7fa4a9b9d700 1 --2- 172.21.15.55:0/2877900562 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa4880670a0 0x7fa488069550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4147a80 tx=0x7fa490009040 comp rx=0 tx=0).stop 2023-10-06T04:48:17.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:17.235+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/2877900562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a40cc8d0 msgr2=0x7fa4a406dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:17.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:17.235+0000 7fa4a9b9d700 1 --2- 172.21.15.55:0/2877900562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a40cc8d0 0x7fa4a406dc40 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fa498006fd0 tx=0x7fa498017000 comp rx=0 tx=0).stop 2023-10-06T04:48:17.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:17.235+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/2877900562 shutdown_connections 2023-10-06T04:48:17.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:17.235+0000 7fa4a9b9d700 1 --2- 172.21.15.55:0/2877900562 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa4880670a0 0x7fa488069550 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:17.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:17.235+0000 7fa4a9b9d700 1 --2- 172.21.15.55:0/2877900562 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4a415bb30 0x7fa4a40a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:17.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:17.235+0000 7fa4a9b9d700 1 --2- 172.21.15.55:0/2877900562 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a40cc8d0 0x7fa4a406dc40 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:17.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:17.235+0000 7fa4a9b9d700 1 --2- 172.21.15.55:0/2877900562 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a415b050 0x7fa4a40b0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:17.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:17.235+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/2877900562 >> 172.21.15.55:0/2877900562 conn(0x7fa4a4068e20 msgr2=0x7fa4a4154760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:17.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:17.235+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/2877900562 shutdown_connections 2023-10-06T04:48:17.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:17.235+0000 7fa4a9b9d700 1 -- 172.21.15.55:0/2877900562 wait complete. 2023-10-06T04:48:17.255 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-10-06T04:48:17.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:17.256 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T04:48:17.296 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_54_00_UTC 2023-10-06T04:48:17.296 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_55_00_UTC 2023-10-06T04:48:17.296 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_56_00_UTC 2023-10-06T04:48:17.296 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_57_00_UTC 2023-10-06T04:48:17.296 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_58_00_UTC 2023-10-06T04:48:17.297 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-03_59_00_UTC 2023-10-06T04:48:17.297 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_00_00_UTC 2023-10-06T04:48:17.297 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_01_00_UTC 2023-10-06T04:48:17.297 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_02_00_UTC 2023-10-06T04:48:17.297 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_03_00_UTC 2023-10-06T04:48:17.297 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_04_00_UTC 2023-10-06T04:48:17.297 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_05_00_UTC 2023-10-06T04:48:17.297 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_06_00_UTC 2023-10-06T04:48:17.297 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_07_00_UTC 2023-10-06T04:48:17.297 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_08_00_UTC 2023-10-06T04:48:17.297 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_09_00_UTC 2023-10-06T04:48:17.297 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_10_00_UTC 2023-10-06T04:48:17.298 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_11_00_UTC 2023-10-06T04:48:17.298 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_12_00_UTC 2023-10-06T04:48:17.298 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_13_01_UTC 2023-10-06T04:48:17.298 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_14_00_UTC 2023-10-06T04:48:17.298 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_15_00_UTC 2023-10-06T04:48:17.298 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_16_00_UTC 2023-10-06T04:48:17.298 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_17_00_UTC 2023-10-06T04:48:17.298 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_18_00_UTC 2023-10-06T04:48:17.298 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_19_00_UTC 2023-10-06T04:48:17.298 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_20_00_UTC 2023-10-06T04:48:17.298 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_21_00_UTC 2023-10-06T04:48:17.299 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_22_00_UTC 2023-10-06T04:48:17.299 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_23_00_UTC 2023-10-06T04:48:17.299 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_24_00_UTC 2023-10-06T04:48:17.299 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_25_00_UTC 2023-10-06T04:48:17.299 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_26_00_UTC 2023-10-06T04:48:17.299 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_27_00_UTC 2023-10-06T04:48:17.299 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_28_00_UTC 2023-10-06T04:48:17.299 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_29_00_UTC 2023-10-06T04:48:17.299 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_30_00_UTC 2023-10-06T04:48:17.299 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_31_00_UTC 2023-10-06T04:48:17.299 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_32_00_UTC 2023-10-06T04:48:17.299 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_33_00_UTC 2023-10-06T04:48:17.300 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_34_00_UTC 2023-10-06T04:48:17.300 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_35_00_UTC 2023-10-06T04:48:17.300 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_36_00_UTC 2023-10-06T04:48:17.300 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_37_00_UTC 2023-10-06T04:48:17.300 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_38_00_UTC 2023-10-06T04:48:17.300 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_39_00_UTC 2023-10-06T04:48:17.300 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_40_00_UTC 2023-10-06T04:48:17.300 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_41_00_UTC 2023-10-06T04:48:17.300 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_42_00_UTC 2023-10-06T04:48:17.300 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_43_00_UTC 2023-10-06T04:48:17.300 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_44_00_UTC 2023-10-06T04:48:17.301 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_45_00_UTC 2023-10-06T04:48:17.301 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_46_00_UTC 2023-10-06T04:48:17.301 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_47_00_UTC 2023-10-06T04:48:17.301 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_48_00_UTC 2023-10-06T04:48:17.302 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-03_54_00_UTC 2023-10-06T04:48:17.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:17.302 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-03_54_00_UTC) 2023-10-06T04:48:17.387 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-03_55_00_UTC 2023-10-06T04:48:17.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:17.388 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-03_55_00_UTC) 2023-10-06T04:48:17.472 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-03_56_00_UTC 2023-10-06T04:48:17.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:17.473 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-03_56_00_UTC) 2023-10-06T04:48:17.557 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-03_57_00_UTC 2023-10-06T04:48:17.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:17.557 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-03_57_00_UTC) 2023-10-06T04:48:17.642 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-03_58_00_UTC 2023-10-06T04:48:17.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:17.643 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-03_58_00_UTC) 2023-10-06T04:48:17.727 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-03_59_00_UTC 2023-10-06T04:48:17.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:17.728 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-03_59_00_UTC) 2023-10-06T04:48:17.812 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_00_00_UTC 2023-10-06T04:48:17.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:17.812 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_00_00_UTC) 2023-10-06T04:48:17.897 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_01_00_UTC 2023-10-06T04:48:17.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:17.897 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_01_00_UTC) 2023-10-06T04:48:17.982 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_02_00_UTC 2023-10-06T04:48:17.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:17.982 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_02_00_UTC) 2023-10-06T04:48:18.067 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_03_00_UTC 2023-10-06T04:48:18.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:18.068 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_03_00_UTC) 2023-10-06T04:48:18.152 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_04_00_UTC 2023-10-06T04:48:18.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:18.153 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_04_00_UTC) 2023-10-06T04:48:18.237 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_05_00_UTC 2023-10-06T04:48:18.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:18.237 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_05_00_UTC) 2023-10-06T04:48:18.321 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_06_00_UTC 2023-10-06T04:48:18.322 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:18.322 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_06_00_UTC) 2023-10-06T04:48:18.406 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_07_00_UTC 2023-10-06T04:48:18.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:18.407 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_07_00_UTC) 2023-10-06T04:48:18.490 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_08_00_UTC 2023-10-06T04:48:18.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:18.491 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_08_00_UTC) 2023-10-06T04:48:18.574 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_09_00_UTC 2023-10-06T04:48:18.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:18.575 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_09_00_UTC) 2023-10-06T04:48:18.659 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_10_00_UTC 2023-10-06T04:48:18.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:18.660 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_10_00_UTC) 2023-10-06T04:48:18.778 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_11_00_UTC 2023-10-06T04:48:18.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:18.779 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_11_00_UTC) 2023-10-06T04:48:18.840 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_12_00_UTC 2023-10-06T04:48:18.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:18.841 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_12_00_UTC) 2023-10-06T04:48:18.884 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_13_01_UTC 2023-10-06T04:48:18.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:18.884 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_13_01_UTC) 2023-10-06T04:48:18.968 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_14_00_UTC 2023-10-06T04:48:18.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:18.969 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_14_00_UTC) 2023-10-06T04:48:19.053 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_15_00_UTC 2023-10-06T04:48:19.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:19.053 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_15_00_UTC) 2023-10-06T04:48:19.137 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_16_00_UTC 2023-10-06T04:48:19.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:19.137 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_16_00_UTC) 2023-10-06T04:48:19.221 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_17_00_UTC 2023-10-06T04:48:19.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:19.222 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_17_00_UTC) 2023-10-06T04:48:19.305 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_18_00_UTC 2023-10-06T04:48:19.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:19.305 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_18_00_UTC) 2023-10-06T04:48:19.389 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_19_00_UTC 2023-10-06T04:48:19.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:19.389 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_19_00_UTC) 2023-10-06T04:48:19.472 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_20_00_UTC 2023-10-06T04:48:19.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:19.473 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_20_00_UTC) 2023-10-06T04:48:19.556 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_21_00_UTC 2023-10-06T04:48:19.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:19.556 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_21_00_UTC) 2023-10-06T04:48:19.639 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_22_00_UTC 2023-10-06T04:48:19.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:19.639 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_22_00_UTC) 2023-10-06T04:48:19.734 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_23_00_UTC 2023-10-06T04:48:19.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:19.734 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_23_00_UTC) 2023-10-06T04:48:19.777 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_24_00_UTC 2023-10-06T04:48:19.777 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:19.778 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_24_00_UTC) 2023-10-06T04:48:19.860 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_25_00_UTC 2023-10-06T04:48:19.861 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:19.861 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_25_00_UTC) 2023-10-06T04:48:19.943 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_26_00_UTC 2023-10-06T04:48:19.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:19.944 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_26_00_UTC) 2023-10-06T04:48:20.029 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_27_00_UTC 2023-10-06T04:48:20.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:20.030 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_27_00_UTC) 2023-10-06T04:48:20.113 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_28_00_UTC 2023-10-06T04:48:20.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:20.113 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_28_00_UTC) 2023-10-06T04:48:20.196 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_29_00_UTC 2023-10-06T04:48:20.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:20.197 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_29_00_UTC) 2023-10-06T04:48:20.286 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_30_00_UTC 2023-10-06T04:48:20.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:20.287 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_30_00_UTC) 2023-10-06T04:48:20.330 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_31_00_UTC 2023-10-06T04:48:20.330 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:20.330 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_31_00_UTC) 2023-10-06T04:48:20.413 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_32_00_UTC 2023-10-06T04:48:20.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:20.413 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_32_00_UTC) 2023-10-06T04:48:20.497 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_33_00_UTC 2023-10-06T04:48:20.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:20.497 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_33_00_UTC) 2023-10-06T04:48:20.582 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_34_00_UTC 2023-10-06T04:48:20.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:20.582 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_34_00_UTC) 2023-10-06T04:48:20.665 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_35_00_UTC 2023-10-06T04:48:20.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:20.666 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_35_00_UTC) 2023-10-06T04:48:20.762 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_36_00_UTC 2023-10-06T04:48:20.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:20.762 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_36_00_UTC) 2023-10-06T04:48:20.817 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_37_00_UTC 2023-10-06T04:48:20.817 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:20.817 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_37_00_UTC) 2023-10-06T04:48:20.860 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_38_00_UTC 2023-10-06T04:48:20.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:20.860 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_38_00_UTC) 2023-10-06T04:48:20.966 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_39_00_UTC 2023-10-06T04:48:20.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:20.967 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_39_00_UTC) 2023-10-06T04:48:21.018 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_40_00_UTC 2023-10-06T04:48:21.018 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:21.019 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_40_00_UTC) 2023-10-06T04:48:21.066 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_41_00_UTC 2023-10-06T04:48:21.066 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:21.066 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_41_00_UTC) 2023-10-06T04:48:21.150 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_42_00_UTC 2023-10-06T04:48:21.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:21.150 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_42_00_UTC) 2023-10-06T04:48:21.233 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_43_00_UTC 2023-10-06T04:48:21.234 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:21.234 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_43_00_UTC) 2023-10-06T04:48:21.317 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_44_00_UTC 2023-10-06T04:48:21.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:21.317 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_44_00_UTC) 2023-10-06T04:48:21.400 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_45_00_UTC 2023-10-06T04:48:21.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:21.400 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_45_00_UTC) 2023-10-06T04:48:21.483 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_46_00_UTC 2023-10-06T04:48:21.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:21.484 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_46_00_UTC) 2023-10-06T04:48:21.566 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_47_00_UTC 2023-10-06T04:48:21.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:21.567 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_47_00_UTC) 2023-10-06T04:48:21.650 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-04_48_00_UTC 2023-10-06T04:48:21.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:21.650 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-04_48_00_UTC) 2023-10-06T04:48:21.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:21.751 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-06T04:48:21.791 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T04:48:22.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.080+0000 7f83d679d700 1 Processor -- start 2023-10-06T04:48:22.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.080+0000 7f83d679d700 1 -- start start 2023-10-06T04:48:22.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.081+0000 7f83d679d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d00cb1a0 0x7f83d015a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:22.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.081+0000 7f83d679d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f83d015a8a0 0x7f83d006f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:22.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.081+0000 7f83d679d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f83d006f5a0 0x7f83d00ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:22.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.081+0000 7f83d679d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d015b2e0 con 0x7f83d015a8a0 2023-10-06T04:48:22.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.081+0000 7f83d679d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d006ebd0 con 0x7f83d006f5a0 2023-10-06T04:48:22.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.081+0000 7f83d679d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d015ad20 con 0x7f83d00cb1a0 2023-10-06T04:48:22.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.082+0000 7f83d4f9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f83d015a8a0 0x7f83d006f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:22.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.082+0000 7f83d4f9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f83d015a8a0 0x7f83d006f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58866/0 (socket says 172.21.15.55:58866) 2023-10-06T04:48:22.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.082+0000 7f83d4f9a700 1 -- 172.21.15.55:0/976634813 learned_addr learned my addr 172.21.15.55:0/976634813 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:22.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.082+0000 7f83d579b700 1 --2- 172.21.15.55:0/976634813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d00cb1a0 0x7f83d015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:22.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.082+0000 7f83d5f9c700 1 --2- 172.21.15.55:0/976634813 >> v2:172.21.15.163:3300/0 conn(0x7f83d006f5a0 0x7f83d00ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:22.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.082+0000 7f83d4f9a700 1 -- 172.21.15.55:0/976634813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d00cb1a0 msgr2=0x7f83d015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:22.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.082+0000 7f83d4f9a700 1 --2- 172.21.15.55:0/976634813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d00cb1a0 0x7f83d015a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:22.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.082+0000 7f83d4f9a700 1 -- 172.21.15.55:0/976634813 >> v2:172.21.15.163:3300/0 conn(0x7f83d006f5a0 msgr2=0x7f83d00ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:22.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.082+0000 7f83d4f9a700 1 --2- 172.21.15.55:0/976634813 >> v2:172.21.15.163:3300/0 conn(0x7f83d006f5a0 0x7f83d00ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:22.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.082+0000 7f83d4f9a700 1 -- 172.21.15.55:0/976634813 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d015b000 con 0x7f83d015a8a0 2023-10-06T04:48:22.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.082+0000 7f83d4f9a700 1 --2- 172.21.15.55:0/976634813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d015a8a0 0x7f83d006f060 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f83d006d580 tx=0x7f83c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5246e88a154da9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:22.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.083+0000 7f83c7fff700 1 -- 172.21.15.55:0/976634813 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c000b8f0 con 0x7f83d015a8a0 2023-10-06T04:48:22.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.083+0000 7f83c7fff700 1 -- 172.21.15.55:0/976634813 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c000ba50 con 0x7f83d015a8a0 2023-10-06T04:48:22.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.083+0000 7f83c7fff700 1 -- 172.21.15.55:0/976634813 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c000bdf0 con 0x7f83d015a8a0 2023-10-06T04:48:22.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.083+0000 7f83d679d700 1 -- 172.21.15.55:0/976634813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d015a8a0 msgr2=0x7f83d006f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:22.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.083+0000 7f83d679d700 1 --2- 172.21.15.55:0/976634813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d015a8a0 0x7f83d006f060 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f83d006d580 tx=0x7f83c0009250 comp rx=0 tx=0).stop 2023-10-06T04:48:22.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.083+0000 7f83d679d700 1 -- 172.21.15.55:0/976634813 shutdown_connections 2023-10-06T04:48:22.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.083+0000 7f83d679d700 1 --2- 172.21.15.55:0/976634813 >> v2:172.21.15.163:3300/0 conn(0x7f83d006f5a0 0x7f83d00ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:22.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.083+0000 7f83d679d700 1 --2- 172.21.15.55:0/976634813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d00cb1a0 0x7f83d015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:22.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.083+0000 7f83d679d700 1 --2- 172.21.15.55:0/976634813 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d015a8a0 0x7f83d006f060 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:22.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.083+0000 7f83d679d700 1 -- 172.21.15.55:0/976634813 >> 172.21.15.55:0/976634813 conn(0x7f83d00cc1e0 msgr2=0x7f83d00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:22.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.083+0000 7f83d679d700 1 -- 172.21.15.55:0/976634813 shutdown_connections 2023-10-06T04:48:22.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.083+0000 7f83d679d700 1 -- 172.21.15.55:0/976634813 wait complete. 2023-10-06T04:48:22.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.085+0000 7f83d679d700 1 Processor -- start 2023-10-06T04:48:22.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.085+0000 7f83d679d700 1 -- start start 2023-10-06T04:48:22.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d679d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d006f5a0 0x7f83d0153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:22.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d679d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d00cb1a0 0x7f83d016a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:22.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d679d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d015d580 0x7f83d015d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:22.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d679d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d015c920 con 0x7f83d00cb1a0 2023-10-06T04:48:22.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d679d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d015c490 con 0x7f83d015d580 2023-10-06T04:48:22.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d679d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d015df10 con 0x7f83d006f5a0 2023-10-06T04:48:22.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d4f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d00cb1a0 0x7f83d016a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:22.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d4f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d00cb1a0 0x7f83d016a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58876/0 (socket says 172.21.15.55:58876) 2023-10-06T04:48:22.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d4f9a700 1 -- 172.21.15.55:0/2645898458 learned_addr learned my addr 172.21.15.55:0/2645898458 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:22.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d579b700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d006f5a0 0x7f83d0153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:22.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d5f9c700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d015d580 0x7f83d015d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:22.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d4f9a700 1 -- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d006f5a0 msgr2=0x7f83d0153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:22.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d4f9a700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d006f5a0 0x7f83d0153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:22.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d4f9a700 1 -- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d015d580 msgr2=0x7f83d015d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:22.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d4f9a700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d015d580 0x7f83d015d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:22.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d4f9a700 1 -- 172.21.15.55:0/2645898458 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c00094f0 con 0x7f83d00cb1a0 2023-10-06T04:48:22.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.086+0000 7f83d5f9c700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d015d580 0x7f83d015d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:48:22.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.087+0000 7f83d4f9a700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d00cb1a0 0x7f83d016a880 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f83c0006fd0 tx=0x7f83c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:22.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.087+0000 7f83c57fa700 1 -- 172.21.15.55:0/2645898458 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c0024440 con 0x7f83d00cb1a0 2023-10-06T04:48:22.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.087+0000 7f83c57fa700 1 -- 172.21.15.55:0/2645898458 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c00245a0 con 0x7f83d00cb1a0 2023-10-06T04:48:22.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.087+0000 7f83c57fa700 1 -- 172.21.15.55:0/2645898458 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c0024960 con 0x7f83d00cb1a0 2023-10-06T04:48:22.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.087+0000 7f83d679d700 1 -- 172.21.15.55:0/2645898458 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d015b880 con 0x7f83d00cb1a0 2023-10-06T04:48:22.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.087+0000 7f83d679d700 1 -- 172.21.15.55:0/2645898458 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d015be40 con 0x7f83d00cb1a0 2023-10-06T04:48:22.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.088+0000 7f83d679d700 1 -- 172.21.15.55:0/2645898458 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d015c600 con 0x7f83d00cb1a0 2023-10-06T04:48:22.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.089+0000 7f83c57fa700 1 -- 172.21.15.55:0/2645898458 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 64) v1 ==== 84871+0+0 (secure 0 0 0) 0x7f83c001a420 con 0x7f83d00cb1a0 2023-10-06T04:48:22.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.090+0000 7f83c57fa700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f83b40670a0 0x7f83b4069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:22.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.090+0000 7f83c57fa700 1 -- 172.21.15.55:0/2645898458 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6882+0+0 (secure 0 0 0) 0x7f83c0086920 con 0x7f83d00cb1a0 2023-10-06T04:48:22.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.090+0000 7f83d579b700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f83b40670a0 0x7f83b4069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:22.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.091+0000 7f83d579b700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f83b40670a0 0x7f83b4069550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f83cc006fd0 tx=0x7f83cc008040 comp rx=0 tx=0).ready entity=mgr.8026 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:22.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.095+0000 7f83c57fa700 1 -- 172.21.15.55:0/2645898458 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83c0053540 con 0x7f83d00cb1a0 2023-10-06T04:48:22.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.350+0000 7f83d679d700 1 -- 172.21.15.55:0/2645898458 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f83d006e860 con 0x7f83d00cb1a0 2023-10-06T04:48:22.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.714+0000 7f83c57fa700 1 -- 172.21.15.55:0/2645898458 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 65) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f83c0053910 con 0x7f83d00cb1a0 2023-10-06T04:48:22.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.715+0000 7f83c57fa700 1 -- 172.21.15.55:0/2645898458 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v65) v1 ==== 93+0+0 (secure 0 0 0) 0x7f83c0056fd0 con 0x7f83d00cb1a0 2023-10-06T04:48:22.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.719+0000 7f83d679d700 1 -- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f83b40670a0 msgr2=0x7f83b4069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:22.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.719+0000 7f83d679d700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f83b40670a0 0x7f83b4069550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f83cc006fd0 tx=0x7f83cc008040 comp rx=0 tx=0).stop 2023-10-06T04:48:22.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.719+0000 7f83d679d700 1 -- 172.21.15.55:0/2645898458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d00cb1a0 msgr2=0x7f83d016a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:22.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.719+0000 7f83d679d700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d00cb1a0 0x7f83d016a880 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f83c0006fd0 tx=0x7f83c0017000 comp rx=0 tx=0).stop 2023-10-06T04:48:22.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.719+0000 7f83d679d700 1 -- 172.21.15.55:0/2645898458 shutdown_connections 2023-10-06T04:48:22.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.719+0000 7f83d679d700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f83b40670a0 0x7f83b4069550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:22.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.719+0000 7f83d679d700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83d015d580 0x7f83d015d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:22.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.719+0000 7f83d679d700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83d006f5a0 0x7f83d0153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:22.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.719+0000 7f83d679d700 1 --2- 172.21.15.55:0/2645898458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83d00cb1a0 0x7f83d016a880 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:22.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.719+0000 7f83d679d700 1 -- 172.21.15.55:0/2645898458 >> 172.21.15.55:0/2645898458 conn(0x7f83d00cc1e0 msgr2=0x7f83d0155ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:22.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.719+0000 7f83d679d700 1 -- 172.21.15.55:0/2645898458 shutdown_connections 2023-10-06T04:48:22.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:22.720+0000 7f83d679d700 1 -- 172.21.15.55:0/2645898458 wait complete. 2023-10-06T04:48:22.752 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:48:22.752 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:48:22.843 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:48:22.927 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:48:22.927 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T04:48:22.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:22.927 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T04:48:23.029 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T04:48:23.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:48:23.029 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:48:23.071 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:48:23.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.299+0000 7f59fcf9d700 1 Processor -- start 2023-10-06T04:48:23.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.299+0000 7f59fcf9d700 1 -- start start 2023-10-06T04:48:23.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.302+0000 7f59fcf9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59f8156310 0x7f59f80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:23.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.302+0000 7f59fcf9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f59f80ae710 0x7f59f80c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:23.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.302+0000 7f59fcf9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f80ca860 con 0x7f59f81566e0 2023-10-06T04:48:23.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.302+0000 7f59fcf9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f80cab40 con 0x7f59f80ae710 2023-10-06T04:48:23.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.302+0000 7f59fcf9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f80cae20 con 0x7f59f8156310 2023-10-06T04:48:23.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.302+0000 7f59f77fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59f8156310 0x7f59f80c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:23.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.303+0000 7f59f77fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59f8156310 0x7f59f80c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:51838/0 (socket says 172.21.15.55:51838) 2023-10-06T04:48:23.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.303+0000 7f59f77fe700 1 -- 172.21.15.55:0/3398121678 learned_addr learned my addr 172.21.15.55:0/3398121678 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:23.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.303+0000 7f59f7fff700 1 --2- 172.21.15.55:0/3398121678 >> v2:172.21.15.55:3300/0 conn(0x7f59f80ae710 0x7f59f80c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:23.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.303+0000 7f59f77fe700 1 -- 172.21.15.55:0/3398121678 >> v2:172.21.15.55:3300/0 conn(0x7f59f80ae710 msgr2=0x7f59f80c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:23.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.303+0000 7f59f77fe700 1 --2- 172.21.15.55:0/3398121678 >> v2:172.21.15.55:3300/0 conn(0x7f59f80ae710 0x7f59f80c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.303+0000 7f59f77fe700 1 -- 172.21.15.55:0/3398121678 >> v1:172.21.15.163:6789/0 conn(0x7f59f81566e0 legacy=0x7f59f80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:23.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.303+0000 7f59f77fe700 1 -- 172.21.15.55:0/3398121678 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f80cb100 con 0x7f59f8156310 2023-10-06T04:48:23.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.304+0000 7f59f77fe700 1 --2- 172.21.15.55:0/3398121678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59f8156310 0x7f59f80c9dc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f59e80099b0 tx=0x7f59e8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da42fa8fdd978dff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:23.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.304+0000 7f59f67fc700 1 -- 172.21.15.55:0/3398121678 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8009ea0 con 0x7f59f8156310 2023-10-06T04:48:23.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.304+0000 7f59f67fc700 1 -- 172.21.15.55:0/3398121678 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e800b860 con 0x7f59f8156310 2023-10-06T04:48:23.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.304+0000 7f59f67fc700 1 -- 172.21.15.55:0/3398121678 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e800bc00 con 0x7f59f8156310 2023-10-06T04:48:23.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.305+0000 7f59fcf9d700 1 -- 172.21.15.55:0/3398121678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59f8156310 msgr2=0x7f59f80c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:23.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.305+0000 7f59fcf9d700 1 --2- 172.21.15.55:0/3398121678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59f8156310 0x7f59f80c9dc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f59e80099b0 tx=0x7f59e8009250 comp rx=0 tx=0).stop 2023-10-06T04:48:23.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.305+0000 7f59fcf9d700 1 -- 172.21.15.55:0/3398121678 shutdown_connections 2023-10-06T04:48:23.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.305+0000 7f59fcf9d700 1 --2- 172.21.15.55:0/3398121678 >> v2:172.21.15.55:3300/0 conn(0x7f59f80ae710 0x7f59f80c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.305+0000 7f59fcf9d700 1 --2- 172.21.15.55:0/3398121678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59f8156310 0x7f59f80c9dc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.305+0000 7f59fcf9d700 1 -- 172.21.15.55:0/3398121678 >> 172.21.15.55:0/3398121678 conn(0x7f59f80cc1f0 msgr2=0x7f59f80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:23.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.306+0000 7f59fcf9d700 1 -- 172.21.15.55:0/3398121678 shutdown_connections 2023-10-06T04:48:23.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.306+0000 7f59fcf9d700 1 -- 172.21.15.55:0/3398121678 wait complete. 2023-10-06T04:48:23.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.308+0000 7f59fcf9d700 1 Processor -- start 2023-10-06T04:48:23.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.309+0000 7f59fcf9d700 1 -- start start 2023-10-06T04:48:23.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.310+0000 7f59fcf9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59f80ae710 0x7f59f80c3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:23.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.310+0000 7f59fcf9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59f8156310 0x7f59f80c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:23.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.311+0000 7f59fcf9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59f81566e0 0x7f59f80c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:23.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.311+0000 7f59fcf9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f8159d60 con 0x7f59f8156310 2023-10-06T04:48:23.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.311+0000 7f59fcf9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f8159150 con 0x7f59f81566e0 2023-10-06T04:48:23.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.311+0000 7f59fcf9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f8159430 con 0x7f59f80ae710 2023-10-06T04:48:23.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.311+0000 7f59f7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59f81566e0 0x7f59f80c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:23.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.311+0000 7f59f77fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59f80ae710 0x7f59f80c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:23.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.311+0000 7f59f7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59f81566e0 0x7f59f80c4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:60988/0 (socket says 172.21.15.55:60988) 2023-10-06T04:48:23.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.311+0000 7f59f7fff700 1 -- 172.21.15.55:0/2354288443 learned_addr learned my addr 172.21.15.55:0/2354288443 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:23.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.312+0000 7f59f6ffd700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59f8156310 0x7f59f80c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:23.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.312+0000 7f59f77fe700 1 -- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59f81566e0 msgr2=0x7f59f80c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.312+0000 7f59f77fe700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59f81566e0 0x7f59f80c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.312+0000 7f59f77fe700 1 -- 172.21.15.55:0/2354288443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59f8156310 msgr2=0x7f59f80c3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.312+0000 7f59f77fe700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59f8156310 0x7f59f80c3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.312+0000 7f59f77fe700 1 -- 172.21.15.55:0/2354288443 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e80094f0 con 0x7f59f80ae710 2023-10-06T04:48:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.313+0000 7f59f7fff700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59f81566e0 0x7f59f80c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.313+0000 7f59f6ffd700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59f8156310 0x7f59f80c3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:23.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.313+0000 7f59f77fe700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59f80ae710 0x7f59f80c3690 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59f806d1e0 tx=0x7f59e8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:23.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.314+0000 7f59df7fe700 1 -- 172.21.15.55:0/2354288443 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e800bbe0 con 0x7f59f80ae710 2023-10-06T04:48:23.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.314+0000 7f59fcf9d700 1 -- 172.21.15.55:0/2354288443 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f8159710 con 0x7f59f80ae710 2023-10-06T04:48:23.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.314+0000 7f59fcf9d700 1 -- 172.21.15.55:0/2354288443 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f80c3bd0 con 0x7f59f80ae710 2023-10-06T04:48:23.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.315+0000 7f59df7fe700 1 -- 172.21.15.55:0/2354288443 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e8011420 con 0x7f59f80ae710 2023-10-06T04:48:23.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.315+0000 7f59df7fe700 1 -- 172.21.15.55:0/2354288443 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e800bbe0 con 0x7f59f80ae710 2023-10-06T04:48:23.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.317+0000 7f59fcf9d700 1 -- 172.21.15.55:0/2354288443 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d80052b0 con 0x7f59f80ae710 2023-10-06T04:48:23.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.318+0000 7f59df7fe700 1 -- 172.21.15.55:0/2354288443 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 65) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f59e8011860 con 0x7f59f80ae710 2023-10-06T04:48:23.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.320+0000 7f59df7fe700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f59e4067130 0x7f59e40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:23.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.320+0000 7f59df7fe700 1 -- 172.21.15.55:0/2354288443 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6882+0+0 (secure 0 0 0) 0x7f59e80862c0 con 0x7f59f80ae710 2023-10-06T04:48:23.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.320+0000 7f59f6ffd700 1 -- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f59e4067130 msgr2=0x7f59e40695e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:23.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.320+0000 7f59f6ffd700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f59e4067130 0x7f59e40695e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:23.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.326+0000 7f59df7fe700 1 -- 172.21.15.55:0/2354288443 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59e8052ee0 con 0x7f59f80ae710 2023-10-06T04:48:23.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.521+0000 7f59f6ffd700 1 -- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f59e4067130 msgr2=0x7f59e40695e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:23.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.521+0000 7f59f6ffd700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f59e4067130 0x7f59e40695e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:23.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.620+0000 7f59fcf9d700 1 -- 172.21.15.55:0/2354288443 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59d8005560 con 0x7f59f80ae710 2023-10-06T04:48:23.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.621+0000 7f59df7fe700 1 -- 172.21.15.55:0/2354288443 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f59e8056500 con 0x7f59f80ae710 2023-10-06T04:48:23.623 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:23.623 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2624033145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2624033145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2624033145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3851023466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3851023466},{"type":"v1","addr":"172.21.15.55:6834","nonce":3851023466}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/2633241022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2633241022},{"type":"v1","addr":"172.21.15.163:6833","nonce":2633241022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:52:30.648704+0000","modified":"2023-10-06T03:52:34.152554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7879":{"gid":7879,"name":"d","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6836/246684871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":246684871},{"type":"v1","addr":"172.21.15.163:6836","nonce":246684871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T04:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.624+0000 7f59fcf9d700 1 -- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f59e4067130 msgr2=0x7f59e40695e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.624+0000 7f59fcf9d700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f59e4067130 0x7f59e40695e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.624+0000 7f59fcf9d700 1 -- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59f80ae710 msgr2=0x7f59f80c3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.624+0000 7f59fcf9d700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59f80ae710 0x7f59f80c3690 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f59f806d1e0 tx=0x7f59e8017000 comp rx=0 tx=0).stop 2023-10-06T04:48:23.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.624+0000 7f59fcf9d700 1 -- 172.21.15.55:0/2354288443 shutdown_connections 2023-10-06T04:48:23.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.624+0000 7f59fcf9d700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f59e4067130 0x7f59e40695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.624+0000 7f59fcf9d700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59f81566e0 0x7f59f80c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.624+0000 7f59fcf9d700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59f80ae710 0x7f59f80c3690 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.624+0000 7f59fcf9d700 1 --2- 172.21.15.55:0/2354288443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59f8156310 0x7f59f80c3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.624+0000 7f59fcf9d700 1 -- 172.21.15.55:0/2354288443 >> 172.21.15.55:0/2354288443 conn(0x7f59f80cc1f0 msgr2=0x7f59f80af650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:23.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.624+0000 7f59fcf9d700 1 -- 172.21.15.55:0/2354288443 shutdown_connections 2023-10-06T04:48:23.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.624+0000 7f59fcf9d700 1 -- 172.21.15.55:0/2354288443 wait complete. 2023-10-06T04:48:23.627 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 70 2023-10-06T04:48:23.644 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:48:23.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.837+0000 7f4978d9d700 1 Processor -- start 2023-10-06T04:48:23.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.837+0000 7f4978d9d700 1 -- start start 2023-10-06T04:48:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.838+0000 7f4978d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4974156310 0x7f49740c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.838+0000 7f4978d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f49741566e0 0x7f49740ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:23.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.838+0000 7f4978d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49740ca300 0x7f49740ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.838+0000 7f4978d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49740ca860 con 0x7f4974156310 2023-10-06T04:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.838+0000 7f4978d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49740cab40 con 0x7f49741566e0 2023-10-06T04:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.838+0000 7f4978d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49740cae20 con 0x7f49740ca300 2023-10-06T04:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.838+0000 7f49737fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4974156310 0x7f49740c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.839+0000 7f49737fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4974156310 0x7f49740c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58334/0 (socket says 172.21.15.55:58334) 2023-10-06T04:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.839+0000 7f49737fe700 1 -- 172.21.15.55:0/2109972602 learned_addr learned my addr 172.21.15.55:0/2109972602 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.839+0000 7f4972ffd700 1 --2- 172.21.15.55:0/2109972602 >> v2:172.21.15.163:3300/0 conn(0x7f49741566e0 0x7f49740ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.839+0000 7f4973fff700 1 --2- 172.21.15.55:0/2109972602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49740ca300 0x7f49740ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.839+0000 7f49737fe700 1 -- 172.21.15.55:0/2109972602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49740ca300 msgr2=0x7f49740ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:23.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.839+0000 7f49737fe700 1 --2- 172.21.15.55:0/2109972602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49740ca300 0x7f49740ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.839+0000 7f49737fe700 1 -- 172.21.15.55:0/2109972602 >> v2:172.21.15.163:3300/0 conn(0x7f49741566e0 msgr2=0x7f49740ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.839+0000 7f49737fe700 1 --2- 172.21.15.55:0/2109972602 >> v2:172.21.15.163:3300/0 conn(0x7f49741566e0 0x7f49740ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.839+0000 7f49737fe700 1 -- 172.21.15.55:0/2109972602 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49740cb100 con 0x7f4974156310 2023-10-06T04:48:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.839+0000 7f49737fe700 1 --2- 172.21.15.55:0/2109972602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4974156310 0x7f49740c9dc0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f497406c000 tx=0x7f4964009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6daea98d95422427 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.840+0000 7f49727fc700 1 -- 172.21.15.55:0/2109972602 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496400b640 con 0x7f4974156310 2023-10-06T04:48:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.840+0000 7f49727fc700 1 -- 172.21.15.55:0/2109972602 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496400b7a0 con 0x7f4974156310 2023-10-06T04:48:23.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.840+0000 7f49727fc700 1 -- 172.21.15.55:0/2109972602 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496400bb60 con 0x7f4974156310 2023-10-06T04:48:23.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.840+0000 7f4978d9d700 1 -- 172.21.15.55:0/2109972602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4974156310 msgr2=0x7f49740c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:23.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.840+0000 7f4978d9d700 1 --2- 172.21.15.55:0/2109972602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4974156310 0x7f49740c9dc0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f497406c000 tx=0x7f4964009250 comp rx=0 tx=0).stop 2023-10-06T04:48:23.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.840+0000 7f4978d9d700 1 -- 172.21.15.55:0/2109972602 shutdown_connections 2023-10-06T04:48:23.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.840+0000 7f4978d9d700 1 --2- 172.21.15.55:0/2109972602 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49740ca300 0x7f49740ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.840+0000 7f4978d9d700 1 --2- 172.21.15.55:0/2109972602 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4974156310 0x7f49740c9dc0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.840+0000 7f4978d9d700 1 --2- 172.21.15.55:0/2109972602 >> v2:172.21.15.163:3300/0 conn(0x7f49741566e0 0x7f49740ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.840+0000 7f4978d9d700 1 -- 172.21.15.55:0/2109972602 >> 172.21.15.55:0/2109972602 conn(0x7f49740cc1f0 msgr2=0x7f49740cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:23.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.841+0000 7f4978d9d700 1 -- 172.21.15.55:0/2109972602 shutdown_connections 2023-10-06T04:48:23.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.841+0000 7f4978d9d700 1 -- 172.21.15.55:0/2109972602 wait complete. 2023-10-06T04:48:23.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.842+0000 7f4978d9d700 1 Processor -- start 2023-10-06T04:48:23.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.842+0000 7f4978d9d700 1 -- start start 2023-10-06T04:48:23.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.843+0000 7f4978d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49740ca300 0x7f49741464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:23.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.843+0000 7f4978d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4974156310 0x7f49740afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:23.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.843+0000 7f4978d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49741566e0 0x7f49740b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:23.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.843+0000 7f4978d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4974158450 con 0x7f4974156310 2023-10-06T04:48:23.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.843+0000 7f4978d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49740af270 con 0x7f49740ca300 2023-10-06T04:48:23.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.843+0000 7f4978d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49741586a0 con 0x7f49741566e0 2023-10-06T04:48:23.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.843+0000 7f49737fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49740ca300 0x7f49741464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:23.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.843+0000 7f4972ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4974156310 0x7f49740afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:23.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.843+0000 7f4973fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49741566e0 0x7f49740b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:23.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.843+0000 7f4972ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4974156310 0x7f49740afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58350/0 (socket says 172.21.15.55:58350) 2023-10-06T04:48:23.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.843+0000 7f49737fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49740ca300 0x7f49741464e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:60998/0 (socket says 172.21.15.55:60998) 2023-10-06T04:48:23.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.843+0000 7f4972ffd700 1 -- 172.21.15.55:0/908005008 learned_addr learned my addr 172.21.15.55:0/908005008 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:23.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.843+0000 7f4973fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49741566e0 0x7f49740b03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:51870/0 (socket says 172.21.15.55:51870) 2023-10-06T04:48:23.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.844+0000 7f4973fff700 1 -- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49740ca300 msgr2=0x7f49741464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:23.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.844+0000 7f4973fff700 1 --2- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49740ca300 0x7f49741464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.844+0000 7f4973fff700 1 -- 172.21.15.55:0/908005008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4974156310 msgr2=0x7f49740afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:23.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.844+0000 7f4973fff700 1 --2- 172.21.15.55:0/908005008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4974156310 0x7f49740afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:23.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.844+0000 7f4973fff700 1 -- 172.21.15.55:0/908005008 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49640094f0 con 0x7f49741566e0 2023-10-06T04:48:23.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.844+0000 7f49737fe700 1 --2- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49740ca300 0x7f49741464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.844+0000 7f4972ffd700 1 --2- 172.21.15.55:0/908005008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4974156310 0x7f49740afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.844+0000 7f4973fff700 1 --2- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49741566e0 0x7f49740b03b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f496c009e50 tx=0x7f496c00b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.845+0000 7f495b7fe700 1 -- 172.21.15.55:0/908005008 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c00caa0 con 0x7f49741566e0 2023-10-06T04:48:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.845+0000 7f495b7fe700 1 -- 172.21.15.55:0/908005008 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c00f040 con 0x7f49741566e0 2023-10-06T04:48:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.845+0000 7f495b7fe700 1 -- 172.21.15.55:0/908005008 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c00ce50 con 0x7f49741566e0 2023-10-06T04:48:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.845+0000 7f4978d9d700 1 -- 172.21.15.55:0/908005008 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4974158980 con 0x7f49741566e0 2023-10-06T04:48:23.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.845+0000 7f4978d9d700 1 -- 172.21.15.55:0/908005008 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4974158f40 con 0x7f49741566e0 2023-10-06T04:48:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.846+0000 7f4978d9d700 1 -- 172.21.15.55:0/908005008 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4974159160 con 0x7f49741566e0 2023-10-06T04:48:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.848+0000 7f495b7fe700 1 -- 172.21.15.55:0/908005008 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 65) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f496c014690 con 0x7f49741566e0 2023-10-06T04:48:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.848+0000 7f495b7fe700 1 --2- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4960067050 0x7f4960069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:23.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.849+0000 7f495b7fe700 1 -- 172.21.15.55:0/908005008 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6882+0+0 (secure 0 0 0) 0x7f496c0857c0 con 0x7f49741566e0 2023-10-06T04:48:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.849+0000 7f49737fe700 1 -- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4960067050 msgr2=0x7f4960069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:23.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.849+0000 7f49737fe700 1 --2- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4960067050 0x7f4960069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:23.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:23.853+0000 7f495b7fe700 1 -- 172.21.15.55:0/908005008 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f496c055e00 con 0x7f49741566e0 2023-10-06T04:48:24.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.050+0000 7f49737fe700 1 -- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4960067050 msgr2=0x7f4960069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:24.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.050+0000 7f49737fe700 1 --2- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4960067050 0x7f4960069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:24.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.064+0000 7f4978d9d700 1 -- 172.21.15.55:0/908005008 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f497404c3c0 con 0x7f49741566e0 2023-10-06T04:48:24.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.066+0000 7f495b7fe700 1 -- 172.21.15.55:0/908005008 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+18184 (secure 0 0 0) 0x7f496c055990 con 0x7f49741566e0 2023-10-06T04:48:24.068 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:24.068 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":146,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T04:48:23.704421+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:52:28.177340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T03:52:29.819781+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":146,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/85191":"2023-10-07T03:52:47.903801+0000","172.21.15.163:6835/85191":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/3682044855":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/851766186":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/27897253":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/141262261":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/1256847914":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/3480679704":"2023-10-07T03:52:47.903801+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":18,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":56,"length":2},{"begin":112,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:48:24.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.068+0000 7f4978d9d700 1 -- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4960067050 msgr2=0x7f4960069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:24.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.068+0000 7f4978d9d700 1 --2- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4960067050 0x7f4960069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.068+0000 7f4978d9d700 1 -- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49741566e0 msgr2=0x7f49740b03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:24.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.068+0000 7f4978d9d700 1 --2- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49741566e0 0x7f49740b03b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f496c009e50 tx=0x7f496c00b040 comp rx=0 tx=0).stop 2023-10-06T04:48:24.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.068+0000 7f4978d9d700 1 -- 172.21.15.55:0/908005008 shutdown_connections 2023-10-06T04:48:24.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.068+0000 7f4978d9d700 1 --2- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f4960067050 0x7f4960069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.068+0000 7f4978d9d700 1 --2- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49741566e0 0x7f49740b03b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.068+0000 7f4978d9d700 1 --2- 172.21.15.55:0/908005008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49740ca300 0x7f49741464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.068+0000 7f4978d9d700 1 --2- 172.21.15.55:0/908005008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4974156310 0x7f49740afe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.068+0000 7f4978d9d700 1 -- 172.21.15.55:0/908005008 >> 172.21.15.55:0/908005008 conn(0x7f49740cc1f0 msgr2=0x7f49740adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:24.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.068+0000 7f4978d9d700 1 -- 172.21.15.55:0/908005008 shutdown_connections 2023-10-06T04:48:24.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.068+0000 7f4978d9d700 1 -- 172.21.15.55:0/908005008 wait complete. 2023-10-06T04:48:24.089 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T03:52:28.177340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T03:52:29.819781+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '146', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 113, 'snap_epoch': 146, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:48:24.089 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:48:24.115 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:24.113+0000 7f6f17ce5180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T04:48:24.185 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:24.184+0000 7fae0d838180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T04:48:24.280 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:24.278+0000 7f6f17ce5180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T04:48:24.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.285+0000 7fcaee19d700 1 Processor -- start 2023-10-06T04:48:24.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.285+0000 7fcaee19d700 1 -- start start 2023-10-06T04:48:24.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.286+0000 7fcaee19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcae80b16d0 0x7fcae80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:24.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.286+0000 7fcaee19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae80b2080 con 0x7fcae80b4e70 2023-10-06T04:48:24.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.286+0000 7fcaee19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae80cfb00 con 0x7fcae80b33b0 2023-10-06T04:48:24.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.286+0000 7fcaee19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae80b5c40 con 0x7fcae80b16d0 2023-10-06T04:48:24.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.287+0000 7fcaed19b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcae80b4e70 0x7fcae80ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37998/0 (socket says 172.21.15.55:37998) 2023-10-06T04:48:24.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.287+0000 7fcaed19b700 1 -- 172.21.15.55:0/635711708 learned_addr learned my addr 172.21.15.55:0/635711708 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:24.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.287+0000 7fcaec99a700 1 --2- 172.21.15.55:0/635711708 >> v2:172.21.15.163:3300/0 conn(0x7fcae80b16d0 0x7fcae80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:24.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.287+0000 7fcadffff700 1 -- 172.21.15.55:0/635711708 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775006642 0 0) 0x7fcae80b2080 con 0x7fcae80b4e70 2023-10-06T04:48:24.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.287+0000 7fcadffff700 1 -- 172.21.15.55:0/635711708 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcad4003570 con 0x7fcae80b4e70 2023-10-06T04:48:24.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.287+0000 7fcaec99a700 1 -- 172.21.15.55:0/635711708 >> v1:172.21.15.163:6789/0 conn(0x7fcae80b33b0 legacy=0x7fcae80b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:24.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.287+0000 7fcadffff700 1 -- 172.21.15.55:0/635711708 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033484221 0 0) 0x7fcae80cfb00 con 0x7fcae80b33b0 2023-10-06T04:48:24.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.287+0000 7fcaec99a700 1 -- 172.21.15.55:0/635711708 >> v1:172.21.15.55:6789/0 conn(0x7fcae80b4e70 legacy=0x7fcae80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:24.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.287+0000 7fcaec99a700 1 -- 172.21.15.55:0/635711708 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae80b5f20 con 0x7fcae80b16d0 2023-10-06T04:48:24.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.288+0000 7fcaec99a700 1 --2- 172.21.15.55:0/635711708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae80b16d0 0x7fcae80b1b40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fcad80098e0 tx=0x7fcad80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbecb305f20ba648 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:24.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.288+0000 7fcadffff700 1 -- 172.21.15.55:0/635711708 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad800b640 con 0x7fcae80b16d0 2023-10-06T04:48:24.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.288+0000 7fcadffff700 1 -- 172.21.15.55:0/635711708 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad800b7a0 con 0x7fcae80b16d0 2023-10-06T04:48:24.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.288+0000 7fcadffff700 1 -- 172.21.15.55:0/635711708 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad800bb40 con 0x7fcae80b16d0 2023-10-06T04:48:24.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.289+0000 7fcaee19d700 1 -- 172.21.15.55:0/635711708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae80b16d0 msgr2=0x7fcae80b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:24.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.289+0000 7fcaee19d700 1 --2- 172.21.15.55:0/635711708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae80b16d0 0x7fcae80b1b40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fcad80098e0 tx=0x7fcad80091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.289+0000 7fcaee19d700 1 -- 172.21.15.55:0/635711708 shutdown_connections 2023-10-06T04:48:24.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.289+0000 7fcaee19d700 1 --2- 172.21.15.55:0/635711708 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae80b16d0 0x7fcae80b1b40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.289+0000 7fcaee19d700 1 -- 172.21.15.55:0/635711708 >> 172.21.15.55:0/635711708 conn(0x7fcae80d10c0 msgr2=0x7fcae80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:24.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.289+0000 7fcaee19d700 1 -- 172.21.15.55:0/635711708 shutdown_connections 2023-10-06T04:48:24.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.289+0000 7fcaee19d700 1 -- 172.21.15.55:0/635711708 wait complete. 2023-10-06T04:48:24.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.290+0000 7fcaee19d700 1 Processor -- start 2023-10-06T04:48:24.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.290+0000 7fcaee19d700 1 -- start start 2023-10-06T04:48:24.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.291+0000 7fcaee19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcae80b16d0 0x7fcae81459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:24.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.291+0000 7fcaee19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcae80b33b0 0x7fcae815e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:24.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.291+0000 7fcaee19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae80b4e70 0x7fcae815e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:24.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.291+0000 7fcaee19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80c7370 con 0x7fcae80b33b0 2023-10-06T04:48:24.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.291+0000 7fcaee19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80c8130 con 0x7fcae80b4e70 2023-10-06T04:48:24.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.291+0000 7fcaee19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80c7090 con 0x7fcae80b16d0 2023-10-06T04:48:24.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.291+0000 7fcaec99a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcae80b33b0 0x7fcae815e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:24.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.291+0000 7fcaec99a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcae80b33b0 0x7fcae815e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58364/0 (socket says 172.21.15.55:58364) 2023-10-06T04:48:24.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.292+0000 7fcaec99a700 1 -- 172.21.15.55:0/4142419425 learned_addr learned my addr 172.21.15.55:0/4142419425 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:24.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.292+0000 7fcaed19b700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcae80b16d0 0x7fcae81459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:24.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.292+0000 7fcaed99c700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae80b4e70 0x7fcae815e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:24.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.292+0000 7fcaec99a700 1 -- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcae80b16d0 msgr2=0x7fcae81459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:24.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.292+0000 7fcaec99a700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcae80b16d0 0x7fcae81459c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.292+0000 7fcaec99a700 1 -- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae80b4e70 msgr2=0x7fcae815e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:24.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.292+0000 7fcaec99a700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae80b4e70 0x7fcae815e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.292+0000 7fcaec99a700 1 -- 172.21.15.55:0/4142419425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad8009420 con 0x7fcae80b33b0 2023-10-06T04:48:24.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.292+0000 7fcaed19b700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcae80b16d0 0x7fcae81459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:48:24.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.292+0000 7fcaec99a700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcae80b33b0 0x7fcae815e420 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fcae8070dd0 tx=0x7fcad8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:24.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.292+0000 7fcadd7fa700 1 -- 172.21.15.55:0/4142419425 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad801f070 con 0x7fcae80b33b0 2023-10-06T04:48:24.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.293+0000 7fcaee19d700 1 -- 172.21.15.55:0/4142419425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae80c7ae0 con 0x7fcae80b33b0 2023-10-06T04:48:24.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.293+0000 7fcadd7fa700 1 -- 172.21.15.55:0/4142419425 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad8002f70 con 0x7fcae80b33b0 2023-10-06T04:48:24.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.293+0000 7fcadd7fa700 1 -- 172.21.15.55:0/4142419425 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad800b910 con 0x7fcae80b33b0 2023-10-06T04:48:24.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.293+0000 7fcaee19d700 1 -- 172.21.15.55:0/4142419425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae8145f00 con 0x7fcae80b33b0 2023-10-06T04:48:24.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.294+0000 7fcaee19d700 1 -- 172.21.15.55:0/4142419425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae80c82c0 con 0x7fcae80b33b0 2023-10-06T04:48:24.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.294+0000 7fcadd7fa700 1 -- 172.21.15.55:0/4142419425 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 65) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fcad8023070 con 0x7fcae80b33b0 2023-10-06T04:48:24.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.295+0000 7fcadd7fa700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcacc067050 0x7fcacc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:24.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.295+0000 7fcadd7fa700 1 -- 172.21.15.55:0/4142419425 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6882+0+0 (secure 0 0 0) 0x7fcad8085f60 con 0x7fcae80b33b0 2023-10-06T04:48:24.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.296+0000 7fcaed19b700 1 -- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcacc067050 msgr2=0x7fcacc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:24.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.296+0000 7fcaed19b700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcacc067050 0x7fcacc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:24.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.300+0000 7fcadd7fa700 1 -- 172.21.15.55:0/4142419425 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcad8052b80 con 0x7fcae80b33b0 2023-10-06T04:48:24.369 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:24.366+0000 7f6f17ce5180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T04:48:24.382 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:24.380+0000 7fae0d838180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T04:48:24.474 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:24.472+0000 7fae0d838180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T04:48:24.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.497+0000 7fcaed19b700 1 -- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcacc067050 msgr2=0x7fcacc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:24.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.497+0000 7fcaed19b700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcacc067050 0x7fcacc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:24.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.531+0000 7fcaee19d700 1 -- 172.21.15.55:0/4142419425 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcae815f3a0 con 0x7fcae80b33b0 2023-10-06T04:48:24.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.531+0000 7fcadd7fa700 1 -- 172.21.15.55:0/4142419425 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v146) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcad80561a0 con 0x7fcae80b33b0 2023-10-06T04:48:24.533 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T04:48:24.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.534+0000 7fcaee19d700 1 -- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcacc067050 msgr2=0x7fcacc069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:24.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.534+0000 7fcaee19d700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcacc067050 0x7fcacc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.534+0000 7fcaee19d700 1 -- 172.21.15.55:0/4142419425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcae80b33b0 msgr2=0x7fcae815e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:24.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.534+0000 7fcaee19d700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcae80b33b0 0x7fcae815e420 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fcae8070dd0 tx=0x7fcad8017000 comp rx=0 tx=0).stop 2023-10-06T04:48:24.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.534+0000 7fcaee19d700 1 -- 172.21.15.55:0/4142419425 shutdown_connections 2023-10-06T04:48:24.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.534+0000 7fcaee19d700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fcacc067050 0x7fcacc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.534+0000 7fcaee19d700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcae80b4e70 0x7fcae815e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.534+0000 7fcaee19d700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcae80b16d0 0x7fcae81459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.534+0000 7fcaee19d700 1 --2- 172.21.15.55:0/4142419425 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcae80b33b0 0x7fcae815e420 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.534+0000 7fcaee19d700 1 -- 172.21.15.55:0/4142419425 >> 172.21.15.55:0/4142419425 conn(0x7fcae80d10c0 msgr2=0x7fcae815c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:24.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.534+0000 7fcaee19d700 1 -- 172.21.15.55:0/4142419425 shutdown_connections 2023-10-06T04:48:24.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.534+0000 7fcaee19d700 1 -- 172.21.15.55:0/4142419425 wait complete. 2023-10-06T04:48:24.555 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:48:24.596 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:24.593+0000 7f6f17ce5180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:48:24.713 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:24.711+0000 7fae0d838180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:48:24.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.749+0000 7ffa3299d700 1 Processor -- start 2023-10-06T04:48:24.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.749+0000 7ffa3299d700 1 -- start start 2023-10-06T04:48:24.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.751+0000 7ffa3299d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffa2c0ccc00 0x7ffa2c069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:24.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.751+0000 7ffa3299d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffa2c06a020 0x7ffa2c0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:24.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.751+0000 7ffa3299d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa2c0c9f30 con 0x7ffa2c159a60 2023-10-06T04:48:24.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.751+0000 7ffa3299d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa2c06dfc0 con 0x7ffa2c06a020 2023-10-06T04:48:24.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.751+0000 7ffa3299d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa2c0ca470 con 0x7ffa2c0ccc00 2023-10-06T04:48:24.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.751+0000 7ffa3219c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffa2c06a020 0x7ffa2c0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:24.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.751+0000 7ffa3199b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ffa2c159a60 0x7ffa2c0c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38000/0 (socket says 172.21.15.55:38000) 2023-10-06T04:48:24.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.751+0000 7ffa3199b700 1 -- 172.21.15.55:0/1655962692 learned_addr learned my addr 172.21.15.55:0/1655962692 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:24.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.751+0000 7ffa3119a700 1 --2- 172.21.15.55:0/1655962692 >> v2:172.21.15.163:3300/0 conn(0x7ffa2c0ccc00 0x7ffa2c069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:24.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.751+0000 7ffa3219c700 1 -- 172.21.15.55:0/1655962692 >> v2:172.21.15.163:3300/0 conn(0x7ffa2c0ccc00 msgr2=0x7ffa2c069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:24.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.751+0000 7ffa3219c700 1 --2- 172.21.15.55:0/1655962692 >> v2:172.21.15.163:3300/0 conn(0x7ffa2c0ccc00 0x7ffa2c069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.751+0000 7ffa30999700 1 -- 172.21.15.55:0/1655962692 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000641505 0 0) 0x7ffa2c0c9f30 con 0x7ffa2c159a60 2023-10-06T04:48:24.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.751+0000 7ffa3219c700 1 -- 172.21.15.55:0/1655962692 >> v1:172.21.15.55:6789/0 conn(0x7ffa2c159a60 legacy=0x7ffa2c0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:24.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.751+0000 7ffa3219c700 1 -- 172.21.15.55:0/1655962692 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2c0ca750 con 0x7ffa2c06a020 2023-10-06T04:48:24.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.752+0000 7ffa3219c700 1 --2- 172.21.15.55:0/1655962692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa2c06a020 0x7ffa2c0cb230 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0cf6a0 tx=0x7ffa24009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5be294038176a4b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:24.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.752+0000 7ffa30999700 1 -- 172.21.15.55:0/1655962692 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2400e070 con 0x7ffa2c06a020 2023-10-06T04:48:24.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.752+0000 7ffa30999700 1 -- 172.21.15.55:0/1655962692 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2400bbf0 con 0x7ffa2c06a020 2023-10-06T04:48:24.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.752+0000 7ffa30999700 1 -- 172.21.15.55:0/1655962692 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24012430 con 0x7ffa2c06a020 2023-10-06T04:48:24.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.753+0000 7ffa3299d700 1 -- 172.21.15.55:0/1655962692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa2c06a020 msgr2=0x7ffa2c0cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:24.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.753+0000 7ffa3299d700 1 --2- 172.21.15.55:0/1655962692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa2c06a020 0x7ffa2c0cb230 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0cf6a0 tx=0x7ffa24009780 comp rx=0 tx=0).stop 2023-10-06T04:48:24.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.753+0000 7ffa3299d700 1 -- 172.21.15.55:0/1655962692 shutdown_connections 2023-10-06T04:48:24.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.753+0000 7ffa3299d700 1 --2- 172.21.15.55:0/1655962692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa2c06a020 0x7ffa2c0cb230 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.753+0000 7ffa3299d700 1 --2- 172.21.15.55:0/1655962692 >> v2:172.21.15.163:3300/0 conn(0x7ffa2c0ccc00 0x7ffa2c069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.753+0000 7ffa3299d700 1 -- 172.21.15.55:0/1655962692 >> 172.21.15.55:0/1655962692 conn(0x7ffa2c06baa0 msgr2=0x7ffa2c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:24.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.753+0000 7ffa3299d700 1 -- 172.21.15.55:0/1655962692 shutdown_connections 2023-10-06T04:48:24.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.753+0000 7ffa3299d700 1 -- 172.21.15.55:0/1655962692 wait complete. 2023-10-06T04:48:24.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.754+0000 7ffa3299d700 1 Processor -- start 2023-10-06T04:48:24.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.754+0000 7ffa3299d700 1 -- start start 2023-10-06T04:48:24.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.755+0000 7ffa3299d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa2c06a020 0x7ffa2c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:24.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.755+0000 7ffa3299d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa2c0ccc00 0x7ffa2c157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:24.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.755+0000 7ffa3299d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa2c159a60 0x7ffa2c157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:24.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.755+0000 7ffa3299d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c0c8a10 con 0x7ffa2c06a020 2023-10-06T04:48:24.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.755+0000 7ffa3299d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c0b1e90 con 0x7ffa2c0ccc00 2023-10-06T04:48:24.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.755+0000 7ffa3299d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c0b1560 con 0x7ffa2c159a60 2023-10-06T04:48:24.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.755+0000 7ffa3199b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa2c06a020 0x7ffa2c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:24.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.756+0000 7ffa3199b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa2c06a020 0x7ffa2c0b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58392/0 (socket says 172.21.15.55:58392) 2023-10-06T04:48:24.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.756+0000 7ffa3199b700 1 -- 172.21.15.55:0/4124036042 learned_addr learned my addr 172.21.15.55:0/4124036042 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:24.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.756+0000 7ffa3119a700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa2c0ccc00 0x7ffa2c157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:24.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.756+0000 7ffa3219c700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa2c159a60 0x7ffa2c157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:24.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.756+0000 7ffa3199b700 1 -- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa2c159a60 msgr2=0x7ffa2c157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:24.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.756+0000 7ffa3199b700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa2c159a60 0x7ffa2c157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.756+0000 7ffa3199b700 1 -- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa2c0ccc00 msgr2=0x7ffa2c157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:24.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.756+0000 7ffa3199b700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa2c0ccc00 0x7ffa2c157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.756+0000 7ffa3199b700 1 -- 172.21.15.55:0/4124036042 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa24009a20 con 0x7ffa2c06a020 2023-10-06T04:48:24.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.756+0000 7ffa3119a700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa2c0ccc00 0x7ffa2c157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:24.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.756+0000 7ffa3199b700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa2c06a020 0x7ffa2c0b3cc0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800c5b0 tx=0x7ffa28001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:24.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.757+0000 7ffa21ffb700 1 -- 172.21.15.55:0/4124036042 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28005b30 con 0x7ffa2c06a020 2023-10-06T04:48:24.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.757+0000 7ffa21ffb700 1 -- 172.21.15.55:0/4124036042 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa28005c90 con 0x7ffa2c06a020 2023-10-06T04:48:24.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.757+0000 7ffa21ffb700 1 -- 172.21.15.55:0/4124036042 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa28007300 con 0x7ffa2c06a020 2023-10-06T04:48:24.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.757+0000 7ffa3299d700 1 -- 172.21.15.55:0/4124036042 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2c0b1840 con 0x7ffa2c06a020 2023-10-06T04:48:24.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.757+0000 7ffa3299d700 1 -- 172.21.15.55:0/4124036042 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2c0b4200 con 0x7ffa2c06a020 2023-10-06T04:48:24.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.758+0000 7ffa3299d700 1 -- 172.21.15.55:0/4124036042 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa2c06eba0 con 0x7ffa2c06a020 2023-10-06T04:48:24.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.760+0000 7ffa21ffb700 1 -- 172.21.15.55:0/4124036042 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 65) v1 ==== 84854+0+0 (secure 0 0 0) 0x7ffa2801b3b0 con 0x7ffa2c06a020 2023-10-06T04:48:24.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.760+0000 7ffa21ffb700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffa10067050 0x7ffa10069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:24.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.761+0000 7ffa21ffb700 1 -- 172.21.15.55:0/4124036042 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6882+0+0 (secure 0 0 0) 0x7ffa28083f80 con 0x7ffa2c06a020 2023-10-06T04:48:24.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.761+0000 7ffa3119a700 1 -- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffa10067050 msgr2=0x7ffa10069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:24.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.761+0000 7ffa3119a700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffa10067050 0x7ffa10069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:24.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.765+0000 7ffa21ffb700 1 -- 172.21.15.55:0/4124036042 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa28050c40 con 0x7ffa2c06a020 2023-10-06T04:48:24.775 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:24.772+0000 7f6f17ce5180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T04:48:24.868 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:24.865+0000 7f6f17ce5180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T04:48:24.904 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:24.902+0000 7fae0d838180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T04:48:24.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.962+0000 7ffa3119a700 1 -- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffa10067050 msgr2=0x7ffa10069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:24.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.962+0000 7ffa3119a700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffa10067050 0x7ffa10069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.967+0000 7ffa3299d700 1 -- 172.21.15.55:0/4124036042 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa2c0b1ce0 con 0x7ffa2c06a020 2023-10-06T04:48:24.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.968+0000 7ffa21ffb700 1 -- 172.21.15.55:0/4124036042 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v146) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa280170a0 con 0x7ffa2c06a020 2023-10-06T04:48:24.970 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T04:48:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.970+0000 7ffa3299d700 1 -- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffa10067050 msgr2=0x7ffa10069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:24.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.970+0000 7ffa3299d700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffa10067050 0x7ffa10069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.970+0000 7ffa3299d700 1 -- 172.21.15.55:0/4124036042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa2c06a020 msgr2=0x7ffa2c0b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.970+0000 7ffa3299d700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa2c06a020 0x7ffa2c0b3cc0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ffa2800c5b0 tx=0x7ffa28001070 comp rx=0 tx=0).stop 2023-10-06T04:48:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.970+0000 7ffa3299d700 1 -- 172.21.15.55:0/4124036042 shutdown_connections 2023-10-06T04:48:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.970+0000 7ffa3299d700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7ffa10067050 0x7ffa10069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.970+0000 7ffa3299d700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffa2c159a60 0x7ffa2c157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.970+0000 7ffa3299d700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffa2c06a020 0x7ffa2c0b3cc0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.970+0000 7ffa3299d700 1 --2- 172.21.15.55:0/4124036042 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffa2c0ccc00 0x7ffa2c157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.970+0000 7ffa3299d700 1 -- 172.21.15.55:0/4124036042 >> 172.21.15.55:0/4124036042 conn(0x7ffa2c06baa0 msgr2=0x7ffa2c1541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.970+0000 7ffa3299d700 1 -- 172.21.15.55:0/4124036042 shutdown_connections 2023-10-06T04:48:24.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:24.971+0000 7ffa3299d700 1 -- 172.21.15.55:0/4124036042 wait complete. 2023-10-06T04:48:24.991 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:48:25.003 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:25.001+0000 7fae0d838180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T04:48:25.030 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:25.027+0000 7f6f17ce5180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T04:48:25.129 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:25.127+0000 7f6f17ce5180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T04:48:25.171 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:25.170+0000 7fae0d838180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T04:48:25.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.182+0000 7fd13af9d700 1 Processor -- start 2023-10-06T04:48:25.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.183+0000 7fd13af9d700 1 -- start start 2023-10-06T04:48:25.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.184+0000 7fd13af9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1340b16d0 0x7fd1340b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:25.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.184+0000 7fd13af9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd1340b33b0 0x7fd1340b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:25.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.184+0000 7fd13af9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1340b2080 con 0x7fd1340b4e70 2023-10-06T04:48:25.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.184+0000 7fd13af9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1340cfb00 con 0x7fd1340b33b0 2023-10-06T04:48:25.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.184+0000 7fd13af9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340b5c40 con 0x7fd1340b16d0 2023-10-06T04:48:25.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.184+0000 7fd13a79c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd1340b33b0 0x7fd1340b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:25.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.184+0000 7fd139f9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fd1340b4e70 0x7fd1340ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38008/0 (socket says 172.21.15.55:38008) 2023-10-06T04:48:25.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.184+0000 7fd139f9b700 1 -- 172.21.15.55:0/2031851880 learned_addr learned my addr 172.21.15.55:0/2031851880 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:25.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.184+0000 7fd13979a700 1 --2- 172.21.15.55:0/2031851880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1340b16d0 0x7fd1340b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:25.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.185+0000 7fd13a79c700 1 -- 172.21.15.55:0/2031851880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1340b16d0 msgr2=0x7fd1340b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:25.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.185+0000 7fd13a79c700 1 --2- 172.21.15.55:0/2031851880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1340b16d0 0x7fd1340b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.185+0000 7fd13a79c700 1 -- 172.21.15.55:0/2031851880 >> v1:172.21.15.55:6789/0 conn(0x7fd1340b4e70 legacy=0x7fd1340ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:25.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.185+0000 7fd13a79c700 1 -- 172.21.15.55:0/2031851880 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1340b5f20 con 0x7fd1340b33b0 2023-10-06T04:48:25.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.185+0000 7fd13a79c700 1 --2- 172.21.15.55:0/2031851880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1340b33b0 0x7fd1340b3830 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009e60 tx=0x7fd12c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2cd13eb4092ec5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:25.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.185+0000 7fd138f99700 1 -- 172.21.15.55:0/2031851880 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c00e070 con 0x7fd1340b33b0 2023-10-06T04:48:25.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.185+0000 7fd138f99700 1 -- 172.21.15.55:0/2031851880 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12c00bb60 con 0x7fd1340b33b0 2023-10-06T04:48:25.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.185+0000 7fd138f99700 1 -- 172.21.15.55:0/2031851880 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c012430 con 0x7fd1340b33b0 2023-10-06T04:48:25.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.186+0000 7fd13af9d700 1 -- 172.21.15.55:0/2031851880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1340b33b0 msgr2=0x7fd1340b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:25.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.186+0000 7fd13af9d700 1 --2- 172.21.15.55:0/2031851880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1340b33b0 0x7fd1340b3830 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009e60 tx=0x7fd12c009780 comp rx=0 tx=0).stop 2023-10-06T04:48:25.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.186+0000 7fd13af9d700 1 -- 172.21.15.55:0/2031851880 shutdown_connections 2023-10-06T04:48:25.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.186+0000 7fd13af9d700 1 --2- 172.21.15.55:0/2031851880 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1340b33b0 0x7fd1340b3830 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.186+0000 7fd13af9d700 1 --2- 172.21.15.55:0/2031851880 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1340b16d0 0x7fd1340b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.186+0000 7fd13af9d700 1 -- 172.21.15.55:0/2031851880 >> 172.21.15.55:0/2031851880 conn(0x7fd1340d10c0 msgr2=0x7fd1340affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:25.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.186+0000 7fd13af9d700 1 -- 172.21.15.55:0/2031851880 shutdown_connections 2023-10-06T04:48:25.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.186+0000 7fd13af9d700 1 -- 172.21.15.55:0/2031851880 wait complete. 2023-10-06T04:48:25.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.187+0000 7fd13af9d700 1 Processor -- start 2023-10-06T04:48:25.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.188+0000 7fd13af9d700 1 -- start start 2023-10-06T04:48:25.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13af9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1340b16d0 0x7fd13415e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13af9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1340b33b0 0x7fd13416d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13af9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1340b4e70 0x7fd13416d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13af9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340c7e00 con 0x7fd1340b4e70 2023-10-06T04:48:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13af9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340c80e0 con 0x7fd1340b33b0 2023-10-06T04:48:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13af9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340c83c0 con 0x7fd1340b16d0 2023-10-06T04:48:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13a79c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1340b4e70 0x7fd13416d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13a79c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1340b4e70 0x7fd13416d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58410/0 (socket says 172.21.15.55:58410) 2023-10-06T04:48:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13a79c700 1 -- 172.21.15.55:0/3472512036 learned_addr learned my addr 172.21.15.55:0/3472512036 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd139f9b700 1 --2- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1340b16d0 0x7fd13415e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13979a700 1 --2- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1340b33b0 0x7fd13416d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:25.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13a79c700 1 -- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1340b16d0 msgr2=0x7fd13415e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:25.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13a79c700 1 --2- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1340b16d0 0x7fd13415e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13a79c700 1 -- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1340b33b0 msgr2=0x7fd13416d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:25.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13a79c700 1 --2- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1340b33b0 0x7fd13416d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.189+0000 7fd13a79c700 1 -- 172.21.15.55:0/3472512036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c009a20 con 0x7fd1340b4e70 2023-10-06T04:48:25.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.190+0000 7fd13a79c700 1 --2- 172.21.15.55:0/3472512036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1340b4e70 0x7fd13416d850 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00b6c0 tx=0x7fd12c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:25.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.190+0000 7fd129ffb700 1 -- 172.21.15.55:0/3472512036 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c025440 con 0x7fd1340b4e70 2023-10-06T04:48:25.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.190+0000 7fd129ffb700 1 -- 172.21.15.55:0/3472512036 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12c0255a0 con 0x7fd1340b4e70 2023-10-06T04:48:25.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.190+0000 7fd13af9d700 1 -- 172.21.15.55:0/3472512036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1340c86a0 con 0x7fd1340b4e70 2023-10-06T04:48:25.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.190+0000 7fd129ffb700 1 -- 172.21.15.55:0/3472512036 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c025940 con 0x7fd1340b4e70 2023-10-06T04:48:25.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.190+0000 7fd13af9d700 1 -- 172.21.15.55:0/3472512036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13415eb80 con 0x7fd1340b4e70 2023-10-06T04:48:25.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.191+0000 7fd13af9d700 1 -- 172.21.15.55:0/3472512036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd134088990 con 0x7fd1340b4e70 2023-10-06T04:48:25.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.193+0000 7fd129ffb700 1 -- 172.21.15.55:0/3472512036 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 65) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fd12c003050 con 0x7fd1340b4e70 2023-10-06T04:48:25.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.194+0000 7fd129ffb700 1 --2- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd118067050 0x7fd118069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:25.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.194+0000 7fd129ffb700 1 -- 172.21.15.55:0/3472512036 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6882+0+0 (secure 0 0 0) 0x7fd12c086e10 con 0x7fd1340b4e70 2023-10-06T04:48:25.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.194+0000 7fd139f9b700 1 -- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd118067050 msgr2=0x7fd118069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:25.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.194+0000 7fd139f9b700 1 --2- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd118067050 0x7fd118069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:25.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.199+0000 7fd129ffb700 1 -- 172.21.15.55:0/3472512036 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd12c08a030 con 0x7fd1340b4e70 2023-10-06T04:48:25.281 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:25.279+0000 7fae0d838180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T04:48:25.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.395+0000 7fd139f9b700 1 -- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd118067050 msgr2=0x7fd118069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:25.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.395+0000 7fd139f9b700 1 --2- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd118067050 0x7fd118069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:25.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.400+0000 7fd13af9d700 1 -- 172.21.15.55:0/3472512036 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd13404bb80 con 0x7fd1340b4e70 2023-10-06T04:48:25.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.401+0000 7fd129ffb700 1 -- 172.21.15.55:0/3472512036 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v146) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd12c053920 con 0x7fd1340b4e70 2023-10-06T04:48:25.403 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T04:48:25.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.403+0000 7fd13af9d700 1 -- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd118067050 msgr2=0x7fd118069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:25.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.403+0000 7fd13af9d700 1 --2- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd118067050 0x7fd118069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.403+0000 7fd13af9d700 1 -- 172.21.15.55:0/3472512036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1340b4e70 msgr2=0x7fd13416d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.403+0000 7fd13af9d700 1 --2- 172.21.15.55:0/3472512036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1340b4e70 0x7fd13416d850 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00b6c0 tx=0x7fd12c019000 comp rx=0 tx=0).stop 2023-10-06T04:48:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.403+0000 7fd13af9d700 1 -- 172.21.15.55:0/3472512036 shutdown_connections 2023-10-06T04:48:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.403+0000 7fd13af9d700 1 --2- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fd118067050 0x7fd118069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.403+0000 7fd13af9d700 1 --2- 172.21.15.55:0/3472512036 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd1340b4e70 0x7fd13416d850 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.403+0000 7fd13af9d700 1 --2- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd1340b16d0 0x7fd13415e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.403+0000 7fd13af9d700 1 --2- 172.21.15.55:0/3472512036 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd1340b33b0 0x7fd13416d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.403+0000 7fd13af9d700 1 -- 172.21.15.55:0/3472512036 >> 172.21.15.55:0/3472512036 conn(0x7fd1340d10c0 msgr2=0x7fd134158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.404+0000 7fd13af9d700 1 -- 172.21.15.55:0/3472512036 shutdown_connections 2023-10-06T04:48:25.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.404+0000 7fd13af9d700 1 -- 172.21.15.55:0/3472512036 wait complete. 2023-10-06T04:48:25.423 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:48:25.613 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:25.611+0000 7f6f17ce5180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T04:48:25.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.657+0000 7f34cffff700 1 Processor -- start 2023-10-06T04:48:25.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.657+0000 7f34cffff700 1 -- start start 2023-10-06T04:48:25.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.658+0000 7f34cffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f34d00cc9d0 0x7f34d015d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:25.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.658+0000 7f34cffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34d00cd0f0 0x7f34d015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:25.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.658+0000 7f34cffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d0150b50 con 0x7f34d00cc600 2023-10-06T04:48:25.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.658+0000 7f34cffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d00cede0 con 0x7f34d00cc9d0 2023-10-06T04:48:25.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.658+0000 7f34cffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d00b2260 con 0x7f34d00cd0f0 2023-10-06T04:48:25.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.658+0000 7f34ce7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f34d00cc9d0 0x7f34d015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:25.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.659+0000 7f34ce7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f34d00cc9d0 0x7f34d015d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58426/0 (socket says 172.21.15.55:58426) 2023-10-06T04:48:25.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.659+0000 7f34ce7fc700 1 -- 172.21.15.55:0/3403100252 learned_addr learned my addr 172.21.15.55:0/3403100252 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:25.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.659+0000 7f34cf7fe700 1 --2- 172.21.15.55:0/3403100252 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34d00cd0f0 0x7f34d015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:25.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.659+0000 7f34ce7fc700 1 -- 172.21.15.55:0/3403100252 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34d00cd0f0 msgr2=0x7f34d015d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:25.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.659+0000 7f34ce7fc700 1 --2- 172.21.15.55:0/3403100252 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34d00cd0f0 0x7f34d015d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.659+0000 7f34ce7fc700 1 -- 172.21.15.55:0/3403100252 >> v1:172.21.15.163:6789/0 conn(0x7f34d00cc600 legacy=0x7f34d00b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:25.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.659+0000 7f34ce7fc700 1 -- 172.21.15.55:0/3403100252 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d00b2540 con 0x7f34d00cc9d0 2023-10-06T04:48:25.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.659+0000 7f34ce7fc700 1 --2- 172.21.15.55:0/3403100252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34d00cc9d0 0x7f34d015d280 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f34c40099b0 tx=0x7f34c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ab47497300c55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:25.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.659+0000 7f34cdffb700 1 -- 172.21.15.55:0/3403100252 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c400b860 con 0x7f34d00cc9d0 2023-10-06T04:48:25.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.659+0000 7f34cdffb700 1 -- 172.21.15.55:0/3403100252 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c400b9c0 con 0x7f34d00cc9d0 2023-10-06T04:48:25.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.660+0000 7f34cdffb700 1 -- 172.21.15.55:0/3403100252 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c400bd60 con 0x7f34d00cc9d0 2023-10-06T04:48:25.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.660+0000 7f34cffff700 1 -- 172.21.15.55:0/3403100252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34d00cc9d0 msgr2=0x7f34d015d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:25.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.660+0000 7f34cffff700 1 --2- 172.21.15.55:0/3403100252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34d00cc9d0 0x7f34d015d280 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f34c40099b0 tx=0x7f34c4009250 comp rx=0 tx=0).stop 2023-10-06T04:48:25.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.660+0000 7f34cffff700 1 -- 172.21.15.55:0/3403100252 shutdown_connections 2023-10-06T04:48:25.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.660+0000 7f34cffff700 1 --2- 172.21.15.55:0/3403100252 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34d00cd0f0 0x7f34d015d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.660+0000 7f34cffff700 1 --2- 172.21.15.55:0/3403100252 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34d00cc9d0 0x7f34d015d280 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.660+0000 7f34cffff700 1 -- 172.21.15.55:0/3403100252 >> 172.21.15.55:0/3403100252 conn(0x7f34d00d0630 msgr2=0x7f34d00d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:25.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.660+0000 7f34cffff700 1 -- 172.21.15.55:0/3403100252 shutdown_connections 2023-10-06T04:48:25.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.660+0000 7f34cffff700 1 -- 172.21.15.55:0/3403100252 wait complete. 2023-10-06T04:48:25.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.662+0000 7f34cffff700 1 Processor -- start 2023-10-06T04:48:25.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.662+0000 7f34cffff700 1 -- start start 2023-10-06T04:48:25.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.663+0000 7f34cffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34d00cc600 0x7f34d0140f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:25.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.663+0000 7f34cffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34d00cc9d0 0x7f34d0154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:25.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.663+0000 7f34cffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34d00cd0f0 0x7f34d01547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:25.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.663+0000 7f34cffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d014dd40 con 0x7f34d00cd0f0 2023-10-06T04:48:25.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.663+0000 7f34cffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d0157590 con 0x7f34d00cc9d0 2023-10-06T04:48:25.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.663+0000 7f34cffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d0156fd0 con 0x7f34d00cc600 2023-10-06T04:48:25.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.663+0000 7f34ce7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34d00cc9d0 0x7f34d0154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:25.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.663+0000 7f34ceffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34d00cc600 0x7f34d0140f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:25.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.663+0000 7f34cf7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34d00cd0f0 0x7f34d01547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:25.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.663+0000 7f34ce7fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34d00cc9d0 0x7f34d0154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:32836/0 (socket says 172.21.15.55:32836) 2023-10-06T04:48:25.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.663+0000 7f34ceffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34d00cc600 0x7f34d0140f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:51922/0 (socket says 172.21.15.55:51922) 2023-10-06T04:48:25.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.663+0000 7f34ce7fc700 1 -- 172.21.15.55:0/611546681 learned_addr learned my addr 172.21.15.55:0/611546681 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:25.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.664+0000 7f34ce7fc700 1 -- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34d00cc600 msgr2=0x7f34d0140f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:25.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.664+0000 7f34ce7fc700 1 --2- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34d00cc600 0x7f34d0140f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.664+0000 7f34ce7fc700 1 -- 172.21.15.55:0/611546681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34d00cd0f0 msgr2=0x7f34d01547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:25.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.664+0000 7f34ce7fc700 1 --2- 172.21.15.55:0/611546681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34d00cd0f0 0x7f34d01547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.664+0000 7f34ce7fc700 1 -- 172.21.15.55:0/611546681 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c40094f0 con 0x7f34d00cc9d0 2023-10-06T04:48:25.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.664+0000 7f34cf7fe700 1 --2- 172.21.15.55:0/611546681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34d00cd0f0 0x7f34d01547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:25.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.664+0000 7f34ceffd700 1 --2- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34d00cc600 0x7f34d0140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:25.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.664+0000 7f34ce7fc700 1 --2- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34d00cc9d0 0x7f34d0154290 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f34d00b3800 tx=0x7f34c4017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:25.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.664+0000 7f34b6ffd700 1 -- 172.21.15.55:0/611546681 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4011e40 con 0x7f34d00cc9d0 2023-10-06T04:48:25.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.664+0000 7f34b6ffd700 1 -- 172.21.15.55:0/611546681 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c4024440 con 0x7f34d00cc9d0 2023-10-06T04:48:25.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.664+0000 7f34b6ffd700 1 -- 172.21.15.55:0/611546681 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4024800 con 0x7f34d00cc9d0 2023-10-06T04:48:25.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.664+0000 7f34cffff700 1 -- 172.21.15.55:0/611546681 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d01572b0 con 0x7f34d00cc9d0 2023-10-06T04:48:25.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.664+0000 7f34cffff700 1 -- 172.21.15.55:0/611546681 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d0151060 con 0x7f34d00cc9d0 2023-10-06T04:48:25.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.665+0000 7f34cffff700 1 -- 172.21.15.55:0/611546681 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d00889d0 con 0x7f34d00cc9d0 2023-10-06T04:48:25.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.666+0000 7f34b6ffd700 1 -- 172.21.15.55:0/611546681 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 65) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f34c401a420 con 0x7f34d00cc9d0 2023-10-06T04:48:25.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.667+0000 7f34b6ffd700 1 --2- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f34bc067050 0x7f34bc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:25.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.667+0000 7f34b6ffd700 1 -- 172.21.15.55:0/611546681 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6882+0+0 (secure 0 0 0) 0x7f34c4086aa0 con 0x7f34d00cc9d0 2023-10-06T04:48:25.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.668+0000 7f34ceffd700 1 -- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f34bc067050 msgr2=0x7f34bc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:25.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.668+0000 7f34ceffd700 1 --2- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f34bc067050 0x7f34bc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:25.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.672+0000 7f34b6ffd700 1 -- 172.21.15.55:0/611546681 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f34c40536c0 con 0x7f34d00cc9d0 2023-10-06T04:48:25.709 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:25.707+0000 7f6f17ce5180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T04:48:25.793 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:25.791+0000 7fae0d838180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T04:48:25.802 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:25.799+0000 7f6f17ce5180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T04:48:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.869+0000 7f34ceffd700 1 -- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f34bc067050 msgr2=0x7f34bc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.869+0000 7f34ceffd700 1 --2- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f34bc067050 0x7f34bc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:25.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.877+0000 7f34cffff700 1 -- 172.21.15.55:0/611546681 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34d00b2820 con 0x7f34d00cc9d0 2023-10-06T04:48:25.880 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.878+0000 7f34b6ffd700 1 -- 172.21.15.55:0/611546681 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v146) v1 ==== 100+0+11 (secure 0 0 0) 0x7f34c4056ce0 con 0x7f34d00cc9d0 2023-10-06T04:48:25.881 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T04:48:25.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.881+0000 7f34cffff700 1 -- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f34bc067050 msgr2=0x7f34bc069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:25.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.881+0000 7f34cffff700 1 --2- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f34bc067050 0x7f34bc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.881+0000 7f34cffff700 1 -- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34d00cc9d0 msgr2=0x7f34d0154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:25.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.881+0000 7f34cffff700 1 --2- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34d00cc9d0 0x7f34d0154290 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f34d00b3800 tx=0x7f34c4017000 comp rx=0 tx=0).stop 2023-10-06T04:48:25.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.881+0000 7f34cffff700 1 -- 172.21.15.55:0/611546681 shutdown_connections 2023-10-06T04:48:25.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.881+0000 7f34cffff700 1 --2- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f34bc067050 0x7f34bc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.881+0000 7f34cffff700 1 --2- 172.21.15.55:0/611546681 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f34d00cd0f0 0x7f34d01547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.881+0000 7f34cffff700 1 --2- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f34d00cc600 0x7f34d0140f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.881+0000 7f34cffff700 1 --2- 172.21.15.55:0/611546681 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f34d00cc9d0 0x7f34d0154290 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:25.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.881+0000 7f34cffff700 1 -- 172.21.15.55:0/611546681 >> 172.21.15.55:0/611546681 conn(0x7f34d00d0630 msgr2=0x7f34d015b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:25.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.881+0000 7f34cffff700 1 -- 172.21.15.55:0/611546681 shutdown_connections 2023-10-06T04:48:25.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:25.881+0000 7f34cffff700 1 -- 172.21.15.55:0/611546681 wait complete. 2023-10-06T04:48:25.893 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:25.891+0000 7fae0d838180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T04:48:25.901 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:48:25.994 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:25.992+0000 7fae0d838180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T04:48:26.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.092+0000 7f045219d700 1 Processor -- start 2023-10-06T04:48:26.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.093+0000 7f045219d700 1 -- start start 2023-10-06T04:48:26.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.094+0000 7f045219d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f044c1597f0 0x7f044c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:26.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.094+0000 7f045219d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f044c15d230 0x7f044c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:26.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.094+0000 7f045219d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f044c15db70 con 0x7f044c159420 2023-10-06T04:48:26.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.094+0000 7f045219d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f044c0b11d0 con 0x7f044c15d230 2023-10-06T04:48:26.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.094+0000 7f045219d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f044c0b14b0 con 0x7f044c1597f0 2023-10-06T04:48:26.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.094+0000 7f045119b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f044c159420 0x7f044c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38010/0 (socket says 172.21.15.55:38010) 2023-10-06T04:48:26.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.094+0000 7f045119b700 1 -- 172.21.15.55:0/2952046867 learned_addr learned my addr 172.21.15.55:0/2952046867 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:26.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.094+0000 7f045199c700 1 --2- 172.21.15.55:0/2952046867 >> v2:172.21.15.55:3300/0 conn(0x7f044c15d230 0x7f044c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:26.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.094+0000 7f045099a700 1 --2- 172.21.15.55:0/2952046867 >> v2:172.21.15.163:3300/0 conn(0x7f044c1597f0 0x7f044c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:26.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.095+0000 7f0443fff700 1 -- 172.21.15.55:0/2952046867 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2820257768 0 0) 0x7f044c15db70 con 0x7f044c159420 2023-10-06T04:48:26.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.095+0000 7f0443fff700 1 -- 172.21.15.55:0/2952046867 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0438003570 con 0x7f044c159420 2023-10-06T04:48:26.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.095+0000 7f045199c700 1 -- 172.21.15.55:0/2952046867 >> v2:172.21.15.163:3300/0 conn(0x7f044c1597f0 msgr2=0x7f044c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:26.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.095+0000 7f045199c700 1 --2- 172.21.15.55:0/2952046867 >> v2:172.21.15.163:3300/0 conn(0x7f044c1597f0 0x7f044c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.095+0000 7f045199c700 1 -- 172.21.15.55:0/2952046867 >> v1:172.21.15.55:6789/0 conn(0x7f044c159420 legacy=0x7f044c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:26.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.095+0000 7f045199c700 1 -- 172.21.15.55:0/2952046867 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c0b1790 con 0x7f044c15d230 2023-10-06T04:48:26.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.095+0000 7f045199c700 1 --2- 172.21.15.55:0/2952046867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f044c15d230 0x7f044c15d630 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f044c089750 tx=0x7f0444009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8adcbe85198bbed server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:26.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.095+0000 7f0443fff700 1 -- 172.21.15.55:0/2952046867 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044400e070 con 0x7f044c15d230 2023-10-06T04:48:26.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.095+0000 7f0443fff700 1 -- 172.21.15.55:0/2952046867 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044400bab0 con 0x7f044c15d230 2023-10-06T04:48:26.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.095+0000 7f0443fff700 1 -- 172.21.15.55:0/2952046867 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044400be70 con 0x7f044c15d230 2023-10-06T04:48:26.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.096+0000 7f045219d700 1 -- 172.21.15.55:0/2952046867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f044c15d230 msgr2=0x7f044c15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:26.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.096+0000 7f045219d700 1 --2- 172.21.15.55:0/2952046867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f044c15d230 0x7f044c15d630 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f044c089750 tx=0x7f0444009780 comp rx=0 tx=0).stop 2023-10-06T04:48:26.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.096+0000 7f045219d700 1 -- 172.21.15.55:0/2952046867 shutdown_connections 2023-10-06T04:48:26.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.096+0000 7f045219d700 1 --2- 172.21.15.55:0/2952046867 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f044c15d230 0x7f044c15d630 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.096+0000 7f045219d700 1 --2- 172.21.15.55:0/2952046867 >> v2:172.21.15.163:3300/0 conn(0x7f044c1597f0 0x7f044c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.096+0000 7f045219d700 1 -- 172.21.15.55:0/2952046867 >> 172.21.15.55:0/2952046867 conn(0x7f044c0d0660 msgr2=0x7f044c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:26.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.096+0000 7f045219d700 1 -- 172.21.15.55:0/2952046867 shutdown_connections 2023-10-06T04:48:26.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.096+0000 7f045219d700 1 -- 172.21.15.55:0/2952046867 wait complete. 2023-10-06T04:48:26.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.097+0000 7f045219d700 1 Processor -- start 2023-10-06T04:48:26.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.098+0000 7f045219d700 1 -- start start 2023-10-06T04:48:26.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.098+0000 7f045219d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f044c159420 0x7f044c15b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:26.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.098+0000 7f045219d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f044c1597f0 0x7f044c16cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:26.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.098+0000 7f045219d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f044c15d230 0x7f044c156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:26.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.098+0000 7f045219d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c16db80 con 0x7f044c15d230 2023-10-06T04:48:26.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.098+0000 7f045219d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c0c6db0 con 0x7f044c159420 2023-10-06T04:48:26.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.098+0000 7f045219d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044c16d2e0 con 0x7f044c1597f0 2023-10-06T04:48:26.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.099+0000 7f045199c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f044c15d230 0x7f044c156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:26.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.099+0000 7f045199c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f044c15d230 0x7f044c156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58454/0 (socket says 172.21.15.55:58454) 2023-10-06T04:48:26.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.099+0000 7f045199c700 1 -- 172.21.15.55:0/2098746208 learned_addr learned my addr 172.21.15.55:0/2098746208 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:26.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.099+0000 7f045099a700 1 --2- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f044c1597f0 0x7f044c16cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:26.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.099+0000 7f045119b700 1 --2- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f044c159420 0x7f044c15b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:26.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.099+0000 7f045199c700 1 -- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f044c1597f0 msgr2=0x7f044c16cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:26.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.099+0000 7f045199c700 1 --2- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f044c1597f0 0x7f044c16cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.099+0000 7f045199c700 1 -- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f044c159420 msgr2=0x7f044c15b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:26.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.099+0000 7f045199c700 1 --2- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f044c159420 0x7f044c15b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.099+0000 7f045199c700 1 -- 172.21.15.55:0/2098746208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0444009a20 con 0x7f044c15d230 2023-10-06T04:48:26.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.099+0000 7f045199c700 1 --2- 172.21.15.55:0/2098746208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f044c15d230 0x7f044c156da0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f04440099f0 tx=0x7f0444019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:26.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.100+0000 7f04417fa700 1 -- 172.21.15.55:0/2098746208 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444012dd0 con 0x7f044c15d230 2023-10-06T04:48:26.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.100+0000 7f04417fa700 1 -- 172.21.15.55:0/2098746208 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0444027470 con 0x7f044c15d230 2023-10-06T04:48:26.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.100+0000 7f045219d700 1 -- 172.21.15.55:0/2098746208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044c16d5c0 con 0x7f044c15d230 2023-10-06T04:48:26.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.100+0000 7f04417fa700 1 -- 172.21.15.55:0/2098746208 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444027830 con 0x7f044c15d230 2023-10-06T04:48:26.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.100+0000 7f045219d700 1 -- 172.21.15.55:0/2098746208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044c0c9260 con 0x7f044c15d230 2023-10-06T04:48:26.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.101+0000 7f045219d700 1 -- 172.21.15.55:0/2098746208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f044c0c6b80 con 0x7f044c15d230 2023-10-06T04:48:26.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.102+0000 7f04417fa700 1 -- 172.21.15.55:0/2098746208 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 65) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f0444024070 con 0x7f044c15d230 2023-10-06T04:48:26.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.103+0000 7f04417fa700 1 --2- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0430067050 0x7f0430069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:26.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.103+0000 7f04417fa700 1 -- 172.21.15.55:0/2098746208 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6882+0+0 (secure 0 0 0) 0x7f0444087160 con 0x7f044c15d230 2023-10-06T04:48:26.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.103+0000 7f045119b700 1 -- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0430067050 msgr2=0x7f0430069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:26.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.103+0000 7f045119b700 1 --2- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0430067050 0x7f0430069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:26.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.107+0000 7f04417fa700 1 -- 172.21.15.55:0/2098746208 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f044c0c6b80 con 0x7f044c15d230 2023-10-06T04:48:26.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.304+0000 7f045119b700 1 -- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0430067050 msgr2=0x7f0430069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:26.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.304+0000 7f045119b700 1 --2- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0430067050 0x7f0430069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:26.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.356+0000 7f045219d700 1 -- 172.21.15.55:0/2098746208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f044c0c6b80 con 0x7f044c15d230 2023-10-06T04:48:26.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.357+0000 7f04417fa700 1 -- 172.21.15.55:0/2098746208 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f044c0c6b80 con 0x7f044c15d230 2023-10-06T04:48:26.359 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:26.359 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2624033145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2624033145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2624033145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3851023466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3851023466},{"type":"v1","addr":"172.21.15.55:6834","nonce":3851023466}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/2633241022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2633241022},{"type":"v1","addr":"172.21.15.163:6833","nonce":2633241022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:52:30.648704+0000","modified":"2023-10-06T03:52:34.152554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7879":{"gid":7879,"name":"d","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6836/246684871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":246684871},{"type":"v1","addr":"172.21.15.163:6836","nonce":246684871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T04:48:26.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.359+0000 7f045219d700 1 -- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0430067050 msgr2=0x7f0430069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:26.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.359+0000 7f045219d700 1 --2- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0430067050 0x7f0430069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.359+0000 7f045219d700 1 -- 172.21.15.55:0/2098746208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f044c15d230 msgr2=0x7f044c156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:26.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.359+0000 7f045219d700 1 --2- 172.21.15.55:0/2098746208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f044c15d230 0x7f044c156da0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f04440099f0 tx=0x7f0444019000 comp rx=0 tx=0).stop 2023-10-06T04:48:26.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.360+0000 7f045219d700 1 -- 172.21.15.55:0/2098746208 shutdown_connections 2023-10-06T04:48:26.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.360+0000 7f045219d700 1 --2- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f0430067050 0x7f0430069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.360+0000 7f045219d700 1 --2- 172.21.15.55:0/2098746208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f044c15d230 0x7f044c156da0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.360+0000 7f045219d700 1 --2- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f044c159420 0x7f044c15b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.360+0000 7f045219d700 1 --2- 172.21.15.55:0/2098746208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f044c1597f0 0x7f044c16cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.360+0000 7f045219d700 1 -- 172.21.15.55:0/2098746208 >> 172.21.15.55:0/2098746208 conn(0x7f044c0d0660 msgr2=0x7f044c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:26.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.360+0000 7f045219d700 1 -- 172.21.15.55:0/2098746208 shutdown_connections 2023-10-06T04:48:26.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.360+0000 7f045219d700 1 -- 172.21.15.55:0/2098746208 wait complete. 2023-10-06T04:48:26.365 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 70 2023-10-06T04:48:26.379 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:48:26.436 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:26.434+0000 7f6f17ce5180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T04:48:26.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.599+0000 7f553519d700 1 Processor -- start 2023-10-06T04:48:26.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.599+0000 7f553519d700 1 -- start start 2023-10-06T04:48:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.600+0000 7f553519d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f553015d230 0x7f553015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.600+0000 7f553519d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55300b2980 con 0x7f55301597f0 2023-10-06T04:48:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.600+0000 7f553519d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f553015db70 con 0x7f5530159420 2023-10-06T04:48:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.600+0000 7f553519d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55300b11d0 con 0x7f553015d230 2023-10-06T04:48:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.600+0000 7f552f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f55301597f0 0x7f55300b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38016/0 (socket says 172.21.15.55:38016) 2023-10-06T04:48:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.600+0000 7f552f7fe700 1 -- 172.21.15.55:0/2037399188 learned_addr learned my addr 172.21.15.55:0/2037399188 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.600+0000 7f553499c700 1 --2- 172.21.15.55:0/2037399188 >> v2:172.21.15.55:3300/0 conn(0x7f553015d230 0x7f553015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.601+0000 7f552effd700 1 -- 172.21.15.55:0/2037399188 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976085641 0 0) 0x7f553015db70 con 0x7f5530159420 2023-10-06T04:48:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.601+0000 7f552effd700 1 -- 172.21.15.55:0/2037399188 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5518003570 con 0x7f5530159420 2023-10-06T04:48:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.601+0000 7f552effd700 1 -- 172.21.15.55:0/2037399188 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037902641 0 0) 0x7f55300b2980 con 0x7f55301597f0 2023-10-06T04:48:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.601+0000 7f552effd700 1 -- 172.21.15.55:0/2037399188 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f553015db70 con 0x7f55301597f0 2023-10-06T04:48:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.601+0000 7f553499c700 1 -- 172.21.15.55:0/2037399188 >> v1:172.21.15.163:6789/0 conn(0x7f5530159420 legacy=0x7f55300b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.601+0000 7f553499c700 1 -- 172.21.15.55:0/2037399188 >> v1:172.21.15.55:6789/0 conn(0x7f55301597f0 legacy=0x7f55300b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.601+0000 7f553499c700 1 -- 172.21.15.55:0/2037399188 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55300b14b0 con 0x7f553015d230 2023-10-06T04:48:26.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.602+0000 7f553499c700 1 --2- 172.21.15.55:0/2037399188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553015d230 0x7f553015d630 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f5528009e60 tx=0x7f5528009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3794864d148246 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:26.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.602+0000 7f552effd700 1 -- 172.21.15.55:0/2037399188 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552800e070 con 0x7f553015d230 2023-10-06T04:48:26.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.602+0000 7f552effd700 1 -- 172.21.15.55:0/2037399188 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552800bab0 con 0x7f553015d230 2023-10-06T04:48:26.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.602+0000 7f552effd700 1 -- 172.21.15.55:0/2037399188 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552800be70 con 0x7f553015d230 2023-10-06T04:48:26.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.602+0000 7f553519d700 1 -- 172.21.15.55:0/2037399188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553015d230 msgr2=0x7f553015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:26.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.602+0000 7f553519d700 1 --2- 172.21.15.55:0/2037399188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553015d230 0x7f553015d630 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f5528009e60 tx=0x7f5528009780 comp rx=0 tx=0).stop 2023-10-06T04:48:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.603+0000 7f553519d700 1 -- 172.21.15.55:0/2037399188 shutdown_connections 2023-10-06T04:48:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.603+0000 7f553519d700 1 --2- 172.21.15.55:0/2037399188 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f553015d230 0x7f553015d630 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.603+0000 7f553519d700 1 -- 172.21.15.55:0/2037399188 >> 172.21.15.55:0/2037399188 conn(0x7f55300d0660 msgr2=0x7f55300d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.603+0000 7f553519d700 1 -- 172.21.15.55:0/2037399188 shutdown_connections 2023-10-06T04:48:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.603+0000 7f553519d700 1 -- 172.21.15.55:0/2037399188 wait complete. 2023-10-06T04:48:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.604+0000 7f553519d700 1 Processor -- start 2023-10-06T04:48:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.604+0000 7f553519d700 1 -- start start 2023-10-06T04:48:26.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.605+0000 7f553519d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5530159420 0x7f5530158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:26.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.605+0000 7f553519d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55301597f0 0x7f55300c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:26.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.605+0000 7f553519d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f553015d230 0x7f55300c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:26.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.605+0000 7f553519d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55300c5e80 con 0x7f55301597f0 2023-10-06T04:48:26.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.605+0000 7f553519d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55300b5e30 con 0x7f553015d230 2023-10-06T04:48:26.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.605+0000 7f553519d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553015aa90 con 0x7f5530159420 2023-10-06T04:48:26.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.605+0000 7f552f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55301597f0 0x7f55300c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:26.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.605+0000 7f552f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55301597f0 0x7f55300c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58468/0 (socket says 172.21.15.55:58468) 2023-10-06T04:48:26.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.605+0000 7f552f7fe700 1 -- 172.21.15.55:0/762372040 learned_addr learned my addr 172.21.15.55:0/762372040 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:26.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.605+0000 7f552ffff700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5530159420 0x7f5530158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:26.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.606+0000 7f553499c700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f553015d230 0x7f55300c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:26.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.606+0000 7f552f7fe700 1 -- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5530159420 msgr2=0x7f5530158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:26.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.606+0000 7f552f7fe700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5530159420 0x7f5530158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.606+0000 7f552f7fe700 1 -- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f553015d230 msgr2=0x7f55300c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:26.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.606+0000 7f552f7fe700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f553015d230 0x7f55300c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.606+0000 7f552f7fe700 1 -- 172.21.15.55:0/762372040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5528009a20 con 0x7f55301597f0 2023-10-06T04:48:26.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.606+0000 7f553499c700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f553015d230 0x7f55300c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:48:26.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.606+0000 7f552f7fe700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55301597f0 0x7f55300c5400 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f552400c7c0 tx=0x7f5524007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:26.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.606+0000 7f5517fff700 1 -- 172.21.15.55:0/762372040 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524005c00 con 0x7f55301597f0 2023-10-06T04:48:26.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.606+0000 7f5517fff700 1 -- 172.21.15.55:0/762372040 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5524005d60 con 0x7f55301597f0 2023-10-06T04:48:26.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.606+0000 7f5517fff700 1 -- 172.21.15.55:0/762372040 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5524007300 con 0x7f55301597f0 2023-10-06T04:48:26.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.606+0000 7f553519d700 1 -- 172.21.15.55:0/762372040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553015ad70 con 0x7f55301597f0 2023-10-06T04:48:26.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.607+0000 7f553519d700 1 -- 172.21.15.55:0/762372040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553015b330 con 0x7f55301597f0 2023-10-06T04:48:26.611 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:26.606+0000 7f6f17ce5180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:48:26.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.608+0000 7f553519d700 1 -- 172.21.15.55:0/762372040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f553015b550 con 0x7f55301597f0 2023-10-06T04:48:26.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.609+0000 7f5517fff700 1 -- 172.21.15.55:0/762372040 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 65) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f552401b3b0 con 0x7f55301597f0 2023-10-06T04:48:26.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.610+0000 7f5517fff700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f551c067050 0x7f551c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:26.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.610+0000 7f5517fff700 1 -- 172.21.15.55:0/762372040 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6882+0+0 (secure 0 0 0) 0x7f5524083eb0 con 0x7f55301597f0 2023-10-06T04:48:26.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.610+0000 7f552ffff700 1 -- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f551c067050 msgr2=0x7f551c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:26.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.610+0000 7f552ffff700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f551c067050 0x7f551c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:26.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.614+0000 7f5517fff700 1 -- 172.21.15.55:0/762372040 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5524050ae0 con 0x7f55301597f0 2023-10-06T04:48:26.660 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:26.658+0000 7fae0d838180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T04:48:26.709 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:26.706+0000 7f6f17ce5180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T04:48:26.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.811+0000 7f552ffff700 1 -- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f551c067050 msgr2=0x7f551c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:26.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.811+0000 7f552ffff700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f551c067050 0x7f551c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:26.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.823+0000 7f553519d700 1 -- 172.21.15.55:0/762372040 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f553004c3c0 con 0x7f55301597f0 2023-10-06T04:48:26.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.824+0000 7f5517fff700 1 -- 172.21.15.55:0/762372040 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f5524017020 con 0x7f55301597f0 2023-10-06T04:48:26.826 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:26.827 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":147,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T04:48:26.659180+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:52:28.177340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T03:52:29.819781+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":146,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/85191":"2023-10-07T03:52:47.903801+0000","172.21.15.163:6835/85191":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/3682044855":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/851766186":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/27897253":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/141262261":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/1256847914":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/3480679704":"2023-10-07T03:52:47.903801+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":18,"snaps":[{"begin":14,"length":44},{"begin":70,"length":44}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":18,"snaps":[{"begin":3,"length":11},{"begin":59,"length":11}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:48:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.827+0000 7f553519d700 1 -- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f551c067050 msgr2=0x7f551c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.827+0000 7f553519d700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f551c067050 0x7f551c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.827+0000 7f553519d700 1 -- 172.21.15.55:0/762372040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55301597f0 msgr2=0x7f55300c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.827+0000 7f553519d700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55301597f0 0x7f55300c5400 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f552400c7c0 tx=0x7f5524007dd0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.827+0000 7f553519d700 1 -- 172.21.15.55:0/762372040 shutdown_connections 2023-10-06T04:48:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.827+0000 7f553519d700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f551c067050 0x7f551c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.827+0000 7f553519d700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f553015d230 0x7f55300c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.827+0000 7f553519d700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5530159420 0x7f5530158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.827+0000 7f553519d700 1 --2- 172.21.15.55:0/762372040 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55301597f0 0x7f55300c5400 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:26.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.827+0000 7f553519d700 1 -- 172.21.15.55:0/762372040 >> 172.21.15.55:0/762372040 conn(0x7f55300d0660 msgr2=0x7f5530155650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:26.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.827+0000 7f553519d700 1 -- 172.21.15.55:0/762372040 shutdown_connections 2023-10-06T04:48:26.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:26.827+0000 7f553519d700 1 -- 172.21.15.55:0/762372040 wait complete. 2023-10-06T04:48:26.848 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T04:48:26.848 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:48:26.848 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:26.846+0000 7fae0d838180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:48:26.952 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:26.950+0000 7fae0d838180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T04:48:27.006 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:27.003+0000 7f6f17ce5180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T04:48:27.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.037+0000 7f2b3119d700 1 Processor -- start 2023-10-06T04:48:27.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.037+0000 7f2b3119d700 1 -- start start 2023-10-06T04:48:27.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.039+0000 7f2b3119d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2b2c06d950 0x7f2b2c0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:27.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.039+0000 7f2b3119d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2b2c06dd20 0x7f2b2c0c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:27.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.039+0000 7f2b3119d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2c06ae70 con 0x7f2b2c06a0a0 2023-10-06T04:48:27.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.039+0000 7f2b3119d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c06a540 con 0x7f2b2c06d950 2023-10-06T04:48:27.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.039+0000 7f2b3119d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c06a820 con 0x7f2b2c06dd20 2023-10-06T04:48:27.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.039+0000 7f2b2bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2b2c06d950 0x7f2b2c0c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:27.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.039+0000 7f2b2bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2b2c06d950 0x7f2b2c0c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58476/0 (socket says 172.21.15.55:58476) 2023-10-06T04:48:27.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.039+0000 7f2b2bfff700 1 -- 172.21.15.55:0/1726892931 learned_addr learned my addr 172.21.15.55:0/1726892931 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:27.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.039+0000 7f2b3099c700 1 --2- 172.21.15.55:0/1726892931 >> v2:172.21.15.163:3300/0 conn(0x7f2b2c06dd20 0x7f2b2c0c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:27.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.040+0000 7f2b2bfff700 1 -- 172.21.15.55:0/1726892931 >> v2:172.21.15.163:3300/0 conn(0x7f2b2c06dd20 msgr2=0x7f2b2c0c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:27.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.040+0000 7f2b2bfff700 1 --2- 172.21.15.55:0/1726892931 >> v2:172.21.15.163:3300/0 conn(0x7f2b2c06dd20 0x7f2b2c0c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.040+0000 7f2b2bfff700 1 -- 172.21.15.55:0/1726892931 >> v1:172.21.15.55:6789/0 conn(0x7f2b2c06a0a0 legacy=0x7f2b2c0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:27.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.040+0000 7f2b2bfff700 1 -- 172.21.15.55:0/1726892931 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c06ab00 con 0x7f2b2c06d950 2023-10-06T04:48:27.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.040+0000 7f2b2affd700 1 -- 172.21.15.55:0/1726892931 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167695707 0 0) 0x7f2b2c06ae70 con 0x7f2b2c06a0a0 2023-10-06T04:48:27.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.040+0000 7f2b2bfff700 1 --2- 172.21.15.55:0/1726892931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c06d950 0x7f2b2c0c8350 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c06e420 tx=0x7f2b1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a319ffd777000a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:27.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.040+0000 7f2b2affd700 1 -- 172.21.15.55:0/1726892931 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c009e70 con 0x7f2b2c06d950 2023-10-06T04:48:27.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.040+0000 7f2b2affd700 1 -- 172.21.15.55:0/1726892931 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c00b860 con 0x7f2b2c06d950 2023-10-06T04:48:27.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.040+0000 7f2b2affd700 1 -- 172.21.15.55:0/1726892931 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00bc00 con 0x7f2b2c06d950 2023-10-06T04:48:27.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.041+0000 7f2b3119d700 1 -- 172.21.15.55:0/1726892931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c06d950 msgr2=0x7f2b2c0c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:27.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.041+0000 7f2b3119d700 1 --2- 172.21.15.55:0/1726892931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c06d950 0x7f2b2c0c8350 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c06e420 tx=0x7f2b1c009250 comp rx=0 tx=0).stop 2023-10-06T04:48:27.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.041+0000 7f2b3119d700 1 -- 172.21.15.55:0/1726892931 shutdown_connections 2023-10-06T04:48:27.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.041+0000 7f2b3119d700 1 --2- 172.21.15.55:0/1726892931 >> v2:172.21.15.163:3300/0 conn(0x7f2b2c06dd20 0x7f2b2c0c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.041+0000 7f2b3119d700 1 --2- 172.21.15.55:0/1726892931 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c06d950 0x7f2b2c0c8350 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.041+0000 7f2b3119d700 1 -- 172.21.15.55:0/1726892931 >> 172.21.15.55:0/1726892931 conn(0x7f2b2c0cc1f0 msgr2=0x7f2b2c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:27.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.041+0000 7f2b3119d700 1 -- 172.21.15.55:0/1726892931 shutdown_connections 2023-10-06T04:48:27.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.041+0000 7f2b3119d700 1 -- 172.21.15.55:0/1726892931 wait complete. 2023-10-06T04:48:27.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.043+0000 7f2b3119d700 1 Processor -- start 2023-10-06T04:48:27.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.044+0000 7f2b3119d700 1 -- start start 2023-10-06T04:48:27.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.045+0000 7f2b3119d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b2c06a0a0 0x7f2b2c147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:27.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.045+0000 7f2b3119d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c06d950 0x7f2b2c154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:27.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.045+0000 7f2b3119d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c06dd20 0x7f2b2c155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:27.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.045+0000 7f2b3119d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c1584c0 con 0x7f2b2c06d950 2023-10-06T04:48:27.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.045+0000 7f2b3119d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0c4290 con 0x7f2b2c06a0a0 2023-10-06T04:48:27.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.045+0000 7f2b3119d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0c3fb0 con 0x7f2b2c06dd20 2023-10-06T04:48:27.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.045+0000 7f2b2b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c06d950 0x7f2b2c154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:27.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.045+0000 7f2b2b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c06d950 0x7f2b2c154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58482/0 (socket says 172.21.15.55:58482) 2023-10-06T04:48:27.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.045+0000 7f2b2b7fe700 1 -- 172.21.15.55:0/3702546102 learned_addr learned my addr 172.21.15.55:0/3702546102 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:27.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.045+0000 7f2b2bfff700 1 --2- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b2c06a0a0 0x7f2b2c147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:27.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.045+0000 7f2b3099c700 1 --2- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c06dd20 0x7f2b2c155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:27.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.046+0000 7f2b2b7fe700 1 -- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c06dd20 msgr2=0x7f2b2c155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:27.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.046+0000 7f2b2b7fe700 1 --2- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c06dd20 0x7f2b2c155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.046+0000 7f2b2b7fe700 1 -- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b2c06a0a0 msgr2=0x7f2b2c147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:27.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.046+0000 7f2b2b7fe700 1 --2- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b2c06a0a0 0x7f2b2c147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.046+0000 7f2b2b7fe700 1 -- 172.21.15.55:0/3702546102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c0094f0 con 0x7f2b2c06d950 2023-10-06T04:48:27.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.046+0000 7f2b2b7fe700 1 --2- 172.21.15.55:0/3702546102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c06d950 0x7f2b2c154b50 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000c7e0 tx=0x7f2b20007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:27.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.046+0000 7f2b13fff700 1 -- 172.21.15.55:0/3702546102 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b200105c0 con 0x7f2b2c06d950 2023-10-06T04:48:27.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.046+0000 7f2b13fff700 1 -- 172.21.15.55:0/3702546102 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b20010740 con 0x7f2b2c06d950 2023-10-06T04:48:27.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.046+0000 7f2b13fff700 1 -- 172.21.15.55:0/3702546102 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20010b00 con 0x7f2b2c06d950 2023-10-06T04:48:27.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.046+0000 7f2b3119d700 1 -- 172.21.15.55:0/3702546102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2c157e70 con 0x7f2b2c06d950 2023-10-06T04:48:27.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.047+0000 7f2b3119d700 1 -- 172.21.15.55:0/3702546102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c1480e0 con 0x7f2b2c06d950 2023-10-06T04:48:27.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.048+0000 7f2b3119d700 1 -- 172.21.15.55:0/3702546102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2c158630 con 0x7f2b2c06d950 2023-10-06T04:48:27.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.049+0000 7f2b13fff700 1 -- 172.21.15.55:0/3702546102 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 65) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f2b2001b3b0 con 0x7f2b2c06d950 2023-10-06T04:48:27.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.050+0000 7f2b13fff700 1 --2- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b18067050 0x7f2b18069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:27.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.050+0000 7f2b13fff700 1 -- 172.21.15.55:0/3702546102 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6882+0+0 (secure 0 0 0) 0x7f2b200841e0 con 0x7f2b2c06d950 2023-10-06T04:48:27.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.050+0000 7f2b2bfff700 1 -- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b18067050 msgr2=0x7f2b18069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:27.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.050+0000 7f2b2bfff700 1 --2- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b18067050 0x7f2b18069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:27.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.054+0000 7f2b13fff700 1 -- 172.21.15.55:0/3702546102 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b20050e00 con 0x7f2b2c06d950 2023-10-06T04:48:27.104 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:27.102+0000 7f6f17ce5180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T04:48:27.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.251+0000 7f2b2bfff700 1 -- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b18067050 msgr2=0x7f2b18069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:27.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.251+0000 7f2b2bfff700 1 --2- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b18067050 0x7f2b18069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:27.262 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:27.260+0000 7fae0d838180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T04:48:27.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.300+0000 7f2b3119d700 1 -- 172.21.15.55:0/3702546102 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b2c04c3c0 con 0x7f2b2c06d950 2023-10-06T04:48:27.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.301+0000 7f2b13fff700 1 -- 172.21.15.55:0/3702546102 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f2b20017090 con 0x7f2b2c06d950 2023-10-06T04:48:27.303 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:27.303 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2624033145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2624033145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2624033145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3851023466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3851023466},{"type":"v1","addr":"172.21.15.55:6834","nonce":3851023466}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/2633241022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2633241022},{"type":"v1","addr":"172.21.15.163:6833","nonce":2633241022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:52:30.648704+0000","modified":"2023-10-06T03:52:34.152554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7879":{"gid":7879,"name":"d","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6836/246684871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":246684871},{"type":"v1","addr":"172.21.15.163:6836","nonce":246684871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T04:48:27.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.303+0000 7f2b3119d700 1 -- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b18067050 msgr2=0x7f2b18069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:27.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.303+0000 7f2b3119d700 1 --2- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b18067050 0x7f2b18069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.303+0000 7f2b3119d700 1 -- 172.21.15.55:0/3702546102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c06d950 msgr2=0x7f2b2c154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:27.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.303+0000 7f2b3119d700 1 --2- 172.21.15.55:0/3702546102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c06d950 0x7f2b2c154b50 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000c7e0 tx=0x7f2b20007df0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.303+0000 7f2b3119d700 1 -- 172.21.15.55:0/3702546102 shutdown_connections 2023-10-06T04:48:27.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.303+0000 7f2b3119d700 1 --2- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b18067050 0x7f2b18069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.303+0000 7f2b3119d700 1 --2- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2b2c06dd20 0x7f2b2c155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.303+0000 7f2b3119d700 1 --2- 172.21.15.55:0/3702546102 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2b2c06a0a0 0x7f2b2c147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.303+0000 7f2b3119d700 1 --2- 172.21.15.55:0/3702546102 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2b2c06d950 0x7f2b2c154b50 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.303+0000 7f2b3119d700 1 -- 172.21.15.55:0/3702546102 >> 172.21.15.55:0/3702546102 conn(0x7f2b2c0cc1f0 msgr2=0x7f2b2c0c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:27.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.303+0000 7f2b3119d700 1 -- 172.21.15.55:0/3702546102 shutdown_connections 2023-10-06T04:48:27.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.303+0000 7f2b3119d700 1 -- 172.21.15.55:0/3702546102 wait complete. 2023-10-06T04:48:27.307 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 70 2023-10-06T04:48:27.323 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:48:27.366 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:27.365+0000 7fae0d838180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T04:48:27.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.513+0000 7fe484d9d700 1 Processor -- start 2023-10-06T04:48:27.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.513+0000 7fe484d9d700 1 -- start start 2023-10-06T04:48:27.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.514+0000 7fe484d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe480142ea0 0x7fe4800b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:27.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.515+0000 7fe484d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe48006c0f0 0x7fe48006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:27.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.515+0000 7fe484d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4800b40f0 con 0x7fe4800ad870 2023-10-06T04:48:27.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.515+0000 7fe484d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe480069640 con 0x7fe480142ea0 2023-10-06T04:48:27.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.515+0000 7fe484d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe480069920 con 0x7fe48006c0f0 2023-10-06T04:48:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.515+0000 7fe47f7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe480142ea0 0x7fe4800b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.515+0000 7fe47f7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe480142ea0 0x7fe4800b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58498/0 (socket says 172.21.15.55:58498) 2023-10-06T04:48:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.515+0000 7fe47f7fe700 1 -- 172.21.15.55:0/2128408978 learned_addr learned my addr 172.21.15.55:0/2128408978 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.515+0000 7fe47ffff700 1 --2- 172.21.15.55:0/2128408978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe48006c0f0 0x7fe48006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.515+0000 7fe47f7fe700 1 -- 172.21.15.55:0/2128408978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe48006c0f0 msgr2=0x7fe48006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.515+0000 7fe47f7fe700 1 --2- 172.21.15.55:0/2128408978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe48006c0f0 0x7fe48006a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.515+0000 7fe47f7fe700 1 -- 172.21.15.55:0/2128408978 >> v1:172.21.15.163:6789/0 conn(0x7fe4800ad870 legacy=0x7fe48006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.515+0000 7fe47f7fe700 1 -- 172.21.15.55:0/2128408978 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe480069c00 con 0x7fe480142ea0 2023-10-06T04:48:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.516+0000 7fe47f7fe700 1 --2- 172.21.15.55:0/2128408978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe480142ea0 0x7fe4800b3630 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fe4700099b0 tx=0x7fe470009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640c41b5b12c8dd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.516+0000 7fe47e7fc700 1 -- 172.21.15.55:0/2128408978 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47000b860 con 0x7fe480142ea0 2023-10-06T04:48:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.516+0000 7fe47e7fc700 1 -- 172.21.15.55:0/2128408978 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47000b9c0 con 0x7fe480142ea0 2023-10-06T04:48:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.516+0000 7fe47e7fc700 1 -- 172.21.15.55:0/2128408978 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47000bd60 con 0x7fe480142ea0 2023-10-06T04:48:27.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.516+0000 7fe484d9d700 1 -- 172.21.15.55:0/2128408978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe480142ea0 msgr2=0x7fe4800b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:27.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.516+0000 7fe484d9d700 1 --2- 172.21.15.55:0/2128408978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe480142ea0 0x7fe4800b3630 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fe4700099b0 tx=0x7fe470009250 comp rx=0 tx=0).stop 2023-10-06T04:48:27.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.516+0000 7fe484d9d700 1 -- 172.21.15.55:0/2128408978 shutdown_connections 2023-10-06T04:48:27.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.516+0000 7fe484d9d700 1 --2- 172.21.15.55:0/2128408978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe48006c0f0 0x7fe48006a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.516+0000 7fe484d9d700 1 --2- 172.21.15.55:0/2128408978 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe480142ea0 0x7fe4800b3630 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.517+0000 7fe484d9d700 1 -- 172.21.15.55:0/2128408978 >> 172.21.15.55:0/2128408978 conn(0x7fe4800ce430 msgr2=0x7fe4800ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:27.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.517+0000 7fe484d9d700 1 -- 172.21.15.55:0/2128408978 shutdown_connections 2023-10-06T04:48:27.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.517+0000 7fe484d9d700 1 -- 172.21.15.55:0/2128408978 wait complete. 2023-10-06T04:48:27.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.518+0000 7fe484d9d700 1 Processor -- start 2023-10-06T04:48:27.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.518+0000 7fe484d9d700 1 -- start start 2023-10-06T04:48:27.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.519+0000 7fe484d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe48006c0f0 0x7fe48015da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:27.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.519+0000 7fe484d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe480142ea0 0x7fe48015cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:27.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.519+0000 7fe484d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe48013f450 0x7fe480141180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:27.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.519+0000 7fe484d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48015d2d0 con 0x7fe48006c0f0 2023-10-06T04:48:27.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.519+0000 7fe484d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48015e0f0 con 0x7fe48013f450 2023-10-06T04:48:27.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.519+0000 7fe484d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe48015e3d0 con 0x7fe480142ea0 2023-10-06T04:48:27.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.519+0000 7fe47f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe48006c0f0 0x7fe48015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:27.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.519+0000 7fe47f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe48006c0f0 0x7fe48015da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58502/0 (socket says 172.21.15.55:58502) 2023-10-06T04:48:27.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.519+0000 7fe47f7fe700 1 -- 172.21.15.55:0/2353030117 learned_addr learned my addr 172.21.15.55:0/2353030117 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:27.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.519+0000 7fe47ffff700 1 --2- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe48013f450 0x7fe480141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:27.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.520+0000 7fe47effd700 1 --2- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe480142ea0 0x7fe48015cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:27.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.520+0000 7fe47f7fe700 1 -- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe480142ea0 msgr2=0x7fe48015cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:27.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.520+0000 7fe47f7fe700 1 --2- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe480142ea0 0x7fe48015cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.520+0000 7fe47f7fe700 1 -- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe48013f450 msgr2=0x7fe480141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:27.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.520+0000 7fe47f7fe700 1 --2- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe48013f450 0x7fe480141180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.520+0000 7fe47f7fe700 1 -- 172.21.15.55:0/2353030117 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4700094f0 con 0x7fe48006c0f0 2023-10-06T04:48:27.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.520+0000 7fe47f7fe700 1 --2- 172.21.15.55:0/2353030117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe48006c0f0 0x7fe48015da10 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe4800cd940 tx=0x7fe470017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:27.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.520+0000 7fe4677fe700 1 -- 172.21.15.55:0/2353030117 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470024440 con 0x7fe48006c0f0 2023-10-06T04:48:27.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.520+0000 7fe4677fe700 1 -- 172.21.15.55:0/2353030117 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4700245c0 con 0x7fe48006c0f0 2023-10-06T04:48:27.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.520+0000 7fe484d9d700 1 -- 172.21.15.55:0/2353030117 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48015e6b0 con 0x7fe48006c0f0 2023-10-06T04:48:27.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.520+0000 7fe4677fe700 1 -- 172.21.15.55:0/2353030117 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470024980 con 0x7fe48006c0f0 2023-10-06T04:48:27.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.521+0000 7fe484d9d700 1 -- 172.21.15.55:0/2353030117 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48015ec70 con 0x7fe48006c0f0 2023-10-06T04:48:27.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.522+0000 7fe484d9d700 1 -- 172.21.15.55:0/2353030117 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4800cdde0 con 0x7fe48006c0f0 2023-10-06T04:48:27.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.523+0000 7fe4677fe700 1 -- 172.21.15.55:0/2353030117 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 65) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fe47001a420 con 0x7fe48006c0f0 2023-10-06T04:48:27.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.523+0000 7fe4677fe700 1 --2- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe46c067050 0x7fe46c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:27.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.524+0000 7fe4677fe700 1 -- 172.21.15.55:0/2353030117 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6882+0+0 (secure 0 0 0) 0x7fe4700867b0 con 0x7fe48006c0f0 2023-10-06T04:48:27.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.524+0000 7fe47effd700 1 -- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe46c067050 msgr2=0x7fe46c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:27.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.524+0000 7fe47effd700 1 --2- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe46c067050 0x7fe46c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:27.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.528+0000 7fe4677fe700 1 -- 172.21.15.55:0/2353030117 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe470053470 con 0x7fe48006c0f0 2023-10-06T04:48:27.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.725+0000 7fe47effd700 1 -- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe46c067050 msgr2=0x7fe46c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:27.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.725+0000 7fe47effd700 1 --2- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe46c067050 0x7fe46c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:27.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.778+0000 7fe484d9d700 1 -- 172.21.15.55:0/2353030117 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe48004c3c0 con 0x7fe48006c0f0 2023-10-06T04:48:27.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.778+0000 7fe4677fe700 1 -- 172.21.15.55:0/2353030117 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fe470056a90 con 0x7fe48006c0f0 2023-10-06T04:48:27.780 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:27.781 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2624033145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2624033145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2624033145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3851023466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3851023466},{"type":"v1","addr":"172.21.15.55:6834","nonce":3851023466}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/2633241022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2633241022},{"type":"v1","addr":"172.21.15.163:6833","nonce":2633241022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T03:52:30.648704+0000","modified":"2023-10-06T03:52:34.152554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7879},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7879":{"gid":7879,"name":"d","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6836/246684871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":246684871},{"type":"v1","addr":"172.21.15.163:6836","nonce":246684871}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-06T04:48:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.781+0000 7fe484d9d700 1 -- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe46c067050 msgr2=0x7fe46c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.781+0000 7fe484d9d700 1 --2- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe46c067050 0x7fe46c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.781+0000 7fe484d9d700 1 -- 172.21.15.55:0/2353030117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe48006c0f0 msgr2=0x7fe48015da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.781+0000 7fe484d9d700 1 --2- 172.21.15.55:0/2353030117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe48006c0f0 0x7fe48015da10 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe4800cd940 tx=0x7fe470017000 comp rx=0 tx=0).stop 2023-10-06T04:48:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.781+0000 7fe484d9d700 1 -- 172.21.15.55:0/2353030117 shutdown_connections 2023-10-06T04:48:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.781+0000 7fe484d9d700 1 --2- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fe46c067050 0x7fe46c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.781+0000 7fe484d9d700 1 --2- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe48013f450 0x7fe480141180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.781+0000 7fe484d9d700 1 --2- 172.21.15.55:0/2353030117 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe48006c0f0 0x7fe48015da10 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.781+0000 7fe484d9d700 1 --2- 172.21.15.55:0/2353030117 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe480142ea0 0x7fe48015cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.781+0000 7fe484d9d700 1 -- 172.21.15.55:0/2353030117 >> 172.21.15.55:0/2353030117 conn(0x7fe4800ce430 msgr2=0x7fe4800ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.781+0000 7fe484d9d700 1 -- 172.21.15.55:0/2353030117 shutdown_connections 2023-10-06T04:48:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:27.781+0000 7fe484d9d700 1 -- 172.21.15.55:0/2353030117 wait complete. 2023-10-06T04:48:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 70 2023-10-06T04:48:27.801 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:48:27.811 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:27.809+0000 7f6f17ce5180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T04:48:28.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.028+0000 7f6b07fff700 1 Processor -- start 2023-10-06T04:48:28.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.029+0000 7f6b07fff700 1 -- start start 2023-10-06T04:48:28.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.030+0000 7f6b07fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b0806ba80 0x7f6b0806bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.030+0000 7f6b07fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b080c8e20 con 0x7f6b08159b30 2023-10-06T04:48:28.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.030+0000 7f6b07fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0806c400 con 0x7f6b080697f0 2023-10-06T04:48:28.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.030+0000 7f6b07fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b080c8c90 con 0x7f6b0806ba80 2023-10-06T04:48:28.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.030+0000 7f6b06ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6b08159b30 0x7f6b08159f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38026/0 (socket says 172.21.15.55:38026) 2023-10-06T04:48:28.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.030+0000 7f6b06ffd700 1 -- 172.21.15.55:0/3047619583 learned_addr learned my addr 172.21.15.55:0/3047619583 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:28.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.030+0000 7f6b077fe700 1 --2- 172.21.15.55:0/3047619583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b0806ba80 0x7f6b0806bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068139342 0 0) 0x7f6b080c8e20 con 0x7f6b08159b30 2023-10-06T04:48:28.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6af0003570 con 0x7f6b08159b30 2023-10-06T04:48:28.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864384224 0 0) 0x7f6b0806c400 con 0x7f6b080697f0 2023-10-06T04:48:28.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b080c8e20 con 0x7f6b080697f0 2023-10-06T04:48:28.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1684274510 0 0) 0x7f6af0003570 con 0x7f6b08159b30 2023-10-06T04:48:28.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b0806ba80 msgr2=0x7f6b0806bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:28.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 --2- 172.21.15.55:0/3047619583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b0806ba80 0x7f6b0806bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 >> v1:172.21.15.163:6789/0 conn(0x7f6b080697f0 legacy=0x7f6b080ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:28.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0815a7c0 con 0x7f6b08159b30 2023-10-06T04:48:28.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f6af8001470 con 0x7f6b08159b30 2023-10-06T04:48:28.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 >> v1:172.21.15.55:6789/0 conn(0x7f6b08159b30 legacy=0x7f6b08159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:28.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 --2- 172.21.15.55:0/3047619583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b0806ba80 0x7f6af0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 --2- 172.21.15.55:0/3047619583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af00041d0 0x7f6af0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 --2- 172.21.15.55:0/3047619583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6af0004d50 0x7f6af00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b080c8c90 con 0x7f6af00041d0 2023-10-06T04:48:28.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af000bbf0 con 0x7f6af0004d50 2023-10-06T04:48:28.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.031+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af000bdb0 con 0x7f6b0806ba80 2023-10-06T04:48:28.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.032+0000 7f6b067fc700 1 --2- 172.21.15.55:0/3047619583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af00041d0 0x7f6af0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.032+0000 7f6b077fe700 1 --2- 172.21.15.55:0/3047619583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6af0004d50 0x7f6af00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.032+0000 7f6b06ffd700 1 --2- 172.21.15.55:0/3047619583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b0806ba80 0x7f6af0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.032+0000 7f6b067fc700 1 -- 172.21.15.55:0/3047619583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b0806ba80 msgr2=0x7f6af0003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:28.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.032+0000 7f6b067fc700 1 --2- 172.21.15.55:0/3047619583 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b0806ba80 0x7f6af0003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.032+0000 7f6b067fc700 1 -- 172.21.15.55:0/3047619583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6af0004d50 msgr2=0x7f6af00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:28.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.032+0000 7f6b067fc700 1 --2- 172.21.15.55:0/3047619583 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6af0004d50 0x7f6af00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.032+0000 7f6b067fc700 1 -- 172.21.15.55:0/3047619583 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af0003570 con 0x7f6af00041d0 2023-10-06T04:48:28.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.032+0000 7f6b067fc700 1 --2- 172.21.15.55:0/3047619583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af00041d0 0x7f6af0004810 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f6afc001a10 tx=0x7f6afc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdb5a708cb283e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:28.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.032+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc00c5d0 con 0x7f6af00041d0 2023-10-06T04:48:28.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.032+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc00c730 con 0x7f6af00041d0 2023-10-06T04:48:28.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.033+0000 7f6b05ffb700 1 -- 172.21.15.55:0/3047619583 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc00cad0 con 0x7f6af00041d0 2023-10-06T04:48:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.033+0000 7f6b07fff700 1 -- 172.21.15.55:0/3047619583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af00041d0 msgr2=0x7f6af0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.033+0000 7f6b07fff700 1 --2- 172.21.15.55:0/3047619583 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af00041d0 0x7f6af0004810 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f6afc001a10 tx=0x7f6afc007d30 comp rx=0 tx=0).stop 2023-10-06T04:48:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.033+0000 7f6b077fe700 1 -- 172.21.15.55:0/3047619583 reap_dead start 2023-10-06T04:48:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.033+0000 7f6b07fff700 1 -- 172.21.15.55:0/3047619583 shutdown_connections 2023-10-06T04:48:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.033+0000 7f6b07fff700 1 -- 172.21.15.55:0/3047619583 >> 172.21.15.55:0/3047619583 conn(0x7f6b080ce650 msgr2=0x7f6b080cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.033+0000 7f6b07fff700 1 -- 172.21.15.55:0/3047619583 shutdown_connections 2023-10-06T04:48:28.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.033+0000 7f6b07fff700 1 -- 172.21.15.55:0/3047619583 wait complete. 2023-10-06T04:48:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.035+0000 7f6b07fff700 1 Processor -- start 2023-10-06T04:48:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.035+0000 7f6b07fff700 1 -- start start 2023-10-06T04:48:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b07fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af00041d0 0x7f6b08156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b07fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b08155820 0x7f6b08153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b07fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b08157da0 0x7f6b081581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b07fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0816a5d0 con 0x7f6af00041d0 2023-10-06T04:48:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b07fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b08158730 con 0x7f6b08155820 2023-10-06T04:48:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b07fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b08169cc0 con 0x7f6b08157da0 2023-10-06T04:48:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b06ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af00041d0 0x7f6b08156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b06ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af00041d0 0x7f6b08156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58510/0 (socket says 172.21.15.55:58510) 2023-10-06T04:48:28.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b067fc700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b08155820 0x7f6b08153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b06ffd700 1 -- 172.21.15.55:0/2925158997 learned_addr learned my addr 172.21.15.55:0/2925158997 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b077fe700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b08157da0 0x7f6b081581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b06ffd700 1 -- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b08157da0 msgr2=0x7f6b081581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b06ffd700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b08157da0 0x7f6b081581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b06ffd700 1 -- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b08155820 msgr2=0x7f6b08153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b06ffd700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b08155820 0x7f6b08153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b06ffd700 1 -- 172.21.15.55:0/2925158997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b08155c90 con 0x7f6af00041d0 2023-10-06T04:48:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b067fc700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b08155820 0x7f6b08153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.036+0000 7f6b077fe700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b08157da0 0x7f6b081581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.037+0000 7f6b06ffd700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af00041d0 0x7f6b08156360 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6af800e840 tx=0x7f6af8001010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:28.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.037+0000 7f6aeeffd700 1 -- 172.21.15.55:0/2925158997 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8012590 con 0x7f6af00041d0 2023-10-06T04:48:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.037+0000 7f6aeeffd700 1 -- 172.21.15.55:0/2925158997 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af8012710 con 0x7f6af00041d0 2023-10-06T04:48:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.037+0000 7f6aeeffd700 1 -- 172.21.15.55:0/2925158997 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8012ad0 con 0x7f6af00041d0 2023-10-06T04:48:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.037+0000 7f6b07fff700 1 -- 172.21.15.55:0/2925158997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b081569e0 con 0x7f6af00041d0 2023-10-06T04:48:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.037+0000 7f6b07fff700 1 -- 172.21.15.55:0/2925158997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0816a260 con 0x7f6af00041d0 2023-10-06T04:48:28.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.038+0000 7f6b07fff700 1 -- 172.21.15.55:0/2925158997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0804c3c0 con 0x7f6af00041d0 2023-10-06T04:48:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.039+0000 7f6aeeffd700 1 -- 172.21.15.55:0/2925158997 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 65) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f6af801d460 con 0x7f6af00041d0 2023-10-06T04:48:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.040+0000 7f6aeeffd700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6af4067050 0x7f6af4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.040+0000 7f6aeeffd700 1 -- 172.21.15.55:0/2925158997 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6882+0+0 (secure 0 0 0) 0x7f6af8084990 con 0x7f6af00041d0 2023-10-06T04:48:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.040+0000 7f6b067fc700 1 -- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6af4067050 msgr2=0x7f6af4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:28.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.040+0000 7f6b067fc700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6af4067050 0x7f6af4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:28.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.044+0000 7f6aeeffd700 1 -- 172.21.15.55:0/2925158997 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6af80515b0 con 0x7f6af00041d0 2023-10-06T04:48:28.100 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:28.098+0000 7f6f17ce5180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T04:48:28.125 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:28.123+0000 7fae0d838180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T04:48:28.193 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:28.190+0000 7f6f17ce5180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T04:48:28.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.241+0000 7f6b067fc700 1 -- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6af4067050 msgr2=0x7f6af4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:28.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.241+0000 7f6b067fc700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6af4067050 0x7f6af4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:28.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.249+0000 7f6b07fff700 1 -- 172.21.15.55:0/2925158997 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b08065910 con 0x7f6af00041d0 2023-10-06T04:48:28.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.250+0000 7f6aeeffd700 1 -- 172.21.15.55:0/2925158997 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+18185 (secure 0 0 0) 0x7f6af8054bd0 con 0x7f6af00041d0 2023-10-06T04:48:28.252 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:28.252 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":147,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T04:48:26.659180+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-10-06T03:52:28.177340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-10-06T03:52:29.819781+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":146,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:6837/85191":"2023-10-07T03:52:47.903801+0000","172.21.15.163:6835/85191":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/3682044855":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/851766186":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/27897253":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/141262261":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/1256847914":"2023-10-07T03:52:47.903801+0000","172.21.15.163:0/3480679704":"2023-10-07T03:52:47.903801+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":18,"snaps":[{"begin":14,"length":44},{"begin":70,"length":44}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":18,"snaps":[{"begin":3,"length":11},{"begin":59,"length":11}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:48:28.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.253+0000 7f6b07fff700 1 -- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6af4067050 msgr2=0x7f6af4069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:28.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.253+0000 7f6b07fff700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6af4067050 0x7f6af4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.253+0000 7f6b07fff700 1 -- 172.21.15.55:0/2925158997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af00041d0 msgr2=0x7f6b08156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:28.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.253+0000 7f6b07fff700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af00041d0 0x7f6b08156360 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f6af800e840 tx=0x7f6af8001010 comp rx=0 tx=0).stop 2023-10-06T04:48:28.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.253+0000 7f6b07fff700 1 -- 172.21.15.55:0/2925158997 shutdown_connections 2023-10-06T04:48:28.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.253+0000 7f6b07fff700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7f6af4067050 0x7f6af4069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.253+0000 7f6b07fff700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6b08157da0 0x7f6b081581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.253+0000 7f6b07fff700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6af00041d0 0x7f6b08156360 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.253+0000 7f6b07fff700 1 --2- 172.21.15.55:0/2925158997 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6b08155820 0x7f6b08153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.253+0000 7f6b07fff700 1 -- 172.21.15.55:0/2925158997 >> 172.21.15.55:0/2925158997 conn(0x7f6b080ce650 msgr2=0x7f6b080c4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:28.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.253+0000 7f6b07fff700 1 -- 172.21.15.55:0/2925158997 shutdown_connections 2023-10-06T04:48:28.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.253+0000 7f6b07fff700 1 -- 172.21.15.55:0/2925158997 wait complete. 2023-10-06T04:48:28.274 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T04:48:28.347 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:28.345+0000 7f6f17ce5180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T04:48:28.423 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:28.421+0000 7fae0d838180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T04:48:28.453 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:28.451+0000 7f6f17ce5180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T04:48:28.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.467+0000 7fa14e59d700 1 Processor -- start 2023-10-06T04:48:28.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.467+0000 7fa14e59d700 1 -- start start 2023-10-06T04:48:28.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.469+0000 7fa14e59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148159420 0x7fa1480b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.469+0000 7fa14e59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa1481597f0 0x7fa1480b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.469+0000 7fa14e59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa14815d230 0x7fa14815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.469+0000 7fa14e59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14815dbd0 con 0x7fa14815d230 2023-10-06T04:48:28.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.469+0000 7fa14e59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1480b11d0 con 0x7fa1481597f0 2023-10-06T04:48:28.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.469+0000 7fa14e59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1480b14b0 con 0x7fa148159420 2023-10-06T04:48:28.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.469+0000 7fa14dd9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa14815d230 0x7fa14815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.469+0000 7fa14dd9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa14815d230 0x7fa14815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58522/0 (socket says 172.21.15.55:58522) 2023-10-06T04:48:28.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.469+0000 7fa14dd9c700 1 -- 172.21.15.55:0/4269438072 learned_addr learned my addr 172.21.15.55:0/4269438072 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:28.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.469+0000 7fa14cd9a700 1 --2- 172.21.15.55:0/4269438072 >> v2:172.21.15.163:3300/0 conn(0x7fa1481597f0 0x7fa1480b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.469+0000 7fa14d59b700 1 --2- 172.21.15.55:0/4269438072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148159420 0x7fa1480b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.470+0000 7fa14dd9c700 1 -- 172.21.15.55:0/4269438072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148159420 msgr2=0x7fa1480b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:28.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.470+0000 7fa14dd9c700 1 --2- 172.21.15.55:0/4269438072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148159420 0x7fa1480b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.470+0000 7fa14dd9c700 1 -- 172.21.15.55:0/4269438072 >> v2:172.21.15.163:3300/0 conn(0x7fa1481597f0 msgr2=0x7fa1480b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:28.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.470+0000 7fa14dd9c700 1 --2- 172.21.15.55:0/4269438072 >> v2:172.21.15.163:3300/0 conn(0x7fa1481597f0 0x7fa1480b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.470+0000 7fa14dd9c700 1 -- 172.21.15.55:0/4269438072 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1480b1790 con 0x7fa14815d230 2023-10-06T04:48:28.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.470+0000 7fa14dd9c700 1 --2- 172.21.15.55:0/4269438072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa14815d230 0x7fa14815d690 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa1480d1770 tx=0x7fa140009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e1d484d70a9951 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:28.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.470+0000 7fa13ffff700 1 -- 172.21.15.55:0/4269438072 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14000e070 con 0x7fa14815d230 2023-10-06T04:48:28.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.470+0000 7fa13ffff700 1 -- 172.21.15.55:0/4269438072 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14000b970 con 0x7fa14815d230 2023-10-06T04:48:28.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.470+0000 7fa13ffff700 1 -- 172.21.15.55:0/4269438072 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1400092e0 con 0x7fa14815d230 2023-10-06T04:48:28.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.471+0000 7fa14e59d700 1 -- 172.21.15.55:0/4269438072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa14815d230 msgr2=0x7fa14815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:28.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.471+0000 7fa14e59d700 1 --2- 172.21.15.55:0/4269438072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa14815d230 0x7fa14815d690 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa1480d1770 tx=0x7fa140009840 comp rx=0 tx=0).stop 2023-10-06T04:48:28.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.471+0000 7fa14e59d700 1 -- 172.21.15.55:0/4269438072 shutdown_connections 2023-10-06T04:48:28.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.471+0000 7fa14e59d700 1 --2- 172.21.15.55:0/4269438072 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa14815d230 0x7fa14815d690 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.471+0000 7fa14e59d700 1 --2- 172.21.15.55:0/4269438072 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148159420 0x7fa1480b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.471+0000 7fa14e59d700 1 --2- 172.21.15.55:0/4269438072 >> v2:172.21.15.163:3300/0 conn(0x7fa1481597f0 0x7fa1480b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.471+0000 7fa14e59d700 1 -- 172.21.15.55:0/4269438072 >> 172.21.15.55:0/4269438072 conn(0x7fa1480d0660 msgr2=0x7fa1480d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:28.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.471+0000 7fa14e59d700 1 -- 172.21.15.55:0/4269438072 shutdown_connections 2023-10-06T04:48:28.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.471+0000 7fa14e59d700 1 -- 172.21.15.55:0/4269438072 wait complete. 2023-10-06T04:48:28.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.473+0000 7fa14e59d700 1 Processor -- start 2023-10-06T04:48:28.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.473+0000 7fa14e59d700 1 -- start start 2023-10-06T04:48:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14e59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148159420 0x7fa1481470f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14e59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1481597f0 0x7fa1480cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14e59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa14815d230 0x7fa1480cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14e59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1480cae30 con 0x7fa14815d230 2023-10-06T04:48:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14e59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa148147630 con 0x7fa1481597f0 2023-10-06T04:48:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14e59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1480c9f80 con 0x7fa148159420 2023-10-06T04:48:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14dd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa14815d230 0x7fa1480cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14d59b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148159420 0x7fa1481470f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14cd9a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1481597f0 0x7fa1480cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14d59b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148159420 0x7fa1481470f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:52008/0 (socket says 172.21.15.55:52008) 2023-10-06T04:48:28.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14dd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa14815d230 0x7fa1480cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58536/0 (socket says 172.21.15.55:58536) 2023-10-06T04:48:28.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14d59b700 1 -- 172.21.15.55:0/3594120828 learned_addr learned my addr 172.21.15.55:0/3594120828 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:28.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14d59b700 1 -- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1481597f0 msgr2=0x7fa1480cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:28.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14d59b700 1 --2- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1481597f0 0x7fa1480cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14d59b700 1 -- 172.21.15.55:0/3594120828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa14815d230 msgr2=0x7fa1480cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:28.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14d59b700 1 --2- 172.21.15.55:0/3594120828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa14815d230 0x7fa1480cfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14d59b700 1 -- 172.21.15.55:0/3594120828 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa140009ae0 con 0x7fa148159420 2023-10-06T04:48:28.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.474+0000 7fa14dd9c700 1 --2- 172.21.15.55:0/3594120828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa14815d230 0x7fa1480cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:28.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.475+0000 7fa14d59b700 1 --2- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148159420 0x7fa1481470f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa14400ec80 tx=0x7fa1440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:28.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.475+0000 7fa13d7fa700 1 -- 172.21.15.55:0/3594120828 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400c850 con 0x7fa148159420 2023-10-06T04:48:28.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.475+0000 7fa13d7fa700 1 -- 172.21.15.55:0/3594120828 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14400c9d0 con 0x7fa148159420 2023-10-06T04:48:28.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.475+0000 7fa13d7fa700 1 -- 172.21.15.55:0/3594120828 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400cd90 con 0x7fa148159420 2023-10-06T04:48:28.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.475+0000 7fa14e59d700 1 -- 172.21.15.55:0/3594120828 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1480ca220 con 0x7fa148159420 2023-10-06T04:48:28.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.475+0000 7fa14e59d700 1 -- 172.21.15.55:0/3594120828 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1480ca7e0 con 0x7fa148159420 2023-10-06T04:48:28.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.476+0000 7fa14e59d700 1 -- 172.21.15.55:0/3594120828 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14804c3c0 con 0x7fa148159420 2023-10-06T04:48:28.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.477+0000 7fa13d7fa700 1 -- 172.21.15.55:0/3594120828 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 65) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fa144013680 con 0x7fa148159420 2023-10-06T04:48:28.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.479+0000 7fa13d7fa700 1 --2- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa12c067050 0x7fa12c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.479+0000 7fa13d7fa700 1 -- 172.21.15.55:0/3594120828 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6882+0+0 (secure 0 0 0) 0x7fa144084e20 con 0x7fa148159420 2023-10-06T04:48:28.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.479+0000 7fa14cd9a700 1 -- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa12c067050 msgr2=0x7fa12c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:28.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.479+0000 7fa14cd9a700 1 --2- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa12c067050 0x7fa12c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:28.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.487+0000 7fa13d7fa700 1 -- 172.21.15.55:0/3594120828 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa144051a40 con 0x7fa148159420 2023-10-06T04:48:28.524 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:28.522+0000 7fae0d838180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T04:48:28.649 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:28.646+0000 7f6f17ce5180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T04:48:28.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.680+0000 7fa14cd9a700 1 -- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa12c067050 msgr2=0x7fa12c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6835/85191 2023-10-06T04:48:28.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.680+0000 7fa14cd9a700 1 --2- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa12c067050 0x7fa12c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:28.686 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:28.685+0000 7fae0d838180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T04:48:28.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.735+0000 7fa14e59d700 1 -- 172.21.15.55:0/3594120828 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa1480b1a70 con 0x7fa148159420 2023-10-06T04:48:28.737 INFO:tasks.ceph.mon.a.smithi055.stderr:2023-10-06T04:48:28.736+0000 7fe524ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T04:48:28.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.742+0000 7fa13d7fa700 1 -- 172.21.15.55:0/3594120828 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v71) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa144055060 con 0x7fa148159420 2023-10-06T04:48:28.745 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:28.742+0000 7f6f17ce5180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T04:48:28.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.745+0000 7fa14e59d700 1 -- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa12c067050 msgr2=0x7fa12c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:28.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.745+0000 7fa14e59d700 1 --2- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa12c067050 0x7fa12c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.745+0000 7fa14e59d700 1 -- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148159420 msgr2=0x7fa1481470f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:28.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.745+0000 7fa14e59d700 1 --2- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148159420 0x7fa1481470f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa14400ec80 tx=0x7fa1440091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.745+0000 7fa14e59d700 1 -- 172.21.15.55:0/3594120828 shutdown_connections 2023-10-06T04:48:28.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.745+0000 7fa14e59d700 1 --2- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x7fa12c067050 0x7fa12c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.745+0000 7fa14e59d700 1 --2- 172.21.15.55:0/3594120828 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa14815d230 0x7fa1480cfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.745+0000 7fa14e59d700 1 --2- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa148159420 0x7fa1481470f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.745+0000 7fa14e59d700 1 --2- 172.21.15.55:0/3594120828 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1481597f0 0x7fa1480cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.745+0000 7fa14e59d700 1 -- 172.21.15.55:0/3594120828 >> 172.21.15.55:0/3594120828 conn(0x7fa1480d0660 msgr2=0x7fa148158250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:28.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.745+0000 7fa14e59d700 1 -- 172.21.15.55:0/3594120828 shutdown_connections 2023-10-06T04:48:28.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.745+0000 7fa14e59d700 1 -- 172.21.15.55:0/3594120828 wait complete. 2023-10-06T04:48:28.749 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T04:48:28.766 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T04:48:28.804 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:28.802+0000 7fae0d838180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T04:48:28.805 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2023-10-06T04:48:28.843 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:28.841+0000 7f6f17ce5180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T04:48:28.937 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:28.935+0000 7f6f17ce5180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T04:48:28.939 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:28.937+0000 7f6f0ddfc700 -1 --2- 172.21.15.163:0/85191 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x559de878a000 0x559de878c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] is using msgr V1 protocol 2023-10-06T04:48:28.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.959+0000 7ff964bff700 1 Processor -- start 2023-10-06T04:48:28.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.959+0000 7ff964bff700 1 -- start start 2023-10-06T04:48:28.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff964bff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff960159890 0x7ff96015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff964bff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff960159c60 0x7ff96015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff964bff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff96015db10 0x7ff960069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff964bff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96015df50 con 0x7ff960159c60 2023-10-06T04:48:28.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff964bff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96015a160 con 0x7ff960159890 2023-10-06T04:48:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff964bff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9600d13a0 con 0x7ff96015db10 2023-10-06T04:48:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff95dd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff960159c60 0x7ff96015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff95e59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff960159890 0x7ff96015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff95dd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff960159c60 0x7ff96015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58550/0 (socket says 172.21.15.55:58550) 2023-10-06T04:48:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff95e59c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff960159890 0x7ff96015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:32892/0 (socket says 172.21.15.55:32892) 2023-10-06T04:48:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff95dd9b700 1 -- 172.21.15.55:0/3012340373 learned_addr learned my addr 172.21.15.55:0/3012340373 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff95ed9d700 1 --2- 172.21.15.55:0/3012340373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff96015db10 0x7ff960069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff95dd9b700 1 -- 172.21.15.55:0/3012340373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff96015db10 msgr2=0x7ff960069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff95dd9b700 1 --2- 172.21.15.55:0/3012340373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff96015db10 0x7ff960069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff95dd9b700 1 -- 172.21.15.55:0/3012340373 >> v2:172.21.15.163:3300/0 conn(0x7ff960159890 msgr2=0x7ff96015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:28.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff95dd9b700 1 --2- 172.21.15.55:0/3012340373 >> v2:172.21.15.163:3300/0 conn(0x7ff960159890 0x7ff96015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.961+0000 7ff95dd9b700 1 -- 172.21.15.55:0/3012340373 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96015e140 con 0x7ff960159c60 2023-10-06T04:48:28.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.962+0000 7ff95dd9b700 1 --2- 172.21.15.55:0/3012340373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff960159c60 0x7ff96015d5d0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff9600d1090 tx=0x7ff954009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb64807919070b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:28.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.962+0000 7ff95d59a700 1 -- 172.21.15.55:0/3012340373 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95400b640 con 0x7ff960159c60 2023-10-06T04:48:28.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.962+0000 7ff95d59a700 1 -- 172.21.15.55:0/3012340373 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95400b7a0 con 0x7ff960159c60 2023-10-06T04:48:28.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.962+0000 7ff95d59a700 1 -- 172.21.15.55:0/3012340373 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95400bb60 con 0x7ff960159c60 2023-10-06T04:48:28.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.963+0000 7ff964bff700 1 -- 172.21.15.55:0/3012340373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff960159c60 msgr2=0x7ff96015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:28.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.963+0000 7ff964bff700 1 --2- 172.21.15.55:0/3012340373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff960159c60 0x7ff96015d5d0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff9600d1090 tx=0x7ff954009250 comp rx=0 tx=0).stop 2023-10-06T04:48:28.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.963+0000 7ff964bff700 1 -- 172.21.15.55:0/3012340373 shutdown_connections 2023-10-06T04:48:28.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.963+0000 7ff964bff700 1 --2- 172.21.15.55:0/3012340373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff96015db10 0x7ff960069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.963+0000 7ff964bff700 1 --2- 172.21.15.55:0/3012340373 >> v2:172.21.15.163:3300/0 conn(0x7ff960159890 0x7ff96015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.963+0000 7ff964bff700 1 --2- 172.21.15.55:0/3012340373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff960159c60 0x7ff96015d5d0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.963+0000 7ff964bff700 1 -- 172.21.15.55:0/3012340373 >> 172.21.15.55:0/3012340373 conn(0x7ff960068e20 msgr2=0x7ff9600d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:28.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.963+0000 7ff964bff700 1 -- 172.21.15.55:0/3012340373 shutdown_connections 2023-10-06T04:48:28.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.963+0000 7ff964bff700 1 -- 172.21.15.55:0/3012340373 wait complete. 2023-10-06T04:48:28.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.965+0000 7ff964bff700 1 Processor -- start 2023-10-06T04:48:28.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.965+0000 7ff964bff700 1 -- start start 2023-10-06T04:48:28.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.966+0000 7ff964bff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff960159890 0x7ff9600b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.966+0000 7ff964bff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff960159c60 0x7ff960157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.966+0000 7ff964bff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff96015db10 0x7ff9600c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:28.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.966+0000 7ff964bff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9600afd20 con 0x7ff960159890 2023-10-06T04:48:28.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.966+0000 7ff964bff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9600b49b0 con 0x7ff96015db10 2023-10-06T04:48:28.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.966+0000 7ff964bff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9600b4c90 con 0x7ff960159c60 2023-10-06T04:48:28.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.966+0000 7ff95e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff960159890 0x7ff9600b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.966+0000 7ff95e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff960159890 0x7ff9600b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58566/0 (socket says 172.21.15.55:58566) 2023-10-06T04:48:28.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.966+0000 7ff95e59c700 1 -- 172.21.15.55:0/1793053515 learned_addr learned my addr 172.21.15.55:0/1793053515 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:28.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.966+0000 7ff95dd9b700 1 --2- 172.21.15.55:0/1793053515 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff960159c60 0x7ff960157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.966+0000 7ff95ed9d700 1 --2- 172.21.15.55:0/1793053515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff96015db10 0x7ff9600c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:28.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.966+0000 7ff95e59c700 1 -- 172.21.15.55:0/1793053515 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff960159c60 msgr2=0x7ff960157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:28.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.966+0000 7ff95e59c700 1 --2- 172.21.15.55:0/1793053515 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff960159c60 0x7ff960157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.967+0000 7ff95e59c700 1 -- 172.21.15.55:0/1793053515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff96015db10 msgr2=0x7ff9600c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:28.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.967+0000 7ff95e59c700 1 --2- 172.21.15.55:0/1793053515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff96015db10 0x7ff9600c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:28.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.967+0000 7ff95e59c700 1 -- 172.21.15.55:0/1793053515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9540094f0 con 0x7ff960159890 2023-10-06T04:48:28.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.967+0000 7ff95e59c700 1 --2- 172.21.15.55:0/1793053515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff960159890 0x7ff9600b5df0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff95000e930 tx=0x7ff9500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:28.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.967+0000 7ff94e7fc700 1 -- 172.21.15.55:0/1793053515 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950013680 con 0x7ff960159890 2023-10-06T04:48:28.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.967+0000 7ff94e7fc700 1 -- 172.21.15.55:0/1793053515 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9500137e0 con 0x7ff960159890 2023-10-06T04:48:28.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.967+0000 7ff964bff700 1 -- 172.21.15.55:0/1793053515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9600b4f70 con 0x7ff960159890 2023-10-06T04:48:28.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.967+0000 7ff94e7fc700 1 -- 172.21.15.55:0/1793053515 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950013b80 con 0x7ff960159890 2023-10-06T04:48:28.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.967+0000 7ff964bff700 1 -- 172.21.15.55:0/1793053515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9600b6330 con 0x7ff960159890 2023-10-06T04:48:28.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.968+0000 7ff964bff700 1 -- 172.21.15.55:0/1793053515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9600c7ab0 con 0x7ff960159890 2023-10-06T04:48:28.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.969+0000 7ff94e7fc700 1 -- 172.21.15.55:0/1793053515 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 66) v1 ==== 84451+0+0 (secure 0 0 0) 0x7ff95000c3e0 con 0x7ff960159890 2023-10-06T04:48:28.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.970+0000 7ff94e7fc700 1 -- 172.21.15.55:0/1793053515 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 7182+0+0 (secure 0 0 0) 0x7ff950084760 con 0x7ff960159890 2023-10-06T04:48:28.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:28.975+0000 7ff94e7fc700 1 -- 172.21.15.55:0/1793053515 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9500516e0 con 0x7ff960159890 2023-10-06T04:48:29.014 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:29.013+0000 7fae0d838180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T04:48:29.020 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:29.017+0000 7f6e95888700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:29.020 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:29.017+0000 7f6e95888700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:29.020 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:29.017+0000 7f6e95888700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:29.020 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:29.017+0000 7f6e95888700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:29.020 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:29.017+0000 7f6e95888700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:29.020 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:29.018+0000 7f6e92882700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:29.020 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:29.018+0000 7f6e92882700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:29.021 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:29.018+0000 7f6e92882700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:29.021 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:29.018+0000 7f6e92882700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:29.021 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:29.018+0000 7f6e92882700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:29.114 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:29.112+0000 7fae0d838180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T04:48:29.211 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:29.209+0000 7fae0d838180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T04:48:29.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:29.229+0000 7ff964bff700 1 -- 172.21.15.55:0/1793053515 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff9600b5470 con 0x7ff960159890 2023-10-06T04:48:29.310 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:29.308+0000 7fae0d838180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T04:48:29.313 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:29.312+0000 7fae039fc700 -1 --2- 172.21.15.55:0/114784 >> [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] conn(0x555c31d42000 0x555c31d44000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.163:6835/85191,v1:172.21.15.163:6837/85191] is using msgr V1 protocol 2023-10-06T04:48:29.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:29.741+0000 7ff94e7fc700 1 -- 172.21.15.55:0/1793053515 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v72) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff950054d00 con 0x7ff960159890 2023-10-06T04:48:29.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:29.744+0000 7ff964bff700 1 -- 172.21.15.55:0/1793053515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff960159890 msgr2=0x7ff9600b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:29.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:29.744+0000 7ff964bff700 1 --2- 172.21.15.55:0/1793053515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff960159890 0x7ff9600b5df0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff95000e930 tx=0x7ff9500091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:29.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:29.744+0000 7ff964bff700 1 -- 172.21.15.55:0/1793053515 shutdown_connections 2023-10-06T04:48:29.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:29.744+0000 7ff964bff700 1 --2- 172.21.15.55:0/1793053515 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff96015db10 0x7ff9600c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:29.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:29.744+0000 7ff964bff700 1 --2- 172.21.15.55:0/1793053515 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff960159890 0x7ff9600b5df0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:29.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:29.744+0000 7ff964bff700 1 --2- 172.21.15.55:0/1793053515 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff960159c60 0x7ff960157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:29.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:29.744+0000 7ff964bff700 1 -- 172.21.15.55:0/1793053515 >> 172.21.15.55:0/1793053515 conn(0x7ff960068e20 msgr2=0x7ff9601589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:29.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:29.744+0000 7ff964bff700 1 -- 172.21.15.55:0/1793053515 shutdown_connections 2023-10-06T04:48:29.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:29.744+0000 7ff964bff700 1 -- 172.21.15.55:0/1793053515 wait complete. 2023-10-06T04:48:29.765 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T04:48:30.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.000+0000 7faa1a19d700 1 Processor -- start 2023-10-06T04:48:30.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.000+0000 7faa1a19d700 1 -- start start 2023-10-06T04:48:30.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.001+0000 7faa1a19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa141597d0 0x7faa140afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:30.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.001+0000 7faa1a19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faa140ac0e0 0x7faa1406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:30.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.001+0000 7faa1a19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faa1406eee0 0x7faa1406f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:30.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.001+0000 7faa1a19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1406e510 con 0x7faa140ac0e0 2023-10-06T04:48:30.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.001+0000 7faa1a19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa14069180 con 0x7faa1406eee0 2023-10-06T04:48:30.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.001+0000 7faa1a19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1406dec0 con 0x7faa141597d0 2023-10-06T04:48:30.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.002+0000 7faa1899a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faa140ac0e0 0x7faa1406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:30.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.002+0000 7faa1899a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faa140ac0e0 0x7faa1406e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58578/0 (socket says 172.21.15.55:58578) 2023-10-06T04:48:30.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.002+0000 7faa1899a700 1 -- 172.21.15.55:0/3811649546 learned_addr learned my addr 172.21.15.55:0/3811649546 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:30.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.002+0000 7faa1919b700 1 --2- 172.21.15.55:0/3811649546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa141597d0 0x7faa140afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:30.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.002+0000 7faa1999c700 1 --2- 172.21.15.55:0/3811649546 >> v2:172.21.15.163:3300/0 conn(0x7faa1406eee0 0x7faa1406f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:30.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.002+0000 7faa1899a700 1 -- 172.21.15.55:0/3811649546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa141597d0 msgr2=0x7faa140afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:30.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.002+0000 7faa1899a700 1 --2- 172.21.15.55:0/3811649546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa141597d0 0x7faa140afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:30.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.002+0000 7faa1899a700 1 -- 172.21.15.55:0/3811649546 >> v2:172.21.15.163:3300/0 conn(0x7faa1406eee0 msgr2=0x7faa1406f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:30.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.002+0000 7faa1899a700 1 --2- 172.21.15.55:0/3811649546 >> v2:172.21.15.163:3300/0 conn(0x7faa1406eee0 0x7faa1406f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:30.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.002+0000 7faa1899a700 1 -- 172.21.15.55:0/3811649546 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1406e1a0 con 0x7faa140ac0e0 2023-10-06T04:48:30.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.003+0000 7faa1899a700 1 --2- 172.21.15.55:0/3811649546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa140ac0e0 0x7faa1406e9a0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7faa14069890 tx=0x7faa04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7baec3402e703942 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:30.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.003+0000 7faa0bfff700 1 -- 172.21.15.55:0/3811649546 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0400b860 con 0x7faa140ac0e0 2023-10-06T04:48:30.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.003+0000 7faa0bfff700 1 -- 172.21.15.55:0/3811649546 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0400b9c0 con 0x7faa140ac0e0 2023-10-06T04:48:30.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.003+0000 7faa0bfff700 1 -- 172.21.15.55:0/3811649546 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0400bd60 con 0x7faa140ac0e0 2023-10-06T04:48:30.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.003+0000 7faa1a19d700 1 -- 172.21.15.55:0/3811649546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa140ac0e0 msgr2=0x7faa1406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:30.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.003+0000 7faa1a19d700 1 --2- 172.21.15.55:0/3811649546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa140ac0e0 0x7faa1406e9a0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7faa14069890 tx=0x7faa04009250 comp rx=0 tx=0).stop 2023-10-06T04:48:30.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.003+0000 7faa1a19d700 1 -- 172.21.15.55:0/3811649546 shutdown_connections 2023-10-06T04:48:30.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.003+0000 7faa1a19d700 1 --2- 172.21.15.55:0/3811649546 >> v2:172.21.15.163:3300/0 conn(0x7faa1406eee0 0x7faa1406f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:30.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.003+0000 7faa1a19d700 1 --2- 172.21.15.55:0/3811649546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa141597d0 0x7faa140afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:30.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.003+0000 7faa1a19d700 1 --2- 172.21.15.55:0/3811649546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa140ac0e0 0x7faa1406e9a0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:30.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.003+0000 7faa1a19d700 1 -- 172.21.15.55:0/3811649546 >> 172.21.15.55:0/3811649546 conn(0x7faa1406adf0 msgr2=0x7faa140ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:30.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.004+0000 7faa1a19d700 1 -- 172.21.15.55:0/3811649546 shutdown_connections 2023-10-06T04:48:30.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.004+0000 7faa1a19d700 1 -- 172.21.15.55:0/3811649546 wait complete. 2023-10-06T04:48:30.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.005+0000 7faa1a19d700 1 Processor -- start 2023-10-06T04:48:30.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.005+0000 7faa1a19d700 1 -- start start 2023-10-06T04:48:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.006+0000 7faa1a19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa1406eee0 0x7faa140cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.006+0000 7faa1a19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa141597d0 0x7faa140ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.006+0000 7faa1a19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa141539a0 0x7faa140acb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.006+0000 7faa1a19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1416e400 con 0x7faa141539a0 2023-10-06T04:48:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.006+0000 7faa1a19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1416d7f0 con 0x7faa1406eee0 2023-10-06T04:48:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.006+0000 7faa1a19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1416dad0 con 0x7faa141597d0 2023-10-06T04:48:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.006+0000 7faa1999c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa141539a0 0x7faa140acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:30.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.006+0000 7faa1999c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa141539a0 0x7faa140acb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58594/0 (socket says 172.21.15.55:58594) 2023-10-06T04:48:30.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.006+0000 7faa1999c700 1 -- 172.21.15.55:0/499543915 learned_addr learned my addr 172.21.15.55:0/499543915 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:30.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.007+0000 7faa1919b700 1 --2- 172.21.15.55:0/499543915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa1406eee0 0x7faa140cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:30.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.007+0000 7faa1899a700 1 --2- 172.21.15.55:0/499543915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa141597d0 0x7faa140ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:30.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.007+0000 7faa1999c700 1 -- 172.21.15.55:0/499543915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa141597d0 msgr2=0x7faa140ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:30.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.007+0000 7faa1999c700 1 --2- 172.21.15.55:0/499543915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa141597d0 0x7faa140ac650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:30.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.007+0000 7faa1999c700 1 -- 172.21.15.55:0/499543915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa1406eee0 msgr2=0x7faa140cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:30.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.007+0000 7faa1999c700 1 --2- 172.21.15.55:0/499543915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa1406eee0 0x7faa140cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:30.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.007+0000 7faa1999c700 1 -- 172.21.15.55:0/499543915 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa040094f0 con 0x7faa141539a0 2023-10-06T04:48:30.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.007+0000 7faa1999c700 1 --2- 172.21.15.55:0/499543915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa141539a0 0x7faa140acb90 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009fa0 tx=0x7faa0c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:30.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.007+0000 7faa097fa700 1 -- 172.21.15.55:0/499543915 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00ed20 con 0x7faa141539a0 2023-10-06T04:48:30.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.007+0000 7faa097fa700 1 -- 172.21.15.55:0/499543915 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c00ee80 con 0x7faa141539a0 2023-10-06T04:48:30.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.007+0000 7faa097fa700 1 -- 172.21.15.55:0/499543915 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c014670 con 0x7faa141539a0 2023-10-06T04:48:30.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.007+0000 7faa1a19d700 1 -- 172.21.15.55:0/499543915 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1416ddb0 con 0x7faa141539a0 2023-10-06T04:48:30.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.007+0000 7faa1a19d700 1 -- 172.21.15.55:0/499543915 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa140cc1f0 con 0x7faa141539a0 2023-10-06T04:48:30.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.008+0000 7faa1a19d700 1 -- 172.21.15.55:0/499543915 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1406d620 con 0x7faa141539a0 2023-10-06T04:48:30.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.009+0000 7faa097fa700 1 -- 172.21.15.55:0/499543915 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 67) v1 ==== 84578+0+0 (secure 0 0 0) 0x7faa0c0149e0 con 0x7faa141539a0 2023-10-06T04:48:30.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.010+0000 7faa097fa700 1 --2- 172.21.15.55:0/499543915 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa9f8066d80 0x7fa9f8069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:30.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.010+0000 7faa097fa700 1 -- 172.21.15.55:0/499543915 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 7182+0+0 (secure 0 0 0) 0x7faa0c085060 con 0x7faa141539a0 2023-10-06T04:48:30.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.011+0000 7faa1919b700 1 --2- 172.21.15.55:0/499543915 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa9f8066d80 0x7fa9f8069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:30.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.011+0000 7faa1919b700 1 --2- 172.21.15.55:0/499543915 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa9f8066d80 0x7fa9f8069230 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faa140ad440 tx=0x7faa10008040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:30.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.014+0000 7faa097fa700 1 -- 172.21.15.55:0/499543915 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faa0c051ef0 con 0x7faa141539a0 2023-10-06T04:48:30.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.215+0000 7faa1a19d700 1 -- 172.21.15.55:0/499543915 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faa1404c3c0 con 0x7faa141539a0 2023-10-06T04:48:30.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.957+0000 7faa097fa700 1 -- 172.21.15.55:0/499543915 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v150) v1 ==== 181+0+0 (secure 0 0 0) 0x7faa0c055510 con 0x7faa141539a0 2023-10-06T04:48:30.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.961+0000 7faa1a19d700 1 -- 172.21.15.55:0/499543915 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa9f8066d80 msgr2=0x7fa9f8069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:30.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.961+0000 7faa1a19d700 1 --2- 172.21.15.55:0/499543915 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa9f8066d80 0x7fa9f8069230 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faa140ad440 tx=0x7faa10008040 comp rx=0 tx=0).stop 2023-10-06T04:48:30.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.961+0000 7faa1a19d700 1 -- 172.21.15.55:0/499543915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa141539a0 msgr2=0x7faa140acb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:30.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.961+0000 7faa1a19d700 1 --2- 172.21.15.55:0/499543915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa141539a0 0x7faa140acb90 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009fa0 tx=0x7faa0c009900 comp rx=0 tx=0).stop 2023-10-06T04:48:30.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.961+0000 7faa1a19d700 1 -- 172.21.15.55:0/499543915 shutdown_connections 2023-10-06T04:48:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.961+0000 7faa1a19d700 1 --2- 172.21.15.55:0/499543915 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa9f8066d80 0x7fa9f8069230 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.961+0000 7faa1a19d700 1 --2- 172.21.15.55:0/499543915 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faa141539a0 0x7faa140acb90 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.961+0000 7faa1a19d700 1 --2- 172.21.15.55:0/499543915 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faa1406eee0 0x7faa140cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.961+0000 7faa1a19d700 1 --2- 172.21.15.55:0/499543915 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faa141597d0 0x7faa140ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.961+0000 7faa1a19d700 1 -- 172.21.15.55:0/499543915 >> 172.21.15.55:0/499543915 conn(0x7faa1406adf0 msgr2=0x7faa140afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.961+0000 7faa1a19d700 1 -- 172.21.15.55:0/499543915 shutdown_connections 2023-10-06T04:48:30.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:30.962+0000 7faa1a19d700 1 -- 172.21.15.55:0/499543915 wait complete. 2023-10-06T04:48:30.965 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2023-10-06T04:48:30.984 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T04:48:31.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.178+0000 7fb967fff700 1 Processor -- start 2023-10-06T04:48:31.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.178+0000 7fb967fff700 1 -- start start 2023-10-06T04:48:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.179+0000 7fb967fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb96815d3f0 0x7fb9681538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.179+0000 7fb967fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb9680689b0 0x7fb968153e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.179+0000 7fb967fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9680c9480 con 0x7fb9680685e0 2023-10-06T04:48:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.179+0000 7fb967fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96815ae00 con 0x7fb9680689b0 2023-10-06T04:48:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.179+0000 7fb967fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96815d880 con 0x7fb96815d3f0 2023-10-06T04:48:31.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.179+0000 7fb966ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb9680685e0 0x7fb96815a6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38034/0 (socket says 172.21.15.55:38034) 2023-10-06T04:48:31.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.179+0000 7fb966ffd700 1 -- 172.21.15.55:0/4180173064 learned_addr learned my addr 172.21.15.55:0/4180173064 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:31.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.179+0000 7fb9677fe700 1 --2- 172.21.15.55:0/4180173064 >> v2:172.21.15.55:3300/0 conn(0x7fb9680689b0 0x7fb968153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:31.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.180+0000 7fb9667fc700 1 --2- 172.21.15.55:0/4180173064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb96815d3f0 0x7fb9681538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:31.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.180+0000 7fb965ffb700 1 -- 172.21.15.55:0/4180173064 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871361936 0 0) 0x7fb9680c9480 con 0x7fb9680685e0 2023-10-06T04:48:31.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.180+0000 7fb965ffb700 1 -- 172.21.15.55:0/4180173064 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb950003570 con 0x7fb9680685e0 2023-10-06T04:48:31.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.180+0000 7fb9677fe700 1 -- 172.21.15.55:0/4180173064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb96815d3f0 msgr2=0x7fb9681538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:31.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.180+0000 7fb9677fe700 1 --2- 172.21.15.55:0/4180173064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb96815d3f0 0x7fb9681538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.180+0000 7fb9677fe700 1 -- 172.21.15.55:0/4180173064 >> v1:172.21.15.55:6789/0 conn(0x7fb9680685e0 legacy=0x7fb96815a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:31.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.180+0000 7fb9677fe700 1 -- 172.21.15.55:0/4180173064 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb96815db60 con 0x7fb9680689b0 2023-10-06T04:48:31.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.180+0000 7fb9677fe700 1 --2- 172.21.15.55:0/4180173064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9680689b0 0x7fb968153e30 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fb960009e60 tx=0x7fb960009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494b1d4a1fb4a72b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:31.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.180+0000 7fb965ffb700 1 -- 172.21.15.55:0/4180173064 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96000e070 con 0x7fb9680689b0 2023-10-06T04:48:31.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.180+0000 7fb965ffb700 1 -- 172.21.15.55:0/4180173064 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96000bbf0 con 0x7fb9680689b0 2023-10-06T04:48:31.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.180+0000 7fb965ffb700 1 -- 172.21.15.55:0/4180173064 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb960012430 con 0x7fb9680689b0 2023-10-06T04:48:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.181+0000 7fb967fff700 1 -- 172.21.15.55:0/4180173064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9680689b0 msgr2=0x7fb968153e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.181+0000 7fb967fff700 1 --2- 172.21.15.55:0/4180173064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9680689b0 0x7fb968153e30 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fb960009e60 tx=0x7fb960009780 comp rx=0 tx=0).stop 2023-10-06T04:48:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.181+0000 7fb967fff700 1 -- 172.21.15.55:0/4180173064 shutdown_connections 2023-10-06T04:48:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.181+0000 7fb967fff700 1 --2- 172.21.15.55:0/4180173064 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9680689b0 0x7fb968153e30 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.181+0000 7fb967fff700 1 --2- 172.21.15.55:0/4180173064 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb96815d3f0 0x7fb9681538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.181+0000 7fb967fff700 1 -- 172.21.15.55:0/4180173064 >> 172.21.15.55:0/4180173064 conn(0x7fb96806b420 msgr2=0x7fb9680cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.181+0000 7fb967fff700 1 -- 172.21.15.55:0/4180173064 shutdown_connections 2023-10-06T04:48:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.181+0000 7fb967fff700 1 -- 172.21.15.55:0/4180173064 wait complete. 2023-10-06T04:48:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.182+0000 7fb967fff700 1 Processor -- start 2023-10-06T04:48:31.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.183+0000 7fb967fff700 1 -- start start 2023-10-06T04:48:31.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb967fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9680685e0 0x7fb9680b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:31.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb967fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9680689b0 0x7fb9680b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:31.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb967fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb96815d3f0 0x7fb9680b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:31.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb967fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9680b2140 con 0x7fb96815d3f0 2023-10-06T04:48:31.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb967fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9680b0750 con 0x7fb9680685e0 2023-10-06T04:48:31.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb967fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9680b1810 con 0x7fb9680689b0 2023-10-06T04:48:31.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb9677fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb96815d3f0 0x7fb9680b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:31.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb9677fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb96815d3f0 0x7fb9680b0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58610/0 (socket says 172.21.15.55:58610) 2023-10-06T04:48:31.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb9677fe700 1 -- 172.21.15.55:0/1118064649 learned_addr learned my addr 172.21.15.55:0/1118064649 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:31.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb9667fc700 1 --2- 172.21.15.55:0/1118064649 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9680689b0 0x7fb9680b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:31.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb966ffd700 1 --2- 172.21.15.55:0/1118064649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9680685e0 0x7fb9680b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:31.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb9677fe700 1 -- 172.21.15.55:0/1118064649 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9680689b0 msgr2=0x7fb9680b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:31.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb9677fe700 1 --2- 172.21.15.55:0/1118064649 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9680689b0 0x7fb9680b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb9677fe700 1 -- 172.21.15.55:0/1118064649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9680685e0 msgr2=0x7fb9680b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:31.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb9677fe700 1 --2- 172.21.15.55:0/1118064649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9680685e0 0x7fb9680b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.184+0000 7fb9677fe700 1 -- 172.21.15.55:0/1118064649 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb960009a20 con 0x7fb96815d3f0 2023-10-06T04:48:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.185+0000 7fb9677fe700 1 --2- 172.21.15.55:0/1118064649 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb96815d3f0 0x7fb9680b0210 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fb968070fe0 tx=0x7fb960019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.185+0000 7fb94effd700 1 -- 172.21.15.55:0/1118064649 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96001d630 con 0x7fb96815d3f0 2023-10-06T04:48:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.185+0000 7fb94effd700 1 -- 172.21.15.55:0/1118064649 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96001d790 con 0x7fb96815d3f0 2023-10-06T04:48:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.185+0000 7fb967fff700 1 -- 172.21.15.55:0/1118064649 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9680b1af0 con 0x7fb96815d3f0 2023-10-06T04:48:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.185+0000 7fb94effd700 1 -- 172.21.15.55:0/1118064649 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96001db50 con 0x7fb96815d3f0 2023-10-06T04:48:31.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.185+0000 7fb967fff700 1 -- 172.21.15.55:0/1118064649 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9680b1500 con 0x7fb96815d3f0 2023-10-06T04:48:31.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.186+0000 7fb967fff700 1 -- 172.21.15.55:0/1118064649 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9680b22b0 con 0x7fb96815d3f0 2023-10-06T04:48:31.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.187+0000 7fb94effd700 1 -- 172.21.15.55:0/1118064649 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 67) v1 ==== 84578+0+0 (secure 0 0 0) 0x7fb960024070 con 0x7fb96815d3f0 2023-10-06T04:48:31.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.188+0000 7fb94effd700 1 --2- 172.21.15.55:0/1118064649 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb954066d80 0x7fb954069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:31.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.188+0000 7fb94effd700 1 -- 172.21.15.55:0/1118064649 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6744+0+0 (secure 0 0 0) 0x7fb960086f40 con 0x7fb96815d3f0 2023-10-06T04:48:31.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.189+0000 7fb966ffd700 1 --2- 172.21.15.55:0/1118064649 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb954066d80 0x7fb954069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:31.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.189+0000 7fb966ffd700 1 --2- 172.21.15.55:0/1118064649 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb954066d80 0x7fb954069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb968153770 tx=0x7fb958009040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:31.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.193+0000 7fb94effd700 1 -- 172.21.15.55:0/1118064649 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb960053f80 con 0x7fb96815d3f0 2023-10-06T04:48:31.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.402+0000 7fb967fff700 1 -- 172.21.15.55:0/1118064649 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb9680c9c60 con 0x7fb96815d3f0 2023-10-06T04:48:31.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.665+0000 7fb94effd700 1 -- 172.21.15.55:0/1118064649 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v151) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb9600575a0 con 0x7fb96815d3f0 2023-10-06T04:48:31.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.670+0000 7fb967fff700 1 -- 172.21.15.55:0/1118064649 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb954066d80 msgr2=0x7fb954069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:31.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.670+0000 7fb967fff700 1 --2- 172.21.15.55:0/1118064649 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb954066d80 0x7fb954069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb968153770 tx=0x7fb958009040 comp rx=0 tx=0).stop 2023-10-06T04:48:31.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.670+0000 7fb967fff700 1 -- 172.21.15.55:0/1118064649 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb96815d3f0 msgr2=0x7fb9680b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:31.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.670+0000 7fb967fff700 1 --2- 172.21.15.55:0/1118064649 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb96815d3f0 0x7fb9680b0210 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fb968070fe0 tx=0x7fb960019000 comp rx=0 tx=0).stop 2023-10-06T04:48:31.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.670+0000 7fb967fff700 1 -- 172.21.15.55:0/1118064649 shutdown_connections 2023-10-06T04:48:31.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.670+0000 7fb967fff700 1 --2- 172.21.15.55:0/1118064649 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb954066d80 0x7fb954069230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.670+0000 7fb967fff700 1 --2- 172.21.15.55:0/1118064649 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb96815d3f0 0x7fb9680b0210 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.670+0000 7fb967fff700 1 --2- 172.21.15.55:0/1118064649 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9680685e0 0x7fb9680b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.670+0000 7fb967fff700 1 --2- 172.21.15.55:0/1118064649 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9680689b0 0x7fb9680b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.670+0000 7fb967fff700 1 -- 172.21.15.55:0/1118064649 >> 172.21.15.55:0/1118064649 conn(0x7fb96806b420 msgr2=0x7fb96815a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:31.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.670+0000 7fb967fff700 1 -- 172.21.15.55:0/1118064649 shutdown_connections 2023-10-06T04:48:31.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.671+0000 7fb967fff700 1 -- 172.21.15.55:0/1118064649 wait complete. 2023-10-06T04:48:31.674 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2023-10-06T04:48:31.692 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T04:48:31.820 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:48:31.827 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:48:31.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.898+0000 7f0d2759e700 1 Processor -- start 2023-10-06T04:48:31.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.898+0000 7f0d2759e700 1 -- start start 2023-10-06T04:48:31.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.899+0000 7f0d2759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0d28153900 0x7f0d280c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:31.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.899+0000 7f0d2759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0d280c9db0 0x7f0d2806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:31.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.899+0000 7f0d2759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d2806e150 con 0x7f0d28157130 2023-10-06T04:48:31.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.899+0000 7f0d2759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d28069e80 con 0x7f0d280c9db0 2023-10-06T04:48:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.899+0000 7f0d2759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d280c7800 con 0x7f0d28153900 2023-10-06T04:48:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.899+0000 7f0d25d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0d280c9db0 0x7f0d2806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.900+0000 7f0d26d9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0d28157130 0x7f0d28069770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38044/0 (socket says 172.21.15.55:38044) 2023-10-06T04:48:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.900+0000 7f0d25d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0d280c9db0 0x7f0d2806a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58618/0 (socket says 172.21.15.55:58618) 2023-10-06T04:48:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.900+0000 7f0d26d9d700 1 -- 172.21.15.55:0/2955443264 learned_addr learned my addr 172.21.15.55:0/2955443264 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.900+0000 7f0d2659c700 1 --2- 172.21.15.55:0/2955443264 >> v2:172.21.15.163:3300/0 conn(0x7f0d28153900 0x7f0d280c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.900+0000 7f0d25d9b700 1 -- 172.21.15.55:0/2955443264 >> v2:172.21.15.163:3300/0 conn(0x7f0d28153900 msgr2=0x7f0d280c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.900+0000 7f0d25d9b700 1 --2- 172.21.15.55:0/2955443264 >> v2:172.21.15.163:3300/0 conn(0x7f0d28153900 0x7f0d280c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.900+0000 7f0d25d9b700 1 -- 172.21.15.55:0/2955443264 >> v1:172.21.15.55:6789/0 conn(0x7f0d28157130 legacy=0x7f0d28069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.900+0000 7f0d25d9b700 1 -- 172.21.15.55:0/2955443264 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d280c7ae0 con 0x7f0d280c9db0 2023-10-06T04:48:31.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.900+0000 7f0d2559a700 1 -- 172.21.15.55:0/2955443264 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 352773707 0 0) 0x7f0d2806e150 con 0x7f0d28157130 2023-10-06T04:48:31.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.900+0000 7f0d25d9b700 1 --2- 172.21.15.55:0/2955443264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d280c9db0 0x7f0d2806a000 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f0d2806aed0 tx=0x7f0d1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a232bde0dd60f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:31.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.901+0000 7f0d2559a700 1 -- 172.21.15.55:0/2955443264 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c011630 con 0x7f0d280c9db0 2023-10-06T04:48:31.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.901+0000 7f0d2559a700 1 -- 172.21.15.55:0/2955443264 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c011790 con 0x7f0d280c9db0 2023-10-06T04:48:31.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.901+0000 7f0d2559a700 1 -- 172.21.15.55:0/2955443264 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c011b30 con 0x7f0d280c9db0 2023-10-06T04:48:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.901+0000 7f0d2759e700 1 -- 172.21.15.55:0/2955443264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d280c9db0 msgr2=0x7f0d2806a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.901+0000 7f0d2759e700 1 --2- 172.21.15.55:0/2955443264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d280c9db0 0x7f0d2806a000 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f0d2806aed0 tx=0x7f0d1c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.901+0000 7f0d2759e700 1 -- 172.21.15.55:0/2955443264 shutdown_connections 2023-10-06T04:48:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.901+0000 7f0d2759e700 1 --2- 172.21.15.55:0/2955443264 >> v2:172.21.15.163:3300/0 conn(0x7f0d28153900 0x7f0d280c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.901+0000 7f0d2759e700 1 --2- 172.21.15.55:0/2955443264 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d280c9db0 0x7f0d2806a000 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.901+0000 7f0d2759e700 1 -- 172.21.15.55:0/2955443264 >> 172.21.15.55:0/2955443264 conn(0x7f0d280cc1e0 msgr2=0x7f0d280cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.901+0000 7f0d2759e700 1 -- 172.21.15.55:0/2955443264 shutdown_connections 2023-10-06T04:48:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.901+0000 7f0d2759e700 1 -- 172.21.15.55:0/2955443264 wait complete. 2023-10-06T04:48:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.903+0000 7f0d2759e700 1 Processor -- start 2023-10-06T04:48:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.903+0000 7f0d2759e700 1 -- start start 2023-10-06T04:48:31.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d2759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d280c9db0 0x7f0d280c5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d2759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d28153900 0x7f0d28142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d2759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d28157130 0x7f0d280aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d2759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d28147190 con 0x7f0d28153900 2023-10-06T04:48:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d2759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d280ac590 con 0x7f0d28157130 2023-10-06T04:48:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d2759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d280ac870 con 0x7f0d280c9db0 2023-10-06T04:48:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d25d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d28153900 0x7f0d28142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d25d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d28153900 0x7f0d28142cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58632/0 (socket says 172.21.15.55:58632) 2023-10-06T04:48:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d25d9b700 1 -- 172.21.15.55:0/492299257 learned_addr learned my addr 172.21.15.55:0/492299257 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d26d9d700 1 --2- 172.21.15.55:0/492299257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d28157130 0x7f0d280aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d2659c700 1 --2- 172.21.15.55:0/492299257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d280c9db0 0x7f0d280c5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d25d9b700 1 -- 172.21.15.55:0/492299257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d280c9db0 msgr2=0x7f0d280c5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d25d9b700 1 --2- 172.21.15.55:0/492299257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d280c9db0 0x7f0d280c5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d25d9b700 1 -- 172.21.15.55:0/492299257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d28157130 msgr2=0x7f0d280aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d25d9b700 1 --2- 172.21.15.55:0/492299257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d28157130 0x7f0d280aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.904+0000 7f0d25d9b700 1 -- 172.21.15.55:0/492299257 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c009420 con 0x7f0d28153900 2023-10-06T04:48:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.905+0000 7f0d25d9b700 1 --2- 172.21.15.55:0/492299257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d28153900 0x7f0d28142cc0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0091e0 tx=0x7f0d1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.905+0000 7f0d167fc700 1 -- 172.21.15.55:0/492299257 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c011b10 con 0x7f0d28153900 2023-10-06T04:48:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.905+0000 7f0d167fc700 1 -- 172.21.15.55:0/492299257 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1c000c40 con 0x7f0d28153900 2023-10-06T04:48:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.905+0000 7f0d2759e700 1 -- 172.21.15.55:0/492299257 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d280acb50 con 0x7f0d28153900 2023-10-06T04:48:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.905+0000 7f0d167fc700 1 -- 172.21.15.55:0/492299257 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c025640 con 0x7f0d28153900 2023-10-06T04:48:31.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.905+0000 7f0d2759e700 1 -- 172.21.15.55:0/492299257 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d280ad110 con 0x7f0d28153900 2023-10-06T04:48:31.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.906+0000 7f0d2759e700 1 -- 172.21.15.55:0/492299257 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d280ad300 con 0x7f0d28153900 2023-10-06T04:48:31.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.907+0000 7f0d167fc700 1 -- 172.21.15.55:0/492299257 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0d1c02e450 con 0x7f0d28153900 2023-10-06T04:48:31.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.908+0000 7f0d167fc700 1 --2- 172.21.15.55:0/492299257 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0d08066f30 0x7f0d080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:31.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.908+0000 7f0d167fc700 1 -- 172.21.15.55:0/492299257 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6330+0+0 (secure 0 0 0) 0x7f0d1c085f50 con 0x7f0d28153900 2023-10-06T04:48:31.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.909+0000 7f0d2659c700 1 --2- 172.21.15.55:0/492299257 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0d08066f30 0x7f0d080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:31.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.909+0000 7f0d2659c700 1 --2- 172.21.15.55:0/492299257 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0d08066f30 0x7f0d080693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009d20 tx=0x7f0d180091a0 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:31.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:31.913+0000 7f0d167fc700 1 -- 172.21.15.55:0/492299257 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d1c052eb0 con 0x7f0d28153900 2023-10-06T04:48:32.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.115+0000 7f0d2759e700 1 -- 172.21.15.55:0/492299257 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0d280c7dc0 con 0x7f0d28153900 2023-10-06T04:48:32.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.119+0000 7f0d167fc700 1 -- 172.21.15.55:0/492299257 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0d1c0564d0 con 0x7f0d28153900 2023-10-06T04:48:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.122+0000 7f0d2759e700 1 -- 172.21.15.55:0/492299257 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0d08066f30 msgr2=0x7f0d080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.122+0000 7f0d2759e700 1 --2- 172.21.15.55:0/492299257 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0d08066f30 0x7f0d080693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009d20 tx=0x7f0d180091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.122+0000 7f0d2759e700 1 -- 172.21.15.55:0/492299257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d28153900 msgr2=0x7f0d28142cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.122+0000 7f0d2759e700 1 --2- 172.21.15.55:0/492299257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d28153900 0x7f0d28142cc0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0091e0 tx=0x7f0d1c017000 comp rx=0 tx=0).stop 2023-10-06T04:48:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.122+0000 7f0d2759e700 1 -- 172.21.15.55:0/492299257 shutdown_connections 2023-10-06T04:48:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.122+0000 7f0d2759e700 1 --2- 172.21.15.55:0/492299257 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0d08066f30 0x7f0d080693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.122+0000 7f0d2759e700 1 --2- 172.21.15.55:0/492299257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0d28157130 0x7f0d280aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.122+0000 7f0d2759e700 1 --2- 172.21.15.55:0/492299257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0d280c9db0 0x7f0d280c5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.122+0000 7f0d2759e700 1 --2- 172.21.15.55:0/492299257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0d28153900 0x7f0d28142cc0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.122+0000 7f0d2759e700 1 -- 172.21.15.55:0/492299257 >> 172.21.15.55:0/492299257 conn(0x7f0d280cc1e0 msgr2=0x7f0d280c2820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.122+0000 7f0d2759e700 1 -- 172.21.15.55:0/492299257 shutdown_connections 2023-10-06T04:48:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.122+0000 7f0d2759e700 1 -- 172.21.15.55:0/492299257 wait complete. 2023-10-06T04:48:32.141 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T04:48:32.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.329+0000 7ffbb759e700 1 Processor -- start 2023-10-06T04:48:32.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.330+0000 7ffbb759e700 1 -- start start 2023-10-06T04:48:32.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.331+0000 7ffbb759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbb8153900 0x7ffbb80c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:32.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.331+0000 7ffbb759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ffbb80c9db0 0x7ffbb806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:32.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.331+0000 7ffbb759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffbb8157130 0x7ffbb806a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:32.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.331+0000 7ffbb759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb806e150 con 0x7ffbb8157130 2023-10-06T04:48:32.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.331+0000 7ffbb759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb80c7800 con 0x7ffbb80c9db0 2023-10-06T04:48:32.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.331+0000 7ffbb759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80c7ae0 con 0x7ffbb8153900 2023-10-06T04:48:32.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.331+0000 7ffbb6d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffbb8157130 0x7ffbb806a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:32.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.331+0000 7ffbb6d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ffbb8157130 0x7ffbb806a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58636/0 (socket says 172.21.15.55:58636) 2023-10-06T04:48:32.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.331+0000 7ffbb6d9d700 1 -- 172.21.15.55:0/2224965186 learned_addr learned my addr 172.21.15.55:0/2224965186 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:32.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.331+0000 7ffbb659c700 1 --2- 172.21.15.55:0/2224965186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbb8153900 0x7ffbb80c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:32.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.331+0000 7ffbb5d9b700 1 --2- 172.21.15.55:0/2224965186 >> v2:172.21.15.163:3300/0 conn(0x7ffbb80c9db0 0x7ffbb806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:32.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.331+0000 7ffbb6d9d700 1 -- 172.21.15.55:0/2224965186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbb8153900 msgr2=0x7ffbb80c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:32.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.332+0000 7ffbb6d9d700 1 --2- 172.21.15.55:0/2224965186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbb8153900 0x7ffbb80c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.332+0000 7ffbb6d9d700 1 -- 172.21.15.55:0/2224965186 >> v2:172.21.15.163:3300/0 conn(0x7ffbb80c9db0 msgr2=0x7ffbb806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:32.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.332+0000 7ffbb6d9d700 1 --2- 172.21.15.55:0/2224965186 >> v2:172.21.15.163:3300/0 conn(0x7ffbb80c9db0 0x7ffbb806a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.332+0000 7ffbb6d9d700 1 -- 172.21.15.55:0/2224965186 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb80c7dc0 con 0x7ffbb8157130 2023-10-06T04:48:32.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.332+0000 7ffbb6d9d700 1 --2- 172.21.15.55:0/2224965186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbb8157130 0x7ffbb806a540 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8070d60 tx=0x7ffbb0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789be59b2390b569 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:32.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.332+0000 7ffbb559a700 1 -- 172.21.15.55:0/2224965186 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000e070 con 0x7ffbb8157130 2023-10-06T04:48:32.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.332+0000 7ffbb559a700 1 -- 172.21.15.55:0/2224965186 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb000ba50 con 0x7ffbb8157130 2023-10-06T04:48:32.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.332+0000 7ffbb559a700 1 -- 172.21.15.55:0/2224965186 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb00092e0 con 0x7ffbb8157130 2023-10-06T04:48:32.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.333+0000 7ffbb759e700 1 -- 172.21.15.55:0/2224965186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbb8157130 msgr2=0x7ffbb806a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:32.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.333+0000 7ffbb759e700 1 --2- 172.21.15.55:0/2224965186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbb8157130 0x7ffbb806a540 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8070d60 tx=0x7ffbb0009840 comp rx=0 tx=0).stop 2023-10-06T04:48:32.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.333+0000 7ffbb759e700 1 -- 172.21.15.55:0/2224965186 shutdown_connections 2023-10-06T04:48:32.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.333+0000 7ffbb759e700 1 --2- 172.21.15.55:0/2224965186 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbb8157130 0x7ffbb806a540 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.333+0000 7ffbb759e700 1 --2- 172.21.15.55:0/2224965186 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbb8153900 0x7ffbb80c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.333+0000 7ffbb759e700 1 --2- 172.21.15.55:0/2224965186 >> v2:172.21.15.163:3300/0 conn(0x7ffbb80c9db0 0x7ffbb806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.333+0000 7ffbb759e700 1 -- 172.21.15.55:0/2224965186 >> 172.21.15.55:0/2224965186 conn(0x7ffbb80cc1e0 msgr2=0x7ffbb80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:32.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.333+0000 7ffbb759e700 1 -- 172.21.15.55:0/2224965186 shutdown_connections 2023-10-06T04:48:32.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.333+0000 7ffbb759e700 1 -- 172.21.15.55:0/2224965186 wait complete. 2023-10-06T04:48:32.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.334+0000 7ffbb759e700 1 Processor -- start 2023-10-06T04:48:32.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.334+0000 7ffbb759e700 1 -- start start 2023-10-06T04:48:32.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.335+0000 7ffbb759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbb80c9db0 0x7ffbb80c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:32.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.335+0000 7ffbb759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbb8153900 0x7ffbb814b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:32.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.335+0000 7ffbb759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbb8157130 0x7ffbb80aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:32.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.335+0000 7ffbb759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80afbe0 con 0x7ffbb8153900 2023-10-06T04:48:32.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.335+0000 7ffbb759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80c0f80 con 0x7ffbb8157130 2023-10-06T04:48:32.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.335+0000 7ffbb759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80c1260 con 0x7ffbb80c9db0 2023-10-06T04:48:32.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.336+0000 7ffbb5d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbb8153900 0x7ffbb814b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:32.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.336+0000 7ffbb5d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbb8153900 0x7ffbb814b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58650/0 (socket says 172.21.15.55:58650) 2023-10-06T04:48:32.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.336+0000 7ffbb5d9b700 1 -- 172.21.15.55:0/2509174094 learned_addr learned my addr 172.21.15.55:0/2509174094 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:32.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.336+0000 7ffbb659c700 1 --2- 172.21.15.55:0/2509174094 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbb80c9db0 0x7ffbb80c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:32.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.336+0000 7ffbb6d9d700 1 --2- 172.21.15.55:0/2509174094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbb8157130 0x7ffbb80aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:32.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.336+0000 7ffbb5d9b700 1 -- 172.21.15.55:0/2509174094 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbb80c9db0 msgr2=0x7ffbb80c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:32.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.336+0000 7ffbb5d9b700 1 --2- 172.21.15.55:0/2509174094 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbb80c9db0 0x7ffbb80c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.336+0000 7ffbb5d9b700 1 -- 172.21.15.55:0/2509174094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbb8157130 msgr2=0x7ffbb80aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:32.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.336+0000 7ffbb5d9b700 1 --2- 172.21.15.55:0/2509174094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbb8157130 0x7ffbb80aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.336+0000 7ffbb5d9b700 1 -- 172.21.15.55:0/2509174094 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb0009ae0 con 0x7ffbb8153900 2023-10-06T04:48:32.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.336+0000 7ffbb5d9b700 1 --2- 172.21.15.55:0/2509174094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbb8153900 0x7ffbb814b2a0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00d860 tx=0x7ffbac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:32.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.337+0000 7ffba67fc700 1 -- 172.21.15.55:0/2509174094 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac0125f0 con 0x7ffbb8153900 2023-10-06T04:48:32.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.337+0000 7ffba67fc700 1 -- 172.21.15.55:0/2509174094 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac012750 con 0x7ffbb8153900 2023-10-06T04:48:32.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.337+0000 7ffbb759e700 1 -- 172.21.15.55:0/2509174094 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb80c1540 con 0x7ffbb8153900 2023-10-06T04:48:32.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.337+0000 7ffba67fc700 1 -- 172.21.15.55:0/2509174094 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac012b10 con 0x7ffbb8153900 2023-10-06T04:48:32.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.337+0000 7ffbb759e700 1 -- 172.21.15.55:0/2509174094 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb80c1b00 con 0x7ffbb8153900 2023-10-06T04:48:32.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.338+0000 7ffbb759e700 1 -- 172.21.15.55:0/2509174094 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb806c450 con 0x7ffbb8153900 2023-10-06T04:48:32.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.339+0000 7ffba67fc700 1 -- 172.21.15.55:0/2509174094 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffbac01e460 con 0x7ffbb8153900 2023-10-06T04:48:32.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.340+0000 7ffba67fc700 1 --2- 172.21.15.55:0/2509174094 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ffb98066f30 0x7ffb980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:32.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.340+0000 7ffba67fc700 1 -- 172.21.15.55:0/2509174094 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6330+0+0 (secure 0 0 0) 0x7ffbac084d70 con 0x7ffbb8153900 2023-10-06T04:48:32.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.340+0000 7ffbb659c700 1 --2- 172.21.15.55:0/2509174094 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ffb98066f30 0x7ffb980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:32.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.341+0000 7ffbb659c700 1 --2- 172.21.15.55:0/2509174094 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ffb98066f30 0x7ffb980693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffba8006b00 tx=0x7ffba8008040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:32.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.344+0000 7ffba67fc700 1 -- 172.21.15.55:0/2509174094 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffbac055760 con 0x7ffbb8153900 2023-10-06T04:48:32.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.548+0000 7ffbb759e700 1 -- 172.21.15.55:0/2509174094 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7ffbb806dd20 con 0x7ffbb8153900 2023-10-06T04:48:32.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.556+0000 7ffba67fc700 1 -- 172.21.15.55:0/2509174094 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v50) v1 ==== 115+0+0 (secure 0 0 0) 0x7ffbac0552f0 con 0x7ffbb8153900 2023-10-06T04:48:32.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.558+0000 7ffbb759e700 1 -- 172.21.15.55:0/2509174094 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ffb98066f30 msgr2=0x7ffb980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:32.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.558+0000 7ffbb759e700 1 --2- 172.21.15.55:0/2509174094 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ffb98066f30 0x7ffb980693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffba8006b00 tx=0x7ffba8008040 comp rx=0 tx=0).stop 2023-10-06T04:48:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.558+0000 7ffbb759e700 1 -- 172.21.15.55:0/2509174094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbb8153900 msgr2=0x7ffbb814b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.558+0000 7ffbb759e700 1 --2- 172.21.15.55:0/2509174094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbb8153900 0x7ffbb814b2a0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00d860 tx=0x7ffbac008040 comp rx=0 tx=0).stop 2023-10-06T04:48:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.558+0000 7ffbb759e700 1 -- 172.21.15.55:0/2509174094 shutdown_connections 2023-10-06T04:48:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.558+0000 7ffbb759e700 1 --2- 172.21.15.55:0/2509174094 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ffb98066f30 0x7ffb980693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.558+0000 7ffbb759e700 1 --2- 172.21.15.55:0/2509174094 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ffbb8157130 0x7ffbb80aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.558+0000 7ffbb759e700 1 --2- 172.21.15.55:0/2509174094 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ffbb80c9db0 0x7ffbb80c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.558+0000 7ffbb759e700 1 --2- 172.21.15.55:0/2509174094 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ffbb8153900 0x7ffbb814b2a0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.558+0000 7ffbb759e700 1 -- 172.21.15.55:0/2509174094 >> 172.21.15.55:0/2509174094 conn(0x7ffbb80cc1e0 msgr2=0x7ffbb80ca1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.558+0000 7ffbb759e700 1 -- 172.21.15.55:0/2509174094 shutdown_connections 2023-10-06T04:48:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.559+0000 7ffbb759e700 1 -- 172.21.15.55:0/2509174094 wait complete. 2023-10-06T04:48:32.577 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T04:48:32.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.767+0000 7f0ab279d700 1 Processor -- start 2023-10-06T04:48:32.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.767+0000 7f0ab279d700 1 -- start start 2023-10-06T04:48:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.769+0000 7f0ab279d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0aac0c9db0 0x7f0aac06a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.769+0000 7f0ab279d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac157130 0x7f0aac06a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.769+0000 7f0ab279d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aac06e150 con 0x7f0aac153900 2023-10-06T04:48:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.769+0000 7f0ab279d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aac0c7610 con 0x7f0aac0c9db0 2023-10-06T04:48:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.769+0000 7f0ab279d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac0c7800 con 0x7f0aac157130 2023-10-06T04:48:32.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.769+0000 7f0ab0f9a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0aac0c9db0 0x7f0aac06a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:32.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.769+0000 7f0ab1f9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac157130 0x7f0aac06a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:32.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.769+0000 7f0ab1f9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac157130 0x7f0aac06a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:52090/0 (socket says 172.21.15.55:52090) 2023-10-06T04:48:32.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.769+0000 7f0ab179b700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f0aac153900 0x7f0aac0c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:33906/0 (socket says 172.21.15.55:33906) 2023-10-06T04:48:32.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.769+0000 7f0ab1f9c700 1 -- 172.21.15.55:0/1088198486 learned_addr learned my addr 172.21.15.55:0/1088198486 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:32.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.770+0000 7f0aa3fff700 1 -- 172.21.15.55:0/1088198486 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257934893 0 0) 0x7f0aac06e150 con 0x7f0aac153900 2023-10-06T04:48:32.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.770+0000 7f0aa3fff700 1 -- 172.21.15.55:0/1088198486 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a98003570 con 0x7f0aac153900 2023-10-06T04:48:32.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.770+0000 7f0ab1f9c700 1 -- 172.21.15.55:0/1088198486 >> v2:172.21.15.163:3300/0 conn(0x7f0aac0c9db0 msgr2=0x7f0aac06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:32.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.770+0000 7f0ab1f9c700 1 --2- 172.21.15.55:0/1088198486 >> v2:172.21.15.163:3300/0 conn(0x7f0aac0c9db0 0x7f0aac06a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.770+0000 7f0ab1f9c700 1 -- 172.21.15.55:0/1088198486 >> v1:172.21.15.163:6789/0 conn(0x7f0aac153900 legacy=0x7f0aac0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:32.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.770+0000 7f0ab1f9c700 1 -- 172.21.15.55:0/1088198486 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac0c7ae0 con 0x7f0aac157130 2023-10-06T04:48:32.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.770+0000 7f0ab0f9a700 1 --2- 172.21.15.55:0/1088198486 >> v2:172.21.15.163:3300/0 conn(0x7f0aac0c9db0 0x7f0aac06a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:32.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.770+0000 7f0ab1f9c700 1 --2- 172.21.15.55:0/1088198486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac157130 0x7f0aac06a540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400d1a0 tx=0x7f0aa400ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cda7f59c6544dad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:32.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.770+0000 7f0aa3fff700 1 -- 172.21.15.55:0/1088198486 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4010070 con 0x7f0aac157130 2023-10-06T04:48:32.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.770+0000 7f0aa3fff700 1 -- 172.21.15.55:0/1088198486 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa400db50 con 0x7f0aac157130 2023-10-06T04:48:32.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.770+0000 7f0aa3fff700 1 -- 172.21.15.55:0/1088198486 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4016400 con 0x7f0aac157130 2023-10-06T04:48:32.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.771+0000 7f0ab279d700 1 -- 172.21.15.55:0/1088198486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac157130 msgr2=0x7f0aac06a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:32.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.771+0000 7f0ab279d700 1 --2- 172.21.15.55:0/1088198486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac157130 0x7f0aac06a540 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400d1a0 tx=0x7f0aa400ba50 comp rx=0 tx=0).stop 2023-10-06T04:48:32.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.771+0000 7f0ab279d700 1 -- 172.21.15.55:0/1088198486 shutdown_connections 2023-10-06T04:48:32.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.771+0000 7f0ab279d700 1 --2- 172.21.15.55:0/1088198486 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac157130 0x7f0aac06a540 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.771+0000 7f0ab279d700 1 --2- 172.21.15.55:0/1088198486 >> v2:172.21.15.163:3300/0 conn(0x7f0aac0c9db0 0x7f0aac06a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.771+0000 7f0ab279d700 1 -- 172.21.15.55:0/1088198486 >> 172.21.15.55:0/1088198486 conn(0x7f0aac0cc1e0 msgr2=0x7f0aac0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:32.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.771+0000 7f0ab279d700 1 -- 172.21.15.55:0/1088198486 shutdown_connections 2023-10-06T04:48:32.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.771+0000 7f0ab279d700 1 -- 172.21.15.55:0/1088198486 wait complete. 2023-10-06T04:48:32.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.772+0000 7f0ab279d700 1 Processor -- start 2023-10-06T04:48:32.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.773+0000 7f0ab279d700 1 -- start start 2023-10-06T04:48:32.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.773+0000 7f0ab279d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aac0c9db0 0x7f0aac1693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab279d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0aac153900 0x7f0aac0aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab279d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac157130 0x7f0aac0ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab279d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac154eb0 con 0x7f0aac153900 2023-10-06T04:48:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab279d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac169900 con 0x7f0aac0c9db0 2023-10-06T04:48:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab279d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac0ab040 con 0x7f0aac157130 2023-10-06T04:48:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab0f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0aac153900 0x7f0aac0aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab0f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0aac153900 0x7f0aac0aa7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58656/0 (socket says 172.21.15.55:58656) 2023-10-06T04:48:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab0f9a700 1 -- 172.21.15.55:0/323876068 learned_addr learned my addr 172.21.15.55:0/323876068 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab1f9c700 1 --2- 172.21.15.55:0/323876068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac157130 0x7f0aac0ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:32.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab179b700 1 --2- 172.21.15.55:0/323876068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aac0c9db0 0x7f0aac1693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:32.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab0f9a700 1 -- 172.21.15.55:0/323876068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac157130 msgr2=0x7f0aac0ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:32.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab0f9a700 1 --2- 172.21.15.55:0/323876068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac157130 0x7f0aac0ac650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab0f9a700 1 -- 172.21.15.55:0/323876068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aac0c9db0 msgr2=0x7f0aac1693c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:32.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab0f9a700 1 --2- 172.21.15.55:0/323876068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aac0c9db0 0x7f0aac1693c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.774+0000 7f0ab0f9a700 1 -- 172.21.15.55:0/323876068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c009420 con 0x7f0aac153900 2023-10-06T04:48:32.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.775+0000 7f0ab0f9a700 1 --2- 172.21.15.55:0/323876068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0aac153900 0x7f0aac0aa7f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00ec80 tx=0x7f0a9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:32.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.775+0000 7f0aa17fa700 1 -- 172.21.15.55:0/323876068 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c00c850 con 0x7f0aac153900 2023-10-06T04:48:32.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.775+0000 7f0aa17fa700 1 -- 172.21.15.55:0/323876068 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c00c9b0 con 0x7f0aac153900 2023-10-06T04:48:32.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.775+0000 7f0ab279d700 1 -- 172.21.15.55:0/323876068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa400bcf0 con 0x7f0aac153900 2023-10-06T04:48:32.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.775+0000 7f0aa17fa700 1 -- 172.21.15.55:0/323876068 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c00cd50 con 0x7f0aac153900 2023-10-06T04:48:32.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.775+0000 7f0ab279d700 1 -- 172.21.15.55:0/323876068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aac154b40 con 0x7f0aac153900 2023-10-06T04:48:32.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.776+0000 7f0ab279d700 1 -- 172.21.15.55:0/323876068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aac1546c0 con 0x7f0aac153900 2023-10-06T04:48:32.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.777+0000 7f0aa17fa700 1 -- 172.21.15.55:0/323876068 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a9c013680 con 0x7f0aac153900 2023-10-06T04:48:32.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.778+0000 7f0aa17fa700 1 --2- 172.21.15.55:0/323876068 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0a90066f30 0x7f0a900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:32.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.778+0000 7f0aa17fa700 1 -- 172.21.15.55:0/323876068 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6330+0+0 (secure 0 0 0) 0x7f0a9c084840 con 0x7f0aac153900 2023-10-06T04:48:32.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.779+0000 7f0ab179b700 1 --2- 172.21.15.55:0/323876068 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0a90066f30 0x7f0a900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:32.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.780+0000 7f0ab179b700 1 --2- 172.21.15.55:0/323876068 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0a90066f30 0x7f0a900693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8004f70 tx=0x7f0aa8007d40 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:32.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.782+0000 7f0aa17fa700 1 -- 172.21.15.55:0/323876068 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0aac1546c0 con 0x7f0aac153900 2023-10-06T04:48:32.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.987+0000 7f0ab279d700 1 -- 172.21.15.55:0/323876068 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f0aac1546c0 con 0x7f0aac153900 2023-10-06T04:48:32.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.991+0000 7f0aa17fa700 1 -- 172.21.15.55:0/323876068 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v51) v1 ==== 120+0+0 (secure 0 0 0) 0x7f0aac1546c0 con 0x7f0aac153900 2023-10-06T04:48:32.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.993+0000 7f0ab279d700 1 -- 172.21.15.55:0/323876068 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0a90066f30 msgr2=0x7f0a900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:32.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.993+0000 7f0ab279d700 1 --2- 172.21.15.55:0/323876068 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0a90066f30 0x7f0a900693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8004f70 tx=0x7f0aa8007d40 comp rx=0 tx=0).stop 2023-10-06T04:48:32.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.993+0000 7f0ab279d700 1 -- 172.21.15.55:0/323876068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0aac153900 msgr2=0x7f0aac0aa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:32.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.993+0000 7f0ab279d700 1 --2- 172.21.15.55:0/323876068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0aac153900 0x7f0aac0aa7f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00ec80 tx=0x7f0a9c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.994+0000 7f0ab279d700 1 -- 172.21.15.55:0/323876068 shutdown_connections 2023-10-06T04:48:32.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.994+0000 7f0ab279d700 1 --2- 172.21.15.55:0/323876068 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0a90066f30 0x7f0a900693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.994+0000 7f0ab279d700 1 --2- 172.21.15.55:0/323876068 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0aac157130 0x7f0aac0ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.994+0000 7f0ab279d700 1 --2- 172.21.15.55:0/323876068 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0aac0c9db0 0x7f0aac1693c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.994+0000 7f0ab279d700 1 --2- 172.21.15.55:0/323876068 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0aac153900 0x7f0aac0aa7f0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:32.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.994+0000 7f0ab279d700 1 -- 172.21.15.55:0/323876068 >> 172.21.15.55:0/323876068 conn(0x7f0aac0cc1e0 msgr2=0x7f0aac0c0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:32.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.994+0000 7f0ab279d700 1 -- 172.21.15.55:0/323876068 shutdown_connections 2023-10-06T04:48:32.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:32.994+0000 7f0ab279d700 1 -- 172.21.15.55:0/323876068 wait complete. 2023-10-06T04:48:33.013 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-06T04:48:33.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.197+0000 7f3492b9d700 1 Processor -- start 2023-10-06T04:48:33.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.198+0000 7f3492b9d700 1 -- start start 2023-10-06T04:48:33.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.199+0000 7f3492b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f348c156310 0x7f348c0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:33.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.199+0000 7f3492b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f348c1566e0 0x7f348c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:33.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.199+0000 7f3492b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f348c0ca300 0x7f348c0ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:33.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.199+0000 7f3492b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c0ca860 con 0x7f348c1566e0 2023-10-06T04:48:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.199+0000 7f3492b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c0cab40 con 0x7f348c0ca300 2023-10-06T04:48:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.199+0000 7f3492b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0cae20 con 0x7f348c156310 2023-10-06T04:48:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.199+0000 7f349139a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f348c1566e0 0x7f348c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.199+0000 7f349139a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f348c1566e0 0x7f348c0ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57588/0 (socket says 172.21.15.55:57588) 2023-10-06T04:48:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.199+0000 7f349139a700 1 -- 172.21.15.55:0/2902335774 learned_addr learned my addr 172.21.15.55:0/2902335774 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.200+0000 7f349139a700 1 -- 172.21.15.55:0/2902335774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f348c156310 msgr2=0x7f348c0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.200+0000 7f349139a700 1 --2- 172.21.15.55:0/2902335774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f348c156310 0x7f348c0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.200+0000 7f349139a700 1 -- 172.21.15.55:0/2902335774 >> v2:172.21.15.163:3300/0 conn(0x7f348c0ca300 msgr2=0x7f348c0ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.200+0000 7f349139a700 1 --2- 172.21.15.55:0/2902335774 >> v2:172.21.15.163:3300/0 conn(0x7f348c0ca300 0x7f348c0ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.200+0000 7f349139a700 1 -- 172.21.15.55:0/2902335774 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c0cb100 con 0x7f348c1566e0 2023-10-06T04:48:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.200+0000 7f349139a700 1 --2- 172.21.15.55:0/2902335774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348c1566e0 0x7f348c0ae000 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f348c06f970 tx=0x7f347c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec8fd86dfd717e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:33.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.200+0000 7f3490b99700 1 -- 172.21.15.55:0/2902335774 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c00b860 con 0x7f348c1566e0 2023-10-06T04:48:33.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.200+0000 7f3490b99700 1 -- 172.21.15.55:0/2902335774 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c00b9c0 con 0x7f348c1566e0 2023-10-06T04:48:33.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.200+0000 7f3490b99700 1 -- 172.21.15.55:0/2902335774 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c00bd60 con 0x7f348c1566e0 2023-10-06T04:48:33.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.201+0000 7f3492b9d700 1 -- 172.21.15.55:0/2902335774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348c1566e0 msgr2=0x7f348c0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:33.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.201+0000 7f3492b9d700 1 --2- 172.21.15.55:0/2902335774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348c1566e0 0x7f348c0ae000 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f348c06f970 tx=0x7f347c009250 comp rx=0 tx=0).stop 2023-10-06T04:48:33.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.201+0000 7f3492b9d700 1 -- 172.21.15.55:0/2902335774 shutdown_connections 2023-10-06T04:48:33.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.201+0000 7f3492b9d700 1 --2- 172.21.15.55:0/2902335774 >> v2:172.21.15.163:3300/0 conn(0x7f348c0ca300 0x7f348c0ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.201+0000 7f3492b9d700 1 --2- 172.21.15.55:0/2902335774 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f348c156310 0x7f348c0c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.201+0000 7f3492b9d700 1 --2- 172.21.15.55:0/2902335774 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348c1566e0 0x7f348c0ae000 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.201+0000 7f3492b9d700 1 -- 172.21.15.55:0/2902335774 >> 172.21.15.55:0/2902335774 conn(0x7f348c0cc1f0 msgr2=0x7f348c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:33.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.201+0000 7f3492b9d700 1 -- 172.21.15.55:0/2902335774 shutdown_connections 2023-10-06T04:48:33.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.201+0000 7f3492b9d700 1 -- 172.21.15.55:0/2902335774 wait complete. 2023-10-06T04:48:33.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.202+0000 7f3492b9d700 1 Processor -- start 2023-10-06T04:48:33.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.203+0000 7f3492b9d700 1 -- start start 2023-10-06T04:48:33.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.203+0000 7f3492b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348c0ca300 0x7f348c157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:33.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.204+0000 7f3492b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f348c156310 0x7f348c0b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:33.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.204+0000 7f3492b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f348c1566e0 0x7f348c142ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:33.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.204+0000 7f3492b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c156b80 con 0x7f348c0ca300 2023-10-06T04:48:33.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.204+0000 7f3492b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c157700 con 0x7f348c1566e0 2023-10-06T04:48:33.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.204+0000 7f3492b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c156dd0 con 0x7f348c156310 2023-10-06T04:48:33.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.204+0000 7f3491b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348c0ca300 0x7f348c157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:33.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.204+0000 7f3491b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348c0ca300 0x7f348c157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57600/0 (socket says 172.21.15.55:57600) 2023-10-06T04:48:33.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.204+0000 7f3491b9b700 1 -- 172.21.15.55:0/210043687 learned_addr learned my addr 172.21.15.55:0/210043687 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:33.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.204+0000 7f3491b9b700 1 -- 172.21.15.55:0/210043687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f348c156310 msgr2=0x7f348c0b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:33.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.204+0000 7f3491b9b700 1 --2- 172.21.15.55:0/210043687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f348c156310 0x7f348c0b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.204+0000 7f3491b9b700 1 -- 172.21.15.55:0/210043687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f348c1566e0 msgr2=0x7f348c142ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:33.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.204+0000 7f3491b9b700 1 --2- 172.21.15.55:0/210043687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f348c1566e0 0x7f348c142ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.204+0000 7f3491b9b700 1 -- 172.21.15.55:0/210043687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c0094f0 con 0x7f348c0ca300 2023-10-06T04:48:33.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.204+0000 7f3491b9b700 1 --2- 172.21.15.55:0/210043687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348c0ca300 0x7f348c157b90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f348800b810 tx=0x7f34880091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:33.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.205+0000 7f3481ffb700 1 -- 172.21.15.55:0/210043687 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348800f570 con 0x7f348c0ca300 2023-10-06T04:48:33.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.205+0000 7f3481ffb700 1 -- 172.21.15.55:0/210043687 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348800f6f0 con 0x7f348c0ca300 2023-10-06T04:48:33.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.205+0000 7f3481ffb700 1 -- 172.21.15.55:0/210043687 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348800fab0 con 0x7f348c0ca300 2023-10-06T04:48:33.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.205+0000 7f3492b9d700 1 -- 172.21.15.55:0/210043687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348c1570b0 con 0x7f348c0ca300 2023-10-06T04:48:33.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.205+0000 7f3492b9d700 1 -- 172.21.15.55:0/210043687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c1580d0 con 0x7f348c0ca300 2023-10-06T04:48:33.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.206+0000 7f3492b9d700 1 -- 172.21.15.55:0/210043687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348c04c3c0 con 0x7f348c0ca300 2023-10-06T04:48:33.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.207+0000 7f3481ffb700 1 -- 172.21.15.55:0/210043687 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f348801a460 con 0x7f348c0ca300 2023-10-06T04:48:33.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.207+0000 7f3481ffb700 1 --2- 172.21.15.55:0/210043687 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f3470066f30 0x7f34700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:33.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.208+0000 7f3481ffb700 1 -- 172.21.15.55:0/210043687 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6330+0+0 (secure 0 0 0) 0x7f3488084570 con 0x7f348c0ca300 2023-10-06T04:48:33.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.208+0000 7f349139a700 1 --2- 172.21.15.55:0/210043687 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f3470066f30 0x7f34700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:33.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.211+0000 7f349139a700 1 --2- 172.21.15.55:0/210043687 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f3470066f30 0x7f34700693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f348c156b50 tx=0x7f347c017000 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:33.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.212+0000 7f3481ffb700 1 -- 172.21.15.55:0/210043687 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3488051550 con 0x7f348c0ca300 2023-10-06T04:48:33.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.494+0000 7f3492b9d700 1 -- 172.21.15.55:0/210043687 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f348c06d900 con 0x7f348c0ca300 2023-10-06T04:48:33.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.759+0000 7f3481ffb700 1 -- 172.21.15.55:0/210043687 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2428) v1 ==== 178+0+0 (secure 0 0 0) 0x7f3488054b70 con 0x7f348c0ca300 2023-10-06T04:48:33.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.761+0000 7f3492b9d700 1 -- 172.21.15.55:0/210043687 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f3470066f30 msgr2=0x7f34700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:33.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.761+0000 7f3492b9d700 1 --2- 172.21.15.55:0/210043687 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f3470066f30 0x7f34700693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f348c156b50 tx=0x7f347c017000 comp rx=0 tx=0).stop 2023-10-06T04:48:33.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.761+0000 7f3492b9d700 1 -- 172.21.15.55:0/210043687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348c0ca300 msgr2=0x7f348c157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:33.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.761+0000 7f3492b9d700 1 --2- 172.21.15.55:0/210043687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348c0ca300 0x7f348c157b90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f348800b810 tx=0x7f34880091d0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.761+0000 7f3492b9d700 1 -- 172.21.15.55:0/210043687 shutdown_connections 2023-10-06T04:48:33.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.761+0000 7f3492b9d700 1 --2- 172.21.15.55:0/210043687 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f3470066f30 0x7f34700693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.761+0000 7f3492b9d700 1 --2- 172.21.15.55:0/210043687 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f348c1566e0 0x7f348c142ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.761+0000 7f3492b9d700 1 --2- 172.21.15.55:0/210043687 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f348c0ca300 0x7f348c157b90 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.761+0000 7f3492b9d700 1 --2- 172.21.15.55:0/210043687 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f348c156310 0x7f348c0b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.761+0000 7f3492b9d700 1 -- 172.21.15.55:0/210043687 >> 172.21.15.55:0/210043687 conn(0x7f348c0cc1f0 msgr2=0x7f348c150b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:33.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.761+0000 7f3492b9d700 1 -- 172.21.15.55:0/210043687 shutdown_connections 2023-10-06T04:48:33.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.761+0000 7f3492b9d700 1 -- 172.21.15.55:0/210043687 wait complete. 2023-10-06T04:48:33.782 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2023-10-06T04:48:33.783 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-10-06T04:48:33.783 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T04:48:33.783 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-06T04:48:33.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.969+0000 7f553f59e700 1 Processor -- start 2023-10-06T04:48:33.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.969+0000 7f553f59e700 1 -- start start 2023-10-06T04:48:33.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.970+0000 7f553f59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5540159420 0x7f55400b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:33.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.970+0000 7f553f59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55400b2980 con 0x7f55401597f0 2023-10-06T04:48:33.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.970+0000 7f553f59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55400b11d0 con 0x7f554015d230 2023-10-06T04:48:33.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.970+0000 7f553f59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55400b14b0 con 0x7f5540159420 2023-10-06T04:48:33.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.970+0000 7f553dd9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f55401597f0 0x7f55400b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35902/0 (socket says 172.21.15.55:35902) 2023-10-06T04:48:33.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.971+0000 7f553dd9b700 1 -- 172.21.15.55:0/2669687763 learned_addr learned my addr 172.21.15.55:0/2669687763 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:33.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.971+0000 7f553e59c700 1 --2- 172.21.15.55:0/2669687763 >> v2:172.21.15.163:3300/0 conn(0x7f5540159420 0x7f55400b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:33.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.971+0000 7f553d59a700 1 -- 172.21.15.55:0/2669687763 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942428864 0 0) 0x7f55400b2980 con 0x7f55401597f0 2023-10-06T04:48:33.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.971+0000 7f553d59a700 1 -- 172.21.15.55:0/2669687763 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5528003570 con 0x7f55401597f0 2023-10-06T04:48:33.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.971+0000 7f553d59a700 1 -- 172.21.15.55:0/2669687763 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914988815 0 0) 0x7f55400b11d0 con 0x7f554015d230 2023-10-06T04:48:33.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.971+0000 7f553d59a700 1 -- 172.21.15.55:0/2669687763 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55400b2980 con 0x7f554015d230 2023-10-06T04:48:33.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.971+0000 7f553e59c700 1 -- 172.21.15.55:0/2669687763 >> v1:172.21.15.163:6789/0 conn(0x7f554015d230 legacy=0x7f554015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:33.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.971+0000 7f553e59c700 1 -- 172.21.15.55:0/2669687763 >> v1:172.21.15.55:6789/0 conn(0x7f55401597f0 legacy=0x7f55400b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:33.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.971+0000 7f553e59c700 1 -- 172.21.15.55:0/2669687763 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55400b1790 con 0x7f5540159420 2023-10-06T04:48:33.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.972+0000 7f553e59c700 1 --2- 172.21.15.55:0/2669687763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5540159420 0x7f55400b5f40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f55300098e0 tx=0x7f55300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc2700e6d8c04c8e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:33.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.972+0000 7f553d59a700 1 -- 172.21.15.55:0/2669687763 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530011630 con 0x7f5540159420 2023-10-06T04:48:33.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.972+0000 7f553d59a700 1 -- 172.21.15.55:0/2669687763 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5530011790 con 0x7f5540159420 2023-10-06T04:48:33.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.972+0000 7f553d59a700 1 -- 172.21.15.55:0/2669687763 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530011b30 con 0x7f5540159420 2023-10-06T04:48:33.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.972+0000 7f553f59e700 1 -- 172.21.15.55:0/2669687763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5540159420 msgr2=0x7f55400b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:33.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.972+0000 7f553f59e700 1 --2- 172.21.15.55:0/2669687763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5540159420 0x7f55400b5f40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f55300098e0 tx=0x7f55300091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.972+0000 7f553f59e700 1 -- 172.21.15.55:0/2669687763 shutdown_connections 2023-10-06T04:48:33.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.972+0000 7f553f59e700 1 --2- 172.21.15.55:0/2669687763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5540159420 0x7f55400b5f40 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.973+0000 7f553f59e700 1 -- 172.21.15.55:0/2669687763 >> 172.21.15.55:0/2669687763 conn(0x7f55400d0660 msgr2=0x7f55400d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:33.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.973+0000 7f553f59e700 1 -- 172.21.15.55:0/2669687763 shutdown_connections 2023-10-06T04:48:33.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.973+0000 7f553f59e700 1 -- 172.21.15.55:0/2669687763 wait complete. 2023-10-06T04:48:33.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.974+0000 7f553f59e700 1 Processor -- start 2023-10-06T04:48:33.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.974+0000 7f553f59e700 1 -- start start 2023-10-06T04:48:33.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.975+0000 7f553f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540159420 0x7f5540155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:33.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.975+0000 7f553f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55401597f0 0x7f55400c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:33.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.975+0000 7f553f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f554015d230 0x7f554015e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:33.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.975+0000 7f553f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55400c7b20 con 0x7f5540159420 2023-10-06T04:48:33.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.975+0000 7f553f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55400c6f10 con 0x7f55401597f0 2023-10-06T04:48:33.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.975+0000 7f553f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55400c71f0 con 0x7f554015d230 2023-10-06T04:48:33.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.975+0000 7f553e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540159420 0x7f5540155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:33.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.975+0000 7f553e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540159420 0x7f5540155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57612/0 (socket says 172.21.15.55:57612) 2023-10-06T04:48:33.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.975+0000 7f553e59c700 1 -- 172.21.15.55:0/117975321 learned_addr learned my addr 172.21.15.55:0/117975321 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:33.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.976+0000 7f553dd9b700 1 --2- 172.21.15.55:0/117975321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55401597f0 0x7f55400c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:33.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.976+0000 7f553ed9d700 1 --2- 172.21.15.55:0/117975321 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f554015d230 0x7f554015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:33.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.976+0000 7f553e59c700 1 -- 172.21.15.55:0/117975321 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f554015d230 msgr2=0x7f554015e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:33.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.976+0000 7f553e59c700 1 --2- 172.21.15.55:0/117975321 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f554015d230 0x7f554015e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.976+0000 7f553e59c700 1 -- 172.21.15.55:0/117975321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55401597f0 msgr2=0x7f55400c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:33.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.976+0000 7f553e59c700 1 --2- 172.21.15.55:0/117975321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55401597f0 0x7f55400c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:33.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.976+0000 7f553e59c700 1 -- 172.21.15.55:0/117975321 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5530009420 con 0x7f5540159420 2023-10-06T04:48:33.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.976+0000 7f553e59c700 1 --2- 172.21.15.55:0/117975321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540159420 0x7f5540155590 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f5540089340 tx=0x7f5530017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:33.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.976+0000 7f552e7fc700 1 -- 172.21.15.55:0/117975321 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553001f070 con 0x7f5540159420 2023-10-06T04:48:33.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.976+0000 7f552e7fc700 1 -- 172.21.15.55:0/117975321 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5530000bc0 con 0x7f5540159420 2023-10-06T04:48:33.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.976+0000 7f553f59e700 1 -- 172.21.15.55:0/117975321 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55400c74d0 con 0x7f5540159420 2023-10-06T04:48:33.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.976+0000 7f552e7fc700 1 -- 172.21.15.55:0/117975321 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530025420 con 0x7f5540159420 2023-10-06T04:48:33.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.977+0000 7f553f59e700 1 -- 172.21.15.55:0/117975321 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5540155ad0 con 0x7f5540159420 2023-10-06T04:48:33.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.977+0000 7f553f59e700 1 -- 172.21.15.55:0/117975321 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f554004c3c0 con 0x7f5540159420 2023-10-06T04:48:33.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.978+0000 7f552e7fc700 1 -- 172.21.15.55:0/117975321 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5530023070 con 0x7f5540159420 2023-10-06T04:48:33.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.980+0000 7f552e7fc700 1 --2- 172.21.15.55:0/117975321 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5520066f30 0x7f55200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:33.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.980+0000 7f552e7fc700 1 -- 172.21.15.55:0/117975321 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6330+0+0 (secure 0 0 0) 0x7f5530085ba0 con 0x7f5540159420 2023-10-06T04:48:33.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.980+0000 7f553dd9b700 1 --2- 172.21.15.55:0/117975321 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5520066f30 0x7f55200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:33.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.981+0000 7f553dd9b700 1 --2- 172.21.15.55:0/117975321 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5520066f30 0x7f55200693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f55400b2220 tx=0x7f5534009040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:33.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:33.984+0000 7f552e7fc700 1 -- 172.21.15.55:0/117975321 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5530052b00 con 0x7f5540159420 2023-10-06T04:48:34.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.268+0000 7f553f59e700 1 -- 172.21.15.55:0/117975321 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f55400b1a70 con 0x7f5540159420 2023-10-06T04:48:34.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.761+0000 7f552e7fc700 1 -- 172.21.15.55:0/117975321 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2429) v1 ==== 177+0+0 (secure 0 0 0) 0x7f55300150f0 con 0x7f5540159420 2023-10-06T04:48:34.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.763+0000 7f553f59e700 1 -- 172.21.15.55:0/117975321 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5520066f30 msgr2=0x7f55200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:34.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.763+0000 7f553f59e700 1 --2- 172.21.15.55:0/117975321 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5520066f30 0x7f55200693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f55400b2220 tx=0x7f5534009040 comp rx=0 tx=0).stop 2023-10-06T04:48:34.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.763+0000 7f553f59e700 1 -- 172.21.15.55:0/117975321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540159420 msgr2=0x7f5540155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:34.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.763+0000 7f553f59e700 1 --2- 172.21.15.55:0/117975321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540159420 0x7f5540155590 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f5540089340 tx=0x7f5530017000 comp rx=0 tx=0).stop 2023-10-06T04:48:34.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.763+0000 7f553f59e700 1 -- 172.21.15.55:0/117975321 shutdown_connections 2023-10-06T04:48:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.763+0000 7f553f59e700 1 --2- 172.21.15.55:0/117975321 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5520066f30 0x7f55200693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.763+0000 7f553f59e700 1 --2- 172.21.15.55:0/117975321 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f554015d230 0x7f554015e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.763+0000 7f553f59e700 1 --2- 172.21.15.55:0/117975321 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5540159420 0x7f5540155590 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.763+0000 7f553f59e700 1 --2- 172.21.15.55:0/117975321 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55401597f0 0x7f55400c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.763+0000 7f553f59e700 1 -- 172.21.15.55:0/117975321 >> 172.21.15.55:0/117975321 conn(0x7f55400d0660 msgr2=0x7f5540069a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.764+0000 7f553f59e700 1 -- 172.21.15.55:0/117975321 shutdown_connections 2023-10-06T04:48:34.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.764+0000 7f553f59e700 1 -- 172.21.15.55:0/117975321 wait complete. 2023-10-06T04:48:34.784 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T04:48:34.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.974+0000 7f0d0279d700 1 Processor -- start 2023-10-06T04:48:34.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.974+0000 7f0d0279d700 1 -- start start 2023-10-06T04:48:34.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.976+0000 7f0d0279d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0cfc159a70 0x7f0cfc0c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:34.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.976+0000 7f0d0279d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0cfc06bfa0 0x7f0cfc0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:34.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.976+0000 7f0d0279d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cfc0ca920 con 0x7f0cfc068390 2023-10-06T04:48:34.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.976+0000 7f0d0279d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cfc06af00 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.976+0000 7f0d0279d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cfc06b1e0 con 0x7f0cfc159a70 2023-10-06T04:48:34.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.976+0000 7f0d00f9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0cfc06bfa0 0x7f0cfc0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:34.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.976+0000 7f0d00f9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0cfc06bfa0 0x7f0cfc0c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57620/0 (socket says 172.21.15.55:57620) 2023-10-06T04:48:34.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.976+0000 7f0d00f9a700 1 -- 172.21.15.55:0/2327549640 learned_addr learned my addr 172.21.15.55:0/2327549640 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:34.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.976+0000 7f0d0179b700 1 --2- 172.21.15.55:0/2327549640 >> v2:172.21.15.163:3300/0 conn(0x7f0cfc159a70 0x7f0cfc0c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:34.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.976+0000 7f0d00f9a700 1 -- 172.21.15.55:0/2327549640 >> v2:172.21.15.163:3300/0 conn(0x7f0cfc159a70 msgr2=0x7f0cfc0c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:34.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.976+0000 7f0d00f9a700 1 --2- 172.21.15.55:0/2327549640 >> v2:172.21.15.163:3300/0 conn(0x7f0cfc159a70 0x7f0cfc0c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:34.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.976+0000 7f0d00f9a700 1 -- 172.21.15.55:0/2327549640 >> v1:172.21.15.163:6789/0 conn(0x7f0cfc068390 legacy=0x7f0cfc0caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:34.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.976+0000 7f0d00f9a700 1 -- 172.21.15.55:0/2327549640 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc06b4c0 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.977+0000 7f0d00f9a700 1 --2- 172.21.15.55:0/2327549640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc06bfa0 0x7f0cfc0c8830 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0640b0 tx=0x7f0cec009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74eaff400cc25df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:34.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.977+0000 7f0cf3fff700 1 -- 172.21.15.55:0/2327549640 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec005c10 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.977+0000 7f0cf3fff700 1 -- 172.21.15.55:0/2327549640 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec005d70 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.977+0000 7f0cf3fff700 1 -- 172.21.15.55:0/2327549640 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec00b840 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.977+0000 7f0d0279d700 1 -- 172.21.15.55:0/2327549640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc06bfa0 msgr2=0x7f0cfc0c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:34.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.977+0000 7f0d0279d700 1 --2- 172.21.15.55:0/2327549640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc06bfa0 0x7f0cfc0c8830 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0640b0 tx=0x7f0cec009220 comp rx=0 tx=0).stop 2023-10-06T04:48:34.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.978+0000 7f0d0279d700 1 -- 172.21.15.55:0/2327549640 shutdown_connections 2023-10-06T04:48:34.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.978+0000 7f0d0279d700 1 --2- 172.21.15.55:0/2327549640 >> v2:172.21.15.163:3300/0 conn(0x7f0cfc159a70 0x7f0cfc0c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:34.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.978+0000 7f0d0279d700 1 --2- 172.21.15.55:0/2327549640 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc06bfa0 0x7f0cfc0c8830 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:34.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.978+0000 7f0d0279d700 1 -- 172.21.15.55:0/2327549640 >> 172.21.15.55:0/2327549640 conn(0x7f0cfc0cddc0 msgr2=0x7f0cfc0c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:34.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.978+0000 7f0d0279d700 1 -- 172.21.15.55:0/2327549640 shutdown_connections 2023-10-06T04:48:34.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.978+0000 7f0d0279d700 1 -- 172.21.15.55:0/2327549640 wait complete. 2023-10-06T04:48:34.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.979+0000 7f0d0279d700 1 Processor -- start 2023-10-06T04:48:34.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.979+0000 7f0d0279d700 1 -- start start 2023-10-06T04:48:34.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.980+0000 7f0d0279d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cfc068390 0x7f0cfc154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:34.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.980+0000 7f0d0279d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc06bfa0 0x7f0cfc16db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:34.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.980+0000 7f0d0279d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cfc159a70 0x7f0cfc16e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:34.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.980+0000 7f0d0279d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc154890 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.980+0000 7f0d0279d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc153c80 con 0x7f0cfc159a70 2023-10-06T04:48:34.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.980+0000 7f0d0279d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc153f60 con 0x7f0cfc068390 2023-10-06T04:48:34.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.980+0000 7f0d00f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc06bfa0 0x7f0cfc16db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:34.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.980+0000 7f0d00f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc06bfa0 0x7f0cfc16db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57624/0 (socket says 172.21.15.55:57624) 2023-10-06T04:48:34.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.980+0000 7f0d00f9a700 1 -- 172.21.15.55:0/2985684392 learned_addr learned my addr 172.21.15.55:0/2985684392 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:34.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.981+0000 7f0d00f9a700 1 -- 172.21.15.55:0/2985684392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cfc068390 msgr2=0x7f0cfc154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:34.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.981+0000 7f0d00f9a700 1 --2- 172.21.15.55:0/2985684392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cfc068390 0x7f0cfc154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:34.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.981+0000 7f0d00f9a700 1 -- 172.21.15.55:0/2985684392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cfc159a70 msgr2=0x7f0cfc16e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:34.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.981+0000 7f0d00f9a700 1 --2- 172.21.15.55:0/2985684392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cfc159a70 0x7f0cfc16e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:34.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.981+0000 7f0d00f9a700 1 -- 172.21.15.55:0/2985684392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec009480 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.981+0000 7f0d00f9a700 1 --2- 172.21.15.55:0/2985684392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc06bfa0 0x7f0cfc16db20 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0030c0 tx=0x7f0cec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:34.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.981+0000 7f0cf17fa700 1 -- 172.21.15.55:0/2985684392 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec011b70 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.981+0000 7f0cf17fa700 1 -- 172.21.15.55:0/2985684392 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec011cf0 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.981+0000 7f0cf17fa700 1 -- 172.21.15.55:0/2985684392 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec01c670 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.981+0000 7f0d0279d700 1 -- 172.21.15.55:0/2985684392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cfc154240 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.982+0000 7f0d0279d700 1 -- 172.21.15.55:0/2985684392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cfc155260 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.982+0000 7f0d0279d700 1 -- 172.21.15.55:0/2985684392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cfc063d10 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.983+0000 7f0cf17fa700 1 -- 172.21.15.55:0/2985684392 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0cec01ca60 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.984+0000 7f0cf17fa700 1 --2- 172.21.15.55:0/2985684392 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0ce0066f30 0x7f0ce00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:34.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.984+0000 7f0cf17fa700 1 -- 172.21.15.55:0/2985684392 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6330+0+0 (secure 0 0 0) 0x7f0cec0860c0 con 0x7f0cfc06bfa0 2023-10-06T04:48:34.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.985+0000 7f0d0179b700 1 --2- 172.21.15.55:0/2985684392 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0ce0066f30 0x7f0ce00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:34.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.985+0000 7f0d0179b700 1 --2- 172.21.15.55:0/2985684392 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0ce0066f30 0x7f0ce00693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc1517d0 tx=0x7f0cf8008040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:34.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:34.989+0000 7f0cf17fa700 1 -- 172.21.15.55:0/2985684392 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cec053020 con 0x7f0cfc06bfa0 2023-10-06T04:48:35.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.190+0000 7f0d0279d700 1 -- 172.21.15.55:0/2985684392 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0cfc1549d0 con 0x7f0cfc06bfa0 2023-10-06T04:48:35.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.195+0000 7f0cf17fa700 1 -- 172.21.15.55:0/2985684392 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0cfc1549d0 con 0x7f0cfc06bfa0 2023-10-06T04:48:35.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.197+0000 7f0d0279d700 1 -- 172.21.15.55:0/2985684392 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0ce0066f30 msgr2=0x7f0ce00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:35.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.197+0000 7f0d0279d700 1 --2- 172.21.15.55:0/2985684392 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0ce0066f30 0x7f0ce00693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc1517d0 tx=0x7f0cf8008040 comp rx=0 tx=0).stop 2023-10-06T04:48:35.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.197+0000 7f0d0279d700 1 -- 172.21.15.55:0/2985684392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc06bfa0 msgr2=0x7f0cfc16db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:35.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.197+0000 7f0d0279d700 1 --2- 172.21.15.55:0/2985684392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc06bfa0 0x7f0cfc16db20 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f0cec0030c0 tx=0x7f0cec015000 comp rx=0 tx=0).stop 2023-10-06T04:48:35.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.197+0000 7f0d0279d700 1 -- 172.21.15.55:0/2985684392 shutdown_connections 2023-10-06T04:48:35.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.197+0000 7f0d0279d700 1 --2- 172.21.15.55:0/2985684392 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0ce0066f30 0x7f0ce00693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:35.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.197+0000 7f0d0279d700 1 --2- 172.21.15.55:0/2985684392 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0cfc159a70 0x7f0cfc16e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:35.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.197+0000 7f0d0279d700 1 --2- 172.21.15.55:0/2985684392 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0cfc068390 0x7f0cfc154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:35.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.197+0000 7f0d0279d700 1 --2- 172.21.15.55:0/2985684392 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0cfc06bfa0 0x7f0cfc16db20 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:35.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.197+0000 7f0d0279d700 1 -- 172.21.15.55:0/2985684392 >> 172.21.15.55:0/2985684392 conn(0x7f0cfc0cddc0 msgr2=0x7f0cfc0c7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:35.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.198+0000 7f0d0279d700 1 -- 172.21.15.55:0/2985684392 shutdown_connections 2023-10-06T04:48:35.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.198+0000 7f0d0279d700 1 -- 172.21.15.55:0/2985684392 wait complete. 2023-10-06T04:48:35.217 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:48:35.217 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:48:35.299 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:48:35.385 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:48:35.385 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T04:48:35.385 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:48:35.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.569+0000 7f19e6b9d700 1 Processor -- start 2023-10-06T04:48:35.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.569+0000 7f19e6b9d700 1 -- start start 2023-10-06T04:48:35.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e6b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f19e0150180 0x7f19e0157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:35.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e6b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19e0150550 0x7f19e00c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:35.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e6b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e006e2a0 con 0x7f19e00c7240 2023-10-06T04:48:35.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e6b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e00c76c0 con 0x7f19e0150180 2023-10-06T04:48:35.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e6b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e006b2e0 con 0x7f19e0150550 2023-10-06T04:48:35.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e5b9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f19e0150180 0x7f19e0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:35.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e5b9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f19e0150180 0x7f19e0157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57630/0 (socket says 172.21.15.55:57630) 2023-10-06T04:48:35.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e5b9b700 1 -- 172.21.15.55:0/1205033699 learned_addr learned my addr 172.21.15.55:0/1205033699 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:35.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e639c700 1 --1- 172.21.15.55:0/1205033699 >> v1:172.21.15.55:6789/0 conn(0x7f19e00c7240 0x7f19e0069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35904/0 (socket says 172.21.15.55:35904) 2023-10-06T04:48:35.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e5b9b700 1 -- 172.21.15.55:0/1205033699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19e0150550 msgr2=0x7f19e00c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:35.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e539a700 1 --2- 172.21.15.55:0/1205033699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19e0150550 0x7f19e00c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:35.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e5b9b700 1 --2- 172.21.15.55:0/1205033699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19e0150550 0x7f19e00c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:35.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e5b9b700 1 -- 172.21.15.55:0/1205033699 >> v1:172.21.15.55:6789/0 conn(0x7f19e00c7240 legacy=0x7f19e0069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:35.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e5b9b700 1 -- 172.21.15.55:0/1205033699 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e00c78e0 con 0x7f19e0150180 2023-10-06T04:48:35.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e4b99700 1 -- 172.21.15.55:0/1205033699 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941229200 0 0) 0x7f19e006e2a0 con 0x7f19e00c7240 2023-10-06T04:48:35.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.571+0000 7f19e539a700 1 --2- 172.21.15.55:0/1205033699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19e0150550 0x7f19e00c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:35.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.572+0000 7f19e5b9b700 1 --2- 172.21.15.55:0/1205033699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19e0150180 0x7f19e0157770 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009a10 tx=0x7f19dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856b5409fd09bd4a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:35.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.572+0000 7f19e4b99700 1 -- 172.21.15.55:0/1205033699 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00b7e0 con 0x7f19e0150180 2023-10-06T04:48:35.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.572+0000 7f19e4b99700 1 -- 172.21.15.55:0/1205033699 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc00b940 con 0x7f19e0150180 2023-10-06T04:48:35.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.572+0000 7f19e4b99700 1 -- 172.21.15.55:0/1205033699 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00bce0 con 0x7f19e0150180 2023-10-06T04:48:35.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.572+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1205033699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19e0150180 msgr2=0x7f19e0157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:35.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.572+0000 7f19e6b9d700 1 --2- 172.21.15.55:0/1205033699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19e0150180 0x7f19e0157770 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009a10 tx=0x7f19dc009250 comp rx=0 tx=0).stop 2023-10-06T04:48:35.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.573+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1205033699 shutdown_connections 2023-10-06T04:48:35.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.573+0000 7f19e6b9d700 1 --2- 172.21.15.55:0/1205033699 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19e0150180 0x7f19e0157770 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:35.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.573+0000 7f19e6b9d700 1 --2- 172.21.15.55:0/1205033699 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19e0150550 0x7f19e00c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:35.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.573+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1205033699 >> 172.21.15.55:0/1205033699 conn(0x7f19e00cc1f0 msgr2=0x7f19e00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:35.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.573+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1205033699 shutdown_connections 2023-10-06T04:48:35.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.573+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1205033699 wait complete. 2023-10-06T04:48:35.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.574+0000 7f19e6b9d700 1 Processor -- start 2023-10-06T04:48:35.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.574+0000 7f19e6b9d700 1 -- start start 2023-10-06T04:48:35.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.575+0000 7f19e6b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19e00c7240 0x7f19e00ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:35.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.575+0000 7f19e6b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19e0150180 0x7f19e00caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:35.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.575+0000 7f19e6b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19e0150550 0x7f19e0148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:35.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.575+0000 7f19e6b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e00ae990 con 0x7f19e00c7240 2023-10-06T04:48:35.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.575+0000 7f19e6b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e00adfc0 con 0x7f19e0150550 2023-10-06T04:48:35.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.575+0000 7f19e6b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e00ad3b0 con 0x7f19e0150180 2023-10-06T04:48:35.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.575+0000 7f19e5b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19e00c7240 0x7f19e00ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:35.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.575+0000 7f19e5b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19e00c7240 0x7f19e00ae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57644/0 (socket says 172.21.15.55:57644) 2023-10-06T04:48:35.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.575+0000 7f19e5b9b700 1 -- 172.21.15.55:0/1740091365 learned_addr learned my addr 172.21.15.55:0/1740091365 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:35.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.576+0000 7f19e539a700 1 --2- 172.21.15.55:0/1740091365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19e0150180 0x7f19e00caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:35.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.576+0000 7f19e639c700 1 --2- 172.21.15.55:0/1740091365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19e0150550 0x7f19e0148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:35.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.576+0000 7f19e5b9b700 1 -- 172.21.15.55:0/1740091365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19e0150180 msgr2=0x7f19e00caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:35.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.576+0000 7f19e5b9b700 1 --2- 172.21.15.55:0/1740091365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19e0150180 0x7f19e00caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:35.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.576+0000 7f19e5b9b700 1 -- 172.21.15.55:0/1740091365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19e0150550 msgr2=0x7f19e0148a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:35.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.576+0000 7f19e5b9b700 1 --2- 172.21.15.55:0/1740091365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19e0150550 0x7f19e0148a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:35.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.576+0000 7f19e5b9b700 1 -- 172.21.15.55:0/1740091365 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc0094f0 con 0x7f19e00c7240 2023-10-06T04:48:35.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.576+0000 7f19e5b9b700 1 --2- 172.21.15.55:0/1740091365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19e00c7240 0x7f19e00ae450 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f19e006fe40 tx=0x7f19dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:35.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.576+0000 7f19d5ffb700 1 -- 172.21.15.55:0/1740091365 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc01a420 con 0x7f19e00c7240 2023-10-06T04:48:35.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.576+0000 7f19d5ffb700 1 -- 172.21.15.55:0/1740091365 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc01a580 con 0x7f19e00c7240 2023-10-06T04:48:35.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.576+0000 7f19d5ffb700 1 -- 172.21.15.55:0/1740091365 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc01a920 con 0x7f19e00c7240 2023-10-06T04:48:35.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.576+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1740091365 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e00ad690 con 0x7f19e00c7240 2023-10-06T04:48:35.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.577+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1740091365 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e00adc50 con 0x7f19e00c7240 2023-10-06T04:48:35.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.577+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1740091365 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e004c3c0 con 0x7f19e00c7240 2023-10-06T04:48:35.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.578+0000 7f19d5ffb700 1 -- 172.21.15.55:0/1740091365 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f19dc01ad10 con 0x7f19e00c7240 2023-10-06T04:48:35.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.579+0000 7f19d5ffb700 1 --2- 172.21.15.55:0/1740091365 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f19c4066f30 0x7f19c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:35.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.579+0000 7f19d5ffb700 1 -- 172.21.15.55:0/1740091365 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6330+0+0 (secure 0 0 0) 0x7f19dc086400 con 0x7f19e00c7240 2023-10-06T04:48:35.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.580+0000 7f19e539a700 1 --2- 172.21.15.55:0/1740091365 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f19c4066f30 0x7f19c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:35.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.580+0000 7f19e539a700 1 --2- 172.21.15.55:0/1740091365 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f19c4066f30 0x7f19c40693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f19d000ac00 tx=0x7f19d0006f90 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:35.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.583+0000 7f19d5ffb700 1 -- 172.21.15.55:0/1740091365 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19dc053360 con 0x7f19e00c7240 2023-10-06T04:48:35.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.831+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1740091365 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19e00ae2a0 con 0x7f19e00c7240 2023-10-06T04:48:35.834 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.832+0000 7f19d5ffb700 1 -- 172.21.15.55:0/1740091365 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f19dc015de0 con 0x7f19e00c7240 2023-10-06T04:48:35.834 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:35.834 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":72,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7001,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2624033145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":2624033145},{"type":"v1","addr":"172.21.15.55:6835","nonce":2624033145}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":7870,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3851023466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3851023466},{"type":"v1","addr":"172.21.15.55:6834","nonce":3851023466}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":7876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6833/2633241022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2633241022},{"type":"v1","addr":"172.21.15.163:6833","nonce":2633241022}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/887855689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6834","nonce":887855689},{"type":"v1","addr":"172.21.15.163:6835","nonce":887855689}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[]} 2023-10-06T04:48:35.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.834+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1740091365 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f19c4066f30 msgr2=0x7f19c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:35.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.834+0000 7f19e6b9d700 1 --2- 172.21.15.55:0/1740091365 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f19c4066f30 0x7f19c40693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f19d000ac00 tx=0x7f19d0006f90 comp rx=0 tx=0).stop 2023-10-06T04:48:35.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.834+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1740091365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19e00c7240 msgr2=0x7f19e00ae450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:35.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.834+0000 7f19e6b9d700 1 --2- 172.21.15.55:0/1740091365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19e00c7240 0x7f19e00ae450 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f19e006fe40 tx=0x7f19dc017000 comp rx=0 tx=0).stop 2023-10-06T04:48:35.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.835+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1740091365 shutdown_connections 2023-10-06T04:48:35.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.835+0000 7f19e6b9d700 1 --2- 172.21.15.55:0/1740091365 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f19c4066f30 0x7f19c40693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:35.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.835+0000 7f19e6b9d700 1 --2- 172.21.15.55:0/1740091365 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f19e0150550 0x7f19e0148a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:35.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.835+0000 7f19e6b9d700 1 --2- 172.21.15.55:0/1740091365 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f19e00c7240 0x7f19e00ae450 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:35.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.835+0000 7f19e6b9d700 1 --2- 172.21.15.55:0/1740091365 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f19e0150180 0x7f19e00caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:35.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.835+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1740091365 >> 172.21.15.55:0/1740091365 conn(0x7f19e00cc1f0 msgr2=0x7f19e0158c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:35.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.835+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1740091365 shutdown_connections 2023-10-06T04:48:35.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:35.835+0000 7f19e6b9d700 1 -- 172.21.15.55:0/1740091365 wait complete. 2023-10-06T04:48:35.838 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 72 2023-10-06T04:48:35.854 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T04:48:35.855 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T04:48:35.855 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T04:48:35.855 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:48:35.855 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:48:35.855 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T04:48:35.855 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:48:35.856 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:48:35.856 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T04:48:35.856 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T04:48:35.856 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T04:48:35.856 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:48:35.856 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T04:48:35.856 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T04:48:35.857 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T04:48:35.857 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:48:35.871 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:48:35.871 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:48:35.872 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T04:48:35.872 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T04:48:35.874 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T04:48:35.874 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T04:48:35.876 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T04:48:35.876 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T04:48:35.915 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:48:35.915 DEBUG:teuthology.parallel:result is None 2023-10-06T04:48:35.918 INFO:tasks.ceph.mds.a:Started 2023-10-06T04:48:35.918 DEBUG:teuthology.parallel:result is None 2023-10-06T04:48:35.919 INFO:tasks.ceph.mds.d:Started 2023-10-06T04:48:35.919 DEBUG:teuthology.parallel:result is None 2023-10-06T04:48:35.922 INFO:tasks.ceph.mds.b:Started 2023-10-06T04:48:35.923 DEBUG:teuthology.parallel:result is None 2023-10-06T04:48:35.923 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T04:48:36.054 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T04:48:36.054 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-10-06T04:48:36.061 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T04:48:36.062 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2023-10-06T04:48:36.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.120+0000 7f01ff39d700 1 Processor -- start 2023-10-06T04:48:36.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.120+0000 7f01ff39d700 1 -- start start 2023-10-06T04:48:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.121+0000 7f01ff39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f01f8159420 0x7f01f80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:36.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.121+0000 7f01ff39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01f815d230 0x7f01f815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:36.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.121+0000 7f01ff39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f80b2980 con 0x7f01f81597f0 2023-10-06T04:48:36.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.121+0000 7f01ff39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f815db70 con 0x7f01f815d230 2023-10-06T04:48:36.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.121+0000 7f01ff39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f80b11d0 con 0x7f01f8159420 2023-10-06T04:48:36.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.122+0000 7f01feb9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01f815d230 0x7f01f815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:36.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.122+0000 7f01fe39b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f01f8159420 0x7f01f80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:36.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.122+0000 7f01feb9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f01f815d230 0x7f01f815d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57692/0 (socket says 172.21.15.55:57692) 2023-10-06T04:48:36.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.122+0000 7f01feb9c700 1 -- 172.21.15.55:0/1220065536 learned_addr learned my addr 172.21.15.55:0/1220065536 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:36.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.122+0000 7f01feb9c700 1 -- 172.21.15.55:0/1220065536 >> v2:172.21.15.163:3300/0 conn(0x7f01f8159420 msgr2=0x7f01f80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:36.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.122+0000 7f01feb9c700 1 --2- 172.21.15.55:0/1220065536 >> v2:172.21.15.163:3300/0 conn(0x7f01f8159420 0x7f01f80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:36.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.122+0000 7f01feb9c700 1 -- 172.21.15.55:0/1220065536 >> v1:172.21.15.163:6789/0 conn(0x7f01f81597f0 legacy=0x7f01f80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:36.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.122+0000 7f01feb9c700 1 -- 172.21.15.55:0/1220065536 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f80b14b0 con 0x7f01f815d230 2023-10-06T04:48:36.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.122+0000 7f01feb9c700 1 --2- 172.21.15.55:0/1220065536 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f815d230 0x7f01f815d630 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f01f80b2f20 tx=0x7f01f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc5777a273aa64a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:36.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.123+0000 7f01fd399700 1 -- 172.21.15.55:0/1220065536 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f000e070 con 0x7f01f815d230 2023-10-06T04:48:36.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.123+0000 7f01fd399700 1 -- 172.21.15.55:0/1220065536 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f000ba20 con 0x7f01f815d230 2023-10-06T04:48:36.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.123+0000 7f01fd399700 1 -- 172.21.15.55:0/1220065536 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f00092e0 con 0x7f01f815d230 2023-10-06T04:48:36.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.123+0000 7f01ff39d700 1 -- 172.21.15.55:0/1220065536 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f815d230 msgr2=0x7f01f815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:36.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.123+0000 7f01ff39d700 1 --2- 172.21.15.55:0/1220065536 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f815d230 0x7f01f815d630 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f01f80b2f20 tx=0x7f01f0009840 comp rx=0 tx=0).stop 2023-10-06T04:48:36.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.123+0000 7f01ff39d700 1 -- 172.21.15.55:0/1220065536 shutdown_connections 2023-10-06T04:48:36.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.123+0000 7f01ff39d700 1 --2- 172.21.15.55:0/1220065536 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f815d230 0x7f01f815d630 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:36.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.123+0000 7f01ff39d700 1 --2- 172.21.15.55:0/1220065536 >> v2:172.21.15.163:3300/0 conn(0x7f01f8159420 0x7f01f80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:36.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.123+0000 7f01ff39d700 1 -- 172.21.15.55:0/1220065536 >> 172.21.15.55:0/1220065536 conn(0x7f01f80d0660 msgr2=0x7f01f80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:36.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.123+0000 7f01ff39d700 1 -- 172.21.15.55:0/1220065536 shutdown_connections 2023-10-06T04:48:36.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.123+0000 7f01ff39d700 1 -- 172.21.15.55:0/1220065536 wait complete. 2023-10-06T04:48:36.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.125+0000 7f01ff39d700 1 Processor -- start 2023-10-06T04:48:36.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.125+0000 7f01ff39d700 1 -- start start 2023-10-06T04:48:36.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01ff39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f8159420 0x7f01f8159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:36.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01ff39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f81597f0 0x7f01f814f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:36.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01ff39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f815d230 0x7f01f815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:36.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01ff39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f815b0e0 con 0x7f01f815d230 2023-10-06T04:48:36.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01ff39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f815a4d0 con 0x7f01f81597f0 2023-10-06T04:48:36.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01ff39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f815a7b0 con 0x7f01f8159420 2023-10-06T04:48:36.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01feb9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f815d230 0x7f01f815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:36.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01feb9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f815d230 0x7f01f815b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57694/0 (socket says 172.21.15.55:57694) 2023-10-06T04:48:36.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01feb9c700 1 -- 172.21.15.55:0/1396528589 learned_addr learned my addr 172.21.15.55:0/1396528589 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:36.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01fe39b700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f8159420 0x7f01f8159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01feb9c700 1 -- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f8159420 msgr2=0x7f01f8159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01fdb9a700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f81597f0 0x7f01f814f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01feb9c700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f8159420 0x7f01f8159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01feb9c700 1 -- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f81597f0 msgr2=0x7f01f814f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01feb9c700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f81597f0 0x7f01f814f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.126+0000 7f01feb9c700 1 -- 172.21.15.55:0/1396528589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f0009ae0 con 0x7f01f815d230 2023-10-06T04:48:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.127+0000 7f01fe39b700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f8159420 0x7f01f8159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.127+0000 7f01fdb9a700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f81597f0 0x7f01f814f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.127+0000 7f01feb9c700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f815d230 0x7f01f815b570 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009880 tx=0x7f01f0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.127+0000 7f01ee7fc700 1 -- 172.21.15.55:0/1396528589 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f0008be0 con 0x7f01f815d230 2023-10-06T04:48:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.127+0000 7f01ee7fc700 1 -- 172.21.15.55:0/1396528589 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f0008d40 con 0x7f01f815d230 2023-10-06T04:48:36.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.127+0000 7f01ff39d700 1 -- 172.21.15.55:0/1396528589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f815aa90 con 0x7f01f815d230 2023-10-06T04:48:36.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.127+0000 7f01ee7fc700 1 -- 172.21.15.55:0/1396528589 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f00256b0 con 0x7f01f815d230 2023-10-06T04:48:36.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.127+0000 7f01ff39d700 1 -- 172.21.15.55:0/1396528589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f815a1e0 con 0x7f01f815d230 2023-10-06T04:48:36.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.128+0000 7f01ff39d700 1 -- 172.21.15.55:0/1396528589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f804c3c0 con 0x7f01f815d230 2023-10-06T04:48:36.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.129+0000 7f01ee7fc700 1 -- 172.21.15.55:0/1396528589 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01f0003040 con 0x7f01f815d230 2023-10-06T04:48:36.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.130+0000 7f01ee7fc700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f01dc066f30 0x7f01dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:36.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.130+0000 7f01ee7fc700 1 -- 172.21.15.55:0/1396528589 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6330+0+0 (secure 0 0 0) 0x7f01f0085e60 con 0x7f01f815d230 2023-10-06T04:48:36.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.130+0000 7f01fe39b700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f01dc066f30 0x7f01dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:36.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.131+0000 7f01fe39b700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f01dc066f30 0x7f01dc0693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f01f4006f00 tx=0x7f01f4008040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:36.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.134+0000 7f01ee7fc700 1 -- 172.21.15.55:0/1396528589 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01f0052e70 con 0x7f01f815d230 2023-10-06T04:48:36.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:36.351+0000 7f01ff39d700 1 -- 172.21.15.55:0/1396528589 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f01f80b1790 con 0x7f01f815d230 2023-10-06T04:48:37.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.691+0000 7f01ee7fc700 1 -- 172.21.15.55:0/1396528589 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v156) v1 ==== 97+0+0 (secure 0 0 0) 0x7f01f0056490 con 0x7f01f815d230 2023-10-06T04:48:37.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.694+0000 7f01ff39d700 1 -- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f01dc066f30 msgr2=0x7f01dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:37.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.694+0000 7f01ff39d700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f01dc066f30 0x7f01dc0693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f01f4006f00 tx=0x7f01f4008040 comp rx=0 tx=0).stop 2023-10-06T04:48:37.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.694+0000 7f01ff39d700 1 -- 172.21.15.55:0/1396528589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f815d230 msgr2=0x7f01f815b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:37.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.694+0000 7f01ff39d700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f815d230 0x7f01f815b570 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009880 tx=0x7f01f0018000 comp rx=0 tx=0).stop 2023-10-06T04:48:37.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.694+0000 7f01ff39d700 1 -- 172.21.15.55:0/1396528589 shutdown_connections 2023-10-06T04:48:37.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.694+0000 7f01ff39d700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f01dc066f30 0x7f01dc0693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:37.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.694+0000 7f01ff39d700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f01f815d230 0x7f01f815b570 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:37.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.694+0000 7f01ff39d700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f01f8159420 0x7f01f8159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:37.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.694+0000 7f01ff39d700 1 --2- 172.21.15.55:0/1396528589 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f01f81597f0 0x7f01f814f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:37.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.694+0000 7f01ff39d700 1 -- 172.21.15.55:0/1396528589 >> 172.21.15.55:0/1396528589 conn(0x7f01f80d0660 msgr2=0x7f01f80cf210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:37.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.695+0000 7f01ff39d700 1 -- 172.21.15.55:0/1396528589 shutdown_connections 2023-10-06T04:48:37.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.695+0000 7f01ff39d700 1 -- 172.21.15.55:0/1396528589 wait complete. 2023-10-06T04:48:37.698 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2023-10-06T04:48:37.718 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T04:48:37.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.909+0000 7fd86c9fe700 1 Processor -- start 2023-10-06T04:48:37.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.909+0000 7fd86c9fe700 1 -- start start 2023-10-06T04:48:37.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.910+0000 7fd86c9fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd868156310 0x7fd8680c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:37.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.910+0000 7fd86c9fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8681566e0 0x7fd8680ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:37.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.910+0000 7fd86c9fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8680ca300 0x7fd8680ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:37.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.910+0000 7fd86c9fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8680ca860 con 0x7fd8681566e0 2023-10-06T04:48:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.910+0000 7fd86c9fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8680cab40 con 0x7fd868156310 2023-10-06T04:48:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.910+0000 7fd86c9fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680cae20 con 0x7fd8680ca300 2023-10-06T04:48:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.910+0000 7fd86659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8681566e0 0x7fd8680ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.911+0000 7fd86659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd8681566e0 0x7fd8680ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57696/0 (socket says 172.21.15.55:57696) 2023-10-06T04:48:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.911+0000 7fd86659c700 1 -- 172.21.15.55:0/85206898 learned_addr learned my addr 172.21.15.55:0/85206898 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.911+0000 7fd86759e700 1 --2- 172.21.15.55:0/85206898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8680ca300 0x7fd8680ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.911+0000 7fd866d9d700 1 --2- 172.21.15.55:0/85206898 >> v2:172.21.15.163:3300/0 conn(0x7fd868156310 0x7fd8680c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.911+0000 7fd86659c700 1 -- 172.21.15.55:0/85206898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8680ca300 msgr2=0x7fd8680ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.911+0000 7fd86659c700 1 --2- 172.21.15.55:0/85206898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8680ca300 0x7fd8680ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.911+0000 7fd86659c700 1 -- 172.21.15.55:0/85206898 >> v2:172.21.15.163:3300/0 conn(0x7fd868156310 msgr2=0x7fd8680c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:37.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.911+0000 7fd86659c700 1 --2- 172.21.15.55:0/85206898 >> v2:172.21.15.163:3300/0 conn(0x7fd868156310 0x7fd8680c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:37.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.911+0000 7fd86659c700 1 -- 172.21.15.55:0/85206898 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8680cb100 con 0x7fd8681566e0 2023-10-06T04:48:37.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.911+0000 7fd86759e700 1 --2- 172.21.15.55:0/85206898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8680ca300 0x7fd8680ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:48:37.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.911+0000 7fd866d9d700 1 --2- 172.21.15.55:0/85206898 >> v2:172.21.15.163:3300/0 conn(0x7fd868156310 0x7fd8680c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:48:37.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.911+0000 7fd86659c700 1 --2- 172.21.15.55:0/85206898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8681566e0 0x7fd8680ae000 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fd8680cce20 tx=0x7fd85c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c657284beee9f3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:37.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.911+0000 7fd865d9b700 1 -- 172.21.15.55:0/85206898 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c00b860 con 0x7fd8681566e0 2023-10-06T04:48:37.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.912+0000 7fd865d9b700 1 -- 172.21.15.55:0/85206898 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c00b9c0 con 0x7fd8681566e0 2023-10-06T04:48:37.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.912+0000 7fd865d9b700 1 -- 172.21.15.55:0/85206898 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c00bd60 con 0x7fd8681566e0 2023-10-06T04:48:37.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.912+0000 7fd86c9fe700 1 -- 172.21.15.55:0/85206898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8681566e0 msgr2=0x7fd8680ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:37.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.912+0000 7fd86c9fe700 1 --2- 172.21.15.55:0/85206898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8681566e0 0x7fd8680ae000 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fd8680cce20 tx=0x7fd85c009250 comp rx=0 tx=0).stop 2023-10-06T04:48:37.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.912+0000 7fd86c9fe700 1 -- 172.21.15.55:0/85206898 shutdown_connections 2023-10-06T04:48:37.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.912+0000 7fd86c9fe700 1 --2- 172.21.15.55:0/85206898 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8680ca300 0x7fd8680ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:37.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.912+0000 7fd86c9fe700 1 --2- 172.21.15.55:0/85206898 >> v2:172.21.15.163:3300/0 conn(0x7fd868156310 0x7fd8680c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:37.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.912+0000 7fd86c9fe700 1 --2- 172.21.15.55:0/85206898 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8681566e0 0x7fd8680ae000 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.912+0000 7fd86c9fe700 1 -- 172.21.15.55:0/85206898 >> 172.21.15.55:0/85206898 conn(0x7fd8680cc1f0 msgr2=0x7fd8680cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.912+0000 7fd86c9fe700 1 -- 172.21.15.55:0/85206898 shutdown_connections 2023-10-06T04:48:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.912+0000 7fd86c9fe700 1 -- 172.21.15.55:0/85206898 wait complete. 2023-10-06T04:48:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.914+0000 7fd86c9fe700 1 Processor -- start 2023-10-06T04:48:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.914+0000 7fd86c9fe700 1 -- start start 2023-10-06T04:48:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.915+0000 7fd86c9fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8680ca300 0x7fd8680aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.915+0000 7fd86c9fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd868156310 0x7fd868153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.915+0000 7fd86c9fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8681566e0 0x7fd86816cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.915+0000 7fd86c9fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680af7e0 con 0x7fd8681566e0 2023-10-06T04:48:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.915+0000 7fd86c9fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680b0e30 con 0x7fd868156310 2023-10-06T04:48:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.915+0000 7fd86c9fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd868140210 con 0x7fd8680ca300 2023-10-06T04:48:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.915+0000 7fd86759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8681566e0 0x7fd86816cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.915+0000 7fd86759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8681566e0 0x7fd86816cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57708/0 (socket says 172.21.15.55:57708) 2023-10-06T04:48:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.915+0000 7fd86759e700 1 -- 172.21.15.55:0/3900005192 learned_addr learned my addr 172.21.15.55:0/3900005192 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.915+0000 7fd866d9d700 1 --2- 172.21.15.55:0/3900005192 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8680ca300 0x7fd8680aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.915+0000 7fd86659c700 1 --2- 172.21.15.55:0/3900005192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd868156310 0x7fd868153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.916+0000 7fd86759e700 1 -- 172.21.15.55:0/3900005192 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8680ca300 msgr2=0x7fd8680aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.916+0000 7fd86759e700 1 --2- 172.21.15.55:0/3900005192 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8680ca300 0x7fd8680aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.916+0000 7fd86759e700 1 -- 172.21.15.55:0/3900005192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd868156310 msgr2=0x7fd868153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.916+0000 7fd86759e700 1 --2- 172.21.15.55:0/3900005192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd868156310 0x7fd868153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.916+0000 7fd86759e700 1 -- 172.21.15.55:0/3900005192 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c0094f0 con 0x7fd8681566e0 2023-10-06T04:48:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.916+0000 7fd86759e700 1 --2- 172.21.15.55:0/3900005192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8681566e0 0x7fd86816cb80 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd860009fc0 tx=0x7fd8600099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.916+0000 7fd84effd700 1 -- 172.21.15.55:0/3900005192 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86000dbe0 con 0x7fd8681566e0 2023-10-06T04:48:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.916+0000 7fd84effd700 1 -- 172.21.15.55:0/3900005192 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86000f040 con 0x7fd8681566e0 2023-10-06T04:48:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.916+0000 7fd84effd700 1 -- 172.21.15.55:0/3900005192 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860015440 con 0x7fd8681566e0 2023-10-06T04:48:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.916+0000 7fd86c9fe700 1 -- 172.21.15.55:0/3900005192 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8681404f0 con 0x7fd8681566e0 2023-10-06T04:48:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.916+0000 7fd86c9fe700 1 -- 172.21.15.55:0/3900005192 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8680af4d0 con 0x7fd8681566e0 2023-10-06T04:48:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.918+0000 7fd86c9fe700 1 -- 172.21.15.55:0/3900005192 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8680b0c00 con 0x7fd8681566e0 2023-10-06T04:48:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.918+0000 7fd84effd700 1 -- 172.21.15.55:0/3900005192 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd86001e3b0 con 0x7fd8681566e0 2023-10-06T04:48:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.920+0000 7fd84effd700 1 --2- 172.21.15.55:0/3900005192 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fd854066f30 0x7fd8540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.920+0000 7fd84effd700 1 -- 172.21.15.55:0/3900005192 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fd860084980 con 0x7fd8681566e0 2023-10-06T04:48:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.920+0000 7fd866d9d700 1 --2- 172.21.15.55:0/3900005192 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fd854066f30 0x7fd8540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:37.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.921+0000 7fd866d9d700 1 --2- 172.21.15.55:0/3900005192 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fd854066f30 0x7fd8540693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd858006fd0 tx=0x7fd858008040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:37.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:37.923+0000 7fd84effd700 1 -- 172.21.15.55:0/3900005192 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8680b0c00 con 0x7fd8681566e0 2023-10-06T04:48:38.207 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.206+0000 7fd86c9fe700 1 -- 172.21.15.55:0/3900005192 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd8680b0c00 con 0x7fd8681566e0 2023-10-06T04:48:38.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.206+0000 7fd84effd700 1 -- 172.21.15.55:0/3900005192 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v36) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd8680b0c00 con 0x7fd8681566e0 2023-10-06T04:48:38.208 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:38.208 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T04:48:38.208 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2023-10-06T04:48:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2023-10-06T04:48:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9li2RaGRAAgRjtbw3h40vylEPjolzlpw==", 2023-10-06T04:48:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:48:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T04:48:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T04:48:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.209 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2023-10-06T04:48:38.210 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lXZu7LxAAu3QAtdDxtD/AuuqGG12sDw==", 2023-10-06T04:48:38.210 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.210 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:48:38.210 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T04:48:38.210 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T04:48:38.210 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.210 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.210 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.210 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.210 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2023-10-06T04:48:38.210 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lS22fJBAAZi6CfyLyJAFgIkaarad1mA==", 2023-10-06T04:48:38.210 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.211 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:48:38.211 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T04:48:38.211 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T04:48:38.211 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.211 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.211 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.211 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.211 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2023-10-06T04:48:38.211 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9ll0RIOhAAGtGdPv4Y2bge7xIv3eu2NQ==", 2023-10-06T04:48:38.211 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.211 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:48:38.211 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T04:48:38.212 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T04:48:38.212 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.212 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.212 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.212 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.212 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2023-10-06T04:48:38.212 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDHfx9lmTeaDRAAuycc9i7fPj8xrDkuZCAGVg==", 2023-10-06T04:48:38.212 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.212 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:48:38.212 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:48:38.212 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.212 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.213 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.213 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.213 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2023-10-06T04:48:38.213 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDKfx9lxOdXEhAA9S8tdOvhnmBeEbutqnUsZg==", 2023-10-06T04:48:38.213 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.213 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:48:38.213 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:48:38.213 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.213 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.213 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.213 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.213 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2023-10-06T04:48:38.213 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDNfx9l76I+GRAAjaj0gM7An9BCyGmfQgaDFw==", 2023-10-06T04:48:38.214 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.214 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:48:38.214 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:48:38.214 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.214 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.214 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.214 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.214 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2023-10-06T04:48:38.214 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDQfx9lsPK+HxAAzQCV0EuBenqOkUpiQW9uog==", 2023-10-06T04:48:38.214 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.214 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:48:38.214 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:48:38.215 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.215 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.215 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.215 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.215 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2023-10-06T04:48:38.215 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDWfx9lSHNrMxAAIn7awDmljO2cRtKyDalWnw==", 2023-10-06T04:48:38.215 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.215 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:48:38.215 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:48:38.215 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.215 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.215 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.216 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.216 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2023-10-06T04:48:38.216 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDZfx9lFoWuOBAAhzQOt3YChvuv0MhrspOjPw==", 2023-10-06T04:48:38.216 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.216 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:48:38.216 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:48:38.216 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.216 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.216 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.216 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.216 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2023-10-06T04:48:38.216 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDcfx9lB8iSOhAAwamSgGIXZ8owFE4FdbOOBw==", 2023-10-06T04:48:38.217 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.217 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:48:38.217 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:48:38.217 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.217 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.217 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.217 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.217 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2023-10-06T04:48:38.217 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDgfx9lUb3mABAAJ75UO2UQg2r+l+tDYVn0Dg==", 2023-10-06T04:48:38.217 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.217 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:48:38.217 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:48:38.218 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.218 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.218 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.218 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.218 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2023-10-06T04:48:38.218 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lfNvCBxAA1ZJq4XsWTZHVY0u0NdOqIw==", 2023-10-06T04:48:38.218 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.218 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:48:38.218 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2023-10-06T04:48:38.218 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T04:48:38.218 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.218 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.219 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.219 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2023-10-06T04:48:38.219 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lnvkoFhAAafxsOmItxmwUr1GJrGYJ6g==", 2023-10-06T04:48:38.219 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.219 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:48:38.219 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T04:48:38.219 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T04:48:38.219 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T04:48:38.219 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.219 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.219 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.219 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2023-10-06T04:48:38.220 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9l+9b4DhAAc5XHdikWbXsEblSE3FGScQ==", 2023-10-06T04:48:38.220 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.220 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:48:38.220 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T04:48:38.220 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T04:48:38.220 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T04:48:38.220 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.220 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.220 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.220 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2023-10-06T04:48:38.220 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9luSXrHBAAg6Ufa1Ep/EOgsQtuOPQgYg==", 2023-10-06T04:48:38.220 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.220 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:48:38.221 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T04:48:38.221 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T04:48:38.221 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T04:48:38.221 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.221 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.221 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.221 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2023-10-06T04:48:38.221 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC+fx9lEb50FxAAt0Oq28gsXtQvol7TV8trgg==", 2023-10-06T04:48:38.221 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.221 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T04:48:38.221 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T04:48:38.221 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T04:48:38.222 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.222 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.222 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.222 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.222 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2023-10-06T04:48:38.222 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lQAANEhAAsNiWPqCNEyUS3IELXSC1Cw==", 2023-10-06T04:48:38.222 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.222 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T04:48:38.222 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T04:48:38.222 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.222 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.222 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:38.223 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:38.223 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2023-10-06T04:48:38.223 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lCwU2ABAAMijxXd8hwgk3XzcdP4NZyg==", 2023-10-06T04:48:38.223 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:48:38.223 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T04:48:38.223 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T04:48:38.223 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:48:38.223 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.223 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:38.223 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2023-10-06T04:48:38.223 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T04:48:38.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.208+0000 7fd86c9fe700 1 -- 172.21.15.55:0/3900005192 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fd854066f30 msgr2=0x7fd8540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:38.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.208+0000 7fd86c9fe700 1 --2- 172.21.15.55:0/3900005192 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fd854066f30 0x7fd8540693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd858006fd0 tx=0x7fd858008040 comp rx=0 tx=0).stop 2023-10-06T04:48:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.208+0000 7fd86c9fe700 1 -- 172.21.15.55:0/3900005192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8681566e0 msgr2=0x7fd86816cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.208+0000 7fd86c9fe700 1 --2- 172.21.15.55:0/3900005192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8681566e0 0x7fd86816cb80 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd860009fc0 tx=0x7fd8600099b0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.209+0000 7fd86c9fe700 1 -- 172.21.15.55:0/3900005192 shutdown_connections 2023-10-06T04:48:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.209+0000 7fd86c9fe700 1 --2- 172.21.15.55:0/3900005192 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fd854066f30 0x7fd8540693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.209+0000 7fd86c9fe700 1 --2- 172.21.15.55:0/3900005192 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd8681566e0 0x7fd86816cb80 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.209+0000 7fd86c9fe700 1 --2- 172.21.15.55:0/3900005192 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd8680ca300 0x7fd8680aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.209+0000 7fd86c9fe700 1 --2- 172.21.15.55:0/3900005192 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd868156310 0x7fd868153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.209+0000 7fd86c9fe700 1 -- 172.21.15.55:0/3900005192 >> 172.21.15.55:0/3900005192 conn(0x7fd8680cc1f0 msgr2=0x7fd8680c2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.209+0000 7fd86c9fe700 1 -- 172.21.15.55:0/3900005192 shutdown_connections 2023-10-06T04:48:38.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.209+0000 7fd86c9fe700 1 -- 172.21.15.55:0/3900005192 wait complete. 2023-10-06T04:48:38.228 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:48:38.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.413+0000 7f911d19d700 1 Processor -- start 2023-10-06T04:48:38.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.413+0000 7f911d19d700 1 -- start start 2023-10-06T04:48:38.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.415+0000 7f911d19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91181595a0 0x7f91180b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:38.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.415+0000 7f911d19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f91180c8830 0x7f9118159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:38.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.415+0000 7f911d19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91180b1410 con 0x7f91180c6c60 2023-10-06T04:48:38.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.415+0000 7f911d19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91180b2b90 con 0x7f91180c8830 2023-10-06T04:48:38.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.415+0000 7f911d19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9118063230 con 0x7f91181595a0 2023-10-06T04:48:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.415+0000 7f9117fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91181595a0 0x7f91180b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.415+0000 7f911c99c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f91180c8830 0x7f9118159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.416+0000 7f9117fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91181595a0 0x7f91180b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:56746/0 (socket says 172.21.15.55:56746) 2023-10-06T04:48:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.416+0000 7f911c99c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f91180c8830 0x7f9118159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:45220/0 (socket says 172.21.15.55:45220) 2023-10-06T04:48:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.416+0000 7f9117fff700 1 -- 172.21.15.55:0/2845003938 learned_addr learned my addr 172.21.15.55:0/2845003938 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.416+0000 7f9117fff700 1 -- 172.21.15.55:0/2845003938 >> v2:172.21.15.163:3300/0 conn(0x7f91180c8830 msgr2=0x7f9118159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.416+0000 7f9117fff700 1 --2- 172.21.15.55:0/2845003938 >> v2:172.21.15.163:3300/0 conn(0x7f91180c8830 0x7f9118159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.416+0000 7f9117fff700 1 -- 172.21.15.55:0/2845003938 >> v1:172.21.15.163:6789/0 conn(0x7f91180c6c60 legacy=0x7f91180c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:38.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.416+0000 7f9117fff700 1 -- 172.21.15.55:0/2845003938 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91180634f0 con 0x7f91181595a0 2023-10-06T04:48:38.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.416+0000 7f9117fff700 1 --2- 172.21.15.55:0/2845003938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91181595a0 0x7f91180b18a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f91080099b0 tx=0x7f9108009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aa8c98173e8d71d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:38.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.417+0000 7f9116ffd700 1 -- 172.21.15.55:0/2845003938 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108009ea0 con 0x7f91181595a0 2023-10-06T04:48:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.417+0000 7f9116ffd700 1 -- 172.21.15.55:0/2845003938 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910800b860 con 0x7f91181595a0 2023-10-06T04:48:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.417+0000 7f9116ffd700 1 -- 172.21.15.55:0/2845003938 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910800bc00 con 0x7f91181595a0 2023-10-06T04:48:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.417+0000 7f911d19d700 1 -- 172.21.15.55:0/2845003938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91181595a0 msgr2=0x7f91180b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.417+0000 7f911d19d700 1 --2- 172.21.15.55:0/2845003938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91181595a0 0x7f91180b18a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f91080099b0 tx=0x7f9108009250 comp rx=0 tx=0).stop 2023-10-06T04:48:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.417+0000 7f911d19d700 1 -- 172.21.15.55:0/2845003938 shutdown_connections 2023-10-06T04:48:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.417+0000 7f911d19d700 1 --2- 172.21.15.55:0/2845003938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91181595a0 0x7f91180b18a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.417+0000 7f911d19d700 1 --2- 172.21.15.55:0/2845003938 >> v2:172.21.15.163:3300/0 conn(0x7f91180c8830 0x7f9118159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.417+0000 7f911d19d700 1 -- 172.21.15.55:0/2845003938 >> 172.21.15.55:0/2845003938 conn(0x7f911806ddb0 msgr2=0x7f911806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.417+0000 7f911d19d700 1 -- 172.21.15.55:0/2845003938 shutdown_connections 2023-10-06T04:48:38.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.417+0000 7f911d19d700 1 -- 172.21.15.55:0/2845003938 wait complete. 2023-10-06T04:48:38.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.419+0000 7f911d19d700 1 Processor -- start 2023-10-06T04:48:38.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.419+0000 7f911d19d700 1 -- start start 2023-10-06T04:48:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f911d19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91180c6c60 0x7f911813b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f911d19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91180c8830 0x7f911813b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f911d19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91181595a0 0x7f91180c1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f911d19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911814ef10 con 0x7f91180c6c60 2023-10-06T04:48:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f911d19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91180c5f90 con 0x7f91181595a0 2023-10-06T04:48:38.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f911d19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91180c32d0 con 0x7f91180c8830 2023-10-06T04:48:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f9117fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91180c6c60 0x7f911813b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f9117fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91180c6c60 0x7f911813b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57720/0 (socket says 172.21.15.55:57720) 2023-10-06T04:48:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f9117fff700 1 -- 172.21.15.55:0/325052732 learned_addr learned my addr 172.21.15.55:0/325052732 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f91177fe700 1 --2- 172.21.15.55:0/325052732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91180c8830 0x7f911813b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f9117fff700 1 -- 172.21.15.55:0/325052732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91180c8830 msgr2=0x7f911813b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f9117fff700 1 --2- 172.21.15.55:0/325052732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91180c8830 0x7f911813b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f9117fff700 1 -- 172.21.15.55:0/325052732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91181595a0 msgr2=0x7f91180c1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f9117fff700 1 --2- 172.21.15.55:0/325052732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91181595a0 0x7f91180c1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f9117fff700 1 -- 172.21.15.55:0/325052732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91080094f0 con 0x7f91180c6c60 2023-10-06T04:48:38.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.420+0000 7f91177fe700 1 --2- 172.21.15.55:0/325052732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91180c8830 0x7f911813b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:38.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.421+0000 7f9117fff700 1 --2- 172.21.15.55:0/325052732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91180c6c60 0x7f911813b120 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f9118063370 tx=0x7f9108017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:38.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.421+0000 7f90fffff700 1 -- 172.21.15.55:0/325052732 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910800bbe0 con 0x7f91180c6c60 2023-10-06T04:48:38.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.421+0000 7f90fffff700 1 -- 172.21.15.55:0/325052732 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910801a450 con 0x7f91180c6c60 2023-10-06T04:48:38.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.421+0000 7f911d19d700 1 -- 172.21.15.55:0/325052732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911813b660 con 0x7f91180c6c60 2023-10-06T04:48:38.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.421+0000 7f90fffff700 1 -- 172.21.15.55:0/325052732 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910801a7f0 con 0x7f91180c6c60 2023-10-06T04:48:38.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.421+0000 7f911d19d700 1 -- 172.21.15.55:0/325052732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91180c29a0 con 0x7f91180c6c60 2023-10-06T04:48:38.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.422+0000 7f911d19d700 1 -- 172.21.15.55:0/325052732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911804c3c0 con 0x7f91180c6c60 2023-10-06T04:48:38.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.423+0000 7f90fffff700 1 -- 172.21.15.55:0/325052732 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f910801ab60 con 0x7f91180c6c60 2023-10-06T04:48:38.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.424+0000 7f90fffff700 1 --2- 172.21.15.55:0/325052732 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f9104066f30 0x7f91040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:38.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.424+0000 7f90fffff700 1 -- 172.21.15.55:0/325052732 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f9108085f40 con 0x7f91180c6c60 2023-10-06T04:48:38.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.425+0000 7f91177fe700 1 --2- 172.21.15.55:0/325052732 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f9104066f30 0x7f91040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:38.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.425+0000 7f91177fe700 1 --2- 172.21.15.55:0/325052732 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f9104066f30 0x7f91040693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f91180c6970 tx=0x7f910c009040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:38.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.428+0000 7f90fffff700 1 -- 172.21.15.55:0/325052732 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9108053180 con 0x7f91180c6c60 2023-10-06T04:48:38.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.631+0000 7f911d19d700 1 -- 172.21.15.55:0/325052732 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91180637d0 con 0x7f91180c6c60 2023-10-06T04:48:38.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.632+0000 7f90fffff700 1 -- 172.21.15.55:0/325052732 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+14303 (secure 0 0 0) 0x7f91080567a0 con 0x7f91180c6c60 2023-10-06T04:48:38.634 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:38.634 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":156,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T04:48:37.684680+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":135,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:48:38.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.634+0000 7f911d19d700 1 -- 172.21.15.55:0/325052732 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f9104066f30 msgr2=0x7f91040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:38.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.634+0000 7f911d19d700 1 --2- 172.21.15.55:0/325052732 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f9104066f30 0x7f91040693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f91180c6970 tx=0x7f910c009040 comp rx=0 tx=0).stop 2023-10-06T04:48:38.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.634+0000 7f911d19d700 1 -- 172.21.15.55:0/325052732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91180c6c60 msgr2=0x7f911813b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.634+0000 7f911d19d700 1 --2- 172.21.15.55:0/325052732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91180c6c60 0x7f911813b120 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f9118063370 tx=0x7f9108017000 comp rx=0 tx=0).stop 2023-10-06T04:48:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.634+0000 7f911d19d700 1 -- 172.21.15.55:0/325052732 shutdown_connections 2023-10-06T04:48:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.634+0000 7f911d19d700 1 --2- 172.21.15.55:0/325052732 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f9104066f30 0x7f91040693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.634+0000 7f911d19d700 1 --2- 172.21.15.55:0/325052732 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f91181595a0 0x7f91180c1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.634+0000 7f911d19d700 1 --2- 172.21.15.55:0/325052732 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f91180c6c60 0x7f911813b120 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.634+0000 7f911d19d700 1 --2- 172.21.15.55:0/325052732 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f91180c8830 0x7f911813b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.634+0000 7f911d19d700 1 -- 172.21.15.55:0/325052732 >> 172.21.15.55:0/325052732 conn(0x7f911806ddb0 msgr2=0x7f91180c70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.634+0000 7f911d19d700 1 -- 172.21.15.55:0/325052732 shutdown_connections 2023-10-06T04:48:38.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.634+0000 7f911d19d700 1 -- 172.21.15.55:0/325052732 wait complete. 2023-10-06T04:48:38.654 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T04:48:38.655 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:48:38.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.843+0000 7f67f7fff700 1 Processor -- start 2023-10-06T04:48:38.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.843+0000 7f67f7fff700 1 -- start start 2023-10-06T04:48:38.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.844+0000 7f67f7fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f67f8159a70 0x7f67f80c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:38.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.844+0000 7f67f7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67f8068aa0 0x7f67f80c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:38.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.844+0000 7f67f7fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f80ca920 con 0x7f67f806bfa0 2023-10-06T04:48:38.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.845+0000 7f67f7fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f806af00 con 0x7f67f8159a70 2023-10-06T04:48:38.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.845+0000 7f67f7fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f806b1e0 con 0x7f67f8068aa0 2023-10-06T04:48:38.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.845+0000 7f67f67fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f67f806bfa0 0x7f67f8068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35932/0 (socket says 172.21.15.55:35932) 2023-10-06T04:48:38.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.845+0000 7f67f67fc700 1 -- 172.21.15.55:0/2586485434 learned_addr learned my addr 172.21.15.55:0/2586485434 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:38.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.845+0000 7f67f6ffd700 1 --2- 172.21.15.55:0/2586485434 >> v2:172.21.15.163:3300/0 conn(0x7f67f8159a70 0x7f67f80c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:38.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.845+0000 7f67f77fe700 1 --2- 172.21.15.55:0/2586485434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67f8068aa0 0x7f67f80c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:38.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.845+0000 7f67f5ffb700 1 -- 172.21.15.55:0/2586485434 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124975617 0 0) 0x7f67f80ca920 con 0x7f67f806bfa0 2023-10-06T04:48:38.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.845+0000 7f67f5ffb700 1 -- 172.21.15.55:0/2586485434 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67e0003570 con 0x7f67f806bfa0 2023-10-06T04:48:38.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.846+0000 7f67f77fe700 1 -- 172.21.15.55:0/2586485434 >> v2:172.21.15.163:3300/0 conn(0x7f67f8159a70 msgr2=0x7f67f80c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:38.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.846+0000 7f67f77fe700 1 --2- 172.21.15.55:0/2586485434 >> v2:172.21.15.163:3300/0 conn(0x7f67f8159a70 0x7f67f80c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.846+0000 7f67f77fe700 1 -- 172.21.15.55:0/2586485434 >> v1:172.21.15.55:6789/0 conn(0x7f67f806bfa0 legacy=0x7f67f8068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:38.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.846+0000 7f67f77fe700 1 -- 172.21.15.55:0/2586485434 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f806b4c0 con 0x7f67f8068aa0 2023-10-06T04:48:38.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.846+0000 7f67f77fe700 1 --2- 172.21.15.55:0/2586485434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67f8068aa0 0x7f67f80c8830 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009e60 tx=0x7f67f0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2b66bce81e9a04a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:38.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.846+0000 7f67f5ffb700 1 -- 172.21.15.55:0/2586485434 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f000e070 con 0x7f67f8068aa0 2023-10-06T04:48:38.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.846+0000 7f67f5ffb700 1 -- 172.21.15.55:0/2586485434 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f000bab0 con 0x7f67f8068aa0 2023-10-06T04:48:38.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.846+0000 7f67f5ffb700 1 -- 172.21.15.55:0/2586485434 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f000be70 con 0x7f67f8068aa0 2023-10-06T04:48:38.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.847+0000 7f67f7fff700 1 -- 172.21.15.55:0/2586485434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67f8068aa0 msgr2=0x7f67f80c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:38.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.847+0000 7f67f7fff700 1 --2- 172.21.15.55:0/2586485434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67f8068aa0 0x7f67f80c8830 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f67f0009e60 tx=0x7f67f0009780 comp rx=0 tx=0).stop 2023-10-06T04:48:38.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.847+0000 7f67f7fff700 1 -- 172.21.15.55:0/2586485434 shutdown_connections 2023-10-06T04:48:38.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.847+0000 7f67f7fff700 1 --2- 172.21.15.55:0/2586485434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67f8068aa0 0x7f67f80c8830 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.847+0000 7f67f7fff700 1 --2- 172.21.15.55:0/2586485434 >> v2:172.21.15.163:3300/0 conn(0x7f67f8159a70 0x7f67f80c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.847+0000 7f67f7fff700 1 -- 172.21.15.55:0/2586485434 >> 172.21.15.55:0/2586485434 conn(0x7f67f80cddc0 msgr2=0x7f67f80c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:38.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.847+0000 7f67f7fff700 1 -- 172.21.15.55:0/2586485434 shutdown_connections 2023-10-06T04:48:38.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.847+0000 7f67f7fff700 1 -- 172.21.15.55:0/2586485434 wait complete. 2023-10-06T04:48:38.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.848+0000 7f67f7fff700 1 Processor -- start 2023-10-06T04:48:38.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.848+0000 7f67f7fff700 1 -- start start 2023-10-06T04:48:38.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.849+0000 7f67f7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67f8068aa0 0x7f67f8150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:38.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.849+0000 7f67f7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67f806bfa0 0x7f67f8153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:38.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.849+0000 7f67f7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67f8159a70 0x7f67f80b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:38.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.849+0000 7f67f7fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f81569f0 con 0x7f67f8068aa0 2023-10-06T04:48:38.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.849+0000 7f67f7fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f80c71e0 con 0x7f67f8159a70 2023-10-06T04:48:38.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.849+0000 7f67f7fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f8157ea0 con 0x7f67f806bfa0 2023-10-06T04:48:38.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.849+0000 7f67f6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67f8068aa0 0x7f67f8150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:38.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.850+0000 7f67f6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67f8068aa0 0x7f67f8150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57724/0 (socket says 172.21.15.55:57724) 2023-10-06T04:48:38.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.850+0000 7f67f6ffd700 1 -- 172.21.15.55:0/331307618 learned_addr learned my addr 172.21.15.55:0/331307618 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:38.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.850+0000 7f67f77fe700 1 --2- 172.21.15.55:0/331307618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67f8159a70 0x7f67f80b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:38.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.850+0000 7f67f6ffd700 1 -- 172.21.15.55:0/331307618 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67f806bfa0 msgr2=0x7f67f8153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:38.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.850+0000 7f67f6ffd700 1 --2- 172.21.15.55:0/331307618 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67f806bfa0 0x7f67f8153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.850+0000 7f67f6ffd700 1 -- 172.21.15.55:0/331307618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67f8159a70 msgr2=0x7f67f80b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:38.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.850+0000 7f67f6ffd700 1 --2- 172.21.15.55:0/331307618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67f8159a70 0x7f67f80b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:38.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.850+0000 7f67f6ffd700 1 -- 172.21.15.55:0/331307618 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f0009a20 con 0x7f67f8068aa0 2023-10-06T04:48:38.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.850+0000 7f67f77fe700 1 --2- 172.21.15.55:0/331307618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67f8159a70 0x7f67f80b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:48:38.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.850+0000 7f67f6ffd700 1 --2- 172.21.15.55:0/331307618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67f8068aa0 0x7f67f8150df0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f67e800ec80 tx=0x7f67e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:38.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.850+0000 7f67deffd700 1 -- 172.21.15.55:0/331307618 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e800c9d0 con 0x7f67f8068aa0 2023-10-06T04:48:38.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.851+0000 7f67deffd700 1 -- 172.21.15.55:0/331307618 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e800cb30 con 0x7f67f8068aa0 2023-10-06T04:48:38.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.851+0000 7f67deffd700 1 -- 172.21.15.55:0/331307618 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8013470 con 0x7f67f8068aa0 2023-10-06T04:48:38.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.851+0000 7f67f7fff700 1 -- 172.21.15.55:0/331307618 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f8156430 con 0x7f67f8068aa0 2023-10-06T04:48:38.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.851+0000 7f67f7fff700 1 -- 172.21.15.55:0/331307618 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f816a690 con 0x7f67f8068aa0 2023-10-06T04:48:38.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.852+0000 7f67f7fff700 1 -- 172.21.15.55:0/331307618 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f81581b0 con 0x7f67f8068aa0 2023-10-06T04:48:38.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.853+0000 7f67deffd700 1 -- 172.21.15.55:0/331307618 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f67e80137e0 con 0x7f67f8068aa0 2023-10-06T04:48:38.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.855+0000 7f67deffd700 1 --2- 172.21.15.55:0/331307618 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f67e4066f00 0x7f67e40693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:38.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.855+0000 7f67deffd700 1 -- 172.21.15.55:0/331307618 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f67e8084950 con 0x7f67f8068aa0 2023-10-06T04:48:38.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.855+0000 7f67f67fc700 1 --2- 172.21.15.55:0/331307618 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f67e4066f00 0x7f67e40693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:38.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.856+0000 7f67f67fc700 1 --2- 172.21.15.55:0/331307618 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f67e4066f00 0x7f67e40693b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f67ec001360 tx=0x7f67ec0078a0 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:38.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:38.859+0000 7f67deffd700 1 -- 172.21.15.55:0/331307618 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67e8051b90 con 0x7f67f8068aa0 2023-10-06T04:48:39.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.061+0000 7f67f7fff700 1 -- 172.21.15.55:0/331307618 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f67f816a4c0 con 0x7f67f8068aa0 2023-10-06T04:48:39.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.062+0000 7f67deffd700 1 -- 172.21.15.55:0/331307618 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v156) v1 ==== 92+0+10 (secure 0 0 0) 0x7f67e80551b0 con 0x7f67f8068aa0 2023-10-06T04:48:39.064 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T04:48:39.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.064+0000 7f67f7fff700 1 -- 172.21.15.55:0/331307618 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f67e4066f00 msgr2=0x7f67e40693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.064+0000 7f67f7fff700 1 --2- 172.21.15.55:0/331307618 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f67e4066f00 0x7f67e40693b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f67ec001360 tx=0x7f67ec0078a0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.064+0000 7f67f7fff700 1 -- 172.21.15.55:0/331307618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67f8068aa0 msgr2=0x7f67f8150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.064+0000 7f67f7fff700 1 --2- 172.21.15.55:0/331307618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67f8068aa0 0x7f67f8150df0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f67e800ec80 tx=0x7f67e80091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.064+0000 7f67f7fff700 1 -- 172.21.15.55:0/331307618 shutdown_connections 2023-10-06T04:48:39.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.064+0000 7f67f7fff700 1 --2- 172.21.15.55:0/331307618 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f67e4066f00 0x7f67e40693b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.064+0000 7f67f7fff700 1 --2- 172.21.15.55:0/331307618 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f67f8159a70 0x7f67f80b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.064+0000 7f67f7fff700 1 --2- 172.21.15.55:0/331307618 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67f8068aa0 0x7f67f8150df0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.064+0000 7f67f7fff700 1 --2- 172.21.15.55:0/331307618 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67f806bfa0 0x7f67f8153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.064+0000 7f67f7fff700 1 -- 172.21.15.55:0/331307618 >> 172.21.15.55:0/331307618 conn(0x7f67f80cddc0 msgr2=0x7f67f8068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:39.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.064+0000 7f67f7fff700 1 -- 172.21.15.55:0/331307618 shutdown_connections 2023-10-06T04:48:39.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.064+0000 7f67f7fff700 1 -- 172.21.15.55:0/331307618 wait complete. 2023-10-06T04:48:39.084 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:48:39.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.272+0000 7f5dd239d700 1 Processor -- start 2023-10-06T04:48:39.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.273+0000 7f5dd239d700 1 -- start start 2023-10-06T04:48:39.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.274+0000 7f5dd239d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5dcc0cb1a0 0x7f5dcc15a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:39.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.274+0000 7f5dd239d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5dcc0ae0d0 0x7f5dcc0ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:39.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.274+0000 7f5dd239d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dcc06ebd0 con 0x7f5dcc15a8a0 2023-10-06T04:48:39.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.274+0000 7f5dd239d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dcc15b2e0 con 0x7f5dcc0ae0d0 2023-10-06T04:48:39.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.274+0000 7f5dd239d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dcc0aea10 con 0x7f5dcc0cb1a0 2023-10-06T04:48:39.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.274+0000 7f5dd1b9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5dcc0ae0d0 0x7f5dcc0ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:39.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.274+0000 7f5dd1b9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5dcc0ae0d0 0x7f5dcc0ae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57730/0 (socket says 172.21.15.55:57730) 2023-10-06T04:48:39.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.274+0000 7f5dd1b9c700 1 -- 172.21.15.55:0/3190246482 learned_addr learned my addr 172.21.15.55:0/3190246482 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:39.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.274+0000 7f5dd139b700 1 --2- 172.21.15.55:0/3190246482 >> v2:172.21.15.163:3300/0 conn(0x7f5dcc0cb1a0 0x7f5dcc15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:39.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.275+0000 7f5dd1b9c700 1 -- 172.21.15.55:0/3190246482 >> v2:172.21.15.163:3300/0 conn(0x7f5dcc0cb1a0 msgr2=0x7f5dcc15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:39.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.275+0000 7f5dd1b9c700 1 --2- 172.21.15.55:0/3190246482 >> v2:172.21.15.163:3300/0 conn(0x7f5dcc0cb1a0 0x7f5dcc15a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.275+0000 7f5dd1b9c700 1 -- 172.21.15.55:0/3190246482 >> v1:172.21.15.163:6789/0 conn(0x7f5dcc15a8a0 legacy=0x7f5dcc06f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.275+0000 7f5dd1b9c700 1 -- 172.21.15.55:0/3190246482 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc15ad20 con 0x7f5dcc0ae0d0 2023-10-06T04:48:39.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.275+0000 7f5dd1b9c700 1 --2- 172.21.15.55:0/3190246482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dcc0ae0d0 0x7f5dcc0ae4d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0cce10 tx=0x7f5dc4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ef990e7c966bef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:39.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.275+0000 7f5dc3fff700 1 -- 172.21.15.55:0/3190246482 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc400e070 con 0x7f5dcc0ae0d0 2023-10-06T04:48:39.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.275+0000 7f5dc3fff700 1 -- 172.21.15.55:0/3190246482 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc400bab0 con 0x7f5dcc0ae0d0 2023-10-06T04:48:39.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.275+0000 7f5dc3fff700 1 -- 172.21.15.55:0/3190246482 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40092e0 con 0x7f5dcc0ae0d0 2023-10-06T04:48:39.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.276+0000 7f5dd239d700 1 -- 172.21.15.55:0/3190246482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dcc0ae0d0 msgr2=0x7f5dcc0ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.276+0000 7f5dd239d700 1 --2- 172.21.15.55:0/3190246482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dcc0ae0d0 0x7f5dcc0ae4d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0cce10 tx=0x7f5dc4009840 comp rx=0 tx=0).stop 2023-10-06T04:48:39.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.276+0000 7f5dd239d700 1 -- 172.21.15.55:0/3190246482 shutdown_connections 2023-10-06T04:48:39.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.276+0000 7f5dd239d700 1 --2- 172.21.15.55:0/3190246482 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dcc0ae0d0 0x7f5dcc0ae4d0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.276+0000 7f5dd239d700 1 --2- 172.21.15.55:0/3190246482 >> v2:172.21.15.163:3300/0 conn(0x7f5dcc0cb1a0 0x7f5dcc15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.276+0000 7f5dd239d700 1 -- 172.21.15.55:0/3190246482 >> 172.21.15.55:0/3190246482 conn(0x7f5dcc0cc1e0 msgr2=0x7f5dcc0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:39.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.276+0000 7f5dd239d700 1 -- 172.21.15.55:0/3190246482 shutdown_connections 2023-10-06T04:48:39.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.276+0000 7f5dd239d700 1 -- 172.21.15.55:0/3190246482 wait complete. 2023-10-06T04:48:39.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.277+0000 7f5dd239d700 1 Processor -- start 2023-10-06T04:48:39.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.278+0000 7f5dd239d700 1 -- start start 2023-10-06T04:48:39.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.278+0000 7f5dd239d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5dcc0ae0d0 0x7f5dcc156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:39.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.278+0000 7f5dd239d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dcc0cb1a0 0x7f5dcc144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:39.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd239d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dcc0c27b0 0x7f5dcc0c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:39.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd239d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc16cf50 con 0x7f5dcc0c27b0 2023-10-06T04:48:39.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd239d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc152770 con 0x7f5dcc0cb1a0 2023-10-06T04:48:39.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd239d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dcc16d1a0 con 0x7f5dcc0ae0d0 2023-10-06T04:48:39.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd1b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dcc0c27b0 0x7f5dcc0c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:39.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd1b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dcc0c27b0 0x7f5dcc0c76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57742/0 (socket says 172.21.15.55:57742) 2023-10-06T04:48:39.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd1b9c700 1 -- 172.21.15.55:0/2914410303 learned_addr learned my addr 172.21.15.55:0/2914410303 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:39.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd139b700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5dcc0ae0d0 0x7f5dcc156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:39.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd0b9a700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dcc0cb1a0 0x7f5dcc144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:39.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd1b9c700 1 -- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5dcc0ae0d0 msgr2=0x7f5dcc156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd1b9c700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5dcc0ae0d0 0x7f5dcc156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd1b9c700 1 -- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dcc0cb1a0 msgr2=0x7f5dcc144d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd1b9c700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dcc0cb1a0 0x7f5dcc144d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd1b9c700 1 -- 172.21.15.55:0/2914410303 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc4009ae0 con 0x7f5dcc0c27b0 2023-10-06T04:48:39.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.279+0000 7f5dd0b9a700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dcc0cb1a0 0x7f5dcc144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:39.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.280+0000 7f5dd1b9c700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dcc0c27b0 0x7f5dcc0c76a0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009880 tx=0x7f5dc4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:39.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.280+0000 7f5dc17fa700 1 -- 172.21.15.55:0/2914410303 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc40032f0 con 0x7f5dcc0c27b0 2023-10-06T04:48:39.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.280+0000 7f5dc17fa700 1 -- 172.21.15.55:0/2914410303 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc4008e80 con 0x7f5dcc0c27b0 2023-10-06T04:48:39.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.280+0000 7f5dc17fa700 1 -- 172.21.15.55:0/2914410303 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc4026690 con 0x7f5dcc0c27b0 2023-10-06T04:48:39.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.280+0000 7f5dd239d700 1 -- 172.21.15.55:0/2914410303 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc16d480 con 0x7f5dcc0c27b0 2023-10-06T04:48:39.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.280+0000 7f5dd239d700 1 -- 172.21.15.55:0/2914410303 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dcc16da40 con 0x7f5dcc0c27b0 2023-10-06T04:48:39.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.281+0000 7f5dd239d700 1 -- 172.21.15.55:0/2914410303 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dcc06c450 con 0x7f5dcc0c27b0 2023-10-06T04:48:39.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.282+0000 7f5dc17fa700 1 -- 172.21.15.55:0/2914410303 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5dc4003040 con 0x7f5dcc0c27b0 2023-10-06T04:48:39.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.283+0000 7f5dc17fa700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5db0066f30 0x7f5db00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:39.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.283+0000 7f5dc17fa700 1 -- 172.21.15.55:0/2914410303 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f5dc4085f20 con 0x7f5dcc0c27b0 2023-10-06T04:48:39.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.283+0000 7f5dd139b700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5db0066f30 0x7f5db00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:39.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.284+0000 7f5dd139b700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5db0066f30 0x7f5db00693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009d40 tx=0x7f5dc8008040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:39.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.287+0000 7f5dc17fa700 1 -- 172.21.15.55:0/2914410303 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5dc4053160 con 0x7f5dcc0c27b0 2023-10-06T04:48:39.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.489+0000 7f5dd239d700 1 -- 172.21.15.55:0/2914410303 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5dcc15b000 con 0x7f5dcc0c27b0 2023-10-06T04:48:39.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.489+0000 7f5dc17fa700 1 -- 172.21.15.55:0/2914410303 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v156) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5dc4056780 con 0x7f5dcc0c27b0 2023-10-06T04:48:39.491 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T04:48:39.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.491+0000 7f5dd239d700 1 -- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5db0066f30 msgr2=0x7f5db00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.491+0000 7f5dd239d700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5db0066f30 0x7f5db00693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009d40 tx=0x7f5dc8008040 comp rx=0 tx=0).stop 2023-10-06T04:48:39.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.491+0000 7f5dd239d700 1 -- 172.21.15.55:0/2914410303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dcc0c27b0 msgr2=0x7f5dcc0c76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.491+0000 7f5dd239d700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dcc0c27b0 0x7f5dcc0c76a0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4009880 tx=0x7f5dc4018000 comp rx=0 tx=0).stop 2023-10-06T04:48:39.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.492+0000 7f5dd239d700 1 -- 172.21.15.55:0/2914410303 shutdown_connections 2023-10-06T04:48:39.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.492+0000 7f5dd239d700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5db0066f30 0x7f5db00693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.492+0000 7f5dd239d700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5dcc0c27b0 0x7f5dcc0c76a0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.492+0000 7f5dd239d700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5dcc0ae0d0 0x7f5dcc156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.492+0000 7f5dd239d700 1 --2- 172.21.15.55:0/2914410303 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5dcc0cb1a0 0x7f5dcc144d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.492+0000 7f5dd239d700 1 -- 172.21.15.55:0/2914410303 >> 172.21.15.55:0/2914410303 conn(0x7f5dcc0cc1e0 msgr2=0x7f5dcc0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:39.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.492+0000 7f5dd239d700 1 -- 172.21.15.55:0/2914410303 shutdown_connections 2023-10-06T04:48:39.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.492+0000 7f5dd239d700 1 -- 172.21.15.55:0/2914410303 wait complete. 2023-10-06T04:48:39.511 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T04:48:39.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.697+0000 7f84c9b9d700 1 Processor -- start 2023-10-06T04:48:39.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.697+0000 7f84c9b9d700 1 -- start start 2023-10-06T04:48:39.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.698+0000 7f84c9b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84c4150550 0x7f84c4157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:39.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.698+0000 7f84c9b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c406e2a0 con 0x7f84c40c7410 2023-10-06T04:48:39.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.698+0000 7f84c9b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c406b2e0 con 0x7f84c4150180 2023-10-06T04:48:39.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.698+0000 7f84c9b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c40c78e0 con 0x7f84c4150550 2023-10-06T04:48:39.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.698+0000 7f84c939c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f84c40c7410 0x7f84c4069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35934/0 (socket says 172.21.15.55:35934) 2023-10-06T04:48:39.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.698+0000 7f84c939c700 1 -- 172.21.15.55:0/2125463901 learned_addr learned my addr 172.21.15.55:0/2125463901 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:39.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.699+0000 7f84c3fff700 1 --2- 172.21.15.55:0/2125463901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84c4150550 0x7f84c4157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:39.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.699+0000 7f84c37fe700 1 -- 172.21.15.55:0/2125463901 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612519435 0 0) 0x7f84c406e2a0 con 0x7f84c40c7410 2023-10-06T04:48:39.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.699+0000 7f84c37fe700 1 -- 172.21.15.55:0/2125463901 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84b4003570 con 0x7f84c40c7410 2023-10-06T04:48:39.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.699+0000 7f84c37fe700 1 -- 172.21.15.55:0/2125463901 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326530155 0 0) 0x7f84c406b2e0 con 0x7f84c4150180 2023-10-06T04:48:39.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.699+0000 7f84c37fe700 1 -- 172.21.15.55:0/2125463901 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84c406e2a0 con 0x7f84c4150180 2023-10-06T04:48:39.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.699+0000 7f84c3fff700 1 -- 172.21.15.55:0/2125463901 >> v1:172.21.15.163:6789/0 conn(0x7f84c4150180 legacy=0x7f84c40c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.699+0000 7f84c3fff700 1 -- 172.21.15.55:0/2125463901 >> v1:172.21.15.55:6789/0 conn(0x7f84c40c7410 legacy=0x7f84c4069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.699+0000 7f84c37fe700 1 -- 172.21.15.55:0/2125463901 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3819068695 0 0) 0x7f84b4003570 con 0x7f84c40c7410 2023-10-06T04:48:39.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.699+0000 7f84c3fff700 1 -- 172.21.15.55:0/2125463901 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c40c7bc0 con 0x7f84c4150550 2023-10-06T04:48:39.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.700+0000 7f84c3fff700 1 --2- 172.21.15.55:0/2125463901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84c4150550 0x7f84c4157770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f84b000aaf0 tx=0x7f84b000a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc569b85c2a1551d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:39.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.700+0000 7f84c37fe700 1 -- 172.21.15.55:0/2125463901 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0017f0 con 0x7f84c4150550 2023-10-06T04:48:39.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.700+0000 7f84c37fe700 1 -- 172.21.15.55:0/2125463901 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b000c660 con 0x7f84c4150550 2023-10-06T04:48:39.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.700+0000 7f84c37fe700 1 -- 172.21.15.55:0/2125463901 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b000ca00 con 0x7f84c4150550 2023-10-06T04:48:39.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.700+0000 7f84c9b9d700 1 -- 172.21.15.55:0/2125463901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84c4150550 msgr2=0x7f84c4157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.700+0000 7f84c9b9d700 1 --2- 172.21.15.55:0/2125463901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84c4150550 0x7f84c4157770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f84b000aaf0 tx=0x7f84b000a330 comp rx=0 tx=0).stop 2023-10-06T04:48:39.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.700+0000 7f84c9b9d700 1 -- 172.21.15.55:0/2125463901 shutdown_connections 2023-10-06T04:48:39.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.700+0000 7f84c9b9d700 1 --2- 172.21.15.55:0/2125463901 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84c4150550 0x7f84c4157770 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.700+0000 7f84c9b9d700 1 -- 172.21.15.55:0/2125463901 >> 172.21.15.55:0/2125463901 conn(0x7f84c40cc1f0 msgr2=0x7f84c40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:39.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.701+0000 7f84c9b9d700 1 -- 172.21.15.55:0/2125463901 shutdown_connections 2023-10-06T04:48:39.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.701+0000 7f84c9b9d700 1 -- 172.21.15.55:0/2125463901 wait complete. 2023-10-06T04:48:39.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.702+0000 7f84c9b9d700 1 Processor -- start 2023-10-06T04:48:39.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.702+0000 7f84c9b9d700 1 -- start start 2023-10-06T04:48:39.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.703+0000 7f84c9b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84c40c7410 0x7f84c40aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:39.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.703+0000 7f84c9b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84c4150180 0x7f84c40aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:39.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.703+0000 7f84c9b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84c4150550 0x7f84c40b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:39.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.703+0000 7f84c9b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c40b0770 con 0x7f84c40c7410 2023-10-06T04:48:39.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.703+0000 7f84c9b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c40b0a50 con 0x7f84c4150550 2023-10-06T04:48:39.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.703+0000 7f84c9b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c40b0d30 con 0x7f84c4150180 2023-10-06T04:48:39.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.703+0000 7f84c8b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84c40c7410 0x7f84c40aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:39.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.703+0000 7f84c8b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84c40c7410 0x7f84c40aa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57752/0 (socket says 172.21.15.55:57752) 2023-10-06T04:48:39.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.703+0000 7f84c8b9b700 1 -- 172.21.15.55:0/336640891 learned_addr learned my addr 172.21.15.55:0/336640891 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:39.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.703+0000 7f84c3fff700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84c4150180 0x7f84c40aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:39.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.703+0000 7f84c939c700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84c4150550 0x7f84c40b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:39.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.704+0000 7f84c8b9b700 1 -- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84c4150180 msgr2=0x7f84c40aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.704+0000 7f84c8b9b700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84c4150180 0x7f84c40aca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.704+0000 7f84c8b9b700 1 -- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84c4150550 msgr2=0x7f84c40b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.704+0000 7f84c8b9b700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84c4150550 0x7f84c40b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.704+0000 7f84c8b9b700 1 -- 172.21.15.55:0/336640891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b000a5d0 con 0x7f84c40c7410 2023-10-06T04:48:39.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.704+0000 7f84c939c700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84c4150550 0x7f84c40b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:48:39.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.704+0000 7f84c8b9b700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84c40c7410 0x7f84c40aa680 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c790 tx=0x7f84b8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:39.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.704+0000 7f84c0ff9700 1 -- 172.21.15.55:0/336640891 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8010510 con 0x7f84c40c7410 2023-10-06T04:48:39.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.704+0000 7f84c0ff9700 1 -- 172.21.15.55:0/336640891 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b8010670 con 0x7f84c40c7410 2023-10-06T04:48:39.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.704+0000 7f84c9b9d700 1 -- 172.21.15.55:0/336640891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c40b1010 con 0x7f84c40c7410 2023-10-06T04:48:39.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.704+0000 7f84c0ff9700 1 -- 172.21.15.55:0/336640891 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8010a30 con 0x7f84c40c7410 2023-10-06T04:48:39.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.704+0000 7f84c9b9d700 1 -- 172.21.15.55:0/336640891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c40aabc0 con 0x7f84c40c7410 2023-10-06T04:48:39.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.705+0000 7f84c9b9d700 1 -- 172.21.15.55:0/336640891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c404c3c0 con 0x7f84c40c7410 2023-10-06T04:48:39.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.706+0000 7f84c0ff9700 1 -- 172.21.15.55:0/336640891 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f84b801b3b0 con 0x7f84c40c7410 2023-10-06T04:48:39.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.707+0000 7f84c0ff9700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f84a8066f30 0x7f84a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:39.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.707+0000 7f84c0ff9700 1 -- 172.21.15.55:0/336640891 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f84b8083de0 con 0x7f84c40c7410 2023-10-06T04:48:39.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.708+0000 7f84c3fff700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f84a8066f30 0x7f84a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:39.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.709+0000 7f84c3fff700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f84a8066f30 0x7f84a80693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84b000aa10 tx=0x7f84b0019000 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:39.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.712+0000 7f84c0ff9700 1 -- 172.21.15.55:0/336640891 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84b80510c0 con 0x7f84c40c7410 2023-10-06T04:48:39.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.919+0000 7f84c9b9d700 1 -- 172.21.15.55:0/336640891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f84c40b08b0 con 0x7f84c40c7410 2023-10-06T04:48:39.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.919+0000 7f84c0ff9700 1 -- 172.21.15.55:0/336640891 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v156) v1 ==== 84+0+126 (secure 0 0 0) 0x7f84c40b08b0 con 0x7f84c40c7410 2023-10-06T04:48:39.922 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:39.922 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T04:48:39.922 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:39.922 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2023-10-06T04:48:39.922 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": "rbd" 2023-10-06T04:48:39.922 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:48:39.922 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:39.923 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 2, 2023-10-06T04:48:39.923 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2023-10-06T04:48:39.923 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:39.923 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T04:48:39.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.922+0000 7f84c9b9d700 1 -- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f84a8066f30 msgr2=0x7f84a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.922+0000 7f84c9b9d700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f84a8066f30 0x7f84a80693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f84b000aa10 tx=0x7f84b0019000 comp rx=0 tx=0).stop 2023-10-06T04:48:39.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.922+0000 7f84c9b9d700 1 -- 172.21.15.55:0/336640891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84c40c7410 msgr2=0x7f84c40aa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:39.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.922+0000 7f84c9b9d700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84c40c7410 0x7f84c40aa680 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c790 tx=0x7f84b8007e10 comp rx=0 tx=0).stop 2023-10-06T04:48:39.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.922+0000 7f84c9b9d700 1 -- 172.21.15.55:0/336640891 shutdown_connections 2023-10-06T04:48:39.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.922+0000 7f84c9b9d700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f84a8066f30 0x7f84a80693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.922+0000 7f84c9b9d700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f84c4150550 0x7f84c40b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.922+0000 7f84c9b9d700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f84c40c7410 0x7f84c40aa680 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.922+0000 7f84c9b9d700 1 --2- 172.21.15.55:0/336640891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f84c4150180 0x7f84c40aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:39.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.922+0000 7f84c9b9d700 1 -- 172.21.15.55:0/336640891 >> 172.21.15.55:0/336640891 conn(0x7f84c40cc1f0 msgr2=0x7f84c40696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:39.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.922+0000 7f84c9b9d700 1 -- 172.21.15.55:0/336640891 shutdown_connections 2023-10-06T04:48:39.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:39.922+0000 7f84c9b9d700 1 -- 172.21.15.55:0/336640891 wait complete. 2023-10-06T04:48:39.941 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T04:48:39.942 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T04:48:40.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.132+0000 7fd9e519d700 1 Processor -- start 2023-10-06T04:48:40.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.132+0000 7fd9e519d700 1 -- start start 2023-10-06T04:48:40.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.133+0000 7fd9e519d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9e00cca20 0x7fd9e00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:40.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.133+0000 7fd9e519d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e00cfa00 con 0x7fd9e00b03c0 2023-10-06T04:48:40.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.133+0000 7fd9e519d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e00cb580 con 0x7fd9e00b0790 2023-10-06T04:48:40.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.133+0000 7fd9e519d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e00cf160 con 0x7fd9e00cca20 2023-10-06T04:48:40.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.134+0000 7fd9e499c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9e00cca20 0x7fd9e00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:40.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.134+0000 7fd9e499c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd9e00cca20 0x7fd9e00cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57760/0 (socket says 172.21.15.55:57760) 2023-10-06T04:48:40.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.134+0000 7fd9e499c700 1 -- 172.21.15.55:0/1944213412 learned_addr learned my addr 172.21.15.55:0/1944213412 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:40.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.134+0000 7fd9dffff700 1 --1- 172.21.15.55:0/1944213412 >> v1:172.21.15.55:6789/0 conn(0x7fd9e00b03c0 0x7fd9e00b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35936/0 (socket says 172.21.15.55:35936) 2023-10-06T04:48:40.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.134+0000 7fd9e499c700 1 -- 172.21.15.55:0/1944213412 >> v1:172.21.15.163:6789/0 conn(0x7fd9e00b0790 legacy=0x7fd9e01591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:40.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.134+0000 7fd9e499c700 1 -- 172.21.15.55:0/1944213412 >> v1:172.21.15.55:6789/0 conn(0x7fd9e00b03c0 legacy=0x7fd9e00b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:40.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.134+0000 7fd9e499c700 1 -- 172.21.15.55:0/1944213412 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00cf440 con 0x7fd9e00cca20 2023-10-06T04:48:40.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.134+0000 7fd9e499c700 1 --2- 172.21.15.55:0/1944213412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00cca20 0x7fd9e00cea70 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009f50 tx=0x7fd9d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8584417fc5c181da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:40.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.135+0000 7fd9deffd700 1 -- 172.21.15.55:0/1944213412 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d800e070 con 0x7fd9e00cca20 2023-10-06T04:48:40.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.135+0000 7fd9deffd700 1 -- 172.21.15.55:0/1944213412 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d800ba70 con 0x7fd9e00cca20 2023-10-06T04:48:40.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.135+0000 7fd9deffd700 1 -- 172.21.15.55:0/1944213412 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80092e0 con 0x7fd9e00cca20 2023-10-06T04:48:40.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.135+0000 7fd9e519d700 1 -- 172.21.15.55:0/1944213412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00cca20 msgr2=0x7fd9e00cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:40.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.135+0000 7fd9e519d700 1 --2- 172.21.15.55:0/1944213412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00cca20 0x7fd9e00cea70 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009f50 tx=0x7fd9d8009840 comp rx=0 tx=0).stop 2023-10-06T04:48:40.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.135+0000 7fd9e519d700 1 -- 172.21.15.55:0/1944213412 shutdown_connections 2023-10-06T04:48:40.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.135+0000 7fd9e519d700 1 --2- 172.21.15.55:0/1944213412 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00cca20 0x7fd9e00cea70 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:40.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.135+0000 7fd9e519d700 1 -- 172.21.15.55:0/1944213412 >> 172.21.15.55:0/1944213412 conn(0x7fd9e0063230 msgr2=0x7fd9e0063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:40.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.135+0000 7fd9e519d700 1 -- 172.21.15.55:0/1944213412 shutdown_connections 2023-10-06T04:48:40.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.135+0000 7fd9e519d700 1 -- 172.21.15.55:0/1944213412 wait complete. 2023-10-06T04:48:40.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.137+0000 7fd9e519d700 1 Processor -- start 2023-10-06T04:48:40.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.137+0000 7fd9e519d700 1 -- start start 2023-10-06T04:48:40.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.138+0000 7fd9e519d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00b03c0 0x7fd9e015a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:40.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.138+0000 7fd9e519d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00b0790 0x7fd9e00c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:40.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.138+0000 7fd9e519d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e00cca20 0x7fd9e00c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:40.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.138+0000 7fd9e519d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00ca240 con 0x7fd9e00b0790 2023-10-06T04:48:40.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.138+0000 7fd9e519d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00c9630 con 0x7fd9e00cca20 2023-10-06T04:48:40.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.138+0000 7fd9e519d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e00c9910 con 0x7fd9e00b03c0 2023-10-06T04:48:40.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.138+0000 7fd9df7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00b0790 0x7fd9e00c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:40.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.138+0000 7fd9df7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00b0790 0x7fd9e00c88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57768/0 (socket says 172.21.15.55:57768) 2023-10-06T04:48:40.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.138+0000 7fd9df7fe700 1 -- 172.21.15.55:0/2172588526 learned_addr learned my addr 172.21.15.55:0/2172588526 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:40.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.138+0000 7fd9dffff700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00b03c0 0x7fd9e015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:40.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.138+0000 7fd9e499c700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e00cca20 0x7fd9e00c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:40.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.139+0000 7fd9df7fe700 1 -- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00b03c0 msgr2=0x7fd9e015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:40.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.139+0000 7fd9df7fe700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00b03c0 0x7fd9e015a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:40.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.139+0000 7fd9df7fe700 1 -- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e00cca20 msgr2=0x7fd9e00c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:40.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.139+0000 7fd9df7fe700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e00cca20 0x7fd9e00c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:40.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.139+0000 7fd9df7fe700 1 -- 172.21.15.55:0/2172588526 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d8009ae0 con 0x7fd9e00b0790 2023-10-06T04:48:40.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.139+0000 7fd9e499c700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e00cca20 0x7fd9e00c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:48:40.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.139+0000 7fd9df7fe700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00b0790 0x7fd9e00c88d0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400c590 tx=0x7fd9d400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:40.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.139+0000 7fd9c7fff700 1 -- 172.21.15.55:0/2172588526 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4011590 con 0x7fd9e00b0790 2023-10-06T04:48:40.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.139+0000 7fd9c7fff700 1 -- 172.21.15.55:0/2172588526 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d4011710 con 0x7fd9e00b0790 2023-10-06T04:48:40.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.139+0000 7fd9e519d700 1 -- 172.21.15.55:0/2172588526 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e00c9bf0 con 0x7fd9e00b0790 2023-10-06T04:48:40.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.139+0000 7fd9c7fff700 1 -- 172.21.15.55:0/2172588526 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4011ad0 con 0x7fd9e00b0790 2023-10-06T04:48:40.143 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.139+0000 7fd9e519d700 1 -- 172.21.15.55:0/2172588526 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e00c9350 con 0x7fd9e00b0790 2023-10-06T04:48:40.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.140+0000 7fd9e519d700 1 -- 172.21.15.55:0/2172588526 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e00ca3f0 con 0x7fd9e00b0790 2023-10-06T04:48:40.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.141+0000 7fd9c7fff700 1 -- 172.21.15.55:0/2172588526 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9d401b3b0 con 0x7fd9e00b0790 2023-10-06T04:48:40.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.142+0000 7fd9c7fff700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9cc066f30 0x7fd9cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:40.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.142+0000 7fd9c7fff700 1 -- 172.21.15.55:0/2172588526 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fd9d4083a70 con 0x7fd9e00b0790 2023-10-06T04:48:40.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.142+0000 7fd9dffff700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9cc066f30 0x7fd9cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:40.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.143+0000 7fd9dffff700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9cc066f30 0x7fd9cc0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0004fd0 tx=0x7fd9d0007eb0 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.146+0000 7fd9c7fff700 1 -- 172.21.15.55:0/2172588526 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9e00ca3f0 con 0x7fd9e00b0790 2023-10-06T04:48:40.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.348+0000 7fd9e519d700 1 -- 172.21.15.55:0/2172588526 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd9e004c3c0 con 0x7fd9e00b0790 2023-10-06T04:48:40.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.802+0000 7fd9c7fff700 1 -- 172.21.15.55:0/2172588526 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v157) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd9d4017070 con 0x7fd9e00b0790 2023-10-06T04:48:40.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.806+0000 7fd9e519d700 1 -- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9cc066f30 msgr2=0x7fd9cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.806+0000 7fd9e519d700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9cc066f30 0x7fd9cc0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0004fd0 tx=0x7fd9d0007eb0 comp rx=0 tx=0).stop 2023-10-06T04:48:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.806+0000 7fd9e519d700 1 -- 172.21.15.55:0/2172588526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00b0790 msgr2=0x7fd9e00c88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.806+0000 7fd9e519d700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00b0790 0x7fd9e00c88d0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400c590 tx=0x7fd9d400c8a0 comp rx=0 tx=0).stop 2023-10-06T04:48:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.806+0000 7fd9e519d700 1 -- 172.21.15.55:0/2172588526 shutdown_connections 2023-10-06T04:48:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.806+0000 7fd9e519d700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fd9cc066f30 0x7fd9cc0693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:40.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.806+0000 7fd9e519d700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd9e00cca20 0x7fd9e00c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:40.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.806+0000 7fd9e519d700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd9e00b03c0 0x7fd9e015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:40.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.806+0000 7fd9e519d700 1 --2- 172.21.15.55:0/2172588526 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd9e00b0790 0x7fd9e00c88d0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:40.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.806+0000 7fd9e519d700 1 -- 172.21.15.55:0/2172588526 >> 172.21.15.55:0/2172588526 conn(0x7fd9e0063230 msgr2=0x7fd9e0154a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:40.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.806+0000 7fd9e519d700 1 -- 172.21.15.55:0/2172588526 shutdown_connections 2023-10-06T04:48:40.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:40.806+0000 7fd9e519d700 1 -- 172.21.15.55:0/2172588526 wait complete. 2023-10-06T04:48:40.811 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2023-10-06T04:48:40.828 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T04:48:41.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.020+0000 7fa25559d700 1 Processor -- start 2023-10-06T04:48:41.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.020+0000 7fa25559d700 1 -- start start 2023-10-06T04:48:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.021+0000 7fa25559d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2501563b0 0x7fa2500ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.021+0000 7fa25559d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa2500ae9c0 0x7fa2500cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.021+0000 7fa25559d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2500cbe50 con 0x7fa250156780 2023-10-06T04:48:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.021+0000 7fa25559d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2500cb370 con 0x7fa2500ae9c0 2023-10-06T04:48:41.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.021+0000 7fa25559d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500cb650 con 0x7fa2501563b0 2023-10-06T04:48:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.022+0000 7fa254d9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa2500ae9c0 0x7fa2500cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.022+0000 7fa24ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2501563b0 0x7fa2500ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.022+0000 7fa254d9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa2500ae9c0 0x7fa2500cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:45280/0 (socket says 172.21.15.55:45280) 2023-10-06T04:48:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.022+0000 7fa254d9c700 1 -- 172.21.15.55:0/3152144978 learned_addr learned my addr 172.21.15.55:0/3152144978 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.022+0000 7fa24effd700 1 -- 172.21.15.55:0/3152144978 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215085729 0 0) 0x7fa2500cbe50 con 0x7fa250156780 2023-10-06T04:48:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.022+0000 7fa24ffff700 1 -- 172.21.15.55:0/3152144978 >> v2:172.21.15.163:3300/0 conn(0x7fa2500ae9c0 msgr2=0x7fa2500cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.022+0000 7fa24ffff700 1 --2- 172.21.15.55:0/3152144978 >> v2:172.21.15.163:3300/0 conn(0x7fa2500ae9c0 0x7fa2500cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.022+0000 7fa24ffff700 1 -- 172.21.15.55:0/3152144978 >> v1:172.21.15.163:6789/0 conn(0x7fa250156780 legacy=0x7fa2500ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.022+0000 7fa24ffff700 1 -- 172.21.15.55:0/3152144978 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2500c6860 con 0x7fa2501563b0 2023-10-06T04:48:41.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.022+0000 7fa254d9c700 1 --2- 172.21.15.55:0/3152144978 >> v2:172.21.15.163:3300/0 conn(0x7fa2500ae9c0 0x7fa2500cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:41.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.023+0000 7fa24ffff700 1 --2- 172.21.15.55:0/3152144978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2501563b0 0x7fa2500ca7d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa2400098e0 tx=0x7fa2400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b25ede9c1b58f13f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:41.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.023+0000 7fa24effd700 1 -- 172.21.15.55:0/3152144978 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24000b640 con 0x7fa2501563b0 2023-10-06T04:48:41.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.023+0000 7fa24effd700 1 -- 172.21.15.55:0/3152144978 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24000b7a0 con 0x7fa2501563b0 2023-10-06T04:48:41.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.023+0000 7fa24effd700 1 -- 172.21.15.55:0/3152144978 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24000bb40 con 0x7fa2501563b0 2023-10-06T04:48:41.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.024+0000 7fa25559d700 1 -- 172.21.15.55:0/3152144978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2501563b0 msgr2=0x7fa2500ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:41.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.024+0000 7fa25559d700 1 --2- 172.21.15.55:0/3152144978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2501563b0 0x7fa2500ca7d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa2400098e0 tx=0x7fa2400091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:41.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.024+0000 7fa25559d700 1 -- 172.21.15.55:0/3152144978 shutdown_connections 2023-10-06T04:48:41.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.024+0000 7fa25559d700 1 --2- 172.21.15.55:0/3152144978 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2501563b0 0x7fa2500ca7d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:41.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.024+0000 7fa25559d700 1 --2- 172.21.15.55:0/3152144978 >> v2:172.21.15.163:3300/0 conn(0x7fa2500ae9c0 0x7fa2500cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:41.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.024+0000 7fa25559d700 1 -- 172.21.15.55:0/3152144978 >> 172.21.15.55:0/3152144978 conn(0x7fa250068e20 msgr2=0x7fa2500cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:41.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.024+0000 7fa25559d700 1 -- 172.21.15.55:0/3152144978 shutdown_connections 2023-10-06T04:48:41.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.024+0000 7fa25559d700 1 -- 172.21.15.55:0/3152144978 wait complete. 2023-10-06T04:48:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.025+0000 7fa25559d700 1 Processor -- start 2023-10-06T04:48:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.025+0000 7fa25559d700 1 -- start start 2023-10-06T04:48:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.026+0000 7fa25559d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2500ae9c0 0x7fa2500c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.026+0000 7fa25559d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2501563b0 0x7fa250135d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.026+0000 7fa25559d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa250156780 0x7fa250136270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.026+0000 7fa25559d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500af9b0 con 0x7fa2501563b0 2023-10-06T04:48:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.026+0000 7fa25559d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500afc90 con 0x7fa250156780 2023-10-06T04:48:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.026+0000 7fa25559d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500aff70 con 0x7fa2500ae9c0 2023-10-06T04:48:41.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.027+0000 7fa24f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2501563b0 0x7fa250135d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:41.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.027+0000 7fa254d9c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa250156780 0x7fa250136270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:41.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.027+0000 7fa24ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2500ae9c0 0x7fa2500c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:41.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.027+0000 7fa24f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2501563b0 0x7fa250135d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57780/0 (socket says 172.21.15.55:57780) 2023-10-06T04:48:41.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.027+0000 7fa24f7fe700 1 -- 172.21.15.55:0/1374136269 learned_addr learned my addr 172.21.15.55:0/1374136269 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:41.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.027+0000 7fa24f7fe700 1 -- 172.21.15.55:0/1374136269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2500ae9c0 msgr2=0x7fa2500c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:41.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.027+0000 7fa24f7fe700 1 --2- 172.21.15.55:0/1374136269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2500ae9c0 0x7fa2500c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:41.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.027+0000 7fa24f7fe700 1 -- 172.21.15.55:0/1374136269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa250156780 msgr2=0x7fa250136270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:41.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.027+0000 7fa24f7fe700 1 --2- 172.21.15.55:0/1374136269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa250156780 0x7fa250136270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:41.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.027+0000 7fa24f7fe700 1 -- 172.21.15.55:0/1374136269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa248009ae0 con 0x7fa2501563b0 2023-10-06T04:48:41.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.027+0000 7fa24f7fe700 1 --2- 172.21.15.55:0/1374136269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2501563b0 0x7fa250135d30 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fa24400c530 tx=0x7fa24400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:41.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.028+0000 7fa237fff700 1 -- 172.21.15.55:0/1374136269 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244011590 con 0x7fa2501563b0 2023-10-06T04:48:41.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.028+0000 7fa237fff700 1 -- 172.21.15.55:0/1374136269 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa244011710 con 0x7fa2501563b0 2023-10-06T04:48:41.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.028+0000 7fa25559d700 1 -- 172.21.15.55:0/1374136269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa240009420 con 0x7fa2501563b0 2023-10-06T04:48:41.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.028+0000 7fa237fff700 1 -- 172.21.15.55:0/1374136269 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa244011ad0 con 0x7fa2501563b0 2023-10-06T04:48:41.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.028+0000 7fa25559d700 1 -- 172.21.15.55:0/1374136269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2500b0530 con 0x7fa2501563b0 2023-10-06T04:48:41.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.029+0000 7fa25559d700 1 -- 172.21.15.55:0/1374136269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2500b0750 con 0x7fa2501563b0 2023-10-06T04:48:41.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.030+0000 7fa237fff700 1 -- 172.21.15.55:0/1374136269 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa24401b3b0 con 0x7fa2501563b0 2023-10-06T04:48:41.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.031+0000 7fa237fff700 1 --2- 172.21.15.55:0/1374136269 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa23c066f30 0x7fa23c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:41.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.031+0000 7fa237fff700 1 -- 172.21.15.55:0/1374136269 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5968+0+0 (secure 0 0 0) 0x7fa244083f80 con 0x7fa2501563b0 2023-10-06T04:48:41.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.031+0000 7fa24ffff700 1 --2- 172.21.15.55:0/1374136269 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa23c066f30 0x7fa23c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:41.033 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.032+0000 7fa24ffff700 1 --2- 172.21.15.55:0/1374136269 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa23c066f30 0x7fa23c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa2400093f0 tx=0x7fa240017000 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:41.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.034+0000 7fa237fff700 1 -- 172.21.15.55:0/1374136269 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa244051070 con 0x7fa2501563b0 2023-10-06T04:48:41.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.233+0000 7fa25559d700 1 -- 172.21.15.55:0/1374136269 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa25004c3c0 con 0x7fa2501563b0 2023-10-06T04:48:41.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.812+0000 7fa237fff700 1 -- 172.21.15.55:0/1374136269 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v158) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa2440170d0 con 0x7fa2501563b0 2023-10-06T04:48:41.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.823+0000 7fa25559d700 1 -- 172.21.15.55:0/1374136269 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa23c066f30 msgr2=0x7fa23c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:41.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.823+0000 7fa25559d700 1 --2- 172.21.15.55:0/1374136269 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa23c066f30 0x7fa23c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa2400093f0 tx=0x7fa240017000 comp rx=0 tx=0).stop 2023-10-06T04:48:41.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.823+0000 7fa25559d700 1 -- 172.21.15.55:0/1374136269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2501563b0 msgr2=0x7fa250135d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:41.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.823+0000 7fa25559d700 1 --2- 172.21.15.55:0/1374136269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2501563b0 0x7fa250135d30 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fa24400c530 tx=0x7fa24400c840 comp rx=0 tx=0).stop 2023-10-06T04:48:41.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.824+0000 7fa25559d700 1 -- 172.21.15.55:0/1374136269 shutdown_connections 2023-10-06T04:48:41.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.824+0000 7fa25559d700 1 --2- 172.21.15.55:0/1374136269 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa23c066f30 0x7fa23c0693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:41.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.824+0000 7fa25559d700 1 --2- 172.21.15.55:0/1374136269 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa250156780 0x7fa250136270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:41.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.824+0000 7fa25559d700 1 --2- 172.21.15.55:0/1374136269 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa2500ae9c0 0x7fa2500c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:41.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.824+0000 7fa25559d700 1 --2- 172.21.15.55:0/1374136269 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa2501563b0 0x7fa250135d30 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:41.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.825+0000 7fa25559d700 1 -- 172.21.15.55:0/1374136269 >> 172.21.15.55:0/1374136269 conn(0x7fa250068e20 msgr2=0x7fa25014ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:41.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.827+0000 7fa25559d700 1 -- 172.21.15.55:0/1374136269 shutdown_connections 2023-10-06T04:48:41.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:41.829+0000 7fa25559d700 1 -- 172.21.15.55:0/1374136269 wait complete. 2023-10-06T04:48:41.837 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2023-10-06T04:48:41.854 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T04:48:42.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.042+0000 7f3b6339d700 1 Processor -- start 2023-10-06T04:48:42.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.042+0000 7f3b6339d700 1 -- start start 2023-10-06T04:48:42.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.044+0000 7f3b6339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b5c159a60 0x7f3b5c0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:42.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.044+0000 7f3b6339d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3b5c0ccc00 0x7f3b5c069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:42.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.044+0000 7f3b6339d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5c0c9cb0 con 0x7f3b5c06a020 2023-10-06T04:48:42.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.044+0000 7f3b6339d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5c06dfc0 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.044+0000 7f3b6339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0c9f30 con 0x7f3b5c159a60 2023-10-06T04:48:42.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.044+0000 7f3b62b9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3b5c06a020 0x7f3b5c0cb230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35938/0 (socket says 172.21.15.55:35938) 2023-10-06T04:48:42.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.044+0000 7f3b62b9c700 1 -- 172.21.15.55:0/1928636055 learned_addr learned my addr 172.21.15.55:0/1928636055 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:42.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.044+0000 7f3b61b9a700 1 --2- 172.21.15.55:0/1928636055 >> v2:172.21.15.163:3300/0 conn(0x7f3b5c0ccc00 0x7f3b5c069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:42.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.044+0000 7f3b6239b700 1 --2- 172.21.15.55:0/1928636055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b5c159a60 0x7f3b5c0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:42.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.045+0000 7f3b61399700 1 -- 172.21.15.55:0/1928636055 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3216772016 0 0) 0x7f3b5c0c9cb0 con 0x7f3b5c06a020 2023-10-06T04:48:42.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.045+0000 7f3b61399700 1 -- 172.21.15.55:0/1928636055 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b48003570 con 0x7f3b5c06a020 2023-10-06T04:48:42.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.045+0000 7f3b61b9a700 1 -- 172.21.15.55:0/1928636055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b5c159a60 msgr2=0x7f3b5c0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:42.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.045+0000 7f3b61b9a700 1 --2- 172.21.15.55:0/1928636055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b5c159a60 0x7f3b5c0c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:42.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.045+0000 7f3b61b9a700 1 -- 172.21.15.55:0/1928636055 >> v1:172.21.15.55:6789/0 conn(0x7f3b5c06a020 legacy=0x7f3b5c0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:42.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.045+0000 7f3b61b9a700 1 -- 172.21.15.55:0/1928636055 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c0ca470 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.045+0000 7f3b61b9a700 1 --2- 172.21.15.55:0/1928636055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b5c0ccc00 0x7f3b5c069ae0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0098e0 tx=0x7f3b4c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=902d8fc6fdc0c3f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:42.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.045+0000 7f3b61399700 1 -- 172.21.15.55:0/1928636055 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c011630 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.045+0000 7f3b61399700 1 -- 172.21.15.55:0/1928636055 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c011790 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.045+0000 7f3b61399700 1 -- 172.21.15.55:0/1928636055 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c011b30 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.046+0000 7f3b6339d700 1 -- 172.21.15.55:0/1928636055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b5c0ccc00 msgr2=0x7f3b5c069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:42.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.046+0000 7f3b6339d700 1 --2- 172.21.15.55:0/1928636055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b5c0ccc00 0x7f3b5c069ae0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0098e0 tx=0x7f3b4c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:42.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.046+0000 7f3b6339d700 1 -- 172.21.15.55:0/1928636055 shutdown_connections 2023-10-06T04:48:42.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.046+0000 7f3b6339d700 1 --2- 172.21.15.55:0/1928636055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b5c159a60 0x7f3b5c0c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:42.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.046+0000 7f3b6339d700 1 --2- 172.21.15.55:0/1928636055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b5c0ccc00 0x7f3b5c069ae0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:42.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.046+0000 7f3b6339d700 1 -- 172.21.15.55:0/1928636055 >> 172.21.15.55:0/1928636055 conn(0x7f3b5c06baa0 msgr2=0x7f3b5c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:42.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.046+0000 7f3b6339d700 1 -- 172.21.15.55:0/1928636055 shutdown_connections 2023-10-06T04:48:42.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.046+0000 7f3b6339d700 1 -- 172.21.15.55:0/1928636055 wait complete. 2023-10-06T04:48:42.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.048+0000 7f3b6339d700 1 Processor -- start 2023-10-06T04:48:42.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.048+0000 7f3b6339d700 1 -- start start 2023-10-06T04:48:42.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b6339d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b5c06a020 0x7f3b5c14c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:42.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b6339d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b5c0ccc00 0x7f3b5c0cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:42.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b6339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b5c159a60 0x7f3b5c0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:42.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b6339d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0c6b00 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b6339d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0c6de0 con 0x7f3b5c06a020 2023-10-06T04:48:42.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b6339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0c70c0 con 0x7f3b5c159a60 2023-10-06T04:48:42.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b61b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b5c0ccc00 0x7f3b5c0cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:42.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b61b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b5c0ccc00 0x7f3b5c0cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57790/0 (socket says 172.21.15.55:57790) 2023-10-06T04:48:42.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b61b9a700 1 -- 172.21.15.55:0/2532268920 learned_addr learned my addr 172.21.15.55:0/2532268920 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:42.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b6239b700 1 --2- 172.21.15.55:0/2532268920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b5c06a020 0x7f3b5c14c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:42.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b62b9c700 1 --2- 172.21.15.55:0/2532268920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b5c159a60 0x7f3b5c0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:42.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b61b9a700 1 -- 172.21.15.55:0/2532268920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b5c159a60 msgr2=0x7f3b5c0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:42.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b61b9a700 1 --2- 172.21.15.55:0/2532268920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b5c159a60 0x7f3b5c0cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:42.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b61b9a700 1 -- 172.21.15.55:0/2532268920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b5c06a020 msgr2=0x7f3b5c14c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:42.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b61b9a700 1 --2- 172.21.15.55:0/2532268920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b5c06a020 0x7f3b5c14c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:42.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.049+0000 7f3b61b9a700 1 -- 172.21.15.55:0/2532268920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c009420 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.050+0000 7f3b61b9a700 1 --2- 172.21.15.55:0/2532268920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b5c0ccc00 0x7f3b5c0cb0f0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c06cdc0 tx=0x7f3b4c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:42.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.050+0000 7f3b527fc700 1 -- 172.21.15.55:0/2532268920 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c011b10 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.050+0000 7f3b527fc700 1 -- 172.21.15.55:0/2532268920 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c0253f0 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.050+0000 7f3b527fc700 1 -- 172.21.15.55:0/2532268920 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c025790 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.050+0000 7f3b6339d700 1 -- 172.21.15.55:0/2532268920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c0c73a0 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.050+0000 7f3b6339d700 1 -- 172.21.15.55:0/2532268920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c0c78c0 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.051+0000 7f3b6339d700 1 -- 172.21.15.55:0/2532268920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5c0c6c40 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.052+0000 7f3b527fc700 1 -- 172.21.15.55:0/2532268920 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b4c025b80 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.053+0000 7f3b527fc700 1 --2- 172.21.15.55:0/2532268920 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f3b40066f30 0x7f3b400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:42.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.053+0000 7f3b527fc700 1 -- 172.21.15.55:0/2532268920 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6350+0+0 (secure 0 0 0) 0x7f3b4c086120 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.054+0000 7f3b6239b700 1 --2- 172.21.15.55:0/2532268920 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f3b40066f30 0x7f3b400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:42.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.054+0000 7f3b6239b700 1 --2- 172.21.15.55:0/2532268920 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f3b40066f30 0x7f3b400693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009ca0 tx=0x7f3b58008040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:42.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.058+0000 7f3b527fc700 1 -- 172.21.15.55:0/2532268920 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b5c0c6c40 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.308+0000 7f3b6339d700 1 -- 172.21.15.55:0/2532268920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3b5c0c6f20 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.813+0000 7f3b527fc700 1 -- 172.21.15.55:0/2532268920 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v74) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3b5c0c6f20 con 0x7f3b5c0ccc00 2023-10-06T04:48:42.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.823+0000 7f3b6339d700 1 -- 172.21.15.55:0/2532268920 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f3b40066f30 msgr2=0x7f3b400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:42.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.823+0000 7f3b6339d700 1 --2- 172.21.15.55:0/2532268920 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f3b40066f30 0x7f3b400693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009ca0 tx=0x7f3b58008040 comp rx=0 tx=0).stop 2023-10-06T04:48:42.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.823+0000 7f3b6339d700 1 -- 172.21.15.55:0/2532268920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b5c0ccc00 msgr2=0x7f3b5c0cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:42.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.823+0000 7f3b6339d700 1 --2- 172.21.15.55:0/2532268920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b5c0ccc00 0x7f3b5c0cb0f0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c06cdc0 tx=0x7f3b4c017000 comp rx=0 tx=0).stop 2023-10-06T04:48:42.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.824+0000 7f3b6339d700 1 -- 172.21.15.55:0/2532268920 shutdown_connections 2023-10-06T04:48:42.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.824+0000 7f3b6339d700 1 --2- 172.21.15.55:0/2532268920 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f3b40066f30 0x7f3b400693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:42.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.824+0000 7f3b6339d700 1 --2- 172.21.15.55:0/2532268920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b5c159a60 0x7f3b5c0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:42.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.824+0000 7f3b6339d700 1 --2- 172.21.15.55:0/2532268920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b5c06a020 0x7f3b5c14c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:42.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.824+0000 7f3b6339d700 1 --2- 172.21.15.55:0/2532268920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b5c0ccc00 0x7f3b5c0cb0f0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:42.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.824+0000 7f3b6339d700 1 -- 172.21.15.55:0/2532268920 >> 172.21.15.55:0/2532268920 conn(0x7f3b5c06baa0 msgr2=0x7f3b5c1577e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:42.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.824+0000 7f3b6339d700 1 -- 172.21.15.55:0/2532268920 shutdown_connections 2023-10-06T04:48:42.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:42.825+0000 7f3b6339d700 1 -- 172.21.15.55:0/2532268920 wait complete. 2023-10-06T04:48:42.836 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 19 and data pool 20 2023-10-06T04:48:42.859 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:48:43.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.044+0000 7f2ab0d9d700 1 Processor -- start 2023-10-06T04:48:43.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.044+0000 7f2ab0d9d700 1 -- start start 2023-10-06T04:48:43.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.045+0000 7f2ab0d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aac150550 0x7f2aac157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.045+0000 7f2ab0d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2aac0c7410 0x7f2aac069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.045+0000 7f2ab0d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aac06e2a0 con 0x7f2aac150180 2023-10-06T04:48:43.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.045+0000 7f2ab0d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aac06b2e0 con 0x7f2aac0c7410 2023-10-06T04:48:43.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.045+0000 7f2ab0d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0c78e0 con 0x7f2aac150550 2023-10-06T04:48:43.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.046+0000 7f2aab7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2aac150180 0x7f2aac0c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:35954/0 (socket says 172.21.15.55:35954) 2023-10-06T04:48:43.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.046+0000 7f2aab7fe700 1 -- 172.21.15.55:0/320232511 learned_addr learned my addr 172.21.15.55:0/320232511 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:43.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.046+0000 7f2aabfff700 1 --2- 172.21.15.55:0/320232511 >> v2:172.21.15.163:3300/0 conn(0x7f2aac0c7410 0x7f2aac069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.046+0000 7f2aaaffd700 1 --2- 172.21.15.55:0/320232511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aac150550 0x7f2aac157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.046+0000 7f2aaa7fc700 1 -- 172.21.15.55:0/320232511 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788791198 0 0) 0x7f2aac06e2a0 con 0x7f2aac150180 2023-10-06T04:48:43.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.046+0000 7f2aaa7fc700 1 -- 172.21.15.55:0/320232511 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a94003570 con 0x7f2aac150180 2023-10-06T04:48:43.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.046+0000 7f2aaaffd700 1 -- 172.21.15.55:0/320232511 >> v2:172.21.15.163:3300/0 conn(0x7f2aac0c7410 msgr2=0x7f2aac069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:43.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.046+0000 7f2aaaffd700 1 --2- 172.21.15.55:0/320232511 >> v2:172.21.15.163:3300/0 conn(0x7f2aac0c7410 0x7f2aac069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.046+0000 7f2aaaffd700 1 -- 172.21.15.55:0/320232511 >> v1:172.21.15.55:6789/0 conn(0x7f2aac150180 legacy=0x7f2aac0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.046+0000 7f2aaaffd700 1 -- 172.21.15.55:0/320232511 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac0c7bc0 con 0x7f2aac150550 2023-10-06T04:48:43.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.046+0000 7f2aabfff700 1 --2- 172.21.15.55:0/320232511 >> v2:172.21.15.163:3300/0 conn(0x7f2aac0c7410 0x7f2aac069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:43.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.047+0000 7f2aaaffd700 1 --2- 172.21.15.55:0/320232511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aac150550 0x7f2aac157770 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009910 tx=0x7f2aa00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=306556f3d8cab379 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:43.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.047+0000 7f2aaa7fc700 1 -- 172.21.15.55:0/320232511 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0011630 con 0x7f2aac150550 2023-10-06T04:48:43.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.047+0000 7f2aaa7fc700 1 -- 172.21.15.55:0/320232511 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa0011790 con 0x7f2aac150550 2023-10-06T04:48:43.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.047+0000 7f2aaa7fc700 1 -- 172.21.15.55:0/320232511 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0011b30 con 0x7f2aac150550 2023-10-06T04:48:43.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.047+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/320232511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aac150550 msgr2=0x7f2aac157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.047+0000 7f2ab0d9d700 1 --2- 172.21.15.55:0/320232511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aac150550 0x7f2aac157770 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0009910 tx=0x7f2aa00091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.048+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/320232511 shutdown_connections 2023-10-06T04:48:43.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.048+0000 7f2ab0d9d700 1 --2- 172.21.15.55:0/320232511 >> v2:172.21.15.163:3300/0 conn(0x7f2aac0c7410 0x7f2aac069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.048+0000 7f2ab0d9d700 1 --2- 172.21.15.55:0/320232511 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aac150550 0x7f2aac157770 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.048+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/320232511 >> 172.21.15.55:0/320232511 conn(0x7f2aac0cc1f0 msgr2=0x7f2aac0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:43.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.048+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/320232511 shutdown_connections 2023-10-06T04:48:43.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.048+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/320232511 wait complete. 2023-10-06T04:48:43.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.049+0000 7f2ab0d9d700 1 Processor -- start 2023-10-06T04:48:43.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.049+0000 7f2ab0d9d700 1 -- start start 2023-10-06T04:48:43.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.050+0000 7f2ab0d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aac0c7410 0x7f2aac0c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.050+0000 7f2ab0d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aac150180 0x7f2aac16b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.050+0000 7f2ab0d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2aac150550 0x7f2aac16b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.050+0000 7f2ab0d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0aef90 con 0x7f2aac150550 2023-10-06T04:48:43.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.050+0000 7f2ab0d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0c5a40 con 0x7f2aac0c7410 2023-10-06T04:48:43.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.050+0000 7f2ab0d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0c5d20 con 0x7f2aac150180 2023-10-06T04:48:43.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.050+0000 7f2aabfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2aac150550 0x7f2aac16b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.050+0000 7f2aabfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2aac150550 0x7f2aac16b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:57806/0 (socket says 172.21.15.55:57806) 2023-10-06T04:48:43.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.050+0000 7f2aabfff700 1 -- 172.21.15.55:0/2448708782 learned_addr learned my addr 172.21.15.55:0/2448708782 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:43.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.051+0000 7f2aab7fe700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aac0c7410 0x7f2aac0c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.051+0000 7f2aaaffd700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aac150180 0x7f2aac16b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.051+0000 7f2aabfff700 1 -- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aac150180 msgr2=0x7f2aac16b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.051+0000 7f2aabfff700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aac150180 0x7f2aac16b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.051+0000 7f2aabfff700 1 -- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aac0c7410 msgr2=0x7f2aac0c4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.051+0000 7f2aabfff700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aac0c7410 0x7f2aac0c4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.051+0000 7f2aabfff700 1 -- 172.21.15.55:0/2448708782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa4009a20 con 0x7f2aac150550 2023-10-06T04:48:43.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.051+0000 7f2aab7fe700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aac0c7410 0x7f2aac0c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:48:43.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.051+0000 7f2aaaffd700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aac150180 0x7f2aac16b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:48:43.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.051+0000 7f2aabfff700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2aac150550 0x7f2aac16b8f0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009e90 tx=0x7f2aa400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:43.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.051+0000 7f2a937fe700 1 -- 172.21.15.55:0/2448708782 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400cc20 con 0x7f2aac150550 2023-10-06T04:48:43.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.051+0000 7f2a937fe700 1 -- 172.21.15.55:0/2448708782 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa400f040 con 0x7f2aac150550 2023-10-06T04:48:43.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.051+0000 7f2a937fe700 1 -- 172.21.15.55:0/2448708782 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4014480 con 0x7f2aac150550 2023-10-06T04:48:43.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.051+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/2448708782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa0009420 con 0x7f2aac150550 2023-10-06T04:48:43.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.052+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/2448708782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac0c62e0 con 0x7f2aac150550 2023-10-06T04:48:43.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.052+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/2448708782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aac04c3c0 con 0x7f2aac150550 2023-10-06T04:48:43.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.054+0000 7f2a937fe700 1 -- 172.21.15.55:0/2448708782 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2aa4014810 con 0x7f2aac150550 2023-10-06T04:48:43.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.054+0000 7f2a937fe700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a98066f30 0x7f2a980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.055+0000 7f2a937fe700 1 -- 172.21.15.55:0/2448708782 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f2aa4085370 con 0x7f2aac150550 2023-10-06T04:48:43.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.055+0000 7f2aab7fe700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a98066f30 0x7f2a980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.056+0000 7f2aab7fe700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a98066f30 0x7f2a980693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0058d0 tx=0x7f2a9c009040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:43.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.058+0000 7f2a937fe700 1 -- 172.21.15.55:0/2448708782 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2aa4052250 con 0x7f2aac150550 2023-10-06T04:48:43.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.260+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/2448708782 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2aac0ae8c0 con 0x7f2aac150550 2023-10-06T04:48:43.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.261+0000 7f2a937fe700 1 -- 172.21.15.55:0/2448708782 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+17536 (secure 0 0 0) 0x7f2aa4055870 con 0x7f2aac150550 2023-10-06T04:48:43.264 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:43.264 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":159,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T04:48:42.309448+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:48:40.391644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T04:48:41.276409+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:48:43.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.264+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a98066f30 msgr2=0x7f2a980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.264+0000 7f2ab0d9d700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a98066f30 0x7f2a980693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0058d0 tx=0x7f2a9c009040 comp rx=0 tx=0).stop 2023-10-06T04:48:43.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.264+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/2448708782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2aac150550 msgr2=0x7f2aac16b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.264+0000 7f2ab0d9d700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2aac150550 0x7f2aac16b8f0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009e90 tx=0x7f2aa400b040 comp rx=0 tx=0).stop 2023-10-06T04:48:43.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.264+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/2448708782 shutdown_connections 2023-10-06T04:48:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.264+0000 7f2ab0d9d700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a98066f30 0x7f2a980693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.264+0000 7f2ab0d9d700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2aac150550 0x7f2aac16b8f0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.264+0000 7f2ab0d9d700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2aac0c7410 0x7f2aac0c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.264+0000 7f2ab0d9d700 1 --2- 172.21.15.55:0/2448708782 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2aac150180 0x7f2aac16b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.264+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/2448708782 >> 172.21.15.55:0/2448708782 conn(0x7f2aac0cc1f0 msgr2=0x7f2aac06a540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.264+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/2448708782 shutdown_connections 2023-10-06T04:48:43.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.264+0000 7f2ab0d9d700 1 -- 172.21.15.55:0/2448708782 wait complete. 2023-10-06T04:48:43.284 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:48:43.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.470+0000 7fe5f2f9d700 1 Processor -- start 2023-10-06T04:48:43.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.470+0000 7fe5f2f9d700 1 -- start start 2023-10-06T04:48:43.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.471+0000 7fe5f2f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe5ec06a0a0 0x7fe5ec15b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.471+0000 7fe5f2f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe5ec06dd20 0x7fe5ec15b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.471+0000 7fe5f2f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ec06ae70 con 0x7fe5ec06d950 2023-10-06T04:48:43.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.471+0000 7fe5f2f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ec0c8a60 con 0x7fe5ec06dd20 2023-10-06T04:48:43.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.471+0000 7fe5f2f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ec06a540 con 0x7fe5ec06a0a0 2023-10-06T04:48:43.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.471+0000 7fe5f279c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe5ec06dd20 0x7fe5ec15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.471+0000 7fe5f279c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe5ec06dd20 0x7fe5ec15b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48352/0 (socket says 172.21.15.55:48352) 2023-10-06T04:48:43.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.471+0000 7fe5f279c700 1 -- 172.21.15.55:0/2052896767 learned_addr learned my addr 172.21.15.55:0/2052896767 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:43.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.472+0000 7fe5f179a700 1 --2- 172.21.15.55:0/2052896767 >> v2:172.21.15.163:3300/0 conn(0x7fe5ec06a0a0 0x7fe5ec15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.472+0000 7fe5f279c700 1 -- 172.21.15.55:0/2052896767 >> v2:172.21.15.163:3300/0 conn(0x7fe5ec06a0a0 msgr2=0x7fe5ec15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:43.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.472+0000 7fe5f279c700 1 --2- 172.21.15.55:0/2052896767 >> v2:172.21.15.163:3300/0 conn(0x7fe5ec06a0a0 0x7fe5ec15b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.472+0000 7fe5f279c700 1 -- 172.21.15.55:0/2052896767 >> v1:172.21.15.163:6789/0 conn(0x7fe5ec06d950 legacy=0x7fe5ec0c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.472+0000 7fe5f279c700 1 -- 172.21.15.55:0/2052896767 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec06a820 con 0x7fe5ec06dd20 2023-10-06T04:48:43.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.472+0000 7fe5f279c700 1 --2- 172.21.15.55:0/2052896767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5ec06dd20 0x7fe5ec15b950 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec06b0c0 tx=0x7fe5e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87176e190a1ef457 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:43.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.472+0000 7fe5f0f99700 1 -- 172.21.15.55:0/2052896767 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e400e070 con 0x7fe5ec06dd20 2023-10-06T04:48:43.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.472+0000 7fe5f0f99700 1 -- 172.21.15.55:0/2052896767 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e400ba50 con 0x7fe5ec06dd20 2023-10-06T04:48:43.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.472+0000 7fe5f0f99700 1 -- 172.21.15.55:0/2052896767 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e40092e0 con 0x7fe5ec06dd20 2023-10-06T04:48:43.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.473+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/2052896767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5ec06dd20 msgr2=0x7fe5ec15b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.473+0000 7fe5f2f9d700 1 --2- 172.21.15.55:0/2052896767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5ec06dd20 0x7fe5ec15b950 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec06b0c0 tx=0x7fe5e4009840 comp rx=0 tx=0).stop 2023-10-06T04:48:43.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.473+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/2052896767 shutdown_connections 2023-10-06T04:48:43.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.473+0000 7fe5f2f9d700 1 --2- 172.21.15.55:0/2052896767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5ec06dd20 0x7fe5ec15b950 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.473+0000 7fe5f2f9d700 1 --2- 172.21.15.55:0/2052896767 >> v2:172.21.15.163:3300/0 conn(0x7fe5ec06a0a0 0x7fe5ec15b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.473+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/2052896767 >> 172.21.15.55:0/2052896767 conn(0x7fe5ec0cc1f0 msgr2=0x7fe5ec0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:43.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.473+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/2052896767 shutdown_connections 2023-10-06T04:48:43.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.473+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/2052896767 wait complete. 2023-10-06T04:48:43.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.475+0000 7fe5f2f9d700 1 Processor -- start 2023-10-06T04:48:43.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.475+0000 7fe5f2f9d700 1 -- start start 2023-10-06T04:48:43.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.476+0000 7fe5f2f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5ec06a0a0 0x7fe5ec151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.476+0000 7fe5f2f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5ec06d950 0x7fe5ec169e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.476+0000 7fe5f2f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5ec06dd20 0x7fe5ec0c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.476+0000 7fe5f2f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec1595b0 con 0x7fe5ec06dd20 2023-10-06T04:48:43.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.476+0000 7fe5f2f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec158fc0 con 0x7fe5ec06d950 2023-10-06T04:48:43.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.476+0000 7fe5f2f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec16af50 con 0x7fe5ec06a0a0 2023-10-06T04:48:43.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.476+0000 7fe5f279c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5ec06dd20 0x7fe5ec0c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.476+0000 7fe5f279c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5ec06dd20 0x7fe5ec0c6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48360/0 (socket says 172.21.15.55:48360) 2023-10-06T04:48:43.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.476+0000 7fe5f279c700 1 -- 172.21.15.55:0/3330675012 learned_addr learned my addr 172.21.15.55:0/3330675012 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:43.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.476+0000 7fe5f179a700 1 --2- 172.21.15.55:0/3330675012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5ec06d950 0x7fe5ec169e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.476+0000 7fe5f1f9b700 1 --2- 172.21.15.55:0/3330675012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5ec06a0a0 0x7fe5ec151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.477+0000 7fe5f279c700 1 -- 172.21.15.55:0/3330675012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5ec06a0a0 msgr2=0x7fe5ec151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.477+0000 7fe5f279c700 1 --2- 172.21.15.55:0/3330675012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5ec06a0a0 0x7fe5ec151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.477+0000 7fe5f279c700 1 -- 172.21.15.55:0/3330675012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5ec06d950 msgr2=0x7fe5ec169e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.477+0000 7fe5f279c700 1 --2- 172.21.15.55:0/3330675012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5ec06d950 0x7fe5ec169e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.477+0000 7fe5f279c700 1 -- 172.21.15.55:0/3330675012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e4009ae0 con 0x7fe5ec06dd20 2023-10-06T04:48:43.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.477+0000 7fe5f279c700 1 --2- 172.21.15.55:0/3330675012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5ec06dd20 0x7fe5ec0c6a40 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec06fe40 tx=0x7fe5e4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:43.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.477+0000 7fe5e1ffb700 1 -- 172.21.15.55:0/3330675012 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e400be60 con 0x7fe5ec06dd20 2023-10-06T04:48:43.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.477+0000 7fe5e1ffb700 1 -- 172.21.15.55:0/3330675012 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e4008d20 con 0x7fe5ec06dd20 2023-10-06T04:48:43.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.477+0000 7fe5e1ffb700 1 -- 172.21.15.55:0/3330675012 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e401c660 con 0x7fe5ec06dd20 2023-10-06T04:48:43.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.477+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/3330675012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ec16ad70 con 0x7fe5ec06dd20 2023-10-06T04:48:43.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.477+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/3330675012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ec16aa90 con 0x7fe5ec06dd20 2023-10-06T04:48:43.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.478+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/3330675012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ec04c3c0 con 0x7fe5ec06dd20 2023-10-06T04:48:43.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.479+0000 7fe5e1ffb700 1 -- 172.21.15.55:0/3330675012 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5e4003040 con 0x7fe5ec06dd20 2023-10-06T04:48:43.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.480+0000 7fe5e1ffb700 1 --2- 172.21.15.55:0/3330675012 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe5d0066f30 0x7fe5d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.480+0000 7fe5e1ffb700 1 -- 172.21.15.55:0/3330675012 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fe5e40860f0 con 0x7fe5ec06dd20 2023-10-06T04:48:43.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.481+0000 7fe5f1f9b700 1 --2- 172.21.15.55:0/3330675012 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe5d0066f30 0x7fe5d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.481+0000 7fe5f1f9b700 1 --2- 172.21.15.55:0/3330675012 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe5d0066f30 0x7fe5d00693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8003a80 tx=0x7fe5e8007ec0 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:43.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.484+0000 7fe5e1ffb700 1 -- 172.21.15.55:0/3330675012 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5e4052fd0 con 0x7fe5ec06dd20 2023-10-06T04:48:43.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.685+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/3330675012 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5ec0c2040 con 0x7fe5ec06dd20 2023-10-06T04:48:43.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.686+0000 7fe5e1ffb700 1 -- 172.21.15.55:0/3330675012 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+17536 (secure 0 0 0) 0x7fe5e4009940 con 0x7fe5ec06dd20 2023-10-06T04:48:43.732 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:43.732 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":159,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T04:48:42.309448+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:48:40.391644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T04:48:41.276409+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:48:43.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.689+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/3330675012 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe5d0066f30 msgr2=0x7fe5d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.689+0000 7fe5f2f9d700 1 --2- 172.21.15.55:0/3330675012 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe5d0066f30 0x7fe5d00693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8003a80 tx=0x7fe5e8007ec0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.689+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/3330675012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5ec06dd20 msgr2=0x7fe5ec0c6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.689+0000 7fe5f2f9d700 1 --2- 172.21.15.55:0/3330675012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5ec06dd20 0x7fe5ec0c6a40 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec06fe40 tx=0x7fe5e4018000 comp rx=0 tx=0).stop 2023-10-06T04:48:43.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.689+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/3330675012 shutdown_connections 2023-10-06T04:48:43.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.689+0000 7fe5f2f9d700 1 --2- 172.21.15.55:0/3330675012 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe5d0066f30 0x7fe5d00693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.689+0000 7fe5f2f9d700 1 --2- 172.21.15.55:0/3330675012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe5ec06dd20 0x7fe5ec0c6a40 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.689+0000 7fe5f2f9d700 1 --2- 172.21.15.55:0/3330675012 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe5ec06a0a0 0x7fe5ec151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.689+0000 7fe5f2f9d700 1 --2- 172.21.15.55:0/3330675012 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe5ec06d950 0x7fe5ec169e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.689+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/3330675012 >> 172.21.15.55:0/3330675012 conn(0x7fe5ec0cc1f0 msgr2=0x7fe5ec1526d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:43.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.689+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/3330675012 shutdown_connections 2023-10-06T04:48:43.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.689+0000 7fe5f2f9d700 1 -- 172.21.15.55:0/3330675012 wait complete. 2023-10-06T04:48:43.735 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T04:48:43.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.924+0000 7fa58759e700 1 Processor -- start 2023-10-06T04:48:43.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.925+0000 7fa58759e700 1 -- start start 2023-10-06T04:48:43.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.926+0000 7fa58759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa588159450 0x7fa5880c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.926+0000 7fa58759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa5880c96e0 0x7fa5880a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.926+0000 7fa58759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5880c8730 con 0x7fa588159820 2023-10-06T04:48:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.926+0000 7fa58759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5880c7b00 con 0x7fa5880c96e0 2023-10-06T04:48:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.926+0000 7fa58759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880c89a0 con 0x7fa588159450 2023-10-06T04:48:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.926+0000 7fa585d9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa5880c96e0 0x7fa5880a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.926+0000 7fa58659c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa588159450 0x7fa5880c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.926+0000 7fa58659c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa588159450 0x7fa5880c91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:50014/0 (socket says 172.21.15.55:50014) 2023-10-06T04:48:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.926+0000 7fa58659c700 1 -- 172.21.15.55:0/2043341769 learned_addr learned my addr 172.21.15.55:0/2043341769 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:43.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.926+0000 7fa585d9b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa5880c96e0 0x7fa5880a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:45862/0 (socket says 172.21.15.55:45862) 2023-10-06T04:48:43.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.927+0000 7fa58659c700 1 -- 172.21.15.55:0/2043341769 >> v2:172.21.15.163:3300/0 conn(0x7fa5880c96e0 msgr2=0x7fa5880a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:43.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.927+0000 7fa58659c700 1 --2- 172.21.15.55:0/2043341769 >> v2:172.21.15.163:3300/0 conn(0x7fa5880c96e0 0x7fa5880a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.927+0000 7fa58659c700 1 -- 172.21.15.55:0/2043341769 >> v1:172.21.15.163:6789/0 conn(0x7fa588159820 legacy=0x7fa58806d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.927+0000 7fa58659c700 1 -- 172.21.15.55:0/2043341769 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5880c8c80 con 0x7fa588159450 2023-10-06T04:48:43.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.927+0000 7fa585d9b700 1 --2- 172.21.15.55:0/2043341769 >> v2:172.21.15.163:3300/0 conn(0x7fa5880c96e0 0x7fa5880a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:43.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.927+0000 7fa58659c700 1 --2- 172.21.15.55:0/2043341769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa588159450 0x7fa5880c91a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa5780099b0 tx=0x7fa578009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b75629101601553 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:43.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.927+0000 7fa58559a700 1 -- 172.21.15.55:0/2043341769 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa578009ea0 con 0x7fa588159450 2023-10-06T04:48:43.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.928+0000 7fa58559a700 1 -- 172.21.15.55:0/2043341769 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57800b860 con 0x7fa588159450 2023-10-06T04:48:43.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.928+0000 7fa58559a700 1 -- 172.21.15.55:0/2043341769 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57800bc00 con 0x7fa588159450 2023-10-06T04:48:43.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.928+0000 7fa58759e700 1 -- 172.21.15.55:0/2043341769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa588159450 msgr2=0x7fa5880c91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.928+0000 7fa58759e700 1 --2- 172.21.15.55:0/2043341769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa588159450 0x7fa5880c91a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa5780099b0 tx=0x7fa578009250 comp rx=0 tx=0).stop 2023-10-06T04:48:43.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.928+0000 7fa58759e700 1 -- 172.21.15.55:0/2043341769 shutdown_connections 2023-10-06T04:48:43.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.928+0000 7fa58759e700 1 --2- 172.21.15.55:0/2043341769 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa588159450 0x7fa5880c91a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.928+0000 7fa58759e700 1 --2- 172.21.15.55:0/2043341769 >> v2:172.21.15.163:3300/0 conn(0x7fa5880c96e0 0x7fa5880a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.928+0000 7fa58759e700 1 -- 172.21.15.55:0/2043341769 >> 172.21.15.55:0/2043341769 conn(0x7fa5880698e0 msgr2=0x7fa588069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:43.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.928+0000 7fa58759e700 1 -- 172.21.15.55:0/2043341769 shutdown_connections 2023-10-06T04:48:43.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.928+0000 7fa58759e700 1 -- 172.21.15.55:0/2043341769 wait complete. 2023-10-06T04:48:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.930+0000 7fa58759e700 1 Processor -- start 2023-10-06T04:48:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.930+0000 7fa58759e700 1 -- start start 2023-10-06T04:48:43.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa58759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5880c96e0 0x7fa58815c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa58759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa588159450 0x7fa5880af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa58759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa588159820 0x7fa5880b1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa58759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58815d3b0 con 0x7fa588159820 2023-10-06T04:48:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa58759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880b21f0 con 0x7fa5880c96e0 2023-10-06T04:48:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa58759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58815d080 con 0x7fa588159450 2023-10-06T04:48:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa586d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa588159820 0x7fa5880b1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa586d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa588159820 0x7fa5880b1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48370/0 (socket says 172.21.15.55:48370) 2023-10-06T04:48:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa586d9d700 1 -- 172.21.15.55:0/399823283 learned_addr learned my addr 172.21.15.55:0/399823283 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa58659c700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5880c96e0 0x7fa58815c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa585d9b700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa588159450 0x7fa5880af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa586d9d700 1 -- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa588159450 msgr2=0x7fa5880af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa586d9d700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa588159450 0x7fa5880af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa586d9d700 1 -- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5880c96e0 msgr2=0x7fa58815c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:43.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa586d9d700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5880c96e0 0x7fa58815c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:43.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa586d9d700 1 -- 172.21.15.55:0/399823283 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c009420 con 0x7fa588159820 2023-10-06T04:48:43.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa58659c700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5880c96e0 0x7fa58815c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:43.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.931+0000 7fa585d9b700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa588159450 0x7fa5880af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:43.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.932+0000 7fa586d9d700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa588159820 0x7fa5880b1300 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa58000c810 tx=0x7fa580001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:43.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.932+0000 7fa5767fc700 1 -- 172.21.15.55:0/399823283 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580012630 con 0x7fa588159820 2023-10-06T04:48:43.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.932+0000 7fa5767fc700 1 -- 172.21.15.55:0/399823283 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5800127b0 con 0x7fa588159820 2023-10-06T04:48:43.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.932+0000 7fa5767fc700 1 -- 172.21.15.55:0/399823283 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580012b70 con 0x7fa588159820 2023-10-06T04:48:43.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.932+0000 7fa58759e700 1 -- 172.21.15.55:0/399823283 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5780094f0 con 0x7fa588159820 2023-10-06T04:48:43.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.932+0000 7fa58759e700 1 -- 172.21.15.55:0/399823283 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5880b1f10 con 0x7fa588159820 2023-10-06T04:48:43.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.933+0000 7fa58759e700 1 -- 172.21.15.55:0/399823283 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58815d520 con 0x7fa588159820 2023-10-06T04:48:43.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.934+0000 7fa5767fc700 1 -- 172.21.15.55:0/399823283 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa580003030 con 0x7fa588159820 2023-10-06T04:48:43.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.935+0000 7fa5767fc700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa568066f30 0x7fa5680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:43.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.935+0000 7fa5767fc700 1 -- 172.21.15.55:0/399823283 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fa5800847c0 con 0x7fa588159820 2023-10-06T04:48:43.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.935+0000 7fa58659c700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa568066f30 0x7fa5680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:43.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.936+0000 7fa58659c700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa568066f30 0x7fa5680693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa5780099b0 tx=0x7fa578017000 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:43.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:43.939+0000 7fa5767fc700 1 -- 172.21.15.55:0/399823283 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa580051740 con 0x7fa588159820 2023-10-06T04:48:44.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:44.189+0000 7fa58759e700 1 -- 172.21.15.55:0/399823283 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa5880c7de0 con 0x7fa588159820 2023-10-06T04:48:44.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:44.840+0000 7fa5767fc700 1 -- 172.21.15.55:0/399823283 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v77) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa580054d60 con 0x7fa588159820 2023-10-06T04:48:44.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:44.843+0000 7fa58759e700 1 -- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa568066f30 msgr2=0x7fa5680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:44.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:44.843+0000 7fa58759e700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa568066f30 0x7fa5680693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa5780099b0 tx=0x7fa578017000 comp rx=0 tx=0).stop 2023-10-06T04:48:44.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:44.843+0000 7fa58759e700 1 -- 172.21.15.55:0/399823283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa588159820 msgr2=0x7fa5880b1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:44.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:44.843+0000 7fa58759e700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa588159820 0x7fa5880b1300 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa58000c810 tx=0x7fa580001650 comp rx=0 tx=0).stop 2023-10-06T04:48:44.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:44.843+0000 7fa58759e700 1 -- 172.21.15.55:0/399823283 shutdown_connections 2023-10-06T04:48:44.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:44.843+0000 7fa58759e700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa568066f30 0x7fa5680693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:44.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:44.843+0000 7fa58759e700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa588159820 0x7fa5880b1300 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:44.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:44.843+0000 7fa58759e700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5880c96e0 0x7fa58815c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:44.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:44.843+0000 7fa58759e700 1 --2- 172.21.15.55:0/399823283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa588159450 0x7fa5880af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:44.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:44.843+0000 7fa58759e700 1 -- 172.21.15.55:0/399823283 >> 172.21.15.55:0/399823283 conn(0x7fa5880698e0 msgr2=0x7fa5880a98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:44.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:44.843+0000 7fa58759e700 1 -- 172.21.15.55:0/399823283 shutdown_connections 2023-10-06T04:48:44.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:44.843+0000 7fa58759e700 1 -- 172.21.15.55:0/399823283 wait complete. 2023-10-06T04:48:44.864 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T04:48:45.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.054+0000 7fe89639d700 1 Processor -- start 2023-10-06T04:48:45.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.055+0000 7fe89639d700 1 -- start start 2023-10-06T04:48:45.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.056+0000 7fe89639d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe8900cb1a0 0x7fe89015a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:45.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.056+0000 7fe89639d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89006ebd0 con 0x7fe8900ae0d0 2023-10-06T04:48:45.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.056+0000 7fe89639d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89015ad20 con 0x7fe89015a8a0 2023-10-06T04:48:45.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.056+0000 7fe89639d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89015b2e0 con 0x7fe8900cb1a0 2023-10-06T04:48:45.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.056+0000 7fe895b9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe8900ae0d0 0x7fe8900ae4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37120/0 (socket says 172.21.15.55:37120) 2023-10-06T04:48:45.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.056+0000 7fe895b9c700 1 -- 172.21.15.55:0/825016124 learned_addr learned my addr 172.21.15.55:0/825016124 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:45.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.056+0000 7fe89539b700 1 --2- 172.21.15.55:0/825016124 >> v2:172.21.15.163:3300/0 conn(0x7fe8900cb1a0 0x7fe89015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:45.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.057+0000 7fe887fff700 1 -- 172.21.15.55:0/825016124 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199478016 0 0) 0x7fe89006ebd0 con 0x7fe8900ae0d0 2023-10-06T04:48:45.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.057+0000 7fe887fff700 1 -- 172.21.15.55:0/825016124 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe87c003570 con 0x7fe8900ae0d0 2023-10-06T04:48:45.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.057+0000 7fe887fff700 1 -- 172.21.15.55:0/825016124 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 677867505 0 0) 0x7fe89015ad20 con 0x7fe89015a8a0 2023-10-06T04:48:45.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.057+0000 7fe887fff700 1 -- 172.21.15.55:0/825016124 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe89006ebd0 con 0x7fe89015a8a0 2023-10-06T04:48:45.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.057+0000 7fe89539b700 1 -- 172.21.15.55:0/825016124 >> v1:172.21.15.163:6789/0 conn(0x7fe89015a8a0 legacy=0x7fe89006f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:45.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.057+0000 7fe89539b700 1 -- 172.21.15.55:0/825016124 >> v1:172.21.15.55:6789/0 conn(0x7fe8900ae0d0 legacy=0x7fe8900ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:45.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.057+0000 7fe89539b700 1 -- 172.21.15.55:0/825016124 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89015b000 con 0x7fe8900cb1a0 2023-10-06T04:48:45.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.057+0000 7fe89539b700 1 --2- 172.21.15.55:0/825016124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8900cb1a0 0x7fe89015a360 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0098e0 tx=0x7fe88c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5ec9a5e09667727 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:45.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.058+0000 7fe887fff700 1 -- 172.21.15.55:0/825016124 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c011630 con 0x7fe8900cb1a0 2023-10-06T04:48:45.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.058+0000 7fe887fff700 1 -- 172.21.15.55:0/825016124 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c011790 con 0x7fe8900cb1a0 2023-10-06T04:48:45.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.058+0000 7fe887fff700 1 -- 172.21.15.55:0/825016124 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c011b30 con 0x7fe8900cb1a0 2023-10-06T04:48:45.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.058+0000 7fe89639d700 1 -- 172.21.15.55:0/825016124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8900cb1a0 msgr2=0x7fe89015a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:45.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.058+0000 7fe89639d700 1 --2- 172.21.15.55:0/825016124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8900cb1a0 0x7fe89015a360 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0098e0 tx=0x7fe88c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:45.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.058+0000 7fe89639d700 1 -- 172.21.15.55:0/825016124 shutdown_connections 2023-10-06T04:48:45.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.058+0000 7fe89639d700 1 --2- 172.21.15.55:0/825016124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8900cb1a0 0x7fe89015a360 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:45.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.058+0000 7fe89639d700 1 -- 172.21.15.55:0/825016124 >> 172.21.15.55:0/825016124 conn(0x7fe8900cc1e0 msgr2=0x7fe8900cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:45.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.058+0000 7fe89639d700 1 -- 172.21.15.55:0/825016124 shutdown_connections 2023-10-06T04:48:45.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.058+0000 7fe89639d700 1 -- 172.21.15.55:0/825016124 wait complete. 2023-10-06T04:48:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.060+0000 7fe89639d700 1 Processor -- start 2023-10-06T04:48:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.060+0000 7fe89639d700 1 -- start start 2023-10-06T04:48:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe89639d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8900ae0d0 0x7fe8900c2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe89639d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8900cb1a0 0x7fe8900c3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe89639d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8900c3ca0 0x7fe8900b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe89639d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8900c4410 con 0x7fe8900ae0d0 2023-10-06T04:48:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe89639d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8900adee0 con 0x7fe8900cb1a0 2023-10-06T04:48:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe89639d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89016a070 con 0x7fe8900c3ca0 2023-10-06T04:48:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe89539b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8900ae0d0 0x7fe8900c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe89539b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8900ae0d0 0x7fe8900c2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48372/0 (socket says 172.21.15.55:48372) 2023-10-06T04:48:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe89539b700 1 -- 172.21.15.55:0/1411939474 learned_addr learned my addr 172.21.15.55:0/1411939474 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe894b9a700 1 --2- 172.21.15.55:0/1411939474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8900cb1a0 0x7fe8900c3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe895b9c700 1 --2- 172.21.15.55:0/1411939474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8900c3ca0 0x7fe8900b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe89539b700 1 -- 172.21.15.55:0/1411939474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8900c3ca0 msgr2=0x7fe8900b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:45.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe89539b700 1 --2- 172.21.15.55:0/1411939474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8900c3ca0 0x7fe8900b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:45.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe89539b700 1 -- 172.21.15.55:0/1411939474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8900cb1a0 msgr2=0x7fe8900c3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:45.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.061+0000 7fe89539b700 1 --2- 172.21.15.55:0/1411939474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8900cb1a0 0x7fe8900c3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:45.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.062+0000 7fe89539b700 1 -- 172.21.15.55:0/1411939474 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c009420 con 0x7fe8900ae0d0 2023-10-06T04:48:45.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.062+0000 7fe89539b700 1 --2- 172.21.15.55:0/1411939474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8900ae0d0 0x7fe8900c2630 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe89006ce80 tx=0x7fe88c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:45.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.062+0000 7fe8857fa700 1 -- 172.21.15.55:0/1411939474 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c01f070 con 0x7fe8900ae0d0 2023-10-06T04:48:45.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.062+0000 7fe8857fa700 1 -- 172.21.15.55:0/1411939474 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c000bc0 con 0x7fe8900ae0d0 2023-10-06T04:48:45.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.062+0000 7fe89639d700 1 -- 172.21.15.55:0/1411939474 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8900c2bd0 con 0x7fe8900ae0d0 2023-10-06T04:48:45.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.062+0000 7fe8857fa700 1 -- 172.21.15.55:0/1411939474 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c025420 con 0x7fe8900ae0d0 2023-10-06T04:48:45.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.062+0000 7fe89639d700 1 -- 172.21.15.55:0/1411939474 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89016a5a0 con 0x7fe8900ae0d0 2023-10-06T04:48:45.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.063+0000 7fe89639d700 1 -- 172.21.15.55:0/1411939474 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89006c450 con 0x7fe8900ae0d0 2023-10-06T04:48:45.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.064+0000 7fe8857fa700 1 -- 172.21.15.55:0/1411939474 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe88c023070 con 0x7fe8900ae0d0 2023-10-06T04:48:45.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.066+0000 7fe8857fa700 1 --2- 172.21.15.55:0/1411939474 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe874066f30 0x7fe8740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:45.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.066+0000 7fe8857fa700 1 -- 172.21.15.55:0/1411939474 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fe88c085aa0 con 0x7fe8900ae0d0 2023-10-06T04:48:45.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.066+0000 7fe894b9a700 1 --2- 172.21.15.55:0/1411939474 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe874066f30 0x7fe8740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:45.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.067+0000 7fe894b9a700 1 --2- 172.21.15.55:0/1411939474 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe874066f30 0x7fe8740693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe8900c3bf0 tx=0x7fe880007d50 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:45.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.070+0000 7fe8857fa700 1 -- 172.21.15.55:0/1411939474 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe88c052980 con 0x7fe8900ae0d0 2023-10-06T04:48:45.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.317+0000 7fe89639d700 1 -- 172.21.15.55:0/1411939474 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe8900c2b70 con 0x7fe8900ae0d0 2023-10-06T04:48:45.850 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.848+0000 7fe8857fa700 1 -- 172.21.15.55:0/1411939474 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v78) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe88c0150c0 con 0x7fe8900ae0d0 2023-10-06T04:48:45.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.851+0000 7fe89639d700 1 -- 172.21.15.55:0/1411939474 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe874066f30 msgr2=0x7fe8740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:45.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.851+0000 7fe89639d700 1 --2- 172.21.15.55:0/1411939474 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe874066f30 0x7fe8740693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe8900c3bf0 tx=0x7fe880007d50 comp rx=0 tx=0).stop 2023-10-06T04:48:45.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.851+0000 7fe89639d700 1 -- 172.21.15.55:0/1411939474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8900ae0d0 msgr2=0x7fe8900c2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:45.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.851+0000 7fe89639d700 1 --2- 172.21.15.55:0/1411939474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8900ae0d0 0x7fe8900c2630 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe89006ce80 tx=0x7fe88c017000 comp rx=0 tx=0).stop 2023-10-06T04:48:45.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.851+0000 7fe89639d700 1 -- 172.21.15.55:0/1411939474 shutdown_connections 2023-10-06T04:48:45.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.851+0000 7fe89639d700 1 --2- 172.21.15.55:0/1411939474 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe874066f30 0x7fe8740693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:45.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.851+0000 7fe89639d700 1 --2- 172.21.15.55:0/1411939474 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8900c3ca0 0x7fe8900b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:45.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.851+0000 7fe89639d700 1 --2- 172.21.15.55:0/1411939474 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8900ae0d0 0x7fe8900c2630 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:45.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.851+0000 7fe89639d700 1 --2- 172.21.15.55:0/1411939474 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8900cb1a0 0x7fe8900c3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:45.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.851+0000 7fe89639d700 1 -- 172.21.15.55:0/1411939474 >> 172.21.15.55:0/1411939474 conn(0x7fe8900cc1e0 msgr2=0x7fe890155cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:45.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.851+0000 7fe89639d700 1 -- 172.21.15.55:0/1411939474 shutdown_connections 2023-10-06T04:48:45.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:45.851+0000 7fe89639d700 1 -- 172.21.15.55:0/1411939474 wait complete. 2023-10-06T04:48:45.873 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:48:46.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.059+0000 7fb2d719d700 1 Processor -- start 2023-10-06T04:48:46.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.059+0000 7fb2d719d700 1 -- start start 2023-10-06T04:48:46.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.061+0000 7fb2d719d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d0156310 0x7fb2d00c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.061+0000 7fb2d719d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb2d01566e0 0x7fb2d00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.061+0000 7fb2d719d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d00ca860 con 0x7fb2d00ca300 2023-10-06T04:48:46.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.061+0000 7fb2d719d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d00cab40 con 0x7fb2d01566e0 2023-10-06T04:48:46.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.061+0000 7fb2d719d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d00cae20 con 0x7fb2d0156310 2023-10-06T04:48:46.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.061+0000 7fb2d619b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d0156310 0x7fb2d00c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.061+0000 7fb2d599a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb2d01566e0 0x7fb2d00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.061+0000 7fb2d699c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb2d00ca300 0x7fb2d00c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:54548/0 (socket says 172.21.15.55:54548) 2023-10-06T04:48:46.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.061+0000 7fb2d699c700 1 -- 172.21.15.55:0/222536020 learned_addr learned my addr 172.21.15.55:0/222536020 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:46.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.062+0000 7fb2d5199700 1 -- 172.21.15.55:0/222536020 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1260129029 0 0) 0x7fb2d00ca860 con 0x7fb2d00ca300 2023-10-06T04:48:46.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.062+0000 7fb2d5199700 1 -- 172.21.15.55:0/222536020 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2bc003570 con 0x7fb2d00ca300 2023-10-06T04:48:46.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.062+0000 7fb2d619b700 1 -- 172.21.15.55:0/222536020 >> v2:172.21.15.163:3300/0 conn(0x7fb2d01566e0 msgr2=0x7fb2d00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:46.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.062+0000 7fb2d619b700 1 --2- 172.21.15.55:0/222536020 >> v2:172.21.15.163:3300/0 conn(0x7fb2d01566e0 0x7fb2d00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.062+0000 7fb2d619b700 1 -- 172.21.15.55:0/222536020 >> v1:172.21.15.163:6789/0 conn(0x7fb2d00ca300 legacy=0x7fb2d00c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.062+0000 7fb2d619b700 1 -- 172.21.15.55:0/222536020 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d00cb100 con 0x7fb2d0156310 2023-10-06T04:48:46.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.062+0000 7fb2d599a700 1 --2- 172.21.15.55:0/222536020 >> v2:172.21.15.163:3300/0 conn(0x7fb2d01566e0 0x7fb2d00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:46.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.062+0000 7fb2d619b700 1 --2- 172.21.15.55:0/222536020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d0156310 0x7fb2d00c9dc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0098e0 tx=0x7fb2cc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eb9154557637a3a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:46.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.062+0000 7fb2d5199700 1 -- 172.21.15.55:0/222536020 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc011630 con 0x7fb2d0156310 2023-10-06T04:48:46.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.062+0000 7fb2d5199700 1 -- 172.21.15.55:0/222536020 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2cc011790 con 0x7fb2d0156310 2023-10-06T04:48:46.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.062+0000 7fb2d5199700 1 -- 172.21.15.55:0/222536020 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc011b30 con 0x7fb2d0156310 2023-10-06T04:48:46.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.063+0000 7fb2d719d700 1 -- 172.21.15.55:0/222536020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d0156310 msgr2=0x7fb2d00c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.063+0000 7fb2d719d700 1 --2- 172.21.15.55:0/222536020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d0156310 0x7fb2d00c9dc0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0098e0 tx=0x7fb2cc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.063+0000 7fb2d719d700 1 -- 172.21.15.55:0/222536020 shutdown_connections 2023-10-06T04:48:46.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.063+0000 7fb2d719d700 1 --2- 172.21.15.55:0/222536020 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d0156310 0x7fb2d00c9dc0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.063+0000 7fb2d719d700 1 --2- 172.21.15.55:0/222536020 >> v2:172.21.15.163:3300/0 conn(0x7fb2d01566e0 0x7fb2d00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.063+0000 7fb2d719d700 1 -- 172.21.15.55:0/222536020 >> 172.21.15.55:0/222536020 conn(0x7fb2d00cc1f0 msgr2=0x7fb2d00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:46.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.063+0000 7fb2d719d700 1 -- 172.21.15.55:0/222536020 shutdown_connections 2023-10-06T04:48:46.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.063+0000 7fb2d719d700 1 -- 172.21.15.55:0/222536020 wait complete. 2023-10-06T04:48:46.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.065+0000 7fb2d719d700 1 Processor -- start 2023-10-06T04:48:46.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.065+0000 7fb2d719d700 1 -- start start 2023-10-06T04:48:46.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.066+0000 7fb2d719d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2d00ca300 0x7fb2d0145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.066+0000 7fb2d719d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2d0156310 0x7fb2d014aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.066+0000 7fb2d719d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d01566e0 0x7fb2d014aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.066+0000 7fb2d719d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d015aad0 con 0x7fb2d00ca300 2023-10-06T04:48:46.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.066+0000 7fb2d719d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0146520 con 0x7fb2d0156310 2023-10-06T04:48:46.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.066+0000 7fb2d719d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d015a7f0 con 0x7fb2d01566e0 2023-10-06T04:48:46.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.066+0000 7fb2d619b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2d00ca300 0x7fb2d0145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.066+0000 7fb2d619b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2d00ca300 0x7fb2d0145fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48382/0 (socket says 172.21.15.55:48382) 2023-10-06T04:48:46.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.066+0000 7fb2d619b700 1 -- 172.21.15.55:0/722617592 learned_addr learned my addr 172.21.15.55:0/722617592 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:46.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.066+0000 7fb2d619b700 1 -- 172.21.15.55:0/722617592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d01566e0 msgr2=0x7fb2d014aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.066+0000 7fb2d619b700 1 --2- 172.21.15.55:0/722617592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d01566e0 0x7fb2d014aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.066+0000 7fb2d619b700 1 -- 172.21.15.55:0/722617592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2d0156310 msgr2=0x7fb2d014aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.066+0000 7fb2d619b700 1 --2- 172.21.15.55:0/722617592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2d0156310 0x7fb2d014aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.066+0000 7fb2d619b700 1 -- 172.21.15.55:0/722617592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c0009420 con 0x7fb2d00ca300 2023-10-06T04:48:46.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.067+0000 7fb2d619b700 1 --2- 172.21.15.55:0/722617592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2d00ca300 0x7fb2d0145fe0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00caf60 tx=0x7fb2cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:46.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.067+0000 7fb2c67fc700 1 -- 172.21.15.55:0/722617592 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc011b10 con 0x7fb2d00ca300 2023-10-06T04:48:46.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.067+0000 7fb2c67fc700 1 -- 172.21.15.55:0/722617592 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2cc0253f0 con 0x7fb2d00ca300 2023-10-06T04:48:46.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.067+0000 7fb2c67fc700 1 -- 172.21.15.55:0/722617592 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc0257b0 con 0x7fb2d00ca300 2023-10-06T04:48:46.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.067+0000 7fb2d719d700 1 -- 172.21.15.55:0/722617592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2cc009420 con 0x7fb2d00ca300 2023-10-06T04:48:46.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.067+0000 7fb2d719d700 1 -- 172.21.15.55:0/722617592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d0148bd0 con 0x7fb2d00ca300 2023-10-06T04:48:46.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.068+0000 7fb2d719d700 1 -- 172.21.15.55:0/722617592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d0148750 con 0x7fb2d00ca300 2023-10-06T04:48:46.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.069+0000 7fb2c67fc700 1 -- 172.21.15.55:0/722617592 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb2cc025ba0 con 0x7fb2d00ca300 2023-10-06T04:48:46.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.070+0000 7fb2c67fc700 1 --2- 172.21.15.55:0/722617592 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb2b4066f30 0x7fb2b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.070+0000 7fb2c67fc700 1 -- 172.21.15.55:0/722617592 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fb2cc0861f0 con 0x7fb2d00ca300 2023-10-06T04:48:46.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.071+0000 7fb2d599a700 1 --2- 172.21.15.55:0/722617592 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb2b4066f30 0x7fb2b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.071+0000 7fb2d599a700 1 --2- 172.21.15.55:0/722617592 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb2b4066f30 0x7fb2b40693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb2d006eab0 tx=0x7fb2c00091a0 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:46.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.075+0000 7fb2c67fc700 1 -- 172.21.15.55:0/722617592 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2d0148750 con 0x7fb2d00ca300 2023-10-06T04:48:46.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.319+0000 7fb2d719d700 1 -- 172.21.15.55:0/722617592 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2d0148750 con 0x7fb2d00ca300 2023-10-06T04:48:46.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.320+0000 7fb2c67fc700 1 -- 172.21.15.55:0/722617592 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fb2d0148750 con 0x7fb2d00ca300 2023-10-06T04:48:46.322 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:46.322 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8308,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/421768289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":421768289},{"type":"v1","addr":"172.21.15.55:6835","nonce":421768289}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8311,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2920562647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2920562647},{"type":"v1","addr":"172.21.15.163:6834","nonce":2920562647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8314,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2324769448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2324769448},{"type":"v1","addr":"172.21.15.163:6835","nonce":2324769448}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:48:42.309816+0000","modified":"2023-10-06T04:48:45.840578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3504492616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3504492616},{"type":"v1","addr":"172.21.15.55:6834","nonce":3504492616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T04:48:46.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.322+0000 7fb2d719d700 1 -- 172.21.15.55:0/722617592 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb2b4066f30 msgr2=0x7fb2b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.322+0000 7fb2d719d700 1 --2- 172.21.15.55:0/722617592 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb2b4066f30 0x7fb2b40693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb2d006eab0 tx=0x7fb2c00091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.322+0000 7fb2d719d700 1 -- 172.21.15.55:0/722617592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2d00ca300 msgr2=0x7fb2d0145fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.322+0000 7fb2d719d700 1 --2- 172.21.15.55:0/722617592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2d00ca300 0x7fb2d0145fe0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb2d00caf60 tx=0x7fb2cc017000 comp rx=0 tx=0).stop 2023-10-06T04:48:46.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.322+0000 7fb2d719d700 1 -- 172.21.15.55:0/722617592 shutdown_connections 2023-10-06T04:48:46.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.322+0000 7fb2d719d700 1 --2- 172.21.15.55:0/722617592 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb2b4066f30 0x7fb2b40693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.322+0000 7fb2d719d700 1 --2- 172.21.15.55:0/722617592 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb2d01566e0 0x7fb2d014aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.322+0000 7fb2d719d700 1 --2- 172.21.15.55:0/722617592 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb2d00ca300 0x7fb2d0145fe0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.322+0000 7fb2d719d700 1 --2- 172.21.15.55:0/722617592 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb2d0156310 0x7fb2d014aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.322+0000 7fb2d719d700 1 -- 172.21.15.55:0/722617592 >> 172.21.15.55:0/722617592 conn(0x7fb2d00cc1f0 msgr2=0x7fb2d00c1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:46.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.322+0000 7fb2d719d700 1 -- 172.21.15.55:0/722617592 shutdown_connections 2023-10-06T04:48:46.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.322+0000 7fb2d719d700 1 -- 172.21.15.55:0/722617592 wait complete. 2023-10-06T04:48:46.325 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 78 2023-10-06T04:48:46.342 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:48:46.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.527+0000 7fce29b9d700 1 Processor -- start 2023-10-06T04:48:46.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.527+0000 7fce29b9d700 1 -- start start 2023-10-06T04:48:46.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.529+0000 7fce29b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fce2406a660 0x7fce24157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.529+0000 7fce29b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2406e2a0 con 0x7fce24150550 2023-10-06T04:48:46.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.529+0000 7fce29b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2406b2e0 con 0x7fce24150180 2023-10-06T04:48:46.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.529+0000 7fce29b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce240c78e0 con 0x7fce2406a660 2023-10-06T04:48:46.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.529+0000 7fce23fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fce24150550 0x7fce24069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37136/0 (socket says 172.21.15.55:37136) 2023-10-06T04:48:46.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.529+0000 7fce23fff700 1 -- 172.21.15.55:0/717704852 learned_addr learned my addr 172.21.15.55:0/717704852 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:46.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.529+0000 7fce2939c700 1 --2- 172.21.15.55:0/717704852 >> v2:172.21.15.55:3300/0 conn(0x7fce2406a660 0x7fce24157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.529+0000 7fce237fe700 1 -- 172.21.15.55:0/717704852 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196343842 0 0) 0x7fce2406e2a0 con 0x7fce24150550 2023-10-06T04:48:46.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.530+0000 7fce237fe700 1 -- 172.21.15.55:0/717704852 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce14003570 con 0x7fce24150550 2023-10-06T04:48:46.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.530+0000 7fce2939c700 1 -- 172.21.15.55:0/717704852 >> v1:172.21.15.163:6789/0 conn(0x7fce24150180 legacy=0x7fce240c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.530+0000 7fce237fe700 1 -- 172.21.15.55:0/717704852 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172379132 0 0) 0x7fce2406b2e0 con 0x7fce24150180 2023-10-06T04:48:46.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.530+0000 7fce2939c700 1 -- 172.21.15.55:0/717704852 >> v1:172.21.15.55:6789/0 conn(0x7fce24150550 legacy=0x7fce24069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.530+0000 7fce2939c700 1 -- 172.21.15.55:0/717704852 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce240c7bc0 con 0x7fce2406a660 2023-10-06T04:48:46.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.530+0000 7fce2939c700 1 --2- 172.21.15.55:0/717704852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce2406a660 0x7fce24157770 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00afa0 tx=0x7fce1c00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512ede27733bc7ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:46.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.530+0000 7fce237fe700 1 -- 172.21.15.55:0/717704852 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00f070 con 0x7fce2406a660 2023-10-06T04:48:46.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.530+0000 7fce237fe700 1 -- 172.21.15.55:0/717704852 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c00a2e0 con 0x7fce2406a660 2023-10-06T04:48:46.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.530+0000 7fce237fe700 1 -- 172.21.15.55:0/717704852 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00cc60 con 0x7fce2406a660 2023-10-06T04:48:46.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.531+0000 7fce29b9d700 1 -- 172.21.15.55:0/717704852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce2406a660 msgr2=0x7fce24157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.531+0000 7fce29b9d700 1 --2- 172.21.15.55:0/717704852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce2406a660 0x7fce24157770 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00afa0 tx=0x7fce1c00a890 comp rx=0 tx=0).stop 2023-10-06T04:48:46.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.531+0000 7fce29b9d700 1 -- 172.21.15.55:0/717704852 shutdown_connections 2023-10-06T04:48:46.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.531+0000 7fce29b9d700 1 --2- 172.21.15.55:0/717704852 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce2406a660 0x7fce24157770 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.531+0000 7fce29b9d700 1 -- 172.21.15.55:0/717704852 >> 172.21.15.55:0/717704852 conn(0x7fce240cc1f0 msgr2=0x7fce240cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:46.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.531+0000 7fce29b9d700 1 -- 172.21.15.55:0/717704852 shutdown_connections 2023-10-06T04:48:46.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.531+0000 7fce29b9d700 1 -- 172.21.15.55:0/717704852 wait complete. 2023-10-06T04:48:46.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.532+0000 7fce29b9d700 1 Processor -- start 2023-10-06T04:48:46.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.532+0000 7fce29b9d700 1 -- start start 2023-10-06T04:48:46.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.533+0000 7fce29b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce2406a660 0x7fce240aa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.533+0000 7fce29b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce24150180 0x7fce240c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.533+0000 7fce29b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce24150550 0x7fce240c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.533+0000 7fce29b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce240ab260 con 0x7fce24150550 2023-10-06T04:48:46.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.533+0000 7fce29b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce240ca5c0 con 0x7fce24150180 2023-10-06T04:48:46.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.533+0000 7fce29b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce240c4ca0 con 0x7fce2406a660 2023-10-06T04:48:46.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.534+0000 7fce2939c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce24150550 0x7fce240c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.534+0000 7fce2939c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce24150550 0x7fce240c32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48408/0 (socket says 172.21.15.55:48408) 2023-10-06T04:48:46.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.534+0000 7fce2939c700 1 -- 172.21.15.55:0/864934917 learned_addr learned my addr 172.21.15.55:0/864934917 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:46.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.534+0000 7fce23fff700 1 --2- 172.21.15.55:0/864934917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce24150180 0x7fce240c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.534+0000 7fce28b9b700 1 --2- 172.21.15.55:0/864934917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce2406a660 0x7fce240aa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.534+0000 7fce2939c700 1 -- 172.21.15.55:0/864934917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce2406a660 msgr2=0x7fce240aa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.534+0000 7fce2939c700 1 --2- 172.21.15.55:0/864934917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce2406a660 0x7fce240aa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.534+0000 7fce2939c700 1 -- 172.21.15.55:0/864934917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce24150180 msgr2=0x7fce240c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.534+0000 7fce2939c700 1 --2- 172.21.15.55:0/864934917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce24150180 0x7fce240c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.534+0000 7fce2939c700 1 -- 172.21.15.55:0/864934917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c00ab30 con 0x7fce24150550 2023-10-06T04:48:46.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.534+0000 7fce2939c700 1 --2- 172.21.15.55:0/864934917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce24150550 0x7fce240c32a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fce2406fe40 tx=0x7fce1c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:46.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.535+0000 7fce20ff9700 1 -- 172.21.15.55:0/864934917 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c009bd0 con 0x7fce24150550 2023-10-06T04:48:46.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.535+0000 7fce20ff9700 1 -- 172.21.15.55:0/864934917 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c009d30 con 0x7fce24150550 2023-10-06T04:48:46.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.535+0000 7fce20ff9700 1 -- 172.21.15.55:0/864934917 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c009ea0 con 0x7fce24150550 2023-10-06T04:48:46.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.535+0000 7fce29b9d700 1 -- 172.21.15.55:0/864934917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce240aa9c0 con 0x7fce24150550 2023-10-06T04:48:46.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.535+0000 7fce29b9d700 1 -- 172.21.15.55:0/864934917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce240aaf80 con 0x7fce24150550 2023-10-06T04:48:46.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.536+0000 7fce29b9d700 1 -- 172.21.15.55:0/864934917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce240ca730 con 0x7fce24150550 2023-10-06T04:48:46.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.537+0000 7fce20ff9700 1 -- 172.21.15.55:0/864934917 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce1c0278f0 con 0x7fce24150550 2023-10-06T04:48:46.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.537+0000 7fce20ff9700 1 --2- 172.21.15.55:0/864934917 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fce08066f30 0x7fce080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.538+0000 7fce20ff9700 1 -- 172.21.15.55:0/864934917 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fce1c086670 con 0x7fce24150550 2023-10-06T04:48:46.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.538+0000 7fce28b9b700 1 --2- 172.21.15.55:0/864934917 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fce08066f30 0x7fce080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.539+0000 7fce28b9b700 1 --2- 172.21.15.55:0/864934917 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fce08066f30 0x7fce080693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fce180011e0 tx=0x7fce18009040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:46.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.542+0000 7fce20ff9700 1 -- 172.21.15.55:0/864934917 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce1c053550 con 0x7fce24150550 2023-10-06T04:48:46.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.747+0000 7fce29b9d700 1 -- 172.21.15.55:0/864934917 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce2404c3c0 con 0x7fce24150550 2023-10-06T04:48:46.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.748+0000 7fce20ff9700 1 -- 172.21.15.55:0/864934917 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+17523 (secure 0 0 0) 0x7fce1c056b70 con 0x7fce24150550 2023-10-06T04:48:46.750 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:46.750 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":160,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T04:48:43.823814+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:48:40.391644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T04:48:41.276409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:48:46.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.750+0000 7fce29b9d700 1 -- 172.21.15.55:0/864934917 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fce08066f30 msgr2=0x7fce080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.750+0000 7fce29b9d700 1 --2- 172.21.15.55:0/864934917 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fce08066f30 0x7fce080693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fce180011e0 tx=0x7fce18009040 comp rx=0 tx=0).stop 2023-10-06T04:48:46.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.750+0000 7fce29b9d700 1 -- 172.21.15.55:0/864934917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce24150550 msgr2=0x7fce240c32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.750+0000 7fce29b9d700 1 --2- 172.21.15.55:0/864934917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce24150550 0x7fce240c32a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fce2406fe40 tx=0x7fce1c019000 comp rx=0 tx=0).stop 2023-10-06T04:48:46.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.750+0000 7fce29b9d700 1 -- 172.21.15.55:0/864934917 shutdown_connections 2023-10-06T04:48:46.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.750+0000 7fce29b9d700 1 --2- 172.21.15.55:0/864934917 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fce08066f30 0x7fce080693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.750+0000 7fce29b9d700 1 --2- 172.21.15.55:0/864934917 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fce24150550 0x7fce240c32a0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.750+0000 7fce29b9d700 1 --2- 172.21.15.55:0/864934917 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fce2406a660 0x7fce240aa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.750+0000 7fce29b9d700 1 --2- 172.21.15.55:0/864934917 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fce24150180 0x7fce240c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.750+0000 7fce29b9d700 1 -- 172.21.15.55:0/864934917 >> 172.21.15.55:0/864934917 conn(0x7fce240cc1f0 msgr2=0x7fce240c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:46.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.750+0000 7fce29b9d700 1 -- 172.21.15.55:0/864934917 shutdown_connections 2023-10-06T04:48:46.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.750+0000 7fce29b9d700 1 -- 172.21.15.55:0/864934917 wait complete. 2023-10-06T04:48:46.770 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T04:48:46.770 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:48:46.781 INFO:tasks.ceph.mds.a.smithi055.stderr:2023-10-06T04:48:46.779+0000 7f8b35bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:48:46.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.957+0000 7fccae39d700 1 Processor -- start 2023-10-06T04:48:46.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.957+0000 7fccae39d700 1 -- start start 2023-10-06T04:48:46.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.958+0000 7fccae39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcca8156310 0x7fcca80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.958+0000 7fccae39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcca81566e0 0x7fcca80ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.958+0000 7fccae39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca80ca860 con 0x7fcca80ca300 2023-10-06T04:48:46.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.958+0000 7fccae39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca80cab40 con 0x7fcca8156310 2023-10-06T04:48:46.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.958+0000 7fccae39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca80cae20 con 0x7fcca81566e0 2023-10-06T04:48:46.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.958+0000 7fccad39b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcca8156310 0x7fcca80c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.958+0000 7fccad39b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcca8156310 0x7fcca80c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48416/0 (socket says 172.21.15.55:48416) 2023-10-06T04:48:46.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.958+0000 7fccad39b700 1 -- 172.21.15.55:0/2768515764 learned_addr learned my addr 172.21.15.55:0/2768515764 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:46.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.959+0000 7fccacb9a700 1 --2- 172.21.15.55:0/2768515764 >> v2:172.21.15.163:3300/0 conn(0x7fcca81566e0 0x7fcca80ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.959+0000 7fccad39b700 1 -- 172.21.15.55:0/2768515764 >> v2:172.21.15.163:3300/0 conn(0x7fcca81566e0 msgr2=0x7fcca80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:46.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.959+0000 7fccad39b700 1 --2- 172.21.15.55:0/2768515764 >> v2:172.21.15.163:3300/0 conn(0x7fcca81566e0 0x7fcca80ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.959+0000 7fccad39b700 1 -- 172.21.15.55:0/2768515764 >> v1:172.21.15.163:6789/0 conn(0x7fcca80ca300 legacy=0x7fcca80c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:46.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.959+0000 7fccad39b700 1 -- 172.21.15.55:0/2768515764 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca80cb100 con 0x7fcca8156310 2023-10-06T04:48:46.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.959+0000 7fccacb9a700 1 --2- 172.21.15.55:0/2768515764 >> v2:172.21.15.163:3300/0 conn(0x7fcca81566e0 0x7fcca80ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:46.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.959+0000 7fccad39b700 1 --2- 172.21.15.55:0/2768515764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcca8156310 0x7fcca80c9dc0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fcca806c000 tx=0x7fcca4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78e7a2817b524bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:46.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.959+0000 7fcc9ffff700 1 -- 172.21.15.55:0/2768515764 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca400b820 con 0x7fcca8156310 2023-10-06T04:48:46.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.959+0000 7fcc9ffff700 1 -- 172.21.15.55:0/2768515764 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca400b980 con 0x7fcca8156310 2023-10-06T04:48:46.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.959+0000 7fcc9ffff700 1 -- 172.21.15.55:0/2768515764 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca400bd20 con 0x7fcca8156310 2023-10-06T04:48:46.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.960+0000 7fccae39d700 1 -- 172.21.15.55:0/2768515764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcca8156310 msgr2=0x7fcca80c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.960+0000 7fccae39d700 1 --2- 172.21.15.55:0/2768515764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcca8156310 0x7fcca80c9dc0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fcca806c000 tx=0x7fcca4009250 comp rx=0 tx=0).stop 2023-10-06T04:48:46.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.960+0000 7fccae39d700 1 -- 172.21.15.55:0/2768515764 shutdown_connections 2023-10-06T04:48:46.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.960+0000 7fccae39d700 1 --2- 172.21.15.55:0/2768515764 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcca8156310 0x7fcca80c9dc0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.960+0000 7fccae39d700 1 --2- 172.21.15.55:0/2768515764 >> v2:172.21.15.163:3300/0 conn(0x7fcca81566e0 0x7fcca80ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.960+0000 7fccae39d700 1 -- 172.21.15.55:0/2768515764 >> 172.21.15.55:0/2768515764 conn(0x7fcca80cc1f0 msgr2=0x7fcca80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:46.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.960+0000 7fccae39d700 1 -- 172.21.15.55:0/2768515764 shutdown_connections 2023-10-06T04:48:46.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.960+0000 7fccae39d700 1 -- 172.21.15.55:0/2768515764 wait complete. 2023-10-06T04:48:46.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.961+0000 7fccae39d700 1 Processor -- start 2023-10-06T04:48:46.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.962+0000 7fccae39d700 1 -- start start 2023-10-06T04:48:46.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.962+0000 7fccae39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcca80ca300 0x7fcca80c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.962+0000 7fccae39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcca8156310 0x7fcca8155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.963+0000 7fccae39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcca81566e0 0x7fcca8143c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.963+0000 7fccae39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca80c5e40 con 0x7fcca80ca300 2023-10-06T04:48:46.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.963+0000 7fccae39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca80c84b0 con 0x7fcca81566e0 2023-10-06T04:48:46.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.963+0000 7fccae39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8155b40 con 0x7fcca8156310 2023-10-06T04:48:46.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.963+0000 7fccad39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcca80ca300 0x7fcca80c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.963+0000 7fccad39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcca80ca300 0x7fcca80c7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48422/0 (socket says 172.21.15.55:48422) 2023-10-06T04:48:46.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.963+0000 7fccad39b700 1 -- 172.21.15.55:0/3323424283 learned_addr learned my addr 172.21.15.55:0/3323424283 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:46.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.963+0000 7fccacb9a700 1 --2- 172.21.15.55:0/3323424283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcca8156310 0x7fcca8155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.963+0000 7fccadb9c700 1 --2- 172.21.15.55:0/3323424283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcca81566e0 0x7fcca8143c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.963+0000 7fccad39b700 1 -- 172.21.15.55:0/3323424283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcca8156310 msgr2=0x7fcca8155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.963+0000 7fccad39b700 1 --2- 172.21.15.55:0/3323424283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcca8156310 0x7fcca8155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.963+0000 7fccad39b700 1 -- 172.21.15.55:0/3323424283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcca81566e0 msgr2=0x7fcca8143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:46.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.963+0000 7fccad39b700 1 --2- 172.21.15.55:0/3323424283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcca81566e0 0x7fcca8143c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:46.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.963+0000 7fccad39b700 1 -- 172.21.15.55:0/3323424283 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca40094f0 con 0x7fcca80ca300 2023-10-06T04:48:46.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.964+0000 7fccad39b700 1 --2- 172.21.15.55:0/3323424283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcca80ca300 0x7fcca80c7200 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fcca40030c0 tx=0x7fcca4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:46.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.964+0000 7fcc9d7fa700 1 -- 172.21.15.55:0/3323424283 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4011d00 con 0x7fcca80ca300 2023-10-06T04:48:46.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.964+0000 7fcc9d7fa700 1 -- 172.21.15.55:0/3323424283 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca4020070 con 0x7fcca80ca300 2023-10-06T04:48:46.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.964+0000 7fcc9d7fa700 1 -- 172.21.15.55:0/3323424283 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca401b6a0 con 0x7fcca80ca300 2023-10-06T04:48:46.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.964+0000 7fccae39d700 1 -- 172.21.15.55:0/3323424283 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca81441b0 con 0x7fcca80ca300 2023-10-06T04:48:46.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.964+0000 7fccae39d700 1 -- 172.21.15.55:0/3323424283 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca80c7b80 con 0x7fcca80ca300 2023-10-06T04:48:46.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.965+0000 7fccae39d700 1 -- 172.21.15.55:0/3323424283 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca804c3c0 con 0x7fcca80ca300 2023-10-06T04:48:46.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.966+0000 7fcc9d7fa700 1 -- 172.21.15.55:0/3323424283 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcca4022030 con 0x7fcca80ca300 2023-10-06T04:48:46.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.967+0000 7fcc9d7fa700 1 --2- 172.21.15.55:0/3323424283 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fcc8c066f30 0x7fcc8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:46.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.967+0000 7fcc9d7fa700 1 -- 172.21.15.55:0/3323424283 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fcca4086680 con 0x7fcca80ca300 2023-10-06T04:48:46.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.968+0000 7fccacb9a700 1 --2- 172.21.15.55:0/3323424283 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fcc8c066f30 0x7fcc8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:46.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.968+0000 7fccacb9a700 1 --2- 172.21.15.55:0/3323424283 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fcc8c066f30 0x7fcc8c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc98006fd0 tx=0x7fcc98008040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:46.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:46.972+0000 7fcc9d7fa700 1 -- 172.21.15.55:0/3323424283 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcca4053560 con 0x7fcca80ca300 2023-10-06T04:48:47.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.169+0000 7fccae39d700 1 -- 172.21.15.55:0/3323424283 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcca80ca190 con 0x7fcc8c066f30 2023-10-06T04:48:47.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.181+0000 7fcc9d7fa700 1 -- 172.21.15.55:0/3323424283 <== mgr.7205 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311672 (secure 0 0 0) 0x7fcca80ca190 con 0x7fcc8c066f30 2023-10-06T04:48:47.187 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:47.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.185+0000 7fccae39d700 1 -- 172.21.15.55:0/3323424283 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fcc8c066f30 msgr2=0x7fcc8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:47.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.185+0000 7fccae39d700 1 --2- 172.21.15.55:0/3323424283 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fcc8c066f30 0x7fcc8c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc98006fd0 tx=0x7fcc98008040 comp rx=0 tx=0).stop 2023-10-06T04:48:47.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.185+0000 7fccae39d700 1 -- 172.21.15.55:0/3323424283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcca80ca300 msgr2=0x7fcca80c7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:47.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.185+0000 7fccae39d700 1 --2- 172.21.15.55:0/3323424283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcca80ca300 0x7fcca80c7200 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fcca40030c0 tx=0x7fcca4016000 comp rx=0 tx=0).stop 2023-10-06T04:48:47.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.185+0000 7fccae39d700 1 -- 172.21.15.55:0/3323424283 shutdown_connections 2023-10-06T04:48:47.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.185+0000 7fccae39d700 1 --2- 172.21.15.55:0/3323424283 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fcc8c066f30 0x7fcc8c0693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.185+0000 7fccae39d700 1 --2- 172.21.15.55:0/3323424283 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcca81566e0 0x7fcca8143c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.185+0000 7fccae39d700 1 --2- 172.21.15.55:0/3323424283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcca80ca300 0x7fcca80c7200 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.185+0000 7fccae39d700 1 --2- 172.21.15.55:0/3323424283 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcca8156310 0x7fcca8155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.185+0000 7fccae39d700 1 -- 172.21.15.55:0/3323424283 >> 172.21.15.55:0/3323424283 conn(0x7fcca80cc1f0 msgr2=0x7fcca8150db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:47.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.185+0000 7fccae39d700 1 -- 172.21.15.55:0/3323424283 shutdown_connections 2023-10-06T04:48:47.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.185+0000 7fccae39d700 1 -- 172.21.15.55:0/3323424283 wait complete. 2023-10-06T04:48:47.190 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T04:48:47.205 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T04:48:46.965696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":483,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2452396,"kb_used_data":16748,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752522324,"statfs":{"total":773094113280,"available":770582859776,"internally_reserved":0,"allocated":17149952,"data_stored":9518291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001578"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.836867+0000","last_change":"2023-10-06T04:48:42.830898+0000","last_active":"2023-10-06T04:48:42.836867+0000","last_peered":"2023-10-06T04:48:42.836867+0000","last_clean":"2023-10-06T04:48:42.836867+0000","last_became_active":"2023-10-06T04:48:42.830631+0000","last_became_peered":"2023-10-06T04:48:42.830631+0000","last_unstale":"2023-10-06T04:48:42.836867+0000","last_undegraded":"2023-10-06T04:48:42.836867+0000","last_fullsized":"2023-10-06T04:48:42.836867+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:18.162595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.838948+0000","last_change":"2023-10-06T04:48:42.832939+0000","last_active":"2023-10-06T04:48:42.838948+0000","last_peered":"2023-10-06T04:48:42.838948+0000","last_clean":"2023-10-06T04:48:42.838948+0000","last_became_active":"2023-10-06T04:48:42.832032+0000","last_became_peered":"2023-10-06T04:48:42.832032+0000","last_unstale":"2023-10-06T04:48:42.838948+0000","last_undegraded":"2023-10-06T04:48:42.838948+0000","last_fullsized":"2023-10-06T04:48:42.838948+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:26.704018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.834818+0000","last_change":"2023-10-06T04:48:42.827090+0000","last_active":"2023-10-06T04:48:42.834818+0000","last_peered":"2023-10-06T04:48:42.834818+0000","last_clean":"2023-10-06T04:48:42.834818+0000","last_became_active":"2023-10-06T04:48:42.826789+0000","last_became_peered":"2023-10-06T04:48:42.826789+0000","last_unstale":"2023-10-06T04:48:42.834818+0000","last_undegraded":"2023-10-06T04:48:42.834818+0000","last_fullsized":"2023-10-06T04:48:42.834818+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:28.443970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.840088+0000","last_change":"2023-10-06T04:48:42.834501+0000","last_active":"2023-10-06T04:48:42.840088+0000","last_peered":"2023-10-06T04:48:42.840088+0000","last_clean":"2023-10-06T04:48:42.840088+0000","last_became_active":"2023-10-06T04:48:42.834170+0000","last_became_peered":"2023-10-06T04:48:42.834170+0000","last_unstale":"2023-10-06T04:48:42.840088+0000","last_undegraded":"2023-10-06T04:48:42.840088+0000","last_fullsized":"2023-10-06T04:48:42.840088+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:40.683509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.837667+0000","last_change":"2023-10-06T04:48:42.833687+0000","last_active":"2023-10-06T04:48:42.837667+0000","last_peered":"2023-10-06T04:48:42.837667+0000","last_clean":"2023-10-06T04:48:42.837667+0000","last_became_active":"2023-10-06T04:48:42.833395+0000","last_became_peered":"2023-10-06T04:48:42.833395+0000","last_unstale":"2023-10-06T04:48:42.837667+0000","last_undegraded":"2023-10-06T04:48:42.837667+0000","last_fullsized":"2023-10-06T04:48:42.837667+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:26.704018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839516+0000","last_change":"2023-10-06T04:48:42.831555+0000","last_active":"2023-10-06T04:48:42.839516+0000","last_peered":"2023-10-06T04:48:42.839516+0000","last_clean":"2023-10-06T04:48:42.839516+0000","last_became_active":"2023-10-06T04:48:42.831159+0000","last_became_peered":"2023-10-06T04:48:42.831159+0000","last_unstale":"2023-10-06T04:48:42.839516+0000","last_undegraded":"2023-10-06T04:48:42.839516+0000","last_fullsized":"2023-10-06T04:48:42.839516+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:51.999284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841625+0000","last_change":"2023-10-06T04:48:42.839020+0000","last_active":"2023-10-06T04:48:42.841625+0000","last_peered":"2023-10-06T04:48:42.841625+0000","last_clean":"2023-10-06T04:48:42.841625+0000","last_became_active":"2023-10-06T04:48:42.838741+0000","last_became_peered":"2023-10-06T04:48:42.838741+0000","last_unstale":"2023-10-06T04:48:42.841625+0000","last_undegraded":"2023-10-06T04:48:42.841625+0000","last_fullsized":"2023-10-06T04:48:42.841625+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:17.582235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.831473+0000","last_change":"2023-10-06T04:48:42.827117+0000","last_active":"2023-10-06T04:48:42.831473+0000","last_peered":"2023-10-06T04:48:42.831473+0000","last_clean":"2023-10-06T04:48:42.831473+0000","last_became_active":"2023-10-06T04:48:42.826836+0000","last_became_peered":"2023-10-06T04:48:42.826836+0000","last_unstale":"2023-10-06T04:48:42.831473+0000","last_undegraded":"2023-10-06T04:48:42.831473+0000","last_fullsized":"2023-10-06T04:48:42.831473+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:51.911529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839555+0000","last_change":"2023-10-06T04:48:42.833226+0000","last_active":"2023-10-06T04:48:42.839555+0000","last_peered":"2023-10-06T04:48:42.839555+0000","last_clean":"2023-10-06T04:48:42.839555+0000","last_became_active":"2023-10-06T04:48:42.832982+0000","last_became_peered":"2023-10-06T04:48:42.832982+0000","last_unstale":"2023-10-06T04:48:42.839555+0000","last_undegraded":"2023-10-06T04:48:42.839555+0000","last_fullsized":"2023-10-06T04:48:42.839555+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:40.683509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.825011+0000","last_change":"2023-10-06T04:48:42.820839+0000","last_active":"2023-10-06T04:48:42.825011+0000","last_peered":"2023-10-06T04:48:42.825011+0000","last_clean":"2023-10-06T04:48:42.825011+0000","last_became_active":"2023-10-06T04:48:42.820573+0000","last_became_peered":"2023-10-06T04:48:42.820573+0000","last_unstale":"2023-10-06T04:48:42.825011+0000","last_undegraded":"2023-10-06T04:48:42.825011+0000","last_fullsized":"2023-10-06T04:48:42.825011+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:17.582235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.840478+0000","last_change":"2023-10-06T04:48:42.838179+0000","last_active":"2023-10-06T04:48:42.840478+0000","last_peered":"2023-10-06T04:48:42.840478+0000","last_clean":"2023-10-06T04:48:42.840478+0000","last_became_active":"2023-10-06T04:48:42.837911+0000","last_became_peered":"2023-10-06T04:48:42.837911+0000","last_unstale":"2023-10-06T04:48:42.840478+0000","last_undegraded":"2023-10-06T04:48:42.840478+0000","last_fullsized":"2023-10-06T04:48:42.840478+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:10.473217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.828076+0000","last_change":"2023-10-06T04:48:42.825994+0000","last_active":"2023-10-06T04:48:42.828076+0000","last_peered":"2023-10-06T04:48:42.828076+0000","last_clean":"2023-10-06T04:48:42.828076+0000","last_became_active":"2023-10-06T04:48:42.825577+0000","last_became_peered":"2023-10-06T04:48:42.825577+0000","last_unstale":"2023-10-06T04:48:42.828076+0000","last_undegraded":"2023-10-06T04:48:42.828076+0000","last_fullsized":"2023-10-06T04:48:42.828076+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:38.137243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839717+0000","last_change":"2023-10-06T04:48:42.833780+0000","last_active":"2023-10-06T04:48:42.839717+0000","last_peered":"2023-10-06T04:48:42.839717+0000","last_clean":"2023-10-06T04:48:42.839717+0000","last_became_active":"2023-10-06T04:48:42.833514+0000","last_became_peered":"2023-10-06T04:48:42.833514+0000","last_unstale":"2023-10-06T04:48:42.839717+0000","last_undegraded":"2023-10-06T04:48:42.839717+0000","last_fullsized":"2023-10-06T04:48:42.839717+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.321447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.845609+0000","last_change":"2023-10-06T04:48:42.843808+0000","last_active":"2023-10-06T04:48:42.845609+0000","last_peered":"2023-10-06T04:48:42.845609+0000","last_clean":"2023-10-06T04:48:42.845609+0000","last_became_active":"2023-10-06T04:48:42.843538+0000","last_became_peered":"2023-10-06T04:48:42.843538+0000","last_unstale":"2023-10-06T04:48:42.845609+0000","last_undegraded":"2023-10-06T04:48:42.845609+0000","last_fullsized":"2023-10-06T04:48:42.845609+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:28.750214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.845649+0000","last_change":"2023-10-06T04:48:42.844400+0000","last_active":"2023-10-06T04:48:42.845649+0000","last_peered":"2023-10-06T04:48:42.845649+0000","last_clean":"2023-10-06T04:48:42.845649+0000","last_became_active":"2023-10-06T04:48:42.844162+0000","last_became_peered":"2023-10-06T04:48:42.844162+0000","last_unstale":"2023-10-06T04:48:42.845649+0000","last_undegraded":"2023-10-06T04:48:42.845649+0000","last_fullsized":"2023-10-06T04:48:42.845649+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:39.033740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839551+0000","last_change":"2023-10-06T04:48:42.833118+0000","last_active":"2023-10-06T04:48:42.839551+0000","last_peered":"2023-10-06T04:48:42.839551+0000","last_clean":"2023-10-06T04:48:42.839551+0000","last_became_active":"2023-10-06T04:48:42.831731+0000","last_became_peered":"2023-10-06T04:48:42.831731+0000","last_unstale":"2023-10-06T04:48:42.839551+0000","last_undegraded":"2023-10-06T04:48:42.839551+0000","last_fullsized":"2023-10-06T04:48:42.839551+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:46.968018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839577+0000","last_change":"2023-10-06T04:48:42.833426+0000","last_active":"2023-10-06T04:48:42.839577+0000","last_peered":"2023-10-06T04:48:42.839577+0000","last_clean":"2023-10-06T04:48:42.839577+0000","last_became_active":"2023-10-06T04:48:42.831599+0000","last_became_peered":"2023-10-06T04:48:42.831599+0000","last_unstale":"2023-10-06T04:48:42.839577+0000","last_undegraded":"2023-10-06T04:48:42.839577+0000","last_fullsized":"2023-10-06T04:48:42.839577+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:36.738377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.837700+0000","last_change":"2023-10-06T04:48:42.832590+0000","last_active":"2023-10-06T04:48:42.837700+0000","last_peered":"2023-10-06T04:48:42.837700+0000","last_clean":"2023-10-06T04:48:42.837700+0000","last_became_active":"2023-10-06T04:48:42.832280+0000","last_became_peered":"2023-10-06T04:48:42.832280+0000","last_unstale":"2023-10-06T04:48:42.837700+0000","last_undegraded":"2023-10-06T04:48:42.837700+0000","last_fullsized":"2023-10-06T04:48:42.837700+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:45.803686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841033+0000","last_change":"2023-10-06T04:48:42.834952+0000","last_active":"2023-10-06T04:48:42.841033+0000","last_peered":"2023-10-06T04:48:42.841033+0000","last_clean":"2023-10-06T04:48:42.841033+0000","last_became_active":"2023-10-06T04:48:42.833442+0000","last_became_peered":"2023-10-06T04:48:42.833442+0000","last_unstale":"2023-10-06T04:48:42.841033+0000","last_undegraded":"2023-10-06T04:48:42.841033+0000","last_fullsized":"2023-10-06T04:48:42.841033+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:02.166083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.838848+0000","last_change":"2023-10-06T04:48:42.835316+0000","last_active":"2023-10-06T04:48:42.838848+0000","last_peered":"2023-10-06T04:48:42.838848+0000","last_clean":"2023-10-06T04:48:42.838848+0000","last_became_active":"2023-10-06T04:48:42.835087+0000","last_became_peered":"2023-10-06T04:48:42.835087+0000","last_unstale":"2023-10-06T04:48:42.838848+0000","last_undegraded":"2023-10-06T04:48:42.838848+0000","last_fullsized":"2023-10-06T04:48:42.838848+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:09.980476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841742+0000","last_change":"2023-10-06T04:48:42.840730+0000","last_active":"2023-10-06T04:48:42.841742+0000","last_peered":"2023-10-06T04:48:42.841742+0000","last_clean":"2023-10-06T04:48:42.841742+0000","last_became_active":"2023-10-06T04:48:42.840410+0000","last_became_peered":"2023-10-06T04:48:42.840410+0000","last_unstale":"2023-10-06T04:48:42.841742+0000","last_undegraded":"2023-10-06T04:48:42.841742+0000","last_fullsized":"2023-10-06T04:48:42.841742+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.321447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.838997+0000","last_change":"2023-10-06T04:48:42.836729+0000","last_active":"2023-10-06T04:48:42.838997+0000","last_peered":"2023-10-06T04:48:42.838997+0000","last_clean":"2023-10-06T04:48:42.838997+0000","last_became_active":"2023-10-06T04:48:42.836464+0000","last_became_peered":"2023-10-06T04:48:42.836464+0000","last_unstale":"2023-10-06T04:48:42.838997+0000","last_undegraded":"2023-10-06T04:48:42.838997+0000","last_fullsized":"2023-10-06T04:48:42.838997+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:29.694522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.840119+0000","last_change":"2023-10-06T04:48:42.833764+0000","last_active":"2023-10-06T04:48:42.840119+0000","last_peered":"2023-10-06T04:48:42.840119+0000","last_clean":"2023-10-06T04:48:42.840119+0000","last_became_active":"2023-10-06T04:48:42.833472+0000","last_became_peered":"2023-10-06T04:48:42.833472+0000","last_unstale":"2023-10-06T04:48:42.840119+0000","last_undegraded":"2023-10-06T04:48:42.840119+0000","last_fullsized":"2023-10-06T04:48:42.840119+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:09.980476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841667+0000","last_change":"2023-10-06T04:48:42.833934+0000","last_active":"2023-10-06T04:48:42.841667+0000","last_peered":"2023-10-06T04:48:42.841667+0000","last_clean":"2023-10-06T04:48:42.841667+0000","last_became_active":"2023-10-06T04:48:42.833651+0000","last_became_peered":"2023-10-06T04:48:42.833651+0000","last_unstale":"2023-10-06T04:48:42.841667+0000","last_undegraded":"2023-10-06T04:48:42.841667+0000","last_fullsized":"2023-10-06T04:48:42.841667+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:24:40.398908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.831427+0000","last_change":"2023-10-06T04:48:42.820806+0000","last_active":"2023-10-06T04:48:42.831427+0000","last_peered":"2023-10-06T04:48:42.831427+0000","last_clean":"2023-10-06T04:48:42.831427+0000","last_became_active":"2023-10-06T04:48:42.820460+0000","last_became_peered":"2023-10-06T04:48:42.820460+0000","last_unstale":"2023-10-06T04:48:42.831427+0000","last_undegraded":"2023-10-06T04:48:42.831427+0000","last_fullsized":"2023-10-06T04:48:42.831427+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:14.349500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.843090+0000","last_change":"2023-10-06T04:48:42.838714+0000","last_active":"2023-10-06T04:48:42.843090+0000","last_peered":"2023-10-06T04:48:42.843090+0000","last_clean":"2023-10-06T04:48:42.843090+0000","last_became_active":"2023-10-06T04:48:42.838378+0000","last_became_peered":"2023-10-06T04:48:42.838378+0000","last_unstale":"2023-10-06T04:48:42.843090+0000","last_undegraded":"2023-10-06T04:48:42.843090+0000","last_fullsized":"2023-10-06T04:48:42.843090+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:28.725438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849736+0000","last_change":"2023-10-06T04:48:42.847388+0000","last_active":"2023-10-06T04:48:42.849736+0000","last_peered":"2023-10-06T04:48:42.849736+0000","last_clean":"2023-10-06T04:48:42.849736+0000","last_became_active":"2023-10-06T04:48:42.844311+0000","last_became_peered":"2023-10-06T04:48:42.844311+0000","last_unstale":"2023-10-06T04:48:42.849736+0000","last_undegraded":"2023-10-06T04:48:42.849736+0000","last_fullsized":"2023-10-06T04:48:42.849736+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:47.069520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841492+0000","last_change":"2023-10-06T04:48:42.827854+0000","last_active":"2023-10-06T04:48:42.841492+0000","last_peered":"2023-10-06T04:48:42.841492+0000","last_clean":"2023-10-06T04:48:42.841492+0000","last_became_active":"2023-10-06T04:48:42.827611+0000","last_became_peered":"2023-10-06T04:48:42.827611+0000","last_unstale":"2023-10-06T04:48:42.841492+0000","last_undegraded":"2023-10-06T04:48:42.841492+0000","last_fullsized":"2023-10-06T04:48:42.841492+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:51.457979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.845022+0000","last_change":"2023-10-06T04:48:42.841510+0000","last_active":"2023-10-06T04:48:42.845022+0000","last_peered":"2023-10-06T04:48:42.845022+0000","last_clean":"2023-10-06T04:48:42.845022+0000","last_became_active":"2023-10-06T04:48:42.841203+0000","last_became_peered":"2023-10-06T04:48:42.841203+0000","last_unstale":"2023-10-06T04:48:42.845022+0000","last_undegraded":"2023-10-06T04:48:42.845022+0000","last_fullsized":"2023-10-06T04:48:42.845022+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:54.317378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.835548+0000","last_change":"2023-10-06T04:48:42.823328+0000","last_active":"2023-10-06T04:48:42.835548+0000","last_peered":"2023-10-06T04:48:42.835548+0000","last_clean":"2023-10-06T04:48:42.835548+0000","last_became_active":"2023-10-06T04:48:42.823062+0000","last_became_peered":"2023-10-06T04:48:42.823062+0000","last_unstale":"2023-10-06T04:48:42.835548+0000","last_undegraded":"2023-10-06T04:48:42.835548+0000","last_fullsized":"2023-10-06T04:48:42.835548+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:01:24.878062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.845375+0000","last_change":"2023-10-06T04:48:42.841657+0000","last_active":"2023-10-06T04:48:42.845375+0000","last_peered":"2023-10-06T04:48:42.845375+0000","last_clean":"2023-10-06T04:48:42.845375+0000","last_became_active":"2023-10-06T04:48:42.841327+0000","last_became_peered":"2023-10-06T04:48:42.841327+0000","last_unstale":"2023-10-06T04:48:42.845375+0000","last_undegraded":"2023-10-06T04:48:42.845375+0000","last_fullsized":"2023-10-06T04:48:42.845375+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:17.582235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.842529+0000","last_change":"2023-10-06T04:48:42.833617+0000","last_active":"2023-10-06T04:48:42.842529+0000","last_peered":"2023-10-06T04:48:42.842529+0000","last_clean":"2023-10-06T04:48:42.842529+0000","last_became_active":"2023-10-06T04:48:42.833372+0000","last_became_peered":"2023-10-06T04:48:42.833372+0000","last_unstale":"2023-10-06T04:48:42.842529+0000","last_undegraded":"2023-10-06T04:48:42.842529+0000","last_fullsized":"2023-10-06T04:48:42.842529+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:15.978210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.831493+0000","last_change":"2023-10-06T04:48:42.828556+0000","last_active":"2023-10-06T04:48:42.831493+0000","last_peered":"2023-10-06T04:48:42.831493+0000","last_clean":"2023-10-06T04:48:42.831493+0000","last_became_active":"2023-10-06T04:48:42.828289+0000","last_became_peered":"2023-10-06T04:48:42.828289+0000","last_unstale":"2023-10-06T04:48:42.831493+0000","last_undegraded":"2023-10-06T04:48:42.831493+0000","last_fullsized":"2023-10-06T04:48:42.831493+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:35.937113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"159'4","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849434+0000","last_change":"2023-10-06T04:48:41.834539+0000","last_active":"2023-10-06T04:48:42.849434+0000","last_peered":"2023-10-06T04:48:42.849434+0000","last_clean":"2023-10-06T04:48:42.849434+0000","last_became_active":"2023-10-06T04:48:41.833970+0000","last_became_peered":"2023-10-06T04:48:41.833970+0000","last_unstale":"2023-10-06T04:48:42.849434+0000","last_undegraded":"2023-10-06T04:48:42.849434+0000","last_fullsized":"2023-10-06T04:48:42.849434+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:06.796613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839033+0000","last_change":"2023-10-06T04:48:42.836745+0000","last_active":"2023-10-06T04:48:42.839033+0000","last_peered":"2023-10-06T04:48:42.839033+0000","last_clean":"2023-10-06T04:48:42.839033+0000","last_became_active":"2023-10-06T04:48:42.836497+0000","last_became_peered":"2023-10-06T04:48:42.836497+0000","last_unstale":"2023-10-06T04:48:42.839033+0000","last_undegraded":"2023-10-06T04:48:42.839033+0000","last_fullsized":"2023-10-06T04:48:42.839033+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:11.590289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"159'1","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.833930+0000","last_change":"2023-10-06T04:48:41.828697+0000","last_active":"2023-10-06T04:48:42.833930+0000","last_peered":"2023-10-06T04:48:42.833930+0000","last_clean":"2023-10-06T04:48:42.833930+0000","last_became_active":"2023-10-06T04:48:41.828450+0000","last_became_peered":"2023-10-06T04:48:41.828450+0000","last_unstale":"2023-10-06T04:48:42.833930+0000","last_undegraded":"2023-10-06T04:48:42.833930+0000","last_fullsized":"2023-10-06T04:48:42.833930+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:57.479209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.840494+0000","last_change":"2023-10-06T04:48:42.837260+0000","last_active":"2023-10-06T04:48:42.840494+0000","last_peered":"2023-10-06T04:48:42.840494+0000","last_clean":"2023-10-06T04:48:42.840494+0000","last_became_active":"2023-10-06T04:48:42.836999+0000","last_became_peered":"2023-10-06T04:48:42.836999+0000","last_unstale":"2023-10-06T04:48:42.840494+0000","last_undegraded":"2023-10-06T04:48:42.840494+0000","last_fullsized":"2023-10-06T04:48:42.840494+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:07.471267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"159'2","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849541+0000","last_change":"2023-10-06T04:48:41.833222+0000","last_active":"2023-10-06T04:48:42.849541+0000","last_peered":"2023-10-06T04:48:42.849541+0000","last_clean":"2023-10-06T04:48:42.849541+0000","last_became_active":"2023-10-06T04:48:41.832978+0000","last_became_peered":"2023-10-06T04:48:41.832978+0000","last_unstale":"2023-10-06T04:48:42.849541+0000","last_undegraded":"2023-10-06T04:48:42.849541+0000","last_fullsized":"2023-10-06T04:48:42.849541+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:40.858246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.840510+0000","last_change":"2023-10-06T04:48:42.838653+0000","last_active":"2023-10-06T04:48:42.840510+0000","last_peered":"2023-10-06T04:48:42.840510+0000","last_clean":"2023-10-06T04:48:42.840510+0000","last_became_active":"2023-10-06T04:48:42.838373+0000","last_became_peered":"2023-10-06T04:48:42.838373+0000","last_unstale":"2023-10-06T04:48:42.840510+0000","last_undegraded":"2023-10-06T04:48:42.840510+0000","last_fullsized":"2023-10-06T04:48:42.840510+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:57.396083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"159'2","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.848854+0000","last_change":"2023-10-06T04:48:41.829961+0000","last_active":"2023-10-06T04:48:42.848854+0000","last_peered":"2023-10-06T04:48:42.848854+0000","last_clean":"2023-10-06T04:48:42.848854+0000","last_became_active":"2023-10-06T04:48:41.829712+0000","last_became_peered":"2023-10-06T04:48:41.829712+0000","last_unstale":"2023-10-06T04:48:42.848854+0000","last_undegraded":"2023-10-06T04:48:42.848854+0000","last_fullsized":"2023-10-06T04:48:42.848854+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:58.474149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.845428+0000","last_change":"2023-10-06T04:48:42.842586+0000","last_active":"2023-10-06T04:48:42.845428+0000","last_peered":"2023-10-06T04:48:42.845428+0000","last_clean":"2023-10-06T04:48:42.845428+0000","last_became_active":"2023-10-06T04:48:42.841482+0000","last_became_peered":"2023-10-06T04:48:42.841482+0000","last_unstale":"2023-10-06T04:48:42.845428+0000","last_undegraded":"2023-10-06T04:48:42.845428+0000","last_fullsized":"2023-10-06T04:48:42.845428+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:47.069520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"159'1","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.843348+0000","last_change":"2023-10-06T04:48:41.830277+0000","last_active":"2023-10-06T04:48:42.843348+0000","last_peered":"2023-10-06T04:48:42.843348+0000","last_clean":"2023-10-06T04:48:42.843348+0000","last_became_active":"2023-10-06T04:48:41.829982+0000","last_became_peered":"2023-10-06T04:48:41.829982+0000","last_unstale":"2023-10-06T04:48:42.843348+0000","last_undegraded":"2023-10-06T04:48:42.843348+0000","last_fullsized":"2023-10-06T04:48:42.843348+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:03.473106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.831436+0000","last_change":"2023-10-06T04:48:42.826353+0000","last_active":"2023-10-06T04:48:42.831436+0000","last_peered":"2023-10-06T04:48:42.831436+0000","last_clean":"2023-10-06T04:48:42.831436+0000","last_became_active":"2023-10-06T04:48:42.826098+0000","last_became_peered":"2023-10-06T04:48:42.826098+0000","last_unstale":"2023-10-06T04:48:42.831436+0000","last_undegraded":"2023-10-06T04:48:42.831436+0000","last_fullsized":"2023-10-06T04:48:42.831436+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:07.264476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"159'1","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.834218+0000","last_change":"2023-10-06T04:48:41.831647+0000","last_active":"2023-10-06T04:48:42.834218+0000","last_peered":"2023-10-06T04:48:42.834218+0000","last_clean":"2023-10-06T04:48:42.834218+0000","last_became_active":"2023-10-06T04:48:41.829510+0000","last_became_peered":"2023-10-06T04:48:41.829510+0000","last_unstale":"2023-10-06T04:48:42.834218+0000","last_undegraded":"2023-10-06T04:48:42.834218+0000","last_fullsized":"2023-10-06T04:48:42.834218+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:37.705377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.838856+0000","last_change":"2023-10-06T04:48:42.835307+0000","last_active":"2023-10-06T04:48:42.838856+0000","last_peered":"2023-10-06T04:48:42.838856+0000","last_clean":"2023-10-06T04:48:42.838856+0000","last_became_active":"2023-10-06T04:48:42.835059+0000","last_became_peered":"2023-10-06T04:48:42.835059+0000","last_unstale":"2023-10-06T04:48:42.838856+0000","last_undegraded":"2023-10-06T04:48:42.838856+0000","last_fullsized":"2023-10-06T04:48:42.838856+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.321447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.818061+0000","last_change":"2023-10-06T04:48:41.833865+0000","last_active":"2023-10-06T04:48:42.818061+0000","last_peered":"2023-10-06T04:48:42.818061+0000","last_clean":"2023-10-06T04:48:42.818061+0000","last_became_active":"2023-10-06T04:48:41.833468+0000","last_became_peered":"2023-10-06T04:48:41.833468+0000","last_unstale":"2023-10-06T04:48:42.818061+0000","last_undegraded":"2023-10-06T04:48:42.818061+0000","last_fullsized":"2023-10-06T04:48:42.818061+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:31.508469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849641+0000","last_change":"2023-10-06T04:48:42.846495+0000","last_active":"2023-10-06T04:48:42.849641+0000","last_peered":"2023-10-06T04:48:42.849641+0000","last_clean":"2023-10-06T04:48:42.849641+0000","last_became_active":"2023-10-06T04:48:42.845649+0000","last_became_peered":"2023-10-06T04:48:42.845649+0000","last_unstale":"2023-10-06T04:48:42.849641+0000","last_undegraded":"2023-10-06T04:48:42.849641+0000","last_fullsized":"2023-10-06T04:48:42.849641+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:51.911529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.817013+0000","last_change":"2023-10-06T04:48:41.829941+0000","last_active":"2023-10-06T04:48:42.817013+0000","last_peered":"2023-10-06T04:48:42.817013+0000","last_clean":"2023-10-06T04:48:42.817013+0000","last_became_active":"2023-10-06T04:48:41.829645+0000","last_became_peered":"2023-10-06T04:48:41.829645+0000","last_unstale":"2023-10-06T04:48:42.817013+0000","last_undegraded":"2023-10-06T04:48:42.817013+0000","last_fullsized":"2023-10-06T04:48:42.817013+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:15.844634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.837987+0000","last_change":"2023-10-06T04:48:42.830724+0000","last_active":"2023-10-06T04:48:42.837987+0000","last_peered":"2023-10-06T04:48:42.837987+0000","last_clean":"2023-10-06T04:48:42.837987+0000","last_became_active":"2023-10-06T04:48:42.830461+0000","last_became_peered":"2023-10-06T04:48:42.830461+0000","last_unstale":"2023-10-06T04:48:42.837987+0000","last_undegraded":"2023-10-06T04:48:42.837987+0000","last_fullsized":"2023-10-06T04:48:42.837987+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:28.648811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"159'1","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.842968+0000","last_change":"2023-10-06T04:48:41.828825+0000","last_active":"2023-10-06T04:48:42.842968+0000","last_peered":"2023-10-06T04:48:42.842968+0000","last_clean":"2023-10-06T04:48:42.842968+0000","last_became_active":"2023-10-06T04:48:41.828516+0000","last_became_peered":"2023-10-06T04:48:41.828516+0000","last_unstale":"2023-10-06T04:48:42.842968+0000","last_undegraded":"2023-10-06T04:48:42.842968+0000","last_fullsized":"2023-10-06T04:48:42.842968+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:41.290550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841097+0000","last_change":"2023-10-06T04:48:42.832064+0000","last_active":"2023-10-06T04:48:42.841097+0000","last_peered":"2023-10-06T04:48:42.841097+0000","last_clean":"2023-10-06T04:48:42.841097+0000","last_became_active":"2023-10-06T04:48:42.831794+0000","last_became_peered":"2023-10-06T04:48:42.831794+0000","last_unstale":"2023-10-06T04:48:42.841097+0000","last_undegraded":"2023-10-06T04:48:42.841097+0000","last_fullsized":"2023-10-06T04:48:42.841097+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:07.264476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"159'2","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849075+0000","last_change":"2023-10-06T04:48:41.830689+0000","last_active":"2023-10-06T04:48:42.849075+0000","last_peered":"2023-10-06T04:48:42.849075+0000","last_clean":"2023-10-06T04:48:42.849075+0000","last_became_active":"2023-10-06T04:48:41.829685+0000","last_became_peered":"2023-10-06T04:48:41.829685+0000","last_unstale":"2023-10-06T04:48:42.849075+0000","last_undegraded":"2023-10-06T04:48:42.849075+0000","last_fullsized":"2023-10-06T04:48:42.849075+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:24.601235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849566+0000","last_change":"2023-10-06T04:48:42.846131+0000","last_active":"2023-10-06T04:48:42.849566+0000","last_peered":"2023-10-06T04:48:42.849566+0000","last_clean":"2023-10-06T04:48:42.849566+0000","last_became_active":"2023-10-06T04:48:42.844413+0000","last_became_peered":"2023-10-06T04:48:42.844413+0000","last_unstale":"2023-10-06T04:48:42.849566+0000","last_undegraded":"2023-10-06T04:48:42.849566+0000","last_fullsized":"2023-10-06T04:48:42.849566+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:15.249396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"159'1","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.846015+0000","last_change":"2023-10-06T04:48:41.833234+0000","last_active":"2023-10-06T04:48:42.846015+0000","last_peered":"2023-10-06T04:48:42.846015+0000","last_clean":"2023-10-06T04:48:42.846015+0000","last_became_active":"2023-10-06T04:48:41.832899+0000","last_became_peered":"2023-10-06T04:48:41.832899+0000","last_unstale":"2023-10-06T04:48:42.846015+0000","last_undegraded":"2023-10-06T04:48:42.846015+0000","last_fullsized":"2023-10-06T04:48:42.846015+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:57.479209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.825044+0000","last_change":"2023-10-06T04:48:42.821574+0000","last_active":"2023-10-06T04:48:42.825044+0000","last_peered":"2023-10-06T04:48:42.825044+0000","last_clean":"2023-10-06T04:48:42.825044+0000","last_became_active":"2023-10-06T04:48:42.820961+0000","last_became_peered":"2023-10-06T04:48:42.820961+0000","last_unstale":"2023-10-06T04:48:42.825044+0000","last_undegraded":"2023-10-06T04:48:42.825044+0000","last_fullsized":"2023-10-06T04:48:42.825044+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:27.780918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.820931+0000","last_change":"2023-10-06T04:48:41.823785+0000","last_active":"2023-10-06T04:48:42.820931+0000","last_peered":"2023-10-06T04:48:42.820931+0000","last_clean":"2023-10-06T04:48:42.820931+0000","last_became_active":"2023-10-06T04:48:41.822675+0000","last_became_peered":"2023-10-06T04:48:41.822675+0000","last_unstale":"2023-10-06T04:48:42.820931+0000","last_undegraded":"2023-10-06T04:48:42.820931+0000","last_fullsized":"2023-10-06T04:48:42.820931+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:47.556535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.829776+0000","last_change":"2023-10-06T04:48:42.827776+0000","last_active":"2023-10-06T04:48:42.829776+0000","last_peered":"2023-10-06T04:48:42.829776+0000","last_clean":"2023-10-06T04:48:42.829776+0000","last_became_active":"2023-10-06T04:48:42.827503+0000","last_became_peered":"2023-10-06T04:48:42.827503+0000","last_unstale":"2023-10-06T04:48:42.829776+0000","last_undegraded":"2023-10-06T04:48:42.829776+0000","last_fullsized":"2023-10-06T04:48:42.829776+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:38.810057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"159'2","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.838811+0000","last_change":"2023-10-06T04:48:41.828483+0000","last_active":"2023-10-06T04:48:42.838811+0000","last_peered":"2023-10-06T04:48:42.838811+0000","last_clean":"2023-10-06T04:48:42.838811+0000","last_became_active":"2023-10-06T04:48:41.828059+0000","last_became_peered":"2023-10-06T04:48:41.828059+0000","last_unstale":"2023-10-06T04:48:42.838811+0000","last_undegraded":"2023-10-06T04:48:42.838811+0000","last_fullsized":"2023-10-06T04:48:42.838811+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:20.307519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.840000+0000","last_change":"2023-10-06T04:48:42.838321+0000","last_active":"2023-10-06T04:48:42.840000+0000","last_peered":"2023-10-06T04:48:42.840000+0000","last_clean":"2023-10-06T04:48:42.840000+0000","last_became_active":"2023-10-06T04:48:42.838071+0000","last_became_peered":"2023-10-06T04:48:42.838071+0000","last_unstale":"2023-10-06T04:48:42.840000+0000","last_undegraded":"2023-10-06T04:48:42.840000+0000","last_fullsized":"2023-10-06T04:48:42.840000+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:47.332244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.820311+0000","last_change":"2023-10-06T04:48:41.830066+0000","last_active":"2023-10-06T04:48:42.820311+0000","last_peered":"2023-10-06T04:48:42.820311+0000","last_clean":"2023-10-06T04:48:42.820311+0000","last_became_active":"2023-10-06T04:48:41.829780+0000","last_became_peered":"2023-10-06T04:48:41.829780+0000","last_unstale":"2023-10-06T04:48:42.820311+0000","last_undegraded":"2023-10-06T04:48:42.820311+0000","last_fullsized":"2023-10-06T04:48:42.820311+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:42.694170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839304+0000","last_change":"2023-10-06T04:48:42.837988+0000","last_active":"2023-10-06T04:48:42.839304+0000","last_peered":"2023-10-06T04:48:42.839304+0000","last_clean":"2023-10-06T04:48:42.839304+0000","last_became_active":"2023-10-06T04:48:42.837643+0000","last_became_peered":"2023-10-06T04:48:42.837643+0000","last_unstale":"2023-10-06T04:48:42.839304+0000","last_undegraded":"2023-10-06T04:48:42.839304+0000","last_fullsized":"2023-10-06T04:48:42.839304+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:10.473217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.821262+0000","last_change":"2023-10-06T04:48:41.832712+0000","last_active":"2023-10-06T04:48:42.821262+0000","last_peered":"2023-10-06T04:48:42.821262+0000","last_clean":"2023-10-06T04:48:42.821262+0000","last_became_active":"2023-10-06T04:48:41.832455+0000","last_became_peered":"2023-10-06T04:48:41.832455+0000","last_unstale":"2023-10-06T04:48:42.821262+0000","last_undegraded":"2023-10-06T04:48:42.821262+0000","last_fullsized":"2023-10-06T04:48:42.821262+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:26.313079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841166+0000","last_change":"2023-10-06T04:48:42.838949+0000","last_active":"2023-10-06T04:48:42.841166+0000","last_peered":"2023-10-06T04:48:42.841166+0000","last_clean":"2023-10-06T04:48:42.841166+0000","last_became_active":"2023-10-06T04:48:42.838621+0000","last_became_peered":"2023-10-06T04:48:42.838621+0000","last_unstale":"2023-10-06T04:48:42.841166+0000","last_undegraded":"2023-10-06T04:48:42.841166+0000","last_fullsized":"2023-10-06T04:48:42.841166+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:48.570506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.818727+0000","last_change":"2023-10-06T04:48:41.832291+0000","last_active":"2023-10-06T04:48:42.818727+0000","last_peered":"2023-10-06T04:48:42.818727+0000","last_clean":"2023-10-06T04:48:42.818727+0000","last_became_active":"2023-10-06T04:48:41.831586+0000","last_became_peered":"2023-10-06T04:48:41.831586+0000","last_unstale":"2023-10-06T04:48:42.818727+0000","last_undegraded":"2023-10-06T04:48:42.818727+0000","last_fullsized":"2023-10-06T04:48:42.818727+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:37.705377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.846528+0000","last_change":"2023-10-06T04:48:42.844382+0000","last_active":"2023-10-06T04:48:42.846528+0000","last_peered":"2023-10-06T04:48:42.846528+0000","last_clean":"2023-10-06T04:48:42.846528+0000","last_became_active":"2023-10-06T04:48:42.843008+0000","last_became_peered":"2023-10-06T04:48:42.843008+0000","last_unstale":"2023-10-06T04:48:42.846528+0000","last_undegraded":"2023-10-06T04:48:42.846528+0000","last_fullsized":"2023-10-06T04:48:42.846528+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:40.683509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"159'2","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.847527+0000","last_change":"2023-10-06T04:48:41.830187+0000","last_active":"2023-10-06T04:48:42.847527+0000","last_peered":"2023-10-06T04:48:42.847527+0000","last_clean":"2023-10-06T04:48:42.847527+0000","last_became_active":"2023-10-06T04:48:41.829868+0000","last_became_peered":"2023-10-06T04:48:41.829868+0000","last_unstale":"2023-10-06T04:48:42.847527+0000","last_undegraded":"2023-10-06T04:48:42.847527+0000","last_fullsized":"2023-10-06T04:48:42.847527+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:00.913165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.836503+0000","last_change":"2023-10-06T04:48:42.827518+0000","last_active":"2023-10-06T04:48:42.836503+0000","last_peered":"2023-10-06T04:48:42.836503+0000","last_clean":"2023-10-06T04:48:42.836503+0000","last_became_active":"2023-10-06T04:48:42.826309+0000","last_became_peered":"2023-10-06T04:48:42.826309+0000","last_unstale":"2023-10-06T04:48:42.836503+0000","last_undegraded":"2023-10-06T04:48:42.836503+0000","last_fullsized":"2023-10-06T04:48:42.836503+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:26.539694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.819682+0000","last_change":"2023-10-06T04:48:41.828270+0000","last_active":"2023-10-06T04:48:42.819682+0000","last_peered":"2023-10-06T04:48:42.819682+0000","last_clean":"2023-10-06T04:48:42.819682+0000","last_became_active":"2023-10-06T04:48:41.827988+0000","last_became_peered":"2023-10-06T04:48:41.827988+0000","last_unstale":"2023-10-06T04:48:42.819682+0000","last_undegraded":"2023-10-06T04:48:42.819682+0000","last_fullsized":"2023-10-06T04:48:42.819682+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:20.321206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841567+0000","last_change":"2023-10-06T04:48:42.839000+0000","last_active":"2023-10-06T04:48:42.841567+0000","last_peered":"2023-10-06T04:48:42.841567+0000","last_clean":"2023-10-06T04:48:42.841567+0000","last_became_active":"2023-10-06T04:48:42.838698+0000","last_became_peered":"2023-10-06T04:48:42.838698+0000","last_unstale":"2023-10-06T04:48:42.841567+0000","last_undegraded":"2023-10-06T04:48:42.841567+0000","last_fullsized":"2023-10-06T04:48:42.841567+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:08.381568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.819504+0000","last_change":"2023-10-06T04:48:41.828065+0000","last_active":"2023-10-06T04:48:42.819504+0000","last_peered":"2023-10-06T04:48:42.819504+0000","last_clean":"2023-10-06T04:48:42.819504+0000","last_became_active":"2023-10-06T04:48:41.827742+0000","last_became_peered":"2023-10-06T04:48:41.827742+0000","last_unstale":"2023-10-06T04:48:42.819504+0000","last_undegraded":"2023-10-06T04:48:42.819504+0000","last_fullsized":"2023-10-06T04:48:42.819504+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:39.162307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.825060+0000","last_change":"2023-10-06T04:48:42.821738+0000","last_active":"2023-10-06T04:48:42.825060+0000","last_peered":"2023-10-06T04:48:42.825060+0000","last_clean":"2023-10-06T04:48:42.825060+0000","last_became_active":"2023-10-06T04:48:42.821324+0000","last_became_peered":"2023-10-06T04:48:42.821324+0000","last_unstale":"2023-10-06T04:48:42.825060+0000","last_undegraded":"2023-10-06T04:48:42.825060+0000","last_fullsized":"2023-10-06T04:48:42.825060+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:15.249396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.826201+0000","last_change":"2023-10-06T04:48:41.834476+0000","last_active":"2023-10-06T04:48:42.826201+0000","last_peered":"2023-10-06T04:48:42.826201+0000","last_clean":"2023-10-06T04:48:42.826201+0000","last_became_active":"2023-10-06T04:48:41.834206+0000","last_became_peered":"2023-10-06T04:48:41.834206+0000","last_unstale":"2023-10-06T04:48:42.826201+0000","last_undegraded":"2023-10-06T04:48:42.826201+0000","last_fullsized":"2023-10-06T04:48:42.826201+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:47.556535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839988+0000","last_change":"2023-10-06T04:48:42.834354+0000","last_active":"2023-10-06T04:48:42.839988+0000","last_peered":"2023-10-06T04:48:42.839988+0000","last_clean":"2023-10-06T04:48:42.839988+0000","last_became_active":"2023-10-06T04:48:42.831790+0000","last_became_peered":"2023-10-06T04:48:42.831790+0000","last_unstale":"2023-10-06T04:48:42.839988+0000","last_undegraded":"2023-10-06T04:48:42.839988+0000","last_fullsized":"2023-10-06T04:48:42.839988+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:26.704018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.821626+0000","last_change":"2023-10-06T04:48:41.832118+0000","last_active":"2023-10-06T04:48:42.821626+0000","last_peered":"2023-10-06T04:48:42.821626+0000","last_clean":"2023-10-06T04:48:42.821626+0000","last_became_active":"2023-10-06T04:48:41.831845+0000","last_became_peered":"2023-10-06T04:48:41.831845+0000","last_unstale":"2023-10-06T04:48:42.821626+0000","last_undegraded":"2023-10-06T04:48:42.821626+0000","last_fullsized":"2023-10-06T04:48:42.821626+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:05.721468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.842385+0000","last_change":"2023-10-06T04:48:42.839801+0000","last_active":"2023-10-06T04:48:42.842385+0000","last_peered":"2023-10-06T04:48:42.842385+0000","last_clean":"2023-10-06T04:48:42.842385+0000","last_became_active":"2023-10-06T04:48:42.839513+0000","last_became_peered":"2023-10-06T04:48:42.839513+0000","last_unstale":"2023-10-06T04:48:42.842385+0000","last_undegraded":"2023-10-06T04:48:42.842385+0000","last_fullsized":"2023-10-06T04:48:42.842385+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:07.471267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.820819+0000","last_change":"2023-10-06T04:48:41.830750+0000","last_active":"2023-10-06T04:48:42.820819+0000","last_peered":"2023-10-06T04:48:42.820819+0000","last_clean":"2023-10-06T04:48:42.820819+0000","last_became_active":"2023-10-06T04:48:41.830418+0000","last_became_peered":"2023-10-06T04:48:41.830418+0000","last_unstale":"2023-10-06T04:48:42.820819+0000","last_undegraded":"2023-10-06T04:48:42.820819+0000","last_fullsized":"2023-10-06T04:48:42.820819+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:22.371060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839770+0000","last_change":"2023-10-06T04:48:42.835149+0000","last_active":"2023-10-06T04:48:42.839770+0000","last_peered":"2023-10-06T04:48:42.839770+0000","last_clean":"2023-10-06T04:48:42.839770+0000","last_became_active":"2023-10-06T04:48:42.834897+0000","last_became_peered":"2023-10-06T04:48:42.834897+0000","last_unstale":"2023-10-06T04:48:42.839770+0000","last_undegraded":"2023-10-06T04:48:42.839770+0000","last_fullsized":"2023-10-06T04:48:42.839770+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:09.980476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.818007+0000","last_change":"2023-10-06T04:48:41.833709+0000","last_active":"2023-10-06T04:48:42.818007+0000","last_peered":"2023-10-06T04:48:42.818007+0000","last_clean":"2023-10-06T04:48:42.818007+0000","last_became_active":"2023-10-06T04:48:41.833117+0000","last_became_peered":"2023-10-06T04:48:41.833117+0000","last_unstale":"2023-10-06T04:48:42.818007+0000","last_undegraded":"2023-10-06T04:48:42.818007+0000","last_fullsized":"2023-10-06T04:48:42.818007+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:05.721468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841507+0000","last_change":"2023-10-06T04:48:42.838320+0000","last_active":"2023-10-06T04:48:42.841507+0000","last_peered":"2023-10-06T04:48:42.841507+0000","last_clean":"2023-10-06T04:48:42.841507+0000","last_became_active":"2023-10-06T04:48:42.838055+0000","last_became_peered":"2023-10-06T04:48:42.838055+0000","last_unstale":"2023-10-06T04:48:42.841507+0000","last_undegraded":"2023-10-06T04:48:42.841507+0000","last_fullsized":"2023-10-06T04:48:42.841507+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:35.937113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.822381+0000","last_change":"2023-10-06T04:48:41.826139+0000","last_active":"2023-10-06T04:48:42.822381+0000","last_peered":"2023-10-06T04:48:42.822381+0000","last_clean":"2023-10-06T04:48:42.822381+0000","last_became_active":"2023-10-06T04:48:41.825864+0000","last_became_peered":"2023-10-06T04:48:41.825864+0000","last_unstale":"2023-10-06T04:48:42.822381+0000","last_undegraded":"2023-10-06T04:48:42.822381+0000","last_fullsized":"2023-10-06T04:48:42.822381+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:31.508469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.848479+0000","last_change":"2023-10-06T04:48:42.846220+0000","last_active":"2023-10-06T04:48:42.848479+0000","last_peered":"2023-10-06T04:48:42.848479+0000","last_clean":"2023-10-06T04:48:42.848479+0000","last_became_active":"2023-10-06T04:48:42.845965+0000","last_became_peered":"2023-10-06T04:48:42.845965+0000","last_unstale":"2023-10-06T04:48:42.848479+0000","last_undegraded":"2023-10-06T04:48:42.848479+0000","last_fullsized":"2023-10-06T04:48:42.848479+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:03.468162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.823224+0000","last_change":"2023-10-06T04:48:41.832712+0000","last_active":"2023-10-06T04:48:42.823224+0000","last_peered":"2023-10-06T04:48:42.823224+0000","last_clean":"2023-10-06T04:48:42.823224+0000","last_became_active":"2023-10-06T04:48:41.832447+0000","last_became_peered":"2023-10-06T04:48:41.832447+0000","last_unstale":"2023-10-06T04:48:42.823224+0000","last_undegraded":"2023-10-06T04:48:42.823224+0000","last_fullsized":"2023-10-06T04:48:42.823224+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:39.093975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":145,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.819573+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T04:48:42.819573+0000","last_peered":"2023-10-06T04:48:42.819573+0000","last_clean":"2023-10-06T04:48:42.819573+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T04:48:42.819573+0000","last_undegraded":"2023-10-06T04:48:42.819573+0000","last_fullsized":"2023-10-06T04:48:42.819573+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839949+0000","last_change":"2023-10-06T04:48:42.831769+0000","last_active":"2023-10-06T04:48:42.839949+0000","last_peered":"2023-10-06T04:48:42.839949+0000","last_clean":"2023-10-06T04:48:42.839949+0000","last_became_active":"2023-10-06T04:48:42.831484+0000","last_became_peered":"2023-10-06T04:48:42.831484+0000","last_unstale":"2023-10-06T04:48:42.839949+0000","last_undegraded":"2023-10-06T04:48:42.839949+0000","last_fullsized":"2023-10-06T04:48:42.839949+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:42.529395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.818356+0000","last_change":"2023-10-06T04:48:41.827048+0000","last_active":"2023-10-06T04:48:42.818356+0000","last_peered":"2023-10-06T04:48:42.818356+0000","last_clean":"2023-10-06T04:48:42.818356+0000","last_became_active":"2023-10-06T04:48:41.826749+0000","last_became_peered":"2023-10-06T04:48:41.826749+0000","last_unstale":"2023-10-06T04:48:42.818356+0000","last_undegraded":"2023-10-06T04:48:42.818356+0000","last_fullsized":"2023-10-06T04:48:42.818356+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:17.691649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":134,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.821038+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T04:48:42.821038+0000","last_peered":"2023-10-06T04:48:42.821038+0000","last_clean":"2023-10-06T04:48:42.821038+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T04:48:42.821038+0000","last_undegraded":"2023-10-06T04:48:42.821038+0000","last_fullsized":"2023-10-06T04:48:42.821038+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.842068+0000","last_change":"2023-10-06T04:48:42.834872+0000","last_active":"2023-10-06T04:48:42.842068+0000","last_peered":"2023-10-06T04:48:42.842068+0000","last_clean":"2023-10-06T04:48:42.842068+0000","last_became_active":"2023-10-06T04:48:42.834610+0000","last_became_peered":"2023-10-06T04:48:42.834610+0000","last_unstale":"2023-10-06T04:48:42.842068+0000","last_undegraded":"2023-10-06T04:48:42.842068+0000","last_fullsized":"2023-10-06T04:48:42.842068+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:47.908424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.815855+0000","last_change":"2023-10-06T04:48:41.826719+0000","last_active":"2023-10-06T04:48:42.815855+0000","last_peered":"2023-10-06T04:48:42.815855+0000","last_clean":"2023-10-06T04:48:42.815855+0000","last_became_active":"2023-10-06T04:48:41.826283+0000","last_became_peered":"2023-10-06T04:48:41.826283+0000","last_unstale":"2023-10-06T04:48:42.815855+0000","last_undegraded":"2023-10-06T04:48:42.815855+0000","last_fullsized":"2023-10-06T04:48:42.815855+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:18.045041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":133,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.816525+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T04:48:42.816525+0000","last_peered":"2023-10-06T04:48:42.816525+0000","last_clean":"2023-10-06T04:48:42.816525+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T04:48:42.816525+0000","last_undegraded":"2023-10-06T04:48:42.816525+0000","last_fullsized":"2023-10-06T04:48:42.816525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"149'271","reported_seq":831,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.823251+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T04:48:42.823251+0000","last_peered":"2023-10-06T04:48:42.823251+0000","last_clean":"2023-10-06T04:48:42.823251+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T04:48:42.823251+0000","last_undegraded":"2023-10-06T04:48:42.823251+0000","last_fullsized":"2023-10-06T04:48:42.823251+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841653+0000","last_change":"2023-10-06T04:48:42.839933+0000","last_active":"2023-10-06T04:48:42.841653+0000","last_peered":"2023-10-06T04:48:42.841653+0000","last_clean":"2023-10-06T04:48:42.841653+0000","last_became_active":"2023-10-06T04:48:42.839525+0000","last_became_peered":"2023-10-06T04:48:42.839525+0000","last_unstale":"2023-10-06T04:48:42.841653+0000","last_undegraded":"2023-10-06T04:48:42.841653+0000","last_fullsized":"2023-10-06T04:48:42.841653+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:07.471267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"159'8","reported_seq":18,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849702+0000","last_change":"2023-10-06T04:48:41.832452+0000","last_active":"2023-10-06T04:48:42.849702+0000","last_peered":"2023-10-06T04:48:42.849702+0000","last_clean":"2023-10-06T04:48:42.849702+0000","last_became_active":"2023-10-06T04:48:41.832038+0000","last_became_peered":"2023-10-06T04:48:41.832038+0000","last_unstale":"2023-10-06T04:48:42.849702+0000","last_undegraded":"2023-10-06T04:48:42.849702+0000","last_fullsized":"2023-10-06T04:48:42.849702+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:39.093975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":136,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.820687+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T04:48:42.820687+0000","last_peered":"2023-10-06T04:48:42.820687+0000","last_clean":"2023-10-06T04:48:42.820687+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T04:48:42.820687+0000","last_undegraded":"2023-10-06T04:48:42.820687+0000","last_fullsized":"2023-10-06T04:48:42.820687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.834851+0000","last_change":"2023-10-06T04:48:42.829970+0000","last_active":"2023-10-06T04:48:42.834851+0000","last_peered":"2023-10-06T04:48:42.834851+0000","last_clean":"2023-10-06T04:48:42.834851+0000","last_became_active":"2023-10-06T04:48:42.827561+0000","last_became_peered":"2023-10-06T04:48:42.827561+0000","last_unstale":"2023-10-06T04:48:42.834851+0000","last_undegraded":"2023-10-06T04:48:42.834851+0000","last_fullsized":"2023-10-06T04:48:42.834851+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:05.627193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.820934+0000","last_change":"2023-10-06T04:48:41.831374+0000","last_active":"2023-10-06T04:48:42.820934+0000","last_peered":"2023-10-06T04:48:42.820934+0000","last_clean":"2023-10-06T04:48:42.820934+0000","last_became_active":"2023-10-06T04:48:41.831108+0000","last_became_peered":"2023-10-06T04:48:41.831108+0000","last_unstale":"2023-10-06T04:48:42.820934+0000","last_undegraded":"2023-10-06T04:48:42.820934+0000","last_fullsized":"2023-10-06T04:48:42.820934+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:50.139672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":136,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.823341+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T04:48:42.823341+0000","last_peered":"2023-10-06T04:48:42.823341+0000","last_clean":"2023-10-06T04:48:42.823341+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T04:48:42.823341+0000","last_undegraded":"2023-10-06T04:48:42.823341+0000","last_fullsized":"2023-10-06T04:48:42.823341+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.832329+0000","last_change":"2023-10-06T04:48:42.825843+0000","last_active":"2023-10-06T04:48:42.832329+0000","last_peered":"2023-10-06T04:48:42.832329+0000","last_clean":"2023-10-06T04:48:42.832329+0000","last_became_active":"2023-10-06T04:48:42.825553+0000","last_became_peered":"2023-10-06T04:48:42.825553+0000","last_unstale":"2023-10-06T04:48:42.832329+0000","last_undegraded":"2023-10-06T04:48:42.832329+0000","last_fullsized":"2023-10-06T04:48:42.832329+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:35.937113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"159'2","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.843058+0000","last_change":"2023-10-06T04:48:41.828558+0000","last_active":"2023-10-06T04:48:42.843058+0000","last_peered":"2023-10-06T04:48:42.843058+0000","last_clean":"2023-10-06T04:48:42.843058+0000","last_became_active":"2023-10-06T04:48:41.828229+0000","last_became_peered":"2023-10-06T04:48:41.828229+0000","last_unstale":"2023-10-06T04:48:42.843058+0000","last_undegraded":"2023-10-06T04:48:42.843058+0000","last_fullsized":"2023-10-06T04:48:42.843058+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:48.868068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":128,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.822830+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T04:48:42.822830+0000","last_peered":"2023-10-06T04:48:42.822830+0000","last_clean":"2023-10-06T04:48:42.822830+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T04:48:42.822830+0000","last_undegraded":"2023-10-06T04:48:42.822830+0000","last_fullsized":"2023-10-06T04:48:42.822830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.834958+0000","last_change":"2023-10-06T04:48:42.824804+0000","last_active":"2023-10-06T04:48:42.834958+0000","last_peered":"2023-10-06T04:48:42.834958+0000","last_clean":"2023-10-06T04:48:42.834958+0000","last_became_active":"2023-10-06T04:48:42.824438+0000","last_became_peered":"2023-10-06T04:48:42.824438+0000","last_unstale":"2023-10-06T04:48:42.834958+0000","last_undegraded":"2023-10-06T04:48:42.834958+0000","last_fullsized":"2023-10-06T04:48:42.834958+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:46.239224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"159'6","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.851210+0000","last_change":"2023-10-06T04:48:41.831354+0000","last_active":"2023-10-06T04:48:42.851210+0000","last_peered":"2023-10-06T04:48:42.851210+0000","last_clean":"2023-10-06T04:48:42.851210+0000","last_became_active":"2023-10-06T04:48:41.831057+0000","last_became_peered":"2023-10-06T04:48:41.831057+0000","last_unstale":"2023-10-06T04:48:42.851210+0000","last_undegraded":"2023-10-06T04:48:42.851210+0000","last_fullsized":"2023-10-06T04:48:42.851210+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:29.137681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":153,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.817095+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T04:48:42.817095+0000","last_peered":"2023-10-06T04:48:42.817095+0000","last_clean":"2023-10-06T04:48:42.817095+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T04:48:42.817095+0000","last_undegraded":"2023-10-06T04:48:42.817095+0000","last_fullsized":"2023-10-06T04:48:42.817095+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.836610+0000","last_change":"2023-10-06T04:48:42.835438+0000","last_active":"2023-10-06T04:48:42.836610+0000","last_peered":"2023-10-06T04:48:42.836610+0000","last_clean":"2023-10-06T04:48:42.836610+0000","last_became_active":"2023-10-06T04:48:42.832631+0000","last_became_peered":"2023-10-06T04:48:42.832631+0000","last_unstale":"2023-10-06T04:48:42.836610+0000","last_undegraded":"2023-10-06T04:48:42.836610+0000","last_fullsized":"2023-10-06T04:48:42.836610+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:26.328578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"159'3","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849707+0000","last_change":"2023-10-06T04:48:41.831964+0000","last_active":"2023-10-06T04:48:42.849707+0000","last_peered":"2023-10-06T04:48:42.849707+0000","last_clean":"2023-10-06T04:48:42.849707+0000","last_became_active":"2023-10-06T04:48:41.831713+0000","last_became_peered":"2023-10-06T04:48:41.831713+0000","last_unstale":"2023-10-06T04:48:42.849707+0000","last_undegraded":"2023-10-06T04:48:42.849707+0000","last_fullsized":"2023-10-06T04:48:42.849707+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:55.389632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":153,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.817240+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T04:48:42.817240+0000","last_peered":"2023-10-06T04:48:42.817240+0000","last_clean":"2023-10-06T04:48:42.817240+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T04:48:42.817240+0000","last_undegraded":"2023-10-06T04:48:42.817240+0000","last_fullsized":"2023-10-06T04:48:42.817240+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575746,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299824,"kb_used_data":2536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072016,"statfs":{"total":96636764160,"available":96329744384,"internally_reserved":0,"allocated":2596864,"data_stored":1634002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71399999999999997,"15min":0.746},"min":{"1min":0.48699999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.92900000000000005,"5min":0.97199999999999998,"15min":1.4930000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72699999999999998,"15min":0.746},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91900000000000004,"5min":0.995,"15min":1.417},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76000000000000001,"15min":0.79600000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.45200000000000001},"max":{"1min":0.90300000000000002,"5min":1.01,"15min":1.5309999999999999},"last":0.746},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.64200000000000002,"5min":0.58399999999999996,"15min":0.58299999999999996},"max":{"1min":0.94599999999999995,"5min":1.0289999999999999,"15min":1.097},"last":0.76400000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72399999999999998,"15min":0.74399999999999999},"min":{"1min":0.53500000000000003,"5min":0.48199999999999998,"15min":0.46400000000000002},"max":{"1min":0.96399999999999997,"5min":0.97999999999999998,"15min":1.5429999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.751,"15min":0.77000000000000002},"min":{"1min":0.57899999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.94699999999999995,"5min":0.96099999999999997,"15min":1.0189999999999999},"last":0.64000000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76400000000000001,"15min":0.77800000000000002},"min":{"1min":0.65500000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.5860000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.73299999999999998,"15min":0.76600000000000001},"min":{"1min":0.55500000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97499999999999998,"5min":1.0229999999999999,"15min":1.071},"last":0.69099999999999995}]},{"osd":4,"last update":"Fri Oct 6 04:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.45200000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85599999999999998,"5min":0.92500000000000004,"15min":1.0640000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65200000000000002,"15min":0.64300000000000002},"min":{"1min":0.51000000000000001,"5min":0.439,"15min":0.42799999999999999},"max":{"1min":0.91400000000000003,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.65700000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63600000000000001,"15min":0.65100000000000002},"min":{"1min":0.46100000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0269999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.48499999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.0469999999999999},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.754,"15min":0.748},"min":{"1min":0.55800000000000005,"5min":0.501,"15min":0.49199999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.093},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.48999999999999999,"15min":0.48699999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.2090000000000001},"last":0.77700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575747,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299240,"kb_used_data":1952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072600,"statfs":{"total":96636764160,"available":96330342400,"internally_reserved":0,"allocated":1998848,"data_stored":1043630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.46000000000000002,"5min":0.40300000000000002,"15min":0.375},"max":{"1min":0.96599999999999997,"5min":1.038,"15min":1.038},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76200000000000001,"15min":0.752},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94699999999999995,"5min":0.999,"15min":1.228},"last":0.621}]},{"osd":1,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.5,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93799999999999994,"5min":1.046,"15min":1.3660000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.748,"15min":0.751},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.438},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.3560000000000001},"last":0.72699999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.47999999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.272},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76900000000000002,"15min":0.77000000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98799999999999999,"5min":1.069,"15min":1.089},"last":0.70799999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94399999999999995,"5min":1.042,"15min":1.1060000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.52000000000000002},"max":{"1min":1.0009999999999999,"5min":1.0409999999999999,"15min":1.3120000000000001},"last":0.81799999999999995}]},{"osd":4,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83499999999999996,"5min":0.88500000000000001,"15min":0.92600000000000005},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.627},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":1.155},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.432},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93799999999999994,"5min":1.056,"15min":1.056},"last":0.76500000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95399999999999996,"5min":1.0580000000000001,"15min":1.3500000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.45800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98599999999999999,"5min":1.024,"15min":1.0389999999999999},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575746,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312104,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059736,"statfs":{"total":96636764160,"available":96317169664,"internally_reserved":0,"allocated":2588672,"data_stored":1633424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.33100000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.93100000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.46100000000000002,"5min":0.44900000000000001,"15min":0.42899999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.441}]},{"osd":2,"last update":"Fri Oct 6 04:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.36699999999999999,"5min":0.36599999999999999,"15min":0.34100000000000003},"max":{"1min":0.64800000000000002,"5min":0.67700000000000005,"15min":0.73299999999999998},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.38200000000000001},"max":{"1min":0.74199999999999999,"5min":0.77600000000000002,"15min":0.97799999999999998},"last":0.41099999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.44800000000000001,"15min":0.437},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.94699999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56299999999999994,"15min":0.55800000000000005},"min":{"1min":0.432,"5min":0.38400000000000001,"15min":0.373},"max":{"1min":0.78000000000000003,"5min":0.85799999999999998,"15min":0.91000000000000003},"last":0.39500000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.57599999999999996,"5min":0.54700000000000004,"15min":0.46899999999999997},"max":{"1min":1.0329999999999999,"5min":1.1970000000000001,"15min":1.407},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.83399999999999996,"15min":0.83099999999999996},"min":{"1min":0.65100000000000002,"5min":0.62,"15min":0.57499999999999996},"max":{"1min":0.996,"5min":1.143,"15min":1.1739999999999999},"last":0.70699999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.84499999999999997,"15min":0.82499999999999996},"min":{"1min":0.65000000000000002,"5min":0.625,"15min":0.55700000000000005},"max":{"1min":0.97199999999999998,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.64300000000000002,"5min":0.59799999999999998,"15min":0.53200000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.724},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.871,"15min":0.85299999999999998},"min":{"1min":0.69999999999999996,"5min":0.60999999999999999,"15min":0.56200000000000006},"max":{"1min":0.96899999999999997,"5min":1.2150000000000001,"15min":1.427},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86399999999999999,"15min":0.85099999999999998},"min":{"1min":0.60999999999999999,"5min":0.56100000000000005,"15min":0.55800000000000005},"max":{"1min":1.0029999999999999,"5min":1.1579999999999999,"15min":1.238},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.89200000000000002,"15min":0.878},"min":{"1min":0.628,"5min":0.628,"15min":0.61899999999999999},"max":{"1min":1.042,"5min":1.125,"15min":1.702},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.877,"15min":0.877},"min":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.98899999999999999,"5min":1.1659999999999999,"15min":1.5920000000000001},"last":0.64600000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575748,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299236,"kb_used_data":1948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072604,"statfs":{"total":96636764160,"available":96330346496,"internally_reserved":0,"allocated":1994752,"data_stored":1042064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51600000000000001,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.81699999999999995,"15min":1.6459999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.32900000000000001},"max":{"1min":0.82199999999999995,"5min":0.84099999999999997,"15min":1.4510000000000001},"last":0.375}]},{"osd":2,"last update":"Fri Oct 6 04:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.57299999999999995},"min":{"1min":0.39100000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.82999999999999996,"5min":0.86899999999999999,"15min":1.7},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.42199999999999999,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":1.6240000000000001},"last":0.42899999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54500000000000004,"15min":0.53800000000000003},"min":{"1min":0.376,"5min":0.35899999999999999,"15min":0.33300000000000002},"max":{"1min":0.72699999999999998,"5min":0.82199999999999995,"15min":1.2529999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64800000000000002,"15min":0.64200000000000002},"min":{"1min":0.48899999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88800000000000001,"5min":0.99099999999999999,"15min":1.7370000000000001},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.85499999999999998,"15min":0.86499999999999999},"min":{"1min":0.66200000000000003,"5min":0.63800000000000001,"15min":0.60399999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":2.3700000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.85799999999999998,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.56699999999999995,"15min":0.52900000000000003},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":2.1280000000000001},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.84999999999999998,"15min":0.85699999999999998},"min":{"1min":0.625,"5min":0.60999999999999999,"15min":0.57599999999999996},"max":{"1min":1.294,"5min":1.294,"15min":2.1739999999999999},"last":0.626},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.71999999999999997,"5min":0.63300000000000001,"15min":0.57799999999999996},"max":{"1min":1.214,"5min":1.214,"15min":2.2429999999999999},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Oct 6 04:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.83699999999999997,"15min":0.83399999999999996},"min":{"1min":0.59299999999999997,"5min":0.56699999999999995,"15min":0.53000000000000003},"max":{"1min":1.1040000000000001,"5min":1.204,"15min":1.605},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.89300000000000002,"15min":0.88400000000000001},"min":{"1min":0.70899999999999996,"5min":0.60199999999999998,"15min":0.58199999999999996},"max":{"1min":1.226,"5min":1.226,"15min":2.3519999999999999},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.873,"15min":0.879},"min":{"1min":0.64600000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.319,"5min":1.319,"15min":2.1019999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.86899999999999999,"15min":0.86799999999999999},"min":{"1min":0.70999999999999996,"5min":0.61899999999999999,"15min":0.56599999999999995},"max":{"1min":1.256,"5min":1.256,"15min":1.9259999999999999},"last":0.78000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575747,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311512,"kb_used_data":1936,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060328,"statfs":{"total":96636764160,"available":96317775872,"internally_reserved":0,"allocated":1982464,"data_stored":1035563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.311},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.34200000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.93999999999999995},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.32000000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.38300000000000001},"max":{"1min":0.79600000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.371},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.498}]},{"osd":4,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.628,"5min":0.60199999999999998,"15min":0.51700000000000002},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.86399999999999999,"15min":0.88100000000000001},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58599999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.381},"last":0.79700000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.59199999999999997},"max":{"1min":1.1719999999999999,"5min":1.2210000000000001,"15min":1.4379999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88300000000000001,"15min":0.89200000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.58899999999999997},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.288},"last":0.82399999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.151,"5min":1.167,"15min":1.2370000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86099999999999999,"15min":0.874},"min":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.57499999999999996},"max":{"1min":1.075,"5min":1.3320000000000001,"15min":1.482},"last":0.877},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.88500000000000001,"15min":0.89600000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.135,"5min":1.2869999999999999,"15min":1.405},"last":0.74399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575747,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311536,"kb_used_data":1960,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060304,"statfs":{"total":96636764160,"available":96317751296,"internally_reserved":0,"allocated":2007040,"data_stored":1044592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47699999999999998,"15min":0.49399999999999999},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67900000000000005,"5min":0.73899999999999999,"15min":0.86399999999999999},"last":0.34499999999999997}]},{"osd":1,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.83599999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.44900000000000001,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.73999999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.40899999999999997},"max":{"1min":0.69699999999999995,"5min":0.79200000000000004,"15min":1.036},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.54000000000000004,"5min":0.47499999999999998,"15min":0.43099999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51900000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83799999999999997},"min":{"1min":0.64300000000000002,"5min":0.56000000000000005,"15min":0.47599999999999998},"max":{"1min":1.101,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.755},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.495},"max":{"1min":1.0760000000000001,"5min":1.387,"15min":1.399},"last":0.79700000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.82599999999999996,"15min":0.82899999999999996},"min":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0049999999999999,"5min":1.2509999999999999,"15min":1.331},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.85799999999999998,"15min":0.85099999999999998},"min":{"1min":0.63600000000000001,"5min":0.58099999999999996,"15min":0.56599999999999995},"max":{"1min":1.0720000000000001,"5min":1.2130000000000001,"15min":1.2330000000000001},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.86499999999999999,"15min":0.85999999999999999},"min":{"1min":0.65100000000000002,"5min":0.53500000000000003,"15min":0.52400000000000002},"max":{"1min":1.175,"5min":1.2090000000000001,"15min":1.2490000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.89900000000000002,"15min":0.89400000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1579999999999999,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88600000000000001,"15min":0.88400000000000001},"min":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.58999999999999997},"max":{"1min":1.127,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85399999999999998,"15min":0.85799999999999998},"min":{"1min":0.60499999999999998,"5min":0.58699999999999997,"15min":0.54900000000000004},"max":{"1min":1.069,"5min":1.276,"15min":1.3029999999999999},"last":0.68700000000000006}]}]},{"osd":4,"up_from":13,"seq":55834575747,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311520,"kb_used_data":1944,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060320,"statfs":{"total":96636764160,"available":96317767680,"internally_reserved":0,"allocated":1990656,"data_stored":1042986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.45600000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.97499999999999998,"5min":1.016,"15min":1.016},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.74099999999999999,"15min":0.74199999999999999},"min":{"1min":0.51300000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95699999999999996,"5min":0.97799999999999998,"15min":1.085},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.752,"15min":0.75},"min":{"1min":0.45100000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.92200000000000004,"5min":0.98999999999999999,"15min":1.012},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.78400000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.95999999999999996,"5min":0.99099999999999999,"15min":1.0289999999999999},"last":0.69899999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76500000000000001,"15min":0.76100000000000001},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.48499999999999999},"max":{"1min":0.92900000000000005,"5min":1.028,"15min":1.0389999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78800000000000003,"15min":0.77900000000000003},"min":{"1min":0.59199999999999997,"5min":0.56499999999999995,"15min":0.52900000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.069},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.753},"min":{"1min":0.59599999999999997,"5min":0.54900000000000004,"15min":0.51800000000000002},"max":{"1min":0.95599999999999996,"5min":1.024,"15min":1.046},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.56299999999999994,"5min":0.498,"15min":0.498},"max":{"1min":0.96499999999999997,"5min":1.002,"15min":1.006},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63600000000000001,"15min":0.623},"min":{"1min":0.47999999999999998,"5min":0.42299999999999999,"15min":0.39700000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.46100000000000002,"5min":0.45000000000000001,"15min":0.41499999999999998},"max":{"1min":0.91100000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.69299999999999995,"15min":0.66100000000000003},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.42099999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45400000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.86399999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70899999999999996,"15min":0.69199999999999995},"min":{"1min":0.52200000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.76800000000000002,"15min":0.749},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.091,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.80200000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575747,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307424,"kb_used_data":1944,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064416,"statfs":{"total":96636764160,"available":96321961984,"internally_reserved":0,"allocated":1990656,"data_stored":1042030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82999999999999996,"5min":1.03,"15min":2.2829999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73599999999999999,"15min":0.751},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91600000000000004,"5min":1.095,"15min":1.3200000000000001},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.751},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.29},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.51400000000000001},"max":{"1min":0.871,"5min":0.98099999999999998,"15min":1.9139999999999999},"last":0.73999999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.57999999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98199999999999998,"5min":1.0589999999999999,"15min":1.532},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95799999999999996,"5min":1.0169999999999999,"15min":1.121},"last":0.76100000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.58899999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.85399999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.40500000000000003,"15min":0.39300000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":2.6989999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38},"max":{"1min":0.99399999999999999,"5min":1.028,"15min":1.694},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.1719999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96399999999999997,"5min":1.077,"15min":1.5880000000000001},"last":0.83799999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.52500000000000002,"5min":0.47499999999999998,"15min":0.40699999999999997},"max":{"1min":1.03,"5min":1.0409999999999999,"15min":1.7390000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.49099999999999999,"15min":0.48599999999999999},"max":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.3380000000000001},"last":0.89200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:48:47.213 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:48:47.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.431+0000 7f6eaf59e700 1 Processor -- start 2023-10-06T04:48:47.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.431+0000 7f6eaf59e700 1 -- start start 2023-10-06T04:48:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.432+0000 7f6eaf59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6eb01566e0 0x7f6eb00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.432+0000 7f6eaf59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb00ca860 con 0x7f6eb00cb900 2023-10-06T04:48:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.432+0000 7f6eaf59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb00cab40 con 0x7f6eb0156310 2023-10-06T04:48:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.432+0000 7f6eaf59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb00cae20 con 0x7f6eb01566e0 2023-10-06T04:48:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.433+0000 7f6eaed9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6eb00cb900 0x7f6eb00c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37152/0 (socket says 172.21.15.55:37152) 2023-10-06T04:48:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.433+0000 7f6eaed9d700 1 -- 172.21.15.55:0/3997372599 learned_addr learned my addr 172.21.15.55:0/3997372599 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.433+0000 7f6eadd9b700 1 --2- 172.21.15.55:0/3997372599 >> v2:172.21.15.163:3300/0 conn(0x7f6eb01566e0 0x7f6eb00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.433+0000 7f6ead59a700 1 -- 172.21.15.55:0/3997372599 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334825360 0 0) 0x7f6eb00ca860 con 0x7f6eb00cb900 2023-10-06T04:48:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.433+0000 7f6ead59a700 1 -- 172.21.15.55:0/3997372599 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e98003570 con 0x7f6eb00cb900 2023-10-06T04:48:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.433+0000 7f6ead59a700 1 -- 172.21.15.55:0/3997372599 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3383461333 0 0) 0x7f6e98003570 con 0x7f6eb00cb900 2023-10-06T04:48:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.433+0000 7f6ead59a700 1 -- 172.21.15.55:0/3997372599 >> v2:172.21.15.163:3300/0 conn(0x7f6eb01566e0 msgr2=0x7f6eb00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.433+0000 7f6ead59a700 1 --2- 172.21.15.55:0/3997372599 >> v2:172.21.15.163:3300/0 conn(0x7f6eb01566e0 0x7f6eb00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.433+0000 7f6ead59a700 1 -- 172.21.15.55:0/3997372599 >> v1:172.21.15.163:6789/0 conn(0x7f6eb0156310 legacy=0x7f6eb00c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:47.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.433+0000 7f6ead59a700 1 -- 172.21.15.55:0/3997372599 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb00cb100 con 0x7f6eb00cb900 2023-10-06T04:48:47.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.433+0000 7f6ead59a700 1 -- 172.21.15.55:0/3997372599 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f6ea80017f0 con 0x7f6eb00cb900 2023-10-06T04:48:47.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.433+0000 7f6ead59a700 1 -- 172.21.15.55:0/3997372599 >> v1:172.21.15.55:6789/0 conn(0x7f6eb00cb900 legacy=0x7f6eb00c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:47.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.433+0000 7f6ead59a700 1 --2- 172.21.15.55:0/3997372599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e98003a60 0x7f6e98003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:47.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6ead59a700 1 --2- 172.21.15.55:0/3997372599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e98004810 0x7f6e98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:47.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6ead59a700 1 --2- 172.21.15.55:0/3997372599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e980051c0 0x7f6e9800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:47.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6ead59a700 1 -- 172.21.15.55:0/3997372599 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb00cae20 con 0x7f6e98004810 2023-10-06T04:48:47.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6ead59a700 1 -- 172.21.15.55:0/3997372599 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9800c050 con 0x7f6e98003a60 2023-10-06T04:48:47.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6ead59a700 1 -- 172.21.15.55:0/3997372599 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9800c210 con 0x7f6e980051c0 2023-10-06T04:48:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6eadd9b700 1 --2- 172.21.15.55:0/3997372599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e98004810 0x7f6e98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6eae59c700 1 --2- 172.21.15.55:0/3997372599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e98003a60 0x7f6e98003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6eaed9d700 1 --2- 172.21.15.55:0/3997372599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e980051c0 0x7f6e9800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6eadd9b700 1 -- 172.21.15.55:0/3997372599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e980051c0 msgr2=0x7f6e9800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6eadd9b700 1 --2- 172.21.15.55:0/3997372599 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6e980051c0 0x7f6e9800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6eadd9b700 1 -- 172.21.15.55:0/3997372599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e98003a60 msgr2=0x7f6e98003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6eadd9b700 1 --2- 172.21.15.55:0/3997372599 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6e98003a60 0x7f6e98003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6eadd9b700 1 -- 172.21.15.55:0/3997372599 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e98003570 con 0x7f6e98004810 2023-10-06T04:48:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6eaed9d700 1 -- 172.21.15.55:0/3997372599 reap_dead start 2023-10-06T04:48:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.434+0000 7f6eadd9b700 1 --2- 172.21.15.55:0/3997372599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e98004810 0x7f6e98004c80 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0156b50 tx=0x7f6ea40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defdc3825069c072 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.435+0000 7f6ead59a700 1 -- 172.21.15.55:0/3997372599 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400e710 con 0x7f6e98004810 2023-10-06T04:48:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.435+0000 7f6ead59a700 1 -- 172.21.15.55:0/3997372599 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea400e870 con 0x7f6e98004810 2023-10-06T04:48:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.435+0000 7f6ead59a700 1 -- 172.21.15.55:0/3997372599 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400ec30 con 0x7f6e98004810 2023-10-06T04:48:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.435+0000 7f6eaf59e700 1 -- 172.21.15.55:0/3997372599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e98004810 msgr2=0x7f6e98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.435+0000 7f6eaf59e700 1 --2- 172.21.15.55:0/3997372599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e98004810 0x7f6e98004c80 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0156b50 tx=0x7f6ea40091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.435+0000 7f6eaf59e700 1 -- 172.21.15.55:0/3997372599 shutdown_connections 2023-10-06T04:48:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.435+0000 7f6eaf59e700 1 --2- 172.21.15.55:0/3997372599 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e98004810 0x7f6e98004c80 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.435+0000 7f6eaf59e700 1 -- 172.21.15.55:0/3997372599 >> 172.21.15.55:0/3997372599 conn(0x7f6eb00cc1f0 msgr2=0x7f6eb00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:47.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.435+0000 7f6eaf59e700 1 -- 172.21.15.55:0/3997372599 shutdown_connections 2023-10-06T04:48:47.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.435+0000 7f6eaf59e700 1 -- 172.21.15.55:0/3997372599 wait complete. 2023-10-06T04:48:47.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.437+0000 7f6eaf59e700 1 Processor -- start 2023-10-06T04:48:47.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.437+0000 7f6eaf59e700 1 -- start start 2023-10-06T04:48:47.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.438+0000 7f6eaf59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e98004810 0x7f6eb016b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.438+0000 7f6eaf59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6eb0149e20 0x7f6eb016bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.438+0000 7f6eaf59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6eb01591c0 0x7f6eb0159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.438+0000 7f6eaf59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb00c82b0 con 0x7f6e98004810 2023-10-06T04:48:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.438+0000 7f6eaf59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb0158d30 con 0x7f6eb0149e20 2023-10-06T04:48:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.438+0000 7f6eaf59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb0157e70 con 0x7f6eb01591c0 2023-10-06T04:48:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.438+0000 7f6eae59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e98004810 0x7f6eb016b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.438+0000 7f6eae59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e98004810 0x7f6eb016b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48450/0 (socket says 172.21.15.55:48450) 2023-10-06T04:48:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.438+0000 7f6eae59c700 1 -- 172.21.15.55:0/2720673938 learned_addr learned my addr 172.21.15.55:0/2720673938 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.438+0000 7f6eae59c700 1 -- 172.21.15.55:0/2720673938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6eb01591c0 msgr2=0x7f6eb0159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.438+0000 7f6eae59c700 1 --2- 172.21.15.55:0/2720673938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6eb01591c0 0x7f6eb0159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.438+0000 7f6eae59c700 1 -- 172.21.15.55:0/2720673938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6eb0149e20 msgr2=0x7f6eb016bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.438+0000 7f6eae59c700 1 --2- 172.21.15.55:0/2720673938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6eb0149e20 0x7f6eb016bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.438+0000 7f6eae59c700 1 -- 172.21.15.55:0/2720673938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb0158120 con 0x7f6e98004810 2023-10-06T04:48:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.439+0000 7f6eae59c700 1 --2- 172.21.15.55:0/2720673938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e98004810 0x7f6eb016b990 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000e810 tx=0x7f6ea0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.439+0000 7f6e9e7fc700 1 -- 172.21.15.55:0/2720673938 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0003350 con 0x7f6e98004810 2023-10-06T04:48:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.439+0000 7f6e9e7fc700 1 -- 172.21.15.55:0/2720673938 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea0013420 con 0x7f6e98004810 2023-10-06T04:48:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.439+0000 7f6e9e7fc700 1 -- 172.21.15.55:0/2720673938 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea00137e0 con 0x7f6e98004810 2023-10-06T04:48:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.439+0000 7f6eaf59e700 1 -- 172.21.15.55:0/2720673938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb0158400 con 0x7f6e98004810 2023-10-06T04:48:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.439+0000 7f6eaf59e700 1 -- 172.21.15.55:0/2720673938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb01589c0 con 0x7f6e98004810 2023-10-06T04:48:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.440+0000 7f6eaf59e700 1 -- 172.21.15.55:0/2720673938 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb006b5e0 con 0x7f6e98004810 2023-10-06T04:48:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.441+0000 7f6e9e7fc700 1 -- 172.21.15.55:0/2720673938 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ea0013b50 con 0x7f6e98004810 2023-10-06T04:48:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.442+0000 7f6e9e7fc700 1 --2- 172.21.15.55:0/2720673938 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6e90066f30 0x7f6e900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.442+0000 7f6e9e7fc700 1 -- 172.21.15.55:0/2720673938 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f6ea0084aa0 con 0x7f6e98004810 2023-10-06T04:48:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.442+0000 7f6eadd9b700 1 --2- 172.21.15.55:0/2720673938 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6e90066f30 0x7f6e900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.443+0000 7f6eadd9b700 1 --2- 172.21.15.55:0/2720673938 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6e90066f30 0x7f6e900693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40091e0 tx=0x7f6ea401a000 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.446+0000 7f6e9e7fc700 1 -- 172.21.15.55:0/2720673938 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ea0051980 con 0x7f6e98004810 2023-10-06T04:48:47.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.644+0000 7f6eaf59e700 1 -- 172.21.15.55:0/2720673938 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6eb004c3c0 con 0x7f6e90066f30 2023-10-06T04:48:47.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.654+0000 7f6e9e7fc700 1 -- 172.21.15.55:0/2720673938 <== mgr.7205 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311672 (secure 0 0 0) 0x7f6eb004c3c0 con 0x7f6e90066f30 2023-10-06T04:48:47.661 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:47.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.658+0000 7f6eaf59e700 1 -- 172.21.15.55:0/2720673938 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6e90066f30 msgr2=0x7f6e900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:47.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.658+0000 7f6eaf59e700 1 --2- 172.21.15.55:0/2720673938 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6e90066f30 0x7f6e900693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40091e0 tx=0x7f6ea401a000 comp rx=0 tx=0).stop 2023-10-06T04:48:47.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.658+0000 7f6eaf59e700 1 -- 172.21.15.55:0/2720673938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e98004810 msgr2=0x7f6eb016b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:47.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.658+0000 7f6eaf59e700 1 --2- 172.21.15.55:0/2720673938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e98004810 0x7f6eb016b990 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000e810 tx=0x7f6ea0009040 comp rx=0 tx=0).stop 2023-10-06T04:48:47.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.659+0000 7f6eaf59e700 1 -- 172.21.15.55:0/2720673938 shutdown_connections 2023-10-06T04:48:47.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.659+0000 7f6eaf59e700 1 --2- 172.21.15.55:0/2720673938 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6e90066f30 0x7f6e900693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.659+0000 7f6eaf59e700 1 --2- 172.21.15.55:0/2720673938 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6eb01591c0 0x7f6eb0159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.659+0000 7f6eaf59e700 1 --2- 172.21.15.55:0/2720673938 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6e98004810 0x7f6eb016b990 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.659+0000 7f6eaf59e700 1 --2- 172.21.15.55:0/2720673938 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6eb0149e20 0x7f6eb016bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.659+0000 7f6eaf59e700 1 -- 172.21.15.55:0/2720673938 >> 172.21.15.55:0/2720673938 conn(0x7f6eb00cc1f0 msgr2=0x7f6eb006ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:47.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.659+0000 7f6eaf59e700 1 -- 172.21.15.55:0/2720673938 shutdown_connections 2023-10-06T04:48:47.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.659+0000 7f6eaf59e700 1 -- 172.21.15.55:0/2720673938 wait complete. 2023-10-06T04:48:47.666 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T04:48:47.678 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-06T04:48:46.965696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":483,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":119,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2452396,"kb_used_data":16748,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752522324,"statfs":{"total":773094113280,"available":770582859776,"internally_reserved":0,"allocated":17149952,"data_stored":9518291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001578"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.836867+0000","last_change":"2023-10-06T04:48:42.830898+0000","last_active":"2023-10-06T04:48:42.836867+0000","last_peered":"2023-10-06T04:48:42.836867+0000","last_clean":"2023-10-06T04:48:42.836867+0000","last_became_active":"2023-10-06T04:48:42.830631+0000","last_became_peered":"2023-10-06T04:48:42.830631+0000","last_unstale":"2023-10-06T04:48:42.836867+0000","last_undegraded":"2023-10-06T04:48:42.836867+0000","last_fullsized":"2023-10-06T04:48:42.836867+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:18.162595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.838948+0000","last_change":"2023-10-06T04:48:42.832939+0000","last_active":"2023-10-06T04:48:42.838948+0000","last_peered":"2023-10-06T04:48:42.838948+0000","last_clean":"2023-10-06T04:48:42.838948+0000","last_became_active":"2023-10-06T04:48:42.832032+0000","last_became_peered":"2023-10-06T04:48:42.832032+0000","last_unstale":"2023-10-06T04:48:42.838948+0000","last_undegraded":"2023-10-06T04:48:42.838948+0000","last_fullsized":"2023-10-06T04:48:42.838948+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:26.704018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.834818+0000","last_change":"2023-10-06T04:48:42.827090+0000","last_active":"2023-10-06T04:48:42.834818+0000","last_peered":"2023-10-06T04:48:42.834818+0000","last_clean":"2023-10-06T04:48:42.834818+0000","last_became_active":"2023-10-06T04:48:42.826789+0000","last_became_peered":"2023-10-06T04:48:42.826789+0000","last_unstale":"2023-10-06T04:48:42.834818+0000","last_undegraded":"2023-10-06T04:48:42.834818+0000","last_fullsized":"2023-10-06T04:48:42.834818+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:27:28.443970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.840088+0000","last_change":"2023-10-06T04:48:42.834501+0000","last_active":"2023-10-06T04:48:42.840088+0000","last_peered":"2023-10-06T04:48:42.840088+0000","last_clean":"2023-10-06T04:48:42.840088+0000","last_became_active":"2023-10-06T04:48:42.834170+0000","last_became_peered":"2023-10-06T04:48:42.834170+0000","last_unstale":"2023-10-06T04:48:42.840088+0000","last_undegraded":"2023-10-06T04:48:42.840088+0000","last_fullsized":"2023-10-06T04:48:42.840088+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:40.683509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.837667+0000","last_change":"2023-10-06T04:48:42.833687+0000","last_active":"2023-10-06T04:48:42.837667+0000","last_peered":"2023-10-06T04:48:42.837667+0000","last_clean":"2023-10-06T04:48:42.837667+0000","last_became_active":"2023-10-06T04:48:42.833395+0000","last_became_peered":"2023-10-06T04:48:42.833395+0000","last_unstale":"2023-10-06T04:48:42.837667+0000","last_undegraded":"2023-10-06T04:48:42.837667+0000","last_fullsized":"2023-10-06T04:48:42.837667+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:26.704018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839516+0000","last_change":"2023-10-06T04:48:42.831555+0000","last_active":"2023-10-06T04:48:42.839516+0000","last_peered":"2023-10-06T04:48:42.839516+0000","last_clean":"2023-10-06T04:48:42.839516+0000","last_became_active":"2023-10-06T04:48:42.831159+0000","last_became_peered":"2023-10-06T04:48:42.831159+0000","last_unstale":"2023-10-06T04:48:42.839516+0000","last_undegraded":"2023-10-06T04:48:42.839516+0000","last_fullsized":"2023-10-06T04:48:42.839516+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:53:51.999284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841625+0000","last_change":"2023-10-06T04:48:42.839020+0000","last_active":"2023-10-06T04:48:42.841625+0000","last_peered":"2023-10-06T04:48:42.841625+0000","last_clean":"2023-10-06T04:48:42.841625+0000","last_became_active":"2023-10-06T04:48:42.838741+0000","last_became_peered":"2023-10-06T04:48:42.838741+0000","last_unstale":"2023-10-06T04:48:42.841625+0000","last_undegraded":"2023-10-06T04:48:42.841625+0000","last_fullsized":"2023-10-06T04:48:42.841625+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:17.582235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.831473+0000","last_change":"2023-10-06T04:48:42.827117+0000","last_active":"2023-10-06T04:48:42.831473+0000","last_peered":"2023-10-06T04:48:42.831473+0000","last_clean":"2023-10-06T04:48:42.831473+0000","last_became_active":"2023-10-06T04:48:42.826836+0000","last_became_peered":"2023-10-06T04:48:42.826836+0000","last_unstale":"2023-10-06T04:48:42.831473+0000","last_undegraded":"2023-10-06T04:48:42.831473+0000","last_fullsized":"2023-10-06T04:48:42.831473+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:51.911529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839555+0000","last_change":"2023-10-06T04:48:42.833226+0000","last_active":"2023-10-06T04:48:42.839555+0000","last_peered":"2023-10-06T04:48:42.839555+0000","last_clean":"2023-10-06T04:48:42.839555+0000","last_became_active":"2023-10-06T04:48:42.832982+0000","last_became_peered":"2023-10-06T04:48:42.832982+0000","last_unstale":"2023-10-06T04:48:42.839555+0000","last_undegraded":"2023-10-06T04:48:42.839555+0000","last_fullsized":"2023-10-06T04:48:42.839555+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:40.683509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.825011+0000","last_change":"2023-10-06T04:48:42.820839+0000","last_active":"2023-10-06T04:48:42.825011+0000","last_peered":"2023-10-06T04:48:42.825011+0000","last_clean":"2023-10-06T04:48:42.825011+0000","last_became_active":"2023-10-06T04:48:42.820573+0000","last_became_peered":"2023-10-06T04:48:42.820573+0000","last_unstale":"2023-10-06T04:48:42.825011+0000","last_undegraded":"2023-10-06T04:48:42.825011+0000","last_fullsized":"2023-10-06T04:48:42.825011+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:17.582235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.840478+0000","last_change":"2023-10-06T04:48:42.838179+0000","last_active":"2023-10-06T04:48:42.840478+0000","last_peered":"2023-10-06T04:48:42.840478+0000","last_clean":"2023-10-06T04:48:42.840478+0000","last_became_active":"2023-10-06T04:48:42.837911+0000","last_became_peered":"2023-10-06T04:48:42.837911+0000","last_unstale":"2023-10-06T04:48:42.840478+0000","last_undegraded":"2023-10-06T04:48:42.840478+0000","last_fullsized":"2023-10-06T04:48:42.840478+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:10.473217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.828076+0000","last_change":"2023-10-06T04:48:42.825994+0000","last_active":"2023-10-06T04:48:42.828076+0000","last_peered":"2023-10-06T04:48:42.828076+0000","last_clean":"2023-10-06T04:48:42.828076+0000","last_became_active":"2023-10-06T04:48:42.825577+0000","last_became_peered":"2023-10-06T04:48:42.825577+0000","last_unstale":"2023-10-06T04:48:42.828076+0000","last_undegraded":"2023-10-06T04:48:42.828076+0000","last_fullsized":"2023-10-06T04:48:42.828076+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:32:38.137243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839717+0000","last_change":"2023-10-06T04:48:42.833780+0000","last_active":"2023-10-06T04:48:42.839717+0000","last_peered":"2023-10-06T04:48:42.839717+0000","last_clean":"2023-10-06T04:48:42.839717+0000","last_became_active":"2023-10-06T04:48:42.833514+0000","last_became_peered":"2023-10-06T04:48:42.833514+0000","last_unstale":"2023-10-06T04:48:42.839717+0000","last_undegraded":"2023-10-06T04:48:42.839717+0000","last_fullsized":"2023-10-06T04:48:42.839717+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.321447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.845609+0000","last_change":"2023-10-06T04:48:42.843808+0000","last_active":"2023-10-06T04:48:42.845609+0000","last_peered":"2023-10-06T04:48:42.845609+0000","last_clean":"2023-10-06T04:48:42.845609+0000","last_became_active":"2023-10-06T04:48:42.843538+0000","last_became_peered":"2023-10-06T04:48:42.843538+0000","last_unstale":"2023-10-06T04:48:42.845609+0000","last_undegraded":"2023-10-06T04:48:42.845609+0000","last_fullsized":"2023-10-06T04:48:42.845609+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:28.750214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.845649+0000","last_change":"2023-10-06T04:48:42.844400+0000","last_active":"2023-10-06T04:48:42.845649+0000","last_peered":"2023-10-06T04:48:42.845649+0000","last_clean":"2023-10-06T04:48:42.845649+0000","last_became_active":"2023-10-06T04:48:42.844162+0000","last_became_peered":"2023-10-06T04:48:42.844162+0000","last_unstale":"2023-10-06T04:48:42.845649+0000","last_undegraded":"2023-10-06T04:48:42.845649+0000","last_fullsized":"2023-10-06T04:48:42.845649+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:21:39.033740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839551+0000","last_change":"2023-10-06T04:48:42.833118+0000","last_active":"2023-10-06T04:48:42.839551+0000","last_peered":"2023-10-06T04:48:42.839551+0000","last_clean":"2023-10-06T04:48:42.839551+0000","last_became_active":"2023-10-06T04:48:42.831731+0000","last_became_peered":"2023-10-06T04:48:42.831731+0000","last_unstale":"2023-10-06T04:48:42.839551+0000","last_undegraded":"2023-10-06T04:48:42.839551+0000","last_fullsized":"2023-10-06T04:48:42.839551+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:09:46.968018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839577+0000","last_change":"2023-10-06T04:48:42.833426+0000","last_active":"2023-10-06T04:48:42.839577+0000","last_peered":"2023-10-06T04:48:42.839577+0000","last_clean":"2023-10-06T04:48:42.839577+0000","last_became_active":"2023-10-06T04:48:42.831599+0000","last_became_peered":"2023-10-06T04:48:42.831599+0000","last_unstale":"2023-10-06T04:48:42.839577+0000","last_undegraded":"2023-10-06T04:48:42.839577+0000","last_fullsized":"2023-10-06T04:48:42.839577+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:36.738377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.837700+0000","last_change":"2023-10-06T04:48:42.832590+0000","last_active":"2023-10-06T04:48:42.837700+0000","last_peered":"2023-10-06T04:48:42.837700+0000","last_clean":"2023-10-06T04:48:42.837700+0000","last_became_active":"2023-10-06T04:48:42.832280+0000","last_became_peered":"2023-10-06T04:48:42.832280+0000","last_unstale":"2023-10-06T04:48:42.837700+0000","last_undegraded":"2023-10-06T04:48:42.837700+0000","last_fullsized":"2023-10-06T04:48:42.837700+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:45.803686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841033+0000","last_change":"2023-10-06T04:48:42.834952+0000","last_active":"2023-10-06T04:48:42.841033+0000","last_peered":"2023-10-06T04:48:42.841033+0000","last_clean":"2023-10-06T04:48:42.841033+0000","last_became_active":"2023-10-06T04:48:42.833442+0000","last_became_peered":"2023-10-06T04:48:42.833442+0000","last_unstale":"2023-10-06T04:48:42.841033+0000","last_undegraded":"2023-10-06T04:48:42.841033+0000","last_fullsized":"2023-10-06T04:48:42.841033+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:56:02.166083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.838848+0000","last_change":"2023-10-06T04:48:42.835316+0000","last_active":"2023-10-06T04:48:42.838848+0000","last_peered":"2023-10-06T04:48:42.838848+0000","last_clean":"2023-10-06T04:48:42.838848+0000","last_became_active":"2023-10-06T04:48:42.835087+0000","last_became_peered":"2023-10-06T04:48:42.835087+0000","last_unstale":"2023-10-06T04:48:42.838848+0000","last_undegraded":"2023-10-06T04:48:42.838848+0000","last_fullsized":"2023-10-06T04:48:42.838848+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:09.980476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841742+0000","last_change":"2023-10-06T04:48:42.840730+0000","last_active":"2023-10-06T04:48:42.841742+0000","last_peered":"2023-10-06T04:48:42.841742+0000","last_clean":"2023-10-06T04:48:42.841742+0000","last_became_active":"2023-10-06T04:48:42.840410+0000","last_became_peered":"2023-10-06T04:48:42.840410+0000","last_unstale":"2023-10-06T04:48:42.841742+0000","last_undegraded":"2023-10-06T04:48:42.841742+0000","last_fullsized":"2023-10-06T04:48:42.841742+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.321447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.838997+0000","last_change":"2023-10-06T04:48:42.836729+0000","last_active":"2023-10-06T04:48:42.838997+0000","last_peered":"2023-10-06T04:48:42.838997+0000","last_clean":"2023-10-06T04:48:42.838997+0000","last_became_active":"2023-10-06T04:48:42.836464+0000","last_became_peered":"2023-10-06T04:48:42.836464+0000","last_unstale":"2023-10-06T04:48:42.838997+0000","last_undegraded":"2023-10-06T04:48:42.838997+0000","last_fullsized":"2023-10-06T04:48:42.838997+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:33:29.694522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.840119+0000","last_change":"2023-10-06T04:48:42.833764+0000","last_active":"2023-10-06T04:48:42.840119+0000","last_peered":"2023-10-06T04:48:42.840119+0000","last_clean":"2023-10-06T04:48:42.840119+0000","last_became_active":"2023-10-06T04:48:42.833472+0000","last_became_peered":"2023-10-06T04:48:42.833472+0000","last_unstale":"2023-10-06T04:48:42.840119+0000","last_undegraded":"2023-10-06T04:48:42.840119+0000","last_fullsized":"2023-10-06T04:48:42.840119+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:09.980476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841667+0000","last_change":"2023-10-06T04:48:42.833934+0000","last_active":"2023-10-06T04:48:42.841667+0000","last_peered":"2023-10-06T04:48:42.841667+0000","last_clean":"2023-10-06T04:48:42.841667+0000","last_became_active":"2023-10-06T04:48:42.833651+0000","last_became_peered":"2023-10-06T04:48:42.833651+0000","last_unstale":"2023-10-06T04:48:42.841667+0000","last_undegraded":"2023-10-06T04:48:42.841667+0000","last_fullsized":"2023-10-06T04:48:42.841667+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:24:40.398908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.831427+0000","last_change":"2023-10-06T04:48:42.820806+0000","last_active":"2023-10-06T04:48:42.831427+0000","last_peered":"2023-10-06T04:48:42.831427+0000","last_clean":"2023-10-06T04:48:42.831427+0000","last_became_active":"2023-10-06T04:48:42.820460+0000","last_became_peered":"2023-10-06T04:48:42.820460+0000","last_unstale":"2023-10-06T04:48:42.831427+0000","last_undegraded":"2023-10-06T04:48:42.831427+0000","last_fullsized":"2023-10-06T04:48:42.831427+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:19:14.349500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.843090+0000","last_change":"2023-10-06T04:48:42.838714+0000","last_active":"2023-10-06T04:48:42.843090+0000","last_peered":"2023-10-06T04:48:42.843090+0000","last_clean":"2023-10-06T04:48:42.843090+0000","last_became_active":"2023-10-06T04:48:42.838378+0000","last_became_peered":"2023-10-06T04:48:42.838378+0000","last_unstale":"2023-10-06T04:48:42.843090+0000","last_undegraded":"2023-10-06T04:48:42.843090+0000","last_fullsized":"2023-10-06T04:48:42.843090+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:59:28.725438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849736+0000","last_change":"2023-10-06T04:48:42.847388+0000","last_active":"2023-10-06T04:48:42.849736+0000","last_peered":"2023-10-06T04:48:42.849736+0000","last_clean":"2023-10-06T04:48:42.849736+0000","last_became_active":"2023-10-06T04:48:42.844311+0000","last_became_peered":"2023-10-06T04:48:42.844311+0000","last_unstale":"2023-10-06T04:48:42.849736+0000","last_undegraded":"2023-10-06T04:48:42.849736+0000","last_fullsized":"2023-10-06T04:48:42.849736+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:47.069520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841492+0000","last_change":"2023-10-06T04:48:42.827854+0000","last_active":"2023-10-06T04:48:42.841492+0000","last_peered":"2023-10-06T04:48:42.841492+0000","last_clean":"2023-10-06T04:48:42.841492+0000","last_became_active":"2023-10-06T04:48:42.827611+0000","last_became_peered":"2023-10-06T04:48:42.827611+0000","last_unstale":"2023-10-06T04:48:42.841492+0000","last_undegraded":"2023-10-06T04:48:42.841492+0000","last_fullsized":"2023-10-06T04:48:42.841492+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:27:51.457979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.845022+0000","last_change":"2023-10-06T04:48:42.841510+0000","last_active":"2023-10-06T04:48:42.845022+0000","last_peered":"2023-10-06T04:48:42.845022+0000","last_clean":"2023-10-06T04:48:42.845022+0000","last_became_active":"2023-10-06T04:48:42.841203+0000","last_became_peered":"2023-10-06T04:48:42.841203+0000","last_unstale":"2023-10-06T04:48:42.845022+0000","last_undegraded":"2023-10-06T04:48:42.845022+0000","last_fullsized":"2023-10-06T04:48:42.845022+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:45:54.317378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.835548+0000","last_change":"2023-10-06T04:48:42.823328+0000","last_active":"2023-10-06T04:48:42.835548+0000","last_peered":"2023-10-06T04:48:42.835548+0000","last_clean":"2023-10-06T04:48:42.835548+0000","last_became_active":"2023-10-06T04:48:42.823062+0000","last_became_peered":"2023-10-06T04:48:42.823062+0000","last_unstale":"2023-10-06T04:48:42.835548+0000","last_undegraded":"2023-10-06T04:48:42.835548+0000","last_fullsized":"2023-10-06T04:48:42.835548+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:01:24.878062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.845375+0000","last_change":"2023-10-06T04:48:42.841657+0000","last_active":"2023-10-06T04:48:42.845375+0000","last_peered":"2023-10-06T04:48:42.845375+0000","last_clean":"2023-10-06T04:48:42.845375+0000","last_became_active":"2023-10-06T04:48:42.841327+0000","last_became_peered":"2023-10-06T04:48:42.841327+0000","last_unstale":"2023-10-06T04:48:42.845375+0000","last_undegraded":"2023-10-06T04:48:42.845375+0000","last_fullsized":"2023-10-06T04:48:42.845375+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:17.582235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.842529+0000","last_change":"2023-10-06T04:48:42.833617+0000","last_active":"2023-10-06T04:48:42.842529+0000","last_peered":"2023-10-06T04:48:42.842529+0000","last_clean":"2023-10-06T04:48:42.842529+0000","last_became_active":"2023-10-06T04:48:42.833372+0000","last_became_peered":"2023-10-06T04:48:42.833372+0000","last_unstale":"2023-10-06T04:48:42.842529+0000","last_undegraded":"2023-10-06T04:48:42.842529+0000","last_fullsized":"2023-10-06T04:48:42.842529+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:15.978210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.831493+0000","last_change":"2023-10-06T04:48:42.828556+0000","last_active":"2023-10-06T04:48:42.831493+0000","last_peered":"2023-10-06T04:48:42.831493+0000","last_clean":"2023-10-06T04:48:42.831493+0000","last_became_active":"2023-10-06T04:48:42.828289+0000","last_became_peered":"2023-10-06T04:48:42.828289+0000","last_unstale":"2023-10-06T04:48:42.831493+0000","last_undegraded":"2023-10-06T04:48:42.831493+0000","last_fullsized":"2023-10-06T04:48:42.831493+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:35.937113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"159'4","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849434+0000","last_change":"2023-10-06T04:48:41.834539+0000","last_active":"2023-10-06T04:48:42.849434+0000","last_peered":"2023-10-06T04:48:42.849434+0000","last_clean":"2023-10-06T04:48:42.849434+0000","last_became_active":"2023-10-06T04:48:41.833970+0000","last_became_peered":"2023-10-06T04:48:41.833970+0000","last_unstale":"2023-10-06T04:48:42.849434+0000","last_undegraded":"2023-10-06T04:48:42.849434+0000","last_fullsized":"2023-10-06T04:48:42.849434+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:23:06.796613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839033+0000","last_change":"2023-10-06T04:48:42.836745+0000","last_active":"2023-10-06T04:48:42.839033+0000","last_peered":"2023-10-06T04:48:42.839033+0000","last_clean":"2023-10-06T04:48:42.839033+0000","last_became_active":"2023-10-06T04:48:42.836497+0000","last_became_peered":"2023-10-06T04:48:42.836497+0000","last_unstale":"2023-10-06T04:48:42.839033+0000","last_undegraded":"2023-10-06T04:48:42.839033+0000","last_fullsized":"2023-10-06T04:48:42.839033+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:50:11.590289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"159'1","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.833930+0000","last_change":"2023-10-06T04:48:41.828697+0000","last_active":"2023-10-06T04:48:42.833930+0000","last_peered":"2023-10-06T04:48:42.833930+0000","last_clean":"2023-10-06T04:48:42.833930+0000","last_became_active":"2023-10-06T04:48:41.828450+0000","last_became_peered":"2023-10-06T04:48:41.828450+0000","last_unstale":"2023-10-06T04:48:42.833930+0000","last_undegraded":"2023-10-06T04:48:42.833930+0000","last_fullsized":"2023-10-06T04:48:42.833930+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:57.479209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.840494+0000","last_change":"2023-10-06T04:48:42.837260+0000","last_active":"2023-10-06T04:48:42.840494+0000","last_peered":"2023-10-06T04:48:42.840494+0000","last_clean":"2023-10-06T04:48:42.840494+0000","last_became_active":"2023-10-06T04:48:42.836999+0000","last_became_peered":"2023-10-06T04:48:42.836999+0000","last_unstale":"2023-10-06T04:48:42.840494+0000","last_undegraded":"2023-10-06T04:48:42.840494+0000","last_fullsized":"2023-10-06T04:48:42.840494+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:07.471267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"159'2","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849541+0000","last_change":"2023-10-06T04:48:41.833222+0000","last_active":"2023-10-06T04:48:42.849541+0000","last_peered":"2023-10-06T04:48:42.849541+0000","last_clean":"2023-10-06T04:48:42.849541+0000","last_became_active":"2023-10-06T04:48:41.832978+0000","last_became_peered":"2023-10-06T04:48:41.832978+0000","last_unstale":"2023-10-06T04:48:42.849541+0000","last_undegraded":"2023-10-06T04:48:42.849541+0000","last_fullsized":"2023-10-06T04:48:42.849541+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:40.858246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.840510+0000","last_change":"2023-10-06T04:48:42.838653+0000","last_active":"2023-10-06T04:48:42.840510+0000","last_peered":"2023-10-06T04:48:42.840510+0000","last_clean":"2023-10-06T04:48:42.840510+0000","last_became_active":"2023-10-06T04:48:42.838373+0000","last_became_peered":"2023-10-06T04:48:42.838373+0000","last_unstale":"2023-10-06T04:48:42.840510+0000","last_undegraded":"2023-10-06T04:48:42.840510+0000","last_fullsized":"2023-10-06T04:48:42.840510+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:57.396083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"159'2","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.848854+0000","last_change":"2023-10-06T04:48:41.829961+0000","last_active":"2023-10-06T04:48:42.848854+0000","last_peered":"2023-10-06T04:48:42.848854+0000","last_clean":"2023-10-06T04:48:42.848854+0000","last_became_active":"2023-10-06T04:48:41.829712+0000","last_became_peered":"2023-10-06T04:48:41.829712+0000","last_unstale":"2023-10-06T04:48:42.848854+0000","last_undegraded":"2023-10-06T04:48:42.848854+0000","last_fullsized":"2023-10-06T04:48:42.848854+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:06:58.474149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.845428+0000","last_change":"2023-10-06T04:48:42.842586+0000","last_active":"2023-10-06T04:48:42.845428+0000","last_peered":"2023-10-06T04:48:42.845428+0000","last_clean":"2023-10-06T04:48:42.845428+0000","last_became_active":"2023-10-06T04:48:42.841482+0000","last_became_peered":"2023-10-06T04:48:42.841482+0000","last_unstale":"2023-10-06T04:48:42.845428+0000","last_undegraded":"2023-10-06T04:48:42.845428+0000","last_fullsized":"2023-10-06T04:48:42.845428+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:47.069520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"159'1","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.843348+0000","last_change":"2023-10-06T04:48:41.830277+0000","last_active":"2023-10-06T04:48:42.843348+0000","last_peered":"2023-10-06T04:48:42.843348+0000","last_clean":"2023-10-06T04:48:42.843348+0000","last_became_active":"2023-10-06T04:48:41.829982+0000","last_became_peered":"2023-10-06T04:48:41.829982+0000","last_unstale":"2023-10-06T04:48:42.843348+0000","last_undegraded":"2023-10-06T04:48:42.843348+0000","last_fullsized":"2023-10-06T04:48:42.843348+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:40:03.473106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.831436+0000","last_change":"2023-10-06T04:48:42.826353+0000","last_active":"2023-10-06T04:48:42.831436+0000","last_peered":"2023-10-06T04:48:42.831436+0000","last_clean":"2023-10-06T04:48:42.831436+0000","last_became_active":"2023-10-06T04:48:42.826098+0000","last_became_peered":"2023-10-06T04:48:42.826098+0000","last_unstale":"2023-10-06T04:48:42.831436+0000","last_undegraded":"2023-10-06T04:48:42.831436+0000","last_fullsized":"2023-10-06T04:48:42.831436+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:07.264476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"159'1","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.834218+0000","last_change":"2023-10-06T04:48:41.831647+0000","last_active":"2023-10-06T04:48:42.834218+0000","last_peered":"2023-10-06T04:48:42.834218+0000","last_clean":"2023-10-06T04:48:42.834218+0000","last_became_active":"2023-10-06T04:48:41.829510+0000","last_became_peered":"2023-10-06T04:48:41.829510+0000","last_unstale":"2023-10-06T04:48:42.834218+0000","last_undegraded":"2023-10-06T04:48:42.834218+0000","last_fullsized":"2023-10-06T04:48:42.834218+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:37.705377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.838856+0000","last_change":"2023-10-06T04:48:42.835307+0000","last_active":"2023-10-06T04:48:42.838856+0000","last_peered":"2023-10-06T04:48:42.838856+0000","last_clean":"2023-10-06T04:48:42.838856+0000","last_became_active":"2023-10-06T04:48:42.835059+0000","last_became_peered":"2023-10-06T04:48:42.835059+0000","last_unstale":"2023-10-06T04:48:42.838856+0000","last_undegraded":"2023-10-06T04:48:42.838856+0000","last_fullsized":"2023-10-06T04:48:42.838856+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:11:57.321447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.818061+0000","last_change":"2023-10-06T04:48:41.833865+0000","last_active":"2023-10-06T04:48:42.818061+0000","last_peered":"2023-10-06T04:48:42.818061+0000","last_clean":"2023-10-06T04:48:42.818061+0000","last_became_active":"2023-10-06T04:48:41.833468+0000","last_became_peered":"2023-10-06T04:48:41.833468+0000","last_unstale":"2023-10-06T04:48:42.818061+0000","last_undegraded":"2023-10-06T04:48:42.818061+0000","last_fullsized":"2023-10-06T04:48:42.818061+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:31.508469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849641+0000","last_change":"2023-10-06T04:48:42.846495+0000","last_active":"2023-10-06T04:48:42.849641+0000","last_peered":"2023-10-06T04:48:42.849641+0000","last_clean":"2023-10-06T04:48:42.849641+0000","last_became_active":"2023-10-06T04:48:42.845649+0000","last_became_peered":"2023-10-06T04:48:42.845649+0000","last_unstale":"2023-10-06T04:48:42.849641+0000","last_undegraded":"2023-10-06T04:48:42.849641+0000","last_fullsized":"2023-10-06T04:48:42.849641+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:34:51.911529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.817013+0000","last_change":"2023-10-06T04:48:41.829941+0000","last_active":"2023-10-06T04:48:42.817013+0000","last_peered":"2023-10-06T04:48:42.817013+0000","last_clean":"2023-10-06T04:48:42.817013+0000","last_became_active":"2023-10-06T04:48:41.829645+0000","last_became_peered":"2023-10-06T04:48:41.829645+0000","last_unstale":"2023-10-06T04:48:42.817013+0000","last_undegraded":"2023-10-06T04:48:42.817013+0000","last_fullsized":"2023-10-06T04:48:42.817013+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:15.844634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.837987+0000","last_change":"2023-10-06T04:48:42.830724+0000","last_active":"2023-10-06T04:48:42.837987+0000","last_peered":"2023-10-06T04:48:42.837987+0000","last_clean":"2023-10-06T04:48:42.837987+0000","last_became_active":"2023-10-06T04:48:42.830461+0000","last_became_peered":"2023-10-06T04:48:42.830461+0000","last_unstale":"2023-10-06T04:48:42.837987+0000","last_undegraded":"2023-10-06T04:48:42.837987+0000","last_fullsized":"2023-10-06T04:48:42.837987+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:28.648811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"159'1","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.842968+0000","last_change":"2023-10-06T04:48:41.828825+0000","last_active":"2023-10-06T04:48:42.842968+0000","last_peered":"2023-10-06T04:48:42.842968+0000","last_clean":"2023-10-06T04:48:42.842968+0000","last_became_active":"2023-10-06T04:48:41.828516+0000","last_became_peered":"2023-10-06T04:48:41.828516+0000","last_unstale":"2023-10-06T04:48:42.842968+0000","last_undegraded":"2023-10-06T04:48:42.842968+0000","last_fullsized":"2023-10-06T04:48:42.842968+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:47:41.290550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841097+0000","last_change":"2023-10-06T04:48:42.832064+0000","last_active":"2023-10-06T04:48:42.841097+0000","last_peered":"2023-10-06T04:48:42.841097+0000","last_clean":"2023-10-06T04:48:42.841097+0000","last_became_active":"2023-10-06T04:48:42.831794+0000","last_became_peered":"2023-10-06T04:48:42.831794+0000","last_unstale":"2023-10-06T04:48:42.841097+0000","last_undegraded":"2023-10-06T04:48:42.841097+0000","last_fullsized":"2023-10-06T04:48:42.841097+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:02:07.264476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"159'2","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849075+0000","last_change":"2023-10-06T04:48:41.830689+0000","last_active":"2023-10-06T04:48:42.849075+0000","last_peered":"2023-10-06T04:48:42.849075+0000","last_clean":"2023-10-06T04:48:42.849075+0000","last_became_active":"2023-10-06T04:48:41.829685+0000","last_became_peered":"2023-10-06T04:48:41.829685+0000","last_unstale":"2023-10-06T04:48:42.849075+0000","last_undegraded":"2023-10-06T04:48:42.849075+0000","last_fullsized":"2023-10-06T04:48:42.849075+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:24.601235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849566+0000","last_change":"2023-10-06T04:48:42.846131+0000","last_active":"2023-10-06T04:48:42.849566+0000","last_peered":"2023-10-06T04:48:42.849566+0000","last_clean":"2023-10-06T04:48:42.849566+0000","last_became_active":"2023-10-06T04:48:42.844413+0000","last_became_peered":"2023-10-06T04:48:42.844413+0000","last_unstale":"2023-10-06T04:48:42.849566+0000","last_undegraded":"2023-10-06T04:48:42.849566+0000","last_fullsized":"2023-10-06T04:48:42.849566+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:15.249396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"159'1","reported_seq":11,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.846015+0000","last_change":"2023-10-06T04:48:41.833234+0000","last_active":"2023-10-06T04:48:42.846015+0000","last_peered":"2023-10-06T04:48:42.846015+0000","last_clean":"2023-10-06T04:48:42.846015+0000","last_became_active":"2023-10-06T04:48:41.832899+0000","last_became_peered":"2023-10-06T04:48:41.832899+0000","last_unstale":"2023-10-06T04:48:42.846015+0000","last_undegraded":"2023-10-06T04:48:42.846015+0000","last_fullsized":"2023-10-06T04:48:42.846015+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:57.479209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.825044+0000","last_change":"2023-10-06T04:48:42.821574+0000","last_active":"2023-10-06T04:48:42.825044+0000","last_peered":"2023-10-06T04:48:42.825044+0000","last_clean":"2023-10-06T04:48:42.825044+0000","last_became_active":"2023-10-06T04:48:42.820961+0000","last_became_peered":"2023-10-06T04:48:42.820961+0000","last_unstale":"2023-10-06T04:48:42.825044+0000","last_undegraded":"2023-10-06T04:48:42.825044+0000","last_fullsized":"2023-10-06T04:48:42.825044+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:27.780918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.820931+0000","last_change":"2023-10-06T04:48:41.823785+0000","last_active":"2023-10-06T04:48:42.820931+0000","last_peered":"2023-10-06T04:48:42.820931+0000","last_clean":"2023-10-06T04:48:42.820931+0000","last_became_active":"2023-10-06T04:48:41.822675+0000","last_became_peered":"2023-10-06T04:48:41.822675+0000","last_unstale":"2023-10-06T04:48:42.820931+0000","last_undegraded":"2023-10-06T04:48:42.820931+0000","last_fullsized":"2023-10-06T04:48:42.820931+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:47.556535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.829776+0000","last_change":"2023-10-06T04:48:42.827776+0000","last_active":"2023-10-06T04:48:42.829776+0000","last_peered":"2023-10-06T04:48:42.829776+0000","last_clean":"2023-10-06T04:48:42.829776+0000","last_became_active":"2023-10-06T04:48:42.827503+0000","last_became_peered":"2023-10-06T04:48:42.827503+0000","last_unstale":"2023-10-06T04:48:42.829776+0000","last_undegraded":"2023-10-06T04:48:42.829776+0000","last_fullsized":"2023-10-06T04:48:42.829776+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:38.810057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"159'2","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.838811+0000","last_change":"2023-10-06T04:48:41.828483+0000","last_active":"2023-10-06T04:48:42.838811+0000","last_peered":"2023-10-06T04:48:42.838811+0000","last_clean":"2023-10-06T04:48:42.838811+0000","last_became_active":"2023-10-06T04:48:41.828059+0000","last_became_peered":"2023-10-06T04:48:41.828059+0000","last_unstale":"2023-10-06T04:48:42.838811+0000","last_undegraded":"2023-10-06T04:48:42.838811+0000","last_fullsized":"2023-10-06T04:48:42.838811+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:20.307519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.840000+0000","last_change":"2023-10-06T04:48:42.838321+0000","last_active":"2023-10-06T04:48:42.840000+0000","last_peered":"2023-10-06T04:48:42.840000+0000","last_clean":"2023-10-06T04:48:42.840000+0000","last_became_active":"2023-10-06T04:48:42.838071+0000","last_became_peered":"2023-10-06T04:48:42.838071+0000","last_unstale":"2023-10-06T04:48:42.840000+0000","last_undegraded":"2023-10-06T04:48:42.840000+0000","last_fullsized":"2023-10-06T04:48:42.840000+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:47.332244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.820311+0000","last_change":"2023-10-06T04:48:41.830066+0000","last_active":"2023-10-06T04:48:42.820311+0000","last_peered":"2023-10-06T04:48:42.820311+0000","last_clean":"2023-10-06T04:48:42.820311+0000","last_became_active":"2023-10-06T04:48:41.829780+0000","last_became_peered":"2023-10-06T04:48:41.829780+0000","last_unstale":"2023-10-06T04:48:42.820311+0000","last_undegraded":"2023-10-06T04:48:42.820311+0000","last_fullsized":"2023-10-06T04:48:42.820311+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:16:42.694170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839304+0000","last_change":"2023-10-06T04:48:42.837988+0000","last_active":"2023-10-06T04:48:42.839304+0000","last_peered":"2023-10-06T04:48:42.839304+0000","last_clean":"2023-10-06T04:48:42.839304+0000","last_became_active":"2023-10-06T04:48:42.837643+0000","last_became_peered":"2023-10-06T04:48:42.837643+0000","last_unstale":"2023-10-06T04:48:42.839304+0000","last_undegraded":"2023-10-06T04:48:42.839304+0000","last_fullsized":"2023-10-06T04:48:42.839304+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:51:10.473217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.821262+0000","last_change":"2023-10-06T04:48:41.832712+0000","last_active":"2023-10-06T04:48:42.821262+0000","last_peered":"2023-10-06T04:48:42.821262+0000","last_clean":"2023-10-06T04:48:42.821262+0000","last_became_active":"2023-10-06T04:48:41.832455+0000","last_became_peered":"2023-10-06T04:48:41.832455+0000","last_unstale":"2023-10-06T04:48:42.821262+0000","last_undegraded":"2023-10-06T04:48:42.821262+0000","last_fullsized":"2023-10-06T04:48:42.821262+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:26.313079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841166+0000","last_change":"2023-10-06T04:48:42.838949+0000","last_active":"2023-10-06T04:48:42.841166+0000","last_peered":"2023-10-06T04:48:42.841166+0000","last_clean":"2023-10-06T04:48:42.841166+0000","last_became_active":"2023-10-06T04:48:42.838621+0000","last_became_peered":"2023-10-06T04:48:42.838621+0000","last_unstale":"2023-10-06T04:48:42.841166+0000","last_undegraded":"2023-10-06T04:48:42.841166+0000","last_fullsized":"2023-10-06T04:48:42.841166+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:14:48.570506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.818727+0000","last_change":"2023-10-06T04:48:41.832291+0000","last_active":"2023-10-06T04:48:42.818727+0000","last_peered":"2023-10-06T04:48:42.818727+0000","last_clean":"2023-10-06T04:48:42.818727+0000","last_became_active":"2023-10-06T04:48:41.831586+0000","last_became_peered":"2023-10-06T04:48:41.831586+0000","last_unstale":"2023-10-06T04:48:42.818727+0000","last_undegraded":"2023-10-06T04:48:42.818727+0000","last_fullsized":"2023-10-06T04:48:42.818727+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:57:37.705377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.846528+0000","last_change":"2023-10-06T04:48:42.844382+0000","last_active":"2023-10-06T04:48:42.846528+0000","last_peered":"2023-10-06T04:48:42.846528+0000","last_clean":"2023-10-06T04:48:42.846528+0000","last_became_active":"2023-10-06T04:48:42.843008+0000","last_became_peered":"2023-10-06T04:48:42.843008+0000","last_unstale":"2023-10-06T04:48:42.846528+0000","last_undegraded":"2023-10-06T04:48:42.846528+0000","last_fullsized":"2023-10-06T04:48:42.846528+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:49:40.683509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"159'2","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.847527+0000","last_change":"2023-10-06T04:48:41.830187+0000","last_active":"2023-10-06T04:48:42.847527+0000","last_peered":"2023-10-06T04:48:42.847527+0000","last_clean":"2023-10-06T04:48:42.847527+0000","last_became_active":"2023-10-06T04:48:41.829868+0000","last_became_peered":"2023-10-06T04:48:41.829868+0000","last_unstale":"2023-10-06T04:48:42.847527+0000","last_undegraded":"2023-10-06T04:48:42.847527+0000","last_fullsized":"2023-10-06T04:48:42.847527+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:42:00.913165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.836503+0000","last_change":"2023-10-06T04:48:42.827518+0000","last_active":"2023-10-06T04:48:42.836503+0000","last_peered":"2023-10-06T04:48:42.836503+0000","last_clean":"2023-10-06T04:48:42.836503+0000","last_became_active":"2023-10-06T04:48:42.826309+0000","last_became_peered":"2023-10-06T04:48:42.826309+0000","last_unstale":"2023-10-06T04:48:42.836503+0000","last_undegraded":"2023-10-06T04:48:42.836503+0000","last_fullsized":"2023-10-06T04:48:42.836503+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:59:26.539694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.819682+0000","last_change":"2023-10-06T04:48:41.828270+0000","last_active":"2023-10-06T04:48:42.819682+0000","last_peered":"2023-10-06T04:48:42.819682+0000","last_clean":"2023-10-06T04:48:42.819682+0000","last_became_active":"2023-10-06T04:48:41.827988+0000","last_became_peered":"2023-10-06T04:48:41.827988+0000","last_unstale":"2023-10-06T04:48:42.819682+0000","last_undegraded":"2023-10-06T04:48:42.819682+0000","last_fullsized":"2023-10-06T04:48:42.819682+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:20.321206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841567+0000","last_change":"2023-10-06T04:48:42.839000+0000","last_active":"2023-10-06T04:48:42.841567+0000","last_peered":"2023-10-06T04:48:42.841567+0000","last_clean":"2023-10-06T04:48:42.841567+0000","last_became_active":"2023-10-06T04:48:42.838698+0000","last_became_peered":"2023-10-06T04:48:42.838698+0000","last_unstale":"2023-10-06T04:48:42.841567+0000","last_undegraded":"2023-10-06T04:48:42.841567+0000","last_fullsized":"2023-10-06T04:48:42.841567+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:01:08.381568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.819504+0000","last_change":"2023-10-06T04:48:41.828065+0000","last_active":"2023-10-06T04:48:42.819504+0000","last_peered":"2023-10-06T04:48:42.819504+0000","last_clean":"2023-10-06T04:48:42.819504+0000","last_became_active":"2023-10-06T04:48:41.827742+0000","last_became_peered":"2023-10-06T04:48:41.827742+0000","last_unstale":"2023-10-06T04:48:42.819504+0000","last_undegraded":"2023-10-06T04:48:42.819504+0000","last_fullsized":"2023-10-06T04:48:42.819504+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:02:39.162307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.825060+0000","last_change":"2023-10-06T04:48:42.821738+0000","last_active":"2023-10-06T04:48:42.825060+0000","last_peered":"2023-10-06T04:48:42.825060+0000","last_clean":"2023-10-06T04:48:42.825060+0000","last_became_active":"2023-10-06T04:48:42.821324+0000","last_became_peered":"2023-10-06T04:48:42.821324+0000","last_unstale":"2023-10-06T04:48:42.825060+0000","last_undegraded":"2023-10-06T04:48:42.825060+0000","last_fullsized":"2023-10-06T04:48:42.825060+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:15.249396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.826201+0000","last_change":"2023-10-06T04:48:41.834476+0000","last_active":"2023-10-06T04:48:42.826201+0000","last_peered":"2023-10-06T04:48:42.826201+0000","last_clean":"2023-10-06T04:48:42.826201+0000","last_became_active":"2023-10-06T04:48:41.834206+0000","last_became_peered":"2023-10-06T04:48:41.834206+0000","last_unstale":"2023-10-06T04:48:42.826201+0000","last_undegraded":"2023-10-06T04:48:42.826201+0000","last_fullsized":"2023-10-06T04:48:42.826201+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:20:47.556535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839988+0000","last_change":"2023-10-06T04:48:42.834354+0000","last_active":"2023-10-06T04:48:42.839988+0000","last_peered":"2023-10-06T04:48:42.839988+0000","last_clean":"2023-10-06T04:48:42.839988+0000","last_became_active":"2023-10-06T04:48:42.831790+0000","last_became_peered":"2023-10-06T04:48:42.831790+0000","last_unstale":"2023-10-06T04:48:42.839988+0000","last_undegraded":"2023-10-06T04:48:42.839988+0000","last_fullsized":"2023-10-06T04:48:42.839988+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:26.704018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.821626+0000","last_change":"2023-10-06T04:48:41.832118+0000","last_active":"2023-10-06T04:48:42.821626+0000","last_peered":"2023-10-06T04:48:42.821626+0000","last_clean":"2023-10-06T04:48:42.821626+0000","last_became_active":"2023-10-06T04:48:41.831845+0000","last_became_peered":"2023-10-06T04:48:41.831845+0000","last_unstale":"2023-10-06T04:48:42.821626+0000","last_undegraded":"2023-10-06T04:48:42.821626+0000","last_fullsized":"2023-10-06T04:48:42.821626+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:05.721468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.842385+0000","last_change":"2023-10-06T04:48:42.839801+0000","last_active":"2023-10-06T04:48:42.842385+0000","last_peered":"2023-10-06T04:48:42.842385+0000","last_clean":"2023-10-06T04:48:42.842385+0000","last_became_active":"2023-10-06T04:48:42.839513+0000","last_became_peered":"2023-10-06T04:48:42.839513+0000","last_unstale":"2023-10-06T04:48:42.842385+0000","last_undegraded":"2023-10-06T04:48:42.842385+0000","last_fullsized":"2023-10-06T04:48:42.842385+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:07.471267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.820819+0000","last_change":"2023-10-06T04:48:41.830750+0000","last_active":"2023-10-06T04:48:42.820819+0000","last_peered":"2023-10-06T04:48:42.820819+0000","last_clean":"2023-10-06T04:48:42.820819+0000","last_became_active":"2023-10-06T04:48:41.830418+0000","last_became_peered":"2023-10-06T04:48:41.830418+0000","last_unstale":"2023-10-06T04:48:42.820819+0000","last_undegraded":"2023-10-06T04:48:42.820819+0000","last_fullsized":"2023-10-06T04:48:42.820819+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:43:22.371060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839770+0000","last_change":"2023-10-06T04:48:42.835149+0000","last_active":"2023-10-06T04:48:42.839770+0000","last_peered":"2023-10-06T04:48:42.839770+0000","last_clean":"2023-10-06T04:48:42.839770+0000","last_became_active":"2023-10-06T04:48:42.834897+0000","last_became_peered":"2023-10-06T04:48:42.834897+0000","last_unstale":"2023-10-06T04:48:42.839770+0000","last_undegraded":"2023-10-06T04:48:42.839770+0000","last_fullsized":"2023-10-06T04:48:42.839770+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:55:09.980476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.818007+0000","last_change":"2023-10-06T04:48:41.833709+0000","last_active":"2023-10-06T04:48:42.818007+0000","last_peered":"2023-10-06T04:48:42.818007+0000","last_clean":"2023-10-06T04:48:42.818007+0000","last_became_active":"2023-10-06T04:48:41.833117+0000","last_became_peered":"2023-10-06T04:48:41.833117+0000","last_unstale":"2023-10-06T04:48:42.818007+0000","last_undegraded":"2023-10-06T04:48:42.818007+0000","last_fullsized":"2023-10-06T04:48:42.818007+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:41:05.721468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841507+0000","last_change":"2023-10-06T04:48:42.838320+0000","last_active":"2023-10-06T04:48:42.841507+0000","last_peered":"2023-10-06T04:48:42.841507+0000","last_clean":"2023-10-06T04:48:42.841507+0000","last_became_active":"2023-10-06T04:48:42.838055+0000","last_became_peered":"2023-10-06T04:48:42.838055+0000","last_unstale":"2023-10-06T04:48:42.841507+0000","last_undegraded":"2023-10-06T04:48:42.841507+0000","last_fullsized":"2023-10-06T04:48:42.841507+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:35.937113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.822381+0000","last_change":"2023-10-06T04:48:41.826139+0000","last_active":"2023-10-06T04:48:42.822381+0000","last_peered":"2023-10-06T04:48:42.822381+0000","last_clean":"2023-10-06T04:48:42.822381+0000","last_became_active":"2023-10-06T04:48:41.825864+0000","last_became_peered":"2023-10-06T04:48:41.825864+0000","last_unstale":"2023-10-06T04:48:42.822381+0000","last_undegraded":"2023-10-06T04:48:42.822381+0000","last_fullsized":"2023-10-06T04:48:42.822381+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:03:31.508469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.848479+0000","last_change":"2023-10-06T04:48:42.846220+0000","last_active":"2023-10-06T04:48:42.848479+0000","last_peered":"2023-10-06T04:48:42.848479+0000","last_clean":"2023-10-06T04:48:42.848479+0000","last_became_active":"2023-10-06T04:48:42.845965+0000","last_became_peered":"2023-10-06T04:48:42.845965+0000","last_unstale":"2023-10-06T04:48:42.848479+0000","last_undegraded":"2023-10-06T04:48:42.848479+0000","last_fullsized":"2023-10-06T04:48:42.848479+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:03.468162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.823224+0000","last_change":"2023-10-06T04:48:41.832712+0000","last_active":"2023-10-06T04:48:42.823224+0000","last_peered":"2023-10-06T04:48:42.823224+0000","last_clean":"2023-10-06T04:48:42.823224+0000","last_became_active":"2023-10-06T04:48:41.832447+0000","last_became_peered":"2023-10-06T04:48:41.832447+0000","last_unstale":"2023-10-06T04:48:42.823224+0000","last_undegraded":"2023-10-06T04:48:42.823224+0000","last_fullsized":"2023-10-06T04:48:42.823224+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:39.093975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":145,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.819573+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T04:48:42.819573+0000","last_peered":"2023-10-06T04:48:42.819573+0000","last_clean":"2023-10-06T04:48:42.819573+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T04:48:42.819573+0000","last_undegraded":"2023-10-06T04:48:42.819573+0000","last_fullsized":"2023-10-06T04:48:42.819573+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.839949+0000","last_change":"2023-10-06T04:48:42.831769+0000","last_active":"2023-10-06T04:48:42.839949+0000","last_peered":"2023-10-06T04:48:42.839949+0000","last_clean":"2023-10-06T04:48:42.839949+0000","last_became_active":"2023-10-06T04:48:42.831484+0000","last_became_peered":"2023-10-06T04:48:42.831484+0000","last_unstale":"2023-10-06T04:48:42.839949+0000","last_undegraded":"2023-10-06T04:48:42.839949+0000","last_fullsized":"2023-10-06T04:48:42.839949+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:25:42.529395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.818356+0000","last_change":"2023-10-06T04:48:41.827048+0000","last_active":"2023-10-06T04:48:42.818356+0000","last_peered":"2023-10-06T04:48:42.818356+0000","last_clean":"2023-10-06T04:48:42.818356+0000","last_became_active":"2023-10-06T04:48:41.826749+0000","last_became_peered":"2023-10-06T04:48:41.826749+0000","last_unstale":"2023-10-06T04:48:42.818356+0000","last_undegraded":"2023-10-06T04:48:42.818356+0000","last_fullsized":"2023-10-06T04:48:42.818356+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:47:17.691649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":134,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.821038+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T04:48:42.821038+0000","last_peered":"2023-10-06T04:48:42.821038+0000","last_clean":"2023-10-06T04:48:42.821038+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T04:48:42.821038+0000","last_undegraded":"2023-10-06T04:48:42.821038+0000","last_fullsized":"2023-10-06T04:48:42.821038+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.842068+0000","last_change":"2023-10-06T04:48:42.834872+0000","last_active":"2023-10-06T04:48:42.842068+0000","last_peered":"2023-10-06T04:48:42.842068+0000","last_clean":"2023-10-06T04:48:42.842068+0000","last_became_active":"2023-10-06T04:48:42.834610+0000","last_became_peered":"2023-10-06T04:48:42.834610+0000","last_unstale":"2023-10-06T04:48:42.842068+0000","last_undegraded":"2023-10-06T04:48:42.842068+0000","last_fullsized":"2023-10-06T04:48:42.842068+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:47.908424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.815855+0000","last_change":"2023-10-06T04:48:41.826719+0000","last_active":"2023-10-06T04:48:42.815855+0000","last_peered":"2023-10-06T04:48:42.815855+0000","last_clean":"2023-10-06T04:48:42.815855+0000","last_became_active":"2023-10-06T04:48:41.826283+0000","last_became_peered":"2023-10-06T04:48:41.826283+0000","last_unstale":"2023-10-06T04:48:42.815855+0000","last_undegraded":"2023-10-06T04:48:42.815855+0000","last_fullsized":"2023-10-06T04:48:42.815855+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:34:18.045041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":133,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.816525+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T04:48:42.816525+0000","last_peered":"2023-10-06T04:48:42.816525+0000","last_clean":"2023-10-06T04:48:42.816525+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T04:48:42.816525+0000","last_undegraded":"2023-10-06T04:48:42.816525+0000","last_fullsized":"2023-10-06T04:48:42.816525+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"149'271","reported_seq":831,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.823251+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T04:48:42.823251+0000","last_peered":"2023-10-06T04:48:42.823251+0000","last_clean":"2023-10-06T04:48:42.823251+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T04:48:42.823251+0000","last_undegraded":"2023-10-06T04:48:42.823251+0000","last_fullsized":"2023-10-06T04:48:42.823251+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.841653+0000","last_change":"2023-10-06T04:48:42.839933+0000","last_active":"2023-10-06T04:48:42.841653+0000","last_peered":"2023-10-06T04:48:42.841653+0000","last_clean":"2023-10-06T04:48:42.841653+0000","last_became_active":"2023-10-06T04:48:42.839525+0000","last_became_peered":"2023-10-06T04:48:42.839525+0000","last_unstale":"2023-10-06T04:48:42.841653+0000","last_undegraded":"2023-10-06T04:48:42.841653+0000","last_fullsized":"2023-10-06T04:48:42.841653+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:26:07.471267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"159'8","reported_seq":18,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849702+0000","last_change":"2023-10-06T04:48:41.832452+0000","last_active":"2023-10-06T04:48:42.849702+0000","last_peered":"2023-10-06T04:48:42.849702+0000","last_clean":"2023-10-06T04:48:42.849702+0000","last_became_active":"2023-10-06T04:48:41.832038+0000","last_became_peered":"2023-10-06T04:48:41.832038+0000","last_unstale":"2023-10-06T04:48:42.849702+0000","last_undegraded":"2023-10-06T04:48:42.849702+0000","last_fullsized":"2023-10-06T04:48:42.849702+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:39.093975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":136,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.820687+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T04:48:42.820687+0000","last_peered":"2023-10-06T04:48:42.820687+0000","last_clean":"2023-10-06T04:48:42.820687+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T04:48:42.820687+0000","last_undegraded":"2023-10-06T04:48:42.820687+0000","last_fullsized":"2023-10-06T04:48:42.820687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.834851+0000","last_change":"2023-10-06T04:48:42.829970+0000","last_active":"2023-10-06T04:48:42.834851+0000","last_peered":"2023-10-06T04:48:42.834851+0000","last_clean":"2023-10-06T04:48:42.834851+0000","last_became_active":"2023-10-06T04:48:42.827561+0000","last_became_peered":"2023-10-06T04:48:42.827561+0000","last_unstale":"2023-10-06T04:48:42.834851+0000","last_undegraded":"2023-10-06T04:48:42.834851+0000","last_fullsized":"2023-10-06T04:48:42.834851+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:58:05.627193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.820934+0000","last_change":"2023-10-06T04:48:41.831374+0000","last_active":"2023-10-06T04:48:42.820934+0000","last_peered":"2023-10-06T04:48:42.820934+0000","last_clean":"2023-10-06T04:48:42.820934+0000","last_became_active":"2023-10-06T04:48:41.831108+0000","last_became_peered":"2023-10-06T04:48:41.831108+0000","last_unstale":"2023-10-06T04:48:42.820934+0000","last_undegraded":"2023-10-06T04:48:42.820934+0000","last_fullsized":"2023-10-06T04:48:42.820934+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:44:50.139672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":136,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.823341+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T04:48:42.823341+0000","last_peered":"2023-10-06T04:48:42.823341+0000","last_clean":"2023-10-06T04:48:42.823341+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T04:48:42.823341+0000","last_undegraded":"2023-10-06T04:48:42.823341+0000","last_fullsized":"2023-10-06T04:48:42.823341+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.832329+0000","last_change":"2023-10-06T04:48:42.825843+0000","last_active":"2023-10-06T04:48:42.832329+0000","last_peered":"2023-10-06T04:48:42.832329+0000","last_clean":"2023-10-06T04:48:42.832329+0000","last_became_active":"2023-10-06T04:48:42.825553+0000","last_became_peered":"2023-10-06T04:48:42.825553+0000","last_unstale":"2023-10-06T04:48:42.832329+0000","last_undegraded":"2023-10-06T04:48:42.832329+0000","last_fullsized":"2023-10-06T04:48:42.832329+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:35.937113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"159'2","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.843058+0000","last_change":"2023-10-06T04:48:41.828558+0000","last_active":"2023-10-06T04:48:42.843058+0000","last_peered":"2023-10-06T04:48:42.843058+0000","last_clean":"2023-10-06T04:48:42.843058+0000","last_became_active":"2023-10-06T04:48:41.828229+0000","last_became_peered":"2023-10-06T04:48:41.828229+0000","last_unstale":"2023-10-06T04:48:42.843058+0000","last_undegraded":"2023-10-06T04:48:42.843058+0000","last_fullsized":"2023-10-06T04:48:42.843058+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:01:48.868068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":128,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.822830+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T04:48:42.822830+0000","last_peered":"2023-10-06T04:48:42.822830+0000","last_clean":"2023-10-06T04:48:42.822830+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T04:48:42.822830+0000","last_undegraded":"2023-10-06T04:48:42.822830+0000","last_fullsized":"2023-10-06T04:48:42.822830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.834958+0000","last_change":"2023-10-06T04:48:42.824804+0000","last_active":"2023-10-06T04:48:42.834958+0000","last_peered":"2023-10-06T04:48:42.834958+0000","last_clean":"2023-10-06T04:48:42.834958+0000","last_became_active":"2023-10-06T04:48:42.824438+0000","last_became_peered":"2023-10-06T04:48:42.824438+0000","last_unstale":"2023-10-06T04:48:42.834958+0000","last_undegraded":"2023-10-06T04:48:42.834958+0000","last_fullsized":"2023-10-06T04:48:42.834958+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:32:46.239224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"159'6","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.851210+0000","last_change":"2023-10-06T04:48:41.831354+0000","last_active":"2023-10-06T04:48:42.851210+0000","last_peered":"2023-10-06T04:48:42.851210+0000","last_clean":"2023-10-06T04:48:42.851210+0000","last_became_active":"2023-10-06T04:48:41.831057+0000","last_became_peered":"2023-10-06T04:48:41.831057+0000","last_unstale":"2023-10-06T04:48:42.851210+0000","last_undegraded":"2023-10-06T04:48:42.851210+0000","last_fullsized":"2023-10-06T04:48:42.851210+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:29.137681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":153,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.817095+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T04:48:42.817095+0000","last_peered":"2023-10-06T04:48:42.817095+0000","last_clean":"2023-10-06T04:48:42.817095+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T04:48:42.817095+0000","last_undegraded":"2023-10-06T04:48:42.817095+0000","last_fullsized":"2023-10-06T04:48:42.817095+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.836610+0000","last_change":"2023-10-06T04:48:42.835438+0000","last_active":"2023-10-06T04:48:42.836610+0000","last_peered":"2023-10-06T04:48:42.836610+0000","last_clean":"2023-10-06T04:48:42.836610+0000","last_became_active":"2023-10-06T04:48:42.832631+0000","last_became_peered":"2023-10-06T04:48:42.832631+0000","last_unstale":"2023-10-06T04:48:42.836610+0000","last_undegraded":"2023-10-06T04:48:42.836610+0000","last_fullsized":"2023-10-06T04:48:42.836610+0000","mapping_epoch":158,"log_start":"0'0","ondisk_log_start":"0'0","created":158,"last_epoch_clean":159,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:41.800581+0000","last_clean_scrub_stamp":"2023-10-06T04:48:41.800581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:26.328578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"159'3","reported_seq":13,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.849707+0000","last_change":"2023-10-06T04:48:41.831964+0000","last_active":"2023-10-06T04:48:42.849707+0000","last_peered":"2023-10-06T04:48:42.849707+0000","last_clean":"2023-10-06T04:48:42.849707+0000","last_became_active":"2023-10-06T04:48:41.831713+0000","last_became_peered":"2023-10-06T04:48:41.831713+0000","last_unstale":"2023-10-06T04:48:42.849707+0000","last_undegraded":"2023-10-06T04:48:42.849707+0000","last_fullsized":"2023-10-06T04:48:42.849707+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:48:40.789894+0000","last_clean_scrub_stamp":"2023-10-06T04:48:40.789894+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:15:55.389632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":153,"reported_epoch":159,"state":"active+clean","last_fresh":"2023-10-06T04:48:42.817240+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T04:48:42.817240+0000","last_peered":"2023-10-06T04:48:42.817240+0000","last_clean":"2023-10-06T04:48:42.817240+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T04:48:42.817240+0000","last_undegraded":"2023-10-06T04:48:42.817240+0000","last_fullsized":"2023-10-06T04:48:42.817240+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575746,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299824,"kb_used_data":2536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072016,"statfs":{"total":96636764160,"available":96329744384,"internally_reserved":0,"allocated":2596864,"data_stored":1634002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71399999999999997,"15min":0.746},"min":{"1min":0.48699999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.92900000000000005,"5min":0.97199999999999998,"15min":1.4930000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72699999999999998,"15min":0.746},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.91900000000000004,"5min":0.995,"15min":1.417},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76000000000000001,"15min":0.79600000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.45200000000000001},"max":{"1min":0.90300000000000002,"5min":1.01,"15min":1.5309999999999999},"last":0.746},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79100000000000004,"15min":0.79900000000000004},"min":{"1min":0.64200000000000002,"5min":0.58399999999999996,"15min":0.58299999999999996},"max":{"1min":0.94599999999999995,"5min":1.0289999999999999,"15min":1.097},"last":0.76400000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72399999999999998,"15min":0.74399999999999999},"min":{"1min":0.53500000000000003,"5min":0.48199999999999998,"15min":0.46400000000000002},"max":{"1min":0.96399999999999997,"5min":0.97999999999999998,"15min":1.5429999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.751,"15min":0.77000000000000002},"min":{"1min":0.57899999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.94699999999999995,"5min":0.96099999999999997,"15min":1.0189999999999999},"last":0.64000000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.76400000000000001,"15min":0.77800000000000002},"min":{"1min":0.65500000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.5860000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.73299999999999998,"15min":0.76600000000000001},"min":{"1min":0.55500000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97499999999999998,"5min":1.0229999999999999,"15min":1.071},"last":0.69099999999999995}]},{"osd":4,"last update":"Fri Oct 6 04:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61599999999999999,"15min":0.61099999999999999},"min":{"1min":0.45200000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85599999999999998,"5min":0.92500000000000004,"15min":1.0640000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.65200000000000002,"15min":0.64300000000000002},"min":{"1min":0.51000000000000001,"5min":0.439,"15min":0.42799999999999999},"max":{"1min":0.91400000000000003,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.65700000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63600000000000001,"15min":0.65100000000000002},"min":{"1min":0.46100000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0269999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.48499999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.0469999999999999},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:47:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.754,"15min":0.748},"min":{"1min":0.55800000000000005,"5min":0.501,"15min":0.49199999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.093},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.72299999999999998,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.48999999999999999,"15min":0.48699999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.2090000000000001},"last":0.77700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575747,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299240,"kb_used_data":1952,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072600,"statfs":{"total":96636764160,"available":96330342400,"internally_reserved":0,"allocated":1998848,"data_stored":1043630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.46000000000000002,"5min":0.40300000000000002,"15min":0.375},"max":{"1min":0.96599999999999997,"5min":1.038,"15min":1.038},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76200000000000001,"15min":0.752},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94699999999999995,"5min":0.999,"15min":1.228},"last":0.621}]},{"osd":1,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.5,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93799999999999994,"5min":1.046,"15min":1.3660000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.748,"15min":0.751},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.438},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.3560000000000001},"last":0.72699999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.47999999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.272},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76900000000000002,"15min":0.77000000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98799999999999999,"5min":1.069,"15min":1.089},"last":0.70799999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94399999999999995,"5min":1.042,"15min":1.1060000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.52000000000000002},"max":{"1min":1.0009999999999999,"5min":1.0409999999999999,"15min":1.3120000000000001},"last":0.81799999999999995}]},{"osd":4,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83499999999999996,"5min":0.88500000000000001,"15min":0.92600000000000005},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.627},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":1.155},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.432},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93799999999999994,"5min":1.056,"15min":1.056},"last":0.76500000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95399999999999996,"5min":1.0580000000000001,"15min":1.3500000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.45800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98599999999999999,"5min":1.024,"15min":1.0389999999999999},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575746,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312104,"kb_used_data":2528,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059736,"statfs":{"total":96636764160,"available":96317169664,"internally_reserved":0,"allocated":2588672,"data_stored":1633424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.33100000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.93100000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.46100000000000002,"5min":0.44900000000000001,"15min":0.42899999999999999},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.441}]},{"osd":2,"last update":"Fri Oct 6 04:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.36699999999999999,"5min":0.36599999999999999,"15min":0.34100000000000003},"max":{"1min":0.64800000000000002,"5min":0.67700000000000005,"15min":0.73299999999999998},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.38200000000000001},"max":{"1min":0.74199999999999999,"5min":0.77600000000000002,"15min":0.97799999999999998},"last":0.41099999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.44800000000000001,"15min":0.437},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.94699999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56299999999999994,"15min":0.55800000000000005},"min":{"1min":0.432,"5min":0.38400000000000001,"15min":0.373},"max":{"1min":0.78000000000000003,"5min":0.85799999999999998,"15min":0.91000000000000003},"last":0.39500000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.79800000000000004,"15min":0.79300000000000004},"min":{"1min":0.57599999999999996,"5min":0.54700000000000004,"15min":0.46899999999999997},"max":{"1min":1.0329999999999999,"5min":1.1970000000000001,"15min":1.407},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.83399999999999996,"15min":0.83099999999999996},"min":{"1min":0.65100000000000002,"5min":0.62,"15min":0.57499999999999996},"max":{"1min":0.996,"5min":1.143,"15min":1.1739999999999999},"last":0.70699999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.84499999999999997,"15min":0.82499999999999996},"min":{"1min":0.65000000000000002,"5min":0.625,"15min":0.55700000000000005},"max":{"1min":0.97199999999999998,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.64300000000000002,"5min":0.59799999999999998,"15min":0.53200000000000003},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.724},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.871,"15min":0.85299999999999998},"min":{"1min":0.69999999999999996,"5min":0.60999999999999999,"15min":0.56200000000000006},"max":{"1min":0.96899999999999997,"5min":1.2150000000000001,"15min":1.427},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.86399999999999999,"15min":0.85099999999999998},"min":{"1min":0.60999999999999999,"5min":0.56100000000000005,"15min":0.55800000000000005},"max":{"1min":1.0029999999999999,"5min":1.1579999999999999,"15min":1.238},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:47:45 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.89200000000000002,"15min":0.878},"min":{"1min":0.628,"5min":0.628,"15min":0.61899999999999999},"max":{"1min":1.042,"5min":1.125,"15min":1.702},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.877,"15min":0.877},"min":{"1min":0.67300000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.98899999999999999,"5min":1.1659999999999999,"15min":1.5920000000000001},"last":0.64600000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575748,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299236,"kb_used_data":1948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072604,"statfs":{"total":96636764160,"available":96330346496,"internally_reserved":0,"allocated":1994752,"data_stored":1042064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.51600000000000001,"15min":0.503},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78200000000000003,"5min":0.81699999999999995,"15min":1.6459999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55800000000000005,"15min":0.55100000000000005},"min":{"1min":0.38400000000000001,"5min":0.34200000000000003,"15min":0.32900000000000001},"max":{"1min":0.82199999999999995,"5min":0.84099999999999997,"15min":1.4510000000000001},"last":0.375}]},{"osd":2,"last update":"Fri Oct 6 04:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.57299999999999995},"min":{"1min":0.39100000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.82999999999999996,"5min":0.86899999999999999,"15min":1.7},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57899999999999996,"15min":0.57499999999999996},"min":{"1min":0.42199999999999999,"5min":0.40000000000000002,"15min":0.373},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":1.6240000000000001},"last":0.42899999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54500000000000004,"15min":0.53800000000000003},"min":{"1min":0.376,"5min":0.35899999999999999,"15min":0.33300000000000002},"max":{"1min":0.72699999999999998,"5min":0.82199999999999995,"15min":1.2529999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64800000000000002,"15min":0.64200000000000002},"min":{"1min":0.48899999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.88800000000000001,"5min":0.99099999999999999,"15min":1.7370000000000001},"last":0.47699999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.85499999999999998,"15min":0.86499999999999999},"min":{"1min":0.66200000000000003,"5min":0.63800000000000001,"15min":0.60399999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":2.3700000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.85799999999999998,"15min":0.85899999999999999},"min":{"1min":0.65400000000000003,"5min":0.56699999999999995,"15min":0.52900000000000003},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":2.1280000000000001},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.84999999999999998,"15min":0.85699999999999998},"min":{"1min":0.625,"5min":0.60999999999999999,"15min":0.57599999999999996},"max":{"1min":1.294,"5min":1.294,"15min":2.1739999999999999},"last":0.626},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.71999999999999997,"5min":0.63300000000000001,"15min":0.57799999999999996},"max":{"1min":1.214,"5min":1.214,"15min":2.2429999999999999},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Oct 6 04:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.83699999999999997,"15min":0.83399999999999996},"min":{"1min":0.59299999999999997,"5min":0.56699999999999995,"15min":0.53000000000000003},"max":{"1min":1.1040000000000001,"5min":1.204,"15min":1.605},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.89300000000000002,"15min":0.88400000000000001},"min":{"1min":0.70899999999999996,"5min":0.60199999999999998,"15min":0.58199999999999996},"max":{"1min":1.226,"5min":1.226,"15min":2.3519999999999999},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:47:40 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.873,"15min":0.879},"min":{"1min":0.64600000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.319,"5min":1.319,"15min":2.1019999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.86899999999999999,"15min":0.86799999999999999},"min":{"1min":0.70999999999999996,"5min":0.61899999999999999,"15min":0.56599999999999995},"max":{"1min":1.256,"5min":1.256,"15min":1.9259999999999999},"last":0.78000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575747,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311512,"kb_used_data":1936,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060328,"statfs":{"total":96636764160,"available":96317775872,"internally_reserved":0,"allocated":1982464,"data_stored":1035563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.311},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.34200000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.93999999999999995},"last":0.40600000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.32000000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.46600000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.38300000000000001},"max":{"1min":0.79600000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.371},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.498}]},{"osd":4,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.628,"5min":0.60199999999999998,"15min":0.51700000000000002},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.86399999999999999,"15min":0.88100000000000001},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58599999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.381},"last":0.79700000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.59199999999999997},"max":{"1min":1.1719999999999999,"5min":1.2210000000000001,"15min":1.4379999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88300000000000001,"15min":0.89200000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.58899999999999997},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.288},"last":0.82399999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.151,"5min":1.167,"15min":1.2370000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86099999999999999,"15min":0.874},"min":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.57499999999999996},"max":{"1min":1.075,"5min":1.3320000000000001,"15min":1.482},"last":0.877},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.88500000000000001,"15min":0.89600000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.135,"5min":1.2869999999999999,"15min":1.405},"last":0.74399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575747,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311536,"kb_used_data":1960,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060304,"statfs":{"total":96636764160,"available":96317751296,"internally_reserved":0,"allocated":2007040,"data_stored":1044592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47699999999999998,"15min":0.49399999999999999},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67900000000000005,"5min":0.73899999999999999,"15min":0.86399999999999999},"last":0.34499999999999997}]},{"osd":1,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.83599999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.44900000000000001,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.73999999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.40899999999999997},"max":{"1min":0.69699999999999995,"5min":0.79200000000000004,"15min":1.036},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.54000000000000004,"5min":0.47499999999999998,"15min":0.43099999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51900000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83799999999999997},"min":{"1min":0.64300000000000002,"5min":0.56000000000000005,"15min":0.47599999999999998},"max":{"1min":1.101,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.755},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.495},"max":{"1min":1.0760000000000001,"5min":1.387,"15min":1.399},"last":0.79700000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.82599999999999996,"15min":0.82899999999999996},"min":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0049999999999999,"5min":1.2509999999999999,"15min":1.331},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.85799999999999998,"15min":0.85099999999999998},"min":{"1min":0.63600000000000001,"5min":0.58099999999999996,"15min":0.56599999999999995},"max":{"1min":1.0720000000000001,"5min":1.2130000000000001,"15min":1.2330000000000001},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.86499999999999999,"15min":0.85999999999999999},"min":{"1min":0.65100000000000002,"5min":0.53500000000000003,"15min":0.52400000000000002},"max":{"1min":1.175,"5min":1.2090000000000001,"15min":1.2490000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.89900000000000002,"15min":0.89400000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1579999999999999,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88600000000000001,"15min":0.88400000000000001},"min":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.58999999999999997},"max":{"1min":1.127,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85399999999999998,"15min":0.85799999999999998},"min":{"1min":0.60499999999999998,"5min":0.58699999999999997,"15min":0.54900000000000004},"max":{"1min":1.069,"5min":1.276,"15min":1.3029999999999999},"last":0.68700000000000006}]}]},{"osd":4,"up_from":13,"seq":55834575747,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311520,"kb_used_data":1944,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060320,"statfs":{"total":96636764160,"available":96317767680,"internally_reserved":0,"allocated":1990656,"data_stored":1042986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.45600000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.97499999999999998,"5min":1.016,"15min":1.016},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.74099999999999999,"15min":0.74199999999999999},"min":{"1min":0.51300000000000001,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95699999999999996,"5min":0.97799999999999998,"15min":1.085},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.752,"15min":0.75},"min":{"1min":0.45100000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.92200000000000004,"5min":0.98999999999999999,"15min":1.012},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.78400000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.95999999999999996,"5min":0.99099999999999999,"15min":1.0289999999999999},"last":0.69899999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76500000000000001,"15min":0.76100000000000001},"min":{"1min":0.50900000000000001,"5min":0.503,"15min":0.48499999999999999},"max":{"1min":0.92900000000000005,"5min":1.028,"15min":1.0389999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78800000000000003,"15min":0.77900000000000003},"min":{"1min":0.59199999999999997,"5min":0.56499999999999995,"15min":0.52900000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.069},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.753},"min":{"1min":0.59599999999999997,"5min":0.54900000000000004,"15min":0.51800000000000002},"max":{"1min":0.95599999999999996,"5min":1.024,"15min":1.046},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79900000000000004,"15min":0.79500000000000004},"min":{"1min":0.56299999999999994,"5min":0.498,"15min":0.498},"max":{"1min":0.96499999999999997,"5min":1.002,"15min":1.006},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.63600000000000001,"15min":0.623},"min":{"1min":0.47999999999999998,"5min":0.42299999999999999,"15min":0.39700000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.64800000000000002,"15min":0.63600000000000001},"min":{"1min":0.46100000000000002,"5min":0.45000000000000001,"15min":0.41499999999999998},"max":{"1min":0.91100000000000003,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.69299999999999995,"15min":0.66100000000000003},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.42099999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45400000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.86399999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:47:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70899999999999996,"15min":0.69199999999999995},"min":{"1min":0.52200000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.76800000000000002,"15min":0.749},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.091,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.80200000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575747,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307424,"kb_used_data":1944,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064416,"statfs":{"total":96636764160,"available":96321961984,"internally_reserved":0,"allocated":1990656,"data_stored":1042030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82999999999999996,"5min":1.03,"15min":2.2829999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73599999999999999,"15min":0.751},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91600000000000004,"5min":1.095,"15min":1.3200000000000001},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.751},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.29},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.51400000000000001},"max":{"1min":0.871,"5min":0.98099999999999998,"15min":1.9139999999999999},"last":0.73999999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.57999999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98199999999999998,"5min":1.0589999999999999,"15min":1.532},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95799999999999996,"5min":1.0169999999999999,"15min":1.121},"last":0.76100000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.58899999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.85399999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.40500000000000003,"15min":0.39300000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":2.6989999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38},"max":{"1min":0.99399999999999999,"5min":1.028,"15min":1.694},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.1719999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96399999999999997,"5min":1.077,"15min":1.5880000000000001},"last":0.83799999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.52500000000000002,"5min":0.47499999999999998,"15min":0.40699999999999997},"max":{"1min":1.03,"5min":1.0409999999999999,"15min":1.7390000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.49099999999999999,"15min":0.48599999999999999},"max":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.3380000000000001},"last":0.89200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:48:47.689 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T04:48:47.689 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T04:48:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.912+0000 7f891159d700 1 Processor -- start 2023-10-06T04:48:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.913+0000 7f891159d700 1 -- start start 2023-10-06T04:48:47.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.914+0000 7f891159d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f890c0ae5f0 0x7f890c0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:47.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.914+0000 7f891159d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f890c0cce70 0x7f890c156f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:47.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.914+0000 7f891159d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f890c0c6990 con 0x7f890c0ae9c0 2023-10-06T04:48:47.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.914+0000 7f891159d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f890c157bb0 con 0x7f890c0ae5f0 2023-10-06T04:48:47.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.914+0000 7f891159d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f890c06d770 con 0x7f890c0cce70 2023-10-06T04:48:47.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.914+0000 7f890bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f890c0ae5f0 0x7f890c0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:47.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.915+0000 7f890bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f890c0ae5f0 0x7f890c0cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:45986/0 (socket says 172.21.15.55:45986) 2023-10-06T04:48:47.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.915+0000 7f8910d9c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f890c0ae9c0 0x7f890c1574a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:54590/0 (socket says 172.21.15.55:54590) 2023-10-06T04:48:47.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.915+0000 7f890bfff700 1 -- 172.21.15.55:0/653509139 learned_addr learned my addr 172.21.15.55:0/653509139 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:47.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.915+0000 7f890b7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f890c0cce70 0x7f890c156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:47.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.915+0000 7f890bfff700 1 -- 172.21.15.55:0/653509139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f890c0cce70 msgr2=0x7f890c156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:47.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.915+0000 7f890bfff700 1 --2- 172.21.15.55:0/653509139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f890c0cce70 0x7f890c156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.915+0000 7f890bfff700 1 -- 172.21.15.55:0/653509139 >> v1:172.21.15.163:6789/0 conn(0x7f890c0ae9c0 legacy=0x7f890c1574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:47.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.915+0000 7f890bfff700 1 -- 172.21.15.55:0/653509139 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c06da50 con 0x7f890c0ae5f0 2023-10-06T04:48:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.915+0000 7f890bfff700 1 --2- 172.21.15.55:0/653509139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f890c0ae5f0 0x7f890c0cc930 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0099b0 tx=0x7f88fc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d473d12cad87a6c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.916+0000 7f890affd700 1 -- 172.21.15.55:0/653509139 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc009ea0 con 0x7f890c0ae5f0 2023-10-06T04:48:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.916+0000 7f890affd700 1 -- 172.21.15.55:0/653509139 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88fc00b860 con 0x7f890c0ae5f0 2023-10-06T04:48:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.916+0000 7f890affd700 1 -- 172.21.15.55:0/653509139 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc00bc00 con 0x7f890c0ae5f0 2023-10-06T04:48:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.916+0000 7f891159d700 1 -- 172.21.15.55:0/653509139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f890c0ae5f0 msgr2=0x7f890c0cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.916+0000 7f891159d700 1 --2- 172.21.15.55:0/653509139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f890c0ae5f0 0x7f890c0cc930 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0099b0 tx=0x7f88fc009250 comp rx=0 tx=0).stop 2023-10-06T04:48:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.916+0000 7f891159d700 1 -- 172.21.15.55:0/653509139 shutdown_connections 2023-10-06T04:48:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.916+0000 7f891159d700 1 --2- 172.21.15.55:0/653509139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f890c0ae5f0 0x7f890c0cc930 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.916+0000 7f891159d700 1 --2- 172.21.15.55:0/653509139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f890c0cce70 0x7f890c156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.916+0000 7f891159d700 1 -- 172.21.15.55:0/653509139 >> 172.21.15.55:0/653509139 conn(0x7f890c068e20 msgr2=0x7f890c0cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:47.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.916+0000 7f891159d700 1 -- 172.21.15.55:0/653509139 shutdown_connections 2023-10-06T04:48:47.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.916+0000 7f891159d700 1 -- 172.21.15.55:0/653509139 wait complete. 2023-10-06T04:48:47.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.918+0000 7f891159d700 1 Processor -- start 2023-10-06T04:48:47.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.918+0000 7f891159d700 1 -- start start 2023-10-06T04:48:47.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.919+0000 7f891159d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f890c0ae5f0 0x7f890c15a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:47.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.919+0000 7f891159d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f890c0ae9c0 0x7f890c157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:47.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.919+0000 7f891159d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f890c0cce70 0x7f890c158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:47.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.919+0000 7f891159d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c0c3e60 con 0x7f890c0cce70 2023-10-06T04:48:47.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.919+0000 7f891159d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c0c4140 con 0x7f890c0ae5f0 2023-10-06T04:48:47.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.919+0000 7f891159d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f890c0c4420 con 0x7f890c0ae9c0 2023-10-06T04:48:47.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.919+0000 7f8910d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f890c0cce70 0x7f890c158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.919+0000 7f8910d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f890c0cce70 0x7f890c158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48464/0 (socket says 172.21.15.55:48464) 2023-10-06T04:48:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.919+0000 7f8910d9c700 1 -- 172.21.15.55:0/4035690084 learned_addr learned my addr 172.21.15.55:0/4035690084 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.919+0000 7f890bfff700 1 --2- 172.21.15.55:0/4035690084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f890c0ae5f0 0x7f890c15a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.919+0000 7f890b7fe700 1 --2- 172.21.15.55:0/4035690084 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f890c0ae9c0 0x7f890c157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.920+0000 7f8910d9c700 1 -- 172.21.15.55:0/4035690084 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f890c0ae9c0 msgr2=0x7f890c157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.920+0000 7f8910d9c700 1 --2- 172.21.15.55:0/4035690084 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f890c0ae9c0 0x7f890c157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.920+0000 7f8910d9c700 1 -- 172.21.15.55:0/4035690084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f890c0ae5f0 msgr2=0x7f890c15a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.920+0000 7f8910d9c700 1 --2- 172.21.15.55:0/4035690084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f890c0ae5f0 0x7f890c15a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.920+0000 7f8910d9c700 1 -- 172.21.15.55:0/4035690084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc0094f0 con 0x7f890c0cce70 2023-10-06T04:48:47.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.920+0000 7f8910d9c700 1 --2- 172.21.15.55:0/4035690084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f890c0cce70 0x7f890c158240 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f890400cd00 tx=0x7f8904001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.920+0000 7f88f3fff700 1 -- 172.21.15.55:0/4035690084 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890400ce60 con 0x7f890c0cce70 2023-10-06T04:48:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.920+0000 7f88f3fff700 1 -- 172.21.15.55:0/4035690084 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8904005df0 con 0x7f890c0cce70 2023-10-06T04:48:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.920+0000 7f891159d700 1 -- 172.21.15.55:0/4035690084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f890c0c4700 con 0x7f890c0cce70 2023-10-06T04:48:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.920+0000 7f88f3fff700 1 -- 172.21.15.55:0/4035690084 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890400ab00 con 0x7f890c0cce70 2023-10-06T04:48:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.921+0000 7f891159d700 1 -- 172.21.15.55:0/4035690084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890c15ac00 con 0x7f890c0cce70 2023-10-06T04:48:47.925 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.921+0000 7f891159d700 1 -- 172.21.15.55:0/4035690084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f890c04c3c0 con 0x7f890c0cce70 2023-10-06T04:48:47.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.922+0000 7f88f3fff700 1 -- 172.21.15.55:0/4035690084 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89040070f0 con 0x7f890c0cce70 2023-10-06T04:48:47.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.923+0000 7f88f3fff700 1 --2- 172.21.15.55:0/4035690084 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f88f8066f30 0x7f88f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:47.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.923+0000 7f88f3fff700 1 -- 172.21.15.55:0/4035690084 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f89040847d0 con 0x7f890c0cce70 2023-10-06T04:48:47.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.924+0000 7f890bfff700 1 --2- 172.21.15.55:0/4035690084 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f88f8066f30 0x7f88f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:47.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.925+0000 7f890bfff700 1 --2- 172.21.15.55:0/4035690084 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f88f8066f30 0x7f88f80693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0099b0 tx=0x7f88fc017000 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:47.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:47.928+0000 7f88f3fff700 1 -- 172.21.15.55:0/4035690084 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89040517d0 con 0x7f890c0cce70 2023-10-06T04:48:48.209 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.207+0000 7f891159d700 1 -- 172.21.15.55:0/4035690084 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f890c06dd30 con 0x7f890c0cce70 2023-10-06T04:48:48.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.212+0000 7f88f3fff700 1 -- 172.21.15.55:0/4035690084 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v37) v1 ==== 190+0+0 (secure 0 0 0) 0x7f8904054df0 con 0x7f890c0cce70 2023-10-06T04:48:48.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.214+0000 7f891159d700 1 -- 172.21.15.55:0/4035690084 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f88f8066f30 msgr2=0x7f88f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:48.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.214+0000 7f891159d700 1 --2- 172.21.15.55:0/4035690084 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f88f8066f30 0x7f88f80693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0099b0 tx=0x7f88fc017000 comp rx=0 tx=0).stop 2023-10-06T04:48:48.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.214+0000 7f891159d700 1 -- 172.21.15.55:0/4035690084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f890c0cce70 msgr2=0x7f890c158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:48.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.214+0000 7f891159d700 1 --2- 172.21.15.55:0/4035690084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f890c0cce70 0x7f890c158240 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f890400cd00 tx=0x7f8904001b10 comp rx=0 tx=0).stop 2023-10-06T04:48:48.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.214+0000 7f891159d700 1 -- 172.21.15.55:0/4035690084 shutdown_connections 2023-10-06T04:48:48.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.214+0000 7f891159d700 1 --2- 172.21.15.55:0/4035690084 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f88f8066f30 0x7f88f80693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:48.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.214+0000 7f891159d700 1 --2- 172.21.15.55:0/4035690084 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f890c0cce70 0x7f890c158240 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:48.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.214+0000 7f891159d700 1 --2- 172.21.15.55:0/4035690084 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f890c0ae5f0 0x7f890c15a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:48.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.214+0000 7f891159d700 1 --2- 172.21.15.55:0/4035690084 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f890c0ae9c0 0x7f890c157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:48.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.214+0000 7f891159d700 1 -- 172.21.15.55:0/4035690084 >> 172.21.15.55:0/4035690084 conn(0x7f890c068e20 msgr2=0x7f890c0c1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:48.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.214+0000 7f891159d700 1 -- 172.21.15.55:0/4035690084 shutdown_connections 2023-10-06T04:48:48.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.214+0000 7f891159d700 1 -- 172.21.15.55:0/4035690084 wait complete. 2023-10-06T04:48:48.217 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2023-10-06T04:48:48.233 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:48:48.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.418+0000 7fec8979d700 1 Processor -- start 2023-10-06T04:48:48.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.418+0000 7fec8979d700 1 -- start start 2023-10-06T04:48:48.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.419+0000 7fec8979d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fec84156310 0x7fec840c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:48.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.419+0000 7fec8979d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec840ca860 con 0x7fec840ae710 2023-10-06T04:48:48.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.420+0000 7fec8979d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec840cab40 con 0x7fec841566e0 2023-10-06T04:48:48.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.420+0000 7fec8979d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec840cae20 con 0x7fec84156310 2023-10-06T04:48:48.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.420+0000 7fec88f9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fec840ae710 0x7fec840c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37164/0 (socket says 172.21.15.55:37164) 2023-10-06T04:48:48.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.420+0000 7fec88f9c700 1 -- 172.21.15.55:0/891831054 learned_addr learned my addr 172.21.15.55:0/891831054 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:48.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.420+0000 7fec83fff700 1 --2- 172.21.15.55:0/891831054 >> v2:172.21.15.163:3300/0 conn(0x7fec84156310 0x7fec840c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:48.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.420+0000 7fec82ffd700 1 -- 172.21.15.55:0/891831054 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512156295 0 0) 0x7fec840ca860 con 0x7fec840ae710 2023-10-06T04:48:48.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.420+0000 7fec82ffd700 1 -- 172.21.15.55:0/891831054 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec6c003570 con 0x7fec840ae710 2023-10-06T04:48:48.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.420+0000 7fec83fff700 1 -- 172.21.15.55:0/891831054 >> v1:172.21.15.163:6789/0 conn(0x7fec841566e0 legacy=0x7fec840ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:48.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.420+0000 7fec82ffd700 1 -- 172.21.15.55:0/891831054 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568698981 0 0) 0x7fec840cab40 con 0x7fec841566e0 2023-10-06T04:48:48.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.420+0000 7fec83fff700 1 -- 172.21.15.55:0/891831054 >> v1:172.21.15.55:6789/0 conn(0x7fec840ae710 legacy=0x7fec840c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:48.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.420+0000 7fec83fff700 1 -- 172.21.15.55:0/891831054 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec840cb100 con 0x7fec84156310 2023-10-06T04:48:48.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.421+0000 7fec83fff700 1 --2- 172.21.15.55:0/891831054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec84156310 0x7fec840c9dc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fec740098e0 tx=0x7fec740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c15b012375fc1e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:48.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.421+0000 7fec82ffd700 1 -- 172.21.15.55:0/891831054 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7400b640 con 0x7fec84156310 2023-10-06T04:48:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.421+0000 7fec82ffd700 1 -- 172.21.15.55:0/891831054 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7400b7a0 con 0x7fec84156310 2023-10-06T04:48:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.421+0000 7fec82ffd700 1 -- 172.21.15.55:0/891831054 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7400bb40 con 0x7fec84156310 2023-10-06T04:48:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.422+0000 7fec8979d700 1 -- 172.21.15.55:0/891831054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec84156310 msgr2=0x7fec840c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.422+0000 7fec8979d700 1 --2- 172.21.15.55:0/891831054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec84156310 0x7fec840c9dc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fec740098e0 tx=0x7fec740091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.422+0000 7fec8979d700 1 -- 172.21.15.55:0/891831054 shutdown_connections 2023-10-06T04:48:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.422+0000 7fec8979d700 1 --2- 172.21.15.55:0/891831054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec84156310 0x7fec840c9dc0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.422+0000 7fec8979d700 1 -- 172.21.15.55:0/891831054 >> 172.21.15.55:0/891831054 conn(0x7fec840cc1f0 msgr2=0x7fec840cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.422+0000 7fec8979d700 1 -- 172.21.15.55:0/891831054 shutdown_connections 2023-10-06T04:48:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.422+0000 7fec8979d700 1 -- 172.21.15.55:0/891831054 wait complete. 2023-10-06T04:48:48.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.423+0000 7fec8979d700 1 Processor -- start 2023-10-06T04:48:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.423+0000 7fec8979d700 1 -- start start 2023-10-06T04:48:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.424+0000 7fec8979d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec840ae710 0x7fec8415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.424+0000 7fec8979d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec84156310 0x7fec84153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.424+0000 7fec8979d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec841566e0 0x7fec8414fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.424+0000 7fec8979d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec84153940 con 0x7fec840ae710 2023-10-06T04:48:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.424+0000 7fec8979d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec84169cc0 con 0x7fec841566e0 2023-10-06T04:48:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.424+0000 7fec8979d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec84154410 con 0x7fec84156310 2023-10-06T04:48:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.424+0000 7fec83fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec840ae710 0x7fec8415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.424+0000 7fec83fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec840ae710 0x7fec8415c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48470/0 (socket says 172.21.15.55:48470) 2023-10-06T04:48:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.424+0000 7fec83fff700 1 -- 172.21.15.55:0/216376550 learned_addr learned my addr 172.21.15.55:0/216376550 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.425+0000 7fec88f9c700 1 --2- 172.21.15.55:0/216376550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec841566e0 0x7fec8414fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.425+0000 7fec83fff700 1 -- 172.21.15.55:0/216376550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec84156310 msgr2=0x7fec84153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.425+0000 7fec83fff700 1 --2- 172.21.15.55:0/216376550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec84156310 0x7fec84153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.425+0000 7fec83fff700 1 -- 172.21.15.55:0/216376550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec841566e0 msgr2=0x7fec8414fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.425+0000 7fec83fff700 1 --2- 172.21.15.55:0/216376550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec841566e0 0x7fec8414fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.425+0000 7fec83fff700 1 -- 172.21.15.55:0/216376550 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec74009420 con 0x7fec840ae710 2023-10-06T04:48:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.425+0000 7fec88f9c700 1 --2- 172.21.15.55:0/216376550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec841566e0 0x7fec8414fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.425+0000 7fec83fff700 1 --2- 172.21.15.55:0/216376550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec840ae710 0x7fec8415c4d0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fec8406fe40 tx=0x7fec74017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.425+0000 7fec6bfff700 1 -- 172.21.15.55:0/216376550 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7401f070 con 0x7fec840ae710 2023-10-06T04:48:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.425+0000 7fec6bfff700 1 -- 172.21.15.55:0/216376550 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7400bb20 con 0x7fec840ae710 2023-10-06T04:48:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.425+0000 7fec6bfff700 1 -- 172.21.15.55:0/216376550 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74011670 con 0x7fec840ae710 2023-10-06T04:48:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.425+0000 7fec8979d700 1 -- 172.21.15.55:0/216376550 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec84153b70 con 0x7fec840ae710 2023-10-06T04:48:48.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.426+0000 7fec8979d700 1 -- 172.21.15.55:0/216376550 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec84154130 con 0x7fec840ae710 2023-10-06T04:48:48.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.427+0000 7fec8979d700 1 -- 172.21.15.55:0/216376550 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8404c3c0 con 0x7fec840ae710 2023-10-06T04:48:48.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.427+0000 7fec6bfff700 1 -- 172.21.15.55:0/216376550 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec74011a60 con 0x7fec840ae710 2023-10-06T04:48:48.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.428+0000 7fec6bfff700 1 --2- 172.21.15.55:0/216376550 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fec70066f30 0x7fec700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:48.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.428+0000 7fec6bfff700 1 -- 172.21.15.55:0/216376550 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fec74085d10 con 0x7fec840ae710 2023-10-06T04:48:48.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.429+0000 7fec837fe700 1 --2- 172.21.15.55:0/216376550 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fec70066f30 0x7fec700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:48.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.429+0000 7fec837fe700 1 --2- 172.21.15.55:0/216376550 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fec70066f30 0x7fec700693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fec8415b0a0 tx=0x7fec780079b0 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:48.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.433+0000 7fec6bfff700 1 -- 172.21.15.55:0/216376550 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec7408b020 con 0x7fec840ae710 2023-10-06T04:48:48.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.680+0000 7fec8979d700 1 -- 172.21.15.55:0/216376550 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec840cd0b0 con 0x7fec840ae710 2023-10-06T04:48:48.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.681+0000 7fec6bfff700 1 -- 172.21.15.55:0/216376550 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fec74028090 con 0x7fec840ae710 2023-10-06T04:48:48.683 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:48:48.683 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8308,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/421768289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":421768289},{"type":"v1","addr":"172.21.15.55:6835","nonce":421768289}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8311,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2920562647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2920562647},{"type":"v1","addr":"172.21.15.163:6834","nonce":2920562647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8314,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2324769448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2324769448},{"type":"v1","addr":"172.21.15.163:6835","nonce":2324769448}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:48:42.309816+0000","modified":"2023-10-06T04:48:45.840578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3504492616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3504492616},{"type":"v1","addr":"172.21.15.55:6834","nonce":3504492616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T04:48:48.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.683+0000 7fec8979d700 1 -- 172.21.15.55:0/216376550 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fec70066f30 msgr2=0x7fec700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:48.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.683+0000 7fec8979d700 1 --2- 172.21.15.55:0/216376550 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fec70066f30 0x7fec700693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fec8415b0a0 tx=0x7fec780079b0 comp rx=0 tx=0).stop 2023-10-06T04:48:48.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.683+0000 7fec8979d700 1 -- 172.21.15.55:0/216376550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec840ae710 msgr2=0x7fec8415c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:48.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.683+0000 7fec8979d700 1 --2- 172.21.15.55:0/216376550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec840ae710 0x7fec8415c4d0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fec8406fe40 tx=0x7fec74017000 comp rx=0 tx=0).stop 2023-10-06T04:48:48.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.683+0000 7fec8979d700 1 -- 172.21.15.55:0/216376550 shutdown_connections 2023-10-06T04:48:48.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.683+0000 7fec8979d700 1 --2- 172.21.15.55:0/216376550 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fec70066f30 0x7fec700693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:48.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.683+0000 7fec8979d700 1 --2- 172.21.15.55:0/216376550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec841566e0 0x7fec8414fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:48.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.683+0000 7fec8979d700 1 --2- 172.21.15.55:0/216376550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec840ae710 0x7fec8415c4d0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:48.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.683+0000 7fec8979d700 1 --2- 172.21.15.55:0/216376550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec84156310 0x7fec84153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:48.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.683+0000 7fec8979d700 1 -- 172.21.15.55:0/216376550 >> 172.21.15.55:0/216376550 conn(0x7fec840cc1f0 msgr2=0x7fec840c2140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:48.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.683+0000 7fec8979d700 1 -- 172.21.15.55:0/216376550 shutdown_connections 2023-10-06T04:48:48.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:48.683+0000 7fec8979d700 1 -- 172.21.15.55:0/216376550 wait complete. 2023-10-06T04:48:48.686 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 78 2023-10-06T04:48:49.106 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:48:42.309816+0000', 'modified': '2023-10-06T04:48:45.840578+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8709}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8709': {'gid': 8709, 'name': 'a', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6834/3504492616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6832', 'nonce': 3504492616}, {'type': 'v1', 'addr': '172.21.15.55:6834', 'nonce': 3504492616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T04:48:49.106 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T04:48:49.106 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T04:48:49.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.294+0000 7fdda9b9d700 1 Processor -- start 2023-10-06T04:48:49.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.294+0000 7fdda9b9d700 1 -- start start 2023-10-06T04:48:49.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.295+0000 7fdda9b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4157270 0x7fdda4068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.295+0000 7fdda9b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdda40b0bd0 0x7fdda415d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.295+0000 7fdda9b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda4068d40 con 0x7fdda40688d0 2023-10-06T04:48:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.295+0000 7fdda9b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda415a980 con 0x7fdda40b0bd0 2023-10-06T04:48:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.295+0000 7fdda9b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda415a050 con 0x7fdda4157270 2023-10-06T04:48:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.296+0000 7fdda3fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fdda40688d0 0x7fdda40b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37174/0 (socket says 172.21.15.55:37174) 2023-10-06T04:48:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.296+0000 7fdda3fff700 1 -- 172.21.15.55:0/2404919711 learned_addr learned my addr 172.21.15.55:0/2404919711 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.296+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/2404919711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4157270 0x7fdda4068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.296+0000 7fdda939c700 1 --2- 172.21.15.55:0/2404919711 >> v2:172.21.15.163:3300/0 conn(0x7fdda40b0bd0 0x7fdda415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.296+0000 7fdda37fe700 1 -- 172.21.15.55:0/2404919711 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766461053 0 0) 0x7fdda4068d40 con 0x7fdda40688d0 2023-10-06T04:48:49.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.296+0000 7fdda37fe700 1 -- 172.21.15.55:0/2404919711 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd94003570 con 0x7fdda40688d0 2023-10-06T04:48:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.296+0000 7fdda8b9b700 1 -- 172.21.15.55:0/2404919711 >> v2:172.21.15.163:3300/0 conn(0x7fdda40b0bd0 msgr2=0x7fdda415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.296+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/2404919711 >> v2:172.21.15.163:3300/0 conn(0x7fdda40b0bd0 0x7fdda415d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.296+0000 7fdda8b9b700 1 -- 172.21.15.55:0/2404919711 >> v1:172.21.15.55:6789/0 conn(0x7fdda40688d0 legacy=0x7fdda40b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.296+0000 7fdda8b9b700 1 -- 172.21.15.55:0/2404919711 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda415a330 con 0x7fdda4157270 2023-10-06T04:48:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.296+0000 7fdda939c700 1 --2- 172.21.15.55:0/2404919711 >> v2:172.21.15.163:3300/0 conn(0x7fdda40b0bd0 0x7fdda415d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.297+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/2404919711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4157270 0x7fdda4068390 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdd980098e0 tx=0x7fdd980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cd5204565204a69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.297+0000 7fdda37fe700 1 -- 172.21.15.55:0/2404919711 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98011630 con 0x7fdda4157270 2023-10-06T04:48:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.297+0000 7fdda37fe700 1 -- 172.21.15.55:0/2404919711 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd98011790 con 0x7fdda4157270 2023-10-06T04:48:49.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.297+0000 7fdda37fe700 1 -- 172.21.15.55:0/2404919711 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98011b30 con 0x7fdda4157270 2023-10-06T04:48:49.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.298+0000 7fdda9b9d700 1 -- 172.21.15.55:0/2404919711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4157270 msgr2=0x7fdda4068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.298+0000 7fdda9b9d700 1 --2- 172.21.15.55:0/2404919711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4157270 0x7fdda4068390 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdd980098e0 tx=0x7fdd980091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.298+0000 7fdda9b9d700 1 -- 172.21.15.55:0/2404919711 shutdown_connections 2023-10-06T04:48:49.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.298+0000 7fdda9b9d700 1 --2- 172.21.15.55:0/2404919711 >> v2:172.21.15.163:3300/0 conn(0x7fdda40b0bd0 0x7fdda415d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.298+0000 7fdda9b9d700 1 --2- 172.21.15.55:0/2404919711 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4157270 0x7fdda4068390 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.298+0000 7fdda9b9d700 1 -- 172.21.15.55:0/2404919711 >> 172.21.15.55:0/2404919711 conn(0x7fdda40cd310 msgr2=0x7fdda40cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:49.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.298+0000 7fdda9b9d700 1 -- 172.21.15.55:0/2404919711 shutdown_connections 2023-10-06T04:48:49.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.298+0000 7fdda9b9d700 1 -- 172.21.15.55:0/2404919711 wait complete. 2023-10-06T04:48:49.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.299+0000 7fdda9b9d700 1 Processor -- start 2023-10-06T04:48:49.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.299+0000 7fdda9b9d700 1 -- start start 2023-10-06T04:48:49.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.300+0000 7fdda9b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda40688d0 0x7fdda40ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.300+0000 7fdda9b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdda40b0bd0 0x7fdda40c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.300+0000 7fdda9b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4157270 0x7fdda41468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.300+0000 7fdda9b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda40aeb10 con 0x7fdda40688d0 2023-10-06T04:48:49.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.300+0000 7fdda9b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda40adf00 con 0x7fdda40b0bd0 2023-10-06T04:48:49.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.300+0000 7fdda9b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda40ae1e0 con 0x7fdda4157270 2023-10-06T04:48:49.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.301+0000 7fdda8b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda40688d0 0x7fdda40ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.301+0000 7fdda8b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda40688d0 0x7fdda40ad8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48478/0 (socket says 172.21.15.55:48478) 2023-10-06T04:48:49.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.301+0000 7fdda8b9b700 1 -- 172.21.15.55:0/3794824331 learned_addr learned my addr 172.21.15.55:0/3794824331 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:49.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.301+0000 7fdda3fff700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdda40b0bd0 0x7fdda40c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.301+0000 7fdda939c700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4157270 0x7fdda41468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.301+0000 7fdda8b9b700 1 -- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4157270 msgr2=0x7fdda41468c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.301+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4157270 0x7fdda41468c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.301+0000 7fdda8b9b700 1 -- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdda40b0bd0 msgr2=0x7fdda40c2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.301+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdda40b0bd0 0x7fdda40c2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.301+0000 7fdda8b9b700 1 -- 172.21.15.55:0/3794824331 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c009a20 con 0x7fdda40688d0 2023-10-06T04:48:49.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.301+0000 7fdda939c700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4157270 0x7fdda41468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:48:49.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.301+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda40688d0 0x7fdda40ad8d0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fdda406b880 tx=0x7fdd98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:49.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.302+0000 7fdda0ff9700 1 -- 172.21.15.55:0/3794824331 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98011900 con 0x7fdda40688d0 2023-10-06T04:48:49.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.302+0000 7fdda0ff9700 1 -- 172.21.15.55:0/3794824331 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd98011a60 con 0x7fdda40688d0 2023-10-06T04:48:49.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.302+0000 7fdda0ff9700 1 -- 172.21.15.55:0/3794824331 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98025670 con 0x7fdda40688d0 2023-10-06T04:48:49.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.302+0000 7fdda9b9d700 1 -- 172.21.15.55:0/3794824331 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd98009420 con 0x7fdda40688d0 2023-10-06T04:48:49.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.302+0000 7fdda9b9d700 1 -- 172.21.15.55:0/3794824331 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda40ae7a0 con 0x7fdda40688d0 2023-10-06T04:48:49.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.304+0000 7fdda0ff9700 1 -- 172.21.15.55:0/3794824331 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd98023070 con 0x7fdda40688d0 2023-10-06T04:48:49.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.305+0000 7fdda0ff9700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fdd88066f60 0x7fdd88069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.305+0000 7fdda0ff9700 1 -- 172.21.15.55:0/3794824331 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fdd98085c40 con 0x7fdda40688d0 2023-10-06T04:48:49.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.305+0000 7fdda3fff700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fdd88066f60 0x7fdd88069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.306+0000 7fdda3fff700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fdd88066f60 0x7fdd88069410 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdda40ce190 tx=0x7fdd90009040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:49.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.307+0000 7fddb94b7b80 1 Processor -- start 2023-10-06T04:48:49.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.307+0000 7fddb94b7b80 1 -- start start 2023-10-06T04:48:49.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.308+0000 7fddb94b7b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb9ee50 0x562c6cb79cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.308+0000 7fddb94b7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb7f2c0 0x562c6cb7a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.308+0000 7fddb94b7b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb80a10 0x562c6cb7a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.308+0000 7fddb94b7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6cad4a70 con 0x562c6cb7f2c0 2023-10-06T04:48:49.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.308+0000 7fddb94b7b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6cad2270 con 0x562c6cb80a10 2023-10-06T04:48:49.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.308+0000 7fddb94b7b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c6cace060 con 0x562c6cb9ee50 2023-10-06T04:48:49.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.308+0000 7fdda3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb7f2c0 0x562c6cb7a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.308+0000 7fdda3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb7f2c0 0x562c6cb7a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48484/0 (socket says 172.21.15.55:48484) 2023-10-06T04:48:49.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.308+0000 7fdda3fff700 1 -- 172.21.15.55:0/523344982 learned_addr learned my addr 172.21.15.55:0/523344982 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:49.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.309+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/523344982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb9ee50 0x562c6cb79cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.309+0000 7fdda939c700 1 --2- 172.21.15.55:0/523344982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb80a10 0x562c6cb7a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.309+0000 7fdda3fff700 1 -- 172.21.15.55:0/523344982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb9ee50 msgr2=0x562c6cb79cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:49.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.309+0000 7fdda3fff700 1 --2- 172.21.15.55:0/523344982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb9ee50 0x562c6cb79cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.309+0000 7fdda3fff700 1 -- 172.21.15.55:0/523344982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb80a10 msgr2=0x562c6cb7a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:49.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.309+0000 7fdda3fff700 1 --2- 172.21.15.55:0/523344982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb80a10 0x562c6cb7a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.309+0000 7fdda3fff700 1 -- 172.21.15.55:0/523344982 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c6c968270 con 0x562c6cb7f2c0 2023-10-06T04:48:49.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.309+0000 7fdda3fff700 1 --2- 172.21.15.55:0/523344982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb7f2c0 0x562c6cb7a200 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000f8b0 tx=0x7fdd90015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997a4de573395dd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:49.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.309+0000 7fdd8dffb700 1 -- 172.21.15.55:0/523344982 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90004830 con 0x562c6cb7f2c0 2023-10-06T04:48:49.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.309+0000 7fdd8dffb700 1 -- 172.21.15.55:0/523344982 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd90004990 con 0x562c6cb7f2c0 2023-10-06T04:48:49.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.309+0000 7fdd8dffb700 1 -- 172.21.15.55:0/523344982 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90004d10 con 0x562c6cb7f2c0 2023-10-06T04:48:49.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.310+0000 7fddb94b7b80 1 -- 172.21.15.55:0/523344982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb7f2c0 msgr2=0x562c6cb7a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.310+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/523344982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb7f2c0 0x562c6cb7a200 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000f8b0 tx=0x7fdd90015000 comp rx=0 tx=0).stop 2023-10-06T04:48:49.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.310+0000 7fddb94b7b80 1 -- 172.21.15.55:0/523344982 shutdown_connections 2023-10-06T04:48:49.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.310+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/523344982 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb80a10 0x562c6cb7a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.310+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/523344982 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb9ee50 0x562c6cb79cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.310+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/523344982 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb7f2c0 0x562c6cb7a200 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.310+0000 7fddb94b7b80 1 -- 172.21.15.55:0/523344982 >> 172.21.15.55:0/523344982 conn(0x562c6cb7ee80 msgr2=0x562c6cb7f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:49.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.310+0000 7fddb94b7b80 1 -- 172.21.15.55:0/523344982 shutdown_connections 2023-10-06T04:48:49.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.310+0000 7fddb94b7b80 1 -- 172.21.15.55:0/523344982 wait complete. 2023-10-06T04:48:49.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.311+0000 7fddb94b7b80 1 -- start start 2023-10-06T04:48:49.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.311+0000 7fddb94b7b80 1 Processor -- start 2023-10-06T04:48:49.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.312+0000 7fddb94b7b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb7f2c0 0x562c6cba6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.312+0000 7fddb94b7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb80a10 0x562c6cbc53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.312+0000 7fddb94b7b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb9ee50 0x562c6cbc82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.312+0000 7fddb94b7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6ca79890 con 0x562c6cb80a10 2023-10-06T04:48:49.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.312+0000 7fddb94b7b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6ca94ea0 con 0x562c6cb9ee50 2023-10-06T04:48:49.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.312+0000 7fddb94b7b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c6c950e30 con 0x562c6cb7f2c0 2023-10-06T04:48:49.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.312+0000 7fddb94b7b80 10 client.0 _ll_register_callbacks cb 0x562c6cb41020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:48:49.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.312+0000 7fdda3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb80a10 0x562c6cbc53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.312+0000 7fdda8b9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb7f2c0 0x562c6cba6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.312+0000 7fdda3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb80a10 0x562c6cbc53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48494/0 (socket says 172.21.15.55:48494) 2023-10-06T04:48:49.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.312+0000 7fdda3fff700 1 -- 172.21.15.55:0/4220314877 learned_addr learned my addr 172.21.15.55:0/4220314877 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:49.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.313+0000 7fdda939c700 1 --2- 172.21.15.55:0/4220314877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb9ee50 0x562c6cbc82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.313+0000 7fdda8b9b700 1 -- 172.21.15.55:0/4220314877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb9ee50 msgr2=0x562c6cbc82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:49.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.313+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/4220314877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb9ee50 0x562c6cbc82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.313+0000 7fdda8b9b700 1 -- 172.21.15.55:0/4220314877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb80a10 msgr2=0x562c6cbc53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:49.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.313+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/4220314877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb80a10 0x562c6cbc53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.313+0000 7fdda8b9b700 1 -- 172.21.15.55:0/4220314877 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd94003570 con 0x562c6cb7f2c0 2023-10-06T04:48:49.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.313+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/4220314877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb7f2c0 0x562c6cba6970 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd98083ed0 tx=0x7fdd98011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=862345e8d2fa83d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:49.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.313+0000 7fdd82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.163:3301/0 2023-10-06T04:48:49.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.313+0000 7fdd82ffd700 1 -- 172.21.15.55:0/4220314877 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9806cea0 con 0x562c6cb7f2c0 2023-10-06T04:48:49.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.313+0000 7fdd82ffd700 1 -- 172.21.15.55:0/4220314877 <== mon.2 v2:172.21.15.163:3301/0 2 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fdd98056880 con 0x562c6cb7f2c0 2023-10-06T04:48:49.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.313+0000 7fddb94b7b80 1 -- 172.21.15.55:0/4220314877 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c6cb72940 con 0x562c6cb7f2c0 2023-10-06T04:48:49.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.314+0000 7fdd82ffd700 1 -- 172.21.15.55:0/4220314877 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd98059ad0 con 0x562c6cb7f2c0 2023-10-06T04:48:49.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.314+0000 7fddb94b7b80 10 client.7442 fetch_fsmap learned FSMap version 78 2023-10-06T04:48:49.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.314+0000 7fddb94b7b80 1 -- 172.21.15.55:0/4220314877 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x562c6c90afe0 con 0x562c6cb7f2c0 2023-10-06T04:48:49.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.314+0000 7fdd82ffd700 1 -- 172.21.15.55:0/4220314877 <== mon.2 v2:172.21.15.163:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562c6cb72940 con 0x562c6cb7f2c0 2023-10-06T04:48:49.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.314+0000 7fdd82ffd700 1 -- 172.21.15.55:0/4220314877 <== mon.2 v2:172.21.15.163:3301/0 5 ==== fsmap(e 78) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdd98072250 con 0x562c6cb7f2c0 2023-10-06T04:48:49.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.314+0000 7fddb94b7b80 10 client.7442 fetch_fsmap finished waiting for FSMap version 78 2023-10-06T04:48:49.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.314+0000 7fddb94b7b80 10 client.7442 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T04:48:49.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.314+0000 7fddb94b7b80 20 client.7442 populate_metadata read hostname 'smithi055' 2023-10-06T04:48:49.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.314+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/4220314877 >> [v2:172.21.15.55:6832/3504492616,v1:172.21.15.55:6834/3504492616] conn(0x562c6cbd44e0 0x562c6cbd48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.314+0000 7fddb94b7b80 4 client.7442 mds_command: new command op to 8709 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T04:48:49.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.314+0000 7fddb94b7b80 1 -- 172.21.15.55:0/4220314877 --> [v2:172.21.15.55:6832/3504492616,v1:172.21.15.55:6834/3504492616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c6cb4f870 con 0x562c6cbd44e0 2023-10-06T04:48:49.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.315+0000 7fdda3fff700 1 --2- 172.21.15.55:0/4220314877 >> [v2:172.21.15.55:6832/3504492616,v1:172.21.15.55:6834/3504492616] conn(0x562c6cbd44e0 0x562c6cbd48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.315+0000 7fdda3fff700 1 --2- 172.21.15.55:0/4220314877 >> [v2:172.21.15.55:6832/3504492616,v1:172.21.15.55:6834/3504492616] conn(0x562c6cbd44e0 0x562c6cbd48b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b44b4e34a8af1f server_cookie=c107669495a4faea in_seq=0 out_seq=0 2023-10-06T04:48:49.320 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.315+0000 7fdd82ffd700 10 client.7442 ms_handle_connect on v2:172.21.15.55:6832/3504492616 2023-10-06T04:48:49.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.316+0000 7fdd82ffd700 1 -- 172.21.15.55:0/4220314877 <== mds.0 v2:172.21.15.55:6832/3504492616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x562c6cb4f870 con 0x562c6cbd44e0 2023-10-06T04:48:49.321 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.316+0000 7fdd82ffd700 10 client.7442 handle_command_reply: tid=0 2023-10-06T04:48:49.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.316+0000 7fddb94b7b80 1 client.7442 shutdown 2023-10-06T04:48:49.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.316+0000 7fddb94b7b80 1 -- 172.21.15.55:0/4220314877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb7f2c0 msgr2=0x562c6cba6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.316+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/4220314877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb7f2c0 0x562c6cba6970 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd98083ed0 tx=0x7fdd98011000 comp rx=0 tx=0).stop 2023-10-06T04:48:49.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.316+0000 7fddb94b7b80 1 -- 172.21.15.55:0/4220314877 shutdown_connections 2023-10-06T04:48:49.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.316+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/4220314877 >> [v2:172.21.15.55:6832/3504492616,v1:172.21.15.55:6834/3504492616] conn(0x562c6cbd44e0 0x562c6cbd48b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.316+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/4220314877 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb9ee50 0x562c6cbc82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.316+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/4220314877 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb7f2c0 0x562c6cba6970 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.316+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/4220314877 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb80a10 0x562c6cbc53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.316+0000 7fddb94b7b80 1 -- 172.21.15.55:0/4220314877 >> 172.21.15.55:0/4220314877 conn(0x562c6cb7ee80 msgr2=0x562c6cb7ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:49.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.316+0000 7fdd82ffd700 0 client.7442 ms_handle_reset on v2:172.21.15.55:6832/3504492616 2023-10-06T04:48:49.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.316+0000 7fddb94b7b80 1 -- 172.21.15.55:0/4220314877 shutdown_connections 2023-10-06T04:48:49.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.317+0000 7fddb94b7b80 1 -- 172.21.15.55:0/4220314877 wait complete. 2023-10-06T04:48:49.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.317+0000 7fddb94b7b80 20 client.7442 trim_cache size 0 max 16384 2023-10-06T04:48:49.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.334+0000 7fddb94b7b80 1 Processor -- start 2023-10-06T04:48:49.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.334+0000 7fddb94b7b80 1 -- start start 2023-10-06T04:48:49.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.335+0000 7fddb94b7b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb7f2c0 0x562c6cb74500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.335+0000 7fddb94b7b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb80a10 0x562c6cb7e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.335+0000 7fddb94b7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb9ee50 0x562c6cbad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.335+0000 7fddb94b7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6cbd36b0 con 0x562c6cb9ee50 2023-10-06T04:48:49.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.335+0000 7fddb94b7b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6cbab900 con 0x562c6cb7f2c0 2023-10-06T04:48:49.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.335+0000 7fddb94b7b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c6cbe5810 con 0x562c6cb80a10 2023-10-06T04:48:49.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.335+0000 7fdda939c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb9ee50 0x562c6cbad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.335+0000 7fdda939c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb9ee50 0x562c6cbad6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48510/0 (socket says 172.21.15.55:48510) 2023-10-06T04:48:49.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.335+0000 7fdda939c700 1 -- 172.21.15.55:0/2862194467 learned_addr learned my addr 172.21.15.55:0/2862194467 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:49.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.336+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/2862194467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb7f2c0 0x562c6cb74500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.336+0000 7fdda939c700 1 -- 172.21.15.55:0/2862194467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb80a10 msgr2=0x562c6cb7e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:49.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.336+0000 7fdda3fff700 1 --2- 172.21.15.55:0/2862194467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb80a10 0x562c6cb7e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.336+0000 7fdda939c700 1 --2- 172.21.15.55:0/2862194467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb80a10 0x562c6cb7e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.336+0000 7fdda939c700 1 -- 172.21.15.55:0/2862194467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb7f2c0 msgr2=0x562c6cb74500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:49.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.336+0000 7fdda939c700 1 --2- 172.21.15.55:0/2862194467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb7f2c0 0x562c6cb74500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.336+0000 7fdda939c700 1 -- 172.21.15.55:0/2862194467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c6ca9c780 con 0x562c6cb9ee50 2023-10-06T04:48:49.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.336+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/2862194467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb7f2c0 0x562c6cb74500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:49.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.336+0000 7fdda3fff700 1 --2- 172.21.15.55:0/2862194467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb80a10 0x562c6cb7e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:49.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.336+0000 7fdda939c700 1 --2- 172.21.15.55:0/2862194467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb9ee50 0x562c6cbad6b0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c007af0 tx=0x7fdd9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cb73c1b10f3c62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:49.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.336+0000 7fdd8e7fc700 1 -- 172.21.15.55:0/2862194467 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c00e050 con 0x562c6cb9ee50 2023-10-06T04:48:49.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.336+0000 7fdd8e7fc700 1 -- 172.21.15.55:0/2862194467 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9c01b070 con 0x562c6cb9ee50 2023-10-06T04:48:49.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.336+0000 7fdd8e7fc700 1 -- 172.21.15.55:0/2862194467 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c0169f0 con 0x562c6cb9ee50 2023-10-06T04:48:49.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.337+0000 7fddb94b7b80 1 -- 172.21.15.55:0/2862194467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb9ee50 msgr2=0x562c6cbad6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.337+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/2862194467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb9ee50 0x562c6cbad6b0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c007af0 tx=0x7fdd9c00b040 comp rx=0 tx=0).stop 2023-10-06T04:48:49.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.337+0000 7fddb94b7b80 1 -- 172.21.15.55:0/2862194467 shutdown_connections 2023-10-06T04:48:49.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.337+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/2862194467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb9ee50 0x562c6cbad6b0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.337+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/2862194467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb7f2c0 0x562c6cb74500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.337+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/2862194467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb80a10 0x562c6cb7e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.337+0000 7fddb94b7b80 1 -- 172.21.15.55:0/2862194467 >> 172.21.15.55:0/2862194467 conn(0x562c6cb7ee80 msgr2=0x562c6cbc5210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:49.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.337+0000 7fddb94b7b80 1 -- 172.21.15.55:0/2862194467 shutdown_connections 2023-10-06T04:48:49.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.337+0000 7fddb94b7b80 1 -- 172.21.15.55:0/2862194467 wait complete. 2023-10-06T04:48:49.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.338+0000 7fddb94b7b80 1 -- start start 2023-10-06T04:48:49.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.338+0000 7fddb94b7b80 1 Processor -- start 2023-10-06T04:48:49.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fddb94b7b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb7f2c0 0x562c6cbdc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fddb94b7b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb80a10 0x562c6cbdce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fddb94b7b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb9ee50 0x562c6cc29f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fddb94b7b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6cba4390 con 0x562c6cb7f2c0 2023-10-06T04:48:49.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fddb94b7b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6cbc7b50 con 0x562c6cb80a10 2023-10-06T04:48:49.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fddb94b7b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c6cba3ec0 con 0x562c6cb9ee50 2023-10-06T04:48:49.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fddb94b7b80 10 client.0 _ll_register_callbacks cb 0x562c6cb41020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:48:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fdda8b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb7f2c0 0x562c6cbdc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fdda8b9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb7f2c0 0x562c6cbdc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48522/0 (socket says 172.21.15.55:48522) 2023-10-06T04:48:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fdda8b9b700 1 -- 172.21.15.55:0/1908564207 learned_addr learned my addr 172.21.15.55:0/1908564207 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fdda8b9b700 1 -- 172.21.15.55:0/1908564207 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb9ee50 msgr2=0x562c6cc29f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/1908564207 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb9ee50 0x562c6cc29f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fdda8b9b700 1 -- 172.21.15.55:0/1908564207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb80a10 msgr2=0x562c6cbdce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fdda3fff700 1 --2- 172.21.15.55:0/1908564207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb80a10 0x562c6cbdce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/1908564207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb80a10 0x562c6cbdce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fdda8b9b700 1 -- 172.21.15.55:0/1908564207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9c007820 con 0x562c6cb7f2c0 2023-10-06T04:48:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.339+0000 7fdda3fff700 1 --2- 172.21.15.55:0/1908564207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb80a10 0x562c6cbdce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:49.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fdda8b9b700 1 --2- 172.21.15.55:0/1908564207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb7f2c0 0x562c6cbdc8f0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fdd98083f50 tx=0x7fdd98031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6007e91bcecc89e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fdd82ffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T04:48:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fdd82ffd700 1 -- 172.21.15.55:0/1908564207 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9806cea0 con 0x562c6cb7f2c0 2023-10-06T04:48:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fdd82ffd700 1 -- 172.21.15.55:0/1908564207 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fdd98056880 con 0x562c6cb7f2c0 2023-10-06T04:48:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fddb94b7b80 1 -- 172.21.15.55:0/1908564207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdda415c3f0 con 0x562c6cb7f2c0 2023-10-06T04:48:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fdd82ffd700 1 -- 172.21.15.55:0/1908564207 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd98059ad0 con 0x562c6cb7f2c0 2023-10-06T04:48:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fddb94b7b80 10 client.8847 fetch_fsmap learned FSMap version 78 2023-10-06T04:48:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fddb94b7b80 1 -- 172.21.15.55:0/1908564207 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x562c6ca83470 con 0x562c6cb7f2c0 2023-10-06T04:48:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fdd82ffd700 1 -- 172.21.15.55:0/1908564207 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda415c3f0 con 0x562c6cb7f2c0 2023-10-06T04:48:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fdd82ffd700 1 -- 172.21.15.55:0/1908564207 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 78) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fdd98072250 con 0x562c6cb7f2c0 2023-10-06T04:48:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fddb94b7b80 10 client.8847 fetch_fsmap finished waiting for FSMap version 78 2023-10-06T04:48:49.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fddb94b7b80 10 client.8847 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T04:48:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fddb94b7b80 20 client.8847 populate_metadata read hostname 'smithi055' 2023-10-06T04:48:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/1908564207 >> [v2:172.21.15.55:6832/3504492616,v1:172.21.15.55:6834/3504492616] conn(0x562c6cc29740 0x562c6cc2a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fddb94b7b80 4 client.8847 mds_command: new command op to 8709 tid=0[{"prefix": "status"}] 2023-10-06T04:48:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.340+0000 7fddb94b7b80 1 -- 172.21.15.55:0/1908564207 --> [v2:172.21.15.55:6832/3504492616,v1:172.21.15.55:6834/3504492616] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fdd90004990 con 0x562c6cc29740 2023-10-06T04:48:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.341+0000 7fdda3fff700 1 --2- 172.21.15.55:0/1908564207 >> [v2:172.21.15.55:6832/3504492616,v1:172.21.15.55:6834/3504492616] conn(0x562c6cc29740 0x562c6cc2a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.341+0000 7fdda3fff700 1 --2- 172.21.15.55:0/1908564207 >> [v2:172.21.15.55:6832/3504492616,v1:172.21.15.55:6834/3504492616] conn(0x562c6cc29740 0x562c6cc2a460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c2842469daf220 server_cookie=91ca244b15033fc4 in_seq=0 out_seq=0 2023-10-06T04:48:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.341+0000 7fdd82ffd700 10 client.8847 ms_handle_connect on v2:172.21.15.55:6832/3504492616 2023-10-06T04:48:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.341+0000 7fdd82ffd700 1 -- 172.21.15.55:0/1908564207 <== mds.0 v2:172.21.15.55:6832/3504492616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7fdd90004990 con 0x562c6cc29740 2023-10-06T04:48:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.341+0000 7fdd82ffd700 10 client.8847 handle_command_reply: tid=0 2023-10-06T04:48:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.341+0000 7fddb94b7b80 1 client.8847 shutdown 2023-10-06T04:48:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.341+0000 7fddb94b7b80 1 -- 172.21.15.55:0/1908564207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb7f2c0 msgr2=0x562c6cbdc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.341+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/1908564207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb7f2c0 0x562c6cbdc8f0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fdd98083f50 tx=0x7fdd98031000 comp rx=0 tx=0).stop 2023-10-06T04:48:49.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.342+0000 7fddb94b7b80 1 -- 172.21.15.55:0/1908564207 shutdown_connections 2023-10-06T04:48:49.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.342+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/1908564207 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x562c6cb9ee50 0x562c6cc29f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.342+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/1908564207 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x562c6cb7f2c0 0x562c6cbdc8f0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.342+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/1908564207 >> [v2:172.21.15.55:6832/3504492616,v1:172.21.15.55:6834/3504492616] conn(0x562c6cc29740 0x562c6cc2a460 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.342+0000 7fddb94b7b80 1 --2- 172.21.15.55:0/1908564207 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x562c6cb80a10 0x562c6cbdce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.342+0000 7fddb94b7b80 1 -- 172.21.15.55:0/1908564207 >> 172.21.15.55:0/1908564207 conn(0x562c6cb7ee80 msgr2=0x562c6cb9feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:49.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.342+0000 7fdd82ffd700 0 client.8847 ms_handle_reset on v2:172.21.15.55:6832/3504492616 2023-10-06T04:48:49.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.342+0000 7fddb94b7b80 1 -- 172.21.15.55:0/1908564207 shutdown_connections 2023-10-06T04:48:49.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.342+0000 7fddb94b7b80 1 -- 172.21.15.55:0/1908564207 wait complete. 2023-10-06T04:48:49.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.342+0000 7fddb94b7b80 20 client.8847 trim_cache size 0 max 16384 2023-10-06T04:48:49.350 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T04:48:49.350 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "8132c8ba-ee1c-41a2-b781-913925034ab6", 2023-10-06T04:48:49.350 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2023-10-06T04:48:49.350 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 8709, 2023-10-06T04:48:49.350 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2023-10-06T04:48:49.350 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2023-10-06T04:48:49.350 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2023-10-06T04:48:49.350 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 6.5328101810000003, 2023-10-06T04:48:49.351 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 78, 2023-10-06T04:48:49.351 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 160, 2023-10-06T04:48:49.351 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 160, 2023-10-06T04:48:49.351 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 13.285883837 2023-10-06T04:48:49.351 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T04:48:49.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.342+0000 7fdda9b9d700 1 -- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fdd88066f60 msgr2=0x7fdd88069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.342+0000 7fdda9b9d700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fdd88066f60 0x7fdd88069410 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdda40ce190 tx=0x7fdd90009040 comp rx=0 tx=0).stop 2023-10-06T04:48:49.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.342+0000 7fdda9b9d700 1 -- 172.21.15.55:0/3794824331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda40688d0 msgr2=0x7fdda40ad8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.342+0000 7fdda9b9d700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda40688d0 0x7fdda40ad8d0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fdda406b880 tx=0x7fdd98017000 comp rx=0 tx=0).stop 2023-10-06T04:48:49.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.343+0000 7fdda9b9d700 1 -- 172.21.15.55:0/3794824331 shutdown_connections 2023-10-06T04:48:49.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.343+0000 7fdda9b9d700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fdd88066f60 0x7fdd88069410 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.343+0000 7fdda9b9d700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdda4157270 0x7fdda41468c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.343+0000 7fdda9b9d700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdda40688d0 0x7fdda40ad8d0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.343+0000 7fdda9b9d700 1 --2- 172.21.15.55:0/3794824331 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdda40b0bd0 0x7fdda40c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.343+0000 7fdda9b9d700 1 -- 172.21.15.55:0/3794824331 >> 172.21.15.55:0/3794824331 conn(0x7fdda40cd310 msgr2=0x7fdda40abfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:49.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.343+0000 7fdda9b9d700 1 -- 172.21.15.55:0/3794824331 shutdown_connections 2023-10-06T04:48:49.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.343+0000 7fdda9b9d700 1 -- 172.21.15.55:0/3794824331 wait complete. 2023-10-06T04:48:49.361 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:48:49.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.545+0000 7fec94b9d700 1 Processor -- start 2023-10-06T04:48:49.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.546+0000 7fec94b9d700 1 -- start start 2023-10-06T04:48:49.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.547+0000 7fec94b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec9006d950 0x7fec900c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.547+0000 7fec94b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fec9006dd20 0x7fec900c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.547+0000 7fec94b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9006ae70 con 0x7fec9006a0a0 2023-10-06T04:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.547+0000 7fec94b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9006a540 con 0x7fec9006dd20 2023-10-06T04:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.547+0000 7fec94b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9006a820 con 0x7fec9006d950 2023-10-06T04:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.547+0000 7fec8ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fec9006dd20 0x7fec900c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.547+0000 7fec8ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fec9006dd20 0x7fec900c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48532/0 (socket says 172.21.15.55:48532) 2023-10-06T04:48:49.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.547+0000 7fec8ffff700 1 -- 172.21.15.55:0/3698011609 learned_addr learned my addr 172.21.15.55:0/3698011609 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:49.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.547+0000 7fec8effd700 1 --1- 172.21.15.55:0/3698011609 >> v1:172.21.15.55:6789/0 conn(0x7fec9006a0a0 0x7fec900c6c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37178/0 (socket says 172.21.15.55:37178) 2023-10-06T04:48:49.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.547+0000 7fec8f7fe700 1 --2- 172.21.15.55:0/3698011609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec9006d950 0x7fec900c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.548+0000 7fec8ffff700 1 -- 172.21.15.55:0/3698011609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec9006d950 msgr2=0x7fec900c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:49.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.548+0000 7fec8ffff700 1 --2- 172.21.15.55:0/3698011609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec9006d950 0x7fec900c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.548+0000 7fec8ffff700 1 -- 172.21.15.55:0/3698011609 >> v1:172.21.15.55:6789/0 conn(0x7fec9006a0a0 legacy=0x7fec900c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.548+0000 7fec8ffff700 1 -- 172.21.15.55:0/3698011609 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9006ab00 con 0x7fec9006dd20 2023-10-06T04:48:49.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.548+0000 7fec8e7fc700 1 -- 172.21.15.55:0/3698011609 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996465672 0 0) 0x7fec9006ae70 con 0x7fec9006a0a0 2023-10-06T04:48:49.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.548+0000 7fec8ffff700 1 --2- 172.21.15.55:0/3698011609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec9006dd20 0x7fec900c7320 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fec88009f20 tx=0x7fec88009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558eb578421e7cb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:49.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.548+0000 7fec8e7fc700 1 -- 172.21.15.55:0/3698011609 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8800e070 con 0x7fec9006dd20 2023-10-06T04:48:49.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.548+0000 7fec8e7fc700 1 -- 172.21.15.55:0/3698011609 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8800b9d0 con 0x7fec9006dd20 2023-10-06T04:48:49.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.548+0000 7fec8e7fc700 1 -- 172.21.15.55:0/3698011609 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec880092e0 con 0x7fec9006dd20 2023-10-06T04:48:49.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.549+0000 7fec94b9d700 1 -- 172.21.15.55:0/3698011609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec9006dd20 msgr2=0x7fec900c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.549+0000 7fec94b9d700 1 --2- 172.21.15.55:0/3698011609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec9006dd20 0x7fec900c7320 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fec88009f20 tx=0x7fec88009840 comp rx=0 tx=0).stop 2023-10-06T04:48:49.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.549+0000 7fec94b9d700 1 -- 172.21.15.55:0/3698011609 shutdown_connections 2023-10-06T04:48:49.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.549+0000 7fec94b9d700 1 --2- 172.21.15.55:0/3698011609 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec9006dd20 0x7fec900c7320 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.549+0000 7fec94b9d700 1 --2- 172.21.15.55:0/3698011609 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec9006d950 0x7fec900c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.549+0000 7fec94b9d700 1 -- 172.21.15.55:0/3698011609 >> 172.21.15.55:0/3698011609 conn(0x7fec900cc1f0 msgr2=0x7fec900cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:49.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.549+0000 7fec94b9d700 1 -- 172.21.15.55:0/3698011609 shutdown_connections 2023-10-06T04:48:49.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.549+0000 7fec94b9d700 1 -- 172.21.15.55:0/3698011609 wait complete. 2023-10-06T04:48:49.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.550+0000 7fec94b9d700 1 Processor -- start 2023-10-06T04:48:49.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.550+0000 7fec94b9d700 1 -- start start 2023-10-06T04:48:49.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.551+0000 7fec94b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec9006a0a0 0x7fec900c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.551+0000 7fec94b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec9006d950 0x7fec90144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.551+0000 7fec94b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec9006dd20 0x7fec9015a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.551+0000 7fec94b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec900c5ac0 con 0x7fec9006dd20 2023-10-06T04:48:49.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.551+0000 7fec94b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec901451c0 con 0x7fec9006d950 2023-10-06T04:48:49.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.551+0000 7fec94b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec900c35b0 con 0x7fec9006a0a0 2023-10-06T04:48:49.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.552+0000 7fec8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec9006dd20 0x7fec9015a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.552+0000 7fec8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec9006dd20 0x7fec9015a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48534/0 (socket says 172.21.15.55:48534) 2023-10-06T04:48:49.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.552+0000 7fec8ffff700 1 -- 172.21.15.55:0/2073048438 learned_addr learned my addr 172.21.15.55:0/2073048438 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:49.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.552+0000 7fec8f7fe700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec9006a0a0 0x7fec900c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.552+0000 7fec8effd700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec9006d950 0x7fec90144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.552+0000 7fec8ffff700 1 -- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec9006a0a0 msgr2=0x7fec900c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.552+0000 7fec8ffff700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec9006a0a0 0x7fec900c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.552+0000 7fec8ffff700 1 -- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec9006d950 msgr2=0x7fec90144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.552+0000 7fec8ffff700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec9006d950 0x7fec90144c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.552+0000 7fec8ffff700 1 -- 172.21.15.55:0/2073048438 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec88009ae0 con 0x7fec9006dd20 2023-10-06T04:48:49.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.552+0000 7fec8effd700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec9006d950 0x7fec90144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:48:49.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.552+0000 7fec8ffff700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec9006dd20 0x7fec9015a290 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fec9006fe40 tx=0x7fec88018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:49.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.553+0000 7fec777fe700 1 -- 172.21.15.55:0/2073048438 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88008bc0 con 0x7fec9006dd20 2023-10-06T04:48:49.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.553+0000 7fec777fe700 1 -- 172.21.15.55:0/2073048438 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec88008d40 con 0x7fec9006dd20 2023-10-06T04:48:49.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.553+0000 7fec777fe700 1 -- 172.21.15.55:0/2073048438 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec880266b0 con 0x7fec9006dd20 2023-10-06T04:48:49.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.553+0000 7fec94b9d700 1 -- 172.21.15.55:0/2073048438 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec900c3890 con 0x7fec9006dd20 2023-10-06T04:48:49.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.553+0000 7fec94b9d700 1 -- 172.21.15.55:0/2073048438 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec900c3e50 con 0x7fec9006dd20 2023-10-06T04:48:49.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.554+0000 7fec94b9d700 1 -- 172.21.15.55:0/2073048438 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec9004c3c0 con 0x7fec9006dd20 2023-10-06T04:48:49.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.555+0000 7fec777fe700 1 -- 172.21.15.55:0/2073048438 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec88003040 con 0x7fec9006dd20 2023-10-06T04:48:49.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.556+0000 7fec777fe700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fec7c066f30 0x7fec7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:49.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.556+0000 7fec777fe700 1 -- 172.21.15.55:0/2073048438 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fec880867c0 con 0x7fec9006dd20 2023-10-06T04:48:49.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.556+0000 7fec8f7fe700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fec7c066f30 0x7fec7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:49.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.557+0000 7fec8f7fe700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fec7c066f30 0x7fec7c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fec80009d10 tx=0x7fec800091a0 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:49.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.560+0000 7fec777fe700 1 -- 172.21.15.55:0/2073048438 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec880536a0 con 0x7fec9006dd20 2023-10-06T04:48:49.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.808+0000 7fec94b9d700 1 -- 172.21.15.55:0/2073048438 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fec901535e0 con 0x7fec9006dd20 2023-10-06T04:48:49.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.808+0000 7fec777fe700 1 -- 172.21.15.55:0/2073048438 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) v1 ==== 53+0+73 (secure 0 0 0) 0x7fec88056cc0 con 0x7fec9006dd20 2023-10-06T04:48:49.811 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:48:49.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.810+0000 7fec94b9d700 1 -- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fec7c066f30 msgr2=0x7fec7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.810+0000 7fec94b9d700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fec7c066f30 0x7fec7c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fec80009d10 tx=0x7fec800091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.810+0000 7fec94b9d700 1 -- 172.21.15.55:0/2073048438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec9006dd20 msgr2=0x7fec9015a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:49.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.810+0000 7fec94b9d700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec9006dd20 0x7fec9015a290 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fec9006fe40 tx=0x7fec88018000 comp rx=0 tx=0).stop 2023-10-06T04:48:49.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.811+0000 7fec94b9d700 1 -- 172.21.15.55:0/2073048438 shutdown_connections 2023-10-06T04:48:49.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.811+0000 7fec94b9d700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fec7c066f30 0x7fec7c0693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.811+0000 7fec94b9d700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fec9006dd20 0x7fec9015a290 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.811+0000 7fec94b9d700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fec9006a0a0 0x7fec900c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.811+0000 7fec94b9d700 1 --2- 172.21.15.55:0/2073048438 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fec9006d950 0x7fec90144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.811+0000 7fec94b9d700 1 -- 172.21.15.55:0/2073048438 >> 172.21.15.55:0/2073048438 conn(0x7fec900cc1f0 msgr2=0x7fec90153de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.811+0000 7fec94b9d700 1 -- 172.21.15.55:0/2073048438 shutdown_connections 2023-10-06T04:48:49.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:49.811+0000 7fec94b9d700 1 -- 172.21.15.55:0/2073048438 wait complete. 2023-10-06T04:48:49.831 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T04:48:49.831 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:48:49.832 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2023-10-06T04:48:49.832 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T04:48:49.832 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T04:48:49.832 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T04:48:49.832 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T04:48:49.832 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:48:49.832 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:48:49.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:48:49.833 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2023-10-06T04:48:49.872 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:48:49.872 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:48:49.872 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:48:49.872 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:48:49.872 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T04:48:49.872 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:48:49.872 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:48:49.873 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:48:49.873 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2023-10-06T04:48:49.873 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:48:49.873 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:48:49.873 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2023-10-06T04:48:49.873 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:48:49.873 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:48:49.873 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2023-10-06T04:48:49.873 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T04:48:49.873 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 37711sec preferred_lft 37711sec 2023-10-06T04:48:49.874 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link 2023-10-06T04:48:49.874 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:48:49.874 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:48:49.874 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:48:49.874 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2023-10-06T04:48:49.874 INFO:teuthology.orchestra.run.smithi055.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:48:49.874 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff 2023-10-06T04:48:49.874 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:48:49.874 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:48:49.875 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::1099:8eff:fe6d:102/64 scope link 2023-10-06T04:48:49.875 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:48:49.875 INFO:teuthology.orchestra.run.smithi055.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:48:49.875 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether f2:5d:53:d2:00:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:48:49.875 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f05d:53ff:fed2:a/64 scope link 2023-10-06T04:48:49.875 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:48:49.875 INFO:teuthology.orchestra.run.smithi055.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:48:49.875 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:48:49.875 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::e037:33ff:feb7:6c8f/64 scope link 2023-10-06T04:48:49.875 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:48:49.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:48:49.876 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:48:49.972 INFO:teuthology.orchestra.run.smithi055.stdout:1 2023-10-06T04:48:49.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:48:49.975 DEBUG:teuthology.orchestra.run.smithi055:> route 2023-10-06T04:48:50.015 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2023-10-06T04:48:50.015 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:48:50.015 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T04:48:50.015 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T04:48:50.015 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:48:50.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:48:50.016 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T04:48:50.017 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T04:48:50.017 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T04:48:50.017 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T04:48:50.017 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T04:48:50.017 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T04:48:50.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:48:50.215 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T04:48:50.251 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T04:48:50.251 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:48:50.252 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:48:50.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:48:50.252 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2023-10-06T04:48:50.354 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:48:50.354 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T04:48:50.391 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:48:50.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:48:50.391 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:48:50.467 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T04:48:50.467 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:48:50.467 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T04:48:50.600 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T04:48:50.600 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T04:48:50.600 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T04:48:50.616 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T04:48:50.616 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T04:48:50.617 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T04:48:50.617 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T04:48:50.617 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T04:48:50.638 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T04:48:50.638 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T04:48:50.638 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T04:48:50.638 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T04:48:50.638 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T04:48:50.638 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T04:48:50.638 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T04:48:50.638 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T04:48:50.642 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T04:48:50.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:48:50.642 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:48:50.706 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T04:48:50.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.894+0000 7f510e19d700 1 Processor -- start 2023-10-06T04:48:50.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.894+0000 7f510e19d700 1 -- start start 2023-10-06T04:48:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.895+0000 7f510e19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5108156310 0x7f51080c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.895+0000 7f510e19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51080ae710 0x7f51080c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:50.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.895+0000 7f510e19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51080ca860 con 0x7f51081566e0 2023-10-06T04:48:50.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.895+0000 7f510e19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51080cab40 con 0x7f5108156310 2023-10-06T04:48:50.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.895+0000 7f510e19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51080cae20 con 0x7f51080ae710 2023-10-06T04:48:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.896+0000 7f510d19b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5108156310 0x7f51080c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.896+0000 7f510d99c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51080ae710 0x7f51080c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.896+0000 7f510d99c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51080ae710 0x7f51080c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:50196/0 (socket says 172.21.15.55:50196) 2023-10-06T04:48:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.896+0000 7f510d99c700 1 -- 172.21.15.55:0/2773603249 learned_addr learned my addr 172.21.15.55:0/2773603249 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.896+0000 7f510c99a700 1 --1- 172.21.15.55:0/2773603249 >> v1:172.21.15.163:6789/0 conn(0x7f51081566e0 0x7f51080ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:54610/0 (socket says 172.21.15.55:54610) 2023-10-06T04:48:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.897+0000 7f510d99c700 1 -- 172.21.15.55:0/2773603249 >> v2:172.21.15.163:3300/0 conn(0x7f5108156310 msgr2=0x7f51080c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:50.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.897+0000 7f510d99c700 1 --2- 172.21.15.55:0/2773603249 >> v2:172.21.15.163:3300/0 conn(0x7f5108156310 0x7f51080c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.897+0000 7f510d99c700 1 -- 172.21.15.55:0/2773603249 >> v1:172.21.15.163:6789/0 conn(0x7f51081566e0 legacy=0x7f51080ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.897+0000 7f510d99c700 1 -- 172.21.15.55:0/2773603249 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51080cb100 con 0x7f51080ae710 2023-10-06T04:48:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.897+0000 7f510d19b700 1 --2- 172.21.15.55:0/2773603249 >> v2:172.21.15.163:3300/0 conn(0x7f5108156310 0x7f51080c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:48:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.897+0000 7f510d99c700 1 --2- 172.21.15.55:0/2773603249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51080ae710 0x7f51080c6830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5100009f20 tx=0x7f5100009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2169f7e7e27dc4fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.897+0000 7f50fffff700 1 -- 172.21.15.55:0/2773603249 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510000e070 con 0x7f51080ae710 2023-10-06T04:48:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.897+0000 7f50fffff700 1 -- 172.21.15.55:0/2773603249 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510000b9d0 con 0x7f51080ae710 2023-10-06T04:48:50.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.897+0000 7f50fffff700 1 -- 172.21.15.55:0/2773603249 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51000092e0 con 0x7f51080ae710 2023-10-06T04:48:50.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.898+0000 7f510e19d700 1 -- 172.21.15.55:0/2773603249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51080ae710 msgr2=0x7f51080c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:50.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.898+0000 7f510e19d700 1 --2- 172.21.15.55:0/2773603249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51080ae710 0x7f51080c6830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5100009f20 tx=0x7f5100009840 comp rx=0 tx=0).stop 2023-10-06T04:48:50.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.898+0000 7f510e19d700 1 -- 172.21.15.55:0/2773603249 shutdown_connections 2023-10-06T04:48:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.898+0000 7f510e19d700 1 --2- 172.21.15.55:0/2773603249 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f51080ae710 0x7f51080c6830 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.898+0000 7f510e19d700 1 --2- 172.21.15.55:0/2773603249 >> v2:172.21.15.163:3300/0 conn(0x7f5108156310 0x7f51080c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.898+0000 7f510e19d700 1 -- 172.21.15.55:0/2773603249 >> 172.21.15.55:0/2773603249 conn(0x7f51080cc1f0 msgr2=0x7f51080cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.898+0000 7f510e19d700 1 -- 172.21.15.55:0/2773603249 shutdown_connections 2023-10-06T04:48:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.898+0000 7f510e19d700 1 -- 172.21.15.55:0/2773603249 wait complete. 2023-10-06T04:48:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.899+0000 7f510e19d700 1 Processor -- start 2023-10-06T04:48:50.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.899+0000 7f510e19d700 1 -- start start 2023-10-06T04:48:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.900+0000 7f510e19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51080ae710 0x7f5108151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.900+0000 7f510e19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5108156310 0x7f510814f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.900+0000 7f510e19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51081566e0 0x7f510814e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.900+0000 7f510e19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5108150430 con 0x7f51081566e0 2023-10-06T04:48:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.900+0000 7f510e19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510814f820 con 0x7f51080ae710 2023-10-06T04:48:50.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.900+0000 7f510e19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510814fb00 con 0x7f5108156310 2023-10-06T04:48:50.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.901+0000 7f510d99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51081566e0 0x7f510814e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:50.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.901+0000 7f510d99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51081566e0 0x7f510814e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48546/0 (socket says 172.21.15.55:48546) 2023-10-06T04:48:50.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.901+0000 7f510d99c700 1 -- 172.21.15.55:0/146467057 learned_addr learned my addr 172.21.15.55:0/146467057 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:50.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.901+0000 7f510d19b700 1 --2- 172.21.15.55:0/146467057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51080ae710 0x7f5108151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:50.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.901+0000 7f510c99a700 1 --2- 172.21.15.55:0/146467057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5108156310 0x7f510814f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:50.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.901+0000 7f510d99c700 1 -- 172.21.15.55:0/146467057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5108156310 msgr2=0x7f510814f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:50.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.901+0000 7f510d99c700 1 --2- 172.21.15.55:0/146467057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5108156310 0x7f510814f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:50.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.901+0000 7f510d99c700 1 -- 172.21.15.55:0/146467057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51080ae710 msgr2=0x7f5108151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:50.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.901+0000 7f510d99c700 1 --2- 172.21.15.55:0/146467057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51080ae710 0x7f5108151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.901+0000 7f510d99c700 1 -- 172.21.15.55:0/146467057 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510400b6d0 con 0x7f51081566e0 2023-10-06T04:48:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.901+0000 7f510d99c700 1 --2- 172.21.15.55:0/146467057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51081566e0 0x7f510814e630 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f51080caf60 tx=0x7f5100018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.902+0000 7f50fd7fa700 1 -- 172.21.15.55:0/146467057 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100008ad0 con 0x7f51081566e0 2023-10-06T04:48:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.902+0000 7f50fd7fa700 1 -- 172.21.15.55:0/146467057 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5100008c30 con 0x7f51081566e0 2023-10-06T04:48:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.902+0000 7f50fd7fa700 1 -- 172.21.15.55:0/146467057 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100026460 con 0x7f51081566e0 2023-10-06T04:48:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.902+0000 7f510e19d700 1 -- 172.21.15.55:0/146467057 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5100009ae0 con 0x7f51081566e0 2023-10-06T04:48:50.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.902+0000 7f510e19d700 1 -- 172.21.15.55:0/146467057 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51081500c0 con 0x7f51081566e0 2023-10-06T04:48:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.903+0000 7f510e19d700 1 -- 172.21.15.55:0/146467057 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510804c3c0 con 0x7f51081566e0 2023-10-06T04:48:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.904+0000 7f50fd7fa700 1 -- 172.21.15.55:0/146467057 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5100003040 con 0x7f51081566e0 2023-10-06T04:48:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.905+0000 7f50fd7fa700 1 --2- 172.21.15.55:0/146467057 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f50ec066f30 0x7f50ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.905+0000 7f50fd7fa700 1 -- 172.21.15.55:0/146467057 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f51000864a0 con 0x7f51081566e0 2023-10-06T04:48:50.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.905+0000 7f510d19b700 1 --2- 172.21.15.55:0/146467057 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f50ec066f30 0x7f50ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:50.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.906+0000 7f510d19b700 1 --2- 172.21.15.55:0/146467057 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f50ec066f30 0x7f50ec0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f510400bfd0 tx=0x7f510400b430 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:50.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:50.909+0000 7f50fd7fa700 1 -- 172.21.15.55:0/146467057 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5100053380 con 0x7f51081566e0 2023-10-06T04:48:51.142 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.140+0000 7f510e19d700 1 -- 172.21.15.55:0/146467057 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f51080cd0b0 con 0x7f50ec066f30 2023-10-06T04:48:51.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.143+0000 7f50fd7fa700 1 -- 172.21.15.55:0/146467057 <== mgr.7205 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f51080cd0b0 con 0x7f50ec066f30 2023-10-06T04:48:51.144 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T04:48:51.144 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:48:51.144 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephfs" 2023-10-06T04:48:51.144 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:48:51.144 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T04:48:51.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.144+0000 7f510e19d700 1 -- 172.21.15.55:0/146467057 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f50ec066f30 msgr2=0x7f50ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:51.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.144+0000 7f510e19d700 1 --2- 172.21.15.55:0/146467057 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f50ec066f30 0x7f50ec0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f510400bfd0 tx=0x7f510400b430 comp rx=0 tx=0).stop 2023-10-06T04:48:51.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.144+0000 7f510e19d700 1 -- 172.21.15.55:0/146467057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51081566e0 msgr2=0x7f510814e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:51.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.144+0000 7f510e19d700 1 --2- 172.21.15.55:0/146467057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51081566e0 0x7f510814e630 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f51080caf60 tx=0x7f5100018000 comp rx=0 tx=0).stop 2023-10-06T04:48:51.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.144+0000 7f510e19d700 1 -- 172.21.15.55:0/146467057 shutdown_connections 2023-10-06T04:48:51.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.144+0000 7f510e19d700 1 --2- 172.21.15.55:0/146467057 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f50ec066f30 0x7f50ec0693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:51.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.144+0000 7f510e19d700 1 --2- 172.21.15.55:0/146467057 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f51081566e0 0x7f510814e630 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:51.147 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.144+0000 7f510e19d700 1 --2- 172.21.15.55:0/146467057 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f51080ae710 0x7f5108151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:51.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.144+0000 7f510e19d700 1 --2- 172.21.15.55:0/146467057 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5108156310 0x7f510814f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:51.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.144+0000 7f510e19d700 1 -- 172.21.15.55:0/146467057 >> 172.21.15.55:0/146467057 conn(0x7f51080cc1f0 msgr2=0x7f51080c1530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:51.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.144+0000 7f510e19d700 1 -- 172.21.15.55:0/146467057 shutdown_connections 2023-10-06T04:48:51.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.144+0000 7f510e19d700 1 -- 172.21.15.55:0/146467057 wait complete. 2023-10-06T04:48:51.164 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T04:48:51.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.352+0000 7fb88359e700 1 Processor -- start 2023-10-06T04:48:51.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.352+0000 7fb88359e700 1 -- start start 2023-10-06T04:48:51.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.353+0000 7fb88359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb8840b4e70 0x7fb8840ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:51.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.353+0000 7fb88359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8840cfb00 con 0x7fb8840b16d0 2023-10-06T04:48:51.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.353+0000 7fb88359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88415e060 con 0x7fb8840b1b40 2023-10-06T04:48:51.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.353+0000 7fb88359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8840b5c40 con 0x7fb8840b4e70 2023-10-06T04:48:51.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.353+0000 7fb88259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb8840b4e70 0x7fb8840ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:51.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.354+0000 7fb88259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb8840b4e70 0x7fb8840ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48550/0 (socket says 172.21.15.55:48550) 2023-10-06T04:48:51.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.354+0000 7fb88259c700 1 -- 172.21.15.55:0/541892682 learned_addr learned my addr 172.21.15.55:0/541892682 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:51.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.354+0000 7fb88259c700 1 -- 172.21.15.55:0/541892682 >> v1:172.21.15.163:6789/0 conn(0x7fb8840b1b40 legacy=0x7fb88415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:51.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.354+0000 7fb88259c700 1 -- 172.21.15.55:0/541892682 >> v1:172.21.15.55:6789/0 conn(0x7fb8840b16d0 legacy=0x7fb88415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:51.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.354+0000 7fb88259c700 1 -- 172.21.15.55:0/541892682 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8840b5f20 con 0x7fb8840b4e70 2023-10-06T04:48:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.354+0000 7fb88259c700 1 --2- 172.21.15.55:0/541892682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8840b4e70 0x7fb8840ccff0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fb8740099b0 tx=0x7fb874009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10e346e72f37cd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.354+0000 7fb88159a700 1 -- 172.21.15.55:0/541892682 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87400b820 con 0x7fb8840b4e70 2023-10-06T04:48:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.355+0000 7fb88159a700 1 -- 172.21.15.55:0/541892682 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87400b980 con 0x7fb8840b4e70 2023-10-06T04:48:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.355+0000 7fb88159a700 1 -- 172.21.15.55:0/541892682 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87400bd20 con 0x7fb8840b4e70 2023-10-06T04:48:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.355+0000 7fb88359e700 1 -- 172.21.15.55:0/541892682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8840b4e70 msgr2=0x7fb8840ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.355+0000 7fb88359e700 1 --2- 172.21.15.55:0/541892682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8840b4e70 0x7fb8840ccff0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fb8740099b0 tx=0x7fb874009250 comp rx=0 tx=0).stop 2023-10-06T04:48:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.355+0000 7fb88359e700 1 -- 172.21.15.55:0/541892682 shutdown_connections 2023-10-06T04:48:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.355+0000 7fb88359e700 1 --2- 172.21.15.55:0/541892682 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8840b4e70 0x7fb8840ccff0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.355+0000 7fb88359e700 1 -- 172.21.15.55:0/541892682 >> 172.21.15.55:0/541892682 conn(0x7fb8840d10c0 msgr2=0x7fb8840affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.355+0000 7fb88359e700 1 -- 172.21.15.55:0/541892682 shutdown_connections 2023-10-06T04:48:51.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.355+0000 7fb88359e700 1 -- 172.21.15.55:0/541892682 wait complete. 2023-10-06T04:48:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.357+0000 7fb88359e700 1 Processor -- start 2023-10-06T04:48:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.357+0000 7fb88359e700 1 -- start start 2023-10-06T04:48:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb88359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8840b16d0 0x7fb8840af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:51.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb88359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8840b1b40 0x7fb8840c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb88359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8840b4e70 0x7fb8840c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb88359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840c80a0 con 0x7fb8840b16d0 2023-10-06T04:48:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb88359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840c6d00 con 0x7fb8840b1b40 2023-10-06T04:48:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb88359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840c7700 con 0x7fb8840b4e70 2023-10-06T04:48:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb88259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8840b16d0 0x7fb8840af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb88259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8840b16d0 0x7fb8840af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48552/0 (socket says 172.21.15.55:48552) 2023-10-06T04:48:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb88259c700 1 -- 172.21.15.55:0/619556393 learned_addr learned my addr 172.21.15.55:0/619556393 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb882d9d700 1 --2- 172.21.15.55:0/619556393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8840b4e70 0x7fb8840c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:51.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb881d9b700 1 --2- 172.21.15.55:0/619556393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8840b1b40 0x7fb8840c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:51.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb88259c700 1 -- 172.21.15.55:0/619556393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8840b4e70 msgr2=0x7fb8840c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:51.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb88259c700 1 --2- 172.21.15.55:0/619556393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8840b4e70 0x7fb8840c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:51.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb88259c700 1 -- 172.21.15.55:0/619556393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8840b1b40 msgr2=0x7fb8840c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:51.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb88259c700 1 --2- 172.21.15.55:0/619556393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8840b1b40 0x7fb8840c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:51.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.358+0000 7fb88259c700 1 -- 172.21.15.55:0/619556393 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8740094f0 con 0x7fb8840b16d0 2023-10-06T04:48:51.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.359+0000 7fb88259c700 1 --2- 172.21.15.55:0/619556393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8840b16d0 0x7fb8840af400 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fb87400c010 tx=0x7fb874016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:51.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.359+0000 7fb8727fc700 1 -- 172.21.15.55:0/619556393 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874011e10 con 0x7fb8840b16d0 2023-10-06T04:48:51.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.359+0000 7fb8727fc700 1 -- 172.21.15.55:0/619556393 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87401b680 con 0x7fb8840b16d0 2023-10-06T04:48:51.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.359+0000 7fb88359e700 1 -- 172.21.15.55:0/619556393 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8840c79e0 con 0x7fb8840b16d0 2023-10-06T04:48:51.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.359+0000 7fb8727fc700 1 -- 172.21.15.55:0/619556393 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87401ba20 con 0x7fb8840b16d0 2023-10-06T04:48:51.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.359+0000 7fb88359e700 1 -- 172.21.15.55:0/619556393 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8840af940 con 0x7fb8840b16d0 2023-10-06T04:48:51.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.360+0000 7fb88359e700 1 -- 172.21.15.55:0/619556393 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb884088990 con 0x7fb8840b16d0 2023-10-06T04:48:51.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.361+0000 7fb8727fc700 1 -- 172.21.15.55:0/619556393 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 68) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb87401c450 con 0x7fb8840b16d0 2023-10-06T04:48:51.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.362+0000 7fb8727fc700 1 --2- 172.21.15.55:0/619556393 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb864066f30 0x7fb8640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:51.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.362+0000 7fb8727fc700 1 -- 172.21.15.55:0/619556393 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fb874086580 con 0x7fb8840b16d0 2023-10-06T04:48:51.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.362+0000 7fb881d9b700 1 --2- 172.21.15.55:0/619556393 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb864066f30 0x7fb8640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:51.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.363+0000 7fb881d9b700 1 --2- 172.21.15.55:0/619556393 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb864066f30 0x7fb8640693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb88414c150 tx=0x7fb878009040 comp rx=0 tx=0).ready entity=mgr.7205 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:51.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.366+0000 7fb8727fc700 1 -- 172.21.15.55:0/619556393 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb874053460 con 0x7fb8840b16d0 2023-10-06T04:48:51.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:51.623+0000 7fb88359e700 1 -- 172.21.15.55:0/619556393 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fb88404bb80 con 0x7fb8840b16d0 2023-10-06T04:48:52.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.036+0000 7fb8727fc700 1 -- 172.21.15.55:0/619556393 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 69) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fb874053830 con 0x7fb8840b16d0 2023-10-06T04:48:52.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.037+0000 7fb8727fc700 1 -- 172.21.15.55:0/619556393 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v69) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb874056ef0 con 0x7fb8840b16d0 2023-10-06T04:48:52.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.041+0000 7fb88359e700 1 -- 172.21.15.55:0/619556393 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb864066f30 msgr2=0x7fb8640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:52.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.041+0000 7fb88359e700 1 --2- 172.21.15.55:0/619556393 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb864066f30 0x7fb8640693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb88414c150 tx=0x7fb878009040 comp rx=0 tx=0).stop 2023-10-06T04:48:52.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.041+0000 7fb88359e700 1 -- 172.21.15.55:0/619556393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8840b16d0 msgr2=0x7fb8840af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:52.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.041+0000 7fb88359e700 1 --2- 172.21.15.55:0/619556393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8840b16d0 0x7fb8840af400 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fb87400c010 tx=0x7fb874016000 comp rx=0 tx=0).stop 2023-10-06T04:48:52.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.041+0000 7fb88359e700 1 -- 172.21.15.55:0/619556393 shutdown_connections 2023-10-06T04:48:52.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.041+0000 7fb88359e700 1 --2- 172.21.15.55:0/619556393 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb864066f30 0x7fb8640693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.041+0000 7fb88359e700 1 --2- 172.21.15.55:0/619556393 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8840b4e70 0x7fb8840c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.041+0000 7fb88359e700 1 --2- 172.21.15.55:0/619556393 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb8840b16d0 0x7fb8840af400 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.041+0000 7fb88359e700 1 --2- 172.21.15.55:0/619556393 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb8840b1b40 0x7fb8840c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.041+0000 7fb88359e700 1 -- 172.21.15.55:0/619556393 >> 172.21.15.55:0/619556393 conn(0x7fb8840d10c0 msgr2=0x7fb8840ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.041+0000 7fb88359e700 1 -- 172.21.15.55:0/619556393 shutdown_connections 2023-10-06T04:48:52.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.041+0000 7fb88359e700 1 -- 172.21.15.55:0/619556393 wait complete. 2023-10-06T04:48:52.062 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T04:48:52.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.254+0000 7fb596f9d700 1 Processor -- start 2023-10-06T04:48:52.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.255+0000 7fb596f9d700 1 -- start start 2023-10-06T04:48:52.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.256+0000 7fb596f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5900cb1a0 0x7fb59015a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:52.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.256+0000 7fb596f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59006ebd0 con 0x7fb59015a8a0 2023-10-06T04:48:52.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.256+0000 7fb596f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59015ad20 con 0x7fb5900ae0d0 2023-10-06T04:48:52.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.256+0000 7fb596f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59015b2e0 con 0x7fb5900cb1a0 2023-10-06T04:48:52.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.256+0000 7fb59579a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb59015a8a0 0x7fb59006f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37200/0 (socket says 172.21.15.55:37200) 2023-10-06T04:48:52.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.256+0000 7fb59579a700 1 -- 172.21.15.55:0/3739163931 learned_addr learned my addr 172.21.15.55:0/3739163931 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:52.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.256+0000 7fb595f9b700 1 --2- 172.21.15.55:0/3739163931 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5900cb1a0 0x7fb59015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:52.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.257+0000 7fb594f99700 1 -- 172.21.15.55:0/3739163931 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397118541 0 0) 0x7fb59006ebd0 con 0x7fb59015a8a0 2023-10-06T04:48:52.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.257+0000 7fb594f99700 1 -- 172.21.15.55:0/3739163931 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb57c003570 con 0x7fb59015a8a0 2023-10-06T04:48:52.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.257+0000 7fb594f99700 1 -- 172.21.15.55:0/3739163931 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058696124 0 0) 0x7fb59015ad20 con 0x7fb5900ae0d0 2023-10-06T04:48:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.257+0000 7fb594f99700 1 -- 172.21.15.55:0/3739163931 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb59006ebd0 con 0x7fb5900ae0d0 2023-10-06T04:48:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.257+0000 7fb595f9b700 1 -- 172.21.15.55:0/3739163931 >> v1:172.21.15.163:6789/0 conn(0x7fb5900ae0d0 legacy=0x7fb5900c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.257+0000 7fb595f9b700 1 -- 172.21.15.55:0/3739163931 >> v1:172.21.15.55:6789/0 conn(0x7fb59015a8a0 legacy=0x7fb59006f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.257+0000 7fb595f9b700 1 -- 172.21.15.55:0/3739163931 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59015b000 con 0x7fb5900cb1a0 2023-10-06T04:48:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.257+0000 7fb595f9b700 1 --2- 172.21.15.55:0/3739163931 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5900cb1a0 0x7fb59015a360 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb58c009910 tx=0x7fb58c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0c182c21ddcdb2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.258+0000 7fb594f99700 1 -- 172.21.15.55:0/3739163931 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c00b690 con 0x7fb5900cb1a0 2023-10-06T04:48:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.258+0000 7fb594f99700 1 -- 172.21.15.55:0/3739163931 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c00b7f0 con 0x7fb5900cb1a0 2023-10-06T04:48:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.258+0000 7fb594f99700 1 -- 172.21.15.55:0/3739163931 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c00bb90 con 0x7fb5900cb1a0 2023-10-06T04:48:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.258+0000 7fb596f9d700 1 -- 172.21.15.55:0/3739163931 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5900cb1a0 msgr2=0x7fb59015a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.258+0000 7fb596f9d700 1 --2- 172.21.15.55:0/3739163931 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5900cb1a0 0x7fb59015a360 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb58c009910 tx=0x7fb58c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.258+0000 7fb596f9d700 1 -- 172.21.15.55:0/3739163931 shutdown_connections 2023-10-06T04:48:52.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.258+0000 7fb596f9d700 1 --2- 172.21.15.55:0/3739163931 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5900cb1a0 0x7fb59015a360 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.258+0000 7fb596f9d700 1 -- 172.21.15.55:0/3739163931 >> 172.21.15.55:0/3739163931 conn(0x7fb5900cc1e0 msgr2=0x7fb5900cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:52.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.258+0000 7fb596f9d700 1 -- 172.21.15.55:0/3739163931 shutdown_connections 2023-10-06T04:48:52.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.258+0000 7fb596f9d700 1 -- 172.21.15.55:0/3739163931 wait complete. 2023-10-06T04:48:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.260+0000 7fb596f9d700 1 Processor -- start 2023-10-06T04:48:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.260+0000 7fb596f9d700 1 -- start start 2023-10-06T04:48:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.261+0000 7fb596f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5900ae0d0 0x7fb590148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.261+0000 7fb596f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5900cb1a0 0x7fb590155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.261+0000 7fb596f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5901598d0 0x7fb590159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:52.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.261+0000 7fb596f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900c0f80 con 0x7fb5901598d0 2023-10-06T04:48:52.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.261+0000 7fb596f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900c11d0 con 0x7fb5900cb1a0 2023-10-06T04:48:52.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.261+0000 7fb596f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5900c14b0 con 0x7fb5900ae0d0 2023-10-06T04:48:52.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.261+0000 7fb59679c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5901598d0 0x7fb590159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:52.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.261+0000 7fb595f9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5900ae0d0 0x7fb590148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:52.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.261+0000 7fb59579a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5900cb1a0 0x7fb590155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:52.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.261+0000 7fb59679c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5901598d0 0x7fb590159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48572/0 (socket says 172.21.15.55:48572) 2023-10-06T04:48:52.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.261+0000 7fb59679c700 1 -- 172.21.15.55:0/1719878216 learned_addr learned my addr 172.21.15.55:0/1719878216 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:52.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.262+0000 7fb59679c700 1 -- 172.21.15.55:0/1719878216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5900ae0d0 msgr2=0x7fb590148930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:52.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.262+0000 7fb59679c700 1 --2- 172.21.15.55:0/1719878216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5900ae0d0 0x7fb590148930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.262+0000 7fb59679c700 1 -- 172.21.15.55:0/1719878216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5900cb1a0 msgr2=0x7fb590155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:52.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.262+0000 7fb59679c700 1 --2- 172.21.15.55:0/1719878216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5900cb1a0 0x7fb590155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.262+0000 7fb59679c700 1 -- 172.21.15.55:0/1719878216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c009420 con 0x7fb5901598d0 2023-10-06T04:48:52.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.262+0000 7fb59679c700 1 --2- 172.21.15.55:0/1719878216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5901598d0 0x7fb590159d30 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb58800c910 tx=0x7fb5880027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:52.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.262+0000 7fb585ffb700 1 -- 172.21.15.55:0/1719878216 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588012630 con 0x7fb5901598d0 2023-10-06T04:48:52.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.262+0000 7fb585ffb700 1 -- 172.21.15.55:0/1719878216 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5880127b0 con 0x7fb5901598d0 2023-10-06T04:48:52.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.262+0000 7fb596f9d700 1 -- 172.21.15.55:0/1719878216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5900c1790 con 0x7fb5901598d0 2023-10-06T04:48:52.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.262+0000 7fb585ffb700 1 -- 172.21.15.55:0/1719878216 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb588012b70 con 0x7fb5901598d0 2023-10-06T04:48:52.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.262+0000 7fb596f9d700 1 -- 172.21.15.55:0/1719878216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5900c1d50 con 0x7fb5901598d0 2023-10-06T04:48:52.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.264+0000 7fb596f9d700 1 -- 172.21.15.55:0/1719878216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59006c450 con 0x7fb5901598d0 2023-10-06T04:48:52.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.265+0000 7fb585ffb700 1 -- 172.21.15.55:0/1719878216 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 69) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fb58801b400 con 0x7fb5901598d0 2023-10-06T04:48:52.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.265+0000 7fb585ffb700 1 --2- 172.21.15.55:0/1719878216 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb574066f80 0x7fb574069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:52.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.266+0000 7fb585ffb700 1 -- 172.21.15.55:0/1719878216 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fb588084c30 con 0x7fb5901598d0 2023-10-06T04:48:52.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.266+0000 7fb595f9b700 1 -- 172.21.15.55:0/1719878216 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb574066f80 msgr2=0x7fb574069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:48:52.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.266+0000 7fb595f9b700 1 --2- 172.21.15.55:0/1719878216 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb574066f80 0x7fb574069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:52.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.270+0000 7fb585ffb700 1 -- 172.21.15.55:0/1719878216 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb588051b10 con 0x7fb5901598d0 2023-10-06T04:48:52.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.467+0000 7fb595f9b700 1 -- 172.21.15.55:0/1719878216 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb574066f80 msgr2=0x7fb574069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:48:52.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.467+0000 7fb595f9b700 1 --2- 172.21.15.55:0/1719878216 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb574066f80 0x7fb574069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:52.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.475+0000 7fb596f9d700 1 -- 172.21.15.55:0/1719878216 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fb5900c6500 con 0x7fb5901598d0 2023-10-06T04:48:52.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.481+0000 7fb585ffb700 1 -- 172.21.15.55:0/1719878216 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v53)=0 v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb588055130 con 0x7fb5901598d0 2023-10-06T04:48:52.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.483+0000 7fb596f9d700 1 -- 172.21.15.55:0/1719878216 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb574066f80 msgr2=0x7fb574069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:52.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.483+0000 7fb596f9d700 1 --2- 172.21.15.55:0/1719878216 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb574066f80 0x7fb574069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.483+0000 7fb596f9d700 1 -- 172.21.15.55:0/1719878216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5901598d0 msgr2=0x7fb590159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:52.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.483+0000 7fb596f9d700 1 --2- 172.21.15.55:0/1719878216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5901598d0 0x7fb590159d30 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fb58800c910 tx=0x7fb5880027a0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.483+0000 7fb596f9d700 1 -- 172.21.15.55:0/1719878216 shutdown_connections 2023-10-06T04:48:52.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.483+0000 7fb596f9d700 1 --2- 172.21.15.55:0/1719878216 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb574066f80 0x7fb574069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.483+0000 7fb596f9d700 1 --2- 172.21.15.55:0/1719878216 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5901598d0 0x7fb590159d30 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.483+0000 7fb596f9d700 1 --2- 172.21.15.55:0/1719878216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5900ae0d0 0x7fb590148930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.483+0000 7fb596f9d700 1 --2- 172.21.15.55:0/1719878216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5900cb1a0 0x7fb590155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.483+0000 7fb596f9d700 1 -- 172.21.15.55:0/1719878216 >> 172.21.15.55:0/1719878216 conn(0x7fb5900cc1e0 msgr2=0x7fb590155cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:52.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.483+0000 7fb596f9d700 1 -- 172.21.15.55:0/1719878216 shutdown_connections 2023-10-06T04:48:52.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.483+0000 7fb596f9d700 1 -- 172.21.15.55:0/1719878216 wait complete. 2023-10-06T04:48:52.503 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T04:48:52.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.697+0000 7fc27ef9d700 1 Processor -- start 2023-10-06T04:48:52.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.697+0000 7fc27ef9d700 1 -- start start 2023-10-06T04:48:52.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.698+0000 7fc27ef9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc2780b16d0 0x7fc2780b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:52.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.698+0000 7fc27ef9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc2780b33b0 0x7fc2780b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:52.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.698+0000 7fc27ef9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2780b2080 con 0x7fc2780b4e70 2023-10-06T04:48:52.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.698+0000 7fc27ef9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2780cfb00 con 0x7fc2780b33b0 2023-10-06T04:48:52.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.698+0000 7fc27ef9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2780b5c40 con 0x7fc2780b16d0 2023-10-06T04:48:52.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.699+0000 7fc27e79c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc2780b33b0 0x7fc2780b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:52.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.699+0000 7fc27e79c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc2780b33b0 0x7fc2780b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48586/0 (socket says 172.21.15.55:48586) 2023-10-06T04:48:52.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.699+0000 7fc27d79a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc2780b16d0 0x7fc2780b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:52.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.699+0000 7fc27e79c700 1 -- 172.21.15.55:0/1998899698 learned_addr learned my addr 172.21.15.55:0/1998899698 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:52.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.699+0000 7fc27e79c700 1 -- 172.21.15.55:0/1998899698 >> v2:172.21.15.163:3300/0 conn(0x7fc2780b16d0 msgr2=0x7fc2780b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:52.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.699+0000 7fc27e79c700 1 --2- 172.21.15.55:0/1998899698 >> v2:172.21.15.163:3300/0 conn(0x7fc2780b16d0 0x7fc2780b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.699+0000 7fc27e79c700 1 -- 172.21.15.55:0/1998899698 >> v1:172.21.15.163:6789/0 conn(0x7fc2780b4e70 legacy=0x7fc2780ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:52.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.699+0000 7fc27e79c700 1 -- 172.21.15.55:0/1998899698 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2780b5f20 con 0x7fc2780b33b0 2023-10-06T04:48:52.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.699+0000 7fc27e79c700 1 --2- 172.21.15.55:0/1998899698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2780b33b0 0x7fc2780b37d0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc278089830 tx=0x7fc270009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4542e5a94106b69 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:52.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.700+0000 7fc27cf99700 1 -- 172.21.15.55:0/1998899698 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27000e070 con 0x7fc2780b33b0 2023-10-06T04:48:52.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.700+0000 7fc27cf99700 1 -- 172.21.15.55:0/1998899698 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27000ba20 con 0x7fc2780b33b0 2023-10-06T04:48:52.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.700+0000 7fc27cf99700 1 -- 172.21.15.55:0/1998899698 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2700092e0 con 0x7fc2780b33b0 2023-10-06T04:48:52.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.700+0000 7fc27ef9d700 1 -- 172.21.15.55:0/1998899698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2780b33b0 msgr2=0x7fc2780b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:52.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.700+0000 7fc27ef9d700 1 --2- 172.21.15.55:0/1998899698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2780b33b0 0x7fc2780b37d0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc278089830 tx=0x7fc270009840 comp rx=0 tx=0).stop 2023-10-06T04:48:52.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.700+0000 7fc27ef9d700 1 -- 172.21.15.55:0/1998899698 shutdown_connections 2023-10-06T04:48:52.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.700+0000 7fc27ef9d700 1 --2- 172.21.15.55:0/1998899698 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2780b33b0 0x7fc2780b37d0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.700+0000 7fc27ef9d700 1 --2- 172.21.15.55:0/1998899698 >> v2:172.21.15.163:3300/0 conn(0x7fc2780b16d0 0x7fc2780b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.700+0000 7fc27ef9d700 1 -- 172.21.15.55:0/1998899698 >> 172.21.15.55:0/1998899698 conn(0x7fc2780d10c0 msgr2=0x7fc2780affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:52.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.701+0000 7fc27ef9d700 1 -- 172.21.15.55:0/1998899698 shutdown_connections 2023-10-06T04:48:52.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.701+0000 7fc27ef9d700 1 -- 172.21.15.55:0/1998899698 wait complete. 2023-10-06T04:48:52.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.702+0000 7fc27ef9d700 1 Processor -- start 2023-10-06T04:48:52.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.702+0000 7fc27ef9d700 1 -- start start 2023-10-06T04:48:52.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.703+0000 7fc27ef9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2780b16d0 0x7fc278145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:52.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.703+0000 7fc27ef9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2780b33b0 0x7fc27815e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:52.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.703+0000 7fc27ef9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2780b4e70 0x7fc27815e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:52.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.703+0000 7fc27ef9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc278148d10 con 0x7fc2780b16d0 2023-10-06T04:48:52.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.703+0000 7fc27ef9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc278148100 con 0x7fc2780b4e70 2023-10-06T04:48:52.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.703+0000 7fc27ef9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781483e0 con 0x7fc2780b33b0 2023-10-06T04:48:52.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.704+0000 7fc27df9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2780b16d0 0x7fc278145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:52.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.704+0000 7fc27df9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2780b16d0 0x7fc278145cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48590/0 (socket says 172.21.15.55:48590) 2023-10-06T04:48:52.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.704+0000 7fc27df9b700 1 -- 172.21.15.55:0/3837105312 learned_addr learned my addr 172.21.15.55:0/3837105312 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:52.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.704+0000 7fc27e79c700 1 --2- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2780b4e70 0x7fc27815e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:52.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.704+0000 7fc27d79a700 1 --2- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2780b33b0 0x7fc27815e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:52.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.704+0000 7fc27df9b700 1 -- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2780b33b0 msgr2=0x7fc27815e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:52.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.704+0000 7fc27df9b700 1 --2- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2780b33b0 0x7fc27815e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.704+0000 7fc27df9b700 1 -- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2780b4e70 msgr2=0x7fc27815e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:52.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.704+0000 7fc27df9b700 1 --2- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2780b4e70 0x7fc27815e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.704+0000 7fc27df9b700 1 -- 172.21.15.55:0/3837105312 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc270009ae0 con 0x7fc2780b16d0 2023-10-06T04:48:52.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.704+0000 7fc27df9b700 1 --2- 172.21.15.55:0/3837105312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2780b16d0 0x7fc278145cb0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fc27400c590 tx=0x7fc27400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:52.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.706+0000 7fc26dffb700 1 -- 172.21.15.55:0/3837105312 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc274011590 con 0x7fc2780b16d0 2023-10-06T04:48:52.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.706+0000 7fc26dffb700 1 -- 172.21.15.55:0/3837105312 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc274011710 con 0x7fc2780b16d0 2023-10-06T04:48:52.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.706+0000 7fc26dffb700 1 -- 172.21.15.55:0/3837105312 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc274011ad0 con 0x7fc2780b16d0 2023-10-06T04:48:52.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.706+0000 7fc27ef9d700 1 -- 172.21.15.55:0/3837105312 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2781486c0 con 0x7fc2780b16d0 2023-10-06T04:48:52.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.706+0000 7fc27ef9d700 1 -- 172.21.15.55:0/3837105312 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2781461f0 con 0x7fc2780b16d0 2023-10-06T04:48:52.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.707+0000 7fc27ef9d700 1 -- 172.21.15.55:0/3837105312 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc278088990 con 0x7fc2780b16d0 2023-10-06T04:48:52.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.708+0000 7fc26dffb700 1 -- 172.21.15.55:0/3837105312 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 69) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fc27401b3b0 con 0x7fc2780b16d0 2023-10-06T04:48:52.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.709+0000 7fc26dffb700 1 --2- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fc25c066f80 0x7fc25c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:52.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.709+0000 7fc26dffb700 1 -- 172.21.15.55:0/3837105312 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fc274083d90 con 0x7fc2780b16d0 2023-10-06T04:48:52.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.709+0000 7fc27d79a700 1 -- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fc25c066f80 msgr2=0x7fc25c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:48:52.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.709+0000 7fc27d79a700 1 --2- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fc25c066f80 0x7fc25c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:52.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.713+0000 7fc26dffb700 1 -- 172.21.15.55:0/3837105312 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc274050c70 con 0x7fc2780b16d0 2023-10-06T04:48:52.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.910+0000 7fc27d79a700 1 -- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fc25c066f80 msgr2=0x7fc25c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:48:52.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.910+0000 7fc27d79a700 1 --2- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fc25c066f80 0x7fc25c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:52.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.920+0000 7fc27ef9d700 1 -- 172.21.15.55:0/3837105312 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fc2780cf330 con 0x7fc2780b16d0 2023-10-06T04:48:52.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.929+0000 7fc26dffb700 1 -- 172.21.15.55:0/3837105312 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v54)=0 v54) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc274017070 con 0x7fc2780b16d0 2023-10-06T04:48:52.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.931+0000 7fc27ef9d700 1 -- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fc25c066f80 msgr2=0x7fc25c069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:52.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.931+0000 7fc27ef9d700 1 --2- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fc25c066f80 0x7fc25c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.931+0000 7fc27ef9d700 1 -- 172.21.15.55:0/3837105312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2780b16d0 msgr2=0x7fc278145cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:52.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.931+0000 7fc27ef9d700 1 --2- 172.21.15.55:0/3837105312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2780b16d0 0x7fc278145cb0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fc27400c590 tx=0x7fc27400c8a0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.931+0000 7fc27ef9d700 1 -- 172.21.15.55:0/3837105312 shutdown_connections 2023-10-06T04:48:52.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.931+0000 7fc27ef9d700 1 --2- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fc25c066f80 0x7fc25c069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.931+0000 7fc27ef9d700 1 --2- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc2780b4e70 0x7fc27815e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.931+0000 7fc27ef9d700 1 --2- 172.21.15.55:0/3837105312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc2780b16d0 0x7fc278145cb0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.931+0000 7fc27ef9d700 1 --2- 172.21.15.55:0/3837105312 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc2780b33b0 0x7fc27815e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:52.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.931+0000 7fc27ef9d700 1 -- 172.21.15.55:0/3837105312 >> 172.21.15.55:0/3837105312 conn(0x7fc2780d10c0 msgr2=0x7fc27815c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:52.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.931+0000 7fc27ef9d700 1 -- 172.21.15.55:0/3837105312 shutdown_connections 2023-10-06T04:48:52.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:52.931+0000 7fc27ef9d700 1 -- 172.21.15.55:0/3837105312 wait complete. 2023-10-06T04:48:52.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:48:52.952 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T04:48:52.993 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1h 2023-10-06T04:48:53.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.226+0000 7f6bde39d700 1 Processor -- start 2023-10-06T04:48:53.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.227+0000 7f6bde39d700 1 -- start start 2023-10-06T04:48:53.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.228+0000 7f6bde39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6bd80b0790 0x7f6bd80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:53.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.228+0000 7f6bde39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bd81591e0 0x7f6bd8159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:53.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.228+0000 7f6bde39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd80cfa00 con 0x7f6bd80b03c0 2023-10-06T04:48:53.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.228+0000 7f6bde39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd80cb580 con 0x7f6bd80b0790 2023-10-06T04:48:53.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.228+0000 7f6bde39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd80cf160 con 0x7f6bd81591e0 2023-10-06T04:48:53.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.228+0000 7f6bdd39b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6bd80b03c0 0x7f6bd80b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:43996/0 (socket says 172.21.15.55:43996) 2023-10-06T04:48:53.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.228+0000 7f6bdd39b700 1 -- 172.21.15.55:0/3535121993 learned_addr learned my addr 172.21.15.55:0/3535121993 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:53.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.228+0000 7f6bddb9c700 1 --2- 172.21.15.55:0/3535121993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bd81591e0 0x7f6bd8159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:53.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.228+0000 7f6bdcb9a700 1 --2- 172.21.15.55:0/3535121993 >> v2:172.21.15.163:3300/0 conn(0x7f6bd80b0790 0x7f6bd80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:53.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.229+0000 7f6bcffff700 1 -- 172.21.15.55:0/3535121993 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4095572676 0 0) 0x7f6bd80cfa00 con 0x7f6bd80b03c0 2023-10-06T04:48:53.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.229+0000 7f6bcffff700 1 -- 172.21.15.55:0/3535121993 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bc4003570 con 0x7f6bd80b03c0 2023-10-06T04:48:53.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.229+0000 7f6bddb9c700 1 -- 172.21.15.55:0/3535121993 >> v2:172.21.15.163:3300/0 conn(0x7f6bd80b0790 msgr2=0x7f6bd80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:53.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.229+0000 7f6bddb9c700 1 --2- 172.21.15.55:0/3535121993 >> v2:172.21.15.163:3300/0 conn(0x7f6bd80b0790 0x7f6bd80cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:53.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.229+0000 7f6bddb9c700 1 -- 172.21.15.55:0/3535121993 >> v1:172.21.15.55:6789/0 conn(0x7f6bd80b03c0 legacy=0x7f6bd80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:53.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.229+0000 7f6bddb9c700 1 -- 172.21.15.55:0/3535121993 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd80cf440 con 0x7f6bd81591e0 2023-10-06T04:48:53.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.229+0000 7f6bddb9c700 1 --2- 172.21.15.55:0/3535121993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bd81591e0 0x7f6bd8159620 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009e90 tx=0x7f6bd0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26555ed444348fe6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:53.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.230+0000 7f6bcffff700 1 -- 172.21.15.55:0/3535121993 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd000e070 con 0x7f6bd81591e0 2023-10-06T04:48:53.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.230+0000 7f6bcffff700 1 -- 172.21.15.55:0/3535121993 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd000bb40 con 0x7f6bd81591e0 2023-10-06T04:48:53.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.230+0000 7f6bcffff700 1 -- 172.21.15.55:0/3535121993 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0012430 con 0x7f6bd81591e0 2023-10-06T04:48:53.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.230+0000 7f6bde39d700 1 -- 172.21.15.55:0/3535121993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bd81591e0 msgr2=0x7f6bd8159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:53.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.230+0000 7f6bde39d700 1 --2- 172.21.15.55:0/3535121993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bd81591e0 0x7f6bd8159620 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0009e90 tx=0x7f6bd0009780 comp rx=0 tx=0).stop 2023-10-06T04:48:53.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.231+0000 7f6bde39d700 1 -- 172.21.15.55:0/3535121993 shutdown_connections 2023-10-06T04:48:53.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.231+0000 7f6bde39d700 1 --2- 172.21.15.55:0/3535121993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bd81591e0 0x7f6bd8159620 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:53.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.231+0000 7f6bde39d700 1 --2- 172.21.15.55:0/3535121993 >> v2:172.21.15.163:3300/0 conn(0x7f6bd80b0790 0x7f6bd80cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:53.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.231+0000 7f6bde39d700 1 -- 172.21.15.55:0/3535121993 >> 172.21.15.55:0/3535121993 conn(0x7f6bd8063230 msgr2=0x7f6bd8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:53.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.231+0000 7f6bde39d700 1 -- 172.21.15.55:0/3535121993 shutdown_connections 2023-10-06T04:48:53.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.231+0000 7f6bde39d700 1 -- 172.21.15.55:0/3535121993 wait complete. 2023-10-06T04:48:53.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.233+0000 7f6bde39d700 1 Processor -- start 2023-10-06T04:48:53.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.233+0000 7f6bde39d700 1 -- start start 2023-10-06T04:48:53.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.235+0000 7f6bde39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bd80b03c0 0x7f6bd80c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:53.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.235+0000 7f6bde39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bd80b0790 0x7f6bd80c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:53.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.235+0000 7f6bde39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bd81591e0 0x7f6bd816c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:53.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.235+0000 7f6bde39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd816acf0 con 0x7f6bd80b03c0 2023-10-06T04:48:53.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.235+0000 7f6bde39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd80ca4d0 con 0x7f6bd80b0790 2023-10-06T04:48:53.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.235+0000 7f6bde39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd80c8b20 con 0x7f6bd81591e0 2023-10-06T04:48:53.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.235+0000 7f6bdd39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bd80b03c0 0x7f6bd80c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:53.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.235+0000 7f6bdd39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bd80b03c0 0x7f6bd80c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41728/0 (socket says 172.21.15.55:41728) 2023-10-06T04:48:53.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.235+0000 7f6bdd39b700 1 -- 172.21.15.55:0/4142513991 learned_addr learned my addr 172.21.15.55:0/4142513991 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:53.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.235+0000 7f6bddb9c700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bd81591e0 0x7f6bd816c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:53.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.235+0000 7f6bdcb9a700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bd80b0790 0x7f6bd80c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:53.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.236+0000 7f6bdd39b700 1 -- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bd81591e0 msgr2=0x7f6bd816c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:53.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.236+0000 7f6bdd39b700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bd81591e0 0x7f6bd816c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:53.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.236+0000 7f6bdd39b700 1 -- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bd80b0790 msgr2=0x7f6bd80c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:53.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.236+0000 7f6bdd39b700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bd80b0790 0x7f6bd80c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:53.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.236+0000 7f6bdd39b700 1 -- 172.21.15.55:0/4142513991 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd0009a20 con 0x7f6bd80b03c0 2023-10-06T04:48:53.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.236+0000 7f6bdd39b700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bd80b03c0 0x7f6bd80c7f60 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400c7c0 tx=0x7f6bd4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:53.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.236+0000 7f6bcd7fa700 1 -- 172.21.15.55:0/4142513991 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4005cd0 con 0x7f6bd80b03c0 2023-10-06T04:48:53.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.236+0000 7f6bcd7fa700 1 -- 172.21.15.55:0/4142513991 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd4005e30 con 0x7f6bd80b03c0 2023-10-06T04:48:53.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.236+0000 7f6bde39d700 1 -- 172.21.15.55:0/4142513991 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd80c8e00 con 0x7f6bd80b03c0 2023-10-06T04:48:53.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.236+0000 7f6bcd7fa700 1 -- 172.21.15.55:0/4142513991 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4007320 con 0x7f6bd80b03c0 2023-10-06T04:48:53.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.236+0000 7f6bde39d700 1 -- 172.21.15.55:0/4142513991 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd816a980 con 0x7f6bd80b03c0 2023-10-06T04:48:53.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.237+0000 7f6bde39d700 1 -- 172.21.15.55:0/4142513991 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd804c3c0 con 0x7f6bd80b03c0 2023-10-06T04:48:53.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.238+0000 7f6bcd7fa700 1 -- 172.21.15.55:0/4142513991 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 69) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f6bd401b3b0 con 0x7f6bd80b03c0 2023-10-06T04:48:53.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.239+0000 7f6bcd7fa700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 0x7f6bbc069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:53.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.239+0000 7f6bcd7fa700 1 -- 172.21.15.55:0/4142513991 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f6bd4083fd0 con 0x7f6bd80b03c0 2023-10-06T04:48:53.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.239+0000 7f6bdcb9a700 1 -- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 msgr2=0x7f6bbc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:48:53.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.239+0000 7f6bdcb9a700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 0x7f6bbc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:48:53.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.243+0000 7f6bcd7fa700 1 -- 172.21.15.55:0/4142513991 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6bd4050eb0 con 0x7f6bd80b03c0 2023-10-06T04:48:53.333 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:53.332+0000 7f137afaa180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T04:48:53.412 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:53.410+0000 7f57e1b40180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T04:48:53.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.440+0000 7f6bdcb9a700 1 -- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 msgr2=0x7f6bbc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:48:53.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.440+0000 7f6bdcb9a700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 0x7f6bbc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:48:53.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.492+0000 7f6bde39d700 1 -- 172.21.15.55:0/4142513991 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bd816ae30 con 0x7f6bbc066f80 2023-10-06T04:48:53.497 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:53.496+0000 7f137afaa180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T04:48:53.583 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:53.581+0000 7f57e1b40180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T04:48:53.584 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:53.582+0000 7f137afaa180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T04:48:53.671 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:53.669+0000 7f57e1b40180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T04:48:53.809 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:53.808+0000 7f137afaa180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:48:53.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.841+0000 7f6bdcb9a700 1 -- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 msgr2=0x7f6bbc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:48:53.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:53.841+0000 7f6bdcb9a700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 0x7f6bbc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T04:48:53.903 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:53.902+0000 7f57e1b40180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:48:53.988 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:53.986+0000 7f137afaa180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T04:48:54.080 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:54.078+0000 7f137afaa180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T04:48:54.087 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:54.085+0000 7f57e1b40180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T04:48:54.182 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:54.180+0000 7f57e1b40180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T04:48:54.242 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:54.241+0000 7f137afaa180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T04:48:54.343 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:54.342+0000 7f137afaa180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T04:48:54.349 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:54.347+0000 7f57e1b40180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T04:48:54.454 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:54.453+0000 7f57e1b40180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T04:48:54.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:54.642+0000 7f6bdcb9a700 1 -- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 msgr2=0x7f6bbc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:48:54.644 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:54.642+0000 7f6bdcb9a700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 0x7f6bbc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T04:48:54.831 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:54.830+0000 7f137afaa180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T04:48:54.926 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:54.925+0000 7f137afaa180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T04:48:54.955 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:54.953+0000 7f57e1b40180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T04:48:55.017 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:55.016+0000 7f137afaa180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T04:48:55.052 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:55.050+0000 7f57e1b40180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T04:48:55.146 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:55.144+0000 7f57e1b40180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T04:48:55.639 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:55.637+0000 7f137afaa180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T04:48:55.791 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:55.789+0000 7f57e1b40180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T04:48:55.812 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:55.811+0000 7f137afaa180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:48:55.909 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:55.907+0000 7f137afaa180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T04:48:55.969 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:55.967+0000 7f57e1b40180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:48:56.068 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:56.067+0000 7f57e1b40180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T04:48:56.203 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:56.202+0000 7f137afaa180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T04:48:56.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:56.244+0000 7f6bdcb9a700 1 -- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 msgr2=0x7f6bbc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:48:56.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:56.244+0000 7f6bdcb9a700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 0x7f6bbc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T04:48:56.300 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:56.299+0000 7f137afaa180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T04:48:56.371 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:56.369+0000 7f57e1b40180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T04:48:56.470 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:56.469+0000 7f57e1b40180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T04:48:57.009 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:57.008+0000 7f137afaa180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T04:48:57.207 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:57.205+0000 7f57e1b40180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T04:48:57.298 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:57.297+0000 7f137afaa180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T04:48:57.391 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:57.390+0000 7f137afaa180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T04:48:57.499 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:57.498+0000 7f57e1b40180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T04:48:57.547 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:57.545+0000 7f137afaa180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T04:48:57.597 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:57.594+0000 7f57e1b40180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T04:48:57.652 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:57.651+0000 7f137afaa180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T04:48:57.757 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:57.755+0000 7f57e1b40180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T04:48:57.846 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:57.845+0000 7f137afaa180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T04:48:57.867 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:57.865+0000 7f57e1b40180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T04:48:57.939 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:57.938+0000 7f137afaa180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T04:48:58.032 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:58.031+0000 7f137afaa180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T04:48:58.064 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:58.062+0000 7f57e1b40180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T04:48:58.127 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:58.125+0000 7f137afaa180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T04:48:58.144 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:58.143+0000 7f6bcd7fa700 1 -- 172.21.15.55:0/4142513991 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 70) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f6bd4051280 con 0x7f6bd80b03c0 2023-10-06T04:48:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:58.143+0000 7f6bcd7fa700 1 -- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 msgr2=0x7f6bbc069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:48:58.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:58.143+0000 7f6bcd7fa700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 0x7f6bbc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:58.165 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:58.163+0000 7f57e1b40180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T04:48:58.269 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:58.266+0000 7f12ecb30700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:58.269 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:58.266+0000 7f12ecb30700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:58.269 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:58.266+0000 7f12ecb30700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:58.269 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:58.266+0000 7f12ecb30700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:58.269 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:58.266+0000 7f12ecb30700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:58.269 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:58.267+0000 7f12f0337700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:58.269 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:58.267+0000 7f12f0337700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:58.269 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:58.267+0000 7f12f0337700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:58.269 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:58.267+0000 7f12f0337700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:58.270 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:48:58.267+0000 7f12f0337700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:48:58.270 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:58.267+0000 7f57e1b40180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T04:48:58.367 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:48:58.365+0000 7f57e1b40180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T04:48:59.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.144+0000 7f6bcd7fa700 1 -- 172.21.15.55:0/4142513991 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mgrmap(e 71) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f6bd4054060 con 0x7f6bd80b03c0 2023-10-06T04:48:59.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.144+0000 7f6bcd7fa700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 0x7f6bbc0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:59.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.144+0000 7f6bcd7fa700 1 -- 172.21.15.55:0/4142513991 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bbc00e8c0 con 0x7f6bbc066f80 2023-10-06T04:48:59.146 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.145+0000 7f6bdcb9a700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 0x7f6bbc0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:59.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.156+0000 7f6bdcb9a700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 0x7f6bbc0693b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80caae0 tx=0x7f6bc80091a0 comp rx=0 tx=0).ready entity=mgr.7472 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:59.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.222+0000 7f6bcd7fa700 1 -- 172.21.15.55:0/4142513991 <== mgr.7472 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6bbc00e8c0 con 0x7f6bbc066f80 2023-10-06T04:48:59.223 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path snap_test_dir1 2023-10-06T04:48:59.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.224+0000 7f6bde39d700 1 -- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 msgr2=0x7f6bbc0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:59.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.224+0000 7f6bde39d700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 0x7f6bbc0693b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80caae0 tx=0x7f6bc80091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.224+0000 7f6bde39d700 1 -- 172.21.15.55:0/4142513991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bd80b03c0 msgr2=0x7f6bd80c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:59.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.224+0000 7f6bde39d700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bd80b03c0 0x7f6bd80c7f60 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400c7c0 tx=0x7f6bd4007e00 comp rx=0 tx=0).stop 2023-10-06T04:48:59.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.224+0000 7f6bde39d700 1 -- 172.21.15.55:0/4142513991 shutdown_connections 2023-10-06T04:48:59.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.224+0000 7f6bde39d700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6bbc066f80 0x7f6bbc0693b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.224+0000 7f6bde39d700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6bd81591e0 0x7f6bd816c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.224+0000 7f6bde39d700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6bd80b03c0 0x7f6bd80c7f60 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.224+0000 7f6bde39d700 1 --2- 172.21.15.55:0/4142513991 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6bd80b0790 0x7f6bd80c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.224+0000 7f6bde39d700 1 -- 172.21.15.55:0/4142513991 >> 172.21.15.55:0/4142513991 conn(0x7f6bd8063230 msgr2=0x7f6bd80b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:59.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.224+0000 7f6bde39d700 1 -- 172.21.15.55:0/4142513991 shutdown_connections 2023-10-06T04:48:59.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.224+0000 7f6bde39d700 1 -- 172.21.15.55:0/4142513991 wait complete. 2023-10-06T04:48:59.245 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-06T04:48:59.245 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-06T04:48:59.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.437+0000 7f7a4ad9d700 1 Processor -- start 2023-10-06T04:48:59.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.437+0000 7f7a4ad9d700 1 -- start start 2023-10-06T04:48:59.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.438+0000 7f7a4ad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7a440b03c0 0x7f7a440cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:59.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.438+0000 7f7a4ad9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7a440b0790 0x7f7a440b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:59.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.438+0000 7f7a4ad9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a440cfa00 con 0x7f7a441591e0 2023-10-06T04:48:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.438+0000 7f7a4ad9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a440cb580 con 0x7f7a440b03c0 2023-10-06T04:48:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.438+0000 7f7a4ad9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a440b2de0 con 0x7f7a440b0790 2023-10-06T04:48:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.438+0000 7f7a49d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7a440b03c0 0x7f7a440cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.439+0000 7f7a4a59c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7a441591e0 0x7f7a44159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44010/0 (socket says 172.21.15.55:44010) 2023-10-06T04:48:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.439+0000 7f7a4a59c700 1 -- 172.21.15.55:0/1576090013 learned_addr learned my addr 172.21.15.55:0/1576090013 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:48:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.439+0000 7f7a49d9b700 1 --2- 172.21.15.55:0/1576090013 >> v2:172.21.15.55:3300/0 conn(0x7f7a440b03c0 0x7f7a440cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41738/0 (socket says 172.21.15.55:41738) 2023-10-06T04:48:59.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.439+0000 7f7a4959a700 1 --2- 172.21.15.55:0/1576090013 >> v2:172.21.15.163:3300/0 conn(0x7f7a440b0790 0x7f7a440b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:59.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.439+0000 7f7a48d99700 1 -- 172.21.15.55:0/1576090013 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211507219 0 0) 0x7f7a440cfa00 con 0x7f7a441591e0 2023-10-06T04:48:59.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.439+0000 7f7a49d9b700 1 -- 172.21.15.55:0/1576090013 >> v2:172.21.15.163:3300/0 conn(0x7f7a440b0790 msgr2=0x7f7a440b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:48:59.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.439+0000 7f7a49d9b700 1 --2- 172.21.15.55:0/1576090013 >> v2:172.21.15.163:3300/0 conn(0x7f7a440b0790 0x7f7a440b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.439+0000 7f7a49d9b700 1 -- 172.21.15.55:0/1576090013 >> v1:172.21.15.55:6789/0 conn(0x7f7a441591e0 legacy=0x7f7a44159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:59.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.439+0000 7f7a49d9b700 1 -- 172.21.15.55:0/1576090013 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a440cf160 con 0x7f7a440b03c0 2023-10-06T04:48:59.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.439+0000 7f7a49d9b700 1 --2- 172.21.15.55:0/1576090013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a440b03c0 0x7f7a440cea70 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f7a400098e0 tx=0x7f7a400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f80f024b63a7de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:59.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.440+0000 7f7a48d99700 1 -- 172.21.15.55:0/1576090013 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40009e80 con 0x7f7a440b03c0 2023-10-06T04:48:59.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.440+0000 7f7a48d99700 1 -- 172.21.15.55:0/1576090013 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4000b820 con 0x7f7a440b03c0 2023-10-06T04:48:59.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.440+0000 7f7a48d99700 1 -- 172.21.15.55:0/1576090013 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000bba0 con 0x7f7a440b03c0 2023-10-06T04:48:59.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.440+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/1576090013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a440b03c0 msgr2=0x7f7a440cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:59.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.440+0000 7f7a4ad9d700 1 --2- 172.21.15.55:0/1576090013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a440b03c0 0x7f7a440cea70 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f7a400098e0 tx=0x7f7a400091a0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.440+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/1576090013 shutdown_connections 2023-10-06T04:48:59.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.440+0000 7f7a4ad9d700 1 --2- 172.21.15.55:0/1576090013 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a440b03c0 0x7f7a440cea70 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.440+0000 7f7a4ad9d700 1 --2- 172.21.15.55:0/1576090013 >> v2:172.21.15.163:3300/0 conn(0x7f7a440b0790 0x7f7a440b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.440+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/1576090013 >> 172.21.15.55:0/1576090013 conn(0x7f7a44063230 msgr2=0x7f7a44063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:59.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.440+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/1576090013 shutdown_connections 2023-10-06T04:48:59.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.440+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/1576090013 wait complete. 2023-10-06T04:48:59.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.442+0000 7f7a4ad9d700 1 Processor -- start 2023-10-06T04:48:59.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.442+0000 7f7a4ad9d700 1 -- start start 2023-10-06T04:48:59.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a4ad9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a440b03c0 0x7f7a441575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:59.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a4ad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a440b0790 0x7f7a440c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:59.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a4ad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a441591e0 0x7f7a441585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:59.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a4ad9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440c8350 con 0x7f7a441591e0 2023-10-06T04:48:59.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a4ad9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440c7740 con 0x7f7a440b0790 2023-10-06T04:48:59.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a4ad9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a440c7a20 con 0x7f7a440b03c0 2023-10-06T04:48:59.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a4a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a441591e0 0x7f7a441585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:59.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a4a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a441591e0 0x7f7a441585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41750/0 (socket says 172.21.15.55:41750) 2023-10-06T04:48:59.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a4a59c700 1 -- 172.21.15.55:0/3275311763 learned_addr learned my addr 172.21.15.55:0/3275311763 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:59.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a49d9b700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a440b03c0 0x7f7a441575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:59.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a4959a700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a440b0790 0x7f7a440c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:59.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a4a59c700 1 -- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a440b03c0 msgr2=0x7f7a441575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:59.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a4a59c700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a440b03c0 0x7f7a441575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a4a59c700 1 -- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a440b0790 msgr2=0x7f7a440c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:59.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.443+0000 7f7a4a59c700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a440b0790 0x7f7a440c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.444+0000 7f7a4a59c700 1 -- 172.21.15.55:0/3275311763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a40009420 con 0x7f7a441591e0 2023-10-06T04:48:59.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.444+0000 7f7a4959a700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a440b0790 0x7f7a440c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:48:59.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.444+0000 7f7a4a59c700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a441591e0 0x7f7a441585c0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00cbf0 tx=0x7f7a3c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:59.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.444+0000 7f7a39ffb700 1 -- 172.21.15.55:0/3275311763 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c00a9c0 con 0x7f7a441591e0 2023-10-06T04:48:59.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.444+0000 7f7a39ffb700 1 -- 172.21.15.55:0/3275311763 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c00ab20 con 0x7f7a441591e0 2023-10-06T04:48:59.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.444+0000 7f7a39ffb700 1 -- 172.21.15.55:0/3275311763 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c0070f0 con 0x7f7a441591e0 2023-10-06T04:48:59.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.444+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/3275311763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a440c7d00 con 0x7f7a441591e0 2023-10-06T04:48:59.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.444+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/3275311763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a44157af0 con 0x7f7a441591e0 2023-10-06T04:48:59.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.445+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/3275311763 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a440c8500 con 0x7f7a441591e0 2023-10-06T04:48:59.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.446+0000 7f7a39ffb700 1 -- 172.21.15.55:0/3275311763 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 71) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f7a3c007520 con 0x7f7a441591e0 2023-10-06T04:48:59.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.447+0000 7f7a39ffb700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a28066dd0 0x7f7a28069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:59.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.447+0000 7f7a39ffb700 1 -- 172.21.15.55:0/3275311763 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f7a3c084b00 con 0x7f7a441591e0 2023-10-06T04:48:59.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.447+0000 7f7a49d9b700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a28066dd0 0x7f7a28069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:59.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.448+0000 7f7a49d9b700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a28066dd0 0x7f7a28069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7a400093f0 tx=0x7f7a40017000 comp rx=0 tx=0).ready entity=mgr.7472 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:59.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.451+0000 7f7a39ffb700 1 -- 172.21.15.55:0/3275311763 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a440c8500 con 0x7f7a441591e0 2023-10-06T04:48:59.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.681+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/3275311763 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a4404c3c0 con 0x7f7a28066dd0 2023-10-06T04:48:59.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.750+0000 7f7a39ffb700 1 -- 172.21.15.55:0/3275311763 <== mgr.7472 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f7a4404c3c0 con 0x7f7a28066dd0 2023-10-06T04:48:59.751 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path snap_test_dir1 2023-10-06T04:48:59.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.752+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a28066dd0 msgr2=0x7f7a28069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:59.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.752+0000 7f7a4ad9d700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a28066dd0 0x7f7a28069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7a400093f0 tx=0x7f7a40017000 comp rx=0 tx=0).stop 2023-10-06T04:48:59.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.752+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/3275311763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a441591e0 msgr2=0x7f7a441585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:59.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.752+0000 7f7a4ad9d700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a441591e0 0x7f7a441585c0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00cbf0 tx=0x7f7a3c0019b0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.752+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/3275311763 shutdown_connections 2023-10-06T04:48:59.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.752+0000 7f7a4ad9d700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a28066dd0 0x7f7a28069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.752+0000 7f7a4ad9d700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a441591e0 0x7f7a441585c0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.752+0000 7f7a4ad9d700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a440b03c0 0x7f7a441575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.752+0000 7f7a4ad9d700 1 --2- 172.21.15.55:0/3275311763 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a440b0790 0x7f7a440c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.752+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/3275311763 >> 172.21.15.55:0/3275311763 conn(0x7f7a44063230 msgr2=0x7f7a440d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:59.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.752+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/3275311763 shutdown_connections 2023-10-06T04:48:59.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.752+0000 7f7a4ad9d700 1 -- 172.21.15.55:0/3275311763 wait complete. 2023-10-06T04:48:59.772 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-06T04:48:59.772 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-10-06T04:48:59.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.961+0000 7f26a959d700 1 Processor -- start 2023-10-06T04:48:59.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.962+0000 7f26a959d700 1 -- start start 2023-10-06T04:48:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.963+0000 7f26a959d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26a415d3f0 0x7f26a41538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.963+0000 7f26a959d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f26a40689b0 0x7f26a4153e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.963+0000 7f26a959d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a40c9480 con 0x7f26a40685e0 2023-10-06T04:48:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.963+0000 7f26a959d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a415ae00 con 0x7f26a415d3f0 2023-10-06T04:48:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.963+0000 7f26a959d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a415d880 con 0x7f26a40689b0 2023-10-06T04:48:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.963+0000 7f26a37fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26a415d3f0 0x7f26a41538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.963+0000 7f26a37fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26a415d3f0 0x7f26a41538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41764/0 (socket says 172.21.15.55:41764) 2023-10-06T04:48:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.963+0000 7f26a37fe700 1 -- 172.21.15.55:0/7550488 learned_addr learned my addr 172.21.15.55:0/7550488 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:59.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.964+0000 7f26a37fe700 1 -- 172.21.15.55:0/7550488 >> v2:172.21.15.163:3300/0 conn(0x7f26a40689b0 msgr2=0x7f26a4153e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T04:48:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.964+0000 7f26a37fe700 1 --2- 172.21.15.55:0/7550488 >> v2:172.21.15.163:3300/0 conn(0x7f26a40689b0 0x7f26a4153e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.964+0000 7f26a37fe700 1 -- 172.21.15.55:0/7550488 >> v1:172.21.15.163:6789/0 conn(0x7f26a40685e0 legacy=0x7f26a415a6f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T04:48:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.964+0000 7f26a37fe700 1 -- 172.21.15.55:0/7550488 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a415db60 con 0x7f26a415d3f0 2023-10-06T04:48:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.964+0000 7f26a37fe700 1 --2- 172.21.15.55:0/7550488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a415d3f0 0x7f26a41538f0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f26a4159df0 tx=0x7f2698009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6766c9b37f826b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.964+0000 7f26a2ffd700 1 -- 172.21.15.55:0/7550488 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698005c10 con 0x7f26a415d3f0 2023-10-06T04:48:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.964+0000 7f26a2ffd700 1 -- 172.21.15.55:0/7550488 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2698005d70 con 0x7f26a415d3f0 2023-10-06T04:48:59.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.964+0000 7f26a2ffd700 1 -- 172.21.15.55:0/7550488 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269800b8d0 con 0x7f26a415d3f0 2023-10-06T04:48:59.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.965+0000 7f26a959d700 1 -- 172.21.15.55:0/7550488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a415d3f0 msgr2=0x7f26a41538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:59.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.965+0000 7f26a959d700 1 --2- 172.21.15.55:0/7550488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a415d3f0 0x7f26a41538f0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f26a4159df0 tx=0x7f2698009220 comp rx=0 tx=0).stop 2023-10-06T04:48:59.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.965+0000 7f26a959d700 1 -- 172.21.15.55:0/7550488 shutdown_connections 2023-10-06T04:48:59.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.965+0000 7f26a959d700 1 --2- 172.21.15.55:0/7550488 >> v2:172.21.15.163:3300/0 conn(0x7f26a40689b0 0x7f26a4153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.965+0000 7f26a959d700 1 --2- 172.21.15.55:0/7550488 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a415d3f0 0x7f26a41538f0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.965+0000 7f26a959d700 1 -- 172.21.15.55:0/7550488 >> 172.21.15.55:0/7550488 conn(0x7f26a406b420 msgr2=0x7f26a40cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:48:59.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.965+0000 7f26a959d700 1 -- 172.21.15.55:0/7550488 shutdown_connections 2023-10-06T04:48:59.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.965+0000 7f26a959d700 1 -- 172.21.15.55:0/7550488 wait complete. 2023-10-06T04:48:59.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.966+0000 7f26a959d700 1 Processor -- start 2023-10-06T04:48:59.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.966+0000 7f26a959d700 1 -- start start 2023-10-06T04:48:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.967+0000 7f26a959d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a40685e0 0x7f26a4142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.967+0000 7f26a959d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a40689b0 0x7f26a40c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:59.971 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.967+0000 7f26a959d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a415d3f0 0x7f26a40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.967+0000 7f26a959d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a416d6a0 con 0x7f26a415d3f0 2023-10-06T04:48:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.967+0000 7f26a959d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a416d980 con 0x7f26a40685e0 2023-10-06T04:48:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.967+0000 7f26a959d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a416dc60 con 0x7f26a40689b0 2023-10-06T04:48:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.967+0000 7f26a8d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a415d3f0 0x7f26a40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.968+0000 7f26a8d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a415d3f0 0x7f26a40ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41772/0 (socket says 172.21.15.55:41772) 2023-10-06T04:48:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.968+0000 7f26a8d9c700 1 -- 172.21.15.55:0/3020992144 learned_addr learned my addr 172.21.15.55:0/3020992144 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:48:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.968+0000 7f26a3fff700 1 --2- 172.21.15.55:0/3020992144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a40685e0 0x7f26a4142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:59.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.968+0000 7f26a8d9c700 1 -- 172.21.15.55:0/3020992144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a40689b0 msgr2=0x7f26a40c9e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:48:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.968+0000 7f26a8d9c700 1 --2- 172.21.15.55:0/3020992144 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26a40689b0 0x7f26a40c9e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.968+0000 7f26a8d9c700 1 -- 172.21.15.55:0/3020992144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a40685e0 msgr2=0x7f26a4142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:48:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.968+0000 7f26a8d9c700 1 --2- 172.21.15.55:0/3020992144 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26a40685e0 0x7f26a4142d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:48:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.968+0000 7f26a8d9c700 1 -- 172.21.15.55:0/3020992144 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2698009480 con 0x7f26a415d3f0 2023-10-06T04:48:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.968+0000 7f26a8d9c700 1 --2- 172.21.15.55:0/3020992144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26a415d3f0 0x7f26a40ca3c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f269c00bbb0 tx=0x7f269c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.968+0000 7f268bfff700 1 -- 172.21.15.55:0/3020992144 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c0092e0 con 0x7f26a415d3f0 2023-10-06T04:48:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.968+0000 7f268bfff700 1 -- 172.21.15.55:0/3020992144 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c00c7a0 con 0x7f26a415d3f0 2023-10-06T04:48:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.968+0000 7f26a959d700 1 -- 172.21.15.55:0/3020992144 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a416df40 con 0x7f26a415d3f0 2023-10-06T04:48:59.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.968+0000 7f268bfff700 1 -- 172.21.15.55:0/3020992144 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c00cb40 con 0x7f26a415d3f0 2023-10-06T04:48:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.968+0000 7f26a959d700 1 -- 172.21.15.55:0/3020992144 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a40b2170 con 0x7f26a415d3f0 2023-10-06T04:48:59.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.969+0000 7f26a959d700 1 -- 172.21.15.55:0/3020992144 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a416e460 con 0x7f26a415d3f0 2023-10-06T04:48:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.970+0000 7f268bfff700 1 -- 172.21.15.55:0/3020992144 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 71) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f269c012070 con 0x7f26a415d3f0 2023-10-06T04:48:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.971+0000 7f268bfff700 1 --2- 172.21.15.55:0/3020992144 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2690066dd0 0x7f2690069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:48:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.971+0000 7f268bfff700 1 -- 172.21.15.55:0/3020992144 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f269c0843a0 con 0x7f26a415d3f0 2023-10-06T04:48:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.972+0000 7f26a3fff700 1 --2- 172.21.15.55:0/3020992144 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2690066dd0 0x7f2690069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:48:59.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.972+0000 7f26a3fff700 1 --2- 172.21.15.55:0/3020992144 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2690066dd0 0x7f2690069280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2694006fd0 tx=0x7f2694008040 comp rx=0 tx=0).ready entity=mgr.7472 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:48:59.977 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:48:59.975+0000 7f268bfff700 1 -- 172.21.15.55:0/3020992144 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26a416e460 con 0x7f26a415d3f0 2023-10-06T04:49:00.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.192+0000 7f268bfff700 1 -- 172.21.15.55:0/3020992144 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 72) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f269c0518a0 con 0x7f26a415d3f0 2023-10-06T04:49:00.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.213+0000 7f26a959d700 1 -- 172.21.15.55:0/3020992144 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f26a416e460 con 0x7f2690066dd0 2023-10-06T04:49:00.222 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:00.220+0000 7f12f7345700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-06T04:49:00.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.220+0000 7f268bfff700 1 -- 172.21.15.55:0/3020992144 <== mgr.7472 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f26a416e460 con 0x7f2690066dd0 2023-10-06T04:49:00.224 INFO:teuthology.orchestra.run.smithi055.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-06T04:49:00.231 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-06T04:49:00.232 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:49:00.232 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-06T04:49:00.271 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T04:49:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.499+0000 7f49fe19d700 1 Processor -- start 2023-10-06T04:49:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.499+0000 7f49fe19d700 1 -- start start 2023-10-06T04:49:00.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.500+0000 7f49fe19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49f80c9db0 0x7f49f806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.500+0000 7f49fe19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f49f8157130 0x7f49f806a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.500+0000 7f49fe19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f806e150 con 0x7f49f8153900 2023-10-06T04:49:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.500+0000 7f49fe19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f80c7610 con 0x7f49f8157130 2023-10-06T04:49:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.500+0000 7f49fe19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f80c7800 con 0x7f49f80c9db0 2023-10-06T04:49:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.501+0000 7f49fc99a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49f80c9db0 0x7f49f806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.501+0000 7f49fd99c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f49f8157130 0x7f49f806a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.501+0000 7f49fc99a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49f80c9db0 0x7f49f806a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:33132/0 (socket says 172.21.15.55:33132) 2023-10-06T04:49:00.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.501+0000 7f49fc99a700 1 -- 172.21.15.55:0/2027630197 learned_addr learned my addr 172.21.15.55:0/2027630197 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.501+0000 7f49fc99a700 1 -- 172.21.15.55:0/2027630197 >> v2:172.21.15.163:3300/0 conn(0x7f49f8157130 msgr2=0x7f49f806a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.501+0000 7f49fc99a700 1 --2- 172.21.15.55:0/2027630197 >> v2:172.21.15.163:3300/0 conn(0x7f49f8157130 0x7f49f806a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.502+0000 7f49fc99a700 1 -- 172.21.15.55:0/2027630197 >> v1:172.21.15.163:6789/0 conn(0x7f49f8153900 legacy=0x7f49f80c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.502+0000 7f49fc99a700 1 -- 172.21.15.55:0/2027630197 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f80c7ae0 con 0x7f49f80c9db0 2023-10-06T04:49:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.502+0000 7f49fd99c700 1 --2- 172.21.15.55:0/2027630197 >> v2:172.21.15.163:3300/0 conn(0x7f49f8157130 0x7f49f806a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:49:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.502+0000 7f49fc99a700 1 --2- 172.21.15.55:0/2027630197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49f80c9db0 0x7f49f806a000 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f49e80099b0 tx=0x7f49e8009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a9b266ad583a1f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.502+0000 7f49effff700 1 -- 172.21.15.55:0/2027630197 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8009ea0 con 0x7f49f80c9db0 2023-10-06T04:49:00.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.502+0000 7f49effff700 1 -- 172.21.15.55:0/2027630197 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e800b860 con 0x7f49f80c9db0 2023-10-06T04:49:00.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.502+0000 7f49effff700 1 -- 172.21.15.55:0/2027630197 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e800bc00 con 0x7f49f80c9db0 2023-10-06T04:49:00.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.503+0000 7f49fe19d700 1 -- 172.21.15.55:0/2027630197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49f80c9db0 msgr2=0x7f49f806a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:00.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.503+0000 7f49fe19d700 1 --2- 172.21.15.55:0/2027630197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49f80c9db0 0x7f49f806a000 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f49e80099b0 tx=0x7f49e8009250 comp rx=0 tx=0).stop 2023-10-06T04:49:00.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.503+0000 7f49fe19d700 1 -- 172.21.15.55:0/2027630197 shutdown_connections 2023-10-06T04:49:00.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.503+0000 7f49fe19d700 1 --2- 172.21.15.55:0/2027630197 >> v2:172.21.15.163:3300/0 conn(0x7f49f8157130 0x7f49f806a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:00.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.503+0000 7f49fe19d700 1 --2- 172.21.15.55:0/2027630197 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49f80c9db0 0x7f49f806a000 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:00.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.503+0000 7f49fe19d700 1 -- 172.21.15.55:0/2027630197 >> 172.21.15.55:0/2027630197 conn(0x7f49f80cc1e0 msgr2=0x7f49f80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:00.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.503+0000 7f49fe19d700 1 -- 172.21.15.55:0/2027630197 shutdown_connections 2023-10-06T04:49:00.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.503+0000 7f49fe19d700 1 -- 172.21.15.55:0/2027630197 wait complete. 2023-10-06T04:49:00.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.504+0000 7f49fe19d700 1 Processor -- start 2023-10-06T04:49:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.504+0000 7f49fe19d700 1 -- start start 2023-10-06T04:49:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.505+0000 7f49fe19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49f80c9db0 0x7f49f814fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.505+0000 7f49fe19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49f8153900 0x7f49f80ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.505+0000 7f49fe19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49f8157130 0x7f49f80aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.505+0000 7f49fe19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f80add50 con 0x7f49f8157130 2023-10-06T04:49:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.506+0000 7f49fe19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f8151750 con 0x7f49f8153900 2023-10-06T04:49:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.506+0000 7f49fe19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f8152570 con 0x7f49f80c9db0 2023-10-06T04:49:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.506+0000 7f49fd99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49f8157130 0x7f49f80aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.506+0000 7f49fd99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49f8157130 0x7f49f80aca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41782/0 (socket says 172.21.15.55:41782) 2023-10-06T04:49:00.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.506+0000 7f49fd99c700 1 -- 172.21.15.55:0/1686532835 learned_addr learned my addr 172.21.15.55:0/1686532835 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.506+0000 7f49fd19b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49f80c9db0 0x7f49f814fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.506+0000 7f49fc99a700 1 --2- 172.21.15.55:0/1686532835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49f8153900 0x7f49f80ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.506+0000 7f49fd99c700 1 -- 172.21.15.55:0/1686532835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49f80c9db0 msgr2=0x7f49f814fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.506+0000 7f49fd99c700 1 --2- 172.21.15.55:0/1686532835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49f80c9db0 0x7f49f814fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.506+0000 7f49fd99c700 1 -- 172.21.15.55:0/1686532835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49f8153900 msgr2=0x7f49f80ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.506+0000 7f49fd99c700 1 --2- 172.21.15.55:0/1686532835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49f8153900 0x7f49f80ac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.506+0000 7f49fd99c700 1 -- 172.21.15.55:0/1686532835 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f0009a20 con 0x7f49f8157130 2023-10-06T04:49:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.507+0000 7f49fd99c700 1 --2- 172.21.15.55:0/1686532835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49f8157130 0x7f49f80aca30 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009f20 tx=0x7f49f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.507+0000 7f49ed7fa700 1 -- 172.21.15.55:0/1686532835 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f000cb70 con 0x7f49f8157130 2023-10-06T04:49:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.507+0000 7f49ed7fa700 1 -- 172.21.15.55:0/1686532835 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f000f040 con 0x7f49f8157130 2023-10-06T04:49:00.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.507+0000 7f49ed7fa700 1 -- 172.21.15.55:0/1686532835 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0014480 con 0x7f49f8157130 2023-10-06T04:49:00.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.507+0000 7f49fe19d700 1 -- 172.21.15.55:0/1686532835 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e80094f0 con 0x7f49f8157130 2023-10-06T04:49:00.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.507+0000 7f49fe19d700 1 -- 172.21.15.55:0/1686532835 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f8151c40 con 0x7f49f8157130 2023-10-06T04:49:00.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.508+0000 7f49fe19d700 1 -- 172.21.15.55:0/1686532835 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f806c450 con 0x7f49f8157130 2023-10-06T04:49:00.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.509+0000 7f49ed7fa700 1 -- 172.21.15.55:0/1686532835 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 72) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f49f0003020 con 0x7f49f8157130 2023-10-06T04:49:00.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.509+0000 7f49ed7fa700 1 --2- 172.21.15.55:0/1686532835 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f49dc067130 0x7f49dc0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:00.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.510+0000 7f49ed7fa700 1 -- 172.21.15.55:0/1686532835 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f49f0085710 con 0x7f49f8157130 2023-10-06T04:49:00.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.510+0000 7f49fd19b700 1 --2- 172.21.15.55:0/1686532835 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f49dc067130 0x7f49dc0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:00.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.511+0000 7f49fd19b700 1 --2- 172.21.15.55:0/1686532835 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f49dc067130 0x7f49dc0695e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f49f40039f0 tx=0x7f49f4007ec0 comp rx=0 tx=0).ready entity=mgr.7472 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:00.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.514+0000 7f49ed7fa700 1 -- 172.21.15.55:0/1686532835 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f49f0052160 con 0x7f49f8157130 2023-10-06T04:49:00.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:00.770+0000 7f49fe19d700 1 -- 172.21.15.55:0/1686532835 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f49f81526b0 con 0x7f49f8157130 2023-10-06T04:49:01.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.194+0000 7f49ed7fa700 1 -- 172.21.15.55:0/1686532835 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 73) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f49f0052530 con 0x7f49f8157130 2023-10-06T04:49:01.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.196+0000 7f49ed7fa700 1 -- 172.21.15.55:0/1686532835 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v73) v1 ==== 93+0+0 (secure 0 0 0) 0x7f49f0052160 con 0x7f49f8157130 2023-10-06T04:49:01.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.199+0000 7f49fe19d700 1 -- 172.21.15.55:0/1686532835 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f49dc067130 msgr2=0x7f49dc0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:01.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.199+0000 7f49fe19d700 1 --2- 172.21.15.55:0/1686532835 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f49dc067130 0x7f49dc0695e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f49f40039f0 tx=0x7f49f4007ec0 comp rx=0 tx=0).stop 2023-10-06T04:49:01.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.199+0000 7f49fe19d700 1 -- 172.21.15.55:0/1686532835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49f8157130 msgr2=0x7f49f80aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:01.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.199+0000 7f49fe19d700 1 --2- 172.21.15.55:0/1686532835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49f8157130 0x7f49f80aca30 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009f20 tx=0x7f49f000b040 comp rx=0 tx=0).stop 2023-10-06T04:49:01.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.199+0000 7f49fe19d700 1 -- 172.21.15.55:0/1686532835 shutdown_connections 2023-10-06T04:49:01.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.199+0000 7f49fe19d700 1 --2- 172.21.15.55:0/1686532835 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f49dc067130 0x7f49dc0695e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:01.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.199+0000 7f49fe19d700 1 --2- 172.21.15.55:0/1686532835 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49f8157130 0x7f49f80aca30 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:01.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.199+0000 7f49fe19d700 1 --2- 172.21.15.55:0/1686532835 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49f80c9db0 0x7f49f814fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:01.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.200+0000 7f49fe19d700 1 --2- 172.21.15.55:0/1686532835 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49f8153900 0x7f49f80ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:01.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.200+0000 7f49fe19d700 1 -- 172.21.15.55:0/1686532835 >> 172.21.15.55:0/1686532835 conn(0x7f49f80cc1e0 msgr2=0x7f49f80aac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:01.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.200+0000 7f49fe19d700 1 -- 172.21.15.55:0/1686532835 shutdown_connections 2023-10-06T04:49:01.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.200+0000 7f49fe19d700 1 -- 172.21.15.55:0/1686532835 wait complete. 2023-10-06T04:49:01.231 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:49:01.231 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:49:01.326 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:49:01.404 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:49:01.404 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T04:49:01.404 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:49:01.404 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T04:49:01.494 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T04:49:01.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:49:01.494 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:49:01.531 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:49:01.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.772+0000 7f5f57fff700 1 Processor -- start 2023-10-06T04:49:01.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.773+0000 7f5f57fff700 1 -- start start 2023-10-06T04:49:01.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.774+0000 7f5f57fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5f5815d230 0x7f5f5815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:01.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.774+0000 7f5f57fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f580b2980 con 0x7f5f581597f0 2023-10-06T04:49:01.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.774+0000 7f5f57fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5815db70 con 0x7f5f58159420 2023-10-06T04:49:01.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.774+0000 7f5f57fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f580b11d0 con 0x7f5f5815d230 2023-10-06T04:49:01.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.774+0000 7f5f567fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5f581597f0 0x7f5f580b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44018/0 (socket says 172.21.15.55:44018) 2023-10-06T04:49:01.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.774+0000 7f5f567fc700 1 -- 172.21.15.55:0/3148700219 learned_addr learned my addr 172.21.15.55:0/3148700219 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:01.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.774+0000 7f5f577fe700 1 --2- 172.21.15.55:0/3148700219 >> v2:172.21.15.163:3300/0 conn(0x7f5f5815d230 0x7f5f5815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.775+0000 7f5f55ffb700 1 -- 172.21.15.55:0/3148700219 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410000988 0 0) 0x7f5f580b2980 con 0x7f5f581597f0 2023-10-06T04:49:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.775+0000 7f5f55ffb700 1 -- 172.21.15.55:0/3148700219 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f40003570 con 0x7f5f581597f0 2023-10-06T04:49:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.775+0000 7f5f577fe700 1 -- 172.21.15.55:0/3148700219 >> v1:172.21.15.163:6789/0 conn(0x7f5f58159420 legacy=0x7f5f580b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.775+0000 7f5f55ffb700 1 -- 172.21.15.55:0/3148700219 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171142862 0 0) 0x7f5f5815db70 con 0x7f5f58159420 2023-10-06T04:49:01.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.775+0000 7f5f577fe700 1 -- 172.21.15.55:0/3148700219 >> v1:172.21.15.55:6789/0 conn(0x7f5f581597f0 legacy=0x7f5f580b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:01.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.775+0000 7f5f577fe700 1 -- 172.21.15.55:0/3148700219 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f580b14b0 con 0x7f5f5815d230 2023-10-06T04:49:01.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.775+0000 7f5f577fe700 1 --2- 172.21.15.55:0/3148700219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f5815d230 0x7f5f5815d630 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009e60 tx=0x7f5f50009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fe97585b0a0f3b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:01.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.775+0000 7f5f55ffb700 1 -- 172.21.15.55:0/3148700219 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5000e070 con 0x7f5f5815d230 2023-10-06T04:49:01.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.776+0000 7f5f55ffb700 1 -- 172.21.15.55:0/3148700219 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5000bb60 con 0x7f5f5815d230 2023-10-06T04:49:01.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.776+0000 7f5f55ffb700 1 -- 172.21.15.55:0/3148700219 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f50012430 con 0x7f5f5815d230 2023-10-06T04:49:01.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.776+0000 7f5f57fff700 1 -- 172.21.15.55:0/3148700219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f5815d230 msgr2=0x7f5f5815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:01.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.776+0000 7f5f57fff700 1 --2- 172.21.15.55:0/3148700219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f5815d230 0x7f5f5815d630 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009e60 tx=0x7f5f50009780 comp rx=0 tx=0).stop 2023-10-06T04:49:01.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.776+0000 7f5f57fff700 1 -- 172.21.15.55:0/3148700219 shutdown_connections 2023-10-06T04:49:01.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.776+0000 7f5f57fff700 1 --2- 172.21.15.55:0/3148700219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f5815d230 0x7f5f5815d630 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:01.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.776+0000 7f5f57fff700 1 -- 172.21.15.55:0/3148700219 >> 172.21.15.55:0/3148700219 conn(0x7f5f580d0660 msgr2=0x7f5f580d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:01.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.776+0000 7f5f57fff700 1 -- 172.21.15.55:0/3148700219 shutdown_connections 2023-10-06T04:49:01.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.776+0000 7f5f57fff700 1 -- 172.21.15.55:0/3148700219 wait complete. 2023-10-06T04:49:01.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.778+0000 7f5f57fff700 1 Processor -- start 2023-10-06T04:49:01.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.778+0000 7f5f57fff700 1 -- start start 2023-10-06T04:49:01.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.779+0000 7f5f57fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f58159420 0x7f5f58169ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:01.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.779+0000 7f5f57fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f581597f0 0x7f5f58154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:01.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.779+0000 7f5f57fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f5815d230 0x7f5f5815bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:01.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.779+0000 7f5f57fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5815a770 con 0x7f5f581597f0 2023-10-06T04:49:01.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.779+0000 7f5f57fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5815aa50 con 0x7f5f5815d230 2023-10-06T04:49:01.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.779+0000 7f5f57fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5815ad30 con 0x7f5f58159420 2023-10-06T04:49:01.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.779+0000 7f5f567fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f581597f0 0x7f5f58154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:01.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.779+0000 7f5f567fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f581597f0 0x7f5f58154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41802/0 (socket says 172.21.15.55:41802) 2023-10-06T04:49:01.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.779+0000 7f5f567fc700 1 -- 172.21.15.55:0/2445955650 learned_addr learned my addr 172.21.15.55:0/2445955650 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:01.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.779+0000 7f5f577fe700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f5815d230 0x7f5f5815bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:01.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.779+0000 7f5f56ffd700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f58159420 0x7f5f58169ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:01.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.780+0000 7f5f577fe700 1 -- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f58159420 msgr2=0x7f5f58169ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:01.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.780+0000 7f5f577fe700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f58159420 0x7f5f58169ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:01.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.780+0000 7f5f577fe700 1 -- 172.21.15.55:0/2445955650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f581597f0 msgr2=0x7f5f58154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:01.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.780+0000 7f5f577fe700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f581597f0 0x7f5f58154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:01.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.780+0000 7f5f577fe700 1 -- 172.21.15.55:0/2445955650 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f50009a20 con 0x7f5f5815d230 2023-10-06T04:49:01.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.780+0000 7f5f567fc700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f581597f0 0x7f5f58154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:49:01.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.780+0000 7f5f577fe700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f5815d230 0x7f5f5815bf80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5f50016000 tx=0x7f5f50019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:01.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.780+0000 7f5f3effd700 1 -- 172.21.15.55:0/2445955650 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f50026480 con 0x7f5f5815d230 2023-10-06T04:49:01.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.780+0000 7f5f3effd700 1 -- 172.21.15.55:0/2445955650 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f50026600 con 0x7f5f5815d230 2023-10-06T04:49:01.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.780+0000 7f5f3effd700 1 -- 172.21.15.55:0/2445955650 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f500269c0 con 0x7f5f5815d230 2023-10-06T04:49:01.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.780+0000 7f5f57fff700 1 -- 172.21.15.55:0/2445955650 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5815b010 con 0x7f5f5815d230 2023-10-06T04:49:01.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.781+0000 7f5f57fff700 1 -- 172.21.15.55:0/2445955650 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5816a000 con 0x7f5f5815d230 2023-10-06T04:49:01.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.781+0000 7f5f57fff700 1 -- 172.21.15.55:0/2445955650 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5804c3c0 con 0x7f5f5815d230 2023-10-06T04:49:01.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.782+0000 7f5f3effd700 1 -- 172.21.15.55:0/2445955650 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 73) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f5f5001c400 con 0x7f5f5815d230 2023-10-06T04:49:01.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.783+0000 7f5f3effd700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5f440670e0 0x7f5f44069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:01.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.784+0000 7f5f3effd700 1 -- 172.21.15.55:0/2445955650 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f5f500870c0 con 0x7f5f5815d230 2023-10-06T04:49:01.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.784+0000 7f5f56ffd700 1 -- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5f440670e0 msgr2=0x7f5f44069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:01.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.784+0000 7f5f56ffd700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5f440670e0 0x7f5f44069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:01.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.788+0000 7f5f3effd700 1 -- 172.21.15.55:0/2445955650 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f50053b10 con 0x7f5f5815d230 2023-10-06T04:49:01.933 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:49:01.941 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:49:01.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.985+0000 7f5f56ffd700 1 -- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5f440670e0 msgr2=0x7f5f44069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:01.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:01.985+0000 7f5f56ffd700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5f440670e0 0x7f5f44069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:02.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.051+0000 7f5f57fff700 1 -- 172.21.15.55:0/2445955650 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f5815ab90 con 0x7f5f5815d230 2023-10-06T04:49:02.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.051+0000 7f5f3effd700 1 -- 172.21.15.55:0/2445955650 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f5f5815ab90 con 0x7f5f5815d230 2023-10-06T04:49:02.054 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:02.054 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8308,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/421768289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":421768289},{"type":"v1","addr":"172.21.15.55:6835","nonce":421768289}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8311,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2920562647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2920562647},{"type":"v1","addr":"172.21.15.163:6834","nonce":2920562647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8314,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2324769448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2324769448},{"type":"v1","addr":"172.21.15.163:6835","nonce":2324769448}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:48:42.309816+0000","modified":"2023-10-06T04:48:45.840578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3504492616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3504492616},{"type":"v1","addr":"172.21.15.55:6834","nonce":3504492616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T04:49:02.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.054+0000 7f5f57fff700 1 -- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5f440670e0 msgr2=0x7f5f44069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:02.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.054+0000 7f5f57fff700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5f440670e0 0x7f5f44069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.054+0000 7f5f57fff700 1 -- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f5815d230 msgr2=0x7f5f5815bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:02.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.054+0000 7f5f57fff700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f5815d230 0x7f5f5815bf80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5f50016000 tx=0x7f5f50019000 comp rx=0 tx=0).stop 2023-10-06T04:49:02.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.054+0000 7f5f57fff700 1 -- 172.21.15.55:0/2445955650 shutdown_connections 2023-10-06T04:49:02.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.054+0000 7f5f57fff700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5f440670e0 0x7f5f44069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.054+0000 7f5f57fff700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5f5815d230 0x7f5f5815bf80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.054+0000 7f5f57fff700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5f58159420 0x7f5f58169ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.054+0000 7f5f57fff700 1 --2- 172.21.15.55:0/2445955650 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5f581597f0 0x7f5f58154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.054+0000 7f5f57fff700 1 -- 172.21.15.55:0/2445955650 >> 172.21.15.55:0/2445955650 conn(0x7f5f580d0660 msgr2=0x7f5f580cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:02.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.054+0000 7f5f57fff700 1 -- 172.21.15.55:0/2445955650 shutdown_connections 2023-10-06T04:49:02.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.054+0000 7f5f57fff700 1 -- 172.21.15.55:0/2445955650 wait complete. 2023-10-06T04:49:02.060 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 78 2023-10-06T04:49:02.076 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:49:02.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.303+0000 7f2a24f9d700 1 Processor -- start 2023-10-06T04:49:02.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.303+0000 7f2a24f9d700 1 -- start start 2023-10-06T04:49:02.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.305+0000 7f2a24f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2a2006d950 0x7f2a200c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:02.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.305+0000 7f2a24f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a2006dd20 0x7f2a200c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:02.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.305+0000 7f2a24f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2006ae70 con 0x7f2a2006a0a0 2023-10-06T04:49:02.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.305+0000 7f2a24f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a2006a540 con 0x7f2a2006d950 2023-10-06T04:49:02.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.305+0000 7f2a24f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2006a820 con 0x7f2a2006dd20 2023-10-06T04:49:02.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.305+0000 7f2a1f7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2a2006d950 0x7f2a200c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:02.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.305+0000 7f2a1effd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2a2006a0a0 0x7f2a200c6c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:44034/0 (socket says 172.21.15.55:44034) 2023-10-06T04:49:02.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.305+0000 7f2a1effd700 1 -- 172.21.15.55:0/822400274 learned_addr learned my addr 172.21.15.55:0/822400274 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:02.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.305+0000 7f2a1ffff700 1 --2- 172.21.15.55:0/822400274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a2006dd20 0x7f2a200c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:02.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.306+0000 7f2a1e7fc700 1 -- 172.21.15.55:0/822400274 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976251482 0 0) 0x7f2a2006ae70 con 0x7f2a2006a0a0 2023-10-06T04:49:02.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.306+0000 7f2a1f7fe700 1 -- 172.21.15.55:0/822400274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a2006dd20 msgr2=0x7f2a200c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:02.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.306+0000 7f2a1f7fe700 1 --2- 172.21.15.55:0/822400274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a2006dd20 0x7f2a200c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.306+0000 7f2a1f7fe700 1 -- 172.21.15.55:0/822400274 >> v1:172.21.15.55:6789/0 conn(0x7f2a2006a0a0 legacy=0x7f2a200c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:02.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.306+0000 7f2a1f7fe700 1 -- 172.21.15.55:0/822400274 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2006ab00 con 0x7f2a2006d950 2023-10-06T04:49:02.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.306+0000 7f2a1f7fe700 1 --2- 172.21.15.55:0/822400274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a2006d950 0x7f2a200c8350 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f2a100098e0 tx=0x7f2a100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc68c79eb49759f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:02.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.306+0000 7f2a1e7fc700 1 -- 172.21.15.55:0/822400274 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1000b640 con 0x7f2a2006d950 2023-10-06T04:49:02.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.306+0000 7f2a1e7fc700 1 -- 172.21.15.55:0/822400274 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1000b7a0 con 0x7f2a2006d950 2023-10-06T04:49:02.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.306+0000 7f2a1e7fc700 1 -- 172.21.15.55:0/822400274 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1000bb40 con 0x7f2a2006d950 2023-10-06T04:49:02.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.307+0000 7f2a24f9d700 1 -- 172.21.15.55:0/822400274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a2006d950 msgr2=0x7f2a200c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:02.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.307+0000 7f2a24f9d700 1 --2- 172.21.15.55:0/822400274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a2006d950 0x7f2a200c8350 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f2a100098e0 tx=0x7f2a100091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.307+0000 7f2a24f9d700 1 -- 172.21.15.55:0/822400274 shutdown_connections 2023-10-06T04:49:02.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.307+0000 7f2a24f9d700 1 --2- 172.21.15.55:0/822400274 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a2006dd20 0x7f2a200c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.307+0000 7f2a24f9d700 1 --2- 172.21.15.55:0/822400274 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a2006d950 0x7f2a200c8350 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.307+0000 7f2a24f9d700 1 -- 172.21.15.55:0/822400274 >> 172.21.15.55:0/822400274 conn(0x7f2a200cc1f0 msgr2=0x7f2a200cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:02.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.307+0000 7f2a24f9d700 1 -- 172.21.15.55:0/822400274 shutdown_connections 2023-10-06T04:49:02.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.307+0000 7f2a24f9d700 1 -- 172.21.15.55:0/822400274 wait complete. 2023-10-06T04:49:02.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.309+0000 7f2a24f9d700 1 Processor -- start 2023-10-06T04:49:02.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.309+0000 7f2a24f9d700 1 -- start start 2023-10-06T04:49:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a24f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a2006a0a0 0x7f2a20154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a24f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a2006d950 0x7f2a201551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a24f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a2006dd20 0x7f2a200c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a24f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2015ad90 con 0x7f2a2006d950 2023-10-06T04:49:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a24f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a20158070 con 0x7f2a2006dd20 2023-10-06T04:49:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a24f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2015a4f0 con 0x7f2a2006a0a0 2023-10-06T04:49:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a1effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a2006d950 0x7f2a201551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a1effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a2006d950 0x7f2a201551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41832/0 (socket says 172.21.15.55:41832) 2023-10-06T04:49:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a1effd700 1 -- 172.21.15.55:0/3072145219 learned_addr learned my addr 172.21.15.55:0/3072145219 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a1f7fe700 1 --2- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a2006a0a0 0x7f2a20154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:02.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a1ffff700 1 --2- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a2006dd20 0x7f2a200c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:02.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a1f7fe700 1 -- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a2006dd20 msgr2=0x7f2a200c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:02.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a1f7fe700 1 --2- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a2006dd20 0x7f2a200c5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a1f7fe700 1 -- 172.21.15.55:0/3072145219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a2006d950 msgr2=0x7f2a201551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:02.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a1f7fe700 1 --2- 172.21.15.55:0/3072145219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a2006d950 0x7f2a201551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.310+0000 7f2a1f7fe700 1 -- 172.21.15.55:0/3072145219 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a10009420 con 0x7f2a2006a0a0 2023-10-06T04:49:02.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.311+0000 7f2a1f7fe700 1 --2- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a2006a0a0 0x7f2a20154c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2a10016000 tx=0x7f2a10017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:02.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.311+0000 7f2a077fe700 1 -- 172.21.15.55:0/3072145219 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1000bb20 con 0x7f2a2006a0a0 2023-10-06T04:49:02.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.311+0000 7f2a077fe700 1 -- 172.21.15.55:0/3072145219 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a10000c90 con 0x7f2a2006a0a0 2023-10-06T04:49:02.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.311+0000 7f2a077fe700 1 -- 172.21.15.55:0/3072145219 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10011650 con 0x7f2a2006a0a0 2023-10-06T04:49:02.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.311+0000 7f2a24f9d700 1 -- 172.21.15.55:0/3072145219 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2015a7d0 con 0x7f2a2006a0a0 2023-10-06T04:49:02.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.311+0000 7f2a24f9d700 1 -- 172.21.15.55:0/3072145219 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a200c3340 con 0x7f2a2006a0a0 2023-10-06T04:49:02.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.312+0000 7f2a24f9d700 1 -- 172.21.15.55:0/3072145219 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a200c3530 con 0x7f2a2006a0a0 2023-10-06T04:49:02.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.314+0000 7f2a077fe700 1 -- 172.21.15.55:0/3072145219 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 73) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f2a10011a00 con 0x7f2a2006a0a0 2023-10-06T04:49:02.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.315+0000 7f2a077fe700 1 --2- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a0c0670e0 0x7f2a0c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:02.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.315+0000 7f2a077fe700 1 -- 172.21.15.55:0/3072145219 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f2a100861b0 con 0x7f2a2006a0a0 2023-10-06T04:49:02.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.315+0000 7f2a1effd700 1 -- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a0c0670e0 msgr2=0x7f2a0c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:02.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.315+0000 7f2a1effd700 1 --2- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a0c0670e0 0x7f2a0c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:02.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.320+0000 7f2a077fe700 1 -- 172.21.15.55:0/3072145219 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a10052d10 con 0x7f2a2006a0a0 2023-10-06T04:49:02.505 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:02.504+0000 7fecea2cb180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T04:49:02.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.516+0000 7f2a1effd700 1 -- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a0c0670e0 msgr2=0x7f2a0c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:02.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.516+0000 7f2a1effd700 1 --2- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a0c0670e0 0x7f2a0c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:02.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.523+0000 7f2a24f9d700 1 -- 172.21.15.55:0/3072145219 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a2004c3c0 con 0x7f2a2006a0a0 2023-10-06T04:49:02.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.524+0000 7f2a077fe700 1 -- 172.21.15.55:0/3072145219 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+17896 (secure 0 0 0) 0x7f2a10011cb0 con 0x7f2a2006a0a0 2023-10-06T04:49:02.527 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:02.527 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":161,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T04:48:58.128628+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:48:40.391644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T04:48:41.276409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3452722174":"2023-10-07T04:48:58.128595+0000","172.21.15.163:6837/85191":"2023-10-07T04:48:58.128595+0000","172.21.15.163:6836/85191":"2023-10-07T04:48:58.128595+0000","172.21.15.163:0/1904936092":"2023-10-07T04:48:58.128595+0000","172.21.15.163:0/3499125134":"2023-10-07T04:48:58.128595+0000","172.21.15.163:0/2488193308":"2023-10-07T04:48:58.128595+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:49:02.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.527+0000 7f2a24f9d700 1 -- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a0c0670e0 msgr2=0x7f2a0c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:02.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.527+0000 7f2a24f9d700 1 --2- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a0c0670e0 0x7f2a0c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.527+0000 7f2a24f9d700 1 -- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a2006a0a0 msgr2=0x7f2a20154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:02.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.527+0000 7f2a24f9d700 1 --2- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a2006a0a0 0x7f2a20154c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2a10016000 tx=0x7f2a10017000 comp rx=0 tx=0).stop 2023-10-06T04:49:02.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.527+0000 7f2a24f9d700 1 -- 172.21.15.55:0/3072145219 shutdown_connections 2023-10-06T04:49:02.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.527+0000 7f2a24f9d700 1 --2- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a0c0670e0 0x7f2a0c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.527+0000 7f2a24f9d700 1 --2- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a2006dd20 0x7f2a200c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.527+0000 7f2a24f9d700 1 --2- 172.21.15.55:0/3072145219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a2006a0a0 0x7f2a20154c80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.527+0000 7f2a24f9d700 1 --2- 172.21.15.55:0/3072145219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a2006d950 0x7f2a201551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.527+0000 7f2a24f9d700 1 -- 172.21.15.55:0/3072145219 >> 172.21.15.55:0/3072145219 conn(0x7f2a200cc1f0 msgr2=0x7f2a20152870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:02.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.527+0000 7f2a24f9d700 1 -- 172.21.15.55:0/3072145219 shutdown_connections 2023-10-06T04:49:02.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.527+0000 7f2a24f9d700 1 -- 172.21.15.55:0/3072145219 wait complete. 2023-10-06T04:49:02.548 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:48:40.391644+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:48:41.276409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '160', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:49:02.549 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:49:02.574 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:02.571+0000 7fb5daa99180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T04:49:02.670 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:02.669+0000 7fecea2cb180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T04:49:02.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.738+0000 7f2e63fff700 1 Processor -- start 2023-10-06T04:49:02.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.739+0000 7f2e63fff700 1 -- start start 2023-10-06T04:49:02.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.740+0000 7f2e63fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2e64159a60 0x7f2e640c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:02.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.740+0000 7f2e63fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2e640ccc00 0x7f2e64069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:02.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.740+0000 7f2e63fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e6406a020 0x7f2e640cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:02.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.740+0000 7f2e63fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6406dfc0 con 0x7f2e640ccc00 2023-10-06T04:49:02.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.740+0000 7f2e63fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e640c9cb0 con 0x7f2e64159a60 2023-10-06T04:49:02.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.740+0000 7f2e63fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640c9f30 con 0x7f2e6406a020 2023-10-06T04:49:02.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.740+0000 7f2e627fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2e640ccc00 0x7f2e64069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:02.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.740+0000 7f2e627fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2e640ccc00 0x7f2e64069ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41838/0 (socket says 172.21.15.55:41838) 2023-10-06T04:49:02.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.740+0000 7f2e627fc700 1 -- 172.21.15.55:0/181177221 learned_addr learned my addr 172.21.15.55:0/181177221 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:02.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.740+0000 7f2e62ffd700 1 --2- 172.21.15.55:0/181177221 >> v2:172.21.15.163:3300/0 conn(0x7f2e64159a60 0x7f2e640c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:02.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.740+0000 7f2e637fe700 1 --2- 172.21.15.55:0/181177221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e6406a020 0x7f2e640cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:02.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.741+0000 7f2e627fc700 1 -- 172.21.15.55:0/181177221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e6406a020 msgr2=0x7f2e640cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:02.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.741+0000 7f2e627fc700 1 --2- 172.21.15.55:0/181177221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e6406a020 0x7f2e640cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.741+0000 7f2e627fc700 1 -- 172.21.15.55:0/181177221 >> v2:172.21.15.163:3300/0 conn(0x7f2e64159a60 msgr2=0x7f2e640c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:02.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.741+0000 7f2e627fc700 1 --2- 172.21.15.55:0/181177221 >> v2:172.21.15.163:3300/0 conn(0x7f2e64159a60 0x7f2e640c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.741+0000 7f2e627fc700 1 -- 172.21.15.55:0/181177221 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e640ca470 con 0x7f2e640ccc00 2023-10-06T04:49:02.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.741+0000 7f2e62ffd700 1 --2- 172.21.15.55:0/181177221 >> v2:172.21.15.163:3300/0 conn(0x7f2e64159a60 0x7f2e640c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:49:02.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.741+0000 7f2e627fc700 1 --2- 172.21.15.55:0/181177221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e640ccc00 0x7f2e64069ae0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f2e6406e510 tx=0x7f2e58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25e01a9aa0a023d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:02.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.741+0000 7f2e61ffb700 1 -- 172.21.15.55:0/181177221 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5800b6d0 con 0x7f2e640ccc00 2023-10-06T04:49:02.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.741+0000 7f2e61ffb700 1 -- 172.21.15.55:0/181177221 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5800b830 con 0x7f2e640ccc00 2023-10-06T04:49:02.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.742+0000 7f2e61ffb700 1 -- 172.21.15.55:0/181177221 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5800bbf0 con 0x7f2e640ccc00 2023-10-06T04:49:02.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.742+0000 7f2e63fff700 1 -- 172.21.15.55:0/181177221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e640ccc00 msgr2=0x7f2e64069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:02.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.742+0000 7f2e63fff700 1 --2- 172.21.15.55:0/181177221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e640ccc00 0x7f2e64069ae0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f2e6406e510 tx=0x7f2e58009250 comp rx=0 tx=0).stop 2023-10-06T04:49:02.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.742+0000 7f2e63fff700 1 -- 172.21.15.55:0/181177221 shutdown_connections 2023-10-06T04:49:02.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.742+0000 7f2e63fff700 1 --2- 172.21.15.55:0/181177221 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e6406a020 0x7f2e640cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.742+0000 7f2e63fff700 1 --2- 172.21.15.55:0/181177221 >> v2:172.21.15.163:3300/0 conn(0x7f2e64159a60 0x7f2e640c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.742+0000 7f2e63fff700 1 --2- 172.21.15.55:0/181177221 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e640ccc00 0x7f2e64069ae0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.742+0000 7f2e63fff700 1 -- 172.21.15.55:0/181177221 >> 172.21.15.55:0/181177221 conn(0x7f2e6406baa0 msgr2=0x7f2e6406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:02.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.742+0000 7f2e63fff700 1 -- 172.21.15.55:0/181177221 shutdown_connections 2023-10-06T04:49:02.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.742+0000 7f2e63fff700 1 -- 172.21.15.55:0/181177221 wait complete. 2023-10-06T04:49:02.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.744+0000 7f2e63fff700 1 Processor -- start 2023-10-06T04:49:02.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.744+0000 7f2e63fff700 1 -- start start 2023-10-06T04:49:02.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e63fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e6406a020 0x7f2e641554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:02.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e63fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e640ccc00 0x7f2e640b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:02.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e63fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e64159a60 0x7f2e640c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:02.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e63fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6416e0d0 con 0x7f2e64159a60 2023-10-06T04:49:02.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e63fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640c5740 con 0x7f2e6406a020 2023-10-06T04:49:02.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e63fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64154c10 con 0x7f2e640ccc00 2023-10-06T04:49:02.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e637fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e64159a60 0x7f2e640c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:02.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e637fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e64159a60 0x7f2e640c5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:41854/0 (socket says 172.21.15.55:41854) 2023-10-06T04:49:02.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e637fe700 1 -- 172.21.15.55:0/4015002499 learned_addr learned my addr 172.21.15.55:0/4015002499 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:02.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e627fc700 1 --2- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e640ccc00 0x7f2e640b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:02.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e62ffd700 1 --2- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e6406a020 0x7f2e641554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:02.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e637fe700 1 -- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e640ccc00 msgr2=0x7f2e640b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:02.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e637fe700 1 --2- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e640ccc00 0x7f2e640b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e637fe700 1 -- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e6406a020 msgr2=0x7f2e641554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:02.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e637fe700 1 --2- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e6406a020 0x7f2e641554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.745+0000 7f2e637fe700 1 -- 172.21.15.55:0/4015002499 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e54009420 con 0x7f2e64159a60 2023-10-06T04:49:02.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.746+0000 7f2e637fe700 1 --2- 172.21.15.55:0/4015002499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e64159a60 0x7f2e640c5eb0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009e90 tx=0x7f2e5c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:02.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.746+0000 7f2e4affd700 1 -- 172.21.15.55:0/4015002499 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c00cb50 con 0x7f2e64159a60 2023-10-06T04:49:02.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.746+0000 7f2e4affd700 1 -- 172.21.15.55:0/4015002499 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c00f040 con 0x7f2e64159a60 2023-10-06T04:49:02.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.746+0000 7f2e4affd700 1 -- 172.21.15.55:0/4015002499 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c014480 con 0x7f2e64159a60 2023-10-06T04:49:02.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.746+0000 7f2e63fff700 1 -- 172.21.15.55:0/4015002499 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e580094f0 con 0x7f2e64159a60 2023-10-06T04:49:02.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.746+0000 7f2e63fff700 1 -- 172.21.15.55:0/4015002499 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e640c5990 con 0x7f2e64159a60 2023-10-06T04:49:02.751 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:02.746+0000 7fb5daa99180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T04:49:02.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.747+0000 7f2e63fff700 1 -- 172.21.15.55:0/4015002499 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e640c5bb0 con 0x7f2e64159a60 2023-10-06T04:49:02.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.748+0000 7f2e4affd700 1 -- 172.21.15.55:0/4015002499 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 73) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f2e5c014830 con 0x7f2e64159a60 2023-10-06T04:49:02.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.749+0000 7f2e4affd700 1 --2- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2e500670e0 0x7f2e50069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:02.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.749+0000 7f2e4affd700 1 -- 172.21.15.55:0/4015002499 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f2e5c085790 con 0x7f2e64159a60 2023-10-06T04:49:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.749+0000 7f2e62ffd700 1 -- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2e500670e0 msgr2=0x7f2e50069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:02.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.749+0000 7f2e62ffd700 1 --2- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2e500670e0 0x7f2e50069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:02.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.754+0000 7f2e4affd700 1 -- 172.21.15.55:0/4015002499 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e5c0521e0 con 0x7f2e64159a60 2023-10-06T04:49:02.756 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:02.755+0000 7fecea2cb180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T04:49:02.839 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:02.836+0000 7fb5daa99180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T04:49:02.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.950+0000 7f2e62ffd700 1 -- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2e500670e0 msgr2=0x7f2e50069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:02.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.950+0000 7f2e62ffd700 1 --2- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2e500670e0 0x7f2e50069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:02.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.954+0000 7f2e63fff700 1 -- 172.21.15.55:0/4015002499 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2e640ca750 con 0x7f2e64159a60 2023-10-06T04:49:02.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.954+0000 7f2e4affd700 1 -- 172.21.15.55:0/4015002499 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v161) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2e5c055800 con 0x7f2e64159a60 2023-10-06T04:49:02.957 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T04:49:02.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.956+0000 7f2e63fff700 1 -- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2e500670e0 msgr2=0x7f2e50069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:02.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.956+0000 7f2e63fff700 1 --2- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2e500670e0 0x7f2e50069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.956+0000 7f2e63fff700 1 -- 172.21.15.55:0/4015002499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e64159a60 msgr2=0x7f2e640c5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:02.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.956+0000 7f2e63fff700 1 --2- 172.21.15.55:0/4015002499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e64159a60 0x7f2e640c5eb0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009e90 tx=0x7f2e5c00b040 comp rx=0 tx=0).stop 2023-10-06T04:49:02.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.956+0000 7f2e63fff700 1 -- 172.21.15.55:0/4015002499 shutdown_connections 2023-10-06T04:49:02.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.956+0000 7f2e63fff700 1 --2- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2e500670e0 0x7f2e50069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.956+0000 7f2e63fff700 1 --2- 172.21.15.55:0/4015002499 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2e64159a60 0x7f2e640c5eb0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.956+0000 7f2e63fff700 1 --2- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2e6406a020 0x7f2e641554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.956+0000 7f2e63fff700 1 --2- 172.21.15.55:0/4015002499 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2e640ccc00 0x7f2e640b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:02.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.956+0000 7f2e63fff700 1 -- 172.21.15.55:0/4015002499 >> 172.21.15.55:0/4015002499 conn(0x7f2e6406baa0 msgr2=0x7f2e64158760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:02.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.957+0000 7f2e63fff700 1 -- 172.21.15.55:0/4015002499 shutdown_connections 2023-10-06T04:49:02.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:02.957+0000 7f2e63fff700 1 -- 172.21.15.55:0/4015002499 wait complete. 2023-10-06T04:49:02.978 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:49:02.982 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:02.981+0000 7fecea2cb180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:49:03.076 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:03.073+0000 7fb5daa99180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:49:03.160 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:03.158+0000 7fecea2cb180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T04:49:03.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.208+0000 7f1c08b9d700 1 Processor -- start 2023-10-06T04:49:03.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.208+0000 7f1c08b9d700 1 -- start start 2023-10-06T04:49:03.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.209+0000 7f1c08b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c040b4e70 0x7f1c040ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:03.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.209+0000 7f1c08b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c040b1b40 0x7f1c0415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:03.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.209+0000 7f1c08b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c040cfb00 con 0x7f1c040b16d0 2023-10-06T04:49:03.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.209+0000 7f1c08b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c040b5c40 con 0x7f1c040b1b40 2023-10-06T04:49:03.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.209+0000 7f1c08b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040b5f20 con 0x7f1c040b4e70 2023-10-06T04:49:03.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.210+0000 7f1c03fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c040b1b40 0x7f1c0415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:03.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.210+0000 7f1c03fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1c040b1b40 0x7f1c0415d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42608/0 (socket says 172.21.15.55:42608) 2023-10-06T04:49:03.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.210+0000 7f1c03fff700 1 -- 172.21.15.55:0/1071589738 learned_addr learned my addr 172.21.15.55:0/1071589738 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:03.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.210+0000 7f1c02ffd700 1 --1- 172.21.15.55:0/1071589738 >> v1:172.21.15.55:6789/0 conn(0x7f1c040b16d0 0x7f1c0415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42512/0 (socket says 172.21.15.55:42512) 2023-10-06T04:49:03.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.210+0000 7f1c037fe700 1 --2- 172.21.15.55:0/1071589738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c040b4e70 0x7f1c040ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:03.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.210+0000 7f1c03fff700 1 -- 172.21.15.55:0/1071589738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c040b4e70 msgr2=0x7f1c040ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:03.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.210+0000 7f1c03fff700 1 --2- 172.21.15.55:0/1071589738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c040b4e70 0x7f1c040ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.210+0000 7f1c03fff700 1 -- 172.21.15.55:0/1071589738 >> v1:172.21.15.55:6789/0 conn(0x7f1c040b16d0 legacy=0x7f1c0415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:03.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.210+0000 7f1c03fff700 1 -- 172.21.15.55:0/1071589738 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c040b6200 con 0x7f1c040b1b40 2023-10-06T04:49:03.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.211+0000 7f1c03fff700 1 --2- 172.21.15.55:0/1071589738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c040b1b40 0x7f1c0415d950 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009f20 tx=0x7f1bfc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac8d402c28ea53f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:03.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.211+0000 7f1c027fc700 1 -- 172.21.15.55:0/1071589738 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc00e070 con 0x7f1c040b1b40 2023-10-06T04:49:03.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.211+0000 7f1c027fc700 1 -- 172.21.15.55:0/1071589738 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bfc00b9d0 con 0x7f1c040b1b40 2023-10-06T04:49:03.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.211+0000 7f1c027fc700 1 -- 172.21.15.55:0/1071589738 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0092e0 con 0x7f1c040b1b40 2023-10-06T04:49:03.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.212+0000 7f1c08b9d700 1 -- 172.21.15.55:0/1071589738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c040b1b40 msgr2=0x7f1c0415d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:03.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.212+0000 7f1c08b9d700 1 --2- 172.21.15.55:0/1071589738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c040b1b40 0x7f1c0415d950 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009f20 tx=0x7f1bfc009840 comp rx=0 tx=0).stop 2023-10-06T04:49:03.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.212+0000 7f1c08b9d700 1 -- 172.21.15.55:0/1071589738 shutdown_connections 2023-10-06T04:49:03.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.212+0000 7f1c08b9d700 1 --2- 172.21.15.55:0/1071589738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c040b1b40 0x7f1c0415d950 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.212+0000 7f1c08b9d700 1 --2- 172.21.15.55:0/1071589738 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c040b4e70 0x7f1c040ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.212+0000 7f1c08b9d700 1 -- 172.21.15.55:0/1071589738 >> 172.21.15.55:0/1071589738 conn(0x7f1c040d10c0 msgr2=0x7f1c040affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:03.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.212+0000 7f1c08b9d700 1 -- 172.21.15.55:0/1071589738 shutdown_connections 2023-10-06T04:49:03.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.212+0000 7f1c08b9d700 1 -- 172.21.15.55:0/1071589738 wait complete. 2023-10-06T04:49:03.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.213+0000 7f1c08b9d700 1 Processor -- start 2023-10-06T04:49:03.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.214+0000 7f1c08b9d700 1 -- start start 2023-10-06T04:49:03.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.214+0000 7f1c08b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c040b16d0 0x7f1c041459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:03.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c08b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c040b1b40 0x7f1c040c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:03.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c08b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c040b4e70 0x7f1c040c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:03.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c08b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c041585c0 con 0x7f1c040b4e70 2023-10-06T04:49:03.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c08b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040c8e00 con 0x7f1c040b16d0 2023-10-06T04:49:03.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c08b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040c90e0 con 0x7f1c040b1b40 2023-10-06T04:49:03.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c03fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c040b4e70 0x7f1c040c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:03.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c03fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c040b4e70 0x7f1c040c84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42612/0 (socket says 172.21.15.55:42612) 2023-10-06T04:49:03.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c03fff700 1 -- 172.21.15.55:0/3573212501 learned_addr learned my addr 172.21.15.55:0/3573212501 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:03.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c037fe700 1 --2- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c040b16d0 0x7f1c041459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:03.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c02ffd700 1 --2- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c040b1b40 0x7f1c040c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:03.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c03fff700 1 -- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c040b1b40 msgr2=0x7f1c040c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:03.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c03fff700 1 --2- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c040b1b40 0x7f1c040c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c03fff700 1 -- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c040b16d0 msgr2=0x7f1c041459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:03.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c03fff700 1 --2- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c040b16d0 0x7f1c041459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.215+0000 7f1c03fff700 1 -- 172.21.15.55:0/3573212501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc009ae0 con 0x7f1c040b4e70 2023-10-06T04:49:03.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.216+0000 7f1c03fff700 1 --2- 172.21.15.55:0/3573212501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c040b4e70 0x7f1c040c84b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f1c04070dd0 tx=0x7f1bfc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:03.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.216+0000 7f1beb7fe700 1 -- 172.21.15.55:0/3573212501 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc008ab0 con 0x7f1c040b4e70 2023-10-06T04:49:03.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.216+0000 7f1beb7fe700 1 -- 172.21.15.55:0/3573212501 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bfc008c30 con 0x7f1c040b4e70 2023-10-06T04:49:03.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.216+0000 7f1c08b9d700 1 -- 172.21.15.55:0/3573212501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c040c93c0 con 0x7f1c040b4e70 2023-10-06T04:49:03.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.216+0000 7f1beb7fe700 1 -- 172.21.15.55:0/3573212501 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc026460 con 0x7f1c040b4e70 2023-10-06T04:49:03.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.216+0000 7f1c08b9d700 1 -- 172.21.15.55:0/3573212501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c040c9980 con 0x7f1c040b4e70 2023-10-06T04:49:03.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.217+0000 7f1c08b9d700 1 -- 172.21.15.55:0/3573212501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c040c9b70 con 0x7f1c040b4e70 2023-10-06T04:49:03.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.218+0000 7f1beb7fe700 1 -- 172.21.15.55:0/3573212501 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 73) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f1bfc003040 con 0x7f1c040b4e70 2023-10-06T04:49:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.219+0000 7f1beb7fe700 1 --2- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1bf00670e0 0x7f1bf0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.219+0000 7f1beb7fe700 1 -- 172.21.15.55:0/3573212501 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f1bfc086d00 con 0x7f1c040b4e70 2023-10-06T04:49:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.219+0000 7f1c037fe700 1 -- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1bf00670e0 msgr2=0x7f1bf0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:03.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.219+0000 7f1c037fe700 1 --2- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1bf00670e0 0x7f1bf0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:03.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.223+0000 7f1beb7fe700 1 -- 172.21.15.55:0/3573212501 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1bfc053800 con 0x7f1c040b4e70 2023-10-06T04:49:03.254 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:03.252+0000 7fecea2cb180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T04:49:03.264 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:03.261+0000 7fb5daa99180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T04:49:03.360 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:03.357+0000 7fb5daa99180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T04:49:03.416 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:03.415+0000 7fecea2cb180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T04:49:03.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.420+0000 7f1c037fe700 1 -- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1bf00670e0 msgr2=0x7f1bf0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:03.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.420+0000 7f1c037fe700 1 --2- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1bf00670e0 0x7f1bf0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:03.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.427+0000 7f1c08b9d700 1 -- 172.21.15.55:0/3573212501 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c0404bb80 con 0x7f1c040b4e70 2023-10-06T04:49:03.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.428+0000 7f1beb7fe700 1 -- 172.21.15.55:0/3573212501 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v161) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1bfc056e20 con 0x7f1c040b4e70 2023-10-06T04:49:03.431 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T04:49:03.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.430+0000 7f1c08b9d700 1 -- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1bf00670e0 msgr2=0x7f1bf0069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:03.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.430+0000 7f1c08b9d700 1 --2- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1bf00670e0 0x7f1bf0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.430+0000 7f1c08b9d700 1 -- 172.21.15.55:0/3573212501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c040b4e70 msgr2=0x7f1c040c84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:03.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.430+0000 7f1c08b9d700 1 --2- 172.21.15.55:0/3573212501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c040b4e70 0x7f1c040c84b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f1c04070dd0 tx=0x7f1bfc018000 comp rx=0 tx=0).stop 2023-10-06T04:49:03.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.431+0000 7f1c08b9d700 1 -- 172.21.15.55:0/3573212501 shutdown_connections 2023-10-06T04:49:03.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.431+0000 7f1c08b9d700 1 --2- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1bf00670e0 0x7f1bf0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.431+0000 7f1c08b9d700 1 --2- 172.21.15.55:0/3573212501 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c040b4e70 0x7f1c040c84b0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.431+0000 7f1c08b9d700 1 --2- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c040b16d0 0x7f1c041459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.431+0000 7f1c08b9d700 1 --2- 172.21.15.55:0/3573212501 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c040b1b40 0x7f1c040c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.431+0000 7f1c08b9d700 1 -- 172.21.15.55:0/3573212501 >> 172.21.15.55:0/3573212501 conn(0x7f1c040d10c0 msgr2=0x7f1c04159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:03.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.431+0000 7f1c08b9d700 1 -- 172.21.15.55:0/3573212501 shutdown_connections 2023-10-06T04:49:03.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.431+0000 7f1c08b9d700 1 -- 172.21.15.55:0/3573212501 wait complete. 2023-10-06T04:49:03.452 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:49:03.517 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:03.516+0000 7fecea2cb180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T04:49:03.530 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:03.527+0000 7fb5daa99180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T04:49:03.637 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:03.634+0000 7fb5daa99180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T04:49:03.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.640+0000 7fa60299d700 1 Processor -- start 2023-10-06T04:49:03.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.640+0000 7fa60299d700 1 -- start start 2023-10-06T04:49:03.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.642+0000 7fa60299d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa5fc0b1b40 0x7fa5fc0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:03.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.642+0000 7fa60299d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5fc0cfb00 con 0x7fa5fc0b16d0 2023-10-06T04:49:03.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.642+0000 7fa60299d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5fc0b5c40 con 0x7fa5fc0b4e70 2023-10-06T04:49:03.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.642+0000 7fa60299d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5fc0b5f20 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.642+0000 7fa60219c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa5fc0b1b40 0x7fa5fc0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:03.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.642+0000 7fa60119a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa5fc0b16d0 0x7fa5fc0b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42524/0 (socket says 172.21.15.55:42524) 2023-10-06T04:49:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.642+0000 7fa60119a700 1 -- 172.21.15.55:0/421282697 learned_addr learned my addr 172.21.15.55:0/421282697 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.642+0000 7fa60219c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa5fc0b1b40 0x7fa5fc0b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42614/0 (socket says 172.21.15.55:42614) 2023-10-06T04:49:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.642+0000 7fa60219c700 1 -- 172.21.15.55:0/421282697 >> v1:172.21.15.163:6789/0 conn(0x7fa5fc0b4e70 legacy=0x7fa5fc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.642+0000 7fa60219c700 1 -- 172.21.15.55:0/421282697 >> v1:172.21.15.55:6789/0 conn(0x7fa5fc0b16d0 legacy=0x7fa5fc0b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.642+0000 7fa60219c700 1 -- 172.21.15.55:0/421282697 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc0b6200 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.643+0000 7fa60219c700 1 --2- 172.21.15.55:0/421282697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5fc0b1b40 0x7fa5fc0b3ac0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009e60 tx=0x7fa5f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c6218b5871225f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.643+0000 7fa600999700 1 -- 172.21.15.55:0/421282697 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f400e070 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.643+0000 7fa600999700 1 -- 172.21.15.55:0/421282697 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f400bc30 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.643+0000 7fa600999700 1 -- 172.21.15.55:0/421282697 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4012430 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.644+0000 7fa60299d700 1 -- 172.21.15.55:0/421282697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5fc0b1b40 msgr2=0x7fa5fc0b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.644+0000 7fa60299d700 1 --2- 172.21.15.55:0/421282697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5fc0b1b40 0x7fa5fc0b3ac0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009e60 tx=0x7fa5f4009780 comp rx=0 tx=0).stop 2023-10-06T04:49:03.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.644+0000 7fa60299d700 1 -- 172.21.15.55:0/421282697 shutdown_connections 2023-10-06T04:49:03.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.644+0000 7fa60299d700 1 --2- 172.21.15.55:0/421282697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5fc0b1b40 0x7fa5fc0b3ac0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.644+0000 7fa60299d700 1 -- 172.21.15.55:0/421282697 >> 172.21.15.55:0/421282697 conn(0x7fa5fc0d10c0 msgr2=0x7fa5fc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:03.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.644+0000 7fa60299d700 1 -- 172.21.15.55:0/421282697 shutdown_connections 2023-10-06T04:49:03.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.644+0000 7fa60299d700 1 -- 172.21.15.55:0/421282697 wait complete. 2023-10-06T04:49:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.645+0000 7fa60299d700 1 Processor -- start 2023-10-06T04:49:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.645+0000 7fa60299d700 1 -- start start 2023-10-06T04:49:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.646+0000 7fa60299d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5fc0b16d0 0x7fa5fc1437b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.646+0000 7fa60299d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5fc0b1b40 0x7fa5fc069510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.646+0000 7fa60299d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5fc0b4e70 0x7fa5fc069a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.646+0000 7fa60299d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc143cf0 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.646+0000 7fa60299d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc16bd20 con 0x7fa5fc0b4e70 2023-10-06T04:49:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.646+0000 7fa60299d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc16b110 con 0x7fa5fc0b16d0 2023-10-06T04:49:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.647+0000 7fa60119a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5fc0b1b40 0x7fa5fc069510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:03.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.647+0000 7fa60119a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5fc0b1b40 0x7fa5fc069510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42626/0 (socket says 172.21.15.55:42626) 2023-10-06T04:49:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.647+0000 7fa60119a700 1 -- 172.21.15.55:0/3056734777 learned_addr learned my addr 172.21.15.55:0/3056734777 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.647+0000 7fa60199b700 1 --2- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5fc0b16d0 0x7fa5fc1437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.647+0000 7fa60219c700 1 --2- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5fc0b4e70 0x7fa5fc069a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.647+0000 7fa60119a700 1 -- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5fc0b16d0 msgr2=0x7fa5fc1437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.647+0000 7fa60119a700 1 --2- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5fc0b16d0 0x7fa5fc1437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.647+0000 7fa60119a700 1 -- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5fc0b4e70 msgr2=0x7fa5fc069a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.647+0000 7fa60119a700 1 --2- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5fc0b4e70 0x7fa5fc069a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.647+0000 7fa60119a700 1 -- 172.21.15.55:0/3056734777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f4009a20 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.647+0000 7fa60119a700 1 --2- 172.21.15.55:0/3056734777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5fc0b1b40 0x7fa5fc069510 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00c7c0 tx=0x7fa5ec007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.648+0000 7fa5f1ffb700 1 -- 172.21.15.55:0/3056734777 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec005b30 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.648+0000 7fa5f1ffb700 1 -- 172.21.15.55:0/3056734777 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ec005c90 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.648+0000 7fa5f1ffb700 1 -- 172.21.15.55:0/3056734777 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec004a80 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.648+0000 7fa60299d700 1 -- 172.21.15.55:0/3056734777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5fc16b3f0 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.648+0000 7fa60299d700 1 -- 172.21.15.55:0/3056734777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5fc16b9b0 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.649+0000 7fa60299d700 1 -- 172.21.15.55:0/3056734777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5fc16be90 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.650+0000 7fa5f1ffb700 1 -- 172.21.15.55:0/3056734777 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 73) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fa5ec0070d0 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.651+0000 7fa5f1ffb700 1 --2- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa5e00670e0 0x7fa5e0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:03.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.651+0000 7fa5f1ffb700 1 -- 172.21.15.55:0/3056734777 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7fa5ec083fb0 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.651+0000 7fa60199b700 1 -- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa5e00670e0 msgr2=0x7fa5e0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:03.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.651+0000 7fa60199b700 1 --2- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa5e00670e0 0x7fa5e0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:03.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.655+0000 7fa5f1ffb700 1 -- 172.21.15.55:0/3056734777 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5ec04b5a0 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.852+0000 7fa60199b700 1 -- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa5e00670e0 msgr2=0x7fa5e0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:03.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.852+0000 7fa60199b700 1 --2- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa5e00670e0 0x7fa5e0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:03.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.855+0000 7fa60299d700 1 -- 172.21.15.55:0/3056734777 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa5fc0b1f90 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.855+0000 7fa5f1ffb700 1 -- 172.21.15.55:0/3056734777 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v161) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa5ec050b10 con 0x7fa5fc0b1b40 2023-10-06T04:49:03.858 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T04:49:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.858+0000 7fa60299d700 1 -- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa5e00670e0 msgr2=0x7fa5e0069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.858+0000 7fa60299d700 1 --2- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa5e00670e0 0x7fa5e0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.858+0000 7fa60299d700 1 -- 172.21.15.55:0/3056734777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5fc0b1b40 msgr2=0x7fa5fc069510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.858+0000 7fa60299d700 1 --2- 172.21.15.55:0/3056734777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5fc0b1b40 0x7fa5fc069510 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00c7c0 tx=0x7fa5ec007f70 comp rx=0 tx=0).stop 2023-10-06T04:49:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.858+0000 7fa60299d700 1 -- 172.21.15.55:0/3056734777 shutdown_connections 2023-10-06T04:49:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.858+0000 7fa60299d700 1 --2- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa5e00670e0 0x7fa5e0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.858+0000 7fa60299d700 1 --2- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5fc0b4e70 0x7fa5fc069a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.858+0000 7fa60299d700 1 --2- 172.21.15.55:0/3056734777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5fc0b16d0 0x7fa5fc1437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.858+0000 7fa60299d700 1 --2- 172.21.15.55:0/3056734777 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5fc0b1b40 0x7fa5fc069510 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:03.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.858+0000 7fa60299d700 1 -- 172.21.15.55:0/3056734777 >> 172.21.15.55:0/3056734777 conn(0x7fa5fc0d10c0 msgr2=0x7fa5fc0b4640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:03.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.858+0000 7fa60299d700 1 -- 172.21.15.55:0/3056734777 shutdown_connections 2023-10-06T04:49:03.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:03.858+0000 7fa60299d700 1 -- 172.21.15.55:0/3056734777 wait complete. 2023-10-06T04:49:03.880 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:49:04.002 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:04.000+0000 7fecea2cb180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T04:49:04.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.074+0000 7f7aa919d700 1 Processor -- start 2023-10-06T04:49:04.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.074+0000 7f7aa919d700 1 -- start start 2023-10-06T04:49:04.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.075+0000 7f7aa919d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7aa40b4e70 0x7f7aa40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:04.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.075+0000 7f7aa919d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa40cfb00 con 0x7f7aa40b1b40 2023-10-06T04:49:04.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.075+0000 7f7aa919d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa415e060 con 0x7f7aa40b16d0 2023-10-06T04:49:04.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.075+0000 7f7aa919d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa40b5c40 con 0x7f7aa40b4e70 2023-10-06T04:49:04.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.075+0000 7f7aa899c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f7aa40b1b40 0x7f7aa415d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42540/0 (socket says 172.21.15.55:42540) 2023-10-06T04:49:04.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.075+0000 7f7aa899c700 1 -- 172.21.15.55:0/930654882 learned_addr learned my addr 172.21.15.55:0/930654882 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:04.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.075+0000 7f7aa3fff700 1 --2- 172.21.15.55:0/930654882 >> v2:172.21.15.163:3300/0 conn(0x7f7aa40b4e70 0x7f7aa40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:04.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.076+0000 7f7aa2ffd700 1 -- 172.21.15.55:0/930654882 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784477668 0 0) 0x7f7aa40cfb00 con 0x7f7aa40b1b40 2023-10-06T04:49:04.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.076+0000 7f7aa2ffd700 1 -- 172.21.15.55:0/930654882 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a8c003570 con 0x7f7aa40b1b40 2023-10-06T04:49:04.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.076+0000 7f7aa2ffd700 1 -- 172.21.15.55:0/930654882 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733853002 0 0) 0x7f7aa415e060 con 0x7f7aa40b16d0 2023-10-06T04:49:04.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.076+0000 7f7aa2ffd700 1 -- 172.21.15.55:0/930654882 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aa40cfb00 con 0x7f7aa40b16d0 2023-10-06T04:49:04.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.076+0000 7f7aa3fff700 1 -- 172.21.15.55:0/930654882 >> v1:172.21.15.163:6789/0 conn(0x7f7aa40b16d0 legacy=0x7f7aa415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:04.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.076+0000 7f7aa3fff700 1 -- 172.21.15.55:0/930654882 >> v1:172.21.15.55:6789/0 conn(0x7f7aa40b1b40 legacy=0x7f7aa415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:04.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.076+0000 7f7aa3fff700 1 -- 172.21.15.55:0/930654882 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa40b5f20 con 0x7f7aa40b4e70 2023-10-06T04:49:04.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.076+0000 7f7aa3fff700 1 --2- 172.21.15.55:0/930654882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aa40b4e70 0x7f7aa40ccff0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7a940098e0 tx=0x7f7a940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2ad509e365cdbba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:04.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.077+0000 7f7aa2ffd700 1 -- 172.21.15.55:0/930654882 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a94011630 con 0x7f7aa40b4e70 2023-10-06T04:49:04.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.077+0000 7f7aa2ffd700 1 -- 172.21.15.55:0/930654882 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a94011790 con 0x7f7aa40b4e70 2023-10-06T04:49:04.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.077+0000 7f7aa2ffd700 1 -- 172.21.15.55:0/930654882 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a94011b30 con 0x7f7aa40b4e70 2023-10-06T04:49:04.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.077+0000 7f7aa919d700 1 -- 172.21.15.55:0/930654882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aa40b4e70 msgr2=0x7f7aa40ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:04.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.077+0000 7f7aa919d700 1 --2- 172.21.15.55:0/930654882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aa40b4e70 0x7f7aa40ccff0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7a940098e0 tx=0x7f7a940091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.077+0000 7f7aa919d700 1 -- 172.21.15.55:0/930654882 shutdown_connections 2023-10-06T04:49:04.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.077+0000 7f7aa919d700 1 --2- 172.21.15.55:0/930654882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aa40b4e70 0x7f7aa40ccff0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.077+0000 7f7aa919d700 1 -- 172.21.15.55:0/930654882 >> 172.21.15.55:0/930654882 conn(0x7f7aa40d10c0 msgr2=0x7f7aa40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:04.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.077+0000 7f7aa919d700 1 -- 172.21.15.55:0/930654882 shutdown_connections 2023-10-06T04:49:04.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.077+0000 7f7aa919d700 1 -- 172.21.15.55:0/930654882 wait complete. 2023-10-06T04:49:04.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.079+0000 7f7aa919d700 1 Processor -- start 2023-10-06T04:49:04.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.079+0000 7f7aa919d700 1 -- start start 2023-10-06T04:49:04.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.080+0000 7f7aa919d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aa40b16d0 0x7f7aa4157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:04.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.080+0000 7f7aa919d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b1b40 0x7f7aa4169c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:04.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.080+0000 7f7aa919d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aa40b4e70 0x7f7aa40c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:04.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.080+0000 7f7aa919d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40af9c0 con 0x7f7aa40b1b40 2023-10-06T04:49:04.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.080+0000 7f7aa919d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40c8b10 con 0x7f7aa40b16d0 2023-10-06T04:49:04.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.080+0000 7f7aa919d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa40c8df0 con 0x7f7aa40b4e70 2023-10-06T04:49:04.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.080+0000 7f7aa37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b1b40 0x7f7aa4169c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:04.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.080+0000 7f7aa37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b1b40 0x7f7aa4169c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42634/0 (socket says 172.21.15.55:42634) 2023-10-06T04:49:04.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.080+0000 7f7aa37fe700 1 -- 172.21.15.55:0/880333888 learned_addr learned my addr 172.21.15.55:0/880333888 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:04.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.080+0000 7f7aa3fff700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aa40b16d0 0x7f7aa4157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:04.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.080+0000 7f7aa899c700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aa40b4e70 0x7f7aa40c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:04.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.081+0000 7f7aa37fe700 1 -- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aa40b4e70 msgr2=0x7f7aa40c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:04.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.081+0000 7f7aa37fe700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aa40b4e70 0x7f7aa40c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.081+0000 7f7aa37fe700 1 -- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aa40b16d0 msgr2=0x7f7aa4157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:04.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.081+0000 7f7aa37fe700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aa40b16d0 0x7f7aa4157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.081+0000 7f7aa37fe700 1 -- 172.21.15.55:0/880333888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a94009420 con 0x7f7aa40b1b40 2023-10-06T04:49:04.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.081+0000 7f7aa3fff700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aa40b16d0 0x7f7aa4157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:49:04.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.081+0000 7f7aa37fe700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b1b40 0x7f7aa4169c10 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800c790 tx=0x7f7a98007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:04.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.081+0000 7f7a8bfff700 1 -- 172.21.15.55:0/880333888 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98010510 con 0x7f7aa40b1b40 2023-10-06T04:49:04.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.081+0000 7f7a8bfff700 1 -- 172.21.15.55:0/880333888 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a98010670 con 0x7f7aa40b1b40 2023-10-06T04:49:04.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.081+0000 7f7a8bfff700 1 -- 172.21.15.55:0/880333888 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98010a10 con 0x7f7aa40b1b40 2023-10-06T04:49:04.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.081+0000 7f7aa919d700 1 -- 172.21.15.55:0/880333888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa40c90d0 con 0x7f7aa40b1b40 2023-10-06T04:49:04.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.081+0000 7f7aa919d700 1 -- 172.21.15.55:0/880333888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa40c9690 con 0x7f7aa40b1b40 2023-10-06T04:49:04.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.083+0000 7f7aa919d700 1 -- 172.21.15.55:0/880333888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa40c9880 con 0x7f7aa40b1b40 2023-10-06T04:49:04.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.084+0000 7f7a8bfff700 1 -- 172.21.15.55:0/880333888 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 73) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f7a9801b3b0 con 0x7f7aa40b1b40 2023-10-06T04:49:04.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.085+0000 7f7a8bfff700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a900670e0 0x7f7a90069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:04.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.085+0000 7f7a8bfff700 1 -- 172.21.15.55:0/880333888 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f7a98084580 con 0x7f7aa40b1b40 2023-10-06T04:49:04.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.085+0000 7f7aa3fff700 1 -- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a900670e0 msgr2=0x7f7a90069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:04.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.085+0000 7f7aa3fff700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a900670e0 0x7f7a90069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:04.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.089+0000 7f7a8bfff700 1 -- 172.21.15.55:0/880333888 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a980510e0 con 0x7f7aa40b1b40 2023-10-06T04:49:04.097 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:04.096+0000 7fecea2cb180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T04:49:04.152 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:04.150+0000 7fb5daa99180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T04:49:04.187 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:04.186+0000 7fecea2cb180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T04:49:04.252 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:04.249+0000 7fb5daa99180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T04:49:04.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.286+0000 7f7aa3fff700 1 -- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a900670e0 msgr2=0x7f7a90069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:04.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.286+0000 7f7aa3fff700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a900670e0 0x7f7a90069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:04.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.290+0000 7f7aa919d700 1 -- 172.21.15.55:0/880333888 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7aa404bb80 con 0x7f7aa40b1b40 2023-10-06T04:49:04.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.291+0000 7f7a8bfff700 1 -- 172.21.15.55:0/880333888 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v161) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7a98017020 con 0x7f7aa40b1b40 2023-10-06T04:49:04.294 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T04:49:04.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.293+0000 7f7aa919d700 1 -- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a900670e0 msgr2=0x7f7a90069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:04.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.293+0000 7f7aa919d700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a900670e0 0x7f7a90069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.293+0000 7f7aa919d700 1 -- 172.21.15.55:0/880333888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b1b40 msgr2=0x7f7aa4169c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.293+0000 7f7aa919d700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b1b40 0x7f7aa4169c10 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800c790 tx=0x7f7a98007e10 comp rx=0 tx=0).stop 2023-10-06T04:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.293+0000 7f7aa919d700 1 -- 172.21.15.55:0/880333888 shutdown_connections 2023-10-06T04:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.293+0000 7f7aa919d700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7a900670e0 0x7f7a90069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.293+0000 7f7aa919d700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7aa40b4e70 0x7f7aa40c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.293+0000 7f7aa919d700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7aa40b16d0 0x7f7aa4157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.293+0000 7f7aa919d700 1 --2- 172.21.15.55:0/880333888 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7aa40b1b40 0x7f7aa4169c10 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.293+0000 7f7aa919d700 1 -- 172.21.15.55:0/880333888 >> 172.21.15.55:0/880333888 conn(0x7f7aa40d10c0 msgr2=0x7f7aa4159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:04.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.293+0000 7f7aa919d700 1 -- 172.21.15.55:0/880333888 shutdown_connections 2023-10-06T04:49:04.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.293+0000 7f7aa919d700 1 -- 172.21.15.55:0/880333888 wait complete. 2023-10-06T04:49:04.315 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:49:04.349 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:04.347+0000 7fb5daa99180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T04:49:04.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.502+0000 7fb775d9d700 1 Processor -- start 2023-10-06T04:49:04.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.502+0000 7fb775d9d700 1 -- start start 2023-10-06T04:49:04.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.503+0000 7fb775d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb770156310 0x7fb7700c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:04.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.503+0000 7fb775d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb7700ae710 0x7fb7700c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:04.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.503+0000 7fb775d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7700ca860 con 0x7fb7701566e0 2023-10-06T04:49:04.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.503+0000 7fb775d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7700cab40 con 0x7fb7700ae710 2023-10-06T04:49:04.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.503+0000 7fb775d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7700cae20 con 0x7fb770156310 2023-10-06T04:49:04.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.503+0000 7fb76ffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb7701566e0 0x7fb7700ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42550/0 (socket says 172.21.15.55:42550) 2023-10-06T04:49:04.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.503+0000 7fb76ffff700 1 -- 172.21.15.55:0/2775974820 learned_addr learned my addr 172.21.15.55:0/2775974820 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:04.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.504+0000 7fb774d9b700 1 --2- 172.21.15.55:0/2775974820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb770156310 0x7fb7700c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:04.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.504+0000 7fb77559c700 1 --2- 172.21.15.55:0/2775974820 >> v2:172.21.15.163:3300/0 conn(0x7fb7700ae710 0x7fb7700c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:04.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.504+0000 7fb76f7fe700 1 -- 172.21.15.55:0/2775974820 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286002525 0 0) 0x7fb7700ca860 con 0x7fb7701566e0 2023-10-06T04:49:04.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.504+0000 7fb76f7fe700 1 -- 172.21.15.55:0/2775974820 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb760003570 con 0x7fb7701566e0 2023-10-06T04:49:04.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.504+0000 7fb77559c700 1 -- 172.21.15.55:0/2775974820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb770156310 msgr2=0x7fb7700c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:04.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.504+0000 7fb77559c700 1 --2- 172.21.15.55:0/2775974820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb770156310 0x7fb7700c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.504+0000 7fb77559c700 1 -- 172.21.15.55:0/2775974820 >> v1:172.21.15.55:6789/0 conn(0x7fb7701566e0 legacy=0x7fb7700ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:04.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.504+0000 7fb77559c700 1 -- 172.21.15.55:0/2775974820 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7700cb100 con 0x7fb7700ae710 2023-10-06T04:49:04.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.504+0000 7fb77559c700 1 --2- 172.21.15.55:0/2775974820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7700ae710 0x7fb7700c6830 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb768009e60 tx=0x7fb768009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff4cfae430e2ca0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:04.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.505+0000 7fb76f7fe700 1 -- 172.21.15.55:0/2775974820 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76800e070 con 0x7fb7700ae710 2023-10-06T04:49:04.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.505+0000 7fb76f7fe700 1 -- 172.21.15.55:0/2775974820 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76800bab0 con 0x7fb7700ae710 2023-10-06T04:49:04.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.505+0000 7fb76f7fe700 1 -- 172.21.15.55:0/2775974820 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76800be70 con 0x7fb7700ae710 2023-10-06T04:49:04.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.505+0000 7fb775d9d700 1 -- 172.21.15.55:0/2775974820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7700ae710 msgr2=0x7fb7700c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:04.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.505+0000 7fb775d9d700 1 --2- 172.21.15.55:0/2775974820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7700ae710 0x7fb7700c6830 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb768009e60 tx=0x7fb768009780 comp rx=0 tx=0).stop 2023-10-06T04:49:04.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.505+0000 7fb775d9d700 1 -- 172.21.15.55:0/2775974820 shutdown_connections 2023-10-06T04:49:04.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.505+0000 7fb775d9d700 1 --2- 172.21.15.55:0/2775974820 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7700ae710 0x7fb7700c6830 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.505+0000 7fb775d9d700 1 --2- 172.21.15.55:0/2775974820 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb770156310 0x7fb7700c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.505+0000 7fb775d9d700 1 -- 172.21.15.55:0/2775974820 >> 172.21.15.55:0/2775974820 conn(0x7fb7700cc1f0 msgr2=0x7fb7700cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:04.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.505+0000 7fb775d9d700 1 -- 172.21.15.55:0/2775974820 shutdown_connections 2023-10-06T04:49:04.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.506+0000 7fb775d9d700 1 -- 172.21.15.55:0/2775974820 wait complete. 2023-10-06T04:49:04.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.507+0000 7fb775d9d700 1 Processor -- start 2023-10-06T04:49:04.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.507+0000 7fb775d9d700 1 -- start start 2023-10-06T04:49:04.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.508+0000 7fb775d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7700ae710 0x7fb7701485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:04.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.508+0000 7fb775d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb770156310 0x7fb7701594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:04.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.508+0000 7fb775d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7701566e0 0x7fb770156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:04.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.508+0000 7fb775d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77015af80 con 0x7fb7700ae710 2023-10-06T04:49:04.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.508+0000 7fb775d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb770148b20 con 0x7fb7701566e0 2023-10-06T04:49:04.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.508+0000 7fb775d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb770159ee0 con 0x7fb770156310 2023-10-06T04:49:04.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.508+0000 7fb774d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7700ae710 0x7fb7701485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:04.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.508+0000 7fb774d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7700ae710 0x7fb7701485e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42648/0 (socket says 172.21.15.55:42648) 2023-10-06T04:49:04.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.508+0000 7fb774d9b700 1 -- 172.21.15.55:0/1253994883 learned_addr learned my addr 172.21.15.55:0/1253994883 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:04.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.508+0000 7fb77559c700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7701566e0 0x7fb770156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:04.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.508+0000 7fb76ffff700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb770156310 0x7fb7701594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:04.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.509+0000 7fb774d9b700 1 -- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb770156310 msgr2=0x7fb7701594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:04.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.509+0000 7fb774d9b700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb770156310 0x7fb7701594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.509+0000 7fb774d9b700 1 -- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7701566e0 msgr2=0x7fb770156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:04.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.509+0000 7fb774d9b700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7701566e0 0x7fb770156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.509+0000 7fb774d9b700 1 -- 172.21.15.55:0/1253994883 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb768009a20 con 0x7fb7700ae710 2023-10-06T04:49:04.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.509+0000 7fb76ffff700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb770156310 0x7fb7701594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:49:04.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.509+0000 7fb774d9b700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7700ae710 0x7fb7701485e0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fb76400ec80 tx=0x7fb7640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:04.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.509+0000 7fb76cff9700 1 -- 172.21.15.55:0/1253994883 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76400c850 con 0x7fb7700ae710 2023-10-06T04:49:04.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.509+0000 7fb76cff9700 1 -- 172.21.15.55:0/1253994883 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76400c9b0 con 0x7fb7700ae710 2023-10-06T04:49:04.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.509+0000 7fb775d9d700 1 -- 172.21.15.55:0/1253994883 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb77015a1c0 con 0x7fb7700ae710 2023-10-06T04:49:04.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.509+0000 7fb76cff9700 1 -- 172.21.15.55:0/1253994883 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76400cd50 con 0x7fb7700ae710 2023-10-06T04:49:04.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.509+0000 7fb775d9d700 1 -- 172.21.15.55:0/1253994883 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77015a780 con 0x7fb7700ae710 2023-10-06T04:49:04.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.510+0000 7fb775d9d700 1 -- 172.21.15.55:0/1253994883 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77014a910 con 0x7fb7700ae710 2023-10-06T04:49:04.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.511+0000 7fb76cff9700 1 -- 172.21.15.55:0/1253994883 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 73) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fb764013680 con 0x7fb7700ae710 2023-10-06T04:49:04.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.512+0000 7fb76cff9700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb7540670e0 0x7fb754069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:04.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.512+0000 7fb76cff9700 1 -- 172.21.15.55:0/1253994883 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7fb764084fa0 con 0x7fb7700ae710 2023-10-06T04:49:04.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.512+0000 7fb76ffff700 1 -- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb7540670e0 msgr2=0x7fb754069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:04.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.513+0000 7fb76ffff700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb7540670e0 0x7fb754069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:04.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.519+0000 7fb76cff9700 1 -- 172.21.15.55:0/1253994883 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb764051b00 con 0x7fb7700ae710 2023-10-06T04:49:04.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.714+0000 7fb76ffff700 1 -- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb7540670e0 msgr2=0x7fb754069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:04.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.714+0000 7fb76ffff700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb7540670e0 0x7fb754069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:04.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.770+0000 7fb775d9d700 1 -- 172.21.15.55:0/1253994883 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb77004c3c0 con 0x7fb7700ae710 2023-10-06T04:49:04.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.771+0000 7fb76cff9700 1 -- 172.21.15.55:0/1253994883 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fb764055120 con 0x7fb7700ae710 2023-10-06T04:49:04.774 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:04.774 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8308,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/421768289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":421768289},{"type":"v1","addr":"172.21.15.55:6835","nonce":421768289}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8311,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2920562647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2920562647},{"type":"v1","addr":"172.21.15.163:6834","nonce":2920562647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8314,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2324769448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2324769448},{"type":"v1","addr":"172.21.15.163:6835","nonce":2324769448}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:48:42.309816+0000","modified":"2023-10-06T04:48:45.840578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3504492616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3504492616},{"type":"v1","addr":"172.21.15.55:6834","nonce":3504492616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T04:49:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.774+0000 7fb775d9d700 1 -- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb7540670e0 msgr2=0x7fb754069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.774+0000 7fb775d9d700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb7540670e0 0x7fb754069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.774+0000 7fb775d9d700 1 -- 172.21.15.55:0/1253994883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7700ae710 msgr2=0x7fb7701485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.774+0000 7fb775d9d700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7700ae710 0x7fb7701485e0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fb76400ec80 tx=0x7fb7640091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.774+0000 7fb775d9d700 1 -- 172.21.15.55:0/1253994883 shutdown_connections 2023-10-06T04:49:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.774+0000 7fb775d9d700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb7540670e0 0x7fb754069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.774+0000 7fb775d9d700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb7701566e0 0x7fb770156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.774+0000 7fb775d9d700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb7700ae710 0x7fb7701485e0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.774+0000 7fb775d9d700 1 --2- 172.21.15.55:0/1253994883 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb770156310 0x7fb7701594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:04.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.774+0000 7fb775d9d700 1 -- 172.21.15.55:0/1253994883 >> 172.21.15.55:0/1253994883 conn(0x7fb7700cc1f0 msgr2=0x7fb7700c2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:04.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.774+0000 7fb775d9d700 1 -- 172.21.15.55:0/1253994883 shutdown_connections 2023-10-06T04:49:04.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:04.774+0000 7fb775d9d700 1 -- 172.21.15.55:0/1253994883 wait complete. 2023-10-06T04:49:04.779 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 78 2023-10-06T04:49:04.795 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:49:04.813 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:04.812+0000 7fecea2cb180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T04:49:04.986 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:04.984+0000 7fecea2cb180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:49:05.013 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:05.010+0000 7fb5daa99180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T04:49:05.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.056+0000 7f6143fff700 1 Processor -- start 2023-10-06T04:49:05.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.056+0000 7f6143fff700 1 -- start start 2023-10-06T04:49:05.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.057+0000 7f6143fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f61440c8830 0x7f6144164800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.057+0000 7f6143fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61440b1410 con 0x7f61441595a0 2023-10-06T04:49:05.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.057+0000 7f6143fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6144063230 con 0x7f61440c6c60 2023-10-06T04:49:05.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.057+0000 7f6143fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61440b2b90 con 0x7f61440c8830 2023-10-06T04:49:05.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.057+0000 7f6142ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f61441595a0 0x7f61440b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42552/0 (socket says 172.21.15.55:42552) 2023-10-06T04:49:05.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.057+0000 7f6142ffd700 1 -- 172.21.15.55:0/3060483056 learned_addr learned my addr 172.21.15.55:0/3060483056 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:05.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.057+0000 7f61437fe700 1 --2- 172.21.15.55:0/3060483056 >> v2:172.21.15.55:3300/0 conn(0x7f61440c8830 0x7f6144164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:05.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.058+0000 7f6141ffb700 1 -- 172.21.15.55:0/3060483056 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735118035 0 0) 0x7f61440b1410 con 0x7f61441595a0 2023-10-06T04:49:05.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.058+0000 7f61437fe700 1 -- 172.21.15.55:0/3060483056 >> v1:172.21.15.163:6789/0 conn(0x7f61440c6c60 legacy=0x7f61440c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.058+0000 7f61437fe700 1 -- 172.21.15.55:0/3060483056 >> v1:172.21.15.55:6789/0 conn(0x7f61441595a0 legacy=0x7f61440b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.058+0000 7f61437fe700 1 -- 172.21.15.55:0/3060483056 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61440634f0 con 0x7f61440c8830 2023-10-06T04:49:05.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.058+0000 7f61437fe700 1 --2- 172.21.15.55:0/3060483056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61440c8830 0x7f6144164800 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f613c009de0 tx=0x7f613c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5414eee3082cff server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:05.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.058+0000 7f6141ffb700 1 -- 172.21.15.55:0/3060483056 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c00e070 con 0x7f61440c8830 2023-10-06T04:49:05.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.058+0000 7f6141ffb700 1 -- 172.21.15.55:0/3060483056 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613c00bbb0 con 0x7f61440c8830 2023-10-06T04:49:05.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.058+0000 7f6141ffb700 1 -- 172.21.15.55:0/3060483056 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613c012430 con 0x7f61440c8830 2023-10-06T04:49:05.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.059+0000 7f6143fff700 1 -- 172.21.15.55:0/3060483056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61440c8830 msgr2=0x7f6144164800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.059+0000 7f6143fff700 1 --2- 172.21.15.55:0/3060483056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61440c8830 0x7f6144164800 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f613c009de0 tx=0x7f613c009780 comp rx=0 tx=0).stop 2023-10-06T04:49:05.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.059+0000 7f6143fff700 1 -- 172.21.15.55:0/3060483056 shutdown_connections 2023-10-06T04:49:05.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.059+0000 7f6143fff700 1 --2- 172.21.15.55:0/3060483056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61440c8830 0x7f6144164800 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.059+0000 7f6143fff700 1 -- 172.21.15.55:0/3060483056 >> 172.21.15.55:0/3060483056 conn(0x7f614406ddb0 msgr2=0x7f614406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:05.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.059+0000 7f6143fff700 1 -- 172.21.15.55:0/3060483056 shutdown_connections 2023-10-06T04:49:05.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.059+0000 7f6143fff700 1 -- 172.21.15.55:0/3060483056 wait complete. 2023-10-06T04:49:05.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.061+0000 7f6143fff700 1 Processor -- start 2023-10-06T04:49:05.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.061+0000 7f6143fff700 1 -- start start 2023-10-06T04:49:05.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.061+0000 7f6143fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61440c6c60 0x7f614415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f6143fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61440c8830 0x7f6144150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f6143fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61441595a0 0x7f614413d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f6143fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61441518f0 con 0x7f61440c8830 2023-10-06T04:49:05.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f6143fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614414bfd0 con 0x7f61440c6c60 2023-10-06T04:49:05.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f6143fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6144151340 con 0x7f61441595a0 2023-10-06T04:49:05.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f61427fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61440c8830 0x7f6144150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:05.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f61427fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61440c8830 0x7f6144150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42664/0 (socket says 172.21.15.55:42664) 2023-10-06T04:49:05.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f61427fc700 1 -- 172.21.15.55:0/2817676066 learned_addr learned my addr 172.21.15.55:0/2817676066 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:05.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f61437fe700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61441595a0 0x7f614413d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:05.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f6142ffd700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61440c6c60 0x7f614415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:05.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f61427fc700 1 -- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61441595a0 msgr2=0x7f614413d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f61427fc700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61441595a0 0x7f614413d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f61427fc700 1 -- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61440c6c60 msgr2=0x7f614415a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f61427fc700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61440c6c60 0x7f614415a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f61427fc700 1 -- 172.21.15.55:0/2817676066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c009a20 con 0x7f61440c8830 2023-10-06T04:49:05.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.062+0000 7f61437fe700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61441595a0 0x7f614413d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:49:05.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.063+0000 7f61427fc700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61440c8830 0x7f6144150e00 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f613800c790 tx=0x7f6138007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:05.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.063+0000 7f612affd700 1 -- 172.21.15.55:0/2817676066 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138001ea0 con 0x7f61440c8830 2023-10-06T04:49:05.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.063+0000 7f612affd700 1 -- 172.21.15.55:0/2817676066 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6138011590 con 0x7f61440c8830 2023-10-06T04:49:05.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.063+0000 7f612affd700 1 -- 172.21.15.55:0/2817676066 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138011950 con 0x7f61440c8830 2023-10-06T04:49:05.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.063+0000 7f6143fff700 1 -- 172.21.15.55:0/2817676066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6144151610 con 0x7f61440c8830 2023-10-06T04:49:05.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.063+0000 7f6143fff700 1 -- 172.21.15.55:0/2817676066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614414bc60 con 0x7f61440c8830 2023-10-06T04:49:05.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.064+0000 7f6143fff700 1 -- 172.21.15.55:0/2817676066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f614414b7e0 con 0x7f61440c8830 2023-10-06T04:49:05.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.065+0000 7f612affd700 1 -- 172.21.15.55:0/2817676066 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 73) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f6138011d00 con 0x7f61440c8830 2023-10-06T04:49:05.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.066+0000 7f612affd700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f61300670e0 0x7f6130069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.066+0000 7f612affd700 1 -- 172.21.15.55:0/2817676066 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f61380841a0 con 0x7f61440c8830 2023-10-06T04:49:05.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.067+0000 7f6142ffd700 1 -- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f61300670e0 msgr2=0x7f6130069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:05.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.067+0000 7f6142ffd700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f61300670e0 0x7f6130069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:05.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.070+0000 7f612affd700 1 -- 172.21.15.55:0/2817676066 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f614414b7e0 con 0x7f61440c8830 2023-10-06T04:49:05.082 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:05.081+0000 7fecea2cb180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T04:49:05.195 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:05.192+0000 7fb5daa99180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:49:05.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.268+0000 7f6142ffd700 1 -- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f61300670e0 msgr2=0x7f6130069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:05.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.268+0000 7f6142ffd700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f61300670e0 0x7f6130069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:05.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.277+0000 7f6143fff700 1 -- 172.21.15.55:0/2817676066 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f614414b7e0 con 0x7f61440c8830 2023-10-06T04:49:05.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.278+0000 7f612affd700 1 -- 172.21.15.55:0/2817676066 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+17896 (secure 0 0 0) 0x7f614414b7e0 con 0x7f61440c8830 2023-10-06T04:49:05.281 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:05.281 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":161,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T04:48:58.128628+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:48:40.391644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T04:48:41.276409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3452722174":"2023-10-07T04:48:58.128595+0000","172.21.15.163:6837/85191":"2023-10-07T04:48:58.128595+0000","172.21.15.163:6836/85191":"2023-10-07T04:48:58.128595+0000","172.21.15.163:0/1904936092":"2023-10-07T04:48:58.128595+0000","172.21.15.163:0/3499125134":"2023-10-07T04:48:58.128595+0000","172.21.15.163:0/2488193308":"2023-10-07T04:48:58.128595+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:49:05.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.280+0000 7f6143fff700 1 -- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f61300670e0 msgr2=0x7f6130069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:05.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.280+0000 7f6143fff700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f61300670e0 0x7f6130069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.280+0000 7f6143fff700 1 -- 172.21.15.55:0/2817676066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61440c8830 msgr2=0x7f6144150e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.280+0000 7f6143fff700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61440c8830 0x7f6144150e00 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f613800c790 tx=0x7f6138007e30 comp rx=0 tx=0).stop 2023-10-06T04:49:05.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.280+0000 7f6143fff700 1 -- 172.21.15.55:0/2817676066 shutdown_connections 2023-10-06T04:49:05.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.280+0000 7f6143fff700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f61300670e0 0x7f6130069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.280+0000 7f6143fff700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61441595a0 0x7f614413d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.280+0000 7f6143fff700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61440c6c60 0x7f614415a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.280+0000 7f6143fff700 1 --2- 172.21.15.55:0/2817676066 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61440c8830 0x7f6144150e00 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.280+0000 7f6143fff700 1 -- 172.21.15.55:0/2817676066 >> 172.21.15.55:0/2817676066 conn(0x7f614406ddb0 msgr2=0x7f6144157240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:05.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.280+0000 7f6143fff700 1 -- 172.21.15.55:0/2817676066 shutdown_connections 2023-10-06T04:49:05.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.280+0000 7f6143fff700 1 -- 172.21.15.55:0/2817676066 wait complete. 2023-10-06T04:49:05.297 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:05.295+0000 7fb5daa99180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T04:49:05.301 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T04:49:05.301 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:49:05.379 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:05.378+0000 7fecea2cb180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T04:49:05.475 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:05.474+0000 7fecea2cb180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T04:49:05.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.494+0000 7fac1c9fe700 1 Processor -- start 2023-10-06T04:49:05.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.494+0000 7fac1c9fe700 1 -- start start 2023-10-06T04:49:05.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.495+0000 7fac1c9fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac18142ea0 0x7fac180b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.495+0000 7fac1c9fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fac1806c0f0 0x7fac1806a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.495+0000 7fac1c9fe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac180b40f0 con 0x7fac180ad870 2023-10-06T04:49:05.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.495+0000 7fac1c9fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac18069640 con 0x7fac1806c0f0 2023-10-06T04:49:05.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.495+0000 7fac1c9fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac18069920 con 0x7fac18142ea0 2023-10-06T04:49:05.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.495+0000 7fac16d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac18142ea0 0x7fac180b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:05.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.496+0000 7fac1759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fac1806c0f0 0x7fac1806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:05.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.496+0000 7fac16d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac18142ea0 0x7fac180b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:37254/0 (socket says 172.21.15.55:37254) 2023-10-06T04:49:05.499 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.496+0000 7fac16d9d700 1 -- 172.21.15.55:0/360364039 learned_addr learned my addr 172.21.15.55:0/360364039 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:05.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.496+0000 7fac1759e700 1 -- 172.21.15.55:0/360364039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac18142ea0 msgr2=0x7fac180b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:05.500 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.496+0000 7fac1759e700 1 --2- 172.21.15.55:0/360364039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac18142ea0 0x7fac180b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.496+0000 7fac1759e700 1 -- 172.21.15.55:0/360364039 >> v1:172.21.15.163:6789/0 conn(0x7fac180ad870 legacy=0x7fac1806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.496+0000 7fac1759e700 1 -- 172.21.15.55:0/360364039 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac18069c00 con 0x7fac1806c0f0 2023-10-06T04:49:05.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.496+0000 7fac15d9b700 1 -- 172.21.15.55:0/360364039 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962463762 0 0) 0x7fac180b40f0 con 0x7fac180ad870 2023-10-06T04:49:05.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.496+0000 7fac16d9d700 1 --2- 172.21.15.55:0/360364039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac18142ea0 0x7fac180b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:49:05.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.497+0000 7fac1759e700 1 --2- 172.21.15.55:0/360364039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1806c0f0 0x7fac1806a120 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fac10009e60 tx=0x7fac10009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4b2af8eab8077d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:05.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.497+0000 7fac15d9b700 1 -- 172.21.15.55:0/360364039 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1000e070 con 0x7fac1806c0f0 2023-10-06T04:49:05.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.497+0000 7fac15d9b700 1 -- 172.21.15.55:0/360364039 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1000bb60 con 0x7fac1806c0f0 2023-10-06T04:49:05.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.497+0000 7fac15d9b700 1 -- 172.21.15.55:0/360364039 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10012430 con 0x7fac1806c0f0 2023-10-06T04:49:05.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.497+0000 7fac1c9fe700 1 -- 172.21.15.55:0/360364039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1806c0f0 msgr2=0x7fac1806a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.497+0000 7fac1c9fe700 1 --2- 172.21.15.55:0/360364039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1806c0f0 0x7fac1806a120 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fac10009e60 tx=0x7fac10009780 comp rx=0 tx=0).stop 2023-10-06T04:49:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.498+0000 7fac1c9fe700 1 -- 172.21.15.55:0/360364039 shutdown_connections 2023-10-06T04:49:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.498+0000 7fac1c9fe700 1 --2- 172.21.15.55:0/360364039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac18142ea0 0x7fac180b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.498+0000 7fac1c9fe700 1 --2- 172.21.15.55:0/360364039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac1806c0f0 0x7fac1806a120 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.498+0000 7fac1c9fe700 1 -- 172.21.15.55:0/360364039 >> 172.21.15.55:0/360364039 conn(0x7fac180ce430 msgr2=0x7fac180ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.498+0000 7fac1c9fe700 1 -- 172.21.15.55:0/360364039 shutdown_connections 2023-10-06T04:49:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.498+0000 7fac1c9fe700 1 -- 172.21.15.55:0/360364039 wait complete. 2023-10-06T04:49:05.502 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.499+0000 7fac1c9fe700 1 Processor -- start 2023-10-06T04:49:05.503 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.499+0000 7fac1c9fe700 1 -- start start 2023-10-06T04:49:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.500+0000 7fac1c9fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1806c0f0 0x7fac1813f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.500+0000 7fac1c9fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac18142ea0 0x7fac1815ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.500+0000 7fac1c9fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1815b060 0x7fac1806a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.500+0000 7fac1c9fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1815b4e0 con 0x7fac1806c0f0 2023-10-06T04:49:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.500+0000 7fac1c9fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1815ebb0 con 0x7fac18142ea0 2023-10-06T04:49:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.500+0000 7fac1c9fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac180c8090 con 0x7fac1815b060 2023-10-06T04:49:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.501+0000 7fac16d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1806c0f0 0x7fac1813f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.501+0000 7fac16d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1806c0f0 0x7fac1813f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42678/0 (socket says 172.21.15.55:42678) 2023-10-06T04:49:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.501+0000 7fac16d9d700 1 -- 172.21.15.55:0/1558678150 learned_addr learned my addr 172.21.15.55:0/1558678150 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.501+0000 7fac1759e700 1 --2- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1815b060 0x7fac1806a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.501+0000 7fac1659c700 1 --2- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac18142ea0 0x7fac1815ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.501+0000 7fac16d9d700 1 -- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1815b060 msgr2=0x7fac1806a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.501+0000 7fac16d9d700 1 --2- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1815b060 0x7fac1806a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.501+0000 7fac16d9d700 1 -- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac18142ea0 msgr2=0x7fac1815ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.501+0000 7fac16d9d700 1 --2- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac18142ea0 0x7fac1815ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.501+0000 7fac16d9d700 1 -- 172.21.15.55:0/1558678150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac080094f0 con 0x7fac1806c0f0 2023-10-06T04:49:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.502+0000 7fac16d9d700 1 --2- 172.21.15.55:0/1558678150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1806c0f0 0x7fac1813f330 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fac0800ec80 tx=0x7fac08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.502+0000 7fabfeffd700 1 -- 172.21.15.55:0/1558678150 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0800c850 con 0x7fac1806c0f0 2023-10-06T04:49:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.502+0000 7fabfeffd700 1 -- 172.21.15.55:0/1558678150 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac0800c9b0 con 0x7fac1806c0f0 2023-10-06T04:49:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.502+0000 7fabfeffd700 1 -- 172.21.15.55:0/1558678150 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0800cd30 con 0x7fac1806c0f0 2023-10-06T04:49:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.502+0000 7fac1c9fe700 1 -- 172.21.15.55:0/1558678150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac10009a20 con 0x7fac1806c0f0 2023-10-06T04:49:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.502+0000 7fac1c9fe700 1 -- 172.21.15.55:0/1558678150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac180c8520 con 0x7fac1806c0f0 2023-10-06T04:49:05.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.503+0000 7fac1c9fe700 1 -- 172.21.15.55:0/1558678150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac1806b4f0 con 0x7fac1806c0f0 2023-10-06T04:49:05.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.504+0000 7fabfeffd700 1 -- 172.21.15.55:0/1558678150 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 73) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fac08013680 con 0x7fac1806c0f0 2023-10-06T04:49:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.505+0000 7fabfeffd700 1 --2- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac040670e0 0x7fac04069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.505+0000 7fabfeffd700 1 -- 172.21.15.55:0/1558678150 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7fac08084e80 con 0x7fac1806c0f0 2023-10-06T04:49:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.505+0000 7fac1659c700 1 -- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac040670e0 msgr2=0x7fac04069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.505+0000 7fac1659c700 1 --2- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac040670e0 0x7fac04069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:05.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.509+0000 7fabfeffd700 1 -- 172.21.15.55:0/1558678150 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac08051a80 con 0x7fac1806c0f0 2023-10-06T04:49:05.605 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:05.602+0000 7fb5daa99180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T04:49:05.705 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:05.703+0000 7fb5daa99180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T04:49:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.706+0000 7fac1659c700 1 -- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac040670e0 msgr2=0x7fac04069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.706+0000 7fac1659c700 1 --2- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac040670e0 0x7fac04069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:05.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.756+0000 7fac1c9fe700 1 -- 172.21.15.55:0/1558678150 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac1804bbe0 con 0x7fac1806c0f0 2023-10-06T04:49:05.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.757+0000 7fabfeffd700 1 -- 172.21.15.55:0/1558678150 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4602 (secure 0 0 0) 0x7fac080550a0 con 0x7fac1806c0f0 2023-10-06T04:49:05.760 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:05.760 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8308,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/421768289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":421768289},{"type":"v1","addr":"172.21.15.55:6835","nonce":421768289}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8311,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2920562647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2920562647},{"type":"v1","addr":"172.21.15.163:6834","nonce":2920562647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8314,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2324769448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2324769448},{"type":"v1","addr":"172.21.15.163:6835","nonce":2324769448}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:48:42.309816+0000","modified":"2023-10-06T04:48:45.840578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3504492616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3504492616},{"type":"v1","addr":"172.21.15.55:6834","nonce":3504492616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T04:49:05.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.759+0000 7fac1c9fe700 1 -- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac040670e0 msgr2=0x7fac04069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:05.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.759+0000 7fac1c9fe700 1 --2- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac040670e0 0x7fac04069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.759+0000 7fac1c9fe700 1 -- 172.21.15.55:0/1558678150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1806c0f0 msgr2=0x7fac1813f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.759+0000 7fac1c9fe700 1 --2- 172.21.15.55:0/1558678150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1806c0f0 0x7fac1813f330 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fac0800ec80 tx=0x7fac08009250 comp rx=0 tx=0).stop 2023-10-06T04:49:05.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.760+0000 7fac1c9fe700 1 -- 172.21.15.55:0/1558678150 shutdown_connections 2023-10-06T04:49:05.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.760+0000 7fac1c9fe700 1 --2- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac040670e0 0x7fac04069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.760+0000 7fac1c9fe700 1 --2- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac1815b060 0x7fac1806a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.760+0000 7fac1c9fe700 1 --2- 172.21.15.55:0/1558678150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac1806c0f0 0x7fac1813f330 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.760+0000 7fac1c9fe700 1 --2- 172.21.15.55:0/1558678150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac18142ea0 0x7fac1815ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.760+0000 7fac1c9fe700 1 -- 172.21.15.55:0/1558678150 >> 172.21.15.55:0/1558678150 conn(0x7fac180ce430 msgr2=0x7fac180caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:05.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.760+0000 7fac1c9fe700 1 -- 172.21.15.55:0/1558678150 shutdown_connections 2023-10-06T04:49:05.764 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.760+0000 7fac1c9fe700 1 -- 172.21.15.55:0/1558678150 wait complete. 2023-10-06T04:49:05.764 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 78 2023-10-06T04:49:05.781 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:49:05.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.971+0000 7f68953ff700 1 Processor -- start 2023-10-06T04:49:05.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.971+0000 7f68953ff700 1 -- start start 2023-10-06T04:49:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.972+0000 7f68953ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f68900cbf30 0x7f68900ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.972+0000 7f68953ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68900ccf90 0x7f68900b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.972+0000 7f68953ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68900cc300 con 0x7f689015a270 2023-10-06T04:49:05.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.972+0000 7f68953ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6890156bf0 con 0x7f68900cbf30 2023-10-06T04:49:05.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.972+0000 7f68953ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68900ae950 con 0x7f68900ccf90 2023-10-06T04:49:05.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.973+0000 7f688f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f689015a270 0x7f68900c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42566/0 (socket says 172.21.15.55:42566) 2023-10-06T04:49:05.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.973+0000 7f688f7fe700 1 -- 172.21.15.55:0/703376233 learned_addr learned my addr 172.21.15.55:0/703376233 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:05.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.973+0000 7f688e7fc700 1 --2- 172.21.15.55:0/703376233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68900ccf90 0x7f68900b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:05.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.973+0000 7f688effd700 1 --2- 172.21.15.55:0/703376233 >> v2:172.21.15.163:3300/0 conn(0x7f68900cbf30 0x7f68900ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.973+0000 7f688dffb700 1 -- 172.21.15.55:0/703376233 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55766307 0 0) 0x7f68900cc300 con 0x7f689015a270 2023-10-06T04:49:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.973+0000 7f688dffb700 1 -- 172.21.15.55:0/703376233 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6878003570 con 0x7f689015a270 2023-10-06T04:49:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.973+0000 7f688e7fc700 1 -- 172.21.15.55:0/703376233 >> v2:172.21.15.163:3300/0 conn(0x7f68900cbf30 msgr2=0x7f68900ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.973+0000 7f688e7fc700 1 --2- 172.21.15.55:0/703376233 >> v2:172.21.15.163:3300/0 conn(0x7f68900cbf30 0x7f68900ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.973+0000 7f688e7fc700 1 -- 172.21.15.55:0/703376233 >> v1:172.21.15.55:6789/0 conn(0x7f689015a270 legacy=0x7f68900c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.973+0000 7f688e7fc700 1 -- 172.21.15.55:0/703376233 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68900ad370 con 0x7f68900ccf90 2023-10-06T04:49:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.973+0000 7f688effd700 1 --2- 172.21.15.55:0/703376233 >> v2:172.21.15.163:3300/0 conn(0x7f68900cbf30 0x7f68900ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:49:05.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.974+0000 7f688e7fc700 1 --2- 172.21.15.55:0/703376233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68900ccf90 0x7f68900b0d10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f68840098e0 tx=0x7f68840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff2ff62b4edee74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.974+0000 7f688dffb700 1 -- 172.21.15.55:0/703376233 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884011630 con 0x7f68900ccf90 2023-10-06T04:49:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.974+0000 7f688dffb700 1 -- 172.21.15.55:0/703376233 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6884011790 con 0x7f68900ccf90 2023-10-06T04:49:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.974+0000 7f688dffb700 1 -- 172.21.15.55:0/703376233 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884011b30 con 0x7f68900ccf90 2023-10-06T04:49:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.974+0000 7f68953ff700 1 -- 172.21.15.55:0/703376233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68900ccf90 msgr2=0x7f68900b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.974+0000 7f68953ff700 1 --2- 172.21.15.55:0/703376233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68900ccf90 0x7f68900b0d10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f68840098e0 tx=0x7f68840091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.975+0000 7f68953ff700 1 -- 172.21.15.55:0/703376233 shutdown_connections 2023-10-06T04:49:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.975+0000 7f68953ff700 1 --2- 172.21.15.55:0/703376233 >> v2:172.21.15.163:3300/0 conn(0x7f68900cbf30 0x7f68900ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.975+0000 7f68953ff700 1 --2- 172.21.15.55:0/703376233 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68900ccf90 0x7f68900b0d10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.975+0000 7f68953ff700 1 -- 172.21.15.55:0/703376233 >> 172.21.15.55:0/703376233 conn(0x7f689006a320 msgr2=0x7f68900cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.975+0000 7f68953ff700 1 -- 172.21.15.55:0/703376233 shutdown_connections 2023-10-06T04:49:05.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.975+0000 7f68953ff700 1 -- 172.21.15.55:0/703376233 wait complete. 2023-10-06T04:49:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.976+0000 7f68953ff700 1 Processor -- start 2023-10-06T04:49:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.976+0000 7f68953ff700 1 -- start start 2023-10-06T04:49:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.977+0000 7f68953ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68900cbf30 0x7f6890161800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.977+0000 7f68953ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68900ccf90 0x7f689016ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.977+0000 7f68953ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f689015a270 0x7f689016bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.977+0000 7f68953ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68901543a0 con 0x7f68900cbf30 2023-10-06T04:49:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.977+0000 7f68953ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689014f3f0 con 0x7f68900ccf90 2023-10-06T04:49:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.977+0000 7f68953ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6890153790 con 0x7f689015a270 2023-10-06T04:49:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.977+0000 7f688effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68900cbf30 0x7f6890161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:05.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.977+0000 7f688f7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f689015a270 0x7f689016bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:05.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.977+0000 7f688effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68900cbf30 0x7f6890161800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42682/0 (socket says 172.21.15.55:42682) 2023-10-06T04:49:05.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.977+0000 7f688effd700 1 -- 172.21.15.55:0/2872097417 learned_addr learned my addr 172.21.15.55:0/2872097417 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:05.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.977+0000 7f688e7fc700 1 --2- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68900ccf90 0x7f689016ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:05.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.978+0000 7f688effd700 1 -- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f689015a270 msgr2=0x7f689016bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.978+0000 7f688effd700 1 --2- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f689015a270 0x7f689016bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.978+0000 7f688effd700 1 -- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68900ccf90 msgr2=0x7f689016ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:05.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.978+0000 7f688effd700 1 --2- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68900ccf90 0x7f689016ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:05.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.978+0000 7f688effd700 1 -- 172.21.15.55:0/2872097417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6880009420 con 0x7f68900cbf30 2023-10-06T04:49:05.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.978+0000 7f688effd700 1 --2- 172.21.15.55:0/2872097417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68900cbf30 0x7f6890161800 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f688000ec80 tx=0x7f68800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:05.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.978+0000 7f6876ffd700 1 -- 172.21.15.55:0/2872097417 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688000c920 con 0x7f68900cbf30 2023-10-06T04:49:05.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.979+0000 7f6876ffd700 1 -- 172.21.15.55:0/2872097417 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688000ca80 con 0x7f68900cbf30 2023-10-06T04:49:05.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.979+0000 7f6876ffd700 1 -- 172.21.15.55:0/2872097417 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688000ce00 con 0x7f68900cbf30 2023-10-06T04:49:05.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.979+0000 7f68953ff700 1 -- 172.21.15.55:0/2872097417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6884009420 con 0x7f68900cbf30 2023-10-06T04:49:05.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.979+0000 7f68953ff700 1 -- 172.21.15.55:0/2872097417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6890153d50 con 0x7f68900cbf30 2023-10-06T04:49:05.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.980+0000 7f68953ff700 1 -- 172.21.15.55:0/2872097417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689004c3c0 con 0x7f68900cbf30 2023-10-06T04:49:05.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.981+0000 7f6876ffd700 1 -- 172.21.15.55:0/2872097417 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 73) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f6880013680 con 0x7f68900cbf30 2023-10-06T04:49:05.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.982+0000 7f6876ffd700 1 --2- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f687c0670e0 0x7f687c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:05.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.982+0000 7f6876ffd700 1 -- 172.21.15.55:0/2872097417 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f6880085330 con 0x7f68900cbf30 2023-10-06T04:49:05.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.982+0000 7f688e7fc700 1 -- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f687c0670e0 msgr2=0x7f687c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:05.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.982+0000 7f688e7fc700 1 --2- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f687c0670e0 0x7f687c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:05.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:05.986+0000 7f6876ffd700 1 -- 172.21.15.55:0/2872097417 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6880051f30 con 0x7f68900cbf30 2023-10-06T04:49:06.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.183+0000 7f688e7fc700 1 -- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f687c0670e0 msgr2=0x7f687c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:06.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.183+0000 7f688e7fc700 1 --2- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f687c0670e0 0x7f687c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:06.187 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:06.186+0000 7fecea2cb180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T04:49:06.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.240+0000 7f68953ff700 1 -- 172.21.15.55:0/2872097417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68900adc00 con 0x7f68900cbf30 2023-10-06T04:49:06.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.240+0000 7f6876ffd700 1 -- 172.21.15.55:0/2872097417 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4602 (secure 0 0 0) 0x7f6880055550 con 0x7f68900cbf30 2023-10-06T04:49:06.243 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:06.243 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":78,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8308,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/421768289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":421768289},{"type":"v1","addr":"172.21.15.55:6835","nonce":421768289}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8311,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2920562647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2920562647},{"type":"v1","addr":"172.21.15.163:6834","nonce":2920562647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8314,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2324769448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2324769448},{"type":"v1","addr":"172.21.15.163:6835","nonce":2324769448}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:48:42.309816+0000","modified":"2023-10-06T04:48:45.840578+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8709":{"gid":8709,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6834/3504492616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3504492616},{"type":"v1","addr":"172.21.15.55:6834","nonce":3504492616}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-06T04:49:06.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.243+0000 7f68953ff700 1 -- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f687c0670e0 msgr2=0x7f687c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:06.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.243+0000 7f68953ff700 1 --2- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f687c0670e0 0x7f687c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.243+0000 7f68953ff700 1 -- 172.21.15.55:0/2872097417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68900cbf30 msgr2=0x7f6890161800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:06.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.243+0000 7f68953ff700 1 --2- 172.21.15.55:0/2872097417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68900cbf30 0x7f6890161800 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f688000ec80 tx=0x7f68800091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.243+0000 7f68953ff700 1 -- 172.21.15.55:0/2872097417 shutdown_connections 2023-10-06T04:49:06.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.243+0000 7f68953ff700 1 --2- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f687c0670e0 0x7f687c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.243+0000 7f68953ff700 1 --2- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f689015a270 0x7f689016bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.243+0000 7f68953ff700 1 --2- 172.21.15.55:0/2872097417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68900cbf30 0x7f6890161800 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.243+0000 7f68953ff700 1 --2- 172.21.15.55:0/2872097417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68900ccf90 0x7f689016ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.243+0000 7f68953ff700 1 -- 172.21.15.55:0/2872097417 >> 172.21.15.55:0/2872097417 conn(0x7f689006a320 msgr2=0x7f6890156120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:06.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.243+0000 7f68953ff700 1 -- 172.21.15.55:0/2872097417 shutdown_connections 2023-10-06T04:49:06.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.243+0000 7f68953ff700 1 -- 172.21.15.55:0/2872097417 wait complete. 2023-10-06T04:49:06.248 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 78 2023-10-06T04:49:06.264 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:49:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.451+0000 7f497e19d700 1 Processor -- start 2023-10-06T04:49:06.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.451+0000 7f497e19d700 1 -- start start 2023-10-06T04:49:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.452+0000 7f497e19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4978159420 0x7f49780b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.452+0000 7f497e19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f497815d230 0x7f497815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.452+0000 7f497e19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49780b2980 con 0x7f49781597f0 2023-10-06T04:49:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.452+0000 7f497e19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f497815db70 con 0x7f4978159420 2023-10-06T04:49:06.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.452+0000 7f497e19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49780b11d0 con 0x7f497815d230 2023-10-06T04:49:06.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.452+0000 7f497d19b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4978159420 0x7f49780b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:06.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.453+0000 7f497c99a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f49781597f0 0x7f49780b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42568/0 (socket says 172.21.15.55:42568) 2023-10-06T04:49:06.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.453+0000 7f497c99a700 1 -- 172.21.15.55:0/3815531721 learned_addr learned my addr 172.21.15.55:0/3815531721 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:06.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.453+0000 7f497d99c700 1 --2- 172.21.15.55:0/3815531721 >> v2:172.21.15.163:3300/0 conn(0x7f497815d230 0x7f497815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:06.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.453+0000 7f497d19b700 1 -- 172.21.15.55:0/3815531721 >> v2:172.21.15.163:3300/0 conn(0x7f497815d230 msgr2=0x7f497815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:06.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.453+0000 7f497d19b700 1 --2- 172.21.15.55:0/3815531721 >> v2:172.21.15.163:3300/0 conn(0x7f497815d230 0x7f497815d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.453+0000 7f497d19b700 1 -- 172.21.15.55:0/3815531721 >> v1:172.21.15.55:6789/0 conn(0x7f49781597f0 legacy=0x7f49780b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:06.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.453+0000 7f497d19b700 1 -- 172.21.15.55:0/3815531721 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49780b14b0 con 0x7f4978159420 2023-10-06T04:49:06.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.453+0000 7f496ffff700 1 -- 172.21.15.55:0/3815531721 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974726397 0 0) 0x7f49780b2980 con 0x7f49781597f0 2023-10-06T04:49:06.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.453+0000 7f497d19b700 1 --2- 172.21.15.55:0/3815531721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4978159420 0x7f49780b5f40 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f497815d170 tx=0x7f49740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b9c4da576f0e99 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:06.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.454+0000 7f496ffff700 1 -- 172.21.15.55:0/3815531721 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974011630 con 0x7f4978159420 2023-10-06T04:49:06.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.454+0000 7f496ffff700 1 -- 172.21.15.55:0/3815531721 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4974011790 con 0x7f4978159420 2023-10-06T04:49:06.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.454+0000 7f496ffff700 1 -- 172.21.15.55:0/3815531721 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974011b30 con 0x7f4978159420 2023-10-06T04:49:06.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.454+0000 7f497e19d700 1 -- 172.21.15.55:0/3815531721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4978159420 msgr2=0x7f49780b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:06.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.454+0000 7f497e19d700 1 --2- 172.21.15.55:0/3815531721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4978159420 0x7f49780b5f40 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f497815d170 tx=0x7f49740091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.454+0000 7f497e19d700 1 -- 172.21.15.55:0/3815531721 shutdown_connections 2023-10-06T04:49:06.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.454+0000 7f497e19d700 1 --2- 172.21.15.55:0/3815531721 >> v2:172.21.15.163:3300/0 conn(0x7f497815d230 0x7f497815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.454+0000 7f497e19d700 1 --2- 172.21.15.55:0/3815531721 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4978159420 0x7f49780b5f40 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.454+0000 7f497e19d700 1 -- 172.21.15.55:0/3815531721 >> 172.21.15.55:0/3815531721 conn(0x7f49780d0660 msgr2=0x7f49780d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:06.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.454+0000 7f497e19d700 1 -- 172.21.15.55:0/3815531721 shutdown_connections 2023-10-06T04:49:06.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.455+0000 7f497e19d700 1 -- 172.21.15.55:0/3815531721 wait complete. 2023-10-06T04:49:06.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.456+0000 7f497e19d700 1 Processor -- start 2023-10-06T04:49:06.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.456+0000 7f497e19d700 1 -- start start 2023-10-06T04:49:06.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497e19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4978159420 0x7f4978145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:06.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497e19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49781597f0 0x7f49780c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:06.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497e19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f497815d230 0x7f497815bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:06.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497e19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49780c7030 con 0x7f4978159420 2023-10-06T04:49:06.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497e19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49780c5d50 con 0x7f49781597f0 2023-10-06T04:49:06.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497e19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49780c6030 con 0x7f497815d230 2023-10-06T04:49:06.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497d19b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4978159420 0x7f4978145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:06.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497d19b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4978159420 0x7f4978145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42706/0 (socket says 172.21.15.55:42706) 2023-10-06T04:49:06.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497d19b700 1 -- 172.21.15.55:0/648302717 learned_addr learned my addr 172.21.15.55:0/648302717 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:06.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497d99c700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f497815d230 0x7f497815bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:06.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497c99a700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49781597f0 0x7f49780c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:06.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497d19b700 1 -- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f497815d230 msgr2=0x7f497815bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:06.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497d19b700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f497815d230 0x7f497815bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497d19b700 1 -- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49781597f0 msgr2=0x7f49780c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:06.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.457+0000 7f497d19b700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49781597f0 0x7f49780c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.458+0000 7f497d19b700 1 -- 172.21.15.55:0/648302717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4974009420 con 0x7f4978159420 2023-10-06T04:49:06.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.458+0000 7f497d99c700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f497815d230 0x7f497815bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:49:06.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.458+0000 7f497d19b700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4978159420 0x7f4978145800 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f4978089340 tx=0x7f4974017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:06.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.458+0000 7f496d7fa700 1 -- 172.21.15.55:0/648302717 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974011900 con 0x7f4978159420 2023-10-06T04:49:06.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.458+0000 7f496d7fa700 1 -- 172.21.15.55:0/648302717 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4974011a60 con 0x7f4978159420 2023-10-06T04:49:06.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.458+0000 7f497e19d700 1 -- 172.21.15.55:0/648302717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49780c6310 con 0x7f4978159420 2023-10-06T04:49:06.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.458+0000 7f496d7fa700 1 -- 172.21.15.55:0/648302717 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974025670 con 0x7f4978159420 2023-10-06T04:49:06.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.458+0000 7f497e19d700 1 -- 172.21.15.55:0/648302717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4978145d40 con 0x7f4978159420 2023-10-06T04:49:06.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.459+0000 7f497e19d700 1 -- 172.21.15.55:0/648302717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497804c3c0 con 0x7f4978159420 2023-10-06T04:49:06.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.460+0000 7f496d7fa700 1 -- 172.21.15.55:0/648302717 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 73) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f4974023070 con 0x7f4978159420 2023-10-06T04:49:06.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.461+0000 7f496d7fa700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f495c0670e0 0x7f495c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:06.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.461+0000 7f496d7fa700 1 -- 172.21.15.55:0/648302717 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f4974086250 con 0x7f4978159420 2023-10-06T04:49:06.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.461+0000 7f497c99a700 1 -- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f495c0670e0 msgr2=0x7f495c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:06.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.461+0000 7f497c99a700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f495c0670e0 0x7f495c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:06.467 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:06.465+0000 7fb5daa99180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T04:49:06.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.465+0000 7f496d7fa700 1 -- 172.21.15.55:0/648302717 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4974052db0 con 0x7f4978159420 2023-10-06T04:49:06.476 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:06.474+0000 7fecea2cb180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T04:49:06.569 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:06.567+0000 7fecea2cb180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T04:49:06.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.662+0000 7f497c99a700 1 -- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f495c0670e0 msgr2=0x7f495c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:06.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.662+0000 7f497c99a700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f495c0670e0 0x7f495c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:06.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.671+0000 7f497e19d700 1 -- 172.21.15.55:0/648302717 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49780b1790 con 0x7f4978159420 2023-10-06T04:49:06.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.674+0000 7f496d7fa700 1 -- 172.21.15.55:0/648302717 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+17896 (secure 0 0 0) 0x7f49740563d0 con 0x7f4978159420 2023-10-06T04:49:06.677 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:06.677 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":161,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T04:48:58.128628+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:48:40.391644+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-10-06T04:48:41.276409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"160","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3452722174":"2023-10-07T04:48:58.128595+0000","172.21.15.163:6837/85191":"2023-10-07T04:48:58.128595+0000","172.21.15.163:6836/85191":"2023-10-07T04:48:58.128595+0000","172.21.15.163:0/1904936092":"2023-10-07T04:48:58.128595+0000","172.21.15.163:0/3499125134":"2023-10-07T04:48:58.128595+0000","172.21.15.163:0/2488193308":"2023-10-07T04:48:58.128595+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:49:06.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.676+0000 7f497e19d700 1 -- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f495c0670e0 msgr2=0x7f495c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:06.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.676+0000 7f497e19d700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f495c0670e0 0x7f495c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.676+0000 7f497e19d700 1 -- 172.21.15.55:0/648302717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4978159420 msgr2=0x7f4978145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.676+0000 7f497e19d700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4978159420 0x7f4978145800 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f4978089340 tx=0x7f4974017000 comp rx=0 tx=0).stop 2023-10-06T04:49:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.676+0000 7f497e19d700 1 -- 172.21.15.55:0/648302717 shutdown_connections 2023-10-06T04:49:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.676+0000 7f497e19d700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f495c0670e0 0x7f495c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.676+0000 7f497e19d700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f497815d230 0x7f497815bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.676+0000 7f497e19d700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4978159420 0x7f4978145800 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.676+0000 7f497e19d700 1 --2- 172.21.15.55:0/648302717 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f49781597f0 0x7f49780c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.676+0000 7f497e19d700 1 -- 172.21.15.55:0/648302717 >> 172.21.15.55:0/648302717 conn(0x7f49780d0660 msgr2=0x7f49780af200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.677+0000 7f497e19d700 1 -- 172.21.15.55:0/648302717 shutdown_connections 2023-10-06T04:49:06.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.677+0000 7f497e19d700 1 -- 172.21.15.55:0/648302717 wait complete. 2023-10-06T04:49:06.698 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T04:49:06.726 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:06.723+0000 7fecea2cb180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T04:49:06.770 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:06.768+0000 7fb5daa99180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T04:49:06.832 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:06.830+0000 7fecea2cb180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T04:49:06.871 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:06.868+0000 7fb5daa99180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T04:49:06.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.895+0000 7faca2f9d700 1 Processor -- start 2023-10-06T04:49:06.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.895+0000 7faca2f9d700 1 -- start start 2023-10-06T04:49:06.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.895+0000 7faca2f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac9c142ea0 0x7fac9c0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:06.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.895+0000 7faca2f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fac9c0ad870 0x7fac9c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:06.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.895+0000 7faca2f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c0b40f0 con 0x7fac9c0b3b70 2023-10-06T04:49:06.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.895+0000 7faca2f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9c069640 con 0x7fac9c0ad870 2023-10-06T04:49:06.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.895+0000 7faca2f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c069920 con 0x7fac9c142ea0 2023-10-06T04:49:06.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.896+0000 7faca279c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fac9c0b3b70 0x7fac9c06cf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42574/0 (socket says 172.21.15.55:42574) 2023-10-06T04:49:06.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.896+0000 7faca279c700 1 -- 172.21.15.55:0/3424779689 learned_addr learned my addr 172.21.15.55:0/3424779689 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:06.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.896+0000 7faca1f9b700 1 --2- 172.21.15.55:0/3424779689 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac9c142ea0 0x7fac9c0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:06.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.896+0000 7faca179a700 1 --2- 172.21.15.55:0/3424779689 >> v2:172.21.15.163:3300/0 conn(0x7fac9c0ad870 0x7fac9c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:06.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.896+0000 7faca0f99700 1 -- 172.21.15.55:0/3424779689 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3766106925 0 0) 0x7fac9c0b40f0 con 0x7fac9c0b3b70 2023-10-06T04:49:06.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.896+0000 7faca0f99700 1 -- 172.21.15.55:0/3424779689 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac88003570 con 0x7fac9c0b3b70 2023-10-06T04:49:06.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.896+0000 7faca1f9b700 1 -- 172.21.15.55:0/3424779689 >> v2:172.21.15.163:3300/0 conn(0x7fac9c0ad870 msgr2=0x7fac9c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:06.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.896+0000 7faca1f9b700 1 --2- 172.21.15.55:0/3424779689 >> v2:172.21.15.163:3300/0 conn(0x7fac9c0ad870 0x7fac9c06b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.896+0000 7faca1f9b700 1 -- 172.21.15.55:0/3424779689 >> v1:172.21.15.55:6789/0 conn(0x7fac9c0b3b70 legacy=0x7fac9c06cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:06.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.896+0000 7faca1f9b700 1 -- 172.21.15.55:0/3424779689 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c069c00 con 0x7fac9c142ea0 2023-10-06T04:49:06.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.897+0000 7faca1f9b700 1 --2- 172.21.15.55:0/3424779689 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac9c142ea0 0x7fac9c0b3630 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fac980098e0 tx=0x7fac980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78847a5ca0eab9c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:06.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.897+0000 7faca0f99700 1 -- 172.21.15.55:0/3424779689 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98011630 con 0x7fac9c142ea0 2023-10-06T04:49:06.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.897+0000 7faca0f99700 1 -- 172.21.15.55:0/3424779689 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac98011790 con 0x7fac9c142ea0 2023-10-06T04:49:06.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.897+0000 7faca0f99700 1 -- 172.21.15.55:0/3424779689 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98011b30 con 0x7fac9c142ea0 2023-10-06T04:49:06.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.897+0000 7faca2f9d700 1 -- 172.21.15.55:0/3424779689 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac9c142ea0 msgr2=0x7fac9c0b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:06.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.897+0000 7faca2f9d700 1 --2- 172.21.15.55:0/3424779689 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac9c142ea0 0x7fac9c0b3630 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fac980098e0 tx=0x7fac980091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.898+0000 7faca2f9d700 1 -- 172.21.15.55:0/3424779689 shutdown_connections 2023-10-06T04:49:06.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.898+0000 7faca2f9d700 1 --2- 172.21.15.55:0/3424779689 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac9c142ea0 0x7fac9c0b3630 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.898+0000 7faca2f9d700 1 --2- 172.21.15.55:0/3424779689 >> v2:172.21.15.163:3300/0 conn(0x7fac9c0ad870 0x7fac9c06b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.898+0000 7faca2f9d700 1 -- 172.21.15.55:0/3424779689 >> 172.21.15.55:0/3424779689 conn(0x7fac9c0ce430 msgr2=0x7fac9c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:06.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.898+0000 7faca2f9d700 1 -- 172.21.15.55:0/3424779689 shutdown_connections 2023-10-06T04:49:06.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.898+0000 7faca2f9d700 1 -- 172.21.15.55:0/3424779689 wait complete. 2023-10-06T04:49:06.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.899+0000 7faca2f9d700 1 Processor -- start 2023-10-06T04:49:06.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.899+0000 7faca2f9d700 1 -- start start 2023-10-06T04:49:06.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.900+0000 7faca2f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac9c0b3b70 0x7fac9c13eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:06.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.900+0000 7faca2f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac9c142ea0 0x7fac9c140810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:06.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.900+0000 7faca2f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac9c140d50 0x7fac9c1695a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:06.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.900+0000 7faca2f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c13f490 con 0x7fac9c142ea0 2023-10-06T04:49:06.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.900+0000 7faca2f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1411c0 con 0x7fac9c0b3b70 2023-10-06T04:49:06.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.900+0000 7faca2f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c16aec0 con 0x7fac9c140d50 2023-10-06T04:49:06.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.900+0000 7faca179a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac9c142ea0 0x7fac9c140810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:06.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.900+0000 7faca179a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac9c142ea0 0x7fac9c140810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42712/0 (socket says 172.21.15.55:42712) 2023-10-06T04:49:06.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.900+0000 7faca179a700 1 -- 172.21.15.55:0/2238090613 learned_addr learned my addr 172.21.15.55:0/2238090613 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:06.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.900+0000 7faca1f9b700 1 --2- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac9c0b3b70 0x7fac9c13eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:06.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.900+0000 7faca279c700 1 --2- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac9c140d50 0x7fac9c1695a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:06.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.901+0000 7faca179a700 1 -- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac9c140d50 msgr2=0x7fac9c1695a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:06.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.901+0000 7faca179a700 1 --2- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac9c140d50 0x7fac9c1695a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.901+0000 7faca179a700 1 -- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac9c0b3b70 msgr2=0x7fac9c13eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:06.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.901+0000 7faca179a700 1 --2- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac9c0b3b70 0x7fac9c13eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:06.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.901+0000 7faca179a700 1 -- 172.21.15.55:0/2238090613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac98009420 con 0x7fac9c142ea0 2023-10-06T04:49:06.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.901+0000 7faca179a700 1 --2- 172.21.15.55:0/2238090613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac9c142ea0 0x7fac9c140810 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00ecb0 tx=0x7fac8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:06.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.901+0000 7fac91ffb700 1 -- 172.21.15.55:0/2238090613 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c00c850 con 0x7fac9c142ea0 2023-10-06T04:49:06.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.901+0000 7fac91ffb700 1 -- 172.21.15.55:0/2238090613 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c00c9b0 con 0x7fac9c142ea0 2023-10-06T04:49:06.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.901+0000 7faca2f9d700 1 -- 172.21.15.55:0/2238090613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9c16a560 con 0x7fac9c142ea0 2023-10-06T04:49:06.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.901+0000 7fac91ffb700 1 -- 172.21.15.55:0/2238090613 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c00cd50 con 0x7fac9c142ea0 2023-10-06T04:49:06.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.902+0000 7faca2f9d700 1 -- 172.21.15.55:0/2238090613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c16a870 con 0x7fac9c142ea0 2023-10-06T04:49:06.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.904+0000 7faca2f9d700 1 -- 172.21.15.55:0/2238090613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9c16b050 con 0x7fac9c142ea0 2023-10-06T04:49:06.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.904+0000 7fac91ffb700 1 -- 172.21.15.55:0/2238090613 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 73) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fac8c013680 con 0x7fac9c142ea0 2023-10-06T04:49:06.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.906+0000 7fac91ffb700 1 --2- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac800670e0 0x7fac80069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:06.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.906+0000 7fac91ffb700 1 -- 172.21.15.55:0/2238090613 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6708+0+0 (secure 0 0 0) 0x7fac8c084ea0 con 0x7fac9c142ea0 2023-10-06T04:49:06.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.906+0000 7faca1f9b700 1 -- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac800670e0 msgr2=0x7fac80069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:06.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.906+0000 7faca1f9b700 1 --2- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac800670e0 0x7fac80069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:06.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:06.911+0000 7fac91ffb700 1 -- 172.21.15.55:0/2238090613 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac8c051aa0 con 0x7fac9c142ea0 2023-10-06T04:49:07.026 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:07.024+0000 7fecea2cb180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T04:49:07.038 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:07.036+0000 7fb5daa99180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T04:49:07.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.107+0000 7faca1f9b700 1 -- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac800670e0 msgr2=0x7fac80069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:07.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.107+0000 7faca1f9b700 1 --2- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac800670e0 0x7fac80069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:07.119 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:07.117+0000 7fecea2cb180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T04:49:07.147 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:07.145+0000 7fb5daa99180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T04:49:07.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.160+0000 7faca2f9d700 1 -- 172.21.15.55:0/2238090613 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fac9c04c3c0 con 0x7fac9c142ea0 2023-10-06T04:49:07.162 INFO:tasks.ceph.mon.a.smithi055.stderr:2023-10-06T04:49:07.160+0000 7fe524ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T04:49:07.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.173+0000 7fac91ffb700 1 -- 172.21.15.55:0/2238090613 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v79) v1 ==== 161+0+0 (secure 0 0 0) 0x7fac8c0550c0 con 0x7fac9c142ea0 2023-10-06T04:49:07.176 INFO:tasks.ceph.mds.a.smithi055.stderr: -2299> 2023-10-06T04:48:46.779+0000 7f8b35bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:49:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.175+0000 7faca2f9d700 1 -- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac800670e0 msgr2=0x7fac80069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.175+0000 7faca2f9d700 1 --2- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac800670e0 0x7fac80069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.175+0000 7faca2f9d700 1 -- 172.21.15.55:0/2238090613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac9c142ea0 msgr2=0x7fac9c140810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:07.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.175+0000 7faca2f9d700 1 --2- 172.21.15.55:0/2238090613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac9c142ea0 0x7fac9c140810 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00ecb0 tx=0x7fac8c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.175+0000 7faca2f9d700 1 -- 172.21.15.55:0/2238090613 shutdown_connections 2023-10-06T04:49:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.175+0000 7faca2f9d700 1 --2- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fac800670e0 0x7fac80069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.175+0000 7faca2f9d700 1 --2- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fac9c140d50 0x7fac9c1695a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.175+0000 7faca2f9d700 1 --2- 172.21.15.55:0/2238090613 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fac9c0b3b70 0x7fac9c13eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.175+0000 7faca2f9d700 1 --2- 172.21.15.55:0/2238090613 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fac9c142ea0 0x7fac9c140810 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.175+0000 7faca2f9d700 1 -- 172.21.15.55:0/2238090613 >> 172.21.15.55:0/2238090613 conn(0x7fac9c0ce430 msgr2=0x7fac9c141ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.175+0000 7faca2f9d700 1 -- 172.21.15.55:0/2238090613 shutdown_connections 2023-10-06T04:49:07.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.175+0000 7faca2f9d700 1 -- 172.21.15.55:0/2238090613 wait complete. 2023-10-06T04:49:07.179 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T04:49:07.199 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T04:49:07.214 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:07.212+0000 7fecea2cb180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T04:49:07.224 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T04:49:07.310 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:07.308+0000 7fecea2cb180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T04:49:07.364 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:07.362+0000 7fb5daa99180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T04:49:07.389 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:07.386+0000 7fec65ec4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:07.389 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:07.386+0000 7fec65ec4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:07.389 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:07.386+0000 7fec65ec4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:07.389 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:07.386+0000 7fec65ec4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:07.389 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:07.386+0000 7fec65ec4700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:07.390 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:07.387+0000 7fec696cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:07.390 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:07.387+0000 7fec696cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:07.390 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:07.387+0000 7fec696cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:07.390 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:07.387+0000 7fec696cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:07.390 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:07.387+0000 7fec696cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:07.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.407+0000 7f054e59d700 1 Processor -- start 2023-10-06T04:49:07.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.407+0000 7f054e59d700 1 -- start start 2023-10-06T04:49:07.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.408+0000 7f054e59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0548159a60 0x7f05480c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:07.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.408+0000 7f054e59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05480ccc00 0x7f0548069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:07.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.408+0000 7f054e59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05480c9cb0 con 0x7f054806a020 2023-10-06T04:49:07.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.408+0000 7f054e59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054806dfc0 con 0x7f0548159a60 2023-10-06T04:49:07.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.408+0000 7f054e59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05480c9f30 con 0x7f05480ccc00 2023-10-06T04:49:07.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.408+0000 7f054dd9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f054806a020 0x7f05480cb230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42576/0 (socket says 172.21.15.55:42576) 2023-10-06T04:49:07.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.408+0000 7f054dd9c700 1 -- 172.21.15.55:0/1513166082 learned_addr learned my addr 172.21.15.55:0/1513166082 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:07.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.408+0000 7f054cd9a700 1 --2- 172.21.15.55:0/1513166082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05480ccc00 0x7f0548069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:07.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.408+0000 7f054d59b700 1 --2- 172.21.15.55:0/1513166082 >> v2:172.21.15.163:3300/0 conn(0x7f0548159a60 0x7f05480c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:07.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.409+0000 7f053ffff700 1 -- 172.21.15.55:0/1513166082 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189799966 0 0) 0x7f05480c9cb0 con 0x7f054806a020 2023-10-06T04:49:07.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.409+0000 7f053ffff700 1 -- 172.21.15.55:0/1513166082 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0534003570 con 0x7f054806a020 2023-10-06T04:49:07.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.409+0000 7f054cd9a700 1 -- 172.21.15.55:0/1513166082 >> v2:172.21.15.163:3300/0 conn(0x7f0548159a60 msgr2=0x7f05480c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:07.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.409+0000 7f054cd9a700 1 --2- 172.21.15.55:0/1513166082 >> v2:172.21.15.163:3300/0 conn(0x7f0548159a60 0x7f05480c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:07.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.409+0000 7f054cd9a700 1 -- 172.21.15.55:0/1513166082 >> v1:172.21.15.55:6789/0 conn(0x7f054806a020 legacy=0x7f05480cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:07.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.409+0000 7f054cd9a700 1 -- 172.21.15.55:0/1513166082 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05480ca470 con 0x7f05480ccc00 2023-10-06T04:49:07.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.409+0000 7f054cd9a700 1 --2- 172.21.15.55:0/1513166082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05480ccc00 0x7f0548069ae0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f05380098e0 tx=0x7f05380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddbe8cc29b1fed50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:07.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.409+0000 7f053ffff700 1 -- 172.21.15.55:0/1513166082 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538011630 con 0x7f05480ccc00 2023-10-06T04:49:07.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.410+0000 7f053ffff700 1 -- 172.21.15.55:0/1513166082 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0538011790 con 0x7f05480ccc00 2023-10-06T04:49:07.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.410+0000 7f053ffff700 1 -- 172.21.15.55:0/1513166082 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538011b30 con 0x7f05480ccc00 2023-10-06T04:49:07.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.410+0000 7f054e59d700 1 -- 172.21.15.55:0/1513166082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05480ccc00 msgr2=0x7f0548069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:07.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.410+0000 7f054e59d700 1 --2- 172.21.15.55:0/1513166082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05480ccc00 0x7f0548069ae0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f05380098e0 tx=0x7f05380091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:07.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.410+0000 7f054e59d700 1 -- 172.21.15.55:0/1513166082 shutdown_connections 2023-10-06T04:49:07.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.410+0000 7f054e59d700 1 --2- 172.21.15.55:0/1513166082 >> v2:172.21.15.163:3300/0 conn(0x7f0548159a60 0x7f05480c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:07.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.410+0000 7f054e59d700 1 --2- 172.21.15.55:0/1513166082 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05480ccc00 0x7f0548069ae0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:07.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.410+0000 7f054e59d700 1 -- 172.21.15.55:0/1513166082 >> 172.21.15.55:0/1513166082 conn(0x7f054806baa0 msgr2=0x7f054806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:07.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.410+0000 7f054e59d700 1 -- 172.21.15.55:0/1513166082 shutdown_connections 2023-10-06T04:49:07.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.410+0000 7f054e59d700 1 -- 172.21.15.55:0/1513166082 wait complete. 2023-10-06T04:49:07.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.412+0000 7f054e59d700 1 Processor -- start 2023-10-06T04:49:07.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.412+0000 7f054e59d700 1 -- start start 2023-10-06T04:49:07.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054e59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054806a020 0x7f05481526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:07.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054e59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05480ccc00 0x7f0548169ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:07.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054e59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0548159a60 0x7f05480b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:07.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054e59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0548154ae0 con 0x7f0548159a60 2023-10-06T04:49:07.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054e59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05481550a0 con 0x7f054806a020 2023-10-06T04:49:07.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054e59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054816a7a0 con 0x7f05480ccc00 2023-10-06T04:49:07.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054dd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0548159a60 0x7f05480b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:07.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054dd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0548159a60 0x7f05480b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42728/0 (socket says 172.21.15.55:42728) 2023-10-06T04:49:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054dd9c700 1 -- 172.21.15.55:0/790602226 learned_addr learned my addr 172.21.15.55:0/790602226 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054cd9a700 1 --2- 172.21.15.55:0/790602226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05480ccc00 0x7f0548169ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054d59b700 1 --2- 172.21.15.55:0/790602226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054806a020 0x7f05481526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054dd9c700 1 -- 172.21.15.55:0/790602226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05480ccc00 msgr2=0x7f0548169ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054dd9c700 1 --2- 172.21.15.55:0/790602226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05480ccc00 0x7f0548169ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054dd9c700 1 -- 172.21.15.55:0/790602226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054806a020 msgr2=0x7f05481526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054dd9c700 1 --2- 172.21.15.55:0/790602226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054806a020 0x7f05481526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.413+0000 7f054dd9c700 1 -- 172.21.15.55:0/790602226 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0538009420 con 0x7f0548159a60 2023-10-06T04:49:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.414+0000 7f054d59b700 1 --2- 172.21.15.55:0/790602226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054806a020 0x7f05481526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:49:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.414+0000 7f054dd9c700 1 --2- 172.21.15.55:0/790602226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0548159a60 0x7f05480b36c0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f054000cca0 tx=0x7f0540001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.414+0000 7f053d7fa700 1 -- 172.21.15.55:0/790602226 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054000a8d0 con 0x7f0548159a60 2023-10-06T04:49:07.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.414+0000 7f053d7fa700 1 -- 172.21.15.55:0/790602226 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054000aa30 con 0x7f0548159a60 2023-10-06T04:49:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.414+0000 7f053d7fa700 1 -- 172.21.15.55:0/790602226 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054000adb0 con 0x7f0548159a60 2023-10-06T04:49:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.414+0000 7f054e59d700 1 -- 172.21.15.55:0/790602226 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054816aa80 con 0x7f0548159a60 2023-10-06T04:49:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.414+0000 7f054e59d700 1 -- 172.21.15.55:0/790602226 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05480b1610 con 0x7f0548159a60 2023-10-06T04:49:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.415+0000 7f054e59d700 1 -- 172.21.15.55:0/790602226 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054806eba0 con 0x7f0548159a60 2023-10-06T04:49:07.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.416+0000 7f053d7fa700 1 -- 172.21.15.55:0/790602226 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 74) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f0540007300 con 0x7f0548159a60 2023-10-06T04:49:07.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.417+0000 7f053d7fa700 1 -- 172.21.15.55:0/790602226 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 7095+0+0 (secure 0 0 0) 0x7f05400849f0 con 0x7f0548159a60 2023-10-06T04:49:07.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.421+0000 7f053d7fa700 1 -- 172.21.15.55:0/790602226 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0540051a40 con 0x7f0548159a60 2023-10-06T04:49:07.468 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:07.466+0000 7fb5daa99180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T04:49:07.564 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:07.563+0000 7fb5daa99180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T04:49:07.663 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:07.661+0000 7fb5daa99180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T04:49:07.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:07.673+0000 7f054e59d700 1 -- 172.21.15.55:0/790602226 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f05480ca750 con 0x7f0548159a60 2023-10-06T04:49:08.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.217+0000 7f053d7fa700 1 -- 172.21.15.55:0/790602226 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v80) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0540055060 con 0x7f0548159a60 2023-10-06T04:49:08.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.220+0000 7f054e59d700 1 -- 172.21.15.55:0/790602226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0548159a60 msgr2=0x7f05480b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:08.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.220+0000 7f054e59d700 1 --2- 172.21.15.55:0/790602226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0548159a60 0x7f05480b36c0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f054000cca0 tx=0x7f0540001ae0 comp rx=0 tx=0).stop 2023-10-06T04:49:08.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.220+0000 7f054e59d700 1 -- 172.21.15.55:0/790602226 shutdown_connections 2023-10-06T04:49:08.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.220+0000 7f054e59d700 1 --2- 172.21.15.55:0/790602226 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0548159a60 0x7f05480b36c0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:08.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.220+0000 7f054e59d700 1 --2- 172.21.15.55:0/790602226 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f054806a020 0x7f05481526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:08.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.220+0000 7f054e59d700 1 --2- 172.21.15.55:0/790602226 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f05480ccc00 0x7f0548169ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:08.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.220+0000 7f054e59d700 1 -- 172.21.15.55:0/790602226 >> 172.21.15.55:0/790602226 conn(0x7f054806baa0 msgr2=0x7f054806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:08.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.220+0000 7f054e59d700 1 -- 172.21.15.55:0/790602226 shutdown_connections 2023-10-06T04:49:08.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.221+0000 7f054e59d700 1 -- 172.21.15.55:0/790602226 wait complete. 2023-10-06T04:49:08.243 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T04:49:08.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.479+0000 7f48ab59e700 1 Processor -- start 2023-10-06T04:49:08.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.479+0000 7f48ab59e700 1 -- start start 2023-10-06T04:49:08.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.480+0000 7f48ab59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48ac15ca70 0x7f48ac1645a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:08.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.480+0000 7f48ab59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ac159db0 con 0x7f48ac0ccb40 2023-10-06T04:49:08.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.480+0000 7f48ab59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ac06a470 con 0x7f48ac0cb560 2023-10-06T04:49:08.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.480+0000 7f48ab59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac06aa30 con 0x7f48ac15ca70 2023-10-06T04:49:08.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.481+0000 7f48a9d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f48ac0ccb40 0x7f48ac0697b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42584/0 (socket says 172.21.15.55:42584) 2023-10-06T04:49:08.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.481+0000 7f48a9d9b700 1 -- 172.21.15.55:0/2873621818 learned_addr learned my addr 172.21.15.55:0/2873621818 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:08.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.481+0000 7f48aad9d700 1 --2- 172.21.15.55:0/2873621818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48ac15ca70 0x7f48ac1645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:08.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.481+0000 7f48a959a700 1 -- 172.21.15.55:0/2873621818 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 579413243 0 0) 0x7f48ac159db0 con 0x7f48ac0ccb40 2023-10-06T04:49:08.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.481+0000 7f48a959a700 1 -- 172.21.15.55:0/2873621818 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4894003570 con 0x7f48ac0ccb40 2023-10-06T04:49:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.481+0000 7f48a959a700 1 -- 172.21.15.55:0/2873621818 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530420204 0 0) 0x7f48ac06a470 con 0x7f48ac0cb560 2023-10-06T04:49:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.481+0000 7f48a959a700 1 -- 172.21.15.55:0/2873621818 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48ac159db0 con 0x7f48ac0cb560 2023-10-06T04:49:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.481+0000 7f48a959a700 1 -- 172.21.15.55:0/2873621818 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 61749470 0 0) 0x7f4894003570 con 0x7f48ac0ccb40 2023-10-06T04:49:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.481+0000 7f48aad9d700 1 -- 172.21.15.55:0/2873621818 >> v1:172.21.15.163:6789/0 conn(0x7f48ac0cb560 legacy=0x7f48ac0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.481+0000 7f48aad9d700 1 -- 172.21.15.55:0/2873621818 >> v1:172.21.15.55:6789/0 conn(0x7f48ac0ccb40 legacy=0x7f48ac0697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.481+0000 7f48aad9d700 1 -- 172.21.15.55:0/2873621818 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac06a750 con 0x7f48ac15ca70 2023-10-06T04:49:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.482+0000 7f48aad9d700 1 --2- 172.21.15.55:0/2873621818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48ac15ca70 0x7f48ac1645a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009ec0 tx=0x7f48a4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ecd882b54196963 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.482+0000 7f48a959a700 1 -- 172.21.15.55:0/2873621818 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a400e070 con 0x7f48ac15ca70 2023-10-06T04:49:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.482+0000 7f48a959a700 1 -- 172.21.15.55:0/2873621818 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a400b870 con 0x7f48ac15ca70 2023-10-06T04:49:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.482+0000 7f48a959a700 1 -- 172.21.15.55:0/2873621818 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0001470 con 0x7f48ac15ca70 2023-10-06T04:49:08.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.483+0000 7f48ab59e700 1 -- 172.21.15.55:0/2873621818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48ac15ca70 msgr2=0x7f48ac1645a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:08.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.483+0000 7f48ab59e700 1 --2- 172.21.15.55:0/2873621818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48ac15ca70 0x7f48ac1645a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009ec0 tx=0x7f48a4009780 comp rx=0 tx=0).stop 2023-10-06T04:49:08.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.483+0000 7f48ab59e700 1 -- 172.21.15.55:0/2873621818 shutdown_connections 2023-10-06T04:49:08.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.483+0000 7f48ab59e700 1 --2- 172.21.15.55:0/2873621818 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48ac15ca70 0x7f48ac1645a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:08.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.483+0000 7f48ab59e700 1 -- 172.21.15.55:0/2873621818 >> 172.21.15.55:0/2873621818 conn(0x7f48ac06baf0 msgr2=0x7f48ac06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:08.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.483+0000 7f48ab59e700 1 -- 172.21.15.55:0/2873621818 shutdown_connections 2023-10-06T04:49:08.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.483+0000 7f48ab59e700 1 -- 172.21.15.55:0/2873621818 wait complete. 2023-10-06T04:49:08.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.484+0000 7f48ab59e700 1 Processor -- start 2023-10-06T04:49:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.484+0000 7f48ab59e700 1 -- start start 2023-10-06T04:49:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.485+0000 7f48ab59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48ac0cb560 0x7f48ac14dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.485+0000 7f48ab59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48ac0ccb40 0x7f48ac15df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.485+0000 7f48ab59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48ac15ca70 0x7f48ac15e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.485+0000 7f48ab59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac14e250 con 0x7f48ac0ccb40 2023-10-06T04:49:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.485+0000 7f48ab59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac0b1dc0 con 0x7f48ac15ca70 2023-10-06T04:49:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.485+0000 7f48ab59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac0b2250 con 0x7f48ac0cb560 2023-10-06T04:49:08.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.485+0000 7f48a9d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48ac0ccb40 0x7f48ac15df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:08.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.485+0000 7f48a9d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48ac0ccb40 0x7f48ac15df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42732/0 (socket says 172.21.15.55:42732) 2023-10-06T04:49:08.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.485+0000 7f48a9d9b700 1 -- 172.21.15.55:0/167702642 learned_addr learned my addr 172.21.15.55:0/167702642 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:08.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.486+0000 7f48aad9d700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48ac15ca70 0x7f48ac15e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:08.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.486+0000 7f48aa59c700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48ac0cb560 0x7f48ac14dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:08.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.486+0000 7f48a9d9b700 1 -- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48ac0cb560 msgr2=0x7f48ac14dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:08.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.486+0000 7f48a9d9b700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48ac0cb560 0x7f48ac14dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:08.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.486+0000 7f48a9d9b700 1 -- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48ac15ca70 msgr2=0x7f48ac15e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:08.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.486+0000 7f48a9d9b700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48ac15ca70 0x7f48ac15e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:08.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.486+0000 7f48a9d9b700 1 -- 172.21.15.55:0/167702642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a4009a20 con 0x7f48ac0ccb40 2023-10-06T04:49:08.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.486+0000 7f48aad9d700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48ac15ca70 0x7f48ac15e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:49:08.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.486+0000 7f48aa59c700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48ac0cb560 0x7f48ac14dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:49:08.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.486+0000 7f48a9d9b700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48ac0ccb40 0x7f48ac15df00 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f48a0001380 tx=0x7f48a0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:08.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.486+0000 7f489a7fc700 1 -- 172.21.15.55:0/167702642 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a000ce40 con 0x7f48ac0ccb40 2023-10-06T04:49:08.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.486+0000 7f489a7fc700 1 -- 172.21.15.55:0/167702642 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a0005ad0 con 0x7f48ac0ccb40 2023-10-06T04:49:08.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.486+0000 7f489a7fc700 1 -- 172.21.15.55:0/167702642 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0005e90 con 0x7f48ac0ccb40 2023-10-06T04:49:08.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.486+0000 7f48ab59e700 1 -- 172.21.15.55:0/167702642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ac0b1490 con 0x7f48ac0ccb40 2023-10-06T04:49:08.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.487+0000 7f48ab59e700 1 -- 172.21.15.55:0/167702642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ac0b1a50 con 0x7f48ac0ccb40 2023-10-06T04:49:08.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.487+0000 7f48ab59e700 1 -- 172.21.15.55:0/167702642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ac0b1f70 con 0x7f48ac0ccb40 2023-10-06T04:49:08.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.488+0000 7f489a7fc700 1 -- 172.21.15.55:0/167702642 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 75) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f48a0007300 con 0x7f48ac0ccb40 2023-10-06T04:49:08.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.489+0000 7f489a7fc700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f488c066d80 0x7f488c069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:08.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.489+0000 7f489a7fc700 1 -- 172.21.15.55:0/167702642 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 7095+0+0 (secure 0 0 0) 0x7f48a00844a0 con 0x7f48ac0ccb40 2023-10-06T04:49:08.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.490+0000 7f48aa59c700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f488c066d80 0x7f488c069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:08.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.490+0000 7f48aa59c700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f488c066d80 0x7f488c069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f48ac0c9830 tx=0x7f489c009040 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:08.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.494+0000 7f489a7fc700 1 -- 172.21.15.55:0/167702642 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48ac0b1f70 con 0x7f48ac0ccb40 2023-10-06T04:49:08.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:08.695+0000 7f48ab59e700 1 -- 172.21.15.55:0/167702642 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f48ac0b1f70 con 0x7f48ac0ccb40 2023-10-06T04:49:09.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.334+0000 7f489a7fc700 1 -- 172.21.15.55:0/167702642 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v164) v1 ==== 181+0+0 (secure 0 0 0) 0x7f48ac0b1f70 con 0x7f48ac0ccb40 2023-10-06T04:49:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.337+0000 7f48ab59e700 1 -- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f488c066d80 msgr2=0x7f488c069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.337+0000 7f48ab59e700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f488c066d80 0x7f488c069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f48ac0c9830 tx=0x7f489c009040 comp rx=0 tx=0).stop 2023-10-06T04:49:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.338+0000 7f48ab59e700 1 -- 172.21.15.55:0/167702642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48ac0ccb40 msgr2=0x7f48ac15df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.338+0000 7f48ab59e700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48ac0ccb40 0x7f48ac15df00 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f48a0001380 tx=0x7f48a0001a00 comp rx=0 tx=0).stop 2023-10-06T04:49:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.338+0000 7f48ab59e700 1 -- 172.21.15.55:0/167702642 shutdown_connections 2023-10-06T04:49:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.338+0000 7f48ab59e700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f488c066d80 0x7f488c069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.338+0000 7f48ab59e700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f48ac15ca70 0x7f48ac15e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:09.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.338+0000 7f48ab59e700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f48ac0cb560 0x7f48ac14dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:09.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.338+0000 7f48ab59e700 1 --2- 172.21.15.55:0/167702642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f48ac0ccb40 0x7f48ac15df00 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:09.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.338+0000 7f48ab59e700 1 -- 172.21.15.55:0/167702642 >> 172.21.15.55:0/167702642 conn(0x7f48ac06baf0 msgr2=0x7f48ac1588c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:09.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.338+0000 7f48ab59e700 1 -- 172.21.15.55:0/167702642 shutdown_connections 2023-10-06T04:49:09.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.338+0000 7f48ab59e700 1 -- 172.21.15.55:0/167702642 wait complete. 2023-10-06T04:49:09.341 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2023-10-06T04:49:09.361 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T04:49:09.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.560+0000 7f7050f9d700 1 Processor -- start 2023-10-06T04:49:09.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.561+0000 7f7050f9d700 1 -- start start 2023-10-06T04:49:09.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.562+0000 7f7050f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f704c15d3f0 0x7f704c1538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:09.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.562+0000 7f7050f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704c0c9480 con 0x7f704c0689b0 2023-10-06T04:49:09.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.562+0000 7f7050f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f704c15d880 con 0x7f704c0685e0 2023-10-06T04:49:09.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.562+0000 7f7050f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704c15ae00 con 0x7f704c15d3f0 2023-10-06T04:49:09.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.562+0000 7f704affd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f704c15d3f0 0x7f704c1538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:09.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.562+0000 7f704bfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f704c0689b0 0x7f704c06eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42592/0 (socket says 172.21.15.55:42592) 2023-10-06T04:49:09.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.562+0000 7f704bfff700 1 -- 172.21.15.55:0/3960437565 learned_addr learned my addr 172.21.15.55:0/3960437565 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:09.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.563+0000 7f704a7fc700 1 -- 172.21.15.55:0/3960437565 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094762223 0 0) 0x7f704c0c9480 con 0x7f704c0689b0 2023-10-06T04:49:09.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.563+0000 7f704affd700 1 -- 172.21.15.55:0/3960437565 >> v1:172.21.15.163:6789/0 conn(0x7f704c0685e0 legacy=0x7f704c15a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:09.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.563+0000 7f704affd700 1 -- 172.21.15.55:0/3960437565 >> v1:172.21.15.55:6789/0 conn(0x7f704c0689b0 legacy=0x7f704c06eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:09.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.563+0000 7f704affd700 1 -- 172.21.15.55:0/3960437565 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c15db60 con 0x7f704c15d3f0 2023-10-06T04:49:09.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.563+0000 7f704affd700 1 --2- 172.21.15.55:0/3960437565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c15d3f0 0x7f704c1538f0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f70400098d0 tx=0x7f7040009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487e3c588d600c0a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.563+0000 7f704a7fc700 1 -- 172.21.15.55:0/3960437565 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040009e70 con 0x7f704c15d3f0 2023-10-06T04:49:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.563+0000 7f704a7fc700 1 -- 172.21.15.55:0/3960437565 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7040005c10 con 0x7f704c15d3f0 2023-10-06T04:49:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.563+0000 7f704a7fc700 1 -- 172.21.15.55:0/3960437565 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000b6a0 con 0x7f704c15d3f0 2023-10-06T04:49:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.564+0000 7f7050f9d700 1 -- 172.21.15.55:0/3960437565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c15d3f0 msgr2=0x7f704c1538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.564+0000 7f7050f9d700 1 --2- 172.21.15.55:0/3960437565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c15d3f0 0x7f704c1538f0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f70400098d0 tx=0x7f7040009120 comp rx=0 tx=0).stop 2023-10-06T04:49:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.564+0000 7f7050f9d700 1 -- 172.21.15.55:0/3960437565 shutdown_connections 2023-10-06T04:49:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.564+0000 7f7050f9d700 1 --2- 172.21.15.55:0/3960437565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c15d3f0 0x7f704c1538f0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.564+0000 7f7050f9d700 1 -- 172.21.15.55:0/3960437565 >> 172.21.15.55:0/3960437565 conn(0x7f704c06b420 msgr2=0x7f704c0cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.564+0000 7f7050f9d700 1 -- 172.21.15.55:0/3960437565 shutdown_connections 2023-10-06T04:49:09.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.564+0000 7f7050f9d700 1 -- 172.21.15.55:0/3960437565 wait complete. 2023-10-06T04:49:09.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.565+0000 7f7050f9d700 1 Processor -- start 2023-10-06T04:49:09.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.566+0000 7f7050f9d700 1 -- start start 2023-10-06T04:49:09.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.566+0000 7f7050f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f704c0685e0 0x7f704c0c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:09.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.566+0000 7f7050f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c0689b0 0x7f704c0b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:09.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.566+0000 7f7050f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f704c15d3f0 0x7f704c0b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:09.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.566+0000 7f7050f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c0c5440 con 0x7f704c0689b0 2023-10-06T04:49:09.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.566+0000 7f7050f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c0c4830 con 0x7f704c15d3f0 2023-10-06T04:49:09.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.566+0000 7f7050f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704c0c4b10 con 0x7f704c0685e0 2023-10-06T04:49:09.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.567+0000 7f704affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c0689b0 0x7f704c0b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:09.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.567+0000 7f704affd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c0689b0 0x7f704c0b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42746/0 (socket says 172.21.15.55:42746) 2023-10-06T04:49:09.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.567+0000 7f704affd700 1 -- 172.21.15.55:0/2722738659 learned_addr learned my addr 172.21.15.55:0/2722738659 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:09.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.567+0000 7f704bfff700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f704c15d3f0 0x7f704c0b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:09.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.567+0000 7f704b7fe700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f704c0685e0 0x7f704c0c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:09.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.567+0000 7f704affd700 1 -- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f704c0685e0 msgr2=0x7f704c0c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:09.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.567+0000 7f704affd700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f704c0685e0 0x7f704c0c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:09.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.567+0000 7f704affd700 1 -- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f704c15d3f0 msgr2=0x7f704c0b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:09.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.567+0000 7f704affd700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f704c15d3f0 0x7f704c0b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:09.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.567+0000 7f704affd700 1 -- 172.21.15.55:0/2722738659 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70400093c0 con 0x7f704c0689b0 2023-10-06T04:49:09.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.567+0000 7f704bfff700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f704c15d3f0 0x7f704c0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:49:09.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.567+0000 7f704b7fe700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f704c0685e0 0x7f704c0c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:49:09.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.567+0000 7f704affd700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c0689b0 0x7f704c0b0bd0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f7040003080 tx=0x7f7040015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.568+0000 7f70337fe700 1 -- 172.21.15.55:0/2722738659 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040006700 con 0x7f704c0689b0 2023-10-06T04:49:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.568+0000 7f70337fe700 1 -- 172.21.15.55:0/2722738659 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7040006860 con 0x7f704c0689b0 2023-10-06T04:49:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.568+0000 7f70337fe700 1 -- 172.21.15.55:0/2722738659 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70400266c0 con 0x7f704c0689b0 2023-10-06T04:49:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.568+0000 7f7050f9d700 1 -- 172.21.15.55:0/2722738659 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704c0c4df0 con 0x7f704c0689b0 2023-10-06T04:49:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.568+0000 7f7050f9d700 1 -- 172.21.15.55:0/2722738659 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704c0c44c0 con 0x7f704c0689b0 2023-10-06T04:49:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.569+0000 7f7050f9d700 1 -- 172.21.15.55:0/2722738659 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704c0c4c50 con 0x7f704c0689b0 2023-10-06T04:49:09.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.570+0000 7f70337fe700 1 -- 172.21.15.55:0/2722738659 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 75) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f7040023070 con 0x7f704c0689b0 2023-10-06T04:49:09.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.571+0000 7f70337fe700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7038066d80 0x7f7038069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:09.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.571+0000 7f70337fe700 1 -- 172.21.15.55:0/2722738659 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6657+0+0 (secure 0 0 0) 0x7f7040086510 con 0x7f704c0689b0 2023-10-06T04:49:09.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.571+0000 7f704b7fe700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7038066d80 0x7f7038069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:09.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.572+0000 7f704b7fe700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7038066d80 0x7f7038069230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f703c007fd0 tx=0x7f703c009040 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:09.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.575+0000 7f70337fe700 1 -- 172.21.15.55:0/2722738659 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f704c0c4c50 con 0x7f704c0689b0 2023-10-06T04:49:09.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:09.778+0000 7f7050f9d700 1 -- 172.21.15.55:0/2722738659 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f704c0c4c50 con 0x7f704c0689b0 2023-10-06T04:49:10.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.232+0000 7f70337fe700 1 -- 172.21.15.55:0/2722738659 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7040053970 con 0x7f704c0689b0 2023-10-06T04:49:10.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.340+0000 7f70337fe700 1 -- 172.21.15.55:0/2722738659 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v165) v1 ==== 169+0+0 (secure 0 0 0) 0x7f70400535a0 con 0x7f704c0689b0 2023-10-06T04:49:10.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.344+0000 7f7050f9d700 1 -- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7038066d80 msgr2=0x7f7038069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:10.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.344+0000 7f7050f9d700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7038066d80 0x7f7038069230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f703c007fd0 tx=0x7f703c009040 comp rx=0 tx=0).stop 2023-10-06T04:49:10.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.344+0000 7f7050f9d700 1 -- 172.21.15.55:0/2722738659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c0689b0 msgr2=0x7f704c0b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:10.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.344+0000 7f7050f9d700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c0689b0 0x7f704c0b0bd0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f7040003080 tx=0x7f7040015000 comp rx=0 tx=0).stop 2023-10-06T04:49:10.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.344+0000 7f7050f9d700 1 -- 172.21.15.55:0/2722738659 shutdown_connections 2023-10-06T04:49:10.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.344+0000 7f7050f9d700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7038066d80 0x7f7038069230 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:10.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.344+0000 7f7050f9d700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f704c15d3f0 0x7f704c0b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:10.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.344+0000 7f7050f9d700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f704c0685e0 0x7f704c0c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:10.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.344+0000 7f7050f9d700 1 --2- 172.21.15.55:0/2722738659 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f704c0689b0 0x7f704c0b0bd0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:10.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.344+0000 7f7050f9d700 1 -- 172.21.15.55:0/2722738659 >> 172.21.15.55:0/2722738659 conn(0x7f704c06b420 msgr2=0x7f704c159170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:10.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.345+0000 7f7050f9d700 1 -- 172.21.15.55:0/2722738659 shutdown_connections 2023-10-06T04:49:10.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.345+0000 7f7050f9d700 1 -- 172.21.15.55:0/2722738659 wait complete. 2023-10-06T04:49:10.348 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2023-10-06T04:49:10.369 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T04:49:10.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.560+0000 7efc3f59e700 1 Processor -- start 2023-10-06T04:49:10.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.561+0000 7efc3f59e700 1 -- start start 2023-10-06T04:49:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.562+0000 7efc3f59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efc400ae7e0 0x7efc400c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.562+0000 7efc3f59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4006ebd0 con 0x7efc4006f060 2023-10-06T04:49:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.562+0000 7efc3f59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4015aa70 con 0x7efc400cb1a0 2023-10-06T04:49:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.562+0000 7efc3f59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4015b2e0 con 0x7efc400ae7e0 2023-10-06T04:49:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.563+0000 7efc3dd9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7efc4006f060 0x7efc400ae0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42606/0 (socket says 172.21.15.55:42606) 2023-10-06T04:49:10.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.563+0000 7efc3dd9b700 1 -- 172.21.15.55:0/3812688379 learned_addr learned my addr 172.21.15.55:0/3812688379 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.563+0000 7efc3ed9d700 1 --2- 172.21.15.55:0/3812688379 >> v2:172.21.15.163:3300/0 conn(0x7efc400ae7e0 0x7efc400c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.563+0000 7efc3ed9d700 1 -- 172.21.15.55:0/3812688379 >> v1:172.21.15.163:6789/0 conn(0x7efc400cb1a0 legacy=0x7efc4015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.563+0000 7efc3ed9d700 1 -- 172.21.15.55:0/3812688379 >> v1:172.21.15.55:6789/0 conn(0x7efc4006f060 legacy=0x7efc400ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:10.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.563+0000 7efc3ed9d700 1 -- 172.21.15.55:0/3812688379 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4015ad20 con 0x7efc400ae7e0 2023-10-06T04:49:10.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.563+0000 7efc3d59a700 1 -- 172.21.15.55:0/3812688379 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008062977 0 0) 0x7efc4006ebd0 con 0x7efc4006f060 2023-10-06T04:49:10.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.564+0000 7efc3ed9d700 1 --2- 172.21.15.55:0/3812688379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc400ae7e0 0x7efc400c9db0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7efc38009e60 tx=0x7efc38009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d05167c6dfa63ab2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.565+0000 7efc3d59a700 1 -- 172.21.15.55:0/3812688379 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3800e070 con 0x7efc400ae7e0 2023-10-06T04:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.565+0000 7efc3d59a700 1 -- 172.21.15.55:0/3812688379 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3800bcc0 con 0x7efc400ae7e0 2023-10-06T04:49:10.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.565+0000 7efc3d59a700 1 -- 172.21.15.55:0/3812688379 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38012680 con 0x7efc400ae7e0 2023-10-06T04:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.566+0000 7efc3f59e700 1 -- 172.21.15.55:0/3812688379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc400ae7e0 msgr2=0x7efc400c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.566+0000 7efc3f59e700 1 --2- 172.21.15.55:0/3812688379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc400ae7e0 0x7efc400c9db0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7efc38009e60 tx=0x7efc38009780 comp rx=0 tx=0).stop 2023-10-06T04:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.566+0000 7efc3f59e700 1 -- 172.21.15.55:0/3812688379 shutdown_connections 2023-10-06T04:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.566+0000 7efc3f59e700 1 --2- 172.21.15.55:0/3812688379 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc400ae7e0 0x7efc400c9db0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.566+0000 7efc3f59e700 1 -- 172.21.15.55:0/3812688379 >> 172.21.15.55:0/3812688379 conn(0x7efc400cc1e0 msgr2=0x7efc400cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.566+0000 7efc3f59e700 1 -- 172.21.15.55:0/3812688379 shutdown_connections 2023-10-06T04:49:10.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.566+0000 7efc3f59e700 1 -- 172.21.15.55:0/3812688379 wait complete. 2023-10-06T04:49:10.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.569+0000 7efc3f59e700 1 Processor -- start 2023-10-06T04:49:10.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.569+0000 7efc3f59e700 1 -- start start 2023-10-06T04:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.570+0000 7efc3f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc4006f060 0x7efc400c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.570+0000 7efc3f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc400ae7e0 0x7efc400c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.570+0000 7efc3f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc400cb1a0 0x7efc40158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.570+0000 7efc3f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc400af870 con 0x7efc400cb1a0 2023-10-06T04:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.570+0000 7efc3f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc400aef80 con 0x7efc400ae7e0 2023-10-06T04:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.570+0000 7efc3f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc400c60a0 con 0x7efc4006f060 2023-10-06T04:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.570+0000 7efc3ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc400cb1a0 0x7efc40158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.570+0000 7efc3ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc400cb1a0 0x7efc40158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42754/0 (socket says 172.21.15.55:42754) 2023-10-06T04:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.570+0000 7efc3ed9d700 1 -- 172.21.15.55:0/1619882791 learned_addr learned my addr 172.21.15.55:0/1619882791 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:10.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.571+0000 7efc3e59c700 1 --2- 172.21.15.55:0/1619882791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc4006f060 0x7efc400c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:10.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.571+0000 7efc3dd9b700 1 --2- 172.21.15.55:0/1619882791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc400ae7e0 0x7efc400c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:10.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.571+0000 7efc3ed9d700 1 -- 172.21.15.55:0/1619882791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc4006f060 msgr2=0x7efc400c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:10.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.571+0000 7efc3ed9d700 1 --2- 172.21.15.55:0/1619882791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc4006f060 0x7efc400c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:10.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.571+0000 7efc3ed9d700 1 -- 172.21.15.55:0/1619882791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc400ae7e0 msgr2=0x7efc400c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:10.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.571+0000 7efc3ed9d700 1 --2- 172.21.15.55:0/1619882791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc400ae7e0 0x7efc400c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:10.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.571+0000 7efc3ed9d700 1 -- 172.21.15.55:0/1619882791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc38009a20 con 0x7efc400cb1a0 2023-10-06T04:49:10.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.571+0000 7efc3ed9d700 1 --2- 172.21.15.55:0/1619882791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc400cb1a0 0x7efc40158fe0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7efc38013000 tx=0x7efc38019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:10.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.572+0000 7efc2e7fc700 1 -- 172.21.15.55:0/1619882791 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3800beb0 con 0x7efc400cb1a0 2023-10-06T04:49:10.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.572+0000 7efc2e7fc700 1 -- 172.21.15.55:0/1619882791 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3801b630 con 0x7efc400cb1a0 2023-10-06T04:49:10.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.572+0000 7efc2e7fc700 1 -- 172.21.15.55:0/1619882791 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3801ba10 con 0x7efc400cb1a0 2023-10-06T04:49:10.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.572+0000 7efc3f59e700 1 -- 172.21.15.55:0/1619882791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc400af220 con 0x7efc400cb1a0 2023-10-06T04:49:10.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.572+0000 7efc3f59e700 1 -- 172.21.15.55:0/1619882791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc400c4430 con 0x7efc400cb1a0 2023-10-06T04:49:10.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.573+0000 7efc3f59e700 1 -- 172.21.15.55:0/1619882791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc400af9e0 con 0x7efc400cb1a0 2023-10-06T04:49:10.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.574+0000 7efc2e7fc700 1 -- 172.21.15.55:0/1619882791 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc3801c400 con 0x7efc400cb1a0 2023-10-06T04:49:10.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.575+0000 7efc2e7fc700 1 --2- 172.21.15.55:0/1619882791 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7efc20066f30 0x7efc200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:10.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.575+0000 7efc2e7fc700 1 -- 172.21.15.55:0/1619882791 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6243+0+0 (secure 0 0 0) 0x7efc380867b0 con 0x7efc400cb1a0 2023-10-06T04:49:10.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.575+0000 7efc3e59c700 1 --2- 172.21.15.55:0/1619882791 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7efc20066f30 0x7efc200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:10.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.576+0000 7efc3e59c700 1 --2- 172.21.15.55:0/1619882791 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7efc20066f30 0x7efc200693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efc300039e0 tx=0x7efc30007ec0 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:10.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.579+0000 7efc2e7fc700 1 -- 172.21.15.55:0/1619882791 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc38053760 con 0x7efc400cb1a0 2023-10-06T04:49:10.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.786+0000 7efc3f59e700 1 -- 172.21.15.55:0/1619882791 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7efc4015b000 con 0x7efc400cb1a0 2023-10-06T04:49:10.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.791+0000 7efc2e7fc700 1 -- 172.21.15.55:0/1619882791 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7efc38056d80 con 0x7efc400cb1a0 2023-10-06T04:49:10.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.793+0000 7efc3f59e700 1 -- 172.21.15.55:0/1619882791 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7efc20066f30 msgr2=0x7efc200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:10.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.793+0000 7efc3f59e700 1 --2- 172.21.15.55:0/1619882791 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7efc20066f30 0x7efc200693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efc300039e0 tx=0x7efc30007ec0 comp rx=0 tx=0).stop 2023-10-06T04:49:10.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.793+0000 7efc3f59e700 1 -- 172.21.15.55:0/1619882791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc400cb1a0 msgr2=0x7efc40158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:10.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.793+0000 7efc3f59e700 1 --2- 172.21.15.55:0/1619882791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc400cb1a0 0x7efc40158fe0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7efc38013000 tx=0x7efc38019000 comp rx=0 tx=0).stop 2023-10-06T04:49:10.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.793+0000 7efc3f59e700 1 -- 172.21.15.55:0/1619882791 shutdown_connections 2023-10-06T04:49:10.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.793+0000 7efc3f59e700 1 --2- 172.21.15.55:0/1619882791 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7efc20066f30 0x7efc200693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:10.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.793+0000 7efc3f59e700 1 --2- 172.21.15.55:0/1619882791 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc400cb1a0 0x7efc40158fe0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:10.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.793+0000 7efc3f59e700 1 --2- 172.21.15.55:0/1619882791 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc4006f060 0x7efc400c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:10.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.793+0000 7efc3f59e700 1 --2- 172.21.15.55:0/1619882791 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc400ae7e0 0x7efc400c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:10.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.793+0000 7efc3f59e700 1 -- 172.21.15.55:0/1619882791 >> 172.21.15.55:0/1619882791 conn(0x7efc400cc1e0 msgr2=0x7efc400c7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:10.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.793+0000 7efc3f59e700 1 -- 172.21.15.55:0/1619882791 shutdown_connections 2023-10-06T04:49:10.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:10.793+0000 7efc3f59e700 1 -- 172.21.15.55:0/1619882791 wait complete. 2023-10-06T04:49:10.813 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T04:49:11.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.000+0000 7fc59b19d700 1 Processor -- start 2023-10-06T04:49:11.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.000+0000 7fc59b19d700 1 -- start start 2023-10-06T04:49:11.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.002+0000 7fc59b19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc5941407e0 0x7fc59406a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.002+0000 7fc59b19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc594140bb0 0x7fc59406a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.002+0000 7fc59b19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5940c6820 con 0x7fc59406d4f0 2023-10-06T04:49:11.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.002+0000 7fc59b19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5940c6de0 con 0x7fc5941407e0 2023-10-06T04:49:11.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.002+0000 7fc59b19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc594141270 con 0x7fc594140bb0 2023-10-06T04:49:11.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.002+0000 7fc59a19b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc5941407e0 0x7fc59406a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.002+0000 7fc59a19b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc5941407e0 0x7fc59406a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42758/0 (socket says 172.21.15.55:42758) 2023-10-06T04:49:11.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.002+0000 7fc59a19b700 1 -- 172.21.15.55:0/3244843444 learned_addr learned my addr 172.21.15.55:0/3244843444 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:11.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.002+0000 7fc59a99c700 1 --1- 172.21.15.55:0/3244843444 >> v1:172.21.15.55:6789/0 conn(0x7fc59406d4f0 0x7fc59415b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42610/0 (socket says 172.21.15.55:42610) 2023-10-06T04:49:11.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.002+0000 7fc59999a700 1 --2- 172.21.15.55:0/3244843444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc594140bb0 0x7fc59406a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.003+0000 7fc59a19b700 1 -- 172.21.15.55:0/3244843444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc594140bb0 msgr2=0x7fc59406a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:11.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.003+0000 7fc59a19b700 1 --2- 172.21.15.55:0/3244843444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc594140bb0 0x7fc59406a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.003+0000 7fc59a19b700 1 -- 172.21.15.55:0/3244843444 >> v1:172.21.15.55:6789/0 conn(0x7fc59406d4f0 legacy=0x7fc59415b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.003+0000 7fc59a19b700 1 -- 172.21.15.55:0/3244843444 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59406acf0 con 0x7fc5941407e0 2023-10-06T04:49:11.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.003+0000 7fc599199700 1 -- 172.21.15.55:0/3244843444 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414772897 0 0) 0x7fc5940c6820 con 0x7fc59406d4f0 2023-10-06T04:49:11.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.003+0000 7fc59a19b700 1 --2- 172.21.15.55:0/3244843444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5941407e0 0x7fc59406a180 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fc5900099b0 tx=0x7fc590009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f74cdaa3d05182 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:11.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.003+0000 7fc599199700 1 -- 172.21.15.55:0/3244843444 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59000b860 con 0x7fc5941407e0 2023-10-06T04:49:11.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.003+0000 7fc599199700 1 -- 172.21.15.55:0/3244843444 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59000b9c0 con 0x7fc5941407e0 2023-10-06T04:49:11.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.003+0000 7fc599199700 1 -- 172.21.15.55:0/3244843444 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59000bd60 con 0x7fc5941407e0 2023-10-06T04:49:11.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.004+0000 7fc59b19d700 1 -- 172.21.15.55:0/3244843444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5941407e0 msgr2=0x7fc59406a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.004+0000 7fc59b19d700 1 --2- 172.21.15.55:0/3244843444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5941407e0 0x7fc59406a180 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fc5900099b0 tx=0x7fc590009250 comp rx=0 tx=0).stop 2023-10-06T04:49:11.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.004+0000 7fc59b19d700 1 -- 172.21.15.55:0/3244843444 shutdown_connections 2023-10-06T04:49:11.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.004+0000 7fc59b19d700 1 --2- 172.21.15.55:0/3244843444 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5941407e0 0x7fc59406a180 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.004+0000 7fc59b19d700 1 --2- 172.21.15.55:0/3244843444 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc594140bb0 0x7fc59406a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.004+0000 7fc59b19d700 1 -- 172.21.15.55:0/3244843444 >> 172.21.15.55:0/3244843444 conn(0x7fc5940cc1e0 msgr2=0x7fc5940cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:11.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.004+0000 7fc59b19d700 1 -- 172.21.15.55:0/3244843444 shutdown_connections 2023-10-06T04:49:11.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.004+0000 7fc59b19d700 1 -- 172.21.15.55:0/3244843444 wait complete. 2023-10-06T04:49:11.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.005+0000 7fc59b19d700 1 Processor -- start 2023-10-06T04:49:11.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.005+0000 7fc59b19d700 1 -- start start 2023-10-06T04:49:11.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.006+0000 7fc59b19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc59406d4f0 0x7fc5940c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.006+0000 7fc59b19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5941407e0 0x7fc59413c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.006+0000 7fc59b19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc594140bb0 0x7fc5940b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59b19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940c4db0 con 0x7fc5941407e0 2023-10-06T04:49:11.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59b19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59415c880 con 0x7fc59406d4f0 2023-10-06T04:49:11.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59b19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59413c890 con 0x7fc594140bb0 2023-10-06T04:49:11.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59999a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5941407e0 0x7fc59413c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59999a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5941407e0 0x7fc59413c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42760/0 (socket says 172.21.15.55:42760) 2023-10-06T04:49:11.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59999a700 1 -- 172.21.15.55:0/4231816299 learned_addr learned my addr 172.21.15.55:0/4231816299 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:11.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59a19b700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc59406d4f0 0x7fc5940c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59a99c700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc594140bb0 0x7fc5940b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59999a700 1 -- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc594140bb0 msgr2=0x7fc5940b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59999a700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc594140bb0 0x7fc5940b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59999a700 1 -- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc59406d4f0 msgr2=0x7fc5940c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59999a700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc59406d4f0 0x7fc5940c4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59999a700 1 -- 172.21.15.55:0/4231816299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5900094f0 con 0x7fc5941407e0 2023-10-06T04:49:11.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59a99c700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc594140bb0 0x7fc5940b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:49:11.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.007+0000 7fc59a19b700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc59406d4f0 0x7fc5940c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:49:11.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.008+0000 7fc59999a700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5941407e0 0x7fc59413c350 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fc58400d860 tx=0x7fc584008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:11.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.008+0000 7fc58a7fc700 1 -- 172.21.15.55:0/4231816299 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584009610 con 0x7fc5941407e0 2023-10-06T04:49:11.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.008+0000 7fc58a7fc700 1 -- 172.21.15.55:0/4231816299 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc584009790 con 0x7fc5941407e0 2023-10-06T04:49:11.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.008+0000 7fc59b19d700 1 -- 172.21.15.55:0/4231816299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59415c270 con 0x7fc5941407e0 2023-10-06T04:49:11.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.008+0000 7fc58a7fc700 1 -- 172.21.15.55:0/4231816299 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc584009b50 con 0x7fc5941407e0 2023-10-06T04:49:11.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.008+0000 7fc59b19d700 1 -- 172.21.15.55:0/4231816299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59415f260 con 0x7fc5941407e0 2023-10-06T04:49:11.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.009+0000 7fc59b19d700 1 -- 172.21.15.55:0/4231816299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59406c450 con 0x7fc5941407e0 2023-10-06T04:49:11.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.010+0000 7fc58a7fc700 1 -- 172.21.15.55:0/4231816299 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc58401e460 con 0x7fc5941407e0 2023-10-06T04:49:11.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.011+0000 7fc58a7fc700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fc578066f30 0x7fc5780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.011+0000 7fc58a7fc700 1 -- 172.21.15.55:0/4231816299 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6243+0+0 (secure 0 0 0) 0x7fc584084dc0 con 0x7fc5941407e0 2023-10-06T04:49:11.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.011+0000 7fc59a19b700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fc578066f30 0x7fc5780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.012+0000 7fc59a19b700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fc578066f30 0x7fc5780693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc5900099b0 tx=0x7fc590017000 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:11.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.015+0000 7fc58a7fc700 1 -- 172.21.15.55:0/4231816299 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc584051d70 con 0x7fc5941407e0 2023-10-06T04:49:11.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.214+0000 7fc59b19d700 1 -- 172.21.15.55:0/4231816299 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fc59406afd0 con 0x7fc5941407e0 2023-10-06T04:49:11.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.222+0000 7fc58a7fc700 1 -- 172.21.15.55:0/4231816299 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v56) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc584055390 con 0x7fc5941407e0 2023-10-06T04:49:11.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.225+0000 7fc59b19d700 1 -- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fc578066f30 msgr2=0x7fc5780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.225+0000 7fc59b19d700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fc578066f30 0x7fc5780693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc5900099b0 tx=0x7fc590017000 comp rx=0 tx=0).stop 2023-10-06T04:49:11.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.225+0000 7fc59b19d700 1 -- 172.21.15.55:0/4231816299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5941407e0 msgr2=0x7fc59413c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.225+0000 7fc59b19d700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5941407e0 0x7fc59413c350 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fc58400d860 tx=0x7fc584008040 comp rx=0 tx=0).stop 2023-10-06T04:49:11.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.225+0000 7fc59b19d700 1 -- 172.21.15.55:0/4231816299 shutdown_connections 2023-10-06T04:49:11.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.225+0000 7fc59b19d700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fc578066f30 0x7fc5780693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.225+0000 7fc59b19d700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc594140bb0 0x7fc5940b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.225+0000 7fc59b19d700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc59406d4f0 0x7fc5940c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.225+0000 7fc59b19d700 1 --2- 172.21.15.55:0/4231816299 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc5941407e0 0x7fc59413c350 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.225+0000 7fc59b19d700 1 -- 172.21.15.55:0/4231816299 >> 172.21.15.55:0/4231816299 conn(0x7fc5940cc1e0 msgr2=0x7fc5940c1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:11.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.225+0000 7fc59b19d700 1 -- 172.21.15.55:0/4231816299 shutdown_connections 2023-10-06T04:49:11.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.225+0000 7fc59b19d700 1 -- 172.21.15.55:0/4231816299 wait complete. 2023-10-06T04:49:11.245 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T04:49:11.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.432+0000 7f264759e700 1 Processor -- start 2023-10-06T04:49:11.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.433+0000 7f264759e700 1 -- start start 2023-10-06T04:49:11.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.434+0000 7f264759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26480b1b40 0x7f26480b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.434+0000 7f264759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26480cfb00 con 0x7f26480b16d0 2023-10-06T04:49:11.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.434+0000 7f264759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26480b5c40 con 0x7f26480b4e70 2023-10-06T04:49:11.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.434+0000 7f264759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26480b5f20 con 0x7f26480b1b40 2023-10-06T04:49:11.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.434+0000 7f2646d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26480b1b40 0x7f26480b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.434+0000 7f2646d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f26480b1b40 0x7f26480b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42762/0 (socket says 172.21.15.55:42762) 2023-10-06T04:49:11.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.434+0000 7f2645d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f26480b16d0 0x7f26480b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42616/0 (socket says 172.21.15.55:42616) 2023-10-06T04:49:11.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.434+0000 7f2646d9d700 1 -- 172.21.15.55:0/1386771062 learned_addr learned my addr 172.21.15.55:0/1386771062 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:11.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.435+0000 7f2646d9d700 1 -- 172.21.15.55:0/1386771062 >> v1:172.21.15.163:6789/0 conn(0x7f26480b4e70 legacy=0x7f26480ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:11.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.435+0000 7f2646d9d700 1 -- 172.21.15.55:0/1386771062 >> v1:172.21.15.55:6789/0 conn(0x7f26480b16d0 legacy=0x7f26480b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.435+0000 7f2646d9d700 1 -- 172.21.15.55:0/1386771062 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26480b6200 con 0x7f26480b1b40 2023-10-06T04:49:11.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.435+0000 7f264559a700 1 -- 172.21.15.55:0/1386771062 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871640237 0 0) 0x7f26480cfb00 con 0x7f26480b16d0 2023-10-06T04:49:11.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.435+0000 7f2646d9d700 1 --2- 172.21.15.55:0/1386771062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26480b1b40 0x7f26480b3ac0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f2640009f20 tx=0x7f2640009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c83f46ee43a260c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:11.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.435+0000 7f264559a700 1 -- 172.21.15.55:0/1386771062 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264000e070 con 0x7f26480b1b40 2023-10-06T04:49:11.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.435+0000 7f264559a700 1 -- 172.21.15.55:0/1386771062 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264000ba40 con 0x7f26480b1b40 2023-10-06T04:49:11.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.435+0000 7f264559a700 1 -- 172.21.15.55:0/1386771062 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26400092e0 con 0x7f26480b1b40 2023-10-06T04:49:11.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.436+0000 7f264759e700 1 -- 172.21.15.55:0/1386771062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26480b1b40 msgr2=0x7f26480b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.436+0000 7f264759e700 1 --2- 172.21.15.55:0/1386771062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26480b1b40 0x7f26480b3ac0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f2640009f20 tx=0x7f2640009840 comp rx=0 tx=0).stop 2023-10-06T04:49:11.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.436+0000 7f264759e700 1 -- 172.21.15.55:0/1386771062 shutdown_connections 2023-10-06T04:49:11.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.436+0000 7f264759e700 1 --2- 172.21.15.55:0/1386771062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26480b1b40 0x7f26480b3ac0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.436+0000 7f264759e700 1 -- 172.21.15.55:0/1386771062 >> 172.21.15.55:0/1386771062 conn(0x7f26480d10c0 msgr2=0x7f26480affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:11.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.436+0000 7f264759e700 1 -- 172.21.15.55:0/1386771062 shutdown_connections 2023-10-06T04:49:11.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.436+0000 7f264759e700 1 -- 172.21.15.55:0/1386771062 wait complete. 2023-10-06T04:49:11.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.437+0000 7f264759e700 1 Processor -- start 2023-10-06T04:49:11.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.438+0000 7f264759e700 1 -- start start 2023-10-06T04:49:11.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.438+0000 7f264759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26480b16d0 0x7f264814b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f264759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26480b1b40 0x7f26480c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f264759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26480b4e70 0x7f264815d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f264759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26480c9a60 con 0x7f26480b4e70 2023-10-06T04:49:11.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f264759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26480c83c0 con 0x7f26480b1b40 2023-10-06T04:49:11.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f264759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26480c77b0 con 0x7f26480b16d0 2023-10-06T04:49:11.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f2646d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26480b4e70 0x7f264815d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f2646d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26480b4e70 0x7f264815d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42772/0 (socket says 172.21.15.55:42772) 2023-10-06T04:49:11.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f2646d9d700 1 -- 172.21.15.55:0/3648372761 learned_addr learned my addr 172.21.15.55:0/3648372761 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:11.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f2645d9b700 1 --2- 172.21.15.55:0/3648372761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26480b1b40 0x7f26480c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f264659c700 1 --2- 172.21.15.55:0/3648372761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26480b16d0 0x7f264814b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f2646d9d700 1 -- 172.21.15.55:0/3648372761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26480b16d0 msgr2=0x7f264814b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f2646d9d700 1 --2- 172.21.15.55:0/3648372761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26480b16d0 0x7f264814b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f2646d9d700 1 -- 172.21.15.55:0/3648372761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26480b1b40 msgr2=0x7f26480c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f2646d9d700 1 --2- 172.21.15.55:0/3648372761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26480b1b40 0x7f26480c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f2646d9d700 1 -- 172.21.15.55:0/3648372761 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2640009ae0 con 0x7f26480b4e70 2023-10-06T04:49:11.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.439+0000 7f2646d9d700 1 --2- 172.21.15.55:0/3648372761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26480b4e70 0x7f264815d840 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f264000b6c0 tx=0x7f2640017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:11.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.440+0000 7f26367fc700 1 -- 172.21.15.55:0/3648372761 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26400092e0 con 0x7f26480b4e70 2023-10-06T04:49:11.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.440+0000 7f26367fc700 1 -- 172.21.15.55:0/3648372761 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264001e430 con 0x7f26480b4e70 2023-10-06T04:49:11.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.440+0000 7f26367fc700 1 -- 172.21.15.55:0/3648372761 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264001e7f0 con 0x7f26480b4e70 2023-10-06T04:49:11.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.440+0000 7f264759e700 1 -- 172.21.15.55:0/3648372761 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26480c7a90 con 0x7f26480b4e70 2023-10-06T04:49:11.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.440+0000 7f264759e700 1 -- 172.21.15.55:0/3648372761 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26480c8050 con 0x7f26480b4e70 2023-10-06T04:49:11.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.441+0000 7f264759e700 1 -- 172.21.15.55:0/3648372761 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26480c8530 con 0x7f26480b4e70 2023-10-06T04:49:11.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.442+0000 7f26367fc700 1 -- 172.21.15.55:0/3648372761 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f264001ebe0 con 0x7f26480b4e70 2023-10-06T04:49:11.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.443+0000 7f26367fc700 1 --2- 172.21.15.55:0/3648372761 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2628066f30 0x7f26280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.443+0000 7f26367fc700 1 -- 172.21.15.55:0/3648372761 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f2640086380 con 0x7f26480b4e70 2023-10-06T04:49:11.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.443+0000 7f264659c700 1 --2- 172.21.15.55:0/3648372761 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2628066f30 0x7f26280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.444+0000 7f264659c700 1 --2- 172.21.15.55:0/3648372761 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2628066f30 0x7f26280693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2638006fd0 tx=0x7f2638008040 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:11.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.447+0000 7f26367fc700 1 -- 172.21.15.55:0/3648372761 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2640053330 con 0x7f26480b4e70 2023-10-06T04:49:11.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.647+0000 7f264759e700 1 -- 172.21.15.55:0/3648372761 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f264804bb80 con 0x7f26480b4e70 2023-10-06T04:49:11.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.651+0000 7f26367fc700 1 -- 172.21.15.55:0/3648372761 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v57) v1 ==== 120+0+0 (secure 0 0 0) 0x7f2640056950 con 0x7f26480b4e70 2023-10-06T04:49:11.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.653+0000 7f264759e700 1 -- 172.21.15.55:0/3648372761 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2628066f30 msgr2=0x7f26280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.653+0000 7f264759e700 1 --2- 172.21.15.55:0/3648372761 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2628066f30 0x7f26280693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2638006fd0 tx=0x7f2638008040 comp rx=0 tx=0).stop 2023-10-06T04:49:11.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.653+0000 7f264759e700 1 -- 172.21.15.55:0/3648372761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26480b4e70 msgr2=0x7f264815d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.653+0000 7f264759e700 1 --2- 172.21.15.55:0/3648372761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26480b4e70 0x7f264815d840 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f264000b6c0 tx=0x7f2640017000 comp rx=0 tx=0).stop 2023-10-06T04:49:11.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.653+0000 7f264759e700 1 -- 172.21.15.55:0/3648372761 shutdown_connections 2023-10-06T04:49:11.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.653+0000 7f264759e700 1 --2- 172.21.15.55:0/3648372761 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2628066f30 0x7f26280693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.653+0000 7f264759e700 1 --2- 172.21.15.55:0/3648372761 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f26480b4e70 0x7f264815d840 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.653+0000 7f264759e700 1 --2- 172.21.15.55:0/3648372761 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f26480b16d0 0x7f264814b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.653+0000 7f264759e700 1 --2- 172.21.15.55:0/3648372761 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f26480b1b40 0x7f26480c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.653+0000 7f264759e700 1 -- 172.21.15.55:0/3648372761 >> 172.21.15.55:0/3648372761 conn(0x7f26480d10c0 msgr2=0x7f26481590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:11.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.654+0000 7f264759e700 1 -- 172.21.15.55:0/3648372761 shutdown_connections 2023-10-06T04:49:11.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.654+0000 7f264759e700 1 -- 172.21.15.55:0/3648372761 wait complete. 2023-10-06T04:49:11.673 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-06T04:49:11.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.859+0000 7f090639d700 1 Processor -- start 2023-10-06T04:49:11.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.859+0000 7f090639d700 1 -- start start 2023-10-06T04:49:11.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.861+0000 7f090639d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f09000c6c60 0x7f09000c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.861+0000 7f090639d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09000b1410 con 0x7f09001595a0 2023-10-06T04:49:11.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.861+0000 7f090639d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0900063230 con 0x7f090006e8c0 2023-10-06T04:49:11.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.861+0000 7f090639d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09000b2b90 con 0x7f09000c6c60 2023-10-06T04:49:11.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.861+0000 7f0904b9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f09000c6c60 0x7f09000c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.861+0000 7f0904b9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f09000c6c60 0x7f09000c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42776/0 (socket says 172.21.15.55:42776) 2023-10-06T04:49:11.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.861+0000 7f0904b9a700 1 -- 172.21.15.55:0/4149042079 learned_addr learned my addr 172.21.15.55:0/4149042079 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:11.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.861+0000 7f090539b700 1 --1- 172.21.15.55:0/4149042079 >> v1:172.21.15.55:6789/0 conn(0x7f09001595a0 0x7f09000b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:42618/0 (socket says 172.21.15.55:42618) 2023-10-06T04:49:11.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.861+0000 7f0904b9a700 1 -- 172.21.15.55:0/4149042079 >> v1:172.21.15.163:6789/0 conn(0x7f090006e8c0 legacy=0x7f0900159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:11.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.862+0000 7f0904b9a700 1 -- 172.21.15.55:0/4149042079 >> v1:172.21.15.55:6789/0 conn(0x7f09001595a0 legacy=0x7f09000b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.862+0000 7f0904b9a700 1 -- 172.21.15.55:0/4149042079 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09000634f0 con 0x7f09000c6c60 2023-10-06T04:49:11.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.862+0000 7f0904b9a700 1 --2- 172.21.15.55:0/4149042079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09000c6c60 0x7f09000c8120 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f08f00099b0 tx=0x7f08f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1815eee3af961444 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:11.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.862+0000 7f08f7fff700 1 -- 172.21.15.55:0/4149042079 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f000b820 con 0x7f09000c6c60 2023-10-06T04:49:11.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.862+0000 7f08f7fff700 1 -- 172.21.15.55:0/4149042079 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f000b980 con 0x7f09000c6c60 2023-10-06T04:49:11.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.862+0000 7f08f7fff700 1 -- 172.21.15.55:0/4149042079 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f000bd20 con 0x7f09000c6c60 2023-10-06T04:49:11.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.863+0000 7f090639d700 1 -- 172.21.15.55:0/4149042079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09000c6c60 msgr2=0x7f09000c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.863+0000 7f090639d700 1 --2- 172.21.15.55:0/4149042079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09000c6c60 0x7f09000c8120 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f08f00099b0 tx=0x7f08f0009250 comp rx=0 tx=0).stop 2023-10-06T04:49:11.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.863+0000 7f090639d700 1 -- 172.21.15.55:0/4149042079 shutdown_connections 2023-10-06T04:49:11.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.863+0000 7f090639d700 1 --2- 172.21.15.55:0/4149042079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f09000c6c60 0x7f09000c8120 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.863+0000 7f090639d700 1 -- 172.21.15.55:0/4149042079 >> 172.21.15.55:0/4149042079 conn(0x7f090006ddb0 msgr2=0x7f090006e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:11.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.863+0000 7f090639d700 1 -- 172.21.15.55:0/4149042079 shutdown_connections 2023-10-06T04:49:11.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.863+0000 7f090639d700 1 -- 172.21.15.55:0/4149042079 wait complete. 2023-10-06T04:49:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.864+0000 7f090639d700 1 Processor -- start 2023-10-06T04:49:11.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.865+0000 7f090639d700 1 -- start start 2023-10-06T04:49:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.865+0000 7f090639d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f090006e8c0 0x7f09000c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.865+0000 7f090639d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09000c6c60 0x7f09000c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.865+0000 7f090639d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09001595a0 0x7f09000c3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.865+0000 7f090639d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09000c2bd0 con 0x7f090006e8c0 2023-10-06T04:49:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.865+0000 7f090639d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09000c1fc0 con 0x7f09001595a0 2023-10-06T04:49:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.865+0000 7f090639d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09000c22a0 con 0x7f09000c6c60 2023-10-06T04:49:11.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.866+0000 7f090539b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f090006e8c0 0x7f09000c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.866+0000 7f090539b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f090006e8c0 0x7f09000c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42782/0 (socket says 172.21.15.55:42782) 2023-10-06T04:49:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.866+0000 7f090539b700 1 -- 172.21.15.55:0/3370401879 learned_addr learned my addr 172.21.15.55:0/3370401879 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.866+0000 7f0905b9c700 1 --2- 172.21.15.55:0/3370401879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09001595a0 0x7f09000c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.866+0000 7f090539b700 1 -- 172.21.15.55:0/3370401879 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09000c6c60 msgr2=0x7f09000c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.866+0000 7f090539b700 1 --2- 172.21.15.55:0/3370401879 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09000c6c60 0x7f09000c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.866+0000 7f090539b700 1 -- 172.21.15.55:0/3370401879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09001595a0 msgr2=0x7f09000c3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.866+0000 7f090539b700 1 --2- 172.21.15.55:0/3370401879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09001595a0 0x7f09000c3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.866+0000 7f090539b700 1 -- 172.21.15.55:0/3370401879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f00094f0 con 0x7f090006e8c0 2023-10-06T04:49:11.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.866+0000 7f090539b700 1 --2- 172.21.15.55:0/3370401879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f090006e8c0 0x7f09000c41c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00c560 tx=0x7f08fc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:11.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.867+0000 7f08f57fa700 1 -- 172.21.15.55:0/3370401879 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc005a70 con 0x7f090006e8c0 2023-10-06T04:49:11.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.867+0000 7f08f57fa700 1 -- 172.21.15.55:0/3370401879 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc005bd0 con 0x7f090006e8c0 2023-10-06T04:49:11.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.867+0000 7f08f57fa700 1 -- 172.21.15.55:0/3370401879 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc0070d0 con 0x7f090006e8c0 2023-10-06T04:49:11.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.867+0000 7f090639d700 1 -- 172.21.15.55:0/3370401879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09000c2580 con 0x7f090006e8c0 2023-10-06T04:49:11.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.867+0000 7f090639d700 1 -- 172.21.15.55:0/3370401879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09000c4700 con 0x7f090006e8c0 2023-10-06T04:49:11.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.868+0000 7f090639d700 1 -- 172.21.15.55:0/3370401879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09000c48f0 con 0x7f090006e8c0 2023-10-06T04:49:11.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.869+0000 7f08f57fa700 1 -- 172.21.15.55:0/3370401879 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f08fc007480 con 0x7f090006e8c0 2023-10-06T04:49:11.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.870+0000 7f08f57fa700 1 --2- 172.21.15.55:0/3370401879 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f08e4066f30 0x7f08e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:11.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.870+0000 7f08f57fa700 1 -- 172.21.15.55:0/3370401879 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f08fc083d00 con 0x7f090006e8c0 2023-10-06T04:49:11.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.870+0000 7f0904b9a700 1 --2- 172.21.15.55:0/3370401879 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f08e4066f30 0x7f08e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:11.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.871+0000 7f0904b9a700 1 --2- 172.21.15.55:0/3370401879 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f08e4066f30 0x7f08e40693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f08f00099b0 tx=0x7f08f0016000 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:11.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:11.874+0000 7f08f57fa700 1 -- 172.21.15.55:0/3370401879 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f08fc050cb0 con 0x7f090006e8c0 2023-10-06T04:49:12.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.157+0000 7f090639d700 1 -- 172.21.15.55:0/3370401879 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7f090004c3c0 con 0x7f090006e8c0 2023-10-06T04:49:12.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.255+0000 7f08f57fa700 1 -- 172.21.15.55:0/3370401879 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2461) v1 ==== 174+0+0 (secure 0 0 0) 0x7f08fc017070 con 0x7f090006e8c0 2023-10-06T04:49:12.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.257+0000 7f090639d700 1 -- 172.21.15.55:0/3370401879 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f08e4066f30 msgr2=0x7f08e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:12.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.257+0000 7f090639d700 1 --2- 172.21.15.55:0/3370401879 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f08e4066f30 0x7f08e40693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f08f00099b0 tx=0x7f08f0016000 comp rx=0 tx=0).stop 2023-10-06T04:49:12.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.257+0000 7f090639d700 1 -- 172.21.15.55:0/3370401879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f090006e8c0 msgr2=0x7f09000c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:12.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.257+0000 7f090639d700 1 --2- 172.21.15.55:0/3370401879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f090006e8c0 0x7f09000c41c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00c560 tx=0x7f08fc007f70 comp rx=0 tx=0).stop 2023-10-06T04:49:12.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.257+0000 7f090639d700 1 -- 172.21.15.55:0/3370401879 shutdown_connections 2023-10-06T04:49:12.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.257+0000 7f090639d700 1 --2- 172.21.15.55:0/3370401879 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f08e4066f30 0x7f08e40693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:12.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.257+0000 7f090639d700 1 --2- 172.21.15.55:0/3370401879 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f09001595a0 0x7f09000c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:12.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.257+0000 7f090639d700 1 --2- 172.21.15.55:0/3370401879 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f090006e8c0 0x7f09000c41c0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:12.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.257+0000 7f090639d700 1 --2- 172.21.15.55:0/3370401879 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09000c6c60 0x7f09000c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:12.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.257+0000 7f090639d700 1 -- 172.21.15.55:0/3370401879 >> 172.21.15.55:0/3370401879 conn(0x7f090006ddb0 msgr2=0x7f09000caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:12.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.257+0000 7f090639d700 1 -- 172.21.15.55:0/3370401879 shutdown_connections 2023-10-06T04:49:12.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.258+0000 7f090639d700 1 -- 172.21.15.55:0/3370401879 wait complete. 2023-10-06T04:49:12.277 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-10-06T04:49:12.277 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-10-06T04:49:12.277 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T04:49:12.278 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-06T04:49:12.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.463+0000 7f086a99d700 1 Processor -- start 2023-10-06T04:49:12.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.463+0000 7f086a99d700 1 -- start start 2023-10-06T04:49:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.465+0000 7f086a99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f08641566e0 0x7f08640ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.465+0000 7f086a99d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f08640cb900 0x7f08640ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.465+0000 7f086a99d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08640ca860 con 0x7f0864156310 2023-10-06T04:49:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.465+0000 7f086a99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08640cab40 con 0x7f08641566e0 2023-10-06T04:49:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.465+0000 7f086a99d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08640cae20 con 0x7f08640cb900 2023-10-06T04:49:12.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.465+0000 7f086919a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f08641566e0 0x7f08640ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:12.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.465+0000 7f086919a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f08641566e0 0x7f08640ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42784/0 (socket says 172.21.15.55:42784) 2023-10-06T04:49:12.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.465+0000 7f086919a700 1 -- 172.21.15.55:0/18892622 learned_addr learned my addr 172.21.15.55:0/18892622 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:12.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.465+0000 7f086a19c700 1 --2- 172.21.15.55:0/18892622 >> v2:172.21.15.163:3300/0 conn(0x7f08640cb900 0x7f08640ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:12.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.465+0000 7f086919a700 1 -- 172.21.15.55:0/18892622 >> v2:172.21.15.163:3300/0 conn(0x7f08640cb900 msgr2=0x7f08640ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:12.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.465+0000 7f086919a700 1 --2- 172.21.15.55:0/18892622 >> v2:172.21.15.163:3300/0 conn(0x7f08640cb900 0x7f08640ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:12.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.465+0000 7f086919a700 1 -- 172.21.15.55:0/18892622 >> v1:172.21.15.163:6789/0 conn(0x7f0864156310 legacy=0x7f08640c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:12.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.465+0000 7f086919a700 1 -- 172.21.15.55:0/18892622 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08640cb100 con 0x7f08641566e0 2023-10-06T04:49:12.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.466+0000 7f086919a700 1 --2- 172.21.15.55:0/18892622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08641566e0 0x7f08640ae000 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f08640addc0 tx=0x7f0854009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c358bb8f5fa3bd33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:12.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.466+0000 7f0868999700 1 -- 172.21.15.55:0/18892622 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085400b860 con 0x7f08641566e0 2023-10-06T04:49:12.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.466+0000 7f0868999700 1 -- 172.21.15.55:0/18892622 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085400b9c0 con 0x7f08641566e0 2023-10-06T04:49:12.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.466+0000 7f0868999700 1 -- 172.21.15.55:0/18892622 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085400bd60 con 0x7f08641566e0 2023-10-06T04:49:12.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.466+0000 7f086a99d700 1 -- 172.21.15.55:0/18892622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08641566e0 msgr2=0x7f08640ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:12.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.466+0000 7f086a99d700 1 --2- 172.21.15.55:0/18892622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08641566e0 0x7f08640ae000 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f08640addc0 tx=0x7f0854009250 comp rx=0 tx=0).stop 2023-10-06T04:49:12.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.466+0000 7f086a99d700 1 -- 172.21.15.55:0/18892622 shutdown_connections 2023-10-06T04:49:12.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.466+0000 7f086a99d700 1 --2- 172.21.15.55:0/18892622 >> v2:172.21.15.163:3300/0 conn(0x7f08640cb900 0x7f08640ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:12.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.466+0000 7f086a99d700 1 --2- 172.21.15.55:0/18892622 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08641566e0 0x7f08640ae000 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:12.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.466+0000 7f086a99d700 1 -- 172.21.15.55:0/18892622 >> 172.21.15.55:0/18892622 conn(0x7f08640cc1f0 msgr2=0x7f08640cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:12.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.467+0000 7f086a99d700 1 -- 172.21.15.55:0/18892622 shutdown_connections 2023-10-06T04:49:12.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.467+0000 7f086a99d700 1 -- 172.21.15.55:0/18892622 wait complete. 2023-10-06T04:49:12.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.468+0000 7f086a99d700 1 Processor -- start 2023-10-06T04:49:12.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.468+0000 7f086a99d700 1 -- start start 2023-10-06T04:49:12.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.469+0000 7f086a99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08640cb900 0x7f086413c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:12.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.469+0000 7f086a99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0864156310 0x7f0864148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:12.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.469+0000 7f086a99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08641566e0 0x7f08641487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.469+0000 7f086a99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0864146750 con 0x7f08641566e0 2023-10-06T04:49:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.469+0000 7f086a99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086413c6b0 con 0x7f0864156310 2023-10-06T04:49:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.469+0000 7f086a99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0864145ad0 con 0x7f08640cb900 2023-10-06T04:49:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.469+0000 7f086a19c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08641566e0 0x7f08641487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.469+0000 7f086a19c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08641566e0 0x7f08641487b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:42798/0 (socket says 172.21.15.55:42798) 2023-10-06T04:49:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.469+0000 7f086a19c700 1 -- 172.21.15.55:0/2371763162 learned_addr learned my addr 172.21.15.55:0/2371763162 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.470+0000 7f086919a700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0864156310 0x7f0864148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.470+0000 7f086999b700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08640cb900 0x7f086413c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.470+0000 7f086a19c700 1 -- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08640cb900 msgr2=0x7f086413c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.470+0000 7f086a19c700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08640cb900 0x7f086413c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.470+0000 7f086a19c700 1 -- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0864156310 msgr2=0x7f0864148270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:12.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.470+0000 7f086a19c700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0864156310 0x7f0864148270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:12.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.470+0000 7f086a19c700 1 -- 172.21.15.55:0/2371763162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08540094f0 con 0x7f08641566e0 2023-10-06T04:49:12.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.470+0000 7f086999b700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08640cb900 0x7f086413c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:49:12.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.470+0000 7f086a19c700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08641566e0 0x7f08641487b0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f085c007f20 tx=0x7f085c009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:12.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.470+0000 7f0859ffb700 1 -- 172.21.15.55:0/2371763162 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c00eca0 con 0x7f08641566e0 2023-10-06T04:49:12.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.470+0000 7f0859ffb700 1 -- 172.21.15.55:0/2371763162 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085c00ee20 con 0x7f08641566e0 2023-10-06T04:49:12.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.470+0000 7f086a99d700 1 -- 172.21.15.55:0/2371763162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0864145db0 con 0x7f08641566e0 2023-10-06T04:49:12.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.470+0000 7f0859ffb700 1 -- 172.21.15.55:0/2371763162 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c014670 con 0x7f08641566e0 2023-10-06T04:49:12.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.471+0000 7f086a99d700 1 -- 172.21.15.55:0/2371763162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0864146370 con 0x7f08641566e0 2023-10-06T04:49:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.472+0000 7f086a99d700 1 -- 172.21.15.55:0/2371763162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086404c3c0 con 0x7f08641566e0 2023-10-06T04:49:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.472+0000 7f0859ffb700 1 -- 172.21.15.55:0/2371763162 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f085c0149e0 con 0x7f08641566e0 2023-10-06T04:49:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.473+0000 7f0859ffb700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0848066f30 0x7f08480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.473+0000 7f0859ffb700 1 -- 172.21.15.55:0/2371763162 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f085c084f20 con 0x7f08641566e0 2023-10-06T04:49:12.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.473+0000 7f086999b700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0848066f30 0x7f08480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:12.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.474+0000 7f086999b700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0848066f30 0x7f08480693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0860001840 tx=0x7f0860009040 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:12.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.477+0000 7f0859ffb700 1 -- 172.21.15.55:0/2371763162 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f085c051ed0 con 0x7f08641566e0 2023-10-06T04:49:12.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:12.757+0000 7f086a99d700 1 -- 172.21.15.55:0/2371763162 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f08640cd0b0 con 0x7f08641566e0 2023-10-06T04:49:13.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.266+0000 7f0859ffb700 1 -- 172.21.15.55:0/2371763162 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2462) v1 ==== 168+0+0 (secure 0 0 0) 0x7f085c0554f0 con 0x7f08641566e0 2023-10-06T04:49:13.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.269+0000 7f086a99d700 1 -- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0848066f30 msgr2=0x7f08480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:13.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.269+0000 7f086a99d700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0848066f30 0x7f08480693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0860001840 tx=0x7f0860009040 comp rx=0 tx=0).stop 2023-10-06T04:49:13.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.269+0000 7f086a99d700 1 -- 172.21.15.55:0/2371763162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08641566e0 msgr2=0x7f08641487b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:13.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.269+0000 7f086a99d700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08641566e0 0x7f08641487b0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f085c007f20 tx=0x7f085c009860 comp rx=0 tx=0).stop 2023-10-06T04:49:13.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.269+0000 7f086a99d700 1 -- 172.21.15.55:0/2371763162 shutdown_connections 2023-10-06T04:49:13.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.269+0000 7f086a99d700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0848066f30 0x7f08480693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:13.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.269+0000 7f086a99d700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f08641566e0 0x7f08641487b0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:13.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.269+0000 7f086a99d700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f08640cb900 0x7f086413c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:13.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.269+0000 7f086a99d700 1 --2- 172.21.15.55:0/2371763162 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0864156310 0x7f0864148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:13.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.269+0000 7f086a99d700 1 -- 172.21.15.55:0/2371763162 >> 172.21.15.55:0/2371763162 conn(0x7f08640cc1f0 msgr2=0x7f08640c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:13.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.269+0000 7f086a99d700 1 -- 172.21.15.55:0/2371763162 shutdown_connections 2023-10-06T04:49:13.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.269+0000 7f086a99d700 1 -- 172.21.15.55:0/2371763162 wait complete. 2023-10-06T04:49:13.289 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T04:49:13.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.480+0000 7febf49fe700 1 Processor -- start 2023-10-06T04:49:13.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.480+0000 7febf49fe700 1 -- start start 2023-10-06T04:49:13.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.482+0000 7febf49fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7febf00cb1a0 0x7febf015a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.482+0000 7febf49fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7febf00ae0d0 0x7febf00ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.482+0000 7febf49fe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf006ebd0 con 0x7febf015a8a0 2023-10-06T04:49:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.482+0000 7febf49fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf015b2e0 con 0x7febf00ae0d0 2023-10-06T04:49:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.482+0000 7febf49fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf00aea10 con 0x7febf00cb1a0 2023-10-06T04:49:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.482+0000 7febef59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7febf00ae0d0 0x7febf00ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.482+0000 7febef59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7febf00ae0d0 0x7febf00ae4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52326/0 (socket says 172.21.15.55:52326) 2023-10-06T04:49:13.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.482+0000 7febef59e700 1 -- 172.21.15.55:0/4143392697 learned_addr learned my addr 172.21.15.55:0/4143392697 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.482+0000 7febeed9d700 1 --2- 172.21.15.55:0/4143392697 >> v2:172.21.15.163:3300/0 conn(0x7febf00cb1a0 0x7febf015a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.482+0000 7febef59e700 1 -- 172.21.15.55:0/4143392697 >> v2:172.21.15.163:3300/0 conn(0x7febf00cb1a0 msgr2=0x7febf015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.482+0000 7febef59e700 1 --2- 172.21.15.55:0/4143392697 >> v2:172.21.15.163:3300/0 conn(0x7febf00cb1a0 0x7febf015a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.482+0000 7febef59e700 1 -- 172.21.15.55:0/4143392697 >> v1:172.21.15.163:6789/0 conn(0x7febf015a8a0 legacy=0x7febf006f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.482+0000 7febef59e700 1 -- 172.21.15.55:0/4143392697 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf015ad20 con 0x7febf00ae0d0 2023-10-06T04:49:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.482+0000 7febeed9d700 1 --2- 172.21.15.55:0/4143392697 >> v2:172.21.15.163:3300/0 conn(0x7febf00cb1a0 0x7febf015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:49:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.483+0000 7febef59e700 1 --2- 172.21.15.55:0/4143392697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febf00ae0d0 0x7febf00ae4d0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7febf00cce10 tx=0x7febe8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a2ef933fb58c50 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.483+0000 7febedd9b700 1 -- 172.21.15.55:0/4143392697 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe800e070 con 0x7febf00ae0d0 2023-10-06T04:49:13.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.483+0000 7febedd9b700 1 -- 172.21.15.55:0/4143392697 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe800baa0 con 0x7febf00ae0d0 2023-10-06T04:49:13.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.483+0000 7febedd9b700 1 -- 172.21.15.55:0/4143392697 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe80092e0 con 0x7febf00ae0d0 2023-10-06T04:49:13.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.483+0000 7febf49fe700 1 -- 172.21.15.55:0/4143392697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febf00ae0d0 msgr2=0x7febf00ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:13.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.483+0000 7febf49fe700 1 --2- 172.21.15.55:0/4143392697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febf00ae0d0 0x7febf00ae4d0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7febf00cce10 tx=0x7febe8009840 comp rx=0 tx=0).stop 2023-10-06T04:49:13.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.483+0000 7febf49fe700 1 -- 172.21.15.55:0/4143392697 shutdown_connections 2023-10-06T04:49:13.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.484+0000 7febf49fe700 1 --2- 172.21.15.55:0/4143392697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febf00ae0d0 0x7febf00ae4d0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:13.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.484+0000 7febf49fe700 1 --2- 172.21.15.55:0/4143392697 >> v2:172.21.15.163:3300/0 conn(0x7febf00cb1a0 0x7febf015a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:13.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.484+0000 7febf49fe700 1 -- 172.21.15.55:0/4143392697 >> 172.21.15.55:0/4143392697 conn(0x7febf00cc1e0 msgr2=0x7febf00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:13.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.484+0000 7febf49fe700 1 -- 172.21.15.55:0/4143392697 shutdown_connections 2023-10-06T04:49:13.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.484+0000 7febf49fe700 1 -- 172.21.15.55:0/4143392697 wait complete. 2023-10-06T04:49:13.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.485+0000 7febf49fe700 1 Processor -- start 2023-10-06T04:49:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.485+0000 7febf49fe700 1 -- start start 2023-10-06T04:49:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.486+0000 7febf49fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febf00ae0d0 0x7febf0146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.486+0000 7febf49fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febf00cb1a0 0x7febf00c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.486+0000 7febf49fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febf0167aa0 0x7febf00c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.486+0000 7febf49fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf00c3d60 con 0x7febf0167aa0 2023-10-06T04:49:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.486+0000 7febf49fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf00c84b0 con 0x7febf00cb1a0 2023-10-06T04:49:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.486+0000 7febf49fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf00c65d0 con 0x7febf00ae0d0 2023-10-06T04:49:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.486+0000 7febef59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febf0167aa0 0x7febf00c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.486+0000 7febef59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febf0167aa0 0x7febf00c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52338/0 (socket says 172.21.15.55:52338) 2023-10-06T04:49:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.486+0000 7febef59e700 1 -- 172.21.15.55:0/3142096833 learned_addr learned my addr 172.21.15.55:0/3142096833 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.487+0000 7febeed9d700 1 --2- 172.21.15.55:0/3142096833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febf00ae0d0 0x7febf0146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:13.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.486+0000 7febee59c700 1 --2- 172.21.15.55:0/3142096833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febf00cb1a0 0x7febf00c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:13.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.487+0000 7febef59e700 1 -- 172.21.15.55:0/3142096833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febf00ae0d0 msgr2=0x7febf0146fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:13.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.487+0000 7febef59e700 1 --2- 172.21.15.55:0/3142096833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febf00ae0d0 0x7febf0146fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:13.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.487+0000 7febef59e700 1 -- 172.21.15.55:0/3142096833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febf00cb1a0 msgr2=0x7febf00c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:13.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.487+0000 7febef59e700 1 --2- 172.21.15.55:0/3142096833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febf00cb1a0 0x7febf00c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:13.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.487+0000 7febef59e700 1 -- 172.21.15.55:0/3142096833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8009ae0 con 0x7febf0167aa0 2023-10-06T04:49:13.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.487+0000 7febef59e700 1 --2- 172.21.15.55:0/3142096833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febf0167aa0 0x7febf00c7f70 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7febe8009fd0 tx=0x7febe8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.487+0000 7febd6ffd700 1 -- 172.21.15.55:0/3142096833 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8012de0 con 0x7febf0167aa0 2023-10-06T04:49:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.487+0000 7febd6ffd700 1 -- 172.21.15.55:0/3142096833 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe801d420 con 0x7febf0167aa0 2023-10-06T04:49:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.487+0000 7febd6ffd700 1 -- 172.21.15.55:0/3142096833 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe801d7e0 con 0x7febf0167aa0 2023-10-06T04:49:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.487+0000 7febf49fe700 1 -- 172.21.15.55:0/3142096833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf00c3430 con 0x7febf0167aa0 2023-10-06T04:49:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.487+0000 7febf49fe700 1 -- 172.21.15.55:0/3142096833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf00c39f0 con 0x7febf0167aa0 2023-10-06T04:49:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.488+0000 7febf49fe700 1 -- 172.21.15.55:0/3142096833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf00c3f10 con 0x7febf0167aa0 2023-10-06T04:49:13.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.489+0000 7febd6ffd700 1 -- 172.21.15.55:0/3142096833 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7febe801dbd0 con 0x7febf0167aa0 2023-10-06T04:49:13.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.490+0000 7febd6ffd700 1 --2- 172.21.15.55:0/3142096833 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7febdc066f30 0x7febdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:13.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.491+0000 7febd6ffd700 1 -- 172.21.15.55:0/3142096833 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6243+0+0 (secure 0 0 0) 0x7febe8086240 con 0x7febf0167aa0 2023-10-06T04:49:13.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.491+0000 7febeed9d700 1 --2- 172.21.15.55:0/3142096833 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7febdc066f30 0x7febdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:13.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.493+0000 7febeed9d700 1 --2- 172.21.15.55:0/3142096833 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7febdc066f30 0x7febdc0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7febf00b01b0 tx=0x7febe0006f90 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:13.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.494+0000 7febd6ffd700 1 -- 172.21.15.55:0/3142096833 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febe80531f0 con 0x7febf0167aa0 2023-10-06T04:49:13.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.696+0000 7febf49fe700 1 -- 172.21.15.55:0/3142096833 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7febf015b000 con 0x7febf0167aa0 2023-10-06T04:49:13.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.700+0000 7febd6ffd700 1 -- 172.21.15.55:0/3142096833 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7febe8056810 con 0x7febf0167aa0 2023-10-06T04:49:13.704 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.702+0000 7febf49fe700 1 -- 172.21.15.55:0/3142096833 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7febdc066f30 msgr2=0x7febdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:13.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.702+0000 7febf49fe700 1 --2- 172.21.15.55:0/3142096833 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7febdc066f30 0x7febdc0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7febf00b01b0 tx=0x7febe0006f90 comp rx=0 tx=0).stop 2023-10-06T04:49:13.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.702+0000 7febf49fe700 1 -- 172.21.15.55:0/3142096833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febf0167aa0 msgr2=0x7febf00c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:13.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.702+0000 7febf49fe700 1 --2- 172.21.15.55:0/3142096833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febf0167aa0 0x7febf00c7f70 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7febe8009fd0 tx=0x7febe8017000 comp rx=0 tx=0).stop 2023-10-06T04:49:13.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.702+0000 7febf49fe700 1 -- 172.21.15.55:0/3142096833 shutdown_connections 2023-10-06T04:49:13.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.702+0000 7febf49fe700 1 --2- 172.21.15.55:0/3142096833 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7febdc066f30 0x7febdc0693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:13.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.702+0000 7febf49fe700 1 --2- 172.21.15.55:0/3142096833 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7febf0167aa0 0x7febf00c7f70 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:13.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.702+0000 7febf49fe700 1 --2- 172.21.15.55:0/3142096833 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7febf00ae0d0 0x7febf0146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:13.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.702+0000 7febf49fe700 1 --2- 172.21.15.55:0/3142096833 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7febf00cb1a0 0x7febf00c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:13.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.702+0000 7febf49fe700 1 -- 172.21.15.55:0/3142096833 >> 172.21.15.55:0/3142096833 conn(0x7febf00cc1e0 msgr2=0x7febf00b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:13.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.703+0000 7febf49fe700 1 -- 172.21.15.55:0/3142096833 shutdown_connections 2023-10-06T04:49:13.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:13.703+0000 7febf49fe700 1 -- 172.21.15.55:0/3142096833 wait complete. 2023-10-06T04:49:13.722 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:49:13.722 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:49:13.799 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:49:13.884 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:49:13.884 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T04:49:13.885 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:49:14.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.070+0000 7f287ad9d700 1 Processor -- start 2023-10-06T04:49:14.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.070+0000 7f287ad9d700 1 -- start start 2023-10-06T04:49:14.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.071+0000 7f287ad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f28740c6c60 0x7f28740c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:14.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.071+0000 7f287ad9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f287406e8c0 0x7f28740c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:14.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.071+0000 7f287ad9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28740b1410 con 0x7f28741595a0 2023-10-06T04:49:14.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.071+0000 7f287ad9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28740b2b90 con 0x7f28740c6c60 2023-10-06T04:49:14.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.071+0000 7f287ad9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2874063230 con 0x7f287406e8c0 2023-10-06T04:49:14.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.071+0000 7f287959a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f28740c6c60 0x7f28740c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:14.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.071+0000 7f2879d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f28741595a0 0x7f28740b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37920/0 (socket says 172.21.15.55:37920) 2023-10-06T04:49:14.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.071+0000 7f2879d9b700 1 -- 172.21.15.55:0/2762779518 learned_addr learned my addr 172.21.15.55:0/2762779518 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:14.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.071+0000 7f287a59c700 1 --2- 172.21.15.55:0/2762779518 >> v2:172.21.15.163:3300/0 conn(0x7f287406e8c0 0x7f28740c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:14.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.072+0000 7f287959a700 1 -- 172.21.15.55:0/2762779518 >> v2:172.21.15.163:3300/0 conn(0x7f287406e8c0 msgr2=0x7f28740c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:14.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.072+0000 7f287959a700 1 --2- 172.21.15.55:0/2762779518 >> v2:172.21.15.163:3300/0 conn(0x7f287406e8c0 0x7f28740c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.072+0000 7f287959a700 1 -- 172.21.15.55:0/2762779518 >> v1:172.21.15.55:6789/0 conn(0x7f28741595a0 legacy=0x7f28740b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:14.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.072+0000 7f287959a700 1 -- 172.21.15.55:0/2762779518 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28740634f0 con 0x7f28740c6c60 2023-10-06T04:49:14.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.072+0000 7f287a59c700 1 --2- 172.21.15.55:0/2762779518 >> v2:172.21.15.163:3300/0 conn(0x7f287406e8c0 0x7f28740c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:49:14.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.072+0000 7f2878d99700 1 -- 172.21.15.55:0/2762779518 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059210414 0 0) 0x7f28740b1410 con 0x7f28741595a0 2023-10-06T04:49:14.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.072+0000 7f287959a700 1 --2- 172.21.15.55:0/2762779518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28740c6c60 0x7f28740c8120 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f287415d390 tx=0x7f28640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9251bb799e91bf40 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:14.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.072+0000 7f2878d99700 1 -- 172.21.15.55:0/2762779518 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286400b6b0 con 0x7f28740c6c60 2023-10-06T04:49:14.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.072+0000 7f2878d99700 1 -- 172.21.15.55:0/2762779518 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286400b810 con 0x7f28740c6c60 2023-10-06T04:49:14.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.072+0000 7f2878d99700 1 -- 172.21.15.55:0/2762779518 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286400bbb0 con 0x7f28740c6c60 2023-10-06T04:49:14.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.073+0000 7f287ad9d700 1 -- 172.21.15.55:0/2762779518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28740c6c60 msgr2=0x7f28740c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:14.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.073+0000 7f287ad9d700 1 --2- 172.21.15.55:0/2762779518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28740c6c60 0x7f28740c8120 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f287415d390 tx=0x7f28640091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.073+0000 7f287ad9d700 1 -- 172.21.15.55:0/2762779518 shutdown_connections 2023-10-06T04:49:14.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.073+0000 7f287ad9d700 1 --2- 172.21.15.55:0/2762779518 >> v2:172.21.15.163:3300/0 conn(0x7f287406e8c0 0x7f28740c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.073+0000 7f287ad9d700 1 --2- 172.21.15.55:0/2762779518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28740c6c60 0x7f28740c8120 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.073+0000 7f287ad9d700 1 -- 172.21.15.55:0/2762779518 >> 172.21.15.55:0/2762779518 conn(0x7f287406ddb0 msgr2=0x7f287406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:14.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.073+0000 7f287ad9d700 1 -- 172.21.15.55:0/2762779518 shutdown_connections 2023-10-06T04:49:14.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.073+0000 7f287ad9d700 1 -- 172.21.15.55:0/2762779518 wait complete. 2023-10-06T04:49:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.074+0000 7f287ad9d700 1 Processor -- start 2023-10-06T04:49:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.075+0000 7f287ad9d700 1 -- start start 2023-10-06T04:49:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.075+0000 7f287ad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f287406e8c0 0x7f2874142550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.075+0000 7f287ad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28740c6c60 0x7f28740c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:14.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.075+0000 7f287ad9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28741595a0 0x7f287415b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.075+0000 7f287ad9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28740c5c40 con 0x7f287406e8c0 2023-10-06T04:49:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.075+0000 7f287ad9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2874142a90 con 0x7f28740c6c60 2023-10-06T04:49:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.075+0000 7f287ad9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287415c120 con 0x7f28741595a0 2023-10-06T04:49:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.076+0000 7f2879d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f287406e8c0 0x7f2874142550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.076+0000 7f2879d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f287406e8c0 0x7f2874142550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52358/0 (socket says 172.21.15.55:52358) 2023-10-06T04:49:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.076+0000 7f2879d9b700 1 -- 172.21.15.55:0/3260199666 learned_addr learned my addr 172.21.15.55:0/3260199666 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.076+0000 7f2879d9b700 1 -- 172.21.15.55:0/3260199666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28741595a0 msgr2=0x7f287415b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.076+0000 7f2879d9b700 1 --2- 172.21.15.55:0/3260199666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28741595a0 0x7f287415b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.076+0000 7f2879d9b700 1 -- 172.21.15.55:0/3260199666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28740c6c60 msgr2=0x7f28740c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.076+0000 7f2879d9b700 1 --2- 172.21.15.55:0/3260199666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28740c6c60 0x7f28740c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.076+0000 7f2879d9b700 1 -- 172.21.15.55:0/3260199666 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2864009420 con 0x7f287406e8c0 2023-10-06T04:49:14.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.076+0000 7f2879d9b700 1 --2- 172.21.15.55:0/3260199666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f287406e8c0 0x7f2874142550 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f287000c5b0 tx=0x7f2870001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:14.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.077+0000 7f2869ffb700 1 -- 172.21.15.55:0/3260199666 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2870005b30 con 0x7f287406e8c0 2023-10-06T04:49:14.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.077+0000 7f2869ffb700 1 -- 172.21.15.55:0/3260199666 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2870005c90 con 0x7f287406e8c0 2023-10-06T04:49:14.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.077+0000 7f2869ffb700 1 -- 172.21.15.55:0/3260199666 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2870007300 con 0x7f287406e8c0 2023-10-06T04:49:14.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.077+0000 7f287ad9d700 1 -- 172.21.15.55:0/3260199666 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28740c5310 con 0x7f287406e8c0 2023-10-06T04:49:14.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.077+0000 7f287ad9d700 1 -- 172.21.15.55:0/3260199666 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28740c58d0 con 0x7f287406e8c0 2023-10-06T04:49:14.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.078+0000 7f287ad9d700 1 -- 172.21.15.55:0/3260199666 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287404c3c0 con 0x7f287406e8c0 2023-10-06T04:49:14.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.079+0000 7f2869ffb700 1 -- 172.21.15.55:0/3260199666 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f287001b3b0 con 0x7f287406e8c0 2023-10-06T04:49:14.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.079+0000 7f2869ffb700 1 --2- 172.21.15.55:0/3260199666 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2858066f30 0x7f28580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:14.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.080+0000 7f2869ffb700 1 -- 172.21.15.55:0/3260199666 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6243+0+0 (secure 0 0 0) 0x7f2870083d20 con 0x7f287406e8c0 2023-10-06T04:49:14.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.081+0000 7f287959a700 1 --2- 172.21.15.55:0/3260199666 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2858066f30 0x7f28580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:14.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.083+0000 7f287959a700 1 --2- 172.21.15.55:0/3260199666 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2858066f30 0x7f28580693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f28740c70c0 tx=0x7f2864017000 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:14.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.084+0000 7f2869ffb700 1 -- 172.21.15.55:0/3260199666 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2870050cd0 con 0x7f287406e8c0 2023-10-06T04:49:14.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.335+0000 7f287ad9d700 1 -- 172.21.15.55:0/3260199666 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28740637d0 con 0x7f287406e8c0 2023-10-06T04:49:14.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.335+0000 7f2869ffb700 1 -- 172.21.15.55:0/3260199666 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f2870017090 con 0x7f287406e8c0 2023-10-06T04:49:14.337 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:14.337 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":80,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8308,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/421768289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":421768289},{"type":"v1","addr":"172.21.15.55:6835","nonce":421768289}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8311,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2920562647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2920562647},{"type":"v1","addr":"172.21.15.163:6834","nonce":2920562647}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8314,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2324769448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2324769448},{"type":"v1","addr":"172.21.15.163:6835","nonce":2324769448}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":9051,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1999869967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1999869967},{"type":"v1","addr":"172.21.15.55:6834","nonce":1999869967}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[]} 2023-10-06T04:49:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.338+0000 7f287ad9d700 1 -- 172.21.15.55:0/3260199666 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2858066f30 msgr2=0x7f28580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.338+0000 7f287ad9d700 1 --2- 172.21.15.55:0/3260199666 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2858066f30 0x7f28580693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f28740c70c0 tx=0x7f2864017000 comp rx=0 tx=0).stop 2023-10-06T04:49:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.338+0000 7f287ad9d700 1 -- 172.21.15.55:0/3260199666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f287406e8c0 msgr2=0x7f2874142550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.338+0000 7f287ad9d700 1 --2- 172.21.15.55:0/3260199666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f287406e8c0 0x7f2874142550 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f287000c5b0 tx=0x7f2870001070 comp rx=0 tx=0).stop 2023-10-06T04:49:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.338+0000 7f287ad9d700 1 -- 172.21.15.55:0/3260199666 shutdown_connections 2023-10-06T04:49:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.338+0000 7f287ad9d700 1 --2- 172.21.15.55:0/3260199666 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2858066f30 0x7f28580693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.338+0000 7f287ad9d700 1 --2- 172.21.15.55:0/3260199666 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28741595a0 0x7f287415b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.338+0000 7f287ad9d700 1 --2- 172.21.15.55:0/3260199666 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f287406e8c0 0x7f2874142550 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.338+0000 7f287ad9d700 1 --2- 172.21.15.55:0/3260199666 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28740c6c60 0x7f28740c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.338+0000 7f287ad9d700 1 -- 172.21.15.55:0/3260199666 >> 172.21.15.55:0/3260199666 conn(0x7f287406ddb0 msgr2=0x7f2874157750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:14.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.338+0000 7f287ad9d700 1 -- 172.21.15.55:0/3260199666 shutdown_connections 2023-10-06T04:49:14.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.338+0000 7f287ad9d700 1 -- 172.21.15.55:0/3260199666 wait complete. 2023-10-06T04:49:14.341 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 80 2023-10-06T04:49:14.357 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T04:49:14.357 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T04:49:14.358 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T04:49:14.358 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:49:14.358 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:49:14.358 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T04:49:14.358 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:49:14.358 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:49:14.359 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T04:49:14.359 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T04:49:14.359 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T04:49:14.359 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:49:14.359 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T04:49:14.359 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T04:49:14.360 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T04:49:14.360 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:49:14.370 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T04:49:14.371 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T04:49:14.372 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:49:14.372 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:49:14.376 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T04:49:14.376 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T04:49:14.377 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T04:49:14.377 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T04:49:14.415 INFO:tasks.ceph.mds.a:Started 2023-10-06T04:49:14.416 DEBUG:teuthology.parallel:result is None 2023-10-06T04:49:14.421 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:49:14.421 DEBUG:teuthology.parallel:result is None 2023-10-06T04:49:14.422 INFO:tasks.ceph.mds.d:Started 2023-10-06T04:49:14.422 DEBUG:teuthology.parallel:result is None 2023-10-06T04:49:14.424 INFO:tasks.ceph.mds.b:Started 2023-10-06T04:49:14.424 DEBUG:teuthology.parallel:result is None 2023-10-06T04:49:14.424 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T04:49:14.552 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T04:49:14.553 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-10-06T04:49:14.561 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T04:49:14.561 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2023-10-06T04:49:14.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.619+0000 7fb471b9d700 1 Processor -- start 2023-10-06T04:49:14.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.620+0000 7fb471b9d700 1 -- start start 2023-10-06T04:49:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.621+0000 7fb471b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb46c142ea0 0x7fb46c0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.621+0000 7fb471b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb46c0ad870 0x7fb46c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.621+0000 7fb471b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46c0b40f0 con 0x7fb46c0b3b70 2023-10-06T04:49:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.621+0000 7fb471b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46c069640 con 0x7fb46c0ad870 2023-10-06T04:49:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.621+0000 7fb471b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46c069920 con 0x7fb46c142ea0 2023-10-06T04:49:14.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.621+0000 7fb46bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb46c0ad870 0x7fb46c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:14.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.621+0000 7fb46bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb46c0ad870 0x7fb46c06b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52404/0 (socket says 172.21.15.55:52404) 2023-10-06T04:49:14.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.621+0000 7fb47139c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb46c0b3b70 0x7fb46c06cf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37944/0 (socket says 172.21.15.55:37944) 2023-10-06T04:49:14.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.621+0000 7fb46bfff700 1 -- 172.21.15.55:0/373502243 learned_addr learned my addr 172.21.15.55:0/373502243 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:14.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.621+0000 7fb470b9b700 1 --2- 172.21.15.55:0/373502243 >> v2:172.21.15.163:3300/0 conn(0x7fb46c142ea0 0x7fb46c0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.622+0000 7fb46bfff700 1 -- 172.21.15.55:0/373502243 >> v2:172.21.15.163:3300/0 conn(0x7fb46c142ea0 msgr2=0x7fb46c0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.622+0000 7fb46bfff700 1 --2- 172.21.15.55:0/373502243 >> v2:172.21.15.163:3300/0 conn(0x7fb46c142ea0 0x7fb46c0b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.622+0000 7fb46bfff700 1 -- 172.21.15.55:0/373502243 >> v1:172.21.15.55:6789/0 conn(0x7fb46c0b3b70 legacy=0x7fb46c06cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:14.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.622+0000 7fb46bfff700 1 -- 172.21.15.55:0/373502243 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c069c00 con 0x7fb46c0ad870 2023-10-06T04:49:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.622+0000 7fb46b7fe700 1 -- 172.21.15.55:0/373502243 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967937712 0 0) 0x7fb46c0b40f0 con 0x7fb46c0b3b70 2023-10-06T04:49:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.622+0000 7fb46bfff700 1 --2- 172.21.15.55:0/373502243 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb46c0ad870 0x7fb46c06b9e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fb46c071240 tx=0x7fb458009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d1707d231e2cb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.622+0000 7fb46b7fe700 1 -- 172.21.15.55:0/373502243 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458009e70 con 0x7fb46c0ad870 2023-10-06T04:49:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.622+0000 7fb46b7fe700 1 -- 172.21.15.55:0/373502243 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45800b860 con 0x7fb46c0ad870 2023-10-06T04:49:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.622+0000 7fb46b7fe700 1 -- 172.21.15.55:0/373502243 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800bc00 con 0x7fb46c0ad870 2023-10-06T04:49:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.623+0000 7fb471b9d700 1 -- 172.21.15.55:0/373502243 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb46c0ad870 msgr2=0x7fb46c06b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.623+0000 7fb471b9d700 1 --2- 172.21.15.55:0/373502243 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb46c0ad870 0x7fb46c06b9e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fb46c071240 tx=0x7fb458009250 comp rx=0 tx=0).stop 2023-10-06T04:49:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.623+0000 7fb471b9d700 1 -- 172.21.15.55:0/373502243 shutdown_connections 2023-10-06T04:49:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.623+0000 7fb471b9d700 1 --2- 172.21.15.55:0/373502243 >> v2:172.21.15.163:3300/0 conn(0x7fb46c142ea0 0x7fb46c0b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.623+0000 7fb471b9d700 1 --2- 172.21.15.55:0/373502243 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb46c0ad870 0x7fb46c06b9e0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.623+0000 7fb471b9d700 1 -- 172.21.15.55:0/373502243 >> 172.21.15.55:0/373502243 conn(0x7fb46c0ce430 msgr2=0x7fb46c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:14.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.623+0000 7fb471b9d700 1 -- 172.21.15.55:0/373502243 shutdown_connections 2023-10-06T04:49:14.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.623+0000 7fb471b9d700 1 -- 172.21.15.55:0/373502243 wait complete. 2023-10-06T04:49:14.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.624+0000 7fb471b9d700 1 Processor -- start 2023-10-06T04:49:14.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.625+0000 7fb471b9d700 1 -- start start 2023-10-06T04:49:14.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb471b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46c0b3b70 0x7fb46c1400f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:14.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb471b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb46c142ea0 0x7fb46c0b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb471b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb46c0ae770 0x7fb46c0aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb471b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c1598e0 con 0x7fb46c0ae770 2023-10-06T04:49:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb471b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c140630 con 0x7fb46c142ea0 2023-10-06T04:49:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb471b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c15d0c0 con 0x7fb46c0b3b70 2023-10-06T04:49:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb47139c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb46c0ae770 0x7fb46c0aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb470b9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46c0b3b70 0x7fb46c1400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb47139c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb46c0ae770 0x7fb46c0aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52410/0 (socket says 172.21.15.55:52410) 2023-10-06T04:49:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb47139c700 1 -- 172.21.15.55:0/1947718870 learned_addr learned my addr 172.21.15.55:0/1947718870 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb470b9b700 1 -- 172.21.15.55:0/1947718870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb46c142ea0 msgr2=0x7fb46c0b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb470b9b700 1 --2- 172.21.15.55:0/1947718870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb46c142ea0 0x7fb46c0b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb470b9b700 1 -- 172.21.15.55:0/1947718870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb46c0ae770 msgr2=0x7fb46c0aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb470b9b700 1 --2- 172.21.15.55:0/1947718870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb46c0ae770 0x7fb46c0aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb470b9b700 1 -- 172.21.15.55:0/1947718870 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4580094f0 con 0x7fb46c0b3b70 2023-10-06T04:49:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.626+0000 7fb47139c700 1 --2- 172.21.15.55:0/1947718870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb46c0ae770 0x7fb46c0aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:49:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.627+0000 7fb470b9b700 1 --2- 172.21.15.55:0/1947718870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46c0b3b70 0x7fb46c1400f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb46000e900 tx=0x7fb4600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.627+0000 7fb468ff9700 1 -- 172.21.15.55:0/1947718870 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460013680 con 0x7fb46c0b3b70 2023-10-06T04:49:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.627+0000 7fb468ff9700 1 -- 172.21.15.55:0/1947718870 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4600137e0 con 0x7fb46c0b3b70 2023-10-06T04:49:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.627+0000 7fb471b9d700 1 -- 172.21.15.55:0/1947718870 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46c0af580 con 0x7fb46c0b3b70 2023-10-06T04:49:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.627+0000 7fb468ff9700 1 -- 172.21.15.55:0/1947718870 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460013b80 con 0x7fb46c0b3b70 2023-10-06T04:49:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.627+0000 7fb471b9d700 1 -- 172.21.15.55:0/1947718870 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46c0afb00 con 0x7fb46c0b3b70 2023-10-06T04:49:14.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.628+0000 7fb471b9d700 1 -- 172.21.15.55:0/1947718870 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46c04c3c0 con 0x7fb46c0b3b70 2023-10-06T04:49:14.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.629+0000 7fb468ff9700 1 -- 172.21.15.55:0/1947718870 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb46000c3e0 con 0x7fb46c0b3b70 2023-10-06T04:49:14.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.630+0000 7fb468ff9700 1 --2- 172.21.15.55:0/1947718870 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb450066f30 0x7fb4500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:14.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.630+0000 7fb468ff9700 1 -- 172.21.15.55:0/1947718870 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6243+0+0 (secure 0 0 0) 0x7fb460084880 con 0x7fb46c0b3b70 2023-10-06T04:49:14.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.630+0000 7fb46bfff700 1 --2- 172.21.15.55:0/1947718870 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb450066f30 0x7fb4500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:14.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.631+0000 7fb46bfff700 1 --2- 172.21.15.55:0/1947718870 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb450066f30 0x7fb4500693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0b12a0 tx=0x7fb458017000 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:14.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.635+0000 7fb468ff9700 1 -- 172.21.15.55:0/1947718870 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb460051830 con 0x7fb46c0b3b70 2023-10-06T04:49:14.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:14.854+0000 7fb471b9d700 1 -- 172.21.15.55:0/1947718870 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb46c143270 con 0x7fb46c0b3b70 2023-10-06T04:49:15.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.291+0000 7fb468ff9700 1 -- 172.21.15.55:0/1947718870 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v167) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb460054e50 con 0x7fb46c0b3b70 2023-10-06T04:49:15.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.294+0000 7fb471b9d700 1 -- 172.21.15.55:0/1947718870 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb450066f30 msgr2=0x7fb4500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:15.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.294+0000 7fb471b9d700 1 --2- 172.21.15.55:0/1947718870 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb450066f30 0x7fb4500693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0b12a0 tx=0x7fb458017000 comp rx=0 tx=0).stop 2023-10-06T04:49:15.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.294+0000 7fb471b9d700 1 -- 172.21.15.55:0/1947718870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46c0b3b70 msgr2=0x7fb46c1400f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:15.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.294+0000 7fb471b9d700 1 --2- 172.21.15.55:0/1947718870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46c0b3b70 0x7fb46c1400f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb46000e900 tx=0x7fb4600091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.294+0000 7fb471b9d700 1 -- 172.21.15.55:0/1947718870 shutdown_connections 2023-10-06T04:49:15.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.294+0000 7fb471b9d700 1 --2- 172.21.15.55:0/1947718870 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb450066f30 0x7fb4500693e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.294+0000 7fb471b9d700 1 --2- 172.21.15.55:0/1947718870 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb46c0ae770 0x7fb46c0aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.294+0000 7fb471b9d700 1 --2- 172.21.15.55:0/1947718870 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46c0b3b70 0x7fb46c1400f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.294+0000 7fb471b9d700 1 --2- 172.21.15.55:0/1947718870 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb46c142ea0 0x7fb46c0b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.294+0000 7fb471b9d700 1 -- 172.21.15.55:0/1947718870 >> 172.21.15.55:0/1947718870 conn(0x7fb46c0ce430 msgr2=0x7fb46c0ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:15.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.294+0000 7fb471b9d700 1 -- 172.21.15.55:0/1947718870 shutdown_connections 2023-10-06T04:49:15.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.294+0000 7fb471b9d700 1 -- 172.21.15.55:0/1947718870 wait complete. 2023-10-06T04:49:15.297 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2023-10-06T04:49:15.316 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T04:49:15.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.503+0000 7f25cf59e700 1 Processor -- start 2023-10-06T04:49:15.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.503+0000 7f25cf59e700 1 -- start start 2023-10-06T04:49:15.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25cf59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f25d0150550 0x7f25d0157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:15.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25cf59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f25d00c7410 0x7f25d0069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:15.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25cf59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d006e2a0 con 0x7f25d0150180 2023-10-06T04:49:15.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25cf59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d006b2e0 con 0x7f25d0150550 2023-10-06T04:49:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25cf59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d00c78e0 con 0x7f25d00c7410 2023-10-06T04:49:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25cdd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f25d0150550 0x7f25d0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25cdd9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f25d0150550 0x7f25d0157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52418/0 (socket says 172.21.15.55:52418) 2023-10-06T04:49:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25ce59c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f25d0150180 0x7f25d00c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37956/0 (socket says 172.21.15.55:37956) 2023-10-06T04:49:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25cdd9b700 1 -- 172.21.15.55:0/2002919389 learned_addr learned my addr 172.21.15.55:0/2002919389 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25ced9d700 1 --2- 172.21.15.55:0/2002919389 >> v2:172.21.15.163:3300/0 conn(0x7f25d00c7410 0x7f25d0069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25cdd9b700 1 -- 172.21.15.55:0/2002919389 >> v2:172.21.15.163:3300/0 conn(0x7f25d00c7410 msgr2=0x7f25d0069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25cdd9b700 1 --2- 172.21.15.55:0/2002919389 >> v2:172.21.15.163:3300/0 conn(0x7f25d00c7410 0x7f25d0069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25cdd9b700 1 -- 172.21.15.55:0/2002919389 >> v1:172.21.15.55:6789/0 conn(0x7f25d0150180 legacy=0x7f25d00c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25cdd9b700 1 -- 172.21.15.55:0/2002919389 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d00c7bc0 con 0x7f25d0150550 2023-10-06T04:49:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.505+0000 7f25cd59a700 1 -- 172.21.15.55:0/2002919389 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907126244 0 0) 0x7f25d006e2a0 con 0x7f25d0150180 2023-10-06T04:49:15.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.506+0000 7f25cdd9b700 1 --2- 172.21.15.55:0/2002919389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d0150550 0x7f25d0157770 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f25d0156270 tx=0x7f25c400a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6760edfcfbcef46b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:15.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.506+0000 7f25cd59a700 1 -- 172.21.15.55:0/2002919389 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400ca40 con 0x7f25d0150550 2023-10-06T04:49:15.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.506+0000 7f25cd59a700 1 -- 172.21.15.55:0/2002919389 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c400cba0 con 0x7f25d0150550 2023-10-06T04:49:15.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.506+0000 7f25cd59a700 1 -- 172.21.15.55:0/2002919389 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c4018400 con 0x7f25d0150550 2023-10-06T04:49:15.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.506+0000 7f25cf59e700 1 -- 172.21.15.55:0/2002919389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d0150550 msgr2=0x7f25d0157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:15.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.506+0000 7f25cf59e700 1 --2- 172.21.15.55:0/2002919389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d0150550 0x7f25d0157770 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f25d0156270 tx=0x7f25c400a3f0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.507+0000 7f25cf59e700 1 -- 172.21.15.55:0/2002919389 shutdown_connections 2023-10-06T04:49:15.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.507+0000 7f25cf59e700 1 --2- 172.21.15.55:0/2002919389 >> v2:172.21.15.163:3300/0 conn(0x7f25d00c7410 0x7f25d0069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.507+0000 7f25cf59e700 1 --2- 172.21.15.55:0/2002919389 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d0150550 0x7f25d0157770 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.507+0000 7f25cf59e700 1 -- 172.21.15.55:0/2002919389 >> 172.21.15.55:0/2002919389 conn(0x7f25d00cc1f0 msgr2=0x7f25d00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:15.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.507+0000 7f25cf59e700 1 -- 172.21.15.55:0/2002919389 shutdown_connections 2023-10-06T04:49:15.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.507+0000 7f25cf59e700 1 -- 172.21.15.55:0/2002919389 wait complete. 2023-10-06T04:49:15.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.508+0000 7f25cf59e700 1 Processor -- start 2023-10-06T04:49:15.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.508+0000 7f25cf59e700 1 -- start start 2023-10-06T04:49:15.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.509+0000 7f25cf59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00c7410 0x7f25d00ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:15.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.509+0000 7f25cf59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25d0150180 0x7f25d00c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:15.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.509+0000 7f25cf59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d0150550 0x7f25d014a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:15.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.509+0000 7f25cf59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00ad420 con 0x7f25d00c7410 2023-10-06T04:49:15.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.509+0000 7f25cf59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00ac810 con 0x7f25d0150550 2023-10-06T04:49:15.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.509+0000 7f25cf59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00acaf0 con 0x7f25d0150180 2023-10-06T04:49:15.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.509+0000 7f25ce59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00c7410 0x7f25d00ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:15.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.510+0000 7f25ce59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00c7410 0x7f25d00ac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52432/0 (socket says 172.21.15.55:52432) 2023-10-06T04:49:15.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.510+0000 7f25ce59c700 1 -- 172.21.15.55:0/4223642581 learned_addr learned my addr 172.21.15.55:0/4223642581 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:15.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.510+0000 7f25cdd9b700 1 --2- 172.21.15.55:0/4223642581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25d0150180 0x7f25d00c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:15.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.510+0000 7f25ced9d700 1 --2- 172.21.15.55:0/4223642581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d0150550 0x7f25d014a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:15.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.510+0000 7f25ce59c700 1 -- 172.21.15.55:0/4223642581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25d0150180 msgr2=0x7f25d00c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:15.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.510+0000 7f25ce59c700 1 --2- 172.21.15.55:0/4223642581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25d0150180 0x7f25d00c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.510+0000 7f25ce59c700 1 -- 172.21.15.55:0/4223642581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d0150550 msgr2=0x7f25d014a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:15.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.510+0000 7f25ce59c700 1 --2- 172.21.15.55:0/4223642581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d0150550 0x7f25d014a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.510+0000 7f25ce59c700 1 -- 172.21.15.55:0/4223642581 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c400a690 con 0x7f25d00c7410 2023-10-06T04:49:15.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.510+0000 7f25ce59c700 1 --2- 172.21.15.55:0/4223642581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00c7410 0x7f25d00ac230 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f25c000c5b0 tx=0x7f25c0001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:15.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.510+0000 7f25be7fc700 1 -- 172.21.15.55:0/4223642581 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c0005a70 con 0x7f25d00c7410 2023-10-06T04:49:15.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.510+0000 7f25be7fc700 1 -- 172.21.15.55:0/4223642581 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c0005bd0 con 0x7f25d00c7410 2023-10-06T04:49:15.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.510+0000 7f25cf59e700 1 -- 172.21.15.55:0/4223642581 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d00acdd0 con 0x7f25d00c7410 2023-10-06T04:49:15.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.510+0000 7f25be7fc700 1 -- 172.21.15.55:0/4223642581 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c00070d0 con 0x7f25d00c7410 2023-10-06T04:49:15.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.511+0000 7f25cf59e700 1 -- 172.21.15.55:0/4223642581 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d00ad8b0 con 0x7f25d00c7410 2023-10-06T04:49:15.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.512+0000 7f25cf59e700 1 -- 172.21.15.55:0/4223642581 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d004c3c0 con 0x7f25d00c7410 2023-10-06T04:49:15.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.512+0000 7f25be7fc700 1 -- 172.21.15.55:0/4223642581 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25c0007440 con 0x7f25d00c7410 2023-10-06T04:49:15.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.513+0000 7f25be7fc700 1 --2- 172.21.15.55:0/4223642581 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f25b0066f30 0x7f25b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:15.516 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.514+0000 7f25be7fc700 1 -- 172.21.15.55:0/4223642581 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f25c0083a50 con 0x7f25d00c7410 2023-10-06T04:49:15.517 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.516+0000 7f25cdd9b700 1 --2- 172.21.15.55:0/4223642581 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f25b0066f30 0x7f25b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:15.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.517+0000 7f25be7fc700 1 -- 172.21.15.55:0/4223642581 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25c0050c90 con 0x7f25d00c7410 2023-10-06T04:49:15.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.518+0000 7f25cdd9b700 1 --2- 172.21.15.55:0/4223642581 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f25b0066f30 0x7f25b00693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f25d0156270 tx=0x7f25c4018000 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:15.801 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.799+0000 7f25cf59e700 1 -- 172.21.15.55:0/4223642581 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f25d00ad700 con 0x7f25d00c7410 2023-10-06T04:49:15.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.800+0000 7f25be7fc700 1 -- 172.21.15.55:0/4223642581 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v39) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f25c0017020 con 0x7f25d00c7410 2023-10-06T04:49:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:15.802 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T04:49:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2023-10-06T04:49:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2023-10-06T04:49:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9li2RaGRAAgRjtbw3h40vylEPjolzlpw==", 2023-10-06T04:49:15.802 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:49:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T04:49:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T04:49:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2023-10-06T04:49:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lXZu7LxAAu3QAtdDxtD/AuuqGG12sDw==", 2023-10-06T04:49:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:49:15.803 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T04:49:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T04:49:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2023-10-06T04:49:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lS22fJBAAZi6CfyLyJAFgIkaarad1mA==", 2023-10-06T04:49:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:49:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T04:49:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T04:49:15.804 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2023-10-06T04:49:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9ll0RIOhAAGtGdPv4Y2bge7xIv3eu2NQ==", 2023-10-06T04:49:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:49:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T04:49:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T04:49:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.805 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2023-10-06T04:49:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDHfx9lmTeaDRAAuycc9i7fPj8xrDkuZCAGVg==", 2023-10-06T04:49:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:49:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:49:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2023-10-06T04:49:15.806 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDKfx9lxOdXEhAA9S8tdOvhnmBeEbutqnUsZg==", 2023-10-06T04:49:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:49:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:49:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2023-10-06T04:49:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDNfx9l76I+GRAAjaj0gM7An9BCyGmfQgaDFw==", 2023-10-06T04:49:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:49:15.807 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:49:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2023-10-06T04:49:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDQfx9lsPK+HxAAzQCV0EuBenqOkUpiQW9uog==", 2023-10-06T04:49:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:49:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:49:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.808 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2023-10-06T04:49:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDWfx9lSHNrMxAAIn7awDmljO2cRtKyDalWnw==", 2023-10-06T04:49:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:49:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:49:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2023-10-06T04:49:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDZfx9lFoWuOBAAhzQOt3YChvuv0MhrspOjPw==", 2023-10-06T04:49:15.809 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:49:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:49:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2023-10-06T04:49:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDcfx9lB8iSOhAAwamSgGIXZ8owFE4FdbOOBw==", 2023-10-06T04:49:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:49:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:49:15.810 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2023-10-06T04:49:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDgfx9lUb3mABAAJ75UO2UQg2r+l+tDYVn0Dg==", 2023-10-06T04:49:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T04:49:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T04:49:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.811 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2023-10-06T04:49:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lfNvCBxAA1ZJq4XsWTZHVY0u0NdOqIw==", 2023-10-06T04:49:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:49:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2023-10-06T04:49:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T04:49:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2023-10-06T04:49:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lnvkoFhAAafxsOmItxmwUr1GJrGYJ6g==", 2023-10-06T04:49:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.812 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:49:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T04:49:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T04:49:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T04:49:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2023-10-06T04:49:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9l+9b4DhAAc5XHdikWbXsEblSE3FGScQ==", 2023-10-06T04:49:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:49:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T04:49:15.813 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T04:49:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T04:49:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2023-10-06T04:49:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9luSXrHBAAg6Ufa1Ep/EOgsQtuOPQgYg==", 2023-10-06T04:49:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T04:49:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T04:49:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T04:49:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T04:49:15.814 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2023-10-06T04:49:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC+fx9lEb50FxAAt0Oq28gsXtQvol7TV8trgg==", 2023-10-06T04:49:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T04:49:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T04:49:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T04:49:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.815 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2023-10-06T04:49:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lQAANEhAAsNiWPqCNEyUS3IELXSC1Cw==", 2023-10-06T04:49:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T04:49:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T04:49:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2023-10-06T04:49:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lCwU2ABAAMijxXd8hwgk3XzcdP4NZyg==", 2023-10-06T04:49:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T04:49:15.816 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T04:49:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T04:49:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T04:49:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:15.817 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2023-10-06T04:49:15.817 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T04:49:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.802+0000 7f25cf59e700 1 -- 172.21.15.55:0/4223642581 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f25b0066f30 msgr2=0x7f25b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.802+0000 7f25cf59e700 1 --2- 172.21.15.55:0/4223642581 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f25b0066f30 0x7f25b00693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f25d0156270 tx=0x7f25c4018000 comp rx=0 tx=0).stop 2023-10-06T04:49:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.802+0000 7f25cf59e700 1 -- 172.21.15.55:0/4223642581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00c7410 msgr2=0x7f25d00ac230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.802+0000 7f25cf59e700 1 --2- 172.21.15.55:0/4223642581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00c7410 0x7f25d00ac230 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f25c000c5b0 tx=0x7f25c0001040 comp rx=0 tx=0).stop 2023-10-06T04:49:15.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.802+0000 7f25cf59e700 1 -- 172.21.15.55:0/4223642581 shutdown_connections 2023-10-06T04:49:15.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.802+0000 7f25cf59e700 1 --2- 172.21.15.55:0/4223642581 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f25b0066f30 0x7f25b00693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.802+0000 7f25cf59e700 1 --2- 172.21.15.55:0/4223642581 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25d0150550 0x7f25d014a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.802+0000 7f25cf59e700 1 --2- 172.21.15.55:0/4223642581 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25d00c7410 0x7f25d00ac230 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.802+0000 7f25cf59e700 1 --2- 172.21.15.55:0/4223642581 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25d0150180 0x7f25d00c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:15.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.802+0000 7f25cf59e700 1 -- 172.21.15.55:0/4223642581 >> 172.21.15.55:0/4223642581 conn(0x7f25d00cc1f0 msgr2=0x7f25d00696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:15.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.802+0000 7f25cf59e700 1 -- 172.21.15.55:0/4223642581 shutdown_connections 2023-10-06T04:49:15.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:15.802+0000 7f25cf59e700 1 -- 172.21.15.55:0/4223642581 wait complete. 2023-10-06T04:49:15.822 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:49:16.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.008+0000 7f8c7939d700 1 Processor -- start 2023-10-06T04:49:16.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.008+0000 7f8c7939d700 1 -- start start 2023-10-06T04:49:16.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.009+0000 7f8c7939d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8c74150180 0x7f8c74157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.010+0000 7f8c7939d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c740c7410 0x7f8c74069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.010+0000 7f8c7939d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7406e2a0 con 0x7f8c74150550 2023-10-06T04:49:16.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.010+0000 7f8c7939d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c7406b2e0 con 0x7f8c74150180 2023-10-06T04:49:16.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.010+0000 7f8c7939d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c740c78e0 con 0x7f8c740c7410 2023-10-06T04:49:16.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.010+0000 7f8c737fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8c74150550 0x7f8c740c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37966/0 (socket says 172.21.15.55:37966) 2023-10-06T04:49:16.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.010+0000 7f8c737fe700 1 -- 172.21.15.55:0/133719138 learned_addr learned my addr 172.21.15.55:0/133719138 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:16.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.010+0000 7f8c73fff700 1 --2- 172.21.15.55:0/133719138 >> v2:172.21.15.163:3300/0 conn(0x7f8c74150180 0x7f8c74157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.010+0000 7f8c78b9c700 1 --2- 172.21.15.55:0/133719138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c740c7410 0x7f8c74069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.010+0000 7f8c72ffd700 1 -- 172.21.15.55:0/133719138 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933323696 0 0) 0x7f8c7406e2a0 con 0x7f8c74150550 2023-10-06T04:49:16.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.010+0000 7f8c72ffd700 1 -- 172.21.15.55:0/133719138 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c5c003570 con 0x7f8c74150550 2023-10-06T04:49:16.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.011+0000 7f8c73fff700 1 -- 172.21.15.55:0/133719138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c740c7410 msgr2=0x7f8c74069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:16.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.011+0000 7f8c73fff700 1 --2- 172.21.15.55:0/133719138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c740c7410 0x7f8c74069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.011+0000 7f8c73fff700 1 -- 172.21.15.55:0/133719138 >> v1:172.21.15.55:6789/0 conn(0x7f8c74150550 legacy=0x7f8c740c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.011+0000 7f8c73fff700 1 -- 172.21.15.55:0/133719138 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c740c7bc0 con 0x7f8c74150180 2023-10-06T04:49:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.011+0000 7f8c73fff700 1 --2- 172.21.15.55:0/133719138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c74150180 0x7f8c74157770 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8c640098e0 tx=0x7f8c640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0a24fb7fdb9ebbe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.011+0000 7f8c72ffd700 1 -- 172.21.15.55:0/133719138 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64011630 con 0x7f8c74150180 2023-10-06T04:49:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.011+0000 7f8c72ffd700 1 -- 172.21.15.55:0/133719138 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c64011790 con 0x7f8c74150180 2023-10-06T04:49:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.011+0000 7f8c72ffd700 1 -- 172.21.15.55:0/133719138 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64011b10 con 0x7f8c74150180 2023-10-06T04:49:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.012+0000 7f8c7939d700 1 -- 172.21.15.55:0/133719138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c74150180 msgr2=0x7f8c74157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.012+0000 7f8c7939d700 1 --2- 172.21.15.55:0/133719138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c74150180 0x7f8c74157770 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8c640098e0 tx=0x7f8c640091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.012+0000 7f8c7939d700 1 -- 172.21.15.55:0/133719138 shutdown_connections 2023-10-06T04:49:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.012+0000 7f8c7939d700 1 --2- 172.21.15.55:0/133719138 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c740c7410 0x7f8c74069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.012+0000 7f8c7939d700 1 --2- 172.21.15.55:0/133719138 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c74150180 0x7f8c74157770 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.012+0000 7f8c7939d700 1 -- 172.21.15.55:0/133719138 >> 172.21.15.55:0/133719138 conn(0x7f8c740cc1f0 msgr2=0x7f8c740cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:16.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.012+0000 7f8c7939d700 1 -- 172.21.15.55:0/133719138 shutdown_connections 2023-10-06T04:49:16.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.012+0000 7f8c7939d700 1 -- 172.21.15.55:0/133719138 wait complete. 2023-10-06T04:49:16.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.013+0000 7f8c7939d700 1 Processor -- start 2023-10-06T04:49:16.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.014+0000 7f8c7939d700 1 -- start start 2023-10-06T04:49:16.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.014+0000 7f8c7939d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c740c7410 0x7f8c740c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.014+0000 7f8c7939d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c74150180 0x7f8c74144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.014+0000 7f8c7939d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c74150550 0x7f8c740adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.014+0000 7f8c7939d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c740abab0 con 0x7f8c740c7410 2023-10-06T04:49:16.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.015+0000 7f8c7939d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c740ac630 con 0x7f8c74150550 2023-10-06T04:49:16.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.015+0000 7f8c7939d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c740abd00 con 0x7f8c74150180 2023-10-06T04:49:16.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.015+0000 7f8c73fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c740c7410 0x7f8c740c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.015+0000 7f8c73fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c740c7410 0x7f8c740c3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52436/0 (socket says 172.21.15.55:52436) 2023-10-06T04:49:16.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.015+0000 7f8c73fff700 1 -- 172.21.15.55:0/3808900707 learned_addr learned my addr 172.21.15.55:0/3808900707 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:16.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.015+0000 7f8c73fff700 1 -- 172.21.15.55:0/3808900707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c74150180 msgr2=0x7f8c74144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.015+0000 7f8c73fff700 1 --2- 172.21.15.55:0/3808900707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c74150180 0x7f8c74144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.015+0000 7f8c73fff700 1 -- 172.21.15.55:0/3808900707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c74150550 msgr2=0x7f8c740adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.015+0000 7f8c73fff700 1 --2- 172.21.15.55:0/3808900707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c74150550 0x7f8c740adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.015+0000 7f8c73fff700 1 -- 172.21.15.55:0/3808900707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c64009420 con 0x7f8c740c7410 2023-10-06T04:49:16.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.015+0000 7f8c73fff700 1 --2- 172.21.15.55:0/3808900707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c740c7410 0x7f8c740c3db0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8c7406fe40 tx=0x7f8c64017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:16.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.016+0000 7f8c5bfff700 1 -- 172.21.15.55:0/3808900707 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64011900 con 0x7f8c740c7410 2023-10-06T04:49:16.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.016+0000 7f8c5bfff700 1 -- 172.21.15.55:0/3808900707 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c64011a60 con 0x7f8c740c7410 2023-10-06T04:49:16.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.016+0000 7f8c5bfff700 1 -- 172.21.15.55:0/3808900707 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64025670 con 0x7f8c740c7410 2023-10-06T04:49:16.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.016+0000 7f8c7939d700 1 -- 172.21.15.55:0/3808900707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c740abfe0 con 0x7f8c740c7410 2023-10-06T04:49:16.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.016+0000 7f8c7939d700 1 -- 172.21.15.55:0/3808900707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c740c42f0 con 0x7f8c740c7410 2023-10-06T04:49:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.017+0000 7f8c7939d700 1 -- 172.21.15.55:0/3808900707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7404c3c0 con 0x7f8c740c7410 2023-10-06T04:49:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.018+0000 7f8c5bfff700 1 -- 172.21.15.55:0/3808900707 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c64023070 con 0x7f8c740c7410 2023-10-06T04:49:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.018+0000 7f8c5bfff700 1 --2- 172.21.15.55:0/3808900707 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f8c60066f30 0x7f8c600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.019+0000 7f8c5bfff700 1 -- 172.21.15.55:0/3808900707 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f8c64085ad0 con 0x7f8c740c7410 2023-10-06T04:49:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.019+0000 7f8c737fe700 1 --2- 172.21.15.55:0/3808900707 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f8c60066f30 0x7f8c600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.020+0000 7f8c737fe700 1 --2- 172.21.15.55:0/3808900707 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f8c60066f30 0x7f8c600693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8c7414f740 tx=0x7f8c680079d0 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:16.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.023+0000 7f8c5bfff700 1 -- 172.21.15.55:0/3808900707 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c64052d10 con 0x7f8c740c7410 2023-10-06T04:49:16.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.225+0000 7f8c7939d700 1 -- 172.21.15.55:0/3808900707 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c7406d650 con 0x7f8c740c7410 2023-10-06T04:49:16.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.226+0000 7f8c5bfff700 1 -- 172.21.15.55:0/3808900707 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+14303 (secure 0 0 0) 0x7f8c64056330 con 0x7f8c740c7410 2023-10-06T04:49:16.228 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:16.228 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":169,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T04:49:15.296584+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":158,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:49:16.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.228+0000 7f8c7939d700 1 -- 172.21.15.55:0/3808900707 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f8c60066f30 msgr2=0x7f8c600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.228+0000 7f8c7939d700 1 --2- 172.21.15.55:0/3808900707 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f8c60066f30 0x7f8c600693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8c7414f740 tx=0x7f8c680079d0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.228+0000 7f8c7939d700 1 -- 172.21.15.55:0/3808900707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c740c7410 msgr2=0x7f8c740c3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.228+0000 7f8c7939d700 1 --2- 172.21.15.55:0/3808900707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c740c7410 0x7f8c740c3db0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8c7406fe40 tx=0x7f8c64017000 comp rx=0 tx=0).stop 2023-10-06T04:49:16.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.228+0000 7f8c7939d700 1 -- 172.21.15.55:0/3808900707 shutdown_connections 2023-10-06T04:49:16.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.228+0000 7f8c7939d700 1 --2- 172.21.15.55:0/3808900707 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f8c60066f30 0x7f8c600693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.228+0000 7f8c7939d700 1 --2- 172.21.15.55:0/3808900707 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8c74150550 0x7f8c740adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.228+0000 7f8c7939d700 1 --2- 172.21.15.55:0/3808900707 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8c740c7410 0x7f8c740c3db0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.228+0000 7f8c7939d700 1 --2- 172.21.15.55:0/3808900707 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8c74150180 0x7f8c74144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.228+0000 7f8c7939d700 1 -- 172.21.15.55:0/3808900707 >> 172.21.15.55:0/3808900707 conn(0x7f8c740cc1f0 msgr2=0x7f8c740c8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:16.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.229+0000 7f8c7939d700 1 -- 172.21.15.55:0/3808900707 shutdown_connections 2023-10-06T04:49:16.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.229+0000 7f8c7939d700 1 -- 172.21.15.55:0/3808900707 wait complete. 2023-10-06T04:49:16.248 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T04:49:16.248 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:49:16.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.440+0000 7fad3df9d700 1 Processor -- start 2023-10-06T04:49:16.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.440+0000 7fad3df9d700 1 -- start start 2023-10-06T04:49:16.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.441+0000 7fad3df9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fad3806f060 0x7fad380ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.441+0000 7fad3df9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad3806ebd0 con 0x7fad3806f4b0 2023-10-06T04:49:16.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.441+0000 7fad3df9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad3815aa70 con 0x7fad380cb1a0 2023-10-06T04:49:16.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.441+0000 7fad3df9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad3815b2e0 con 0x7fad3806f060 2023-10-06T04:49:16.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.441+0000 7fad3d79c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fad3806f4b0 0x7fad380c9db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37970/0 (socket says 172.21.15.55:37970) 2023-10-06T04:49:16.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.441+0000 7fad3d79c700 1 -- 172.21.15.55:0/4120855352 learned_addr learned my addr 172.21.15.55:0/4120855352 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad37fff700 1 --2- 172.21.15.55:0/4120855352 >> v2:172.21.15.163:3300/0 conn(0x7fad3806f060 0x7fad380ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 -- 172.21.15.55:0/4120855352 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65160114 0 0) 0x7fad3806ebd0 con 0x7fad3806f4b0 2023-10-06T04:49:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 -- 172.21.15.55:0/4120855352 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad28003570 con 0x7fad3806f4b0 2023-10-06T04:49:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 -- 172.21.15.55:0/4120855352 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2937338496 0 0) 0x7fad28003570 con 0x7fad3806f4b0 2023-10-06T04:49:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 -- 172.21.15.55:0/4120855352 >> v2:172.21.15.163:3300/0 conn(0x7fad3806f060 msgr2=0x7fad380ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:16.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 --2- 172.21.15.55:0/4120855352 >> v2:172.21.15.163:3300/0 conn(0x7fad3806f060 0x7fad380ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 -- 172.21.15.55:0/4120855352 >> v1:172.21.15.163:6789/0 conn(0x7fad380cb1a0 legacy=0x7fad3815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 -- 172.21.15.55:0/4120855352 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3815ad20 con 0x7fad3806f4b0 2023-10-06T04:49:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 -- 172.21.15.55:0/4120855352 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fad300017f0 con 0x7fad3806f4b0 2023-10-06T04:49:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 -- 172.21.15.55:0/4120855352 >> v1:172.21.15.55:6789/0 conn(0x7fad3806f4b0 legacy=0x7fad380c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 --2- 172.21.15.55:0/4120855352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28003a60 0x7fad28003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 --2- 172.21.15.55:0/4120855352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad28004810 0x7fad28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 --2- 172.21.15.55:0/4120855352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad280051c0 0x7fad2800dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 -- 172.21.15.55:0/4120855352 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3815b2e0 con 0x7fad28003a60 2023-10-06T04:49:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 -- 172.21.15.55:0/4120855352 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2800e260 con 0x7fad280051c0 2023-10-06T04:49:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.442+0000 7fad377fe700 1 -- 172.21.15.55:0/4120855352 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2800e420 con 0x7fad28004810 2023-10-06T04:49:16.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.443+0000 7fad3cf9b700 1 --2- 172.21.15.55:0/4120855352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28003a60 0x7fad28003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.443+0000 7fad3d79c700 1 --2- 172.21.15.55:0/4120855352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad280051c0 0x7fad2800dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.443+0000 7fad3cf9b700 1 -- 172.21.15.55:0/4120855352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad28004810 msgr2=0x7fad28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.443+0000 7fad3cf9b700 1 --2- 172.21.15.55:0/4120855352 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad28004810 0x7fad28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.443+0000 7fad3cf9b700 1 -- 172.21.15.55:0/4120855352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad280051c0 msgr2=0x7fad2800dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.443+0000 7fad3cf9b700 1 --2- 172.21.15.55:0/4120855352 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad280051c0 0x7fad2800dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.443+0000 7fad3cf9b700 1 -- 172.21.15.55:0/4120855352 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad28003570 con 0x7fad28003a60 2023-10-06T04:49:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.443+0000 7fad3d79c700 1 -- 172.21.15.55:0/4120855352 reap_dead start 2023-10-06T04:49:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.443+0000 7fad3cf9b700 1 --2- 172.21.15.55:0/4120855352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28003a60 0x7fad28003e30 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fad2c001a60 tx=0x7fad2c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f250c86d61f0536 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.444+0000 7fad377fe700 1 -- 172.21.15.55:0/4120855352 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c001ea0 con 0x7fad28003a60 2023-10-06T04:49:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.444+0000 7fad377fe700 1 -- 172.21.15.55:0/4120855352 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c00c590 con 0x7fad28003a60 2023-10-06T04:49:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.444+0000 7fad377fe700 1 -- 172.21.15.55:0/4120855352 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c00c930 con 0x7fad28003a60 2023-10-06T04:49:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.444+0000 7fad3df9d700 1 -- 172.21.15.55:0/4120855352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28003a60 msgr2=0x7fad28003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.444+0000 7fad3df9d700 1 --2- 172.21.15.55:0/4120855352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28003a60 0x7fad28003e30 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fad2c001a60 tx=0x7fad2c007d30 comp rx=0 tx=0).stop 2023-10-06T04:49:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.444+0000 7fad3df9d700 1 -- 172.21.15.55:0/4120855352 shutdown_connections 2023-10-06T04:49:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.444+0000 7fad3df9d700 1 --2- 172.21.15.55:0/4120855352 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28003a60 0x7fad28003e30 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.444+0000 7fad3df9d700 1 -- 172.21.15.55:0/4120855352 >> 172.21.15.55:0/4120855352 conn(0x7fad380cc1e0 msgr2=0x7fad380cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.444+0000 7fad3df9d700 1 -- 172.21.15.55:0/4120855352 shutdown_connections 2023-10-06T04:49:16.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.444+0000 7fad3df9d700 1 -- 172.21.15.55:0/4120855352 wait complete. 2023-10-06T04:49:16.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.446+0000 7fad3df9d700 1 Processor -- start 2023-10-06T04:49:16.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.446+0000 7fad3df9d700 1 -- start start 2023-10-06T04:49:16.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.447+0000 7fad3df9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28003a60 0x7fad3816a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.447+0000 7fad3df9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad38149e60 0x7fad3816d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.447+0000 7fad3df9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad3816d630 0x7fad380c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.447+0000 7fad3df9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3816e090 con 0x7fad28003a60 2023-10-06T04:49:16.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.447+0000 7fad3df9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad38169af0 con 0x7fad3816d630 2023-10-06T04:49:16.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.447+0000 7fad3df9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3816dad0 con 0x7fad38149e60 2023-10-06T04:49:16.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.447+0000 7fad3cf9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28003a60 0x7fad3816a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.447+0000 7fad3cf9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28003a60 0x7fad3816a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52464/0 (socket says 172.21.15.55:52464) 2023-10-06T04:49:16.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.447+0000 7fad3cf9b700 1 -- 172.21.15.55:0/1503820737 learned_addr learned my addr 172.21.15.55:0/1503820737 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:16.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.447+0000 7fad37fff700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad38149e60 0x7fad3816d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.447+0000 7fad3d79c700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad3816d630 0x7fad380c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.447+0000 7fad3cf9b700 1 -- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad38149e60 msgr2=0x7fad3816d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.448+0000 7fad3cf9b700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad38149e60 0x7fad3816d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.448+0000 7fad3cf9b700 1 -- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad3816d630 msgr2=0x7fad380c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.448+0000 7fad3cf9b700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad3816d630 0x7fad380c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.448+0000 7fad3cf9b700 1 -- 172.21.15.55:0/1503820737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3816ddb0 con 0x7fad28003a60 2023-10-06T04:49:16.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.448+0000 7fad3d79c700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad3816d630 0x7fad380c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:49:16.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.448+0000 7fad3cf9b700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28003a60 0x7fad3816a500 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fad2c001500 tx=0x7fad2c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:16.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.448+0000 7fad34ff9700 1 -- 172.21.15.55:0/1503820737 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c001ea0 con 0x7fad28003a60 2023-10-06T04:49:16.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.448+0000 7fad34ff9700 1 -- 172.21.15.55:0/1503820737 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c00cca0 con 0x7fad28003a60 2023-10-06T04:49:16.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.448+0000 7fad34ff9700 1 -- 172.21.15.55:0/1503820737 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c006820 con 0x7fad28003a60 2023-10-06T04:49:16.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.448+0000 7fad3df9d700 1 -- 172.21.15.55:0/1503820737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad38169d00 con 0x7fad28003a60 2023-10-06T04:49:16.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.448+0000 7fad3df9d700 1 -- 172.21.15.55:0/1503820737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3816aa40 con 0x7fad28003a60 2023-10-06T04:49:16.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.449+0000 7fad3df9d700 1 -- 172.21.15.55:0/1503820737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3806c450 con 0x7fad28003a60 2023-10-06T04:49:16.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.450+0000 7fad34ff9700 1 -- 172.21.15.55:0/1503820737 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad2c006c10 con 0x7fad28003a60 2023-10-06T04:49:16.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.451+0000 7fad34ff9700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fad1c066f30 0x7fad1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.451+0000 7fad34ff9700 1 -- 172.21.15.55:0/1503820737 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fad2c084fd0 con 0x7fad28003a60 2023-10-06T04:49:16.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.452+0000 7fad37fff700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fad1c066f30 0x7fad1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.452+0000 7fad37fff700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fad1c066f30 0x7fad1c0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fad380c83e0 tx=0x7fad240091a0 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:16.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.455+0000 7fad34ff9700 1 -- 172.21.15.55:0/1503820737 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad2c052370 con 0x7fad28003a60 2023-10-06T04:49:16.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.657+0000 7fad3df9d700 1 -- 172.21.15.55:0/1503820737 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fad3815b000 con 0x7fad28003a60 2023-10-06T04:49:16.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.658+0000 7fad34ff9700 1 -- 172.21.15.55:0/1503820737 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v169) v1 ==== 92+0+10 (secure 0 0 0) 0x7fad2c017070 con 0x7fad28003a60 2023-10-06T04:49:16.660 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T04:49:16.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.660+0000 7fad3df9d700 1 -- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fad1c066f30 msgr2=0x7fad1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.660+0000 7fad3df9d700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fad1c066f30 0x7fad1c0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fad380c83e0 tx=0x7fad240091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.660+0000 7fad3df9d700 1 -- 172.21.15.55:0/1503820737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28003a60 msgr2=0x7fad3816a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.660+0000 7fad3df9d700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28003a60 0x7fad3816a500 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fad2c001500 tx=0x7fad2c015000 comp rx=0 tx=0).stop 2023-10-06T04:49:16.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.660+0000 7fad3df9d700 1 -- 172.21.15.55:0/1503820737 shutdown_connections 2023-10-06T04:49:16.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.660+0000 7fad3df9d700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fad1c066f30 0x7fad1c0693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.660+0000 7fad3df9d700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad3816d630 0x7fad380c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.660+0000 7fad3df9d700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad28003a60 0x7fad3816a500 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.660+0000 7fad3df9d700 1 --2- 172.21.15.55:0/1503820737 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad38149e60 0x7fad3816d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.660+0000 7fad3df9d700 1 -- 172.21.15.55:0/1503820737 >> 172.21.15.55:0/1503820737 conn(0x7fad380cc1e0 msgr2=0x7fad380c0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:16.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.660+0000 7fad3df9d700 1 -- 172.21.15.55:0/1503820737 shutdown_connections 2023-10-06T04:49:16.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.660+0000 7fad3df9d700 1 -- 172.21.15.55:0/1503820737 wait complete. 2023-10-06T04:49:16.679 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:49:16.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.868+0000 7faffd39d700 1 Processor -- start 2023-10-06T04:49:16.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.868+0000 7faffd39d700 1 -- start start 2023-10-06T04:49:16.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.869+0000 7faffd39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff8153900 0x7faff80c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.869+0000 7faffd39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7faff80c9db0 0x7faff806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.869+0000 7faffd39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff806e150 con 0x7faff8157130 2023-10-06T04:49:16.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.869+0000 7faffd39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff80c7800 con 0x7faff80c9db0 2023-10-06T04:49:16.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.869+0000 7faffd39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff80c7ae0 con 0x7faff8153900 2023-10-06T04:49:16.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.869+0000 7faffcb9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7faff8157130 0x7faff8159ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37976/0 (socket says 172.21.15.55:37976) 2023-10-06T04:49:16.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.869+0000 7faffcb9c700 1 -- 172.21.15.55:0/2983311173 learned_addr learned my addr 172.21.15.55:0/2983311173 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:16.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.870+0000 7faff7fff700 1 --2- 172.21.15.55:0/2983311173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff8153900 0x7faff80c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.870+0000 7faff77fe700 1 --2- 172.21.15.55:0/2983311173 >> v2:172.21.15.163:3300/0 conn(0x7faff80c9db0 0x7faff806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.870+0000 7faff6ffd700 1 -- 172.21.15.55:0/2983311173 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3051224909 0 0) 0x7faff806e150 con 0x7faff8157130 2023-10-06T04:49:16.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.870+0000 7faff6ffd700 1 -- 172.21.15.55:0/2983311173 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafe0003570 con 0x7faff8157130 2023-10-06T04:49:16.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.870+0000 7faff7fff700 1 -- 172.21.15.55:0/2983311173 >> v2:172.21.15.163:3300/0 conn(0x7faff80c9db0 msgr2=0x7faff806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:16.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.870+0000 7faff7fff700 1 --2- 172.21.15.55:0/2983311173 >> v2:172.21.15.163:3300/0 conn(0x7faff80c9db0 0x7faff806a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.870+0000 7faff7fff700 1 -- 172.21.15.55:0/2983311173 >> v1:172.21.15.55:6789/0 conn(0x7faff8157130 legacy=0x7faff8159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.870+0000 7faff7fff700 1 -- 172.21.15.55:0/2983311173 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff80c7dc0 con 0x7faff8153900 2023-10-06T04:49:16.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.870+0000 7faff77fe700 1 --2- 172.21.15.55:0/2983311173 >> v2:172.21.15.163:3300/0 conn(0x7faff80c9db0 0x7faff806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:49:16.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.870+0000 7faff7fff700 1 --2- 172.21.15.55:0/2983311173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff8153900 0x7faff80c6f00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fafe80098e0 tx=0x7fafe80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f020b84e2bbcb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:16.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.871+0000 7faff6ffd700 1 -- 172.21.15.55:0/2983311173 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8011630 con 0x7faff8153900 2023-10-06T04:49:16.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.871+0000 7faff6ffd700 1 -- 172.21.15.55:0/2983311173 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe8011790 con 0x7faff8153900 2023-10-06T04:49:16.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.871+0000 7faff6ffd700 1 -- 172.21.15.55:0/2983311173 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8011b30 con 0x7faff8153900 2023-10-06T04:49:16.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.871+0000 7faffd39d700 1 -- 172.21.15.55:0/2983311173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff8153900 msgr2=0x7faff80c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.871+0000 7faffd39d700 1 --2- 172.21.15.55:0/2983311173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff8153900 0x7faff80c6f00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fafe80098e0 tx=0x7fafe80091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.871+0000 7faffd39d700 1 -- 172.21.15.55:0/2983311173 shutdown_connections 2023-10-06T04:49:16.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.871+0000 7faffd39d700 1 --2- 172.21.15.55:0/2983311173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff8153900 0x7faff80c6f00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.871+0000 7faffd39d700 1 --2- 172.21.15.55:0/2983311173 >> v2:172.21.15.163:3300/0 conn(0x7faff80c9db0 0x7faff806a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.871+0000 7faffd39d700 1 -- 172.21.15.55:0/2983311173 >> 172.21.15.55:0/2983311173 conn(0x7faff80cc1e0 msgr2=0x7faff80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:16.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.871+0000 7faffd39d700 1 -- 172.21.15.55:0/2983311173 shutdown_connections 2023-10-06T04:49:16.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.871+0000 7faffd39d700 1 -- 172.21.15.55:0/2983311173 wait complete. 2023-10-06T04:49:16.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.873+0000 7faffd39d700 1 Processor -- start 2023-10-06T04:49:16.880 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.873+0000 7faffd39d700 1 -- start start 2023-10-06T04:49:16.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.874+0000 7faffd39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff80c9db0 0x7faff8151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.874+0000 7faffd39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faff8153900 0x7faff80ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.874+0000 7faffd39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff8157130 0x7faff8158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.874+0000 7faffd39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff8152680 con 0x7faff80c9db0 2023-10-06T04:49:16.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.874+0000 7faffd39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff8150d80 con 0x7faff8153900 2023-10-06T04:49:16.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.874+0000 7faffd39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff816a8d0 con 0x7faff8157130 2023-10-06T04:49:16.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.874+0000 7faff7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff80c9db0 0x7faff8151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.874+0000 7faff7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff80c9db0 0x7faff8151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52472/0 (socket says 172.21.15.55:52472) 2023-10-06T04:49:16.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.874+0000 7faff7fff700 1 -- 172.21.15.55:0/1752113765 learned_addr learned my addr 172.21.15.55:0/1752113765 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:16.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7faff7fff700 1 -- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff8157130 msgr2=0x7faff8158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7faffcb9c700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff8157130 0x7faff8158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7faff77fe700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faff8153900 0x7faff80ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7faff7fff700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff8157130 0x7faff8158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7faff7fff700 1 -- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faff8153900 msgr2=0x7faff80ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:16.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7faff7fff700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faff8153900 0x7faff80ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:16.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7faff7fff700 1 -- 172.21.15.55:0/1752113765 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec009420 con 0x7faff80c9db0 2023-10-06T04:49:16.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7faff77fe700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faff8153900 0x7faff80ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:49:16.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7faffcb9c700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff8157130 0x7faff8158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:49:16.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7faff7fff700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff80c9db0 0x7faff8151210 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7faff806f5f0 tx=0x7fafe8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:16.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7fafdffff700 1 -- 172.21.15.55:0/1752113765 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8011b10 con 0x7faff80c9db0 2023-10-06T04:49:16.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7fafdffff700 1 -- 172.21.15.55:0/1752113765 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe80253f0 con 0x7faff80c9db0 2023-10-06T04:49:16.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7faffd39d700 1 -- 172.21.15.55:0/1752113765 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe8009420 con 0x7faff80c9db0 2023-10-06T04:49:16.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7fafdffff700 1 -- 172.21.15.55:0/1752113765 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe80257b0 con 0x7faff80c9db0 2023-10-06T04:49:16.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.875+0000 7faffd39d700 1 -- 172.21.15.55:0/1752113765 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff8150a10 con 0x7faff80c9db0 2023-10-06T04:49:16.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.876+0000 7faffd39d700 1 -- 172.21.15.55:0/1752113765 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff8151060 con 0x7faff80c9db0 2023-10-06T04:49:16.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.877+0000 7fafdffff700 1 -- 172.21.15.55:0/1752113765 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fafe8025ba0 con 0x7faff80c9db0 2023-10-06T04:49:16.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.878+0000 7fafdffff700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fafe4066f30 0x7fafe40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:16.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.878+0000 7fafdffff700 1 -- 172.21.15.55:0/1752113765 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fafe8085dd0 con 0x7faff80c9db0 2023-10-06T04:49:16.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.879+0000 7faff77fe700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fafe4066f30 0x7fafe40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:16.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.880+0000 7faff77fe700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fafe4066f30 0x7fafe40693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fafec0093f0 tx=0x7fafec008040 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:16.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:16.883+0000 7fafdffff700 1 -- 172.21.15.55:0/1752113765 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fafe8053010 con 0x7faff80c9db0 2023-10-06T04:49:17.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.082+0000 7faffd39d700 1 -- 172.21.15.55:0/1752113765 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faff806dd20 con 0x7faff80c9db0 2023-10-06T04:49:17.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.083+0000 7fafdffff700 1 -- 172.21.15.55:0/1752113765 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v169) v1 ==== 93+0+10 (secure 0 0 0) 0x7fafe8056630 con 0x7faff80c9db0 2023-10-06T04:49:17.085 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T04:49:17.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.085+0000 7faffd39d700 1 -- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fafe4066f30 msgr2=0x7fafe40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:17.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.085+0000 7faffd39d700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fafe4066f30 0x7fafe40693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fafec0093f0 tx=0x7fafec008040 comp rx=0 tx=0).stop 2023-10-06T04:49:17.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.085+0000 7faffd39d700 1 -- 172.21.15.55:0/1752113765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff80c9db0 msgr2=0x7faff8151210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:17.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.085+0000 7faffd39d700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff80c9db0 0x7faff8151210 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7faff806f5f0 tx=0x7fafe8017000 comp rx=0 tx=0).stop 2023-10-06T04:49:17.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.085+0000 7faffd39d700 1 -- 172.21.15.55:0/1752113765 shutdown_connections 2023-10-06T04:49:17.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.085+0000 7faffd39d700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fafe4066f30 0x7fafe40693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.085+0000 7faffd39d700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faff8157130 0x7faff8158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.085+0000 7faffd39d700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faff80c9db0 0x7faff8151210 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.085+0000 7faffd39d700 1 --2- 172.21.15.55:0/1752113765 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faff8153900 0x7faff80ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.085+0000 7faffd39d700 1 -- 172.21.15.55:0/1752113765 >> 172.21.15.55:0/1752113765 conn(0x7faff80cc1e0 msgr2=0x7faff814fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:17.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.085+0000 7faffd39d700 1 -- 172.21.15.55:0/1752113765 shutdown_connections 2023-10-06T04:49:17.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.086+0000 7faffd39d700 1 -- 172.21.15.55:0/1752113765 wait complete. 2023-10-06T04:49:17.105 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T04:49:17.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.290+0000 7fa8fb39d700 1 Processor -- start 2023-10-06T04:49:17.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.290+0000 7fa8fb39d700 1 -- start start 2023-10-06T04:49:17.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.291+0000 7fa8fb39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa8f406d950 0x7fa8f40c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:17.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.291+0000 7fa8fb39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8f406a0a0 0x7fa8f40c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:17.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.291+0000 7fa8fb39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8f406dd20 0x7fa8f40c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:17.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.291+0000 7fa8fb39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f406ae70 con 0x7fa8f406d950 2023-10-06T04:49:17.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.291+0000 7fa8fb39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f406a540 con 0x7fa8f406dd20 2023-10-06T04:49:17.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.291+0000 7fa8fb39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f406a820 con 0x7fa8f406a0a0 2023-10-06T04:49:17.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.291+0000 7fa8fa39b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa8f406d950 0x7fa8f40c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:17.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.291+0000 7fa8fa39b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa8f406d950 0x7fa8f40c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52476/0 (socket says 172.21.15.55:52476) 2023-10-06T04:49:17.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.291+0000 7fa8fa39b700 1 -- 172.21.15.55:0/1207914745 learned_addr learned my addr 172.21.15.55:0/1207914745 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:17.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.291+0000 7fa8f9b9a700 1 --2- 172.21.15.55:0/1207914745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8f406a0a0 0x7fa8f40c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:17.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.292+0000 7fa8fab9c700 1 --2- 172.21.15.55:0/1207914745 >> v2:172.21.15.163:3300/0 conn(0x7fa8f406dd20 0x7fa8f40c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:17.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.292+0000 7fa8fa39b700 1 -- 172.21.15.55:0/1207914745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8f406a0a0 msgr2=0x7fa8f40c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:17.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.292+0000 7fa8fa39b700 1 --2- 172.21.15.55:0/1207914745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8f406a0a0 0x7fa8f40c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.292+0000 7fa8fa39b700 1 -- 172.21.15.55:0/1207914745 >> v2:172.21.15.163:3300/0 conn(0x7fa8f406dd20 msgr2=0x7fa8f40c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:17.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.292+0000 7fa8fa39b700 1 --2- 172.21.15.55:0/1207914745 >> v2:172.21.15.163:3300/0 conn(0x7fa8f406dd20 0x7fa8f40c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.292+0000 7fa8fa39b700 1 -- 172.21.15.55:0/1207914745 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f406ab00 con 0x7fa8f406d950 2023-10-06T04:49:17.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.292+0000 7fa8fab9c700 1 --2- 172.21.15.55:0/1207914745 >> v2:172.21.15.163:3300/0 conn(0x7fa8f406dd20 0x7fa8f40c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:49:17.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.292+0000 7fa8fa39b700 1 --2- 172.21.15.55:0/1207914745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8f406d950 0x7fa8f40c8350 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fa8f406e1c0 tx=0x7fa8f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8192254140e0fd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:17.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.294+0000 7fa8f9399700 1 -- 172.21.15.55:0/1207914745 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0009e70 con 0x7fa8f406d950 2023-10-06T04:49:17.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.294+0000 7fa8f9399700 1 -- 172.21.15.55:0/1207914745 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f000b860 con 0x7fa8f406d950 2023-10-06T04:49:17.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.294+0000 7fa8f9399700 1 -- 172.21.15.55:0/1207914745 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f000bc00 con 0x7fa8f406d950 2023-10-06T04:49:17.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.294+0000 7fa8fb39d700 1 -- 172.21.15.55:0/1207914745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8f406d950 msgr2=0x7fa8f40c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:17.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.294+0000 7fa8fb39d700 1 --2- 172.21.15.55:0/1207914745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8f406d950 0x7fa8f40c8350 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fa8f406e1c0 tx=0x7fa8f0009250 comp rx=0 tx=0).stop 2023-10-06T04:49:17.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.294+0000 7fa8fb39d700 1 -- 172.21.15.55:0/1207914745 shutdown_connections 2023-10-06T04:49:17.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.294+0000 7fa8fb39d700 1 --2- 172.21.15.55:0/1207914745 >> v2:172.21.15.163:3300/0 conn(0x7fa8f406dd20 0x7fa8f40c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.294+0000 7fa8fb39d700 1 --2- 172.21.15.55:0/1207914745 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8f406d950 0x7fa8f40c8350 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.294+0000 7fa8fb39d700 1 --2- 172.21.15.55:0/1207914745 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8f406a0a0 0x7fa8f40c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.294+0000 7fa8fb39d700 1 -- 172.21.15.55:0/1207914745 >> 172.21.15.55:0/1207914745 conn(0x7fa8f40cc1f0 msgr2=0x7fa8f40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:17.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.295+0000 7fa8fb39d700 1 -- 172.21.15.55:0/1207914745 shutdown_connections 2023-10-06T04:49:17.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.295+0000 7fa8fb39d700 1 -- 172.21.15.55:0/1207914745 wait complete. 2023-10-06T04:49:17.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.296+0000 7fa8fb39d700 1 Processor -- start 2023-10-06T04:49:17.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.296+0000 7fa8fb39d700 1 -- start start 2023-10-06T04:49:17.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.297+0000 7fa8fb39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8f406a0a0 0x7fa8f4157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:17.300 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.297+0000 7fa8fb39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8f406d950 0x7fa8f4144b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:17.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.297+0000 7fa8fb39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8f406dd20 0x7fa8f40c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:17.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.297+0000 7fa8fb39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f4168bf0 con 0x7fa8f406a0a0 2023-10-06T04:49:17.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.297+0000 7fa8fb39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f4167fe0 con 0x7fa8f406d950 2023-10-06T04:49:17.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.297+0000 7fa8fb39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f41682c0 con 0x7fa8f406dd20 2023-10-06T04:49:17.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.297+0000 7fa8fa39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8f406a0a0 0x7fa8f4157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:17.301 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.298+0000 7fa8fa39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8f406a0a0 0x7fa8f4157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52484/0 (socket says 172.21.15.55:52484) 2023-10-06T04:49:17.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.298+0000 7fa8fa39b700 1 -- 172.21.15.55:0/3893474690 learned_addr learned my addr 172.21.15.55:0/3893474690 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:17.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.298+0000 7fa8f9b9a700 1 --2- 172.21.15.55:0/3893474690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8f406d950 0x7fa8f4144b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:17.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.298+0000 7fa8fa39b700 1 -- 172.21.15.55:0/3893474690 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8f406dd20 msgr2=0x7fa8f40c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:17.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.298+0000 7fa8fa39b700 1 --2- 172.21.15.55:0/3893474690 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8f406dd20 0x7fa8f40c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.298+0000 7fa8fa39b700 1 -- 172.21.15.55:0/3893474690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8f406d950 msgr2=0x7fa8f4144b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:17.303 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.298+0000 7fa8fa39b700 1 --2- 172.21.15.55:0/3893474690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8f406d950 0x7fa8f4144b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.298+0000 7fa8fa39b700 1 -- 172.21.15.55:0/3893474690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f00094f0 con 0x7fa8f406a0a0 2023-10-06T04:49:17.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.298+0000 7fa8fa39b700 1 --2- 172.21.15.55:0/3893474690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8f406a0a0 0x7fa8f4157c10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009290 tx=0x7fa8f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:17.305 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.298+0000 7fa8ea7fc700 1 -- 172.21.15.55:0/3893474690 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f000bbe0 con 0x7fa8f406a0a0 2023-10-06T04:49:17.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.299+0000 7fa8ea7fc700 1 -- 172.21.15.55:0/3893474690 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f001f070 con 0x7fa8f406a0a0 2023-10-06T04:49:17.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.299+0000 7fa8fb39d700 1 -- 172.21.15.55:0/3893474690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f41685a0 con 0x7fa8f406a0a0 2023-10-06T04:49:17.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.299+0000 7fa8ea7fc700 1 -- 172.21.15.55:0/3893474690 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00266b0 con 0x7fa8f406a0a0 2023-10-06T04:49:17.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.299+0000 7fa8fb39d700 1 -- 172.21.15.55:0/3893474690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f4167c40 con 0x7fa8f406a0a0 2023-10-06T04:49:17.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.300+0000 7fa8fb39d700 1 -- 172.21.15.55:0/3893474690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f404c3c0 con 0x7fa8f406a0a0 2023-10-06T04:49:17.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.301+0000 7fa8ea7fc700 1 -- 172.21.15.55:0/3893474690 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8f001a430 con 0x7fa8f406a0a0 2023-10-06T04:49:17.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.301+0000 7fa8ea7fc700 1 --2- 172.21.15.55:0/3893474690 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa8d8066f30 0x7fa8d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:17.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.301+0000 7fa8ea7fc700 1 -- 172.21.15.55:0/3893474690 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5598+0+0 (secure 0 0 0) 0x7fa8f0021070 con 0x7fa8f406a0a0 2023-10-06T04:49:17.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.302+0000 7fa8f9b9a700 1 --2- 172.21.15.55:0/3893474690 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa8d8066f30 0x7fa8d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:17.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.303+0000 7fa8f9b9a700 1 --2- 172.21.15.55:0/3893474690 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa8d8066f30 0x7fa8d80693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4156140 tx=0x7fa8e40091a0 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:17.309 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.306+0000 7fa8ea7fc700 1 -- 172.21.15.55:0/3893474690 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8f00b75a0 con 0x7fa8f406a0a0 2023-10-06T04:49:17.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.515+0000 7fa8fb39d700 1 -- 172.21.15.55:0/3893474690 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa8f40b2020 con 0x7fa8f406a0a0 2023-10-06T04:49:17.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.516+0000 7fa8ea7fc700 1 -- 172.21.15.55:0/3893474690 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v169) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa8f0015090 con 0x7fa8f406a0a0 2023-10-06T04:49:17.519 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:17.519 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T04:49:17.519 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:17.520 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2023-10-06T04:49:17.520 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": "rbd" 2023-10-06T04:49:17.520 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T04:49:17.520 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:17.520 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 2, 2023-10-06T04:49:17.521 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2023-10-06T04:49:17.521 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:17.521 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T04:49:17.522 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.518+0000 7fa8fb39d700 1 -- 172.21.15.55:0/3893474690 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa8d8066f30 msgr2=0x7fa8d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:17.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.518+0000 7fa8fb39d700 1 --2- 172.21.15.55:0/3893474690 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa8d8066f30 0x7fa8d80693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4156140 tx=0x7fa8e40091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.518+0000 7fa8fb39d700 1 -- 172.21.15.55:0/3893474690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8f406a0a0 msgr2=0x7fa8f4157c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:17.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.518+0000 7fa8fb39d700 1 --2- 172.21.15.55:0/3893474690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8f406a0a0 0x7fa8f4157c10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009290 tx=0x7fa8f0017000 comp rx=0 tx=0).stop 2023-10-06T04:49:17.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.518+0000 7fa8fb39d700 1 -- 172.21.15.55:0/3893474690 shutdown_connections 2023-10-06T04:49:17.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.518+0000 7fa8fb39d700 1 --2- 172.21.15.55:0/3893474690 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa8d8066f30 0x7fa8d80693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.518+0000 7fa8fb39d700 1 --2- 172.21.15.55:0/3893474690 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8f406dd20 0x7fa8f40c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.518+0000 7fa8fb39d700 1 --2- 172.21.15.55:0/3893474690 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8f406a0a0 0x7fa8f4157c10 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.518+0000 7fa8fb39d700 1 --2- 172.21.15.55:0/3893474690 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8f406d950 0x7fa8f4144b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.518+0000 7fa8fb39d700 1 -- 172.21.15.55:0/3893474690 >> 172.21.15.55:0/3893474690 conn(0x7fa8f40cc1f0 msgr2=0x7fa8f4152990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:17.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.519+0000 7fa8fb39d700 1 -- 172.21.15.55:0/3893474690 shutdown_connections 2023-10-06T04:49:17.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.519+0000 7fa8fb39d700 1 -- 172.21.15.55:0/3893474690 wait complete. 2023-10-06T04:49:17.538 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T04:49:17.539 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T04:49:17.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.731+0000 7f6a3359e700 1 Processor -- start 2023-10-06T04:49:17.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.731+0000 7f6a3359e700 1 -- start start 2023-10-06T04:49:17.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.732+0000 7f6a3359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6a340b03c0 0x7f6a340cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:17.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.732+0000 7f6a3359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a340b0790 0x7f6a340b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:17.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.732+0000 7f6a3359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a340cfa00 con 0x7f6a341591e0 2023-10-06T04:49:17.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.732+0000 7f6a3359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a340cb580 con 0x7f6a340b03c0 2023-10-06T04:49:17.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.732+0000 7f6a3359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340b2de0 con 0x7f6a340b0790 2023-10-06T04:49:17.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.733+0000 7f6a31d9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a340b0790 0x7f6a340b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:17.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.733+0000 7f6a3259c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6a340b03c0 0x7f6a340cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:17.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.733+0000 7f6a31d9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a340b0790 0x7f6a340b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:45592/0 (socket says 172.21.15.55:45592) 2023-10-06T04:49:17.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.733+0000 7f6a31d9b700 1 -- 172.21.15.55:0/3935496813 learned_addr learned my addr 172.21.15.55:0/3935496813 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:17.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.733+0000 7f6a32d9d700 1 --1- 172.21.15.55:0/3935496813 >> v1:172.21.15.163:6789/0 conn(0x7f6a341591e0 0x7f6a34159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:37798/0 (socket says 172.21.15.55:37798) 2023-10-06T04:49:17.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.733+0000 7f6a3259c700 1 -- 172.21.15.55:0/3935496813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a340b0790 msgr2=0x7f6a340b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:17.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.733+0000 7f6a3259c700 1 --2- 172.21.15.55:0/3935496813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a340b0790 0x7f6a340b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.733+0000 7f6a3259c700 1 -- 172.21.15.55:0/3935496813 >> v1:172.21.15.163:6789/0 conn(0x7f6a341591e0 legacy=0x7f6a34159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:17.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.733+0000 7f6a3259c700 1 -- 172.21.15.55:0/3935496813 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340cf160 con 0x7f6a340b03c0 2023-10-06T04:49:17.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.734+0000 7f6a3259c700 1 --2- 172.21.15.55:0/3935496813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a340b03c0 0x7f6a340cea70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6a240098e0 tx=0x7f6a240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=884f599c01cd1275 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.734+0000 7f6a3159a700 1 -- 172.21.15.55:0/3935496813 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400b640 con 0x7f6a340b03c0 2023-10-06T04:49:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.734+0000 7f6a3159a700 1 -- 172.21.15.55:0/3935496813 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2400b7a0 con 0x7f6a340b03c0 2023-10-06T04:49:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.734+0000 7f6a3159a700 1 -- 172.21.15.55:0/3935496813 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400bb40 con 0x7f6a340b03c0 2023-10-06T04:49:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.734+0000 7f6a3359e700 1 -- 172.21.15.55:0/3935496813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a340b03c0 msgr2=0x7f6a340cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.734+0000 7f6a3359e700 1 --2- 172.21.15.55:0/3935496813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a340b03c0 0x7f6a340cea70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6a240098e0 tx=0x7f6a240091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.735+0000 7f6a3359e700 1 -- 172.21.15.55:0/3935496813 shutdown_connections 2023-10-06T04:49:17.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.735+0000 7f6a3359e700 1 --2- 172.21.15.55:0/3935496813 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a340b03c0 0x7f6a340cea70 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.735+0000 7f6a3359e700 1 --2- 172.21.15.55:0/3935496813 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a340b0790 0x7f6a340b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.735+0000 7f6a3359e700 1 -- 172.21.15.55:0/3935496813 >> 172.21.15.55:0/3935496813 conn(0x7f6a34063230 msgr2=0x7f6a34063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:17.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.735+0000 7f6a3359e700 1 -- 172.21.15.55:0/3935496813 shutdown_connections 2023-10-06T04:49:17.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.735+0000 7f6a3359e700 1 -- 172.21.15.55:0/3935496813 wait complete. 2023-10-06T04:49:17.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.736+0000 7f6a3359e700 1 Processor -- start 2023-10-06T04:49:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.736+0000 7f6a3359e700 1 -- start start 2023-10-06T04:49:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.737+0000 7f6a3359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a340b03c0 0x7f6a34155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.737+0000 7f6a3359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a340b0790 0x7f6a340c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.737+0000 7f6a3359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a341591e0 0x7f6a340c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.737+0000 7f6a3359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340c5770 con 0x7f6a341591e0 2023-10-06T04:49:17.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.737+0000 7f6a3359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340c9190 con 0x7f6a340b0790 2023-10-06T04:49:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.737+0000 7f6a3359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340c4b60 con 0x7f6a340b03c0 2023-10-06T04:49:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.737+0000 7f6a32d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a341591e0 0x7f6a340c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.737+0000 7f6a32d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a341591e0 0x7f6a340c4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52488/0 (socket says 172.21.15.55:52488) 2023-10-06T04:49:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.737+0000 7f6a32d9d700 1 -- 172.21.15.55:0/681026267 learned_addr learned my addr 172.21.15.55:0/681026267 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.738+0000 7f6a3259c700 1 --2- 172.21.15.55:0/681026267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a340b03c0 0x7f6a34155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.738+0000 7f6a31d9b700 1 --2- 172.21.15.55:0/681026267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a340b0790 0x7f6a340c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.738+0000 7f6a32d9d700 1 -- 172.21.15.55:0/681026267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a340b03c0 msgr2=0x7f6a34155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.738+0000 7f6a32d9d700 1 --2- 172.21.15.55:0/681026267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a340b03c0 0x7f6a34155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.738+0000 7f6a32d9d700 1 -- 172.21.15.55:0/681026267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a340b0790 msgr2=0x7f6a340c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:17.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.738+0000 7f6a32d9d700 1 --2- 172.21.15.55:0/681026267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a340b0790 0x7f6a340c3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.738+0000 7f6a32d9d700 1 -- 172.21.15.55:0/681026267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a24009420 con 0x7f6a341591e0 2023-10-06T04:49:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.738+0000 7f6a32d9d700 1 --2- 172.21.15.55:0/681026267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a341591e0 0x7f6a340c4530 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00cd00 tx=0x7f6a2c001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.738+0000 7f6a227fc700 1 -- 172.21.15.55:0/681026267 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00ce60 con 0x7f6a341591e0 2023-10-06T04:49:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.738+0000 7f6a227fc700 1 -- 172.21.15.55:0/681026267 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c005df0 con 0x7f6a341591e0 2023-10-06T04:49:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.738+0000 7f6a3359e700 1 -- 172.21.15.55:0/681026267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a340c4e40 con 0x7f6a341591e0 2023-10-06T04:49:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.738+0000 7f6a227fc700 1 -- 172.21.15.55:0/681026267 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00ab00 con 0x7f6a341591e0 2023-10-06T04:49:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.739+0000 7f6a3359e700 1 -- 172.21.15.55:0/681026267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a340c5400 con 0x7f6a341591e0 2023-10-06T04:49:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.740+0000 7f6a3359e700 1 -- 172.21.15.55:0/681026267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3404c3c0 con 0x7f6a341591e0 2023-10-06T04:49:17.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.740+0000 7f6a227fc700 1 -- 172.21.15.55:0/681026267 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6a2c0070f0 con 0x7f6a341591e0 2023-10-06T04:49:17.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.741+0000 7f6a227fc700 1 --2- 172.21.15.55:0/681026267 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6a14066f30 0x7f6a140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:17.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.741+0000 7f6a227fc700 1 -- 172.21.15.55:0/681026267 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5598+0+0 (secure 0 0 0) 0x7f6a2c084550 con 0x7f6a341591e0 2023-10-06T04:49:17.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.742+0000 7f6a3259c700 1 --2- 172.21.15.55:0/681026267 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6a14066f30 0x7f6a140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:17.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.742+0000 7f6a3259c700 1 --2- 172.21.15.55:0/681026267 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6a14066f30 0x7f6a140693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6a240093f0 tx=0x7f6a24017000 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:17.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.746+0000 7f6a227fc700 1 -- 172.21.15.55:0/681026267 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a2c051810 con 0x7f6a341591e0 2023-10-06T04:49:17.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:17.949+0000 7f6a3359e700 1 -- 172.21.15.55:0/681026267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6a340c58b0 con 0x7f6a341591e0 2023-10-06T04:49:18.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.301+0000 7f6a227fc700 1 -- 172.21.15.55:0/681026267 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v170) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6a2c082e60 con 0x7f6a341591e0 2023-10-06T04:49:18.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.304+0000 7f6a3359e700 1 -- 172.21.15.55:0/681026267 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6a14066f30 msgr2=0x7f6a140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:18.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.304+0000 7f6a3359e700 1 --2- 172.21.15.55:0/681026267 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6a14066f30 0x7f6a140693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6a240093f0 tx=0x7f6a24017000 comp rx=0 tx=0).stop 2023-10-06T04:49:18.306 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.304+0000 7f6a3359e700 1 -- 172.21.15.55:0/681026267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a341591e0 msgr2=0x7f6a340c4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:18.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.304+0000 7f6a3359e700 1 --2- 172.21.15.55:0/681026267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a341591e0 0x7f6a340c4530 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00cd00 tx=0x7f6a2c001b10 comp rx=0 tx=0).stop 2023-10-06T04:49:18.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.305+0000 7f6a3359e700 1 -- 172.21.15.55:0/681026267 shutdown_connections 2023-10-06T04:49:18.307 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.305+0000 7f6a3359e700 1 --2- 172.21.15.55:0/681026267 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f6a14066f30 0x7f6a140693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:18.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.305+0000 7f6a3359e700 1 --2- 172.21.15.55:0/681026267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6a341591e0 0x7f6a340c4530 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:18.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.305+0000 7f6a3359e700 1 --2- 172.21.15.55:0/681026267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6a340b03c0 0x7f6a34155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:18.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.305+0000 7f6a3359e700 1 --2- 172.21.15.55:0/681026267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6a340b0790 0x7f6a340c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:18.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.305+0000 7f6a3359e700 1 -- 172.21.15.55:0/681026267 >> 172.21.15.55:0/681026267 conn(0x7f6a34063230 msgr2=0x7f6a34158a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:18.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.305+0000 7f6a3359e700 1 -- 172.21.15.55:0/681026267 shutdown_connections 2023-10-06T04:49:18.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.305+0000 7f6a3359e700 1 -- 172.21.15.55:0/681026267 wait complete. 2023-10-06T04:49:18.309 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2023-10-06T04:49:18.329 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T04:49:18.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.523+0000 7f119ffff700 1 Processor -- start 2023-10-06T04:49:18.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.524+0000 7f119ffff700 1 -- start start 2023-10-06T04:49:18.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.525+0000 7f119ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11a0068aa0 0x7f11a006bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:18.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.525+0000 7f119ffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a015ad70 con 0x7f11a01573f0 2023-10-06T04:49:18.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.525+0000 7f119ffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a015a160 con 0x7f11a01577c0 2023-10-06T04:49:18.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.525+0000 7f119ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00c8610 con 0x7f11a0068aa0 2023-10-06T04:49:18.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.525+0000 7f119effd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f11a01573f0 0x7f11a006c6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37992/0 (socket says 172.21.15.55:37992) 2023-10-06T04:49:18.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.525+0000 7f119effd700 1 -- 172.21.15.55:0/367231574 learned_addr learned my addr 172.21.15.55:0/367231574 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:18.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.525+0000 7f119f7fe700 1 --2- 172.21.15.55:0/367231574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11a0068aa0 0x7f11a006bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:18.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.526+0000 7f119dffb700 1 -- 172.21.15.55:0/367231574 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227557877 0 0) 0x7f11a015ad70 con 0x7f11a01573f0 2023-10-06T04:49:18.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.526+0000 7f119dffb700 1 -- 172.21.15.55:0/367231574 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1188003570 con 0x7f11a01573f0 2023-10-06T04:49:18.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.526+0000 7f119dffb700 1 -- 172.21.15.55:0/367231574 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477807971 0 0) 0x7f11a015a160 con 0x7f11a01577c0 2023-10-06T04:49:18.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.526+0000 7f119dffb700 1 -- 172.21.15.55:0/367231574 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11a015ad70 con 0x7f11a01577c0 2023-10-06T04:49:18.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.526+0000 7f119dffb700 1 -- 172.21.15.55:0/367231574 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4032324359 0 0) 0x7f1188003570 con 0x7f11a01573f0 2023-10-06T04:49:18.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.526+0000 7f119f7fe700 1 -- 172.21.15.55:0/367231574 >> v1:172.21.15.163:6789/0 conn(0x7f11a01577c0 legacy=0x7f11a0068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:18.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.526+0000 7f119f7fe700 1 -- 172.21.15.55:0/367231574 >> v1:172.21.15.55:6789/0 conn(0x7f11a01573f0 legacy=0x7f11a006c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:18.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.526+0000 7f119f7fe700 1 -- 172.21.15.55:0/367231574 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a015a440 con 0x7f11a0068aa0 2023-10-06T04:49:18.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.526+0000 7f119f7fe700 1 --2- 172.21.15.55:0/367231574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11a0068aa0 0x7f11a006bcb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1198009e90 tx=0x7f1198009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d06e87ced43b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:18.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.527+0000 7f119dffb700 1 -- 172.21.15.55:0/367231574 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119800e070 con 0x7f11a0068aa0 2023-10-06T04:49:18.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.527+0000 7f119dffb700 1 -- 172.21.15.55:0/367231574 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119800b870 con 0x7f11a0068aa0 2023-10-06T04:49:18.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.527+0000 7f119dffb700 1 -- 172.21.15.55:0/367231574 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190001470 con 0x7f11a0068aa0 2023-10-06T04:49:18.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.527+0000 7f119ffff700 1 -- 172.21.15.55:0/367231574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11a0068aa0 msgr2=0x7f11a006bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:18.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.527+0000 7f119ffff700 1 --2- 172.21.15.55:0/367231574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11a0068aa0 0x7f11a006bcb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1198009e90 tx=0x7f1198009780 comp rx=0 tx=0).stop 2023-10-06T04:49:18.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.527+0000 7f119ffff700 1 -- 172.21.15.55:0/367231574 shutdown_connections 2023-10-06T04:49:18.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.527+0000 7f119ffff700 1 --2- 172.21.15.55:0/367231574 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11a0068aa0 0x7f11a006bcb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:18.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.527+0000 7f119ffff700 1 -- 172.21.15.55:0/367231574 >> 172.21.15.55:0/367231574 conn(0x7f11a00cddc0 msgr2=0x7f11a00c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:18.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.527+0000 7f119ffff700 1 -- 172.21.15.55:0/367231574 shutdown_connections 2023-10-06T04:49:18.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.527+0000 7f119ffff700 1 -- 172.21.15.55:0/367231574 wait complete. 2023-10-06T04:49:18.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.529+0000 7f119ffff700 1 Processor -- start 2023-10-06T04:49:18.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.529+0000 7f119ffff700 1 -- start start 2023-10-06T04:49:18.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11a0068aa0 0x7f11a00cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a01573f0 0x7f11a014b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11a01577c0 0x7f11a016ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00cb8b0 con 0x7f11a01573f0 2023-10-06T04:49:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119ffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a00cc150 con 0x7f11a0068aa0 2023-10-06T04:49:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a016aa00 con 0x7f11a01577c0 2023-10-06T04:49:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a01573f0 0x7f11a014b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119e7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a01573f0 0x7f11a014b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52498/0 (socket says 172.21.15.55:52498) 2023-10-06T04:49:18.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119e7fc700 1 -- 172.21.15.55:0/3957447758 learned_addr learned my addr 172.21.15.55:0/3957447758 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:18.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119f7fe700 1 --2- 172.21.15.55:0/3957447758 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11a01577c0 0x7f11a016ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:18.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119effd700 1 --2- 172.21.15.55:0/3957447758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11a0068aa0 0x7f11a00cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:18.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119e7fc700 1 -- 172.21.15.55:0/3957447758 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11a01577c0 msgr2=0x7f11a016ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:18.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119e7fc700 1 --2- 172.21.15.55:0/3957447758 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11a01577c0 0x7f11a016ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:18.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119e7fc700 1 -- 172.21.15.55:0/3957447758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11a0068aa0 msgr2=0x7f11a00cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:18.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119e7fc700 1 --2- 172.21.15.55:0/3957447758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11a0068aa0 0x7f11a00cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:18.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.530+0000 7f119e7fc700 1 -- 172.21.15.55:0/3957447758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1198009a20 con 0x7f11a01573f0 2023-10-06T04:49:18.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.531+0000 7f119e7fc700 1 --2- 172.21.15.55:0/3957447758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a01573f0 0x7f11a014b430 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f119400c580 tx=0x7f1194007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:18.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.531+0000 7f1186ffd700 1 -- 172.21.15.55:0/3957447758 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194010510 con 0x7f11a01573f0 2023-10-06T04:49:18.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.531+0000 7f1186ffd700 1 -- 172.21.15.55:0/3957447758 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1194010670 con 0x7f11a01573f0 2023-10-06T04:49:18.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.531+0000 7f119ffff700 1 -- 172.21.15.55:0/3957447758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a00cbb00 con 0x7f11a01573f0 2023-10-06T04:49:18.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.531+0000 7f1186ffd700 1 -- 172.21.15.55:0/3957447758 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194010a30 con 0x7f11a01573f0 2023-10-06T04:49:18.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.531+0000 7f119ffff700 1 -- 172.21.15.55:0/3957447758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a00c6670 con 0x7f11a01573f0 2023-10-06T04:49:18.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.532+0000 7f119ffff700 1 -- 172.21.15.55:0/3957447758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a004c3c0 con 0x7f11a01573f0 2023-10-06T04:49:18.541 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.534+0000 7f1186ffd700 1 -- 172.21.15.55:0/3957447758 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1194025430 con 0x7f11a01573f0 2023-10-06T04:49:18.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.535+0000 7f1186ffd700 1 --2- 172.21.15.55:0/3957447758 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f118c066f30 0x7f118c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:18.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.535+0000 7f1186ffd700 1 -- 172.21.15.55:0/3957447758 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5968+0+0 (secure 0 0 0) 0x7f1194084e00 con 0x7f11a01573f0 2023-10-06T04:49:18.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.535+0000 7f119effd700 1 --2- 172.21.15.55:0/3957447758 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f118c066f30 0x7f118c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:18.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.536+0000 7f119effd700 1 --2- 172.21.15.55:0/3957447758 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f118c066f30 0x7f118c0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1190005c20 tx=0x7f1190000bc0 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:18.542 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.538+0000 7f1186ffd700 1 -- 172.21.15.55:0/3957447758 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1194052020 con 0x7f11a01573f0 2023-10-06T04:49:18.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:18.742+0000 7f119ffff700 1 -- 172.21.15.55:0/3957447758 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f11a00cbfa0 con 0x7f11a01573f0 2023-10-06T04:49:19.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.312+0000 7f1186ffd700 1 -- 172.21.15.55:0/3957447758 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v171) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1194055640 con 0x7f11a01573f0 2023-10-06T04:49:19.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.325+0000 7f119ffff700 1 -- 172.21.15.55:0/3957447758 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f118c066f30 msgr2=0x7f118c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:19.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.325+0000 7f119ffff700 1 --2- 172.21.15.55:0/3957447758 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f118c066f30 0x7f118c0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1190005c20 tx=0x7f1190000bc0 comp rx=0 tx=0).stop 2023-10-06T04:49:19.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.325+0000 7f119ffff700 1 -- 172.21.15.55:0/3957447758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a01573f0 msgr2=0x7f11a014b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:19.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.325+0000 7f119ffff700 1 --2- 172.21.15.55:0/3957447758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a01573f0 0x7f11a014b430 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f119400c580 tx=0x7f1194007e10 comp rx=0 tx=0).stop 2023-10-06T04:49:19.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.325+0000 7f119ffff700 1 -- 172.21.15.55:0/3957447758 shutdown_connections 2023-10-06T04:49:19.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.325+0000 7f119ffff700 1 --2- 172.21.15.55:0/3957447758 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f118c066f30 0x7f118c0693e0 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1190005c20 tx=0x7f1190000bc0 comp rx=0 tx=0).stop 2023-10-06T04:49:19.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.325+0000 7f119ffff700 1 --2- 172.21.15.55:0/3957447758 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11a01577c0 0x7f11a016ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:19.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.325+0000 7f119ffff700 1 --2- 172.21.15.55:0/3957447758 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11a0068aa0 0x7f11a00cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:19.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.325+0000 7f119ffff700 1 --2- 172.21.15.55:0/3957447758 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11a01573f0 0x7f11a014b430 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:19.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.325+0000 7f119ffff700 1 -- 172.21.15.55:0/3957447758 >> 172.21.15.55:0/3957447758 conn(0x7f11a00cddc0 msgr2=0x7f11a0156070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:19.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.326+0000 7f119ffff700 1 -- 172.21.15.55:0/3957447758 shutdown_connections 2023-10-06T04:49:19.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.326+0000 7f119ffff700 1 -- 172.21.15.55:0/3957447758 wait complete. 2023-10-06T04:49:19.332 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2023-10-06T04:49:19.352 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T04:49:19.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.543+0000 7f25fa99d700 1 Processor -- start 2023-10-06T04:49:19.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.543+0000 7f25fa99d700 1 -- start start 2023-10-06T04:49:19.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.544+0000 7f25fa99d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f25f40ccc00 0x7f25f4069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:19.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.544+0000 7f25fa99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f25f406a020 0x7f25f40cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:19.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.544+0000 7f25fa99d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25f40c9f30 con 0x7f25f4159a60 2023-10-06T04:49:19.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.544+0000 7f25fa99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25f406dfc0 con 0x7f25f406a020 2023-10-06T04:49:19.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.544+0000 7f25fa99d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25f40ca470 con 0x7f25f40ccc00 2023-10-06T04:49:19.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.544+0000 7f25f999b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f25f4159a60 0x7f25f40c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:37994/0 (socket says 172.21.15.55:37994) 2023-10-06T04:49:19.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.545+0000 7f25f999b700 1 -- 172.21.15.55:0/190226069 learned_addr learned my addr 172.21.15.55:0/190226069 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:19.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.545+0000 7f25fa19c700 1 --2- 172.21.15.55:0/190226069 >> v2:172.21.15.55:3300/0 conn(0x7f25f406a020 0x7f25f40cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:19.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.545+0000 7f25f8999700 1 -- 172.21.15.55:0/190226069 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516991502 0 0) 0x7f25f40c9f30 con 0x7f25f4159a60 2023-10-06T04:49:19.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.545+0000 7f25f8999700 1 -- 172.21.15.55:0/190226069 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25e0003570 con 0x7f25f4159a60 2023-10-06T04:49:19.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.545+0000 7f25fa19c700 1 -- 172.21.15.55:0/190226069 >> v2:172.21.15.163:3300/0 conn(0x7f25f40ccc00 msgr2=0x7f25f4069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:19.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.545+0000 7f25fa19c700 1 --2- 172.21.15.55:0/190226069 >> v2:172.21.15.163:3300/0 conn(0x7f25f40ccc00 0x7f25f4069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:19.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.545+0000 7f25fa19c700 1 -- 172.21.15.55:0/190226069 >> v1:172.21.15.55:6789/0 conn(0x7f25f4159a60 legacy=0x7f25f40c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:19.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.545+0000 7f25fa19c700 1 -- 172.21.15.55:0/190226069 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f40ca750 con 0x7f25f406a020 2023-10-06T04:49:19.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.545+0000 7f25fa19c700 1 --2- 172.21.15.55:0/190226069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25f406a020 0x7f25f40cb230 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f25f40cf6a0 tx=0x7f25ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8460baa14378a8de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:19.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.546+0000 7f25f8999700 1 -- 172.21.15.55:0/190226069 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec00e070 con 0x7f25f406a020 2023-10-06T04:49:19.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.546+0000 7f25f8999700 1 -- 172.21.15.55:0/190226069 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ec00bbf0 con 0x7f25f406a020 2023-10-06T04:49:19.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.546+0000 7f25f8999700 1 -- 172.21.15.55:0/190226069 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec012430 con 0x7f25f406a020 2023-10-06T04:49:19.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.546+0000 7f25fa99d700 1 -- 172.21.15.55:0/190226069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25f406a020 msgr2=0x7f25f40cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:19.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.546+0000 7f25fa99d700 1 --2- 172.21.15.55:0/190226069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25f406a020 0x7f25f40cb230 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f25f40cf6a0 tx=0x7f25ec009780 comp rx=0 tx=0).stop 2023-10-06T04:49:19.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.546+0000 7f25fa99d700 1 -- 172.21.15.55:0/190226069 shutdown_connections 2023-10-06T04:49:19.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.546+0000 7f25fa99d700 1 --2- 172.21.15.55:0/190226069 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25f406a020 0x7f25f40cb230 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:19.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.546+0000 7f25fa99d700 1 --2- 172.21.15.55:0/190226069 >> v2:172.21.15.163:3300/0 conn(0x7f25f40ccc00 0x7f25f4069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:19.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.546+0000 7f25fa99d700 1 -- 172.21.15.55:0/190226069 >> 172.21.15.55:0/190226069 conn(0x7f25f406baa0 msgr2=0x7f25f406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:19.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.546+0000 7f25fa99d700 1 -- 172.21.15.55:0/190226069 shutdown_connections 2023-10-06T04:49:19.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.546+0000 7f25fa99d700 1 -- 172.21.15.55:0/190226069 wait complete. 2023-10-06T04:49:19.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.548+0000 7f25fa99d700 1 Processor -- start 2023-10-06T04:49:19.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.548+0000 7f25fa99d700 1 -- start start 2023-10-06T04:49:19.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.549+0000 7f25fa99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25f406a020 0x7f25f40b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:19.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.549+0000 7f25fa99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25f40ccc00 0x7f25f4142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:19.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.549+0000 7f25fa99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25f4159a60 0x7f25f4143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:19.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.549+0000 7f25fa99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f40b1b10 con 0x7f25f40ccc00 2023-10-06T04:49:19.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.549+0000 7f25fa99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f40b3be0 con 0x7f25f406a020 2023-10-06T04:49:19.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.549+0000 7f25fa99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f4154ae0 con 0x7f25f4159a60 2023-10-06T04:49:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.549+0000 7f25f919a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25f40ccc00 0x7f25f4142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.549+0000 7f25f919a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25f40ccc00 0x7f25f4142cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52518/0 (socket says 172.21.15.55:52518) 2023-10-06T04:49:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.549+0000 7f25f919a700 1 -- 172.21.15.55:0/2012695360 learned_addr learned my addr 172.21.15.55:0/2012695360 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.549+0000 7f25f919a700 1 -- 172.21.15.55:0/2012695360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25f4159a60 msgr2=0x7f25f4143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.549+0000 7f25f919a700 1 --2- 172.21.15.55:0/2012695360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25f4159a60 0x7f25f4143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.549+0000 7f25f919a700 1 -- 172.21.15.55:0/2012695360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25f406a020 msgr2=0x7f25f40b1430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:49:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.549+0000 7f25f919a700 1 --2- 172.21.15.55:0/2012695360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25f406a020 0x7f25f40b1430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:19.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.549+0000 7f25f919a700 1 -- 172.21.15.55:0/2012695360 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec009a20 con 0x7f25f40ccc00 2023-10-06T04:49:19.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.550+0000 7f25f919a700 1 --2- 172.21.15.55:0/2012695360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25f40ccc00 0x7f25f4142cf0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f25e400b810 tx=0x7f25e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:19.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.550+0000 7f25e9ffb700 1 -- 172.21.15.55:0/2012695360 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e400f5f0 con 0x7f25f40ccc00 2023-10-06T04:49:19.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.550+0000 7f25e9ffb700 1 -- 172.21.15.55:0/2012695360 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e400f770 con 0x7f25f40ccc00 2023-10-06T04:49:19.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.550+0000 7f25e9ffb700 1 -- 172.21.15.55:0/2012695360 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e400fb30 con 0x7f25f40ccc00 2023-10-06T04:49:19.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.550+0000 7f25fa99d700 1 -- 172.21.15.55:0/2012695360 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f40b3ea0 con 0x7f25f40ccc00 2023-10-06T04:49:19.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.550+0000 7f25fa99d700 1 -- 172.21.15.55:0/2012695360 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f416a210 con 0x7f25f40ccc00 2023-10-06T04:49:19.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.551+0000 7f25fa99d700 1 -- 172.21.15.55:0/2012695360 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f406eba0 con 0x7f25f40ccc00 2023-10-06T04:49:19.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.553+0000 7f25e9ffb700 1 -- 172.21.15.55:0/2012695360 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25e4010480 con 0x7f25f40ccc00 2023-10-06T04:49:19.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.553+0000 7f25e9ffb700 1 --2- 172.21.15.55:0/2012695360 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f25d8066f30 0x7f25d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:19.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.553+0000 7f25e9ffb700 1 -- 172.21.15.55:0/2012695360 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6350+0+0 (secure 0 0 0) 0x7f25e4085230 con 0x7f25f40ccc00 2023-10-06T04:49:19.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.554+0000 7f25f999b700 1 --2- 172.21.15.55:0/2012695360 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f25d8066f30 0x7f25d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:19.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.554+0000 7f25f999b700 1 --2- 172.21.15.55:0/2012695360 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f25d8066f30 0x7f25d80693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f25f00058d0 tx=0x7f25f00010a0 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:19.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.557+0000 7f25e9ffb700 1 -- 172.21.15.55:0/2012695360 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25e4052280 con 0x7f25f40ccc00 2023-10-06T04:49:19.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:19.806+0000 7f25fa99d700 1 -- 172.21.15.55:0/2012695360 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f25f4154ca0 con 0x7f25f40ccc00 2023-10-06T04:49:20.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.315+0000 7f25e9ffb700 1 -- 172.21.15.55:0/2012695360 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v83) v1 ==== 174+0+0 (secure 0 0 0) 0x7f25e40558a0 con 0x7f25f40ccc00 2023-10-06T04:49:20.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.327+0000 7f25fa99d700 1 -- 172.21.15.55:0/2012695360 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f25d8066f30 msgr2=0x7f25d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:20.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.327+0000 7f25fa99d700 1 --2- 172.21.15.55:0/2012695360 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f25d8066f30 0x7f25d80693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f25f00058d0 tx=0x7f25f00010a0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.327+0000 7f25fa99d700 1 -- 172.21.15.55:0/2012695360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25f40ccc00 msgr2=0x7f25f4142cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:20.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.327+0000 7f25fa99d700 1 --2- 172.21.15.55:0/2012695360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25f40ccc00 0x7f25f4142cf0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f25e400b810 tx=0x7f25e4009250 comp rx=0 tx=0).stop 2023-10-06T04:49:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.330+0000 7f25fa99d700 1 -- 172.21.15.55:0/2012695360 shutdown_connections 2023-10-06T04:49:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.330+0000 7f25fa99d700 1 --2- 172.21.15.55:0/2012695360 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f25d8066f30 0x7f25d80693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.330+0000 7f25fa99d700 1 --2- 172.21.15.55:0/2012695360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f25f4159a60 0x7f25f4143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.330+0000 7f25fa99d700 1 --2- 172.21.15.55:0/2012695360 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f25f406a020 0x7f25f40b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.330+0000 7f25fa99d700 1 --2- 172.21.15.55:0/2012695360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f25f40ccc00 0x7f25f4142cf0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.330+0000 7f25fa99d700 1 -- 172.21.15.55:0/2012695360 >> 172.21.15.55:0/2012695360 conn(0x7f25f406baa0 msgr2=0x7f25f41541d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:20.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.335+0000 7f25fa99d700 1 -- 172.21.15.55:0/2012695360 shutdown_connections 2023-10-06T04:49:20.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.335+0000 7f25fa99d700 1 -- 172.21.15.55:0/2012695360 wait complete. 2023-10-06T04:49:20.345 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 21 and data pool 22 2023-10-06T04:49:20.366 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:49:20.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.552+0000 7f1082f9d700 1 Processor -- start 2023-10-06T04:49:20.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.552+0000 7f1082f9d700 1 -- start start 2023-10-06T04:49:20.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.553+0000 7f1082f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f107c1597f0 0x7f107c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:20.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.553+0000 7f1082f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f107c15d230 0x7f107c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:20.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.553+0000 7f1082f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107c15db70 con 0x7f107c159420 2023-10-06T04:49:20.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.553+0000 7f1082f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f107c0b11d0 con 0x7f107c15d230 2023-10-06T04:49:20.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.553+0000 7f1082f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f107c0b14b0 con 0x7f107c1597f0 2023-10-06T04:49:20.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.554+0000 7f108279c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f107c15d230 0x7f107c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:20.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.554+0000 7f1081f9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f107c159420 0x7f107c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38002/0 (socket says 172.21.15.55:38002) 2023-10-06T04:49:20.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.554+0000 7f1081f9b700 1 -- 172.21.15.55:0/2937876116 learned_addr learned my addr 172.21.15.55:0/2937876116 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:20.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.554+0000 7f108179a700 1 --2- 172.21.15.55:0/2937876116 >> v2:172.21.15.163:3300/0 conn(0x7f107c1597f0 0x7f107c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:20.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.554+0000 7f108279c700 1 -- 172.21.15.55:0/2937876116 >> v2:172.21.15.163:3300/0 conn(0x7f107c1597f0 msgr2=0x7f107c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:20.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.554+0000 7f108279c700 1 --2- 172.21.15.55:0/2937876116 >> v2:172.21.15.163:3300/0 conn(0x7f107c1597f0 0x7f107c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.554+0000 7f108279c700 1 -- 172.21.15.55:0/2937876116 >> v1:172.21.15.55:6789/0 conn(0x7f107c159420 legacy=0x7f107c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:20.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.554+0000 7f108279c700 1 -- 172.21.15.55:0/2937876116 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c0b1790 con 0x7f107c15d230 2023-10-06T04:49:20.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.554+0000 7f1080f99700 1 -- 172.21.15.55:0/2937876116 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520565563 0 0) 0x7f107c15db70 con 0x7f107c159420 2023-10-06T04:49:20.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.555+0000 7f108279c700 1 --2- 172.21.15.55:0/2937876116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107c15d230 0x7f107c15d630 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f107c089750 tx=0x7f1074009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea10f42c56020a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:20.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.555+0000 7f1080f99700 1 -- 172.21.15.55:0/2937876116 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107400e070 con 0x7f107c15d230 2023-10-06T04:49:20.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.555+0000 7f1080f99700 1 -- 172.21.15.55:0/2937876116 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107400bb60 con 0x7f107c15d230 2023-10-06T04:49:20.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.555+0000 7f1080f99700 1 -- 172.21.15.55:0/2937876116 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074012430 con 0x7f107c15d230 2023-10-06T04:49:20.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.555+0000 7f1082f9d700 1 -- 172.21.15.55:0/2937876116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107c15d230 msgr2=0x7f107c15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:20.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.555+0000 7f1082f9d700 1 --2- 172.21.15.55:0/2937876116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107c15d230 0x7f107c15d630 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f107c089750 tx=0x7f1074009780 comp rx=0 tx=0).stop 2023-10-06T04:49:20.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.555+0000 7f1082f9d700 1 -- 172.21.15.55:0/2937876116 shutdown_connections 2023-10-06T04:49:20.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.555+0000 7f1082f9d700 1 --2- 172.21.15.55:0/2937876116 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107c15d230 0x7f107c15d630 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.555+0000 7f1082f9d700 1 --2- 172.21.15.55:0/2937876116 >> v2:172.21.15.163:3300/0 conn(0x7f107c1597f0 0x7f107c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.555+0000 7f1082f9d700 1 -- 172.21.15.55:0/2937876116 >> 172.21.15.55:0/2937876116 conn(0x7f107c0d0660 msgr2=0x7f107c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:20.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.555+0000 7f1082f9d700 1 -- 172.21.15.55:0/2937876116 shutdown_connections 2023-10-06T04:49:20.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.556+0000 7f1082f9d700 1 -- 172.21.15.55:0/2937876116 wait complete. 2023-10-06T04:49:20.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.557+0000 7f1082f9d700 1 Processor -- start 2023-10-06T04:49:20.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.557+0000 7f1082f9d700 1 -- start start 2023-10-06T04:49:20.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.558+0000 7f1082f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f107c159420 0x7f107c155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:20.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.558+0000 7f1082f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107c1597f0 0x7f107c168b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:20.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.558+0000 7f1082f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f107c15d230 0x7f107c0cf210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:20.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.558+0000 7f1082f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c1564f0 con 0x7f107c1597f0 2023-10-06T04:49:20.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.558+0000 7f1082f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c14fad0 con 0x7f107c15d230 2023-10-06T04:49:20.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.558+0000 7f1082f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107c155970 con 0x7f107c159420 2023-10-06T04:49:20.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.558+0000 7f108179a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107c1597f0 0x7f107c168b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:20.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.558+0000 7f108179a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107c1597f0 0x7f107c168b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52522/0 (socket says 172.21.15.55:52522) 2023-10-06T04:49:20.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.558+0000 7f108179a700 1 -- 172.21.15.55:0/4005471424 learned_addr learned my addr 172.21.15.55:0/4005471424 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:20.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.559+0000 7f1081f9b700 1 --2- 172.21.15.55:0/4005471424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f107c159420 0x7f107c155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:20.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.559+0000 7f108279c700 1 --2- 172.21.15.55:0/4005471424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f107c15d230 0x7f107c0cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:20.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.559+0000 7f108179a700 1 -- 172.21.15.55:0/4005471424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f107c159420 msgr2=0x7f107c155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:20.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.559+0000 7f108179a700 1 --2- 172.21.15.55:0/4005471424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f107c159420 0x7f107c155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.559+0000 7f108179a700 1 -- 172.21.15.55:0/4005471424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f107c15d230 msgr2=0x7f107c0cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:20.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.559+0000 7f108179a700 1 --2- 172.21.15.55:0/4005471424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f107c15d230 0x7f107c0cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.559+0000 7f108179a700 1 -- 172.21.15.55:0/4005471424 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1074009a20 con 0x7f107c1597f0 2023-10-06T04:49:20.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.559+0000 7f108179a700 1 --2- 172.21.15.55:0/4005471424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107c1597f0 0x7f107c168b00 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f106c00ec50 tx=0x7f106c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:20.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.559+0000 7f1071ffb700 1 -- 172.21.15.55:0/4005471424 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c00c9d0 con 0x7f107c1597f0 2023-10-06T04:49:20.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.559+0000 7f1071ffb700 1 -- 172.21.15.55:0/4005471424 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c00cb30 con 0x7f107c1597f0 2023-10-06T04:49:20.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.559+0000 7f1071ffb700 1 -- 172.21.15.55:0/4005471424 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c013470 con 0x7f107c1597f0 2023-10-06T04:49:20.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.559+0000 7f1082f9d700 1 -- 172.21.15.55:0/4005471424 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107c155c50 con 0x7f107c1597f0 2023-10-06T04:49:20.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.559+0000 7f1082f9d700 1 -- 172.21.15.55:0/4005471424 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107c156210 con 0x7f107c1597f0 2023-10-06T04:49:20.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.560+0000 7f1082f9d700 1 -- 172.21.15.55:0/4005471424 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107c1556c0 con 0x7f107c1597f0 2023-10-06T04:49:20.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.561+0000 7f1071ffb700 1 -- 172.21.15.55:0/4005471424 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f106c013860 con 0x7f107c1597f0 2023-10-06T04:49:20.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.562+0000 7f1071ffb700 1 --2- 172.21.15.55:0/4005471424 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1060066f30 0x7f10600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:20.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.562+0000 7f1071ffb700 1 -- 172.21.15.55:0/4005471424 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f106c084c10 con 0x7f107c1597f0 2023-10-06T04:49:20.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.563+0000 7f1081f9b700 1 --2- 172.21.15.55:0/4005471424 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1060066f30 0x7f10600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:20.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.563+0000 7f1081f9b700 1 --2- 172.21.15.55:0/4005471424 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1060066f30 0x7f10600693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f10780011e0 tx=0x7f1078009040 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:20.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.566+0000 7f1071ffb700 1 -- 172.21.15.55:0/4005471424 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f106c051af0 con 0x7f107c1597f0 2023-10-06T04:49:20.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.770+0000 7f1082f9d700 1 -- 172.21.15.55:0/4005471424 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f107c04c3c0 con 0x7f107c1597f0 2023-10-06T04:49:20.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.771+0000 7f1071ffb700 1 -- 172.21.15.55:0/4005471424 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+17536 (secure 0 0 0) 0x7f106c055110 con 0x7f107c1597f0 2023-10-06T04:49:20.773 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:20.774 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":172,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T04:49:19.807525+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:49:17.992792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T04:49:18.785942+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:49:20.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.774+0000 7f1082f9d700 1 -- 172.21.15.55:0/4005471424 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1060066f30 msgr2=0x7f10600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:20.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.774+0000 7f1082f9d700 1 --2- 172.21.15.55:0/4005471424 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1060066f30 0x7f10600693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f10780011e0 tx=0x7f1078009040 comp rx=0 tx=0).stop 2023-10-06T04:49:20.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.774+0000 7f1082f9d700 1 -- 172.21.15.55:0/4005471424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107c1597f0 msgr2=0x7f107c168b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:20.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.774+0000 7f1082f9d700 1 --2- 172.21.15.55:0/4005471424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107c1597f0 0x7f107c168b00 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f106c00ec50 tx=0x7f106c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.774+0000 7f1082f9d700 1 -- 172.21.15.55:0/4005471424 shutdown_connections 2023-10-06T04:49:20.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.774+0000 7f1082f9d700 1 --2- 172.21.15.55:0/4005471424 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1060066f30 0x7f10600693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.774+0000 7f1082f9d700 1 --2- 172.21.15.55:0/4005471424 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f107c15d230 0x7f107c0cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.774+0000 7f1082f9d700 1 --2- 172.21.15.55:0/4005471424 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f107c159420 0x7f107c155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.774+0000 7f1082f9d700 1 --2- 172.21.15.55:0/4005471424 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f107c1597f0 0x7f107c168b00 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.774+0000 7f1082f9d700 1 -- 172.21.15.55:0/4005471424 >> 172.21.15.55:0/4005471424 conn(0x7f107c0d0660 msgr2=0x7f107c1582c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:20.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.774+0000 7f1082f9d700 1 -- 172.21.15.55:0/4005471424 shutdown_connections 2023-10-06T04:49:20.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.774+0000 7f1082f9d700 1 -- 172.21.15.55:0/4005471424 wait complete. 2023-10-06T04:49:20.794 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:49:20.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.979+0000 7f463759e700 1 Processor -- start 2023-10-06T04:49:20.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.979+0000 7f463759e700 1 -- start start 2023-10-06T04:49:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.981+0000 7f463759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f46381566e0 0x7f46380ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.981+0000 7f463759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f46380cb900 0x7f46380ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.981+0000 7f463759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46380ca860 con 0x7f4638156310 2023-10-06T04:49:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.981+0000 7f463759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46380cab40 con 0x7f46381566e0 2023-10-06T04:49:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.981+0000 7f463759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46380cae20 con 0x7f46380cb900 2023-10-06T04:49:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.981+0000 7f463659c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4638156310 0x7f46380c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:38012/0 (socket says 172.21.15.55:38012) 2023-10-06T04:49:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.981+0000 7f463659c700 1 -- 172.21.15.55:0/2870580142 learned_addr learned my addr 172.21.15.55:0/2870580142 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:20.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.981+0000 7f4635d9b700 1 --2- 172.21.15.55:0/2870580142 >> v2:172.21.15.55:3300/0 conn(0x7f46381566e0 0x7f46380ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:20.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.981+0000 7f4636d9d700 1 --2- 172.21.15.55:0/2870580142 >> v2:172.21.15.163:3300/0 conn(0x7f46380cb900 0x7f46380ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:20.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.982+0000 7f4635d9b700 1 -- 172.21.15.55:0/2870580142 >> v2:172.21.15.163:3300/0 conn(0x7f46380cb900 msgr2=0x7f46380ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:20.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.982+0000 7f4635d9b700 1 --2- 172.21.15.55:0/2870580142 >> v2:172.21.15.163:3300/0 conn(0x7f46380cb900 0x7f46380ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.982+0000 7f4635d9b700 1 -- 172.21.15.55:0/2870580142 >> v1:172.21.15.55:6789/0 conn(0x7f4638156310 legacy=0x7f46380c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:20.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.982+0000 7f4635d9b700 1 -- 172.21.15.55:0/2870580142 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380cb100 con 0x7f46381566e0 2023-10-06T04:49:20.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.982+0000 7f4635d9b700 1 --2- 172.21.15.55:0/2870580142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46381566e0 0x7f46380ae000 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f46380addc0 tx=0x7f462c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529c301c4dc5d913 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:20.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.982+0000 7f463559a700 1 -- 172.21.15.55:0/2870580142 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c00b6b0 con 0x7f46381566e0 2023-10-06T04:49:20.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.982+0000 7f463559a700 1 -- 172.21.15.55:0/2870580142 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f462c00b810 con 0x7f46381566e0 2023-10-06T04:49:20.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.982+0000 7f463559a700 1 -- 172.21.15.55:0/2870580142 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c00bbb0 con 0x7f46381566e0 2023-10-06T04:49:20.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.983+0000 7f463759e700 1 -- 172.21.15.55:0/2870580142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46381566e0 msgr2=0x7f46380ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:20.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.983+0000 7f463759e700 1 --2- 172.21.15.55:0/2870580142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46381566e0 0x7f46380ae000 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f46380addc0 tx=0x7f462c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.983+0000 7f463759e700 1 -- 172.21.15.55:0/2870580142 shutdown_connections 2023-10-06T04:49:20.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.983+0000 7f463759e700 1 --2- 172.21.15.55:0/2870580142 >> v2:172.21.15.163:3300/0 conn(0x7f46380cb900 0x7f46380ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.983+0000 7f463759e700 1 --2- 172.21.15.55:0/2870580142 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46381566e0 0x7f46380ae000 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.983+0000 7f463759e700 1 -- 172.21.15.55:0/2870580142 >> 172.21.15.55:0/2870580142 conn(0x7f46380cc1f0 msgr2=0x7f46380cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:20.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.983+0000 7f463759e700 1 -- 172.21.15.55:0/2870580142 shutdown_connections 2023-10-06T04:49:20.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.983+0000 7f463759e700 1 -- 172.21.15.55:0/2870580142 wait complete. 2023-10-06T04:49:20.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.984+0000 7f463759e700 1 Processor -- start 2023-10-06T04:49:20.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.984+0000 7f463759e700 1 -- start start 2023-10-06T04:49:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.985+0000 7f463759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46380cb900 0x7f463814ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.985+0000 7f463759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4638156310 0x7f4638147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.985+0000 7f463759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46381566e0 0x7f463814e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.985+0000 7f463759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463814d8a0 con 0x7f46381566e0 2023-10-06T04:49:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.985+0000 7f463759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463814c5c0 con 0x7f4638156310 2023-10-06T04:49:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.985+0000 7f463759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f463814b9b0 con 0x7f46380cb900 2023-10-06T04:49:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.986+0000 7f4636d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46381566e0 0x7f463814e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.986+0000 7f4636d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46381566e0 0x7f463814e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52544/0 (socket says 172.21.15.55:52544) 2023-10-06T04:49:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.986+0000 7f4636d9d700 1 -- 172.21.15.55:0/4204854508 learned_addr learned my addr 172.21.15.55:0/4204854508 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.986+0000 7f463659c700 1 --2- 172.21.15.55:0/4204854508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46380cb900 0x7f463814ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.986+0000 7f4635d9b700 1 --2- 172.21.15.55:0/4204854508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4638156310 0x7f4638147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:20.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.986+0000 7f4636d9d700 1 -- 172.21.15.55:0/4204854508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46380cb900 msgr2=0x7f463814ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:20.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.986+0000 7f4636d9d700 1 --2- 172.21.15.55:0/4204854508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46380cb900 0x7f463814ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.986+0000 7f4636d9d700 1 -- 172.21.15.55:0/4204854508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4638156310 msgr2=0x7f4638147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:20.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.986+0000 7f4636d9d700 1 --2- 172.21.15.55:0/4204854508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4638156310 0x7f4638147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:20.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.986+0000 7f4636d9d700 1 -- 172.21.15.55:0/4204854508 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c009420 con 0x7f46381566e0 2023-10-06T04:49:20.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.986+0000 7f4636d9d700 1 --2- 172.21.15.55:0/4204854508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46381566e0 0x7f463814e660 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f4630007f20 tx=0x7f4630009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:20.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.987+0000 7f46267fc700 1 -- 172.21.15.55:0/4204854508 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463000ec20 con 0x7f46381566e0 2023-10-06T04:49:20.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.987+0000 7f46267fc700 1 -- 172.21.15.55:0/4204854508 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463000eda0 con 0x7f46381566e0 2023-10-06T04:49:20.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.987+0000 7f46267fc700 1 -- 172.21.15.55:0/4204854508 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630014670 con 0x7f46381566e0 2023-10-06T04:49:20.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.987+0000 7f463759e700 1 -- 172.21.15.55:0/4204854508 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463814bc90 con 0x7f46381566e0 2023-10-06T04:49:20.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.987+0000 7f463759e700 1 -- 172.21.15.55:0/4204854508 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463814c250 con 0x7f46381566e0 2023-10-06T04:49:20.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.988+0000 7f463759e700 1 -- 172.21.15.55:0/4204854508 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f463814c730 con 0x7f46381566e0 2023-10-06T04:49:20.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.989+0000 7f46267fc700 1 -- 172.21.15.55:0/4204854508 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f46300149e0 con 0x7f46381566e0 2023-10-06T04:49:20.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.990+0000 7f46267fc700 1 --2- 172.21.15.55:0/4204854508 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4618066f30 0x7f46180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:20.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.990+0000 7f46267fc700 1 -- 172.21.15.55:0/4204854508 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f4630084f50 con 0x7f46381566e0 2023-10-06T04:49:20.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.990+0000 7f463659c700 1 --2- 172.21.15.55:0/4204854508 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4618066f30 0x7f46180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:20.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.991+0000 7f463659c700 1 --2- 172.21.15.55:0/4204854508 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4618066f30 0x7f46180693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4628007130 tx=0x7f4628007eb0 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:20.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:20.994+0000 7f46267fc700 1 -- 172.21.15.55:0/4204854508 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4630051e30 con 0x7f46381566e0 2023-10-06T04:49:21.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.197+0000 7f463759e700 1 -- 172.21.15.55:0/4204854508 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f463804c3c0 con 0x7f46381566e0 2023-10-06T04:49:21.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.198+0000 7f46267fc700 1 -- 172.21.15.55:0/4204854508 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+17536 (secure 0 0 0) 0x7f4630055450 con 0x7f46381566e0 2023-10-06T04:49:21.200 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:21.200 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":172,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T04:49:19.807525+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:49:17.992792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T04:49:18.785942+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:49:21.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.200+0000 7f463759e700 1 -- 172.21.15.55:0/4204854508 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4618066f30 msgr2=0x7f46180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:21.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.200+0000 7f463759e700 1 --2- 172.21.15.55:0/4204854508 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4618066f30 0x7f46180693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4628007130 tx=0x7f4628007eb0 comp rx=0 tx=0).stop 2023-10-06T04:49:21.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.200+0000 7f463759e700 1 -- 172.21.15.55:0/4204854508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46381566e0 msgr2=0x7f463814e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:21.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.200+0000 7f463759e700 1 --2- 172.21.15.55:0/4204854508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46381566e0 0x7f463814e660 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f4630007f20 tx=0x7f4630009860 comp rx=0 tx=0).stop 2023-10-06T04:49:21.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.200+0000 7f463759e700 1 -- 172.21.15.55:0/4204854508 shutdown_connections 2023-10-06T04:49:21.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.200+0000 7f463759e700 1 --2- 172.21.15.55:0/4204854508 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4618066f30 0x7f46180693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:21.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.200+0000 7f463759e700 1 --2- 172.21.15.55:0/4204854508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46381566e0 0x7f463814e660 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:21.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.200+0000 7f463759e700 1 --2- 172.21.15.55:0/4204854508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46380cb900 0x7f463814ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:21.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.200+0000 7f463759e700 1 --2- 172.21.15.55:0/4204854508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4638156310 0x7f4638147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:21.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.200+0000 7f463759e700 1 -- 172.21.15.55:0/4204854508 >> 172.21.15.55:0/4204854508 conn(0x7f46380cc1f0 msgr2=0x7f463806a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:21.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.200+0000 7f463759e700 1 -- 172.21.15.55:0/4204854508 shutdown_connections 2023-10-06T04:49:21.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.200+0000 7f463759e700 1 -- 172.21.15.55:0/4204854508 wait complete. 2023-10-06T04:49:21.220 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T04:49:21.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.409+0000 7ff7baf9d700 1 Processor -- start 2023-10-06T04:49:21.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.409+0000 7ff7baf9d700 1 -- start start 2023-10-06T04:49:21.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.411+0000 7ff7baf9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff7b4140bb0 0x7ff7b406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:21.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.411+0000 7ff7baf9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff7b406da30 0x7ff7b40c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:21.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.411+0000 7ff7baf9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b40c6820 con 0x7ff7b41407e0 2023-10-06T04:49:21.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.411+0000 7ff7baf9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b406deb0 con 0x7ff7b406da30 2023-10-06T04:49:21.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.411+0000 7ff7baf9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b40c6de0 con 0x7ff7b4140bb0 2023-10-06T04:49:21.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.411+0000 7ff7ba79c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff7b406da30 0x7ff7b40c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:21.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.411+0000 7ff7ba79c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff7b406da30 0x7ff7b40c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52546/0 (socket says 172.21.15.55:52546) 2023-10-06T04:49:21.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.411+0000 7ff7ba79c700 1 -- 172.21.15.55:0/3805148808 learned_addr learned my addr 172.21.15.55:0/3805148808 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.411+0000 7ff7b979a700 1 --2- 172.21.15.55:0/3805148808 >> v2:172.21.15.163:3300/0 conn(0x7ff7b4140bb0 0x7ff7b406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.412+0000 7ff7ba79c700 1 -- 172.21.15.55:0/3805148808 >> v2:172.21.15.163:3300/0 conn(0x7ff7b4140bb0 msgr2=0x7ff7b406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.412+0000 7ff7ba79c700 1 --2- 172.21.15.55:0/3805148808 >> v2:172.21.15.163:3300/0 conn(0x7ff7b4140bb0 0x7ff7b406d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:21.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.412+0000 7ff7ba79c700 1 -- 172.21.15.55:0/3805148808 >> v1:172.21.15.163:6789/0 conn(0x7ff7b41407e0 legacy=0x7ff7b406a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:21.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.412+0000 7ff7ba79c700 1 -- 172.21.15.55:0/3805148808 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b4141270 con 0x7ff7b406da30 2023-10-06T04:49:21.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.412+0000 7ff7b979a700 1 --2- 172.21.15.55:0/3805148808 >> v2:172.21.15.163:3300/0 conn(0x7ff7b4140bb0 0x7ff7b406d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:49:21.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.412+0000 7ff7ba79c700 1 --2- 172.21.15.55:0/3805148808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b406da30 0x7ff7b40c7270 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40c3240 tx=0x7ff7ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1535f022bdc3a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:21.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.412+0000 7ff7b8f99700 1 -- 172.21.15.55:0/3805148808 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac00e070 con 0x7ff7b406da30 2023-10-06T04:49:21.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.412+0000 7ff7b8f99700 1 -- 172.21.15.55:0/3805148808 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac00ba50 con 0x7ff7b406da30 2023-10-06T04:49:21.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.412+0000 7ff7b8f99700 1 -- 172.21.15.55:0/3805148808 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac0092e0 con 0x7ff7b406da30 2023-10-06T04:49:21.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.413+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3805148808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b406da30 msgr2=0x7ff7b40c7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:21.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.413+0000 7ff7baf9d700 1 --2- 172.21.15.55:0/3805148808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b406da30 0x7ff7b40c7270 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40c3240 tx=0x7ff7ac009840 comp rx=0 tx=0).stop 2023-10-06T04:49:21.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.413+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3805148808 shutdown_connections 2023-10-06T04:49:21.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.413+0000 7ff7baf9d700 1 --2- 172.21.15.55:0/3805148808 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b406da30 0x7ff7b40c7270 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:21.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.413+0000 7ff7baf9d700 1 --2- 172.21.15.55:0/3805148808 >> v2:172.21.15.163:3300/0 conn(0x7ff7b4140bb0 0x7ff7b406d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:21.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.413+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3805148808 >> 172.21.15.55:0/3805148808 conn(0x7ff7b40cc1e0 msgr2=0x7ff7b40cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:21.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.413+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3805148808 shutdown_connections 2023-10-06T04:49:21.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.413+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3805148808 wait complete. 2023-10-06T04:49:21.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.414+0000 7ff7baf9d700 1 Processor -- start 2023-10-06T04:49:21.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.414+0000 7ff7baf9d700 1 -- start start 2023-10-06T04:49:21.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.415+0000 7ff7baf9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b406da30 0x7ff7b41325a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:21.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.415+0000 7ff7baf9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b41407e0 0x7ff7b4167530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:21.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.415+0000 7ff7baf9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b4140bb0 0x7ff7b40a7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.415+0000 7ff7baf9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b413cb30 con 0x7ff7b41407e0 2023-10-06T04:49:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.415+0000 7ff7baf9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b413d3d0 con 0x7ff7b406da30 2023-10-06T04:49:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.415+0000 7ff7baf9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b4132ae0 con 0x7ff7b4140bb0 2023-10-06T04:49:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.416+0000 7ff7b979a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b41407e0 0x7ff7b4167530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.416+0000 7ff7b979a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b41407e0 0x7ff7b4167530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52550/0 (socket says 172.21.15.55:52550) 2023-10-06T04:49:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.416+0000 7ff7b979a700 1 -- 172.21.15.55:0/3296934354 learned_addr learned my addr 172.21.15.55:0/3296934354 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.416+0000 7ff7ba79c700 1 --2- 172.21.15.55:0/3296934354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b4140bb0 0x7ff7b40a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.416+0000 7ff7b9f9b700 1 --2- 172.21.15.55:0/3296934354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b406da30 0x7ff7b41325a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.416+0000 7ff7b979a700 1 -- 172.21.15.55:0/3296934354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b4140bb0 msgr2=0x7ff7b40a7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.416+0000 7ff7b979a700 1 --2- 172.21.15.55:0/3296934354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b4140bb0 0x7ff7b40a7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.416+0000 7ff7b979a700 1 -- 172.21.15.55:0/3296934354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b406da30 msgr2=0x7ff7b41325a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:21.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.416+0000 7ff7b979a700 1 --2- 172.21.15.55:0/3296934354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b406da30 0x7ff7b41325a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.416+0000 7ff7b979a700 1 -- 172.21.15.55:0/3296934354 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac009ae0 con 0x7ff7b41407e0 2023-10-06T04:49:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.416+0000 7ff7b979a700 1 --2- 172.21.15.55:0/3296934354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b41407e0 0x7ff7b4167530 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400c880 tx=0x7ff7a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.417+0000 7ff7a9ffb700 1 -- 172.21.15.55:0/3296934354 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4011600 con 0x7ff7b41407e0 2023-10-06T04:49:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.417+0000 7ff7a9ffb700 1 -- 172.21.15.55:0/3296934354 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a4011760 con 0x7ff7b41407e0 2023-10-06T04:49:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.417+0000 7ff7a9ffb700 1 -- 172.21.15.55:0/3296934354 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4011b00 con 0x7ff7b41407e0 2023-10-06T04:49:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.417+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3296934354 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b40a6b50 con 0x7ff7b41407e0 2023-10-06T04:49:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.417+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3296934354 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b413d060 con 0x7ff7b41407e0 2023-10-06T04:49:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.418+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3296934354 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b413d540 con 0x7ff7b41407e0 2023-10-06T04:49:21.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.419+0000 7ff7a9ffb700 1 -- 172.21.15.55:0/3296934354 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7a4018350 con 0x7ff7b41407e0 2023-10-06T04:49:21.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.419+0000 7ff7a9ffb700 1 --2- 172.21.15.55:0/3296934354 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ff798066f30 0x7ff7980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:21.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.420+0000 7ff7a9ffb700 1 -- 172.21.15.55:0/3296934354 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7ff7a4084800 con 0x7ff7b41407e0 2023-10-06T04:49:21.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.420+0000 7ff7b9f9b700 1 --2- 172.21.15.55:0/3296934354 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ff798066f30 0x7ff7980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:21.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.421+0000 7ff7b9f9b700 1 --2- 172.21.15.55:0/3296934354 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ff798066f30 0x7ff7980693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0003860 tx=0x7ff7b0009040 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:21.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.424+0000 7ff7a9ffb700 1 -- 172.21.15.55:0/3296934354 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7a40516e0 con 0x7ff7b41407e0 2023-10-06T04:49:21.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:21.674+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3296934354 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff7b406acf0 con 0x7ff7b41407e0 2023-10-06T04:49:22.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.342+0000 7ff7a9ffb700 1 -- 172.21.15.55:0/3296934354 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v86) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff7a4054d00 con 0x7ff7b41407e0 2023-10-06T04:49:22.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.345+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3296934354 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ff798066f30 msgr2=0x7ff7980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:22.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.345+0000 7ff7baf9d700 1 --2- 172.21.15.55:0/3296934354 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ff798066f30 0x7ff7980693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0003860 tx=0x7ff7b0009040 comp rx=0 tx=0).stop 2023-10-06T04:49:22.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.345+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3296934354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b41407e0 msgr2=0x7ff7b4167530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:22.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.345+0000 7ff7baf9d700 1 --2- 172.21.15.55:0/3296934354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b41407e0 0x7ff7b4167530 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400c880 tx=0x7ff7a4008040 comp rx=0 tx=0).stop 2023-10-06T04:49:22.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.345+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3296934354 shutdown_connections 2023-10-06T04:49:22.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.345+0000 7ff7baf9d700 1 --2- 172.21.15.55:0/3296934354 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ff798066f30 0x7ff7980693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:22.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.345+0000 7ff7baf9d700 1 --2- 172.21.15.55:0/3296934354 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff7b4140bb0 0x7ff7b40a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:22.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.345+0000 7ff7baf9d700 1 --2- 172.21.15.55:0/3296934354 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff7b406da30 0x7ff7b41325a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:22.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.345+0000 7ff7baf9d700 1 --2- 172.21.15.55:0/3296934354 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff7b41407e0 0x7ff7b4167530 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:22.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.345+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3296934354 >> 172.21.15.55:0/3296934354 conn(0x7ff7b40cc1e0 msgr2=0x7ff7b40c0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:22.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.346+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3296934354 shutdown_connections 2023-10-06T04:49:22.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.346+0000 7ff7baf9d700 1 -- 172.21.15.55:0/3296934354 wait complete. 2023-10-06T04:49:22.368 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T04:49:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.562+0000 7f2ab759d700 1 Processor -- start 2023-10-06T04:49:22.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.563+0000 7f2ab759d700 1 -- start start 2023-10-06T04:49:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.564+0000 7f2ab759d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2ab0153900 0x7f2ab00c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.564+0000 7f2ab759d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ab00c9db0 0x7f2ab006a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.564+0000 7f2ab759d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2ab0157130 0x7f2ab006a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.564+0000 7f2ab759d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab006e150 con 0x7f2ab0153900 2023-10-06T04:49:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.564+0000 7f2ab759d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab00c7800 con 0x7f2ab0157130 2023-10-06T04:49:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.564+0000 7f2ab759d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00c7ae0 con 0x7f2ab00c9db0 2023-10-06T04:49:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.564+0000 7f2ab659b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2ab0153900 0x7f2ab00c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.564+0000 7f2ab659b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2ab0153900 0x7f2ab00c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52556/0 (socket says 172.21.15.55:52556) 2023-10-06T04:49:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.564+0000 7f2ab659b700 1 -- 172.21.15.55:0/2070824178 learned_addr learned my addr 172.21.15.55:0/2070824178 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:22.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.564+0000 7f2ab5d9a700 1 --2- 172.21.15.55:0/2070824178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ab00c9db0 0x7f2ab006a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.564+0000 7f2ab6d9c700 1 --2- 172.21.15.55:0/2070824178 >> v2:172.21.15.163:3300/0 conn(0x7f2ab0157130 0x7f2ab006a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.565+0000 7f2ab659b700 1 -- 172.21.15.55:0/2070824178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ab00c9db0 msgr2=0x7f2ab006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.565+0000 7f2ab659b700 1 --2- 172.21.15.55:0/2070824178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ab00c9db0 0x7f2ab006a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.565+0000 7f2ab659b700 1 -- 172.21.15.55:0/2070824178 >> v2:172.21.15.163:3300/0 conn(0x7f2ab0157130 msgr2=0x7f2ab006a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.565+0000 7f2ab659b700 1 --2- 172.21.15.55:0/2070824178 >> v2:172.21.15.163:3300/0 conn(0x7f2ab0157130 0x7f2ab006a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.565+0000 7f2ab659b700 1 -- 172.21.15.55:0/2070824178 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab00c7dc0 con 0x7f2ab0153900 2023-10-06T04:49:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.565+0000 7f2ab659b700 1 --2- 172.21.15.55:0/2070824178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ab0153900 0x7f2ab00c6f00 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f2ab01575b0 tx=0x7f2aac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324db053cbb0ab2e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.565+0000 7f2ab5599700 1 -- 172.21.15.55:0/2070824178 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00b640 con 0x7f2ab0153900 2023-10-06T04:49:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.565+0000 7f2ab5599700 1 -- 172.21.15.55:0/2070824178 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac00b7a0 con 0x7f2ab0153900 2023-10-06T04:49:22.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.565+0000 7f2ab5599700 1 -- 172.21.15.55:0/2070824178 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00bb60 con 0x7f2ab0153900 2023-10-06T04:49:22.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.566+0000 7f2ab759d700 1 -- 172.21.15.55:0/2070824178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ab0153900 msgr2=0x7f2ab00c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:22.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.566+0000 7f2ab759d700 1 --2- 172.21.15.55:0/2070824178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ab0153900 0x7f2ab00c6f00 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f2ab01575b0 tx=0x7f2aac009250 comp rx=0 tx=0).stop 2023-10-06T04:49:22.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.566+0000 7f2ab759d700 1 -- 172.21.15.55:0/2070824178 shutdown_connections 2023-10-06T04:49:22.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.566+0000 7f2ab759d700 1 --2- 172.21.15.55:0/2070824178 >> v2:172.21.15.163:3300/0 conn(0x7f2ab0157130 0x7f2ab006a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:22.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.566+0000 7f2ab759d700 1 --2- 172.21.15.55:0/2070824178 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ab0153900 0x7f2ab00c6f00 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:22.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.566+0000 7f2ab759d700 1 --2- 172.21.15.55:0/2070824178 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ab00c9db0 0x7f2ab006a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:22.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.566+0000 7f2ab759d700 1 -- 172.21.15.55:0/2070824178 >> 172.21.15.55:0/2070824178 conn(0x7f2ab00cc1e0 msgr2=0x7f2ab00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:22.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.566+0000 7f2ab759d700 1 -- 172.21.15.55:0/2070824178 shutdown_connections 2023-10-06T04:49:22.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.566+0000 7f2ab759d700 1 -- 172.21.15.55:0/2070824178 wait complete. 2023-10-06T04:49:22.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.567+0000 7f2ab759d700 1 Processor -- start 2023-10-06T04:49:22.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.567+0000 7f2ab759d700 1 -- start start 2023-10-06T04:49:22.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.568+0000 7f2ab759d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ab00c9db0 0x7f2ab0148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:22.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.568+0000 7f2ab759d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ab0153900 0x7f2ab0142f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:22.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.568+0000 7f2ab759d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ab0157130 0x7f2ab01558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:22.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.568+0000 7f2ab759d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0156050 con 0x7f2ab00c9db0 2023-10-06T04:49:22.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.568+0000 7f2ab759d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab00ad2c0 con 0x7f2ab0157130 2023-10-06T04:49:22.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.568+0000 7f2ab759d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0143450 con 0x7f2ab0153900 2023-10-06T04:49:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.569+0000 7f2ab659b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ab00c9db0 0x7f2ab0148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.569+0000 7f2ab659b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ab00c9db0 0x7f2ab0148ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52570/0 (socket says 172.21.15.55:52570) 2023-10-06T04:49:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.569+0000 7f2ab659b700 1 -- 172.21.15.55:0/3588233908 learned_addr learned my addr 172.21.15.55:0/3588233908 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.569+0000 7f2ab6d9c700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ab0157130 0x7f2ab01558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.569+0000 7f2ab5d9a700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ab0153900 0x7f2ab0142f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.569+0000 7f2ab659b700 1 -- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ab0153900 msgr2=0x7f2ab0142f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.569+0000 7f2ab659b700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ab0153900 0x7f2ab0142f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.569+0000 7f2ab659b700 1 -- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ab0157130 msgr2=0x7f2ab01558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.569+0000 7f2ab659b700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ab0157130 0x7f2ab01558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.569+0000 7f2ab659b700 1 -- 172.21.15.55:0/3588233908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac0094f0 con 0x7f2ab00c9db0 2023-10-06T04:49:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.569+0000 7f2ab5d9a700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ab0153900 0x7f2ab0142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:49:22.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.569+0000 7f2ab6d9c700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ab0157130 0x7f2ab01558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:49:22.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.569+0000 7f2ab659b700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ab00c9db0 0x7f2ab0148ea0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f2ab006ce80 tx=0x7f2aac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:22.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.570+0000 7f2aa67fc700 1 -- 172.21.15.55:0/3588233908 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac000d10 con 0x7f2ab00c9db0 2023-10-06T04:49:22.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.570+0000 7f2aa67fc700 1 -- 172.21.15.55:0/3588233908 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac00bb20 con 0x7f2ab00c9db0 2023-10-06T04:49:22.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.570+0000 7f2aa67fc700 1 -- 172.21.15.55:0/3588233908 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac011670 con 0x7f2ab00c9db0 2023-10-06T04:49:22.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.570+0000 7f2ab759d700 1 -- 172.21.15.55:0/3588233908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab016e190 con 0x7f2ab00c9db0 2023-10-06T04:49:22.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.570+0000 7f2ab759d700 1 -- 172.21.15.55:0/3588233908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab0155e00 con 0x7f2ab00c9db0 2023-10-06T04:49:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.571+0000 7f2ab759d700 1 -- 172.21.15.55:0/3588233908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab016e470 con 0x7f2ab00c9db0 2023-10-06T04:49:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.572+0000 7f2aa67fc700 1 -- 172.21.15.55:0/3588233908 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2aac0119e0 con 0x7f2ab00c9db0 2023-10-06T04:49:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.572+0000 7f2aa67fc700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a94066f30 0x7f2a940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.573+0000 7f2aa67fc700 1 -- 172.21.15.55:0/3588233908 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f2aac0860d0 con 0x7f2ab00c9db0 2023-10-06T04:49:22.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.573+0000 7f2ab5d9a700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a94066f30 0x7f2a940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:22.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.574+0000 7f2ab5d9a700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a94066f30 0x7f2a940693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0006f00 tx=0x7f2aa0008040 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:22.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.577+0000 7f2aa67fc700 1 -- 172.21.15.55:0/3588233908 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2aac053050 con 0x7f2ab00c9db0 2023-10-06T04:49:22.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:22.829+0000 7f2ab759d700 1 -- 172.21.15.55:0/3588233908 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2ab006dd20 con 0x7f2ab00c9db0 2023-10-06T04:49:23.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.346+0000 7f2aa67fc700 1 -- 172.21.15.55:0/3588233908 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v87) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2aac056670 con 0x7f2ab00c9db0 2023-10-06T04:49:23.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.349+0000 7f2ab759d700 1 -- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a94066f30 msgr2=0x7f2a940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:23.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.349+0000 7f2ab759d700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a94066f30 0x7f2a940693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0006f00 tx=0x7f2aa0008040 comp rx=0 tx=0).stop 2023-10-06T04:49:23.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.349+0000 7f2ab759d700 1 -- 172.21.15.55:0/3588233908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ab00c9db0 msgr2=0x7f2ab0148ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:23.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.349+0000 7f2ab759d700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ab00c9db0 0x7f2ab0148ea0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f2ab006ce80 tx=0x7f2aac017000 comp rx=0 tx=0).stop 2023-10-06T04:49:23.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.349+0000 7f2ab759d700 1 -- 172.21.15.55:0/3588233908 shutdown_connections 2023-10-06T04:49:23.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.349+0000 7f2ab759d700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2a94066f30 0x7f2a940693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:23.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.349+0000 7f2ab759d700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ab0157130 0x7f2ab01558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:23.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.349+0000 7f2ab759d700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ab00c9db0 0x7f2ab0148ea0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:23.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.349+0000 7f2ab759d700 1 --2- 172.21.15.55:0/3588233908 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ab0153900 0x7f2ab0142f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:23.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.349+0000 7f2ab759d700 1 -- 172.21.15.55:0/3588233908 >> 172.21.15.55:0/3588233908 conn(0x7f2ab00cc1e0 msgr2=0x7f2ab014d410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:23.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.349+0000 7f2ab759d700 1 -- 172.21.15.55:0/3588233908 shutdown_connections 2023-10-06T04:49:23.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.349+0000 7f2ab759d700 1 -- 172.21.15.55:0/3588233908 wait complete. 2023-10-06T04:49:23.369 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:49:23.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.553+0000 7f8bcd39d700 1 Processor -- start 2023-10-06T04:49:23.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.553+0000 7f8bcd39d700 1 -- start start 2023-10-06T04:49:23.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.555+0000 7f8bcd39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bc81595a0 0x7f8bc80b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:23.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.555+0000 7f8bcd39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc80b1410 con 0x7f8bc80c8830 2023-10-06T04:49:23.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.555+0000 7f8bcd39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc8063230 con 0x7f8bc80c6c60 2023-10-06T04:49:23.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.555+0000 7f8bcd39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc80b2b90 con 0x7f8bc81595a0 2023-10-06T04:49:23.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.555+0000 7f8bccb9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f8bc80c8830 0x7f8bc8159970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46690/0 (socket says 172.21.15.55:46690) 2023-10-06T04:49:23.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.555+0000 7f8bccb9c700 1 -- 172.21.15.55:0/1714563434 learned_addr learned my addr 172.21.15.55:0/1714563434 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:23.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.555+0000 7f8bc7fff700 1 --2- 172.21.15.55:0/1714563434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bc81595a0 0x7f8bc80b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:23.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 -- 172.21.15.55:0/1714563434 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836623095 0 0) 0x7f8bc80b1410 con 0x7f8bc80c8830 2023-10-06T04:49:23.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 -- 172.21.15.55:0/1714563434 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bb0003570 con 0x7f8bc80c8830 2023-10-06T04:49:23.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 -- 172.21.15.55:0/1714563434 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1126267363 0 0) 0x7f8bb0003570 con 0x7f8bc80c8830 2023-10-06T04:49:23.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 -- 172.21.15.55:0/1714563434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bc81595a0 msgr2=0x7f8bc80b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:23.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 --2- 172.21.15.55:0/1714563434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bc81595a0 0x7f8bc80b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:23.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 -- 172.21.15.55:0/1714563434 >> v1:172.21.15.163:6789/0 conn(0x7f8bc80c6c60 legacy=0x7f8bc80c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:23.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 -- 172.21.15.55:0/1714563434 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc80634f0 con 0x7f8bc80c8830 2023-10-06T04:49:23.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 -- 172.21.15.55:0/1714563434 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f8bc00017f0 con 0x7f8bc80c8830 2023-10-06T04:49:23.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 -- 172.21.15.55:0/1714563434 >> v1:172.21.15.55:6789/0 conn(0x7f8bc80c8830 legacy=0x7f8bc8159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:23.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 --2- 172.21.15.55:0/1714563434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bc81595a0 0x7f8bb0003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:23.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 --2- 172.21.15.55:0/1714563434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb00040e0 0x7f8bb0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:23.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 --2- 172.21.15.55:0/1714563434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bb0004da0 0x7f8bb0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:23.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 -- 172.21.15.55:0/1714563434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc80b2b90 con 0x7f8bb0004da0 2023-10-06T04:49:23.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 -- 172.21.15.55:0/1714563434 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb000bca0 con 0x7f8bb00040e0 2023-10-06T04:49:23.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bc6ffd700 1 -- 172.21.15.55:0/1714563434 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb000be60 con 0x7f8bc81595a0 2023-10-06T04:49:23.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.556+0000 7f8bccb9c700 1 --2- 172.21.15.55:0/1714563434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bb0004da0 0x7f8bb0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:23.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.557+0000 7f8bc77fe700 1 --2- 172.21.15.55:0/1714563434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb00040e0 0x7f8bb0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:23.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.557+0000 7f8bccb9c700 1 -- 172.21.15.55:0/1714563434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bc81595a0 msgr2=0x7f8bb0003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:23.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.557+0000 7f8bccb9c700 1 --2- 172.21.15.55:0/1714563434 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bc81595a0 0x7f8bb0003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:23.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.557+0000 7f8bccb9c700 1 -- 172.21.15.55:0/1714563434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb00040e0 msgr2=0x7f8bb0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:23.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.557+0000 7f8bccb9c700 1 --2- 172.21.15.55:0/1714563434 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bb00040e0 0x7f8bb0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:23.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.557+0000 7f8bccb9c700 1 -- 172.21.15.55:0/1714563434 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb0003570 con 0x7f8bb0004da0 2023-10-06T04:49:23.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.557+0000 7f8bccb9c700 1 --2- 172.21.15.55:0/1714563434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bb0004da0 0x7f8bb0005210 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80c9510 tx=0x7f8bc0002100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3385182a0001c3f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:23.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.557+0000 7f8bc6ffd700 1 -- 172.21.15.55:0/1714563434 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc000a210 con 0x7f8bb0004da0 2023-10-06T04:49:23.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.557+0000 7f8bc6ffd700 1 -- 172.21.15.55:0/1714563434 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc00070d0 con 0x7f8bb0004da0 2023-10-06T04:49:23.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.557+0000 7f8bc6ffd700 1 -- 172.21.15.55:0/1714563434 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0007490 con 0x7f8bb0004da0 2023-10-06T04:49:23.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.558+0000 7f8bcd39d700 1 -- 172.21.15.55:0/1714563434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bb0004da0 msgr2=0x7f8bb0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:23.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.558+0000 7f8bcd39d700 1 --2- 172.21.15.55:0/1714563434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bb0004da0 0x7f8bb0005210 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80c9510 tx=0x7f8bc0002100 comp rx=0 tx=0).stop 2023-10-06T04:49:23.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.558+0000 7f8bccb9c700 1 -- 172.21.15.55:0/1714563434 reap_dead start 2023-10-06T04:49:23.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.558+0000 7f8bcd39d700 1 -- 172.21.15.55:0/1714563434 shutdown_connections 2023-10-06T04:49:23.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.558+0000 7f8bcd39d700 1 -- 172.21.15.55:0/1714563434 >> 172.21.15.55:0/1714563434 conn(0x7f8bc806ddb0 msgr2=0x7f8bc806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:23.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.558+0000 7f8bcd39d700 1 -- 172.21.15.55:0/1714563434 shutdown_connections 2023-10-06T04:49:23.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.558+0000 7f8bcd39d700 1 -- 172.21.15.55:0/1714563434 wait complete. 2023-10-06T04:49:23.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.559+0000 7f8bcd39d700 1 Processor -- start 2023-10-06T04:49:23.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.560+0000 7f8bcd39d700 1 -- start start 2023-10-06T04:49:23.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.560+0000 7f8bcd39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb0004da0 0x7f8bc815bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:23.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.560+0000 7f8bcd39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bc815a380 0x7f8bc81543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:23.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bcd39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc815c620 0x7f8bc815caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:23.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bcd39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc8169c90 con 0x7f8bc815c620 2023-10-06T04:49:23.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bcd39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc815cfe0 con 0x7f8bc815a380 2023-10-06T04:49:23.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bcd39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc815b390 con 0x7f8bb0004da0 2023-10-06T04:49:23.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bccb9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc815c620 0x7f8bc815caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:23.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bccb9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc815c620 0x7f8bc815caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34438/0 (socket says 172.21.15.55:34438) 2023-10-06T04:49:23.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bccb9c700 1 -- 172.21.15.55:0/3516102373 learned_addr learned my addr 172.21.15.55:0/3516102373 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:23.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bc77fe700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bc815a380 0x7f8bc81543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:23.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bc7fff700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb0004da0 0x7f8bc815bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:23.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bccb9c700 1 -- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb0004da0 msgr2=0x7f8bc815bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:23.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bccb9c700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb0004da0 0x7f8bc815bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:23.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bccb9c700 1 -- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bc815a380 msgr2=0x7f8bc81543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:23.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bccb9c700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bc815a380 0x7f8bc81543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:23.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bccb9c700 1 -- 172.21.15.55:0/3516102373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc815a810 con 0x7f8bc815c620 2023-10-06T04:49:23.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bc77fe700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bc815a380 0x7f8bc81543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:49:23.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.561+0000 7f8bccb9c700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc815c620 0x7f8bc815caa0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0016040 tx=0x7f8bc0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:23.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.562+0000 7f8baffff700 1 -- 172.21.15.55:0/3516102373 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0007450 con 0x7f8bc815c620 2023-10-06T04:49:23.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.562+0000 7f8baffff700 1 -- 172.21.15.55:0/3516102373 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc001f430 con 0x7f8bc815c620 2023-10-06T04:49:23.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.562+0000 7f8baffff700 1 -- 172.21.15.55:0/3516102373 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc001f7f0 con 0x7f8bc815c620 2023-10-06T04:49:23.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.562+0000 7f8bcd39d700 1 -- 172.21.15.55:0/3516102373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc815aaf0 con 0x7f8bc815c620 2023-10-06T04:49:23.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.562+0000 7f8bcd39d700 1 -- 172.21.15.55:0/3516102373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc815b0b0 con 0x7f8bc815c620 2023-10-06T04:49:23.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.563+0000 7f8bcd39d700 1 -- 172.21.15.55:0/3516102373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc8169de0 con 0x7f8bc815c620 2023-10-06T04:49:23.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.564+0000 7f8baffff700 1 -- 172.21.15.55:0/3516102373 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8bc001fb60 con 0x7f8bc815c620 2023-10-06T04:49:23.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.565+0000 7f8baffff700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f8bb4066f30 0x7f8bb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:23.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.565+0000 7f8baffff700 1 -- 172.21.15.55:0/3516102373 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f8bc00859a0 con 0x7f8bc815c620 2023-10-06T04:49:23.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.565+0000 7f8bc7fff700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f8bb4066f30 0x7f8bb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:23.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.566+0000 7f8bc7fff700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f8bb4066f30 0x7f8bb40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009db0 tx=0x7f8bb80091a0 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:23.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.569+0000 7f8baffff700 1 -- 172.21.15.55:0/3516102373 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bc8169de0 con 0x7f8bc815c620 2023-10-06T04:49:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.816+0000 7f8bcd39d700 1 -- 172.21.15.55:0/3516102373 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bc8169de0 con 0x7f8bc815c620 2023-10-06T04:49:23.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.817+0000 7f8baffff700 1 -- 172.21.15.55:0/3516102373 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f8bc8169de0 con 0x7f8bc815c620 2023-10-06T04:49:23.819 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:23.819 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7682,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1565234675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1565234675},{"type":"v1","addr":"172.21.15.55:6834","nonce":1565234675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3042024952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3042024952},{"type":"v1","addr":"172.21.15.163:6835","nonce":3042024952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8737,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/304924610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":304924610},{"type":"v1","addr":"172.21.15.55:6835","nonce":304924610}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:49:19.807866+0000","modified":"2023-10-06T04:49:23.339948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8740":{"gid":8740,"name":"b","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6834/3095914721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3095914721},{"type":"v1","addr":"172.21.15.163:6834","nonce":3095914721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T04:49:23.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.819+0000 7f8bcd39d700 1 -- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f8bb4066f30 msgr2=0x7f8bb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:23.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.819+0000 7f8bcd39d700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f8bb4066f30 0x7f8bb40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009db0 tx=0x7f8bb80091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:23.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.819+0000 7f8bcd39d700 1 -- 172.21.15.55:0/3516102373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc815c620 msgr2=0x7f8bc815caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:23.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.819+0000 7f8bcd39d700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc815c620 0x7f8bc815caa0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0016040 tx=0x7f8bc0012000 comp rx=0 tx=0).stop 2023-10-06T04:49:23.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.819+0000 7f8bcd39d700 1 -- 172.21.15.55:0/3516102373 shutdown_connections 2023-10-06T04:49:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.819+0000 7f8bcd39d700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f8bb4066f30 0x7f8bb40693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.819+0000 7f8bcd39d700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8bc815c620 0x7f8bc815caa0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.819+0000 7f8bcd39d700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8bb0004da0 0x7f8bc815bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.819+0000 7f8bcd39d700 1 --2- 172.21.15.55:0/3516102373 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8bc815a380 0x7f8bc81543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.819+0000 7f8bcd39d700 1 -- 172.21.15.55:0/3516102373 >> 172.21.15.55:0/3516102373 conn(0x7f8bc806ddb0 msgr2=0x7f8bc8156ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.819+0000 7f8bcd39d700 1 -- 172.21.15.55:0/3516102373 shutdown_connections 2023-10-06T04:49:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:23.819+0000 7f8bcd39d700 1 -- 172.21.15.55:0/3516102373 wait complete. 2023-10-06T04:49:23.822 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 87 2023-10-06T04:49:23.838 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:49:24.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.025+0000 7f2ba499d700 1 Processor -- start 2023-10-06T04:49:24.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.025+0000 7f2ba499d700 1 -- start start 2023-10-06T04:49:24.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.026+0000 7f2ba499d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba01597f0 0x7f2ba00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.026+0000 7f2ba499d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba00b11d0 con 0x7f2ba0159420 2023-10-06T04:49:24.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.026+0000 7f2ba499d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba00b14b0 con 0x7f2ba015d230 2023-10-06T04:49:24.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.026+0000 7f2ba499d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba00b1790 con 0x7f2ba01597f0 2023-10-06T04:49:24.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.027+0000 7f2b9f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2ba0159420 0x7f2ba00b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46694/0 (socket says 172.21.15.55:46694) 2023-10-06T04:49:24.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.027+0000 7f2b9f7fe700 1 -- 172.21.15.55:0/1434230421 learned_addr learned my addr 172.21.15.55:0/1434230421 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:24.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.027+0000 7f2b9effd700 1 --2- 172.21.15.55:0/1434230421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba01597f0 0x7f2ba00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.027+0000 7f2b9e7fc700 1 -- 172.21.15.55:0/1434230421 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815479597 0 0) 0x7f2ba00b11d0 con 0x7f2ba0159420 2023-10-06T04:49:24.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.027+0000 7f2b9e7fc700 1 -- 172.21.15.55:0/1434230421 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b88003570 con 0x7f2ba0159420 2023-10-06T04:49:24.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.027+0000 7f2b9e7fc700 1 -- 172.21.15.55:0/1434230421 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296511542 0 0) 0x7f2ba00b14b0 con 0x7f2ba015d230 2023-10-06T04:49:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.027+0000 7f2b9effd700 1 -- 172.21.15.55:0/1434230421 >> v1:172.21.15.163:6789/0 conn(0x7f2ba015d230 legacy=0x7f2ba00b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.027+0000 7f2b9effd700 1 -- 172.21.15.55:0/1434230421 >> v1:172.21.15.55:6789/0 conn(0x7f2ba0159420 legacy=0x7f2ba00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.027+0000 7f2b9effd700 1 -- 172.21.15.55:0/1434230421 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba00b1a70 con 0x7f2ba01597f0 2023-10-06T04:49:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.027+0000 7f2b9effd700 1 --2- 172.21.15.55:0/1434230421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba01597f0 0x7f2ba00b2270 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009910 tx=0x7f2b940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f66e365da1b15e70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.028+0000 7f2b9e7fc700 1 -- 172.21.15.55:0/1434230421 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9400b680 con 0x7f2ba01597f0 2023-10-06T04:49:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.028+0000 7f2b9e7fc700 1 -- 172.21.15.55:0/1434230421 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9400b7e0 con 0x7f2ba01597f0 2023-10-06T04:49:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.028+0000 7f2b9e7fc700 1 -- 172.21.15.55:0/1434230421 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9400bba0 con 0x7f2ba01597f0 2023-10-06T04:49:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.028+0000 7f2ba499d700 1 -- 172.21.15.55:0/1434230421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba01597f0 msgr2=0x7f2ba00b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.028+0000 7f2ba499d700 1 --2- 172.21.15.55:0/1434230421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba01597f0 0x7f2ba00b2270 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009910 tx=0x7f2b940091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.028+0000 7f2ba499d700 1 -- 172.21.15.55:0/1434230421 shutdown_connections 2023-10-06T04:49:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.028+0000 7f2ba499d700 1 --2- 172.21.15.55:0/1434230421 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba01597f0 0x7f2ba00b2270 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.028+0000 7f2ba499d700 1 -- 172.21.15.55:0/1434230421 >> 172.21.15.55:0/1434230421 conn(0x7f2ba00d0660 msgr2=0x7f2ba00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:24.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.028+0000 7f2ba499d700 1 -- 172.21.15.55:0/1434230421 shutdown_connections 2023-10-06T04:49:24.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.029+0000 7f2ba499d700 1 -- 172.21.15.55:0/1434230421 wait complete. 2023-10-06T04:49:24.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.030+0000 7f2ba499d700 1 Processor -- start 2023-10-06T04:49:24.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.030+0000 7f2ba499d700 1 -- start start 2023-10-06T04:49:24.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.031+0000 7f2ba499d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba0159420 0x7f2ba014f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.031+0000 7f2ba499d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba01597f0 0x7f2ba0155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.031+0000 7f2ba499d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba015d230 0x7f2ba01561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.031+0000 7f2ba499d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba01554e0 con 0x7f2ba01597f0 2023-10-06T04:49:24.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.031+0000 7f2ba499d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba015a110 con 0x7f2ba0159420 2023-10-06T04:49:24.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.031+0000 7f2ba499d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba0154f20 con 0x7f2ba015d230 2023-10-06T04:49:24.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.031+0000 7f2b9effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba01597f0 0x7f2ba0155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.031+0000 7f2b9effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba01597f0 0x7f2ba0155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34452/0 (socket says 172.21.15.55:34452) 2023-10-06T04:49:24.034 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.031+0000 7f2b9effd700 1 -- 172.21.15.55:0/1729948257 learned_addr learned my addr 172.21.15.55:0/1729948257 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.031+0000 7f2b9f7fe700 1 --2- 172.21.15.55:0/1729948257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba0159420 0x7f2ba014f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.031+0000 7f2b9ffff700 1 --2- 172.21.15.55:0/1729948257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba015d230 0x7f2ba01561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.032+0000 7f2b9effd700 1 -- 172.21.15.55:0/1729948257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba015d230 msgr2=0x7f2ba01561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.032+0000 7f2b9effd700 1 --2- 172.21.15.55:0/1729948257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba015d230 0x7f2ba01561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.032+0000 7f2b9effd700 1 -- 172.21.15.55:0/1729948257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba0159420 msgr2=0x7f2ba014f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.032+0000 7f2b9effd700 1 --2- 172.21.15.55:0/1729948257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba0159420 0x7f2ba014f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.032+0000 7f2b9effd700 1 -- 172.21.15.55:0/1729948257 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b94009420 con 0x7f2ba01597f0 2023-10-06T04:49:24.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.032+0000 7f2b9effd700 1 --2- 172.21.15.55:0/1729948257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba01597f0 0x7f2ba0155c80 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0089340 tx=0x7f2b94017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.032+0000 7f2b877fe700 1 -- 172.21.15.55:0/1729948257 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9400bb60 con 0x7f2ba01597f0 2023-10-06T04:49:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.032+0000 7f2b877fe700 1 -- 172.21.15.55:0/1729948257 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b94000c90 con 0x7f2ba01597f0 2023-10-06T04:49:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.033+0000 7f2b877fe700 1 -- 172.21.15.55:0/1729948257 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b94027680 con 0x7f2ba01597f0 2023-10-06T04:49:24.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.033+0000 7f2ba499d700 1 -- 172.21.15.55:0/1729948257 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba0155200 con 0x7f2ba01597f0 2023-10-06T04:49:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.033+0000 7f2ba499d700 1 -- 172.21.15.55:0/1729948257 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba014fa50 con 0x7f2ba01597f0 2023-10-06T04:49:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.033+0000 7f2ba499d700 1 -- 172.21.15.55:0/1729948257 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba0159bc0 con 0x7f2ba01597f0 2023-10-06T04:49:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.035+0000 7f2b877fe700 1 -- 172.21.15.55:0/1729948257 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b9401a430 con 0x7f2ba01597f0 2023-10-06T04:49:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.035+0000 7f2b877fe700 1 --2- 172.21.15.55:0/1729948257 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b8c066f30 0x7f2b8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.037 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.036+0000 7f2b877fe700 1 -- 172.21.15.55:0/1729948257 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f2b940867d0 con 0x7f2ba01597f0 2023-10-06T04:49:24.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.036+0000 7f2b9f7fe700 1 --2- 172.21.15.55:0/1729948257 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b8c066f30 0x7f2b8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.038 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.037+0000 7f2b9f7fe700 1 --2- 172.21.15.55:0/1729948257 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b8c066f30 0x7f2b8c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2b90007fd0 tx=0x7f2b90009040 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:24.041 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.040+0000 7f2b877fe700 1 -- 172.21.15.55:0/1729948257 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b940536b0 con 0x7f2ba01597f0 2023-10-06T04:49:24.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.243+0000 7f2ba499d700 1 -- 172.21.15.55:0/1729948257 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ba015a280 con 0x7f2ba01597f0 2023-10-06T04:49:24.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.244+0000 7f2b877fe700 1 -- 172.21.15.55:0/1729948257 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+17523 (secure 0 0 0) 0x7f2b94056cd0 con 0x7f2ba01597f0 2023-10-06T04:49:24.246 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:24.247 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":173,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T04:49:21.324141+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:49:17.992792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T04:49:18.785942+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:49:24.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.247+0000 7f2ba499d700 1 -- 172.21.15.55:0/1729948257 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b8c066f30 msgr2=0x7f2b8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.247+0000 7f2ba499d700 1 --2- 172.21.15.55:0/1729948257 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b8c066f30 0x7f2b8c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2b90007fd0 tx=0x7f2b90009040 comp rx=0 tx=0).stop 2023-10-06T04:49:24.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.247+0000 7f2ba499d700 1 -- 172.21.15.55:0/1729948257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba01597f0 msgr2=0x7f2ba0155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.247+0000 7f2ba499d700 1 --2- 172.21.15.55:0/1729948257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba01597f0 0x7f2ba0155c80 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0089340 tx=0x7f2b94017000 comp rx=0 tx=0).stop 2023-10-06T04:49:24.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.247+0000 7f2ba499d700 1 -- 172.21.15.55:0/1729948257 shutdown_connections 2023-10-06T04:49:24.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.247+0000 7f2ba499d700 1 --2- 172.21.15.55:0/1729948257 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f2b8c066f30 0x7f2b8c0693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.247+0000 7f2ba499d700 1 --2- 172.21.15.55:0/1729948257 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2ba015d230 0x7f2ba01561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.247+0000 7f2ba499d700 1 --2- 172.21.15.55:0/1729948257 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2ba0159420 0x7f2ba014f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.247+0000 7f2ba499d700 1 --2- 172.21.15.55:0/1729948257 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2ba01597f0 0x7f2ba0155c80 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.247+0000 7f2ba499d700 1 -- 172.21.15.55:0/1729948257 >> 172.21.15.55:0/1729948257 conn(0x7f2ba00d0660 msgr2=0x7f2ba00ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:24.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.247+0000 7f2ba499d700 1 -- 172.21.15.55:0/1729948257 shutdown_connections 2023-10-06T04:49:24.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.247+0000 7f2ba499d700 1 -- 172.21.15.55:0/1729948257 wait complete. 2023-10-06T04:49:24.266 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T04:49:24.267 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:49:24.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.452+0000 7f4d9f59e700 1 Processor -- start 2023-10-06T04:49:24.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.452+0000 7f4d9f59e700 1 -- start start 2023-10-06T04:49:24.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.453+0000 7f4d9f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4da006a0a0 0x7f4da015b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.453+0000 7f4d9f59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4da006dd20 0x7f4da015b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.453+0000 7f4d9f59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da006ae70 con 0x7f4da006d950 2023-10-06T04:49:24.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.453+0000 7f4d9f59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da00c8a60 con 0x7f4da006dd20 2023-10-06T04:49:24.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.453+0000 7f4d9f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da006a540 con 0x7f4da006a0a0 2023-10-06T04:49:24.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.454+0000 7f4d9e59c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4da006d950 0x7f4da00c8350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46708/0 (socket says 172.21.15.55:46708) 2023-10-06T04:49:24.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.454+0000 7f4d9e59c700 1 -- 172.21.15.55:0/1010686777 learned_addr learned my addr 172.21.15.55:0/1010686777 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:24.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.454+0000 7f4d9dd9b700 1 --2- 172.21.15.55:0/1010686777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4da006a0a0 0x7f4da015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.454+0000 7f4d9ed9d700 1 --2- 172.21.15.55:0/1010686777 >> v2:172.21.15.163:3300/0 conn(0x7f4da006dd20 0x7f4da015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.454+0000 7f4d9d59a700 1 -- 172.21.15.55:0/1010686777 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415451892 0 0) 0x7f4da006ae70 con 0x7f4da006d950 2023-10-06T04:49:24.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.454+0000 7f4d9d59a700 1 -- 172.21.15.55:0/1010686777 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d88003570 con 0x7f4da006d950 2023-10-06T04:49:24.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.454+0000 7f4d9dd9b700 1 -- 172.21.15.55:0/1010686777 >> v2:172.21.15.163:3300/0 conn(0x7f4da006dd20 msgr2=0x7f4da015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:24.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.454+0000 7f4d9dd9b700 1 --2- 172.21.15.55:0/1010686777 >> v2:172.21.15.163:3300/0 conn(0x7f4da006dd20 0x7f4da015b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.454+0000 7f4d9dd9b700 1 -- 172.21.15.55:0/1010686777 >> v1:172.21.15.55:6789/0 conn(0x7f4da006d950 legacy=0x7f4da00c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.454+0000 7f4d9d59a700 1 -- 172.21.15.55:0/1010686777 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1020177286 0 0) 0x7f4d88003570 con 0x7f4da006d950 2023-10-06T04:49:24.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.454+0000 7f4d9dd9b700 1 -- 172.21.15.55:0/1010686777 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da006a820 con 0x7f4da006a0a0 2023-10-06T04:49:24.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.454+0000 7f4d9ed9d700 1 --2- 172.21.15.55:0/1010686777 >> v2:172.21.15.163:3300/0 conn(0x7f4da006dd20 0x7f4da015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:49:24.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.455+0000 7f4d9dd9b700 1 --2- 172.21.15.55:0/1010686777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4da006a0a0 0x7f4da015b410 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4d940098e0 tx=0x7f4d940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db936c61e916ec0c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:24.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.455+0000 7f4d9d59a700 1 -- 172.21.15.55:0/1010686777 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90001470 con 0x7f4da006a0a0 2023-10-06T04:49:24.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.455+0000 7f4d9d59a700 1 -- 172.21.15.55:0/1010686777 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d94011420 con 0x7f4da006a0a0 2023-10-06T04:49:24.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.455+0000 7f4d9d59a700 1 -- 172.21.15.55:0/1010686777 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d940117c0 con 0x7f4da006a0a0 2023-10-06T04:49:24.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.455+0000 7f4d9f59e700 1 -- 172.21.15.55:0/1010686777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4da006a0a0 msgr2=0x7f4da015b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.455+0000 7f4d9f59e700 1 --2- 172.21.15.55:0/1010686777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4da006a0a0 0x7f4da015b410 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4d940098e0 tx=0x7f4d940091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.456+0000 7f4d9f59e700 1 -- 172.21.15.55:0/1010686777 shutdown_connections 2023-10-06T04:49:24.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.456+0000 7f4d9f59e700 1 --2- 172.21.15.55:0/1010686777 >> v2:172.21.15.163:3300/0 conn(0x7f4da006dd20 0x7f4da015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.456+0000 7f4d9f59e700 1 --2- 172.21.15.55:0/1010686777 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4da006a0a0 0x7f4da015b410 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.456+0000 7f4d9f59e700 1 -- 172.21.15.55:0/1010686777 >> 172.21.15.55:0/1010686777 conn(0x7f4da00cc1f0 msgr2=0x7f4da00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:24.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.456+0000 7f4d9f59e700 1 -- 172.21.15.55:0/1010686777 shutdown_connections 2023-10-06T04:49:24.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.456+0000 7f4d9f59e700 1 -- 172.21.15.55:0/1010686777 wait complete. 2023-10-06T04:49:24.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.457+0000 7f4d9f59e700 1 Processor -- start 2023-10-06T04:49:24.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.457+0000 7f4d9f59e700 1 -- start start 2023-10-06T04:49:24.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.458+0000 7f4d9f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da006a0a0 0x7f4da00c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.458+0000 7f4d9f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4da006d950 0x7f4da0169fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.458+0000 7f4d9f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4da006dd20 0x7f4da0149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.458+0000 7f4d9f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da00c5b50 con 0x7f4da006a0a0 2023-10-06T04:49:24.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.458+0000 7f4d9f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da016a520 con 0x7f4da006dd20 2023-10-06T04:49:24.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.458+0000 7f4d9f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da00af410 con 0x7f4da006d950 2023-10-06T04:49:24.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.458+0000 7f4d9e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da006a0a0 0x7f4da00c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.458+0000 7f4d9e59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da006a0a0 0x7f4da00c5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34460/0 (socket says 172.21.15.55:34460) 2023-10-06T04:49:24.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.458+0000 7f4d9e59c700 1 -- 172.21.15.55:0/2137429330 learned_addr learned my addr 172.21.15.55:0/2137429330 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:24.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.458+0000 7f4d9ed9d700 1 --2- 172.21.15.55:0/2137429330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4da006dd20 0x7f4da0149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.458+0000 7f4d9dd9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4da006d950 0x7f4da0169fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.459+0000 7f4d9e59c700 1 -- 172.21.15.55:0/2137429330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4da006d950 msgr2=0x7f4da0169fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.459+0000 7f4d9e59c700 1 --2- 172.21.15.55:0/2137429330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4da006d950 0x7f4da0169fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.459+0000 7f4d9e59c700 1 -- 172.21.15.55:0/2137429330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4da006dd20 msgr2=0x7f4da0149410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.459+0000 7f4d9e59c700 1 --2- 172.21.15.55:0/2137429330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4da006dd20 0x7f4da0149410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.459+0000 7f4d9e59c700 1 -- 172.21.15.55:0/2137429330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d98009a20 con 0x7f4da006a0a0 2023-10-06T04:49:24.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.459+0000 7f4d9e59c700 1 --2- 172.21.15.55:0/2137429330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da006a0a0 0x7f4da00c5610 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000cb90 tx=0x7f4d90001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:24.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.459+0000 7f4d8e7fc700 1 -- 172.21.15.55:0/2137429330 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90007150 con 0x7f4da006a0a0 2023-10-06T04:49:24.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.459+0000 7f4d8e7fc700 1 -- 172.21.15.55:0/2137429330 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d900072d0 con 0x7f4da006a0a0 2023-10-06T04:49:24.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.459+0000 7f4d8e7fc700 1 -- 172.21.15.55:0/2137429330 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90007690 con 0x7f4da006a0a0 2023-10-06T04:49:24.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.459+0000 7f4d9f59e700 1 -- 172.21.15.55:0/2137429330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d94009420 con 0x7f4da006a0a0 2023-10-06T04:49:24.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.460+0000 7f4d9f59e700 1 -- 172.21.15.55:0/2137429330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da016a7d0 con 0x7f4da006a0a0 2023-10-06T04:49:24.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.461+0000 7f4d9f59e700 1 -- 172.21.15.55:0/2137429330 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da004c3c0 con 0x7f4da006a0a0 2023-10-06T04:49:24.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.462+0000 7f4d8e7fc700 1 -- 172.21.15.55:0/2137429330 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4d90004a40 con 0x7f4da006a0a0 2023-10-06T04:49:24.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.463+0000 7f4d8e7fc700 1 --2- 172.21.15.55:0/2137429330 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4d80066f30 0x7f4d800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.463+0000 7f4d8e7fc700 1 -- 172.21.15.55:0/2137429330 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f4d90083f90 con 0x7f4da006a0a0 2023-10-06T04:49:24.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.463+0000 7f4d9dd9b700 1 --2- 172.21.15.55:0/2137429330 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4d80066f30 0x7f4d800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.464+0000 7f4d9dd9b700 1 --2- 172.21.15.55:0/2137429330 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4d80066f30 0x7f4d800693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d940093f0 tx=0x7f4d94016000 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:24.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.466+0000 7f4d8e7fc700 1 -- 172.21.15.55:0/2137429330 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d90050fb0 con 0x7f4da006a0a0 2023-10-06T04:49:24.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.663+0000 7f4d9f59e700 1 -- 172.21.15.55:0/2137429330 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4da00caba0 con 0x7f4d80066f30 2023-10-06T04:49:24.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.676+0000 7f4d8e7fc700 1 -- 172.21.15.55:0/2137429330 <== mgr.8557 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309845 (secure 0 0 0) 0x7f4da00caba0 con 0x7f4d80066f30 2023-10-06T04:49:24.685 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:24.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.680+0000 7f4d9f59e700 1 -- 172.21.15.55:0/2137429330 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4d80066f30 msgr2=0x7f4d800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.680+0000 7f4d9f59e700 1 --2- 172.21.15.55:0/2137429330 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4d80066f30 0x7f4d800693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d940093f0 tx=0x7f4d94016000 comp rx=0 tx=0).stop 2023-10-06T04:49:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.680+0000 7f4d9f59e700 1 -- 172.21.15.55:0/2137429330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da006a0a0 msgr2=0x7f4da00c5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.680+0000 7f4d9f59e700 1 --2- 172.21.15.55:0/2137429330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da006a0a0 0x7f4da00c5610 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000cb90 tx=0x7f4d90001a80 comp rx=0 tx=0).stop 2023-10-06T04:49:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.680+0000 7f4d9f59e700 1 -- 172.21.15.55:0/2137429330 shutdown_connections 2023-10-06T04:49:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.680+0000 7f4d9f59e700 1 --2- 172.21.15.55:0/2137429330 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4d80066f30 0x7f4d800693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.680+0000 7f4d9f59e700 1 --2- 172.21.15.55:0/2137429330 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4da006dd20 0x7f4da0149410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.680+0000 7f4d9f59e700 1 --2- 172.21.15.55:0/2137429330 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4da006a0a0 0x7f4da00c5610 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.680+0000 7f4d9f59e700 1 --2- 172.21.15.55:0/2137429330 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4da006d950 0x7f4da0169fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.680+0000 7f4d9f59e700 1 -- 172.21.15.55:0/2137429330 >> 172.21.15.55:0/2137429330 conn(0x7f4da00cc1f0 msgr2=0x7f4da01551c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:24.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.680+0000 7f4d9f59e700 1 -- 172.21.15.55:0/2137429330 shutdown_connections 2023-10-06T04:49:24.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.681+0000 7f4d9f59e700 1 -- 172.21.15.55:0/2137429330 wait complete. 2023-10-06T04:49:24.689 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T04:49:24.700 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-06T04:49:23.337226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":523,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":330,"ondisk_log_size":330,"up":190,"acting":190,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2453660,"kb_used_data":18012,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752521060,"statfs":{"total":773094113280,"available":770581565440,"internally_reserved":0,"allocated":18444288,"data_stored":10203569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.036418"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346440+0000","last_change":"2023-10-06T04:49:20.341012+0000","last_active":"2023-10-06T04:49:20.346440+0000","last_peered":"2023-10-06T04:49:20.346440+0000","last_clean":"2023-10-06T04:49:20.346440+0000","last_became_active":"2023-10-06T04:49:20.340740+0000","last_became_peered":"2023-10-06T04:49:20.340740+0000","last_unstale":"2023-10-06T04:49:20.346440+0000","last_undegraded":"2023-10-06T04:49:20.346440+0000","last_fullsized":"2023-10-06T04:49:20.346440+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341092+0000","last_change":"2023-10-06T04:49:20.336745+0000","last_active":"2023-10-06T04:49:20.341092+0000","last_peered":"2023-10-06T04:49:20.341092+0000","last_clean":"2023-10-06T04:49:20.341092+0000","last_became_active":"2023-10-06T04:49:20.336500+0000","last_became_peered":"2023-10-06T04:49:20.336500+0000","last_unstale":"2023-10-06T04:49:20.341092+0000","last_undegraded":"2023-10-06T04:49:20.341092+0000","last_fullsized":"2023-10-06T04:49:20.341092+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:28.214277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341469+0000","last_change":"2023-10-06T04:49:20.334944+0000","last_active":"2023-10-06T04:49:20.341469+0000","last_peered":"2023-10-06T04:49:20.341469+0000","last_clean":"2023-10-06T04:49:20.341469+0000","last_became_active":"2023-10-06T04:49:20.333924+0000","last_became_peered":"2023-10-06T04:49:20.333924+0000","last_unstale":"2023-10-06T04:49:20.341469+0000","last_undegraded":"2023-10-06T04:49:20.341469+0000","last_fullsized":"2023-10-06T04:49:20.341469+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341150+0000","last_change":"2023-10-06T04:49:20.333903+0000","last_active":"2023-10-06T04:49:20.341150+0000","last_peered":"2023-10-06T04:49:20.341150+0000","last_clean":"2023-10-06T04:49:20.341150+0000","last_became_active":"2023-10-06T04:49:20.333572+0000","last_became_peered":"2023-10-06T04:49:20.333572+0000","last_unstale":"2023-10-06T04:49:20.341150+0000","last_undegraded":"2023-10-06T04:49:20.341150+0000","last_fullsized":"2023-10-06T04:49:20.341150+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:49.977768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339683+0000","last_change":"2023-10-06T04:49:20.338306+0000","last_active":"2023-10-06T04:49:20.339683+0000","last_peered":"2023-10-06T04:49:20.339683+0000","last_clean":"2023-10-06T04:49:20.339683+0000","last_became_active":"2023-10-06T04:49:20.338034+0000","last_became_peered":"2023-10-06T04:49:20.338034+0000","last_unstale":"2023-10-06T04:49:20.339683+0000","last_undegraded":"2023-10-06T04:49:20.339683+0000","last_fullsized":"2023-10-06T04:49:20.339683+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339447+0000","last_change":"2023-10-06T04:49:20.336417+0000","last_active":"2023-10-06T04:49:20.339447+0000","last_peered":"2023-10-06T04:49:20.339447+0000","last_clean":"2023-10-06T04:49:20.339447+0000","last_became_active":"2023-10-06T04:49:20.334348+0000","last_became_peered":"2023-10-06T04:49:20.334348+0000","last_unstale":"2023-10-06T04:49:20.339447+0000","last_undegraded":"2023-10-06T04:49:20.339447+0000","last_fullsized":"2023-10-06T04:49:20.339447+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:54.036275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.345389+0000","last_change":"2023-10-06T04:49:20.339825+0000","last_active":"2023-10-06T04:49:20.345389+0000","last_peered":"2023-10-06T04:49:20.345389+0000","last_clean":"2023-10-06T04:49:20.345389+0000","last_became_active":"2023-10-06T04:49:20.339565+0000","last_became_peered":"2023-10-06T04:49:20.339565+0000","last_unstale":"2023-10-06T04:49:20.345389+0000","last_undegraded":"2023-10-06T04:49:20.345389+0000","last_fullsized":"2023-10-06T04:49:20.345389+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:54.036275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.340009+0000","last_change":"2023-10-06T04:49:20.336646+0000","last_active":"2023-10-06T04:49:20.340009+0000","last_peered":"2023-10-06T04:49:20.340009+0000","last_clean":"2023-10-06T04:49:20.340009+0000","last_became_active":"2023-10-06T04:49:20.334441+0000","last_became_peered":"2023-10-06T04:49:20.334441+0000","last_unstale":"2023-10-06T04:49:20.340009+0000","last_undegraded":"2023-10-06T04:49:20.340009+0000","last_fullsized":"2023-10-06T04:49:20.340009+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:23.582332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333816+0000","last_change":"2023-10-06T04:49:20.331180+0000","last_active":"2023-10-06T04:49:20.333816+0000","last_peered":"2023-10-06T04:49:20.333816+0000","last_clean":"2023-10-06T04:49:20.333816+0000","last_became_active":"2023-10-06T04:49:20.330817+0000","last_became_peered":"2023-10-06T04:49:20.330817+0000","last_unstale":"2023-10-06T04:49:20.333816+0000","last_undegraded":"2023-10-06T04:49:20.333816+0000","last_fullsized":"2023-10-06T04:49:20.333816+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.336853+0000","last_change":"2023-10-06T04:49:20.336069+0000","last_active":"2023-10-06T04:49:20.336853+0000","last_peered":"2023-10-06T04:49:20.336853+0000","last_clean":"2023-10-06T04:49:20.336853+0000","last_became_active":"2023-10-06T04:49:20.335808+0000","last_became_peered":"2023-10-06T04:49:20.335808+0000","last_unstale":"2023-10-06T04:49:20.336853+0000","last_undegraded":"2023-10-06T04:49:20.336853+0000","last_fullsized":"2023-10-06T04:49:20.336853+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:53.477421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.334968+0000","last_change":"2023-10-06T04:49:20.328201+0000","last_active":"2023-10-06T04:49:20.334968+0000","last_peered":"2023-10-06T04:49:20.334968+0000","last_clean":"2023-10-06T04:49:20.334968+0000","last_became_active":"2023-10-06T04:49:20.327560+0000","last_became_peered":"2023-10-06T04:49:20.327560+0000","last_unstale":"2023-10-06T04:49:20.334968+0000","last_undegraded":"2023-10-06T04:49:20.334968+0000","last_fullsized":"2023-10-06T04:49:20.334968+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:40.721698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333739+0000","last_change":"2023-10-06T04:49:20.331130+0000","last_active":"2023-10-06T04:49:20.333739+0000","last_peered":"2023-10-06T04:49:20.333739+0000","last_clean":"2023-10-06T04:49:20.333739+0000","last_became_active":"2023-10-06T04:49:20.330637+0000","last_became_peered":"2023-10-06T04:49:20.330637+0000","last_unstale":"2023-10-06T04:49:20.333739+0000","last_undegraded":"2023-10-06T04:49:20.333739+0000","last_fullsized":"2023-10-06T04:49:20.333739+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341277+0000","last_change":"2023-10-06T04:49:20.337139+0000","last_active":"2023-10-06T04:49:20.341277+0000","last_peered":"2023-10-06T04:49:20.341277+0000","last_clean":"2023-10-06T04:49:20.341277+0000","last_became_active":"2023-10-06T04:49:20.336396+0000","last_became_peered":"2023-10-06T04:49:20.336396+0000","last_unstale":"2023-10-06T04:49:20.341277+0000","last_undegraded":"2023-10-06T04:49:20.341277+0000","last_fullsized":"2023-10-06T04:49:20.341277+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:22.496792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341177+0000","last_change":"2023-10-06T04:49:20.337332+0000","last_active":"2023-10-06T04:49:20.341177+0000","last_peered":"2023-10-06T04:49:20.341177+0000","last_clean":"2023-10-06T04:49:20.341177+0000","last_became_active":"2023-10-06T04:49:20.337091+0000","last_became_peered":"2023-10-06T04:49:20.337091+0000","last_unstale":"2023-10-06T04:49:20.341177+0000","last_undegraded":"2023-10-06T04:49:20.341177+0000","last_fullsized":"2023-10-06T04:49:20.341177+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:57.445018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333411+0000","last_change":"2023-10-06T04:49:20.329112+0000","last_active":"2023-10-06T04:49:20.333411+0000","last_peered":"2023-10-06T04:49:20.333411+0000","last_clean":"2023-10-06T04:49:20.333411+0000","last_became_active":"2023-10-06T04:49:20.328156+0000","last_became_peered":"2023-10-06T04:49:20.328156+0000","last_unstale":"2023-10-06T04:49:20.333411+0000","last_undegraded":"2023-10-06T04:49:20.333411+0000","last_fullsized":"2023-10-06T04:49:20.333411+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:25.017277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341591+0000","last_change":"2023-10-06T04:49:20.334020+0000","last_active":"2023-10-06T04:49:20.341591+0000","last_peered":"2023-10-06T04:49:20.341591+0000","last_clean":"2023-10-06T04:49:20.341591+0000","last_became_active":"2023-10-06T04:49:20.333516+0000","last_became_peered":"2023-10-06T04:49:20.333516+0000","last_unstale":"2023-10-06T04:49:20.341591+0000","last_undegraded":"2023-10-06T04:49:20.341591+0000","last_fullsized":"2023-10-06T04:49:20.341591+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:43.552397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347965+0000","last_change":"2023-10-06T04:49:20.346422+0000","last_active":"2023-10-06T04:49:20.347965+0000","last_peered":"2023-10-06T04:49:20.347965+0000","last_clean":"2023-10-06T04:49:20.347965+0000","last_became_active":"2023-10-06T04:49:20.346126+0000","last_became_peered":"2023-10-06T04:49:20.346126+0000","last_unstale":"2023-10-06T04:49:20.347965+0000","last_undegraded":"2023-10-06T04:49:20.347965+0000","last_fullsized":"2023-10-06T04:49:20.347965+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:58.644879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355577+0000","last_change":"2023-10-06T04:49:20.350471+0000","last_active":"2023-10-06T04:49:20.355577+0000","last_peered":"2023-10-06T04:49:20.355577+0000","last_clean":"2023-10-06T04:49:20.355577+0000","last_became_active":"2023-10-06T04:49:20.350195+0000","last_became_peered":"2023-10-06T04:49:20.350195+0000","last_unstale":"2023-10-06T04:49:20.355577+0000","last_undegraded":"2023-10-06T04:49:20.355577+0000","last_fullsized":"2023-10-06T04:49:20.355577+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:12.727118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339262+0000","last_change":"2023-10-06T04:49:20.334837+0000","last_active":"2023-10-06T04:49:20.339262+0000","last_peered":"2023-10-06T04:49:20.339262+0000","last_clean":"2023-10-06T04:49:20.339262+0000","last_became_active":"2023-10-06T04:49:20.334580+0000","last_became_peered":"2023-10-06T04:49:20.334580+0000","last_unstale":"2023-10-06T04:49:20.339262+0000","last_undegraded":"2023-10-06T04:49:20.339262+0000","last_fullsized":"2023-10-06T04:49:20.339262+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:02.862072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.338389+0000","last_change":"2023-10-06T04:49:20.334048+0000","last_active":"2023-10-06T04:49:20.338389+0000","last_peered":"2023-10-06T04:49:20.338389+0000","last_clean":"2023-10-06T04:49:20.338389+0000","last_became_active":"2023-10-06T04:49:20.333761+0000","last_became_peered":"2023-10-06T04:49:20.333761+0000","last_unstale":"2023-10-06T04:49:20.338389+0000","last_undegraded":"2023-10-06T04:49:20.338389+0000","last_fullsized":"2023-10-06T04:49:20.338389+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344972+0000","last_change":"2023-10-06T04:49:20.336537+0000","last_active":"2023-10-06T04:49:20.344972+0000","last_peered":"2023-10-06T04:49:20.344972+0000","last_clean":"2023-10-06T04:49:20.344972+0000","last_became_active":"2023-10-06T04:49:20.336251+0000","last_became_peered":"2023-10-06T04:49:20.336251+0000","last_unstale":"2023-10-06T04:49:20.344972+0000","last_undegraded":"2023-10-06T04:49:20.344972+0000","last_fullsized":"2023-10-06T04:49:20.344972+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.329722+0000","last_change":"2023-10-06T04:49:20.326750+0000","last_active":"2023-10-06T04:49:20.329722+0000","last_peered":"2023-10-06T04:49:20.329722+0000","last_clean":"2023-10-06T04:49:20.329722+0000","last_became_active":"2023-10-06T04:49:20.326466+0000","last_became_peered":"2023-10-06T04:49:20.326466+0000","last_unstale":"2023-10-06T04:49:20.329722+0000","last_undegraded":"2023-10-06T04:49:20.329722+0000","last_fullsized":"2023-10-06T04:49:20.329722+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:52.748607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.345875+0000","last_change":"2023-10-06T04:49:20.337383+0000","last_active":"2023-10-06T04:49:20.345875+0000","last_peered":"2023-10-06T04:49:20.345875+0000","last_clean":"2023-10-06T04:49:20.345875+0000","last_became_active":"2023-10-06T04:49:20.337130+0000","last_became_peered":"2023-10-06T04:49:20.337130+0000","last_unstale":"2023-10-06T04:49:20.345875+0000","last_undegraded":"2023-10-06T04:49:20.345875+0000","last_fullsized":"2023-10-06T04:49:20.345875+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355403+0000","last_change":"2023-10-06T04:49:20.347903+0000","last_active":"2023-10-06T04:49:20.355403+0000","last_peered":"2023-10-06T04:49:20.355403+0000","last_clean":"2023-10-06T04:49:20.355403+0000","last_became_active":"2023-10-06T04:49:20.346956+0000","last_became_peered":"2023-10-06T04:49:20.346956+0000","last_unstale":"2023-10-06T04:49:20.355403+0000","last_undegraded":"2023-10-06T04:49:20.355403+0000","last_fullsized":"2023-10-06T04:49:20.355403+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:20.026936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339401+0000","last_change":"2023-10-06T04:49:20.336472+0000","last_active":"2023-10-06T04:49:20.339401+0000","last_peered":"2023-10-06T04:49:20.339401+0000","last_clean":"2023-10-06T04:49:20.339401+0000","last_became_active":"2023-10-06T04:49:20.334373+0000","last_became_peered":"2023-10-06T04:49:20.334373+0000","last_unstale":"2023-10-06T04:49:20.339401+0000","last_undegraded":"2023-10-06T04:49:20.339401+0000","last_fullsized":"2023-10-06T04:49:20.339401+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:20.026936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346959+0000","last_change":"2023-10-06T04:49:20.344850+0000","last_active":"2023-10-06T04:49:20.346959+0000","last_peered":"2023-10-06T04:49:20.346959+0000","last_clean":"2023-10-06T04:49:20.346959+0000","last_became_active":"2023-10-06T04:49:20.343891+0000","last_became_peered":"2023-10-06T04:49:20.343891+0000","last_unstale":"2023-10-06T04:49:20.346959+0000","last_undegraded":"2023-10-06T04:49:20.346959+0000","last_fullsized":"2023-10-06T04:49:20.346959+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:28.957190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347992+0000","last_change":"2023-10-06T04:49:20.344409+0000","last_active":"2023-10-06T04:49:20.347992+0000","last_peered":"2023-10-06T04:49:20.347992+0000","last_clean":"2023-10-06T04:49:20.347992+0000","last_became_active":"2023-10-06T04:49:20.343734+0000","last_became_peered":"2023-10-06T04:49:20.343734+0000","last_unstale":"2023-10-06T04:49:20.347992+0000","last_undegraded":"2023-10-06T04:49:20.347992+0000","last_fullsized":"2023-10-06T04:49:20.347992+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:27.555498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333496+0000","last_change":"2023-10-06T04:49:20.327245+0000","last_active":"2023-10-06T04:49:20.333496+0000","last_peered":"2023-10-06T04:49:20.333496+0000","last_clean":"2023-10-06T04:49:20.333496+0000","last_became_active":"2023-10-06T04:49:20.326914+0000","last_became_peered":"2023-10-06T04:49:20.326914+0000","last_unstale":"2023-10-06T04:49:20.333496+0000","last_undegraded":"2023-10-06T04:49:20.333496+0000","last_fullsized":"2023-10-06T04:49:20.333496+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:19.514355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341007+0000","last_change":"2023-10-06T04:49:20.333556+0000","last_active":"2023-10-06T04:49:20.341007+0000","last_peered":"2023-10-06T04:49:20.341007+0000","last_clean":"2023-10-06T04:49:20.341007+0000","last_became_active":"2023-10-06T04:49:20.333251+0000","last_became_peered":"2023-10-06T04:49:20.333251+0000","last_unstale":"2023-10-06T04:49:20.341007+0000","last_undegraded":"2023-10-06T04:49:20.341007+0000","last_fullsized":"2023-10-06T04:49:20.341007+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"172'4","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344554+0000","last_change":"2023-10-06T04:49:19.320190+0000","last_active":"2023-10-06T04:49:20.344554+0000","last_peered":"2023-10-06T04:49:20.344554+0000","last_clean":"2023-10-06T04:49:20.344554+0000","last_became_active":"2023-10-06T04:49:19.319917+0000","last_became_peered":"2023-10-06T04:49:19.319917+0000","last_unstale":"2023-10-06T04:49:20.344554+0000","last_undegraded":"2023-10-06T04:49:20.344554+0000","last_fullsized":"2023-10-06T04:49:20.344554+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:26.266062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341489+0000","last_change":"2023-10-06T04:49:20.336237+0000","last_active":"2023-10-06T04:49:20.341489+0000","last_peered":"2023-10-06T04:49:20.341489+0000","last_clean":"2023-10-06T04:49:20.341489+0000","last_became_active":"2023-10-06T04:49:20.335986+0000","last_became_peered":"2023-10-06T04:49:20.335986+0000","last_unstale":"2023-10-06T04:49:20.341489+0000","last_undegraded":"2023-10-06T04:49:20.341489+0000","last_fullsized":"2023-10-06T04:49:20.341489+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355736+0000","last_change":"2023-10-06T04:49:19.335778+0000","last_active":"2023-10-06T04:49:20.355736+0000","last_peered":"2023-10-06T04:49:20.355736+0000","last_clean":"2023-10-06T04:49:20.355736+0000","last_became_active":"2023-10-06T04:49:19.335470+0000","last_became_peered":"2023-10-06T04:49:19.335470+0000","last_unstale":"2023-10-06T04:49:20.355736+0000","last_undegraded":"2023-10-06T04:49:20.355736+0000","last_fullsized":"2023-10-06T04:49:20.355736+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:01.854229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356161+0000","last_change":"2023-10-06T04:49:19.335543+0000","last_active":"2023-10-06T04:49:20.356161+0000","last_peered":"2023-10-06T04:49:20.356161+0000","last_clean":"2023-10-06T04:49:20.356161+0000","last_became_active":"2023-10-06T04:49:19.335265+0000","last_became_peered":"2023-10-06T04:49:19.335265+0000","last_unstale":"2023-10-06T04:49:20.356161+0000","last_undegraded":"2023-10-06T04:49:20.356161+0000","last_fullsized":"2023-10-06T04:49:20.356161+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:41.377516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350856+0000","last_change":"2023-10-06T04:49:20.346902+0000","last_active":"2023-10-06T04:49:20.350856+0000","last_peered":"2023-10-06T04:49:20.350856+0000","last_clean":"2023-10-06T04:49:20.350856+0000","last_became_active":"2023-10-06T04:49:20.346594+0000","last_became_peered":"2023-10-06T04:49:20.346594+0000","last_unstale":"2023-10-06T04:49:20.350856+0000","last_undegraded":"2023-10-06T04:49:20.350856+0000","last_fullsized":"2023-10-06T04:49:20.350856+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:00.206328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.349018+0000","last_change":"2023-10-06T04:49:19.328489+0000","last_active":"2023-10-06T04:49:20.349018+0000","last_peered":"2023-10-06T04:49:20.349018+0000","last_clean":"2023-10-06T04:49:20.349018+0000","last_became_active":"2023-10-06T04:49:19.328220+0000","last_became_peered":"2023-10-06T04:49:19.328220+0000","last_unstale":"2023-10-06T04:49:20.349018+0000","last_undegraded":"2023-10-06T04:49:20.349018+0000","last_fullsized":"2023-10-06T04:49:20.349018+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:26.266062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.337318+0000","last_change":"2023-10-06T04:49:20.330590+0000","last_active":"2023-10-06T04:49:20.337318+0000","last_peered":"2023-10-06T04:49:20.337318+0000","last_clean":"2023-10-06T04:49:20.337318+0000","last_became_active":"2023-10-06T04:49:20.330306+0000","last_became_peered":"2023-10-06T04:49:20.330306+0000","last_unstale":"2023-10-06T04:49:20.337318+0000","last_undegraded":"2023-10-06T04:49:20.337318+0000","last_fullsized":"2023-10-06T04:49:20.337318+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:42.535859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:49:18.291949+0000","last_change":"2023-10-06T04:49:18.303303+0000","last_active":"2023-10-06T04:49:18.291949+0000","last_peered":"2023-10-06T04:49:18.291949+0000","last_clean":"2023-10-06T04:49:18.291949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:18.291949+0000","last_undegraded":"2023-10-06T04:49:18.291949+0000","last_fullsized":"2023-10-06T04:49:18.291949+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:49.003200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341432+0000","last_change":"2023-10-06T04:49:20.334543+0000","last_active":"2023-10-06T04:49:20.341432+0000","last_peered":"2023-10-06T04:49:20.341432+0000","last_clean":"2023-10-06T04:49:20.341432+0000","last_became_active":"2023-10-06T04:49:20.333665+0000","last_became_peered":"2023-10-06T04:49:20.333665+0000","last_unstale":"2023-10-06T04:49:20.341432+0000","last_undegraded":"2023-10-06T04:49:20.341432+0000","last_fullsized":"2023-10-06T04:49:20.341432+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:20.869185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347200+0000","last_change":"2023-10-06T04:49:19.330786+0000","last_active":"2023-10-06T04:49:20.347200+0000","last_peered":"2023-10-06T04:49:20.347200+0000","last_clean":"2023-10-06T04:49:20.347200+0000","last_became_active":"2023-10-06T04:49:19.330506+0000","last_became_peered":"2023-10-06T04:49:19.330506+0000","last_unstale":"2023-10-06T04:49:20.347200+0000","last_undegraded":"2023-10-06T04:49:20.347200+0000","last_fullsized":"2023-10-06T04:49:20.347200+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:54.415706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333277+0000","last_change":"2023-10-06T04:49:20.332521+0000","last_active":"2023-10-06T04:49:20.333277+0000","last_peered":"2023-10-06T04:49:20.333277+0000","last_clean":"2023-10-06T04:49:20.333277+0000","last_became_active":"2023-10-06T04:49:20.332264+0000","last_became_peered":"2023-10-06T04:49:20.332264+0000","last_unstale":"2023-10-06T04:49:20.333277+0000","last_undegraded":"2023-10-06T04:49:20.333277+0000","last_fullsized":"2023-10-06T04:49:20.333277+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:16.543656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332961+0000","last_change":"2023-10-06T04:49:19.332296+0000","last_active":"2023-10-06T04:49:20.332961+0000","last_peered":"2023-10-06T04:49:20.332961+0000","last_clean":"2023-10-06T04:49:20.332961+0000","last_became_active":"2023-10-06T04:49:19.332022+0000","last_became_peered":"2023-10-06T04:49:19.332022+0000","last_unstale":"2023-10-06T04:49:20.332961+0000","last_undegraded":"2023-10-06T04:49:20.332961+0000","last_fullsized":"2023-10-06T04:49:20.332961+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:18.846190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350851+0000","last_change":"2023-10-06T04:49:20.341492+0000","last_active":"2023-10-06T04:49:20.350851+0000","last_peered":"2023-10-06T04:49:20.350851+0000","last_clean":"2023-10-06T04:49:20.350851+0000","last_became_active":"2023-10-06T04:49:20.341257+0000","last_became_peered":"2023-10-06T04:49:20.341257+0000","last_unstale":"2023-10-06T04:49:20.350851+0000","last_undegraded":"2023-10-06T04:49:20.350851+0000","last_fullsized":"2023-10-06T04:49:20.350851+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:28.214277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.315096+0000","last_change":"2023-10-06T04:49:19.324312+0000","last_active":"2023-10-06T04:49:20.315096+0000","last_peered":"2023-10-06T04:49:20.315096+0000","last_clean":"2023-10-06T04:49:20.315096+0000","last_became_active":"2023-10-06T04:49:19.324046+0000","last_became_peered":"2023-10-06T04:49:19.324046+0000","last_unstale":"2023-10-06T04:49:20.315096+0000","last_undegraded":"2023-10-06T04:49:20.315096+0000","last_fullsized":"2023-10-06T04:49:20.315096+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:49.588344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344426+0000","last_change":"2023-10-06T04:49:20.341875+0000","last_active":"2023-10-06T04:49:20.344426+0000","last_peered":"2023-10-06T04:49:20.344426+0000","last_clean":"2023-10-06T04:49:20.344426+0000","last_became_active":"2023-10-06T04:49:20.341312+0000","last_became_peered":"2023-10-06T04:49:20.341312+0000","last_unstale":"2023-10-06T04:49:20.344426+0000","last_undegraded":"2023-10-06T04:49:20.344426+0000","last_fullsized":"2023-10-06T04:49:20.344426+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:07.881803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.351368+0000","last_change":"2023-10-06T04:49:19.331266+0000","last_active":"2023-10-06T04:49:20.351368+0000","last_peered":"2023-10-06T04:49:20.351368+0000","last_clean":"2023-10-06T04:49:20.351368+0000","last_became_active":"2023-10-06T04:49:19.331010+0000","last_became_peered":"2023-10-06T04:49:19.331010+0000","last_unstale":"2023-10-06T04:49:20.351368+0000","last_undegraded":"2023-10-06T04:49:20.351368+0000","last_fullsized":"2023-10-06T04:49:20.351368+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:41.647975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355558+0000","last_change":"2023-10-06T04:49:20.350352+0000","last_active":"2023-10-06T04:49:20.355558+0000","last_peered":"2023-10-06T04:49:20.355558+0000","last_clean":"2023-10-06T04:49:20.355558+0000","last_became_active":"2023-10-06T04:49:20.350084+0000","last_became_peered":"2023-10-06T04:49:20.350084+0000","last_unstale":"2023-10-06T04:49:20.355558+0000","last_undegraded":"2023-10-06T04:49:20.355558+0000","last_fullsized":"2023-10-06T04:49:20.355558+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:54.488863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.353329+0000","last_change":"2023-10-06T04:49:19.333467+0000","last_active":"2023-10-06T04:49:20.353329+0000","last_peered":"2023-10-06T04:49:20.353329+0000","last_clean":"2023-10-06T04:49:20.353329+0000","last_became_active":"2023-10-06T04:49:19.333191+0000","last_became_peered":"2023-10-06T04:49:19.333191+0000","last_unstale":"2023-10-06T04:49:20.353329+0000","last_undegraded":"2023-10-06T04:49:20.353329+0000","last_fullsized":"2023-10-06T04:49:20.353329+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:50.631194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346482+0000","last_change":"2023-10-06T04:49:20.340075+0000","last_active":"2023-10-06T04:49:20.346482+0000","last_peered":"2023-10-06T04:49:20.346482+0000","last_clean":"2023-10-06T04:49:20.346482+0000","last_became_active":"2023-10-06T04:49:20.339781+0000","last_became_peered":"2023-10-06T04:49:20.339781+0000","last_unstale":"2023-10-06T04:49:20.346482+0000","last_undegraded":"2023-10-06T04:49:20.346482+0000","last_fullsized":"2023-10-06T04:49:20.346482+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:57.445018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344697+0000","last_change":"2023-10-06T04:49:19.324434+0000","last_active":"2023-10-06T04:49:20.344697+0000","last_peered":"2023-10-06T04:49:20.344697+0000","last_clean":"2023-10-06T04:49:20.344697+0000","last_became_active":"2023-10-06T04:49:19.324162+0000","last_became_peered":"2023-10-06T04:49:19.324162+0000","last_unstale":"2023-10-06T04:49:20.344697+0000","last_undegraded":"2023-10-06T04:49:20.344697+0000","last_fullsized":"2023-10-06T04:49:20.344697+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350745+0000","last_change":"2023-10-06T04:49:20.345980+0000","last_active":"2023-10-06T04:49:20.350745+0000","last_peered":"2023-10-06T04:49:20.350745+0000","last_clean":"2023-10-06T04:49:20.350745+0000","last_became_active":"2023-10-06T04:49:20.345691+0000","last_became_peered":"2023-10-06T04:49:20.345691+0000","last_unstale":"2023-10-06T04:49:20.350745+0000","last_undegraded":"2023-10-06T04:49:20.350745+0000","last_fullsized":"2023-10-06T04:49:20.350745+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:23.582332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.319623+0000","last_change":"2023-10-06T04:49:19.330962+0000","last_active":"2023-10-06T04:49:20.319623+0000","last_peered":"2023-10-06T04:49:20.319623+0000","last_clean":"2023-10-06T04:49:20.319623+0000","last_became_active":"2023-10-06T04:49:19.330616+0000","last_became_peered":"2023-10-06T04:49:19.330616+0000","last_unstale":"2023-10-06T04:49:20.319623+0000","last_undegraded":"2023-10-06T04:49:20.319623+0000","last_fullsized":"2023-10-06T04:49:20.319623+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:04.272287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355580+0000","last_change":"2023-10-06T04:49:20.350356+0000","last_active":"2023-10-06T04:49:20.355580+0000","last_peered":"2023-10-06T04:49:20.355580+0000","last_clean":"2023-10-06T04:49:20.355580+0000","last_became_active":"2023-10-06T04:49:20.350105+0000","last_became_peered":"2023-10-06T04:49:20.350105+0000","last_unstale":"2023-10-06T04:49:20.355580+0000","last_undegraded":"2023-10-06T04:49:20.355580+0000","last_fullsized":"2023-10-06T04:49:20.355580+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:10.637938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346898+0000","last_change":"2023-10-06T04:49:19.333331+0000","last_active":"2023-10-06T04:49:20.346898+0000","last_peered":"2023-10-06T04:49:20.346898+0000","last_clean":"2023-10-06T04:49:20.346898+0000","last_became_active":"2023-10-06T04:49:19.332774+0000","last_became_peered":"2023-10-06T04:49:19.332774+0000","last_unstale":"2023-10-06T04:49:20.346898+0000","last_undegraded":"2023-10-06T04:49:20.346898+0000","last_fullsized":"2023-10-06T04:49:20.346898+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:19.019093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346151+0000","last_change":"2023-10-06T04:49:20.345404+0000","last_active":"2023-10-06T04:49:20.346151+0000","last_peered":"2023-10-06T04:49:20.346151+0000","last_clean":"2023-10-06T04:49:20.346151+0000","last_became_active":"2023-10-06T04:49:20.345129+0000","last_became_peered":"2023-10-06T04:49:20.345129+0000","last_unstale":"2023-10-06T04:49:20.346151+0000","last_undegraded":"2023-10-06T04:49:20.346151+0000","last_fullsized":"2023-10-06T04:49:20.346151+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:25.017277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:49:18.291949+0000","last_change":"2023-10-06T04:49:18.303757+0000","last_active":"2023-10-06T04:49:18.291949+0000","last_peered":"2023-10-06T04:49:18.291949+0000","last_clean":"2023-10-06T04:49:18.291949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:18.291949+0000","last_undegraded":"2023-10-06T04:49:18.291949+0000","last_fullsized":"2023-10-06T04:49:18.291949+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:22.574489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350774+0000","last_change":"2023-10-06T04:49:20.347069+0000","last_active":"2023-10-06T04:49:20.350774+0000","last_peered":"2023-10-06T04:49:20.350774+0000","last_clean":"2023-10-06T04:49:20.350774+0000","last_became_active":"2023-10-06T04:49:20.346801+0000","last_became_peered":"2023-10-06T04:49:20.346801+0000","last_unstale":"2023-10-06T04:49:20.350774+0000","last_undegraded":"2023-10-06T04:49:20.350774+0000","last_fullsized":"2023-10-06T04:49:20.350774+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:34.988176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321187+0000","last_change":"2023-10-06T04:49:19.329229+0000","last_active":"2023-10-06T04:49:20.321187+0000","last_peered":"2023-10-06T04:49:20.321187+0000","last_clean":"2023-10-06T04:49:20.321187+0000","last_became_active":"2023-10-06T04:49:19.328964+0000","last_became_peered":"2023-10-06T04:49:19.328964+0000","last_unstale":"2023-10-06T04:49:20.321187+0000","last_undegraded":"2023-10-06T04:49:20.321187+0000","last_fullsized":"2023-10-06T04:49:20.321187+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:53.870884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341139+0000","last_change":"2023-10-06T04:49:20.334997+0000","last_active":"2023-10-06T04:49:20.341139+0000","last_peered":"2023-10-06T04:49:20.341139+0000","last_clean":"2023-10-06T04:49:20.341139+0000","last_became_active":"2023-10-06T04:49:20.334723+0000","last_became_peered":"2023-10-06T04:49:20.334723+0000","last_unstale":"2023-10-06T04:49:20.341139+0000","last_undegraded":"2023-10-06T04:49:20.341139+0000","last_fullsized":"2023-10-06T04:49:20.341139+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:41.560870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:49:18.291949+0000","last_change":"2023-10-06T04:49:18.303604+0000","last_active":"2023-10-06T04:49:18.291949+0000","last_peered":"2023-10-06T04:49:18.291949+0000","last_clean":"2023-10-06T04:49:18.291949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:18.291949+0000","last_undegraded":"2023-10-06T04:49:18.291949+0000","last_fullsized":"2023-10-06T04:49:18.291949+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:53.028433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339461+0000","last_change":"2023-10-06T04:49:20.337671+0000","last_active":"2023-10-06T04:49:20.339461+0000","last_peered":"2023-10-06T04:49:20.339461+0000","last_clean":"2023-10-06T04:49:20.339461+0000","last_became_active":"2023-10-06T04:49:20.337383+0000","last_became_peered":"2023-10-06T04:49:20.337383+0000","last_unstale":"2023-10-06T04:49:20.339461+0000","last_undegraded":"2023-10-06T04:49:20.339461+0000","last_fullsized":"2023-10-06T04:49:20.339461+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:41.316112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.342104+0000","last_change":"2023-10-06T04:49:19.328915+0000","last_active":"2023-10-06T04:49:20.342104+0000","last_peered":"2023-10-06T04:49:20.342104+0000","last_clean":"2023-10-06T04:49:20.342104+0000","last_became_active":"2023-10-06T04:49:19.328630+0000","last_became_peered":"2023-10-06T04:49:19.328630+0000","last_unstale":"2023-10-06T04:49:20.342104+0000","last_undegraded":"2023-10-06T04:49:20.342104+0000","last_fullsized":"2023-10-06T04:49:20.342104+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:27.949348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.343843+0000","last_change":"2023-10-06T04:49:20.341772+0000","last_active":"2023-10-06T04:49:20.343843+0000","last_peered":"2023-10-06T04:49:20.343843+0000","last_clean":"2023-10-06T04:49:20.343843+0000","last_became_active":"2023-10-06T04:49:20.341489+0000","last_became_peered":"2023-10-06T04:49:20.341489+0000","last_unstale":"2023-10-06T04:49:20.343843+0000","last_undegraded":"2023-10-06T04:49:20.343843+0000","last_fullsized":"2023-10-06T04:49:20.343843+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:10.460143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.330303+0000","last_change":"2023-10-06T04:49:19.331671+0000","last_active":"2023-10-06T04:49:20.330303+0000","last_peered":"2023-10-06T04:49:20.330303+0000","last_clean":"2023-10-06T04:49:20.330303+0000","last_became_active":"2023-10-06T04:49:19.330061+0000","last_became_peered":"2023-10-06T04:49:19.330061+0000","last_unstale":"2023-10-06T04:49:20.330303+0000","last_undegraded":"2023-10-06T04:49:20.330303+0000","last_fullsized":"2023-10-06T04:49:20.330303+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:09.188792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356042+0000","last_change":"2023-10-06T04:49:20.352177+0000","last_active":"2023-10-06T04:49:20.356042+0000","last_peered":"2023-10-06T04:49:20.356042+0000","last_clean":"2023-10-06T04:49:20.356042+0000","last_became_active":"2023-10-06T04:49:20.351885+0000","last_became_peered":"2023-10-06T04:49:20.351885+0000","last_unstale":"2023-10-06T04:49:20.356042+0000","last_undegraded":"2023-10-06T04:49:20.356042+0000","last_fullsized":"2023-10-06T04:49:20.356042+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320737+0000","last_change":"2023-10-06T04:49:19.329132+0000","last_active":"2023-10-06T04:49:20.320737+0000","last_peered":"2023-10-06T04:49:20.320737+0000","last_clean":"2023-10-06T04:49:20.320737+0000","last_became_active":"2023-10-06T04:49:19.328841+0000","last_became_peered":"2023-10-06T04:49:19.328841+0000","last_unstale":"2023-10-06T04:49:20.320737+0000","last_undegraded":"2023-10-06T04:49:20.320737+0000","last_fullsized":"2023-10-06T04:49:20.320737+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:54.415706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344647+0000","last_change":"2023-10-06T04:49:20.343226+0000","last_active":"2023-10-06T04:49:20.344647+0000","last_peered":"2023-10-06T04:49:20.344647+0000","last_clean":"2023-10-06T04:49:20.344647+0000","last_became_active":"2023-10-06T04:49:20.342932+0000","last_became_peered":"2023-10-06T04:49:20.342932+0000","last_unstale":"2023-10-06T04:49:20.344647+0000","last_undegraded":"2023-10-06T04:49:20.344647+0000","last_fullsized":"2023-10-06T04:49:20.344647+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:28.172716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.316079+0000","last_change":"2023-10-06T04:49:19.328230+0000","last_active":"2023-10-06T04:49:20.316079+0000","last_peered":"2023-10-06T04:49:20.316079+0000","last_clean":"2023-10-06T04:49:20.316079+0000","last_became_active":"2023-10-06T04:49:19.327917+0000","last_became_peered":"2023-10-06T04:49:19.327917+0000","last_unstale":"2023-10-06T04:49:20.316079+0000","last_undegraded":"2023-10-06T04:49:20.316079+0000","last_fullsized":"2023-10-06T04:49:20.316079+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.900580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.325259+0000","last_change":"2023-10-06T04:49:19.331624+0000","last_active":"2023-10-06T04:49:20.325259+0000","last_peered":"2023-10-06T04:49:20.325259+0000","last_clean":"2023-10-06T04:49:20.325259+0000","last_became_active":"2023-10-06T04:49:19.331287+0000","last_became_peered":"2023-10-06T04:49:19.331287+0000","last_unstale":"2023-10-06T04:49:20.325259+0000","last_undegraded":"2023-10-06T04:49:20.325259+0000","last_fullsized":"2023-10-06T04:49:20.325259+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.288607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341412+0000","last_change":"2023-10-06T04:49:20.334336+0000","last_active":"2023-10-06T04:49:20.341412+0000","last_peered":"2023-10-06T04:49:20.341412+0000","last_clean":"2023-10-06T04:49:20.341412+0000","last_became_active":"2023-10-06T04:49:20.334097+0000","last_became_peered":"2023-10-06T04:49:20.334097+0000","last_unstale":"2023-10-06T04:49:20.341412+0000","last_undegraded":"2023-10-06T04:49:20.341412+0000","last_fullsized":"2023-10-06T04:49:20.341412+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:35.183960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320080+0000","last_change":"2023-10-06T04:49:19.332817+0000","last_active":"2023-10-06T04:49:20.320080+0000","last_peered":"2023-10-06T04:49:20.320080+0000","last_clean":"2023-10-06T04:49:20.320080+0000","last_became_active":"2023-10-06T04:49:19.331466+0000","last_became_peered":"2023-10-06T04:49:19.331466+0000","last_unstale":"2023-10-06T04:49:20.320080+0000","last_undegraded":"2023-10-06T04:49:20.320080+0000","last_fullsized":"2023-10-06T04:49:20.320080+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:01.585470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332830+0000","last_change":"2023-10-06T04:49:20.327613+0000","last_active":"2023-10-06T04:49:20.332830+0000","last_peered":"2023-10-06T04:49:20.332830+0000","last_clean":"2023-10-06T04:49:20.332830+0000","last_became_active":"2023-10-06T04:49:20.327331+0000","last_became_peered":"2023-10-06T04:49:20.327331+0000","last_unstale":"2023-10-06T04:49:20.332830+0000","last_undegraded":"2023-10-06T04:49:20.332830+0000","last_fullsized":"2023-10-06T04:49:20.332830+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:21.888629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339559+0000","last_change":"2023-10-06T04:49:19.327134+0000","last_active":"2023-10-06T04:49:20.339559+0000","last_peered":"2023-10-06T04:49:20.339559+0000","last_clean":"2023-10-06T04:49:20.339559+0000","last_became_active":"2023-10-06T04:49:19.326884+0000","last_became_peered":"2023-10-06T04:49:19.326884+0000","last_unstale":"2023-10-06T04:49:20.339559+0000","last_undegraded":"2023-10-06T04:49:20.339559+0000","last_fullsized":"2023-10-06T04:49:20.339559+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:40.308269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331074+0000","last_change":"2023-10-06T04:49:19.336399+0000","last_active":"2023-10-06T04:49:20.331074+0000","last_peered":"2023-10-06T04:49:20.331074+0000","last_clean":"2023-10-06T04:49:20.331074+0000","last_became_active":"2023-10-06T04:49:19.336144+0000","last_became_peered":"2023-10-06T04:49:19.336144+0000","last_unstale":"2023-10-06T04:49:20.331074+0000","last_undegraded":"2023-10-06T04:49:20.331074+0000","last_fullsized":"2023-10-06T04:49:20.331074+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:48.611176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333465+0000","last_change":"2023-10-06T04:49:20.325062+0000","last_active":"2023-10-06T04:49:20.333465+0000","last_peered":"2023-10-06T04:49:20.333465+0000","last_clean":"2023-10-06T04:49:20.333465+0000","last_became_active":"2023-10-06T04:49:20.324091+0000","last_became_peered":"2023-10-06T04:49:20.324091+0000","last_unstale":"2023-10-06T04:49:20.333465+0000","last_undegraded":"2023-10-06T04:49:20.333465+0000","last_fullsized":"2023-10-06T04:49:20.333465+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:28.109738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321039+0000","last_change":"2023-10-06T04:49:19.327314+0000","last_active":"2023-10-06T04:49:20.321039+0000","last_peered":"2023-10-06T04:49:20.321039+0000","last_clean":"2023-10-06T04:49:20.321039+0000","last_became_active":"2023-10-06T04:49:19.326851+0000","last_became_peered":"2023-10-06T04:49:19.326851+0000","last_unstale":"2023-10-06T04:49:20.321039+0000","last_undegraded":"2023-10-06T04:49:20.321039+0000","last_fullsized":"2023-10-06T04:49:20.321039+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:26.547655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":153,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-10-06T04:49:15.303291+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T04:49:15.303291+0000","last_peered":"2023-10-06T04:49:15.303291+0000","last_clean":"2023-10-06T04:49:15.303291+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T04:49:15.303291+0000","last_undegraded":"2023-10-06T04:49:15.303291+0000","last_fullsized":"2023-10-06T04:49:15.303291+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332841+0000","last_change":"2023-10-06T04:49:20.327658+0000","last_active":"2023-10-06T04:49:20.332841+0000","last_peered":"2023-10-06T04:49:20.332841+0000","last_clean":"2023-10-06T04:49:20.332841+0000","last_became_active":"2023-10-06T04:49:20.327377+0000","last_became_peered":"2023-10-06T04:49:20.327377+0000","last_unstale":"2023-10-06T04:49:20.332841+0000","last_undegraded":"2023-10-06T04:49:20.332841+0000","last_fullsized":"2023-10-06T04:49:20.332841+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"172'8","reported_seq":18,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.351338+0000","last_change":"2023-10-06T04:49:19.329982+0000","last_active":"2023-10-06T04:49:20.351338+0000","last_peered":"2023-10-06T04:49:20.351338+0000","last_clean":"2023-10-06T04:49:20.351338+0000","last_became_active":"2023-10-06T04:49:19.329677+0000","last_became_peered":"2023-10-06T04:49:19.329677+0000","last_unstale":"2023-10-06T04:49:20.351338+0000","last_undegraded":"2023-10-06T04:49:20.351338+0000","last_fullsized":"2023-10-06T04:49:20.351338+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":143,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.316165+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T04:49:20.316165+0000","last_peered":"2023-10-06T04:49:20.316165+0000","last_clean":"2023-10-06T04:49:20.316165+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T04:49:20.316165+0000","last_undegraded":"2023-10-06T04:49:20.316165+0000","last_fullsized":"2023-10-06T04:49:20.316165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.330157+0000","last_change":"2023-10-06T04:49:19.330514+0000","last_active":"2023-10-06T04:49:20.330157+0000","last_peered":"2023-10-06T04:49:20.330157+0000","last_clean":"2023-10-06T04:49:20.330157+0000","last_became_active":"2023-10-06T04:49:19.330226+0000","last_became_peered":"2023-10-06T04:49:19.330226+0000","last_unstale":"2023-10-06T04:49:20.330157+0000","last_undegraded":"2023-10-06T04:49:20.330157+0000","last_fullsized":"2023-10-06T04:49:20.330157+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:55.142830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":144,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.319996+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T04:49:20.319996+0000","last_peered":"2023-10-06T04:49:20.319996+0000","last_clean":"2023-10-06T04:49:20.319996+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T04:49:20.319996+0000","last_undegraded":"2023-10-06T04:49:20.319996+0000","last_fullsized":"2023-10-06T04:49:20.319996+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"163'293","reported_seq":919,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331027+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T04:49:20.331027+0000","last_peered":"2023-10-06T04:49:20.331027+0000","last_clean":"2023-10-06T04:49:20.331027+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T04:49:20.331027+0000","last_undegraded":"2023-10-06T04:49:20.331027+0000","last_fullsized":"2023-10-06T04:49:20.331027+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.336779+0000","last_change":"2023-10-06T04:49:20.335358+0000","last_active":"2023-10-06T04:49:20.336779+0000","last_peered":"2023-10-06T04:49:20.336779+0000","last_clean":"2023-10-06T04:49:20.336779+0000","last_became_active":"2023-10-06T04:49:20.335073+0000","last_became_peered":"2023-10-06T04:49:20.335073+0000","last_unstale":"2023-10-06T04:49:20.336779+0000","last_undegraded":"2023-10-06T04:49:20.336779+0000","last_fullsized":"2023-10-06T04:49:20.336779+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:05.280129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.330408+0000","last_change":"2023-10-06T04:49:19.331617+0000","last_active":"2023-10-06T04:49:20.330408+0000","last_peered":"2023-10-06T04:49:20.330408+0000","last_clean":"2023-10-06T04:49:20.330408+0000","last_became_active":"2023-10-06T04:49:19.329690+0000","last_became_peered":"2023-10-06T04:49:19.329690+0000","last_unstale":"2023-10-06T04:49:20.330408+0000","last_undegraded":"2023-10-06T04:49:20.330408+0000","last_fullsized":"2023-10-06T04:49:20.330408+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:04.983797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":146,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.325305+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T04:49:20.325305+0000","last_peered":"2023-10-06T04:49:20.325305+0000","last_clean":"2023-10-06T04:49:20.325305+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T04:49:20.325305+0000","last_undegraded":"2023-10-06T04:49:20.325305+0000","last_fullsized":"2023-10-06T04:49:20.325305+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"172'6","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356034+0000","last_change":"2023-10-06T04:49:19.331731+0000","last_active":"2023-10-06T04:49:20.356034+0000","last_peered":"2023-10-06T04:49:20.356034+0000","last_clean":"2023-10-06T04:49:20.356034+0000","last_became_active":"2023-10-06T04:49:19.331458+0000","last_became_peered":"2023-10-06T04:49:19.331458+0000","last_unstale":"2023-10-06T04:49:20.356034+0000","last_undegraded":"2023-10-06T04:49:20.356034+0000","last_fullsized":"2023-10-06T04:49:20.356034+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:55.142830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":146,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331105+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T04:49:20.331105+0000","last_peered":"2023-10-06T04:49:20.331105+0000","last_clean":"2023-10-06T04:49:20.331105+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T04:49:20.331105+0000","last_undegraded":"2023-10-06T04:49:20.331105+0000","last_fullsized":"2023-10-06T04:49:20.331105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347917+0000","last_change":"2023-10-06T04:49:20.345754+0000","last_active":"2023-10-06T04:49:20.347917+0000","last_peered":"2023-10-06T04:49:20.347917+0000","last_clean":"2023-10-06T04:49:20.347917+0000","last_became_active":"2023-10-06T04:49:20.345446+0000","last_became_peered":"2023-10-06T04:49:20.345446+0000","last_unstale":"2023-10-06T04:49:20.347917+0000","last_undegraded":"2023-10-06T04:49:20.347917+0000","last_fullsized":"2023-10-06T04:49:20.347917+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:00.217906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"172'3","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.342594+0000","last_change":"2023-10-06T04:49:19.327118+0000","last_active":"2023-10-06T04:49:20.342594+0000","last_peered":"2023-10-06T04:49:20.342594+0000","last_clean":"2023-10-06T04:49:20.342594+0000","last_became_active":"2023-10-06T04:49:19.326839+0000","last_became_peered":"2023-10-06T04:49:19.326839+0000","last_unstale":"2023-10-06T04:49:20.342594+0000","last_undegraded":"2023-10-06T04:49:20.342594+0000","last_fullsized":"2023-10-06T04:49:20.342594+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:17.629867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":139,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.335923+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T04:49:20.335923+0000","last_peered":"2023-10-06T04:49:20.335923+0000","last_clean":"2023-10-06T04:49:20.335923+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T04:49:20.335923+0000","last_undegraded":"2023-10-06T04:49:20.335923+0000","last_fullsized":"2023-10-06T04:49:20.335923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341195+0000","last_change":"2023-10-06T04:49:20.338084+0000","last_active":"2023-10-06T04:49:20.341195+0000","last_peered":"2023-10-06T04:49:20.341195+0000","last_clean":"2023-10-06T04:49:20.341195+0000","last_became_active":"2023-10-06T04:49:20.337738+0000","last_became_peered":"2023-10-06T04:49:20.337738+0000","last_unstale":"2023-10-06T04:49:20.341195+0000","last_undegraded":"2023-10-06T04:49:20.341195+0000","last_fullsized":"2023-10-06T04:49:20.341195+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:28.172716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321149+0000","last_change":"2023-10-06T04:49:19.332809+0000","last_active":"2023-10-06T04:49:20.321149+0000","last_peered":"2023-10-06T04:49:20.321149+0000","last_clean":"2023-10-06T04:49:20.321149+0000","last_became_active":"2023-10-06T04:49:19.331892+0000","last_became_peered":"2023-10-06T04:49:19.331892+0000","last_unstale":"2023-10-06T04:49:20.321149+0000","last_undegraded":"2023-10-06T04:49:20.321149+0000","last_fullsized":"2023-10-06T04:49:20.321149+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:52.469578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320842+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T04:49:20.320842+0000","last_peered":"2023-10-06T04:49:20.320842+0000","last_clean":"2023-10-06T04:49:20.320842+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T04:49:20.320842+0000","last_undegraded":"2023-10-06T04:49:20.320842+0000","last_fullsized":"2023-10-06T04:49:20.320842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344451+0000","last_change":"2023-10-06T04:49:19.321982+0000","last_active":"2023-10-06T04:49:20.344451+0000","last_peered":"2023-10-06T04:49:20.344451+0000","last_clean":"2023-10-06T04:49:20.344451+0000","last_became_active":"2023-10-06T04:49:19.321679+0000","last_became_peered":"2023-10-06T04:49:19.321679+0000","last_unstale":"2023-10-06T04:49:20.344451+0000","last_undegraded":"2023-10-06T04:49:20.344451+0000","last_fullsized":"2023-10-06T04:49:20.344451+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320746+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T04:49:20.320746+0000","last_peered":"2023-10-06T04:49:20.320746+0000","last_clean":"2023-10-06T04:49:20.320746+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T04:49:20.320746+0000","last_undegraded":"2023-10-06T04:49:20.320746+0000","last_fullsized":"2023-10-06T04:49:20.320746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575752,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299976,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071864,"statfs":{"total":96636764160,"available":96329588736,"internally_reserved":0,"allocated":2752512,"data_stored":1720883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69799999999999995,"15min":0.73899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.89500000000000002,"5min":0.97199999999999998,"15min":1.4930000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.48599999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90900000000000003,"5min":0.995,"15min":1.417},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.755,"15min":0.79500000000000004},"min":{"1min":0.53500000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93100000000000005,"5min":1.01,"15min":1.5309999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77600000000000002,"15min":0.79300000000000004},"min":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.58299999999999996},"max":{"1min":0.93600000000000005,"5min":1.0289999999999999,"15min":1.097},"last":0.81000000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70899999999999996,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.46400000000000002},"max":{"1min":0.92300000000000004,"5min":0.97999999999999998,"15min":1.5429999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.751,"15min":0.77100000000000002},"min":{"1min":0.60199999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0189999999999999},"last":0.82999999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.75700000000000001,"15min":0.77500000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.91000000000000003,"5min":1.03,"15min":1.5860000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73299999999999998,"15min":0.76400000000000001},"min":{"1min":0.64200000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.96099999999999997}]},{"osd":4,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63400000000000001,"15min":0.61899999999999999},"min":{"1min":0.50800000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.0640000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.439,"15min":0.42799999999999999},"max":{"1min":1.1339999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.90400000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.53800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.55200000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.72999999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.64200000000000002,"5min":0.501,"15min":0.49199999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":1.1020000000000001,"15min":1.2090000000000001},"last":0.93899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575753,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299400,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072440,"statfs":{"total":96636764160,"available":96330178560,"internally_reserved":0,"allocated":2162688,"data_stored":1131541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.46000000000000002,"5min":0.40300000000000002,"15min":0.375},"max":{"1min":0.96599999999999997,"5min":1.038,"15min":1.038},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76200000000000001,"15min":0.752},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94699999999999995,"5min":0.999,"15min":1.228},"last":0.80000000000000004}]},{"osd":1,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.5,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93799999999999994,"5min":1.046,"15min":1.3660000000000001},"last":0.63},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.748,"15min":0.751},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.438},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.3560000000000001},"last":0.76500000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.47999999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.272},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76900000000000002,"15min":0.77000000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98799999999999999,"5min":1.069,"15min":1.089},"last":0.66300000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94399999999999995,"5min":1.042,"15min":1.1060000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.52000000000000002},"max":{"1min":1.0009999999999999,"5min":1.0409999999999999,"15min":1.3120000000000001},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83499999999999996,"5min":0.88500000000000001,"15min":0.92600000000000005},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.627},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":1.155},"last":0.84499999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.432},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93799999999999994,"5min":1.056,"15min":1.056},"last":0.82799999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95399999999999996,"5min":1.0580000000000001,"15min":1.3500000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.45800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98599999999999999,"5min":1.024,"15min":1.0389999999999999},"last":0.78300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575752,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312268,"kb_used_data":2692,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059572,"statfs":{"total":96636764160,"available":96317001728,"internally_reserved":0,"allocated":2756608,"data_stored":1721909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.79600000000000004,"5min":0.89000000000000001,"15min":0.93100000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42899999999999999},"max":{"1min":0.90000000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.41399999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63800000000000001,"5min":0.67700000000000005,"15min":0.73299999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.38200000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.97799999999999998},"last":0.46899999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.44,"5min":0.44,"15min":0.437},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.94699999999999995},"last":0.497},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.373},"max":{"1min":0.748,"5min":0.85799999999999998,"15min":0.88900000000000001},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79200000000000004,"15min":0.79500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.46899999999999997},"max":{"1min":1.159,"5min":1.1970000000000001,"15min":1.407},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82599999999999996,"15min":0.83099999999999996},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.57499999999999996},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.84399999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83699999999999997,"15min":0.82699999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.55700000000000005},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81699999999999995,"15min":0.83899999999999997},"min":{"1min":0.66300000000000003,"5min":0.59799999999999998,"15min":0.53200000000000003},"max":{"1min":1.097,"5min":1.1060000000000001,"15min":1.724},"last":0.88300000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.64700000000000002,"5min":0.626,"15min":0.56200000000000006},"max":{"1min":0.97899999999999998,"5min":1.2150000000000001,"15min":1.427},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.55800000000000005},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.93600000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.877,"15min":0.878},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1140000000000001,"5min":1.125,"15min":1.702},"last":0.874},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85799999999999998,"15min":0.872},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0269999999999999,"5min":1.1659999999999999,"15min":1.5920000000000001},"last":0.72199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575754,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299416,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072424,"statfs":{"total":96636764160,"available":96330162176,"internally_reserved":0,"allocated":2179072,"data_stored":1132707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.378,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76600000000000001,"5min":0.81699999999999995,"15min":1.6459999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.34200000000000003,"15min":0.32900000000000001},"max":{"1min":0.68300000000000005,"5min":0.84099999999999997,"15min":1.4510000000000001},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55400000000000005,"15min":0.57099999999999995},"min":{"1min":0.434,"5min":0.308,"15min":0.308},"max":{"1min":0.74099999999999999,"5min":0.86899999999999999,"15min":1.7},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.373},"max":{"1min":0.81699999999999995,"5min":0.82099999999999995,"15min":1.6240000000000001},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":1.2529999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91300000000000003,"5min":0.99099999999999999,"15min":1.7370000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.86599999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.60399999999999998},"max":{"1min":1.0940000000000001,"5min":1.2769999999999999,"15min":2.3700000000000001},"last":0.873},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.85099999999999998,"15min":0.85999999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.52900000000000003},"max":{"1min":1.111,"5min":1.3069999999999999,"15min":2.1280000000000001},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84099999999999997,"15min":0.85599999999999998},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.57599999999999996},"max":{"1min":1.048,"5min":1.294,"15min":2.1739999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84099999999999997,"15min":0.84799999999999998},"min":{"1min":0.66000000000000003,"5min":0.63300000000000001,"15min":0.57799999999999996},"max":{"1min":1.087,"5min":1.214,"15min":2.2429999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60599999999999998,"5min":0.59299999999999997,"15min":0.53000000000000003},"max":{"1min":1.1160000000000001,"5min":1.204,"15min":1.605},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89600000000000002,"15min":0.88700000000000001},"min":{"1min":0.72899999999999998,"5min":0.65800000000000003,"15min":0.58199999999999996},"max":{"1min":1.181,"5min":1.226,"15min":2.3519999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.86699999999999999,"15min":0.878},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.091,"5min":1.319,"15min":2.1019999999999999},"last":0.746},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.86299999999999999,"15min":0.86799999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0580000000000001,"5min":1.256,"15min":1.9259999999999999},"last":0.79500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575753,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311688,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060152,"statfs":{"total":96636764160,"available":96317595648,"internally_reserved":0,"allocated":2162688,"data_stored":1130605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.311},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.376},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.34200000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.93999999999999995},"last":0.505}]},{"osd":1,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.32000000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.42399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.38300000000000001},"max":{"1min":0.79600000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.371},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.45400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.628,"5min":0.60199999999999998,"15min":0.51700000000000002},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.86399999999999999,"15min":0.88100000000000001},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58599999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.381},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.59199999999999997},"max":{"1min":1.1719999999999999,"5min":1.2210000000000001,"15min":1.4379999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88300000000000001,"15min":0.89200000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.58899999999999997},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.288},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.151,"5min":1.167,"15min":1.2370000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.61499999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86099999999999999,"15min":0.874},"min":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.57499999999999996},"max":{"1min":1.075,"5min":1.3320000000000001,"15min":1.482},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.88500000000000001,"15min":0.89600000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.135,"5min":1.2869999999999999,"15min":1.405},"last":0.755}]}]},{"osd":3,"up_from":13,"seq":55834575753,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311692,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060148,"statfs":{"total":96636764160,"available":96317591552,"internally_reserved":0,"allocated":2166784,"data_stored":1132059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47699999999999998,"15min":0.49399999999999999},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67900000000000005,"5min":0.73899999999999999,"15min":0.86399999999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.83599999999999997},"last":0.501},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.44900000000000001,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.73999999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.40899999999999997},"max":{"1min":0.69699999999999995,"5min":0.79200000000000004,"15min":1.036},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.54000000000000004,"5min":0.47499999999999998,"15min":0.43099999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83799999999999997},"min":{"1min":0.64300000000000002,"5min":0.56000000000000005,"15min":0.47599999999999998},"max":{"1min":1.101,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.495},"max":{"1min":1.0760000000000001,"5min":1.387,"15min":1.399},"last":1.034}]},{"osd":5,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.82599999999999996,"15min":0.82899999999999996},"min":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0049999999999999,"5min":1.2509999999999999,"15min":1.331},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.85799999999999998,"15min":0.85099999999999998},"min":{"1min":0.63600000000000001,"5min":0.58099999999999996,"15min":0.56599999999999995},"max":{"1min":1.0720000000000001,"5min":1.2130000000000001,"15min":1.2330000000000001},"last":1.0549999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.86499999999999999,"15min":0.85999999999999999},"min":{"1min":0.65100000000000002,"5min":0.53500000000000003,"15min":0.52400000000000002},"max":{"1min":1.175,"5min":1.2090000000000001,"15min":1.2490000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.89900000000000002,"15min":0.89400000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1579999999999999,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88600000000000001,"15min":0.88400000000000001},"min":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.58999999999999997},"max":{"1min":1.127,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85399999999999998,"15min":0.85799999999999998},"min":{"1min":0.60499999999999998,"5min":0.58699999999999997,"15min":0.54900000000000004},"max":{"1min":1.069,"5min":1.276,"15min":1.3029999999999999},"last":1.097}]}]},{"osd":4,"up_from":13,"seq":55834575752,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311636,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060204,"statfs":{"total":96636764160,"available":96317648896,"internally_reserved":0,"allocated":2109440,"data_stored":1103372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.71699999999999997,"15min":0.74299999999999999},"min":{"1min":0.47899999999999998,"5min":0.36699999999999999,"15min":0.35599999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.72399999999999998,"15min":0.73499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.76000000000000001,"15min":0.77800000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.72099999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.745,"15min":0.75900000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.76600000000000001,"15min":0.77100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91200000000000003,"5min":1.01,"15min":1.069},"last":0.76500000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.996,"5min":1.002,"15min":1.002},"last":0.90700000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81699999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.871},{"interface":"front","average":{"1min":0.624,"5min":0.65000000000000002,"15min":0.63500000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69099999999999995,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.46000000000000002,"15min":0.42099999999999999},"max":{"1min":0.996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71499999999999997,"15min":0.70199999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0629999999999999,"5min":1.105,"15min":1.105},"last":1.014}]},{"osd":7,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71399999999999997,"15min":0.68799999999999994},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97999999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.75600000000000001,"15min":0.745},"min":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89700000000000002,"5min":1.091,"15min":1.0940000000000001},"last":1.079}]}]},{"osd":5,"up_from":13,"seq":55834575753,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307584,"kb_used_data":2104,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064256,"statfs":{"total":96636764160,"available":96321798144,"internally_reserved":0,"allocated":2154496,"data_stored":1130493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82999999999999996,"5min":1.03,"15min":2.2829999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73599999999999999,"15min":0.751},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91600000000000004,"5min":1.095,"15min":1.3200000000000001},"last":0.67600000000000005}]},{"osd":1,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.751},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.29},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.51400000000000001},"max":{"1min":0.871,"5min":0.98099999999999998,"15min":1.9139999999999999},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.57999999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98199999999999998,"5min":1.0589999999999999,"15min":1.532},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95799999999999996,"5min":1.0169999999999999,"15min":1.121},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.58899999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.40500000000000003,"15min":0.39300000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":2.6989999999999998},"last":0.75},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38},"max":{"1min":0.99399999999999999,"5min":1.028,"15min":1.694},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.1719999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96399999999999997,"5min":1.077,"15min":1.5880000000000001},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.52500000000000002,"5min":0.47499999999999998,"15min":0.40699999999999997},"max":{"1min":1.03,"5min":1.0409999999999999,"15min":1.7390000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.49099999999999999,"15min":0.48599999999999999},"max":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.3380000000000001},"last":0.75800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:49:24.710 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:49:24.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.925+0000 7f0e63fff700 1 Processor -- start 2023-10-06T04:49:24.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.925+0000 7f0e63fff700 1 -- start start 2023-10-06T04:49:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.926+0000 7f0e63fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e64150550 0x7f0e64157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.927+0000 7f0e63fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6406e2a0 con 0x7f0e64150180 2023-10-06T04:49:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.927+0000 7f0e63fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6406b2e0 con 0x7f0e640c7410 2023-10-06T04:49:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.927+0000 7f0e63fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640c78e0 con 0x7f0e64150550 2023-10-06T04:49:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.927+0000 7f0e62ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0e64150180 0x7f0e640c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46722/0 (socket says 172.21.15.55:46722) 2023-10-06T04:49:24.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.927+0000 7f0e62ffd700 1 -- 172.21.15.55:0/367437465 learned_addr learned my addr 172.21.15.55:0/367437465 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:24.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.927+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321238190 0 0) 0x7f0e6406e2a0 con 0x7f0e64150180 2023-10-06T04:49:24.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.927+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e4c003570 con 0x7f0e64150180 2023-10-06T04:49:24.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.927+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056626755 0 0) 0x7f0e6406b2e0 con 0x7f0e640c7410 2023-10-06T04:49:24.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.927+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e6406e2a0 con 0x7f0e640c7410 2023-10-06T04:49:24.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3789261574 0 0) 0x7f0e4c003570 con 0x7f0e64150180 2023-10-06T04:49:24.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e64150550 msgr2=0x7f0e64157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:24.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e627fc700 1 --2- 172.21.15.55:0/367437465 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e64150550 0x7f0e64157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e61ffb700 1 --2- 172.21.15.55:0/367437465 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e64150550 0x7f0e64157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 >> v1:172.21.15.163:6789/0 conn(0x7f0e640c7410 legacy=0x7f0e64069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e640c7bc0 con 0x7f0e64150180 2023-10-06T04:49:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f0e54001470 con 0x7f0e64150180 2023-10-06T04:49:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e627fc700 1 --2- 172.21.15.55:0/367437465 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e64150550 0x7f0e64157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:49:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 >> v1:172.21.15.55:6789/0 conn(0x7f0e64150180 legacy=0x7f0e640c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e61ffb700 1 --2- 172.21.15.55:0/367437465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e4c003c20 0x7f0e4c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e61ffb700 1 --2- 172.21.15.55:0/367437465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e4c004810 0x7f0e4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e61ffb700 1 --2- 172.21.15.55:0/367437465 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e64150550 0x7f0e4c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640c78e0 con 0x7f0e4c004810 2023-10-06T04:49:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c0055c0 con 0x7f0e4c003c20 2023-10-06T04:49:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c00bfa0 con 0x7f0e64150550 2023-10-06T04:49:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e627fc700 1 --2- 172.21.15.55:0/367437465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e4c004810 0x7f0e4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.928+0000 7f0e637fe700 1 --2- 172.21.15.55:0/367437465 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e64150550 0x7f0e4c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.929+0000 7f0e627fc700 1 -- 172.21.15.55:0/367437465 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e64150550 msgr2=0x7f0e4c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.929+0000 7f0e627fc700 1 --2- 172.21.15.55:0/367437465 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e64150550 0x7f0e4c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.929+0000 7f0e627fc700 1 -- 172.21.15.55:0/367437465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e4c003c20 msgr2=0x7f0e4c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.929+0000 7f0e627fc700 1 --2- 172.21.15.55:0/367437465 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e4c003c20 0x7f0e4c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.929+0000 7f0e627fc700 1 -- 172.21.15.55:0/367437465 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c003570 con 0x7f0e4c004810 2023-10-06T04:49:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.929+0000 7f0e627fc700 1 --2- 172.21.15.55:0/367437465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e4c004810 0x7f0e4c004c80 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800abf0 tx=0x7f0e58008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3873f86fa9e2d8c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.929+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800a530 con 0x7f0e4c004810 2023-10-06T04:49:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.929+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5800c670 con 0x7f0e4c004810 2023-10-06T04:49:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.929+0000 7f0e61ffb700 1 -- 172.21.15.55:0/367437465 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800ca10 con 0x7f0e4c004810 2023-10-06T04:49:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.930+0000 7f0e63fff700 1 -- 172.21.15.55:0/367437465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e4c004810 msgr2=0x7f0e4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.930+0000 7f0e63fff700 1 --2- 172.21.15.55:0/367437465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e4c004810 0x7f0e4c004c80 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800abf0 tx=0x7f0e58008040 comp rx=0 tx=0).stop 2023-10-06T04:49:24.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.930+0000 7f0e637fe700 1 -- 172.21.15.55:0/367437465 reap_dead start 2023-10-06T04:49:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.930+0000 7f0e63fff700 1 -- 172.21.15.55:0/367437465 shutdown_connections 2023-10-06T04:49:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.930+0000 7f0e63fff700 1 -- 172.21.15.55:0/367437465 >> 172.21.15.55:0/367437465 conn(0x7f0e640cc1f0 msgr2=0x7f0e640cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.930+0000 7f0e63fff700 1 -- 172.21.15.55:0/367437465 shutdown_connections 2023-10-06T04:49:24.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.930+0000 7f0e63fff700 1 -- 172.21.15.55:0/367437465 wait complete. 2023-10-06T04:49:24.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.931+0000 7f0e63fff700 1 Processor -- start 2023-10-06T04:49:24.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.931+0000 7f0e63fff700 1 -- start start 2023-10-06T04:49:24.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.932+0000 7f0e63fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e4c004810 0x7f0e640c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.932+0000 7f0e63fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e640cbd60 0x7f0e640ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.932+0000 7f0e63fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e64168ce0 0x7f0e64169160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.932+0000 7f0e63fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e641696a0 con 0x7f0e4c004810 2023-10-06T04:49:24.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.932+0000 7f0e63fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640c4f70 con 0x7f0e640cbd60 2023-10-06T04:49:24.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.932+0000 7f0e63fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e640c5250 con 0x7f0e64168ce0 2023-10-06T04:49:24.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.933+0000 7f0e62ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e4c004810 0x7f0e640c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.933+0000 7f0e62ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e4c004810 0x7f0e640c3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34470/0 (socket says 172.21.15.55:34470) 2023-10-06T04:49:24.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.933+0000 7f0e62ffd700 1 -- 172.21.15.55:0/3471709260 learned_addr learned my addr 172.21.15.55:0/3471709260 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:24.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.933+0000 7f0e627fc700 1 --2- 172.21.15.55:0/3471709260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e640cbd60 0x7f0e640ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.933+0000 7f0e637fe700 1 --2- 172.21.15.55:0/3471709260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e64168ce0 0x7f0e64169160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.933+0000 7f0e62ffd700 1 -- 172.21.15.55:0/3471709260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e64168ce0 msgr2=0x7f0e64169160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.933+0000 7f0e62ffd700 1 --2- 172.21.15.55:0/3471709260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e64168ce0 0x7f0e64169160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.933+0000 7f0e62ffd700 1 -- 172.21.15.55:0/3471709260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e640cbd60 msgr2=0x7f0e640ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.933+0000 7f0e62ffd700 1 --2- 172.21.15.55:0/3471709260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e640cbd60 0x7f0e640ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.933+0000 7f0e62ffd700 1 -- 172.21.15.55:0/3471709260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e640c5530 con 0x7f0e4c004810 2023-10-06T04:49:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.933+0000 7f0e62ffd700 1 --2- 172.21.15.55:0/3471709260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e4c004810 0x7f0e640c3cb0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c710 tx=0x7f0e540019d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.934+0000 7f0e4affd700 1 -- 172.21.15.55:0/3471709260 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54007360 con 0x7f0e4c004810 2023-10-06T04:49:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.934+0000 7f0e4affd700 1 -- 172.21.15.55:0/3471709260 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e540074c0 con 0x7f0e4c004810 2023-10-06T04:49:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.934+0000 7f0e4affd700 1 -- 172.21.15.55:0/3471709260 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54004830 con 0x7f0e4c004810 2023-10-06T04:49:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.934+0000 7f0e63fff700 1 -- 172.21.15.55:0/3471709260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e640aef90 con 0x7f0e4c004810 2023-10-06T04:49:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.934+0000 7f0e63fff700 1 -- 172.21.15.55:0/3471709260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e640c41f0 con 0x7f0e4c004810 2023-10-06T04:49:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.935+0000 7f0e63fff700 1 -- 172.21.15.55:0/3471709260 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6404c3c0 con 0x7f0e4c004810 2023-10-06T04:49:24.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.936+0000 7f0e4affd700 1 -- 172.21.15.55:0/3471709260 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e54004c60 con 0x7f0e4c004810 2023-10-06T04:49:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.937+0000 7f0e4affd700 1 --2- 172.21.15.55:0/3471709260 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0e50066f30 0x7f0e500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.937+0000 7f0e4affd700 1 -- 172.21.15.55:0/3471709260 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f0e540846d0 con 0x7f0e4c004810 2023-10-06T04:49:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.937+0000 7f0e627fc700 1 --2- 172.21.15.55:0/3471709260 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0e50066f30 0x7f0e500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:24.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.938+0000 7f0e627fc700 1 --2- 172.21.15.55:0/3471709260 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0e50066f30 0x7f0e500693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800abf0 tx=0x7f0e5801a000 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:24.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:24.940+0000 7f0e4affd700 1 -- 172.21.15.55:0/3471709260 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e54051630 con 0x7f0e4c004810 2023-10-06T04:49:25.139 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.137+0000 7f0e63fff700 1 -- 172.21.15.55:0/3471709260 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e6406ada0 con 0x7f0e50066f30 2023-10-06T04:49:25.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.147+0000 7f0e4affd700 1 -- 172.21.15.55:0/3471709260 <== mgr.8557 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+309845 (secure 0 0 0) 0x7f0e6406ada0 con 0x7f0e50066f30 2023-10-06T04:49:25.153 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:25.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.152+0000 7f0e63fff700 1 -- 172.21.15.55:0/3471709260 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0e50066f30 msgr2=0x7f0e500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:25.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.152+0000 7f0e63fff700 1 --2- 172.21.15.55:0/3471709260 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0e50066f30 0x7f0e500693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800abf0 tx=0x7f0e5801a000 comp rx=0 tx=0).stop 2023-10-06T04:49:25.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.152+0000 7f0e63fff700 1 -- 172.21.15.55:0/3471709260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e4c004810 msgr2=0x7f0e640c3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:25.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.152+0000 7f0e63fff700 1 --2- 172.21.15.55:0/3471709260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e4c004810 0x7f0e640c3cb0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c710 tx=0x7f0e540019d0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.152+0000 7f0e63fff700 1 -- 172.21.15.55:0/3471709260 shutdown_connections 2023-10-06T04:49:25.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.152+0000 7f0e63fff700 1 --2- 172.21.15.55:0/3471709260 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f0e50066f30 0x7f0e500693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.152+0000 7f0e63fff700 1 --2- 172.21.15.55:0/3471709260 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e64168ce0 0x7f0e64169160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.152+0000 7f0e63fff700 1 --2- 172.21.15.55:0/3471709260 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e4c004810 0x7f0e640c3cb0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.152+0000 7f0e63fff700 1 --2- 172.21.15.55:0/3471709260 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e640cbd60 0x7f0e640ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.152+0000 7f0e63fff700 1 -- 172.21.15.55:0/3471709260 >> 172.21.15.55:0/3471709260 conn(0x7f0e640cc1f0 msgr2=0x7f0e640c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:25.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.152+0000 7f0e63fff700 1 -- 172.21.15.55:0/3471709260 shutdown_connections 2023-10-06T04:49:25.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.152+0000 7f0e63fff700 1 -- 172.21.15.55:0/3471709260 wait complete. 2023-10-06T04:49:25.157 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T04:49:25.172 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-06T04:49:23.337226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":523,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":330,"ondisk_log_size":330,"up":190,"acting":190,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2453660,"kb_used_data":18012,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752521060,"statfs":{"total":773094113280,"available":770581565440,"internally_reserved":0,"allocated":18444288,"data_stored":10203569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.036418"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346440+0000","last_change":"2023-10-06T04:49:20.341012+0000","last_active":"2023-10-06T04:49:20.346440+0000","last_peered":"2023-10-06T04:49:20.346440+0000","last_clean":"2023-10-06T04:49:20.346440+0000","last_became_active":"2023-10-06T04:49:20.340740+0000","last_became_peered":"2023-10-06T04:49:20.340740+0000","last_unstale":"2023-10-06T04:49:20.346440+0000","last_undegraded":"2023-10-06T04:49:20.346440+0000","last_fullsized":"2023-10-06T04:49:20.346440+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341092+0000","last_change":"2023-10-06T04:49:20.336745+0000","last_active":"2023-10-06T04:49:20.341092+0000","last_peered":"2023-10-06T04:49:20.341092+0000","last_clean":"2023-10-06T04:49:20.341092+0000","last_became_active":"2023-10-06T04:49:20.336500+0000","last_became_peered":"2023-10-06T04:49:20.336500+0000","last_unstale":"2023-10-06T04:49:20.341092+0000","last_undegraded":"2023-10-06T04:49:20.341092+0000","last_fullsized":"2023-10-06T04:49:20.341092+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:28.214277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341469+0000","last_change":"2023-10-06T04:49:20.334944+0000","last_active":"2023-10-06T04:49:20.341469+0000","last_peered":"2023-10-06T04:49:20.341469+0000","last_clean":"2023-10-06T04:49:20.341469+0000","last_became_active":"2023-10-06T04:49:20.333924+0000","last_became_peered":"2023-10-06T04:49:20.333924+0000","last_unstale":"2023-10-06T04:49:20.341469+0000","last_undegraded":"2023-10-06T04:49:20.341469+0000","last_fullsized":"2023-10-06T04:49:20.341469+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341150+0000","last_change":"2023-10-06T04:49:20.333903+0000","last_active":"2023-10-06T04:49:20.341150+0000","last_peered":"2023-10-06T04:49:20.341150+0000","last_clean":"2023-10-06T04:49:20.341150+0000","last_became_active":"2023-10-06T04:49:20.333572+0000","last_became_peered":"2023-10-06T04:49:20.333572+0000","last_unstale":"2023-10-06T04:49:20.341150+0000","last_undegraded":"2023-10-06T04:49:20.341150+0000","last_fullsized":"2023-10-06T04:49:20.341150+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:49.977768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339683+0000","last_change":"2023-10-06T04:49:20.338306+0000","last_active":"2023-10-06T04:49:20.339683+0000","last_peered":"2023-10-06T04:49:20.339683+0000","last_clean":"2023-10-06T04:49:20.339683+0000","last_became_active":"2023-10-06T04:49:20.338034+0000","last_became_peered":"2023-10-06T04:49:20.338034+0000","last_unstale":"2023-10-06T04:49:20.339683+0000","last_undegraded":"2023-10-06T04:49:20.339683+0000","last_fullsized":"2023-10-06T04:49:20.339683+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339447+0000","last_change":"2023-10-06T04:49:20.336417+0000","last_active":"2023-10-06T04:49:20.339447+0000","last_peered":"2023-10-06T04:49:20.339447+0000","last_clean":"2023-10-06T04:49:20.339447+0000","last_became_active":"2023-10-06T04:49:20.334348+0000","last_became_peered":"2023-10-06T04:49:20.334348+0000","last_unstale":"2023-10-06T04:49:20.339447+0000","last_undegraded":"2023-10-06T04:49:20.339447+0000","last_fullsized":"2023-10-06T04:49:20.339447+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:54.036275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.345389+0000","last_change":"2023-10-06T04:49:20.339825+0000","last_active":"2023-10-06T04:49:20.345389+0000","last_peered":"2023-10-06T04:49:20.345389+0000","last_clean":"2023-10-06T04:49:20.345389+0000","last_became_active":"2023-10-06T04:49:20.339565+0000","last_became_peered":"2023-10-06T04:49:20.339565+0000","last_unstale":"2023-10-06T04:49:20.345389+0000","last_undegraded":"2023-10-06T04:49:20.345389+0000","last_fullsized":"2023-10-06T04:49:20.345389+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:54.036275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.340009+0000","last_change":"2023-10-06T04:49:20.336646+0000","last_active":"2023-10-06T04:49:20.340009+0000","last_peered":"2023-10-06T04:49:20.340009+0000","last_clean":"2023-10-06T04:49:20.340009+0000","last_became_active":"2023-10-06T04:49:20.334441+0000","last_became_peered":"2023-10-06T04:49:20.334441+0000","last_unstale":"2023-10-06T04:49:20.340009+0000","last_undegraded":"2023-10-06T04:49:20.340009+0000","last_fullsized":"2023-10-06T04:49:20.340009+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:23.582332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333816+0000","last_change":"2023-10-06T04:49:20.331180+0000","last_active":"2023-10-06T04:49:20.333816+0000","last_peered":"2023-10-06T04:49:20.333816+0000","last_clean":"2023-10-06T04:49:20.333816+0000","last_became_active":"2023-10-06T04:49:20.330817+0000","last_became_peered":"2023-10-06T04:49:20.330817+0000","last_unstale":"2023-10-06T04:49:20.333816+0000","last_undegraded":"2023-10-06T04:49:20.333816+0000","last_fullsized":"2023-10-06T04:49:20.333816+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.336853+0000","last_change":"2023-10-06T04:49:20.336069+0000","last_active":"2023-10-06T04:49:20.336853+0000","last_peered":"2023-10-06T04:49:20.336853+0000","last_clean":"2023-10-06T04:49:20.336853+0000","last_became_active":"2023-10-06T04:49:20.335808+0000","last_became_peered":"2023-10-06T04:49:20.335808+0000","last_unstale":"2023-10-06T04:49:20.336853+0000","last_undegraded":"2023-10-06T04:49:20.336853+0000","last_fullsized":"2023-10-06T04:49:20.336853+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:53.477421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.334968+0000","last_change":"2023-10-06T04:49:20.328201+0000","last_active":"2023-10-06T04:49:20.334968+0000","last_peered":"2023-10-06T04:49:20.334968+0000","last_clean":"2023-10-06T04:49:20.334968+0000","last_became_active":"2023-10-06T04:49:20.327560+0000","last_became_peered":"2023-10-06T04:49:20.327560+0000","last_unstale":"2023-10-06T04:49:20.334968+0000","last_undegraded":"2023-10-06T04:49:20.334968+0000","last_fullsized":"2023-10-06T04:49:20.334968+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:40.721698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333739+0000","last_change":"2023-10-06T04:49:20.331130+0000","last_active":"2023-10-06T04:49:20.333739+0000","last_peered":"2023-10-06T04:49:20.333739+0000","last_clean":"2023-10-06T04:49:20.333739+0000","last_became_active":"2023-10-06T04:49:20.330637+0000","last_became_peered":"2023-10-06T04:49:20.330637+0000","last_unstale":"2023-10-06T04:49:20.333739+0000","last_undegraded":"2023-10-06T04:49:20.333739+0000","last_fullsized":"2023-10-06T04:49:20.333739+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341277+0000","last_change":"2023-10-06T04:49:20.337139+0000","last_active":"2023-10-06T04:49:20.341277+0000","last_peered":"2023-10-06T04:49:20.341277+0000","last_clean":"2023-10-06T04:49:20.341277+0000","last_became_active":"2023-10-06T04:49:20.336396+0000","last_became_peered":"2023-10-06T04:49:20.336396+0000","last_unstale":"2023-10-06T04:49:20.341277+0000","last_undegraded":"2023-10-06T04:49:20.341277+0000","last_fullsized":"2023-10-06T04:49:20.341277+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:22.496792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341177+0000","last_change":"2023-10-06T04:49:20.337332+0000","last_active":"2023-10-06T04:49:20.341177+0000","last_peered":"2023-10-06T04:49:20.341177+0000","last_clean":"2023-10-06T04:49:20.341177+0000","last_became_active":"2023-10-06T04:49:20.337091+0000","last_became_peered":"2023-10-06T04:49:20.337091+0000","last_unstale":"2023-10-06T04:49:20.341177+0000","last_undegraded":"2023-10-06T04:49:20.341177+0000","last_fullsized":"2023-10-06T04:49:20.341177+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:57.445018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333411+0000","last_change":"2023-10-06T04:49:20.329112+0000","last_active":"2023-10-06T04:49:20.333411+0000","last_peered":"2023-10-06T04:49:20.333411+0000","last_clean":"2023-10-06T04:49:20.333411+0000","last_became_active":"2023-10-06T04:49:20.328156+0000","last_became_peered":"2023-10-06T04:49:20.328156+0000","last_unstale":"2023-10-06T04:49:20.333411+0000","last_undegraded":"2023-10-06T04:49:20.333411+0000","last_fullsized":"2023-10-06T04:49:20.333411+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:25.017277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341591+0000","last_change":"2023-10-06T04:49:20.334020+0000","last_active":"2023-10-06T04:49:20.341591+0000","last_peered":"2023-10-06T04:49:20.341591+0000","last_clean":"2023-10-06T04:49:20.341591+0000","last_became_active":"2023-10-06T04:49:20.333516+0000","last_became_peered":"2023-10-06T04:49:20.333516+0000","last_unstale":"2023-10-06T04:49:20.341591+0000","last_undegraded":"2023-10-06T04:49:20.341591+0000","last_fullsized":"2023-10-06T04:49:20.341591+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:43.552397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347965+0000","last_change":"2023-10-06T04:49:20.346422+0000","last_active":"2023-10-06T04:49:20.347965+0000","last_peered":"2023-10-06T04:49:20.347965+0000","last_clean":"2023-10-06T04:49:20.347965+0000","last_became_active":"2023-10-06T04:49:20.346126+0000","last_became_peered":"2023-10-06T04:49:20.346126+0000","last_unstale":"2023-10-06T04:49:20.347965+0000","last_undegraded":"2023-10-06T04:49:20.347965+0000","last_fullsized":"2023-10-06T04:49:20.347965+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:58.644879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355577+0000","last_change":"2023-10-06T04:49:20.350471+0000","last_active":"2023-10-06T04:49:20.355577+0000","last_peered":"2023-10-06T04:49:20.355577+0000","last_clean":"2023-10-06T04:49:20.355577+0000","last_became_active":"2023-10-06T04:49:20.350195+0000","last_became_peered":"2023-10-06T04:49:20.350195+0000","last_unstale":"2023-10-06T04:49:20.355577+0000","last_undegraded":"2023-10-06T04:49:20.355577+0000","last_fullsized":"2023-10-06T04:49:20.355577+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:12.727118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339262+0000","last_change":"2023-10-06T04:49:20.334837+0000","last_active":"2023-10-06T04:49:20.339262+0000","last_peered":"2023-10-06T04:49:20.339262+0000","last_clean":"2023-10-06T04:49:20.339262+0000","last_became_active":"2023-10-06T04:49:20.334580+0000","last_became_peered":"2023-10-06T04:49:20.334580+0000","last_unstale":"2023-10-06T04:49:20.339262+0000","last_undegraded":"2023-10-06T04:49:20.339262+0000","last_fullsized":"2023-10-06T04:49:20.339262+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:02.862072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.338389+0000","last_change":"2023-10-06T04:49:20.334048+0000","last_active":"2023-10-06T04:49:20.338389+0000","last_peered":"2023-10-06T04:49:20.338389+0000","last_clean":"2023-10-06T04:49:20.338389+0000","last_became_active":"2023-10-06T04:49:20.333761+0000","last_became_peered":"2023-10-06T04:49:20.333761+0000","last_unstale":"2023-10-06T04:49:20.338389+0000","last_undegraded":"2023-10-06T04:49:20.338389+0000","last_fullsized":"2023-10-06T04:49:20.338389+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344972+0000","last_change":"2023-10-06T04:49:20.336537+0000","last_active":"2023-10-06T04:49:20.344972+0000","last_peered":"2023-10-06T04:49:20.344972+0000","last_clean":"2023-10-06T04:49:20.344972+0000","last_became_active":"2023-10-06T04:49:20.336251+0000","last_became_peered":"2023-10-06T04:49:20.336251+0000","last_unstale":"2023-10-06T04:49:20.344972+0000","last_undegraded":"2023-10-06T04:49:20.344972+0000","last_fullsized":"2023-10-06T04:49:20.344972+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.329722+0000","last_change":"2023-10-06T04:49:20.326750+0000","last_active":"2023-10-06T04:49:20.329722+0000","last_peered":"2023-10-06T04:49:20.329722+0000","last_clean":"2023-10-06T04:49:20.329722+0000","last_became_active":"2023-10-06T04:49:20.326466+0000","last_became_peered":"2023-10-06T04:49:20.326466+0000","last_unstale":"2023-10-06T04:49:20.329722+0000","last_undegraded":"2023-10-06T04:49:20.329722+0000","last_fullsized":"2023-10-06T04:49:20.329722+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:52.748607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.345875+0000","last_change":"2023-10-06T04:49:20.337383+0000","last_active":"2023-10-06T04:49:20.345875+0000","last_peered":"2023-10-06T04:49:20.345875+0000","last_clean":"2023-10-06T04:49:20.345875+0000","last_became_active":"2023-10-06T04:49:20.337130+0000","last_became_peered":"2023-10-06T04:49:20.337130+0000","last_unstale":"2023-10-06T04:49:20.345875+0000","last_undegraded":"2023-10-06T04:49:20.345875+0000","last_fullsized":"2023-10-06T04:49:20.345875+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355403+0000","last_change":"2023-10-06T04:49:20.347903+0000","last_active":"2023-10-06T04:49:20.355403+0000","last_peered":"2023-10-06T04:49:20.355403+0000","last_clean":"2023-10-06T04:49:20.355403+0000","last_became_active":"2023-10-06T04:49:20.346956+0000","last_became_peered":"2023-10-06T04:49:20.346956+0000","last_unstale":"2023-10-06T04:49:20.355403+0000","last_undegraded":"2023-10-06T04:49:20.355403+0000","last_fullsized":"2023-10-06T04:49:20.355403+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:20.026936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339401+0000","last_change":"2023-10-06T04:49:20.336472+0000","last_active":"2023-10-06T04:49:20.339401+0000","last_peered":"2023-10-06T04:49:20.339401+0000","last_clean":"2023-10-06T04:49:20.339401+0000","last_became_active":"2023-10-06T04:49:20.334373+0000","last_became_peered":"2023-10-06T04:49:20.334373+0000","last_unstale":"2023-10-06T04:49:20.339401+0000","last_undegraded":"2023-10-06T04:49:20.339401+0000","last_fullsized":"2023-10-06T04:49:20.339401+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:20.026936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346959+0000","last_change":"2023-10-06T04:49:20.344850+0000","last_active":"2023-10-06T04:49:20.346959+0000","last_peered":"2023-10-06T04:49:20.346959+0000","last_clean":"2023-10-06T04:49:20.346959+0000","last_became_active":"2023-10-06T04:49:20.343891+0000","last_became_peered":"2023-10-06T04:49:20.343891+0000","last_unstale":"2023-10-06T04:49:20.346959+0000","last_undegraded":"2023-10-06T04:49:20.346959+0000","last_fullsized":"2023-10-06T04:49:20.346959+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:28.957190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347992+0000","last_change":"2023-10-06T04:49:20.344409+0000","last_active":"2023-10-06T04:49:20.347992+0000","last_peered":"2023-10-06T04:49:20.347992+0000","last_clean":"2023-10-06T04:49:20.347992+0000","last_became_active":"2023-10-06T04:49:20.343734+0000","last_became_peered":"2023-10-06T04:49:20.343734+0000","last_unstale":"2023-10-06T04:49:20.347992+0000","last_undegraded":"2023-10-06T04:49:20.347992+0000","last_fullsized":"2023-10-06T04:49:20.347992+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:27.555498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333496+0000","last_change":"2023-10-06T04:49:20.327245+0000","last_active":"2023-10-06T04:49:20.333496+0000","last_peered":"2023-10-06T04:49:20.333496+0000","last_clean":"2023-10-06T04:49:20.333496+0000","last_became_active":"2023-10-06T04:49:20.326914+0000","last_became_peered":"2023-10-06T04:49:20.326914+0000","last_unstale":"2023-10-06T04:49:20.333496+0000","last_undegraded":"2023-10-06T04:49:20.333496+0000","last_fullsized":"2023-10-06T04:49:20.333496+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:19.514355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341007+0000","last_change":"2023-10-06T04:49:20.333556+0000","last_active":"2023-10-06T04:49:20.341007+0000","last_peered":"2023-10-06T04:49:20.341007+0000","last_clean":"2023-10-06T04:49:20.341007+0000","last_became_active":"2023-10-06T04:49:20.333251+0000","last_became_peered":"2023-10-06T04:49:20.333251+0000","last_unstale":"2023-10-06T04:49:20.341007+0000","last_undegraded":"2023-10-06T04:49:20.341007+0000","last_fullsized":"2023-10-06T04:49:20.341007+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"172'4","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344554+0000","last_change":"2023-10-06T04:49:19.320190+0000","last_active":"2023-10-06T04:49:20.344554+0000","last_peered":"2023-10-06T04:49:20.344554+0000","last_clean":"2023-10-06T04:49:20.344554+0000","last_became_active":"2023-10-06T04:49:19.319917+0000","last_became_peered":"2023-10-06T04:49:19.319917+0000","last_unstale":"2023-10-06T04:49:20.344554+0000","last_undegraded":"2023-10-06T04:49:20.344554+0000","last_fullsized":"2023-10-06T04:49:20.344554+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:26.266062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341489+0000","last_change":"2023-10-06T04:49:20.336237+0000","last_active":"2023-10-06T04:49:20.341489+0000","last_peered":"2023-10-06T04:49:20.341489+0000","last_clean":"2023-10-06T04:49:20.341489+0000","last_became_active":"2023-10-06T04:49:20.335986+0000","last_became_peered":"2023-10-06T04:49:20.335986+0000","last_unstale":"2023-10-06T04:49:20.341489+0000","last_undegraded":"2023-10-06T04:49:20.341489+0000","last_fullsized":"2023-10-06T04:49:20.341489+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355736+0000","last_change":"2023-10-06T04:49:19.335778+0000","last_active":"2023-10-06T04:49:20.355736+0000","last_peered":"2023-10-06T04:49:20.355736+0000","last_clean":"2023-10-06T04:49:20.355736+0000","last_became_active":"2023-10-06T04:49:19.335470+0000","last_became_peered":"2023-10-06T04:49:19.335470+0000","last_unstale":"2023-10-06T04:49:20.355736+0000","last_undegraded":"2023-10-06T04:49:20.355736+0000","last_fullsized":"2023-10-06T04:49:20.355736+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:01.854229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356161+0000","last_change":"2023-10-06T04:49:19.335543+0000","last_active":"2023-10-06T04:49:20.356161+0000","last_peered":"2023-10-06T04:49:20.356161+0000","last_clean":"2023-10-06T04:49:20.356161+0000","last_became_active":"2023-10-06T04:49:19.335265+0000","last_became_peered":"2023-10-06T04:49:19.335265+0000","last_unstale":"2023-10-06T04:49:20.356161+0000","last_undegraded":"2023-10-06T04:49:20.356161+0000","last_fullsized":"2023-10-06T04:49:20.356161+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:41.377516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350856+0000","last_change":"2023-10-06T04:49:20.346902+0000","last_active":"2023-10-06T04:49:20.350856+0000","last_peered":"2023-10-06T04:49:20.350856+0000","last_clean":"2023-10-06T04:49:20.350856+0000","last_became_active":"2023-10-06T04:49:20.346594+0000","last_became_peered":"2023-10-06T04:49:20.346594+0000","last_unstale":"2023-10-06T04:49:20.350856+0000","last_undegraded":"2023-10-06T04:49:20.350856+0000","last_fullsized":"2023-10-06T04:49:20.350856+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:00.206328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.349018+0000","last_change":"2023-10-06T04:49:19.328489+0000","last_active":"2023-10-06T04:49:20.349018+0000","last_peered":"2023-10-06T04:49:20.349018+0000","last_clean":"2023-10-06T04:49:20.349018+0000","last_became_active":"2023-10-06T04:49:19.328220+0000","last_became_peered":"2023-10-06T04:49:19.328220+0000","last_unstale":"2023-10-06T04:49:20.349018+0000","last_undegraded":"2023-10-06T04:49:20.349018+0000","last_fullsized":"2023-10-06T04:49:20.349018+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:26.266062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.337318+0000","last_change":"2023-10-06T04:49:20.330590+0000","last_active":"2023-10-06T04:49:20.337318+0000","last_peered":"2023-10-06T04:49:20.337318+0000","last_clean":"2023-10-06T04:49:20.337318+0000","last_became_active":"2023-10-06T04:49:20.330306+0000","last_became_peered":"2023-10-06T04:49:20.330306+0000","last_unstale":"2023-10-06T04:49:20.337318+0000","last_undegraded":"2023-10-06T04:49:20.337318+0000","last_fullsized":"2023-10-06T04:49:20.337318+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:42.535859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:49:18.291949+0000","last_change":"2023-10-06T04:49:18.303303+0000","last_active":"2023-10-06T04:49:18.291949+0000","last_peered":"2023-10-06T04:49:18.291949+0000","last_clean":"2023-10-06T04:49:18.291949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:18.291949+0000","last_undegraded":"2023-10-06T04:49:18.291949+0000","last_fullsized":"2023-10-06T04:49:18.291949+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:49.003200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341432+0000","last_change":"2023-10-06T04:49:20.334543+0000","last_active":"2023-10-06T04:49:20.341432+0000","last_peered":"2023-10-06T04:49:20.341432+0000","last_clean":"2023-10-06T04:49:20.341432+0000","last_became_active":"2023-10-06T04:49:20.333665+0000","last_became_peered":"2023-10-06T04:49:20.333665+0000","last_unstale":"2023-10-06T04:49:20.341432+0000","last_undegraded":"2023-10-06T04:49:20.341432+0000","last_fullsized":"2023-10-06T04:49:20.341432+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:20.869185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347200+0000","last_change":"2023-10-06T04:49:19.330786+0000","last_active":"2023-10-06T04:49:20.347200+0000","last_peered":"2023-10-06T04:49:20.347200+0000","last_clean":"2023-10-06T04:49:20.347200+0000","last_became_active":"2023-10-06T04:49:19.330506+0000","last_became_peered":"2023-10-06T04:49:19.330506+0000","last_unstale":"2023-10-06T04:49:20.347200+0000","last_undegraded":"2023-10-06T04:49:20.347200+0000","last_fullsized":"2023-10-06T04:49:20.347200+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:54.415706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333277+0000","last_change":"2023-10-06T04:49:20.332521+0000","last_active":"2023-10-06T04:49:20.333277+0000","last_peered":"2023-10-06T04:49:20.333277+0000","last_clean":"2023-10-06T04:49:20.333277+0000","last_became_active":"2023-10-06T04:49:20.332264+0000","last_became_peered":"2023-10-06T04:49:20.332264+0000","last_unstale":"2023-10-06T04:49:20.333277+0000","last_undegraded":"2023-10-06T04:49:20.333277+0000","last_fullsized":"2023-10-06T04:49:20.333277+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:16.543656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332961+0000","last_change":"2023-10-06T04:49:19.332296+0000","last_active":"2023-10-06T04:49:20.332961+0000","last_peered":"2023-10-06T04:49:20.332961+0000","last_clean":"2023-10-06T04:49:20.332961+0000","last_became_active":"2023-10-06T04:49:19.332022+0000","last_became_peered":"2023-10-06T04:49:19.332022+0000","last_unstale":"2023-10-06T04:49:20.332961+0000","last_undegraded":"2023-10-06T04:49:20.332961+0000","last_fullsized":"2023-10-06T04:49:20.332961+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:18.846190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350851+0000","last_change":"2023-10-06T04:49:20.341492+0000","last_active":"2023-10-06T04:49:20.350851+0000","last_peered":"2023-10-06T04:49:20.350851+0000","last_clean":"2023-10-06T04:49:20.350851+0000","last_became_active":"2023-10-06T04:49:20.341257+0000","last_became_peered":"2023-10-06T04:49:20.341257+0000","last_unstale":"2023-10-06T04:49:20.350851+0000","last_undegraded":"2023-10-06T04:49:20.350851+0000","last_fullsized":"2023-10-06T04:49:20.350851+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:28.214277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.315096+0000","last_change":"2023-10-06T04:49:19.324312+0000","last_active":"2023-10-06T04:49:20.315096+0000","last_peered":"2023-10-06T04:49:20.315096+0000","last_clean":"2023-10-06T04:49:20.315096+0000","last_became_active":"2023-10-06T04:49:19.324046+0000","last_became_peered":"2023-10-06T04:49:19.324046+0000","last_unstale":"2023-10-06T04:49:20.315096+0000","last_undegraded":"2023-10-06T04:49:20.315096+0000","last_fullsized":"2023-10-06T04:49:20.315096+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:49.588344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344426+0000","last_change":"2023-10-06T04:49:20.341875+0000","last_active":"2023-10-06T04:49:20.344426+0000","last_peered":"2023-10-06T04:49:20.344426+0000","last_clean":"2023-10-06T04:49:20.344426+0000","last_became_active":"2023-10-06T04:49:20.341312+0000","last_became_peered":"2023-10-06T04:49:20.341312+0000","last_unstale":"2023-10-06T04:49:20.344426+0000","last_undegraded":"2023-10-06T04:49:20.344426+0000","last_fullsized":"2023-10-06T04:49:20.344426+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:07.881803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.351368+0000","last_change":"2023-10-06T04:49:19.331266+0000","last_active":"2023-10-06T04:49:20.351368+0000","last_peered":"2023-10-06T04:49:20.351368+0000","last_clean":"2023-10-06T04:49:20.351368+0000","last_became_active":"2023-10-06T04:49:19.331010+0000","last_became_peered":"2023-10-06T04:49:19.331010+0000","last_unstale":"2023-10-06T04:49:20.351368+0000","last_undegraded":"2023-10-06T04:49:20.351368+0000","last_fullsized":"2023-10-06T04:49:20.351368+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:41.647975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355558+0000","last_change":"2023-10-06T04:49:20.350352+0000","last_active":"2023-10-06T04:49:20.355558+0000","last_peered":"2023-10-06T04:49:20.355558+0000","last_clean":"2023-10-06T04:49:20.355558+0000","last_became_active":"2023-10-06T04:49:20.350084+0000","last_became_peered":"2023-10-06T04:49:20.350084+0000","last_unstale":"2023-10-06T04:49:20.355558+0000","last_undegraded":"2023-10-06T04:49:20.355558+0000","last_fullsized":"2023-10-06T04:49:20.355558+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:54.488863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.353329+0000","last_change":"2023-10-06T04:49:19.333467+0000","last_active":"2023-10-06T04:49:20.353329+0000","last_peered":"2023-10-06T04:49:20.353329+0000","last_clean":"2023-10-06T04:49:20.353329+0000","last_became_active":"2023-10-06T04:49:19.333191+0000","last_became_peered":"2023-10-06T04:49:19.333191+0000","last_unstale":"2023-10-06T04:49:20.353329+0000","last_undegraded":"2023-10-06T04:49:20.353329+0000","last_fullsized":"2023-10-06T04:49:20.353329+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:50.631194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346482+0000","last_change":"2023-10-06T04:49:20.340075+0000","last_active":"2023-10-06T04:49:20.346482+0000","last_peered":"2023-10-06T04:49:20.346482+0000","last_clean":"2023-10-06T04:49:20.346482+0000","last_became_active":"2023-10-06T04:49:20.339781+0000","last_became_peered":"2023-10-06T04:49:20.339781+0000","last_unstale":"2023-10-06T04:49:20.346482+0000","last_undegraded":"2023-10-06T04:49:20.346482+0000","last_fullsized":"2023-10-06T04:49:20.346482+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:57.445018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344697+0000","last_change":"2023-10-06T04:49:19.324434+0000","last_active":"2023-10-06T04:49:20.344697+0000","last_peered":"2023-10-06T04:49:20.344697+0000","last_clean":"2023-10-06T04:49:20.344697+0000","last_became_active":"2023-10-06T04:49:19.324162+0000","last_became_peered":"2023-10-06T04:49:19.324162+0000","last_unstale":"2023-10-06T04:49:20.344697+0000","last_undegraded":"2023-10-06T04:49:20.344697+0000","last_fullsized":"2023-10-06T04:49:20.344697+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350745+0000","last_change":"2023-10-06T04:49:20.345980+0000","last_active":"2023-10-06T04:49:20.350745+0000","last_peered":"2023-10-06T04:49:20.350745+0000","last_clean":"2023-10-06T04:49:20.350745+0000","last_became_active":"2023-10-06T04:49:20.345691+0000","last_became_peered":"2023-10-06T04:49:20.345691+0000","last_unstale":"2023-10-06T04:49:20.350745+0000","last_undegraded":"2023-10-06T04:49:20.350745+0000","last_fullsized":"2023-10-06T04:49:20.350745+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:23.582332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.319623+0000","last_change":"2023-10-06T04:49:19.330962+0000","last_active":"2023-10-06T04:49:20.319623+0000","last_peered":"2023-10-06T04:49:20.319623+0000","last_clean":"2023-10-06T04:49:20.319623+0000","last_became_active":"2023-10-06T04:49:19.330616+0000","last_became_peered":"2023-10-06T04:49:19.330616+0000","last_unstale":"2023-10-06T04:49:20.319623+0000","last_undegraded":"2023-10-06T04:49:20.319623+0000","last_fullsized":"2023-10-06T04:49:20.319623+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:04.272287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355580+0000","last_change":"2023-10-06T04:49:20.350356+0000","last_active":"2023-10-06T04:49:20.355580+0000","last_peered":"2023-10-06T04:49:20.355580+0000","last_clean":"2023-10-06T04:49:20.355580+0000","last_became_active":"2023-10-06T04:49:20.350105+0000","last_became_peered":"2023-10-06T04:49:20.350105+0000","last_unstale":"2023-10-06T04:49:20.355580+0000","last_undegraded":"2023-10-06T04:49:20.355580+0000","last_fullsized":"2023-10-06T04:49:20.355580+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:10.637938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346898+0000","last_change":"2023-10-06T04:49:19.333331+0000","last_active":"2023-10-06T04:49:20.346898+0000","last_peered":"2023-10-06T04:49:20.346898+0000","last_clean":"2023-10-06T04:49:20.346898+0000","last_became_active":"2023-10-06T04:49:19.332774+0000","last_became_peered":"2023-10-06T04:49:19.332774+0000","last_unstale":"2023-10-06T04:49:20.346898+0000","last_undegraded":"2023-10-06T04:49:20.346898+0000","last_fullsized":"2023-10-06T04:49:20.346898+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:19.019093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346151+0000","last_change":"2023-10-06T04:49:20.345404+0000","last_active":"2023-10-06T04:49:20.346151+0000","last_peered":"2023-10-06T04:49:20.346151+0000","last_clean":"2023-10-06T04:49:20.346151+0000","last_became_active":"2023-10-06T04:49:20.345129+0000","last_became_peered":"2023-10-06T04:49:20.345129+0000","last_unstale":"2023-10-06T04:49:20.346151+0000","last_undegraded":"2023-10-06T04:49:20.346151+0000","last_fullsized":"2023-10-06T04:49:20.346151+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:25.017277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:49:18.291949+0000","last_change":"2023-10-06T04:49:18.303757+0000","last_active":"2023-10-06T04:49:18.291949+0000","last_peered":"2023-10-06T04:49:18.291949+0000","last_clean":"2023-10-06T04:49:18.291949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:18.291949+0000","last_undegraded":"2023-10-06T04:49:18.291949+0000","last_fullsized":"2023-10-06T04:49:18.291949+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:22.574489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350774+0000","last_change":"2023-10-06T04:49:20.347069+0000","last_active":"2023-10-06T04:49:20.350774+0000","last_peered":"2023-10-06T04:49:20.350774+0000","last_clean":"2023-10-06T04:49:20.350774+0000","last_became_active":"2023-10-06T04:49:20.346801+0000","last_became_peered":"2023-10-06T04:49:20.346801+0000","last_unstale":"2023-10-06T04:49:20.350774+0000","last_undegraded":"2023-10-06T04:49:20.350774+0000","last_fullsized":"2023-10-06T04:49:20.350774+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:34.988176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321187+0000","last_change":"2023-10-06T04:49:19.329229+0000","last_active":"2023-10-06T04:49:20.321187+0000","last_peered":"2023-10-06T04:49:20.321187+0000","last_clean":"2023-10-06T04:49:20.321187+0000","last_became_active":"2023-10-06T04:49:19.328964+0000","last_became_peered":"2023-10-06T04:49:19.328964+0000","last_unstale":"2023-10-06T04:49:20.321187+0000","last_undegraded":"2023-10-06T04:49:20.321187+0000","last_fullsized":"2023-10-06T04:49:20.321187+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:53.870884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341139+0000","last_change":"2023-10-06T04:49:20.334997+0000","last_active":"2023-10-06T04:49:20.341139+0000","last_peered":"2023-10-06T04:49:20.341139+0000","last_clean":"2023-10-06T04:49:20.341139+0000","last_became_active":"2023-10-06T04:49:20.334723+0000","last_became_peered":"2023-10-06T04:49:20.334723+0000","last_unstale":"2023-10-06T04:49:20.341139+0000","last_undegraded":"2023-10-06T04:49:20.341139+0000","last_fullsized":"2023-10-06T04:49:20.341139+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:41.560870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-06T04:49:18.291949+0000","last_change":"2023-10-06T04:49:18.303604+0000","last_active":"2023-10-06T04:49:18.291949+0000","last_peered":"2023-10-06T04:49:18.291949+0000","last_clean":"2023-10-06T04:49:18.291949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:18.291949+0000","last_undegraded":"2023-10-06T04:49:18.291949+0000","last_fullsized":"2023-10-06T04:49:18.291949+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:53.028433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339461+0000","last_change":"2023-10-06T04:49:20.337671+0000","last_active":"2023-10-06T04:49:20.339461+0000","last_peered":"2023-10-06T04:49:20.339461+0000","last_clean":"2023-10-06T04:49:20.339461+0000","last_became_active":"2023-10-06T04:49:20.337383+0000","last_became_peered":"2023-10-06T04:49:20.337383+0000","last_unstale":"2023-10-06T04:49:20.339461+0000","last_undegraded":"2023-10-06T04:49:20.339461+0000","last_fullsized":"2023-10-06T04:49:20.339461+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:41.316112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.342104+0000","last_change":"2023-10-06T04:49:19.328915+0000","last_active":"2023-10-06T04:49:20.342104+0000","last_peered":"2023-10-06T04:49:20.342104+0000","last_clean":"2023-10-06T04:49:20.342104+0000","last_became_active":"2023-10-06T04:49:19.328630+0000","last_became_peered":"2023-10-06T04:49:19.328630+0000","last_unstale":"2023-10-06T04:49:20.342104+0000","last_undegraded":"2023-10-06T04:49:20.342104+0000","last_fullsized":"2023-10-06T04:49:20.342104+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:27.949348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.343843+0000","last_change":"2023-10-06T04:49:20.341772+0000","last_active":"2023-10-06T04:49:20.343843+0000","last_peered":"2023-10-06T04:49:20.343843+0000","last_clean":"2023-10-06T04:49:20.343843+0000","last_became_active":"2023-10-06T04:49:20.341489+0000","last_became_peered":"2023-10-06T04:49:20.341489+0000","last_unstale":"2023-10-06T04:49:20.343843+0000","last_undegraded":"2023-10-06T04:49:20.343843+0000","last_fullsized":"2023-10-06T04:49:20.343843+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:10.460143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.330303+0000","last_change":"2023-10-06T04:49:19.331671+0000","last_active":"2023-10-06T04:49:20.330303+0000","last_peered":"2023-10-06T04:49:20.330303+0000","last_clean":"2023-10-06T04:49:20.330303+0000","last_became_active":"2023-10-06T04:49:19.330061+0000","last_became_peered":"2023-10-06T04:49:19.330061+0000","last_unstale":"2023-10-06T04:49:20.330303+0000","last_undegraded":"2023-10-06T04:49:20.330303+0000","last_fullsized":"2023-10-06T04:49:20.330303+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:09.188792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356042+0000","last_change":"2023-10-06T04:49:20.352177+0000","last_active":"2023-10-06T04:49:20.356042+0000","last_peered":"2023-10-06T04:49:20.356042+0000","last_clean":"2023-10-06T04:49:20.356042+0000","last_became_active":"2023-10-06T04:49:20.351885+0000","last_became_peered":"2023-10-06T04:49:20.351885+0000","last_unstale":"2023-10-06T04:49:20.356042+0000","last_undegraded":"2023-10-06T04:49:20.356042+0000","last_fullsized":"2023-10-06T04:49:20.356042+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320737+0000","last_change":"2023-10-06T04:49:19.329132+0000","last_active":"2023-10-06T04:49:20.320737+0000","last_peered":"2023-10-06T04:49:20.320737+0000","last_clean":"2023-10-06T04:49:20.320737+0000","last_became_active":"2023-10-06T04:49:19.328841+0000","last_became_peered":"2023-10-06T04:49:19.328841+0000","last_unstale":"2023-10-06T04:49:20.320737+0000","last_undegraded":"2023-10-06T04:49:20.320737+0000","last_fullsized":"2023-10-06T04:49:20.320737+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:54.415706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344647+0000","last_change":"2023-10-06T04:49:20.343226+0000","last_active":"2023-10-06T04:49:20.344647+0000","last_peered":"2023-10-06T04:49:20.344647+0000","last_clean":"2023-10-06T04:49:20.344647+0000","last_became_active":"2023-10-06T04:49:20.342932+0000","last_became_peered":"2023-10-06T04:49:20.342932+0000","last_unstale":"2023-10-06T04:49:20.344647+0000","last_undegraded":"2023-10-06T04:49:20.344647+0000","last_fullsized":"2023-10-06T04:49:20.344647+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:28.172716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.316079+0000","last_change":"2023-10-06T04:49:19.328230+0000","last_active":"2023-10-06T04:49:20.316079+0000","last_peered":"2023-10-06T04:49:20.316079+0000","last_clean":"2023-10-06T04:49:20.316079+0000","last_became_active":"2023-10-06T04:49:19.327917+0000","last_became_peered":"2023-10-06T04:49:19.327917+0000","last_unstale":"2023-10-06T04:49:20.316079+0000","last_undegraded":"2023-10-06T04:49:20.316079+0000","last_fullsized":"2023-10-06T04:49:20.316079+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.900580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.325259+0000","last_change":"2023-10-06T04:49:19.331624+0000","last_active":"2023-10-06T04:49:20.325259+0000","last_peered":"2023-10-06T04:49:20.325259+0000","last_clean":"2023-10-06T04:49:20.325259+0000","last_became_active":"2023-10-06T04:49:19.331287+0000","last_became_peered":"2023-10-06T04:49:19.331287+0000","last_unstale":"2023-10-06T04:49:20.325259+0000","last_undegraded":"2023-10-06T04:49:20.325259+0000","last_fullsized":"2023-10-06T04:49:20.325259+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.288607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341412+0000","last_change":"2023-10-06T04:49:20.334336+0000","last_active":"2023-10-06T04:49:20.341412+0000","last_peered":"2023-10-06T04:49:20.341412+0000","last_clean":"2023-10-06T04:49:20.341412+0000","last_became_active":"2023-10-06T04:49:20.334097+0000","last_became_peered":"2023-10-06T04:49:20.334097+0000","last_unstale":"2023-10-06T04:49:20.341412+0000","last_undegraded":"2023-10-06T04:49:20.341412+0000","last_fullsized":"2023-10-06T04:49:20.341412+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:35.183960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320080+0000","last_change":"2023-10-06T04:49:19.332817+0000","last_active":"2023-10-06T04:49:20.320080+0000","last_peered":"2023-10-06T04:49:20.320080+0000","last_clean":"2023-10-06T04:49:20.320080+0000","last_became_active":"2023-10-06T04:49:19.331466+0000","last_became_peered":"2023-10-06T04:49:19.331466+0000","last_unstale":"2023-10-06T04:49:20.320080+0000","last_undegraded":"2023-10-06T04:49:20.320080+0000","last_fullsized":"2023-10-06T04:49:20.320080+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:01.585470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332830+0000","last_change":"2023-10-06T04:49:20.327613+0000","last_active":"2023-10-06T04:49:20.332830+0000","last_peered":"2023-10-06T04:49:20.332830+0000","last_clean":"2023-10-06T04:49:20.332830+0000","last_became_active":"2023-10-06T04:49:20.327331+0000","last_became_peered":"2023-10-06T04:49:20.327331+0000","last_unstale":"2023-10-06T04:49:20.332830+0000","last_undegraded":"2023-10-06T04:49:20.332830+0000","last_fullsized":"2023-10-06T04:49:20.332830+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:21.888629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339559+0000","last_change":"2023-10-06T04:49:19.327134+0000","last_active":"2023-10-06T04:49:20.339559+0000","last_peered":"2023-10-06T04:49:20.339559+0000","last_clean":"2023-10-06T04:49:20.339559+0000","last_became_active":"2023-10-06T04:49:19.326884+0000","last_became_peered":"2023-10-06T04:49:19.326884+0000","last_unstale":"2023-10-06T04:49:20.339559+0000","last_undegraded":"2023-10-06T04:49:20.339559+0000","last_fullsized":"2023-10-06T04:49:20.339559+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:40.308269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331074+0000","last_change":"2023-10-06T04:49:19.336399+0000","last_active":"2023-10-06T04:49:20.331074+0000","last_peered":"2023-10-06T04:49:20.331074+0000","last_clean":"2023-10-06T04:49:20.331074+0000","last_became_active":"2023-10-06T04:49:19.336144+0000","last_became_peered":"2023-10-06T04:49:19.336144+0000","last_unstale":"2023-10-06T04:49:20.331074+0000","last_undegraded":"2023-10-06T04:49:20.331074+0000","last_fullsized":"2023-10-06T04:49:20.331074+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:48.611176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333465+0000","last_change":"2023-10-06T04:49:20.325062+0000","last_active":"2023-10-06T04:49:20.333465+0000","last_peered":"2023-10-06T04:49:20.333465+0000","last_clean":"2023-10-06T04:49:20.333465+0000","last_became_active":"2023-10-06T04:49:20.324091+0000","last_became_peered":"2023-10-06T04:49:20.324091+0000","last_unstale":"2023-10-06T04:49:20.333465+0000","last_undegraded":"2023-10-06T04:49:20.333465+0000","last_fullsized":"2023-10-06T04:49:20.333465+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:28.109738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321039+0000","last_change":"2023-10-06T04:49:19.327314+0000","last_active":"2023-10-06T04:49:20.321039+0000","last_peered":"2023-10-06T04:49:20.321039+0000","last_clean":"2023-10-06T04:49:20.321039+0000","last_became_active":"2023-10-06T04:49:19.326851+0000","last_became_peered":"2023-10-06T04:49:19.326851+0000","last_unstale":"2023-10-06T04:49:20.321039+0000","last_undegraded":"2023-10-06T04:49:20.321039+0000","last_fullsized":"2023-10-06T04:49:20.321039+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:26.547655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":153,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-10-06T04:49:15.303291+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T04:49:15.303291+0000","last_peered":"2023-10-06T04:49:15.303291+0000","last_clean":"2023-10-06T04:49:15.303291+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T04:49:15.303291+0000","last_undegraded":"2023-10-06T04:49:15.303291+0000","last_fullsized":"2023-10-06T04:49:15.303291+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332841+0000","last_change":"2023-10-06T04:49:20.327658+0000","last_active":"2023-10-06T04:49:20.332841+0000","last_peered":"2023-10-06T04:49:20.332841+0000","last_clean":"2023-10-06T04:49:20.332841+0000","last_became_active":"2023-10-06T04:49:20.327377+0000","last_became_peered":"2023-10-06T04:49:20.327377+0000","last_unstale":"2023-10-06T04:49:20.332841+0000","last_undegraded":"2023-10-06T04:49:20.332841+0000","last_fullsized":"2023-10-06T04:49:20.332841+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"172'8","reported_seq":18,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.351338+0000","last_change":"2023-10-06T04:49:19.329982+0000","last_active":"2023-10-06T04:49:20.351338+0000","last_peered":"2023-10-06T04:49:20.351338+0000","last_clean":"2023-10-06T04:49:20.351338+0000","last_became_active":"2023-10-06T04:49:19.329677+0000","last_became_peered":"2023-10-06T04:49:19.329677+0000","last_unstale":"2023-10-06T04:49:20.351338+0000","last_undegraded":"2023-10-06T04:49:20.351338+0000","last_fullsized":"2023-10-06T04:49:20.351338+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":143,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.316165+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T04:49:20.316165+0000","last_peered":"2023-10-06T04:49:20.316165+0000","last_clean":"2023-10-06T04:49:20.316165+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T04:49:20.316165+0000","last_undegraded":"2023-10-06T04:49:20.316165+0000","last_fullsized":"2023-10-06T04:49:20.316165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.330157+0000","last_change":"2023-10-06T04:49:19.330514+0000","last_active":"2023-10-06T04:49:20.330157+0000","last_peered":"2023-10-06T04:49:20.330157+0000","last_clean":"2023-10-06T04:49:20.330157+0000","last_became_active":"2023-10-06T04:49:19.330226+0000","last_became_peered":"2023-10-06T04:49:19.330226+0000","last_unstale":"2023-10-06T04:49:20.330157+0000","last_undegraded":"2023-10-06T04:49:20.330157+0000","last_fullsized":"2023-10-06T04:49:20.330157+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:55.142830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":144,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.319996+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T04:49:20.319996+0000","last_peered":"2023-10-06T04:49:20.319996+0000","last_clean":"2023-10-06T04:49:20.319996+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T04:49:20.319996+0000","last_undegraded":"2023-10-06T04:49:20.319996+0000","last_fullsized":"2023-10-06T04:49:20.319996+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"163'293","reported_seq":919,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331027+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T04:49:20.331027+0000","last_peered":"2023-10-06T04:49:20.331027+0000","last_clean":"2023-10-06T04:49:20.331027+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T04:49:20.331027+0000","last_undegraded":"2023-10-06T04:49:20.331027+0000","last_fullsized":"2023-10-06T04:49:20.331027+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.336779+0000","last_change":"2023-10-06T04:49:20.335358+0000","last_active":"2023-10-06T04:49:20.336779+0000","last_peered":"2023-10-06T04:49:20.336779+0000","last_clean":"2023-10-06T04:49:20.336779+0000","last_became_active":"2023-10-06T04:49:20.335073+0000","last_became_peered":"2023-10-06T04:49:20.335073+0000","last_unstale":"2023-10-06T04:49:20.336779+0000","last_undegraded":"2023-10-06T04:49:20.336779+0000","last_fullsized":"2023-10-06T04:49:20.336779+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:05.280129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.330408+0000","last_change":"2023-10-06T04:49:19.331617+0000","last_active":"2023-10-06T04:49:20.330408+0000","last_peered":"2023-10-06T04:49:20.330408+0000","last_clean":"2023-10-06T04:49:20.330408+0000","last_became_active":"2023-10-06T04:49:19.329690+0000","last_became_peered":"2023-10-06T04:49:19.329690+0000","last_unstale":"2023-10-06T04:49:20.330408+0000","last_undegraded":"2023-10-06T04:49:20.330408+0000","last_fullsized":"2023-10-06T04:49:20.330408+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:04.983797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":146,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.325305+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T04:49:20.325305+0000","last_peered":"2023-10-06T04:49:20.325305+0000","last_clean":"2023-10-06T04:49:20.325305+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T04:49:20.325305+0000","last_undegraded":"2023-10-06T04:49:20.325305+0000","last_fullsized":"2023-10-06T04:49:20.325305+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"172'6","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356034+0000","last_change":"2023-10-06T04:49:19.331731+0000","last_active":"2023-10-06T04:49:20.356034+0000","last_peered":"2023-10-06T04:49:20.356034+0000","last_clean":"2023-10-06T04:49:20.356034+0000","last_became_active":"2023-10-06T04:49:19.331458+0000","last_became_peered":"2023-10-06T04:49:19.331458+0000","last_unstale":"2023-10-06T04:49:20.356034+0000","last_undegraded":"2023-10-06T04:49:20.356034+0000","last_fullsized":"2023-10-06T04:49:20.356034+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:55.142830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":146,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331105+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T04:49:20.331105+0000","last_peered":"2023-10-06T04:49:20.331105+0000","last_clean":"2023-10-06T04:49:20.331105+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T04:49:20.331105+0000","last_undegraded":"2023-10-06T04:49:20.331105+0000","last_fullsized":"2023-10-06T04:49:20.331105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347917+0000","last_change":"2023-10-06T04:49:20.345754+0000","last_active":"2023-10-06T04:49:20.347917+0000","last_peered":"2023-10-06T04:49:20.347917+0000","last_clean":"2023-10-06T04:49:20.347917+0000","last_became_active":"2023-10-06T04:49:20.345446+0000","last_became_peered":"2023-10-06T04:49:20.345446+0000","last_unstale":"2023-10-06T04:49:20.347917+0000","last_undegraded":"2023-10-06T04:49:20.347917+0000","last_fullsized":"2023-10-06T04:49:20.347917+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:00.217906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"172'3","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.342594+0000","last_change":"2023-10-06T04:49:19.327118+0000","last_active":"2023-10-06T04:49:20.342594+0000","last_peered":"2023-10-06T04:49:20.342594+0000","last_clean":"2023-10-06T04:49:20.342594+0000","last_became_active":"2023-10-06T04:49:19.326839+0000","last_became_peered":"2023-10-06T04:49:19.326839+0000","last_unstale":"2023-10-06T04:49:20.342594+0000","last_undegraded":"2023-10-06T04:49:20.342594+0000","last_fullsized":"2023-10-06T04:49:20.342594+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:17.629867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":139,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.335923+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T04:49:20.335923+0000","last_peered":"2023-10-06T04:49:20.335923+0000","last_clean":"2023-10-06T04:49:20.335923+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T04:49:20.335923+0000","last_undegraded":"2023-10-06T04:49:20.335923+0000","last_fullsized":"2023-10-06T04:49:20.335923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341195+0000","last_change":"2023-10-06T04:49:20.338084+0000","last_active":"2023-10-06T04:49:20.341195+0000","last_peered":"2023-10-06T04:49:20.341195+0000","last_clean":"2023-10-06T04:49:20.341195+0000","last_became_active":"2023-10-06T04:49:20.337738+0000","last_became_peered":"2023-10-06T04:49:20.337738+0000","last_unstale":"2023-10-06T04:49:20.341195+0000","last_undegraded":"2023-10-06T04:49:20.341195+0000","last_fullsized":"2023-10-06T04:49:20.341195+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:28.172716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321149+0000","last_change":"2023-10-06T04:49:19.332809+0000","last_active":"2023-10-06T04:49:20.321149+0000","last_peered":"2023-10-06T04:49:20.321149+0000","last_clean":"2023-10-06T04:49:20.321149+0000","last_became_active":"2023-10-06T04:49:19.331892+0000","last_became_peered":"2023-10-06T04:49:19.331892+0000","last_unstale":"2023-10-06T04:49:20.321149+0000","last_undegraded":"2023-10-06T04:49:20.321149+0000","last_fullsized":"2023-10-06T04:49:20.321149+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:52.469578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320842+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T04:49:20.320842+0000","last_peered":"2023-10-06T04:49:20.320842+0000","last_clean":"2023-10-06T04:49:20.320842+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T04:49:20.320842+0000","last_undegraded":"2023-10-06T04:49:20.320842+0000","last_fullsized":"2023-10-06T04:49:20.320842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:49:19.299792+0000","last_change":"2023-10-06T04:49:19.299792+0000","last_active":"2023-10-06T04:49:19.299792+0000","last_peered":"2023-10-06T04:49:19.299792+0000","last_clean":"2023-10-06T04:49:19.299792+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:49:19.299792+0000","last_undegraded":"2023-10-06T04:49:19.299792+0000","last_fullsized":"2023-10-06T04:49:19.299792+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344451+0000","last_change":"2023-10-06T04:49:19.321982+0000","last_active":"2023-10-06T04:49:20.344451+0000","last_peered":"2023-10-06T04:49:20.344451+0000","last_clean":"2023-10-06T04:49:20.344451+0000","last_became_active":"2023-10-06T04:49:19.321679+0000","last_became_peered":"2023-10-06T04:49:19.321679+0000","last_unstale":"2023-10-06T04:49:20.344451+0000","last_undegraded":"2023-10-06T04:49:20.344451+0000","last_fullsized":"2023-10-06T04:49:20.344451+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320746+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T04:49:20.320746+0000","last_peered":"2023-10-06T04:49:20.320746+0000","last_clean":"2023-10-06T04:49:20.320746+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T04:49:20.320746+0000","last_undegraded":"2023-10-06T04:49:20.320746+0000","last_fullsized":"2023-10-06T04:49:20.320746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":108,"acting":108,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575752,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299976,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071864,"statfs":{"total":96636764160,"available":96329588736,"internally_reserved":0,"allocated":2752512,"data_stored":1720883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69799999999999995,"15min":0.73899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.89500000000000002,"5min":0.97199999999999998,"15min":1.4930000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.48599999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90900000000000003,"5min":0.995,"15min":1.417},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.755,"15min":0.79500000000000004},"min":{"1min":0.53500000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93100000000000005,"5min":1.01,"15min":1.5309999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77600000000000002,"15min":0.79300000000000004},"min":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.58299999999999996},"max":{"1min":0.93600000000000005,"5min":1.0289999999999999,"15min":1.097},"last":0.81000000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70899999999999996,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.46400000000000002},"max":{"1min":0.92300000000000004,"5min":0.97999999999999998,"15min":1.5429999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.751,"15min":0.77100000000000002},"min":{"1min":0.60199999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0189999999999999},"last":0.82999999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.75700000000000001,"15min":0.77500000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.91000000000000003,"5min":1.03,"15min":1.5860000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73299999999999998,"15min":0.76400000000000001},"min":{"1min":0.64200000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.96099999999999997}]},{"osd":4,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63400000000000001,"15min":0.61899999999999999},"min":{"1min":0.50800000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.0640000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.439,"15min":0.42799999999999999},"max":{"1min":1.1339999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.90400000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.53800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.55200000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.72999999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.64200000000000002,"5min":0.501,"15min":0.49199999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":1.1020000000000001,"15min":1.2090000000000001},"last":0.93899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575753,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299400,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072440,"statfs":{"total":96636764160,"available":96330178560,"internally_reserved":0,"allocated":2162688,"data_stored":1131541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.46000000000000002,"5min":0.40300000000000002,"15min":0.375},"max":{"1min":0.96599999999999997,"5min":1.038,"15min":1.038},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76200000000000001,"15min":0.752},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94699999999999995,"5min":0.999,"15min":1.228},"last":0.80000000000000004}]},{"osd":1,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.5,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93799999999999994,"5min":1.046,"15min":1.3660000000000001},"last":0.63},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.748,"15min":0.751},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.438},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.3560000000000001},"last":0.76500000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.47999999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.272},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76900000000000002,"15min":0.77000000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98799999999999999,"5min":1.069,"15min":1.089},"last":0.66300000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94399999999999995,"5min":1.042,"15min":1.1060000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.52000000000000002},"max":{"1min":1.0009999999999999,"5min":1.0409999999999999,"15min":1.3120000000000001},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83499999999999996,"5min":0.88500000000000001,"15min":0.92600000000000005},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.627},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":1.155},"last":0.84499999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.432},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93799999999999994,"5min":1.056,"15min":1.056},"last":0.82799999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95399999999999996,"5min":1.0580000000000001,"15min":1.3500000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.45800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98599999999999999,"5min":1.024,"15min":1.0389999999999999},"last":0.78300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575752,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312268,"kb_used_data":2692,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059572,"statfs":{"total":96636764160,"available":96317001728,"internally_reserved":0,"allocated":2756608,"data_stored":1721909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.79600000000000004,"5min":0.89000000000000001,"15min":0.93100000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42899999999999999},"max":{"1min":0.90000000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.41399999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63800000000000001,"5min":0.67700000000000005,"15min":0.73299999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.38200000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.97799999999999998},"last":0.46899999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.44,"5min":0.44,"15min":0.437},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.94699999999999995},"last":0.497},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.373},"max":{"1min":0.748,"5min":0.85799999999999998,"15min":0.88900000000000001},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79200000000000004,"15min":0.79500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.46899999999999997},"max":{"1min":1.159,"5min":1.1970000000000001,"15min":1.407},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82599999999999996,"15min":0.83099999999999996},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.57499999999999996},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.84399999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83699999999999997,"15min":0.82699999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.55700000000000005},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81699999999999995,"15min":0.83899999999999997},"min":{"1min":0.66300000000000003,"5min":0.59799999999999998,"15min":0.53200000000000003},"max":{"1min":1.097,"5min":1.1060000000000001,"15min":1.724},"last":0.88300000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.64700000000000002,"5min":0.626,"15min":0.56200000000000006},"max":{"1min":0.97899999999999998,"5min":1.2150000000000001,"15min":1.427},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.55800000000000005},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.93600000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.877,"15min":0.878},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1140000000000001,"5min":1.125,"15min":1.702},"last":0.874},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85799999999999998,"15min":0.872},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0269999999999999,"5min":1.1659999999999999,"15min":1.5920000000000001},"last":0.72199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575754,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299416,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072424,"statfs":{"total":96636764160,"available":96330162176,"internally_reserved":0,"allocated":2179072,"data_stored":1132707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.378,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76600000000000001,"5min":0.81699999999999995,"15min":1.6459999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.34200000000000003,"15min":0.32900000000000001},"max":{"1min":0.68300000000000005,"5min":0.84099999999999997,"15min":1.4510000000000001},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55400000000000005,"15min":0.57099999999999995},"min":{"1min":0.434,"5min":0.308,"15min":0.308},"max":{"1min":0.74099999999999999,"5min":0.86899999999999999,"15min":1.7},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.373},"max":{"1min":0.81699999999999995,"5min":0.82099999999999995,"15min":1.6240000000000001},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":1.2529999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91300000000000003,"5min":0.99099999999999999,"15min":1.7370000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.86599999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.60399999999999998},"max":{"1min":1.0940000000000001,"5min":1.2769999999999999,"15min":2.3700000000000001},"last":0.873},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.85099999999999998,"15min":0.85999999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.52900000000000003},"max":{"1min":1.111,"5min":1.3069999999999999,"15min":2.1280000000000001},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84099999999999997,"15min":0.85599999999999998},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.57599999999999996},"max":{"1min":1.048,"5min":1.294,"15min":2.1739999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84099999999999997,"15min":0.84799999999999998},"min":{"1min":0.66000000000000003,"5min":0.63300000000000001,"15min":0.57799999999999996},"max":{"1min":1.087,"5min":1.214,"15min":2.2429999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60599999999999998,"5min":0.59299999999999997,"15min":0.53000000000000003},"max":{"1min":1.1160000000000001,"5min":1.204,"15min":1.605},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89600000000000002,"15min":0.88700000000000001},"min":{"1min":0.72899999999999998,"5min":0.65800000000000003,"15min":0.58199999999999996},"max":{"1min":1.181,"5min":1.226,"15min":2.3519999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.86699999999999999,"15min":0.878},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.091,"5min":1.319,"15min":2.1019999999999999},"last":0.746},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.86299999999999999,"15min":0.86799999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0580000000000001,"5min":1.256,"15min":1.9259999999999999},"last":0.79500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575753,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311688,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060152,"statfs":{"total":96636764160,"available":96317595648,"internally_reserved":0,"allocated":2162688,"data_stored":1130605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.311},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.376},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.34200000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.93999999999999995},"last":0.505}]},{"osd":1,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.32000000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.42399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.38300000000000001},"max":{"1min":0.79600000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.371},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.45400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.628,"5min":0.60199999999999998,"15min":0.51700000000000002},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.86399999999999999,"15min":0.88100000000000001},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58599999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.381},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.59199999999999997},"max":{"1min":1.1719999999999999,"5min":1.2210000000000001,"15min":1.4379999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88300000000000001,"15min":0.89200000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.58899999999999997},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.288},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.151,"5min":1.167,"15min":1.2370000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.61499999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86099999999999999,"15min":0.874},"min":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.57499999999999996},"max":{"1min":1.075,"5min":1.3320000000000001,"15min":1.482},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.88500000000000001,"15min":0.89600000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.135,"5min":1.2869999999999999,"15min":1.405},"last":0.755}]}]},{"osd":3,"up_from":13,"seq":55834575753,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311692,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060148,"statfs":{"total":96636764160,"available":96317591552,"internally_reserved":0,"allocated":2166784,"data_stored":1132059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47699999999999998,"15min":0.49399999999999999},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67900000000000005,"5min":0.73899999999999999,"15min":0.86399999999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.83599999999999997},"last":0.501},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.44900000000000001,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.73999999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.40899999999999997},"max":{"1min":0.69699999999999995,"5min":0.79200000000000004,"15min":1.036},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.54000000000000004,"5min":0.47499999999999998,"15min":0.43099999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83799999999999997},"min":{"1min":0.64300000000000002,"5min":0.56000000000000005,"15min":0.47599999999999998},"max":{"1min":1.101,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.495},"max":{"1min":1.0760000000000001,"5min":1.387,"15min":1.399},"last":1.034}]},{"osd":5,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.82599999999999996,"15min":0.82899999999999996},"min":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0049999999999999,"5min":1.2509999999999999,"15min":1.331},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.85799999999999998,"15min":0.85099999999999998},"min":{"1min":0.63600000000000001,"5min":0.58099999999999996,"15min":0.56599999999999995},"max":{"1min":1.0720000000000001,"5min":1.2130000000000001,"15min":1.2330000000000001},"last":1.0549999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.86499999999999999,"15min":0.85999999999999999},"min":{"1min":0.65100000000000002,"5min":0.53500000000000003,"15min":0.52400000000000002},"max":{"1min":1.175,"5min":1.2090000000000001,"15min":1.2490000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.89900000000000002,"15min":0.89400000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1579999999999999,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88600000000000001,"15min":0.88400000000000001},"min":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.58999999999999997},"max":{"1min":1.127,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85399999999999998,"15min":0.85799999999999998},"min":{"1min":0.60499999999999998,"5min":0.58699999999999997,"15min":0.54900000000000004},"max":{"1min":1.069,"5min":1.276,"15min":1.3029999999999999},"last":1.097}]}]},{"osd":4,"up_from":13,"seq":55834575752,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311636,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060204,"statfs":{"total":96636764160,"available":96317648896,"internally_reserved":0,"allocated":2109440,"data_stored":1103372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.71699999999999997,"15min":0.74299999999999999},"min":{"1min":0.47899999999999998,"5min":0.36699999999999999,"15min":0.35599999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.72399999999999998,"15min":0.73499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.76000000000000001,"15min":0.77800000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.72099999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.745,"15min":0.75900000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.76600000000000001,"15min":0.77100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91200000000000003,"5min":1.01,"15min":1.069},"last":0.76500000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.996,"5min":1.002,"15min":1.002},"last":0.90700000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81699999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.871},{"interface":"front","average":{"1min":0.624,"5min":0.65000000000000002,"15min":0.63500000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69099999999999995,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.46000000000000002,"15min":0.42099999999999999},"max":{"1min":0.996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71499999999999997,"15min":0.70199999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0629999999999999,"5min":1.105,"15min":1.105},"last":1.014}]},{"osd":7,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71399999999999997,"15min":0.68799999999999994},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97999999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.75600000000000001,"15min":0.745},"min":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89700000000000002,"5min":1.091,"15min":1.0940000000000001},"last":1.079}]}]},{"osd":5,"up_from":13,"seq":55834575753,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307584,"kb_used_data":2104,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064256,"statfs":{"total":96636764160,"available":96321798144,"internally_reserved":0,"allocated":2154496,"data_stored":1130493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82999999999999996,"5min":1.03,"15min":2.2829999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73599999999999999,"15min":0.751},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91600000000000004,"5min":1.095,"15min":1.3200000000000001},"last":0.67600000000000005}]},{"osd":1,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.751},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.29},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.51400000000000001},"max":{"1min":0.871,"5min":0.98099999999999998,"15min":1.9139999999999999},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.57999999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98199999999999998,"5min":1.0589999999999999,"15min":1.532},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95799999999999996,"5min":1.0169999999999999,"15min":1.121},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.58899999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.40500000000000003,"15min":0.39300000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":2.6989999999999998},"last":0.75},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38},"max":{"1min":0.99399999999999999,"5min":1.028,"15min":1.694},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.1719999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96399999999999997,"5min":1.077,"15min":1.5880000000000001},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.52500000000000002,"5min":0.47499999999999998,"15min":0.40699999999999997},"max":{"1min":1.03,"5min":1.0409999999999999,"15min":1.7390000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.49099999999999999,"15min":0.48599999999999999},"max":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.3380000000000001},"last":0.75800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:49:25.178 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:49:25.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.399+0000 7ff5ef39d700 1 Processor -- start 2023-10-06T04:49:25.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.399+0000 7ff5ef39d700 1 -- start start 2023-10-06T04:49:25.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.400+0000 7ff5ef39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff5e8068aa0 0x7ff5e80b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.400+0000 7ff5ef39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e80b0a00 0x7ff5e815d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.400+0000 7ff5ef39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e815a980 con 0x7ff5e8157270 2023-10-06T04:49:25.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.400+0000 7ff5ef39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e815a050 con 0x7ff5e8068aa0 2023-10-06T04:49:25.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.400+0000 7ff5ef39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e815a330 con 0x7ff5e80b0a00 2023-10-06T04:49:25.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.400+0000 7ff5ee39b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff5e8157270 0x7ff5e8068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46732/0 (socket says 172.21.15.55:46732) 2023-10-06T04:49:25.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.401+0000 7ff5ee39b700 1 -- 172.21.15.55:0/2296425279 learned_addr learned my addr 172.21.15.55:0/2296425279 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:25.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.401+0000 7ff5edb9a700 1 --2- 172.21.15.55:0/2296425279 >> v2:172.21.15.163:3300/0 conn(0x7ff5e8068aa0 0x7ff5e80b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:25.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.401+0000 7ff5eeb9c700 1 --2- 172.21.15.55:0/2296425279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e80b0a00 0x7ff5e815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.401+0000 7ff5ed399700 1 -- 172.21.15.55:0/2296425279 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732585868 0 0) 0x7ff5e815a980 con 0x7ff5e8157270 2023-10-06T04:49:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.401+0000 7ff5ed399700 1 -- 172.21.15.55:0/2296425279 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5d4003570 con 0x7ff5e8157270 2023-10-06T04:49:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.401+0000 7ff5ed399700 1 -- 172.21.15.55:0/2296425279 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1756300785 0 0) 0x7ff5d4003570 con 0x7ff5e8157270 2023-10-06T04:49:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.401+0000 7ff5ed399700 1 -- 172.21.15.55:0/2296425279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e80b0a00 msgr2=0x7ff5e815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.401+0000 7ff5ed399700 1 --2- 172.21.15.55:0/2296425279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e80b0a00 0x7ff5e815d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.401+0000 7ff5ed399700 1 -- 172.21.15.55:0/2296425279 >> v2:172.21.15.163:3300/0 conn(0x7ff5e8068aa0 msgr2=0x7ff5e80b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:25.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.401+0000 7ff5ed399700 1 --2- 172.21.15.55:0/2296425279 >> v2:172.21.15.163:3300/0 conn(0x7ff5e8068aa0 0x7ff5e80b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.401+0000 7ff5ed399700 1 -- 172.21.15.55:0/2296425279 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e815a610 con 0x7ff5e8157270 2023-10-06T04:49:25.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.401+0000 7ff5ed399700 1 -- 172.21.15.55:0/2296425279 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7ff5e4001470 con 0x7ff5e8157270 2023-10-06T04:49:25.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.401+0000 7ff5ed399700 1 -- 172.21.15.55:0/2296425279 >> v1:172.21.15.55:6789/0 conn(0x7ff5e8157270 legacy=0x7ff5e8068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:25.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5ed399700 1 --2- 172.21.15.55:0/2296425279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d4003ae0 0x7ff5d4003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5ed399700 1 --2- 172.21.15.55:0/2296425279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e80b0a00 0x7ff5d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5ed399700 1 --2- 172.21.15.55:0/2296425279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5d4004d50 0x7ff5d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5ed399700 1 -- 172.21.15.55:0/2296425279 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e815a050 con 0x7ff5d4003ae0 2023-10-06T04:49:25.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5ed399700 1 -- 172.21.15.55:0/2296425279 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e815a330 con 0x7ff5d4004d50 2023-10-06T04:49:25.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5ed399700 1 -- 172.21.15.55:0/2296425279 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d400bb10 con 0x7ff5e80b0a00 2023-10-06T04:49:25.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5ee39b700 1 --2- 172.21.15.55:0/2296425279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d4003ae0 0x7ff5d4003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:25.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5edb9a700 1 --2- 172.21.15.55:0/2296425279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e80b0a00 0x7ff5d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:25.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5eeb9c700 1 --2- 172.21.15.55:0/2296425279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5d4004d50 0x7ff5d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:25.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5ee39b700 1 -- 172.21.15.55:0/2296425279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e80b0a00 msgr2=0x7ff5d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:25.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5ee39b700 1 --2- 172.21.15.55:0/2296425279 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e80b0a00 0x7ff5d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5ee39b700 1 -- 172.21.15.55:0/2296425279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5d4004d50 msgr2=0x7ff5d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:25.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5ee39b700 1 --2- 172.21.15.55:0/2296425279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5d4004d50 0x7ff5d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5ee39b700 1 -- 172.21.15.55:0/2296425279 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d4003570 con 0x7ff5d4003ae0 2023-10-06T04:49:25.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.402+0000 7ff5eeb9c700 1 --2- 172.21.15.55:0/2296425279 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5d4004d50 0x7ff5d40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:49:25.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.403+0000 7ff5ee39b700 1 --2- 172.21.15.55:0/2296425279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d4003ae0 0x7ff5d4003ef0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400c300 tx=0x7ff5e4001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475ff1133a19b84d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:25.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.403+0000 7ff5ed399700 1 -- 172.21.15.55:0/2296425279 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400cb90 con 0x7ff5d4003ae0 2023-10-06T04:49:25.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.403+0000 7ff5ed399700 1 -- 172.21.15.55:0/2296425279 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e400ccf0 con 0x7ff5d4003ae0 2023-10-06T04:49:25.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.403+0000 7ff5ed399700 1 -- 172.21.15.55:0/2296425279 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4005b70 con 0x7ff5d4003ae0 2023-10-06T04:49:25.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.403+0000 7ff5ef39d700 1 -- 172.21.15.55:0/2296425279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d4003ae0 msgr2=0x7ff5d4003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:25.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.403+0000 7ff5ef39d700 1 --2- 172.21.15.55:0/2296425279 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d4003ae0 0x7ff5d4003ef0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400c300 tx=0x7ff5e4001dc0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.403+0000 7ff5eeb9c700 1 -- 172.21.15.55:0/2296425279 reap_dead start 2023-10-06T04:49:25.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.403+0000 7ff5ef39d700 1 -- 172.21.15.55:0/2296425279 shutdown_connections 2023-10-06T04:49:25.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.403+0000 7ff5ef39d700 1 -- 172.21.15.55:0/2296425279 >> 172.21.15.55:0/2296425279 conn(0x7ff5e80cd310 msgr2=0x7ff5e80cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:25.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.403+0000 7ff5ef39d700 1 -- 172.21.15.55:0/2296425279 shutdown_connections 2023-10-06T04:49:25.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.403+0000 7ff5ef39d700 1 -- 172.21.15.55:0/2296425279 wait complete. 2023-10-06T04:49:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.405+0000 7ff5ef39d700 1 Processor -- start 2023-10-06T04:49:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.405+0000 7ff5ef39d700 1 -- start start 2023-10-06T04:49:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.406+0000 7ff5ef39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d4003ae0 0x7ff5e8155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.406+0000 7ff5ef39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e8152220 0x7ff5e81550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.406+0000 7ff5ef39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e80c38e0 0x7ff5e80c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.406+0000 7ff5ef39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80c4290 con 0x7ff5d4003ae0 2023-10-06T04:49:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.406+0000 7ff5ef39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e8153000 con 0x7ff5e80c38e0 2023-10-06T04:49:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.406+0000 7ff5ef39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e8152690 con 0x7ff5e8152220 2023-10-06T04:49:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.406+0000 7ff5ee39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d4003ae0 0x7ff5e8155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:25.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.406+0000 7ff5ee39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d4003ae0 0x7ff5e8155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34474/0 (socket says 172.21.15.55:34474) 2023-10-06T04:49:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.406+0000 7ff5ee39b700 1 -- 172.21.15.55:0/3845652513 learned_addr learned my addr 172.21.15.55:0/3845652513 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.407+0000 7ff5ee39b700 1 -- 172.21.15.55:0/3845652513 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e8152220 msgr2=0x7ff5e81550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.407+0000 7ff5ee39b700 1 --2- 172.21.15.55:0/3845652513 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e8152220 0x7ff5e81550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.407+0000 7ff5ee39b700 1 -- 172.21.15.55:0/3845652513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e80c38e0 msgr2=0x7ff5e80c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.407+0000 7ff5ee39b700 1 --2- 172.21.15.55:0/3845652513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e80c38e0 0x7ff5e80c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.407+0000 7ff5ee39b700 1 -- 172.21.15.55:0/3845652513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e80c2a70 con 0x7ff5d4003ae0 2023-10-06T04:49:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.407+0000 7ff5ee39b700 1 --2- 172.21.15.55:0/3845652513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d4003ae0 0x7ff5e8155d40 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400c8f0 tx=0x7ff5e4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.407+0000 7ff5de7fc700 1 -- 172.21.15.55:0/3845652513 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400ce60 con 0x7ff5d4003ae0 2023-10-06T04:49:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.407+0000 7ff5de7fc700 1 -- 172.21.15.55:0/3845652513 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e40072e0 con 0x7ff5d4003ae0 2023-10-06T04:49:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.407+0000 7ff5de7fc700 1 -- 172.21.15.55:0/3845652513 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e40076a0 con 0x7ff5d4003ae0 2023-10-06T04:49:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.407+0000 7ff5ef39d700 1 -- 172.21.15.55:0/3845652513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e8155610 con 0x7ff5d4003ae0 2023-10-06T04:49:25.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.407+0000 7ff5ef39d700 1 -- 172.21.15.55:0/3845652513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e80c2ff0 con 0x7ff5d4003ae0 2023-10-06T04:49:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.408+0000 7ff5ef39d700 1 -- 172.21.15.55:0/3845652513 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e804c3c0 con 0x7ff5d4003ae0 2023-10-06T04:49:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.409+0000 7ff5de7fc700 1 -- 172.21.15.55:0/3845652513 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5e4019030 con 0x7ff5d4003ae0 2023-10-06T04:49:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.410+0000 7ff5de7fc700 1 --2- 172.21.15.55:0/3845652513 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ff5cc066f30 0x7ff5cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.410+0000 7ff5de7fc700 1 -- 172.21.15.55:0/3845652513 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7ff5e40859a0 con 0x7ff5d4003ae0 2023-10-06T04:49:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.410+0000 7ff5edb9a700 1 --2- 172.21.15.55:0/3845652513 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ff5cc066f30 0x7ff5cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:25.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.411+0000 7ff5edb9a700 1 --2- 172.21.15.55:0/3845652513 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ff5cc066f30 0x7ff5cc0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff5e816e1f0 tx=0x7ff5d80091a0 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:25.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.414+0000 7ff5de7fc700 1 -- 172.21.15.55:0/3845652513 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5e4052880 con 0x7ff5d4003ae0 2023-10-06T04:49:25.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.611+0000 7ff5ef39d700 1 -- 172.21.15.55:0/3845652513 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5e81558e0 con 0x7ff5cc066f30 2023-10-06T04:49:25.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.621+0000 7ff5de7fc700 1 -- 172.21.15.55:0/3845652513 <== mgr.8557 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311431 (secure 0 0 0) 0x7ff5d8011740 con 0x7ff5cc066f30 2023-10-06T04:49:25.627 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:25.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.625+0000 7ff5ef39d700 1 -- 172.21.15.55:0/3845652513 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ff5cc066f30 msgr2=0x7ff5cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:25.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.625+0000 7ff5ef39d700 1 --2- 172.21.15.55:0/3845652513 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ff5cc066f30 0x7ff5cc0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff5e816e1f0 tx=0x7ff5d80091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.625+0000 7ff5ef39d700 1 -- 172.21.15.55:0/3845652513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d4003ae0 msgr2=0x7ff5e8155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:25.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.625+0000 7ff5ef39d700 1 --2- 172.21.15.55:0/3845652513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d4003ae0 0x7ff5e8155d40 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400c8f0 tx=0x7ff5e4012000 comp rx=0 tx=0).stop 2023-10-06T04:49:25.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.625+0000 7ff5ef39d700 1 -- 172.21.15.55:0/3845652513 shutdown_connections 2023-10-06T04:49:25.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.625+0000 7ff5ef39d700 1 --2- 172.21.15.55:0/3845652513 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7ff5cc066f30 0x7ff5cc0693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.625+0000 7ff5ef39d700 1 --2- 172.21.15.55:0/3845652513 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff5e80c38e0 0x7ff5e80c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.625+0000 7ff5ef39d700 1 --2- 172.21.15.55:0/3845652513 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff5d4003ae0 0x7ff5e8155d40 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.625+0000 7ff5ef39d700 1 --2- 172.21.15.55:0/3845652513 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff5e8152220 0x7ff5e81550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.625+0000 7ff5ef39d700 1 -- 172.21.15.55:0/3845652513 >> 172.21.15.55:0/3845652513 conn(0x7ff5e80cd310 msgr2=0x7ff5e806bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:25.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.625+0000 7ff5ef39d700 1 -- 172.21.15.55:0/3845652513 shutdown_connections 2023-10-06T04:49:25.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.626+0000 7ff5ef39d700 1 -- 172.21.15.55:0/3845652513 wait complete. 2023-10-06T04:49:25.631 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T04:49:25.645 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T04:49:25.337829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":523,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2453720,"kb_used_data":18072,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752521000,"statfs":{"total":773094113280,"available":770581504000,"internally_reserved":0,"allocated":18505728,"data_stored":10233204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.027170"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346440+0000","last_change":"2023-10-06T04:49:20.341012+0000","last_active":"2023-10-06T04:49:20.346440+0000","last_peered":"2023-10-06T04:49:20.346440+0000","last_clean":"2023-10-06T04:49:20.346440+0000","last_became_active":"2023-10-06T04:49:20.340740+0000","last_became_peered":"2023-10-06T04:49:20.340740+0000","last_unstale":"2023-10-06T04:49:20.346440+0000","last_undegraded":"2023-10-06T04:49:20.346440+0000","last_fullsized":"2023-10-06T04:49:20.346440+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341092+0000","last_change":"2023-10-06T04:49:20.336745+0000","last_active":"2023-10-06T04:49:20.341092+0000","last_peered":"2023-10-06T04:49:20.341092+0000","last_clean":"2023-10-06T04:49:20.341092+0000","last_became_active":"2023-10-06T04:49:20.336500+0000","last_became_peered":"2023-10-06T04:49:20.336500+0000","last_unstale":"2023-10-06T04:49:20.341092+0000","last_undegraded":"2023-10-06T04:49:20.341092+0000","last_fullsized":"2023-10-06T04:49:20.341092+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:28.214277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341469+0000","last_change":"2023-10-06T04:49:20.334944+0000","last_active":"2023-10-06T04:49:20.341469+0000","last_peered":"2023-10-06T04:49:20.341469+0000","last_clean":"2023-10-06T04:49:20.341469+0000","last_became_active":"2023-10-06T04:49:20.333924+0000","last_became_peered":"2023-10-06T04:49:20.333924+0000","last_unstale":"2023-10-06T04:49:20.341469+0000","last_undegraded":"2023-10-06T04:49:20.341469+0000","last_fullsized":"2023-10-06T04:49:20.341469+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341150+0000","last_change":"2023-10-06T04:49:20.333903+0000","last_active":"2023-10-06T04:49:20.341150+0000","last_peered":"2023-10-06T04:49:20.341150+0000","last_clean":"2023-10-06T04:49:20.341150+0000","last_became_active":"2023-10-06T04:49:20.333572+0000","last_became_peered":"2023-10-06T04:49:20.333572+0000","last_unstale":"2023-10-06T04:49:20.341150+0000","last_undegraded":"2023-10-06T04:49:20.341150+0000","last_fullsized":"2023-10-06T04:49:20.341150+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:49.977768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339683+0000","last_change":"2023-10-06T04:49:20.338306+0000","last_active":"2023-10-06T04:49:20.339683+0000","last_peered":"2023-10-06T04:49:20.339683+0000","last_clean":"2023-10-06T04:49:20.339683+0000","last_became_active":"2023-10-06T04:49:20.338034+0000","last_became_peered":"2023-10-06T04:49:20.338034+0000","last_unstale":"2023-10-06T04:49:20.339683+0000","last_undegraded":"2023-10-06T04:49:20.339683+0000","last_fullsized":"2023-10-06T04:49:20.339683+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339447+0000","last_change":"2023-10-06T04:49:20.336417+0000","last_active":"2023-10-06T04:49:20.339447+0000","last_peered":"2023-10-06T04:49:20.339447+0000","last_clean":"2023-10-06T04:49:20.339447+0000","last_became_active":"2023-10-06T04:49:20.334348+0000","last_became_peered":"2023-10-06T04:49:20.334348+0000","last_unstale":"2023-10-06T04:49:20.339447+0000","last_undegraded":"2023-10-06T04:49:20.339447+0000","last_fullsized":"2023-10-06T04:49:20.339447+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:54.036275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.345389+0000","last_change":"2023-10-06T04:49:20.339825+0000","last_active":"2023-10-06T04:49:20.345389+0000","last_peered":"2023-10-06T04:49:20.345389+0000","last_clean":"2023-10-06T04:49:20.345389+0000","last_became_active":"2023-10-06T04:49:20.339565+0000","last_became_peered":"2023-10-06T04:49:20.339565+0000","last_unstale":"2023-10-06T04:49:20.345389+0000","last_undegraded":"2023-10-06T04:49:20.345389+0000","last_fullsized":"2023-10-06T04:49:20.345389+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:54.036275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347030+0000","last_change":"2023-10-06T04:49:20.336205+0000","last_active":"2023-10-06T04:49:20.347030+0000","last_peered":"2023-10-06T04:49:20.347030+0000","last_clean":"2023-10-06T04:49:20.347030+0000","last_became_active":"2023-10-06T04:49:20.334936+0000","last_became_peered":"2023-10-06T04:49:20.334936+0000","last_unstale":"2023-10-06T04:49:20.347030+0000","last_undegraded":"2023-10-06T04:49:20.347030+0000","last_fullsized":"2023-10-06T04:49:20.347030+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:34.988176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.340009+0000","last_change":"2023-10-06T04:49:20.336646+0000","last_active":"2023-10-06T04:49:20.340009+0000","last_peered":"2023-10-06T04:49:20.340009+0000","last_clean":"2023-10-06T04:49:20.340009+0000","last_became_active":"2023-10-06T04:49:20.334441+0000","last_became_peered":"2023-10-06T04:49:20.334441+0000","last_unstale":"2023-10-06T04:49:20.340009+0000","last_undegraded":"2023-10-06T04:49:20.340009+0000","last_fullsized":"2023-10-06T04:49:20.340009+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:23.582332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333816+0000","last_change":"2023-10-06T04:49:20.331180+0000","last_active":"2023-10-06T04:49:20.333816+0000","last_peered":"2023-10-06T04:49:20.333816+0000","last_clean":"2023-10-06T04:49:20.333816+0000","last_became_active":"2023-10-06T04:49:20.330817+0000","last_became_peered":"2023-10-06T04:49:20.330817+0000","last_unstale":"2023-10-06T04:49:20.333816+0000","last_undegraded":"2023-10-06T04:49:20.333816+0000","last_fullsized":"2023-10-06T04:49:20.333816+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.336853+0000","last_change":"2023-10-06T04:49:20.336069+0000","last_active":"2023-10-06T04:49:20.336853+0000","last_peered":"2023-10-06T04:49:20.336853+0000","last_clean":"2023-10-06T04:49:20.336853+0000","last_became_active":"2023-10-06T04:49:20.335808+0000","last_became_peered":"2023-10-06T04:49:20.335808+0000","last_unstale":"2023-10-06T04:49:20.336853+0000","last_undegraded":"2023-10-06T04:49:20.336853+0000","last_fullsized":"2023-10-06T04:49:20.336853+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:53.477421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.334968+0000","last_change":"2023-10-06T04:49:20.328201+0000","last_active":"2023-10-06T04:49:20.334968+0000","last_peered":"2023-10-06T04:49:20.334968+0000","last_clean":"2023-10-06T04:49:20.334968+0000","last_became_active":"2023-10-06T04:49:20.327560+0000","last_became_peered":"2023-10-06T04:49:20.327560+0000","last_unstale":"2023-10-06T04:49:20.334968+0000","last_undegraded":"2023-10-06T04:49:20.334968+0000","last_fullsized":"2023-10-06T04:49:20.334968+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:40.721698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333739+0000","last_change":"2023-10-06T04:49:20.331130+0000","last_active":"2023-10-06T04:49:20.333739+0000","last_peered":"2023-10-06T04:49:20.333739+0000","last_clean":"2023-10-06T04:49:20.333739+0000","last_became_active":"2023-10-06T04:49:20.330637+0000","last_became_peered":"2023-10-06T04:49:20.330637+0000","last_unstale":"2023-10-06T04:49:20.333739+0000","last_undegraded":"2023-10-06T04:49:20.333739+0000","last_fullsized":"2023-10-06T04:49:20.333739+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341277+0000","last_change":"2023-10-06T04:49:20.337139+0000","last_active":"2023-10-06T04:49:20.341277+0000","last_peered":"2023-10-06T04:49:20.341277+0000","last_clean":"2023-10-06T04:49:20.341277+0000","last_became_active":"2023-10-06T04:49:20.336396+0000","last_became_peered":"2023-10-06T04:49:20.336396+0000","last_unstale":"2023-10-06T04:49:20.341277+0000","last_undegraded":"2023-10-06T04:49:20.341277+0000","last_fullsized":"2023-10-06T04:49:20.341277+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:22.496792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341177+0000","last_change":"2023-10-06T04:49:20.337332+0000","last_active":"2023-10-06T04:49:20.341177+0000","last_peered":"2023-10-06T04:49:20.341177+0000","last_clean":"2023-10-06T04:49:20.341177+0000","last_became_active":"2023-10-06T04:49:20.337091+0000","last_became_peered":"2023-10-06T04:49:20.337091+0000","last_unstale":"2023-10-06T04:49:20.341177+0000","last_undegraded":"2023-10-06T04:49:20.341177+0000","last_fullsized":"2023-10-06T04:49:20.341177+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:57.445018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333411+0000","last_change":"2023-10-06T04:49:20.329112+0000","last_active":"2023-10-06T04:49:20.333411+0000","last_peered":"2023-10-06T04:49:20.333411+0000","last_clean":"2023-10-06T04:49:20.333411+0000","last_became_active":"2023-10-06T04:49:20.328156+0000","last_became_peered":"2023-10-06T04:49:20.328156+0000","last_unstale":"2023-10-06T04:49:20.333411+0000","last_undegraded":"2023-10-06T04:49:20.333411+0000","last_fullsized":"2023-10-06T04:49:20.333411+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:25.017277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341591+0000","last_change":"2023-10-06T04:49:20.334020+0000","last_active":"2023-10-06T04:49:20.341591+0000","last_peered":"2023-10-06T04:49:20.341591+0000","last_clean":"2023-10-06T04:49:20.341591+0000","last_became_active":"2023-10-06T04:49:20.333516+0000","last_became_peered":"2023-10-06T04:49:20.333516+0000","last_unstale":"2023-10-06T04:49:20.341591+0000","last_undegraded":"2023-10-06T04:49:20.341591+0000","last_fullsized":"2023-10-06T04:49:20.341591+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:43.552397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347965+0000","last_change":"2023-10-06T04:49:20.346422+0000","last_active":"2023-10-06T04:49:20.347965+0000","last_peered":"2023-10-06T04:49:20.347965+0000","last_clean":"2023-10-06T04:49:20.347965+0000","last_became_active":"2023-10-06T04:49:20.346126+0000","last_became_peered":"2023-10-06T04:49:20.346126+0000","last_unstale":"2023-10-06T04:49:20.347965+0000","last_undegraded":"2023-10-06T04:49:20.347965+0000","last_fullsized":"2023-10-06T04:49:20.347965+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:58.644879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355577+0000","last_change":"2023-10-06T04:49:20.350471+0000","last_active":"2023-10-06T04:49:20.355577+0000","last_peered":"2023-10-06T04:49:20.355577+0000","last_clean":"2023-10-06T04:49:20.355577+0000","last_became_active":"2023-10-06T04:49:20.350195+0000","last_became_peered":"2023-10-06T04:49:20.350195+0000","last_unstale":"2023-10-06T04:49:20.355577+0000","last_undegraded":"2023-10-06T04:49:20.355577+0000","last_fullsized":"2023-10-06T04:49:20.355577+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:12.727118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339262+0000","last_change":"2023-10-06T04:49:20.334837+0000","last_active":"2023-10-06T04:49:20.339262+0000","last_peered":"2023-10-06T04:49:20.339262+0000","last_clean":"2023-10-06T04:49:20.339262+0000","last_became_active":"2023-10-06T04:49:20.334580+0000","last_became_peered":"2023-10-06T04:49:20.334580+0000","last_unstale":"2023-10-06T04:49:20.339262+0000","last_undegraded":"2023-10-06T04:49:20.339262+0000","last_fullsized":"2023-10-06T04:49:20.339262+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:02.862072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.338389+0000","last_change":"2023-10-06T04:49:20.334048+0000","last_active":"2023-10-06T04:49:20.338389+0000","last_peered":"2023-10-06T04:49:20.338389+0000","last_clean":"2023-10-06T04:49:20.338389+0000","last_became_active":"2023-10-06T04:49:20.333761+0000","last_became_peered":"2023-10-06T04:49:20.333761+0000","last_unstale":"2023-10-06T04:49:20.338389+0000","last_undegraded":"2023-10-06T04:49:20.338389+0000","last_fullsized":"2023-10-06T04:49:20.338389+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344972+0000","last_change":"2023-10-06T04:49:20.336537+0000","last_active":"2023-10-06T04:49:20.344972+0000","last_peered":"2023-10-06T04:49:20.344972+0000","last_clean":"2023-10-06T04:49:20.344972+0000","last_became_active":"2023-10-06T04:49:20.336251+0000","last_became_peered":"2023-10-06T04:49:20.336251+0000","last_unstale":"2023-10-06T04:49:20.344972+0000","last_undegraded":"2023-10-06T04:49:20.344972+0000","last_fullsized":"2023-10-06T04:49:20.344972+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347191+0000","last_change":"2023-10-06T04:49:20.344026+0000","last_active":"2023-10-06T04:49:20.347191+0000","last_peered":"2023-10-06T04:49:20.347191+0000","last_clean":"2023-10-06T04:49:20.347191+0000","last_became_active":"2023-10-06T04:49:20.343761+0000","last_became_peered":"2023-10-06T04:49:20.343761+0000","last_unstale":"2023-10-06T04:49:20.347191+0000","last_undegraded":"2023-10-06T04:49:20.347191+0000","last_fullsized":"2023-10-06T04:49:20.347191+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:07.881803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.329722+0000","last_change":"2023-10-06T04:49:20.326750+0000","last_active":"2023-10-06T04:49:20.329722+0000","last_peered":"2023-10-06T04:49:20.329722+0000","last_clean":"2023-10-06T04:49:20.329722+0000","last_became_active":"2023-10-06T04:49:20.326466+0000","last_became_peered":"2023-10-06T04:49:20.326466+0000","last_unstale":"2023-10-06T04:49:20.329722+0000","last_undegraded":"2023-10-06T04:49:20.329722+0000","last_fullsized":"2023-10-06T04:49:20.329722+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:52.748607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.345875+0000","last_change":"2023-10-06T04:49:20.337383+0000","last_active":"2023-10-06T04:49:20.345875+0000","last_peered":"2023-10-06T04:49:20.345875+0000","last_clean":"2023-10-06T04:49:20.345875+0000","last_became_active":"2023-10-06T04:49:20.337130+0000","last_became_peered":"2023-10-06T04:49:20.337130+0000","last_unstale":"2023-10-06T04:49:20.345875+0000","last_undegraded":"2023-10-06T04:49:20.345875+0000","last_fullsized":"2023-10-06T04:49:20.345875+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355403+0000","last_change":"2023-10-06T04:49:20.347903+0000","last_active":"2023-10-06T04:49:20.355403+0000","last_peered":"2023-10-06T04:49:20.355403+0000","last_clean":"2023-10-06T04:49:20.355403+0000","last_became_active":"2023-10-06T04:49:20.346956+0000","last_became_peered":"2023-10-06T04:49:20.346956+0000","last_unstale":"2023-10-06T04:49:20.355403+0000","last_undegraded":"2023-10-06T04:49:20.355403+0000","last_fullsized":"2023-10-06T04:49:20.355403+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:20.026936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339401+0000","last_change":"2023-10-06T04:49:20.336472+0000","last_active":"2023-10-06T04:49:20.339401+0000","last_peered":"2023-10-06T04:49:20.339401+0000","last_clean":"2023-10-06T04:49:20.339401+0000","last_became_active":"2023-10-06T04:49:20.334373+0000","last_became_peered":"2023-10-06T04:49:20.334373+0000","last_unstale":"2023-10-06T04:49:20.339401+0000","last_undegraded":"2023-10-06T04:49:20.339401+0000","last_fullsized":"2023-10-06T04:49:20.339401+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:20.026936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346959+0000","last_change":"2023-10-06T04:49:20.344850+0000","last_active":"2023-10-06T04:49:20.346959+0000","last_peered":"2023-10-06T04:49:20.346959+0000","last_clean":"2023-10-06T04:49:20.346959+0000","last_became_active":"2023-10-06T04:49:20.343891+0000","last_became_peered":"2023-10-06T04:49:20.343891+0000","last_unstale":"2023-10-06T04:49:20.346959+0000","last_undegraded":"2023-10-06T04:49:20.346959+0000","last_fullsized":"2023-10-06T04:49:20.346959+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:28.957190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347052+0000","last_change":"2023-10-06T04:49:20.343469+0000","last_active":"2023-10-06T04:49:20.347052+0000","last_peered":"2023-10-06T04:49:20.347052+0000","last_clean":"2023-10-06T04:49:20.347052+0000","last_became_active":"2023-10-06T04:49:20.343165+0000","last_became_peered":"2023-10-06T04:49:20.343165+0000","last_unstale":"2023-10-06T04:49:20.347052+0000","last_undegraded":"2023-10-06T04:49:20.347052+0000","last_fullsized":"2023-10-06T04:49:20.347052+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:12.727118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347992+0000","last_change":"2023-10-06T04:49:20.344409+0000","last_active":"2023-10-06T04:49:20.347992+0000","last_peered":"2023-10-06T04:49:20.347992+0000","last_clean":"2023-10-06T04:49:20.347992+0000","last_became_active":"2023-10-06T04:49:20.343734+0000","last_became_peered":"2023-10-06T04:49:20.343734+0000","last_unstale":"2023-10-06T04:49:20.347992+0000","last_undegraded":"2023-10-06T04:49:20.347992+0000","last_fullsized":"2023-10-06T04:49:20.347992+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:27.555498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333496+0000","last_change":"2023-10-06T04:49:20.327245+0000","last_active":"2023-10-06T04:49:20.333496+0000","last_peered":"2023-10-06T04:49:20.333496+0000","last_clean":"2023-10-06T04:49:20.333496+0000","last_became_active":"2023-10-06T04:49:20.326914+0000","last_became_peered":"2023-10-06T04:49:20.326914+0000","last_unstale":"2023-10-06T04:49:20.333496+0000","last_undegraded":"2023-10-06T04:49:20.333496+0000","last_fullsized":"2023-10-06T04:49:20.333496+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:19.514355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347097+0000","last_change":"2023-10-06T04:49:20.339379+0000","last_active":"2023-10-06T04:49:20.347097+0000","last_peered":"2023-10-06T04:49:20.347097+0000","last_clean":"2023-10-06T04:49:20.347097+0000","last_became_active":"2023-10-06T04:49:20.339031+0000","last_became_peered":"2023-10-06T04:49:20.339031+0000","last_unstale":"2023-10-06T04:49:20.347097+0000","last_undegraded":"2023-10-06T04:49:20.347097+0000","last_fullsized":"2023-10-06T04:49:20.347097+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:51.900026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341007+0000","last_change":"2023-10-06T04:49:20.333556+0000","last_active":"2023-10-06T04:49:20.341007+0000","last_peered":"2023-10-06T04:49:20.341007+0000","last_clean":"2023-10-06T04:49:20.341007+0000","last_became_active":"2023-10-06T04:49:20.333251+0000","last_became_peered":"2023-10-06T04:49:20.333251+0000","last_unstale":"2023-10-06T04:49:20.341007+0000","last_undegraded":"2023-10-06T04:49:20.341007+0000","last_fullsized":"2023-10-06T04:49:20.341007+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"172'4","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344554+0000","last_change":"2023-10-06T04:49:19.320190+0000","last_active":"2023-10-06T04:49:20.344554+0000","last_peered":"2023-10-06T04:49:20.344554+0000","last_clean":"2023-10-06T04:49:20.344554+0000","last_became_active":"2023-10-06T04:49:19.319917+0000","last_became_peered":"2023-10-06T04:49:19.319917+0000","last_unstale":"2023-10-06T04:49:20.344554+0000","last_undegraded":"2023-10-06T04:49:20.344554+0000","last_fullsized":"2023-10-06T04:49:20.344554+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:26.266062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341489+0000","last_change":"2023-10-06T04:49:20.336237+0000","last_active":"2023-10-06T04:49:20.341489+0000","last_peered":"2023-10-06T04:49:20.341489+0000","last_clean":"2023-10-06T04:49:20.341489+0000","last_became_active":"2023-10-06T04:49:20.335986+0000","last_became_peered":"2023-10-06T04:49:20.335986+0000","last_unstale":"2023-10-06T04:49:20.341489+0000","last_undegraded":"2023-10-06T04:49:20.341489+0000","last_fullsized":"2023-10-06T04:49:20.341489+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355736+0000","last_change":"2023-10-06T04:49:19.335778+0000","last_active":"2023-10-06T04:49:20.355736+0000","last_peered":"2023-10-06T04:49:20.355736+0000","last_clean":"2023-10-06T04:49:20.355736+0000","last_became_active":"2023-10-06T04:49:19.335470+0000","last_became_peered":"2023-10-06T04:49:19.335470+0000","last_unstale":"2023-10-06T04:49:20.355736+0000","last_undegraded":"2023-10-06T04:49:20.355736+0000","last_fullsized":"2023-10-06T04:49:20.355736+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:01.854229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347502+0000","last_change":"2023-10-06T04:49:20.339407+0000","last_active":"2023-10-06T04:49:20.347502+0000","last_peered":"2023-10-06T04:49:20.347502+0000","last_clean":"2023-10-06T04:49:20.347502+0000","last_became_active":"2023-10-06T04:49:20.335811+0000","last_became_peered":"2023-10-06T04:49:20.335811+0000","last_unstale":"2023-10-06T04:49:20.347502+0000","last_undegraded":"2023-10-06T04:49:20.347502+0000","last_fullsized":"2023-10-06T04:49:20.347502+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:10.637938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356161+0000","last_change":"2023-10-06T04:49:19.335543+0000","last_active":"2023-10-06T04:49:20.356161+0000","last_peered":"2023-10-06T04:49:20.356161+0000","last_clean":"2023-10-06T04:49:20.356161+0000","last_became_active":"2023-10-06T04:49:19.335265+0000","last_became_peered":"2023-10-06T04:49:19.335265+0000","last_unstale":"2023-10-06T04:49:20.356161+0000","last_undegraded":"2023-10-06T04:49:20.356161+0000","last_fullsized":"2023-10-06T04:49:20.356161+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:41.377516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350856+0000","last_change":"2023-10-06T04:49:20.346902+0000","last_active":"2023-10-06T04:49:20.350856+0000","last_peered":"2023-10-06T04:49:20.350856+0000","last_clean":"2023-10-06T04:49:20.350856+0000","last_became_active":"2023-10-06T04:49:20.346594+0000","last_became_peered":"2023-10-06T04:49:20.346594+0000","last_unstale":"2023-10-06T04:49:20.350856+0000","last_undegraded":"2023-10-06T04:49:20.350856+0000","last_fullsized":"2023-10-06T04:49:20.350856+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:00.206328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.349018+0000","last_change":"2023-10-06T04:49:19.328489+0000","last_active":"2023-10-06T04:49:20.349018+0000","last_peered":"2023-10-06T04:49:20.349018+0000","last_clean":"2023-10-06T04:49:20.349018+0000","last_became_active":"2023-10-06T04:49:19.328220+0000","last_became_peered":"2023-10-06T04:49:19.328220+0000","last_unstale":"2023-10-06T04:49:20.349018+0000","last_undegraded":"2023-10-06T04:49:20.349018+0000","last_fullsized":"2023-10-06T04:49:20.349018+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:26.266062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.337318+0000","last_change":"2023-10-06T04:49:20.330590+0000","last_active":"2023-10-06T04:49:20.337318+0000","last_peered":"2023-10-06T04:49:20.337318+0000","last_clean":"2023-10-06T04:49:20.337318+0000","last_became_active":"2023-10-06T04:49:20.330306+0000","last_became_peered":"2023-10-06T04:49:20.330306+0000","last_unstale":"2023-10-06T04:49:20.337318+0000","last_undegraded":"2023-10-06T04:49:20.337318+0000","last_fullsized":"2023-10-06T04:49:20.337318+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:42.535859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.351406+0000","last_change":"2023-10-06T04:49:19.324990+0000","last_active":"2023-10-06T04:49:20.351406+0000","last_peered":"2023-10-06T04:49:20.351406+0000","last_clean":"2023-10-06T04:49:20.351406+0000","last_became_active":"2023-10-06T04:49:19.324681+0000","last_became_peered":"2023-10-06T04:49:19.324681+0000","last_unstale":"2023-10-06T04:49:20.351406+0000","last_undegraded":"2023-10-06T04:49:20.351406+0000","last_fullsized":"2023-10-06T04:49:20.351406+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:40.637063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341432+0000","last_change":"2023-10-06T04:49:20.334543+0000","last_active":"2023-10-06T04:49:20.341432+0000","last_peered":"2023-10-06T04:49:20.341432+0000","last_clean":"2023-10-06T04:49:20.341432+0000","last_became_active":"2023-10-06T04:49:20.333665+0000","last_became_peered":"2023-10-06T04:49:20.333665+0000","last_unstale":"2023-10-06T04:49:20.341432+0000","last_undegraded":"2023-10-06T04:49:20.341432+0000","last_fullsized":"2023-10-06T04:49:20.341432+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:20.869185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347200+0000","last_change":"2023-10-06T04:49:19.330786+0000","last_active":"2023-10-06T04:49:20.347200+0000","last_peered":"2023-10-06T04:49:20.347200+0000","last_clean":"2023-10-06T04:49:20.347200+0000","last_became_active":"2023-10-06T04:49:19.330506+0000","last_became_peered":"2023-10-06T04:49:19.330506+0000","last_unstale":"2023-10-06T04:49:20.347200+0000","last_undegraded":"2023-10-06T04:49:20.347200+0000","last_fullsized":"2023-10-06T04:49:20.347200+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:54.415706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333277+0000","last_change":"2023-10-06T04:49:20.332521+0000","last_active":"2023-10-06T04:49:20.333277+0000","last_peered":"2023-10-06T04:49:20.333277+0000","last_clean":"2023-10-06T04:49:20.333277+0000","last_became_active":"2023-10-06T04:49:20.332264+0000","last_became_peered":"2023-10-06T04:49:20.332264+0000","last_unstale":"2023-10-06T04:49:20.333277+0000","last_undegraded":"2023-10-06T04:49:20.333277+0000","last_fullsized":"2023-10-06T04:49:20.333277+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:16.543656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332961+0000","last_change":"2023-10-06T04:49:19.332296+0000","last_active":"2023-10-06T04:49:20.332961+0000","last_peered":"2023-10-06T04:49:20.332961+0000","last_clean":"2023-10-06T04:49:20.332961+0000","last_became_active":"2023-10-06T04:49:19.332022+0000","last_became_peered":"2023-10-06T04:49:19.332022+0000","last_unstale":"2023-10-06T04:49:20.332961+0000","last_undegraded":"2023-10-06T04:49:20.332961+0000","last_fullsized":"2023-10-06T04:49:20.332961+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:18.846190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350851+0000","last_change":"2023-10-06T04:49:20.341492+0000","last_active":"2023-10-06T04:49:20.350851+0000","last_peered":"2023-10-06T04:49:20.350851+0000","last_clean":"2023-10-06T04:49:20.350851+0000","last_became_active":"2023-10-06T04:49:20.341257+0000","last_became_peered":"2023-10-06T04:49:20.341257+0000","last_unstale":"2023-10-06T04:49:20.350851+0000","last_undegraded":"2023-10-06T04:49:20.350851+0000","last_fullsized":"2023-10-06T04:49:20.350851+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:28.214277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.315096+0000","last_change":"2023-10-06T04:49:19.324312+0000","last_active":"2023-10-06T04:49:20.315096+0000","last_peered":"2023-10-06T04:49:20.315096+0000","last_clean":"2023-10-06T04:49:20.315096+0000","last_became_active":"2023-10-06T04:49:19.324046+0000","last_became_peered":"2023-10-06T04:49:19.324046+0000","last_unstale":"2023-10-06T04:49:20.315096+0000","last_undegraded":"2023-10-06T04:49:20.315096+0000","last_fullsized":"2023-10-06T04:49:20.315096+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:49.588344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344426+0000","last_change":"2023-10-06T04:49:20.341875+0000","last_active":"2023-10-06T04:49:20.344426+0000","last_peered":"2023-10-06T04:49:20.344426+0000","last_clean":"2023-10-06T04:49:20.344426+0000","last_became_active":"2023-10-06T04:49:20.341312+0000","last_became_peered":"2023-10-06T04:49:20.341312+0000","last_unstale":"2023-10-06T04:49:20.344426+0000","last_undegraded":"2023-10-06T04:49:20.344426+0000","last_fullsized":"2023-10-06T04:49:20.344426+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:07.881803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.351368+0000","last_change":"2023-10-06T04:49:19.331266+0000","last_active":"2023-10-06T04:49:20.351368+0000","last_peered":"2023-10-06T04:49:20.351368+0000","last_clean":"2023-10-06T04:49:20.351368+0000","last_became_active":"2023-10-06T04:49:19.331010+0000","last_became_peered":"2023-10-06T04:49:19.331010+0000","last_unstale":"2023-10-06T04:49:20.351368+0000","last_undegraded":"2023-10-06T04:49:20.351368+0000","last_fullsized":"2023-10-06T04:49:20.351368+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:41.647975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355558+0000","last_change":"2023-10-06T04:49:20.350352+0000","last_active":"2023-10-06T04:49:20.355558+0000","last_peered":"2023-10-06T04:49:20.355558+0000","last_clean":"2023-10-06T04:49:20.355558+0000","last_became_active":"2023-10-06T04:49:20.350084+0000","last_became_peered":"2023-10-06T04:49:20.350084+0000","last_unstale":"2023-10-06T04:49:20.355558+0000","last_undegraded":"2023-10-06T04:49:20.355558+0000","last_fullsized":"2023-10-06T04:49:20.355558+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:54.488863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.353329+0000","last_change":"2023-10-06T04:49:19.333467+0000","last_active":"2023-10-06T04:49:20.353329+0000","last_peered":"2023-10-06T04:49:20.353329+0000","last_clean":"2023-10-06T04:49:20.353329+0000","last_became_active":"2023-10-06T04:49:19.333191+0000","last_became_peered":"2023-10-06T04:49:19.333191+0000","last_unstale":"2023-10-06T04:49:20.353329+0000","last_undegraded":"2023-10-06T04:49:20.353329+0000","last_fullsized":"2023-10-06T04:49:20.353329+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:50.631194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346482+0000","last_change":"2023-10-06T04:49:20.340075+0000","last_active":"2023-10-06T04:49:20.346482+0000","last_peered":"2023-10-06T04:49:20.346482+0000","last_clean":"2023-10-06T04:49:20.346482+0000","last_became_active":"2023-10-06T04:49:20.339781+0000","last_became_peered":"2023-10-06T04:49:20.339781+0000","last_unstale":"2023-10-06T04:49:20.346482+0000","last_undegraded":"2023-10-06T04:49:20.346482+0000","last_fullsized":"2023-10-06T04:49:20.346482+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:57.445018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344697+0000","last_change":"2023-10-06T04:49:19.324434+0000","last_active":"2023-10-06T04:49:20.344697+0000","last_peered":"2023-10-06T04:49:20.344697+0000","last_clean":"2023-10-06T04:49:20.344697+0000","last_became_active":"2023-10-06T04:49:19.324162+0000","last_became_peered":"2023-10-06T04:49:19.324162+0000","last_unstale":"2023-10-06T04:49:20.344697+0000","last_undegraded":"2023-10-06T04:49:20.344697+0000","last_fullsized":"2023-10-06T04:49:20.344697+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350745+0000","last_change":"2023-10-06T04:49:20.345980+0000","last_active":"2023-10-06T04:49:20.350745+0000","last_peered":"2023-10-06T04:49:20.350745+0000","last_clean":"2023-10-06T04:49:20.350745+0000","last_became_active":"2023-10-06T04:49:20.345691+0000","last_became_peered":"2023-10-06T04:49:20.345691+0000","last_unstale":"2023-10-06T04:49:20.350745+0000","last_undegraded":"2023-10-06T04:49:20.350745+0000","last_fullsized":"2023-10-06T04:49:20.350745+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:23.582332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.319623+0000","last_change":"2023-10-06T04:49:19.330962+0000","last_active":"2023-10-06T04:49:20.319623+0000","last_peered":"2023-10-06T04:49:20.319623+0000","last_clean":"2023-10-06T04:49:20.319623+0000","last_became_active":"2023-10-06T04:49:19.330616+0000","last_became_peered":"2023-10-06T04:49:19.330616+0000","last_unstale":"2023-10-06T04:49:20.319623+0000","last_undegraded":"2023-10-06T04:49:20.319623+0000","last_fullsized":"2023-10-06T04:49:20.319623+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:04.272287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355580+0000","last_change":"2023-10-06T04:49:20.350356+0000","last_active":"2023-10-06T04:49:20.355580+0000","last_peered":"2023-10-06T04:49:20.355580+0000","last_clean":"2023-10-06T04:49:20.355580+0000","last_became_active":"2023-10-06T04:49:20.350105+0000","last_became_peered":"2023-10-06T04:49:20.350105+0000","last_unstale":"2023-10-06T04:49:20.355580+0000","last_undegraded":"2023-10-06T04:49:20.355580+0000","last_fullsized":"2023-10-06T04:49:20.355580+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:10.637938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346898+0000","last_change":"2023-10-06T04:49:19.333331+0000","last_active":"2023-10-06T04:49:20.346898+0000","last_peered":"2023-10-06T04:49:20.346898+0000","last_clean":"2023-10-06T04:49:20.346898+0000","last_became_active":"2023-10-06T04:49:19.332774+0000","last_became_peered":"2023-10-06T04:49:19.332774+0000","last_unstale":"2023-10-06T04:49:20.346898+0000","last_undegraded":"2023-10-06T04:49:20.346898+0000","last_fullsized":"2023-10-06T04:49:20.346898+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:19.019093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346151+0000","last_change":"2023-10-06T04:49:20.345404+0000","last_active":"2023-10-06T04:49:20.346151+0000","last_peered":"2023-10-06T04:49:20.346151+0000","last_clean":"2023-10-06T04:49:20.346151+0000","last_became_active":"2023-10-06T04:49:20.345129+0000","last_became_peered":"2023-10-06T04:49:20.345129+0000","last_unstale":"2023-10-06T04:49:20.346151+0000","last_undegraded":"2023-10-06T04:49:20.346151+0000","last_fullsized":"2023-10-06T04:49:20.346151+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:25.017277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.319374+0000","last_change":"2023-10-06T04:49:19.322897+0000","last_active":"2023-10-06T04:49:20.319374+0000","last_peered":"2023-10-06T04:49:20.319374+0000","last_clean":"2023-10-06T04:49:20.319374+0000","last_became_active":"2023-10-06T04:49:19.322635+0000","last_became_peered":"2023-10-06T04:49:19.322635+0000","last_unstale":"2023-10-06T04:49:20.319374+0000","last_undegraded":"2023-10-06T04:49:20.319374+0000","last_fullsized":"2023-10-06T04:49:20.319374+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:19.019093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350774+0000","last_change":"2023-10-06T04:49:20.347069+0000","last_active":"2023-10-06T04:49:20.350774+0000","last_peered":"2023-10-06T04:49:20.350774+0000","last_clean":"2023-10-06T04:49:20.350774+0000","last_became_active":"2023-10-06T04:49:20.346801+0000","last_became_peered":"2023-10-06T04:49:20.346801+0000","last_unstale":"2023-10-06T04:49:20.350774+0000","last_undegraded":"2023-10-06T04:49:20.350774+0000","last_fullsized":"2023-10-06T04:49:20.350774+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:34.988176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321187+0000","last_change":"2023-10-06T04:49:19.329229+0000","last_active":"2023-10-06T04:49:20.321187+0000","last_peered":"2023-10-06T04:49:20.321187+0000","last_clean":"2023-10-06T04:49:20.321187+0000","last_became_active":"2023-10-06T04:49:19.328964+0000","last_became_peered":"2023-10-06T04:49:19.328964+0000","last_unstale":"2023-10-06T04:49:20.321187+0000","last_undegraded":"2023-10-06T04:49:20.321187+0000","last_fullsized":"2023-10-06T04:49:20.321187+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:53.870884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341139+0000","last_change":"2023-10-06T04:49:20.334997+0000","last_active":"2023-10-06T04:49:20.341139+0000","last_peered":"2023-10-06T04:49:20.341139+0000","last_clean":"2023-10-06T04:49:20.341139+0000","last_became_active":"2023-10-06T04:49:20.334723+0000","last_became_peered":"2023-10-06T04:49:20.334723+0000","last_unstale":"2023-10-06T04:49:20.341139+0000","last_undegraded":"2023-10-06T04:49:20.341139+0000","last_fullsized":"2023-10-06T04:49:20.341139+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:41.560870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321711+0000","last_change":"2023-10-06T04:49:19.324862+0000","last_active":"2023-10-06T04:49:20.321711+0000","last_peered":"2023-10-06T04:49:20.321711+0000","last_clean":"2023-10-06T04:49:20.321711+0000","last_became_active":"2023-10-06T04:49:19.324595+0000","last_became_peered":"2023-10-06T04:49:19.324595+0000","last_unstale":"2023-10-06T04:49:20.321711+0000","last_undegraded":"2023-10-06T04:49:20.321711+0000","last_fullsized":"2023-10-06T04:49:20.321711+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:53.870884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339461+0000","last_change":"2023-10-06T04:49:20.337671+0000","last_active":"2023-10-06T04:49:20.339461+0000","last_peered":"2023-10-06T04:49:20.339461+0000","last_clean":"2023-10-06T04:49:20.339461+0000","last_became_active":"2023-10-06T04:49:20.337383+0000","last_became_peered":"2023-10-06T04:49:20.337383+0000","last_unstale":"2023-10-06T04:49:20.339461+0000","last_undegraded":"2023-10-06T04:49:20.339461+0000","last_fullsized":"2023-10-06T04:49:20.339461+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:41.316112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.342104+0000","last_change":"2023-10-06T04:49:19.328915+0000","last_active":"2023-10-06T04:49:20.342104+0000","last_peered":"2023-10-06T04:49:20.342104+0000","last_clean":"2023-10-06T04:49:20.342104+0000","last_became_active":"2023-10-06T04:49:19.328630+0000","last_became_peered":"2023-10-06T04:49:19.328630+0000","last_unstale":"2023-10-06T04:49:20.342104+0000","last_undegraded":"2023-10-06T04:49:20.342104+0000","last_fullsized":"2023-10-06T04:49:20.342104+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:27.949348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.343843+0000","last_change":"2023-10-06T04:49:20.341772+0000","last_active":"2023-10-06T04:49:20.343843+0000","last_peered":"2023-10-06T04:49:20.343843+0000","last_clean":"2023-10-06T04:49:20.343843+0000","last_became_active":"2023-10-06T04:49:20.341489+0000","last_became_peered":"2023-10-06T04:49:20.341489+0000","last_unstale":"2023-10-06T04:49:20.343843+0000","last_undegraded":"2023-10-06T04:49:20.343843+0000","last_fullsized":"2023-10-06T04:49:20.343843+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:10.460143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.330303+0000","last_change":"2023-10-06T04:49:19.331671+0000","last_active":"2023-10-06T04:49:20.330303+0000","last_peered":"2023-10-06T04:49:20.330303+0000","last_clean":"2023-10-06T04:49:20.330303+0000","last_became_active":"2023-10-06T04:49:19.330061+0000","last_became_peered":"2023-10-06T04:49:19.330061+0000","last_unstale":"2023-10-06T04:49:20.330303+0000","last_undegraded":"2023-10-06T04:49:20.330303+0000","last_fullsized":"2023-10-06T04:49:20.330303+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:09.188792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356042+0000","last_change":"2023-10-06T04:49:20.352177+0000","last_active":"2023-10-06T04:49:20.356042+0000","last_peered":"2023-10-06T04:49:20.356042+0000","last_clean":"2023-10-06T04:49:20.356042+0000","last_became_active":"2023-10-06T04:49:20.351885+0000","last_became_peered":"2023-10-06T04:49:20.351885+0000","last_unstale":"2023-10-06T04:49:20.356042+0000","last_undegraded":"2023-10-06T04:49:20.356042+0000","last_fullsized":"2023-10-06T04:49:20.356042+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320737+0000","last_change":"2023-10-06T04:49:19.329132+0000","last_active":"2023-10-06T04:49:20.320737+0000","last_peered":"2023-10-06T04:49:20.320737+0000","last_clean":"2023-10-06T04:49:20.320737+0000","last_became_active":"2023-10-06T04:49:19.328841+0000","last_became_peered":"2023-10-06T04:49:19.328841+0000","last_unstale":"2023-10-06T04:49:20.320737+0000","last_undegraded":"2023-10-06T04:49:20.320737+0000","last_fullsized":"2023-10-06T04:49:20.320737+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:54.415706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344647+0000","last_change":"2023-10-06T04:49:20.343226+0000","last_active":"2023-10-06T04:49:20.344647+0000","last_peered":"2023-10-06T04:49:20.344647+0000","last_clean":"2023-10-06T04:49:20.344647+0000","last_became_active":"2023-10-06T04:49:20.342932+0000","last_became_peered":"2023-10-06T04:49:20.342932+0000","last_unstale":"2023-10-06T04:49:20.344647+0000","last_undegraded":"2023-10-06T04:49:20.344647+0000","last_fullsized":"2023-10-06T04:49:20.344647+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:28.172716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.316079+0000","last_change":"2023-10-06T04:49:19.328230+0000","last_active":"2023-10-06T04:49:20.316079+0000","last_peered":"2023-10-06T04:49:20.316079+0000","last_clean":"2023-10-06T04:49:20.316079+0000","last_became_active":"2023-10-06T04:49:19.327917+0000","last_became_peered":"2023-10-06T04:49:19.327917+0000","last_unstale":"2023-10-06T04:49:20.316079+0000","last_undegraded":"2023-10-06T04:49:20.316079+0000","last_fullsized":"2023-10-06T04:49:20.316079+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.900580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347059+0000","last_change":"2023-10-06T04:49:20.336712+0000","last_active":"2023-10-06T04:49:20.347059+0000","last_peered":"2023-10-06T04:49:20.347059+0000","last_clean":"2023-10-06T04:49:20.347059+0000","last_became_active":"2023-10-06T04:49:20.335319+0000","last_became_peered":"2023-10-06T04:49:20.335319+0000","last_unstale":"2023-10-06T04:49:20.347059+0000","last_undegraded":"2023-10-06T04:49:20.347059+0000","last_fullsized":"2023-10-06T04:49:20.347059+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:00.206328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.325259+0000","last_change":"2023-10-06T04:49:19.331624+0000","last_active":"2023-10-06T04:49:20.325259+0000","last_peered":"2023-10-06T04:49:20.325259+0000","last_clean":"2023-10-06T04:49:20.325259+0000","last_became_active":"2023-10-06T04:49:19.331287+0000","last_became_peered":"2023-10-06T04:49:19.331287+0000","last_unstale":"2023-10-06T04:49:20.325259+0000","last_undegraded":"2023-10-06T04:49:20.325259+0000","last_fullsized":"2023-10-06T04:49:20.325259+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.288607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341412+0000","last_change":"2023-10-06T04:49:20.334336+0000","last_active":"2023-10-06T04:49:20.341412+0000","last_peered":"2023-10-06T04:49:20.341412+0000","last_clean":"2023-10-06T04:49:20.341412+0000","last_became_active":"2023-10-06T04:49:20.334097+0000","last_became_peered":"2023-10-06T04:49:20.334097+0000","last_unstale":"2023-10-06T04:49:20.341412+0000","last_undegraded":"2023-10-06T04:49:20.341412+0000","last_fullsized":"2023-10-06T04:49:20.341412+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:35.183960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320080+0000","last_change":"2023-10-06T04:49:19.332817+0000","last_active":"2023-10-06T04:49:20.320080+0000","last_peered":"2023-10-06T04:49:20.320080+0000","last_clean":"2023-10-06T04:49:20.320080+0000","last_became_active":"2023-10-06T04:49:19.331466+0000","last_became_peered":"2023-10-06T04:49:19.331466+0000","last_unstale":"2023-10-06T04:49:20.320080+0000","last_undegraded":"2023-10-06T04:49:20.320080+0000","last_fullsized":"2023-10-06T04:49:20.320080+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:01.585470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332830+0000","last_change":"2023-10-06T04:49:20.327613+0000","last_active":"2023-10-06T04:49:20.332830+0000","last_peered":"2023-10-06T04:49:20.332830+0000","last_clean":"2023-10-06T04:49:20.332830+0000","last_became_active":"2023-10-06T04:49:20.327331+0000","last_became_peered":"2023-10-06T04:49:20.327331+0000","last_unstale":"2023-10-06T04:49:20.332830+0000","last_undegraded":"2023-10-06T04:49:20.332830+0000","last_fullsized":"2023-10-06T04:49:20.332830+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:21.888629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339559+0000","last_change":"2023-10-06T04:49:19.327134+0000","last_active":"2023-10-06T04:49:20.339559+0000","last_peered":"2023-10-06T04:49:20.339559+0000","last_clean":"2023-10-06T04:49:20.339559+0000","last_became_active":"2023-10-06T04:49:19.326884+0000","last_became_peered":"2023-10-06T04:49:19.326884+0000","last_unstale":"2023-10-06T04:49:20.339559+0000","last_undegraded":"2023-10-06T04:49:20.339559+0000","last_fullsized":"2023-10-06T04:49:20.339559+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:40.308269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347026+0000","last_change":"2023-10-06T04:49:20.343182+0000","last_active":"2023-10-06T04:49:20.347026+0000","last_peered":"2023-10-06T04:49:20.347026+0000","last_clean":"2023-10-06T04:49:20.347026+0000","last_became_active":"2023-10-06T04:49:20.342662+0000","last_became_peered":"2023-10-06T04:49:20.342662+0000","last_unstale":"2023-10-06T04:49:20.347026+0000","last_undegraded":"2023-10-06T04:49:20.347026+0000","last_fullsized":"2023-10-06T04:49:20.347026+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:54.488863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331074+0000","last_change":"2023-10-06T04:49:19.336399+0000","last_active":"2023-10-06T04:49:20.331074+0000","last_peered":"2023-10-06T04:49:20.331074+0000","last_clean":"2023-10-06T04:49:20.331074+0000","last_became_active":"2023-10-06T04:49:19.336144+0000","last_became_peered":"2023-10-06T04:49:19.336144+0000","last_unstale":"2023-10-06T04:49:20.331074+0000","last_undegraded":"2023-10-06T04:49:20.331074+0000","last_fullsized":"2023-10-06T04:49:20.331074+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:48.611176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333465+0000","last_change":"2023-10-06T04:49:20.325062+0000","last_active":"2023-10-06T04:49:20.333465+0000","last_peered":"2023-10-06T04:49:20.333465+0000","last_clean":"2023-10-06T04:49:20.333465+0000","last_became_active":"2023-10-06T04:49:20.324091+0000","last_became_peered":"2023-10-06T04:49:20.324091+0000","last_unstale":"2023-10-06T04:49:20.333465+0000","last_undegraded":"2023-10-06T04:49:20.333465+0000","last_fullsized":"2023-10-06T04:49:20.333465+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:28.109738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321039+0000","last_change":"2023-10-06T04:49:19.327314+0000","last_active":"2023-10-06T04:49:20.321039+0000","last_peered":"2023-10-06T04:49:20.321039+0000","last_clean":"2023-10-06T04:49:20.321039+0000","last_became_active":"2023-10-06T04:49:19.326851+0000","last_became_peered":"2023-10-06T04:49:19.326851+0000","last_unstale":"2023-10-06T04:49:20.321039+0000","last_undegraded":"2023-10-06T04:49:20.321039+0000","last_fullsized":"2023-10-06T04:49:20.321039+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:26.547655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":156,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321556+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T04:49:20.321556+0000","last_peered":"2023-10-06T04:49:20.321556+0000","last_clean":"2023-10-06T04:49:20.321556+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T04:49:20.321556+0000","last_undegraded":"2023-10-06T04:49:20.321556+0000","last_fullsized":"2023-10-06T04:49:20.321556+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332841+0000","last_change":"2023-10-06T04:49:20.327658+0000","last_active":"2023-10-06T04:49:20.332841+0000","last_peered":"2023-10-06T04:49:20.332841+0000","last_clean":"2023-10-06T04:49:20.332841+0000","last_became_active":"2023-10-06T04:49:20.327377+0000","last_became_peered":"2023-10-06T04:49:20.327377+0000","last_unstale":"2023-10-06T04:49:20.332841+0000","last_undegraded":"2023-10-06T04:49:20.332841+0000","last_fullsized":"2023-10-06T04:49:20.332841+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"172'8","reported_seq":18,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.351338+0000","last_change":"2023-10-06T04:49:19.329982+0000","last_active":"2023-10-06T04:49:20.351338+0000","last_peered":"2023-10-06T04:49:20.351338+0000","last_clean":"2023-10-06T04:49:20.351338+0000","last_became_active":"2023-10-06T04:49:19.329677+0000","last_became_peered":"2023-10-06T04:49:19.329677+0000","last_unstale":"2023-10-06T04:49:20.351338+0000","last_undegraded":"2023-10-06T04:49:20.351338+0000","last_fullsized":"2023-10-06T04:49:20.351338+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":143,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.316165+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T04:49:20.316165+0000","last_peered":"2023-10-06T04:49:20.316165+0000","last_clean":"2023-10-06T04:49:20.316165+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T04:49:20.316165+0000","last_undegraded":"2023-10-06T04:49:20.316165+0000","last_fullsized":"2023-10-06T04:49:20.316165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.343279+0000","last_change":"2023-10-06T04:49:20.330255+0000","last_active":"2023-10-06T04:49:20.343279+0000","last_peered":"2023-10-06T04:49:20.343279+0000","last_clean":"2023-10-06T04:49:20.343279+0000","last_became_active":"2023-10-06T04:49:20.329995+0000","last_became_peered":"2023-10-06T04:49:20.329995+0000","last_unstale":"2023-10-06T04:49:20.343279+0000","last_undegraded":"2023-10-06T04:49:20.343279+0000","last_fullsized":"2023-10-06T04:49:20.343279+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:16.754707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.330157+0000","last_change":"2023-10-06T04:49:19.330514+0000","last_active":"2023-10-06T04:49:20.330157+0000","last_peered":"2023-10-06T04:49:20.330157+0000","last_clean":"2023-10-06T04:49:20.330157+0000","last_became_active":"2023-10-06T04:49:19.330226+0000","last_became_peered":"2023-10-06T04:49:19.330226+0000","last_unstale":"2023-10-06T04:49:20.330157+0000","last_undegraded":"2023-10-06T04:49:20.330157+0000","last_fullsized":"2023-10-06T04:49:20.330157+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:55.142830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":144,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.319996+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T04:49:20.319996+0000","last_peered":"2023-10-06T04:49:20.319996+0000","last_clean":"2023-10-06T04:49:20.319996+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T04:49:20.319996+0000","last_undegraded":"2023-10-06T04:49:20.319996+0000","last_fullsized":"2023-10-06T04:49:20.319996+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"163'293","reported_seq":919,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331027+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T04:49:20.331027+0000","last_peered":"2023-10-06T04:49:20.331027+0000","last_clean":"2023-10-06T04:49:20.331027+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T04:49:20.331027+0000","last_undegraded":"2023-10-06T04:49:20.331027+0000","last_fullsized":"2023-10-06T04:49:20.331027+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.336779+0000","last_change":"2023-10-06T04:49:20.335358+0000","last_active":"2023-10-06T04:49:20.336779+0000","last_peered":"2023-10-06T04:49:20.336779+0000","last_clean":"2023-10-06T04:49:20.336779+0000","last_became_active":"2023-10-06T04:49:20.335073+0000","last_became_peered":"2023-10-06T04:49:20.335073+0000","last_unstale":"2023-10-06T04:49:20.336779+0000","last_undegraded":"2023-10-06T04:49:20.336779+0000","last_fullsized":"2023-10-06T04:49:20.336779+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:05.280129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.330408+0000","last_change":"2023-10-06T04:49:19.331617+0000","last_active":"2023-10-06T04:49:20.330408+0000","last_peered":"2023-10-06T04:49:20.330408+0000","last_clean":"2023-10-06T04:49:20.330408+0000","last_became_active":"2023-10-06T04:49:19.329690+0000","last_became_peered":"2023-10-06T04:49:19.329690+0000","last_unstale":"2023-10-06T04:49:20.330408+0000","last_undegraded":"2023-10-06T04:49:20.330408+0000","last_fullsized":"2023-10-06T04:49:20.330408+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:04.983797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":146,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.325305+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T04:49:20.325305+0000","last_peered":"2023-10-06T04:49:20.325305+0000","last_clean":"2023-10-06T04:49:20.325305+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T04:49:20.325305+0000","last_undegraded":"2023-10-06T04:49:20.325305+0000","last_fullsized":"2023-10-06T04:49:20.325305+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339557+0000","last_change":"2023-10-06T04:49:20.331740+0000","last_active":"2023-10-06T04:49:20.339557+0000","last_peered":"2023-10-06T04:49:20.339557+0000","last_clean":"2023-10-06T04:49:20.339557+0000","last_became_active":"2023-10-06T04:49:20.331463+0000","last_became_peered":"2023-10-06T04:49:20.331463+0000","last_unstale":"2023-10-06T04:49:20.339557+0000","last_undegraded":"2023-10-06T04:49:20.339557+0000","last_fullsized":"2023-10-06T04:49:20.339557+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:01.313087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"172'6","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356034+0000","last_change":"2023-10-06T04:49:19.331731+0000","last_active":"2023-10-06T04:49:20.356034+0000","last_peered":"2023-10-06T04:49:20.356034+0000","last_clean":"2023-10-06T04:49:20.356034+0000","last_became_active":"2023-10-06T04:49:19.331458+0000","last_became_peered":"2023-10-06T04:49:19.331458+0000","last_unstale":"2023-10-06T04:49:20.356034+0000","last_undegraded":"2023-10-06T04:49:20.356034+0000","last_fullsized":"2023-10-06T04:49:20.356034+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:55.142830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":146,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331105+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T04:49:20.331105+0000","last_peered":"2023-10-06T04:49:20.331105+0000","last_clean":"2023-10-06T04:49:20.331105+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T04:49:20.331105+0000","last_undegraded":"2023-10-06T04:49:20.331105+0000","last_fullsized":"2023-10-06T04:49:20.331105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347917+0000","last_change":"2023-10-06T04:49:20.345754+0000","last_active":"2023-10-06T04:49:20.347917+0000","last_peered":"2023-10-06T04:49:20.347917+0000","last_clean":"2023-10-06T04:49:20.347917+0000","last_became_active":"2023-10-06T04:49:20.345446+0000","last_became_peered":"2023-10-06T04:49:20.345446+0000","last_unstale":"2023-10-06T04:49:20.347917+0000","last_undegraded":"2023-10-06T04:49:20.347917+0000","last_fullsized":"2023-10-06T04:49:20.347917+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:00.217906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"172'3","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.342594+0000","last_change":"2023-10-06T04:49:19.327118+0000","last_active":"2023-10-06T04:49:20.342594+0000","last_peered":"2023-10-06T04:49:20.342594+0000","last_clean":"2023-10-06T04:49:20.342594+0000","last_became_active":"2023-10-06T04:49:19.326839+0000","last_became_peered":"2023-10-06T04:49:19.326839+0000","last_unstale":"2023-10-06T04:49:20.342594+0000","last_undegraded":"2023-10-06T04:49:20.342594+0000","last_fullsized":"2023-10-06T04:49:20.342594+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:17.629867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":139,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.335923+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T04:49:20.335923+0000","last_peered":"2023-10-06T04:49:20.335923+0000","last_clean":"2023-10-06T04:49:20.335923+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T04:49:20.335923+0000","last_undegraded":"2023-10-06T04:49:20.335923+0000","last_fullsized":"2023-10-06T04:49:20.335923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341195+0000","last_change":"2023-10-06T04:49:20.338084+0000","last_active":"2023-10-06T04:49:20.341195+0000","last_peered":"2023-10-06T04:49:20.341195+0000","last_clean":"2023-10-06T04:49:20.341195+0000","last_became_active":"2023-10-06T04:49:20.337738+0000","last_became_peered":"2023-10-06T04:49:20.337738+0000","last_unstale":"2023-10-06T04:49:20.341195+0000","last_undegraded":"2023-10-06T04:49:20.341195+0000","last_fullsized":"2023-10-06T04:49:20.341195+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:28.172716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321149+0000","last_change":"2023-10-06T04:49:19.332809+0000","last_active":"2023-10-06T04:49:20.321149+0000","last_peered":"2023-10-06T04:49:20.321149+0000","last_clean":"2023-10-06T04:49:20.321149+0000","last_became_active":"2023-10-06T04:49:19.331892+0000","last_became_peered":"2023-10-06T04:49:19.331892+0000","last_unstale":"2023-10-06T04:49:20.321149+0000","last_undegraded":"2023-10-06T04:49:20.321149+0000","last_fullsized":"2023-10-06T04:49:20.321149+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:52.469578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320842+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T04:49:20.320842+0000","last_peered":"2023-10-06T04:49:20.320842+0000","last_clean":"2023-10-06T04:49:20.320842+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T04:49:20.320842+0000","last_undegraded":"2023-10-06T04:49:20.320842+0000","last_fullsized":"2023-10-06T04:49:20.320842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347079+0000","last_change":"2023-10-06T04:49:20.336896+0000","last_active":"2023-10-06T04:49:20.347079+0000","last_peered":"2023-10-06T04:49:20.347079+0000","last_clean":"2023-10-06T04:49:20.347079+0000","last_became_active":"2023-10-06T04:49:20.336451+0000","last_became_peered":"2023-10-06T04:49:20.336451+0000","last_unstale":"2023-10-06T04:49:20.347079+0000","last_undegraded":"2023-10-06T04:49:20.347079+0000","last_fullsized":"2023-10-06T04:49:20.347079+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:09.120814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344451+0000","last_change":"2023-10-06T04:49:19.321982+0000","last_active":"2023-10-06T04:49:20.344451+0000","last_peered":"2023-10-06T04:49:20.344451+0000","last_clean":"2023-10-06T04:49:20.344451+0000","last_became_active":"2023-10-06T04:49:19.321679+0000","last_became_peered":"2023-10-06T04:49:19.321679+0000","last_unstale":"2023-10-06T04:49:20.344451+0000","last_undegraded":"2023-10-06T04:49:20.344451+0000","last_fullsized":"2023-10-06T04:49:20.344451+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320746+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T04:49:20.320746+0000","last_peered":"2023-10-06T04:49:20.320746+0000","last_clean":"2023-10-06T04:49:20.320746+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T04:49:20.320746+0000","last_undegraded":"2023-10-06T04:49:20.320746+0000","last_fullsized":"2023-10-06T04:49:20.320746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575752,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299976,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071864,"statfs":{"total":96636764160,"available":96329588736,"internally_reserved":0,"allocated":2752512,"data_stored":1720883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69799999999999995,"15min":0.73899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.89500000000000002,"5min":0.97199999999999998,"15min":1.4930000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.48599999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90900000000000003,"5min":0.995,"15min":1.417},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.755,"15min":0.79500000000000004},"min":{"1min":0.53500000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93100000000000005,"5min":1.01,"15min":1.5309999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77600000000000002,"15min":0.79300000000000004},"min":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.58299999999999996},"max":{"1min":0.93600000000000005,"5min":1.0289999999999999,"15min":1.097},"last":0.81000000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70899999999999996,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.46400000000000002},"max":{"1min":0.92300000000000004,"5min":0.97999999999999998,"15min":1.5429999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.751,"15min":0.77100000000000002},"min":{"1min":0.60199999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0189999999999999},"last":0.82999999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.75700000000000001,"15min":0.77500000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.91000000000000003,"5min":1.03,"15min":1.5860000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73299999999999998,"15min":0.76400000000000001},"min":{"1min":0.64200000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.96099999999999997}]},{"osd":4,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63400000000000001,"15min":0.61899999999999999},"min":{"1min":0.50800000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.0640000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.439,"15min":0.42799999999999999},"max":{"1min":1.1339999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.90400000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.53800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.55200000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.72999999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.64200000000000002,"5min":0.501,"15min":0.49199999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":1.1020000000000001,"15min":1.2090000000000001},"last":0.93899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575753,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299400,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072440,"statfs":{"total":96636764160,"available":96330178560,"internally_reserved":0,"allocated":2162688,"data_stored":1131541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.46000000000000002,"5min":0.40300000000000002,"15min":0.375},"max":{"1min":0.96599999999999997,"5min":1.038,"15min":1.038},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76200000000000001,"15min":0.752},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94699999999999995,"5min":0.999,"15min":1.228},"last":0.80000000000000004}]},{"osd":1,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.5,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93799999999999994,"5min":1.046,"15min":1.3660000000000001},"last":0.63},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.748,"15min":0.751},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.438},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.3560000000000001},"last":0.76500000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.47999999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.272},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76900000000000002,"15min":0.77000000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98799999999999999,"5min":1.069,"15min":1.089},"last":0.66300000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94399999999999995,"5min":1.042,"15min":1.1060000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.52000000000000002},"max":{"1min":1.0009999999999999,"5min":1.0409999999999999,"15min":1.3120000000000001},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83499999999999996,"5min":0.88500000000000001,"15min":0.92600000000000005},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.627},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":1.155},"last":0.84499999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.432},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93799999999999994,"5min":1.056,"15min":1.056},"last":0.82799999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95399999999999996,"5min":1.0580000000000001,"15min":1.3500000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.45800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98599999999999999,"5min":1.024,"15min":1.0389999999999999},"last":0.78300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575752,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312268,"kb_used_data":2692,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059572,"statfs":{"total":96636764160,"available":96317001728,"internally_reserved":0,"allocated":2756608,"data_stored":1721909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.79600000000000004,"5min":0.89000000000000001,"15min":0.93100000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42899999999999999},"max":{"1min":0.90000000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.41399999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63800000000000001,"5min":0.67700000000000005,"15min":0.73299999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.38200000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.97799999999999998},"last":0.46899999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.44,"5min":0.44,"15min":0.437},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.94699999999999995},"last":0.497},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.373},"max":{"1min":0.748,"5min":0.85799999999999998,"15min":0.88900000000000001},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79200000000000004,"15min":0.79500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.46899999999999997},"max":{"1min":1.159,"5min":1.1970000000000001,"15min":1.407},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82599999999999996,"15min":0.83099999999999996},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.57499999999999996},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.84399999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83699999999999997,"15min":0.82699999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.55700000000000005},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81699999999999995,"15min":0.83899999999999997},"min":{"1min":0.66300000000000003,"5min":0.59799999999999998,"15min":0.53200000000000003},"max":{"1min":1.097,"5min":1.1060000000000001,"15min":1.724},"last":0.88300000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.64700000000000002,"5min":0.626,"15min":0.56200000000000006},"max":{"1min":0.97899999999999998,"5min":1.2150000000000001,"15min":1.427},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.55800000000000005},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.93600000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.877,"15min":0.878},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1140000000000001,"5min":1.125,"15min":1.702},"last":0.874},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85799999999999998,"15min":0.872},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0269999999999999,"5min":1.1659999999999999,"15min":1.5920000000000001},"last":0.72199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575754,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299416,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072424,"statfs":{"total":96636764160,"available":96330162176,"internally_reserved":0,"allocated":2179072,"data_stored":1132707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.378,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76600000000000001,"5min":0.81699999999999995,"15min":1.6459999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.34200000000000003,"15min":0.32900000000000001},"max":{"1min":0.68300000000000005,"5min":0.84099999999999997,"15min":1.4510000000000001},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55400000000000005,"15min":0.57099999999999995},"min":{"1min":0.434,"5min":0.308,"15min":0.308},"max":{"1min":0.74099999999999999,"5min":0.86899999999999999,"15min":1.7},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.373},"max":{"1min":0.81699999999999995,"5min":0.82099999999999995,"15min":1.6240000000000001},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":1.2529999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91300000000000003,"5min":0.99099999999999999,"15min":1.7370000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.86599999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.60399999999999998},"max":{"1min":1.0940000000000001,"5min":1.2769999999999999,"15min":2.3700000000000001},"last":0.873},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.85099999999999998,"15min":0.85999999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.52900000000000003},"max":{"1min":1.111,"5min":1.3069999999999999,"15min":2.1280000000000001},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84099999999999997,"15min":0.85599999999999998},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.57599999999999996},"max":{"1min":1.048,"5min":1.294,"15min":2.1739999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84099999999999997,"15min":0.84799999999999998},"min":{"1min":0.66000000000000003,"5min":0.63300000000000001,"15min":0.57799999999999996},"max":{"1min":1.087,"5min":1.214,"15min":2.2429999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60599999999999998,"5min":0.59299999999999997,"15min":0.53000000000000003},"max":{"1min":1.1160000000000001,"5min":1.204,"15min":1.605},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89600000000000002,"15min":0.88700000000000001},"min":{"1min":0.72899999999999998,"5min":0.65800000000000003,"15min":0.58199999999999996},"max":{"1min":1.181,"5min":1.226,"15min":2.3519999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.86699999999999999,"15min":0.878},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.091,"5min":1.319,"15min":2.1019999999999999},"last":0.746},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.86299999999999999,"15min":0.86799999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0580000000000001,"5min":1.256,"15min":1.9259999999999999},"last":0.79500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575753,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311688,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060152,"statfs":{"total":96636764160,"available":96317595648,"internally_reserved":0,"allocated":2162688,"data_stored":1130605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.311},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.376},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.34200000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.93999999999999995},"last":0.505}]},{"osd":1,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.32000000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.42399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.38300000000000001},"max":{"1min":0.79600000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.371},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.45400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.628,"5min":0.60199999999999998,"15min":0.51700000000000002},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.86399999999999999,"15min":0.88100000000000001},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58599999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.381},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.59199999999999997},"max":{"1min":1.1719999999999999,"5min":1.2210000000000001,"15min":1.4379999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88300000000000001,"15min":0.89200000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.58899999999999997},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.288},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.151,"5min":1.167,"15min":1.2370000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.61499999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86099999999999999,"15min":0.874},"min":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.57499999999999996},"max":{"1min":1.075,"5min":1.3320000000000001,"15min":1.482},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.88500000000000001,"15min":0.89600000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.135,"5min":1.2869999999999999,"15min":1.405},"last":0.755}]}]},{"osd":3,"up_from":13,"seq":55834575753,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311692,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060148,"statfs":{"total":96636764160,"available":96317591552,"internally_reserved":0,"allocated":2166784,"data_stored":1132059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47699999999999998,"15min":0.49399999999999999},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67900000000000005,"5min":0.73899999999999999,"15min":0.86399999999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.83599999999999997},"last":0.501},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.44900000000000001,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.73999999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.40899999999999997},"max":{"1min":0.69699999999999995,"5min":0.79200000000000004,"15min":1.036},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.54000000000000004,"5min":0.47499999999999998,"15min":0.43099999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83799999999999997},"min":{"1min":0.64300000000000002,"5min":0.56000000000000005,"15min":0.47599999999999998},"max":{"1min":1.101,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.495},"max":{"1min":1.0760000000000001,"5min":1.387,"15min":1.399},"last":1.034}]},{"osd":5,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.82599999999999996,"15min":0.82899999999999996},"min":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0049999999999999,"5min":1.2509999999999999,"15min":1.331},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.85799999999999998,"15min":0.85099999999999998},"min":{"1min":0.63600000000000001,"5min":0.58099999999999996,"15min":0.56599999999999995},"max":{"1min":1.0720000000000001,"5min":1.2130000000000001,"15min":1.2330000000000001},"last":1.0549999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.86499999999999999,"15min":0.85999999999999999},"min":{"1min":0.65100000000000002,"5min":0.53500000000000003,"15min":0.52400000000000002},"max":{"1min":1.175,"5min":1.2090000000000001,"15min":1.2490000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.89900000000000002,"15min":0.89400000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1579999999999999,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88600000000000001,"15min":0.88400000000000001},"min":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.58999999999999997},"max":{"1min":1.127,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85399999999999998,"15min":0.85799999999999998},"min":{"1min":0.60499999999999998,"5min":0.58699999999999997,"15min":0.54900000000000004},"max":{"1min":1.069,"5min":1.276,"15min":1.3029999999999999},"last":1.097}]}]},{"osd":4,"up_from":13,"seq":55834575753,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311696,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060144,"statfs":{"total":96636764160,"available":96317587456,"internally_reserved":0,"allocated":2170880,"data_stored":1133007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.71699999999999997,"15min":0.74299999999999999},"min":{"1min":0.47899999999999998,"5min":0.36699999999999999,"15min":0.35599999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.72399999999999998,"15min":0.73499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.79300000000000004}]},{"osd":1,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.76000000000000001,"15min":0.77800000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.69999999999999996}]},{"osd":2,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.745,"15min":0.75900000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.755},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.76600000000000001,"15min":0.77100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91200000000000003,"5min":1.01,"15min":1.069},"last":0.876}]},{"osd":3,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.996,"5min":1.002,"15min":1.002},"last":0.65900000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81699999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.65000000000000002,"15min":0.63500000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.61099999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69099999999999995,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.46000000000000002,"15min":0.42099999999999999},"max":{"1min":0.996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71499999999999997,"15min":0.70199999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0629999999999999,"5min":1.105,"15min":1.105},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71399999999999997,"15min":0.68799999999999994},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97999999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.75600000000000001,"15min":0.745},"min":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89700000000000002,"5min":1.091,"15min":1.0940000000000001},"last":0.85699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575753,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307584,"kb_used_data":2104,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064256,"statfs":{"total":96636764160,"available":96321798144,"internally_reserved":0,"allocated":2154496,"data_stored":1130493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82999999999999996,"5min":1.03,"15min":2.2829999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73599999999999999,"15min":0.751},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91600000000000004,"5min":1.095,"15min":1.3200000000000001},"last":0.67600000000000005}]},{"osd":1,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.751},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.29},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.51400000000000001},"max":{"1min":0.871,"5min":0.98099999999999998,"15min":1.9139999999999999},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.57999999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98199999999999998,"5min":1.0589999999999999,"15min":1.532},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95799999999999996,"5min":1.0169999999999999,"15min":1.121},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.58899999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.40500000000000003,"15min":0.39300000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":2.6989999999999998},"last":0.75},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38},"max":{"1min":0.99399999999999999,"5min":1.028,"15min":1.694},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.1719999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96399999999999997,"5min":1.077,"15min":1.5880000000000001},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.52500000000000002,"5min":0.47499999999999998,"15min":0.40699999999999997},"max":{"1min":1.03,"5min":1.0409999999999999,"15min":1.7390000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.49099999999999999,"15min":0.48599999999999999},"max":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.3380000000000001},"last":0.75800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:49:25.653 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T04:49:25.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.869+0000 7f22c759e700 1 Processor -- start 2023-10-06T04:49:25.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.869+0000 7f22c759e700 1 -- start start 2023-10-06T04:49:25.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.870+0000 7f22c759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22c81563e0 0x7f22c81567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.870+0000 7f22c759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22c806eeb0 0x7f22c80c6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.870+0000 7f22c759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c8157780 con 0x7f22c80c9350 2023-10-06T04:49:25.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.870+0000 7f22c759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c8156e50 con 0x7f22c81563e0 2023-10-06T04:49:25.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.870+0000 7f22c759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c8157130 con 0x7f22c806eeb0 2023-10-06T04:49:25.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.870+0000 7f22c659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22c81563e0 0x7f22c81567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:25.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.870+0000 7f22c659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f22c81563e0 0x7f22c81567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34482/0 (socket says 172.21.15.55:34482) 2023-10-06T04:49:25.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.870+0000 7f22c659c700 1 -- 172.21.15.55:0/3533385550 learned_addr learned my addr 172.21.15.55:0/3533385550 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:25.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.870+0000 7f22c5d9b700 1 --1- 172.21.15.55:0/3533385550 >> v1:172.21.15.55:6789/0 conn(0x7f22c80c9350 0x7f22c806e7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46734/0 (socket says 172.21.15.55:46734) 2023-10-06T04:49:25.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.871+0000 7f22c6d9d700 1 --2- 172.21.15.55:0/3533385550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22c806eeb0 0x7f22c80c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:25.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.871+0000 7f22c659c700 1 -- 172.21.15.55:0/3533385550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22c806eeb0 msgr2=0x7f22c80c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:25.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.871+0000 7f22c659c700 1 --2- 172.21.15.55:0/3533385550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22c806eeb0 0x7f22c80c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.871+0000 7f22c659c700 1 -- 172.21.15.55:0/3533385550 >> v1:172.21.15.55:6789/0 conn(0x7f22c80c9350 legacy=0x7f22c806e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:25.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.871+0000 7f22c659c700 1 -- 172.21.15.55:0/3533385550 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c8157410 con 0x7f22c81563e0 2023-10-06T04:49:25.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.871+0000 7f22c559a700 1 -- 172.21.15.55:0/3533385550 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884222266 0 0) 0x7f22c8157780 con 0x7f22c80c9350 2023-10-06T04:49:25.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.871+0000 7f22c659c700 1 --2- 172.21.15.55:0/3533385550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c81563e0 0x7f22c81567b0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f22b800bc50 tx=0x7f22b800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7626198ba6178642 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:25.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.871+0000 7f22c559a700 1 -- 172.21.15.55:0/3533385550 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b800dbf0 con 0x7f22c81563e0 2023-10-06T04:49:25.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.871+0000 7f22c559a700 1 -- 172.21.15.55:0/3533385550 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b800dd50 con 0x7f22c81563e0 2023-10-06T04:49:25.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.872+0000 7f22c559a700 1 -- 172.21.15.55:0/3533385550 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b80196d0 con 0x7f22c81563e0 2023-10-06T04:49:25.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.872+0000 7f22c759e700 1 -- 172.21.15.55:0/3533385550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c81563e0 msgr2=0x7f22c81567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:25.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.872+0000 7f22c759e700 1 --2- 172.21.15.55:0/3533385550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c81563e0 0x7f22c81567b0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f22b800bc50 tx=0x7f22b800b4f0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.872+0000 7f22c759e700 1 -- 172.21.15.55:0/3533385550 shutdown_connections 2023-10-06T04:49:25.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.872+0000 7f22c759e700 1 --2- 172.21.15.55:0/3533385550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22c806eeb0 0x7f22c80c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.872+0000 7f22c759e700 1 --2- 172.21.15.55:0/3533385550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c81563e0 0x7f22c81567b0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.872+0000 7f22c759e700 1 -- 172.21.15.55:0/3533385550 >> 172.21.15.55:0/3533385550 conn(0x7f22c80ccca0 msgr2=0x7f22c80c8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:25.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.872+0000 7f22c759e700 1 -- 172.21.15.55:0/3533385550 shutdown_connections 2023-10-06T04:49:25.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.872+0000 7f22c759e700 1 -- 172.21.15.55:0/3533385550 wait complete. 2023-10-06T04:49:25.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.874+0000 7f22c759e700 1 Processor -- start 2023-10-06T04:49:25.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.874+0000 7f22c759e700 1 -- start start 2023-10-06T04:49:25.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22c806eeb0 0x7f22c80af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22c80c9350 0x7f22c80c2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c81563e0 0x7f22c81527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80c19a0 con 0x7f22c81563e0 2023-10-06T04:49:25.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c8158b50 con 0x7f22c806eeb0 2023-10-06T04:49:25.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80c2ad0 con 0x7f22c80c9350 2023-10-06T04:49:25.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c81563e0 0x7f22c81527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:25.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c81563e0 0x7f22c81527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34486/0 (socket says 172.21.15.55:34486) 2023-10-06T04:49:25.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c6d9d700 1 -- 172.21.15.55:0/3849088006 learned_addr learned my addr 172.21.15.55:0/3849088006 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:25.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c5d9b700 1 --2- 172.21.15.55:0/3849088006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22c80c9350 0x7f22c80c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:25.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c6d9d700 1 -- 172.21.15.55:0/3849088006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22c80c9350 msgr2=0x7f22c80c2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:25.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c6d9d700 1 --2- 172.21.15.55:0/3849088006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22c80c9350 0x7f22c80c2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c6d9d700 1 -- 172.21.15.55:0/3849088006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22c806eeb0 msgr2=0x7f22c80af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:25.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c6d9d700 1 --2- 172.21.15.55:0/3849088006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22c806eeb0 0x7f22c80af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:25.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c6d9d700 1 -- 172.21.15.55:0/3849088006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b800b790 con 0x7f22c81563e0 2023-10-06T04:49:25.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.875+0000 7f22c5d9b700 1 --2- 172.21.15.55:0/3849088006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22c80c9350 0x7f22c80c2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:49:25.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.876+0000 7f22c6d9d700 1 --2- 172.21.15.55:0/3849088006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c81563e0 0x7f22c81527e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009ec0 tx=0x7f22c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:25.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.876+0000 7f22b67fc700 1 -- 172.21.15.55:0/3849088006 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c000cc20 con 0x7f22c81563e0 2023-10-06T04:49:25.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.876+0000 7f22b67fc700 1 -- 172.21.15.55:0/3849088006 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c000f040 con 0x7f22c81563e0 2023-10-06T04:49:25.879 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.876+0000 7f22b67fc700 1 -- 172.21.15.55:0/3849088006 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0014480 con 0x7f22c81563e0 2023-10-06T04:49:25.880 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.876+0000 7f22c759e700 1 -- 172.21.15.55:0/3849088006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c8152d20 con 0x7f22c81563e0 2023-10-06T04:49:25.880 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.876+0000 7f22c759e700 1 -- 172.21.15.55:0/3849088006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c80af640 con 0x7f22c81563e0 2023-10-06T04:49:25.880 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.877+0000 7f22c759e700 1 -- 172.21.15.55:0/3849088006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c8158940 con 0x7f22c81563e0 2023-10-06T04:49:25.880 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.878+0000 7f22b67fc700 1 -- 172.21.15.55:0/3849088006 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f22c0014810 con 0x7f22c81563e0 2023-10-06T04:49:25.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.879+0000 7f22b67fc700 1 --2- 172.21.15.55:0/3849088006 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f22a8066f30 0x7f22a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:25.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.879+0000 7f22b67fc700 1 -- 172.21.15.55:0/3849088006 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f22c0085680 con 0x7f22c81563e0 2023-10-06T04:49:25.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.880+0000 7f22c659c700 1 --2- 172.21.15.55:0/3849088006 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f22a8066f30 0x7f22a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:25.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.882+0000 7f22c659c700 1 --2- 172.21.15.55:0/3849088006 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f22a8066f30 0x7f22a80693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f22b800bc50 tx=0x7f22b801b000 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:25.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:25.883+0000 7f22b67fc700 1 -- 172.21.15.55:0/3849088006 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22c0052560 con 0x7f22c81563e0 2023-10-06T04:49:26.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.165+0000 7f22c759e700 1 -- 172.21.15.55:0/3849088006 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f22c8158cf0 con 0x7f22c81563e0 2023-10-06T04:49:26.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.166+0000 7f22b67fc700 1 -- 172.21.15.55:0/3849088006 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1022 (secure 0 0 0) 0x7f22c0055b80 con 0x7f22c81563e0 2023-10-06T04:49:26.168 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:26.168 INFO:teuthology.orchestra.run.smithi055.stdout:{"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":4590,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":173,"num_osds":8,"num_up_osds":8,"osd_up_since":1696563187,"num_in_osds":8,"osd_in_since":1696563178,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":23,"data_bytes":592674,"bytes_used":2512609280,"bytes_avail":770581504000,"bytes_total":773094113280,"write_bytes_sec":2208,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":87,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"b","status":"up:active","gid":8740}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2023-10-06T04:38:50.699493+0000","services":{}},"progress_events":{}} 2023-10-06T04:49:26.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.168+0000 7f22c759e700 1 -- 172.21.15.55:0/3849088006 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f22a8066f30 msgr2=0x7f22a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:26.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.168+0000 7f22c759e700 1 --2- 172.21.15.55:0/3849088006 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f22a8066f30 0x7f22a80693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f22b800bc50 tx=0x7f22b801b000 comp rx=0 tx=0).stop 2023-10-06T04:49:26.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.168+0000 7f22c759e700 1 -- 172.21.15.55:0/3849088006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c81563e0 msgr2=0x7f22c81527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:26.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.168+0000 7f22c759e700 1 --2- 172.21.15.55:0/3849088006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c81563e0 0x7f22c81527e0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f22c0009ec0 tx=0x7f22c000b040 comp rx=0 tx=0).stop 2023-10-06T04:49:26.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.168+0000 7f22c759e700 1 -- 172.21.15.55:0/3849088006 shutdown_connections 2023-10-06T04:49:26.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.168+0000 7f22c759e700 1 --2- 172.21.15.55:0/3849088006 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f22a8066f30 0x7f22a80693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.168+0000 7f22c759e700 1 --2- 172.21.15.55:0/3849088006 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f22c81563e0 0x7f22c81527e0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.168+0000 7f22c759e700 1 --2- 172.21.15.55:0/3849088006 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f22c806eeb0 0x7f22c80af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.168+0000 7f22c759e700 1 --2- 172.21.15.55:0/3849088006 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f22c80c9350 0x7f22c80c2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.168+0000 7f22c759e700 1 -- 172.21.15.55:0/3849088006 >> 172.21.15.55:0/3849088006 conn(0x7f22c80ccca0 msgr2=0x7f22c80c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:26.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.168+0000 7f22c759e700 1 -- 172.21.15.55:0/3849088006 shutdown_connections 2023-10-06T04:49:26.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.168+0000 7f22c759e700 1 -- 172.21.15.55:0/3849088006 wait complete. 2023-10-06T04:49:26.188 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T04:49:26.188 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:49:26.298 INFO:tasks.ceph.mds.b.smithi163.stderr:2023-10-06T04:49:26.296+0000 7fa5a85f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:49:26.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.373+0000 7fcdeb39d700 1 Processor -- start 2023-10-06T04:49:26.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.373+0000 7fcdeb39d700 1 -- start start 2023-10-06T04:49:26.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.374+0000 7fcdeb39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcde4157270 0x7fcde4068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:26.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.374+0000 7fcdeb39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcde40688d0 0x7fcde40b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:26.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.374+0000 7fcdeb39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcde40b0a00 0x7fcde406a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:26.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.374+0000 7fcdeb39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde4068d50 con 0x7fcde40688d0 2023-10-06T04:49:26.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.374+0000 7fcdeb39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde415a980 con 0x7fcde4157270 2023-10-06T04:49:26.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.374+0000 7fcdeb39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde415a050 con 0x7fcde40b0a00 2023-10-06T04:49:26.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.375+0000 7fcde9b9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcde40688d0 0x7fcde40b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:26.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.375+0000 7fcde9b9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fcde40688d0 0x7fcde40b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34488/0 (socket says 172.21.15.55:34488) 2023-10-06T04:49:26.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.375+0000 7fcde9b9a700 1 -- 172.21.15.55:0/1370692546 learned_addr learned my addr 172.21.15.55:0/1370692546 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:26.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.375+0000 7fcdeab9c700 1 --2- 172.21.15.55:0/1370692546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcde40b0a00 0x7fcde406a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:26.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.375+0000 7fcde9b9a700 1 -- 172.21.15.55:0/1370692546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcde40b0a00 msgr2=0x7fcde406a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:26.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.375+0000 7fcde9b9a700 1 --2- 172.21.15.55:0/1370692546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcde40b0a00 0x7fcde406a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.375+0000 7fcde9b9a700 1 -- 172.21.15.55:0/1370692546 >> v2:172.21.15.163:3300/0 conn(0x7fcde4157270 msgr2=0x7fcde4068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:26.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.375+0000 7fcde9b9a700 1 --2- 172.21.15.55:0/1370692546 >> v2:172.21.15.163:3300/0 conn(0x7fcde4157270 0x7fcde4068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.375+0000 7fcde9b9a700 1 -- 172.21.15.55:0/1370692546 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde415a330 con 0x7fcde40688d0 2023-10-06T04:49:26.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.375+0000 7fcdeab9c700 1 --2- 172.21.15.55:0/1370692546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcde40b0a00 0x7fcde406a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:49:26.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.375+0000 7fcde9b9a700 1 --2- 172.21.15.55:0/1370692546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde40688d0 0x7fcde40b04c0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fcde40b3360 tx=0x7fcdd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9ed77fee322031 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:26.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.376+0000 7fcde9399700 1 -- 172.21.15.55:0/1370692546 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd400b860 con 0x7fcde40688d0 2023-10-06T04:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.376+0000 7fcde9399700 1 -- 172.21.15.55:0/1370692546 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd400b9c0 con 0x7fcde40688d0 2023-10-06T04:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.376+0000 7fcde9399700 1 -- 172.21.15.55:0/1370692546 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd400bd60 con 0x7fcde40688d0 2023-10-06T04:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.376+0000 7fcdeb39d700 1 -- 172.21.15.55:0/1370692546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde40688d0 msgr2=0x7fcde40b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.376+0000 7fcdeb39d700 1 --2- 172.21.15.55:0/1370692546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde40688d0 0x7fcde40b04c0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fcde40b3360 tx=0x7fcdd4009250 comp rx=0 tx=0).stop 2023-10-06T04:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.376+0000 7fcdeb39d700 1 -- 172.21.15.55:0/1370692546 shutdown_connections 2023-10-06T04:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.376+0000 7fcdeb39d700 1 --2- 172.21.15.55:0/1370692546 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcde40b0a00 0x7fcde406a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.376+0000 7fcdeb39d700 1 --2- 172.21.15.55:0/1370692546 >> v2:172.21.15.163:3300/0 conn(0x7fcde4157270 0x7fcde4068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.376+0000 7fcdeb39d700 1 --2- 172.21.15.55:0/1370692546 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde40688d0 0x7fcde40b04c0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.376+0000 7fcdeb39d700 1 -- 172.21.15.55:0/1370692546 >> 172.21.15.55:0/1370692546 conn(0x7fcde40cd310 msgr2=0x7fcde40cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:26.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.376+0000 7fcdeb39d700 1 -- 172.21.15.55:0/1370692546 shutdown_connections 2023-10-06T04:49:26.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.376+0000 7fcdeb39d700 1 -- 172.21.15.55:0/1370692546 wait complete. 2023-10-06T04:49:26.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.378+0000 7fcdeb39d700 1 Processor -- start 2023-10-06T04:49:26.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.378+0000 7fcdeb39d700 1 -- start start 2023-10-06T04:49:26.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.379+0000 7fcdeb39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcde40688d0 0x7fcde4152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:26.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.379+0000 7fcdeb39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcde40b0a00 0x7fcde41555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:26.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.379+0000 7fcdeb39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde4157270 0x7fcde4155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:26.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.379+0000 7fcdeb39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde415d460 con 0x7fcde4157270 2023-10-06T04:49:26.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.379+0000 7fcdeb39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde415c850 con 0x7fcde40688d0 2023-10-06T04:49:26.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.379+0000 7fcdeb39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde415cb30 con 0x7fcde40b0a00 2023-10-06T04:49:26.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.379+0000 7fcdeab9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde4157270 0x7fcde4155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:26.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.379+0000 7fcdeab9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde4157270 0x7fcde4155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34502/0 (socket says 172.21.15.55:34502) 2023-10-06T04:49:26.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.379+0000 7fcdeab9c700 1 -- 172.21.15.55:0/2876446458 learned_addr learned my addr 172.21.15.55:0/2876446458 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:26.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.379+0000 7fcde9b9a700 1 --2- 172.21.15.55:0/2876446458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcde40b0a00 0x7fcde41555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:26.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.379+0000 7fcdea39b700 1 --2- 172.21.15.55:0/2876446458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcde40688d0 0x7fcde4152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:26.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.380+0000 7fcdeab9c700 1 -- 172.21.15.55:0/2876446458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcde40b0a00 msgr2=0x7fcde41555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:26.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.380+0000 7fcdeab9c700 1 --2- 172.21.15.55:0/2876446458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcde40b0a00 0x7fcde41555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.380+0000 7fcdeab9c700 1 -- 172.21.15.55:0/2876446458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcde40688d0 msgr2=0x7fcde4152220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:26.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.380+0000 7fcdeab9c700 1 --2- 172.21.15.55:0/2876446458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcde40688d0 0x7fcde4152220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.380+0000 7fcdeab9c700 1 -- 172.21.15.55:0/2876446458 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd40094f0 con 0x7fcde4157270 2023-10-06T04:49:26.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.380+0000 7fcdeab9c700 1 --2- 172.21.15.55:0/2876446458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde4157270 0x7fcde4155b30 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00a380 tx=0x7fcddc007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:26.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.380+0000 7fcdda7fc700 1 -- 172.21.15.55:0/2876446458 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00ae10 con 0x7fcde4157270 2023-10-06T04:49:26.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.380+0000 7fcdda7fc700 1 -- 172.21.15.55:0/2876446458 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc00d650 con 0x7fcde4157270 2023-10-06T04:49:26.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.380+0000 7fcdda7fc700 1 -- 172.21.15.55:0/2876446458 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00da10 con 0x7fcde4157270 2023-10-06T04:49:26.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.380+0000 7fcdeb39d700 1 -- 172.21.15.55:0/2876446458 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde415ce10 con 0x7fcde4157270 2023-10-06T04:49:26.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.381+0000 7fcdeb39d700 1 -- 172.21.15.55:0/2876446458 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde4152760 con 0x7fcde4157270 2023-10-06T04:49:26.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.381+0000 7fcdeb39d700 1 -- 172.21.15.55:0/2876446458 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde415d5d0 con 0x7fcde4157270 2023-10-06T04:49:26.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.382+0000 7fcdda7fc700 1 -- 172.21.15.55:0/2876446458 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcddc01e3b0 con 0x7fcde4157270 2023-10-06T04:49:26.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.383+0000 7fcdda7fc700 1 --2- 172.21.15.55:0/2876446458 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fcdc8066f30 0x7fcdc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:26.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.383+0000 7fcdda7fc700 1 -- 172.21.15.55:0/2876446458 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fcddc0852d0 con 0x7fcde4157270 2023-10-06T04:49:26.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.384+0000 7fcdea39b700 1 --2- 172.21.15.55:0/2876446458 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fcdc8066f30 0x7fcdc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:26.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.384+0000 7fcdea39b700 1 --2- 172.21.15.55:0/2876446458 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fcdc8066f30 0x7fcdc80693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcde00099d0 tx=0x7fcde0008040 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:26.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.387+0000 7fcdda7fc700 1 -- 172.21.15.55:0/2876446458 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcddc0521b0 con 0x7fcde4157270 2023-10-06T04:49:26.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.584+0000 7fcdeb39d700 1 -- 172.21.15.55:0/2876446458 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcde415c5a0 con 0x7fcdc8066f30 2023-10-06T04:49:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.594+0000 7fcdda7fc700 1 -- 172.21.15.55:0/2876446458 <== mgr.8557 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311431 (secure 0 0 0) 0x7fcde415c5a0 con 0x7fcdc8066f30 2023-10-06T04:49:26.600 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.598+0000 7fcdeb39d700 1 -- 172.21.15.55:0/2876446458 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fcdc8066f30 msgr2=0x7fcdc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.598+0000 7fcdeb39d700 1 --2- 172.21.15.55:0/2876446458 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fcdc8066f30 0x7fcdc80693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcde00099d0 tx=0x7fcde0008040 comp rx=0 tx=0).stop 2023-10-06T04:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.598+0000 7fcdeb39d700 1 -- 172.21.15.55:0/2876446458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde4157270 msgr2=0x7fcde4155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.598+0000 7fcdeb39d700 1 --2- 172.21.15.55:0/2876446458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde4157270 0x7fcde4155b30 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00a380 tx=0x7fcddc007900 comp rx=0 tx=0).stop 2023-10-06T04:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.598+0000 7fcdeb39d700 1 -- 172.21.15.55:0/2876446458 shutdown_connections 2023-10-06T04:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.598+0000 7fcdeb39d700 1 --2- 172.21.15.55:0/2876446458 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fcdc8066f30 0x7fcdc80693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.598+0000 7fcdeb39d700 1 --2- 172.21.15.55:0/2876446458 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcde4157270 0x7fcde4155b30 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.598+0000 7fcdeb39d700 1 --2- 172.21.15.55:0/2876446458 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcde40688d0 0x7fcde4152220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.598+0000 7fcdeb39d700 1 --2- 172.21.15.55:0/2876446458 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcde40b0a00 0x7fcde41555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.599+0000 7fcdeb39d700 1 -- 172.21.15.55:0/2876446458 >> 172.21.15.55:0/2876446458 conn(0x7fcde40cd310 msgr2=0x7fcde4152bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.599+0000 7fcdeb39d700 1 -- 172.21.15.55:0/2876446458 shutdown_connections 2023-10-06T04:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:26.599+0000 7fcdeb39d700 1 -- 172.21.15.55:0/2876446458 wait complete. 2023-10-06T04:49:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T04:49:26.618 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T04:49:25.337829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":523,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2453720,"kb_used_data":18072,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752521000,"statfs":{"total":773094113280,"available":770581504000,"internally_reserved":0,"allocated":18505728,"data_stored":10233204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.027170"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346440+0000","last_change":"2023-10-06T04:49:20.341012+0000","last_active":"2023-10-06T04:49:20.346440+0000","last_peered":"2023-10-06T04:49:20.346440+0000","last_clean":"2023-10-06T04:49:20.346440+0000","last_became_active":"2023-10-06T04:49:20.340740+0000","last_became_peered":"2023-10-06T04:49:20.340740+0000","last_unstale":"2023-10-06T04:49:20.346440+0000","last_undegraded":"2023-10-06T04:49:20.346440+0000","last_fullsized":"2023-10-06T04:49:20.346440+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341092+0000","last_change":"2023-10-06T04:49:20.336745+0000","last_active":"2023-10-06T04:49:20.341092+0000","last_peered":"2023-10-06T04:49:20.341092+0000","last_clean":"2023-10-06T04:49:20.341092+0000","last_became_active":"2023-10-06T04:49:20.336500+0000","last_became_peered":"2023-10-06T04:49:20.336500+0000","last_unstale":"2023-10-06T04:49:20.341092+0000","last_undegraded":"2023-10-06T04:49:20.341092+0000","last_fullsized":"2023-10-06T04:49:20.341092+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:28.214277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341469+0000","last_change":"2023-10-06T04:49:20.334944+0000","last_active":"2023-10-06T04:49:20.341469+0000","last_peered":"2023-10-06T04:49:20.341469+0000","last_clean":"2023-10-06T04:49:20.341469+0000","last_became_active":"2023-10-06T04:49:20.333924+0000","last_became_peered":"2023-10-06T04:49:20.333924+0000","last_unstale":"2023-10-06T04:49:20.341469+0000","last_undegraded":"2023-10-06T04:49:20.341469+0000","last_fullsized":"2023-10-06T04:49:20.341469+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341150+0000","last_change":"2023-10-06T04:49:20.333903+0000","last_active":"2023-10-06T04:49:20.341150+0000","last_peered":"2023-10-06T04:49:20.341150+0000","last_clean":"2023-10-06T04:49:20.341150+0000","last_became_active":"2023-10-06T04:49:20.333572+0000","last_became_peered":"2023-10-06T04:49:20.333572+0000","last_unstale":"2023-10-06T04:49:20.341150+0000","last_undegraded":"2023-10-06T04:49:20.341150+0000","last_fullsized":"2023-10-06T04:49:20.341150+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:49.977768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339683+0000","last_change":"2023-10-06T04:49:20.338306+0000","last_active":"2023-10-06T04:49:20.339683+0000","last_peered":"2023-10-06T04:49:20.339683+0000","last_clean":"2023-10-06T04:49:20.339683+0000","last_became_active":"2023-10-06T04:49:20.338034+0000","last_became_peered":"2023-10-06T04:49:20.338034+0000","last_unstale":"2023-10-06T04:49:20.339683+0000","last_undegraded":"2023-10-06T04:49:20.339683+0000","last_fullsized":"2023-10-06T04:49:20.339683+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339447+0000","last_change":"2023-10-06T04:49:20.336417+0000","last_active":"2023-10-06T04:49:20.339447+0000","last_peered":"2023-10-06T04:49:20.339447+0000","last_clean":"2023-10-06T04:49:20.339447+0000","last_became_active":"2023-10-06T04:49:20.334348+0000","last_became_peered":"2023-10-06T04:49:20.334348+0000","last_unstale":"2023-10-06T04:49:20.339447+0000","last_undegraded":"2023-10-06T04:49:20.339447+0000","last_fullsized":"2023-10-06T04:49:20.339447+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:54.036275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.345389+0000","last_change":"2023-10-06T04:49:20.339825+0000","last_active":"2023-10-06T04:49:20.345389+0000","last_peered":"2023-10-06T04:49:20.345389+0000","last_clean":"2023-10-06T04:49:20.345389+0000","last_became_active":"2023-10-06T04:49:20.339565+0000","last_became_peered":"2023-10-06T04:49:20.339565+0000","last_unstale":"2023-10-06T04:49:20.345389+0000","last_undegraded":"2023-10-06T04:49:20.345389+0000","last_fullsized":"2023-10-06T04:49:20.345389+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:54.036275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347030+0000","last_change":"2023-10-06T04:49:20.336205+0000","last_active":"2023-10-06T04:49:20.347030+0000","last_peered":"2023-10-06T04:49:20.347030+0000","last_clean":"2023-10-06T04:49:20.347030+0000","last_became_active":"2023-10-06T04:49:20.334936+0000","last_became_peered":"2023-10-06T04:49:20.334936+0000","last_unstale":"2023-10-06T04:49:20.347030+0000","last_undegraded":"2023-10-06T04:49:20.347030+0000","last_fullsized":"2023-10-06T04:49:20.347030+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:34.988176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.340009+0000","last_change":"2023-10-06T04:49:20.336646+0000","last_active":"2023-10-06T04:49:20.340009+0000","last_peered":"2023-10-06T04:49:20.340009+0000","last_clean":"2023-10-06T04:49:20.340009+0000","last_became_active":"2023-10-06T04:49:20.334441+0000","last_became_peered":"2023-10-06T04:49:20.334441+0000","last_unstale":"2023-10-06T04:49:20.340009+0000","last_undegraded":"2023-10-06T04:49:20.340009+0000","last_fullsized":"2023-10-06T04:49:20.340009+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:23.582332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333816+0000","last_change":"2023-10-06T04:49:20.331180+0000","last_active":"2023-10-06T04:49:20.333816+0000","last_peered":"2023-10-06T04:49:20.333816+0000","last_clean":"2023-10-06T04:49:20.333816+0000","last_became_active":"2023-10-06T04:49:20.330817+0000","last_became_peered":"2023-10-06T04:49:20.330817+0000","last_unstale":"2023-10-06T04:49:20.333816+0000","last_undegraded":"2023-10-06T04:49:20.333816+0000","last_fullsized":"2023-10-06T04:49:20.333816+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.336853+0000","last_change":"2023-10-06T04:49:20.336069+0000","last_active":"2023-10-06T04:49:20.336853+0000","last_peered":"2023-10-06T04:49:20.336853+0000","last_clean":"2023-10-06T04:49:20.336853+0000","last_became_active":"2023-10-06T04:49:20.335808+0000","last_became_peered":"2023-10-06T04:49:20.335808+0000","last_unstale":"2023-10-06T04:49:20.336853+0000","last_undegraded":"2023-10-06T04:49:20.336853+0000","last_fullsized":"2023-10-06T04:49:20.336853+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:53.477421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.334968+0000","last_change":"2023-10-06T04:49:20.328201+0000","last_active":"2023-10-06T04:49:20.334968+0000","last_peered":"2023-10-06T04:49:20.334968+0000","last_clean":"2023-10-06T04:49:20.334968+0000","last_became_active":"2023-10-06T04:49:20.327560+0000","last_became_peered":"2023-10-06T04:49:20.327560+0000","last_unstale":"2023-10-06T04:49:20.334968+0000","last_undegraded":"2023-10-06T04:49:20.334968+0000","last_fullsized":"2023-10-06T04:49:20.334968+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:40.721698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333739+0000","last_change":"2023-10-06T04:49:20.331130+0000","last_active":"2023-10-06T04:49:20.333739+0000","last_peered":"2023-10-06T04:49:20.333739+0000","last_clean":"2023-10-06T04:49:20.333739+0000","last_became_active":"2023-10-06T04:49:20.330637+0000","last_became_peered":"2023-10-06T04:49:20.330637+0000","last_unstale":"2023-10-06T04:49:20.333739+0000","last_undegraded":"2023-10-06T04:49:20.333739+0000","last_fullsized":"2023-10-06T04:49:20.333739+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341277+0000","last_change":"2023-10-06T04:49:20.337139+0000","last_active":"2023-10-06T04:49:20.341277+0000","last_peered":"2023-10-06T04:49:20.341277+0000","last_clean":"2023-10-06T04:49:20.341277+0000","last_became_active":"2023-10-06T04:49:20.336396+0000","last_became_peered":"2023-10-06T04:49:20.336396+0000","last_unstale":"2023-10-06T04:49:20.341277+0000","last_undegraded":"2023-10-06T04:49:20.341277+0000","last_fullsized":"2023-10-06T04:49:20.341277+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:22.496792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341177+0000","last_change":"2023-10-06T04:49:20.337332+0000","last_active":"2023-10-06T04:49:20.341177+0000","last_peered":"2023-10-06T04:49:20.341177+0000","last_clean":"2023-10-06T04:49:20.341177+0000","last_became_active":"2023-10-06T04:49:20.337091+0000","last_became_peered":"2023-10-06T04:49:20.337091+0000","last_unstale":"2023-10-06T04:49:20.341177+0000","last_undegraded":"2023-10-06T04:49:20.341177+0000","last_fullsized":"2023-10-06T04:49:20.341177+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:57.445018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333411+0000","last_change":"2023-10-06T04:49:20.329112+0000","last_active":"2023-10-06T04:49:20.333411+0000","last_peered":"2023-10-06T04:49:20.333411+0000","last_clean":"2023-10-06T04:49:20.333411+0000","last_became_active":"2023-10-06T04:49:20.328156+0000","last_became_peered":"2023-10-06T04:49:20.328156+0000","last_unstale":"2023-10-06T04:49:20.333411+0000","last_undegraded":"2023-10-06T04:49:20.333411+0000","last_fullsized":"2023-10-06T04:49:20.333411+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:25.017277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341591+0000","last_change":"2023-10-06T04:49:20.334020+0000","last_active":"2023-10-06T04:49:20.341591+0000","last_peered":"2023-10-06T04:49:20.341591+0000","last_clean":"2023-10-06T04:49:20.341591+0000","last_became_active":"2023-10-06T04:49:20.333516+0000","last_became_peered":"2023-10-06T04:49:20.333516+0000","last_unstale":"2023-10-06T04:49:20.341591+0000","last_undegraded":"2023-10-06T04:49:20.341591+0000","last_fullsized":"2023-10-06T04:49:20.341591+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:43.552397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347965+0000","last_change":"2023-10-06T04:49:20.346422+0000","last_active":"2023-10-06T04:49:20.347965+0000","last_peered":"2023-10-06T04:49:20.347965+0000","last_clean":"2023-10-06T04:49:20.347965+0000","last_became_active":"2023-10-06T04:49:20.346126+0000","last_became_peered":"2023-10-06T04:49:20.346126+0000","last_unstale":"2023-10-06T04:49:20.347965+0000","last_undegraded":"2023-10-06T04:49:20.347965+0000","last_fullsized":"2023-10-06T04:49:20.347965+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:58.644879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355577+0000","last_change":"2023-10-06T04:49:20.350471+0000","last_active":"2023-10-06T04:49:20.355577+0000","last_peered":"2023-10-06T04:49:20.355577+0000","last_clean":"2023-10-06T04:49:20.355577+0000","last_became_active":"2023-10-06T04:49:20.350195+0000","last_became_peered":"2023-10-06T04:49:20.350195+0000","last_unstale":"2023-10-06T04:49:20.355577+0000","last_undegraded":"2023-10-06T04:49:20.355577+0000","last_fullsized":"2023-10-06T04:49:20.355577+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:12.727118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339262+0000","last_change":"2023-10-06T04:49:20.334837+0000","last_active":"2023-10-06T04:49:20.339262+0000","last_peered":"2023-10-06T04:49:20.339262+0000","last_clean":"2023-10-06T04:49:20.339262+0000","last_became_active":"2023-10-06T04:49:20.334580+0000","last_became_peered":"2023-10-06T04:49:20.334580+0000","last_unstale":"2023-10-06T04:49:20.339262+0000","last_undegraded":"2023-10-06T04:49:20.339262+0000","last_fullsized":"2023-10-06T04:49:20.339262+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:02.862072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.338389+0000","last_change":"2023-10-06T04:49:20.334048+0000","last_active":"2023-10-06T04:49:20.338389+0000","last_peered":"2023-10-06T04:49:20.338389+0000","last_clean":"2023-10-06T04:49:20.338389+0000","last_became_active":"2023-10-06T04:49:20.333761+0000","last_became_peered":"2023-10-06T04:49:20.333761+0000","last_unstale":"2023-10-06T04:49:20.338389+0000","last_undegraded":"2023-10-06T04:49:20.338389+0000","last_fullsized":"2023-10-06T04:49:20.338389+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344972+0000","last_change":"2023-10-06T04:49:20.336537+0000","last_active":"2023-10-06T04:49:20.344972+0000","last_peered":"2023-10-06T04:49:20.344972+0000","last_clean":"2023-10-06T04:49:20.344972+0000","last_became_active":"2023-10-06T04:49:20.336251+0000","last_became_peered":"2023-10-06T04:49:20.336251+0000","last_unstale":"2023-10-06T04:49:20.344972+0000","last_undegraded":"2023-10-06T04:49:20.344972+0000","last_fullsized":"2023-10-06T04:49:20.344972+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347191+0000","last_change":"2023-10-06T04:49:20.344026+0000","last_active":"2023-10-06T04:49:20.347191+0000","last_peered":"2023-10-06T04:49:20.347191+0000","last_clean":"2023-10-06T04:49:20.347191+0000","last_became_active":"2023-10-06T04:49:20.343761+0000","last_became_peered":"2023-10-06T04:49:20.343761+0000","last_unstale":"2023-10-06T04:49:20.347191+0000","last_undegraded":"2023-10-06T04:49:20.347191+0000","last_fullsized":"2023-10-06T04:49:20.347191+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:07.881803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.329722+0000","last_change":"2023-10-06T04:49:20.326750+0000","last_active":"2023-10-06T04:49:20.329722+0000","last_peered":"2023-10-06T04:49:20.329722+0000","last_clean":"2023-10-06T04:49:20.329722+0000","last_became_active":"2023-10-06T04:49:20.326466+0000","last_became_peered":"2023-10-06T04:49:20.326466+0000","last_unstale":"2023-10-06T04:49:20.329722+0000","last_undegraded":"2023-10-06T04:49:20.329722+0000","last_fullsized":"2023-10-06T04:49:20.329722+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:52.748607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.345875+0000","last_change":"2023-10-06T04:49:20.337383+0000","last_active":"2023-10-06T04:49:20.345875+0000","last_peered":"2023-10-06T04:49:20.345875+0000","last_clean":"2023-10-06T04:49:20.345875+0000","last_became_active":"2023-10-06T04:49:20.337130+0000","last_became_peered":"2023-10-06T04:49:20.337130+0000","last_unstale":"2023-10-06T04:49:20.345875+0000","last_undegraded":"2023-10-06T04:49:20.345875+0000","last_fullsized":"2023-10-06T04:49:20.345875+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355403+0000","last_change":"2023-10-06T04:49:20.347903+0000","last_active":"2023-10-06T04:49:20.355403+0000","last_peered":"2023-10-06T04:49:20.355403+0000","last_clean":"2023-10-06T04:49:20.355403+0000","last_became_active":"2023-10-06T04:49:20.346956+0000","last_became_peered":"2023-10-06T04:49:20.346956+0000","last_unstale":"2023-10-06T04:49:20.355403+0000","last_undegraded":"2023-10-06T04:49:20.355403+0000","last_fullsized":"2023-10-06T04:49:20.355403+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:20.026936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339401+0000","last_change":"2023-10-06T04:49:20.336472+0000","last_active":"2023-10-06T04:49:20.339401+0000","last_peered":"2023-10-06T04:49:20.339401+0000","last_clean":"2023-10-06T04:49:20.339401+0000","last_became_active":"2023-10-06T04:49:20.334373+0000","last_became_peered":"2023-10-06T04:49:20.334373+0000","last_unstale":"2023-10-06T04:49:20.339401+0000","last_undegraded":"2023-10-06T04:49:20.339401+0000","last_fullsized":"2023-10-06T04:49:20.339401+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:20.026936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346959+0000","last_change":"2023-10-06T04:49:20.344850+0000","last_active":"2023-10-06T04:49:20.346959+0000","last_peered":"2023-10-06T04:49:20.346959+0000","last_clean":"2023-10-06T04:49:20.346959+0000","last_became_active":"2023-10-06T04:49:20.343891+0000","last_became_peered":"2023-10-06T04:49:20.343891+0000","last_unstale":"2023-10-06T04:49:20.346959+0000","last_undegraded":"2023-10-06T04:49:20.346959+0000","last_fullsized":"2023-10-06T04:49:20.346959+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:28.957190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347052+0000","last_change":"2023-10-06T04:49:20.343469+0000","last_active":"2023-10-06T04:49:20.347052+0000","last_peered":"2023-10-06T04:49:20.347052+0000","last_clean":"2023-10-06T04:49:20.347052+0000","last_became_active":"2023-10-06T04:49:20.343165+0000","last_became_peered":"2023-10-06T04:49:20.343165+0000","last_unstale":"2023-10-06T04:49:20.347052+0000","last_undegraded":"2023-10-06T04:49:20.347052+0000","last_fullsized":"2023-10-06T04:49:20.347052+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:12.727118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347992+0000","last_change":"2023-10-06T04:49:20.344409+0000","last_active":"2023-10-06T04:49:20.347992+0000","last_peered":"2023-10-06T04:49:20.347992+0000","last_clean":"2023-10-06T04:49:20.347992+0000","last_became_active":"2023-10-06T04:49:20.343734+0000","last_became_peered":"2023-10-06T04:49:20.343734+0000","last_unstale":"2023-10-06T04:49:20.347992+0000","last_undegraded":"2023-10-06T04:49:20.347992+0000","last_fullsized":"2023-10-06T04:49:20.347992+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:27.555498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333496+0000","last_change":"2023-10-06T04:49:20.327245+0000","last_active":"2023-10-06T04:49:20.333496+0000","last_peered":"2023-10-06T04:49:20.333496+0000","last_clean":"2023-10-06T04:49:20.333496+0000","last_became_active":"2023-10-06T04:49:20.326914+0000","last_became_peered":"2023-10-06T04:49:20.326914+0000","last_unstale":"2023-10-06T04:49:20.333496+0000","last_undegraded":"2023-10-06T04:49:20.333496+0000","last_fullsized":"2023-10-06T04:49:20.333496+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:19.514355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347097+0000","last_change":"2023-10-06T04:49:20.339379+0000","last_active":"2023-10-06T04:49:20.347097+0000","last_peered":"2023-10-06T04:49:20.347097+0000","last_clean":"2023-10-06T04:49:20.347097+0000","last_became_active":"2023-10-06T04:49:20.339031+0000","last_became_peered":"2023-10-06T04:49:20.339031+0000","last_unstale":"2023-10-06T04:49:20.347097+0000","last_undegraded":"2023-10-06T04:49:20.347097+0000","last_fullsized":"2023-10-06T04:49:20.347097+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:51.900026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341007+0000","last_change":"2023-10-06T04:49:20.333556+0000","last_active":"2023-10-06T04:49:20.341007+0000","last_peered":"2023-10-06T04:49:20.341007+0000","last_clean":"2023-10-06T04:49:20.341007+0000","last_became_active":"2023-10-06T04:49:20.333251+0000","last_became_peered":"2023-10-06T04:49:20.333251+0000","last_unstale":"2023-10-06T04:49:20.341007+0000","last_undegraded":"2023-10-06T04:49:20.341007+0000","last_fullsized":"2023-10-06T04:49:20.341007+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"172'4","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344554+0000","last_change":"2023-10-06T04:49:19.320190+0000","last_active":"2023-10-06T04:49:20.344554+0000","last_peered":"2023-10-06T04:49:20.344554+0000","last_clean":"2023-10-06T04:49:20.344554+0000","last_became_active":"2023-10-06T04:49:19.319917+0000","last_became_peered":"2023-10-06T04:49:19.319917+0000","last_unstale":"2023-10-06T04:49:20.344554+0000","last_undegraded":"2023-10-06T04:49:20.344554+0000","last_fullsized":"2023-10-06T04:49:20.344554+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:26.266062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341489+0000","last_change":"2023-10-06T04:49:20.336237+0000","last_active":"2023-10-06T04:49:20.341489+0000","last_peered":"2023-10-06T04:49:20.341489+0000","last_clean":"2023-10-06T04:49:20.341489+0000","last_became_active":"2023-10-06T04:49:20.335986+0000","last_became_peered":"2023-10-06T04:49:20.335986+0000","last_unstale":"2023-10-06T04:49:20.341489+0000","last_undegraded":"2023-10-06T04:49:20.341489+0000","last_fullsized":"2023-10-06T04:49:20.341489+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355736+0000","last_change":"2023-10-06T04:49:19.335778+0000","last_active":"2023-10-06T04:49:20.355736+0000","last_peered":"2023-10-06T04:49:20.355736+0000","last_clean":"2023-10-06T04:49:20.355736+0000","last_became_active":"2023-10-06T04:49:19.335470+0000","last_became_peered":"2023-10-06T04:49:19.335470+0000","last_unstale":"2023-10-06T04:49:20.355736+0000","last_undegraded":"2023-10-06T04:49:20.355736+0000","last_fullsized":"2023-10-06T04:49:20.355736+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:01.854229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347502+0000","last_change":"2023-10-06T04:49:20.339407+0000","last_active":"2023-10-06T04:49:20.347502+0000","last_peered":"2023-10-06T04:49:20.347502+0000","last_clean":"2023-10-06T04:49:20.347502+0000","last_became_active":"2023-10-06T04:49:20.335811+0000","last_became_peered":"2023-10-06T04:49:20.335811+0000","last_unstale":"2023-10-06T04:49:20.347502+0000","last_undegraded":"2023-10-06T04:49:20.347502+0000","last_fullsized":"2023-10-06T04:49:20.347502+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:10.637938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356161+0000","last_change":"2023-10-06T04:49:19.335543+0000","last_active":"2023-10-06T04:49:20.356161+0000","last_peered":"2023-10-06T04:49:20.356161+0000","last_clean":"2023-10-06T04:49:20.356161+0000","last_became_active":"2023-10-06T04:49:19.335265+0000","last_became_peered":"2023-10-06T04:49:19.335265+0000","last_unstale":"2023-10-06T04:49:20.356161+0000","last_undegraded":"2023-10-06T04:49:20.356161+0000","last_fullsized":"2023-10-06T04:49:20.356161+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:41.377516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350856+0000","last_change":"2023-10-06T04:49:20.346902+0000","last_active":"2023-10-06T04:49:20.350856+0000","last_peered":"2023-10-06T04:49:20.350856+0000","last_clean":"2023-10-06T04:49:20.350856+0000","last_became_active":"2023-10-06T04:49:20.346594+0000","last_became_peered":"2023-10-06T04:49:20.346594+0000","last_unstale":"2023-10-06T04:49:20.350856+0000","last_undegraded":"2023-10-06T04:49:20.350856+0000","last_fullsized":"2023-10-06T04:49:20.350856+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:00.206328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.349018+0000","last_change":"2023-10-06T04:49:19.328489+0000","last_active":"2023-10-06T04:49:20.349018+0000","last_peered":"2023-10-06T04:49:20.349018+0000","last_clean":"2023-10-06T04:49:20.349018+0000","last_became_active":"2023-10-06T04:49:19.328220+0000","last_became_peered":"2023-10-06T04:49:19.328220+0000","last_unstale":"2023-10-06T04:49:20.349018+0000","last_undegraded":"2023-10-06T04:49:20.349018+0000","last_fullsized":"2023-10-06T04:49:20.349018+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:26.266062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.337318+0000","last_change":"2023-10-06T04:49:20.330590+0000","last_active":"2023-10-06T04:49:20.337318+0000","last_peered":"2023-10-06T04:49:20.337318+0000","last_clean":"2023-10-06T04:49:20.337318+0000","last_became_active":"2023-10-06T04:49:20.330306+0000","last_became_peered":"2023-10-06T04:49:20.330306+0000","last_unstale":"2023-10-06T04:49:20.337318+0000","last_undegraded":"2023-10-06T04:49:20.337318+0000","last_fullsized":"2023-10-06T04:49:20.337318+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:42.535859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.351406+0000","last_change":"2023-10-06T04:49:19.324990+0000","last_active":"2023-10-06T04:49:20.351406+0000","last_peered":"2023-10-06T04:49:20.351406+0000","last_clean":"2023-10-06T04:49:20.351406+0000","last_became_active":"2023-10-06T04:49:19.324681+0000","last_became_peered":"2023-10-06T04:49:19.324681+0000","last_unstale":"2023-10-06T04:49:20.351406+0000","last_undegraded":"2023-10-06T04:49:20.351406+0000","last_fullsized":"2023-10-06T04:49:20.351406+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:40.637063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341432+0000","last_change":"2023-10-06T04:49:20.334543+0000","last_active":"2023-10-06T04:49:20.341432+0000","last_peered":"2023-10-06T04:49:20.341432+0000","last_clean":"2023-10-06T04:49:20.341432+0000","last_became_active":"2023-10-06T04:49:20.333665+0000","last_became_peered":"2023-10-06T04:49:20.333665+0000","last_unstale":"2023-10-06T04:49:20.341432+0000","last_undegraded":"2023-10-06T04:49:20.341432+0000","last_fullsized":"2023-10-06T04:49:20.341432+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:20.869185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347200+0000","last_change":"2023-10-06T04:49:19.330786+0000","last_active":"2023-10-06T04:49:20.347200+0000","last_peered":"2023-10-06T04:49:20.347200+0000","last_clean":"2023-10-06T04:49:20.347200+0000","last_became_active":"2023-10-06T04:49:19.330506+0000","last_became_peered":"2023-10-06T04:49:19.330506+0000","last_unstale":"2023-10-06T04:49:20.347200+0000","last_undegraded":"2023-10-06T04:49:20.347200+0000","last_fullsized":"2023-10-06T04:49:20.347200+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:54.415706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333277+0000","last_change":"2023-10-06T04:49:20.332521+0000","last_active":"2023-10-06T04:49:20.333277+0000","last_peered":"2023-10-06T04:49:20.333277+0000","last_clean":"2023-10-06T04:49:20.333277+0000","last_became_active":"2023-10-06T04:49:20.332264+0000","last_became_peered":"2023-10-06T04:49:20.332264+0000","last_unstale":"2023-10-06T04:49:20.333277+0000","last_undegraded":"2023-10-06T04:49:20.333277+0000","last_fullsized":"2023-10-06T04:49:20.333277+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:16.543656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332961+0000","last_change":"2023-10-06T04:49:19.332296+0000","last_active":"2023-10-06T04:49:20.332961+0000","last_peered":"2023-10-06T04:49:20.332961+0000","last_clean":"2023-10-06T04:49:20.332961+0000","last_became_active":"2023-10-06T04:49:19.332022+0000","last_became_peered":"2023-10-06T04:49:19.332022+0000","last_unstale":"2023-10-06T04:49:20.332961+0000","last_undegraded":"2023-10-06T04:49:20.332961+0000","last_fullsized":"2023-10-06T04:49:20.332961+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:18.846190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350851+0000","last_change":"2023-10-06T04:49:20.341492+0000","last_active":"2023-10-06T04:49:20.350851+0000","last_peered":"2023-10-06T04:49:20.350851+0000","last_clean":"2023-10-06T04:49:20.350851+0000","last_became_active":"2023-10-06T04:49:20.341257+0000","last_became_peered":"2023-10-06T04:49:20.341257+0000","last_unstale":"2023-10-06T04:49:20.350851+0000","last_undegraded":"2023-10-06T04:49:20.350851+0000","last_fullsized":"2023-10-06T04:49:20.350851+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:28.214277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.315096+0000","last_change":"2023-10-06T04:49:19.324312+0000","last_active":"2023-10-06T04:49:20.315096+0000","last_peered":"2023-10-06T04:49:20.315096+0000","last_clean":"2023-10-06T04:49:20.315096+0000","last_became_active":"2023-10-06T04:49:19.324046+0000","last_became_peered":"2023-10-06T04:49:19.324046+0000","last_unstale":"2023-10-06T04:49:20.315096+0000","last_undegraded":"2023-10-06T04:49:20.315096+0000","last_fullsized":"2023-10-06T04:49:20.315096+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:49.588344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344426+0000","last_change":"2023-10-06T04:49:20.341875+0000","last_active":"2023-10-06T04:49:20.344426+0000","last_peered":"2023-10-06T04:49:20.344426+0000","last_clean":"2023-10-06T04:49:20.344426+0000","last_became_active":"2023-10-06T04:49:20.341312+0000","last_became_peered":"2023-10-06T04:49:20.341312+0000","last_unstale":"2023-10-06T04:49:20.344426+0000","last_undegraded":"2023-10-06T04:49:20.344426+0000","last_fullsized":"2023-10-06T04:49:20.344426+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:07.881803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.351368+0000","last_change":"2023-10-06T04:49:19.331266+0000","last_active":"2023-10-06T04:49:20.351368+0000","last_peered":"2023-10-06T04:49:20.351368+0000","last_clean":"2023-10-06T04:49:20.351368+0000","last_became_active":"2023-10-06T04:49:19.331010+0000","last_became_peered":"2023-10-06T04:49:19.331010+0000","last_unstale":"2023-10-06T04:49:20.351368+0000","last_undegraded":"2023-10-06T04:49:20.351368+0000","last_fullsized":"2023-10-06T04:49:20.351368+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:41.647975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355558+0000","last_change":"2023-10-06T04:49:20.350352+0000","last_active":"2023-10-06T04:49:20.355558+0000","last_peered":"2023-10-06T04:49:20.355558+0000","last_clean":"2023-10-06T04:49:20.355558+0000","last_became_active":"2023-10-06T04:49:20.350084+0000","last_became_peered":"2023-10-06T04:49:20.350084+0000","last_unstale":"2023-10-06T04:49:20.355558+0000","last_undegraded":"2023-10-06T04:49:20.355558+0000","last_fullsized":"2023-10-06T04:49:20.355558+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:54.488863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.353329+0000","last_change":"2023-10-06T04:49:19.333467+0000","last_active":"2023-10-06T04:49:20.353329+0000","last_peered":"2023-10-06T04:49:20.353329+0000","last_clean":"2023-10-06T04:49:20.353329+0000","last_became_active":"2023-10-06T04:49:19.333191+0000","last_became_peered":"2023-10-06T04:49:19.333191+0000","last_unstale":"2023-10-06T04:49:20.353329+0000","last_undegraded":"2023-10-06T04:49:20.353329+0000","last_fullsized":"2023-10-06T04:49:20.353329+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:50.631194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346482+0000","last_change":"2023-10-06T04:49:20.340075+0000","last_active":"2023-10-06T04:49:20.346482+0000","last_peered":"2023-10-06T04:49:20.346482+0000","last_clean":"2023-10-06T04:49:20.346482+0000","last_became_active":"2023-10-06T04:49:20.339781+0000","last_became_peered":"2023-10-06T04:49:20.339781+0000","last_unstale":"2023-10-06T04:49:20.346482+0000","last_undegraded":"2023-10-06T04:49:20.346482+0000","last_fullsized":"2023-10-06T04:49:20.346482+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:57.445018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344697+0000","last_change":"2023-10-06T04:49:19.324434+0000","last_active":"2023-10-06T04:49:20.344697+0000","last_peered":"2023-10-06T04:49:20.344697+0000","last_clean":"2023-10-06T04:49:20.344697+0000","last_became_active":"2023-10-06T04:49:19.324162+0000","last_became_peered":"2023-10-06T04:49:19.324162+0000","last_unstale":"2023-10-06T04:49:20.344697+0000","last_undegraded":"2023-10-06T04:49:20.344697+0000","last_fullsized":"2023-10-06T04:49:20.344697+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350745+0000","last_change":"2023-10-06T04:49:20.345980+0000","last_active":"2023-10-06T04:49:20.350745+0000","last_peered":"2023-10-06T04:49:20.350745+0000","last_clean":"2023-10-06T04:49:20.350745+0000","last_became_active":"2023-10-06T04:49:20.345691+0000","last_became_peered":"2023-10-06T04:49:20.345691+0000","last_unstale":"2023-10-06T04:49:20.350745+0000","last_undegraded":"2023-10-06T04:49:20.350745+0000","last_fullsized":"2023-10-06T04:49:20.350745+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:23.582332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.319623+0000","last_change":"2023-10-06T04:49:19.330962+0000","last_active":"2023-10-06T04:49:20.319623+0000","last_peered":"2023-10-06T04:49:20.319623+0000","last_clean":"2023-10-06T04:49:20.319623+0000","last_became_active":"2023-10-06T04:49:19.330616+0000","last_became_peered":"2023-10-06T04:49:19.330616+0000","last_unstale":"2023-10-06T04:49:20.319623+0000","last_undegraded":"2023-10-06T04:49:20.319623+0000","last_fullsized":"2023-10-06T04:49:20.319623+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:04.272287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355580+0000","last_change":"2023-10-06T04:49:20.350356+0000","last_active":"2023-10-06T04:49:20.355580+0000","last_peered":"2023-10-06T04:49:20.355580+0000","last_clean":"2023-10-06T04:49:20.355580+0000","last_became_active":"2023-10-06T04:49:20.350105+0000","last_became_peered":"2023-10-06T04:49:20.350105+0000","last_unstale":"2023-10-06T04:49:20.355580+0000","last_undegraded":"2023-10-06T04:49:20.355580+0000","last_fullsized":"2023-10-06T04:49:20.355580+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:10.637938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346898+0000","last_change":"2023-10-06T04:49:19.333331+0000","last_active":"2023-10-06T04:49:20.346898+0000","last_peered":"2023-10-06T04:49:20.346898+0000","last_clean":"2023-10-06T04:49:20.346898+0000","last_became_active":"2023-10-06T04:49:19.332774+0000","last_became_peered":"2023-10-06T04:49:19.332774+0000","last_unstale":"2023-10-06T04:49:20.346898+0000","last_undegraded":"2023-10-06T04:49:20.346898+0000","last_fullsized":"2023-10-06T04:49:20.346898+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:19.019093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346151+0000","last_change":"2023-10-06T04:49:20.345404+0000","last_active":"2023-10-06T04:49:20.346151+0000","last_peered":"2023-10-06T04:49:20.346151+0000","last_clean":"2023-10-06T04:49:20.346151+0000","last_became_active":"2023-10-06T04:49:20.345129+0000","last_became_peered":"2023-10-06T04:49:20.345129+0000","last_unstale":"2023-10-06T04:49:20.346151+0000","last_undegraded":"2023-10-06T04:49:20.346151+0000","last_fullsized":"2023-10-06T04:49:20.346151+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:25.017277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.319374+0000","last_change":"2023-10-06T04:49:19.322897+0000","last_active":"2023-10-06T04:49:20.319374+0000","last_peered":"2023-10-06T04:49:20.319374+0000","last_clean":"2023-10-06T04:49:20.319374+0000","last_became_active":"2023-10-06T04:49:19.322635+0000","last_became_peered":"2023-10-06T04:49:19.322635+0000","last_unstale":"2023-10-06T04:49:20.319374+0000","last_undegraded":"2023-10-06T04:49:20.319374+0000","last_fullsized":"2023-10-06T04:49:20.319374+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:19.019093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350774+0000","last_change":"2023-10-06T04:49:20.347069+0000","last_active":"2023-10-06T04:49:20.350774+0000","last_peered":"2023-10-06T04:49:20.350774+0000","last_clean":"2023-10-06T04:49:20.350774+0000","last_became_active":"2023-10-06T04:49:20.346801+0000","last_became_peered":"2023-10-06T04:49:20.346801+0000","last_unstale":"2023-10-06T04:49:20.350774+0000","last_undegraded":"2023-10-06T04:49:20.350774+0000","last_fullsized":"2023-10-06T04:49:20.350774+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:34.988176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321187+0000","last_change":"2023-10-06T04:49:19.329229+0000","last_active":"2023-10-06T04:49:20.321187+0000","last_peered":"2023-10-06T04:49:20.321187+0000","last_clean":"2023-10-06T04:49:20.321187+0000","last_became_active":"2023-10-06T04:49:19.328964+0000","last_became_peered":"2023-10-06T04:49:19.328964+0000","last_unstale":"2023-10-06T04:49:20.321187+0000","last_undegraded":"2023-10-06T04:49:20.321187+0000","last_fullsized":"2023-10-06T04:49:20.321187+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:53.870884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341139+0000","last_change":"2023-10-06T04:49:20.334997+0000","last_active":"2023-10-06T04:49:20.341139+0000","last_peered":"2023-10-06T04:49:20.341139+0000","last_clean":"2023-10-06T04:49:20.341139+0000","last_became_active":"2023-10-06T04:49:20.334723+0000","last_became_peered":"2023-10-06T04:49:20.334723+0000","last_unstale":"2023-10-06T04:49:20.341139+0000","last_undegraded":"2023-10-06T04:49:20.341139+0000","last_fullsized":"2023-10-06T04:49:20.341139+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:41.560870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321711+0000","last_change":"2023-10-06T04:49:19.324862+0000","last_active":"2023-10-06T04:49:20.321711+0000","last_peered":"2023-10-06T04:49:20.321711+0000","last_clean":"2023-10-06T04:49:20.321711+0000","last_became_active":"2023-10-06T04:49:19.324595+0000","last_became_peered":"2023-10-06T04:49:19.324595+0000","last_unstale":"2023-10-06T04:49:20.321711+0000","last_undegraded":"2023-10-06T04:49:20.321711+0000","last_fullsized":"2023-10-06T04:49:20.321711+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:53.870884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339461+0000","last_change":"2023-10-06T04:49:20.337671+0000","last_active":"2023-10-06T04:49:20.339461+0000","last_peered":"2023-10-06T04:49:20.339461+0000","last_clean":"2023-10-06T04:49:20.339461+0000","last_became_active":"2023-10-06T04:49:20.337383+0000","last_became_peered":"2023-10-06T04:49:20.337383+0000","last_unstale":"2023-10-06T04:49:20.339461+0000","last_undegraded":"2023-10-06T04:49:20.339461+0000","last_fullsized":"2023-10-06T04:49:20.339461+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:41.316112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.342104+0000","last_change":"2023-10-06T04:49:19.328915+0000","last_active":"2023-10-06T04:49:20.342104+0000","last_peered":"2023-10-06T04:49:20.342104+0000","last_clean":"2023-10-06T04:49:20.342104+0000","last_became_active":"2023-10-06T04:49:19.328630+0000","last_became_peered":"2023-10-06T04:49:19.328630+0000","last_unstale":"2023-10-06T04:49:20.342104+0000","last_undegraded":"2023-10-06T04:49:20.342104+0000","last_fullsized":"2023-10-06T04:49:20.342104+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:27.949348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.343843+0000","last_change":"2023-10-06T04:49:20.341772+0000","last_active":"2023-10-06T04:49:20.343843+0000","last_peered":"2023-10-06T04:49:20.343843+0000","last_clean":"2023-10-06T04:49:20.343843+0000","last_became_active":"2023-10-06T04:49:20.341489+0000","last_became_peered":"2023-10-06T04:49:20.341489+0000","last_unstale":"2023-10-06T04:49:20.343843+0000","last_undegraded":"2023-10-06T04:49:20.343843+0000","last_fullsized":"2023-10-06T04:49:20.343843+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:10.460143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.330303+0000","last_change":"2023-10-06T04:49:19.331671+0000","last_active":"2023-10-06T04:49:20.330303+0000","last_peered":"2023-10-06T04:49:20.330303+0000","last_clean":"2023-10-06T04:49:20.330303+0000","last_became_active":"2023-10-06T04:49:19.330061+0000","last_became_peered":"2023-10-06T04:49:19.330061+0000","last_unstale":"2023-10-06T04:49:20.330303+0000","last_undegraded":"2023-10-06T04:49:20.330303+0000","last_fullsized":"2023-10-06T04:49:20.330303+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:09.188792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356042+0000","last_change":"2023-10-06T04:49:20.352177+0000","last_active":"2023-10-06T04:49:20.356042+0000","last_peered":"2023-10-06T04:49:20.356042+0000","last_clean":"2023-10-06T04:49:20.356042+0000","last_became_active":"2023-10-06T04:49:20.351885+0000","last_became_peered":"2023-10-06T04:49:20.351885+0000","last_unstale":"2023-10-06T04:49:20.356042+0000","last_undegraded":"2023-10-06T04:49:20.356042+0000","last_fullsized":"2023-10-06T04:49:20.356042+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320737+0000","last_change":"2023-10-06T04:49:19.329132+0000","last_active":"2023-10-06T04:49:20.320737+0000","last_peered":"2023-10-06T04:49:20.320737+0000","last_clean":"2023-10-06T04:49:20.320737+0000","last_became_active":"2023-10-06T04:49:19.328841+0000","last_became_peered":"2023-10-06T04:49:19.328841+0000","last_unstale":"2023-10-06T04:49:20.320737+0000","last_undegraded":"2023-10-06T04:49:20.320737+0000","last_fullsized":"2023-10-06T04:49:20.320737+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:54.415706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344647+0000","last_change":"2023-10-06T04:49:20.343226+0000","last_active":"2023-10-06T04:49:20.344647+0000","last_peered":"2023-10-06T04:49:20.344647+0000","last_clean":"2023-10-06T04:49:20.344647+0000","last_became_active":"2023-10-06T04:49:20.342932+0000","last_became_peered":"2023-10-06T04:49:20.342932+0000","last_unstale":"2023-10-06T04:49:20.344647+0000","last_undegraded":"2023-10-06T04:49:20.344647+0000","last_fullsized":"2023-10-06T04:49:20.344647+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:28.172716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.316079+0000","last_change":"2023-10-06T04:49:19.328230+0000","last_active":"2023-10-06T04:49:20.316079+0000","last_peered":"2023-10-06T04:49:20.316079+0000","last_clean":"2023-10-06T04:49:20.316079+0000","last_became_active":"2023-10-06T04:49:19.327917+0000","last_became_peered":"2023-10-06T04:49:19.327917+0000","last_unstale":"2023-10-06T04:49:20.316079+0000","last_undegraded":"2023-10-06T04:49:20.316079+0000","last_fullsized":"2023-10-06T04:49:20.316079+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.900580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347059+0000","last_change":"2023-10-06T04:49:20.336712+0000","last_active":"2023-10-06T04:49:20.347059+0000","last_peered":"2023-10-06T04:49:20.347059+0000","last_clean":"2023-10-06T04:49:20.347059+0000","last_became_active":"2023-10-06T04:49:20.335319+0000","last_became_peered":"2023-10-06T04:49:20.335319+0000","last_unstale":"2023-10-06T04:49:20.347059+0000","last_undegraded":"2023-10-06T04:49:20.347059+0000","last_fullsized":"2023-10-06T04:49:20.347059+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:00.206328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.325259+0000","last_change":"2023-10-06T04:49:19.331624+0000","last_active":"2023-10-06T04:49:20.325259+0000","last_peered":"2023-10-06T04:49:20.325259+0000","last_clean":"2023-10-06T04:49:20.325259+0000","last_became_active":"2023-10-06T04:49:19.331287+0000","last_became_peered":"2023-10-06T04:49:19.331287+0000","last_unstale":"2023-10-06T04:49:20.325259+0000","last_undegraded":"2023-10-06T04:49:20.325259+0000","last_fullsized":"2023-10-06T04:49:20.325259+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.288607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341412+0000","last_change":"2023-10-06T04:49:20.334336+0000","last_active":"2023-10-06T04:49:20.341412+0000","last_peered":"2023-10-06T04:49:20.341412+0000","last_clean":"2023-10-06T04:49:20.341412+0000","last_became_active":"2023-10-06T04:49:20.334097+0000","last_became_peered":"2023-10-06T04:49:20.334097+0000","last_unstale":"2023-10-06T04:49:20.341412+0000","last_undegraded":"2023-10-06T04:49:20.341412+0000","last_fullsized":"2023-10-06T04:49:20.341412+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:35.183960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320080+0000","last_change":"2023-10-06T04:49:19.332817+0000","last_active":"2023-10-06T04:49:20.320080+0000","last_peered":"2023-10-06T04:49:20.320080+0000","last_clean":"2023-10-06T04:49:20.320080+0000","last_became_active":"2023-10-06T04:49:19.331466+0000","last_became_peered":"2023-10-06T04:49:19.331466+0000","last_unstale":"2023-10-06T04:49:20.320080+0000","last_undegraded":"2023-10-06T04:49:20.320080+0000","last_fullsized":"2023-10-06T04:49:20.320080+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:01.585470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332830+0000","last_change":"2023-10-06T04:49:20.327613+0000","last_active":"2023-10-06T04:49:20.332830+0000","last_peered":"2023-10-06T04:49:20.332830+0000","last_clean":"2023-10-06T04:49:20.332830+0000","last_became_active":"2023-10-06T04:49:20.327331+0000","last_became_peered":"2023-10-06T04:49:20.327331+0000","last_unstale":"2023-10-06T04:49:20.332830+0000","last_undegraded":"2023-10-06T04:49:20.332830+0000","last_fullsized":"2023-10-06T04:49:20.332830+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:21.888629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339559+0000","last_change":"2023-10-06T04:49:19.327134+0000","last_active":"2023-10-06T04:49:20.339559+0000","last_peered":"2023-10-06T04:49:20.339559+0000","last_clean":"2023-10-06T04:49:20.339559+0000","last_became_active":"2023-10-06T04:49:19.326884+0000","last_became_peered":"2023-10-06T04:49:19.326884+0000","last_unstale":"2023-10-06T04:49:20.339559+0000","last_undegraded":"2023-10-06T04:49:20.339559+0000","last_fullsized":"2023-10-06T04:49:20.339559+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:40.308269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347026+0000","last_change":"2023-10-06T04:49:20.343182+0000","last_active":"2023-10-06T04:49:20.347026+0000","last_peered":"2023-10-06T04:49:20.347026+0000","last_clean":"2023-10-06T04:49:20.347026+0000","last_became_active":"2023-10-06T04:49:20.342662+0000","last_became_peered":"2023-10-06T04:49:20.342662+0000","last_unstale":"2023-10-06T04:49:20.347026+0000","last_undegraded":"2023-10-06T04:49:20.347026+0000","last_fullsized":"2023-10-06T04:49:20.347026+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:54.488863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331074+0000","last_change":"2023-10-06T04:49:19.336399+0000","last_active":"2023-10-06T04:49:20.331074+0000","last_peered":"2023-10-06T04:49:20.331074+0000","last_clean":"2023-10-06T04:49:20.331074+0000","last_became_active":"2023-10-06T04:49:19.336144+0000","last_became_peered":"2023-10-06T04:49:19.336144+0000","last_unstale":"2023-10-06T04:49:20.331074+0000","last_undegraded":"2023-10-06T04:49:20.331074+0000","last_fullsized":"2023-10-06T04:49:20.331074+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:48.611176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333465+0000","last_change":"2023-10-06T04:49:20.325062+0000","last_active":"2023-10-06T04:49:20.333465+0000","last_peered":"2023-10-06T04:49:20.333465+0000","last_clean":"2023-10-06T04:49:20.333465+0000","last_became_active":"2023-10-06T04:49:20.324091+0000","last_became_peered":"2023-10-06T04:49:20.324091+0000","last_unstale":"2023-10-06T04:49:20.333465+0000","last_undegraded":"2023-10-06T04:49:20.333465+0000","last_fullsized":"2023-10-06T04:49:20.333465+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:28.109738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321039+0000","last_change":"2023-10-06T04:49:19.327314+0000","last_active":"2023-10-06T04:49:20.321039+0000","last_peered":"2023-10-06T04:49:20.321039+0000","last_clean":"2023-10-06T04:49:20.321039+0000","last_became_active":"2023-10-06T04:49:19.326851+0000","last_became_peered":"2023-10-06T04:49:19.326851+0000","last_unstale":"2023-10-06T04:49:20.321039+0000","last_undegraded":"2023-10-06T04:49:20.321039+0000","last_fullsized":"2023-10-06T04:49:20.321039+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:26.547655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":156,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321556+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T04:49:20.321556+0000","last_peered":"2023-10-06T04:49:20.321556+0000","last_clean":"2023-10-06T04:49:20.321556+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T04:49:20.321556+0000","last_undegraded":"2023-10-06T04:49:20.321556+0000","last_fullsized":"2023-10-06T04:49:20.321556+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332841+0000","last_change":"2023-10-06T04:49:20.327658+0000","last_active":"2023-10-06T04:49:20.332841+0000","last_peered":"2023-10-06T04:49:20.332841+0000","last_clean":"2023-10-06T04:49:20.332841+0000","last_became_active":"2023-10-06T04:49:20.327377+0000","last_became_peered":"2023-10-06T04:49:20.327377+0000","last_unstale":"2023-10-06T04:49:20.332841+0000","last_undegraded":"2023-10-06T04:49:20.332841+0000","last_fullsized":"2023-10-06T04:49:20.332841+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"172'8","reported_seq":18,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.351338+0000","last_change":"2023-10-06T04:49:19.329982+0000","last_active":"2023-10-06T04:49:20.351338+0000","last_peered":"2023-10-06T04:49:20.351338+0000","last_clean":"2023-10-06T04:49:20.351338+0000","last_became_active":"2023-10-06T04:49:19.329677+0000","last_became_peered":"2023-10-06T04:49:19.329677+0000","last_unstale":"2023-10-06T04:49:20.351338+0000","last_undegraded":"2023-10-06T04:49:20.351338+0000","last_fullsized":"2023-10-06T04:49:20.351338+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":143,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.316165+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T04:49:20.316165+0000","last_peered":"2023-10-06T04:49:20.316165+0000","last_clean":"2023-10-06T04:49:20.316165+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T04:49:20.316165+0000","last_undegraded":"2023-10-06T04:49:20.316165+0000","last_fullsized":"2023-10-06T04:49:20.316165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.343279+0000","last_change":"2023-10-06T04:49:20.330255+0000","last_active":"2023-10-06T04:49:20.343279+0000","last_peered":"2023-10-06T04:49:20.343279+0000","last_clean":"2023-10-06T04:49:20.343279+0000","last_became_active":"2023-10-06T04:49:20.329995+0000","last_became_peered":"2023-10-06T04:49:20.329995+0000","last_unstale":"2023-10-06T04:49:20.343279+0000","last_undegraded":"2023-10-06T04:49:20.343279+0000","last_fullsized":"2023-10-06T04:49:20.343279+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:16.754707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.330157+0000","last_change":"2023-10-06T04:49:19.330514+0000","last_active":"2023-10-06T04:49:20.330157+0000","last_peered":"2023-10-06T04:49:20.330157+0000","last_clean":"2023-10-06T04:49:20.330157+0000","last_became_active":"2023-10-06T04:49:19.330226+0000","last_became_peered":"2023-10-06T04:49:19.330226+0000","last_unstale":"2023-10-06T04:49:20.330157+0000","last_undegraded":"2023-10-06T04:49:20.330157+0000","last_fullsized":"2023-10-06T04:49:20.330157+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:55.142830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":144,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.319996+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T04:49:20.319996+0000","last_peered":"2023-10-06T04:49:20.319996+0000","last_clean":"2023-10-06T04:49:20.319996+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T04:49:20.319996+0000","last_undegraded":"2023-10-06T04:49:20.319996+0000","last_fullsized":"2023-10-06T04:49:20.319996+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"163'293","reported_seq":919,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331027+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T04:49:20.331027+0000","last_peered":"2023-10-06T04:49:20.331027+0000","last_clean":"2023-10-06T04:49:20.331027+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T04:49:20.331027+0000","last_undegraded":"2023-10-06T04:49:20.331027+0000","last_fullsized":"2023-10-06T04:49:20.331027+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.336779+0000","last_change":"2023-10-06T04:49:20.335358+0000","last_active":"2023-10-06T04:49:20.336779+0000","last_peered":"2023-10-06T04:49:20.336779+0000","last_clean":"2023-10-06T04:49:20.336779+0000","last_became_active":"2023-10-06T04:49:20.335073+0000","last_became_peered":"2023-10-06T04:49:20.335073+0000","last_unstale":"2023-10-06T04:49:20.336779+0000","last_undegraded":"2023-10-06T04:49:20.336779+0000","last_fullsized":"2023-10-06T04:49:20.336779+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:05.280129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.330408+0000","last_change":"2023-10-06T04:49:19.331617+0000","last_active":"2023-10-06T04:49:20.330408+0000","last_peered":"2023-10-06T04:49:20.330408+0000","last_clean":"2023-10-06T04:49:20.330408+0000","last_became_active":"2023-10-06T04:49:19.329690+0000","last_became_peered":"2023-10-06T04:49:19.329690+0000","last_unstale":"2023-10-06T04:49:20.330408+0000","last_undegraded":"2023-10-06T04:49:20.330408+0000","last_fullsized":"2023-10-06T04:49:20.330408+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:04.983797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":146,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.325305+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T04:49:20.325305+0000","last_peered":"2023-10-06T04:49:20.325305+0000","last_clean":"2023-10-06T04:49:20.325305+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T04:49:20.325305+0000","last_undegraded":"2023-10-06T04:49:20.325305+0000","last_fullsized":"2023-10-06T04:49:20.325305+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339557+0000","last_change":"2023-10-06T04:49:20.331740+0000","last_active":"2023-10-06T04:49:20.339557+0000","last_peered":"2023-10-06T04:49:20.339557+0000","last_clean":"2023-10-06T04:49:20.339557+0000","last_became_active":"2023-10-06T04:49:20.331463+0000","last_became_peered":"2023-10-06T04:49:20.331463+0000","last_unstale":"2023-10-06T04:49:20.339557+0000","last_undegraded":"2023-10-06T04:49:20.339557+0000","last_fullsized":"2023-10-06T04:49:20.339557+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:01.313087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"172'6","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356034+0000","last_change":"2023-10-06T04:49:19.331731+0000","last_active":"2023-10-06T04:49:20.356034+0000","last_peered":"2023-10-06T04:49:20.356034+0000","last_clean":"2023-10-06T04:49:20.356034+0000","last_became_active":"2023-10-06T04:49:19.331458+0000","last_became_peered":"2023-10-06T04:49:19.331458+0000","last_unstale":"2023-10-06T04:49:20.356034+0000","last_undegraded":"2023-10-06T04:49:20.356034+0000","last_fullsized":"2023-10-06T04:49:20.356034+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:55.142830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":146,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331105+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T04:49:20.331105+0000","last_peered":"2023-10-06T04:49:20.331105+0000","last_clean":"2023-10-06T04:49:20.331105+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T04:49:20.331105+0000","last_undegraded":"2023-10-06T04:49:20.331105+0000","last_fullsized":"2023-10-06T04:49:20.331105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347917+0000","last_change":"2023-10-06T04:49:20.345754+0000","last_active":"2023-10-06T04:49:20.347917+0000","last_peered":"2023-10-06T04:49:20.347917+0000","last_clean":"2023-10-06T04:49:20.347917+0000","last_became_active":"2023-10-06T04:49:20.345446+0000","last_became_peered":"2023-10-06T04:49:20.345446+0000","last_unstale":"2023-10-06T04:49:20.347917+0000","last_undegraded":"2023-10-06T04:49:20.347917+0000","last_fullsized":"2023-10-06T04:49:20.347917+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:00.217906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"172'3","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.342594+0000","last_change":"2023-10-06T04:49:19.327118+0000","last_active":"2023-10-06T04:49:20.342594+0000","last_peered":"2023-10-06T04:49:20.342594+0000","last_clean":"2023-10-06T04:49:20.342594+0000","last_became_active":"2023-10-06T04:49:19.326839+0000","last_became_peered":"2023-10-06T04:49:19.326839+0000","last_unstale":"2023-10-06T04:49:20.342594+0000","last_undegraded":"2023-10-06T04:49:20.342594+0000","last_fullsized":"2023-10-06T04:49:20.342594+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:17.629867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":139,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.335923+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T04:49:20.335923+0000","last_peered":"2023-10-06T04:49:20.335923+0000","last_clean":"2023-10-06T04:49:20.335923+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T04:49:20.335923+0000","last_undegraded":"2023-10-06T04:49:20.335923+0000","last_fullsized":"2023-10-06T04:49:20.335923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341195+0000","last_change":"2023-10-06T04:49:20.338084+0000","last_active":"2023-10-06T04:49:20.341195+0000","last_peered":"2023-10-06T04:49:20.341195+0000","last_clean":"2023-10-06T04:49:20.341195+0000","last_became_active":"2023-10-06T04:49:20.337738+0000","last_became_peered":"2023-10-06T04:49:20.337738+0000","last_unstale":"2023-10-06T04:49:20.341195+0000","last_undegraded":"2023-10-06T04:49:20.341195+0000","last_fullsized":"2023-10-06T04:49:20.341195+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:28.172716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321149+0000","last_change":"2023-10-06T04:49:19.332809+0000","last_active":"2023-10-06T04:49:20.321149+0000","last_peered":"2023-10-06T04:49:20.321149+0000","last_clean":"2023-10-06T04:49:20.321149+0000","last_became_active":"2023-10-06T04:49:19.331892+0000","last_became_peered":"2023-10-06T04:49:19.331892+0000","last_unstale":"2023-10-06T04:49:20.321149+0000","last_undegraded":"2023-10-06T04:49:20.321149+0000","last_fullsized":"2023-10-06T04:49:20.321149+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:52.469578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320842+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T04:49:20.320842+0000","last_peered":"2023-10-06T04:49:20.320842+0000","last_clean":"2023-10-06T04:49:20.320842+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T04:49:20.320842+0000","last_undegraded":"2023-10-06T04:49:20.320842+0000","last_fullsized":"2023-10-06T04:49:20.320842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347079+0000","last_change":"2023-10-06T04:49:20.336896+0000","last_active":"2023-10-06T04:49:20.347079+0000","last_peered":"2023-10-06T04:49:20.347079+0000","last_clean":"2023-10-06T04:49:20.347079+0000","last_became_active":"2023-10-06T04:49:20.336451+0000","last_became_peered":"2023-10-06T04:49:20.336451+0000","last_unstale":"2023-10-06T04:49:20.347079+0000","last_undegraded":"2023-10-06T04:49:20.347079+0000","last_fullsized":"2023-10-06T04:49:20.347079+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:09.120814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344451+0000","last_change":"2023-10-06T04:49:19.321982+0000","last_active":"2023-10-06T04:49:20.344451+0000","last_peered":"2023-10-06T04:49:20.344451+0000","last_clean":"2023-10-06T04:49:20.344451+0000","last_became_active":"2023-10-06T04:49:19.321679+0000","last_became_peered":"2023-10-06T04:49:19.321679+0000","last_unstale":"2023-10-06T04:49:20.344451+0000","last_undegraded":"2023-10-06T04:49:20.344451+0000","last_fullsized":"2023-10-06T04:49:20.344451+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320746+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T04:49:20.320746+0000","last_peered":"2023-10-06T04:49:20.320746+0000","last_clean":"2023-10-06T04:49:20.320746+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T04:49:20.320746+0000","last_undegraded":"2023-10-06T04:49:20.320746+0000","last_fullsized":"2023-10-06T04:49:20.320746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575752,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299976,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071864,"statfs":{"total":96636764160,"available":96329588736,"internally_reserved":0,"allocated":2752512,"data_stored":1720883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69799999999999995,"15min":0.73899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.89500000000000002,"5min":0.97199999999999998,"15min":1.4930000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.48599999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90900000000000003,"5min":0.995,"15min":1.417},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.755,"15min":0.79500000000000004},"min":{"1min":0.53500000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93100000000000005,"5min":1.01,"15min":1.5309999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77600000000000002,"15min":0.79300000000000004},"min":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.58299999999999996},"max":{"1min":0.93600000000000005,"5min":1.0289999999999999,"15min":1.097},"last":0.81000000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70899999999999996,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.46400000000000002},"max":{"1min":0.92300000000000004,"5min":0.97999999999999998,"15min":1.5429999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.751,"15min":0.77100000000000002},"min":{"1min":0.60199999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0189999999999999},"last":0.82999999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.75700000000000001,"15min":0.77500000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.91000000000000003,"5min":1.03,"15min":1.5860000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73299999999999998,"15min":0.76400000000000001},"min":{"1min":0.64200000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.96099999999999997}]},{"osd":4,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63400000000000001,"15min":0.61899999999999999},"min":{"1min":0.50800000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.0640000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.439,"15min":0.42799999999999999},"max":{"1min":1.1339999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.90400000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.53800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.55200000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.72999999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.64200000000000002,"5min":0.501,"15min":0.49199999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":1.1020000000000001,"15min":1.2090000000000001},"last":0.93899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575753,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299400,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072440,"statfs":{"total":96636764160,"available":96330178560,"internally_reserved":0,"allocated":2162688,"data_stored":1131541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.46000000000000002,"5min":0.40300000000000002,"15min":0.375},"max":{"1min":0.96599999999999997,"5min":1.038,"15min":1.038},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76200000000000001,"15min":0.752},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94699999999999995,"5min":0.999,"15min":1.228},"last":0.80000000000000004}]},{"osd":1,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.5,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93799999999999994,"5min":1.046,"15min":1.3660000000000001},"last":0.63},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.748,"15min":0.751},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.438},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.3560000000000001},"last":0.76500000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.47999999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.272},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76900000000000002,"15min":0.77000000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98799999999999999,"5min":1.069,"15min":1.089},"last":0.66300000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94399999999999995,"5min":1.042,"15min":1.1060000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.52000000000000002},"max":{"1min":1.0009999999999999,"5min":1.0409999999999999,"15min":1.3120000000000001},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83499999999999996,"5min":0.88500000000000001,"15min":0.92600000000000005},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.627},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":1.155},"last":0.84499999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.432},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93799999999999994,"5min":1.056,"15min":1.056},"last":0.82799999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95399999999999996,"5min":1.0580000000000001,"15min":1.3500000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.45800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98599999999999999,"5min":1.024,"15min":1.0389999999999999},"last":0.78300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575752,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312268,"kb_used_data":2692,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059572,"statfs":{"total":96636764160,"available":96317001728,"internally_reserved":0,"allocated":2756608,"data_stored":1721909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.79600000000000004,"5min":0.89000000000000001,"15min":0.93100000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42899999999999999},"max":{"1min":0.90000000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.41399999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63800000000000001,"5min":0.67700000000000005,"15min":0.73299999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.38200000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.97799999999999998},"last":0.46899999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.44,"5min":0.44,"15min":0.437},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.94699999999999995},"last":0.497},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.373},"max":{"1min":0.748,"5min":0.85799999999999998,"15min":0.88900000000000001},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79200000000000004,"15min":0.79500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.46899999999999997},"max":{"1min":1.159,"5min":1.1970000000000001,"15min":1.407},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82599999999999996,"15min":0.83099999999999996},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.57499999999999996},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.84399999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83699999999999997,"15min":0.82699999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.55700000000000005},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81699999999999995,"15min":0.83899999999999997},"min":{"1min":0.66300000000000003,"5min":0.59799999999999998,"15min":0.53200000000000003},"max":{"1min":1.097,"5min":1.1060000000000001,"15min":1.724},"last":0.88300000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.64700000000000002,"5min":0.626,"15min":0.56200000000000006},"max":{"1min":0.97899999999999998,"5min":1.2150000000000001,"15min":1.427},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.55800000000000005},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.93600000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.877,"15min":0.878},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1140000000000001,"5min":1.125,"15min":1.702},"last":0.874},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85799999999999998,"15min":0.872},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0269999999999999,"5min":1.1659999999999999,"15min":1.5920000000000001},"last":0.72199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575754,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299416,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072424,"statfs":{"total":96636764160,"available":96330162176,"internally_reserved":0,"allocated":2179072,"data_stored":1132707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.378,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76600000000000001,"5min":0.81699999999999995,"15min":1.6459999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.34200000000000003,"15min":0.32900000000000001},"max":{"1min":0.68300000000000005,"5min":0.84099999999999997,"15min":1.4510000000000001},"last":0.437}]},{"osd":2,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55400000000000005,"15min":0.57099999999999995},"min":{"1min":0.434,"5min":0.308,"15min":0.308},"max":{"1min":0.74099999999999999,"5min":0.86899999999999999,"15min":1.7},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.373},"max":{"1min":0.81699999999999995,"5min":0.82099999999999995,"15min":1.6240000000000001},"last":0.46800000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":1.2529999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91300000000000003,"5min":0.99099999999999999,"15min":1.7370000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.86599999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.60399999999999998},"max":{"1min":1.0940000000000001,"5min":1.2769999999999999,"15min":2.3700000000000001},"last":0.873},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.85099999999999998,"15min":0.85999999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.52900000000000003},"max":{"1min":1.111,"5min":1.3069999999999999,"15min":2.1280000000000001},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84099999999999997,"15min":0.85599999999999998},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.57599999999999996},"max":{"1min":1.048,"5min":1.294,"15min":2.1739999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84099999999999997,"15min":0.84799999999999998},"min":{"1min":0.66000000000000003,"5min":0.63300000000000001,"15min":0.57799999999999996},"max":{"1min":1.087,"5min":1.214,"15min":2.2429999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60599999999999998,"5min":0.59299999999999997,"15min":0.53000000000000003},"max":{"1min":1.1160000000000001,"5min":1.204,"15min":1.605},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89600000000000002,"15min":0.88700000000000001},"min":{"1min":0.72899999999999998,"5min":0.65800000000000003,"15min":0.58199999999999996},"max":{"1min":1.181,"5min":1.226,"15min":2.3519999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.86699999999999999,"15min":0.878},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.091,"5min":1.319,"15min":2.1019999999999999},"last":0.746},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.86299999999999999,"15min":0.86799999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0580000000000001,"5min":1.256,"15min":1.9259999999999999},"last":0.79500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575753,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311688,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060152,"statfs":{"total":96636764160,"available":96317595648,"internally_reserved":0,"allocated":2162688,"data_stored":1130605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.311},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.376},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.34200000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.93999999999999995},"last":0.505}]},{"osd":1,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.32000000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.42399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.38300000000000001},"max":{"1min":0.79600000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.371},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.45400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.628,"5min":0.60199999999999998,"15min":0.51700000000000002},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.86399999999999999,"15min":0.88100000000000001},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58599999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.381},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.59199999999999997},"max":{"1min":1.1719999999999999,"5min":1.2210000000000001,"15min":1.4379999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88300000000000001,"15min":0.89200000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.58899999999999997},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.288},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.151,"5min":1.167,"15min":1.2370000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.61499999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86099999999999999,"15min":0.874},"min":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.57499999999999996},"max":{"1min":1.075,"5min":1.3320000000000001,"15min":1.482},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.88500000000000001,"15min":0.89600000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.135,"5min":1.2869999999999999,"15min":1.405},"last":0.755}]}]},{"osd":3,"up_from":13,"seq":55834575753,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311692,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060148,"statfs":{"total":96636764160,"available":96317591552,"internally_reserved":0,"allocated":2166784,"data_stored":1132059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47699999999999998,"15min":0.49399999999999999},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67900000000000005,"5min":0.73899999999999999,"15min":0.86399999999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.83599999999999997},"last":0.501},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.44900000000000001,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.73999999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.40899999999999997},"max":{"1min":0.69699999999999995,"5min":0.79200000000000004,"15min":1.036},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.54000000000000004,"5min":0.47499999999999998,"15min":0.43099999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83799999999999997},"min":{"1min":0.64300000000000002,"5min":0.56000000000000005,"15min":0.47599999999999998},"max":{"1min":1.101,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.495},"max":{"1min":1.0760000000000001,"5min":1.387,"15min":1.399},"last":1.034}]},{"osd":5,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.82599999999999996,"15min":0.82899999999999996},"min":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0049999999999999,"5min":1.2509999999999999,"15min":1.331},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.85799999999999998,"15min":0.85099999999999998},"min":{"1min":0.63600000000000001,"5min":0.58099999999999996,"15min":0.56599999999999995},"max":{"1min":1.0720000000000001,"5min":1.2130000000000001,"15min":1.2330000000000001},"last":1.0549999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.86499999999999999,"15min":0.85999999999999999},"min":{"1min":0.65100000000000002,"5min":0.53500000000000003,"15min":0.52400000000000002},"max":{"1min":1.175,"5min":1.2090000000000001,"15min":1.2490000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.89900000000000002,"15min":0.89400000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1579999999999999,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88600000000000001,"15min":0.88400000000000001},"min":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.58999999999999997},"max":{"1min":1.127,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85399999999999998,"15min":0.85799999999999998},"min":{"1min":0.60499999999999998,"5min":0.58699999999999997,"15min":0.54900000000000004},"max":{"1min":1.069,"5min":1.276,"15min":1.3029999999999999},"last":1.097}]}]},{"osd":4,"up_from":13,"seq":55834575753,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311696,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060144,"statfs":{"total":96636764160,"available":96317587456,"internally_reserved":0,"allocated":2170880,"data_stored":1133007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.71699999999999997,"15min":0.74299999999999999},"min":{"1min":0.47899999999999998,"5min":0.36699999999999999,"15min":0.35599999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.72399999999999998,"15min":0.73499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.79300000000000004}]},{"osd":1,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.76000000000000001,"15min":0.77800000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.69999999999999996}]},{"osd":2,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.745,"15min":0.75900000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.755},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.76600000000000001,"15min":0.77100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91200000000000003,"5min":1.01,"15min":1.069},"last":0.876}]},{"osd":3,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.996,"5min":1.002,"15min":1.002},"last":0.65900000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81699999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.65000000000000002,"15min":0.63500000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.61099999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69099999999999995,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.46000000000000002,"15min":0.42099999999999999},"max":{"1min":0.996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71499999999999997,"15min":0.70199999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0629999999999999,"5min":1.105,"15min":1.105},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71399999999999997,"15min":0.68799999999999994},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97999999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.75600000000000001,"15min":0.745},"min":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89700000000000002,"5min":1.091,"15min":1.0940000000000001},"last":0.85699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575753,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307584,"kb_used_data":2104,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064256,"statfs":{"total":96636764160,"available":96321798144,"internally_reserved":0,"allocated":2154496,"data_stored":1130493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82999999999999996,"5min":1.03,"15min":2.2829999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73599999999999999,"15min":0.751},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91600000000000004,"5min":1.095,"15min":1.3200000000000001},"last":0.67600000000000005}]},{"osd":1,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.751},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.29},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.51400000000000001},"max":{"1min":0.871,"5min":0.98099999999999998,"15min":1.9139999999999999},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.57999999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98199999999999998,"5min":1.0589999999999999,"15min":1.532},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95799999999999996,"5min":1.0169999999999999,"15min":1.121},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.58899999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.40500000000000003,"15min":0.39300000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":2.6989999999999998},"last":0.75},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38},"max":{"1min":0.99399999999999999,"5min":1.028,"15min":1.694},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.1719999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96399999999999997,"5min":1.077,"15min":1.5880000000000001},"last":0.79100000000000004}]},{"osd":7,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.52500000000000002,"5min":0.47499999999999998,"15min":0.40699999999999997},"max":{"1min":1.03,"5min":1.0409999999999999,"15min":1.7390000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.49099999999999999,"15min":0.48599999999999999},"max":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.3380000000000001},"last":0.75800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:49:29.662 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:49:29.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.859+0000 7fb85759e700 1 Processor -- start 2023-10-06T04:49:29.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.860+0000 7fb85759e700 1 -- start start 2023-10-06T04:49:29.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.861+0000 7fb85759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8580ae710 0x7fb8580c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:29.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.861+0000 7fb85759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8580ca860 con 0x7fb858156310 2023-10-06T04:49:29.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.861+0000 7fb85759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8580cab40 con 0x7fb8581566e0 2023-10-06T04:49:29.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.861+0000 7fb85759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580cae20 con 0x7fb8580ae710 2023-10-06T04:49:29.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.861+0000 7fb85659c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb858156310 0x7fb8580c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46740/0 (socket says 172.21.15.55:46740) 2023-10-06T04:49:29.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.861+0000 7fb85659c700 1 -- 172.21.15.55:0/2595658447 learned_addr learned my addr 172.21.15.55:0/2595658447 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:29.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.861+0000 7fb856d9d700 1 --2- 172.21.15.55:0/2595658447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8580ae710 0x7fb8580c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:29.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884868700 0 0) 0x7fb8580ca860 con 0x7fb858156310 2023-10-06T04:49:29.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb840003570 con 0x7fb858156310 2023-10-06T04:49:29.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4257023678 0 0) 0x7fb8580cab40 con 0x7fb8581566e0 2023-10-06T04:49:29.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8580ca860 con 0x7fb8581566e0 2023-10-06T04:49:29.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1030998170 0 0) 0x7fb840003570 con 0x7fb858156310 2023-10-06T04:49:29.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8580ae710 msgr2=0x7fb8580c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:29.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 --2- 172.21.15.55:0/2595658447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8580ae710 0x7fb8580c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:29.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 >> v1:172.21.15.163:6789/0 conn(0x7fb8581566e0 legacy=0x7fb8580ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:29.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8580cb100 con 0x7fb858156310 2023-10-06T04:49:29.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fb848001470 con 0x7fb858156310 2023-10-06T04:49:29.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 >> v1:172.21.15.55:6789/0 conn(0x7fb858156310 legacy=0x7fb8580c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:29.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 --2- 172.21.15.55:0/2595658447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb840003c20 0x7fb840004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:29.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 --2- 172.21.15.55:0/2595658447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8580ae710 0x7fb840004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:29.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 --2- 172.21.15.55:0/2595658447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb840004d50 0x7fb8400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:29.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580cae20 con 0x7fb840003c20 2023-10-06T04:49:29.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84000bbf0 con 0x7fb840004d50 2023-10-06T04:49:29.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84000bd60 con 0x7fb8580ae710 2023-10-06T04:49:29.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb85659c700 1 --2- 172.21.15.55:0/2595658447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb840003c20 0x7fb840004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:29.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb855d9b700 1 --2- 172.21.15.55:0/2595658447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8580ae710 0x7fb840004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:29.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.862+0000 7fb856d9d700 1 --2- 172.21.15.55:0/2595658447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb840004d50 0x7fb8400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:29.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.863+0000 7fb85659c700 1 -- 172.21.15.55:0/2595658447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8580ae710 msgr2=0x7fb840004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:29.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.863+0000 7fb85659c700 1 --2- 172.21.15.55:0/2595658447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb8580ae710 0x7fb840004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:29.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.863+0000 7fb85659c700 1 -- 172.21.15.55:0/2595658447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb840004d50 msgr2=0x7fb8400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:29.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.863+0000 7fb85659c700 1 --2- 172.21.15.55:0/2595658447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb840004d50 0x7fb8400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:29.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.863+0000 7fb85659c700 1 -- 172.21.15.55:0/2595658447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb840003570 con 0x7fb840003c20 2023-10-06T04:49:29.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.863+0000 7fb85659c700 1 --2- 172.21.15.55:0/2595658447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb840003c20 0x7fb840004010 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb848007fd0 tx=0x7fb848001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ffa923037489db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:29.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.863+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84800cae0 con 0x7fb840003c20 2023-10-06T04:49:29.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.863+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84800cc40 con 0x7fb840003c20 2023-10-06T04:49:29.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.863+0000 7fb85559a700 1 -- 172.21.15.55:0/2595658447 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb848005920 con 0x7fb840003c20 2023-10-06T04:49:29.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.864+0000 7fb85759e700 1 -- 172.21.15.55:0/2595658447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb840003c20 msgr2=0x7fb840004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:29.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.864+0000 7fb85759e700 1 --2- 172.21.15.55:0/2595658447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb840003c20 0x7fb840004010 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb848007fd0 tx=0x7fb848001d40 comp rx=0 tx=0).stop 2023-10-06T04:49:29.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.864+0000 7fb856d9d700 1 -- 172.21.15.55:0/2595658447 reap_dead start 2023-10-06T04:49:29.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.864+0000 7fb85759e700 1 -- 172.21.15.55:0/2595658447 shutdown_connections 2023-10-06T04:49:29.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.864+0000 7fb85759e700 1 -- 172.21.15.55:0/2595658447 >> 172.21.15.55:0/2595658447 conn(0x7fb8580cc1f0 msgr2=0x7fb8580cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:29.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.864+0000 7fb85759e700 1 -- 172.21.15.55:0/2595658447 shutdown_connections 2023-10-06T04:49:29.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.864+0000 7fb85759e700 1 -- 172.21.15.55:0/2595658447 wait complete. 2023-10-06T04:49:29.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.865+0000 7fb85759e700 1 Processor -- start 2023-10-06T04:49:29.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.865+0000 7fb85759e700 1 -- start start 2023-10-06T04:49:29.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.866+0000 7fb85759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb840003c20 0x7fb8580c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:29.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.866+0000 7fb85759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb85815a440 0x7fb858143c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:29.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.866+0000 7fb85759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb858157b50 0x7fb8580c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:29.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.866+0000 7fb85759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580c38f0 con 0x7fb840003c20 2023-10-06T04:49:29.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.866+0000 7fb85759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85815ac20 con 0x7fb858157b50 2023-10-06T04:49:29.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.866+0000 7fb85759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85815af00 con 0x7fb85815a440 2023-10-06T04:49:29.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.866+0000 7fb85659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb840003c20 0x7fb8580c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:29.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.867+0000 7fb85659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb840003c20 0x7fb8580c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34508/0 (socket says 172.21.15.55:34508) 2023-10-06T04:49:29.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.867+0000 7fb85659c700 1 -- 172.21.15.55:0/1808258619 learned_addr learned my addr 172.21.15.55:0/1808258619 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:29.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.867+0000 7fb855d9b700 1 --2- 172.21.15.55:0/1808258619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb85815a440 0x7fb858143c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:29.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.867+0000 7fb856d9d700 1 --2- 172.21.15.55:0/1808258619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb858157b50 0x7fb8580c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:29.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.867+0000 7fb85659c700 1 -- 172.21.15.55:0/1808258619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb85815a440 msgr2=0x7fb858143c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:29.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.867+0000 7fb85659c700 1 --2- 172.21.15.55:0/1808258619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb85815a440 0x7fb858143c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:29.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.867+0000 7fb85659c700 1 -- 172.21.15.55:0/1808258619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb858157b50 msgr2=0x7fb8580c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:29.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.867+0000 7fb85659c700 1 --2- 172.21.15.55:0/1808258619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb858157b50 0x7fb8580c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:29.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.867+0000 7fb85659c700 1 -- 172.21.15.55:0/1808258619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8580c32a0 con 0x7fb840003c20 2023-10-06T04:49:29.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.867+0000 7fb85659c700 1 --2- 172.21.15.55:0/1808258619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb840003c20 0x7fb8580c7e30 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fb848001120 tx=0x7fb848015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:29.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.867+0000 7fb8467fc700 1 -- 172.21.15.55:0/1808258619 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb848005c90 con 0x7fb840003c20 2023-10-06T04:49:29.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.868+0000 7fb8467fc700 1 -- 172.21.15.55:0/1808258619 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb848005df0 con 0x7fb840003c20 2023-10-06T04:49:29.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.868+0000 7fb85759e700 1 -- 172.21.15.55:0/1808258619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8580c3580 con 0x7fb840003c20 2023-10-06T04:49:29.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.868+0000 7fb8467fc700 1 -- 172.21.15.55:0/1808258619 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb848007320 con 0x7fb840003c20 2023-10-06T04:49:29.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.868+0000 7fb85759e700 1 -- 172.21.15.55:0/1808258619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85815a8b0 con 0x7fb840003c20 2023-10-06T04:49:29.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.869+0000 7fb85759e700 1 -- 172.21.15.55:0/1808258619 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85804c3c0 con 0x7fb840003c20 2023-10-06T04:49:29.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.869+0000 7fb8467fc700 1 -- 172.21.15.55:0/1808258619 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb848028450 con 0x7fb840003c20 2023-10-06T04:49:29.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.871+0000 7fb8467fc700 1 --2- 172.21.15.55:0/1808258619 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb838066f30 0x7fb8380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:29.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.871+0000 7fb8467fc700 1 -- 172.21.15.55:0/1808258619 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fb848085e50 con 0x7fb840003c20 2023-10-06T04:49:29.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.871+0000 7fb855d9b700 1 --2- 172.21.15.55:0/1808258619 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb838066f30 0x7fb8380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:29.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.872+0000 7fb855d9b700 1 --2- 172.21.15.55:0/1808258619 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb838066f30 0x7fb8380693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0010e0 tx=0x7fb84c007d30 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:29.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:29.875+0000 7fb8467fc700 1 -- 172.21.15.55:0/1808258619 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb848052d30 con 0x7fb840003c20 2023-10-06T04:49:30.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.071+0000 7fb85759e700 1 -- 172.21.15.55:0/1808258619 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb8580c8740 con 0x7fb838066f30 2023-10-06T04:49:30.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.081+0000 7fb8467fc700 1 -- 172.21.15.55:0/1808258619 <== mgr.8557 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311263 (secure 0 0 0) 0x7fb8580c8740 con 0x7fb838066f30 2023-10-06T04:49:30.087 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:30.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.085+0000 7fb85759e700 1 -- 172.21.15.55:0/1808258619 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb838066f30 msgr2=0x7fb8380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:30.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.085+0000 7fb85759e700 1 --2- 172.21.15.55:0/1808258619 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb838066f30 0x7fb8380693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb84c0010e0 tx=0x7fb84c007d30 comp rx=0 tx=0).stop 2023-10-06T04:49:30.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.085+0000 7fb85759e700 1 -- 172.21.15.55:0/1808258619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb840003c20 msgr2=0x7fb8580c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:30.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.085+0000 7fb85759e700 1 --2- 172.21.15.55:0/1808258619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb840003c20 0x7fb8580c7e30 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fb848001120 tx=0x7fb848015000 comp rx=0 tx=0).stop 2023-10-06T04:49:30.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.085+0000 7fb85759e700 1 -- 172.21.15.55:0/1808258619 shutdown_connections 2023-10-06T04:49:30.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.085+0000 7fb85759e700 1 --2- 172.21.15.55:0/1808258619 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb838066f30 0x7fb8380693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.085+0000 7fb85759e700 1 --2- 172.21.15.55:0/1808258619 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb858157b50 0x7fb8580c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.085+0000 7fb85759e700 1 --2- 172.21.15.55:0/1808258619 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb840003c20 0x7fb8580c7e30 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.085+0000 7fb85759e700 1 --2- 172.21.15.55:0/1808258619 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb85815a440 0x7fb858143c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.086+0000 7fb85759e700 1 -- 172.21.15.55:0/1808258619 >> 172.21.15.55:0/1808258619 conn(0x7fb8580cc1f0 msgr2=0x7fb858155120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:30.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.086+0000 7fb85759e700 1 -- 172.21.15.55:0/1808258619 shutdown_connections 2023-10-06T04:49:30.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.086+0000 7fb85759e700 1 -- 172.21.15.55:0/1808258619 wait complete. 2023-10-06T04:49:30.091 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T04:49:30.105 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-06T04:49:29.339118+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":524,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":332,"ondisk_log_size":332,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2453720,"kb_used_data":18072,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752521000,"statfs":{"total":773094113280,"available":770581504000,"internally_reserved":0,"allocated":18505728,"data_stored":10233204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.023370"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346440+0000","last_change":"2023-10-06T04:49:20.341012+0000","last_active":"2023-10-06T04:49:20.346440+0000","last_peered":"2023-10-06T04:49:20.346440+0000","last_clean":"2023-10-06T04:49:20.346440+0000","last_became_active":"2023-10-06T04:49:20.340740+0000","last_became_peered":"2023-10-06T04:49:20.340740+0000","last_unstale":"2023-10-06T04:49:20.346440+0000","last_undegraded":"2023-10-06T04:49:20.346440+0000","last_fullsized":"2023-10-06T04:49:20.346440+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341092+0000","last_change":"2023-10-06T04:49:20.336745+0000","last_active":"2023-10-06T04:49:20.341092+0000","last_peered":"2023-10-06T04:49:20.341092+0000","last_clean":"2023-10-06T04:49:20.341092+0000","last_became_active":"2023-10-06T04:49:20.336500+0000","last_became_peered":"2023-10-06T04:49:20.336500+0000","last_unstale":"2023-10-06T04:49:20.341092+0000","last_undegraded":"2023-10-06T04:49:20.341092+0000","last_fullsized":"2023-10-06T04:49:20.341092+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:28.214277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341469+0000","last_change":"2023-10-06T04:49:20.334944+0000","last_active":"2023-10-06T04:49:20.341469+0000","last_peered":"2023-10-06T04:49:20.341469+0000","last_clean":"2023-10-06T04:49:20.341469+0000","last_became_active":"2023-10-06T04:49:20.333924+0000","last_became_peered":"2023-10-06T04:49:20.333924+0000","last_unstale":"2023-10-06T04:49:20.341469+0000","last_undegraded":"2023-10-06T04:49:20.341469+0000","last_fullsized":"2023-10-06T04:49:20.341469+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341150+0000","last_change":"2023-10-06T04:49:20.333903+0000","last_active":"2023-10-06T04:49:20.341150+0000","last_peered":"2023-10-06T04:49:20.341150+0000","last_clean":"2023-10-06T04:49:20.341150+0000","last_became_active":"2023-10-06T04:49:20.333572+0000","last_became_peered":"2023-10-06T04:49:20.333572+0000","last_unstale":"2023-10-06T04:49:20.341150+0000","last_undegraded":"2023-10-06T04:49:20.341150+0000","last_fullsized":"2023-10-06T04:49:20.341150+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:49.977768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339683+0000","last_change":"2023-10-06T04:49:20.338306+0000","last_active":"2023-10-06T04:49:20.339683+0000","last_peered":"2023-10-06T04:49:20.339683+0000","last_clean":"2023-10-06T04:49:20.339683+0000","last_became_active":"2023-10-06T04:49:20.338034+0000","last_became_peered":"2023-10-06T04:49:20.338034+0000","last_unstale":"2023-10-06T04:49:20.339683+0000","last_undegraded":"2023-10-06T04:49:20.339683+0000","last_fullsized":"2023-10-06T04:49:20.339683+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339447+0000","last_change":"2023-10-06T04:49:20.336417+0000","last_active":"2023-10-06T04:49:20.339447+0000","last_peered":"2023-10-06T04:49:20.339447+0000","last_clean":"2023-10-06T04:49:20.339447+0000","last_became_active":"2023-10-06T04:49:20.334348+0000","last_became_peered":"2023-10-06T04:49:20.334348+0000","last_unstale":"2023-10-06T04:49:20.339447+0000","last_undegraded":"2023-10-06T04:49:20.339447+0000","last_fullsized":"2023-10-06T04:49:20.339447+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:54.036275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.345389+0000","last_change":"2023-10-06T04:49:20.339825+0000","last_active":"2023-10-06T04:49:20.345389+0000","last_peered":"2023-10-06T04:49:20.345389+0000","last_clean":"2023-10-06T04:49:20.345389+0000","last_became_active":"2023-10-06T04:49:20.339565+0000","last_became_peered":"2023-10-06T04:49:20.339565+0000","last_unstale":"2023-10-06T04:49:20.345389+0000","last_undegraded":"2023-10-06T04:49:20.345389+0000","last_fullsized":"2023-10-06T04:49:20.345389+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:03:54.036275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347030+0000","last_change":"2023-10-06T04:49:20.336205+0000","last_active":"2023-10-06T04:49:20.347030+0000","last_peered":"2023-10-06T04:49:20.347030+0000","last_clean":"2023-10-06T04:49:20.347030+0000","last_became_active":"2023-10-06T04:49:20.334936+0000","last_became_peered":"2023-10-06T04:49:20.334936+0000","last_unstale":"2023-10-06T04:49:20.347030+0000","last_undegraded":"2023-10-06T04:49:20.347030+0000","last_fullsized":"2023-10-06T04:49:20.347030+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:34.988176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.340009+0000","last_change":"2023-10-06T04:49:20.336646+0000","last_active":"2023-10-06T04:49:20.340009+0000","last_peered":"2023-10-06T04:49:20.340009+0000","last_clean":"2023-10-06T04:49:20.340009+0000","last_became_active":"2023-10-06T04:49:20.334441+0000","last_became_peered":"2023-10-06T04:49:20.334441+0000","last_unstale":"2023-10-06T04:49:20.340009+0000","last_undegraded":"2023-10-06T04:49:20.340009+0000","last_fullsized":"2023-10-06T04:49:20.340009+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:23.582332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333816+0000","last_change":"2023-10-06T04:49:20.331180+0000","last_active":"2023-10-06T04:49:20.333816+0000","last_peered":"2023-10-06T04:49:20.333816+0000","last_clean":"2023-10-06T04:49:20.333816+0000","last_became_active":"2023-10-06T04:49:20.330817+0000","last_became_peered":"2023-10-06T04:49:20.330817+0000","last_unstale":"2023-10-06T04:49:20.333816+0000","last_undegraded":"2023-10-06T04:49:20.333816+0000","last_fullsized":"2023-10-06T04:49:20.333816+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.336853+0000","last_change":"2023-10-06T04:49:20.336069+0000","last_active":"2023-10-06T04:49:20.336853+0000","last_peered":"2023-10-06T04:49:20.336853+0000","last_clean":"2023-10-06T04:49:20.336853+0000","last_became_active":"2023-10-06T04:49:20.335808+0000","last_became_peered":"2023-10-06T04:49:20.335808+0000","last_unstale":"2023-10-06T04:49:20.336853+0000","last_undegraded":"2023-10-06T04:49:20.336853+0000","last_fullsized":"2023-10-06T04:49:20.336853+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:53.477421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.334968+0000","last_change":"2023-10-06T04:49:20.328201+0000","last_active":"2023-10-06T04:49:20.334968+0000","last_peered":"2023-10-06T04:49:20.334968+0000","last_clean":"2023-10-06T04:49:20.334968+0000","last_became_active":"2023-10-06T04:49:20.327560+0000","last_became_peered":"2023-10-06T04:49:20.327560+0000","last_unstale":"2023-10-06T04:49:20.334968+0000","last_undegraded":"2023-10-06T04:49:20.334968+0000","last_fullsized":"2023-10-06T04:49:20.334968+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:16:40.721698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333739+0000","last_change":"2023-10-06T04:49:20.331130+0000","last_active":"2023-10-06T04:49:20.333739+0000","last_peered":"2023-10-06T04:49:20.333739+0000","last_clean":"2023-10-06T04:49:20.333739+0000","last_became_active":"2023-10-06T04:49:20.330637+0000","last_became_peered":"2023-10-06T04:49:20.330637+0000","last_unstale":"2023-10-06T04:49:20.333739+0000","last_undegraded":"2023-10-06T04:49:20.333739+0000","last_fullsized":"2023-10-06T04:49:20.333739+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341277+0000","last_change":"2023-10-06T04:49:20.337139+0000","last_active":"2023-10-06T04:49:20.341277+0000","last_peered":"2023-10-06T04:49:20.341277+0000","last_clean":"2023-10-06T04:49:20.341277+0000","last_became_active":"2023-10-06T04:49:20.336396+0000","last_became_peered":"2023-10-06T04:49:20.336396+0000","last_unstale":"2023-10-06T04:49:20.341277+0000","last_undegraded":"2023-10-06T04:49:20.341277+0000","last_fullsized":"2023-10-06T04:49:20.341277+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:58:22.496792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341177+0000","last_change":"2023-10-06T04:49:20.337332+0000","last_active":"2023-10-06T04:49:20.341177+0000","last_peered":"2023-10-06T04:49:20.341177+0000","last_clean":"2023-10-06T04:49:20.341177+0000","last_became_active":"2023-10-06T04:49:20.337091+0000","last_became_peered":"2023-10-06T04:49:20.337091+0000","last_unstale":"2023-10-06T04:49:20.341177+0000","last_undegraded":"2023-10-06T04:49:20.341177+0000","last_fullsized":"2023-10-06T04:49:20.341177+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:57.445018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333411+0000","last_change":"2023-10-06T04:49:20.329112+0000","last_active":"2023-10-06T04:49:20.333411+0000","last_peered":"2023-10-06T04:49:20.333411+0000","last_clean":"2023-10-06T04:49:20.333411+0000","last_became_active":"2023-10-06T04:49:20.328156+0000","last_became_peered":"2023-10-06T04:49:20.328156+0000","last_unstale":"2023-10-06T04:49:20.333411+0000","last_undegraded":"2023-10-06T04:49:20.333411+0000","last_fullsized":"2023-10-06T04:49:20.333411+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:25.017277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341591+0000","last_change":"2023-10-06T04:49:20.334020+0000","last_active":"2023-10-06T04:49:20.341591+0000","last_peered":"2023-10-06T04:49:20.341591+0000","last_clean":"2023-10-06T04:49:20.341591+0000","last_became_active":"2023-10-06T04:49:20.333516+0000","last_became_peered":"2023-10-06T04:49:20.333516+0000","last_unstale":"2023-10-06T04:49:20.341591+0000","last_undegraded":"2023-10-06T04:49:20.341591+0000","last_fullsized":"2023-10-06T04:49:20.341591+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:03:43.552397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347965+0000","last_change":"2023-10-06T04:49:20.346422+0000","last_active":"2023-10-06T04:49:20.347965+0000","last_peered":"2023-10-06T04:49:20.347965+0000","last_clean":"2023-10-06T04:49:20.347965+0000","last_became_active":"2023-10-06T04:49:20.346126+0000","last_became_peered":"2023-10-06T04:49:20.346126+0000","last_unstale":"2023-10-06T04:49:20.347965+0000","last_undegraded":"2023-10-06T04:49:20.347965+0000","last_fullsized":"2023-10-06T04:49:20.347965+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:58.644879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355577+0000","last_change":"2023-10-06T04:49:20.350471+0000","last_active":"2023-10-06T04:49:20.355577+0000","last_peered":"2023-10-06T04:49:20.355577+0000","last_clean":"2023-10-06T04:49:20.355577+0000","last_became_active":"2023-10-06T04:49:20.350195+0000","last_became_peered":"2023-10-06T04:49:20.350195+0000","last_unstale":"2023-10-06T04:49:20.355577+0000","last_undegraded":"2023-10-06T04:49:20.355577+0000","last_fullsized":"2023-10-06T04:49:20.355577+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:12.727118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339262+0000","last_change":"2023-10-06T04:49:20.334837+0000","last_active":"2023-10-06T04:49:20.339262+0000","last_peered":"2023-10-06T04:49:20.339262+0000","last_clean":"2023-10-06T04:49:20.339262+0000","last_became_active":"2023-10-06T04:49:20.334580+0000","last_became_peered":"2023-10-06T04:49:20.334580+0000","last_unstale":"2023-10-06T04:49:20.339262+0000","last_undegraded":"2023-10-06T04:49:20.339262+0000","last_fullsized":"2023-10-06T04:49:20.339262+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:02.862072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.338389+0000","last_change":"2023-10-06T04:49:20.334048+0000","last_active":"2023-10-06T04:49:20.338389+0000","last_peered":"2023-10-06T04:49:20.338389+0000","last_clean":"2023-10-06T04:49:20.338389+0000","last_became_active":"2023-10-06T04:49:20.333761+0000","last_became_peered":"2023-10-06T04:49:20.333761+0000","last_unstale":"2023-10-06T04:49:20.338389+0000","last_undegraded":"2023-10-06T04:49:20.338389+0000","last_fullsized":"2023-10-06T04:49:20.338389+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344972+0000","last_change":"2023-10-06T04:49:20.336537+0000","last_active":"2023-10-06T04:49:20.344972+0000","last_peered":"2023-10-06T04:49:20.344972+0000","last_clean":"2023-10-06T04:49:20.344972+0000","last_became_active":"2023-10-06T04:49:20.336251+0000","last_became_peered":"2023-10-06T04:49:20.336251+0000","last_unstale":"2023-10-06T04:49:20.344972+0000","last_undegraded":"2023-10-06T04:49:20.344972+0000","last_fullsized":"2023-10-06T04:49:20.344972+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:11:50.011042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347191+0000","last_change":"2023-10-06T04:49:20.344026+0000","last_active":"2023-10-06T04:49:20.347191+0000","last_peered":"2023-10-06T04:49:20.347191+0000","last_clean":"2023-10-06T04:49:20.347191+0000","last_became_active":"2023-10-06T04:49:20.343761+0000","last_became_peered":"2023-10-06T04:49:20.343761+0000","last_unstale":"2023-10-06T04:49:20.347191+0000","last_undegraded":"2023-10-06T04:49:20.347191+0000","last_fullsized":"2023-10-06T04:49:20.347191+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:07.881803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.329722+0000","last_change":"2023-10-06T04:49:20.326750+0000","last_active":"2023-10-06T04:49:20.329722+0000","last_peered":"2023-10-06T04:49:20.329722+0000","last_clean":"2023-10-06T04:49:20.329722+0000","last_became_active":"2023-10-06T04:49:20.326466+0000","last_became_peered":"2023-10-06T04:49:20.326466+0000","last_unstale":"2023-10-06T04:49:20.329722+0000","last_undegraded":"2023-10-06T04:49:20.329722+0000","last_fullsized":"2023-10-06T04:49:20.329722+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:47:52.748607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.345875+0000","last_change":"2023-10-06T04:49:20.337383+0000","last_active":"2023-10-06T04:49:20.345875+0000","last_peered":"2023-10-06T04:49:20.345875+0000","last_clean":"2023-10-06T04:49:20.345875+0000","last_became_active":"2023-10-06T04:49:20.337130+0000","last_became_peered":"2023-10-06T04:49:20.337130+0000","last_unstale":"2023-10-06T04:49:20.345875+0000","last_undegraded":"2023-10-06T04:49:20.345875+0000","last_fullsized":"2023-10-06T04:49:20.345875+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355403+0000","last_change":"2023-10-06T04:49:20.347903+0000","last_active":"2023-10-06T04:49:20.355403+0000","last_peered":"2023-10-06T04:49:20.355403+0000","last_clean":"2023-10-06T04:49:20.355403+0000","last_became_active":"2023-10-06T04:49:20.346956+0000","last_became_peered":"2023-10-06T04:49:20.346956+0000","last_unstale":"2023-10-06T04:49:20.355403+0000","last_undegraded":"2023-10-06T04:49:20.355403+0000","last_fullsized":"2023-10-06T04:49:20.355403+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:20.026936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339401+0000","last_change":"2023-10-06T04:49:20.336472+0000","last_active":"2023-10-06T04:49:20.339401+0000","last_peered":"2023-10-06T04:49:20.339401+0000","last_clean":"2023-10-06T04:49:20.339401+0000","last_became_active":"2023-10-06T04:49:20.334373+0000","last_became_peered":"2023-10-06T04:49:20.334373+0000","last_unstale":"2023-10-06T04:49:20.339401+0000","last_undegraded":"2023-10-06T04:49:20.339401+0000","last_fullsized":"2023-10-06T04:49:20.339401+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:20.026936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346959+0000","last_change":"2023-10-06T04:49:20.344850+0000","last_active":"2023-10-06T04:49:20.346959+0000","last_peered":"2023-10-06T04:49:20.346959+0000","last_clean":"2023-10-06T04:49:20.346959+0000","last_became_active":"2023-10-06T04:49:20.343891+0000","last_became_peered":"2023-10-06T04:49:20.343891+0000","last_unstale":"2023-10-06T04:49:20.346959+0000","last_undegraded":"2023-10-06T04:49:20.346959+0000","last_fullsized":"2023-10-06T04:49:20.346959+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:28.957190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347052+0000","last_change":"2023-10-06T04:49:20.343469+0000","last_active":"2023-10-06T04:49:20.347052+0000","last_peered":"2023-10-06T04:49:20.347052+0000","last_clean":"2023-10-06T04:49:20.347052+0000","last_became_active":"2023-10-06T04:49:20.343165+0000","last_became_peered":"2023-10-06T04:49:20.343165+0000","last_unstale":"2023-10-06T04:49:20.347052+0000","last_undegraded":"2023-10-06T04:49:20.347052+0000","last_fullsized":"2023-10-06T04:49:20.347052+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:12.727118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347992+0000","last_change":"2023-10-06T04:49:20.344409+0000","last_active":"2023-10-06T04:49:20.347992+0000","last_peered":"2023-10-06T04:49:20.347992+0000","last_clean":"2023-10-06T04:49:20.347992+0000","last_became_active":"2023-10-06T04:49:20.343734+0000","last_became_peered":"2023-10-06T04:49:20.343734+0000","last_unstale":"2023-10-06T04:49:20.347992+0000","last_undegraded":"2023-10-06T04:49:20.347992+0000","last_fullsized":"2023-10-06T04:49:20.347992+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:27.555498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333496+0000","last_change":"2023-10-06T04:49:20.327245+0000","last_active":"2023-10-06T04:49:20.333496+0000","last_peered":"2023-10-06T04:49:20.333496+0000","last_clean":"2023-10-06T04:49:20.333496+0000","last_became_active":"2023-10-06T04:49:20.326914+0000","last_became_peered":"2023-10-06T04:49:20.326914+0000","last_unstale":"2023-10-06T04:49:20.333496+0000","last_undegraded":"2023-10-06T04:49:20.333496+0000","last_fullsized":"2023-10-06T04:49:20.333496+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:41:19.514355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347097+0000","last_change":"2023-10-06T04:49:20.339379+0000","last_active":"2023-10-06T04:49:20.347097+0000","last_peered":"2023-10-06T04:49:20.347097+0000","last_clean":"2023-10-06T04:49:20.347097+0000","last_became_active":"2023-10-06T04:49:20.339031+0000","last_became_peered":"2023-10-06T04:49:20.339031+0000","last_unstale":"2023-10-06T04:49:20.347097+0000","last_undegraded":"2023-10-06T04:49:20.347097+0000","last_fullsized":"2023-10-06T04:49:20.347097+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:21:51.900026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341007+0000","last_change":"2023-10-06T04:49:20.333556+0000","last_active":"2023-10-06T04:49:20.341007+0000","last_peered":"2023-10-06T04:49:20.341007+0000","last_clean":"2023-10-06T04:49:20.341007+0000","last_became_active":"2023-10-06T04:49:20.333251+0000","last_became_peered":"2023-10-06T04:49:20.333251+0000","last_unstale":"2023-10-06T04:49:20.341007+0000","last_undegraded":"2023-10-06T04:49:20.341007+0000","last_fullsized":"2023-10-06T04:49:20.341007+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:52:18.866925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"172'4","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344554+0000","last_change":"2023-10-06T04:49:19.320190+0000","last_active":"2023-10-06T04:49:20.344554+0000","last_peered":"2023-10-06T04:49:20.344554+0000","last_clean":"2023-10-06T04:49:20.344554+0000","last_became_active":"2023-10-06T04:49:19.319917+0000","last_became_peered":"2023-10-06T04:49:19.319917+0000","last_unstale":"2023-10-06T04:49:20.344554+0000","last_undegraded":"2023-10-06T04:49:20.344554+0000","last_fullsized":"2023-10-06T04:49:20.344554+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:26.266062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341489+0000","last_change":"2023-10-06T04:49:20.336237+0000","last_active":"2023-10-06T04:49:20.341489+0000","last_peered":"2023-10-06T04:49:20.341489+0000","last_clean":"2023-10-06T04:49:20.341489+0000","last_became_active":"2023-10-06T04:49:20.335986+0000","last_became_peered":"2023-10-06T04:49:20.335986+0000","last_unstale":"2023-10-06T04:49:20.341489+0000","last_undegraded":"2023-10-06T04:49:20.341489+0000","last_fullsized":"2023-10-06T04:49:20.341489+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:40:19.333066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355736+0000","last_change":"2023-10-06T04:49:19.335778+0000","last_active":"2023-10-06T04:49:20.355736+0000","last_peered":"2023-10-06T04:49:20.355736+0000","last_clean":"2023-10-06T04:49:20.355736+0000","last_became_active":"2023-10-06T04:49:19.335470+0000","last_became_peered":"2023-10-06T04:49:19.335470+0000","last_unstale":"2023-10-06T04:49:20.355736+0000","last_undegraded":"2023-10-06T04:49:20.355736+0000","last_fullsized":"2023-10-06T04:49:20.355736+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:16:01.854229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347502+0000","last_change":"2023-10-06T04:49:20.339407+0000","last_active":"2023-10-06T04:49:20.347502+0000","last_peered":"2023-10-06T04:49:20.347502+0000","last_clean":"2023-10-06T04:49:20.347502+0000","last_became_active":"2023-10-06T04:49:20.335811+0000","last_became_peered":"2023-10-06T04:49:20.335811+0000","last_unstale":"2023-10-06T04:49:20.347502+0000","last_undegraded":"2023-10-06T04:49:20.347502+0000","last_fullsized":"2023-10-06T04:49:20.347502+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:10.637938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356161+0000","last_change":"2023-10-06T04:49:19.335543+0000","last_active":"2023-10-06T04:49:20.356161+0000","last_peered":"2023-10-06T04:49:20.356161+0000","last_clean":"2023-10-06T04:49:20.356161+0000","last_became_active":"2023-10-06T04:49:19.335265+0000","last_became_peered":"2023-10-06T04:49:19.335265+0000","last_unstale":"2023-10-06T04:49:20.356161+0000","last_undegraded":"2023-10-06T04:49:20.356161+0000","last_fullsized":"2023-10-06T04:49:20.356161+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:41.377516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350856+0000","last_change":"2023-10-06T04:49:20.346902+0000","last_active":"2023-10-06T04:49:20.350856+0000","last_peered":"2023-10-06T04:49:20.350856+0000","last_clean":"2023-10-06T04:49:20.350856+0000","last_became_active":"2023-10-06T04:49:20.346594+0000","last_became_peered":"2023-10-06T04:49:20.346594+0000","last_unstale":"2023-10-06T04:49:20.350856+0000","last_undegraded":"2023-10-06T04:49:20.350856+0000","last_fullsized":"2023-10-06T04:49:20.350856+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:00.206328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.349018+0000","last_change":"2023-10-06T04:49:19.328489+0000","last_active":"2023-10-06T04:49:20.349018+0000","last_peered":"2023-10-06T04:49:20.349018+0000","last_clean":"2023-10-06T04:49:20.349018+0000","last_became_active":"2023-10-06T04:49:19.328220+0000","last_became_peered":"2023-10-06T04:49:19.328220+0000","last_unstale":"2023-10-06T04:49:20.349018+0000","last_undegraded":"2023-10-06T04:49:20.349018+0000","last_fullsized":"2023-10-06T04:49:20.349018+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:18:26.266062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.337318+0000","last_change":"2023-10-06T04:49:20.330590+0000","last_active":"2023-10-06T04:49:20.337318+0000","last_peered":"2023-10-06T04:49:20.337318+0000","last_clean":"2023-10-06T04:49:20.337318+0000","last_became_active":"2023-10-06T04:49:20.330306+0000","last_became_peered":"2023-10-06T04:49:20.330306+0000","last_unstale":"2023-10-06T04:49:20.337318+0000","last_undegraded":"2023-10-06T04:49:20.337318+0000","last_fullsized":"2023-10-06T04:49:20.337318+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:40:42.535859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.351406+0000","last_change":"2023-10-06T04:49:19.324990+0000","last_active":"2023-10-06T04:49:20.351406+0000","last_peered":"2023-10-06T04:49:20.351406+0000","last_clean":"2023-10-06T04:49:20.351406+0000","last_became_active":"2023-10-06T04:49:19.324681+0000","last_became_peered":"2023-10-06T04:49:19.324681+0000","last_unstale":"2023-10-06T04:49:20.351406+0000","last_undegraded":"2023-10-06T04:49:20.351406+0000","last_fullsized":"2023-10-06T04:49:20.351406+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:25:40.637063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341432+0000","last_change":"2023-10-06T04:49:20.334543+0000","last_active":"2023-10-06T04:49:20.341432+0000","last_peered":"2023-10-06T04:49:20.341432+0000","last_clean":"2023-10-06T04:49:20.341432+0000","last_became_active":"2023-10-06T04:49:20.333665+0000","last_became_peered":"2023-10-06T04:49:20.333665+0000","last_unstale":"2023-10-06T04:49:20.341432+0000","last_undegraded":"2023-10-06T04:49:20.341432+0000","last_fullsized":"2023-10-06T04:49:20.341432+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:12:20.869185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347200+0000","last_change":"2023-10-06T04:49:19.330786+0000","last_active":"2023-10-06T04:49:20.347200+0000","last_peered":"2023-10-06T04:49:20.347200+0000","last_clean":"2023-10-06T04:49:20.347200+0000","last_became_active":"2023-10-06T04:49:19.330506+0000","last_became_peered":"2023-10-06T04:49:19.330506+0000","last_unstale":"2023-10-06T04:49:20.347200+0000","last_undegraded":"2023-10-06T04:49:20.347200+0000","last_fullsized":"2023-10-06T04:49:20.347200+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:54.415706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333277+0000","last_change":"2023-10-06T04:49:20.332521+0000","last_active":"2023-10-06T04:49:20.333277+0000","last_peered":"2023-10-06T04:49:20.333277+0000","last_clean":"2023-10-06T04:49:20.333277+0000","last_became_active":"2023-10-06T04:49:20.332264+0000","last_became_peered":"2023-10-06T04:49:20.332264+0000","last_unstale":"2023-10-06T04:49:20.333277+0000","last_undegraded":"2023-10-06T04:49:20.333277+0000","last_fullsized":"2023-10-06T04:49:20.333277+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:56:16.543656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332961+0000","last_change":"2023-10-06T04:49:19.332296+0000","last_active":"2023-10-06T04:49:20.332961+0000","last_peered":"2023-10-06T04:49:20.332961+0000","last_clean":"2023-10-06T04:49:20.332961+0000","last_became_active":"2023-10-06T04:49:19.332022+0000","last_became_peered":"2023-10-06T04:49:19.332022+0000","last_unstale":"2023-10-06T04:49:20.332961+0000","last_undegraded":"2023-10-06T04:49:20.332961+0000","last_fullsized":"2023-10-06T04:49:20.332961+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:18.846190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350851+0000","last_change":"2023-10-06T04:49:20.341492+0000","last_active":"2023-10-06T04:49:20.350851+0000","last_peered":"2023-10-06T04:49:20.350851+0000","last_clean":"2023-10-06T04:49:20.350851+0000","last_became_active":"2023-10-06T04:49:20.341257+0000","last_became_peered":"2023-10-06T04:49:20.341257+0000","last_unstale":"2023-10-06T04:49:20.350851+0000","last_undegraded":"2023-10-06T04:49:20.350851+0000","last_fullsized":"2023-10-06T04:49:20.350851+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:26:28.214277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.315096+0000","last_change":"2023-10-06T04:49:19.324312+0000","last_active":"2023-10-06T04:49:20.315096+0000","last_peered":"2023-10-06T04:49:20.315096+0000","last_clean":"2023-10-06T04:49:20.315096+0000","last_became_active":"2023-10-06T04:49:19.324046+0000","last_became_peered":"2023-10-06T04:49:19.324046+0000","last_unstale":"2023-10-06T04:49:20.315096+0000","last_undegraded":"2023-10-06T04:49:20.315096+0000","last_fullsized":"2023-10-06T04:49:20.315096+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:53:49.588344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344426+0000","last_change":"2023-10-06T04:49:20.341875+0000","last_active":"2023-10-06T04:49:20.344426+0000","last_peered":"2023-10-06T04:49:20.344426+0000","last_clean":"2023-10-06T04:49:20.344426+0000","last_became_active":"2023-10-06T04:49:20.341312+0000","last_became_peered":"2023-10-06T04:49:20.341312+0000","last_unstale":"2023-10-06T04:49:20.344426+0000","last_undegraded":"2023-10-06T04:49:20.344426+0000","last_fullsized":"2023-10-06T04:49:20.344426+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:30:07.881803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.351368+0000","last_change":"2023-10-06T04:49:19.331266+0000","last_active":"2023-10-06T04:49:20.351368+0000","last_peered":"2023-10-06T04:49:20.351368+0000","last_clean":"2023-10-06T04:49:20.351368+0000","last_became_active":"2023-10-06T04:49:19.331010+0000","last_became_peered":"2023-10-06T04:49:19.331010+0000","last_unstale":"2023-10-06T04:49:20.351368+0000","last_undegraded":"2023-10-06T04:49:20.351368+0000","last_fullsized":"2023-10-06T04:49:20.351368+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:56:41.647975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355558+0000","last_change":"2023-10-06T04:49:20.350352+0000","last_active":"2023-10-06T04:49:20.355558+0000","last_peered":"2023-10-06T04:49:20.355558+0000","last_clean":"2023-10-06T04:49:20.355558+0000","last_became_active":"2023-10-06T04:49:20.350084+0000","last_became_peered":"2023-10-06T04:49:20.350084+0000","last_unstale":"2023-10-06T04:49:20.355558+0000","last_undegraded":"2023-10-06T04:49:20.355558+0000","last_fullsized":"2023-10-06T04:49:20.355558+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:54.488863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.353329+0000","last_change":"2023-10-06T04:49:19.333467+0000","last_active":"2023-10-06T04:49:20.353329+0000","last_peered":"2023-10-06T04:49:20.353329+0000","last_clean":"2023-10-06T04:49:20.353329+0000","last_became_active":"2023-10-06T04:49:19.333191+0000","last_became_peered":"2023-10-06T04:49:19.333191+0000","last_unstale":"2023-10-06T04:49:20.353329+0000","last_undegraded":"2023-10-06T04:49:20.353329+0000","last_fullsized":"2023-10-06T04:49:20.353329+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:50.631194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346482+0000","last_change":"2023-10-06T04:49:20.340075+0000","last_active":"2023-10-06T04:49:20.346482+0000","last_peered":"2023-10-06T04:49:20.346482+0000","last_clean":"2023-10-06T04:49:20.346482+0000","last_became_active":"2023-10-06T04:49:20.339781+0000","last_became_peered":"2023-10-06T04:49:20.339781+0000","last_unstale":"2023-10-06T04:49:20.346482+0000","last_undegraded":"2023-10-06T04:49:20.346482+0000","last_fullsized":"2023-10-06T04:49:20.346482+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:57.445018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344697+0000","last_change":"2023-10-06T04:49:19.324434+0000","last_active":"2023-10-06T04:49:20.344697+0000","last_peered":"2023-10-06T04:49:20.344697+0000","last_clean":"2023-10-06T04:49:20.344697+0000","last_became_active":"2023-10-06T04:49:19.324162+0000","last_became_peered":"2023-10-06T04:49:19.324162+0000","last_unstale":"2023-10-06T04:49:20.344697+0000","last_undegraded":"2023-10-06T04:49:20.344697+0000","last_fullsized":"2023-10-06T04:49:20.344697+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350745+0000","last_change":"2023-10-06T04:49:20.345980+0000","last_active":"2023-10-06T04:49:20.350745+0000","last_peered":"2023-10-06T04:49:20.350745+0000","last_clean":"2023-10-06T04:49:20.350745+0000","last_became_active":"2023-10-06T04:49:20.345691+0000","last_became_peered":"2023-10-06T04:49:20.345691+0000","last_unstale":"2023-10-06T04:49:20.350745+0000","last_undegraded":"2023-10-06T04:49:20.350745+0000","last_fullsized":"2023-10-06T04:49:20.350745+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:44:23.582332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.319623+0000","last_change":"2023-10-06T04:49:19.330962+0000","last_active":"2023-10-06T04:49:20.319623+0000","last_peered":"2023-10-06T04:49:20.319623+0000","last_clean":"2023-10-06T04:49:20.319623+0000","last_became_active":"2023-10-06T04:49:19.330616+0000","last_became_peered":"2023-10-06T04:49:19.330616+0000","last_unstale":"2023-10-06T04:49:20.319623+0000","last_undegraded":"2023-10-06T04:49:20.319623+0000","last_fullsized":"2023-10-06T04:49:20.319623+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:04.272287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.355580+0000","last_change":"2023-10-06T04:49:20.350356+0000","last_active":"2023-10-06T04:49:20.355580+0000","last_peered":"2023-10-06T04:49:20.355580+0000","last_clean":"2023-10-06T04:49:20.355580+0000","last_became_active":"2023-10-06T04:49:20.350105+0000","last_became_peered":"2023-10-06T04:49:20.350105+0000","last_unstale":"2023-10-06T04:49:20.355580+0000","last_undegraded":"2023-10-06T04:49:20.355580+0000","last_fullsized":"2023-10-06T04:49:20.355580+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:10.637938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346898+0000","last_change":"2023-10-06T04:49:19.333331+0000","last_active":"2023-10-06T04:49:20.346898+0000","last_peered":"2023-10-06T04:49:20.346898+0000","last_clean":"2023-10-06T04:49:20.346898+0000","last_became_active":"2023-10-06T04:49:19.332774+0000","last_became_peered":"2023-10-06T04:49:19.332774+0000","last_unstale":"2023-10-06T04:49:20.346898+0000","last_undegraded":"2023-10-06T04:49:20.346898+0000","last_fullsized":"2023-10-06T04:49:20.346898+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:19.019093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.346151+0000","last_change":"2023-10-06T04:49:20.345404+0000","last_active":"2023-10-06T04:49:20.346151+0000","last_peered":"2023-10-06T04:49:20.346151+0000","last_clean":"2023-10-06T04:49:20.346151+0000","last_became_active":"2023-10-06T04:49:20.345129+0000","last_became_peered":"2023-10-06T04:49:20.345129+0000","last_unstale":"2023-10-06T04:49:20.346151+0000","last_undegraded":"2023-10-06T04:49:20.346151+0000","last_fullsized":"2023-10-06T04:49:20.346151+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:17:25.017277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.319374+0000","last_change":"2023-10-06T04:49:19.322897+0000","last_active":"2023-10-06T04:49:20.319374+0000","last_peered":"2023-10-06T04:49:20.319374+0000","last_clean":"2023-10-06T04:49:20.319374+0000","last_became_active":"2023-10-06T04:49:19.322635+0000","last_became_peered":"2023-10-06T04:49:19.322635+0000","last_unstale":"2023-10-06T04:49:20.319374+0000","last_undegraded":"2023-10-06T04:49:20.319374+0000","last_fullsized":"2023-10-06T04:49:20.319374+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:04:19.019093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.350774+0000","last_change":"2023-10-06T04:49:20.347069+0000","last_active":"2023-10-06T04:49:20.350774+0000","last_peered":"2023-10-06T04:49:20.350774+0000","last_clean":"2023-10-06T04:49:20.350774+0000","last_became_active":"2023-10-06T04:49:20.346801+0000","last_became_peered":"2023-10-06T04:49:20.346801+0000","last_unstale":"2023-10-06T04:49:20.350774+0000","last_undegraded":"2023-10-06T04:49:20.350774+0000","last_fullsized":"2023-10-06T04:49:20.350774+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:50:34.988176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321187+0000","last_change":"2023-10-06T04:49:19.329229+0000","last_active":"2023-10-06T04:49:20.321187+0000","last_peered":"2023-10-06T04:49:20.321187+0000","last_clean":"2023-10-06T04:49:20.321187+0000","last_became_active":"2023-10-06T04:49:19.328964+0000","last_became_peered":"2023-10-06T04:49:19.328964+0000","last_unstale":"2023-10-06T04:49:20.321187+0000","last_undegraded":"2023-10-06T04:49:20.321187+0000","last_fullsized":"2023-10-06T04:49:20.321187+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:53.870884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341139+0000","last_change":"2023-10-06T04:49:20.334997+0000","last_active":"2023-10-06T04:49:20.341139+0000","last_peered":"2023-10-06T04:49:20.341139+0000","last_clean":"2023-10-06T04:49:20.341139+0000","last_became_active":"2023-10-06T04:49:20.334723+0000","last_became_peered":"2023-10-06T04:49:20.334723+0000","last_unstale":"2023-10-06T04:49:20.341139+0000","last_undegraded":"2023-10-06T04:49:20.341139+0000","last_fullsized":"2023-10-06T04:49:20.341139+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:37:41.560870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321711+0000","last_change":"2023-10-06T04:49:19.324862+0000","last_active":"2023-10-06T04:49:20.321711+0000","last_peered":"2023-10-06T04:49:20.321711+0000","last_clean":"2023-10-06T04:49:20.321711+0000","last_became_active":"2023-10-06T04:49:19.324595+0000","last_became_peered":"2023-10-06T04:49:19.324595+0000","last_unstale":"2023-10-06T04:49:20.321711+0000","last_undegraded":"2023-10-06T04:49:20.321711+0000","last_fullsized":"2023-10-06T04:49:20.321711+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:48:53.870884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339461+0000","last_change":"2023-10-06T04:49:20.337671+0000","last_active":"2023-10-06T04:49:20.339461+0000","last_peered":"2023-10-06T04:49:20.339461+0000","last_clean":"2023-10-06T04:49:20.339461+0000","last_became_active":"2023-10-06T04:49:20.337383+0000","last_became_peered":"2023-10-06T04:49:20.337383+0000","last_unstale":"2023-10-06T04:49:20.339461+0000","last_undegraded":"2023-10-06T04:49:20.339461+0000","last_fullsized":"2023-10-06T04:49:20.339461+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:41.316112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.342104+0000","last_change":"2023-10-06T04:49:19.328915+0000","last_active":"2023-10-06T04:49:20.342104+0000","last_peered":"2023-10-06T04:49:20.342104+0000","last_clean":"2023-10-06T04:49:20.342104+0000","last_became_active":"2023-10-06T04:49:19.328630+0000","last_became_peered":"2023-10-06T04:49:19.328630+0000","last_unstale":"2023-10-06T04:49:20.342104+0000","last_undegraded":"2023-10-06T04:49:20.342104+0000","last_fullsized":"2023-10-06T04:49:20.342104+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:28:27.949348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.343843+0000","last_change":"2023-10-06T04:49:20.341772+0000","last_active":"2023-10-06T04:49:20.343843+0000","last_peered":"2023-10-06T04:49:20.343843+0000","last_clean":"2023-10-06T04:49:20.343843+0000","last_became_active":"2023-10-06T04:49:20.341489+0000","last_became_peered":"2023-10-06T04:49:20.341489+0000","last_unstale":"2023-10-06T04:49:20.343843+0000","last_undegraded":"2023-10-06T04:49:20.343843+0000","last_fullsized":"2023-10-06T04:49:20.343843+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:51:10.460143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"173'1","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-10-06T04:49:22.371843+0000","last_change":"2023-10-06T04:49:19.331671+0000","last_active":"2023-10-06T04:49:22.371843+0000","last_peered":"2023-10-06T04:49:22.371843+0000","last_clean":"2023-10-06T04:49:22.371843+0000","last_became_active":"2023-10-06T04:49:19.330061+0000","last_became_peered":"2023-10-06T04:49:19.330061+0000","last_unstale":"2023-10-06T04:49:22.371843+0000","last_undegraded":"2023-10-06T04:49:22.371843+0000","last_fullsized":"2023-10-06T04:49:22.371843+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:23:09.188792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356042+0000","last_change":"2023-10-06T04:49:20.352177+0000","last_active":"2023-10-06T04:49:20.356042+0000","last_peered":"2023-10-06T04:49:20.356042+0000","last_clean":"2023-10-06T04:49:20.356042+0000","last_became_active":"2023-10-06T04:49:20.351885+0000","last_became_peered":"2023-10-06T04:49:20.351885+0000","last_unstale":"2023-10-06T04:49:20.356042+0000","last_undegraded":"2023-10-06T04:49:20.356042+0000","last_fullsized":"2023-10-06T04:49:20.356042+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320737+0000","last_change":"2023-10-06T04:49:19.329132+0000","last_active":"2023-10-06T04:49:20.320737+0000","last_peered":"2023-10-06T04:49:20.320737+0000","last_clean":"2023-10-06T04:49:20.320737+0000","last_became_active":"2023-10-06T04:49:19.328841+0000","last_became_peered":"2023-10-06T04:49:19.328841+0000","last_unstale":"2023-10-06T04:49:20.320737+0000","last_undegraded":"2023-10-06T04:49:20.320737+0000","last_fullsized":"2023-10-06T04:49:20.320737+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:28:54.415706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344647+0000","last_change":"2023-10-06T04:49:20.343226+0000","last_active":"2023-10-06T04:49:20.344647+0000","last_peered":"2023-10-06T04:49:20.344647+0000","last_clean":"2023-10-06T04:49:20.344647+0000","last_became_active":"2023-10-06T04:49:20.342932+0000","last_became_peered":"2023-10-06T04:49:20.342932+0000","last_unstale":"2023-10-06T04:49:20.344647+0000","last_undegraded":"2023-10-06T04:49:20.344647+0000","last_fullsized":"2023-10-06T04:49:20.344647+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:28.172716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.316079+0000","last_change":"2023-10-06T04:49:19.328230+0000","last_active":"2023-10-06T04:49:20.316079+0000","last_peered":"2023-10-06T04:49:20.316079+0000","last_clean":"2023-10-06T04:49:20.316079+0000","last_became_active":"2023-10-06T04:49:19.327917+0000","last_became_peered":"2023-10-06T04:49:19.327917+0000","last_unstale":"2023-10-06T04:49:20.316079+0000","last_undegraded":"2023-10-06T04:49:20.316079+0000","last_fullsized":"2023-10-06T04:49:20.316079+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:11:05.900580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347059+0000","last_change":"2023-10-06T04:49:20.336712+0000","last_active":"2023-10-06T04:49:20.347059+0000","last_peered":"2023-10-06T04:49:20.347059+0000","last_clean":"2023-10-06T04:49:20.347059+0000","last_became_active":"2023-10-06T04:49:20.335319+0000","last_became_peered":"2023-10-06T04:49:20.335319+0000","last_unstale":"2023-10-06T04:49:20.347059+0000","last_undegraded":"2023-10-06T04:49:20.347059+0000","last_fullsized":"2023-10-06T04:49:20.347059+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:20:00.206328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.325259+0000","last_change":"2023-10-06T04:49:19.331624+0000","last_active":"2023-10-06T04:49:20.325259+0000","last_peered":"2023-10-06T04:49:20.325259+0000","last_clean":"2023-10-06T04:49:20.325259+0000","last_became_active":"2023-10-06T04:49:19.331287+0000","last_became_peered":"2023-10-06T04:49:19.331287+0000","last_unstale":"2023-10-06T04:49:20.325259+0000","last_undegraded":"2023-10-06T04:49:20.325259+0000","last_fullsized":"2023-10-06T04:49:20.325259+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:07:26.288607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341412+0000","last_change":"2023-10-06T04:49:20.334336+0000","last_active":"2023-10-06T04:49:20.341412+0000","last_peered":"2023-10-06T04:49:20.341412+0000","last_clean":"2023-10-06T04:49:20.341412+0000","last_became_active":"2023-10-06T04:49:20.334097+0000","last_became_peered":"2023-10-06T04:49:20.334097+0000","last_unstale":"2023-10-06T04:49:20.341412+0000","last_undegraded":"2023-10-06T04:49:20.341412+0000","last_fullsized":"2023-10-06T04:49:20.341412+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:35.183960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320080+0000","last_change":"2023-10-06T04:49:19.332817+0000","last_active":"2023-10-06T04:49:20.320080+0000","last_peered":"2023-10-06T04:49:20.320080+0000","last_clean":"2023-10-06T04:49:20.320080+0000","last_became_active":"2023-10-06T04:49:19.331466+0000","last_became_peered":"2023-10-06T04:49:19.331466+0000","last_unstale":"2023-10-06T04:49:20.320080+0000","last_undegraded":"2023-10-06T04:49:20.320080+0000","last_fullsized":"2023-10-06T04:49:20.320080+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:14:01.585470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332830+0000","last_change":"2023-10-06T04:49:20.327613+0000","last_active":"2023-10-06T04:49:20.332830+0000","last_peered":"2023-10-06T04:49:20.332830+0000","last_clean":"2023-10-06T04:49:20.332830+0000","last_became_active":"2023-10-06T04:49:20.327331+0000","last_became_peered":"2023-10-06T04:49:20.327331+0000","last_unstale":"2023-10-06T04:49:20.332830+0000","last_undegraded":"2023-10-06T04:49:20.332830+0000","last_fullsized":"2023-10-06T04:49:20.332830+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:21.888629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339559+0000","last_change":"2023-10-06T04:49:19.327134+0000","last_active":"2023-10-06T04:49:20.339559+0000","last_peered":"2023-10-06T04:49:20.339559+0000","last_clean":"2023-10-06T04:49:20.339559+0000","last_became_active":"2023-10-06T04:49:19.326884+0000","last_became_peered":"2023-10-06T04:49:19.326884+0000","last_unstale":"2023-10-06T04:49:20.339559+0000","last_undegraded":"2023-10-06T04:49:20.339559+0000","last_fullsized":"2023-10-06T04:49:20.339559+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:38:40.308269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347026+0000","last_change":"2023-10-06T04:49:20.343182+0000","last_active":"2023-10-06T04:49:20.347026+0000","last_peered":"2023-10-06T04:49:20.347026+0000","last_clean":"2023-10-06T04:49:20.347026+0000","last_became_active":"2023-10-06T04:49:20.342662+0000","last_became_peered":"2023-10-06T04:49:20.342662+0000","last_unstale":"2023-10-06T04:49:20.347026+0000","last_undegraded":"2023-10-06T04:49:20.347026+0000","last_fullsized":"2023-10-06T04:49:20.347026+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:54.488863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331074+0000","last_change":"2023-10-06T04:49:19.336399+0000","last_active":"2023-10-06T04:49:20.331074+0000","last_peered":"2023-10-06T04:49:20.331074+0000","last_clean":"2023-10-06T04:49:20.331074+0000","last_became_active":"2023-10-06T04:49:19.336144+0000","last_became_peered":"2023-10-06T04:49:19.336144+0000","last_unstale":"2023-10-06T04:49:20.331074+0000","last_undegraded":"2023-10-06T04:49:20.331074+0000","last_fullsized":"2023-10-06T04:49:20.331074+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:54:48.611176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.333465+0000","last_change":"2023-10-06T04:49:20.325062+0000","last_active":"2023-10-06T04:49:20.333465+0000","last_peered":"2023-10-06T04:49:20.333465+0000","last_clean":"2023-10-06T04:49:20.333465+0000","last_became_active":"2023-10-06T04:49:20.324091+0000","last_became_peered":"2023-10-06T04:49:20.324091+0000","last_unstale":"2023-10-06T04:49:20.333465+0000","last_undegraded":"2023-10-06T04:49:20.333465+0000","last_fullsized":"2023-10-06T04:49:20.333465+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:28.109738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321039+0000","last_change":"2023-10-06T04:49:19.327314+0000","last_active":"2023-10-06T04:49:20.321039+0000","last_peered":"2023-10-06T04:49:20.321039+0000","last_clean":"2023-10-06T04:49:20.321039+0000","last_became_active":"2023-10-06T04:49:19.326851+0000","last_became_peered":"2023-10-06T04:49:19.326851+0000","last_unstale":"2023-10-06T04:49:20.321039+0000","last_undegraded":"2023-10-06T04:49:20.321039+0000","last_fullsized":"2023-10-06T04:49:20.321039+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:04:26.547655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":156,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321556+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T04:49:20.321556+0000","last_peered":"2023-10-06T04:49:20.321556+0000","last_clean":"2023-10-06T04:49:20.321556+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T04:49:20.321556+0000","last_undegraded":"2023-10-06T04:49:20.321556+0000","last_fullsized":"2023-10-06T04:49:20.321556+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.332841+0000","last_change":"2023-10-06T04:49:20.327658+0000","last_active":"2023-10-06T04:49:20.332841+0000","last_peered":"2023-10-06T04:49:20.332841+0000","last_clean":"2023-10-06T04:49:20.332841+0000","last_became_active":"2023-10-06T04:49:20.327377+0000","last_became_peered":"2023-10-06T04:49:20.327377+0000","last_unstale":"2023-10-06T04:49:20.332841+0000","last_undegraded":"2023-10-06T04:49:20.332841+0000","last_fullsized":"2023-10-06T04:49:20.332841+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:49:29.783670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"172'8","reported_seq":18,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.351338+0000","last_change":"2023-10-06T04:49:19.329982+0000","last_active":"2023-10-06T04:49:20.351338+0000","last_peered":"2023-10-06T04:49:20.351338+0000","last_clean":"2023-10-06T04:49:20.351338+0000","last_became_active":"2023-10-06T04:49:19.329677+0000","last_became_peered":"2023-10-06T04:49:19.329677+0000","last_unstale":"2023-10-06T04:49:20.351338+0000","last_undegraded":"2023-10-06T04:49:20.351338+0000","last_fullsized":"2023-10-06T04:49:20.351338+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":143,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.316165+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T04:49:20.316165+0000","last_peered":"2023-10-06T04:49:20.316165+0000","last_clean":"2023-10-06T04:49:20.316165+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T04:49:20.316165+0000","last_undegraded":"2023-10-06T04:49:20.316165+0000","last_fullsized":"2023-10-06T04:49:20.316165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.343279+0000","last_change":"2023-10-06T04:49:20.330255+0000","last_active":"2023-10-06T04:49:20.343279+0000","last_peered":"2023-10-06T04:49:20.343279+0000","last_clean":"2023-10-06T04:49:20.343279+0000","last_became_active":"2023-10-06T04:49:20.329995+0000","last_became_peered":"2023-10-06T04:49:20.329995+0000","last_unstale":"2023-10-06T04:49:20.343279+0000","last_undegraded":"2023-10-06T04:49:20.343279+0000","last_fullsized":"2023-10-06T04:49:20.343279+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:16.754707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.330157+0000","last_change":"2023-10-06T04:49:19.330514+0000","last_active":"2023-10-06T04:49:20.330157+0000","last_peered":"2023-10-06T04:49:20.330157+0000","last_clean":"2023-10-06T04:49:20.330157+0000","last_became_active":"2023-10-06T04:49:19.330226+0000","last_became_peered":"2023-10-06T04:49:19.330226+0000","last_unstale":"2023-10-06T04:49:20.330157+0000","last_undegraded":"2023-10-06T04:49:20.330157+0000","last_fullsized":"2023-10-06T04:49:20.330157+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:55.142830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":144,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.319996+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T04:49:20.319996+0000","last_peered":"2023-10-06T04:49:20.319996+0000","last_clean":"2023-10-06T04:49:20.319996+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T04:49:20.319996+0000","last_undegraded":"2023-10-06T04:49:20.319996+0000","last_fullsized":"2023-10-06T04:49:20.319996+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"163'293","reported_seq":919,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331027+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T04:49:20.331027+0000","last_peered":"2023-10-06T04:49:20.331027+0000","last_clean":"2023-10-06T04:49:20.331027+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T04:49:20.331027+0000","last_undegraded":"2023-10-06T04:49:20.331027+0000","last_fullsized":"2023-10-06T04:49:20.331027+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.336779+0000","last_change":"2023-10-06T04:49:20.335358+0000","last_active":"2023-10-06T04:49:20.336779+0000","last_peered":"2023-10-06T04:49:20.336779+0000","last_clean":"2023-10-06T04:49:20.336779+0000","last_became_active":"2023-10-06T04:49:20.335073+0000","last_became_peered":"2023-10-06T04:49:20.335073+0000","last_unstale":"2023-10-06T04:49:20.336779+0000","last_undegraded":"2023-10-06T04:49:20.336779+0000","last_fullsized":"2023-10-06T04:49:20.336779+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:18:05.280129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.330408+0000","last_change":"2023-10-06T04:49:19.331617+0000","last_active":"2023-10-06T04:49:20.330408+0000","last_peered":"2023-10-06T04:49:20.330408+0000","last_clean":"2023-10-06T04:49:20.330408+0000","last_became_active":"2023-10-06T04:49:19.329690+0000","last_became_peered":"2023-10-06T04:49:19.329690+0000","last_unstale":"2023-10-06T04:49:20.330408+0000","last_undegraded":"2023-10-06T04:49:20.330408+0000","last_fullsized":"2023-10-06T04:49:20.330408+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:09:04.983797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":146,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.325305+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T04:49:20.325305+0000","last_peered":"2023-10-06T04:49:20.325305+0000","last_clean":"2023-10-06T04:49:20.325305+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T04:49:20.325305+0000","last_undegraded":"2023-10-06T04:49:20.325305+0000","last_fullsized":"2023-10-06T04:49:20.325305+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.339557+0000","last_change":"2023-10-06T04:49:20.331740+0000","last_active":"2023-10-06T04:49:20.339557+0000","last_peered":"2023-10-06T04:49:20.339557+0000","last_clean":"2023-10-06T04:49:20.339557+0000","last_became_active":"2023-10-06T04:49:20.331463+0000","last_became_peered":"2023-10-06T04:49:20.331463+0000","last_unstale":"2023-10-06T04:49:20.339557+0000","last_undegraded":"2023-10-06T04:49:20.339557+0000","last_fullsized":"2023-10-06T04:49:20.339557+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:20:01.313087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"172'6","reported_seq":16,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.356034+0000","last_change":"2023-10-06T04:49:19.331731+0000","last_active":"2023-10-06T04:49:20.356034+0000","last_peered":"2023-10-06T04:49:20.356034+0000","last_clean":"2023-10-06T04:49:20.356034+0000","last_became_active":"2023-10-06T04:49:19.331458+0000","last_became_peered":"2023-10-06T04:49:19.331458+0000","last_unstale":"2023-10-06T04:49:20.356034+0000","last_undegraded":"2023-10-06T04:49:20.356034+0000","last_fullsized":"2023-10-06T04:49:20.356034+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:55.142830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":146,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.331105+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T04:49:20.331105+0000","last_peered":"2023-10-06T04:49:20.331105+0000","last_clean":"2023-10-06T04:49:20.331105+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T04:49:20.331105+0000","last_undegraded":"2023-10-06T04:49:20.331105+0000","last_fullsized":"2023-10-06T04:49:20.331105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347917+0000","last_change":"2023-10-06T04:49:20.345754+0000","last_active":"2023-10-06T04:49:20.347917+0000","last_peered":"2023-10-06T04:49:20.347917+0000","last_clean":"2023-10-06T04:49:20.347917+0000","last_became_active":"2023-10-06T04:49:20.345446+0000","last_became_peered":"2023-10-06T04:49:20.345446+0000","last_unstale":"2023-10-06T04:49:20.347917+0000","last_undegraded":"2023-10-06T04:49:20.347917+0000","last_fullsized":"2023-10-06T04:49:20.347917+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:34:00.217906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"172'3","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.342594+0000","last_change":"2023-10-06T04:49:19.327118+0000","last_active":"2023-10-06T04:49:20.342594+0000","last_peered":"2023-10-06T04:49:20.342594+0000","last_clean":"2023-10-06T04:49:20.342594+0000","last_became_active":"2023-10-06T04:49:19.326839+0000","last_became_peered":"2023-10-06T04:49:19.326839+0000","last_unstale":"2023-10-06T04:49:20.342594+0000","last_undegraded":"2023-10-06T04:49:20.342594+0000","last_fullsized":"2023-10-06T04:49:20.342594+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:10:17.629867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":139,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.335923+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T04:49:20.335923+0000","last_peered":"2023-10-06T04:49:20.335923+0000","last_clean":"2023-10-06T04:49:20.335923+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T04:49:20.335923+0000","last_undegraded":"2023-10-06T04:49:20.335923+0000","last_fullsized":"2023-10-06T04:49:20.335923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.341195+0000","last_change":"2023-10-06T04:49:20.338084+0000","last_active":"2023-10-06T04:49:20.341195+0000","last_peered":"2023-10-06T04:49:20.341195+0000","last_clean":"2023-10-06T04:49:20.341195+0000","last_became_active":"2023-10-06T04:49:20.337738+0000","last_became_peered":"2023-10-06T04:49:20.337738+0000","last_unstale":"2023-10-06T04:49:20.341195+0000","last_undegraded":"2023-10-06T04:49:20.341195+0000","last_fullsized":"2023-10-06T04:49:20.341195+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:38:28.172716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.321149+0000","last_change":"2023-10-06T04:49:19.332809+0000","last_active":"2023-10-06T04:49:20.321149+0000","last_peered":"2023-10-06T04:49:20.321149+0000","last_clean":"2023-10-06T04:49:20.321149+0000","last_became_active":"2023-10-06T04:49:19.331892+0000","last_became_peered":"2023-10-06T04:49:19.331892+0000","last_unstale":"2023-10-06T04:49:20.321149+0000","last_undegraded":"2023-10-06T04:49:20.321149+0000","last_fullsized":"2023-10-06T04:49:20.321149+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:24:52.469578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320842+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T04:49:20.320842+0000","last_peered":"2023-10-06T04:49:20.320842+0000","last_clean":"2023-10-06T04:49:20.320842+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T04:49:20.320842+0000","last_undegraded":"2023-10-06T04:49:20.320842+0000","last_fullsized":"2023-10-06T04:49:20.320842+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.347079+0000","last_change":"2023-10-06T04:49:20.336896+0000","last_active":"2023-10-06T04:49:20.347079+0000","last_peered":"2023-10-06T04:49:20.347079+0000","last_clean":"2023-10-06T04:49:20.347079+0000","last_became_active":"2023-10-06T04:49:20.336451+0000","last_became_peered":"2023-10-06T04:49:20.336451+0000","last_unstale":"2023-10-06T04:49:20.347079+0000","last_undegraded":"2023-10-06T04:49:20.347079+0000","last_fullsized":"2023-10-06T04:49:20.347079+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:19.299792+0000","last_clean_scrub_stamp":"2023-10-06T04:49:19.299792+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:09.120814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.344451+0000","last_change":"2023-10-06T04:49:19.321982+0000","last_active":"2023-10-06T04:49:20.344451+0000","last_peered":"2023-10-06T04:49:20.344451+0000","last_clean":"2023-10-06T04:49:20.344451+0000","last_became_active":"2023-10-06T04:49:19.321679+0000","last_became_peered":"2023-10-06T04:49:19.321679+0000","last_unstale":"2023-10-06T04:49:20.344451+0000","last_undegraded":"2023-10-06T04:49:20.344451+0000","last_fullsized":"2023-10-06T04:49:20.344451+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:49:18.291949+0000","last_clean_scrub_stamp":"2023-10-06T04:49:18.291949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:07:15.052260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":166,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-10-06T04:49:20.320746+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T04:49:20.320746+0000","last_peered":"2023-10-06T04:49:20.320746+0000","last_clean":"2023-10-06T04:49:20.320746+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T04:49:20.320746+0000","last_undegraded":"2023-10-06T04:49:20.320746+0000","last_fullsized":"2023-10-06T04:49:20.320746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575753,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299976,"kb_used_data":2688,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071864,"statfs":{"total":96636764160,"available":96329588736,"internally_reserved":0,"allocated":2752512,"data_stored":1720883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69799999999999995,"15min":0.73899999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.89500000000000002,"5min":0.97199999999999998,"15min":1.4930000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.74099999999999999},"min":{"1min":0.48599999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90900000000000003,"5min":0.995,"15min":1.417},"last":0.84899999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.755,"15min":0.79500000000000004},"min":{"1min":0.53500000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93100000000000005,"5min":1.01,"15min":1.5309999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77600000000000002,"15min":0.79300000000000004},"min":{"1min":0.58499999999999996,"5min":0.58399999999999996,"15min":0.58299999999999996},"max":{"1min":0.93600000000000005,"5min":1.0289999999999999,"15min":1.097},"last":0.76500000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70899999999999996,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.46400000000000002},"max":{"1min":0.92300000000000004,"5min":0.97999999999999998,"15min":1.5429999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.751,"15min":0.77100000000000002},"min":{"1min":0.60199999999999998,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0189999999999999},"last":0.745}]},{"osd":3,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.75700000000000001,"15min":0.77500000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.91000000000000003,"5min":1.03,"15min":1.5860000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.73299999999999998,"15min":0.76400000000000001},"min":{"1min":0.64200000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.90300000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.63400000000000001,"15min":0.61899999999999999},"min":{"1min":0.50800000000000001,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.0640000000000001},"last":0.505},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.67300000000000004,"15min":0.64900000000000002},"min":{"1min":0.47399999999999998,"5min":0.439,"15min":0.42799999999999999},"max":{"1min":1.1339999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.65300000000000002,"15min":0.65600000000000003},"min":{"1min":0.53800000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.55200000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.70099999999999996}]},{"osd":6,"last update":"Fri Oct 6 04:48:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.64200000000000002,"5min":0.501,"15min":0.49199999999999999},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.50800000000000001,"5min":0.48999999999999999,"15min":0.48699999999999999},"max":{"1min":0.96299999999999997,"5min":1.1020000000000001,"15min":1.2090000000000001},"last":0.56000000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575754,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299400,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072440,"statfs":{"total":96636764160,"available":96330178560,"internally_reserved":0,"allocated":2162688,"data_stored":1131541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71399999999999997,"15min":0.71499999999999997},"min":{"1min":0.46000000000000002,"5min":0.40300000000000002,"15min":0.375},"max":{"1min":0.96599999999999997,"5min":1.038,"15min":1.038},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76200000000000001,"15min":0.752},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.94699999999999995,"5min":0.999,"15min":1.228},"last":0.78300000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.747,"15min":0.75900000000000001},"min":{"1min":0.5,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.93799999999999994,"5min":1.046,"15min":1.3660000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.748,"15min":0.751},"min":{"1min":0.495,"5min":0.47899999999999998,"15min":0.438},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.3560000000000001},"last":0.755}]},{"osd":2,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.75600000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.47999999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.272},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76900000000000002,"15min":0.77000000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.98799999999999999,"5min":1.069,"15min":1.089},"last":0.90200000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.77600000000000002,"15min":0.78600000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.94399999999999995,"5min":1.042,"15min":1.1060000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.52000000000000002},"max":{"1min":1.0009999999999999,"5min":1.0409999999999999,"15min":1.3120000000000001},"last":0.86399999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.38300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83499999999999996,"5min":0.88500000000000001,"15min":0.92600000000000005},"last":0.503},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.627},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.38600000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":1.155},"last":0.56499999999999995}]},{"osd":5,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70099999999999996,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.442,"15min":0.432},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.45300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.93799999999999994,"5min":1.056,"15min":1.056},"last":0.627}]},{"osd":7,"last update":"Fri Oct 6 04:48:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.44500000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95399999999999996,"5min":1.0580000000000001,"15min":1.3500000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.71899999999999997,"15min":0.72799999999999998},"min":{"1min":0.45800000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.98599999999999999,"5min":1.024,"15min":1.0389999999999999},"last":0.66500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575753,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312268,"kb_used_data":2692,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059572,"statfs":{"total":96636764160,"available":96317001728,"internally_reserved":0,"allocated":2756608,"data_stored":1721909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.51800000000000002,"15min":0.51400000000000001},"min":{"1min":0.373,"5min":0.307,"15min":0.307},"max":{"1min":0.79600000000000004,"5min":0.89000000000000001,"15min":0.93100000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42899999999999999},"max":{"1min":0.90000000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.627}]},{"osd":2,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48699999999999999,"15min":0.49099999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63800000000000001,"5min":0.67700000000000005,"15min":0.73299999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.38200000000000001},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.97799999999999998},"last":0.59099999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.44,"5min":0.44,"15min":0.437},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.94699999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.56000000000000005},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.373},"max":{"1min":0.748,"5min":0.85799999999999998,"15min":0.88900000000000001},"last":0.70799999999999996}]},{"osd":4,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79200000000000004,"15min":0.79500000000000004},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.46899999999999997},"max":{"1min":1.159,"5min":1.1970000000000001,"15min":1.407},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82599999999999996,"15min":0.83099999999999996},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.57499999999999996},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.754}]},{"osd":5,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.83699999999999997,"15min":0.82699999999999996},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.55700000000000005},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81699999999999995,"15min":0.83899999999999997},"min":{"1min":0.66300000000000003,"5min":0.59799999999999998,"15min":0.53200000000000003},"max":{"1min":1.097,"5min":1.1060000000000001,"15min":1.724},"last":0.87}]},{"osd":6,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.64700000000000002,"5min":0.626,"15min":0.56200000000000006},"max":{"1min":0.97899999999999998,"5min":1.2150000000000001,"15min":1.427},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.84599999999999997,"15min":0.85099999999999998},"min":{"1min":0.59199999999999997,"5min":0.56100000000000005,"15min":0.55800000000000005},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.73099999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:48:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.877,"15min":0.878},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1140000000000001,"5min":1.125,"15min":1.702},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85799999999999998,"15min":0.872},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.0269999999999999,"5min":1.1659999999999999,"15min":1.5920000000000001},"last":0.82399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575755,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299416,"kb_used_data":2128,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072424,"statfs":{"total":96636764160,"available":96330162176,"internally_reserved":0,"allocated":2179072,"data_stored":1132707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.378,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76600000000000001,"5min":0.81699999999999995,"15min":1.6459999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.36299999999999999,"5min":0.34200000000000003,"15min":0.32900000000000001},"max":{"1min":0.68300000000000005,"5min":0.84099999999999997,"15min":1.4510000000000001},"last":0.36099999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55400000000000005,"15min":0.57099999999999995},"min":{"1min":0.434,"5min":0.308,"15min":0.308},"max":{"1min":0.74099999999999999,"5min":0.86899999999999999,"15min":1.7},"last":0.443},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.373},"max":{"1min":0.81699999999999995,"5min":0.82099999999999995,"15min":1.6240000000000001},"last":0.436}]},{"osd":3,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.55100000000000005,"15min":0.54200000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":1.2529999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.64400000000000002},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91300000000000003,"5min":0.99099999999999999,"15min":1.7370000000000001},"last":0.46800000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.86599999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.60399999999999998},"max":{"1min":1.0940000000000001,"5min":1.2769999999999999,"15min":2.3700000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.85099999999999998,"15min":0.85999999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.52900000000000003},"max":{"1min":1.111,"5min":1.3069999999999999,"15min":2.1280000000000001},"last":1.0089999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.84099999999999997,"15min":0.85599999999999998},"min":{"1min":0.626,"5min":0.60999999999999999,"15min":0.57599999999999996},"max":{"1min":1.048,"5min":1.294,"15min":2.1739999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.84099999999999997,"15min":0.84799999999999998},"min":{"1min":0.66000000000000003,"5min":0.63300000000000001,"15min":0.57799999999999996},"max":{"1min":1.087,"5min":1.214,"15min":2.2429999999999999},"last":0.89500000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60599999999999998,"5min":0.59299999999999997,"15min":0.53000000000000003},"max":{"1min":1.1160000000000001,"5min":1.204,"15min":1.605},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89600000000000002,"15min":0.88700000000000001},"min":{"1min":0.72899999999999998,"5min":0.65800000000000003,"15min":0.58199999999999996},"max":{"1min":1.181,"5min":1.226,"15min":2.3519999999999999},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:48:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.86699999999999999,"15min":0.878},"min":{"1min":0.63200000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.091,"5min":1.319,"15min":2.1019999999999999},"last":0.745},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.86299999999999999,"15min":0.86799999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0580000000000001,"5min":1.256,"15min":1.9259999999999999},"last":1.0589999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575754,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311688,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060152,"statfs":{"total":96636764160,"available":96317595648,"internally_reserved":0,"allocated":2162688,"data_stored":1130605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51600000000000001,"15min":0.501},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.311},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.34200000000000003},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.93999999999999995},"last":0.34999999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51000000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.32000000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.56599999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60299999999999998,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.38300000000000001},"max":{"1min":0.79600000000000004,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.371},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.502}]},{"osd":4,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.86499999999999999,"15min":0.86099999999999999},"min":{"1min":0.628,"5min":0.60199999999999998,"15min":0.51700000000000002},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.86399999999999999,"15min":0.88100000000000001},"min":{"1min":0.621,"5min":0.58999999999999997,"15min":0.58599999999999997},"max":{"1min":1.228,"5min":1.228,"15min":1.381},"last":0.77800000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.63,"5min":0.63,"15min":0.59199999999999997},"max":{"1min":1.1719999999999999,"5min":1.2210000000000001,"15min":1.4379999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88300000000000001,"15min":0.89200000000000002},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.58899999999999997},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.288},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.86499999999999999,"15min":0.86699999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.151,"5min":1.167,"15min":1.2370000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.746}]},{"osd":7,"last update":"Fri Oct 6 04:48:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.86099999999999999,"15min":0.874},"min":{"1min":0.58599999999999997,"5min":0.57799999999999996,"15min":0.57499999999999996},"max":{"1min":1.075,"5min":1.3320000000000001,"15min":1.482},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.88500000000000001,"15min":0.89600000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.135,"5min":1.2869999999999999,"15min":1.405},"last":0.76600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575754,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311692,"kb_used_data":2116,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060148,"statfs":{"total":96636764160,"available":96317591552,"internally_reserved":0,"allocated":2166784,"data_stored":1132059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47699999999999998,"15min":0.49399999999999999},"min":{"1min":0.32100000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67900000000000005,"5min":0.73899999999999999,"15min":0.86399999999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.51900000000000002,"15min":0.51400000000000001},"min":{"1min":0.42799999999999999,"5min":0.35799999999999998,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.83599999999999997},"last":0.501},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59699999999999998,"15min":0.58499999999999996},"min":{"1min":0.44900000000000001,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.73999999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.45000000000000001,"5min":0.41799999999999998,"15min":0.40899999999999997},"max":{"1min":0.69699999999999995,"5min":0.79200000000000004,"15min":1.036},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.64500000000000002,"15min":0.64000000000000001},"min":{"1min":0.54000000000000004,"5min":0.47499999999999998,"15min":0.43099999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51400000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83299999999999996,"15min":0.83799999999999997},"min":{"1min":0.64300000000000002,"5min":0.56000000000000005,"15min":0.47599999999999998},"max":{"1min":1.101,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.86499999999999999,"15min":0.879},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.495},"max":{"1min":1.0760000000000001,"5min":1.387,"15min":1.399},"last":1.034}]},{"osd":5,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.82599999999999996,"15min":0.82899999999999996},"min":{"1min":0.64300000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.0049999999999999,"5min":1.2509999999999999,"15min":1.331},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.85799999999999998,"15min":0.85099999999999998},"min":{"1min":0.63600000000000001,"5min":0.58099999999999996,"15min":0.56599999999999995},"max":{"1min":1.0720000000000001,"5min":1.2130000000000001,"15min":1.2330000000000001},"last":1.0549999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.86499999999999999,"15min":0.85999999999999999},"min":{"1min":0.65100000000000002,"5min":0.53500000000000003,"15min":0.52400000000000002},"max":{"1min":1.175,"5min":1.2090000000000001,"15min":1.2490000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.89900000000000002,"15min":0.89400000000000002},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1579999999999999,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:48:26 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88600000000000001,"15min":0.88400000000000001},"min":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.58999999999999997},"max":{"1min":1.127,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85399999999999998,"15min":0.85799999999999998},"min":{"1min":0.60499999999999998,"5min":0.58699999999999997,"15min":0.54900000000000004},"max":{"1min":1.069,"5min":1.276,"15min":1.3029999999999999},"last":1.097}]}]},{"osd":4,"up_from":13,"seq":55834575754,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311696,"kb_used_data":2120,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060144,"statfs":{"total":96636764160,"available":96317587456,"internally_reserved":0,"allocated":2170880,"data_stored":1133007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.71699999999999997,"15min":0.74299999999999999},"min":{"1min":0.47899999999999998,"5min":0.36699999999999999,"15min":0.35599999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.72399999999999998,"15min":0.73499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.41199999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.68799999999999994}]},{"osd":1,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.76000000000000001,"15min":0.77800000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.63400000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.745,"15min":0.75900000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.76600000000000001,"15min":0.77100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.91200000000000003,"5min":1.01,"15min":1.069},"last":0.81100000000000005}]},{"osd":3,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.74199999999999999,"15min":0.749},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78200000000000003,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.996,"5min":1.002,"15min":1.002},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.63100000000000001,"15min":0.623},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.81699999999999995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.65000000000000002,"15min":0.63500000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.879}]},{"osd":6,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.69099999999999995,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.46000000000000002,"15min":0.42099999999999999},"max":{"1min":0.996,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.71499999999999997,"15min":0.70199999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0629999999999999,"5min":1.105,"15min":1.105},"last":0.86199999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:48:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.71399999999999997,"15min":0.68799999999999994},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97999999999999998,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.75600000000000001,"15min":0.745},"min":{"1min":0.48499999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89700000000000002,"5min":1.091,"15min":1.0940000000000001},"last":0.89600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575754,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307584,"kb_used_data":2104,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064256,"statfs":{"total":96636764160,"available":96321798144,"internally_reserved":0,"allocated":2154496,"data_stored":1130493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82999999999999996,"5min":1.03,"15min":2.2829999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73599999999999999,"15min":0.751},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91600000000000004,"5min":1.095,"15min":1.3200000000000001},"last":0.80800000000000005}]},{"osd":1,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75800000000000001,"15min":0.751},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.29},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.74299999999999999,"15min":0.76900000000000002},"min":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.51400000000000001},"max":{"1min":0.871,"5min":0.98099999999999998,"15min":1.9139999999999999},"last":0.76400000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79000000000000004,"15min":0.80600000000000005},"min":{"1min":0.57999999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98199999999999998,"5min":1.0589999999999999,"15min":1.532},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79400000000000004,"15min":0.81599999999999995},"min":{"1min":0.58799999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95799999999999996,"5min":1.0169999999999999,"15min":1.121},"last":0.78100000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77000000000000002,"15min":0.77800000000000002},"min":{"1min":0.58899999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.60299999999999998,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.40500000000000003,"15min":0.39300000000000002},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":2.6989999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.64100000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.38},"max":{"1min":0.99399999999999999,"5min":1.028,"15min":1.694},"last":0.748}]},{"osd":6,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66300000000000003,"15min":0.66500000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.1719999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.96399999999999997,"5min":1.077,"15min":1.5880000000000001},"last":0.629}]},{"osd":7,"last update":"Fri Oct 6 04:48:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69299999999999995,"15min":0.70199999999999996},"min":{"1min":0.52500000000000002,"5min":0.47499999999999998,"15min":0.40699999999999997},"max":{"1min":1.03,"5min":1.0409999999999999,"15min":1.7390000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.51300000000000001,"5min":0.49099999999999999,"15min":0.48599999999999999},"max":{"1min":1.0429999999999999,"5min":1.0589999999999999,"15min":1.3380000000000001},"last":0.59399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:49:30.112 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T04:49:30.112 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T04:49:30.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.338+0000 7f03d219d700 1 Processor -- start 2023-10-06T04:49:30.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.338+0000 7f03d219d700 1 -- start start 2023-10-06T04:49:30.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.339+0000 7f03d219d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f03cc0ae9c0 0x7f03cc156f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:30.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.339+0000 7f03d219d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03cc1574a0 0x7f03cc0c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:30.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.339+0000 7f03d219d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03cc0c6990 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.340+0000 7f03d219d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03cc06d770 con 0x7f03cc0ae9c0 2023-10-06T04:49:30.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.340+0000 7f03d219d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc06da50 con 0x7f03cc1574a0 2023-10-06T04:49:30.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.340+0000 7f03d099a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f03cc0ae9c0 0x7f03cc156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:30.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.340+0000 7f03d119b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f03cc0ae5f0 0x7f03cc0cc930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46748/0 (socket says 172.21.15.55:46748) 2023-10-06T04:49:30.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.340+0000 7f03d119b700 1 -- 172.21.15.55:0/3745813157 learned_addr learned my addr 172.21.15.55:0/3745813157 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:30.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.340+0000 7f03d099a700 1 -- 172.21.15.55:0/3745813157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03cc1574a0 msgr2=0x7f03cc0c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:30.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.340+0000 7f03d099a700 1 --2- 172.21.15.55:0/3745813157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03cc1574a0 0x7f03cc0c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.340+0000 7f03d099a700 1 -- 172.21.15.55:0/3745813157 >> v1:172.21.15.55:6789/0 conn(0x7f03cc0ae5f0 legacy=0x7f03cc0cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:30.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.340+0000 7f03c3fff700 1 -- 172.21.15.55:0/3745813157 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821843648 0 0) 0x7f03cc0c6990 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.340+0000 7f03d099a700 1 -- 172.21.15.55:0/3745813157 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc06dd30 con 0x7f03cc0ae9c0 2023-10-06T04:49:30.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.341+0000 7f03d099a700 1 --2- 172.21.15.55:0/3745813157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03cc0ae9c0 0x7f03cc156f60 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0098e0 tx=0x7f03bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fb6d4f2f9dbe74 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:30.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.341+0000 7f03c3fff700 1 -- 172.21.15.55:0/3745813157 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc009e80 con 0x7f03cc0ae9c0 2023-10-06T04:49:30.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.341+0000 7f03c3fff700 1 -- 172.21.15.55:0/3745813157 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03bc00b860 con 0x7f03cc0ae9c0 2023-10-06T04:49:30.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.341+0000 7f03c3fff700 1 -- 172.21.15.55:0/3745813157 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc00bbe0 con 0x7f03cc0ae9c0 2023-10-06T04:49:30.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.341+0000 7f03d219d700 1 -- 172.21.15.55:0/3745813157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03cc0ae9c0 msgr2=0x7f03cc156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:30.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.341+0000 7f03d219d700 1 --2- 172.21.15.55:0/3745813157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03cc0ae9c0 0x7f03cc156f60 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f03bc0098e0 tx=0x7f03bc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.341+0000 7f03d219d700 1 -- 172.21.15.55:0/3745813157 shutdown_connections 2023-10-06T04:49:30.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.341+0000 7f03d219d700 1 --2- 172.21.15.55:0/3745813157 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03cc1574a0 0x7f03cc0c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.341+0000 7f03d219d700 1 --2- 172.21.15.55:0/3745813157 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03cc0ae9c0 0x7f03cc156f60 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.341+0000 7f03d219d700 1 -- 172.21.15.55:0/3745813157 >> 172.21.15.55:0/3745813157 conn(0x7f03cc068e20 msgr2=0x7f03cc0cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:30.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.341+0000 7f03d219d700 1 -- 172.21.15.55:0/3745813157 shutdown_connections 2023-10-06T04:49:30.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.341+0000 7f03d219d700 1 -- 172.21.15.55:0/3745813157 wait complete. 2023-10-06T04:49:30.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.343+0000 7f03d219d700 1 Processor -- start 2023-10-06T04:49:30.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.343+0000 7f03d219d700 1 -- start start 2023-10-06T04:49:30.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.344+0000 7f03d219d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03cc0ae5f0 0x7f03cc15c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:30.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.344+0000 7f03d219d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03cc0ae9c0 0x7f03cc0c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:30.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.344+0000 7f03d219d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03cc1574a0 0x7f03cc152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:30.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.344+0000 7f03d219d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0cace0 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.344+0000 7f03d219d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc172910 con 0x7f03cc1574a0 2023-10-06T04:49:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.344+0000 7f03d219d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0ca3b0 con 0x7f03cc0ae9c0 2023-10-06T04:49:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.344+0000 7f03d119b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03cc0ae5f0 0x7f03cc15c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.344+0000 7f03d119b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03cc0ae5f0 0x7f03cc15c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34538/0 (socket says 172.21.15.55:34538) 2023-10-06T04:49:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.344+0000 7f03d119b700 1 -- 172.21.15.55:0/141134861 learned_addr learned my addr 172.21.15.55:0/141134861 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.345+0000 7f03d119b700 1 -- 172.21.15.55:0/141134861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03cc0ae9c0 msgr2=0x7f03cc0c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.345+0000 7f03d119b700 1 --2- 172.21.15.55:0/141134861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03cc0ae9c0 0x7f03cc0c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.345+0000 7f03d199c700 1 --2- 172.21.15.55:0/141134861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03cc1574a0 0x7f03cc152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.345+0000 7f03d119b700 1 -- 172.21.15.55:0/141134861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03cc1574a0 msgr2=0x7f03cc152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.345+0000 7f03d119b700 1 --2- 172.21.15.55:0/141134861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03cc1574a0 0x7f03cc152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.345+0000 7f03d119b700 1 -- 172.21.15.55:0/141134861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc009420 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.345+0000 7f03d199c700 1 --2- 172.21.15.55:0/141134861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03cc1574a0 0x7f03cc152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:49:30.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.345+0000 7f03d119b700 1 --2- 172.21.15.55:0/141134861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03cc0ae5f0 0x7f03cc15c520 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f03c800c7e0 tx=0x7f03c8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:30.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.345+0000 7f03c17fa700 1 -- 172.21.15.55:0/141134861 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8005b30 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.345+0000 7f03c17fa700 1 -- 172.21.15.55:0/141134861 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c8005c90 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.345+0000 7f03c17fa700 1 -- 172.21.15.55:0/141134861 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8007300 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.345+0000 7f03d219d700 1 -- 172.21.15.55:0/141134861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03cc0ca690 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.345+0000 7f03d219d700 1 -- 172.21.15.55:0/141134861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03cc15ca60 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.346+0000 7f03d219d700 1 -- 172.21.15.55:0/141134861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03cc04c3c0 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.347+0000 7f03c17fa700 1 -- 172.21.15.55:0/141134861 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f03c801b400 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.348+0000 7f03c17fa700 1 --2- 172.21.15.55:0/141134861 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f03b0066f30 0x7f03b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:30.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.348+0000 7f03c17fa700 1 -- 172.21.15.55:0/141134861 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f03c8084350 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.349+0000 7f03d099a700 1 --2- 172.21.15.55:0/141134861 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f03b0066f30 0x7f03b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:30.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.349+0000 7f03d099a700 1 --2- 172.21.15.55:0/141134861 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f03b0066f30 0x7f03b00693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f03cc155f90 tx=0x7f03bc017000 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:30.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.352+0000 7f03c17fa700 1 -- 172.21.15.55:0/141134861 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03c8051360 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.639+0000 7f03d219d700 1 -- 172.21.15.55:0/141134861 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f03cc06e010 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.644+0000 7f03c17fa700 1 -- 172.21.15.55:0/141134861 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v40) v1 ==== 190+0+0 (secure 0 0 0) 0x7f03c8007730 con 0x7f03cc0ae5f0 2023-10-06T04:49:30.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.647+0000 7f03d219d700 1 -- 172.21.15.55:0/141134861 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f03b0066f30 msgr2=0x7f03b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:30.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.647+0000 7f03d219d700 1 --2- 172.21.15.55:0/141134861 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f03b0066f30 0x7f03b00693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f03cc155f90 tx=0x7f03bc017000 comp rx=0 tx=0).stop 2023-10-06T04:49:30.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.647+0000 7f03d219d700 1 -- 172.21.15.55:0/141134861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03cc0ae5f0 msgr2=0x7f03cc15c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:30.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.647+0000 7f03d219d700 1 --2- 172.21.15.55:0/141134861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03cc0ae5f0 0x7f03cc15c520 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f03c800c7e0 tx=0x7f03c8007df0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.647+0000 7f03d219d700 1 -- 172.21.15.55:0/141134861 shutdown_connections 2023-10-06T04:49:30.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.647+0000 7f03d219d700 1 --2- 172.21.15.55:0/141134861 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f03b0066f30 0x7f03b00693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.647+0000 7f03d219d700 1 --2- 172.21.15.55:0/141134861 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03cc1574a0 0x7f03cc152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.647+0000 7f03d219d700 1 --2- 172.21.15.55:0/141134861 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03cc0ae5f0 0x7f03cc15c520 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.647+0000 7f03d219d700 1 --2- 172.21.15.55:0/141134861 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03cc0ae9c0 0x7f03cc0c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.647+0000 7f03d219d700 1 -- 172.21.15.55:0/141134861 >> 172.21.15.55:0/141134861 conn(0x7f03cc068e20 msgr2=0x7f03cc1560f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:30.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.647+0000 7f03d219d700 1 -- 172.21.15.55:0/141134861 shutdown_connections 2023-10-06T04:49:30.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.647+0000 7f03d219d700 1 -- 172.21.15.55:0/141134861 wait complete. 2023-10-06T04:49:30.650 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2023-10-06T04:49:30.666 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:49:30.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.852+0000 7f4a40d9d700 1 Processor -- start 2023-10-06T04:49:30.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.852+0000 7f4a40d9d700 1 -- start start 2023-10-06T04:49:30.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.854+0000 7f4a40d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a3c156310 0x7f4a3c0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:30.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.854+0000 7f4a40d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3c0ca860 con 0x7f4a3c0ae710 2023-10-06T04:49:30.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.854+0000 7f4a40d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3c0cab40 con 0x7f4a3c1566e0 2023-10-06T04:49:30.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.854+0000 7f4a40d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0cae20 con 0x7f4a3c156310 2023-10-06T04:49:30.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.854+0000 7f4a3bfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4a3c0ae710 0x7f4a3c0c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46762/0 (socket says 172.21.15.55:46762) 2023-10-06T04:49:30.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.854+0000 7f4a3bfff700 1 -- 172.21.15.55:0/3088871084 learned_addr learned my addr 172.21.15.55:0/3088871084 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:30.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.854+0000 7f4a3b7fe700 1 --2- 172.21.15.55:0/3088871084 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a3c156310 0x7f4a3c0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:30.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.854+0000 7f4a3a7fc700 1 -- 172.21.15.55:0/3088871084 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689009859 0 0) 0x7f4a3c0ca860 con 0x7f4a3c0ae710 2023-10-06T04:49:30.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.855+0000 7f4a3a7fc700 1 -- 172.21.15.55:0/3088871084 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a24003570 con 0x7f4a3c0ae710 2023-10-06T04:49:30.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.855+0000 7f4a3a7fc700 1 -- 172.21.15.55:0/3088871084 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135274266 0 0) 0x7f4a3c0cab40 con 0x7f4a3c1566e0 2023-10-06T04:49:30.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.855+0000 7f4a3a7fc700 1 -- 172.21.15.55:0/3088871084 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a3c0ca860 con 0x7f4a3c1566e0 2023-10-06T04:49:30.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.855+0000 7f4a3a7fc700 1 -- 172.21.15.55:0/3088871084 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2993361672 0 0) 0x7f4a24003570 con 0x7f4a3c0ae710 2023-10-06T04:49:30.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.855+0000 7f4a3b7fe700 1 -- 172.21.15.55:0/3088871084 >> v1:172.21.15.163:6789/0 conn(0x7f4a3c1566e0 legacy=0x7f4a3c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:30.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.855+0000 7f4a3b7fe700 1 -- 172.21.15.55:0/3088871084 >> v1:172.21.15.55:6789/0 conn(0x7f4a3c0ae710 legacy=0x7f4a3c0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:30.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.855+0000 7f4a3b7fe700 1 -- 172.21.15.55:0/3088871084 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c0cb100 con 0x7f4a3c156310 2023-10-06T04:49:30.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.855+0000 7f4a3b7fe700 1 --2- 172.21.15.55:0/3088871084 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a3c156310 0x7f4a3c0c9dc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009910 tx=0x7f4a2c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a78e018c1b7f66 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:30.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.855+0000 7f4a3a7fc700 1 -- 172.21.15.55:0/3088871084 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a340017f0 con 0x7f4a3c156310 2023-10-06T04:49:30.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.856+0000 7f4a3a7fc700 1 -- 172.21.15.55:0/3088871084 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2c011420 con 0x7f4a3c156310 2023-10-06T04:49:30.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.856+0000 7f4a3a7fc700 1 -- 172.21.15.55:0/3088871084 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0117c0 con 0x7f4a3c156310 2023-10-06T04:49:30.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.856+0000 7f4a40d9d700 1 -- 172.21.15.55:0/3088871084 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a3c156310 msgr2=0x7f4a3c0c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:30.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.856+0000 7f4a40d9d700 1 --2- 172.21.15.55:0/3088871084 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a3c156310 0x7f4a3c0c9dc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c009910 tx=0x7f4a2c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.856+0000 7f4a40d9d700 1 -- 172.21.15.55:0/3088871084 shutdown_connections 2023-10-06T04:49:30.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.856+0000 7f4a40d9d700 1 --2- 172.21.15.55:0/3088871084 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a3c156310 0x7f4a3c0c9dc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.856+0000 7f4a40d9d700 1 -- 172.21.15.55:0/3088871084 >> 172.21.15.55:0/3088871084 conn(0x7f4a3c0cc1f0 msgr2=0x7f4a3c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:30.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.856+0000 7f4a40d9d700 1 -- 172.21.15.55:0/3088871084 shutdown_connections 2023-10-06T04:49:30.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.856+0000 7f4a40d9d700 1 -- 172.21.15.55:0/3088871084 wait complete. 2023-10-06T04:49:30.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.858+0000 7f4a40d9d700 1 Processor -- start 2023-10-06T04:49:30.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.858+0000 7f4a40d9d700 1 -- start start 2023-10-06T04:49:30.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a40d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a3c0ae710 0x7f4a3c1547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:30.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a40d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a3c156310 0x7f4a3c168fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:30.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a40d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a3c1566e0 0x7f4a3c0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:30.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a40d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0c6840 con 0x7f4a3c1566e0 2023-10-06T04:49:30.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a40d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c0c5ca0 con 0x7f4a3c156310 2023-10-06T04:49:30.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a40d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3c154d20 con 0x7f4a3c0ae710 2023-10-06T04:49:30.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a3bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a3c1566e0 0x7f4a3c0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:30.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a3bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a3c1566e0 0x7f4a3c0a5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34544/0 (socket says 172.21.15.55:34544) 2023-10-06T04:49:30.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a3bfff700 1 -- 172.21.15.55:0/2177427198 learned_addr learned my addr 172.21.15.55:0/2177427198 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:30.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a3b7fe700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a3c0ae710 0x7f4a3c1547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:30.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a3affd700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a3c156310 0x7f4a3c168fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:30.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a3bfff700 1 -- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a3c0ae710 msgr2=0x7f4a3c1547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:30.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a3bfff700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a3c0ae710 0x7f4a3c1547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a3bfff700 1 -- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a3c156310 msgr2=0x7f4a3c168fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:30.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a3bfff700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a3c156310 0x7f4a3c168fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:30.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a3bfff700 1 -- 172.21.15.55:0/2177427198 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2c009420 con 0x7f4a3c1566e0 2023-10-06T04:49:30.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.859+0000 7f4a3affd700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a3c156310 0x7f4a3c168fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:49:30.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.860+0000 7f4a3bfff700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a3c1566e0 0x7f4a3c0a5aa0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400cc70 tx=0x7f4a34007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:30.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.860+0000 7f4a237fe700 1 -- 172.21.15.55:0/2177427198 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3400ce80 con 0x7f4a3c1566e0 2023-10-06T04:49:30.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.860+0000 7f4a237fe700 1 -- 172.21.15.55:0/2177427198 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a34004dc0 con 0x7f4a3c1566e0 2023-10-06T04:49:30.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.860+0000 7f4a237fe700 1 -- 172.21.15.55:0/2177427198 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3400ab10 con 0x7f4a3c1566e0 2023-10-06T04:49:30.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.860+0000 7f4a40d9d700 1 -- 172.21.15.55:0/2177427198 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3c0c5f10 con 0x7f4a3c1566e0 2023-10-06T04:49:30.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.860+0000 7f4a40d9d700 1 -- 172.21.15.55:0/2177427198 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3c0c64d0 con 0x7f4a3c1566e0 2023-10-06T04:49:30.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.861+0000 7f4a40d9d700 1 -- 172.21.15.55:0/2177427198 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3c04c3c0 con 0x7f4a3c1566e0 2023-10-06T04:49:30.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.862+0000 7f4a237fe700 1 -- 172.21.15.55:0/2177427198 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a34007150 con 0x7f4a3c1566e0 2023-10-06T04:49:30.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.863+0000 7f4a237fe700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4a28066f30 0x7f4a280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:30.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.863+0000 7f4a237fe700 1 -- 172.21.15.55:0/2177427198 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f4a34083f80 con 0x7f4a3c1566e0 2023-10-06T04:49:30.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.864+0000 7f4a3b7fe700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4a28066f30 0x7f4a280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:30.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.864+0000 7f4a3b7fe700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4a28066f30 0x7f4a280693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0030c0 tx=0x7f4a2c016000 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:30.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:30.867+0000 7f4a237fe700 1 -- 172.21.15.55:0/2177427198 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a34050f80 con 0x7f4a3c1566e0 2023-10-06T04:49:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.113+0000 7f4a40d9d700 1 -- 172.21.15.55:0/2177427198 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a3c0cd0b0 con 0x7f4a3c1566e0 2023-10-06T04:49:31.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.113+0000 7f4a237fe700 1 -- 172.21.15.55:0/2177427198 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f4a34018090 con 0x7f4a3c1566e0 2023-10-06T04:49:31.115 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T04:49:31.115 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7682,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1565234675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1565234675},{"type":"v1","addr":"172.21.15.55:6834","nonce":1565234675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3042024952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3042024952},{"type":"v1","addr":"172.21.15.163:6835","nonce":3042024952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8737,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/304924610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":304924610},{"type":"v1","addr":"172.21.15.55:6835","nonce":304924610}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:49:19.807866+0000","modified":"2023-10-06T04:49:23.339948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8740":{"gid":8740,"name":"b","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6834/3095914721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3095914721},{"type":"v1","addr":"172.21.15.163:6834","nonce":3095914721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T04:49:31.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.116+0000 7f4a40d9d700 1 -- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4a28066f30 msgr2=0x7f4a280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.116+0000 7f4a40d9d700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4a28066f30 0x7f4a280693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0030c0 tx=0x7f4a2c016000 comp rx=0 tx=0).stop 2023-10-06T04:49:31.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.116+0000 7f4a40d9d700 1 -- 172.21.15.55:0/2177427198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a3c1566e0 msgr2=0x7f4a3c0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.116+0000 7f4a40d9d700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a3c1566e0 0x7f4a3c0a5aa0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400cc70 tx=0x7f4a34007d20 comp rx=0 tx=0).stop 2023-10-06T04:49:31.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.116+0000 7f4a40d9d700 1 -- 172.21.15.55:0/2177427198 shutdown_connections 2023-10-06T04:49:31.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.116+0000 7f4a40d9d700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4a28066f30 0x7f4a280693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.116+0000 7f4a40d9d700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4a3c1566e0 0x7f4a3c0a5aa0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.116+0000 7f4a40d9d700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4a3c0ae710 0x7f4a3c1547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.116+0000 7f4a40d9d700 1 --2- 172.21.15.55:0/2177427198 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4a3c156310 0x7f4a3c168fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.116+0000 7f4a40d9d700 1 -- 172.21.15.55:0/2177427198 >> 172.21.15.55:0/2177427198 conn(0x7f4a3c0cc1f0 msgr2=0x7f4a3c0b1790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.116+0000 7f4a40d9d700 1 -- 172.21.15.55:0/2177427198 shutdown_connections 2023-10-06T04:49:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.116+0000 7f4a40d9d700 1 -- 172.21.15.55:0/2177427198 wait complete. 2023-10-06T04:49:31.119 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 87 2023-10-06T04:49:31.538 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:49:19.807866+0000', 'modified': '2023-10-06T04:49:23.339948+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8740}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8740': {'gid': 8740, 'name': 'b', 'rank': 0, 'incarnation': 83, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.163:6834/3095914721', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.163:6832', 'nonce': 3095914721}, {'type': 'v1', 'addr': '172.21.15.163:6834', 'nonce': 3095914721}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T04:49:31.538 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T04:49:31.538 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-06T04:49:31.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.727+0000 7fe89679d700 1 Processor -- start 2023-10-06T04:49:31.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.727+0000 7fe89679d700 1 -- start start 2023-10-06T04:49:31.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.729+0000 7fe89679d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8901566e0 0x7fe8900ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.729+0000 7fe89679d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8900ca860 con 0x7fe890156310 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.729+0000 7fe89679d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8900cab40 con 0x7fe8900cb900 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.729+0000 7fe89679d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8900cae20 con 0x7fe8901566e0 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.729+0000 7fe894f9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8901566e0 0x7fe8900ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.729+0000 7fe894f9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8901566e0 0x7fe8900ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34550/0 (socket says 172.21.15.55:34550) 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.729+0000 7fe894f9a700 1 -- 172.21.15.55:0/1364691362 learned_addr learned my addr 172.21.15.55:0/1364691362 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.729+0000 7fe89579b700 1 --1- 172.21.15.55:0/1364691362 >> v1:172.21.15.55:6789/0 conn(0x7fe890156310 0x7fe8900c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46776/0 (socket says 172.21.15.55:46776) 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.729+0000 7fe894f9a700 1 -- 172.21.15.55:0/1364691362 >> v1:172.21.15.163:6789/0 conn(0x7fe8900cb900 legacy=0x7fe8900c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.729+0000 7fe894f9a700 1 -- 172.21.15.55:0/1364691362 >> v1:172.21.15.55:6789/0 conn(0x7fe890156310 legacy=0x7fe8900c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.730+0000 7fe894f9a700 1 -- 172.21.15.55:0/1364691362 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8900cb100 con 0x7fe8901566e0 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.730+0000 7fe887fff700 1 -- 172.21.15.55:0/1364691362 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558903 0 0) 0x7fe8900ca860 con 0x7fe890156310 2023-10-06T04:49:31.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.730+0000 7fe894f9a700 1 --2- 172.21.15.55:0/1364691362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8901566e0 0x7fe8900ae000 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe8800099b0 tx=0x7fe880009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6f5007ce22b023 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:31.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.730+0000 7fe887fff700 1 -- 172.21.15.55:0/1364691362 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88000b860 con 0x7fe8901566e0 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.730+0000 7fe887fff700 1 -- 172.21.15.55:0/1364691362 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88000b9c0 con 0x7fe8901566e0 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.730+0000 7fe887fff700 1 -- 172.21.15.55:0/1364691362 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88000bd60 con 0x7fe8901566e0 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.731+0000 7fe89679d700 1 -- 172.21.15.55:0/1364691362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8901566e0 msgr2=0x7fe8900ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.731+0000 7fe89679d700 1 --2- 172.21.15.55:0/1364691362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8901566e0 0x7fe8900ae000 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe8800099b0 tx=0x7fe880009250 comp rx=0 tx=0).stop 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.731+0000 7fe89679d700 1 -- 172.21.15.55:0/1364691362 shutdown_connections 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.731+0000 7fe89679d700 1 --2- 172.21.15.55:0/1364691362 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8901566e0 0x7fe8900ae000 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.731+0000 7fe89679d700 1 -- 172.21.15.55:0/1364691362 >> 172.21.15.55:0/1364691362 conn(0x7fe8900cc1f0 msgr2=0x7fe8900cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.731+0000 7fe89679d700 1 -- 172.21.15.55:0/1364691362 shutdown_connections 2023-10-06T04:49:31.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.731+0000 7fe89679d700 1 -- 172.21.15.55:0/1364691362 wait complete. 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.732+0000 7fe89679d700 1 Processor -- start 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.732+0000 7fe89679d700 1 -- start start 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.733+0000 7fe89679d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8900cb900 0x7fe8900aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.733+0000 7fe89679d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe890156310 0x7fe890169a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.733+0000 7fe89679d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8901566e0 0x7fe89014e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.733+0000 7fe89679d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89016aa70 con 0x7fe8900cb900 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.733+0000 7fe89679d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe890169fc0 con 0x7fe890156310 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.733+0000 7fe89679d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89014f000 con 0x7fe8901566e0 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.734+0000 7fe89579b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8900cb900 0x7fe8900aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.734+0000 7fe89579b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8900cb900 0x7fe8900aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34558/0 (socket says 172.21.15.55:34558) 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.734+0000 7fe89579b700 1 -- 172.21.15.55:0/30006275 learned_addr learned my addr 172.21.15.55:0/30006275 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.734+0000 7fe894f9a700 1 --2- 172.21.15.55:0/30006275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe890156310 0x7fe890169a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.734+0000 7fe895f9c700 1 --2- 172.21.15.55:0/30006275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8901566e0 0x7fe89014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.734+0000 7fe89579b700 1 -- 172.21.15.55:0/30006275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8901566e0 msgr2=0x7fe89014e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.734+0000 7fe89579b700 1 --2- 172.21.15.55:0/30006275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8901566e0 0x7fe89014e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.734+0000 7fe89579b700 1 -- 172.21.15.55:0/30006275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe890156310 msgr2=0x7fe890169a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.734+0000 7fe89579b700 1 --2- 172.21.15.55:0/30006275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe890156310 0x7fe890169a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.734+0000 7fe89579b700 1 -- 172.21.15.55:0/30006275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8800094f0 con 0x7fe8900cb900 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.734+0000 7fe89579b700 1 --2- 172.21.15.55:0/30006275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8900cb900 0x7fe8900aef90 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00c810 tx=0x7fe88c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.735+0000 7fe8857fa700 1 -- 172.21.15.55:0/30006275 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c005b40 con 0x7fe8900cb900 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.735+0000 7fe8857fa700 1 -- 172.21.15.55:0/30006275 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c005ca0 con 0x7fe8900cb900 2023-10-06T04:49:31.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.735+0000 7fe89679d700 1 -- 172.21.15.55:0/30006275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89014e5a0 con 0x7fe8900cb900 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.735+0000 7fe8857fa700 1 -- 172.21.15.55:0/30006275 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c007320 con 0x7fe8900cb900 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.735+0000 7fe89679d700 1 -- 172.21.15.55:0/30006275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89016a4b0 con 0x7fe8900cb900 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.737+0000 7fe8857fa700 1 -- 172.21.15.55:0/30006275 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe88c01b3b0 con 0x7fe8900cb900 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.738+0000 7fe8857fa700 1 --2- 172.21.15.55:0/30006275 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe874066f60 0x7fe874069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.738+0000 7fe8857fa700 1 -- 172.21.15.55:0/30006275 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fe88c083d60 con 0x7fe8900cb900 2023-10-06T04:49:31.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.738+0000 7fe894f9a700 1 --2- 172.21.15.55:0/30006275 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe874066f60 0x7fe874069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.739+0000 7fe894f9a700 1 --2- 172.21.15.55:0/30006275 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe874066f60 0x7fe874069410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe890156b50 tx=0x7fe880017000 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:31.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.739+0000 7fe8a61dab80 1 Processor -- start 2023-10-06T04:49:31.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.739+0000 7fe8a61dab80 1 -- start start 2023-10-06T04:49:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.740+0000 7fe8a61dab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d36de50 0x55662d348cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.740+0000 7fe8a61dab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34e2c0 0x55662d349200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.740+0000 7fe8a61dab80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 0x55662d349740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.740+0000 7fe8a61dab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662d2a3a70 con 0x55662d34e2c0 2023-10-06T04:49:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.740+0000 7fe8a61dab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662d2a1270 con 0x55662d36de50 2023-10-06T04:49:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.740+0000 7fe8a61dab80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55662d29d060 con 0x55662d34fa10 2023-10-06T04:49:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.741+0000 7fe894f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34e2c0 0x55662d349200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.741+0000 7fe894f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34e2c0 0x55662d349200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34562/0 (socket says 172.21.15.55:34562) 2023-10-06T04:49:31.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.741+0000 7fe894f9a700 1 -- 172.21.15.55:0/3922189934 learned_addr learned my addr 172.21.15.55:0/3922189934 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:31.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.741+0000 7fe89579b700 1 --2- 172.21.15.55:0/3922189934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d36de50 0x55662d348cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.741+0000 7fe895f9c700 1 --2- 172.21.15.55:0/3922189934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 0x55662d349740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.741+0000 7fe894f9a700 1 -- 172.21.15.55:0/3922189934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 msgr2=0x55662d349740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:31.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.741+0000 7fe894f9a700 1 --2- 172.21.15.55:0/3922189934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 0x55662d349740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.741+0000 7fe894f9a700 1 -- 172.21.15.55:0/3922189934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d36de50 msgr2=0x55662d348cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:31.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.741+0000 7fe894f9a700 1 --2- 172.21.15.55:0/3922189934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d36de50 0x55662d348cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.741+0000 7fe894f9a700 1 -- 172.21.15.55:0/3922189934 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55662d137270 con 0x55662d34e2c0 2023-10-06T04:49:31.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.741+0000 7fe894f9a700 1 --2- 172.21.15.55:0/3922189934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34e2c0 0x55662d349200 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fe89006fe40 tx=0x7fe88001d050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0f36f399884336 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:31.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.742+0000 7fe87a7fc700 1 -- 172.21.15.55:0/3922189934 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88002e070 con 0x55662d34e2c0 2023-10-06T04:49:31.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.742+0000 7fe87a7fc700 1 -- 172.21.15.55:0/3922189934 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe880021040 con 0x55662d34e2c0 2023-10-06T04:49:31.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.742+0000 7fe87a7fc700 1 -- 172.21.15.55:0/3922189934 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880011ce0 con 0x55662d34e2c0 2023-10-06T04:49:31.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.742+0000 7fe8a61dab80 1 -- 172.21.15.55:0/3922189934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34e2c0 msgr2=0x55662d349200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.742+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/3922189934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34e2c0 0x55662d349200 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fe89006fe40 tx=0x7fe88001d050 comp rx=0 tx=0).stop 2023-10-06T04:49:31.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.742+0000 7fe8a61dab80 1 -- 172.21.15.55:0/3922189934 shutdown_connections 2023-10-06T04:49:31.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.742+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/3922189934 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 0x55662d349740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.742+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/3922189934 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d36de50 0x55662d348cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.742+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/3922189934 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34e2c0 0x55662d349200 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.742+0000 7fe8a61dab80 1 -- 172.21.15.55:0/3922189934 >> 172.21.15.55:0/3922189934 conn(0x55662d34de80 msgr2=0x55662d34e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:31.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.742+0000 7fe8a61dab80 1 -- 172.21.15.55:0/3922189934 shutdown_connections 2023-10-06T04:49:31.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.742+0000 7fe8a61dab80 1 -- 172.21.15.55:0/3922189934 wait complete. 2023-10-06T04:49:31.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.743+0000 7fe8a61dab80 1 -- start start 2023-10-06T04:49:31.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.743+0000 7fe8a61dab80 1 Processor -- start 2023-10-06T04:49:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.744+0000 7fe8a61dab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d34e2c0 0x55662d348bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.744+0000 7fe8a61dab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34fa10 0x55662d3490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.744+0000 7fe8a61dab80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d36de50 0x55662d393940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.744+0000 7fe8a61dab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662d248890 con 0x55662d34fa10 2023-10-06T04:49:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.744+0000 7fe8a61dab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662d263ea0 con 0x55662d34e2c0 2023-10-06T04:49:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.744+0000 7fe8a61dab80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55662d11fe30 con 0x55662d36de50 2023-10-06T04:49:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.744+0000 7fe8a61dab80 10 client.0 _ll_register_callbacks cb 0x55662d310020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:49:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.744+0000 7fe894f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34fa10 0x55662d3490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.744+0000 7fe894f9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34fa10 0x55662d3490f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34570/0 (socket says 172.21.15.55:34570) 2023-10-06T04:49:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.744+0000 7fe894f9a700 1 -- 172.21.15.55:0/3223263124 learned_addr learned my addr 172.21.15.55:0/3223263124 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.745+0000 7fe895f9c700 1 --2- 172.21.15.55:0/3223263124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d36de50 0x55662d393940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.745+0000 7fe89579b700 1 --2- 172.21.15.55:0/3223263124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d34e2c0 0x55662d348bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.745+0000 7fe894f9a700 1 -- 172.21.15.55:0/3223263124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d36de50 msgr2=0x55662d393940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:31.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.745+0000 7fe894f9a700 1 --2- 172.21.15.55:0/3223263124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d36de50 0x55662d393940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.745+0000 7fe894f9a700 1 -- 172.21.15.55:0/3223263124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d34e2c0 msgr2=0x55662d348bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.745+0000 7fe894f9a700 1 --2- 172.21.15.55:0/3223263124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d34e2c0 0x55662d348bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.745+0000 7fe894f9a700 1 -- 172.21.15.55:0/3223263124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8900cb100 con 0x55662d34fa10 2023-10-06T04:49:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.745+0000 7fe894f9a700 1 --2- 172.21.15.55:0/3223263124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34fa10 0x55662d3490f0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe88000b3c0 tx=0x7fe88002b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa91023d824d93f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.745+0000 7fe86f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T04:49:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.745+0000 7fe86f7fe700 1 -- 172.21.15.55:0/3223263124 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88002e040 con 0x55662d34fa10 2023-10-06T04:49:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.745+0000 7fe86f7fe700 1 -- 172.21.15.55:0/3223263124 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fe88001a840 con 0x55662d34fa10 2023-10-06T04:49:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.745+0000 7fe8a61dab80 1 -- 172.21.15.55:0/3223263124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55662d341940 con 0x55662d34fa10 2023-10-06T04:49:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.746+0000 7fe86f7fe700 1 -- 172.21.15.55:0/3223263124 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe880011e70 con 0x55662d34fa10 2023-10-06T04:49:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.746+0000 7fe8a61dab80 10 client.9360 fetch_fsmap learned FSMap version 87 2023-10-06T04:49:31.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.746+0000 7fe8a61dab80 1 -- 172.21.15.55:0/3223263124 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x55662d0d9fe0 con 0x55662d34fa10 2023-10-06T04:49:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.746+0000 7fe86f7fe700 1 -- 172.21.15.55:0/3223263124 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55662d341940 con 0x55662d34fa10 2023-10-06T04:49:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.746+0000 7fe86f7fe700 1 -- 172.21.15.55:0/3223263124 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 87) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe88001ac10 con 0x55662d34fa10 2023-10-06T04:49:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.746+0000 7fe8a61dab80 10 client.9360 fetch_fsmap finished waiting for FSMap version 87 2023-10-06T04:49:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.746+0000 7fe8a61dab80 10 client.9360 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T04:49:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.746+0000 7fe8a61dab80 20 client.9360 populate_metadata read hostname 'smithi055' 2023-10-06T04:49:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.746+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/3223263124 >> [v2:172.21.15.163:6832/3095914721,v1:172.21.15.163:6834/3095914721] conn(0x55662d393e80 0x55662d394250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.746+0000 7fe8a61dab80 4 client.9360 mds_command: new command op to 8740 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T04:49:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.746+0000 7fe8a61dab80 1 -- 172.21.15.55:0/3223263124 --> [v2:172.21.15.163:6832/3095914721,v1:172.21.15.163:6834/3095914721] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55662d31e870 con 0x55662d393e80 2023-10-06T04:49:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.746+0000 7fe89579b700 1 --2- 172.21.15.55:0/3223263124 >> [v2:172.21.15.163:6832/3095914721,v1:172.21.15.163:6834/3095914721] conn(0x55662d393e80 0x55662d394250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.747+0000 7fe89579b700 1 --2- 172.21.15.55:0/3223263124 >> [v2:172.21.15.163:6832/3095914721,v1:172.21.15.163:6834/3095914721] conn(0x55662d393e80 0x55662d394250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=323520e33330616a server_cookie=c2aa71e9fd9303cf in_seq=0 out_seq=0 2023-10-06T04:49:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.747+0000 7fe86f7fe700 10 client.9360 ms_handle_connect on v2:172.21.15.163:6832/3095914721 2023-10-06T04:49:31.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.748+0000 7fe86f7fe700 1 -- 172.21.15.55:0/3223263124 <== mds.0 v2:172.21.15.163:6832/3095914721 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55662d31e870 con 0x55662d393e80 2023-10-06T04:49:31.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.748+0000 7fe86f7fe700 10 client.9360 handle_command_reply: tid=0 2023-10-06T04:49:31.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.748+0000 7fe8a61dab80 1 client.9360 shutdown 2023-10-06T04:49:31.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.748+0000 7fe8a61dab80 1 -- 172.21.15.55:0/3223263124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34fa10 msgr2=0x55662d3490f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.748+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/3223263124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34fa10 0x55662d3490f0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fe88000b3c0 tx=0x7fe88002b050 comp rx=0 tx=0).stop 2023-10-06T04:49:31.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.748+0000 7fe8a61dab80 1 -- 172.21.15.55:0/3223263124 shutdown_connections 2023-10-06T04:49:31.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.748+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/3223263124 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d36de50 0x55662d393940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.748+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/3223263124 >> [v2:172.21.15.163:6832/3095914721,v1:172.21.15.163:6834/3095914721] conn(0x55662d393e80 0x55662d394250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.748+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/3223263124 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d34e2c0 0x55662d348bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.748+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/3223263124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34fa10 0x55662d3490f0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.748+0000 7fe8a61dab80 1 -- 172.21.15.55:0/3223263124 >> 172.21.15.55:0/3223263124 conn(0x55662d34de80 msgr2=0x55662d34cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:31.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.748+0000 7fe86f7fe700 0 client.9360 ms_handle_reset on v2:172.21.15.163:6832/3095914721 2023-10-06T04:49:31.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.748+0000 7fe8a61dab80 1 -- 172.21.15.55:0/3223263124 shutdown_connections 2023-10-06T04:49:31.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.748+0000 7fe8a61dab80 1 -- 172.21.15.55:0/3223263124 wait complete. 2023-10-06T04:49:31.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.749+0000 7fe8a61dab80 20 client.9360 trim_cache size 0 max 16384 2023-10-06T04:49:31.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.766+0000 7fe8a61dab80 1 Processor -- start 2023-10-06T04:49:31.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.766+0000 7fe8a61dab80 1 -- start start 2023-10-06T04:49:31.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe8a61dab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34e2c0 0x55662d348ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe8a61dab80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 0x55662d349010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe8a61dab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d36de50 0x55662d34e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe8a61dab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662d36db40 con 0x55662d34e2c0 2023-10-06T04:49:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe8a61dab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662d3ad110 con 0x55662d36de50 2023-10-06T04:49:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe8a61dab80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55662d36ef90 con 0x55662d34fa10 2023-10-06T04:49:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe89579b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34e2c0 0x55662d348ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe89579b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34e2c0 0x55662d348ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34586/0 (socket says 172.21.15.55:34586) 2023-10-06T04:49:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe89579b700 1 -- 172.21.15.55:0/2233709601 learned_addr learned my addr 172.21.15.55:0/2233709601 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe895f9c700 1 --2- 172.21.15.55:0/2233709601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d36de50 0x55662d34e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe894f9a700 1 --2- 172.21.15.55:0/2233709601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 0x55662d349010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe89579b700 1 -- 172.21.15.55:0/2233709601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 msgr2=0x55662d349010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe89579b700 1 --2- 172.21.15.55:0/2233709601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 0x55662d349010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe89579b700 1 -- 172.21.15.55:0/2233709601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d36de50 msgr2=0x55662d34e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe89579b700 1 --2- 172.21.15.55:0/2233709601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d36de50 0x55662d34e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.767+0000 7fe89579b700 1 -- 172.21.15.55:0/2233709601 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55662d26b780 con 0x55662d34e2c0 2023-10-06T04:49:31.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.768+0000 7fe89579b700 1 --2- 172.21.15.55:0/2233709601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34e2c0 0x55662d348ad0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00ce00 tx=0x7fe88c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7da2998c85d2199 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:31.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.768+0000 7fe87affd700 1 -- 172.21.15.55:0/2233709601 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c054900 con 0x55662d34e2c0 2023-10-06T04:49:31.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.768+0000 7fe87affd700 1 -- 172.21.15.55:0/2233709601 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c057b50 con 0x55662d34e2c0 2023-10-06T04:49:31.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.768+0000 7fe87affd700 1 -- 172.21.15.55:0/2233709601 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c05b8a0 con 0x55662d34e2c0 2023-10-06T04:49:31.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.768+0000 7fe8a61dab80 1 -- 172.21.15.55:0/2233709601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34e2c0 msgr2=0x55662d348ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.768+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/2233709601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34e2c0 0x55662d348ad0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00ce00 tx=0x7fe88c001e50 comp rx=0 tx=0).stop 2023-10-06T04:49:31.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.768+0000 7fe8a61dab80 1 -- 172.21.15.55:0/2233709601 shutdown_connections 2023-10-06T04:49:31.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.768+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/2233709601 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d36de50 0x55662d34e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.768+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/2233709601 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d34e2c0 0x55662d348ad0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.768+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/2233709601 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 0x55662d349010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.768+0000 7fe8a61dab80 1 -- 172.21.15.55:0/2233709601 >> 172.21.15.55:0/2233709601 conn(0x55662d34de80 msgr2=0x55662d34a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:31.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.768+0000 7fe8a61dab80 1 -- 172.21.15.55:0/2233709601 shutdown_connections 2023-10-06T04:49:31.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.768+0000 7fe8a61dab80 1 -- 172.21.15.55:0/2233709601 wait complete. 2023-10-06T04:49:31.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.769+0000 7fe8a61dab80 1 -- start start 2023-10-06T04:49:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.769+0000 7fe8a61dab80 1 Processor -- start 2023-10-06T04:49:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.770+0000 7fe8a61dab80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d34e2c0 0x55662d3f8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.770+0000 7fe8a61dab80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 0x55662d36fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.770+0000 7fe8a61dab80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d36de50 0x55662d3703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.770+0000 7fe8a61dab80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662d3943c0 con 0x55662d36de50 2023-10-06T04:49:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.770+0000 7fe8a61dab80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x55662d3f8de0 con 0x55662d34e2c0 2023-10-06T04:49:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.770+0000 7fe8a61dab80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x55662d3945d0 con 0x55662d34fa10 2023-10-06T04:49:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.770+0000 7fe8a61dab80 10 client.0 _ll_register_callbacks cb 0x55662d310020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:49:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.771+0000 7fe895f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d36de50 0x55662d3703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.771+0000 7fe895f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d36de50 0x55662d3703d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34596/0 (socket says 172.21.15.55:34596) 2023-10-06T04:49:31.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.771+0000 7fe895f9c700 1 -- 172.21.15.55:0/901222183 learned_addr learned my addr 172.21.15.55:0/901222183 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:31.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.771+0000 7fe89579b700 1 --2- 172.21.15.55:0/901222183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d34e2c0 0x55662d3f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.771+0000 7fe894f9a700 1 --2- 172.21.15.55:0/901222183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 0x55662d36fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.771+0000 7fe895f9c700 1 -- 172.21.15.55:0/901222183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 msgr2=0x55662d36fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:31.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.771+0000 7fe895f9c700 1 --2- 172.21.15.55:0/901222183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 0x55662d36fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.771+0000 7fe895f9c700 1 -- 172.21.15.55:0/901222183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d34e2c0 msgr2=0x55662d3f8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:31.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.771+0000 7fe895f9c700 1 --2- 172.21.15.55:0/901222183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d34e2c0 0x55662d3f8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.771+0000 7fe895f9c700 1 -- 172.21.15.55:0/901222183 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89016a4b0 con 0x55662d36de50 2023-10-06T04:49:31.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.771+0000 7fe895f9c700 1 --2- 172.21.15.55:0/901222183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d36de50 0x55662d3703d0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fe888016e20 tx=0x7fe8880015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884a8a938a77e07b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:31.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.771+0000 7fe86f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T04:49:31.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.772+0000 7fe86f7fe700 1 -- 172.21.15.55:0/901222183 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe888016e50 con 0x55662d36de50 2023-10-06T04:49:31.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.772+0000 7fe86f7fe700 1 -- 172.21.15.55:0/901222183 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fe888021e20 con 0x55662d36de50 2023-10-06T04:49:31.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.772+0000 7fe8a61dab80 1 -- 172.21.15.55:0/901222183 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8900c1e30 con 0x55662d36de50 2023-10-06T04:49:31.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.772+0000 7fe86f7fe700 1 -- 172.21.15.55:0/901222183 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe888014ab0 con 0x55662d36de50 2023-10-06T04:49:31.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.772+0000 7fe8a61dab80 10 client.9366 fetch_fsmap learned FSMap version 87 2023-10-06T04:49:31.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.772+0000 7fe8a61dab80 1 -- 172.21.15.55:0/901222183 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x55662d252470 con 0x55662d36de50 2023-10-06T04:49:31.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.772+0000 7fe86f7fe700 1 -- 172.21.15.55:0/901222183 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8900c1e30 con 0x55662d36de50 2023-10-06T04:49:31.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.772+0000 7fe86f7fe700 1 -- 172.21.15.55:0/901222183 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 87) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe888014c10 con 0x55662d36de50 2023-10-06T04:49:31.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.772+0000 7fe8a61dab80 10 client.9366 fetch_fsmap finished waiting for FSMap version 87 2023-10-06T04:49:31.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.772+0000 7fe8a61dab80 10 client.9366 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-06T04:49:31.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.772+0000 7fe8a61dab80 20 client.9366 populate_metadata read hostname 'smithi055' 2023-10-06T04:49:31.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.772+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/901222183 >> [v2:172.21.15.163:6832/3095914721,v1:172.21.15.163:6834/3095914721] conn(0x55662d369af0 0x55662d369ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.772+0000 7fe8a61dab80 4 client.9366 mds_command: new command op to 8740 tid=0[{"prefix": "status"}] 2023-10-06T04:49:31.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.772+0000 7fe8a61dab80 1 -- 172.21.15.55:0/901222183 --> [v2:172.21.15.163:6832/3095914721,v1:172.21.15.163:6834/3095914721] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fe880021040 con 0x55662d369af0 2023-10-06T04:49:31.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.773+0000 7fe89579b700 1 --2- 172.21.15.55:0/901222183 >> [v2:172.21.15.163:6832/3095914721,v1:172.21.15.163:6834/3095914721] conn(0x55662d369af0 0x55662d369ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.773+0000 7fe89579b700 1 --2- 172.21.15.55:0/901222183 >> [v2:172.21.15.163:6832/3095914721,v1:172.21.15.163:6834/3095914721] conn(0x55662d369af0 0x55662d369ec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afdd7bc6ac5f1c76 server_cookie=ebf753924545a261 in_seq=0 out_seq=0 2023-10-06T04:49:31.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.773+0000 7fe86f7fe700 10 client.9366 ms_handle_connect on v2:172.21.15.163:6832/3095914721 2023-10-06T04:49:31.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe86f7fe700 1 -- 172.21.15.55:0/901222183 <== mds.0 v2:172.21.15.163:6832/3095914721 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x7fe880021040 con 0x55662d369af0 2023-10-06T04:49:31.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe86f7fe700 10 client.9366 handle_command_reply: tid=0 2023-10-06T04:49:31.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe8a61dab80 1 client.9366 shutdown 2023-10-06T04:49:31.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe8a61dab80 1 -- 172.21.15.55:0/901222183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d36de50 msgr2=0x55662d3703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/901222183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d36de50 0x55662d3703d0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fe888016e20 tx=0x7fe8880015f0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe8a61dab80 1 -- 172.21.15.55:0/901222183 shutdown_connections 2023-10-06T04:49:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/901222183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x55662d36de50 0x55662d3703d0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/901222183 >> [v2:172.21.15.163:6832/3095914721,v1:172.21.15.163:6834/3095914721] conn(0x55662d369af0 0x55662d369ec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/901222183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x55662d34e2c0 0x55662d3f8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe8a61dab80 1 --2- 172.21.15.55:0/901222183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x55662d34fa10 0x55662d36fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe8a61dab80 1 -- 172.21.15.55:0/901222183 >> 172.21.15.55:0/901222183 conn(0x55662d34de80 msgr2=0x55662d34c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe86f7fe700 0 client.9366 ms_handle_reset on v2:172.21.15.163:6832/3095914721 2023-10-06T04:49:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe8a61dab80 1 -- 172.21.15.55:0/901222183 shutdown_connections 2023-10-06T04:49:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe8a61dab80 1 -- 172.21.15.55:0/901222183 wait complete. 2023-10-06T04:49:31.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.774+0000 7fe8a61dab80 20 client.9366 trim_cache size 0 max 16384 2023-10-06T04:49:31.782 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T04:49:31.782 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "8132c8ba-ee1c-41a2-b781-913925034ab6", 2023-10-06T04:49:31.782 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2023-10-06T04:49:31.782 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 8740, 2023-10-06T04:49:31.782 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2023-10-06T04:49:31.782 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2023-10-06T04:49:31.782 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2023-10-06T04:49:31.782 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 11.46108579, 2023-10-06T04:49:31.782 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 87, 2023-10-06T04:49:31.782 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 173, 2023-10-06T04:49:31.783 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 173, 2023-10-06T04:49:31.783 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 17.211066129999999 2023-10-06T04:49:31.783 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T04:49:31.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.775+0000 7fe89679d700 1 -- 172.21.15.55:0/30006275 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe874066f60 msgr2=0x7fe874069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.775+0000 7fe89679d700 1 --2- 172.21.15.55:0/30006275 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe874066f60 0x7fe874069410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe890156b50 tx=0x7fe880017000 comp rx=0 tx=0).stop 2023-10-06T04:49:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.775+0000 7fe89679d700 1 -- 172.21.15.55:0/30006275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8900cb900 msgr2=0x7fe8900aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.775+0000 7fe89679d700 1 --2- 172.21.15.55:0/30006275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8900cb900 0x7fe8900aef90 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fe88c00c810 tx=0x7fe88c007d90 comp rx=0 tx=0).stop 2023-10-06T04:49:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.775+0000 7fe89679d700 1 -- 172.21.15.55:0/30006275 shutdown_connections 2023-10-06T04:49:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.775+0000 7fe89679d700 1 --2- 172.21.15.55:0/30006275 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fe874066f60 0x7fe874069410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.775+0000 7fe89679d700 1 --2- 172.21.15.55:0/30006275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8901566e0 0x7fe89014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.775+0000 7fe89679d700 1 --2- 172.21.15.55:0/30006275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8900cb900 0x7fe8900aef90 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.775+0000 7fe89679d700 1 --2- 172.21.15.55:0/30006275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe890156310 0x7fe890169a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.775+0000 7fe89679d700 1 -- 172.21.15.55:0/30006275 >> 172.21.15.55:0/30006275 conn(0x7fe8900cc1f0 msgr2=0x7fe890151eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.775+0000 7fe89679d700 1 -- 172.21.15.55:0/30006275 shutdown_connections 2023-10-06T04:49:31.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.775+0000 7fe89679d700 1 -- 172.21.15.55:0/30006275 wait complete. 2023-10-06T04:49:31.793 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:49:31.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.978+0000 7f1ca899d700 1 Processor -- start 2023-10-06T04:49:31.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.978+0000 7f1ca899d700 1 -- start start 2023-10-06T04:49:31.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.979+0000 7f1ca899d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ca41566e0 0x7f1ca40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.979+0000 7f1ca899d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1ca40cb900 0x7f1ca40ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.979+0000 7f1ca899d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca40ca860 con 0x7f1ca4156310 2023-10-06T04:49:31.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.979+0000 7f1ca899d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca40cab40 con 0x7f1ca40cb900 2023-10-06T04:49:31.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.979+0000 7f1ca899d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40cae20 con 0x7f1ca41566e0 2023-10-06T04:49:31.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.980+0000 7f1ca2ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ca41566e0 0x7f1ca40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.980+0000 7f1ca37fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1ca4156310 0x7f1ca40c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:42208/0 (socket says 172.21.15.55:42208) 2023-10-06T04:49:31.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.980+0000 7f1ca3fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1ca40cb900 0x7f1ca40ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.980+0000 7f1ca37fe700 1 -- 172.21.15.55:0/3966018700 learned_addr learned my addr 172.21.15.55:0/3966018700 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:31.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.981+0000 7f1ca27fc700 1 -- 172.21.15.55:0/3966018700 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3868919139 0 0) 0x7f1ca40ca860 con 0x7f1ca4156310 2023-10-06T04:49:31.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.981+0000 7f1ca2ffd700 1 -- 172.21.15.55:0/3966018700 >> v2:172.21.15.163:3300/0 conn(0x7f1ca40cb900 msgr2=0x7f1ca40ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:31.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.981+0000 7f1ca2ffd700 1 --2- 172.21.15.55:0/3966018700 >> v2:172.21.15.163:3300/0 conn(0x7f1ca40cb900 0x7f1ca40ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.981+0000 7f1ca2ffd700 1 -- 172.21.15.55:0/3966018700 >> v1:172.21.15.163:6789/0 conn(0x7f1ca4156310 legacy=0x7f1ca40c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.981+0000 7f1ca2ffd700 1 -- 172.21.15.55:0/3966018700 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca40cb100 con 0x7f1ca41566e0 2023-10-06T04:49:31.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.981+0000 7f1ca2ffd700 1 --2- 172.21.15.55:0/3966018700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ca41566e0 0x7f1ca40ae000 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1c980098e0 tx=0x7f1c980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4194f5e02f1d7b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:31.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.981+0000 7f1ca27fc700 1 -- 172.21.15.55:0/3966018700 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9800b640 con 0x7f1ca41566e0 2023-10-06T04:49:31.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.981+0000 7f1ca27fc700 1 -- 172.21.15.55:0/3966018700 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9800b7a0 con 0x7f1ca41566e0 2023-10-06T04:49:31.984 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.981+0000 7f1ca27fc700 1 -- 172.21.15.55:0/3966018700 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9800bb40 con 0x7f1ca41566e0 2023-10-06T04:49:31.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.982+0000 7f1ca899d700 1 -- 172.21.15.55:0/3966018700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ca41566e0 msgr2=0x7f1ca40ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.982+0000 7f1ca899d700 1 --2- 172.21.15.55:0/3966018700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ca41566e0 0x7f1ca40ae000 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1c980098e0 tx=0x7f1c980091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.982+0000 7f1ca899d700 1 -- 172.21.15.55:0/3966018700 shutdown_connections 2023-10-06T04:49:31.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.982+0000 7f1ca899d700 1 --2- 172.21.15.55:0/3966018700 >> v2:172.21.15.163:3300/0 conn(0x7f1ca40cb900 0x7f1ca40ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.982+0000 7f1ca899d700 1 --2- 172.21.15.55:0/3966018700 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ca41566e0 0x7f1ca40ae000 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.982+0000 7f1ca899d700 1 -- 172.21.15.55:0/3966018700 >> 172.21.15.55:0/3966018700 conn(0x7f1ca40cc1f0 msgr2=0x7f1ca40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:31.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.982+0000 7f1ca899d700 1 -- 172.21.15.55:0/3966018700 shutdown_connections 2023-10-06T04:49:31.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.982+0000 7f1ca899d700 1 -- 172.21.15.55:0/3966018700 wait complete. 2023-10-06T04:49:31.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.983+0000 7f1ca899d700 1 Processor -- start 2023-10-06T04:49:31.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.984+0000 7f1ca899d700 1 -- start start 2023-10-06T04:49:31.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.984+0000 7f1ca899d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca40cb900 0x7f1ca4150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.984+0000 7f1ca899d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca4156310 0x7f1ca40af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.984+0000 7f1ca899d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ca41566e0 0x7f1ca414b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.984+0000 7f1ca899d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40afe90 con 0x7f1ca40cb900 2023-10-06T04:49:31.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.985+0000 7f1ca899d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40b0110 con 0x7f1ca4156310 2023-10-06T04:49:31.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.985+0000 7f1ca899d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40b03f0 con 0x7f1ca41566e0 2023-10-06T04:49:31.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.985+0000 7f1ca37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca40cb900 0x7f1ca4150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.985+0000 7f1ca37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca40cb900 0x7f1ca4150a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:34602/0 (socket says 172.21.15.55:34602) 2023-10-06T04:49:31.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.985+0000 7f1ca37fe700 1 -- 172.21.15.55:0/3436369517 learned_addr learned my addr 172.21.15.55:0/3436369517 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:31.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.985+0000 7f1ca37fe700 1 -- 172.21.15.55:0/3436369517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ca41566e0 msgr2=0x7f1ca414b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.985+0000 7f1ca37fe700 1 --2- 172.21.15.55:0/3436369517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ca41566e0 0x7f1ca414b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.985+0000 7f1ca37fe700 1 -- 172.21.15.55:0/3436369517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca4156310 msgr2=0x7f1ca40af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:31.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.985+0000 7f1ca37fe700 1 --2- 172.21.15.55:0/3436369517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca4156310 0x7f1ca40af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:31.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.985+0000 7f1ca37fe700 1 -- 172.21.15.55:0/3436369517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c98009420 con 0x7f1ca40cb900 2023-10-06T04:49:31.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.985+0000 7f1ca37fe700 1 --2- 172.21.15.55:0/3436369517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca40cb900 0x7f1ca4150a60 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400c7e0 tx=0x7f1c94007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:31.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.986+0000 7f1c8b7fe700 1 -- 172.21.15.55:0/3436369517 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94005b30 con 0x7f1ca40cb900 2023-10-06T04:49:31.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.986+0000 7f1c8b7fe700 1 -- 172.21.15.55:0/3436369517 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c94005c90 con 0x7f1ca40cb900 2023-10-06T04:49:31.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.986+0000 7f1c8b7fe700 1 -- 172.21.15.55:0/3436369517 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94007300 con 0x7f1ca40cb900 2023-10-06T04:49:31.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.986+0000 7f1ca899d700 1 -- 172.21.15.55:0/3436369517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca40b06d0 con 0x7f1ca40cb900 2023-10-06T04:49:31.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.986+0000 7f1ca899d700 1 -- 172.21.15.55:0/3436369517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca4157670 con 0x7f1ca40cb900 2023-10-06T04:49:31.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.987+0000 7f1ca899d700 1 -- 172.21.15.55:0/3436369517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca40ae3f0 con 0x7f1ca40cb900 2023-10-06T04:49:31.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.988+0000 7f1c8b7fe700 1 -- 172.21.15.55:0/3436369517 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c9401b400 con 0x7f1ca40cb900 2023-10-06T04:49:31.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.989+0000 7f1c8b7fe700 1 --2- 172.21.15.55:0/3436369517 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c90066f30 0x7f1c900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:31.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.989+0000 7f1c8b7fe700 1 -- 172.21.15.55:0/3436369517 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f1c94084430 con 0x7f1ca40cb900 2023-10-06T04:49:31.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.989+0000 7f1ca2ffd700 1 --2- 172.21.15.55:0/3436369517 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c90066f30 0x7f1c900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:31.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.990+0000 7f1ca2ffd700 1 --2- 172.21.15.55:0/3436369517 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c90066f30 0x7f1c900693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1c980098e0 tx=0x7f1c98017000 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:31.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:31.993+0000 7f1c8b7fe700 1 -- 172.21.15.55:0/3436369517 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c94051390 con 0x7f1ca40cb900 2023-10-06T04:49:32.047 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:49:32.052 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:49:32.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:32.243+0000 7f1ca899d700 1 -- 172.21.15.55:0/3436369517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1ca404c3c0 con 0x7f1ca40cb900 2023-10-06T04:49:32.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:32.243+0000 7f1c8b7fe700 1 -- 172.21.15.55:0/3436369517 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v87) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1c940549b0 con 0x7f1ca40cb900 2023-10-06T04:49:32.245 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:49:32.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:32.245+0000 7f1ca899d700 1 -- 172.21.15.55:0/3436369517 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c90066f30 msgr2=0x7f1c900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:32.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:32.245+0000 7f1ca899d700 1 --2- 172.21.15.55:0/3436369517 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c90066f30 0x7f1c900693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1c980098e0 tx=0x7f1c98017000 comp rx=0 tx=0).stop 2023-10-06T04:49:32.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:32.245+0000 7f1ca899d700 1 -- 172.21.15.55:0/3436369517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca40cb900 msgr2=0x7f1ca4150a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:32.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:32.245+0000 7f1ca899d700 1 --2- 172.21.15.55:0/3436369517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca40cb900 0x7f1ca4150a60 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400c7e0 tx=0x7f1c94007df0 comp rx=0 tx=0).stop 2023-10-06T04:49:32.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:32.245+0000 7f1ca899d700 1 -- 172.21.15.55:0/3436369517 shutdown_connections 2023-10-06T04:49:32.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:32.245+0000 7f1ca899d700 1 --2- 172.21.15.55:0/3436369517 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f1c90066f30 0x7f1c900693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:32.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:32.245+0000 7f1ca899d700 1 --2- 172.21.15.55:0/3436369517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ca41566e0 0x7f1ca414b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:32.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:32.245+0000 7f1ca899d700 1 --2- 172.21.15.55:0/3436369517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ca40cb900 0x7f1ca4150a60 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:32.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:32.245+0000 7f1ca899d700 1 --2- 172.21.15.55:0/3436369517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ca4156310 0x7f1ca40af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:32.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:32.245+0000 7f1ca899d700 1 -- 172.21.15.55:0/3436369517 >> 172.21.15.55:0/3436369517 conn(0x7f1ca40cc1f0 msgr2=0x7f1ca40adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:32.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:32.246+0000 7f1ca899d700 1 -- 172.21.15.55:0/3436369517 shutdown_connections 2023-10-06T04:49:32.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:32.246+0000 7f1ca899d700 1 -- 172.21.15.55:0/3436369517 wait complete. 2023-10-06T04:49:32.265 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T04:49:32.265 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:49:32.266 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2023-10-06T04:49:32.266 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T04:49:32.266 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T04:49:32.266 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T04:49:32.266 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T04:49:32.266 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:49:32.266 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:49:32.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:49:32.266 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2023-10-06T04:49:32.304 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:49:32.304 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:49:32.304 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:49:32.304 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:49:32.304 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T04:49:32.304 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:49:32.304 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:49:32.304 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:49:32.304 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2023-10-06T04:49:32.305 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:49:32.305 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:49:32.305 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2023-10-06T04:49:32.305 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:49:32.305 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:49:32.305 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2023-10-06T04:49:32.305 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T04:49:32.305 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 37669sec preferred_lft 37669sec 2023-10-06T04:49:32.305 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link 2023-10-06T04:49:32.305 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:49:32.305 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:49:32.306 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:49:32.306 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2023-10-06T04:49:32.306 INFO:teuthology.orchestra.run.smithi055.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:49:32.306 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff 2023-10-06T04:49:32.306 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:49:32.306 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:49:32.306 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::1099:8eff:fe6d:102/64 scope link 2023-10-06T04:49:32.306 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:49:32.306 INFO:teuthology.orchestra.run.smithi055.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:49:32.306 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether f2:5d:53:d2:00:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:49:32.306 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f05d:53ff:fed2:a/64 scope link 2023-10-06T04:49:32.306 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:49:32.307 INFO:teuthology.orchestra.run.smithi055.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:49:32.307 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:49:32.307 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::e037:33ff:feb7:6c8f/64 scope link 2023-10-06T04:49:32.307 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:49:32.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:49:32.307 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:49:32.404 INFO:teuthology.orchestra.run.smithi055.stdout:1 2023-10-06T04:49:32.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:49:32.407 DEBUG:teuthology.orchestra.run.smithi055:> route 2023-10-06T04:49:32.445 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2023-10-06T04:49:32.445 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:49:32.445 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T04:49:32.445 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T04:49:32.445 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:49:32.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:49:32.447 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T04:49:32.447 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T04:49:32.447 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T04:49:32.447 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T04:49:32.447 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T04:49:32.447 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T04:49:32.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:49:32.646 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T04:49:32.682 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T04:49:32.682 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:49:32.683 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:49:32.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:49:32.683 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2023-10-06T04:49:32.786 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:49:32.786 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T04:49:32.823 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:49:32.823 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:49:32.823 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:49:32.901 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T04:49:32.903 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:49:32.903 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T04:49:33.033 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T04:49:33.033 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T04:49:33.033 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T04:49:33.049 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T04:49:33.049 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T04:49:33.049 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T04:49:33.049 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T04:49:33.049 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T04:49:33.071 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T04:49:33.071 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T04:49:33.072 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T04:49:33.072 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T04:49:33.072 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T04:49:33.072 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T04:49:33.072 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T04:49:33.072 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T04:49:33.078 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T04:49:33.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:49:33.078 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:49:33.141 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T04:49:33.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.327+0000 7fb133fff700 1 Processor -- start 2023-10-06T04:49:33.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.327+0000 7fb133fff700 1 -- start start 2023-10-06T04:49:33.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.328+0000 7fb133fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1340ae710 0x7fb1340c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.328+0000 7fb133fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1340ca860 con 0x7fb134156310 2023-10-06T04:49:33.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.328+0000 7fb133fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1340cab40 con 0x7fb1341566e0 2023-10-06T04:49:33.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.328+0000 7fb133fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340cae20 con 0x7fb1340ae710 2023-10-06T04:49:33.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.328+0000 7fb132ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb134156310 0x7fb1340c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56566/0 (socket says 172.21.15.55:56566) 2023-10-06T04:49:33.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.329+0000 7fb132ffd700 1 -- 172.21.15.55:0/2820963005 learned_addr learned my addr 172.21.15.55:0/2820963005 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:33.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.329+0000 7fb1337fe700 1 --2- 172.21.15.55:0/2820963005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1340ae710 0x7fb1340c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:33.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.329+0000 7fb131ffb700 1 -- 172.21.15.55:0/2820963005 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4168912172 0 0) 0x7fb1340ca860 con 0x7fb134156310 2023-10-06T04:49:33.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.329+0000 7fb131ffb700 1 -- 172.21.15.55:0/2820963005 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb11c003570 con 0x7fb134156310 2023-10-06T04:49:33.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.329+0000 7fb131ffb700 1 -- 172.21.15.55:0/2820963005 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1443337501 0 0) 0x7fb11c003570 con 0x7fb134156310 2023-10-06T04:49:33.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.329+0000 7fb131ffb700 1 -- 172.21.15.55:0/2820963005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1340ae710 msgr2=0x7fb1340c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:33.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.329+0000 7fb131ffb700 1 --2- 172.21.15.55:0/2820963005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1340ae710 0x7fb1340c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.329+0000 7fb131ffb700 1 -- 172.21.15.55:0/2820963005 >> v1:172.21.15.163:6789/0 conn(0x7fb1341566e0 legacy=0x7fb1340ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:33.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.329+0000 7fb131ffb700 1 -- 172.21.15.55:0/2820963005 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1340cb100 con 0x7fb134156310 2023-10-06T04:49:33.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.329+0000 7fb131ffb700 1 -- 172.21.15.55:0/2820963005 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fb124001470 con 0x7fb134156310 2023-10-06T04:49:33.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.329+0000 7fb131ffb700 1 -- 172.21.15.55:0/2820963005 >> v1:172.21.15.55:6789/0 conn(0x7fb134156310 legacy=0x7fb1340c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:33.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.329+0000 7fb131ffb700 1 --2- 172.21.15.55:0/2820963005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1340ae710 0x7fb11c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.329+0000 7fb131ffb700 1 --2- 172.21.15.55:0/2820963005 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb11c003fd0 0x7fb11c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.329+0000 7fb131ffb700 1 --2- 172.21.15.55:0/2820963005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb11c004d50 0x7fb11c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.330+0000 7fb131ffb700 1 -- 172.21.15.55:0/2820963005 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340cae20 con 0x7fb11c003fd0 2023-10-06T04:49:33.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.330+0000 7fb131ffb700 1 -- 172.21.15.55:0/2820963005 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c00bba0 con 0x7fb11c004d50 2023-10-06T04:49:33.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.330+0000 7fb131ffb700 1 -- 172.21.15.55:0/2820963005 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c00bd60 con 0x7fb1340ae710 2023-10-06T04:49:33.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.330+0000 7fb1327fc700 1 --2- 172.21.15.55:0/2820963005 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb11c003fd0 0x7fb11c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:33.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.330+0000 7fb1337fe700 1 --2- 172.21.15.55:0/2820963005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb11c004d50 0x7fb11c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:33.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.330+0000 7fb132ffd700 1 --2- 172.21.15.55:0/2820963005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1340ae710 0x7fb11c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:33.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.330+0000 7fb1327fc700 1 -- 172.21.15.55:0/2820963005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1340ae710 msgr2=0x7fb11c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:33.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.330+0000 7fb1327fc700 1 --2- 172.21.15.55:0/2820963005 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb1340ae710 0x7fb11c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.330+0000 7fb1327fc700 1 -- 172.21.15.55:0/2820963005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb11c004d50 msgr2=0x7fb11c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:33.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.330+0000 7fb1327fc700 1 --2- 172.21.15.55:0/2820963005 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb11c004d50 0x7fb11c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.330+0000 7fb1327fc700 1 -- 172.21.15.55:0/2820963005 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c003570 con 0x7fb11c003fd0 2023-10-06T04:49:33.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.330+0000 7fb1327fc700 1 --2- 172.21.15.55:0/2820963005 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb11c003fd0 0x7fb11c004810 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fb128001400 tx=0x7fb128007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e7e6745df9aa28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:33.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.331+0000 7fb131ffb700 1 -- 172.21.15.55:0/2820963005 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12800c5d0 con 0x7fb11c003fd0 2023-10-06T04:49:33.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.331+0000 7fb131ffb700 1 -- 172.21.15.55:0/2820963005 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12800c750 con 0x7fb11c003fd0 2023-10-06T04:49:33.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.331+0000 7fb131ffb700 1 -- 172.21.15.55:0/2820963005 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12800cb10 con 0x7fb11c003fd0 2023-10-06T04:49:33.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.331+0000 7fb133fff700 1 -- 172.21.15.55:0/2820963005 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb11c003fd0 msgr2=0x7fb11c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:33.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.331+0000 7fb133fff700 1 --2- 172.21.15.55:0/2820963005 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb11c003fd0 0x7fb11c004810 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fb128001400 tx=0x7fb128007d10 comp rx=0 tx=0).stop 2023-10-06T04:49:33.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.331+0000 7fb1337fe700 1 -- 172.21.15.55:0/2820963005 reap_dead start 2023-10-06T04:49:33.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.331+0000 7fb133fff700 1 -- 172.21.15.55:0/2820963005 shutdown_connections 2023-10-06T04:49:33.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.331+0000 7fb133fff700 1 -- 172.21.15.55:0/2820963005 >> 172.21.15.55:0/2820963005 conn(0x7fb1340cc1f0 msgr2=0x7fb1340cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:33.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.331+0000 7fb133fff700 1 -- 172.21.15.55:0/2820963005 shutdown_connections 2023-10-06T04:49:33.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.331+0000 7fb133fff700 1 -- 172.21.15.55:0/2820963005 wait complete. 2023-10-06T04:49:33.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.333+0000 7fb133fff700 1 Processor -- start 2023-10-06T04:49:33.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.333+0000 7fb133fff700 1 -- start start 2023-10-06T04:49:33.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.334+0000 7fb133fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb11c003fd0 0x7fb134169930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.334+0000 7fb133fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb13414c440 0x7fb1340c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.334+0000 7fb133fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb13414c8b0 0x7fb1340c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.334+0000 7fb133fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134150d70 con 0x7fb13414c440 2023-10-06T04:49:33.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.334+0000 7fb133fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340b0510 con 0x7fb13414c8b0 2023-10-06T04:49:33.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.334+0000 7fb133fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340b07f0 con 0x7fb11c003fd0 2023-10-06T04:49:33.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.334+0000 7fb1327fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb13414c440 0x7fb1340c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:33.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.334+0000 7fb1327fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb13414c440 0x7fb1340c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44576/0 (socket says 172.21.15.55:44576) 2023-10-06T04:49:33.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.334+0000 7fb1327fc700 1 -- 172.21.15.55:0/2199608977 learned_addr learned my addr 172.21.15.55:0/2199608977 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:33.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.334+0000 7fb132ffd700 1 --2- 172.21.15.55:0/2199608977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb11c003fd0 0x7fb134169930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:33.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.334+0000 7fb1337fe700 1 --2- 172.21.15.55:0/2199608977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb13414c8b0 0x7fb1340c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:33.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.335+0000 7fb1327fc700 1 -- 172.21.15.55:0/2199608977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb11c003fd0 msgr2=0x7fb134169930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:33.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.335+0000 7fb1327fc700 1 --2- 172.21.15.55:0/2199608977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb11c003fd0 0x7fb134169930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.335+0000 7fb1327fc700 1 -- 172.21.15.55:0/2199608977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb13414c8b0 msgr2=0x7fb1340c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:33.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.335+0000 7fb1327fc700 1 --2- 172.21.15.55:0/2199608977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb13414c8b0 0x7fb1340c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.335+0000 7fb1327fc700 1 -- 172.21.15.55:0/2199608977 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb134150720 con 0x7fb13414c440 2023-10-06T04:49:33.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.335+0000 7fb1327fc700 1 --2- 172.21.15.55:0/2199608977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb13414c440 0x7fb1340c76b0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb1280117a0 tx=0x7fb128017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:33.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.335+0000 7fb11affd700 1 -- 172.21.15.55:0/2199608977 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12801f070 con 0x7fb13414c440 2023-10-06T04:49:33.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.335+0000 7fb11affd700 1 -- 172.21.15.55:0/2199608977 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12800c5d0 con 0x7fb13414c440 2023-10-06T04:49:33.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.335+0000 7fb133fff700 1 -- 172.21.15.55:0/2199608977 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb134150d70 con 0x7fb13414c440 2023-10-06T04:49:33.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.335+0000 7fb11affd700 1 -- 172.21.15.55:0/2199608977 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb128007af0 con 0x7fb13414c440 2023-10-06T04:49:33.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.335+0000 7fb133fff700 1 -- 172.21.15.55:0/2199608977 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1340af7b0 con 0x7fb13414c440 2023-10-06T04:49:33.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.336+0000 7fb133fff700 1 -- 172.21.15.55:0/2199608977 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13404c3c0 con 0x7fb13414c440 2023-10-06T04:49:33.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.337+0000 7fb11affd700 1 -- 172.21.15.55:0/2199608977 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb1280193f0 con 0x7fb13414c440 2023-10-06T04:49:33.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.338+0000 7fb11affd700 1 --2- 172.21.15.55:0/2199608977 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb120066f30 0x7fb1200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.338+0000 7fb11affd700 1 -- 172.21.15.55:0/2199608977 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fb128085450 con 0x7fb13414c440 2023-10-06T04:49:33.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.339+0000 7fb132ffd700 1 --2- 172.21.15.55:0/2199608977 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb120066f30 0x7fb1200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:33.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.339+0000 7fb132ffd700 1 --2- 172.21.15.55:0/2199608977 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb120066f30 0x7fb1200693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb124000e50 tx=0x7fb124001da0 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:33.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.343+0000 7fb11affd700 1 -- 172.21.15.55:0/2199608977 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb128052490 con 0x7fb13414c440 2023-10-06T04:49:33.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.572+0000 7fb133fff700 1 -- 172.21.15.55:0/2199608977 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1340cd0b0 con 0x7fb120066f30 2023-10-06T04:49:33.576 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.574+0000 7fb11affd700 1 -- 172.21.15.55:0/2199608977 <== mgr.8557 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb1340cd0b0 con 0x7fb120066f30 2023-10-06T04:49:33.576 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T04:49:33.576 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T04:49:33.577 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephfs" 2023-10-06T04:49:33.577 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T04:49:33.577 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T04:49:33.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.576+0000 7fb133fff700 1 -- 172.21.15.55:0/2199608977 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb120066f30 msgr2=0x7fb1200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:33.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.576+0000 7fb133fff700 1 --2- 172.21.15.55:0/2199608977 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb120066f30 0x7fb1200693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb124000e50 tx=0x7fb124001da0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.576+0000 7fb133fff700 1 -- 172.21.15.55:0/2199608977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb13414c440 msgr2=0x7fb1340c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:33.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.576+0000 7fb133fff700 1 --2- 172.21.15.55:0/2199608977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb13414c440 0x7fb1340c76b0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb1280117a0 tx=0x7fb128017000 comp rx=0 tx=0).stop 2023-10-06T04:49:33.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.576+0000 7fb133fff700 1 -- 172.21.15.55:0/2199608977 shutdown_connections 2023-10-06T04:49:33.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.576+0000 7fb133fff700 1 --2- 172.21.15.55:0/2199608977 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fb120066f30 0x7fb1200693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.576+0000 7fb133fff700 1 --2- 172.21.15.55:0/2199608977 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb13414c8b0 0x7fb1340c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.576+0000 7fb133fff700 1 --2- 172.21.15.55:0/2199608977 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb11c003fd0 0x7fb134169930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.576+0000 7fb133fff700 1 --2- 172.21.15.55:0/2199608977 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb13414c440 0x7fb1340c76b0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.576+0000 7fb133fff700 1 -- 172.21.15.55:0/2199608977 >> 172.21.15.55:0/2199608977 conn(0x7fb1340cc1f0 msgr2=0x7fb1340c9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:33.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.576+0000 7fb133fff700 1 -- 172.21.15.55:0/2199608977 shutdown_connections 2023-10-06T04:49:33.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.576+0000 7fb133fff700 1 -- 172.21.15.55:0/2199608977 wait complete. 2023-10-06T04:49:33.595 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T04:49:33.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.784+0000 7f42a359e700 1 Processor -- start 2023-10-06T04:49:33.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.784+0000 7f42a359e700 1 -- start start 2023-10-06T04:49:33.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.785+0000 7f42a359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f42a40b1b40 0x7f42a40b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.785+0000 7f42a359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a40cfb00 con 0x7f42a40b16d0 2023-10-06T04:49:33.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.785+0000 7f42a359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a40b5c40 con 0x7f42a40b4e70 2023-10-06T04:49:33.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.785+0000 7f42a359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a40b5f20 con 0x7f42a40b1b40 2023-10-06T04:49:33.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.786+0000 7f42a1d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f42a40b16d0 0x7f42a40b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56578/0 (socket says 172.21.15.55:56578) 2023-10-06T04:49:33.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.786+0000 7f42a1d9b700 1 -- 172.21.15.55:0/3607657921 learned_addr learned my addr 172.21.15.55:0/3607657921 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:33.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.786+0000 7f42a2d9d700 1 --2- 172.21.15.55:0/3607657921 >> v2:172.21.15.163:3300/0 conn(0x7f42a40b1b40 0x7f42a40b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:33.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.786+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149725511 0 0) 0x7f42a40cfb00 con 0x7f42a40b16d0 2023-10-06T04:49:33.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.786+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f428c003570 con 0x7f42a40b16d0 2023-10-06T04:49:33.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.786+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1422764676 0 0) 0x7f42a40b5c40 con 0x7f42a40b4e70 2023-10-06T04:49:33.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.786+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42a40cfb00 con 0x7f42a40b4e70 2023-10-06T04:49:33.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.786+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1666366666 0 0) 0x7f428c003570 con 0x7f42a40b16d0 2023-10-06T04:49:33.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.786+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 >> v2:172.21.15.163:3300/0 conn(0x7f42a40b1b40 msgr2=0x7f42a40b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:33.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.786+0000 7f42a159a700 1 --2- 172.21.15.55:0/3607657921 >> v2:172.21.15.163:3300/0 conn(0x7f42a40b1b40 0x7f42a40b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.786+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 >> v1:172.21.15.163:6789/0 conn(0x7f42a40b4e70 legacy=0x7f42a40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:33.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.786+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a40b6200 con 0x7f42a40b16d0 2023-10-06T04:49:33.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.786+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f4298001470 con 0x7f42a40b16d0 2023-10-06T04:49:33.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.786+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 >> v1:172.21.15.55:6789/0 conn(0x7f42a40b16d0 legacy=0x7f42a40b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:33.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a159a700 1 --2- 172.21.15.55:0/3607657921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f428c003bf0 0x7f428c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a159a700 1 --2- 172.21.15.55:0/3607657921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f428c004810 0x7f428c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a159a700 1 --2- 172.21.15.55:0/3607657921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c005230 0x7f428c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40b5f20 con 0x7f428c005230 2023-10-06T04:49:33.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c00c200 con 0x7f428c004810 2023-10-06T04:49:33.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428c00c3c0 con 0x7f428c003bf0 2023-10-06T04:49:33.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a1d9b700 1 --2- 172.21.15.55:0/3607657921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f428c004810 0x7f428c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:33.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a2d9d700 1 --2- 172.21.15.55:0/3607657921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c005230 0x7f428c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:33.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a259c700 1 --2- 172.21.15.55:0/3607657921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f428c003bf0 0x7f428c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:33.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a2d9d700 1 -- 172.21.15.55:0/3607657921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f428c003bf0 msgr2=0x7f428c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:33.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a2d9d700 1 --2- 172.21.15.55:0/3607657921 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f428c003bf0 0x7f428c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a2d9d700 1 -- 172.21.15.55:0/3607657921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f428c004810 msgr2=0x7f428c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:33.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a2d9d700 1 --2- 172.21.15.55:0/3607657921 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f428c004810 0x7f428c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a2d9d700 1 -- 172.21.15.55:0/3607657921 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428c003570 con 0x7f428c005230 2023-10-06T04:49:33.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.787+0000 7f42a2d9d700 1 -- 172.21.15.55:0/3607657921 reap_dead start 2023-10-06T04:49:33.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.788+0000 7f42a2d9d700 1 --2- 172.21.15.55:0/3607657921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c005230 0x7f428c00ba60 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f429c00e210 tx=0x7f429c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ac36591b9a97e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:33.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.788+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c00eac0 con 0x7f428c005230 2023-10-06T04:49:33.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.788+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c00f040 con 0x7f428c005230 2023-10-06T04:49:33.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.788+0000 7f42a159a700 1 -- 172.21.15.55:0/3607657921 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c00ee70 con 0x7f428c005230 2023-10-06T04:49:33.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.788+0000 7f42a359e700 1 -- 172.21.15.55:0/3607657921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c005230 msgr2=0x7f428c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:33.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.788+0000 7f42a359e700 1 --2- 172.21.15.55:0/3607657921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c005230 0x7f428c00ba60 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f429c00e210 tx=0x7f429c00b040 comp rx=0 tx=0).stop 2023-10-06T04:49:33.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.788+0000 7f42a359e700 1 -- 172.21.15.55:0/3607657921 shutdown_connections 2023-10-06T04:49:33.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.788+0000 7f42a359e700 1 --2- 172.21.15.55:0/3607657921 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c005230 0x7f428c00ba60 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.788+0000 7f42a359e700 1 -- 172.21.15.55:0/3607657921 >> 172.21.15.55:0/3607657921 conn(0x7f42a40d10c0 msgr2=0x7f42a40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:33.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.788+0000 7f42a359e700 1 -- 172.21.15.55:0/3607657921 shutdown_connections 2023-10-06T04:49:33.797 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.789+0000 7f42a359e700 1 -- 172.21.15.55:0/3607657921 wait complete. 2023-10-06T04:49:33.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.790+0000 7f42a359e700 1 Processor -- start 2023-10-06T04:49:33.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.790+0000 7f42a359e700 1 -- start start 2023-10-06T04:49:33.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.791+0000 7f42a359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c005230 0x7f42a4147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.791+0000 7f42a359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42a40c8560 0x7f42a40c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.791+0000 7f42a359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42a40a6f00 0x7f42a40a7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.791+0000 7f42a359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40ca6c0 con 0x7f428c005230 2023-10-06T04:49:33.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.791+0000 7f42a359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40c8f10 con 0x7f42a40c8560 2023-10-06T04:49:33.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.791+0000 7f42a359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40c9ab0 con 0x7f42a40a6f00 2023-10-06T04:49:33.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.791+0000 7f42a259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c005230 0x7f42a4147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:33.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.791+0000 7f42a259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c005230 0x7f42a4147440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44596/0 (socket says 172.21.15.55:44596) 2023-10-06T04:49:33.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.791+0000 7f42a259c700 1 -- 172.21.15.55:0/2599777014 learned_addr learned my addr 172.21.15.55:0/2599777014 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:33.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.791+0000 7f42a259c700 1 -- 172.21.15.55:0/2599777014 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42a40a6f00 msgr2=0x7f42a40a7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:33.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.791+0000 7f42a259c700 1 --2- 172.21.15.55:0/2599777014 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42a40a6f00 0x7f42a40a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.791+0000 7f42a259c700 1 -- 172.21.15.55:0/2599777014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42a40c8560 msgr2=0x7f42a40c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:33.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.791+0000 7f42a259c700 1 --2- 172.21.15.55:0/2599777014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42a40c8560 0x7f42a40c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:33.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.791+0000 7f42a259c700 1 -- 172.21.15.55:0/2599777014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a40c9d90 con 0x7f428c005230 2023-10-06T04:49:33.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.792+0000 7f42a259c700 1 --2- 172.21.15.55:0/2599777014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c005230 0x7f42a4147440 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f4294002b30 tx=0x7f4294007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:33.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.792+0000 7f42927fc700 1 -- 172.21.15.55:0/2599777014 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294006a30 con 0x7f428c005230 2023-10-06T04:49:33.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.792+0000 7f42927fc700 1 -- 172.21.15.55:0/2599777014 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4294006b90 con 0x7f428c005230 2023-10-06T04:49:33.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.792+0000 7f42927fc700 1 -- 172.21.15.55:0/2599777014 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294007050 con 0x7f428c005230 2023-10-06T04:49:33.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.792+0000 7f42a359e700 1 -- 172.21.15.55:0/2599777014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a40ca070 con 0x7f428c005230 2023-10-06T04:49:33.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.792+0000 7f42a359e700 1 -- 172.21.15.55:0/2599777014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a40a78b0 con 0x7f428c005230 2023-10-06T04:49:33.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.793+0000 7f42a359e700 1 -- 172.21.15.55:0/2599777014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a4088990 con 0x7f428c005230 2023-10-06T04:49:33.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.794+0000 7f42927fc700 1 -- 172.21.15.55:0/2599777014 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 76) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4294007400 con 0x7f428c005230 2023-10-06T04:49:33.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.795+0000 7f42927fc700 1 --2- 172.21.15.55:0/2599777014 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4284066f30 0x7f42840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:33.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.795+0000 7f42927fc700 1 -- 172.21.15.55:0/2599777014 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f4294084cb0 con 0x7f428c005230 2023-10-06T04:49:33.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.795+0000 7f42a1d9b700 1 --2- 172.21.15.55:0/2599777014 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4284066f30 0x7f42840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:33.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.796+0000 7f42a1d9b700 1 --2- 172.21.15.55:0/2599777014 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4284066f30 0x7f42840693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f42a4068760 tx=0x7f4298001e10 comp rx=0 tx=0).ready entity=mgr.8557 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:33.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:33.799+0000 7f42927fc700 1 -- 172.21.15.55:0/2599777014 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4294051b90 con 0x7f428c005230 2023-10-06T04:49:34.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.058+0000 7f42a359e700 1 -- 172.21.15.55:0/2599777014 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f42a404bb80 con 0x7f428c005230 2023-10-06T04:49:34.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.728+0000 7f42927fc700 1 -- 172.21.15.55:0/2599777014 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 77) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f4294051f60 con 0x7f428c005230 2023-10-06T04:49:34.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.730+0000 7f42927fc700 1 -- 172.21.15.55:0/2599777014 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v77) v1 ==== 92+0+0 (secure 0 0 0) 0x7f4294051b90 con 0x7f428c005230 2023-10-06T04:49:34.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.734+0000 7f42a359e700 1 -- 172.21.15.55:0/2599777014 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4284066f30 msgr2=0x7f42840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:34.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.734+0000 7f42a359e700 1 --2- 172.21.15.55:0/2599777014 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4284066f30 0x7f42840693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f42a4068760 tx=0x7f4298001e10 comp rx=0 tx=0).stop 2023-10-06T04:49:34.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.734+0000 7f42a359e700 1 -- 172.21.15.55:0/2599777014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c005230 msgr2=0x7f42a4147440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:34.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.734+0000 7f42a359e700 1 --2- 172.21.15.55:0/2599777014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c005230 0x7f42a4147440 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f4294002b30 tx=0x7f4294007d30 comp rx=0 tx=0).stop 2023-10-06T04:49:34.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.734+0000 7f42a359e700 1 -- 172.21.15.55:0/2599777014 shutdown_connections 2023-10-06T04:49:34.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.734+0000 7f42a359e700 1 --2- 172.21.15.55:0/2599777014 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f4284066f30 0x7f42840693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:34.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.734+0000 7f42a359e700 1 --2- 172.21.15.55:0/2599777014 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f42a40a6f00 0x7f42a40a7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:34.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.734+0000 7f42a359e700 1 --2- 172.21.15.55:0/2599777014 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f428c005230 0x7f42a4147440 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:34.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.734+0000 7f42a359e700 1 --2- 172.21.15.55:0/2599777014 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f42a40c8560 0x7f42a40c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:34.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.734+0000 7f42a359e700 1 -- 172.21.15.55:0/2599777014 >> 172.21.15.55:0/2599777014 conn(0x7f42a40d10c0 msgr2=0x7f42a40b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:34.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.734+0000 7f42a359e700 1 -- 172.21.15.55:0/2599777014 shutdown_connections 2023-10-06T04:49:34.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.734+0000 7f42a359e700 1 -- 172.21.15.55:0/2599777014 wait complete. 2023-10-06T04:49:34.756 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T04:49:34.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.953+0000 7f14f339d700 1 Processor -- start 2023-10-06T04:49:34.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.953+0000 7f14f339d700 1 -- start start 2023-10-06T04:49:34.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.955+0000 7f14f339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14ec159a60 0x7f14ec0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:34.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.955+0000 7f14f339d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ec0c9cb0 con 0x7f14ec0ccc00 2023-10-06T04:49:34.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.955+0000 7f14f339d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ec0c9f30 con 0x7f14ec0cb230 2023-10-06T04:49:34.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.955+0000 7f14f339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec06dfc0 con 0x7f14ec159a60 2023-10-06T04:49:34.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.955+0000 7f14f1b9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f14ec0ccc00 0x7f14ec069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56586/0 (socket says 172.21.15.55:56586) 2023-10-06T04:49:34.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.955+0000 7f14f1b9a700 1 -- 172.21.15.55:0/133665505 learned_addr learned my addr 172.21.15.55:0/133665505 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:34.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.955+0000 7f14f239b700 1 --2- 172.21.15.55:0/133665505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14ec159a60 0x7f14ec0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:34.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.956+0000 7f14f1399700 1 -- 172.21.15.55:0/133665505 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345126562 0 0) 0x7f14ec0c9cb0 con 0x7f14ec0ccc00 2023-10-06T04:49:34.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.956+0000 7f14f1399700 1 -- 172.21.15.55:0/133665505 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14d8003570 con 0x7f14ec0ccc00 2023-10-06T04:49:34.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.956+0000 7f14f1399700 1 -- 172.21.15.55:0/133665505 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290220514 0 0) 0x7f14ec0c9f30 con 0x7f14ec0cb230 2023-10-06T04:49:34.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.956+0000 7f14f1399700 1 -- 172.21.15.55:0/133665505 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14ec0c9cb0 con 0x7f14ec0cb230 2023-10-06T04:49:34.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.956+0000 7f14f239b700 1 -- 172.21.15.55:0/133665505 >> v1:172.21.15.163:6789/0 conn(0x7f14ec0cb230 legacy=0x7f14ec06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:34.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.956+0000 7f14f239b700 1 -- 172.21.15.55:0/133665505 >> v1:172.21.15.55:6789/0 conn(0x7f14ec0ccc00 legacy=0x7f14ec069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:34.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.956+0000 7f14f239b700 1 -- 172.21.15.55:0/133665505 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec0ca470 con 0x7f14ec159a60 2023-10-06T04:49:34.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.956+0000 7f14f239b700 1 --2- 172.21.15.55:0/133665505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14ec159a60 0x7f14ec0c9770 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009910 tx=0x7f14e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8255b57408a320c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:34.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.956+0000 7f14f1399700 1 -- 172.21.15.55:0/133665505 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e800b690 con 0x7f14ec159a60 2023-10-06T04:49:34.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.956+0000 7f14f1399700 1 -- 172.21.15.55:0/133665505 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e800b7f0 con 0x7f14ec159a60 2023-10-06T04:49:34.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.956+0000 7f14f1399700 1 -- 172.21.15.55:0/133665505 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e800bb90 con 0x7f14ec159a60 2023-10-06T04:49:34.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.957+0000 7f14f339d700 1 -- 172.21.15.55:0/133665505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14ec159a60 msgr2=0x7f14ec0c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:34.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.957+0000 7f14f339d700 1 --2- 172.21.15.55:0/133665505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14ec159a60 0x7f14ec0c9770 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009910 tx=0x7f14e80091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:34.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.957+0000 7f14f339d700 1 -- 172.21.15.55:0/133665505 shutdown_connections 2023-10-06T04:49:34.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.957+0000 7f14f339d700 1 --2- 172.21.15.55:0/133665505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14ec159a60 0x7f14ec0c9770 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:34.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.957+0000 7f14f339d700 1 -- 172.21.15.55:0/133665505 >> 172.21.15.55:0/133665505 conn(0x7f14ec06baa0 msgr2=0x7f14ec06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:34.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.957+0000 7f14f339d700 1 -- 172.21.15.55:0/133665505 shutdown_connections 2023-10-06T04:49:34.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.957+0000 7f14f339d700 1 -- 172.21.15.55:0/133665505 wait complete. 2023-10-06T04:49:34.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.959+0000 7f14f339d700 1 Processor -- start 2023-10-06T04:49:34.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.959+0000 7f14f339d700 1 -- start start 2023-10-06T04:49:34.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f339d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14ec0cb230 0x7f14ec155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:34.966 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14ec0ccc00 0x7f14ec15bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:34.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f339d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14ec159a60 0x7f14ec15c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:34.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f339d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec0b4280 con 0x7f14ec0cb230 2023-10-06T04:49:34.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f339d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec15afc0 con 0x7f14ec159a60 2023-10-06T04:49:34.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec0b3fa0 con 0x7f14ec0ccc00 2023-10-06T04:49:34.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f239b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14ec0cb230 0x7f14ec155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:34.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f239b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14ec0cb230 0x7f14ec155440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44614/0 (socket says 172.21.15.55:44614) 2023-10-06T04:49:34.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f239b700 1 -- 172.21.15.55:0/3040479520 learned_addr learned my addr 172.21.15.55:0/3040479520 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:34.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f1b9a700 1 --2- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14ec0ccc00 0x7f14ec15bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:34.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f2b9c700 1 --2- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14ec159a60 0x7f14ec15c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:34.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f239b700 1 -- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14ec0ccc00 msgr2=0x7f14ec15bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:34.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f239b700 1 --2- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14ec0ccc00 0x7f14ec15bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:34.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f239b700 1 -- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14ec159a60 msgr2=0x7f14ec15c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:34.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.960+0000 7f14f239b700 1 --2- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14ec159a60 0x7f14ec15c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:34.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.961+0000 7f14f239b700 1 -- 172.21.15.55:0/3040479520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e8009420 con 0x7f14ec0cb230 2023-10-06T04:49:34.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.961+0000 7f14f239b700 1 --2- 172.21.15.55:0/3040479520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14ec0cb230 0x7f14ec155440 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009f00 tx=0x7f14e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:34.968 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.961+0000 7f14e27fc700 1 -- 172.21.15.55:0/3040479520 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e801f070 con 0x7f14ec0cb230 2023-10-06T04:49:34.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.961+0000 7f14e27fc700 1 -- 172.21.15.55:0/3040479520 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e8000bc0 con 0x7f14ec0cb230 2023-10-06T04:49:34.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.961+0000 7f14f339d700 1 -- 172.21.15.55:0/3040479520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec15a970 con 0x7f14ec0cb230 2023-10-06T04:49:34.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.961+0000 7f14e27fc700 1 -- 172.21.15.55:0/3040479520 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8025420 con 0x7f14ec0cb230 2023-10-06T04:49:34.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.961+0000 7f14f339d700 1 -- 172.21.15.55:0/3040479520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec155980 con 0x7f14ec0cb230 2023-10-06T04:49:34.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.962+0000 7f14f339d700 1 -- 172.21.15.55:0/3040479520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec06eba0 con 0x7f14ec0cb230 2023-10-06T04:49:34.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.963+0000 7f14e27fc700 1 -- 172.21.15.55:0/3040479520 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 77) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f14e8023070 con 0x7f14ec0cb230 2023-10-06T04:49:34.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.964+0000 7f14e27fc700 1 --2- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f14d0066f80 0x7f14d0069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:34.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.964+0000 7f14e27fc700 1 -- 172.21.15.55:0/3040479520 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f14e8085bb0 con 0x7f14ec0cb230 2023-10-06T04:49:34.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.964+0000 7f14f1b9a700 1 -- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f14d0066f80 msgr2=0x7f14d0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:34.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.964+0000 7f14f1b9a700 1 --2- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f14d0066f80 0x7f14d0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:34.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:34.968+0000 7f14e27fc700 1 -- 172.21.15.55:0/3040479520 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14e8052b40 con 0x7f14ec0cb230 2023-10-06T04:49:35.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.165+0000 7f14f1b9a700 1 -- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f14d0066f80 msgr2=0x7f14d0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:35.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.165+0000 7f14f1b9a700 1 --2- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f14d0066f80 0x7f14d0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:35.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.174+0000 7f14f339d700 1 -- 172.21.15.55:0/3040479520 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f14ec0ca750 con 0x7f14ec0cb230 2023-10-06T04:49:35.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.178+0000 7f14e27fc700 1 -- 172.21.15.55:0/3040479520 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v59)=0 v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7f14e80150d0 con 0x7f14ec0cb230 2023-10-06T04:49:35.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.181+0000 7f14f339d700 1 -- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f14d0066f80 msgr2=0x7f14d0069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:35.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.181+0000 7f14f339d700 1 --2- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f14d0066f80 0x7f14d0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:35.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.181+0000 7f14f339d700 1 -- 172.21.15.55:0/3040479520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14ec0cb230 msgr2=0x7f14ec155440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:35.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.181+0000 7f14f339d700 1 --2- 172.21.15.55:0/3040479520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14ec0cb230 0x7f14ec155440 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f14e8009f00 tx=0x7f14e8017000 comp rx=0 tx=0).stop 2023-10-06T04:49:35.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.181+0000 7f14f339d700 1 -- 172.21.15.55:0/3040479520 shutdown_connections 2023-10-06T04:49:35.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.181+0000 7f14f339d700 1 --2- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f14d0066f80 0x7f14d0069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:35.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.181+0000 7f14f339d700 1 --2- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f14ec159a60 0x7f14ec15c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:35.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.181+0000 7f14f339d700 1 --2- 172.21.15.55:0/3040479520 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f14ec0cb230 0x7f14ec155440 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:35.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.181+0000 7f14f339d700 1 --2- 172.21.15.55:0/3040479520 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f14ec0ccc00 0x7f14ec15bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:35.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.181+0000 7f14f339d700 1 -- 172.21.15.55:0/3040479520 >> 172.21.15.55:0/3040479520 conn(0x7f14ec06baa0 msgr2=0x7f14ec06a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:35.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.181+0000 7f14f339d700 1 -- 172.21.15.55:0/3040479520 shutdown_connections 2023-10-06T04:49:35.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.181+0000 7f14f339d700 1 -- 172.21.15.55:0/3040479520 wait complete. 2023-10-06T04:49:35.201 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T04:49:35.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.392+0000 7fa05359e700 1 Processor -- start 2023-10-06T04:49:35.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.392+0000 7fa05359e700 1 -- start start 2023-10-06T04:49:35.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.393+0000 7fa05359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0540b16d0 0x7fa0540b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:35.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.393+0000 7fa05359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0540b2080 con 0x7fa0540b33b0 2023-10-06T04:49:35.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.393+0000 7fa05359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0540cfb00 con 0x7fa0540b4e70 2023-10-06T04:49:35.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.393+0000 7fa05359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0540b5c40 con 0x7fa0540b16d0 2023-10-06T04:49:35.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.394+0000 7fa052d9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa0540b33b0 0x7fa0540b37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56596/0 (socket says 172.21.15.55:56596) 2023-10-06T04:49:35.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.394+0000 7fa052d9d700 1 -- 172.21.15.55:0/610230683 learned_addr learned my addr 172.21.15.55:0/610230683 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:35.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.394+0000 7fa051d9b700 1 --2- 172.21.15.55:0/610230683 >> v2:172.21.15.163:3300/0 conn(0x7fa0540b16d0 0x7fa0540b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:35.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.394+0000 7fa05259c700 1 --1- 172.21.15.55:0/610230683 >> v1:172.21.15.163:6789/0 conn(0x7fa0540b4e70 0x7fa0540ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:57976/0 (socket says 172.21.15.55:57976) 2023-10-06T04:49:35.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.394+0000 7fa05159a700 1 -- 172.21.15.55:0/610230683 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111453167 0 0) 0x7fa0540cfb00 con 0x7fa0540b4e70 2023-10-06T04:49:35.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.394+0000 7fa05159a700 1 -- 172.21.15.55:0/610230683 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa03c003570 con 0x7fa0540b4e70 2023-10-06T04:49:35.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.394+0000 7fa051d9b700 1 -- 172.21.15.55:0/610230683 >> v1:172.21.15.163:6789/0 conn(0x7fa0540b4e70 legacy=0x7fa0540ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:35.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.394+0000 7fa051d9b700 1 -- 172.21.15.55:0/610230683 >> v1:172.21.15.55:6789/0 conn(0x7fa0540b33b0 legacy=0x7fa0540b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:35.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.394+0000 7fa05159a700 1 -- 172.21.15.55:0/610230683 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396858675 0 0) 0x7fa0540b2080 con 0x7fa0540b33b0 2023-10-06T04:49:35.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.394+0000 7fa051d9b700 1 -- 172.21.15.55:0/610230683 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0540b5f20 con 0x7fa0540b16d0 2023-10-06T04:49:35.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.395+0000 7fa051d9b700 1 --2- 172.21.15.55:0/610230683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540b16d0 0x7fa0540b1b40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa0480098e0 tx=0x7fa0480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=280e27755248f334 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:35.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.395+0000 7fa05159a700 1 -- 172.21.15.55:0/610230683 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04800b640 con 0x7fa0540b16d0 2023-10-06T04:49:35.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.395+0000 7fa05159a700 1 -- 172.21.15.55:0/610230683 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04800b7a0 con 0x7fa0540b16d0 2023-10-06T04:49:35.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.395+0000 7fa05159a700 1 -- 172.21.15.55:0/610230683 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04800bb40 con 0x7fa0540b16d0 2023-10-06T04:49:35.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.395+0000 7fa05359e700 1 -- 172.21.15.55:0/610230683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540b16d0 msgr2=0x7fa0540b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:35.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.395+0000 7fa05359e700 1 --2- 172.21.15.55:0/610230683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540b16d0 0x7fa0540b1b40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa0480098e0 tx=0x7fa0480091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:35.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.395+0000 7fa05359e700 1 -- 172.21.15.55:0/610230683 shutdown_connections 2023-10-06T04:49:35.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.395+0000 7fa05359e700 1 --2- 172.21.15.55:0/610230683 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540b16d0 0x7fa0540b1b40 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:35.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.395+0000 7fa05359e700 1 -- 172.21.15.55:0/610230683 >> 172.21.15.55:0/610230683 conn(0x7fa0540d10c0 msgr2=0x7fa0540affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:35.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.396+0000 7fa05359e700 1 -- 172.21.15.55:0/610230683 shutdown_connections 2023-10-06T04:49:35.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.396+0000 7fa05359e700 1 -- 172.21.15.55:0/610230683 wait complete. 2023-10-06T04:49:35.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.397+0000 7fa05359e700 1 Processor -- start 2023-10-06T04:49:35.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.397+0000 7fa05359e700 1 -- start start 2023-10-06T04:49:35.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.398+0000 7fa05359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540b16d0 0x7fa0541477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:35.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.398+0000 7fa05359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0540b33b0 0x7fa0540a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:35.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.398+0000 7fa05359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0540b4e70 0x7fa0540a7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:35.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.398+0000 7fa05359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540c8460 con 0x7fa0540b4e70 2023-10-06T04:49:35.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.398+0000 7fa05359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540c8740 con 0x7fa0540b16d0 2023-10-06T04:49:35.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.398+0000 7fa05359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0540c8a20 con 0x7fa0540b33b0 2023-10-06T04:49:35.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.398+0000 7fa05259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540b16d0 0x7fa0541477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:35.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.398+0000 7fa052d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0540b4e70 0x7fa0540a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:35.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.398+0000 7fa051d9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0540b33b0 0x7fa0540a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:35.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.398+0000 7fa052d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0540b4e70 0x7fa0540a7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44616/0 (socket says 172.21.15.55:44616) 2023-10-06T04:49:35.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.398+0000 7fa05259c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540b16d0 0x7fa0541477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:33526/0 (socket says 172.21.15.55:33526) 2023-10-06T04:49:35.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.398+0000 7fa051d9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0540b33b0 0x7fa0540a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:36290/0 (socket says 172.21.15.55:36290) 2023-10-06T04:49:35.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.398+0000 7fa052d9d700 1 -- 172.21.15.55:0/613014170 learned_addr learned my addr 172.21.15.55:0/613014170 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:35.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.399+0000 7fa052d9d700 1 -- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0540b33b0 msgr2=0x7fa0540a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:35.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.399+0000 7fa052d9d700 1 --2- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0540b33b0 0x7fa0540a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:35.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.399+0000 7fa052d9d700 1 -- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540b16d0 msgr2=0x7fa0541477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:35.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.399+0000 7fa052d9d700 1 --2- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540b16d0 0x7fa0541477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:35.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.399+0000 7fa052d9d700 1 -- 172.21.15.55:0/613014170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa048009420 con 0x7fa0540b4e70 2023-10-06T04:49:35.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.399+0000 7fa051d9b700 1 --2- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0540b33b0 0x7fa0540a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:49:35.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.399+0000 7fa052d9d700 1 --2- 172.21.15.55:0/613014170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0540b4e70 0x7fa0540a7440 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00cc90 tx=0x7fa04c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:35.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.399+0000 7fa0427fc700 1 -- 172.21.15.55:0/613014170 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c00a850 con 0x7fa0540b4e70 2023-10-06T04:49:35.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.399+0000 7fa0427fc700 1 -- 172.21.15.55:0/613014170 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04c00a9b0 con 0x7fa0540b4e70 2023-10-06T04:49:35.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.399+0000 7fa05359e700 1 -- 172.21.15.55:0/613014170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0540c8d00 con 0x7fa0540b4e70 2023-10-06T04:49:35.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.399+0000 7fa0427fc700 1 -- 172.21.15.55:0/613014170 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04c00ad70 con 0x7fa0540b4e70 2023-10-06T04:49:35.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.400+0000 7fa05359e700 1 -- 172.21.15.55:0/613014170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0540c7710 con 0x7fa0540b4e70 2023-10-06T04:49:35.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.401+0000 7fa05359e700 1 -- 172.21.15.55:0/613014170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0540b4660 con 0x7fa0540b4e70 2023-10-06T04:49:35.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.403+0000 7fa0427fc700 1 -- 172.21.15.55:0/613014170 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 77) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fa04c007300 con 0x7fa0540b4e70 2023-10-06T04:49:35.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.404+0000 7fa0427fc700 1 --2- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa034066f80 0x7fa034069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:35.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.404+0000 7fa0427fc700 1 -- 172.21.15.55:0/613014170 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fa04c084b40 con 0x7fa0540b4e70 2023-10-06T04:49:35.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.404+0000 7fa05259c700 1 -- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa034066f80 msgr2=0x7fa034069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:35.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.404+0000 7fa05259c700 1 --2- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa034066f80 0x7fa034069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:35.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.408+0000 7fa0427fc700 1 -- 172.21.15.55:0/613014170 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa04c051b40 con 0x7fa0540b4e70 2023-10-06T04:49:35.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.605+0000 7fa05259c700 1 -- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa034066f80 msgr2=0x7fa034069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:35.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.605+0000 7fa05259c700 1 --2- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa034066f80 0x7fa034069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:35.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.610+0000 7fa05359e700 1 -- 172.21.15.55:0/613014170 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fa05404bb80 con 0x7fa0540b4e70 2023-10-06T04:49:35.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.618+0000 7fa0427fc700 1 -- 172.21.15.55:0/613014170 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v60)=0 v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa04c055160 con 0x7fa0540b4e70 2023-10-06T04:49:35.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.621+0000 7fa05359e700 1 -- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa034066f80 msgr2=0x7fa034069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:35.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.621+0000 7fa05359e700 1 --2- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa034066f80 0x7fa034069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:35.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.621+0000 7fa05359e700 1 -- 172.21.15.55:0/613014170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0540b4e70 msgr2=0x7fa0540a7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:35.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.621+0000 7fa05359e700 1 --2- 172.21.15.55:0/613014170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0540b4e70 0x7fa0540a7440 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00cc90 tx=0x7fa04c001a80 comp rx=0 tx=0).stop 2023-10-06T04:49:35.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.621+0000 7fa05359e700 1 -- 172.21.15.55:0/613014170 shutdown_connections 2023-10-06T04:49:35.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.621+0000 7fa05359e700 1 --2- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fa034066f80 0x7fa034069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:35.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.621+0000 7fa05359e700 1 --2- 172.21.15.55:0/613014170 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0540b4e70 0x7fa0540a7440 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:35.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.621+0000 7fa05359e700 1 --2- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0540b16d0 0x7fa0541477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:35.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.621+0000 7fa05359e700 1 --2- 172.21.15.55:0/613014170 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0540b33b0 0x7fa0540a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:35.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.621+0000 7fa05359e700 1 -- 172.21.15.55:0/613014170 >> 172.21.15.55:0/613014170 conn(0x7fa0540d10c0 msgr2=0x7fa05415c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:35.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.621+0000 7fa05359e700 1 -- 172.21.15.55:0/613014170 shutdown_connections 2023-10-06T04:49:35.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.621+0000 7fa05359e700 1 -- 172.21.15.55:0/613014170 wait complete. 2023-10-06T04:49:35.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:49:35.641 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T04:49:35.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:49:35.683 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-10-06T04:49:35.762 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/test_restart 1M 2023-10-06T04:49:35.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.996+0000 7f5c71f9d700 1 Processor -- start 2023-10-06T04:49:35.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.996+0000 7f5c71f9d700 1 -- start start 2023-10-06T04:49:35.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.997+0000 7f5c71f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c6c0b03c0 0x7f5c6c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:35.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.997+0000 7f5c71f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5c6c0b0790 0x7f5c6c0b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:35.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.997+0000 7f5c71f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6c0cfa00 con 0x7f5c6c1591e0 2023-10-06T04:49:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.997+0000 7f5c71f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6c0cb580 con 0x7f5c6c0b0790 2023-10-06T04:49:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.997+0000 7f5c71f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6c0b2de0 con 0x7f5c6c0b03c0 2023-10-06T04:49:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.997+0000 7f5c6bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5c6c0b0790 0x7f5c6c0b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.997+0000 7f5c7179c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5c6c1591e0 0x7f5c6c159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:56604/0 (socket says 172.21.15.55:56604) 2023-10-06T04:49:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.997+0000 7f5c7179c700 1 -- 172.21.15.55:0/228970263 learned_addr learned my addr 172.21.15.55:0/228970263 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T04:49:36.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.998+0000 7f5c70f9b700 1 --2- 172.21.15.55:0/228970263 >> v2:172.21.15.163:3300/0 conn(0x7f5c6c0b03c0 0x7f5c6c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:36.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.998+0000 7f5c6b7fe700 1 -- 172.21.15.55:0/228970263 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171604641 0 0) 0x7f5c6c0cfa00 con 0x7f5c6c1591e0 2023-10-06T04:49:36.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.998+0000 7f5c6bfff700 1 -- 172.21.15.55:0/228970263 >> v2:172.21.15.163:3300/0 conn(0x7f5c6c0b03c0 msgr2=0x7f5c6c0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:36.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.998+0000 7f5c6bfff700 1 --2- 172.21.15.55:0/228970263 >> v2:172.21.15.163:3300/0 conn(0x7f5c6c0b03c0 0x7f5c6c0cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:36.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.998+0000 7f5c6bfff700 1 -- 172.21.15.55:0/228970263 >> v1:172.21.15.55:6789/0 conn(0x7f5c6c1591e0 legacy=0x7f5c6c159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:36.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.998+0000 7f5c6bfff700 1 -- 172.21.15.55:0/228970263 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c0cf160 con 0x7f5c6c0b0790 2023-10-06T04:49:36.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.998+0000 7f5c6bfff700 1 --2- 172.21.15.55:0/228970263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c6c0b0790 0x7f5c6c0b28a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f5c580098e0 tx=0x7f5c580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b268d65f381652b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:36.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.999+0000 7f5c6b7fe700 1 -- 172.21.15.55:0/228970263 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c58011630 con 0x7f5c6c0b0790 2023-10-06T04:49:36.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.999+0000 7f5c6b7fe700 1 -- 172.21.15.55:0/228970263 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c58011790 con 0x7f5c6c0b0790 2023-10-06T04:49:36.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.999+0000 7f5c6b7fe700 1 -- 172.21.15.55:0/228970263 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c58011b30 con 0x7f5c6c0b0790 2023-10-06T04:49:36.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.999+0000 7f5c71f9d700 1 -- 172.21.15.55:0/228970263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c6c0b0790 msgr2=0x7f5c6c0b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:36.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:35.999+0000 7f5c71f9d700 1 --2- 172.21.15.55:0/228970263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c6c0b0790 0x7f5c6c0b28a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f5c580098e0 tx=0x7f5c580091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:36.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.000+0000 7f5c71f9d700 1 -- 172.21.15.55:0/228970263 shutdown_connections 2023-10-06T04:49:36.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.000+0000 7f5c71f9d700 1 --2- 172.21.15.55:0/228970263 >> v2:172.21.15.163:3300/0 conn(0x7f5c6c0b03c0 0x7f5c6c0cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:36.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.000+0000 7f5c71f9d700 1 --2- 172.21.15.55:0/228970263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c6c0b0790 0x7f5c6c0b28a0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:36.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.000+0000 7f5c71f9d700 1 -- 172.21.15.55:0/228970263 >> 172.21.15.55:0/228970263 conn(0x7f5c6c063230 msgr2=0x7f5c6c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:36.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.000+0000 7f5c71f9d700 1 -- 172.21.15.55:0/228970263 shutdown_connections 2023-10-06T04:49:36.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.000+0000 7f5c71f9d700 1 -- 172.21.15.55:0/228970263 wait complete. 2023-10-06T04:49:36.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.002+0000 7f5c71f9d700 1 Processor -- start 2023-10-06T04:49:36.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.002+0000 7f5c71f9d700 1 -- start start 2023-10-06T04:49:36.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.004+0000 7f5c71f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c6c0b03c0 0x7f5c6c16b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:36.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.004+0000 7f5c71f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c6c0b0790 0x7f5c6c16c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:36.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.004+0000 7f5c71f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c1591e0 0x7f5c6c068420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:36.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.004+0000 7f5c71f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c16b900 con 0x7f5c6c0b03c0 2023-10-06T04:49:36.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.004+0000 7f5c71f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c16bbe0 con 0x7f5c6c0b0790 2023-10-06T04:49:36.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.004+0000 7f5c71f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c16bec0 con 0x7f5c6c1591e0 2023-10-06T04:49:36.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.004+0000 7f5c70f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c6c0b03c0 0x7f5c6c16b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:36.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.004+0000 7f5c70f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c6c0b03c0 0x7f5c6c16b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44640/0 (socket says 172.21.15.55:44640) 2023-10-06T04:49:36.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.004+0000 7f5c70f9b700 1 -- 172.21.15.55:0/3321617322 learned_addr learned my addr 172.21.15.55:0/3321617322 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:36.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.004+0000 7f5c6bfff700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c6c0b0790 0x7f5c6c16c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:36.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.004+0000 7f5c7179c700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c1591e0 0x7f5c6c068420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:36.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.005+0000 7f5c70f9b700 1 -- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c1591e0 msgr2=0x7f5c6c068420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:36.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.005+0000 7f5c70f9b700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c1591e0 0x7f5c6c068420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:36.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.005+0000 7f5c70f9b700 1 -- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c6c0b0790 msgr2=0x7f5c6c16c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:36.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.005+0000 7f5c70f9b700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c6c0b0790 0x7f5c6c16c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:36.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.005+0000 7f5c70f9b700 1 -- 172.21.15.55:0/3321617322 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c58009420 con 0x7f5c6c0b03c0 2023-10-06T04:49:36.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.005+0000 7f5c7179c700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c1591e0 0x7f5c6c068420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:49:36.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.005+0000 7f5c70f9b700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c6c0b03c0 0x7f5c6c16b270 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000ec80 tx=0x7f5c600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:36.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.005+0000 7f5c68ff9700 1 -- 172.21.15.55:0/3321617322 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000c850 con 0x7f5c6c0b03c0 2023-10-06T04:49:36.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.005+0000 7f5c68ff9700 1 -- 172.21.15.55:0/3321617322 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6000c9b0 con 0x7f5c6c0b03c0 2023-10-06T04:49:36.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.005+0000 7f5c68ff9700 1 -- 172.21.15.55:0/3321617322 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000cd50 con 0x7f5c6c0b03c0 2023-10-06T04:49:36.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.005+0000 7f5c71f9d700 1 -- 172.21.15.55:0/3321617322 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c16c1a0 con 0x7f5c6c0b03c0 2023-10-06T04:49:36.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.005+0000 7f5c71f9d700 1 -- 172.21.15.55:0/3321617322 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c16cee0 con 0x7f5c6c0b03c0 2023-10-06T04:49:36.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.006+0000 7f5c71f9d700 1 -- 172.21.15.55:0/3321617322 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c04c3c0 con 0x7f5c6c0b03c0 2023-10-06T04:49:36.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.007+0000 7f5c68ff9700 1 -- 172.21.15.55:0/3321617322 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 77) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f5c60013680 con 0x7f5c6c0b03c0 2023-10-06T04:49:36.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.008+0000 7f5c68ff9700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 0x7f5c50069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:36.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.008+0000 7f5c68ff9700 1 -- 172.21.15.55:0/3321617322 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f5c60084be0 con 0x7f5c6c0b03c0 2023-10-06T04:49:36.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.009+0000 7f5c6bfff700 1 -- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 msgr2=0x7f5c50069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:36.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.009+0000 7f5c6bfff700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 0x7f5c50069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:49:36.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.012+0000 7f5c68ff9700 1 -- 172.21.15.55:0/3321617322 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c60051ac0 con 0x7f5c6c0b03c0 2023-10-06T04:49:36.029 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:36.028+0000 7f7356342180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T04:49:36.089 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:36.087+0000 7f933ba9b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T04:49:36.193 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:36.192+0000 7f7356342180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T04:49:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.210+0000 7f5c6bfff700 1 -- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 msgr2=0x7f5c50069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:36.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.210+0000 7f5c6bfff700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 0x7f5c50069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T04:49:36.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.244+0000 7f5c71f9d700 1 -- 172.21.15.55:0/3321617322 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c6c16d190 con 0x7f5c50066f80 2023-10-06T04:49:36.259 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:36.258+0000 7f933ba9b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T04:49:36.279 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:36.278+0000 7f7356342180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T04:49:36.349 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:36.348+0000 7f933ba9b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T04:49:36.508 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:36.507+0000 7f7356342180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:49:36.580 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:36.578+0000 7f933ba9b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:49:36.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.611+0000 7f5c6bfff700 1 -- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 msgr2=0x7f5c50069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:36.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:36.611+0000 7f5c6bfff700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 0x7f5c50069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T04:49:36.685 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:36.684+0000 7f7356342180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T04:49:36.765 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:36.763+0000 7f933ba9b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T04:49:36.780 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:36.779+0000 7f7356342180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T04:49:36.861 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:36.859+0000 7f933ba9b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T04:49:36.943 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:36.941+0000 7f7356342180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T04:49:37.027 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:37.025+0000 7f933ba9b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T04:49:37.043 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:37.042+0000 7f7356342180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T04:49:37.131 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:37.129+0000 7f933ba9b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T04:49:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:37.412+0000 7f5c6bfff700 1 -- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 msgr2=0x7f5c50069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:37.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:37.412+0000 7f5c6bfff700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 0x7f5c50069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T04:49:37.528 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:37.527+0000 7f7356342180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T04:49:37.623 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:37.621+0000 7f7356342180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T04:49:37.633 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:37.631+0000 7f933ba9b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T04:49:37.714 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:37.712+0000 7f7356342180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T04:49:37.731 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:37.729+0000 7f933ba9b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T04:49:37.825 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:37.823+0000 7f933ba9b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T04:49:38.337 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:38.336+0000 7f7356342180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T04:49:38.469 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:38.467+0000 7f933ba9b180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T04:49:38.509 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:38.508+0000 7f7356342180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:49:38.606 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:38.605+0000 7f7356342180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T04:49:38.647 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:38.645+0000 7f933ba9b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:49:38.746 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:38.744+0000 7f933ba9b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T04:49:38.901 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:38.900+0000 7f7356342180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T04:49:38.998 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:38.997+0000 7f7356342180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T04:49:39.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:39.014+0000 7f5c6bfff700 1 -- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 msgr2=0x7f5c50069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/85191 2023-10-06T04:49:39.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:39.014+0000 7f5c6bfff700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 0x7f5c50069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T04:49:39.047 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:39.045+0000 7f933ba9b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T04:49:39.147 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:39.145+0000 7f933ba9b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T04:49:39.699 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:39.697+0000 7f7356342180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T04:49:39.880 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:39.878+0000 7f933ba9b180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T04:49:39.984 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:39.983+0000 7f7356342180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T04:49:40.076 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.074+0000 7f7356342180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T04:49:40.172 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:40.170+0000 7f933ba9b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T04:49:40.227 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.226+0000 7f7356342180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T04:49:40.268 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:40.267+0000 7f933ba9b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T04:49:40.331 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.329+0000 7f7356342180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T04:49:40.429 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:40.427+0000 7f933ba9b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T04:49:40.519 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.518+0000 7f7356342180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T04:49:40.537 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:40.535+0000 7f933ba9b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T04:49:40.611 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.610+0000 7f7356342180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T04:49:40.702 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.701+0000 7f7356342180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T04:49:40.735 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:40.733+0000 7f933ba9b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T04:49:40.796 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.794+0000 7f7356342180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T04:49:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:40.807+0000 7f5c68ff9700 1 -- 172.21.15.55:0/3321617322 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 78) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f5c60051e90 con 0x7f5c6c0b03c0 2023-10-06T04:49:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:40.808+0000 7f5c68ff9700 1 -- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 msgr2=0x7f5c50069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:49:40.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:40.808+0000 7f5c68ff9700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 0x7f5c50069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:40.836 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:40.835+0000 7f933ba9b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T04:49:40.938 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.936+0000 7f72c6e6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:40.938 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.936+0000 7f72c6e6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:40.938 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.936+0000 7f72c6e6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:40.939 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.936+0000 7f72c6e6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:40.939 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.936+0000 7f72c6e6e700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:40.939 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.937+0000 7f72cb677700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:40.939 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.937+0000 7f72cb677700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:40.939 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.937+0000 7f72cb677700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:40.939 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.937+0000 7f72cb677700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:40.939 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:49:40.937+0000 7f72cb677700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:49:40.941 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:40.940+0000 7f933ba9b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T04:49:41.038 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T04:49:41.037+0000 7f933ba9b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T04:49:41.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.813+0000 7f5c68ff9700 1 -- 172.21.15.55:0/3321617322 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mgrmap(e 79) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f5c60053530 con 0x7f5c6c0b03c0 2023-10-06T04:49:41.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.813+0000 7f5c68ff9700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 0x7f5c50069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:41.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.813+0000 7f5c68ff9700 1 -- 172.21.15.55:0/3321617322 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c6c16d190 con 0x7f5c50066f80 2023-10-06T04:49:41.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.814+0000 7f5c6bfff700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 0x7f5c50069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:41.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.818+0000 7f5c6bfff700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 0x7f5c50069430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800a010 tx=0x7f5c58017000 comp rx=0 tx=0).ready entity=mgr.7808 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:41.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.889+0000 7f5c68ff9700 1 -- 172.21.15.55:0/3321617322 <== mgr.7808 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f5c6c16d190 con 0x7f5c50066f80 2023-10-06T04:49:41.891 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-10-06T04:49:41.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.892+0000 7f5c71f9d700 1 -- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 msgr2=0x7f5c50069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:41.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.892+0000 7f5c71f9d700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 0x7f5c50069430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5c5800a010 tx=0x7f5c58017000 comp rx=0 tx=0).stop 2023-10-06T04:49:41.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.892+0000 7f5c71f9d700 1 -- 172.21.15.55:0/3321617322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c6c0b03c0 msgr2=0x7f5c6c16b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:41.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.892+0000 7f5c71f9d700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c6c0b03c0 0x7f5c6c16b270 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000ec80 tx=0x7f5c600091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:41.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.892+0000 7f5c71f9d700 1 -- 172.21.15.55:0/3321617322 shutdown_connections 2023-10-06T04:49:41.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.892+0000 7f5c71f9d700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f5c50066f80 0x7f5c50069430 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:41.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.892+0000 7f5c71f9d700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c6c1591e0 0x7f5c6c068420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:41.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.892+0000 7f5c71f9d700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c6c0b03c0 0x7f5c6c16b270 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:41.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.892+0000 7f5c71f9d700 1 --2- 172.21.15.55:0/3321617322 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c6c0b0790 0x7f5c6c16c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:41.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.892+0000 7f5c71f9d700 1 -- 172.21.15.55:0/3321617322 >> 172.21.15.55:0/3321617322 conn(0x7f5c6c063230 msgr2=0x7f5c6c158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:41.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.893+0000 7f5c71f9d700 1 -- 172.21.15.55:0/3321617322 shutdown_connections 2023-10-06T04:49:41.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:41.893+0000 7f5c71f9d700 1 -- 172.21.15.55:0/3321617322 wait complete. 2023-10-06T04:49:41.916 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-10-06T04:49:41.916 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/test_restart 1M 2023-10-06T04:49:42.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.112+0000 7f717cb9d700 1 Processor -- start 2023-10-06T04:49:42.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.112+0000 7f717cb9d700 1 -- start start 2023-10-06T04:49:42.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.113+0000 7f717cb9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7178156350 0x7f7178157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:42.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.113+0000 7f717cb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71780b0760 0x7f71780b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:42.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.113+0000 7f717cb9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71780b1110 con 0x7f7178159f20 2023-10-06T04:49:42.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.113+0000 7f717cb9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71780b0570 con 0x7f7178156350 2023-10-06T04:49:42.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.113+0000 7f717cb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71781578b0 con 0x7f71780b0760 2023-10-06T04:49:42.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.114+0000 7f7176ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71780b0760 0x7f71780b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:42.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.114+0000 7f71777fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7178156350 0x7f7178157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:42.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.114+0000 7f7176ffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71780b0760 0x7f71780b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:36310/0 (socket says 172.21.15.55:36310) 2023-10-06T04:49:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.114+0000 7f7176ffd700 1 -- 172.21.15.55:0/336465214 learned_addr learned my addr 172.21.15.55:0/336465214 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.114+0000 7f7176ffd700 1 -- 172.21.15.55:0/336465214 >> v2:172.21.15.163:3300/0 conn(0x7f7178156350 msgr2=0x7f7178157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:49:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.114+0000 7f7176ffd700 1 --2- 172.21.15.55:0/336465214 >> v2:172.21.15.163:3300/0 conn(0x7f7178156350 0x7f7178157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.114+0000 7f7176ffd700 1 -- 172.21.15.55:0/336465214 >> v1:172.21.15.163:6789/0 conn(0x7f7178159f20 legacy=0x7f717815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.114+0000 7f7176ffd700 1 -- 172.21.15.55:0/336465214 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71780af4d0 con 0x7f71780b0760 2023-10-06T04:49:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.114+0000 7f71777fe700 1 --2- 172.21.15.55:0/336465214 >> v2:172.21.15.163:3300/0 conn(0x7f7178156350 0x7f7178157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:49:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.115+0000 7f7176ffd700 1 --2- 172.21.15.55:0/336465214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71780b0760 0x7f71780b0bd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f716c00bc50 tx=0x7f716c00b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8abc94a81a974a6a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.115+0000 7f71767fc700 1 -- 172.21.15.55:0/336465214 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c00dbf0 con 0x7f71780b0760 2023-10-06T04:49:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.115+0000 7f71767fc700 1 -- 172.21.15.55:0/336465214 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c00dd50 con 0x7f71780b0760 2023-10-06T04:49:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.115+0000 7f71767fc700 1 -- 172.21.15.55:0/336465214 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c0196b0 con 0x7f71780b0760 2023-10-06T04:49:42.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.115+0000 7f717cb9d700 1 -- 172.21.15.55:0/336465214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71780b0760 msgr2=0x7f71780b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:42.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.115+0000 7f717cb9d700 1 --2- 172.21.15.55:0/336465214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71780b0760 0x7f71780b0bd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f716c00bc50 tx=0x7f716c00b4f0 comp rx=0 tx=0).stop 2023-10-06T04:49:42.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.115+0000 7f717cb9d700 1 -- 172.21.15.55:0/336465214 shutdown_connections 2023-10-06T04:49:42.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.115+0000 7f717cb9d700 1 --2- 172.21.15.55:0/336465214 >> v2:172.21.15.163:3300/0 conn(0x7f7178156350 0x7f7178157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:42.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.115+0000 7f717cb9d700 1 --2- 172.21.15.55:0/336465214 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f71780b0760 0x7f71780b0bd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:42.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.115+0000 7f717cb9d700 1 -- 172.21.15.55:0/336465214 >> 172.21.15.55:0/336465214 conn(0x7f7178068e20 msgr2=0x7f71780cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:42.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.116+0000 7f717cb9d700 1 -- 172.21.15.55:0/336465214 shutdown_connections 2023-10-06T04:49:42.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.116+0000 7f717cb9d700 1 -- 172.21.15.55:0/336465214 wait complete. 2023-10-06T04:49:42.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.117+0000 7f717cb9d700 1 Processor -- start 2023-10-06T04:49:42.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.117+0000 7f717cb9d700 1 -- start start 2023-10-06T04:49:42.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.118+0000 7f717cb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71780b0760 0x7f717814edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:42.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.118+0000 7f717cb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7178156350 0x7f71780ab400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:42.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.118+0000 7f717cb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7178159f20 0x7f71780c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:42.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.118+0000 7f717cb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717814fd00 con 0x7f7178156350 2023-10-06T04:49:42.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.118+0000 7f717cb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717814f740 con 0x7f71780b0760 2023-10-06T04:49:42.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.118+0000 7f717cb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717814fa20 con 0x7f7178159f20 2023-10-06T04:49:42.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.118+0000 7f7176ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7178156350 0x7f71780ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:42.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.118+0000 7f7176ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7178156350 0x7f71780ab400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:44646/0 (socket says 172.21.15.55:44646) 2023-10-06T04:49:42.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.118+0000 7f7176ffd700 1 -- 172.21.15.55:0/306346173 learned_addr learned my addr 172.21.15.55:0/306346173 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T04:49:42.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.119+0000 7f7177fff700 1 --2- 172.21.15.55:0/306346173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7178159f20 0x7f71780c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.119+0000 7f71777fe700 1 --2- 172.21.15.55:0/306346173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71780b0760 0x7f717814edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.119+0000 7f7176ffd700 1 -- 172.21.15.55:0/306346173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7178159f20 msgr2=0x7f71780c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.119+0000 7f7176ffd700 1 --2- 172.21.15.55:0/306346173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7178159f20 0x7f71780c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.119+0000 7f7176ffd700 1 -- 172.21.15.55:0/306346173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71780b0760 msgr2=0x7f717814edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.119+0000 7f7176ffd700 1 --2- 172.21.15.55:0/306346173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71780b0760 0x7f717814edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:42.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.119+0000 7f7176ffd700 1 -- 172.21.15.55:0/306346173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7168009420 con 0x7f7178156350 2023-10-06T04:49:42.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.119+0000 7f7176ffd700 1 --2- 172.21.15.55:0/306346173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7178156350 0x7f71780ab400 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f71781579f0 tx=0x7f716c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:42.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.119+0000 7f715f7fe700 1 -- 172.21.15.55:0/306346173 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c010680 con 0x7f7178156350 2023-10-06T04:49:42.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.119+0000 7f715f7fe700 1 -- 172.21.15.55:0/306346173 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c0107e0 con 0x7f7178156350 2023-10-06T04:49:42.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.119+0000 7f717cb9d700 1 -- 172.21.15.55:0/306346173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c00b790 con 0x7f7178156350 2023-10-06T04:49:42.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.120+0000 7f715f7fe700 1 -- 172.21.15.55:0/306346173 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c010b80 con 0x7f7178156350 2023-10-06T04:49:42.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.120+0000 7f717cb9d700 1 -- 172.21.15.55:0/306346173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717806d870 con 0x7f7178156350 2023-10-06T04:49:42.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.121+0000 7f717cb9d700 1 -- 172.21.15.55:0/306346173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717806da00 con 0x7f7178156350 2023-10-06T04:49:42.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.121+0000 7f715f7fe700 1 -- 172.21.15.55:0/306346173 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 79) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f716c02c430 con 0x7f7178156350 2023-10-06T04:49:42.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.123+0000 7f715f7fe700 1 --2- 172.21.15.55:0/306346173 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7164066dd0 0x7f7164069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:49:42.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.123+0000 7f715f7fe700 1 -- 172.21.15.55:0/306346173 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6708+0+0 (secure 0 0 0) 0x7f716c014070 con 0x7f7178156350 2023-10-06T04:49:42.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.123+0000 7f71777fe700 1 --2- 172.21.15.55:0/306346173 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7164066dd0 0x7f7164069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:49:42.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.124+0000 7f71777fe700 1 --2- 172.21.15.55:0/306346173 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7164066dd0 0x7f7164069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7168009ea0 tx=0x7f71680091a0 comp rx=0 tx=0).ready entity=mgr.7808 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:49:42.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.128+0000 7f715f7fe700 1 -- 172.21.15.55:0/306346173 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f716c053400 con 0x7f7178156350 2023-10-06T04:49:42.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.354+0000 7f717cb9d700 1 -- 172.21.15.55:0/306346173 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f717804c3c0 con 0x7f7164066dd0 2023-10-06T04:49:42.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.404+0000 7f715f7fe700 1 -- 172.21.15.55:0/306346173 <== mgr.7808 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f717804c3c0 con 0x7f7164066dd0 2023-10-06T04:49:42.406 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-10-06T04:49:42.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.406+0000 7f717cb9d700 1 -- 172.21.15.55:0/306346173 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7164066dd0 msgr2=0x7f7164069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:42.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.406+0000 7f717cb9d700 1 --2- 172.21.15.55:0/306346173 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7164066dd0 0x7f7164069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7168009ea0 tx=0x7f71680091a0 comp rx=0 tx=0).stop 2023-10-06T04:49:42.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.406+0000 7f717cb9d700 1 -- 172.21.15.55:0/306346173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7178156350 msgr2=0x7f71780ab400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:49:42.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.406+0000 7f717cb9d700 1 --2- 172.21.15.55:0/306346173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7178156350 0x7f71780ab400 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f71781579f0 tx=0x7f716c01b000 comp rx=0 tx=0).stop 2023-10-06T04:49:42.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.406+0000 7f717cb9d700 1 -- 172.21.15.55:0/306346173 shutdown_connections 2023-10-06T04:49:42.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.406+0000 7f717cb9d700 1 --2- 172.21.15.55:0/306346173 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f7164066dd0 0x7f7164069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:42.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.406+0000 7f717cb9d700 1 --2- 172.21.15.55:0/306346173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7178159f20 0x7f71780c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:42.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.406+0000 7f717cb9d700 1 --2- 172.21.15.55:0/306346173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f71780b0760 0x7f717814edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:42.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.406+0000 7f717cb9d700 1 --2- 172.21.15.55:0/306346173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7178156350 0x7f71780ab400 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:49:42.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.406+0000 7f717cb9d700 1 -- 172.21.15.55:0/306346173 >> 172.21.15.55:0/306346173 conn(0x7f7178068e20 msgr2=0x7f7178069ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:49:42.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.406+0000 7f717cb9d700 1 -- 172.21.15.55:0/306346173 shutdown_connections 2023-10-06T04:49:42.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T04:49:42.406+0000 7f717cb9d700 1 -- 172.21.15.55:0/306346173 wait complete. 2023-10-06T04:49:42.425 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-10-06T04:50:00.418 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:50:00.415+0000 7f72ba655700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:50:00.418 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:50:00.415+0000 7f72ba655700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:50:00.418 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:50:00.415+0000 7f72ba655700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:50:00.418 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:50:00.415+0000 7f72ba655700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:50:00.418 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T04:50:00.415+0000 7f72ba655700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T04:50:02.244 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:50:02.249 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:50:32.423 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:50:32.429 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:51:02.529 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:51:02.534 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:51:32.716 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:51:32.722 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:52:02.855 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:52:02.861 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:52:33.044 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:52:33.050 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:53:03.240 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:53:03.246 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:53:33.412 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:53:33.418 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:54:03.576 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:54:03.581 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:54:33.763 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:54:33.769 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:55:03.885 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:55:03.890 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:55:34.024 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:55:34.029 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:56:04.204 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:56:04.209 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:56:34.401 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:56:34.408 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:57:04.514 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:57:04.520 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:57:34.622 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:57:35.427 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:58:05.548 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:58:05.554 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:58:35.740 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:58:35.746 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:59:05.916 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:59:05.922 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:59:36.063 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:59:36.069 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:00:06.251 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:00:06.257 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:00:16.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:00:16.427 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-06T05:00:16.472 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_50_00_UTC 2023-10-06T05:00:16.473 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_51_00_UTC 2023-10-06T05:00:16.473 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_52_00_UTC 2023-10-06T05:00:16.473 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_53_00_UTC 2023-10-06T05:00:16.473 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_54_00_UTC 2023-10-06T05:00:16.473 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_55_00_UTC 2023-10-06T05:00:16.473 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_56_00_UTC 2023-10-06T05:00:16.473 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_57_00_UTC 2023-10-06T05:00:16.473 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_58_00_UTC 2023-10-06T05:00:16.473 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_59_00_UTC 2023-10-06T05:00:16.473 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_00_00_UTC 2023-10-06T05:00:16.474 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-04_50_00_UTC', 'scheduled-2023-10-06-04_51_00_UTC', 'scheduled-2023-10-06-04_52_00_UTC', 'scheduled-2023-10-06-04_53_00_UTC', 'scheduled-2023-10-06-04_54_00_UTC', 'scheduled-2023-10-06-04_55_00_UTC', 'scheduled-2023-10-06-04_56_00_UTC', 'scheduled-2023-10-06-04_57_00_UTC', 'scheduled-2023-10-06-04_58_00_UTC', 'scheduled-2023-10-06-04_59_00_UTC', 'scheduled-2023-10-06-05_00_00_UTC'] 2023-10-06T05:00:16.475 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-10-06T05:00:16.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.713+0000 7fbfca59d700 1 Processor -- start 2023-10-06T05:00:16.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.713+0000 7fbfca59d700 1 -- start start 2023-10-06T05:00:16.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.714+0000 7fbfca59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbfc40cb560 0x7fbfc40cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:16.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.714+0000 7fbfca59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfc4069ec0 0x7fbfc415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:16.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.714+0000 7fbfca59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc4159db0 con 0x7fbfc40cc970 2023-10-06T05:00:16.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.714+0000 7fbfca59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc406aa30 con 0x7fbfc40cb560 2023-10-06T05:00:16.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.714+0000 7fbfca59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc40cce10 con 0x7fbfc4069ec0 2023-10-06T05:00:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.715+0000 7fbfc9d9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfc4069ec0 0x7fbfc415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.715+0000 7fbfc959b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fbfc40cb560 0x7fbfc40cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.715+0000 7fbfc9d9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfc4069ec0 0x7fbfc415ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:55384/0 (socket says 172.21.15.55:55384) 2023-10-06T05:00:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.715+0000 7fbfc9d9c700 1 -- 172.21.15.55:0/4155815936 learned_addr learned my addr 172.21.15.55:0/4155815936 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:00:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.716+0000 7fbfc959b700 1 -- 172.21.15.55:0/4155815936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfc4069ec0 msgr2=0x7fbfc415ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.716+0000 7fbfc959b700 1 --2- 172.21.15.55:0/4155815936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfc4069ec0 0x7fbfc415ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.716+0000 7fbfc959b700 1 -- 172.21.15.55:0/4155815936 >> v1:172.21.15.163:6789/0 conn(0x7fbfc40cc970 legacy=0x7fbfc40697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.716+0000 7fbfc959b700 1 -- 172.21.15.55:0/4155815936 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc406a470 con 0x7fbfc40cb560 2023-10-06T05:00:16.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.716+0000 7fbfc9d9c700 1 --2- 172.21.15.55:0/4155815936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfc4069ec0 0x7fbfc415ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:00:16.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.716+0000 7fbfbbfff700 1 -- 172.21.15.55:0/4155815936 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901129876 0 0) 0x7fbfc4159db0 con 0x7fbfc40cc970 2023-10-06T05:00:16.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.716+0000 7fbfc959b700 1 --2- 172.21.15.55:0/4155815936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc40cb560 0x7fbfc40cc430 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009910 tx=0x7fbfc00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e5ba5d10c593863 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:16.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.716+0000 7fbfbbfff700 1 -- 172.21.15.55:0/4155815936 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc000b640 con 0x7fbfc40cb560 2023-10-06T05:00:16.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.716+0000 7fbfbbfff700 1 -- 172.21.15.55:0/4155815936 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc000b7a0 con 0x7fbfc40cb560 2023-10-06T05:00:16.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.716+0000 7fbfbbfff700 1 -- 172.21.15.55:0/4155815936 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc000bb60 con 0x7fbfc40cb560 2023-10-06T05:00:16.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.717+0000 7fbfca59d700 1 -- 172.21.15.55:0/4155815936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc40cb560 msgr2=0x7fbfc40cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:16.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.717+0000 7fbfca59d700 1 --2- 172.21.15.55:0/4155815936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc40cb560 0x7fbfc40cc430 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009910 tx=0x7fbfc00091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:16.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.717+0000 7fbfca59d700 1 -- 172.21.15.55:0/4155815936 shutdown_connections 2023-10-06T05:00:16.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.717+0000 7fbfca59d700 1 --2- 172.21.15.55:0/4155815936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfc4069ec0 0x7fbfc415ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:16.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.717+0000 7fbfca59d700 1 --2- 172.21.15.55:0/4155815936 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc40cb560 0x7fbfc40cc430 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:16.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.717+0000 7fbfca59d700 1 -- 172.21.15.55:0/4155815936 >> 172.21.15.55:0/4155815936 conn(0x7fbfc406baf0 msgr2=0x7fbfc406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:16.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.717+0000 7fbfca59d700 1 -- 172.21.15.55:0/4155815936 shutdown_connections 2023-10-06T05:00:16.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.717+0000 7fbfca59d700 1 -- 172.21.15.55:0/4155815936 wait complete. 2023-10-06T05:00:16.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.719+0000 7fbfca59d700 1 Processor -- start 2023-10-06T05:00:16.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.719+0000 7fbfca59d700 1 -- start start 2023-10-06T05:00:16.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfca59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfc4069ec0 0x7fbfc40c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:16.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfca59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc40cb560 0x7fbfc414fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:16.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfca59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfc40cc970 0x7fbfc4150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:16.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfca59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc416aa50 con 0x7fbfc40cc970 2023-10-06T05:00:16.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfca59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc416a630 con 0x7fbfc40cb560 2023-10-06T05:00:16.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfca59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc4169a20 con 0x7fbfc4069ec0 2023-10-06T05:00:16.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfc9d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfc40cc970 0x7fbfc4150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:16.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfc9d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfc40cc970 0x7fbfc4150400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40274/0 (socket says 172.21.15.55:40274) 2023-10-06T05:00:16.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfc9d9c700 1 -- 172.21.15.55:0/2084033571 learned_addr learned my addr 172.21.15.55:0/2084033571 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:00:16.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfc8d9a700 1 --2- 172.21.15.55:0/2084033571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc40cb560 0x7fbfc414fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:16.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfc959b700 1 --2- 172.21.15.55:0/2084033571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfc4069ec0 0x7fbfc40c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:16.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfc9d9c700 1 -- 172.21.15.55:0/2084033571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfc4069ec0 msgr2=0x7fbfc40c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:16.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfc9d9c700 1 --2- 172.21.15.55:0/2084033571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfc4069ec0 0x7fbfc40c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:16.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfc9d9c700 1 -- 172.21.15.55:0/2084033571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc40cb560 msgr2=0x7fbfc414fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:16.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfc9d9c700 1 --2- 172.21.15.55:0/2084033571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc40cb560 0x7fbfc414fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:16.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.720+0000 7fbfc9d9c700 1 -- 172.21.15.55:0/2084033571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc009ae0 con 0x7fbfc40cc970 2023-10-06T05:00:16.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.721+0000 7fbfc9d9c700 1 --2- 172.21.15.55:0/2084033571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfc40cc970 0x7fbfc4150400 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009fd0 tx=0x7fbfbc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:16.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.721+0000 7fbfb97fa700 1 -- 172.21.15.55:0/2084033571 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc018070 con 0x7fbfc40cc970 2023-10-06T05:00:16.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.721+0000 7fbfb97fa700 1 -- 172.21.15.55:0/2084033571 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc00f040 con 0x7fbfc40cc970 2023-10-06T05:00:16.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.721+0000 7fbfb97fa700 1 -- 172.21.15.55:0/2084033571 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc00cbd0 con 0x7fbfc40cc970 2023-10-06T05:00:16.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.721+0000 7fbfca59d700 1 -- 172.21.15.55:0/2084033571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc0009420 con 0x7fbfc40cc970 2023-10-06T05:00:16.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.721+0000 7fbfca59d700 1 -- 172.21.15.55:0/2084033571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc4169fe0 con 0x7fbfc40cc970 2023-10-06T05:00:16.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.722+0000 7fbfca59d700 1 -- 172.21.15.55:0/2084033571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc416a7e0 con 0x7fbfc40cc970 2023-10-06T05:00:16.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.723+0000 7fbfb97fa700 1 -- 172.21.15.55:0/2084033571 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 82) v1 ==== 84871+0+0 (secure 0 0 0) 0x7fbfbc003020 con 0x7fbfc40cc970 2023-10-06T05:00:16.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.724+0000 7fbfb97fa700 1 --2- 172.21.15.55:0/2084033571 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fbfa80670a0 0x7fbfa8069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:16.726 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.725+0000 7fbfb97fa700 1 -- 172.21.15.55:0/2084033571 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6795+0+0 (secure 0 0 0) 0x7fbfbc084b90 con 0x7fbfc40cc970 2023-10-06T05:00:16.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.725+0000 7fbfc959b700 1 --2- 172.21.15.55:0/2084033571 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fbfa80670a0 0x7fbfa8069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:16.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.726+0000 7fbfc959b700 1 --2- 172.21.15.55:0/2084033571 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fbfa80670a0 0x7fbfa8069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00091e0 tx=0x7fbfc0017000 comp rx=0 tx=0).ready entity=mgr.7808 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:16.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.729+0000 7fbfb97fa700 1 -- 172.21.15.55:0/2084033571 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfbc04d5a0 con 0x7fbfc40cc970 2023-10-06T05:00:16.963 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.961+0000 7fbfca59d700 1 -- 172.21.15.55:0/2084033571 --> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfc404c3c0 con 0x7fbfa80670a0 2023-10-06T05:00:16.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.970+0000 7fbfb97fa700 1 -- 172.21.15.55:0/2084033571 <== mgr.7808 v2:172.21.15.163:6836/85191 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fbfc404c3c0 con 0x7fbfa80670a0 2023-10-06T05:00:16.974 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:00:16.974 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T04:49:41", "first": "2023-10-06T04:50:00", "last": "2023-10-06T05:00:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-06T05:00:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.973+0000 7fbfca59d700 1 -- 172.21.15.55:0/2084033571 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fbfa80670a0 msgr2=0x7fbfa8069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.973+0000 7fbfca59d700 1 --2- 172.21.15.55:0/2084033571 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fbfa80670a0 0x7fbfa8069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00091e0 tx=0x7fbfc0017000 comp rx=0 tx=0).stop 2023-10-06T05:00:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.973+0000 7fbfca59d700 1 -- 172.21.15.55:0/2084033571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfc40cc970 msgr2=0x7fbfc4150400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:16.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.973+0000 7fbfca59d700 1 --2- 172.21.15.55:0/2084033571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfc40cc970 0x7fbfc4150400 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009fd0 tx=0x7fbfbc00b040 comp rx=0 tx=0).stop 2023-10-06T05:00:16.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.973+0000 7fbfca59d700 1 -- 172.21.15.55:0/2084033571 shutdown_connections 2023-10-06T05:00:16.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.973+0000 7fbfca59d700 1 --2- 172.21.15.55:0/2084033571 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7fbfa80670a0 0x7fbfa8069550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:16.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.973+0000 7fbfca59d700 1 --2- 172.21.15.55:0/2084033571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbfc40cc970 0x7fbfc4150400 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:16.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.973+0000 7fbfca59d700 1 --2- 172.21.15.55:0/2084033571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbfc4069ec0 0x7fbfc40c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:16.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.973+0000 7fbfca59d700 1 --2- 172.21.15.55:0/2084033571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbfc40cb560 0x7fbfc414fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:16.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.973+0000 7fbfca59d700 1 -- 172.21.15.55:0/2084033571 >> 172.21.15.55:0/2084033571 conn(0x7fbfc406baf0 msgr2=0x7fbfc40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:16.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.973+0000 7fbfca59d700 1 -- 172.21.15.55:0/2084033571 shutdown_connections 2023-10-06T05:00:16.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:16.973+0000 7fbfca59d700 1 -- 172.21.15.55:0/2084033571 wait complete. 2023-10-06T05:00:16.992 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T04:49:41", "first": "2023-10-06T04:50:00", "last": "2023-10-06T05:00:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-06T05:00:16.993 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T04:49:41', 'first': '2023-10-06T04:50:00', 'last': '2023-10-06T05:00:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-10-06T05:00:16.993 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-06T05:00:16.993 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-06T05:00:16.993 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T05:00:17.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.179+0000 7f02fd39d700 1 Processor -- start 2023-10-06T05:00:17.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.179+0000 7f02fd39d700 1 -- start start 2023-10-06T05:00:17.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.180+0000 7f02fd39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f02f8159420 0x7f02f80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:17.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.180+0000 7f02fd39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f02f815d230 0x7f02f815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:17.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.180+0000 7f02fd39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f80b2980 con 0x7f02f81597f0 2023-10-06T05:00:17.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.180+0000 7f02fd39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f815db70 con 0x7f02f8159420 2023-10-06T05:00:17.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.180+0000 7f02fd39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f80b11d0 con 0x7f02f815d230 2023-10-06T05:00:17.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.181+0000 7f02f7fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f02f8159420 0x7f02f80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:17.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.181+0000 7f02f77fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f02f81597f0 0x7f02f80b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52840/0 (socket says 172.21.15.55:52840) 2023-10-06T05:00:17.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.181+0000 7f02f77fe700 1 -- 172.21.15.55:0/668503827 learned_addr learned my addr 172.21.15.55:0/668503827 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:00:17.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.181+0000 7f02f7fff700 1 -- 172.21.15.55:0/668503827 >> v2:172.21.15.163:3300/0 conn(0x7f02f815d230 msgr2=0x7f02f815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:00:17.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.181+0000 7f02fcb9c700 1 --2- 172.21.15.55:0/668503827 >> v2:172.21.15.163:3300/0 conn(0x7f02f815d230 0x7f02f815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:17.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.181+0000 7f02f7fff700 1 --2- 172.21.15.55:0/668503827 >> v2:172.21.15.163:3300/0 conn(0x7f02f815d230 0x7f02f815d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:17.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.181+0000 7f02f6ffd700 1 -- 172.21.15.55:0/668503827 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108269587 0 0) 0x7f02f80b2980 con 0x7f02f81597f0 2023-10-06T05:00:17.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.181+0000 7f02f7fff700 1 -- 172.21.15.55:0/668503827 >> v1:172.21.15.55:6789/0 conn(0x7f02f81597f0 legacy=0x7f02f80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:17.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.181+0000 7f02f7fff700 1 -- 172.21.15.55:0/668503827 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f80b14b0 con 0x7f02f8159420 2023-10-06T05:00:17.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.181+0000 7f02fcb9c700 1 --2- 172.21.15.55:0/668503827 >> v2:172.21.15.163:3300/0 conn(0x7f02f815d230 0x7f02f815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:17.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.182+0000 7f02f7fff700 1 --2- 172.21.15.55:0/668503827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02f8159420 0x7f02f80b5f40 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f02f815d170 tx=0x7f02e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ae3428890c7728 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:17.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.182+0000 7f02f6ffd700 1 -- 172.21.15.55:0/668503827 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e800b6b0 con 0x7f02f8159420 2023-10-06T05:00:17.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.182+0000 7f02f6ffd700 1 -- 172.21.15.55:0/668503827 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e800b810 con 0x7f02f8159420 2023-10-06T05:00:17.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.182+0000 7f02f6ffd700 1 -- 172.21.15.55:0/668503827 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e800bbb0 con 0x7f02f8159420 2023-10-06T05:00:17.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.182+0000 7f02fd39d700 1 -- 172.21.15.55:0/668503827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02f8159420 msgr2=0x7f02f80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:17.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.182+0000 7f02fd39d700 1 --2- 172.21.15.55:0/668503827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02f8159420 0x7f02f80b5f40 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f02f815d170 tx=0x7f02e80091a0 comp rx=0 tx=0).stop 2023-10-06T05:00:17.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.182+0000 7f02fd39d700 1 -- 172.21.15.55:0/668503827 shutdown_connections 2023-10-06T05:00:17.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.182+0000 7f02fd39d700 1 --2- 172.21.15.55:0/668503827 >> v2:172.21.15.163:3300/0 conn(0x7f02f815d230 0x7f02f815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:17.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.182+0000 7f02fd39d700 1 --2- 172.21.15.55:0/668503827 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02f8159420 0x7f02f80b5f40 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:17.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.182+0000 7f02fd39d700 1 -- 172.21.15.55:0/668503827 >> 172.21.15.55:0/668503827 conn(0x7f02f80d0660 msgr2=0x7f02f80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:17.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.182+0000 7f02fd39d700 1 -- 172.21.15.55:0/668503827 shutdown_connections 2023-10-06T05:00:17.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.183+0000 7f02fd39d700 1 -- 172.21.15.55:0/668503827 wait complete. 2023-10-06T05:00:17.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.184+0000 7f02fd39d700 1 Processor -- start 2023-10-06T05:00:17.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.184+0000 7f02fd39d700 1 -- start start 2023-10-06T05:00:17.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.185+0000 7f02fd39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02f8159420 0x7f02f8157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:17.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.185+0000 7f02fd39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02f81597f0 0x7f02f80ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:17.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.185+0000 7f02fd39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02f815d230 0x7f02f80ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:17.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.185+0000 7f02fd39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f80c6c40 con 0x7f02f81597f0 2023-10-06T05:00:17.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.185+0000 7f02fd39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f80cab30 con 0x7f02f815d230 2023-10-06T05:00:17.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.185+0000 7f02fd39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f80c6030 con 0x7f02f8159420 2023-10-06T05:00:17.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.185+0000 7f02f77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02f81597f0 0x7f02f80ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:17.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.185+0000 7f02f77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02f81597f0 0x7f02f80ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40280/0 (socket says 172.21.15.55:40280) 2023-10-06T05:00:17.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.185+0000 7f02f77fe700 1 -- 172.21.15.55:0/3888923550 learned_addr learned my addr 172.21.15.55:0/3888923550 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:00:17.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02fcb9c700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02f815d230 0x7f02f80ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:17.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02f7fff700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02f8159420 0x7f02f8157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:17.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02f77fe700 1 -- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02f8159420 msgr2=0x7f02f8157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:17.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02f77fe700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02f8159420 0x7f02f8157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:17.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02f77fe700 1 -- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02f815d230 msgr2=0x7f02f80ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:17.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02f77fe700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02f815d230 0x7f02f80ca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:17.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02f77fe700 1 -- 172.21.15.55:0/3888923550 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e8009420 con 0x7f02f81597f0 2023-10-06T05:00:17.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02fcb9c700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02f815d230 0x7f02f80ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:17.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02f7fff700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02f8159420 0x7f02f8157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:00:17.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02f77fe700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02f81597f0 0x7f02f80ca0b0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f02ec00c810 tx=0x7f02ec007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:17.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02dffff700 1 -- 172.21.15.55:0/3888923550 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec005c00 con 0x7f02f81597f0 2023-10-06T05:00:17.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02dffff700 1 -- 172.21.15.55:0/3888923550 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ec005d60 con 0x7f02f81597f0 2023-10-06T05:00:17.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02dffff700 1 -- 172.21.15.55:0/3888923550 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ec007320 con 0x7f02f81597f0 2023-10-06T05:00:17.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02fd39d700 1 -- 172.21.15.55:0/3888923550 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f80c6310 con 0x7f02f81597f0 2023-10-06T05:00:17.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.186+0000 7f02fd39d700 1 -- 172.21.15.55:0/3888923550 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f80c68d0 con 0x7f02f81597f0 2023-10-06T05:00:17.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.187+0000 7f02fd39d700 1 -- 172.21.15.55:0/3888923550 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f804c3c0 con 0x7f02f81597f0 2023-10-06T05:00:17.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.189+0000 7f02dffff700 1 -- 172.21.15.55:0/3888923550 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 82) v1 ==== 84871+0+0 (secure 0 0 0) 0x7f02ec01b400 con 0x7f02f81597f0 2023-10-06T05:00:17.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.189+0000 7f02dffff700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f02e40670a0 0x7f02e4069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:00:17.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.190+0000 7f02dffff700 1 -- 172.21.15.55:0/3888923550 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6795+0+0 (secure 0 0 0) 0x7f02ec0846b0 con 0x7f02f81597f0 2023-10-06T05:00:17.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.190+0000 7f02f7fff700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f02e40670a0 0x7f02e4069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:00:17.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.191+0000 7f02f7fff700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f02e40670a0 0x7f02e4069550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f02e8003080 tx=0x7f02e8017000 comp rx=0 tx=0).ready entity=mgr.7808 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:00:17.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.194+0000 7f02dffff700 1 -- 172.21.15.55:0/3888923550 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02ec0513b0 con 0x7f02f81597f0 2023-10-06T05:00:17.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.450+0000 7f02fd39d700 1 -- 172.21.15.55:0/3888923550 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f02f80b1790 con 0x7f02f81597f0 2023-10-06T05:00:17.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.454+0000 7f02dffff700 1 -- 172.21.15.55:0/3888923550 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v82) v1 ==== 74+0+163914 (secure 0 0 0) 0x7f02ec0549d0 con 0x7f02f81597f0 2023-10-06T05:00:17.458 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:00:17.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.457+0000 7f02fd39d700 1 -- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f02e40670a0 msgr2=0x7f02e4069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:17.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.457+0000 7f02fd39d700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f02e40670a0 0x7f02e4069550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f02e8003080 tx=0x7f02e8017000 comp rx=0 tx=0).stop 2023-10-06T05:00:17.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.457+0000 7f02fd39d700 1 -- 172.21.15.55:0/3888923550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02f81597f0 msgr2=0x7f02f80ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:00:17.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.457+0000 7f02fd39d700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02f81597f0 0x7f02f80ca0b0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f02ec00c810 tx=0x7f02ec007db0 comp rx=0 tx=0).stop 2023-10-06T05:00:17.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.457+0000 7f02fd39d700 1 -- 172.21.15.55:0/3888923550 shutdown_connections 2023-10-06T05:00:17.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.457+0000 7f02fd39d700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:6836/85191,v1:172.21.15.163:6837/85191] conn(0x7f02e40670a0 0x7f02e4069550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:17.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.457+0000 7f02fd39d700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f02f815d230 0x7f02f80ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:17.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.457+0000 7f02fd39d700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f02f8159420 0x7f02f8157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:17.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.457+0000 7f02fd39d700 1 --2- 172.21.15.55:0/3888923550 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f02f81597f0 0x7f02f80ca0b0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:00:17.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.457+0000 7f02fd39d700 1 -- 172.21.15.55:0/3888923550 >> 172.21.15.55:0/3888923550 conn(0x7f02f80d0660 msgr2=0x7f02f80ccec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:00:17.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.457+0000 7f02fd39d700 1 -- 172.21.15.55:0/3888923550 shutdown_connections 2023-10-06T05:00:17.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:00:17.458+0000 7f02fd39d700 1 -- 172.21.15.55:0/3888923550 wait complete. 2023-10-06T05:00:17.477 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":82,"active_gid":7808,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6836","nonce":85191},{"type":"v1","addr":"172.21.15.163:6837","nonce":85191}]},"active_addr":"172.21.15.163:6837/85191","active_change":"2023-10-06T04:49:40.797920+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":8893,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":174,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":3141284388}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":3594252861}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":3085093567}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":1915465424}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":167943248}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.163:0","nonce":2727475042}]}]} 2023-10-06T05:00:17.482 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: x 2023-10-06T05:00:17.482 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T05:00:17.482 INFO:tasks.ceph.mgr.x:Stopping old one... 2023-10-06T05:00:17.483 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T05:00:17.483 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:00:17.555 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T05:00:17.556 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T05:00:17.560 INFO:tasks.ceph.mgr.x:Started 2023-10-06T05:00:18.892 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:18.890+0000 7fb7e26de180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:00:19.055 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:19.054+0000 7fb7e26de180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:00:19.141 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:19.139+0000 7fb7e26de180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:00:19.367 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:19.366+0000 7fb7e26de180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:00:19.546 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:19.544+0000 7fb7e26de180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:00:19.638 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:19.636+0000 7fb7e26de180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:00:19.802 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:19.801+0000 7fb7e26de180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:00:19.902 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:19.901+0000 7fb7e26de180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:00:20.388 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:20.386+0000 7fb7e26de180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:00:20.483 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:20.481+0000 7fb7e26de180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:00:20.573 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:20.572+0000 7fb7e26de180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:00:21.195 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:21.194+0000 7fb7e26de180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:00:21.368 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:21.367+0000 7fb7e26de180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:00:21.465 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:21.463+0000 7fb7e26de180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:00:21.761 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:21.760+0000 7fb7e26de180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:00:21.858 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:21.857+0000 7fb7e26de180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:00:22.560 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:22.559+0000 7fb7e26de180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:00:22.847 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:22.846+0000 7fb7e26de180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:00:22.940 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:22.939+0000 7fb7e26de180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:00:23.094 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.093+0000 7fb7e26de180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:00:23.200 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.199+0000 7fb7e26de180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:00:23.392 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.390+0000 7fb7e26de180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:00:23.486 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.485+0000 7fb7e26de180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:00:23.580 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.579+0000 7fb7e26de180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:00:23.676 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.675+0000 7fb7e26de180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:00:23.844 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.843+0000 7fb757236700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:00:23.844 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.843+0000 7fb757236700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:00:23.845 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.843+0000 7fb757236700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:00:23.845 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.843+0000 7fb757236700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:00:23.845 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.843+0000 7fb757236700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:00:23.845 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.843+0000 7fb754230700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:00:23.846 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.843+0000 7fb754230700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:00:23.846 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.843+0000 7fb754230700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:00:23.846 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.843+0000 7fb754230700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:00:23.846 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:00:23.843+0000 7fb754230700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:00:36.447 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:00:36.453 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:01:00.842 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:01:00.840+0000 7fb75f246700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:01:00.842 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:01:00.840+0000 7fb75f246700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:01:00.842 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:01:00.840+0000 7fb75f246700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:01:00.842 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:01:00.840+0000 7fb75f246700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:01:00.842 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:01:00.840+0000 7fb75f246700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:01:06.670 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:01:06.675 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:01:36.824 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:01:36.830 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:02:06.971 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:02:07.083 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:02:37.217 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:02:37.223 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:03:07.405 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:03:07.410 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:03:37.593 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:03:37.599 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:04:07.756 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:04:07.761 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:04:37.936 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:04:37.942 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:05:08.108 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:05:08.113 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:05:17.562 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/test_restart 1M 2023-10-06T05:05:17.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.765+0000 7efef4f9d700 1 Processor -- start 2023-10-06T05:05:17.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.765+0000 7efef4f9d700 1 -- start start 2023-10-06T05:05:17.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efef4f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efef0156350 0x7efef0157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:17.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efef4f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efef00b0760 0x7efef00b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:17.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efef4f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef00b1110 con 0x7efef0159f20 2023-10-06T05:05:17.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efef4f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef00b0570 con 0x7efef0156350 2023-10-06T05:05:17.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efef4f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef01578b0 con 0x7efef00b0760 2023-10-06T05:05:17.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efeef7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efef0156350 0x7efef0157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:17.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efeeffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7efef0159f20 0x7efef015a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:33872/0 (socket says 172.21.15.55:33872) 2023-10-06T05:05:17.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efeeffff700 1 -- 172.21.15.55:0/1641495204 learned_addr learned my addr 172.21.15.55:0/1641495204 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:17.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efeeeffd700 1 --2- 172.21.15.55:0/1641495204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efef00b0760 0x7efef00b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:17.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efeee7fc700 1 -- 172.21.15.55:0/1641495204 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729533673 0 0) 0x7efef00b1110 con 0x7efef0159f20 2023-10-06T05:05:17.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efeee7fc700 1 -- 172.21.15.55:0/1641495204 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efed8003570 con 0x7efef0159f20 2023-10-06T05:05:17.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efeef7fe700 1 -- 172.21.15.55:0/1641495204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efef00b0760 msgr2=0x7efef00b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:17.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efeef7fe700 1 --2- 172.21.15.55:0/1641495204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efef00b0760 0x7efef00b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:17.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efeef7fe700 1 -- 172.21.15.55:0/1641495204 >> v1:172.21.15.55:6789/0 conn(0x7efef0159f20 legacy=0x7efef015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:17.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.767+0000 7efeef7fe700 1 -- 172.21.15.55:0/1641495204 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef00af4d0 con 0x7efef0156350 2023-10-06T05:05:17.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.768+0000 7efeeeffd700 1 --2- 172.21.15.55:0/1641495204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efef00b0760 0x7efef00b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:17.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.768+0000 7efeef7fe700 1 --2- 172.21.15.55:0/1641495204 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef0156350 0x7efef0157370 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7efee00098e0 tx=0x7efee00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3848d7ac8b6c5a65 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:17.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.768+0000 7efeee7fc700 1 -- 172.21.15.55:0/1641495204 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee000b640 con 0x7efef0156350 2023-10-06T05:05:17.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.768+0000 7efeee7fc700 1 -- 172.21.15.55:0/1641495204 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee000b7a0 con 0x7efef0156350 2023-10-06T05:05:17.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.768+0000 7efeee7fc700 1 -- 172.21.15.55:0/1641495204 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee000bb40 con 0x7efef0156350 2023-10-06T05:05:17.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.769+0000 7efef4f9d700 1 -- 172.21.15.55:0/1641495204 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef0156350 msgr2=0x7efef0157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:17.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.769+0000 7efef4f9d700 1 --2- 172.21.15.55:0/1641495204 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef0156350 0x7efef0157370 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7efee00098e0 tx=0x7efee00091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:17.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.769+0000 7efef4f9d700 1 -- 172.21.15.55:0/1641495204 shutdown_connections 2023-10-06T05:05:17.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.769+0000 7efef4f9d700 1 --2- 172.21.15.55:0/1641495204 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef0156350 0x7efef0157370 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:17.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.769+0000 7efef4f9d700 1 --2- 172.21.15.55:0/1641495204 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efef00b0760 0x7efef00b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:17.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.769+0000 7efef4f9d700 1 -- 172.21.15.55:0/1641495204 >> 172.21.15.55:0/1641495204 conn(0x7efef0068e20 msgr2=0x7efef00cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:17.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.769+0000 7efef4f9d700 1 -- 172.21.15.55:0/1641495204 shutdown_connections 2023-10-06T05:05:17.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.769+0000 7efef4f9d700 1 -- 172.21.15.55:0/1641495204 wait complete. 2023-10-06T05:05:17.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.770+0000 7efef4f9d700 1 Processor -- start 2023-10-06T05:05:17.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.770+0000 7efef4f9d700 1 -- start start 2023-10-06T05:05:17.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.771+0000 7efef4f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef00b0760 0x7efef00c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:17.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.771+0000 7efef4f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efef0156350 0x7efef00ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:17.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.771+0000 7efef4f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efef0159f20 0x7efef00caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:17.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.771+0000 7efef4f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef0169cb0 con 0x7efef00b0760 2023-10-06T05:05:17.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.771+0000 7efef4f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef00aa830 con 0x7efef0156350 2023-10-06T05:05:17.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.771+0000 7efef4f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef00a9cb0 con 0x7efef0159f20 2023-10-06T05:05:17.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.772+0000 7efeef7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef00b0760 0x7efef00c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:17.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.772+0000 7efeef7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef00b0760 0x7efef00c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56006/0 (socket says 172.21.15.55:56006) 2023-10-06T05:05:17.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.772+0000 7efeef7fe700 1 -- 172.21.15.55:0/3280577445 learned_addr learned my addr 172.21.15.55:0/3280577445 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:17.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.772+0000 7efeef7fe700 1 -- 172.21.15.55:0/3280577445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efef0159f20 msgr2=0x7efef00caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:17.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.772+0000 7efeef7fe700 1 --2- 172.21.15.55:0/3280577445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efef0159f20 0x7efef00caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:17.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.772+0000 7efeef7fe700 1 -- 172.21.15.55:0/3280577445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efef0156350 msgr2=0x7efef00ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:17.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.772+0000 7efeef7fe700 1 --2- 172.21.15.55:0/3280577445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efef0156350 0x7efef00ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:17.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.772+0000 7efeef7fe700 1 -- 172.21.15.55:0/3280577445 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee0009420 con 0x7efef00b0760 2023-10-06T05:05:17.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.772+0000 7efeef7fe700 1 --2- 172.21.15.55:0/3280577445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef00b0760 0x7efef00c90c0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7efef00714a0 tx=0x7efee0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:17.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.772+0000 7efed77fe700 1 -- 172.21.15.55:0/3280577445 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee001bda0 con 0x7efef00b0760 2023-10-06T05:05:17.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.773+0000 7efed77fe700 1 -- 172.21.15.55:0/3280577445 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee0011420 con 0x7efef00b0760 2023-10-06T05:05:17.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.773+0000 7efef4f9d700 1 -- 172.21.15.55:0/3280577445 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef00a9f90 con 0x7efef00b0760 2023-10-06T05:05:17.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.773+0000 7efed77fe700 1 -- 172.21.15.55:0/3280577445 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee00117e0 con 0x7efef00b0760 2023-10-06T05:05:17.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.773+0000 7efef4f9d700 1 -- 172.21.15.55:0/3280577445 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef00aa550 con 0x7efef00b0760 2023-10-06T05:05:17.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.774+0000 7efef4f9d700 1 -- 172.21.15.55:0/3280577445 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef0169df0 con 0x7efef00b0760 2023-10-06T05:05:17.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.775+0000 7efed77fe700 1 -- 172.21.15.55:0/3280577445 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 87) v1 ==== 84871+0+0 (secure 0 0 0) 0x7efee0011b90 con 0x7efef00b0760 2023-10-06T05:05:17.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.775+0000 7efed77fe700 1 --2- 172.21.15.55:0/3280577445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efedc0670a0 0x7efedc069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:17.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.776+0000 7efed77fe700 1 -- 172.21.15.55:0/3280577445 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7053+0+0 (secure 0 0 0) 0x7efee0086280 con 0x7efef00b0760 2023-10-06T05:05:17.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.776+0000 7efeeeffd700 1 --2- 172.21.15.55:0/3280577445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efedc0670a0 0x7efedc069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:17.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.777+0000 7efeeeffd700 1 --2- 172.21.15.55:0/3280577445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efedc0670a0 0x7efedc069550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efee4006fd0 tx=0x7efee4008040 comp rx=0 tx=0).ready entity=mgr.8989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:17.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:17.780+0000 7efed77fe700 1 -- 172.21.15.55:0/3280577445 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efee0052f50 con 0x7efef00b0760 2023-10-06T05:05:18.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.010+0000 7efef4f9d700 1 -- 172.21.15.55:0/3280577445 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7efef004c3c0 con 0x7efedc0670a0 2023-10-06T05:05:18.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.071+0000 7efed77fe700 1 -- 172.21.15.55:0/3280577445 <== mgr.8989 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7efef004c3c0 con 0x7efedc0670a0 2023-10-06T05:05:18.073 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-06T05:05:18.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.073+0000 7efef4f9d700 1 -- 172.21.15.55:0/3280577445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efedc0670a0 msgr2=0x7efedc069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:18.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.073+0000 7efef4f9d700 1 --2- 172.21.15.55:0/3280577445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efedc0670a0 0x7efedc069550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efee4006fd0 tx=0x7efee4008040 comp rx=0 tx=0).stop 2023-10-06T05:05:18.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.073+0000 7efef4f9d700 1 -- 172.21.15.55:0/3280577445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef00b0760 msgr2=0x7efef00c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:18.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.073+0000 7efef4f9d700 1 --2- 172.21.15.55:0/3280577445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef00b0760 0x7efef00c90c0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7efef00714a0 tx=0x7efee0017000 comp rx=0 tx=0).stop 2023-10-06T05:05:18.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.074+0000 7efef4f9d700 1 -- 172.21.15.55:0/3280577445 shutdown_connections 2023-10-06T05:05:18.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.074+0000 7efef4f9d700 1 --2- 172.21.15.55:0/3280577445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efedc0670a0 0x7efedc069550 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.074+0000 7efef4f9d700 1 --2- 172.21.15.55:0/3280577445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efef0159f20 0x7efef00caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.074+0000 7efef4f9d700 1 --2- 172.21.15.55:0/3280577445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efef00b0760 0x7efef00c90c0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.074+0000 7efef4f9d700 1 --2- 172.21.15.55:0/3280577445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efef0156350 0x7efef00ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.074+0000 7efef4f9d700 1 -- 172.21.15.55:0/3280577445 >> 172.21.15.55:0/3280577445 conn(0x7efef0068e20 msgr2=0x7efef0152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:18.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.074+0000 7efef4f9d700 1 -- 172.21.15.55:0/3280577445 shutdown_connections 2023-10-06T05:05:18.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.074+0000 7efef4f9d700 1 -- 172.21.15.55:0/3280577445 wait complete. 2023-10-06T05:05:18.092 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-06T05:05:18.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:18.093 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-06T05:05:18.131 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_50_00_UTC 2023-10-06T05:05:18.131 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_51_00_UTC 2023-10-06T05:05:18.131 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_52_00_UTC 2023-10-06T05:05:18.132 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_53_00_UTC 2023-10-06T05:05:18.132 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_54_00_UTC 2023-10-06T05:05:18.132 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_55_00_UTC 2023-10-06T05:05:18.132 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_56_00_UTC 2023-10-06T05:05:18.132 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_57_00_UTC 2023-10-06T05:05:18.132 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_58_00_UTC 2023-10-06T05:05:18.132 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_59_00_UTC 2023-10-06T05:05:18.132 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_00_00_UTC 2023-10-06T05:05:18.132 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_01_00_UTC 2023-10-06T05:05:18.132 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_02_00_UTC 2023-10-06T05:05:18.132 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_03_00_UTC 2023-10-06T05:05:18.133 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_04_00_UTC 2023-10-06T05:05:18.133 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_05_00_UTC 2023-10-06T05:05:18.134 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-06-04_50_00_UTC', 'scheduled-2023-10-06-04_51_00_UTC', 'scheduled-2023-10-06-04_52_00_UTC', 'scheduled-2023-10-06-04_53_00_UTC', 'scheduled-2023-10-06-04_54_00_UTC', 'scheduled-2023-10-06-04_55_00_UTC', 'scheduled-2023-10-06-04_56_00_UTC', 'scheduled-2023-10-06-04_57_00_UTC', 'scheduled-2023-10-06-04_58_00_UTC', 'scheduled-2023-10-06-04_59_00_UTC', 'scheduled-2023-10-06-05_00_00_UTC', 'scheduled-2023-10-06-05_01_00_UTC', 'scheduled-2023-10-06-05_02_00_UTC', 'scheduled-2023-10-06-05_03_00_UTC', 'scheduled-2023-10-06-05_04_00_UTC', 'scheduled-2023-10-06-05_05_00_UTC'] 2023-10-06T05:05:18.134 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-10-06T05:05:18.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.364+0000 7f46fc9fe700 1 Processor -- start 2023-10-06T05:05:18.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.364+0000 7f46fc9fe700 1 -- start start 2023-10-06T05:05:18.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.365+0000 7f46fc9fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f46f80685e0 0x7f46f815a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:18.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.365+0000 7f46fc9fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f46f8154000 0x7f46f806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:18.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.365+0000 7f46fc9fe700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f80c9480 con 0x7f46f815d3f0 2023-10-06T05:05:18.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.365+0000 7f46fc9fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f815d880 con 0x7f46f8154000 2023-10-06T05:05:18.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.365+0000 7f46fc9fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f815db60 con 0x7f46f80685e0 2023-10-06T05:05:18.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.366+0000 7f46f759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f46f8154000 0x7f46f806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:18.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.366+0000 7f46f659c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f46f815d3f0 0x7f46f81538f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:33878/0 (socket says 172.21.15.55:33878) 2023-10-06T05:05:18.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.366+0000 7f46f659c700 1 -- 172.21.15.55:0/3581326769 learned_addr learned my addr 172.21.15.55:0/3581326769 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:18.369 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.366+0000 7f46f6d9d700 1 --2- 172.21.15.55:0/3581326769 >> v2:172.21.15.163:3300/0 conn(0x7f46f80685e0 0x7f46f815a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:18.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.366+0000 7f46f759e700 1 -- 172.21.15.55:0/3581326769 >> v2:172.21.15.163:3300/0 conn(0x7f46f80685e0 msgr2=0x7f46f815a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:18.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.366+0000 7f46f759e700 1 --2- 172.21.15.55:0/3581326769 >> v2:172.21.15.163:3300/0 conn(0x7f46f80685e0 0x7f46f815a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.366+0000 7f46f759e700 1 -- 172.21.15.55:0/3581326769 >> v1:172.21.15.55:6789/0 conn(0x7f46f815d3f0 legacy=0x7f46f81538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:18.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.366+0000 7f46f759e700 1 -- 172.21.15.55:0/3581326769 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f815de40 con 0x7f46f8154000 2023-10-06T05:05:18.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.366+0000 7f46f6d9d700 1 --2- 172.21.15.55:0/3581326769 >> v2:172.21.15.163:3300/0 conn(0x7f46f80685e0 0x7f46f815a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:18.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.366+0000 7f46f5d9b700 1 -- 172.21.15.55:0/3581326769 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470611902 0 0) 0x7f46f80c9480 con 0x7f46f815d3f0 2023-10-06T05:05:18.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.366+0000 7f46f759e700 1 --2- 172.21.15.55:0/3581326769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f8154000 0x7f46f806eb00 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f46f8065920 tx=0x7f46f0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7f7b0998d22b60 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:18.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.367+0000 7f46f5d9b700 1 -- 172.21.15.55:0/3581326769 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f000e070 con 0x7f46f8154000 2023-10-06T05:05:18.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.367+0000 7f46f5d9b700 1 -- 172.21.15.55:0/3581326769 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f000bcc0 con 0x7f46f8154000 2023-10-06T05:05:18.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.367+0000 7f46f5d9b700 1 -- 172.21.15.55:0/3581326769 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0012680 con 0x7f46f8154000 2023-10-06T05:05:18.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.367+0000 7f46fc9fe700 1 -- 172.21.15.55:0/3581326769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f8154000 msgr2=0x7f46f806eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:18.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.367+0000 7f46fc9fe700 1 --2- 172.21.15.55:0/3581326769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f8154000 0x7f46f806eb00 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f46f8065920 tx=0x7f46f0009780 comp rx=0 tx=0).stop 2023-10-06T05:05:18.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.367+0000 7f46fc9fe700 1 -- 172.21.15.55:0/3581326769 shutdown_connections 2023-10-06T05:05:18.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.367+0000 7f46fc9fe700 1 --2- 172.21.15.55:0/3581326769 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f8154000 0x7f46f806eb00 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.367+0000 7f46fc9fe700 1 --2- 172.21.15.55:0/3581326769 >> v2:172.21.15.163:3300/0 conn(0x7f46f80685e0 0x7f46f815a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.367+0000 7f46fc9fe700 1 -- 172.21.15.55:0/3581326769 >> 172.21.15.55:0/3581326769 conn(0x7f46f806b420 msgr2=0x7f46f80cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:18.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.367+0000 7f46fc9fe700 1 -- 172.21.15.55:0/3581326769 shutdown_connections 2023-10-06T05:05:18.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.367+0000 7f46fc9fe700 1 -- 172.21.15.55:0/3581326769 wait complete. 2023-10-06T05:05:18.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.369+0000 7f46fc9fe700 1 Processor -- start 2023-10-06T05:05:18.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.369+0000 7f46fc9fe700 1 -- start start 2023-10-06T05:05:18.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.370+0000 7f46fc9fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f80685e0 0x7f46f81508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:18.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.370+0000 7f46fc9fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f8154000 0x7f46f80c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:18.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.370+0000 7f46fc9fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46f815d3f0 0x7f46f80b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:18.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.370+0000 7f46fc9fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f80b0440 con 0x7f46f80685e0 2023-10-06T05:05:18.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.370+0000 7f46fc9fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f8159470 con 0x7f46f815d3f0 2023-10-06T05:05:18.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.370+0000 7f46fc9fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f8159750 con 0x7f46f8154000 2023-10-06T05:05:18.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.370+0000 7f46f6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f80685e0 0x7f46f81508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:18.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.370+0000 7f46f6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f80685e0 0x7f46f81508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56016/0 (socket says 172.21.15.55:56016) 2023-10-06T05:05:18.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.370+0000 7f46f6d9d700 1 -- 172.21.15.55:0/2426528338 learned_addr learned my addr 172.21.15.55:0/2426528338 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:18.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46f759e700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46f815d3f0 0x7f46f80b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:18.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46f659c700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f8154000 0x7f46f80c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:18.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46f6d9d700 1 -- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f8154000 msgr2=0x7f46f80c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:18.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46f6d9d700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f8154000 0x7f46f80c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46f6d9d700 1 -- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46f815d3f0 msgr2=0x7f46f80b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:18.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46f6d9d700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46f815d3f0 0x7f46f80b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46f6d9d700 1 -- 172.21.15.55:0/2426528338 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f0009a20 con 0x7f46f80685e0 2023-10-06T05:05:18.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46f759e700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46f815d3f0 0x7f46f80b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:18.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46f659c700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f8154000 0x7f46f80c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:18.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46f6d9d700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f80685e0 0x7f46f81508e0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c830 tx=0x7f46e8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:18.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46deffd700 1 -- 172.21.15.55:0/2426528338 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e8011600 con 0x7f46f80685e0 2023-10-06T05:05:18.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46deffd700 1 -- 172.21.15.55:0/2426528338 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e8011760 con 0x7f46f80685e0 2023-10-06T05:05:18.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46fc9fe700 1 -- 172.21.15.55:0/2426528338 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f8159a30 con 0x7f46f80685e0 2023-10-06T05:05:18.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46deffd700 1 -- 172.21.15.55:0/2426528338 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e8011b20 con 0x7f46f80685e0 2023-10-06T05:05:18.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.371+0000 7f46fc9fe700 1 -- 172.21.15.55:0/2426528338 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f80c5760 con 0x7f46f80685e0 2023-10-06T05:05:18.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.373+0000 7f46fc9fe700 1 -- 172.21.15.55:0/2426528338 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f80b0720 con 0x7f46f80685e0 2023-10-06T05:05:18.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.373+0000 7f46deffd700 1 -- 172.21.15.55:0/2426528338 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 87) v1 ==== 84871+0+0 (secure 0 0 0) 0x7f46e800a3e0 con 0x7f46f80685e0 2023-10-06T05:05:18.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.374+0000 7f46deffd700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f46e40670a0 0x7f46e4069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:18.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.374+0000 7f46deffd700 1 -- 172.21.15.55:0/2426528338 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7053+0+0 (secure 0 0 0) 0x7f46e8085290 con 0x7f46f80685e0 2023-10-06T05:05:18.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.375+0000 7f46f659c700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f46e40670a0 0x7f46e4069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:18.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.375+0000 7f46f659c700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f46e40670a0 0x7f46e4069550 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f46f80c6c70 tx=0x7f46ec009040 comp rx=0 tx=0).ready entity=mgr.8989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:18.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.378+0000 7f46deffd700 1 -- 172.21.15.55:0/2426528338 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46e8051d00 con 0x7f46f80685e0 2023-10-06T05:05:18.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.607+0000 7f46fc9fe700 1 -- 172.21.15.55:0/2426528338 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46f80b0210 con 0x7f46e40670a0 2023-10-06T05:05:18.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.617+0000 7f46deffd700 1 -- 172.21.15.55:0/2426528338 <== mgr.8989 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7f46f80b0210 con 0x7f46e40670a0 2023-10-06T05:05:18.621 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:18.621 INFO:teuthology.orchestra.run.smithi055.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T04:49:41", "first": "2023-10-06T04:50:00", "last": "2023-10-06T05:05:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-06T05:05:18.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.619+0000 7f46fc9fe700 1 -- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f46e40670a0 msgr2=0x7f46e4069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:18.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.619+0000 7f46fc9fe700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f46e40670a0 0x7f46e4069550 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f46f80c6c70 tx=0x7f46ec009040 comp rx=0 tx=0).stop 2023-10-06T05:05:18.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.619+0000 7f46fc9fe700 1 -- 172.21.15.55:0/2426528338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f80685e0 msgr2=0x7f46f81508e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:18.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.619+0000 7f46fc9fe700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f80685e0 0x7f46f81508e0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f46e800c830 tx=0x7f46e8008040 comp rx=0 tx=0).stop 2023-10-06T05:05:18.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.619+0000 7f46fc9fe700 1 -- 172.21.15.55:0/2426528338 shutdown_connections 2023-10-06T05:05:18.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.619+0000 7f46fc9fe700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f46e40670a0 0x7f46e4069550 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.619+0000 7f46fc9fe700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46f815d3f0 0x7f46f80b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.619+0000 7f46fc9fe700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f80685e0 0x7f46f81508e0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.619+0000 7f46fc9fe700 1 --2- 172.21.15.55:0/2426528338 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f8154000 0x7f46f80c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.619+0000 7f46fc9fe700 1 -- 172.21.15.55:0/2426528338 >> 172.21.15.55:0/2426528338 conn(0x7f46f806b420 msgr2=0x7f46f815e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:18.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.619+0000 7f46fc9fe700 1 -- 172.21.15.55:0/2426528338 shutdown_connections 2023-10-06T05:05:18.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.619+0000 7f46fc9fe700 1 -- 172.21.15.55:0/2426528338 wait complete. 2023-10-06T05:05:18.638 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-10-06T00:00:00", "created": "2023-10-06T04:49:41", "first": "2023-10-06T04:50:00", "last": "2023-10-06T05:05:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-06T05:05:18.638 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-10-06T00:00:00', 'created': '2023-10-06T04:49:41', 'first': '2023-10-06T04:50:00', 'last': '2023-10-06T05:05:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-10-06T05:05:18.639 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-10-06T05:05:18.639 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-10-06T05:05:18.639 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/test_restart 1M 2023-10-06T05:05:18.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.830+0000 7f5b1499d700 1 Processor -- start 2023-10-06T05:05:18.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.830+0000 7f5b1499d700 1 -- start start 2023-10-06T05:05:18.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.831+0000 7f5b1499d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5b100cca20 0x7f5b100cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:18.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.831+0000 7f5b1499d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b100cfa00 con 0x7f5b100b03c0 2023-10-06T05:05:18.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.831+0000 7f5b1499d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b100cb580 con 0x7f5b100b0790 2023-10-06T05:05:18.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.831+0000 7f5b1499d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b100cf160 con 0x7f5b100cca20 2023-10-06T05:05:18.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.831+0000 7f5b0f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5b100b03c0 0x7f5b100b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:33890/0 (socket says 172.21.15.55:33890) 2023-10-06T05:05:18.834 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.831+0000 7f5b0f7fe700 1 -- 172.21.15.55:0/3048295173 learned_addr learned my addr 172.21.15.55:0/3048295173 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.832+0000 7f5b0e7fc700 1 -- 172.21.15.55:0/3048295173 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239737456 0 0) 0x7f5b100cfa00 con 0x7f5b100b03c0 2023-10-06T05:05:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.832+0000 7f5b0e7fc700 1 -- 172.21.15.55:0/3048295173 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af8003570 con 0x7f5b100b03c0 2023-10-06T05:05:18.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.832+0000 7f5b0ffff700 1 --2- 172.21.15.55:0/3048295173 >> v2:172.21.15.163:3300/0 conn(0x7f5b100cca20 0x7f5b100cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.832+0000 7f5b0e7fc700 1 -- 172.21.15.55:0/3048295173 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 816121619 0 0) 0x7f5af8003570 con 0x7f5b100b03c0 2023-10-06T05:05:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.832+0000 7f5b0e7fc700 1 -- 172.21.15.55:0/3048295173 >> v2:172.21.15.163:3300/0 conn(0x7f5b100cca20 msgr2=0x7f5b100cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.832+0000 7f5b0e7fc700 1 --2- 172.21.15.55:0/3048295173 >> v2:172.21.15.163:3300/0 conn(0x7f5b100cca20 0x7f5b100cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.832+0000 7f5b0e7fc700 1 -- 172.21.15.55:0/3048295173 >> v1:172.21.15.163:6789/0 conn(0x7f5b100b0790 legacy=0x7f5b101591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.832+0000 7f5b0e7fc700 1 -- 172.21.15.55:0/3048295173 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b100cf440 con 0x7f5b100b03c0 2023-10-06T05:05:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.832+0000 7f5b0e7fc700 1 -- 172.21.15.55:0/3048295173 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f5b00001470 con 0x7f5b100b03c0 2023-10-06T05:05:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.832+0000 7f5b0e7fc700 1 -- 172.21.15.55:0/3048295173 >> v1:172.21.15.55:6789/0 conn(0x7f5b100b03c0 legacy=0x7f5b100b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.832+0000 7f5b0e7fc700 1 --2- 172.21.15.55:0/3048295173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5af8003ab0 0x7f5af8003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.832+0000 7f5b0e7fc700 1 --2- 172.21.15.55:0/3048295173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5af8004810 0x7f5af8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.832+0000 7f5b0e7fc700 1 --2- 172.21.15.55:0/3048295173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5af8005210 0x7f5af800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:18.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.833+0000 7f5b0e7fc700 1 -- 172.21.15.55:0/3048295173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b100cf160 con 0x7f5af8003ab0 2023-10-06T05:05:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.833+0000 7f5b0e7fc700 1 -- 172.21.15.55:0/3048295173 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af800c1b0 con 0x7f5af8004810 2023-10-06T05:05:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.833+0000 7f5b0e7fc700 1 -- 172.21.15.55:0/3048295173 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af800c370 con 0x7f5af8005210 2023-10-06T05:05:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.833+0000 7f5b0f7fe700 1 --2- 172.21.15.55:0/3048295173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5af8003ab0 0x7f5af8003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.833+0000 7f5b0effd700 1 --2- 172.21.15.55:0/3048295173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5af8004810 0x7f5af8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.833+0000 7f5b0ffff700 1 --2- 172.21.15.55:0/3048295173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5af8005210 0x7f5af800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.833+0000 7f5b0f7fe700 1 -- 172.21.15.55:0/3048295173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5af8005210 msgr2=0x7f5af800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.833+0000 7f5b0f7fe700 1 --2- 172.21.15.55:0/3048295173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5af8005210 0x7f5af800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.833+0000 7f5b0f7fe700 1 -- 172.21.15.55:0/3048295173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5af8004810 msgr2=0x7f5af8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.833+0000 7f5b0f7fe700 1 --2- 172.21.15.55:0/3048295173 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5af8004810 0x7f5af8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.833+0000 7f5b0f7fe700 1 -- 172.21.15.55:0/3048295173 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af8003570 con 0x7f5af8003ab0 2023-10-06T05:05:18.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.833+0000 7f5b0ffff700 1 --2- 172.21.15.55:0/3048295173 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5af8005210 0x7f5af800ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.833+0000 7f5b0ffff700 1 -- 172.21.15.55:0/3048295173 reap_dead start 2023-10-06T05:05:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.833+0000 7f5b0f7fe700 1 --2- 172.21.15.55:0/3048295173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5af8003ab0 0x7f5af8003ec0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000c2a0 tx=0x7f5b100b1a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e3b9ab19d88463 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.834+0000 7f5b0e7fc700 1 -- 172.21.15.55:0/3048295173 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0000cb70 con 0x7f5af8003ab0 2023-10-06T05:05:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.834+0000 7f5b0e7fc700 1 -- 172.21.15.55:0/3048295173 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0000ccd0 con 0x7f5af8003ab0 2023-10-06T05:05:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.834+0000 7f5b0e7fc700 1 -- 172.21.15.55:0/3048295173 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b00005b70 con 0x7f5af8003ab0 2023-10-06T05:05:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.834+0000 7f5b1499d700 1 -- 172.21.15.55:0/3048295173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5af8003ab0 msgr2=0x7f5af8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.834+0000 7f5b1499d700 1 --2- 172.21.15.55:0/3048295173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5af8003ab0 0x7f5af8003ec0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000c2a0 tx=0x7f5b100b1a10 comp rx=0 tx=0).stop 2023-10-06T05:05:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.834+0000 7f5b1499d700 1 -- 172.21.15.55:0/3048295173 shutdown_connections 2023-10-06T05:05:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.834+0000 7f5b1499d700 1 --2- 172.21.15.55:0/3048295173 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5af8003ab0 0x7f5af8003ec0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.834+0000 7f5b1499d700 1 -- 172.21.15.55:0/3048295173 >> 172.21.15.55:0/3048295173 conn(0x7f5b10063230 msgr2=0x7f5b10063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.834+0000 7f5b1499d700 1 -- 172.21.15.55:0/3048295173 shutdown_connections 2023-10-06T05:05:18.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.834+0000 7f5b1499d700 1 -- 172.21.15.55:0/3048295173 wait complete. 2023-10-06T05:05:18.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.836+0000 7f5b1499d700 1 Processor -- start 2023-10-06T05:05:18.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.836+0000 7f5b1499d700 1 -- start start 2023-10-06T05:05:18.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.837+0000 7f5b1499d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5af8003ab0 0x7f5b10152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:18.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.837+0000 7f5b1499d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b100c66c0 0x7f5b100c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:18.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.837+0000 7f5b1499d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b100d0520 0x7f5b100d0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:18.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.837+0000 7f5b1499d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b10153090 con 0x7f5b100d0520 2023-10-06T05:05:18.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.837+0000 7f5b1499d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b100d0ed0 con 0x7f5af8003ab0 2023-10-06T05:05:18.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.837+0000 7f5b1499d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b100c4c90 con 0x7f5b100c66c0 2023-10-06T05:05:18.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.837+0000 7f5b0ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b100d0520 0x7f5b100d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:18.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.837+0000 7f5b0ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b100d0520 0x7f5b100d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56036/0 (socket says 172.21.15.55:56036) 2023-10-06T05:05:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.837+0000 7f5b0ffff700 1 -- 172.21.15.55:0/2160690077 learned_addr learned my addr 172.21.15.55:0/2160690077 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.837+0000 7f5b0f7fe700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5af8003ab0 0x7f5b10152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.838+0000 7f5b0ffff700 1 -- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b100c66c0 msgr2=0x7f5b100c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.837+0000 7f5b0effd700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b100c66c0 0x7f5b100c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.838+0000 7f5b0ffff700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b100c66c0 0x7f5b100c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.838+0000 7f5b0ffff700 1 -- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5af8003ab0 msgr2=0x7f5b10152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.838+0000 7f5b0ffff700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5af8003ab0 0x7f5b10152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.838+0000 7f5b0ffff700 1 -- 172.21.15.55:0/2160690077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b100c4f70 con 0x7f5b100d0520 2023-10-06T05:05:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.838+0000 7f5b0effd700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b100c66c0 0x7f5b100c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.838+0000 7f5b0ffff700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b100d0520 0x7f5b100d0990 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5b100d04f0 tx=0x7f5b0800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:18.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.838+0000 7f5af77fe700 1 -- 172.21.15.55:0/2160690077 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0801b070 con 0x7f5b100d0520 2023-10-06T05:05:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.838+0000 7f5af77fe700 1 -- 172.21.15.55:0/2160690077 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0800f040 con 0x7f5b100d0520 2023-10-06T05:05:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.838+0000 7f5b1499d700 1 -- 172.21.15.55:0/2160690077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b100c5250 con 0x7f5b100d0520 2023-10-06T05:05:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.838+0000 7f5af77fe700 1 -- 172.21.15.55:0/2160690077 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800e050 con 0x7f5b100d0520 2023-10-06T05:05:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.838+0000 7f5b1499d700 1 -- 172.21.15.55:0/2160690077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b100c6270 con 0x7f5b100d0520 2023-10-06T05:05:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.839+0000 7f5b1499d700 1 -- 172.21.15.55:0/2160690077 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1004c3c0 con 0x7f5b100d0520 2023-10-06T05:05:18.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.840+0000 7f5af77fe700 1 -- 172.21.15.55:0/2160690077 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 87) v1 ==== 84871+0+0 (secure 0 0 0) 0x7f5b08003020 con 0x7f5b100d0520 2023-10-06T05:05:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.841+0000 7f5af77fe700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5afc0670a0 0x7f5afc069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:18.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.841+0000 7f5af77fe700 1 -- 172.21.15.55:0/2160690077 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 7053+0+0 (secure 0 0 0) 0x7f5b080852b0 con 0x7f5b100d0520 2023-10-06T05:05:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.841+0000 7f5b0f7fe700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5afc0670a0 0x7f5afc069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:18.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.842+0000 7f5b0f7fe700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5afc0670a0 0x7f5afc069550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5b00007c00 tx=0x7f5b00015000 comp rx=0 tx=0).ready entity=mgr.8989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:18.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:18.845+0000 7f5af77fe700 1 -- 172.21.15.55:0/2160690077 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b08051d20 con 0x7f5b100d0520 2023-10-06T05:05:19.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:19.074+0000 7f5b1499d700 1 -- 172.21.15.55:0/2160690077 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b100cf720 con 0x7f5afc0670a0 2023-10-06T05:05:19.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:19.148+0000 7f5af77fe700 1 -- 172.21.15.55:0/2160690077 <== mgr.8989 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f5b100cf720 con 0x7f5afc0670a0 2023-10-06T05:05:19.150 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-10-06T05:05:19.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:19.150+0000 7f5b1499d700 1 -- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5afc0670a0 msgr2=0x7f5afc069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:19.152 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:19.150+0000 7f5b1499d700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5afc0670a0 0x7f5afc069550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5b00007c00 tx=0x7f5b00015000 comp rx=0 tx=0).stop 2023-10-06T05:05:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:19.150+0000 7f5b1499d700 1 -- 172.21.15.55:0/2160690077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b100d0520 msgr2=0x7f5b100d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:19.150+0000 7f5b1499d700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b100d0520 0x7f5b100d0990 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f5b100d04f0 tx=0x7f5b0800b040 comp rx=0 tx=0).stop 2023-10-06T05:05:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:19.150+0000 7f5b1499d700 1 -- 172.21.15.55:0/2160690077 shutdown_connections 2023-10-06T05:05:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:19.150+0000 7f5b1499d700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5afc0670a0 0x7f5afc069550 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:19.150+0000 7f5b1499d700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5b100d0520 0x7f5b100d0990 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:19.150+0000 7f5b1499d700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5af8003ab0 0x7f5b10152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:19.150+0000 7f5b1499d700 1 --2- 172.21.15.55:0/2160690077 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5b100c66c0 0x7f5b100c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:19.150+0000 7f5b1499d700 1 -- 172.21.15.55:0/2160690077 >> 172.21.15.55:0/2160690077 conn(0x7f5b10063230 msgr2=0x7f5b100b5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:19.150+0000 7f5b1499d700 1 -- 172.21.15.55:0/2160690077 shutdown_connections 2023-10-06T05:05:19.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:19.150+0000 7f5b1499d700 1 -- 172.21.15.55:0/2160690077 wait complete. 2023-10-06T05:05:19.169 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-10-06T05:05:19.170 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:19.170 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-06T05:05:19.207 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_50_00_UTC 2023-10-06T05:05:19.208 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_51_00_UTC 2023-10-06T05:05:19.208 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_52_00_UTC 2023-10-06T05:05:19.208 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_53_00_UTC 2023-10-06T05:05:19.208 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_54_00_UTC 2023-10-06T05:05:19.208 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_55_00_UTC 2023-10-06T05:05:19.208 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_56_00_UTC 2023-10-06T05:05:19.209 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_57_00_UTC 2023-10-06T05:05:19.209 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_58_00_UTC 2023-10-06T05:05:19.209 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-04_59_00_UTC 2023-10-06T05:05:19.209 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_00_00_UTC 2023-10-06T05:05:19.209 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_01_00_UTC 2023-10-06T05:05:19.209 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_02_00_UTC 2023-10-06T05:05:19.209 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_03_00_UTC 2023-10-06T05:05:19.209 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_04_00_UTC 2023-10-06T05:05:19.209 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_05_00_UTC 2023-10-06T05:05:19.210 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_50_00_UTC 2023-10-06T05:05:19.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:19.210 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_50_00_UTC) 2023-10-06T05:05:19.295 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_51_00_UTC 2023-10-06T05:05:19.295 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:19.295 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_51_00_UTC) 2023-10-06T05:05:19.378 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_52_00_UTC 2023-10-06T05:05:19.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:19.379 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_52_00_UTC) 2023-10-06T05:05:19.462 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_53_00_UTC 2023-10-06T05:05:19.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:19.463 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_53_00_UTC) 2023-10-06T05:05:19.547 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_54_00_UTC 2023-10-06T05:05:19.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:19.547 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_54_00_UTC) 2023-10-06T05:05:19.631 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_55_00_UTC 2023-10-06T05:05:19.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:19.632 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_55_00_UTC) 2023-10-06T05:05:19.715 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_56_00_UTC 2023-10-06T05:05:19.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:19.715 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_56_00_UTC) 2023-10-06T05:05:19.809 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_57_00_UTC 2023-10-06T05:05:19.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:19.809 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_57_00_UTC) 2023-10-06T05:05:19.852 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_58_00_UTC 2023-10-06T05:05:19.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:19.852 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_58_00_UTC) 2023-10-06T05:05:19.936 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_59_00_UTC 2023-10-06T05:05:19.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:19.937 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-04_59_00_UTC) 2023-10-06T05:05:20.020 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-05_00_00_UTC 2023-10-06T05:05:20.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:20.020 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-05_00_00_UTC) 2023-10-06T05:05:20.106 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-05_01_00_UTC 2023-10-06T05:05:20.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:20.106 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-05_01_00_UTC) 2023-10-06T05:05:20.191 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-05_02_00_UTC 2023-10-06T05:05:20.191 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:20.191 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-05_02_00_UTC) 2023-10-06T05:05:20.275 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-05_03_00_UTC 2023-10-06T05:05:20.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:20.275 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-05_03_00_UTC) 2023-10-06T05:05:20.360 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-05_04_00_UTC 2023-10-06T05:05:20.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:20.360 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-05_04_00_UTC) 2023-10-06T05:05:20.453 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-05_05_00_UTC 2023-10-06T05:05:20.453 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:20.453 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-06-05_05_00_UTC) 2023-10-06T05:05:20.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:20.497 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-10-06T05:05:20.577 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T05:05:20.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.807+0000 7fa73759e700 1 Processor -- start 2023-10-06T05:05:20.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.807+0000 7fa73759e700 1 -- start start 2023-10-06T05:05:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.809+0000 7fa73759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa7380ae7e0 0x7fa7380c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.809+0000 7fa73759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa73806ebd0 con 0x7fa7380cb1a0 2023-10-06T05:05:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.809+0000 7fa73759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa73815aa70 con 0x7fa73806f060 2023-10-06T05:05:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.809+0000 7fa73759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73815b2e0 con 0x7fa7380ae7e0 2023-10-06T05:05:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.809+0000 7fa73659c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa7380cb1a0 0x7fa73815a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:33896/0 (socket says 172.21.15.55:33896) 2023-10-06T05:05:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.809+0000 7fa73659c700 1 -- 172.21.15.55:0/399317151 learned_addr learned my addr 172.21.15.55:0/399317151 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:20.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.809+0000 7fa736d9d700 1 --2- 172.21.15.55:0/399317151 >> v2:172.21.15.163:3300/0 conn(0x7fa7380ae7e0 0x7fa7380c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.809+0000 7fa73559a700 1 -- 172.21.15.55:0/399317151 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839516755 0 0) 0x7fa73806ebd0 con 0x7fa7380cb1a0 2023-10-06T05:05:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.810+0000 7fa73559a700 1 -- 172.21.15.55:0/399317151 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa720003570 con 0x7fa7380cb1a0 2023-10-06T05:05:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.810+0000 7fa73559a700 1 -- 172.21.15.55:0/399317151 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935379875 0 0) 0x7fa73815aa70 con 0x7fa73806f060 2023-10-06T05:05:20.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.810+0000 7fa73559a700 1 -- 172.21.15.55:0/399317151 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa73806ebd0 con 0x7fa73806f060 2023-10-06T05:05:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.810+0000 7fa736d9d700 1 -- 172.21.15.55:0/399317151 >> v1:172.21.15.163:6789/0 conn(0x7fa73806f060 legacy=0x7fa7380ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.810+0000 7fa736d9d700 1 -- 172.21.15.55:0/399317151 >> v1:172.21.15.55:6789/0 conn(0x7fa7380cb1a0 legacy=0x7fa73815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.810+0000 7fa736d9d700 1 -- 172.21.15.55:0/399317151 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73815ad20 con 0x7fa7380ae7e0 2023-10-06T05:05:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.810+0000 7fa736d9d700 1 --2- 172.21.15.55:0/399317151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7380ae7e0 0x7fa7380c9db0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa730009e60 tx=0x7fa730009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2978f144e09efdf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.810+0000 7fa73559a700 1 -- 172.21.15.55:0/399317151 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73000e070 con 0x7fa7380ae7e0 2023-10-06T05:05:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.810+0000 7fa73559a700 1 -- 172.21.15.55:0/399317151 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73000bb40 con 0x7fa7380ae7e0 2023-10-06T05:05:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.810+0000 7fa73559a700 1 -- 172.21.15.55:0/399317151 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa730012430 con 0x7fa7380ae7e0 2023-10-06T05:05:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.811+0000 7fa73759e700 1 -- 172.21.15.55:0/399317151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7380ae7e0 msgr2=0x7fa7380c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.811+0000 7fa73759e700 1 --2- 172.21.15.55:0/399317151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7380ae7e0 0x7fa7380c9db0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa730009e60 tx=0x7fa730009780 comp rx=0 tx=0).stop 2023-10-06T05:05:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.811+0000 7fa73759e700 1 -- 172.21.15.55:0/399317151 shutdown_connections 2023-10-06T05:05:20.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.811+0000 7fa73759e700 1 --2- 172.21.15.55:0/399317151 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa7380ae7e0 0x7fa7380c9db0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:20.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.811+0000 7fa73759e700 1 -- 172.21.15.55:0/399317151 >> 172.21.15.55:0/399317151 conn(0x7fa7380cc1e0 msgr2=0x7fa7380cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:20.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.811+0000 7fa73759e700 1 -- 172.21.15.55:0/399317151 shutdown_connections 2023-10-06T05:05:20.815 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.811+0000 7fa73759e700 1 -- 172.21.15.55:0/399317151 wait complete. 2023-10-06T05:05:20.816 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.813+0000 7fa73759e700 1 Processor -- start 2023-10-06T05:05:20.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.813+0000 7fa73759e700 1 -- start start 2023-10-06T05:05:20.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.813+0000 7fa73759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa73806f060 0x7fa738151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:20.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.813+0000 7fa73759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7380ae7e0 0x7fa738157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:20.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa73759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380cb1a0 0x7fa738157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:20.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa73759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa738165d60 con 0x7fa7380ae7e0 2023-10-06T05:05:20.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa73759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa738165750 con 0x7fa73806f060 2023-10-06T05:05:20.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa73759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7380c5df0 con 0x7fa7380cb1a0 2023-10-06T05:05:20.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa735d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7380ae7e0 0x7fa738157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:20.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa735d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7380ae7e0 0x7fa738157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56038/0 (socket says 172.21.15.55:56038) 2023-10-06T05:05:20.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa735d9b700 1 -- 172.21.15.55:0/3782429800 learned_addr learned my addr 172.21.15.55:0/3782429800 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa73659c700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa73806f060 0x7fa738151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa736d9d700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380cb1a0 0x7fa738157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa735d9b700 1 -- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380cb1a0 msgr2=0x7fa738157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa735d9b700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380cb1a0 0x7fa738157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa735d9b700 1 -- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa73806f060 msgr2=0x7fa738151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa735d9b700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa73806f060 0x7fa738151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa735d9b700 1 -- 172.21.15.55:0/3782429800 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa730009a20 con 0x7fa7380ae7e0 2023-10-06T05:05:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa73659c700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa73806f060 0x7fa738151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.814+0000 7fa735d9b700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7380ae7e0 0x7fa738157600 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00c790 tx=0x7fa72c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.815+0000 7fa7267fc700 1 -- 172.21.15.55:0/3782429800 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c012630 con 0x7fa7380ae7e0 2023-10-06T05:05:20.818 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.815+0000 7fa7267fc700 1 -- 172.21.15.55:0/3782429800 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c012790 con 0x7fa7380ae7e0 2023-10-06T05:05:20.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.815+0000 7fa7267fc700 1 -- 172.21.15.55:0/3782429800 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c012b30 con 0x7fa7380ae7e0 2023-10-06T05:05:20.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.815+0000 7fa73759e700 1 -- 172.21.15.55:0/3782429800 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7381659f0 con 0x7fa7380ae7e0 2023-10-06T05:05:20.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.815+0000 7fa73759e700 1 -- 172.21.15.55:0/3782429800 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa738151a80 con 0x7fa7380ae7e0 2023-10-06T05:05:20.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.816+0000 7fa73759e700 1 -- 172.21.15.55:0/3782429800 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa738165ed0 con 0x7fa7380ae7e0 2023-10-06T05:05:20.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.817+0000 7fa7267fc700 1 -- 172.21.15.55:0/3782429800 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 87) v1 ==== 84871+0+0 (secure 0 0 0) 0x7fa72c01b400 con 0x7fa7380ae7e0 2023-10-06T05:05:20.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.818+0000 7fa7267fc700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa7180670a0 0x7fa718069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:20.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.818+0000 7fa7267fc700 1 -- 172.21.15.55:0/3782429800 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 7141+0+0 (secure 0 0 0) 0x7fa72c0848d0 con 0x7fa7380ae7e0 2023-10-06T05:05:20.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.818+0000 7fa73659c700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa7180670a0 0x7fa718069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:20.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.819+0000 7fa73659c700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa7180670a0 0x7fa718069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa7280058d0 tx=0x7fa728009040 comp rx=0 tx=0).ready entity=mgr.8989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:20.824 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:20.822+0000 7fa7267fc700 1 -- 172.21.15.55:0/3782429800 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa72c0513f0 con 0x7fa7380ae7e0 2023-10-06T05:05:21.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.088+0000 7fa73759e700 1 -- 172.21.15.55:0/3782429800 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fa73815b000 con 0x7fa7380ae7e0 2023-10-06T05:05:21.801 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.799+0000 7fa7267fc700 1 -- 172.21.15.55:0/3782429800 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 88) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fa72c0517c0 con 0x7fa7380ae7e0 2023-10-06T05:05:21.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.801+0000 7fa7267fc700 1 -- 172.21.15.55:0/3782429800 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v88) v1 ==== 93+0+0 (secure 0 0 0) 0x7fa72c0513f0 con 0x7fa7380ae7e0 2023-10-06T05:05:21.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.805+0000 7fa73759e700 1 -- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa7180670a0 msgr2=0x7fa718069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:21.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.805+0000 7fa73759e700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa7180670a0 0x7fa718069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa7280058d0 tx=0x7fa728009040 comp rx=0 tx=0).stop 2023-10-06T05:05:21.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.805+0000 7fa73759e700 1 -- 172.21.15.55:0/3782429800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7380ae7e0 msgr2=0x7fa738157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:21.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.805+0000 7fa73759e700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7380ae7e0 0x7fa738157600 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00c790 tx=0x7fa72c007e10 comp rx=0 tx=0).stop 2023-10-06T05:05:21.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.806+0000 7fa73759e700 1 -- 172.21.15.55:0/3782429800 shutdown_connections 2023-10-06T05:05:21.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.806+0000 7fa73759e700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa7180670a0 0x7fa718069550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:21.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.806+0000 7fa73759e700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa7380cb1a0 0x7fa738157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.806+0000 7fa73759e700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa73806f060 0x7fa738151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.806+0000 7fa73759e700 1 --2- 172.21.15.55:0/3782429800 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa7380ae7e0 0x7fa738157600 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.806+0000 7fa73759e700 1 -- 172.21.15.55:0/3782429800 >> 172.21.15.55:0/3782429800 conn(0x7fa7380cc1e0 msgr2=0x7fa7380c6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.806+0000 7fa73759e700 1 -- 172.21.15.55:0/3782429800 shutdown_connections 2023-10-06T05:05:21.809 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:21.807+0000 7fa73759e700 1 -- 172.21.15.55:0/3782429800 wait complete. 2023-10-06T05:05:21.843 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:05:21.843 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:05:21.943 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:05:22.024 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:05:22.024 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:05:22.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:22.025 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:05:22.117 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T05:05:22.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:05:22.117 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:05:22.156 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:05:22.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.384+0000 7f7372d9d700 1 Processor -- start 2023-10-06T05:05:22.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.384+0000 7f7372d9d700 1 -- start start 2023-10-06T05:05:22.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.386+0000 7f7372d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f736c1595a0 0x7f736c0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:22.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.386+0000 7f7372d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f736c0c6c60 0x7f736c0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:22.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.386+0000 7f7372d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f736c0b1de0 0x7f736c0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:22.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.386+0000 7f7372d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f736c0b2b90 con 0x7f736c0c6c60 2023-10-06T05:05:22.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.386+0000 7f7372d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f736c0b1410 con 0x7f736c0b1de0 2023-10-06T05:05:22.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.386+0000 7f7372d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736c063230 con 0x7f736c1595a0 2023-10-06T05:05:22.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.386+0000 7f737159a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f736c0c6c60 0x7f736c0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:22.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.386+0000 7f737159a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f736c0c6c60 0x7f736c0c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56070/0 (socket says 172.21.15.55:56070) 2023-10-06T05:05:22.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.386+0000 7f737159a700 1 -- 172.21.15.55:0/3614021420 learned_addr learned my addr 172.21.15.55:0/3614021420 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:22.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.386+0000 7f7371d9b700 1 --2- 172.21.15.55:0/3614021420 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f736c1595a0 0x7f736c0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:22.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.386+0000 7f737259c700 1 --2- 172.21.15.55:0/3614021420 >> v2:172.21.15.163:3300/0 conn(0x7f736c0b1de0 0x7f736c0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:22.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.387+0000 7f737159a700 1 -- 172.21.15.55:0/3614021420 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f736c1595a0 msgr2=0x7f736c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:22.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.387+0000 7f737159a700 1 --2- 172.21.15.55:0/3614021420 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f736c1595a0 0x7f736c0b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.387+0000 7f737159a700 1 -- 172.21.15.55:0/3614021420 >> v2:172.21.15.163:3300/0 conn(0x7f736c0b1de0 msgr2=0x7f736c0c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:22.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.387+0000 7f737159a700 1 --2- 172.21.15.55:0/3614021420 >> v2:172.21.15.163:3300/0 conn(0x7f736c0b1de0 0x7f736c0c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.387+0000 7f737159a700 1 -- 172.21.15.55:0/3614021420 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c0634f0 con 0x7f736c0c6c60 2023-10-06T05:05:22.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.387+0000 7f737159a700 1 --2- 172.21.15.55:0/3614021420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f736c0c6c60 0x7f736c0c8120 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f736c15d3c0 tx=0x7f735c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79116036e343b954 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:22.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.387+0000 7f7370d99700 1 -- 172.21.15.55:0/3614021420 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c00b640 con 0x7f736c0c6c60 2023-10-06T05:05:22.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.387+0000 7f7370d99700 1 -- 172.21.15.55:0/3614021420 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735c00b7a0 con 0x7f736c0c6c60 2023-10-06T05:05:22.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.387+0000 7f7370d99700 1 -- 172.21.15.55:0/3614021420 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c00bb60 con 0x7f736c0c6c60 2023-10-06T05:05:22.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.388+0000 7f7372d9d700 1 -- 172.21.15.55:0/3614021420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f736c0c6c60 msgr2=0x7f736c0c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:22.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.388+0000 7f7372d9d700 1 --2- 172.21.15.55:0/3614021420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f736c0c6c60 0x7f736c0c8120 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f736c15d3c0 tx=0x7f735c009250 comp rx=0 tx=0).stop 2023-10-06T05:05:22.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.388+0000 7f7372d9d700 1 -- 172.21.15.55:0/3614021420 shutdown_connections 2023-10-06T05:05:22.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.388+0000 7f7372d9d700 1 --2- 172.21.15.55:0/3614021420 >> v2:172.21.15.163:3300/0 conn(0x7f736c0b1de0 0x7f736c0c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.388+0000 7f7372d9d700 1 --2- 172.21.15.55:0/3614021420 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f736c1595a0 0x7f736c0b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.388+0000 7f7372d9d700 1 --2- 172.21.15.55:0/3614021420 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f736c0c6c60 0x7f736c0c8120 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.388+0000 7f7372d9d700 1 -- 172.21.15.55:0/3614021420 >> 172.21.15.55:0/3614021420 conn(0x7f736c06ddb0 msgr2=0x7f736c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:22.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.388+0000 7f7372d9d700 1 -- 172.21.15.55:0/3614021420 shutdown_connections 2023-10-06T05:05:22.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.388+0000 7f7372d9d700 1 -- 172.21.15.55:0/3614021420 wait complete. 2023-10-06T05:05:22.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.389+0000 7f7372d9d700 1 Processor -- start 2023-10-06T05:05:22.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.390+0000 7f7372d9d700 1 -- start start 2023-10-06T05:05:22.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.390+0000 7f7372d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f736c0b1de0 0x7f736c13e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:22.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.390+0000 7f7372d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f736c0c6c60 0x7f736c14c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:22.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f7372d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f736c1595a0 0x7f736c06e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:22.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f7372d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c16aa40 con 0x7f736c1595a0 2023-10-06T05:05:22.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f7372d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736c13eac0 con 0x7f736c0c6c60 2023-10-06T05:05:22.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f7372d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736c16aed0 con 0x7f736c0b1de0 2023-10-06T05:05:22.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f737259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f736c1595a0 0x7f736c06e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:22.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f7371d9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f736c0b1de0 0x7f736c13e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:22.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f737159a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f736c0c6c60 0x7f736c14c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:22.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f737259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f736c1595a0 0x7f736c06e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56082/0 (socket says 172.21.15.55:56082) 2023-10-06T05:05:22.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f737259c700 1 -- 172.21.15.55:0/359044150 learned_addr learned my addr 172.21.15.55:0/359044150 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:22.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f7371d9b700 1 -- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f736c0c6c60 msgr2=0x7f736c14c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:22.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f7371d9b700 1 --2- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f736c0c6c60 0x7f736c14c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f7371d9b700 1 -- 172.21.15.55:0/359044150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f736c1595a0 msgr2=0x7f736c06e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:22.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f7371d9b700 1 --2- 172.21.15.55:0/359044150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f736c1595a0 0x7f736c06e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f7371d9b700 1 -- 172.21.15.55:0/359044150 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c0094f0 con 0x7f736c0b1de0 2023-10-06T05:05:22.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f737259c700 1 --2- 172.21.15.55:0/359044150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f736c1595a0 0x7f736c06e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:22.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.391+0000 7f737159a700 1 --2- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f736c0c6c60 0x7f736c14c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:22.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.392+0000 7f7371d9b700 1 --2- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f736c0b1de0 0x7f736c13e580 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f736800e850 tx=0x7f73680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:22.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.392+0000 7f7361ffb700 1 -- 172.21.15.55:0/359044150 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7368009430 con 0x7f736c0b1de0 2023-10-06T05:05:22.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.392+0000 7f7361ffb700 1 -- 172.21.15.55:0/359044150 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7368013680 con 0x7f736c0b1de0 2023-10-06T05:05:22.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.392+0000 7f7361ffb700 1 -- 172.21.15.55:0/359044150 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7368013a40 con 0x7f736c0b1de0 2023-10-06T05:05:22.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.392+0000 7f7372d9d700 1 -- 172.21.15.55:0/359044150 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f736c16a110 con 0x7f736c0b1de0 2023-10-06T05:05:22.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.392+0000 7f7372d9d700 1 -- 172.21.15.55:0/359044150 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f736c16a6d0 con 0x7f736c0b1de0 2023-10-06T05:05:22.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.394+0000 7f7372d9d700 1 -- 172.21.15.55:0/359044150 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736c16ad20 con 0x7f736c0b1de0 2023-10-06T05:05:22.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.394+0000 7f7361ffb700 1 -- 172.21.15.55:0/359044150 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 88) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f73680123d0 con 0x7f736c0b1de0 2023-10-06T05:05:22.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.395+0000 7f7361ffb700 1 --2- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7350067050 0x7f7350069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:22.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.395+0000 7f7361ffb700 1 -- 172.21.15.55:0/359044150 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 7141+0+0 (secure 0 0 0) 0x7f7368084a10 con 0x7f736c0b1de0 2023-10-06T05:05:22.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.395+0000 7f737159a700 1 -- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7350067050 msgr2=0x7f7350069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:22.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.395+0000 7f737159a700 1 --2- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7350067050 0x7f7350069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:22.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.402+0000 7f7361ffb700 1 -- 172.21.15.55:0/359044150 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7368051530 con 0x7f736c0b1de0 2023-10-06T05:05:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.596+0000 7f737159a700 1 -- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7350067050 msgr2=0x7f7350069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:22.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.596+0000 7f737159a700 1 --2- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7350067050 0x7f7350069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:22.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.675+0000 7f7372d9d700 1 -- 172.21.15.55:0/359044150 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f736c04c3c0 con 0x7f736c0b1de0 2023-10-06T05:05:22.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.676+0000 7f7361ffb700 1 -- 172.21.15.55:0/359044150 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f7368054b50 con 0x7f736c0b1de0 2023-10-06T05:05:22.678 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:22.678 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7682,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1565234675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1565234675},{"type":"v1","addr":"172.21.15.55:6834","nonce":1565234675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3042024952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3042024952},{"type":"v1","addr":"172.21.15.163:6835","nonce":3042024952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8737,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/304924610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":304924610},{"type":"v1","addr":"172.21.15.55:6835","nonce":304924610}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:49:19.807866+0000","modified":"2023-10-06T04:49:23.339948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8740":{"gid":8740,"name":"b","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6834/3095914721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3095914721},{"type":"v1","addr":"172.21.15.163:6834","nonce":3095914721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:05:22.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.678+0000 7f7372d9d700 1 -- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7350067050 msgr2=0x7f7350069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:22.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.679+0000 7f7372d9d700 1 --2- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7350067050 0x7f7350069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.679+0000 7f7372d9d700 1 -- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f736c0b1de0 msgr2=0x7f736c13e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:22.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.679+0000 7f7372d9d700 1 --2- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f736c0b1de0 0x7f736c13e580 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f736800e850 tx=0x7f73680091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.679+0000 7f7372d9d700 1 -- 172.21.15.55:0/359044150 shutdown_connections 2023-10-06T05:05:22.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.679+0000 7f7372d9d700 1 --2- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7350067050 0x7f7350069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.679+0000 7f7372d9d700 1 --2- 172.21.15.55:0/359044150 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f736c1595a0 0x7f736c06e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.679+0000 7f7372d9d700 1 --2- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f736c0b1de0 0x7f736c13e580 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.679+0000 7f7372d9d700 1 --2- 172.21.15.55:0/359044150 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f736c0c6c60 0x7f736c14c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.679+0000 7f7372d9d700 1 -- 172.21.15.55:0/359044150 >> 172.21.15.55:0/359044150 conn(0x7f736c06ddb0 msgr2=0x7f736c157270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:22.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.679+0000 7f7372d9d700 1 -- 172.21.15.55:0/359044150 shutdown_connections 2023-10-06T05:05:22.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.679+0000 7f7372d9d700 1 -- 172.21.15.55:0/359044150 wait complete. 2023-10-06T05:05:22.682 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 87 2023-10-06T05:05:22.699 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:05:22.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.890+0000 7fa5d259d700 1 Processor -- start 2023-10-06T05:05:22.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.890+0000 7fa5d259d700 1 -- start start 2023-10-06T05:05:22.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.891+0000 7fa5d259d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa5cc159420 0x7fa5cc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:22.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.891+0000 7fa5d259d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa5cc1597f0 0x7fa5cc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:22.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.891+0000 7fa5d259d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc0b11d0 con 0x7fa5cc15d230 2023-10-06T05:05:22.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.891+0000 7fa5d259d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5cc0b14b0 con 0x7fa5cc1597f0 2023-10-06T05:05:22.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.891+0000 7fa5d259d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5cc0b1790 con 0x7fa5cc159420 2023-10-06T05:05:22.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.892+0000 7fa5d0d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa5cc1597f0 0x7fa5cc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:22.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.892+0000 7fa5d0d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa5cc1597f0 0x7fa5cc0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56086/0 (socket says 172.21.15.55:56086) 2023-10-06T05:05:22.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.892+0000 7fa5d0d9a700 1 -- 172.21.15.55:0/1378492144 learned_addr learned my addr 172.21.15.55:0/1378492144 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:22.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.892+0000 7fa5d1d9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa5cc15d230 0x7fa5cc15d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:33916/0 (socket says 172.21.15.55:33916) 2023-10-06T05:05:22.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.892+0000 7fa5d159b700 1 --2- 172.21.15.55:0/1378492144 >> v2:172.21.15.163:3300/0 conn(0x7fa5cc159420 0x7fa5cc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:22.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.892+0000 7fa5d0d9a700 1 -- 172.21.15.55:0/1378492144 >> v2:172.21.15.163:3300/0 conn(0x7fa5cc159420 msgr2=0x7fa5cc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:22.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.892+0000 7fa5d0d9a700 1 --2- 172.21.15.55:0/1378492144 >> v2:172.21.15.163:3300/0 conn(0x7fa5cc159420 0x7fa5cc0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.892+0000 7fa5d0d9a700 1 -- 172.21.15.55:0/1378492144 >> v1:172.21.15.55:6789/0 conn(0x7fa5cc15d230 legacy=0x7fa5cc15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:22.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.892+0000 7fa5d0d9a700 1 -- 172.21.15.55:0/1378492144 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc0b1a70 con 0x7fa5cc1597f0 2023-10-06T05:05:22.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.892+0000 7fa5d159b700 1 --2- 172.21.15.55:0/1378492144 >> v2:172.21.15.163:3300/0 conn(0x7fa5cc159420 0x7fa5cc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:22.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.892+0000 7fa5c3fff700 1 -- 172.21.15.55:0/1378492144 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3913463129 0 0) 0x7fa5cc0b11d0 con 0x7fa5cc15d230 2023-10-06T05:05:22.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.892+0000 7fa5d0d9a700 1 --2- 172.21.15.55:0/1378492144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5cc1597f0 0x7fa5cc0b2270 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0b2f20 tx=0x7fa5bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd59bac46685fdd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:22.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.893+0000 7fa5c3fff700 1 -- 172.21.15.55:0/1378492144 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00b860 con 0x7fa5cc1597f0 2023-10-06T05:05:22.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.893+0000 7fa5c3fff700 1 -- 172.21.15.55:0/1378492144 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc00b9c0 con 0x7fa5cc1597f0 2023-10-06T05:05:22.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.893+0000 7fa5c3fff700 1 -- 172.21.15.55:0/1378492144 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00bd60 con 0x7fa5cc1597f0 2023-10-06T05:05:22.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.893+0000 7fa5d259d700 1 -- 172.21.15.55:0/1378492144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5cc1597f0 msgr2=0x7fa5cc0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:22.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.893+0000 7fa5d259d700 1 --2- 172.21.15.55:0/1378492144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5cc1597f0 0x7fa5cc0b2270 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc0b2f20 tx=0x7fa5bc009250 comp rx=0 tx=0).stop 2023-10-06T05:05:22.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.893+0000 7fa5d259d700 1 -- 172.21.15.55:0/1378492144 shutdown_connections 2023-10-06T05:05:22.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.893+0000 7fa5d259d700 1 --2- 172.21.15.55:0/1378492144 >> v2:172.21.15.163:3300/0 conn(0x7fa5cc159420 0x7fa5cc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.893+0000 7fa5d259d700 1 --2- 172.21.15.55:0/1378492144 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5cc1597f0 0x7fa5cc0b2270 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.893+0000 7fa5d259d700 1 -- 172.21.15.55:0/1378492144 >> 172.21.15.55:0/1378492144 conn(0x7fa5cc0d0660 msgr2=0x7fa5cc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:22.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.894+0000 7fa5d259d700 1 -- 172.21.15.55:0/1378492144 shutdown_connections 2023-10-06T05:05:22.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.894+0000 7fa5d259d700 1 -- 172.21.15.55:0/1378492144 wait complete. 2023-10-06T05:05:22.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.895+0000 7fa5d259d700 1 Processor -- start 2023-10-06T05:05:22.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.895+0000 7fa5d259d700 1 -- start start 2023-10-06T05:05:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.896+0000 7fa5d259d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5cc159420 0x7fa5cc0cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.896+0000 7fa5d259d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5cc1597f0 0x7fa5cc1540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.896+0000 7fa5d259d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5cc15d230 0x7fa5cc0cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.896+0000 7fa5d259d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0b5ec0 con 0x7fa5cc159420 2023-10-06T05:05:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.896+0000 7fa5d259d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0b4f60 con 0x7fa5cc15d230 2023-10-06T05:05:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.896+0000 7fa5d259d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc0cc390 con 0x7fa5cc1597f0 2023-10-06T05:05:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.896+0000 7fa5d159b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5cc159420 0x7fa5cc0cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.896+0000 7fa5d159b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5cc159420 0x7fa5cc0cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:56098/0 (socket says 172.21.15.55:56098) 2023-10-06T05:05:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.896+0000 7fa5d159b700 1 -- 172.21.15.55:0/1621256629 learned_addr learned my addr 172.21.15.55:0/1621256629 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.897+0000 7fa5d0d9a700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5cc1597f0 0x7fa5cc1540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:22.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.897+0000 7fa5d1d9c700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5cc15d230 0x7fa5cc0cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.897+0000 7fa5d159b700 1 -- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5cc1597f0 msgr2=0x7fa5cc1540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.897+0000 7fa5d159b700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5cc1597f0 0x7fa5cc1540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.897+0000 7fa5d159b700 1 -- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5cc15d230 msgr2=0x7fa5cc0cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.897+0000 7fa5d159b700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5cc15d230 0x7fa5cc0cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.897+0000 7fa5d159b700 1 -- 172.21.15.55:0/1621256629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc0094f0 con 0x7fa5cc159420 2023-10-06T05:05:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.897+0000 7fa5d1d9c700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5cc15d230 0x7fa5cc0cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.897+0000 7fa5d159b700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5cc159420 0x7fa5cc0cbe50 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800e930 tx=0x7fa5c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.897+0000 7fa5c17fa700 1 -- 172.21.15.55:0/1621256629 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8013410 con 0x7fa5cc159420 2023-10-06T05:05:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.897+0000 7fa5c17fa700 1 -- 172.21.15.55:0/1621256629 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c8013570 con 0x7fa5cc159420 2023-10-06T05:05:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.897+0000 7fa5d259d700 1 -- 172.21.15.55:0/1621256629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5cc0b4350 con 0x7fa5cc159420 2023-10-06T05:05:22.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.897+0000 7fa5c17fa700 1 -- 172.21.15.55:0/1621256629 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8013910 con 0x7fa5cc159420 2023-10-06T05:05:22.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.898+0000 7fa5d259d700 1 -- 172.21.15.55:0/1621256629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5cc0b4910 con 0x7fa5cc159420 2023-10-06T05:05:22.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.899+0000 7fa5d259d700 1 -- 172.21.15.55:0/1621256629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5cc04c3c0 con 0x7fa5cc159420 2023-10-06T05:05:22.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.900+0000 7fa5c17fa700 1 -- 172.21.15.55:0/1621256629 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 88) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fa5c8013cc0 con 0x7fa5cc159420 2023-10-06T05:05:22.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.900+0000 7fa5c17fa700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa5b0067050 0x7fa5b0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:22.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.901+0000 7fa5c17fa700 1 -- 172.21.15.55:0/1621256629 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 7141+0+0 (secure 0 0 0) 0x7fa5c8084b20 con 0x7fa5cc159420 2023-10-06T05:05:22.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.901+0000 7fa5d0d9a700 1 -- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa5b0067050 msgr2=0x7fa5b0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:22.903 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.901+0000 7fa5d0d9a700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa5b0067050 0x7fa5b0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:22.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:22.909+0000 7fa5c17fa700 1 -- 172.21.15.55:0/1621256629 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5c80515d0 con 0x7fa5cc159420 2023-10-06T05:05:23.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.102+0000 7fa5d0d9a700 1 -- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa5b0067050 msgr2=0x7fa5b0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:23.104 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.102+0000 7fa5d0d9a700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa5b0067050 0x7fa5b0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:23.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.117+0000 7fa5d259d700 1 -- 172.21.15.55:0/1621256629 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5cc0b1d50 con 0x7fa5cc159420 2023-10-06T05:05:23.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.119+0000 7fa5c17fa700 1 -- 172.21.15.55:0/1621256629 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+18584 (secure 0 0 0) 0x7fa5c8054bf0 con 0x7fa5cc159420 2023-10-06T05:05:23.121 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:23.121 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":180,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:05:21.780167+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:49:17.992792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T04:49:18.785942+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"180","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":32,"snap_epoch":180,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.12","mappings":[{"from":3,"to":0}]},{"pgid":"22.23","mappings":[{"from":3,"to":1}]},{"pgid":"22.37","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/1915465424":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/3085093567":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/2727475042":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/167943248":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/3594252861":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/3141284388":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/2833364221":"2023-10-07T04:49:40.797751+0000","172.21.15.163:6836/85191":"2023-10-07T04:49:40.797751+0000","172.21.15.163:0/4127328436":"2023-10-07T04:49:40.797751+0000","172.21.15.163:6837/85191":"2023-10-07T04:49:40.797751+0000","172.21.15.163:0/2876114056":"2023-10-07T04:49:40.797751+0000","172.21.15.163:0/3803111782":"2023-10-07T04:49:40.797751+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":15},{"begin":18,"length":15}]}],"new_removed_snaps":[{"pool":22,"snaps":[{"begin":7,"length":10},{"begin":23,"length":10}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:05:23.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.121+0000 7fa5d259d700 1 -- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa5b0067050 msgr2=0x7fa5b0069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:23.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.121+0000 7fa5d259d700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa5b0067050 0x7fa5b0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.121+0000 7fa5d259d700 1 -- 172.21.15.55:0/1621256629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5cc159420 msgr2=0x7fa5cc0cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:23.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.121+0000 7fa5d259d700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5cc159420 0x7fa5cc0cbe50 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800e930 tx=0x7fa5c8008040 comp rx=0 tx=0).stop 2023-10-06T05:05:23.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.121+0000 7fa5d259d700 1 -- 172.21.15.55:0/1621256629 shutdown_connections 2023-10-06T05:05:23.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.121+0000 7fa5d259d700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa5b0067050 0x7fa5b0069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.121+0000 7fa5d259d700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5cc15d230 0x7fa5cc0cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.121+0000 7fa5d259d700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5cc159420 0x7fa5cc0cbe50 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.121+0000 7fa5d259d700 1 --2- 172.21.15.55:0/1621256629 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5cc1597f0 0x7fa5cc1540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.121+0000 7fa5d259d700 1 -- 172.21.15.55:0/1621256629 >> 172.21.15.55:0/1621256629 conn(0x7fa5cc0d0660 msgr2=0x7fa5cc156a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:23.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.121+0000 7fa5d259d700 1 -- 172.21.15.55:0/1621256629 shutdown_connections 2023-10-06T05:05:23.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.121+0000 7fa5d259d700 1 -- 172.21.15.55:0/1621256629 wait complete. 2023-10-06T05:05:23.142 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:49:17.992792+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:49:18.785942+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '180', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 32, 'snap_epoch': 180, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:05:23.143 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:05:23.167 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:23.165+0000 7f00f806e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:05:23.234 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:23.232+0000 7fdef7987180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:05:23.332 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:23.331+0000 7f00f806e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:05:23.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.335+0000 7f668759e700 1 Processor -- start 2023-10-06T05:05:23.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.335+0000 7f668759e700 1 -- start start 2023-10-06T05:05:23.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.336+0000 7f668759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66880b4e70 0x7f66880ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:23.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.337+0000 7f668759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66880cfb00 con 0x7f66880b16d0 2023-10-06T05:05:23.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.337+0000 7f668759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f668815e060 con 0x7f66880b1b40 2023-10-06T05:05:23.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.337+0000 7f668759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66880b5c40 con 0x7f66880b4e70 2023-10-06T05:05:23.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.337+0000 7f6685d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f66880b16d0 0x7f668815d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52524/0 (socket says 172.21.15.55:52524) 2023-10-06T05:05:23.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.337+0000 7f6685d9b700 1 -- 172.21.15.55:0/3420094565 learned_addr learned my addr 172.21.15.55:0/3420094565 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:23.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.337+0000 7f668659c700 1 --2- 172.21.15.55:0/3420094565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66880b4e70 0x7f66880ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:23.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.337+0000 7f668559a700 1 -- 172.21.15.55:0/3420094565 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599225456 0 0) 0x7f66880cfb00 con 0x7f66880b16d0 2023-10-06T05:05:23.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.337+0000 7f668559a700 1 -- 172.21.15.55:0/3420094565 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6670003570 con 0x7f66880b16d0 2023-10-06T05:05:23.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.337+0000 7f668559a700 1 -- 172.21.15.55:0/3420094565 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052951844 0 0) 0x7f668815e060 con 0x7f66880b1b40 2023-10-06T05:05:23.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.337+0000 7f668659c700 1 -- 172.21.15.55:0/3420094565 >> v1:172.21.15.163:6789/0 conn(0x7f66880b1b40 legacy=0x7f668815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:23.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.337+0000 7f668659c700 1 -- 172.21.15.55:0/3420094565 >> v1:172.21.15.55:6789/0 conn(0x7f66880b16d0 legacy=0x7f668815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:23.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.338+0000 7f668659c700 1 -- 172.21.15.55:0/3420094565 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66880b5f20 con 0x7f66880b4e70 2023-10-06T05:05:23.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.338+0000 7f668659c700 1 --2- 172.21.15.55:0/3420094565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66880b4e70 0x7f66880ccff0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6678009910 tx=0x7f66780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=902e86bffc6992f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:23.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.338+0000 7f668559a700 1 -- 172.21.15.55:0/3420094565 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667800b680 con 0x7f66880b4e70 2023-10-06T05:05:23.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.338+0000 7f668559a700 1 -- 172.21.15.55:0/3420094565 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667800b7e0 con 0x7f66880b4e70 2023-10-06T05:05:23.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.338+0000 7f668559a700 1 -- 172.21.15.55:0/3420094565 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667800bba0 con 0x7f66880b4e70 2023-10-06T05:05:23.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.339+0000 7f668759e700 1 -- 172.21.15.55:0/3420094565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66880b4e70 msgr2=0x7f66880ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:23.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.339+0000 7f668759e700 1 --2- 172.21.15.55:0/3420094565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66880b4e70 0x7f66880ccff0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6678009910 tx=0x7f66780091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.339+0000 7f668759e700 1 -- 172.21.15.55:0/3420094565 shutdown_connections 2023-10-06T05:05:23.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.339+0000 7f668759e700 1 --2- 172.21.15.55:0/3420094565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66880b4e70 0x7f66880ccff0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.339+0000 7f668759e700 1 -- 172.21.15.55:0/3420094565 >> 172.21.15.55:0/3420094565 conn(0x7f66880d10c0 msgr2=0x7f66880affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:23.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.339+0000 7f668759e700 1 -- 172.21.15.55:0/3420094565 shutdown_connections 2023-10-06T05:05:23.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.339+0000 7f668759e700 1 -- 172.21.15.55:0/3420094565 wait complete. 2023-10-06T05:05:23.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.340+0000 7f668759e700 1 Processor -- start 2023-10-06T05:05:23.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.341+0000 7f668759e700 1 -- start start 2023-10-06T05:05:23.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f668759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66880b16d0 0x7f66880cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:23.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f668759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66880b1b40 0x7f668814cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:23.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f668759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66880b4e70 0x7f668814d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:23.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f668759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66880c7d50 con 0x7f66880b16d0 2023-10-06T05:05:23.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f668759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66880c7140 con 0x7f66880b1b40 2023-10-06T05:05:23.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f668759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66880c7420 con 0x7f66880b4e70 2023-10-06T05:05:23.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f668659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66880b16d0 0x7f66880cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:23.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f668659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66880b16d0 0x7f66880cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58090/0 (socket says 172.21.15.55:58090) 2023-10-06T05:05:23.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f668659c700 1 -- 172.21.15.55:0/3558473056 learned_addr learned my addr 172.21.15.55:0/3558473056 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:23.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f6685d9b700 1 --2- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66880b1b40 0x7f668814cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:23.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f6686d9d700 1 --2- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66880b4e70 0x7f668814d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:23.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f668659c700 1 -- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66880b4e70 msgr2=0x7f668814d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:23.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f668659c700 1 --2- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66880b4e70 0x7f668814d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f668659c700 1 -- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66880b1b40 msgr2=0x7f668814cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:23.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f668659c700 1 --2- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66880b1b40 0x7f668814cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.342+0000 7f668659c700 1 -- 172.21.15.55:0/3558473056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6678009420 con 0x7f66880b16d0 2023-10-06T05:05:23.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.343+0000 7f668659c700 1 --2- 172.21.15.55:0/3558473056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66880b16d0 0x7f66880cab30 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f6688070dd0 tx=0x7f6678017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:23.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.343+0000 7f66767fc700 1 -- 172.21.15.55:0/3558473056 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667800bb60 con 0x7f66880b16d0 2023-10-06T05:05:23.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.343+0000 7f66767fc700 1 -- 172.21.15.55:0/3558473056 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6678000c40 con 0x7f66880b16d0 2023-10-06T05:05:23.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.343+0000 7f668759e700 1 -- 172.21.15.55:0/3558473056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66880c7700 con 0x7f66880b16d0 2023-10-06T05:05:23.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.343+0000 7f66767fc700 1 -- 172.21.15.55:0/3558473056 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678027680 con 0x7f66880b16d0 2023-10-06T05:05:23.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.343+0000 7f668759e700 1 -- 172.21.15.55:0/3558473056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66880cb070 con 0x7f66880b16d0 2023-10-06T05:05:23.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.344+0000 7f668759e700 1 -- 172.21.15.55:0/3558473056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6688088990 con 0x7f66880b16d0 2023-10-06T05:05:23.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.345+0000 7f66767fc700 1 -- 172.21.15.55:0/3558473056 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 88) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f667801a430 con 0x7f66880b16d0 2023-10-06T05:05:23.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.346+0000 7f66767fc700 1 --2- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6668067050 0x7f6668069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:23.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.346+0000 7f66767fc700 1 -- 172.21.15.55:0/3558473056 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 7141+0+0 (secure 0 0 0) 0x7f6678086b70 con 0x7f66880b16d0 2023-10-06T05:05:23.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.346+0000 7f6685d9b700 1 -- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6668067050 msgr2=0x7f6668069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:23.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.346+0000 7f6685d9b700 1 --2- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6668067050 0x7f6668069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:23.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.350+0000 7f66767fc700 1 -- 172.21.15.55:0/3558473056 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6678053740 con 0x7f66880b16d0 2023-10-06T05:05:23.409 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:23.408+0000 7fdef7987180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:05:23.418 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:23.417+0000 7f00f806e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:05:23.500 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:23.498+0000 7fdef7987180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:05:23.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.547+0000 7f6685d9b700 1 -- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6668067050 msgr2=0x7f6668069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:23.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.547+0000 7f6685d9b700 1 --2- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6668067050 0x7f6668069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:23.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.553+0000 7f668759e700 1 -- 172.21.15.55:0/3558473056 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f66880b1080 con 0x7f66880b16d0 2023-10-06T05:05:23.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.554+0000 7f66767fc700 1 -- 172.21.15.55:0/3558473056 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v180) v1 ==== 92+0+10 (secure 0 0 0) 0x7f667802c020 con 0x7f66880b16d0 2023-10-06T05:05:23.556 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T05:05:23.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.556+0000 7f668759e700 1 -- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6668067050 msgr2=0x7f6668069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:23.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.556+0000 7f668759e700 1 --2- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6668067050 0x7f6668069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.556+0000 7f668759e700 1 -- 172.21.15.55:0/3558473056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66880b16d0 msgr2=0x7f66880cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.556+0000 7f668759e700 1 --2- 172.21.15.55:0/3558473056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66880b16d0 0x7f66880cab30 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f6688070dd0 tx=0x7f6678017000 comp rx=0 tx=0).stop 2023-10-06T05:05:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.556+0000 7f668759e700 1 -- 172.21.15.55:0/3558473056 shutdown_connections 2023-10-06T05:05:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.556+0000 7f668759e700 1 --2- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6668067050 0x7f6668069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.556+0000 7f668759e700 1 --2- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f66880b4e70 0x7f668814d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.556+0000 7f668759e700 1 --2- 172.21.15.55:0/3558473056 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f66880b16d0 0x7f66880cab30 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.556+0000 7f668759e700 1 --2- 172.21.15.55:0/3558473056 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f66880b1b40 0x7f668814cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.556+0000 7f668759e700 1 -- 172.21.15.55:0/3558473056 >> 172.21.15.55:0/3558473056 conn(0x7f66880d10c0 msgr2=0x7f66880ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.556+0000 7f668759e700 1 -- 172.21.15.55:0/3558473056 shutdown_connections 2023-10-06T05:05:23.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.556+0000 7f668759e700 1 -- 172.21.15.55:0/3558473056 wait complete. 2023-10-06T05:05:23.578 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:05:23.645 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:23.644+0000 7f00f806e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:05:23.736 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:23.734+0000 7fdef7987180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:05:23.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.768+0000 7f242a59d700 1 Processor -- start 2023-10-06T05:05:23.770 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.768+0000 7f242a59d700 1 -- start start 2023-10-06T05:05:23.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.769+0000 7f242a59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f24240b16d0 0x7f24240b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:23.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.769+0000 7f242a59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f24240b33b0 0x7f24240b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:23.771 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.769+0000 7f242a59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24240b2080 con 0x7f24240b4e70 2023-10-06T05:05:23.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.769+0000 7f242a59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24240cfb00 con 0x7f24240b16d0 2023-10-06T05:05:23.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.769+0000 7f242a59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24240b5c40 con 0x7f24240b33b0 2023-10-06T05:05:23.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.769+0000 7f2428d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f24240b16d0 0x7f24240b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:23.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.770+0000 7f2428d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f24240b16d0 0x7f24240b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58098/0 (socket says 172.21.15.55:58098) 2023-10-06T05:05:23.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.770+0000 7f2428d9a700 1 -- 172.21.15.55:0/1346887693 learned_addr learned my addr 172.21.15.55:0/1346887693 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:23.772 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.770+0000 7f2429d9c700 1 --2- 172.21.15.55:0/1346887693 >> v2:172.21.15.163:3300/0 conn(0x7f24240b33b0 0x7f24240b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:23.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.770+0000 7f2428d9a700 1 -- 172.21.15.55:0/1346887693 >> v2:172.21.15.163:3300/0 conn(0x7f24240b33b0 msgr2=0x7f24240b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:23.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.770+0000 7f2428d9a700 1 --2- 172.21.15.55:0/1346887693 >> v2:172.21.15.163:3300/0 conn(0x7f24240b33b0 0x7f24240b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.770+0000 7f2428d9a700 1 -- 172.21.15.55:0/1346887693 >> v1:172.21.15.163:6789/0 conn(0x7f24240b4e70 legacy=0x7f24240ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:23.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.770+0000 7f241bfff700 1 -- 172.21.15.55:0/1346887693 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813644918 0 0) 0x7f24240b2080 con 0x7f24240b4e70 2023-10-06T05:05:23.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.770+0000 7f2428d9a700 1 -- 172.21.15.55:0/1346887693 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24240b5f20 con 0x7f24240b16d0 2023-10-06T05:05:23.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.770+0000 7f2428d9a700 1 --2- 172.21.15.55:0/1346887693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24240b16d0 0x7f24240b1b40 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f2424071380 tx=0x7f2414009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273e13101bb6f55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:23.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.771+0000 7f241bfff700 1 -- 172.21.15.55:0/1346887693 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414009e70 con 0x7f24240b16d0 2023-10-06T05:05:23.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.771+0000 7f241bfff700 1 -- 172.21.15.55:0/1346887693 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241400b860 con 0x7f24240b16d0 2023-10-06T05:05:23.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.771+0000 7f241bfff700 1 -- 172.21.15.55:0/1346887693 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400bc20 con 0x7f24240b16d0 2023-10-06T05:05:23.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.771+0000 7f242a59d700 1 -- 172.21.15.55:0/1346887693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24240b16d0 msgr2=0x7f24240b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:23.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.771+0000 7f242a59d700 1 --2- 172.21.15.55:0/1346887693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24240b16d0 0x7f24240b1b40 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f2424071380 tx=0x7f2414009250 comp rx=0 tx=0).stop 2023-10-06T05:05:23.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.771+0000 7f242a59d700 1 -- 172.21.15.55:0/1346887693 shutdown_connections 2023-10-06T05:05:23.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.771+0000 7f242a59d700 1 --2- 172.21.15.55:0/1346887693 >> v2:172.21.15.163:3300/0 conn(0x7f24240b33b0 0x7f24240b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.771+0000 7f242a59d700 1 --2- 172.21.15.55:0/1346887693 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24240b16d0 0x7f24240b1b40 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.771+0000 7f242a59d700 1 -- 172.21.15.55:0/1346887693 >> 172.21.15.55:0/1346887693 conn(0x7f24240d10c0 msgr2=0x7f24240affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:23.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.772+0000 7f242a59d700 1 -- 172.21.15.55:0/1346887693 shutdown_connections 2023-10-06T05:05:23.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.772+0000 7f242a59d700 1 -- 172.21.15.55:0/1346887693 wait complete. 2023-10-06T05:05:23.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.773+0000 7f242a59d700 1 Processor -- start 2023-10-06T05:05:23.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.773+0000 7f242a59d700 1 -- start start 2023-10-06T05:05:23.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.774+0000 7f242a59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240b16d0 0x7f242416d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:23.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.774+0000 7f242a59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24240b33b0 0x7f24241699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:23.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.774+0000 7f242a59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24240b4e70 0x7f24240cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:23.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.774+0000 7f242a59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24240c7020 con 0x7f24240b4e70 2023-10-06T05:05:23.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.774+0000 7f242a59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24240c7270 con 0x7f24240b16d0 2023-10-06T05:05:23.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.774+0000 7f242a59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24240c7550 con 0x7f24240b33b0 2023-10-06T05:05:23.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.774+0000 7f2429d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24240b4e70 0x7f24240cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:23.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.774+0000 7f2429d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24240b4e70 0x7f24240cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58112/0 (socket says 172.21.15.55:58112) 2023-10-06T05:05:23.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.774+0000 7f2429d9c700 1 -- 172.21.15.55:0/4071006597 learned_addr learned my addr 172.21.15.55:0/4071006597 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:23.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.774+0000 7f2428d9a700 1 --2- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24240b33b0 0x7f24241699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:23.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.774+0000 7f242959b700 1 --2- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240b16d0 0x7f242416d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:23.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.775+0000 7f2429d9c700 1 -- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24240b33b0 msgr2=0x7f24241699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:23.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.775+0000 7f2429d9c700 1 --2- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24240b33b0 0x7f24241699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.775+0000 7f2429d9c700 1 -- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240b16d0 msgr2=0x7f242416d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:23.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.775+0000 7f2429d9c700 1 --2- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240b16d0 0x7f242416d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.775+0000 7f2429d9c700 1 -- 172.21.15.55:0/4071006597 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24140094f0 con 0x7f24240b4e70 2023-10-06T05:05:23.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.775+0000 7f2429d9c700 1 --2- 172.21.15.55:0/4071006597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24240b4e70 0x7f24240cb430 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f241c009e90 tx=0x7f241c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:23.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.775+0000 7f24197fa700 1 -- 172.21.15.55:0/4071006597 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00cb70 con 0x7f24240b4e70 2023-10-06T05:05:23.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.775+0000 7f24197fa700 1 -- 172.21.15.55:0/4071006597 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c00f040 con 0x7f24240b4e70 2023-10-06T05:05:23.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.775+0000 7f242a59d700 1 -- 172.21.15.55:0/4071006597 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24240c7830 con 0x7f24240b4e70 2023-10-06T05:05:23.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.775+0000 7f24197fa700 1 -- 172.21.15.55:0/4071006597 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c014480 con 0x7f24240b4e70 2023-10-06T05:05:23.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.776+0000 7f242a59d700 1 -- 172.21.15.55:0/4071006597 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24240c7e70 con 0x7f24240b4e70 2023-10-06T05:05:23.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.777+0000 7f242a59d700 1 -- 172.21.15.55:0/4071006597 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2424088990 con 0x7f24240b4e70 2023-10-06T05:05:23.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.777+0000 7f24197fa700 1 -- 172.21.15.55:0/4071006597 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 88) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f241c014830 con 0x7f24240b4e70 2023-10-06T05:05:23.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.778+0000 7f24197fa700 1 --2- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2408067050 0x7f2408069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:23.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.778+0000 7f24197fa700 1 -- 172.21.15.55:0/4071006597 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 7141+0+0 (secure 0 0 0) 0x7f241c085830 con 0x7f24240b4e70 2023-10-06T05:05:23.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.778+0000 7f242959b700 1 -- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2408067050 msgr2=0x7f2408069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:23.781 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.778+0000 7f242959b700 1 --2- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2408067050 0x7f2408069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:23.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.786+0000 7f24197fa700 1 -- 172.21.15.55:0/4071006597 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f241c052240 con 0x7f24240b4e70 2023-10-06T05:05:23.825 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:23.824+0000 7f00f806e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:05:23.918 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:23.917+0000 7f00f806e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:05:23.927 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:23.926+0000 7fdef7987180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:05:23.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.979+0000 7f242959b700 1 -- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2408067050 msgr2=0x7f2408069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:23.981 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.979+0000 7f242959b700 1 --2- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2408067050 0x7f2408069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:23.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.991+0000 7f242a59d700 1 -- 172.21.15.55:0/4071006597 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f242404bb80 con 0x7f24240b4e70 2023-10-06T05:05:23.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.991+0000 7f24197fa700 1 -- 172.21.15.55:0/4071006597 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v180) v1 ==== 93+0+10 (secure 0 0 0) 0x7f241c055860 con 0x7f24240b4e70 2023-10-06T05:05:23.993 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T05:05:23.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.993+0000 7f242a59d700 1 -- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2408067050 msgr2=0x7f2408069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:23.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.993+0000 7f242a59d700 1 --2- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2408067050 0x7f2408069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.993+0000 7f242a59d700 1 -- 172.21.15.55:0/4071006597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24240b4e70 msgr2=0x7f24240cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:23.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.993+0000 7f242a59d700 1 --2- 172.21.15.55:0/4071006597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24240b4e70 0x7f24240cb430 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f241c009e90 tx=0x7f241c00b040 comp rx=0 tx=0).stop 2023-10-06T05:05:23.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.993+0000 7f242a59d700 1 -- 172.21.15.55:0/4071006597 shutdown_connections 2023-10-06T05:05:23.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.993+0000 7f242a59d700 1 --2- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2408067050 0x7f2408069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.993+0000 7f242a59d700 1 --2- 172.21.15.55:0/4071006597 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f24240b4e70 0x7f24240cb430 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.993+0000 7f242a59d700 1 --2- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f24240b16d0 0x7f242416d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.993+0000 7f242a59d700 1 --2- 172.21.15.55:0/4071006597 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f24240b33b0 0x7f24241699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:23.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.993+0000 7f242a59d700 1 -- 172.21.15.55:0/4071006597 >> 172.21.15.55:0/4071006597 conn(0x7f24240d10c0 msgr2=0x7f2424069710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:23.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.994+0000 7f242a59d700 1 -- 172.21.15.55:0/4071006597 shutdown_connections 2023-10-06T05:05:23.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:23.994+0000 7f242a59d700 1 -- 172.21.15.55:0/4071006597 wait complete. 2023-10-06T05:05:24.014 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:05:24.027 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:24.025+0000 7fdef7987180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:05:24.088 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:24.087+0000 7f00f806e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:05:24.188 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:24.187+0000 7f00f806e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:05:24.197 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:24.195+0000 7fdef7987180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:05:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.206+0000 7f219359e700 1 Processor -- start 2023-10-06T05:05:24.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.206+0000 7f219359e700 1 -- start start 2023-10-06T05:05:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.208+0000 7f219359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f21940b4e70 0x7f21940ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.208+0000 7f219359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21940cfb00 con 0x7f21940b1b40 2023-10-06T05:05:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.208+0000 7f219359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f219415e060 con 0x7f21940b16d0 2023-10-06T05:05:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.208+0000 7f219359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21940b5c40 con 0x7f21940b4e70 2023-10-06T05:05:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.208+0000 7f219259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f21940b4e70 0x7f21940ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.208+0000 7f219259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f21940b4e70 0x7f21940ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58118/0 (socket says 172.21.15.55:58118) 2023-10-06T05:05:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.208+0000 7f219259c700 1 -- 172.21.15.55:0/2838942223 learned_addr learned my addr 172.21.15.55:0/2838942223 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:24.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.208+0000 7f2192d9d700 1 --1- 172.21.15.55:0/2838942223 >> v1:172.21.15.55:6789/0 conn(0x7f21940b1b40 0x7f219415d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52534/0 (socket says 172.21.15.55:52534) 2023-10-06T05:05:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.208+0000 7f219259c700 1 -- 172.21.15.55:0/2838942223 >> v1:172.21.15.163:6789/0 conn(0x7f21940b16d0 legacy=0x7f219415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.208+0000 7f219259c700 1 -- 172.21.15.55:0/2838942223 >> v1:172.21.15.55:6789/0 conn(0x7f21940b1b40 legacy=0x7f219415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.208+0000 7f219259c700 1 -- 172.21.15.55:0/2838942223 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21940b5f20 con 0x7f21940b4e70 2023-10-06T05:05:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.209+0000 7f219259c700 1 --2- 172.21.15.55:0/2838942223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21940b4e70 0x7f21940ccff0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f21840099b0 tx=0x7f2184009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acd581c7373c389 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.209+0000 7f219159a700 1 -- 172.21.15.55:0/2838942223 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218400b860 con 0x7f21940b4e70 2023-10-06T05:05:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.209+0000 7f219159a700 1 -- 172.21.15.55:0/2838942223 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218400b9c0 con 0x7f21940b4e70 2023-10-06T05:05:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.209+0000 7f219159a700 1 -- 172.21.15.55:0/2838942223 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218400bd60 con 0x7f21940b4e70 2023-10-06T05:05:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.210+0000 7f219359e700 1 -- 172.21.15.55:0/2838942223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21940b4e70 msgr2=0x7f21940ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.210+0000 7f219359e700 1 --2- 172.21.15.55:0/2838942223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21940b4e70 0x7f21940ccff0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f21840099b0 tx=0x7f2184009250 comp rx=0 tx=0).stop 2023-10-06T05:05:24.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.210+0000 7f219359e700 1 -- 172.21.15.55:0/2838942223 shutdown_connections 2023-10-06T05:05:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.210+0000 7f219359e700 1 --2- 172.21.15.55:0/2838942223 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21940b4e70 0x7f21940ccff0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.210+0000 7f219359e700 1 -- 172.21.15.55:0/2838942223 >> 172.21.15.55:0/2838942223 conn(0x7f21940d10c0 msgr2=0x7f21940affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.210+0000 7f219359e700 1 -- 172.21.15.55:0/2838942223 shutdown_connections 2023-10-06T05:05:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.210+0000 7f219359e700 1 -- 172.21.15.55:0/2838942223 wait complete. 2023-10-06T05:05:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.211+0000 7f219359e700 1 Processor -- start 2023-10-06T05:05:24.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.211+0000 7f219359e700 1 -- start start 2023-10-06T05:05:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.212+0000 7f219359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21940b16d0 0x7f21940c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:24.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.212+0000 7f219359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21940b1b40 0x7f219416cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.212+0000 7f219359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21940b4e70 0x7f2194169dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.212+0000 7f219359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21940c8ed0 con 0x7f21940b1b40 2023-10-06T05:05:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.212+0000 7f219359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21940c5530 con 0x7f21940b4e70 2023-10-06T05:05:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.212+0000 7f219359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21940c5810 con 0x7f21940b16d0 2023-10-06T05:05:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.213+0000 7f2191d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21940b1b40 0x7f219416cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.213+0000 7f2191d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21940b1b40 0x7f219416cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58128/0 (socket says 172.21.15.55:58128) 2023-10-06T05:05:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.213+0000 7f2191d9b700 1 -- 172.21.15.55:0/1174556836 learned_addr learned my addr 172.21.15.55:0/1174556836 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.213+0000 7f219259c700 1 --2- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21940b16d0 0x7f21940c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.213+0000 7f2192d9d700 1 --2- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21940b4e70 0x7f2194169dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:24.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.213+0000 7f2191d9b700 1 -- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21940b16d0 msgr2=0x7f21940c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.213+0000 7f2191d9b700 1 --2- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21940b16d0 0x7f21940c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.213+0000 7f2191d9b700 1 -- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21940b4e70 msgr2=0x7f2194169dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.213+0000 7f2191d9b700 1 --2- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21940b4e70 0x7f2194169dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.213+0000 7f2191d9b700 1 -- 172.21.15.55:0/1174556836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21840094f0 con 0x7f21940b1b40 2023-10-06T05:05:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.213+0000 7f2191d9b700 1 --2- 172.21.15.55:0/1174556836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21940b1b40 0x7f219416cac0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f218800c590 tx=0x7f218800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.214+0000 7f21827fc700 1 -- 172.21.15.55:0/1174556836 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188011590 con 0x7f21940b1b40 2023-10-06T05:05:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.214+0000 7f21827fc700 1 -- 172.21.15.55:0/1174556836 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2188011710 con 0x7f21940b1b40 2023-10-06T05:05:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.214+0000 7f219359e700 1 -- 172.21.15.55:0/1174556836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21940c5af0 con 0x7f21940b1b40 2023-10-06T05:05:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.214+0000 7f21827fc700 1 -- 172.21.15.55:0/1174556836 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188011ad0 con 0x7f21940b1b40 2023-10-06T05:05:24.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.214+0000 7f219359e700 1 -- 172.21.15.55:0/1174556836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219416d000 con 0x7f21940b1b40 2023-10-06T05:05:24.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.215+0000 7f219359e700 1 -- 172.21.15.55:0/1174556836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21940c5f90 con 0x7f21940b1b40 2023-10-06T05:05:24.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.216+0000 7f21827fc700 1 -- 172.21.15.55:0/1174556836 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 88) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f218801b3b0 con 0x7f21940b1b40 2023-10-06T05:05:24.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.216+0000 7f21827fc700 1 --2- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2174067050 0x7f2174069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:24.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.217+0000 7f21827fc700 1 -- 172.21.15.55:0/1174556836 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 7141+0+0 (secure 0 0 0) 0x7f2188084160 con 0x7f21940b1b40 2023-10-06T05:05:24.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.217+0000 7f219259c700 1 -- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2174067050 msgr2=0x7f2174069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:24.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.217+0000 7f219259c700 1 --2- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2174067050 0x7f2174069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:24.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.221+0000 7f21827fc700 1 -- 172.21.15.55:0/1174556836 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2188050c80 con 0x7f21940b1b40 2023-10-06T05:05:24.306 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:24.304+0000 7fdef7987180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:05:24.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.418+0000 7f219259c700 1 -- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2174067050 msgr2=0x7f2174069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:24.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.418+0000 7f219259c700 1 --2- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2174067050 0x7f2174069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:24.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.422+0000 7f219359e700 1 -- 172.21.15.55:0/1174556836 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f219404bb80 con 0x7f21940b1b40 2023-10-06T05:05:24.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.423+0000 7f21827fc700 1 -- 172.21.15.55:0/1174556836 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v180) v1 ==== 104+0+11 (secure 0 0 0) 0x7f21880542a0 con 0x7f21940b1b40 2023-10-06T05:05:24.425 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T05:05:24.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.425+0000 7f219359e700 1 -- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2174067050 msgr2=0x7f2174069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:24.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.425+0000 7f219359e700 1 --2- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2174067050 0x7f2174069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.425+0000 7f219359e700 1 -- 172.21.15.55:0/1174556836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21940b1b40 msgr2=0x7f219416cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:24.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.425+0000 7f219359e700 1 --2- 172.21.15.55:0/1174556836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21940b1b40 0x7f219416cac0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f218800c590 tx=0x7f218800c8a0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.425+0000 7f219359e700 1 -- 172.21.15.55:0/1174556836 shutdown_connections 2023-10-06T05:05:24.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.425+0000 7f219359e700 1 --2- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2174067050 0x7f2174069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.425+0000 7f219359e700 1 --2- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f21940b4e70 0x7f2194169dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.425+0000 7f219359e700 1 --2- 172.21.15.55:0/1174556836 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f21940b16d0 0x7f21940c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.425+0000 7f219359e700 1 --2- 172.21.15.55:0/1174556836 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f21940b1b40 0x7f219416cac0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.425+0000 7f219359e700 1 -- 172.21.15.55:0/1174556836 >> 172.21.15.55:0/1174556836 conn(0x7f21940d10c0 msgr2=0x7f2194159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:24.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.425+0000 7f219359e700 1 -- 172.21.15.55:0/1174556836 shutdown_connections 2023-10-06T05:05:24.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.425+0000 7f219359e700 1 -- 172.21.15.55:0/1174556836 wait complete. 2023-10-06T05:05:24.446 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:05:24.673 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:24.672+0000 7f00f806e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:05:24.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.680+0000 7f172a39d700 1 Processor -- start 2023-10-06T05:05:24.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.680+0000 7f172a39d700 1 -- start start 2023-10-06T05:05:24.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.681+0000 7f172a39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1724159a60 0x7f17240c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:24.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.681+0000 7f172a39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f17240cb230 0x7f17240cb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:24.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.681+0000 7f172a39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17240cbb90 con 0x7f17240ccc00 2023-10-06T05:05:24.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.681+0000 7f172a39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172406dfc0 con 0x7f17240cb230 2023-10-06T05:05:24.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.681+0000 7f172a39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17240c9cb0 con 0x7f1724159a60 2023-10-06T05:05:24.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.681+0000 7f1728b9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f17240ccc00 0x7f1724069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52538/0 (socket says 172.21.15.55:52538) 2023-10-06T05:05:24.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.682+0000 7f1728b9a700 1 -- 172.21.15.55:0/1373759927 learned_addr learned my addr 172.21.15.55:0/1373759927 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:24.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.682+0000 7f1729b9c700 1 --2- 172.21.15.55:0/1373759927 >> v2:172.21.15.55:3300/0 conn(0x7f17240cb230 0x7f17240cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:24.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.682+0000 7f172939b700 1 --2- 172.21.15.55:0/1373759927 >> v2:172.21.15.163:3300/0 conn(0x7f1724159a60 0x7f17240c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:24.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.682+0000 7f171bfff700 1 -- 172.21.15.55:0/1373759927 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955236624 0 0) 0x7f17240cbb90 con 0x7f17240ccc00 2023-10-06T05:05:24.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.682+0000 7f171bfff700 1 -- 172.21.15.55:0/1373759927 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1710003570 con 0x7f17240ccc00 2023-10-06T05:05:24.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.682+0000 7f172939b700 1 -- 172.21.15.55:0/1373759927 >> v2:172.21.15.55:3300/0 conn(0x7f17240cb230 msgr2=0x7f17240cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:24.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.682+0000 7f172939b700 1 --2- 172.21.15.55:0/1373759927 >> v2:172.21.15.55:3300/0 conn(0x7f17240cb230 0x7f17240cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.682+0000 7f172939b700 1 -- 172.21.15.55:0/1373759927 >> v1:172.21.15.55:6789/0 conn(0x7f17240ccc00 legacy=0x7f1724069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:24.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.682+0000 7f172939b700 1 -- 172.21.15.55:0/1373759927 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17240c9f30 con 0x7f1724159a60 2023-10-06T05:05:24.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.682+0000 7f1729b9c700 1 --2- 172.21.15.55:0/1373759927 >> v2:172.21.15.55:3300/0 conn(0x7f17240cb230 0x7f17240cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:24.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.683+0000 7f172939b700 1 --2- 172.21.15.55:0/1373759927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1724159a60 0x7f17240c9770 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f172406e510 tx=0x7f17200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b36d38a8c19b529 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:24.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.683+0000 7f171bfff700 1 -- 172.21.15.55:0/1373759927 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720011630 con 0x7f1724159a60 2023-10-06T05:05:24.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.683+0000 7f171bfff700 1 -- 172.21.15.55:0/1373759927 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17200117b0 con 0x7f1724159a60 2023-10-06T05:05:24.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.683+0000 7f171bfff700 1 -- 172.21.15.55:0/1373759927 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720011b70 con 0x7f1724159a60 2023-10-06T05:05:24.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.683+0000 7f172a39d700 1 -- 172.21.15.55:0/1373759927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1724159a60 msgr2=0x7f17240c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:24.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.683+0000 7f172a39d700 1 --2- 172.21.15.55:0/1373759927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1724159a60 0x7f17240c9770 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f172406e510 tx=0x7f17200091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.684+0000 7f172a39d700 1 -- 172.21.15.55:0/1373759927 shutdown_connections 2023-10-06T05:05:24.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.684+0000 7f172a39d700 1 --2- 172.21.15.55:0/1373759927 >> v2:172.21.15.55:3300/0 conn(0x7f17240cb230 0x7f17240cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.684+0000 7f172a39d700 1 --2- 172.21.15.55:0/1373759927 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1724159a60 0x7f17240c9770 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.684+0000 7f172a39d700 1 -- 172.21.15.55:0/1373759927 >> 172.21.15.55:0/1373759927 conn(0x7f172406baa0 msgr2=0x7f172406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.684+0000 7f172a39d700 1 -- 172.21.15.55:0/1373759927 shutdown_connections 2023-10-06T05:05:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.684+0000 7f172a39d700 1 -- 172.21.15.55:0/1373759927 wait complete. 2023-10-06T05:05:24.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.685+0000 7f172a39d700 1 Processor -- start 2023-10-06T05:05:24.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.685+0000 7f172a39d700 1 -- start start 2023-10-06T05:05:24.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.686+0000 7f172a39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17240cb230 0x7f17240c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:24.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.686+0000 7f172a39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17240ccc00 0x7f1724146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:24.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.686+0000 7f172a39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1724159a60 0x7f17240b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:24.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.686+0000 7f172a39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17240a9bd0 con 0x7f17240cb230 2023-10-06T05:05:24.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.686+0000 7f172a39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17240b4340 con 0x7f1724159a60 2023-10-06T05:05:24.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.686+0000 7f172a39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17240b1f00 con 0x7f17240ccc00 2023-10-06T05:05:24.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.686+0000 7f172939b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17240cb230 0x7f17240c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:24.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.686+0000 7f172939b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17240cb230 0x7f17240c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58142/0 (socket says 172.21.15.55:58142) 2023-10-06T05:05:24.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.687+0000 7f172939b700 1 -- 172.21.15.55:0/3609889565 learned_addr learned my addr 172.21.15.55:0/3609889565 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:24.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.687+0000 7f1729b9c700 1 --2- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1724159a60 0x7f17240b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:24.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.687+0000 7f1728b9a700 1 --2- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17240ccc00 0x7f1724146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:24.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.687+0000 7f172939b700 1 -- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17240ccc00 msgr2=0x7f1724146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:24.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.687+0000 7f172939b700 1 --2- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17240ccc00 0x7f1724146ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.687+0000 7f172939b700 1 -- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1724159a60 msgr2=0x7f17240b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:24.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.687+0000 7f172939b700 1 --2- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1724159a60 0x7f17240b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.687+0000 7f172939b700 1 -- 172.21.15.55:0/3609889565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c009a20 con 0x7f17240cb230 2023-10-06T05:05:24.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.687+0000 7f172939b700 1 --2- 172.21.15.55:0/3609889565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17240cb230 0x7f17240c7530 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f1720013000 tx=0x7f1720017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:24.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.688+0000 7f17197fa700 1 -- 172.21.15.55:0/3609889565 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172001f070 con 0x7f17240cb230 2023-10-06T05:05:24.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.688+0000 7f17197fa700 1 -- 172.21.15.55:0/3609889565 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1720000bc0 con 0x7f17240cb230 2023-10-06T05:05:24.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.688+0000 7f17197fa700 1 -- 172.21.15.55:0/3609889565 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720011b70 con 0x7f17240cb230 2023-10-06T05:05:24.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.688+0000 7f172a39d700 1 -- 172.21.15.55:0/3609889565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1720009420 con 0x7f17240cb230 2023-10-06T05:05:24.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.688+0000 7f172a39d700 1 -- 172.21.15.55:0/3609889565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17240b18b0 con 0x7f17240cb230 2023-10-06T05:05:24.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.689+0000 7f172a39d700 1 -- 172.21.15.55:0/3609889565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172406eba0 con 0x7f17240cb230 2023-10-06T05:05:24.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.690+0000 7f17197fa700 1 -- 172.21.15.55:0/3609889565 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 88) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f1720023070 con 0x7f17240cb230 2023-10-06T05:05:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.690+0000 7f17197fa700 1 --2- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1708067050 0x7f1708069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.691+0000 7f17197fa700 1 -- 172.21.15.55:0/3609889565 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 7141+0+0 (secure 0 0 0) 0x7f1720085df0 con 0x7f17240cb230 2023-10-06T05:05:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.691+0000 7f1728b9a700 1 -- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1708067050 msgr2=0x7f1708069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:24.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.691+0000 7f1728b9a700 1 --2- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1708067050 0x7f1708069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:24.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.695+0000 7f17197fa700 1 -- 172.21.15.55:0/3609889565 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1720052a70 con 0x7f17240cb230 2023-10-06T05:05:24.770 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:24.768+0000 7f00f806e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:05:24.830 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:24.829+0000 7fdef7987180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:05:24.864 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:24.863+0000 7f00f806e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:05:24.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.892+0000 7f1728b9a700 1 -- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1708067050 msgr2=0x7f1708069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:24.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.892+0000 7f1728b9a700 1 --2- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1708067050 0x7f1708069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:24.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.908+0000 7f172a39d700 1 -- 172.21.15.55:0/3609889565 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f17240ca470 con 0x7f17240cb230 2023-10-06T05:05:24.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.908+0000 7f17197fa700 1 -- 172.21.15.55:0/3609889565 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v181) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1720056090 con 0x7f17240cb230 2023-10-06T05:05:24.910 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T05:05:24.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.910+0000 7f172a39d700 1 -- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1708067050 msgr2=0x7f1708069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:24.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.910+0000 7f172a39d700 1 --2- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1708067050 0x7f1708069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.910+0000 7f172a39d700 1 -- 172.21.15.55:0/3609889565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17240cb230 msgr2=0x7f17240c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:24.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.910+0000 7f172a39d700 1 --2- 172.21.15.55:0/3609889565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17240cb230 0x7f17240c7530 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f1720013000 tx=0x7f1720017000 comp rx=0 tx=0).stop 2023-10-06T05:05:24.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.911+0000 7f172a39d700 1 -- 172.21.15.55:0/3609889565 shutdown_connections 2023-10-06T05:05:24.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.911+0000 7f172a39d700 1 --2- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1708067050 0x7f1708069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.911+0000 7f172a39d700 1 --2- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1724159a60 0x7f17240b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.911+0000 7f172a39d700 1 --2- 172.21.15.55:0/3609889565 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17240cb230 0x7f17240c7530 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.911+0000 7f172a39d700 1 --2- 172.21.15.55:0/3609889565 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17240ccc00 0x7f1724146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:24.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.911+0000 7f172a39d700 1 -- 172.21.15.55:0/3609889565 >> 172.21.15.55:0/3609889565 conn(0x7f172406baa0 msgr2=0x7f172406ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:24.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.911+0000 7f172a39d700 1 -- 172.21.15.55:0/3609889565 shutdown_connections 2023-10-06T05:05:24.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:24.911+0000 7f172a39d700 1 -- 172.21.15.55:0/3609889565 wait complete. 2023-10-06T05:05:24.931 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:05:24.932 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:24.931+0000 7fdef7987180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:05:25.035 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:25.033+0000 7fdef7987180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:05:25.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.124+0000 7fc3b1d9d700 1 Processor -- start 2023-10-06T05:05:25.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.125+0000 7fc3b1d9d700 1 -- start start 2023-10-06T05:05:25.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.126+0000 7fc3b1d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc3ac159420 0x7fc3ac0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:25.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.126+0000 7fc3b1d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc3ac15d230 0x7fc3ac15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:25.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.126+0000 7fc3b1d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3ac0b2980 con 0x7fc3ac1597f0 2023-10-06T05:05:25.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.126+0000 7fc3b1d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ac15db70 con 0x7fc3ac159420 2023-10-06T05:05:25.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.126+0000 7fc3b1d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ac0b11d0 con 0x7fc3ac15d230 2023-10-06T05:05:25.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.126+0000 7fc3b0d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc3ac159420 0x7fc3ac0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:25.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.126+0000 7fc3b0d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc3ac159420 0x7fc3ac0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58154/0 (socket says 172.21.15.55:58154) 2023-10-06T05:05:25.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.126+0000 7fc3b0d9b700 1 -- 172.21.15.55:0/242665370 learned_addr learned my addr 172.21.15.55:0/242665370 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:25.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.126+0000 7fc3b159c700 1 --2- 172.21.15.55:0/242665370 >> v2:172.21.15.163:3300/0 conn(0x7fc3ac15d230 0x7fc3ac15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:25.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.127+0000 7fc3b0d9b700 1 -- 172.21.15.55:0/242665370 >> v2:172.21.15.163:3300/0 conn(0x7fc3ac15d230 msgr2=0x7fc3ac15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:25.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.127+0000 7fc3b0d9b700 1 --2- 172.21.15.55:0/242665370 >> v2:172.21.15.163:3300/0 conn(0x7fc3ac15d230 0x7fc3ac15d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.127+0000 7fc3b0d9b700 1 -- 172.21.15.55:0/242665370 >> v1:172.21.15.163:6789/0 conn(0x7fc3ac1597f0 legacy=0x7fc3ac0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:25.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.127+0000 7fc3b0d9b700 1 -- 172.21.15.55:0/242665370 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ac0b14b0 con 0x7fc3ac159420 2023-10-06T05:05:25.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.127+0000 7fc3b0d9b700 1 --2- 172.21.15.55:0/242665370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ac159420 0x7fc3ac0b5f40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac15d170 tx=0x7fc3a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94d03c1593193a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:25.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.127+0000 7fc3ab7fe700 1 -- 172.21.15.55:0/242665370 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0009e70 con 0x7fc3ac159420 2023-10-06T05:05:25.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.127+0000 7fc3ab7fe700 1 -- 172.21.15.55:0/242665370 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a000b860 con 0x7fc3ac159420 2023-10-06T05:05:25.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.127+0000 7fc3ab7fe700 1 -- 172.21.15.55:0/242665370 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a000bc00 con 0x7fc3ac159420 2023-10-06T05:05:25.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.128+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/242665370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ac159420 msgr2=0x7fc3ac0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:25.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.128+0000 7fc3b1d9d700 1 --2- 172.21.15.55:0/242665370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ac159420 0x7fc3ac0b5f40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac15d170 tx=0x7fc3a0009250 comp rx=0 tx=0).stop 2023-10-06T05:05:25.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.128+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/242665370 shutdown_connections 2023-10-06T05:05:25.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.128+0000 7fc3b1d9d700 1 --2- 172.21.15.55:0/242665370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ac159420 0x7fc3ac0b5f40 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.128+0000 7fc3b1d9d700 1 --2- 172.21.15.55:0/242665370 >> v2:172.21.15.163:3300/0 conn(0x7fc3ac15d230 0x7fc3ac15d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.128+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/242665370 >> 172.21.15.55:0/242665370 conn(0x7fc3ac0d0660 msgr2=0x7fc3ac0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:25.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.128+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/242665370 shutdown_connections 2023-10-06T05:05:25.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.128+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/242665370 wait complete. 2023-10-06T05:05:25.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.130+0000 7fc3b1d9d700 1 Processor -- start 2023-10-06T05:05:25.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.130+0000 7fc3b1d9d700 1 -- start start 2023-10-06T05:05:25.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.131+0000 7fc3b1d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3ac159420 0x7fc3ac155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:25.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.131+0000 7fc3b1d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3ac1597f0 0x7fc3ac169bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:25.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.131+0000 7fc3b1d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ac15d230 0x7fc3ac16a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:25.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.131+0000 7fc3b1d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac1566b0 con 0x7fc3ac15d230 2023-10-06T05:05:25.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.131+0000 7fc3b1d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac16a7b0 con 0x7fc3ac159420 2023-10-06T05:05:25.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.131+0000 7fc3b1d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ac16aa90 con 0x7fc3ac1597f0 2023-10-06T05:05:25.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.131+0000 7fc3b0d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3ac159420 0x7fc3ac155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:25.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.131+0000 7fc3b159c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ac15d230 0x7fc3ac16a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:25.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.131+0000 7fc3b0d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3ac159420 0x7fc3ac155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:42766/0 (socket says 172.21.15.55:42766) 2023-10-06T05:05:25.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.131+0000 7fc3b0d9b700 1 -- 172.21.15.55:0/3282157595 learned_addr learned my addr 172.21.15.55:0/3282157595 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:25.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.131+0000 7fc3abfff700 1 --2- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3ac1597f0 0x7fc3ac169bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:25.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.132+0000 7fc3b0d9b700 1 -- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3ac1597f0 msgr2=0x7fc3ac169bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:25.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.132+0000 7fc3b0d9b700 1 --2- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3ac1597f0 0x7fc3ac169bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.132+0000 7fc3b0d9b700 1 -- 172.21.15.55:0/3282157595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ac15d230 msgr2=0x7fc3ac16a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:25.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.132+0000 7fc3b0d9b700 1 --2- 172.21.15.55:0/3282157595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ac15d230 0x7fc3ac16a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.132+0000 7fc3b0d9b700 1 -- 172.21.15.55:0/3282157595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a00094f0 con 0x7fc3ac159420 2023-10-06T05:05:25.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.132+0000 7fc3b0d9b700 1 --2- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3ac159420 0x7fc3ac155c20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac089340 tx=0x7fc3a0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:25.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.132+0000 7fc3a8ff9700 1 -- 172.21.15.55:0/3282157595 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a000bbe0 con 0x7fc3ac159420 2023-10-06T05:05:25.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.132+0000 7fc3a8ff9700 1 -- 172.21.15.55:0/3282157595 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a0011420 con 0x7fc3ac159420 2023-10-06T05:05:25.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.132+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/3282157595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ac159ca0 con 0x7fc3ac159420 2023-10-06T05:05:25.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.132+0000 7fc3a8ff9700 1 -- 172.21.15.55:0/3282157595 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a00117c0 con 0x7fc3ac159420 2023-10-06T05:05:25.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.132+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/3282157595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ac15a260 con 0x7fc3ac159420 2023-10-06T05:05:25.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.134+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/3282157595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3ac15a480 con 0x7fc3ac159420 2023-10-06T05:05:25.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.134+0000 7fc3a8ff9700 1 -- 172.21.15.55:0/3282157595 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 88) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fc3a0011bf0 con 0x7fc3ac159420 2023-10-06T05:05:25.137 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.135+0000 7fc3a8ff9700 1 --2- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc390067050 0x7fc390069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:25.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.135+0000 7fc3a8ff9700 1 -- 172.21.15.55:0/3282157595 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fc3a0086240 con 0x7fc3ac159420 2023-10-06T05:05:25.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.135+0000 7fc3abfff700 1 -- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc390067050 msgr2=0x7fc390069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:25.138 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.135+0000 7fc3abfff700 1 --2- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc390067050 0x7fc390069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:25.141 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.140+0000 7fc3a8ff9700 1 -- 172.21.15.55:0/3282157595 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3a0052df0 con 0x7fc3ac159420 2023-10-06T05:05:25.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.336+0000 7fc3abfff700 1 -- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc390067050 msgr2=0x7fc390069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:25.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.336+0000 7fc3abfff700 1 --2- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc390067050 0x7fc390069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:25.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.390+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/3282157595 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3ac16a8f0 con 0x7fc3ac159420 2023-10-06T05:05:25.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.391+0000 7fc3a8ff9700 1 -- 172.21.15.55:0/3282157595 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4604 (secure 0 0 0) 0x7fc3ac16a8f0 con 0x7fc3ac159420 2023-10-06T05:05:25.393 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:25.394 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7682,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1565234675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1565234675},{"type":"v1","addr":"172.21.15.55:6834","nonce":1565234675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3042024952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3042024952},{"type":"v1","addr":"172.21.15.163:6835","nonce":3042024952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8737,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/304924610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":304924610},{"type":"v1","addr":"172.21.15.55:6835","nonce":304924610}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:49:19.807866+0000","modified":"2023-10-06T04:49:23.339948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8740":{"gid":8740,"name":"b","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6834/3095914721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3095914721},{"type":"v1","addr":"172.21.15.163:6834","nonce":3095914721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:05:25.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.394+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc390067050 msgr2=0x7fc390069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:25.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.394+0000 7fc3b1d9d700 1 --2- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc390067050 0x7fc390069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.394+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3ac159420 msgr2=0x7fc3ac155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:25.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.394+0000 7fc3b1d9d700 1 --2- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3ac159420 0x7fc3ac155c20 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac089340 tx=0x7fc3a0017000 comp rx=0 tx=0).stop 2023-10-06T05:05:25.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.394+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/3282157595 shutdown_connections 2023-10-06T05:05:25.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.394+0000 7fc3b1d9d700 1 --2- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc390067050 0x7fc390069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.394+0000 7fc3b1d9d700 1 --2- 172.21.15.55:0/3282157595 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc3ac15d230 0x7fc3ac16a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.394+0000 7fc3b1d9d700 1 --2- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc3ac159420 0x7fc3ac155c20 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.394+0000 7fc3b1d9d700 1 --2- 172.21.15.55:0/3282157595 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc3ac1597f0 0x7fc3ac169bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.394+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/3282157595 >> 172.21.15.55:0/3282157595 conn(0x7fc3ac0d0660 msgr2=0x7fc3ac069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:25.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.394+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/3282157595 shutdown_connections 2023-10-06T05:05:25.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.394+0000 7fc3b1d9d700 1 -- 172.21.15.55:0/3282157595 wait complete. 2023-10-06T05:05:25.397 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 87 2023-10-06T05:05:25.419 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:05:25.494 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:25.492+0000 7f00f806e180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:05:25.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.636+0000 7fb9effff700 1 Processor -- start 2023-10-06T05:05:25.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.636+0000 7fb9effff700 1 -- start start 2023-10-06T05:05:25.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.638+0000 7fb9effff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9f015daa0 0x7fb9f006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:25.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.638+0000 7fb9effff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb9f00696c0 0x7fb9f006ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:25.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.638+0000 7fb9effff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9f006e8c0 con 0x7fb9f015de70 2023-10-06T05:05:25.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.638+0000 7fb9effff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f00ca740 con 0x7fb9f00696c0 2023-10-06T05:05:25.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.638+0000 7fb9effff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f006e5e0 con 0x7fb9f015daa0 2023-10-06T05:05:25.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.639+0000 7fb9ee7fc700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fb9f015de70 0x7fb9f00c7110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:51354/0 (socket says 172.21.15.55:51354) 2023-10-06T05:05:25.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.639+0000 7fb9ee7fc700 1 -- 172.21.15.55:0/1815435746 learned_addr learned my addr 172.21.15.55:0/1815435746 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:25.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.639+0000 7fb9eeffd700 1 --2- 172.21.15.55:0/1815435746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9f015daa0 0x7fb9f006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:25.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.639+0000 7fb9ef7fe700 1 --2- 172.21.15.55:0/1815435746 >> v2:172.21.15.163:3300/0 conn(0x7fb9f00696c0 0x7fb9f006ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:25.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.639+0000 7fb9edffb700 1 -- 172.21.15.55:0/1815435746 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2607751064 0 0) 0x7fb9f006e8c0 con 0x7fb9f015de70 2023-10-06T05:05:25.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.639+0000 7fb9edffb700 1 -- 172.21.15.55:0/1815435746 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9dc003570 con 0x7fb9f015de70 2023-10-06T05:05:25.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.639+0000 7fb9eeffd700 1 -- 172.21.15.55:0/1815435746 >> v2:172.21.15.163:3300/0 conn(0x7fb9f00696c0 msgr2=0x7fb9f006ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:25.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.639+0000 7fb9eeffd700 1 --2- 172.21.15.55:0/1815435746 >> v2:172.21.15.163:3300/0 conn(0x7fb9f00696c0 0x7fb9f006ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.639+0000 7fb9eeffd700 1 -- 172.21.15.55:0/1815435746 >> v1:172.21.15.163:6789/0 conn(0x7fb9f015de70 legacy=0x7fb9f00c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:25.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.639+0000 7fb9eeffd700 1 -- 172.21.15.55:0/1815435746 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f00ca0f0 con 0x7fb9f015daa0 2023-10-06T05:05:25.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.639+0000 7fb9ef7fe700 1 --2- 172.21.15.55:0/1815435746 >> v2:172.21.15.163:3300/0 conn(0x7fb9f00696c0 0x7fb9f006ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:25.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.640+0000 7fb9eeffd700 1 --2- 172.21.15.55:0/1815435746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9f015daa0 0x7fb9f006e000 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80098e0 tx=0x7fb9e80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5cdb6edb2162702 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:25.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.640+0000 7fb9edffb700 1 -- 172.21.15.55:0/1815435746 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e800b640 con 0x7fb9f015daa0 2023-10-06T05:05:25.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.640+0000 7fb9edffb700 1 -- 172.21.15.55:0/1815435746 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e800b7a0 con 0x7fb9f015daa0 2023-10-06T05:05:25.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.640+0000 7fb9edffb700 1 -- 172.21.15.55:0/1815435746 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e800bb40 con 0x7fb9f015daa0 2023-10-06T05:05:25.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.641+0000 7fb9effff700 1 -- 172.21.15.55:0/1815435746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9f015daa0 msgr2=0x7fb9f006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:25.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.641+0000 7fb9effff700 1 --2- 172.21.15.55:0/1815435746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9f015daa0 0x7fb9f006e000 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80098e0 tx=0x7fb9e80091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.641+0000 7fb9effff700 1 -- 172.21.15.55:0/1815435746 shutdown_connections 2023-10-06T05:05:25.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.641+0000 7fb9effff700 1 --2- 172.21.15.55:0/1815435746 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9f015daa0 0x7fb9f006e000 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.641+0000 7fb9effff700 1 --2- 172.21.15.55:0/1815435746 >> v2:172.21.15.163:3300/0 conn(0x7fb9f00696c0 0x7fb9f006ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.641+0000 7fb9effff700 1 -- 172.21.15.55:0/1815435746 >> 172.21.15.55:0/1815435746 conn(0x7fb9f00cc1f0 msgr2=0x7fb9f00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:25.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.641+0000 7fb9effff700 1 -- 172.21.15.55:0/1815435746 shutdown_connections 2023-10-06T05:05:25.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.642+0000 7fb9effff700 1 -- 172.21.15.55:0/1815435746 wait complete. 2023-10-06T05:05:25.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.643+0000 7fb9effff700 1 Processor -- start 2023-10-06T05:05:25.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.643+0000 7fb9effff700 1 -- start start 2023-10-06T05:05:25.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.644+0000 7fb9effff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9f00696c0 0x7fb9f01480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:25.648 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.644+0000 7fb9effff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9f015daa0 0x7fb9f014ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:25.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.644+0000 7fb9effff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9f015de70 0x7fb9f00b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:25.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.644+0000 7fb9effff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00a7cf0 con 0x7fb9f015de70 2023-10-06T05:05:25.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.644+0000 7fb9effff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00a8870 con 0x7fb9f00696c0 2023-10-06T05:05:25.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.644+0000 7fb9effff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00a7f40 con 0x7fb9f015daa0 2023-10-06T05:05:25.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.644+0000 7fb9ef7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9f015de70 0x7fb9f00b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:25.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.644+0000 7fb9ee7fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9f015daa0 0x7fb9f014ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:25.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.644+0000 7fb9ee7fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9f015daa0 0x7fb9f014ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:36806/0 (socket says 172.21.15.55:36806) 2023-10-06T05:05:25.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.644+0000 7fb9ee7fc700 1 -- 172.21.15.55:0/2934214023 learned_addr learned my addr 172.21.15.55:0/2934214023 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:25.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.644+0000 7fb9eeffd700 1 --2- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9f00696c0 0x7fb9f01480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:25.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.645+0000 7fb9ee7fc700 1 -- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9f00696c0 msgr2=0x7fb9f01480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:25.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.645+0000 7fb9ee7fc700 1 --2- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9f00696c0 0x7fb9f01480c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.645+0000 7fb9ee7fc700 1 -- 172.21.15.55:0/2934214023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9f015de70 msgr2=0x7fb9f00b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:25.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.645+0000 7fb9ee7fc700 1 --2- 172.21.15.55:0/2934214023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9f015de70 0x7fb9f00b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.645+0000 7fb9ee7fc700 1 -- 172.21.15.55:0/2934214023 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e8009420 con 0x7fb9f015daa0 2023-10-06T05:05:25.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.645+0000 7fb9ee7fc700 1 --2- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9f015daa0 0x7fb9f014ab10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800cca0 tx=0x7fb9d8001aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:25.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.645+0000 7fb9e6ffd700 1 -- 172.21.15.55:0/2934214023 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d800a8d0 con 0x7fb9f015daa0 2023-10-06T05:05:25.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.645+0000 7fb9e6ffd700 1 -- 172.21.15.55:0/2934214023 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d800aa50 con 0x7fb9f015daa0 2023-10-06T05:05:25.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.645+0000 7fb9effff700 1 -- 172.21.15.55:0/2934214023 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f00a8220 con 0x7fb9f015daa0 2023-10-06T05:05:25.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.645+0000 7fb9e6ffd700 1 -- 172.21.15.55:0/2934214023 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d800ae10 con 0x7fb9f015daa0 2023-10-06T05:05:25.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.645+0000 7fb9effff700 1 -- 172.21.15.55:0/2934214023 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f014b050 con 0x7fb9f015daa0 2023-10-06T05:05:25.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.646+0000 7fb9effff700 1 -- 172.21.15.55:0/2934214023 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f00a8a00 con 0x7fb9f015daa0 2023-10-06T05:05:25.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.647+0000 7fb9e6ffd700 1 -- 172.21.15.55:0/2934214023 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 88) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fb9d8007300 con 0x7fb9f015daa0 2023-10-06T05:05:25.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.648+0000 7fb9e6ffd700 1 --2- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb9d0067050 0x7fb9d0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:25.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.648+0000 7fb9e6ffd700 1 -- 172.21.15.55:0/2934214023 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fb9d8084f70 con 0x7fb9f015daa0 2023-10-06T05:05:25.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.648+0000 7fb9eeffd700 1 -- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb9d0067050 msgr2=0x7fb9d0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:25.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.648+0000 7fb9eeffd700 1 --2- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb9d0067050 0x7fb9d0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:25.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.652+0000 7fb9e6ffd700 1 -- 172.21.15.55:0/2934214023 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9d8051b20 con 0x7fb9f015daa0 2023-10-06T05:05:25.666 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:25.664+0000 7f00f806e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:05:25.707 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:25.705+0000 7fdef7987180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:05:25.764 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:25.762+0000 7f00f806e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:05:25.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.849+0000 7fb9eeffd700 1 -- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb9d0067050 msgr2=0x7fb9d0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:25.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.849+0000 7fb9eeffd700 1 --2- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb9d0067050 0x7fb9d0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:25.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.860+0000 7fb9effff700 1 -- 172.21.15.55:0/2934214023 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9f004c3c0 con 0x7fb9f015daa0 2023-10-06T05:05:25.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.862+0000 7fb9e6ffd700 1 -- 172.21.15.55:0/2934214023 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+18558 (secure 0 0 0) 0x7fb9d8055140 con 0x7fb9f015daa0 2023-10-06T05:05:25.864 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:25.864 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":181,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:05:24.813738+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:49:17.992792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T04:49:18.785942+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":181,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.12","mappings":[{"from":3,"to":0}]},{"pgid":"22.23","mappings":[{"from":3,"to":1}]},{"pgid":"22.37","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/1915465424":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/3085093567":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/2727475042":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/167943248":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/3594252861":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/3141284388":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/2833364221":"2023-10-07T04:49:40.797751+0000","172.21.15.163:6836/85191":"2023-10-07T04:49:40.797751+0000","172.21.15.163:0/4127328436":"2023-10-07T04:49:40.797751+0000","172.21.15.163:6837/85191":"2023-10-07T04:49:40.797751+0000","172.21.15.163:0/2876114056":"2023-10-07T04:49:40.797751+0000","172.21.15.163:0/3803111782":"2023-10-07T04:49:40.797751+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":22,"snaps":[{"begin":17,"length":1},{"begin":33,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:05:25.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.864+0000 7fb9effff700 1 -- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb9d0067050 msgr2=0x7fb9d0069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:25.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.864+0000 7fb9effff700 1 --2- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb9d0067050 0x7fb9d0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.864+0000 7fb9effff700 1 -- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9f015daa0 msgr2=0x7fb9f014ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:25.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.864+0000 7fb9effff700 1 --2- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9f015daa0 0x7fb9f014ab10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800cca0 tx=0x7fb9d8001aa0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.864+0000 7fb9effff700 1 -- 172.21.15.55:0/2934214023 shutdown_connections 2023-10-06T05:05:25.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.864+0000 7fb9effff700 1 --2- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb9d0067050 0x7fb9d0069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.864+0000 7fb9effff700 1 --2- 172.21.15.55:0/2934214023 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb9f015de70 0x7fb9f00b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.864+0000 7fb9effff700 1 --2- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb9f00696c0 0x7fb9f01480c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.864+0000 7fb9effff700 1 --2- 172.21.15.55:0/2934214023 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb9f015daa0 0x7fb9f014ab10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:25.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.864+0000 7fb9effff700 1 -- 172.21.15.55:0/2934214023 >> 172.21.15.55:0/2934214023 conn(0x7fb9f00cc1f0 msgr2=0x7fb9f00c1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:25.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.864+0000 7fb9effff700 1 -- 172.21.15.55:0/2934214023 shutdown_connections 2023-10-06T05:05:25.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:25.864+0000 7fb9effff700 1 -- 172.21.15.55:0/2934214023 wait complete. 2023-10-06T05:05:25.884 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:05:25.885 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:05:25.890 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:25.889+0000 7fdef7987180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:05:25.994 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:25.992+0000 7fdef7987180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:05:26.062 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:26.060+0000 7f00f806e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:05:26.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.072+0000 7fc4eb59e700 1 Processor -- start 2023-10-06T05:05:26.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.072+0000 7fc4eb59e700 1 -- start start 2023-10-06T05:05:26.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.074+0000 7fc4eb59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc4ec1595a0 0x7fc4ec0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:26.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.074+0000 7fc4eb59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc4ec0c6c60 0x7fc4ec0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.074+0000 7fc4eb59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4ec0b1410 con 0x7fc4ec0b1de0 2023-10-06T05:05:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.074+0000 7fc4eb59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4ec0b2b90 con 0x7fc4ec1595a0 2023-10-06T05:05:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.074+0000 7fc4eb59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4ec063230 con 0x7fc4ec0c6c60 2023-10-06T05:05:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.074+0000 7fc4ea59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc4ec1595a0 0x7fc4ec0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.074+0000 7fc4ea59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc4ec1595a0 0x7fc4ec0b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58170/0 (socket says 172.21.15.55:58170) 2023-10-06T05:05:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.074+0000 7fc4ea59c700 1 -- 172.21.15.55:0/806449767 learned_addr learned my addr 172.21.15.55:0/806449767 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.074+0000 7fc4e9d9b700 1 --2- 172.21.15.55:0/806449767 >> v2:172.21.15.163:3300/0 conn(0x7fc4ec0c6c60 0x7fc4ec0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.074+0000 7fc4ea59c700 1 -- 172.21.15.55:0/806449767 >> v2:172.21.15.163:3300/0 conn(0x7fc4ec0c6c60 msgr2=0x7fc4ec0c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:26.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.074+0000 7fc4ea59c700 1 --2- 172.21.15.55:0/806449767 >> v2:172.21.15.163:3300/0 conn(0x7fc4ec0c6c60 0x7fc4ec0c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.074+0000 7fc4ea59c700 1 -- 172.21.15.55:0/806449767 >> v1:172.21.15.163:6789/0 conn(0x7fc4ec0b1de0 legacy=0x7fc4ec15d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:26.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.074+0000 7fc4ea59c700 1 -- 172.21.15.55:0/806449767 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec0634f0 con 0x7fc4ec1595a0 2023-10-06T05:05:26.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.074+0000 7fc4e9d9b700 1 --2- 172.21.15.55:0/806449767 >> v2:172.21.15.163:3300/0 conn(0x7fc4ec0c6c60 0x7fc4ec0c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:26.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.075+0000 7fc4ea59c700 1 --2- 172.21.15.55:0/806449767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4ec1595a0 0x7fc4ec0b18a0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0b07d0 tx=0x7fc4dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392443f613434505 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:26.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.075+0000 7fc4e959a700 1 -- 172.21.15.55:0/806449767 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc00b860 con 0x7fc4ec1595a0 2023-10-06T05:05:26.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.075+0000 7fc4e959a700 1 -- 172.21.15.55:0/806449767 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc00b9c0 con 0x7fc4ec1595a0 2023-10-06T05:05:26.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.075+0000 7fc4e959a700 1 -- 172.21.15.55:0/806449767 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc00bd60 con 0x7fc4ec1595a0 2023-10-06T05:05:26.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.076+0000 7fc4eb59e700 1 -- 172.21.15.55:0/806449767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4ec1595a0 msgr2=0x7fc4ec0b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:26.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.076+0000 7fc4eb59e700 1 --2- 172.21.15.55:0/806449767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4ec1595a0 0x7fc4ec0b18a0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0b07d0 tx=0x7fc4dc009250 comp rx=0 tx=0).stop 2023-10-06T05:05:26.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.076+0000 7fc4eb59e700 1 -- 172.21.15.55:0/806449767 shutdown_connections 2023-10-06T05:05:26.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.076+0000 7fc4eb59e700 1 --2- 172.21.15.55:0/806449767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4ec1595a0 0x7fc4ec0b18a0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.076+0000 7fc4eb59e700 1 --2- 172.21.15.55:0/806449767 >> v2:172.21.15.163:3300/0 conn(0x7fc4ec0c6c60 0x7fc4ec0c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.076+0000 7fc4eb59e700 1 -- 172.21.15.55:0/806449767 >> 172.21.15.55:0/806449767 conn(0x7fc4ec06ddb0 msgr2=0x7fc4ec06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:26.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.076+0000 7fc4eb59e700 1 -- 172.21.15.55:0/806449767 shutdown_connections 2023-10-06T05:05:26.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.076+0000 7fc4eb59e700 1 -- 172.21.15.55:0/806449767 wait complete. 2023-10-06T05:05:26.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.077+0000 7fc4eb59e700 1 Processor -- start 2023-10-06T05:05:26.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.077+0000 7fc4eb59e700 1 -- start start 2023-10-06T05:05:26.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.078+0000 7fc4eb59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4ec0b1de0 0x7fc4ec141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:26.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.078+0000 7fc4eb59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4ec0c6c60 0x7fc4ec169f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:26.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.078+0000 7fc4eb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4ec1595a0 0x7fc4ec1410e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:26.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.078+0000 7fc4eb59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec0c2450 con 0x7fc4ec1595a0 2023-10-06T05:05:26.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.078+0000 7fc4eb59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec16aec0 con 0x7fc4ec0c6c60 2023-10-06T05:05:26.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.078+0000 7fc4eb59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec16a620 con 0x7fc4ec0b1de0 2023-10-06T05:05:26.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.079+0000 7fc4ead9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4ec1595a0 0x7fc4ec1410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:26.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.079+0000 7fc4e9d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4ec0c6c60 0x7fc4ec169f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:26.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.079+0000 7fc4ead9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4ec1595a0 0x7fc4ec1410e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58178/0 (socket says 172.21.15.55:58178) 2023-10-06T05:05:26.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.079+0000 7fc4ead9d700 1 -- 172.21.15.55:0/1013808254 learned_addr learned my addr 172.21.15.55:0/1013808254 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:26.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.079+0000 7fc4ea59c700 1 --2- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4ec0b1de0 0x7fc4ec141fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:26.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.079+0000 7fc4ead9d700 1 -- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4ec0b1de0 msgr2=0x7fc4ec141fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:26.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.079+0000 7fc4ead9d700 1 --2- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4ec0b1de0 0x7fc4ec141fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.079+0000 7fc4ead9d700 1 -- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4ec0c6c60 msgr2=0x7fc4ec169f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:26.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.079+0000 7fc4ead9d700 1 --2- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4ec0c6c60 0x7fc4ec169f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.079+0000 7fc4ead9d700 1 -- 172.21.15.55:0/1013808254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc0094f0 con 0x7fc4ec1595a0 2023-10-06T05:05:26.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.079+0000 7fc4ead9d700 1 --2- 172.21.15.55:0/1013808254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4ec1595a0 0x7fc4ec1410e0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c910 tx=0x7fc4e4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:26.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.080+0000 7fc4da7fc700 1 -- 172.21.15.55:0/1013808254 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4012630 con 0x7fc4ec1595a0 2023-10-06T05:05:26.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.080+0000 7fc4da7fc700 1 -- 172.21.15.55:0/1013808254 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e40127b0 con 0x7fc4ec1595a0 2023-10-06T05:05:26.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.080+0000 7fc4da7fc700 1 -- 172.21.15.55:0/1013808254 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4012b70 con 0x7fc4ec1595a0 2023-10-06T05:05:26.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.080+0000 7fc4eb59e700 1 -- 172.21.15.55:0/1013808254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ec16a900 con 0x7fc4ec1595a0 2023-10-06T05:05:26.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.080+0000 7fc4eb59e700 1 -- 172.21.15.55:0/1013808254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ec159aa0 con 0x7fc4ec1595a0 2023-10-06T05:05:26.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.081+0000 7fc4eb59e700 1 -- 172.21.15.55:0/1013808254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ec04c3c0 con 0x7fc4ec1595a0 2023-10-06T05:05:26.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.082+0000 7fc4da7fc700 1 -- 172.21.15.55:0/1013808254 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 88) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fc4e4003030 con 0x7fc4ec1595a0 2023-10-06T05:05:26.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.083+0000 7fc4da7fc700 1 --2- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc4cc067050 0x7fc4cc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:26.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.083+0000 7fc4da7fc700 1 -- 172.21.15.55:0/1013808254 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fc4e4084c00 con 0x7fc4ec1595a0 2023-10-06T05:05:26.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.083+0000 7fc4ea59c700 1 -- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc4cc067050 msgr2=0x7fc4cc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:26.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.083+0000 7fc4ea59c700 1 --2- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc4cc067050 0x7fc4cc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:26.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.087+0000 7fc4da7fc700 1 -- 172.21.15.55:0/1013808254 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4e4051730 con 0x7fc4ec1595a0 2023-10-06T05:05:26.159 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:26.158+0000 7f00f806e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:05:26.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.284+0000 7fc4ea59c700 1 -- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc4cc067050 msgr2=0x7fc4cc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:26.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.284+0000 7fc4ea59c700 1 --2- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc4cc067050 0x7fc4cc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:26.300 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:26.298+0000 7fdef7987180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:05:26.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.336+0000 7fc4eb59e700 1 -- 172.21.15.55:0/1013808254 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4ec0637d0 con 0x7fc4ec1595a0 2023-10-06T05:05:26.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.336+0000 7fc4da7fc700 1 -- 172.21.15.55:0/1013808254 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4604 (secure 0 0 0) 0x7fc4e4054d50 con 0x7fc4ec1595a0 2023-10-06T05:05:26.338 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:26.338 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7682,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1565234675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1565234675},{"type":"v1","addr":"172.21.15.55:6834","nonce":1565234675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3042024952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3042024952},{"type":"v1","addr":"172.21.15.163:6835","nonce":3042024952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8737,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/304924610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":304924610},{"type":"v1","addr":"172.21.15.55:6835","nonce":304924610}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:49:19.807866+0000","modified":"2023-10-06T04:49:23.339948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8740":{"gid":8740,"name":"b","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6834/3095914721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3095914721},{"type":"v1","addr":"172.21.15.163:6834","nonce":3095914721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:05:26.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.338+0000 7fc4eb59e700 1 -- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc4cc067050 msgr2=0x7fc4cc069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:26.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.338+0000 7fc4eb59e700 1 --2- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc4cc067050 0x7fc4cc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.338+0000 7fc4eb59e700 1 -- 172.21.15.55:0/1013808254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4ec1595a0 msgr2=0x7fc4ec1410e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:26.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.338+0000 7fc4eb59e700 1 --2- 172.21.15.55:0/1013808254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4ec1595a0 0x7fc4ec1410e0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc4e400c910 tx=0x7fc4e4001710 comp rx=0 tx=0).stop 2023-10-06T05:05:26.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.339+0000 7fc4eb59e700 1 -- 172.21.15.55:0/1013808254 shutdown_connections 2023-10-06T05:05:26.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.339+0000 7fc4eb59e700 1 --2- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc4cc067050 0x7fc4cc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.339+0000 7fc4eb59e700 1 --2- 172.21.15.55:0/1013808254 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc4ec1595a0 0x7fc4ec1410e0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.339+0000 7fc4eb59e700 1 --2- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc4ec0b1de0 0x7fc4ec141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.339+0000 7fc4eb59e700 1 --2- 172.21.15.55:0/1013808254 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc4ec0c6c60 0x7fc4ec169f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.339+0000 7fc4eb59e700 1 -- 172.21.15.55:0/1013808254 >> 172.21.15.55:0/1013808254 conn(0x7fc4ec06ddb0 msgr2=0x7fc4ec15c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:26.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.339+0000 7fc4eb59e700 1 -- 172.21.15.55:0/1013808254 shutdown_connections 2023-10-06T05:05:26.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.339+0000 7fc4eb59e700 1 -- 172.21.15.55:0/1013808254 wait complete. 2023-10-06T05:05:26.342 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 87 2023-10-06T05:05:26.359 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:05:26.405 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:26.400+0000 7fdef7987180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:05:26.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.587+0000 7f1fee7ff700 1 Processor -- start 2023-10-06T05:05:26.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.587+0000 7f1fee7ff700 1 -- start start 2023-10-06T05:05:26.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.589+0000 7f1fee7ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1fe8159890 0x7f1fe815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:26.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.589+0000 7f1fee7ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe815a160 con 0x7f1fe8069560 2023-10-06T05:05:26.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.589+0000 7f1fee7ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe80d13a0 con 0x7f1fe8159c60 2023-10-06T05:05:26.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.589+0000 7f1fee7ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe815e140 con 0x7f1fe8159890 2023-10-06T05:05:26.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.589+0000 7f1fecd9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1fe8069560 0x7f1fe80ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52548/0 (socket says 172.21.15.55:52548) 2023-10-06T05:05:26.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.589+0000 7f1fecd9c700 1 -- 172.21.15.55:0/2308828315 learned_addr learned my addr 172.21.15.55:0/2308828315 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:26.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.589+0000 7f1fe7fff700 1 --2- 172.21.15.55:0/2308828315 >> v2:172.21.15.163:3300/0 conn(0x7f1fe8159890 0x7f1fe815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:26.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.589+0000 7f1fe6ffd700 1 -- 172.21.15.55:0/2308828315 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536436164 0 0) 0x7f1fe815a160 con 0x7f1fe8069560 2023-10-06T05:05:26.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.590+0000 7f1fe6ffd700 1 -- 172.21.15.55:0/2308828315 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fd0003570 con 0x7f1fe8069560 2023-10-06T05:05:26.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.590+0000 7f1fe6ffd700 1 -- 172.21.15.55:0/2308828315 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002143954 0 0) 0x7f1fe80d13a0 con 0x7f1fe8159c60 2023-10-06T05:05:26.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.590+0000 7f1fe6ffd700 1 -- 172.21.15.55:0/2308828315 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fe815a160 con 0x7f1fe8159c60 2023-10-06T05:05:26.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.590+0000 7f1fe7fff700 1 -- 172.21.15.55:0/2308828315 >> v1:172.21.15.163:6789/0 conn(0x7f1fe8159c60 legacy=0x7f1fe815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:26.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.590+0000 7f1fe7fff700 1 -- 172.21.15.55:0/2308828315 >> v1:172.21.15.55:6789/0 conn(0x7f1fe8069560 legacy=0x7f1fe80ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:26.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.590+0000 7f1fe7fff700 1 -- 172.21.15.55:0/2308828315 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe815a3d0 con 0x7f1fe8159890 2023-10-06T05:05:26.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.590+0000 7f1fe7fff700 1 --2- 172.21.15.55:0/2308828315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe8159890 0x7f1fe815abd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80098e0 tx=0x7f1fd80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dd05f1bbee93f84 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:26.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.590+0000 7f1fe6ffd700 1 -- 172.21.15.55:0/2308828315 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8011630 con 0x7f1fe8159890 2023-10-06T05:05:26.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.590+0000 7f1fe6ffd700 1 -- 172.21.15.55:0/2308828315 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd8011790 con 0x7f1fe8159890 2023-10-06T05:05:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.590+0000 7f1fe6ffd700 1 -- 172.21.15.55:0/2308828315 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8011b30 con 0x7f1fe8159890 2023-10-06T05:05:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.591+0000 7f1fee7ff700 1 -- 172.21.15.55:0/2308828315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe8159890 msgr2=0x7f1fe815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.591+0000 7f1fee7ff700 1 --2- 172.21.15.55:0/2308828315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe8159890 0x7f1fe815abd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80098e0 tx=0x7f1fd80091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.591+0000 7f1fee7ff700 1 -- 172.21.15.55:0/2308828315 shutdown_connections 2023-10-06T05:05:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.591+0000 7f1fee7ff700 1 --2- 172.21.15.55:0/2308828315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe8159890 0x7f1fe815abd0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.591+0000 7f1fee7ff700 1 -- 172.21.15.55:0/2308828315 >> 172.21.15.55:0/2308828315 conn(0x7f1fe8068e20 msgr2=0x7f1fe80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.591+0000 7f1fee7ff700 1 -- 172.21.15.55:0/2308828315 shutdown_connections 2023-10-06T05:05:26.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.591+0000 7f1fee7ff700 1 -- 172.21.15.55:0/2308828315 wait complete. 2023-10-06T05:05:26.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.592+0000 7f1fee7ff700 1 Processor -- start 2023-10-06T05:05:26.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.593+0000 7f1fee7ff700 1 -- start start 2023-10-06T05:05:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fee7ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe8069560 0x7f1fe80ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fee7ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fe8159890 0x7f1fe8152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fee7ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fe8159c60 0x7f1fe80b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fee7ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80ca750 con 0x7f1fe8159c60 2023-10-06T05:05:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fee7ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80c9620 con 0x7f1fe8069560 2023-10-06T05:05:26.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fee7ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80c8a10 con 0x7f1fe8159890 2023-10-06T05:05:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fecd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fe8159c60 0x7f1fe80b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fecd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fe8159c60 0x7f1fe80b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58186/0 (socket says 172.21.15.55:58186) 2023-10-06T05:05:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fecd9c700 1 -- 172.21.15.55:0/107449480 learned_addr learned my addr 172.21.15.55:0/107449480 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fe7fff700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe8069560 0x7f1fe80ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:26.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fe77fe700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fe8159890 0x7f1fe8152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fecd9c700 1 -- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fe8159890 msgr2=0x7f1fe8152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fecd9c700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fe8159890 0x7f1fe8152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fecd9c700 1 -- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe8069560 msgr2=0x7f1fe80ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fecd9c700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe8069560 0x7f1fe80ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fecd9c700 1 -- 172.21.15.55:0/107449480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd8009420 con 0x7f1fe8159c60 2023-10-06T05:05:26.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.594+0000 7f1fe7fff700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe8069560 0x7f1fe80ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:26.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.595+0000 7f1fecd9c700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fe8159c60 0x7f1fe80b3590 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000cd00 tx=0x7f1fe0001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:26.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.595+0000 7f1fcffff700 1 -- 172.21.15.55:0/107449480 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe000ce00 con 0x7f1fe8159c60 2023-10-06T05:05:26.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.595+0000 7f1fcffff700 1 -- 172.21.15.55:0/107449480 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe000aac0 con 0x7f1fe8159c60 2023-10-06T05:05:26.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.595+0000 7f1fcffff700 1 -- 172.21.15.55:0/107449480 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe000ae80 con 0x7f1fe8159c60 2023-10-06T05:05:26.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.595+0000 7f1fee7ff700 1 -- 172.21.15.55:0/107449480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe80c8cf0 con 0x7f1fe8159c60 2023-10-06T05:05:26.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.595+0000 7f1fee7ff700 1 -- 172.21.15.55:0/107449480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe80c92b0 con 0x7f1fe8159c60 2023-10-06T05:05:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.596+0000 7f1fee7ff700 1 -- 172.21.15.55:0/107449480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe8069bb0 con 0x7f1fe8159c60 2023-10-06T05:05:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.597+0000 7f1fcffff700 1 -- 172.21.15.55:0/107449480 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 88) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f1fe0007300 con 0x7f1fe8159c60 2023-10-06T05:05:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.598+0000 7f1fcffff700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1fd4067050 0x7f1fd4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.598+0000 7f1fcffff700 1 -- 172.21.15.55:0/107449480 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 7125+0+0 (secure 0 0 0) 0x7f1fe0084ff0 con 0x7f1fe8159c60 2023-10-06T05:05:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.598+0000 7f1fe7fff700 1 -- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1fd4067050 msgr2=0x7f1fd4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.598+0000 7f1fe7fff700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1fd4067050 0x7f1fd4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:26.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.602+0000 7f1fcffff700 1 -- 172.21.15.55:0/107449480 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1fe0051b60 con 0x7f1fe8159c60 2023-10-06T05:05:26.801 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.799+0000 7f1fe7fff700 1 -- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1fd4067050 msgr2=0x7f1fd4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:26.801 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.799+0000 7f1fe7fff700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1fd4067050 0x7f1fd4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:26.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.855+0000 7f1fee7ff700 1 -- 172.21.15.55:0/107449480 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fe80c9790 con 0x7f1fe8159c60 2023-10-06T05:05:26.857 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.855+0000 7f1fcffff700 1 -- 172.21.15.55:0/107449480 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f1fe0055180 con 0x7f1fe8159c60 2023-10-06T05:05:26.857 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:26.858 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7682,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1565234675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1565234675},{"type":"v1","addr":"172.21.15.55:6834","nonce":1565234675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3042024952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3042024952},{"type":"v1","addr":"172.21.15.163:6835","nonce":3042024952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8737,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/304924610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":304924610},{"type":"v1","addr":"172.21.15.55:6835","nonce":304924610}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:49:19.807866+0000","modified":"2023-10-06T04:49:23.339948+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8740":{"gid":8740,"name":"b","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.163:6834/3095914721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3095914721},{"type":"v1","addr":"172.21.15.163:6834","nonce":3095914721}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-06T05:05:26.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.858+0000 7f1fee7ff700 1 -- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1fd4067050 msgr2=0x7f1fd4069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:26.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.858+0000 7f1fee7ff700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1fd4067050 0x7f1fd4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.858+0000 7f1fee7ff700 1 -- 172.21.15.55:0/107449480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fe8159c60 msgr2=0x7f1fe80b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:26.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.858+0000 7f1fee7ff700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fe8159c60 0x7f1fe80b3590 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000cd00 tx=0x7f1fe0001ac0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.858+0000 7f1fee7ff700 1 -- 172.21.15.55:0/107449480 shutdown_connections 2023-10-06T05:05:26.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.858+0000 7f1fee7ff700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1fd4067050 0x7f1fd4069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.858+0000 7f1fee7ff700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fe8159c60 0x7f1fe80b3590 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.858+0000 7f1fee7ff700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fe8069560 0x7f1fe80ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.858+0000 7f1fee7ff700 1 --2- 172.21.15.55:0/107449480 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fe8159890 0x7f1fe8152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:26.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.858+0000 7f1fee7ff700 1 -- 172.21.15.55:0/107449480 >> 172.21.15.55:0/107449480 conn(0x7f1fe8068e20 msgr2=0x7f1fe8152b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:26.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.858+0000 7f1fee7ff700 1 -- 172.21.15.55:0/107449480 shutdown_connections 2023-10-06T05:05:26.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:26.858+0000 7f1fee7ff700 1 -- 172.21.15.55:0/107449480 wait complete. 2023-10-06T05:05:26.862 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 87 2023-10-06T05:05:26.865 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:26.864+0000 7f00f806e180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:05:26.878 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:05:27.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.068+0000 7eff6cb9d700 1 Processor -- start 2023-10-06T05:05:27.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.068+0000 7eff6cb9d700 1 -- start start 2023-10-06T05:05:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.069+0000 7eff6cb9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7eff681597f0 0x7eff680b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.069+0000 7eff6cb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff6815d230 0x7eff6815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.069+0000 7eff6cb9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff6815dbb0 con 0x7eff68159420 2023-10-06T05:05:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.069+0000 7eff6cb9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff680b11d0 con 0x7eff681597f0 2023-10-06T05:05:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.069+0000 7eff6cb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff680b14b0 con 0x7eff6815d230 2023-10-06T05:05:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.070+0000 7eff66ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7eff681597f0 0x7eff680b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.070+0000 7eff67fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff6815d230 0x7eff6815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.070+0000 7eff66ffd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7eff681597f0 0x7eff680b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:42812/0 (socket says 172.21.15.55:42812) 2023-10-06T05:05:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.070+0000 7eff677fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7eff68159420 0x7eff680b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:51380/0 (socket says 172.21.15.55:51380) 2023-10-06T05:05:27.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.070+0000 7eff66ffd700 1 -- 172.21.15.55:0/1411723052 learned_addr learned my addr 172.21.15.55:0/1411723052 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:27.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.070+0000 7eff667fc700 1 -- 172.21.15.55:0/1411723052 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615381571 0 0) 0x7eff6815dbb0 con 0x7eff68159420 2023-10-06T05:05:27.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.070+0000 7eff667fc700 1 -- 172.21.15.55:0/1411723052 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff50003570 con 0x7eff68159420 2023-10-06T05:05:27.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.070+0000 7eff66ffd700 1 -- 172.21.15.55:0/1411723052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff6815d230 msgr2=0x7eff6815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:27.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.070+0000 7eff66ffd700 1 --2- 172.21.15.55:0/1411723052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff6815d230 0x7eff6815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.070+0000 7eff66ffd700 1 -- 172.21.15.55:0/1411723052 >> v1:172.21.15.163:6789/0 conn(0x7eff68159420 legacy=0x7eff680b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:27.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.070+0000 7eff66ffd700 1 -- 172.21.15.55:0/1411723052 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680b1790 con 0x7eff681597f0 2023-10-06T05:05:27.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.070+0000 7eff67fff700 1 --2- 172.21.15.55:0/1411723052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff6815d230 0x7eff6815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:27.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.071+0000 7eff66ffd700 1 --2- 172.21.15.55:0/1411723052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff681597f0 0x7eff680b2270 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0099b0 tx=0x7eff5c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8943ec076fa13ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:27.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.071+0000 7eff667fc700 1 -- 172.21.15.55:0/1411723052 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00b680 con 0x7eff681597f0 2023-10-06T05:05:27.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.071+0000 7eff667fc700 1 -- 172.21.15.55:0/1411723052 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c00b7e0 con 0x7eff681597f0 2023-10-06T05:05:27.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.071+0000 7eff667fc700 1 -- 172.21.15.55:0/1411723052 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00bb80 con 0x7eff681597f0 2023-10-06T05:05:27.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.071+0000 7eff6cb9d700 1 -- 172.21.15.55:0/1411723052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff681597f0 msgr2=0x7eff680b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:27.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.071+0000 7eff6cb9d700 1 --2- 172.21.15.55:0/1411723052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff681597f0 0x7eff680b2270 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0099b0 tx=0x7eff5c009250 comp rx=0 tx=0).stop 2023-10-06T05:05:27.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.072+0000 7eff6cb9d700 1 -- 172.21.15.55:0/1411723052 shutdown_connections 2023-10-06T05:05:27.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.072+0000 7eff6cb9d700 1 --2- 172.21.15.55:0/1411723052 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff6815d230 0x7eff6815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.072+0000 7eff6cb9d700 1 --2- 172.21.15.55:0/1411723052 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff681597f0 0x7eff680b2270 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.072+0000 7eff6cb9d700 1 -- 172.21.15.55:0/1411723052 >> 172.21.15.55:0/1411723052 conn(0x7eff680d0660 msgr2=0x7eff680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:27.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.072+0000 7eff6cb9d700 1 -- 172.21.15.55:0/1411723052 shutdown_connections 2023-10-06T05:05:27.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.072+0000 7eff6cb9d700 1 -- 172.21.15.55:0/1411723052 wait complete. 2023-10-06T05:05:27.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.073+0000 7eff6cb9d700 1 Processor -- start 2023-10-06T05:05:27.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.073+0000 7eff6cb9d700 1 -- start start 2023-10-06T05:05:27.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.074+0000 7eff6cb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff68159420 0x7eff6814c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:27.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.074+0000 7eff6cb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff681597f0 0x7eff680c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:27.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.074+0000 7eff6cb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff6815d230 0x7eff680c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:27.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.074+0000 7eff6cb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6814c710 con 0x7eff6815d230 2023-10-06T05:05:27.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.074+0000 7eff6cb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff680ca6d0 con 0x7eff681597f0 2023-10-06T05:05:27.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.074+0000 7eff6cb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6816ceb0 con 0x7eff68159420 2023-10-06T05:05:27.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.075+0000 7eff67fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff6815d230 0x7eff680c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:27.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.075+0000 7eff677fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff68159420 0x7eff6814c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:27.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.075+0000 7eff66ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff681597f0 0x7eff680c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:27.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.075+0000 7eff67fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff6815d230 0x7eff680c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58190/0 (socket says 172.21.15.55:58190) 2023-10-06T05:05:27.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.075+0000 7eff67fff700 1 -- 172.21.15.55:0/4248025753 learned_addr learned my addr 172.21.15.55:0/4248025753 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:27.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.075+0000 7eff677fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff68159420 0x7eff6814c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:36834/0 (socket says 172.21.15.55:36834) 2023-10-06T05:05:27.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.075+0000 7eff66ffd700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff681597f0 0x7eff680c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:42824/0 (socket says 172.21.15.55:42824) 2023-10-06T05:05:27.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.075+0000 7eff67fff700 1 -- 172.21.15.55:0/4248025753 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff68159420 msgr2=0x7eff6814c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:27.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.075+0000 7eff67fff700 1 --2- 172.21.15.55:0/4248025753 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff68159420 0x7eff6814c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.075+0000 7eff67fff700 1 -- 172.21.15.55:0/4248025753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff681597f0 msgr2=0x7eff680c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:27.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.075+0000 7eff67fff700 1 --2- 172.21.15.55:0/4248025753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff681597f0 0x7eff680c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.075+0000 7eff67fff700 1 -- 172.21.15.55:0/4248025753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff60009a20 con 0x7eff6815d230 2023-10-06T05:05:27.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.075+0000 7eff67fff700 1 --2- 172.21.15.55:0/4248025753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff6815d230 0x7eff680c7ae0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7eff60009f50 tx=0x7eff6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:27.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.075+0000 7eff4f7fe700 1 -- 172.21.15.55:0/4248025753 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6000caf0 con 0x7eff6815d230 2023-10-06T05:05:27.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.076+0000 7eff4f7fe700 1 -- 172.21.15.55:0/4248025753 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6000f040 con 0x7eff6815d230 2023-10-06T05:05:27.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.076+0000 7eff6cb9d700 1 -- 172.21.15.55:0/4248025753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5c0094f0 con 0x7eff6815d230 2023-10-06T05:05:27.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.076+0000 7eff4f7fe700 1 -- 172.21.15.55:0/4248025753 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6000cea0 con 0x7eff6815d230 2023-10-06T05:05:27.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.076+0000 7eff6cb9d700 1 -- 172.21.15.55:0/4248025753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff680c9e30 con 0x7eff6815d230 2023-10-06T05:05:27.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.077+0000 7eff6cb9d700 1 -- 172.21.15.55:0/4248025753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff680d03e0 con 0x7eff6815d230 2023-10-06T05:05:27.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.078+0000 7eff4f7fe700 1 -- 172.21.15.55:0/4248025753 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 88) v1 ==== 84854+0+0 (secure 0 0 0) 0x7eff60003020 con 0x7eff6815d230 2023-10-06T05:05:27.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.078+0000 7eff4f7fe700 1 --2- 172.21.15.55:0/4248025753 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7eff54067050 0x7eff54069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:27.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.079+0000 7eff4f7fe700 1 -- 172.21.15.55:0/4248025753 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 7125+0+0 (secure 0 0 0) 0x7eff60085630 con 0x7eff6815d230 2023-10-06T05:05:27.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.079+0000 7eff677fe700 1 -- 172.21.15.55:0/4248025753 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7eff54067050 msgr2=0x7eff54069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:27.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.079+0000 7eff677fe700 1 --2- 172.21.15.55:0/4248025753 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7eff54067050 0x7eff54069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:27.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.083+0000 7eff4f7fe700 1 -- 172.21.15.55:0/4248025753 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff600520f0 con 0x7eff6815d230 2023-10-06T05:05:27.157 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:27.156+0000 7f00f806e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:05:27.160 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:27.158+0000 7fdef7987180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:05:27.251 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:27.250+0000 7f00f806e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:05:27.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.280+0000 7eff677fe700 1 -- 172.21.15.55:0/4248025753 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7eff54067050 msgr2=0x7eff54069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:27.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.280+0000 7eff677fe700 1 --2- 172.21.15.55:0/4248025753 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7eff54067050 0x7eff54069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:27.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.289+0000 7eff6cb9d700 1 -- 172.21.15.55:0/4248025753 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff6804c3c0 con 0x7eff6815d230 2023-10-06T05:05:27.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.290+0000 7eff4f7fe700 1 -- 172.21.15.55:0/4248025753 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+18558 (secure 0 0 0) 0x7eff60055710 con 0x7eff6815d230 2023-10-06T05:05:27.292 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:27.293 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":181,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:05:24.813738+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:49:17.992792+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-10-06T04:49:18.785942+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":181,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.12","mappings":[{"from":3,"to":0}]},{"pgid":"22.23","mappings":[{"from":3,"to":1}]},{"pgid":"22.37","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/1915465424":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/3085093567":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/2727475042":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/167943248":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/3594252861":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/3141284388":"2023-10-07T05:00:23.678125+0000","172.21.15.163:0/2833364221":"2023-10-07T04:49:40.797751+0000","172.21.15.163:6836/85191":"2023-10-07T04:49:40.797751+0000","172.21.15.163:0/4127328436":"2023-10-07T04:49:40.797751+0000","172.21.15.163:6837/85191":"2023-10-07T04:49:40.797751+0000","172.21.15.163:0/2876114056":"2023-10-07T04:49:40.797751+0000","172.21.15.163:0/3803111782":"2023-10-07T04:49:40.797751+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":22,"snaps":[{"begin":17,"length":1},{"begin":33,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:05:27.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.292+0000 7eff6cb9d700 1 -- 172.21.15.55:0/4248025753 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7eff54067050 msgr2=0x7eff54069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:27.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.292+0000 7eff6cb9d700 1 --2- 172.21.15.55:0/4248025753 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7eff54067050 0x7eff54069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.293+0000 7eff6cb9d700 1 -- 172.21.15.55:0/4248025753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff6815d230 msgr2=0x7eff680c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:27.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.293+0000 7eff6cb9d700 1 --2- 172.21.15.55:0/4248025753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff6815d230 0x7eff680c7ae0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7eff60009f50 tx=0x7eff6000b040 comp rx=0 tx=0).stop 2023-10-06T05:05:27.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.293+0000 7eff6cb9d700 1 -- 172.21.15.55:0/4248025753 shutdown_connections 2023-10-06T05:05:27.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.293+0000 7eff6cb9d700 1 --2- 172.21.15.55:0/4248025753 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7eff54067050 0x7eff54069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.293+0000 7eff6cb9d700 1 --2- 172.21.15.55:0/4248025753 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7eff6815d230 0x7eff680c7ae0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.293+0000 7eff6cb9d700 1 --2- 172.21.15.55:0/4248025753 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7eff68159420 0x7eff6814c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.293+0000 7eff6cb9d700 1 --2- 172.21.15.55:0/4248025753 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7eff681597f0 0x7eff680c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.293+0000 7eff6cb9d700 1 -- 172.21.15.55:0/4248025753 >> 172.21.15.55:0/4248025753 conn(0x7eff680d0660 msgr2=0x7eff680cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:27.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.293+0000 7eff6cb9d700 1 -- 172.21.15.55:0/4248025753 shutdown_connections 2023-10-06T05:05:27.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.293+0000 7eff6cb9d700 1 -- 172.21.15.55:0/4248025753 wait complete. 2023-10-06T05:05:27.314 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:05:27.409 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:27.407+0000 7f00f806e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:05:27.461 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:27.459+0000 7fdef7987180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:05:27.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.508+0000 7fadeffff700 1 Processor -- start 2023-10-06T05:05:27.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.508+0000 7fadeffff700 1 -- start start 2023-10-06T05:05:27.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.509+0000 7fadeffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fadf00b1e50 0x7fadf0159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:27.511 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.509+0000 7fadeffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf00cfe20 con 0x7fadf00ceaa0 2023-10-06T05:05:27.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.509+0000 7fadeffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf00cf210 con 0x7fadf01593b0 2023-10-06T05:05:27.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.509+0000 7fadeffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf00cf4f0 con 0x7fadf00b1e50 2023-10-06T05:05:27.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.510+0000 7fadef7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fadf00ceaa0 0x7fadf00b3420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52558/0 (socket says 172.21.15.55:52558) 2023-10-06T05:05:27.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.510+0000 7fadef7fe700 1 -- 172.21.15.55:0/3471894984 learned_addr learned my addr 172.21.15.55:0/3471894984 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:27.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.510+0000 7fadee7fc700 1 --2- 172.21.15.55:0/3471894984 >> v2:172.21.15.163:3300/0 conn(0x7fadf00b1e50 0x7fadf0159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:27.512 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:27.510+0000 7f00f806e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:05:27.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.510+0000 7fadedffb700 1 -- 172.21.15.55:0/3471894984 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897484665 0 0) 0x7fadf00cfe20 con 0x7fadf00ceaa0 2023-10-06T05:05:27.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.510+0000 7fadedffb700 1 -- 172.21.15.55:0/3471894984 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadd8003570 con 0x7fadf00ceaa0 2023-10-06T05:05:27.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.510+0000 7fadedffb700 1 -- 172.21.15.55:0/3471894984 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 240064922 0 0) 0x7fadf00cf210 con 0x7fadf01593b0 2023-10-06T05:05:27.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.510+0000 7fadedffb700 1 -- 172.21.15.55:0/3471894984 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadf00cfe20 con 0x7fadf01593b0 2023-10-06T05:05:27.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.510+0000 7fadee7fc700 1 -- 172.21.15.55:0/3471894984 >> v1:172.21.15.163:6789/0 conn(0x7fadf01593b0 legacy=0x7fadf00b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:27.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.510+0000 7fadee7fc700 1 -- 172.21.15.55:0/3471894984 >> v1:172.21.15.55:6789/0 conn(0x7fadf00ceaa0 legacy=0x7fadf00b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:27.514 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.510+0000 7fadee7fc700 1 -- 172.21.15.55:0/3471894984 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf00cf7d0 con 0x7fadf00b1e50 2023-10-06T05:05:27.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.511+0000 7fadee7fc700 1 --2- 172.21.15.55:0/3471894984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf00b1e50 0x7fadf0159780 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fade40098e0 tx=0x7fade40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0c9c188385e4206 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:27.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.511+0000 7fadedffb700 1 -- 172.21.15.55:0/3471894984 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4011630 con 0x7fadf00b1e50 2023-10-06T05:05:27.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.511+0000 7fadedffb700 1 -- 172.21.15.55:0/3471894984 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade4011790 con 0x7fadf00b1e50 2023-10-06T05:05:27.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.511+0000 7fadedffb700 1 -- 172.21.15.55:0/3471894984 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4011b30 con 0x7fadf00b1e50 2023-10-06T05:05:27.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.511+0000 7fadeffff700 1 -- 172.21.15.55:0/3471894984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf00b1e50 msgr2=0x7fadf0159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:27.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.511+0000 7fadeffff700 1 --2- 172.21.15.55:0/3471894984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf00b1e50 0x7fadf0159780 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fade40098e0 tx=0x7fade40091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.512+0000 7fadeffff700 1 -- 172.21.15.55:0/3471894984 shutdown_connections 2023-10-06T05:05:27.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.512+0000 7fadeffff700 1 --2- 172.21.15.55:0/3471894984 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf00b1e50 0x7fadf0159780 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.512+0000 7fadeffff700 1 -- 172.21.15.55:0/3471894984 >> 172.21.15.55:0/3471894984 conn(0x7fadf00d1130 msgr2=0x7fadf00b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:27.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.512+0000 7fadeffff700 1 -- 172.21.15.55:0/3471894984 shutdown_connections 2023-10-06T05:05:27.515 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.512+0000 7fadeffff700 1 -- 172.21.15.55:0/3471894984 wait complete. 2023-10-06T05:05:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.513+0000 7fadeffff700 1 Processor -- start 2023-10-06T05:05:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.513+0000 7fadeffff700 1 -- start start 2023-10-06T05:05:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.514+0000 7fadeffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadf00b1e50 0x7fadf015b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.514+0000 7fadeffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf00ceaa0 0x7fadf00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.514+0000 7fadeffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadf01593b0 0x7fadf00caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.514+0000 7fadeffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf0147da0 con 0x7fadf01593b0 2023-10-06T05:05:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.514+0000 7fadeffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf015ba60 con 0x7fadf00ceaa0 2023-10-06T05:05:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.514+0000 7fadeffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf015bd40 con 0x7fadf00b1e50 2023-10-06T05:05:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.514+0000 7fadef7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadf01593b0 0x7fadf00caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:27.518 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.514+0000 7fadef7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadf01593b0 0x7fadf00caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58200/0 (socket says 172.21.15.55:58200) 2023-10-06T05:05:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.514+0000 7fadef7fe700 1 -- 172.21.15.55:0/2030179217 learned_addr learned my addr 172.21.15.55:0/2030179217 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.514+0000 7fadee7fc700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf00ceaa0 0x7fadf00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.514+0000 7fadeeffd700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadf00b1e50 0x7fadf015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.515+0000 7fadef7fe700 1 -- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadf00b1e50 msgr2=0x7fadf015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.515+0000 7fadef7fe700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadf00b1e50 0x7fadf015b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.515+0000 7fadef7fe700 1 -- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf00ceaa0 msgr2=0x7fadf00ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.515+0000 7fadef7fe700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf00ceaa0 0x7fadf00ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.515+0000 7fadef7fe700 1 -- 172.21.15.55:0/2030179217 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4009420 con 0x7fadf01593b0 2023-10-06T05:05:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.515+0000 7fadee7fc700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf00ceaa0 0x7fadf00ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.515+0000 7fadef7fe700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadf01593b0 0x7fadf00caa30 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fade800cc60 tx=0x7fade8001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.515+0000 7fadd6ffd700 1 -- 172.21.15.55:0/2030179217 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade800a960 con 0x7fadf01593b0 2023-10-06T05:05:27.519 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.515+0000 7fadd6ffd700 1 -- 172.21.15.55:0/2030179217 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade800aae0 con 0x7fadf01593b0 2023-10-06T05:05:27.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.515+0000 7fadd6ffd700 1 -- 172.21.15.55:0/2030179217 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade800aea0 con 0x7fadf01593b0 2023-10-06T05:05:27.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.515+0000 7fadeffff700 1 -- 172.21.15.55:0/2030179217 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf015c020 con 0x7fadf01593b0 2023-10-06T05:05:27.520 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.515+0000 7fadeffff700 1 -- 172.21.15.55:0/2030179217 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf015b670 con 0x7fadf01593b0 2023-10-06T05:05:27.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.516+0000 7fadeffff700 1 -- 172.21.15.55:0/2030179217 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf004c3c0 con 0x7fadf01593b0 2023-10-06T05:05:27.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.517+0000 7fadd6ffd700 1 -- 172.21.15.55:0/2030179217 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 88) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fade8007300 con 0x7fadf01593b0 2023-10-06T05:05:27.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.518+0000 7fadd6ffd700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faddc067050 0x7faddc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:27.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.518+0000 7fadd6ffd700 1 -- 172.21.15.55:0/2030179217 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 7125+0+0 (secure 0 0 0) 0x7fade8085100 con 0x7fadf01593b0 2023-10-06T05:05:27.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.519+0000 7fadeeffd700 1 -- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faddc067050 msgr2=0x7faddc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:27.521 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.519+0000 7fadeeffd700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faddc067050 0x7faddc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:27.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.522+0000 7fadd6ffd700 1 -- 172.21.15.55:0/2030179217 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fade8055740 con 0x7fadf01593b0 2023-10-06T05:05:27.561 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:27.559+0000 7fdef7987180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:05:27.705 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:27.704+0000 7f00f806e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:05:27.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.720+0000 7fadeeffd700 1 -- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faddc067050 msgr2=0x7faddc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:27.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.720+0000 7fadeeffd700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faddc067050 0x7faddc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:27.724 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:27.722+0000 7fdef7987180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:05:27.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.772+0000 7fadeffff700 1 -- 172.21.15.55:0/2030179217 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fadf0147ee0 con 0x7fadf01593b0 2023-10-06T05:05:27.774 INFO:tasks.ceph.mon.a.smithi055.stderr:2023-10-06T05:05:27.772+0000 7fe524ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:05:27.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.778+0000 7fadd6ffd700 1 -- 172.21.15.55:0/2030179217 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) v1 ==== 161+0+0 (secure 0 0 0) 0x7fade80552d0 con 0x7fadf01593b0 2023-10-06T05:05:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.781+0000 7fadeffff700 1 -- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faddc067050 msgr2=0x7faddc069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.781+0000 7fadeffff700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faddc067050 0x7faddc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.781+0000 7fadeffff700 1 -- 172.21.15.55:0/2030179217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadf01593b0 msgr2=0x7fadf00caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.781+0000 7fadeffff700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadf01593b0 0x7fadf00caa30 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fade800cc60 tx=0x7fade8001ae0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.781+0000 7fadeffff700 1 -- 172.21.15.55:0/2030179217 shutdown_connections 2023-10-06T05:05:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.781+0000 7fadeffff700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faddc067050 0x7faddc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.783 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.781+0000 7fadeffff700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadf01593b0 0x7fadf00caa30 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.781+0000 7fadeffff700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadf00b1e50 0x7fadf015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.781+0000 7fadeffff700 1 --2- 172.21.15.55:0/2030179217 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadf00ceaa0 0x7fadf00ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.781+0000 7fadeffff700 1 -- 172.21.15.55:0/2030179217 >> 172.21.15.55:0/2030179217 conn(0x7fadf00d1130 msgr2=0x7fadf00b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.781+0000 7fadeffff700 1 -- 172.21.15.55:0/2030179217 shutdown_connections 2023-10-06T05:05:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.781+0000 7fadeffff700 1 -- 172.21.15.55:0/2030179217 wait complete. 2023-10-06T05:05:27.784 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:05:27.801 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:05:27.803 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:27.802+0000 7f00f806e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:05:27.842 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:27.840+0000 7fdef7987180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:05:27.842 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T05:05:27.900 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:27.899+0000 7f00f806e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:05:27.994 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:27.992+0000 7f00f806e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:05:27.996 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.995+0000 7fa4a8b9d700 1 Processor -- start 2023-10-06T05:05:27.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.995+0000 7fa4a8b9d700 1 -- start start 2023-10-06T05:05:27.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.996+0000 7fa4a8b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa4a415a3b0 0x7fa4a415a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:27.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.996+0000 7fa4a8b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa4a406e470 0x7fa4a406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:27.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.996+0000 7fa4a8b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a40aea70 con 0x7fa4a40aef00 2023-10-06T05:05:27.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.996+0000 7fa4a8b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a406ee30 con 0x7fa4a406e470 2023-10-06T05:05:27.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.996+0000 7fa4a8b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a415dad0 con 0x7fa4a415a3b0 2023-10-06T05:05:27.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.997+0000 7fa4a2ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa4a40aef00 0x7fa4a415d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52564/0 (socket says 172.21.15.55:52564) 2023-10-06T05:05:27.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.997+0000 7fa4a37fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa4a415a3b0 0x7fa4a415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.997+0000 7fa4a2ffd700 1 -- 172.21.15.55:0/4075433343 learned_addr learned my addr 172.21.15.55:0/4075433343 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.997+0000 7fa4a3fff700 1 --2- 172.21.15.55:0/4075433343 >> v2:172.21.15.55:3300/0 conn(0x7fa4a406e470 0x7fa4a406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.997+0000 7fa4a27fc700 1 -- 172.21.15.55:0/4075433343 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151339001 0 0) 0x7fa4a40aea70 con 0x7fa4a40aef00 2023-10-06T05:05:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.997+0000 7fa4a27fc700 1 -- 172.21.15.55:0/4075433343 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa48c003570 con 0x7fa4a40aef00 2023-10-06T05:05:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.997+0000 7fa4a37fe700 1 -- 172.21.15.55:0/4075433343 >> v2:172.21.15.55:3300/0 conn(0x7fa4a406e470 msgr2=0x7fa4a406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.997+0000 7fa4a37fe700 1 --2- 172.21.15.55:0/4075433343 >> v2:172.21.15.55:3300/0 conn(0x7fa4a406e470 0x7fa4a406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.997+0000 7fa4a37fe700 1 -- 172.21.15.55:0/4075433343 >> v1:172.21.15.55:6789/0 conn(0x7fa4a40aef00 legacy=0x7fa4a415d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.997+0000 7fa4a37fe700 1 -- 172.21.15.55:0/4075433343 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a40ae480 con 0x7fa4a415a3b0 2023-10-06T05:05:28.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.997+0000 7fa4a3fff700 1 --2- 172.21.15.55:0/4075433343 >> v2:172.21.15.55:3300/0 conn(0x7fa4a406e470 0x7fa4a406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:28.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.998+0000 7fa4a37fe700 1 --2- 172.21.15.55:0/4075433343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a415a3b0 0x7fa4a415a780 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4063350 tx=0x7fa4940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60bea7f707119de1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:28.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.998+0000 7fa4a27fc700 1 -- 172.21.15.55:0/4075433343 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494011630 con 0x7fa4a415a3b0 2023-10-06T05:05:28.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.998+0000 7fa4a27fc700 1 -- 172.21.15.55:0/4075433343 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4940117b0 con 0x7fa4a415a3b0 2023-10-06T05:05:28.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.998+0000 7fa4a27fc700 1 -- 172.21.15.55:0/4075433343 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa494011b70 con 0x7fa4a415a3b0 2023-10-06T05:05:28.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.999+0000 7fa4a8b9d700 1 -- 172.21.15.55:0/4075433343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a415a3b0 msgr2=0x7fa4a415a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:28.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.999+0000 7fa4a8b9d700 1 --2- 172.21.15.55:0/4075433343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a415a3b0 0x7fa4a415a780 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4063350 tx=0x7fa4940091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:28.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.999+0000 7fa4a8b9d700 1 -- 172.21.15.55:0/4075433343 shutdown_connections 2023-10-06T05:05:28.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.999+0000 7fa4a8b9d700 1 --2- 172.21.15.55:0/4075433343 >> v2:172.21.15.55:3300/0 conn(0x7fa4a406e470 0x7fa4a406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:28.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.999+0000 7fa4a8b9d700 1 --2- 172.21.15.55:0/4075433343 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a415a3b0 0x7fa4a415a780 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:28.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.999+0000 7fa4a8b9d700 1 -- 172.21.15.55:0/4075433343 >> 172.21.15.55:0/4075433343 conn(0x7fa4a40c9f70 msgr2=0x7fa4a406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:28.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.999+0000 7fa4a8b9d700 1 -- 172.21.15.55:0/4075433343 shutdown_connections 2023-10-06T05:05:28.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:27.999+0000 7fa4a8b9d700 1 -- 172.21.15.55:0/4075433343 wait complete. 2023-10-06T05:05:28.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.001+0000 7fa4a8b9d700 1 Processor -- start 2023-10-06T05:05:28.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.003+0000 7fa4a8b9d700 1 -- start start 2023-10-06T05:05:28.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.004+0000 7fa4a8b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a406e470 0x7fa4a416b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:28.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.004+0000 7fa4a8b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a40aef00 0x7fa4a40b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:28.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.004+0000 7fa4a8b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4a415a3b0 0x7fa4a41512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:28.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.004+0000 7fa4a8b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40c52f0 con 0x7fa4a40aef00 2023-10-06T05:05:28.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.004+0000 7fa4a8b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40c5590 con 0x7fa4a406e470 2023-10-06T05:05:28.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.004+0000 7fa4a8b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a40c5870 con 0x7fa4a415a3b0 2023-10-06T05:05:28.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.004+0000 7fa4a2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a40aef00 0x7fa4a40b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:28.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.004+0000 7fa4a2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a40aef00 0x7fa4a40b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58220/0 (socket says 172.21.15.55:58220) 2023-10-06T05:05:28.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.004+0000 7fa4a2ffd700 1 -- 172.21.15.55:0/1737850647 learned_addr learned my addr 172.21.15.55:0/1737850647 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:28.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.004+0000 7fa4a3fff700 1 --2- 172.21.15.55:0/1737850647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4a415a3b0 0x7fa4a41512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.005+0000 7fa4a2ffd700 1 -- 172.21.15.55:0/1737850647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4a415a3b0 msgr2=0x7fa4a41512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.005+0000 7fa4a2ffd700 1 --2- 172.21.15.55:0/1737850647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4a415a3b0 0x7fa4a41512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.005+0000 7fa4a37fe700 1 --2- 172.21.15.55:0/1737850647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a406e470 0x7fa4a416b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.005+0000 7fa4a2ffd700 1 -- 172.21.15.55:0/1737850647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a406e470 msgr2=0x7fa4a416b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:28.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.005+0000 7fa4a2ffd700 1 --2- 172.21.15.55:0/1737850647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a406e470 0x7fa4a416b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.005+0000 7fa4a2ffd700 1 -- 172.21.15.55:0/1737850647 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c009a20 con 0x7fa4a40aef00 2023-10-06T05:05:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.005+0000 7fa4a37fe700 1 --2- 172.21.15.55:0/1737850647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a406e470 0x7fa4a416b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.005+0000 7fa4a2ffd700 1 --2- 172.21.15.55:0/1737850647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a40aef00 0x7fa4a40b1c10 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fa49800c7c0 tx=0x7fa498007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.005+0000 7fa48b7fe700 1 -- 172.21.15.55:0/1737850647 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498005b30 con 0x7fa4a40aef00 2023-10-06T05:05:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.005+0000 7fa48b7fe700 1 -- 172.21.15.55:0/1737850647 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa498005c90 con 0x7fa4a40aef00 2023-10-06T05:05:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.005+0000 7fa4a8b9d700 1 -- 172.21.15.55:0/1737850647 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa494009420 con 0x7fa4a40aef00 2023-10-06T05:05:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.005+0000 7fa48b7fe700 1 -- 172.21.15.55:0/1737850647 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa498007340 con 0x7fa4a40aef00 2023-10-06T05:05:28.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.006+0000 7fa4a8b9d700 1 -- 172.21.15.55:0/1737850647 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a4155cd0 con 0x7fa4a40aef00 2023-10-06T05:05:28.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.007+0000 7fa4a8b9d700 1 -- 172.21.15.55:0/1737850647 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a406cfa0 con 0x7fa4a40aef00 2023-10-06T05:05:28.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.008+0000 7fa48b7fe700 1 -- 172.21.15.55:0/1737850647 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 89) v1 ==== 84451+0+0 (secure 0 0 0) 0x7fa49801b400 con 0x7fa4a40aef00 2023-10-06T05:05:28.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.009+0000 7fa48b7fe700 1 -- 172.21.15.55:0/1737850647 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 7511+0+0 (secure 0 0 0) 0x7fa4980842e0 con 0x7fa4a40aef00 2023-10-06T05:05:28.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.014+0000 7fa48b7fe700 1 -- 172.21.15.55:0/1737850647 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa498051250 con 0x7fa4a40aef00 2023-10-06T05:05:28.054 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:28.052+0000 7fdef7987180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:05:28.088 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:28.086+0000 7f00773cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:28.088 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:28.086+0000 7f00773cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:28.088 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:28.086+0000 7f00773cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:28.088 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:28.086+0000 7f00773cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:28.088 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:28.086+0000 7f00773cb700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:28.088 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:28.086+0000 7f007037d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:28.088 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:28.086+0000 7f007037d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:28.088 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:28.086+0000 7f007037d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:28.088 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:28.086+0000 7f007037d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:28.089 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:28.086+0000 7f007037d700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:28.157 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:28.155+0000 7fdef7987180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:05:28.254 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:28.252+0000 7fdef7987180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:05:28.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.265+0000 7fa4a8b9d700 1 -- 172.21.15.55:0/1737850647 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa4a40c91e0 con 0x7fa4a40aef00 2023-10-06T05:05:28.352 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:28.350+0000 7fdef7987180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:05:28.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.833+0000 7fa48b7fe700 1 -- 172.21.15.55:0/1737850647 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v89) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa498054870 con 0x7fa4a40aef00 2023-10-06T05:05:28.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.836+0000 7fa4a8b9d700 1 -- 172.21.15.55:0/1737850647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a40aef00 msgr2=0x7fa4a40b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:28.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.836+0000 7fa4a8b9d700 1 --2- 172.21.15.55:0/1737850647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a40aef00 0x7fa4a40b1c10 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fa49800c7c0 tx=0x7fa498007e00 comp rx=0 tx=0).stop 2023-10-06T05:05:28.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.836+0000 7fa4a8b9d700 1 -- 172.21.15.55:0/1737850647 shutdown_connections 2023-10-06T05:05:28.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.836+0000 7fa4a8b9d700 1 --2- 172.21.15.55:0/1737850647 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa4a415a3b0 0x7fa4a41512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:28.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.836+0000 7fa4a8b9d700 1 --2- 172.21.15.55:0/1737850647 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa4a406e470 0x7fa4a416b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:28.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.836+0000 7fa4a8b9d700 1 --2- 172.21.15.55:0/1737850647 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa4a40aef00 0x7fa4a40b1c10 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:28.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.836+0000 7fa4a8b9d700 1 -- 172.21.15.55:0/1737850647 >> 172.21.15.55:0/1737850647 conn(0x7fa4a40c9f70 msgr2=0x7fa4a4152620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:28.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.836+0000 7fa4a8b9d700 1 -- 172.21.15.55:0/1737850647 shutdown_connections 2023-10-06T05:05:28.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:28.836+0000 7fa4a8b9d700 1 -- 172.21.15.55:0/1737850647 wait complete. 2023-10-06T05:05:28.857 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:05:29.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.096+0000 7fd67759e700 1 Processor -- start 2023-10-06T05:05:29.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.096+0000 7fd67759e700 1 -- start start 2023-10-06T05:05:29.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.097+0000 7fd67759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6780b0760 0x7fd6780b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:29.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.097+0000 7fd67759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd6780ace60 0x7fd6780ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:29.099 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.097+0000 7fd67759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6780b1110 con 0x7fd678156350 2023-10-06T05:05:29.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.097+0000 7fd67759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6780b0570 con 0x7fd6780ace60 2023-10-06T05:05:29.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.097+0000 7fd67759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780af4d0 con 0x7fd6780b0760 2023-10-06T05:05:29.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.098+0000 7fd676d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd6780ace60 0x7fd6780ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:29.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.098+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6780b0760 0x7fd6780b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:29.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.098+0000 7fd67659c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fd678156350 0x7fd678157370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:51390/0 (socket says 172.21.15.55:51390) 2023-10-06T05:05:29.100 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.098+0000 7fd67659c700 1 -- 172.21.15.55:0/2809760179 learned_addr learned my addr 172.21.15.55:0/2809760179 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:29.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.098+0000 7fd67559a700 1 -- 172.21.15.55:0/2809760179 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179547211 0 0) 0x7fd6780b1110 con 0x7fd678156350 2023-10-06T05:05:29.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.098+0000 7fd67559a700 1 -- 172.21.15.55:0/2809760179 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd660003570 con 0x7fd678156350 2023-10-06T05:05:29.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.098+0000 7fd676d9d700 1 -- 172.21.15.55:0/2809760179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6780b0760 msgr2=0x7fd6780b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:29.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.098+0000 7fd676d9d700 1 --2- 172.21.15.55:0/2809760179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6780b0760 0x7fd6780b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:29.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.098+0000 7fd676d9d700 1 -- 172.21.15.55:0/2809760179 >> v1:172.21.15.163:6789/0 conn(0x7fd678156350 legacy=0x7fd678157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:29.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.098+0000 7fd676d9d700 1 -- 172.21.15.55:0/2809760179 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6780af7b0 con 0x7fd6780ace60 2023-10-06T05:05:29.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.098+0000 7fd675d9b700 1 --2- 172.21.15.55:0/2809760179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6780b0760 0x7fd6780b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:29.102 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.099+0000 7fd676d9d700 1 --2- 172.21.15.55:0/2809760179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6780ace60 0x7fd6780ad2a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd670009e60 tx=0x7fd670009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a527c5591787056 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:29.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.099+0000 7fd67559a700 1 -- 172.21.15.55:0/2809760179 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67000e070 con 0x7fd6780ace60 2023-10-06T05:05:29.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.099+0000 7fd67559a700 1 -- 172.21.15.55:0/2809760179 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67000bab0 con 0x7fd6780ace60 2023-10-06T05:05:29.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.099+0000 7fd67559a700 1 -- 172.21.15.55:0/2809760179 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67000be50 con 0x7fd6780ace60 2023-10-06T05:05:29.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.099+0000 7fd67759e700 1 -- 172.21.15.55:0/2809760179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6780ace60 msgr2=0x7fd6780ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:29.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.099+0000 7fd67759e700 1 --2- 172.21.15.55:0/2809760179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6780ace60 0x7fd6780ad2a0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fd670009e60 tx=0x7fd670009780 comp rx=0 tx=0).stop 2023-10-06T05:05:29.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.100+0000 7fd67759e700 1 -- 172.21.15.55:0/2809760179 shutdown_connections 2023-10-06T05:05:29.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.100+0000 7fd67759e700 1 --2- 172.21.15.55:0/2809760179 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6780ace60 0x7fd6780ad2a0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:29.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.100+0000 7fd67759e700 1 --2- 172.21.15.55:0/2809760179 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6780b0760 0x7fd6780b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:29.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.100+0000 7fd67759e700 1 -- 172.21.15.55:0/2809760179 >> 172.21.15.55:0/2809760179 conn(0x7fd678068e20 msgr2=0x7fd6780cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:29.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.100+0000 7fd67759e700 1 -- 172.21.15.55:0/2809760179 shutdown_connections 2023-10-06T05:05:29.103 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.100+0000 7fd67759e700 1 -- 172.21.15.55:0/2809760179 wait complete. 2023-10-06T05:05:29.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.101+0000 7fd67759e700 1 Processor -- start 2023-10-06T05:05:29.105 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.101+0000 7fd67759e700 1 -- start start 2023-10-06T05:05:29.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.102+0000 7fd67759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6780ace60 0x7fd67815be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:29.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.102+0000 7fd67759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6780b0760 0x7fd67816a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:29.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.102+0000 7fd67759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd678156350 0x7fd678151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:29.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.102+0000 7fd67759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780cb160 con 0x7fd678156350 2023-10-06T05:05:29.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.102+0000 7fd67759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67816a500 con 0x7fd6780ace60 2023-10-06T05:05:29.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.102+0000 7fd67759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6780cab10 con 0x7fd6780b0760 2023-10-06T05:05:29.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.103+0000 7fd676d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd678156350 0x7fd678151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:29.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.103+0000 7fd67659c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6780ace60 0x7fd67815be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:29.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.103+0000 7fd676d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd678156350 0x7fd678151f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58228/0 (socket says 172.21.15.55:58228) 2023-10-06T05:05:29.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.103+0000 7fd676d9d700 1 -- 172.21.15.55:0/1346861858 learned_addr learned my addr 172.21.15.55:0/1346861858 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:29.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.103+0000 7fd675d9b700 1 --2- 172.21.15.55:0/1346861858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6780b0760 0x7fd67816a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:29.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.103+0000 7fd676d9d700 1 -- 172.21.15.55:0/1346861858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6780b0760 msgr2=0x7fd67816a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:29.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.103+0000 7fd676d9d700 1 --2- 172.21.15.55:0/1346861858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6780b0760 0x7fd67816a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:29.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.103+0000 7fd676d9d700 1 -- 172.21.15.55:0/1346861858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6780ace60 msgr2=0x7fd67815be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:29.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.103+0000 7fd676d9d700 1 --2- 172.21.15.55:0/1346861858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6780ace60 0x7fd67815be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:29.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.103+0000 7fd676d9d700 1 -- 172.21.15.55:0/1346861858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66c009420 con 0x7fd678156350 2023-10-06T05:05:29.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.103+0000 7fd67659c700 1 --2- 172.21.15.55:0/1346861858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6780ace60 0x7fd67815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:29.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.103+0000 7fd676d9d700 1 --2- 172.21.15.55:0/1346861858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd678156350 0x7fd678151f20 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fd6780714a0 tx=0x7fd670019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:29.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.104+0000 7fd6667fc700 1 -- 172.21.15.55:0/1346861858 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd670012e10 con 0x7fd678156350 2023-10-06T05:05:29.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.104+0000 7fd6667fc700 1 -- 172.21.15.55:0/1346861858 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd670027470 con 0x7fd678156350 2023-10-06T05:05:29.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.104+0000 7fd6667fc700 1 -- 172.21.15.55:0/1346861858 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd670027830 con 0x7fd678156350 2023-10-06T05:05:29.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.104+0000 7fd67759e700 1 -- 172.21.15.55:0/1346861858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd670009a20 con 0x7fd678156350 2023-10-06T05:05:29.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.104+0000 7fd67759e700 1 -- 172.21.15.55:0/1346861858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6780ca7c0 con 0x7fd678156350 2023-10-06T05:05:29.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.105+0000 7fd67759e700 1 -- 172.21.15.55:0/1346861858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6780cb2d0 con 0x7fd678156350 2023-10-06T05:05:29.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.106+0000 7fd6667fc700 1 -- 172.21.15.55:0/1346861858 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 90) v1 ==== 84578+0+0 (secure 0 0 0) 0x7fd670024070 con 0x7fd678156350 2023-10-06T05:05:29.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.107+0000 7fd6667fc700 1 --2- 172.21.15.55:0/1346861858 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fd658066d80 0x7fd658069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:29.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.107+0000 7fd6667fc700 1 -- 172.21.15.55:0/1346861858 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 7511+0+0 (secure 0 0 0) 0x7fd670086e80 con 0x7fd678156350 2023-10-06T05:05:29.109 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.107+0000 7fd67659c700 1 --2- 172.21.15.55:0/1346861858 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fd658066d80 0x7fd658069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:29.110 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.108+0000 7fd67659c700 1 --2- 172.21.15.55:0/1346861858 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fd658066d80 0x7fd658069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd668007fd0 tx=0x7fd668009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:29.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.111+0000 7fd6667fc700 1 -- 172.21.15.55:0/1346861858 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd670053c90 con 0x7fd678156350 2023-10-06T05:05:29.314 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:29.312+0000 7fd67759e700 1 -- 172.21.15.55:0/1346861858 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd67804c3c0 con 0x7fd678156350 2023-10-06T05:05:30.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.014+0000 7fd6667fc700 1 -- 172.21.15.55:0/1346861858 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v184) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd67002c0d0 con 0x7fd678156350 2023-10-06T05:05:30.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.018+0000 7fd67759e700 1 -- 172.21.15.55:0/1346861858 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fd658066d80 msgr2=0x7fd658069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:30.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.018+0000 7fd67759e700 1 --2- 172.21.15.55:0/1346861858 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fd658066d80 0x7fd658069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd668007fd0 tx=0x7fd668009040 comp rx=0 tx=0).stop 2023-10-06T05:05:30.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.018+0000 7fd67759e700 1 -- 172.21.15.55:0/1346861858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd678156350 msgr2=0x7fd678151f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:30.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.018+0000 7fd67759e700 1 --2- 172.21.15.55:0/1346861858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd678156350 0x7fd678151f20 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fd6780714a0 tx=0x7fd670019000 comp rx=0 tx=0).stop 2023-10-06T05:05:30.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.019+0000 7fd67759e700 1 -- 172.21.15.55:0/1346861858 shutdown_connections 2023-10-06T05:05:30.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.019+0000 7fd67759e700 1 --2- 172.21.15.55:0/1346861858 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fd658066d80 0x7fd658069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:30.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.019+0000 7fd67759e700 1 --2- 172.21.15.55:0/1346861858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd678156350 0x7fd678151f20 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:30.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.019+0000 7fd67759e700 1 --2- 172.21.15.55:0/1346861858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd6780ace60 0x7fd67815be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:30.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.019+0000 7fd67759e700 1 --2- 172.21.15.55:0/1346861858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd6780b0760 0x7fd67816a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:30.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.019+0000 7fd67759e700 1 -- 172.21.15.55:0/1346861858 >> 172.21.15.55:0/1346861858 conn(0x7fd678068e20 msgr2=0x7fd678152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:30.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.019+0000 7fd67759e700 1 -- 172.21.15.55:0/1346861858 shutdown_connections 2023-10-06T05:05:30.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.019+0000 7fd67759e700 1 -- 172.21.15.55:0/1346861858 wait complete. 2023-10-06T05:05:30.022 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:05:30.042 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:05:30.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.238+0000 7fc13ad9d700 1 Processor -- start 2023-10-06T05:05:30.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.238+0000 7fc13ad9d700 1 -- start start 2023-10-06T05:05:30.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.239+0000 7fc13ad9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc1340685e0 0x7fc13415a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:30.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.239+0000 7fc13ad9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc13415d3f0 0x7fc1341538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:30.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.239+0000 7fc13ad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc1340689b0 0x7fc134153e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.239+0000 7fc13ad9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1340c9480 con 0x7fc1340689b0 2023-10-06T05:05:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.239+0000 7fc13ad9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc13415d880 con 0x7fc1340685e0 2023-10-06T05:05:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.239+0000 7fc13ad9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13415db60 con 0x7fc13415d3f0 2023-10-06T05:05:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.240+0000 7fc13a59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc1340689b0 0x7fc134153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.240+0000 7fc13a59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc1340689b0 0x7fc134153e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58236/0 (socket says 172.21.15.55:58236) 2023-10-06T05:05:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.240+0000 7fc13a59c700 1 -- 172.21.15.55:0/1714809050 learned_addr learned my addr 172.21.15.55:0/1714809050 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.240+0000 7fc13959a700 1 --2- 172.21.15.55:0/1714809050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc13415d3f0 0x7fc1341538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.240+0000 7fc139d9b700 1 --2- 172.21.15.55:0/1714809050 >> v2:172.21.15.163:3300/0 conn(0x7fc1340685e0 0x7fc13415a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.240+0000 7fc13a59c700 1 -- 172.21.15.55:0/1714809050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc13415d3f0 msgr2=0x7fc1341538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.240+0000 7fc13a59c700 1 --2- 172.21.15.55:0/1714809050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc13415d3f0 0x7fc1341538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.240+0000 7fc13a59c700 1 -- 172.21.15.55:0/1714809050 >> v2:172.21.15.163:3300/0 conn(0x7fc1340685e0 msgr2=0x7fc13415a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:30.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.240+0000 7fc13a59c700 1 --2- 172.21.15.55:0/1714809050 >> v2:172.21.15.163:3300/0 conn(0x7fc1340685e0 0x7fc13415a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:30.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.240+0000 7fc13a59c700 1 -- 172.21.15.55:0/1714809050 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13415de40 con 0x7fc1340689b0 2023-10-06T05:05:30.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.240+0000 7fc13a59c700 1 --2- 172.21.15.55:0/1714809050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1340689b0 0x7fc134153e30 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc134065920 tx=0x7fc12c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed156d577f29271e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:30.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.241+0000 7fc138d99700 1 -- 172.21.15.55:0/1714809050 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c00e070 con 0x7fc1340689b0 2023-10-06T05:05:30.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.241+0000 7fc138d99700 1 -- 172.21.15.55:0/1714809050 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12c00ba00 con 0x7fc1340689b0 2023-10-06T05:05:30.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.241+0000 7fc138d99700 1 -- 172.21.15.55:0/1714809050 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c0092e0 con 0x7fc1340689b0 2023-10-06T05:05:30.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.241+0000 7fc13ad9d700 1 -- 172.21.15.55:0/1714809050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1340689b0 msgr2=0x7fc134153e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:30.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.241+0000 7fc13ad9d700 1 --2- 172.21.15.55:0/1714809050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1340689b0 0x7fc134153e30 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc134065920 tx=0x7fc12c009840 comp rx=0 tx=0).stop 2023-10-06T05:05:30.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.241+0000 7fc13ad9d700 1 -- 172.21.15.55:0/1714809050 shutdown_connections 2023-10-06T05:05:30.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.241+0000 7fc13ad9d700 1 --2- 172.21.15.55:0/1714809050 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1340689b0 0x7fc134153e30 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:30.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.241+0000 7fc13ad9d700 1 --2- 172.21.15.55:0/1714809050 >> v2:172.21.15.163:3300/0 conn(0x7fc1340685e0 0x7fc13415a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:30.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.241+0000 7fc13ad9d700 1 --2- 172.21.15.55:0/1714809050 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc13415d3f0 0x7fc1341538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:30.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.241+0000 7fc13ad9d700 1 -- 172.21.15.55:0/1714809050 >> 172.21.15.55:0/1714809050 conn(0x7fc13406b420 msgr2=0x7fc1340cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:30.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.241+0000 7fc13ad9d700 1 -- 172.21.15.55:0/1714809050 shutdown_connections 2023-10-06T05:05:30.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.241+0000 7fc13ad9d700 1 -- 172.21.15.55:0/1714809050 wait complete. 2023-10-06T05:05:30.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.243+0000 7fc13ad9d700 1 Processor -- start 2023-10-06T05:05:30.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.243+0000 7fc13ad9d700 1 -- start start 2023-10-06T05:05:30.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.244+0000 7fc13ad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1340685e0 0x7fc1341518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:30.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.244+0000 7fc13ad9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1340689b0 0x7fc1340b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:30.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.244+0000 7fc13ad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc13415d3f0 0x7fc1340c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:30.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.244+0000 7fc13ad9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc134151e30 con 0x7fc1340685e0 2023-10-06T05:05:30.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.244+0000 7fc13ad9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1341599e0 con 0x7fc13415d3f0 2023-10-06T05:05:30.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.244+0000 7fc13ad9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc134169a90 con 0x7fc1340689b0 2023-10-06T05:05:30.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.244+0000 7fc139d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1340685e0 0x7fc1341518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:30.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.244+0000 7fc139d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1340685e0 0x7fc1341518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58250/0 (socket says 172.21.15.55:58250) 2023-10-06T05:05:30.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.244+0000 7fc139d9b700 1 -- 172.21.15.55:0/4187984467 learned_addr learned my addr 172.21.15.55:0/4187984467 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:30.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.244+0000 7fc139d9b700 1 -- 172.21.15.55:0/4187984467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1340689b0 msgr2=0x7fc1340b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:30.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.244+0000 7fc139d9b700 1 --2- 172.21.15.55:0/4187984467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1340689b0 0x7fc1340b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:30.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.244+0000 7fc139d9b700 1 -- 172.21.15.55:0/4187984467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc13415d3f0 msgr2=0x7fc1340c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:30.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.244+0000 7fc139d9b700 1 --2- 172.21.15.55:0/4187984467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc13415d3f0 0x7fc1340c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:30.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.244+0000 7fc139d9b700 1 -- 172.21.15.55:0/4187984467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c009ae0 con 0x7fc1340685e0 2023-10-06T05:05:30.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.245+0000 7fc139d9b700 1 --2- 172.21.15.55:0/4187984467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1340685e0 0x7fc1341518f0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fc13000ec80 tx=0x7fc1300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:30.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.245+0000 7fc129ffb700 1 -- 172.21.15.55:0/4187984467 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13000c9d0 con 0x7fc1340685e0 2023-10-06T05:05:30.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.245+0000 7fc129ffb700 1 -- 172.21.15.55:0/4187984467 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13000cb30 con 0x7fc1340685e0 2023-10-06T05:05:30.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.245+0000 7fc129ffb700 1 -- 172.21.15.55:0/4187984467 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130013470 con 0x7fc1340685e0 2023-10-06T05:05:30.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.245+0000 7fc13ad9d700 1 -- 172.21.15.55:0/4187984467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1340b1120 con 0x7fc1340685e0 2023-10-06T05:05:30.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.245+0000 7fc13ad9d700 1 -- 172.21.15.55:0/4187984467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1340b16e0 con 0x7fc1340685e0 2023-10-06T05:05:30.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.246+0000 7fc13ad9d700 1 -- 172.21.15.55:0/4187984467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc13415ce10 con 0x7fc1340685e0 2023-10-06T05:05:30.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.247+0000 7fc129ffb700 1 -- 172.21.15.55:0/4187984467 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 90) v1 ==== 84578+0+0 (secure 0 0 0) 0x7fc130013820 con 0x7fc1340685e0 2023-10-06T05:05:30.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.248+0000 7fc129ffb700 1 --2- 172.21.15.55:0/4187984467 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc118066d80 0x7fc118069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:30.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.248+0000 7fc129ffb700 1 -- 172.21.15.55:0/4187984467 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 7073+0+0 (secure 0 0 0) 0x7fc130084bd0 con 0x7fc1340685e0 2023-10-06T05:05:30.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.248+0000 7fc13959a700 1 --2- 172.21.15.55:0/4187984467 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc118066d80 0x7fc118069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:30.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.249+0000 7fc13959a700 1 --2- 172.21.15.55:0/4187984467 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc118066d80 0x7fc118069230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc124009b00 tx=0x7fc124009120 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:30.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.252+0000 7fc129ffb700 1 -- 172.21.15.55:0/4187984467 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc130051ac0 con 0x7fc1340685e0 2023-10-06T05:05:30.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.454+0000 7fc13ad9d700 1 -- 172.21.15.55:0/4187984467 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc1340c9c60 con 0x7fc1340685e0 2023-10-06T05:05:30.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:30.845+0000 7fc129ffb700 1 -- 172.21.15.55:0/4187984467 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc130051e90 con 0x7fc1340685e0 2023-10-06T05:05:31.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.020+0000 7fc129ffb700 1 -- 172.21.15.55:0/4187984467 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v185) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc130070560 con 0x7fc1340685e0 2023-10-06T05:05:31.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.024+0000 7fc13ad9d700 1 -- 172.21.15.55:0/4187984467 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc118066d80 msgr2=0x7fc118069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.024+0000 7fc13ad9d700 1 --2- 172.21.15.55:0/4187984467 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc118066d80 0x7fc118069230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc124009b00 tx=0x7fc124009120 comp rx=0 tx=0).stop 2023-10-06T05:05:31.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.024+0000 7fc13ad9d700 1 -- 172.21.15.55:0/4187984467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1340685e0 msgr2=0x7fc1341518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.024+0000 7fc13ad9d700 1 --2- 172.21.15.55:0/4187984467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1340685e0 0x7fc1341518f0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fc13000ec80 tx=0x7fc1300091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.025+0000 7fc13ad9d700 1 -- 172.21.15.55:0/4187984467 shutdown_connections 2023-10-06T05:05:31.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.025+0000 7fc13ad9d700 1 --2- 172.21.15.55:0/4187984467 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc118066d80 0x7fc118069230 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.025+0000 7fc13ad9d700 1 --2- 172.21.15.55:0/4187984467 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc13415d3f0 0x7fc1340c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.025+0000 7fc13ad9d700 1 --2- 172.21.15.55:0/4187984467 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc1340685e0 0x7fc1341518f0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.025+0000 7fc13ad9d700 1 --2- 172.21.15.55:0/4187984467 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc1340689b0 0x7fc1340b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.025+0000 7fc13ad9d700 1 -- 172.21.15.55:0/4187984467 >> 172.21.15.55:0/4187984467 conn(0x7fc13406b420 msgr2=0x7fc13415c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:31.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.025+0000 7fc13ad9d700 1 -- 172.21.15.55:0/4187984467 shutdown_connections 2023-10-06T05:05:31.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.025+0000 7fc13ad9d700 1 -- 172.21.15.55:0/4187984467 wait complete. 2023-10-06T05:05:31.029 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2023-10-06T05:05:31.050 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:05:31.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.248+0000 7f18aef9d700 1 Processor -- start 2023-10-06T05:05:31.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.249+0000 7f18aef9d700 1 -- start start 2023-10-06T05:05:31.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.250+0000 7f18aef9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f18a806ec60 0x7f18a806f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:31.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.250+0000 7f18aef9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a80ca780 con 0x7f18a815de10 2023-10-06T05:05:31.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.250+0000 7f18aef9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a806f5e0 con 0x7f18a80c72c0 2023-10-06T05:05:31.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.250+0000 7f18aef9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a80c7760 con 0x7f18a806ec60 2023-10-06T05:05:31.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.250+0000 7f18adf9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f18a815de10 0x7f18a806df40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52566/0 (socket says 172.21.15.55:52566) 2023-10-06T05:05:31.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.250+0000 7f18adf9b700 1 -- 172.21.15.55:0/3179629208 learned_addr learned my addr 172.21.15.55:0/3179629208 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:31.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18ae79c700 1 --2- 172.21.15.55:0/3179629208 >> v2:172.21.15.163:3300/0 conn(0x7f18a806ec60 0x7f18a806f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:31.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 -- 172.21.15.55:0/3179629208 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2558011121 0 0) 0x7f18a80ca780 con 0x7f18a815de10 2023-10-06T05:05:31.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 -- 172.21.15.55:0/3179629208 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1894003570 con 0x7f18a815de10 2023-10-06T05:05:31.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 -- 172.21.15.55:0/3179629208 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1727486581 0 0) 0x7f1894003570 con 0x7f18a815de10 2023-10-06T05:05:31.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 -- 172.21.15.55:0/3179629208 >> v2:172.21.15.163:3300/0 conn(0x7f18a806ec60 msgr2=0x7f18a806f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:31.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 --2- 172.21.15.55:0/3179629208 >> v2:172.21.15.163:3300/0 conn(0x7f18a806ec60 0x7f18a806f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 -- 172.21.15.55:0/3179629208 >> v1:172.21.15.163:6789/0 conn(0x7f18a80c72c0 legacy=0x7f18a8069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 -- 172.21.15.55:0/3179629208 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a806e7c0 con 0x7f18a815de10 2023-10-06T05:05:31.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 -- 172.21.15.55:0/3179629208 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f18a4001470 con 0x7f18a815de10 2023-10-06T05:05:31.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 -- 172.21.15.55:0/3179629208 >> v1:172.21.15.55:6789/0 conn(0x7f18a815de10 legacy=0x7f18a806df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 --2- 172.21.15.55:0/3179629208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1894003a60 0x7f1894003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 --2- 172.21.15.55:0/3179629208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1894004810 0x7f1894004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 --2- 172.21.15.55:0/3179629208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18940051c0 0x7f189400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 -- 172.21.15.55:0/3179629208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80c7760 con 0x7f1894003a60 2023-10-06T05:05:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 -- 172.21.15.55:0/3179629208 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189400c0e0 con 0x7f18940051c0 2023-10-06T05:05:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18acf99700 1 -- 172.21.15.55:0/3179629208 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189400c2a0 con 0x7f1894004810 2023-10-06T05:05:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.251+0000 7f18adf9b700 1 --2- 172.21.15.55:0/3179629208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1894003a60 0x7f1894003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.252+0000 7f18ad79a700 1 --2- 172.21.15.55:0/3179629208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1894004810 0x7f1894004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.252+0000 7f18ae79c700 1 --2- 172.21.15.55:0/3179629208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18940051c0 0x7f189400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.252+0000 7f18adf9b700 1 -- 172.21.15.55:0/3179629208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1894004810 msgr2=0x7f1894004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.252+0000 7f18adf9b700 1 --2- 172.21.15.55:0/3179629208 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1894004810 0x7f1894004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.252+0000 7f18adf9b700 1 -- 172.21.15.55:0/3179629208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18940051c0 msgr2=0x7f189400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:31.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.252+0000 7f18adf9b700 1 --2- 172.21.15.55:0/3179629208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18940051c0 0x7f189400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.252+0000 7f18adf9b700 1 -- 172.21.15.55:0/3179629208 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1894003570 con 0x7f1894003a60 2023-10-06T05:05:31.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.252+0000 7f18ae79c700 1 --2- 172.21.15.55:0/3179629208 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18940051c0 0x7f189400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:31.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.252+0000 7f18ae79c700 1 -- 172.21.15.55:0/3179629208 reap_dead start 2023-10-06T05:05:31.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.252+0000 7f18adf9b700 1 --2- 172.21.15.55:0/3179629208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1894003a60 0x7f1894003e30 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f18a400c2d0 tx=0x7f18a80c8b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208d271f23d2d0a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:31.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.252+0000 7f18acf99700 1 -- 172.21.15.55:0/3179629208 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a400cc00 con 0x7f1894003a60 2023-10-06T05:05:31.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.252+0000 7f18acf99700 1 -- 172.21.15.55:0/3179629208 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a400cd80 con 0x7f1894003a60 2023-10-06T05:05:31.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.252+0000 7f18acf99700 1 -- 172.21.15.55:0/3179629208 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4005b70 con 0x7f1894003a60 2023-10-06T05:05:31.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.253+0000 7f18aef9d700 1 -- 172.21.15.55:0/3179629208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1894003a60 msgr2=0x7f1894003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.253+0000 7f18aef9d700 1 --2- 172.21.15.55:0/3179629208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1894003a60 0x7f1894003e30 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f18a400c2d0 tx=0x7f18a80c8b20 comp rx=0 tx=0).stop 2023-10-06T05:05:31.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.253+0000 7f18aef9d700 1 -- 172.21.15.55:0/3179629208 shutdown_connections 2023-10-06T05:05:31.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.253+0000 7f18aef9d700 1 --2- 172.21.15.55:0/3179629208 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1894003a60 0x7f1894003e30 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.253+0000 7f18aef9d700 1 -- 172.21.15.55:0/3179629208 >> 172.21.15.55:0/3179629208 conn(0x7f18a80cc1e0 msgr2=0x7f18a80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:31.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.253+0000 7f18aef9d700 1 -- 172.21.15.55:0/3179629208 shutdown_connections 2023-10-06T05:05:31.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.253+0000 7f18aef9d700 1 -- 172.21.15.55:0/3179629208 wait complete. 2023-10-06T05:05:31.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.254+0000 7f18aef9d700 1 Processor -- start 2023-10-06T05:05:31.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.255+0000 7f18aef9d700 1 -- start start 2023-10-06T05:05:31.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.255+0000 7f18aef9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1894003a60 0x7f18a80aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:31.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.255+0000 7f18aef9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18a815c710 0x7f18a814b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:31.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18aef9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18a80c3290 0x7f18a80c3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:31.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18aef9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80b0220 con 0x7f1894003a60 2023-10-06T05:05:31.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18aef9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a8160480 con 0x7f18a815c710 2023-10-06T05:05:31.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18aef9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a814bbd0 con 0x7f18a80c3290 2023-10-06T05:05:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18adf9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1894003a60 0x7f18a80aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18adf9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1894003a60 0x7f18a80aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58264/0 (socket says 172.21.15.55:58264) 2023-10-06T05:05:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18adf9b700 1 -- 172.21.15.55:0/1887779194 learned_addr learned my addr 172.21.15.55:0/1887779194 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18ad79a700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18a815c710 0x7f18a814b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18ae79c700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18a80c3290 0x7f18a80c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18adf9b700 1 -- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18a80c3290 msgr2=0x7f18a80c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18adf9b700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18a80c3290 0x7f18a80c3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18adf9b700 1 -- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18a815c710 msgr2=0x7f18a814b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18adf9b700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18a815c710 0x7f18a814b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.263 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18adf9b700 1 -- 172.21.15.55:0/1887779194 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a80af610 con 0x7f1894003a60 2023-10-06T05:05:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18ae79c700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18a80c3290 0x7f18a80c3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.256+0000 7f18adf9b700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1894003a60 0x7f18a80aef80 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f18a400c8c0 tx=0x7f18a4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.257+0000 7f189dffb700 1 -- 172.21.15.55:0/1887779194 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4022070 con 0x7f1894003a60 2023-10-06T05:05:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.257+0000 7f189dffb700 1 -- 172.21.15.55:0/1887779194 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a400cc00 con 0x7f1894003a60 2023-10-06T05:05:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.257+0000 7f189dffb700 1 -- 172.21.15.55:0/1887779194 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a4007320 con 0x7f1894003a60 2023-10-06T05:05:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.257+0000 7f18aef9d700 1 -- 172.21.15.55:0/1887779194 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a80af8f0 con 0x7f1894003a60 2023-10-06T05:05:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.257+0000 7f18aef9d700 1 -- 172.21.15.55:0/1887779194 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a80afeb0 con 0x7f1894003a60 2023-10-06T05:05:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.258+0000 7f18aef9d700 1 -- 172.21.15.55:0/1887779194 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a806c450 con 0x7f1894003a60 2023-10-06T05:05:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.259+0000 7f189dffb700 1 -- 172.21.15.55:0/1887779194 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18a401e430 con 0x7f1894003a60 2023-10-06T05:05:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.260+0000 7f189dffb700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f188c066f30 0x7f188c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.260+0000 7f189dffb700 1 -- 172.21.15.55:0/1887779194 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6572+0+0 (secure 0 0 0) 0x7f18a4085ab0 con 0x7f1894003a60 2023-10-06T05:05:31.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.260+0000 7f18ad79a700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f188c066f30 0x7f188c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:31.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.261+0000 7f18ad79a700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f188c066f30 0x7f188c0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1898007fd0 tx=0x7f1898009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:31.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.264+0000 7f189dffb700 1 -- 172.21.15.55:0/1887779194 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18a40529d0 con 0x7f1894003a60 2023-10-06T05:05:31.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.470+0000 7f18aef9d700 1 -- 172.21.15.55:0/1887779194 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f18a80b0500 con 0x7f1894003a60 2023-10-06T05:05:31.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.474+0000 7f189dffb700 1 -- 172.21.15.55:0/1887779194 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7f18a4007710 con 0x7f1894003a60 2023-10-06T05:05:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.476+0000 7f18aef9d700 1 -- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f188c066f30 msgr2=0x7f188c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.476+0000 7f18aef9d700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f188c066f30 0x7f188c0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1898007fd0 tx=0x7f1898009040 comp rx=0 tx=0).stop 2023-10-06T05:05:31.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.476+0000 7f18aef9d700 1 -- 172.21.15.55:0/1887779194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1894003a60 msgr2=0x7f18a80aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.476+0000 7f18aef9d700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1894003a60 0x7f18a80aef80 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f18a400c8c0 tx=0x7f18a4015000 comp rx=0 tx=0).stop 2023-10-06T05:05:31.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.476+0000 7f18aef9d700 1 -- 172.21.15.55:0/1887779194 shutdown_connections 2023-10-06T05:05:31.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.476+0000 7f18aef9d700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f188c066f30 0x7f188c0693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.476+0000 7f18aef9d700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18a80c3290 0x7f18a80c3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.476+0000 7f18aef9d700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1894003a60 0x7f18a80aef80 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.477+0000 7f18aef9d700 1 --2- 172.21.15.55:0/1887779194 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18a815c710 0x7f18a814b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.477+0000 7f18aef9d700 1 -- 172.21.15.55:0/1887779194 >> 172.21.15.55:0/1887779194 conn(0x7f18a80cc1e0 msgr2=0x7f18a815e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:31.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.477+0000 7f18aef9d700 1 -- 172.21.15.55:0/1887779194 shutdown_connections 2023-10-06T05:05:31.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.477+0000 7f18aef9d700 1 -- 172.21.15.55:0/1887779194 wait complete. 2023-10-06T05:05:31.496 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T05:05:31.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.683+0000 7f2febfff700 1 Processor -- start 2023-10-06T05:05:31.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.683+0000 7f2febfff700 1 -- start start 2023-10-06T05:05:31.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.684+0000 7f2febfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f2fec159a60 0x7f2fec0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:31.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.684+0000 7f2febfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fec0cb230 0x7f2fec0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:31.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.685+0000 7f2febfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fec0c9cb0 con 0x7f2fec0ccc00 2023-10-06T05:05:31.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.685+0000 7f2febfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fec0cbbd0 con 0x7f2fec159a60 2023-10-06T05:05:31.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.685+0000 7f2febfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec06dfc0 con 0x7f2fec0cb230 2023-10-06T05:05:31.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.685+0000 7f2fea7fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2fec0ccc00 0x7f2fec069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52580/0 (socket says 172.21.15.55:52580) 2023-10-06T05:05:31.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.685+0000 7f2fea7fc700 1 -- 172.21.15.55:0/2776545305 learned_addr learned my addr 172.21.15.55:0/2776545305 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:31.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.685+0000 7f2feaffd700 1 --2- 172.21.15.55:0/2776545305 >> v2:172.21.15.163:3300/0 conn(0x7f2fec159a60 0x7f2fec0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:31.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.685+0000 7f2feb7fe700 1 --2- 172.21.15.55:0/2776545305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fec0cb230 0x7f2fec0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:31.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.685+0000 7f2fe9ffb700 1 -- 172.21.15.55:0/2776545305 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842378657 0 0) 0x7f2fec0c9cb0 con 0x7f2fec0ccc00 2023-10-06T05:05:31.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.685+0000 7f2fe9ffb700 1 -- 172.21.15.55:0/2776545305 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fd4003570 con 0x7f2fec0ccc00 2023-10-06T05:05:31.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.686+0000 7f2feaffd700 1 -- 172.21.15.55:0/2776545305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fec0cb230 msgr2=0x7f2fec0cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:31.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.686+0000 7f2feaffd700 1 --2- 172.21.15.55:0/2776545305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fec0cb230 0x7f2fec0cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.686+0000 7f2feaffd700 1 -- 172.21.15.55:0/2776545305 >> v1:172.21.15.55:6789/0 conn(0x7f2fec0ccc00 legacy=0x7f2fec069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.686+0000 7f2feaffd700 1 -- 172.21.15.55:0/2776545305 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec0c9f30 con 0x7f2fec159a60 2023-10-06T05:05:31.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.686+0000 7f2feb7fe700 1 --2- 172.21.15.55:0/2776545305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fec0cb230 0x7f2fec0cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:31.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.686+0000 7f2feaffd700 1 --2- 172.21.15.55:0/2776545305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fec159a60 0x7f2fec0c9770 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009910 tx=0x7f2fdc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de14a484c22b11bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:31.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.686+0000 7f2fe9ffb700 1 -- 172.21.15.55:0/2776545305 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc011630 con 0x7f2fec159a60 2023-10-06T05:05:31.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.686+0000 7f2fe9ffb700 1 -- 172.21.15.55:0/2776545305 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc011790 con 0x7f2fec159a60 2023-10-06T05:05:31.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.687+0000 7f2fe9ffb700 1 -- 172.21.15.55:0/2776545305 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc011b30 con 0x7f2fec159a60 2023-10-06T05:05:31.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.687+0000 7f2febfff700 1 -- 172.21.15.55:0/2776545305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fec159a60 msgr2=0x7f2fec0c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.687+0000 7f2febfff700 1 --2- 172.21.15.55:0/2776545305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fec159a60 0x7f2fec0c9770 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009910 tx=0x7f2fdc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.687+0000 7f2febfff700 1 -- 172.21.15.55:0/2776545305 shutdown_connections 2023-10-06T05:05:31.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.687+0000 7f2febfff700 1 --2- 172.21.15.55:0/2776545305 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fec0cb230 0x7f2fec0cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.687+0000 7f2febfff700 1 --2- 172.21.15.55:0/2776545305 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fec159a60 0x7f2fec0c9770 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.687+0000 7f2febfff700 1 -- 172.21.15.55:0/2776545305 >> 172.21.15.55:0/2776545305 conn(0x7f2fec06baa0 msgr2=0x7f2fec06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:31.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.687+0000 7f2febfff700 1 -- 172.21.15.55:0/2776545305 shutdown_connections 2023-10-06T05:05:31.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.687+0000 7f2febfff700 1 -- 172.21.15.55:0/2776545305 wait complete. 2023-10-06T05:05:31.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.689+0000 7f2febfff700 1 Processor -- start 2023-10-06T05:05:31.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.689+0000 7f2febfff700 1 -- start start 2023-10-06T05:05:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2febfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fec0cb230 0x7f2fec149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2febfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fec0ccc00 0x7f2fec0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2febfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fec159a60 0x7f2fec0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2febfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0b1e10 con 0x7f2fec0cb230 2023-10-06T05:05:31.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2febfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0b20f0 con 0x7f2fec159a60 2023-10-06T05:05:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2febfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0b23d0 con 0x7f2fec0ccc00 2023-10-06T05:05:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2feaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fec0cb230 0x7f2fec149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2feaffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fec0cb230 0x7f2fec149030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58270/0 (socket says 172.21.15.55:58270) 2023-10-06T05:05:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2feaffd700 1 -- 172.21.15.55:0/206753471 learned_addr learned my addr 172.21.15.55:0/206753471 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2feb7fe700 1 --2- 172.21.15.55:0/206753471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fec159a60 0x7f2fec0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2fea7fc700 1 --2- 172.21.15.55:0/206753471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fec0ccc00 0x7f2fec0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2feaffd700 1 -- 172.21.15.55:0/206753471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fec0ccc00 msgr2=0x7f2fec0b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2feaffd700 1 --2- 172.21.15.55:0/206753471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fec0ccc00 0x7f2fec0b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2feaffd700 1 -- 172.21.15.55:0/206753471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fec159a60 msgr2=0x7f2fec0b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.690+0000 7f2feaffd700 1 --2- 172.21.15.55:0/206753471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fec159a60 0x7f2fec0b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.691+0000 7f2feaffd700 1 -- 172.21.15.55:0/206753471 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe4009a20 con 0x7f2fec0cb230 2023-10-06T05:05:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.691+0000 7f2feaffd700 1 --2- 172.21.15.55:0/206753471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fec0cb230 0x7f2fec149030 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f2fec06cdc0 tx=0x7f2fdc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.691+0000 7f2fd2ffd700 1 -- 172.21.15.55:0/206753471 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc011900 con 0x7f2fec0cb230 2023-10-06T05:05:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.691+0000 7f2fd2ffd700 1 -- 172.21.15.55:0/206753471 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc011a60 con 0x7f2fec0cb230 2023-10-06T05:05:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.691+0000 7f2fd2ffd700 1 -- 172.21.15.55:0/206753471 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc025670 con 0x7f2fec0cb230 2023-10-06T05:05:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.691+0000 7f2febfff700 1 -- 172.21.15.55:0/206753471 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc009420 con 0x7f2fec0cb230 2023-10-06T05:05:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.691+0000 7f2febfff700 1 -- 172.21.15.55:0/206753471 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fec0b2990 con 0x7f2fec0cb230 2023-10-06T05:05:31.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.692+0000 7f2febfff700 1 -- 172.21.15.55:0/206753471 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fec0b2bb0 con 0x7f2fec0cb230 2023-10-06T05:05:31.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.693+0000 7f2fd2ffd700 1 -- 172.21.15.55:0/206753471 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2fdc023070 con 0x7f2fec0cb230 2023-10-06T05:05:31.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.694+0000 7f2fd2ffd700 1 --2- 172.21.15.55:0/206753471 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2fd8066f30 0x7f2fd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:31.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.694+0000 7f2fd2ffd700 1 -- 172.21.15.55:0/206753471 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6572+0+0 (secure 0 0 0) 0x7f2fdc085da0 con 0x7f2fec0cb230 2023-10-06T05:05:31.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.695+0000 7f2fea7fc700 1 --2- 172.21.15.55:0/206753471 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2fd8066f30 0x7f2fd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:31.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.696+0000 7f2fea7fc700 1 --2- 172.21.15.55:0/206753471 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2fd8066f30 0x7f2fd80693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2fec154430 tx=0x7f2fe0009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:31.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.699+0000 7f2fd2ffd700 1 -- 172.21.15.55:0/206753471 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fdc052cd0 con 0x7f2fec0cb230 2023-10-06T05:05:31.902 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.900+0000 7f2febfff700 1 -- 172.21.15.55:0/206753471 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f2fec0ca470 con 0x7f2fec0cb230 2023-10-06T05:05:31.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.904+0000 7f2fd2ffd700 1 -- 172.21.15.55:0/206753471 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v62) v1 ==== 115+0+0 (secure 0 0 0) 0x7f2fdc02a020 con 0x7f2fec0cb230 2023-10-06T05:05:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.906+0000 7f2febfff700 1 -- 172.21.15.55:0/206753471 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2fd8066f30 msgr2=0x7f2fd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.906+0000 7f2febfff700 1 --2- 172.21.15.55:0/206753471 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2fd8066f30 0x7f2fd80693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2fec154430 tx=0x7f2fe0009040 comp rx=0 tx=0).stop 2023-10-06T05:05:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.906+0000 7f2febfff700 1 -- 172.21.15.55:0/206753471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fec0cb230 msgr2=0x7f2fec149030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.906+0000 7f2febfff700 1 --2- 172.21.15.55:0/206753471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fec0cb230 0x7f2fec149030 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f2fec06cdc0 tx=0x7f2fdc017000 comp rx=0 tx=0).stop 2023-10-06T05:05:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.907+0000 7f2febfff700 1 -- 172.21.15.55:0/206753471 shutdown_connections 2023-10-06T05:05:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.907+0000 7f2febfff700 1 --2- 172.21.15.55:0/206753471 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2fd8066f30 0x7f2fd80693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.907+0000 7f2febfff700 1 --2- 172.21.15.55:0/206753471 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2fec159a60 0x7f2fec0b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.907+0000 7f2febfff700 1 --2- 172.21.15.55:0/206753471 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2fec0cb230 0x7f2fec149030 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.907+0000 7f2febfff700 1 --2- 172.21.15.55:0/206753471 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2fec0ccc00 0x7f2fec0b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.907+0000 7f2febfff700 1 -- 172.21.15.55:0/206753471 >> 172.21.15.55:0/206753471 conn(0x7f2fec06baa0 msgr2=0x7f2fec06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.907+0000 7f2febfff700 1 -- 172.21.15.55:0/206753471 shutdown_connections 2023-10-06T05:05:31.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:31.907+0000 7f2febfff700 1 -- 172.21.15.55:0/206753471 wait complete. 2023-10-06T05:05:31.926 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T05:05:32.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.114+0000 7f29e759e700 1 Processor -- start 2023-10-06T05:05:32.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.114+0000 7f29e759e700 1 -- start start 2023-10-06T05:05:32.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.116+0000 7f29e759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29e815a3b0 0x7f29e815a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:32.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.116+0000 7f29e759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f29e806e470 0x7f29e806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:32.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.116+0000 7f29e759e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e80aea70 con 0x7f29e80aef00 2023-10-06T05:05:32.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.116+0000 7f29e759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e806ee30 con 0x7f29e815a3b0 2023-10-06T05:05:32.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.116+0000 7f29e759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e815dad0 con 0x7f29e806e470 2023-10-06T05:05:32.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.116+0000 7f29e659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29e815a3b0 0x7f29e815a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:32.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.116+0000 7f29e659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f29e815a3b0 0x7f29e815a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58276/0 (socket says 172.21.15.55:58276) 2023-10-06T05:05:32.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.116+0000 7f29e659c700 1 -- 172.21.15.55:0/3233999358 learned_addr learned my addr 172.21.15.55:0/3233999358 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:32.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.116+0000 7f29e5d9b700 1 --1- 172.21.15.55:0/3233999358 >> v1:172.21.15.55:6789/0 conn(0x7f29e80aef00 0x7f29e815d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52596/0 (socket says 172.21.15.55:52596) 2023-10-06T05:05:32.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.116+0000 7f29e659c700 1 -- 172.21.15.55:0/3233999358 >> v2:172.21.15.163:3300/0 conn(0x7f29e806e470 msgr2=0x7f29e806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:32.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.116+0000 7f29e659c700 1 --2- 172.21.15.55:0/3233999358 >> v2:172.21.15.163:3300/0 conn(0x7f29e806e470 0x7f29e806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.116+0000 7f29e659c700 1 -- 172.21.15.55:0/3233999358 >> v1:172.21.15.55:6789/0 conn(0x7f29e80aef00 legacy=0x7f29e815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:32.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.116+0000 7f29e659c700 1 -- 172.21.15.55:0/3233999358 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e80ae480 con 0x7f29e815a3b0 2023-10-06T05:05:32.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.117+0000 7f29e659c700 1 --2- 172.21.15.55:0/3233999358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29e815a3b0 0x7f29e815a780 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f29e80c8660 tx=0x7f29d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae01c3d333c61f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:32.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.117+0000 7f29e559a700 1 -- 172.21.15.55:0/3233999358 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d800b860 con 0x7f29e815a3b0 2023-10-06T05:05:32.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.117+0000 7f29e559a700 1 -- 172.21.15.55:0/3233999358 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d800b9c0 con 0x7f29e815a3b0 2023-10-06T05:05:32.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.117+0000 7f29e559a700 1 -- 172.21.15.55:0/3233999358 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d800bd60 con 0x7f29e815a3b0 2023-10-06T05:05:32.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.118+0000 7f29e759e700 1 -- 172.21.15.55:0/3233999358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29e815a3b0 msgr2=0x7f29e815a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:32.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.118+0000 7f29e759e700 1 --2- 172.21.15.55:0/3233999358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29e815a3b0 0x7f29e815a780 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f29e80c8660 tx=0x7f29d8009250 comp rx=0 tx=0).stop 2023-10-06T05:05:32.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.118+0000 7f29e759e700 1 -- 172.21.15.55:0/3233999358 shutdown_connections 2023-10-06T05:05:32.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.118+0000 7f29e759e700 1 --2- 172.21.15.55:0/3233999358 >> v2:172.21.15.163:3300/0 conn(0x7f29e806e470 0x7f29e806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.118+0000 7f29e759e700 1 --2- 172.21.15.55:0/3233999358 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29e815a3b0 0x7f29e815a780 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.118+0000 7f29e759e700 1 -- 172.21.15.55:0/3233999358 >> 172.21.15.55:0/3233999358 conn(0x7f29e80c9f70 msgr2=0x7f29e806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:32.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.118+0000 7f29e759e700 1 -- 172.21.15.55:0/3233999358 shutdown_connections 2023-10-06T05:05:32.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.118+0000 7f29e759e700 1 -- 172.21.15.55:0/3233999358 wait complete. 2023-10-06T05:05:32.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.119+0000 7f29e759e700 1 Processor -- start 2023-10-06T05:05:32.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.119+0000 7f29e759e700 1 -- start start 2023-10-06T05:05:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.120+0000 7f29e759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29e806e470 0x7f29e8151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.120+0000 7f29e759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29e80aef00 0x7f29e816b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.120+0000 7f29e759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29e815a3b0 0x7f29e81550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.120+0000 7f29e759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e80c5ec0 con 0x7f29e815a3b0 2023-10-06T05:05:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.120+0000 7f29e759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e80c52b0 con 0x7f29e806e470 2023-10-06T05:05:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.120+0000 7f29e759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e80c5590 con 0x7f29e80aef00 2023-10-06T05:05:32.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.121+0000 7f29e6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29e815a3b0 0x7f29e81550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.121+0000 7f29e6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29e815a3b0 0x7f29e81550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58284/0 (socket says 172.21.15.55:58284) 2023-10-06T05:05:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.121+0000 7f29e6d9d700 1 -- 172.21.15.55:0/647780062 learned_addr learned my addr 172.21.15.55:0/647780062 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.121+0000 7f29e5d9b700 1 --2- 172.21.15.55:0/647780062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29e80aef00 0x7f29e816b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.121+0000 7f29e659c700 1 --2- 172.21.15.55:0/647780062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29e806e470 0x7f29e8151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.121+0000 7f29e6d9d700 1 -- 172.21.15.55:0/647780062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29e80aef00 msgr2=0x7f29e816b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.121+0000 7f29e6d9d700 1 --2- 172.21.15.55:0/647780062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29e80aef00 0x7f29e816b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.121+0000 7f29e6d9d700 1 -- 172.21.15.55:0/647780062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29e806e470 msgr2=0x7f29e8151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.121+0000 7f29e6d9d700 1 --2- 172.21.15.55:0/647780062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29e806e470 0x7f29e8151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.121+0000 7f29e6d9d700 1 -- 172.21.15.55:0/647780062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d80094f0 con 0x7f29e815a3b0 2023-10-06T05:05:32.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.121+0000 7f29e6d9d700 1 --2- 172.21.15.55:0/647780062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29e815a3b0 0x7f29e81550b0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f29e000bc30 tx=0x7f29e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.122+0000 7f29d67fc700 1 -- 172.21.15.55:0/647780062 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e00092e0 con 0x7f29e815a3b0 2023-10-06T05:05:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.122+0000 7f29d67fc700 1 -- 172.21.15.55:0/647780062 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e000c630 con 0x7f29e815a3b0 2023-10-06T05:05:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.122+0000 7f29d67fc700 1 -- 172.21.15.55:0/647780062 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e000ca10 con 0x7f29e815a3b0 2023-10-06T05:05:32.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.122+0000 7f29e759e700 1 -- 172.21.15.55:0/647780062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e80c5870 con 0x7f29e815a3b0 2023-10-06T05:05:32.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.122+0000 7f29e759e700 1 -- 172.21.15.55:0/647780062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e816b680 con 0x7f29e815a3b0 2023-10-06T05:05:32.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.123+0000 7f29e759e700 1 -- 172.21.15.55:0/647780062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e80c5d10 con 0x7f29e815a3b0 2023-10-06T05:05:32.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.124+0000 7f29d67fc700 1 -- 172.21.15.55:0/647780062 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f29e0012070 con 0x7f29e815a3b0 2023-10-06T05:05:32.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.124+0000 7f29d67fc700 1 --2- 172.21.15.55:0/647780062 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f29c8066f30 0x7f29c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:32.126 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.125+0000 7f29d67fc700 1 -- 172.21.15.55:0/647780062 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6572+0+0 (secure 0 0 0) 0x7f29e0084450 con 0x7f29e815a3b0 2023-10-06T05:05:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.125+0000 7f29e659c700 1 --2- 172.21.15.55:0/647780062 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f29c8066f30 0x7f29c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:32.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.126+0000 7f29e659c700 1 --2- 172.21.15.55:0/647780062 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f29c8066f30 0x7f29c80693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29e80c8660 tx=0x7f29d8017000 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:32.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.129+0000 7f29d67fc700 1 -- 172.21.15.55:0/647780062 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29e00512c0 con 0x7f29e815a3b0 2023-10-06T05:05:32.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.333+0000 7f29e759e700 1 -- 172.21.15.55:0/647780062 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f29e80c8110 con 0x7f29e815a3b0 2023-10-06T05:05:32.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.341+0000 7f29d67fc700 1 -- 172.21.15.55:0/647780062 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v63) v1 ==== 120+0+0 (secure 0 0 0) 0x7f29e00548e0 con 0x7f29e815a3b0 2023-10-06T05:05:32.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.343+0000 7f29e759e700 1 -- 172.21.15.55:0/647780062 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f29c8066f30 msgr2=0x7f29c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:32.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.343+0000 7f29e759e700 1 --2- 172.21.15.55:0/647780062 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f29c8066f30 0x7f29c80693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f29e80c8660 tx=0x7f29d8017000 comp rx=0 tx=0).stop 2023-10-06T05:05:32.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.343+0000 7f29e759e700 1 -- 172.21.15.55:0/647780062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29e815a3b0 msgr2=0x7f29e81550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:32.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.343+0000 7f29e759e700 1 --2- 172.21.15.55:0/647780062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29e815a3b0 0x7f29e81550b0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f29e000bc30 tx=0x7f29e0009840 comp rx=0 tx=0).stop 2023-10-06T05:05:32.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.343+0000 7f29e759e700 1 -- 172.21.15.55:0/647780062 shutdown_connections 2023-10-06T05:05:32.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.343+0000 7f29e759e700 1 --2- 172.21.15.55:0/647780062 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f29c8066f30 0x7f29c80693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.343+0000 7f29e759e700 1 --2- 172.21.15.55:0/647780062 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f29e815a3b0 0x7f29e81550b0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.343+0000 7f29e759e700 1 --2- 172.21.15.55:0/647780062 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f29e806e470 0x7f29e8151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.343+0000 7f29e759e700 1 --2- 172.21.15.55:0/647780062 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f29e80aef00 0x7f29e816b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.343+0000 7f29e759e700 1 -- 172.21.15.55:0/647780062 >> 172.21.15.55:0/647780062 conn(0x7f29e80c9f70 msgr2=0x7f29e8153060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:32.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.344+0000 7f29e759e700 1 -- 172.21.15.55:0/647780062 shutdown_connections 2023-10-06T05:05:32.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.344+0000 7f29e759e700 1 -- 172.21.15.55:0/647780062 wait complete. 2023-10-06T05:05:32.365 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-06T05:05:32.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.551+0000 7f490999d700 1 Processor -- start 2023-10-06T05:05:32.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.551+0000 7f490999d700 1 -- start start 2023-10-06T05:05:32.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.552+0000 7f490999d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f49041566e0 0x7f49040ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.552+0000 7f490999d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49040ca860 con 0x7f49040cb900 2023-10-06T05:05:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.552+0000 7f490999d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49040cab40 con 0x7f4904156310 2023-10-06T05:05:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.552+0000 7f490999d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49040cae20 con 0x7f49041566e0 2023-10-06T05:05:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.553+0000 7f4903fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f49041566e0 0x7f49040ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.553+0000 7f490919c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f49040cb900 0x7f49040c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52612/0 (socket says 172.21.15.55:52612) 2023-10-06T05:05:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.553+0000 7f490919c700 1 -- 172.21.15.55:0/578815026 learned_addr learned my addr 172.21.15.55:0/578815026 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.553+0000 7f4903fff700 1 -- 172.21.15.55:0/578815026 >> v1:172.21.15.163:6789/0 conn(0x7f4904156310 legacy=0x7f49040c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.553+0000 7f4903fff700 1 -- 172.21.15.55:0/578815026 >> v1:172.21.15.55:6789/0 conn(0x7f49040cb900 legacy=0x7f49040c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:32.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.553+0000 7f4903fff700 1 -- 172.21.15.55:0/578815026 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49040cb100 con 0x7f49041566e0 2023-10-06T05:05:32.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.553+0000 7f4903fff700 1 --2- 172.21.15.55:0/578815026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49041566e0 0x7f49040ae000 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f48f00098e0 tx=0x7f48f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47fd09da4b30b774 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:32.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.553+0000 7f49037fe700 1 -- 172.21.15.55:0/578815026 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0009e80 con 0x7f49041566e0 2023-10-06T05:05:32.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.554+0000 7f49037fe700 1 -- 172.21.15.55:0/578815026 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f000b820 con 0x7f49041566e0 2023-10-06T05:05:32.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.554+0000 7f49037fe700 1 -- 172.21.15.55:0/578815026 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f000bba0 con 0x7f49041566e0 2023-10-06T05:05:32.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.554+0000 7f490999d700 1 -- 172.21.15.55:0/578815026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49041566e0 msgr2=0x7f49040ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:32.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.554+0000 7f490999d700 1 --2- 172.21.15.55:0/578815026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49041566e0 0x7f49040ae000 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f48f00098e0 tx=0x7f48f00091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.554+0000 7f490999d700 1 -- 172.21.15.55:0/578815026 shutdown_connections 2023-10-06T05:05:32.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.554+0000 7f490999d700 1 --2- 172.21.15.55:0/578815026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49041566e0 0x7f49040ae000 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.554+0000 7f490999d700 1 -- 172.21.15.55:0/578815026 >> 172.21.15.55:0/578815026 conn(0x7f49040cc1f0 msgr2=0x7f49040cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:32.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.554+0000 7f490999d700 1 -- 172.21.15.55:0/578815026 shutdown_connections 2023-10-06T05:05:32.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.554+0000 7f490999d700 1 -- 172.21.15.55:0/578815026 wait complete. 2023-10-06T05:05:32.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.556+0000 7f490999d700 1 Processor -- start 2023-10-06T05:05:32.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.556+0000 7f490999d700 1 -- start start 2023-10-06T05:05:32.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f490999d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49040cb900 0x7f4904169a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:32.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f490999d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4904156310 0x7f49041414e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:32.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f490999d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49041566e0 0x7f49041405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:32.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f490999d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49040b1270 con 0x7f49041566e0 2023-10-06T05:05:32.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f490999d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490413f560 con 0x7f4904156310 2023-10-06T05:05:32.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f490999d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490413f840 con 0x7f49040cb900 2023-10-06T05:05:32.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f490919c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49041566e0 0x7f49041405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:32.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f490919c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49041566e0 0x7f49041405e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:58312/0 (socket says 172.21.15.55:58312) 2023-10-06T05:05:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f490919c700 1 -- 172.21.15.55:0/4177372313 learned_addr learned my addr 172.21.15.55:0/4177372313 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f490899b700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49040cb900 0x7f4904169a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f4903fff700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4904156310 0x7f49041414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f490919c700 1 -- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49040cb900 msgr2=0x7f4904169a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f490919c700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49040cb900 0x7f4904169a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f490919c700 1 -- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4904156310 msgr2=0x7f49041414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.557+0000 7f490919c700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4904156310 0x7f49041414e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.558+0000 7f490919c700 1 -- 172.21.15.55:0/4177372313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f0009420 con 0x7f49041566e0 2023-10-06T05:05:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.558+0000 7f490899b700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49040cb900 0x7f4904169a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.558+0000 7f4903fff700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4904156310 0x7f49041414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:32.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.558+0000 7f490919c700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49041566e0 0x7f49041405e0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00cbc0 tx=0x7f48fc0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.558+0000 7f4900ff9700 1 -- 172.21.15.55:0/4177372313 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00a940 con 0x7f49041566e0 2023-10-06T05:05:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.558+0000 7f4900ff9700 1 -- 172.21.15.55:0/4177372313 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc00aaa0 con 0x7f49041566e0 2023-10-06T05:05:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.558+0000 7f490999d700 1 -- 172.21.15.55:0/4177372313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490413fb20 con 0x7f49041566e0 2023-10-06T05:05:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.558+0000 7f4900ff9700 1 -- 172.21.15.55:0/4177372313 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00ae20 con 0x7f49041566e0 2023-10-06T05:05:32.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.558+0000 7f490999d700 1 -- 172.21.15.55:0/4177372313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4904169f50 con 0x7f49041566e0 2023-10-06T05:05:32.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.559+0000 7f490999d700 1 -- 172.21.15.55:0/4177372313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49041402a0 con 0x7f49041566e0 2023-10-06T05:05:32.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.560+0000 7f4900ff9700 1 -- 172.21.15.55:0/4177372313 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48fc007300 con 0x7f49041566e0 2023-10-06T05:05:32.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.561+0000 7f4900ff9700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f48e8066f30 0x7f48e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:32.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.561+0000 7f4900ff9700 1 -- 172.21.15.55:0/4177372313 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6572+0+0 (secure 0 0 0) 0x7f48fc084cf0 con 0x7f49041566e0 2023-10-06T05:05:32.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.561+0000 7f490899b700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f48e8066f30 0x7f48e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:32.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.562+0000 7f490899b700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f48e8066f30 0x7f48e80693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f48f8007fd0 tx=0x7f48f8009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:32.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.565+0000 7f4900ff9700 1 -- 172.21.15.55:0/4177372313 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48fc051be0 con 0x7f49041566e0 2023-10-06T05:05:32.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.845+0000 7f490999d700 1 -- 172.21.15.55:0/4177372313 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f4904140450 con 0x7f49041566e0 2023-10-06T05:05:32.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.859+0000 7f4900ff9700 1 -- 172.21.15.55:0/4177372313 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2971) v1 ==== 165+0+0 (secure 0 0 0) 0x7f48fc055200 con 0x7f49041566e0 2023-10-06T05:05:32.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.862+0000 7f490999d700 1 -- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f48e8066f30 msgr2=0x7f48e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:32.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.862+0000 7f490999d700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f48e8066f30 0x7f48e80693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f48f8007fd0 tx=0x7f48f8009040 comp rx=0 tx=0).stop 2023-10-06T05:05:32.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.862+0000 7f490999d700 1 -- 172.21.15.55:0/4177372313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49041566e0 msgr2=0x7f49041405e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:32.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.862+0000 7f490999d700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49041566e0 0x7f49041405e0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00cbc0 tx=0x7f48fc0019c0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.862+0000 7f490999d700 1 -- 172.21.15.55:0/4177372313 shutdown_connections 2023-10-06T05:05:32.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.862+0000 7f490999d700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f48e8066f30 0x7f48e80693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.862+0000 7f490999d700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f49041566e0 0x7f49041405e0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.862+0000 7f490999d700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f49040cb900 0x7f4904169a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.862+0000 7f490999d700 1 --2- 172.21.15.55:0/4177372313 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4904156310 0x7f49041414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:32.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.862+0000 7f490999d700 1 -- 172.21.15.55:0/4177372313 >> 172.21.15.55:0/4177372313 conn(0x7f49040cc1f0 msgr2=0x7f4904151db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:32.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.862+0000 7f490999d700 1 -- 172.21.15.55:0/4177372313 shutdown_connections 2023-10-06T05:05:32.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:32.862+0000 7f490999d700 1 -- 172.21.15.55:0/4177372313 wait complete. 2023-10-06T05:05:32.881 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-10-06T05:05:32.881 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-10-06T05:05:32.881 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-06T05:05:32.882 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-06T05:05:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.069+0000 7f3be539d700 1 Processor -- start 2023-10-06T05:05:33.071 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.069+0000 7f3be539d700 1 -- start start 2023-10-06T05:05:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.070+0000 7f3be539d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3be00ad870 0x7f3be006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.070+0000 7f3be539d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3be006bf20 0x7f3be006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.070+0000 7f3be539d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be00b40f0 con 0x7f3be0142ea0 2023-10-06T05:05:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.070+0000 7f3be539d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be006c3a0 con 0x7f3be006bf20 2023-10-06T05:05:33.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.070+0000 7f3be539d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be0069640 con 0x7f3be00ad870 2023-10-06T05:05:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.070+0000 7f3be4b9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3be006bf20 0x7f3be006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.070+0000 7f3be4b9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3be006bf20 0x7f3be006a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38992/0 (socket says 172.21.15.55:38992) 2023-10-06T05:05:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.070+0000 7f3be4b9c700 1 -- 172.21.15.55:0/1112775381 learned_addr learned my addr 172.21.15.55:0/1112775381 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:33.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.071+0000 7f3bdf7fe700 1 --2- 172.21.15.55:0/1112775381 >> v2:172.21.15.163:3300/0 conn(0x7f3be00ad870 0x7f3be006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.071+0000 7f3be4b9c700 1 -- 172.21.15.55:0/1112775381 >> v2:172.21.15.163:3300/0 conn(0x7f3be00ad870 msgr2=0x7f3be006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:33.074 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.071+0000 7f3be4b9c700 1 --2- 172.21.15.55:0/1112775381 >> v2:172.21.15.163:3300/0 conn(0x7f3be00ad870 0x7f3be006b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:33.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.071+0000 7f3be4b9c700 1 -- 172.21.15.55:0/1112775381 >> v1:172.21.15.163:6789/0 conn(0x7f3be0142ea0 legacy=0x7f3be00b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:33.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.071+0000 7f3be4b9c700 1 -- 172.21.15.55:0/1112775381 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be0069920 con 0x7f3be006bf20 2023-10-06T05:05:33.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.071+0000 7f3be4b9c700 1 --2- 172.21.15.55:0/1112775381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be006bf20 0x7f3be006a120 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f3be00b0e10 tx=0x7f3bd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba968cd4f9ae8ef server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:33.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.071+0000 7f3bdeffd700 1 -- 172.21.15.55:0/1112775381 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd800e070 con 0x7f3be006bf20 2023-10-06T05:05:33.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.071+0000 7f3bdeffd700 1 -- 172.21.15.55:0/1112775381 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd800ba20 con 0x7f3be006bf20 2023-10-06T05:05:33.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.071+0000 7f3bdeffd700 1 -- 172.21.15.55:0/1112775381 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80092e0 con 0x7f3be006bf20 2023-10-06T05:05:33.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.072+0000 7f3be539d700 1 -- 172.21.15.55:0/1112775381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be006bf20 msgr2=0x7f3be006a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:33.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.072+0000 7f3be539d700 1 --2- 172.21.15.55:0/1112775381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be006bf20 0x7f3be006a120 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f3be00b0e10 tx=0x7f3bd8009840 comp rx=0 tx=0).stop 2023-10-06T05:05:33.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.072+0000 7f3be539d700 1 -- 172.21.15.55:0/1112775381 shutdown_connections 2023-10-06T05:05:33.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.072+0000 7f3be539d700 1 --2- 172.21.15.55:0/1112775381 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be006bf20 0x7f3be006a120 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:33.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.072+0000 7f3be539d700 1 --2- 172.21.15.55:0/1112775381 >> v2:172.21.15.163:3300/0 conn(0x7f3be00ad870 0x7f3be006b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:33.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.072+0000 7f3be539d700 1 -- 172.21.15.55:0/1112775381 >> 172.21.15.55:0/1112775381 conn(0x7f3be00ce430 msgr2=0x7f3be00ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:33.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.072+0000 7f3be539d700 1 -- 172.21.15.55:0/1112775381 shutdown_connections 2023-10-06T05:05:33.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.072+0000 7f3be539d700 1 -- 172.21.15.55:0/1112775381 wait complete. 2023-10-06T05:05:33.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.074+0000 7f3be539d700 1 Processor -- start 2023-10-06T05:05:33.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.074+0000 7f3be539d700 1 -- start start 2023-10-06T05:05:33.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3be539d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be006bf20 0x7f3be0169c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:33.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3be539d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be0142ea0 0x7f3be00acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:33.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3be539d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be00c85b0 0x7f3be015c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:33.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3be539d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be00c9a80 con 0x7f3be00c85b0 2023-10-06T05:05:33.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3be539d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be00c8e00 con 0x7f3be0142ea0 2023-10-06T05:05:33.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3be539d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be00c90e0 con 0x7f3be006bf20 2023-10-06T05:05:33.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3be4b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be00c85b0 0x7f3be015c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:33.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3be4b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be00c85b0 0x7f3be015c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:38998/0 (socket says 172.21.15.55:38998) 2023-10-06T05:05:33.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3be4b9c700 1 -- 172.21.15.55:0/52376894 learned_addr learned my addr 172.21.15.55:0/52376894 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:33.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3bdffff700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be006bf20 0x7f3be0169c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:33.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3bdf7fe700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be0142ea0 0x7f3be00acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:33.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3be4b9c700 1 -- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be006bf20 msgr2=0x7f3be0169c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:33.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3be4b9c700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be006bf20 0x7f3be0169c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:33.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3be4b9c700 1 -- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be0142ea0 msgr2=0x7f3be00acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:33.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3be4b9c700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be0142ea0 0x7f3be00acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:33.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3be4b9c700 1 -- 172.21.15.55:0/52376894 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd8009ae0 con 0x7f3be00c85b0 2023-10-06T05:05:33.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3bdffff700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be006bf20 0x7f3be0169c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:33.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.075+0000 7f3bdf7fe700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be0142ea0 0x7f3be00acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:33.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.076+0000 7f3be4b9c700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be00c85b0 0x7f3be015c8f0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009880 tx=0x7f3bd8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:33.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.076+0000 7f3bc7fff700 1 -- 172.21.15.55:0/52376894 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8008be0 con 0x7f3be00c85b0 2023-10-06T05:05:33.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.076+0000 7f3bc7fff700 1 -- 172.21.15.55:0/52376894 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd8008d60 con 0x7f3be00c85b0 2023-10-06T05:05:33.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.076+0000 7f3bc7fff700 1 -- 172.21.15.55:0/52376894 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd80266b0 con 0x7f3be00c85b0 2023-10-06T05:05:33.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.076+0000 7f3be539d700 1 -- 172.21.15.55:0/52376894 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be00c93c0 con 0x7f3be00c85b0 2023-10-06T05:05:33.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.076+0000 7f3be539d700 1 -- 172.21.15.55:0/52376894 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be00ad120 con 0x7f3be00c85b0 2023-10-06T05:05:33.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.077+0000 7f3be539d700 1 -- 172.21.15.55:0/52376894 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be00b3e60 con 0x7f3be00c85b0 2023-10-06T05:05:33.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.078+0000 7f3bc7fff700 1 -- 172.21.15.55:0/52376894 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3bd8003040 con 0x7f3be00c85b0 2023-10-06T05:05:33.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.079+0000 7f3bc7fff700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3bcc066f30 0x7f3bcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:33.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.079+0000 7f3bc7fff700 1 -- 172.21.15.55:0/52376894 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6572+0+0 (secure 0 0 0) 0x7f3bd80863a0 con 0x7f3be00c85b0 2023-10-06T05:05:33.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.079+0000 7f3bdffff700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3bcc066f30 0x7f3bcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:33.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.080+0000 7f3bdffff700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3bcc066f30 0x7f3bcc0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0007fd0 tx=0x7f3bd0009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:33.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.083+0000 7f3bc7fff700 1 -- 172.21.15.55:0/52376894 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3bd8053210 con 0x7f3be00c85b0 2023-10-06T05:05:33.369 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.367+0000 7f3be539d700 1 -- 172.21.15.55:0/52376894 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f3be004c3c0 con 0x7f3be00c85b0 2023-10-06T05:05:33.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.857+0000 7f3bc7fff700 1 -- 172.21.15.55:0/52376894 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v2972) v1 ==== 166+0+0 (secure 0 0 0) 0x7f3bd805ae60 con 0x7f3be00c85b0 2023-10-06T05:05:33.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.860+0000 7f3be539d700 1 -- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3bcc066f30 msgr2=0x7f3bcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:33.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.860+0000 7f3be539d700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3bcc066f30 0x7f3bcc0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0007fd0 tx=0x7f3bd0009040 comp rx=0 tx=0).stop 2023-10-06T05:05:33.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.860+0000 7f3be539d700 1 -- 172.21.15.55:0/52376894 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be00c85b0 msgr2=0x7f3be015c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:33.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.860+0000 7f3be539d700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be00c85b0 0x7f3be015c8f0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009880 tx=0x7f3bd8018000 comp rx=0 tx=0).stop 2023-10-06T05:05:33.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.860+0000 7f3be539d700 1 -- 172.21.15.55:0/52376894 shutdown_connections 2023-10-06T05:05:33.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.860+0000 7f3be539d700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3bcc066f30 0x7f3bcc0693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:33.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.860+0000 7f3be539d700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3be00c85b0 0x7f3be015c8f0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:33.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.860+0000 7f3be539d700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3be006bf20 0x7f3be0169c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:33.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.860+0000 7f3be539d700 1 --2- 172.21.15.55:0/52376894 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3be0142ea0 0x7f3be00acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:33.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.860+0000 7f3be539d700 1 -- 172.21.15.55:0/52376894 >> 172.21.15.55:0/52376894 conn(0x7f3be00ce430 msgr2=0x7f3be00b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:33.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.860+0000 7f3be539d700 1 -- 172.21.15.55:0/52376894 shutdown_connections 2023-10-06T05:05:33.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:33.860+0000 7f3be539d700 1 -- 172.21.15.55:0/52376894 wait complete. 2023-10-06T05:05:33.881 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:05:34.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.073+0000 7f1ad6b9d700 1 Processor -- start 2023-10-06T05:05:34.075 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.073+0000 7f1ad6b9d700 1 -- start start 2023-10-06T05:05:34.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.074+0000 7f1ad6b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ad00b16d0 0x7f1ad00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:34.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.074+0000 7f1ad6b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad00b2080 con 0x7f1ad00b4e70 2023-10-06T05:05:34.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.074+0000 7f1ad6b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad00cfb00 con 0x7f1ad00b33b0 2023-10-06T05:05:34.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.074+0000 7f1ad6b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00b5c40 con 0x7f1ad00b16d0 2023-10-06T05:05:34.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.075+0000 7f1ad5b9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1ad00b4e70 0x7f1ad00ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45974/0 (socket says 172.21.15.55:45974) 2023-10-06T05:05:34.077 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.075+0000 7f1ad5b9b700 1 -- 172.21.15.55:0/1654607176 learned_addr learned my addr 172.21.15.55:0/1654607176 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:34.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.075+0000 7f1ad539a700 1 --2- 172.21.15.55:0/1654607176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ad00b16d0 0x7f1ad00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:34.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.075+0000 7f1ad4b99700 1 -- 172.21.15.55:0/1654607176 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860216795 0 0) 0x7f1ad00b2080 con 0x7f1ad00b4e70 2023-10-06T05:05:34.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.075+0000 7f1ad4b99700 1 -- 172.21.15.55:0/1654607176 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1abc003570 con 0x7f1ad00b4e70 2023-10-06T05:05:34.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.075+0000 7f1ad4b99700 1 -- 172.21.15.55:0/1654607176 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1279587749 0 0) 0x7f1ad00cfb00 con 0x7f1ad00b33b0 2023-10-06T05:05:34.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.075+0000 7f1ad4b99700 1 -- 172.21.15.55:0/1654607176 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ad00b2080 con 0x7f1ad00b33b0 2023-10-06T05:05:34.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.075+0000 7f1ad539a700 1 -- 172.21.15.55:0/1654607176 >> v1:172.21.15.163:6789/0 conn(0x7f1ad00b33b0 legacy=0x7f1ad00b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:34.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.075+0000 7f1ad539a700 1 -- 172.21.15.55:0/1654607176 >> v1:172.21.15.55:6789/0 conn(0x7f1ad00b4e70 legacy=0x7f1ad00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:34.078 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.075+0000 7f1ad539a700 1 -- 172.21.15.55:0/1654607176 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad00b5f20 con 0x7f1ad00b16d0 2023-10-06T05:05:34.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.076+0000 7f1ad539a700 1 --2- 172.21.15.55:0/1654607176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ad00b16d0 0x7f1ad00b1b40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009910 tx=0x7f1ac00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c63f1d4b9a60079 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:34.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.076+0000 7f1ad4b99700 1 -- 172.21.15.55:0/1654607176 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0011630 con 0x7f1ad00b16d0 2023-10-06T05:05:34.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.076+0000 7f1ad4b99700 1 -- 172.21.15.55:0/1654607176 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac0011790 con 0x7f1ad00b16d0 2023-10-06T05:05:34.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.076+0000 7f1ad4b99700 1 -- 172.21.15.55:0/1654607176 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0011b30 con 0x7f1ad00b16d0 2023-10-06T05:05:34.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.076+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/1654607176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ad00b16d0 msgr2=0x7f1ad00b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:34.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.076+0000 7f1ad6b9d700 1 --2- 172.21.15.55:0/1654607176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ad00b16d0 0x7f1ad00b1b40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009910 tx=0x7f1ac00091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.077+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/1654607176 shutdown_connections 2023-10-06T05:05:34.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.077+0000 7f1ad6b9d700 1 --2- 172.21.15.55:0/1654607176 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ad00b16d0 0x7f1ad00b1b40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.077+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/1654607176 >> 172.21.15.55:0/1654607176 conn(0x7f1ad00d10c0 msgr2=0x7f1ad00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:34.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.077+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/1654607176 shutdown_connections 2023-10-06T05:05:34.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.077+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/1654607176 wait complete. 2023-10-06T05:05:34.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.078+0000 7f1ad6b9d700 1 Processor -- start 2023-10-06T05:05:34.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.079+0000 7f1ad6b9d700 1 -- start start 2023-10-06T05:05:34.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.079+0000 7f1ad6b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ad00b16d0 0x7f1ad01692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:34.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.079+0000 7f1ad6b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ad00b33b0 0x7f1ad00c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:34.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.079+0000 7f1ad6b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad00b4e70 0x7f1ad00cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:34.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.080+0000 7f1ad6b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00ca6c0 con 0x7f1ad00b33b0 2023-10-06T05:05:34.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.080+0000 7f1ad6b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00ca9a0 con 0x7f1ad00b4e70 2023-10-06T05:05:34.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.080+0000 7f1ad6b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00cac80 con 0x7f1ad00b16d0 2023-10-06T05:05:34.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.080+0000 7f1ad539a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ad00b33b0 0x7f1ad00c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:34.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.080+0000 7f1ad539a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ad00b33b0 0x7f1ad00c9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39002/0 (socket says 172.21.15.55:39002) 2023-10-06T05:05:34.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.080+0000 7f1ad539a700 1 -- 172.21.15.55:0/3804569712 learned_addr learned my addr 172.21.15.55:0/3804569712 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:34.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.080+0000 7f1ad5b9b700 1 --2- 172.21.15.55:0/3804569712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ad00b16d0 0x7f1ad01692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:34.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.080+0000 7f1ad639c700 1 --2- 172.21.15.55:0/3804569712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad00b4e70 0x7f1ad00cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:34.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.080+0000 7f1ad539a700 1 -- 172.21.15.55:0/3804569712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ad00b16d0 msgr2=0x7f1ad01692a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:34.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.080+0000 7f1ad539a700 1 --2- 172.21.15.55:0/3804569712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ad00b16d0 0x7f1ad01692a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.080+0000 7f1ad539a700 1 -- 172.21.15.55:0/3804569712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad00b4e70 msgr2=0x7f1ad00cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:34.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.080+0000 7f1ad539a700 1 --2- 172.21.15.55:0/3804569712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad00b4e70 0x7f1ad00cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.080+0000 7f1ad539a700 1 -- 172.21.15.55:0/3804569712 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac0009420 con 0x7f1ad00b33b0 2023-10-06T05:05:34.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.081+0000 7f1ad539a700 1 --2- 172.21.15.55:0/3804569712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ad00b33b0 0x7f1ad00c9c70 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009f00 tx=0x7f1ac0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:34.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.081+0000 7f1ac5ffb700 1 -- 172.21.15.55:0/3804569712 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac001f070 con 0x7f1ad00b33b0 2023-10-06T05:05:34.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.081+0000 7f1ac5ffb700 1 -- 172.21.15.55:0/3804569712 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac0000bc0 con 0x7f1ad00b33b0 2023-10-06T05:05:34.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.081+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/3804569712 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad00caf60 con 0x7f1ad00b33b0 2023-10-06T05:05:34.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.081+0000 7f1ac5ffb700 1 -- 172.21.15.55:0/3804569712 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0011b30 con 0x7f1ad00b33b0 2023-10-06T05:05:34.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.081+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/3804569712 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad00a8a00 con 0x7f1ad00b33b0 2023-10-06T05:05:34.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.082+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/3804569712 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad00cb400 con 0x7f1ad00b33b0 2023-10-06T05:05:34.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.083+0000 7f1ac5ffb700 1 -- 172.21.15.55:0/3804569712 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1ac0023070 con 0x7f1ad00b33b0 2023-10-06T05:05:34.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.084+0000 7f1ac5ffb700 1 --2- 172.21.15.55:0/3804569712 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1ab4066f30 0x7f1ab40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:34.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.084+0000 7f1ac5ffb700 1 -- 172.21.15.55:0/3804569712 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6572+0+0 (secure 0 0 0) 0x7f1ac0085c60 con 0x7f1ad00b33b0 2023-10-06T05:05:34.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.084+0000 7f1ad5b9b700 1 --2- 172.21.15.55:0/3804569712 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1ab4066f30 0x7f1ab40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:34.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.085+0000 7f1ad5b9b700 1 --2- 172.21.15.55:0/3804569712 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1ab4066f30 0x7f1ab40693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0058d0 tx=0x7f1acc009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:34.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.088+0000 7f1ac5ffb700 1 -- 172.21.15.55:0/3804569712 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1ac0052b50 con 0x7f1ad00b33b0 2023-10-06T05:05:34.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.289+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/3804569712 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1ad004bb80 con 0x7f1ad00b33b0 2023-10-06T05:05:34.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.294+0000 7f1ac5ffb700 1 -- 172.21.15.55:0/3804569712 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1ac00560f0 con 0x7f1ad00b33b0 2023-10-06T05:05:34.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.296+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/3804569712 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1ab4066f30 msgr2=0x7f1ab40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:34.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.296+0000 7f1ad6b9d700 1 --2- 172.21.15.55:0/3804569712 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1ab4066f30 0x7f1ab40693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0058d0 tx=0x7f1acc009040 comp rx=0 tx=0).stop 2023-10-06T05:05:34.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.296+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/3804569712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ad00b33b0 msgr2=0x7f1ad00c9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:34.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.296+0000 7f1ad6b9d700 1 --2- 172.21.15.55:0/3804569712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ad00b33b0 0x7f1ad00c9c70 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009f00 tx=0x7f1ac0017000 comp rx=0 tx=0).stop 2023-10-06T05:05:34.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.296+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/3804569712 shutdown_connections 2023-10-06T05:05:34.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.296+0000 7f1ad6b9d700 1 --2- 172.21.15.55:0/3804569712 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1ab4066f30 0x7f1ab40693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.296+0000 7f1ad6b9d700 1 --2- 172.21.15.55:0/3804569712 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1ad00b4e70 0x7f1ad00cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.296+0000 7f1ad6b9d700 1 --2- 172.21.15.55:0/3804569712 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1ad00b16d0 0x7f1ad01692a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.296+0000 7f1ad6b9d700 1 --2- 172.21.15.55:0/3804569712 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1ad00b33b0 0x7f1ad00c9c70 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.296+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/3804569712 >> 172.21.15.55:0/3804569712 conn(0x7f1ad00d10c0 msgr2=0x7f1ad00b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:34.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.296+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/3804569712 shutdown_connections 2023-10-06T05:05:34.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.296+0000 7f1ad6b9d700 1 -- 172.21.15.55:0/3804569712 wait complete. 2023-10-06T05:05:34.316 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:05:34.316 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:05:34.397 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:05:34.484 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:05:34.484 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:05:34.484 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:05:34.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.670+0000 7fad1d59d700 1 Processor -- start 2023-10-06T05:05:34.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.670+0000 7fad1d59d700 1 -- start start 2023-10-06T05:05:34.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.671+0000 7fad1d59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fad1815d230 0x7fad1815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:34.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.671+0000 7fad1d59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad180b2980 con 0x7fad181597f0 2023-10-06T05:05:34.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.671+0000 7fad1d59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1815db70 con 0x7fad18159420 2023-10-06T05:05:34.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.671+0000 7fad1d59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad180b11d0 con 0x7fad1815d230 2023-10-06T05:05:34.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.671+0000 7fad1cd9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fad1815d230 0x7fad1815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:34.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.671+0000 7fad177fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fad181597f0 0x7fad180b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45988/0 (socket says 172.21.15.55:45988) 2023-10-06T05:05:34.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.671+0000 7fad1cd9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fad1815d230 0x7fad1815d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39008/0 (socket says 172.21.15.55:39008) 2023-10-06T05:05:34.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.671+0000 7fad177fe700 1 -- 172.21.15.55:0/1997194876 learned_addr learned my addr 172.21.15.55:0/1997194876 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:34.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.672+0000 7fad1cd9c700 1 -- 172.21.15.55:0/1997194876 >> v1:172.21.15.163:6789/0 conn(0x7fad18159420 legacy=0x7fad180b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:34.674 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.672+0000 7fad1cd9c700 1 -- 172.21.15.55:0/1997194876 >> v1:172.21.15.55:6789/0 conn(0x7fad181597f0 legacy=0x7fad180b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:34.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.672+0000 7fad1cd9c700 1 -- 172.21.15.55:0/1997194876 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad180b14b0 con 0x7fad1815d230 2023-10-06T05:05:34.675 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.672+0000 7fad1cd9c700 1 --2- 172.21.15.55:0/1997194876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad1815d230 0x7fad1815d630 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fad10009e60 tx=0x7fad10009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cefabc8a22f882b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:34.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.672+0000 7fad16ffd700 1 -- 172.21.15.55:0/1997194876 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1000e070 con 0x7fad1815d230 2023-10-06T05:05:34.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.672+0000 7fad16ffd700 1 -- 172.21.15.55:0/1997194876 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1000bc30 con 0x7fad1815d230 2023-10-06T05:05:34.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.672+0000 7fad16ffd700 1 -- 172.21.15.55:0/1997194876 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10012430 con 0x7fad1815d230 2023-10-06T05:05:34.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.673+0000 7fad1d59d700 1 -- 172.21.15.55:0/1997194876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad1815d230 msgr2=0x7fad1815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:34.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.673+0000 7fad1d59d700 1 --2- 172.21.15.55:0/1997194876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad1815d230 0x7fad1815d630 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fad10009e60 tx=0x7fad10009780 comp rx=0 tx=0).stop 2023-10-06T05:05:34.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.673+0000 7fad1d59d700 1 -- 172.21.15.55:0/1997194876 shutdown_connections 2023-10-06T05:05:34.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.673+0000 7fad1d59d700 1 --2- 172.21.15.55:0/1997194876 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad1815d230 0x7fad1815d630 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.673+0000 7fad1d59d700 1 -- 172.21.15.55:0/1997194876 >> 172.21.15.55:0/1997194876 conn(0x7fad180d0660 msgr2=0x7fad180d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:34.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.673+0000 7fad1d59d700 1 -- 172.21.15.55:0/1997194876 shutdown_connections 2023-10-06T05:05:34.676 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.673+0000 7fad1d59d700 1 -- 172.21.15.55:0/1997194876 wait complete. 2023-10-06T05:05:34.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.675+0000 7fad1d59d700 1 Processor -- start 2023-10-06T05:05:34.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.675+0000 7fad1d59d700 1 -- start start 2023-10-06T05:05:34.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad1d59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad18159420 0x7fad18153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:34.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad1d59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad181597f0 0x7fad18155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:34.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad1d59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad1815d230 0x7fad181560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:34.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad1d59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad180c54a0 con 0x7fad18159420 2023-10-06T05:05:34.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad1d59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad180c5710 con 0x7fad1815d230 2023-10-06T05:05:34.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad1d59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad180c59f0 con 0x7fad181597f0 2023-10-06T05:05:34.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad17fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad18159420 0x7fad18153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:34.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad17fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad18159420 0x7fad18153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39024/0 (socket says 172.21.15.55:39024) 2023-10-06T05:05:34.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad17fff700 1 -- 172.21.15.55:0/2550727235 learned_addr learned my addr 172.21.15.55:0/2550727235 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:34.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad1cd9c700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad1815d230 0x7fad181560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:34.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad177fe700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad181597f0 0x7fad18155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:34.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad17fff700 1 -- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad181597f0 msgr2=0x7fad18155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:34.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad17fff700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad181597f0 0x7fad18155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad17fff700 1 -- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad1815d230 msgr2=0x7fad181560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:34.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad17fff700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad1815d230 0x7fad181560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad17fff700 1 -- 172.21.15.55:0/2550727235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad10009a20 con 0x7fad18159420 2023-10-06T05:05:34.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad1cd9c700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad1815d230 0x7fad181560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:34.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.676+0000 7fad177fe700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad181597f0 0x7fad18155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:34.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.677+0000 7fad17fff700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad18159420 0x7fad18153be0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fad0800c790 tx=0x7fad08007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:34.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.677+0000 7facfffff700 1 -- 172.21.15.55:0/2550727235 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08011590 con 0x7fad18159420 2023-10-06T05:05:34.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.677+0000 7facfffff700 1 -- 172.21.15.55:0/2550727235 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad08011710 con 0x7fad18159420 2023-10-06T05:05:34.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.677+0000 7facfffff700 1 -- 172.21.15.55:0/2550727235 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08011ad0 con 0x7fad18159420 2023-10-06T05:05:34.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.677+0000 7fad1d59d700 1 -- 172.21.15.55:0/2550727235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad180c5cd0 con 0x7fad18159420 2023-10-06T05:05:34.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.677+0000 7fad1d59d700 1 -- 172.21.15.55:0/2550727235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad18154120 con 0x7fad18159420 2023-10-06T05:05:34.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.678+0000 7fad1d59d700 1 -- 172.21.15.55:0/2550727235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad180ccb90 con 0x7fad18159420 2023-10-06T05:05:34.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.679+0000 7facfffff700 1 -- 172.21.15.55:0/2550727235 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad0801b3b0 con 0x7fad18159420 2023-10-06T05:05:34.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.680+0000 7facfffff700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad04066f30 0x7fad040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:34.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.680+0000 7facfffff700 1 -- 172.21.15.55:0/2550727235 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6572+0+0 (secure 0 0 0) 0x7fad08083dc0 con 0x7fad18159420 2023-10-06T05:05:34.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.680+0000 7fad177fe700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad04066f30 0x7fad040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:34.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.681+0000 7fad177fe700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad04066f30 0x7fad040693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fad180cc540 tx=0x7fad0c009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:34.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.684+0000 7facfffff700 1 -- 172.21.15.55:0/2550727235 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad08050cd0 con 0x7fad18159420 2023-10-06T05:05:34.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.932+0000 7fad1d59d700 1 -- 172.21.15.55:0/2550727235 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad1804c3c0 con 0x7fad18159420 2023-10-06T05:05:34.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.932+0000 7facfffff700 1 -- 172.21.15.55:0/2550727235 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fad08017070 con 0x7fad18159420 2023-10-06T05:05:34.934 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:34.935 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":89,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7682,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/1565234675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":1565234675},{"type":"v1","addr":"172.21.15.55:6834","nonce":1565234675}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":7691,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/3042024952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":3042024952},{"type":"v1","addr":"172.21.15.163:6835","nonce":3042024952}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8003,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/1551855381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":1551855381},{"type":"v1","addr":"172.21.15.163:6834","nonce":1551855381}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":8737,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/304924610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":304924610},{"type":"v1","addr":"172.21.15.55:6835","nonce":304924610}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[]} 2023-10-06T05:05:34.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.935+0000 7fad1d59d700 1 -- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad04066f30 msgr2=0x7fad040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:34.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.935+0000 7fad1d59d700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad04066f30 0x7fad040693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fad180cc540 tx=0x7fad0c009040 comp rx=0 tx=0).stop 2023-10-06T05:05:34.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.935+0000 7fad1d59d700 1 -- 172.21.15.55:0/2550727235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad18159420 msgr2=0x7fad18153be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:34.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.935+0000 7fad1d59d700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad18159420 0x7fad18153be0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fad0800c790 tx=0x7fad08007f40 comp rx=0 tx=0).stop 2023-10-06T05:05:34.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.935+0000 7fad1d59d700 1 -- 172.21.15.55:0/2550727235 shutdown_connections 2023-10-06T05:05:34.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.935+0000 7fad1d59d700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad04066f30 0x7fad040693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.935+0000 7fad1d59d700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad1815d230 0x7fad181560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.935+0000 7fad1d59d700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad18159420 0x7fad18153be0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.935+0000 7fad1d59d700 1 --2- 172.21.15.55:0/2550727235 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad181597f0 0x7fad18155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:34.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.935+0000 7fad1d59d700 1 -- 172.21.15.55:0/2550727235 >> 172.21.15.55:0/2550727235 conn(0x7fad180d0660 msgr2=0x7fad180d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:34.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.935+0000 7fad1d59d700 1 -- 172.21.15.55:0/2550727235 shutdown_connections 2023-10-06T05:05:34.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:34.935+0000 7fad1d59d700 1 -- 172.21.15.55:0/2550727235 wait complete. 2023-10-06T05:05:34.938 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 89 2023-10-06T05:05:34.954 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:05:34.955 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:05:34.955 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:05:34.955 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:05:34.955 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:05:34.955 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:05:34.956 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:05:34.956 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:05:34.956 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:05:34.956 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:05:34.956 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:05:34.956 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:05:34.957 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:05:34.957 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:05:34.957 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:05:34.957 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:05:34.970 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:05:34.970 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:05:34.971 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:05:34.971 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:05:34.973 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:05:34.973 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:05:34.974 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:05:34.974 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:05:35.014 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:05:35.014 DEBUG:teuthology.parallel:result is None 2023-10-06T05:05:35.017 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:05:35.017 DEBUG:teuthology.parallel:result is None 2023-10-06T05:05:35.019 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:05:35.019 DEBUG:teuthology.parallel:result is None 2023-10-06T05:05:35.022 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:05:35.022 DEBUG:teuthology.parallel:result is None 2023-10-06T05:05:35.022 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:05:35.156 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-10-06T05:05:35.156 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T05:05:35.158 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2023-10-06T05:05:35.158 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T05:05:35.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.224+0000 7f209759e700 1 Processor -- start 2023-10-06T05:05:35.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.224+0000 7f209759e700 1 -- start start 2023-10-06T05:05:35.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f209759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2098157270 0x7f2098068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:35.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f209759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20980688d0 0x7f20980b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:35.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f209759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f20980b0a00 0x7f209806a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:35.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f209759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209815a980 con 0x7f2098157270 2023-10-06T05:05:35.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f209759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2098068d20 con 0x7f20980b0a00 2023-10-06T05:05:35.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f209759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209815a050 con 0x7f20980688d0 2023-10-06T05:05:35.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f209659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2098157270 0x7f2098068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:35.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f209659c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2098157270 0x7f2098068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39052/0 (socket says 172.21.15.55:39052) 2023-10-06T05:05:35.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f209659c700 1 -- 172.21.15.55:0/2503150784 learned_addr learned my addr 172.21.15.55:0/2503150784 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:35.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f2095d9b700 1 --2- 172.21.15.55:0/2503150784 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20980688d0 0x7f20980b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:35.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f2096d9d700 1 --2- 172.21.15.55:0/2503150784 >> v2:172.21.15.163:3300/0 conn(0x7f20980b0a00 0x7f209806a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:35.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f209659c700 1 -- 172.21.15.55:0/2503150784 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20980688d0 msgr2=0x7f20980b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:35.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f209659c700 1 --2- 172.21.15.55:0/2503150784 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20980688d0 0x7f20980b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:35.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f209659c700 1 -- 172.21.15.55:0/2503150784 >> v2:172.21.15.163:3300/0 conn(0x7f20980b0a00 msgr2=0x7f209806a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:35.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f209659c700 1 --2- 172.21.15.55:0/2503150784 >> v2:172.21.15.163:3300/0 conn(0x7f20980b0a00 0x7f209806a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:35.230 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f209659c700 1 -- 172.21.15.55:0/2503150784 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209815a330 con 0x7f2098157270 2023-10-06T05:05:35.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.226+0000 7f2095d9b700 1 --2- 172.21.15.55:0/2503150784 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20980688d0 0x7f20980b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:35.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.227+0000 7f209659c700 1 --2- 172.21.15.55:0/2503150784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2098157270 0x7f2098068390 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f209806ed10 tx=0x7f2088009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=927f32d76fcb5a9b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:35.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.227+0000 7f209559a700 1 -- 172.21.15.55:0/2503150784 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208800b640 con 0x7f2098157270 2023-10-06T05:05:35.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.227+0000 7f209559a700 1 -- 172.21.15.55:0/2503150784 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208800b7a0 con 0x7f2098157270 2023-10-06T05:05:35.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.227+0000 7f209559a700 1 -- 172.21.15.55:0/2503150784 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208800bb60 con 0x7f2098157270 2023-10-06T05:05:35.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.227+0000 7f209759e700 1 -- 172.21.15.55:0/2503150784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2098157270 msgr2=0x7f2098068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:35.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.227+0000 7f209759e700 1 --2- 172.21.15.55:0/2503150784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2098157270 0x7f2098068390 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f209806ed10 tx=0x7f2088009250 comp rx=0 tx=0).stop 2023-10-06T05:05:35.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.228+0000 7f209759e700 1 -- 172.21.15.55:0/2503150784 shutdown_connections 2023-10-06T05:05:35.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.228+0000 7f209759e700 1 --2- 172.21.15.55:0/2503150784 >> v2:172.21.15.163:3300/0 conn(0x7f20980b0a00 0x7f209806a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:35.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.228+0000 7f209759e700 1 --2- 172.21.15.55:0/2503150784 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2098157270 0x7f2098068390 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:35.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.228+0000 7f209759e700 1 --2- 172.21.15.55:0/2503150784 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20980688d0 0x7f20980b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:35.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.228+0000 7f209759e700 1 -- 172.21.15.55:0/2503150784 >> 172.21.15.55:0/2503150784 conn(0x7f20980cd310 msgr2=0x7f20980cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:35.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.228+0000 7f209759e700 1 -- 172.21.15.55:0/2503150784 shutdown_connections 2023-10-06T05:05:35.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.228+0000 7f209759e700 1 -- 172.21.15.55:0/2503150784 wait complete. 2023-10-06T05:05:35.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.229+0000 7f209759e700 1 Processor -- start 2023-10-06T05:05:35.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.229+0000 7f209759e700 1 -- start start 2023-10-06T05:05:35.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.230+0000 7f209759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20980688d0 0x7f2098152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:35.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.230+0000 7f209759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20980b0a00 0x7f2098151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:35.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.230+0000 7f209759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2098157270 0x7f20980aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:35.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.230+0000 7f209759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2098169d70 con 0x7f2098157270 2023-10-06T05:05:35.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.230+0000 7f209759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2098169ff0 con 0x7f20980b0a00 2023-10-06T05:05:35.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.230+0000 7f209759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2098153710 con 0x7f20980688d0 2023-10-06T05:05:35.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.230+0000 7f2096d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2098157270 0x7f20980aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:35.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.230+0000 7f2095d9b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20980b0a00 0x7f2098151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:35.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.230+0000 7f2096d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2098157270 0x7f20980aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39064/0 (socket says 172.21.15.55:39064) 2023-10-06T05:05:35.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.230+0000 7f2096d9d700 1 -- 172.21.15.55:0/900834611 learned_addr learned my addr 172.21.15.55:0/900834611 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:35.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.230+0000 7f209659c700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20980688d0 0x7f2098152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:35.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.231+0000 7f2095d9b700 1 -- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20980688d0 msgr2=0x7f2098152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:35.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.231+0000 7f2095d9b700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20980688d0 0x7f2098152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:35.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.231+0000 7f2095d9b700 1 -- 172.21.15.55:0/900834611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2098157270 msgr2=0x7f20980aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:35.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.231+0000 7f2095d9b700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2098157270 0x7f20980aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:35.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.231+0000 7f2095d9b700 1 -- 172.21.15.55:0/900834611 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c009420 con 0x7f20980b0a00 2023-10-06T05:05:35.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.231+0000 7f2096d9d700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2098157270 0x7f20980aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:35.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.231+0000 7f209659c700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20980688d0 0x7f2098152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:35.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.231+0000 7f2095d9b700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20980b0a00 0x7f2098151f60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f208c00ec80 tx=0x7f208c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:35.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.231+0000 7f20867fc700 1 -- 172.21.15.55:0/900834611 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c00c850 con 0x7f20980b0a00 2023-10-06T05:05:35.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.231+0000 7f20867fc700 1 -- 172.21.15.55:0/900834611 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208c00c9b0 con 0x7f20980b0a00 2023-10-06T05:05:35.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.231+0000 7f20867fc700 1 -- 172.21.15.55:0/900834611 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c00cd70 con 0x7f20980b0a00 2023-10-06T05:05:35.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.231+0000 7f209759e700 1 -- 172.21.15.55:0/900834611 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20880094f0 con 0x7f20980b0a00 2023-10-06T05:05:35.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.232+0000 7f209759e700 1 -- 172.21.15.55:0/900834611 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2098153cd0 con 0x7f20980b0a00 2023-10-06T05:05:35.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.232+0000 7f209759e700 1 -- 172.21.15.55:0/900834611 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209816a1a0 con 0x7f20980b0a00 2023-10-06T05:05:35.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.234+0000 7f20867fc700 1 -- 172.21.15.55:0/900834611 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f208c013680 con 0x7f20980b0a00 2023-10-06T05:05:35.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.235+0000 7f20867fc700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2078066f30 0x7f20780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:35.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.235+0000 7f20867fc700 1 -- 172.21.15.55:0/900834611 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6572+0+0 (secure 0 0 0) 0x7f208c084b00 con 0x7f20980b0a00 2023-10-06T05:05:35.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.235+0000 7f209659c700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2078066f30 0x7f20780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:35.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.236+0000 7f209659c700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2078066f30 0x7f20780693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f209806ed10 tx=0x7f2088017000 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:35.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.239+0000 7f20867fc700 1 -- 172.21.15.55:0/900834611 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f209816a1a0 con 0x7f20980b0a00 2023-10-06T05:05:35.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.457+0000 7f209759e700 1 -- 172.21.15.55:0/900834611 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f209816a130 con 0x7f20980b0a00 2023-10-06T05:05:35.893 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.891+0000 7f20867fc700 1 -- 172.21.15.55:0/900834611 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v187) v1 ==== 97+0+0 (secure 0 0 0) 0x7f208c086e60 con 0x7f20980b0a00 2023-10-06T05:05:35.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.894+0000 7f209759e700 1 -- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2078066f30 msgr2=0x7f20780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:35.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.894+0000 7f209759e700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2078066f30 0x7f20780693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f209806ed10 tx=0x7f2088017000 comp rx=0 tx=0).stop 2023-10-06T05:05:35.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.894+0000 7f209759e700 1 -- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20980b0a00 msgr2=0x7f2098151f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:35.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.894+0000 7f209759e700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20980b0a00 0x7f2098151f60 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f208c00ec80 tx=0x7f208c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:35.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.894+0000 7f209759e700 1 -- 172.21.15.55:0/900834611 shutdown_connections 2023-10-06T05:05:35.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.894+0000 7f209759e700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2078066f30 0x7f20780693e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:35.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.894+0000 7f209759e700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2098157270 0x7f20980aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:35.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.894+0000 7f209759e700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f20980688d0 0x7f2098152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:35.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.894+0000 7f209759e700 1 --2- 172.21.15.55:0/900834611 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f20980b0a00 0x7f2098151f60 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:35.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.894+0000 7f209759e700 1 -- 172.21.15.55:0/900834611 >> 172.21.15.55:0/900834611 conn(0x7f20980cd310 msgr2=0x7f20980ac200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:35.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.895+0000 7f209759e700 1 -- 172.21.15.55:0/900834611 shutdown_connections 2023-10-06T05:05:35.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:35.895+0000 7f209759e700 1 -- 172.21.15.55:0/900834611 wait complete. 2023-10-06T05:05:35.898 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2023-10-06T05:05:35.917 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:05:36.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.104+0000 7f606bfff700 1 Processor -- start 2023-10-06T05:05:36.106 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.104+0000 7f606bfff700 1 -- start start 2023-10-06T05:05:36.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.106+0000 7f606bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f606c0ae710 0x7f606c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.106+0000 7f606bfff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f606c0ca860 con 0x7f606c156310 2023-10-06T05:05:36.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.106+0000 7f606bfff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f606c0cab40 con 0x7f606c1566e0 2023-10-06T05:05:36.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.106+0000 7f606bfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f606c0cae20 con 0x7f606c0ae710 2023-10-06T05:05:36.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.106+0000 7f606affd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f606c156310 0x7f606c0c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46014/0 (socket says 172.21.15.55:46014) 2023-10-06T05:05:36.108 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.106+0000 7f606affd700 1 -- 172.21.15.55:0/3029148426 learned_addr learned my addr 172.21.15.55:0/3029148426 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:36.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.106+0000 7f6069ffb700 1 -- 172.21.15.55:0/3029148426 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286560058 0 0) 0x7f606c0ca860 con 0x7f606c156310 2023-10-06T05:05:36.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.106+0000 7f606b7fe700 1 --2- 172.21.15.55:0/3029148426 >> v2:172.21.15.163:3300/0 conn(0x7f606c0ae710 0x7f606c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.106+0000 7f6069ffb700 1 -- 172.21.15.55:0/3029148426 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6054003570 con 0x7f606c156310 2023-10-06T05:05:36.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f6069ffb700 1 -- 172.21.15.55:0/3029148426 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3052120758 0 0) 0x7f6054003570 con 0x7f606c156310 2023-10-06T05:05:36.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f6069ffb700 1 -- 172.21.15.55:0/3029148426 >> v2:172.21.15.163:3300/0 conn(0x7f606c0ae710 msgr2=0x7f606c0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:36.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f6069ffb700 1 --2- 172.21.15.55:0/3029148426 >> v2:172.21.15.163:3300/0 conn(0x7f606c0ae710 0x7f606c0c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f6069ffb700 1 -- 172.21.15.55:0/3029148426 >> v1:172.21.15.163:6789/0 conn(0x7f606c1566e0 legacy=0x7f606c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:36.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f6069ffb700 1 -- 172.21.15.55:0/3029148426 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c0cb100 con 0x7f606c156310 2023-10-06T05:05:36.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f6069ffb700 1 -- 172.21.15.55:0/3029148426 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f605c001470 con 0x7f606c156310 2023-10-06T05:05:36.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f6069ffb700 1 -- 172.21.15.55:0/3029148426 >> v1:172.21.15.55:6789/0 conn(0x7f606c156310 legacy=0x7f606c0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:36.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f6069ffb700 1 --2- 172.21.15.55:0/3029148426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6054003a60 0x7f6054003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f6069ffb700 1 --2- 172.21.15.55:0/3029148426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6054004810 0x7f6054004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.112 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f6069ffb700 1 --2- 172.21.15.55:0/3029148426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60540051c0 0x7f605400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f6069ffb700 1 -- 172.21.15.55:0/3029148426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c0cae20 con 0x7f60540051c0 2023-10-06T05:05:36.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f6069ffb700 1 -- 172.21.15.55:0/3029148426 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605400c0d0 con 0x7f6054004810 2023-10-06T05:05:36.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f6069ffb700 1 -- 172.21.15.55:0/3029148426 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605400c290 con 0x7f6054003a60 2023-10-06T05:05:36.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f606b7fe700 1 --2- 172.21.15.55:0/3029148426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60540051c0 0x7f605400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f606affd700 1 --2- 172.21.15.55:0/3029148426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6054003a60 0x7f6054003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.107+0000 7f606a7fc700 1 --2- 172.21.15.55:0/3029148426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6054004810 0x7f6054004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.108+0000 7f606b7fe700 1 -- 172.21.15.55:0/3029148426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6054003a60 msgr2=0x7f6054003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:36.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.108+0000 7f606b7fe700 1 --2- 172.21.15.55:0/3029148426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6054003a60 0x7f6054003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.108+0000 7f606b7fe700 1 -- 172.21.15.55:0/3029148426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6054004810 msgr2=0x7f6054004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:36.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.108+0000 7f606b7fe700 1 --2- 172.21.15.55:0/3029148426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6054004810 0x7f6054004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.108+0000 7f606b7fe700 1 -- 172.21.15.55:0/3029148426 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6054003570 con 0x7f60540051c0 2023-10-06T05:05:36.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.108+0000 7f606affd700 1 --2- 172.21.15.55:0/3029148426 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6054003a60 0x7f6054003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:36.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.108+0000 7f606a7fc700 1 --2- 172.21.15.55:0/3029148426 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6054004810 0x7f6054004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:36.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.108+0000 7f606b7fe700 1 -- 172.21.15.55:0/3029148426 reap_dead start 2023-10-06T05:05:36.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.108+0000 7f606b7fe700 1 --2- 172.21.15.55:0/3029148426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60540051c0 0x7f605400ba60 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f606c0aeb10 tx=0x7f606400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f10d47337e4f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:36.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.108+0000 7f6069ffb700 1 -- 172.21.15.55:0/3029148426 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400ea90 con 0x7f60540051c0 2023-10-06T05:05:36.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.108+0000 7f6069ffb700 1 -- 172.21.15.55:0/3029148426 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606400f040 con 0x7f60540051c0 2023-10-06T05:05:36.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.108+0000 7f6069ffb700 1 -- 172.21.15.55:0/3029148426 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400ee40 con 0x7f60540051c0 2023-10-06T05:05:36.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.109+0000 7f606bfff700 1 -- 172.21.15.55:0/3029148426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60540051c0 msgr2=0x7f605400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.109+0000 7f606bfff700 1 --2- 172.21.15.55:0/3029148426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60540051c0 0x7f605400ba60 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f606c0aeb10 tx=0x7f606400b040 comp rx=0 tx=0).stop 2023-10-06T05:05:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.109+0000 7f606bfff700 1 -- 172.21.15.55:0/3029148426 shutdown_connections 2023-10-06T05:05:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.109+0000 7f606bfff700 1 --2- 172.21.15.55:0/3029148426 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f60540051c0 0x7f605400ba60 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.109+0000 7f606bfff700 1 -- 172.21.15.55:0/3029148426 >> 172.21.15.55:0/3029148426 conn(0x7f606c0cc1f0 msgr2=0x7f606c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.109+0000 7f606bfff700 1 -- 172.21.15.55:0/3029148426 shutdown_connections 2023-10-06T05:05:36.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.109+0000 7f606bfff700 1 -- 172.21.15.55:0/3029148426 wait complete. 2023-10-06T05:05:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.110+0000 7f606bfff700 1 Processor -- start 2023-10-06T05:05:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.110+0000 7f606bfff700 1 -- start start 2023-10-06T05:05:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.111+0000 7f606bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60540051c0 0x7f606c15b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.111+0000 7f606bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f606c15d250 0x7f606c0b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.111+0000 7f606bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f606c15dde0 0x7f606c15e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.111+0000 7f606bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c15e7c0 con 0x7f606c15dde0 2023-10-06T05:05:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.111+0000 7f606bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c16bf40 con 0x7f606c15d250 2023-10-06T05:05:36.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.111+0000 7f606bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606c16b330 con 0x7f60540051c0 2023-10-06T05:05:36.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.112+0000 7f606b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f606c15dde0 0x7f606c15e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.112+0000 7f606b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f606c15dde0 0x7f606c15e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39074/0 (socket says 172.21.15.55:39074) 2023-10-06T05:05:36.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.112+0000 7f606b7fe700 1 -- 172.21.15.55:0/2298296993 learned_addr learned my addr 172.21.15.55:0/2298296993 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:36.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.112+0000 7f606affd700 1 --2- 172.21.15.55:0/2298296993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60540051c0 0x7f606c15b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.112+0000 7f606a7fc700 1 --2- 172.21.15.55:0/2298296993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f606c15d250 0x7f606c0b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.112+0000 7f606b7fe700 1 -- 172.21.15.55:0/2298296993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60540051c0 msgr2=0x7f606c15b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:36.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.112+0000 7f606b7fe700 1 --2- 172.21.15.55:0/2298296993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60540051c0 0x7f606c15b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.112+0000 7f606b7fe700 1 -- 172.21.15.55:0/2298296993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f606c15d250 msgr2=0x7f606c0b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:36.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.112+0000 7f606b7fe700 1 --2- 172.21.15.55:0/2298296993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f606c15d250 0x7f606c0b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.112+0000 7f606b7fe700 1 -- 172.21.15.55:0/2298296993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c16b610 con 0x7f606c15dde0 2023-10-06T05:05:36.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.112+0000 7f606b7fe700 1 --2- 172.21.15.55:0/2298296993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f606c15dde0 0x7f606c15e280 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f606400afd0 tx=0x7f6064019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:36.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.113+0000 7f6052ffd700 1 -- 172.21.15.55:0/2298296993 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606401d660 con 0x7f606c15dde0 2023-10-06T05:05:36.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.113+0000 7f6052ffd700 1 -- 172.21.15.55:0/2298296993 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6064022070 con 0x7f606c15dde0 2023-10-06T05:05:36.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.113+0000 7f6052ffd700 1 -- 172.21.15.55:0/2298296993 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606401da10 con 0x7f606c15dde0 2023-10-06T05:05:36.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.113+0000 7f606bfff700 1 -- 172.21.15.55:0/2298296993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606c16b8f0 con 0x7f606c15dde0 2023-10-06T05:05:36.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.113+0000 7f606bfff700 1 -- 172.21.15.55:0/2298296993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606c16af50 con 0x7f606c15dde0 2023-10-06T05:05:36.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.114+0000 7f606bfff700 1 -- 172.21.15.55:0/2298296993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f606c16c0f0 con 0x7f606c15dde0 2023-10-06T05:05:36.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.115+0000 7f6052ffd700 1 -- 172.21.15.55:0/2298296993 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6064003020 con 0x7f606c15dde0 2023-10-06T05:05:36.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.116+0000 7f6052ffd700 1 --2- 172.21.15.55:0/2298296993 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6058066f30 0x7f60580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.116+0000 7f6052ffd700 1 -- 172.21.15.55:0/2298296993 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5626+0+0 (secure 0 0 0) 0x7f6064024070 con 0x7f606c15dde0 2023-10-06T05:05:36.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.116+0000 7f606affd700 1 --2- 172.21.15.55:0/2298296993 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6058066f30 0x7f60580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.117+0000 7f606affd700 1 --2- 172.21.15.55:0/2298296993 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6058066f30 0x7f60580693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f605c007cd0 tx=0x7f606c0caa30 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:36.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.120+0000 7f6052ffd700 1 -- 172.21.15.55:0/2298296993 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f606c16c0f0 con 0x7f606c15dde0 2023-10-06T05:05:36.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.404+0000 7f606bfff700 1 -- 172.21.15.55:0/2298296993 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f606c16c0b0 con 0x7f606c15dde0 2023-10-06T05:05:36.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.405+0000 7f6052ffd700 1 -- 172.21.15.55:0/2298296993 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v45) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6064053bf0 con 0x7f606c15dde0 2023-10-06T05:05:36.407 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:36.407 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T05:05:36.407 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2023-10-06T05:05:36.407 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.407 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2023-10-06T05:05:36.408 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9li2RaGRAAgRjtbw3h40vylEPjolzlpw==", 2023-10-06T05:05:36.408 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.408 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:05:36.408 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T05:05:36.408 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T05:05:36.408 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.408 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.409 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.409 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.409 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2023-10-06T05:05:36.409 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lXZu7LxAAu3QAtdDxtD/AuuqGG12sDw==", 2023-10-06T05:05:36.409 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.409 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:05:36.409 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T05:05:36.409 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T05:05:36.409 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.409 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.409 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.410 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.410 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2023-10-06T05:05:36.410 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lS22fJBAAZi6CfyLyJAFgIkaarad1mA==", 2023-10-06T05:05:36.410 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.410 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:05:36.410 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T05:05:36.410 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T05:05:36.410 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.410 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.410 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.411 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.411 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2023-10-06T05:05:36.411 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9ll0RIOhAAGtGdPv4Y2bge7xIv3eu2NQ==", 2023-10-06T05:05:36.411 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.411 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:05:36.411 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T05:05:36.411 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T05:05:36.411 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.411 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.411 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.412 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.412 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2023-10-06T05:05:36.412 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDHfx9lmTeaDRAAuycc9i7fPj8xrDkuZCAGVg==", 2023-10-06T05:05:36.412 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.412 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:05:36.412 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:05:36.412 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.412 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.412 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.412 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.413 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2023-10-06T05:05:36.413 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDKfx9lxOdXEhAA9S8tdOvhnmBeEbutqnUsZg==", 2023-10-06T05:05:36.413 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.413 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:05:36.413 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:05:36.413 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.413 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.414 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.414 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.414 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2023-10-06T05:05:36.414 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDNfx9l76I+GRAAjaj0gM7An9BCyGmfQgaDFw==", 2023-10-06T05:05:36.414 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.414 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:05:36.414 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:05:36.414 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.414 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.414 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.414 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.415 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2023-10-06T05:05:36.415 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDQfx9lsPK+HxAAzQCV0EuBenqOkUpiQW9uog==", 2023-10-06T05:05:36.415 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.415 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:05:36.415 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:05:36.415 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.415 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.415 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.416 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.416 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2023-10-06T05:05:36.416 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDWfx9lSHNrMxAAIn7awDmljO2cRtKyDalWnw==", 2023-10-06T05:05:36.416 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.416 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:05:36.416 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:05:36.416 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.416 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.416 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.416 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.416 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2023-10-06T05:05:36.417 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDZfx9lFoWuOBAAhzQOt3YChvuv0MhrspOjPw==", 2023-10-06T05:05:36.417 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.417 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:05:36.417 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:05:36.417 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.417 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.417 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.417 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.417 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2023-10-06T05:05:36.417 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDcfx9lB8iSOhAAwamSgGIXZ8owFE4FdbOOBw==", 2023-10-06T05:05:36.418 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.418 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:05:36.418 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:05:36.418 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.418 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.418 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.418 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.418 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2023-10-06T05:05:36.418 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDgfx9lUb3mABAAJ75UO2UQg2r+l+tDYVn0Dg==", 2023-10-06T05:05:36.418 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.418 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:05:36.419 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:05:36.419 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.419 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.419 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.419 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.419 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2023-10-06T05:05:36.419 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lfNvCBxAA1ZJq4XsWTZHVY0u0NdOqIw==", 2023-10-06T05:05:36.419 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.419 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:05:36.419 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2023-10-06T05:05:36.419 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:05:36.420 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.420 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.420 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.420 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2023-10-06T05:05:36.420 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lnvkoFhAAafxsOmItxmwUr1GJrGYJ6g==", 2023-10-06T05:05:36.420 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.420 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:05:36.420 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T05:05:36.421 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T05:05:36.421 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T05:05:36.421 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.421 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.421 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.422 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2023-10-06T05:05:36.422 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9l+9b4DhAAc5XHdikWbXsEblSE3FGScQ==", 2023-10-06T05:05:36.422 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.422 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:05:36.422 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T05:05:36.422 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T05:05:36.423 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T05:05:36.423 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.423 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.423 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.424 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2023-10-06T05:05:36.425 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9luSXrHBAAg6Ufa1Ep/EOgsQtuOPQgYg==", 2023-10-06T05:05:36.425 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.425 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:05:36.425 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T05:05:36.425 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T05:05:36.425 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T05:05:36.425 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.425 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.425 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.425 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2023-10-06T05:05:36.425 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC+fx9lEb50FxAAt0Oq28gsXtQvol7TV8trgg==", 2023-10-06T05:05:36.426 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.426 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T05:05:36.426 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T05:05:36.426 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T05:05:36.426 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.426 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.426 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.426 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.426 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2023-10-06T05:05:36.426 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lQAANEhAAsNiWPqCNEyUS3IELXSC1Cw==", 2023-10-06T05:05:36.426 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.427 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T05:05:36.427 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T05:05:36.427 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.427 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.427 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:36.427 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:36.427 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2023-10-06T05:05:36.427 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lCwU2ABAAMijxXd8hwgk3XzcdP4NZyg==", 2023-10-06T05:05:36.427 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:05:36.427 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T05:05:36.427 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T05:05:36.427 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:05:36.428 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.428 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:36.428 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2023-10-06T05:05:36.428 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T05:05:36.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.407+0000 7f606bfff700 1 -- 172.21.15.55:0/2298296993 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6058066f30 msgr2=0x7f60580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:36.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.407+0000 7f606bfff700 1 --2- 172.21.15.55:0/2298296993 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6058066f30 0x7f60580693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f605c007cd0 tx=0x7f606c0caa30 comp rx=0 tx=0).stop 2023-10-06T05:05:36.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.407+0000 7f606bfff700 1 -- 172.21.15.55:0/2298296993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f606c15dde0 msgr2=0x7f606c15e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:36.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.407+0000 7f606bfff700 1 --2- 172.21.15.55:0/2298296993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f606c15dde0 0x7f606c15e280 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f606400afd0 tx=0x7f6064019000 comp rx=0 tx=0).stop 2023-10-06T05:05:36.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.407+0000 7f606bfff700 1 -- 172.21.15.55:0/2298296993 shutdown_connections 2023-10-06T05:05:36.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.407+0000 7f606bfff700 1 --2- 172.21.15.55:0/2298296993 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6058066f30 0x7f60580693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.407+0000 7f606bfff700 1 --2- 172.21.15.55:0/2298296993 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f606c15dde0 0x7f606c15e280 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.407+0000 7f606bfff700 1 --2- 172.21.15.55:0/2298296993 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f60540051c0 0x7f606c15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.407+0000 7f606bfff700 1 --2- 172.21.15.55:0/2298296993 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f606c15d250 0x7f606c0b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.407+0000 7f606bfff700 1 -- 172.21.15.55:0/2298296993 >> 172.21.15.55:0/2298296993 conn(0x7f606c0cc1f0 msgr2=0x7f606c0c9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:36.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.408+0000 7f606bfff700 1 -- 172.21.15.55:0/2298296993 shutdown_connections 2023-10-06T05:05:36.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.408+0000 7f606bfff700 1 -- 172.21.15.55:0/2298296993 wait complete. 2023-10-06T05:05:36.431 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:05:36.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.615+0000 7f964899d700 1 Processor -- start 2023-10-06T05:05:36.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.616+0000 7f964899d700 1 -- start start 2023-10-06T05:05:36.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.617+0000 7f964899d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96440c8830 0x7f9644164800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.617+0000 7f964899d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96440b1410 con 0x7f96440c6c60 2023-10-06T05:05:36.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.617+0000 7f964899d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9644063230 con 0x7f96441595a0 2023-10-06T05:05:36.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.617+0000 7f964899d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96440b2b90 con 0x7f96440c8830 2023-10-06T05:05:36.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.617+0000 7f9642ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f96440c6c60 0x7f96440c8120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46016/0 (socket says 172.21.15.55:46016) 2023-10-06T05:05:36.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.617+0000 7f9642ffd700 1 -- 172.21.15.55:0/319118447 learned_addr learned my addr 172.21.15.55:0/319118447 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:36.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.617+0000 7f96427fc700 1 -- 172.21.15.55:0/319118447 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 40111069 0 0) 0x7f96440b1410 con 0x7f96440c6c60 2023-10-06T05:05:36.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.617+0000 7f96427fc700 1 -- 172.21.15.55:0/319118447 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f962c003570 con 0x7f96440c6c60 2023-10-06T05:05:36.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.617+0000 7f9643fff700 1 --2- 172.21.15.55:0/319118447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96440c8830 0x7f9644164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f96427fc700 1 -- 172.21.15.55:0/319118447 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3406451167 0 0) 0x7f962c003570 con 0x7f96440c6c60 2023-10-06T05:05:36.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f96427fc700 1 -- 172.21.15.55:0/319118447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96440c8830 msgr2=0x7f9644164800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:36.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f96427fc700 1 --2- 172.21.15.55:0/319118447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96440c8830 0x7f9644164800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f96427fc700 1 -- 172.21.15.55:0/319118447 >> v1:172.21.15.163:6789/0 conn(0x7f96441595a0 legacy=0x7f96440b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:36.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f96427fc700 1 -- 172.21.15.55:0/319118447 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96440634f0 con 0x7f96440c6c60 2023-10-06T05:05:36.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f96427fc700 1 -- 172.21.15.55:0/319118447 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f9638001470 con 0x7f96440c6c60 2023-10-06T05:05:36.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f96427fc700 1 -- 172.21.15.55:0/319118447 >> v1:172.21.15.55:6789/0 conn(0x7f96440c6c60 legacy=0x7f96440c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:36.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f96427fc700 1 --2- 172.21.15.55:0/319118447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96440c8830 0x7f962c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f96427fc700 1 --2- 172.21.15.55:0/319118447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f962c003fd0 0x7f962c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f96427fc700 1 --2- 172.21.15.55:0/319118447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c004d50 0x7f962c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f96427fc700 1 -- 172.21.15.55:0/319118447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96440b2b90 con 0x7f962c004d50 2023-10-06T05:05:36.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f96427fc700 1 -- 172.21.15.55:0/319118447 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c00bb40 con 0x7f962c003fd0 2023-10-06T05:05:36.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f96427fc700 1 -- 172.21.15.55:0/319118447 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c00bd00 con 0x7f96440c8830 2023-10-06T05:05:36.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f9643fff700 1 --2- 172.21.15.55:0/319118447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c004d50 0x7f962c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f96437fe700 1 --2- 172.21.15.55:0/319118447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96440c8830 0x7f962c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.618+0000 7f9642ffd700 1 --2- 172.21.15.55:0/319118447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f962c003fd0 0x7f962c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.619+0000 7f9643fff700 1 -- 172.21.15.55:0/319118447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96440c8830 msgr2=0x7f962c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:36.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.619+0000 7f9643fff700 1 --2- 172.21.15.55:0/319118447 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f96440c8830 0x7f962c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.619+0000 7f9643fff700 1 -- 172.21.15.55:0/319118447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f962c003fd0 msgr2=0x7f962c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:36.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.619+0000 7f9643fff700 1 --2- 172.21.15.55:0/319118447 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f962c003fd0 0x7f962c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.619+0000 7f9643fff700 1 -- 172.21.15.55:0/319118447 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c003570 con 0x7f962c004d50 2023-10-06T05:05:36.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.619+0000 7f9643fff700 1 --2- 172.21.15.55:0/319118447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c004d50 0x7f962c0051c0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f96440c9510 tx=0x7f963c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638a47ca88b70e62 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:36.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.619+0000 7f96427fc700 1 -- 172.21.15.55:0/319118447 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c00ea60 con 0x7f962c004d50 2023-10-06T05:05:36.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.619+0000 7f96427fc700 1 -- 172.21.15.55:0/319118447 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963c00f040 con 0x7f962c004d50 2023-10-06T05:05:36.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.619+0000 7f96427fc700 1 -- 172.21.15.55:0/319118447 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c00ee10 con 0x7f962c004d50 2023-10-06T05:05:36.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.620+0000 7f964899d700 1 -- 172.21.15.55:0/319118447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c004d50 msgr2=0x7f962c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:36.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.620+0000 7f964899d700 1 --2- 172.21.15.55:0/319118447 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c004d50 0x7f962c0051c0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f96440c9510 tx=0x7f963c00b040 comp rx=0 tx=0).stop 2023-10-06T05:05:36.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.620+0000 7f9643fff700 1 -- 172.21.15.55:0/319118447 reap_dead start 2023-10-06T05:05:36.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.620+0000 7f964899d700 1 -- 172.21.15.55:0/319118447 shutdown_connections 2023-10-06T05:05:36.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.620+0000 7f964899d700 1 -- 172.21.15.55:0/319118447 >> 172.21.15.55:0/319118447 conn(0x7f964406ddb0 msgr2=0x7f964406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:36.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.620+0000 7f964899d700 1 -- 172.21.15.55:0/319118447 shutdown_connections 2023-10-06T05:05:36.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.620+0000 7f964899d700 1 -- 172.21.15.55:0/319118447 wait complete. 2023-10-06T05:05:36.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.621+0000 7f964899d700 1 Processor -- start 2023-10-06T05:05:36.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.622+0000 7f964899d700 1 -- start start 2023-10-06T05:05:36.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.622+0000 7f964899d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c004d50 0x7f9644152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.622+0000 7f964899d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9644154b00 0x7f964414dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.622+0000 7f964899d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96440c5ae0 0x7f96440c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.622+0000 7f964899d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96441535c0 con 0x7f962c004d50 2023-10-06T05:05:36.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.622+0000 7f964899d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964414e520 con 0x7f96440c5ae0 2023-10-06T05:05:36.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.622+0000 7f964899d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9644154f70 con 0x7f9644154b00 2023-10-06T05:05:36.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.623+0000 7f96437fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c004d50 0x7f9644152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.623+0000 7f96437fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c004d50 0x7f9644152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39092/0 (socket says 172.21.15.55:39092) 2023-10-06T05:05:36.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.623+0000 7f96437fe700 1 -- 172.21.15.55:0/1400492406 learned_addr learned my addr 172.21.15.55:0/1400492406 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:36.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.623+0000 7f9642ffd700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9644154b00 0x7f964414dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.623+0000 7f9643fff700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96440c5ae0 0x7f96440c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.623+0000 7f96437fe700 1 -- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9644154b00 msgr2=0x7f964414dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:36.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.623+0000 7f96437fe700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9644154b00 0x7f964414dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.623+0000 7f96437fe700 1 -- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96440c5ae0 msgr2=0x7f96440c5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:36.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.623+0000 7f96437fe700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96440c5ae0 0x7f96440c5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.623+0000 7f96437fe700 1 -- 172.21.15.55:0/1400492406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9644152d20 con 0x7f962c004d50 2023-10-06T05:05:36.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.623+0000 7f9642ffd700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9644154b00 0x7f964414dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:36.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.623+0000 7f9643fff700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96440c5ae0 0x7f96440c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:36.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.623+0000 7f96437fe700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c004d50 0x7f9644152640 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f96340029e0 tx=0x7f9634007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:36.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.624+0000 7f962b7fe700 1 -- 172.21.15.55:0/1400492406 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963400f8d0 con 0x7f962c004d50 2023-10-06T05:05:36.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.624+0000 7f962b7fe700 1 -- 172.21.15.55:0/1400492406 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963400fa50 con 0x7f962c004d50 2023-10-06T05:05:36.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.624+0000 7f962b7fe700 1 -- 172.21.15.55:0/1400492406 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963400fe10 con 0x7f962c004d50 2023-10-06T05:05:36.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.624+0000 7f964899d700 1 -- 172.21.15.55:0/1400492406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9644153000 con 0x7f962c004d50 2023-10-06T05:05:36.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.624+0000 7f964899d700 1 -- 172.21.15.55:0/1400492406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96440c1a80 con 0x7f962c004d50 2023-10-06T05:05:36.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.625+0000 7f964899d700 1 -- 172.21.15.55:0/1400492406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96440c1ca0 con 0x7f962c004d50 2023-10-06T05:05:36.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.626+0000 7f962b7fe700 1 -- 172.21.15.55:0/1400492406 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9634013600 con 0x7f962c004d50 2023-10-06T05:05:36.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.626+0000 7f962b7fe700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9630066f60 0x7f9630069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:36.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.627+0000 7f962b7fe700 1 -- 172.21.15.55:0/1400492406 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5626+0+0 (secure 0 0 0) 0x7f9634084b00 con 0x7f962c004d50 2023-10-06T05:05:36.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.627+0000 7f9642ffd700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9630066f60 0x7f9630069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:36.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.628+0000 7f9642ffd700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9630066f60 0x7f9630069410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f96380010e0 tx=0x7f9638009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:36.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.630+0000 7f962b7fe700 1 -- 172.21.15.55:0/1400492406 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9634086e60 con 0x7f962c004d50 2023-10-06T05:05:36.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.835+0000 7f964899d700 1 -- 172.21.15.55:0/1400492406 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f964404c3c0 con 0x7f962c004d50 2023-10-06T05:05:36.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.836+0000 7f962b7fe700 1 -- 172.21.15.55:0/1400492406 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+14349 (secure 0 0 0) 0x7f9634017090 con 0x7f962c004d50 2023-10-06T05:05:36.838 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:36.838 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":189,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:05:35.895780+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:05:36.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.838+0000 7f964899d700 1 -- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9630066f60 msgr2=0x7f9630069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:36.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.838+0000 7f964899d700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9630066f60 0x7f9630069410 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f96380010e0 tx=0x7f9638009040 comp rx=0 tx=0).stop 2023-10-06T05:05:36.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.838+0000 7f964899d700 1 -- 172.21.15.55:0/1400492406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c004d50 msgr2=0x7f9644152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:36.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.838+0000 7f964899d700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c004d50 0x7f9644152640 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f96340029e0 tx=0x7f9634007d10 comp rx=0 tx=0).stop 2023-10-06T05:05:36.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.838+0000 7f964899d700 1 -- 172.21.15.55:0/1400492406 shutdown_connections 2023-10-06T05:05:36.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.838+0000 7f964899d700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9630066f60 0x7f9630069410 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.838+0000 7f964899d700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f96440c5ae0 0x7f96440c5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.838+0000 7f964899d700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f962c004d50 0x7f9644152640 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.838+0000 7f964899d700 1 --2- 172.21.15.55:0/1400492406 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9644154b00 0x7f964414dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:36.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.838+0000 7f964899d700 1 -- 172.21.15.55:0/1400492406 >> 172.21.15.55:0/1400492406 conn(0x7f964406ddb0 msgr2=0x7f96440cb440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:36.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.839+0000 7f964899d700 1 -- 172.21.15.55:0/1400492406 shutdown_connections 2023-10-06T05:05:36.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:36.839+0000 7f964899d700 1 -- 172.21.15.55:0/1400492406 wait complete. 2023-10-06T05:05:36.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:05:36.859 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:05:37.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.050+0000 7fae3e19d700 1 Processor -- start 2023-10-06T05:05:37.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.050+0000 7fae3e19d700 1 -- start start 2023-10-06T05:05:37.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.051+0000 7fae3e19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fae380cb230 0x7fae380cb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.051+0000 7fae3e19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae380cbb90 con 0x7fae380ccc00 2023-10-06T05:05:37.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.051+0000 7fae3e19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae380c9f30 con 0x7fae38159a60 2023-10-06T05:05:37.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.051+0000 7fae3e19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3806dfc0 con 0x7fae380cb230 2023-10-06T05:05:37.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.051+0000 7fae3d99c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fae380cb230 0x7fae380cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.051+0000 7fae3c99a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fae380ccc00 0x7fae38069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46032/0 (socket says 172.21.15.55:46032) 2023-10-06T05:05:37.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.051+0000 7fae3d99c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fae380cb230 0x7fae380cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39102/0 (socket says 172.21.15.55:39102) 2023-10-06T05:05:37.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.051+0000 7fae3c99a700 1 -- 172.21.15.55:0/4133663416 learned_addr learned my addr 172.21.15.55:0/4133663416 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:37.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.052+0000 7fae3d99c700 1 -- 172.21.15.55:0/4133663416 >> v1:172.21.15.163:6789/0 conn(0x7fae38159a60 legacy=0x7fae380c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.052+0000 7fae3d99c700 1 -- 172.21.15.55:0/4133663416 >> v1:172.21.15.55:6789/0 conn(0x7fae380ccc00 legacy=0x7fae38069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.052+0000 7fae3d99c700 1 -- 172.21.15.55:0/4133663416 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae380ca470 con 0x7fae380cb230 2023-10-06T05:05:37.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.052+0000 7fae2ffff700 1 -- 172.21.15.55:0/4133663416 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106436599 0 0) 0x7fae380cbb90 con 0x7fae380ccc00 2023-10-06T05:05:37.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.052+0000 7fae3d99c700 1 --2- 172.21.15.55:0/4133663416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae380cb230 0x7fae380cb650 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fae30009e60 tx=0x7fae30009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6df908083be3cf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:37.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.052+0000 7fae2ffff700 1 -- 172.21.15.55:0/4133663416 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3000e070 con 0x7fae380cb230 2023-10-06T05:05:37.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.052+0000 7fae2ffff700 1 -- 172.21.15.55:0/4133663416 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3000bcc0 con 0x7fae380cb230 2023-10-06T05:05:37.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.052+0000 7fae2ffff700 1 -- 172.21.15.55:0/4133663416 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30012680 con 0x7fae380cb230 2023-10-06T05:05:37.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.053+0000 7fae3e19d700 1 -- 172.21.15.55:0/4133663416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae380cb230 msgr2=0x7fae380cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.053+0000 7fae3e19d700 1 --2- 172.21.15.55:0/4133663416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae380cb230 0x7fae380cb650 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fae30009e60 tx=0x7fae30009780 comp rx=0 tx=0).stop 2023-10-06T05:05:37.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.053+0000 7fae3e19d700 1 -- 172.21.15.55:0/4133663416 shutdown_connections 2023-10-06T05:05:37.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.053+0000 7fae3e19d700 1 --2- 172.21.15.55:0/4133663416 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae380cb230 0x7fae380cb650 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.053+0000 7fae3e19d700 1 -- 172.21.15.55:0/4133663416 >> 172.21.15.55:0/4133663416 conn(0x7fae3806baa0 msgr2=0x7fae3806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:37.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.053+0000 7fae3e19d700 1 -- 172.21.15.55:0/4133663416 shutdown_connections 2023-10-06T05:05:37.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.053+0000 7fae3e19d700 1 -- 172.21.15.55:0/4133663416 wait complete. 2023-10-06T05:05:37.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.054+0000 7fae3e19d700 1 Processor -- start 2023-10-06T05:05:37.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.055+0000 7fae3e19d700 1 -- start start 2023-10-06T05:05:37.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3e19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae380cb230 0x7fae381522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3e19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380ccc00 0x7fae380c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3e19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38159a60 0x7fae380c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3e19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae38152830 con 0x7fae38159a60 2023-10-06T05:05:37.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3e19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae380c7830 con 0x7fae380cb230 2023-10-06T05:05:37.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3e19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae380c7ad0 con 0x7fae380ccc00 2023-10-06T05:05:37.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3d99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38159a60 0x7fae380c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3d99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38159a60 0x7fae380c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39106/0 (socket says 172.21.15.55:39106) 2023-10-06T05:05:37.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3d99c700 1 -- 172.21.15.55:0/21033804 learned_addr learned my addr 172.21.15.55:0/21033804 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:37.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3d19b700 1 --2- 172.21.15.55:0/21033804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae380cb230 0x7fae381522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3c99a700 1 --2- 172.21.15.55:0/21033804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380ccc00 0x7fae380c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3d99c700 1 -- 172.21.15.55:0/21033804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380ccc00 msgr2=0x7fae380c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3d99c700 1 --2- 172.21.15.55:0/21033804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380ccc00 0x7fae380c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3d99c700 1 -- 172.21.15.55:0/21033804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae380cb230 msgr2=0x7fae381522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3d99c700 1 --2- 172.21.15.55:0/21033804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae380cb230 0x7fae381522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.056+0000 7fae3d99c700 1 -- 172.21.15.55:0/21033804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae30009a20 con 0x7fae38159a60 2023-10-06T05:05:37.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.057+0000 7fae3d99c700 1 --2- 172.21.15.55:0/21033804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38159a60 0x7fae380c5fd0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fae3806cdc0 tx=0x7fae30019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:37.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.057+0000 7fae2d7fa700 1 -- 172.21.15.55:0/21033804 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30012e10 con 0x7fae38159a60 2023-10-06T05:05:37.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.057+0000 7fae2d7fa700 1 -- 172.21.15.55:0/21033804 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3001d630 con 0x7fae38159a60 2023-10-06T05:05:37.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.057+0000 7fae2d7fa700 1 -- 172.21.15.55:0/21033804 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3001d9f0 con 0x7fae38159a60 2023-10-06T05:05:37.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.057+0000 7fae3e19d700 1 -- 172.21.15.55:0/21033804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae380c7db0 con 0x7fae38159a60 2023-10-06T05:05:37.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.057+0000 7fae3e19d700 1 -- 172.21.15.55:0/21033804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae380c6510 con 0x7fae38159a60 2023-10-06T05:05:37.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.058+0000 7fae3e19d700 1 -- 172.21.15.55:0/21033804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae380c85b0 con 0x7fae38159a60 2023-10-06T05:05:37.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.059+0000 7fae2d7fa700 1 -- 172.21.15.55:0/21033804 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae30024070 con 0x7fae38159a60 2023-10-06T05:05:37.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.060+0000 7fae2d7fa700 1 --2- 172.21.15.55:0/21033804 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fae1c066f30 0x7fae1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.060+0000 7fae2d7fa700 1 -- 172.21.15.55:0/21033804 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5626+0+0 (secure 0 0 0) 0x7fae30086e70 con 0x7fae38159a60 2023-10-06T05:05:37.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.060+0000 7fae3d19b700 1 --2- 172.21.15.55:0/21033804 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fae1c066f30 0x7fae1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.061+0000 7fae3d19b700 1 --2- 172.21.15.55:0/21033804 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fae1c066f30 0x7fae1c0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fae340039e0 tx=0x7fae34007ec0 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:37.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.065+0000 7fae2d7fa700 1 -- 172.21.15.55:0/21033804 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae380c85b0 con 0x7fae38159a60 2023-10-06T05:05:37.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.266+0000 7fae3e19d700 1 -- 172.21.15.55:0/21033804 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fae380c85b0 con 0x7fae38159a60 2023-10-06T05:05:37.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.267+0000 7fae2d7fa700 1 -- 172.21.15.55:0/21033804 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v189) v1 ==== 92+0+10 (secure 0 0 0) 0x7fae380c85b0 con 0x7fae38159a60 2023-10-06T05:05:37.270 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T05:05:37.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.269+0000 7fae3e19d700 1 -- 172.21.15.55:0/21033804 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fae1c066f30 msgr2=0x7fae1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.269+0000 7fae3e19d700 1 --2- 172.21.15.55:0/21033804 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fae1c066f30 0x7fae1c0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fae340039e0 tx=0x7fae34007ec0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.269+0000 7fae3e19d700 1 -- 172.21.15.55:0/21033804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38159a60 msgr2=0x7fae380c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.269+0000 7fae3e19d700 1 --2- 172.21.15.55:0/21033804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38159a60 0x7fae380c5fd0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fae3806cdc0 tx=0x7fae30019000 comp rx=0 tx=0).stop 2023-10-06T05:05:37.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.269+0000 7fae3e19d700 1 -- 172.21.15.55:0/21033804 shutdown_connections 2023-10-06T05:05:37.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.269+0000 7fae3e19d700 1 --2- 172.21.15.55:0/21033804 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fae1c066f30 0x7fae1c0693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.269+0000 7fae3e19d700 1 --2- 172.21.15.55:0/21033804 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fae38159a60 0x7fae380c5fd0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.269+0000 7fae3e19d700 1 --2- 172.21.15.55:0/21033804 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fae380cb230 0x7fae381522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.269+0000 7fae3e19d700 1 --2- 172.21.15.55:0/21033804 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fae380ccc00 0x7fae380c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.269+0000 7fae3e19d700 1 -- 172.21.15.55:0/21033804 >> 172.21.15.55:0/21033804 conn(0x7fae3806baa0 msgr2=0x7fae38154130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:37.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.269+0000 7fae3e19d700 1 -- 172.21.15.55:0/21033804 shutdown_connections 2023-10-06T05:05:37.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.269+0000 7fae3e19d700 1 -- 172.21.15.55:0/21033804 wait complete. 2023-10-06T05:05:37.290 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:05:37.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.477+0000 7f390759e700 1 Processor -- start 2023-10-06T05:05:37.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.477+0000 7f390759e700 1 -- start start 2023-10-06T05:05:37.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.479+0000 7f390759e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f390806f060 0x7f39080ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.479+0000 7f390759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f390806f4b0 0x7f39080ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.479+0000 7f390759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390806ebd0 con 0x7f39080cb1a0 2023-10-06T05:05:37.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.479+0000 7f390759e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390815b2e0 con 0x7f390806f060 2023-10-06T05:05:37.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.479+0000 7f390759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390815aa70 con 0x7f390806f4b0 2023-10-06T05:05:37.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.479+0000 7f3905d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f390806f060 0x7f39080ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.479+0000 7f3905d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f390806f060 0x7f39080ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39112/0 (socket says 172.21.15.55:39112) 2023-10-06T05:05:37.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.479+0000 7f3905d9b700 1 -- 172.21.15.55:0/2132241606 learned_addr learned my addr 172.21.15.55:0/2132241606 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:37.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.479+0000 7f3906d9d700 1 --2- 172.21.15.55:0/2132241606 >> v2:172.21.15.163:3300/0 conn(0x7f390806f4b0 0x7f39080ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.479+0000 7f3905d9b700 1 -- 172.21.15.55:0/2132241606 >> v2:172.21.15.163:3300/0 conn(0x7f390806f4b0 msgr2=0x7f39080ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:37.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.479+0000 7f3905d9b700 1 --2- 172.21.15.55:0/2132241606 >> v2:172.21.15.163:3300/0 conn(0x7f390806f4b0 0x7f39080ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.480+0000 7f3905d9b700 1 -- 172.21.15.55:0/2132241606 >> v1:172.21.15.163:6789/0 conn(0x7f39080cb1a0 legacy=0x7f390815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.480+0000 7f3905d9b700 1 -- 172.21.15.55:0/2132241606 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390815ad20 con 0x7f390806f060 2023-10-06T05:05:37.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.480+0000 7f3905d9b700 1 --2- 172.21.15.55:0/2132241606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f390806f060 0x7f39080ae0d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f390806d480 tx=0x7f38fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c863e7018c19e8f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:37.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.480+0000 7f390559a700 1 -- 172.21.15.55:0/2132241606 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00b8f0 con 0x7f390806f060 2023-10-06T05:05:37.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.480+0000 7f390559a700 1 -- 172.21.15.55:0/2132241606 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc00ba50 con 0x7f390806f060 2023-10-06T05:05:37.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.480+0000 7f390559a700 1 -- 172.21.15.55:0/2132241606 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00bdf0 con 0x7f390806f060 2023-10-06T05:05:37.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.481+0000 7f390759e700 1 -- 172.21.15.55:0/2132241606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f390806f060 msgr2=0x7f39080ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.481+0000 7f390759e700 1 --2- 172.21.15.55:0/2132241606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f390806f060 0x7f39080ae0d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f390806d480 tx=0x7f38fc009250 comp rx=0 tx=0).stop 2023-10-06T05:05:37.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.481+0000 7f390759e700 1 -- 172.21.15.55:0/2132241606 shutdown_connections 2023-10-06T05:05:37.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.481+0000 7f390759e700 1 --2- 172.21.15.55:0/2132241606 >> v2:172.21.15.163:3300/0 conn(0x7f390806f4b0 0x7f39080ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.481+0000 7f390759e700 1 --2- 172.21.15.55:0/2132241606 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f390806f060 0x7f39080ae0d0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.481+0000 7f390759e700 1 -- 172.21.15.55:0/2132241606 >> 172.21.15.55:0/2132241606 conn(0x7f39080cc1e0 msgr2=0x7f39080cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:37.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.481+0000 7f390759e700 1 -- 172.21.15.55:0/2132241606 shutdown_connections 2023-10-06T05:05:37.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.481+0000 7f390759e700 1 -- 172.21.15.55:0/2132241606 wait complete. 2023-10-06T05:05:37.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.482+0000 7f390759e700 1 Processor -- start 2023-10-06T05:05:37.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.482+0000 7f390759e700 1 -- start start 2023-10-06T05:05:37.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.483+0000 7f390759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390806f060 0x7f3908153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.483+0000 7f390759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f390806f4b0 0x7f390816a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.483+0000 7f390759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39080cb1a0 0x7f390815c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.483+0000 7f390759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39081542a0 con 0x7f390806f4b0 2023-10-06T05:05:37.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.483+0000 7f390759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3908158cd0 con 0x7f39080cb1a0 2023-10-06T05:05:37.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.483+0000 7f390759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390815b620 con 0x7f390806f060 2023-10-06T05:05:37.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.483+0000 7f3905d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f390806f4b0 0x7f390816a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.483+0000 7f3905d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f390806f4b0 0x7f390816a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39128/0 (socket says 172.21.15.55:39128) 2023-10-06T05:05:37.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.483+0000 7f3905d9b700 1 -- 172.21.15.55:0/2455071752 learned_addr learned my addr 172.21.15.55:0/2455071752 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:37.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.484+0000 7f3906d9d700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39080cb1a0 0x7f390815c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.484+0000 7f390659c700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390806f060 0x7f3908153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.484+0000 7f3905d9b700 1 -- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390806f060 msgr2=0x7f3908153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.484+0000 7f3905d9b700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390806f060 0x7f3908153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.484+0000 7f3905d9b700 1 -- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39080cb1a0 msgr2=0x7f390815c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.484+0000 7f3905d9b700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39080cb1a0 0x7f390815c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.484+0000 7f3905d9b700 1 -- 172.21.15.55:0/2455071752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0094f0 con 0x7f390806f4b0 2023-10-06T05:05:37.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.484+0000 7f390659c700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390806f060 0x7f3908153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:37.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.484+0000 7f3905d9b700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f390806f4b0 0x7f390816a060 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009290 tx=0x7f38fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:37.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.484+0000 7f38f67fc700 1 -- 172.21.15.55:0/2455071752 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc006bc0 con 0x7f390806f4b0 2023-10-06T05:05:37.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.484+0000 7f38f67fc700 1 -- 172.21.15.55:0/2455071752 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc011e20 con 0x7f390806f4b0 2023-10-06T05:05:37.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.484+0000 7f38f67fc700 1 -- 172.21.15.55:0/2455071752 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc01a880 con 0x7f390806f4b0 2023-10-06T05:05:37.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.484+0000 7f390759e700 1 -- 172.21.15.55:0/2455071752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390815b900 con 0x7f390806f4b0 2023-10-06T05:05:37.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.485+0000 7f390759e700 1 -- 172.21.15.55:0/2455071752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390816a5a0 con 0x7f390806f4b0 2023-10-06T05:05:37.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.485+0000 7f390759e700 1 -- 172.21.15.55:0/2455071752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390806c450 con 0x7f390806f4b0 2023-10-06T05:05:37.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.486+0000 7f38f67fc700 1 -- 172.21.15.55:0/2455071752 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f38fc01acb0 con 0x7f390806f4b0 2023-10-06T05:05:37.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.487+0000 7f38f67fc700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f38e8066f30 0x7f38e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.487+0000 7f38f67fc700 1 -- 172.21.15.55:0/2455071752 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5626+0+0 (secure 0 0 0) 0x7f38fc086220 con 0x7f390806f4b0 2023-10-06T05:05:37.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.488+0000 7f390659c700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f38e8066f30 0x7f38e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.489+0000 7f390659c700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f38e8066f30 0x7f38e80693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f38f80017a0 tx=0x7f38f8009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:37.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.492+0000 7f38f67fc700 1 -- 172.21.15.55:0/2455071752 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f38fc053440 con 0x7f390806f4b0 2023-10-06T05:05:37.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.693+0000 7f390759e700 1 -- 172.21.15.55:0/2455071752 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f390815b000 con 0x7f390806f4b0 2023-10-06T05:05:37.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.693+0000 7f38f67fc700 1 -- 172.21.15.55:0/2455071752 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v189) v1 ==== 93+0+10 (secure 0 0 0) 0x7f38fc02b340 con 0x7f390806f4b0 2023-10-06T05:05:37.696 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T05:05:37.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.696+0000 7f390759e700 1 -- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f38e8066f30 msgr2=0x7f38e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.696+0000 7f390759e700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f38e8066f30 0x7f38e80693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f38f80017a0 tx=0x7f38f8009040 comp rx=0 tx=0).stop 2023-10-06T05:05:37.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.696+0000 7f390759e700 1 -- 172.21.15.55:0/2455071752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f390806f4b0 msgr2=0x7f390816a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.696+0000 7f390759e700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f390806f4b0 0x7f390816a060 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009290 tx=0x7f38fc017000 comp rx=0 tx=0).stop 2023-10-06T05:05:37.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.696+0000 7f390759e700 1 -- 172.21.15.55:0/2455071752 shutdown_connections 2023-10-06T05:05:37.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.696+0000 7f390759e700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f38e8066f30 0x7f38e80693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.696+0000 7f390759e700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f39080cb1a0 0x7f390815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.696+0000 7f390759e700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f390806f060 0x7f3908153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.696+0000 7f390759e700 1 --2- 172.21.15.55:0/2455071752 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f390806f4b0 0x7f390816a060 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.696+0000 7f390759e700 1 -- 172.21.15.55:0/2455071752 >> 172.21.15.55:0/2455071752 conn(0x7f39080cc1e0 msgr2=0x7f3908155ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:37.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.696+0000 7f390759e700 1 -- 172.21.15.55:0/2455071752 shutdown_connections 2023-10-06T05:05:37.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.696+0000 7f390759e700 1 -- 172.21.15.55:0/2455071752 wait complete. 2023-10-06T05:05:37.717 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:05:37.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.902+0000 7f1f6e59d700 1 Processor -- start 2023-10-06T05:05:37.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.902+0000 7f1f6e59d700 1 -- start start 2023-10-06T05:05:37.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.903+0000 7f1f6e59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1f681597f0 0x7f1f680b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.903+0000 7f1f6e59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f6815d230 0x7f1f6815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.904+0000 7f1f6e59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6815dbb0 con 0x7f1f68159420 2023-10-06T05:05:37.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.904+0000 7f1f6e59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f680b11d0 con 0x7f1f681597f0 2023-10-06T05:05:37.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.904+0000 7f1f6e59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f680b14b0 con 0x7f1f6815d230 2023-10-06T05:05:37.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.904+0000 7f1f6cd9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1f681597f0 0x7f1f680b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.904+0000 7f1f6cd9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1f681597f0 0x7f1f680b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39142/0 (socket says 172.21.15.55:39142) 2023-10-06T05:05:37.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.904+0000 7f1f6cd9a700 1 -- 172.21.15.55:0/1301727974 learned_addr learned my addr 172.21.15.55:0/1301727974 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:37.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.904+0000 7f1f6d59b700 1 --1- 172.21.15.55:0/1301727974 >> v1:172.21.15.55:6789/0 conn(0x7f1f68159420 0x7f1f680b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46048/0 (socket says 172.21.15.55:46048) 2023-10-06T05:05:37.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.904+0000 7f1f6dd9c700 1 --2- 172.21.15.55:0/1301727974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f6815d230 0x7f1f6815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.904+0000 7f1f6cd9a700 1 -- 172.21.15.55:0/1301727974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f6815d230 msgr2=0x7f1f6815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:37.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.904+0000 7f1f6cd9a700 1 --2- 172.21.15.55:0/1301727974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f6815d230 0x7f1f6815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.904+0000 7f1f6cd9a700 1 -- 172.21.15.55:0/1301727974 >> v1:172.21.15.55:6789/0 conn(0x7f1f68159420 legacy=0x7f1f680b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.904+0000 7f1f6cd9a700 1 -- 172.21.15.55:0/1301727974 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f680b1790 con 0x7f1f681597f0 2023-10-06T05:05:37.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.904+0000 7f1f5ffff700 1 -- 172.21.15.55:0/1301727974 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354953654 0 0) 0x7f1f6815dbb0 con 0x7f1f68159420 2023-10-06T05:05:37.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.905+0000 7f1f6cd9a700 1 --2- 172.21.15.55:0/1301727974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f681597f0 0x7f1f680b2270 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f1f580099b0 tx=0x7f1f58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc79dd5157bb8b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:37.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.905+0000 7f1f5ffff700 1 -- 172.21.15.55:0/1301727974 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800b860 con 0x7f1f681597f0 2023-10-06T05:05:37.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.905+0000 7f1f5ffff700 1 -- 172.21.15.55:0/1301727974 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5800b9c0 con 0x7f1f681597f0 2023-10-06T05:05:37.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.905+0000 7f1f5ffff700 1 -- 172.21.15.55:0/1301727974 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800bd60 con 0x7f1f681597f0 2023-10-06T05:05:37.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.905+0000 7f1f6e59d700 1 -- 172.21.15.55:0/1301727974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f681597f0 msgr2=0x7f1f680b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.905+0000 7f1f6e59d700 1 --2- 172.21.15.55:0/1301727974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f681597f0 0x7f1f680b2270 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f1f580099b0 tx=0x7f1f58009250 comp rx=0 tx=0).stop 2023-10-06T05:05:37.912 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.905+0000 7f1f6e59d700 1 -- 172.21.15.55:0/1301727974 shutdown_connections 2023-10-06T05:05:37.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.905+0000 7f1f6e59d700 1 --2- 172.21.15.55:0/1301727974 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f6815d230 0x7f1f6815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.905+0000 7f1f6e59d700 1 --2- 172.21.15.55:0/1301727974 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f681597f0 0x7f1f680b2270 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.905+0000 7f1f6e59d700 1 -- 172.21.15.55:0/1301727974 >> 172.21.15.55:0/1301727974 conn(0x7f1f680d0660 msgr2=0x7f1f680d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:37.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.906+0000 7f1f6e59d700 1 -- 172.21.15.55:0/1301727974 shutdown_connections 2023-10-06T05:05:37.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.906+0000 7f1f6e59d700 1 -- 172.21.15.55:0/1301727974 wait complete. 2023-10-06T05:05:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.907+0000 7f1f6e59d700 1 Processor -- start 2023-10-06T05:05:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.907+0000 7f1f6e59d700 1 -- start start 2023-10-06T05:05:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.908+0000 7f1f6e59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f68159420 0x7f1f681575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.918 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.908+0000 7f1f6e59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f681597f0 0x7f1f6815b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.908+0000 7f1f6e59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f6815d230 0x7f1f68153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.908+0000 7f1f6e59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f680c5530 con 0x7f1f6815d230 2023-10-06T05:05:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.908+0000 7f1f6e59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6815cba0 con 0x7f1f68159420 2023-10-06T05:05:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.908+0000 7f1f6e59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6815c300 con 0x7f1f681597f0 2023-10-06T05:05:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.908+0000 7f1f6dd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f6815d230 0x7f1f68153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.908+0000 7f1f6dd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f6815d230 0x7f1f68153f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39146/0 (socket says 172.21.15.55:39146) 2023-10-06T05:05:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.908+0000 7f1f6dd9c700 1 -- 172.21.15.55:0/3111601838 learned_addr learned my addr 172.21.15.55:0/3111601838 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.909+0000 7f1f6cd9a700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f681597f0 0x7f1f6815b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.919 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.909+0000 7f1f6d59b700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f68159420 0x7f1f681575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.909+0000 7f1f6dd9c700 1 -- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f681597f0 msgr2=0x7f1f6815b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.909+0000 7f1f6dd9c700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f681597f0 0x7f1f6815b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.909+0000 7f1f6dd9c700 1 -- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f68159420 msgr2=0x7f1f681575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.909+0000 7f1f6dd9c700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f68159420 0x7f1f681575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.909+0000 7f1f6dd9c700 1 -- 172.21.15.55:0/3111601838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f580094f0 con 0x7f1f6815d230 2023-10-06T05:05:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.909+0000 7f1f6d59b700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f68159420 0x7f1f681575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:37.920 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.909+0000 7f1f6cd9a700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f681597f0 0x7f1f6815b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.909+0000 7f1f6dd9c700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f6815d230 0x7f1f68153f70 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f1f60007f20 tx=0x7f1f60009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.909+0000 7f1f5d7fa700 1 -- 172.21.15.55:0/3111601838 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6000eca0 con 0x7f1f6815d230 2023-10-06T05:05:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.909+0000 7f1f5d7fa700 1 -- 172.21.15.55:0/3111601838 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6000ee20 con 0x7f1f6815d230 2023-10-06T05:05:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.909+0000 7f1f5d7fa700 1 -- 172.21.15.55:0/3111601838 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60014670 con 0x7f1f6815d230 2023-10-06T05:05:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.909+0000 7f1f6e59d700 1 -- 172.21.15.55:0/3111601838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6815c5e0 con 0x7f1f6815d230 2023-10-06T05:05:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.910+0000 7f1f6e59d700 1 -- 172.21.15.55:0/3111601838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f68159e50 con 0x7f1f6815d230 2023-10-06T05:05:37.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.910+0000 7f1f6e59d700 1 -- 172.21.15.55:0/3111601838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6804c3c0 con 0x7f1f6815d230 2023-10-06T05:05:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.911+0000 7f1f5d7fa700 1 -- 172.21.15.55:0/3111601838 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1f600149e0 con 0x7f1f6815d230 2023-10-06T05:05:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.912+0000 7f1f5d7fa700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1f4c066f30 0x7f1f4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.912+0000 7f1f5d7fa700 1 -- 172.21.15.55:0/3111601838 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5626+0+0 (secure 0 0 0) 0x7f1f60084cb0 con 0x7f1f6815d230 2023-10-06T05:05:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.913+0000 7f1f6d59b700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1f4c066f30 0x7f1f4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:37.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.914+0000 7f1f6d59b700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1f4c066f30 0x7f1f4c0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1f640018d0 tx=0x7f1f64009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:37.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:37.916+0000 7f1f5d7fa700 1 -- 172.21.15.55:0/3111601838 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f60051ed0 con 0x7f1f6815d230 2023-10-06T05:05:38.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.126+0000 7f1f6e59d700 1 -- 172.21.15.55:0/3111601838 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1f680c5670 con 0x7f1f6815d230 2023-10-06T05:05:38.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.127+0000 7f1f5d7fa700 1 -- 172.21.15.55:0/3111601838 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v189) v1 ==== 84+0+126 (secure 0 0 0) 0x7f1f600554f0 con 0x7f1f6815d230 2023-10-06T05:05:38.130 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:38.130 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T05:05:38.130 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:38.130 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2023-10-06T05:05:38.130 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": "rbd" 2023-10-06T05:05:38.130 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:05:38.130 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:38.130 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 2, 2023-10-06T05:05:38.130 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2023-10-06T05:05:38.131 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:38.131 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T05:05:38.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.129+0000 7f1f6e59d700 1 -- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1f4c066f30 msgr2=0x7f1f4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:38.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.129+0000 7f1f6e59d700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1f4c066f30 0x7f1f4c0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1f640018d0 tx=0x7f1f64009040 comp rx=0 tx=0).stop 2023-10-06T05:05:38.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.129+0000 7f1f6e59d700 1 -- 172.21.15.55:0/3111601838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f6815d230 msgr2=0x7f1f68153f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:38.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.129+0000 7f1f6e59d700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f6815d230 0x7f1f68153f70 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f1f60007f20 tx=0x7f1f60009860 comp rx=0 tx=0).stop 2023-10-06T05:05:38.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.129+0000 7f1f6e59d700 1 -- 172.21.15.55:0/3111601838 shutdown_connections 2023-10-06T05:05:38.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.129+0000 7f1f6e59d700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1f4c066f30 0x7f1f4c0693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:38.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.129+0000 7f1f6e59d700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1f6815d230 0x7f1f68153f70 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:38.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.129+0000 7f1f6e59d700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1f68159420 0x7f1f681575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:38.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.129+0000 7f1f6e59d700 1 --2- 172.21.15.55:0/3111601838 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1f681597f0 0x7f1f6815b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:38.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.129+0000 7f1f6e59d700 1 -- 172.21.15.55:0/3111601838 >> 172.21.15.55:0/3111601838 conn(0x7f1f680d0660 msgr2=0x7f1f68156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:38.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.129+0000 7f1f6e59d700 1 -- 172.21.15.55:0/3111601838 shutdown_connections 2023-10-06T05:05:38.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.129+0000 7f1f6e59d700 1 -- 172.21.15.55:0/3111601838 wait complete. 2023-10-06T05:05:38.150 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:05:38.150 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:05:38.304 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:05:38.309 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:05:38.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.339+0000 7f5a04f9d700 1 Processor -- start 2023-10-06T05:05:38.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.339+0000 7f5a04f9d700 1 -- start start 2023-10-06T05:05:38.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.340+0000 7f5a04f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a000b0790 0x7f5a000cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:38.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.340+0000 7f5a04f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a001591e0 0x7f5a00159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:38.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.340+0000 7f5a04f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a000cfa00 con 0x7f5a000b03c0 2023-10-06T05:05:38.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.340+0000 7f5a04f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a000cb580 con 0x7f5a001591e0 2023-10-06T05:05:38.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.340+0000 7f5a04f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a000cf160 con 0x7f5a000b0790 2023-10-06T05:05:38.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.340+0000 7f59feffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a000b0790 0x7f5a000cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:38.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.340+0000 7f59fffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5a001591e0 0x7f5a00159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:38.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.340+0000 7f59ff7fe700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f5a000b03c0 0x7f5a000b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:40772/0 (socket says 172.21.15.55:40772) 2023-10-06T05:05:38.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.340+0000 7f59feffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a000b0790 0x7f5a000cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:59206/0 (socket says 172.21.15.55:59206) 2023-10-06T05:05:38.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.341+0000 7f59ff7fe700 1 -- 172.21.15.55:0/4150099027 learned_addr learned my addr 172.21.15.55:0/4150099027 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:38.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.341+0000 7f59fe7fc700 1 -- 172.21.15.55:0/4150099027 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601402959 0 0) 0x7f5a000cfa00 con 0x7f5a000b03c0 2023-10-06T05:05:38.346 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.341+0000 7f59fe7fc700 1 -- 172.21.15.55:0/4150099027 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59e8003570 con 0x7f5a000b03c0 2023-10-06T05:05:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.341+0000 7f59fffff700 1 -- 172.21.15.55:0/4150099027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a000b0790 msgr2=0x7f5a000cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.341+0000 7f59fffff700 1 --2- 172.21.15.55:0/4150099027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a000b0790 0x7f5a000cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.341+0000 7f59fffff700 1 -- 172.21.15.55:0/4150099027 >> v1:172.21.15.163:6789/0 conn(0x7f5a000b03c0 legacy=0x7f5a000b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.341+0000 7f59fffff700 1 -- 172.21.15.55:0/4150099027 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a000cf440 con 0x7f5a001591e0 2023-10-06T05:05:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.341+0000 7f59feffd700 1 --2- 172.21.15.55:0/4150099027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a000b0790 0x7f5a000cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.342+0000 7f59fffff700 1 --2- 172.21.15.55:0/4150099027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a001591e0 0x7f5a00159620 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009e60 tx=0x7f59f8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f507436d8ba70bb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.342+0000 7f59fe7fc700 1 -- 172.21.15.55:0/4150099027 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f800e070 con 0x7f5a001591e0 2023-10-06T05:05:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.342+0000 7f59fe7fc700 1 -- 172.21.15.55:0/4150099027 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f800bab0 con 0x7f5a001591e0 2023-10-06T05:05:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.342+0000 7f59fe7fc700 1 -- 172.21.15.55:0/4150099027 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f800be70 con 0x7f5a001591e0 2023-10-06T05:05:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.342+0000 7f5a04f9d700 1 -- 172.21.15.55:0/4150099027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a001591e0 msgr2=0x7f5a00159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:38.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.342+0000 7f5a04f9d700 1 --2- 172.21.15.55:0/4150099027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a001591e0 0x7f5a00159620 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009e60 tx=0x7f59f8009780 comp rx=0 tx=0).stop 2023-10-06T05:05:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.343+0000 7f5a04f9d700 1 -- 172.21.15.55:0/4150099027 shutdown_connections 2023-10-06T05:05:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.343+0000 7f5a04f9d700 1 --2- 172.21.15.55:0/4150099027 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a001591e0 0x7f5a00159620 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.343+0000 7f5a04f9d700 1 --2- 172.21.15.55:0/4150099027 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a000b0790 0x7f5a000cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.343+0000 7f5a04f9d700 1 -- 172.21.15.55:0/4150099027 >> 172.21.15.55:0/4150099027 conn(0x7f5a00063230 msgr2=0x7f5a00063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.343+0000 7f5a04f9d700 1 -- 172.21.15.55:0/4150099027 shutdown_connections 2023-10-06T05:05:38.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.343+0000 7f5a04f9d700 1 -- 172.21.15.55:0/4150099027 wait complete. 2023-10-06T05:05:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.344+0000 7f5a04f9d700 1 Processor -- start 2023-10-06T05:05:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.344+0000 7f5a04f9d700 1 -- start start 2023-10-06T05:05:38.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.345+0000 7f5a04f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a000b03c0 0x7f5a00157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.345+0000 7f5a04f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a000b0790 0x7f5a0015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.345+0000 7f5a04f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a001591e0 0x7f5a0015a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.345+0000 7f5a04f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0015c4b0 con 0x7f5a001591e0 2023-10-06T05:05:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.345+0000 7f5a04f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0015cd10 con 0x7f5a000b03c0 2023-10-06T05:05:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.345+0000 7f5a04f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a00157960 con 0x7f5a000b0790 2023-10-06T05:05:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.346+0000 7f59fffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a001591e0 0x7f5a0015a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.346+0000 7f59fffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a001591e0 0x7f5a0015a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39152/0 (socket says 172.21.15.55:39152) 2023-10-06T05:05:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.346+0000 7f59fffff700 1 -- 172.21.15.55:0/2942836516 learned_addr learned my addr 172.21.15.55:0/2942836516 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.346+0000 7f59feffd700 1 --2- 172.21.15.55:0/2942836516 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a000b0790 0x7f5a0015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.346+0000 7f59ff7fe700 1 --2- 172.21.15.55:0/2942836516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a000b03c0 0x7f5a00157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:38.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.346+0000 7f59fffff700 1 -- 172.21.15.55:0/2942836516 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a000b0790 msgr2=0x7f5a0015d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.346+0000 7f59fffff700 1 --2- 172.21.15.55:0/2942836516 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a000b0790 0x7f5a0015d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.346+0000 7f59fffff700 1 -- 172.21.15.55:0/2942836516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a000b03c0 msgr2=0x7f5a00157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.346+0000 7f59fffff700 1 --2- 172.21.15.55:0/2942836516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a000b03c0 0x7f5a00157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.346+0000 7f59fffff700 1 -- 172.21.15.55:0/2942836516 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f4009420 con 0x7f5a001591e0 2023-10-06T05:05:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.347+0000 7f59fffff700 1 --2- 172.21.15.55:0/2942836516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a001591e0 0x7f5a0015a400 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f5a000cfe40 tx=0x7f59f8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.347+0000 7f59e77fe700 1 -- 172.21.15.55:0/2942836516 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8012e50 con 0x7f5a001591e0 2023-10-06T05:05:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.347+0000 7f59e77fe700 1 -- 172.21.15.55:0/2942836516 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f8027470 con 0x7f5a001591e0 2023-10-06T05:05:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.347+0000 7f5a04f9d700 1 -- 172.21.15.55:0/2942836516 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f8009a20 con 0x7f5a001591e0 2023-10-06T05:05:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.347+0000 7f59e77fe700 1 -- 172.21.15.55:0/2942836516 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8027830 con 0x7f5a001591e0 2023-10-06T05:05:38.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.347+0000 7f5a04f9d700 1 -- 172.21.15.55:0/2942836516 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0015c740 con 0x7f5a001591e0 2023-10-06T05:05:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.348+0000 7f5a04f9d700 1 -- 172.21.15.55:0/2942836516 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0004c3c0 con 0x7f5a001591e0 2023-10-06T05:05:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.349+0000 7f59e77fe700 1 -- 172.21.15.55:0/2942836516 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59f8024070 con 0x7f5a001591e0 2023-10-06T05:05:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.350+0000 7f59e77fe700 1 --2- 172.21.15.55:0/2942836516 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f59ec066f30 0x7f59ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:38.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.350+0000 7f59e77fe700 1 -- 172.21.15.55:0/2942836516 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5626+0+0 (secure 0 0 0) 0x7f59f8086b60 con 0x7f5a001591e0 2023-10-06T05:05:38.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.351+0000 7f59ff7fe700 1 --2- 172.21.15.55:0/2942836516 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f59ec066f30 0x7f59ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:38.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.351+0000 7f59ff7fe700 1 --2- 172.21.15.55:0/2942836516 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f59ec066f30 0x7f59ec0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f59f0007fd0 tx=0x7f59f0009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:38.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.357+0000 7f59e77fe700 1 -- 172.21.15.55:0/2942836516 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59f8053d80 con 0x7f5a001591e0 2023-10-06T05:05:38.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.560+0000 7f5a04f9d700 1 -- 172.21.15.55:0/2942836516 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5a0015ce50 con 0x7f5a001591e0 2023-10-06T05:05:38.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.893+0000 7f59e77fe700 1 -- 172.21.15.55:0/2942836516 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v190) v1 ==== 138+0+0 (secure 0 0 0) 0x7f59f80573a0 con 0x7f5a001591e0 2023-10-06T05:05:38.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.897+0000 7f5a04f9d700 1 -- 172.21.15.55:0/2942836516 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f59ec066f30 msgr2=0x7f59ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:38.899 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.897+0000 7f5a04f9d700 1 --2- 172.21.15.55:0/2942836516 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f59ec066f30 0x7f59ec0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f59f0007fd0 tx=0x7f59f0009040 comp rx=0 tx=0).stop 2023-10-06T05:05:38.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.897+0000 7f5a04f9d700 1 -- 172.21.15.55:0/2942836516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a001591e0 msgr2=0x7f5a0015a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:38.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.897+0000 7f5a04f9d700 1 --2- 172.21.15.55:0/2942836516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a001591e0 0x7f5a0015a400 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f5a000cfe40 tx=0x7f59f8019000 comp rx=0 tx=0).stop 2023-10-06T05:05:38.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.897+0000 7f5a04f9d700 1 -- 172.21.15.55:0/2942836516 shutdown_connections 2023-10-06T05:05:38.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.897+0000 7f5a04f9d700 1 --2- 172.21.15.55:0/2942836516 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f59ec066f30 0x7f59ec0693e0 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f59f0007fd0 tx=0x7f59f0009040 comp rx=0 tx=0).stop 2023-10-06T05:05:38.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.897+0000 7f5a04f9d700 1 --2- 172.21.15.55:0/2942836516 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5a001591e0 0x7f5a0015a400 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:38.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.897+0000 7f5a04f9d700 1 --2- 172.21.15.55:0/2942836516 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5a000b03c0 0x7f5a00157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:38.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.897+0000 7f5a04f9d700 1 --2- 172.21.15.55:0/2942836516 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5a000b0790 0x7f5a0015d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:38.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.897+0000 7f5a04f9d700 1 -- 172.21.15.55:0/2942836516 >> 172.21.15.55:0/2942836516 conn(0x7f5a00063230 msgr2=0x7f5a000b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:38.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.897+0000 7f5a04f9d700 1 -- 172.21.15.55:0/2942836516 shutdown_connections 2023-10-06T05:05:38.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:38.898+0000 7f5a04f9d700 1 -- 172.21.15.55:0/2942836516 wait complete. 2023-10-06T05:05:38.906 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2023-10-06T05:05:38.924 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:05:39.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.116+0000 7fb46359e700 1 Processor -- start 2023-10-06T05:05:39.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.116+0000 7fb46359e700 1 -- start start 2023-10-06T05:05:39.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.117+0000 7fb46359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4641595d0 0x7fb4640ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:39.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.117+0000 7fb46359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb46415d930 0x7fb4640b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:39.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.117+0000 7fb46359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4640b10f0 con 0x7fb4640cf480 2023-10-06T05:05:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.117+0000 7fb46359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4640cd530 con 0x7fb46415d930 2023-10-06T05:05:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.117+0000 7fb46359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46415e000 con 0x7fb4641595d0 2023-10-06T05:05:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.118+0000 7fb461d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb4640cf480 0x7fb46415d220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46062/0 (socket says 172.21.15.55:46062) 2023-10-06T05:05:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.118+0000 7fb461d9b700 1 -- 172.21.15.55:0/3214214216 learned_addr learned my addr 172.21.15.55:0/3214214216 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.118+0000 7fb462d9d700 1 --2- 172.21.15.55:0/3214214216 >> v2:172.21.15.163:3300/0 conn(0x7fb46415d930 0x7fb4640b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.118+0000 7fb46259c700 1 --2- 172.21.15.55:0/3214214216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4641595d0 0x7fb4640ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.118+0000 7fb46159a700 1 -- 172.21.15.55:0/3214214216 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255176326 0 0) 0x7fb4640b10f0 con 0x7fb4640cf480 2023-10-06T05:05:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.118+0000 7fb46159a700 1 -- 172.21.15.55:0/3214214216 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb44c003570 con 0x7fb4640cf480 2023-10-06T05:05:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.118+0000 7fb462d9d700 1 -- 172.21.15.55:0/3214214216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4641595d0 msgr2=0x7fb4640ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:39.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.118+0000 7fb462d9d700 1 --2- 172.21.15.55:0/3214214216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4641595d0 0x7fb4640ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.118+0000 7fb462d9d700 1 -- 172.21.15.55:0/3214214216 >> v1:172.21.15.55:6789/0 conn(0x7fb4640cf480 legacy=0x7fb46415d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.118+0000 7fb462d9d700 1 -- 172.21.15.55:0/3214214216 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4640cf990 con 0x7fb46415d930 2023-10-06T05:05:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.118+0000 7fb46259c700 1 --2- 172.21.15.55:0/3214214216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4641595d0 0x7fb4640ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:39.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.119+0000 7fb462d9d700 1 --2- 172.21.15.55:0/3214214216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb46415d930 0x7fb4640b1580 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009e60 tx=0x7fb45c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=282ae90791a6df2b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:39.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.119+0000 7fb46159a700 1 -- 172.21.15.55:0/3214214216 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00e070 con 0x7fb46415d930 2023-10-06T05:05:39.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.119+0000 7fb46159a700 1 -- 172.21.15.55:0/3214214216 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c00bab0 con 0x7fb46415d930 2023-10-06T05:05:39.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.119+0000 7fb46159a700 1 -- 172.21.15.55:0/3214214216 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00be70 con 0x7fb46415d930 2023-10-06T05:05:39.124 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.119+0000 7fb46359e700 1 -- 172.21.15.55:0/3214214216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb46415d930 msgr2=0x7fb4640b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.119+0000 7fb46359e700 1 --2- 172.21.15.55:0/3214214216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb46415d930 0x7fb4640b1580 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009e60 tx=0x7fb45c009780 comp rx=0 tx=0).stop 2023-10-06T05:05:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.120+0000 7fb46359e700 1 -- 172.21.15.55:0/3214214216 shutdown_connections 2023-10-06T05:05:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.120+0000 7fb46359e700 1 --2- 172.21.15.55:0/3214214216 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb46415d930 0x7fb4640b1580 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.120+0000 7fb46359e700 1 --2- 172.21.15.55:0/3214214216 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb4641595d0 0x7fb4640ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.120+0000 7fb46359e700 1 -- 172.21.15.55:0/3214214216 >> 172.21.15.55:0/3214214216 conn(0x7fb4640d05f0 msgr2=0x7fb4640d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.120+0000 7fb46359e700 1 -- 172.21.15.55:0/3214214216 shutdown_connections 2023-10-06T05:05:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.120+0000 7fb46359e700 1 -- 172.21.15.55:0/3214214216 wait complete. 2023-10-06T05:05:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.121+0000 7fb46359e700 1 Processor -- start 2023-10-06T05:05:39.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.121+0000 7fb46359e700 1 -- start start 2023-10-06T05:05:39.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.122+0000 7fb46359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4640cf480 0x7fb464151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.122+0000 7fb46359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4641595d0 0x7fb464158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.122+0000 7fb46359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46415d930 0x7fb464156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.122+0000 7fb46359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464152290 con 0x7fb4640cf480 2023-10-06T05:05:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.122+0000 7fb46359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb464158200 con 0x7fb4641595d0 2023-10-06T05:05:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.122+0000 7fb46359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4641575f0 con 0x7fb46415d930 2023-10-06T05:05:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.123+0000 7fb46259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4640cf480 0x7fb464151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.123+0000 7fb46259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4640cf480 0x7fb464151d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39168/0 (socket says 172.21.15.55:39168) 2023-10-06T05:05:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.123+0000 7fb46259c700 1 -- 172.21.15.55:0/2614072273 learned_addr learned my addr 172.21.15.55:0/2614072273 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:39.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.123+0000 7fb461d9b700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4641595d0 0x7fb464158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:39.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.123+0000 7fb462d9d700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46415d930 0x7fb464156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:39.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.123+0000 7fb46259c700 1 -- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46415d930 msgr2=0x7fb464156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:39.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.123+0000 7fb46259c700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46415d930 0x7fb464156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:39.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.123+0000 7fb46259c700 1 -- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4641595d0 msgr2=0x7fb464158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:39.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.123+0000 7fb46259c700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4641595d0 0x7fb464158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:39.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.123+0000 7fb46259c700 1 -- 172.21.15.55:0/2614072273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4540093c0 con 0x7fb4640cf480 2023-10-06T05:05:39.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.123+0000 7fb461d9b700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4641595d0 0x7fb464158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:39.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.123+0000 7fb462d9d700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46415d930 0x7fb464156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:39.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.123+0000 7fb46259c700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4640cf480 0x7fb464151d50 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fb45400ec80 tx=0x7fb454009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:39.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.124+0000 7fb4527fc700 1 -- 172.21.15.55:0/2614072273 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454005a00 con 0x7fb4640cf480 2023-10-06T05:05:39.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.124+0000 7fb4527fc700 1 -- 172.21.15.55:0/2614072273 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb454005b60 con 0x7fb4640cf480 2023-10-06T05:05:39.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.124+0000 7fb4527fc700 1 -- 172.21.15.55:0/2614072273 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45400c9d0 con 0x7fb4640cf480 2023-10-06T05:05:39.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.124+0000 7fb46359e700 1 -- 172.21.15.55:0/2614072273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45c009a20 con 0x7fb4640cf480 2023-10-06T05:05:39.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.124+0000 7fb46359e700 1 -- 172.21.15.55:0/2614072273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb464157bb0 con 0x7fb4640cf480 2023-10-06T05:05:39.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.125+0000 7fb46359e700 1 -- 172.21.15.55:0/2614072273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4641584e0 con 0x7fb4640cf480 2023-10-06T05:05:39.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.126+0000 7fb4527fc700 1 -- 172.21.15.55:0/2614072273 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb45400cd80 con 0x7fb4640cf480 2023-10-06T05:05:39.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.127+0000 7fb4527fc700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb444066f30 0x7fb4440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:39.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.127+0000 7fb4527fc700 1 -- 172.21.15.55:0/2614072273 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5996+0+0 (secure 0 0 0) 0x7fb454085130 con 0x7fb4640cf480 2023-10-06T05:05:39.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.128+0000 7fb461d9b700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb444066f30 0x7fb4440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:39.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.128+0000 7fb461d9b700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb444066f30 0x7fb4440693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb4640d14d0 tx=0x7fb458009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:39.135 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.131+0000 7fb4527fc700 1 -- 172.21.15.55:0/2614072273 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb454052260 con 0x7fb4640cf480 2023-10-06T05:05:39.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.333+0000 7fb46359e700 1 -- 172.21.15.55:0/2614072273 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb4640cf2c0 con 0x7fb4640cf480 2023-10-06T05:05:39.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.906+0000 7fb4527fc700 1 -- 172.21.15.55:0/2614072273 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v191) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb454055880 con 0x7fb4640cf480 2023-10-06T05:05:39.922 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.915+0000 7fb46359e700 1 -- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb444066f30 msgr2=0x7fb4440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:39.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.915+0000 7fb46359e700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb444066f30 0x7fb4440693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb4640d14d0 tx=0x7fb458009040 comp rx=0 tx=0).stop 2023-10-06T05:05:39.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.915+0000 7fb46359e700 1 -- 172.21.15.55:0/2614072273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4640cf480 msgr2=0x7fb464151d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:39.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.915+0000 7fb46359e700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4640cf480 0x7fb464151d50 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fb45400ec80 tx=0x7fb454009120 comp rx=0 tx=0).stop 2023-10-06T05:05:39.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.916+0000 7fb46359e700 1 -- 172.21.15.55:0/2614072273 shutdown_connections 2023-10-06T05:05:39.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.916+0000 7fb46359e700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb444066f30 0x7fb4440693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:39.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.916+0000 7fb46359e700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb46415d930 0x7fb464156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:39.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.916+0000 7fb46359e700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb4640cf480 0x7fb464151d50 secure :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fb45400ec80 tx=0x7fb454009120 comp rx=0 tx=0).stop 2023-10-06T05:05:39.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.916+0000 7fb46359e700 1 --2- 172.21.15.55:0/2614072273 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb4641595d0 0x7fb464158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:39.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.916+0000 7fb46359e700 1 -- 172.21.15.55:0/2614072273 >> 172.21.15.55:0/2614072273 conn(0x7fb4640d05f0 msgr2=0x7fb464152990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:39.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.921+0000 7fb46359e700 1 -- 172.21.15.55:0/2614072273 shutdown_connections 2023-10-06T05:05:39.926 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:39.923+0000 7fb46359e700 1 -- 172.21.15.55:0/2614072273 wait complete. 2023-10-06T05:05:39.932 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2023-10-06T05:05:39.951 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:05:40.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.142+0000 7f3b5359e700 1 Processor -- start 2023-10-06T05:05:40.145 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.142+0000 7f3b5359e700 1 -- start start 2023-10-06T05:05:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.144+0000 7f3b5359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3b540cb1a0 0x7f3b5415a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.144+0000 7f3b5359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3b540ae0d0 0x7f3b540ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.144+0000 7f3b5359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5406ebd0 con 0x7f3b5415a8a0 2023-10-06T05:05:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.144+0000 7f3b5359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5415b2e0 con 0x7f3b540cb1a0 2023-10-06T05:05:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.144+0000 7f3b5359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b540aea10 con 0x7f3b540ae0d0 2023-10-06T05:05:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.144+0000 7f3b5259c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f3b540cb1a0 0x7f3b5415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.144+0000 7f3b51d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3b5415a8a0 0x7f3b5406f060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46074/0 (socket says 172.21.15.55:46074) 2023-10-06T05:05:40.148 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.144+0000 7f3b51d9b700 1 -- 172.21.15.55:0/885762429 learned_addr learned my addr 172.21.15.55:0/885762429 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:40.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.144+0000 7f3b52d9d700 1 --2- 172.21.15.55:0/885762429 >> v2:172.21.15.163:3300/0 conn(0x7f3b540ae0d0 0x7f3b540ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:40.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.145+0000 7f3b5259c700 1 -- 172.21.15.55:0/885762429 >> v2:172.21.15.163:3300/0 conn(0x7f3b540ae0d0 msgr2=0x7f3b540ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:40.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.145+0000 7f3b5259c700 1 --2- 172.21.15.55:0/885762429 >> v2:172.21.15.163:3300/0 conn(0x7f3b540ae0d0 0x7f3b540ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:40.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.145+0000 7f3b5259c700 1 -- 172.21.15.55:0/885762429 >> v1:172.21.15.55:6789/0 conn(0x7f3b5415a8a0 legacy=0x7f3b5406f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:40.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.145+0000 7f3b5259c700 1 -- 172.21.15.55:0/885762429 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5415ad20 con 0x7f3b540cb1a0 2023-10-06T05:05:40.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.145+0000 7f3b5159a700 1 -- 172.21.15.55:0/885762429 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11670923 0 0) 0x7f3b5406ebd0 con 0x7f3b5415a8a0 2023-10-06T05:05:40.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.145+0000 7f3b5259c700 1 --2- 172.21.15.55:0/885762429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b540cb1a0 0x7f3b5415a360 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f3b540c67b0 tx=0x7f3b440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54eb34f2e844aa6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:40.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.145+0000 7f3b5159a700 1 -- 172.21.15.55:0/885762429 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4400b740 con 0x7f3b540cb1a0 2023-10-06T05:05:40.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.145+0000 7f3b5159a700 1 -- 172.21.15.55:0/885762429 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4400b8a0 con 0x7f3b540cb1a0 2023-10-06T05:05:40.149 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.145+0000 7f3b5159a700 1 -- 172.21.15.55:0/885762429 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4400bc40 con 0x7f3b540cb1a0 2023-10-06T05:05:40.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.146+0000 7f3b5359e700 1 -- 172.21.15.55:0/885762429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b540cb1a0 msgr2=0x7f3b5415a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:40.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.146+0000 7f3b5359e700 1 --2- 172.21.15.55:0/885762429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b540cb1a0 0x7f3b5415a360 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f3b540c67b0 tx=0x7f3b440091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:40.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.146+0000 7f3b5359e700 1 -- 172.21.15.55:0/885762429 shutdown_connections 2023-10-06T05:05:40.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.146+0000 7f3b5359e700 1 --2- 172.21.15.55:0/885762429 >> v2:172.21.15.163:3300/0 conn(0x7f3b540ae0d0 0x7f3b540ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:40.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.146+0000 7f3b5359e700 1 --2- 172.21.15.55:0/885762429 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b540cb1a0 0x7f3b5415a360 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:40.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.146+0000 7f3b5359e700 1 -- 172.21.15.55:0/885762429 >> 172.21.15.55:0/885762429 conn(0x7f3b540cc1e0 msgr2=0x7f3b540cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:40.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.146+0000 7f3b5359e700 1 -- 172.21.15.55:0/885762429 shutdown_connections 2023-10-06T05:05:40.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.146+0000 7f3b5359e700 1 -- 172.21.15.55:0/885762429 wait complete. 2023-10-06T05:05:40.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.147+0000 7f3b5359e700 1 Processor -- start 2023-10-06T05:05:40.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.147+0000 7f3b5359e700 1 -- start start 2023-10-06T05:05:40.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.148+0000 7f3b5359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b540ae0d0 0x7f3b540b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:40.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.148+0000 7f3b5359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b540cb1a0 0x7f3b54151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:40.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.148+0000 7f3b5359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b54151580 0x7f3b54158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:40.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.148+0000 7f3b5359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b54159a30 con 0x7f3b540ae0d0 2023-10-06T05:05:40.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.148+0000 7f3b5359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b54151a00 con 0x7f3b54151580 2023-10-06T05:05:40.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.148+0000 7f3b5359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5416aa00 con 0x7f3b540cb1a0 2023-10-06T05:05:40.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.149+0000 7f3b5259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b540ae0d0 0x7f3b540b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:40.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.149+0000 7f3b5259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b540ae0d0 0x7f3b540b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39186/0 (socket says 172.21.15.55:39186) 2023-10-06T05:05:40.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.149+0000 7f3b5259c700 1 -- 172.21.15.55:0/2665331505 learned_addr learned my addr 172.21.15.55:0/2665331505 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:40.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.149+0000 7f3b52d9d700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b54151580 0x7f3b54158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:40.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.149+0000 7f3b51d9b700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b540cb1a0 0x7f3b54151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:40.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.149+0000 7f3b5259c700 1 -- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b540cb1a0 msgr2=0x7f3b54151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:40.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.149+0000 7f3b5259c700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b540cb1a0 0x7f3b54151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:40.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.149+0000 7f3b5259c700 1 -- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b54151580 msgr2=0x7f3b54158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:40.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.149+0000 7f3b5259c700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b54151580 0x7f3b54158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:40.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.149+0000 7f3b5259c700 1 -- 172.21.15.55:0/2665331505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b44009420 con 0x7f3b540ae0d0 2023-10-06T05:05:40.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.149+0000 7f3b52d9d700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b54151580 0x7f3b54158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:40.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.149+0000 7f3b5259c700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b540ae0d0 0x7f3b540b0060 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400b0e0 tx=0x7f3b44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:40.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.150+0000 7f3b427fc700 1 -- 172.21.15.55:0/2665331505 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44011420 con 0x7f3b540ae0d0 2023-10-06T05:05:40.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.150+0000 7f3b427fc700 1 -- 172.21.15.55:0/2665331505 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b44011580 con 0x7f3b540ae0d0 2023-10-06T05:05:40.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.150+0000 7f3b5359e700 1 -- 172.21.15.55:0/2665331505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b54169c40 con 0x7f3b540ae0d0 2023-10-06T05:05:40.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.150+0000 7f3b427fc700 1 -- 172.21.15.55:0/2665331505 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44011940 con 0x7f3b540ae0d0 2023-10-06T05:05:40.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.150+0000 7f3b5359e700 1 -- 172.21.15.55:0/2665331505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5416a200 con 0x7f3b540ae0d0 2023-10-06T05:05:40.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.151+0000 7f3b5359e700 1 -- 172.21.15.55:0/2665331505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5406c450 con 0x7f3b540ae0d0 2023-10-06T05:05:40.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.152+0000 7f3b427fc700 1 -- 172.21.15.55:0/2665331505 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b44011cb0 con 0x7f3b540ae0d0 2023-10-06T05:05:40.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.153+0000 7f3b427fc700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3b34066f30 0x7f3b340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:40.156 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.153+0000 7f3b427fc700 1 -- 172.21.15.55:0/2665331505 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f3b44086700 con 0x7f3b540ae0d0 2023-10-06T05:05:40.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.153+0000 7f3b51d9b700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3b34066f30 0x7f3b340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:40.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.154+0000 7f3b51d9b700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3b34066f30 0x7f3b340693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3b54155130 tx=0x7f3b48009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:40.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.156+0000 7f3b427fc700 1 -- 172.21.15.55:0/2665331505 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b440536b0 con 0x7f3b540ae0d0 2023-10-06T05:05:40.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.403+0000 7f3b5359e700 1 -- 172.21.15.55:0/2665331505 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3b5415b000 con 0x7f3b540ae0d0 2023-10-06T05:05:40.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.911+0000 7f3b427fc700 1 -- 172.21.15.55:0/2665331505 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v92) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3b44056cd0 con 0x7f3b540ae0d0 2023-10-06T05:05:40.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.920+0000 7f3b5359e700 1 -- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3b34066f30 msgr2=0x7f3b340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:40.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.920+0000 7f3b5359e700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3b34066f30 0x7f3b340693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3b54155130 tx=0x7f3b48009040 comp rx=0 tx=0).stop 2023-10-06T05:05:40.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.920+0000 7f3b5359e700 1 -- 172.21.15.55:0/2665331505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b540ae0d0 msgr2=0x7f3b540b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:40.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.920+0000 7f3b5359e700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b540ae0d0 0x7f3b540b0060 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400b0e0 tx=0x7f3b44017000 comp rx=0 tx=0).stop 2023-10-06T05:05:40.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.921+0000 7f3b5359e700 1 -- 172.21.15.55:0/2665331505 shutdown_connections 2023-10-06T05:05:40.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.921+0000 7f3b5359e700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3b34066f30 0x7f3b340693e0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3b54155130 tx=0x7f3b48009040 comp rx=0 tx=0).stop 2023-10-06T05:05:40.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.921+0000 7f3b5359e700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3b54151580 0x7f3b54158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:40.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.921+0000 7f3b5359e700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3b540ae0d0 0x7f3b540b0060 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:40.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.921+0000 7f3b5359e700 1 --2- 172.21.15.55:0/2665331505 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3b540cb1a0 0x7f3b54151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:40.924 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.921+0000 7f3b5359e700 1 -- 172.21.15.55:0/2665331505 >> 172.21.15.55:0/2665331505 conn(0x7f3b540cc1e0 msgr2=0x7f3b5406a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:40.927 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.924+0000 7f3b5359e700 1 -- 172.21.15.55:0/2665331505 shutdown_connections 2023-10-06T05:05:40.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:40.927+0000 7f3b5359e700 1 -- 172.21.15.55:0/2665331505 wait complete. 2023-10-06T05:05:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 23 and data pool 24 2023-10-06T05:05:40.964 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:05:41.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.150+0000 7f1fa6d9d700 1 Processor -- start 2023-10-06T05:05:41.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.150+0000 7f1fa6d9d700 1 -- start start 2023-10-06T05:05:41.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.151+0000 7f1fa6d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f1fa0159420 0x7f1fa00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:41.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.151+0000 7f1fa6d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa01597f0 0x7f1fa00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:41.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.151+0000 7f1fa6d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa00b11d0 con 0x7f1fa015d230 2023-10-06T05:05:41.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.151+0000 7f1fa6d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa00b14b0 con 0x7f1fa0159420 2023-10-06T05:05:41.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.151+0000 7f1fa6d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00b1790 con 0x7f1fa01597f0 2023-10-06T05:05:41.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.152+0000 7f1fa559a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa01597f0 0x7f1fa00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:41.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.152+0000 7f1fa659c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f1fa015d230 0x7f1fa00b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:40774/0 (socket says 172.21.15.55:40774) 2023-10-06T05:05:41.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.152+0000 7f1fa659c700 1 -- 172.21.15.55:0/2624751404 learned_addr learned my addr 172.21.15.55:0/2624751404 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:41.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.152+0000 7f1fa5d9b700 1 --2- 172.21.15.55:0/2624751404 >> v2:172.21.15.163:3300/0 conn(0x7f1fa0159420 0x7f1fa00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:41.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.152+0000 7f1fa4d99700 1 -- 172.21.15.55:0/2624751404 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195869762 0 0) 0x7f1fa00b11d0 con 0x7f1fa015d230 2023-10-06T05:05:41.157 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.152+0000 7f1fa4d99700 1 -- 172.21.15.55:0/2624751404 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f8c003570 con 0x7f1fa015d230 2023-10-06T05:05:41.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.152+0000 7f1fa5d9b700 1 -- 172.21.15.55:0/2624751404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa01597f0 msgr2=0x7f1fa00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:41.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.152+0000 7f1fa5d9b700 1 --2- 172.21.15.55:0/2624751404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa01597f0 0x7f1fa00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.152+0000 7f1fa5d9b700 1 -- 172.21.15.55:0/2624751404 >> v1:172.21.15.163:6789/0 conn(0x7f1fa015d230 legacy=0x7f1fa00b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:41.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.152+0000 7f1fa5d9b700 1 -- 172.21.15.55:0/2624751404 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa00b1a70 con 0x7f1fa0159420 2023-10-06T05:05:41.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.152+0000 7f1fa559a700 1 --2- 172.21.15.55:0/2624751404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa01597f0 0x7f1fa00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:41.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.153+0000 7f1fa5d9b700 1 --2- 172.21.15.55:0/2624751404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa0159420 0x7f1fa00b5f40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0098e0 tx=0x7f1f9c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88278b3297ab6b4d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:41.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.153+0000 7f1fa4d99700 1 -- 172.21.15.55:0/2624751404 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c011630 con 0x7f1fa0159420 2023-10-06T05:05:41.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.153+0000 7f1fa4d99700 1 -- 172.21.15.55:0/2624751404 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c011790 con 0x7f1fa0159420 2023-10-06T05:05:41.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.153+0000 7f1fa4d99700 1 -- 172.21.15.55:0/2624751404 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c011b10 con 0x7f1fa0159420 2023-10-06T05:05:41.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.154+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/2624751404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa0159420 msgr2=0x7f1fa00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:41.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.154+0000 7f1fa6d9d700 1 --2- 172.21.15.55:0/2624751404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa0159420 0x7f1fa00b5f40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0098e0 tx=0x7f1f9c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.154+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/2624751404 shutdown_connections 2023-10-06T05:05:41.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.154+0000 7f1fa6d9d700 1 --2- 172.21.15.55:0/2624751404 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa0159420 0x7f1fa00b5f40 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.154+0000 7f1fa6d9d700 1 --2- 172.21.15.55:0/2624751404 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa01597f0 0x7f1fa00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.154+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/2624751404 >> 172.21.15.55:0/2624751404 conn(0x7f1fa00d0660 msgr2=0x7f1fa00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:41.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.154+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/2624751404 shutdown_connections 2023-10-06T05:05:41.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.154+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/2624751404 wait complete. 2023-10-06T05:05:41.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.155+0000 7f1fa6d9d700 1 Processor -- start 2023-10-06T05:05:41.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.155+0000 7f1fa6d9d700 1 -- start start 2023-10-06T05:05:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.156+0000 7f1fa6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fa0159420 0x7f1fa016a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.156+0000 7f1fa6d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa01597f0 0x7f1fa0145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.156+0000 7f1fa6d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa015d230 0x7f1fa00cf500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.156+0000 7f1fa6d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa00cac90 con 0x7f1fa0159420 2023-10-06T05:05:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.156+0000 7f1fa6d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa016cf80 con 0x7f1fa01597f0 2023-10-06T05:05:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.156+0000 7f1fa6d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa016ad60 con 0x7f1fa015d230 2023-10-06T05:05:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.157+0000 7f1fa5d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fa0159420 0x7f1fa016a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.157+0000 7f1fa5d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fa0159420 0x7f1fa016a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39198/0 (socket says 172.21.15.55:39198) 2023-10-06T05:05:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.157+0000 7f1fa5d9b700 1 -- 172.21.15.55:0/3796007829 learned_addr learned my addr 172.21.15.55:0/3796007829 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:41.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.157+0000 7f1fa659c700 1 --2- 172.21.15.55:0/3796007829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa015d230 0x7f1fa00cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.157+0000 7f1fa559a700 1 --2- 172.21.15.55:0/3796007829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa01597f0 0x7f1fa0145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.157+0000 7f1fa5d9b700 1 -- 172.21.15.55:0/3796007829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa015d230 msgr2=0x7f1fa00cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.157+0000 7f1fa5d9b700 1 --2- 172.21.15.55:0/3796007829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa015d230 0x7f1fa00cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.157+0000 7f1fa5d9b700 1 -- 172.21.15.55:0/3796007829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa01597f0 msgr2=0x7f1fa0145a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.157+0000 7f1fa5d9b700 1 --2- 172.21.15.55:0/3796007829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa01597f0 0x7f1fa0145a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.157+0000 7f1fa5d9b700 1 -- 172.21.15.55:0/3796007829 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f90009420 con 0x7f1fa0159420 2023-10-06T05:05:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.157+0000 7f1fa5d9b700 1 --2- 172.21.15.55:0/3796007829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fa0159420 0x7f1fa016a5b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0089340 tx=0x7f1f9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.158+0000 7f1f95ffb700 1 -- 172.21.15.55:0/3796007829 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c011900 con 0x7f1fa0159420 2023-10-06T05:05:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.158+0000 7f1f95ffb700 1 -- 172.21.15.55:0/3796007829 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c011a60 con 0x7f1fa0159420 2023-10-06T05:05:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.158+0000 7f1f95ffb700 1 -- 172.21.15.55:0/3796007829 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c01a680 con 0x7f1fa0159420 2023-10-06T05:05:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.158+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/3796007829 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9c009420 con 0x7f1fa0159420 2023-10-06T05:05:41.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.158+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/3796007829 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa00c6a80 con 0x7f1fa0159420 2023-10-06T05:05:41.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.159+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/3796007829 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa004c3c0 con 0x7f1fa0159420 2023-10-06T05:05:41.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.160+0000 7f1f95ffb700 1 -- 172.21.15.55:0/3796007829 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1f9c01aa70 con 0x7f1fa0159420 2023-10-06T05:05:41.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.161+0000 7f1f95ffb700 1 --2- 172.21.15.55:0/3796007829 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1f84066f30 0x7f1f840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:41.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.161+0000 7f1f95ffb700 1 -- 172.21.15.55:0/3796007829 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f1f9c086430 con 0x7f1fa0159420 2023-10-06T05:05:41.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.161+0000 7f1fa559a700 1 --2- 172.21.15.55:0/3796007829 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1f84066f30 0x7f1f840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:41.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.162+0000 7f1fa559a700 1 --2- 172.21.15.55:0/3796007829 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1f84066f30 0x7f1f840693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00b25d0 tx=0x7f1f900091a0 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:41.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.165+0000 7f1f95ffb700 1 -- 172.21.15.55:0/3796007829 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f9c053300 con 0x7f1fa0159420 2023-10-06T05:05:41.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.367+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/3796007829 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fa00b1d50 con 0x7f1fa0159420 2023-10-06T05:05:41.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.368+0000 7f1f95ffb700 1 -- 172.21.15.55:0/3796007829 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+17582 (secure 0 0 0) 0x7f1f9c056920 con 0x7f1fa0159420 2023-10-06T05:05:41.371 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:41.371 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":192,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:05:40.405383+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:05:38.605485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:05:39.377918+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:05:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.370+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/3796007829 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1f84066f30 msgr2=0x7f1f840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.370+0000 7f1fa6d9d700 1 --2- 172.21.15.55:0/3796007829 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1f84066f30 0x7f1f840693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00b25d0 tx=0x7f1f900091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.370+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/3796007829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fa0159420 msgr2=0x7f1fa016a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.370+0000 7f1fa6d9d700 1 --2- 172.21.15.55:0/3796007829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fa0159420 0x7f1fa016a5b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0089340 tx=0x7f1f9c017000 comp rx=0 tx=0).stop 2023-10-06T05:05:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.371+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/3796007829 shutdown_connections 2023-10-06T05:05:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.371+0000 7f1fa6d9d700 1 --2- 172.21.15.55:0/3796007829 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1f84066f30 0x7f1f840693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.371+0000 7f1fa6d9d700 1 --2- 172.21.15.55:0/3796007829 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1fa015d230 0x7f1fa00cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.371+0000 7f1fa6d9d700 1 --2- 172.21.15.55:0/3796007829 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1fa0159420 0x7f1fa016a5b0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.371+0000 7f1fa6d9d700 1 --2- 172.21.15.55:0/3796007829 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1fa01597f0 0x7f1fa0145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.371+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/3796007829 >> 172.21.15.55:0/3796007829 conn(0x7f1fa00d0660 msgr2=0x7f1fa015bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:41.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.371+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/3796007829 shutdown_connections 2023-10-06T05:05:41.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.371+0000 7f1fa6d9d700 1 -- 172.21.15.55:0/3796007829 wait complete. 2023-10-06T05:05:41.394 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:05:41.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.577+0000 7f9e0119d700 1 Processor -- start 2023-10-06T05:05:41.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.578+0000 7f9e0119d700 1 -- start start 2023-10-06T05:05:41.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.579+0000 7f9e0119d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dfc156310 0x7f9dfc0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:41.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.579+0000 7f9e0119d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc0ca860 con 0x7f9dfc0ae710 2023-10-06T05:05:41.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.579+0000 7f9e0119d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dfc0cab40 con 0x7f9dfc1566e0 2023-10-06T05:05:41.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.579+0000 7f9e0119d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc0cae20 con 0x7f9dfc156310 2023-10-06T05:05:41.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.579+0000 7f9e0099c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9dfc0ae710 0x7f9dfc0c6830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46090/0 (socket says 172.21.15.55:46090) 2023-10-06T05:05:41.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.579+0000 7f9e0099c700 1 -- 172.21.15.55:0/3856321282 learned_addr learned my addr 172.21.15.55:0/3856321282 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:41.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.580+0000 7f9dfbfff700 1 --2- 172.21.15.55:0/3856321282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dfc156310 0x7f9dfc0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:41.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.580+0000 7f9dfb7fe700 1 -- 172.21.15.55:0/3856321282 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162068200 0 0) 0x7f9dfc0ca860 con 0x7f9dfc0ae710 2023-10-06T05:05:41.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.580+0000 7f9dfb7fe700 1 -- 172.21.15.55:0/3856321282 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9de8003570 con 0x7f9dfc0ae710 2023-10-06T05:05:41.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.580+0000 7f9dfb7fe700 1 -- 172.21.15.55:0/3856321282 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178335585 0 0) 0x7f9dfc0cab40 con 0x7f9dfc1566e0 2023-10-06T05:05:41.586 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.580+0000 7f9dfb7fe700 1 -- 172.21.15.55:0/3856321282 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dfc0ca860 con 0x7f9dfc1566e0 2023-10-06T05:05:41.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.580+0000 7f9dfbfff700 1 -- 172.21.15.55:0/3856321282 >> v1:172.21.15.163:6789/0 conn(0x7f9dfc1566e0 legacy=0x7f9dfc0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:41.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.580+0000 7f9dfbfff700 1 -- 172.21.15.55:0/3856321282 >> v1:172.21.15.55:6789/0 conn(0x7f9dfc0ae710 legacy=0x7f9dfc0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:41.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.580+0000 7f9dfbfff700 1 -- 172.21.15.55:0/3856321282 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc0cb100 con 0x7f9dfc156310 2023-10-06T05:05:41.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.580+0000 7f9dfbfff700 1 --2- 172.21.15.55:0/3856321282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dfc156310 0x7f9dfc0c9dc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009910 tx=0x7f9dec0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87e127247e1f54f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:41.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.581+0000 7f9dfb7fe700 1 -- 172.21.15.55:0/3856321282 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec011630 con 0x7f9dfc156310 2023-10-06T05:05:41.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.581+0000 7f9dfb7fe700 1 -- 172.21.15.55:0/3856321282 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec011790 con 0x7f9dfc156310 2023-10-06T05:05:41.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.581+0000 7f9dfb7fe700 1 -- 172.21.15.55:0/3856321282 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec011b30 con 0x7f9dfc156310 2023-10-06T05:05:41.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.581+0000 7f9e0119d700 1 -- 172.21.15.55:0/3856321282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dfc156310 msgr2=0x7f9dfc0c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:41.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.581+0000 7f9e0119d700 1 --2- 172.21.15.55:0/3856321282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dfc156310 0x7f9dfc0c9dc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009910 tx=0x7f9dec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.581+0000 7f9e0119d700 1 -- 172.21.15.55:0/3856321282 shutdown_connections 2023-10-06T05:05:41.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.581+0000 7f9e0119d700 1 --2- 172.21.15.55:0/3856321282 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dfc156310 0x7f9dfc0c9dc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.581+0000 7f9e0119d700 1 -- 172.21.15.55:0/3856321282 >> 172.21.15.55:0/3856321282 conn(0x7f9dfc0cc1f0 msgr2=0x7f9dfc0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:41.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.581+0000 7f9e0119d700 1 -- 172.21.15.55:0/3856321282 shutdown_connections 2023-10-06T05:05:41.588 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.581+0000 7f9e0119d700 1 -- 172.21.15.55:0/3856321282 wait complete. 2023-10-06T05:05:41.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.583+0000 7f9e0119d700 1 Processor -- start 2023-10-06T05:05:41.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.583+0000 7f9e0119d700 1 -- start start 2023-10-06T05:05:41.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.584+0000 7f9e0119d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dfc0ae710 0x7f9dfc169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:41.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.584+0000 7f9e0119d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9dfc156310 0x7f9dfc159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:41.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.584+0000 7f9e0119d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9dfc1566e0 0x7f9dfc1409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:41.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.584+0000 7f9e0119d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc141440 con 0x7f9dfc1566e0 2023-10-06T05:05:41.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.584+0000 7f9e0119d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc15ac70 con 0x7f9dfc156310 2023-10-06T05:05:41.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.584+0000 7f9e0119d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc15a060 con 0x7f9dfc0ae710 2023-10-06T05:05:41.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.584+0000 7f9e0099c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9dfc1566e0 0x7f9dfc1409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:41.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.584+0000 7f9e0099c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9dfc1566e0 0x7f9dfc1409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39214/0 (socket says 172.21.15.55:39214) 2023-10-06T05:05:41.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.584+0000 7f9e0099c700 1 -- 172.21.15.55:0/3186729011 learned_addr learned my addr 172.21.15.55:0/3186729011 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:41.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.584+0000 7f9dfbfff700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dfc0ae710 0x7f9dfc169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:41.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.584+0000 7f9df3fff700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9dfc156310 0x7f9dfc159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:41.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.585+0000 7f9e0099c700 1 -- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dfc0ae710 msgr2=0x7f9dfc169a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:41.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.585+0000 7f9e0099c700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dfc0ae710 0x7f9dfc169a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.585+0000 7f9e0099c700 1 -- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9dfc156310 msgr2=0x7f9dfc159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:41.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.585+0000 7f9e0099c700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9dfc156310 0x7f9dfc159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.585+0000 7f9e0099c700 1 -- 172.21.15.55:0/3186729011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec009420 con 0x7f9dfc1566e0 2023-10-06T05:05:41.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.585+0000 7f9dfbfff700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dfc0ae710 0x7f9dfc169a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:41.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.585+0000 7f9df3fff700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9dfc156310 0x7f9dfc159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:41.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.585+0000 7f9e0099c700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9dfc1566e0 0x7f9dfc1409c0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f9df4001460 tx=0x7f9df4001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:41.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.585+0000 7f9df8ff9700 1 -- 172.21.15.55:0/3186729011 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df400ce80 con 0x7f9dfc1566e0 2023-10-06T05:05:41.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.585+0000 7f9df8ff9700 1 -- 172.21.15.55:0/3186729011 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df400ab70 con 0x7f9dfc1566e0 2023-10-06T05:05:41.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.585+0000 7f9df8ff9700 1 -- 172.21.15.55:0/3186729011 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df40070f0 con 0x7f9dfc1566e0 2023-10-06T05:05:41.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.585+0000 7f9e0119d700 1 -- 172.21.15.55:0/3186729011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc15a340 con 0x7f9dfc1566e0 2023-10-06T05:05:41.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.585+0000 7f9e0119d700 1 -- 172.21.15.55:0/3186729011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc15a900 con 0x7f9dfc1566e0 2023-10-06T05:05:41.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.587+0000 7f9e0119d700 1 -- 172.21.15.55:0/3186729011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dfc15ae20 con 0x7f9dfc1566e0 2023-10-06T05:05:41.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.588+0000 7f9df8ff9700 1 -- 172.21.15.55:0/3186729011 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9df40074a0 con 0x7f9dfc1566e0 2023-10-06T05:05:41.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.589+0000 7f9df8ff9700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9ddc066f30 0x7f9ddc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:41.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.589+0000 7f9df8ff9700 1 -- 172.21.15.55:0/3186729011 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f9df4084d00 con 0x7f9dfc1566e0 2023-10-06T05:05:41.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.589+0000 7f9dfbfff700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9ddc066f30 0x7f9ddc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:41.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.590+0000 7f9dfbfff700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9ddc066f30 0x7f9ddc0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc16abf0 tx=0x7f9dec017000 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:41.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.593+0000 7f9df8ff9700 1 -- 172.21.15.55:0/3186729011 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9dfc15ae20 con 0x7f9dfc1566e0 2023-10-06T05:05:41.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.797+0000 7f9e0119d700 1 -- 172.21.15.55:0/3186729011 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dfc15b000 con 0x7f9dfc1566e0 2023-10-06T05:05:41.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.798+0000 7f9df8ff9700 1 -- 172.21.15.55:0/3186729011 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+17582 (secure 0 0 0) 0x7f9df4051c50 con 0x7f9dfc1566e0 2023-10-06T05:05:41.884 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:41.884 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":192,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:05:40.405383+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:05:38.605485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:05:39.377918+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:05:41.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.801+0000 7f9e0119d700 1 -- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9ddc066f30 msgr2=0x7f9ddc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.801+0000 7f9e0119d700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9ddc066f30 0x7f9ddc0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc16abf0 tx=0x7f9dec017000 comp rx=0 tx=0).stop 2023-10-06T05:05:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.801+0000 7f9e0119d700 1 -- 172.21.15.55:0/3186729011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9dfc1566e0 msgr2=0x7f9dfc1409c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.801+0000 7f9e0119d700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9dfc1566e0 0x7f9dfc1409c0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f9df4001460 tx=0x7f9df4001ac0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.801+0000 7f9e0119d700 1 -- 172.21.15.55:0/3186729011 shutdown_connections 2023-10-06T05:05:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.801+0000 7f9e0119d700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9ddc066f30 0x7f9ddc0693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.801+0000 7f9e0119d700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9dfc1566e0 0x7f9dfc1409c0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.801+0000 7f9e0119d700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9dfc0ae710 0x7f9dfc169a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.801+0000 7f9e0119d700 1 --2- 172.21.15.55:0/3186729011 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9dfc156310 0x7f9dfc159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.801+0000 7f9e0119d700 1 -- 172.21.15.55:0/3186729011 >> 172.21.15.55:0/3186729011 conn(0x7f9dfc0cc1f0 msgr2=0x7f9dfc149580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:41.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.801+0000 7f9e0119d700 1 -- 172.21.15.55:0/3186729011 shutdown_connections 2023-10-06T05:05:41.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:41.801+0000 7f9e0119d700 1 -- 172.21.15.55:0/3186729011 wait complete. 2023-10-06T05:05:41.887 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:05:42.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.078+0000 7f3c4ffff700 1 Processor -- start 2023-10-06T05:05:42.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.078+0000 7f3c4ffff700 1 -- start start 2023-10-06T05:05:42.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.079+0000 7f3c4ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3c5015a190 0x7f3c501540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:42.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.079+0000 7f3c4ffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c50158640 con 0x7f3c50153880 2023-10-06T05:05:42.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.079+0000 7f3c4ffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c50157a30 con 0x7f3c50153c50 2023-10-06T05:05:42.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.079+0000 7f3c4ffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c501573c0 con 0x7f3c5015a190 2023-10-06T05:05:42.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.080+0000 7f3c4effd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3c50153880 0x7f3c500b0290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:46092/0 (socket says 172.21.15.55:46092) 2023-10-06T05:05:42.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.080+0000 7f3c4effd700 1 -- 172.21.15.55:0/314049469 learned_addr learned my addr 172.21.15.55:0/314049469 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:42.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.080+0000 7f3c4f7fe700 1 --2- 172.21.15.55:0/314049469 >> v2:172.21.15.163:3300/0 conn(0x7f3c5015a190 0x7f3c501540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:42.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.080+0000 7f3c4dffb700 1 -- 172.21.15.55:0/314049469 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306948890 0 0) 0x7f3c50158640 con 0x7f3c50153880 2023-10-06T05:05:42.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.080+0000 7f3c4dffb700 1 -- 172.21.15.55:0/314049469 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c38003570 con 0x7f3c50153880 2023-10-06T05:05:42.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.080+0000 7f3c4dffb700 1 -- 172.21.15.55:0/314049469 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373019587 0 0) 0x7f3c50157a30 con 0x7f3c50153c50 2023-10-06T05:05:42.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.080+0000 7f3c4dffb700 1 -- 172.21.15.55:0/314049469 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c50158640 con 0x7f3c50153c50 2023-10-06T05:05:42.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.080+0000 7f3c4f7fe700 1 -- 172.21.15.55:0/314049469 >> v1:172.21.15.163:6789/0 conn(0x7f3c50153c50 legacy=0x7f3c50159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:42.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.080+0000 7f3c4f7fe700 1 -- 172.21.15.55:0/314049469 >> v1:172.21.15.55:6789/0 conn(0x7f3c50153880 legacy=0x7f3c500b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:42.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.080+0000 7f3c4f7fe700 1 -- 172.21.15.55:0/314049469 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c50157d10 con 0x7f3c5015a190 2023-10-06T05:05:42.085 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.081+0000 7f3c4f7fe700 1 --2- 172.21.15.55:0/314049469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c5015a190 0x7f3c501540c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009e60 tx=0x7f3c48009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=504bc1c6fa10317c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:42.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.081+0000 7f3c4dffb700 1 -- 172.21.15.55:0/314049469 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4800e070 con 0x7f3c5015a190 2023-10-06T05:05:42.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.081+0000 7f3c4dffb700 1 -- 172.21.15.55:0/314049469 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4800bab0 con 0x7f3c5015a190 2023-10-06T05:05:42.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.081+0000 7f3c4dffb700 1 -- 172.21.15.55:0/314049469 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4800be50 con 0x7f3c5015a190 2023-10-06T05:05:42.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.081+0000 7f3c4ffff700 1 -- 172.21.15.55:0/314049469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c5015a190 msgr2=0x7f3c501540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:42.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.081+0000 7f3c4ffff700 1 --2- 172.21.15.55:0/314049469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c5015a190 0x7f3c501540c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009e60 tx=0x7f3c48009780 comp rx=0 tx=0).stop 2023-10-06T05:05:42.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.082+0000 7f3c4ffff700 1 -- 172.21.15.55:0/314049469 shutdown_connections 2023-10-06T05:05:42.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.082+0000 7f3c4ffff700 1 --2- 172.21.15.55:0/314049469 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c5015a190 0x7f3c501540c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:42.086 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.082+0000 7f3c4ffff700 1 -- 172.21.15.55:0/314049469 >> 172.21.15.55:0/314049469 conn(0x7f3c50068390 msgr2=0x7f3c50068790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:42.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.082+0000 7f3c4ffff700 1 -- 172.21.15.55:0/314049469 shutdown_connections 2023-10-06T05:05:42.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.082+0000 7f3c4ffff700 1 -- 172.21.15.55:0/314049469 wait complete. 2023-10-06T05:05:42.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.083+0000 7f3c4ffff700 1 Processor -- start 2023-10-06T05:05:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.083+0000 7f3c4ffff700 1 -- start start 2023-10-06T05:05:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.084+0000 7f3c4ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c50153880 0x7f3c500a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.084+0000 7f3c4ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c50153c50 0x7f3c5014b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.084+0000 7f3c4ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c5015a190 0x7f3c5014a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.084+0000 7f3c4ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5016ae60 con 0x7f3c50153880 2023-10-06T05:05:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.084+0000 7f3c4ffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c50167d90 con 0x7f3c50153c50 2023-10-06T05:05:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.084+0000 7f3c4ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c50168070 con 0x7f3c5015a190 2023-10-06T05:05:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.085+0000 7f3c4effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c50153880 0x7f3c500a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.085+0000 7f3c4effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c50153880 0x7f3c500a5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:39222/0 (socket says 172.21.15.55:39222) 2023-10-06T05:05:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.085+0000 7f3c4effd700 1 -- 172.21.15.55:0/3072828972 learned_addr learned my addr 172.21.15.55:0/3072828972 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:42.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.085+0000 7f3c4e7fc700 1 --2- 172.21.15.55:0/3072828972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c50153c50 0x7f3c5014b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:42.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.085+0000 7f3c4f7fe700 1 --2- 172.21.15.55:0/3072828972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c5015a190 0x7f3c5014a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:42.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.085+0000 7f3c4effd700 1 -- 172.21.15.55:0/3072828972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c5015a190 msgr2=0x7f3c5014a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:42.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.085+0000 7f3c4effd700 1 --2- 172.21.15.55:0/3072828972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c5015a190 0x7f3c5014a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:42.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.085+0000 7f3c4effd700 1 -- 172.21.15.55:0/3072828972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c50153c50 msgr2=0x7f3c5014b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:42.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.085+0000 7f3c4effd700 1 --2- 172.21.15.55:0/3072828972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c50153c50 0x7f3c5014b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:42.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.085+0000 7f3c4effd700 1 -- 172.21.15.55:0/3072828972 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c48009a20 con 0x7f3c50153880 2023-10-06T05:05:42.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.085+0000 7f3c4effd700 1 --2- 172.21.15.55:0/3072828972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c50153880 0x7f3c500a5710 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000c790 tx=0x7f3c40007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:42.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.086+0000 7f3c36ffd700 1 -- 172.21.15.55:0/3072828972 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c40005b30 con 0x7f3c50153880 2023-10-06T05:05:42.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.086+0000 7f3c36ffd700 1 -- 172.21.15.55:0/3072828972 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c40005cd0 con 0x7f3c50153880 2023-10-06T05:05:42.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.086+0000 7f3c4ffff700 1 -- 172.21.15.55:0/3072828972 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c50168350 con 0x7f3c50153880 2023-10-06T05:05:42.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.086+0000 7f3c36ffd700 1 -- 172.21.15.55:0/3072828972 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c40007320 con 0x7f3c50153880 2023-10-06T05:05:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.086+0000 7f3c4ffff700 1 -- 172.21.15.55:0/3072828972 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c50168910 con 0x7f3c50153880 2023-10-06T05:05:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.087+0000 7f3c4ffff700 1 -- 172.21.15.55:0/3072828972 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5006c450 con 0x7f3c50153880 2023-10-06T05:05:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.088+0000 7f3c36ffd700 1 -- 172.21.15.55:0/3072828972 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c40012400 con 0x7f3c50153880 2023-10-06T05:05:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.089+0000 7f3c36ffd700 1 --2- 172.21.15.55:0/3072828972 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3c3c066f30 0x7f3c3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.089+0000 7f3c36ffd700 1 -- 172.21.15.55:0/3072828972 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f3c40083c90 con 0x7f3c50153880 2023-10-06T05:05:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.089+0000 7f3c4e7fc700 1 --2- 172.21.15.55:0/3072828972 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3c3c066f30 0x7f3c3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.090+0000 7f3c4e7fc700 1 --2- 172.21.15.55:0/3072828972 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3c3c066f30 0x7f3c3c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3c44007fd0 tx=0x7f3c44007d50 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:42.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.093+0000 7f3c36ffd700 1 -- 172.21.15.55:0/3072828972 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c4004b5a0 con 0x7f3c50153880 2023-10-06T05:05:42.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.340+0000 7f3c4ffff700 1 -- 172.21.15.55:0/3072828972 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3c50157ff0 con 0x7f3c50153880 2023-10-06T05:05:42.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.944+0000 7f3c36ffd700 1 -- 172.21.15.55:0/3072828972 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v95) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3c400170b0 con 0x7f3c50153880 2023-10-06T05:05:42.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.947+0000 7f3c4ffff700 1 -- 172.21.15.55:0/3072828972 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3c3c066f30 msgr2=0x7f3c3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:42.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.947+0000 7f3c4ffff700 1 --2- 172.21.15.55:0/3072828972 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3c3c066f30 0x7f3c3c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3c44007fd0 tx=0x7f3c44007d50 comp rx=0 tx=0).stop 2023-10-06T05:05:42.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.947+0000 7f3c4ffff700 1 -- 172.21.15.55:0/3072828972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c50153880 msgr2=0x7f3c500a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:42.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.947+0000 7f3c4ffff700 1 --2- 172.21.15.55:0/3072828972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c50153880 0x7f3c500a5710 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000c790 tx=0x7f3c40007e80 comp rx=0 tx=0).stop 2023-10-06T05:05:42.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.947+0000 7f3c4ffff700 1 -- 172.21.15.55:0/3072828972 shutdown_connections 2023-10-06T05:05:42.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.947+0000 7f3c4ffff700 1 --2- 172.21.15.55:0/3072828972 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3c3c066f30 0x7f3c3c0693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:42.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.947+0000 7f3c4ffff700 1 --2- 172.21.15.55:0/3072828972 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3c5015a190 0x7f3c5014a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:42.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.947+0000 7f3c4ffff700 1 --2- 172.21.15.55:0/3072828972 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3c50153880 0x7f3c500a5710 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:42.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.947+0000 7f3c4ffff700 1 --2- 172.21.15.55:0/3072828972 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3c50153c50 0x7f3c5014b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:42.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.947+0000 7f3c4ffff700 1 -- 172.21.15.55:0/3072828972 >> 172.21.15.55:0/3072828972 conn(0x7f3c50068390 msgr2=0x7f3c50156bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:42.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.947+0000 7f3c4ffff700 1 -- 172.21.15.55:0/3072828972 shutdown_connections 2023-10-06T05:05:42.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:42.947+0000 7f3c4ffff700 1 -- 172.21.15.55:0/3072828972 wait complete. 2023-10-06T05:05:42.971 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:05:43.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.165+0000 7f80b9f9d700 1 Processor -- start 2023-10-06T05:05:43.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.165+0000 7f80b9f9d700 1 -- start start 2023-10-06T05:05:43.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b9f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f80b4140bb0 0x7f80b406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:43.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b9f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b406da30 0x7f80b40c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:43.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b9f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b40c6820 con 0x7f80b41407e0 2023-10-06T05:05:43.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b9f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b406deb0 con 0x7f80b4140bb0 2023-10-06T05:05:43.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b9f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40c6de0 con 0x7f80b406da30 2023-10-06T05:05:43.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f80b4140bb0 0x7f80b406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:43.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f80b4140bb0 0x7f80b406d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45388/0 (socket says 172.21.15.55:45388) 2023-10-06T05:05:43.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b3fff700 1 -- 172.21.15.55:0/405800263 learned_addr learned my addr 172.21.15.55:0/405800263 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:43.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b3fff700 1 -- 172.21.15.55:0/405800263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b406da30 msgr2=0x7f80b40c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:43.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b979c700 1 --2- 172.21.15.55:0/405800263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b406da30 0x7f80b40c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:43.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b3fff700 1 --2- 172.21.15.55:0/405800263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b406da30 0x7f80b40c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:43.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b3fff700 1 -- 172.21.15.55:0/405800263 >> v1:172.21.15.163:6789/0 conn(0x7f80b41407e0 legacy=0x7f80b406a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:43.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b3fff700 1 -- 172.21.15.55:0/405800263 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b4141270 con 0x7f80b4140bb0 2023-10-06T05:05:43.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b979c700 1 --2- 172.21.15.55:0/405800263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b406da30 0x7f80b40c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:43.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b3fff700 1 --2- 172.21.15.55:0/405800263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b4140bb0 0x7f80b406d4f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f80a00099b0 tx=0x7f80a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b56e37a5d994bf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:43.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.166+0000 7f80b37fe700 1 -- 172.21.15.55:0/405800263 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a000b860 con 0x7f80b4140bb0 2023-10-06T05:05:43.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.167+0000 7f80b37fe700 1 -- 172.21.15.55:0/405800263 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a000b9c0 con 0x7f80b4140bb0 2023-10-06T05:05:43.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.167+0000 7f80b37fe700 1 -- 172.21.15.55:0/405800263 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a000bd60 con 0x7f80b4140bb0 2023-10-06T05:05:43.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.167+0000 7f80b9f9d700 1 -- 172.21.15.55:0/405800263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b4140bb0 msgr2=0x7f80b406d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:43.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.167+0000 7f80b9f9d700 1 --2- 172.21.15.55:0/405800263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b4140bb0 0x7f80b406d4f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f80a00099b0 tx=0x7f80a0009250 comp rx=0 tx=0).stop 2023-10-06T05:05:43.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.167+0000 7f80b9f9d700 1 -- 172.21.15.55:0/405800263 shutdown_connections 2023-10-06T05:05:43.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.167+0000 7f80b9f9d700 1 --2- 172.21.15.55:0/405800263 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b406da30 0x7f80b40c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:43.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.167+0000 7f80b9f9d700 1 --2- 172.21.15.55:0/405800263 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b4140bb0 0x7f80b406d4f0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:43.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.167+0000 7f80b9f9d700 1 -- 172.21.15.55:0/405800263 >> 172.21.15.55:0/405800263 conn(0x7f80b40cc1e0 msgr2=0x7f80b40cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:43.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.167+0000 7f80b9f9d700 1 -- 172.21.15.55:0/405800263 shutdown_connections 2023-10-06T05:05:43.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.167+0000 7f80b9f9d700 1 -- 172.21.15.55:0/405800263 wait complete. 2023-10-06T05:05:43.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.169+0000 7f80b9f9d700 1 Processor -- start 2023-10-06T05:05:43.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.169+0000 7f80b9f9d700 1 -- start start 2023-10-06T05:05:43.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b9f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b406da30 0x7f80b4157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:43.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b9f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80b41407e0 0x7f80b40a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:43.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b9f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b4140bb0 0x7f80b413aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:43.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b9f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40c42a0 con 0x7f80b4140bb0 2023-10-06T05:05:43.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b9f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40c4960 con 0x7f80b41407e0 2023-10-06T05:05:43.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b9f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b41575b0 con 0x7f80b406da30 2023-10-06T05:05:43.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b979c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b4140bb0 0x7f80b413aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:43.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b979c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b4140bb0 0x7f80b413aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45394/0 (socket says 172.21.15.55:45394) 2023-10-06T05:05:43.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b979c700 1 -- 172.21.15.55:0/2495647387 learned_addr learned my addr 172.21.15.55:0/2495647387 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:43.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b8f9b700 1 --2- 172.21.15.55:0/2495647387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b406da30 0x7f80b4157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:43.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b3fff700 1 --2- 172.21.15.55:0/2495647387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80b41407e0 0x7f80b40a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:43.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b979c700 1 -- 172.21.15.55:0/2495647387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b406da30 msgr2=0x7f80b4157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:43.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b979c700 1 --2- 172.21.15.55:0/2495647387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b406da30 0x7f80b4157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:43.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b979c700 1 -- 172.21.15.55:0/2495647387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80b41407e0 msgr2=0x7f80b40a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:43.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b979c700 1 --2- 172.21.15.55:0/2495647387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80b41407e0 0x7f80b40a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:43.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.170+0000 7f80b979c700 1 -- 172.21.15.55:0/2495647387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a00094f0 con 0x7f80b4140bb0 2023-10-06T05:05:43.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.171+0000 7f80b979c700 1 --2- 172.21.15.55:0/2495647387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b4140bb0 0x7f80b413aa80 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00a380 tx=0x7f80ac007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:43.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.171+0000 7f80b0ff9700 1 -- 172.21.15.55:0/2495647387 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac015440 con 0x7f80b4140bb0 2023-10-06T05:05:43.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.171+0000 7f80b0ff9700 1 -- 172.21.15.55:0/2495647387 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ac0155a0 con 0x7f80b4140bb0 2023-10-06T05:05:43.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.171+0000 7f80b9f9d700 1 -- 172.21.15.55:0/2495647387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b40ae350 con 0x7f80b4140bb0 2023-10-06T05:05:43.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.171+0000 7f80b0ff9700 1 -- 172.21.15.55:0/2495647387 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ac015960 con 0x7f80b4140bb0 2023-10-06T05:05:43.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.171+0000 7f80b9f9d700 1 -- 172.21.15.55:0/2495647387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b40ae910 con 0x7f80b4140bb0 2023-10-06T05:05:43.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.172+0000 7f80b9f9d700 1 -- 172.21.15.55:0/2495647387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b40c4ad0 con 0x7f80b4140bb0 2023-10-06T05:05:43.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.173+0000 7f80b0ff9700 1 -- 172.21.15.55:0/2495647387 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f80ac00d440 con 0x7f80b4140bb0 2023-10-06T05:05:43.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.174+0000 7f80b0ff9700 1 --2- 172.21.15.55:0/2495647387 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8098066f30 0x7f80980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:43.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.174+0000 7f80b0ff9700 1 -- 172.21.15.55:0/2495647387 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f80ac0853d0 con 0x7f80b4140bb0 2023-10-06T05:05:43.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.176+0000 7f80b8f9b700 1 --2- 172.21.15.55:0/2495647387 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8098066f30 0x7f80980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:43.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.176+0000 7f80b8f9b700 1 --2- 172.21.15.55:0/2495647387 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8098066f30 0x7f80980693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f80a8003a80 tx=0x7f80a8007ec0 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:43.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.179+0000 7f80b0ff9700 1 -- 172.21.15.55:0/2495647387 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80ac0522a0 con 0x7f80b4140bb0 2023-10-06T05:05:43.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.427+0000 7f80b9f9d700 1 -- 172.21.15.55:0/2495647387 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f80b40aeb00 con 0x7f80b4140bb0 2023-10-06T05:05:43.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.946+0000 7f80b0ff9700 1 -- 172.21.15.55:0/2495647387 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v96) v1 ==== 122+0+0 (secure 0 0 0) 0x7f80ac0558c0 con 0x7f80b4140bb0 2023-10-06T05:05:43.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.948+0000 7f80b9f9d700 1 -- 172.21.15.55:0/2495647387 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8098066f30 msgr2=0x7f80980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:43.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.948+0000 7f80b9f9d700 1 --2- 172.21.15.55:0/2495647387 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8098066f30 0x7f80980693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f80a8003a80 tx=0x7f80a8007ec0 comp rx=0 tx=0).stop 2023-10-06T05:05:43.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.948+0000 7f80b9f9d700 1 -- 172.21.15.55:0/2495647387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b4140bb0 msgr2=0x7f80b413aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:43.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.949+0000 7f80b9f9d700 1 --2- 172.21.15.55:0/2495647387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b4140bb0 0x7f80b413aa80 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f80ac00a380 tx=0x7f80ac007900 comp rx=0 tx=0).stop 2023-10-06T05:05:43.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.949+0000 7f80b9f9d700 1 -- 172.21.15.55:0/2495647387 shutdown_connections 2023-10-06T05:05:43.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.949+0000 7f80b9f9d700 1 --2- 172.21.15.55:0/2495647387 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8098066f30 0x7f80980693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:43.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.949+0000 7f80b9f9d700 1 --2- 172.21.15.55:0/2495647387 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f80b4140bb0 0x7f80b413aa80 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:43.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.949+0000 7f80b9f9d700 1 --2- 172.21.15.55:0/2495647387 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f80b406da30 0x7f80b4157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:43.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.949+0000 7f80b9f9d700 1 --2- 172.21.15.55:0/2495647387 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f80b41407e0 0x7f80b40a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:43.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.949+0000 7f80b9f9d700 1 -- 172.21.15.55:0/2495647387 >> 172.21.15.55:0/2495647387 conn(0x7f80b40cc1e0 msgr2=0x7f80b40c0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:43.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.949+0000 7f80b9f9d700 1 -- 172.21.15.55:0/2495647387 shutdown_connections 2023-10-06T05:05:43.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:43.949+0000 7f80b9f9d700 1 -- 172.21.15.55:0/2495647387 wait complete. 2023-10-06T05:05:43.970 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:05:44.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.158+0000 7f815b59d700 1 Processor -- start 2023-10-06T05:05:44.160 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.158+0000 7f815b59d700 1 -- start start 2023-10-06T05:05:44.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.159+0000 7f815b59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f81540ad870 0x7f815406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:44.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.159+0000 7f815b59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81540b40f0 con 0x7f815406bf20 2023-10-06T05:05:44.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.159+0000 7f815b59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8154069640 con 0x7f8154142ea0 2023-10-06T05:05:44.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.159+0000 7f815b59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f815406c3a0 con 0x7f81540ad870 2023-10-06T05:05:44.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.159+0000 7f8159d9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f81540ad870 0x7f815406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:44.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.159+0000 7f815ad9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f815406bf20 0x7f815406a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55938/0 (socket says 172.21.15.55:55938) 2023-10-06T05:05:44.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.159+0000 7f815ad9c700 1 -- 172.21.15.55:0/3749908054 learned_addr learned my addr 172.21.15.55:0/3749908054 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.160+0000 7f8159d9a700 1 -- 172.21.15.55:0/3749908054 >> v1:172.21.15.163:6789/0 conn(0x7f8154142ea0 legacy=0x7f81540b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.160+0000 7f8159d9a700 1 -- 172.21.15.55:0/3749908054 >> v1:172.21.15.55:6789/0 conn(0x7f815406bf20 legacy=0x7f815406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.160+0000 7f8159d9a700 1 -- 172.21.15.55:0/3749908054 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8154069920 con 0x7f81540ad870 2023-10-06T05:05:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.160+0000 7f8159599700 1 -- 172.21.15.55:0/3749908054 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575504974 0 0) 0x7f81540b40f0 con 0x7f815406bf20 2023-10-06T05:05:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.160+0000 7f8159d9a700 1 --2- 172.21.15.55:0/3749908054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81540ad870 0x7f815406b9e0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f81440098e0 tx=0x7f81440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e558d490c0b2a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.160+0000 7f8159599700 1 -- 172.21.15.55:0/3749908054 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144009e80 con 0x7f81540ad870 2023-10-06T05:05:44.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.160+0000 7f8159599700 1 -- 172.21.15.55:0/3749908054 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814400b860 con 0x7f81540ad870 2023-10-06T05:05:44.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.160+0000 7f8159599700 1 -- 172.21.15.55:0/3749908054 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814400bbe0 con 0x7f81540ad870 2023-10-06T05:05:44.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.161+0000 7f815b59d700 1 -- 172.21.15.55:0/3749908054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81540ad870 msgr2=0x7f815406b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:44.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.161+0000 7f815b59d700 1 --2- 172.21.15.55:0/3749908054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81540ad870 0x7f815406b9e0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f81440098e0 tx=0x7f81440091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.161+0000 7f815b59d700 1 -- 172.21.15.55:0/3749908054 shutdown_connections 2023-10-06T05:05:44.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.161+0000 7f815b59d700 1 --2- 172.21.15.55:0/3749908054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81540ad870 0x7f815406b9e0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.161+0000 7f815b59d700 1 -- 172.21.15.55:0/3749908054 >> 172.21.15.55:0/3749908054 conn(0x7f81540ce430 msgr2=0x7f81540ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:44.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.161+0000 7f815b59d700 1 -- 172.21.15.55:0/3749908054 shutdown_connections 2023-10-06T05:05:44.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.161+0000 7f815b59d700 1 -- 172.21.15.55:0/3749908054 wait complete. 2023-10-06T05:05:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.162+0000 7f815b59d700 1 Processor -- start 2023-10-06T05:05:44.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.163+0000 7f815b59d700 1 -- start start 2023-10-06T05:05:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.163+0000 7f815b59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f815406bf20 0x7f81540c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.163+0000 7f815b59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8154142ea0 0x7f8154158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.163+0000 7f815b59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81540b0210 0x7f81541412b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.163+0000 7f815b59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81541698e0 con 0x7f81540b0210 2023-10-06T05:05:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.163+0000 7f815b59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8154169b20 con 0x7f815406bf20 2023-10-06T05:05:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.163+0000 7f815b59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8154169e00 con 0x7f8154142ea0 2023-10-06T05:05:44.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.164+0000 7f815ad9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81540b0210 0x7f81541412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:44.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.164+0000 7f815ad9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81540b0210 0x7f81541412b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45422/0 (socket says 172.21.15.55:45422) 2023-10-06T05:05:44.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.164+0000 7f815ad9c700 1 -- 172.21.15.55:0/1789303603 learned_addr learned my addr 172.21.15.55:0/1789303603 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:44.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.164+0000 7f8159d9a700 1 --2- 172.21.15.55:0/1789303603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8154142ea0 0x7f8154158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:44.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.164+0000 7f815a59b700 1 --2- 172.21.15.55:0/1789303603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f815406bf20 0x7f81540c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:44.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.164+0000 7f815ad9c700 1 -- 172.21.15.55:0/1789303603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8154142ea0 msgr2=0x7f8154158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:44.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.164+0000 7f815ad9c700 1 --2- 172.21.15.55:0/1789303603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8154142ea0 0x7f8154158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.164+0000 7f815ad9c700 1 -- 172.21.15.55:0/1789303603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f815406bf20 msgr2=0x7f81540c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:44.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.164+0000 7f815ad9c700 1 --2- 172.21.15.55:0/1789303603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f815406bf20 0x7f81540c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.164+0000 7f815ad9c700 1 -- 172.21.15.55:0/1789303603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8144009420 con 0x7f81540b0210 2023-10-06T05:05:44.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.164+0000 7f815ad9c700 1 --2- 172.21.15.55:0/1789303603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81540b0210 0x7f81541412b0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f814c00cc90 tx=0x7f814c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:44.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.165+0000 7f814a7fc700 1 -- 172.21.15.55:0/1789303603 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c00cea0 con 0x7f81540b0210 2023-10-06T05:05:44.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.165+0000 7f814a7fc700 1 -- 172.21.15.55:0/1789303603 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c00ab70 con 0x7f81540b0210 2023-10-06T05:05:44.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.165+0000 7f815b59d700 1 -- 172.21.15.55:0/1789303603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815416a0e0 con 0x7f81540b0210 2023-10-06T05:05:44.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.165+0000 7f814a7fc700 1 -- 172.21.15.55:0/1789303603 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0070f0 con 0x7f81540b0210 2023-10-06T05:05:44.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.165+0000 7f815b59d700 1 -- 172.21.15.55:0/1789303603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81541591c0 con 0x7f81540b0210 2023-10-06T05:05:44.169 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.166+0000 7f815b59d700 1 -- 172.21.15.55:0/1789303603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f815416a6f0 con 0x7f81540b0210 2023-10-06T05:05:44.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.167+0000 7f814a7fc700 1 -- 172.21.15.55:0/1789303603 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f814c0074a0 con 0x7f81540b0210 2023-10-06T05:05:44.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.167+0000 7f814a7fc700 1 --2- 172.21.15.55:0/1789303603 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8138066f30 0x7f81380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:44.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.168+0000 7f814a7fc700 1 -- 172.21.15.55:0/1789303603 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f814c084d50 con 0x7f81540b0210 2023-10-06T05:05:44.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.168+0000 7f815a59b700 1 --2- 172.21.15.55:0/1789303603 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8138066f30 0x7f81380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:44.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.169+0000 7f815a59b700 1 --2- 172.21.15.55:0/1789303603 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8138066f30 0x7f81380693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f81500037c0 tx=0x7f8150009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:44.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.172+0000 7f814a7fc700 1 -- 172.21.15.55:0/1789303603 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f814c051ca0 con 0x7f81540b0210 2023-10-06T05:05:44.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.416+0000 7f815b59d700 1 -- 172.21.15.55:0/1789303603 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f815404c3c0 con 0x7f81540b0210 2023-10-06T05:05:44.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.417+0000 7f814a7fc700 1 -- 172.21.15.55:0/1789303603 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f814c0552c0 con 0x7f81540b0210 2023-10-06T05:05:44.419 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:44.419 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8078,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1255308005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1255308005},{"type":"v1","addr":"172.21.15.163:6835","nonce":1255308005}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":9274,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2468487159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2468487159},{"type":"v1","addr":"172.21.15.163:6834","nonce":2468487159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/4105265850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":4105265850},{"type":"v1","addr":"172.21.15.55:6834","nonce":4105265850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:05:40.405710+0000","modified":"2023-10-06T05:05:43.944287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9771":{"gid":9771,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/312761019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":312761019},{"type":"v1","addr":"172.21.15.55:6835","nonce":312761019}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:05:44.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.419+0000 7f815b59d700 1 -- 172.21.15.55:0/1789303603 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8138066f30 msgr2=0x7f81380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:44.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.419+0000 7f815b59d700 1 --2- 172.21.15.55:0/1789303603 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8138066f30 0x7f81380693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f81500037c0 tx=0x7f8150009040 comp rx=0 tx=0).stop 2023-10-06T05:05:44.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.419+0000 7f815b59d700 1 -- 172.21.15.55:0/1789303603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81540b0210 msgr2=0x7f81541412b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:44.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.419+0000 7f815b59d700 1 --2- 172.21.15.55:0/1789303603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81540b0210 0x7f81541412b0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f814c00cc90 tx=0x7f814c001a80 comp rx=0 tx=0).stop 2023-10-06T05:05:44.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.419+0000 7f815b59d700 1 -- 172.21.15.55:0/1789303603 shutdown_connections 2023-10-06T05:05:44.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.419+0000 7f815b59d700 1 --2- 172.21.15.55:0/1789303603 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8138066f30 0x7f81380693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.419+0000 7f815b59d700 1 --2- 172.21.15.55:0/1789303603 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f81540b0210 0x7f81541412b0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.419+0000 7f815b59d700 1 --2- 172.21.15.55:0/1789303603 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f815406bf20 0x7f81540c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.419+0000 7f815b59d700 1 --2- 172.21.15.55:0/1789303603 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8154142ea0 0x7f8154158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.419+0000 7f815b59d700 1 -- 172.21.15.55:0/1789303603 >> 172.21.15.55:0/1789303603 conn(0x7f81540ce430 msgr2=0x7f81540cd220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:44.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.419+0000 7f815b59d700 1 -- 172.21.15.55:0/1789303603 shutdown_connections 2023-10-06T05:05:44.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.419+0000 7f815b59d700 1 -- 172.21.15.55:0/1789303603 wait complete. 2023-10-06T05:05:44.423 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 96 2023-10-06T05:05:44.439 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:05:44.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.623+0000 7f884b59e700 1 Processor -- start 2023-10-06T05:05:44.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.623+0000 7f884b59e700 1 -- start start 2023-10-06T05:05:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.625+0000 7f884b59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f884c06c0f0 0x7f884c1642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.625+0000 7f884b59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884c0b40f0 con 0x7f884c142ea0 2023-10-06T05:05:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.625+0000 7f884b59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884c069640 con 0x7f884c0ad870 2023-10-06T05:05:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.625+0000 7f884b59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884c069920 con 0x7f884c06c0f0 2023-10-06T05:05:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.625+0000 7f884a59c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f884c142ea0 0x7f884c0b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55946/0 (socket says 172.21.15.55:55946) 2023-10-06T05:05:44.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.625+0000 7f884a59c700 1 -- 172.21.15.55:0/3299049767 learned_addr learned my addr 172.21.15.55:0/3299049767 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:44.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.625+0000 7f884ad9d700 1 --2- 172.21.15.55:0/3299049767 >> v2:172.21.15.163:3300/0 conn(0x7f884c06c0f0 0x7f884c1642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:44.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.625+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4214499167 0 0) 0x7f884c0b40f0 con 0x7f884c142ea0 2023-10-06T05:05:44.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8834003570 con 0x7f884c142ea0 2023-10-06T05:05:44.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3920670785 0 0) 0x7f884c069640 con 0x7f884c0ad870 2023-10-06T05:05:44.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f884c0b40f0 con 0x7f884c0ad870 2023-10-06T05:05:44.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 641237227 0 0) 0x7f8834003570 con 0x7f884c142ea0 2023-10-06T05:05:44.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 >> v2:172.21.15.163:3300/0 conn(0x7f884c06c0f0 msgr2=0x7f884c1642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:44.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 --2- 172.21.15.55:0/3299049767 >> v2:172.21.15.163:3300/0 conn(0x7f884c06c0f0 0x7f884c1642e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 >> v1:172.21.15.163:6789/0 conn(0x7f884c0ad870 legacy=0x7f884c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:44.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c069c00 con 0x7f884c142ea0 2023-10-06T05:05:44.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7f883c001470 con 0x7f884c142ea0 2023-10-06T05:05:44.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 >> v1:172.21.15.55:6789/0 conn(0x7f884c142ea0 legacy=0x7f884c0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:44.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 --2- 172.21.15.55:0/3299049767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8834003bf0 0x7f8834003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:44.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 --2- 172.21.15.55:0/3299049767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8834004810 0x7f8834004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:44.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 --2- 172.21.15.55:0/3299049767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88340051c0 0x7f883400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:44.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c069920 con 0x7f8834003bf0 2023-10-06T05:05:44.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883400c0b0 con 0x7f8834004810 2023-10-06T05:05:44.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883400c270 con 0x7f88340051c0 2023-10-06T05:05:44.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884a59c700 1 --2- 172.21.15.55:0/3299049767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8834003bf0 0x7f8834003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:44.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f8849d9b700 1 --2- 172.21.15.55:0/3299049767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8834004810 0x7f8834004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:44.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.626+0000 7f884ad9d700 1 --2- 172.21.15.55:0/3299049767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88340051c0 0x7f883400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:44.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.627+0000 7f884a59c700 1 -- 172.21.15.55:0/3299049767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88340051c0 msgr2=0x7f883400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:44.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.627+0000 7f884a59c700 1 --2- 172.21.15.55:0/3299049767 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88340051c0 0x7f883400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.627+0000 7f884a59c700 1 -- 172.21.15.55:0/3299049767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8834004810 msgr2=0x7f8834004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:44.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.627+0000 7f884a59c700 1 --2- 172.21.15.55:0/3299049767 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8834004810 0x7f8834004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.627+0000 7f884a59c700 1 -- 172.21.15.55:0/3299049767 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8834003570 con 0x7f8834003bf0 2023-10-06T05:05:44.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.627+0000 7f884ad9d700 1 -- 172.21.15.55:0/3299049767 reap_dead start 2023-10-06T05:05:44.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.627+0000 7f884a59c700 1 --2- 172.21.15.55:0/3299049767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8834003bf0 0x7f8834003fe0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f883c00c2d0 tx=0x7f883c001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac945026c7e59cd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:44.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.627+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c00cba0 con 0x7f8834003bf0 2023-10-06T05:05:44.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.627+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c00cd00 con 0x7f8834003bf0 2023-10-06T05:05:44.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.627+0000 7f884959a700 1 -- 172.21.15.55:0/3299049767 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c005b70 con 0x7f8834003bf0 2023-10-06T05:05:44.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.628+0000 7f884b59e700 1 -- 172.21.15.55:0/3299049767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8834003bf0 msgr2=0x7f8834003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:44.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.628+0000 7f884b59e700 1 --2- 172.21.15.55:0/3299049767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8834003bf0 0x7f8834003fe0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f883c00c2d0 tx=0x7f883c001e10 comp rx=0 tx=0).stop 2023-10-06T05:05:44.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.628+0000 7f884b59e700 1 -- 172.21.15.55:0/3299049767 shutdown_connections 2023-10-06T05:05:44.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.628+0000 7f884b59e700 1 --2- 172.21.15.55:0/3299049767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8834003bf0 0x7f8834003fe0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.628+0000 7f884b59e700 1 -- 172.21.15.55:0/3299049767 >> 172.21.15.55:0/3299049767 conn(0x7f884c0ce430 msgr2=0x7f884c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:44.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.628+0000 7f884b59e700 1 -- 172.21.15.55:0/3299049767 shutdown_connections 2023-10-06T05:05:44.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.628+0000 7f884b59e700 1 -- 172.21.15.55:0/3299049767 wait complete. 2023-10-06T05:05:44.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.629+0000 7f884b59e700 1 Processor -- start 2023-10-06T05:05:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.630+0000 7f884b59e700 1 -- start start 2023-10-06T05:05:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.630+0000 7f884b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8834003bf0 0x7f884c0c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.630+0000 7f884b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f884c15ece0 0x7f884c0acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.630+0000 7f884b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c15acd0 0x7f884c15b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.630+0000 7f884b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c0c8b80 con 0x7f8834003bf0 2023-10-06T05:05:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.630+0000 7f884b59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884c15f150 con 0x7f884c15ece0 2023-10-06T05:05:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.630+0000 7f884b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884c15b680 con 0x7f884c15acd0 2023-10-06T05:05:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.631+0000 7f884a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8834003bf0 0x7f884c0c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.631+0000 7f884a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8834003bf0 0x7f884c0c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45434/0 (socket says 172.21.15.55:45434) 2023-10-06T05:05:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.631+0000 7f884a59c700 1 -- 172.21.15.55:0/1760562403 learned_addr learned my addr 172.21.15.55:0/1760562403 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:44.636 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.631+0000 7f884a59c700 1 -- 172.21.15.55:0/1760562403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c15acd0 msgr2=0x7f884c15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.631+0000 7f884a59c700 1 --2- 172.21.15.55:0/1760562403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c15acd0 0x7f884c15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.631+0000 7f884a59c700 1 -- 172.21.15.55:0/1760562403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f884c15ece0 msgr2=0x7f884c0acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.631+0000 7f884a59c700 1 --2- 172.21.15.55:0/1760562403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f884c15ece0 0x7f884c0acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.631+0000 7f884a59c700 1 -- 172.21.15.55:0/1760562403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c0c7c70 con 0x7f8834003bf0 2023-10-06T05:05:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.631+0000 7f884a59c700 1 --2- 172.21.15.55:0/1760562403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8834003bf0 0x7f884c0c75e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f883c001e50 tx=0x7f883c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.632+0000 7f883a7fc700 1 -- 172.21.15.55:0/1760562403 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c004a40 con 0x7f8834003bf0 2023-10-06T05:05:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.632+0000 7f883a7fc700 1 -- 172.21.15.55:0/1760562403 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c004ba0 con 0x7f8834003bf0 2023-10-06T05:05:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.632+0000 7f884b59e700 1 -- 172.21.15.55:0/1760562403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884c0c7f50 con 0x7f8834003bf0 2023-10-06T05:05:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.632+0000 7f883a7fc700 1 -- 172.21.15.55:0/1760562403 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c0070d0 con 0x7f8834003bf0 2023-10-06T05:05:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.632+0000 7f884b59e700 1 -- 172.21.15.55:0/1760562403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884c0c8510 con 0x7f8834003bf0 2023-10-06T05:05:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.633+0000 7f884b59e700 1 -- 172.21.15.55:0/1760562403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f884c04c3c0 con 0x7f8834003bf0 2023-10-06T05:05:44.637 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.634+0000 7f883a7fc700 1 -- 172.21.15.55:0/1760562403 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f883c007440 con 0x7f8834003bf0 2023-10-06T05:05:44.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.635+0000 7f883a7fc700 1 --2- 172.21.15.55:0/1760562403 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f882c066f30 0x7f882c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:44.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.635+0000 7f883a7fc700 1 -- 172.21.15.55:0/1760562403 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f883c085560 con 0x7f8834003bf0 2023-10-06T05:05:44.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.635+0000 7f8849d9b700 1 --2- 172.21.15.55:0/1760562403 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f882c066f30 0x7f882c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:44.639 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.636+0000 7f8849d9b700 1 --2- 172.21.15.55:0/1760562403 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f882c066f30 0x7f882c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f88400010e0 tx=0x7f8840007d30 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:44.640 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.639+0000 7f883a7fc700 1 -- 172.21.15.55:0/1760562403 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f883c052430 con 0x7f8834003bf0 2023-10-06T05:05:44.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.842+0000 7f884b59e700 1 -- 172.21.15.55:0/1760562403 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f884c143270 con 0x7f8834003bf0 2023-10-06T05:05:44.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.843+0000 7f883a7fc700 1 -- 172.21.15.55:0/1760562403 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+17569 (secure 0 0 0) 0x7f883c017070 con 0x7f8834003bf0 2023-10-06T05:05:44.846 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:44.846 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":193,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:05:41.905806+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:05:38.605485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:05:39.377918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:05:44.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.846+0000 7f884b59e700 1 -- 172.21.15.55:0/1760562403 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f882c066f30 msgr2=0x7f882c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:44.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.846+0000 7f884b59e700 1 --2- 172.21.15.55:0/1760562403 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f882c066f30 0x7f882c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f88400010e0 tx=0x7f8840007d30 comp rx=0 tx=0).stop 2023-10-06T05:05:44.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.846+0000 7f884b59e700 1 -- 172.21.15.55:0/1760562403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8834003bf0 msgr2=0x7f884c0c75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:44.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.846+0000 7f884b59e700 1 --2- 172.21.15.55:0/1760562403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8834003bf0 0x7f884c0c75e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f883c001e50 tx=0x7f883c015000 comp rx=0 tx=0).stop 2023-10-06T05:05:44.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.846+0000 7f884b59e700 1 -- 172.21.15.55:0/1760562403 shutdown_connections 2023-10-06T05:05:44.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.846+0000 7f884b59e700 1 --2- 172.21.15.55:0/1760562403 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f882c066f30 0x7f882c0693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.846+0000 7f884b59e700 1 --2- 172.21.15.55:0/1760562403 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f884c15acd0 0x7f884c15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.846+0000 7f884b59e700 1 --2- 172.21.15.55:0/1760562403 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8834003bf0 0x7f884c0c75e0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.846+0000 7f884b59e700 1 --2- 172.21.15.55:0/1760562403 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f884c15ece0 0x7f884c0acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:44.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.846+0000 7f884b59e700 1 -- 172.21.15.55:0/1760562403 >> 172.21.15.55:0/1760562403 conn(0x7f884c0ce430 msgr2=0x7f884c0cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:44.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.846+0000 7f884b59e700 1 -- 172.21.15.55:0/1760562403 shutdown_connections 2023-10-06T05:05:44.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:44.846+0000 7f884b59e700 1 -- 172.21.15.55:0/1760562403 wait complete. 2023-10-06T05:05:44.866 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:05:44.866 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:05:45.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.052+0000 7f0e8f59e700 1 Processor -- start 2023-10-06T05:05:45.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.052+0000 7f0e8f59e700 1 -- start start 2023-10-06T05:05:45.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.053+0000 7f0e8f59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0e900ae810 0x7f0e900abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:45.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.053+0000 7f0e8f59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0e90069f10 0x7f0e9006d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:45.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.054+0000 7f0e8f59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9006b0d0 con 0x7f0e900ac100 2023-10-06T05:05:45.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.054+0000 7f0e8f59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e9006a4c0 con 0x7f0e900ae810 2023-10-06T05:05:45.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.054+0000 7f0e8f59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e9006a7a0 con 0x7f0e90069f10 2023-10-06T05:05:45.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.054+0000 7f0e8e59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0e900ae810 0x7f0e900abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:45.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.054+0000 7f0e8e59c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0e900ae810 0x7f0e900abbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45438/0 (socket says 172.21.15.55:45438) 2023-10-06T05:05:45.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.054+0000 7f0e8e59c700 1 -- 172.21.15.55:0/3303391996 learned_addr learned my addr 172.21.15.55:0/3303391996 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:45.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.054+0000 7f0e8ed9d700 1 --2- 172.21.15.55:0/3303391996 >> v2:172.21.15.163:3300/0 conn(0x7f0e90069f10 0x7f0e9006d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:45.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.054+0000 7f0e8e59c700 1 -- 172.21.15.55:0/3303391996 >> v2:172.21.15.163:3300/0 conn(0x7f0e90069f10 msgr2=0x7f0e9006d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:45.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.054+0000 7f0e8e59c700 1 --2- 172.21.15.55:0/3303391996 >> v2:172.21.15.163:3300/0 conn(0x7f0e90069f10 0x7f0e9006d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.054+0000 7f0e8e59c700 1 -- 172.21.15.55:0/3303391996 >> v1:172.21.15.163:6789/0 conn(0x7f0e900ac100 legacy=0x7f0e90069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:45.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.054+0000 7f0e8e59c700 1 -- 172.21.15.55:0/3303391996 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e900ac570 con 0x7f0e900ae810 2023-10-06T05:05:45.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.055+0000 7f0e8e59c700 1 --2- 172.21.15.55:0/3303391996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e900ae810 0x7f0e900abbc0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0e9006d480 tx=0x7f0e80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1601676d16137b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:45.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.055+0000 7f0e8d59a700 1 -- 172.21.15.55:0/3303391996 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000b860 con 0x7f0e900ae810 2023-10-06T05:05:45.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.055+0000 7f0e8d59a700 1 -- 172.21.15.55:0/3303391996 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8000b9c0 con 0x7f0e900ae810 2023-10-06T05:05:45.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.055+0000 7f0e8d59a700 1 -- 172.21.15.55:0/3303391996 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8000bd60 con 0x7f0e900ae810 2023-10-06T05:05:45.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.055+0000 7f0e8f59e700 1 -- 172.21.15.55:0/3303391996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e900ae810 msgr2=0x7f0e900abbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:45.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.055+0000 7f0e8f59e700 1 --2- 172.21.15.55:0/3303391996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e900ae810 0x7f0e900abbc0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f0e9006d480 tx=0x7f0e80009250 comp rx=0 tx=0).stop 2023-10-06T05:05:45.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.056+0000 7f0e8f59e700 1 -- 172.21.15.55:0/3303391996 shutdown_connections 2023-10-06T05:05:45.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.056+0000 7f0e8f59e700 1 --2- 172.21.15.55:0/3303391996 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e900ae810 0x7f0e900abbc0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.056+0000 7f0e8f59e700 1 --2- 172.21.15.55:0/3303391996 >> v2:172.21.15.163:3300/0 conn(0x7f0e90069f10 0x7f0e9006d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.056+0000 7f0e8f59e700 1 -- 172.21.15.55:0/3303391996 >> 172.21.15.55:0/3303391996 conn(0x7f0e900c8bf0 msgr2=0x7f0e900c8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:45.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.056+0000 7f0e8f59e700 1 -- 172.21.15.55:0/3303391996 shutdown_connections 2023-10-06T05:05:45.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.056+0000 7f0e8f59e700 1 -- 172.21.15.55:0/3303391996 wait complete. 2023-10-06T05:05:45.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.057+0000 7f0e8f59e700 1 Processor -- start 2023-10-06T05:05:45.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.057+0000 7f0e8f59e700 1 -- start start 2023-10-06T05:05:45.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.058+0000 7f0e8f59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e90069f10 0x7f0e90157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:45.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.058+0000 7f0e8f59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e900ac100 0x7f0e900afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:45.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.058+0000 7f0e8f59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e900ae810 0x7f0e9016e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:45.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.058+0000 7f0e8f59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e90154410 con 0x7f0e900ae810 2023-10-06T05:05:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.058+0000 7f0e8f59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e900b0070 con 0x7f0e90069f10 2023-10-06T05:05:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.058+0000 7f0e8f59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e900b0350 con 0x7f0e900ac100 2023-10-06T05:05:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.058+0000 7f0e8ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e900ae810 0x7f0e9016e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.059+0000 7f0e8ed9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e900ae810 0x7f0e9016e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45454/0 (socket says 172.21.15.55:45454) 2023-10-06T05:05:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.059+0000 7f0e8ed9d700 1 -- 172.21.15.55:0/1732768046 learned_addr learned my addr 172.21.15.55:0/1732768046 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.059+0000 7f0e8e59c700 1 --2- 172.21.15.55:0/1732768046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e90069f10 0x7f0e90157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.059+0000 7f0e8dd9b700 1 --2- 172.21.15.55:0/1732768046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e900ac100 0x7f0e900afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.059+0000 7f0e8ed9d700 1 -- 172.21.15.55:0/1732768046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e900ac100 msgr2=0x7f0e900afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.059+0000 7f0e8ed9d700 1 --2- 172.21.15.55:0/1732768046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e900ac100 0x7f0e900afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.059+0000 7f0e8ed9d700 1 -- 172.21.15.55:0/1732768046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e90069f10 msgr2=0x7f0e90157c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.059+0000 7f0e8ed9d700 1 --2- 172.21.15.55:0/1732768046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e90069f10 0x7f0e90157c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.059+0000 7f0e8ed9d700 1 -- 172.21.15.55:0/1732768046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800094f0 con 0x7f0e900ae810 2023-10-06T05:05:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.059+0000 7f0e8ed9d700 1 --2- 172.21.15.55:0/1732768046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e900ae810 0x7f0e9016e1b0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f0e88007f20 tx=0x7f0e88009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.059+0000 7f0e7e7fc700 1 -- 172.21.15.55:0/1732768046 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8800ebf0 con 0x7f0e900ae810 2023-10-06T05:05:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.059+0000 7f0e7e7fc700 1 -- 172.21.15.55:0/1732768046 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8800ed70 con 0x7f0e900ae810 2023-10-06T05:05:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.060+0000 7f0e8f59e700 1 -- 172.21.15.55:0/1732768046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e900b0630 con 0x7f0e900ae810 2023-10-06T05:05:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.060+0000 7f0e7e7fc700 1 -- 172.21.15.55:0/1732768046 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e88014670 con 0x7f0e900ae810 2023-10-06T05:05:45.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.060+0000 7f0e8f59e700 1 -- 172.21.15.55:0/1732768046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9016f380 con 0x7f0e900ae810 2023-10-06T05:05:45.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.061+0000 7f0e8f59e700 1 -- 172.21.15.55:0/1732768046 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9004c3c0 con 0x7f0e900ae810 2023-10-06T05:05:45.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.062+0000 7f0e7e7fc700 1 -- 172.21.15.55:0/1732768046 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e880149e0 con 0x7f0e900ae810 2023-10-06T05:05:45.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.063+0000 7f0e7e7fc700 1 --2- 172.21.15.55:0/1732768046 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0e70066f30 0x7f0e700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:45.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.063+0000 7f0e7e7fc700 1 -- 172.21.15.55:0/1732768046 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f0e88084f10 con 0x7f0e900ae810 2023-10-06T05:05:45.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.063+0000 7f0e8e59c700 1 --2- 172.21.15.55:0/1732768046 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0e70066f30 0x7f0e700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:45.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.064+0000 7f0e8e59c700 1 --2- 172.21.15.55:0/1732768046 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0e70066f30 0x7f0e700693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0e9006d480 tx=0x7f0e80017000 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:45.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.066+0000 7f0e7e7fc700 1 -- 172.21.15.55:0/1732768046 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e88051de0 con 0x7f0e900ae810 2023-10-06T05:05:45.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.264+0000 7f0e8f59e700 1 -- 172.21.15.55:0/1732768046 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e9006b570 con 0x7f0e70066f30 2023-10-06T05:05:45.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.277+0000 7f0e7e7fc700 1 -- 172.21.15.55:0/1732768046 <== mgr.7946 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311812 (secure 0 0 0) 0x7f0e9006b570 con 0x7f0e70066f30 2023-10-06T05:05:45.284 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:45.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.282+0000 7f0e8f59e700 1 -- 172.21.15.55:0/1732768046 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0e70066f30 msgr2=0x7f0e700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:45.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.282+0000 7f0e8f59e700 1 --2- 172.21.15.55:0/1732768046 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0e70066f30 0x7f0e700693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0e9006d480 tx=0x7f0e80017000 comp rx=0 tx=0).stop 2023-10-06T05:05:45.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.282+0000 7f0e8f59e700 1 -- 172.21.15.55:0/1732768046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e900ae810 msgr2=0x7f0e9016e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:45.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.282+0000 7f0e8f59e700 1 --2- 172.21.15.55:0/1732768046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e900ae810 0x7f0e9016e1b0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f0e88007f20 tx=0x7f0e88009860 comp rx=0 tx=0).stop 2023-10-06T05:05:45.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.282+0000 7f0e8f59e700 1 -- 172.21.15.55:0/1732768046 shutdown_connections 2023-10-06T05:05:45.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.282+0000 7f0e8f59e700 1 --2- 172.21.15.55:0/1732768046 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0e70066f30 0x7f0e700693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.282+0000 7f0e8f59e700 1 --2- 172.21.15.55:0/1732768046 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e900ae810 0x7f0e9016e1b0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.282+0000 7f0e8f59e700 1 --2- 172.21.15.55:0/1732768046 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e90069f10 0x7f0e90157c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.282+0000 7f0e8f59e700 1 --2- 172.21.15.55:0/1732768046 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e900ac100 0x7f0e900afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.282+0000 7f0e8f59e700 1 -- 172.21.15.55:0/1732768046 >> 172.21.15.55:0/1732768046 conn(0x7f0e900c8bf0 msgr2=0x7f0e90156ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:45.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.282+0000 7f0e8f59e700 1 -- 172.21.15.55:0/1732768046 shutdown_connections 2023-10-06T05:05:45.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.282+0000 7f0e8f59e700 1 -- 172.21.15.55:0/1732768046 wait complete. 2023-10-06T05:05:45.290 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T05:05:45.302 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-06T05:05:44.023870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":583,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":364,"ondisk_log_size":364,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2472088,"kb_used_data":20056,"kb_used_omap":0,"kb_used_meta":2451968,"kb_avail":752502632,"statfs":{"total":773094113280,"available":770562695168,"internally_reserved":0,"allocated":20537344,"data_stored":11381292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2510815232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.125490"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.919707+0000","last_change":"2023-10-06T05:05:40.916091+0000","last_active":"2023-10-06T05:05:40.919707+0000","last_peered":"2023-10-06T05:05:40.919707+0000","last_clean":"2023-10-06T05:05:40.919707+0000","last_became_active":"2023-10-06T05:05:40.915665+0000","last_became_peered":"2023-10-06T05:05:40.915665+0000","last_unstale":"2023-10-06T05:05:40.919707+0000","last_undegraded":"2023-10-06T05:05:40.919707+0000","last_fullsized":"2023-10-06T05:05:40.919707+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:44.300844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.931643+0000","last_change":"2023-10-06T05:05:40.926850+0000","last_active":"2023-10-06T05:05:40.931643+0000","last_peered":"2023-10-06T05:05:40.931643+0000","last_clean":"2023-10-06T05:05:40.931643+0000","last_became_active":"2023-10-06T05:05:40.926419+0000","last_became_peered":"2023-10-06T05:05:40.926419+0000","last_unstale":"2023-10-06T05:05:40.931643+0000","last_undegraded":"2023-10-06T05:05:40.931643+0000","last_fullsized":"2023-10-06T05:05:40.931643+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:19.765709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.933709+0000","last_change":"2023-10-06T05:05:40.930290+0000","last_active":"2023-10-06T05:05:40.933709+0000","last_peered":"2023-10-06T05:05:40.933709+0000","last_clean":"2023-10-06T05:05:40.933709+0000","last_became_active":"2023-10-06T05:05:40.930001+0000","last_became_peered":"2023-10-06T05:05:40.930001+0000","last_unstale":"2023-10-06T05:05:40.933709+0000","last_undegraded":"2023-10-06T05:05:40.933709+0000","last_fullsized":"2023-10-06T05:05:40.933709+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:02.396017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.935251+0000","last_change":"2023-10-06T05:05:40.931283+0000","last_active":"2023-10-06T05:05:40.935251+0000","last_peered":"2023-10-06T05:05:40.935251+0000","last_clean":"2023-10-06T05:05:40.935251+0000","last_became_active":"2023-10-06T05:05:40.930980+0000","last_became_peered":"2023-10-06T05:05:40.930980+0000","last_unstale":"2023-10-06T05:05:40.935251+0000","last_undegraded":"2023-10-06T05:05:40.935251+0000","last_fullsized":"2023-10-06T05:05:40.935251+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:28.839013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.925383+0000","last_change":"2023-10-06T05:05:40.923538+0000","last_active":"2023-10-06T05:05:40.925383+0000","last_peered":"2023-10-06T05:05:40.925383+0000","last_clean":"2023-10-06T05:05:40.925383+0000","last_became_active":"2023-10-06T05:05:40.923282+0000","last_became_peered":"2023-10-06T05:05:40.923282+0000","last_unstale":"2023-10-06T05:05:40.925383+0000","last_undegraded":"2023-10-06T05:05:40.925383+0000","last_fullsized":"2023-10-06T05:05:40.925383+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:10.104501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.932195+0000","last_change":"2023-10-06T05:05:40.926888+0000","last_active":"2023-10-06T05:05:40.932195+0000","last_peered":"2023-10-06T05:05:40.932195+0000","last_clean":"2023-10-06T05:05:40.932195+0000","last_became_active":"2023-10-06T05:05:40.926585+0000","last_became_peered":"2023-10-06T05:05:40.926585+0000","last_unstale":"2023-10-06T05:05:40.932195+0000","last_undegraded":"2023-10-06T05:05:40.932195+0000","last_fullsized":"2023-10-06T05:05:40.932195+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:44.300844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.947042+0000","last_change":"2023-10-06T05:05:40.939478+0000","last_active":"2023-10-06T05:05:40.947042+0000","last_peered":"2023-10-06T05:05:40.947042+0000","last_clean":"2023-10-06T05:05:40.947042+0000","last_became_active":"2023-10-06T05:05:40.939179+0000","last_became_peered":"2023-10-06T05:05:40.939179+0000","last_unstale":"2023-10-06T05:05:40.947042+0000","last_undegraded":"2023-10-06T05:05:40.947042+0000","last_fullsized":"2023-10-06T05:05:40.947042+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:44.501528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.945326+0000","last_change":"2023-10-06T05:05:40.938714+0000","last_active":"2023-10-06T05:05:40.945326+0000","last_peered":"2023-10-06T05:05:40.945326+0000","last_clean":"2023-10-06T05:05:40.945326+0000","last_became_active":"2023-10-06T05:05:40.936963+0000","last_became_peered":"2023-10-06T05:05:40.936963+0000","last_unstale":"2023-10-06T05:05:40.945326+0000","last_undegraded":"2023-10-06T05:05:40.945326+0000","last_fullsized":"2023-10-06T05:05:40.945326+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:53.688046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950412+0000","last_change":"2023-10-06T05:05:40.944935+0000","last_active":"2023-10-06T05:05:40.950412+0000","last_peered":"2023-10-06T05:05:40.950412+0000","last_clean":"2023-10-06T05:05:40.950412+0000","last_became_active":"2023-10-06T05:05:40.944657+0000","last_became_peered":"2023-10-06T05:05:40.944657+0000","last_unstale":"2023-10-06T05:05:40.950412+0000","last_undegraded":"2023-10-06T05:05:40.950412+0000","last_fullsized":"2023-10-06T05:05:40.950412+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:09.021499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950779+0000","last_change":"2023-10-06T05:05:40.946638+0000","last_active":"2023-10-06T05:05:40.950779+0000","last_peered":"2023-10-06T05:05:40.950779+0000","last_clean":"2023-10-06T05:05:40.950779+0000","last_became_active":"2023-10-06T05:05:40.946390+0000","last_became_peered":"2023-10-06T05:05:40.946390+0000","last_unstale":"2023-10-06T05:05:40.950779+0000","last_undegraded":"2023-10-06T05:05:40.950779+0000","last_fullsized":"2023-10-06T05:05:40.950779+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:18.198649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.929570+0000","last_change":"2023-10-06T05:05:40.927717+0000","last_active":"2023-10-06T05:05:40.929570+0000","last_peered":"2023-10-06T05:05:40.929570+0000","last_clean":"2023-10-06T05:05:40.929570+0000","last_became_active":"2023-10-06T05:05:40.927371+0000","last_became_peered":"2023-10-06T05:05:40.927371+0000","last_unstale":"2023-10-06T05:05:40.929570+0000","last_undegraded":"2023-10-06T05:05:40.929570+0000","last_fullsized":"2023-10-06T05:05:40.929570+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:09.021499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.935862+0000","last_change":"2023-10-06T05:05:40.934420+0000","last_active":"2023-10-06T05:05:40.935862+0000","last_peered":"2023-10-06T05:05:40.935862+0000","last_clean":"2023-10-06T05:05:40.935862+0000","last_became_active":"2023-10-06T05:05:40.934140+0000","last_became_peered":"2023-10-06T05:05:40.934140+0000","last_unstale":"2023-10-06T05:05:40.935862+0000","last_undegraded":"2023-10-06T05:05:40.935862+0000","last_fullsized":"2023-10-06T05:05:40.935862+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:21.918262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.940048+0000","last_change":"2023-10-06T05:05:40.937744+0000","last_active":"2023-10-06T05:05:40.940048+0000","last_peered":"2023-10-06T05:05:40.940048+0000","last_clean":"2023-10-06T05:05:40.940048+0000","last_became_active":"2023-10-06T05:05:40.937359+0000","last_became_peered":"2023-10-06T05:05:40.937359+0000","last_unstale":"2023-10-06T05:05:40.940048+0000","last_undegraded":"2023-10-06T05:05:40.940048+0000","last_fullsized":"2023-10-06T05:05:40.940048+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:57.524605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.943926+0000","last_change":"2023-10-06T05:05:40.932645+0000","last_active":"2023-10-06T05:05:40.943926+0000","last_peered":"2023-10-06T05:05:40.943926+0000","last_clean":"2023-10-06T05:05:40.943926+0000","last_became_active":"2023-10-06T05:05:40.932034+0000","last_became_peered":"2023-10-06T05:05:40.932034+0000","last_unstale":"2023-10-06T05:05:40.943926+0000","last_undegraded":"2023-10-06T05:05:40.943926+0000","last_fullsized":"2023-10-06T05:05:40.943926+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:28.839013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.942396+0000","last_change":"2023-10-06T05:05:40.941300+0000","last_active":"2023-10-06T05:05:40.942396+0000","last_peered":"2023-10-06T05:05:40.942396+0000","last_clean":"2023-10-06T05:05:40.942396+0000","last_became_active":"2023-10-06T05:05:40.940987+0000","last_became_peered":"2023-10-06T05:05:40.940987+0000","last_unstale":"2023-10-06T05:05:40.942396+0000","last_undegraded":"2023-10-06T05:05:40.942396+0000","last_fullsized":"2023-10-06T05:05:40.942396+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:46.429634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.924229+0000","last_change":"2023-10-06T05:05:40.922202+0000","last_active":"2023-10-06T05:05:40.924229+0000","last_peered":"2023-10-06T05:05:40.924229+0000","last_clean":"2023-10-06T05:05:40.924229+0000","last_became_active":"2023-10-06T05:05:40.921936+0000","last_became_peered":"2023-10-06T05:05:40.921936+0000","last_unstale":"2023-10-06T05:05:40.924229+0000","last_undegraded":"2023-10-06T05:05:40.924229+0000","last_fullsized":"2023-10-06T05:05:40.924229+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:21.918262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950518+0000","last_change":"2023-10-06T05:05:40.944935+0000","last_active":"2023-10-06T05:05:40.950518+0000","last_peered":"2023-10-06T05:05:40.950518+0000","last_clean":"2023-10-06T05:05:40.950518+0000","last_became_active":"2023-10-06T05:05:40.940363+0000","last_became_peered":"2023-10-06T05:05:40.940363+0000","last_unstale":"2023-10-06T05:05:40.950518+0000","last_undegraded":"2023-10-06T05:05:40.950518+0000","last_fullsized":"2023-10-06T05:05:40.950518+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:11.565930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946556+0000","last_change":"2023-10-06T05:05:40.945791+0000","last_active":"2023-10-06T05:05:40.946556+0000","last_peered":"2023-10-06T05:05:40.946556+0000","last_clean":"2023-10-06T05:05:40.946556+0000","last_became_active":"2023-10-06T05:05:40.945506+0000","last_became_peered":"2023-10-06T05:05:40.945506+0000","last_unstale":"2023-10-06T05:05:40.946556+0000","last_undegraded":"2023-10-06T05:05:40.946556+0000","last_fullsized":"2023-10-06T05:05:40.946556+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:21.067657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.942400+0000","last_change":"2023-10-06T05:05:40.941160+0000","last_active":"2023-10-06T05:05:40.942400+0000","last_peered":"2023-10-06T05:05:40.942400+0000","last_clean":"2023-10-06T05:05:40.942400+0000","last_became_active":"2023-10-06T05:05:40.940858+0000","last_became_peered":"2023-10-06T05:05:40.940858+0000","last_unstale":"2023-10-06T05:05:40.942400+0000","last_undegraded":"2023-10-06T05:05:40.942400+0000","last_fullsized":"2023-10-06T05:05:40.942400+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:22.225371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.933540+0000","last_change":"2023-10-06T05:05:40.929792+0000","last_active":"2023-10-06T05:05:40.933540+0000","last_peered":"2023-10-06T05:05:40.933540+0000","last_clean":"2023-10-06T05:05:40.933540+0000","last_became_active":"2023-10-06T05:05:40.929348+0000","last_became_peered":"2023-10-06T05:05:40.929348+0000","last_unstale":"2023-10-06T05:05:40.933540+0000","last_undegraded":"2023-10-06T05:05:40.933540+0000","last_fullsized":"2023-10-06T05:05:40.933540+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:02.907910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.926528+0000","last_change":"2023-10-06T05:05:40.922805+0000","last_active":"2023-10-06T05:05:40.926528+0000","last_peered":"2023-10-06T05:05:40.926528+0000","last_clean":"2023-10-06T05:05:40.926528+0000","last_became_active":"2023-10-06T05:05:40.922405+0000","last_became_peered":"2023-10-06T05:05:40.922405+0000","last_unstale":"2023-10-06T05:05:40.926528+0000","last_undegraded":"2023-10-06T05:05:40.926528+0000","last_fullsized":"2023-10-06T05:05:40.926528+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:17.058870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.931883+0000","last_change":"2023-10-06T05:05:40.926329+0000","last_active":"2023-10-06T05:05:40.931883+0000","last_peered":"2023-10-06T05:05:40.931883+0000","last_clean":"2023-10-06T05:05:40.931883+0000","last_became_active":"2023-10-06T05:05:40.926026+0000","last_became_peered":"2023-10-06T05:05:40.926026+0000","last_unstale":"2023-10-06T05:05:40.931883+0000","last_undegraded":"2023-10-06T05:05:40.931883+0000","last_fullsized":"2023-10-06T05:05:40.931883+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:17.058870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.917799+0000","last_change":"2023-10-06T05:05:40.915087+0000","last_active":"2023-10-06T05:05:40.917799+0000","last_peered":"2023-10-06T05:05:40.917799+0000","last_clean":"2023-10-06T05:05:40.917799+0000","last_became_active":"2023-10-06T05:05:40.914807+0000","last_became_peered":"2023-10-06T05:05:40.914807+0000","last_unstale":"2023-10-06T05:05:40.917799+0000","last_undegraded":"2023-10-06T05:05:40.917799+0000","last_fullsized":"2023-10-06T05:05:40.917799+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:44.300844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.923541+0000","last_change":"2023-10-06T05:05:40.922026+0000","last_active":"2023-10-06T05:05:40.923541+0000","last_peered":"2023-10-06T05:05:40.923541+0000","last_clean":"2023-10-06T05:05:40.923541+0000","last_became_active":"2023-10-06T05:05:40.921531+0000","last_became_peered":"2023-10-06T05:05:40.921531+0000","last_unstale":"2023-10-06T05:05:40.923541+0000","last_undegraded":"2023-10-06T05:05:40.923541+0000","last_fullsized":"2023-10-06T05:05:40.923541+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:48.689929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.935321+0000","last_change":"2023-10-06T05:05:40.927795+0000","last_active":"2023-10-06T05:05:40.935321+0000","last_peered":"2023-10-06T05:05:40.935321+0000","last_clean":"2023-10-06T05:05:40.935321+0000","last_became_active":"2023-10-06T05:05:40.927347+0000","last_became_peered":"2023-10-06T05:05:40.927347+0000","last_unstale":"2023-10-06T05:05:40.935321+0000","last_undegraded":"2023-10-06T05:05:40.935321+0000","last_fullsized":"2023-10-06T05:05:40.935321+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:34:49.506769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.942408+0000","last_change":"2023-10-06T05:05:40.938367+0000","last_active":"2023-10-06T05:05:40.942408+0000","last_peered":"2023-10-06T05:05:40.942408+0000","last_clean":"2023-10-06T05:05:40.942408+0000","last_became_active":"2023-10-06T05:05:40.938143+0000","last_became_peered":"2023-10-06T05:05:40.938143+0000","last_unstale":"2023-10-06T05:05:40.942408+0000","last_undegraded":"2023-10-06T05:05:40.942408+0000","last_fullsized":"2023-10-06T05:05:40.942408+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:46.577112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946345+0000","last_change":"2023-10-06T05:05:40.943345+0000","last_active":"2023-10-06T05:05:40.946345+0000","last_peered":"2023-10-06T05:05:40.946345+0000","last_clean":"2023-10-06T05:05:40.946345+0000","last_became_active":"2023-10-06T05:05:40.943071+0000","last_became_peered":"2023-10-06T05:05:40.943071+0000","last_unstale":"2023-10-06T05:05:40.946345+0000","last_undegraded":"2023-10-06T05:05:40.946345+0000","last_fullsized":"2023-10-06T05:05:40.946345+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:50.110698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.923559+0000","last_change":"2023-10-06T05:05:40.921659+0000","last_active":"2023-10-06T05:05:40.923559+0000","last_peered":"2023-10-06T05:05:40.923559+0000","last_clean":"2023-10-06T05:05:40.923559+0000","last_became_active":"2023-10-06T05:05:40.921381+0000","last_became_peered":"2023-10-06T05:05:40.921381+0000","last_unstale":"2023-10-06T05:05:40.923559+0000","last_undegraded":"2023-10-06T05:05:40.923559+0000","last_fullsized":"2023-10-06T05:05:40.923559+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:47.808228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946914+0000","last_change":"2023-10-06T05:05:40.946429+0000","last_active":"2023-10-06T05:05:40.946914+0000","last_peered":"2023-10-06T05:05:40.946914+0000","last_clean":"2023-10-06T05:05:40.946914+0000","last_became_active":"2023-10-06T05:05:40.946095+0000","last_became_peered":"2023-10-06T05:05:40.946095+0000","last_unstale":"2023-10-06T05:05:40.946914+0000","last_undegraded":"2023-10-06T05:05:40.946914+0000","last_fullsized":"2023-10-06T05:05:40.946914+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:02.396017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.926500+0000","last_change":"2023-10-06T05:05:40.923308+0000","last_active":"2023-10-06T05:05:40.926500+0000","last_peered":"2023-10-06T05:05:40.926500+0000","last_clean":"2023-10-06T05:05:40.926500+0000","last_became_active":"2023-10-06T05:05:40.922449+0000","last_became_peered":"2023-10-06T05:05:40.922449+0000","last_unstale":"2023-10-06T05:05:40.926500+0000","last_undegraded":"2023-10-06T05:05:40.926500+0000","last_fullsized":"2023-10-06T05:05:40.926500+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:03:18.924378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.926173+0000","last_change":"2023-10-06T05:05:40.922469+0000","last_active":"2023-10-06T05:05:40.926173+0000","last_peered":"2023-10-06T05:05:40.926173+0000","last_clean":"2023-10-06T05:05:40.926173+0000","last_became_active":"2023-10-06T05:05:40.922155+0000","last_became_peered":"2023-10-06T05:05:40.922155+0000","last_unstale":"2023-10-06T05:05:40.926173+0000","last_undegraded":"2023-10-06T05:05:40.926173+0000","last_fullsized":"2023-10-06T05:05:40.926173+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:42.339342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.934664+0000","last_change":"2023-10-06T05:05:40.933192+0000","last_active":"2023-10-06T05:05:40.934664+0000","last_peered":"2023-10-06T05:05:40.934664+0000","last_clean":"2023-10-06T05:05:40.934664+0000","last_became_active":"2023-10-06T05:05:40.932740+0000","last_became_peered":"2023-10-06T05:05:40.932740+0000","last_unstale":"2023-10-06T05:05:40.934664+0000","last_undegraded":"2023-10-06T05:05:40.934664+0000","last_fullsized":"2023-10-06T05:05:40.934664+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:04.491084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.943829+0000","last_change":"2023-10-06T05:05:40.942695+0000","last_active":"2023-10-06T05:05:40.943829+0000","last_peered":"2023-10-06T05:05:40.943829+0000","last_clean":"2023-10-06T05:05:40.943829+0000","last_became_active":"2023-10-06T05:05:40.942183+0000","last_became_peered":"2023-10-06T05:05:40.942183+0000","last_unstale":"2023-10-06T05:05:40.943829+0000","last_undegraded":"2023-10-06T05:05:40.943829+0000","last_fullsized":"2023-10-06T05:05:40.943829+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:10.104501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"192'4","reported_seq":14,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.943127+0000","last_change":"2023-10-06T05:05:39.925290+0000","last_active":"2023-10-06T05:05:40.943127+0000","last_peered":"2023-10-06T05:05:40.943127+0000","last_clean":"2023-10-06T05:05:40.943127+0000","last_became_active":"2023-10-06T05:05:39.924872+0000","last_became_peered":"2023-10-06T05:05:39.924872+0000","last_unstale":"2023-10-06T05:05:40.943127+0000","last_undegraded":"2023-10-06T05:05:40.943127+0000","last_fullsized":"2023-10-06T05:05:40.943127+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:46.798368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.922648+0000","last_change":"2023-10-06T05:05:40.919939+0000","last_active":"2023-10-06T05:05:40.922648+0000","last_peered":"2023-10-06T05:05:40.922648+0000","last_clean":"2023-10-06T05:05:40.922648+0000","last_became_active":"2023-10-06T05:05:40.919646+0000","last_became_peered":"2023-10-06T05:05:40.919646+0000","last_unstale":"2023-10-06T05:05:40.922648+0000","last_undegraded":"2023-10-06T05:05:40.922648+0000","last_fullsized":"2023-10-06T05:05:40.922648+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:14.653476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950992+0000","last_change":"2023-10-06T05:05:39.918565+0000","last_active":"2023-10-06T05:05:40.950992+0000","last_peered":"2023-10-06T05:05:40.950992+0000","last_clean":"2023-10-06T05:05:40.950992+0000","last_became_active":"2023-10-06T05:05:39.918281+0000","last_became_peered":"2023-10-06T05:05:39.918281+0000","last_unstale":"2023-10-06T05:05:40.950992+0000","last_undegraded":"2023-10-06T05:05:40.950992+0000","last_fullsized":"2023-10-06T05:05:40.950992+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:17.188789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.943946+0000","last_change":"2023-10-06T05:05:40.932809+0000","last_active":"2023-10-06T05:05:40.943946+0000","last_peered":"2023-10-06T05:05:40.943946+0000","last_clean":"2023-10-06T05:05:40.943946+0000","last_became_active":"2023-10-06T05:05:40.932380+0000","last_became_peered":"2023-10-06T05:05:40.932380+0000","last_unstale":"2023-10-06T05:05:40.943946+0000","last_undegraded":"2023-10-06T05:05:40.943946+0000","last_fullsized":"2023-10-06T05:05:40.943946+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:48.689929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.943613+0000","last_change":"2023-10-06T05:05:39.925369+0000","last_active":"2023-10-06T05:05:40.943613+0000","last_peered":"2023-10-06T05:05:40.943613+0000","last_clean":"2023-10-06T05:05:40.943613+0000","last_became_active":"2023-10-06T05:05:39.923479+0000","last_became_peered":"2023-10-06T05:05:39.923479+0000","last_unstale":"2023-10-06T05:05:40.943613+0000","last_undegraded":"2023-10-06T05:05:40.943613+0000","last_fullsized":"2023-10-06T05:05:40.943613+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:41.289552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.943792+0000","last_change":"2023-10-06T05:05:40.942581+0000","last_active":"2023-10-06T05:05:40.943792+0000","last_peered":"2023-10-06T05:05:40.943792+0000","last_clean":"2023-10-06T05:05:40.943792+0000","last_became_active":"2023-10-06T05:05:40.942285+0000","last_became_peered":"2023-10-06T05:05:40.942285+0000","last_unstale":"2023-10-06T05:05:40.943792+0000","last_undegraded":"2023-10-06T05:05:40.943792+0000","last_fullsized":"2023-10-06T05:05:40.943792+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:28.839013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.941293+0000","last_change":"2023-10-06T05:05:39.926002+0000","last_active":"2023-10-06T05:05:40.941293+0000","last_peered":"2023-10-06T05:05:40.941293+0000","last_clean":"2023-10-06T05:05:40.941293+0000","last_became_active":"2023-10-06T05:05:39.925738+0000","last_became_peered":"2023-10-06T05:05:39.925738+0000","last_unstale":"2023-10-06T05:05:40.941293+0000","last_undegraded":"2023-10-06T05:05:40.941293+0000","last_fullsized":"2023-10-06T05:05:40.941293+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:38.682187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950753+0000","last_change":"2023-10-06T05:05:40.949012+0000","last_active":"2023-10-06T05:05:40.950753+0000","last_peered":"2023-10-06T05:05:40.950753+0000","last_clean":"2023-10-06T05:05:40.950753+0000","last_became_active":"2023-10-06T05:05:40.946073+0000","last_became_peered":"2023-10-06T05:05:40.946073+0000","last_unstale":"2023-10-06T05:05:40.950753+0000","last_undegraded":"2023-10-06T05:05:40.950753+0000","last_fullsized":"2023-10-06T05:05:40.950753+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:17.899816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.948623+0000","last_change":"2023-10-06T05:05:39.923467+0000","last_active":"2023-10-06T05:05:40.948623+0000","last_peered":"2023-10-06T05:05:40.948623+0000","last_clean":"2023-10-06T05:05:40.948623+0000","last_became_active":"2023-10-06T05:05:39.923175+0000","last_became_peered":"2023-10-06T05:05:39.923175+0000","last_unstale":"2023-10-06T05:05:40.948623+0000","last_undegraded":"2023-10-06T05:05:40.948623+0000","last_fullsized":"2023-10-06T05:05:40.948623+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:58.965000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.942490+0000","last_change":"2023-10-06T05:05:40.939550+0000","last_active":"2023-10-06T05:05:40.942490+0000","last_peered":"2023-10-06T05:05:40.942490+0000","last_clean":"2023-10-06T05:05:40.942490+0000","last_became_active":"2023-10-06T05:05:40.939327+0000","last_became_peered":"2023-10-06T05:05:40.939327+0000","last_unstale":"2023-10-06T05:05:40.942490+0000","last_undegraded":"2023-10-06T05:05:40.942490+0000","last_fullsized":"2023-10-06T05:05:40.942490+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:48.324228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946700+0000","last_change":"2023-10-06T05:05:39.926962+0000","last_active":"2023-10-06T05:05:40.946700+0000","last_peered":"2023-10-06T05:05:40.946700+0000","last_clean":"2023-10-06T05:05:40.946700+0000","last_became_active":"2023-10-06T05:05:39.926698+0000","last_became_peered":"2023-10-06T05:05:39.926698+0000","last_unstale":"2023-10-06T05:05:40.946700+0000","last_undegraded":"2023-10-06T05:05:40.946700+0000","last_fullsized":"2023-10-06T05:05:40.946700+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:43.491668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.933580+0000","last_change":"2023-10-06T05:05:40.928951+0000","last_active":"2023-10-06T05:05:40.933580+0000","last_peered":"2023-10-06T05:05:40.933580+0000","last_clean":"2023-10-06T05:05:40.933580+0000","last_became_active":"2023-10-06T05:05:40.928695+0000","last_became_peered":"2023-10-06T05:05:40.928695+0000","last_unstale":"2023-10-06T05:05:40.933580+0000","last_undegraded":"2023-10-06T05:05:40.933580+0000","last_fullsized":"2023-10-06T05:05:40.933580+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:03.406905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.920630+0000","last_change":"2023-10-06T05:05:39.926602+0000","last_active":"2023-10-06T05:05:40.920630+0000","last_peered":"2023-10-06T05:05:40.920630+0000","last_clean":"2023-10-06T05:05:40.920630+0000","last_became_active":"2023-10-06T05:05:39.926304+0000","last_became_peered":"2023-10-06T05:05:39.926304+0000","last_unstale":"2023-10-06T05:05:40.920630+0000","last_undegraded":"2023-10-06T05:05:40.920630+0000","last_fullsized":"2023-10-06T05:05:40.920630+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:34.314159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.925483+0000","last_change":"2023-10-06T05:05:40.923945+0000","last_active":"2023-10-06T05:05:40.925483+0000","last_peered":"2023-10-06T05:05:40.925483+0000","last_clean":"2023-10-06T05:05:40.925483+0000","last_became_active":"2023-10-06T05:05:40.923610+0000","last_became_peered":"2023-10-06T05:05:40.923610+0000","last_unstale":"2023-10-06T05:05:40.925483+0000","last_undegraded":"2023-10-06T05:05:40.925483+0000","last_fullsized":"2023-10-06T05:05:40.925483+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:10.104501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.918673+0000","last_change":"2023-10-06T05:05:39.928895+0000","last_active":"2023-10-06T05:05:40.918673+0000","last_peered":"2023-10-06T05:05:40.918673+0000","last_clean":"2023-10-06T05:05:40.918673+0000","last_became_active":"2023-10-06T05:05:39.928643+0000","last_became_peered":"2023-10-06T05:05:39.928643+0000","last_unstale":"2023-10-06T05:05:40.918673+0000","last_undegraded":"2023-10-06T05:05:40.918673+0000","last_fullsized":"2023-10-06T05:05:40.918673+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:49.012909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.945303+0000","last_change":"2023-10-06T05:05:40.937216+0000","last_active":"2023-10-06T05:05:40.945303+0000","last_peered":"2023-10-06T05:05:40.945303+0000","last_clean":"2023-10-06T05:05:40.945303+0000","last_became_active":"2023-10-06T05:05:40.936666+0000","last_became_peered":"2023-10-06T05:05:40.936666+0000","last_unstale":"2023-10-06T05:05:40.945303+0000","last_undegraded":"2023-10-06T05:05:40.945303+0000","last_fullsized":"2023-10-06T05:05:40.945303+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:26.963559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.940832+0000","last_change":"2023-10-06T05:05:39.920059+0000","last_active":"2023-10-06T05:05:40.940832+0000","last_peered":"2023-10-06T05:05:40.940832+0000","last_clean":"2023-10-06T05:05:40.940832+0000","last_became_active":"2023-10-06T05:05:39.919769+0000","last_became_peered":"2023-10-06T05:05:39.919769+0000","last_unstale":"2023-10-06T05:05:40.940832+0000","last_undegraded":"2023-10-06T05:05:40.940832+0000","last_fullsized":"2023-10-06T05:05:40.940832+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:39.631553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.927801+0000","last_change":"2023-10-06T05:05:40.923956+0000","last_active":"2023-10-06T05:05:40.927801+0000","last_peered":"2023-10-06T05:05:40.927801+0000","last_clean":"2023-10-06T05:05:40.927801+0000","last_became_active":"2023-10-06T05:05:40.923640+0000","last_became_peered":"2023-10-06T05:05:40.923640+0000","last_unstale":"2023-10-06T05:05:40.927801+0000","last_undegraded":"2023-10-06T05:05:40.927801+0000","last_fullsized":"2023-10-06T05:05:40.927801+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:14.959671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950727+0000","last_change":"2023-10-06T05:05:39.925659+0000","last_active":"2023-10-06T05:05:40.950727+0000","last_peered":"2023-10-06T05:05:40.950727+0000","last_clean":"2023-10-06T05:05:40.950727+0000","last_became_active":"2023-10-06T05:05:39.925413+0000","last_became_peered":"2023-10-06T05:05:39.925413+0000","last_unstale":"2023-10-06T05:05:40.950727+0000","last_undegraded":"2023-10-06T05:05:40.950727+0000","last_fullsized":"2023-10-06T05:05:40.950727+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:41.289552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.925429+0000","last_change":"2023-10-06T05:05:40.921842+0000","last_active":"2023-10-06T05:05:40.925429+0000","last_peered":"2023-10-06T05:05:40.925429+0000","last_clean":"2023-10-06T05:05:40.925429+0000","last_became_active":"2023-10-06T05:05:40.921579+0000","last_became_peered":"2023-10-06T05:05:40.921579+0000","last_unstale":"2023-10-06T05:05:40.925429+0000","last_undegraded":"2023-10-06T05:05:40.925429+0000","last_fullsized":"2023-10-06T05:05:40.925429+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:36.500257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946487+0000","last_change":"2023-10-06T05:05:39.920441+0000","last_active":"2023-10-06T05:05:40.946487+0000","last_peered":"2023-10-06T05:05:40.946487+0000","last_clean":"2023-10-06T05:05:40.946487+0000","last_became_active":"2023-10-06T05:05:39.920172+0000","last_became_peered":"2023-10-06T05:05:39.920172+0000","last_unstale":"2023-10-06T05:05:40.946487+0000","last_undegraded":"2023-10-06T05:05:40.946487+0000","last_fullsized":"2023-10-06T05:05:40.946487+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:46.798368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.929553+0000","last_change":"2023-10-06T05:05:40.927217+0000","last_active":"2023-10-06T05:05:40.929553+0000","last_peered":"2023-10-06T05:05:40.929553+0000","last_clean":"2023-10-06T05:05:40.929553+0000","last_became_active":"2023-10-06T05:05:40.926817+0000","last_became_peered":"2023-10-06T05:05:40.926817+0000","last_unstale":"2023-10-06T05:05:40.929553+0000","last_undegraded":"2023-10-06T05:05:40.929553+0000","last_fullsized":"2023-10-06T05:05:40.929553+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:01.165703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.919218+0000","last_change":"2023-10-06T05:05:39.919987+0000","last_active":"2023-10-06T05:05:40.919218+0000","last_peered":"2023-10-06T05:05:40.919218+0000","last_clean":"2023-10-06T05:05:40.919218+0000","last_became_active":"2023-10-06T05:05:39.919692+0000","last_became_peered":"2023-10-06T05:05:39.919692+0000","last_unstale":"2023-10-06T05:05:40.919218+0000","last_undegraded":"2023-10-06T05:05:40.919218+0000","last_fullsized":"2023-10-06T05:05:40.919218+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:01.125242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.923540+0000","last_change":"2023-10-06T05:05:40.921248+0000","last_active":"2023-10-06T05:05:40.923540+0000","last_peered":"2023-10-06T05:05:40.923540+0000","last_clean":"2023-10-06T05:05:40.923540+0000","last_became_active":"2023-10-06T05:05:40.920924+0000","last_became_peered":"2023-10-06T05:05:40.920924+0000","last_unstale":"2023-10-06T05:05:40.923540+0000","last_undegraded":"2023-10-06T05:05:40.923540+0000","last_fullsized":"2023-10-06T05:05:40.923540+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:53.688046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946972+0000","last_change":"2023-10-06T05:05:39.928080+0000","last_active":"2023-10-06T05:05:40.946972+0000","last_peered":"2023-10-06T05:05:40.946972+0000","last_clean":"2023-10-06T05:05:40.946972+0000","last_became_active":"2023-10-06T05:05:39.927814+0000","last_became_peered":"2023-10-06T05:05:39.927814+0000","last_unstale":"2023-10-06T05:05:40.946972+0000","last_undegraded":"2023-10-06T05:05:40.946972+0000","last_fullsized":"2023-10-06T05:05:40.946972+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:52.678186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946542+0000","last_change":"2023-10-06T05:05:40.941982+0000","last_active":"2023-10-06T05:05:40.946542+0000","last_peered":"2023-10-06T05:05:40.946542+0000","last_clean":"2023-10-06T05:05:40.946542+0000","last_became_active":"2023-10-06T05:05:40.941660+0000","last_became_peered":"2023-10-06T05:05:40.941660+0000","last_unstale":"2023-10-06T05:05:40.946542+0000","last_undegraded":"2023-10-06T05:05:40.946542+0000","last_fullsized":"2023-10-06T05:05:40.946542+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:21.918262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.916446+0000","last_change":"2023-10-06T05:05:39.926300+0000","last_active":"2023-10-06T05:05:40.916446+0000","last_peered":"2023-10-06T05:05:40.916446+0000","last_clean":"2023-10-06T05:05:40.916446+0000","last_became_active":"2023-10-06T05:05:39.926009+0000","last_became_peered":"2023-10-06T05:05:39.926009+0000","last_unstale":"2023-10-06T05:05:40.916446+0000","last_undegraded":"2023-10-06T05:05:40.916446+0000","last_fullsized":"2023-10-06T05:05:40.916446+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:01.125242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950051+0000","last_change":"2023-10-06T05:05:40.947297+0000","last_active":"2023-10-06T05:05:40.950051+0000","last_peered":"2023-10-06T05:05:40.950051+0000","last_clean":"2023-10-06T05:05:40.950051+0000","last_became_active":"2023-10-06T05:05:40.947004+0000","last_became_peered":"2023-10-06T05:05:40.947004+0000","last_unstale":"2023-10-06T05:05:40.950051+0000","last_undegraded":"2023-10-06T05:05:40.950051+0000","last_fullsized":"2023-10-06T05:05:40.950051+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:19.716383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.921497+0000","last_change":"2023-10-06T05:05:39.923141+0000","last_active":"2023-10-06T05:05:40.921497+0000","last_peered":"2023-10-06T05:05:40.921497+0000","last_clean":"2023-10-06T05:05:40.921497+0000","last_became_active":"2023-10-06T05:05:39.922780+0000","last_became_peered":"2023-10-06T05:05:39.922780+0000","last_unstale":"2023-10-06T05:05:40.921497+0000","last_undegraded":"2023-10-06T05:05:40.921497+0000","last_fullsized":"2023-10-06T05:05:40.921497+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:42.468662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.933364+0000","last_change":"2023-10-06T05:05:40.932037+0000","last_active":"2023-10-06T05:05:40.933364+0000","last_peered":"2023-10-06T05:05:40.933364+0000","last_clean":"2023-10-06T05:05:40.933364+0000","last_became_active":"2023-10-06T05:05:40.930683+0000","last_became_peered":"2023-10-06T05:05:40.930683+0000","last_unstale":"2023-10-06T05:05:40.933364+0000","last_undegraded":"2023-10-06T05:05:40.933364+0000","last_fullsized":"2023-10-06T05:05:40.933364+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:19.716383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.920703+0000","last_change":"2023-10-06T05:05:39.928537+0000","last_active":"2023-10-06T05:05:40.920703+0000","last_peered":"2023-10-06T05:05:40.920703+0000","last_clean":"2023-10-06T05:05:40.920703+0000","last_became_active":"2023-10-06T05:05:39.928200+0000","last_became_peered":"2023-10-06T05:05:39.928200+0000","last_unstale":"2023-10-06T05:05:40.920703+0000","last_undegraded":"2023-10-06T05:05:40.920703+0000","last_fullsized":"2023-10-06T05:05:40.920703+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:05.926590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.926142+0000","last_change":"2023-10-06T05:05:40.921820+0000","last_active":"2023-10-06T05:05:40.926142+0000","last_peered":"2023-10-06T05:05:40.926142+0000","last_clean":"2023-10-06T05:05:40.926142+0000","last_became_active":"2023-10-06T05:05:40.921541+0000","last_became_peered":"2023-10-06T05:05:40.921541+0000","last_unstale":"2023-10-06T05:05:40.926142+0000","last_undegraded":"2023-10-06T05:05:40.926142+0000","last_fullsized":"2023-10-06T05:05:40.926142+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:11.565930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.931437+0000","last_change":"2023-10-06T05:05:39.922396+0000","last_active":"2023-10-06T05:05:40.931437+0000","last_peered":"2023-10-06T05:05:40.931437+0000","last_clean":"2023-10-06T05:05:40.931437+0000","last_became_active":"2023-10-06T05:05:39.922116+0000","last_became_peered":"2023-10-06T05:05:39.922116+0000","last_unstale":"2023-10-06T05:05:40.931437+0000","last_undegraded":"2023-10-06T05:05:40.931437+0000","last_fullsized":"2023-10-06T05:05:40.931437+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:25.953699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.947023+0000","last_change":"2023-10-06T05:05:40.938953+0000","last_active":"2023-10-06T05:05:40.947023+0000","last_peered":"2023-10-06T05:05:40.947023+0000","last_clean":"2023-10-06T05:05:40.947023+0000","last_became_active":"2023-10-06T05:05:40.938643+0000","last_became_peered":"2023-10-06T05:05:40.938643+0000","last_unstale":"2023-10-06T05:05:40.947023+0000","last_undegraded":"2023-10-06T05:05:40.947023+0000","last_fullsized":"2023-10-06T05:05:40.947023+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:46.429634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.916613+0000","last_change":"2023-10-06T05:05:39.929045+0000","last_active":"2023-10-06T05:05:40.916613+0000","last_peered":"2023-10-06T05:05:40.916613+0000","last_clean":"2023-10-06T05:05:40.916613+0000","last_became_active":"2023-10-06T05:05:39.928706+0000","last_became_peered":"2023-10-06T05:05:39.928706+0000","last_unstale":"2023-10-06T05:05:40.916613+0000","last_undegraded":"2023-10-06T05:05:40.916613+0000","last_fullsized":"2023-10-06T05:05:40.916613+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:29.272255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.931295+0000","last_change":"2023-10-06T05:05:40.928471+0000","last_active":"2023-10-06T05:05:40.931295+0000","last_peered":"2023-10-06T05:05:40.931295+0000","last_clean":"2023-10-06T05:05:40.931295+0000","last_became_active":"2023-10-06T05:05:40.928138+0000","last_became_peered":"2023-10-06T05:05:40.928138+0000","last_unstale":"2023-10-06T05:05:40.931295+0000","last_undegraded":"2023-10-06T05:05:40.931295+0000","last_fullsized":"2023-10-06T05:05:40.931295+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:21.333982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.915210+0000","last_change":"2023-10-06T05:05:39.930030+0000","last_active":"2023-10-06T05:05:40.915210+0000","last_peered":"2023-10-06T05:05:40.915210+0000","last_clean":"2023-10-06T05:05:40.915210+0000","last_became_active":"2023-10-06T05:05:39.929713+0000","last_became_peered":"2023-10-06T05:05:39.929713+0000","last_unstale":"2023-10-06T05:05:40.915210+0000","last_undegraded":"2023-10-06T05:05:40.915210+0000","last_fullsized":"2023-10-06T05:05:40.915210+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:00.155842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950501+0000","last_change":"2023-10-06T05:05:40.945260+0000","last_active":"2023-10-06T05:05:40.950501+0000","last_peered":"2023-10-06T05:05:40.950501+0000","last_clean":"2023-10-06T05:05:40.950501+0000","last_became_active":"2023-10-06T05:05:40.944947+0000","last_became_peered":"2023-10-06T05:05:40.944947+0000","last_unstale":"2023-10-06T05:05:40.950501+0000","last_undegraded":"2023-10-06T05:05:40.950501+0000","last_fullsized":"2023-10-06T05:05:40.950501+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:22.225371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.921397+0000","last_change":"2023-10-06T05:05:39.927197+0000","last_active":"2023-10-06T05:05:40.921397+0000","last_peered":"2023-10-06T05:05:40.921397+0000","last_clean":"2023-10-06T05:05:40.921397+0000","last_became_active":"2023-10-06T05:05:39.926927+0000","last_became_peered":"2023-10-06T05:05:39.926927+0000","last_unstale":"2023-10-06T05:05:40.921397+0000","last_undegraded":"2023-10-06T05:05:40.921397+0000","last_fullsized":"2023-10-06T05:05:40.921397+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:22.307584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.920707+0000","last_change":"2023-10-06T05:05:40.916765+0000","last_active":"2023-10-06T05:05:40.920707+0000","last_peered":"2023-10-06T05:05:40.920707+0000","last_clean":"2023-10-06T05:05:40.920707+0000","last_became_active":"2023-10-06T05:05:40.916428+0000","last_became_peered":"2023-10-06T05:05:40.916428+0000","last_unstale":"2023-10-06T05:05:40.920707+0000","last_undegraded":"2023-10-06T05:05:40.920707+0000","last_fullsized":"2023-10-06T05:05:40.920707+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:10.851897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.910908+0000","last_change":"2023-10-06T05:05:39.921721+0000","last_active":"2023-10-06T05:05:40.910908+0000","last_peered":"2023-10-06T05:05:40.910908+0000","last_clean":"2023-10-06T05:05:40.910908+0000","last_became_active":"2023-10-06T05:05:39.921430+0000","last_became_peered":"2023-10-06T05:05:39.921430+0000","last_unstale":"2023-10-06T05:05:40.910908+0000","last_undegraded":"2023-10-06T05:05:40.910908+0000","last_fullsized":"2023-10-06T05:05:40.910908+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:57.786548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.940595+0000","last_change":"2023-10-06T05:05:40.937499+0000","last_active":"2023-10-06T05:05:40.940595+0000","last_peered":"2023-10-06T05:05:40.940595+0000","last_clean":"2023-10-06T05:05:40.940595+0000","last_became_active":"2023-10-06T05:05:40.937206+0000","last_became_peered":"2023-10-06T05:05:40.937206+0000","last_unstale":"2023-10-06T05:05:40.940595+0000","last_undegraded":"2023-10-06T05:05:40.940595+0000","last_fullsized":"2023-10-06T05:05:40.940595+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:07.458157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.915086+0000","last_change":"2023-10-06T05:05:39.923515+0000","last_active":"2023-10-06T05:05:40.915086+0000","last_peered":"2023-10-06T05:05:40.915086+0000","last_clean":"2023-10-06T05:05:40.915086+0000","last_became_active":"2023-10-06T05:05:39.923082+0000","last_became_peered":"2023-10-06T05:05:39.923082+0000","last_unstale":"2023-10-06T05:05:40.915086+0000","last_undegraded":"2023-10-06T05:05:40.915086+0000","last_fullsized":"2023-10-06T05:05:40.915086+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:07.808357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.925608+0000","last_change":"2023-10-06T05:05:40.923428+0000","last_active":"2023-10-06T05:05:40.925608+0000","last_peered":"2023-10-06T05:05:40.925608+0000","last_clean":"2023-10-06T05:05:40.925608+0000","last_became_active":"2023-10-06T05:05:40.923167+0000","last_became_peered":"2023-10-06T05:05:40.923167+0000","last_unstale":"2023-10-06T05:05:40.925608+0000","last_undegraded":"2023-10-06T05:05:40.925608+0000","last_fullsized":"2023-10-06T05:05:40.925608+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:18.198649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.916967+0000","last_change":"2023-10-06T05:05:39.923687+0000","last_active":"2023-10-06T05:05:40.916967+0000","last_peered":"2023-10-06T05:05:40.916967+0000","last_clean":"2023-10-06T05:05:40.916967+0000","last_became_active":"2023-10-06T05:05:39.923378+0000","last_became_peered":"2023-10-06T05:05:39.923378+0000","last_unstale":"2023-10-06T05:05:40.916967+0000","last_undegraded":"2023-10-06T05:05:40.916967+0000","last_fullsized":"2023-10-06T05:05:40.916967+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:49.012909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.933837+0000","last_change":"2023-10-06T05:05:40.929228+0000","last_active":"2023-10-06T05:05:40.933837+0000","last_peered":"2023-10-06T05:05:40.933837+0000","last_clean":"2023-10-06T05:05:40.933837+0000","last_became_active":"2023-10-06T05:05:40.928352+0000","last_became_peered":"2023-10-06T05:05:40.928352+0000","last_unstale":"2023-10-06T05:05:40.933837+0000","last_undegraded":"2023-10-06T05:05:40.933837+0000","last_fullsized":"2023-10-06T05:05:40.933837+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:10.201272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.909941+0000","last_change":"2023-10-06T05:05:39.930235+0000","last_active":"2023-10-06T05:05:40.909941+0000","last_peered":"2023-10-06T05:05:40.909941+0000","last_clean":"2023-10-06T05:05:40.909941+0000","last_became_active":"2023-10-06T05:05:39.929890+0000","last_became_peered":"2023-10-06T05:05:39.929890+0000","last_unstale":"2023-10-06T05:05:40.909941+0000","last_undegraded":"2023-10-06T05:05:40.909941+0000","last_fullsized":"2023-10-06T05:05:40.909941+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:16.889956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.930569+0000","last_change":"2023-10-06T05:05:40.928894+0000","last_active":"2023-10-06T05:05:40.930569+0000","last_peered":"2023-10-06T05:05:40.930569+0000","last_clean":"2023-10-06T05:05:40.930569+0000","last_became_active":"2023-10-06T05:05:40.928638+0000","last_became_peered":"2023-10-06T05:05:40.928638+0000","last_unstale":"2023-10-06T05:05:40.930569+0000","last_undegraded":"2023-10-06T05:05:40.930569+0000","last_fullsized":"2023-10-06T05:05:40.930569+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:49.513363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946359+0000","last_change":"2023-10-06T05:05:39.924825+0000","last_active":"2023-10-06T05:05:40.946359+0000","last_peered":"2023-10-06T05:05:40.946359+0000","last_clean":"2023-10-06T05:05:40.946359+0000","last_became_active":"2023-10-06T05:05:39.923420+0000","last_became_peered":"2023-10-06T05:05:39.923420+0000","last_unstale":"2023-10-06T05:05:40.946359+0000","last_undegraded":"2023-10-06T05:05:40.946359+0000","last_fullsized":"2023-10-06T05:05:40.946359+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:57.786548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":175,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.910780+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T05:05:40.910780+0000","last_peered":"2023-10-06T05:05:40.910780+0000","last_clean":"2023-10-06T05:05:40.910780+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T05:05:40.910780+0000","last_undegraded":"2023-10-06T05:05:40.910780+0000","last_fullsized":"2023-10-06T05:05:40.910780+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950557+0000","last_change":"2023-10-06T05:05:40.945808+0000","last_active":"2023-10-06T05:05:40.950557+0000","last_peered":"2023-10-06T05:05:40.950557+0000","last_clean":"2023-10-06T05:05:40.950557+0000","last_became_active":"2023-10-06T05:05:40.945531+0000","last_became_peered":"2023-10-06T05:05:40.945531+0000","last_unstale":"2023-10-06T05:05:40.950557+0000","last_undegraded":"2023-10-06T05:05:40.950557+0000","last_fullsized":"2023-10-06T05:05:40.950557+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:06.936450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.914985+0000","last_change":"2023-10-06T05:05:39.923348+0000","last_active":"2023-10-06T05:05:40.914985+0000","last_peered":"2023-10-06T05:05:40.914985+0000","last_clean":"2023-10-06T05:05:40.914985+0000","last_became_active":"2023-10-06T05:05:39.922810+0000","last_became_peered":"2023-10-06T05:05:39.922810+0000","last_unstale":"2023-10-06T05:05:40.914985+0000","last_undegraded":"2023-10-06T05:05:40.914985+0000","last_fullsized":"2023-10-06T05:05:40.914985+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:20:19.361912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":163,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.915504+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T05:05:40.915504+0000","last_peered":"2023-10-06T05:05:40.915504+0000","last_clean":"2023-10-06T05:05:40.915504+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T05:05:40.915504+0000","last_undegraded":"2023-10-06T05:05:40.915504+0000","last_fullsized":"2023-10-06T05:05:40.915504+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.949849+0000","last_change":"2023-10-06T05:05:40.947853+0000","last_active":"2023-10-06T05:05:40.949849+0000","last_peered":"2023-10-06T05:05:40.949849+0000","last_clean":"2023-10-06T05:05:40.949849+0000","last_became_active":"2023-10-06T05:05:40.947476+0000","last_became_peered":"2023-10-06T05:05:40.947476+0000","last_unstale":"2023-10-06T05:05:40.949849+0000","last_undegraded":"2023-10-06T05:05:40.949849+0000","last_fullsized":"2023-10-06T05:05:40.949849+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:58.796408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"192'3","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.936421+0000","last_change":"2023-10-06T05:05:39.922291+0000","last_active":"2023-10-06T05:05:40.936421+0000","last_peered":"2023-10-06T05:05:40.936421+0000","last_clean":"2023-10-06T05:05:40.936421+0000","last_became_active":"2023-10-06T05:05:39.922032+0000","last_became_peered":"2023-10-06T05:05:39.922032+0000","last_unstale":"2023-10-06T05:05:40.936421+0000","last_undegraded":"2023-10-06T05:05:40.936421+0000","last_fullsized":"2023-10-06T05:05:40.936421+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:52.678186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":160,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.924883+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T05:05:40.924883+0000","last_peered":"2023-10-06T05:05:40.924883+0000","last_clean":"2023-10-06T05:05:40.924883+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T05:05:40.924883+0000","last_undegraded":"2023-10-06T05:05:40.924883+0000","last_fullsized":"2023-10-06T05:05:40.924883+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"183'326","reported_seq":1053,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.923602+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T05:05:40.923602+0000","last_peered":"2023-10-06T05:05:40.923602+0000","last_clean":"2023-10-06T05:05:40.923602+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T05:05:40.923602+0000","last_undegraded":"2023-10-06T05:05:40.923602+0000","last_fullsized":"2023-10-06T05:05:40.923602+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.935830+0000","last_change":"2023-10-06T05:05:40.931126+0000","last_active":"2023-10-06T05:05:40.935830+0000","last_peered":"2023-10-06T05:05:40.935830+0000","last_clean":"2023-10-06T05:05:40.935830+0000","last_became_active":"2023-10-06T05:05:40.930016+0000","last_became_peered":"2023-10-06T05:05:40.930016+0000","last_unstale":"2023-10-06T05:05:40.935830+0000","last_undegraded":"2023-10-06T05:05:40.935830+0000","last_fullsized":"2023-10-06T05:05:40.935830+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:49.846088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"192'6","reported_seq":16,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.952749+0000","last_change":"2023-10-06T05:05:39.928213+0000","last_active":"2023-10-06T05:05:40.952749+0000","last_peered":"2023-10-06T05:05:40.952749+0000","last_clean":"2023-10-06T05:05:40.952749+0000","last_became_active":"2023-10-06T05:05:39.927927+0000","last_became_peered":"2023-10-06T05:05:39.927927+0000","last_unstale":"2023-10-06T05:05:40.952749+0000","last_undegraded":"2023-10-06T05:05:40.952749+0000","last_fullsized":"2023-10-06T05:05:40.952749+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:40.706784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":163,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.915543+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T05:05:40.915543+0000","last_peered":"2023-10-06T05:05:40.915543+0000","last_clean":"2023-10-06T05:05:40.915543+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T05:05:40.915543+0000","last_undegraded":"2023-10-06T05:05:40.915543+0000","last_fullsized":"2023-10-06T05:05:40.915543+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.932909+0000","last_change":"2023-10-06T05:05:40.929922+0000","last_active":"2023-10-06T05:05:40.932909+0000","last_peered":"2023-10-06T05:05:40.932909+0000","last_clean":"2023-10-06T05:05:40.932909+0000","last_became_active":"2023-10-06T05:05:40.929618+0000","last_became_peered":"2023-10-06T05:05:40.929618+0000","last_unstale":"2023-10-06T05:05:40.932909+0000","last_undegraded":"2023-10-06T05:05:40.932909+0000","last_fullsized":"2023-10-06T05:05:40.932909+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:42.478177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.916517+0000","last_change":"2023-10-06T05:05:39.921775+0000","last_active":"2023-10-06T05:05:40.916517+0000","last_peered":"2023-10-06T05:05:40.916517+0000","last_clean":"2023-10-06T05:05:40.916517+0000","last_became_active":"2023-10-06T05:05:39.921525+0000","last_became_peered":"2023-10-06T05:05:39.921525+0000","last_unstale":"2023-10-06T05:05:40.916517+0000","last_undegraded":"2023-10-06T05:05:40.916517+0000","last_fullsized":"2023-10-06T05:05:40.916517+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:12.112490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":163,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.925284+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T05:05:40.925284+0000","last_peered":"2023-10-06T05:05:40.925284+0000","last_clean":"2023-10-06T05:05:40.925284+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T05:05:40.925284+0000","last_undegraded":"2023-10-06T05:05:40.925284+0000","last_fullsized":"2023-10-06T05:05:40.925284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.943745+0000","last_change":"2023-10-06T05:05:40.942382+0000","last_active":"2023-10-06T05:05:40.943745+0000","last_peered":"2023-10-06T05:05:40.943745+0000","last_clean":"2023-10-06T05:05:40.943745+0000","last_became_active":"2023-10-06T05:05:40.941889+0000","last_became_peered":"2023-10-06T05:05:40.941889+0000","last_unstale":"2023-10-06T05:05:40.943745+0000","last_undegraded":"2023-10-06T05:05:40.943745+0000","last_fullsized":"2023-10-06T05:05:40.943745+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:10.851897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.915883+0000","last_change":"2023-10-06T05:05:39.929195+0000","last_active":"2023-10-06T05:05:40.915883+0000","last_peered":"2023-10-06T05:05:40.915883+0000","last_clean":"2023-10-06T05:05:40.915883+0000","last_became_active":"2023-10-06T05:05:39.928902+0000","last_became_peered":"2023-10-06T05:05:39.928902+0000","last_unstale":"2023-10-06T05:05:40.915883+0000","last_undegraded":"2023-10-06T05:05:40.915883+0000","last_fullsized":"2023-10-06T05:05:40.915883+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:23.422305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":155,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.917075+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T05:05:40.917075+0000","last_peered":"2023-10-06T05:05:40.917075+0000","last_clean":"2023-10-06T05:05:40.917075+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T05:05:40.917075+0000","last_undegraded":"2023-10-06T05:05:40.917075+0000","last_fullsized":"2023-10-06T05:05:40.917075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.942379+0000","last_change":"2023-10-06T05:05:40.939000+0000","last_active":"2023-10-06T05:05:40.942379+0000","last_peered":"2023-10-06T05:05:40.942379+0000","last_clean":"2023-10-06T05:05:40.942379+0000","last_became_active":"2023-10-06T05:05:40.938765+0000","last_became_peered":"2023-10-06T05:05:40.938765+0000","last_unstale":"2023-10-06T05:05:40.942379+0000","last_undegraded":"2023-10-06T05:05:40.942379+0000","last_fullsized":"2023-10-06T05:05:40.942379+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:16.297316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"192'8","reported_seq":18,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946955+0000","last_change":"2023-10-06T05:05:39.927836+0000","last_active":"2023-10-06T05:05:40.946955+0000","last_peered":"2023-10-06T05:05:40.946955+0000","last_clean":"2023-10-06T05:05:40.946955+0000","last_became_active":"2023-10-06T05:05:39.927333+0000","last_became_peered":"2023-10-06T05:05:39.927333+0000","last_unstale":"2023-10-06T05:05:40.946955+0000","last_undegraded":"2023-10-06T05:05:40.946955+0000","last_fullsized":"2023-10-06T05:05:40.946955+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:04.303464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":186,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.911654+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T05:05:40.911654+0000","last_peered":"2023-10-06T05:05:40.911654+0000","last_clean":"2023-10-06T05:05:40.911654+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T05:05:40.911654+0000","last_undegraded":"2023-10-06T05:05:40.911654+0000","last_fullsized":"2023-10-06T05:05:40.911654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.936102+0000","last_change":"2023-10-06T05:05:40.932466+0000","last_active":"2023-10-06T05:05:40.936102+0000","last_peered":"2023-10-06T05:05:40.936102+0000","last_clean":"2023-10-06T05:05:40.936102+0000","last_became_active":"2023-10-06T05:05:40.932197+0000","last_became_peered":"2023-10-06T05:05:40.932197+0000","last_unstale":"2023-10-06T05:05:40.936102+0000","last_undegraded":"2023-10-06T05:05:40.936102+0000","last_fullsized":"2023-10-06T05:05:40.936102+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:06.936450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.916863+0000","last_change":"2023-10-06T05:05:39.922809+0000","last_active":"2023-10-06T05:05:40.916863+0000","last_peered":"2023-10-06T05:05:40.916863+0000","last_clean":"2023-10-06T05:05:40.916863+0000","last_became_active":"2023-10-06T05:05:39.922518+0000","last_became_peered":"2023-10-06T05:05:39.922518+0000","last_unstale":"2023-10-06T05:05:40.916863+0000","last_undegraded":"2023-10-06T05:05:40.916863+0000","last_fullsized":"2023-10-06T05:05:40.916863+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:04.303464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":186,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.910317+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T05:05:40.910317+0000","last_peered":"2023-10-06T05:05:40.910317+0000","last_clean":"2023-10-06T05:05:40.910317+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T05:05:40.910317+0000","last_undegraded":"2023-10-06T05:05:40.910317+0000","last_fullsized":"2023-10-06T05:05:40.910317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575946,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304316,"kb_used_data":2932,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067524,"statfs":{"total":96636764160,"available":96325144576,"internally_reserved":0,"allocated":3002368,"data_stored":1864372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.75,"15min":0.74299999999999999},"min":{"1min":0.498,"5min":0.48499999999999999,"15min":0.44500000000000001},"max":{"1min":0.94199999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.64900000000000002,"5min":0.48699999999999999,"15min":0.48399999999999999},"max":{"1min":0.94099999999999995,"5min":0.94599999999999995,"15min":1.0529999999999999},"last":0.79300000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82299999999999995,"15min":0.80700000000000005},"min":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.50800000000000001},"max":{"1min":1.0640000000000001,"5min":1.0720000000000001,"15min":1.073},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81000000000000005,"15min":0.81299999999999994},"min":{"1min":0.63400000000000001,"5min":0.627,"15min":0.55900000000000005},"max":{"1min":0.96599999999999997,"5min":0.99299999999999999,"15min":1.0860000000000001},"last":0.92200000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.502},"max":{"1min":1.034,"5min":1.034,"15min":1.095},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78500000000000003,"15min":0.78400000000000003},"min":{"1min":0.61899999999999999,"5min":0.54300000000000004,"15min":0.50900000000000001},"max":{"1min":1.107,"5min":1.107,"15min":1.1160000000000001},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.77000000000000002,"15min":0.77700000000000002},"min":{"1min":0.68300000000000005,"5min":0.55300000000000005,"15min":0.51400000000000001},"max":{"1min":0.99099999999999999,"5min":1.0029999999999999,"15min":1.101},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.63200000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0149999999999999,"5min":1.024,"15min":1.0369999999999999},"last":0.89300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.62},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77700000000000002,"5min":1.282,"15min":1.355},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89700000000000002,"5min":1.196,"15min":1.208},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63500000000000001,"15min":0.65800000000000003},"min":{"1min":0.42299999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0009999999999999,"5min":1.044,"15min":1.054},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66400000000000003,"15min":0.69299999999999995},"min":{"1min":0.50800000000000001,"5min":0.48399999999999999,"15min":0.47699999999999998},"max":{"1min":0.871,"5min":0.98999999999999999,"15min":1.0760000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.57799999999999996,"5min":0.497,"15min":0.48799999999999999},"max":{"1min":0.92200000000000004,"5min":1.02,"15min":1.044},"last":0.747},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.69699999999999995,"15min":0.71799999999999997},"min":{"1min":0.56599999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95699999999999996,"5min":1.0229999999999999,"15min":1.0469999999999999},"last":0.71899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575947,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299644,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072196,"statfs":{"total":96636764160,"available":96329928704,"internally_reserved":0,"allocated":2412544,"data_stored":1274982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73299999999999998,"15min":0.72799999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.40699999999999997},"max":{"1min":1.0009999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.749,"15min":0.76100000000000001},"min":{"1min":0.53000000000000003,"5min":0.48799999999999999,"15min":0.46600000000000003},"max":{"1min":0.96399999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.96799999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.77100000000000002,"15min":0.75900000000000001},"min":{"1min":0.53200000000000003,"5min":0.47099999999999997,"15min":0.378},"max":{"1min":0.92300000000000004,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.77800000000000002,"15min":0.76500000000000001},"min":{"1min":0.60199999999999998,"5min":0.50600000000000001,"15min":0.42399999999999999},"max":{"1min":0.94899999999999995,"5min":1.425,"15min":1.425},"last":0.88400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77500000000000002,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.44700000000000001},"max":{"1min":1.0069999999999999,"5min":1.083,"15min":1.147},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77200000000000002,"15min":0.77400000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48299999999999998},"max":{"1min":1.03,"5min":1.0640000000000001,"15min":1.097},"last":0.98799999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.59599999999999997,"5min":0.54900000000000004,"15min":0.505},"max":{"1min":0.97999999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":1.016},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81699999999999995,"15min":0.79600000000000004},"min":{"1min":0.627,"5min":0.60799999999999998,"15min":0.53700000000000003},"max":{"1min":0.97399999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.92500000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60799999999999998,"15min":0.58899999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.38500000000000001},"max":{"1min":0.73199999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.435},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64400000000000002,"15min":0.63900000000000001},"min":{"1min":0.46600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94299999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69699999999999995,"15min":0.70199999999999996},"min":{"1min":0.5,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98099999999999998,"5min":1.0680000000000001,"15min":1.069},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.746,"15min":0.73399999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.41199999999999998},"max":{"1min":0.95399999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.50900000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.92700000000000005,"5min":1.0369999999999999,"15min":1.0580000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.747,"15min":0.73299999999999998},"min":{"1min":0.56799999999999995,"5min":0.45100000000000001,"15min":0.45000000000000001},"max":{"1min":0.93000000000000005,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.68300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575946,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312524,"kb_used_data":2948,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059316,"statfs":{"total":96636764160,"available":96316739584,"internally_reserved":0,"allocated":3018752,"data_stored":1866886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32400000000000001},"max":{"1min":0.73199999999999998,"5min":1.155,"15min":1.155},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.38900000000000001},"max":{"1min":0.77100000000000002,"5min":0.88900000000000001,"15min":0.90100000000000002},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.52100000000000002},"min":{"1min":0.379,"5min":0.378,"15min":0.35499999999999998},"max":{"1min":0.64800000000000002,"5min":1.153,"15min":1.153},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.37},"max":{"1min":0.86099999999999999,"5min":1.018,"15min":1.018},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.41799999999999998},"max":{"1min":0.82999999999999996,"5min":1.131,"15min":1.131},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.44,"5min":0.40500000000000003,"15min":0.38100000000000001},"max":{"1min":0.78100000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.79500000000000004,"15min":0.80800000000000005},"min":{"1min":0.69199999999999995,"5min":0.50800000000000001,"15min":0.46400000000000002},"max":{"1min":1.161,"5min":1.1739999999999999,"15min":1.2250000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.84299999999999997,"15min":0.85099999999999998},"min":{"1min":0.70999999999999996,"5min":0.61799999999999999,"15min":0.57799999999999996},"max":{"1min":1.04,"5min":1.236,"15min":1.448},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83899999999999997,"15min":0.84799999999999998},"min":{"1min":0.62,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.083,"5min":1.21,"15min":1.484},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.84199999999999997,"15min":0.84799999999999998},"min":{"1min":0.624,"5min":0.51000000000000001,"15min":0.502},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.2829999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.85799999999999998,"15min":0.85899999999999999},"min":{"1min":0.755,"5min":0.60699999999999998,"15min":0.58099999999999996},"max":{"1min":1.1639999999999999,"5min":1.2090000000000001,"15min":1.2709999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.86199999999999999,"15min":0.87},"min":{"1min":0.65900000000000003,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1950000000000001,"5min":1.242,"15min":1.4690000000000001},"last":0.73399999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88700000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.59899999999999998},"max":{"1min":1.167,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.88400000000000001,"15min":0.88500000000000001},"min":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.61699999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.3080000000000001},"last":0.80200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575948,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303744,"kb_used_data":2360,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068096,"statfs":{"total":96636764160,"available":96325730304,"internally_reserved":0,"allocated":2416640,"data_stored":1274108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:05:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49199999999999999,"15min":0.504},"min":{"1min":0.35599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.747,"5min":0.80800000000000005,"15min":0.83399999999999996},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.55600000000000005},"min":{"1min":0.377,"5min":0.35799999999999998,"15min":0.34399999999999997},"max":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:05:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83299999999999996,"5min":0.86899999999999999,"15min":0.91000000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.38900000000000001},"max":{"1min":0.69999999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.38300000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:05:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.34300000000000003},"max":{"1min":0.71799999999999997,"5min":0.753,"15min":0.86699999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.42899999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.48499999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:05:12 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.86599999999999999,"15min":0.86499999999999999},"min":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.54300000000000004},"max":{"1min":1.1419999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.85999999999999999,"15min":0.86499999999999999},"min":{"1min":0.75,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1220000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:05:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.84799999999999998,"15min":0.84699999999999998},"min":{"1min":0.69499999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1140000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85199999999999998,"15min":0.85399999999999998},"min":{"1min":0.63900000000000001,"5min":0.57399999999999995,"15min":0.45600000000000002},"max":{"1min":1.204,"5min":1.2110000000000001,"15min":1.466},"last":0.78600000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:05:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.83999999999999997,"15min":0.84499999999999997},"min":{"1min":0.67000000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1240000000000001,"5min":1.2869999999999999,"15min":1.4419999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.875,"5min":0.88100000000000001,"15min":0.88800000000000001},"min":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.21,"5min":1.321,"15min":1.54},"last":0.80300000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:05:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88400000000000001,"15min":0.877},"min":{"1min":0.624,"5min":0.624,"15min":0.59799999999999998},"max":{"1min":1.2490000000000001,"5min":1.367,"15min":1.367},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88600000000000001,"15min":0.88300000000000001},"min":{"1min":0.65600000000000003,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0549999999999999,"5min":1.294,"15min":1.6399999999999999},"last":0.64400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575947,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316024,"kb_used_data":2352,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055816,"statfs":{"total":96636764160,"available":96313155584,"internally_reserved":0,"allocated":2408448,"data_stored":1274004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.33400000000000002,"5min":0.31900000000000001,"15min":0.29199999999999998},"max":{"1min":0.746,"5min":0.748,"15min":0.80300000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.37,"5min":0.35999999999999999,"15min":0.31},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.91200000000000003},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78300000000000003,"5min":0.89400000000000002,"15min":0.92400000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.432,"5min":0.41399999999999998,"15min":0.35999999999999999},"max":{"1min":0.70299999999999996,"5min":0.86599999999999999,"15min":0.95299999999999996},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.751,"5min":0.78400000000000003,"15min":0.85999999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.42199999999999999,"5min":0.40799999999999997,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.66500000000000004,"5min":0.57799999999999996,"15min":0.55900000000000005},"max":{"1min":1.046,"5min":1.1910000000000001,"15min":1.383},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86799999999999999,"15min":0.876},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.59199999999999997},"max":{"1min":1.0700000000000001,"5min":1.246,"15min":1.254},"last":1.0149999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.879,"15min":0.88300000000000001},"min":{"1min":0.66800000000000004,"5min":0.59099999999999997,"15min":0.57599999999999996},"max":{"1min":1.0309999999999999,"5min":1.345,"15min":1.345},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.89000000000000001,"15min":0.89400000000000002},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0249999999999999,"5min":1.329,"15min":1.339},"last":1.0229999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.874,"15min":0.86299999999999999},"min":{"1min":0.66200000000000003,"5min":0.60499999999999998,"15min":0.58199999999999996},"max":{"1min":1.0580000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.84199999999999997,"15min":0.85799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55000000000000004},"max":{"1min":1.054,"5min":1.135,"15min":1.4450000000000001},"last":0.89700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.86699999999999999,"15min":0.86399999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.997,"5min":1.276,"15min":1.425},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.91100000000000003,"15min":0.90400000000000003},"min":{"1min":0.66600000000000004,"5min":0.61399999999999999,"15min":0.59399999999999997},"max":{"1min":1.1970000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.97299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575947,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316048,"kb_used_data":2376,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055792,"statfs":{"total":96636764160,"available":96313131008,"internally_reserved":0,"allocated":2433024,"data_stored":1277094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.29799999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.40300000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.65200000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.49399999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.34899999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80500000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.307},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.58299999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.371}]},{"osd":2,"last update":"Fri Oct 6 05:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59499999999999997,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.39700000000000002},"max":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.81499999999999995},"last":0.432},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65000000000000002,"15min":0.63900000000000001},"min":{"1min":0.48499999999999999,"5min":0.47599999999999998,"15min":0.38500000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.92300000000000004},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.55700000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0249999999999999,"5min":1.1559999999999999,"15min":1.2849999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.88200000000000001,"15min":0.876},"min":{"1min":0.70999999999999996,"5min":0.64400000000000002,"15min":0.52500000000000002},"max":{"1min":1.044,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82899999999999996,"15min":0.82199999999999995},"min":{"1min":0.624,"5min":0.57499999999999996,"15min":0.51300000000000001},"max":{"1min":1.115,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84899999999999998,"15min":0.84399999999999997},"min":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.56100000000000005},"max":{"1min":1.0209999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.55900000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.66500000000000004,"5min":0.55700000000000005,"15min":0.52800000000000002},"max":{"1min":1.0640000000000001,"5min":1.2150000000000001,"15min":1.288},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.89600000000000002,"15min":0.89900000000000002},"min":{"1min":0.76700000000000002,"5min":0.63,"15min":0.57499999999999996},"max":{"1min":1.123,"5min":1.2669999999999999,"15min":1.411},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.871,"15min":0.875},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.57599999999999996},"max":{"1min":1.0649999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.86099999999999999,"15min":0.85399999999999998},"min":{"1min":0.62,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.089,"5min":1.365,"15min":1.389},"last":0.622}]}]},{"osd":4,"up_from":13,"seq":55834575947,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311948,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059892,"statfs":{"total":96636764160,"available":96317329408,"internally_reserved":0,"allocated":2428928,"data_stored":1275718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.42399999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.2},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76800000000000002,"15min":0.76100000000000001},"min":{"1min":0.57899999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.018,"5min":1.048,"15min":1.048},"last":0.745}]},{"osd":1,"last update":"Fri Oct 6 05:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.56499999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78000000000000003,"15min":0.78100000000000003},"min":{"1min":0.52400000000000002,"5min":0.48499999999999999,"15min":0.46700000000000003},"max":{"1min":0.99099999999999999,"5min":1.026,"15min":1.258},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.77500000000000002},"min":{"1min":0.58599999999999997,"5min":0.55000000000000004,"15min":0.48299999999999998},"max":{"1min":0.92200000000000004,"5min":1.0089999999999999,"15min":1.0309999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78000000000000003,"15min":0.78700000000000003},"min":{"1min":0.60799999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.88600000000000001,"5min":1.0049999999999999,"15min":1.2929999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76600000000000001,"15min":0.76400000000000001},"min":{"1min":0.59199999999999997,"5min":0.52000000000000002,"15min":0.49299999999999999},"max":{"1min":0.91500000000000004,"5min":0.97999999999999998,"15min":1.2330000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.79900000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.95499999999999996,"5min":1.012,"15min":1.034},"last":0.85599999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.48299999999999998,"5min":0.41599999999999998,"15min":0.41199999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.323},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.39700000000000002},"max":{"1min":0.877,"5min":1.01,"15min":1.1910000000000001},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.67500000000000004,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.42999999999999999,"15min":0.42099999999999999},"max":{"1min":0.88800000000000001,"5min":1.024,"15min":1.079},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69199999999999995,"15min":0.71299999999999997},"min":{"1min":0.54300000000000004,"5min":0.44400000000000001,"15min":0.41999999999999998},"max":{"1min":0.93000000000000005,"5min":1.022,"15min":1.3540000000000001},"last":0.51000000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.71699999999999997},"min":{"1min":0.502,"5min":0.47699999999999998,"15min":0.45200000000000001},"max":{"1min":0.97299999999999998,"5min":1.0640000000000001,"15min":1.155},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73999999999999999,"15min":0.75700000000000001},"min":{"1min":0.56499999999999995,"5min":0.505,"15min":0.47299999999999998},"max":{"1min":1.0069999999999999,"5min":1.079,"15min":1.149},"last":0.79100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575947,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307840,"kb_used_data":2360,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064000,"statfs":{"total":96636764160,"available":96321536000,"internally_reserved":0,"allocated":2416640,"data_stored":1274128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.63700000000000001,"5min":0.504,"15min":0.39800000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.171},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.751,"15min":0.73899999999999999},"min":{"1min":0.60699999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.999,"5min":0.999,"15min":1.0209999999999999},"last":0.78700000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.64000000000000001,"5min":0.505,"15min":0.42099999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.2330000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.77100000000000002},"min":{"1min":0.61099999999999999,"5min":0.55000000000000004,"15min":0.42699999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.095},"last":0.84899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81299999999999994,"15min":0.80600000000000005},"min":{"1min":0.63700000000000001,"5min":0.51800000000000002,"15min":0.46800000000000003},"max":{"1min":1.0700000000000001,"5min":1.0780000000000001,"15min":1.325},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.83199999999999996,"15min":0.82499999999999996},"min":{"1min":0.71099999999999997,"5min":0.628,"15min":0.52100000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.349},"last":0.77400000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.66100000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.2809999999999999},"last":0.747},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.80100000000000005,"15min":0.79300000000000004},"min":{"1min":0.61399999999999999,"5min":0.57899999999999996,"15min":0.55400000000000005},"max":{"1min":1.095,"5min":1.095,"15min":1.298},"last":0.86499999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.624,"15min":0.63400000000000001},"min":{"1min":0.47799999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93799999999999994,"5min":0.97699999999999998,"15min":1.0049999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.40400000000000003,"15min":0.38300000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.042},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.47899999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93999999999999995,"5min":0.98799999999999999,"15min":1.0369999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.53600000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.496,"5min":0.45700000000000002,"15min":0.44500000000000001},"max":{"1min":0.99099999999999999,"5min":1.0269999999999999,"15min":1.0669999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.54600000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.67000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:05:45.313 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:05:45.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.527+0000 7f0a4e39d700 1 Processor -- start 2023-10-06T05:05:45.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.528+0000 7f0a4e39d700 1 -- start start 2023-10-06T05:05:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.529+0000 7f0a4e39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f0a480ad870 0x7f0a4806b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.529+0000 7f0a4e39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0a4806bf20 0x7f0a4806a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.529+0000 7f0a4e39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a480b40f0 con 0x7f0a48142ea0 2023-10-06T05:05:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.529+0000 7f0a4e39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4806c3a0 con 0x7f0a4806bf20 2023-10-06T05:05:45.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.529+0000 7f0a4e39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a48069640 con 0x7f0a480ad870 2023-10-06T05:05:45.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.529+0000 7f0a4db9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0a4806bf20 0x7f0a4806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:45.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.529+0000 7f0a4db9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0a4806bf20 0x7f0a4806a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45466/0 (socket says 172.21.15.55:45466) 2023-10-06T05:05:45.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.529+0000 7f0a4db9c700 1 -- 172.21.15.55:0/2652286772 learned_addr learned my addr 172.21.15.55:0/2652286772 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:45.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.530+0000 7f0a4db9c700 1 -- 172.21.15.55:0/2652286772 >> v2:172.21.15.163:3300/0 conn(0x7f0a480ad870 msgr2=0x7f0a4806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:45.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.530+0000 7f0a4cb9a700 1 --2- 172.21.15.55:0/2652286772 >> v2:172.21.15.163:3300/0 conn(0x7f0a480ad870 0x7f0a4806b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:45.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.530+0000 7f0a4db9c700 1 --2- 172.21.15.55:0/2652286772 >> v2:172.21.15.163:3300/0 conn(0x7f0a480ad870 0x7f0a4806b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.530+0000 7f0a4db9c700 1 -- 172.21.15.55:0/2652286772 >> v1:172.21.15.163:6789/0 conn(0x7f0a48142ea0 legacy=0x7f0a480b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:45.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.530+0000 7f0a4db9c700 1 -- 172.21.15.55:0/2652286772 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a48069920 con 0x7f0a4806bf20 2023-10-06T05:05:45.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.530+0000 7f0a4cb9a700 1 --2- 172.21.15.55:0/2652286772 >> v2:172.21.15.163:3300/0 conn(0x7f0a480ad870 0x7f0a4806b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:45.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.530+0000 7f0a4db9c700 1 --2- 172.21.15.55:0/2652286772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a4806bf20 0x7f0a4806a120 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f0a480b0e10 tx=0x7f0a40009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8053e6f0076fe08d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:45.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.530+0000 7f0a3ffff700 1 -- 172.21.15.55:0/2652286772 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4000e070 con 0x7f0a4806bf20 2023-10-06T05:05:45.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.530+0000 7f0a3ffff700 1 -- 172.21.15.55:0/2652286772 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4000ba10 con 0x7f0a4806bf20 2023-10-06T05:05:45.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.530+0000 7f0a3ffff700 1 -- 172.21.15.55:0/2652286772 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a400092e0 con 0x7f0a4806bf20 2023-10-06T05:05:45.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.531+0000 7f0a4e39d700 1 -- 172.21.15.55:0/2652286772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a4806bf20 msgr2=0x7f0a4806a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:45.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.531+0000 7f0a4e39d700 1 --2- 172.21.15.55:0/2652286772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a4806bf20 0x7f0a4806a120 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f0a480b0e10 tx=0x7f0a40009840 comp rx=0 tx=0).stop 2023-10-06T05:05:45.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.531+0000 7f0a4e39d700 1 -- 172.21.15.55:0/2652286772 shutdown_connections 2023-10-06T05:05:45.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.531+0000 7f0a4e39d700 1 --2- 172.21.15.55:0/2652286772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a4806bf20 0x7f0a4806a120 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.531+0000 7f0a4e39d700 1 --2- 172.21.15.55:0/2652286772 >> v2:172.21.15.163:3300/0 conn(0x7f0a480ad870 0x7f0a4806b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.531+0000 7f0a4e39d700 1 -- 172.21.15.55:0/2652286772 >> 172.21.15.55:0/2652286772 conn(0x7f0a480ce430 msgr2=0x7f0a480ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:45.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.531+0000 7f0a4e39d700 1 -- 172.21.15.55:0/2652286772 shutdown_connections 2023-10-06T05:05:45.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.531+0000 7f0a4e39d700 1 -- 172.21.15.55:0/2652286772 wait complete. 2023-10-06T05:05:45.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.532+0000 7f0a4e39d700 1 Processor -- start 2023-10-06T05:05:45.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.533+0000 7f0a4e39d700 1 -- start start 2023-10-06T05:05:45.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.533+0000 7f0a4e39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a4806bf20 0x7f0a4815cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:45.543 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.533+0000 7f0a4e39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a48142ea0 0x7f0a480c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4e39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a480c9500 0x7f0a481589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4e39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4815a020 con 0x7f0a4806bf20 2023-10-06T05:05:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4e39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a480c9a10 con 0x7f0a480c9500 2023-10-06T05:05:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4e39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a480c9cf0 con 0x7f0a48142ea0 2023-10-06T05:05:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4d39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a4806bf20 0x7f0a4815cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:45.544 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4d39b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a4806bf20 0x7f0a4815cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45476/0 (socket says 172.21.15.55:45476) 2023-10-06T05:05:45.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4d39b700 1 -- 172.21.15.55:0/3465843787 learned_addr learned my addr 172.21.15.55:0/3465843787 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:45.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4cb9a700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a48142ea0 0x7f0a480c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:45.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4db9c700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a480c9500 0x7f0a481589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:45.545 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4d39b700 1 -- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a48142ea0 msgr2=0x7f0a480c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:45.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4d39b700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a48142ea0 0x7f0a480c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4d39b700 1 -- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a480c9500 msgr2=0x7f0a481589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:45.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4d39b700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a480c9500 0x7f0a481589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4d39b700 1 -- 172.21.15.55:0/3465843787 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a40009ae0 con 0x7f0a4806bf20 2023-10-06T05:05:45.546 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.534+0000 7f0a4cb9a700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a48142ea0 0x7f0a480c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:45.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.535+0000 7f0a4d39b700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a4806bf20 0x7f0a4815cc00 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400b5e0 tx=0x7f0a44009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:45.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.535+0000 7f0a3d7fa700 1 -- 172.21.15.55:0/3465843787 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400f510 con 0x7f0a4806bf20 2023-10-06T05:05:45.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.535+0000 7f0a3d7fa700 1 -- 172.21.15.55:0/3465843787 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4400f670 con 0x7f0a4806bf20 2023-10-06T05:05:45.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.535+0000 7f0a3d7fa700 1 -- 172.21.15.55:0/3465843787 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400fa30 con 0x7f0a4806bf20 2023-10-06T05:05:45.547 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.535+0000 7f0a4e39d700 1 -- 172.21.15.55:0/3465843787 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a480c9fd0 con 0x7f0a4806bf20 2023-10-06T05:05:45.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.535+0000 7f0a4e39d700 1 -- 172.21.15.55:0/3465843787 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a481400e0 con 0x7f0a4806bf20 2023-10-06T05:05:45.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.536+0000 7f0a4e39d700 1 -- 172.21.15.55:0/3465843787 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4815a4b0 con 0x7f0a4806bf20 2023-10-06T05:05:45.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.537+0000 7f0a3d7fa700 1 -- 172.21.15.55:0/3465843787 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a4401a400 con 0x7f0a4806bf20 2023-10-06T05:05:45.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.538+0000 7f0a3d7fa700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0a2c066f30 0x7f0a2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:45.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.538+0000 7f0a3d7fa700 1 -- 172.21.15.55:0/3465843787 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f0a44084370 con 0x7f0a4806bf20 2023-10-06T05:05:45.548 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.538+0000 7f0a4cb9a700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0a2c066f30 0x7f0a2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:45.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.539+0000 7f0a4cb9a700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0a2c066f30 0x7f0a2c0693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a480cb590 tx=0x7f0a38008040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:45.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.542+0000 7f0a3d7fa700 1 -- 172.21.15.55:0/3465843787 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a4815a4b0 con 0x7f0a4806bf20 2023-10-06T05:05:45.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.738+0000 7f0a4e39d700 1 -- 172.21.15.55:0/3465843787 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a4815a4b0 con 0x7f0a2c066f30 2023-10-06T05:05:45.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.747+0000 7f0a3d7fa700 1 -- 172.21.15.55:0/3465843787 <== mgr.7946 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311812 (secure 0 0 0) 0x7f0a4815a4b0 con 0x7f0a2c066f30 2023-10-06T05:05:45.753 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:45.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.752+0000 7f0a4e39d700 1 -- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0a2c066f30 msgr2=0x7f0a2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:45.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.752+0000 7f0a4e39d700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0a2c066f30 0x7f0a2c0693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a480cb590 tx=0x7f0a38008040 comp rx=0 tx=0).stop 2023-10-06T05:05:45.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.752+0000 7f0a4e39d700 1 -- 172.21.15.55:0/3465843787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a4806bf20 msgr2=0x7f0a4815cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:45.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.752+0000 7f0a4e39d700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a4806bf20 0x7f0a4815cc00 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400b5e0 tx=0x7f0a44009120 comp rx=0 tx=0).stop 2023-10-06T05:05:45.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.752+0000 7f0a4e39d700 1 -- 172.21.15.55:0/3465843787 shutdown_connections 2023-10-06T05:05:45.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.752+0000 7f0a4e39d700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0a2c066f30 0x7f0a2c0693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.752+0000 7f0a4e39d700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a480c9500 0x7f0a481589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.752+0000 7f0a4e39d700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a4806bf20 0x7f0a4815cc00 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.752+0000 7f0a4e39d700 1 --2- 172.21.15.55:0/3465843787 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a48142ea0 0x7f0a480c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:45.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.752+0000 7f0a4e39d700 1 -- 172.21.15.55:0/3465843787 >> 172.21.15.55:0/3465843787 conn(0x7f0a480ce430 msgr2=0x7f0a480b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:45.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.752+0000 7f0a4e39d700 1 -- 172.21.15.55:0/3465843787 shutdown_connections 2023-10-06T05:05:45.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:45.752+0000 7f0a4e39d700 1 -- 172.21.15.55:0/3465843787 wait complete. 2023-10-06T05:05:45.757 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T05:05:45.771 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-06T05:05:44.023870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":583,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":364,"ondisk_log_size":364,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2472088,"kb_used_data":20056,"kb_used_omap":0,"kb_used_meta":2451968,"kb_avail":752502632,"statfs":{"total":773094113280,"available":770562695168,"internally_reserved":0,"allocated":20537344,"data_stored":11381292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2510815232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.125490"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.919707+0000","last_change":"2023-10-06T05:05:40.916091+0000","last_active":"2023-10-06T05:05:40.919707+0000","last_peered":"2023-10-06T05:05:40.919707+0000","last_clean":"2023-10-06T05:05:40.919707+0000","last_became_active":"2023-10-06T05:05:40.915665+0000","last_became_peered":"2023-10-06T05:05:40.915665+0000","last_unstale":"2023-10-06T05:05:40.919707+0000","last_undegraded":"2023-10-06T05:05:40.919707+0000","last_fullsized":"2023-10-06T05:05:40.919707+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:44.300844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.931643+0000","last_change":"2023-10-06T05:05:40.926850+0000","last_active":"2023-10-06T05:05:40.931643+0000","last_peered":"2023-10-06T05:05:40.931643+0000","last_clean":"2023-10-06T05:05:40.931643+0000","last_became_active":"2023-10-06T05:05:40.926419+0000","last_became_peered":"2023-10-06T05:05:40.926419+0000","last_unstale":"2023-10-06T05:05:40.931643+0000","last_undegraded":"2023-10-06T05:05:40.931643+0000","last_fullsized":"2023-10-06T05:05:40.931643+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:19.765709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.933709+0000","last_change":"2023-10-06T05:05:40.930290+0000","last_active":"2023-10-06T05:05:40.933709+0000","last_peered":"2023-10-06T05:05:40.933709+0000","last_clean":"2023-10-06T05:05:40.933709+0000","last_became_active":"2023-10-06T05:05:40.930001+0000","last_became_peered":"2023-10-06T05:05:40.930001+0000","last_unstale":"2023-10-06T05:05:40.933709+0000","last_undegraded":"2023-10-06T05:05:40.933709+0000","last_fullsized":"2023-10-06T05:05:40.933709+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:02.396017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.935251+0000","last_change":"2023-10-06T05:05:40.931283+0000","last_active":"2023-10-06T05:05:40.935251+0000","last_peered":"2023-10-06T05:05:40.935251+0000","last_clean":"2023-10-06T05:05:40.935251+0000","last_became_active":"2023-10-06T05:05:40.930980+0000","last_became_peered":"2023-10-06T05:05:40.930980+0000","last_unstale":"2023-10-06T05:05:40.935251+0000","last_undegraded":"2023-10-06T05:05:40.935251+0000","last_fullsized":"2023-10-06T05:05:40.935251+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:28.839013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.925383+0000","last_change":"2023-10-06T05:05:40.923538+0000","last_active":"2023-10-06T05:05:40.925383+0000","last_peered":"2023-10-06T05:05:40.925383+0000","last_clean":"2023-10-06T05:05:40.925383+0000","last_became_active":"2023-10-06T05:05:40.923282+0000","last_became_peered":"2023-10-06T05:05:40.923282+0000","last_unstale":"2023-10-06T05:05:40.925383+0000","last_undegraded":"2023-10-06T05:05:40.925383+0000","last_fullsized":"2023-10-06T05:05:40.925383+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:10.104501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.932195+0000","last_change":"2023-10-06T05:05:40.926888+0000","last_active":"2023-10-06T05:05:40.932195+0000","last_peered":"2023-10-06T05:05:40.932195+0000","last_clean":"2023-10-06T05:05:40.932195+0000","last_became_active":"2023-10-06T05:05:40.926585+0000","last_became_peered":"2023-10-06T05:05:40.926585+0000","last_unstale":"2023-10-06T05:05:40.932195+0000","last_undegraded":"2023-10-06T05:05:40.932195+0000","last_fullsized":"2023-10-06T05:05:40.932195+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:44.300844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.947042+0000","last_change":"2023-10-06T05:05:40.939478+0000","last_active":"2023-10-06T05:05:40.947042+0000","last_peered":"2023-10-06T05:05:40.947042+0000","last_clean":"2023-10-06T05:05:40.947042+0000","last_became_active":"2023-10-06T05:05:40.939179+0000","last_became_peered":"2023-10-06T05:05:40.939179+0000","last_unstale":"2023-10-06T05:05:40.947042+0000","last_undegraded":"2023-10-06T05:05:40.947042+0000","last_fullsized":"2023-10-06T05:05:40.947042+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:44.501528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.945326+0000","last_change":"2023-10-06T05:05:40.938714+0000","last_active":"2023-10-06T05:05:40.945326+0000","last_peered":"2023-10-06T05:05:40.945326+0000","last_clean":"2023-10-06T05:05:40.945326+0000","last_became_active":"2023-10-06T05:05:40.936963+0000","last_became_peered":"2023-10-06T05:05:40.936963+0000","last_unstale":"2023-10-06T05:05:40.945326+0000","last_undegraded":"2023-10-06T05:05:40.945326+0000","last_fullsized":"2023-10-06T05:05:40.945326+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:53.688046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950412+0000","last_change":"2023-10-06T05:05:40.944935+0000","last_active":"2023-10-06T05:05:40.950412+0000","last_peered":"2023-10-06T05:05:40.950412+0000","last_clean":"2023-10-06T05:05:40.950412+0000","last_became_active":"2023-10-06T05:05:40.944657+0000","last_became_peered":"2023-10-06T05:05:40.944657+0000","last_unstale":"2023-10-06T05:05:40.950412+0000","last_undegraded":"2023-10-06T05:05:40.950412+0000","last_fullsized":"2023-10-06T05:05:40.950412+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:09.021499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950779+0000","last_change":"2023-10-06T05:05:40.946638+0000","last_active":"2023-10-06T05:05:40.950779+0000","last_peered":"2023-10-06T05:05:40.950779+0000","last_clean":"2023-10-06T05:05:40.950779+0000","last_became_active":"2023-10-06T05:05:40.946390+0000","last_became_peered":"2023-10-06T05:05:40.946390+0000","last_unstale":"2023-10-06T05:05:40.950779+0000","last_undegraded":"2023-10-06T05:05:40.950779+0000","last_fullsized":"2023-10-06T05:05:40.950779+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:18.198649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.929570+0000","last_change":"2023-10-06T05:05:40.927717+0000","last_active":"2023-10-06T05:05:40.929570+0000","last_peered":"2023-10-06T05:05:40.929570+0000","last_clean":"2023-10-06T05:05:40.929570+0000","last_became_active":"2023-10-06T05:05:40.927371+0000","last_became_peered":"2023-10-06T05:05:40.927371+0000","last_unstale":"2023-10-06T05:05:40.929570+0000","last_undegraded":"2023-10-06T05:05:40.929570+0000","last_fullsized":"2023-10-06T05:05:40.929570+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:09.021499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.935862+0000","last_change":"2023-10-06T05:05:40.934420+0000","last_active":"2023-10-06T05:05:40.935862+0000","last_peered":"2023-10-06T05:05:40.935862+0000","last_clean":"2023-10-06T05:05:40.935862+0000","last_became_active":"2023-10-06T05:05:40.934140+0000","last_became_peered":"2023-10-06T05:05:40.934140+0000","last_unstale":"2023-10-06T05:05:40.935862+0000","last_undegraded":"2023-10-06T05:05:40.935862+0000","last_fullsized":"2023-10-06T05:05:40.935862+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:21.918262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.940048+0000","last_change":"2023-10-06T05:05:40.937744+0000","last_active":"2023-10-06T05:05:40.940048+0000","last_peered":"2023-10-06T05:05:40.940048+0000","last_clean":"2023-10-06T05:05:40.940048+0000","last_became_active":"2023-10-06T05:05:40.937359+0000","last_became_peered":"2023-10-06T05:05:40.937359+0000","last_unstale":"2023-10-06T05:05:40.940048+0000","last_undegraded":"2023-10-06T05:05:40.940048+0000","last_fullsized":"2023-10-06T05:05:40.940048+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:56:57.524605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.943926+0000","last_change":"2023-10-06T05:05:40.932645+0000","last_active":"2023-10-06T05:05:40.943926+0000","last_peered":"2023-10-06T05:05:40.943926+0000","last_clean":"2023-10-06T05:05:40.943926+0000","last_became_active":"2023-10-06T05:05:40.932034+0000","last_became_peered":"2023-10-06T05:05:40.932034+0000","last_unstale":"2023-10-06T05:05:40.943926+0000","last_undegraded":"2023-10-06T05:05:40.943926+0000","last_fullsized":"2023-10-06T05:05:40.943926+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:28.839013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.942396+0000","last_change":"2023-10-06T05:05:40.941300+0000","last_active":"2023-10-06T05:05:40.942396+0000","last_peered":"2023-10-06T05:05:40.942396+0000","last_clean":"2023-10-06T05:05:40.942396+0000","last_became_active":"2023-10-06T05:05:40.940987+0000","last_became_peered":"2023-10-06T05:05:40.940987+0000","last_unstale":"2023-10-06T05:05:40.942396+0000","last_undegraded":"2023-10-06T05:05:40.942396+0000","last_fullsized":"2023-10-06T05:05:40.942396+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:46.429634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.924229+0000","last_change":"2023-10-06T05:05:40.922202+0000","last_active":"2023-10-06T05:05:40.924229+0000","last_peered":"2023-10-06T05:05:40.924229+0000","last_clean":"2023-10-06T05:05:40.924229+0000","last_became_active":"2023-10-06T05:05:40.921936+0000","last_became_peered":"2023-10-06T05:05:40.921936+0000","last_unstale":"2023-10-06T05:05:40.924229+0000","last_undegraded":"2023-10-06T05:05:40.924229+0000","last_fullsized":"2023-10-06T05:05:40.924229+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:21.918262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950518+0000","last_change":"2023-10-06T05:05:40.944935+0000","last_active":"2023-10-06T05:05:40.950518+0000","last_peered":"2023-10-06T05:05:40.950518+0000","last_clean":"2023-10-06T05:05:40.950518+0000","last_became_active":"2023-10-06T05:05:40.940363+0000","last_became_peered":"2023-10-06T05:05:40.940363+0000","last_unstale":"2023-10-06T05:05:40.950518+0000","last_undegraded":"2023-10-06T05:05:40.950518+0000","last_fullsized":"2023-10-06T05:05:40.950518+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:11.565930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946556+0000","last_change":"2023-10-06T05:05:40.945791+0000","last_active":"2023-10-06T05:05:40.946556+0000","last_peered":"2023-10-06T05:05:40.946556+0000","last_clean":"2023-10-06T05:05:40.946556+0000","last_became_active":"2023-10-06T05:05:40.945506+0000","last_became_peered":"2023-10-06T05:05:40.945506+0000","last_unstale":"2023-10-06T05:05:40.946556+0000","last_undegraded":"2023-10-06T05:05:40.946556+0000","last_fullsized":"2023-10-06T05:05:40.946556+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:42:21.067657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.942400+0000","last_change":"2023-10-06T05:05:40.941160+0000","last_active":"2023-10-06T05:05:40.942400+0000","last_peered":"2023-10-06T05:05:40.942400+0000","last_clean":"2023-10-06T05:05:40.942400+0000","last_became_active":"2023-10-06T05:05:40.940858+0000","last_became_peered":"2023-10-06T05:05:40.940858+0000","last_unstale":"2023-10-06T05:05:40.942400+0000","last_undegraded":"2023-10-06T05:05:40.942400+0000","last_fullsized":"2023-10-06T05:05:40.942400+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:22.225371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.933540+0000","last_change":"2023-10-06T05:05:40.929792+0000","last_active":"2023-10-06T05:05:40.933540+0000","last_peered":"2023-10-06T05:05:40.933540+0000","last_clean":"2023-10-06T05:05:40.933540+0000","last_became_active":"2023-10-06T05:05:40.929348+0000","last_became_peered":"2023-10-06T05:05:40.929348+0000","last_unstale":"2023-10-06T05:05:40.933540+0000","last_undegraded":"2023-10-06T05:05:40.933540+0000","last_fullsized":"2023-10-06T05:05:40.933540+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:35:02.907910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.926528+0000","last_change":"2023-10-06T05:05:40.922805+0000","last_active":"2023-10-06T05:05:40.926528+0000","last_peered":"2023-10-06T05:05:40.926528+0000","last_clean":"2023-10-06T05:05:40.926528+0000","last_became_active":"2023-10-06T05:05:40.922405+0000","last_became_peered":"2023-10-06T05:05:40.922405+0000","last_unstale":"2023-10-06T05:05:40.926528+0000","last_undegraded":"2023-10-06T05:05:40.926528+0000","last_fullsized":"2023-10-06T05:05:40.926528+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:17.058870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.931883+0000","last_change":"2023-10-06T05:05:40.926329+0000","last_active":"2023-10-06T05:05:40.931883+0000","last_peered":"2023-10-06T05:05:40.931883+0000","last_clean":"2023-10-06T05:05:40.931883+0000","last_became_active":"2023-10-06T05:05:40.926026+0000","last_became_peered":"2023-10-06T05:05:40.926026+0000","last_unstale":"2023-10-06T05:05:40.931883+0000","last_undegraded":"2023-10-06T05:05:40.931883+0000","last_fullsized":"2023-10-06T05:05:40.931883+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:37:17.058870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.917799+0000","last_change":"2023-10-06T05:05:40.915087+0000","last_active":"2023-10-06T05:05:40.917799+0000","last_peered":"2023-10-06T05:05:40.917799+0000","last_clean":"2023-10-06T05:05:40.917799+0000","last_became_active":"2023-10-06T05:05:40.914807+0000","last_became_peered":"2023-10-06T05:05:40.914807+0000","last_unstale":"2023-10-06T05:05:40.917799+0000","last_undegraded":"2023-10-06T05:05:40.917799+0000","last_fullsized":"2023-10-06T05:05:40.917799+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:44.300844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.923541+0000","last_change":"2023-10-06T05:05:40.922026+0000","last_active":"2023-10-06T05:05:40.923541+0000","last_peered":"2023-10-06T05:05:40.923541+0000","last_clean":"2023-10-06T05:05:40.923541+0000","last_became_active":"2023-10-06T05:05:40.921531+0000","last_became_peered":"2023-10-06T05:05:40.921531+0000","last_unstale":"2023-10-06T05:05:40.923541+0000","last_undegraded":"2023-10-06T05:05:40.923541+0000","last_fullsized":"2023-10-06T05:05:40.923541+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:48.689929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.935321+0000","last_change":"2023-10-06T05:05:40.927795+0000","last_active":"2023-10-06T05:05:40.935321+0000","last_peered":"2023-10-06T05:05:40.935321+0000","last_clean":"2023-10-06T05:05:40.935321+0000","last_became_active":"2023-10-06T05:05:40.927347+0000","last_became_peered":"2023-10-06T05:05:40.927347+0000","last_unstale":"2023-10-06T05:05:40.935321+0000","last_undegraded":"2023-10-06T05:05:40.935321+0000","last_fullsized":"2023-10-06T05:05:40.935321+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:34:49.506769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.942408+0000","last_change":"2023-10-06T05:05:40.938367+0000","last_active":"2023-10-06T05:05:40.942408+0000","last_peered":"2023-10-06T05:05:40.942408+0000","last_clean":"2023-10-06T05:05:40.942408+0000","last_became_active":"2023-10-06T05:05:40.938143+0000","last_became_peered":"2023-10-06T05:05:40.938143+0000","last_unstale":"2023-10-06T05:05:40.942408+0000","last_undegraded":"2023-10-06T05:05:40.942408+0000","last_fullsized":"2023-10-06T05:05:40.942408+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:10:46.577112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946345+0000","last_change":"2023-10-06T05:05:40.943345+0000","last_active":"2023-10-06T05:05:40.946345+0000","last_peered":"2023-10-06T05:05:40.946345+0000","last_clean":"2023-10-06T05:05:40.946345+0000","last_became_active":"2023-10-06T05:05:40.943071+0000","last_became_peered":"2023-10-06T05:05:40.943071+0000","last_unstale":"2023-10-06T05:05:40.946345+0000","last_undegraded":"2023-10-06T05:05:40.946345+0000","last_fullsized":"2023-10-06T05:05:40.946345+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:42:50.110698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.923559+0000","last_change":"2023-10-06T05:05:40.921659+0000","last_active":"2023-10-06T05:05:40.923559+0000","last_peered":"2023-10-06T05:05:40.923559+0000","last_clean":"2023-10-06T05:05:40.923559+0000","last_became_active":"2023-10-06T05:05:40.921381+0000","last_became_peered":"2023-10-06T05:05:40.921381+0000","last_unstale":"2023-10-06T05:05:40.923559+0000","last_undegraded":"2023-10-06T05:05:40.923559+0000","last_fullsized":"2023-10-06T05:05:40.923559+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:47.808228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946914+0000","last_change":"2023-10-06T05:05:40.946429+0000","last_active":"2023-10-06T05:05:40.946914+0000","last_peered":"2023-10-06T05:05:40.946914+0000","last_clean":"2023-10-06T05:05:40.946914+0000","last_became_active":"2023-10-06T05:05:40.946095+0000","last_became_peered":"2023-10-06T05:05:40.946095+0000","last_unstale":"2023-10-06T05:05:40.946914+0000","last_undegraded":"2023-10-06T05:05:40.946914+0000","last_fullsized":"2023-10-06T05:05:40.946914+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:17:02.396017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.926500+0000","last_change":"2023-10-06T05:05:40.923308+0000","last_active":"2023-10-06T05:05:40.926500+0000","last_peered":"2023-10-06T05:05:40.926500+0000","last_clean":"2023-10-06T05:05:40.926500+0000","last_became_active":"2023-10-06T05:05:40.922449+0000","last_became_peered":"2023-10-06T05:05:40.922449+0000","last_unstale":"2023-10-06T05:05:40.926500+0000","last_undegraded":"2023-10-06T05:05:40.926500+0000","last_fullsized":"2023-10-06T05:05:40.926500+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:03:18.924378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.926173+0000","last_change":"2023-10-06T05:05:40.922469+0000","last_active":"2023-10-06T05:05:40.926173+0000","last_peered":"2023-10-06T05:05:40.926173+0000","last_clean":"2023-10-06T05:05:40.926173+0000","last_became_active":"2023-10-06T05:05:40.922155+0000","last_became_peered":"2023-10-06T05:05:40.922155+0000","last_unstale":"2023-10-06T05:05:40.926173+0000","last_undegraded":"2023-10-06T05:05:40.926173+0000","last_fullsized":"2023-10-06T05:05:40.926173+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:11:42.339342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.934664+0000","last_change":"2023-10-06T05:05:40.933192+0000","last_active":"2023-10-06T05:05:40.934664+0000","last_peered":"2023-10-06T05:05:40.934664+0000","last_clean":"2023-10-06T05:05:40.934664+0000","last_became_active":"2023-10-06T05:05:40.932740+0000","last_became_peered":"2023-10-06T05:05:40.932740+0000","last_unstale":"2023-10-06T05:05:40.934664+0000","last_undegraded":"2023-10-06T05:05:40.934664+0000","last_fullsized":"2023-10-06T05:05:40.934664+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:57:04.491084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.943829+0000","last_change":"2023-10-06T05:05:40.942695+0000","last_active":"2023-10-06T05:05:40.943829+0000","last_peered":"2023-10-06T05:05:40.943829+0000","last_clean":"2023-10-06T05:05:40.943829+0000","last_became_active":"2023-10-06T05:05:40.942183+0000","last_became_peered":"2023-10-06T05:05:40.942183+0000","last_unstale":"2023-10-06T05:05:40.943829+0000","last_undegraded":"2023-10-06T05:05:40.943829+0000","last_fullsized":"2023-10-06T05:05:40.943829+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:10.104501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"192'4","reported_seq":14,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.943127+0000","last_change":"2023-10-06T05:05:39.925290+0000","last_active":"2023-10-06T05:05:40.943127+0000","last_peered":"2023-10-06T05:05:40.943127+0000","last_clean":"2023-10-06T05:05:40.943127+0000","last_became_active":"2023-10-06T05:05:39.924872+0000","last_became_peered":"2023-10-06T05:05:39.924872+0000","last_unstale":"2023-10-06T05:05:40.943127+0000","last_undegraded":"2023-10-06T05:05:40.943127+0000","last_fullsized":"2023-10-06T05:05:40.943127+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:46.798368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.922648+0000","last_change":"2023-10-06T05:05:40.919939+0000","last_active":"2023-10-06T05:05:40.922648+0000","last_peered":"2023-10-06T05:05:40.922648+0000","last_clean":"2023-10-06T05:05:40.922648+0000","last_became_active":"2023-10-06T05:05:40.919646+0000","last_became_peered":"2023-10-06T05:05:40.919646+0000","last_unstale":"2023-10-06T05:05:40.922648+0000","last_undegraded":"2023-10-06T05:05:40.922648+0000","last_fullsized":"2023-10-06T05:05:40.922648+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:18:14.653476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950992+0000","last_change":"2023-10-06T05:05:39.918565+0000","last_active":"2023-10-06T05:05:40.950992+0000","last_peered":"2023-10-06T05:05:40.950992+0000","last_clean":"2023-10-06T05:05:40.950992+0000","last_became_active":"2023-10-06T05:05:39.918281+0000","last_became_peered":"2023-10-06T05:05:39.918281+0000","last_unstale":"2023-10-06T05:05:40.950992+0000","last_undegraded":"2023-10-06T05:05:40.950992+0000","last_fullsized":"2023-10-06T05:05:40.950992+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:17.188789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.943946+0000","last_change":"2023-10-06T05:05:40.932809+0000","last_active":"2023-10-06T05:05:40.943946+0000","last_peered":"2023-10-06T05:05:40.943946+0000","last_clean":"2023-10-06T05:05:40.943946+0000","last_became_active":"2023-10-06T05:05:40.932380+0000","last_became_peered":"2023-10-06T05:05:40.932380+0000","last_unstale":"2023-10-06T05:05:40.943946+0000","last_undegraded":"2023-10-06T05:05:40.943946+0000","last_fullsized":"2023-10-06T05:05:40.943946+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:50:48.689929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.943613+0000","last_change":"2023-10-06T05:05:39.925369+0000","last_active":"2023-10-06T05:05:40.943613+0000","last_peered":"2023-10-06T05:05:40.943613+0000","last_clean":"2023-10-06T05:05:40.943613+0000","last_became_active":"2023-10-06T05:05:39.923479+0000","last_became_peered":"2023-10-06T05:05:39.923479+0000","last_unstale":"2023-10-06T05:05:40.943613+0000","last_undegraded":"2023-10-06T05:05:40.943613+0000","last_fullsized":"2023-10-06T05:05:40.943613+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:41.289552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.943792+0000","last_change":"2023-10-06T05:05:40.942581+0000","last_active":"2023-10-06T05:05:40.943792+0000","last_peered":"2023-10-06T05:05:40.943792+0000","last_clean":"2023-10-06T05:05:40.943792+0000","last_became_active":"2023-10-06T05:05:40.942285+0000","last_became_peered":"2023-10-06T05:05:40.942285+0000","last_unstale":"2023-10-06T05:05:40.943792+0000","last_undegraded":"2023-10-06T05:05:40.943792+0000","last_fullsized":"2023-10-06T05:05:40.943792+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:28.839013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.941293+0000","last_change":"2023-10-06T05:05:39.926002+0000","last_active":"2023-10-06T05:05:40.941293+0000","last_peered":"2023-10-06T05:05:40.941293+0000","last_clean":"2023-10-06T05:05:40.941293+0000","last_became_active":"2023-10-06T05:05:39.925738+0000","last_became_peered":"2023-10-06T05:05:39.925738+0000","last_unstale":"2023-10-06T05:05:40.941293+0000","last_undegraded":"2023-10-06T05:05:40.941293+0000","last_fullsized":"2023-10-06T05:05:40.941293+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:46:38.682187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950753+0000","last_change":"2023-10-06T05:05:40.949012+0000","last_active":"2023-10-06T05:05:40.950753+0000","last_peered":"2023-10-06T05:05:40.950753+0000","last_clean":"2023-10-06T05:05:40.950753+0000","last_became_active":"2023-10-06T05:05:40.946073+0000","last_became_peered":"2023-10-06T05:05:40.946073+0000","last_unstale":"2023-10-06T05:05:40.950753+0000","last_undegraded":"2023-10-06T05:05:40.950753+0000","last_fullsized":"2023-10-06T05:05:40.950753+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:17.899816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.948623+0000","last_change":"2023-10-06T05:05:39.923467+0000","last_active":"2023-10-06T05:05:40.948623+0000","last_peered":"2023-10-06T05:05:40.948623+0000","last_clean":"2023-10-06T05:05:40.948623+0000","last_became_active":"2023-10-06T05:05:39.923175+0000","last_became_peered":"2023-10-06T05:05:39.923175+0000","last_unstale":"2023-10-06T05:05:40.948623+0000","last_undegraded":"2023-10-06T05:05:40.948623+0000","last_fullsized":"2023-10-06T05:05:40.948623+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:29:58.965000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.942490+0000","last_change":"2023-10-06T05:05:40.939550+0000","last_active":"2023-10-06T05:05:40.942490+0000","last_peered":"2023-10-06T05:05:40.942490+0000","last_clean":"2023-10-06T05:05:40.942490+0000","last_became_active":"2023-10-06T05:05:40.939327+0000","last_became_peered":"2023-10-06T05:05:40.939327+0000","last_unstale":"2023-10-06T05:05:40.942490+0000","last_undegraded":"2023-10-06T05:05:40.942490+0000","last_fullsized":"2023-10-06T05:05:40.942490+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:23:48.324228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946700+0000","last_change":"2023-10-06T05:05:39.926962+0000","last_active":"2023-10-06T05:05:40.946700+0000","last_peered":"2023-10-06T05:05:40.946700+0000","last_clean":"2023-10-06T05:05:40.946700+0000","last_became_active":"2023-10-06T05:05:39.926698+0000","last_became_peered":"2023-10-06T05:05:39.926698+0000","last_unstale":"2023-10-06T05:05:40.946700+0000","last_undegraded":"2023-10-06T05:05:40.946700+0000","last_fullsized":"2023-10-06T05:05:40.946700+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:43:43.491668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.933580+0000","last_change":"2023-10-06T05:05:40.928951+0000","last_active":"2023-10-06T05:05:40.933580+0000","last_peered":"2023-10-06T05:05:40.933580+0000","last_clean":"2023-10-06T05:05:40.933580+0000","last_became_active":"2023-10-06T05:05:40.928695+0000","last_became_peered":"2023-10-06T05:05:40.928695+0000","last_unstale":"2023-10-06T05:05:40.933580+0000","last_undegraded":"2023-10-06T05:05:40.933580+0000","last_fullsized":"2023-10-06T05:05:40.933580+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:59:03.406905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.920630+0000","last_change":"2023-10-06T05:05:39.926602+0000","last_active":"2023-10-06T05:05:40.920630+0000","last_peered":"2023-10-06T05:05:40.920630+0000","last_clean":"2023-10-06T05:05:40.920630+0000","last_became_active":"2023-10-06T05:05:39.926304+0000","last_became_peered":"2023-10-06T05:05:39.926304+0000","last_unstale":"2023-10-06T05:05:40.920630+0000","last_undegraded":"2023-10-06T05:05:40.920630+0000","last_fullsized":"2023-10-06T05:05:40.920630+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:15:34.314159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.925483+0000","last_change":"2023-10-06T05:05:40.923945+0000","last_active":"2023-10-06T05:05:40.925483+0000","last_peered":"2023-10-06T05:05:40.925483+0000","last_clean":"2023-10-06T05:05:40.925483+0000","last_became_active":"2023-10-06T05:05:40.923610+0000","last_became_peered":"2023-10-06T05:05:40.923610+0000","last_unstale":"2023-10-06T05:05:40.925483+0000","last_undegraded":"2023-10-06T05:05:40.925483+0000","last_fullsized":"2023-10-06T05:05:40.925483+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:10.104501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.918673+0000","last_change":"2023-10-06T05:05:39.928895+0000","last_active":"2023-10-06T05:05:40.918673+0000","last_peered":"2023-10-06T05:05:40.918673+0000","last_clean":"2023-10-06T05:05:40.918673+0000","last_became_active":"2023-10-06T05:05:39.928643+0000","last_became_peered":"2023-10-06T05:05:39.928643+0000","last_unstale":"2023-10-06T05:05:40.918673+0000","last_undegraded":"2023-10-06T05:05:40.918673+0000","last_fullsized":"2023-10-06T05:05:40.918673+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:49.012909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.945303+0000","last_change":"2023-10-06T05:05:40.937216+0000","last_active":"2023-10-06T05:05:40.945303+0000","last_peered":"2023-10-06T05:05:40.945303+0000","last_clean":"2023-10-06T05:05:40.945303+0000","last_became_active":"2023-10-06T05:05:40.936666+0000","last_became_peered":"2023-10-06T05:05:40.936666+0000","last_unstale":"2023-10-06T05:05:40.945303+0000","last_undegraded":"2023-10-06T05:05:40.945303+0000","last_fullsized":"2023-10-06T05:05:40.945303+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:26.963559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.940832+0000","last_change":"2023-10-06T05:05:39.920059+0000","last_active":"2023-10-06T05:05:40.940832+0000","last_peered":"2023-10-06T05:05:40.940832+0000","last_clean":"2023-10-06T05:05:40.940832+0000","last_became_active":"2023-10-06T05:05:39.919769+0000","last_became_peered":"2023-10-06T05:05:39.919769+0000","last_unstale":"2023-10-06T05:05:40.940832+0000","last_undegraded":"2023-10-06T05:05:40.940832+0000","last_fullsized":"2023-10-06T05:05:40.940832+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:29:39.631553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.927801+0000","last_change":"2023-10-06T05:05:40.923956+0000","last_active":"2023-10-06T05:05:40.927801+0000","last_peered":"2023-10-06T05:05:40.927801+0000","last_clean":"2023-10-06T05:05:40.927801+0000","last_became_active":"2023-10-06T05:05:40.923640+0000","last_became_peered":"2023-10-06T05:05:40.923640+0000","last_unstale":"2023-10-06T05:05:40.927801+0000","last_undegraded":"2023-10-06T05:05:40.927801+0000","last_fullsized":"2023-10-06T05:05:40.927801+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:05:14.959671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950727+0000","last_change":"2023-10-06T05:05:39.925659+0000","last_active":"2023-10-06T05:05:40.950727+0000","last_peered":"2023-10-06T05:05:40.950727+0000","last_clean":"2023-10-06T05:05:40.950727+0000","last_became_active":"2023-10-06T05:05:39.925413+0000","last_became_peered":"2023-10-06T05:05:39.925413+0000","last_unstale":"2023-10-06T05:05:40.950727+0000","last_undegraded":"2023-10-06T05:05:40.950727+0000","last_fullsized":"2023-10-06T05:05:40.950727+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:24:41.289552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.925429+0000","last_change":"2023-10-06T05:05:40.921842+0000","last_active":"2023-10-06T05:05:40.925429+0000","last_peered":"2023-10-06T05:05:40.925429+0000","last_clean":"2023-10-06T05:05:40.925429+0000","last_became_active":"2023-10-06T05:05:40.921579+0000","last_became_peered":"2023-10-06T05:05:40.921579+0000","last_unstale":"2023-10-06T05:05:40.925429+0000","last_undegraded":"2023-10-06T05:05:40.925429+0000","last_fullsized":"2023-10-06T05:05:40.925429+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:39:36.500257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"192'1","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946487+0000","last_change":"2023-10-06T05:05:39.920441+0000","last_active":"2023-10-06T05:05:40.946487+0000","last_peered":"2023-10-06T05:05:40.946487+0000","last_clean":"2023-10-06T05:05:40.946487+0000","last_became_active":"2023-10-06T05:05:39.920172+0000","last_became_peered":"2023-10-06T05:05:39.920172+0000","last_unstale":"2023-10-06T05:05:40.946487+0000","last_undegraded":"2023-10-06T05:05:40.946487+0000","last_fullsized":"2023-10-06T05:05:40.946487+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:59:46.798368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.929553+0000","last_change":"2023-10-06T05:05:40.927217+0000","last_active":"2023-10-06T05:05:40.929553+0000","last_peered":"2023-10-06T05:05:40.929553+0000","last_clean":"2023-10-06T05:05:40.929553+0000","last_became_active":"2023-10-06T05:05:40.926817+0000","last_became_peered":"2023-10-06T05:05:40.926817+0000","last_unstale":"2023-10-06T05:05:40.929553+0000","last_undegraded":"2023-10-06T05:05:40.929553+0000","last_fullsized":"2023-10-06T05:05:40.929553+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:01.165703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.919218+0000","last_change":"2023-10-06T05:05:39.919987+0000","last_active":"2023-10-06T05:05:40.919218+0000","last_peered":"2023-10-06T05:05:40.919218+0000","last_clean":"2023-10-06T05:05:40.919218+0000","last_became_active":"2023-10-06T05:05:39.919692+0000","last_became_peered":"2023-10-06T05:05:39.919692+0000","last_unstale":"2023-10-06T05:05:40.919218+0000","last_undegraded":"2023-10-06T05:05:40.919218+0000","last_fullsized":"2023-10-06T05:05:40.919218+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:01.125242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.923540+0000","last_change":"2023-10-06T05:05:40.921248+0000","last_active":"2023-10-06T05:05:40.923540+0000","last_peered":"2023-10-06T05:05:40.923540+0000","last_clean":"2023-10-06T05:05:40.923540+0000","last_became_active":"2023-10-06T05:05:40.920924+0000","last_became_peered":"2023-10-06T05:05:40.920924+0000","last_unstale":"2023-10-06T05:05:40.923540+0000","last_undegraded":"2023-10-06T05:05:40.923540+0000","last_fullsized":"2023-10-06T05:05:40.923540+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:53.688046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946972+0000","last_change":"2023-10-06T05:05:39.928080+0000","last_active":"2023-10-06T05:05:40.946972+0000","last_peered":"2023-10-06T05:05:40.946972+0000","last_clean":"2023-10-06T05:05:40.946972+0000","last_became_active":"2023-10-06T05:05:39.927814+0000","last_became_peered":"2023-10-06T05:05:39.927814+0000","last_unstale":"2023-10-06T05:05:40.946972+0000","last_undegraded":"2023-10-06T05:05:40.946972+0000","last_fullsized":"2023-10-06T05:05:40.946972+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:52.678186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946542+0000","last_change":"2023-10-06T05:05:40.941982+0000","last_active":"2023-10-06T05:05:40.946542+0000","last_peered":"2023-10-06T05:05:40.946542+0000","last_clean":"2023-10-06T05:05:40.946542+0000","last_became_active":"2023-10-06T05:05:40.941660+0000","last_became_peered":"2023-10-06T05:05:40.941660+0000","last_unstale":"2023-10-06T05:05:40.946542+0000","last_undegraded":"2023-10-06T05:05:40.946542+0000","last_fullsized":"2023-10-06T05:05:40.946542+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:02:21.918262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.916446+0000","last_change":"2023-10-06T05:05:39.926300+0000","last_active":"2023-10-06T05:05:40.916446+0000","last_peered":"2023-10-06T05:05:40.916446+0000","last_clean":"2023-10-06T05:05:40.916446+0000","last_became_active":"2023-10-06T05:05:39.926009+0000","last_became_peered":"2023-10-06T05:05:39.926009+0000","last_unstale":"2023-10-06T05:05:40.916446+0000","last_undegraded":"2023-10-06T05:05:40.916446+0000","last_fullsized":"2023-10-06T05:05:40.916446+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:00:01.125242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950051+0000","last_change":"2023-10-06T05:05:40.947297+0000","last_active":"2023-10-06T05:05:40.950051+0000","last_peered":"2023-10-06T05:05:40.950051+0000","last_clean":"2023-10-06T05:05:40.950051+0000","last_became_active":"2023-10-06T05:05:40.947004+0000","last_became_peered":"2023-10-06T05:05:40.947004+0000","last_unstale":"2023-10-06T05:05:40.950051+0000","last_undegraded":"2023-10-06T05:05:40.950051+0000","last_fullsized":"2023-10-06T05:05:40.950051+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:19.716383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.921497+0000","last_change":"2023-10-06T05:05:39.923141+0000","last_active":"2023-10-06T05:05:40.921497+0000","last_peered":"2023-10-06T05:05:40.921497+0000","last_clean":"2023-10-06T05:05:40.921497+0000","last_became_active":"2023-10-06T05:05:39.922780+0000","last_became_peered":"2023-10-06T05:05:39.922780+0000","last_unstale":"2023-10-06T05:05:40.921497+0000","last_undegraded":"2023-10-06T05:05:40.921497+0000","last_fullsized":"2023-10-06T05:05:40.921497+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:42.468662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.933364+0000","last_change":"2023-10-06T05:05:40.932037+0000","last_active":"2023-10-06T05:05:40.933364+0000","last_peered":"2023-10-06T05:05:40.933364+0000","last_clean":"2023-10-06T05:05:40.933364+0000","last_became_active":"2023-10-06T05:05:40.930683+0000","last_became_peered":"2023-10-06T05:05:40.930683+0000","last_unstale":"2023-10-06T05:05:40.933364+0000","last_undegraded":"2023-10-06T05:05:40.933364+0000","last_fullsized":"2023-10-06T05:05:40.933364+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:52:19.716383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.920703+0000","last_change":"2023-10-06T05:05:39.928537+0000","last_active":"2023-10-06T05:05:40.920703+0000","last_peered":"2023-10-06T05:05:40.920703+0000","last_clean":"2023-10-06T05:05:40.920703+0000","last_became_active":"2023-10-06T05:05:39.928200+0000","last_became_peered":"2023-10-06T05:05:39.928200+0000","last_unstale":"2023-10-06T05:05:40.920703+0000","last_undegraded":"2023-10-06T05:05:40.920703+0000","last_fullsized":"2023-10-06T05:05:40.920703+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:05.926590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.926142+0000","last_change":"2023-10-06T05:05:40.921820+0000","last_active":"2023-10-06T05:05:40.926142+0000","last_peered":"2023-10-06T05:05:40.926142+0000","last_clean":"2023-10-06T05:05:40.926142+0000","last_became_active":"2023-10-06T05:05:40.921541+0000","last_became_peered":"2023-10-06T05:05:40.921541+0000","last_unstale":"2023-10-06T05:05:40.926142+0000","last_undegraded":"2023-10-06T05:05:40.926142+0000","last_fullsized":"2023-10-06T05:05:40.926142+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:39:11.565930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.931437+0000","last_change":"2023-10-06T05:05:39.922396+0000","last_active":"2023-10-06T05:05:40.931437+0000","last_peered":"2023-10-06T05:05:40.931437+0000","last_clean":"2023-10-06T05:05:40.931437+0000","last_became_active":"2023-10-06T05:05:39.922116+0000","last_became_peered":"2023-10-06T05:05:39.922116+0000","last_unstale":"2023-10-06T05:05:40.931437+0000","last_undegraded":"2023-10-06T05:05:40.931437+0000","last_fullsized":"2023-10-06T05:05:40.931437+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:04:25.953699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.947023+0000","last_change":"2023-10-06T05:05:40.938953+0000","last_active":"2023-10-06T05:05:40.947023+0000","last_peered":"2023-10-06T05:05:40.947023+0000","last_clean":"2023-10-06T05:05:40.947023+0000","last_became_active":"2023-10-06T05:05:40.938643+0000","last_became_peered":"2023-10-06T05:05:40.938643+0000","last_unstale":"2023-10-06T05:05:40.947023+0000","last_undegraded":"2023-10-06T05:05:40.947023+0000","last_fullsized":"2023-10-06T05:05:40.947023+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:19:46.429634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.916613+0000","last_change":"2023-10-06T05:05:39.929045+0000","last_active":"2023-10-06T05:05:40.916613+0000","last_peered":"2023-10-06T05:05:40.916613+0000","last_clean":"2023-10-06T05:05:40.916613+0000","last_became_active":"2023-10-06T05:05:39.928706+0000","last_became_peered":"2023-10-06T05:05:39.928706+0000","last_unstale":"2023-10-06T05:05:40.916613+0000","last_undegraded":"2023-10-06T05:05:40.916613+0000","last_fullsized":"2023-10-06T05:05:40.916613+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:49:29.272255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.931295+0000","last_change":"2023-10-06T05:05:40.928471+0000","last_active":"2023-10-06T05:05:40.931295+0000","last_peered":"2023-10-06T05:05:40.931295+0000","last_clean":"2023-10-06T05:05:40.931295+0000","last_became_active":"2023-10-06T05:05:40.928138+0000","last_became_peered":"2023-10-06T05:05:40.928138+0000","last_unstale":"2023-10-06T05:05:40.931295+0000","last_undegraded":"2023-10-06T05:05:40.931295+0000","last_fullsized":"2023-10-06T05:05:40.931295+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:58:21.333982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.915210+0000","last_change":"2023-10-06T05:05:39.930030+0000","last_active":"2023-10-06T05:05:40.915210+0000","last_peered":"2023-10-06T05:05:40.915210+0000","last_clean":"2023-10-06T05:05:40.915210+0000","last_became_active":"2023-10-06T05:05:39.929713+0000","last_became_peered":"2023-10-06T05:05:39.929713+0000","last_unstale":"2023-10-06T05:05:40.915210+0000","last_undegraded":"2023-10-06T05:05:40.915210+0000","last_fullsized":"2023-10-06T05:05:40.915210+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:35:00.155842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950501+0000","last_change":"2023-10-06T05:05:40.945260+0000","last_active":"2023-10-06T05:05:40.950501+0000","last_peered":"2023-10-06T05:05:40.950501+0000","last_clean":"2023-10-06T05:05:40.950501+0000","last_became_active":"2023-10-06T05:05:40.944947+0000","last_became_peered":"2023-10-06T05:05:40.944947+0000","last_unstale":"2023-10-06T05:05:40.950501+0000","last_undegraded":"2023-10-06T05:05:40.950501+0000","last_fullsized":"2023-10-06T05:05:40.950501+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:22.225371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.921397+0000","last_change":"2023-10-06T05:05:39.927197+0000","last_active":"2023-10-06T05:05:40.921397+0000","last_peered":"2023-10-06T05:05:40.921397+0000","last_clean":"2023-10-06T05:05:40.921397+0000","last_became_active":"2023-10-06T05:05:39.926927+0000","last_became_peered":"2023-10-06T05:05:39.926927+0000","last_unstale":"2023-10-06T05:05:40.921397+0000","last_undegraded":"2023-10-06T05:05:40.921397+0000","last_fullsized":"2023-10-06T05:05:40.921397+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:20:22.307584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.920707+0000","last_change":"2023-10-06T05:05:40.916765+0000","last_active":"2023-10-06T05:05:40.920707+0000","last_peered":"2023-10-06T05:05:40.920707+0000","last_clean":"2023-10-06T05:05:40.920707+0000","last_became_active":"2023-10-06T05:05:40.916428+0000","last_became_peered":"2023-10-06T05:05:40.916428+0000","last_unstale":"2023-10-06T05:05:40.920707+0000","last_undegraded":"2023-10-06T05:05:40.920707+0000","last_fullsized":"2023-10-06T05:05:40.920707+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:10.851897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.910908+0000","last_change":"2023-10-06T05:05:39.921721+0000","last_active":"2023-10-06T05:05:40.910908+0000","last_peered":"2023-10-06T05:05:40.910908+0000","last_clean":"2023-10-06T05:05:40.910908+0000","last_became_active":"2023-10-06T05:05:39.921430+0000","last_became_peered":"2023-10-06T05:05:39.921430+0000","last_unstale":"2023-10-06T05:05:40.910908+0000","last_undegraded":"2023-10-06T05:05:40.910908+0000","last_fullsized":"2023-10-06T05:05:40.910908+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:57.786548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.940595+0000","last_change":"2023-10-06T05:05:40.937499+0000","last_active":"2023-10-06T05:05:40.940595+0000","last_peered":"2023-10-06T05:05:40.940595+0000","last_clean":"2023-10-06T05:05:40.940595+0000","last_became_active":"2023-10-06T05:05:40.937206+0000","last_became_peered":"2023-10-06T05:05:40.937206+0000","last_unstale":"2023-10-06T05:05:40.940595+0000","last_undegraded":"2023-10-06T05:05:40.940595+0000","last_fullsized":"2023-10-06T05:05:40.940595+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:34:07.458157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.915086+0000","last_change":"2023-10-06T05:05:39.923515+0000","last_active":"2023-10-06T05:05:40.915086+0000","last_peered":"2023-10-06T05:05:40.915086+0000","last_clean":"2023-10-06T05:05:40.915086+0000","last_became_active":"2023-10-06T05:05:39.923082+0000","last_became_peered":"2023-10-06T05:05:39.923082+0000","last_unstale":"2023-10-06T05:05:40.915086+0000","last_undegraded":"2023-10-06T05:05:40.915086+0000","last_fullsized":"2023-10-06T05:05:40.915086+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:05:07.808357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.925608+0000","last_change":"2023-10-06T05:05:40.923428+0000","last_active":"2023-10-06T05:05:40.925608+0000","last_peered":"2023-10-06T05:05:40.925608+0000","last_clean":"2023-10-06T05:05:40.925608+0000","last_became_active":"2023-10-06T05:05:40.923167+0000","last_became_peered":"2023-10-06T05:05:40.923167+0000","last_unstale":"2023-10-06T05:05:40.925608+0000","last_undegraded":"2023-10-06T05:05:40.925608+0000","last_fullsized":"2023-10-06T05:05:40.925608+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:31:18.198649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.916967+0000","last_change":"2023-10-06T05:05:39.923687+0000","last_active":"2023-10-06T05:05:40.916967+0000","last_peered":"2023-10-06T05:05:40.916967+0000","last_clean":"2023-10-06T05:05:40.916967+0000","last_became_active":"2023-10-06T05:05:39.923378+0000","last_became_peered":"2023-10-06T05:05:39.923378+0000","last_unstale":"2023-10-06T05:05:40.916967+0000","last_undegraded":"2023-10-06T05:05:40.916967+0000","last_fullsized":"2023-10-06T05:05:40.916967+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:58:49.012909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.933837+0000","last_change":"2023-10-06T05:05:40.929228+0000","last_active":"2023-10-06T05:05:40.933837+0000","last_peered":"2023-10-06T05:05:40.933837+0000","last_clean":"2023-10-06T05:05:40.933837+0000","last_became_active":"2023-10-06T05:05:40.928352+0000","last_became_peered":"2023-10-06T05:05:40.928352+0000","last_unstale":"2023-10-06T05:05:40.933837+0000","last_undegraded":"2023-10-06T05:05:40.933837+0000","last_fullsized":"2023-10-06T05:05:40.933837+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:25:10.201272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.909941+0000","last_change":"2023-10-06T05:05:39.930235+0000","last_active":"2023-10-06T05:05:40.909941+0000","last_peered":"2023-10-06T05:05:40.909941+0000","last_clean":"2023-10-06T05:05:40.909941+0000","last_became_active":"2023-10-06T05:05:39.929890+0000","last_became_peered":"2023-10-06T05:05:39.929890+0000","last_unstale":"2023-10-06T05:05:40.909941+0000","last_undegraded":"2023-10-06T05:05:40.909941+0000","last_fullsized":"2023-10-06T05:05:40.909941+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:31:16.889956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.930569+0000","last_change":"2023-10-06T05:05:40.928894+0000","last_active":"2023-10-06T05:05:40.930569+0000","last_peered":"2023-10-06T05:05:40.930569+0000","last_clean":"2023-10-06T05:05:40.930569+0000","last_became_active":"2023-10-06T05:05:40.928638+0000","last_became_peered":"2023-10-06T05:05:40.928638+0000","last_unstale":"2023-10-06T05:05:40.930569+0000","last_undegraded":"2023-10-06T05:05:40.930569+0000","last_fullsized":"2023-10-06T05:05:40.930569+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:49.513363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"192'2","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946359+0000","last_change":"2023-10-06T05:05:39.924825+0000","last_active":"2023-10-06T05:05:40.946359+0000","last_peered":"2023-10-06T05:05:40.946359+0000","last_clean":"2023-10-06T05:05:40.946359+0000","last_became_active":"2023-10-06T05:05:39.923420+0000","last_became_peered":"2023-10-06T05:05:39.923420+0000","last_unstale":"2023-10-06T05:05:40.946359+0000","last_undegraded":"2023-10-06T05:05:40.946359+0000","last_fullsized":"2023-10-06T05:05:40.946359+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:57.786548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":175,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.910780+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T05:05:40.910780+0000","last_peered":"2023-10-06T05:05:40.910780+0000","last_clean":"2023-10-06T05:05:40.910780+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T05:05:40.910780+0000","last_undegraded":"2023-10-06T05:05:40.910780+0000","last_fullsized":"2023-10-06T05:05:40.910780+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.950557+0000","last_change":"2023-10-06T05:05:40.945808+0000","last_active":"2023-10-06T05:05:40.950557+0000","last_peered":"2023-10-06T05:05:40.950557+0000","last_clean":"2023-10-06T05:05:40.950557+0000","last_became_active":"2023-10-06T05:05:40.945531+0000","last_became_peered":"2023-10-06T05:05:40.945531+0000","last_unstale":"2023-10-06T05:05:40.950557+0000","last_undegraded":"2023-10-06T05:05:40.950557+0000","last_fullsized":"2023-10-06T05:05:40.950557+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:06.936450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.914985+0000","last_change":"2023-10-06T05:05:39.923348+0000","last_active":"2023-10-06T05:05:40.914985+0000","last_peered":"2023-10-06T05:05:40.914985+0000","last_clean":"2023-10-06T05:05:40.914985+0000","last_became_active":"2023-10-06T05:05:39.922810+0000","last_became_peered":"2023-10-06T05:05:39.922810+0000","last_unstale":"2023-10-06T05:05:40.914985+0000","last_undegraded":"2023-10-06T05:05:40.914985+0000","last_fullsized":"2023-10-06T05:05:40.914985+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:20:19.361912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":163,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.915504+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T05:05:40.915504+0000","last_peered":"2023-10-06T05:05:40.915504+0000","last_clean":"2023-10-06T05:05:40.915504+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T05:05:40.915504+0000","last_undegraded":"2023-10-06T05:05:40.915504+0000","last_fullsized":"2023-10-06T05:05:40.915504+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.949849+0000","last_change":"2023-10-06T05:05:40.947853+0000","last_active":"2023-10-06T05:05:40.949849+0000","last_peered":"2023-10-06T05:05:40.949849+0000","last_clean":"2023-10-06T05:05:40.949849+0000","last_became_active":"2023-10-06T05:05:40.947476+0000","last_became_peered":"2023-10-06T05:05:40.947476+0000","last_unstale":"2023-10-06T05:05:40.949849+0000","last_undegraded":"2023-10-06T05:05:40.949849+0000","last_fullsized":"2023-10-06T05:05:40.949849+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:55:58.796408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"192'3","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.936421+0000","last_change":"2023-10-06T05:05:39.922291+0000","last_active":"2023-10-06T05:05:40.936421+0000","last_peered":"2023-10-06T05:05:40.936421+0000","last_clean":"2023-10-06T05:05:40.936421+0000","last_became_active":"2023-10-06T05:05:39.922032+0000","last_became_peered":"2023-10-06T05:05:39.922032+0000","last_unstale":"2023-10-06T05:05:40.936421+0000","last_undegraded":"2023-10-06T05:05:40.936421+0000","last_fullsized":"2023-10-06T05:05:40.936421+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:51:52.678186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":160,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.924883+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T05:05:40.924883+0000","last_peered":"2023-10-06T05:05:40.924883+0000","last_clean":"2023-10-06T05:05:40.924883+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T05:05:40.924883+0000","last_undegraded":"2023-10-06T05:05:40.924883+0000","last_fullsized":"2023-10-06T05:05:40.924883+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"183'326","reported_seq":1053,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.923602+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T05:05:40.923602+0000","last_peered":"2023-10-06T05:05:40.923602+0000","last_clean":"2023-10-06T05:05:40.923602+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T05:05:40.923602+0000","last_undegraded":"2023-10-06T05:05:40.923602+0000","last_fullsized":"2023-10-06T05:05:40.923602+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.935830+0000","last_change":"2023-10-06T05:05:40.931126+0000","last_active":"2023-10-06T05:05:40.935830+0000","last_peered":"2023-10-06T05:05:40.935830+0000","last_clean":"2023-10-06T05:05:40.935830+0000","last_became_active":"2023-10-06T05:05:40.930016+0000","last_became_peered":"2023-10-06T05:05:40.930016+0000","last_unstale":"2023-10-06T05:05:40.935830+0000","last_undegraded":"2023-10-06T05:05:40.935830+0000","last_fullsized":"2023-10-06T05:05:40.935830+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:49.846088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"192'6","reported_seq":16,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.952749+0000","last_change":"2023-10-06T05:05:39.928213+0000","last_active":"2023-10-06T05:05:40.952749+0000","last_peered":"2023-10-06T05:05:40.952749+0000","last_clean":"2023-10-06T05:05:40.952749+0000","last_became_active":"2023-10-06T05:05:39.927927+0000","last_became_peered":"2023-10-06T05:05:39.927927+0000","last_unstale":"2023-10-06T05:05:40.952749+0000","last_undegraded":"2023-10-06T05:05:40.952749+0000","last_fullsized":"2023-10-06T05:05:40.952749+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:55:40.706784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":163,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.915543+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T05:05:40.915543+0000","last_peered":"2023-10-06T05:05:40.915543+0000","last_clean":"2023-10-06T05:05:40.915543+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T05:05:40.915543+0000","last_undegraded":"2023-10-06T05:05:40.915543+0000","last_fullsized":"2023-10-06T05:05:40.915543+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.932909+0000","last_change":"2023-10-06T05:05:40.929922+0000","last_active":"2023-10-06T05:05:40.932909+0000","last_peered":"2023-10-06T05:05:40.932909+0000","last_clean":"2023-10-06T05:05:40.932909+0000","last_became_active":"2023-10-06T05:05:40.929618+0000","last_became_peered":"2023-10-06T05:05:40.929618+0000","last_unstale":"2023-10-06T05:05:40.932909+0000","last_undegraded":"2023-10-06T05:05:40.932909+0000","last_fullsized":"2023-10-06T05:05:40.932909+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:25:42.478177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.916517+0000","last_change":"2023-10-06T05:05:39.921775+0000","last_active":"2023-10-06T05:05:40.916517+0000","last_peered":"2023-10-06T05:05:40.916517+0000","last_clean":"2023-10-06T05:05:40.916517+0000","last_became_active":"2023-10-06T05:05:39.921525+0000","last_became_peered":"2023-10-06T05:05:39.921525+0000","last_unstale":"2023-10-06T05:05:40.916517+0000","last_undegraded":"2023-10-06T05:05:40.916517+0000","last_fullsized":"2023-10-06T05:05:40.916517+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:17:12.112490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":163,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.925284+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T05:05:40.925284+0000","last_peered":"2023-10-06T05:05:40.925284+0000","last_clean":"2023-10-06T05:05:40.925284+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T05:05:40.925284+0000","last_undegraded":"2023-10-06T05:05:40.925284+0000","last_fullsized":"2023-10-06T05:05:40.925284+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.943745+0000","last_change":"2023-10-06T05:05:40.942382+0000","last_active":"2023-10-06T05:05:40.943745+0000","last_peered":"2023-10-06T05:05:40.943745+0000","last_clean":"2023-10-06T05:05:40.943745+0000","last_became_active":"2023-10-06T05:05:40.941889+0000","last_became_peered":"2023-10-06T05:05:40.941889+0000","last_unstale":"2023-10-06T05:05:40.943745+0000","last_undegraded":"2023-10-06T05:05:40.943745+0000","last_fullsized":"2023-10-06T05:05:40.943745+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:00:10.851897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.915883+0000","last_change":"2023-10-06T05:05:39.929195+0000","last_active":"2023-10-06T05:05:40.915883+0000","last_peered":"2023-10-06T05:05:40.915883+0000","last_clean":"2023-10-06T05:05:40.915883+0000","last_became_active":"2023-10-06T05:05:39.928902+0000","last_became_peered":"2023-10-06T05:05:39.928902+0000","last_unstale":"2023-10-06T05:05:40.915883+0000","last_undegraded":"2023-10-06T05:05:40.915883+0000","last_fullsized":"2023-10-06T05:05:40.915883+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:03:23.422305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":155,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.917075+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T05:05:40.917075+0000","last_peered":"2023-10-06T05:05:40.917075+0000","last_clean":"2023-10-06T05:05:40.917075+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T05:05:40.917075+0000","last_undegraded":"2023-10-06T05:05:40.917075+0000","last_fullsized":"2023-10-06T05:05:40.917075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.942379+0000","last_change":"2023-10-06T05:05:40.939000+0000","last_active":"2023-10-06T05:05:40.942379+0000","last_peered":"2023-10-06T05:05:40.942379+0000","last_clean":"2023-10-06T05:05:40.942379+0000","last_became_active":"2023-10-06T05:05:40.938765+0000","last_became_peered":"2023-10-06T05:05:40.938765+0000","last_unstale":"2023-10-06T05:05:40.942379+0000","last_undegraded":"2023-10-06T05:05:40.942379+0000","last_fullsized":"2023-10-06T05:05:40.942379+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:07:16.297316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"192'8","reported_seq":18,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.946955+0000","last_change":"2023-10-06T05:05:39.927836+0000","last_active":"2023-10-06T05:05:40.946955+0000","last_peered":"2023-10-06T05:05:40.946955+0000","last_clean":"2023-10-06T05:05:40.946955+0000","last_became_active":"2023-10-06T05:05:39.927333+0000","last_became_peered":"2023-10-06T05:05:39.927333+0000","last_unstale":"2023-10-06T05:05:40.946955+0000","last_undegraded":"2023-10-06T05:05:40.946955+0000","last_fullsized":"2023-10-06T05:05:40.946955+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:04.303464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":186,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.911654+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T05:05:40.911654+0000","last_peered":"2023-10-06T05:05:40.911654+0000","last_clean":"2023-10-06T05:05:40.911654+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T05:05:40.911654+0000","last_undegraded":"2023-10-06T05:05:40.911654+0000","last_fullsized":"2023-10-06T05:05:40.911654+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":9,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.936102+0000","last_change":"2023-10-06T05:05:40.932466+0000","last_active":"2023-10-06T05:05:40.936102+0000","last_peered":"2023-10-06T05:05:40.936102+0000","last_clean":"2023-10-06T05:05:40.936102+0000","last_became_active":"2023-10-06T05:05:40.932197+0000","last_became_peered":"2023-10-06T05:05:40.932197+0000","last_unstale":"2023-10-06T05:05:40.936102+0000","last_undegraded":"2023-10-06T05:05:40.936102+0000","last_fullsized":"2023-10-06T05:05:40.936102+0000","mapping_epoch":191,"log_start":"0'0","ondisk_log_start":"0'0","created":191,"last_epoch_clean":192,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:39.893997+0000","last_clean_scrub_stamp":"2023-10-06T05:05:39.893997+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:03:06.936450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.916863+0000","last_change":"2023-10-06T05:05:39.922809+0000","last_active":"2023-10-06T05:05:40.916863+0000","last_peered":"2023-10-06T05:05:40.916863+0000","last_clean":"2023-10-06T05:05:40.916863+0000","last_became_active":"2023-10-06T05:05:39.922518+0000","last_became_peered":"2023-10-06T05:05:39.922518+0000","last_unstale":"2023-10-06T05:05:40.916863+0000","last_undegraded":"2023-10-06T05:05:40.916863+0000","last_fullsized":"2023-10-06T05:05:40.916863+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:05:38.884137+0000","last_clean_scrub_stamp":"2023-10-06T05:05:38.884137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:54:04.303464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":186,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-10-06T05:05:40.910317+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T05:05:40.910317+0000","last_peered":"2023-10-06T05:05:40.910317+0000","last_clean":"2023-10-06T05:05:40.910317+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T05:05:40.910317+0000","last_undegraded":"2023-10-06T05:05:40.910317+0000","last_fullsized":"2023-10-06T05:05:40.910317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575946,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304316,"kb_used_data":2932,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067524,"statfs":{"total":96636764160,"available":96325144576,"internally_reserved":0,"allocated":3002368,"data_stored":1864372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.75,"15min":0.74299999999999999},"min":{"1min":0.498,"5min":0.48499999999999999,"15min":0.44500000000000001},"max":{"1min":0.94199999999999995,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.64900000000000002,"5min":0.48699999999999999,"15min":0.48399999999999999},"max":{"1min":0.94099999999999995,"5min":0.94599999999999995,"15min":1.0529999999999999},"last":0.79300000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.82299999999999995,"15min":0.80700000000000005},"min":{"1min":0.67500000000000004,"5min":0.65000000000000002,"15min":0.50800000000000001},"max":{"1min":1.0640000000000001,"5min":1.0720000000000001,"15min":1.073},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.81000000000000005,"15min":0.81299999999999994},"min":{"1min":0.63400000000000001,"5min":0.627,"15min":0.55900000000000005},"max":{"1min":0.96599999999999997,"5min":0.99299999999999999,"15min":1.0860000000000001},"last":0.92200000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.502},"max":{"1min":1.034,"5min":1.034,"15min":1.095},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78500000000000003,"15min":0.78400000000000003},"min":{"1min":0.61899999999999999,"5min":0.54300000000000004,"15min":0.50900000000000001},"max":{"1min":1.107,"5min":1.107,"15min":1.1160000000000001},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.77000000000000002,"15min":0.77700000000000002},"min":{"1min":0.68300000000000005,"5min":0.55300000000000005,"15min":0.51400000000000001},"max":{"1min":0.99099999999999999,"5min":1.0029999999999999,"15min":1.101},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76700000000000002,"15min":0.77000000000000002},"min":{"1min":0.63200000000000001,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0149999999999999,"5min":1.024,"15min":1.0369999999999999},"last":0.89300000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60599999999999998,"15min":0.62},"min":{"1min":0.45600000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.77700000000000002,"5min":1.282,"15min":1.355},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.45900000000000002,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89700000000000002,"5min":1.196,"15min":1.208},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.63500000000000001,"15min":0.65800000000000003},"min":{"1min":0.42299999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0009999999999999,"5min":1.044,"15min":1.054},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66400000000000003,"15min":0.69299999999999995},"min":{"1min":0.50800000000000001,"5min":0.48399999999999999,"15min":0.47699999999999998},"max":{"1min":0.871,"5min":0.98999999999999999,"15min":1.0760000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.57799999999999996,"5min":0.497,"15min":0.48799999999999999},"max":{"1min":0.92200000000000004,"5min":1.02,"15min":1.044},"last":0.747},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.69699999999999995,"15min":0.71799999999999997},"min":{"1min":0.56599999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95699999999999996,"5min":1.0229999999999999,"15min":1.0469999999999999},"last":0.71899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575947,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299644,"kb_used_data":2356,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072196,"statfs":{"total":96636764160,"available":96329928704,"internally_reserved":0,"allocated":2412544,"data_stored":1274982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73299999999999998,"15min":0.72799999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.40699999999999997},"max":{"1min":1.0009999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.749,"15min":0.76100000000000001},"min":{"1min":0.53000000000000003,"5min":0.48799999999999999,"15min":0.46600000000000003},"max":{"1min":0.96399999999999997,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.96799999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.77100000000000002,"15min":0.75900000000000001},"min":{"1min":0.53200000000000003,"5min":0.47099999999999997,"15min":0.378},"max":{"1min":0.92300000000000004,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.77800000000000002,"15min":0.76500000000000001},"min":{"1min":0.60199999999999998,"5min":0.50600000000000001,"15min":0.42399999999999999},"max":{"1min":0.94899999999999995,"5min":1.425,"15min":1.425},"last":0.88400000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77500000000000002,"15min":0.76700000000000002},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.44700000000000001},"max":{"1min":1.0069999999999999,"5min":1.083,"15min":1.147},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77200000000000002,"15min":0.77400000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48299999999999998},"max":{"1min":1.03,"5min":1.0640000000000001,"15min":1.097},"last":0.98799999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.59599999999999997,"5min":0.54900000000000004,"15min":0.505},"max":{"1min":0.97999999999999998,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":1.016},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81699999999999995,"15min":0.79600000000000004},"min":{"1min":0.627,"5min":0.60799999999999998,"15min":0.53700000000000003},"max":{"1min":0.97399999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.92500000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60799999999999998,"15min":0.58899999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.38500000000000001},"max":{"1min":0.73199999999999998,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.435},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.64400000000000002,"15min":0.63900000000000001},"min":{"1min":0.46600000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94299999999999995,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.69699999999999995,"15min":0.70199999999999996},"min":{"1min":0.5,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.98099999999999998,"5min":1.0680000000000001,"15min":1.069},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.746,"15min":0.73399999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.41199999999999998},"max":{"1min":0.95399999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.50900000000000001,"5min":0.441,"15min":0.441},"max":{"1min":0.92700000000000005,"5min":1.0369999999999999,"15min":1.0580000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.747,"15min":0.73299999999999998},"min":{"1min":0.56799999999999995,"5min":0.45100000000000001,"15min":0.45000000000000001},"max":{"1min":0.93000000000000005,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.68300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575946,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312524,"kb_used_data":2948,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059316,"statfs":{"total":96636764160,"available":96316739584,"internally_reserved":0,"allocated":3018752,"data_stored":1866886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32400000000000001},"max":{"1min":0.73199999999999998,"5min":1.155,"15min":1.155},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.59899999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.38900000000000001},"max":{"1min":0.77100000000000002,"5min":0.88900000000000001,"15min":0.90100000000000002},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.52100000000000002},"min":{"1min":0.379,"5min":0.378,"15min":0.35499999999999998},"max":{"1min":0.64800000000000002,"5min":1.153,"15min":1.153},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.37},"max":{"1min":0.86099999999999999,"5min":1.018,"15min":1.018},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64000000000000001,"15min":0.63700000000000001},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.41799999999999998},"max":{"1min":0.82999999999999996,"5min":1.131,"15min":1.131},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.44,"5min":0.40500000000000003,"15min":0.38100000000000001},"max":{"1min":0.78100000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.48999999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.79500000000000004,"15min":0.80800000000000005},"min":{"1min":0.69199999999999995,"5min":0.50800000000000001,"15min":0.46400000000000002},"max":{"1min":1.161,"5min":1.1739999999999999,"15min":1.2250000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.84299999999999997,"15min":0.85099999999999998},"min":{"1min":0.70999999999999996,"5min":0.61799999999999999,"15min":0.57799999999999996},"max":{"1min":1.04,"5min":1.236,"15min":1.448},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83899999999999997,"15min":0.84799999999999998},"min":{"1min":0.62,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.083,"5min":1.21,"15min":1.484},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.84199999999999997,"15min":0.84799999999999998},"min":{"1min":0.624,"5min":0.51000000000000001,"15min":0.502},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.2829999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.85799999999999998,"15min":0.85899999999999999},"min":{"1min":0.755,"5min":0.60699999999999998,"15min":0.58099999999999996},"max":{"1min":1.1639999999999999,"5min":1.2090000000000001,"15min":1.2709999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.86199999999999999,"15min":0.87},"min":{"1min":0.65900000000000003,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.1950000000000001,"5min":1.242,"15min":1.4690000000000001},"last":0.73399999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88700000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.59899999999999998},"max":{"1min":1.167,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.877,"5min":0.88400000000000001,"15min":0.88500000000000001},"min":{"1min":0.67300000000000004,"5min":0.65300000000000002,"15min":0.61699999999999999},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.3080000000000001},"last":0.80200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575948,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303744,"kb_used_data":2360,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068096,"statfs":{"total":96636764160,"available":96325730304,"internally_reserved":0,"allocated":2416640,"data_stored":1274108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:05:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.49199999999999999,"15min":0.504},"min":{"1min":0.35599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.747,"5min":0.80800000000000005,"15min":0.83399999999999996},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54400000000000004,"15min":0.55600000000000005},"min":{"1min":0.377,"5min":0.35799999999999998,"15min":0.34399999999999997},"max":{"1min":0.80700000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:05:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.42699999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83299999999999996,"5min":0.86899999999999999,"15min":0.91000000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.57999999999999996},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.38900000000000001},"max":{"1min":0.69999999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.38300000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:05:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.40999999999999998,"5min":0.379,"15min":0.34300000000000003},"max":{"1min":0.71799999999999997,"5min":0.753,"15min":0.86699999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.47699999999999998,"5min":0.46200000000000002,"15min":0.42899999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.48499999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:05:12 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.86599999999999999,"15min":0.86499999999999999},"min":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.54300000000000004},"max":{"1min":1.1419999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.85999999999999999,"15min":0.86499999999999999},"min":{"1min":0.75,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.1220000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:05:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.84799999999999998,"15min":0.84699999999999998},"min":{"1min":0.69499999999999995,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.1140000000000001,"5min":1.2729999999999999,"15min":1.2729999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.85199999999999998,"15min":0.85399999999999998},"min":{"1min":0.63900000000000001,"5min":0.57399999999999995,"15min":0.45600000000000002},"max":{"1min":1.204,"5min":1.2110000000000001,"15min":1.466},"last":0.78600000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:05:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.83999999999999997,"15min":0.84499999999999997},"min":{"1min":0.67000000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1240000000000001,"5min":1.2869999999999999,"15min":1.4419999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.875,"5min":0.88100000000000001,"15min":0.88800000000000001},"min":{"1min":0.66500000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.21,"5min":1.321,"15min":1.54},"last":0.80300000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:05:12 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88400000000000001,"15min":0.877},"min":{"1min":0.624,"5min":0.624,"15min":0.59799999999999998},"max":{"1min":1.2490000000000001,"5min":1.367,"15min":1.367},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88600000000000001,"15min":0.88300000000000001},"min":{"1min":0.65600000000000003,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.0549999999999999,"5min":1.294,"15min":1.6399999999999999},"last":0.64400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575947,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316024,"kb_used_data":2352,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055816,"statfs":{"total":96636764160,"available":96313155584,"internally_reserved":0,"allocated":2408448,"data_stored":1274004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.33400000000000002,"5min":0.31900000000000001,"15min":0.29199999999999998},"max":{"1min":0.746,"5min":0.748,"15min":0.80300000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55800000000000005,"15min":0.55400000000000005},"min":{"1min":0.37,"5min":0.35999999999999999,"15min":0.31},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.91200000000000003},"last":0.35799999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78300000000000003,"5min":0.89400000000000002,"15min":0.92400000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.432,"5min":0.41399999999999998,"15min":0.35999999999999999},"max":{"1min":0.70299999999999996,"5min":0.86599999999999999,"15min":0.95299999999999996},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.60599999999999998},"min":{"1min":0.45200000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.751,"5min":0.78400000000000003,"15min":0.85999999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59299999999999997,"15min":0.59799999999999998},"min":{"1min":0.42199999999999999,"5min":0.40799999999999997,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.66500000000000004,"5min":0.57799999999999996,"15min":0.55900000000000005},"max":{"1min":1.046,"5min":1.1910000000000001,"15min":1.383},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86799999999999999,"15min":0.876},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.59199999999999997},"max":{"1min":1.0700000000000001,"5min":1.246,"15min":1.254},"last":1.0149999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.879,"15min":0.88300000000000001},"min":{"1min":0.66800000000000004,"5min":0.59099999999999997,"15min":0.57599999999999996},"max":{"1min":1.0309999999999999,"5min":1.345,"15min":1.345},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.89000000000000001,"15min":0.89400000000000002},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0249999999999999,"5min":1.329,"15min":1.339},"last":1.0229999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.874,"15min":0.86299999999999999},"min":{"1min":0.66200000000000003,"5min":0.60499999999999998,"15min":0.58199999999999996},"max":{"1min":1.0580000000000001,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.84199999999999997,"15min":0.85799999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55000000000000004},"max":{"1min":1.054,"5min":1.135,"15min":1.4450000000000001},"last":0.89700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.86699999999999999,"15min":0.86399999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.997,"5min":1.276,"15min":1.425},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.91100000000000003,"15min":0.90400000000000003},"min":{"1min":0.66600000000000004,"5min":0.61399999999999999,"15min":0.59399999999999997},"max":{"1min":1.1970000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.97299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575947,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316048,"kb_used_data":2376,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055792,"statfs":{"total":96636764160,"available":96313131008,"internally_reserved":0,"allocated":2433024,"data_stored":1277094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.29799999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.40300000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.65200000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.49399999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51400000000000001,"15min":0.51200000000000001},"min":{"1min":0.34899999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80500000000000005,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.307},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.58299999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76000000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.371}]},{"osd":2,"last update":"Fri Oct 6 05:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59499999999999997,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.39700000000000002},"max":{"1min":0.80000000000000004,"5min":0.81200000000000006,"15min":0.81499999999999995},"last":0.432},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65000000000000002,"15min":0.63900000000000001},"min":{"1min":0.48499999999999999,"5min":0.47599999999999998,"15min":0.38500000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.92300000000000004},"last":0.46000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.55700000000000005,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0249999999999999,"5min":1.1559999999999999,"15min":1.2849999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.88200000000000001,"15min":0.876},"min":{"1min":0.70999999999999996,"5min":0.64400000000000002,"15min":0.52500000000000002},"max":{"1min":1.044,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82899999999999996,"15min":0.82199999999999995},"min":{"1min":0.624,"5min":0.57499999999999996,"15min":0.51300000000000001},"max":{"1min":1.115,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84899999999999998,"15min":0.84399999999999997},"min":{"1min":0.64600000000000002,"5min":0.60799999999999998,"15min":0.56100000000000005},"max":{"1min":1.0209999999999999,"5min":1.3300000000000001,"15min":1.3300000000000001},"last":0.55900000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.84499999999999997,"15min":0.85299999999999998},"min":{"1min":0.66500000000000004,"5min":0.55700000000000005,"15min":0.52800000000000002},"max":{"1min":1.0640000000000001,"5min":1.2150000000000001,"15min":1.288},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.89600000000000002,"15min":0.89900000000000002},"min":{"1min":0.76700000000000002,"5min":0.63,"15min":0.57499999999999996},"max":{"1min":1.123,"5min":1.2669999999999999,"15min":1.411},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.871,"15min":0.875},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.57599999999999996},"max":{"1min":1.0649999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.86099999999999999,"15min":0.85399999999999998},"min":{"1min":0.62,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.089,"5min":1.365,"15min":1.389},"last":0.622}]}]},{"osd":4,"up_from":13,"seq":55834575947,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311948,"kb_used_data":2372,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059892,"statfs":{"total":96636764160,"available":96317329408,"internally_reserved":0,"allocated":2428928,"data_stored":1275718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.42399999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.2},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76800000000000002,"15min":0.76100000000000001},"min":{"1min":0.57899999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.018,"5min":1.048,"15min":1.048},"last":0.745}]},{"osd":1,"last update":"Fri Oct 6 05:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.74099999999999999,"15min":0.749},"min":{"1min":0.56499999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.78000000000000003,"15min":0.78100000000000003},"min":{"1min":0.52400000000000002,"5min":0.48499999999999999,"15min":0.46700000000000003},"max":{"1min":0.99099999999999999,"5min":1.026,"15min":1.258},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.77500000000000002},"min":{"1min":0.58599999999999997,"5min":0.55000000000000004,"15min":0.48299999999999998},"max":{"1min":0.92200000000000004,"5min":1.0089999999999999,"15min":1.0309999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78000000000000003,"15min":0.78700000000000003},"min":{"1min":0.60799999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.88600000000000001,"5min":1.0049999999999999,"15min":1.2929999999999999},"last":0.77300000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76600000000000001,"15min":0.76400000000000001},"min":{"1min":0.59199999999999997,"5min":0.52000000000000002,"15min":0.49299999999999999},"max":{"1min":0.91500000000000004,"5min":0.97999999999999998,"15min":1.2330000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78400000000000003,"15min":0.79900000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.95499999999999996,"5min":1.012,"15min":1.034},"last":0.85599999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.625,"15min":0.64200000000000002},"min":{"1min":0.48299999999999998,"5min":0.41599999999999998,"15min":0.41199999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.323},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.39700000000000002},"max":{"1min":0.877,"5min":1.01,"15min":1.1910000000000001},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.67500000000000004,"15min":0.68000000000000005},"min":{"1min":0.49399999999999999,"5min":0.42999999999999999,"15min":0.42099999999999999},"max":{"1min":0.88800000000000001,"5min":1.024,"15min":1.079},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69199999999999995,"15min":0.71299999999999997},"min":{"1min":0.54300000000000004,"5min":0.44400000000000001,"15min":0.41999999999999998},"max":{"1min":0.93000000000000005,"5min":1.022,"15min":1.3540000000000001},"last":0.51000000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.69999999999999996,"15min":0.71699999999999997},"min":{"1min":0.502,"5min":0.47699999999999998,"15min":0.45200000000000001},"max":{"1min":0.97299999999999998,"5min":1.0640000000000001,"15min":1.155},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73999999999999999,"15min":0.75700000000000001},"min":{"1min":0.56499999999999995,"5min":0.505,"15min":0.47299999999999998},"max":{"1min":1.0069999999999999,"5min":1.079,"15min":1.149},"last":0.79100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575947,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307840,"kb_used_data":2360,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064000,"statfs":{"total":96636764160,"available":96321536000,"internally_reserved":0,"allocated":2416640,"data_stored":1274128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.63700000000000001,"5min":0.504,"15min":0.39800000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.171},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.751,"15min":0.73899999999999999},"min":{"1min":0.60699999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.999,"5min":0.999,"15min":1.0209999999999999},"last":0.78700000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.64000000000000001,"5min":0.505,"15min":0.42099999999999999},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.2330000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.77100000000000002},"min":{"1min":0.61099999999999999,"5min":0.55000000000000004,"15min":0.42699999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.095},"last":0.84899999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.81299999999999994,"15min":0.80600000000000005},"min":{"1min":0.63700000000000001,"5min":0.51800000000000002,"15min":0.46800000000000003},"max":{"1min":1.0700000000000001,"5min":1.0780000000000001,"15min":1.325},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.83199999999999996,"15min":0.82499999999999996},"min":{"1min":0.71099999999999997,"5min":0.628,"15min":0.52100000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.349},"last":0.77400000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.76300000000000001,"15min":0.75900000000000001},"min":{"1min":0.66100000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.2809999999999999},"last":0.747},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.80100000000000005,"15min":0.79300000000000004},"min":{"1min":0.61399999999999999,"5min":0.57899999999999996,"15min":0.55400000000000005},"max":{"1min":1.095,"5min":1.095,"15min":1.298},"last":0.86499999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.624,"15min":0.63400000000000001},"min":{"1min":0.47799999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93799999999999994,"5min":0.97699999999999998,"15min":1.0049999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.64800000000000002,"15min":0.64600000000000002},"min":{"1min":0.439,"5min":0.40400000000000003,"15min":0.38300000000000001},"max":{"1min":1.01,"5min":1.01,"15min":1.042},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.47899999999999998,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.93999999999999995,"5min":0.98799999999999999,"15min":1.0369999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.71499999999999997,"15min":0.71299999999999997},"min":{"1min":0.53600000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:05:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.69499999999999995},"min":{"1min":0.496,"5min":0.45700000000000002,"15min":0.44500000000000001},"max":{"1min":0.99099999999999999,"5min":1.0269999999999999,"15min":1.0669999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73599999999999999,"15min":0.74199999999999999},"min":{"1min":0.54600000000000004,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.67000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:05:45.779 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:05:45.779 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:05:46.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.006+0000 7fc7289fe700 1 Processor -- start 2023-10-06T05:05:46.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.007+0000 7fc7289fe700 1 -- start start 2023-10-06T05:05:46.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.008+0000 7fc7289fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7240c6e80 0x7fc7240ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:46.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.008+0000 7fc7289fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc7240c9b60 0x7fc7240cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:46.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.008+0000 7fc7289fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc7240c7250 0x7fc724142b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:46.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.008+0000 7fc7289fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7240cb5b0 con 0x7fc7240c7250 2023-10-06T05:05:46.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.008+0000 7fc7289fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7240ca700 con 0x7fc7240c9b60 2023-10-06T05:05:46.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.008+0000 7fc7289fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240ca9a0 con 0x7fc7240c6e80 2023-10-06T05:05:46.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.008+0000 7fc72359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc7240c7250 0x7fc724142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:46.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.008+0000 7fc72359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc7240c7250 0x7fc724142b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45490/0 (socket says 172.21.15.55:45490) 2023-10-06T05:05:46.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.008+0000 7fc72359e700 1 -- 172.21.15.55:0/2377618222 learned_addr learned my addr 172.21.15.55:0/2377618222 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:46.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.008+0000 7fc72259c700 1 --2- 172.21.15.55:0/2377618222 >> v2:172.21.15.163:3300/0 conn(0x7fc7240c9b60 0x7fc7240cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:46.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.008+0000 7fc722d9d700 1 --2- 172.21.15.55:0/2377618222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7240c6e80 0x7fc7240ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:46.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.009+0000 7fc72359e700 1 -- 172.21.15.55:0/2377618222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7240c6e80 msgr2=0x7fc7240ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:46.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.009+0000 7fc72359e700 1 --2- 172.21.15.55:0/2377618222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7240c6e80 0x7fc7240ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.009+0000 7fc72359e700 1 -- 172.21.15.55:0/2377618222 >> v2:172.21.15.163:3300/0 conn(0x7fc7240c9b60 msgr2=0x7fc7240cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:46.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.009+0000 7fc72359e700 1 --2- 172.21.15.55:0/2377618222 >> v2:172.21.15.163:3300/0 conn(0x7fc7240c9b60 0x7fc7240cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.009+0000 7fc72359e700 1 -- 172.21.15.55:0/2377618222 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7240cac80 con 0x7fc7240c7250 2023-10-06T05:05:46.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.009+0000 7fc72259c700 1 --2- 172.21.15.55:0/2377618222 >> v2:172.21.15.163:3300/0 conn(0x7fc7240c9b60 0x7fc7240cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:46.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.009+0000 7fc72359e700 1 --2- 172.21.15.55:0/2377618222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7240c7250 0x7fc724142b90 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fc72406cde0 tx=0x7fc71c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d884cf3d8c73cedc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:46.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.009+0000 7fc721d9b700 1 -- 172.21.15.55:0/2377618222 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c00e070 con 0x7fc7240c7250 2023-10-06T05:05:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.009+0000 7fc721d9b700 1 -- 172.21.15.55:0/2377618222 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71c00ba50 con 0x7fc7240c7250 2023-10-06T05:05:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.009+0000 7fc721d9b700 1 -- 172.21.15.55:0/2377618222 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0092e0 con 0x7fc7240c7250 2023-10-06T05:05:46.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.010+0000 7fc7289fe700 1 -- 172.21.15.55:0/2377618222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7240c7250 msgr2=0x7fc724142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.010+0000 7fc7289fe700 1 --2- 172.21.15.55:0/2377618222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7240c7250 0x7fc724142b90 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fc72406cde0 tx=0x7fc71c009840 comp rx=0 tx=0).stop 2023-10-06T05:05:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.010+0000 7fc7289fe700 1 -- 172.21.15.55:0/2377618222 shutdown_connections 2023-10-06T05:05:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.010+0000 7fc7289fe700 1 --2- 172.21.15.55:0/2377618222 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7240c7250 0x7fc724142b90 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.010+0000 7fc7289fe700 1 --2- 172.21.15.55:0/2377618222 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7240c6e80 0x7fc7240ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.010+0000 7fc7289fe700 1 --2- 172.21.15.55:0/2377618222 >> v2:172.21.15.163:3300/0 conn(0x7fc7240c9b60 0x7fc7240cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.010+0000 7fc7289fe700 1 -- 172.21.15.55:0/2377618222 >> 172.21.15.55:0/2377618222 conn(0x7fc7240cd0c0 msgr2=0x7fc7240c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.010+0000 7fc7289fe700 1 -- 172.21.15.55:0/2377618222 shutdown_connections 2023-10-06T05:05:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.010+0000 7fc7289fe700 1 -- 172.21.15.55:0/2377618222 wait complete. 2023-10-06T05:05:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.011+0000 7fc7289fe700 1 Processor -- start 2023-10-06T05:05:46.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.011+0000 7fc7289fe700 1 -- start start 2023-10-06T05:05:46.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.012+0000 7fc7289fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7240c6e80 0x7fc72415ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:46.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.012+0000 7fc7289fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7240c7250 0x7fc7240c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:46.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.012+0000 7fc7289fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7240c9b60 0x7fc7240c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:46.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.012+0000 7fc7289fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240c5b40 con 0x7fc7240c9b60 2023-10-06T05:05:46.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.012+0000 7fc7289fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240c6510 con 0x7fc7240c7250 2023-10-06T05:05:46.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.012+0000 7fc7289fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240c3cf0 con 0x7fc7240c6e80 2023-10-06T05:05:46.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.013+0000 7fc72359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7240c9b60 0x7fc7240c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:46.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.013+0000 7fc72359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7240c9b60 0x7fc7240c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45502/0 (socket says 172.21.15.55:45502) 2023-10-06T05:05:46.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.013+0000 7fc72359e700 1 -- 172.21.15.55:0/386292366 learned_addr learned my addr 172.21.15.55:0/386292366 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:46.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.013+0000 7fc72259c700 1 --2- 172.21.15.55:0/386292366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7240c7250 0x7fc7240c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:46.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.013+0000 7fc722d9d700 1 --2- 172.21.15.55:0/386292366 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7240c6e80 0x7fc72415ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:46.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.013+0000 7fc72359e700 1 -- 172.21.15.55:0/386292366 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7240c6e80 msgr2=0x7fc72415ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:46.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.013+0000 7fc72359e700 1 --2- 172.21.15.55:0/386292366 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7240c6e80 0x7fc72415ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.013+0000 7fc72359e700 1 -- 172.21.15.55:0/386292366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7240c7250 msgr2=0x7fc7240c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:46.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.013+0000 7fc72359e700 1 --2- 172.21.15.55:0/386292366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7240c7250 0x7fc7240c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.013+0000 7fc72359e700 1 -- 172.21.15.55:0/386292366 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c009ae0 con 0x7fc7240c9b60 2023-10-06T05:05:46.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.013+0000 7fc72359e700 1 --2- 172.21.15.55:0/386292366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7240c9b60 0x7fc7240c5fd0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009880 tx=0x7fc71c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:46.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.014+0000 7fc70affd700 1 -- 172.21.15.55:0/386292366 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0032f0 con 0x7fc7240c9b60 2023-10-06T05:05:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.014+0000 7fc70affd700 1 -- 172.21.15.55:0/386292366 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71c008d20 con 0x7fc7240c9b60 2023-10-06T05:05:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.014+0000 7fc70affd700 1 -- 172.21.15.55:0/386292366 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c01c640 con 0x7fc7240c9b60 2023-10-06T05:05:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.014+0000 7fc7289fe700 1 -- 172.21.15.55:0/386292366 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7240c4f30 con 0x7fc7240c9b60 2023-10-06T05:05:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.014+0000 7fc7289fe700 1 -- 172.21.15.55:0/386292366 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7240c54f0 con 0x7fc7240c9b60 2023-10-06T05:05:46.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.015+0000 7fc7289fe700 1 -- 172.21.15.55:0/386292366 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7240c5cb0 con 0x7fc7240c9b60 2023-10-06T05:05:46.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.016+0000 7fc70affd700 1 -- 172.21.15.55:0/386292366 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc71c003040 con 0x7fc7240c9b60 2023-10-06T05:05:46.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.017+0000 7fc70affd700 1 --2- 172.21.15.55:0/386292366 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc710066f30 0x7fc7100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:46.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.017+0000 7fc70affd700 1 -- 172.21.15.55:0/386292366 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fc71c085fe0 con 0x7fc7240c9b60 2023-10-06T05:05:46.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.017+0000 7fc722d9d700 1 --2- 172.21.15.55:0/386292366 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc710066f30 0x7fc7100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:46.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.018+0000 7fc722d9d700 1 --2- 172.21.15.55:0/386292366 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc710066f30 0x7fc7100693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc714006fd0 tx=0x7fc714008040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:46.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.021+0000 7fc70affd700 1 -- 172.21.15.55:0/386292366 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc71c052ef0 con 0x7fc7240c9b60 2023-10-06T05:05:46.304 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.302+0000 7fc7289fe700 1 -- 172.21.15.55:0/386292366 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc72404c3c0 con 0x7fc7240c9b60 2023-10-06T05:05:46.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.306+0000 7fc70affd700 1 -- 172.21.15.55:0/386292366 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v46) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc71c056510 con 0x7fc7240c9b60 2023-10-06T05:05:46.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.309+0000 7fc7289fe700 1 -- 172.21.15.55:0/386292366 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc710066f30 msgr2=0x7fc7100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:46.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.309+0000 7fc7289fe700 1 --2- 172.21.15.55:0/386292366 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc710066f30 0x7fc7100693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc714006fd0 tx=0x7fc714008040 comp rx=0 tx=0).stop 2023-10-06T05:05:46.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.309+0000 7fc7289fe700 1 -- 172.21.15.55:0/386292366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7240c9b60 msgr2=0x7fc7240c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:46.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.309+0000 7fc7289fe700 1 --2- 172.21.15.55:0/386292366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7240c9b60 0x7fc7240c5fd0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009880 tx=0x7fc71c018000 comp rx=0 tx=0).stop 2023-10-06T05:05:46.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.309+0000 7fc7289fe700 1 -- 172.21.15.55:0/386292366 shutdown_connections 2023-10-06T05:05:46.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.309+0000 7fc7289fe700 1 --2- 172.21.15.55:0/386292366 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc710066f30 0x7fc7100693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.309+0000 7fc7289fe700 1 --2- 172.21.15.55:0/386292366 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc7240c9b60 0x7fc7240c5fd0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.309+0000 7fc7289fe700 1 --2- 172.21.15.55:0/386292366 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc7240c6e80 0x7fc72415ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.309+0000 7fc7289fe700 1 --2- 172.21.15.55:0/386292366 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc7240c7250 0x7fc7240c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.309+0000 7fc7289fe700 1 -- 172.21.15.55:0/386292366 >> 172.21.15.55:0/386292366 conn(0x7fc7240cd0c0 msgr2=0x7fc724141f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:46.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.309+0000 7fc7289fe700 1 -- 172.21.15.55:0/386292366 shutdown_connections 2023-10-06T05:05:46.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.310+0000 7fc7289fe700 1 -- 172.21.15.55:0/386292366 wait complete. 2023-10-06T05:05:46.313 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2023-10-06T05:05:46.336 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:05:46.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.521+0000 7f41e7bff700 1 Processor -- start 2023-10-06T05:05:46.523 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.521+0000 7f41e7bff700 1 -- start start 2023-10-06T05:05:46.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.522+0000 7f41e7bff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f41e00afc40 0x7f41e00b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:46.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.522+0000 7f41e7bff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41e00b2140 0x7f41e015d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:46.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.522+0000 7f41e7bff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e00b0750 con 0x7f41e00b0010 2023-10-06T05:05:46.524 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.522+0000 7f41e7bff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e00aca70 con 0x7f41e00afc40 2023-10-06T05:05:46.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.522+0000 7f41e7bff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e00acd50 con 0x7f41e00b2140 2023-10-06T05:05:46.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.522+0000 7f41e599b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f41e00afc40 0x7f41e00b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:46.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.523+0000 7f41e599b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f41e00afc40 0x7f41e00b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45506/0 (socket says 172.21.15.55:45506) 2023-10-06T05:05:46.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.523+0000 7f41e599b700 1 -- 172.21.15.55:0/302029248 learned_addr learned my addr 172.21.15.55:0/302029248 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:46.525 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.523+0000 7f41e519a700 1 --2- 172.21.15.55:0/302029248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41e00b2140 0x7f41e015d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:46.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.523+0000 7f41e599b700 1 -- 172.21.15.55:0/302029248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41e00b2140 msgr2=0x7f41e015d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:46.526 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.523+0000 7f41e599b700 1 --2- 172.21.15.55:0/302029248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41e00b2140 0x7f41e015d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.523+0000 7f41e599b700 1 -- 172.21.15.55:0/302029248 >> v1:172.21.15.163:6789/0 conn(0x7f41e00b0010 legacy=0x7f41e015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.523+0000 7f41e599b700 1 -- 172.21.15.55:0/302029248 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e00ad030 con 0x7f41e00afc40 2023-10-06T05:05:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.523+0000 7f41e599b700 1 --2- 172.21.15.55:0/302029248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00afc40 0x7f41e00b1c00 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0099b0 tx=0x7f41dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757949ab05c38d2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.523+0000 7f41e4999700 1 -- 172.21.15.55:0/302029248 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc009ea0 con 0x7f41e00afc40 2023-10-06T05:05:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.524+0000 7f41e4999700 1 -- 172.21.15.55:0/302029248 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc00b860 con 0x7f41e00afc40 2023-10-06T05:05:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.524+0000 7f41e4999700 1 -- 172.21.15.55:0/302029248 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc00bc00 con 0x7f41e00afc40 2023-10-06T05:05:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.524+0000 7f41e7bff700 1 -- 172.21.15.55:0/302029248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00afc40 msgr2=0x7f41e00b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.524+0000 7f41e7bff700 1 --2- 172.21.15.55:0/302029248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00afc40 0x7f41e00b1c00 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0099b0 tx=0x7f41dc009250 comp rx=0 tx=0).stop 2023-10-06T05:05:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.524+0000 7f41e7bff700 1 -- 172.21.15.55:0/302029248 shutdown_connections 2023-10-06T05:05:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.524+0000 7f41e7bff700 1 --2- 172.21.15.55:0/302029248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00afc40 0x7f41e00b1c00 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.524+0000 7f41e7bff700 1 --2- 172.21.15.55:0/302029248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41e00b2140 0x7f41e015d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.527 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.524+0000 7f41e7bff700 1 -- 172.21.15.55:0/302029248 >> 172.21.15.55:0/302029248 conn(0x7f41e006a320 msgr2=0x7f41e00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:46.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.524+0000 7f41e7bff700 1 -- 172.21.15.55:0/302029248 shutdown_connections 2023-10-06T05:05:46.528 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.524+0000 7f41e7bff700 1 -- 172.21.15.55:0/302029248 wait complete. 2023-10-06T05:05:46.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.526+0000 7f41e7bff700 1 Processor -- start 2023-10-06T05:05:46.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.526+0000 7f41e7bff700 1 -- start start 2023-10-06T05:05:46.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e7bff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00afc40 0x7f41e015f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:46.529 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e7bff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41e00b0010 0x7f41e01621a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:46.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e7bff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41e00b2140 0x7f41e01626e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:46.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e7bff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e0151f50 con 0x7f41e00afc40 2023-10-06T05:05:46.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e7bff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e01606c0 con 0x7f41e00b0010 2023-10-06T05:05:46.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e7bff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e0162d80 con 0x7f41e00b2140 2023-10-06T05:05:46.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e599b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00afc40 0x7f41e015f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:46.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e599b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00afc40 0x7f41e015f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45522/0 (socket says 172.21.15.55:45522) 2023-10-06T05:05:46.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e599b700 1 -- 172.21.15.55:0/1436428380 learned_addr learned my addr 172.21.15.55:0/1436428380 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:46.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e599b700 1 -- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41e00b2140 msgr2=0x7f41e01626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:46.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e519a700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41e00b0010 0x7f41e01621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:46.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e619c700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41e00b2140 0x7f41e01626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:46.530 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e599b700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41e00b2140 0x7f41e01626e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e599b700 1 -- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41e00b0010 msgr2=0x7f41e01621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:46.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e599b700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41e00b0010 0x7f41e01621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e599b700 1 -- 172.21.15.55:0/1436428380 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc0094f0 con 0x7f41e00afc40 2023-10-06T05:05:46.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e519a700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41e00b0010 0x7f41e01621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:46.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.527+0000 7f41e619c700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41e00b2140 0x7f41e01626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:46.531 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.528+0000 7f41e599b700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00afc40 0x7f41e015f750 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f41e00695d0 tx=0x7f41dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:46.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.528+0000 7f41d5ffb700 1 -- 172.21.15.55:0/1436428380 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc00bbe0 con 0x7f41e00afc40 2023-10-06T05:05:46.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.528+0000 7f41d5ffb700 1 -- 172.21.15.55:0/1436428380 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc011420 con 0x7f41e00afc40 2023-10-06T05:05:46.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.528+0000 7f41d5ffb700 1 -- 172.21.15.55:0/1436428380 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0117c0 con 0x7f41e00afc40 2023-10-06T05:05:46.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.528+0000 7f41e7bff700 1 -- 172.21.15.55:0/1436428380 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e0160910 con 0x7f41e00afc40 2023-10-06T05:05:46.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.528+0000 7f41e7bff700 1 -- 172.21.15.55:0/1436428380 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e0160ed0 con 0x7f41e00afc40 2023-10-06T05:05:46.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.529+0000 7f41e7bff700 1 -- 172.21.15.55:0/1436428380 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e004c3c0 con 0x7f41e00afc40 2023-10-06T05:05:46.532 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.530+0000 7f41d5ffb700 1 -- 172.21.15.55:0/1436428380 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f41dc011bf0 con 0x7f41e00afc40 2023-10-06T05:05:46.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.531+0000 7f41d5ffb700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f41c4066f30 0x7f41c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:46.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.531+0000 7f41d5ffb700 1 -- 172.21.15.55:0/1436428380 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f41dc086280 con 0x7f41e00afc40 2023-10-06T05:05:46.533 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.531+0000 7f41e519a700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f41c4066f30 0x7f41c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:46.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.532+0000 7f41e519a700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f41c4066f30 0x7f41c40693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f41d0006f00 tx=0x7f41d0008040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:46.536 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.535+0000 7f41d5ffb700 1 -- 172.21.15.55:0/1436428380 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41dc053150 con 0x7f41e00afc40 2023-10-06T05:05:46.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.782+0000 7f41e7bff700 1 -- 172.21.15.55:0/1436428380 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41e00ad340 con 0x7f41e00afc40 2023-10-06T05:05:46.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.783+0000 7f41d5ffb700 1 -- 172.21.15.55:0/1436428380 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f41dc056770 con 0x7f41e00afc40 2023-10-06T05:05:46.785 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:46.785 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8078,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1255308005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1255308005},{"type":"v1","addr":"172.21.15.163:6835","nonce":1255308005}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":9274,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2468487159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2468487159},{"type":"v1","addr":"172.21.15.163:6834","nonce":2468487159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/4105265850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":4105265850},{"type":"v1","addr":"172.21.15.55:6834","nonce":4105265850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:05:40.405710+0000","modified":"2023-10-06T05:05:43.944287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9771":{"gid":9771,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/312761019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":312761019},{"type":"v1","addr":"172.21.15.55:6835","nonce":312761019}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:05:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.785+0000 7f41e7bff700 1 -- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f41c4066f30 msgr2=0x7f41c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.785+0000 7f41e7bff700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f41c4066f30 0x7f41c40693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f41d0006f00 tx=0x7f41d0008040 comp rx=0 tx=0).stop 2023-10-06T05:05:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.785+0000 7f41e7bff700 1 -- 172.21.15.55:0/1436428380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00afc40 msgr2=0x7f41e015f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.785+0000 7f41e7bff700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00afc40 0x7f41e015f750 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f41e00695d0 tx=0x7f41dc017000 comp rx=0 tx=0).stop 2023-10-06T05:05:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.785+0000 7f41e7bff700 1 -- 172.21.15.55:0/1436428380 shutdown_connections 2023-10-06T05:05:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.785+0000 7f41e7bff700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f41c4066f30 0x7f41c40693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.785+0000 7f41e7bff700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f41e00b2140 0x7f41e01626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.785+0000 7f41e7bff700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f41e00afc40 0x7f41e015f750 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.785+0000 7f41e7bff700 1 --2- 172.21.15.55:0/1436428380 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f41e00b0010 0x7f41e01621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.785+0000 7f41e7bff700 1 -- 172.21.15.55:0/1436428380 >> 172.21.15.55:0/1436428380 conn(0x7f41e006a320 msgr2=0x7f41e015c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.785+0000 7f41e7bff700 1 -- 172.21.15.55:0/1436428380 shutdown_connections 2023-10-06T05:05:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:46.785+0000 7f41e7bff700 1 -- 172.21.15.55:0/1436428380 wait complete. 2023-10-06T05:05:46.788 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 96 2023-10-06T05:05:46.889 INFO:tasks.ceph.mds.a.smithi055.stderr:2023-10-06T05:05:46.887+0000 7f969f1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:05:47.207 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:05:40.405710+0000', 'modified': '2023-10-06T05:05:43.944287+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9771}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9771': {'gid': 9771, 'name': 'a', 'rank': 0, 'incarnation': 92, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6835/312761019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6833', 'nonce': 312761019}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 312761019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:05:47.208 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:05:47.208 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:05:47.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.397+0000 7fbb94b9d700 1 Processor -- start 2023-10-06T05:05:47.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.397+0000 7fbb94b9d700 1 -- start start 2023-10-06T05:05:47.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.398+0000 7fbb94b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb90150180 0x7fbb90157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.398+0000 7fbb94b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb90150550 0x7fbb900c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.398+0000 7fbb94b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9006e2a0 con 0x7fbb900c7240 2023-10-06T05:05:47.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.398+0000 7fbb94b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb900c76c0 con 0x7fbb90150180 2023-10-06T05:05:47.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.398+0000 7fbb94b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9006b2e0 con 0x7fbb90150550 2023-10-06T05:05:47.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.398+0000 7fbb8f7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fbb90150180 0x7fbb90157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.398+0000 7fbb8ffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fbb900c7240 0x7fbb90069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55954/0 (socket says 172.21.15.55:55954) 2023-10-06T05:05:47.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.398+0000 7fbb8ffff700 1 -- 172.21.15.55:0/3790973994 learned_addr learned my addr 172.21.15.55:0/3790973994 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:47.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.399+0000 7fbb8effd700 1 --2- 172.21.15.55:0/3790973994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb90150550 0x7fbb900c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.399+0000 7fbb8f7fe700 1 -- 172.21.15.55:0/3790973994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb90150550 msgr2=0x7fbb900c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:47.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.399+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/3790973994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb90150550 0x7fbb900c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.399+0000 7fbb8f7fe700 1 -- 172.21.15.55:0/3790973994 >> v1:172.21.15.55:6789/0 conn(0x7fbb900c7240 legacy=0x7fbb90069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.399+0000 7fbb8f7fe700 1 -- 172.21.15.55:0/3790973994 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb900c78e0 con 0x7fbb90150180 2023-10-06T05:05:47.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.399+0000 7fbb8e7fc700 1 -- 172.21.15.55:0/3790973994 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346515454 0 0) 0x7fbb9006e2a0 con 0x7fbb900c7240 2023-10-06T05:05:47.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.399+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/3790973994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb90150180 0x7fbb90157770 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009910 tx=0x7fbb800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8398389ef301a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:47.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.399+0000 7fbb8e7fc700 1 -- 172.21.15.55:0/3790973994 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000b820 con 0x7fbb90150180 2023-10-06T05:05:47.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.399+0000 7fbb8e7fc700 1 -- 172.21.15.55:0/3790973994 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8000b980 con 0x7fbb90150180 2023-10-06T05:05:47.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.400+0000 7fbb8e7fc700 1 -- 172.21.15.55:0/3790973994 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8000bd20 con 0x7fbb90150180 2023-10-06T05:05:47.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.400+0000 7fbb94b9d700 1 -- 172.21.15.55:0/3790973994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb90150180 msgr2=0x7fbb90157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.400+0000 7fbb94b9d700 1 --2- 172.21.15.55:0/3790973994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb90150180 0x7fbb90157770 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009910 tx=0x7fbb800091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.400+0000 7fbb94b9d700 1 -- 172.21.15.55:0/3790973994 shutdown_connections 2023-10-06T05:05:47.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.400+0000 7fbb94b9d700 1 --2- 172.21.15.55:0/3790973994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb90150180 0x7fbb90157770 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.400+0000 7fbb94b9d700 1 --2- 172.21.15.55:0/3790973994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb90150550 0x7fbb900c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.400+0000 7fbb94b9d700 1 -- 172.21.15.55:0/3790973994 >> 172.21.15.55:0/3790973994 conn(0x7fbb900cc1f0 msgr2=0x7fbb900cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:47.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.400+0000 7fbb94b9d700 1 -- 172.21.15.55:0/3790973994 shutdown_connections 2023-10-06T05:05:47.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.400+0000 7fbb94b9d700 1 -- 172.21.15.55:0/3790973994 wait complete. 2023-10-06T05:05:47.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.402+0000 7fbb94b9d700 1 Processor -- start 2023-10-06T05:05:47.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.402+0000 7fbb94b9d700 1 -- start start 2023-10-06T05:05:47.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb94b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb900c7240 0x7fbb90148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb94b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb90150180 0x7fbb900c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb94b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb90150550 0x7fbb900c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb94b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb90149300 con 0x7fbb900c7240 2023-10-06T05:05:47.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb94b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb900abab0 con 0x7fbb90150180 2023-10-06T05:05:47.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb94b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb900c4dc0 con 0x7fbb90150550 2023-10-06T05:05:47.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb8f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb900c7240 0x7fbb90148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb8f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb900c7240 0x7fbb90148630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45534/0 (socket says 172.21.15.55:45534) 2023-10-06T05:05:47.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb8f7fe700 1 -- 172.21.15.55:0/4037537073 learned_addr learned my addr 172.21.15.55:0/4037537073 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:47.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb8effd700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb90150180 0x7fbb900c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb8ffff700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb90150550 0x7fbb900c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb8f7fe700 1 -- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb90150550 msgr2=0x7fbb900c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb90150550 0x7fbb900c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb8f7fe700 1 -- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb90150180 msgr2=0x7fbb900c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb90150180 0x7fbb900c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb8f7fe700 1 -- 172.21.15.55:0/4037537073 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb80009420 con 0x7fbb900c7240 2023-10-06T05:05:47.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb8effd700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb90150180 0x7fbb900c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:47.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.403+0000 7fbb8ffff700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb90150550 0x7fbb900c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:05:47.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.404+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb900c7240 0x7fbb90148630 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000b3c0 tx=0x7fbb80017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:47.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.404+0000 7fbb777fe700 1 -- 172.21.15.55:0/4037537073 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80024440 con 0x7fbb900c7240 2023-10-06T05:05:47.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.404+0000 7fbb777fe700 1 -- 172.21.15.55:0/4037537073 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb800245a0 con 0x7fbb900c7240 2023-10-06T05:05:47.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.404+0000 7fbb777fe700 1 -- 172.21.15.55:0/4037537073 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80024960 con 0x7fbb900c7240 2023-10-06T05:05:47.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.404+0000 7fbb94b9d700 1 -- 172.21.15.55:0/4037537073 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb900c50a0 con 0x7fbb900c7240 2023-10-06T05:05:47.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.404+0000 7fbb94b9d700 1 -- 172.21.15.55:0/4037537073 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb900abd00 con 0x7fbb900c7240 2023-10-06T05:05:47.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.406+0000 7fbb777fe700 1 -- 172.21.15.55:0/4037537073 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb80024d50 con 0x7fbb900c7240 2023-10-06T05:05:47.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.407+0000 7fbb777fe700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fbb7c066f30 0x7fbb7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.408+0000 7fbb777fe700 1 -- 172.21.15.55:0/4037537073 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fbb800864c0 con 0x7fbb900c7240 2023-10-06T05:05:47.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.408+0000 7fbb8effd700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fbb7c066f30 0x7fbb7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.409+0000 7fbb8effd700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fbb7c066f30 0x7fbb7c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009990 tx=0x7fbb84008040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:47.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.409+0000 7fbba453eb80 1 Processor -- start 2023-10-06T05:05:47.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.410+0000 7fbba453eb80 1 -- start start 2023-10-06T05:05:47.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.410+0000 7fbba453eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a239ee50 0x5636a2379cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.410+0000 7fbba453eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a237f2c0 0x5636a237a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.410+0000 7fbba453eb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a2380a10 0x5636a237a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.410+0000 7fbba453eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a22d4a70 con 0x5636a237f2c0 2023-10-06T05:05:47.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.410+0000 7fbba453eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a22d2270 con 0x5636a239ee50 2023-10-06T05:05:47.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.410+0000 7fbba453eb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636a22ce060 con 0x5636a2380a10 2023-10-06T05:05:47.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.411+0000 7fbb8effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a237f2c0 0x5636a237a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.411+0000 7fbb8effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a237f2c0 0x5636a237a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45536/0 (socket says 172.21.15.55:45536) 2023-10-06T05:05:47.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.411+0000 7fbb8effd700 1 -- 172.21.15.55:0/4117537191 learned_addr learned my addr 172.21.15.55:0/4117537191 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:47.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.411+0000 7fbb8ffff700 1 --2- 172.21.15.55:0/4117537191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a2380a10 0x5636a237a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.411+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/4117537191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a239ee50 0x5636a2379cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.411+0000 7fbb8effd700 1 -- 172.21.15.55:0/4117537191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a2380a10 msgr2=0x5636a237a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:47.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.411+0000 7fbb8effd700 1 --2- 172.21.15.55:0/4117537191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a2380a10 0x5636a237a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.411+0000 7fbb8effd700 1 -- 172.21.15.55:0/4117537191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a239ee50 msgr2=0x5636a2379cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:47.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.411+0000 7fbb8effd700 1 --2- 172.21.15.55:0/4117537191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a239ee50 0x5636a2379cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.411+0000 7fbb8effd700 1 -- 172.21.15.55:0/4117537191 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636a2168270 con 0x5636a237f2c0 2023-10-06T05:05:47.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.411+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/4117537191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a239ee50 0x5636a2379cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:47.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.411+0000 7fbb8effd700 1 --2- 172.21.15.55:0/4117537191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a237f2c0 0x5636a237a200 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400e990 tx=0x7fbb84013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca6295f0f36a2ce server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:47.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.412+0000 7fbb74ff9700 1 -- 172.21.15.55:0/4117537191 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8401d070 con 0x5636a237f2c0 2023-10-06T05:05:47.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.412+0000 7fbb74ff9700 1 -- 172.21.15.55:0/4117537191 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb840181a0 con 0x5636a237f2c0 2023-10-06T05:05:47.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.412+0000 7fbb74ff9700 1 -- 172.21.15.55:0/4117537191 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84018840 con 0x5636a237f2c0 2023-10-06T05:05:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.412+0000 7fbba453eb80 1 -- 172.21.15.55:0/4117537191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a237f2c0 msgr2=0x5636a237a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.412+0000 7fbba453eb80 1 --2- 172.21.15.55:0/4117537191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a237f2c0 0x5636a237a200 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400e990 tx=0x7fbb84013000 comp rx=0 tx=0).stop 2023-10-06T05:05:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.412+0000 7fbba453eb80 1 -- 172.21.15.55:0/4117537191 shutdown_connections 2023-10-06T05:05:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.412+0000 7fbba453eb80 1 --2- 172.21.15.55:0/4117537191 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a2380a10 0x5636a237a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.412+0000 7fbba453eb80 1 --2- 172.21.15.55:0/4117537191 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a239ee50 0x5636a2379cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.412+0000 7fbba453eb80 1 --2- 172.21.15.55:0/4117537191 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a237f2c0 0x5636a237a200 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.412+0000 7fbba453eb80 1 -- 172.21.15.55:0/4117537191 >> 172.21.15.55:0/4117537191 conn(0x5636a237ee80 msgr2=0x5636a237f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:47.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.412+0000 7fbba453eb80 1 -- 172.21.15.55:0/4117537191 shutdown_connections 2023-10-06T05:05:47.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.412+0000 7fbba453eb80 1 -- 172.21.15.55:0/4117537191 wait complete. 2023-10-06T05:05:47.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.413+0000 7fbba453eb80 1 -- start start 2023-10-06T05:05:47.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.413+0000 7fbba453eb80 1 Processor -- start 2023-10-06T05:05:47.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.414+0000 7fbba453eb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 0x5636a23c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.414+0000 7fbba453eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a2380a10 0x5636a23a8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.414+0000 7fbba453eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a239ee50 0x5636a23ac290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.414+0000 7fbba453eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a2279890 con 0x5636a239ee50 2023-10-06T05:05:47.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.414+0000 7fbba453eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a2294ea0 con 0x5636a2380a10 2023-10-06T05:05:47.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.414+0000 7fbba453eb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636a2150e30 con 0x5636a237f2c0 2023-10-06T05:05:47.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.414+0000 7fbba453eb80 10 client.0 _ll_register_callbacks cb 0x5636a2341020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:05:47.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.415+0000 7fbb8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a239ee50 0x5636a23ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.415+0000 7fbb8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a239ee50 0x5636a23ac290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45540/0 (socket says 172.21.15.55:45540) 2023-10-06T05:05:47.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.415+0000 7fbb8ffff700 1 -- 172.21.15.55:0/1119928261 learned_addr learned my addr 172.21.15.55:0/1119928261 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:47.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.415+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/1119928261 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 0x5636a23c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.415+0000 7fbb8effd700 1 --2- 172.21.15.55:0/1119928261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a2380a10 0x5636a23a8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.415+0000 7fbb8ffff700 1 -- 172.21.15.55:0/1119928261 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 msgr2=0x5636a23c3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:47.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.415+0000 7fbb8ffff700 1 --2- 172.21.15.55:0/1119928261 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 0x5636a23c3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.415+0000 7fbb8ffff700 1 -- 172.21.15.55:0/1119928261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a2380a10 msgr2=0x5636a23a8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:47.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.415+0000 7fbb8ffff700 1 --2- 172.21.15.55:0/1119928261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a2380a10 0x5636a23a8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.415+0000 7fbb8ffff700 1 -- 172.21.15.55:0/1119928261 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8400e670 con 0x5636a239ee50 2023-10-06T05:05:47.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.415+0000 7fbb8effd700 1 --2- 172.21.15.55:0/1119928261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a2380a10 0x5636a23a8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:47.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.415+0000 7fbb8ffff700 1 --2- 172.21.15.55:0/1119928261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a239ee50 0x5636a23ac290 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fbb88010df0 tx=0x7fbb88009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da068d46e0339ba7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:47.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.415+0000 7fbb6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T05:05:47.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.416+0000 7fbb6dffb700 1 -- 172.21.15.55:0/1119928261 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88001fd0 con 0x5636a239ee50 2023-10-06T05:05:47.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.416+0000 7fbba453eb80 1 -- 172.21.15.55:0/1119928261 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636a2372940 con 0x5636a239ee50 2023-10-06T05:05:47.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.416+0000 7fbb6dffb700 1 -- 172.21.15.55:0/1119928261 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fbb8800ea40 con 0x5636a239ee50 2023-10-06T05:05:47.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.416+0000 7fbb6dffb700 1 -- 172.21.15.55:0/1119928261 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb8800ee10 con 0x5636a239ee50 2023-10-06T05:05:47.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.416+0000 7fbba453eb80 10 client.9906 fetch_fsmap learned FSMap version 96 2023-10-06T05:05:47.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.416+0000 7fbba453eb80 1 -- 172.21.15.55:0/1119928261 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x5636a210afe0 con 0x5636a239ee50 2023-10-06T05:05:47.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.416+0000 7fbb6dffb700 1 -- 172.21.15.55:0/1119928261 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5636a2372940 con 0x5636a239ee50 2023-10-06T05:05:47.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.416+0000 7fbb6dffb700 1 -- 172.21.15.55:0/1119928261 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 96) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fbb88012040 con 0x5636a239ee50 2023-10-06T05:05:47.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.416+0000 7fbba453eb80 10 client.9906 fetch_fsmap finished waiting for FSMap version 96 2023-10-06T05:05:47.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.416+0000 7fbba453eb80 10 client.9906 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:05:47.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.416+0000 7fbba453eb80 20 client.9906 populate_metadata read hostname 'smithi055' 2023-10-06T05:05:47.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.416+0000 7fbba453eb80 1 --2- 172.21.15.55:0/1119928261 >> [v2:172.21.15.55:6833/312761019,v1:172.21.15.55:6835/312761019] conn(0x5636a2398370 0x5636a2398740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.416+0000 7fbba453eb80 4 client.9906 mds_command: new command op to 9771 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:05:47.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.416+0000 7fbba453eb80 1 -- 172.21.15.55:0/1119928261 --> [v2:172.21.15.55:6833/312761019,v1:172.21.15.55:6835/312761019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636a234f870 con 0x5636a2398370 2023-10-06T05:05:47.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.417+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/1119928261 >> [v2:172.21.15.55:6833/312761019,v1:172.21.15.55:6835/312761019] conn(0x5636a2398370 0x5636a2398740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.417+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/1119928261 >> [v2:172.21.15.55:6833/312761019,v1:172.21.15.55:6835/312761019] conn(0x5636a2398370 0x5636a2398740 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30ede1935f3264a0 server_cookie=4301ea8ad993f6dd in_seq=0 out_seq=0 2023-10-06T05:05:47.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.417+0000 7fbb6dffb700 10 client.9906 ms_handle_connect on v2:172.21.15.55:6833/312761019 2023-10-06T05:05:47.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbb6dffb700 1 -- 172.21.15.55:0/1119928261 <== mds.0 v2:172.21.15.55:6833/312761019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5636a234f870 con 0x5636a2398370 2023-10-06T05:05:47.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbb6dffb700 10 client.9906 handle_command_reply: tid=0 2023-10-06T05:05:47.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbba453eb80 1 client.9906 shutdown 2023-10-06T05:05:47.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbba453eb80 1 -- 172.21.15.55:0/1119928261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a239ee50 msgr2=0x5636a23ac290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbba453eb80 1 --2- 172.21.15.55:0/1119928261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a239ee50 0x5636a23ac290 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fbb88010df0 tx=0x7fbb88009040 comp rx=0 tx=0).stop 2023-10-06T05:05:47.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbba453eb80 1 -- 172.21.15.55:0/1119928261 shutdown_connections 2023-10-06T05:05:47.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbba453eb80 1 --2- 172.21.15.55:0/1119928261 >> [v2:172.21.15.55:6833/312761019,v1:172.21.15.55:6835/312761019] conn(0x5636a2398370 0x5636a2398740 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbba453eb80 1 --2- 172.21.15.55:0/1119928261 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a239ee50 0x5636a23ac290 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbba453eb80 1 --2- 172.21.15.55:0/1119928261 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 0x5636a23c3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbb6dffb700 0 client.9906 ms_handle_reset on v2:172.21.15.55:6833/312761019 2023-10-06T05:05:47.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbba453eb80 1 --2- 172.21.15.55:0/1119928261 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a2380a10 0x5636a23a8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbba453eb80 1 -- 172.21.15.55:0/1119928261 >> 172.21.15.55:0/1119928261 conn(0x5636a237ee80 msgr2=0x5636a237fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:47.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbba453eb80 1 -- 172.21.15.55:0/1119928261 shutdown_connections 2023-10-06T05:05:47.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbba453eb80 1 -- 172.21.15.55:0/1119928261 wait complete. 2023-10-06T05:05:47.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.418+0000 7fbba453eb80 20 client.9906 trim_cache size 0 max 16384 2023-10-06T05:05:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.436+0000 7fbba453eb80 1 Processor -- start 2023-10-06T05:05:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.436+0000 7fbba453eb80 1 -- start start 2023-10-06T05:05:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbba453eb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 0x5636a2420a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbba453eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a2380a10 0x5636a237d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbba453eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a239ee50 0x5636a237de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbba453eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a237ed40 con 0x5636a239ee50 2023-10-06T05:05:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbba453eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a2393fe0 con 0x5636a2380a10 2023-10-06T05:05:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbba453eb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636a23dcce0 con 0x5636a237f2c0 2023-10-06T05:05:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbb8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a239ee50 0x5636a237de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbb8ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a239ee50 0x5636a237de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45554/0 (socket says 172.21.15.55:45554) 2023-10-06T05:05:47.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbb8ffff700 1 -- 172.21.15.55:0/1573295798 learned_addr learned my addr 172.21.15.55:0/1573295798 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/1573295798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 0x5636a2420a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbb8effd700 1 --2- 172.21.15.55:0/1573295798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a2380a10 0x5636a237d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbb8ffff700 1 -- 172.21.15.55:0/1573295798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 msgr2=0x5636a2420a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbb8ffff700 1 --2- 172.21.15.55:0/1573295798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 0x5636a2420a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbb8ffff700 1 -- 172.21.15.55:0/1573295798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a2380a10 msgr2=0x5636a237d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbb8ffff700 1 --2- 172.21.15.55:0/1573295798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a2380a10 0x5636a237d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbb8ffff700 1 -- 172.21.15.55:0/1573295798 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636a229c780 con 0x5636a239ee50 2023-10-06T05:05:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.437+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/1573295798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 0x5636a2420a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.438+0000 7fbb8ffff700 1 --2- 172.21.15.55:0/1573295798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a239ee50 0x5636a237de40 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fbb88001cc0 tx=0x7fbb8801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce76650fc3f3893f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.438+0000 7fbb757fa700 1 -- 172.21.15.55:0/1573295798 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8800e830 con 0x5636a239ee50 2023-10-06T05:05:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.438+0000 7fbb757fa700 1 -- 172.21.15.55:0/1573295798 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8800d050 con 0x5636a239ee50 2023-10-06T05:05:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.438+0000 7fbb757fa700 1 -- 172.21.15.55:0/1573295798 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8800ea40 con 0x5636a239ee50 2023-10-06T05:05:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.438+0000 7fbba453eb80 1 -- 172.21.15.55:0/1573295798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a239ee50 msgr2=0x5636a237de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.438+0000 7fbba453eb80 1 --2- 172.21.15.55:0/1573295798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a239ee50 0x5636a237de40 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fbb88001cc0 tx=0x7fbb8801f000 comp rx=0 tx=0).stop 2023-10-06T05:05:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.438+0000 7fbba453eb80 1 -- 172.21.15.55:0/1573295798 shutdown_connections 2023-10-06T05:05:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.438+0000 7fbba453eb80 1 --2- 172.21.15.55:0/1573295798 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a239ee50 0x5636a237de40 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.438+0000 7fbba453eb80 1 --2- 172.21.15.55:0/1573295798 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 0x5636a2420a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.438+0000 7fbba453eb80 1 --2- 172.21.15.55:0/1573295798 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a2380a10 0x5636a237d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.439+0000 7fbba453eb80 1 -- 172.21.15.55:0/1573295798 >> 172.21.15.55:0/1573295798 conn(0x5636a237ee80 msgr2=0x5636a2374400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.439+0000 7fbba453eb80 1 -- 172.21.15.55:0/1573295798 shutdown_connections 2023-10-06T05:05:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.439+0000 7fbba453eb80 1 -- 172.21.15.55:0/1573295798 wait complete. 2023-10-06T05:05:47.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.439+0000 7fbba453eb80 1 -- start start 2023-10-06T05:05:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.439+0000 7fbba453eb80 1 Processor -- start 2023-10-06T05:05:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.440+0000 7fbba453eb80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 0x5636a241ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.440+0000 7fbba453eb80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a2380a10 0x5636a23acee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.440+0000 7fbba453eb80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a239ee50 0x5636a23ad420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.440+0000 7fbba453eb80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a23811f0 con 0x5636a2380a10 2023-10-06T05:05:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.440+0000 7fbba453eb80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636a237e6d0 con 0x5636a239ee50 2023-10-06T05:05:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.440+0000 7fbba453eb80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636a242aa40 con 0x5636a237f2c0 2023-10-06T05:05:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.440+0000 7fbba453eb80 10 client.0 _ll_register_callbacks cb 0x5636a2341020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:05:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.441+0000 7fbb8effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a2380a10 0x5636a23acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.441+0000 7fbb8effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a2380a10 0x5636a23acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45558/0 (socket says 172.21.15.55:45558) 2023-10-06T05:05:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.441+0000 7fbb8effd700 1 -- 172.21.15.55:0/3340269110 learned_addr learned my addr 172.21.15.55:0/3340269110 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:47.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.441+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/3340269110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 0x5636a241ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.441+0000 7fbb8ffff700 1 --2- 172.21.15.55:0/3340269110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a239ee50 0x5636a23ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.441+0000 7fbb8effd700 1 -- 172.21.15.55:0/3340269110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 msgr2=0x5636a241ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.441+0000 7fbb8effd700 1 --2- 172.21.15.55:0/3340269110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 0x5636a241ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.441+0000 7fbb8effd700 1 -- 172.21.15.55:0/3340269110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a239ee50 msgr2=0x5636a23ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.441+0000 7fbb8effd700 1 --2- 172.21.15.55:0/3340269110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a239ee50 0x5636a23ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.441+0000 7fbb8effd700 1 -- 172.21.15.55:0/3340269110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636a210afe0 con 0x5636a2380a10 2023-10-06T05:05:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.441+0000 7fbb8ffff700 1 --2- 172.21.15.55:0/3340269110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a239ee50 0x5636a23ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.441+0000 7fbb8effd700 1 --2- 172.21.15.55:0/3340269110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a2380a10 0x5636a23acee0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fbb840030a0 tx=0x7fbb84018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b2840275291fc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.441+0000 7fbb6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T05:05:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.442+0000 7fbb6dffb700 1 -- 172.21.15.55:0/3340269110 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb84005de0 con 0x5636a2380a10 2023-10-06T05:05:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.442+0000 7fbb6dffb700 1 -- 172.21.15.55:0/3340269110 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fbb840064a0 con 0x5636a2380a10 2023-10-06T05:05:47.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.442+0000 7fbba453eb80 1 -- 172.21.15.55:0/3340269110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb900c1a50 con 0x5636a2380a10 2023-10-06T05:05:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.442+0000 7fbb6dffb700 1 -- 172.21.15.55:0/3340269110 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb84006870 con 0x5636a2380a10 2023-10-06T05:05:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.442+0000 7fbba453eb80 10 client.9912 fetch_fsmap learned FSMap version 96 2023-10-06T05:05:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.442+0000 7fbba453eb80 1 -- 172.21.15.55:0/3340269110 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x5636a2283470 con 0x5636a2380a10 2023-10-06T05:05:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.442+0000 7fbb6dffb700 1 -- 172.21.15.55:0/3340269110 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb900c1a50 con 0x5636a2380a10 2023-10-06T05:05:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.442+0000 7fbb6dffb700 1 -- 172.21.15.55:0/3340269110 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 96) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fbb84029040 con 0x5636a2380a10 2023-10-06T05:05:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.442+0000 7fbba453eb80 10 client.9912 fetch_fsmap finished waiting for FSMap version 96 2023-10-06T05:05:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.442+0000 7fbba453eb80 10 client.9912 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:05:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.442+0000 7fbba453eb80 20 client.9912 populate_metadata read hostname 'smithi055' 2023-10-06T05:05:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.442+0000 7fbba453eb80 1 --2- 172.21.15.55:0/3340269110 >> [v2:172.21.15.55:6833/312761019,v1:172.21.15.55:6835/312761019] conn(0x5636a23c2a50 0x5636a23c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.442+0000 7fbba453eb80 4 client.9912 mds_command: new command op to 9771 tid=0[{"prefix": "status"}] 2023-10-06T05:05:47.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.442+0000 7fbba453eb80 1 -- 172.21.15.55:0/3340269110 --> [v2:172.21.15.55:6833/312761019,v1:172.21.15.55:6835/312761019] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fbb840181a0 con 0x5636a23c2a50 2023-10-06T05:05:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.443+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/3340269110 >> [v2:172.21.15.55:6833/312761019,v1:172.21.15.55:6835/312761019] conn(0x5636a23c2a50 0x5636a23c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.443+0000 7fbb8f7fe700 1 --2- 172.21.15.55:0/3340269110 >> [v2:172.21.15.55:6833/312761019,v1:172.21.15.55:6835/312761019] conn(0x5636a23c2a50 0x5636a23c2e20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e22591fc98b9100 server_cookie=a3474e36afa533ed in_seq=0 out_seq=0 2023-10-06T05:05:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.443+0000 7fbb6dffb700 10 client.9912 ms_handle_connect on v2:172.21.15.55:6833/312761019 2023-10-06T05:05:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.443+0000 7fbb6dffb700 1 -- 172.21.15.55:0/3340269110 <== mds.0 v2:172.21.15.55:6833/312761019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7fbb840181a0 con 0x5636a23c2a50 2023-10-06T05:05:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.443+0000 7fbb6dffb700 10 client.9912 handle_command_reply: tid=0 2023-10-06T05:05:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.443+0000 7fbba453eb80 1 client.9912 shutdown 2023-10-06T05:05:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.443+0000 7fbba453eb80 1 -- 172.21.15.55:0/3340269110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a2380a10 msgr2=0x5636a23acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.443+0000 7fbba453eb80 1 --2- 172.21.15.55:0/3340269110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a2380a10 0x5636a23acee0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fbb840030a0 tx=0x7fbb84018000 comp rx=0 tx=0).stop 2023-10-06T05:05:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.443+0000 7fbba453eb80 1 -- 172.21.15.55:0/3340269110 shutdown_connections 2023-10-06T05:05:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.443+0000 7fbba453eb80 1 --2- 172.21.15.55:0/3340269110 >> [v2:172.21.15.55:6833/312761019,v1:172.21.15.55:6835/312761019] conn(0x5636a23c2a50 0x5636a23c2e20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.443+0000 7fbba453eb80 1 --2- 172.21.15.55:0/3340269110 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x5636a239ee50 0x5636a23ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.443+0000 7fbba453eb80 1 --2- 172.21.15.55:0/3340269110 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x5636a237f2c0 0x5636a241ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.443+0000 7fbba453eb80 1 --2- 172.21.15.55:0/3340269110 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x5636a2380a10 0x5636a23acee0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.443+0000 7fbba453eb80 1 -- 172.21.15.55:0/3340269110 >> 172.21.15.55:0/3340269110 conn(0x5636a237ee80 msgr2=0x5636a237d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:47.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.444+0000 7fbb6dffb700 0 client.9912 ms_handle_reset on v2:172.21.15.55:6833/312761019 2023-10-06T05:05:47.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.444+0000 7fbba453eb80 1 -- 172.21.15.55:0/3340269110 shutdown_connections 2023-10-06T05:05:47.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.444+0000 7fbba453eb80 1 -- 172.21.15.55:0/3340269110 wait complete. 2023-10-06T05:05:47.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.444+0000 7fbba453eb80 20 client.9912 trim_cache size 0 max 16384 2023-10-06T05:05:47.453 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T05:05:47.453 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "8132c8ba-ee1c-41a2-b781-913925034ab6", 2023-10-06T05:05:47.453 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2023-10-06T05:05:47.453 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 9771, 2023-10-06T05:05:47.453 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2023-10-06T05:05:47.453 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2023-10-06T05:05:47.453 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2023-10-06T05:05:47.453 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 6.5378674129999998, 2023-10-06T05:05:47.454 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 96, 2023-10-06T05:05:47.454 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 193, 2023-10-06T05:05:47.454 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 193, 2023-10-06T05:05:47.454 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 12.283187863 2023-10-06T05:05:47.454 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T05:05:47.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.444+0000 7fbb94b9d700 1 -- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fbb7c066f30 msgr2=0x7fbb7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.444+0000 7fbb94b9d700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fbb7c066f30 0x7fbb7c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009990 tx=0x7fbb84008040 comp rx=0 tx=0).stop 2023-10-06T05:05:47.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.444+0000 7fbb94b9d700 1 -- 172.21.15.55:0/4037537073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb900c7240 msgr2=0x7fbb90148630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.444+0000 7fbb94b9d700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb900c7240 0x7fbb90148630 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000b3c0 tx=0x7fbb80017000 comp rx=0 tx=0).stop 2023-10-06T05:05:47.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.445+0000 7fbb94b9d700 1 -- 172.21.15.55:0/4037537073 shutdown_connections 2023-10-06T05:05:47.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.445+0000 7fbb94b9d700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fbb7c066f30 0x7fbb7c0693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.445+0000 7fbb94b9d700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fbb90150550 0x7fbb900c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.445+0000 7fbb94b9d700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fbb900c7240 0x7fbb90148630 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.445+0000 7fbb94b9d700 1 --2- 172.21.15.55:0/4037537073 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fbb90150180 0x7fbb900c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.445+0000 7fbb94b9d700 1 -- 172.21.15.55:0/4037537073 >> 172.21.15.55:0/4037537073 conn(0x7fbb900cc1f0 msgr2=0x7fbb90157a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:47.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.445+0000 7fbb94b9d700 1 -- 172.21.15.55:0/4037537073 shutdown_connections 2023-10-06T05:05:47.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.445+0000 7fbb94b9d700 1 -- 172.21.15.55:0/4037537073 wait complete. 2023-10-06T05:05:47.464 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:05:47.650 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.648+0000 7fb5c759e700 1 Processor -- start 2023-10-06T05:05:47.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.648+0000 7fb5c759e700 1 -- start start 2023-10-06T05:05:47.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.650+0000 7fb5c759e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb5c80c6c60 0x7fb5c80c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.650+0000 7fb5c759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5c806e8c0 0x7fb5c80c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.650+0000 7fb5c759e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c80b1410 con 0x7fb5c81595a0 2023-10-06T05:05:47.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.650+0000 7fb5c759e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c80b2b90 con 0x7fb5c80c6c60 2023-10-06T05:05:47.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.650+0000 7fb5c759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c8063230 con 0x7fb5c806e8c0 2023-10-06T05:05:47.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.650+0000 7fb5c6d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5c806e8c0 0x7fb5c80c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.650+0000 7fb5c6d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5c806e8c0 0x7fb5c80c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:59508/0 (socket says 172.21.15.55:59508) 2023-10-06T05:05:47.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.651+0000 7fb5c6d9d700 1 -- 172.21.15.55:0/1643750522 learned_addr learned my addr 172.21.15.55:0/1643750522 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:47.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.651+0000 7fb5c5d9b700 1 --2- 172.21.15.55:0/1643750522 >> v2:172.21.15.163:3300/0 conn(0x7fb5c80c6c60 0x7fb5c80c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.651+0000 7fb5c6d9d700 1 -- 172.21.15.55:0/1643750522 >> v2:172.21.15.163:3300/0 conn(0x7fb5c80c6c60 msgr2=0x7fb5c80c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:47.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.651+0000 7fb5c6d9d700 1 --2- 172.21.15.55:0/1643750522 >> v2:172.21.15.163:3300/0 conn(0x7fb5c80c6c60 0x7fb5c80c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.651+0000 7fb5c6d9d700 1 -- 172.21.15.55:0/1643750522 >> v1:172.21.15.163:6789/0 conn(0x7fb5c81595a0 legacy=0x7fb5c80b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.651+0000 7fb5c6d9d700 1 -- 172.21.15.55:0/1643750522 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c80634f0 con 0x7fb5c806e8c0 2023-10-06T05:05:47.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.651+0000 7fb5c559a700 1 -- 172.21.15.55:0/1643750522 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469926625 0 0) 0x7fb5c80b1410 con 0x7fb5c81595a0 2023-10-06T05:05:47.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.651+0000 7fb5c6d9d700 1 --2- 172.21.15.55:0/1643750522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5c806e8c0 0x7fb5c80c8660 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009f20 tx=0x7fb5c0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24be6d3390ed9460 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:47.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.652+0000 7fb5c559a700 1 -- 172.21.15.55:0/1643750522 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000e070 con 0x7fb5c806e8c0 2023-10-06T05:05:47.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.652+0000 7fb5c559a700 1 -- 172.21.15.55:0/1643750522 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c000b9d0 con 0x7fb5c806e8c0 2023-10-06T05:05:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.652+0000 7fb5c559a700 1 -- 172.21.15.55:0/1643750522 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00092e0 con 0x7fb5c806e8c0 2023-10-06T05:05:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.652+0000 7fb5c759e700 1 -- 172.21.15.55:0/1643750522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5c806e8c0 msgr2=0x7fb5c80c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.652+0000 7fb5c759e700 1 --2- 172.21.15.55:0/1643750522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5c806e8c0 0x7fb5c80c8660 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009f20 tx=0x7fb5c0009840 comp rx=0 tx=0).stop 2023-10-06T05:05:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.652+0000 7fb5c759e700 1 -- 172.21.15.55:0/1643750522 shutdown_connections 2023-10-06T05:05:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.652+0000 7fb5c759e700 1 --2- 172.21.15.55:0/1643750522 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5c806e8c0 0x7fb5c80c8660 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.652+0000 7fb5c759e700 1 --2- 172.21.15.55:0/1643750522 >> v2:172.21.15.163:3300/0 conn(0x7fb5c80c6c60 0x7fb5c80c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.652+0000 7fb5c759e700 1 -- 172.21.15.55:0/1643750522 >> 172.21.15.55:0/1643750522 conn(0x7fb5c806ddb0 msgr2=0x7fb5c806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:47.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.652+0000 7fb5c759e700 1 -- 172.21.15.55:0/1643750522 shutdown_connections 2023-10-06T05:05:47.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.652+0000 7fb5c759e700 1 -- 172.21.15.55:0/1643750522 wait complete. 2023-10-06T05:05:47.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.654+0000 7fb5c759e700 1 Processor -- start 2023-10-06T05:05:47.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.654+0000 7fb5c759e700 1 -- start start 2023-10-06T05:05:47.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.655+0000 7fb5c759e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5c806e8c0 0x7fb5c816a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.655+0000 7fb5c759e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5c80c6c60 0x7fb5c80aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.655+0000 7fb5c759e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5c81595a0 0x7fb5c80c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.655+0000 7fb5c759e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80c3ba0 con 0x7fb5c81595a0 2023-10-06T05:05:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.655+0000 7fb5c759e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c816a470 con 0x7fb5c80c6c60 2023-10-06T05:05:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.655+0000 7fb5c759e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80c3db0 con 0x7fb5c806e8c0 2023-10-06T05:05:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.655+0000 7fb5c6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5c81595a0 0x7fb5c80c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.655+0000 7fb5c6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5c81595a0 0x7fb5c80c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45572/0 (socket says 172.21.15.55:45572) 2023-10-06T05:05:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.655+0000 7fb5c6d9d700 1 -- 172.21.15.55:0/2685282532 learned_addr learned my addr 172.21.15.55:0/2685282532 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:47.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.655+0000 7fb5c659c700 1 --2- 172.21.15.55:0/2685282532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5c806e8c0 0x7fb5c816a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.655+0000 7fb5c5d9b700 1 --2- 172.21.15.55:0/2685282532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5c80c6c60 0x7fb5c80aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.655+0000 7fb5c6d9d700 1 -- 172.21.15.55:0/2685282532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5c806e8c0 msgr2=0x7fb5c816a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.655+0000 7fb5c6d9d700 1 --2- 172.21.15.55:0/2685282532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5c806e8c0 0x7fb5c816a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.656+0000 7fb5c6d9d700 1 -- 172.21.15.55:0/2685282532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5c80c6c60 msgr2=0x7fb5c80aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.656+0000 7fb5c6d9d700 1 --2- 172.21.15.55:0/2685282532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5c80c6c60 0x7fb5c80aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.656+0000 7fb5c6d9d700 1 -- 172.21.15.55:0/2685282532 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c0009ae0 con 0x7fb5c81595a0 2023-10-06T05:05:47.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.656+0000 7fb5c6d9d700 1 --2- 172.21.15.55:0/2685282532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5c81595a0 0x7fb5c80c3660 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000b610 tx=0x7fb5c0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:47.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.656+0000 7fb5b67fc700 1 -- 172.21.15.55:0/2685282532 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0008c30 con 0x7fb5c81595a0 2023-10-06T05:05:47.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.656+0000 7fb5b67fc700 1 -- 172.21.15.55:0/2685282532 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c0008d90 con 0x7fb5c81595a0 2023-10-06T05:05:47.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.656+0000 7fb5b67fc700 1 -- 172.21.15.55:0/2685282532 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0026690 con 0x7fb5c81595a0 2023-10-06T05:05:47.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.656+0000 7fb5c759e700 1 -- 172.21.15.55:0/2685282532 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c80c4090 con 0x7fb5c81595a0 2023-10-06T05:05:47.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.656+0000 7fb5c759e700 1 -- 172.21.15.55:0/2685282532 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c816a100 con 0x7fb5c81595a0 2023-10-06T05:05:47.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.657+0000 7fb5c759e700 1 -- 172.21.15.55:0/2685282532 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c804c3c0 con 0x7fb5c81595a0 2023-10-06T05:05:47.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.658+0000 7fb5b67fc700 1 -- 172.21.15.55:0/2685282532 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5c0003040 con 0x7fb5c81595a0 2023-10-06T05:05:47.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.659+0000 7fb5b67fc700 1 --2- 172.21.15.55:0/2685282532 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb5a8066f30 0x7fb5a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:47.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.659+0000 7fb5b67fc700 1 -- 172.21.15.55:0/2685282532 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fb5c00867c0 con 0x7fb5c81595a0 2023-10-06T05:05:47.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.659+0000 7fb5c659c700 1 --2- 172.21.15.55:0/2685282532 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb5a8066f30 0x7fb5a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:47.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.660+0000 7fb5c659c700 1 --2- 172.21.15.55:0/2685282532 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb5a8066f30 0x7fb5a80693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8003820 tx=0x7fb5b8009040 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:47.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.664+0000 7fb5b67fc700 1 -- 172.21.15.55:0/2685282532 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5c0053730 con 0x7fb5c81595a0 2023-10-06T05:05:47.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.909+0000 7fb5c759e700 1 -- 172.21.15.55:0/2685282532 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb5c80637d0 con 0x7fb5c81595a0 2023-10-06T05:05:47.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.910+0000 7fb5b67fc700 1 -- 172.21.15.55:0/2685282532 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v96) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb5c0056d50 con 0x7fb5c81595a0 2023-10-06T05:05:47.911 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:05:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.912+0000 7fb5c759e700 1 -- 172.21.15.55:0/2685282532 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb5a8066f30 msgr2=0x7fb5a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.912+0000 7fb5c759e700 1 --2- 172.21.15.55:0/2685282532 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb5a8066f30 0x7fb5a80693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8003820 tx=0x7fb5b8009040 comp rx=0 tx=0).stop 2023-10-06T05:05:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.912+0000 7fb5c759e700 1 -- 172.21.15.55:0/2685282532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5c81595a0 msgr2=0x7fb5c80c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.912+0000 7fb5c759e700 1 --2- 172.21.15.55:0/2685282532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5c81595a0 0x7fb5c80c3660 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000b610 tx=0x7fb5c0018000 comp rx=0 tx=0).stop 2023-10-06T05:05:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.912+0000 7fb5c759e700 1 -- 172.21.15.55:0/2685282532 shutdown_connections 2023-10-06T05:05:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.912+0000 7fb5c759e700 1 --2- 172.21.15.55:0/2685282532 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb5a8066f30 0x7fb5a80693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.912+0000 7fb5c759e700 1 --2- 172.21.15.55:0/2685282532 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb5c81595a0 0x7fb5c80c3660 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.912+0000 7fb5c759e700 1 --2- 172.21.15.55:0/2685282532 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb5c806e8c0 0x7fb5c816a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.912+0000 7fb5c759e700 1 --2- 172.21.15.55:0/2685282532 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb5c80c6c60 0x7fb5c80aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:47.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.912+0000 7fb5c759e700 1 -- 172.21.15.55:0/2685282532 >> 172.21.15.55:0/2685282532 conn(0x7fb5c806ddb0 msgr2=0x7fb5c80c7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:47.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.912+0000 7fb5c759e700 1 -- 172.21.15.55:0/2685282532 shutdown_connections 2023-10-06T05:05:47.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:47.912+0000 7fb5c759e700 1 -- 172.21.15.55:0/2685282532 wait complete. 2023-10-06T05:05:47.933 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:05:47.933 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:05:47.933 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2023-10-06T05:05:47.934 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:05:47.934 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:05:47.934 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:05:47.934 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:05:47.934 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:05:47.934 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:05:47.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:05:47.935 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2023-10-06T05:05:47.973 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:05:47.973 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:05:47.973 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:05:47.973 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:05:47.973 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T05:05:47.973 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:05:47.973 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:05:47.974 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:05:47.974 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2023-10-06T05:05:47.974 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:05:47.974 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:05:47.974 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2023-10-06T05:05:47.974 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:05:47.974 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:05:47.974 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2023-10-06T05:05:47.974 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:05:47.974 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 36693sec preferred_lft 36693sec 2023-10-06T05:05:47.974 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link 2023-10-06T05:05:47.975 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:05:47.975 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:05:47.975 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:05:47.975 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2023-10-06T05:05:47.975 INFO:teuthology.orchestra.run.smithi055.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:05:47.975 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff 2023-10-06T05:05:47.975 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:05:47.975 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:05:47.975 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::1099:8eff:fe6d:102/64 scope link 2023-10-06T05:05:47.975 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:05:47.975 INFO:teuthology.orchestra.run.smithi055.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:05:47.975 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether f2:5d:53:d2:00:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:05:47.975 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f05d:53ff:fed2:a/64 scope link 2023-10-06T05:05:47.976 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:05:47.976 INFO:teuthology.orchestra.run.smithi055.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:05:47.976 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:05:47.976 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::e037:33ff:feb7:6c8f/64 scope link 2023-10-06T05:05:47.976 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:05:47.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:05:47.976 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:05:48.074 INFO:teuthology.orchestra.run.smithi055.stdout:1 2023-10-06T05:05:48.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:05:48.077 DEBUG:teuthology.orchestra.run.smithi055:> route 2023-10-06T05:05:48.116 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2023-10-06T05:05:48.116 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:05:48.116 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:05:48.116 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:05:48.116 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:05:48.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:05:48.118 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T05:05:48.118 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T05:05:48.118 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:05:48.118 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:05:48.118 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:05:48.118 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T05:05:48.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:05:48.316 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T05:05:48.353 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:05:48.354 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:05:48.354 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:05:48.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:05:48.354 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2023-10-06T05:05:48.456 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:05:48.456 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:05:48.492 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:05:48.493 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:05:48.493 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:05:48.569 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:05:48.569 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:05:48.569 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:05:48.702 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:05:48.702 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:05:48.703 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T05:05:48.719 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:05:48.719 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:05:48.719 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T05:05:48.719 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T05:05:48.719 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:05:48.740 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:05:48.740 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:05:48.740 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T05:05:48.741 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T05:05:48.741 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:05:48.741 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T05:05:48.741 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T05:05:48.741 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T05:05:48.745 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:05:48.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:05:48.745 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:05:48.808 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T05:05:49.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:48.999+0000 7fadb139d700 1 Processor -- start 2023-10-06T05:05:49.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:48.999+0000 7fadb139d700 1 -- start start 2023-10-06T05:05:49.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.001+0000 7fadb139d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fadac150550 0x7fadac157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:49.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.001+0000 7fadb139d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadac0c7410 0x7fadac069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:49.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.001+0000 7fadb139d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadac06e2a0 con 0x7fadac150180 2023-10-06T05:05:49.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.001+0000 7fadb139d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadac06b2e0 con 0x7fadac150550 2023-10-06T05:05:49.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.001+0000 7fadb139d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac0c78e0 con 0x7fadac0c7410 2023-10-06T05:05:49.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.001+0000 7fadab7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fadac150550 0x7fadac157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:49.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.001+0000 7fadb0b9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadac0c7410 0x7fadac069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:49.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.001+0000 7fadab7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fadac150550 0x7fadac157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:40746/0 (socket says 172.21.15.55:40746) 2023-10-06T05:05:49.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.002+0000 7fadab7fe700 1 -- 172.21.15.55:0/3100186500 learned_addr learned my addr 172.21.15.55:0/3100186500 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:49.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.001+0000 7fadabfff700 1 --1- 172.21.15.55:0/3100186500 >> v1:172.21.15.163:6789/0 conn(0x7fadac150180 0x7fadac0c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:41318/0 (socket says 172.21.15.55:41318) 2023-10-06T05:05:49.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.002+0000 7fadab7fe700 1 -- 172.21.15.55:0/3100186500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadac0c7410 msgr2=0x7fadac069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:49.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.002+0000 7fadab7fe700 1 --2- 172.21.15.55:0/3100186500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadac0c7410 0x7fadac069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:49.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.002+0000 7fadab7fe700 1 -- 172.21.15.55:0/3100186500 >> v1:172.21.15.163:6789/0 conn(0x7fadac150180 legacy=0x7fadac0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:49.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.002+0000 7fadab7fe700 1 -- 172.21.15.55:0/3100186500 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac0c7bc0 con 0x7fadac150550 2023-10-06T05:05:49.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.002+0000 7fadb0b9c700 1 --2- 172.21.15.55:0/3100186500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadac0c7410 0x7fadac069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:49.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.002+0000 7fadaaffd700 1 -- 172.21.15.55:0/3100186500 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36429854 0 0) 0x7fadac06e2a0 con 0x7fadac150180 2023-10-06T05:05:49.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.002+0000 7fadab7fe700 1 --2- 172.21.15.55:0/3100186500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadac150550 0x7fadac157770 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fada00099b0 tx=0x7fada0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b24d542e9c70e506 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:49.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.003+0000 7fadaaffd700 1 -- 172.21.15.55:0/3100186500 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0009ea0 con 0x7fadac150550 2023-10-06T05:05:49.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.003+0000 7fadaaffd700 1 -- 172.21.15.55:0/3100186500 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada000b860 con 0x7fadac150550 2023-10-06T05:05:49.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.003+0000 7fadaaffd700 1 -- 172.21.15.55:0/3100186500 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada000bc00 con 0x7fadac150550 2023-10-06T05:05:49.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.003+0000 7fadb139d700 1 -- 172.21.15.55:0/3100186500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadac150550 msgr2=0x7fadac157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:49.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.003+0000 7fadb139d700 1 --2- 172.21.15.55:0/3100186500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadac150550 0x7fadac157770 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fada00099b0 tx=0x7fada0009250 comp rx=0 tx=0).stop 2023-10-06T05:05:49.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.003+0000 7fadb139d700 1 -- 172.21.15.55:0/3100186500 shutdown_connections 2023-10-06T05:05:49.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.003+0000 7fadb139d700 1 --2- 172.21.15.55:0/3100186500 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadac0c7410 0x7fadac069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:49.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.003+0000 7fadb139d700 1 --2- 172.21.15.55:0/3100186500 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadac150550 0x7fadac157770 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:49.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.003+0000 7fadb139d700 1 -- 172.21.15.55:0/3100186500 >> 172.21.15.55:0/3100186500 conn(0x7fadac0cc1f0 msgr2=0x7fadac0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:49.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.003+0000 7fadb139d700 1 -- 172.21.15.55:0/3100186500 shutdown_connections 2023-10-06T05:05:49.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.003+0000 7fadb139d700 1 -- 172.21.15.55:0/3100186500 wait complete. 2023-10-06T05:05:49.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.005+0000 7fadb139d700 1 Processor -- start 2023-10-06T05:05:49.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.005+0000 7fadb139d700 1 -- start start 2023-10-06T05:05:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadb139d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadac0c7410 0x7fadac1694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadb139d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadac150180 0x7fadac149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadb139d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadac150550 0x7fadac1654d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadb139d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac169a20 con 0x7fadac150550 2023-10-06T05:05:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadb139d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac168a90 con 0x7fadac0c7410 2023-10-06T05:05:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadb139d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac169050 con 0x7fadac150180 2023-10-06T05:05:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadb0b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadac150550 0x7fadac1654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadb0b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadac150550 0x7fadac1654d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45582/0 (socket says 172.21.15.55:45582) 2023-10-06T05:05:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadb0b9c700 1 -- 172.21.15.55:0/3476170152 learned_addr learned my addr 172.21.15.55:0/3476170152 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadab7fe700 1 --2- 172.21.15.55:0/3476170152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadac150180 0x7fadac149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:49.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadabfff700 1 --2- 172.21.15.55:0/3476170152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadac0c7410 0x7fadac1694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:49.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadb0b9c700 1 -- 172.21.15.55:0/3476170152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadac150180 msgr2=0x7fadac149d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:49.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadb0b9c700 1 --2- 172.21.15.55:0/3476170152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadac150180 0x7fadac149d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:49.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadb0b9c700 1 -- 172.21.15.55:0/3476170152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadac0c7410 msgr2=0x7fadac1694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:49.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadb0b9c700 1 --2- 172.21.15.55:0/3476170152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadac0c7410 0x7fadac1694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:49.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.006+0000 7fadb0b9c700 1 -- 172.21.15.55:0/3476170152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada400aa40 con 0x7fadac150550 2023-10-06T05:05:49.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.007+0000 7fadb0b9c700 1 --2- 172.21.15.55:0/3476170152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadac150550 0x7fadac1654d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fada4008f10 tx=0x7fada400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:49.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.007+0000 7fad93fff700 1 -- 172.21.15.55:0/3476170152 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada400f040 con 0x7fadac150550 2023-10-06T05:05:49.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.007+0000 7fad93fff700 1 -- 172.21.15.55:0/3476170152 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada400ebe0 con 0x7fadac150550 2023-10-06T05:05:49.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.007+0000 7fad93fff700 1 -- 172.21.15.55:0/3476170152 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada4016480 con 0x7fadac150550 2023-10-06T05:05:49.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.007+0000 7fadb139d700 1 -- 172.21.15.55:0/3476170152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada00094f0 con 0x7fadac150550 2023-10-06T05:05:49.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.007+0000 7fadb139d700 1 -- 172.21.15.55:0/3476170152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadac1662a0 con 0x7fadac150550 2023-10-06T05:05:49.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.008+0000 7fadb139d700 1 -- 172.21.15.55:0/3476170152 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadac04c3c0 con 0x7fadac150550 2023-10-06T05:05:49.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.009+0000 7fad93fff700 1 -- 172.21.15.55:0/3476170152 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fada4016830 con 0x7fadac150550 2023-10-06T05:05:49.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.010+0000 7fad93fff700 1 --2- 172.21.15.55:0/3476170152 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad98066f30 0x7fad980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:49.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.010+0000 7fad93fff700 1 -- 172.21.15.55:0/3476170152 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fada40854e0 con 0x7fadac150550 2023-10-06T05:05:49.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.011+0000 7fadabfff700 1 --2- 172.21.15.55:0/3476170152 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad98066f30 0x7fad980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:49.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.011+0000 7fadabfff700 1 --2- 172.21.15.55:0/3476170152 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad98066f30 0x7fad980693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fadac0c2930 tx=0x7fad9c009f60 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:49.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.014+0000 7fad93fff700 1 -- 172.21.15.55:0/3476170152 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fada4052450 con 0x7fadac150550 2023-10-06T05:05:49.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.246+0000 7fadb139d700 1 -- 172.21.15.55:0/3476170152 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fadac14ba90 con 0x7fad98066f30 2023-10-06T05:05:49.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.248+0000 7fad93fff700 1 -- 172.21.15.55:0/3476170152 <== mgr.7946 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fadac14ba90 con 0x7fad98066f30 2023-10-06T05:05:49.250 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T05:05:49.250 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:05:49.250 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephfs" 2023-10-06T05:05:49.250 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:05:49.251 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T05:05:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.250+0000 7fadb139d700 1 -- 172.21.15.55:0/3476170152 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad98066f30 msgr2=0x7fad980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.250+0000 7fadb139d700 1 --2- 172.21.15.55:0/3476170152 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad98066f30 0x7fad980693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fadac0c2930 tx=0x7fad9c009f60 comp rx=0 tx=0).stop 2023-10-06T05:05:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.251+0000 7fadb139d700 1 -- 172.21.15.55:0/3476170152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadac150550 msgr2=0x7fadac1654d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.251+0000 7fadb139d700 1 --2- 172.21.15.55:0/3476170152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadac150550 0x7fadac1654d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fada4008f10 tx=0x7fada400c040 comp rx=0 tx=0).stop 2023-10-06T05:05:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.251+0000 7fadb139d700 1 -- 172.21.15.55:0/3476170152 shutdown_connections 2023-10-06T05:05:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.251+0000 7fadb139d700 1 --2- 172.21.15.55:0/3476170152 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad98066f30 0x7fad980693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.251+0000 7fadb139d700 1 --2- 172.21.15.55:0/3476170152 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fadac150550 0x7fadac1654d0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.251+0000 7fadb139d700 1 --2- 172.21.15.55:0/3476170152 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fadac0c7410 0x7fadac1694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.251+0000 7fadb139d700 1 --2- 172.21.15.55:0/3476170152 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fadac150180 0x7fadac149d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:49.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.251+0000 7fadb139d700 1 -- 172.21.15.55:0/3476170152 >> 172.21.15.55:0/3476170152 conn(0x7fadac0cc1f0 msgr2=0x7fadac156b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:49.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.251+0000 7fadb139d700 1 -- 172.21.15.55:0/3476170152 shutdown_connections 2023-10-06T05:05:49.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.251+0000 7fadb139d700 1 -- 172.21.15.55:0/3476170152 wait complete. 2023-10-06T05:05:49.270 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T05:05:49.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.462+0000 7fa883fff700 1 Processor -- start 2023-10-06T05:05:49.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.462+0000 7fa883fff700 1 -- start start 2023-10-06T05:05:49.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.463+0000 7fa883fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8840b16d0 0x7fa8840b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:49.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.463+0000 7fa883fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8840b33b0 0x7fa8840b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:49.465 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.463+0000 7fa883fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8840b2080 con 0x7fa8840b4e70 2023-10-06T05:05:49.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.463+0000 7fa883fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8840cfb00 con 0x7fa8840b33b0 2023-10-06T05:05:49.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.463+0000 7fa883fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8840b5c40 con 0x7fa8840b16d0 2023-10-06T05:05:49.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.464+0000 7fa8827fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8840b16d0 0x7fa8840b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:49.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.464+0000 7fa8837fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8840b33b0 0x7fa8840b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:49.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.464+0000 7fa8837fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa8840b33b0 0x7fa8840b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:40762/0 (socket says 172.21.15.55:40762) 2023-10-06T05:05:49.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.464+0000 7fa8837fe700 1 -- 172.21.15.55:0/1242985973 learned_addr learned my addr 172.21.15.55:0/1242985973 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:49.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.464+0000 7fa882ffd700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fa8840b4e70 0x7fa8840ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:41320/0 (socket says 172.21.15.55:41320) 2023-10-06T05:05:49.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.464+0000 7fa8827fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8840b16d0 0x7fa8840b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:59530/0 (socket says 172.21.15.55:59530) 2023-10-06T05:05:49.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.464+0000 7fa881ffb700 1 -- 172.21.15.55:0/1242985973 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321737653 0 0) 0x7fa8840b2080 con 0x7fa8840b4e70 2023-10-06T05:05:49.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.464+0000 7fa881ffb700 1 -- 172.21.15.55:0/1242985973 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa86c003570 con 0x7fa8840b4e70 2023-10-06T05:05:49.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.464+0000 7fa8837fe700 1 -- 172.21.15.55:0/1242985973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8840b16d0 msgr2=0x7fa8840b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:49.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.464+0000 7fa8837fe700 1 --2- 172.21.15.55:0/1242985973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8840b16d0 0x7fa8840b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:49.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.464+0000 7fa8837fe700 1 -- 172.21.15.55:0/1242985973 >> v1:172.21.15.163:6789/0 conn(0x7fa8840b4e70 legacy=0x7fa8840ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:49.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.464+0000 7fa8837fe700 1 -- 172.21.15.55:0/1242985973 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8840b5f20 con 0x7fa8840b33b0 2023-10-06T05:05:49.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.464+0000 7fa8827fc700 1 --2- 172.21.15.55:0/1242985973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8840b16d0 0x7fa8840b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:49.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.465+0000 7fa8837fe700 1 --2- 172.21.15.55:0/1242985973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8840b33b0 0x7fa8840b3830 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009f20 tx=0x7fa87c009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ff9ff3eabe374da server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:49.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.465+0000 7fa881ffb700 1 -- 172.21.15.55:0/1242985973 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c00e070 con 0x7fa8840b33b0 2023-10-06T05:05:49.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.465+0000 7fa881ffb700 1 -- 172.21.15.55:0/1242985973 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c00b970 con 0x7fa8840b33b0 2023-10-06T05:05:49.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.465+0000 7fa881ffb700 1 -- 172.21.15.55:0/1242985973 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c0092e0 con 0x7fa8840b33b0 2023-10-06T05:05:49.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.465+0000 7fa883fff700 1 -- 172.21.15.55:0/1242985973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8840b33b0 msgr2=0x7fa8840b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:49.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.465+0000 7fa883fff700 1 --2- 172.21.15.55:0/1242985973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8840b33b0 0x7fa8840b3830 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009f20 tx=0x7fa87c009840 comp rx=0 tx=0).stop 2023-10-06T05:05:49.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.465+0000 7fa883fff700 1 -- 172.21.15.55:0/1242985973 shutdown_connections 2023-10-06T05:05:49.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.465+0000 7fa883fff700 1 --2- 172.21.15.55:0/1242985973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8840b33b0 0x7fa8840b3830 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:49.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.465+0000 7fa883fff700 1 --2- 172.21.15.55:0/1242985973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8840b16d0 0x7fa8840b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:49.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.465+0000 7fa883fff700 1 -- 172.21.15.55:0/1242985973 >> 172.21.15.55:0/1242985973 conn(0x7fa8840d10c0 msgr2=0x7fa8840affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:49.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.466+0000 7fa883fff700 1 -- 172.21.15.55:0/1242985973 shutdown_connections 2023-10-06T05:05:49.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.466+0000 7fa883fff700 1 -- 172.21.15.55:0/1242985973 wait complete. 2023-10-06T05:05:49.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.467+0000 7fa883fff700 1 Processor -- start 2023-10-06T05:05:49.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.467+0000 7fa883fff700 1 -- start start 2023-10-06T05:05:49.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.468+0000 7fa883fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8840b16d0 0x7fa8840c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:49.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.468+0000 7fa883fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8840b33b0 0x7fa8840ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:49.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.468+0000 7fa883fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8840b4e70 0x7fa8840c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:49.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.468+0000 7fa883fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8840c8220 con 0x7fa8840b33b0 2023-10-06T05:05:49.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.468+0000 7fa883fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8840c8500 con 0x7fa8840b16d0 2023-10-06T05:05:49.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.468+0000 7fa883fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8840c87e0 con 0x7fa8840b4e70 2023-10-06T05:05:49.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.469+0000 7fa8827fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8840b33b0 0x7fa8840ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:49.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.469+0000 7fa8827fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8840b33b0 0x7fa8840ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45586/0 (socket says 172.21.15.55:45586) 2023-10-06T05:05:49.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.469+0000 7fa8827fc700 1 -- 172.21.15.55:0/694355803 learned_addr learned my addr 172.21.15.55:0/694355803 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:49.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.469+0000 7fa882ffd700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8840b16d0 0x7fa8840c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:49.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.469+0000 7fa8837fe700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8840b4e70 0x7fa8840c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:49.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.469+0000 7fa8827fc700 1 -- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8840b4e70 msgr2=0x7fa8840c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:49.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.469+0000 7fa8827fc700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8840b4e70 0x7fa8840c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:49.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.469+0000 7fa8827fc700 1 -- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8840b16d0 msgr2=0x7fa8840c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:49.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.469+0000 7fa8827fc700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8840b16d0 0x7fa8840c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:49.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.469+0000 7fa8827fc700 1 -- 172.21.15.55:0/694355803 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa878009420 con 0x7fa8840b33b0 2023-10-06T05:05:49.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.469+0000 7fa8837fe700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8840b4e70 0x7fa8840c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:49.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.469+0000 7fa8827fc700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8840b33b0 0x7fa8840ca140 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fa87800ec80 tx=0x7fa8780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:49.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.470+0000 7fa86affd700 1 -- 172.21.15.55:0/694355803 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87800c8f0 con 0x7fa8840b33b0 2023-10-06T05:05:49.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.470+0000 7fa86affd700 1 -- 172.21.15.55:0/694355803 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87800ca50 con 0x7fa8840b33b0 2023-10-06T05:05:49.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.470+0000 7fa883fff700 1 -- 172.21.15.55:0/694355803 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87c009ae0 con 0x7fa8840b33b0 2023-10-06T05:05:49.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.470+0000 7fa86affd700 1 -- 172.21.15.55:0/694355803 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87800cdd0 con 0x7fa8840b33b0 2023-10-06T05:05:49.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.470+0000 7fa883fff700 1 -- 172.21.15.55:0/694355803 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8840c8da0 con 0x7fa8840b33b0 2023-10-06T05:05:49.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.471+0000 7fa883fff700 1 -- 172.21.15.55:0/694355803 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa884088990 con 0x7fa8840b33b0 2023-10-06T05:05:49.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.473+0000 7fa86affd700 1 -- 172.21.15.55:0/694355803 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 91) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa878013680 con 0x7fa8840b33b0 2023-10-06T05:05:49.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.474+0000 7fa86affd700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa870066f30 0x7fa8700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:49.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.474+0000 7fa86affd700 1 -- 172.21.15.55:0/694355803 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fa878084c50 con 0x7fa8840b33b0 2023-10-06T05:05:49.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.475+0000 7fa882ffd700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa870066f30 0x7fa8700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:49.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.476+0000 7fa882ffd700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa870066f30 0x7fa8700693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa88415eb80 tx=0x7fa874007d40 comp rx=0 tx=0).ready entity=mgr.7946 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:49.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.479+0000 7fa86affd700 1 -- 172.21.15.55:0/694355803 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa878051b20 con 0x7fa8840b33b0 2023-10-06T05:05:49.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.741+0000 7fa883fff700 1 -- 172.21.15.55:0/694355803 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fa88404bb80 con 0x7fa8840b33b0 2023-10-06T05:05:50.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:49.999+0000 7fa86affd700 1 -- 172.21.15.55:0/694355803 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 92) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fa878051ef0 con 0x7fa8840b33b0 2023-10-06T05:05:50.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.001+0000 7fa86affd700 1 -- 172.21.15.55:0/694355803 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v92) v1 ==== 92+0+0 (secure 0 0 0) 0x7fa878053920 con 0x7fa8840b33b0 2023-10-06T05:05:50.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.004+0000 7fa883fff700 1 -- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa870066f30 msgr2=0x7fa8700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:50.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.004+0000 7fa883fff700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa870066f30 0x7fa8700693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa88415eb80 tx=0x7fa874007d40 comp rx=0 tx=0).stop 2023-10-06T05:05:50.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.004+0000 7fa883fff700 1 -- 172.21.15.55:0/694355803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8840b33b0 msgr2=0x7fa8840ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:50.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.004+0000 7fa883fff700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8840b33b0 0x7fa8840ca140 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fa87800ec80 tx=0x7fa8780091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.004+0000 7fa883fff700 1 -- 172.21.15.55:0/694355803 shutdown_connections 2023-10-06T05:05:50.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.004+0000 7fa883fff700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa870066f30 0x7fa8700693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.004+0000 7fa883fff700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa8840b4e70 0x7fa8840c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.004+0000 7fa883fff700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa8840b16d0 0x7fa8840c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.004+0000 7fa883fff700 1 --2- 172.21.15.55:0/694355803 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa8840b33b0 0x7fa8840ca140 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.004+0000 7fa883fff700 1 -- 172.21.15.55:0/694355803 >> 172.21.15.55:0/694355803 conn(0x7fa8840d10c0 msgr2=0x7fa88415f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:50.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.004+0000 7fa883fff700 1 -- 172.21.15.55:0/694355803 shutdown_connections 2023-10-06T05:05:50.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.004+0000 7fa883fff700 1 -- 172.21.15.55:0/694355803 wait complete. 2023-10-06T05:05:50.026 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T05:05:50.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.232+0000 7f88219ff700 1 Processor -- start 2023-10-06T05:05:50.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.233+0000 7f88219ff700 1 -- start start 2023-10-06T05:05:50.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.234+0000 7f88219ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f881c159890 0x7f881c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:50.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.234+0000 7f88219ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f881c159c60 0x7f881c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:50.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.234+0000 7f88219ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f881c15db10 0x7f881c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:50.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.234+0000 7f88219ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f881c15df50 con 0x7f881c159c60 2023-10-06T05:05:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.234+0000 7f88219ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f881c15a160 con 0x7f881c159890 2023-10-06T05:05:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.234+0000 7f88219ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881c0d13a0 con 0x7f881c15db10 2023-10-06T05:05:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.234+0000 7f881a7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f881c159c60 0x7f881c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.234+0000 7f881a7fc700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f881c159c60 0x7f881c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45608/0 (socket says 172.21.15.55:45608) 2023-10-06T05:05:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.234+0000 7f881a7fc700 1 -- 172.21.15.55:0/3793739231 learned_addr learned my addr 172.21.15.55:0/3793739231 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:50.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.234+0000 7f881affd700 1 --2- 172.21.15.55:0/3793739231 >> v2:172.21.15.163:3300/0 conn(0x7f881c159890 0x7f881c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:50.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.234+0000 7f881b7fe700 1 --2- 172.21.15.55:0/3793739231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f881c15db10 0x7f881c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:50.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.235+0000 7f881a7fc700 1 -- 172.21.15.55:0/3793739231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f881c15db10 msgr2=0x7f881c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:50.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.235+0000 7f881a7fc700 1 --2- 172.21.15.55:0/3793739231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f881c15db10 0x7f881c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.235+0000 7f881a7fc700 1 -- 172.21.15.55:0/3793739231 >> v2:172.21.15.163:3300/0 conn(0x7f881c159890 msgr2=0x7f881c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:50.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.235+0000 7f881a7fc700 1 --2- 172.21.15.55:0/3793739231 >> v2:172.21.15.163:3300/0 conn(0x7f881c159890 0x7f881c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.235+0000 7f881a7fc700 1 -- 172.21.15.55:0/3793739231 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881c15e140 con 0x7f881c159c60 2023-10-06T05:05:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.235+0000 7f881a7fc700 1 --2- 172.21.15.55:0/3793739231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f881c159c60 0x7f881c15d5d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f881c0d1090 tx=0x7f8810009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a96f89b30dd3571 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.235+0000 7f8819ffb700 1 -- 172.21.15.55:0/3793739231 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881000b640 con 0x7f881c159c60 2023-10-06T05:05:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.236+0000 7f8819ffb700 1 -- 172.21.15.55:0/3793739231 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881000b7a0 con 0x7f881c159c60 2023-10-06T05:05:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.236+0000 7f8819ffb700 1 -- 172.21.15.55:0/3793739231 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881000bb60 con 0x7f881c159c60 2023-10-06T05:05:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.236+0000 7f88219ff700 1 -- 172.21.15.55:0/3793739231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f881c159c60 msgr2=0x7f881c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:50.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.236+0000 7f88219ff700 1 --2- 172.21.15.55:0/3793739231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f881c159c60 0x7f881c15d5d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f881c0d1090 tx=0x7f8810009250 comp rx=0 tx=0).stop 2023-10-06T05:05:50.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.236+0000 7f88219ff700 1 -- 172.21.15.55:0/3793739231 shutdown_connections 2023-10-06T05:05:50.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.236+0000 7f88219ff700 1 --2- 172.21.15.55:0/3793739231 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f881c15db10 0x7f881c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.236+0000 7f88219ff700 1 --2- 172.21.15.55:0/3793739231 >> v2:172.21.15.163:3300/0 conn(0x7f881c159890 0x7f881c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.236+0000 7f88219ff700 1 --2- 172.21.15.55:0/3793739231 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f881c159c60 0x7f881c15d5d0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.236+0000 7f88219ff700 1 -- 172.21.15.55:0/3793739231 >> 172.21.15.55:0/3793739231 conn(0x7f881c068e20 msgr2=0x7f881c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:50.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.236+0000 7f88219ff700 1 -- 172.21.15.55:0/3793739231 shutdown_connections 2023-10-06T05:05:50.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.236+0000 7f88219ff700 1 -- 172.21.15.55:0/3793739231 wait complete. 2023-10-06T05:05:50.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.238+0000 7f88219ff700 1 Processor -- start 2023-10-06T05:05:50.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.238+0000 7f88219ff700 1 -- start start 2023-10-06T05:05:50.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.239+0000 7f88219ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f881c159890 0x7f881c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:50.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.239+0000 7f88219ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f881c159c60 0x7f881c1471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:50.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.239+0000 7f88219ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f881c15db10 0x7f881c154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:50.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.239+0000 7f88219ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c0b5b90 con 0x7f881c159c60 2023-10-06T05:05:50.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.239+0000 7f88219ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881c0b6430 con 0x7f881c15db10 2023-10-06T05:05:50.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.239+0000 7f88219ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881c15d0b0 con 0x7f881c159890 2023-10-06T05:05:50.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.239+0000 7f881a7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f881c159c60 0x7f881c1471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:50.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.240+0000 7f881b7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f881c15db10 0x7f881c154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:50.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.240+0000 7f881a7fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f881c159c60 0x7f881c1471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45618/0 (socket says 172.21.15.55:45618) 2023-10-06T05:05:50.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.240+0000 7f881a7fc700 1 -- 172.21.15.55:0/1925066034 learned_addr learned my addr 172.21.15.55:0/1925066034 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:50.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.240+0000 7f881affd700 1 --2- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f881c159890 0x7f881c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:50.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.240+0000 7f881a7fc700 1 -- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f881c159890 msgr2=0x7f881c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:50.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.240+0000 7f881a7fc700 1 --2- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f881c159890 0x7f881c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.240+0000 7f881a7fc700 1 -- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f881c15db10 msgr2=0x7f881c154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:50.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.240+0000 7f881a7fc700 1 --2- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f881c15db10 0x7f881c154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.240+0000 7f881a7fc700 1 -- 172.21.15.55:0/1925066034 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88100094f0 con 0x7f881c159c60 2023-10-06T05:05:50.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.240+0000 7f881b7fe700 1 --2- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f881c15db10 0x7f881c154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:50.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.240+0000 7f881a7fc700 1 --2- 172.21.15.55:0/1925066034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f881c159c60 0x7f881c1471f0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f881c070e20 tx=0x7f8810017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:50.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.241+0000 7f8802ffd700 1 -- 172.21.15.55:0/1925066034 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881000b910 con 0x7f881c159c60 2023-10-06T05:05:50.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.241+0000 7f8802ffd700 1 -- 172.21.15.55:0/1925066034 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881000ba70 con 0x7f881c159c60 2023-10-06T05:05:50.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.241+0000 7f8802ffd700 1 -- 172.21.15.55:0/1925066034 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810011630 con 0x7f881c159c60 2023-10-06T05:05:50.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.241+0000 7f88219ff700 1 -- 172.21.15.55:0/1925066034 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881c0b5de0 con 0x7f881c159c60 2023-10-06T05:05:50.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.241+0000 7f88219ff700 1 -- 172.21.15.55:0/1925066034 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881c0b3ad0 con 0x7f881c159c60 2023-10-06T05:05:50.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.242+0000 7f88219ff700 1 -- 172.21.15.55:0/1925066034 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f881c0cd100 con 0x7f881c159c60 2023-10-06T05:05:50.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.243+0000 7f8802ffd700 1 -- 172.21.15.55:0/1925066034 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 92) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f88100119a0 con 0x7f881c159c60 2023-10-06T05:05:50.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.245+0000 7f8802ffd700 1 --2- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8808066f80 0x7f8808069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:50.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.245+0000 7f8802ffd700 1 -- 172.21.15.55:0/1925066034 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f8810086070 con 0x7f881c159c60 2023-10-06T05:05:50.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.245+0000 7f881affd700 1 -- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8808066f80 msgr2=0x7f8808069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:50.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.245+0000 7f881affd700 1 --2- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8808066f80 0x7f8808069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:50.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.249+0000 7f8802ffd700 1 -- 172.21.15.55:0/1925066034 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f88100b75a0 con 0x7f881c159c60 2023-10-06T05:05:50.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.446+0000 7f881affd700 1 -- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8808066f80 msgr2=0x7f8808069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:50.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.446+0000 7f881affd700 1 --2- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8808066f80 0x7f8808069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:50.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.462+0000 7f88219ff700 1 -- 172.21.15.55:0/1925066034 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f881c04c3c0 con 0x7f881c159c60 2023-10-06T05:05:50.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.472+0000 7f8802ffd700 1 -- 172.21.15.55:0/1925066034 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v65)=0 v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f881c0cd100 con 0x7f881c159c60 2023-10-06T05:05:50.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.474+0000 7f88219ff700 1 -- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8808066f80 msgr2=0x7f8808069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:50.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.474+0000 7f88219ff700 1 --2- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8808066f80 0x7f8808069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.474+0000 7f88219ff700 1 -- 172.21.15.55:0/1925066034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f881c159c60 msgr2=0x7f881c1471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:50.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.474+0000 7f88219ff700 1 --2- 172.21.15.55:0/1925066034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f881c159c60 0x7f881c1471f0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f881c070e20 tx=0x7f8810017000 comp rx=0 tx=0).stop 2023-10-06T05:05:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.474+0000 7f88219ff700 1 -- 172.21.15.55:0/1925066034 shutdown_connections 2023-10-06T05:05:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.474+0000 7f88219ff700 1 --2- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8808066f80 0x7f8808069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.474+0000 7f88219ff700 1 --2- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f881c15db10 0x7f881c154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.474+0000 7f88219ff700 1 --2- 172.21.15.55:0/1925066034 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f881c159890 0x7f881c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.474+0000 7f88219ff700 1 --2- 172.21.15.55:0/1925066034 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f881c159c60 0x7f881c1471f0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.474+0000 7f88219ff700 1 -- 172.21.15.55:0/1925066034 >> 172.21.15.55:0/1925066034 conn(0x7f881c068e20 msgr2=0x7f881c153390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.474+0000 7f88219ff700 1 -- 172.21.15.55:0/1925066034 shutdown_connections 2023-10-06T05:05:50.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.474+0000 7f88219ff700 1 -- 172.21.15.55:0/1925066034 wait complete. 2023-10-06T05:05:50.497 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T05:05:50.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.695+0000 7fea5219d700 1 Processor -- start 2023-10-06T05:05:50.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.695+0000 7fea5219d700 1 -- start start 2023-10-06T05:05:50.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.696+0000 7fea5219d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea4c0b4e70 0x7fea4c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:50.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.696+0000 7fea5219d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fea4c0b1b40 0x7fea4c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:50.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.696+0000 7fea5219d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c0cfb00 con 0x7fea4c0b16d0 2023-10-06T05:05:50.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.696+0000 7fea5219d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4c0b5c40 con 0x7fea4c0b1b40 2023-10-06T05:05:50.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.696+0000 7fea5219d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0b5f20 con 0x7fea4c0b4e70 2023-10-06T05:05:50.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.697+0000 7fea5099a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fea4c0b16d0 0x7fea4c15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55970/0 (socket says 172.21.15.55:55970) 2023-10-06T05:05:50.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.697+0000 7fea5099a700 1 -- 172.21.15.55:0/1121089041 learned_addr learned my addr 172.21.15.55:0/1121089041 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:50.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.697+0000 7fea5119b700 1 --2- 172.21.15.55:0/1121089041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea4c0b4e70 0x7fea4c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:50.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.697+0000 7fea5199c700 1 --2- 172.21.15.55:0/1121089041 >> v2:172.21.15.163:3300/0 conn(0x7fea4c0b1b40 0x7fea4c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:50.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.697+0000 7fea43fff700 1 -- 172.21.15.55:0/1121089041 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491485402 0 0) 0x7fea4c0cfb00 con 0x7fea4c0b16d0 2023-10-06T05:05:50.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.697+0000 7fea43fff700 1 -- 172.21.15.55:0/1121089041 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea38003570 con 0x7fea4c0b16d0 2023-10-06T05:05:50.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.697+0000 7fea5119b700 1 -- 172.21.15.55:0/1121089041 >> v2:172.21.15.163:3300/0 conn(0x7fea4c0b1b40 msgr2=0x7fea4c15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:50.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.697+0000 7fea5119b700 1 --2- 172.21.15.55:0/1121089041 >> v2:172.21.15.163:3300/0 conn(0x7fea4c0b1b40 0x7fea4c15d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.697+0000 7fea5119b700 1 -- 172.21.15.55:0/1121089041 >> v1:172.21.15.55:6789/0 conn(0x7fea4c0b16d0 legacy=0x7fea4c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:50.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.697+0000 7fea5119b700 1 -- 172.21.15.55:0/1121089041 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c0b6200 con 0x7fea4c0b4e70 2023-10-06T05:05:50.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.697+0000 7fea5199c700 1 --2- 172.21.15.55:0/1121089041 >> v2:172.21.15.163:3300/0 conn(0x7fea4c0b1b40 0x7fea4c15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:50.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.698+0000 7fea5119b700 1 --2- 172.21.15.55:0/1121089041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea4c0b4e70 0x7fea4c0ccff0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fea480098e0 tx=0x7fea480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3069377855cd674a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:50.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.698+0000 7fea43fff700 1 -- 172.21.15.55:0/1121089041 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea48011630 con 0x7fea4c0b4e70 2023-10-06T05:05:50.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.698+0000 7fea43fff700 1 -- 172.21.15.55:0/1121089041 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea48011790 con 0x7fea4c0b4e70 2023-10-06T05:05:50.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.698+0000 7fea43fff700 1 -- 172.21.15.55:0/1121089041 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea48011b30 con 0x7fea4c0b4e70 2023-10-06T05:05:50.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.698+0000 7fea5219d700 1 -- 172.21.15.55:0/1121089041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea4c0b4e70 msgr2=0x7fea4c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:50.702 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.698+0000 7fea5219d700 1 --2- 172.21.15.55:0/1121089041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea4c0b4e70 0x7fea4c0ccff0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fea480098e0 tx=0x7fea480091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.699+0000 7fea5219d700 1 -- 172.21.15.55:0/1121089041 shutdown_connections 2023-10-06T05:05:50.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.699+0000 7fea5219d700 1 --2- 172.21.15.55:0/1121089041 >> v2:172.21.15.163:3300/0 conn(0x7fea4c0b1b40 0x7fea4c15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.699+0000 7fea5219d700 1 --2- 172.21.15.55:0/1121089041 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea4c0b4e70 0x7fea4c0ccff0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.699+0000 7fea5219d700 1 -- 172.21.15.55:0/1121089041 >> 172.21.15.55:0/1121089041 conn(0x7fea4c0d10c0 msgr2=0x7fea4c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:50.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.699+0000 7fea5219d700 1 -- 172.21.15.55:0/1121089041 shutdown_connections 2023-10-06T05:05:50.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.699+0000 7fea5219d700 1 -- 172.21.15.55:0/1121089041 wait complete. 2023-10-06T05:05:50.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.700+0000 7fea5219d700 1 Processor -- start 2023-10-06T05:05:50.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.700+0000 7fea5219d700 1 -- start start 2023-10-06T05:05:50.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.701+0000 7fea5219d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea4c0b16d0 0x7fea4c145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:50.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.701+0000 7fea5219d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea4c0b1b40 0x7fea4c0b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:50.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.701+0000 7fea5219d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea4c0b4e70 0x7fea4c0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:50.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.701+0000 7fea5219d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c158910 con 0x7fea4c0b16d0 2023-10-06T05:05:50.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.701+0000 7fea5219d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c16b4b0 con 0x7fea4c0b1b40 2023-10-06T05:05:50.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.701+0000 7fea5219d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0c6f70 con 0x7fea4c0b4e70 2023-10-06T05:05:50.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.701+0000 7fea5119b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea4c0b16d0 0x7fea4c145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:50.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.702+0000 7fea5119b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea4c0b16d0 0x7fea4c145a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45626/0 (socket says 172.21.15.55:45626) 2023-10-06T05:05:50.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.702+0000 7fea5119b700 1 -- 172.21.15.55:0/3795269008 learned_addr learned my addr 172.21.15.55:0/3795269008 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:50.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.702+0000 7fea5099a700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea4c0b1b40 0x7fea4c0b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:50.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.702+0000 7fea5199c700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea4c0b4e70 0x7fea4c0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:50.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.702+0000 7fea5099a700 1 -- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea4c0b4e70 msgr2=0x7fea4c0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:50.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.702+0000 7fea5099a700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea4c0b4e70 0x7fea4c0b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.702+0000 7fea5099a700 1 -- 172.21.15.55:0/3795269008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea4c0b16d0 msgr2=0x7fea4c145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:50.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.702+0000 7fea5099a700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea4c0b16d0 0x7fea4c145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.702+0000 7fea5099a700 1 -- 172.21.15.55:0/3795269008 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea44009a20 con 0x7fea4c0b1b40 2023-10-06T05:05:50.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.702+0000 7fea5119b700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea4c0b16d0 0x7fea4c145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:50.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.702+0000 7fea5099a700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea4c0b1b40 0x7fea4c0b3390 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c580 tx=0x7fea3c007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:50.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.703+0000 7fea417fa700 1 -- 172.21.15.55:0/3795269008 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c005c00 con 0x7fea4c0b1b40 2023-10-06T05:05:50.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.703+0000 7fea417fa700 1 -- 172.21.15.55:0/3795269008 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c005d60 con 0x7fea4c0b1b40 2023-10-06T05:05:50.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.703+0000 7fea417fa700 1 -- 172.21.15.55:0/3795269008 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c007320 con 0x7fea4c0b1b40 2023-10-06T05:05:50.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.703+0000 7fea5219d700 1 -- 172.21.15.55:0/3795269008 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea48009420 con 0x7fea4c0b1b40 2023-10-06T05:05:50.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.703+0000 7fea5219d700 1 -- 172.21.15.55:0/3795269008 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c16b140 con 0x7fea4c0b1b40 2023-10-06T05:05:50.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.704+0000 7fea5219d700 1 -- 172.21.15.55:0/3795269008 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4c16b640 con 0x7fea4c0b1b40 2023-10-06T05:05:50.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.705+0000 7fea417fa700 1 -- 172.21.15.55:0/3795269008 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 92) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fea3c01b3b0 con 0x7fea4c0b1b40 2023-10-06T05:05:50.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.706+0000 7fea417fa700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fea30066f80 0x7fea30069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:50.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.706+0000 7fea417fa700 1 -- 172.21.15.55:0/3795269008 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7fea3c083cc0 con 0x7fea4c0b1b40 2023-10-06T05:05:50.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.706+0000 7fea5119b700 1 -- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fea30066f80 msgr2=0x7fea30069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:50.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.706+0000 7fea5119b700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fea30066f80 0x7fea30069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:50.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.710+0000 7fea417fa700 1 -- 172.21.15.55:0/3795269008 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea3c050cf0 con 0x7fea4c0b1b40 2023-10-06T05:05:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.907+0000 7fea5119b700 1 -- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fea30066f80 msgr2=0x7fea30069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:50.909 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.907+0000 7fea5119b700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fea30066f80 0x7fea30069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:50.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.919+0000 7fea5219d700 1 -- 172.21.15.55:0/3795269008 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fea4c0cd860 con 0x7fea4c0b1b40 2023-10-06T05:05:50.931 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.930+0000 7fea417fa700 1 -- 172.21.15.55:0/3795269008 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v66)=0 v66) v1 ==== 133+0+0 (secure 0 0 0) 0x7fea3c017070 con 0x7fea4c0b1b40 2023-10-06T05:05:50.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.932+0000 7fea5219d700 1 -- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fea30066f80 msgr2=0x7fea30069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:50.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.932+0000 7fea5219d700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fea30066f80 0x7fea30069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.932+0000 7fea5219d700 1 -- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea4c0b1b40 msgr2=0x7fea4c0b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:50.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.932+0000 7fea5219d700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea4c0b1b40 0x7fea4c0b3390 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c580 tx=0x7fea3c007d50 comp rx=0 tx=0).stop 2023-10-06T05:05:50.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.932+0000 7fea5219d700 1 -- 172.21.15.55:0/3795269008 shutdown_connections 2023-10-06T05:05:50.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.932+0000 7fea5219d700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fea30066f80 0x7fea30069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.932+0000 7fea5219d700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fea4c0b4e70 0x7fea4c0b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.932+0000 7fea5219d700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fea4c0b16d0 0x7fea4c145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.932+0000 7fea5219d700 1 --2- 172.21.15.55:0/3795269008 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fea4c0b1b40 0x7fea4c0b3390 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:50.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.932+0000 7fea5219d700 1 -- 172.21.15.55:0/3795269008 >> 172.21.15.55:0/3795269008 conn(0x7fea4c0d10c0 msgr2=0x7fea4c15c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:50.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.932+0000 7fea5219d700 1 -- 172.21.15.55:0/3795269008 shutdown_connections 2023-10-06T05:05:50.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:50.932+0000 7fea5219d700 1 -- 172.21.15.55:0/3795269008 wait complete. 2023-10-06T05:05:50.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:50.953 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T05:05:51.004 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-10-06T05:05:51.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.206+0000 7f836e59d700 1 Processor -- start 2023-10-06T05:05:51.208 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.206+0000 7f836e59d700 1 -- start start 2023-10-06T05:05:51.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.208+0000 7f836e59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f83680b03c0 0x7f83680cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:51.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.208+0000 7f836e59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f83681591e0 0x7f8368159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:51.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.208+0000 7f836e59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83680cfa00 con 0x7f83680b0790 2023-10-06T05:05:51.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.208+0000 7f836e59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83680cb580 con 0x7f83681591e0 2023-10-06T05:05:51.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.208+0000 7f836e59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83680cf160 con 0x7f83680b03c0 2023-10-06T05:05:51.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.208+0000 7f836dd9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f83681591e0 0x7f8368159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:51.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.208+0000 7f836cd9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f83680b0790 0x7f83680b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:55984/0 (socket says 172.21.15.55:55984) 2023-10-06T05:05:51.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.208+0000 7f836dd9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f83681591e0 0x7f8368159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45628/0 (socket says 172.21.15.55:45628) 2023-10-06T05:05:51.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.208+0000 7f836cd9a700 1 -- 172.21.15.55:0/2834320443 learned_addr learned my addr 172.21.15.55:0/2834320443 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:05:51.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.208+0000 7f836d59b700 1 --2- 172.21.15.55:0/2834320443 >> v2:172.21.15.163:3300/0 conn(0x7f83680b03c0 0x7f83680cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:51.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.209+0000 7f835ffff700 1 -- 172.21.15.55:0/2834320443 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646177704 0 0) 0x7f83680cfa00 con 0x7f83680b0790 2023-10-06T05:05:51.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.209+0000 7f836d59b700 1 -- 172.21.15.55:0/2834320443 >> v2:172.21.15.55:3300/0 conn(0x7f83681591e0 msgr2=0x7f8368159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:51.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.209+0000 7f836d59b700 1 --2- 172.21.15.55:0/2834320443 >> v2:172.21.15.55:3300/0 conn(0x7f83681591e0 0x7f8368159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:51.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.209+0000 7f836d59b700 1 -- 172.21.15.55:0/2834320443 >> v1:172.21.15.55:6789/0 conn(0x7f83680b0790 legacy=0x7f83680b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:51.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.209+0000 7f836d59b700 1 -- 172.21.15.55:0/2834320443 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83680cf440 con 0x7f83680b03c0 2023-10-06T05:05:51.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.209+0000 7f836dd9c700 1 --2- 172.21.15.55:0/2834320443 >> v2:172.21.15.55:3300/0 conn(0x7f83681591e0 0x7f8368159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:05:51.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.209+0000 7f836d59b700 1 --2- 172.21.15.55:0/2834320443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83680b03c0 0x7f83680cea70 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f83640098e0 tx=0x7f83640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a1ff219661f6452 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:51.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.209+0000 7f835ffff700 1 -- 172.21.15.55:0/2834320443 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364011630 con 0x7f83680b03c0 2023-10-06T05:05:51.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.209+0000 7f835ffff700 1 -- 172.21.15.55:0/2834320443 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8364011790 con 0x7f83680b03c0 2023-10-06T05:05:51.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.209+0000 7f835ffff700 1 -- 172.21.15.55:0/2834320443 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364011b30 con 0x7f83680b03c0 2023-10-06T05:05:51.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.210+0000 7f836e59d700 1 -- 172.21.15.55:0/2834320443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83680b03c0 msgr2=0x7f83680cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:51.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.210+0000 7f836e59d700 1 --2- 172.21.15.55:0/2834320443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83680b03c0 0x7f83680cea70 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f83640098e0 tx=0x7f83640091a0 comp rx=0 tx=0).stop 2023-10-06T05:05:51.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.210+0000 7f836e59d700 1 -- 172.21.15.55:0/2834320443 shutdown_connections 2023-10-06T05:05:51.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.210+0000 7f836e59d700 1 --2- 172.21.15.55:0/2834320443 >> v2:172.21.15.55:3300/0 conn(0x7f83681591e0 0x7f8368159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:51.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.210+0000 7f836e59d700 1 --2- 172.21.15.55:0/2834320443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83680b03c0 0x7f83680cea70 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:51.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.210+0000 7f836e59d700 1 -- 172.21.15.55:0/2834320443 >> 172.21.15.55:0/2834320443 conn(0x7f8368063230 msgr2=0x7f8368063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:51.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.210+0000 7f836e59d700 1 -- 172.21.15.55:0/2834320443 shutdown_connections 2023-10-06T05:05:51.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.210+0000 7f836e59d700 1 -- 172.21.15.55:0/2834320443 wait complete. 2023-10-06T05:05:51.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.211+0000 7f836e59d700 1 Processor -- start 2023-10-06T05:05:51.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.212+0000 7f836e59d700 1 -- start start 2023-10-06T05:05:51.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836e59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83680b03c0 0x7f83680c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:51.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836e59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83680b0790 0x7f83680ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:51.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836e59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83681591e0 0x7f8368168c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:51.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836e59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83680c9230 con 0x7f83680b03c0 2023-10-06T05:05:51.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836e59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83680c8620 con 0x7f83680b0790 2023-10-06T05:05:51.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836e59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83680c8900 con 0x7f83681591e0 2023-10-06T05:05:51.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836d59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83680b03c0 0x7f83680c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:51.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836d59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83680b03c0 0x7f83680c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45638/0 (socket says 172.21.15.55:45638) 2023-10-06T05:05:51.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836d59b700 1 -- 172.21.15.55:0/2154682821 learned_addr learned my addr 172.21.15.55:0/2154682821 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:51.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836cd9a700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83680b0790 0x7f83680ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:51.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836dd9c700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83681591e0 0x7f8368168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:51.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836d59b700 1 -- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83681591e0 msgr2=0x7f8368168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:51.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836d59b700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83681591e0 0x7f8368168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:51.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836d59b700 1 -- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83680b0790 msgr2=0x7f83680ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:51.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836d59b700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83680b0790 0x7f83680ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:51.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836d59b700 1 -- 172.21.15.55:0/2154682821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8360009a20 con 0x7f83680b03c0 2023-10-06T05:05:51.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.213+0000 7f836dd9c700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83681591e0 0x7f8368168c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:51.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.214+0000 7f836d59b700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83680b03c0 0x7f83680c7f50 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8364009f80 tx=0x7f8364017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:51.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.214+0000 7f835d7fa700 1 -- 172.21.15.55:0/2154682821 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364000d10 con 0x7f83680b03c0 2023-10-06T05:05:51.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.214+0000 7f835d7fa700 1 -- 172.21.15.55:0/2154682821 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8364011b10 con 0x7f83680b03c0 2023-10-06T05:05:51.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.214+0000 7f836e59d700 1 -- 172.21.15.55:0/2154682821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8364009420 con 0x7f83680b03c0 2023-10-06T05:05:51.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.214+0000 7f835d7fa700 1 -- 172.21.15.55:0/2154682821 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364025600 con 0x7f83680b03c0 2023-10-06T05:05:51.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.214+0000 7f836e59d700 1 -- 172.21.15.55:0/2154682821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83680c8ec0 con 0x7f83680b03c0 2023-10-06T05:05:51.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.216+0000 7f836e59d700 1 -- 172.21.15.55:0/2154682821 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83680c9510 con 0x7f83680b03c0 2023-10-06T05:05:51.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.217+0000 7f835d7fa700 1 -- 172.21.15.55:0/2154682821 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 92) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f8364025a30 con 0x7f83680b03c0 2023-10-06T05:05:51.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.218+0000 7f835d7fa700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 0x7f834c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:51.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.218+0000 7f835d7fa700 1 -- 172.21.15.55:0/2154682821 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6478+0+0 (secure 0 0 0) 0x7f83640860b0 con 0x7f83680b03c0 2023-10-06T05:05:51.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.218+0000 7f836cd9a700 1 -- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 msgr2=0x7f834c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:51.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.218+0000 7f836cd9a700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 0x7f834c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:05:51.229 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.226+0000 7f835d7fa700 1 -- 172.21.15.55:0/2154682821 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8364052f80 con 0x7f83680b03c0 2023-10-06T05:05:51.300 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:51.297+0000 7fe4584d0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:05:51.406 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:51.404+0000 7f3ba3056180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:05:51.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.419+0000 7f836cd9a700 1 -- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 msgr2=0x7f834c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:51.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.419+0000 7f836cd9a700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 0x7f834c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:05:51.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.455+0000 7f836e59d700 1 -- 172.21.15.55:0/2154682821 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f83680c9370 con 0x7f834c066f80 2023-10-06T05:05:51.463 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:51.461+0000 7fe4584d0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:05:51.550 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:51.547+0000 7fe4584d0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:05:51.578 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:51.576+0000 7f3ba3056180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:05:51.666 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:51.665+0000 7f3ba3056180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:05:51.774 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:51.772+0000 7fe4584d0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:05:51.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.820+0000 7f836cd9a700 1 -- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 msgr2=0x7f834c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:51.822 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:51.820+0000 7f836cd9a700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 0x7f834c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T05:05:51.900 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:51.897+0000 7f3ba3056180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:05:51.952 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:51.950+0000 7fe4584d0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:05:52.045 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:52.043+0000 7fe4584d0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:05:52.086 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:52.084+0000 7f3ba3056180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:05:52.182 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:52.180+0000 7f3ba3056180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:05:52.209 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:52.207+0000 7fe4584d0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:05:52.311 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:52.308+0000 7fe4584d0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:05:52.348 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:52.346+0000 7f3ba3056180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:05:52.452 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:52.450+0000 7f3ba3056180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:05:52.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:52.621+0000 7f836cd9a700 1 -- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 msgr2=0x7f834c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:52.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:52.621+0000 7f836cd9a700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 0x7f834c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T05:05:52.798 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:52.796+0000 7fe4584d0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:05:52.893 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:52.891+0000 7fe4584d0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:05:52.952 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:52.950+0000 7f3ba3056180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:05:52.984 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:52.982+0000 7fe4584d0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:05:53.049 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:53.047+0000 7f3ba3056180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:05:53.142 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:53.141+0000 7f3ba3056180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:05:53.610 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:53.608+0000 7fe4584d0180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:05:53.783 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:53.781+0000 7fe4584d0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:05:53.790 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:53.788+0000 7f3ba3056180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:05:53.881 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:53.878+0000 7fe4584d0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:05:53.969 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:53.967+0000 7f3ba3056180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:05:54.069 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:54.067+0000 7f3ba3056180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:05:54.178 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:54.175+0000 7fe4584d0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:05:54.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:54.223+0000 7f836cd9a700 1 -- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 msgr2=0x7f834c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:05:54.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:54.223+0000 7f836cd9a700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 0x7f834c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T05:05:54.275 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:54.272+0000 7fe4584d0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:05:54.370 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:54.368+0000 7f3ba3056180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:05:54.469 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:54.467+0000 7f3ba3056180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:05:54.984 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:54.982+0000 7fe4584d0180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:05:55.200 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:55.198+0000 7f3ba3056180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:05:55.272 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:55.269+0000 7fe4584d0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:05:55.365 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:55.363+0000 7fe4584d0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:05:55.493 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:55.490+0000 7f3ba3056180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:05:55.522 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:55.520+0000 7fe4584d0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:05:55.588 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:55.586+0000 7f3ba3056180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:05:55.628 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:55.626+0000 7fe4584d0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:05:55.747 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:55.746+0000 7f3ba3056180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:05:55.819 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:55.817+0000 7fe4584d0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:05:55.855 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:55.853+0000 7f3ba3056180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:05:55.913 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:55.911+0000 7fe4584d0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:05:56.009 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:56.007+0000 7fe4584d0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:05:56.053 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:56.052+0000 7f3ba3056180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:05:56.103 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:56.101+0000 7fe4584d0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:05:56.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:56.114+0000 7f835d7fa700 1 -- 172.21.15.55:0/2154682821 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 93) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f8364053350 con 0x7f83680b03c0 2023-10-06T05:05:56.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:56.114+0000 7f835d7fa700 1 -- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 msgr2=0x7f834c069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:05:56.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:56.114+0000 7f835d7fa700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 0x7f834c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:56.154 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:56.152+0000 7f3ba3056180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:05:56.252 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:56.249+0000 7fe3cc835700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:56.253 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:56.249+0000 7fe3cc835700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:56.253 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:56.249+0000 7fe3cc835700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:56.253 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:56.249+0000 7fe3cc835700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:56.253 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:56.249+0000 7fe3cc835700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:56.253 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:56.250+0000 7fe3ca831700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:56.253 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:56.250+0000 7fe3ca831700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:56.253 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:56.250+0000 7fe3ca831700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:56.253 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:56.250+0000 7fe3ca831700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:56.253 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:05:56.250+0000 7fe3ca831700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:05:56.260 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:56.258+0000 7f3ba3056180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:05:56.356 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:05:56.355+0000 7f3ba3056180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:05:57.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.119+0000 7f835d7fa700 1 -- 172.21.15.55:0/2154682821 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mgrmap(e 94) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f8364006820 con 0x7f83680b03c0 2023-10-06T05:05:57.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.119+0000 7f835d7fa700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 0x7f834c0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:57.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.119+0000 7f835d7fa700 1 -- 172.21.15.55:0/2154682821 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f834c00e8c0 con 0x7f834c066f80 2023-10-06T05:05:57.125 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.123+0000 7f836cd9a700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 0x7f834c0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:57.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.127+0000 7f836cd9a700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 0x7f834c0693b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f83680c7110 tx=0x7f8358007d30 comp rx=0 tx=0).ready entity=mgr.9442 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:57.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.198+0000 7f835d7fa700 1 -- 172.21.15.55:0/2154682821 <== mgr.9442 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f834c00e8c0 con 0x7f834c066f80 2023-10-06T05:05:57.201 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path snap_test_dir1 2023-10-06T05:05:57.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.201+0000 7f836e59d700 1 -- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 msgr2=0x7f834c0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:57.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.201+0000 7f836e59d700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 0x7f834c0693b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f83680c7110 tx=0x7f8358007d30 comp rx=0 tx=0).stop 2023-10-06T05:05:57.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.201+0000 7f836e59d700 1 -- 172.21.15.55:0/2154682821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83680b03c0 msgr2=0x7f83680c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:57.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.201+0000 7f836e59d700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83680b03c0 0x7f83680c7f50 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8364009f80 tx=0x7f8364017000 comp rx=0 tx=0).stop 2023-10-06T05:05:57.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.201+0000 7f836e59d700 1 -- 172.21.15.55:0/2154682821 shutdown_connections 2023-10-06T05:05:57.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.201+0000 7f836e59d700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f834c066f80 0x7f834c0693b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.201+0000 7f836e59d700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f83681591e0 0x7f8368168c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.201+0000 7f836e59d700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f83680b03c0 0x7f83680c7f50 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.201+0000 7f836e59d700 1 --2- 172.21.15.55:0/2154682821 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f83680b0790 0x7f83680ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.201+0000 7f836e59d700 1 -- 172.21.15.55:0/2154682821 >> 172.21.15.55:0/2154682821 conn(0x7f8368063230 msgr2=0x7f83680695b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:57.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.201+0000 7f836e59d700 1 -- 172.21.15.55:0/2154682821 shutdown_connections 2023-10-06T05:05:57.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.201+0000 7f836e59d700 1 -- 172.21.15.55:0/2154682821 wait complete. 2023-10-06T05:05:57.223 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-06T05:05:57.223 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-10-06T05:05:57.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.422+0000 7f4f73fff700 1 Processor -- start 2023-10-06T05:05:57.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.422+0000 7f4f73fff700 1 -- start start 2023-10-06T05:05:57.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.423+0000 7f4f73fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4f741563b0 0x7f4f740ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:57.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.423+0000 7f4f73fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f74156780 0x7f4f740cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:57.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.423+0000 7f4f73fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4f740ae2b0 0x7f4f740ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:57.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.423+0000 7f4f73fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f740aec70 con 0x7f4f740ae2b0 2023-10-06T05:05:57.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.423+0000 7f4f73fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f740cbe50 con 0x7f4f741563b0 2023-10-06T05:05:57.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.423+0000 7f4f73fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f740cb370 con 0x7f4f74156780 2023-10-06T05:05:57.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.423+0000 7f4f737fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4f740ae2b0 0x7f4f740ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:57.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.423+0000 7f4f737fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4f740ae2b0 0x7f4f740ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47808/0 (socket says 172.21.15.55:47808) 2023-10-06T05:05:57.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.423+0000 7f4f737fe700 1 -- 172.21.15.55:0/289343125 learned_addr learned my addr 172.21.15.55:0/289343125 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:57.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.423+0000 7f4f72ffd700 1 --2- 172.21.15.55:0/289343125 >> v2:172.21.15.163:3300/0 conn(0x7f4f741563b0 0x7f4f740ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:57.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.423+0000 7f4f727fc700 1 --2- 172.21.15.55:0/289343125 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f74156780 0x7f4f740cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:57.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.424+0000 7f4f737fe700 1 -- 172.21.15.55:0/289343125 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f74156780 msgr2=0x7f4f740cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:57.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.424+0000 7f4f737fe700 1 --2- 172.21.15.55:0/289343125 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f74156780 0x7f4f740cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.424+0000 7f4f737fe700 1 -- 172.21.15.55:0/289343125 >> v2:172.21.15.163:3300/0 conn(0x7f4f741563b0 msgr2=0x7f4f740ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:05:57.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.424+0000 7f4f737fe700 1 --2- 172.21.15.55:0/289343125 >> v2:172.21.15.163:3300/0 conn(0x7f4f741563b0 0x7f4f740ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.424+0000 7f4f737fe700 1 -- 172.21.15.55:0/289343125 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f740cb650 con 0x7f4f740ae2b0 2023-10-06T05:05:57.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.424+0000 7f4f72ffd700 1 --2- 172.21.15.55:0/289343125 >> v2:172.21.15.163:3300/0 conn(0x7f4f741563b0 0x7f4f740ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:57.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.424+0000 7f4f727fc700 1 --2- 172.21.15.55:0/289343125 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f74156780 0x7f4f740cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:05:57.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.424+0000 7f4f737fe700 1 --2- 172.21.15.55:0/289343125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f740ae2b0 0x7f4f740ae730 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f4f74071150 tx=0x7f4f6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff321581657f53d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:57.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.424+0000 7f4f71ffb700 1 -- 172.21.15.55:0/289343125 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c00e070 con 0x7f4f740ae2b0 2023-10-06T05:05:57.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.424+0000 7f4f71ffb700 1 -- 172.21.15.55:0/289343125 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6c00ba50 con 0x7f4f740ae2b0 2023-10-06T05:05:57.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.424+0000 7f4f71ffb700 1 -- 172.21.15.55:0/289343125 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c0092e0 con 0x7f4f740ae2b0 2023-10-06T05:05:57.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.425+0000 7f4f73fff700 1 -- 172.21.15.55:0/289343125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f740ae2b0 msgr2=0x7f4f740ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:57.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.425+0000 7f4f73fff700 1 --2- 172.21.15.55:0/289343125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f740ae2b0 0x7f4f740ae730 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f4f74071150 tx=0x7f4f6c009840 comp rx=0 tx=0).stop 2023-10-06T05:05:57.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.425+0000 7f4f73fff700 1 -- 172.21.15.55:0/289343125 shutdown_connections 2023-10-06T05:05:57.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.425+0000 7f4f73fff700 1 --2- 172.21.15.55:0/289343125 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f740ae2b0 0x7f4f740ae730 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.425+0000 7f4f73fff700 1 --2- 172.21.15.55:0/289343125 >> v2:172.21.15.163:3300/0 conn(0x7f4f741563b0 0x7f4f740ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.425+0000 7f4f73fff700 1 --2- 172.21.15.55:0/289343125 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f74156780 0x7f4f740cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.425+0000 7f4f73fff700 1 -- 172.21.15.55:0/289343125 >> 172.21.15.55:0/289343125 conn(0x7f4f74068e20 msgr2=0x7f4f740cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:57.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.425+0000 7f4f73fff700 1 -- 172.21.15.55:0/289343125 shutdown_connections 2023-10-06T05:05:57.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.425+0000 7f4f73fff700 1 -- 172.21.15.55:0/289343125 wait complete. 2023-10-06T05:05:57.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.426+0000 7f4f73fff700 1 Processor -- start 2023-10-06T05:05:57.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.427+0000 7f4f73fff700 1 -- start start 2023-10-06T05:05:57.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.427+0000 7f4f73fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f740ae2b0 0x7f4f740acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:57.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.427+0000 7f4f73fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f741563b0 0x7f4f7416cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:57.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.428+0000 7f4f73fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f74156780 0x7f4f740af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:57.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.428+0000 7f4f73fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f740b1b70 con 0x7f4f741563b0 2023-10-06T05:05:57.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.428+0000 7f4f73fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f740ac580 con 0x7f4f740ae2b0 2023-10-06T05:05:57.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.428+0000 7f4f73fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f740ac860 con 0x7f4f74156780 2023-10-06T05:05:57.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.428+0000 7f4f727fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f741563b0 0x7f4f7416cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:57.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.428+0000 7f4f727fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f741563b0 0x7f4f7416cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47812/0 (socket says 172.21.15.55:47812) 2023-10-06T05:05:57.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.428+0000 7f4f727fc700 1 -- 172.21.15.55:0/1314406588 learned_addr learned my addr 172.21.15.55:0/1314406588 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:57.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.428+0000 7f4f72ffd700 1 --2- 172.21.15.55:0/1314406588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f740ae2b0 0x7f4f740acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:57.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.428+0000 7f4f737fe700 1 --2- 172.21.15.55:0/1314406588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f74156780 0x7f4f740af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:57.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.428+0000 7f4f727fc700 1 -- 172.21.15.55:0/1314406588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f74156780 msgr2=0x7f4f740af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:57.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.428+0000 7f4f727fc700 1 --2- 172.21.15.55:0/1314406588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f74156780 0x7f4f740af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.428+0000 7f4f727fc700 1 -- 172.21.15.55:0/1314406588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f740ae2b0 msgr2=0x7f4f740acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:57.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.428+0000 7f4f727fc700 1 --2- 172.21.15.55:0/1314406588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f740ae2b0 0x7f4f740acd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.428+0000 7f4f727fc700 1 -- 172.21.15.55:0/1314406588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c009ae0 con 0x7f4f741563b0 2023-10-06T05:05:57.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.429+0000 7f4f727fc700 1 --2- 172.21.15.55:0/1314406588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f741563b0 0x7f4f7416cf70 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800d860 tx=0x7f4f68008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:57.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.429+0000 7f4f5affd700 1 -- 172.21.15.55:0/1314406588 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f680125f0 con 0x7f4f741563b0 2023-10-06T05:05:57.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.429+0000 7f4f5affd700 1 -- 172.21.15.55:0/1314406588 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f68012750 con 0x7f4f741563b0 2023-10-06T05:05:57.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.429+0000 7f4f73fff700 1 -- 172.21.15.55:0/1314406588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f740c3760 con 0x7f4f741563b0 2023-10-06T05:05:57.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.429+0000 7f4f5affd700 1 -- 172.21.15.55:0/1314406588 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f68012af0 con 0x7f4f741563b0 2023-10-06T05:05:57.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.429+0000 7f4f73fff700 1 -- 172.21.15.55:0/1314406588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f740b1e50 con 0x7f4f741563b0 2023-10-06T05:05:57.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.430+0000 7f4f73fff700 1 -- 172.21.15.55:0/1314406588 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f740aca20 con 0x7f4f741563b0 2023-10-06T05:05:57.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.431+0000 7f4f5affd700 1 -- 172.21.15.55:0/1314406588 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 94) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f4f68014430 con 0x7f4f741563b0 2023-10-06T05:05:57.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.432+0000 7f4f5affd700 1 --2- 172.21.15.55:0/1314406588 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4f60066dd0 0x7f4f60069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:57.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.432+0000 7f4f5affd700 1 -- 172.21.15.55:0/1314406588 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6736+0+0 (secure 0 0 0) 0x7f4f68084980 con 0x7f4f741563b0 2023-10-06T05:05:57.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.432+0000 7f4f72ffd700 1 --2- 172.21.15.55:0/1314406588 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4f60066dd0 0x7f4f60069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:57.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.433+0000 7f4f72ffd700 1 --2- 172.21.15.55:0/1314406588 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4f60066dd0 0x7f4f60069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4f64006f00 tx=0x7f4f64008040 comp rx=0 tx=0).ready entity=mgr.9442 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:57.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.436+0000 7f4f5affd700 1 -- 172.21.15.55:0/1314406588 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f680519f0 con 0x7f4f741563b0 2023-10-06T05:05:57.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.663+0000 7f4f73fff700 1 -- 172.21.15.55:0/1314406588 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f740b1cb0 con 0x7f4f60066dd0 2023-10-06T05:05:57.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.724+0000 7f4f5affd700 1 -- 172.21.15.55:0/1314406588 <== mgr.9442 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f4f740b1cb0 con 0x7f4f60066dd0 2023-10-06T05:05:57.726 INFO:teuthology.orchestra.run.smithi055.stdout:Retention added to path snap_test_dir1 2023-10-06T05:05:57.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.726+0000 7f4f73fff700 1 -- 172.21.15.55:0/1314406588 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4f60066dd0 msgr2=0x7f4f60069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:57.728 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.726+0000 7f4f73fff700 1 --2- 172.21.15.55:0/1314406588 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4f60066dd0 0x7f4f60069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4f64006f00 tx=0x7f4f64008040 comp rx=0 tx=0).stop 2023-10-06T05:05:57.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.726+0000 7f4f73fff700 1 -- 172.21.15.55:0/1314406588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f741563b0 msgr2=0x7f4f7416cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:57.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.726+0000 7f4f73fff700 1 --2- 172.21.15.55:0/1314406588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f741563b0 0x7f4f7416cf70 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800d860 tx=0x7f4f68008040 comp rx=0 tx=0).stop 2023-10-06T05:05:57.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.726+0000 7f4f73fff700 1 -- 172.21.15.55:0/1314406588 shutdown_connections 2023-10-06T05:05:57.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.726+0000 7f4f73fff700 1 --2- 172.21.15.55:0/1314406588 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4f60066dd0 0x7f4f60069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.726+0000 7f4f73fff700 1 --2- 172.21.15.55:0/1314406588 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4f74156780 0x7f4f740af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.726+0000 7f4f73fff700 1 --2- 172.21.15.55:0/1314406588 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4f740ae2b0 0x7f4f740acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.726+0000 7f4f73fff700 1 --2- 172.21.15.55:0/1314406588 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4f741563b0 0x7f4f7416cf70 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.726+0000 7f4f73fff700 1 -- 172.21.15.55:0/1314406588 >> 172.21.15.55:0/1314406588 conn(0x7f4f74068e20 msgr2=0x7f4f7406e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:57.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.726+0000 7f4f73fff700 1 -- 172.21.15.55:0/1314406588 shutdown_connections 2023-10-06T05:05:57.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.727+0000 7f4f73fff700 1 -- 172.21.15.55:0/1314406588 wait complete. 2023-10-06T05:05:57.746 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-06T05:05:57.746 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-06-05_06_00 in ~3s... 2023-10-06T05:05:57.747 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-10-06T05:05:57.747 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-10-06T05:05:57.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.936+0000 7fa1bf39d700 1 Processor -- start 2023-10-06T05:05:57.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.936+0000 7fa1bf39d700 1 -- start start 2023-10-06T05:05:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.938+0000 7fa1bf39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa1b8154000 0x7fa1b806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.938+0000 7fa1bf39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b80c9480 con 0x7fa1b80685e0 2023-10-06T05:05:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.938+0000 7fa1bf39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1b815d880 con 0x7fa1b815d3f0 2023-10-06T05:05:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.938+0000 7fa1bf39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1b815ae00 con 0x7fa1b8154000 2023-10-06T05:05:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.938+0000 7fa1beb9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa1b8154000 0x7fa1b806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.938+0000 7fa1beb9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa1b8154000 0x7fa1b806eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47822/0 (socket says 172.21.15.55:47822) 2023-10-06T05:05:57.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.938+0000 7fa1beb9c700 1 -- 172.21.15.55:0/3249204544 learned_addr learned my addr 172.21.15.55:0/3249204544 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:57.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.938+0000 7fa1beb9c700 1 -- 172.21.15.55:0/3249204544 >> v1:172.21.15.163:6789/0 conn(0x7fa1b815d3f0 legacy=0x7fa1b81538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:57.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.938+0000 7fa1beb9c700 1 -- 172.21.15.55:0/3249204544 >> v1:172.21.15.55:6789/0 conn(0x7fa1b80685e0 legacy=0x7fa1b815a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:57.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.938+0000 7fa1beb9c700 1 -- 172.21.15.55:0/3249204544 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b815db60 con 0x7fa1b8154000 2023-10-06T05:05:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.939+0000 7fa1beb9c700 1 --2- 172.21.15.55:0/3249204544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1b8154000 0x7fa1b806eb00 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009f20 tx=0x7fa1b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e016b58565773f70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.939+0000 7fa1bd399700 1 -- 172.21.15.55:0/3249204544 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b000e070 con 0x7fa1b8154000 2023-10-06T05:05:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.939+0000 7fa1bd399700 1 -- 172.21.15.55:0/3249204544 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b000bb10 con 0x7fa1b8154000 2023-10-06T05:05:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.939+0000 7fa1bd399700 1 -- 172.21.15.55:0/3249204544 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00092e0 con 0x7fa1b8154000 2023-10-06T05:05:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.939+0000 7fa1bf39d700 1 -- 172.21.15.55:0/3249204544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1b8154000 msgr2=0x7fa1b806eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:57.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.939+0000 7fa1bf39d700 1 --2- 172.21.15.55:0/3249204544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1b8154000 0x7fa1b806eb00 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009f20 tx=0x7fa1b0009840 comp rx=0 tx=0).stop 2023-10-06T05:05:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.940+0000 7fa1bf39d700 1 -- 172.21.15.55:0/3249204544 shutdown_connections 2023-10-06T05:05:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.940+0000 7fa1bf39d700 1 --2- 172.21.15.55:0/3249204544 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1b8154000 0x7fa1b806eb00 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.940+0000 7fa1bf39d700 1 -- 172.21.15.55:0/3249204544 >> 172.21.15.55:0/3249204544 conn(0x7fa1b806b420 msgr2=0x7fa1b80cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.940+0000 7fa1bf39d700 1 -- 172.21.15.55:0/3249204544 shutdown_connections 2023-10-06T05:05:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.940+0000 7fa1bf39d700 1 -- 172.21.15.55:0/3249204544 wait complete. 2023-10-06T05:05:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.941+0000 7fa1bf39d700 1 Processor -- start 2023-10-06T05:05:57.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.941+0000 7fa1bf39d700 1 -- start start 2023-10-06T05:05:57.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.942+0000 7fa1bf39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80685e0 0x7fa1b80b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:57.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.942+0000 7fa1bf39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1b8154000 0x7fa1b80c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:57.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.942+0000 7fa1bf39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1b815d3f0 0x7fa1b8145ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:57.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.942+0000 7fa1bf39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80b1e30 con 0x7fa1b815d3f0 2023-10-06T05:05:57.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.942+0000 7fa1bf39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80b28c0 con 0x7fa1b8154000 2023-10-06T05:05:57.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.942+0000 7fa1bf39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b80c4a00 con 0x7fa1b80685e0 2023-10-06T05:05:57.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.942+0000 7fa1beb9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1b815d3f0 0x7fa1b8145ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:57.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.943+0000 7fa1beb9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1b815d3f0 0x7fa1b8145ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:47838/0 (socket says 172.21.15.55:47838) 2023-10-06T05:05:57.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.943+0000 7fa1beb9c700 1 -- 172.21.15.55:0/2652584071 learned_addr learned my addr 172.21.15.55:0/2652584071 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:05:57.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.943+0000 7fa1bdb9a700 1 --2- 172.21.15.55:0/2652584071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1b8154000 0x7fa1b80c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:57.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.943+0000 7fa1be39b700 1 --2- 172.21.15.55:0/2652584071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80685e0 0x7fa1b80b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:57.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.943+0000 7fa1beb9c700 1 -- 172.21.15.55:0/2652584071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80685e0 msgr2=0x7fa1b80b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:57.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.943+0000 7fa1beb9c700 1 --2- 172.21.15.55:0/2652584071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80685e0 0x7fa1b80b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.943+0000 7fa1beb9c700 1 -- 172.21.15.55:0/2652584071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1b8154000 msgr2=0x7fa1b80c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:57.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.943+0000 7fa1beb9c700 1 --2- 172.21.15.55:0/2652584071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1b8154000 0x7fa1b80c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:57.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.943+0000 7fa1beb9c700 1 -- 172.21.15.55:0/2652584071 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b0009ae0 con 0x7fa1b815d3f0 2023-10-06T05:05:57.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.943+0000 7fa1beb9c700 1 --2- 172.21.15.55:0/2652584071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1b815d3f0 0x7fa1b8145ec0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000b6c0 tx=0x7fa1b0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:57.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.943+0000 7fa1ae7fc700 1 -- 172.21.15.55:0/2652584071 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0008ce0 con 0x7fa1b815d3f0 2023-10-06T05:05:57.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.944+0000 7fa1ae7fc700 1 -- 172.21.15.55:0/2652584071 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b0008e80 con 0x7fa1b815d3f0 2023-10-06T05:05:57.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.944+0000 7fa1bf39d700 1 -- 172.21.15.55:0/2652584071 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b8151370 con 0x7fa1b815d3f0 2023-10-06T05:05:57.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.944+0000 7fa1ae7fc700 1 -- 172.21.15.55:0/2652584071 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00266b0 con 0x7fa1b815d3f0 2023-10-06T05:05:57.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.944+0000 7fa1bf39d700 1 -- 172.21.15.55:0/2652584071 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b80b2270 con 0x7fa1b815d3f0 2023-10-06T05:05:57.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.945+0000 7fa1bf39d700 1 -- 172.21.15.55:0/2652584071 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b804c3c0 con 0x7fa1b815d3f0 2023-10-06T05:05:57.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.945+0000 7fa1ae7fc700 1 -- 172.21.15.55:0/2652584071 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 94) v1 ==== 84595+0+0 (secure 0 0 0) 0x7fa1b0003040 con 0x7fa1b815d3f0 2023-10-06T05:05:57.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.946+0000 7fa1ae7fc700 1 --2- 172.21.15.55:0/2652584071 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa19c066dd0 0x7fa19c069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:05:57.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.946+0000 7fa1ae7fc700 1 -- 172.21.15.55:0/2652584071 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6736+0+0 (secure 0 0 0) 0x7fa1b00862a0 con 0x7fa1b815d3f0 2023-10-06T05:05:57.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.947+0000 7fa1be39b700 1 --2- 172.21.15.55:0/2652584071 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa19c066dd0 0x7fa19c069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:05:57.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.947+0000 7fa1be39b700 1 --2- 172.21.15.55:0/2652584071 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa19c066dd0 0x7fa19c069280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa1b816d150 tx=0x7fa1b4009040 comp rx=0 tx=0).ready entity=mgr.9442 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:05:57.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:57.950+0000 7fa1ae7fc700 1 -- 172.21.15.55:0/2652584071 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1b00533c0 con 0x7fa1b815d3f0 2023-10-06T05:05:58.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.166+0000 7fa1ae7fc700 1 -- 172.21.15.55:0/2652584071 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 95) v1 ==== 84916+0+0 (secure 0 0 0) 0x7fa1b0053790 con 0x7fa1b815d3f0 2023-10-06T05:05:58.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.182+0000 7fa1bf39d700 1 -- 172.21.15.55:0/2652584071 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1b815de40 con 0x7fa19c066dd0 2023-10-06T05:05:58.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.191+0000 7fa1ae7fc700 1 -- 172.21.15.55:0/2652584071 <== mgr.9442 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7fa1b815de40 con 0x7fa19c066dd0 2023-10-06T05:05:58.193 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:05:58.193 INFO:teuthology.orchestra.run.smithi055.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-06T05:05:58.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.194+0000 7fa1bf39d700 1 -- 172.21.15.55:0/2652584071 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa19c066dd0 msgr2=0x7fa19c069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:58.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.194+0000 7fa1bf39d700 1 --2- 172.21.15.55:0/2652584071 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa19c066dd0 0x7fa19c069280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa1b816d150 tx=0x7fa1b4009040 comp rx=0 tx=0).stop 2023-10-06T05:05:58.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.194+0000 7fa1bf39d700 1 -- 172.21.15.55:0/2652584071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1b815d3f0 msgr2=0x7fa1b8145ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:05:58.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.194+0000 7fa1bf39d700 1 --2- 172.21.15.55:0/2652584071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1b815d3f0 0x7fa1b8145ec0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fa1b000b6c0 tx=0x7fa1b0018000 comp rx=0 tx=0).stop 2023-10-06T05:05:58.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.194+0000 7fa1bf39d700 1 -- 172.21.15.55:0/2652584071 shutdown_connections 2023-10-06T05:05:58.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.194+0000 7fa1bf39d700 1 --2- 172.21.15.55:0/2652584071 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa19c066dd0 0x7fa19c069280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:58.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.194+0000 7fa1bf39d700 1 --2- 172.21.15.55:0/2652584071 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa1b815d3f0 0x7fa1b8145ec0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:58.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.194+0000 7fa1bf39d700 1 --2- 172.21.15.55:0/2652584071 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa1b80685e0 0x7fa1b80b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:58.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.194+0000 7fa1bf39d700 1 --2- 172.21.15.55:0/2652584071 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa1b8154000 0x7fa1b80c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:05:58.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.194+0000 7fa1bf39d700 1 -- 172.21.15.55:0/2652584071 >> 172.21.15.55:0/2652584071 conn(0x7fa1b806b420 msgr2=0x7fa1b80c9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:05:58.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.194+0000 7fa1bf39d700 1 -- 172.21.15.55:0/2652584071 shutdown_connections 2023-10-06T05:05:58.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:05:58.194+0000 7fa1bf39d700 1 -- 172.21.15.55:0/2652584071 wait complete. 2023-10-06T05:05:58.214 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-06T05:05:58.214 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-10-06T05:05:58.214 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:58.214 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:05:58.253 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T05:05:58.253 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:05:58.253 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:05:59.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:05:59.255 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:05:59.298 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T05:05:59.298 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:05:59.298 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:00.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:00.299 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:00.342 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-06T05:06:00.343 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:00.343 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:00.740 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:06:00.737+0000 7fe3bc815700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:06:00.740 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:06:00.737+0000 7fe3bc815700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:06:00.740 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:06:00.737+0000 7fe3bc815700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:06:00.740 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:06:00.737+0000 7fe3bc815700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:06:00.740 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:06:00.737+0000 7fe3bc815700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:06:01.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:01.344 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:01.386 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:01.387 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:01.387 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:01.387 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:01.387 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-06-05_06_00_UTC'] 2023-10-06T05:06:01.387 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696568757.7469144, now = 1696568761.3875585, timo = 3 2023-10-06T05:06:02.388 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-10-06-05_06_00 in ~63s... 2023-10-06T05:06:02.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:02.389 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:02.432 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:02.433 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:02.433 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:02.434 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:03.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:03.435 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:03.479 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:03.480 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:03.480 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:03.480 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:04.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:04.482 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:04.523 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:04.524 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:04.525 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:04.525 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:05.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:05.746 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:05.788 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:05.789 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:05.789 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:05.789 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:06.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:06.790 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:06.833 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:06.834 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:06.834 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:06.834 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:07.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:07.835 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:07.887 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:07.888 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:07.888 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:07.888 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:08.543 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:06:08.548 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:06:08.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:08.889 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:08.931 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:08.932 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:08.932 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:08.932 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:09.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:09.933 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:09.976 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:09.977 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:09.977 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:09.977 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:10.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:10.978 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:11.020 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:11.021 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:11.021 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:11.021 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:12.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:12.022 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:12.065 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:12.066 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:12.066 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:12.066 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:13.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:13.068 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:13.110 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:13.111 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:13.111 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:13.111 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:14.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:14.113 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:14.155 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:14.156 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:14.156 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:14.156 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:15.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:15.157 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:15.199 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:15.200 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:15.200 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:15.200 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:16.201 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:16.202 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:16.244 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:16.245 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:16.245 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:16.245 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:17.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:17.247 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:17.289 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:17.291 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:17.291 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:17.291 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:18.293 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:18.293 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:18.336 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:18.337 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:18.337 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:18.337 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:19.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:19.338 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:19.381 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:19.383 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:19.383 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:19.383 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:20.384 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:20.384 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:20.426 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:20.427 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:20.428 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:20.428 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:21.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:21.429 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:21.471 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:21.472 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:21.472 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:21.472 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:22.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:22.473 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:22.515 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:22.516 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:22.517 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:22.517 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:23.517 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:23.518 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:23.560 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:23.561 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:23.562 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:23.562 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:24.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:24.563 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:24.604 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:24.605 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:24.605 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:24.606 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:25.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:25.607 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:25.649 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:25.650 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:25.650 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:25.650 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:26.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:26.651 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:26.693 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:26.694 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:26.694 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:26.695 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:27.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:27.696 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:27.737 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:27.739 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:27.739 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:27.739 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:28.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:28.740 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:28.784 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:28.785 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:28.785 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:28.785 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:29.786 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:29.786 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:29.829 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:29.830 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:29.830 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:29.830 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:30.831 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:30.832 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:30.873 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:30.875 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:30.875 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:30.875 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:31.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:31.876 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:31.918 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:31.919 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:31.919 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:31.919 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:32.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:32.920 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:32.963 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:32.964 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:32.964 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:32.964 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:33.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:33.965 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:34.008 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:34.010 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:34.010 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:34.010 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:35.011 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:35.011 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:35.057 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:35.059 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:35.059 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:35.059 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:36.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:36.060 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:36.102 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:36.103 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:36.103 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:36.104 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:37.105 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:37.105 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:37.147 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:37.148 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:37.148 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:37.149 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:38.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:38.150 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:38.192 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:38.193 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:38.193 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:38.193 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:38.647 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:06:38.653 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:06:39.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:39.195 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:39.237 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:39.238 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:39.238 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:39.238 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:40.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:40.239 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:40.281 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:40.282 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:40.282 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:40.283 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:41.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:41.283 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:41.326 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:41.328 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:41.328 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:41.328 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:42.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:42.329 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:42.372 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:42.373 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:42.373 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:42.373 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:43.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:43.374 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:43.416 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:43.417 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:43.417 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:43.417 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:44.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:44.419 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:44.461 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:44.462 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:44.462 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:44.463 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:45.463 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:45.463 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:45.506 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:45.507 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:45.507 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:45.507 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:46.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:46.509 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:46.551 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:46.552 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:46.553 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:46.553 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:47.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:47.554 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:47.596 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:47.597 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:47.597 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:47.597 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:48.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:48.598 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:48.640 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:48.641 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:48.641 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:48.641 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:49.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:49.643 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:49.689 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:49.690 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:49.691 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:49.691 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:50.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:50.692 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:50.734 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:50.735 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:50.735 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:50.736 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:51.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:51.737 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:51.779 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:51.780 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:51.780 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:51.780 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:52.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:52.782 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:52.824 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:52.825 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:52.825 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:52.825 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:53.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:53.827 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:53.869 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:53.870 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:53.870 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:53.870 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:54.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:54.871 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:54.913 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:54.914 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:54.914 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:54.915 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:55.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:55.916 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:55.958 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:55.959 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:55.959 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:55.960 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:56.960 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:56.960 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:57.002 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:57.004 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:57.004 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:57.004 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:58.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:58.005 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:58.047 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:58.048 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:58.048 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:58.048 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:06:59.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:06:59.049 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:06:59.092 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:06:59.093 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:06:59.093 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:06:59.093 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:07:00.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:07:00.095 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:07:00.139 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_06_00_UTC 2023-10-06T05:07:00.140 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:07:00.140 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-06T05:07:00.140 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-06T05:07:01.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:07:01.141 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:07:01.182 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_07_00_UTC 2023-10-06T05:07:01.183 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-06-05_07_00_UTC'} 2023-10-06T05:07:01.183 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-06-05_07_00_UTC'} 2023-10-06T05:07:01.183 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2023-10-06-05_06_00_UTC'} 2023-10-06T05:07:01.183 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-10-06-05_06_00_UTC'] 2023-10-06T05:07:01.183 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696568757.7469144, now = 1696568821.1836205, timo = 63 2023-10-06T05:07:02.185 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-06T05:07:02.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.386+0000 7f0a6959d700 1 Processor -- start 2023-10-06T05:07:02.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.386+0000 7f0a6959d700 1 -- start start 2023-10-06T05:07:02.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.387+0000 7f0a6959d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0a640cc8d0 0x7f0a6415b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:02.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.387+0000 7f0a6959d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a6415b960 0x7f0a640ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:02.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.387+0000 7f0a6959d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a64157a50 con 0x7f0a6415b050 2023-10-06T05:07:02.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.387+0000 7f0a6959d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a64158f10 con 0x7f0a640cc8d0 2023-10-06T05:07:02.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.387+0000 7f0a6959d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a641591b0 con 0x7f0a6415b960 2023-10-06T05:07:02.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.387+0000 7f0a637fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0a640cc8d0 0x7f0a6415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.387+0000 7f0a637fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0a640cc8d0 0x7f0a6415b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48082/0 (socket says 172.21.15.55:48082) 2023-10-06T05:07:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.387+0000 7f0a637fe700 1 -- 172.21.15.55:0/1637092558 learned_addr learned my addr 172.21.15.55:0/1637092558 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.387+0000 7f0a68d9c700 1 --2- 172.21.15.55:0/1637092558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a6415b960 0x7f0a640ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.388+0000 7f0a637fe700 1 -- 172.21.15.55:0/1637092558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a6415b960 msgr2=0x7f0a640ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.388+0000 7f0a637fe700 1 --2- 172.21.15.55:0/1637092558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a6415b960 0x7f0a640ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.388+0000 7f0a637fe700 1 -- 172.21.15.55:0/1637092558 >> v1:172.21.15.163:6789/0 conn(0x7f0a6415b050 legacy=0x7f0a640ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.388+0000 7f0a637fe700 1 -- 172.21.15.55:0/1637092558 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a640ca9e0 con 0x7f0a640cc8d0 2023-10-06T05:07:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.388+0000 7f0a637fe700 1 --2- 172.21.15.55:0/1637092558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a640cc8d0 0x7f0a6415b420 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f0a580099b0 tx=0x7f0a58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6a7f649ffaf907 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.388+0000 7f0a62ffd700 1 -- 172.21.15.55:0/1637092558 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a58009ea0 con 0x7f0a640cc8d0 2023-10-06T05:07:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.388+0000 7f0a62ffd700 1 -- 172.21.15.55:0/1637092558 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5800b860 con 0x7f0a640cc8d0 2023-10-06T05:07:02.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.388+0000 7f0a62ffd700 1 -- 172.21.15.55:0/1637092558 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5800bc00 con 0x7f0a640cc8d0 2023-10-06T05:07:02.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.389+0000 7f0a6959d700 1 -- 172.21.15.55:0/1637092558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a640cc8d0 msgr2=0x7f0a6415b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:02.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.389+0000 7f0a6959d700 1 --2- 172.21.15.55:0/1637092558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a640cc8d0 0x7f0a6415b420 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f0a580099b0 tx=0x7f0a58009250 comp rx=0 tx=0).stop 2023-10-06T05:07:02.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.389+0000 7f0a6959d700 1 -- 172.21.15.55:0/1637092558 shutdown_connections 2023-10-06T05:07:02.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.389+0000 7f0a6959d700 1 --2- 172.21.15.55:0/1637092558 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a6415b960 0x7f0a640ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:02.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.389+0000 7f0a6959d700 1 --2- 172.21.15.55:0/1637092558 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a640cc8d0 0x7f0a6415b420 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:02.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.389+0000 7f0a6959d700 1 -- 172.21.15.55:0/1637092558 >> 172.21.15.55:0/1637092558 conn(0x7f0a64068e20 msgr2=0x7f0a640cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:02.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.389+0000 7f0a6959d700 1 -- 172.21.15.55:0/1637092558 shutdown_connections 2023-10-06T05:07:02.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.389+0000 7f0a6959d700 1 -- 172.21.15.55:0/1637092558 wait complete. 2023-10-06T05:07:02.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.390+0000 7f0a6959d700 1 Processor -- start 2023-10-06T05:07:02.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.391+0000 7f0a6959d700 1 -- start start 2023-10-06T05:07:02.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.391+0000 7f0a6959d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a640cc8d0 0x7f0a6414ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:02.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.391+0000 7f0a6959d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a6415b050 0x7f0a640c1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:02.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a6959d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a6415b960 0x7f0a640b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:02.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a6959d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a640c18c0 con 0x7f0a640cc8d0 2023-10-06T05:07:02.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a6959d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a640c1ba0 con 0x7f0a6415b960 2023-10-06T05:07:02.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a6959d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a640c1e80 con 0x7f0a6415b050 2023-10-06T05:07:02.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a63fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a640cc8d0 0x7f0a6414ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:02.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a63fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a640cc8d0 0x7f0a6414ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:48094/0 (socket says 172.21.15.55:48094) 2023-10-06T05:07:02.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a63fff700 1 -- 172.21.15.55:0/3138426810 learned_addr learned my addr 172.21.15.55:0/3138426810 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:02.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a637fe700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a6415b050 0x7f0a640c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:02.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a68d9c700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a6415b960 0x7f0a640b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:02.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a63fff700 1 -- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a6415b050 msgr2=0x7f0a640c1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:02.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a63fff700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a6415b050 0x7f0a640c1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:02.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a63fff700 1 -- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a6415b960 msgr2=0x7f0a640b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:02.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a63fff700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a6415b960 0x7f0a640b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:02.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a63fff700 1 -- 172.21.15.55:0/3138426810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a580094f0 con 0x7f0a640cc8d0 2023-10-06T05:07:02.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a637fe700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a6415b050 0x7f0a640c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:02.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.392+0000 7f0a68d9c700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a6415b960 0x7f0a640b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:02.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.393+0000 7f0a63fff700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a640cc8d0 0x7f0a6414ed10 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400c4e0 tx=0x7f0a5400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:02.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.393+0000 7f0a4bfff700 1 -- 172.21.15.55:0/3138426810 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54011590 con 0x7f0a640cc8d0 2023-10-06T05:07:02.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.393+0000 7f0a4bfff700 1 -- 172.21.15.55:0/3138426810 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a54011710 con 0x7f0a640cc8d0 2023-10-06T05:07:02.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.393+0000 7f0a4bfff700 1 -- 172.21.15.55:0/3138426810 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54011ad0 con 0x7f0a640cc8d0 2023-10-06T05:07:02.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.393+0000 7f0a6959d700 1 -- 172.21.15.55:0/3138426810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a640c2160 con 0x7f0a640cc8d0 2023-10-06T05:07:02.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.393+0000 7f0a6959d700 1 -- 172.21.15.55:0/3138426810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a64168500 con 0x7f0a640cc8d0 2023-10-06T05:07:02.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.394+0000 7f0a6959d700 1 -- 172.21.15.55:0/3138426810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6404c3c0 con 0x7f0a640cc8d0 2023-10-06T05:07:02.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.395+0000 7f0a4bfff700 1 -- 172.21.15.55:0/3138426810 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 96) v1 ==== 84973+0+0 (secure 0 0 0) 0x7f0a5401b3b0 con 0x7f0a640cc8d0 2023-10-06T05:07:02.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.395+0000 7f0a4bfff700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0a500671c0 0x7f0a50069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:02.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.396+0000 7f0a4bfff700 1 -- 172.21.15.55:0/3138426810 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6736+0+0 (secure 0 0 0) 0x7f0a54084220 con 0x7f0a640cc8d0 2023-10-06T05:07:02.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.396+0000 7f0a637fe700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0a500671c0 0x7f0a50069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:02.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.397+0000 7f0a637fe700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0a500671c0 0x7f0a50069670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0a580099b0 tx=0x7f0a58017000 comp rx=0 tx=0).ready entity=mgr.9442 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:02.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.400+0000 7f0a4bfff700 1 -- 172.21.15.55:0/3138426810 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a54050e00 con 0x7f0a640cc8d0 2023-10-06T05:07:02.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.628+0000 7f0a6959d700 1 -- 172.21.15.55:0/3138426810 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a640c27a0 con 0x7f0a500671c0 2023-10-06T05:07:02.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.701+0000 7f0a4bfff700 1 -- 172.21.15.55:0/3138426810 <== mgr.9442 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f0a640c27a0 con 0x7f0a500671c0 2023-10-06T05:07:02.703 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path snap_test_dir1 2023-10-06T05:07:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.703+0000 7f0a6959d700 1 -- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0a500671c0 msgr2=0x7f0a50069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.703+0000 7f0a6959d700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0a500671c0 0x7f0a50069670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0a580099b0 tx=0x7f0a58017000 comp rx=0 tx=0).stop 2023-10-06T05:07:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.703+0000 7f0a6959d700 1 -- 172.21.15.55:0/3138426810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a640cc8d0 msgr2=0x7f0a6414ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.703+0000 7f0a6959d700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a640cc8d0 0x7f0a6414ed10 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f0a5400c4e0 tx=0x7f0a5400c8a0 comp rx=0 tx=0).stop 2023-10-06T05:07:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.704+0000 7f0a6959d700 1 -- 172.21.15.55:0/3138426810 shutdown_connections 2023-10-06T05:07:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.704+0000 7f0a6959d700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0a500671c0 0x7f0a50069670 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.704+0000 7f0a6959d700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0a6415b960 0x7f0a640b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.704+0000 7f0a6959d700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0a640cc8d0 0x7f0a6414ed10 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.704+0000 7f0a6959d700 1 --2- 172.21.15.55:0/3138426810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0a6415b050 0x7f0a640c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.704+0000 7f0a6959d700 1 -- 172.21.15.55:0/3138426810 >> 172.21.15.55:0/3138426810 conn(0x7f0a64068e20 msgr2=0x7f0a6406d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.704+0000 7f0a6959d700 1 -- 172.21.15.55:0/3138426810 shutdown_connections 2023-10-06T05:07:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:02.704+0000 7f0a6959d700 1 -- 172.21.15.55:0/3138426810 wait complete. 2023-10-06T05:07:02.724 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-06T05:07:02.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:07:02.724 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:07:02.762 INFO:teuthology.orchestra.run.smithi055.stdout:scheduled-2023-10-06-05_07_00_UTC 2023-10-06T05:07:02.763 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-06-05_07_00_UTC 2023-10-06T05:07:02.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:07:02.763 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-06-05_07_00_UTC) 2023-10-06T05:07:02.846 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:07:02.846 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-06T05:07:02.927 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T05:07:03.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.156+0000 7fa0bf19d700 1 Processor -- start 2023-10-06T05:07:03.158 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.156+0000 7fa0bf19d700 1 -- start start 2023-10-06T05:07:03.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.157+0000 7fa0bf19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa0b80cb1a0 0x7fa0b815a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:03.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.157+0000 7fa0bf19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b815a8a0 0x7fa0b806f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:03.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.157+0000 7fa0bf19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0b806f5a0 0x7fa0b80ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:03.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.157+0000 7fa0bf19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b815b2e0 con 0x7fa0b80cb1a0 2023-10-06T05:07:03.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.157+0000 7fa0bf19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b806ebd0 con 0x7fa0b806f5a0 2023-10-06T05:07:03.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.157+0000 7fa0bf19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b815ad20 con 0x7fa0b815a8a0 2023-10-06T05:07:03.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.158+0000 7fa0be19b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa0b80cb1a0 0x7fa0b815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:03.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.158+0000 7fa0be19b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa0b80cb1a0 0x7fa0b815a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50726/0 (socket says 172.21.15.55:50726) 2023-10-06T05:07:03.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.158+0000 7fa0be19b700 1 -- 172.21.15.55:0/688859225 learned_addr learned my addr 172.21.15.55:0/688859225 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:03.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.158+0000 7fa0bd99a700 1 --2- 172.21.15.55:0/688859225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b815a8a0 0x7fa0b806f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:03.164 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.158+0000 7fa0be99c700 1 --2- 172.21.15.55:0/688859225 >> v2:172.21.15.163:3300/0 conn(0x7fa0b806f5a0 0x7fa0b80ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:03.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.158+0000 7fa0be19b700 1 -- 172.21.15.55:0/688859225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b815a8a0 msgr2=0x7fa0b806f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:03.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.158+0000 7fa0be19b700 1 --2- 172.21.15.55:0/688859225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b815a8a0 0x7fa0b806f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:03.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.158+0000 7fa0be19b700 1 -- 172.21.15.55:0/688859225 >> v2:172.21.15.163:3300/0 conn(0x7fa0b806f5a0 msgr2=0x7fa0b80ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:03.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.158+0000 7fa0be19b700 1 --2- 172.21.15.55:0/688859225 >> v2:172.21.15.163:3300/0 conn(0x7fa0b806f5a0 0x7fa0b80ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:03.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.158+0000 7fa0be19b700 1 -- 172.21.15.55:0/688859225 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b815b000 con 0x7fa0b80cb1a0 2023-10-06T05:07:03.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.158+0000 7fa0be99c700 1 --2- 172.21.15.55:0/688859225 >> v2:172.21.15.163:3300/0 conn(0x7fa0b806f5a0 0x7fa0b80ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:03.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.158+0000 7fa0be19b700 1 --2- 172.21.15.55:0/688859225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b80cb1a0 0x7fa0b815a360 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fa0b806d580 tx=0x7fa0b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f70fdb8e8baa13 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:03.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.159+0000 7fa0bd199700 1 -- 172.21.15.55:0/688859225 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4009e70 con 0x7fa0b80cb1a0 2023-10-06T05:07:03.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.159+0000 7fa0bd199700 1 -- 172.21.15.55:0/688859225 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b400b8f0 con 0x7fa0b80cb1a0 2023-10-06T05:07:03.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.159+0000 7fa0bd199700 1 -- 172.21.15.55:0/688859225 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b400bc90 con 0x7fa0b80cb1a0 2023-10-06T05:07:03.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.159+0000 7fa0bf19d700 1 -- 172.21.15.55:0/688859225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b80cb1a0 msgr2=0x7fa0b815a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:03.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.159+0000 7fa0bf19d700 1 --2- 172.21.15.55:0/688859225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b80cb1a0 0x7fa0b815a360 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fa0b806d580 tx=0x7fa0b4009250 comp rx=0 tx=0).stop 2023-10-06T05:07:03.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.159+0000 7fa0bf19d700 1 -- 172.21.15.55:0/688859225 shutdown_connections 2023-10-06T05:07:03.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.159+0000 7fa0bf19d700 1 --2- 172.21.15.55:0/688859225 >> v2:172.21.15.163:3300/0 conn(0x7fa0b806f5a0 0x7fa0b80ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:03.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.159+0000 7fa0bf19d700 1 --2- 172.21.15.55:0/688859225 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b80cb1a0 0x7fa0b815a360 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:03.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.159+0000 7fa0bf19d700 1 --2- 172.21.15.55:0/688859225 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b815a8a0 0x7fa0b806f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:03.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.159+0000 7fa0bf19d700 1 -- 172.21.15.55:0/688859225 >> 172.21.15.55:0/688859225 conn(0x7fa0b80cc1e0 msgr2=0x7fa0b80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:03.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.159+0000 7fa0bf19d700 1 -- 172.21.15.55:0/688859225 shutdown_connections 2023-10-06T05:07:03.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.159+0000 7fa0bf19d700 1 -- 172.21.15.55:0/688859225 wait complete. 2023-10-06T05:07:03.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.161+0000 7fa0bf19d700 1 Processor -- start 2023-10-06T05:07:03.168 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.161+0000 7fa0bf19d700 1 -- start start 2023-10-06T05:07:03.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.162+0000 7fa0bf19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b806f5a0 0x7fa0b81547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:03.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.162+0000 7fa0bf19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b80cb1a0 0x7fa0b813ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:03.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.162+0000 7fa0bf19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b816b130 0x7fa0b816c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:03.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.162+0000 7fa0bf19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b814fb40 con 0x7fa0b816b130 2023-10-06T05:07:03.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.162+0000 7fa0bf19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b816b5d0 con 0x7fa0b80cb1a0 2023-10-06T05:07:03.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.162+0000 7fa0bf19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b816c420 con 0x7fa0b806f5a0 2023-10-06T05:07:03.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.162+0000 7fa0be99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b816b130 0x7fa0b816c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:03.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.162+0000 7fa0be99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b816b130 0x7fa0b816c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50728/0 (socket says 172.21.15.55:50728) 2023-10-06T05:07:03.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.162+0000 7fa0be99c700 1 -- 172.21.15.55:0/2452287267 learned_addr learned my addr 172.21.15.55:0/2452287267 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:03.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.162+0000 7fa0bd99a700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b80cb1a0 0x7fa0b813ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:03.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.162+0000 7fa0be19b700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b806f5a0 0x7fa0b81547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:03.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.162+0000 7fa0be99c700 1 -- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b806f5a0 msgr2=0x7fa0b81547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:03.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.162+0000 7fa0be99c700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b806f5a0 0x7fa0b81547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:03.172 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.163+0000 7fa0be99c700 1 -- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b80cb1a0 msgr2=0x7fa0b813ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:03.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.163+0000 7fa0be99c700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b80cb1a0 0x7fa0b813ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:03.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.163+0000 7fa0be99c700 1 -- 172.21.15.55:0/2452287267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b40094f0 con 0x7fa0b816b130 2023-10-06T05:07:03.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.163+0000 7fa0bd99a700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b80cb1a0 0x7fa0b813ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:03.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.163+0000 7fa0be99c700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b816b130 0x7fa0b816c8b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000d180 tx=0x7fa0b0009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:03.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.163+0000 7fa0ae7fc700 1 -- 172.21.15.55:0/2452287267 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000dd60 con 0x7fa0b816b130 2023-10-06T05:07:03.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.163+0000 7fa0ae7fc700 1 -- 172.21.15.55:0/2452287267 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b000f040 con 0x7fa0b816b130 2023-10-06T05:07:03.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.163+0000 7fa0bf19d700 1 -- 172.21.15.55:0/2452287267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b816b810 con 0x7fa0b816b130 2023-10-06T05:07:03.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.163+0000 7fa0ae7fc700 1 -- 172.21.15.55:0/2452287267 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0015650 con 0x7fa0b816b130 2023-10-06T05:07:03.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.163+0000 7fa0bf19d700 1 -- 172.21.15.55:0/2452287267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b816bdd0 con 0x7fa0b816b130 2023-10-06T05:07:03.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.164+0000 7fa0bf19d700 1 -- 172.21.15.55:0/2452287267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b806c450 con 0x7fa0b816b130 2023-10-06T05:07:03.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.165+0000 7fa0ae7fc700 1 -- 172.21.15.55:0/2452287267 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 96) v1 ==== 84973+0+0 (secure 0 0 0) 0x7fa0b0015a40 con 0x7fa0b816b130 2023-10-06T05:07:03.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.166+0000 7fa0ae7fc700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa09c0671c0 0x7fa09c069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:03.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.166+0000 7fa0ae7fc700 1 -- 172.21.15.55:0/2452287267 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6736+0+0 (secure 0 0 0) 0x7fa0b00856c0 con 0x7fa0b816b130 2023-10-06T05:07:03.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.167+0000 7fa0be19b700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa09c0671c0 0x7fa09c069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:03.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.167+0000 7fa0be19b700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa09c0671c0 0x7fa09c069670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0b806d580 tx=0x7fa0b4017000 comp rx=0 tx=0).ready entity=mgr.9442 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:03.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.170+0000 7fa0ae7fc700 1 -- 172.21.15.55:0/2452287267 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0b0052110 con 0x7fa0b816b130 2023-10-06T05:07:03.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.425+0000 7fa0bf19d700 1 -- 172.21.15.55:0/2452287267 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fa0b806e860 con 0x7fa0b816b130 2023-10-06T05:07:03.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.992+0000 7fa0ae7fc700 1 -- 172.21.15.55:0/2452287267 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 97) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fa0b00524e0 con 0x7fa0b816b130 2023-10-06T05:07:03.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.993+0000 7fa0ae7fc700 1 -- 172.21.15.55:0/2452287267 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v97) v1 ==== 93+0+0 (secure 0 0 0) 0x7fa0b0070bb0 con 0x7fa0b816b130 2023-10-06T05:07:03.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.996+0000 7fa0bf19d700 1 -- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa09c0671c0 msgr2=0x7fa09c069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:03.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.996+0000 7fa0bf19d700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa09c0671c0 0x7fa09c069670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa0b806d580 tx=0x7fa0b4017000 comp rx=0 tx=0).stop 2023-10-06T05:07:03.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.996+0000 7fa0bf19d700 1 -- 172.21.15.55:0/2452287267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b816b130 msgr2=0x7fa0b816c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:03.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.996+0000 7fa0bf19d700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b816b130 0x7fa0b816c8b0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000d180 tx=0x7fa0b0009a50 comp rx=0 tx=0).stop 2023-10-06T05:07:03.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.996+0000 7fa0bf19d700 1 -- 172.21.15.55:0/2452287267 shutdown_connections 2023-10-06T05:07:03.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.996+0000 7fa0bf19d700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa09c0671c0 0x7fa09c069670 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:03.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.996+0000 7fa0bf19d700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b816b130 0x7fa0b816c8b0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:03.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.996+0000 7fa0bf19d700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b806f5a0 0x7fa0b81547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.996+0000 7fa0bf19d700 1 --2- 172.21.15.55:0/2452287267 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b80cb1a0 0x7fa0b813ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.996+0000 7fa0bf19d700 1 -- 172.21.15.55:0/2452287267 >> 172.21.15.55:0/2452287267 conn(0x7fa0b80cc1e0 msgr2=0x7fa0b8069eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.997+0000 7fa0bf19d700 1 -- 172.21.15.55:0/2452287267 shutdown_connections 2023-10-06T05:07:04.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:03.997+0000 7fa0bf19d700 1 -- 172.21.15.55:0/2452287267 wait complete. 2023-10-06T05:07:04.019 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:07:04.020 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:07:04.121 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:07:04.202 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:07:04.202 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:07:04.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:07:04.202 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:07:04.293 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T05:07:04.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:04.293 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:07:04.330 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:07:04.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.559+0000 7f88ced9d700 1 Processor -- start 2023-10-06T05:07:04.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.559+0000 7f88ced9d700 1 -- start start 2023-10-06T05:07:04.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.560+0000 7f88ced9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f88c806c0f0 0x7f88c81642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:04.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.560+0000 7f88ced9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c80b40f0 con 0x7f88c8142ea0 2023-10-06T05:07:04.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.561+0000 7f88ced9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c8069640 con 0x7f88c80ad870 2023-10-06T05:07:04.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.561+0000 7f88ced9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c8069920 con 0x7f88c806c0f0 2023-10-06T05:07:04.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.561+0000 7f88cdd9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f88c8142ea0 0x7f88c80b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52772/0 (socket says 172.21.15.55:52772) 2023-10-06T05:07:04.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.561+0000 7f88cdd9b700 1 -- 172.21.15.55:0/2073858134 learned_addr learned my addr 172.21.15.55:0/2073858134 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:04.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.561+0000 7f88ce59c700 1 --2- 172.21.15.55:0/2073858134 >> v2:172.21.15.163:3300/0 conn(0x7f88c806c0f0 0x7f88c81642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:04.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.561+0000 7f88ccd99700 1 -- 172.21.15.55:0/2073858134 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388647880 0 0) 0x7f88c80b40f0 con 0x7f88c8142ea0 2023-10-06T05:07:04.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.561+0000 7f88ccd99700 1 -- 172.21.15.55:0/2073858134 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88b4003570 con 0x7f88c8142ea0 2023-10-06T05:07:04.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.562+0000 7f88ce59c700 1 -- 172.21.15.55:0/2073858134 >> v1:172.21.15.163:6789/0 conn(0x7f88c80ad870 legacy=0x7f88c806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:04.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.562+0000 7f88ce59c700 1 -- 172.21.15.55:0/2073858134 >> v1:172.21.15.55:6789/0 conn(0x7f88c8142ea0 legacy=0x7f88c80b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:04.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.562+0000 7f88ce59c700 1 -- 172.21.15.55:0/2073858134 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c8069c00 con 0x7f88c806c0f0 2023-10-06T05:07:04.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.562+0000 7f88ce59c700 1 --2- 172.21.15.55:0/2073858134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c806c0f0 0x7f88c81642e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009e60 tx=0x7f88c0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da2ecacf83523ec5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:04.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.562+0000 7f88ccd99700 1 -- 172.21.15.55:0/2073858134 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c000e070 con 0x7f88c806c0f0 2023-10-06T05:07:04.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.562+0000 7f88ccd99700 1 -- 172.21.15.55:0/2073858134 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c000bb60 con 0x7f88c806c0f0 2023-10-06T05:07:04.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.562+0000 7f88ccd99700 1 -- 172.21.15.55:0/2073858134 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c0012430 con 0x7f88c806c0f0 2023-10-06T05:07:04.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.563+0000 7f88ced9d700 1 -- 172.21.15.55:0/2073858134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c806c0f0 msgr2=0x7f88c81642e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:04.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.563+0000 7f88ced9d700 1 --2- 172.21.15.55:0/2073858134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c806c0f0 0x7f88c81642e0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009e60 tx=0x7f88c0009780 comp rx=0 tx=0).stop 2023-10-06T05:07:04.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.563+0000 7f88ced9d700 1 -- 172.21.15.55:0/2073858134 shutdown_connections 2023-10-06T05:07:04.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.563+0000 7f88ced9d700 1 --2- 172.21.15.55:0/2073858134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c806c0f0 0x7f88c81642e0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:04.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.563+0000 7f88ced9d700 1 -- 172.21.15.55:0/2073858134 >> 172.21.15.55:0/2073858134 conn(0x7f88c80ce430 msgr2=0x7f88c80ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:04.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.563+0000 7f88ced9d700 1 -- 172.21.15.55:0/2073858134 shutdown_connections 2023-10-06T05:07:04.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.563+0000 7f88ced9d700 1 -- 172.21.15.55:0/2073858134 wait complete. 2023-10-06T05:07:04.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.565+0000 7f88ced9d700 1 Processor -- start 2023-10-06T05:07:04.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.566+0000 7f88ced9d700 1 -- start start 2023-10-06T05:07:04.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.567+0000 7f88ced9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c806c0f0 0x7f88c80c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:04.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.567+0000 7f88ced9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c8142ea0 0x7f88c815cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:04.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.567+0000 7f88ced9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c815d1e0 0x7f88c8169d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:04.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.567+0000 7f88ced9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80c9f00 con 0x7f88c806c0f0 2023-10-06T05:07:04.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.567+0000 7f88ced9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80c8920 con 0x7f88c8142ea0 2023-10-06T05:07:04.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.567+0000 7f88ced9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80c8c00 con 0x7f88c815d1e0 2023-10-06T05:07:04.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.567+0000 7f88cdd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c806c0f0 0x7f88c80c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:04.571 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.568+0000 7f88cdd9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c806c0f0 0x7f88c80c99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50744/0 (socket says 172.21.15.55:50744) 2023-10-06T05:07:04.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.568+0000 7f88cdd9b700 1 -- 172.21.15.55:0/2086940018 learned_addr learned my addr 172.21.15.55:0/2086940018 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:04.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.568+0000 7f88cd59a700 1 --2- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c8142ea0 0x7f88c815cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:04.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.568+0000 7f88ce59c700 1 --2- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c815d1e0 0x7f88c8169d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:04.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.568+0000 7f88cdd9b700 1 -- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c815d1e0 msgr2=0x7f88c8169d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:04.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.568+0000 7f88cdd9b700 1 --2- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c815d1e0 0x7f88c8169d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:04.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.568+0000 7f88cdd9b700 1 -- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c8142ea0 msgr2=0x7f88c815cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:04.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.568+0000 7f88cdd9b700 1 --2- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c8142ea0 0x7f88c815cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:04.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.568+0000 7f88cdd9b700 1 -- 172.21.15.55:0/2086940018 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c0009a20 con 0x7f88c806c0f0 2023-10-06T05:07:04.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.568+0000 7f88cdd9b700 1 --2- 172.21.15.55:0/2086940018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c806c0f0 0x7f88c80c99c0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f88c400c590 tx=0x7f88c4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:04.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.569+0000 7f88bdffb700 1 -- 172.21.15.55:0/2086940018 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4005cd0 con 0x7f88c806c0f0 2023-10-06T05:07:04.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.569+0000 7f88bdffb700 1 -- 172.21.15.55:0/2086940018 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c4005e50 con 0x7f88c806c0f0 2023-10-06T05:07:04.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.569+0000 7f88ced9d700 1 -- 172.21.15.55:0/2086940018 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c80c8ee0 con 0x7f88c806c0f0 2023-10-06T05:07:04.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.569+0000 7f88bdffb700 1 -- 172.21.15.55:0/2086940018 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4007320 con 0x7f88c806c0f0 2023-10-06T05:07:04.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.569+0000 7f88ced9d700 1 -- 172.21.15.55:0/2086940018 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c80c94a0 con 0x7f88c806c0f0 2023-10-06T05:07:04.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.570+0000 7f88ced9d700 1 -- 172.21.15.55:0/2086940018 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c804c3c0 con 0x7f88c806c0f0 2023-10-06T05:07:04.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.571+0000 7f88bdffb700 1 -- 172.21.15.55:0/2086940018 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 97) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f88c4007690 con 0x7f88c806c0f0 2023-10-06T05:07:04.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.572+0000 7f88bdffb700 1 --2- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f88ac067170 0x7f88ac069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:04.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.572+0000 7f88bdffb700 1 -- 172.21.15.55:0/2086940018 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6736+0+0 (secure 0 0 0) 0x7f88c4084370 con 0x7f88c806c0f0 2023-10-06T05:07:04.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.572+0000 7f88cd59a700 1 -- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f88ac067170 msgr2=0x7f88ac069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:04.574 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.572+0000 7f88cd59a700 1 --2- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f88ac067170 0x7f88ac069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:04.577 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.576+0000 7f88bdffb700 1 -- 172.21.15.55:0/2086940018 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f88c4050ed0 con 0x7f88c806c0f0 2023-10-06T05:07:04.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.773+0000 7f88cd59a700 1 -- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f88ac067170 msgr2=0x7f88ac069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:04.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.773+0000 7f88cd59a700 1 --2- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f88ac067170 0x7f88ac069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:04.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.830+0000 7f88ced9d700 1 -- 172.21.15.55:0/2086940018 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88c80b3d40 con 0x7f88c806c0f0 2023-10-06T05:07:04.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.831+0000 7f88bdffb700 1 -- 172.21.15.55:0/2086940018 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f88c40544f0 con 0x7f88c806c0f0 2023-10-06T05:07:04.833 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:04.833 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8078,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1255308005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1255308005},{"type":"v1","addr":"172.21.15.163:6835","nonce":1255308005}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":9274,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2468487159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2468487159},{"type":"v1","addr":"172.21.15.163:6834","nonce":2468487159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/4105265850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":4105265850},{"type":"v1","addr":"172.21.15.55:6834","nonce":4105265850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:05:40.405710+0000","modified":"2023-10-06T05:05:43.944287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9771":{"gid":9771,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/312761019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":312761019},{"type":"v1","addr":"172.21.15.55:6835","nonce":312761019}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:07:04.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.834+0000 7f88ced9d700 1 -- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f88ac067170 msgr2=0x7f88ac069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:04.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.834+0000 7f88ced9d700 1 --2- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f88ac067170 0x7f88ac069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:04.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.834+0000 7f88ced9d700 1 -- 172.21.15.55:0/2086940018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c806c0f0 msgr2=0x7f88c80c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:04.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.834+0000 7f88ced9d700 1 --2- 172.21.15.55:0/2086940018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c806c0f0 0x7f88c80c99c0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f88c400c590 tx=0x7f88c4007e00 comp rx=0 tx=0).stop 2023-10-06T05:07:04.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.834+0000 7f88ced9d700 1 -- 172.21.15.55:0/2086940018 shutdown_connections 2023-10-06T05:07:04.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.834+0000 7f88ced9d700 1 --2- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f88ac067170 0x7f88ac069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:04.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.834+0000 7f88ced9d700 1 --2- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f88c815d1e0 0x7f88c8169d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:04.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.834+0000 7f88ced9d700 1 --2- 172.21.15.55:0/2086940018 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f88c806c0f0 0x7f88c80c99c0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:04.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.834+0000 7f88ced9d700 1 --2- 172.21.15.55:0/2086940018 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f88c8142ea0 0x7f88c815cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:04.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.834+0000 7f88ced9d700 1 -- 172.21.15.55:0/2086940018 >> 172.21.15.55:0/2086940018 conn(0x7f88c80ce430 msgr2=0x7f88c80b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:04.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.834+0000 7f88ced9d700 1 -- 172.21.15.55:0/2086940018 shutdown_connections 2023-10-06T05:07:04.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:04.834+0000 7f88ced9d700 1 -- 172.21.15.55:0/2086940018 wait complete. 2023-10-06T05:07:04.837 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 96 2023-10-06T05:07:04.855 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:07:05.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.042+0000 7fa85319d700 1 Processor -- start 2023-10-06T05:07:05.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.042+0000 7fa85319d700 1 -- start start 2023-10-06T05:07:05.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.044+0000 7fa85319d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa84c0c8830 0x7fa84c164800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.044+0000 7fa85319d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84c0b1410 con 0x7fa84c1595a0 2023-10-06T05:07:05.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.044+0000 7fa85319d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84c063230 con 0x7fa84c0c6c60 2023-10-06T05:07:05.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.044+0000 7fa85319d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84c0b2b90 con 0x7fa84c0c8830 2023-10-06T05:07:05.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.044+0000 7fa85219b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa84c1595a0 0x7fa84c0b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52780/0 (socket says 172.21.15.55:52780) 2023-10-06T05:07:05.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.044+0000 7fa85219b700 1 -- 172.21.15.55:0/3864668246 learned_addr learned my addr 172.21.15.55:0/3864668246 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:05.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.044+0000 7fa85299c700 1 --2- 172.21.15.55:0/3864668246 >> v2:172.21.15.163:3300/0 conn(0x7fa84c0c8830 0x7fa84c164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:05.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.044+0000 7fa851199700 1 -- 172.21.15.55:0/3864668246 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792622908 0 0) 0x7fa84c0b1410 con 0x7fa84c1595a0 2023-10-06T05:07:05.047 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.044+0000 7fa851199700 1 -- 172.21.15.55:0/3864668246 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa838003570 con 0x7fa84c1595a0 2023-10-06T05:07:05.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.045+0000 7fa851199700 1 -- 172.21.15.55:0/3864668246 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155873573 0 0) 0x7fa84c063230 con 0x7fa84c0c6c60 2023-10-06T05:07:05.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.045+0000 7fa851199700 1 -- 172.21.15.55:0/3864668246 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa84c0b1410 con 0x7fa84c0c6c60 2023-10-06T05:07:05.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.045+0000 7fa85299c700 1 -- 172.21.15.55:0/3864668246 >> v1:172.21.15.163:6789/0 conn(0x7fa84c0c6c60 legacy=0x7fa84c0c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.045+0000 7fa85299c700 1 -- 172.21.15.55:0/3864668246 >> v1:172.21.15.55:6789/0 conn(0x7fa84c1595a0 legacy=0x7fa84c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.045+0000 7fa85299c700 1 -- 172.21.15.55:0/3864668246 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84c0634f0 con 0x7fa84c0c8830 2023-10-06T05:07:05.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.045+0000 7fa85299c700 1 --2- 172.21.15.55:0/3864668246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa84c0c8830 0x7fa84c164800 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa844009e60 tx=0x7fa844009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdd1b5f8aec0db78 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:05.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.045+0000 7fa851199700 1 -- 172.21.15.55:0/3864668246 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84400e070 con 0x7fa84c0c8830 2023-10-06T05:07:05.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.045+0000 7fa851199700 1 -- 172.21.15.55:0/3864668246 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84400bab0 con 0x7fa84c0c8830 2023-10-06T05:07:05.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.045+0000 7fa851199700 1 -- 172.21.15.55:0/3864668246 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84400be70 con 0x7fa84c0c8830 2023-10-06T05:07:05.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.046+0000 7fa85319d700 1 -- 172.21.15.55:0/3864668246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa84c0c8830 msgr2=0x7fa84c164800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.046+0000 7fa85319d700 1 --2- 172.21.15.55:0/3864668246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa84c0c8830 0x7fa84c164800 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa844009e60 tx=0x7fa844009780 comp rx=0 tx=0).stop 2023-10-06T05:07:05.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.046+0000 7fa85319d700 1 -- 172.21.15.55:0/3864668246 shutdown_connections 2023-10-06T05:07:05.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.046+0000 7fa85319d700 1 --2- 172.21.15.55:0/3864668246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa84c0c8830 0x7fa84c164800 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.046+0000 7fa85319d700 1 -- 172.21.15.55:0/3864668246 >> 172.21.15.55:0/3864668246 conn(0x7fa84c06ddb0 msgr2=0x7fa84c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:05.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.046+0000 7fa85319d700 1 -- 172.21.15.55:0/3864668246 shutdown_connections 2023-10-06T05:07:05.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.046+0000 7fa85319d700 1 -- 172.21.15.55:0/3864668246 wait complete. 2023-10-06T05:07:05.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.047+0000 7fa85319d700 1 Processor -- start 2023-10-06T05:07:05.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.048+0000 7fa85319d700 1 -- start start 2023-10-06T05:07:05.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.048+0000 7fa85319d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84c0c6c60 0x7fa84c14e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85319d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa84c0c8830 0x7fa84c13d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85319d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa84c1595a0 0x7fa84c150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85319d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c14c5a0 con 0x7fa84c1595a0 2023-10-06T05:07:05.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85319d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c14c880 con 0x7fa84c0c8830 2023-10-06T05:07:05.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85319d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84c14cb60 con 0x7fa84c0c6c60 2023-10-06T05:07:05.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85299c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa84c1595a0 0x7fa84c150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:05.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85219b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84c0c6c60 0x7fa84c14e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:05.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85299c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa84c1595a0 0x7fa84c150460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50758/0 (socket says 172.21.15.55:50758) 2023-10-06T05:07:05.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85299c700 1 -- 172.21.15.55:0/1946708278 learned_addr learned my addr 172.21.15.55:0/1946708278 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:05.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85199a700 1 --2- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa84c0c8830 0x7fa84c13d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:05.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85299c700 1 -- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84c0c6c60 msgr2=0x7fa84c14e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85299c700 1 --2- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84c0c6c60 0x7fa84c14e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85299c700 1 -- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa84c0c8830 msgr2=0x7fa84c13d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85299c700 1 --2- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa84c0c8830 0x7fa84c13d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.049+0000 7fa85299c700 1 -- 172.21.15.55:0/1946708278 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa844009a20 con 0x7fa84c1595a0 2023-10-06T05:07:05.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.050+0000 7fa85299c700 1 --2- 172.21.15.55:0/1946708278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa84c1595a0 0x7fa84c150460 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fa844013000 tx=0x7fa844019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:05.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.050+0000 7fa8427fc700 1 -- 172.21.15.55:0/1946708278 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844012db0 con 0x7fa84c1595a0 2023-10-06T05:07:05.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.050+0000 7fa8427fc700 1 -- 172.21.15.55:0/1946708278 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa844026480 con 0x7fa84c1595a0 2023-10-06T05:07:05.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.050+0000 7fa85319d700 1 -- 172.21.15.55:0/1946708278 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84c14ce40 con 0x7fa84c1595a0 2023-10-06T05:07:05.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.050+0000 7fa8427fc700 1 -- 172.21.15.55:0/1946708278 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844026840 con 0x7fa84c1595a0 2023-10-06T05:07:05.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.050+0000 7fa85319d700 1 -- 172.21.15.55:0/1946708278 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84c14e840 con 0x7fa84c1595a0 2023-10-06T05:07:05.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.051+0000 7fa85319d700 1 -- 172.21.15.55:0/1946708278 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84c04c3c0 con 0x7fa84c1595a0 2023-10-06T05:07:05.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.052+0000 7fa8427fc700 1 -- 172.21.15.55:0/1946708278 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 97) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fa84401c400 con 0x7fa84c1595a0 2023-10-06T05:07:05.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.053+0000 7fa8427fc700 1 --2- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa830067170 0x7fa830069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.053+0000 7fa8427fc700 1 -- 172.21.15.55:0/1946708278 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6736+0+0 (secure 0 0 0) 0x7fa844086cd0 con 0x7fa84c1595a0 2023-10-06T05:07:05.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.053+0000 7fa85219b700 1 -- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa830067170 msgr2=0x7fa830069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:05.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.053+0000 7fa85219b700 1 --2- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa830067170 0x7fa830069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:05.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.057+0000 7fa8427fc700 1 -- 172.21.15.55:0/1946708278 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa844053720 con 0x7fa84c1595a0 2023-10-06T05:07:05.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.254+0000 7fa85219b700 1 -- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa830067170 msgr2=0x7fa830069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:05.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.254+0000 7fa85219b700 1 --2- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa830067170 0x7fa830069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:05.264 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.262+0000 7fa85319d700 1 -- 172.21.15.55:0/1946708278 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa84c15a070 con 0x7fa84c1595a0 2023-10-06T05:07:05.265 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.263+0000 7fa8427fc700 1 -- 172.21.15.55:0/1946708278 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+17944 (secure 0 0 0) 0x7fa84402b090 con 0x7fa84c1595a0 2023-10-06T05:07:05.265 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:05.266 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":194,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:05:56.105399+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:05:38.605485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:05:39.377918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3244631548":"2023-10-07T05:05:56.105364+0000","172.21.15.163:0/3391882979":"2023-10-07T05:05:56.105364+0000","172.21.15.163:6837/101089":"2023-10-07T05:05:56.105364+0000","172.21.15.163:6836/101089":"2023-10-07T05:05:56.105364+0000","172.21.15.163:0/3718998698":"2023-10-07T05:05:56.105364+0000","172.21.15.163:0/2082256652":"2023-10-07T05:05:56.105364+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:07:05.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.265+0000 7fa85319d700 1 -- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa830067170 msgr2=0x7fa830069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:05.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.265+0000 7fa85319d700 1 --2- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa830067170 0x7fa830069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.265+0000 7fa85319d700 1 -- 172.21.15.55:0/1946708278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa84c1595a0 msgr2=0x7fa84c150460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.265+0000 7fa85319d700 1 --2- 172.21.15.55:0/1946708278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa84c1595a0 0x7fa84c150460 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fa844013000 tx=0x7fa844019000 comp rx=0 tx=0).stop 2023-10-06T05:07:05.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.265+0000 7fa85319d700 1 -- 172.21.15.55:0/1946708278 shutdown_connections 2023-10-06T05:07:05.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.265+0000 7fa85319d700 1 --2- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa830067170 0x7fa830069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.265+0000 7fa85319d700 1 --2- 172.21.15.55:0/1946708278 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa84c1595a0 0x7fa84c150460 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.265+0000 7fa85319d700 1 --2- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa84c0c6c60 0x7fa84c14e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.265+0000 7fa85319d700 1 --2- 172.21.15.55:0/1946708278 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa84c0c8830 0x7fa84c13d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.265+0000 7fa85319d700 1 -- 172.21.15.55:0/1946708278 >> 172.21.15.55:0/1946708278 conn(0x7fa84c06ddb0 msgr2=0x7fa84c0cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:05.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.265+0000 7fa85319d700 1 -- 172.21.15.55:0/1946708278 shutdown_connections 2023-10-06T05:07:05.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.265+0000 7fa85319d700 1 -- 172.21.15.55:0/1946708278 wait complete. 2023-10-06T05:07:05.287 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:05:38.605485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '192', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:05:39.377918+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '193', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:07:05.288 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:07:05.297 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:05.296+0000 7f47bb684180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:07:05.354 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:05.350+0000 7f6aede8b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:07:05.460 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:05.459+0000 7f47bb684180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:07:05.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.485+0000 7f4788f9d700 1 Processor -- start 2023-10-06T05:07:05.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.485+0000 7f4788f9d700 1 -- start start 2023-10-06T05:07:05.492 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.486+0000 7f4788f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f47840b16d0 0x7f47840b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.486+0000 7f4788f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47840b33b0 0x7f47840b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.486+0000 7f4788f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47840b2080 con 0x7f47840b4e70 2023-10-06T05:07:05.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.486+0000 7f4788f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47840cfb00 con 0x7f47840b16d0 2023-10-06T05:07:05.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.486+0000 7f4788f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47840b5c40 con 0x7f47840b33b0 2023-10-06T05:07:05.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.487+0000 7f47837fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f47840b4e70 0x7f47840ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52796/0 (socket says 172.21.15.55:52796) 2023-10-06T05:07:05.493 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.487+0000 7f47837fe700 1 -- 172.21.15.55:0/1127235223 learned_addr learned my addr 172.21.15.55:0/1127235223 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:05.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.487+0000 7f4782ffd700 1 --2- 172.21.15.55:0/1127235223 >> v2:172.21.15.163:3300/0 conn(0x7f47840b16d0 0x7f47840b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:05.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.487+0000 7f4783fff700 1 --2- 172.21.15.55:0/1127235223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47840b33b0 0x7f47840b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:05.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.487+0000 7f47827fc700 1 -- 172.21.15.55:0/1127235223 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1775625730 0 0) 0x7f47840b2080 con 0x7f47840b4e70 2023-10-06T05:07:05.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.487+0000 7f47827fc700 1 -- 172.21.15.55:0/1127235223 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f476c003570 con 0x7f47840b4e70 2023-10-06T05:07:05.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.487+0000 7f4782ffd700 1 -- 172.21.15.55:0/1127235223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47840b33b0 msgr2=0x7f47840b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:05.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.487+0000 7f4782ffd700 1 --2- 172.21.15.55:0/1127235223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47840b33b0 0x7f47840b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.487+0000 7f4782ffd700 1 -- 172.21.15.55:0/1127235223 >> v1:172.21.15.55:6789/0 conn(0x7f47840b4e70 legacy=0x7f47840ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.487+0000 7f4782ffd700 1 -- 172.21.15.55:0/1127235223 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47840b5f20 con 0x7f47840b16d0 2023-10-06T05:07:05.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.487+0000 7f4783fff700 1 --2- 172.21.15.55:0/1127235223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47840b33b0 0x7f47840b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:05.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.488+0000 7f4782ffd700 1 --2- 172.21.15.55:0/1127235223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47840b16d0 0x7f47840b1b40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f47780098e0 tx=0x7f47780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=747d124a2c84e71f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:05.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.488+0000 7f47827fc700 1 -- 172.21.15.55:0/1127235223 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778011630 con 0x7f47840b16d0 2023-10-06T05:07:05.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.488+0000 7f47827fc700 1 -- 172.21.15.55:0/1127235223 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4778011790 con 0x7f47840b16d0 2023-10-06T05:07:05.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.488+0000 7f47827fc700 1 -- 172.21.15.55:0/1127235223 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778011b30 con 0x7f47840b16d0 2023-10-06T05:07:05.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.488+0000 7f4788f9d700 1 -- 172.21.15.55:0/1127235223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47840b16d0 msgr2=0x7f47840b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.488+0000 7f4788f9d700 1 --2- 172.21.15.55:0/1127235223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47840b16d0 0x7f47840b1b40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f47780098e0 tx=0x7f47780091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.488+0000 7f4788f9d700 1 -- 172.21.15.55:0/1127235223 shutdown_connections 2023-10-06T05:07:05.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.488+0000 7f4788f9d700 1 --2- 172.21.15.55:0/1127235223 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47840b33b0 0x7f47840b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.489+0000 7f4788f9d700 1 --2- 172.21.15.55:0/1127235223 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47840b16d0 0x7f47840b1b40 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.489+0000 7f4788f9d700 1 -- 172.21.15.55:0/1127235223 >> 172.21.15.55:0/1127235223 conn(0x7f47840d10c0 msgr2=0x7f47840affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:05.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.489+0000 7f4788f9d700 1 -- 172.21.15.55:0/1127235223 shutdown_connections 2023-10-06T05:07:05.497 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.489+0000 7f4788f9d700 1 -- 172.21.15.55:0/1127235223 wait complete. 2023-10-06T05:07:05.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.490+0000 7f4788f9d700 1 Processor -- start 2023-10-06T05:07:05.498 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.490+0000 7f4788f9d700 1 -- start start 2023-10-06T05:07:05.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.491+0000 7f4788f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47840b16d0 0x7f478415c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.491+0000 7f4788f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47840b33b0 0x7f4784157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.491+0000 7f4788f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47840b4e70 0x7f4784158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.491+0000 7f4788f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478415b8d0 con 0x7f47840b33b0 2023-10-06T05:07:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.491+0000 7f4788f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478415b5f0 con 0x7f47840b16d0 2023-10-06T05:07:05.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.491+0000 7f4788f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4784153e00 con 0x7f47840b4e70 2023-10-06T05:07:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.491+0000 7f4782ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47840b33b0 0x7f4784157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.491+0000 7f4782ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47840b33b0 0x7f4784157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50760/0 (socket says 172.21.15.55:50760) 2023-10-06T05:07:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.491+0000 7f47837fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47840b16d0 0x7f478415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.491+0000 7f4782ffd700 1 -- 172.21.15.55:0/89412508 learned_addr learned my addr 172.21.15.55:0/89412508 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.491+0000 7f4783fff700 1 --2- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47840b4e70 0x7f4784158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:05.506 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.492+0000 7f4782ffd700 1 -- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47840b4e70 msgr2=0x7f4784158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.492+0000 7f4782ffd700 1 --2- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47840b4e70 0x7f4784158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.492+0000 7f4782ffd700 1 -- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47840b16d0 msgr2=0x7f478415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.492+0000 7f4782ffd700 1 --2- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47840b16d0 0x7f478415c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.492+0000 7f4782ffd700 1 -- 172.21.15.55:0/89412508 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c009a20 con 0x7f47840b33b0 2023-10-06T05:07:05.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.492+0000 7f4782ffd700 1 --2- 172.21.15.55:0/89412508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47840b33b0 0x7f4784157d30 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f4778009ed0 tx=0x7f4778017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:05.507 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.492+0000 7f476b7fe700 1 -- 172.21.15.55:0/89412508 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778000d10 con 0x7f47840b33b0 2023-10-06T05:07:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.492+0000 7f476b7fe700 1 -- 172.21.15.55:0/89412508 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4778011b10 con 0x7f47840b33b0 2023-10-06T05:07:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.492+0000 7f476b7fe700 1 -- 172.21.15.55:0/89412508 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778025620 con 0x7f47840b33b0 2023-10-06T05:07:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.492+0000 7f4788f9d700 1 -- 172.21.15.55:0/89412508 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4778009420 con 0x7f47840b33b0 2023-10-06T05:07:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.493+0000 7f4788f9d700 1 -- 172.21.15.55:0/89412508 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47841543c0 con 0x7f47840b33b0 2023-10-06T05:07:05.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.494+0000 7f4788f9d700 1 -- 172.21.15.55:0/89412508 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4784153c60 con 0x7f47840b33b0 2023-10-06T05:07:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.495+0000 7f476b7fe700 1 -- 172.21.15.55:0/89412508 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 97) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f4778025a50 con 0x7f47840b33b0 2023-10-06T05:07:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.496+0000 7f476b7fe700 1 --2- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4770067170 0x7f4770069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.496+0000 7f476b7fe700 1 -- 172.21.15.55:0/89412508 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6736+0+0 (secure 0 0 0) 0x7f47780863a0 con 0x7f47840b33b0 2023-10-06T05:07:05.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.496+0000 7f47837fe700 1 -- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4770067170 msgr2=0x7f4770069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:05.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.496+0000 7f47837fe700 1 --2- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4770067170 0x7f4770069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:05.510 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.500+0000 7f476b7fe700 1 -- 172.21.15.55:0/89412508 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4784153c60 con 0x7f47840b33b0 2023-10-06T05:07:05.525 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:05.523+0000 7f6aede8b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:07:05.548 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:05.546+0000 7f47bb684180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:07:05.614 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:05.613+0000 7f6aede8b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:07:05.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.697+0000 7f47837fe700 1 -- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4770067170 msgr2=0x7f4770069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:05.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.697+0000 7f47837fe700 1 --2- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4770067170 0x7f4770069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.703+0000 7f4788f9d700 1 -- 172.21.15.55:0/89412508 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4784153c60 con 0x7f47840b33b0 2023-10-06T05:07:05.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.704+0000 7f476b7fe700 1 -- 172.21.15.55:0/89412508 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v194) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4784153c60 con 0x7f47840b33b0 2023-10-06T05:07:05.707 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T05:07:05.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.706+0000 7f4788f9d700 1 -- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4770067170 msgr2=0x7f4770069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:05.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.706+0000 7f4788f9d700 1 --2- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4770067170 0x7f4770069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.706+0000 7f4788f9d700 1 -- 172.21.15.55:0/89412508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47840b33b0 msgr2=0x7f4784157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.706+0000 7f4788f9d700 1 --2- 172.21.15.55:0/89412508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47840b33b0 0x7f4784157d30 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f4778009ed0 tx=0x7f4778017000 comp rx=0 tx=0).stop 2023-10-06T05:07:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.706+0000 7f4788f9d700 1 -- 172.21.15.55:0/89412508 shutdown_connections 2023-10-06T05:07:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.706+0000 7f4788f9d700 1 --2- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4770067170 0x7f4770069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.706+0000 7f4788f9d700 1 --2- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f47840b4e70 0x7f4784158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.706+0000 7f4788f9d700 1 --2- 172.21.15.55:0/89412508 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f47840b16d0 0x7f478415c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.706+0000 7f4788f9d700 1 --2- 172.21.15.55:0/89412508 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f47840b33b0 0x7f4784157d30 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.706+0000 7f4788f9d700 1 -- 172.21.15.55:0/89412508 >> 172.21.15.55:0/89412508 conn(0x7f47840d10c0 msgr2=0x7f4784159e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.706+0000 7f4788f9d700 1 -- 172.21.15.55:0/89412508 shutdown_connections 2023-10-06T05:07:05.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.706+0000 7f4788f9d700 1 -- 172.21.15.55:0/89412508 wait complete. 2023-10-06T05:07:05.727 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:07:05.774 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:05.772+0000 7f47bb684180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:07:05.850 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:05.848+0000 7f6aede8b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:07:05.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.920+0000 7f337d19d700 1 Processor -- start 2023-10-06T05:07:05.923 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.921+0000 7f337d19d700 1 -- start start 2023-10-06T05:07:05.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.922+0000 7f337d19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f33780b1b40 0x7f33780b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.922+0000 7f337d19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33780cfb00 con 0x7f33780b16d0 2023-10-06T05:07:05.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.922+0000 7f337d19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33780b5c40 con 0x7f33780b4e70 2023-10-06T05:07:05.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.922+0000 7f337d19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33780b5f20 con 0x7f33780b1b40 2023-10-06T05:07:05.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.922+0000 7f33777fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f33780b16d0 0x7f33780b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52806/0 (socket says 172.21.15.55:52806) 2023-10-06T05:07:05.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.922+0000 7f33777fe700 1 -- 172.21.15.55:0/2327821554 learned_addr learned my addr 172.21.15.55:0/2327821554 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:05.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.923+0000 7f337c99c700 1 --2- 172.21.15.55:0/2327821554 >> v2:172.21.15.163:3300/0 conn(0x7f33780b1b40 0x7f33780b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:05.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.923+0000 7f3376ffd700 1 -- 172.21.15.55:0/2327821554 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196699954 0 0) 0x7f33780cfb00 con 0x7f33780b16d0 2023-10-06T05:07:05.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.923+0000 7f3376ffd700 1 -- 172.21.15.55:0/2327821554 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3360003570 con 0x7f33780b16d0 2023-10-06T05:07:05.928 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.923+0000 7f3376ffd700 1 -- 172.21.15.55:0/2327821554 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450568861 0 0) 0x7f33780b5c40 con 0x7f33780b4e70 2023-10-06T05:07:05.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.923+0000 7f337c99c700 1 -- 172.21.15.55:0/2327821554 >> v1:172.21.15.163:6789/0 conn(0x7f33780b4e70 legacy=0x7f33780ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.923+0000 7f337c99c700 1 -- 172.21.15.55:0/2327821554 >> v1:172.21.15.55:6789/0 conn(0x7f33780b16d0 legacy=0x7f33780b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.923+0000 7f337c99c700 1 -- 172.21.15.55:0/2327821554 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33780b6200 con 0x7f33780b1b40 2023-10-06T05:07:05.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.923+0000 7f337c99c700 1 --2- 172.21.15.55:0/2327821554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33780b1b40 0x7f33780b3ac0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3370009e60 tx=0x7f3370009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9e68ff39852f8f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:05.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.924+0000 7f3376ffd700 1 -- 172.21.15.55:0/2327821554 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337000e070 con 0x7f33780b1b40 2023-10-06T05:07:05.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.924+0000 7f3376ffd700 1 -- 172.21.15.55:0/2327821554 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337000bb60 con 0x7f33780b1b40 2023-10-06T05:07:05.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.924+0000 7f3376ffd700 1 -- 172.21.15.55:0/2327821554 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370012430 con 0x7f33780b1b40 2023-10-06T05:07:05.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.924+0000 7f337d19d700 1 -- 172.21.15.55:0/2327821554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33780b1b40 msgr2=0x7f33780b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.924+0000 7f337d19d700 1 --2- 172.21.15.55:0/2327821554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33780b1b40 0x7f33780b3ac0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3370009e60 tx=0x7f3370009780 comp rx=0 tx=0).stop 2023-10-06T05:07:05.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.924+0000 7f337d19d700 1 -- 172.21.15.55:0/2327821554 shutdown_connections 2023-10-06T05:07:05.929 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.924+0000 7f337d19d700 1 --2- 172.21.15.55:0/2327821554 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33780b1b40 0x7f33780b3ac0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.924+0000 7f337d19d700 1 -- 172.21.15.55:0/2327821554 >> 172.21.15.55:0/2327821554 conn(0x7f33780d10c0 msgr2=0x7f33780affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:05.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.924+0000 7f337d19d700 1 -- 172.21.15.55:0/2327821554 shutdown_connections 2023-10-06T05:07:05.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.924+0000 7f337d19d700 1 -- 172.21.15.55:0/2327821554 wait complete. 2023-10-06T05:07:05.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.926+0000 7f337d19d700 1 Processor -- start 2023-10-06T05:07:05.930 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.926+0000 7f337d19d700 1 -- start start 2023-10-06T05:07:05.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.927+0000 7f337d19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33780b16d0 0x7f3378157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.927+0000 7f337d19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33780b1b40 0x7f33780c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.927+0000 7f337d19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33780b4e70 0x7f337815b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.927+0000 7f337d19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33780c6760 con 0x7f33780b16d0 2023-10-06T05:07:05.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.927+0000 7f337d19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3378158050 con 0x7f33780b4e70 2023-10-06T05:07:05.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.927+0000 7f337d19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33780c9460 con 0x7f33780b1b40 2023-10-06T05:07:05.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.927+0000 7f3377fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33780b16d0 0x7f3378157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:05.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.927+0000 7f33777fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33780b1b40 0x7f33780c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:05.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.927+0000 7f3377fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33780b16d0 0x7f3378157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50776/0 (socket says 172.21.15.55:50776) 2023-10-06T05:07:05.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.927+0000 7f337c99c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33780b4e70 0x7f337815b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:05.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.927+0000 7f3377fff700 1 -- 172.21.15.55:0/2267521493 learned_addr learned my addr 172.21.15.55:0/2267521493 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:05.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.928+0000 7f3377fff700 1 -- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33780b1b40 msgr2=0x7f33780c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.928+0000 7f3377fff700 1 --2- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33780b1b40 0x7f33780c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.928+0000 7f3377fff700 1 -- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33780b4e70 msgr2=0x7f337815b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:05.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.928+0000 7f3377fff700 1 --2- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33780b4e70 0x7f337815b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:05.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.928+0000 7f3377fff700 1 -- 172.21.15.55:0/2267521493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3370009a20 con 0x7f33780b16d0 2023-10-06T05:07:05.934 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.928+0000 7f33777fe700 1 --2- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33780b1b40 0x7f33780c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:05.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.928+0000 7f3377fff700 1 --2- 172.21.15.55:0/2267521493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33780b16d0 0x7f3378157b10 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f336800c560 tx=0x7f336800c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:05.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.928+0000 7f335ffff700 1 -- 172.21.15.55:0/2267521493 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3368011590 con 0x7f33780b16d0 2023-10-06T05:07:05.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.929+0000 7f335ffff700 1 -- 172.21.15.55:0/2267521493 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3368011710 con 0x7f33780b16d0 2023-10-06T05:07:05.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.929+0000 7f335ffff700 1 -- 172.21.15.55:0/2267521493 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3368011ad0 con 0x7f33780b16d0 2023-10-06T05:07:05.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.929+0000 7f337d19d700 1 -- 172.21.15.55:0/2267521493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33780c5be0 con 0x7f33780b16d0 2023-10-06T05:07:05.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.929+0000 7f337d19d700 1 -- 172.21.15.55:0/2267521493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33780c61a0 con 0x7f33780b16d0 2023-10-06T05:07:05.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.930+0000 7f337d19d700 1 -- 172.21.15.55:0/2267521493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3378088990 con 0x7f33780b16d0 2023-10-06T05:07:05.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.932+0000 7f335ffff700 1 -- 172.21.15.55:0/2267521493 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 97) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f336801b3b0 con 0x7f33780b16d0 2023-10-06T05:07:05.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.933+0000 7f335ffff700 1 --2- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3364067170 0x7f3364069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:05.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.933+0000 7f335ffff700 1 -- 172.21.15.55:0/2267521493 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6736+0+0 (secure 0 0 0) 0x7f3368084180 con 0x7f33780b16d0 2023-10-06T05:07:05.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.933+0000 7f33777fe700 1 -- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3364067170 msgr2=0x7f3364069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:05.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.933+0000 7f33777fe700 1 --2- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3364067170 0x7f3364069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:05.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:05.937+0000 7f335ffff700 1 -- 172.21.15.55:0/2267521493 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3368050ce0 con 0x7f33780b16d0 2023-10-06T05:07:05.951 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:05.950+0000 7f47bb684180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:07:06.045 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:06.043+0000 7f6aede8b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:07:06.054 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:06.050+0000 7f47bb684180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:07:06.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.134+0000 7f33777fe700 1 -- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3364067170 msgr2=0x7f3364069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:06.136 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.134+0000 7f33777fe700 1 --2- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3364067170 0x7f3364069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:06.145 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:06.144+0000 7f6aede8b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:07:06.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.148+0000 7f337d19d700 1 -- 172.21.15.55:0/2267521493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33780cd820 con 0x7f33780b16d0 2023-10-06T05:07:06.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.148+0000 7f335ffff700 1 -- 172.21.15.55:0/2267521493 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v195) v1 ==== 93+0+10 (secure 0 0 0) 0x7f33680170a0 con 0x7f33780b16d0 2023-10-06T05:07:06.150 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T05:07:06.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.150+0000 7f337d19d700 1 -- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3364067170 msgr2=0x7f3364069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:06.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.150+0000 7f337d19d700 1 --2- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3364067170 0x7f3364069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.150+0000 7f337d19d700 1 -- 172.21.15.55:0/2267521493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33780b16d0 msgr2=0x7f3378157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:06.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.150+0000 7f337d19d700 1 --2- 172.21.15.55:0/2267521493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33780b16d0 0x7f3378157b10 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f336800c560 tx=0x7f336800c870 comp rx=0 tx=0).stop 2023-10-06T05:07:06.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.151+0000 7f337d19d700 1 -- 172.21.15.55:0/2267521493 shutdown_connections 2023-10-06T05:07:06.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.151+0000 7f337d19d700 1 --2- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3364067170 0x7f3364069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.151+0000 7f337d19d700 1 --2- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f33780b4e70 0x7f337815b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.151+0000 7f337d19d700 1 --2- 172.21.15.55:0/2267521493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f33780b16d0 0x7f3378157b10 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.153 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.151+0000 7f337d19d700 1 --2- 172.21.15.55:0/2267521493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f33780b1b40 0x7f33780c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.151+0000 7f337d19d700 1 -- 172.21.15.55:0/2267521493 >> 172.21.15.55:0/2267521493 conn(0x7f33780d10c0 msgr2=0x7f33780b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.151+0000 7f337d19d700 1 -- 172.21.15.55:0/2267521493 shutdown_connections 2023-10-06T05:07:06.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.151+0000 7f337d19d700 1 -- 172.21.15.55:0/2267521493 wait complete. 2023-10-06T05:07:06.171 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:07:06.216 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:06.215+0000 7f47bb684180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:07:06.324 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:06.323+0000 7f47bb684180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:07:06.324 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:06.322+0000 7f6aede8b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:07:06.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.374+0000 7fcb4199d700 1 Processor -- start 2023-10-06T05:07:06.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.374+0000 7fcb4199d700 1 -- start start 2023-10-06T05:07:06.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.375+0000 7fcb4199d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcb3c0aef00 0x7fcb3c15d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:06.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.375+0000 7fcb4199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb3c06e470 0x7fcb3c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:06.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.375+0000 7fcb4199d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3c0aea70 con 0x7fcb3c15a3b0 2023-10-06T05:07:06.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.375+0000 7fcb4199d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3c06ee30 con 0x7fcb3c0aef00 2023-10-06T05:07:06.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.375+0000 7fcb4199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c0ae480 con 0x7fcb3c06e470 2023-10-06T05:07:06.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.375+0000 7fcb4099b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcb3c15a3b0 0x7fcb3c15a780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52818/0 (socket says 172.21.15.55:52818) 2023-10-06T05:07:06.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.375+0000 7fcb4099b700 1 -- 172.21.15.55:0/1179980147 learned_addr learned my addr 172.21.15.55:0/1179980147 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:06.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.376+0000 7fcb3bfff700 1 --2- 172.21.15.55:0/1179980147 >> v2:172.21.15.163:3300/0 conn(0x7fcb3c0aef00 0x7fcb3c15d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:06.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.376+0000 7fcb4119c700 1 --2- 172.21.15.55:0/1179980147 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb3c06e470 0x7fcb3c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:06.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.376+0000 7fcb3b7fe700 1 -- 172.21.15.55:0/1179980147 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181276262 0 0) 0x7fcb3c0aea70 con 0x7fcb3c15a3b0 2023-10-06T05:07:06.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.376+0000 7fcb3b7fe700 1 -- 172.21.15.55:0/1179980147 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb2c003570 con 0x7fcb3c15a3b0 2023-10-06T05:07:06.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.376+0000 7fcb4119c700 1 -- 172.21.15.55:0/1179980147 >> v2:172.21.15.163:3300/0 conn(0x7fcb3c0aef00 msgr2=0x7fcb3c15d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:06.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.376+0000 7fcb4119c700 1 --2- 172.21.15.55:0/1179980147 >> v2:172.21.15.163:3300/0 conn(0x7fcb3c0aef00 0x7fcb3c15d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.376+0000 7fcb4119c700 1 -- 172.21.15.55:0/1179980147 >> v1:172.21.15.55:6789/0 conn(0x7fcb3c15a3b0 legacy=0x7fcb3c15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:06.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.376+0000 7fcb4119c700 1 -- 172.21.15.55:0/1179980147 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c15dd30 con 0x7fcb3c06e470 2023-10-06T05:07:06.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.376+0000 7fcb3bfff700 1 --2- 172.21.15.55:0/1179980147 >> v2:172.21.15.163:3300/0 conn(0x7fcb3c0aef00 0x7fcb3c15d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:06.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.376+0000 7fcb4119c700 1 --2- 172.21.15.55:0/1179980147 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb3c06e470 0x7fcb3c06e8f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009e60 tx=0x7fcb34009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c506272919d4a770 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:06.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.377+0000 7fcb3b7fe700 1 -- 172.21.15.55:0/1179980147 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400e070 con 0x7fcb3c06e470 2023-10-06T05:07:06.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.377+0000 7fcb3b7fe700 1 -- 172.21.15.55:0/1179980147 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3400bab0 con 0x7fcb3c06e470 2023-10-06T05:07:06.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.377+0000 7fcb3b7fe700 1 -- 172.21.15.55:0/1179980147 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400be70 con 0x7fcb3c06e470 2023-10-06T05:07:06.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.377+0000 7fcb4199d700 1 -- 172.21.15.55:0/1179980147 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb3c06e470 msgr2=0x7fcb3c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:06.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.377+0000 7fcb4199d700 1 --2- 172.21.15.55:0/1179980147 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb3c06e470 0x7fcb3c06e8f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009e60 tx=0x7fcb34009780 comp rx=0 tx=0).stop 2023-10-06T05:07:06.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.377+0000 7fcb4199d700 1 -- 172.21.15.55:0/1179980147 shutdown_connections 2023-10-06T05:07:06.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.377+0000 7fcb4199d700 1 --2- 172.21.15.55:0/1179980147 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb3c06e470 0x7fcb3c06e8f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.377+0000 7fcb4199d700 1 --2- 172.21.15.55:0/1179980147 >> v2:172.21.15.163:3300/0 conn(0x7fcb3c0aef00 0x7fcb3c15d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.377+0000 7fcb4199d700 1 -- 172.21.15.55:0/1179980147 >> 172.21.15.55:0/1179980147 conn(0x7fcb3c0c9f70 msgr2=0x7fcb3c06dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:06.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.378+0000 7fcb4199d700 1 -- 172.21.15.55:0/1179980147 shutdown_connections 2023-10-06T05:07:06.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.378+0000 7fcb4199d700 1 -- 172.21.15.55:0/1179980147 wait complete. 2023-10-06T05:07:06.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.379+0000 7fcb4199d700 1 Processor -- start 2023-10-06T05:07:06.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.379+0000 7fcb4199d700 1 -- start start 2023-10-06T05:07:06.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.380+0000 7fcb4199d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb3c06e470 0x7fcb3c0c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:06.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.380+0000 7fcb4199d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb3c0aef00 0x7fcb3c153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:06.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.380+0000 7fcb4199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb3c15a3b0 0x7fcb3c1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:06.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.380+0000 7fcb4199d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c158620 con 0x7fcb3c0aef00 2023-10-06T05:07:06.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.380+0000 7fcb4199d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c153b70 con 0x7fcb3c06e470 2023-10-06T05:07:06.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.380+0000 7fcb4199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c0c63c0 con 0x7fcb3c15a3b0 2023-10-06T05:07:06.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.380+0000 7fcb3bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb3c0aef00 0x7fcb3c153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:06.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.380+0000 7fcb3bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb3c0aef00 0x7fcb3c153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50778/0 (socket says 172.21.15.55:50778) 2023-10-06T05:07:06.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.380+0000 7fcb3bfff700 1 -- 172.21.15.55:0/1340128029 learned_addr learned my addr 172.21.15.55:0/1340128029 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:06.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.380+0000 7fcb4099b700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb3c06e470 0x7fcb3c0c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:06.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.381+0000 7fcb4119c700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb3c15a3b0 0x7fcb3c1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:06.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.381+0000 7fcb3bfff700 1 -- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb3c15a3b0 msgr2=0x7fcb3c1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:06.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.381+0000 7fcb3bfff700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb3c15a3b0 0x7fcb3c1564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.381+0000 7fcb3bfff700 1 -- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb3c06e470 msgr2=0x7fcb3c0c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:06.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.381+0000 7fcb3bfff700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb3c06e470 0x7fcb3c0c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.381+0000 7fcb3bfff700 1 -- 172.21.15.55:0/1340128029 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb28009420 con 0x7fcb3c0aef00 2023-10-06T05:07:06.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.381+0000 7fcb4099b700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb3c06e470 0x7fcb3c0c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:06.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.381+0000 7fcb3bfff700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb3c0aef00 0x7fcb3c153630 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800ec80 tx=0x7fcb280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:06.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.381+0000 7fcb38ff9700 1 -- 172.21.15.55:0/1340128029 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800c920 con 0x7fcb3c0aef00 2023-10-06T05:07:06.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.381+0000 7fcb38ff9700 1 -- 172.21.15.55:0/1340128029 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2800ca80 con 0x7fcb3c0aef00 2023-10-06T05:07:06.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.381+0000 7fcb38ff9700 1 -- 172.21.15.55:0/1340128029 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2800ce00 con 0x7fcb3c0aef00 2023-10-06T05:07:06.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.382+0000 7fcb4199d700 1 -- 172.21.15.55:0/1340128029 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb34009a20 con 0x7fcb3c0aef00 2023-10-06T05:07:06.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.382+0000 7fcb4199d700 1 -- 172.21.15.55:0/1340128029 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3c1582b0 con 0x7fcb3c0aef00 2023-10-06T05:07:06.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.383+0000 7fcb4199d700 1 -- 172.21.15.55:0/1340128029 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3c0c4290 con 0x7fcb3c0aef00 2023-10-06T05:07:06.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.385+0000 7fcb38ff9700 1 -- 172.21.15.55:0/1340128029 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 97) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fcb28013680 con 0x7fcb3c0aef00 2023-10-06T05:07:06.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.385+0000 7fcb38ff9700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcb20067170 0x7fcb20069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:06.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.386+0000 7fcb38ff9700 1 -- 172.21.15.55:0/1340128029 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6792+0+0 (secure 0 0 0) 0x7fcb28085440 con 0x7fcb3c0aef00 2023-10-06T05:07:06.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.386+0000 7fcb4099b700 1 -- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcb20067170 msgr2=0x7fcb20069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:06.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.386+0000 7fcb4099b700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcb20067170 0x7fcb20069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:06.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.393+0000 7fcb38ff9700 1 -- 172.21.15.55:0/1340128029 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb3c0c4290 con 0x7fcb3c0aef00 2023-10-06T05:07:06.438 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:06.436+0000 7f6aede8b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:07:06.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.587+0000 7fcb4099b700 1 -- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcb20067170 msgr2=0x7fcb20069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:06.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.587+0000 7fcb4099b700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcb20067170 0x7fcb20069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:06.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.595+0000 7fcb4199d700 1 -- 172.21.15.55:0/1340128029 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcb3c0c4290 con 0x7fcb3c0aef00 2023-10-06T05:07:06.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.596+0000 7fcb38ff9700 1 -- 172.21.15.55:0/1340128029 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v195) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcb3c0c4290 con 0x7fcb3c0aef00 2023-10-06T05:07:06.598 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T05:07:06.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.598+0000 7fcb4199d700 1 -- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcb20067170 msgr2=0x7fcb20069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:06.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.598+0000 7fcb4199d700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcb20067170 0x7fcb20069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.598+0000 7fcb4199d700 1 -- 172.21.15.55:0/1340128029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb3c0aef00 msgr2=0x7fcb3c153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:06.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.598+0000 7fcb4199d700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb3c0aef00 0x7fcb3c153630 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800ec80 tx=0x7fcb280091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.598+0000 7fcb4199d700 1 -- 172.21.15.55:0/1340128029 shutdown_connections 2023-10-06T05:07:06.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.598+0000 7fcb4199d700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcb20067170 0x7fcb20069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.598+0000 7fcb4199d700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcb3c15a3b0 0x7fcb3c1564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.598+0000 7fcb4199d700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcb3c06e470 0x7fcb3c0c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.598+0000 7fcb4199d700 1 --2- 172.21.15.55:0/1340128029 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcb3c0aef00 0x7fcb3c153630 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.598+0000 7fcb4199d700 1 -- 172.21.15.55:0/1340128029 >> 172.21.15.55:0/1340128029 conn(0x7fcb3c0c9f70 msgr2=0x7fcb3c15d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:06.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.598+0000 7fcb4199d700 1 -- 172.21.15.55:0/1340128029 shutdown_connections 2023-10-06T05:07:06.601 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.598+0000 7fcb4199d700 1 -- 172.21.15.55:0/1340128029 wait complete. 2023-10-06T05:07:06.619 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:07:06.809 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:06.807+0000 7f47bb684180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:07:06.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.815+0000 7fc6b49fe700 1 Processor -- start 2023-10-06T05:07:06.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.815+0000 7fc6b49fe700 1 -- start start 2023-10-06T05:07:06.819 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.817+0000 7fc6b49fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc6b00b4e70 0x7fc6b00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:06.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.817+0000 7fc6b49fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc6b00b16d0 0x7fc6b00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:06.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.817+0000 7fc6b49fe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b00b2080 con 0x7fc6b015d240 2023-10-06T05:07:06.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.817+0000 7fc6b49fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b00cfb00 con 0x7fc6b00b4e70 2023-10-06T05:07:06.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.817+0000 7fc6b49fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b00b5c40 con 0x7fc6b00b16d0 2023-10-06T05:07:06.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.817+0000 7fc6aed9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc6b00b4e70 0x7fc6b00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:06.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.817+0000 7fc6aed9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc6b00b4e70 0x7fc6b00ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50780/0 (socket says 172.21.15.55:50780) 2023-10-06T05:07:06.820 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.817+0000 7fc6aed9d700 1 -- 172.21.15.55:0/4293636469 learned_addr learned my addr 172.21.15.55:0/4293636469 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:06.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.817+0000 7fc6ae59c700 1 --2- 172.21.15.55:0/4293636469 >> v2:172.21.15.163:3300/0 conn(0x7fc6b00b16d0 0x7fc6b00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:06.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.817+0000 7fc6aed9d700 1 -- 172.21.15.55:0/4293636469 >> v2:172.21.15.163:3300/0 conn(0x7fc6b00b16d0 msgr2=0x7fc6b00b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:06.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.817+0000 7fc6aed9d700 1 --2- 172.21.15.55:0/4293636469 >> v2:172.21.15.163:3300/0 conn(0x7fc6b00b16d0 0x7fc6b00b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.818+0000 7fc6aed9d700 1 -- 172.21.15.55:0/4293636469 >> v1:172.21.15.163:6789/0 conn(0x7fc6b015d240 legacy=0x7fc6b015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:06.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.818+0000 7fc6aed9d700 1 -- 172.21.15.55:0/4293636469 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b00b5f20 con 0x7fc6b00b4e70 2023-10-06T05:07:06.821 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.818+0000 7fc6aed9d700 1 --2- 172.21.15.55:0/4293636469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b00b4e70 0x7fc6b00ccff0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0089830 tx=0x7fc6a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69cbc773dc30dac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:06.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.818+0000 7fc6add9b700 1 -- 172.21.15.55:0/4293636469 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0009e70 con 0x7fc6b00b4e70 2023-10-06T05:07:06.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.818+0000 7fc6add9b700 1 -- 172.21.15.55:0/4293636469 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a000b860 con 0x7fc6b00b4e70 2023-10-06T05:07:06.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.818+0000 7fc6add9b700 1 -- 172.21.15.55:0/4293636469 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a000bc00 con 0x7fc6b00b4e70 2023-10-06T05:07:06.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.819+0000 7fc6b49fe700 1 -- 172.21.15.55:0/4293636469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b00b4e70 msgr2=0x7fc6b00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:06.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.819+0000 7fc6b49fe700 1 --2- 172.21.15.55:0/4293636469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b00b4e70 0x7fc6b00ccff0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fc6b0089830 tx=0x7fc6a0009250 comp rx=0 tx=0).stop 2023-10-06T05:07:06.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.819+0000 7fc6b49fe700 1 -- 172.21.15.55:0/4293636469 shutdown_connections 2023-10-06T05:07:06.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.819+0000 7fc6b49fe700 1 --2- 172.21.15.55:0/4293636469 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b00b4e70 0x7fc6b00ccff0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.825 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.819+0000 7fc6b49fe700 1 --2- 172.21.15.55:0/4293636469 >> v2:172.21.15.163:3300/0 conn(0x7fc6b00b16d0 0x7fc6b00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.819+0000 7fc6b49fe700 1 -- 172.21.15.55:0/4293636469 >> 172.21.15.55:0/4293636469 conn(0x7fc6b00d10c0 msgr2=0x7fc6b00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.819+0000 7fc6b49fe700 1 -- 172.21.15.55:0/4293636469 shutdown_connections 2023-10-06T05:07:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.819+0000 7fc6b49fe700 1 -- 172.21.15.55:0/4293636469 wait complete. 2023-10-06T05:07:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.820+0000 7fc6b49fe700 1 Processor -- start 2023-10-06T05:07:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.820+0000 7fc6b49fe700 1 -- start start 2023-10-06T05:07:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.821+0000 7fc6b49fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6b00b16d0 0x7fc6b00c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.821+0000 7fc6b49fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6b00b4e70 0x7fc6b0157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.821+0000 7fc6b49fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b015d240 0x7fc6b015e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:06.826 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.821+0000 7fc6b49fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b016d420 con 0x7fc6b015d240 2023-10-06T05:07:06.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.821+0000 7fc6b49fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b0157cb0 con 0x7fc6b00b16d0 2023-10-06T05:07:06.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.821+0000 7fc6b49fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b016d0f0 con 0x7fc6b00b4e70 2023-10-06T05:07:06.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.822+0000 7fc6af59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b015d240 0x7fc6b015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:06.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.822+0000 7fc6aed9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6b00b16d0 0x7fc6b00c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:06.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.822+0000 7fc6af59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b015d240 0x7fc6b015e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50786/0 (socket says 172.21.15.55:50786) 2023-10-06T05:07:06.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.822+0000 7fc6af59e700 1 -- 172.21.15.55:0/2749367112 learned_addr learned my addr 172.21.15.55:0/2749367112 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:06.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.822+0000 7fc6ae59c700 1 --2- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6b00b4e70 0x7fc6b0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:06.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.822+0000 7fc6af59e700 1 -- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6b00b4e70 msgr2=0x7fc6b0157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.822+0000 7fc6af59e700 1 --2- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6b00b4e70 0x7fc6b0157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.822+0000 7fc6af59e700 1 -- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6b00b16d0 msgr2=0x7fc6b00c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.822+0000 7fc6af59e700 1 --2- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6b00b16d0 0x7fc6b00c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.822+0000 7fc6af59e700 1 -- 172.21.15.55:0/2749367112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a00094f0 con 0x7fc6b015d240 2023-10-06T05:07:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.822+0000 7fc6af59e700 1 --2- 172.21.15.55:0/2749367112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b015d240 0x7fc6b015e630 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800c840 tx=0x7fc6a8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.823+0000 7fc696ffd700 1 -- 172.21.15.55:0/2749367112 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8012630 con 0x7fc6b015d240 2023-10-06T05:07:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.823+0000 7fc696ffd700 1 -- 172.21.15.55:0/2749367112 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a80127b0 con 0x7fc6b015d240 2023-10-06T05:07:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.823+0000 7fc696ffd700 1 -- 172.21.15.55:0/2749367112 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8012b70 con 0x7fc6b015d240 2023-10-06T05:07:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.823+0000 7fc6b49fe700 1 -- 172.21.15.55:0/2749367112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b0158720 con 0x7fc6b015d240 2023-10-06T05:07:06.828 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.823+0000 7fc6b49fe700 1 -- 172.21.15.55:0/2749367112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b00c9ed0 con 0x7fc6b015d240 2023-10-06T05:07:06.829 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.824+0000 7fc6b49fe700 1 -- 172.21.15.55:0/2749367112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b016d590 con 0x7fc6b015d240 2023-10-06T05:07:06.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.825+0000 7fc696ffd700 1 -- 172.21.15.55:0/2749367112 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 97) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fc6a801a3a0 con 0x7fc6b015d240 2023-10-06T05:07:06.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.827+0000 7fc696ffd700 1 --2- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc69c067170 0x7fc69c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:06.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.827+0000 7fc696ffd700 1 -- 172.21.15.55:0/2749367112 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6792+0+0 (secure 0 0 0) 0x7fc6a8084ea0 con 0x7fc6b015d240 2023-10-06T05:07:06.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.827+0000 7fc6aed9d700 1 -- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc69c067170 msgr2=0x7fc69c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:06.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.827+0000 7fc6aed9d700 1 --2- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc69c067170 0x7fc69c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:06.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:06.831+0000 7fc696ffd700 1 -- 172.21.15.55:0/2749367112 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6a80519a0 con 0x7fc6b015d240 2023-10-06T05:07:06.904 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:06.903+0000 7f47bb684180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:07:06.996 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:06.995+0000 7f47bb684180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:07:07.011 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:07.010+0000 7f6aede8b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:07:07.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.028+0000 7fc6aed9d700 1 -- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc69c067170 msgr2=0x7fc69c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:07.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.028+0000 7fc6aed9d700 1 --2- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc69c067170 0x7fc69c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:07.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.109+0000 7fc6b49fe700 1 -- 172.21.15.55:0/2749367112 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6b004bb80 con 0x7fc6b015d240 2023-10-06T05:07:07.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.109+0000 7fc696ffd700 1 -- 172.21.15.55:0/2749367112 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v195) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6a8054fc0 con 0x7fc6b015d240 2023-10-06T05:07:07.111 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:07.109+0000 7f6aede8b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:07:07.112 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T05:07:07.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.112+0000 7fc6b49fe700 1 -- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc69c067170 msgr2=0x7fc69c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:07.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.112+0000 7fc6b49fe700 1 --2- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc69c067170 0x7fc69c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.112+0000 7fc6b49fe700 1 -- 172.21.15.55:0/2749367112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b015d240 msgr2=0x7fc6b015e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:07.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.112+0000 7fc6b49fe700 1 --2- 172.21.15.55:0/2749367112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b015d240 0x7fc6b015e630 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800c840 tx=0x7fc6a8001670 comp rx=0 tx=0).stop 2023-10-06T05:07:07.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.112+0000 7fc6b49fe700 1 -- 172.21.15.55:0/2749367112 shutdown_connections 2023-10-06T05:07:07.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.112+0000 7fc6b49fe700 1 --2- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc69c067170 0x7fc69c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.112+0000 7fc6b49fe700 1 --2- 172.21.15.55:0/2749367112 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc6b015d240 0x7fc6b015e630 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.112+0000 7fc6b49fe700 1 --2- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc6b00b16d0 0x7fc6b00c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.112+0000 7fc6b49fe700 1 --2- 172.21.15.55:0/2749367112 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc6b00b4e70 0x7fc6b0157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.112+0000 7fc6b49fe700 1 -- 172.21.15.55:0/2749367112 >> 172.21.15.55:0/2749367112 conn(0x7fc6b00d10c0 msgr2=0x7fc6b0158db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:07.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.112+0000 7fc6b49fe700 1 -- 172.21.15.55:0/2749367112 shutdown_connections 2023-10-06T05:07:07.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.112+0000 7fc6b49fe700 1 -- 172.21.15.55:0/2749367112 wait complete. 2023-10-06T05:07:07.133 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:07:07.210 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:07.209+0000 7f6aede8b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:07:07.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.323+0000 7f922199d700 1 Processor -- start 2023-10-06T05:07:07.325 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.323+0000 7f922199d700 1 -- start start 2023-10-06T05:07:07.326 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.324+0000 7f922199d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f921c142ea0 0x7f921c0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:07.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.324+0000 7f922199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c06c0f0 0x7f921c06a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:07.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.324+0000 7f922199d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f921c0b40f0 con 0x7f921c0ad870 2023-10-06T05:07:07.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.324+0000 7f922199d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f921c069640 con 0x7f921c142ea0 2023-10-06T05:07:07.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.324+0000 7f922199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921c069920 con 0x7f921c06c0f0 2023-10-06T05:07:07.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.325+0000 7f922099b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f921c142ea0 0x7f921c0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:07.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.325+0000 7f922119c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c06c0f0 0x7f921c06a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:07.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.325+0000 7f921bfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f921c0ad870 0x7f921c06b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:33100/0 (socket says 172.21.15.55:33100) 2023-10-06T05:07:07.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.325+0000 7f921bfff700 1 -- 172.21.15.55:0/1957895936 learned_addr learned my addr 172.21.15.55:0/1957895936 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:07.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.325+0000 7f921b7fe700 1 -- 172.21.15.55:0/1957895936 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930481792 0 0) 0x7f921c0b40f0 con 0x7f921c0ad870 2023-10-06T05:07:07.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.325+0000 7f921b7fe700 1 -- 172.21.15.55:0/1957895936 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f920c003570 con 0x7f921c0ad870 2023-10-06T05:07:07.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.325+0000 7f922119c700 1 -- 172.21.15.55:0/1957895936 >> v2:172.21.15.163:3300/0 conn(0x7f921c142ea0 msgr2=0x7f921c0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:07.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.325+0000 7f922119c700 1 --2- 172.21.15.55:0/1957895936 >> v2:172.21.15.163:3300/0 conn(0x7f921c142ea0 0x7f921c0b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.325+0000 7f922119c700 1 -- 172.21.15.55:0/1957895936 >> v1:172.21.15.163:6789/0 conn(0x7f921c0ad870 legacy=0x7f921c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:07.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.325+0000 7f922119c700 1 -- 172.21.15.55:0/1957895936 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c069c00 con 0x7f921c06c0f0 2023-10-06T05:07:07.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.325+0000 7f922099b700 1 --2- 172.21.15.55:0/1957895936 >> v2:172.21.15.163:3300/0 conn(0x7f921c142ea0 0x7f921c0b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:07.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.326+0000 7f922119c700 1 --2- 172.21.15.55:0/1957895936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c06c0f0 0x7f921c06a120 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9214009e60 tx=0x7f9214009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=541ba3c174cab74e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:07.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.326+0000 7f921b7fe700 1 -- 172.21.15.55:0/1957895936 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921400e070 con 0x7f921c06c0f0 2023-10-06T05:07:07.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.326+0000 7f921b7fe700 1 -- 172.21.15.55:0/1957895936 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921400bab0 con 0x7f921c06c0f0 2023-10-06T05:07:07.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.326+0000 7f921b7fe700 1 -- 172.21.15.55:0/1957895936 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921400be70 con 0x7f921c06c0f0 2023-10-06T05:07:07.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.326+0000 7f922199d700 1 -- 172.21.15.55:0/1957895936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c06c0f0 msgr2=0x7f921c06a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:07.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.326+0000 7f922199d700 1 --2- 172.21.15.55:0/1957895936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c06c0f0 0x7f921c06a120 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9214009e60 tx=0x7f9214009780 comp rx=0 tx=0).stop 2023-10-06T05:07:07.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.326+0000 7f922199d700 1 -- 172.21.15.55:0/1957895936 shutdown_connections 2023-10-06T05:07:07.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.326+0000 7f922199d700 1 --2- 172.21.15.55:0/1957895936 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c06c0f0 0x7f921c06a120 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.326+0000 7f922199d700 1 --2- 172.21.15.55:0/1957895936 >> v2:172.21.15.163:3300/0 conn(0x7f921c142ea0 0x7f921c0b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.327+0000 7f922199d700 1 -- 172.21.15.55:0/1957895936 >> 172.21.15.55:0/1957895936 conn(0x7f921c0ce430 msgr2=0x7f921c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:07.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.327+0000 7f922199d700 1 -- 172.21.15.55:0/1957895936 shutdown_connections 2023-10-06T05:07:07.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.327+0000 7f922199d700 1 -- 172.21.15.55:0/1957895936 wait complete. 2023-10-06T05:07:07.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.328+0000 7f922199d700 1 Processor -- start 2023-10-06T05:07:07.332 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.328+0000 7f922199d700 1 -- start start 2023-10-06T05:07:07.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.329+0000 7f922199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c06c0f0 0x7f921c159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:07.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.329+0000 7f922199d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f921c142ea0 0x7f921c15eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:07.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.329+0000 7f922199d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c159180 0x7f921c15b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:07.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.329+0000 7f922199d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c15af20 con 0x7f921c142ea0 2023-10-06T05:07:07.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.329+0000 7f922199d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c15e8c0 con 0x7f921c159180 2023-10-06T05:07:07.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.329+0000 7f922199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921c15b8f0 con 0x7f921c06c0f0 2023-10-06T05:07:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.329+0000 7f921bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f921c142ea0 0x7f921c15eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.329+0000 7f922119c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c159180 0x7f921c15b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.329+0000 7f922099b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c06c0f0 0x7f921c159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.329+0000 7f922119c700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c159180 0x7f921c15b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:52408/0 (socket says 172.21.15.55:52408) 2023-10-06T05:07:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.329+0000 7f922119c700 1 -- 172.21.15.55:0/2746223973 learned_addr learned my addr 172.21.15.55:0/2746223973 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.329+0000 7f921bfff700 1 --2- 172.21.15.55:0/2746223973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f921c142ea0 0x7f921c15eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50798/0 (socket says 172.21.15.55:50798) 2023-10-06T05:07:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.330+0000 7f922099b700 1 -- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c159180 msgr2=0x7f921c15b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.330+0000 7f922099b700 1 --2- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c159180 0x7f921c15b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.330+0000 7f922099b700 1 -- 172.21.15.55:0/2746223973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f921c142ea0 msgr2=0x7f921c15eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.330+0000 7f922099b700 1 --2- 172.21.15.55:0/2746223973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f921c142ea0 0x7f921c15eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.335 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.330+0000 7f922099b700 1 -- 172.21.15.55:0/2746223973 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9210009420 con 0x7f921c06c0f0 2023-10-06T05:07:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.330+0000 7f922099b700 1 --2- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c06c0f0 0x7f921c159d10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f921000ea30 tx=0x7f92100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.330+0000 7f9218ff9700 1 -- 172.21.15.55:0/2746223973 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921000ee70 con 0x7f921c06c0f0 2023-10-06T05:07:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.330+0000 7f9218ff9700 1 -- 172.21.15.55:0/2746223973 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921000c850 con 0x7f921c06c0f0 2023-10-06T05:07:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.330+0000 7f9218ff9700 1 -- 172.21.15.55:0/2746223973 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921000cbf0 con 0x7f921c06c0f0 2023-10-06T05:07:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.330+0000 7f922199d700 1 -- 172.21.15.55:0/2746223973 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9214009a20 con 0x7f921c06c0f0 2023-10-06T05:07:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.331+0000 7f922199d700 1 -- 172.21.15.55:0/2746223973 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921c15a310 con 0x7f921c06c0f0 2023-10-06T05:07:07.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.332+0000 7f922199d700 1 -- 172.21.15.55:0/2746223973 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f921c04c3c0 con 0x7f921c06c0f0 2023-10-06T05:07:07.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.333+0000 7f9218ff9700 1 -- 172.21.15.55:0/2746223973 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 97) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f9210013470 con 0x7f921c06c0f0 2023-10-06T05:07:07.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.333+0000 7f9218ff9700 1 --2- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9200067170 0x7f9200069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:07.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.333+0000 7f9218ff9700 1 -- 172.21.15.55:0/2746223973 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6792+0+0 (secure 0 0 0) 0x7f9210084fa0 con 0x7f921c06c0f0 2023-10-06T05:07:07.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.334+0000 7f921bfff700 1 -- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9200067170 msgr2=0x7f9200069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:07.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.334+0000 7f921bfff700 1 --2- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9200067170 0x7f9200069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:07.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.338+0000 7f9218ff9700 1 -- 172.21.15.55:0/2746223973 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92100519d0 con 0x7f921c06c0f0 2023-10-06T05:07:07.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.535+0000 7f921bfff700 1 -- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9200067170 msgr2=0x7f9200069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:07.537 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.535+0000 7f921bfff700 1 --2- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9200067170 0x7f9200069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:07.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.587+0000 7f922199d700 1 -- 172.21.15.55:0/2746223973 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f921c143270 con 0x7f921c06c0f0 2023-10-06T05:07:07.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.588+0000 7f9218ff9700 1 -- 172.21.15.55:0/2746223973 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f9210054ff0 con 0x7f921c06c0f0 2023-10-06T05:07:07.590 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:07.590 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8078,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1255308005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1255308005},{"type":"v1","addr":"172.21.15.163:6835","nonce":1255308005}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":9274,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2468487159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2468487159},{"type":"v1","addr":"172.21.15.163:6834","nonce":2468487159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/4105265850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":4105265850},{"type":"v1","addr":"172.21.15.55:6834","nonce":4105265850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:05:40.405710+0000","modified":"2023-10-06T05:05:43.944287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9771":{"gid":9771,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/312761019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":312761019},{"type":"v1","addr":"172.21.15.55:6835","nonce":312761019}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:07:07.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.590+0000 7f922199d700 1 -- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9200067170 msgr2=0x7f9200069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.590+0000 7f922199d700 1 --2- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9200067170 0x7f9200069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.590+0000 7f922199d700 1 -- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c06c0f0 msgr2=0x7f921c159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.590+0000 7f922199d700 1 --2- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c06c0f0 0x7f921c159d10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f921000ea30 tx=0x7f92100091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.590+0000 7f922199d700 1 -- 172.21.15.55:0/2746223973 shutdown_connections 2023-10-06T05:07:07.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.590+0000 7f922199d700 1 --2- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9200067170 0x7f9200069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.590+0000 7f922199d700 1 --2- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f921c159180 0x7f921c15b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.590+0000 7f922199d700 1 --2- 172.21.15.55:0/2746223973 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f921c06c0f0 0x7f921c159d10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.590+0000 7f922199d700 1 --2- 172.21.15.55:0/2746223973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f921c142ea0 0x7f921c15eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.590+0000 7f922199d700 1 -- 172.21.15.55:0/2746223973 >> 172.21.15.55:0/2746223973 conn(0x7f921c0ce430 msgr2=0x7f921c0cc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.590+0000 7f922199d700 1 -- 172.21.15.55:0/2746223973 shutdown_connections 2023-10-06T05:07:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.590+0000 7f922199d700 1 -- 172.21.15.55:0/2746223973 wait complete. 2023-10-06T05:07:07.594 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 96 2023-10-06T05:07:07.610 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:07:07.626 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:07.625+0000 7f47bb684180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:07:07.800 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:07.798+0000 7f47bb684180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:07:07.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.828+0000 7fc8a5d9d700 1 Processor -- start 2023-10-06T05:07:07.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.828+0000 7fc8a5d9d700 1 -- start start 2023-10-06T05:07:07.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.830+0000 7fc8a5d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc8a01597f0 0x7fc8a00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:07.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.830+0000 7fc8a5d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc8a015d230 0x7fc8a015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:07.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.830+0000 7fc8a5d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a015db70 con 0x7fc8a0159420 2023-10-06T05:07:07.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.830+0000 7fc8a5d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a00b11d0 con 0x7fc8a015d230 2023-10-06T05:07:07.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.830+0000 7fc8a5d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a00b14b0 con 0x7fc8a01597f0 2023-10-06T05:07:07.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.830+0000 7fc8a559c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc8a015d230 0x7fc8a015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:07.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.830+0000 7fc8a559c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc8a015d230 0x7fc8a015d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50808/0 (socket says 172.21.15.55:50808) 2023-10-06T05:07:07.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.830+0000 7fc8a559c700 1 -- 172.21.15.55:0/1122438019 learned_addr learned my addr 172.21.15.55:0/1122438019 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:07.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.830+0000 7fc89ffff700 1 --2- 172.21.15.55:0/1122438019 >> v2:172.21.15.163:3300/0 conn(0x7fc8a01597f0 0x7fc8a00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:07.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.831+0000 7fc8a559c700 1 -- 172.21.15.55:0/1122438019 >> v2:172.21.15.163:3300/0 conn(0x7fc8a01597f0 msgr2=0x7fc8a00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:07.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.831+0000 7fc8a559c700 1 --2- 172.21.15.55:0/1122438019 >> v2:172.21.15.163:3300/0 conn(0x7fc8a01597f0 0x7fc8a00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.831+0000 7fc8a559c700 1 -- 172.21.15.55:0/1122438019 >> v1:172.21.15.55:6789/0 conn(0x7fc8a0159420 legacy=0x7fc8a00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:07.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.831+0000 7fc8a559c700 1 -- 172.21.15.55:0/1122438019 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a00b1790 con 0x7fc8a015d230 2023-10-06T05:07:07.833 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.831+0000 7fc8a559c700 1 --2- 172.21.15.55:0/1122438019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a015d230 0x7fc8a015d630 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0089750 tx=0x7fc898009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296b8c7601f82d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:07.834 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.831+0000 7fc89f7fe700 1 -- 172.21.15.55:0/1122438019 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89800e070 con 0x7fc8a015d230 2023-10-06T05:07:07.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.831+0000 7fc89f7fe700 1 -- 172.21.15.55:0/1122438019 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89800ba20 con 0x7fc8a015d230 2023-10-06T05:07:07.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.831+0000 7fc89f7fe700 1 -- 172.21.15.55:0/1122438019 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8980092e0 con 0x7fc8a015d230 2023-10-06T05:07:07.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.832+0000 7fc8a5d9d700 1 -- 172.21.15.55:0/1122438019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a015d230 msgr2=0x7fc8a015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:07.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.832+0000 7fc8a5d9d700 1 --2- 172.21.15.55:0/1122438019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a015d230 0x7fc8a015d630 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0089750 tx=0x7fc898009840 comp rx=0 tx=0).stop 2023-10-06T05:07:07.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.832+0000 7fc8a5d9d700 1 -- 172.21.15.55:0/1122438019 shutdown_connections 2023-10-06T05:07:07.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.832+0000 7fc8a5d9d700 1 --2- 172.21.15.55:0/1122438019 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a015d230 0x7fc8a015d630 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.832+0000 7fc8a5d9d700 1 --2- 172.21.15.55:0/1122438019 >> v2:172.21.15.163:3300/0 conn(0x7fc8a01597f0 0x7fc8a00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.832+0000 7fc8a5d9d700 1 -- 172.21.15.55:0/1122438019 >> 172.21.15.55:0/1122438019 conn(0x7fc8a00d0660 msgr2=0x7fc8a00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:07.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.832+0000 7fc8a5d9d700 1 -- 172.21.15.55:0/1122438019 shutdown_connections 2023-10-06T05:07:07.835 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.832+0000 7fc8a5d9d700 1 -- 172.21.15.55:0/1122438019 wait complete. 2023-10-06T05:07:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.833+0000 7fc8a5d9d700 1 Processor -- start 2023-10-06T05:07:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.833+0000 7fc8a5d9d700 1 -- start start 2023-10-06T05:07:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.834+0000 7fc8a5d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8a0159420 0x7fc8a015c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.834+0000 7fc8a5d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a01597f0 0x7fc8a015a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:07.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.834+0000 7fc8a5d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a015d230 0x7fc8a0153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.834+0000 7fc8a5d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a015b650 con 0x7fc8a015d230 2023-10-06T05:07:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.834+0000 7fc8a5d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a015b090 con 0x7fc8a01597f0 2023-10-06T05:07:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.834+0000 7fc8a5d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a015b370 con 0x7fc8a0159420 2023-10-06T05:07:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.835+0000 7fc8a559c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a015d230 0x7fc8a0153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.835+0000 7fc89ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a01597f0 0x7fc8a015a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.835+0000 7fc8a4d9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8a0159420 0x7fc8a015c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.835+0000 7fc89ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a01597f0 0x7fc8a015a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:52428/0 (socket says 172.21.15.55:52428) 2023-10-06T05:07:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.835+0000 7fc89ffff700 1 -- 172.21.15.55:0/1029158315 learned_addr learned my addr 172.21.15.55:0/1029158315 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.835+0000 7fc89ffff700 1 -- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8a0159420 msgr2=0x7fc8a015c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.835+0000 7fc89ffff700 1 --2- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8a0159420 0x7fc8a015c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.835+0000 7fc89ffff700 1 -- 172.21.15.55:0/1029158315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a015d230 msgr2=0x7fc8a0153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:07.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.835+0000 7fc89ffff700 1 --2- 172.21.15.55:0/1029158315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a015d230 0x7fc8a0153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.835+0000 7fc89ffff700 1 -- 172.21.15.55:0/1029158315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc898009ae0 con 0x7fc8a01597f0 2023-10-06T05:07:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.836+0000 7fc89ffff700 1 --2- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a01597f0 0x7fc8a015a660 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00ec50 tx=0x7fc88c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:07.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.836+0000 7fc89cff9700 1 -- 172.21.15.55:0/1029158315 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c009690 con 0x7fc8a01597f0 2023-10-06T05:07:07.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.836+0000 7fc89cff9700 1 -- 172.21.15.55:0/1029158315 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c00c850 con 0x7fc8a01597f0 2023-10-06T05:07:07.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.836+0000 7fc8a5d9d700 1 -- 172.21.15.55:0/1029158315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a00c55a0 con 0x7fc8a01597f0 2023-10-06T05:07:07.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.836+0000 7fc89cff9700 1 -- 172.21.15.55:0/1029158315 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c00cc10 con 0x7fc8a01597f0 2023-10-06T05:07:07.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.836+0000 7fc8a5d9d700 1 -- 172.21.15.55:0/1029158315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a00c7430 con 0x7fc8a01597f0 2023-10-06T05:07:07.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.837+0000 7fc88a7fc700 1 -- 172.21.15.55:0/1029158315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8780052b0 con 0x7fc8a01597f0 2023-10-06T05:07:07.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.838+0000 7fc89cff9700 1 -- 172.21.15.55:0/1029158315 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 97) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fc88c013470 con 0x7fc8a01597f0 2023-10-06T05:07:07.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.839+0000 7fc89cff9700 1 --2- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc884067170 0x7fc884069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:07.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.839+0000 7fc89cff9700 1 -- 172.21.15.55:0/1029158315 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6792+0+0 (secure 0 0 0) 0x7fc88c084f10 con 0x7fc8a01597f0 2023-10-06T05:07:07.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.839+0000 7fc8a4d9b700 1 -- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc884067170 msgr2=0x7fc884069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:07.841 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.839+0000 7fc8a4d9b700 1 --2- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc884067170 0x7fc884069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:07.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:07.843+0000 7fc89cff9700 1 -- 172.21.15.55:0/1029158315 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc88c051a10 con 0x7fc8a01597f0 2023-10-06T05:07:07.874 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:07.873+0000 7f6aede8b180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:07:07.898 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:07.896+0000 7f47bb684180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:07:08.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.040+0000 7fc8a4d9b700 1 -- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc884067170 msgr2=0x7fc884069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:08.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.040+0000 7fc8a4d9b700 1 --2- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc884067170 0x7fc884069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:08.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.052+0000 7fc88a7fc700 1 -- 172.21.15.55:0/1029158315 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc878005830 con 0x7fc8a01597f0 2023-10-06T05:07:08.055 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.053+0000 7fc89cff9700 1 -- 172.21.15.55:0/1029158315 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+18056 (secure 0 0 0) 0x7fc88c055030 con 0x7fc8a01597f0 2023-10-06T05:07:08.055 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:08.055 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":195,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:07:06.001411+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:05:38.605485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:05:39.377918+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":195,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3244631548":"2023-10-07T05:05:56.105364+0000","172.21.15.163:0/3391882979":"2023-10-07T05:05:56.105364+0000","172.21.15.163:6837/101089":"2023-10-07T05:05:56.105364+0000","172.21.15.163:6836/101089":"2023-10-07T05:05:56.105364+0000","172.21.15.163:0/3718998698":"2023-10-07T05:05:56.105364+0000","172.21.15.163:0/2082256652":"2023-10-07T05:05:56.105364+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:07:08.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.055+0000 7fc88a7fc700 1 -- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc884067170 msgr2=0x7fc884069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:08.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.055+0000 7fc88a7fc700 1 --2- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc884067170 0x7fc884069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.055+0000 7fc88a7fc700 1 -- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a01597f0 msgr2=0x7fc8a015a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:08.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.055+0000 7fc88a7fc700 1 --2- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a01597f0 0x7fc8a015a660 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00ec50 tx=0x7fc88c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.055+0000 7fc88a7fc700 1 -- 172.21.15.55:0/1029158315 shutdown_connections 2023-10-06T05:07:08.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.055+0000 7fc88a7fc700 1 --2- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc884067170 0x7fc884069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.055+0000 7fc88a7fc700 1 --2- 172.21.15.55:0/1029158315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc8a015d230 0x7fc8a0153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.055+0000 7fc88a7fc700 1 --2- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc8a0159420 0x7fc8a015c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.055+0000 7fc88a7fc700 1 --2- 172.21.15.55:0/1029158315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc8a01597f0 0x7fc8a015a660 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.055+0000 7fc88a7fc700 1 -- 172.21.15.55:0/1029158315 >> 172.21.15.55:0/1029158315 conn(0x7fc8a00d0660 msgr2=0x7fc8a00b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:08.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.055+0000 7fc88a7fc700 1 -- 172.21.15.55:0/1029158315 shutdown_connections 2023-10-06T05:07:08.061 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.055+0000 7fc88a7fc700 1 -- 172.21.15.55:0/1029158315 wait complete. 2023-10-06T05:07:08.061 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:08.057+0000 7f6aede8b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:07:08.076 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:07:08.077 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:07:08.162 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:08.160+0000 7f6aede8b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:07:08.197 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:08.196+0000 7f47bb684180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:07:08.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.265+0000 7f309d39d700 1 Processor -- start 2023-10-06T05:07:08.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.266+0000 7f309d39d700 1 -- start start 2023-10-06T05:07:08.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.267+0000 7f309d39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f30981595a0 0x7f30980b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:08.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.267+0000 7f309d39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30980c6c60 0x7f30980c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:08.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.267+0000 7f309d39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f30980b1de0 0x7f30980c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:08.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.267+0000 7f309d39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30980b2b90 con 0x7f30980b1de0 2023-10-06T05:07:08.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.267+0000 7f309d39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30980b1410 con 0x7f30981595a0 2023-10-06T05:07:08.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.267+0000 7f309d39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3098063230 con 0x7f30980c6c60 2023-10-06T05:07:08.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.267+0000 7f309cb9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f30980b1de0 0x7f30980c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:08.273 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.267+0000 7f309cb9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f30980b1de0 0x7f30980c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50828/0 (socket says 172.21.15.55:50828) 2023-10-06T05:07:08.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.267+0000 7f30977fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30980c6c60 0x7f30980c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:08.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.267+0000 7f309cb9c700 1 -- 172.21.15.55:0/1985636953 learned_addr learned my addr 172.21.15.55:0/1985636953 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:08.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.267+0000 7f3097fff700 1 --2- 172.21.15.55:0/1985636953 >> v2:172.21.15.163:3300/0 conn(0x7f30981595a0 0x7f30980b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:08.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.268+0000 7f30977fe700 1 -- 172.21.15.55:0/1985636953 >> v2:172.21.15.163:3300/0 conn(0x7f30981595a0 msgr2=0x7f30980b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:08.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.268+0000 7f30977fe700 1 --2- 172.21.15.55:0/1985636953 >> v2:172.21.15.163:3300/0 conn(0x7f30981595a0 0x7f30980b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.268+0000 7f30977fe700 1 -- 172.21.15.55:0/1985636953 >> v2:172.21.15.55:3300/0 conn(0x7f30980b1de0 msgr2=0x7f30980c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:08.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.268+0000 7f30977fe700 1 --2- 172.21.15.55:0/1985636953 >> v2:172.21.15.55:3300/0 conn(0x7f30980b1de0 0x7f30980c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.268+0000 7f30977fe700 1 -- 172.21.15.55:0/1985636953 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30980634f0 con 0x7f30980c6c60 2023-10-06T05:07:08.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.268+0000 7f309cb9c700 1 --2- 172.21.15.55:0/1985636953 >> v2:172.21.15.55:3300/0 conn(0x7f30980b1de0 0x7f30980c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:08.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.268+0000 7f30977fe700 1 --2- 172.21.15.55:0/1985636953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30980c6c60 0x7f30980c8120 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f30980c8d10 tx=0x7f308c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9fb90867daf6657 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:08.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.268+0000 7f3096ffd700 1 -- 172.21.15.55:0/1985636953 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c011630 con 0x7f30980c6c60 2023-10-06T05:07:08.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.268+0000 7f3096ffd700 1 -- 172.21.15.55:0/1985636953 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c0117b0 con 0x7f30980c6c60 2023-10-06T05:07:08.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.269+0000 7f3096ffd700 1 -- 172.21.15.55:0/1985636953 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c011b70 con 0x7f30980c6c60 2023-10-06T05:07:08.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.269+0000 7f309d39d700 1 -- 172.21.15.55:0/1985636953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30980c6c60 msgr2=0x7f30980c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:08.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.269+0000 7f309d39d700 1 --2- 172.21.15.55:0/1985636953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30980c6c60 0x7f30980c8120 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f30980c8d10 tx=0x7f308c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.269+0000 7f309d39d700 1 -- 172.21.15.55:0/1985636953 shutdown_connections 2023-10-06T05:07:08.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.269+0000 7f309d39d700 1 --2- 172.21.15.55:0/1985636953 >> v2:172.21.15.55:3300/0 conn(0x7f30980b1de0 0x7f30980c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.269+0000 7f309d39d700 1 --2- 172.21.15.55:0/1985636953 >> v2:172.21.15.163:3300/0 conn(0x7f30981595a0 0x7f30980b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.269+0000 7f309d39d700 1 --2- 172.21.15.55:0/1985636953 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30980c6c60 0x7f30980c8120 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.269+0000 7f309d39d700 1 -- 172.21.15.55:0/1985636953 >> 172.21.15.55:0/1985636953 conn(0x7f309806ddb0 msgr2=0x7f309806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:08.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.269+0000 7f309d39d700 1 -- 172.21.15.55:0/1985636953 shutdown_connections 2023-10-06T05:07:08.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.269+0000 7f309d39d700 1 -- 172.21.15.55:0/1985636953 wait complete. 2023-10-06T05:07:08.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.271+0000 7f309d39d700 1 Processor -- start 2023-10-06T05:07:08.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.271+0000 7f309d39d700 1 -- start start 2023-10-06T05:07:08.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.272+0000 7f309d39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30980b1de0 0x7f309814d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:08.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.272+0000 7f309d39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30980c6c60 0x7f309815c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:08.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.272+0000 7f309d39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30981595a0 0x7f309815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:08.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.272+0000 7f309d39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309814e050 con 0x7f30980b1de0 2023-10-06T05:07:08.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.272+0000 7f309d39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3098159a10 con 0x7f30980c6c60 2023-10-06T05:07:08.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.272+0000 7f309d39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3098159cf0 con 0x7f30981595a0 2023-10-06T05:07:08.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.272+0000 7f3097fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30980b1de0 0x7f309814d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:08.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.272+0000 7f30977fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30980c6c60 0x7f309815c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:08.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.272+0000 7f309cb9c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30981595a0 0x7f309815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:08.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.272+0000 7f3097fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30980b1de0 0x7f309814d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50838/0 (socket says 172.21.15.55:50838) 2023-10-06T05:07:08.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.272+0000 7f30977fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30980c6c60 0x7f309815c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:52448/0 (socket says 172.21.15.55:52448) 2023-10-06T05:07:08.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.272+0000 7f3097fff700 1 -- 172.21.15.55:0/2411019417 learned_addr learned my addr 172.21.15.55:0/2411019417 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:08.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.273+0000 7f309cb9c700 1 -- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30980c6c60 msgr2=0x7f309815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:08.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.273+0000 7f309cb9c700 1 --2- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30980c6c60 0x7f309815c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.273+0000 7f309cb9c700 1 -- 172.21.15.55:0/2411019417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30980b1de0 msgr2=0x7f309814d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:08.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.273+0000 7f309cb9c700 1 --2- 172.21.15.55:0/2411019417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30980b1de0 0x7f309814d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.273+0000 7f309cb9c700 1 -- 172.21.15.55:0/2411019417 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3090009ae0 con 0x7f30981595a0 2023-10-06T05:07:08.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.273+0000 7f3097fff700 1 --2- 172.21.15.55:0/2411019417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30980b1de0 0x7f309814d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:08.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.273+0000 7f309cb9c700 1 --2- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30981595a0 0x7f309815ccf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3090009ee0 tx=0x7f309000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:08.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.273+0000 7f307ffff700 1 -- 172.21.15.55:0/2411019417 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090018070 con 0x7f30981595a0 2023-10-06T05:07:08.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.273+0000 7f307ffff700 1 -- 172.21.15.55:0/2411019417 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309000f040 con 0x7f30981595a0 2023-10-06T05:07:08.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.273+0000 7f309d39d700 1 -- 172.21.15.55:0/2411019417 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308c009420 con 0x7f30981595a0 2023-10-06T05:07:08.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.273+0000 7f307ffff700 1 -- 172.21.15.55:0/2411019417 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309000cb90 con 0x7f30981595a0 2023-10-06T05:07:08.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.274+0000 7f309d39d700 1 -- 172.21.15.55:0/2411019417 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309815a2b0 con 0x7f30981595a0 2023-10-06T05:07:08.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.275+0000 7f309d39d700 1 -- 172.21.15.55:0/2411019417 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309815a4a0 con 0x7f30981595a0 2023-10-06T05:07:08.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.276+0000 7f307ffff700 1 -- 172.21.15.55:0/2411019417 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 97) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f3090003020 con 0x7f30981595a0 2023-10-06T05:07:08.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.277+0000 7f307ffff700 1 --2- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3084067170 0x7f3084069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:08.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.277+0000 7f307ffff700 1 -- 172.21.15.55:0/2411019417 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6792+0+0 (secure 0 0 0) 0x7f30900851b0 con 0x7f30981595a0 2023-10-06T05:07:08.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.277+0000 7f3097fff700 1 -- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3084067170 msgr2=0x7f3084069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:08.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.277+0000 7f3097fff700 1 --2- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3084067170 0x7f3084069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:08.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.285+0000 7f307ffff700 1 -- 172.21.15.55:0/2411019417 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3090051ba0 con 0x7f30981595a0 2023-10-06T05:07:08.298 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:08.297+0000 7f47bb684180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:07:08.468 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:08.466+0000 7f6aede8b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:07:08.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.478+0000 7f3097fff700 1 -- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3084067170 msgr2=0x7f3084069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:08.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.478+0000 7f3097fff700 1 --2- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3084067170 0x7f3084069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:08.534 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.532+0000 7f309d39d700 1 -- 172.21.15.55:0/2411019417 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f309804c3c0 con 0x7f30981595a0 2023-10-06T05:07:08.535 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.533+0000 7f307ffff700 1 -- 172.21.15.55:0/2411019417 <== mon.2 v2:172.21.15.163:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4604 (secure 0 0 0) 0x7f30900551c0 con 0x7f30981595a0 2023-10-06T05:07:08.535 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:08.535 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8078,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1255308005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1255308005},{"type":"v1","addr":"172.21.15.163:6835","nonce":1255308005}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":9274,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2468487159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2468487159},{"type":"v1","addr":"172.21.15.163:6834","nonce":2468487159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/4105265850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":4105265850},{"type":"v1","addr":"172.21.15.55:6834","nonce":4105265850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:05:40.405710+0000","modified":"2023-10-06T05:05:43.944287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9771":{"gid":9771,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/312761019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":312761019},{"type":"v1","addr":"172.21.15.55:6835","nonce":312761019}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:07:08.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.536+0000 7f309d39d700 1 -- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3084067170 msgr2=0x7f3084069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:08.538 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.536+0000 7f309d39d700 1 --2- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3084067170 0x7f3084069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.536+0000 7f309d39d700 1 -- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30981595a0 msgr2=0x7f309815ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.536+0000 7f309d39d700 1 --2- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30981595a0 0x7f309815ccf0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3090009ee0 tx=0x7f309000b040 comp rx=0 tx=0).stop 2023-10-06T05:07:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.536+0000 7f309d39d700 1 -- 172.21.15.55:0/2411019417 shutdown_connections 2023-10-06T05:07:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.536+0000 7f309d39d700 1 --2- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3084067170 0x7f3084069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.536+0000 7f309d39d700 1 --2- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f30981595a0 0x7f309815ccf0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.536+0000 7f309d39d700 1 --2- 172.21.15.55:0/2411019417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f30980b1de0 0x7f309814d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.536+0000 7f309d39d700 1 --2- 172.21.15.55:0/2411019417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f30980c6c60 0x7f309815c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.536+0000 7f309d39d700 1 -- 172.21.15.55:0/2411019417 >> 172.21.15.55:0/2411019417 conn(0x7f309806ddb0 msgr2=0x7f309814af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:08.539 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.536+0000 7f309d39d700 1 -- 172.21.15.55:0/2411019417 shutdown_connections 2023-10-06T05:07:08.540 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.536+0000 7f309d39d700 1 -- 172.21.15.55:0/2411019417 wait complete. 2023-10-06T05:07:08.540 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 96 2023-10-06T05:07:08.557 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:07:08.573 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:08.571+0000 7f6aede8b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:07:08.748 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.746+0000 7fcde45ff700 1 Processor -- start 2023-10-06T05:07:08.749 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.746+0000 7fcde45ff700 1 -- start start 2023-10-06T05:07:08.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.748+0000 7fcde45ff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcddc0b1420 0x7fcddc0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:08.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.748+0000 7fcde45ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcddc0cc300 con 0x7fcddc0ccf90 2023-10-06T05:07:08.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.748+0000 7fcde45ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcddc0ad370 con 0x7fcddc0cbf30 2023-10-06T05:07:08.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.748+0000 7fcde45ff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcddc156bf0 con 0x7fcddc0b1420 2023-10-06T05:07:08.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.748+0000 7fcde2b9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcddc0ccf90 0x7fcddc0c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52834/0 (socket says 172.21.15.55:52834) 2023-10-06T05:07:08.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.748+0000 7fcde2b9c700 1 -- 172.21.15.55:0/1159731585 learned_addr learned my addr 172.21.15.55:0/1159731585 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:08.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.748+0000 7fcde1b9a700 1 --2- 172.21.15.55:0/1159731585 >> v2:172.21.15.163:3300/0 conn(0x7fcddc0b1420 0x7fcddc0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:08.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.749+0000 7fcde1399700 1 -- 172.21.15.55:0/1159731585 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177199918 0 0) 0x7fcddc0cc300 con 0x7fcddc0ccf90 2023-10-06T05:07:08.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.749+0000 7fcde1399700 1 -- 172.21.15.55:0/1159731585 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdc8003570 con 0x7fcddc0ccf90 2023-10-06T05:07:08.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.749+0000 7fcde1b9a700 1 -- 172.21.15.55:0/1159731585 >> v1:172.21.15.163:6789/0 conn(0x7fcddc0cbf30 legacy=0x7fcddc0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:08.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.749+0000 7fcde1b9a700 1 -- 172.21.15.55:0/1159731585 >> v1:172.21.15.55:6789/0 conn(0x7fcddc0ccf90 legacy=0x7fcddc0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:08.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.749+0000 7fcde1b9a700 1 -- 172.21.15.55:0/1159731585 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc0ad650 con 0x7fcddc0b1420 2023-10-06T05:07:08.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.749+0000 7fcde1b9a700 1 --2- 172.21.15.55:0/1159731585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcddc0b1420 0x7fcddc0ae410 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0098e0 tx=0x7fcdcc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9b68e1dd981de7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:08.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.749+0000 7fcde1399700 1 -- 172.21.15.55:0/1159731585 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00b640 con 0x7fcddc0b1420 2023-10-06T05:07:08.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.749+0000 7fcde1399700 1 -- 172.21.15.55:0/1159731585 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc00b7a0 con 0x7fcddc0b1420 2023-10-06T05:07:08.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.749+0000 7fcde1399700 1 -- 172.21.15.55:0/1159731585 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00bb40 con 0x7fcddc0b1420 2023-10-06T05:07:08.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.750+0000 7fcde45ff700 1 -- 172.21.15.55:0/1159731585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcddc0b1420 msgr2=0x7fcddc0ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:08.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.750+0000 7fcde45ff700 1 --2- 172.21.15.55:0/1159731585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcddc0b1420 0x7fcddc0ae410 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc0098e0 tx=0x7fcdcc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.750+0000 7fcde45ff700 1 -- 172.21.15.55:0/1159731585 shutdown_connections 2023-10-06T05:07:08.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.750+0000 7fcde45ff700 1 --2- 172.21.15.55:0/1159731585 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcddc0b1420 0x7fcddc0ae410 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.750+0000 7fcde45ff700 1 -- 172.21.15.55:0/1159731585 >> 172.21.15.55:0/1159731585 conn(0x7fcddc06a320 msgr2=0x7fcddc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:08.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.750+0000 7fcde45ff700 1 -- 172.21.15.55:0/1159731585 shutdown_connections 2023-10-06T05:07:08.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.750+0000 7fcde45ff700 1 -- 172.21.15.55:0/1159731585 wait complete. 2023-10-06T05:07:08.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.752+0000 7fcde45ff700 1 Processor -- start 2023-10-06T05:07:08.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.752+0000 7fcde45ff700 1 -- start start 2023-10-06T05:07:08.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.753+0000 7fcde45ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcddc0b1420 0x7fcddc147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:08.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.753+0000 7fcde45ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcddc0cbf30 0x7fcddc0c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:08.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.753+0000 7fcde45ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcddc0ccf90 0x7fcddc16d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:08.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.753+0000 7fcde45ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0c8190 con 0x7fcddc0ccf90 2023-10-06T05:07:08.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.753+0000 7fcde45ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0c7d00 con 0x7fcddc0cbf30 2023-10-06T05:07:08.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.753+0000 7fcde45ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc0c70f0 con 0x7fcddc0b1420 2023-10-06T05:07:08.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.753+0000 7fcde2b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcddc0ccf90 0x7fcddc16d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:08.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.753+0000 7fcde2b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcddc0ccf90 0x7fcddc16d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50846/0 (socket says 172.21.15.55:50846) 2023-10-06T05:07:08.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.753+0000 7fcde2b9c700 1 -- 172.21.15.55:0/1319740026 learned_addr learned my addr 172.21.15.55:0/1319740026 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:08.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.753+0000 7fcde1b9a700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcddc0cbf30 0x7fcddc0c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:08.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.753+0000 7fcde239b700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcddc0b1420 0x7fcddc147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:08.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.754+0000 7fcde1b9a700 1 -- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcddc0b1420 msgr2=0x7fcddc147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:08.758 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.754+0000 7fcde1b9a700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcddc0b1420 0x7fcddc147d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.754+0000 7fcde1b9a700 1 -- 172.21.15.55:0/1319740026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcddc0ccf90 msgr2=0x7fcddc16d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:08.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.754+0000 7fcde1b9a700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcddc0ccf90 0x7fcddc16d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:08.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.754+0000 7fcde1b9a700 1 -- 172.21.15.55:0/1319740026 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdcc009420 con 0x7fcddc0cbf30 2023-10-06T05:07:08.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.754+0000 7fcde2b9c700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcddc0ccf90 0x7fcddc16d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:08.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.754+0000 7fcde1b9a700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcddc0cbf30 0x7fcddc0c6650 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc016000 tx=0x7fcdcc017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:08.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.754+0000 7fcdd27fc700 1 -- 172.21.15.55:0/1319740026 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00bb20 con 0x7fcddc0cbf30 2023-10-06T05:07:08.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.754+0000 7fcdd27fc700 1 -- 172.21.15.55:0/1319740026 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc000c90 con 0x7fcddc0cbf30 2023-10-06T05:07:08.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.754+0000 7fcdd27fc700 1 -- 172.21.15.55:0/1319740026 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc011630 con 0x7fcddc0cbf30 2023-10-06T05:07:08.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.754+0000 7fcde45ff700 1 -- 172.21.15.55:0/1319740026 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcddc0c73d0 con 0x7fcddc0cbf30 2023-10-06T05:07:08.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.755+0000 7fcde45ff700 1 -- 172.21.15.55:0/1319740026 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcddc0c7990 con 0x7fcddc0cbf30 2023-10-06T05:07:08.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.755+0000 7fcde45ff700 1 -- 172.21.15.55:0/1319740026 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcddc0c7e70 con 0x7fcddc0cbf30 2023-10-06T05:07:08.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.756+0000 7fcdd27fc700 1 -- 172.21.15.55:0/1319740026 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 97) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fcdcc011a60 con 0x7fcddc0cbf30 2023-10-06T05:07:08.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.757+0000 7fcdd27fc700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcdc0067170 0x7fcdc0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:08.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.757+0000 7fcdd27fc700 1 -- 172.21.15.55:0/1319740026 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6792+0+0 (secure 0 0 0) 0x7fcdcc0865e0 con 0x7fcddc0cbf30 2023-10-06T05:07:08.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.757+0000 7fcde239b700 1 -- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcdc0067170 msgr2=0x7fcdc0069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:08.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.757+0000 7fcde239b700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcdc0067170 0x7fcdc0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:08.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.761+0000 7fcdd27fc700 1 -- 172.21.15.55:0/1319740026 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdcc0530e0 con 0x7fcddc0cbf30 2023-10-06T05:07:08.810 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:07:08.814 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:07:08.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.958+0000 7fcde239b700 1 -- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcdc0067170 msgr2=0x7fcdc0069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:08.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:08.958+0000 7fcde239b700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcdc0067170 0x7fcdc0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:09.015 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:09.014+0000 7f47bb684180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:07:09.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.018+0000 7fcde45ff700 1 -- 172.21.15.55:0/1319740026 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcddc04c3c0 con 0x7fcddc0cbf30 2023-10-06T05:07:09.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.021+0000 7fcdd27fc700 1 -- 172.21.15.55:0/1319740026 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4604 (secure 0 0 0) 0x7fcdcc056700 con 0x7fcddc0cbf30 2023-10-06T05:07:09.023 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:09.023 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":96,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8078,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1255308005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1255308005},{"type":"v1","addr":"172.21.15.163:6835","nonce":1255308005}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":9274,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2468487159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2468487159},{"type":"v1","addr":"172.21.15.163:6834","nonce":2468487159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/4105265850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":4105265850},{"type":"v1","addr":"172.21.15.55:6834","nonce":4105265850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:05:40.405710+0000","modified":"2023-10-06T05:05:43.944287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9771},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9771":{"gid":9771,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/312761019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":312761019},{"type":"v1","addr":"172.21.15.55:6835","nonce":312761019}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-06T05:07:09.025 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.023+0000 7fcde45ff700 1 -- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcdc0067170 msgr2=0x7fcdc0069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.023+0000 7fcde45ff700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcdc0067170 0x7fcdc0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.023+0000 7fcde45ff700 1 -- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcddc0cbf30 msgr2=0x7fcddc0c6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.023+0000 7fcde45ff700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcddc0cbf30 0x7fcddc0c6650 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc016000 tx=0x7fcdcc017000 comp rx=0 tx=0).stop 2023-10-06T05:07:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.023+0000 7fcde45ff700 1 -- 172.21.15.55:0/1319740026 shutdown_connections 2023-10-06T05:07:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.023+0000 7fcde45ff700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcdc0067170 0x7fcdc0069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.023+0000 7fcde45ff700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcddc0ccf90 0x7fcddc16d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.023+0000 7fcde45ff700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcddc0b1420 0x7fcddc147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.023+0000 7fcde45ff700 1 --2- 172.21.15.55:0/1319740026 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcddc0cbf30 0x7fcddc0c6650 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.023+0000 7fcde45ff700 1 -- 172.21.15.55:0/1319740026 >> 172.21.15.55:0/1319740026 conn(0x7fcddc06a320 msgr2=0x7fcddc0c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:09.026 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.023+0000 7fcde45ff700 1 -- 172.21.15.55:0/1319740026 shutdown_connections 2023-10-06T05:07:09.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.023+0000 7fcde45ff700 1 -- 172.21.15.55:0/1319740026 wait complete. 2023-10-06T05:07:09.027 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 96 2023-10-06T05:07:09.044 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:07:09.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.233+0000 7f9c5239d700 1 Processor -- start 2023-10-06T05:07:09.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.233+0000 7f9c5239d700 1 -- start start 2023-10-06T05:07:09.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.234+0000 7f9c5239d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9c4c15d230 0x7f9c4c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:09.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.234+0000 7f9c5239d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4c0b2980 con 0x7f9c4c159420 2023-10-06T05:07:09.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.234+0000 7f9c5239d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4c15db70 con 0x7f9c4c1597f0 2023-10-06T05:07:09.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.234+0000 7f9c5239d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4c0b11d0 con 0x7f9c4c15d230 2023-10-06T05:07:09.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.235+0000 7f9c5139b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9c4c159420 0x7f9c4c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52846/0 (socket says 172.21.15.55:52846) 2023-10-06T05:07:09.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.235+0000 7f9c5139b700 1 -- 172.21.15.55:0/155814388 learned_addr learned my addr 172.21.15.55:0/155814388 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:09.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.235+0000 7f9c51b9c700 1 --2- 172.21.15.55:0/155814388 >> v2:172.21.15.163:3300/0 conn(0x7f9c4c15d230 0x7f9c4c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:09.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.235+0000 7f9c43fff700 1 -- 172.21.15.55:0/155814388 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948399972 0 0) 0x7f9c4c15db70 con 0x7f9c4c1597f0 2023-10-06T05:07:09.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.235+0000 7f9c43fff700 1 -- 172.21.15.55:0/155814388 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c38003570 con 0x7f9c4c1597f0 2023-10-06T05:07:09.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.235+0000 7f9c43fff700 1 -- 172.21.15.55:0/155814388 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849774248 0 0) 0x7f9c4c0b2980 con 0x7f9c4c159420 2023-10-06T05:07:09.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.235+0000 7f9c43fff700 1 -- 172.21.15.55:0/155814388 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c4c15db70 con 0x7f9c4c159420 2023-10-06T05:07:09.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.235+0000 7f9c51b9c700 1 -- 172.21.15.55:0/155814388 >> v1:172.21.15.163:6789/0 conn(0x7f9c4c1597f0 legacy=0x7f9c4c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:09.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.235+0000 7f9c51b9c700 1 -- 172.21.15.55:0/155814388 >> v1:172.21.15.55:6789/0 conn(0x7f9c4c159420 legacy=0x7f9c4c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:09.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.235+0000 7f9c51b9c700 1 -- 172.21.15.55:0/155814388 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0b14b0 con 0x7f9c4c15d230 2023-10-06T05:07:09.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.236+0000 7f9c51b9c700 1 --2- 172.21.15.55:0/155814388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4c15d230 0x7f9c4c15d630 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009e60 tx=0x7f9c44009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1dd16930c7989b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:09.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.236+0000 7f9c43fff700 1 -- 172.21.15.55:0/155814388 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4400e070 con 0x7f9c4c15d230 2023-10-06T05:07:09.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.236+0000 7f9c43fff700 1 -- 172.21.15.55:0/155814388 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4400bab0 con 0x7f9c4c15d230 2023-10-06T05:07:09.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.236+0000 7f9c43fff700 1 -- 172.21.15.55:0/155814388 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4400be70 con 0x7f9c4c15d230 2023-10-06T05:07:09.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.236+0000 7f9c5239d700 1 -- 172.21.15.55:0/155814388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4c15d230 msgr2=0x7f9c4c15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:09.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.236+0000 7f9c5239d700 1 --2- 172.21.15.55:0/155814388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4c15d230 0x7f9c4c15d630 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009e60 tx=0x7f9c44009780 comp rx=0 tx=0).stop 2023-10-06T05:07:09.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.237+0000 7f9c5239d700 1 -- 172.21.15.55:0/155814388 shutdown_connections 2023-10-06T05:07:09.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.237+0000 7f9c5239d700 1 --2- 172.21.15.55:0/155814388 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4c15d230 0x7f9c4c15d630 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.237+0000 7f9c5239d700 1 -- 172.21.15.55:0/155814388 >> 172.21.15.55:0/155814388 conn(0x7f9c4c0d0660 msgr2=0x7f9c4c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:09.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.237+0000 7f9c5239d700 1 -- 172.21.15.55:0/155814388 shutdown_connections 2023-10-06T05:07:09.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.237+0000 7f9c5239d700 1 -- 172.21.15.55:0/155814388 wait complete. 2023-10-06T05:07:09.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.238+0000 7f9c5239d700 1 Processor -- start 2023-10-06T05:07:09.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.238+0000 7f9c5239d700 1 -- start start 2023-10-06T05:07:09.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.239+0000 7f9c5239d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4c159420 0x7f9c4c16ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:09.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.239+0000 7f9c5239d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c4c1597f0 0x7f9c4c154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:09.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.239+0000 7f9c5239d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c4c15d230 0x7f9c4c169650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:09.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.239+0000 7f9c5239d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c155330 con 0x7f9c4c1597f0 2023-10-06T05:07:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.239+0000 7f9c5239d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c159e20 con 0x7f9c4c159420 2023-10-06T05:07:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.239+0000 7f9c5239d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c16a060 con 0x7f9c4c15d230 2023-10-06T05:07:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.239+0000 7f9c5139b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4c159420 0x7f9c4c16ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.239+0000 7f9c5139b700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4c159420 0x7f9c4c16ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:52504/0 (socket says 172.21.15.55:52504) 2023-10-06T05:07:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.239+0000 7f9c5139b700 1 -- 172.21.15.55:0/2296380089 learned_addr learned my addr 172.21.15.55:0/2296380089 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.239+0000 7f9c50b9a700 1 --2- 172.21.15.55:0/2296380089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c4c1597f0 0x7f9c4c154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.240+0000 7f9c51b9c700 1 --2- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c4c15d230 0x7f9c4c169650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.240+0000 7f9c5139b700 1 -- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c4c15d230 msgr2=0x7f9c4c169650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:09.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.240+0000 7f9c5139b700 1 --2- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c4c15d230 0x7f9c4c169650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.240+0000 7f9c5139b700 1 -- 172.21.15.55:0/2296380089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c4c1597f0 msgr2=0x7f9c4c154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:09.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.240+0000 7f9c5139b700 1 --2- 172.21.15.55:0/2296380089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c4c1597f0 0x7f9c4c154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.240+0000 7f9c5139b700 1 -- 172.21.15.55:0/2296380089 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c44009a20 con 0x7f9c4c159420 2023-10-06T05:07:09.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.240+0000 7f9c5139b700 1 --2- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4c159420 0x7f9c4c16ab20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800c7f0 tx=0x7f9c48007e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:09.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.241+0000 7f9c417fa700 1 -- 172.21.15.55:0/2296380089 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c48005b30 con 0x7f9c4c159420 2023-10-06T05:07:09.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.241+0000 7f9c417fa700 1 -- 172.21.15.55:0/2296380089 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c48005c90 con 0x7f9c4c159420 2023-10-06T05:07:09.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.241+0000 7f9c417fa700 1 -- 172.21.15.55:0/2296380089 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c48007320 con 0x7f9c4c159420 2023-10-06T05:07:09.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.241+0000 7f9c5239d700 1 -- 172.21.15.55:0/2296380089 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4c16a340 con 0x7f9c4c159420 2023-10-06T05:07:09.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.241+0000 7f9c5239d700 1 -- 172.21.15.55:0/2296380089 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4c15cb20 con 0x7f9c4c159420 2023-10-06T05:07:09.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.242+0000 7f9c5239d700 1 -- 172.21.15.55:0/2296380089 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4c15a020 con 0x7f9c4c159420 2023-10-06T05:07:09.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.243+0000 7f9c417fa700 1 -- 172.21.15.55:0/2296380089 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 97) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f9c4801b3b0 con 0x7f9c4c159420 2023-10-06T05:07:09.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.243+0000 7f9c417fa700 1 --2- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9c30067170 0x7f9c30069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:09.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.244+0000 7f9c417fa700 1 -- 172.21.15.55:0/2296380089 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6792+0+0 (secure 0 0 0) 0x7f9c480842d0 con 0x7f9c4c159420 2023-10-06T05:07:09.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.244+0000 7f9c50b9a700 1 -- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9c30067170 msgr2=0x7f9c30069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:09.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.244+0000 7f9c50b9a700 1 --2- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9c30067170 0x7f9c30069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:09.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.247+0000 7f9c417fa700 1 -- 172.21.15.55:0/2296380089 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c48050da0 con 0x7f9c4c159420 2023-10-06T05:07:09.307 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:09.306+0000 7f47bb684180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:07:09.335 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:09.334+0000 7f6aede8b180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:07:09.399 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:09.398+0000 7f47bb684180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:07:09.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.445+0000 7f9c50b9a700 1 -- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9c30067170 msgr2=0x7f9c30069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:09.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.445+0000 7f9c50b9a700 1 --2- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9c30067170 0x7f9c30069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:09.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.452+0000 7f9c5239d700 1 -- 172.21.15.55:0/2296380089 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c4c15cd40 con 0x7f9c4c159420 2023-10-06T05:07:09.455 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.453+0000 7f9c417fa700 1 -- 172.21.15.55:0/2296380089 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+18056 (secure 0 0 0) 0x7f9c480543c0 con 0x7f9c4c159420 2023-10-06T05:07:09.456 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:09.456 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":195,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:07:06.001411+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:05:38.605485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-10-06T05:05:39.377918+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":195,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/3244631548":"2023-10-07T05:05:56.105364+0000","172.21.15.163:0/3391882979":"2023-10-07T05:05:56.105364+0000","172.21.15.163:6837/101089":"2023-10-07T05:05:56.105364+0000","172.21.15.163:6836/101089":"2023-10-07T05:05:56.105364+0000","172.21.15.163:0/3718998698":"2023-10-07T05:05:56.105364+0000","172.21.15.163:0/2082256652":"2023-10-07T05:05:56.105364+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:07:09.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.457+0000 7f9c5239d700 1 -- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9c30067170 msgr2=0x7f9c30069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:09.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.457+0000 7f9c5239d700 1 --2- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9c30067170 0x7f9c30069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.457+0000 7f9c5239d700 1 -- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4c159420 msgr2=0x7f9c4c16ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:09.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.457+0000 7f9c5239d700 1 --2- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4c159420 0x7f9c4c16ab20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800c7f0 tx=0x7f9c48007e00 comp rx=0 tx=0).stop 2023-10-06T05:07:09.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.457+0000 7f9c5239d700 1 -- 172.21.15.55:0/2296380089 shutdown_connections 2023-10-06T05:07:09.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.457+0000 7f9c5239d700 1 --2- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9c30067170 0x7f9c30069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.457+0000 7f9c5239d700 1 --2- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c4c15d230 0x7f9c4c169650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.457+0000 7f9c5239d700 1 --2- 172.21.15.55:0/2296380089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c4c159420 0x7f9c4c16ab20 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.457+0000 7f9c5239d700 1 --2- 172.21.15.55:0/2296380089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c4c1597f0 0x7f9c4c154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.457+0000 7f9c5239d700 1 -- 172.21.15.55:0/2296380089 >> 172.21.15.55:0/2296380089 conn(0x7f9c4c0d0660 msgr2=0x7f9c4c0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:09.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.457+0000 7f9c5239d700 1 -- 172.21.15.55:0/2296380089 shutdown_connections 2023-10-06T05:07:09.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.457+0000 7f9c5239d700 1 -- 172.21.15.55:0/2296380089 wait complete. 2023-10-06T05:07:09.484 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:07:09.556 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:09.554+0000 7f47bb684180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:07:09.641 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:09.639+0000 7f6aede8b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:07:09.660 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:09.658+0000 7f47bb684180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:07:09.677 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.675+0000 7f3e94f9d700 1 Processor -- start 2023-10-06T05:07:09.678 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.676+0000 7f3e94f9d700 1 -- start start 2023-10-06T05:07:09.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.677+0000 7f3e94f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f3e901597f0 0x7f3e900b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:09.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.677+0000 7f3e94f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e9015d230 0x7f3e9015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:09.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.677+0000 7f3e94f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9015dbb0 con 0x7f3e90159420 2023-10-06T05:07:09.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.677+0000 7f3e94f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e900b11d0 con 0x7f3e901597f0 2023-10-06T05:07:09.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.677+0000 7f3e94f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900b14b0 con 0x7f3e9015d230 2023-10-06T05:07:09.679 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.677+0000 7f3e8f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3e90159420 0x7f3e900b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52862/0 (socket says 172.21.15.55:52862) 2023-10-06T05:07:09.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.677+0000 7f3e8f7fe700 1 -- 172.21.15.55:0/4115077678 learned_addr learned my addr 172.21.15.55:0/4115077678 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:09.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.677+0000 7f3e8ffff700 1 --2- 172.21.15.55:0/4115077678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e9015d230 0x7f3e9015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:09.680 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.677+0000 7f3e8effd700 1 --2- 172.21.15.55:0/4115077678 >> v2:172.21.15.163:3300/0 conn(0x7f3e901597f0 0x7f3e900b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.678+0000 7f3e8e7fc700 1 -- 172.21.15.55:0/4115077678 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501208300 0 0) 0x7f3e9015dbb0 con 0x7f3e90159420 2023-10-06T05:07:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.678+0000 7f3e8e7fc700 1 -- 172.21.15.55:0/4115077678 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e78003570 con 0x7f3e90159420 2023-10-06T05:07:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.678+0000 7f3e8effd700 1 -- 172.21.15.55:0/4115077678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e9015d230 msgr2=0x7f3e9015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.678+0000 7f3e8effd700 1 --2- 172.21.15.55:0/4115077678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e9015d230 0x7f3e9015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.678+0000 7f3e8effd700 1 -- 172.21.15.55:0/4115077678 >> v1:172.21.15.55:6789/0 conn(0x7f3e90159420 legacy=0x7f3e900b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.678+0000 7f3e8effd700 1 -- 172.21.15.55:0/4115077678 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e900b1790 con 0x7f3e901597f0 2023-10-06T05:07:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.678+0000 7f3e8ffff700 1 --2- 172.21.15.55:0/4115077678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e9015d230 0x7f3e9015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.678+0000 7f3e8effd700 1 --2- 172.21.15.55:0/4115077678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e901597f0 0x7f3e900b2270 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3e840098e0 tx=0x7f3e840091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5451f5e650b8602 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.678+0000 7f3e8e7fc700 1 -- 172.21.15.55:0/4115077678 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84011630 con 0x7f3e901597f0 2023-10-06T05:07:09.682 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.678+0000 7f3e8e7fc700 1 -- 172.21.15.55:0/4115077678 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e84011790 con 0x7f3e901597f0 2023-10-06T05:07:09.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.678+0000 7f3e8e7fc700 1 -- 172.21.15.55:0/4115077678 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84011b30 con 0x7f3e901597f0 2023-10-06T05:07:09.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.679+0000 7f3e94f9d700 1 -- 172.21.15.55:0/4115077678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e901597f0 msgr2=0x7f3e900b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:09.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.679+0000 7f3e94f9d700 1 --2- 172.21.15.55:0/4115077678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e901597f0 0x7f3e900b2270 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f3e840098e0 tx=0x7f3e840091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.679+0000 7f3e94f9d700 1 -- 172.21.15.55:0/4115077678 shutdown_connections 2023-10-06T05:07:09.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.679+0000 7f3e94f9d700 1 --2- 172.21.15.55:0/4115077678 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e9015d230 0x7f3e9015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.679+0000 7f3e94f9d700 1 --2- 172.21.15.55:0/4115077678 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e901597f0 0x7f3e900b2270 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.679+0000 7f3e94f9d700 1 -- 172.21.15.55:0/4115077678 >> 172.21.15.55:0/4115077678 conn(0x7f3e900d0660 msgr2=0x7f3e900d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:09.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.679+0000 7f3e94f9d700 1 -- 172.21.15.55:0/4115077678 shutdown_connections 2023-10-06T05:07:09.683 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.679+0000 7f3e94f9d700 1 -- 172.21.15.55:0/4115077678 wait complete. 2023-10-06T05:07:09.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.681+0000 7f3e94f9d700 1 Processor -- start 2023-10-06T05:07:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.681+0000 7f3e94f9d700 1 -- start start 2023-10-06T05:07:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e94f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e90159420 0x7f3e90154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e94f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e901597f0 0x7f3e900c5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e94f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e9015d230 0x7f3e9015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:09.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e94f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900c6870 con 0x7f3e90159420 2023-10-06T05:07:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e94f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9015bc10 con 0x7f3e9015d230 2023-10-06T05:07:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e94f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9015cb30 con 0x7f3e901597f0 2023-10-06T05:07:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e8f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e90159420 0x7f3e90154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e8f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e90159420 0x7f3e90154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50858/0 (socket says 172.21.15.55:50858) 2023-10-06T05:07:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e8f7fe700 1 -- 172.21.15.55:0/2896181230 learned_addr learned my addr 172.21.15.55:0/2896181230 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:09.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e8ffff700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e9015d230 0x7f3e9015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e8effd700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e901597f0 0x7f3e900c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e8f7fe700 1 -- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e901597f0 msgr2=0x7f3e900c5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e8f7fe700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e901597f0 0x7f3e900c5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e8f7fe700 1 -- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e9015d230 msgr2=0x7f3e9015c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e8f7fe700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e9015d230 0x7f3e9015c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e8f7fe700 1 -- 172.21.15.55:0/2896181230 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e88009a20 con 0x7f3e90159420 2023-10-06T05:07:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.682+0000 7f3e8ffff700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e9015d230 0x7f3e9015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.683+0000 7f3e8f7fe700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e90159420 0x7f3e90154110 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f3e8000c7f0 tx=0x7f3e80007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.683+0000 7f3e777fe700 1 -- 172.21.15.55:0/2896181230 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80005b30 con 0x7f3e90159420 2023-10-06T05:07:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.683+0000 7f3e777fe700 1 -- 172.21.15.55:0/2896181230 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e80005cb0 con 0x7f3e90159420 2023-10-06T05:07:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.683+0000 7f3e94f9d700 1 -- 172.21.15.55:0/2896181230 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e84009420 con 0x7f3e90159420 2023-10-06T05:07:09.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.683+0000 7f3e777fe700 1 -- 172.21.15.55:0/2896181230 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80007320 con 0x7f3e90159420 2023-10-06T05:07:09.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.683+0000 7f3e94f9d700 1 -- 172.21.15.55:0/2896181230 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9015b780 con 0x7f3e90159420 2023-10-06T05:07:09.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.684+0000 7f3e94f9d700 1 -- 172.21.15.55:0/2896181230 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9004c3c0 con 0x7f3e90159420 2023-10-06T05:07:09.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.685+0000 7f3e777fe700 1 -- 172.21.15.55:0/2896181230 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 97) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f3e8001b3b0 con 0x7f3e90159420 2023-10-06T05:07:09.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.686+0000 7f3e777fe700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3e7c067170 0x7f3e7c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:09.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.686+0000 7f3e777fe700 1 -- 172.21.15.55:0/2896181230 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6792+0+0 (secure 0 0 0) 0x7f3e80084320 con 0x7f3e90159420 2023-10-06T05:07:09.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.686+0000 7f3e8effd700 1 -- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3e7c067170 msgr2=0x7f3e7c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:09.690 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.686+0000 7f3e8effd700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3e7c067170 0x7f3e7c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:09.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.690+0000 7f3e777fe700 1 -- 172.21.15.55:0/2896181230 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e80050d50 con 0x7f3e90159420 2023-10-06T05:07:09.742 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:09.740+0000 7f6aede8b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:07:09.854 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:09.853+0000 7f47bb684180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:07:09.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.887+0000 7f3e8effd700 1 -- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3e7c067170 msgr2=0x7f3e7c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:09.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.887+0000 7f3e8effd700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3e7c067170 0x7f3e7c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:09.904 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:09.902+0000 7f6aede8b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:07:09.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.940+0000 7f3e94f9d700 1 -- 172.21.15.55:0/2896181230 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3e900b1a70 con 0x7f3e90159420 2023-10-06T05:07:09.942 INFO:tasks.ceph.mon.a.smithi055.stderr:2023-10-06T05:07:09.940+0000 7fe524ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:07:09.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.947+0000 7f3e777fe700 1 -- 172.21.15.55:0/2896181230 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v97) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3e80017090 con 0x7f3e90159420 2023-10-06T05:07:09.950 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:09.949+0000 7f47bb684180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:07:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.949+0000 7f3e94f9d700 1 -- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3e7c067170 msgr2=0x7f3e7c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.949+0000 7f3e94f9d700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3e7c067170 0x7f3e7c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.949+0000 7f3e94f9d700 1 -- 172.21.15.55:0/2896181230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e90159420 msgr2=0x7f3e90154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.949+0000 7f3e94f9d700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e90159420 0x7f3e90154110 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f3e8000c7f0 tx=0x7f3e80007e00 comp rx=0 tx=0).stop 2023-10-06T05:07:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.950+0000 7f3e94f9d700 1 -- 172.21.15.55:0/2896181230 shutdown_connections 2023-10-06T05:07:09.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.950+0000 7f3e94f9d700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3e7c067170 0x7f3e7c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.950+0000 7f3e94f9d700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3e9015d230 0x7f3e9015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.950+0000 7f3e94f9d700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3e90159420 0x7f3e90154110 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.950+0000 7f3e94f9d700 1 --2- 172.21.15.55:0/2896181230 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3e901597f0 0x7f3e900c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:09.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.950+0000 7f3e94f9d700 1 -- 172.21.15.55:0/2896181230 >> 172.21.15.55:0/2896181230 conn(0x7f3e900d0660 msgr2=0x7f3e901561c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:09.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.950+0000 7f3e94f9d700 1 -- 172.21.15.55:0/2896181230 shutdown_connections 2023-10-06T05:07:09.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:09.950+0000 7f3e94f9d700 1 -- 172.21.15.55:0/2896181230 wait complete. 2023-10-06T05:07:09.958 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:07:09.970 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:07:10.009 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T05:07:10.019 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:10.017+0000 7f6aede8b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:07:10.045 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:10.044+0000 7f47bb684180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:07:10.140 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:10.138+0000 7f47bb684180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:07:10.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.168+0000 7f59f1f9d700 1 Processor -- start 2023-10-06T05:07:10.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.169+0000 7f59f1f9d700 1 -- start start 2023-10-06T05:07:10.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.170+0000 7f59f1f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f59ec0b4e70 0x7f59ec0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:10.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.170+0000 7f59f1f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec0cfb00 con 0x7f59ec0b1b40 2023-10-06T05:07:10.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.170+0000 7f59f1f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec15e060 con 0x7f59ec0b16d0 2023-10-06T05:07:10.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.170+0000 7f59f1f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ec0b5c40 con 0x7f59ec0b4e70 2023-10-06T05:07:10.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.170+0000 7f59f179c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f59ec0b1b40 0x7f59ec15d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52866/0 (socket says 172.21.15.55:52866) 2023-10-06T05:07:10.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.170+0000 7f59f179c700 1 -- 172.21.15.55:0/719634114 learned_addr learned my addr 172.21.15.55:0/719634114 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:10.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.170+0000 7f59f0f9b700 1 --2- 172.21.15.55:0/719634114 >> v2:172.21.15.163:3300/0 conn(0x7f59ec0b4e70 0x7f59ec0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:10.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.171+0000 7f59f0f9b700 1 -- 172.21.15.55:0/719634114 >> v1:172.21.15.163:6789/0 conn(0x7f59ec0b16d0 legacy=0x7f59ec15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:10.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.171+0000 7f59f0f9b700 1 -- 172.21.15.55:0/719634114 >> v1:172.21.15.55:6789/0 conn(0x7f59ec0b1b40 legacy=0x7f59ec15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:10.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.171+0000 7f59f0f9b700 1 -- 172.21.15.55:0/719634114 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec0b5f20 con 0x7f59ec0b4e70 2023-10-06T05:07:10.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.171+0000 7f59eb7fe700 1 -- 172.21.15.55:0/719634114 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783976116 0 0) 0x7f59ec0cfb00 con 0x7f59ec0b1b40 2023-10-06T05:07:10.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.171+0000 7f59f0f9b700 1 --2- 172.21.15.55:0/719634114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59ec0b4e70 0x7f59ec0ccff0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f59e00098e0 tx=0x7f59e00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f79df366e739e0e7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:10.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.171+0000 7f59eb7fe700 1 -- 172.21.15.55:0/719634114 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0009e80 con 0x7f59ec0b4e70 2023-10-06T05:07:10.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.171+0000 7f59eb7fe700 1 -- 172.21.15.55:0/719634114 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e000b860 con 0x7f59ec0b4e70 2023-10-06T05:07:10.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.171+0000 7f59eb7fe700 1 -- 172.21.15.55:0/719634114 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e000bbe0 con 0x7f59ec0b4e70 2023-10-06T05:07:10.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.172+0000 7f59f1f9d700 1 -- 172.21.15.55:0/719634114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59ec0b4e70 msgr2=0x7f59ec0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:10.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.172+0000 7f59f1f9d700 1 --2- 172.21.15.55:0/719634114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59ec0b4e70 0x7f59ec0ccff0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f59e00098e0 tx=0x7f59e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:10.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.172+0000 7f59f1f9d700 1 -- 172.21.15.55:0/719634114 shutdown_connections 2023-10-06T05:07:10.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.172+0000 7f59f1f9d700 1 --2- 172.21.15.55:0/719634114 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59ec0b4e70 0x7f59ec0ccff0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:10.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.172+0000 7f59f1f9d700 1 -- 172.21.15.55:0/719634114 >> 172.21.15.55:0/719634114 conn(0x7f59ec0d10c0 msgr2=0x7f59ec0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:10.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.172+0000 7f59f1f9d700 1 -- 172.21.15.55:0/719634114 shutdown_connections 2023-10-06T05:07:10.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.172+0000 7f59f1f9d700 1 -- 172.21.15.55:0/719634114 wait complete. 2023-10-06T05:07:10.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.173+0000 7f59f1f9d700 1 Processor -- start 2023-10-06T05:07:10.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.174+0000 7f59f1f9d700 1 -- start start 2023-10-06T05:07:10.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.174+0000 7f59f1f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59ec0b16d0 0x7f59ec16aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:10.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.174+0000 7f59f1f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59ec0b1b40 0x7f59ec0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:10.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59f1f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59ec0b4e70 0x7f59ec0cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:10.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59f1f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec0c6d40 con 0x7f59ec0b4e70 2023-10-06T05:07:10.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59f1f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec0c6130 con 0x7f59ec0b1b40 2023-10-06T05:07:10.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59f1f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec0c6410 con 0x7f59ec0b16d0 2023-10-06T05:07:10.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59f0f9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59ec0b16d0 0x7f59ec16aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:10.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59ebfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59ec0b1b40 0x7f59ec0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:10.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59f179c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59ec0b4e70 0x7f59ec0cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:10.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59f0f9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59ec0b16d0 0x7f59ec16aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:40594/0 (socket says 172.21.15.55:40594) 2023-10-06T05:07:10.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59f0f9b700 1 -- 172.21.15.55:0/3803321055 learned_addr learned my addr 172.21.15.55:0/3803321055 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:10.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59f179c700 1 -- 172.21.15.55:0/3803321055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59ec0b16d0 msgr2=0x7f59ec16aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:10.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59f179c700 1 --2- 172.21.15.55:0/3803321055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59ec0b16d0 0x7f59ec16aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:10.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59f179c700 1 -- 172.21.15.55:0/3803321055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59ec0b1b40 msgr2=0x7f59ec0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:10.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59f179c700 1 --2- 172.21.15.55:0/3803321055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59ec0b1b40 0x7f59ec0caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:10.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59f179c700 1 -- 172.21.15.55:0/3803321055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e0009420 con 0x7f59ec0b4e70 2023-10-06T05:07:10.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59ebfff700 1 --2- 172.21.15.55:0/3803321055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59ec0b1b40 0x7f59ec0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:10.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.175+0000 7f59f0f9b700 1 --2- 172.21.15.55:0/3803321055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59ec0b16d0 0x7f59ec16aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:10.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.176+0000 7f59f179c700 1 --2- 172.21.15.55:0/3803321055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59ec0b4e70 0x7f59ec0cb030 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f59e400cbe0 tx=0x7f59e40019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:10.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.176+0000 7f59e8ff9700 1 -- 172.21.15.55:0/3803321055 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e400a8b0 con 0x7f59ec0b4e70 2023-10-06T05:07:10.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.176+0000 7f59e8ff9700 1 -- 172.21.15.55:0/3803321055 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e400aa30 con 0x7f59ec0b4e70 2023-10-06T05:07:10.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.176+0000 7f59f1f9d700 1 -- 172.21.15.55:0/3803321055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec0c66f0 con 0x7f59ec0b4e70 2023-10-06T05:07:10.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.176+0000 7f59e8ff9700 1 -- 172.21.15.55:0/3803321055 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e400adf0 con 0x7f59ec0b4e70 2023-10-06T05:07:10.184 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.176+0000 7f59f1f9d700 1 -- 172.21.15.55:0/3803321055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ec0c5400 con 0x7f59ec0b4e70 2023-10-06T05:07:10.185 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.177+0000 7f59f1f9d700 1 -- 172.21.15.55:0/3803321055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ec088990 con 0x7f59ec0b4e70 2023-10-06T05:07:10.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.178+0000 7f59e8ff9700 1 -- 172.21.15.55:0/3803321055 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 98) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f59e4007300 con 0x7f59ec0b4e70 2023-10-06T05:07:10.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.180+0000 7f59e8ff9700 1 -- 172.21.15.55:0/3803321055 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7194+0+0 (secure 0 0 0) 0x7f59e4084970 con 0x7f59ec0b4e70 2023-10-06T05:07:10.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.185+0000 7f59e8ff9700 1 -- 172.21.15.55:0/3803321055 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59e4051a10 con 0x7f59ec0b4e70 2023-10-06T05:07:10.227 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:10.224+0000 7f4735981700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:10.228 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:10.224+0000 7f4735981700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:10.228 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:10.224+0000 7f4735981700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:10.228 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:10.224+0000 7f4735981700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:10.228 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:10.224+0000 7f4735981700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:10.229 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:10.224+0000 7f4739989700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:10.229 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:10.224+0000 7f4739989700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:10.229 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:10.224+0000 7f4739989700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:10.229 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:10.224+0000 7f4739989700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:10.229 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:10.224+0000 7f4739989700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:10.232 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:10.230+0000 7f6aede8b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:07:10.330 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:10.329+0000 7f6aede8b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:07:10.428 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:10.426+0000 7f6aede8b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:07:10.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.443+0000 7f59f1f9d700 1 -- 172.21.15.55:0/3803321055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f59ec0b5440 con 0x7f59ec0b4e70 2023-10-06T05:07:10.526 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:10.524+0000 7f6aede8b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:07:10.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.946+0000 7f59e8ff9700 1 -- 172.21.15.55:0/3803321055 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v98) v1 ==== 104+0+0 (secure 0 0 0) 0x7f59e4055030 con 0x7f59ec0b4e70 2023-10-06T05:07:10.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.949+0000 7f59f1f9d700 1 -- 172.21.15.55:0/3803321055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59ec0b4e70 msgr2=0x7f59ec0cb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:10.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.949+0000 7f59f1f9d700 1 --2- 172.21.15.55:0/3803321055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59ec0b4e70 0x7f59ec0cb030 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f59e400cbe0 tx=0x7f59e40019a0 comp rx=0 tx=0).stop 2023-10-06T05:07:10.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.949+0000 7f59f1f9d700 1 -- 172.21.15.55:0/3803321055 shutdown_connections 2023-10-06T05:07:10.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.949+0000 7f59f1f9d700 1 --2- 172.21.15.55:0/3803321055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f59ec0b4e70 0x7f59ec0cb030 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:10.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.949+0000 7f59f1f9d700 1 --2- 172.21.15.55:0/3803321055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f59ec0b16d0 0x7f59ec16aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:10.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.949+0000 7f59f1f9d700 1 --2- 172.21.15.55:0/3803321055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f59ec0b1b40 0x7f59ec0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:10.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.949+0000 7f59f1f9d700 1 -- 172.21.15.55:0/3803321055 >> 172.21.15.55:0/3803321055 conn(0x7f59ec0d10c0 msgr2=0x7f59ec068e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:10.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.949+0000 7f59f1f9d700 1 -- 172.21.15.55:0/3803321055 shutdown_connections 2023-10-06T05:07:10.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:10.949+0000 7f59f1f9d700 1 -- 172.21.15.55:0/3803321055 wait complete. 2023-10-06T05:07:10.971 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:07:11.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.208+0000 7f071159d700 1 Processor -- start 2023-10-06T05:07:11.210 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.208+0000 7f071159d700 1 -- start start 2023-10-06T05:07:11.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.209+0000 7f071159d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f070c1572f0 0x7f070c159e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:11.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.209+0000 7f071159d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f070c0acaa0 0x7f070c15a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:11.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.209+0000 7f071159d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f070c15dcb0 con 0x7f070c156f20 2023-10-06T05:07:11.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.209+0000 7f071159d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070c15d380 con 0x7f070c0acaa0 2023-10-06T05:07:11.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.209+0000 7f071159d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070c15d660 con 0x7f070c1572f0 2023-10-06T05:07:11.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.210+0000 7f070bfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f070c156f20 0x7f070c0ac390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52882/0 (socket says 172.21.15.55:52882) 2023-10-06T05:07:11.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.210+0000 7f070bfff700 1 -- 172.21.15.55:0/4027091243 learned_addr learned my addr 172.21.15.55:0/4027091243 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:11.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.210+0000 7f070b7fe700 1 --2- 172.21.15.55:0/4027091243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f070c1572f0 0x7f070c159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:11.213 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.210+0000 7f0710d9c700 1 --2- 172.21.15.55:0/4027091243 >> v2:172.21.15.163:3300/0 conn(0x7f070c0acaa0 0x7f070c15a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:11.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.210+0000 7f070affd700 1 -- 172.21.15.55:0/4027091243 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2846914367 0 0) 0x7f070c15dcb0 con 0x7f070c156f20 2023-10-06T05:07:11.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.210+0000 7f070affd700 1 -- 172.21.15.55:0/4027091243 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06f4003570 con 0x7f070c156f20 2023-10-06T05:07:11.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.210+0000 7f0710d9c700 1 -- 172.21.15.55:0/4027091243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f070c1572f0 msgr2=0x7f070c159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:11.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.210+0000 7f0710d9c700 1 --2- 172.21.15.55:0/4027091243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f070c1572f0 0x7f070c159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:11.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.210+0000 7f0710d9c700 1 -- 172.21.15.55:0/4027091243 >> v1:172.21.15.55:6789/0 conn(0x7f070c156f20 legacy=0x7f070c0ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:11.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.210+0000 7f0710d9c700 1 -- 172.21.15.55:0/4027091243 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070c15d940 con 0x7f070c0acaa0 2023-10-06T05:07:11.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.211+0000 7f0710d9c700 1 --2- 172.21.15.55:0/4027091243 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f070c0acaa0 0x7f070c15a3c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0704009e60 tx=0x7f0704009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62976f5a7dc32d09 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:11.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.211+0000 7f070affd700 1 -- 172.21.15.55:0/4027091243 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070400e070 con 0x7f070c0acaa0 2023-10-06T05:07:11.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.211+0000 7f070affd700 1 -- 172.21.15.55:0/4027091243 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070400bab0 con 0x7f070c0acaa0 2023-10-06T05:07:11.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.211+0000 7f070affd700 1 -- 172.21.15.55:0/4027091243 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070400be30 con 0x7f070c0acaa0 2023-10-06T05:07:11.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.211+0000 7f071159d700 1 -- 172.21.15.55:0/4027091243 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f070c0acaa0 msgr2=0x7f070c15a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:11.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.211+0000 7f071159d700 1 --2- 172.21.15.55:0/4027091243 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f070c0acaa0 0x7f070c15a3c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0704009e60 tx=0x7f0704009780 comp rx=0 tx=0).stop 2023-10-06T05:07:11.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.212+0000 7f071159d700 1 -- 172.21.15.55:0/4027091243 shutdown_connections 2023-10-06T05:07:11.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.212+0000 7f071159d700 1 --2- 172.21.15.55:0/4027091243 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f070c0acaa0 0x7f070c15a3c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:11.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.212+0000 7f071159d700 1 --2- 172.21.15.55:0/4027091243 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f070c1572f0 0x7f070c159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:11.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.212+0000 7f071159d700 1 -- 172.21.15.55:0/4027091243 >> 172.21.15.55:0/4027091243 conn(0x7f070c0c9f10 msgr2=0x7f070c06dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:11.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.212+0000 7f071159d700 1 -- 172.21.15.55:0/4027091243 shutdown_connections 2023-10-06T05:07:11.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.212+0000 7f071159d700 1 -- 172.21.15.55:0/4027091243 wait complete. 2023-10-06T05:07:11.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.213+0000 7f071159d700 1 Processor -- start 2023-10-06T05:07:11.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.213+0000 7f071159d700 1 -- start start 2023-10-06T05:07:11.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.214+0000 7f071159d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f070c0acaa0 0x7f070c0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:11.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.214+0000 7f071159d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f070c156f20 0x7f070c1478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:11.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.214+0000 7f071159d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f070c1572f0 0x7f070c16a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:11.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.214+0000 7f071159d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c0c8250 con 0x7f070c1572f0 2023-10-06T05:07:11.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.214+0000 7f071159d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070c0c7240 con 0x7f070c156f20 2023-10-06T05:07:11.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.214+0000 7f071159d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070c0c7490 con 0x7f070c0acaa0 2023-10-06T05:07:11.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.214+0000 7f0710d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f070c1572f0 0x7f070c16a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:11.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.214+0000 7f0710d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f070c1572f0 0x7f070c16a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50898/0 (socket says 172.21.15.55:50898) 2023-10-06T05:07:11.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.214+0000 7f0710d9c700 1 -- 172.21.15.55:0/281254370 learned_addr learned my addr 172.21.15.55:0/281254370 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:11.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.214+0000 7f070bfff700 1 --2- 172.21.15.55:0/281254370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f070c0acaa0 0x7f070c0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:11.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.214+0000 7f070b7fe700 1 --2- 172.21.15.55:0/281254370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f070c156f20 0x7f070c1478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:11.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.215+0000 7f0710d9c700 1 -- 172.21.15.55:0/281254370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f070c0acaa0 msgr2=0x7f070c0c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:11.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.215+0000 7f0710d9c700 1 --2- 172.21.15.55:0/281254370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f070c0acaa0 0x7f070c0c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:11.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.215+0000 7f0710d9c700 1 -- 172.21.15.55:0/281254370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f070c156f20 msgr2=0x7f070c1478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:11.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.215+0000 7f0710d9c700 1 --2- 172.21.15.55:0/281254370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f070c156f20 0x7f070c1478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:11.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.215+0000 7f0710d9c700 1 -- 172.21.15.55:0/281254370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0704009a20 con 0x7f070c1572f0 2023-10-06T05:07:11.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.215+0000 7f0710d9c700 1 --2- 172.21.15.55:0/281254370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f070c1572f0 0x7f070c16a3a0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f070c0c3990 tx=0x7f0704019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:11.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.215+0000 7f06f3fff700 1 -- 172.21.15.55:0/281254370 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704012e10 con 0x7f070c1572f0 2023-10-06T05:07:11.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.215+0000 7f06f3fff700 1 -- 172.21.15.55:0/281254370 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0704027470 con 0x7f070c1572f0 2023-10-06T05:07:11.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.215+0000 7f06f3fff700 1 -- 172.21.15.55:0/281254370 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0704027830 con 0x7f070c1572f0 2023-10-06T05:07:11.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.215+0000 7f071159d700 1 -- 172.21.15.55:0/281254370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f070c0c7770 con 0x7f070c1572f0 2023-10-06T05:07:11.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.216+0000 7f071159d700 1 -- 172.21.15.55:0/281254370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f070c0c7d30 con 0x7f070c1572f0 2023-10-06T05:07:11.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.216+0000 7f071159d700 1 -- 172.21.15.55:0/281254370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070c0c7f10 con 0x7f070c1572f0 2023-10-06T05:07:11.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.217+0000 7f06f3fff700 1 -- 172.21.15.55:0/281254370 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 99) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f0704024070 con 0x7f070c1572f0 2023-10-06T05:07:11.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.218+0000 7f06f3fff700 1 --2- 172.21.15.55:0/281254370 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f06f8066d80 0x7f06f8069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:11.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.218+0000 7f06f3fff700 1 -- 172.21.15.55:0/281254370 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 7194+0+0 (secure 0 0 0) 0x7f0704086e10 con 0x7f070c1572f0 2023-10-06T05:07:11.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.219+0000 7f070bfff700 1 --2- 172.21.15.55:0/281254370 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f06f8066d80 0x7f06f8069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:11.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.219+0000 7f070bfff700 1 --2- 172.21.15.55:0/281254370 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f06f8066d80 0x7f06f8069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f06fc007fd0 tx=0x7f06fc009040 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:11.225 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.223+0000 7f06f3fff700 1 -- 172.21.15.55:0/281254370 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f070408a030 con 0x7f070c1572f0 2023-10-06T05:07:11.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:11.427+0000 7f071159d700 1 -- 172.21.15.55:0/281254370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f070c0c8fb0 con 0x7f070c1572f0 2023-10-06T05:07:12.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.172+0000 7f06f3fff700 1 -- 172.21.15.55:0/281254370 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v198) v1 ==== 181+0+0 (secure 0 0 0) 0x7f070402c0d0 con 0x7f070c1572f0 2023-10-06T05:07:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.176+0000 7f071159d700 1 -- 172.21.15.55:0/281254370 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f06f8066d80 msgr2=0x7f06f8069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.176+0000 7f071159d700 1 --2- 172.21.15.55:0/281254370 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f06f8066d80 0x7f06f8069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f06fc007fd0 tx=0x7f06fc009040 comp rx=0 tx=0).stop 2023-10-06T05:07:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.176+0000 7f071159d700 1 -- 172.21.15.55:0/281254370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f070c1572f0 msgr2=0x7f070c16a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:12.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.176+0000 7f071159d700 1 --2- 172.21.15.55:0/281254370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f070c1572f0 0x7f070c16a3a0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f070c0c3990 tx=0x7f0704019000 comp rx=0 tx=0).stop 2023-10-06T05:07:12.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.176+0000 7f071159d700 1 -- 172.21.15.55:0/281254370 shutdown_connections 2023-10-06T05:07:12.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.176+0000 7f071159d700 1 --2- 172.21.15.55:0/281254370 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f06f8066d80 0x7f06f8069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:12.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.176+0000 7f071159d700 1 --2- 172.21.15.55:0/281254370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f070c1572f0 0x7f070c16a3a0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:12.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.176+0000 7f071159d700 1 --2- 172.21.15.55:0/281254370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f070c0acaa0 0x7f070c0c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:12.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.176+0000 7f071159d700 1 --2- 172.21.15.55:0/281254370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f070c156f20 0x7f070c1478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:12.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.176+0000 7f071159d700 1 -- 172.21.15.55:0/281254370 >> 172.21.15.55:0/281254370 conn(0x7f070c0c9f10 msgr2=0x7f070c150ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:12.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.176+0000 7f071159d700 1 -- 172.21.15.55:0/281254370 shutdown_connections 2023-10-06T05:07:12.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.176+0000 7f071159d700 1 -- 172.21.15.55:0/281254370 wait complete. 2023-10-06T05:07:12.180 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:07:12.197 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:07:12.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.392+0000 7f5093fff700 1 Processor -- start 2023-10-06T05:07:12.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.392+0000 7f5093fff700 1 -- start start 2023-10-06T05:07:12.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.393+0000 7f5093fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f509415bb30 0x7f50941623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:12.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.393+0000 7f5093fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5094157a50 con 0x7f509415b050 2023-10-06T05:07:12.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.393+0000 7f5093fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5094158f10 con 0x7f50940cc8d0 2023-10-06T05:07:12.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.393+0000 7f5093fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50941591b0 con 0x7f509415bb30 2023-10-06T05:07:12.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.394+0000 7f50937fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f509415bb30 0x7f50941623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:12.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.394+0000 7f5092ffd700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f509415b050 0x7f50940ca100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:52898/0 (socket says 172.21.15.55:52898) 2023-10-06T05:07:12.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.394+0000 7f5092ffd700 1 -- 172.21.15.55:0/2768718376 learned_addr learned my addr 172.21.15.55:0/2768718376 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:12.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.394+0000 7f5091ffb700 1 -- 172.21.15.55:0/2768718376 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195516304 0 0) 0x7f5094157a50 con 0x7f509415b050 2023-10-06T05:07:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.394+0000 7f5091ffb700 1 -- 172.21.15.55:0/2768718376 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f507c003570 con 0x7f509415b050 2023-10-06T05:07:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.394+0000 7f50937fe700 1 -- 172.21.15.55:0/2768718376 >> v1:172.21.15.163:6789/0 conn(0x7f50940cc8d0 legacy=0x7f509415b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.394+0000 7f50937fe700 1 -- 172.21.15.55:0/2768718376 >> v1:172.21.15.55:6789/0 conn(0x7f509415b050 legacy=0x7f50940ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.394+0000 7f50937fe700 1 -- 172.21.15.55:0/2768718376 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50940ca9e0 con 0x7f509415bb30 2023-10-06T05:07:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.394+0000 7f50937fe700 1 --2- 172.21.15.55:0/2768718376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f509415bb30 0x7f50941623f0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f508c009e60 tx=0x7f508c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8679995818a0b0eb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.395+0000 7f5091ffb700 1 -- 172.21.15.55:0/2768718376 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c00e070 con 0x7f509415bb30 2023-10-06T05:07:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.395+0000 7f5091ffb700 1 -- 172.21.15.55:0/2768718376 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c00bb60 con 0x7f509415bb30 2023-10-06T05:07:12.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.395+0000 7f5091ffb700 1 -- 172.21.15.55:0/2768718376 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c012430 con 0x7f509415bb30 2023-10-06T05:07:12.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.395+0000 7f5093fff700 1 -- 172.21.15.55:0/2768718376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f509415bb30 msgr2=0x7f50941623f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:12.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.395+0000 7f5093fff700 1 --2- 172.21.15.55:0/2768718376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f509415bb30 0x7f50941623f0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f508c009e60 tx=0x7f508c009780 comp rx=0 tx=0).stop 2023-10-06T05:07:12.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.395+0000 7f5093fff700 1 -- 172.21.15.55:0/2768718376 shutdown_connections 2023-10-06T05:07:12.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.395+0000 7f5093fff700 1 --2- 172.21.15.55:0/2768718376 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f509415bb30 0x7f50941623f0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:12.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.395+0000 7f5093fff700 1 -- 172.21.15.55:0/2768718376 >> 172.21.15.55:0/2768718376 conn(0x7f5094068e20 msgr2=0x7f50940cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:12.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.395+0000 7f5093fff700 1 -- 172.21.15.55:0/2768718376 shutdown_connections 2023-10-06T05:07:12.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.395+0000 7f5093fff700 1 -- 172.21.15.55:0/2768718376 wait complete. 2023-10-06T05:07:12.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.397+0000 7f5093fff700 1 Processor -- start 2023-10-06T05:07:12.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.397+0000 7f5093fff700 1 -- start start 2023-10-06T05:07:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.398+0000 7f5093fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50940cc8d0 0x7f5094147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.398+0000 7f5093fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f509415b050 0x7f509406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.398+0000 7f5093fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f509415bb30 0x7f509406da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.398+0000 7f5093fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509416d760 con 0x7f509415b050 2023-10-06T05:07:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.398+0000 7f5093fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509416cb50 con 0x7f50940cc8d0 2023-10-06T05:07:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.398+0000 7f5093fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509416ce30 con 0x7f509415bb30 2023-10-06T05:07:12.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.398+0000 7f50927fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f509415b050 0x7f509406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:12.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.398+0000 7f50927fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f509415b050 0x7f509406d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50910/0 (socket says 172.21.15.55:50910) 2023-10-06T05:07:12.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.398+0000 7f50927fc700 1 -- 172.21.15.55:0/2642928139 learned_addr learned my addr 172.21.15.55:0/2642928139 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:12.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.398+0000 7f5092ffd700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50940cc8d0 0x7f5094147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:12.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.398+0000 7f50937fe700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f509415bb30 0x7f509406da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:12.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.399+0000 7f50927fc700 1 -- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f509415bb30 msgr2=0x7f509406da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:12.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.399+0000 7f50927fc700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f509415bb30 0x7f509406da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:12.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.399+0000 7f50927fc700 1 -- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50940cc8d0 msgr2=0x7f5094147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:12.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.399+0000 7f50927fc700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50940cc8d0 0x7f5094147ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:12.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.399+0000 7f50927fc700 1 -- 172.21.15.55:0/2642928139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c009a20 con 0x7f509415b050 2023-10-06T05:07:12.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.399+0000 7f5092ffd700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50940cc8d0 0x7f5094147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:12.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.399+0000 7f50927fc700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f509415b050 0x7f509406d4f0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f508800c4e0 tx=0x7f508800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:12.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.399+0000 7f507affd700 1 -- 172.21.15.55:0/2642928139 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088011590 con 0x7f509415b050 2023-10-06T05:07:12.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.399+0000 7f507affd700 1 -- 172.21.15.55:0/2642928139 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5088011710 con 0x7f509415b050 2023-10-06T05:07:12.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.399+0000 7f507affd700 1 -- 172.21.15.55:0/2642928139 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088011ad0 con 0x7f509415b050 2023-10-06T05:07:12.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.399+0000 7f5093fff700 1 -- 172.21.15.55:0/2642928139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509416d110 con 0x7f509415b050 2023-10-06T05:07:12.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.399+0000 7f5093fff700 1 -- 172.21.15.55:0/2642928139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50940c5480 con 0x7f509415b050 2023-10-06T05:07:12.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.400+0000 7f5093fff700 1 -- 172.21.15.55:0/2642928139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509416d910 con 0x7f509415b050 2023-10-06T05:07:12.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.401+0000 7f507affd700 1 -- 172.21.15.55:0/2642928139 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 99) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f508801b3b0 con 0x7f509415b050 2023-10-06T05:07:12.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.402+0000 7f507affd700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5080066d80 0x7f5080069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:12.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.402+0000 7f507affd700 1 -- 172.21.15.55:0/2642928139 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6756+0+0 (secure 0 0 0) 0x7f5088083ae0 con 0x7f509415b050 2023-10-06T05:07:12.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.403+0000 7f5092ffd700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5080066d80 0x7f5080069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:12.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.403+0000 7f5092ffd700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5080066d80 0x7f5080069230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f50840058d0 tx=0x7f5084009040 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:12.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.406+0000 7f507affd700 1 -- 172.21.15.55:0/2642928139 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f509416d910 con 0x7f509415b050 2023-10-06T05:07:12.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.608+0000 7f5093fff700 1 -- 172.21.15.55:0/2642928139 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f509404c3c0 con 0x7f509415b050 2023-10-06T05:07:12.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:12.976+0000 7f507affd700 1 -- 172.21.15.55:0/2642928139 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5088050ee0 con 0x7f509415b050 2023-10-06T05:07:13.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.174+0000 7f507affd700 1 -- 172.21.15.55:0/2642928139 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v199) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5088050b10 con 0x7f509415b050 2023-10-06T05:07:13.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.178+0000 7f5093fff700 1 -- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5080066d80 msgr2=0x7f5080069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:13.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.178+0000 7f5093fff700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5080066d80 0x7f5080069230 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f50840058d0 tx=0x7f5084009040 comp rx=0 tx=0).stop 2023-10-06T05:07:13.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.178+0000 7f5093fff700 1 -- 172.21.15.55:0/2642928139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f509415b050 msgr2=0x7f509406d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.178+0000 7f5093fff700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f509415b050 0x7f509406d4f0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f508800c4e0 tx=0x7f508800c7f0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.178+0000 7f5093fff700 1 -- 172.21.15.55:0/2642928139 shutdown_connections 2023-10-06T05:07:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.178+0000 7f5093fff700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5080066d80 0x7f5080069230 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.178+0000 7f5093fff700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f509415bb30 0x7f509406da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.178+0000 7f5093fff700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f50940cc8d0 0x7f5094147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.178+0000 7f5093fff700 1 --2- 172.21.15.55:0/2642928139 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f509415b050 0x7f509406d4f0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.178+0000 7f5093fff700 1 -- 172.21.15.55:0/2642928139 >> 172.21.15.55:0/2642928139 conn(0x7f5094068e20 msgr2=0x7f5094151eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.178+0000 7f5093fff700 1 -- 172.21.15.55:0/2642928139 shutdown_connections 2023-10-06T05:07:13.181 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.178+0000 7f5093fff700 1 -- 172.21.15.55:0/2642928139 wait complete. 2023-10-06T05:07:13.182 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2023-10-06T05:07:13.201 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:07:13.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.390+0000 7fe0cd39d700 1 Processor -- start 2023-10-06T05:07:13.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.390+0000 7fe0cd39d700 1 -- start start 2023-10-06T05:07:13.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.391+0000 7fe0cd39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c806a710 0x7fe0c8160e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:13.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.391+0000 7fe0cd39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c806e150 con 0x7fe0c8153900 2023-10-06T05:07:13.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.391+0000 7fe0cd39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c80c7800 con 0x7fe0c80c9db0 2023-10-06T05:07:13.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.391+0000 7fe0cd39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c80c7610 con 0x7fe0c806a710 2023-10-06T05:07:13.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.392+0000 7fe0c7fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe0c8153900 0x7fe0c80c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36566/0 (socket says 172.21.15.55:36566) 2023-10-06T05:07:13.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.392+0000 7fe0c7fff700 1 -- 172.21.15.55:0/2146663882 learned_addr learned my addr 172.21.15.55:0/2146663882 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:13.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.392+0000 7fe0ccb9c700 1 --2- 172.21.15.55:0/2146663882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c806a710 0x7fe0c8160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:13.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.392+0000 7fe0c6ffd700 1 -- 172.21.15.55:0/2146663882 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3353100258 0 0) 0x7fe0c806e150 con 0x7fe0c8153900 2023-10-06T05:07:13.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.392+0000 7fe0c6ffd700 1 -- 172.21.15.55:0/2146663882 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0b0003570 con 0x7fe0c8153900 2023-10-06T05:07:13.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.392+0000 7fe0c6ffd700 1 -- 172.21.15.55:0/2146663882 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1292041949 0 0) 0x7fe0b0003570 con 0x7fe0c8153900 2023-10-06T05:07:13.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c6ffd700 1 -- 172.21.15.55:0/2146663882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c806a710 msgr2=0x7fe0c8160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:13.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c6ffd700 1 --2- 172.21.15.55:0/2146663882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c806a710 0x7fe0c8160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c6ffd700 1 -- 172.21.15.55:0/2146663882 >> v1:172.21.15.163:6789/0 conn(0x7fe0c80c9db0 legacy=0x7fe0c806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:13.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c6ffd700 1 -- 172.21.15.55:0/2146663882 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c80c7ae0 con 0x7fe0c8153900 2023-10-06T05:07:13.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c6ffd700 1 -- 172.21.15.55:0/2146663882 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fe0b8001470 con 0x7fe0c8153900 2023-10-06T05:07:13.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c6ffd700 1 -- 172.21.15.55:0/2146663882 >> v1:172.21.15.55:6789/0 conn(0x7fe0c8153900 legacy=0x7fe0c80c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c6ffd700 1 --2- 172.21.15.55:0/2146663882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0b0003a90 0x7fe0b0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c6ffd700 1 --2- 172.21.15.55:0/2146663882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0b0004810 0x7fe0b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c6ffd700 1 --2- 172.21.15.55:0/2146663882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b00051c0 0x7fe0b000dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c6ffd700 1 -- 172.21.15.55:0/2146663882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b000e1e0 con 0x7fe0b0003a90 2023-10-06T05:07:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c6ffd700 1 -- 172.21.15.55:0/2146663882 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b000e350 con 0x7fe0b00051c0 2023-10-06T05:07:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c6ffd700 1 -- 172.21.15.55:0/2146663882 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b000e510 con 0x7fe0b0004810 2023-10-06T05:07:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0ccb9c700 1 --2- 172.21.15.55:0/2146663882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c806a710 0x7fe0c8160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c7fff700 1 --2- 172.21.15.55:0/2146663882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0b0003a90 0x7fe0b0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c77fe700 1 --2- 172.21.15.55:0/2146663882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0b0004810 0x7fe0b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:13.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0ccb9c700 1 --2- 172.21.15.55:0/2146663882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b00051c0 0x7fe0b000dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:13.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c7fff700 1 -- 172.21.15.55:0/2146663882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0b0004810 msgr2=0x7fe0b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:13.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c7fff700 1 --2- 172.21.15.55:0/2146663882 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0b0004810 0x7fe0b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c7fff700 1 -- 172.21.15.55:0/2146663882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b00051c0 msgr2=0x7fe0b000dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:13.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c7fff700 1 --2- 172.21.15.55:0/2146663882 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b00051c0 0x7fe0b000dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.393+0000 7fe0c7fff700 1 -- 172.21.15.55:0/2146663882 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b0003570 con 0x7fe0b0003a90 2023-10-06T05:07:13.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.394+0000 7fe0c7fff700 1 --2- 172.21.15.55:0/2146663882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0b0003a90 0x7fe0b0003e60 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800c2a0 tx=0x7fe0b8001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbfb3eefe80b192 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:13.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.394+0000 7fe0c6ffd700 1 -- 172.21.15.55:0/2146663882 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b800cb70 con 0x7fe0b0003a90 2023-10-06T05:07:13.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.394+0000 7fe0c6ffd700 1 -- 172.21.15.55:0/2146663882 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b800ccf0 con 0x7fe0b0003a90 2023-10-06T05:07:13.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.394+0000 7fe0c6ffd700 1 -- 172.21.15.55:0/2146663882 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8005b70 con 0x7fe0b0003a90 2023-10-06T05:07:13.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.394+0000 7fe0cd39d700 1 -- 172.21.15.55:0/2146663882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0b0003a90 msgr2=0x7fe0b0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:13.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.394+0000 7fe0cd39d700 1 --2- 172.21.15.55:0/2146663882 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0b0003a90 0x7fe0b0003e60 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800c2a0 tx=0x7fe0b8001da0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.394+0000 7fe0ccb9c700 1 -- 172.21.15.55:0/2146663882 reap_dead start 2023-10-06T05:07:13.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.395+0000 7fe0cd39d700 1 -- 172.21.15.55:0/2146663882 shutdown_connections 2023-10-06T05:07:13.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.395+0000 7fe0cd39d700 1 -- 172.21.15.55:0/2146663882 >> 172.21.15.55:0/2146663882 conn(0x7fe0c80cc1e0 msgr2=0x7fe0c80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:13.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.395+0000 7fe0cd39d700 1 -- 172.21.15.55:0/2146663882 shutdown_connections 2023-10-06T05:07:13.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.395+0000 7fe0cd39d700 1 -- 172.21.15.55:0/2146663882 wait complete. 2023-10-06T05:07:13.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.396+0000 7fe0cd39d700 1 Processor -- start 2023-10-06T05:07:13.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.396+0000 7fe0cd39d700 1 -- start start 2023-10-06T05:07:13.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.397+0000 7fe0cd39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0003a90 0x7fe0c8153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:13.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.397+0000 7fe0cd39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c80caf40 0x7fe0c814e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:13.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.397+0000 7fe0cd39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c80a98d0 0x7fe0c814fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:13.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.397+0000 7fe0cd39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c80ac640 con 0x7fe0c80a98d0 2023-10-06T05:07:13.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.397+0000 7fe0cd39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c80aaae0 con 0x7fe0b0003a90 2023-10-06T05:07:13.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.397+0000 7fe0cd39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c80ac900 con 0x7fe0c80caf40 2023-10-06T05:07:13.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.398+0000 7fe0ccb9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c80a98d0 0x7fe0c814fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:13.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.398+0000 7fe0ccb9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c80a98d0 0x7fe0c814fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45448/0 (socket says 172.21.15.55:45448) 2023-10-06T05:07:13.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.398+0000 7fe0ccb9c700 1 -- 172.21.15.55:0/3633199054 learned_addr learned my addr 172.21.15.55:0/3633199054 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:13.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.398+0000 7fe0c7fff700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0003a90 0x7fe0c8153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:13.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.398+0000 7fe0c77fe700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c80caf40 0x7fe0c814e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:13.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.398+0000 7fe0ccb9c700 1 -- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c80caf40 msgr2=0x7fe0c814e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:13.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.398+0000 7fe0ccb9c700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c80caf40 0x7fe0c814e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.398+0000 7fe0ccb9c700 1 -- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0003a90 msgr2=0x7fe0c8153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:13.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.398+0000 7fe0ccb9c700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0003a90 0x7fe0c8153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.398+0000 7fe0ccb9c700 1 -- 172.21.15.55:0/3633199054 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c80aa240 con 0x7fe0c80a98d0 2023-10-06T05:07:13.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.398+0000 7fe0c7fff700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0003a90 0x7fe0c8153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:13.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.398+0000 7fe0ccb9c700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c80a98d0 0x7fe0c814fb00 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe0c806b9d0 tx=0x7fe0c000c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:13.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.398+0000 7fe0affff700 1 -- 172.21.15.55:0/3633199054 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c000f040 con 0x7fe0c80a98d0 2023-10-06T05:07:13.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.399+0000 7fe0affff700 1 -- 172.21.15.55:0/3633199054 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c000a280 con 0x7fe0c80a98d0 2023-10-06T05:07:13.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.399+0000 7fe0affff700 1 -- 172.21.15.55:0/3633199054 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0010d20 con 0x7fe0c80a98d0 2023-10-06T05:07:13.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.399+0000 7fe0cd39d700 1 -- 172.21.15.55:0/3633199054 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c80aa520 con 0x7fe0c80a98d0 2023-10-06T05:07:13.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.399+0000 7fe0cd39d700 1 -- 172.21.15.55:0/3633199054 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c814e7a0 con 0x7fe0c80a98d0 2023-10-06T05:07:13.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.400+0000 7fe0cd39d700 1 -- 172.21.15.55:0/3633199054 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c80aca90 con 0x7fe0c80a98d0 2023-10-06T05:07:13.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.401+0000 7fe0affff700 1 -- 172.21.15.55:0/3633199054 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe0c0015660 con 0x7fe0c80a98d0 2023-10-06T05:07:13.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.401+0000 7fe0affff700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe0b4066f30 0x7fe0b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:13.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.402+0000 7fe0affff700 1 -- 172.21.15.55:0/3633199054 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6342+0+0 (secure 0 0 0) 0x7fe0c0085220 con 0x7fe0c80a98d0 2023-10-06T05:07:13.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.402+0000 7fe0c7fff700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe0b4066f30 0x7fe0b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:13.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.403+0000 7fe0c7fff700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe0b4066f30 0x7fe0b40693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8007c20 tx=0x7fe0b8015000 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:13.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.406+0000 7fe0affff700 1 -- 172.21.15.55:0/3633199054 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe0c0052170 con 0x7fe0c80a98d0 2023-10-06T05:07:13.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.617+0000 7fe0cd39d700 1 -- 172.21.15.55:0/3633199054 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe0c80c7dc0 con 0x7fe0c80a98d0 2023-10-06T05:07:13.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.621+0000 7fe0affff700 1 -- 172.21.15.55:0/3633199054 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe0c0055790 con 0x7fe0c80a98d0 2023-10-06T05:07:13.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.623+0000 7fe0cd39d700 1 -- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe0b4066f30 msgr2=0x7fe0b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:13.626 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.623+0000 7fe0cd39d700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe0b4066f30 0x7fe0b40693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8007c20 tx=0x7fe0b8015000 comp rx=0 tx=0).stop 2023-10-06T05:07:13.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.623+0000 7fe0cd39d700 1 -- 172.21.15.55:0/3633199054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c80a98d0 msgr2=0x7fe0c814fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:13.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.623+0000 7fe0cd39d700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c80a98d0 0x7fe0c814fb00 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe0c806b9d0 tx=0x7fe0c000c040 comp rx=0 tx=0).stop 2023-10-06T05:07:13.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.623+0000 7fe0cd39d700 1 -- 172.21.15.55:0/3633199054 shutdown_connections 2023-10-06T05:07:13.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.623+0000 7fe0cd39d700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe0b4066f30 0x7fe0b40693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.623+0000 7fe0cd39d700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe0c80a98d0 0x7fe0c814fb00 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.623+0000 7fe0cd39d700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe0b0003a90 0x7fe0c8153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.623+0000 7fe0cd39d700 1 --2- 172.21.15.55:0/3633199054 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe0c80caf40 0x7fe0c814e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.623+0000 7fe0cd39d700 1 -- 172.21.15.55:0/3633199054 >> 172.21.15.55:0/3633199054 conn(0x7fe0c80cc1e0 msgr2=0x7fe0c80c2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:13.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.623+0000 7fe0cd39d700 1 -- 172.21.15.55:0/3633199054 shutdown_connections 2023-10-06T05:07:13.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.624+0000 7fe0cd39d700 1 -- 172.21.15.55:0/3633199054 wait complete. 2023-10-06T05:07:13.643 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T05:07:13.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.830+0000 7f18d7fff700 1 Processor -- start 2023-10-06T05:07:13.832 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.831+0000 7f18d7fff700 1 -- start start 2023-10-06T05:07:13.836 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.832+0000 7f18d7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18d80aac80 0x7f18d80ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:13.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.832+0000 7f18d7fff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f18d8159de0 0x7f18d815a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:13.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.832+0000 7f18d7fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d80c89a0 con 0x7f18d8159a10 2023-10-06T05:07:13.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.832+0000 7f18d7fff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d80caee0 con 0x7f18d8159de0 2023-10-06T05:07:13.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.832+0000 7f18d7fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d80c7db0 con 0x7f18d80aac80 2023-10-06T05:07:13.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.832+0000 7f18d77fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f18d8159de0 0x7f18d815a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:13.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.832+0000 7f18d67fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18d80aac80 0x7f18d80ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:13.837 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.832+0000 7f18d77fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f18d8159de0 0x7f18d815a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:35686/0 (socket says 172.21.15.55:35686) 2023-10-06T05:07:13.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.832+0000 7f18d77fe700 1 -- 172.21.15.55:0/3648384905 learned_addr learned my addr 172.21.15.55:0/3648384905 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:13.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.832+0000 7f18d6ffd700 1 --1- 172.21.15.55:0/3648384905 >> v1:172.21.15.163:6789/0 conn(0x7f18d8159a10 0x7f18d80c81e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:39712/0 (socket says 172.21.15.55:39712) 2023-10-06T05:07:13.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.833+0000 7f18d5ffb700 1 -- 172.21.15.55:0/3648384905 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767898985 0 0) 0x7f18d80c89a0 con 0x7f18d8159a10 2023-10-06T05:07:13.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.833+0000 7f18d77fe700 1 -- 172.21.15.55:0/3648384905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18d80aac80 msgr2=0x7f18d80ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:13.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.833+0000 7f18d77fe700 1 --2- 172.21.15.55:0/3648384905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18d80aac80 0x7f18d80ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.838 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.833+0000 7f18d77fe700 1 -- 172.21.15.55:0/3648384905 >> v1:172.21.15.163:6789/0 conn(0x7f18d8159a10 legacy=0x7f18d80c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:13.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.833+0000 7f18d77fe700 1 -- 172.21.15.55:0/3648384905 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d80c7510 con 0x7f18d8159de0 2023-10-06T05:07:13.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.833+0000 7f18d67fc700 1 --2- 172.21.15.55:0/3648384905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18d80aac80 0x7f18d80ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:13.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.833+0000 7f18d77fe700 1 --2- 172.21.15.55:0/3648384905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d8159de0 0x7f18d815a1d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009f50 tx=0x7f18d0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46b6b852f1251020 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:13.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.833+0000 7f18d5ffb700 1 -- 172.21.15.55:0/3648384905 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d000e070 con 0x7f18d8159de0 2023-10-06T05:07:13.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.834+0000 7f18d5ffb700 1 -- 172.21.15.55:0/3648384905 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d000b9c0 con 0x7f18d8159de0 2023-10-06T05:07:13.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.834+0000 7f18d5ffb700 1 -- 172.21.15.55:0/3648384905 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d00092e0 con 0x7f18d8159de0 2023-10-06T05:07:13.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.834+0000 7f18d7fff700 1 -- 172.21.15.55:0/3648384905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d8159de0 msgr2=0x7f18d815a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.834+0000 7f18d7fff700 1 --2- 172.21.15.55:0/3648384905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d8159de0 0x7f18d815a1d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009f50 tx=0x7f18d0009840 comp rx=0 tx=0).stop 2023-10-06T05:07:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.834+0000 7f18d7fff700 1 -- 172.21.15.55:0/3648384905 shutdown_connections 2023-10-06T05:07:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.834+0000 7f18d7fff700 1 --2- 172.21.15.55:0/3648384905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d8159de0 0x7f18d815a1d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.834+0000 7f18d7fff700 1 --2- 172.21.15.55:0/3648384905 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18d80aac80 0x7f18d80ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.834+0000 7f18d7fff700 1 -- 172.21.15.55:0/3648384905 >> 172.21.15.55:0/3648384905 conn(0x7f18d80cc1e0 msgr2=0x7f18d80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.834+0000 7f18d7fff700 1 -- 172.21.15.55:0/3648384905 shutdown_connections 2023-10-06T05:07:13.842 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.834+0000 7f18d7fff700 1 -- 172.21.15.55:0/3648384905 wait complete. 2023-10-06T05:07:13.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.836+0000 7f18d7fff700 1 Processor -- start 2023-10-06T05:07:13.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.836+0000 7f18d7fff700 1 -- start start 2023-10-06T05:07:13.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d7fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18d80aac80 0x7f18d80c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:13.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d8159a10 0x7f18d80adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:13.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d7fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18d8159de0 0x7f18d8153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:13.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d7fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d80abd50 con 0x7f18d8159de0 2023-10-06T05:07:13.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d7fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d80ac030 con 0x7f18d8159a10 2023-10-06T05:07:13.845 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d7fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d80ac310 con 0x7f18d80aac80 2023-10-06T05:07:13.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18d8159de0 0x7f18d8153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:13.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18d8159de0 0x7f18d8153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45462/0 (socket says 172.21.15.55:45462) 2023-10-06T05:07:13.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d77fe700 1 -- 172.21.15.55:0/2363066875 learned_addr learned my addr 172.21.15.55:0/2363066875 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:13.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d6ffd700 1 --2- 172.21.15.55:0/2363066875 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18d80aac80 0x7f18d80c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:13.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d67fc700 1 --2- 172.21.15.55:0/2363066875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d8159a10 0x7f18d80adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:13.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d77fe700 1 -- 172.21.15.55:0/2363066875 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18d80aac80 msgr2=0x7f18d80c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:13.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d77fe700 1 --2- 172.21.15.55:0/2363066875 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18d80aac80 0x7f18d80c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d77fe700 1 -- 172.21.15.55:0/2363066875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d8159a10 msgr2=0x7f18d80adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:13.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d77fe700 1 --2- 172.21.15.55:0/2363066875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d8159a10 0x7f18d80adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:13.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.837+0000 7f18d77fe700 1 -- 172.21.15.55:0/2363066875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc009420 con 0x7f18d8159de0 2023-10-06T05:07:13.846 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.838+0000 7f18d77fe700 1 --2- 172.21.15.55:0/2363066875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18d8159de0 0x7f18d8153060 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f18d000b6c0 tx=0x7f18d0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:13.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.838+0000 7f18beffd700 1 -- 172.21.15.55:0/2363066875 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0008b10 con 0x7f18d8159de0 2023-10-06T05:07:13.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.838+0000 7f18beffd700 1 -- 172.21.15.55:0/2363066875 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d0008cb0 con 0x7f18d8159de0 2023-10-06T05:07:13.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.838+0000 7f18beffd700 1 -- 172.21.15.55:0/2363066875 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d00256b0 con 0x7f18d8159de0 2023-10-06T05:07:13.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.838+0000 7f18d7fff700 1 -- 172.21.15.55:0/2363066875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d0009ae0 con 0x7f18d8159de0 2023-10-06T05:07:13.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.838+0000 7f18d7fff700 1 -- 172.21.15.55:0/2363066875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d80ac8d0 con 0x7f18d8159de0 2023-10-06T05:07:13.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.839+0000 7f18d7fff700 1 -- 172.21.15.55:0/2363066875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d80acac0 con 0x7f18d8159de0 2023-10-06T05:07:13.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.840+0000 7f18beffd700 1 -- 172.21.15.55:0/2363066875 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18d0003040 con 0x7f18d8159de0 2023-10-06T05:07:13.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.841+0000 7f18beffd700 1 --2- 172.21.15.55:0/2363066875 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f18c4066f30 0x7f18c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:13.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.841+0000 7f18beffd700 1 -- 172.21.15.55:0/2363066875 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6342+0+0 (secure 0 0 0) 0x7f18d0085d20 con 0x7f18d8159de0 2023-10-06T05:07:13.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.842+0000 7f18d6ffd700 1 --2- 172.21.15.55:0/2363066875 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f18c4066f30 0x7f18c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:13.847 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.842+0000 7f18d6ffd700 1 --2- 172.21.15.55:0/2363066875 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f18c4066f30 0x7f18c40693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f18c8007fd0 tx=0x7f18c8007d50 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:13.848 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:13.845+0000 7f18beffd700 1 -- 172.21.15.55:0/2363066875 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18d0052c70 con 0x7f18d8159de0 2023-10-06T05:07:14.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.049+0000 7f18d7fff700 1 -- 172.21.15.55:0/2363066875 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f18d80c77f0 con 0x7f18d8159de0 2023-10-06T05:07:14.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.054+0000 7f18beffd700 1 -- 172.21.15.55:0/2363066875 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v68) v1 ==== 115+0+0 (secure 0 0 0) 0x7f18d0056290 con 0x7f18d8159de0 2023-10-06T05:07:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.056+0000 7f18d7fff700 1 -- 172.21.15.55:0/2363066875 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f18c4066f30 msgr2=0x7f18c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.056+0000 7f18d7fff700 1 --2- 172.21.15.55:0/2363066875 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f18c4066f30 0x7f18c40693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f18c8007fd0 tx=0x7f18c8007d50 comp rx=0 tx=0).stop 2023-10-06T05:07:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.056+0000 7f18d7fff700 1 -- 172.21.15.55:0/2363066875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18d8159de0 msgr2=0x7f18d8153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.056+0000 7f18d7fff700 1 --2- 172.21.15.55:0/2363066875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18d8159de0 0x7f18d8153060 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f18d000b6c0 tx=0x7f18d0018000 comp rx=0 tx=0).stop 2023-10-06T05:07:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.057+0000 7f18d7fff700 1 -- 172.21.15.55:0/2363066875 shutdown_connections 2023-10-06T05:07:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.057+0000 7f18d7fff700 1 --2- 172.21.15.55:0/2363066875 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f18c4066f30 0x7f18c40693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.057+0000 7f18d7fff700 1 --2- 172.21.15.55:0/2363066875 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f18d8159de0 0x7f18d8153060 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.057+0000 7f18d7fff700 1 --2- 172.21.15.55:0/2363066875 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f18d80aac80 0x7f18d80c21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.059 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.057+0000 7f18d7fff700 1 --2- 172.21.15.55:0/2363066875 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f18d8159a10 0x7f18d80adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.057+0000 7f18d7fff700 1 -- 172.21.15.55:0/2363066875 >> 172.21.15.55:0/2363066875 conn(0x7f18d80cc1e0 msgr2=0x7f18d8155290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.057+0000 7f18d7fff700 1 -- 172.21.15.55:0/2363066875 shutdown_connections 2023-10-06T05:07:14.060 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.057+0000 7f18d7fff700 1 -- 172.21.15.55:0/2363066875 wait complete. 2023-10-06T05:07:14.076 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T05:07:14.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.264+0000 7fad2119d700 1 Processor -- start 2023-10-06T05:07:14.266 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.264+0000 7fad2119d700 1 -- start start 2023-10-06T05:07:14.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.266+0000 7fad2119d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad1c0cb1a0 0x7fad1c15a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:14.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.266+0000 7fad2119d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fad1c15a8a0 0x7fad1c06f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:14.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.266+0000 7fad2119d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1c06ebd0 con 0x7fad1c06f5a0 2023-10-06T05:07:14.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.266+0000 7fad2119d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1c15b2e0 con 0x7fad1c15a8a0 2023-10-06T05:07:14.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.266+0000 7fad2119d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c15ad20 con 0x7fad1c0cb1a0 2023-10-06T05:07:14.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.266+0000 7fad1b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fad1c15a8a0 0x7fad1c06f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:14.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.266+0000 7fad1b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fad1c15a8a0 0x7fad1c06f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45466/0 (socket says 172.21.15.55:45466) 2023-10-06T05:07:14.268 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.266+0000 7fad1b7fe700 1 -- 172.21.15.55:0/3005066410 learned_addr learned my addr 172.21.15.55:0/3005066410 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:14.269 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.266+0000 7fad1bfff700 1 --2- 172.21.15.55:0/3005066410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad1c0cb1a0 0x7fad1c15a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:14.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.266+0000 7fad1b7fe700 1 -- 172.21.15.55:0/3005066410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad1c0cb1a0 msgr2=0x7fad1c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:14.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.266+0000 7fad1b7fe700 1 --2- 172.21.15.55:0/3005066410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad1c0cb1a0 0x7fad1c15a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.266+0000 7fad1b7fe700 1 -- 172.21.15.55:0/3005066410 >> v1:172.21.15.163:6789/0 conn(0x7fad1c06f5a0 legacy=0x7fad1c0ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:14.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.266+0000 7fad1b7fe700 1 -- 172.21.15.55:0/3005066410 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c15b000 con 0x7fad1c15a8a0 2023-10-06T05:07:14.270 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.267+0000 7fad1b7fe700 1 --2- 172.21.15.55:0/3005066410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad1c15a8a0 0x7fad1c06f060 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fad100099b0 tx=0x7fad10009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7ee9b3fa47fd0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:14.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.267+0000 7fad1affd700 1 -- 172.21.15.55:0/3005066410 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1000b8f0 con 0x7fad1c15a8a0 2023-10-06T05:07:14.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.267+0000 7fad1affd700 1 -- 172.21.15.55:0/3005066410 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1000ba50 con 0x7fad1c15a8a0 2023-10-06T05:07:14.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.267+0000 7fad1affd700 1 -- 172.21.15.55:0/3005066410 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1000bdf0 con 0x7fad1c15a8a0 2023-10-06T05:07:14.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.267+0000 7fad2119d700 1 -- 172.21.15.55:0/3005066410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad1c15a8a0 msgr2=0x7fad1c06f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:14.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.267+0000 7fad2119d700 1 --2- 172.21.15.55:0/3005066410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad1c15a8a0 0x7fad1c06f060 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fad100099b0 tx=0x7fad10009250 comp rx=0 tx=0).stop 2023-10-06T05:07:14.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.268+0000 7fad2119d700 1 -- 172.21.15.55:0/3005066410 shutdown_connections 2023-10-06T05:07:14.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.268+0000 7fad2119d700 1 --2- 172.21.15.55:0/3005066410 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad1c0cb1a0 0x7fad1c15a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.268+0000 7fad2119d700 1 --2- 172.21.15.55:0/3005066410 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad1c15a8a0 0x7fad1c06f060 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.268+0000 7fad2119d700 1 -- 172.21.15.55:0/3005066410 >> 172.21.15.55:0/3005066410 conn(0x7fad1c0cc1e0 msgr2=0x7fad1c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.268+0000 7fad2119d700 1 -- 172.21.15.55:0/3005066410 shutdown_connections 2023-10-06T05:07:14.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.268+0000 7fad2119d700 1 -- 172.21.15.55:0/3005066410 wait complete. 2023-10-06T05:07:14.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.269+0000 7fad2119d700 1 Processor -- start 2023-10-06T05:07:14.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.269+0000 7fad2119d700 1 -- start start 2023-10-06T05:07:14.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.270+0000 7fad2119d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad1c06f5a0 0x7fad1c16adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:14.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.270+0000 7fad2119d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad1c0cb1a0 0x7fad1c169c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:14.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.270+0000 7fad2119d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad1c159ec0 0x7fad1c16c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:14.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.270+0000 7fad2119d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c151d20 con 0x7fad1c159ec0 2023-10-06T05:07:14.274 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.270+0000 7fad2119d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c16a440 con 0x7fad1c0cb1a0 2023-10-06T05:07:14.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.270+0000 7fad2119d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c16a720 con 0x7fad1c06f5a0 2023-10-06T05:07:14.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.270+0000 7fad2099c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad1c159ec0 0x7fad1c16c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:14.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.270+0000 7fad1bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad1c06f5a0 0x7fad1c16adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:14.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.271+0000 7fad1bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad1c06f5a0 0x7fad1c16adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:39564/0 (socket says 172.21.15.55:39564) 2023-10-06T05:07:14.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.271+0000 7fad1bfff700 1 -- 172.21.15.55:0/1313061671 learned_addr learned my addr 172.21.15.55:0/1313061671 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:14.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.271+0000 7fad1b7fe700 1 --2- 172.21.15.55:0/1313061671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad1c0cb1a0 0x7fad1c169c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:14.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.271+0000 7fad2099c700 1 -- 172.21.15.55:0/1313061671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad1c06f5a0 msgr2=0x7fad1c16adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:14.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.271+0000 7fad2099c700 1 --2- 172.21.15.55:0/1313061671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad1c06f5a0 0x7fad1c16adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.271+0000 7fad2099c700 1 -- 172.21.15.55:0/1313061671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad1c0cb1a0 msgr2=0x7fad1c169c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:14.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.271+0000 7fad2099c700 1 --2- 172.21.15.55:0/1313061671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad1c0cb1a0 0x7fad1c169c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.271+0000 7fad2099c700 1 -- 172.21.15.55:0/1313061671 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad100094f0 con 0x7fad1c159ec0 2023-10-06T05:07:14.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.271+0000 7fad2099c700 1 --2- 172.21.15.55:0/1313061671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad1c159ec0 0x7fad1c16c960 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fad1c06f9a0 tx=0x7fad14001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:14.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.271+0000 7fad03fff700 1 -- 172.21.15.55:0/1313061671 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad14012630 con 0x7fad1c159ec0 2023-10-06T05:07:14.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.272+0000 7fad03fff700 1 -- 172.21.15.55:0/1313061671 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad140127b0 con 0x7fad1c159ec0 2023-10-06T05:07:14.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.272+0000 7fad03fff700 1 -- 172.21.15.55:0/1313061671 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad14012b70 con 0x7fad1c159ec0 2023-10-06T05:07:14.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.272+0000 7fad2119d700 1 -- 172.21.15.55:0/1313061671 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c16aa00 con 0x7fad1c159ec0 2023-10-06T05:07:14.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.272+0000 7fad2119d700 1 -- 172.21.15.55:0/1313061671 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c16a160 con 0x7fad1c159ec0 2023-10-06T05:07:14.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.273+0000 7fad2119d700 1 -- 172.21.15.55:0/1313061671 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1c06c450 con 0x7fad1c159ec0 2023-10-06T05:07:14.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.273+0000 7fad03fff700 1 -- 172.21.15.55:0/1313061671 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad14003030 con 0x7fad1c159ec0 2023-10-06T05:07:14.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.274+0000 7fad03fff700 1 --2- 172.21.15.55:0/1313061671 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad08066f30 0x7fad080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:14.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.274+0000 7fad03fff700 1 -- 172.21.15.55:0/1313061671 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6342+0+0 (secure 0 0 0) 0x7fad140846a0 con 0x7fad1c159ec0 2023-10-06T05:07:14.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.275+0000 7fad1bfff700 1 --2- 172.21.15.55:0/1313061671 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad08066f30 0x7fad080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:14.278 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.276+0000 7fad1bfff700 1 --2- 172.21.15.55:0/1313061671 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad08066f30 0x7fad080693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fad0c006fd0 tx=0x7fad0c008040 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:14.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.279+0000 7fad03fff700 1 -- 172.21.15.55:0/1313061671 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad140515f0 con 0x7fad1c159ec0 2023-10-06T05:07:14.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.479+0000 7fad2119d700 1 -- 172.21.15.55:0/1313061671 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fad1c06e860 con 0x7fad1c159ec0 2023-10-06T05:07:14.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.483+0000 7fad03fff700 1 -- 172.21.15.55:0/1313061671 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v69) v1 ==== 120+0+0 (secure 0 0 0) 0x7fad14054c10 con 0x7fad1c159ec0 2023-10-06T05:07:14.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.485+0000 7fad2119d700 1 -- 172.21.15.55:0/1313061671 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad08066f30 msgr2=0x7fad080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:14.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.485+0000 7fad2119d700 1 --2- 172.21.15.55:0/1313061671 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad08066f30 0x7fad080693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fad0c006fd0 tx=0x7fad0c008040 comp rx=0 tx=0).stop 2023-10-06T05:07:14.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.485+0000 7fad2119d700 1 -- 172.21.15.55:0/1313061671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad1c159ec0 msgr2=0x7fad1c16c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:14.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.485+0000 7fad2119d700 1 --2- 172.21.15.55:0/1313061671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad1c159ec0 0x7fad1c16c960 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fad1c06f9a0 tx=0x7fad14001690 comp rx=0 tx=0).stop 2023-10-06T05:07:14.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.486+0000 7fad2119d700 1 -- 172.21.15.55:0/1313061671 shutdown_connections 2023-10-06T05:07:14.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.486+0000 7fad2119d700 1 --2- 172.21.15.55:0/1313061671 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fad08066f30 0x7fad080693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.486+0000 7fad2119d700 1 --2- 172.21.15.55:0/1313061671 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fad1c159ec0 0x7fad1c16c960 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.486+0000 7fad2119d700 1 --2- 172.21.15.55:0/1313061671 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fad1c06f5a0 0x7fad1c16adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.486+0000 7fad2119d700 1 --2- 172.21.15.55:0/1313061671 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fad1c0cb1a0 0x7fad1c169c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.486+0000 7fad2119d700 1 -- 172.21.15.55:0/1313061671 >> 172.21.15.55:0/1313061671 conn(0x7fad1c0cc1e0 msgr2=0x7fad1c150900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:14.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.486+0000 7fad2119d700 1 -- 172.21.15.55:0/1313061671 shutdown_connections 2023-10-06T05:07:14.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.486+0000 7fad2119d700 1 -- 172.21.15.55:0/1313061671 wait complete. 2023-10-06T05:07:14.506 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-06T05:07:14.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.692+0000 7fdec999d700 1 Processor -- start 2023-10-06T05:07:14.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.692+0000 7fdec999d700 1 -- start start 2023-10-06T05:07:14.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.694+0000 7fdec999d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdec406d950 0x7fdec40c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:14.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.694+0000 7fdec999d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec406a0a0 0x7fdec40c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:14.696 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.694+0000 7fdec999d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec406ae70 con 0x7fdec406dd20 2023-10-06T05:07:14.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.694+0000 7fdec999d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec406a540 con 0x7fdec406d950 2023-10-06T05:07:14.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.694+0000 7fdec999d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec406a820 con 0x7fdec406a0a0 2023-10-06T05:07:14.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.694+0000 7fdec899b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdec406d950 0x7fdec40c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:14.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.694+0000 7fdec899b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdec406d950 0x7fdec40c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45482/0 (socket says 172.21.15.55:45482) 2023-10-06T05:07:14.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.694+0000 7fdec899b700 1 -- 172.21.15.55:0/363667288 learned_addr learned my addr 172.21.15.55:0/363667288 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:14.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.694+0000 7fdec3fff700 1 --2- 172.21.15.55:0/363667288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec406a0a0 0x7fdec40c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:14.697 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.694+0000 7fdec899b700 1 -- 172.21.15.55:0/363667288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec406a0a0 msgr2=0x7fdec40c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:14.698 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.694+0000 7fdec899b700 1 --2- 172.21.15.55:0/363667288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec406a0a0 0x7fdec40c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.698 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.695+0000 7fdec899b700 1 -- 172.21.15.55:0/363667288 >> v1:172.21.15.163:6789/0 conn(0x7fdec406dd20 legacy=0x7fdec40c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:14.698 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.695+0000 7fdec899b700 1 -- 172.21.15.55:0/363667288 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec406ab00 con 0x7fdec406d950 2023-10-06T05:07:14.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.695+0000 7fdec899b700 1 --2- 172.21.15.55:0/363667288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec406d950 0x7fdec40c8350 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80099b0 tx=0x7fdeb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe4dd3bd5611d7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:14.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.695+0000 7fdec37fe700 1 -- 172.21.15.55:0/363667288 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb800b860 con 0x7fdec406d950 2023-10-06T05:07:14.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.695+0000 7fdec37fe700 1 -- 172.21.15.55:0/363667288 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb800b9c0 con 0x7fdec406d950 2023-10-06T05:07:14.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.695+0000 7fdec37fe700 1 -- 172.21.15.55:0/363667288 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb800bd60 con 0x7fdec406d950 2023-10-06T05:07:14.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.696+0000 7fdec999d700 1 -- 172.21.15.55:0/363667288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec406d950 msgr2=0x7fdec40c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:14.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.696+0000 7fdec999d700 1 --2- 172.21.15.55:0/363667288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec406d950 0x7fdec40c8350 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80099b0 tx=0x7fdeb8009250 comp rx=0 tx=0).stop 2023-10-06T05:07:14.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.696+0000 7fdec999d700 1 -- 172.21.15.55:0/363667288 shutdown_connections 2023-10-06T05:07:14.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.696+0000 7fdec999d700 1 --2- 172.21.15.55:0/363667288 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec406d950 0x7fdec40c8350 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.696+0000 7fdec999d700 1 --2- 172.21.15.55:0/363667288 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec406a0a0 0x7fdec40c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.696+0000 7fdec999d700 1 -- 172.21.15.55:0/363667288 >> 172.21.15.55:0/363667288 conn(0x7fdec40cc1f0 msgr2=0x7fdec40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:14.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.696+0000 7fdec999d700 1 -- 172.21.15.55:0/363667288 shutdown_connections 2023-10-06T05:07:14.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.696+0000 7fdec999d700 1 -- 172.21.15.55:0/363667288 wait complete. 2023-10-06T05:07:14.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.697+0000 7fdec999d700 1 Processor -- start 2023-10-06T05:07:14.701 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.697+0000 7fdec999d700 1 -- start start 2023-10-06T05:07:14.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.698+0000 7fdec999d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec406a0a0 0x7fdec4157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:14.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.698+0000 7fdec999d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec406d950 0x7fdec4146d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:14.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.698+0000 7fdec999d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec406dd20 0x7fdec4153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:14.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.698+0000 7fdec999d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec4159a20 con 0x7fdec406d950 2023-10-06T05:07:14.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.698+0000 7fdec999d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec4158520 con 0x7fdec406dd20 2023-10-06T05:07:14.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.698+0000 7fdec999d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40c4200 con 0x7fdec406a0a0 2023-10-06T05:07:14.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.699+0000 7fdec3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec406d950 0x7fdec4146d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:14.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.699+0000 7fdec3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec406d950 0x7fdec4146d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45496/0 (socket says 172.21.15.55:45496) 2023-10-06T05:07:14.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.699+0000 7fdec3fff700 1 -- 172.21.15.55:0/20133517 learned_addr learned my addr 172.21.15.55:0/20133517 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:14.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.699+0000 7fdec899b700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec406a0a0 0x7fdec4157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:14.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.699+0000 7fdec919c700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec406dd20 0x7fdec4153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:14.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.699+0000 7fdec3fff700 1 -- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec406a0a0 msgr2=0x7fdec4157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.699+0000 7fdec3fff700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec406a0a0 0x7fdec4157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.699+0000 7fdec3fff700 1 -- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec406dd20 msgr2=0x7fdec4153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.699+0000 7fdec3fff700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec406dd20 0x7fdec4153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.699+0000 7fdec3fff700 1 -- 172.21.15.55:0/20133517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb80094f0 con 0x7fdec406d950 2023-10-06T05:07:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.699+0000 7fdec919c700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec406dd20 0x7fdec4153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.699+0000 7fdec3fff700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec406d950 0x7fdec4146d30 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000d8c0 tx=0x7fdeb0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.700+0000 7fdec0ff9700 1 -- 172.21.15.55:0/20133517 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb00125f0 con 0x7fdec406d950 2023-10-06T05:07:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.700+0000 7fdec0ff9700 1 -- 172.21.15.55:0/20133517 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb0012770 con 0x7fdec406d950 2023-10-06T05:07:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.700+0000 7fdec999d700 1 -- 172.21.15.55:0/20133517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec40c3d70 con 0x7fdec406d950 2023-10-06T05:07:14.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.700+0000 7fdec0ff9700 1 -- 172.21.15.55:0/20133517 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0012b30 con 0x7fdec406d950 2023-10-06T05:07:14.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.700+0000 7fdec999d700 1 -- 172.21.15.55:0/20133517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec40c3720 con 0x7fdec406d950 2023-10-06T05:07:14.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.701+0000 7fdec999d700 1 -- 172.21.15.55:0/20133517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec404c3c0 con 0x7fdec406d950 2023-10-06T05:07:14.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.702+0000 7fdec0ff9700 1 -- 172.21.15.55:0/20133517 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdeb001e460 con 0x7fdec406d950 2023-10-06T05:07:14.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.702+0000 7fdec0ff9700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdea8066f30 0x7fdea80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:14.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.703+0000 7fdec0ff9700 1 -- 172.21.15.55:0/20133517 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6342+0+0 (secure 0 0 0) 0x7fdeb0084e20 con 0x7fdec406d950 2023-10-06T05:07:14.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.703+0000 7fdec899b700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdea8066f30 0x7fdea80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:14.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.704+0000 7fdec899b700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdea8066f30 0x7fdea80693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80099b0 tx=0x7fdeb8017000 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:14.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.707+0000 7fdec0ff9700 1 -- 172.21.15.55:0/20133517 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdeb0051d70 con 0x7fdec406d950 2023-10-06T05:07:14.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:14.988+0000 7fdec999d700 1 -- 172.21.15.55:0/20133517 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7fdec41536c0 con 0x7fdec406d950 2023-10-06T05:07:15.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.173+0000 7fdec0ff9700 1 -- 172.21.15.55:0/20133517 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3036) v1 ==== 163+0+0 (secure 0 0 0) 0x7fdeb0055390 con 0x7fdec406d950 2023-10-06T05:07:15.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.176+0000 7fdec999d700 1 -- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdea8066f30 msgr2=0x7fdea80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:15.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.176+0000 7fdec999d700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdea8066f30 0x7fdea80693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80099b0 tx=0x7fdeb8017000 comp rx=0 tx=0).stop 2023-10-06T05:07:15.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.176+0000 7fdec999d700 1 -- 172.21.15.55:0/20133517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec406d950 msgr2=0x7fdec4146d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:15.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.176+0000 7fdec999d700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec406d950 0x7fdec4146d30 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000d8c0 tx=0x7fdeb0008040 comp rx=0 tx=0).stop 2023-10-06T05:07:15.178 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.176+0000 7fdec999d700 1 -- 172.21.15.55:0/20133517 shutdown_connections 2023-10-06T05:07:15.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.176+0000 7fdec999d700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdea8066f30 0x7fdea80693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:15.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.176+0000 7fdec999d700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdec406dd20 0x7fdec4153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:15.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.176+0000 7fdec999d700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdec406a0a0 0x7fdec4157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:15.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.176+0000 7fdec999d700 1 --2- 172.21.15.55:0/20133517 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdec406d950 0x7fdec4146d30 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:15.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.176+0000 7fdec999d700 1 -- 172.21.15.55:0/20133517 >> 172.21.15.55:0/20133517 conn(0x7fdec40cc1f0 msgr2=0x7fdec40c17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:15.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.176+0000 7fdec999d700 1 -- 172.21.15.55:0/20133517 shutdown_connections 2023-10-06T05:07:15.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.176+0000 7fdec999d700 1 -- 172.21.15.55:0/20133517 wait complete. 2023-10-06T05:07:15.196 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-10-06T05:07:15.196 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-10-06T05:07:15.196 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-06T05:07:15.196 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-06T05:07:15.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.383+0000 7f6230b9d700 1 Processor -- start 2023-10-06T05:07:15.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.383+0000 7f6230b9d700 1 -- start start 2023-10-06T05:07:15.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.384+0000 7f6230b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f622c156310 0x7f622c0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:15.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.384+0000 7f6230b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f622c1566e0 0x7f622c0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:15.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.384+0000 7f6230b9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f622c0ca860 con 0x7f622c0ca300 2023-10-06T05:07:15.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.384+0000 7f6230b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622c0cab40 con 0x7f622c156310 2023-10-06T05:07:15.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.384+0000 7f6230b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622c0cae20 con 0x7f622c1566e0 2023-10-06T05:07:15.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.385+0000 7f622b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f622c156310 0x7f622c0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:15.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.385+0000 7f622b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f622c156310 0x7f622c0c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45504/0 (socket says 172.21.15.55:45504) 2023-10-06T05:07:15.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.385+0000 7f622b7fe700 1 -- 172.21.15.55:0/2294958630 learned_addr learned my addr 172.21.15.55:0/2294958630 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:15.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.385+0000 7f622affd700 1 --2- 172.21.15.55:0/2294958630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f622c1566e0 0x7f622c0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:15.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.385+0000 7f622b7fe700 1 -- 172.21.15.55:0/2294958630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f622c1566e0 msgr2=0x7f622c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:15.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.385+0000 7f622b7fe700 1 --2- 172.21.15.55:0/2294958630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f622c1566e0 0x7f622c0ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:15.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.385+0000 7f622b7fe700 1 -- 172.21.15.55:0/2294958630 >> v1:172.21.15.163:6789/0 conn(0x7f622c0ca300 legacy=0x7f622c0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:15.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.385+0000 7f622b7fe700 1 -- 172.21.15.55:0/2294958630 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c0cb100 con 0x7f622c156310 2023-10-06T05:07:15.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.385+0000 7f622b7fe700 1 --2- 172.21.15.55:0/2294958630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f622c156310 0x7f622c0c9dc0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f621c00bc50 tx=0x7f621c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6f763b77bb0aae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.386+0000 7f622a7fc700 1 -- 172.21.15.55:0/2294958630 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c00dbf0 con 0x7f622c156310 2023-10-06T05:07:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.386+0000 7f622a7fc700 1 -- 172.21.15.55:0/2294958630 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c00dd50 con 0x7f622c156310 2023-10-06T05:07:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.386+0000 7f622a7fc700 1 -- 172.21.15.55:0/2294958630 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c0196b0 con 0x7f622c156310 2023-10-06T05:07:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.386+0000 7f6230b9d700 1 -- 172.21.15.55:0/2294958630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f622c156310 msgr2=0x7f622c0c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.386+0000 7f6230b9d700 1 --2- 172.21.15.55:0/2294958630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f622c156310 0x7f622c0c9dc0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f621c00bc50 tx=0x7f621c00b4f0 comp rx=0 tx=0).stop 2023-10-06T05:07:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.386+0000 7f6230b9d700 1 -- 172.21.15.55:0/2294958630 shutdown_connections 2023-10-06T05:07:15.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.386+0000 7f6230b9d700 1 --2- 172.21.15.55:0/2294958630 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f622c156310 0x7f622c0c9dc0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:15.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.386+0000 7f6230b9d700 1 --2- 172.21.15.55:0/2294958630 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f622c1566e0 0x7f622c0ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:15.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.386+0000 7f6230b9d700 1 -- 172.21.15.55:0/2294958630 >> 172.21.15.55:0/2294958630 conn(0x7f622c0cc1f0 msgr2=0x7f622c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:15.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.386+0000 7f6230b9d700 1 -- 172.21.15.55:0/2294958630 shutdown_connections 2023-10-06T05:07:15.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.386+0000 7f6230b9d700 1 -- 172.21.15.55:0/2294958630 wait complete. 2023-10-06T05:07:15.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.388+0000 7f6230b9d700 1 Processor -- start 2023-10-06T05:07:15.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.388+0000 7f6230b9d700 1 -- start start 2023-10-06T05:07:15.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f6230b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f622c0ca300 0x7f622c14c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:15.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f6230b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f622c156310 0x7f622c0b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:15.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f6230b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f622c1566e0 0x7f622c0b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:15.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f6230b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c149910 con 0x7f622c1566e0 2023-10-06T05:07:15.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f6230b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c148a60 con 0x7f622c156310 2023-10-06T05:07:15.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f6230b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622c148d00 con 0x7f622c0ca300 2023-10-06T05:07:15.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f622bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f622c1566e0 0x7f622c0b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:15.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f622bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f622c1566e0 0x7f622c0b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45514/0 (socket says 172.21.15.55:45514) 2023-10-06T05:07:15.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f622bfff700 1 -- 172.21.15.55:0/1527595785 learned_addr learned my addr 172.21.15.55:0/1527595785 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:15.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f622b7fe700 1 --2- 172.21.15.55:0/1527595785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f622c0ca300 0x7f622c14c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:15.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f622affd700 1 --2- 172.21.15.55:0/1527595785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f622c156310 0x7f622c0b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:15.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f622bfff700 1 -- 172.21.15.55:0/1527595785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f622c0ca300 msgr2=0x7f622c14c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:15.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f622bfff700 1 --2- 172.21.15.55:0/1527595785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f622c0ca300 0x7f622c14c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f622bfff700 1 -- 172.21.15.55:0/1527595785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f622c156310 msgr2=0x7f622c0b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f622bfff700 1 --2- 172.21.15.55:0/1527595785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f622c156310 0x7f622c0b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.389+0000 7f622bfff700 1 -- 172.21.15.55:0/1527595785 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c00b790 con 0x7f622c1566e0 2023-10-06T05:07:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.390+0000 7f622bfff700 1 --2- 172.21.15.55:0/1527595785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f622c1566e0 0x7f622c0b0dc0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f622400c840 tx=0x7f6224001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.390+0000 7f62137fe700 1 -- 172.21.15.55:0/1527595785 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224012630 con 0x7f622c1566e0 2023-10-06T05:07:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.390+0000 7f62137fe700 1 -- 172.21.15.55:0/1527595785 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62240127b0 con 0x7f622c1566e0 2023-10-06T05:07:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.390+0000 7f6230b9d700 1 -- 172.21.15.55:0/1527595785 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c148fe0 con 0x7f622c1566e0 2023-10-06T05:07:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.390+0000 7f62137fe700 1 -- 172.21.15.55:0/1527595785 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224012b70 con 0x7f622c1566e0 2023-10-06T05:07:15.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.390+0000 7f6230b9d700 1 -- 172.21.15.55:0/1527595785 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c1495a0 con 0x7f622c1566e0 2023-10-06T05:07:15.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.391+0000 7f6230b9d700 1 -- 172.21.15.55:0/1527595785 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622c149ac0 con 0x7f622c1566e0 2023-10-06T05:07:15.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.392+0000 7f62137fe700 1 -- 172.21.15.55:0/1527595785 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f622401a3a0 con 0x7f622c1566e0 2023-10-06T05:07:15.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.393+0000 7f62137fe700 1 --2- 172.21.15.55:0/1527595785 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6218066f30 0x7f62180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:15.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.393+0000 7f62137fe700 1 -- 172.21.15.55:0/1527595785 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6342+0+0 (secure 0 0 0) 0x7f6224084a00 con 0x7f622c1566e0 2023-10-06T05:07:15.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.393+0000 7f622b7fe700 1 --2- 172.21.15.55:0/1527595785 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6218066f30 0x7f62180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:15.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.394+0000 7f622b7fe700 1 --2- 172.21.15.55:0/1527595785 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6218066f30 0x7f62180693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f621c00bc50 tx=0x7f621c01b000 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:15.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.397+0000 7f62137fe700 1 -- 172.21.15.55:0/1527595785 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f622c149ac0 con 0x7f622c1566e0 2023-10-06T05:07:15.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:15.679+0000 7f6230b9d700 1 -- 172.21.15.55:0/1527595785 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f622c04c3c0 con 0x7f622c1566e0 2023-10-06T05:07:16.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.181+0000 7f62137fe700 1 -- 172.21.15.55:0/1527595785 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3037) v1 ==== 158+0+0 (secure 0 0 0) 0x7f6224051950 con 0x7f622c1566e0 2023-10-06T05:07:16.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.183+0000 7f6230b9d700 1 -- 172.21.15.55:0/1527595785 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6218066f30 msgr2=0x7f62180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:16.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.183+0000 7f6230b9d700 1 --2- 172.21.15.55:0/1527595785 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6218066f30 0x7f62180693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f621c00bc50 tx=0x7f621c01b000 comp rx=0 tx=0).stop 2023-10-06T05:07:16.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.183+0000 7f6230b9d700 1 -- 172.21.15.55:0/1527595785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f622c1566e0 msgr2=0x7f622c0b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:16.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.183+0000 7f6230b9d700 1 --2- 172.21.15.55:0/1527595785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f622c1566e0 0x7f622c0b0dc0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f622400c840 tx=0x7f6224001670 comp rx=0 tx=0).stop 2023-10-06T05:07:16.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.183+0000 7f6230b9d700 1 -- 172.21.15.55:0/1527595785 shutdown_connections 2023-10-06T05:07:16.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.183+0000 7f6230b9d700 1 --2- 172.21.15.55:0/1527595785 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6218066f30 0x7f62180693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.183+0000 7f6230b9d700 1 --2- 172.21.15.55:0/1527595785 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f622c1566e0 0x7f622c0b0dc0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.183+0000 7f6230b9d700 1 --2- 172.21.15.55:0/1527595785 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f622c0ca300 0x7f622c14c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.183+0000 7f6230b9d700 1 --2- 172.21.15.55:0/1527595785 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f622c156310 0x7f622c0b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.184+0000 7f6230b9d700 1 -- 172.21.15.55:0/1527595785 >> 172.21.15.55:0/1527595785 conn(0x7f622c0cc1f0 msgr2=0x7f622c0c2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:16.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.184+0000 7f6230b9d700 1 -- 172.21.15.55:0/1527595785 shutdown_connections 2023-10-06T05:07:16.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.184+0000 7f6230b9d700 1 -- 172.21.15.55:0/1527595785 wait complete. 2023-10-06T05:07:16.204 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T05:07:16.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.392+0000 7f9c8ef9d700 1 Processor -- start 2023-10-06T05:07:16.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.393+0000 7f9c8ef9d700 1 -- start start 2023-10-06T05:07:16.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.394+0000 7f9c8ef9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c8806bfa0 0x7f9c88068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:16.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.394+0000 7f9c8ef9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9c880688d0 0x7f9c880caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:16.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.394+0000 7f9c8ef9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c880ca920 con 0x7f9c88159a70 2023-10-06T05:07:16.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.394+0000 7f9c8ef9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c88068d50 con 0x7f9c880688d0 2023-10-06T05:07:16.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.394+0000 7f9c8ef9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8806af00 con 0x7f9c8806bfa0 2023-10-06T05:07:16.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.394+0000 7f9c8d79a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c8806bfa0 0x7f9c88068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:16.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.394+0000 7f9c8e79c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9c880688d0 0x7f9c880caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:16.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.394+0000 7f9c8d79a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c8806bfa0 0x7f9c88068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:39598/0 (socket says 172.21.15.55:39598) 2023-10-06T05:07:16.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.394+0000 7f9c8d79a700 1 -- 172.21.15.55:0/2969516232 learned_addr learned my addr 172.21.15.55:0/2969516232 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:16.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.395+0000 7f9c8d79a700 1 -- 172.21.15.55:0/2969516232 >> v2:172.21.15.163:3300/0 conn(0x7f9c880688d0 msgr2=0x7f9c880caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:16.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.395+0000 7f9c8d79a700 1 --2- 172.21.15.55:0/2969516232 >> v2:172.21.15.163:3300/0 conn(0x7f9c880688d0 0x7f9c880caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.395+0000 7f9c8d79a700 1 -- 172.21.15.55:0/2969516232 >> v1:172.21.15.163:6789/0 conn(0x7f9c88159a70 legacy=0x7f9c880c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:16.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.395+0000 7f9c8d79a700 1 -- 172.21.15.55:0/2969516232 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8806b1e0 con 0x7f9c8806bfa0 2023-10-06T05:07:16.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.395+0000 7f9c8cf99700 1 -- 172.21.15.55:0/2969516232 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962497360 0 0) 0x7f9c880ca920 con 0x7f9c88159a70 2023-10-06T05:07:16.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.395+0000 7f9c8d79a700 1 --2- 172.21.15.55:0/2969516232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c8806bfa0 0x7f9c88068390 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009990 tx=0x7f9c78009220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a03e93cc57c6171 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:16.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.396+0000 7f9c8cf99700 1 -- 172.21.15.55:0/2969516232 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78009e80 con 0x7f9c8806bfa0 2023-10-06T05:07:16.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.396+0000 7f9c8cf99700 1 -- 172.21.15.55:0/2969516232 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c78005c10 con 0x7f9c8806bfa0 2023-10-06T05:07:16.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.396+0000 7f9c8cf99700 1 -- 172.21.15.55:0/2969516232 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800b650 con 0x7f9c8806bfa0 2023-10-06T05:07:16.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.396+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/2969516232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c8806bfa0 msgr2=0x7f9c88068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:16.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.396+0000 7f9c8ef9d700 1 --2- 172.21.15.55:0/2969516232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c8806bfa0 0x7f9c88068390 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009990 tx=0x7f9c78009220 comp rx=0 tx=0).stop 2023-10-06T05:07:16.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.396+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/2969516232 shutdown_connections 2023-10-06T05:07:16.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.396+0000 7f9c8ef9d700 1 --2- 172.21.15.55:0/2969516232 >> v2:172.21.15.163:3300/0 conn(0x7f9c880688d0 0x7f9c880caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.396+0000 7f9c8ef9d700 1 --2- 172.21.15.55:0/2969516232 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c8806bfa0 0x7f9c88068390 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.396+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/2969516232 >> 172.21.15.55:0/2969516232 conn(0x7f9c880cddc0 msgr2=0x7f9c880c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:16.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.396+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/2969516232 shutdown_connections 2023-10-06T05:07:16.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.396+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/2969516232 wait complete. 2023-10-06T05:07:16.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.398+0000 7f9c8ef9d700 1 Processor -- start 2023-10-06T05:07:16.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.398+0000 7f9c8ef9d700 1 -- start start 2023-10-06T05:07:16.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.399+0000 7f9c8ef9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c880688d0 0x7f9c88147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:16.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.399+0000 7f9c8ef9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8806bfa0 0x7f9c880b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:16.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.399+0000 7f9c8ef9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c88159a70 0x7f9c880cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:16.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.399+0000 7f9c8ef9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880c68f0 con 0x7f9c8806bfa0 2023-10-06T05:07:16.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.399+0000 7f9c8ef9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880c5ce0 con 0x7f9c88159a70 2023-10-06T05:07:16.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.399+0000 7f9c8ef9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c880c5fc0 con 0x7f9c880688d0 2023-10-06T05:07:16.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.399+0000 7f9c8d79a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8806bfa0 0x7f9c880b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:16.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.399+0000 7f9c8d79a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8806bfa0 0x7f9c880b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45520/0 (socket says 172.21.15.55:45520) 2023-10-06T05:07:16.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.399+0000 7f9c8d79a700 1 -- 172.21.15.55:0/1953482577 learned_addr learned my addr 172.21.15.55:0/1953482577 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:16.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.399+0000 7f9c8df9b700 1 --2- 172.21.15.55:0/1953482577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c880688d0 0x7f9c88147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:16.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.399+0000 7f9c8e79c700 1 --2- 172.21.15.55:0/1953482577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c88159a70 0x7f9c880cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:16.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.400+0000 7f9c8d79a700 1 -- 172.21.15.55:0/1953482577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c880688d0 msgr2=0x7f9c88147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:16.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.400+0000 7f9c8d79a700 1 --2- 172.21.15.55:0/1953482577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c880688d0 0x7f9c88147100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.400+0000 7f9c8d79a700 1 -- 172.21.15.55:0/1953482577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c88159a70 msgr2=0x7f9c880cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:16.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.400+0000 7f9c8d79a700 1 --2- 172.21.15.55:0/1953482577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c88159a70 0x7f9c880cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.400+0000 7f9c8d79a700 1 -- 172.21.15.55:0/1953482577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c78009480 con 0x7f9c8806bfa0 2023-10-06T05:07:16.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.400+0000 7f9c8d79a700 1 --2- 172.21.15.55:0/1953482577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8806bfa0 0x7f9c880b1580 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f9c8806b040 tx=0x7f9c78016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:16.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.400+0000 7f9c7dffb700 1 -- 172.21.15.55:0/1953482577 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7801dc30 con 0x7f9c8806bfa0 2023-10-06T05:07:16.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.400+0000 7f9c7dffb700 1 -- 172.21.15.55:0/1953482577 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7801dd90 con 0x7f9c8806bfa0 2023-10-06T05:07:16.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.400+0000 7f9c7dffb700 1 -- 172.21.15.55:0/1953482577 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7801a670 con 0x7f9c8806bfa0 2023-10-06T05:07:16.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.400+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/1953482577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c880c62a0 con 0x7f9c8806bfa0 2023-10-06T05:07:16.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.400+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/1953482577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8816ae10 con 0x7f9c8806bfa0 2023-10-06T05:07:16.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.401+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/1953482577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c88063d10 con 0x7f9c8806bfa0 2023-10-06T05:07:16.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.402+0000 7f9c7dffb700 1 -- 172.21.15.55:0/1953482577 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9c78011420 con 0x7f9c8806bfa0 2023-10-06T05:07:16.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.403+0000 7f9c7dffb700 1 --2- 172.21.15.55:0/1953482577 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9c6c066f30 0x7f9c6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:16.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.403+0000 7f9c7dffb700 1 -- 172.21.15.55:0/1953482577 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6342+0+0 (secure 0 0 0) 0x7f9c780859b0 con 0x7f9c8806bfa0 2023-10-06T05:07:16.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.404+0000 7f9c8df9b700 1 --2- 172.21.15.55:0/1953482577 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9c6c066f30 0x7f9c6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:16.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.405+0000 7f9c8df9b700 1 --2- 172.21.15.55:0/1953482577 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9c6c066f30 0x7f9c6c0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9c84007fd0 tx=0x7f9c84007d30 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:16.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.408+0000 7f9c7dffb700 1 -- 172.21.15.55:0/1953482577 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c780529b0 con 0x7f9c8806bfa0 2023-10-06T05:07:16.613 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.611+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/1953482577 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9c8806b4c0 con 0x7f9c8806bfa0 2023-10-06T05:07:16.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.616+0000 7f9c7dffb700 1 -- 172.21.15.55:0/1953482577 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v70)=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9c78055fd0 con 0x7f9c8806bfa0 2023-10-06T05:07:16.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.619+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/1953482577 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9c6c066f30 msgr2=0x7f9c6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:16.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.619+0000 7f9c8ef9d700 1 --2- 172.21.15.55:0/1953482577 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9c6c066f30 0x7f9c6c0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9c84007fd0 tx=0x7f9c84007d30 comp rx=0 tx=0).stop 2023-10-06T05:07:16.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.619+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/1953482577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8806bfa0 msgr2=0x7f9c880b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:16.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.619+0000 7f9c8ef9d700 1 --2- 172.21.15.55:0/1953482577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8806bfa0 0x7f9c880b1580 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f9c8806b040 tx=0x7f9c78016000 comp rx=0 tx=0).stop 2023-10-06T05:07:16.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.619+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/1953482577 shutdown_connections 2023-10-06T05:07:16.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.619+0000 7f9c8ef9d700 1 --2- 172.21.15.55:0/1953482577 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9c6c066f30 0x7f9c6c0693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.619+0000 7f9c8ef9d700 1 --2- 172.21.15.55:0/1953482577 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9c88159a70 0x7f9c880cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.619+0000 7f9c8ef9d700 1 --2- 172.21.15.55:0/1953482577 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9c880688d0 0x7f9c88147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.619+0000 7f9c8ef9d700 1 --2- 172.21.15.55:0/1953482577 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9c8806bfa0 0x7f9c880b1580 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.619+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/1953482577 >> 172.21.15.55:0/1953482577 conn(0x7f9c880cddc0 msgr2=0x7f9c88153280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:16.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.619+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/1953482577 shutdown_connections 2023-10-06T05:07:16.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.619+0000 7f9c8ef9d700 1 -- 172.21.15.55:0/1953482577 wait complete. 2023-10-06T05:07:16.638 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:07:16.639 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:07:16.719 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:07:16.804 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:07:16.804 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T05:07:16.804 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:07:16.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.990+0000 7f7a6df9d700 1 Processor -- start 2023-10-06T05:07:16.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.990+0000 7f7a6df9d700 1 -- start start 2023-10-06T05:07:16.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.992+0000 7f7a6df9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7a681595a0 0x7f7a680b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:16.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.992+0000 7f7a6df9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f7a680c8830 0x7f7a68159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:16.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.992+0000 7f7a6df9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a680b1410 con 0x7f7a680c6c60 2023-10-06T05:07:16.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.992+0000 7f7a6df9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a680b2b90 con 0x7f7a681595a0 2023-10-06T05:07:16.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.992+0000 7f7a6df9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a68063230 con 0x7f7a680c8830 2023-10-06T05:07:16.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.992+0000 7f7a6cf9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7a681595a0 0x7f7a680b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:16.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.992+0000 7f7a6cf9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7a681595a0 0x7f7a680b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45524/0 (socket says 172.21.15.55:45524) 2023-10-06T05:07:16.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.992+0000 7f7a6cf9b700 1 -- 172.21.15.55:0/2102840258 learned_addr learned my addr 172.21.15.55:0/2102840258 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:16.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.992+0000 7f7a6d79c700 1 --2- 172.21.15.55:0/2102840258 >> v2:172.21.15.163:3300/0 conn(0x7f7a680c8830 0x7f7a68159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:16.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.992+0000 7f7a6cf9b700 1 -- 172.21.15.55:0/2102840258 >> v2:172.21.15.163:3300/0 conn(0x7f7a680c8830 msgr2=0x7f7a68159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:16.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.992+0000 7f7a6cf9b700 1 --2- 172.21.15.55:0/2102840258 >> v2:172.21.15.163:3300/0 conn(0x7f7a680c8830 0x7f7a68159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.992+0000 7f7a6cf9b700 1 -- 172.21.15.55:0/2102840258 >> v1:172.21.15.163:6789/0 conn(0x7f7a680c6c60 legacy=0x7f7a680c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:16.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.992+0000 7f7a6cf9b700 1 -- 172.21.15.55:0/2102840258 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680634f0 con 0x7f7a681595a0 2023-10-06T05:07:16.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.993+0000 7f7a6cf9b700 1 --2- 172.21.15.55:0/2102840258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a681595a0 0x7f7a680b18a0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f7a6815d390 tx=0x7f7a5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adaf90c60bd7ea95 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:16.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.993+0000 7f7a677fe700 1 -- 172.21.15.55:0/2102840258 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c00b820 con 0x7f7a681595a0 2023-10-06T05:07:16.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.993+0000 7f7a677fe700 1 -- 172.21.15.55:0/2102840258 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c00b980 con 0x7f7a681595a0 2023-10-06T05:07:16.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.993+0000 7f7a677fe700 1 -- 172.21.15.55:0/2102840258 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c00bd20 con 0x7f7a681595a0 2023-10-06T05:07:16.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.993+0000 7f7a6df9d700 1 -- 172.21.15.55:0/2102840258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a681595a0 msgr2=0x7f7a680b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:16.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.993+0000 7f7a6df9d700 1 --2- 172.21.15.55:0/2102840258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a681595a0 0x7f7a680b18a0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f7a6815d390 tx=0x7f7a5c009250 comp rx=0 tx=0).stop 2023-10-06T05:07:16.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.994+0000 7f7a6df9d700 1 -- 172.21.15.55:0/2102840258 shutdown_connections 2023-10-06T05:07:16.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.994+0000 7f7a6df9d700 1 --2- 172.21.15.55:0/2102840258 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a681595a0 0x7f7a680b18a0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.994+0000 7f7a6df9d700 1 --2- 172.21.15.55:0/2102840258 >> v2:172.21.15.163:3300/0 conn(0x7f7a680c8830 0x7f7a68159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:16.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.994+0000 7f7a6df9d700 1 -- 172.21.15.55:0/2102840258 >> 172.21.15.55:0/2102840258 conn(0x7f7a6806ddb0 msgr2=0x7f7a6806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:16.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.994+0000 7f7a6df9d700 1 -- 172.21.15.55:0/2102840258 shutdown_connections 2023-10-06T05:07:16.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.994+0000 7f7a6df9d700 1 -- 172.21.15.55:0/2102840258 wait complete. 2023-10-06T05:07:16.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.995+0000 7f7a6df9d700 1 Processor -- start 2023-10-06T05:07:16.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.995+0000 7f7a6df9d700 1 -- start start 2023-10-06T05:07:17.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.996+0000 7f7a6df9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a680c6c60 0x7f7a68134340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:17.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.996+0000 7f7a6df9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a680c8830 0x7f7a6815eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:17.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.996+0000 7f7a6df9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a681595a0 0x7f7a6815f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:17.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.996+0000 7f7a6df9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6815a8d0 con 0x7f7a680c8830 2023-10-06T05:07:17.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.996+0000 7f7a6df9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6815a440 con 0x7f7a681595a0 2023-10-06T05:07:17.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.996+0000 7f7a6df9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68159b10 con 0x7f7a680c6c60 2023-10-06T05:07:17.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.996+0000 7f7a67fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a680c8830 0x7f7a6815eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:17.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.996+0000 7f7a67fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a680c8830 0x7f7a6815eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45538/0 (socket says 172.21.15.55:45538) 2023-10-06T05:07:17.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.996+0000 7f7a67fff700 1 -- 172.21.15.55:0/246367004 learned_addr learned my addr 172.21.15.55:0/246367004 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:17.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.996+0000 7f7a6d79c700 1 --2- 172.21.15.55:0/246367004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a681595a0 0x7f7a6815f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:17.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.996+0000 7f7a6cf9b700 1 --2- 172.21.15.55:0/246367004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a680c6c60 0x7f7a68134340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:17.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.997+0000 7f7a67fff700 1 -- 172.21.15.55:0/246367004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a680c6c60 msgr2=0x7f7a68134340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:17.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.997+0000 7f7a67fff700 1 --2- 172.21.15.55:0/246367004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a680c6c60 0x7f7a68134340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:17.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.997+0000 7f7a67fff700 1 -- 172.21.15.55:0/246367004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a681595a0 msgr2=0x7f7a6815f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:17.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.997+0000 7f7a67fff700 1 --2- 172.21.15.55:0/246367004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a681595a0 0x7f7a6815f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:17.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.997+0000 7f7a67fff700 1 -- 172.21.15.55:0/246367004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c0094f0 con 0x7f7a680c8830 2023-10-06T05:07:17.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.997+0000 7f7a67fff700 1 --2- 172.21.15.55:0/246367004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a680c8830 0x7f7a6815eac0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400c590 tx=0x7f7a54007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:17.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.997+0000 7f7a64ff9700 1 -- 172.21.15.55:0/246367004 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54011590 con 0x7f7a680c8830 2023-10-06T05:07:17.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.997+0000 7f7a64ff9700 1 -- 172.21.15.55:0/246367004 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a540116f0 con 0x7f7a680c8830 2023-10-06T05:07:17.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.997+0000 7f7a6df9d700 1 -- 172.21.15.55:0/246367004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a68159df0 con 0x7f7a680c8830 2023-10-06T05:07:17.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.997+0000 7f7a64ff9700 1 -- 172.21.15.55:0/246367004 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54011ab0 con 0x7f7a680c8830 2023-10-06T05:07:17.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.998+0000 7f7a6df9d700 1 -- 172.21.15.55:0/246367004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a68134880 con 0x7f7a680c8830 2023-10-06T05:07:17.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.998+0000 7f7a6df9d700 1 -- 172.21.15.55:0/246367004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6804c3c0 con 0x7f7a680c8830 2023-10-06T05:07:17.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:16.999+0000 7f7a64ff9700 1 -- 172.21.15.55:0/246367004 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a5401c460 con 0x7f7a680c8830 2023-10-06T05:07:17.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.000+0000 7f7a64ff9700 1 --2- 172.21.15.55:0/246367004 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7a4c066f30 0x7f7a4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:17.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.000+0000 7f7a64ff9700 1 -- 172.21.15.55:0/246367004 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6342+0+0 (secure 0 0 0) 0x7f7a54084480 con 0x7f7a680c8830 2023-10-06T05:07:17.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.001+0000 7f7a6cf9b700 1 --2- 172.21.15.55:0/246367004 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7a4c066f30 0x7f7a4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:17.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.001+0000 7f7a6cf9b700 1 --2- 172.21.15.55:0/246367004 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7a4c066f30 0x7f7a4c0693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7a6815d390 tx=0x7f7a5c016000 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:17.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.004+0000 7f7a64ff9700 1 -- 172.21.15.55:0/246367004 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a540513d0 con 0x7f7a680c8830 2023-10-06T05:07:17.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.254+0000 7f7a6df9d700 1 -- 172.21.15.55:0/246367004 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a680637d0 con 0x7f7a680c8830 2023-10-06T05:07:17.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.254+0000 7f7a64ff9700 1 -- 172.21.15.55:0/246367004 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f7a540549f0 con 0x7f7a680c8830 2023-10-06T05:07:17.257 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:17.258 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":98,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8078,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/1255308005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":1255308005},{"type":"v1","addr":"172.21.15.163:6835","nonce":1255308005}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":9274,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/2468487159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":2468487159},{"type":"v1","addr":"172.21.15.163:6834","nonce":2468487159}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91},{"gid":9646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/457472947","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":457472947},{"type":"v1","addr":"172.21.15.55:6835","nonce":457472947}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":98},{"gid":9762,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/4105265850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":4105265850},{"type":"v1","addr":"172.21.15.55:6834","nonce":4105265850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[]} 2023-10-06T05:07:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.256+0000 7f7a6df9d700 1 -- 172.21.15.55:0/246367004 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7a4c066f30 msgr2=0x7f7a4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.257+0000 7f7a6df9d700 1 --2- 172.21.15.55:0/246367004 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7a4c066f30 0x7f7a4c0693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7a6815d390 tx=0x7f7a5c016000 comp rx=0 tx=0).stop 2023-10-06T05:07:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.257+0000 7f7a6df9d700 1 -- 172.21.15.55:0/246367004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a680c8830 msgr2=0x7f7a6815eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:17.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.257+0000 7f7a6df9d700 1 --2- 172.21.15.55:0/246367004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a680c8830 0x7f7a6815eac0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f7a5400c590 tx=0x7f7a54007f30 comp rx=0 tx=0).stop 2023-10-06T05:07:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.257+0000 7f7a6df9d700 1 -- 172.21.15.55:0/246367004 shutdown_connections 2023-10-06T05:07:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.257+0000 7f7a6df9d700 1 --2- 172.21.15.55:0/246367004 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7a4c066f30 0x7f7a4c0693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.257+0000 7f7a6df9d700 1 --2- 172.21.15.55:0/246367004 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7a681595a0 0x7f7a6815f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.257+0000 7f7a6df9d700 1 --2- 172.21.15.55:0/246367004 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7a680c6c60 0x7f7a68134340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.257+0000 7f7a6df9d700 1 --2- 172.21.15.55:0/246367004 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7a680c8830 0x7f7a6815eac0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.257+0000 7f7a6df9d700 1 -- 172.21.15.55:0/246367004 >> 172.21.15.55:0/246367004 conn(0x7f7a6806ddb0 msgr2=0x7f7a681577c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.257+0000 7f7a6df9d700 1 -- 172.21.15.55:0/246367004 shutdown_connections 2023-10-06T05:07:17.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.257+0000 7f7a6df9d700 1 -- 172.21.15.55:0/246367004 wait complete. 2023-10-06T05:07:17.262 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 98 2023-10-06T05:07:17.278 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T05:07:17.278 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T05:07:17.278 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:07:17.279 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:07:17.279 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T05:07:17.279 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T05:07:17.280 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:07:17.280 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:07:17.280 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T05:07:17.280 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T05:07:17.280 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:07:17.280 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:07:17.281 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T05:07:17.281 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T05:07:17.281 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:07:17.281 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:07:17.293 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:07:17.293 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T05:07:17.295 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:07:17.295 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T05:07:17.298 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:07:17.299 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T05:07:17.300 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:07:17.300 DEBUG:teuthology.orchestra.run.smithi163:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T05:07:17.339 INFO:tasks.ceph.mds.a:Started 2023-10-06T05:07:17.340 DEBUG:teuthology.parallel:result is None 2023-10-06T05:07:17.343 INFO:tasks.ceph.mds.c:Started 2023-10-06T05:07:17.343 DEBUG:teuthology.parallel:result is None 2023-10-06T05:07:17.344 INFO:tasks.ceph.mds.d:Started 2023-10-06T05:07:17.344 DEBUG:teuthology.parallel:result is None 2023-10-06T05:07:17.347 INFO:tasks.ceph.mds.b:Started 2023-10-06T05:07:17.347 DEBUG:teuthology.parallel:result is None 2023-10-06T05:07:17.347 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T05:07:17.480 INFO:tasks.ceph.mds.c.smithi055.stdout:starting mds.c at 2023-10-06T05:07:17.481 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T05:07:17.482 INFO:tasks.ceph.mds.a.smithi055.stderr:2023-10-06T05:07:17.479+0000 7f37065fd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-06T05:07:17.484 INFO:tasks.ceph.mds.b.smithi163.stdout:starting mds.b at 2023-10-06T05:07:17.485 INFO:tasks.ceph.mds.d.smithi163.stdout:starting mds.d at 2023-10-06T05:07:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.546+0000 7f11cffff700 1 Processor -- start 2023-10-06T05:07:17.549 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.546+0000 7f11cffff700 1 -- start start 2023-10-06T05:07:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.547+0000 7f11cffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f11d015a330 0x7f11d00c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.547+0000 7f11cffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d00c7e50 con 0x7f11d01566b0 2023-10-06T05:07:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.547+0000 7f11cffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d00c7240 con 0x7f11d00c6d00 2023-10-06T05:07:17.550 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.547+0000 7f11cffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d00c7520 con 0x7f11d015a330 2023-10-06T05:07:17.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.548+0000 7f11cf7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f11d01566b0 0x7f11d00ac960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36584/0 (socket says 172.21.15.55:36584) 2023-10-06T05:07:17.551 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.548+0000 7f11cf7fe700 1 -- 172.21.15.55:0/442438168 learned_addr learned my addr 172.21.15.55:0/442438168 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:17.552 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.548+0000 7f11ceffd700 1 --2- 172.21.15.55:0/442438168 >> v2:172.21.15.163:3300/0 conn(0x7f11d015a330 0x7f11d00c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.548+0000 7f11cdffb700 1 -- 172.21.15.55:0/442438168 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097571322 0 0) 0x7f11d00c7e50 con 0x7f11d01566b0 2023-10-06T05:07:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.548+0000 7f11cdffb700 1 -- 172.21.15.55:0/442438168 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b8003570 con 0x7f11d01566b0 2023-10-06T05:07:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.548+0000 7f11cdffb700 1 -- 172.21.15.55:0/442438168 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356966159 0 0) 0x7f11d00c7240 con 0x7f11d00c6d00 2023-10-06T05:07:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.548+0000 7f11cdffb700 1 -- 172.21.15.55:0/442438168 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11d00c7e50 con 0x7f11d00c6d00 2023-10-06T05:07:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.548+0000 7f11ceffd700 1 -- 172.21.15.55:0/442438168 >> v1:172.21.15.163:6789/0 conn(0x7f11d00c6d00 legacy=0x7f11d0155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.548+0000 7f11ceffd700 1 -- 172.21.15.55:0/442438168 >> v1:172.21.15.55:6789/0 conn(0x7f11d01566b0 legacy=0x7f11d00ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.548+0000 7f11cdffb700 1 -- 172.21.15.55:0/442438168 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1137294355 0 0) 0x7f11b8003570 con 0x7f11d01566b0 2023-10-06T05:07:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.548+0000 7f11ceffd700 1 -- 172.21.15.55:0/442438168 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d00c7800 con 0x7f11d015a330 2023-10-06T05:07:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.549+0000 7f11ceffd700 1 --2- 172.21.15.55:0/442438168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11d015a330 0x7f11d00c67c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f11c00098e0 tx=0x7f11c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f6ced5a72f4ee8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.549+0000 7f11cdffb700 1 -- 172.21.15.55:0/442438168 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c80017f0 con 0x7f11d015a330 2023-10-06T05:07:17.553 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.549+0000 7f11cdffb700 1 -- 172.21.15.55:0/442438168 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c0011420 con 0x7f11d015a330 2023-10-06T05:07:17.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.549+0000 7f11cdffb700 1 -- 172.21.15.55:0/442438168 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c00117c0 con 0x7f11d015a330 2023-10-06T05:07:17.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.550+0000 7f11cffff700 1 -- 172.21.15.55:0/442438168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11d015a330 msgr2=0x7f11d00c67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:17.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.550+0000 7f11cffff700 1 --2- 172.21.15.55:0/442438168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11d015a330 0x7f11d00c67c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f11c00098e0 tx=0x7f11c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:17.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.550+0000 7f11cffff700 1 -- 172.21.15.55:0/442438168 shutdown_connections 2023-10-06T05:07:17.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.550+0000 7f11cffff700 1 --2- 172.21.15.55:0/442438168 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11d015a330 0x7f11d00c67c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:17.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.550+0000 7f11cffff700 1 -- 172.21.15.55:0/442438168 >> 172.21.15.55:0/442438168 conn(0x7f11d00c8bf0 msgr2=0x7f11d00c8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:17.554 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.550+0000 7f11cffff700 1 -- 172.21.15.55:0/442438168 shutdown_connections 2023-10-06T05:07:17.555 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.550+0000 7f11cffff700 1 -- 172.21.15.55:0/442438168 wait complete. 2023-10-06T05:07:17.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.551+0000 7f11cffff700 1 Processor -- start 2023-10-06T05:07:17.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.551+0000 7f11cffff700 1 -- start start 2023-10-06T05:07:17.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.552+0000 7f11cffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11d00c6d00 0x7f11d01590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:17.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.552+0000 7f11cffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11d01566b0 0x7f11d00c19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:17.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.552+0000 7f11cffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d015a330 0x7f11d00b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:17.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.552+0000 7f11cffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d00c2090 con 0x7f11d015a330 2023-10-06T05:07:17.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.552+0000 7f11cffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d0158c70 con 0x7f11d01566b0 2023-10-06T05:07:17.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.552+0000 7f11cffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d01599e0 con 0x7f11d00c6d00 2023-10-06T05:07:17.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.552+0000 7f11cf7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d015a330 0x7f11d00b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:17.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.553+0000 7f11cf7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d015a330 0x7f11d00b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45570/0 (socket says 172.21.15.55:45570) 2023-10-06T05:07:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.553+0000 7f11cf7fe700 1 -- 172.21.15.55:0/1562111654 learned_addr learned my addr 172.21.15.55:0/1562111654 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.553+0000 7f11ceffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11d00c6d00 0x7f11d01590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.553+0000 7f11ce7fc700 1 --2- 172.21.15.55:0/1562111654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11d01566b0 0x7f11d00c19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.553+0000 7f11cf7fe700 1 -- 172.21.15.55:0/1562111654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11d00c6d00 msgr2=0x7f11d01590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.553+0000 7f11cf7fe700 1 --2- 172.21.15.55:0/1562111654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11d00c6d00 0x7f11d01590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.553+0000 7f11cf7fe700 1 -- 172.21.15.55:0/1562111654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11d01566b0 msgr2=0x7f11d00c19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.553+0000 7f11cf7fe700 1 --2- 172.21.15.55:0/1562111654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11d01566b0 0x7f11d00c19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.553+0000 7f11cf7fe700 1 -- 172.21.15.55:0/1562111654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c0009420 con 0x7f11d015a330 2023-10-06T05:07:17.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.553+0000 7f11cf7fe700 1 --2- 172.21.15.55:0/1562111654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d015a330 0x7f11d00b14c0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f11c800cc80 tx=0x7f11c8007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:17.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.553+0000 7f11b6ffd700 1 -- 172.21.15.55:0/1562111654 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8004dc0 con 0x7f11d015a330 2023-10-06T05:07:17.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.553+0000 7f11b6ffd700 1 -- 172.21.15.55:0/1562111654 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c800a660 con 0x7f11d015a330 2023-10-06T05:07:17.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.553+0000 7f11b6ffd700 1 -- 172.21.15.55:0/1562111654 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c800aa20 con 0x7f11d015a330 2023-10-06T05:07:17.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.553+0000 7f11cffff700 1 -- 172.21.15.55:0/1562111654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d0157e90 con 0x7f11d015a330 2023-10-06T05:07:17.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.554+0000 7f11cffff700 1 -- 172.21.15.55:0/1562111654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d0157bb0 con 0x7f11d015a330 2023-10-06T05:07:17.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.555+0000 7f11cffff700 1 -- 172.21.15.55:0/1562111654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d0158de0 con 0x7f11d015a330 2023-10-06T05:07:17.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.555+0000 7f11b6ffd700 1 -- 172.21.15.55:0/1562111654 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f11c8007150 con 0x7f11d015a330 2023-10-06T05:07:17.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.557+0000 7f11b6ffd700 1 --2- 172.21.15.55:0/1562111654 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f11bc066f60 0x7f11bc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:17.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.557+0000 7f11b6ffd700 1 -- 172.21.15.55:0/1562111654 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6342+0+0 (secure 0 0 0) 0x7f11c8083dc0 con 0x7f11d015a330 2023-10-06T05:07:17.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.558+0000 7f11ceffd700 1 --2- 172.21.15.55:0/1562111654 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f11bc066f60 0x7f11bc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:17.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.558+0000 7f11ceffd700 1 --2- 172.21.15.55:0/1562111654 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f11bc066f60 0x7f11bc069410 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f11c00093f0 tx=0x7f11c0016000 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:17.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.561+0000 7f11b6ffd700 1 -- 172.21.15.55:0/1562111654 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f11d0158de0 con 0x7f11d015a330 2023-10-06T05:07:17.782 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:17.780+0000 7f11cffff700 1 -- 172.21.15.55:0/1562111654 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f11d0158de0 con 0x7f11d015a330 2023-10-06T05:07:18.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.194+0000 7f11b6ffd700 1 -- 172.21.15.55:0/1562111654 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v201) v1 ==== 97+0+0 (secure 0 0 0) 0x7f11d0158de0 con 0x7f11d015a330 2023-10-06T05:07:18.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.197+0000 7f11cffff700 1 -- 172.21.15.55:0/1562111654 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f11bc066f60 msgr2=0x7f11bc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:18.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.197+0000 7f11cffff700 1 --2- 172.21.15.55:0/1562111654 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f11bc066f60 0x7f11bc069410 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f11c00093f0 tx=0x7f11c0016000 comp rx=0 tx=0).stop 2023-10-06T05:07:18.201 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.197+0000 7f11cffff700 1 -- 172.21.15.55:0/1562111654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d015a330 msgr2=0x7f11d00b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:18.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.197+0000 7f11cffff700 1 --2- 172.21.15.55:0/1562111654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d015a330 0x7f11d00b14c0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f11c800cc80 tx=0x7f11c8007d20 comp rx=0 tx=0).stop 2023-10-06T05:07:18.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.197+0000 7f11cffff700 1 -- 172.21.15.55:0/1562111654 shutdown_connections 2023-10-06T05:07:18.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.197+0000 7f11cffff700 1 --2- 172.21.15.55:0/1562111654 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f11bc066f60 0x7f11bc069410 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.197+0000 7f11cffff700 1 --2- 172.21.15.55:0/1562111654 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f11d015a330 0x7f11d00b14c0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.197+0000 7f11cffff700 1 --2- 172.21.15.55:0/1562111654 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f11d00c6d00 0x7f11d01590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.197+0000 7f11cffff700 1 --2- 172.21.15.55:0/1562111654 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f11d01566b0 0x7f11d00c19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.203 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.197+0000 7f11cffff700 1 -- 172.21.15.55:0/1562111654 >> 172.21.15.55:0/1562111654 conn(0x7f11d00c8bf0 msgr2=0x7f11d01530e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:18.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.197+0000 7f11cffff700 1 -- 172.21.15.55:0/1562111654 shutdown_connections 2023-10-06T05:07:18.204 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.197+0000 7f11cffff700 1 -- 172.21.15.55:0/1562111654 wait complete. 2023-10-06T05:07:18.205 INFO:teuthology.orchestra.run.smithi055.stderr: removed all blocklist entries 2023-10-06T05:07:18.220 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T05:07:18.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.417+0000 7f6db359e700 1 Processor -- start 2023-10-06T05:07:18.427 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.417+0000 7f6db359e700 1 -- start start 2023-10-06T05:07:18.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.418+0000 7f6db359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6db40ae710 0x7f6db40c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:18.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.418+0000 7f6db359e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db40ca860 con 0x7f6db4156310 2023-10-06T05:07:18.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.418+0000 7f6db359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db40cab40 con 0x7f6db41566e0 2023-10-06T05:07:18.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.418+0000 7f6db359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db40cae20 con 0x7f6db40ae710 2023-10-06T05:07:18.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.419+0000 7f6db259c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6db4156310 0x7f6db40c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36592/0 (socket says 172.21.15.55:36592) 2023-10-06T05:07:18.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.419+0000 7f6db259c700 1 -- 172.21.15.55:0/4218608472 learned_addr learned my addr 172.21.15.55:0/4218608472 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:18.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.419+0000 7f6db2d9d700 1 --2- 172.21.15.55:0/4218608472 >> v2:172.21.15.163:3300/0 conn(0x7f6db40ae710 0x7f6db40c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:18.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.419+0000 7f6db159a700 1 -- 172.21.15.55:0/4218608472 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861151208 0 0) 0x7f6db40ca860 con 0x7f6db4156310 2023-10-06T05:07:18.434 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.419+0000 7f6db159a700 1 -- 172.21.15.55:0/4218608472 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d9c003570 con 0x7f6db4156310 2023-10-06T05:07:18.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.419+0000 7f6db2d9d700 1 -- 172.21.15.55:0/4218608472 >> v1:172.21.15.163:6789/0 conn(0x7f6db41566e0 legacy=0x7f6db40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:18.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.419+0000 7f6db2d9d700 1 -- 172.21.15.55:0/4218608472 >> v1:172.21.15.55:6789/0 conn(0x7f6db4156310 legacy=0x7f6db40c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:18.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.419+0000 7f6db159a700 1 -- 172.21.15.55:0/4218608472 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 360716571 0 0) 0x7f6d9c003570 con 0x7f6db4156310 2023-10-06T05:07:18.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.419+0000 7f6db2d9d700 1 -- 172.21.15.55:0/4218608472 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db40cb100 con 0x7f6db40ae710 2023-10-06T05:07:18.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.420+0000 7f6db2d9d700 1 --2- 172.21.15.55:0/4218608472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db40ae710 0x7f6db40c6830 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009e60 tx=0x7f6dac009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49430c917cd1684e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:18.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.420+0000 7f6db159a700 1 -- 172.21.15.55:0/4218608472 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00e070 con 0x7f6db40ae710 2023-10-06T05:07:18.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.420+0000 7f6db159a700 1 -- 172.21.15.55:0/4218608472 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac00b850 con 0x7f6db40ae710 2023-10-06T05:07:18.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.420+0000 7f6db159a700 1 -- 172.21.15.55:0/4218608472 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4001470 con 0x7f6db40ae710 2023-10-06T05:07:18.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.421+0000 7f6db359e700 1 -- 172.21.15.55:0/4218608472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db40ae710 msgr2=0x7f6db40c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:18.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.421+0000 7f6db359e700 1 --2- 172.21.15.55:0/4218608472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db40ae710 0x7f6db40c6830 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6dac009e60 tx=0x7f6dac009780 comp rx=0 tx=0).stop 2023-10-06T05:07:18.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.421+0000 7f6db359e700 1 -- 172.21.15.55:0/4218608472 shutdown_connections 2023-10-06T05:07:18.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.421+0000 7f6db359e700 1 --2- 172.21.15.55:0/4218608472 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db40ae710 0x7f6db40c6830 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.421+0000 7f6db359e700 1 -- 172.21.15.55:0/4218608472 >> 172.21.15.55:0/4218608472 conn(0x7f6db40cc1f0 msgr2=0x7f6db40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:18.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.421+0000 7f6db359e700 1 -- 172.21.15.55:0/4218608472 shutdown_connections 2023-10-06T05:07:18.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.421+0000 7f6db359e700 1 -- 172.21.15.55:0/4218608472 wait complete. 2023-10-06T05:07:18.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.422+0000 7f6db359e700 1 Processor -- start 2023-10-06T05:07:18.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.422+0000 7f6db359e700 1 -- start start 2023-10-06T05:07:18.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.423+0000 7f6db359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db40ae710 0x7f6db40b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:18.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.423+0000 7f6db359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db4156310 0x7f6db414ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:18.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.423+0000 7f6db359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db41566e0 0x7f6db4069fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:18.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.423+0000 7f6db359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40b0e50 con 0x7f6db40ae710 2023-10-06T05:07:18.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.423+0000 7f6db359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db415b270 con 0x7f6db4156310 2023-10-06T05:07:18.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.423+0000 7f6db359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40b1e20 con 0x7f6db41566e0 2023-10-06T05:07:18.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.423+0000 7f6db259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db40ae710 0x7f6db40b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:18.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.423+0000 7f6db259c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db40ae710 0x7f6db40b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45582/0 (socket says 172.21.15.55:45582) 2023-10-06T05:07:18.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.423+0000 7f6db259c700 1 -- 172.21.15.55:0/3849879571 learned_addr learned my addr 172.21.15.55:0/3849879571 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:18.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.424+0000 7f6db1d9b700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db4156310 0x7f6db414ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:18.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.424+0000 7f6db2d9d700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db41566e0 0x7f6db4069fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:18.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.424+0000 7f6db259c700 1 -- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db41566e0 msgr2=0x7f6db4069fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:18.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.424+0000 7f6db259c700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db41566e0 0x7f6db4069fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.424+0000 7f6db259c700 1 -- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db4156310 msgr2=0x7f6db414ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.424+0000 7f6db259c700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db4156310 0x7f6db414ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.424+0000 7f6db259c700 1 -- 172.21.15.55:0/3849879571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac009a20 con 0x7f6db40ae710 2023-10-06T05:07:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.424+0000 7f6db2d9d700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db41566e0 0x7f6db4069fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.424+0000 7f6db259c700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db40ae710 0x7f6db40b1760 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f6da400cb90 tx=0x7f6da4001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.424+0000 7f6da27fc700 1 -- 172.21.15.55:0/3849879571 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4001d60 con 0x7f6db40ae710 2023-10-06T05:07:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.424+0000 7f6da27fc700 1 -- 172.21.15.55:0/3849879571 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da4007360 con 0x7f6db40ae710 2023-10-06T05:07:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.424+0000 7f6da27fc700 1 -- 172.21.15.55:0/3849879571 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4007700 con 0x7f6db40ae710 2023-10-06T05:07:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.424+0000 7f6db359e700 1 -- 172.21.15.55:0/3849879571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db40b0090 con 0x7f6db40ae710 2023-10-06T05:07:18.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.425+0000 7f6db359e700 1 -- 172.21.15.55:0/3849879571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db40b0650 con 0x7f6db40ae710 2023-10-06T05:07:18.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.426+0000 7f6db359e700 1 -- 172.21.15.55:0/3849879571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db404c3c0 con 0x7f6db40ae710 2023-10-06T05:07:18.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.426+0000 7f6da27fc700 1 -- 172.21.15.55:0/3849879571 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6da4004a40 con 0x7f6db40ae710 2023-10-06T05:07:18.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.427+0000 7f6da27fc700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6d94066f30 0x7f6d940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:18.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.428+0000 7f6da27fc700 1 -- 172.21.15.55:0/3849879571 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5654+0+0 (secure 0 0 0) 0x7f6da40838f0 con 0x7f6db40ae710 2023-10-06T05:07:18.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.428+0000 7f6db1d9b700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6d94066f30 0x7f6d940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:18.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.429+0000 7f6db1d9b700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6d94066f30 0x7f6d940693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6db415c760 tx=0x7f6da8007ec0 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:18.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.431+0000 7f6da27fc700 1 -- 172.21.15.55:0/3849879571 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6da4050b70 con 0x7f6db40ae710 2023-10-06T05:07:18.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.711+0000 7f6db359e700 1 -- 172.21.15.55:0/3849879571 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6db406ae20 con 0x7f6db40ae710 2023-10-06T05:07:18.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.711+0000 7f6da27fc700 1 -- 172.21.15.55:0/3849879571 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v46) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6da4018020 con 0x7f6db40ae710 2023-10-06T05:07:18.717 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:18.717 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T05:07:18.717 INFO:teuthology.orchestra.run.smithi055.stdout: "auth_dump": [ 2023-10-06T05:07:18.717 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.717 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.a", 2023-10-06T05:07:18.717 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9li2RaGRAAgRjtbw3h40vylEPjolzlpw==", 2023-10-06T05:07:18.717 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.717 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:07:18.718 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T05:07:18.718 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T05:07:18.718 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.718 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.718 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.718 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.718 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.b", 2023-10-06T05:07:18.718 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lXZu7LxAAu3QAtdDxtD/AuuqGG12sDw==", 2023-10-06T05:07:18.719 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.719 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:07:18.719 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T05:07:18.719 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T05:07:18.719 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.719 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.719 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.719 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.719 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.c", 2023-10-06T05:07:18.719 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lS22fJBAAZi6CfyLyJAFgIkaarad1mA==", 2023-10-06T05:07:18.720 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.720 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:07:18.721 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T05:07:18.721 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T05:07:18.721 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.721 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.721 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.721 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.722 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mds.d", 2023-10-06T05:07:18.722 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9ll0RIOhAAGtGdPv4Y2bge7xIv3eu2NQ==", 2023-10-06T05:07:18.722 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.722 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:07:18.722 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T05:07:18.722 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T05:07:18.723 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.723 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.723 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.723 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.723 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.0", 2023-10-06T05:07:18.723 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDHfx9lmTeaDRAAuycc9i7fPj8xrDkuZCAGVg==", 2023-10-06T05:07:18.723 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.723 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:07:18.723 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:07:18.723 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.724 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.724 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.724 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.724 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.1", 2023-10-06T05:07:18.724 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDKfx9lxOdXEhAA9S8tdOvhnmBeEbutqnUsZg==", 2023-10-06T05:07:18.724 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.724 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:07:18.724 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:07:18.724 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.725 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.725 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.725 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.725 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.2", 2023-10-06T05:07:18.725 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDNfx9l76I+GRAAjaj0gM7An9BCyGmfQgaDFw==", 2023-10-06T05:07:18.725 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.725 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:07:18.726 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:07:18.726 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.726 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.726 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.726 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.726 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.3", 2023-10-06T05:07:18.726 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDQfx9lsPK+HxAAzQCV0EuBenqOkUpiQW9uog==", 2023-10-06T05:07:18.726 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.726 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:07:18.726 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:07:18.726 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.726 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.727 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.727 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.727 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.4", 2023-10-06T05:07:18.727 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDWfx9lSHNrMxAAIn7awDmljO2cRtKyDalWnw==", 2023-10-06T05:07:18.727 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.727 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:07:18.727 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:07:18.727 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.728 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.728 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.728 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.728 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.5", 2023-10-06T05:07:18.728 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDZfx9lFoWuOBAAhzQOt3YChvuv0MhrspOjPw==", 2023-10-06T05:07:18.728 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.728 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:07:18.728 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:07:18.728 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.728 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.729 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.729 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.729 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.6", 2023-10-06T05:07:18.729 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDcfx9lB8iSOhAAwamSgGIXZ8owFE4FdbOOBw==", 2023-10-06T05:07:18.729 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.729 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:07:18.729 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:07:18.729 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.729 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.730 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.730 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.730 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "osd.7", 2023-10-06T05:07:18.730 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDgfx9lUb3mABAAJ75UO2UQg2r+l+tDYVn0Dg==", 2023-10-06T05:07:18.730 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.730 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow profile osd", 2023-10-06T05:07:18.730 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile osd", 2023-10-06T05:07:18.730 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.731 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.731 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.731 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.731 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.0", 2023-10-06T05:07:18.731 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lfNvCBxAA1ZJq4XsWTZHVY0u0NdOqIw==", 2023-10-06T05:07:18.731 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.731 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:07:18.731 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow r", 2023-10-06T05:07:18.732 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-06T05:07:18.732 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.733 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.733 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.733 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.1", 2023-10-06T05:07:18.733 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9lnvkoFhAAafxsOmItxmwUr1GJrGYJ6g==", 2023-10-06T05:07:18.733 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.733 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:07:18.733 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T05:07:18.733 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T05:07:18.734 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T05:07:18.734 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.734 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.734 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.734 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.2", 2023-10-06T05:07:18.734 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9l+9b4DhAAc5XHdikWbXsEblSE3FGScQ==", 2023-10-06T05:07:18.734 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.734 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:07:18.735 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T05:07:18.735 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T05:07:18.735 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T05:07:18.735 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.735 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.735 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.736 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.3", 2023-10-06T05:07:18.736 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQDAfx9luSXrHBAAg6Ufa1Ep/EOgsQtuOPQgYg==", 2023-10-06T05:07:18.736 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.736 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow", 2023-10-06T05:07:18.736 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow r", 2023-10-06T05:07:18.736 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow rw", 2023-10-06T05:07:18.736 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow rwx" 2023-10-06T05:07:18.736 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.736 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.736 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.737 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "client.admin", 2023-10-06T05:07:18.737 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC+fx9lEb50FxAAt0Oq28gsXtQvol7TV8trgg==", 2023-10-06T05:07:18.737 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.737 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T05:07:18.738 INFO:teuthology.orchestra.run.smithi055.stdout: "mgr": "allow *", 2023-10-06T05:07:18.738 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow *", 2023-10-06T05:07:18.738 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.738 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.738 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.741 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.741 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.x", 2023-10-06T05:07:18.741 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lQAANEhAAsNiWPqCNEyUS3IELXSC1Cw==", 2023-10-06T05:07:18.741 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.741 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T05:07:18.741 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T05:07:18.741 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.741 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.741 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:18.741 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:18.742 INFO:teuthology.orchestra.run.smithi055.stdout: "entity": "mgr.y", 2023-10-06T05:07:18.742 INFO:teuthology.orchestra.run.smithi055.stdout: "key": "AQC/fx9lCwU2ABAAMijxXd8hwgk3XzcdP4NZyg==", 2023-10-06T05:07:18.742 INFO:teuthology.orchestra.run.smithi055.stdout: "caps": { 2023-10-06T05:07:18.742 INFO:teuthology.orchestra.run.smithi055.stdout: "mds": "allow *", 2023-10-06T05:07:18.742 INFO:teuthology.orchestra.run.smithi055.stdout: "mon": "allow profile mgr", 2023-10-06T05:07:18.742 INFO:teuthology.orchestra.run.smithi055.stdout: "osd": "allow *" 2023-10-06T05:07:18.742 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.742 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:18.742 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2023-10-06T05:07:18.742 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T05:07:18.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.713+0000 7f6db359e700 1 -- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6d94066f30 msgr2=0x7f6d940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:18.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.713+0000 7f6db359e700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6d94066f30 0x7f6d940693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6db415c760 tx=0x7f6da8007ec0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.713+0000 7f6db359e700 1 -- 172.21.15.55:0/3849879571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db40ae710 msgr2=0x7f6db40b1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:18.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.713+0000 7f6db359e700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db40ae710 0x7f6db40b1760 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f6da400cb90 tx=0x7f6da4001ad0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.714+0000 7f6db359e700 1 -- 172.21.15.55:0/3849879571 shutdown_connections 2023-10-06T05:07:18.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.714+0000 7f6db359e700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6d94066f30 0x7f6d940693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.714+0000 7f6db359e700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6db41566e0 0x7f6db4069fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.714+0000 7f6db359e700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6db40ae710 0x7f6db40b1760 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.714+0000 7f6db359e700 1 --2- 172.21.15.55:0/3849879571 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6db4156310 0x7f6db414ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.714+0000 7f6db359e700 1 -- 172.21.15.55:0/3849879571 >> 172.21.15.55:0/3849879571 conn(0x7f6db40cc1f0 msgr2=0x7f6db41517d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:18.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.714+0000 7f6db359e700 1 -- 172.21.15.55:0/3849879571 shutdown_connections 2023-10-06T05:07:18.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.714+0000 7f6db359e700 1 -- 172.21.15.55:0/3849879571 wait complete. 2023-10-06T05:07:18.745 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:07:18.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.935+0000 7fe31299d700 1 Processor -- start 2023-10-06T05:07:18.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.935+0000 7fe31299d700 1 -- start start 2023-10-06T05:07:18.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.936+0000 7fe31299d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe30c06d950 0x7fe30c0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:18.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.936+0000 7fe31299d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe30c06dd20 0x7fe30c0c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:18.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.936+0000 7fe31299d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c06ae70 con 0x7fe30c06a0a0 2023-10-06T05:07:18.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.936+0000 7fe31299d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c06a540 con 0x7fe30c06d950 2023-10-06T05:07:18.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.936+0000 7fe31299d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c06a820 con 0x7fe30c06dd20 2023-10-06T05:07:18.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.937+0000 7fe31199b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe30c06d950 0x7fe30c0c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:18.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.937+0000 7fe31119a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe30c06a0a0 0x7fe30c0c6c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36604/0 (socket says 172.21.15.55:36604) 2023-10-06T05:07:18.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.937+0000 7fe31119a700 1 -- 172.21.15.55:0/1120285692 learned_addr learned my addr 172.21.15.55:0/1120285692 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:18.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.937+0000 7fe31219c700 1 --2- 172.21.15.55:0/1120285692 >> v2:172.21.15.163:3300/0 conn(0x7fe30c06dd20 0x7fe30c0c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:18.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.937+0000 7fe310999700 1 -- 172.21.15.55:0/1120285692 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017175063 0 0) 0x7fe30c06ae70 con 0x7fe30c06a0a0 2023-10-06T05:07:18.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.937+0000 7fe310999700 1 -- 172.21.15.55:0/1120285692 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2f8003570 con 0x7fe30c06a0a0 2023-10-06T05:07:18.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.937+0000 7fe31199b700 1 -- 172.21.15.55:0/1120285692 >> v2:172.21.15.163:3300/0 conn(0x7fe30c06dd20 msgr2=0x7fe30c0c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:18.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.937+0000 7fe31199b700 1 --2- 172.21.15.55:0/1120285692 >> v2:172.21.15.163:3300/0 conn(0x7fe30c06dd20 0x7fe30c0c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.937+0000 7fe31199b700 1 -- 172.21.15.55:0/1120285692 >> v1:172.21.15.55:6789/0 conn(0x7fe30c06a0a0 legacy=0x7fe30c0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:18.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.937+0000 7fe31199b700 1 -- 172.21.15.55:0/1120285692 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c06ab00 con 0x7fe30c06d950 2023-10-06T05:07:18.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.937+0000 7fe31199b700 1 --2- 172.21.15.55:0/1120285692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06d950 0x7fe30c0c8350 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fe30c06e420 tx=0x7fe3080091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bde26295bbe61153 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:18.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.938+0000 7fe310999700 1 -- 172.21.15.55:0/1120285692 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308011630 con 0x7fe30c06d950 2023-10-06T05:07:18.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.938+0000 7fe310999700 1 -- 172.21.15.55:0/1120285692 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3080117b0 con 0x7fe30c06d950 2023-10-06T05:07:18.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.938+0000 7fe310999700 1 -- 172.21.15.55:0/1120285692 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308011b70 con 0x7fe30c06d950 2023-10-06T05:07:18.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.938+0000 7fe31299d700 1 -- 172.21.15.55:0/1120285692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06d950 msgr2=0x7fe30c0c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:18.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.938+0000 7fe31299d700 1 --2- 172.21.15.55:0/1120285692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06d950 0x7fe30c0c8350 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fe30c06e420 tx=0x7fe3080091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.938+0000 7fe31299d700 1 -- 172.21.15.55:0/1120285692 shutdown_connections 2023-10-06T05:07:18.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.938+0000 7fe31299d700 1 --2- 172.21.15.55:0/1120285692 >> v2:172.21.15.163:3300/0 conn(0x7fe30c06dd20 0x7fe30c0c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.938+0000 7fe31299d700 1 --2- 172.21.15.55:0/1120285692 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06d950 0x7fe30c0c8350 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.938+0000 7fe31299d700 1 -- 172.21.15.55:0/1120285692 >> 172.21.15.55:0/1120285692 conn(0x7fe30c0cc1f0 msgr2=0x7fe30c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:18.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.938+0000 7fe31299d700 1 -- 172.21.15.55:0/1120285692 shutdown_connections 2023-10-06T05:07:18.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.938+0000 7fe31299d700 1 -- 172.21.15.55:0/1120285692 wait complete. 2023-10-06T05:07:18.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.940+0000 7fe31299d700 1 Processor -- start 2023-10-06T05:07:18.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.940+0000 7fe31299d700 1 -- start start 2023-10-06T05:07:18.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.941+0000 7fe31299d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c06a0a0 0x7fe30c16db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:18.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.941+0000 7fe31299d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe30c06d950 0x7fe30c15a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:18.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.941+0000 7fe31299d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06dd20 0x7fe30c156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:18.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.941+0000 7fe31299d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c16e080 con 0x7fe30c06dd20 2023-10-06T05:07:18.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.941+0000 7fe31299d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c1680c0 con 0x7fe30c06a0a0 2023-10-06T05:07:18.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.941+0000 7fe31299d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c15ad00 con 0x7fe30c06d950 2023-10-06T05:07:18.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.941+0000 7fe31219c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06dd20 0x7fe30c156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:18.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.941+0000 7fe31219c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06dd20 0x7fe30c156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45604/0 (socket says 172.21.15.55:45604) 2023-10-06T05:07:18.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.941+0000 7fe31219c700 1 -- 172.21.15.55:0/3664580015 learned_addr learned my addr 172.21.15.55:0/3664580015 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:18.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.941+0000 7fe31199b700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c06a0a0 0x7fe30c16db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:18.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.942+0000 7fe31119a700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe30c06d950 0x7fe30c15a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:18.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.942+0000 7fe31219c700 1 -- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe30c06d950 msgr2=0x7fe30c15a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:18.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.942+0000 7fe31219c700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe30c06d950 0x7fe30c15a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.942+0000 7fe31219c700 1 -- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c06a0a0 msgr2=0x7fe30c16db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:18.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.942+0000 7fe31219c700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c06a0a0 0x7fe30c16db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:18.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.942+0000 7fe31219c700 1 -- 172.21.15.55:0/3664580015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe308009420 con 0x7fe30c06dd20 2023-10-06T05:07:18.949 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.942+0000 7fe31199b700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c06a0a0 0x7fe30c16db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:07:18.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.942+0000 7fe31119a700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe30c06d950 0x7fe30c15a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:07:18.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.942+0000 7fe31219c700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06dd20 0x7fe30c156f80 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fe304009e20 tx=0x7fe30400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:18.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.942+0000 7fe301ffb700 1 -- 172.21.15.55:0/3664580015 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30400cbc0 con 0x7fe30c06dd20 2023-10-06T05:07:18.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.942+0000 7fe301ffb700 1 -- 172.21.15.55:0/3664580015 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30400f040 con 0x7fe30c06dd20 2023-10-06T05:07:18.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.942+0000 7fe31299d700 1 -- 172.21.15.55:0/3664580015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30c0c9f80 con 0x7fe30c06dd20 2023-10-06T05:07:18.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.942+0000 7fe301ffb700 1 -- 172.21.15.55:0/3664580015 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304014480 con 0x7fe30c06dd20 2023-10-06T05:07:18.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.942+0000 7fe31299d700 1 -- 172.21.15.55:0/3664580015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c16e290 con 0x7fe30c06dd20 2023-10-06T05:07:18.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.943+0000 7fe31299d700 1 -- 172.21.15.55:0/3664580015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30c04c3c0 con 0x7fe30c06dd20 2023-10-06T05:07:18.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.944+0000 7fe301ffb700 1 -- 172.21.15.55:0/3664580015 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe304014830 con 0x7fe30c06dd20 2023-10-06T05:07:18.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.945+0000 7fe301ffb700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe2f0066f30 0x7fe2f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:18.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.945+0000 7fe301ffb700 1 -- 172.21.15.55:0/3664580015 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5654+0+0 (secure 0 0 0) 0x7fe304084f10 con 0x7fe30c06dd20 2023-10-06T05:07:18.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.945+0000 7fe31199b700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe2f0066f30 0x7fe2f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:18.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.946+0000 7fe31199b700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe2f0066f30 0x7fe2f00693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe30c06e420 tx=0x7fe308017000 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:18.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:18.949+0000 7fe301ffb700 1 -- 172.21.15.55:0/3664580015 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe304052110 con 0x7fe30c06dd20 2023-10-06T05:07:19.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.152+0000 7fe31299d700 1 -- 172.21.15.55:0/3664580015 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe30c0c2280 con 0x7fe30c06dd20 2023-10-06T05:07:19.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.153+0000 7fe301ffb700 1 -- 172.21.15.55:0/3664580015 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v203) v1 ==== 74+0+14394 (secure 0 0 0) 0x7fe304055730 con 0x7fe30c06dd20 2023-10-06T05:07:19.160 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:19.160 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":203,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:07:18.201958+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":191,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:07:19.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.155+0000 7fe31299d700 1 -- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe2f0066f30 msgr2=0x7fe2f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:19.161 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.155+0000 7fe31299d700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe2f0066f30 0x7fe2f00693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe30c06e420 tx=0x7fe308017000 comp rx=0 tx=0).stop 2023-10-06T05:07:19.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.155+0000 7fe31299d700 1 -- 172.21.15.55:0/3664580015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06dd20 msgr2=0x7fe30c156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:19.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.155+0000 7fe31299d700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06dd20 0x7fe30c156f80 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fe304009e20 tx=0x7fe30400b040 comp rx=0 tx=0).stop 2023-10-06T05:07:19.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.155+0000 7fe31299d700 1 -- 172.21.15.55:0/3664580015 shutdown_connections 2023-10-06T05:07:19.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.155+0000 7fe31299d700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe2f0066f30 0x7fe2f00693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.155+0000 7fe31299d700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe30c06dd20 0x7fe30c156f80 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.155+0000 7fe31299d700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe30c06a0a0 0x7fe30c16db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.155+0000 7fe31299d700 1 --2- 172.21.15.55:0/3664580015 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe30c06d950 0x7fe30c15a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.155+0000 7fe31299d700 1 -- 172.21.15.55:0/3664580015 >> 172.21.15.55:0/3664580015 conn(0x7fe30c0cc1f0 msgr2=0x7fe30c0c2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:19.162 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.155+0000 7fe31299d700 1 -- 172.21.15.55:0/3664580015 shutdown_connections 2023-10-06T05:07:19.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.156+0000 7fe31299d700 1 -- 172.21.15.55:0/3664580015 wait complete. 2023-10-06T05:07:19.178 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-06T05:07:19.179 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:07:19.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.370+0000 7f124b59e700 1 Processor -- start 2023-10-06T05:07:19.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.371+0000 7f124b59e700 1 -- start start 2023-10-06T05:07:19.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.372+0000 7f124b59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f124c159450 0x7f124c0c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:19.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.372+0000 7f124b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f124c159820 0x7f124c06d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:19.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.372+0000 7f124b59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f124c0c8730 con 0x7f124c0c96e0 2023-10-06T05:07:19.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.372+0000 7f124b59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124c0c7b00 con 0x7f124c159820 2023-10-06T05:07:19.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.372+0000 7f124b59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f124c0c89a0 con 0x7f124c159450 2023-10-06T05:07:19.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.372+0000 7f124ad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f124c159820 0x7f124c06d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:19.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.372+0000 7f124ad9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f124c159820 0x7f124c06d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45608/0 (socket says 172.21.15.55:45608) 2023-10-06T05:07:19.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.372+0000 7f124ad9d700 1 -- 172.21.15.55:0/4201585642 learned_addr learned my addr 172.21.15.55:0/4201585642 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:19.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.372+0000 7f124a59c700 1 --2- 172.21.15.55:0/4201585642 >> v2:172.21.15.163:3300/0 conn(0x7f124c159450 0x7f124c0c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:19.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.372+0000 7f124ad9d700 1 -- 172.21.15.55:0/4201585642 >> v2:172.21.15.163:3300/0 conn(0x7f124c159450 msgr2=0x7f124c0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:19.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.372+0000 7f124ad9d700 1 --2- 172.21.15.55:0/4201585642 >> v2:172.21.15.163:3300/0 conn(0x7f124c159450 0x7f124c0c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.373+0000 7f124ad9d700 1 -- 172.21.15.55:0/4201585642 >> v1:172.21.15.163:6789/0 conn(0x7f124c0c96e0 legacy=0x7f124c0a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:19.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.373+0000 7f124ad9d700 1 -- 172.21.15.55:0/4201585642 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c0c8c80 con 0x7f124c159820 2023-10-06T05:07:19.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.373+0000 7f124ad9d700 1 --2- 172.21.15.55:0/4201585642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124c159820 0x7f124c06d750 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f124c06d3f0 tx=0x7f1244009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fba9344854cfe46 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:19.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.373+0000 7f124959a700 1 -- 172.21.15.55:0/4201585642 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124400e070 con 0x7f124c159820 2023-10-06T05:07:19.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.373+0000 7f124959a700 1 -- 172.21.15.55:0/4201585642 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f124400ba10 con 0x7f124c159820 2023-10-06T05:07:19.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.373+0000 7f124959a700 1 -- 172.21.15.55:0/4201585642 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12440092e0 con 0x7f124c159820 2023-10-06T05:07:19.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.374+0000 7f124b59e700 1 -- 172.21.15.55:0/4201585642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124c159820 msgr2=0x7f124c06d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:19.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.374+0000 7f124b59e700 1 --2- 172.21.15.55:0/4201585642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124c159820 0x7f124c06d750 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f124c06d3f0 tx=0x7f1244009840 comp rx=0 tx=0).stop 2023-10-06T05:07:19.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.374+0000 7f124b59e700 1 -- 172.21.15.55:0/4201585642 shutdown_connections 2023-10-06T05:07:19.379 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.374+0000 7f124b59e700 1 --2- 172.21.15.55:0/4201585642 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124c159820 0x7f124c06d750 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.374+0000 7f124b59e700 1 --2- 172.21.15.55:0/4201585642 >> v2:172.21.15.163:3300/0 conn(0x7f124c159450 0x7f124c0c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.374+0000 7f124b59e700 1 -- 172.21.15.55:0/4201585642 >> 172.21.15.55:0/4201585642 conn(0x7f124c0698e0 msgr2=0x7f124c069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:19.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.374+0000 7f124b59e700 1 -- 172.21.15.55:0/4201585642 shutdown_connections 2023-10-06T05:07:19.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.374+0000 7f124b59e700 1 -- 172.21.15.55:0/4201585642 wait complete. 2023-10-06T05:07:19.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.375+0000 7f124b59e700 1 Processor -- start 2023-10-06T05:07:19.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.375+0000 7f124b59e700 1 -- start start 2023-10-06T05:07:19.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.376+0000 7f124b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124c0c96e0 0x7f124c150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:19.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.376+0000 7f124b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f124c159450 0x7f124c0aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:19.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.376+0000 7f124b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f124c159820 0x7f124c1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:19.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.376+0000 7f124b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0ac610 con 0x7f124c0c96e0 2023-10-06T05:07:19.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.376+0000 7f124b59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0ab4f0 con 0x7f124c159450 2023-10-06T05:07:19.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.376+0000 7f124b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124c0abce0 con 0x7f124c159820 2023-10-06T05:07:19.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.376+0000 7f124a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124c0c96e0 0x7f124c150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:19.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f124a59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124c0c96e0 0x7f124c150ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45616/0 (socket says 172.21.15.55:45616) 2023-10-06T05:07:19.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f124a59c700 1 -- 172.21.15.55:0/1569841141 learned_addr learned my addr 172.21.15.55:0/1569841141 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:19.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f1249d9b700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f124c159450 0x7f124c0aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:19.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f124ad9d700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f124c159820 0x7f124c1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:19.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f124a59c700 1 -- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f124c159820 msgr2=0x7f124c1582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:19.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f124a59c700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f124c159820 0x7f124c1582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f124a59c700 1 -- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f124c159450 msgr2=0x7f124c0aaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:19.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f124a59c700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f124c159450 0x7f124c0aaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f124a59c700 1 -- 172.21.15.55:0/1569841141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1244009ae0 con 0x7f124c0c96e0 2023-10-06T05:07:19.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f1249d9b700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f124c159450 0x7f124c0aaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:19.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f124a59c700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124c0c96e0 0x7f124c150ab0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f123c00d890 tx=0x7f123c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:19.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f123a7fc700 1 -- 172.21.15.55:0/1569841141 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c00b600 con 0x7f124c0c96e0 2023-10-06T05:07:19.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f123a7fc700 1 -- 172.21.15.55:0/1569841141 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f123c00b760 con 0x7f124c0c96e0 2023-10-06T05:07:19.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f123a7fc700 1 -- 172.21.15.55:0/1569841141 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f123c00bae0 con 0x7f124c0c96e0 2023-10-06T05:07:19.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.377+0000 7f124b59e700 1 -- 172.21.15.55:0/1569841141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c0abfc0 con 0x7f124c0c96e0 2023-10-06T05:07:19.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.378+0000 7f124b59e700 1 -- 172.21.15.55:0/1569841141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c150ff0 con 0x7f124c0c96e0 2023-10-06T05:07:19.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.379+0000 7f124b59e700 1 -- 172.21.15.55:0/1569841141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124c06ca90 con 0x7f124c0c96e0 2023-10-06T05:07:19.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.380+0000 7f123a7fc700 1 -- 172.21.15.55:0/1569841141 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f123c01e460 con 0x7f124c0c96e0 2023-10-06T05:07:19.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.381+0000 7f123a7fc700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f122c066f30 0x7f122c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:19.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.381+0000 7f123a7fc700 1 -- 172.21.15.55:0/1569841141 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5654+0+0 (secure 0 0 0) 0x7f123c084be0 con 0x7f124c0c96e0 2023-10-06T05:07:19.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.381+0000 7f1249d9b700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f122c066f30 0x7f122c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:19.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.382+0000 7f1249d9b700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f122c066f30 0x7f122c0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1240007fd0 tx=0x7f1240009040 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:19.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.385+0000 7f123a7fc700 1 -- 172.21.15.55:0/1569841141 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f123c051e80 con 0x7f124c0c96e0 2023-10-06T05:07:19.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.586+0000 7f124b59e700 1 -- 172.21.15.55:0/1569841141 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f124c0c7de0 con 0x7f124c0c96e0 2023-10-06T05:07:19.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.587+0000 7f123a7fc700 1 -- 172.21.15.55:0/1569841141 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v203) v1 ==== 92+0+10 (secure 0 0 0) 0x7f123c0554a0 con 0x7f124c0c96e0 2023-10-06T05:07:19.590 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T05:07:19.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.589+0000 7f124b59e700 1 -- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f122c066f30 msgr2=0x7f122c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:19.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.589+0000 7f124b59e700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f122c066f30 0x7f122c0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1240007fd0 tx=0x7f1240009040 comp rx=0 tx=0).stop 2023-10-06T05:07:19.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.589+0000 7f124b59e700 1 -- 172.21.15.55:0/1569841141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124c0c96e0 msgr2=0x7f124c150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:19.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.589+0000 7f124b59e700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124c0c96e0 0x7f124c150ab0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f123c00d890 tx=0x7f123c008040 comp rx=0 tx=0).stop 2023-10-06T05:07:19.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.589+0000 7f124b59e700 1 -- 172.21.15.55:0/1569841141 shutdown_connections 2023-10-06T05:07:19.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.589+0000 7f124b59e700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f122c066f30 0x7f122c0693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.589+0000 7f124b59e700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f124c159820 0x7f124c1582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.589+0000 7f124b59e700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f124c0c96e0 0x7f124c150ab0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.589+0000 7f124b59e700 1 --2- 172.21.15.55:0/1569841141 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f124c159450 0x7f124c0aaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.589+0000 7f124b59e700 1 -- 172.21.15.55:0/1569841141 >> 172.21.15.55:0/1569841141 conn(0x7f124c0698e0 msgr2=0x7f124c1528f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:19.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.589+0000 7f124b59e700 1 -- 172.21.15.55:0/1569841141 shutdown_connections 2023-10-06T05:07:19.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.589+0000 7f124b59e700 1 -- 172.21.15.55:0/1569841141 wait complete. 2023-10-06T05:07:19.610 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:07:19.801 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.798+0000 7faed759d700 1 Processor -- start 2023-10-06T05:07:19.801 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.798+0000 7faed759d700 1 -- start start 2023-10-06T05:07:19.802 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.799+0000 7faed759d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00c9db0 0x7faed006a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:19.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.799+0000 7faed759d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed006e150 con 0x7faed0153900 2023-10-06T05:07:19.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.799+0000 7faed759d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed00c7800 con 0x7faed0157130 2023-10-06T05:07:19.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.799+0000 7faed759d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed00c7610 con 0x7faed00c9db0 2023-10-06T05:07:19.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.799+0000 7faed659b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7faed0153900 0x7faed00c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36612/0 (socket says 172.21.15.55:36612) 2023-10-06T05:07:19.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.799+0000 7faed659b700 1 -- 172.21.15.55:0/2799624830 learned_addr learned my addr 172.21.15.55:0/2799624830 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:19.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 -- 172.21.15.55:0/2799624830 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174774348 0 0) 0x7faed006e150 con 0x7faed0153900 2023-10-06T05:07:19.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 -- 172.21.15.55:0/2799624830 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faebc004680 con 0x7faed0153900 2023-10-06T05:07:19.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5d9a700 1 --2- 172.21.15.55:0/2799624830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00c9db0 0x7faed006a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:19.805 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 -- 172.21.15.55:0/2799624830 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 583116580 0 0) 0x7faebc004680 con 0x7faed0153900 2023-10-06T05:07:19.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 -- 172.21.15.55:0/2799624830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00c9db0 msgr2=0x7faed006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:19.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 --2- 172.21.15.55:0/2799624830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00c9db0 0x7faed006a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 -- 172.21.15.55:0/2799624830 >> v1:172.21.15.163:6789/0 conn(0x7faed0157130 legacy=0x7faed006e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:19.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 -- 172.21.15.55:0/2799624830 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00c7ae0 con 0x7faed0153900 2023-10-06T05:07:19.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 -- 172.21.15.55:0/2799624830 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7faecc001470 con 0x7faed0153900 2023-10-06T05:07:19.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 -- 172.21.15.55:0/2799624830 >> v1:172.21.15.55:6789/0 conn(0x7faed0153900 legacy=0x7faed00c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:19.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 --2- 172.21.15.55:0/2799624830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faebc004ba0 0x7faebc004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:19.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 --2- 172.21.15.55:0/2799624830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00c9db0 0x7faebc005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:19.806 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 --2- 172.21.15.55:0/2799624830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faebc005e60 0x7faebc0062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:19.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 -- 172.21.15.55:0/2799624830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc00cd00 con 0x7faebc004ba0 2023-10-06T05:07:19.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 -- 172.21.15.55:0/2799624830 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc00ce40 con 0x7faebc005e60 2023-10-06T05:07:19.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.800+0000 7faed5599700 1 -- 172.21.15.55:0/2799624830 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc0034e0 con 0x7faed00c9db0 2023-10-06T05:07:19.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.801+0000 7faed659b700 1 --2- 172.21.15.55:0/2799624830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faebc004ba0 0x7faebc004fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:19.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.801+0000 7faed6d9c700 1 --2- 172.21.15.55:0/2799624830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faebc005e60 0x7faebc0062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:19.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.801+0000 7faed5d9a700 1 --2- 172.21.15.55:0/2799624830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00c9db0 0x7faebc005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:19.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.801+0000 7faed659b700 1 -- 172.21.15.55:0/2799624830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00c9db0 msgr2=0x7faebc005920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:19.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.801+0000 7faed659b700 1 --2- 172.21.15.55:0/2799624830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00c9db0 0x7faebc005920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.801+0000 7faed659b700 1 -- 172.21.15.55:0/2799624830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faebc005e60 msgr2=0x7faebc0062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:19.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.801+0000 7faed659b700 1 --2- 172.21.15.55:0/2799624830 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faebc005e60 0x7faebc0062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.801+0000 7faed659b700 1 -- 172.21.15.55:0/2799624830 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc004680 con 0x7faebc004ba0 2023-10-06T05:07:19.807 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.801+0000 7faed5d9a700 1 --2- 172.21.15.55:0/2799624830 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00c9db0 0x7faebc005920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:19.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.801+0000 7faed659b700 1 --2- 172.21.15.55:0/2799624830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faebc004ba0 0x7faebc004fb0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7faecc00c2d0 tx=0x7faecc001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75bee680b94eb55 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:19.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.801+0000 7faed5599700 1 -- 172.21.15.55:0/2799624830 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc00cba0 con 0x7faebc004ba0 2023-10-06T05:07:19.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.801+0000 7faed5599700 1 -- 172.21.15.55:0/2799624830 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faecc00cd20 con 0x7faebc004ba0 2023-10-06T05:07:19.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.801+0000 7faed5599700 1 -- 172.21.15.55:0/2799624830 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc005b70 con 0x7faebc004ba0 2023-10-06T05:07:19.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.802+0000 7faed759d700 1 -- 172.21.15.55:0/2799624830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faebc004ba0 msgr2=0x7faebc004fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:19.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.802+0000 7faed759d700 1 --2- 172.21.15.55:0/2799624830 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faebc004ba0 0x7faebc004fb0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7faecc00c2d0 tx=0x7faecc001d80 comp rx=0 tx=0).stop 2023-10-06T05:07:19.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.802+0000 7faed6d9c700 1 -- 172.21.15.55:0/2799624830 reap_dead start 2023-10-06T05:07:19.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.802+0000 7faed759d700 1 -- 172.21.15.55:0/2799624830 shutdown_connections 2023-10-06T05:07:19.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.802+0000 7faed759d700 1 -- 172.21.15.55:0/2799624830 >> 172.21.15.55:0/2799624830 conn(0x7faed00cc1e0 msgr2=0x7faed00cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:19.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.802+0000 7faed759d700 1 -- 172.21.15.55:0/2799624830 shutdown_connections 2023-10-06T05:07:19.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.802+0000 7faed759d700 1 -- 172.21.15.55:0/2799624830 wait complete. 2023-10-06T05:07:19.810 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.804+0000 7faed759d700 1 Processor -- start 2023-10-06T05:07:19.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.804+0000 7faed759d700 1 -- start start 2023-10-06T05:07:19.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.805+0000 7faed759d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faebc004ba0 0x7faed013fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:19.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.805+0000 7faed759d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00aef80 0x7faed00af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:19.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.805+0000 7faed759d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed00b1090 0x7faed00b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:19.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.805+0000 7faed759d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed0151030 con 0x7faed00b1090 2023-10-06T05:07:19.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.805+0000 7faed759d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed00af930 con 0x7faebc004ba0 2023-10-06T05:07:19.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.805+0000 7faed759d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed00ac250 con 0x7faed00aef80 2023-10-06T05:07:19.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.805+0000 7faed6d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed00b1090 0x7faed00b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:19.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.805+0000 7faed6d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed00b1090 0x7faed00b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45642/0 (socket says 172.21.15.55:45642) 2023-10-06T05:07:19.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.805+0000 7faed6d9c700 1 -- 172.21.15.55:0/303465007 learned_addr learned my addr 172.21.15.55:0/303465007 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.805+0000 7faed5d9a700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00aef80 0x7faed00af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.805+0000 7faed659b700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faebc004ba0 0x7faed013fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.806+0000 7faed6d9c700 1 -- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00aef80 msgr2=0x7faed00af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.806+0000 7faed6d9c700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00aef80 0x7faed00af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.806+0000 7faed6d9c700 1 -- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faebc004ba0 msgr2=0x7faed013fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.806+0000 7faed6d9c700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faebc004ba0 0x7faed013fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.806+0000 7faed6d9c700 1 -- 172.21.15.55:0/303465007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed00ac4f0 con 0x7faed00b1090 2023-10-06T05:07:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.806+0000 7faed659b700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faebc004ba0 0x7faed013fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.806+0000 7faed6d9c700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed00b1090 0x7faed00b14e0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7faec8007fd0 tx=0x7faec8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.806+0000 7faec67fc700 1 -- 172.21.15.55:0/303465007 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8006db0 con 0x7faed00b1090 2023-10-06T05:07:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.806+0000 7faec67fc700 1 -- 172.21.15.55:0/303465007 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec8003030 con 0x7faed00b1090 2023-10-06T05:07:19.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.806+0000 7faec67fc700 1 -- 172.21.15.55:0/303465007 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec80033f0 con 0x7faed00b1090 2023-10-06T05:07:19.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.806+0000 7faed759d700 1 -- 172.21.15.55:0/303465007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed00ac7d0 con 0x7faed00b1090 2023-10-06T05:07:19.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.806+0000 7faed759d700 1 -- 172.21.15.55:0/303465007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed00b1d50 con 0x7faed00b1090 2023-10-06T05:07:19.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.807+0000 7faed759d700 1 -- 172.21.15.55:0/303465007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed00accf0 con 0x7faed00b1090 2023-10-06T05:07:19.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.808+0000 7faec67fc700 1 -- 172.21.15.55:0/303465007 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faec8001790 con 0x7faed00b1090 2023-10-06T05:07:19.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.809+0000 7faec67fc700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faeb4066f30 0x7faeb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:19.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.809+0000 7faec67fc700 1 -- 172.21.15.55:0/303465007 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5654+0+0 (secure 0 0 0) 0x7faec8083560 con 0x7faed00b1090 2023-10-06T05:07:19.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.810+0000 7faed659b700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faeb4066f30 0x7faeb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:19.814 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.810+0000 7faed659b700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faeb4066f30 0x7faeb40693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faecc007c00 tx=0x7faecc015000 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:19.817 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:19.813+0000 7faec67fc700 1 -- 172.21.15.55:0/303465007 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faed00accf0 con 0x7faed00b1090 2023-10-06T05:07:20.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.013+0000 7faed759d700 1 -- 172.21.15.55:0/303465007 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faed00accf0 con 0x7faed00b1090 2023-10-06T05:07:20.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.014+0000 7faec67fc700 1 -- 172.21.15.55:0/303465007 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v203) v1 ==== 93+0+10 (secure 0 0 0) 0x7faed00accf0 con 0x7faed00b1090 2023-10-06T05:07:20.017 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T05:07:20.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.016+0000 7faed759d700 1 -- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faeb4066f30 msgr2=0x7faeb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:20.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.016+0000 7faed759d700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faeb4066f30 0x7faeb40693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7faecc007c00 tx=0x7faecc015000 comp rx=0 tx=0).stop 2023-10-06T05:07:20.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.016+0000 7faed759d700 1 -- 172.21.15.55:0/303465007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed00b1090 msgr2=0x7faed00b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:20.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.016+0000 7faed759d700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed00b1090 0x7faed00b14e0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7faec8007fd0 tx=0x7faec8009040 comp rx=0 tx=0).stop 2023-10-06T05:07:20.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.016+0000 7faed759d700 1 -- 172.21.15.55:0/303465007 shutdown_connections 2023-10-06T05:07:20.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.016+0000 7faed759d700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faeb4066f30 0x7faeb40693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.016+0000 7faed759d700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faed00b1090 0x7faed00b14e0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.016+0000 7faed759d700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faebc004ba0 0x7faed013fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.016+0000 7faed759d700 1 --2- 172.21.15.55:0/303465007 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faed00aef80 0x7faed00af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.016+0000 7faed759d700 1 -- 172.21.15.55:0/303465007 >> 172.21.15.55:0/303465007 conn(0x7faed00cc1e0 msgr2=0x7faed0069c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:20.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.016+0000 7faed759d700 1 -- 172.21.15.55:0/303465007 shutdown_connections 2023-10-06T05:07:20.020 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.016+0000 7faed759d700 1 -- 172.21.15.55:0/303465007 wait complete. 2023-10-06T05:07:20.037 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T05:07:20.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.223+0000 7f8fabfff700 1 Processor -- start 2023-10-06T05:07:20.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.223+0000 7f8fabfff700 1 -- start start 2023-10-06T05:07:20.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.225+0000 7f8fabfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8fac0c6c60 0x7f8fac0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:20.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.225+0000 7f8fabfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fac06e8c0 0x7f8fac0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:20.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.225+0000 7f8fabfff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fac0b1410 con 0x7f8fac1595a0 2023-10-06T05:07:20.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.225+0000 7f8fabfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fac0b2b90 con 0x7f8fac0c6c60 2023-10-06T05:07:20.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.225+0000 7f8fabfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac063230 con 0x7f8fac06e8c0 2023-10-06T05:07:20.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.225+0000 7f8faa7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8fac0c6c60 0x7f8fac0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:20.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.225+0000 7f8fab7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fac06e8c0 0x7f8fac0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:20.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.225+0000 7f8faa7fc700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8fac0c6c60 0x7f8fac0c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:35870/0 (socket says 172.21.15.55:35870) 2023-10-06T05:07:20.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.225+0000 7f8faa7fc700 1 -- 172.21.15.55:0/3801260669 learned_addr learned my addr 172.21.15.55:0/3801260669 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:20.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.225+0000 7f8faaffd700 1 --1- 172.21.15.55:0/3801260669 >> v1:172.21.15.163:6789/0 conn(0x7f8fac1595a0 0x7f8fac0b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:39840/0 (socket says 172.21.15.55:39840) 2023-10-06T05:07:20.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.226+0000 7f8faa7fc700 1 -- 172.21.15.55:0/3801260669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fac06e8c0 msgr2=0x7f8fac0c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:20.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.226+0000 7f8faa7fc700 1 --2- 172.21.15.55:0/3801260669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fac06e8c0 0x7f8fac0c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.226+0000 7f8faa7fc700 1 -- 172.21.15.55:0/3801260669 >> v1:172.21.15.163:6789/0 conn(0x7f8fac1595a0 legacy=0x7f8fac0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:20.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.226+0000 7f8faa7fc700 1 -- 172.21.15.55:0/3801260669 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac0634f0 con 0x7f8fac0c6c60 2023-10-06T05:07:20.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.226+0000 7f8fab7fe700 1 --2- 172.21.15.55:0/3801260669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fac06e8c0 0x7f8fac0c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:20.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.226+0000 7f8faa7fc700 1 --2- 172.21.15.55:0/3801260669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fac0c6c60 0x7f8fac0c8120 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00099b0 tx=0x7f8fa0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d482eb966c7d467 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:20.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.226+0000 7f8fa9ffb700 1 -- 172.21.15.55:0/3801260669 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0009ea0 con 0x7f8fac0c6c60 2023-10-06T05:07:20.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.226+0000 7f8fa9ffb700 1 -- 172.21.15.55:0/3801260669 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa000b860 con 0x7f8fac0c6c60 2023-10-06T05:07:20.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.226+0000 7f8fa9ffb700 1 -- 172.21.15.55:0/3801260669 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000bc00 con 0x7f8fac0c6c60 2023-10-06T05:07:20.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.227+0000 7f8fabfff700 1 -- 172.21.15.55:0/3801260669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fac0c6c60 msgr2=0x7f8fac0c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:20.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.227+0000 7f8fabfff700 1 --2- 172.21.15.55:0/3801260669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fac0c6c60 0x7f8fac0c8120 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00099b0 tx=0x7f8fa0009250 comp rx=0 tx=0).stop 2023-10-06T05:07:20.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.227+0000 7f8fabfff700 1 -- 172.21.15.55:0/3801260669 shutdown_connections 2023-10-06T05:07:20.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.227+0000 7f8fabfff700 1 --2- 172.21.15.55:0/3801260669 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fac06e8c0 0x7f8fac0c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.227+0000 7f8fabfff700 1 --2- 172.21.15.55:0/3801260669 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fac0c6c60 0x7f8fac0c8120 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.227+0000 7f8fabfff700 1 -- 172.21.15.55:0/3801260669 >> 172.21.15.55:0/3801260669 conn(0x7f8fac06ddb0 msgr2=0x7f8fac06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:20.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.227+0000 7f8fabfff700 1 -- 172.21.15.55:0/3801260669 shutdown_connections 2023-10-06T05:07:20.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.227+0000 7f8fabfff700 1 -- 172.21.15.55:0/3801260669 wait complete. 2023-10-06T05:07:20.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.228+0000 7f8fabfff700 1 Processor -- start 2023-10-06T05:07:20.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.229+0000 7f8fabfff700 1 -- start start 2023-10-06T05:07:20.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.229+0000 7f8fabfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fac06e8c0 0x7f8fac14c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:20.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.229+0000 7f8fabfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fac0c6c60 0x7f8fac14a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:20.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.230+0000 7f8fabfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8fac1595a0 0x7f8fac152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:20.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.230+0000 7f8fabfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac16d790 con 0x7f8fac1595a0 2023-10-06T05:07:20.242 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.230+0000 7f8fabfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac14ca70 con 0x7f8fac06e8c0 2023-10-06T05:07:20.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.230+0000 7f8fabfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac15bc70 con 0x7f8fac0c6c60 2023-10-06T05:07:20.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.230+0000 7f8fab7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8fac1595a0 0x7f8fac152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:20.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.230+0000 7f8faa7fc700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fac0c6c60 0x7f8fac14a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:20.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.230+0000 7f8fab7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8fac1595a0 0x7f8fac152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45656/0 (socket says 172.21.15.55:45656) 2023-10-06T05:07:20.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.230+0000 7f8fab7fe700 1 -- 172.21.15.55:0/2176304224 learned_addr learned my addr 172.21.15.55:0/2176304224 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:20.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.230+0000 7f8faaffd700 1 --2- 172.21.15.55:0/2176304224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fac06e8c0 0x7f8fac14c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:20.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.230+0000 7f8fab7fe700 1 -- 172.21.15.55:0/2176304224 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fac0c6c60 msgr2=0x7f8fac14a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:20.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.230+0000 7f8fab7fe700 1 --2- 172.21.15.55:0/2176304224 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fac0c6c60 0x7f8fac14a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.230+0000 7f8fab7fe700 1 -- 172.21.15.55:0/2176304224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fac06e8c0 msgr2=0x7f8fac14c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:20.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.230+0000 7f8fab7fe700 1 --2- 172.21.15.55:0/2176304224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fac06e8c0 0x7f8fac14c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.230+0000 7f8fab7fe700 1 -- 172.21.15.55:0/2176304224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa4009a20 con 0x7f8fac1595a0 2023-10-06T05:07:20.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.231+0000 7f8fab7fe700 1 --2- 172.21.15.55:0/2176304224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8fac1595a0 0x7f8fac152370 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009fc0 tx=0x7f8fa400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:20.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.231+0000 7f8f92ffd700 1 -- 172.21.15.55:0/2176304224 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa400cbd0 con 0x7f8fac1595a0 2023-10-06T05:07:20.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.231+0000 7f8f92ffd700 1 -- 172.21.15.55:0/2176304224 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa400f040 con 0x7f8fac1595a0 2023-10-06T05:07:20.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.231+0000 7f8fabfff700 1 -- 172.21.15.55:0/2176304224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa00094f0 con 0x7f8fac1595a0 2023-10-06T05:07:20.246 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.231+0000 7f8f92ffd700 1 -- 172.21.15.55:0/2176304224 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4014480 con 0x7f8fac1595a0 2023-10-06T05:07:20.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.231+0000 7f8fabfff700 1 -- 172.21.15.55:0/2176304224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fac15c230 con 0x7f8fac1595a0 2023-10-06T05:07:20.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.232+0000 7f8fabfff700 1 -- 172.21.15.55:0/2176304224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fac16d900 con 0x7f8fac1595a0 2023-10-06T05:07:20.247 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.233+0000 7f8f92ffd700 1 -- 172.21.15.55:0/2176304224 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8fa4003020 con 0x7f8fac1595a0 2023-10-06T05:07:20.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.234+0000 7f8f92ffd700 1 --2- 172.21.15.55:0/2176304224 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8f98066f30 0x7f8f980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:20.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.234+0000 7f8f92ffd700 1 -- 172.21.15.55:0/2176304224 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5654+0+0 (secure 0 0 0) 0x7f8fa4084ec0 con 0x7f8fac1595a0 2023-10-06T05:07:20.248 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.234+0000 7f8faaffd700 1 --2- 172.21.15.55:0/2176304224 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8f98066f30 0x7f8f980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:20.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.235+0000 7f8faaffd700 1 --2- 172.21.15.55:0/2176304224 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8f98066f30 0x7f8f980693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c004fd0 tx=0x7f8f9c007eb0 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:20.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.238+0000 7f8f92ffd700 1 -- 172.21.15.55:0/2176304224 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8fa40520c0 con 0x7f8fac1595a0 2023-10-06T05:07:20.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.442+0000 7f8fabfff700 1 -- 172.21.15.55:0/2176304224 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8fac04c3c0 con 0x7f8fac1595a0 2023-10-06T05:07:20.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.442+0000 7f8f92ffd700 1 -- 172.21.15.55:0/2176304224 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v203) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8fa4018090 con 0x7f8fac1595a0 2023-10-06T05:07:20.446 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:20.446 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T05:07:20.446 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:20.446 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 1, 2023-10-06T05:07:20.446 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": "rbd" 2023-10-06T05:07:20.447 INFO:teuthology.orchestra.run.smithi055.stdout: }, 2023-10-06T05:07:20.447 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:20.447 INFO:teuthology.orchestra.run.smithi055.stdout: "poolnum": 2, 2023-10-06T05:07:20.447 INFO:teuthology.orchestra.run.smithi055.stdout: "poolname": ".mgr" 2023-10-06T05:07:20.447 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:20.447 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T05:07:20.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.444+0000 7f8fabfff700 1 -- 172.21.15.55:0/2176304224 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8f98066f30 msgr2=0x7f8f980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.444+0000 7f8fabfff700 1 --2- 172.21.15.55:0/2176304224 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8f98066f30 0x7f8f980693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c004fd0 tx=0x7f8f9c007eb0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.444+0000 7f8fabfff700 1 -- 172.21.15.55:0/2176304224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8fac1595a0 msgr2=0x7f8fac152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.444+0000 7f8fabfff700 1 --2- 172.21.15.55:0/2176304224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8fac1595a0 0x7f8fac152370 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4009fc0 tx=0x7f8fa400b040 comp rx=0 tx=0).stop 2023-10-06T05:07:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.444+0000 7f8fabfff700 1 -- 172.21.15.55:0/2176304224 shutdown_connections 2023-10-06T05:07:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.444+0000 7f8fabfff700 1 --2- 172.21.15.55:0/2176304224 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8f98066f30 0x7f8f980693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.444+0000 7f8fabfff700 1 --2- 172.21.15.55:0/2176304224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8fac1595a0 0x7f8fac152370 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.444+0000 7f8fabfff700 1 --2- 172.21.15.55:0/2176304224 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8fac06e8c0 0x7f8fac14c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.444+0000 7f8fabfff700 1 --2- 172.21.15.55:0/2176304224 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8fac0c6c60 0x7f8fac14a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.444+0000 7f8fabfff700 1 -- 172.21.15.55:0/2176304224 >> 172.21.15.55:0/2176304224 conn(0x7f8fac06ddb0 msgr2=0x7f8fac0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:20.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.445+0000 7f8fabfff700 1 -- 172.21.15.55:0/2176304224 shutdown_connections 2023-10-06T05:07:20.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.445+0000 7f8fabfff700 1 -- 172.21.15.55:0/2176304224 wait complete. 2023-10-06T05:07:20.466 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T05:07:20.466 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T05:07:20.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.655+0000 7f464899d700 1 Processor -- start 2023-10-06T05:07:20.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.656+0000 7f464899d700 1 -- start start 2023-10-06T05:07:20.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.657+0000 7f464899d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4644153480 0x7f46440ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:20.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.657+0000 7f464899d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f4644153850 0x7f46441596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:20.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.657+0000 7f464899d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f464415a030 con 0x7f4644159bf0 2023-10-06T05:07:20.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.657+0000 7f464899d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46440aec90 con 0x7f4644153480 2023-10-06T05:07:20.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.657+0000 7f464899d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f464415a480 con 0x7f4644153850 2023-10-06T05:07:20.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.657+0000 7f4643fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4644159bf0 0x7f46441560b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36628/0 (socket says 172.21.15.55:36628) 2023-10-06T05:07:20.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.657+0000 7f4643fff700 1 -- 172.21.15.55:0/926873177 learned_addr learned my addr 172.21.15.55:0/926873177 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:20.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.657+0000 7f46437fe700 1 --2- 172.21.15.55:0/926873177 >> v2:172.21.15.55:3300/0 conn(0x7f4644153480 0x7f46440ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:20.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.658+0000 7f4642ffd700 1 --2- 172.21.15.55:0/926873177 >> v2:172.21.15.163:3300/0 conn(0x7f4644153850 0x7f46441596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:20.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.658+0000 7f46437fe700 1 -- 172.21.15.55:0/926873177 >> v2:172.21.15.163:3300/0 conn(0x7f4644153850 msgr2=0x7f46441596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:20.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.658+0000 7f46437fe700 1 --2- 172.21.15.55:0/926873177 >> v2:172.21.15.163:3300/0 conn(0x7f4644153850 0x7f46441596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.658+0000 7f46427fc700 1 -- 172.21.15.55:0/926873177 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404171283 0 0) 0x7f464415a030 con 0x7f4644159bf0 2023-10-06T05:07:20.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.658+0000 7f46437fe700 1 -- 172.21.15.55:0/926873177 >> v1:172.21.15.55:6789/0 conn(0x7f4644159bf0 legacy=0x7f46441560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:20.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.658+0000 7f46437fe700 1 -- 172.21.15.55:0/926873177 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46440afb40 con 0x7f4644153480 2023-10-06T05:07:20.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.658+0000 7f46437fe700 1 --2- 172.21.15.55:0/926873177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4644153480 0x7f46440ae750 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f4644064f60 tx=0x7f46340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09aabd392a59791 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.658+0000 7f46427fc700 1 -- 172.21.15.55:0/926873177 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463400b6b0 con 0x7f4644153480 2023-10-06T05:07:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.658+0000 7f46427fc700 1 -- 172.21.15.55:0/926873177 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463400b810 con 0x7f4644153480 2023-10-06T05:07:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.658+0000 7f46427fc700 1 -- 172.21.15.55:0/926873177 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463400bbb0 con 0x7f4644153480 2023-10-06T05:07:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.659+0000 7f464899d700 1 -- 172.21.15.55:0/926873177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4644153480 msgr2=0x7f46440ae750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.659+0000 7f464899d700 1 --2- 172.21.15.55:0/926873177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4644153480 0x7f46440ae750 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f4644064f60 tx=0x7f46340091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.659+0000 7f464899d700 1 -- 172.21.15.55:0/926873177 shutdown_connections 2023-10-06T05:07:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.659+0000 7f464899d700 1 --2- 172.21.15.55:0/926873177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4644153480 0x7f46440ae750 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.659+0000 7f464899d700 1 --2- 172.21.15.55:0/926873177 >> v2:172.21.15.163:3300/0 conn(0x7f4644153850 0x7f46441596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.659+0000 7f464899d700 1 -- 172.21.15.55:0/926873177 >> 172.21.15.55:0/926873177 conn(0x7f4644068e20 msgr2=0x7f46440cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.659+0000 7f464899d700 1 -- 172.21.15.55:0/926873177 shutdown_connections 2023-10-06T05:07:20.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.659+0000 7f464899d700 1 -- 172.21.15.55:0/926873177 wait complete. 2023-10-06T05:07:20.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.660+0000 7f464899d700 1 Processor -- start 2023-10-06T05:07:20.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.661+0000 7f464899d700 1 -- start start 2023-10-06T05:07:20.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.661+0000 7f464899d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4644153480 0x7f4644147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:20.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.661+0000 7f464899d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4644153850 0x7f4644149a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:20.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f464899d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4644159bf0 0x7f4644149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:20.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f464899d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464414a490 con 0x7f4644159bf0 2023-10-06T05:07:20.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f464899d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4644169d90 con 0x7f4644153480 2023-10-06T05:07:20.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f464899d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46441560a0 con 0x7f4644153850 2023-10-06T05:07:20.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f4643fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4644159bf0 0x7f4644149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:20.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f4643fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4644159bf0 0x7f4644149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45678/0 (socket says 172.21.15.55:45678) 2023-10-06T05:07:20.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f4643fff700 1 -- 172.21.15.55:0/540697315 learned_addr learned my addr 172.21.15.55:0/540697315 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:20.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f46437fe700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4644153480 0x7f4644147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:20.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f4642ffd700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4644153850 0x7f4644149a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:20.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f4643fff700 1 -- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4644153850 msgr2=0x7f4644149a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:20.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f4643fff700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4644153850 0x7f4644149a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f4643fff700 1 -- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4644153480 msgr2=0x7f4644147ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:20.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f4643fff700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4644153480 0x7f4644147ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:20.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f4643fff700 1 -- 172.21.15.55:0/540697315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634009420 con 0x7f4644159bf0 2023-10-06T05:07:20.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.662+0000 7f4642ffd700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4644153850 0x7f4644149a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:20.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.663+0000 7f4643fff700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4644159bf0 0x7f4644149f50 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f463c00cc20 tx=0x7f463c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:20.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.663+0000 7f462b7fe700 1 -- 172.21.15.55:0/540697315 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c004bb0 con 0x7f4644159bf0 2023-10-06T05:07:20.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.663+0000 7f462b7fe700 1 -- 172.21.15.55:0/540697315 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463c004d10 con 0x7f4644159bf0 2023-10-06T05:07:20.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.663+0000 7f462b7fe700 1 -- 172.21.15.55:0/540697315 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c00ab20 con 0x7f4644159bf0 2023-10-06T05:07:20.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.663+0000 7f464899d700 1 -- 172.21.15.55:0/540697315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4644156380 con 0x7f4644159bf0 2023-10-06T05:07:20.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.663+0000 7f464899d700 1 -- 172.21.15.55:0/540697315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4644156940 con 0x7f4644159bf0 2023-10-06T05:07:20.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.664+0000 7f464899d700 1 -- 172.21.15.55:0/540697315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464404c3c0 con 0x7f4644159bf0 2023-10-06T05:07:20.670 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.665+0000 7f462b7fe700 1 -- 172.21.15.55:0/540697315 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f463c0070f0 con 0x7f4644159bf0 2023-10-06T05:07:20.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.666+0000 7f462b7fe700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4630066f30 0x7f46300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:20.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.666+0000 7f462b7fe700 1 -- 172.21.15.55:0/540697315 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5654+0+0 (secure 0 0 0) 0x7f463c084290 con 0x7f4644159bf0 2023-10-06T05:07:20.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.666+0000 7f46437fe700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4630066f30 0x7f46300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:20.671 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.667+0000 7f46437fe700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4630066f30 0x7f46300693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f46340097b0 tx=0x7f4634017000 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:20.673 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.670+0000 7f462b7fe700 1 -- 172.21.15.55:0/540697315 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f463c0515b0 con 0x7f4644159bf0 2023-10-06T05:07:20.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:20.871+0000 7f464899d700 1 -- 172.21.15.55:0/540697315 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f46440afe20 con 0x7f4644159bf0 2023-10-06T05:07:21.219 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.216+0000 7f462b7fe700 1 -- 172.21.15.55:0/540697315 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v204) v1 ==== 138+0+0 (secure 0 0 0) 0x7f463c0180b0 con 0x7f4644159bf0 2023-10-06T05:07:21.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.222+0000 7f464899d700 1 -- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4630066f30 msgr2=0x7f46300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:21.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.222+0000 7f464899d700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4630066f30 0x7f46300693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f46340097b0 tx=0x7f4634017000 comp rx=0 tx=0).stop 2023-10-06T05:07:21.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.222+0000 7f464899d700 1 -- 172.21.15.55:0/540697315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4644159bf0 msgr2=0x7f4644149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:21.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.222+0000 7f464899d700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4644159bf0 0x7f4644149f50 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f463c00cc20 tx=0x7f463c0019b0 comp rx=0 tx=0).stop 2023-10-06T05:07:21.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.224+0000 7f464899d700 1 -- 172.21.15.55:0/540697315 shutdown_connections 2023-10-06T05:07:21.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.224+0000 7f464899d700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4630066f30 0x7f46300693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:21.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.224+0000 7f464899d700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4644159bf0 0x7f4644149f50 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:21.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.224+0000 7f464899d700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4644153480 0x7f4644147ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:21.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.224+0000 7f464899d700 1 --2- 172.21.15.55:0/540697315 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4644153850 0x7f4644149a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:21.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.224+0000 7f464899d700 1 -- 172.21.15.55:0/540697315 >> 172.21.15.55:0/540697315 conn(0x7f4644068e20 msgr2=0x7f46441580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:21.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.224+0000 7f464899d700 1 -- 172.21.15.55:0/540697315 shutdown_connections 2023-10-06T05:07:21.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.224+0000 7f464899d700 1 -- 172.21.15.55:0/540697315 wait complete. 2023-10-06T05:07:21.229 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' created 2023-10-06T05:07:21.247 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T05:07:21.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.444+0000 7efede19d700 1 Processor -- start 2023-10-06T05:07:21.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.444+0000 7efede19d700 1 -- start start 2023-10-06T05:07:21.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.446+0000 7efede19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7efed81563b0 0x7efed80ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:21.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.446+0000 7efede19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efed80ae9c0 0x7efed80cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:21.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.446+0000 7efede19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efed80cbe50 con 0x7efed8156780 2023-10-06T05:07:21.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.446+0000 7efede19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed80cb370 con 0x7efed80ae9c0 2023-10-06T05:07:21.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.446+0000 7efede19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed80cb650 con 0x7efed81563b0 2023-10-06T05:07:21.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.446+0000 7efedd99c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efed80ae9c0 0x7efed80cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:21.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.446+0000 7efedc99a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7efed8156780 0x7efed80ae2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36638/0 (socket says 172.21.15.55:36638) 2023-10-06T05:07:21.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.446+0000 7efedc99a700 1 -- 172.21.15.55:0/1024064802 learned_addr learned my addr 172.21.15.55:0/1024064802 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:21.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.446+0000 7efedd19b700 1 --2- 172.21.15.55:0/1024064802 >> v2:172.21.15.163:3300/0 conn(0x7efed81563b0 0x7efed80ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:21.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.447+0000 7efedd99c700 1 -- 172.21.15.55:0/1024064802 >> v2:172.21.15.163:3300/0 conn(0x7efed81563b0 msgr2=0x7efed80ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:21.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.447+0000 7efedd99c700 1 --2- 172.21.15.55:0/1024064802 >> v2:172.21.15.163:3300/0 conn(0x7efed81563b0 0x7efed80ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:21.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.447+0000 7efedd99c700 1 -- 172.21.15.55:0/1024064802 >> v1:172.21.15.55:6789/0 conn(0x7efed8156780 legacy=0x7efed80ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:21.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.447+0000 7efedd99c700 1 -- 172.21.15.55:0/1024064802 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed80c6860 con 0x7efed80ae9c0 2023-10-06T05:07:21.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.447+0000 7efedd19b700 1 --2- 172.21.15.55:0/1024064802 >> v2:172.21.15.163:3300/0 conn(0x7efed81563b0 0x7efed80ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:21.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.447+0000 7efecffff700 1 -- 172.21.15.55:0/1024064802 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770567969 0 0) 0x7efed80cbe50 con 0x7efed8156780 2023-10-06T05:07:21.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.447+0000 7efedd99c700 1 --2- 172.21.15.55:0/1024064802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efed80ae9c0 0x7efed80cad10 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7efed8071150 tx=0x7efed0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebc44d3d9a51485 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:21.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.447+0000 7efecffff700 1 -- 172.21.15.55:0/1024064802 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed000e070 con 0x7efed80ae9c0 2023-10-06T05:07:21.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.447+0000 7efecffff700 1 -- 172.21.15.55:0/1024064802 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed000bc30 con 0x7efed80ae9c0 2023-10-06T05:07:21.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.447+0000 7efecffff700 1 -- 172.21.15.55:0/1024064802 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0012430 con 0x7efed80ae9c0 2023-10-06T05:07:21.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.448+0000 7efede19d700 1 -- 172.21.15.55:0/1024064802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efed80ae9c0 msgr2=0x7efed80cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:21.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.448+0000 7efede19d700 1 --2- 172.21.15.55:0/1024064802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efed80ae9c0 0x7efed80cad10 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7efed8071150 tx=0x7efed0009780 comp rx=0 tx=0).stop 2023-10-06T05:07:21.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.448+0000 7efede19d700 1 -- 172.21.15.55:0/1024064802 shutdown_connections 2023-10-06T05:07:21.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.448+0000 7efede19d700 1 --2- 172.21.15.55:0/1024064802 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efed80ae9c0 0x7efed80cad10 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:21.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.448+0000 7efede19d700 1 --2- 172.21.15.55:0/1024064802 >> v2:172.21.15.163:3300/0 conn(0x7efed81563b0 0x7efed80ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:21.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.448+0000 7efede19d700 1 -- 172.21.15.55:0/1024064802 >> 172.21.15.55:0/1024064802 conn(0x7efed8068e20 msgr2=0x7efed80cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:21.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.448+0000 7efede19d700 1 -- 172.21.15.55:0/1024064802 shutdown_connections 2023-10-06T05:07:21.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.448+0000 7efede19d700 1 -- 172.21.15.55:0/1024064802 wait complete. 2023-10-06T05:07:21.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.449+0000 7efede19d700 1 Processor -- start 2023-10-06T05:07:21.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.449+0000 7efede19d700 1 -- start start 2023-10-06T05:07:21.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.450+0000 7efede19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efed80ae9c0 0x7efed80c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:21.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.450+0000 7efede19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed81563b0 0x7efed8166530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:21.461 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.450+0000 7efede19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efed8156780 0x7efed8166a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:21.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.450+0000 7efede19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed813ff50 con 0x7efed8156780 2023-10-06T05:07:21.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.450+0000 7efede19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed8166fb0 con 0x7efed80ae9c0 2023-10-06T05:07:21.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.450+0000 7efede19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed813f620 con 0x7efed81563b0 2023-10-06T05:07:21.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.451+0000 7efedd99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efed8156780 0x7efed8166a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:21.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.451+0000 7efedd99c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efed8156780 0x7efed8166a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45692/0 (socket says 172.21.15.55:45692) 2023-10-06T05:07:21.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.451+0000 7efedd99c700 1 -- 172.21.15.55:0/183241350 learned_addr learned my addr 172.21.15.55:0/183241350 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:21.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.451+0000 7efedd19b700 1 --2- 172.21.15.55:0/183241350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efed80ae9c0 0x7efed80c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:21.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.451+0000 7efedc99a700 1 --2- 172.21.15.55:0/183241350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed81563b0 0x7efed8166530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:21.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.451+0000 7efedd99c700 1 -- 172.21.15.55:0/183241350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed81563b0 msgr2=0x7efed8166530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:21.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.451+0000 7efedd99c700 1 --2- 172.21.15.55:0/183241350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed81563b0 0x7efed8166530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:21.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.451+0000 7efedd99c700 1 -- 172.21.15.55:0/183241350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efed80ae9c0 msgr2=0x7efed80c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:21.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.451+0000 7efedd99c700 1 --2- 172.21.15.55:0/183241350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efed80ae9c0 0x7efed80c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:21.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.451+0000 7efedd99c700 1 -- 172.21.15.55:0/183241350 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed0009a20 con 0x7efed8156780 2023-10-06T05:07:21.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.451+0000 7efedd99c700 1 --2- 172.21.15.55:0/183241350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efed8156780 0x7efed8166a70 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7efed8069370 tx=0x7efed0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:21.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.452+0000 7efecd7fa700 1 -- 172.21.15.55:0/183241350 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0027470 con 0x7efed8156780 2023-10-06T05:07:21.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.452+0000 7efecd7fa700 1 -- 172.21.15.55:0/183241350 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed00275d0 con 0x7efed8156780 2023-10-06T05:07:21.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.452+0000 7efecd7fa700 1 -- 172.21.15.55:0/183241350 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed0027990 con 0x7efed8156780 2023-10-06T05:07:21.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.452+0000 7efede19d700 1 -- 172.21.15.55:0/183241350 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed813f900 con 0x7efed8156780 2023-10-06T05:07:21.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.452+0000 7efede19d700 1 -- 172.21.15.55:0/183241350 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed80c79d0 con 0x7efed8156780 2023-10-06T05:07:21.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.453+0000 7efede19d700 1 -- 172.21.15.55:0/183241350 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed8140100 con 0x7efed8156780 2023-10-06T05:07:21.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.454+0000 7efecd7fa700 1 -- 172.21.15.55:0/183241350 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efed0024070 con 0x7efed8156780 2023-10-06T05:07:21.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.455+0000 7efecd7fa700 1 --2- 172.21.15.55:0/183241350 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efebc066f30 0x7efebc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:21.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.455+0000 7efecd7fa700 1 -- 172.21.15.55:0/183241350 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6024+0+0 (secure 0 0 0) 0x7efed0087080 con 0x7efed8156780 2023-10-06T05:07:21.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.456+0000 7efedd19b700 1 --2- 172.21.15.55:0/183241350 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efebc066f30 0x7efebc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:21.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.456+0000 7efedd19b700 1 --2- 172.21.15.55:0/183241350 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efebc066f30 0x7efebc0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efed80c64e0 tx=0x7efed4008040 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:21.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.459+0000 7efecd7fa700 1 -- 172.21.15.55:0/183241350 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efed0054190 con 0x7efed8156780 2023-10-06T05:07:21.664 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:21.661+0000 7efede19d700 1 -- 172.21.15.55:0/183241350 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efed804c3c0 con 0x7efed8156780 2023-10-06T05:07:22.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.226+0000 7efecd7fa700 1 -- 172.21.15.55:0/183241350 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v205) v1 ==== 170+0+0 (secure 0 0 0) 0x7efed00577b0 con 0x7efed8156780 2023-10-06T05:07:22.236 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.233+0000 7efede19d700 1 -- 172.21.15.55:0/183241350 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efebc066f30 msgr2=0x7efebc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:22.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.233+0000 7efede19d700 1 --2- 172.21.15.55:0/183241350 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efebc066f30 0x7efebc0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efed80c64e0 tx=0x7efed4008040 comp rx=0 tx=0).stop 2023-10-06T05:07:22.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.233+0000 7efede19d700 1 -- 172.21.15.55:0/183241350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efed8156780 msgr2=0x7efed8166a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:22.237 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.233+0000 7efede19d700 1 --2- 172.21.15.55:0/183241350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efed8156780 0x7efed8166a70 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7efed8069370 tx=0x7efed0019000 comp rx=0 tx=0).stop 2023-10-06T05:07:22.239 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.236+0000 7efede19d700 1 -- 172.21.15.55:0/183241350 shutdown_connections 2023-10-06T05:07:22.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.236+0000 7efede19d700 1 --2- 172.21.15.55:0/183241350 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efebc066f30 0x7efebc0693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:22.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.236+0000 7efede19d700 1 --2- 172.21.15.55:0/183241350 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efed8156780 0x7efed8166a70 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:22.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.236+0000 7efede19d700 1 --2- 172.21.15.55:0/183241350 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efed80ae9c0 0x7efed80c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:22.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.236+0000 7efede19d700 1 --2- 172.21.15.55:0/183241350 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efed81563b0 0x7efed8166530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:22.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.236+0000 7efede19d700 1 -- 172.21.15.55:0/183241350 >> 172.21.15.55:0/183241350 conn(0x7efed8068e20 msgr2=0x7efed8154db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:22.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.241+0000 7efede19d700 1 -- 172.21.15.55:0/183241350 shutdown_connections 2023-10-06T05:07:22.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.244+0000 7efede19d700 1 -- 172.21.15.55:0/183241350 wait complete. 2023-10-06T05:07:22.252 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' created 2023-10-06T05:07:22.270 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T05:07:22.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.459+0000 7fdbdf59e700 1 Processor -- start 2023-10-06T05:07:22.462 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.459+0000 7fdbdf59e700 1 -- start start 2023-10-06T05:07:22.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.460+0000 7fdbdf59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdbe00b16d0 0x7fdbe00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:22.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.460+0000 7fdbdf59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbe00b33b0 0x7fdbe00b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:22.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.460+0000 7fdbdf59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe00b2080 con 0x7fdbe00b4e70 2023-10-06T05:07:22.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.460+0000 7fdbdf59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe00cfb00 con 0x7fdbe00b16d0 2023-10-06T05:07:22.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.460+0000 7fdbdf59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00b5c40 con 0x7fdbe00b33b0 2023-10-06T05:07:22.463 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.460+0000 7fdbde59c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fdbe00b4e70 0x7fdbe00ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:36652/0 (socket says 172.21.15.55:36652) 2023-10-06T05:07:22.464 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.460+0000 7fdbde59c700 1 -- 172.21.15.55:0/1025442405 learned_addr learned my addr 172.21.15.55:0/1025442405 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:22.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbddd9b700 1 --2- 172.21.15.55:0/1025442405 >> v2:172.21.15.163:3300/0 conn(0x7fdbe00b16d0 0x7fdbe00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:22.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 -- 172.21.15.55:0/1025442405 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057736554 0 0) 0x7fdbe00b2080 con 0x7fdbe00b4e70 2023-10-06T05:07:22.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbded9d700 1 --2- 172.21.15.55:0/1025442405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbe00b33b0 0x7fdbe00b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:22.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 -- 172.21.15.55:0/1025442405 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbc8003570 con 0x7fdbe00b4e70 2023-10-06T05:07:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 -- 172.21.15.55:0/1025442405 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3675484123 0 0) 0x7fdbc8003570 con 0x7fdbe00b4e70 2023-10-06T05:07:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 -- 172.21.15.55:0/1025442405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbe00b33b0 msgr2=0x7fdbe00b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 --2- 172.21.15.55:0/1025442405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbe00b33b0 0x7fdbe00b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 -- 172.21.15.55:0/1025442405 >> v2:172.21.15.163:3300/0 conn(0x7fdbe00b16d0 msgr2=0x7fdbe00b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 --2- 172.21.15.55:0/1025442405 >> v2:172.21.15.163:3300/0 conn(0x7fdbe00b16d0 0x7fdbe00b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 -- 172.21.15.55:0/1025442405 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe00b5f20 con 0x7fdbe00b4e70 2023-10-06T05:07:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 -- 172.21.15.55:0/1025442405 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7fdbd0001470 con 0x7fdbe00b4e70 2023-10-06T05:07:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 -- 172.21.15.55:0/1025442405 >> v1:172.21.15.55:6789/0 conn(0x7fdbe00b4e70 legacy=0x7fdbe00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 --2- 172.21.15.55:0/1025442405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbc8003ae0 0x7fdbc8003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 --2- 172.21.15.55:0/1025442405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbe00b33b0 0x7fdbc8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:22.467 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 --2- 172.21.15.55:0/1025442405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbc8004d50 0x7fdbc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:22.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 -- 172.21.15.55:0/1025442405 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00cfb00 con 0x7fdbc8004d50 2023-10-06T05:07:22.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 -- 172.21.15.55:0/1025442405 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00b5c40 con 0x7fdbc8003ae0 2023-10-06T05:07:22.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.461+0000 7fdbdd59a700 1 -- 172.21.15.55:0/1025442405 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc800bbc0 con 0x7fdbe00b33b0 2023-10-06T05:07:22.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.462+0000 7fdbddd9b700 1 --2- 172.21.15.55:0/1025442405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbe00b33b0 0x7fdbc8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:22.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.462+0000 7fdbde59c700 1 --2- 172.21.15.55:0/1025442405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbc8003ae0 0x7fdbc8003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:22.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.462+0000 7fdbded9d700 1 --2- 172.21.15.55:0/1025442405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbc8004d50 0x7fdbc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:22.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.462+0000 7fdbddd9b700 1 -- 172.21.15.55:0/1025442405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbc8003ae0 msgr2=0x7fdbc8003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:22.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.462+0000 7fdbddd9b700 1 --2- 172.21.15.55:0/1025442405 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbc8003ae0 0x7fdbc8003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:22.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.462+0000 7fdbddd9b700 1 -- 172.21.15.55:0/1025442405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbc8004d50 msgr2=0x7fdbc80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:22.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.462+0000 7fdbddd9b700 1 --2- 172.21.15.55:0/1025442405 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbc8004d50 0x7fdbc80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:22.468 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.462+0000 7fdbddd9b700 1 -- 172.21.15.55:0/1025442405 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc8003570 con 0x7fdbe00b33b0 2023-10-06T05:07:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.462+0000 7fdbddd9b700 1 --2- 172.21.15.55:0/1025442405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbe00b33b0 0x7fdbc8004810 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009c00 tx=0x7fdbd40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e9ca1377fc86ec8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.463+0000 7fdbdd59a700 1 -- 172.21.15.55:0/1025442405 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd400e710 con 0x7fdbe00b33b0 2023-10-06T05:07:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.463+0000 7fdbdd59a700 1 -- 172.21.15.55:0/1025442405 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd400e870 con 0x7fdbe00b33b0 2023-10-06T05:07:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.463+0000 7fdbdd59a700 1 -- 172.21.15.55:0/1025442405 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd400ec30 con 0x7fdbe00b33b0 2023-10-06T05:07:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.463+0000 7fdbdf59e700 1 -- 172.21.15.55:0/1025442405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbe00b33b0 msgr2=0x7fdbc8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.463+0000 7fdbdf59e700 1 --2- 172.21.15.55:0/1025442405 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbe00b33b0 0x7fdbc8004810 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009c00 tx=0x7fdbd40091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.463+0000 7fdbded9d700 1 -- 172.21.15.55:0/1025442405 reap_dead start 2023-10-06T05:07:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.463+0000 7fdbdf59e700 1 -- 172.21.15.55:0/1025442405 shutdown_connections 2023-10-06T05:07:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.463+0000 7fdbdf59e700 1 -- 172.21.15.55:0/1025442405 >> 172.21.15.55:0/1025442405 conn(0x7fdbe00d10c0 msgr2=0x7fdbe00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.463+0000 7fdbdf59e700 1 -- 172.21.15.55:0/1025442405 shutdown_connections 2023-10-06T05:07:22.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.463+0000 7fdbdf59e700 1 -- 172.21.15.55:0/1025442405 wait complete. 2023-10-06T05:07:22.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.465+0000 7fdbdf59e700 1 Processor -- start 2023-10-06T05:07:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.465+0000 7fdbdf59e700 1 -- start start 2023-10-06T05:07:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.466+0000 7fdbdf59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbe00b33b0 0x7fdbe0153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.466+0000 7fdbdf59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbe00ca320 0x7fdbe00ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.466+0000 7fdbdf59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbe00c81f0 0x7fdbe00c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.466+0000 7fdbdf59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe00c8bd0 con 0x7fdbe00c81f0 2023-10-06T05:07:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.466+0000 7fdbdf59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe0154a00 con 0x7fdbe00ca320 2023-10-06T05:07:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.466+0000 7fdbdf59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe0155580 con 0x7fdbe00b33b0 2023-10-06T05:07:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.466+0000 7fdbded9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbe00c81f0 0x7fdbe00c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.466+0000 7fdbded9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbe00c81f0 0x7fdbe00c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:45708/0 (socket says 172.21.15.55:45708) 2023-10-06T05:07:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.466+0000 7fdbded9d700 1 -- 172.21.15.55:0/572135885 learned_addr learned my addr 172.21.15.55:0/572135885 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:22.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.466+0000 7fdbde59c700 1 --2- 172.21.15.55:0/572135885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbe00b33b0 0x7fdbe0153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.466+0000 7fdbddd9b700 1 --2- 172.21.15.55:0/572135885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbe00ca320 0x7fdbe00ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.467+0000 7fdbded9d700 1 -- 172.21.15.55:0/572135885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbe00b33b0 msgr2=0x7fdbe0153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.467+0000 7fdbded9d700 1 --2- 172.21.15.55:0/572135885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbe00b33b0 0x7fdbe0153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.467+0000 7fdbded9d700 1 -- 172.21.15.55:0/572135885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbe00ca320 msgr2=0x7fdbe00ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.467+0000 7fdbded9d700 1 --2- 172.21.15.55:0/572135885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbe00ca320 0x7fdbe00ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.467+0000 7fdbded9d700 1 -- 172.21.15.55:0/572135885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe0154c50 con 0x7fdbe00c81f0 2023-10-06T05:07:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.467+0000 7fdbded9d700 1 --2- 172.21.15.55:0/572135885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbe00c81f0 0x7fdbe00c8690 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80094d0 tx=0x7fdbd800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.467+0000 7fdbce7fc700 1 -- 172.21.15.55:0/572135885 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800e040 con 0x7fdbe00c81f0 2023-10-06T05:07:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.467+0000 7fdbce7fc700 1 -- 172.21.15.55:0/572135885 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd800f040 con 0x7fdbe00c81f0 2023-10-06T05:07:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.467+0000 7fdbce7fc700 1 -- 172.21.15.55:0/572135885 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800fd40 con 0x7fdbe00c81f0 2023-10-06T05:07:22.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.467+0000 7fdbdf59e700 1 -- 172.21.15.55:0/572135885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe0154f30 con 0x7fdbe00c81f0 2023-10-06T05:07:22.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.467+0000 7fdbdf59e700 1 -- 172.21.15.55:0/572135885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe0153680 con 0x7fdbe00c81f0 2023-10-06T05:07:22.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.468+0000 7fdbdf59e700 1 -- 172.21.15.55:0/572135885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe0088990 con 0x7fdbe00c81f0 2023-10-06T05:07:22.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.470+0000 7fdbce7fc700 1 -- 172.21.15.55:0/572135885 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdbd8015640 con 0x7fdbe00c81f0 2023-10-06T05:07:22.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.471+0000 7fdbce7fc700 1 --2- 172.21.15.55:0/572135885 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdbc0066f30 0x7fdbc00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:22.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.471+0000 7fdbce7fc700 1 -- 172.21.15.55:0/572135885 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6406+0+0 (secure 0 0 0) 0x7fdbd8084df0 con 0x7fdbe00c81f0 2023-10-06T05:07:22.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.471+0000 7fdbde59c700 1 --2- 172.21.15.55:0/572135885 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdbc0066f30 0x7fdbc00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:22.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.472+0000 7fdbde59c700 1 --2- 172.21.15.55:0/572135885 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdbc0066f30 0x7fdbc00693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0001120 tx=0x7fdbd0001dc0 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:22.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.475+0000 7fdbce7fc700 1 -- 172.21.15.55:0/572135885 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbd8051da0 con 0x7fdbe00c81f0 2023-10-06T05:07:22.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:22.721+0000 7fdbdf59e700 1 -- 172.21.15.55:0/572135885 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdbe01556c0 con 0x7fdbe00c81f0 2023-10-06T05:07:23.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.226+0000 7fdbce7fc700 1 -- 172.21.15.55:0/572135885 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v101) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdbd800fea0 con 0x7fdbe00c81f0 2023-10-06T05:07:23.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.230+0000 7fdbdf59e700 1 -- 172.21.15.55:0/572135885 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdbc0066f30 msgr2=0x7fdbc00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.230+0000 7fdbdf59e700 1 --2- 172.21.15.55:0/572135885 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdbc0066f30 0x7fdbc00693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0001120 tx=0x7fdbd0001dc0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.230+0000 7fdbdf59e700 1 -- 172.21.15.55:0/572135885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbe00c81f0 msgr2=0x7fdbe00c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.230+0000 7fdbdf59e700 1 --2- 172.21.15.55:0/572135885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbe00c81f0 0x7fdbe00c8690 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80094d0 tx=0x7fdbd800b040 comp rx=0 tx=0).stop 2023-10-06T05:07:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.230+0000 7fdbdf59e700 1 -- 172.21.15.55:0/572135885 shutdown_connections 2023-10-06T05:07:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.230+0000 7fdbdf59e700 1 --2- 172.21.15.55:0/572135885 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdbc0066f30 0x7fdbc00693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.230+0000 7fdbdf59e700 1 --2- 172.21.15.55:0/572135885 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdbe00c81f0 0x7fdbe00c8690 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.230+0000 7fdbdf59e700 1 --2- 172.21.15.55:0/572135885 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdbe00b33b0 0x7fdbe0153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.230+0000 7fdbdf59e700 1 --2- 172.21.15.55:0/572135885 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdbe00ca320 0x7fdbe00ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.233 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.230+0000 7fdbdf59e700 1 -- 172.21.15.55:0/572135885 >> 172.21.15.55:0/572135885 conn(0x7fdbe00d10c0 msgr2=0x7fdbe0069c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:23.234 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.231+0000 7fdbdf59e700 1 -- 172.21.15.55:0/572135885 shutdown_connections 2023-10-06T05:07:23.235 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.232+0000 7fdbdf59e700 1 -- 172.21.15.55:0/572135885 wait complete. 2023-10-06T05:07:23.239 INFO:teuthology.orchestra.run.smithi055.stderr:new fs with metadata pool 25 and data pool 26 2023-10-06T05:07:23.257 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:07:23.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.443+0000 7ff58259d700 1 Processor -- start 2023-10-06T05:07:23.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.443+0000 7ff58259d700 1 -- start start 2023-10-06T05:07:23.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff58259d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff57c06d950 0x7ff57c0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:23.449 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff58259d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff57c06a0a0 0x7ff57c0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:23.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff58259d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff57c06ae70 con 0x7ff57c06dd20 2023-10-06T05:07:23.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff58259d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff57c06a540 con 0x7ff57c06d950 2023-10-06T05:07:23.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff58259d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff57c06a820 con 0x7ff57c06a0a0 2023-10-06T05:07:23.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff58159b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff57c06d950 0x7ff57c0c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:23.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff58159b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7ff57c06d950 0x7ff57c0c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43734/0 (socket says 172.21.15.55:43734) 2023-10-06T05:07:23.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff58159b700 1 -- 172.21.15.55:0/1604815353 learned_addr learned my addr 172.21.15.55:0/1604815353 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:23.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff580d9a700 1 --2- 172.21.15.55:0/1604815353 >> v2:172.21.15.163:3300/0 conn(0x7ff57c06a0a0 0x7ff57c0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:23.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff58159b700 1 -- 172.21.15.55:0/1604815353 >> v2:172.21.15.163:3300/0 conn(0x7ff57c06a0a0 msgr2=0x7ff57c0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:23.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff58159b700 1 --2- 172.21.15.55:0/1604815353 >> v2:172.21.15.163:3300/0 conn(0x7ff57c06a0a0 0x7ff57c0c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.451 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff58159b700 1 -- 172.21.15.55:0/1604815353 >> v1:172.21.15.163:6789/0 conn(0x7ff57c06dd20 legacy=0x7ff57c0c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:23.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff58159b700 1 -- 172.21.15.55:0/1604815353 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff57c06ab00 con 0x7ff57c06d950 2023-10-06T05:07:23.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff580d9a700 1 --2- 172.21.15.55:0/1604815353 >> v2:172.21.15.163:3300/0 conn(0x7ff57c06a0a0 0x7ff57c0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:23.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff58159b700 1 --2- 172.21.15.55:0/1604815353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff57c06d950 0x7ff57c0c8350 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7ff57c06b0c0 tx=0x7ff578009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a55c1cb3f033c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:23.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff573fff700 1 -- 172.21.15.55:0/1604815353 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57800b860 con 0x7ff57c06d950 2023-10-06T05:07:23.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff573fff700 1 -- 172.21.15.55:0/1604815353 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57800b9c0 con 0x7ff57c06d950 2023-10-06T05:07:23.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.445+0000 7ff573fff700 1 -- 172.21.15.55:0/1604815353 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57800bd60 con 0x7ff57c06d950 2023-10-06T05:07:23.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.446+0000 7ff58259d700 1 -- 172.21.15.55:0/1604815353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff57c06d950 msgr2=0x7ff57c0c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:23.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.446+0000 7ff58259d700 1 --2- 172.21.15.55:0/1604815353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff57c06d950 0x7ff57c0c8350 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7ff57c06b0c0 tx=0x7ff578009250 comp rx=0 tx=0).stop 2023-10-06T05:07:23.453 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.446+0000 7ff58259d700 1 -- 172.21.15.55:0/1604815353 shutdown_connections 2023-10-06T05:07:23.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.446+0000 7ff58259d700 1 --2- 172.21.15.55:0/1604815353 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff57c06d950 0x7ff57c0c8350 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.446+0000 7ff58259d700 1 --2- 172.21.15.55:0/1604815353 >> v2:172.21.15.163:3300/0 conn(0x7ff57c06a0a0 0x7ff57c0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.446+0000 7ff58259d700 1 -- 172.21.15.55:0/1604815353 >> 172.21.15.55:0/1604815353 conn(0x7ff57c0cc1f0 msgr2=0x7ff57c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:23.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.446+0000 7ff58259d700 1 -- 172.21.15.55:0/1604815353 shutdown_connections 2023-10-06T05:07:23.454 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.446+0000 7ff58259d700 1 -- 172.21.15.55:0/1604815353 wait complete. 2023-10-06T05:07:23.456 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.447+0000 7ff58259d700 1 Processor -- start 2023-10-06T05:07:23.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.447+0000 7ff58259d700 1 -- start start 2023-10-06T05:07:23.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.448+0000 7ff58259d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff57c06a0a0 0x7ff57c0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:23.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.448+0000 7ff58259d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff57c06d950 0x7ff57c151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:23.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.448+0000 7ff58259d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff57c06dd20 0x7ff57c0c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:23.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.448+0000 7ff58259d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c0cbc00 con 0x7ff57c06a0a0 2023-10-06T05:07:23.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.448+0000 7ff58259d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c0c4050 con 0x7ff57c06d950 2023-10-06T05:07:23.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.448+0000 7ff58259d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff57c0c3440 con 0x7ff57c06dd20 2023-10-06T05:07:23.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.448+0000 7ff58159b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff57c06a0a0 0x7ff57c0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:23.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.448+0000 7ff58159b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff57c06a0a0 0x7ff57c0c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43742/0 (socket says 172.21.15.55:43742) 2023-10-06T05:07:23.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.448+0000 7ff58159b700 1 -- 172.21.15.55:0/1240952920 learned_addr learned my addr 172.21.15.55:0/1240952920 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:23.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.449+0000 7ff580d9a700 1 --2- 172.21.15.55:0/1240952920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff57c06d950 0x7ff57c151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:23.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.449+0000 7ff581d9c700 1 --2- 172.21.15.55:0/1240952920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff57c06dd20 0x7ff57c0c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:23.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.449+0000 7ff58159b700 1 -- 172.21.15.55:0/1240952920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff57c06dd20 msgr2=0x7ff57c0c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:23.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.449+0000 7ff58159b700 1 --2- 172.21.15.55:0/1240952920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff57c06dd20 0x7ff57c0c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.449+0000 7ff58159b700 1 -- 172.21.15.55:0/1240952920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff57c06d950 msgr2=0x7ff57c151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:23.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.449+0000 7ff58159b700 1 --2- 172.21.15.55:0/1240952920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff57c06d950 0x7ff57c151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.449+0000 7ff58159b700 1 -- 172.21.15.55:0/1240952920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5780094f0 con 0x7ff57c06a0a0 2023-10-06T05:07:23.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.449+0000 7ff58159b700 1 --2- 172.21.15.55:0/1240952920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff57c06a0a0 0x7ff57c0c9dc0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7ff578009290 tx=0x7ff578017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:23.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.449+0000 7ff5717fa700 1 -- 172.21.15.55:0/1240952920 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff578026480 con 0x7ff57c06a0a0 2023-10-06T05:07:23.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.449+0000 7ff5717fa700 1 -- 172.21.15.55:0/1240952920 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57801f070 con 0x7ff57c06a0a0 2023-10-06T05:07:23.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.450+0000 7ff58259d700 1 -- 172.21.15.55:0/1240952920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff57c0c3720 con 0x7ff57c06a0a0 2023-10-06T05:07:23.458 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.450+0000 7ff5717fa700 1 -- 172.21.15.55:0/1240952920 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff578026850 con 0x7ff57c06a0a0 2023-10-06T05:07:23.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.450+0000 7ff58259d700 1 -- 172.21.15.55:0/1240952920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57c0c3ce0 con 0x7ff57c06a0a0 2023-10-06T05:07:23.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.451+0000 7ff58259d700 1 -- 172.21.15.55:0/1240952920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff57c04c3c0 con 0x7ff57c06a0a0 2023-10-06T05:07:23.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.451+0000 7ff5717fa700 1 -- 172.21.15.55:0/1240952920 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff57801a430 con 0x7ff57c06a0a0 2023-10-06T05:07:23.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.452+0000 7ff5717fa700 1 --2- 172.21.15.55:0/1240952920 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff560066f30 0x7ff5600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:23.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.452+0000 7ff5717fa700 1 -- 172.21.15.55:0/1240952920 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ff578021070 con 0x7ff57c06a0a0 2023-10-06T05:07:23.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.453+0000 7ff580d9a700 1 --2- 172.21.15.55:0/1240952920 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff560066f30 0x7ff5600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:23.459 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.455+0000 7ff580d9a700 1 --2- 172.21.15.55:0/1240952920 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff560066f30 0x7ff5600693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0cb750 tx=0x7ff56c008040 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:23.460 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.457+0000 7ff5717fa700 1 -- 172.21.15.55:0/1240952920 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff578053720 con 0x7ff57c06a0a0 2023-10-06T05:07:23.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.660+0000 7ff58259d700 1 -- 172.21.15.55:0/1240952920 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff57c0c62d0 con 0x7ff57c06a0a0 2023-10-06T05:07:23.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.661+0000 7ff5717fa700 1 -- 172.21.15.55:0/1240952920 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) v1 ==== 74+0+17627 (secure 0 0 0) 0x7ff578056d40 con 0x7ff57c06a0a0 2023-10-06T05:07:23.663 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:23.664 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":206,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:07:22.723830+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:07:20.916360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:07:21.706951+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:07:23.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.664+0000 7ff58259d700 1 -- 172.21.15.55:0/1240952920 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff560066f30 msgr2=0x7ff5600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:23.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.664+0000 7ff58259d700 1 --2- 172.21.15.55:0/1240952920 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff560066f30 0x7ff5600693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff57c0cb750 tx=0x7ff56c008040 comp rx=0 tx=0).stop 2023-10-06T05:07:23.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.664+0000 7ff58259d700 1 -- 172.21.15.55:0/1240952920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff57c06a0a0 msgr2=0x7ff57c0c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:23.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.664+0000 7ff58259d700 1 --2- 172.21.15.55:0/1240952920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff57c06a0a0 0x7ff57c0c9dc0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7ff578009290 tx=0x7ff578017000 comp rx=0 tx=0).stop 2023-10-06T05:07:23.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.664+0000 7ff58259d700 1 -- 172.21.15.55:0/1240952920 shutdown_connections 2023-10-06T05:07:23.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.664+0000 7ff58259d700 1 --2- 172.21.15.55:0/1240952920 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff560066f30 0x7ff5600693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.664+0000 7ff58259d700 1 --2- 172.21.15.55:0/1240952920 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff57c06dd20 0x7ff57c0c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.664+0000 7ff58259d700 1 --2- 172.21.15.55:0/1240952920 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff57c06a0a0 0x7ff57c0c9dc0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.664+0000 7ff58259d700 1 --2- 172.21.15.55:0/1240952920 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff57c06d950 0x7ff57c151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.664+0000 7ff58259d700 1 -- 172.21.15.55:0/1240952920 >> 172.21.15.55:0/1240952920 conn(0x7ff57c0cc1f0 msgr2=0x7ff57c0c1150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:23.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.664+0000 7ff58259d700 1 -- 172.21.15.55:0/1240952920 shutdown_connections 2023-10-06T05:07:23.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.664+0000 7ff58259d700 1 -- 172.21.15.55:0/1240952920 wait complete. 2023-10-06T05:07:23.684 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:07:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.870+0000 7f17f0f9d700 1 Processor -- start 2023-10-06T05:07:23.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.870+0000 7f17f0f9d700 1 -- start start 2023-10-06T05:07:23.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.871+0000 7f17f0f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f17ec159420 0x7f17ec0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:23.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.871+0000 7f17f0f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17ec1597f0 0x7f17ec0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:23.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.871+0000 7f17f0f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17ec0b11d0 con 0x7f17ec15d230 2023-10-06T05:07:23.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.871+0000 7f17f0f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17ec0b14b0 con 0x7f17ec159420 2023-10-06T05:07:23.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.871+0000 7f17f0f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec0b1790 con 0x7f17ec1597f0 2023-10-06T05:07:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.872+0000 7f17eb7fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f17ec159420 0x7f17ec0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.872+0000 7f17eaffd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17ec1597f0 0x7f17ec0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.872+0000 7f17ebfff700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f17ec15d230 0x7f17ec0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:59642/0 (socket says 172.21.15.55:59642) 2023-10-06T05:07:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.872+0000 7f17ebfff700 1 -- 172.21.15.55:0/2083039479 learned_addr learned my addr 172.21.15.55:0/2083039479 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:23.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.872+0000 7f17ea7fc700 1 -- 172.21.15.55:0/2083039479 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3932876759 0 0) 0x7f17ec0b11d0 con 0x7f17ec15d230 2023-10-06T05:07:23.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.872+0000 7f17ea7fc700 1 -- 172.21.15.55:0/2083039479 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17d4003570 con 0x7f17ec15d230 2023-10-06T05:07:23.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.873+0000 7f17eaffd700 1 -- 172.21.15.55:0/2083039479 >> v2:172.21.15.163:3300/0 conn(0x7f17ec159420 msgr2=0x7f17ec0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:23.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.873+0000 7f17eaffd700 1 --2- 172.21.15.55:0/2083039479 >> v2:172.21.15.163:3300/0 conn(0x7f17ec159420 0x7f17ec0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.873+0000 7f17eaffd700 1 -- 172.21.15.55:0/2083039479 >> v1:172.21.15.163:6789/0 conn(0x7f17ec15d230 legacy=0x7f17ec0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:23.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.873+0000 7f17eaffd700 1 -- 172.21.15.55:0/2083039479 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ec0b1a70 con 0x7f17ec1597f0 2023-10-06T05:07:23.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.873+0000 7f17eb7fe700 1 --2- 172.21.15.55:0/2083039479 >> v2:172.21.15.163:3300/0 conn(0x7f17ec159420 0x7f17ec0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:23.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.873+0000 7f17eaffd700 1 --2- 172.21.15.55:0/2083039479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17ec1597f0 0x7f17ec0b2270 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f17e00098e0 tx=0x7f17e00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a95e6a3845b04be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:23.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.873+0000 7f17ea7fc700 1 -- 172.21.15.55:0/2083039479 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0011630 con 0x7f17ec1597f0 2023-10-06T05:07:23.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.873+0000 7f17ea7fc700 1 -- 172.21.15.55:0/2083039479 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e0011790 con 0x7f17ec1597f0 2023-10-06T05:07:23.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.873+0000 7f17ea7fc700 1 -- 172.21.15.55:0/2083039479 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0011b30 con 0x7f17ec1597f0 2023-10-06T05:07:23.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.874+0000 7f17f0f9d700 1 -- 172.21.15.55:0/2083039479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17ec1597f0 msgr2=0x7f17ec0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.874+0000 7f17f0f9d700 1 --2- 172.21.15.55:0/2083039479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17ec1597f0 0x7f17ec0b2270 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f17e00098e0 tx=0x7f17e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.874+0000 7f17f0f9d700 1 -- 172.21.15.55:0/2083039479 shutdown_connections 2023-10-06T05:07:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.874+0000 7f17f0f9d700 1 --2- 172.21.15.55:0/2083039479 >> v2:172.21.15.163:3300/0 conn(0x7f17ec159420 0x7f17ec0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.874+0000 7f17f0f9d700 1 --2- 172.21.15.55:0/2083039479 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17ec1597f0 0x7f17ec0b2270 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.874+0000 7f17f0f9d700 1 -- 172.21.15.55:0/2083039479 >> 172.21.15.55:0/2083039479 conn(0x7f17ec0d0660 msgr2=0x7f17ec0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.874+0000 7f17f0f9d700 1 -- 172.21.15.55:0/2083039479 shutdown_connections 2023-10-06T05:07:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.874+0000 7f17f0f9d700 1 -- 172.21.15.55:0/2083039479 wait complete. 2023-10-06T05:07:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.875+0000 7f17f0f9d700 1 Processor -- start 2023-10-06T05:07:23.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.876+0000 7f17f0f9d700 1 -- start start 2023-10-06T05:07:23.880 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.876+0000 7f17f0f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17ec159420 0x7f17ec15e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:23.880 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.876+0000 7f17f0f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17ec1597f0 0x7f17ec0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:23.880 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.876+0000 7f17f0f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17ec15d230 0x7f17ec0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:23.880 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.876+0000 7f17f0f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec0c6ef0 con 0x7f17ec159420 2023-10-06T05:07:23.880 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.876+0000 7f17f0f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec15f560 con 0x7f17ec15d230 2023-10-06T05:07:23.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.876+0000 7f17f0f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec15e950 con 0x7f17ec1597f0 2023-10-06T05:07:23.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.877+0000 7f17eb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17ec159420 0x7f17ec15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:23.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.877+0000 7f17eb7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17ec159420 0x7f17ec15e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43746/0 (socket says 172.21.15.55:43746) 2023-10-06T05:07:23.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.877+0000 7f17eb7fe700 1 -- 172.21.15.55:0/1925739952 learned_addr learned my addr 172.21.15.55:0/1925739952 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:23.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.877+0000 7f17ebfff700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17ec15d230 0x7f17ec0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:23.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.877+0000 7f17eaffd700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17ec1597f0 0x7f17ec0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:23.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.877+0000 7f17eb7fe700 1 -- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17ec1597f0 msgr2=0x7f17ec0c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:23.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.877+0000 7f17eb7fe700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17ec1597f0 0x7f17ec0c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.877+0000 7f17eb7fe700 1 -- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17ec15d230 msgr2=0x7f17ec0c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:23.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.877+0000 7f17eb7fe700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17ec15d230 0x7f17ec0c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:23.881 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.877+0000 7f17eb7fe700 1 -- 172.21.15.55:0/1925739952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc009420 con 0x7f17ec159420 2023-10-06T05:07:23.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.877+0000 7f17ebfff700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17ec15d230 0x7f17ec0c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:23.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.877+0000 7f17eb7fe700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17ec159420 0x7f17ec15e370 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00ec80 tx=0x7f17dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:23.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.878+0000 7f17d37fe700 1 -- 172.21.15.55:0/1925739952 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc00c9d0 con 0x7f17ec159420 2023-10-06T05:07:23.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.878+0000 7f17d37fe700 1 -- 172.21.15.55:0/1925739952 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc00cb30 con 0x7f17ec159420 2023-10-06T05:07:23.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.878+0000 7f17f0f9d700 1 -- 172.21.15.55:0/1925739952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e0009420 con 0x7f17ec159420 2023-10-06T05:07:23.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.878+0000 7f17d37fe700 1 -- 172.21.15.55:0/1925739952 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc00ceb0 con 0x7f17ec159420 2023-10-06T05:07:23.882 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.878+0000 7f17f0f9d700 1 -- 172.21.15.55:0/1925739952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ec15ef10 con 0x7f17ec159420 2023-10-06T05:07:23.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.879+0000 7f17f0f9d700 1 -- 172.21.15.55:0/1925739952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17ec15f840 con 0x7f17ec159420 2023-10-06T05:07:23.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.880+0000 7f17d37fe700 1 -- 172.21.15.55:0/1925739952 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f17dc013680 con 0x7f17ec159420 2023-10-06T05:07:23.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.881+0000 7f17d37fe700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f17d8066f30 0x7f17d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:23.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.881+0000 7f17d37fe700 1 -- 172.21.15.55:0/1925739952 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f17dc084d40 con 0x7f17ec159420 2023-10-06T05:07:23.883 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.881+0000 7f17eaffd700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f17d8066f30 0x7f17d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:23.885 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.882+0000 7f17eaffd700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f17d8066f30 0x7f17d80693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f17e00091e0 tx=0x7f17e0017000 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:23.886 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:23.885+0000 7f17d37fe700 1 -- 172.21.15.55:0/1925739952 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17dc051bf0 con 0x7f17ec159420 2023-10-06T05:07:24.093 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.091+0000 7f17f0f9d700 1 -- 172.21.15.55:0/1925739952 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17ec0b1d50 con 0x7f17ec159420 2023-10-06T05:07:24.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.092+0000 7f17d37fe700 1 -- 172.21.15.55:0/1925739952 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v207) v1 ==== 74+0+17614 (secure 0 0 0) 0x7f17dc055210 con 0x7f17ec159420 2023-10-06T05:07:24.095 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:24.095 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":207,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:07:23.969436+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:07:20.916360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:07:21.706951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"207","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:07:24.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.095+0000 7f17f0f9d700 1 -- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f17d8066f30 msgr2=0x7f17d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:24.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.095+0000 7f17f0f9d700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f17d8066f30 0x7f17d80693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f17e00091e0 tx=0x7f17e0017000 comp rx=0 tx=0).stop 2023-10-06T05:07:24.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.095+0000 7f17f0f9d700 1 -- 172.21.15.55:0/1925739952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17ec159420 msgr2=0x7f17ec15e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:24.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.095+0000 7f17f0f9d700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17ec159420 0x7f17ec15e370 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00ec80 tx=0x7f17dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:24.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.095+0000 7f17f0f9d700 1 -- 172.21.15.55:0/1925739952 shutdown_connections 2023-10-06T05:07:24.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.095+0000 7f17f0f9d700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f17d8066f30 0x7f17d80693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:24.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.095+0000 7f17f0f9d700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f17ec15d230 0x7f17ec0c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:24.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.095+0000 7f17f0f9d700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f17ec159420 0x7f17ec15e370 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:24.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.095+0000 7f17f0f9d700 1 --2- 172.21.15.55:0/1925739952 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f17ec1597f0 0x7f17ec0c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:24.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.095+0000 7f17f0f9d700 1 -- 172.21.15.55:0/1925739952 >> 172.21.15.55:0/1925739952 conn(0x7f17ec0d0660 msgr2=0x7f17ec0af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:24.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.095+0000 7f17f0f9d700 1 -- 172.21.15.55:0/1925739952 shutdown_connections 2023-10-06T05:07:24.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.095+0000 7f17f0f9d700 1 -- 172.21.15.55:0/1925739952 wait complete. 2023-10-06T05:07:24.115 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T05:07:24.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.306+0000 7f616359e700 1 Processor -- start 2023-10-06T05:07:24.308 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.306+0000 7f616359e700 1 -- start start 2023-10-06T05:07:24.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.307+0000 7f616359e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f61640b4e70 0x7f61640ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:24.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.307+0000 7f616359e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f61640b16d0 0x7f61640b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:24.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.307+0000 7f616359e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61640b2080 con 0x7f616415d240 2023-10-06T05:07:24.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.307+0000 7f616359e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61640cfb00 con 0x7f61640b16d0 2023-10-06T05:07:24.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.307+0000 7f616359e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61640b5c40 con 0x7f61640b4e70 2023-10-06T05:07:24.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.307+0000 7f6161d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f61640b16d0 0x7f61640b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:24.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.307+0000 7f6161d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f61640b16d0 0x7f61640b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43750/0 (socket says 172.21.15.55:43750) 2023-10-06T05:07:24.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.307+0000 7f6161d9b700 1 -- 172.21.15.55:0/3889664909 learned_addr learned my addr 172.21.15.55:0/3889664909 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:24.310 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.308+0000 7f616259c700 1 --2- 172.21.15.55:0/3889664909 >> v2:172.21.15.163:3300/0 conn(0x7f61640b4e70 0x7f61640ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:24.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.308+0000 7f6161d9b700 1 -- 172.21.15.55:0/3889664909 >> v2:172.21.15.163:3300/0 conn(0x7f61640b4e70 msgr2=0x7f61640ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:24.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.308+0000 7f6161d9b700 1 --2- 172.21.15.55:0/3889664909 >> v2:172.21.15.163:3300/0 conn(0x7f61640b4e70 0x7f61640ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:24.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.308+0000 7f6161d9b700 1 -- 172.21.15.55:0/3889664909 >> v1:172.21.15.163:6789/0 conn(0x7f616415d240 legacy=0x7f616415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:24.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.308+0000 7f6161d9b700 1 -- 172.21.15.55:0/3889664909 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640b5f20 con 0x7f61640b16d0 2023-10-06T05:07:24.311 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.308+0000 7f6161d9b700 1 --2- 172.21.15.55:0/3889664909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61640b16d0 0x7f61640b1b40 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f6164064910 tx=0x7f6158009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3497a6244ddb6d4c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:24.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.308+0000 7f616159a700 1 -- 172.21.15.55:0/3889664909 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800b820 con 0x7f61640b16d0 2023-10-06T05:07:24.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.308+0000 7f616159a700 1 -- 172.21.15.55:0/3889664909 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615800b980 con 0x7f61640b16d0 2023-10-06T05:07:24.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.308+0000 7f616159a700 1 -- 172.21.15.55:0/3889664909 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800bd20 con 0x7f61640b16d0 2023-10-06T05:07:24.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.309+0000 7f616359e700 1 -- 172.21.15.55:0/3889664909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61640b16d0 msgr2=0x7f61640b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:24.312 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.309+0000 7f616359e700 1 --2- 172.21.15.55:0/3889664909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61640b16d0 0x7f61640b1b40 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f6164064910 tx=0x7f6158009250 comp rx=0 tx=0).stop 2023-10-06T05:07:24.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.309+0000 7f616359e700 1 -- 172.21.15.55:0/3889664909 shutdown_connections 2023-10-06T05:07:24.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.309+0000 7f616359e700 1 --2- 172.21.15.55:0/3889664909 >> v2:172.21.15.163:3300/0 conn(0x7f61640b4e70 0x7f61640ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:24.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.309+0000 7f616359e700 1 --2- 172.21.15.55:0/3889664909 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f61640b16d0 0x7f61640b1b40 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:24.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.309+0000 7f616359e700 1 -- 172.21.15.55:0/3889664909 >> 172.21.15.55:0/3889664909 conn(0x7f61640d10c0 msgr2=0x7f61640affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:24.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.309+0000 7f616359e700 1 -- 172.21.15.55:0/3889664909 shutdown_connections 2023-10-06T05:07:24.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.309+0000 7f616359e700 1 -- 172.21.15.55:0/3889664909 wait complete. 2023-10-06T05:07:24.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.310+0000 7f616359e700 1 Processor -- start 2023-10-06T05:07:24.313 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.311+0000 7f616359e700 1 -- start start 2023-10-06T05:07:24.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.311+0000 7f616359e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61640b16d0 0x7f616414c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:24.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.311+0000 7f616359e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61640b4e70 0x7f6164068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:24.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.312+0000 7f616359e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f616415d240 0x7f61640688d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:24.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.312+0000 7f616359e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61640c6e30 con 0x7f616415d240 2023-10-06T05:07:24.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.312+0000 7f616359e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61640c6500 con 0x7f61640b4e70 2023-10-06T05:07:24.315 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.312+0000 7f616359e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61640c67e0 con 0x7f61640b16d0 2023-10-06T05:07:24.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.312+0000 7f6162d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f616415d240 0x7f61640688d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:24.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.312+0000 7f6162d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f616415d240 0x7f61640688d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43754/0 (socket says 172.21.15.55:43754) 2023-10-06T05:07:24.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.312+0000 7f6162d9d700 1 -- 172.21.15.55:0/2710183786 learned_addr learned my addr 172.21.15.55:0/2710183786 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:24.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.312+0000 7f6161d9b700 1 --2- 172.21.15.55:0/2710183786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61640b4e70 0x7f6164068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:24.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.312+0000 7f616259c700 1 --2- 172.21.15.55:0/2710183786 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61640b16d0 0x7f616414c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:24.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.312+0000 7f6162d9d700 1 -- 172.21.15.55:0/2710183786 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61640b16d0 msgr2=0x7f616414c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:24.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.312+0000 7f6162d9d700 1 --2- 172.21.15.55:0/2710183786 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61640b16d0 0x7f616414c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:24.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.312+0000 7f6162d9d700 1 -- 172.21.15.55:0/2710183786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61640b4e70 msgr2=0x7f6164068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:24.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.312+0000 7f6162d9d700 1 --2- 172.21.15.55:0/2710183786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61640b4e70 0x7f6164068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:24.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.312+0000 7f6162d9d700 1 -- 172.21.15.55:0/2710183786 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61580094f0 con 0x7f616415d240 2023-10-06T05:07:24.316 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.313+0000 7f6162d9d700 1 --2- 172.21.15.55:0/2710183786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f616415d240 0x7f61640688d0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f615c00bc30 tx=0x7f615c009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:24.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.313+0000 7f61527fc700 1 -- 172.21.15.55:0/2710183786 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c00c840 con 0x7f616415d240 2023-10-06T05:07:24.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.313+0000 7f616359e700 1 -- 172.21.15.55:0/2710183786 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61640c6ac0 con 0x7f616415d240 2023-10-06T05:07:24.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.313+0000 7f61527fc700 1 -- 172.21.15.55:0/2710183786 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c00c9e0 con 0x7f616415d240 2023-10-06T05:07:24.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.313+0000 7f61527fc700 1 -- 172.21.15.55:0/2710183786 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c00cda0 con 0x7f616415d240 2023-10-06T05:07:24.317 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.313+0000 7f616359e700 1 -- 172.21.15.55:0/2710183786 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61640c72c0 con 0x7f616415d240 2023-10-06T05:07:24.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.314+0000 7f616359e700 1 -- 172.21.15.55:0/2710183786 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61640c6fa0 con 0x7f616415d240 2023-10-06T05:07:24.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.315+0000 7f61527fc700 1 -- 172.21.15.55:0/2710183786 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f615c014620 con 0x7f616415d240 2023-10-06T05:07:24.318 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.317+0000 7f61527fc700 1 --2- 172.21.15.55:0/2710183786 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6144066f30 0x7f61440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:24.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.317+0000 7f61527fc700 1 -- 172.21.15.55:0/2710183786 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f615c084890 con 0x7f616415d240 2023-10-06T05:07:24.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.317+0000 7f616259c700 1 --2- 172.21.15.55:0/2710183786 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6144066f30 0x7f61440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:24.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.318+0000 7f616259c700 1 --2- 172.21.15.55:0/2710183786 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6144066f30 0x7f61440693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6154006fd0 tx=0x7f6154008040 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:24.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.321+0000 7f61527fc700 1 -- 172.21.15.55:0/2710183786 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f615c051740 con 0x7f616415d240 2023-10-06T05:07:24.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:24.571+0000 7f616359e700 1 -- 172.21.15.55:0/2710183786 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f61640af610 con 0x7f616415d240 2023-10-06T05:07:25.258 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.256+0000 7f61527fc700 1 -- 172.21.15.55:0/2710183786 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v104) v1 ==== 118+0+0 (secure 0 0 0) 0x7f615c054d60 con 0x7f616415d240 2023-10-06T05:07:25.260 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.258+0000 7f616359e700 1 -- 172.21.15.55:0/2710183786 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6144066f30 msgr2=0x7f61440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:25.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.258+0000 7f616359e700 1 --2- 172.21.15.55:0/2710183786 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6144066f30 0x7f61440693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6154006fd0 tx=0x7f6154008040 comp rx=0 tx=0).stop 2023-10-06T05:07:25.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.258+0000 7f616359e700 1 -- 172.21.15.55:0/2710183786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f616415d240 msgr2=0x7f61640688d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:25.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.258+0000 7f616359e700 1 --2- 172.21.15.55:0/2710183786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f616415d240 0x7f61640688d0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f615c00bc30 tx=0x7f615c009820 comp rx=0 tx=0).stop 2023-10-06T05:07:25.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.259+0000 7f616359e700 1 -- 172.21.15.55:0/2710183786 shutdown_connections 2023-10-06T05:07:25.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.259+0000 7f616359e700 1 --2- 172.21.15.55:0/2710183786 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6144066f30 0x7f61440693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:25.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.259+0000 7f616359e700 1 --2- 172.21.15.55:0/2710183786 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f616415d240 0x7f61640688d0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:25.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.259+0000 7f616359e700 1 --2- 172.21.15.55:0/2710183786 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f61640b16d0 0x7f616414c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:25.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.259+0000 7f616359e700 1 --2- 172.21.15.55:0/2710183786 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f61640b4e70 0x7f6164068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:25.261 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.259+0000 7f616359e700 1 -- 172.21.15.55:0/2710183786 >> 172.21.15.55:0/2710183786 conn(0x7f61640d10c0 msgr2=0x7f6164069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.259+0000 7f616359e700 1 -- 172.21.15.55:0/2710183786 shutdown_connections 2023-10-06T05:07:25.262 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.259+0000 7f616359e700 1 -- 172.21.15.55:0/2710183786 wait complete. 2023-10-06T05:07:25.281 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T05:07:25.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.474+0000 7f112119d700 1 Processor -- start 2023-10-06T05:07:25.476 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.474+0000 7f112119d700 1 -- start start 2023-10-06T05:07:25.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.475+0000 7f112119d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f111c140bb0 0x7f111c06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:25.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.475+0000 7f112119d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c06da30 0x7f111c0c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:25.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.475+0000 7f112119d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111c0c6820 con 0x7f111c1407e0 2023-10-06T05:07:25.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.475+0000 7f112119d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111c06deb0 con 0x7f111c140bb0 2023-10-06T05:07:25.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.475+0000 7f112119d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0c6de0 con 0x7f111c06da30 2023-10-06T05:07:25.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.476+0000 7f111b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f111c140bb0 0x7f111c06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:25.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.476+0000 7f111bfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f111c1407e0 0x7f111c06a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:33244/0 (socket says 172.21.15.55:33244) 2023-10-06T05:07:25.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.476+0000 7f111b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f111c140bb0 0x7f111c06d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43764/0 (socket says 172.21.15.55:43764) 2023-10-06T05:07:25.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.476+0000 7f111bfff700 1 -- 172.21.15.55:0/3916094080 learned_addr learned my addr 172.21.15.55:0/3916094080 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:25.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.476+0000 7f112099c700 1 --2- 172.21.15.55:0/3916094080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c06da30 0x7f111c0c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:25.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.476+0000 7f111b7fe700 1 -- 172.21.15.55:0/3916094080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c06da30 msgr2=0x7f111c0c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:25.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.476+0000 7f111b7fe700 1 --2- 172.21.15.55:0/3916094080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c06da30 0x7f111c0c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:25.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.476+0000 7f111b7fe700 1 -- 172.21.15.55:0/3916094080 >> v1:172.21.15.55:6789/0 conn(0x7f111c1407e0 legacy=0x7f111c06a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:25.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.476+0000 7f111b7fe700 1 -- 172.21.15.55:0/3916094080 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c141270 con 0x7f111c140bb0 2023-10-06T05:07:25.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.476+0000 7f111affd700 1 -- 172.21.15.55:0/3916094080 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852561655 0 0) 0x7f111c0c6820 con 0x7f111c1407e0 2023-10-06T05:07:25.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.476+0000 7f111b7fe700 1 --2- 172.21.15.55:0/3916094080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c140bb0 0x7f111c06d4f0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f11100098e0 tx=0x7f11100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b0a3348a94c890 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:25.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.477+0000 7f111affd700 1 -- 172.21.15.55:0/3916094080 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110009e80 con 0x7f111c140bb0 2023-10-06T05:07:25.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.477+0000 7f111affd700 1 -- 172.21.15.55:0/3916094080 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111000b860 con 0x7f111c140bb0 2023-10-06T05:07:25.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.477+0000 7f111affd700 1 -- 172.21.15.55:0/3916094080 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111000bbe0 con 0x7f111c140bb0 2023-10-06T05:07:25.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.477+0000 7f112119d700 1 -- 172.21.15.55:0/3916094080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c140bb0 msgr2=0x7f111c06d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:25.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.477+0000 7f112119d700 1 --2- 172.21.15.55:0/3916094080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c140bb0 0x7f111c06d4f0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f11100098e0 tx=0x7f11100091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:25.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.477+0000 7f112119d700 1 -- 172.21.15.55:0/3916094080 shutdown_connections 2023-10-06T05:07:25.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.477+0000 7f112119d700 1 --2- 172.21.15.55:0/3916094080 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c06da30 0x7f111c0c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:25.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.477+0000 7f112119d700 1 --2- 172.21.15.55:0/3916094080 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c140bb0 0x7f111c06d4f0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:25.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.477+0000 7f112119d700 1 -- 172.21.15.55:0/3916094080 >> 172.21.15.55:0/3916094080 conn(0x7f111c0cc1e0 msgr2=0x7f111c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:25.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.477+0000 7f112119d700 1 -- 172.21.15.55:0/3916094080 shutdown_connections 2023-10-06T05:07:25.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.477+0000 7f112119d700 1 -- 172.21.15.55:0/3916094080 wait complete. 2023-10-06T05:07:25.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.479+0000 7f112119d700 1 Processor -- start 2023-10-06T05:07:25.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.479+0000 7f112119d700 1 -- start start 2023-10-06T05:07:25.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.480+0000 7f112119d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111c06da30 0x7f111c0a7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:25.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.480+0000 7f112119d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c1407e0 0x7f111c0a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:25.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.480+0000 7f112119d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c140bb0 0x7f111c167a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:25.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.480+0000 7f112119d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0a6be0 con 0x7f111c1407e0 2023-10-06T05:07:25.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.480+0000 7f112119d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0a5f60 con 0x7f111c06da30 2023-10-06T05:07:25.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.480+0000 7f112119d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0a6240 con 0x7f111c140bb0 2023-10-06T05:07:25.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.480+0000 7f111b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c1407e0 0x7f111c0a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:25.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.480+0000 7f111b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c1407e0 0x7f111c0a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43780/0 (socket says 172.21.15.55:43780) 2023-10-06T05:07:25.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.480+0000 7f111b7fe700 1 -- 172.21.15.55:0/680068248 learned_addr learned my addr 172.21.15.55:0/680068248 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:25.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.480+0000 7f112099c700 1 --2- 172.21.15.55:0/680068248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c140bb0 0x7f111c167a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:25.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.480+0000 7f111bfff700 1 --2- 172.21.15.55:0/680068248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111c06da30 0x7f111c0a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:25.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.480+0000 7f111b7fe700 1 -- 172.21.15.55:0/680068248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c140bb0 msgr2=0x7f111c167a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:25.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.480+0000 7f111b7fe700 1 --2- 172.21.15.55:0/680068248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c140bb0 0x7f111c167a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:25.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.481+0000 7f111b7fe700 1 -- 172.21.15.55:0/680068248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111c06da30 msgr2=0x7f111c0a7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:25.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.481+0000 7f111b7fe700 1 --2- 172.21.15.55:0/680068248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111c06da30 0x7f111c0a7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:25.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.481+0000 7f111b7fe700 1 -- 172.21.15.55:0/680068248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1110009420 con 0x7f111c1407e0 2023-10-06T05:07:25.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.481+0000 7f111b7fe700 1 --2- 172.21.15.55:0/680068248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c1407e0 0x7f111c0a54c0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f111c06cd50 tx=0x7f1110017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:25.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.481+0000 7f1103fff700 1 -- 172.21.15.55:0/680068248 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111000bbe0 con 0x7f111c1407e0 2023-10-06T05:07:25.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.481+0000 7f1103fff700 1 -- 172.21.15.55:0/680068248 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1110011420 con 0x7f111c1407e0 2023-10-06T05:07:25.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.481+0000 7f1103fff700 1 -- 172.21.15.55:0/680068248 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11100117c0 con 0x7f111c1407e0 2023-10-06T05:07:25.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.481+0000 7f112119d700 1 -- 172.21.15.55:0/680068248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c0a6520 con 0x7f111c1407e0 2023-10-06T05:07:25.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.481+0000 7f112119d700 1 -- 172.21.15.55:0/680068248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c0c5760 con 0x7f111c1407e0 2023-10-06T05:07:25.487 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.482+0000 7f112119d700 1 -- 172.21.15.55:0/680068248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c06c450 con 0x7f111c1407e0 2023-10-06T05:07:25.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.483+0000 7f1103fff700 1 -- 172.21.15.55:0/680068248 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1110011bb0 con 0x7f111c1407e0 2023-10-06T05:07:25.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.484+0000 7f1103fff700 1 --2- 172.21.15.55:0/680068248 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1108066f30 0x7f11080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:25.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.484+0000 7f1103fff700 1 -- 172.21.15.55:0/680068248 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f1110086250 con 0x7f111c1407e0 2023-10-06T05:07:25.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.484+0000 7f111bfff700 1 --2- 172.21.15.55:0/680068248 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1108066f30 0x7f11080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:25.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.485+0000 7f111bfff700 1 --2- 172.21.15.55:0/680068248 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1108066f30 0x7f11080693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f110c0011e0 tx=0x7f110c009040 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:25.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.488+0000 7f1103fff700 1 -- 172.21.15.55:0/680068248 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1110053100 con 0x7f111c1407e0 2023-10-06T05:07:25.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:25.735+0000 7f112119d700 1 -- 172.21.15.55:0/680068248 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f111c06acf0 con 0x7f111c1407e0 2023-10-06T05:07:26.267 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.265+0000 7f1103fff700 1 -- 172.21.15.55:0/680068248 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v105) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1110011e60 con 0x7f111c1407e0 2023-10-06T05:07:26.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.268+0000 7f112119d700 1 -- 172.21.15.55:0/680068248 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1108066f30 msgr2=0x7f11080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:26.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.268+0000 7f112119d700 1 --2- 172.21.15.55:0/680068248 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1108066f30 0x7f11080693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f110c0011e0 tx=0x7f110c009040 comp rx=0 tx=0).stop 2023-10-06T05:07:26.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.268+0000 7f112119d700 1 -- 172.21.15.55:0/680068248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c1407e0 msgr2=0x7f111c0a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:26.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.268+0000 7f112119d700 1 --2- 172.21.15.55:0/680068248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c1407e0 0x7f111c0a54c0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f111c06cd50 tx=0x7f1110017000 comp rx=0 tx=0).stop 2023-10-06T05:07:26.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.268+0000 7f112119d700 1 -- 172.21.15.55:0/680068248 shutdown_connections 2023-10-06T05:07:26.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.268+0000 7f112119d700 1 --2- 172.21.15.55:0/680068248 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1108066f30 0x7f11080693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.268+0000 7f112119d700 1 --2- 172.21.15.55:0/680068248 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c140bb0 0x7f111c167a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.268+0000 7f112119d700 1 --2- 172.21.15.55:0/680068248 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111c06da30 0x7f111c0a7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.268+0000 7f112119d700 1 --2- 172.21.15.55:0/680068248 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c1407e0 0x7f111c0a54c0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.268+0000 7f112119d700 1 -- 172.21.15.55:0/680068248 >> 172.21.15.55:0/680068248 conn(0x7f111c0cc1e0 msgr2=0x7f111c0c2630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:26.271 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.269+0000 7f112119d700 1 -- 172.21.15.55:0/680068248 shutdown_connections 2023-10-06T05:07:26.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.269+0000 7f112119d700 1 -- 172.21.15.55:0/680068248 wait complete. 2023-10-06T05:07:26.289 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:07:26.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.478+0000 7fa88bfff700 1 Processor -- start 2023-10-06T05:07:26.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.478+0000 7fa88bfff700 1 -- start start 2023-10-06T05:07:26.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.479+0000 7fa88bfff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa88c1597f0 0x7fa88c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:26.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.479+0000 7fa88bfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa88c15d230 0x7fa88c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:26.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.479+0000 7fa88bfff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88c15db70 con 0x7fa88c159420 2023-10-06T05:07:26.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.479+0000 7fa88bfff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88c0b11d0 con 0x7fa88c15d230 2023-10-06T05:07:26.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.479+0000 7fa88bfff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88c0b14b0 con 0x7fa88c1597f0 2023-10-06T05:07:26.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.479+0000 7fa88b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa88c15d230 0x7fa88c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:26.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.480+0000 7fa88b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa88c15d230 0x7fa88c15d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43792/0 (socket says 172.21.15.55:43792) 2023-10-06T05:07:26.482 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.480+0000 7fa88b7fe700 1 -- 172.21.15.55:0/2917869320 learned_addr learned my addr 172.21.15.55:0/2917869320 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:26.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.480+0000 7fa88a7fc700 1 --2- 172.21.15.55:0/2917869320 >> v2:172.21.15.163:3300/0 conn(0x7fa88c1597f0 0x7fa88c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:26.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.480+0000 7fa88b7fe700 1 -- 172.21.15.55:0/2917869320 >> v2:172.21.15.163:3300/0 conn(0x7fa88c1597f0 msgr2=0x7fa88c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:26.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.480+0000 7fa88b7fe700 1 --2- 172.21.15.55:0/2917869320 >> v2:172.21.15.163:3300/0 conn(0x7fa88c1597f0 0x7fa88c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.480+0000 7fa88b7fe700 1 -- 172.21.15.55:0/2917869320 >> v1:172.21.15.163:6789/0 conn(0x7fa88c159420 legacy=0x7fa88c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:26.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.480+0000 7fa88b7fe700 1 -- 172.21.15.55:0/2917869320 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c0b1790 con 0x7fa88c15d230 2023-10-06T05:07:26.483 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.480+0000 7fa88b7fe700 1 --2- 172.21.15.55:0/2917869320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c15d230 0x7fa88c15d630 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fa88c089750 tx=0x7fa884009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4467eeb6f533f2aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:26.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.480+0000 7fa889ffb700 1 -- 172.21.15.55:0/2917869320 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88400e070 con 0x7fa88c15d230 2023-10-06T05:07:26.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.480+0000 7fa889ffb700 1 -- 172.21.15.55:0/2917869320 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88400ba10 con 0x7fa88c15d230 2023-10-06T05:07:26.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.480+0000 7fa889ffb700 1 -- 172.21.15.55:0/2917869320 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8840092e0 con 0x7fa88c15d230 2023-10-06T05:07:26.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.481+0000 7fa88bfff700 1 -- 172.21.15.55:0/2917869320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c15d230 msgr2=0x7fa88c15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:26.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.481+0000 7fa88bfff700 1 --2- 172.21.15.55:0/2917869320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c15d230 0x7fa88c15d630 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fa88c089750 tx=0x7fa884009840 comp rx=0 tx=0).stop 2023-10-06T05:07:26.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.481+0000 7fa88bfff700 1 -- 172.21.15.55:0/2917869320 shutdown_connections 2023-10-06T05:07:26.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.481+0000 7fa88bfff700 1 --2- 172.21.15.55:0/2917869320 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c15d230 0x7fa88c15d630 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.481+0000 7fa88bfff700 1 --2- 172.21.15.55:0/2917869320 >> v2:172.21.15.163:3300/0 conn(0x7fa88c1597f0 0x7fa88c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.481+0000 7fa88bfff700 1 -- 172.21.15.55:0/2917869320 >> 172.21.15.55:0/2917869320 conn(0x7fa88c0d0660 msgr2=0x7fa88c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:26.485 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.481+0000 7fa88bfff700 1 -- 172.21.15.55:0/2917869320 shutdown_connections 2023-10-06T05:07:26.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.481+0000 7fa88bfff700 1 -- 172.21.15.55:0/2917869320 wait complete. 2023-10-06T05:07:26.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.483+0000 7fa88bfff700 1 Processor -- start 2023-10-06T05:07:26.486 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.483+0000 7fa88bfff700 1 -- start start 2023-10-06T05:07:26.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88bfff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c159420 0x7fa88c14a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:26.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88bfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa88c1597f0 0x7fa88c15c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:26.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c15d230 0x7fa88c0c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:26.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88bfff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0c66d0 con 0x7fa88c15d230 2023-10-06T05:07:26.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88bfff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0c5ac0 con 0x7fa88c1597f0 2023-10-06T05:07:26.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88bfff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0c5da0 con 0x7fa88c159420 2023-10-06T05:07:26.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c15d230 0x7fa88c0c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:26.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c15d230 0x7fa88c0c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43804/0 (socket says 172.21.15.55:43804) 2023-10-06T05:07:26.488 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88b7fe700 1 -- 172.21.15.55:0/2374149031 learned_addr learned my addr 172.21.15.55:0/2374149031 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:26.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88a7fc700 1 --2- 172.21.15.55:0/2374149031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa88c1597f0 0x7fa88c15c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:26.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88affd700 1 --2- 172.21.15.55:0/2374149031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c159420 0x7fa88c14a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:26.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88b7fe700 1 -- 172.21.15.55:0/2374149031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c159420 msgr2=0x7fa88c14a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:26.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88b7fe700 1 --2- 172.21.15.55:0/2374149031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c159420 0x7fa88c14a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88b7fe700 1 -- 172.21.15.55:0/2374149031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa88c1597f0 msgr2=0x7fa88c15c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:26.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88b7fe700 1 --2- 172.21.15.55:0/2374149031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa88c1597f0 0x7fa88c15c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88b7fe700 1 -- 172.21.15.55:0/2374149031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa884009ae0 con 0x7fa88c15d230 2023-10-06T05:07:26.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.484+0000 7fa88b7fe700 1 --2- 172.21.15.55:0/2374149031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c15d230 0x7fa88c0c9f30 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fa884009880 tx=0x7fa884017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:26.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.485+0000 7fa872ffd700 1 -- 172.21.15.55:0/2374149031 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884012e60 con 0x7fa88c15d230 2023-10-06T05:07:26.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.485+0000 7fa872ffd700 1 -- 172.21.15.55:0/2374149031 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8840092e0 con 0x7fa88c15d230 2023-10-06T05:07:26.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.485+0000 7fa872ffd700 1 -- 172.21.15.55:0/2374149031 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88401d670 con 0x7fa88c15d230 2023-10-06T05:07:26.489 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.485+0000 7fa88bfff700 1 -- 172.21.15.55:0/2374149031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88c0c6080 con 0x7fa88c15d230 2023-10-06T05:07:26.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.485+0000 7fa88bfff700 1 -- 172.21.15.55:0/2374149031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88c14a5d0 con 0x7fa88c15d230 2023-10-06T05:07:26.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.486+0000 7fa88bfff700 1 -- 172.21.15.55:0/2374149031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88c0c6840 con 0x7fa88c15d230 2023-10-06T05:07:26.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.487+0000 7fa872ffd700 1 -- 172.21.15.55:0/2374149031 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa88401da60 con 0x7fa88c15d230 2023-10-06T05:07:26.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.488+0000 7fa872ffd700 1 --2- 172.21.15.55:0/2374149031 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa878066f30 0x7fa8780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.488+0000 7fa872ffd700 1 -- 172.21.15.55:0/2374149031 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fa884086130 con 0x7fa88c15d230 2023-10-06T05:07:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.489+0000 7fa88affd700 1 --2- 172.21.15.55:0/2374149031 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa878066f30 0x7fa8780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:26.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.489+0000 7fa88affd700 1 --2- 172.21.15.55:0/2374149031 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa878066f30 0x7fa8780693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa87c003870 tx=0x7fa87c009040 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:26.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.492+0000 7fa872ffd700 1 -- 172.21.15.55:0/2374149031 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa884052fe0 con 0x7fa88c15d230 2023-10-06T05:07:26.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.741+0000 7fa88bfff700 1 -- 172.21.15.55:0/2374149031 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa88c04c3c0 con 0x7fa88c15d230 2023-10-06T05:07:26.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.741+0000 7fa872ffd700 1 -- 172.21.15.55:0/2374149031 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4616 (secure 0 0 0) 0x7fa884056600 con 0x7fa88c15d230 2023-10-06T05:07:26.743 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:26.743 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3704044820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3704044820},{"type":"v1","addr":"172.21.15.163:6834","nonce":3704044820}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":9772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2254022733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2254022733},{"type":"v1","addr":"172.21.15.163:6835","nonce":2254022733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":10224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3844040929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3844040929},{"type":"v1","addr":"172.21.15.55:6834","nonce":3844040929}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:07:22.724188+0000","modified":"2023-10-06T05:07:26.255934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10227":{"gid":10227,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/3632968604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3632968604},{"type":"v1","addr":"172.21.15.55:6835","nonce":3632968604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:07:26.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.744+0000 7fa88bfff700 1 -- 172.21.15.55:0/2374149031 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa878066f30 msgr2=0x7fa8780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:26.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.744+0000 7fa88bfff700 1 --2- 172.21.15.55:0/2374149031 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa878066f30 0x7fa8780693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa87c003870 tx=0x7fa87c009040 comp rx=0 tx=0).stop 2023-10-06T05:07:26.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.744+0000 7fa88bfff700 1 -- 172.21.15.55:0/2374149031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c15d230 msgr2=0x7fa88c0c9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:26.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.744+0000 7fa88bfff700 1 --2- 172.21.15.55:0/2374149031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c15d230 0x7fa88c0c9f30 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fa884009880 tx=0x7fa884017000 comp rx=0 tx=0).stop 2023-10-06T05:07:26.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.744+0000 7fa88bfff700 1 -- 172.21.15.55:0/2374149031 shutdown_connections 2023-10-06T05:07:26.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.744+0000 7fa88bfff700 1 --2- 172.21.15.55:0/2374149031 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa878066f30 0x7fa8780693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.744+0000 7fa88bfff700 1 --2- 172.21.15.55:0/2374149031 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa88c15d230 0x7fa88c0c9f30 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.744+0000 7fa88bfff700 1 --2- 172.21.15.55:0/2374149031 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa88c159420 0x7fa88c14a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.744+0000 7fa88bfff700 1 --2- 172.21.15.55:0/2374149031 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa88c1597f0 0x7fa88c15c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.744+0000 7fa88bfff700 1 -- 172.21.15.55:0/2374149031 >> 172.21.15.55:0/2374149031 conn(0x7fa88c0d0660 msgr2=0x7fa88c156010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:26.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.744+0000 7fa88bfff700 1 -- 172.21.15.55:0/2374149031 shutdown_connections 2023-10-06T05:07:26.747 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.744+0000 7fa88bfff700 1 -- 172.21.15.55:0/2374149031 wait complete. 2023-10-06T05:07:26.747 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 105 2023-10-06T05:07:26.763 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:07:26.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.948+0000 7f95af59d700 1 Processor -- start 2023-10-06T05:07:26.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.948+0000 7f95af59d700 1 -- start start 2023-10-06T05:07:26.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.950+0000 7f95af59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f95a8150180 0x7f95a8157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:26.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.950+0000 7f95af59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95a8150550 0x7f95a80c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:26.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.950+0000 7f95af59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a806e2a0 con 0x7f95a80c7240 2023-10-06T05:07:26.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.950+0000 7f95af59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a80c76c0 con 0x7f95a8150180 2023-10-06T05:07:26.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.950+0000 7f95af59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a806b2e0 con 0x7f95a8150550 2023-10-06T05:07:26.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.950+0000 7f95ae59b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f95a8150180 0x7f95a8157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:26.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.950+0000 7f95add9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95a8150550 0x7f95a80c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:26.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.950+0000 7f95aed9c700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7f95a80c7240 0x7f95a8069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:59662/0 (socket says 172.21.15.55:59662) 2023-10-06T05:07:26.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.950+0000 7f95aed9c700 1 -- 172.21.15.55:0/3572793912 learned_addr learned my addr 172.21.15.55:0/3572793912 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:26.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.951+0000 7f95ad599700 1 -- 172.21.15.55:0/3572793912 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2460358843 0 0) 0x7f95a806e2a0 con 0x7f95a80c7240 2023-10-06T05:07:26.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.951+0000 7f95ad599700 1 -- 172.21.15.55:0/3572793912 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9594003570 con 0x7f95a80c7240 2023-10-06T05:07:26.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.951+0000 7f95add9a700 1 -- 172.21.15.55:0/3572793912 >> v2:172.21.15.163:3300/0 conn(0x7f95a8150180 msgr2=0x7f95a8157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:26.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.951+0000 7f95add9a700 1 --2- 172.21.15.55:0/3572793912 >> v2:172.21.15.163:3300/0 conn(0x7f95a8150180 0x7f95a8157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.951+0000 7f95add9a700 1 -- 172.21.15.55:0/3572793912 >> v1:172.21.15.163:6789/0 conn(0x7f95a80c7240 legacy=0x7f95a8069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:26.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.951+0000 7f95add9a700 1 -- 172.21.15.55:0/3572793912 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a80c78e0 con 0x7f95a8150550 2023-10-06T05:07:26.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.951+0000 7f95ae59b700 1 --2- 172.21.15.55:0/3572793912 >> v2:172.21.15.163:3300/0 conn(0x7f95a8150180 0x7f95a8157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:26.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.951+0000 7f95add9a700 1 --2- 172.21.15.55:0/3572793912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95a8150550 0x7f95a80c6d00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f959800b9d0 tx=0x7f95980090f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1676462d8419ea58 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:26.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.951+0000 7f95ad599700 1 -- 172.21.15.55:0/3572793912 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959800c870 con 0x7f95a8150550 2023-10-06T05:07:26.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.952+0000 7f95ad599700 1 -- 172.21.15.55:0/3572793912 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959800c9d0 con 0x7f95a8150550 2023-10-06T05:07:26.955 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.952+0000 7f95ad599700 1 -- 172.21.15.55:0/3572793912 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959800cd90 con 0x7f95a8150550 2023-10-06T05:07:26.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.952+0000 7f95af59d700 1 -- 172.21.15.55:0/3572793912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95a8150550 msgr2=0x7f95a80c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:26.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.952+0000 7f95af59d700 1 --2- 172.21.15.55:0/3572793912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95a8150550 0x7f95a80c6d00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f959800b9d0 tx=0x7f95980090f0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.952+0000 7f95af59d700 1 -- 172.21.15.55:0/3572793912 shutdown_connections 2023-10-06T05:07:26.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.952+0000 7f95af59d700 1 --2- 172.21.15.55:0/3572793912 >> v2:172.21.15.163:3300/0 conn(0x7f95a8150180 0x7f95a8157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.952+0000 7f95af59d700 1 --2- 172.21.15.55:0/3572793912 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95a8150550 0x7f95a80c6d00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.952+0000 7f95af59d700 1 -- 172.21.15.55:0/3572793912 >> 172.21.15.55:0/3572793912 conn(0x7f95a80cc1f0 msgr2=0x7f95a80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:26.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.952+0000 7f95af59d700 1 -- 172.21.15.55:0/3572793912 shutdown_connections 2023-10-06T05:07:26.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.952+0000 7f95af59d700 1 -- 172.21.15.55:0/3572793912 wait complete. 2023-10-06T05:07:26.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.954+0000 7f95af59d700 1 Processor -- start 2023-10-06T05:07:26.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.954+0000 7f95af59d700 1 -- start start 2023-10-06T05:07:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95af59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95a80c7240 0x7f95a80c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95af59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95a8150180 0x7f95a8148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95af59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95a8150550 0x7f95a80cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95af59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80ace50 con 0x7f95a8150180 2023-10-06T05:07:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95af59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80ac840 con 0x7f95a80c7240 2023-10-06T05:07:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95af59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80c6070 con 0x7f95a8150550 2023-10-06T05:07:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95add9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95a8150180 0x7f95a8148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95add9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95a8150180 0x7f95a8148cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43814/0 (socket says 172.21.15.55:43814) 2023-10-06T05:07:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95add9a700 1 -- 172.21.15.55:0/2894267370 learned_addr learned my addr 172.21.15.55:0/2894267370 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95aed9c700 1 --2- 172.21.15.55:0/2894267370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95a8150550 0x7f95a80cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:26.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95ae59b700 1 --2- 172.21.15.55:0/2894267370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95a80c7240 0x7f95a80c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:26.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95add9a700 1 -- 172.21.15.55:0/2894267370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95a8150550 msgr2=0x7f95a80cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:26.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95add9a700 1 --2- 172.21.15.55:0/2894267370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95a8150550 0x7f95a80cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95add9a700 1 -- 172.21.15.55:0/2894267370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95a80c7240 msgr2=0x7f95a80c3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:26.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95add9a700 1 --2- 172.21.15.55:0/2894267370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95a80c7240 0x7f95a80c3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:26.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.955+0000 7f95add9a700 1 -- 172.21.15.55:0/2894267370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a400d8d0 con 0x7f95a8150180 2023-10-06T05:07:26.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.956+0000 7f95add9a700 1 --2- 172.21.15.55:0/2894267370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95a8150180 0x7f95a8148cd0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f95a806b420 tx=0x7f9598017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:26.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.956+0000 7f959e7fc700 1 -- 172.21.15.55:0/2894267370 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598022400 con 0x7f95a8150180 2023-10-06T05:07:26.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.956+0000 7f959e7fc700 1 -- 172.21.15.55:0/2894267370 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9598022560 con 0x7f95a8150180 2023-10-06T05:07:26.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.956+0000 7f959e7fc700 1 -- 172.21.15.55:0/2894267370 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9598022900 con 0x7f95a8150180 2023-10-06T05:07:26.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.956+0000 7f95af59d700 1 -- 172.21.15.55:0/2894267370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959800b4e0 con 0x7f95a8150180 2023-10-06T05:07:26.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.956+0000 7f95af59d700 1 -- 172.21.15.55:0/2894267370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a80acae0 con 0x7f95a8150180 2023-10-06T05:07:26.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.957+0000 7f95af59d700 1 -- 172.21.15.55:0/2894267370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a80acfc0 con 0x7f95a8150180 2023-10-06T05:07:26.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.958+0000 7f959e7fc700 1 -- 172.21.15.55:0/2894267370 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9598017400 con 0x7f95a8150180 2023-10-06T05:07:26.961 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.959+0000 7f959e7fc700 1 --2- 172.21.15.55:0/2894267370 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f958c066f30 0x7f958c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:26.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.959+0000 7f959e7fc700 1 -- 172.21.15.55:0/2894267370 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f9598086400 con 0x7f95a8150180 2023-10-06T05:07:26.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.959+0000 7f95ae59b700 1 --2- 172.21.15.55:0/2894267370 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f958c066f30 0x7f958c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:26.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.960+0000 7f95ae59b700 1 --2- 172.21.15.55:0/2894267370 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f958c066f30 0x7f958c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95a400d8a0 tx=0x7f95a400f040 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:26.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:26.963+0000 7f959e7fc700 1 -- 172.21.15.55:0/2894267370 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95980532b0 con 0x7f95a8150180 2023-10-06T05:07:27.171 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.170+0000 7f95af59d700 1 -- 172.21.15.55:0/2894267370 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95a804c3c0 con 0x7f95a8150180 2023-10-06T05:07:27.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.171+0000 7f959e7fc700 1 -- 172.21.15.55:0/2894267370 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v207) v1 ==== 74+0+17614 (secure 0 0 0) 0x7f95980568d0 con 0x7f95a8150180 2023-10-06T05:07:27.173 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:27.174 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":207,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:07:23.969436+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:07:20.916360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:07:21.706951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"207","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:07:27.175 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.173+0000 7f95af59d700 1 -- 172.21.15.55:0/2894267370 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f958c066f30 msgr2=0x7f958c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:27.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.173+0000 7f95af59d700 1 --2- 172.21.15.55:0/2894267370 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f958c066f30 0x7f958c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95a400d8a0 tx=0x7f95a400f040 comp rx=0 tx=0).stop 2023-10-06T05:07:27.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.173+0000 7f95af59d700 1 -- 172.21.15.55:0/2894267370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95a8150180 msgr2=0x7f95a8148cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:27.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.173+0000 7f95af59d700 1 --2- 172.21.15.55:0/2894267370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95a8150180 0x7f95a8148cd0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f95a806b420 tx=0x7f9598017000 comp rx=0 tx=0).stop 2023-10-06T05:07:27.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.173+0000 7f95af59d700 1 -- 172.21.15.55:0/2894267370 shutdown_connections 2023-10-06T05:07:27.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.173+0000 7f95af59d700 1 --2- 172.21.15.55:0/2894267370 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f958c066f30 0x7f958c0693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.173+0000 7f95af59d700 1 --2- 172.21.15.55:0/2894267370 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f95a8150550 0x7f95a80cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.173+0000 7f95af59d700 1 --2- 172.21.15.55:0/2894267370 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f95a80c7240 0x7f95a80c3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.173+0000 7f95af59d700 1 --2- 172.21.15.55:0/2894267370 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f95a8150180 0x7f95a8148cd0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.176 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.173+0000 7f95af59d700 1 -- 172.21.15.55:0/2894267370 >> 172.21.15.55:0/2894267370 conn(0x7f95a80cc1f0 msgr2=0x7f95a806a520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:27.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.173+0000 7f95af59d700 1 -- 172.21.15.55:0/2894267370 shutdown_connections 2023-10-06T05:07:27.177 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.173+0000 7f95af59d700 1 -- 172.21.15.55:0/2894267370 wait complete. 2023-10-06T05:07:27.194 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T05:07:27.194 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:07:27.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.381+0000 7f6c45f9d700 1 Processor -- start 2023-10-06T05:07:27.383 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.381+0000 7f6c45f9d700 1 -- start start 2023-10-06T05:07:27.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.382+0000 7f6c45f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6c40150180 0x7f6c40157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:27.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.382+0000 7f6c45f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c400c7410 0x7f6c40069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:27.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.382+0000 7f6c45f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c4006e2a0 con 0x7f6c40150550 2023-10-06T05:07:27.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.382+0000 7f6c45f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c4006b2e0 con 0x7f6c40150180 2023-10-06T05:07:27.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.382+0000 7f6c45f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400c78e0 con 0x7f6c400c7410 2023-10-06T05:07:27.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.382+0000 7f6c3ffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f6c40150550 0x7f6c400c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:33248/0 (socket says 172.21.15.55:33248) 2023-10-06T05:07:27.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.382+0000 7f6c3ffff700 1 -- 172.21.15.55:0/462463939 learned_addr learned my addr 172.21.15.55:0/462463939 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:27.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.382+0000 7f6c44f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6c40150180 0x7f6c40157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:27.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.383+0000 7f6c4579c700 1 --2- 172.21.15.55:0/462463939 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c400c7410 0x7f6c40069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:27.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.383+0000 7f6c44f9b700 1 -- 172.21.15.55:0/462463939 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c400c7410 msgr2=0x7f6c40069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:27.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.383+0000 7f6c44f9b700 1 --2- 172.21.15.55:0/462463939 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c400c7410 0x7f6c40069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.383+0000 7f6c44f9b700 1 -- 172.21.15.55:0/462463939 >> v1:172.21.15.55:6789/0 conn(0x7f6c40150550 legacy=0x7f6c400c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:27.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.383+0000 7f6c44f9b700 1 -- 172.21.15.55:0/462463939 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c400c7bc0 con 0x7f6c40150180 2023-10-06T05:07:27.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.383+0000 7f6c3f7fe700 1 -- 172.21.15.55:0/462463939 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600618603 0 0) 0x7f6c4006e2a0 con 0x7f6c40150550 2023-10-06T05:07:27.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.383+0000 7f6c44f9b700 1 --2- 172.21.15.55:0/462463939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c40150180 0x7f6c40157770 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f6c340098e0 tx=0x7f6c340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4af746f4389510 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:27.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.383+0000 7f6c3f7fe700 1 -- 172.21.15.55:0/462463939 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3400c750 con 0x7f6c40150180 2023-10-06T05:07:27.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.384+0000 7f6c3f7fe700 1 -- 172.21.15.55:0/462463939 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3400c8b0 con 0x7f6c40150180 2023-10-06T05:07:27.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.384+0000 7f6c3f7fe700 1 -- 172.21.15.55:0/462463939 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3400cc50 con 0x7f6c40150180 2023-10-06T05:07:27.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.384+0000 7f6c45f9d700 1 -- 172.21.15.55:0/462463939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c40150180 msgr2=0x7f6c40157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:27.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.384+0000 7f6c45f9d700 1 --2- 172.21.15.55:0/462463939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c40150180 0x7f6c40157770 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f6c340098e0 tx=0x7f6c340091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.384+0000 7f6c45f9d700 1 -- 172.21.15.55:0/462463939 shutdown_connections 2023-10-06T05:07:27.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.384+0000 7f6c45f9d700 1 --2- 172.21.15.55:0/462463939 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c400c7410 0x7f6c40069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.384+0000 7f6c45f9d700 1 --2- 172.21.15.55:0/462463939 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c40150180 0x7f6c40157770 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.384+0000 7f6c45f9d700 1 -- 172.21.15.55:0/462463939 >> 172.21.15.55:0/462463939 conn(0x7f6c400cc1f0 msgr2=0x7f6c400cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:27.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.384+0000 7f6c45f9d700 1 -- 172.21.15.55:0/462463939 shutdown_connections 2023-10-06T05:07:27.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.384+0000 7f6c45f9d700 1 -- 172.21.15.55:0/462463939 wait complete. 2023-10-06T05:07:27.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.386+0000 7f6c45f9d700 1 Processor -- start 2023-10-06T05:07:27.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.386+0000 7f6c45f9d700 1 -- start start 2023-10-06T05:07:27.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c45f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c400c7410 0x7f6c40142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:27.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c45f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c40150180 0x7f6c40154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:27.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c45f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c40150550 0x7f6c40155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:27.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c45f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400abab0 con 0x7f6c40150550 2023-10-06T05:07:27.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c45f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400c5680 con 0x7f6c40150180 2023-10-06T05:07:27.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c45f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400c5960 con 0x7f6c400c7410 2023-10-06T05:07:27.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c4579c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c40150550 0x7f6c40155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:27.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c4579c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c40150550 0x7f6c40155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43834/0 (socket says 172.21.15.55:43834) 2023-10-06T05:07:27.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c4579c700 1 -- 172.21.15.55:0/3884389919 learned_addr learned my addr 172.21.15.55:0/3884389919 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:27.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c44f9b700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c400c7410 0x7f6c40142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:27.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c3ffff700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c40150180 0x7f6c40154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:27.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c4579c700 1 -- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c400c7410 msgr2=0x7f6c40142bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:27.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c4579c700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c400c7410 0x7f6c40142bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c4579c700 1 -- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c40150180 msgr2=0x7f6c40154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:27.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c4579c700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c40150180 0x7f6c40154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c4579c700 1 -- 172.21.15.55:0/3884389919 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c34009420 con 0x7f6c40150550 2023-10-06T05:07:27.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.387+0000 7f6c3ffff700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c40150180 0x7f6c40154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:07:27.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.388+0000 7f6c4579c700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c40150550 0x7f6c40155270 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009ec0 tx=0x7f6c3800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:27.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.388+0000 7f6c3cff9700 1 -- 172.21.15.55:0/3884389919 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800cc20 con 0x7f6c40150550 2023-10-06T05:07:27.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.388+0000 7f6c3cff9700 1 -- 172.21.15.55:0/3884389919 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3800f040 con 0x7f6c40150550 2023-10-06T05:07:27.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.388+0000 7f6c3cff9700 1 -- 172.21.15.55:0/3884389919 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38014480 con 0x7f6c40150550 2023-10-06T05:07:27.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.388+0000 7f6c45f9d700 1 -- 172.21.15.55:0/3884389919 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c400c5c40 con 0x7f6c40150550 2023-10-06T05:07:27.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.388+0000 7f6c45f9d700 1 -- 172.21.15.55:0/3884389919 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c40143130 con 0x7f6c40150550 2023-10-06T05:07:27.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.389+0000 7f6c45f9d700 1 -- 172.21.15.55:0/3884389919 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c400abc20 con 0x7f6c40150550 2023-10-06T05:07:27.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.390+0000 7f6c3cff9700 1 -- 172.21.15.55:0/3884389919 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c38014810 con 0x7f6c40150550 2023-10-06T05:07:27.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.391+0000 7f6c3cff9700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6c24066f30 0x7f6c240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:27.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.392+0000 7f6c3cff9700 1 -- 172.21.15.55:0/3884389919 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f6c38085610 con 0x7f6c40150550 2023-10-06T05:07:27.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.392+0000 7f6c44f9b700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6c24066f30 0x7f6c240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:27.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.393+0000 7f6c44f9b700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6c24066f30 0x7f6c240693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6c340093f0 tx=0x7f6c34018000 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:27.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.396+0000 7f6c3cff9700 1 -- 172.21.15.55:0/3884389919 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c38052560 con 0x7f6c40150550 2023-10-06T05:07:27.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.595+0000 7f6c45f9d700 1 -- 172.21.15.55:0/3884389919 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c400c6160 con 0x7f6c24066f30 2023-10-06T05:07:27.609 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.608+0000 7f6c3cff9700 1 -- 172.21.15.55:0/3884389919 <== mgr.9544 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311686 (secure 0 0 0) 0x7f6c400c6160 con 0x7f6c24066f30 2023-10-06T05:07:27.613 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:27.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.612+0000 7f6c45f9d700 1 -- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6c24066f30 msgr2=0x7f6c240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:27.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.612+0000 7f6c45f9d700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6c24066f30 0x7f6c240693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6c340093f0 tx=0x7f6c34018000 comp rx=0 tx=0).stop 2023-10-06T05:07:27.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.612+0000 7f6c45f9d700 1 -- 172.21.15.55:0/3884389919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c40150550 msgr2=0x7f6c40155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:27.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.612+0000 7f6c45f9d700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c40150550 0x7f6c40155270 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009ec0 tx=0x7f6c3800b040 comp rx=0 tx=0).stop 2023-10-06T05:07:27.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.612+0000 7f6c45f9d700 1 -- 172.21.15.55:0/3884389919 shutdown_connections 2023-10-06T05:07:27.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.612+0000 7f6c45f9d700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6c24066f30 0x7f6c240693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.616 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.612+0000 7f6c45f9d700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6c40150550 0x7f6c40155270 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.612+0000 7f6c45f9d700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6c400c7410 0x7f6c40142bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.612+0000 7f6c45f9d700 1 --2- 172.21.15.55:0/3884389919 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6c40150180 0x7f6c40154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.612+0000 7f6c45f9d700 1 -- 172.21.15.55:0/3884389919 >> 172.21.15.55:0/3884389919 conn(0x7f6c400cc1f0 msgr2=0x7f6c40147010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:27.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.612+0000 7f6c45f9d700 1 -- 172.21.15.55:0/3884389919 shutdown_connections 2023-10-06T05:07:27.617 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.612+0000 7f6c45f9d700 1 -- 172.21.15.55:0/3884389919 wait complete. 2023-10-06T05:07:27.617 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T05:07:27.632 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-06T05:07:26.167609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":623,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":386,"ondisk_log_size":386,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":193,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2457112,"kb_used_data":21464,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752517608,"statfs":{"total":773094113280,"available":770578030592,"internally_reserved":0,"allocated":21979136,"data_stored":12155996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.950411"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994948+0000","last_change":"2023-10-06T05:07:22.992773+0000","last_active":"2023-10-06T05:07:22.994948+0000","last_peered":"2023-10-06T05:07:22.994948+0000","last_clean":"2023-10-06T05:07:22.994948+0000","last_became_active":"2023-10-06T05:07:22.992522+0000","last_became_peered":"2023-10-06T05:07:22.992522+0000","last_unstale":"2023-10-06T05:07:22.994948+0000","last_undegraded":"2023-10-06T05:07:22.994948+0000","last_fullsized":"2023-10-06T05:07:22.994948+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:46.673236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.998948+0000","last_change":"2023-10-06T05:07:22.997391+0000","last_active":"2023-10-06T05:07:22.998948+0000","last_peered":"2023-10-06T05:07:22.998948+0000","last_clean":"2023-10-06T05:07:22.998948+0000","last_became_active":"2023-10-06T05:07:22.997114+0000","last_became_peered":"2023-10-06T05:07:22.997114+0000","last_unstale":"2023-10-06T05:07:22.998948+0000","last_undegraded":"2023-10-06T05:07:22.998948+0000","last_fullsized":"2023-10-06T05:07:22.998948+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:25.484832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.984731+0000","last_change":"2023-10-06T05:07:22.981446+0000","last_active":"2023-10-06T05:07:22.984731+0000","last_peered":"2023-10-06T05:07:22.984731+0000","last_clean":"2023-10-06T05:07:22.984731+0000","last_became_active":"2023-10-06T05:07:22.981175+0000","last_became_peered":"2023-10-06T05:07:22.981175+0000","last_unstale":"2023-10-06T05:07:22.984731+0000","last_undegraded":"2023-10-06T05:07:22.984731+0000","last_fullsized":"2023-10-06T05:07:22.984731+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:20.552363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.991409+0000","last_change":"2023-10-06T05:07:22.990554+0000","last_active":"2023-10-06T05:07:22.991409+0000","last_peered":"2023-10-06T05:07:22.991409+0000","last_clean":"2023-10-06T05:07:22.991409+0000","last_became_active":"2023-10-06T05:07:22.990271+0000","last_became_peered":"2023-10-06T05:07:22.990271+0000","last_unstale":"2023-10-06T05:07:22.991409+0000","last_undegraded":"2023-10-06T05:07:22.991409+0000","last_fullsized":"2023-10-06T05:07:22.991409+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:50.560382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996000+0000","last_change":"2023-10-06T05:07:22.995029+0000","last_active":"2023-10-06T05:07:22.996000+0000","last_peered":"2023-10-06T05:07:22.996000+0000","last_clean":"2023-10-06T05:07:22.996000+0000","last_became_active":"2023-10-06T05:07:22.994716+0000","last_became_peered":"2023-10-06T05:07:22.994716+0000","last_unstale":"2023-10-06T05:07:22.996000+0000","last_undegraded":"2023-10-06T05:07:22.996000+0000","last_fullsized":"2023-10-06T05:07:22.996000+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:22.229060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.985808+0000","last_change":"2023-10-06T05:07:22.981616+0000","last_active":"2023-10-06T05:07:22.985808+0000","last_peered":"2023-10-06T05:07:22.985808+0000","last_clean":"2023-10-06T05:07:22.985808+0000","last_became_active":"2023-10-06T05:07:22.981350+0000","last_became_peered":"2023-10-06T05:07:22.981350+0000","last_unstale":"2023-10-06T05:07:22.985808+0000","last_undegraded":"2023-10-06T05:07:22.985808+0000","last_fullsized":"2023-10-06T05:07:22.985808+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:54.689049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994794+0000","last_change":"2023-10-06T05:07:22.994181+0000","last_active":"2023-10-06T05:07:22.994794+0000","last_peered":"2023-10-06T05:07:22.994794+0000","last_clean":"2023-10-06T05:07:22.994794+0000","last_became_active":"2023-10-06T05:07:22.993934+0000","last_became_peered":"2023-10-06T05:07:22.993934+0000","last_unstale":"2023-10-06T05:07:22.994794+0000","last_undegraded":"2023-10-06T05:07:22.994794+0000","last_fullsized":"2023-10-06T05:07:22.994794+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:31.816501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.999843+0000","last_change":"2023-10-06T05:07:22.997887+0000","last_active":"2023-10-06T05:07:22.999843+0000","last_peered":"2023-10-06T05:07:22.999843+0000","last_clean":"2023-10-06T05:07:22.999843+0000","last_became_active":"2023-10-06T05:07:22.997631+0000","last_became_peered":"2023-10-06T05:07:22.997631+0000","last_unstale":"2023-10-06T05:07:22.999843+0000","last_undegraded":"2023-10-06T05:07:22.999843+0000","last_fullsized":"2023-10-06T05:07:22.999843+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:36.114697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.984712+0000","last_change":"2023-10-06T05:07:22.983043+0000","last_active":"2023-10-06T05:07:22.984712+0000","last_peered":"2023-10-06T05:07:22.984712+0000","last_clean":"2023-10-06T05:07:22.984712+0000","last_became_active":"2023-10-06T05:07:22.982751+0000","last_became_peered":"2023-10-06T05:07:22.982751+0000","last_unstale":"2023-10-06T05:07:22.984712+0000","last_undegraded":"2023-10-06T05:07:22.984712+0000","last_fullsized":"2023-10-06T05:07:22.984712+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:55.760224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.993461+0000","last_change":"2023-10-06T05:07:22.991404+0000","last_active":"2023-10-06T05:07:22.993461+0000","last_peered":"2023-10-06T05:07:22.993461+0000","last_clean":"2023-10-06T05:07:22.993461+0000","last_became_active":"2023-10-06T05:07:22.991125+0000","last_became_peered":"2023-10-06T05:07:22.991125+0000","last_unstale":"2023-10-06T05:07:22.993461+0000","last_undegraded":"2023-10-06T05:07:22.993461+0000","last_fullsized":"2023-10-06T05:07:22.993461+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:04.044282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996519+0000","last_change":"2023-10-06T05:07:22.991645+0000","last_active":"2023-10-06T05:07:22.996519+0000","last_peered":"2023-10-06T05:07:22.996519+0000","last_clean":"2023-10-06T05:07:22.996519+0000","last_became_active":"2023-10-06T05:07:22.991336+0000","last_became_peered":"2023-10-06T05:07:22.991336+0000","last_unstale":"2023-10-06T05:07:22.996519+0000","last_undegraded":"2023-10-06T05:07:22.996519+0000","last_fullsized":"2023-10-06T05:07:22.996519+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:49.756579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994995+0000","last_change":"2023-10-06T05:07:22.991271+0000","last_active":"2023-10-06T05:07:22.994995+0000","last_peered":"2023-10-06T05:07:22.994995+0000","last_clean":"2023-10-06T05:07:22.994995+0000","last_became_active":"2023-10-06T05:07:22.991004+0000","last_became_peered":"2023-10-06T05:07:22.991004+0000","last_unstale":"2023-10-06T05:07:22.994995+0000","last_undegraded":"2023-10-06T05:07:22.994995+0000","last_fullsized":"2023-10-06T05:07:22.994995+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:16.715921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.989657+0000","last_change":"2023-10-06T05:07:22.988147+0000","last_active":"2023-10-06T05:07:22.989657+0000","last_peered":"2023-10-06T05:07:22.989657+0000","last_clean":"2023-10-06T05:07:22.989657+0000","last_became_active":"2023-10-06T05:07:22.987426+0000","last_became_peered":"2023-10-06T05:07:22.987426+0000","last_unstale":"2023-10-06T05:07:22.989657+0000","last_undegraded":"2023-10-06T05:07:22.989657+0000","last_fullsized":"2023-10-06T05:07:22.989657+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:04.044282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994654+0000","last_change":"2023-10-06T05:07:22.993664+0000","last_active":"2023-10-06T05:07:22.994654+0000","last_peered":"2023-10-06T05:07:22.994654+0000","last_clean":"2023-10-06T05:07:22.994654+0000","last_became_active":"2023-10-06T05:07:22.993371+0000","last_became_peered":"2023-10-06T05:07:22.993371+0000","last_unstale":"2023-10-06T05:07:22.994654+0000","last_undegraded":"2023-10-06T05:07:22.994654+0000","last_fullsized":"2023-10-06T05:07:22.994654+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:50.062759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994955+0000","last_change":"2023-10-06T05:07:22.991327+0000","last_active":"2023-10-06T05:07:22.994955+0000","last_peered":"2023-10-06T05:07:22.994955+0000","last_clean":"2023-10-06T05:07:22.994955+0000","last_became_active":"2023-10-06T05:07:22.990999+0000","last_became_peered":"2023-10-06T05:07:22.990999+0000","last_unstale":"2023-10-06T05:07:22.994955+0000","last_undegraded":"2023-10-06T05:07:22.994955+0000","last_fullsized":"2023-10-06T05:07:22.994955+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:36.114697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.995907+0000","last_change":"2023-10-06T05:07:22.994009+0000","last_active":"2023-10-06T05:07:22.995907+0000","last_peered":"2023-10-06T05:07:22.995907+0000","last_clean":"2023-10-06T05:07:22.995907+0000","last_became_active":"2023-10-06T05:07:22.993730+0000","last_became_peered":"2023-10-06T05:07:22.993730+0000","last_unstale":"2023-10-06T05:07:22.995907+0000","last_undegraded":"2023-10-06T05:07:22.995907+0000","last_fullsized":"2023-10-06T05:07:22.995907+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:25.484832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.995076+0000","last_change":"2023-10-06T05:07:22.992053+0000","last_active":"2023-10-06T05:07:22.995076+0000","last_peered":"2023-10-06T05:07:22.995076+0000","last_clean":"2023-10-06T05:07:22.995076+0000","last_became_active":"2023-10-06T05:07:22.991759+0000","last_became_peered":"2023-10-06T05:07:22.991759+0000","last_unstale":"2023-10-06T05:07:22.995076+0000","last_undegraded":"2023-10-06T05:07:22.995076+0000","last_fullsized":"2023-10-06T05:07:22.995076+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:55.760224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996254+0000","last_change":"2023-10-06T05:07:22.993408+0000","last_active":"2023-10-06T05:07:22.996254+0000","last_peered":"2023-10-06T05:07:22.996254+0000","last_clean":"2023-10-06T05:07:22.996254+0000","last_became_active":"2023-10-06T05:07:22.991581+0000","last_became_peered":"2023-10-06T05:07:22.991581+0000","last_unstale":"2023-10-06T05:07:22.996254+0000","last_undegraded":"2023-10-06T05:07:22.996254+0000","last_fullsized":"2023-10-06T05:07:22.996254+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:46.536627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.990724+0000","last_change":"2023-10-06T05:07:22.988878+0000","last_active":"2023-10-06T05:07:22.990724+0000","last_peered":"2023-10-06T05:07:22.990724+0000","last_clean":"2023-10-06T05:07:22.990724+0000","last_became_active":"2023-10-06T05:07:22.988386+0000","last_became_peered":"2023-10-06T05:07:22.988386+0000","last_unstale":"2023-10-06T05:07:22.990724+0000","last_undegraded":"2023-10-06T05:07:22.990724+0000","last_fullsized":"2023-10-06T05:07:22.990724+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:50.960453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.995088+0000","last_change":"2023-10-06T05:07:22.992111+0000","last_active":"2023-10-06T05:07:22.995088+0000","last_peered":"2023-10-06T05:07:22.995088+0000","last_clean":"2023-10-06T05:07:22.995088+0000","last_became_active":"2023-10-06T05:07:22.991813+0000","last_became_peered":"2023-10-06T05:07:22.991813+0000","last_unstale":"2023-10-06T05:07:22.995088+0000","last_undegraded":"2023-10-06T05:07:22.995088+0000","last_fullsized":"2023-10-06T05:07:22.995088+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:50.560382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.983319+0000","last_change":"2023-10-06T05:07:22.980598+0000","last_active":"2023-10-06T05:07:22.983319+0000","last_peered":"2023-10-06T05:07:22.983319+0000","last_clean":"2023-10-06T05:07:22.983319+0000","last_became_active":"2023-10-06T05:07:22.980349+0000","last_became_peered":"2023-10-06T05:07:22.980349+0000","last_unstale":"2023-10-06T05:07:22.983319+0000","last_undegraded":"2023-10-06T05:07:22.983319+0000","last_fullsized":"2023-10-06T05:07:22.983319+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:32.561371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.983388+0000","last_change":"2023-10-06T05:07:22.980732+0000","last_active":"2023-10-06T05:07:22.983388+0000","last_peered":"2023-10-06T05:07:22.983388+0000","last_clean":"2023-10-06T05:07:22.983388+0000","last_became_active":"2023-10-06T05:07:22.980483+0000","last_became_peered":"2023-10-06T05:07:22.980483+0000","last_unstale":"2023-10-06T05:07:22.983388+0000","last_undegraded":"2023-10-06T05:07:22.983388+0000","last_fullsized":"2023-10-06T05:07:22.983388+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:45.649162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996984+0000","last_change":"2023-10-06T05:07:22.992095+0000","last_active":"2023-10-06T05:07:22.996984+0000","last_peered":"2023-10-06T05:07:22.996984+0000","last_clean":"2023-10-06T05:07:22.996984+0000","last_became_active":"2023-10-06T05:07:22.991829+0000","last_became_peered":"2023-10-06T05:07:22.991829+0000","last_unstale":"2023-10-06T05:07:22.996984+0000","last_undegraded":"2023-10-06T05:07:22.996984+0000","last_fullsized":"2023-10-06T05:07:22.996984+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:16.715921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.995202+0000","last_change":"2023-10-06T05:07:22.993233+0000","last_active":"2023-10-06T05:07:22.995202+0000","last_peered":"2023-10-06T05:07:22.995202+0000","last_clean":"2023-10-06T05:07:22.995202+0000","last_became_active":"2023-10-06T05:07:22.992975+0000","last_became_peered":"2023-10-06T05:07:22.992975+0000","last_unstale":"2023-10-06T05:07:22.995202+0000","last_undegraded":"2023-10-06T05:07:22.995202+0000","last_fullsized":"2023-10-06T05:07:22.995202+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:34.441593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.991045+0000","last_change":"2023-10-06T05:07:22.989327+0000","last_active":"2023-10-06T05:07:22.991045+0000","last_peered":"2023-10-06T05:07:22.991045+0000","last_clean":"2023-10-06T05:07:22.991045+0000","last_became_active":"2023-10-06T05:07:22.989037+0000","last_became_peered":"2023-10-06T05:07:22.989037+0000","last_unstale":"2023-10-06T05:07:22.991045+0000","last_undegraded":"2023-10-06T05:07:22.991045+0000","last_fullsized":"2023-10-06T05:07:22.991045+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:42.686463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.999157+0000","last_change":"2023-10-06T05:07:22.996658+0000","last_active":"2023-10-06T05:07:22.999157+0000","last_peered":"2023-10-06T05:07:22.999157+0000","last_clean":"2023-10-06T05:07:22.999157+0000","last_became_active":"2023-10-06T05:07:22.996353+0000","last_became_peered":"2023-10-06T05:07:22.996353+0000","last_unstale":"2023-10-06T05:07:22.999157+0000","last_undegraded":"2023-10-06T05:07:22.999157+0000","last_fullsized":"2023-10-06T05:07:22.999157+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:04.708912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994677+0000","last_change":"2023-10-06T05:07:22.991190+0000","last_active":"2023-10-06T05:07:22.994677+0000","last_peered":"2023-10-06T05:07:22.994677+0000","last_clean":"2023-10-06T05:07:22.994677+0000","last_became_active":"2023-10-06T05:07:22.990924+0000","last_became_peered":"2023-10-06T05:07:22.990924+0000","last_unstale":"2023-10-06T05:07:22.994677+0000","last_undegraded":"2023-10-06T05:07:22.994677+0000","last_fullsized":"2023-10-06T05:07:22.994677+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:55.760224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.999053+0000","last_change":"2023-10-06T05:07:22.998216+0000","last_active":"2023-10-06T05:07:22.999053+0000","last_peered":"2023-10-06T05:07:22.999053+0000","last_clean":"2023-10-06T05:07:22.999053+0000","last_became_active":"2023-10-06T05:07:22.997947+0000","last_became_peered":"2023-10-06T05:07:22.997947+0000","last_unstale":"2023-10-06T05:07:22.999053+0000","last_undegraded":"2023-10-06T05:07:22.999053+0000","last_fullsized":"2023-10-06T05:07:22.999053+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:50.960453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996675+0000","last_change":"2023-10-06T05:07:22.991470+0000","last_active":"2023-10-06T05:07:22.996675+0000","last_peered":"2023-10-06T05:07:22.996675+0000","last_clean":"2023-10-06T05:07:22.996675+0000","last_became_active":"2023-10-06T05:07:22.991159+0000","last_became_peered":"2023-10-06T05:07:22.991159+0000","last_unstale":"2023-10-06T05:07:22.996675+0000","last_undegraded":"2023-10-06T05:07:22.996675+0000","last_fullsized":"2023-10-06T05:07:22.996675+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:50.947942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996554+0000","last_change":"2023-10-06T05:07:22.990803+0000","last_active":"2023-10-06T05:07:22.996554+0000","last_peered":"2023-10-06T05:07:22.996554+0000","last_clean":"2023-10-06T05:07:22.996554+0000","last_became_active":"2023-10-06T05:07:22.990518+0000","last_became_peered":"2023-10-06T05:07:22.990518+0000","last_unstale":"2023-10-06T05:07:22.996554+0000","last_undegraded":"2023-10-06T05:07:22.996554+0000","last_fullsized":"2023-10-06T05:07:22.996554+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:08.974065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.981861+0000","last_change":"2023-10-06T05:07:22.981032+0000","last_active":"2023-10-06T05:07:22.981861+0000","last_peered":"2023-10-06T05:07:22.981861+0000","last_clean":"2023-10-06T05:07:22.981861+0000","last_became_active":"2023-10-06T05:07:22.980735+0000","last_became_peered":"2023-10-06T05:07:22.980735+0000","last_unstale":"2023-10-06T05:07:22.981861+0000","last_undegraded":"2023-10-06T05:07:22.981861+0000","last_fullsized":"2023-10-06T05:07:22.981861+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:46.916320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.989745+0000","last_change":"2023-10-06T05:07:22.988539+0000","last_active":"2023-10-06T05:07:22.989745+0000","last_peered":"2023-10-06T05:07:22.989745+0000","last_clean":"2023-10-06T05:07:22.989745+0000","last_became_active":"2023-10-06T05:07:22.988190+0000","last_became_peered":"2023-10-06T05:07:22.988190+0000","last_unstale":"2023-10-06T05:07:22.989745+0000","last_undegraded":"2023-10-06T05:07:22.989745+0000","last_fullsized":"2023-10-06T05:07:22.989745+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:17.286805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.997061+0000","last_change":"2023-10-06T05:07:22.993849+0000","last_active":"2023-10-06T05:07:22.997061+0000","last_peered":"2023-10-06T05:07:22.997061+0000","last_clean":"2023-10-06T05:07:22.997061+0000","last_became_active":"2023-10-06T05:07:22.993583+0000","last_became_peered":"2023-10-06T05:07:22.993583+0000","last_unstale":"2023-10-06T05:07:22.997061+0000","last_undegraded":"2023-10-06T05:07:22.997061+0000","last_fullsized":"2023-10-06T05:07:22.997061+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:50.560382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.977379+0000","last_change":"2023-10-06T05:07:22.252541+0000","last_active":"2023-10-06T05:07:22.977379+0000","last_peered":"2023-10-06T05:07:22.977379+0000","last_clean":"2023-10-06T05:07:22.977379+0000","last_became_active":"2023-10-06T05:07:22.252270+0000","last_became_peered":"2023-10-06T05:07:22.252270+0000","last_unstale":"2023-10-06T05:07:22.977379+0000","last_undegraded":"2023-10-06T05:07:22.977379+0000","last_fullsized":"2023-10-06T05:07:22.977379+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:46.127273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.986983+0000","last_change":"2023-10-06T05:07:22.986268+0000","last_active":"2023-10-06T05:07:22.986983+0000","last_peered":"2023-10-06T05:07:22.986983+0000","last_clean":"2023-10-06T05:07:22.986983+0000","last_became_active":"2023-10-06T05:07:22.985683+0000","last_became_peered":"2023-10-06T05:07:22.985683+0000","last_unstale":"2023-10-06T05:07:22.986983+0000","last_undegraded":"2023-10-06T05:07:22.986983+0000","last_fullsized":"2023-10-06T05:07:22.986983+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:36.114697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.977602+0000","last_change":"2023-10-06T05:07:22.249160+0000","last_active":"2023-10-06T05:07:22.977602+0000","last_peered":"2023-10-06T05:07:22.977602+0000","last_clean":"2023-10-06T05:07:22.977602+0000","last_became_active":"2023-10-06T05:07:22.248890+0000","last_became_peered":"2023-10-06T05:07:22.248890+0000","last_unstale":"2023-10-06T05:07:22.977602+0000","last_undegraded":"2023-10-06T05:07:22.977602+0000","last_fullsized":"2023-10-06T05:07:22.977602+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:58:05.453298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.999740+0000","last_change":"2023-10-06T05:07:22.997771+0000","last_active":"2023-10-06T05:07:22.999740+0000","last_peered":"2023-10-06T05:07:22.999740+0000","last_clean":"2023-10-06T05:07:22.999740+0000","last_became_active":"2023-10-06T05:07:22.997496+0000","last_became_peered":"2023-10-06T05:07:22.997496+0000","last_unstale":"2023-10-06T05:07:22.999740+0000","last_undegraded":"2023-10-06T05:07:22.999740+0000","last_fullsized":"2023-10-06T05:07:22.999740+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:52.327857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"206'3","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.238221+0000","last_change":"2023-10-06T05:07:22.253022+0000","last_active":"2023-10-06T05:07:23.238221+0000","last_peered":"2023-10-06T05:07:23.238221+0000","last_clean":"2023-10-06T05:07:23.238221+0000","last_became_active":"2023-10-06T05:07:22.252768+0000","last_became_peered":"2023-10-06T05:07:22.252768+0000","last_unstale":"2023-10-06T05:07:23.238221+0000","last_undegraded":"2023-10-06T05:07:23.238221+0000","last_fullsized":"2023-10-06T05:07:23.238221+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:35.851836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.998937+0000","last_change":"2023-10-06T05:07:22.997444+0000","last_active":"2023-10-06T05:07:22.998937+0000","last_peered":"2023-10-06T05:07:22.998937+0000","last_clean":"2023-10-06T05:07:22.998937+0000","last_became_active":"2023-10-06T05:07:22.997129+0000","last_became_peered":"2023-10-06T05:07:22.997129+0000","last_unstale":"2023-10-06T05:07:22.998937+0000","last_undegraded":"2023-10-06T05:07:22.998937+0000","last_fullsized":"2023-10-06T05:07:22.998937+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:43.015189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"206'6","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.241809+0000","last_change":"2023-10-06T05:07:22.245618+0000","last_active":"2023-10-06T05:07:23.241809+0000","last_peered":"2023-10-06T05:07:23.241809+0000","last_clean":"2023-10-06T05:07:23.241809+0000","last_became_active":"2023-10-06T05:07:22.244969+0000","last_became_peered":"2023-10-06T05:07:22.244969+0000","last_unstale":"2023-10-06T05:07:23.241809+0000","last_undegraded":"2023-10-06T05:07:23.241809+0000","last_fullsized":"2023-10-06T05:07:23.241809+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:54.121389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.993797+0000","last_change":"2023-10-06T05:07:22.993048+0000","last_active":"2023-10-06T05:07:22.993797+0000","last_peered":"2023-10-06T05:07:22.993797+0000","last_clean":"2023-10-06T05:07:22.993797+0000","last_became_active":"2023-10-06T05:07:22.992628+0000","last_became_peered":"2023-10-06T05:07:22.992628+0000","last_unstale":"2023-10-06T05:07:22.993797+0000","last_undegraded":"2023-10-06T05:07:22.993797+0000","last_fullsized":"2023-10-06T05:07:22.993797+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:53.257097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"206'2","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.238403+0000","last_change":"2023-10-06T05:07:22.249263+0000","last_active":"2023-10-06T05:07:23.238403+0000","last_peered":"2023-10-06T05:07:23.238403+0000","last_clean":"2023-10-06T05:07:23.238403+0000","last_became_active":"2023-10-06T05:07:22.248975+0000","last_became_peered":"2023-10-06T05:07:22.248975+0000","last_unstale":"2023-10-06T05:07:23.238403+0000","last_undegraded":"2023-10-06T05:07:23.238403+0000","last_fullsized":"2023-10-06T05:07:23.238403+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.549744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.985546+0000","last_change":"2023-10-06T05:07:22.982444+0000","last_active":"2023-10-06T05:07:22.985546+0000","last_peered":"2023-10-06T05:07:22.985546+0000","last_clean":"2023-10-06T05:07:22.985546+0000","last_became_active":"2023-10-06T05:07:22.982113+0000","last_became_peered":"2023-10-06T05:07:22.982113+0000","last_unstale":"2023-10-06T05:07:22.985546+0000","last_undegraded":"2023-10-06T05:07:22.985546+0000","last_fullsized":"2023-10-06T05:07:22.985546+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:16.715921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.976079+0000","last_change":"2023-10-06T05:07:22.234506+0000","last_active":"2023-10-06T05:07:22.976079+0000","last_peered":"2023-10-06T05:07:22.976079+0000","last_clean":"2023-10-06T05:07:22.976079+0000","last_became_active":"2023-10-06T05:07:22.234252+0000","last_became_peered":"2023-10-06T05:07:22.234252+0000","last_unstale":"2023-10-06T05:07:22.976079+0000","last_undegraded":"2023-10-06T05:07:22.976079+0000","last_fullsized":"2023-10-06T05:07:22.976079+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:35.650646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.995084+0000","last_change":"2023-10-06T05:07:22.990094+0000","last_active":"2023-10-06T05:07:22.995084+0000","last_peered":"2023-10-06T05:07:22.995084+0000","last_clean":"2023-10-06T05:07:22.995084+0000","last_became_active":"2023-10-06T05:07:22.989715+0000","last_became_peered":"2023-10-06T05:07:22.989715+0000","last_unstale":"2023-10-06T05:07:22.995084+0000","last_undegraded":"2023-10-06T05:07:22.995084+0000","last_fullsized":"2023-10-06T05:07:22.995084+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:20:58.784752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.978932+0000","last_change":"2023-10-06T05:07:22.239853+0000","last_active":"2023-10-06T05:07:22.978932+0000","last_peered":"2023-10-06T05:07:22.978932+0000","last_clean":"2023-10-06T05:07:22.978932+0000","last_became_active":"2023-10-06T05:07:22.239583+0000","last_became_peered":"2023-10-06T05:07:22.239583+0000","last_unstale":"2023-10-06T05:07:22.978932+0000","last_undegraded":"2023-10-06T05:07:22.978932+0000","last_fullsized":"2023-10-06T05:07:22.978932+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:48.478428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.988751+0000","last_change":"2023-10-06T05:07:22.986035+0000","last_active":"2023-10-06T05:07:22.988751+0000","last_peered":"2023-10-06T05:07:22.988751+0000","last_clean":"2023-10-06T05:07:22.988751+0000","last_became_active":"2023-10-06T05:07:22.985733+0000","last_became_peered":"2023-10-06T05:07:22.985733+0000","last_unstale":"2023-10-06T05:07:22.988751+0000","last_undegraded":"2023-10-06T05:07:22.988751+0000","last_fullsized":"2023-10-06T05:07:22.988751+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:58.355609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"206'8","reported_seq":18,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.239325+0000","last_change":"2023-10-06T05:07:22.249114+0000","last_active":"2023-10-06T05:07:23.239325+0000","last_peered":"2023-10-06T05:07:23.239325+0000","last_clean":"2023-10-06T05:07:23.239325+0000","last_became_active":"2023-10-06T05:07:22.248828+0000","last_became_peered":"2023-10-06T05:07:22.248828+0000","last_unstale":"2023-10-06T05:07:23.239325+0000","last_undegraded":"2023-10-06T05:07:23.239325+0000","last_fullsized":"2023-10-06T05:07:23.239325+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:35.316785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.984391+0000","last_change":"2023-10-06T05:07:22.981348+0000","last_active":"2023-10-06T05:07:22.984391+0000","last_peered":"2023-10-06T05:07:22.984391+0000","last_clean":"2023-10-06T05:07:22.984391+0000","last_became_active":"2023-10-06T05:07:22.981098+0000","last_became_peered":"2023-10-06T05:07:22.981098+0000","last_unstale":"2023-10-06T05:07:22.984391+0000","last_undegraded":"2023-10-06T05:07:22.984391+0000","last_fullsized":"2023-10-06T05:07:22.984391+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:27.433904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"206'1","reported_seq":11,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.238419+0000","last_change":"2023-10-06T05:07:22.250493+0000","last_active":"2023-10-06T05:07:23.238419+0000","last_peered":"2023-10-06T05:07:23.238419+0000","last_clean":"2023-10-06T05:07:23.238419+0000","last_became_active":"2023-10-06T05:07:22.250183+0000","last_became_peered":"2023-10-06T05:07:22.250183+0000","last_unstale":"2023-10-06T05:07:23.238419+0000","last_undegraded":"2023-10-06T05:07:23.238419+0000","last_fullsized":"2023-10-06T05:07:23.238419+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:09.087113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.992093+0000","last_change":"2023-10-06T05:07:22.990813+0000","last_active":"2023-10-06T05:07:22.992093+0000","last_peered":"2023-10-06T05:07:22.992093+0000","last_clean":"2023-10-06T05:07:22.992093+0000","last_became_active":"2023-10-06T05:07:22.990561+0000","last_became_peered":"2023-10-06T05:07:22.990561+0000","last_unstale":"2023-10-06T05:07:22.992093+0000","last_undegraded":"2023-10-06T05:07:22.992093+0000","last_fullsized":"2023-10-06T05:07:22.992093+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:20:58.784752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"206'2","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.239187+0000","last_change":"2023-10-06T05:07:22.240451+0000","last_active":"2023-10-06T05:07:23.239187+0000","last_peered":"2023-10-06T05:07:23.239187+0000","last_clean":"2023-10-06T05:07:23.239187+0000","last_became_active":"2023-10-06T05:07:22.239346+0000","last_became_peered":"2023-10-06T05:07:22.239346+0000","last_unstale":"2023-10-06T05:07:23.239187+0000","last_undegraded":"2023-10-06T05:07:23.239187+0000","last_fullsized":"2023-10-06T05:07:23.239187+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:44.637535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996583+0000","last_change":"2023-10-06T05:07:22.991073+0000","last_active":"2023-10-06T05:07:22.996583+0000","last_peered":"2023-10-06T05:07:22.996583+0000","last_clean":"2023-10-06T05:07:22.996583+0000","last_became_active":"2023-10-06T05:07:22.990784+0000","last_became_peered":"2023-10-06T05:07:22.990784+0000","last_unstale":"2023-10-06T05:07:22.996583+0000","last_undegraded":"2023-10-06T05:07:22.996583+0000","last_fullsized":"2023-10-06T05:07:22.996583+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:26.338526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"206'1","reported_seq":11,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.237098+0000","last_change":"2023-10-06T05:07:22.250085+0000","last_active":"2023-10-06T05:07:23.237098+0000","last_peered":"2023-10-06T05:07:23.237098+0000","last_clean":"2023-10-06T05:07:23.237098+0000","last_became_active":"2023-10-06T05:07:22.249805+0000","last_became_peered":"2023-10-06T05:07:22.249805+0000","last_unstale":"2023-10-06T05:07:23.237098+0000","last_undegraded":"2023-10-06T05:07:23.237098+0000","last_fullsized":"2023-10-06T05:07:23.237098+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:52.245469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.990455+0000","last_change":"2023-10-06T05:07:22.986800+0000","last_active":"2023-10-06T05:07:22.990455+0000","last_peered":"2023-10-06T05:07:22.990455+0000","last_clean":"2023-10-06T05:07:22.990455+0000","last_became_active":"2023-10-06T05:07:22.986362+0000","last_became_peered":"2023-10-06T05:07:22.986362+0000","last_unstale":"2023-10-06T05:07:22.990455+0000","last_undegraded":"2023-10-06T05:07:22.990455+0000","last_fullsized":"2023-10-06T05:07:22.990455+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:36.863463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.985274+0000","last_change":"2023-10-06T05:07:22.243171+0000","last_active":"2023-10-06T05:07:22.985274+0000","last_peered":"2023-10-06T05:07:22.985274+0000","last_clean":"2023-10-06T05:07:22.985274+0000","last_became_active":"2023-10-06T05:07:22.242886+0000","last_became_peered":"2023-10-06T05:07:22.242886+0000","last_unstale":"2023-10-06T05:07:22.985274+0000","last_undegraded":"2023-10-06T05:07:22.985274+0000","last_fullsized":"2023-10-06T05:07:22.985274+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:17:55.853364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.985593+0000","last_change":"2023-10-06T05:07:22.984226+0000","last_active":"2023-10-06T05:07:22.985593+0000","last_peered":"2023-10-06T05:07:22.985593+0000","last_clean":"2023-10-06T05:07:22.985593+0000","last_became_active":"2023-10-06T05:07:22.983966+0000","last_became_peered":"2023-10-06T05:07:22.983966+0000","last_unstale":"2023-10-06T05:07:22.985593+0000","last_undegraded":"2023-10-06T05:07:22.985593+0000","last_fullsized":"2023-10-06T05:07:22.985593+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:55.133017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"206'2","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.237553+0000","last_change":"2023-10-06T05:07:22.251066+0000","last_active":"2023-10-06T05:07:23.237553+0000","last_peered":"2023-10-06T05:07:23.237553+0000","last_clean":"2023-10-06T05:07:23.237553+0000","last_became_active":"2023-10-06T05:07:22.250746+0000","last_became_peered":"2023-10-06T05:07:22.250746+0000","last_unstale":"2023-10-06T05:07:23.237553+0000","last_undegraded":"2023-10-06T05:07:23.237553+0000","last_fullsized":"2023-10-06T05:07:23.237553+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.429965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.995231+0000","last_change":"2023-10-06T05:07:22.993053+0000","last_active":"2023-10-06T05:07:22.995231+0000","last_peered":"2023-10-06T05:07:22.995231+0000","last_clean":"2023-10-06T05:07:22.995231+0000","last_became_active":"2023-10-06T05:07:22.992709+0000","last_became_peered":"2023-10-06T05:07:22.992709+0000","last_unstale":"2023-10-06T05:07:22.995231+0000","last_undegraded":"2023-10-06T05:07:22.995231+0000","last_fullsized":"2023-10-06T05:07:22.995231+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:55.760224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.978661+0000","last_change":"2023-10-06T05:07:22.241530+0000","last_active":"2023-10-06T05:07:22.978661+0000","last_peered":"2023-10-06T05:07:22.978661+0000","last_clean":"2023-10-06T05:07:22.978661+0000","last_became_active":"2023-10-06T05:07:22.241279+0000","last_became_peered":"2023-10-06T05:07:22.241279+0000","last_unstale":"2023-10-06T05:07:22.978661+0000","last_undegraded":"2023-10-06T05:07:22.978661+0000","last_fullsized":"2023-10-06T05:07:22.978661+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:54.121389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996332+0000","last_change":"2023-10-06T05:07:22.995761+0000","last_active":"2023-10-06T05:07:22.996332+0000","last_peered":"2023-10-06T05:07:22.996332+0000","last_clean":"2023-10-06T05:07:22.996332+0000","last_became_active":"2023-10-06T05:07:22.995461+0000","last_became_peered":"2023-10-06T05:07:22.995461+0000","last_unstale":"2023-10-06T05:07:22.996332+0000","last_undegraded":"2023-10-06T05:07:22.996332+0000","last_fullsized":"2023-10-06T05:07:22.996332+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:25.484832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.971495+0000","last_change":"2023-10-06T05:07:22.245918+0000","last_active":"2023-10-06T05:07:22.971495+0000","last_peered":"2023-10-06T05:07:22.971495+0000","last_clean":"2023-10-06T05:07:22.971495+0000","last_became_active":"2023-10-06T05:07:22.245621+0000","last_became_peered":"2023-10-06T05:07:22.245621+0000","last_unstale":"2023-10-06T05:07:22.971495+0000","last_undegraded":"2023-10-06T05:07:22.971495+0000","last_fullsized":"2023-10-06T05:07:22.971495+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:58.677381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.987560+0000","last_change":"2023-10-06T05:07:22.983578+0000","last_active":"2023-10-06T05:07:22.987560+0000","last_peered":"2023-10-06T05:07:22.987560+0000","last_clean":"2023-10-06T05:07:22.987560+0000","last_became_active":"2023-10-06T05:07:22.983307+0000","last_became_peered":"2023-10-06T05:07:22.983307+0000","last_unstale":"2023-10-06T05:07:22.987560+0000","last_undegraded":"2023-10-06T05:07:22.987560+0000","last_fullsized":"2023-10-06T05:07:22.987560+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:36.863463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.975806+0000","last_change":"2023-10-06T05:07:22.243399+0000","last_active":"2023-10-06T05:07:22.975806+0000","last_peered":"2023-10-06T05:07:22.975806+0000","last_clean":"2023-10-06T05:07:22.975806+0000","last_became_active":"2023-10-06T05:07:22.243143+0000","last_became_peered":"2023-10-06T05:07:22.243143+0000","last_unstale":"2023-10-06T05:07:22.975806+0000","last_undegraded":"2023-10-06T05:07:22.975806+0000","last_fullsized":"2023-10-06T05:07:22.975806+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:53.431494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.998364+0000","last_change":"2023-10-06T05:07:22.997381+0000","last_active":"2023-10-06T05:07:22.998364+0000","last_peered":"2023-10-06T05:07:22.998364+0000","last_clean":"2023-10-06T05:07:22.998364+0000","last_became_active":"2023-10-06T05:07:22.997108+0000","last_became_peered":"2023-10-06T05:07:22.997108+0000","last_unstale":"2023-10-06T05:07:22.998364+0000","last_undegraded":"2023-10-06T05:07:22.998364+0000","last_fullsized":"2023-10-06T05:07:22.998364+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:23.608664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"206'2","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.236508+0000","last_change":"2023-10-06T05:07:22.246704+0000","last_active":"2023-10-06T05:07:23.236508+0000","last_peered":"2023-10-06T05:07:23.236508+0000","last_clean":"2023-10-06T05:07:23.236508+0000","last_became_active":"2023-10-06T05:07:22.246211+0000","last_became_peered":"2023-10-06T05:07:22.246211+0000","last_unstale":"2023-10-06T05:07:23.236508+0000","last_undegraded":"2023-10-06T05:07:23.236508+0000","last_fullsized":"2023-10-06T05:07:23.236508+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.429965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.984356+0000","last_change":"2023-10-06T05:07:22.982242+0000","last_active":"2023-10-06T05:07:22.984356+0000","last_peered":"2023-10-06T05:07:22.984356+0000","last_clean":"2023-10-06T05:07:22.984356+0000","last_became_active":"2023-10-06T05:07:22.981960+0000","last_became_peered":"2023-10-06T05:07:22.981960+0000","last_unstale":"2023-10-06T05:07:22.984356+0000","last_undegraded":"2023-10-06T05:07:22.984356+0000","last_fullsized":"2023-10-06T05:07:22.984356+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.611744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.971732+0000","last_change":"2023-10-06T05:07:22.247530+0000","last_active":"2023-10-06T05:07:22.971732+0000","last_peered":"2023-10-06T05:07:22.971732+0000","last_clean":"2023-10-06T05:07:22.971732+0000","last_became_active":"2023-10-06T05:07:22.247057+0000","last_became_peered":"2023-10-06T05:07:22.247057+0000","last_unstale":"2023-10-06T05:07:22.971732+0000","last_undegraded":"2023-10-06T05:07:22.971732+0000","last_fullsized":"2023-10-06T05:07:22.971732+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:56.908733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994689+0000","last_change":"2023-10-06T05:07:22.991246+0000","last_active":"2023-10-06T05:07:22.994689+0000","last_peered":"2023-10-06T05:07:22.994689+0000","last_clean":"2023-10-06T05:07:22.994689+0000","last_became_active":"2023-10-06T05:07:22.990966+0000","last_became_peered":"2023-10-06T05:07:22.990966+0000","last_unstale":"2023-10-06T05:07:22.994689+0000","last_undegraded":"2023-10-06T05:07:22.994689+0000","last_fullsized":"2023-10-06T05:07:22.994689+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:25.484832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.977216+0000","last_change":"2023-10-06T05:07:22.253563+0000","last_active":"2023-10-06T05:07:22.977216+0000","last_peered":"2023-10-06T05:07:22.977216+0000","last_clean":"2023-10-06T05:07:22.977216+0000","last_became_active":"2023-10-06T05:07:22.252014+0000","last_became_peered":"2023-10-06T05:07:22.252014+0000","last_unstale":"2023-10-06T05:07:22.977216+0000","last_undegraded":"2023-10-06T05:07:22.977216+0000","last_fullsized":"2023-10-06T05:07:22.977216+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:56.558391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.998385+0000","last_change":"2023-10-06T05:07:22.997430+0000","last_active":"2023-10-06T05:07:22.998385+0000","last_peered":"2023-10-06T05:07:22.998385+0000","last_clean":"2023-10-06T05:07:22.998385+0000","last_became_active":"2023-10-06T05:07:22.996987+0000","last_became_peered":"2023-10-06T05:07:22.996987+0000","last_unstale":"2023-10-06T05:07:22.998385+0000","last_undegraded":"2023-10-06T05:07:22.998385+0000","last_fullsized":"2023-10-06T05:07:22.998385+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:35.845988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.978187+0000","last_change":"2023-10-06T05:07:22.245801+0000","last_active":"2023-10-06T05:07:22.978187+0000","last_peered":"2023-10-06T05:07:22.978187+0000","last_clean":"2023-10-06T05:07:22.978187+0000","last_became_active":"2023-10-06T05:07:22.245510+0000","last_became_peered":"2023-10-06T05:07:22.245510+0000","last_unstale":"2023-10-06T05:07:22.978187+0000","last_undegraded":"2023-10-06T05:07:22.978187+0000","last_fullsized":"2023-10-06T05:07:22.978187+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:03.697285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.991741+0000","last_change":"2023-10-06T05:07:22.987740+0000","last_active":"2023-10-06T05:07:22.991741+0000","last_peered":"2023-10-06T05:07:22.991741+0000","last_clean":"2023-10-06T05:07:22.991741+0000","last_became_active":"2023-10-06T05:07:22.987272+0000","last_became_peered":"2023-10-06T05:07:22.987272+0000","last_unstale":"2023-10-06T05:07:22.991741+0000","last_undegraded":"2023-10-06T05:07:22.991741+0000","last_fullsized":"2023-10-06T05:07:22.991741+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:58.923948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.984020+0000","last_change":"2023-10-06T05:07:22.245459+0000","last_active":"2023-10-06T05:07:22.984020+0000","last_peered":"2023-10-06T05:07:22.984020+0000","last_clean":"2023-10-06T05:07:22.984020+0000","last_became_active":"2023-10-06T05:07:22.245139+0000","last_became_peered":"2023-10-06T05:07:22.245139+0000","last_unstale":"2023-10-06T05:07:22.984020+0000","last_undegraded":"2023-10-06T05:07:22.984020+0000","last_fullsized":"2023-10-06T05:07:22.984020+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:44.637535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.989750+0000","last_change":"2023-10-06T05:07:22.986997+0000","last_active":"2023-10-06T05:07:22.989750+0000","last_peered":"2023-10-06T05:07:22.989750+0000","last_clean":"2023-10-06T05:07:22.989750+0000","last_became_active":"2023-10-06T05:07:22.986743+0000","last_became_peered":"2023-10-06T05:07:22.986743+0000","last_unstale":"2023-10-06T05:07:22.989750+0000","last_undegraded":"2023-10-06T05:07:22.989750+0000","last_fullsized":"2023-10-06T05:07:22.989750+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:22.229060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.982724+0000","last_change":"2023-10-06T05:07:22.248671+0000","last_active":"2023-10-06T05:07:22.982724+0000","last_peered":"2023-10-06T05:07:22.982724+0000","last_clean":"2023-10-06T05:07:22.982724+0000","last_became_active":"2023-10-06T05:07:22.248428+0000","last_became_peered":"2023-10-06T05:07:22.248428+0000","last_unstale":"2023-10-06T05:07:22.982724+0000","last_undegraded":"2023-10-06T05:07:22.982724+0000","last_fullsized":"2023-10-06T05:07:22.982724+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:07.631313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.986796+0000","last_change":"2023-10-06T05:07:22.984836+0000","last_active":"2023-10-06T05:07:22.986796+0000","last_peered":"2023-10-06T05:07:22.986796+0000","last_clean":"2023-10-06T05:07:22.986796+0000","last_became_active":"2023-10-06T05:07:22.984428+0000","last_became_peered":"2023-10-06T05:07:22.984428+0000","last_unstale":"2023-10-06T05:07:22.986796+0000","last_undegraded":"2023-10-06T05:07:22.986796+0000","last_fullsized":"2023-10-06T05:07:22.986796+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:49.490055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.971965+0000","last_change":"2023-10-06T05:07:22.245009+0000","last_active":"2023-10-06T05:07:22.971965+0000","last_peered":"2023-10-06T05:07:22.971965+0000","last_clean":"2023-10-06T05:07:22.971965+0000","last_became_active":"2023-10-06T05:07:22.244713+0000","last_became_peered":"2023-10-06T05:07:22.244713+0000","last_unstale":"2023-10-06T05:07:22.971965+0000","last_undegraded":"2023-10-06T05:07:22.971965+0000","last_fullsized":"2023-10-06T05:07:22.971965+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:22.144994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.997027+0000","last_change":"2023-10-06T05:07:22.992108+0000","last_active":"2023-10-06T05:07:22.997027+0000","last_peered":"2023-10-06T05:07:22.997027+0000","last_clean":"2023-10-06T05:07:22.997027+0000","last_became_active":"2023-10-06T05:07:22.991844+0000","last_became_peered":"2023-10-06T05:07:22.991844+0000","last_unstale":"2023-10-06T05:07:22.997027+0000","last_undegraded":"2023-10-06T05:07:22.997027+0000","last_fullsized":"2023-10-06T05:07:22.997027+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:54.689049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.975914+0000","last_change":"2023-10-06T05:07:22.245770+0000","last_active":"2023-10-06T05:07:22.975914+0000","last_peered":"2023-10-06T05:07:22.975914+0000","last_clean":"2023-10-06T05:07:22.975914+0000","last_became_active":"2023-10-06T05:07:22.245457+0000","last_became_peered":"2023-10-06T05:07:22.245457+0000","last_unstale":"2023-10-06T05:07:22.975914+0000","last_undegraded":"2023-10-06T05:07:22.975914+0000","last_fullsized":"2023-10-06T05:07:22.975914+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:22.597037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.998995+0000","last_change":"2023-10-06T05:07:22.992250+0000","last_active":"2023-10-06T05:07:22.998995+0000","last_peered":"2023-10-06T05:07:22.998995+0000","last_clean":"2023-10-06T05:07:22.998995+0000","last_became_active":"2023-10-06T05:07:22.991899+0000","last_became_peered":"2023-10-06T05:07:22.991899+0000","last_unstale":"2023-10-06T05:07:22.998995+0000","last_undegraded":"2023-10-06T05:07:22.998995+0000","last_fullsized":"2023-10-06T05:07:22.998995+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:04.285399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"206'2","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.239845+0000","last_change":"2023-10-06T05:07:22.246756+0000","last_active":"2023-10-06T05:07:23.239845+0000","last_peered":"2023-10-06T05:07:23.239845+0000","last_clean":"2023-10-06T05:07:23.239845+0000","last_became_active":"2023-10-06T05:07:22.246492+0000","last_became_peered":"2023-10-06T05:07:22.246492+0000","last_unstale":"2023-10-06T05:07:23.239845+0000","last_undegraded":"2023-10-06T05:07:23.239845+0000","last_fullsized":"2023-10-06T05:07:23.239845+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:24:23.626791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":188,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.982030+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T05:07:22.982030+0000","last_peered":"2023-10-06T05:07:22.982030+0000","last_clean":"2023-10-06T05:07:22.982030+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T05:07:22.982030+0000","last_undegraded":"2023-10-06T05:07:22.982030+0000","last_fullsized":"2023-10-06T05:07:22.982030+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.998503+0000","last_change":"2023-10-06T05:07:22.997733+0000","last_active":"2023-10-06T05:07:22.998503+0000","last_peered":"2023-10-06T05:07:22.998503+0000","last_clean":"2023-10-06T05:07:22.998503+0000","last_became_active":"2023-10-06T05:07:22.997429+0000","last_became_peered":"2023-10-06T05:07:22.997429+0000","last_unstale":"2023-10-06T05:07:22.998503+0000","last_undegraded":"2023-10-06T05:07:22.998503+0000","last_fullsized":"2023-10-06T05:07:22.998503+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:34.497285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"206'2","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.239515+0000","last_change":"2023-10-06T05:07:22.243146+0000","last_active":"2023-10-06T05:07:23.239515+0000","last_peered":"2023-10-06T05:07:23.239515+0000","last_clean":"2023-10-06T05:07:23.239515+0000","last_became_active":"2023-10-06T05:07:22.242856+0000","last_became_peered":"2023-10-06T05:07:22.242856+0000","last_unstale":"2023-10-06T05:07:23.239515+0000","last_undegraded":"2023-10-06T05:07:23.239515+0000","last_fullsized":"2023-10-06T05:07:23.239515+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:42.003562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":175,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.978278+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T05:07:22.978278+0000","last_peered":"2023-10-06T05:07:22.978278+0000","last_clean":"2023-10-06T05:07:22.978278+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T05:07:22.978278+0000","last_undegraded":"2023-10-06T05:07:22.978278+0000","last_fullsized":"2023-10-06T05:07:22.978278+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.989625+0000","last_change":"2023-10-06T05:07:22.987826+0000","last_active":"2023-10-06T05:07:22.989625+0000","last_peered":"2023-10-06T05:07:22.989625+0000","last_clean":"2023-10-06T05:07:22.989625+0000","last_became_active":"2023-10-06T05:07:22.987361+0000","last_became_peered":"2023-10-06T05:07:22.987361+0000","last_unstale":"2023-10-06T05:07:22.989625+0000","last_undegraded":"2023-10-06T05:07:22.989625+0000","last_fullsized":"2023-10-06T05:07:22.989625+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:02.906690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"206'1","reported_seq":11,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.237913+0000","last_change":"2023-10-06T05:07:22.252280+0000","last_active":"2023-10-06T05:07:23.237913+0000","last_peered":"2023-10-06T05:07:23.237913+0000","last_clean":"2023-10-06T05:07:23.237913+0000","last_became_active":"2023-10-06T05:07:22.251990+0000","last_became_peered":"2023-10-06T05:07:22.251990+0000","last_unstale":"2023-10-06T05:07:23.237913+0000","last_undegraded":"2023-10-06T05:07:23.237913+0000","last_fullsized":"2023-10-06T05:07:23.237913+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:28.600117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":171,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.975618+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T05:07:22.975618+0000","last_peered":"2023-10-06T05:07:22.975618+0000","last_clean":"2023-10-06T05:07:22.975618+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T05:07:22.975618+0000","last_undegraded":"2023-10-06T05:07:22.975618+0000","last_fullsized":"2023-10-06T05:07:22.975618+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"197'348","reported_seq":1141,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.983663+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T05:07:22.983663+0000","last_peered":"2023-10-06T05:07:22.983663+0000","last_clean":"2023-10-06T05:07:22.983663+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T05:07:22.983663+0000","last_undegraded":"2023-10-06T05:07:22.983663+0000","last_fullsized":"2023-10-06T05:07:22.983663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.988007+0000","last_change":"2023-10-06T05:07:22.982519+0000","last_active":"2023-10-06T05:07:22.988007+0000","last_peered":"2023-10-06T05:07:22.988007+0000","last_clean":"2023-10-06T05:07:22.988007+0000","last_became_active":"2023-10-06T05:07:22.982260+0000","last_became_peered":"2023-10-06T05:07:22.982260+0000","last_unstale":"2023-10-06T05:07:22.988007+0000","last_undegraded":"2023-10-06T05:07:22.988007+0000","last_fullsized":"2023-10-06T05:07:22.988007+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:49.999035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"206'1","reported_seq":11,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.236852+0000","last_change":"2023-10-06T05:07:22.239525+0000","last_active":"2023-10-06T05:07:23.236852+0000","last_peered":"2023-10-06T05:07:23.236852+0000","last_clean":"2023-10-06T05:07:23.236852+0000","last_became_active":"2023-10-06T05:07:22.239167+0000","last_became_peered":"2023-10-06T05:07:22.239167+0000","last_unstale":"2023-10-06T05:07:23.236852+0000","last_undegraded":"2023-10-06T05:07:23.236852+0000","last_fullsized":"2023-10-06T05:07:23.236852+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.549744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":173,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.982820+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T05:07:22.982820+0000","last_peered":"2023-10-06T05:07:22.982820+0000","last_clean":"2023-10-06T05:07:22.982820+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T05:07:22.982820+0000","last_undegraded":"2023-10-06T05:07:22.982820+0000","last_fullsized":"2023-10-06T05:07:22.982820+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.989634+0000","last_change":"2023-10-06T05:07:22.986052+0000","last_active":"2023-10-06T05:07:22.989634+0000","last_peered":"2023-10-06T05:07:22.989634+0000","last_clean":"2023-10-06T05:07:22.989634+0000","last_became_active":"2023-10-06T05:07:22.985743+0000","last_became_peered":"2023-10-06T05:07:22.985743+0000","last_unstale":"2023-10-06T05:07:22.989634+0000","last_undegraded":"2023-10-06T05:07:22.989634+0000","last_fullsized":"2023-10-06T05:07:22.989634+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:43.329682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.978259+0000","last_change":"2023-10-06T05:07:22.242260+0000","last_active":"2023-10-06T05:07:22.978259+0000","last_peered":"2023-10-06T05:07:22.978259+0000","last_clean":"2023-10-06T05:07:22.978259+0000","last_became_active":"2023-10-06T05:07:22.242008+0000","last_became_peered":"2023-10-06T05:07:22.242008+0000","last_unstale":"2023-10-06T05:07:22.978259+0000","last_undegraded":"2023-10-06T05:07:22.978259+0000","last_fullsized":"2023-10-06T05:07:22.978259+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:07.631313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":173,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.983749+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T05:07:22.983749+0000","last_peered":"2023-10-06T05:07:22.983749+0000","last_clean":"2023-10-06T05:07:22.983749+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T05:07:22.983749+0000","last_undegraded":"2023-10-06T05:07:22.983749+0000","last_fullsized":"2023-10-06T05:07:22.983749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.995151+0000","last_change":"2023-10-06T05:07:22.991943+0000","last_active":"2023-10-06T05:07:22.995151+0000","last_peered":"2023-10-06T05:07:22.995151+0000","last_clean":"2023-10-06T05:07:22.995151+0000","last_became_active":"2023-10-06T05:07:22.991651+0000","last_became_peered":"2023-10-06T05:07:22.991651+0000","last_unstale":"2023-10-06T05:07:22.995151+0000","last_undegraded":"2023-10-06T05:07:22.995151+0000","last_fullsized":"2023-10-06T05:07:22.995151+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:59:11.511887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.978030+0000","last_change":"2023-10-06T05:07:22.243198+0000","last_active":"2023-10-06T05:07:22.978030+0000","last_peered":"2023-10-06T05:07:22.978030+0000","last_clean":"2023-10-06T05:07:22.978030+0000","last_became_active":"2023-10-06T05:07:22.242921+0000","last_became_peered":"2023-10-06T05:07:22.242921+0000","last_unstale":"2023-10-06T05:07:22.978030+0000","last_undegraded":"2023-10-06T05:07:22.978030+0000","last_fullsized":"2023-10-06T05:07:22.978030+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:35.851836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":166,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.982314+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T05:07:22.982314+0000","last_peered":"2023-10-06T05:07:22.982314+0000","last_clean":"2023-10-06T05:07:22.982314+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T05:07:22.982314+0000","last_undegraded":"2023-10-06T05:07:22.982314+0000","last_fullsized":"2023-10-06T05:07:22.982314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.990102+0000","last_change":"2023-10-06T05:07:22.987611+0000","last_active":"2023-10-06T05:07:22.990102+0000","last_peered":"2023-10-06T05:07:22.990102+0000","last_clean":"2023-10-06T05:07:22.990102+0000","last_became_active":"2023-10-06T05:07:22.987355+0000","last_became_peered":"2023-10-06T05:07:22.987355+0000","last_unstale":"2023-10-06T05:07:22.990102+0000","last_undegraded":"2023-10-06T05:07:22.990102+0000","last_fullsized":"2023-10-06T05:07:22.990102+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:54.443121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"206'1","reported_seq":11,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.238233+0000","last_change":"2023-10-06T05:07:22.252918+0000","last_active":"2023-10-06T05:07:23.238233+0000","last_peered":"2023-10-06T05:07:23.238233+0000","last_clean":"2023-10-06T05:07:23.238233+0000","last_became_active":"2023-10-06T05:07:22.252617+0000","last_became_peered":"2023-10-06T05:07:22.252617+0000","last_unstale":"2023-10-06T05:07:23.238233+0000","last_undegraded":"2023-10-06T05:07:23.238233+0000","last_fullsized":"2023-10-06T05:07:23.238233+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:10.626108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":200,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.975348+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T05:07:22.975348+0000","last_peered":"2023-10-06T05:07:22.975348+0000","last_clean":"2023-10-06T05:07:22.975348+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T05:07:22.975348+0000","last_undegraded":"2023-10-06T05:07:22.975348+0000","last_fullsized":"2023-10-06T05:07:22.975348+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994881+0000","last_change":"2023-10-06T05:07:22.990552+0000","last_active":"2023-10-06T05:07:22.994881+0000","last_peered":"2023-10-06T05:07:22.994881+0000","last_clean":"2023-10-06T05:07:22.994881+0000","last_became_active":"2023-10-06T05:07:22.990264+0000","last_became_peered":"2023-10-06T05:07:22.990264+0000","last_unstale":"2023-10-06T05:07:22.994881+0000","last_undegraded":"2023-10-06T05:07:22.994881+0000","last_fullsized":"2023-10-06T05:07:22.994881+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:17.286805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"206'4","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.237664+0000","last_change":"2023-10-06T05:07:22.246494+0000","last_active":"2023-10-06T05:07:23.237664+0000","last_peered":"2023-10-06T05:07:23.237664+0000","last_clean":"2023-10-06T05:07:23.237664+0000","last_became_active":"2023-10-06T05:07:22.245691+0000","last_became_peered":"2023-10-06T05:07:22.245691+0000","last_unstale":"2023-10-06T05:07:23.237664+0000","last_undegraded":"2023-10-06T05:07:23.237664+0000","last_fullsized":"2023-10-06T05:07:23.237664+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:22:25.877007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":200,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.975864+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T05:07:22.975864+0000","last_peered":"2023-10-06T05:07:22.975864+0000","last_clean":"2023-10-06T05:07:22.975864+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T05:07:22.975864+0000","last_undegraded":"2023-10-06T05:07:22.975864+0000","last_fullsized":"2023-10-06T05:07:22.975864+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304492,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067348,"statfs":{"total":96636764160,"available":96324964352,"internally_reserved":0,"allocated":3182592,"data_stored":1961210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94999999999999996,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0529999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79300000000000004,"15min":0.80600000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.996,"5min":1.0640000000000001,"15min":1.073},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.80300000000000005,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.0860000000000001},"last":0.442}]},{"osd":2,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72199999999999998,"15min":0.74099999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.751},{"interface":"front","average":{"1min":0.754,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.52500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0509999999999999,"5min":1.107,"15min":1.1160000000000001},"last":0.50700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.101},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.753,"15min":0.76600000000000001},"min":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96099999999999997,"5min":1.0149999999999999,"15min":1.0369999999999999},"last":0.61499999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.355},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64000000000000001,"15min":0.64500000000000002},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.40999999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.208},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47699999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.754,"15min":0.749},"min":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.48799999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.51500000000000001,"5min":0.48999999999999999,"15min":0.46700000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.80500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575966,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299824,"kb_used_data":2536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072016,"statfs":{"total":96636764160,"available":96329744384,"internally_reserved":0,"allocated":2596864,"data_stored":1371890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":1.0009999999999999,"15min":1.2969999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94999999999999996,"5min":1.083,"15min":1.083},"last":0.79100000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0640000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0960000000000001,"5min":1.425,"15min":1.425},"last":0.76900000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.752,"15min":0.76100000000000001},"min":{"1min":0.48099999999999998,"5min":0.47299999999999998,"15min":0.44700000000000001},"max":{"1min":0.91800000000000004,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96499999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78200000000000003,"15min":0.78600000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":0.998,"5min":1.016,"15min":1.1699999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.79800000000000004},"min":{"1min":0.61299999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94599999999999995,"5min":1.0409999999999999,"15min":1.1479999999999999},"last":0.88400000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.40200000000000002,"15min":0.38900000000000001},"max":{"1min":0.81399999999999995,"5min":0.95999999999999996,"15min":1.0269999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.64400000000000002},"min":{"1min":0.51800000000000002,"5min":0.46600000000000003,"15min":0.39300000000000002},"max":{"1min":0.89400000000000002,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.51400000000000001,"5min":0.5,"15min":0.41699999999999998},"max":{"1min":0.94199999999999995,"5min":1.02,"15min":1.069},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.41199999999999998},"max":{"1min":0.94199999999999995,"5min":1.0349999999999999,"15min":1.1950000000000001},"last":0.60799999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93999999999999995,"5min":1.208,"15min":1.208},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.48599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97399999999999998,"5min":1.0449999999999999,"15min":1.1579999999999999},"last":0.63100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575965,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312688,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059152,"statfs":{"total":96636764160,"available":96316571648,"internally_reserved":0,"allocated":3186688,"data_stored":1962188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.32700000000000001},"max":{"1min":1.093,"5min":1.155,"15min":1.155},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.59299999999999997},"min":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.38900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.90100000000000002},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.72699999999999998,"5min":0.76400000000000001,"15min":1.153},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.42199999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1,"5min":1.018,"15min":1.018},"last":0.50900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.435,"15min":0.41799999999999998},"max":{"1min":0.95499999999999996,"5min":1.113,"15min":1.131},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92200000000000004,"5min":0.95499999999999996,"15min":0.96999999999999997},"last":0.433}]},{"osd":4,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79600000000000004,"15min":0.80600000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.02,"5min":1.2210000000000001,"15min":1.2250000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.82199999999999995,"15min":0.84299999999999997},"min":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.147,"5min":1.1930000000000001,"15min":1.236},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.82699999999999996,"15min":0.83999999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.53600000000000003},"max":{"1min":1.0660000000000001,"5min":1.1240000000000001,"15min":1.228},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.51000000000000001},"max":{"1min":1.139,"5min":1.1799999999999999,"15min":1.2829999999999999},"last":0.873}]},{"osd":6,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85199999999999998,"15min":0.85699999999999998},"min":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.59199999999999997},"max":{"1min":1.1779999999999999,"5min":1.2490000000000001,"15min":1.2709999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.85999999999999999,"15min":0.86399999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.163,"5min":1.242,"15min":1.2470000000000001},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.63,"5min":0.623,"15min":0.623},"max":{"1min":1.198,"5min":1.222,"15min":1.2949999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.872,"15min":0.88},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.224,"5min":1.224,"15min":1.3080000000000001},"last":0.94399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575967,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303928,"kb_used_data":2544,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067912,"statfs":{"total":96636764160,"available":96325541888,"internally_reserved":0,"allocated":2605056,"data_stored":1371542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.28599999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.35999999999999999,"15min":0.34399999999999997},"max":{"1min":0.67600000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55500000000000005,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.38700000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.91000000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.86699999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.63700000000000001,"15min":0.64700000000000002},"min":{"1min":0.49299999999999999,"5min":0.439,"15min":0.42899999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.67100000000000004,"5min":0.57899999999999996,"15min":0.54300000000000004},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.3049999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.63600000000000001,"5min":0.55700000000000005,"15min":0.55300000000000005},"max":{"1min":1.085,"5min":1.127,"15min":1.3440000000000001},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.83499999999999996,"15min":0.84399999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.2729999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82899999999999996,"15min":0.85099999999999998},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.45600000000000002},"max":{"1min":1.1339999999999999,"5min":1.204,"15min":1.218},"last":0.66600000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81999999999999995,"15min":0.83799999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.53800000000000003},"max":{"1min":1.0529999999999999,"5min":1.1240000000000001,"15min":1.2869999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86199999999999999,"15min":0.88300000000000001},"min":{"1min":0.67700000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.246,"5min":1.246,"15min":1.4399999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.85399999999999998,"15min":0.876},"min":{"1min":0.626,"5min":0.624,"15min":0.59799999999999998},"max":{"1min":1.216,"5min":1.2490000000000001,"15min":1.367},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.68300000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.294},"last":0.53800000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575966,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316204,"kb_used_data":2532,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055636,"statfs":{"total":96636764160,"available":96312971264,"internally_reserved":0,"allocated":2592768,"data_stored":1371396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47299999999999998,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.32000000000000001,"15min":0.312},"max":{"1min":0.73699999999999999,"5min":0.746,"15min":0.80300000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.91200000000000003},"last":0.377}]},{"osd":1,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73899999999999999,"5min":0.78300000000000003,"15min":0.92400000000000004},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.42899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.51300000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.32400000000000001},"max":{"1min":0.76100000000000001,"5min":0.89800000000000002,"15min":0.90600000000000003},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.84799999999999998,"15min":0.85399999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0580000000000001,"5min":1.286,"15min":1.286},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.59199999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.254},"last":0.66100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.88400000000000001,"15min":0.88800000000000001},"min":{"1min":0.67100000000000004,"5min":0.59099999999999997,"15min":0.57599999999999996},"max":{"1min":1.165,"5min":1.26,"15min":1.345},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.88,"5min":0.89000000000000001,"15min":0.89700000000000002},"min":{"1min":0.65500000000000003,"5min":0.628,"15min":0.628},"max":{"1min":1.085,"5min":1.2090000000000001,"15min":1.339},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58199999999999996},"max":{"1min":0.997,"5min":1.1279999999999999,"15min":1.2989999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1439999999999999,"5min":1.226,"15min":1.226},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85999999999999999,"15min":0.86599999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1000000000000001,"5min":1.141,"15min":1.276},"last":0.752},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90700000000000003,"15min":0.90900000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.59399999999999997},"max":{"1min":1.1850000000000001,"5min":1.2170000000000001,"15min":1.3680000000000001},"last":0.58199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575966,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299828,"kb_used_data":2540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072012,"statfs":{"total":96636764160,"available":96329740288,"internally_reserved":0,"allocated":2600960,"data_stored":1371980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51100000000000001,"15min":0.49399999999999999},"min":{"1min":0.34399999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67400000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85799999999999998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.76400000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.58299999999999996},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.48899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.74199999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65000000000000002,"15min":0.63500000000000001},"min":{"1min":0.48899999999999999,"5min":0.45100000000000001,"15min":0.38500000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.83599999999999997,"15min":0.82699999999999996},"min":{"1min":0.625,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1599999999999999,"5min":1.232,"15min":1.2849999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.87},"min":{"1min":0.64800000000000002,"5min":0.53900000000000003,"15min":0.52500000000000002},"max":{"1min":1.0860000000000001,"5min":1.232,"15min":1.4019999999999999},"last":0.77700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80600000000000005,"15min":0.81999999999999995},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.52200000000000002},"max":{"1min":1.01,"5min":1.22,"15min":1.5760000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0980000000000001,"5min":1.1419999999999999,"15min":1.3300000000000001},"last":0.85699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.84899999999999998,"15min":0.85099999999999998},"min":{"1min":0.66100000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0900000000000001,"5min":1.212,"15min":1.2150000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.87,"5min":0.88700000000000001,"15min":0.89100000000000001},"min":{"1min":0.64900000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1399999999999999,"5min":1.2929999999999999,"15min":1.321},"last":0.90300000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.87,"15min":0.87},"min":{"1min":0.70299999999999996,"5min":0.59399999999999997,"15min":0.57599999999999996},"max":{"1min":1.1619999999999999,"5min":1.198,"15min":1.2769999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.86599999999999999,"15min":0.85399999999999998},"min":{"1min":0.69099999999999995,"5min":0.59299999999999997,"15min":0.55800000000000005},"max":{"1min":1.1200000000000001,"5min":1.1890000000000001,"15min":1.389},"last":0.76400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575966,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312124,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059716,"statfs":{"total":96636764160,"available":96317149184,"internally_reserved":0,"allocated":2609152,"data_stored":1373390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.72799999999999998,"15min":0.749},"min":{"1min":0.49099999999999999,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.84199999999999997,"5min":1.0820000000000001,"15min":1.2},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.75900000000000001,"15min":0.76100000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.874,"5min":1.048,"15min":1.048},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72799999999999998,"15min":0.745},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.83099999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.76900000000000002,"15min":0.78100000000000003},"min":{"1min":0.58299999999999996,"5min":0.52400000000000002,"15min":0.48499999999999999},"max":{"1min":0.89000000000000001,"5min":0.99099999999999999,"15min":1.258},"last":0.624}]},{"osd":2,"last update":"Fri Oct 6 05:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85599999999999998,"5min":1.0089999999999999,"15min":1.0309999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.76100000000000001,"15min":0.78600000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.91500000000000004,"5min":1.0049999999999999,"15min":1.2929999999999999},"last":0.92000000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.74399999999999999,"15min":0.76100000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.49299999999999999},"max":{"1min":0.79500000000000004,"5min":0.97999999999999998,"15min":1.2330000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.77500000000000002,"15min":0.79400000000000004},"min":{"1min":0.56200000000000006,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88300000000000001,"5min":0.97399999999999998,"15min":1.034},"last":0.89300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.63900000000000001},"min":{"1min":0.44400000000000001,"5min":0.41599999999999998,"15min":0.41199999999999998},"max":{"1min":0.84299999999999997,"5min":1.04,"15min":1.323},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64600000000000002,"15min":0.65100000000000002},"min":{"1min":0.45800000000000002,"5min":0.44700000000000001,"15min":0.39700000000000002},"max":{"1min":0.77800000000000002,"5min":1.01,"15min":1.1910000000000001},"last":0.74299999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66000000000000003,"15min":0.67600000000000005},"min":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42099999999999999},"max":{"1min":0.82799999999999996,"5min":1.024,"15min":1.079},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.44400000000000001,"15min":0.41999999999999998},"max":{"1min":0.85399999999999998,"5min":0.98499999999999999,"15min":1.3540000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68700000000000006,"15min":0.71199999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.80400000000000005,"5min":1.0640000000000001,"15min":1.155},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72699999999999998,"15min":0.751},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":0.84199999999999997,"5min":1.079,"15min":1.149},"last":0.93000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575966,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308024,"kb_used_data":2544,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063816,"statfs":{"total":96636764160,"available":96321347584,"internally_reserved":0,"allocated":2605056,"data_stored":1372400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.748},"min":{"1min":0.53200000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.93500000000000005,"5min":1.079,"15min":1.079},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.72999999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88800000000000001,"5min":0.999,"15min":1.02},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.745,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.53600000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98399999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.46700000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79600000000000004,"15min":0.79400000000000004},"min":{"1min":0.65300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.99199999999999999,"5min":1.0700000000000001,"15min":1.0780000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80100000000000005,"15min":0.80700000000000005},"min":{"1min":0.57599999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0149999999999999,"5min":1.048,"15min":1.048},"last":0.64300000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.48599999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.94399999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.627},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.63500000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0169999999999999,"5min":1.095,"15min":1.095},"last":0.754}]},{"osd":4,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.629},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.38200000000000001},"max":{"1min":0.84399999999999997,"5min":0.97699999999999998,"15min":1.0049999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.5,"5min":0.38,"15min":0.38},"max":{"1min":0.89000000000000001,"5min":1.01,"15min":1.042},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.48499999999999999,"5min":0.45900000000000002,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.0369999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.42799999999999999},"max":{"1min":0.90700000000000003,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.55000000000000004,"5min":0.48499999999999999,"15min":0.44500000000000001},"max":{"1min":0.92600000000000005,"5min":1.0269999999999999,"15min":1.0669999999999999},"last":0.503},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.54000000000000004,"5min":0.51300000000000001,"15min":0.47199999999999998},"max":{"1min":0.93000000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.72999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:07:27.640 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:07:27.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.859+0000 7fdc3359d700 1 Processor -- start 2023-10-06T05:07:27.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.859+0000 7fdc3359d700 1 -- start start 2023-10-06T05:07:27.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.861+0000 7fdc3359d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fdc2c06d950 0x7fdc2c0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:27.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.861+0000 7fdc3359d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2c06a0a0 0x7fdc2c0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:27.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.861+0000 7fdc3359d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdc2c06dd20 0x7fdc2c0c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:27.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.861+0000 7fdc3359d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c06ae70 con 0x7fdc2c06dd20 2023-10-06T05:07:27.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.861+0000 7fdc3359d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c06a540 con 0x7fdc2c06d950 2023-10-06T05:07:27.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.861+0000 7fdc3359d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c06a820 con 0x7fdc2c06a0a0 2023-10-06T05:07:27.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.861+0000 7fdc32d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdc2c06dd20 0x7fdc2c0c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:27.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.861+0000 7fdc32d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fdc2c06dd20 0x7fdc2c0c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43850/0 (socket says 172.21.15.55:43850) 2023-10-06T05:07:27.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.861+0000 7fdc32d9c700 1 -- 172.21.15.55:0/2467732889 learned_addr learned my addr 172.21.15.55:0/2467732889 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:27.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.861+0000 7fdc31d9a700 1 --2- 172.21.15.55:0/2467732889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2c06a0a0 0x7fdc2c0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:27.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.861+0000 7fdc3259b700 1 --2- 172.21.15.55:0/2467732889 >> v2:172.21.15.163:3300/0 conn(0x7fdc2c06d950 0x7fdc2c0c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:27.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.861+0000 7fdc32d9c700 1 -- 172.21.15.55:0/2467732889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2c06a0a0 msgr2=0x7fdc2c0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:27.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.861+0000 7fdc32d9c700 1 --2- 172.21.15.55:0/2467732889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2c06a0a0 0x7fdc2c0c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.862+0000 7fdc32d9c700 1 -- 172.21.15.55:0/2467732889 >> v2:172.21.15.163:3300/0 conn(0x7fdc2c06d950 msgr2=0x7fdc2c0c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:27.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.862+0000 7fdc32d9c700 1 --2- 172.21.15.55:0/2467732889 >> v2:172.21.15.163:3300/0 conn(0x7fdc2c06d950 0x7fdc2c0c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.862+0000 7fdc32d9c700 1 -- 172.21.15.55:0/2467732889 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c06ab00 con 0x7fdc2c06dd20 2023-10-06T05:07:27.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.862+0000 7fdc3259b700 1 --2- 172.21.15.55:0/2467732889 >> v2:172.21.15.163:3300/0 conn(0x7fdc2c06d950 0x7fdc2c0c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:07:27.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.862+0000 7fdc32d9c700 1 --2- 172.21.15.55:0/2467732889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc2c06dd20 0x7fdc2c0c7150 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0c9c70 tx=0x7fdc24009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55dd993c37799faa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:27.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.862+0000 7fdc31599700 1 -- 172.21.15.55:0/2467732889 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2400e070 con 0x7fdc2c06dd20 2023-10-06T05:07:27.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.862+0000 7fdc31599700 1 -- 172.21.15.55:0/2467732889 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2400ba50 con 0x7fdc2c06dd20 2023-10-06T05:07:27.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.862+0000 7fdc31599700 1 -- 172.21.15.55:0/2467732889 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc240092e0 con 0x7fdc2c06dd20 2023-10-06T05:07:27.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.863+0000 7fdc3359d700 1 -- 172.21.15.55:0/2467732889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc2c06dd20 msgr2=0x7fdc2c0c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:27.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.863+0000 7fdc3359d700 1 --2- 172.21.15.55:0/2467732889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc2c06dd20 0x7fdc2c0c7150 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0c9c70 tx=0x7fdc24009840 comp rx=0 tx=0).stop 2023-10-06T05:07:27.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.863+0000 7fdc3359d700 1 -- 172.21.15.55:0/2467732889 shutdown_connections 2023-10-06T05:07:27.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.863+0000 7fdc3359d700 1 --2- 172.21.15.55:0/2467732889 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc2c06dd20 0x7fdc2c0c7150 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.863+0000 7fdc3359d700 1 --2- 172.21.15.55:0/2467732889 >> v2:172.21.15.163:3300/0 conn(0x7fdc2c06d950 0x7fdc2c0c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.863+0000 7fdc3359d700 1 --2- 172.21.15.55:0/2467732889 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2c06a0a0 0x7fdc2c0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.863+0000 7fdc3359d700 1 -- 172.21.15.55:0/2467732889 >> 172.21.15.55:0/2467732889 conn(0x7fdc2c0cc1f0 msgr2=0x7fdc2c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:27.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.863+0000 7fdc3359d700 1 -- 172.21.15.55:0/2467732889 shutdown_connections 2023-10-06T05:07:27.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.863+0000 7fdc3359d700 1 -- 172.21.15.55:0/2467732889 wait complete. 2023-10-06T05:07:27.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.864+0000 7fdc3359d700 1 Processor -- start 2023-10-06T05:07:27.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.864+0000 7fdc3359d700 1 -- start start 2023-10-06T05:07:27.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.865+0000 7fdc3359d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc2c06a0a0 0x7fdc2c0b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:27.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.865+0000 7fdc3359d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2c06d950 0x7fdc2c154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:27.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.865+0000 7fdc3359d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc2c06dd20 0x7fdc2c13bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:27.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.865+0000 7fdc3359d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0c5f80 con 0x7fdc2c06a0a0 2023-10-06T05:07:27.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.865+0000 7fdc3359d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0c53b0 con 0x7fdc2c06dd20 2023-10-06T05:07:27.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.865+0000 7fdc3359d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0c5650 con 0x7fdc2c06d950 2023-10-06T05:07:27.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.866+0000 7fdc3259b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc2c06a0a0 0x7fdc2c0b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:27.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.866+0000 7fdc3259b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc2c06a0a0 0x7fdc2c0b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43854/0 (socket says 172.21.15.55:43854) 2023-10-06T05:07:27.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.866+0000 7fdc3259b700 1 -- 172.21.15.55:0/1264397280 learned_addr learned my addr 172.21.15.55:0/1264397280 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:27.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.866+0000 7fdc32d9c700 1 --2- 172.21.15.55:0/1264397280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc2c06dd20 0x7fdc2c13bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:27.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.866+0000 7fdc31d9a700 1 --2- 172.21.15.55:0/1264397280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2c06d950 0x7fdc2c154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:27.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.866+0000 7fdc3259b700 1 -- 172.21.15.55:0/1264397280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2c06d950 msgr2=0x7fdc2c154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:27.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.866+0000 7fdc3259b700 1 --2- 172.21.15.55:0/1264397280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2c06d950 0x7fdc2c154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.866+0000 7fdc3259b700 1 -- 172.21.15.55:0/1264397280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc2c06dd20 msgr2=0x7fdc2c13bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:27.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.866+0000 7fdc3259b700 1 --2- 172.21.15.55:0/1264397280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc2c06dd20 0x7fdc2c13bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:27.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.866+0000 7fdc3259b700 1 -- 172.21.15.55:0/1264397280 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc24009ae0 con 0x7fdc2c06a0a0 2023-10-06T05:07:27.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.866+0000 7fdc3259b700 1 --2- 172.21.15.55:0/1264397280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc2c06a0a0 0x7fdc2c0b1820 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800c830 tx=0x7fdc28008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:27.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.867+0000 7fdc227fc700 1 -- 172.21.15.55:0/1264397280 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28011600 con 0x7fdc2c06a0a0 2023-10-06T05:07:27.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.867+0000 7fdc227fc700 1 -- 172.21.15.55:0/1264397280 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc28011760 con 0x7fdc2c06a0a0 2023-10-06T05:07:27.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.867+0000 7fdc227fc700 1 -- 172.21.15.55:0/1264397280 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28011b20 con 0x7fdc2c06a0a0 2023-10-06T05:07:27.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.867+0000 7fdc3359d700 1 -- 172.21.15.55:0/1264397280 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c0c5930 con 0x7fdc2c06a0a0 2023-10-06T05:07:27.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.867+0000 7fdc3359d700 1 -- 172.21.15.55:0/1264397280 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c0b1d60 con 0x7fdc2c06a0a0 2023-10-06T05:07:27.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.868+0000 7fdc3359d700 1 -- 172.21.15.55:0/1264397280 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2c04c3c0 con 0x7fdc2c06a0a0 2023-10-06T05:07:27.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.869+0000 7fdc227fc700 1 -- 172.21.15.55:0/1264397280 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdc28018360 con 0x7fdc2c06a0a0 2023-10-06T05:07:27.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.870+0000 7fdc227fc700 1 --2- 172.21.15.55:0/1264397280 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdc10066f30 0x7fdc100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:27.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.870+0000 7fdc227fc700 1 -- 172.21.15.55:0/1264397280 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fdc28084800 con 0x7fdc2c06a0a0 2023-10-06T05:07:27.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.870+0000 7fdc31d9a700 1 --2- 172.21.15.55:0/1264397280 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdc10066f30 0x7fdc100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:27.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.871+0000 7fdc31d9a700 1 --2- 172.21.15.55:0/1264397280 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdc10066f30 0x7fdc100693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0c7860 tx=0x7fdc1c008040 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:27.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:27.874+0000 7fdc227fc700 1 -- 172.21.15.55:0/1264397280 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc28051730 con 0x7fdc2c06a0a0 2023-10-06T05:07:28.072 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.071+0000 7fdc3359d700 1 -- 172.21.15.55:0/1264397280 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc2c0c60c0 con 0x7fdc10066f30 2023-10-06T05:07:28.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.080+0000 7fdc227fc700 1 -- 172.21.15.55:0/1264397280 <== mgr.9544 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311686 (secure 0 0 0) 0x7fdc1c007da0 con 0x7fdc10066f30 2023-10-06T05:07:28.086 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:28.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.085+0000 7fdc3359d700 1 -- 172.21.15.55:0/1264397280 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdc10066f30 msgr2=0x7fdc100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:28.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.085+0000 7fdc3359d700 1 --2- 172.21.15.55:0/1264397280 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdc10066f30 0x7fdc100693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0c7860 tx=0x7fdc1c008040 comp rx=0 tx=0).stop 2023-10-06T05:07:28.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.085+0000 7fdc3359d700 1 -- 172.21.15.55:0/1264397280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc2c06a0a0 msgr2=0x7fdc2c0b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:28.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.085+0000 7fdc3359d700 1 --2- 172.21.15.55:0/1264397280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc2c06a0a0 0x7fdc2c0b1820 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800c830 tx=0x7fdc28008040 comp rx=0 tx=0).stop 2023-10-06T05:07:28.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.085+0000 7fdc3359d700 1 -- 172.21.15.55:0/1264397280 shutdown_connections 2023-10-06T05:07:28.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.085+0000 7fdc3359d700 1 --2- 172.21.15.55:0/1264397280 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fdc10066f30 0x7fdc100693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.085+0000 7fdc3359d700 1 --2- 172.21.15.55:0/1264397280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fdc2c06dd20 0x7fdc2c13bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.085+0000 7fdc3359d700 1 --2- 172.21.15.55:0/1264397280 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fdc2c06a0a0 0x7fdc2c0b1820 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.085+0000 7fdc3359d700 1 --2- 172.21.15.55:0/1264397280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fdc2c06d950 0x7fdc2c154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.085+0000 7fdc3359d700 1 -- 172.21.15.55:0/1264397280 >> 172.21.15.55:0/1264397280 conn(0x7fdc2c0cc1f0 msgr2=0x7fdc2c0c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:28.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.085+0000 7fdc3359d700 1 -- 172.21.15.55:0/1264397280 shutdown_connections 2023-10-06T05:07:28.090 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.085+0000 7fdc3359d700 1 -- 172.21.15.55:0/1264397280 wait complete. 2023-10-06T05:07:28.090 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T05:07:28.105 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-06T05:07:26.167609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":623,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":386,"ondisk_log_size":386,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":193,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2457112,"kb_used_data":21464,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752517608,"statfs":{"total":773094113280,"available":770578030592,"internally_reserved":0,"allocated":21979136,"data_stored":12155996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.950411"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994948+0000","last_change":"2023-10-06T05:07:22.992773+0000","last_active":"2023-10-06T05:07:22.994948+0000","last_peered":"2023-10-06T05:07:22.994948+0000","last_clean":"2023-10-06T05:07:22.994948+0000","last_became_active":"2023-10-06T05:07:22.992522+0000","last_became_peered":"2023-10-06T05:07:22.992522+0000","last_unstale":"2023-10-06T05:07:22.994948+0000","last_undegraded":"2023-10-06T05:07:22.994948+0000","last_fullsized":"2023-10-06T05:07:22.994948+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:44:46.673236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.998948+0000","last_change":"2023-10-06T05:07:22.997391+0000","last_active":"2023-10-06T05:07:22.998948+0000","last_peered":"2023-10-06T05:07:22.998948+0000","last_clean":"2023-10-06T05:07:22.998948+0000","last_became_active":"2023-10-06T05:07:22.997114+0000","last_became_peered":"2023-10-06T05:07:22.997114+0000","last_unstale":"2023-10-06T05:07:22.998948+0000","last_undegraded":"2023-10-06T05:07:22.998948+0000","last_fullsized":"2023-10-06T05:07:22.998948+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:25.484832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.984731+0000","last_change":"2023-10-06T05:07:22.981446+0000","last_active":"2023-10-06T05:07:22.984731+0000","last_peered":"2023-10-06T05:07:22.984731+0000","last_clean":"2023-10-06T05:07:22.984731+0000","last_became_active":"2023-10-06T05:07:22.981175+0000","last_became_peered":"2023-10-06T05:07:22.981175+0000","last_unstale":"2023-10-06T05:07:22.984731+0000","last_undegraded":"2023-10-06T05:07:22.984731+0000","last_fullsized":"2023-10-06T05:07:22.984731+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:34:20.552363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.991409+0000","last_change":"2023-10-06T05:07:22.990554+0000","last_active":"2023-10-06T05:07:22.991409+0000","last_peered":"2023-10-06T05:07:22.991409+0000","last_clean":"2023-10-06T05:07:22.991409+0000","last_became_active":"2023-10-06T05:07:22.990271+0000","last_became_peered":"2023-10-06T05:07:22.990271+0000","last_unstale":"2023-10-06T05:07:22.991409+0000","last_undegraded":"2023-10-06T05:07:22.991409+0000","last_fullsized":"2023-10-06T05:07:22.991409+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:50.560382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996000+0000","last_change":"2023-10-06T05:07:22.995029+0000","last_active":"2023-10-06T05:07:22.996000+0000","last_peered":"2023-10-06T05:07:22.996000+0000","last_clean":"2023-10-06T05:07:22.996000+0000","last_became_active":"2023-10-06T05:07:22.994716+0000","last_became_peered":"2023-10-06T05:07:22.994716+0000","last_unstale":"2023-10-06T05:07:22.996000+0000","last_undegraded":"2023-10-06T05:07:22.996000+0000","last_fullsized":"2023-10-06T05:07:22.996000+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:22.229060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.985808+0000","last_change":"2023-10-06T05:07:22.981616+0000","last_active":"2023-10-06T05:07:22.985808+0000","last_peered":"2023-10-06T05:07:22.985808+0000","last_clean":"2023-10-06T05:07:22.985808+0000","last_became_active":"2023-10-06T05:07:22.981350+0000","last_became_peered":"2023-10-06T05:07:22.981350+0000","last_unstale":"2023-10-06T05:07:22.985808+0000","last_undegraded":"2023-10-06T05:07:22.985808+0000","last_fullsized":"2023-10-06T05:07:22.985808+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:54.689049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994794+0000","last_change":"2023-10-06T05:07:22.994181+0000","last_active":"2023-10-06T05:07:22.994794+0000","last_peered":"2023-10-06T05:07:22.994794+0000","last_clean":"2023-10-06T05:07:22.994794+0000","last_became_active":"2023-10-06T05:07:22.993934+0000","last_became_peered":"2023-10-06T05:07:22.993934+0000","last_unstale":"2023-10-06T05:07:22.994794+0000","last_undegraded":"2023-10-06T05:07:22.994794+0000","last_fullsized":"2023-10-06T05:07:22.994794+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:31.816501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.999843+0000","last_change":"2023-10-06T05:07:22.997887+0000","last_active":"2023-10-06T05:07:22.999843+0000","last_peered":"2023-10-06T05:07:22.999843+0000","last_clean":"2023-10-06T05:07:22.999843+0000","last_became_active":"2023-10-06T05:07:22.997631+0000","last_became_peered":"2023-10-06T05:07:22.997631+0000","last_unstale":"2023-10-06T05:07:22.999843+0000","last_undegraded":"2023-10-06T05:07:22.999843+0000","last_fullsized":"2023-10-06T05:07:22.999843+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:36.114697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.984712+0000","last_change":"2023-10-06T05:07:22.983043+0000","last_active":"2023-10-06T05:07:22.984712+0000","last_peered":"2023-10-06T05:07:22.984712+0000","last_clean":"2023-10-06T05:07:22.984712+0000","last_became_active":"2023-10-06T05:07:22.982751+0000","last_became_peered":"2023-10-06T05:07:22.982751+0000","last_unstale":"2023-10-06T05:07:22.984712+0000","last_undegraded":"2023-10-06T05:07:22.984712+0000","last_fullsized":"2023-10-06T05:07:22.984712+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:55.760224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.993461+0000","last_change":"2023-10-06T05:07:22.991404+0000","last_active":"2023-10-06T05:07:22.993461+0000","last_peered":"2023-10-06T05:07:22.993461+0000","last_clean":"2023-10-06T05:07:22.993461+0000","last_became_active":"2023-10-06T05:07:22.991125+0000","last_became_peered":"2023-10-06T05:07:22.991125+0000","last_unstale":"2023-10-06T05:07:22.993461+0000","last_undegraded":"2023-10-06T05:07:22.993461+0000","last_fullsized":"2023-10-06T05:07:22.993461+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:04.044282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996519+0000","last_change":"2023-10-06T05:07:22.991645+0000","last_active":"2023-10-06T05:07:22.996519+0000","last_peered":"2023-10-06T05:07:22.996519+0000","last_clean":"2023-10-06T05:07:22.996519+0000","last_became_active":"2023-10-06T05:07:22.991336+0000","last_became_peered":"2023-10-06T05:07:22.991336+0000","last_unstale":"2023-10-06T05:07:22.996519+0000","last_undegraded":"2023-10-06T05:07:22.996519+0000","last_fullsized":"2023-10-06T05:07:22.996519+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:20:49.756579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994995+0000","last_change":"2023-10-06T05:07:22.991271+0000","last_active":"2023-10-06T05:07:22.994995+0000","last_peered":"2023-10-06T05:07:22.994995+0000","last_clean":"2023-10-06T05:07:22.994995+0000","last_became_active":"2023-10-06T05:07:22.991004+0000","last_became_peered":"2023-10-06T05:07:22.991004+0000","last_unstale":"2023-10-06T05:07:22.994995+0000","last_undegraded":"2023-10-06T05:07:22.994995+0000","last_fullsized":"2023-10-06T05:07:22.994995+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:16.715921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.989657+0000","last_change":"2023-10-06T05:07:22.988147+0000","last_active":"2023-10-06T05:07:22.989657+0000","last_peered":"2023-10-06T05:07:22.989657+0000","last_clean":"2023-10-06T05:07:22.989657+0000","last_became_active":"2023-10-06T05:07:22.987426+0000","last_became_peered":"2023-10-06T05:07:22.987426+0000","last_unstale":"2023-10-06T05:07:22.989657+0000","last_undegraded":"2023-10-06T05:07:22.989657+0000","last_fullsized":"2023-10-06T05:07:22.989657+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:39:04.044282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994654+0000","last_change":"2023-10-06T05:07:22.993664+0000","last_active":"2023-10-06T05:07:22.994654+0000","last_peered":"2023-10-06T05:07:22.994654+0000","last_clean":"2023-10-06T05:07:22.994654+0000","last_became_active":"2023-10-06T05:07:22.993371+0000","last_became_peered":"2023-10-06T05:07:22.993371+0000","last_unstale":"2023-10-06T05:07:22.994654+0000","last_undegraded":"2023-10-06T05:07:22.994654+0000","last_fullsized":"2023-10-06T05:07:22.994654+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:33:50.062759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994955+0000","last_change":"2023-10-06T05:07:22.991327+0000","last_active":"2023-10-06T05:07:22.994955+0000","last_peered":"2023-10-06T05:07:22.994955+0000","last_clean":"2023-10-06T05:07:22.994955+0000","last_became_active":"2023-10-06T05:07:22.990999+0000","last_became_peered":"2023-10-06T05:07:22.990999+0000","last_unstale":"2023-10-06T05:07:22.994955+0000","last_undegraded":"2023-10-06T05:07:22.994955+0000","last_fullsized":"2023-10-06T05:07:22.994955+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:36.114697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.995907+0000","last_change":"2023-10-06T05:07:22.994009+0000","last_active":"2023-10-06T05:07:22.995907+0000","last_peered":"2023-10-06T05:07:22.995907+0000","last_clean":"2023-10-06T05:07:22.995907+0000","last_became_active":"2023-10-06T05:07:22.993730+0000","last_became_peered":"2023-10-06T05:07:22.993730+0000","last_unstale":"2023-10-06T05:07:22.995907+0000","last_undegraded":"2023-10-06T05:07:22.995907+0000","last_fullsized":"2023-10-06T05:07:22.995907+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:25.484832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.995076+0000","last_change":"2023-10-06T05:07:22.992053+0000","last_active":"2023-10-06T05:07:22.995076+0000","last_peered":"2023-10-06T05:07:22.995076+0000","last_clean":"2023-10-06T05:07:22.995076+0000","last_became_active":"2023-10-06T05:07:22.991759+0000","last_became_peered":"2023-10-06T05:07:22.991759+0000","last_unstale":"2023-10-06T05:07:22.995076+0000","last_undegraded":"2023-10-06T05:07:22.995076+0000","last_fullsized":"2023-10-06T05:07:22.995076+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:55.760224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996254+0000","last_change":"2023-10-06T05:07:22.993408+0000","last_active":"2023-10-06T05:07:22.996254+0000","last_peered":"2023-10-06T05:07:22.996254+0000","last_clean":"2023-10-06T05:07:22.996254+0000","last_became_active":"2023-10-06T05:07:22.991581+0000","last_became_peered":"2023-10-06T05:07:22.991581+0000","last_unstale":"2023-10-06T05:07:22.996254+0000","last_undegraded":"2023-10-06T05:07:22.996254+0000","last_fullsized":"2023-10-06T05:07:22.996254+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:15:46.536627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.990724+0000","last_change":"2023-10-06T05:07:22.988878+0000","last_active":"2023-10-06T05:07:22.990724+0000","last_peered":"2023-10-06T05:07:22.990724+0000","last_clean":"2023-10-06T05:07:22.990724+0000","last_became_active":"2023-10-06T05:07:22.988386+0000","last_became_peered":"2023-10-06T05:07:22.988386+0000","last_unstale":"2023-10-06T05:07:22.990724+0000","last_undegraded":"2023-10-06T05:07:22.990724+0000","last_fullsized":"2023-10-06T05:07:22.990724+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:50.960453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.995088+0000","last_change":"2023-10-06T05:07:22.992111+0000","last_active":"2023-10-06T05:07:22.995088+0000","last_peered":"2023-10-06T05:07:22.995088+0000","last_clean":"2023-10-06T05:07:22.995088+0000","last_became_active":"2023-10-06T05:07:22.991813+0000","last_became_peered":"2023-10-06T05:07:22.991813+0000","last_unstale":"2023-10-06T05:07:22.995088+0000","last_undegraded":"2023-10-06T05:07:22.995088+0000","last_fullsized":"2023-10-06T05:07:22.995088+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:50.560382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.983319+0000","last_change":"2023-10-06T05:07:22.980598+0000","last_active":"2023-10-06T05:07:22.983319+0000","last_peered":"2023-10-06T05:07:22.983319+0000","last_clean":"2023-10-06T05:07:22.983319+0000","last_became_active":"2023-10-06T05:07:22.980349+0000","last_became_peered":"2023-10-06T05:07:22.980349+0000","last_unstale":"2023-10-06T05:07:22.983319+0000","last_undegraded":"2023-10-06T05:07:22.983319+0000","last_fullsized":"2023-10-06T05:07:22.983319+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:32.561371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.983388+0000","last_change":"2023-10-06T05:07:22.980732+0000","last_active":"2023-10-06T05:07:22.983388+0000","last_peered":"2023-10-06T05:07:22.983388+0000","last_clean":"2023-10-06T05:07:22.983388+0000","last_became_active":"2023-10-06T05:07:22.980483+0000","last_became_peered":"2023-10-06T05:07:22.980483+0000","last_unstale":"2023-10-06T05:07:22.983388+0000","last_undegraded":"2023-10-06T05:07:22.983388+0000","last_fullsized":"2023-10-06T05:07:22.983388+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:45.649162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996984+0000","last_change":"2023-10-06T05:07:22.992095+0000","last_active":"2023-10-06T05:07:22.996984+0000","last_peered":"2023-10-06T05:07:22.996984+0000","last_clean":"2023-10-06T05:07:22.996984+0000","last_became_active":"2023-10-06T05:07:22.991829+0000","last_became_peered":"2023-10-06T05:07:22.991829+0000","last_unstale":"2023-10-06T05:07:22.996984+0000","last_undegraded":"2023-10-06T05:07:22.996984+0000","last_fullsized":"2023-10-06T05:07:22.996984+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:16.715921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.995202+0000","last_change":"2023-10-06T05:07:22.993233+0000","last_active":"2023-10-06T05:07:22.995202+0000","last_peered":"2023-10-06T05:07:22.995202+0000","last_clean":"2023-10-06T05:07:22.995202+0000","last_became_active":"2023-10-06T05:07:22.992975+0000","last_became_peered":"2023-10-06T05:07:22.992975+0000","last_unstale":"2023-10-06T05:07:22.995202+0000","last_undegraded":"2023-10-06T05:07:22.995202+0000","last_fullsized":"2023-10-06T05:07:22.995202+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:34.441593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.991045+0000","last_change":"2023-10-06T05:07:22.989327+0000","last_active":"2023-10-06T05:07:22.991045+0000","last_peered":"2023-10-06T05:07:22.991045+0000","last_clean":"2023-10-06T05:07:22.991045+0000","last_became_active":"2023-10-06T05:07:22.989037+0000","last_became_peered":"2023-10-06T05:07:22.989037+0000","last_unstale":"2023-10-06T05:07:22.991045+0000","last_undegraded":"2023-10-06T05:07:22.991045+0000","last_fullsized":"2023-10-06T05:07:22.991045+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:32:42.686463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.999157+0000","last_change":"2023-10-06T05:07:22.996658+0000","last_active":"2023-10-06T05:07:22.999157+0000","last_peered":"2023-10-06T05:07:22.999157+0000","last_clean":"2023-10-06T05:07:22.999157+0000","last_became_active":"2023-10-06T05:07:22.996353+0000","last_became_peered":"2023-10-06T05:07:22.996353+0000","last_unstale":"2023-10-06T05:07:22.999157+0000","last_undegraded":"2023-10-06T05:07:22.999157+0000","last_fullsized":"2023-10-06T05:07:22.999157+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:04.708912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994677+0000","last_change":"2023-10-06T05:07:22.991190+0000","last_active":"2023-10-06T05:07:22.994677+0000","last_peered":"2023-10-06T05:07:22.994677+0000","last_clean":"2023-10-06T05:07:22.994677+0000","last_became_active":"2023-10-06T05:07:22.990924+0000","last_became_peered":"2023-10-06T05:07:22.990924+0000","last_unstale":"2023-10-06T05:07:22.994677+0000","last_undegraded":"2023-10-06T05:07:22.994677+0000","last_fullsized":"2023-10-06T05:07:22.994677+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:55.760224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.999053+0000","last_change":"2023-10-06T05:07:22.998216+0000","last_active":"2023-10-06T05:07:22.999053+0000","last_peered":"2023-10-06T05:07:22.999053+0000","last_clean":"2023-10-06T05:07:22.999053+0000","last_became_active":"2023-10-06T05:07:22.997947+0000","last_became_peered":"2023-10-06T05:07:22.997947+0000","last_unstale":"2023-10-06T05:07:22.999053+0000","last_undegraded":"2023-10-06T05:07:22.999053+0000","last_fullsized":"2023-10-06T05:07:22.999053+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:50.960453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996675+0000","last_change":"2023-10-06T05:07:22.991470+0000","last_active":"2023-10-06T05:07:22.996675+0000","last_peered":"2023-10-06T05:07:22.996675+0000","last_clean":"2023-10-06T05:07:22.996675+0000","last_became_active":"2023-10-06T05:07:22.991159+0000","last_became_peered":"2023-10-06T05:07:22.991159+0000","last_unstale":"2023-10-06T05:07:22.996675+0000","last_undegraded":"2023-10-06T05:07:22.996675+0000","last_fullsized":"2023-10-06T05:07:22.996675+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:25:50.947942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996554+0000","last_change":"2023-10-06T05:07:22.990803+0000","last_active":"2023-10-06T05:07:22.996554+0000","last_peered":"2023-10-06T05:07:22.996554+0000","last_clean":"2023-10-06T05:07:22.996554+0000","last_became_active":"2023-10-06T05:07:22.990518+0000","last_became_peered":"2023-10-06T05:07:22.990518+0000","last_unstale":"2023-10-06T05:07:22.996554+0000","last_undegraded":"2023-10-06T05:07:22.996554+0000","last_fullsized":"2023-10-06T05:07:22.996554+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:11:08.974065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.981861+0000","last_change":"2023-10-06T05:07:22.981032+0000","last_active":"2023-10-06T05:07:22.981861+0000","last_peered":"2023-10-06T05:07:22.981861+0000","last_clean":"2023-10-06T05:07:22.981861+0000","last_became_active":"2023-10-06T05:07:22.980735+0000","last_became_peered":"2023-10-06T05:07:22.980735+0000","last_unstale":"2023-10-06T05:07:22.981861+0000","last_undegraded":"2023-10-06T05:07:22.981861+0000","last_fullsized":"2023-10-06T05:07:22.981861+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:43:46.916320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.989745+0000","last_change":"2023-10-06T05:07:22.988539+0000","last_active":"2023-10-06T05:07:22.989745+0000","last_peered":"2023-10-06T05:07:22.989745+0000","last_clean":"2023-10-06T05:07:22.989745+0000","last_became_active":"2023-10-06T05:07:22.988190+0000","last_became_peered":"2023-10-06T05:07:22.988190+0000","last_unstale":"2023-10-06T05:07:22.989745+0000","last_undegraded":"2023-10-06T05:07:22.989745+0000","last_fullsized":"2023-10-06T05:07:22.989745+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:17.286805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.997061+0000","last_change":"2023-10-06T05:07:22.993849+0000","last_active":"2023-10-06T05:07:22.997061+0000","last_peered":"2023-10-06T05:07:22.997061+0000","last_clean":"2023-10-06T05:07:22.997061+0000","last_became_active":"2023-10-06T05:07:22.993583+0000","last_became_peered":"2023-10-06T05:07:22.993583+0000","last_unstale":"2023-10-06T05:07:22.997061+0000","last_undegraded":"2023-10-06T05:07:22.997061+0000","last_fullsized":"2023-10-06T05:07:22.997061+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:43:50.560382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.977379+0000","last_change":"2023-10-06T05:07:22.252541+0000","last_active":"2023-10-06T05:07:22.977379+0000","last_peered":"2023-10-06T05:07:22.977379+0000","last_clean":"2023-10-06T05:07:22.977379+0000","last_became_active":"2023-10-06T05:07:22.252270+0000","last_became_peered":"2023-10-06T05:07:22.252270+0000","last_unstale":"2023-10-06T05:07:22.977379+0000","last_undegraded":"2023-10-06T05:07:22.977379+0000","last_fullsized":"2023-10-06T05:07:22.977379+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:23:46.127273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.986983+0000","last_change":"2023-10-06T05:07:22.986268+0000","last_active":"2023-10-06T05:07:22.986983+0000","last_peered":"2023-10-06T05:07:22.986983+0000","last_clean":"2023-10-06T05:07:22.986983+0000","last_became_active":"2023-10-06T05:07:22.985683+0000","last_became_peered":"2023-10-06T05:07:22.985683+0000","last_unstale":"2023-10-06T05:07:22.986983+0000","last_undegraded":"2023-10-06T05:07:22.986983+0000","last_fullsized":"2023-10-06T05:07:22.986983+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:15:36.114697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.977602+0000","last_change":"2023-10-06T05:07:22.249160+0000","last_active":"2023-10-06T05:07:22.977602+0000","last_peered":"2023-10-06T05:07:22.977602+0000","last_clean":"2023-10-06T05:07:22.977602+0000","last_became_active":"2023-10-06T05:07:22.248890+0000","last_became_peered":"2023-10-06T05:07:22.248890+0000","last_unstale":"2023-10-06T05:07:22.977602+0000","last_undegraded":"2023-10-06T05:07:22.977602+0000","last_fullsized":"2023-10-06T05:07:22.977602+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:58:05.453298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.999740+0000","last_change":"2023-10-06T05:07:22.997771+0000","last_active":"2023-10-06T05:07:22.999740+0000","last_peered":"2023-10-06T05:07:22.999740+0000","last_clean":"2023-10-06T05:07:22.999740+0000","last_became_active":"2023-10-06T05:07:22.997496+0000","last_became_peered":"2023-10-06T05:07:22.997496+0000","last_unstale":"2023-10-06T05:07:22.999740+0000","last_undegraded":"2023-10-06T05:07:22.999740+0000","last_fullsized":"2023-10-06T05:07:22.999740+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:38:52.327857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"206'3","reported_seq":13,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.238221+0000","last_change":"2023-10-06T05:07:22.253022+0000","last_active":"2023-10-06T05:07:23.238221+0000","last_peered":"2023-10-06T05:07:23.238221+0000","last_clean":"2023-10-06T05:07:23.238221+0000","last_became_active":"2023-10-06T05:07:22.252768+0000","last_became_peered":"2023-10-06T05:07:22.252768+0000","last_unstale":"2023-10-06T05:07:23.238221+0000","last_undegraded":"2023-10-06T05:07:23.238221+0000","last_fullsized":"2023-10-06T05:07:23.238221+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:35.851836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.998937+0000","last_change":"2023-10-06T05:07:22.997444+0000","last_active":"2023-10-06T05:07:22.998937+0000","last_peered":"2023-10-06T05:07:22.998937+0000","last_clean":"2023-10-06T05:07:22.998937+0000","last_became_active":"2023-10-06T05:07:22.997129+0000","last_became_peered":"2023-10-06T05:07:22.997129+0000","last_unstale":"2023-10-06T05:07:22.998937+0000","last_undegraded":"2023-10-06T05:07:22.998937+0000","last_fullsized":"2023-10-06T05:07:22.998937+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:43.015189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"206'6","reported_seq":16,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.241809+0000","last_change":"2023-10-06T05:07:22.245618+0000","last_active":"2023-10-06T05:07:23.241809+0000","last_peered":"2023-10-06T05:07:23.241809+0000","last_clean":"2023-10-06T05:07:23.241809+0000","last_became_active":"2023-10-06T05:07:22.244969+0000","last_became_peered":"2023-10-06T05:07:22.244969+0000","last_unstale":"2023-10-06T05:07:23.241809+0000","last_undegraded":"2023-10-06T05:07:23.241809+0000","last_fullsized":"2023-10-06T05:07:23.241809+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:54.121389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.993797+0000","last_change":"2023-10-06T05:07:22.993048+0000","last_active":"2023-10-06T05:07:22.993797+0000","last_peered":"2023-10-06T05:07:22.993797+0000","last_clean":"2023-10-06T05:07:22.993797+0000","last_became_active":"2023-10-06T05:07:22.992628+0000","last_became_peered":"2023-10-06T05:07:22.992628+0000","last_unstale":"2023-10-06T05:07:22.993797+0000","last_undegraded":"2023-10-06T05:07:22.993797+0000","last_fullsized":"2023-10-06T05:07:22.993797+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:53.257097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"206'2","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.238403+0000","last_change":"2023-10-06T05:07:22.249263+0000","last_active":"2023-10-06T05:07:23.238403+0000","last_peered":"2023-10-06T05:07:23.238403+0000","last_clean":"2023-10-06T05:07:23.238403+0000","last_became_active":"2023-10-06T05:07:22.248975+0000","last_became_peered":"2023-10-06T05:07:22.248975+0000","last_unstale":"2023-10-06T05:07:23.238403+0000","last_undegraded":"2023-10-06T05:07:23.238403+0000","last_fullsized":"2023-10-06T05:07:23.238403+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.549744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.985546+0000","last_change":"2023-10-06T05:07:22.982444+0000","last_active":"2023-10-06T05:07:22.985546+0000","last_peered":"2023-10-06T05:07:22.985546+0000","last_clean":"2023-10-06T05:07:22.985546+0000","last_became_active":"2023-10-06T05:07:22.982113+0000","last_became_peered":"2023-10-06T05:07:22.982113+0000","last_unstale":"2023-10-06T05:07:22.985546+0000","last_undegraded":"2023-10-06T05:07:22.985546+0000","last_fullsized":"2023-10-06T05:07:22.985546+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:22:16.715921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.976079+0000","last_change":"2023-10-06T05:07:22.234506+0000","last_active":"2023-10-06T05:07:22.976079+0000","last_peered":"2023-10-06T05:07:22.976079+0000","last_clean":"2023-10-06T05:07:22.976079+0000","last_became_active":"2023-10-06T05:07:22.234252+0000","last_became_peered":"2023-10-06T05:07:22.234252+0000","last_unstale":"2023-10-06T05:07:22.976079+0000","last_undegraded":"2023-10-06T05:07:22.976079+0000","last_fullsized":"2023-10-06T05:07:22.976079+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:35:35.650646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.995084+0000","last_change":"2023-10-06T05:07:22.990094+0000","last_active":"2023-10-06T05:07:22.995084+0000","last_peered":"2023-10-06T05:07:22.995084+0000","last_clean":"2023-10-06T05:07:22.995084+0000","last_became_active":"2023-10-06T05:07:22.989715+0000","last_became_peered":"2023-10-06T05:07:22.989715+0000","last_unstale":"2023-10-06T05:07:22.995084+0000","last_undegraded":"2023-10-06T05:07:22.995084+0000","last_fullsized":"2023-10-06T05:07:22.995084+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:20:58.784752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.978932+0000","last_change":"2023-10-06T05:07:22.239853+0000","last_active":"2023-10-06T05:07:22.978932+0000","last_peered":"2023-10-06T05:07:22.978932+0000","last_clean":"2023-10-06T05:07:22.978932+0000","last_became_active":"2023-10-06T05:07:22.239583+0000","last_became_peered":"2023-10-06T05:07:22.239583+0000","last_unstale":"2023-10-06T05:07:22.978932+0000","last_undegraded":"2023-10-06T05:07:22.978932+0000","last_fullsized":"2023-10-06T05:07:22.978932+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:48.478428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.988751+0000","last_change":"2023-10-06T05:07:22.986035+0000","last_active":"2023-10-06T05:07:22.988751+0000","last_peered":"2023-10-06T05:07:22.988751+0000","last_clean":"2023-10-06T05:07:22.988751+0000","last_became_active":"2023-10-06T05:07:22.985733+0000","last_became_peered":"2023-10-06T05:07:22.985733+0000","last_unstale":"2023-10-06T05:07:22.988751+0000","last_undegraded":"2023-10-06T05:07:22.988751+0000","last_fullsized":"2023-10-06T05:07:22.988751+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:42:58.355609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"206'8","reported_seq":18,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.239325+0000","last_change":"2023-10-06T05:07:22.249114+0000","last_active":"2023-10-06T05:07:23.239325+0000","last_peered":"2023-10-06T05:07:23.239325+0000","last_clean":"2023-10-06T05:07:23.239325+0000","last_became_active":"2023-10-06T05:07:22.248828+0000","last_became_peered":"2023-10-06T05:07:22.248828+0000","last_unstale":"2023-10-06T05:07:23.239325+0000","last_undegraded":"2023-10-06T05:07:23.239325+0000","last_fullsized":"2023-10-06T05:07:23.239325+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:35.316785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.984391+0000","last_change":"2023-10-06T05:07:22.981348+0000","last_active":"2023-10-06T05:07:22.984391+0000","last_peered":"2023-10-06T05:07:22.984391+0000","last_clean":"2023-10-06T05:07:22.984391+0000","last_became_active":"2023-10-06T05:07:22.981098+0000","last_became_peered":"2023-10-06T05:07:22.981098+0000","last_unstale":"2023-10-06T05:07:22.984391+0000","last_undegraded":"2023-10-06T05:07:22.984391+0000","last_fullsized":"2023-10-06T05:07:22.984391+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:56:27.433904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"206'1","reported_seq":11,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.238419+0000","last_change":"2023-10-06T05:07:22.250493+0000","last_active":"2023-10-06T05:07:23.238419+0000","last_peered":"2023-10-06T05:07:23.238419+0000","last_clean":"2023-10-06T05:07:23.238419+0000","last_became_active":"2023-10-06T05:07:22.250183+0000","last_became_peered":"2023-10-06T05:07:22.250183+0000","last_unstale":"2023-10-06T05:07:23.238419+0000","last_undegraded":"2023-10-06T05:07:23.238419+0000","last_fullsized":"2023-10-06T05:07:23.238419+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:23:09.087113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.992093+0000","last_change":"2023-10-06T05:07:22.990813+0000","last_active":"2023-10-06T05:07:22.992093+0000","last_peered":"2023-10-06T05:07:22.992093+0000","last_clean":"2023-10-06T05:07:22.992093+0000","last_became_active":"2023-10-06T05:07:22.990561+0000","last_became_peered":"2023-10-06T05:07:22.990561+0000","last_unstale":"2023-10-06T05:07:22.992093+0000","last_undegraded":"2023-10-06T05:07:22.992093+0000","last_fullsized":"2023-10-06T05:07:22.992093+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:20:58.784752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"206'2","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.239187+0000","last_change":"2023-10-06T05:07:22.240451+0000","last_active":"2023-10-06T05:07:23.239187+0000","last_peered":"2023-10-06T05:07:23.239187+0000","last_clean":"2023-10-06T05:07:23.239187+0000","last_became_active":"2023-10-06T05:07:22.239346+0000","last_became_peered":"2023-10-06T05:07:22.239346+0000","last_unstale":"2023-10-06T05:07:23.239187+0000","last_undegraded":"2023-10-06T05:07:23.239187+0000","last_fullsized":"2023-10-06T05:07:23.239187+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:44.637535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996583+0000","last_change":"2023-10-06T05:07:22.991073+0000","last_active":"2023-10-06T05:07:22.996583+0000","last_peered":"2023-10-06T05:07:22.996583+0000","last_clean":"2023-10-06T05:07:22.996583+0000","last_became_active":"2023-10-06T05:07:22.990784+0000","last_became_peered":"2023-10-06T05:07:22.990784+0000","last_unstale":"2023-10-06T05:07:22.996583+0000","last_undegraded":"2023-10-06T05:07:22.996583+0000","last_fullsized":"2023-10-06T05:07:22.996583+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:02:26.338526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"206'1","reported_seq":11,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.237098+0000","last_change":"2023-10-06T05:07:22.250085+0000","last_active":"2023-10-06T05:07:23.237098+0000","last_peered":"2023-10-06T05:07:23.237098+0000","last_clean":"2023-10-06T05:07:23.237098+0000","last_became_active":"2023-10-06T05:07:22.249805+0000","last_became_peered":"2023-10-06T05:07:22.249805+0000","last_unstale":"2023-10-06T05:07:23.237098+0000","last_undegraded":"2023-10-06T05:07:23.237098+0000","last_fullsized":"2023-10-06T05:07:23.237098+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:52.245469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.990455+0000","last_change":"2023-10-06T05:07:22.986800+0000","last_active":"2023-10-06T05:07:22.990455+0000","last_peered":"2023-10-06T05:07:22.990455+0000","last_clean":"2023-10-06T05:07:22.990455+0000","last_became_active":"2023-10-06T05:07:22.986362+0000","last_became_peered":"2023-10-06T05:07:22.986362+0000","last_unstale":"2023-10-06T05:07:22.990455+0000","last_undegraded":"2023-10-06T05:07:22.990455+0000","last_fullsized":"2023-10-06T05:07:22.990455+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:36.863463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.985274+0000","last_change":"2023-10-06T05:07:22.243171+0000","last_active":"2023-10-06T05:07:22.985274+0000","last_peered":"2023-10-06T05:07:22.985274+0000","last_clean":"2023-10-06T05:07:22.985274+0000","last_became_active":"2023-10-06T05:07:22.242886+0000","last_became_peered":"2023-10-06T05:07:22.242886+0000","last_unstale":"2023-10-06T05:07:22.985274+0000","last_undegraded":"2023-10-06T05:07:22.985274+0000","last_fullsized":"2023-10-06T05:07:22.985274+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:17:55.853364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.985593+0000","last_change":"2023-10-06T05:07:22.984226+0000","last_active":"2023-10-06T05:07:22.985593+0000","last_peered":"2023-10-06T05:07:22.985593+0000","last_clean":"2023-10-06T05:07:22.985593+0000","last_became_active":"2023-10-06T05:07:22.983966+0000","last_became_peered":"2023-10-06T05:07:22.983966+0000","last_unstale":"2023-10-06T05:07:22.985593+0000","last_undegraded":"2023-10-06T05:07:22.985593+0000","last_fullsized":"2023-10-06T05:07:22.985593+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:55.133017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"206'2","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.237553+0000","last_change":"2023-10-06T05:07:22.251066+0000","last_active":"2023-10-06T05:07:23.237553+0000","last_peered":"2023-10-06T05:07:23.237553+0000","last_clean":"2023-10-06T05:07:23.237553+0000","last_became_active":"2023-10-06T05:07:22.250746+0000","last_became_peered":"2023-10-06T05:07:22.250746+0000","last_unstale":"2023-10-06T05:07:23.237553+0000","last_undegraded":"2023-10-06T05:07:23.237553+0000","last_fullsized":"2023-10-06T05:07:23.237553+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.429965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.995231+0000","last_change":"2023-10-06T05:07:22.993053+0000","last_active":"2023-10-06T05:07:22.995231+0000","last_peered":"2023-10-06T05:07:22.995231+0000","last_clean":"2023-10-06T05:07:22.995231+0000","last_became_active":"2023-10-06T05:07:22.992709+0000","last_became_peered":"2023-10-06T05:07:22.992709+0000","last_unstale":"2023-10-06T05:07:22.995231+0000","last_undegraded":"2023-10-06T05:07:22.995231+0000","last_fullsized":"2023-10-06T05:07:22.995231+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:49:55.760224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.978661+0000","last_change":"2023-10-06T05:07:22.241530+0000","last_active":"2023-10-06T05:07:22.978661+0000","last_peered":"2023-10-06T05:07:22.978661+0000","last_clean":"2023-10-06T05:07:22.978661+0000","last_became_active":"2023-10-06T05:07:22.241279+0000","last_became_peered":"2023-10-06T05:07:22.241279+0000","last_unstale":"2023-10-06T05:07:22.978661+0000","last_undegraded":"2023-10-06T05:07:22.978661+0000","last_fullsized":"2023-10-06T05:07:22.978661+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:54:54.121389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.996332+0000","last_change":"2023-10-06T05:07:22.995761+0000","last_active":"2023-10-06T05:07:22.996332+0000","last_peered":"2023-10-06T05:07:22.996332+0000","last_clean":"2023-10-06T05:07:22.996332+0000","last_became_active":"2023-10-06T05:07:22.995461+0000","last_became_peered":"2023-10-06T05:07:22.995461+0000","last_unstale":"2023-10-06T05:07:22.996332+0000","last_undegraded":"2023-10-06T05:07:22.996332+0000","last_fullsized":"2023-10-06T05:07:22.996332+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:25.484832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.971495+0000","last_change":"2023-10-06T05:07:22.245918+0000","last_active":"2023-10-06T05:07:22.971495+0000","last_peered":"2023-10-06T05:07:22.971495+0000","last_clean":"2023-10-06T05:07:22.971495+0000","last_became_active":"2023-10-06T05:07:22.245621+0000","last_became_peered":"2023-10-06T05:07:22.245621+0000","last_unstale":"2023-10-06T05:07:22.971495+0000","last_undegraded":"2023-10-06T05:07:22.971495+0000","last_fullsized":"2023-10-06T05:07:22.971495+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:58.677381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.987560+0000","last_change":"2023-10-06T05:07:22.983578+0000","last_active":"2023-10-06T05:07:22.987560+0000","last_peered":"2023-10-06T05:07:22.987560+0000","last_clean":"2023-10-06T05:07:22.987560+0000","last_became_active":"2023-10-06T05:07:22.983307+0000","last_became_peered":"2023-10-06T05:07:22.983307+0000","last_unstale":"2023-10-06T05:07:22.987560+0000","last_undegraded":"2023-10-06T05:07:22.987560+0000","last_fullsized":"2023-10-06T05:07:22.987560+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:36.863463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.975806+0000","last_change":"2023-10-06T05:07:22.243399+0000","last_active":"2023-10-06T05:07:22.975806+0000","last_peered":"2023-10-06T05:07:22.975806+0000","last_clean":"2023-10-06T05:07:22.975806+0000","last_became_active":"2023-10-06T05:07:22.243143+0000","last_became_peered":"2023-10-06T05:07:22.243143+0000","last_unstale":"2023-10-06T05:07:22.975806+0000","last_undegraded":"2023-10-06T05:07:22.975806+0000","last_fullsized":"2023-10-06T05:07:22.975806+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:53.431494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.998364+0000","last_change":"2023-10-06T05:07:22.997381+0000","last_active":"2023-10-06T05:07:22.998364+0000","last_peered":"2023-10-06T05:07:22.998364+0000","last_clean":"2023-10-06T05:07:22.998364+0000","last_became_active":"2023-10-06T05:07:22.997108+0000","last_became_peered":"2023-10-06T05:07:22.997108+0000","last_unstale":"2023-10-06T05:07:22.998364+0000","last_undegraded":"2023-10-06T05:07:22.998364+0000","last_fullsized":"2023-10-06T05:07:22.998364+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:23.608664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"206'2","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.236508+0000","last_change":"2023-10-06T05:07:22.246704+0000","last_active":"2023-10-06T05:07:23.236508+0000","last_peered":"2023-10-06T05:07:23.236508+0000","last_clean":"2023-10-06T05:07:23.236508+0000","last_became_active":"2023-10-06T05:07:22.246211+0000","last_became_peered":"2023-10-06T05:07:22.246211+0000","last_unstale":"2023-10-06T05:07:23.236508+0000","last_undegraded":"2023-10-06T05:07:23.236508+0000","last_fullsized":"2023-10-06T05:07:23.236508+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:33.429965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.984356+0000","last_change":"2023-10-06T05:07:22.982242+0000","last_active":"2023-10-06T05:07:22.984356+0000","last_peered":"2023-10-06T05:07:22.984356+0000","last_clean":"2023-10-06T05:07:22.984356+0000","last_became_active":"2023-10-06T05:07:22.981960+0000","last_became_peered":"2023-10-06T05:07:22.981960+0000","last_unstale":"2023-10-06T05:07:22.984356+0000","last_undegraded":"2023-10-06T05:07:22.984356+0000","last_fullsized":"2023-10-06T05:07:22.984356+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:29.611744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.971732+0000","last_change":"2023-10-06T05:07:22.247530+0000","last_active":"2023-10-06T05:07:22.971732+0000","last_peered":"2023-10-06T05:07:22.971732+0000","last_clean":"2023-10-06T05:07:22.971732+0000","last_became_active":"2023-10-06T05:07:22.247057+0000","last_became_peered":"2023-10-06T05:07:22.247057+0000","last_unstale":"2023-10-06T05:07:22.971732+0000","last_undegraded":"2023-10-06T05:07:22.971732+0000","last_fullsized":"2023-10-06T05:07:22.971732+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:27:56.908733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994689+0000","last_change":"2023-10-06T05:07:22.991246+0000","last_active":"2023-10-06T05:07:22.994689+0000","last_peered":"2023-10-06T05:07:22.994689+0000","last_clean":"2023-10-06T05:07:22.994689+0000","last_became_active":"2023-10-06T05:07:22.990966+0000","last_became_peered":"2023-10-06T05:07:22.990966+0000","last_unstale":"2023-10-06T05:07:22.994689+0000","last_undegraded":"2023-10-06T05:07:22.994689+0000","last_fullsized":"2023-10-06T05:07:22.994689+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:19:25.484832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.977216+0000","last_change":"2023-10-06T05:07:22.253563+0000","last_active":"2023-10-06T05:07:22.977216+0000","last_peered":"2023-10-06T05:07:22.977216+0000","last_clean":"2023-10-06T05:07:22.977216+0000","last_became_active":"2023-10-06T05:07:22.252014+0000","last_became_peered":"2023-10-06T05:07:22.252014+0000","last_unstale":"2023-10-06T05:07:22.977216+0000","last_undegraded":"2023-10-06T05:07:22.977216+0000","last_fullsized":"2023-10-06T05:07:22.977216+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:00:56.558391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.998385+0000","last_change":"2023-10-06T05:07:22.997430+0000","last_active":"2023-10-06T05:07:22.998385+0000","last_peered":"2023-10-06T05:07:22.998385+0000","last_clean":"2023-10-06T05:07:22.998385+0000","last_became_active":"2023-10-06T05:07:22.996987+0000","last_became_peered":"2023-10-06T05:07:22.996987+0000","last_unstale":"2023-10-06T05:07:22.998385+0000","last_undegraded":"2023-10-06T05:07:22.998385+0000","last_fullsized":"2023-10-06T05:07:22.998385+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:34:35.845988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.978187+0000","last_change":"2023-10-06T05:07:22.245801+0000","last_active":"2023-10-06T05:07:22.978187+0000","last_peered":"2023-10-06T05:07:22.978187+0000","last_clean":"2023-10-06T05:07:22.978187+0000","last_became_active":"2023-10-06T05:07:22.245510+0000","last_became_peered":"2023-10-06T05:07:22.245510+0000","last_unstale":"2023-10-06T05:07:22.978187+0000","last_undegraded":"2023-10-06T05:07:22.978187+0000","last_fullsized":"2023-10-06T05:07:22.978187+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:44:03.697285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.991741+0000","last_change":"2023-10-06T05:07:22.987740+0000","last_active":"2023-10-06T05:07:22.991741+0000","last_peered":"2023-10-06T05:07:22.991741+0000","last_clean":"2023-10-06T05:07:22.991741+0000","last_became_active":"2023-10-06T05:07:22.987272+0000","last_became_peered":"2023-10-06T05:07:22.987272+0000","last_unstale":"2023-10-06T05:07:22.991741+0000","last_undegraded":"2023-10-06T05:07:22.991741+0000","last_fullsized":"2023-10-06T05:07:22.991741+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:48:58.923948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.984020+0000","last_change":"2023-10-06T05:07:22.245459+0000","last_active":"2023-10-06T05:07:22.984020+0000","last_peered":"2023-10-06T05:07:22.984020+0000","last_clean":"2023-10-06T05:07:22.984020+0000","last_became_active":"2023-10-06T05:07:22.245139+0000","last_became_peered":"2023-10-06T05:07:22.245139+0000","last_unstale":"2023-10-06T05:07:22.984020+0000","last_undegraded":"2023-10-06T05:07:22.984020+0000","last_fullsized":"2023-10-06T05:07:22.984020+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:41:44.637535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.989750+0000","last_change":"2023-10-06T05:07:22.986997+0000","last_active":"2023-10-06T05:07:22.989750+0000","last_peered":"2023-10-06T05:07:22.989750+0000","last_clean":"2023-10-06T05:07:22.989750+0000","last_became_active":"2023-10-06T05:07:22.986743+0000","last_became_peered":"2023-10-06T05:07:22.986743+0000","last_unstale":"2023-10-06T05:07:22.989750+0000","last_undegraded":"2023-10-06T05:07:22.989750+0000","last_fullsized":"2023-10-06T05:07:22.989750+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:19:22.229060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.982724+0000","last_change":"2023-10-06T05:07:22.248671+0000","last_active":"2023-10-06T05:07:22.982724+0000","last_peered":"2023-10-06T05:07:22.982724+0000","last_clean":"2023-10-06T05:07:22.982724+0000","last_became_active":"2023-10-06T05:07:22.248428+0000","last_became_peered":"2023-10-06T05:07:22.248428+0000","last_unstale":"2023-10-06T05:07:22.982724+0000","last_undegraded":"2023-10-06T05:07:22.982724+0000","last_fullsized":"2023-10-06T05:07:22.982724+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:07.631313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.986796+0000","last_change":"2023-10-06T05:07:22.984836+0000","last_active":"2023-10-06T05:07:22.986796+0000","last_peered":"2023-10-06T05:07:22.986796+0000","last_clean":"2023-10-06T05:07:22.986796+0000","last_became_active":"2023-10-06T05:07:22.984428+0000","last_became_peered":"2023-10-06T05:07:22.984428+0000","last_unstale":"2023-10-06T05:07:22.986796+0000","last_undegraded":"2023-10-06T05:07:22.986796+0000","last_fullsized":"2023-10-06T05:07:22.986796+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:51:49.490055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.971965+0000","last_change":"2023-10-06T05:07:22.245009+0000","last_active":"2023-10-06T05:07:22.971965+0000","last_peered":"2023-10-06T05:07:22.971965+0000","last_clean":"2023-10-06T05:07:22.971965+0000","last_became_active":"2023-10-06T05:07:22.244713+0000","last_became_peered":"2023-10-06T05:07:22.244713+0000","last_unstale":"2023-10-06T05:07:22.971965+0000","last_undegraded":"2023-10-06T05:07:22.971965+0000","last_fullsized":"2023-10-06T05:07:22.971965+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:37:22.144994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.997027+0000","last_change":"2023-10-06T05:07:22.992108+0000","last_active":"2023-10-06T05:07:22.997027+0000","last_peered":"2023-10-06T05:07:22.997027+0000","last_clean":"2023-10-06T05:07:22.997027+0000","last_became_active":"2023-10-06T05:07:22.991844+0000","last_became_peered":"2023-10-06T05:07:22.991844+0000","last_unstale":"2023-10-06T05:07:22.997027+0000","last_undegraded":"2023-10-06T05:07:22.997027+0000","last_fullsized":"2023-10-06T05:07:22.997027+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:05:54.689049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.975914+0000","last_change":"2023-10-06T05:07:22.245770+0000","last_active":"2023-10-06T05:07:22.975914+0000","last_peered":"2023-10-06T05:07:22.975914+0000","last_clean":"2023-10-06T05:07:22.975914+0000","last_became_active":"2023-10-06T05:07:22.245457+0000","last_became_peered":"2023-10-06T05:07:22.245457+0000","last_unstale":"2023-10-06T05:07:22.975914+0000","last_undegraded":"2023-10-06T05:07:22.975914+0000","last_fullsized":"2023-10-06T05:07:22.975914+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:36:22.597037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.998995+0000","last_change":"2023-10-06T05:07:22.992250+0000","last_active":"2023-10-06T05:07:22.998995+0000","last_peered":"2023-10-06T05:07:22.998995+0000","last_clean":"2023-10-06T05:07:22.998995+0000","last_became_active":"2023-10-06T05:07:22.991899+0000","last_became_peered":"2023-10-06T05:07:22.991899+0000","last_unstale":"2023-10-06T05:07:22.998995+0000","last_undegraded":"2023-10-06T05:07:22.998995+0000","last_fullsized":"2023-10-06T05:07:22.998995+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:06:04.285399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"206'2","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.239845+0000","last_change":"2023-10-06T05:07:22.246756+0000","last_active":"2023-10-06T05:07:23.239845+0000","last_peered":"2023-10-06T05:07:23.239845+0000","last_clean":"2023-10-06T05:07:23.239845+0000","last_became_active":"2023-10-06T05:07:22.246492+0000","last_became_peered":"2023-10-06T05:07:22.246492+0000","last_unstale":"2023-10-06T05:07:23.239845+0000","last_undegraded":"2023-10-06T05:07:23.239845+0000","last_fullsized":"2023-10-06T05:07:23.239845+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:24:23.626791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":188,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.982030+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T05:07:22.982030+0000","last_peered":"2023-10-06T05:07:22.982030+0000","last_clean":"2023-10-06T05:07:22.982030+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T05:07:22.982030+0000","last_undegraded":"2023-10-06T05:07:22.982030+0000","last_fullsized":"2023-10-06T05:07:22.982030+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.998503+0000","last_change":"2023-10-06T05:07:22.997733+0000","last_active":"2023-10-06T05:07:22.998503+0000","last_peered":"2023-10-06T05:07:22.998503+0000","last_clean":"2023-10-06T05:07:22.998503+0000","last_became_active":"2023-10-06T05:07:22.997429+0000","last_became_peered":"2023-10-06T05:07:22.997429+0000","last_unstale":"2023-10-06T05:07:22.998503+0000","last_undegraded":"2023-10-06T05:07:22.998503+0000","last_fullsized":"2023-10-06T05:07:22.998503+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:30:34.497285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"206'2","reported_seq":12,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.239515+0000","last_change":"2023-10-06T05:07:22.243146+0000","last_active":"2023-10-06T05:07:23.239515+0000","last_peered":"2023-10-06T05:07:23.239515+0000","last_clean":"2023-10-06T05:07:23.239515+0000","last_became_active":"2023-10-06T05:07:22.242856+0000","last_became_peered":"2023-10-06T05:07:22.242856+0000","last_unstale":"2023-10-06T05:07:23.239515+0000","last_undegraded":"2023-10-06T05:07:23.239515+0000","last_fullsized":"2023-10-06T05:07:23.239515+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:10:42.003562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":175,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.978278+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T05:07:22.978278+0000","last_peered":"2023-10-06T05:07:22.978278+0000","last_clean":"2023-10-06T05:07:22.978278+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T05:07:22.978278+0000","last_undegraded":"2023-10-06T05:07:22.978278+0000","last_fullsized":"2023-10-06T05:07:22.978278+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.989625+0000","last_change":"2023-10-06T05:07:22.987826+0000","last_active":"2023-10-06T05:07:22.989625+0000","last_peered":"2023-10-06T05:07:22.989625+0000","last_clean":"2023-10-06T05:07:22.989625+0000","last_became_active":"2023-10-06T05:07:22.987361+0000","last_became_peered":"2023-10-06T05:07:22.987361+0000","last_unstale":"2023-10-06T05:07:22.989625+0000","last_undegraded":"2023-10-06T05:07:22.989625+0000","last_fullsized":"2023-10-06T05:07:22.989625+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:16:02.906690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"206'1","reported_seq":11,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.237913+0000","last_change":"2023-10-06T05:07:22.252280+0000","last_active":"2023-10-06T05:07:23.237913+0000","last_peered":"2023-10-06T05:07:23.237913+0000","last_clean":"2023-10-06T05:07:23.237913+0000","last_became_active":"2023-10-06T05:07:22.251990+0000","last_became_peered":"2023-10-06T05:07:22.251990+0000","last_unstale":"2023-10-06T05:07:23.237913+0000","last_undegraded":"2023-10-06T05:07:23.237913+0000","last_fullsized":"2023-10-06T05:07:23.237913+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:12:28.600117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":171,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.975618+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T05:07:22.975618+0000","last_peered":"2023-10-06T05:07:22.975618+0000","last_clean":"2023-10-06T05:07:22.975618+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T05:07:22.975618+0000","last_undegraded":"2023-10-06T05:07:22.975618+0000","last_fullsized":"2023-10-06T05:07:22.975618+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"197'348","reported_seq":1141,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.983663+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T05:07:22.983663+0000","last_peered":"2023-10-06T05:07:22.983663+0000","last_clean":"2023-10-06T05:07:22.983663+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T05:07:22.983663+0000","last_undegraded":"2023-10-06T05:07:22.983663+0000","last_fullsized":"2023-10-06T05:07:22.983663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.988007+0000","last_change":"2023-10-06T05:07:22.982519+0000","last_active":"2023-10-06T05:07:22.988007+0000","last_peered":"2023-10-06T05:07:22.988007+0000","last_clean":"2023-10-06T05:07:22.988007+0000","last_became_active":"2023-10-06T05:07:22.982260+0000","last_became_peered":"2023-10-06T05:07:22.982260+0000","last_unstale":"2023-10-06T05:07:22.988007+0000","last_undegraded":"2023-10-06T05:07:22.988007+0000","last_fullsized":"2023-10-06T05:07:22.988007+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:47:49.999035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"206'1","reported_seq":11,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.236852+0000","last_change":"2023-10-06T05:07:22.239525+0000","last_active":"2023-10-06T05:07:23.236852+0000","last_peered":"2023-10-06T05:07:23.236852+0000","last_clean":"2023-10-06T05:07:23.236852+0000","last_became_active":"2023-10-06T05:07:22.239167+0000","last_became_peered":"2023-10-06T05:07:22.239167+0000","last_unstale":"2023-10-06T05:07:23.236852+0000","last_undegraded":"2023-10-06T05:07:23.236852+0000","last_fullsized":"2023-10-06T05:07:23.236852+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:46:31.549744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":173,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.982820+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T05:07:22.982820+0000","last_peered":"2023-10-06T05:07:22.982820+0000","last_clean":"2023-10-06T05:07:22.982820+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T05:07:22.982820+0000","last_undegraded":"2023-10-06T05:07:22.982820+0000","last_fullsized":"2023-10-06T05:07:22.982820+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.989634+0000","last_change":"2023-10-06T05:07:22.986052+0000","last_active":"2023-10-06T05:07:22.989634+0000","last_peered":"2023-10-06T05:07:22.989634+0000","last_clean":"2023-10-06T05:07:22.989634+0000","last_became_active":"2023-10-06T05:07:22.985743+0000","last_became_peered":"2023-10-06T05:07:22.985743+0000","last_unstale":"2023-10-06T05:07:22.989634+0000","last_undegraded":"2023-10-06T05:07:22.989634+0000","last_fullsized":"2023-10-06T05:07:22.989634+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:33:43.329682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.978259+0000","last_change":"2023-10-06T05:07:22.242260+0000","last_active":"2023-10-06T05:07:22.978259+0000","last_peered":"2023-10-06T05:07:22.978259+0000","last_clean":"2023-10-06T05:07:22.978259+0000","last_became_active":"2023-10-06T05:07:22.242008+0000","last_became_peered":"2023-10-06T05:07:22.242008+0000","last_unstale":"2023-10-06T05:07:22.978259+0000","last_undegraded":"2023-10-06T05:07:22.978259+0000","last_fullsized":"2023-10-06T05:07:22.978259+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:08:07.631313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":173,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.983749+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T05:07:22.983749+0000","last_peered":"2023-10-06T05:07:22.983749+0000","last_clean":"2023-10-06T05:07:22.983749+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T05:07:22.983749+0000","last_undegraded":"2023-10-06T05:07:22.983749+0000","last_fullsized":"2023-10-06T05:07:22.983749+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.995151+0000","last_change":"2023-10-06T05:07:22.991943+0000","last_active":"2023-10-06T05:07:22.995151+0000","last_peered":"2023-10-06T05:07:22.995151+0000","last_clean":"2023-10-06T05:07:22.995151+0000","last_became_active":"2023-10-06T05:07:22.991651+0000","last_became_peered":"2023-10-06T05:07:22.991651+0000","last_unstale":"2023-10-06T05:07:22.995151+0000","last_undegraded":"2023-10-06T05:07:22.995151+0000","last_fullsized":"2023-10-06T05:07:22.995151+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:59:11.511887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":10,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.978030+0000","last_change":"2023-10-06T05:07:22.243198+0000","last_active":"2023-10-06T05:07:22.978030+0000","last_peered":"2023-10-06T05:07:22.978030+0000","last_clean":"2023-10-06T05:07:22.978030+0000","last_became_active":"2023-10-06T05:07:22.242921+0000","last_became_peered":"2023-10-06T05:07:22.242921+0000","last_unstale":"2023-10-06T05:07:22.978030+0000","last_undegraded":"2023-10-06T05:07:22.978030+0000","last_fullsized":"2023-10-06T05:07:22.978030+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:48:35.851836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":166,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.982314+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T05:07:22.982314+0000","last_peered":"2023-10-06T05:07:22.982314+0000","last_clean":"2023-10-06T05:07:22.982314+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T05:07:22.982314+0000","last_undegraded":"2023-10-06T05:07:22.982314+0000","last_fullsized":"2023-10-06T05:07:22.982314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.990102+0000","last_change":"2023-10-06T05:07:22.987611+0000","last_active":"2023-10-06T05:07:22.990102+0000","last_peered":"2023-10-06T05:07:22.990102+0000","last_clean":"2023-10-06T05:07:22.990102+0000","last_became_active":"2023-10-06T05:07:22.987355+0000","last_became_peered":"2023-10-06T05:07:22.987355+0000","last_unstale":"2023-10-06T05:07:22.990102+0000","last_undegraded":"2023-10-06T05:07:22.990102+0000","last_fullsized":"2023-10-06T05:07:22.990102+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:00:54.443121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"206'1","reported_seq":11,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.238233+0000","last_change":"2023-10-06T05:07:22.252918+0000","last_active":"2023-10-06T05:07:23.238233+0000","last_peered":"2023-10-06T05:07:23.238233+0000","last_clean":"2023-10-06T05:07:23.238233+0000","last_became_active":"2023-10-06T05:07:22.252617+0000","last_became_peered":"2023-10-06T05:07:22.252617+0000","last_unstale":"2023-10-06T05:07:23.238233+0000","last_undegraded":"2023-10-06T05:07:23.238233+0000","last_fullsized":"2023-10-06T05:07:23.238233+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:10.626108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":200,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.975348+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T05:07:22.975348+0000","last_peered":"2023-10-06T05:07:22.975348+0000","last_clean":"2023-10-06T05:07:22.975348+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T05:07:22.975348+0000","last_undegraded":"2023-10-06T05:07:22.975348+0000","last_fullsized":"2023-10-06T05:07:22.975348+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":9,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.994881+0000","last_change":"2023-10-06T05:07:22.990552+0000","last_active":"2023-10-06T05:07:22.994881+0000","last_peered":"2023-10-06T05:07:22.994881+0000","last_clean":"2023-10-06T05:07:22.994881+0000","last_became_active":"2023-10-06T05:07:22.990264+0000","last_became_peered":"2023-10-06T05:07:22.990264+0000","last_unstale":"2023-10-06T05:07:22.994881+0000","last_undegraded":"2023-10-06T05:07:22.994881+0000","last_fullsized":"2023-10-06T05:07:22.994881+0000","mapping_epoch":205,"log_start":"0'0","ondisk_log_start":"0'0","created":205,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:22.216568+0000","last_clean_scrub_stamp":"2023-10-06T05:07:22.216568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:17.286805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"206'4","reported_seq":14,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:23.237664+0000","last_change":"2023-10-06T05:07:22.246494+0000","last_active":"2023-10-06T05:07:23.237664+0000","last_peered":"2023-10-06T05:07:23.237664+0000","last_clean":"2023-10-06T05:07:23.237664+0000","last_became_active":"2023-10-06T05:07:22.245691+0000","last_became_peered":"2023-10-06T05:07:22.245691+0000","last_unstale":"2023-10-06T05:07:23.237664+0000","last_undegraded":"2023-10-06T05:07:23.237664+0000","last_fullsized":"2023-10-06T05:07:23.237664+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:07:21.204941+0000","last_clean_scrub_stamp":"2023-10-06T05:07:21.204941+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:22:25.877007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":200,"reported_epoch":206,"state":"active+clean","last_fresh":"2023-10-06T05:07:22.975864+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T05:07:22.975864+0000","last_peered":"2023-10-06T05:07:22.975864+0000","last_clean":"2023-10-06T05:07:22.975864+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T05:07:22.975864+0000","last_undegraded":"2023-10-06T05:07:22.975864+0000","last_fullsized":"2023-10-06T05:07:22.975864+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304492,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067348,"statfs":{"total":96636764160,"available":96324964352,"internally_reserved":0,"allocated":3182592,"data_stored":1961210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94999999999999996,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0529999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79300000000000004,"15min":0.80600000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.996,"5min":1.0640000000000001,"15min":1.073},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.80300000000000005,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.0860000000000001},"last":0.442}]},{"osd":2,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72199999999999998,"15min":0.74099999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.751},{"interface":"front","average":{"1min":0.754,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.52500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0509999999999999,"5min":1.107,"15min":1.1160000000000001},"last":0.50700000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.101},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.753,"15min":0.76600000000000001},"min":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96099999999999997,"5min":1.0149999999999999,"15min":1.0369999999999999},"last":0.61499999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.355},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64000000000000001,"15min":0.64500000000000002},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.40999999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.208},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47699999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.754,"15min":0.749},"min":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.48799999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.51500000000000001,"5min":0.48999999999999999,"15min":0.46700000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.80500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575966,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299824,"kb_used_data":2536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072016,"statfs":{"total":96636764160,"available":96329744384,"internally_reserved":0,"allocated":2596864,"data_stored":1371890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":1.0009999999999999,"15min":1.2969999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94999999999999996,"5min":1.083,"15min":1.083},"last":0.79100000000000004}]},{"osd":1,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0640000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0960000000000001,"5min":1.425,"15min":1.425},"last":0.76900000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.752,"15min":0.76100000000000001},"min":{"1min":0.48099999999999998,"5min":0.47299999999999998,"15min":0.44700000000000001},"max":{"1min":0.91800000000000004,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96499999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78200000000000003,"15min":0.78600000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":0.998,"5min":1.016,"15min":1.1699999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.79800000000000004},"min":{"1min":0.61299999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94599999999999995,"5min":1.0409999999999999,"15min":1.1479999999999999},"last":0.88400000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.40200000000000002,"15min":0.38900000000000001},"max":{"1min":0.81399999999999995,"5min":0.95999999999999996,"15min":1.0269999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.64400000000000002},"min":{"1min":0.51800000000000002,"5min":0.46600000000000003,"15min":0.39300000000000002},"max":{"1min":0.89400000000000002,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.51400000000000001,"5min":0.5,"15min":0.41699999999999998},"max":{"1min":0.94199999999999995,"5min":1.02,"15min":1.069},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.41199999999999998},"max":{"1min":0.94199999999999995,"5min":1.0349999999999999,"15min":1.1950000000000001},"last":0.60799999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93999999999999995,"5min":1.208,"15min":1.208},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.48599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97399999999999998,"5min":1.0449999999999999,"15min":1.1579999999999999},"last":0.63100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575965,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312688,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059152,"statfs":{"total":96636764160,"available":96316571648,"internally_reserved":0,"allocated":3186688,"data_stored":1962188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.32700000000000001},"max":{"1min":1.093,"5min":1.155,"15min":1.155},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.59299999999999997},"min":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.38900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.90100000000000002},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.72699999999999998,"5min":0.76400000000000001,"15min":1.153},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.42199999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1,"5min":1.018,"15min":1.018},"last":0.50900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.435,"15min":0.41799999999999998},"max":{"1min":0.95499999999999996,"5min":1.113,"15min":1.131},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92200000000000004,"5min":0.95499999999999996,"15min":0.96999999999999997},"last":0.433}]},{"osd":4,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79600000000000004,"15min":0.80600000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.02,"5min":1.2210000000000001,"15min":1.2250000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.82199999999999995,"15min":0.84299999999999997},"min":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.147,"5min":1.1930000000000001,"15min":1.236},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.82699999999999996,"15min":0.83999999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.53600000000000003},"max":{"1min":1.0660000000000001,"5min":1.1240000000000001,"15min":1.228},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.51000000000000001},"max":{"1min":1.139,"5min":1.1799999999999999,"15min":1.2829999999999999},"last":0.873}]},{"osd":6,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85199999999999998,"15min":0.85699999999999998},"min":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.59199999999999997},"max":{"1min":1.1779999999999999,"5min":1.2490000000000001,"15min":1.2709999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.85999999999999999,"15min":0.86399999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.163,"5min":1.242,"15min":1.2470000000000001},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.63,"5min":0.623,"15min":0.623},"max":{"1min":1.198,"5min":1.222,"15min":1.2949999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.872,"15min":0.88},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.224,"5min":1.224,"15min":1.3080000000000001},"last":0.94399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575967,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303928,"kb_used_data":2544,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067912,"statfs":{"total":96636764160,"available":96325541888,"internally_reserved":0,"allocated":2605056,"data_stored":1371542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.28599999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.35999999999999999,"15min":0.34399999999999997},"max":{"1min":0.67600000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55500000000000005,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.38700000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.91000000000000003},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.86699999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.63700000000000001,"15min":0.64700000000000002},"min":{"1min":0.49299999999999999,"5min":0.439,"15min":0.42899999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.67100000000000004,"5min":0.57899999999999996,"15min":0.54300000000000004},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.3049999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.63600000000000001,"5min":0.55700000000000005,"15min":0.55300000000000005},"max":{"1min":1.085,"5min":1.127,"15min":1.3440000000000001},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.83499999999999996,"15min":0.84399999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.2729999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82899999999999996,"15min":0.85099999999999998},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.45600000000000002},"max":{"1min":1.1339999999999999,"5min":1.204,"15min":1.218},"last":0.66600000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81999999999999995,"15min":0.83799999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.53800000000000003},"max":{"1min":1.0529999999999999,"5min":1.1240000000000001,"15min":1.2869999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86199999999999999,"15min":0.88300000000000001},"min":{"1min":0.67700000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.246,"5min":1.246,"15min":1.4399999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.85399999999999998,"15min":0.876},"min":{"1min":0.626,"5min":0.624,"15min":0.59799999999999998},"max":{"1min":1.216,"5min":1.2490000000000001,"15min":1.367},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.68300000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.294},"last":0.53800000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575966,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316204,"kb_used_data":2532,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055636,"statfs":{"total":96636764160,"available":96312971264,"internally_reserved":0,"allocated":2592768,"data_stored":1371396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47299999999999998,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.32000000000000001,"15min":0.312},"max":{"1min":0.73699999999999999,"5min":0.746,"15min":0.80300000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.91200000000000003},"last":0.377}]},{"osd":1,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73899999999999999,"5min":0.78300000000000003,"15min":0.92400000000000004},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.42899999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.51300000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.32400000000000001},"max":{"1min":0.76100000000000001,"5min":0.89800000000000002,"15min":0.90600000000000003},"last":0.45600000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.84799999999999998,"15min":0.85399999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0580000000000001,"5min":1.286,"15min":1.286},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.59199999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.254},"last":0.66100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.88400000000000001,"15min":0.88800000000000001},"min":{"1min":0.67100000000000004,"5min":0.59099999999999997,"15min":0.57599999999999996},"max":{"1min":1.165,"5min":1.26,"15min":1.345},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.88,"5min":0.89000000000000001,"15min":0.89700000000000002},"min":{"1min":0.65500000000000003,"5min":0.628,"15min":0.628},"max":{"1min":1.085,"5min":1.2090000000000001,"15min":1.339},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58199999999999996},"max":{"1min":0.997,"5min":1.1279999999999999,"15min":1.2989999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1439999999999999,"5min":1.226,"15min":1.226},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85999999999999999,"15min":0.86599999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1000000000000001,"5min":1.141,"15min":1.276},"last":0.752},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90700000000000003,"15min":0.90900000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.59399999999999997},"max":{"1min":1.1850000000000001,"5min":1.2170000000000001,"15min":1.3680000000000001},"last":0.58199999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575966,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299828,"kb_used_data":2540,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072012,"statfs":{"total":96636764160,"available":96329740288,"internally_reserved":0,"allocated":2600960,"data_stored":1371980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51100000000000001,"15min":0.49399999999999999},"min":{"1min":0.34399999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67400000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85799999999999998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.76400000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.58299999999999996},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60099999999999998}]},{"osd":2,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.48899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.74199999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65000000000000002,"15min":0.63500000000000001},"min":{"1min":0.48899999999999999,"5min":0.45100000000000001,"15min":0.38500000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46200000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.83599999999999997,"15min":0.82699999999999996},"min":{"1min":0.625,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1599999999999999,"5min":1.232,"15min":1.2849999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.87},"min":{"1min":0.64800000000000002,"5min":0.53900000000000003,"15min":0.52500000000000002},"max":{"1min":1.0860000000000001,"5min":1.232,"15min":1.4019999999999999},"last":0.77700000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80600000000000005,"15min":0.81999999999999995},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.52200000000000002},"max":{"1min":1.01,"5min":1.22,"15min":1.5760000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0980000000000001,"5min":1.1419999999999999,"15min":1.3300000000000001},"last":0.85699999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.84899999999999998,"15min":0.85099999999999998},"min":{"1min":0.66100000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0900000000000001,"5min":1.212,"15min":1.2150000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.87,"5min":0.88700000000000001,"15min":0.89100000000000001},"min":{"1min":0.64900000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1399999999999999,"5min":1.2929999999999999,"15min":1.321},"last":0.90300000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.87,"15min":0.87},"min":{"1min":0.70299999999999996,"5min":0.59399999999999997,"15min":0.57599999999999996},"max":{"1min":1.1619999999999999,"5min":1.198,"15min":1.2769999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.86599999999999999,"15min":0.85399999999999998},"min":{"1min":0.69099999999999995,"5min":0.59299999999999997,"15min":0.55800000000000005},"max":{"1min":1.1200000000000001,"5min":1.1890000000000001,"15min":1.389},"last":0.76400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575966,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312124,"kb_used_data":2548,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059716,"statfs":{"total":96636764160,"available":96317149184,"internally_reserved":0,"allocated":2609152,"data_stored":1373390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.72799999999999998,"15min":0.749},"min":{"1min":0.49099999999999999,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.84199999999999997,"5min":1.0820000000000001,"15min":1.2},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.75900000000000001,"15min":0.76100000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.874,"5min":1.048,"15min":1.048},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Oct 6 05:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.72799999999999998,"15min":0.745},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.83099999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.76900000000000002,"15min":0.78100000000000003},"min":{"1min":0.58299999999999996,"5min":0.52400000000000002,"15min":0.48499999999999999},"max":{"1min":0.89000000000000001,"5min":0.99099999999999999,"15min":1.258},"last":0.624}]},{"osd":2,"last update":"Fri Oct 6 05:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.75800000000000001,"15min":0.77200000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.85599999999999998,"5min":1.0089999999999999,"15min":1.0309999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.76100000000000001,"15min":0.78600000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.91500000000000004,"5min":1.0049999999999999,"15min":1.2929999999999999},"last":0.92000000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.74399999999999999,"15min":0.76100000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.49299999999999999},"max":{"1min":0.79500000000000004,"5min":0.97999999999999998,"15min":1.2330000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.77500000000000002,"15min":0.79400000000000004},"min":{"1min":0.56200000000000006,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.88300000000000001,"5min":0.97399999999999998,"15min":1.034},"last":0.89300000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61799999999999999,"15min":0.63900000000000001},"min":{"1min":0.44400000000000001,"5min":0.41599999999999998,"15min":0.41199999999999998},"max":{"1min":0.84299999999999997,"5min":1.04,"15min":1.323},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64600000000000002,"15min":0.65100000000000002},"min":{"1min":0.45800000000000002,"5min":0.44700000000000001,"15min":0.39700000000000002},"max":{"1min":0.77800000000000002,"5min":1.01,"15min":1.1910000000000001},"last":0.74299999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.66000000000000003,"15min":0.67600000000000005},"min":{"1min":0.434,"5min":0.42999999999999999,"15min":0.42099999999999999},"max":{"1min":0.82799999999999996,"5min":1.024,"15min":1.079},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.504,"5min":0.44400000000000001,"15min":0.41999999999999998},"max":{"1min":0.85399999999999998,"5min":0.98499999999999999,"15min":1.3540000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68700000000000006,"15min":0.71199999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.80400000000000005,"5min":1.0640000000000001,"15min":1.155},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72699999999999998,"15min":0.751},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.47299999999999998},"max":{"1min":0.84199999999999997,"5min":1.079,"15min":1.149},"last":0.93000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575966,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308024,"kb_used_data":2544,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063816,"statfs":{"total":96636764160,"available":96321347584,"internally_reserved":0,"allocated":2605056,"data_stored":1372400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.748},"min":{"1min":0.53200000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.93500000000000005,"5min":1.079,"15min":1.079},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.72999999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88800000000000001,"5min":0.999,"15min":1.02},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.745,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.53600000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98399999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.46700000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79600000000000004,"15min":0.79400000000000004},"min":{"1min":0.65300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.99199999999999999,"5min":1.0700000000000001,"15min":1.0780000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80100000000000005,"15min":0.80700000000000005},"min":{"1min":0.57599999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0149999999999999,"5min":1.048,"15min":1.048},"last":0.64300000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.48599999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.94399999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.627},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.63500000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0169999999999999,"5min":1.095,"15min":1.095},"last":0.754}]},{"osd":4,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.629},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.38200000000000001},"max":{"1min":0.84399999999999997,"5min":0.97699999999999998,"15min":1.0049999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.5,"5min":0.38,"15min":0.38},"max":{"1min":0.89000000000000001,"5min":1.01,"15min":1.042},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.48499999999999999,"5min":0.45900000000000002,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.0369999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.42799999999999999},"max":{"1min":0.90700000000000003,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.55000000000000004,"5min":0.48499999999999999,"15min":0.44500000000000001},"max":{"1min":0.92600000000000005,"5min":1.0269999999999999,"15min":1.0669999999999999},"last":0.503},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.54000000000000004,"5min":0.51300000000000001,"15min":0.47199999999999998},"max":{"1min":0.93000000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.72999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:07:28.113 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T05:07:28.114 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T05:07:28.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.341+0000 7f1d9659d700 1 Processor -- start 2023-10-06T05:07:28.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.341+0000 7f1d9659d700 1 -- start start 2023-10-06T05:07:28.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.342+0000 7f1d9659d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1d901595d0 0x7f1d900ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:28.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.342+0000 7f1d9659d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d900cf480 0x7f1d9015d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:28.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.342+0000 7f1d9659d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d9015dbc0 con 0x7f1d9015d760 2023-10-06T05:07:28.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.342+0000 7f1d9659d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d900b10f0 con 0x7f1d901595d0 2023-10-06T05:07:28.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.342+0000 7f1d9659d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d900cd530 con 0x7f1d900cf480 2023-10-06T05:07:28.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.343+0000 7f1d95d9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1d9015d760 0x7f1d901599a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:33256/0 (socket says 172.21.15.55:33256) 2023-10-06T05:07:28.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.343+0000 7f1d95d9c700 1 -- 172.21.15.55:0/2569399518 learned_addr learned my addr 172.21.15.55:0/2569399518 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:28.345 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.343+0000 7f1d9559b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1d901595d0 0x7f1d900ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:28.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.343+0000 7f1d94d9a700 1 --2- 172.21.15.55:0/2569399518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d900cf480 0x7f1d9015d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:28.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.343+0000 7f1d9559b700 1 -- 172.21.15.55:0/2569399518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d900cf480 msgr2=0x7f1d9015d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:28.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.343+0000 7f1d9559b700 1 --2- 172.21.15.55:0/2569399518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d900cf480 0x7f1d9015d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.343+0000 7f1d9559b700 1 -- 172.21.15.55:0/2569399518 >> v1:172.21.15.55:6789/0 conn(0x7f1d9015d760 legacy=0x7f1d901599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:28.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.343+0000 7f1d9559b700 1 -- 172.21.15.55:0/2569399518 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9015e000 con 0x7f1d901595d0 2023-10-06T05:07:28.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.343+0000 7f1d87fff700 1 -- 172.21.15.55:0/2569399518 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659443175 0 0) 0x7f1d88001110 con 0x7f1d9015d760 2023-10-06T05:07:28.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.343+0000 7f1d94d9a700 1 --2- 172.21.15.55:0/2569399518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d900cf480 0x7f1d9015d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:28.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.343+0000 7f1d9559b700 1 --2- 172.21.15.55:0/2569399518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d901595d0 0x7f1d900ccff0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009eb0 tx=0x7f1d8c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b52b40db81d03e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:28.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.344+0000 7f1d87fff700 1 -- 172.21.15.55:0/2569399518 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c00e070 con 0x7f1d901595d0 2023-10-06T05:07:28.348 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.344+0000 7f1d87fff700 1 -- 172.21.15.55:0/2569399518 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c00bc30 con 0x7f1d901595d0 2023-10-06T05:07:28.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.344+0000 7f1d87fff700 1 -- 172.21.15.55:0/2569399518 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c012430 con 0x7f1d901595d0 2023-10-06T05:07:28.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.344+0000 7f1d9659d700 1 -- 172.21.15.55:0/2569399518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d901595d0 msgr2=0x7f1d900ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:28.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.344+0000 7f1d9659d700 1 --2- 172.21.15.55:0/2569399518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d901595d0 0x7f1d900ccff0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009eb0 tx=0x7f1d8c009780 comp rx=0 tx=0).stop 2023-10-06T05:07:28.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.344+0000 7f1d9659d700 1 -- 172.21.15.55:0/2569399518 shutdown_connections 2023-10-06T05:07:28.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.344+0000 7f1d9659d700 1 --2- 172.21.15.55:0/2569399518 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d901595d0 0x7f1d900ccff0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.344+0000 7f1d9659d700 1 --2- 172.21.15.55:0/2569399518 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d900cf480 0x7f1d9015d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.344+0000 7f1d9659d700 1 -- 172.21.15.55:0/2569399518 >> 172.21.15.55:0/2569399518 conn(0x7f1d900d05f0 msgr2=0x7f1d900d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:28.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.344+0000 7f1d9659d700 1 -- 172.21.15.55:0/2569399518 shutdown_connections 2023-10-06T05:07:28.350 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.344+0000 7f1d9659d700 1 -- 172.21.15.55:0/2569399518 wait complete. 2023-10-06T05:07:28.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.346+0000 7f1d9659d700 1 Processor -- start 2023-10-06T05:07:28.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.346+0000 7f1d9659d700 1 -- start start 2023-10-06T05:07:28.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.347+0000 7f1d9659d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d900cf480 0x7f1d90143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:28.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.347+0000 7f1d9659d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d901595d0 0x7f1d90155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:28.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.347+0000 7f1d9659d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d9015d760 0x7f1d900b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:28.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.347+0000 7f1d9659d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9016dc80 con 0x7f1d901595d0 2023-10-06T05:07:28.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.347+0000 7f1d9659d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9016d070 con 0x7f1d900cf480 2023-10-06T05:07:28.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.347+0000 7f1d9659d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9016d350 con 0x7f1d9015d760 2023-10-06T05:07:28.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.347+0000 7f1d94d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d901595d0 0x7f1d90155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:28.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.347+0000 7f1d94d9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d901595d0 0x7f1d90155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43868/0 (socket says 172.21.15.55:43868) 2023-10-06T05:07:28.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.347+0000 7f1d94d9a700 1 -- 172.21.15.55:0/2292822275 learned_addr learned my addr 172.21.15.55:0/2292822275 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:28.354 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.347+0000 7f1d95d9c700 1 --2- 172.21.15.55:0/2292822275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d9015d760 0x7f1d900b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:28.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.347+0000 7f1d9559b700 1 --2- 172.21.15.55:0/2292822275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d900cf480 0x7f1d90143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:28.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.348+0000 7f1d94d9a700 1 -- 172.21.15.55:0/2292822275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d9015d760 msgr2=0x7f1d900b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:28.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.348+0000 7f1d94d9a700 1 --2- 172.21.15.55:0/2292822275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d9015d760 0x7f1d900b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.348+0000 7f1d94d9a700 1 -- 172.21.15.55:0/2292822275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d900cf480 msgr2=0x7f1d90143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:28.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.348+0000 7f1d94d9a700 1 --2- 172.21.15.55:0/2292822275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d900cf480 0x7f1d90143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.355 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.348+0000 7f1d94d9a700 1 -- 172.21.15.55:0/2292822275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c009a20 con 0x7f1d901595d0 2023-10-06T05:07:28.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.348+0000 7f1d94d9a700 1 --2- 172.21.15.55:0/2292822275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d901595d0 0x7f1d90155390 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000e8d0 tx=0x7f1d80008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:28.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.348+0000 7f1d857fa700 1 -- 172.21.15.55:0/2292822275 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000ee70 con 0x7f1d901595d0 2023-10-06T05:07:28.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.348+0000 7f1d857fa700 1 -- 172.21.15.55:0/2292822275 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8000c5e0 con 0x7f1d901595d0 2023-10-06T05:07:28.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.348+0000 7f1d857fa700 1 -- 172.21.15.55:0/2292822275 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8000c980 con 0x7f1d901595d0 2023-10-06T05:07:28.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.348+0000 7f1d9659d700 1 -- 172.21.15.55:0/2292822275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9016d630 con 0x7f1d901595d0 2023-10-06T05:07:28.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.348+0000 7f1d9659d700 1 -- 172.21.15.55:0/2292822275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d90144460 con 0x7f1d901595d0 2023-10-06T05:07:28.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.349+0000 7f1d9659d700 1 -- 172.21.15.55:0/2292822275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9016de30 con 0x7f1d901595d0 2023-10-06T05:07:28.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.350+0000 7f1d857fa700 1 -- 172.21.15.55:0/2292822275 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1d80013410 con 0x7f1d901595d0 2023-10-06T05:07:28.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.351+0000 7f1d857fa700 1 --2- 172.21.15.55:0/2292822275 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1d74066f30 0x7f1d740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:28.357 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.351+0000 7f1d857fa700 1 -- 172.21.15.55:0/2292822275 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f1d80084700 con 0x7f1d901595d0 2023-10-06T05:07:28.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.351+0000 7f1d9559b700 1 --2- 172.21.15.55:0/2292822275 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1d74066f30 0x7f1d740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:28.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.352+0000 7f1d9559b700 1 --2- 172.21.15.55:0/2292822275 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1d74066f30 0x7f1d740693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009eb0 tx=0x7f1d8c017000 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:28.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.355+0000 7f1d857fa700 1 -- 172.21.15.55:0/2292822275 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1d9016de30 con 0x7f1d901595d0 2023-10-06T05:07:28.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.636+0000 7f1d9659d700 1 -- 172.21.15.55:0/2292822275 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1d9004c3c0 con 0x7f1d901595d0 2023-10-06T05:07:28.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.640+0000 7f1d857fa700 1 -- 172.21.15.55:0/2292822275 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v47) v1 ==== 190+0+0 (secure 0 0 0) 0x7f1d80055040 con 0x7f1d901595d0 2023-10-06T05:07:28.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.643+0000 7f1d9659d700 1 -- 172.21.15.55:0/2292822275 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1d74066f30 msgr2=0x7f1d740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:28.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.643+0000 7f1d9659d700 1 --2- 172.21.15.55:0/2292822275 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1d74066f30 0x7f1d740693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c009eb0 tx=0x7f1d8c017000 comp rx=0 tx=0).stop 2023-10-06T05:07:28.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.643+0000 7f1d9659d700 1 -- 172.21.15.55:0/2292822275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d901595d0 msgr2=0x7f1d90155390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:28.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.643+0000 7f1d9659d700 1 --2- 172.21.15.55:0/2292822275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d901595d0 0x7f1d90155390 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000e8d0 tx=0x7f1d80008040 comp rx=0 tx=0).stop 2023-10-06T05:07:28.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.643+0000 7f1d9659d700 1 -- 172.21.15.55:0/2292822275 shutdown_connections 2023-10-06T05:07:28.645 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.643+0000 7f1d9659d700 1 --2- 172.21.15.55:0/2292822275 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1d74066f30 0x7f1d740693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.643+0000 7f1d9659d700 1 --2- 172.21.15.55:0/2292822275 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1d9015d760 0x7f1d900b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.643+0000 7f1d9659d700 1 --2- 172.21.15.55:0/2292822275 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1d900cf480 0x7f1d90143f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.643+0000 7f1d9659d700 1 --2- 172.21.15.55:0/2292822275 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1d901595d0 0x7f1d90155390 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.643+0000 7f1d9659d700 1 -- 172.21.15.55:0/2292822275 >> 172.21.15.55:0/2292822275 conn(0x7f1d900d05f0 msgr2=0x7f1d90152d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:28.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.643+0000 7f1d9659d700 1 -- 172.21.15.55:0/2292822275 shutdown_connections 2023-10-06T05:07:28.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.643+0000 7f1d9659d700 1 -- 172.21.15.55:0/2292822275 wait complete. 2023-10-06T05:07:28.646 INFO:teuthology.orchestra.run.smithi055.stderr:updated caps for client.0 2023-10-06T05:07:28.662 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:07:28.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.847+0000 7f9f86d9d700 1 Processor -- start 2023-10-06T05:07:28.849 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.847+0000 7f9f86d9d700 1 -- start start 2023-10-06T05:07:28.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.849+0000 7f9f86d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f9f800ae710 0x7f9f800c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:28.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.849+0000 7f9f86d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f800ca860 con 0x7f9f80156310 2023-10-06T05:07:28.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.849+0000 7f9f86d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f800cab40 con 0x7f9f801566e0 2023-10-06T05:07:28.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.849+0000 7f9f86d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f800cae20 con 0x7f9f800ae710 2023-10-06T05:07:28.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.849+0000 7f9f85d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9f80156310 0x7f9f800c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:33260/0 (socket says 172.21.15.55:33260) 2023-10-06T05:07:28.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.849+0000 7f9f85d9b700 1 -- 172.21.15.55:0/3461396837 learned_addr learned my addr 172.21.15.55:0/3461396837 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:28.851 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.849+0000 7f9f8659c700 1 --2- 172.21.15.55:0/3461396837 >> v2:172.21.15.55:3300/0 conn(0x7f9f800ae710 0x7f9f800c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:28.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.849+0000 7f9f84d99700 1 -- 172.21.15.55:0/3461396837 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568634363 0 0) 0x7f9f800ca860 con 0x7f9f80156310 2023-10-06T05:07:28.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.849+0000 7f9f84d99700 1 -- 172.21.15.55:0/3461396837 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f6c003570 con 0x7f9f80156310 2023-10-06T05:07:28.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.849+0000 7f9f8659c700 1 -- 172.21.15.55:0/3461396837 >> v1:172.21.15.163:6789/0 conn(0x7f9f801566e0 legacy=0x7f9f800ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:28.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.850+0000 7f9f8659c700 1 -- 172.21.15.55:0/3461396837 >> v1:172.21.15.55:6789/0 conn(0x7f9f80156310 legacy=0x7f9f800c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:28.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.850+0000 7f9f8659c700 1 -- 172.21.15.55:0/3461396837 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f800cb100 con 0x7f9f800ae710 2023-10-06T05:07:28.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.850+0000 7f9f8659c700 1 --2- 172.21.15.55:0/3461396837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f800ae710 0x7f9f800c6830 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009e60 tx=0x7f9f78009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b157997516e09ba server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:28.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.850+0000 7f9f84d99700 1 -- 172.21.15.55:0/3461396837 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7800e070 con 0x7f9f800ae710 2023-10-06T05:07:28.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.850+0000 7f9f84d99700 1 -- 172.21.15.55:0/3461396837 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7800bb60 con 0x7f9f800ae710 2023-10-06T05:07:28.853 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.850+0000 7f9f84d99700 1 -- 172.21.15.55:0/3461396837 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78012430 con 0x7f9f800ae710 2023-10-06T05:07:28.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.851+0000 7f9f86d9d700 1 -- 172.21.15.55:0/3461396837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f800ae710 msgr2=0x7f9f800c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:28.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.851+0000 7f9f86d9d700 1 --2- 172.21.15.55:0/3461396837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f800ae710 0x7f9f800c6830 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009e60 tx=0x7f9f78009780 comp rx=0 tx=0).stop 2023-10-06T05:07:28.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.851+0000 7f9f86d9d700 1 -- 172.21.15.55:0/3461396837 shutdown_connections 2023-10-06T05:07:28.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.851+0000 7f9f86d9d700 1 --2- 172.21.15.55:0/3461396837 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f800ae710 0x7f9f800c6830 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.851+0000 7f9f86d9d700 1 -- 172.21.15.55:0/3461396837 >> 172.21.15.55:0/3461396837 conn(0x7f9f800cc1f0 msgr2=0x7f9f800cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:28.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.851+0000 7f9f86d9d700 1 -- 172.21.15.55:0/3461396837 shutdown_connections 2023-10-06T05:07:28.854 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.851+0000 7f9f86d9d700 1 -- 172.21.15.55:0/3461396837 wait complete. 2023-10-06T05:07:28.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.852+0000 7f9f86d9d700 1 Processor -- start 2023-10-06T05:07:28.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.852+0000 7f9f86d9d700 1 -- start start 2023-10-06T05:07:28.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.853+0000 7f9f86d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f800ae710 0x7f9f8013fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:28.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.853+0000 7f9f86d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f80156310 0x7f9f800c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:28.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.853+0000 7f9f86d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f801566e0 0x7f9f800c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:28.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.853+0000 7f9f86d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f80140220 con 0x7f9f801566e0 2023-10-06T05:07:28.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.853+0000 7f9f86d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8016a0b0 con 0x7f9f80156310 2023-10-06T05:07:28.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.853+0000 7f9f86d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f80169780 con 0x7f9f800ae710 2023-10-06T05:07:28.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.854+0000 7f9f8659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f801566e0 0x7f9f800c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:28.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.854+0000 7f9f8659c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f801566e0 0x7f9f800c2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43884/0 (socket says 172.21.15.55:43884) 2023-10-06T05:07:28.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.854+0000 7f9f8659c700 1 -- 172.21.15.55:0/2146276646 learned_addr learned my addr 172.21.15.55:0/2146276646 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:28.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.854+0000 7f9f85d9b700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f800ae710 0x7f9f8013fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:28.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.854+0000 7f9f8559a700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f80156310 0x7f9f800c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:28.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.854+0000 7f9f8659c700 1 -- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f800ae710 msgr2=0x7f9f8013fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:28.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.854+0000 7f9f8659c700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f800ae710 0x7f9f8013fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.854+0000 7f9f8659c700 1 -- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f80156310 msgr2=0x7f9f800c26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:28.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.854+0000 7f9f8659c700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f80156310 0x7f9f800c26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:28.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.854+0000 7f9f8659c700 1 -- 172.21.15.55:0/2146276646 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f78009a20 con 0x7f9f801566e0 2023-10-06T05:07:28.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.854+0000 7f9f8559a700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f80156310 0x7f9f800c26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:07:28.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.854+0000 7f9f85d9b700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f800ae710 0x7f9f8013fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:07:28.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.854+0000 7f9f8659c700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f801566e0 0x7f9f800c2c20 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f9f78013000 tx=0x7f9f78019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:28.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.854+0000 7f9f75ffb700 1 -- 172.21.15.55:0/2146276646 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78012e90 con 0x7f9f801566e0 2023-10-06T05:07:28.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.855+0000 7f9f75ffb700 1 -- 172.21.15.55:0/2146276646 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7801d630 con 0x7f9f801566e0 2023-10-06T05:07:28.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.855+0000 7f9f86d9d700 1 -- 172.21.15.55:0/2146276646 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f80169a60 con 0x7f9f801566e0 2023-10-06T05:07:28.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.855+0000 7f9f75ffb700 1 -- 172.21.15.55:0/2146276646 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7801d9f0 con 0x7f9f801566e0 2023-10-06T05:07:28.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.855+0000 7f9f86d9d700 1 -- 172.21.15.55:0/2146276646 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f80169470 con 0x7f9f801566e0 2023-10-06T05:07:28.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.856+0000 7f9f86d9d700 1 -- 172.21.15.55:0/2146276646 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f8016a420 con 0x7f9f801566e0 2023-10-06T05:07:28.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.856+0000 7f9f75ffb700 1 -- 172.21.15.55:0/2146276646 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f78024070 con 0x7f9f801566e0 2023-10-06T05:07:28.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.857+0000 7f9f75ffb700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9f64066f30 0x7f9f640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:28.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.857+0000 7f9f75ffb700 1 -- 172.21.15.55:0/2146276646 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f9f78087150 con 0x7f9f801566e0 2023-10-06T05:07:28.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.858+0000 7f9f85d9b700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9f64066f30 0x7f9f640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:28.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.858+0000 7f9f85d9b700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9f64066f30 0x7f9f640693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c007fd0 tx=0x7f9f7c009040 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:28.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:28.861+0000 7f9f75ffb700 1 -- 172.21.15.55:0/2146276646 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f78054000 con 0x7f9f801566e0 2023-10-06T05:07:29.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.111+0000 7f9f86d9d700 1 -- 172.21.15.55:0/2146276646 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f8004c3c0 con 0x7f9f801566e0 2023-10-06T05:07:29.113 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.112+0000 7f9f75ffb700 1 -- 172.21.15.55:0/2146276646 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4616 (secure 0 0 0) 0x7f9f78057620 con 0x7f9f801566e0 2023-10-06T05:07:29.114 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:29.114 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3704044820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3704044820},{"type":"v1","addr":"172.21.15.163:6834","nonce":3704044820}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":9772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2254022733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2254022733},{"type":"v1","addr":"172.21.15.163:6835","nonce":2254022733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":10224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3844040929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3844040929},{"type":"v1","addr":"172.21.15.55:6834","nonce":3844040929}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:07:22.724188+0000","modified":"2023-10-06T05:07:26.255934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10227":{"gid":10227,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/3632968604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3632968604},{"type":"v1","addr":"172.21.15.55:6835","nonce":3632968604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:07:29.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.114+0000 7f9f86d9d700 1 -- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9f64066f30 msgr2=0x7f9f640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.114+0000 7f9f86d9d700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9f64066f30 0x7f9f640693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c007fd0 tx=0x7f9f7c009040 comp rx=0 tx=0).stop 2023-10-06T05:07:29.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.114+0000 7f9f86d9d700 1 -- 172.21.15.55:0/2146276646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f801566e0 msgr2=0x7f9f800c2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.114+0000 7f9f86d9d700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f801566e0 0x7f9f800c2c20 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f9f78013000 tx=0x7f9f78019000 comp rx=0 tx=0).stop 2023-10-06T05:07:29.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.114+0000 7f9f86d9d700 1 -- 172.21.15.55:0/2146276646 shutdown_connections 2023-10-06T05:07:29.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.114+0000 7f9f86d9d700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9f64066f30 0x7f9f640693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.114+0000 7f9f86d9d700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9f801566e0 0x7f9f800c2c20 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.114+0000 7f9f86d9d700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9f800ae710 0x7f9f8013fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.114+0000 7f9f86d9d700 1 --2- 172.21.15.55:0/2146276646 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9f80156310 0x7f9f800c26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.114+0000 7f9f86d9d700 1 -- 172.21.15.55:0/2146276646 >> 172.21.15.55:0/2146276646 conn(0x7f9f800cc1f0 msgr2=0x7f9f800c1370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:29.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.114+0000 7f9f86d9d700 1 -- 172.21.15.55:0/2146276646 shutdown_connections 2023-10-06T05:07:29.117 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.114+0000 7f9f86d9d700 1 -- 172.21.15.55:0/2146276646 wait complete. 2023-10-06T05:07:29.117 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 105 2023-10-06T05:07:29.199 INFO:tasks.ceph.mds.a.smithi055.stderr:2023-10-06T05:07:29.198+0000 7f37005f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:07:29.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T05:07:22.724188+0000', 'modified': '2023-10-06T05:07:26.255934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10227}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10227': {'gid': 10227, 'name': 'a', 'rank': 0, 'incarnation': 101, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.55:6835/3632968604', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.55:6833', 'nonce': 3632968604}, {'type': 'v1', 'addr': '172.21.15.55:6835', 'nonce': 3632968604}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T05:07:29.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T05:07:29.538 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T05:07:29.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.725+0000 7f5c39d9d700 1 Processor -- start 2023-10-06T05:07:29.727 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.725+0000 7f5c39d9d700 1 -- start start 2023-10-06T05:07:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.727+0000 7f5c39d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5c34150180 0x7f5c34157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.727+0000 7f5c39d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5c340c7410 0x7f5c34069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.727+0000 7f5c39d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c3406e2a0 con 0x7f5c34150550 2023-10-06T05:07:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.727+0000 7f5c39d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c3406b2e0 con 0x7f5c34150180 2023-10-06T05:07:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.727+0000 7f5c39d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c340c78e0 con 0x7f5c340c7410 2023-10-06T05:07:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.727+0000 7f5c33fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5c34150550 0x7f5c340c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:33268/0 (socket says 172.21.15.55:33268) 2023-10-06T05:07:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.727+0000 7f5c38d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f5c34150180 0x7f5c34157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.729 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.727+0000 7f5c33fff700 1 -- 172.21.15.55:0/3548855224 learned_addr learned my addr 172.21.15.55:0/3548855224 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:29.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.727+0000 7f5c3959c700 1 --2- 172.21.15.55:0/3548855224 >> v2:172.21.15.163:3300/0 conn(0x7f5c340c7410 0x7f5c34069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.727+0000 7f5c38d9b700 1 -- 172.21.15.55:0/3548855224 >> v2:172.21.15.163:3300/0 conn(0x7f5c340c7410 msgr2=0x7f5c34069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:29.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.727+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/3548855224 >> v2:172.21.15.163:3300/0 conn(0x7f5c340c7410 0x7f5c34069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.727+0000 7f5c38d9b700 1 -- 172.21.15.55:0/3548855224 >> v1:172.21.15.55:6789/0 conn(0x7f5c34150550 legacy=0x7f5c340c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.727+0000 7f5c38d9b700 1 -- 172.21.15.55:0/3548855224 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c340c7bc0 con 0x7f5c34150180 2023-10-06T05:07:29.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.728+0000 7f5c3959c700 1 --2- 172.21.15.55:0/3548855224 >> v2:172.21.15.163:3300/0 conn(0x7f5c340c7410 0x7f5c34069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:07:29.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.727+0000 7f5c337fe700 1 -- 172.21.15.55:0/3548855224 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454880505 0 0) 0x7f5c3406e2a0 con 0x7f5c34150550 2023-10-06T05:07:29.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.728+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/3548855224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c34150180 0x7f5c34157770 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f5c34156270 tx=0x7f5c280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e946bac3a007c86e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:29.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.728+0000 7f5c337fe700 1 -- 172.21.15.55:0/3548855224 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2800b6b0 con 0x7f5c34150180 2023-10-06T05:07:29.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.728+0000 7f5c337fe700 1 -- 172.21.15.55:0/3548855224 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2800b810 con 0x7f5c34150180 2023-10-06T05:07:29.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.728+0000 7f5c337fe700 1 -- 172.21.15.55:0/3548855224 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2800bbb0 con 0x7f5c34150180 2023-10-06T05:07:29.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.728+0000 7f5c39d9d700 1 -- 172.21.15.55:0/3548855224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c34150180 msgr2=0x7f5c34157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.728+0000 7f5c39d9d700 1 --2- 172.21.15.55:0/3548855224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c34150180 0x7f5c34157770 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f5c34156270 tx=0x7f5c280091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.729+0000 7f5c39d9d700 1 -- 172.21.15.55:0/3548855224 shutdown_connections 2023-10-06T05:07:29.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.729+0000 7f5c39d9d700 1 --2- 172.21.15.55:0/3548855224 >> v2:172.21.15.163:3300/0 conn(0x7f5c340c7410 0x7f5c34069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.729+0000 7f5c39d9d700 1 --2- 172.21.15.55:0/3548855224 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c34150180 0x7f5c34157770 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.729+0000 7f5c39d9d700 1 -- 172.21.15.55:0/3548855224 >> 172.21.15.55:0/3548855224 conn(0x7f5c340cc1f0 msgr2=0x7f5c340cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:29.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.729+0000 7f5c39d9d700 1 -- 172.21.15.55:0/3548855224 shutdown_connections 2023-10-06T05:07:29.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.729+0000 7f5c39d9d700 1 -- 172.21.15.55:0/3548855224 wait complete. 2023-10-06T05:07:29.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.730+0000 7f5c39d9d700 1 Processor -- start 2023-10-06T05:07:29.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.730+0000 7f5c39d9d700 1 -- start start 2023-10-06T05:07:29.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.731+0000 7f5c39d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c340c7410 0x7f5c341671b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.731+0000 7f5c39d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c34150180 0x7f5c340697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.731+0000 7f5c39d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c34150550 0x7f5c34069d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.731+0000 7f5c39d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c341676f0 con 0x7f5c34150180 2023-10-06T05:07:29.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.731+0000 7f5c39d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3414e120 con 0x7f5c340c7410 2023-10-06T05:07:29.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.731+0000 7f5c39d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3414e400 con 0x7f5c34150550 2023-10-06T05:07:29.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.731+0000 7f5c33fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c34150180 0x7f5c340697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.731+0000 7f5c33fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c34150180 0x7f5c340697c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43908/0 (socket says 172.21.15.55:43908) 2023-10-06T05:07:29.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.731+0000 7f5c33fff700 1 -- 172.21.15.55:0/3596426134 learned_addr learned my addr 172.21.15.55:0/3596426134 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:29.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.732+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/3596426134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c340c7410 0x7f5c341671b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.732+0000 7f5c3959c700 1 --2- 172.21.15.55:0/3596426134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c34150550 0x7f5c34069d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.732+0000 7f5c33fff700 1 -- 172.21.15.55:0/3596426134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c34150550 msgr2=0x7f5c34069d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.732+0000 7f5c33fff700 1 --2- 172.21.15.55:0/3596426134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c34150550 0x7f5c34069d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.732+0000 7f5c33fff700 1 -- 172.21.15.55:0/3596426134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c340c7410 msgr2=0x7f5c341671b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.732+0000 7f5c33fff700 1 --2- 172.21.15.55:0/3596426134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c340c7410 0x7f5c341671b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.732+0000 7f5c33fff700 1 -- 172.21.15.55:0/3596426134 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c28009420 con 0x7f5c34150180 2023-10-06T05:07:29.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.732+0000 7f5c33fff700 1 --2- 172.21.15.55:0/3596426134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c34150180 0x7f5c340697c0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000c7e0 tx=0x7f5c20007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:29.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.732+0000 7f5c30ff9700 1 -- 172.21.15.55:0/3596426134 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20005b30 con 0x7f5c34150180 2023-10-06T05:07:29.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.732+0000 7f5c30ff9700 1 -- 172.21.15.55:0/3596426134 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c20005c90 con 0x7f5c34150180 2023-10-06T05:07:29.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.732+0000 7f5c39d9d700 1 -- 172.21.15.55:0/3596426134 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3414e6e0 con 0x7f5c34150180 2023-10-06T05:07:29.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.732+0000 7f5c30ff9700 1 -- 172.21.15.55:0/3596426134 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c20007300 con 0x7f5c34150180 2023-10-06T05:07:29.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.733+0000 7f5c39d9d700 1 -- 172.21.15.55:0/3596426134 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3414eca0 con 0x7f5c34150180 2023-10-06T05:07:29.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.735+0000 7f5c30ff9700 1 -- 172.21.15.55:0/3596426134 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c2001b3b0 con 0x7f5c34150180 2023-10-06T05:07:29.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.736+0000 7f5c30ff9700 1 --2- 172.21.15.55:0/3596426134 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5c18066f30 0x7f5c180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.736+0000 7f5c30ff9700 1 -- 172.21.15.55:0/3596426134 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f5c20083d70 con 0x7f5c34150180 2023-10-06T05:07:29.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.736+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/3596426134 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5c18066f30 0x7f5c180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.737+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/3596426134 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5c18066f30 0x7f5c180693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5c280093f0 tx=0x7f5c28017000 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:29.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.737+0000 7f5c497e3b80 1 Processor -- start 2023-10-06T05:07:29.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.738+0000 7f5c497e3b80 1 -- start start 2023-10-06T05:07:29.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.738+0000 7f5c497e3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 0x558d7221fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.738+0000 7f5c497e3b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d72220200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.738+0000 7f5c497e3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72226a10 0x558d72220740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c497e3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d7217aa70 con 0x558d72226a10 2023-10-06T05:07:29.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c497e3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d72178270 con 0x558d72244e50 2023-10-06T05:07:29.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c497e3b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d72174060 con 0x558d722252c0 2023-10-06T05:07:29.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c3959c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72226a10 0x558d72220740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c3959c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72226a10 0x558d72220740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43912/0 (socket says 172.21.15.55:43912) 2023-10-06T05:07:29.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c3959c700 1 -- 172.21.15.55:0/3145886729 learned_addr learned my addr 172.21.15.55:0/3145886729 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:29.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/3145886729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 0x558d7221fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c33fff700 1 --2- 172.21.15.55:0/3145886729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d72220200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c3959c700 1 -- 172.21.15.55:0/3145886729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 msgr2=0x558d72220200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:29.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c3959c700 1 --2- 172.21.15.55:0/3145886729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d72220200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c3959c700 1 -- 172.21.15.55:0/3145886729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 msgr2=0x558d7221fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:29.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c3959c700 1 --2- 172.21.15.55:0/3145886729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 0x558d7221fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c3959c700 1 -- 172.21.15.55:0/3145886729 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d7200e270 con 0x558d72226a10 2023-10-06T05:07:29.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/3145886729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 0x558d7221fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:29.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c33fff700 1 --2- 172.21.15.55:0/3145886729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d72220200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:29.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.739+0000 7f5c3959c700 1 --2- 172.21.15.55:0/3145886729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72226a10 0x558d72220740 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00f590 tx=0x7f5c2c0077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c870302b5cc6fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:29.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.740+0000 7f5c1dffb700 1 -- 172.21.15.55:0/3145886729 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c00a580 con 0x558d72226a10 2023-10-06T05:07:29.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.740+0000 7f5c1dffb700 1 -- 172.21.15.55:0/3145886729 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2c00a6e0 con 0x558d72226a10 2023-10-06T05:07:29.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.740+0000 7f5c1dffb700 1 -- 172.21.15.55:0/3145886729 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c00aaa0 con 0x558d72226a10 2023-10-06T05:07:29.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.740+0000 7f5c497e3b80 1 -- 172.21.15.55:0/3145886729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72226a10 msgr2=0x558d72220740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.740+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/3145886729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72226a10 0x558d72220740 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00f590 tx=0x7f5c2c0077a0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.740+0000 7f5c497e3b80 1 -- 172.21.15.55:0/3145886729 shutdown_connections 2023-10-06T05:07:29.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.740+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/3145886729 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72226a10 0x558d72220740 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.740+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/3145886729 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 0x558d7221fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.740+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/3145886729 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d72220200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.750 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.740+0000 7f5c497e3b80 1 -- 172.21.15.55:0/3145886729 >> 172.21.15.55:0/3145886729 conn(0x558d72224e80 msgr2=0x558d722256b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:29.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.741+0000 7f5c497e3b80 1 -- 172.21.15.55:0/3145886729 shutdown_connections 2023-10-06T05:07:29.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.741+0000 7f5c497e3b80 1 -- 172.21.15.55:0/3145886729 wait complete. 2023-10-06T05:07:29.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.741+0000 7f5c497e3b80 1 -- start start 2023-10-06T05:07:29.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.741+0000 7f5c497e3b80 1 Processor -- start 2023-10-06T05:07:29.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.742+0000 7f5c497e3b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d72267b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.742+0000 7f5c497e3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72226a10 0x558d7226b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.742+0000 7f5c497e3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 0x558d7226d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.742+0000 7f5c497e3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d7211f890 con 0x558d72226a10 2023-10-06T05:07:29.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.742+0000 7f5c497e3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d7213aea0 con 0x558d72244e50 2023-10-06T05:07:29.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.742+0000 7f5c497e3b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d71ff6e30 con 0x558d722252c0 2023-10-06T05:07:29.751 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.742+0000 7f5c497e3b80 10 client.0 _ll_register_callbacks cb 0x558d721e7020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:07:29.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c33fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72226a10 0x558d7226b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c33fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72226a10 0x558d7226b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43918/0 (socket says 172.21.15.55:43918) 2023-10-06T05:07:29.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c33fff700 1 -- 172.21.15.55:0/2027038891 learned_addr learned my addr 172.21.15.55:0/2027038891 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:29.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c3959c700 1 --2- 172.21.15.55:0/2027038891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 0x558d7226d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/2027038891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d72267b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c33fff700 1 -- 172.21.15.55:0/2027038891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 msgr2=0x558d72267b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:29.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c33fff700 1 --2- 172.21.15.55:0/2027038891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d72267b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c33fff700 1 -- 172.21.15.55:0/2027038891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 msgr2=0x558d7226d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:29.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c33fff700 1 --2- 172.21.15.55:0/2027038891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 0x558d7226d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c33fff700 1 -- 172.21.15.55:0/2027038891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c2c007dc0 con 0x558d72226a10 2023-10-06T05:07:29.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/2027038891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d72267b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:07:29.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c33fff700 1 --2- 172.21.15.55:0/2027038891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72226a10 0x558d7226b1b0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f5c20007fd0 tx=0x7f5c20081ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63a76e137bedbdc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:29.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T05:07:29.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c12ffd700 1 -- 172.21.15.55:0/2027038891 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2005b920 con 0x558d72226a10 2023-10-06T05:07:29.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c12ffd700 1 -- 172.21.15.55:0/2027038891 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f5c20005e00 con 0x558d72226a10 2023-10-06T05:07:29.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.743+0000 7f5c497e3b80 1 -- 172.21.15.55:0/2027038891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d72218940 con 0x558d72226a10 2023-10-06T05:07:29.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.744+0000 7f5c12ffd700 1 -- 172.21.15.55:0/2027038891 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c20080020 con 0x558d72226a10 2023-10-06T05:07:29.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.744+0000 7f5c497e3b80 10 client.10374 fetch_fsmap learned FSMap version 105 2023-10-06T05:07:29.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.744+0000 7f5c497e3b80 1 -- 172.21.15.55:0/2027038891 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x558d71fb0fe0 con 0x558d72226a10 2023-10-06T05:07:29.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.744+0000 7f5c12ffd700 1 -- 172.21.15.55:0/2027038891 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558d72218940 con 0x558d72226a10 2023-10-06T05:07:29.753 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.744+0000 7f5c12ffd700 1 -- 172.21.15.55:0/2027038891 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 105) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5c2005b490 con 0x558d72226a10 2023-10-06T05:07:29.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.744+0000 7f5c497e3b80 10 client.10374 fetch_fsmap finished waiting for FSMap version 105 2023-10-06T05:07:29.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.744+0000 7f5c497e3b80 10 client.10374 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:07:29.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.744+0000 7f5c497e3b80 20 client.10374 populate_metadata read hostname 'smithi055' 2023-10-06T05:07:29.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.744+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/2027038891 >> [v2:172.21.15.55:6833/3632968604,v1:172.21.15.55:6835/3632968604] conn(0x558d722522f0 0x558d722526c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.744+0000 7f5c497e3b80 4 client.10374 mds_command: new command op to 10227 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T05:07:29.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.744+0000 7f5c497e3b80 1 -- 172.21.15.55:0/2027038891 --> [v2:172.21.15.55:6833/3632968604,v1:172.21.15.55:6835/3632968604] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d721f5870 con 0x558d722522f0 2023-10-06T05:07:29.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.744+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/2027038891 >> [v2:172.21.15.55:6833/3632968604,v1:172.21.15.55:6835/3632968604] conn(0x558d722522f0 0x558d722526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.745+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/2027038891 >> [v2:172.21.15.55:6833/3632968604,v1:172.21.15.55:6835/3632968604] conn(0x558d722522f0 0x558d722526c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cca03821232e589 server_cookie=3f55f32eca4a1cda in_seq=0 out_seq=0 2023-10-06T05:07:29.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.745+0000 7f5c12ffd700 10 client.10374 ms_handle_connect on v2:172.21.15.55:6833/3632968604 2023-10-06T05:07:29.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c12ffd700 1 -- 172.21.15.55:0/2027038891 <== mds.0 v2:172.21.15.55:6833/3632968604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x558d721f5870 con 0x558d722522f0 2023-10-06T05:07:29.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c12ffd700 10 client.10374 handle_command_reply: tid=0 2023-10-06T05:07:29.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c497e3b80 1 client.10374 shutdown 2023-10-06T05:07:29.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c497e3b80 1 -- 172.21.15.55:0/2027038891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72226a10 msgr2=0x558d7226b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/2027038891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72226a10 0x558d7226b1b0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f5c20007fd0 tx=0x7f5c20081ed0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c497e3b80 1 -- 172.21.15.55:0/2027038891 shutdown_connections 2023-10-06T05:07:29.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/2027038891 >> [v2:172.21.15.55:6833/3632968604,v1:172.21.15.55:6835/3632968604] conn(0x558d722522f0 0x558d722526c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/2027038891 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 0x558d7226d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/2027038891 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d72267b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/2027038891 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72226a10 0x558d7226b1b0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c12ffd700 0 client.10374 ms_handle_reset on v2:172.21.15.55:6833/3632968604 2023-10-06T05:07:29.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c497e3b80 1 -- 172.21.15.55:0/2027038891 >> 172.21.15.55:0/2027038891 conn(0x558d72224e80 msgr2=0x558d72223b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:29.755 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c497e3b80 1 -- 172.21.15.55:0/2027038891 shutdown_connections 2023-10-06T05:07:29.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c497e3b80 1 -- 172.21.15.55:0/2027038891 wait complete. 2023-10-06T05:07:29.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.746+0000 7f5c497e3b80 20 client.10374 trim_cache size 0 max 16384 2023-10-06T05:07:29.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.764+0000 7f5c497e3b80 1 Processor -- start 2023-10-06T05:07:29.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.764+0000 7f5c497e3b80 1 -- start start 2023-10-06T05:07:29.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c497e3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d722252c0 0x558d7223a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c497e3b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d72226a10 0x558d7221a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.767 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c497e3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 0x558d7221a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c497e3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d7226c100 con 0x558d722252c0 2023-10-06T05:07:29.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c497e3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d7226a550 con 0x558d72244e50 2023-10-06T05:07:29.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c497e3b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d72253f00 con 0x558d72226a10 2023-10-06T05:07:29.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c38d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d722252c0 0x558d7223a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c38d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d722252c0 0x558d7223a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43924/0 (socket says 172.21.15.55:43924) 2023-10-06T05:07:29.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c38d9b700 1 -- 172.21.15.55:0/2199580099 learned_addr learned my addr 172.21.15.55:0/2199580099 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:29.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c33fff700 1 --2- 172.21.15.55:0/2199580099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d72226a10 0x558d7221a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c3959c700 1 --2- 172.21.15.55:0/2199580099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 0x558d7221a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c38d9b700 1 -- 172.21.15.55:0/2199580099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d72226a10 msgr2=0x558d7221a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:29.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/2199580099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d72226a10 0x558d7221a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c38d9b700 1 -- 172.21.15.55:0/2199580099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 msgr2=0x558d7221a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:29.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/2199580099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 0x558d7221a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.765+0000 7f5c38d9b700 1 -- 172.21.15.55:0/2199580099 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d72142780 con 0x558d722252c0 2023-10-06T05:07:29.769 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.766+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/2199580099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d722252c0 0x558d7223a990 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f5c3406fe40 tx=0x7f5c28014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c72a2ad8278cf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:29.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.766+0000 7f5c1e7fc700 1 -- 172.21.15.55:0/2199580099 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28030050 con 0x558d722252c0 2023-10-06T05:07:29.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.766+0000 7f5c1e7fc700 1 -- 172.21.15.55:0/2199580099 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c28031040 con 0x558d722252c0 2023-10-06T05:07:29.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.766+0000 7f5c1e7fc700 1 -- 172.21.15.55:0/2199580099 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c28034250 con 0x558d722252c0 2023-10-06T05:07:29.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.766+0000 7f5c497e3b80 1 -- 172.21.15.55:0/2199580099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d722252c0 msgr2=0x558d7223a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.766+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/2199580099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d722252c0 0x558d7223a990 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f5c3406fe40 tx=0x7f5c28014000 comp rx=0 tx=0).stop 2023-10-06T05:07:29.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.766+0000 7f5c497e3b80 1 -- 172.21.15.55:0/2199580099 shutdown_connections 2023-10-06T05:07:29.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.766+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/2199580099 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72244e50 0x558d7221a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.767+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/2199580099 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d722252c0 0x558d7223a990 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.767+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/2199580099 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d72226a10 0x558d7221a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.767+0000 7f5c497e3b80 1 -- 172.21.15.55:0/2199580099 >> 172.21.15.55:0/2199580099 conn(0x558d72224e80 msgr2=0x558d7226b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:29.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.767+0000 7f5c497e3b80 1 -- 172.21.15.55:0/2199580099 shutdown_connections 2023-10-06T05:07:29.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.767+0000 7f5c497e3b80 1 -- 172.21.15.55:0/2199580099 wait complete. 2023-10-06T05:07:29.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.767+0000 7f5c497e3b80 1 -- start start 2023-10-06T05:07:29.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.767+0000 7f5c497e3b80 1 Processor -- start 2023-10-06T05:07:29.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.768+0000 7f5c497e3b80 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d722d3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.768+0000 7f5c497e3b80 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72226a10 0x558d7221a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.768+0000 7f5c497e3b80 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72244e50 0x558d722cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.768+0000 7f5c497e3b80 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d722d55e0 con 0x558d72244e50 2023-10-06T05:07:29.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.768+0000 7f5c497e3b80 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d722d44d0 con 0x558d72226a10 2023-10-06T05:07:29.775 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.768+0000 7f5c497e3b80 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x558d7221ada0 con 0x558d722252c0 2023-10-06T05:07:29.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.768+0000 7f5c497e3b80 10 client.0 _ll_register_callbacks cb 0x558d721e7020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T05:07:29.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.769+0000 7f5c3959c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72244e50 0x558d722cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.769+0000 7f5c3959c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72244e50 0x558d722cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43940/0 (socket says 172.21.15.55:43940) 2023-10-06T05:07:29.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.769+0000 7f5c3959c700 1 -- 172.21.15.55:0/3908857417 learned_addr learned my addr 172.21.15.55:0/3908857417 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:29.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.769+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d722d3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.769+0000 7f5c33fff700 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72226a10 0x558d7221a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.769+0000 7f5c3959c700 1 -- 172.21.15.55:0/3908857417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 msgr2=0x558d722d3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:29.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.769+0000 7f5c3959c700 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d722d3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.769+0000 7f5c3959c700 1 -- 172.21.15.55:0/3908857417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72226a10 msgr2=0x558d7221a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:29.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.769+0000 7f5c3959c700 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72226a10 0x558d7221a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.769+0000 7f5c3959c700 1 -- 172.21.15.55:0/3908857417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c340c7bc0 con 0x558d72244e50 2023-10-06T05:07:29.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.769+0000 7f5c33fff700 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72226a10 0x558d7221a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:29.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.769+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d722d3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:29.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.769+0000 7f5c3959c700 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72244e50 0x558d722cd960 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0072f0 tx=0x7f5c2c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f223b5ba446b530 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:29.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.769+0000 7f5c12ffd700 10 client.0 ms_handle_connect on v2:172.21.15.55:3300/0 2023-10-06T05:07:29.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c12ffd700 1 -- 172.21.15.55:0/3908857417 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c000f40 con 0x558d72244e50 2023-10-06T05:07:29.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c497e3b80 1 -- 172.21.15.55:0/3908857417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c3414d070 con 0x558d72244e50 2023-10-06T05:07:29.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c12ffd700 1 -- 172.21.15.55:0/3908857417 <== mon.0 v2:172.21.15.55:3300/0 2 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f5c2c025470 con 0x558d72244e50 2023-10-06T05:07:29.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c12ffd700 1 -- 172.21.15.55:0/3908857417 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c2c0010a0 con 0x558d72244e50 2023-10-06T05:07:29.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c497e3b80 10 client.10380 fetch_fsmap learned FSMap version 105 2023-10-06T05:07:29.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c497e3b80 1 -- 172.21.15.55:0/3908857417 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) v3 -- 0x558d72129470 con 0x558d72244e50 2023-10-06T05:07:29.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c12ffd700 1 -- 172.21.15.55:0/3908857417 <== mon.0 v2:172.21.15.55:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3414d070 con 0x558d72244e50 2023-10-06T05:07:29.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c12ffd700 1 -- 172.21.15.55:0/3908857417 <== mon.0 v2:172.21.15.55:3300/0 5 ==== fsmap(e 105) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5c2c025860 con 0x558d72244e50 2023-10-06T05:07:29.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c497e3b80 10 client.10380 fetch_fsmap finished waiting for FSMap version 105 2023-10-06T05:07:29.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c497e3b80 10 client.10380 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T05:07:29.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c497e3b80 20 client.10380 populate_metadata read hostname 'smithi055' 2023-10-06T05:07:29.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.55:6833/3632968604,v1:172.21.15.55:6835/3632968604] conn(0x558d7228b910 0x558d7228bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c497e3b80 4 client.10380 mds_command: new command op to 10227 tid=0[{"prefix": "status"}] 2023-10-06T05:07:29.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c497e3b80 1 -- 172.21.15.55:0/3908857417 --> [v2:172.21.15.55:6833/3632968604,v1:172.21.15.55:6835/3632968604] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f5c2c00a6e0 con 0x558d7228b910 2023-10-06T05:07:29.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.770+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.55:6833/3632968604,v1:172.21.15.55:6835/3632968604] conn(0x558d7228b910 0x558d7228bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.771+0000 7f5c38d9b700 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.55:6833/3632968604,v1:172.21.15.55:6835/3632968604] conn(0x558d7228b910 0x558d7228bce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe02445a95e26d6 server_cookie=f66abc75594e695d in_seq=0 out_seq=0 2023-10-06T05:07:29.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.771+0000 7f5c12ffd700 10 client.10380 ms_handle_connect on v2:172.21.15.55:6833/3632968604 2023-10-06T05:07:29.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.771+0000 7f5c12ffd700 1 -- 172.21.15.55:0/3908857417 <== mds.0 v2:172.21.15.55:6833/3632968604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x7f5c2c00a6e0 con 0x558d7228b910 2023-10-06T05:07:29.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.771+0000 7f5c12ffd700 10 client.10380 handle_command_reply: tid=0 2023-10-06T05:07:29.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.771+0000 7f5c497e3b80 1 client.10380 shutdown 2023-10-06T05:07:29.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.771+0000 7f5c497e3b80 1 -- 172.21.15.55:0/3908857417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72244e50 msgr2=0x558d722cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.771+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72244e50 0x558d722cd960 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0072f0 tx=0x7f5c2c019000 comp rx=0 tx=0).stop 2023-10-06T05:07:29.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.771+0000 7f5c497e3b80 1 -- 172.21.15.55:0/3908857417 shutdown_connections 2023-10-06T05:07:29.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.771+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x558d72244e50 0x558d722cd960 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.771+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x558d722252c0 0x558d722d3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.771+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.55:6833/3632968604,v1:172.21.15.55:6835/3632968604] conn(0x558d7228b910 0x558d7228bce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.771+0000 7f5c497e3b80 1 --2- 172.21.15.55:0/3908857417 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x558d72226a10 0x558d7221a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.771+0000 7f5c12ffd700 0 client.10380 ms_handle_reset on v2:172.21.15.55:6833/3632968604 2023-10-06T05:07:29.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.771+0000 7f5c497e3b80 1 -- 172.21.15.55:0/3908857417 >> 172.21.15.55:0/3908857417 conn(0x558d72224e80 msgr2=0x558d722690c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:29.779 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.772+0000 7f5c497e3b80 1 -- 172.21.15.55:0/3908857417 shutdown_connections 2023-10-06T05:07:29.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.772+0000 7f5c497e3b80 1 -- 172.21.15.55:0/3908857417 wait complete. 2023-10-06T05:07:29.780 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.772+0000 7f5c497e3b80 20 client.10380 trim_cache size 0 max 16384 2023-10-06T05:07:29.782 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T05:07:29.782 INFO:teuthology.orchestra.run.smithi055.stdout: "cluster_fsid": "8132c8ba-ee1c-41a2-b781-913925034ab6", 2023-10-06T05:07:29.782 INFO:teuthology.orchestra.run.smithi055.stdout: "whoami": 0, 2023-10-06T05:07:29.782 INFO:teuthology.orchestra.run.smithi055.stdout: "id": 10227, 2023-10-06T05:07:29.782 INFO:teuthology.orchestra.run.smithi055.stdout: "want_state": "up:active", 2023-10-06T05:07:29.782 INFO:teuthology.orchestra.run.smithi055.stdout: "state": "up:active", 2023-10-06T05:07:29.782 INFO:teuthology.orchestra.run.smithi055.stdout: "fs_name": "cephfs", 2023-10-06T05:07:29.782 INFO:teuthology.orchestra.run.smithi055.stdout: "rank_uptime": 6.5459386029999997, 2023-10-06T05:07:29.782 INFO:teuthology.orchestra.run.smithi055.stdout: "mdsmap_epoch": 105, 2023-10-06T05:07:29.783 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch": 207, 2023-10-06T05:07:29.783 INFO:teuthology.orchestra.run.smithi055.stdout: "osdmap_epoch_barrier": 207, 2023-10-06T05:07:29.783 INFO:teuthology.orchestra.run.smithi055.stdout: "uptime": 12.288551508999999 2023-10-06T05:07:29.783 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T05:07:29.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.772+0000 7f5c39d9d700 1 -- 172.21.15.55:0/3596426134 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5c18066f30 msgr2=0x7f5c180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.772+0000 7f5c39d9d700 1 --2- 172.21.15.55:0/3596426134 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5c18066f30 0x7f5c180693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5c280093f0 tx=0x7f5c28017000 comp rx=0 tx=0).stop 2023-10-06T05:07:29.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.772+0000 7f5c39d9d700 1 -- 172.21.15.55:0/3596426134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c34150180 msgr2=0x7f5c340697c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.772+0000 7f5c39d9d700 1 --2- 172.21.15.55:0/3596426134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c34150180 0x7f5c340697c0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000c7e0 tx=0x7f5c20007df0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.772+0000 7f5c39d9d700 1 -- 172.21.15.55:0/3596426134 shutdown_connections 2023-10-06T05:07:29.784 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.772+0000 7f5c39d9d700 1 --2- 172.21.15.55:0/3596426134 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5c18066f30 0x7f5c180693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.772+0000 7f5c39d9d700 1 --2- 172.21.15.55:0/3596426134 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5c34150550 0x7f5c34069d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.772+0000 7f5c39d9d700 1 --2- 172.21.15.55:0/3596426134 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5c340c7410 0x7f5c341671b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.772+0000 7f5c39d9d700 1 --2- 172.21.15.55:0/3596426134 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5c34150180 0x7f5c340697c0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.772+0000 7f5c39d9d700 1 -- 172.21.15.55:0/3596426134 >> 172.21.15.55:0/3596426134 conn(0x7f5c340cc1f0 msgr2=0x7f5c3414b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:29.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.773+0000 7f5c39d9d700 1 -- 172.21.15.55:0/3596426134 shutdown_connections 2023-10-06T05:07:29.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.773+0000 7f5c39d9d700 1 -- 172.21.15.55:0/3596426134 wait complete. 2023-10-06T05:07:29.791 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T05:07:29.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.976+0000 7f75cb19d700 1 Processor -- start 2023-10-06T05:07:29.978 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.976+0000 7f75cb19d700 1 -- start start 2023-10-06T05:07:29.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.977+0000 7f75cb19d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f75c4156310 0x7f75c40c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.978+0000 7f75cb19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c40ca860 con 0x7f75c41566e0 2023-10-06T05:07:29.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.978+0000 7f75cb19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c40cab40 con 0x7f75c40ae710 2023-10-06T05:07:29.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.978+0000 7f75cb19d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c40cae20 con 0x7f75c4156310 2023-10-06T05:07:29.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.978+0000 7f75c999a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f75c41566e0 0x7f75c40ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:33284/0 (socket says 172.21.15.55:33284) 2023-10-06T05:07:29.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.978+0000 7f75c999a700 1 -- 172.21.15.55:0/1582788386 learned_addr learned my addr 172.21.15.55:0/1582788386 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:29.980 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.978+0000 7f75ca19b700 1 --2- 172.21.15.55:0/1582788386 >> v2:172.21.15.55:3300/0 conn(0x7f75c4156310 0x7f75c40c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.978+0000 7f75c9199700 1 -- 172.21.15.55:0/1582788386 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1435813582 0 0) 0x7f75c40ca860 con 0x7f75c41566e0 2023-10-06T05:07:29.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.978+0000 7f75ca19b700 1 -- 172.21.15.55:0/1582788386 >> v1:172.21.15.163:6789/0 conn(0x7f75c40ae710 legacy=0x7f75c40c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.978+0000 7f75ca19b700 1 -- 172.21.15.55:0/1582788386 >> v1:172.21.15.55:6789/0 conn(0x7f75c41566e0 legacy=0x7f75c40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.978+0000 7f75ca19b700 1 -- 172.21.15.55:0/1582788386 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c40cb100 con 0x7f75c4156310 2023-10-06T05:07:29.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.979+0000 7f75ca19b700 1 --2- 172.21.15.55:0/1582788386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75c4156310 0x7f75c40c9dc0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f75c00098e0 tx=0x7f75c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19416f23c8fcf01 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:29.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.979+0000 7f75c9199700 1 -- 172.21.15.55:0/1582788386 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0009e80 con 0x7f75c4156310 2023-10-06T05:07:29.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.979+0000 7f75c9199700 1 -- 172.21.15.55:0/1582788386 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c000b860 con 0x7f75c4156310 2023-10-06T05:07:29.982 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.979+0000 7f75c9199700 1 -- 172.21.15.55:0/1582788386 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c000bbe0 con 0x7f75c4156310 2023-10-06T05:07:29.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.979+0000 7f75cb19d700 1 -- 172.21.15.55:0/1582788386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75c4156310 msgr2=0x7f75c40c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.979+0000 7f75cb19d700 1 --2- 172.21.15.55:0/1582788386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75c4156310 0x7f75c40c9dc0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f75c00098e0 tx=0x7f75c00091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.979+0000 7f75cb19d700 1 -- 172.21.15.55:0/1582788386 shutdown_connections 2023-10-06T05:07:29.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.979+0000 7f75cb19d700 1 --2- 172.21.15.55:0/1582788386 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75c4156310 0x7f75c40c9dc0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.979+0000 7f75cb19d700 1 -- 172.21.15.55:0/1582788386 >> 172.21.15.55:0/1582788386 conn(0x7f75c40cc1f0 msgr2=0x7f75c40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:29.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.980+0000 7f75cb19d700 1 -- 172.21.15.55:0/1582788386 shutdown_connections 2023-10-06T05:07:29.983 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.980+0000 7f75cb19d700 1 -- 172.21.15.55:0/1582788386 wait complete. 2023-10-06T05:07:29.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.981+0000 7f75cb19d700 1 Processor -- start 2023-10-06T05:07:29.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.981+0000 7f75cb19d700 1 -- start start 2023-10-06T05:07:29.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.982+0000 7f75cb19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75c40ae710 0x7f75c415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.985 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.982+0000 7f75cb19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75c4156310 0x7f75c40c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.982+0000 7f75cb19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75c41566e0 0x7f75c40c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.982+0000 7f75cb19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c415d0d0 con 0x7f75c40ae710 2023-10-06T05:07:29.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.982+0000 7f75cb19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c40c3ab0 con 0x7f75c4156310 2023-10-06T05:07:29.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.982+0000 7f75cb19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c40c37e0 con 0x7f75c41566e0 2023-10-06T05:07:29.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.982+0000 7f75ca19b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75c40ae710 0x7f75c415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.982+0000 7f75ca19b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75c40ae710 0x7f75c415beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43962/0 (socket says 172.21.15.55:43962) 2023-10-06T05:07:29.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.982+0000 7f75ca19b700 1 -- 172.21.15.55:0/3847457720 learned_addr learned my addr 172.21.15.55:0/3847457720 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:29.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.983+0000 7f75c999a700 1 --2- 172.21.15.55:0/3847457720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75c4156310 0x7f75c40c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.982+0000 7f75ca99c700 1 --2- 172.21.15.55:0/3847457720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75c41566e0 0x7f75c40c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.986 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.983+0000 7f75ca19b700 1 -- 172.21.15.55:0/3847457720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75c41566e0 msgr2=0x7f75c40c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.983+0000 7f75ca19b700 1 --2- 172.21.15.55:0/3847457720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75c41566e0 0x7f75c40c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.983+0000 7f75ca19b700 1 -- 172.21.15.55:0/3847457720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75c4156310 msgr2=0x7f75c40c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:29.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.983+0000 7f75ca19b700 1 --2- 172.21.15.55:0/3847457720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75c4156310 0x7f75c40c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:29.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.983+0000 7f75ca19b700 1 -- 172.21.15.55:0/3847457720 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c0009420 con 0x7f75c40ae710 2023-10-06T05:07:29.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.983+0000 7f75ca19b700 1 --2- 172.21.15.55:0/3847457720 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75c40ae710 0x7f75c415beb0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f75c000b3c0 tx=0x7f75c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:29.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.983+0000 7f75ba7fc700 1 -- 172.21.15.55:0/3847457720 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0011420 con 0x7f75c40ae710 2023-10-06T05:07:29.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.983+0000 7f75ba7fc700 1 -- 172.21.15.55:0/3847457720 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c0011580 con 0x7f75c40ae710 2023-10-06T05:07:29.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.983+0000 7f75ba7fc700 1 -- 172.21.15.55:0/3847457720 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c0011940 con 0x7f75c40ae710 2023-10-06T05:07:29.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.983+0000 7f75cb19d700 1 -- 172.21.15.55:0/3847457720 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c415c7a0 con 0x7f75c40ae710 2023-10-06T05:07:29.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.984+0000 7f75cb19d700 1 -- 172.21.15.55:0/3847457720 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c415cd60 con 0x7f75c40ae710 2023-10-06T05:07:29.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.984+0000 7f75cb19d700 1 -- 172.21.15.55:0/3847457720 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c404c3c0 con 0x7f75c40ae710 2023-10-06T05:07:29.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.985+0000 7f75ba7fc700 1 -- 172.21.15.55:0/3847457720 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f75c0011cf0 con 0x7f75c40ae710 2023-10-06T05:07:29.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.987+0000 7f75ba7fc700 1 --2- 172.21.15.55:0/3847457720 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f75a8066f30 0x7f75a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:29.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.987+0000 7f75ba7fc700 1 -- 172.21.15.55:0/3847457720 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f75c00863f0 con 0x7f75c40ae710 2023-10-06T05:07:29.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.987+0000 7f75c999a700 1 --2- 172.21.15.55:0/3847457720 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f75a8066f30 0x7f75a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:29.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.988+0000 7f75c999a700 1 --2- 172.21.15.55:0/3847457720 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f75a8066f30 0x7f75a80693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f75c40afc70 tx=0x7f75b4009040 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:29.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:29.991+0000 7f75ba7fc700 1 -- 172.21.15.55:0/3847457720 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f75c00532a0 con 0x7f75c40ae710 2023-10-06T05:07:30.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:30.238+0000 7f75cb19d700 1 -- 172.21.15.55:0/3847457720 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f75c40cd0b0 con 0x7f75c40ae710 2023-10-06T05:07:30.240 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:30.238+0000 7f75ba7fc700 1 -- 172.21.15.55:0/3847457720 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) v1 ==== 53+0+73 (secure 0 0 0) 0x7f75c00568c0 con 0x7f75c40ae710 2023-10-06T05:07:30.240 INFO:teuthology.orchestra.run.smithi055.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T05:07:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:30.241+0000 7f75cb19d700 1 -- 172.21.15.55:0/3847457720 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f75a8066f30 msgr2=0x7f75a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:30.241+0000 7f75cb19d700 1 --2- 172.21.15.55:0/3847457720 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f75a8066f30 0x7f75a80693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f75c40afc70 tx=0x7f75b4009040 comp rx=0 tx=0).stop 2023-10-06T05:07:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:30.241+0000 7f75cb19d700 1 -- 172.21.15.55:0/3847457720 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75c40ae710 msgr2=0x7f75c415beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:30.241+0000 7f75cb19d700 1 --2- 172.21.15.55:0/3847457720 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75c40ae710 0x7f75c415beb0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f75c000b3c0 tx=0x7f75c0017000 comp rx=0 tx=0).stop 2023-10-06T05:07:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:30.241+0000 7f75cb19d700 1 -- 172.21.15.55:0/3847457720 shutdown_connections 2023-10-06T05:07:30.243 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:30.241+0000 7f75cb19d700 1 --2- 172.21.15.55:0/3847457720 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f75a8066f30 0x7f75a80693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:30.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:30.241+0000 7f75cb19d700 1 --2- 172.21.15.55:0/3847457720 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f75c41566e0 0x7f75c40c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:30.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:30.241+0000 7f75cb19d700 1 --2- 172.21.15.55:0/3847457720 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f75c40ae710 0x7f75c415beb0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:30.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:30.241+0000 7f75cb19d700 1 --2- 172.21.15.55:0/3847457720 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f75c4156310 0x7f75c40c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:30.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:30.241+0000 7f75cb19d700 1 -- 172.21.15.55:0/3847457720 >> 172.21.15.55:0/3847457720 conn(0x7f75c40cc1f0 msgr2=0x7f75c4151a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:30.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:30.241+0000 7f75cb19d700 1 -- 172.21.15.55:0/3847457720 shutdown_connections 2023-10-06T05:07:30.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:30.241+0000 7f75cb19d700 1 -- 172.21.15.55:0/3847457720 wait complete. 2023-10-06T05:07:30.262 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-06T05:07:30.262 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T05:07:30.262 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi055.front.sepia.ceph.com 2023-10-06T05:07:30.262 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T05:07:30.262 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T05:07:30.262 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-06T05:07:30.262 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-06T05:07:30.262 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T05:07:30.262 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:07:30.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:30.263 DEBUG:teuthology.orchestra.run.smithi055:> ip addr 2023-10-06T05:07:30.301 INFO:teuthology.orchestra.run.smithi055.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T05:07:30.301 INFO:teuthology.orchestra.run.smithi055.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T05:07:30.301 INFO:teuthology.orchestra.run.smithi055.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T05:07:30.301 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:07:30.301 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 ::1/128 scope host 2023-10-06T05:07:30.302 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:07:30.302 INFO:teuthology.orchestra.run.smithi055.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:07:30.302 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d6 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:07:30.302 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f0 2023-10-06T05:07:30.302 INFO:teuthology.orchestra.run.smithi055.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:07:30.302 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether 0c:c4:7a:6b:fd:d7 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:07:30.302 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp7s0f1 2023-10-06T05:07:30.302 INFO:teuthology.orchestra.run.smithi055.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T05:07:30.302 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:06 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:07:30.302 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f0 2023-10-06T05:07:30.302 INFO:teuthology.orchestra.run.smithi055.stdout: inet 172.21.15.55/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-10-06T05:07:30.302 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft 36591sec preferred_lft 36591sec 2023-10-06T05:07:30.303 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::ae1f:6bff:fef8:2606/64 scope link 2023-10-06T05:07:30.303 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:07:30.303 INFO:teuthology.orchestra.run.smithi055.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T05:07:30.303 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether ac:1f:6b:f8:26:07 brd ff:ff:ff:ff:ff:ff 2023-10-06T05:07:30.303 INFO:teuthology.orchestra.run.smithi055.stdout: altname enp1s0f1 2023-10-06T05:07:30.303 INFO:teuthology.orchestra.run.smithi055.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T05:07:30.303 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff 2023-10-06T05:07:30.303 INFO:teuthology.orchestra.run.smithi055.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T05:07:30.303 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:07:30.303 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::1099:8eff:fe6d:102/64 scope link 2023-10-06T05:07:30.303 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:07:30.304 INFO:teuthology.orchestra.run.smithi055.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:07:30.304 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether f2:5d:53:d2:00:0a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:07:30.304 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::f05d:53ff:fed2:a/64 scope link 2023-10-06T05:07:30.304 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:07:30.304 INFO:teuthology.orchestra.run.smithi055.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T05:07:30.304 INFO:teuthology.orchestra.run.smithi055.stdout: link/ether e2:37:33:b7:6c:8f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:07:30.304 INFO:teuthology.orchestra.run.smithi055.stdout: inet6 fe80::e037:33ff:feb7:6c8f/64 scope link 2023-10-06T05:07:30.304 INFO:teuthology.orchestra.run.smithi055.stdout: valid_lft forever preferred_lft forever 2023-10-06T05:07:30.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:30.305 DEBUG:teuthology.orchestra.run.smithi055:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T05:07:30.402 INFO:teuthology.orchestra.run.smithi055.stdout:1 2023-10-06T05:07:30.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:30.405 DEBUG:teuthology.orchestra.run.smithi055:> route 2023-10-06T05:07:30.444 INFO:teuthology.orchestra.run.smithi055.stdout:Kernel IP routing table 2023-10-06T05:07:30.444 INFO:teuthology.orchestra.run.smithi055.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T05:07:30.445 INFO:teuthology.orchestra.run.smithi055.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-10-06T05:07:30.445 INFO:teuthology.orchestra.run.smithi055.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-10-06T05:07:30.445 INFO:teuthology.orchestra.run.smithi055.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T05:07:30.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:30.445 DEBUG:teuthology.orchestra.run.smithi055:> (cd / && exec bash -c ' 2023-10-06T05:07:30.445 DEBUG:teuthology.orchestra.run.smithi055:> set -e 2023-10-06T05:07:30.446 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-06T05:07:30.446 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-06T05:07:30.446 DEBUG:teuthology.orchestra.run.smithi055:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-06T05:07:30.446 DEBUG:teuthology.orchestra.run.smithi055:> ') 2023-10-06T05:07:30.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:30.644 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T05:07:30.680 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:07:30.680 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:07:30.681 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:07:30.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:30.681 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link set brx.0 up 2023-10-06T05:07:30.783 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:07:30.783 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T05:07:30.819 INFO:teuthology.orchestra.run.smithi055.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T05:07:30.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T05:07:30.820 DEBUG:teuthology.orchestra.run.smithi055:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:07:30.896 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-10-06T05:07:30.896 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T05:07:30.896 DEBUG:teuthology.orchestra.run.smithi055:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:07:31.031 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:07:31.031 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:07:31.031 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T05:07:31.048 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:07:31.048 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:07:31.048 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T05:07:31.049 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T05:07:31.049 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:07:31.070 INFO:teuthology.orchestra.run.smithi055.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-10-06T05:07:31.070 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-10-06T05:07:31.070 INFO:teuthology.orchestra.run.smithi055.stdout:invalid new device string format 2023-10-06T05:07:31.070 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:3300,172.21.15.163:3300,172.21.15.163:3301" 2023-10-06T05:07:31.070 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: switching to using v1 address with old syntax 2023-10-06T05:07:31.071 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: resolved to: "172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790" 2023-10-06T05:07:31.071 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.55:6789,172.21.15.163:6789,172.21.15.163:6790:/ 2023-10-06T05:07:31.071 INFO:teuthology.orchestra.run.smithi055.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=8132c8ba-ee1c-41a2-b781-913925034ab6" will pass to kernel 2023-10-06T05:07:31.076 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-06T05:07:31.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:31.076 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T05:07:31.139 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-06T05:07:31.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.327+0000 7fa07d19d700 1 Processor -- start 2023-10-06T05:07:31.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.327+0000 7fa07d19d700 1 -- start start 2023-10-06T05:07:31.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.328+0000 7fa07d19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa078159420 0x7fa0780b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:31.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.328+0000 7fa07d19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0781597f0 0x7fa0780b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:31.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.328+0000 7fa07d19d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0780b11d0 con 0x7fa07815d230 2023-10-06T05:07:31.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.328+0000 7fa07d19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0780b14b0 con 0x7fa0781597f0 2023-10-06T05:07:31.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.328+0000 7fa07d19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0780b1790 con 0x7fa078159420 2023-10-06T05:07:31.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.329+0000 7fa07c99c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa07815d230 0x7fa0780b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:33286/0 (socket says 172.21.15.55:33286) 2023-10-06T05:07:31.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.329+0000 7fa07c99c700 1 -- 172.21.15.55:0/1691745413 learned_addr learned my addr 172.21.15.55:0/1691745413 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:31.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.329+0000 7fa077fff700 1 --2- 172.21.15.55:0/1691745413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa078159420 0x7fa0780b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:31.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.329+0000 7fa0777fe700 1 --2- 172.21.15.55:0/1691745413 >> v2:172.21.15.163:3300/0 conn(0x7fa0781597f0 0x7fa0780b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:31.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.329+0000 7fa076ffd700 1 -- 172.21.15.55:0/1691745413 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694156891 0 0) 0x7fa0780b11d0 con 0x7fa07815d230 2023-10-06T05:07:31.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.329+0000 7fa076ffd700 1 -- 172.21.15.55:0/1691745413 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa060003570 con 0x7fa07815d230 2023-10-06T05:07:31.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.329+0000 7fa0777fe700 1 -- 172.21.15.55:0/1691745413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa078159420 msgr2=0x7fa0780b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:31.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.329+0000 7fa0777fe700 1 --2- 172.21.15.55:0/1691745413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa078159420 0x7fa0780b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.329+0000 7fa0777fe700 1 -- 172.21.15.55:0/1691745413 >> v1:172.21.15.55:6789/0 conn(0x7fa07815d230 legacy=0x7fa0780b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:31.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.329+0000 7fa076ffd700 1 -- 172.21.15.55:0/1691745413 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 33874366 0 0) 0x7fa060003570 con 0x7fa07815d230 2023-10-06T05:07:31.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.329+0000 7fa0777fe700 1 -- 172.21.15.55:0/1691745413 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0780b1a70 con 0x7fa0781597f0 2023-10-06T05:07:31.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.330+0000 7fa0777fe700 1 --2- 172.21.15.55:0/1691745413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0781597f0 0x7fa0780b2270 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0098e0 tx=0x7fa06c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a105fa299ac03edb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:31.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.330+0000 7fa076ffd700 1 -- 172.21.15.55:0/1691745413 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0700017f0 con 0x7fa0781597f0 2023-10-06T05:07:31.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.330+0000 7fa076ffd700 1 -- 172.21.15.55:0/1691745413 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06c011420 con 0x7fa0781597f0 2023-10-06T05:07:31.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.330+0000 7fa076ffd700 1 -- 172.21.15.55:0/1691745413 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06c0117c0 con 0x7fa0781597f0 2023-10-06T05:07:31.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.331+0000 7fa07d19d700 1 -- 172.21.15.55:0/1691745413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0781597f0 msgr2=0x7fa0780b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:31.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.331+0000 7fa07d19d700 1 --2- 172.21.15.55:0/1691745413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0781597f0 0x7fa0780b2270 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0098e0 tx=0x7fa06c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.331+0000 7fa07d19d700 1 -- 172.21.15.55:0/1691745413 shutdown_connections 2023-10-06T05:07:31.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.331+0000 7fa07d19d700 1 --2- 172.21.15.55:0/1691745413 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa078159420 0x7fa0780b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.331+0000 7fa07d19d700 1 --2- 172.21.15.55:0/1691745413 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0781597f0 0x7fa0780b2270 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.331+0000 7fa07d19d700 1 -- 172.21.15.55:0/1691745413 >> 172.21.15.55:0/1691745413 conn(0x7fa0780d0660 msgr2=0x7fa0780d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:31.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.331+0000 7fa07d19d700 1 -- 172.21.15.55:0/1691745413 shutdown_connections 2023-10-06T05:07:31.334 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.331+0000 7fa07d19d700 1 -- 172.21.15.55:0/1691745413 wait complete. 2023-10-06T05:07:31.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.332+0000 7fa07d19d700 1 Processor -- start 2023-10-06T05:07:31.336 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.332+0000 7fa07d19d700 1 -- start start 2023-10-06T05:07:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.333+0000 7fa07d19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa078159420 0x7fa07814f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.333+0000 7fa07d19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0781597f0 0x7fa078169c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.333+0000 7fa07d19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa07815d230 0x7fa07814c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.333+0000 7fa07d19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07814ebc0 con 0x7fa078159420 2023-10-06T05:07:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.333+0000 7fa07d19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07814cb40 con 0x7fa0781597f0 2023-10-06T05:07:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.333+0000 7fa07d19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07814e570 con 0x7fa07815d230 2023-10-06T05:07:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.333+0000 7fa077fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa078159420 0x7fa07814f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.333+0000 7fa077fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa078159420 0x7fa07814f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43964/0 (socket says 172.21.15.55:43964) 2023-10-06T05:07:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.333+0000 7fa077fff700 1 -- 172.21.15.55:0/1229643345 learned_addr learned my addr 172.21.15.55:0/1229643345 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.334+0000 7fa0777fe700 1 --2- 172.21.15.55:0/1229643345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0781597f0 0x7fa078169c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:31.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.334+0000 7fa07c99c700 1 --2- 172.21.15.55:0/1229643345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa07815d230 0x7fa07814c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.334+0000 7fa077fff700 1 -- 172.21.15.55:0/1229643345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa07815d230 msgr2=0x7fa07814c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.334+0000 7fa077fff700 1 --2- 172.21.15.55:0/1229643345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa07815d230 0x7fa07814c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.334+0000 7fa077fff700 1 -- 172.21.15.55:0/1229643345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0781597f0 msgr2=0x7fa078169c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.334+0000 7fa077fff700 1 --2- 172.21.15.55:0/1229643345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0781597f0 0x7fa078169c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.334+0000 7fa077fff700 1 -- 172.21.15.55:0/1229643345 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c009420 con 0x7fa078159420 2023-10-06T05:07:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.334+0000 7fa077fff700 1 --2- 172.21.15.55:0/1229643345 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa078159420 0x7fa07814f050 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fa06800ec80 tx=0x7fa0680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.334+0000 7fa05ffff700 1 -- 172.21.15.55:0/1229643345 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06800c9d0 con 0x7fa078159420 2023-10-06T05:07:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.334+0000 7fa05ffff700 1 -- 172.21.15.55:0/1229643345 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06800cb30 con 0x7fa078159420 2023-10-06T05:07:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.334+0000 7fa05ffff700 1 -- 172.21.15.55:0/1229643345 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa068013470 con 0x7fa078159420 2023-10-06T05:07:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.334+0000 7fa07d19d700 1 -- 172.21.15.55:0/1229643345 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07814e850 con 0x7fa078159420 2023-10-06T05:07:31.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.334+0000 7fa07d19d700 1 -- 172.21.15.55:0/1229643345 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07816a190 con 0x7fa078159420 2023-10-06T05:07:31.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.335+0000 7fa07d19d700 1 -- 172.21.15.55:0/1229643345 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07816a3e0 con 0x7fa078159420 2023-10-06T05:07:31.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.336+0000 7fa05ffff700 1 -- 172.21.15.55:0/1229643345 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa068013820 con 0x7fa078159420 2023-10-06T05:07:31.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.338+0000 7fa05ffff700 1 --2- 172.21.15.55:0/1229643345 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa064066f30 0x7fa0640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:31.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.338+0000 7fa05ffff700 1 -- 172.21.15.55:0/1229643345 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7fa068084c80 con 0x7fa078159420 2023-10-06T05:07:31.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.338+0000 7fa0777fe700 1 --2- 172.21.15.55:0/1229643345 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa064066f30 0x7fa0640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:31.340 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.339+0000 7fa0777fe700 1 --2- 172.21.15.55:0/1229643345 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa064066f30 0x7fa0640693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0093f0 tx=0x7fa06c016000 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:31.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.342+0000 7fa05ffff700 1 -- 172.21.15.55:0/1229643345 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa068051b30 con 0x7fa078159420 2023-10-06T05:07:31.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.571+0000 7fa07d19d700 1 -- 172.21.15.55:0/1229643345 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa07814ed00 con 0x7fa064066f30 2023-10-06T05:07:31.575 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.573+0000 7fa05ffff700 1 -- 172.21.15.55:0/1229643345 <== mgr.9544 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa06c0116c0 con 0x7fa064066f30 2023-10-06T05:07:31.575 INFO:teuthology.orchestra.run.smithi055.stdout:[ 2023-10-06T05:07:31.575 INFO:teuthology.orchestra.run.smithi055.stdout: { 2023-10-06T05:07:31.575 INFO:teuthology.orchestra.run.smithi055.stdout: "name": "cephfs" 2023-10-06T05:07:31.576 INFO:teuthology.orchestra.run.smithi055.stdout: } 2023-10-06T05:07:31.576 INFO:teuthology.orchestra.run.smithi055.stdout:] 2023-10-06T05:07:31.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.576+0000 7fa07d19d700 1 -- 172.21.15.55:0/1229643345 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa064066f30 msgr2=0x7fa0640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:31.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.576+0000 7fa07d19d700 1 --2- 172.21.15.55:0/1229643345 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa064066f30 0x7fa0640693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa06c0093f0 tx=0x7fa06c016000 comp rx=0 tx=0).stop 2023-10-06T05:07:31.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.576+0000 7fa07d19d700 1 -- 172.21.15.55:0/1229643345 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa078159420 msgr2=0x7fa07814f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:31.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.576+0000 7fa07d19d700 1 --2- 172.21.15.55:0/1229643345 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa078159420 0x7fa07814f050 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fa06800ec80 tx=0x7fa0680091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.576+0000 7fa07d19d700 1 -- 172.21.15.55:0/1229643345 shutdown_connections 2023-10-06T05:07:31.578 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.576+0000 7fa07d19d700 1 --2- 172.21.15.55:0/1229643345 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa064066f30 0x7fa0640693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.576+0000 7fa07d19d700 1 --2- 172.21.15.55:0/1229643345 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa07815d230 0x7fa07814c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.576+0000 7fa07d19d700 1 --2- 172.21.15.55:0/1229643345 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa078159420 0x7fa07814f050 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.576+0000 7fa07d19d700 1 --2- 172.21.15.55:0/1229643345 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0781597f0 0x7fa078169c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.576+0000 7fa07d19d700 1 -- 172.21.15.55:0/1229643345 >> 172.21.15.55:0/1229643345 conn(0x7fa0780d0660 msgr2=0x7fa078155060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:31.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.576+0000 7fa07d19d700 1 -- 172.21.15.55:0/1229643345 shutdown_connections 2023-10-06T05:07:31.579 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.576+0000 7fa07d19d700 1 -- 172.21.15.55:0/1229643345 wait complete. 2023-10-06T05:07:31.595 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-06T05:07:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.783+0000 7ff42e79d700 1 Processor -- start 2023-10-06T05:07:31.785 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.783+0000 7ff42e79d700 1 -- start start 2023-10-06T05:07:31.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.784+0000 7ff42e79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42806f060 0x7ff4280ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:31.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.784+0000 7ff42e79d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff42806f4b0 0x7ff4280ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:31.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.785+0000 7ff42e79d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42806ebd0 con 0x7ff4280cb1a0 2023-10-06T05:07:31.786 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.785+0000 7ff42e79d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff42815b2e0 con 0x7ff42806f4b0 2023-10-06T05:07:31.787 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.785+0000 7ff42e79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42815aa70 con 0x7ff42806f060 2023-10-06T05:07:31.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.785+0000 7ff42df9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff42806f4b0 0x7ff4280ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:31.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.785+0000 7ff42cf9a700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42806f060 0x7ff4280ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:31.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.785+0000 7ff42df9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7ff42806f4b0 0x7ff4280ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:42936/0 (socket says 172.21.15.55:42936) 2023-10-06T05:07:31.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.785+0000 7ff42df9c700 1 -- 172.21.15.55:0/1886823291 learned_addr learned my addr 172.21.15.55:0/1886823291 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:31.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.785+0000 7ff42df9c700 1 -- 172.21.15.55:0/1886823291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42806f060 msgr2=0x7ff4280ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:31.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.785+0000 7ff42df9c700 1 --2- 172.21.15.55:0/1886823291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42806f060 0x7ff4280ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.785+0000 7ff42df9c700 1 -- 172.21.15.55:0/1886823291 >> v1:172.21.15.163:6789/0 conn(0x7ff4280cb1a0 legacy=0x7ff42815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:31.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.785+0000 7ff42df9c700 1 -- 172.21.15.55:0/1886823291 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42815ad20 con 0x7ff42806f4b0 2023-10-06T05:07:31.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.786+0000 7ff42df9c700 1 --2- 172.21.15.55:0/1886823291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42806f4b0 0x7ff4280ae610 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff420009f20 tx=0x7ff420009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dda5f9eb00e0f2c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:31.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.786+0000 7ff41ffff700 1 -- 172.21.15.55:0/1886823291 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42000e070 con 0x7ff42806f4b0 2023-10-06T05:07:31.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.786+0000 7ff41ffff700 1 -- 172.21.15.55:0/1886823291 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42000ba60 con 0x7ff42806f4b0 2023-10-06T05:07:31.789 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.786+0000 7ff41ffff700 1 -- 172.21.15.55:0/1886823291 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4200092e0 con 0x7ff42806f4b0 2023-10-06T05:07:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.787+0000 7ff42e79d700 1 -- 172.21.15.55:0/1886823291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42806f4b0 msgr2=0x7ff4280ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.787+0000 7ff42e79d700 1 --2- 172.21.15.55:0/1886823291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42806f4b0 0x7ff4280ae610 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff420009f20 tx=0x7ff420009840 comp rx=0 tx=0).stop 2023-10-06T05:07:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.787+0000 7ff42e79d700 1 -- 172.21.15.55:0/1886823291 shutdown_connections 2023-10-06T05:07:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.787+0000 7ff42e79d700 1 --2- 172.21.15.55:0/1886823291 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42806f4b0 0x7ff4280ae610 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.787+0000 7ff42e79d700 1 --2- 172.21.15.55:0/1886823291 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42806f060 0x7ff4280ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.787+0000 7ff42e79d700 1 -- 172.21.15.55:0/1886823291 >> 172.21.15.55:0/1886823291 conn(0x7ff4280cc1e0 msgr2=0x7ff4280cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.787+0000 7ff42e79d700 1 -- 172.21.15.55:0/1886823291 shutdown_connections 2023-10-06T05:07:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.787+0000 7ff42e79d700 1 -- 172.21.15.55:0/1886823291 wait complete. 2023-10-06T05:07:31.790 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.788+0000 7ff42e79d700 1 Processor -- start 2023-10-06T05:07:31.791 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.788+0000 7ff42e79d700 1 -- start start 2023-10-06T05:07:31.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.789+0000 7ff42e79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42806f060 0x7ff428165cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:31.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.789+0000 7ff42e79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42806f4b0 0x7ff4280c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.789+0000 7ff42e79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4280cb1a0 0x7ff4280c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.789+0000 7ff42e79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42815a470 con 0x7ff4280cb1a0 2023-10-06T05:07:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.789+0000 7ff42e79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff428159b40 con 0x7ff42806f060 2023-10-06T05:07:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.789+0000 7ff42e79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff428159e20 con 0x7ff42806f4b0 2023-10-06T05:07:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.790+0000 7ff42df9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4280cb1a0 0x7ff4280c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.790+0000 7ff42df9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4280cb1a0 0x7ff4280c6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:43972/0 (socket says 172.21.15.55:43972) 2023-10-06T05:07:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.790+0000 7ff42df9c700 1 -- 172.21.15.55:0/306388864 learned_addr learned my addr 172.21.15.55:0/306388864 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.790+0000 7ff42cf9a700 1 --2- 172.21.15.55:0/306388864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42806f4b0 0x7ff4280c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:31.793 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.790+0000 7ff42d79b700 1 --2- 172.21.15.55:0/306388864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42806f060 0x7ff428165cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:31.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.790+0000 7ff42df9c700 1 -- 172.21.15.55:0/306388864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42806f4b0 msgr2=0x7ff4280c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:31.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.790+0000 7ff42df9c700 1 --2- 172.21.15.55:0/306388864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42806f4b0 0x7ff4280c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.790+0000 7ff42df9c700 1 -- 172.21.15.55:0/306388864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42806f060 msgr2=0x7ff428165cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:31.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.790+0000 7ff42df9c700 1 --2- 172.21.15.55:0/306388864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42806f060 0x7ff428165cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:31.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.790+0000 7ff42df9c700 1 -- 172.21.15.55:0/306388864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff420009ae0 con 0x7ff4280cb1a0 2023-10-06T05:07:31.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.790+0000 7ff42df9c700 1 --2- 172.21.15.55:0/306388864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4280cb1a0 0x7ff4280c6750 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7ff42806ce80 tx=0x7ff420018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:31.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.791+0000 7ff41d7fa700 1 -- 172.21.15.55:0/306388864 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4200092e0 con 0x7ff4280cb1a0 2023-10-06T05:07:31.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.791+0000 7ff41d7fa700 1 -- 172.21.15.55:0/306388864 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff420008d20 con 0x7ff4280cb1a0 2023-10-06T05:07:31.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.791+0000 7ff41d7fa700 1 -- 172.21.15.55:0/306388864 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff420026690 con 0x7ff4280cb1a0 2023-10-06T05:07:31.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.791+0000 7ff42e79d700 1 -- 172.21.15.55:0/306388864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42815a100 con 0x7ff4280cb1a0 2023-10-06T05:07:31.794 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.791+0000 7ff42e79d700 1 -- 172.21.15.55:0/306388864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff428158e10 con 0x7ff4280cb1a0 2023-10-06T05:07:31.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.792+0000 7ff42e79d700 1 -- 172.21.15.55:0/306388864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4281664a0 con 0x7ff4280cb1a0 2023-10-06T05:07:31.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.793+0000 7ff41d7fa700 1 -- 172.21.15.55:0/306388864 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 100) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff420003040 con 0x7ff4280cb1a0 2023-10-06T05:07:31.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.793+0000 7ff41d7fa700 1 --2- 172.21.15.55:0/306388864 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff40c066f30 0x7ff40c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:31.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.794+0000 7ff41d7fa700 1 -- 172.21.15.55:0/306388864 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7ff420086420 con 0x7ff4280cb1a0 2023-10-06T05:07:31.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.794+0000 7ff42d79b700 1 --2- 172.21.15.55:0/306388864 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff40c066f30 0x7ff40c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:31.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.795+0000 7ff42d79b700 1 --2- 172.21.15.55:0/306388864 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff40c066f30 0x7ff40c0693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff4280c1630 tx=0x7ff424009040 comp rx=0 tx=0).ready entity=mgr.9544 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:31.799 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:31.798+0000 7ff41d7fa700 1 -- 172.21.15.55:0/306388864 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff4281664a0 con 0x7ff4280cb1a0 2023-10-06T05:07:32.058 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.056+0000 7ff42e79d700 1 -- 172.21.15.55:0/306388864 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7ff4281664a0 con 0x7ff4280cb1a0 2023-10-06T05:07:32.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.985+0000 7ff41d7fa700 1 -- 172.21.15.55:0/306388864 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 101) v1 ==== 84757+0+0 (secure 0 0 0) 0x7ff4200537f0 con 0x7ff4280cb1a0 2023-10-06T05:07:32.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.987+0000 7ff41d7fa700 1 -- 172.21.15.55:0/306388864 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v101) v1 ==== 92+0+0 (secure 0 0 0) 0x7ff420056eb0 con 0x7ff4280cb1a0 2023-10-06T05:07:32.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.990+0000 7ff42e79d700 1 -- 172.21.15.55:0/306388864 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff40c066f30 msgr2=0x7ff40c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:32.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.990+0000 7ff42e79d700 1 --2- 172.21.15.55:0/306388864 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff40c066f30 0x7ff40c0693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff4280c1630 tx=0x7ff424009040 comp rx=0 tx=0).stop 2023-10-06T05:07:32.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.990+0000 7ff42e79d700 1 -- 172.21.15.55:0/306388864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4280cb1a0 msgr2=0x7ff4280c6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:32.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.990+0000 7ff42e79d700 1 --2- 172.21.15.55:0/306388864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4280cb1a0 0x7ff4280c6750 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7ff42806ce80 tx=0x7ff420018000 comp rx=0 tx=0).stop 2023-10-06T05:07:32.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.990+0000 7ff42e79d700 1 -- 172.21.15.55:0/306388864 shutdown_connections 2023-10-06T05:07:32.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.990+0000 7ff42e79d700 1 --2- 172.21.15.55:0/306388864 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff40c066f30 0x7ff40c0693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:32.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.990+0000 7ff42e79d700 1 --2- 172.21.15.55:0/306388864 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff4280cb1a0 0x7ff4280c6750 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:32.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.990+0000 7ff42e79d700 1 --2- 172.21.15.55:0/306388864 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff42806f060 0x7ff428165cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:32.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.990+0000 7ff42e79d700 1 --2- 172.21.15.55:0/306388864 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff42806f4b0 0x7ff4280c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:32.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.990+0000 7ff42e79d700 1 -- 172.21.15.55:0/306388864 >> 172.21.15.55:0/306388864 conn(0x7ff4280cc1e0 msgr2=0x7ff4280c28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:32.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.990+0000 7ff42e79d700 1 -- 172.21.15.55:0/306388864 shutdown_connections 2023-10-06T05:07:32.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:32.990+0000 7ff42e79d700 1 -- 172.21.15.55:0/306388864 wait complete. 2023-10-06T05:07:33.014 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-06T05:07:33.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.210+0000 7f0e42b9d700 1 Processor -- start 2023-10-06T05:07:33.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.210+0000 7f0e42b9d700 1 -- start start 2023-10-06T05:07:33.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.212+0000 7f0e42b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e3c0aef00 0x7f0e3c15d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:33.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.212+0000 7f0e42b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0e3c06e470 0x7f0e3c06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:33.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.212+0000 7f0e42b9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3c0aea70 con 0x7f0e3c15a3b0 2023-10-06T05:07:33.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.212+0000 7f0e42b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e3c06ee00 con 0x7f0e3c06e470 2023-10-06T05:07:33.214 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.212+0000 7f0e42b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c0ae480 con 0x7f0e3c0aef00 2023-10-06T05:07:33.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.212+0000 7f0e41b9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f0e3c15a3b0 0x7f0e3c15a780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34486/0 (socket says 172.21.15.55:34486) 2023-10-06T05:07:33.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.212+0000 7f0e41b9b700 1 -- 172.21.15.55:0/1751269778 learned_addr learned my addr 172.21.15.55:0/1751269778 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:33.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.212+0000 7f0e4239c700 1 --2- 172.21.15.55:0/1751269778 >> v2:172.21.15.55:3300/0 conn(0x7f0e3c06e470 0x7f0e3c06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:33.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.212+0000 7f0e4139a700 1 --2- 172.21.15.55:0/1751269778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e3c0aef00 0x7f0e3c15d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:33.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.213+0000 7f0e40b99700 1 -- 172.21.15.55:0/1751269778 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874234488 0 0) 0x7f0e3c0aea70 con 0x7f0e3c15a3b0 2023-10-06T05:07:33.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.213+0000 7f0e40b99700 1 -- 172.21.15.55:0/1751269778 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e28003570 con 0x7f0e3c15a3b0 2023-10-06T05:07:33.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.213+0000 7f0e4239c700 1 -- 172.21.15.55:0/1751269778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e3c0aef00 msgr2=0x7f0e3c15d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:33.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.213+0000 7f0e4239c700 1 --2- 172.21.15.55:0/1751269778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e3c0aef00 0x7f0e3c15d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.213+0000 7f0e4239c700 1 -- 172.21.15.55:0/1751269778 >> v1:172.21.15.55:6789/0 conn(0x7f0e3c15a3b0 legacy=0x7f0e3c15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:33.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.213+0000 7f0e4239c700 1 -- 172.21.15.55:0/1751269778 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c15dd30 con 0x7f0e3c06e470 2023-10-06T05:07:33.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.213+0000 7f0e4239c700 1 --2- 172.21.15.55:0/1751269778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e3c06e470 0x7f0e3c06e8c0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009e60 tx=0x7f0e34009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a16a21edf86db8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:33.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.213+0000 7f0e40b99700 1 -- 172.21.15.55:0/1751269778 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400e070 con 0x7f0e3c06e470 2023-10-06T05:07:33.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.214+0000 7f0e40b99700 1 -- 172.21.15.55:0/1751269778 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3400bab0 con 0x7f0e3c06e470 2023-10-06T05:07:33.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.214+0000 7f0e40b99700 1 -- 172.21.15.55:0/1751269778 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400be70 con 0x7f0e3c06e470 2023-10-06T05:07:33.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.214+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1751269778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e3c06e470 msgr2=0x7f0e3c06e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:33.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.214+0000 7f0e42b9d700 1 --2- 172.21.15.55:0/1751269778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e3c06e470 0x7f0e3c06e8c0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009e60 tx=0x7f0e34009780 comp rx=0 tx=0).stop 2023-10-06T05:07:33.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.214+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1751269778 shutdown_connections 2023-10-06T05:07:33.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.214+0000 7f0e42b9d700 1 --2- 172.21.15.55:0/1751269778 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e3c06e470 0x7f0e3c06e8c0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.214+0000 7f0e42b9d700 1 --2- 172.21.15.55:0/1751269778 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e3c0aef00 0x7f0e3c15d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.214+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1751269778 >> 172.21.15.55:0/1751269778 conn(0x7f0e3c0c9f70 msgr2=0x7f0e3c06dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:33.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.214+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1751269778 shutdown_connections 2023-10-06T05:07:33.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.214+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1751269778 wait complete. 2023-10-06T05:07:33.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.216+0000 7f0e42b9d700 1 Processor -- start 2023-10-06T05:07:33.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.216+0000 7f0e42b9d700 1 -- start start 2023-10-06T05:07:33.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.217+0000 7f0e42b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e3c06e470 0x7f0e3c169f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:33.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.217+0000 7f0e42b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e3c0aef00 0x7f0e3c168e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:33.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.217+0000 7f0e42b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e3c15a3b0 0x7f0e3c0c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:33.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.217+0000 7f0e42b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c0c6210 con 0x7f0e3c0aef00 2023-10-06T05:07:33.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.217+0000 7f0e42b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c16a4a0 con 0x7f0e3c15a3b0 2023-10-06T05:07:33.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.217+0000 7f0e42b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c169340 con 0x7f0e3c06e470 2023-10-06T05:07:33.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.217+0000 7f0e4139a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e3c0aef00 0x7f0e3c168e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:33.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.217+0000 7f0e4139a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e3c0aef00 0x7f0e3c168e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50824/0 (socket says 172.21.15.55:50824) 2023-10-06T05:07:33.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.217+0000 7f0e4139a700 1 -- 172.21.15.55:0/1123023166 learned_addr learned my addr 172.21.15.55:0/1123023166 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:33.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.217+0000 7f0e41b9b700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e3c06e470 0x7f0e3c169f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:33.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.217+0000 7f0e4239c700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e3c15a3b0 0x7f0e3c0c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:33.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.218+0000 7f0e4139a700 1 -- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e3c06e470 msgr2=0x7f0e3c169f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:33.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.218+0000 7f0e4139a700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e3c06e470 0x7f0e3c169f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.218+0000 7f0e4139a700 1 -- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e3c15a3b0 msgr2=0x7f0e3c0c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:33.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.218+0000 7f0e4139a700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e3c15a3b0 0x7f0e3c0c4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.218+0000 7f0e4139a700 1 -- 172.21.15.55:0/1123023166 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e34009a20 con 0x7f0e3c0aef00 2023-10-06T05:07:33.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.218+0000 7f0e4239c700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e3c15a3b0 0x7f0e3c0c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:33.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.218+0000 7f0e4139a700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e3c0aef00 0x7f0e3c168e00 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00ecb0 tx=0x7f0e2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:33.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.218+0000 7f0e31ffb700 1 -- 172.21.15.55:0/1123023166 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c00c920 con 0x7f0e3c0aef00 2023-10-06T05:07:33.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.218+0000 7f0e31ffb700 1 -- 172.21.15.55:0/1123023166 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c00ca80 con 0x7f0e3c0aef00 2023-10-06T05:07:33.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.218+0000 7f0e31ffb700 1 -- 172.21.15.55:0/1123023166 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c00ce20 con 0x7f0e3c0aef00 2023-10-06T05:07:33.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.218+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1123023166 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3c0c5340 con 0x7f0e3c0aef00 2023-10-06T05:07:33.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.218+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1123023166 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c0c5960 con 0x7f0e3c0aef00 2023-10-06T05:07:33.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.220+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1123023166 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3c16a650 con 0x7f0e3c0aef00 2023-10-06T05:07:33.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.220+0000 7f0e31ffb700 1 -- 172.21.15.55:0/1123023166 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 101) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f0e2c013680 con 0x7f0e3c0aef00 2023-10-06T05:07:33.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.221+0000 7f0e31ffb700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0e20066f80 0x7f0e20069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:33.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.221+0000 7f0e31ffb700 1 -- 172.21.15.55:0/1123023166 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f0e2c084c00 con 0x7f0e3c0aef00 2023-10-06T05:07:33.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.221+0000 7f0e41b9b700 1 -- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0e20066f80 msgr2=0x7f0e20069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:33.224 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.221+0000 7f0e41b9b700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0e20066f80 0x7f0e20069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:33.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.226+0000 7f0e31ffb700 1 -- 172.21.15.55:0/1123023166 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e3c16a650 con 0x7f0e3c0aef00 2023-10-06T05:07:33.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.423+0000 7f0e41b9b700 1 -- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0e20066f80 msgr2=0x7f0e20069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:33.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.423+0000 7f0e41b9b700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0e20066f80 0x7f0e20069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:33.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.431+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1123023166 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f0e3c16a650 con 0x7f0e3c0aef00 2023-10-06T05:07:33.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.437+0000 7f0e31ffb700 1 -- 172.21.15.55:0/1123023166 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v71)=0 v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0e3c16a650 con 0x7f0e3c0aef00 2023-10-06T05:07:33.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.439+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0e20066f80 msgr2=0x7f0e20069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:33.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.439+0000 7f0e42b9d700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0e20066f80 0x7f0e20069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.439+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1123023166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e3c0aef00 msgr2=0x7f0e3c168e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:33.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.439+0000 7f0e42b9d700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e3c0aef00 0x7f0e3c168e00 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c00ecb0 tx=0x7f0e2c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.439+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1123023166 shutdown_connections 2023-10-06T05:07:33.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.439+0000 7f0e42b9d700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0e20066f80 0x7f0e20069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.439+0000 7f0e42b9d700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0e3c15a3b0 0x7f0e3c0c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.439+0000 7f0e42b9d700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f0e3c06e470 0x7f0e3c169f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.439+0000 7f0e42b9d700 1 --2- 172.21.15.55:0/1123023166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0e3c0aef00 0x7f0e3c168e00 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.439+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1123023166 >> 172.21.15.55:0/1123023166 conn(0x7f0e3c0c9f70 msgr2=0x7f0e3c0c8690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:33.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.439+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1123023166 shutdown_connections 2023-10-06T05:07:33.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.439+0000 7f0e42b9d700 1 -- 172.21.15.55:0/1123023166 wait complete. 2023-10-06T05:07:33.460 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-06T05:07:33.657 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.655+0000 7f85eb59e700 1 Processor -- start 2023-10-06T05:07:33.658 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.656+0000 7f85eb59e700 1 -- start start 2023-10-06T05:07:33.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.657+0000 7f85eb59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85ec0b4e70 0x7f85ec0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:33.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.657+0000 7f85eb59e700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f85ec0b16d0 0x7f85ec0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:33.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.657+0000 7f85eb59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85ec0b2080 con 0x7f85ec15d240 2023-10-06T05:07:33.659 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.657+0000 7f85eb59e700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85ec0cfb00 con 0x7f85ec0b16d0 2023-10-06T05:07:33.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.657+0000 7f85eb59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec0b5c40 con 0x7f85ec0b4e70 2023-10-06T05:07:33.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.657+0000 7f85ead9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f85ec15d240 0x7f85ec15d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34488/0 (socket says 172.21.15.55:34488) 2023-10-06T05:07:33.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.657+0000 7f85ead9d700 1 -- 172.21.15.55:0/4130156311 learned_addr learned my addr 172.21.15.55:0/4130156311 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:33.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.657+0000 7f85e9d9b700 1 --2- 172.21.15.55:0/4130156311 >> v2:172.21.15.163:3300/0 conn(0x7f85ec0b16d0 0x7f85ec0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:33.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.657+0000 7f85ea59c700 1 --2- 172.21.15.55:0/4130156311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85ec0b4e70 0x7f85ec0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:33.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.658+0000 7f85e959a700 1 -- 172.21.15.55:0/4130156311 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289239695 0 0) 0x7f85ec0b2080 con 0x7f85ec15d240 2023-10-06T05:07:33.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.658+0000 7f85e959a700 1 -- 172.21.15.55:0/4130156311 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85d4003570 con 0x7f85ec15d240 2023-10-06T05:07:33.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.658+0000 7f85ea59c700 1 -- 172.21.15.55:0/4130156311 >> v2:172.21.15.163:3300/0 conn(0x7f85ec0b16d0 msgr2=0x7f85ec0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:33.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.658+0000 7f85ea59c700 1 --2- 172.21.15.55:0/4130156311 >> v2:172.21.15.163:3300/0 conn(0x7f85ec0b16d0 0x7f85ec0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.658+0000 7f85ea59c700 1 -- 172.21.15.55:0/4130156311 >> v1:172.21.15.55:6789/0 conn(0x7f85ec15d240 legacy=0x7f85ec15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:33.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.658+0000 7f85ea59c700 1 -- 172.21.15.55:0/4130156311 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ec0b5f20 con 0x7f85ec0b4e70 2023-10-06T05:07:33.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.658+0000 7f85e9d9b700 1 --2- 172.21.15.55:0/4130156311 >> v2:172.21.15.163:3300/0 conn(0x7f85ec0b16d0 0x7f85ec0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:33.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.658+0000 7f85ea59c700 1 --2- 172.21.15.55:0/4130156311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85ec0b4e70 0x7f85ec0ccff0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0098e0 tx=0x7f85dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=875f27d452e555ae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:33.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.658+0000 7f85e959a700 1 -- 172.21.15.55:0/4130156311 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc011630 con 0x7f85ec0b4e70 2023-10-06T05:07:33.662 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.659+0000 7f85e959a700 1 -- 172.21.15.55:0/4130156311 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc011790 con 0x7f85ec0b4e70 2023-10-06T05:07:33.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.659+0000 7f85e959a700 1 -- 172.21.15.55:0/4130156311 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc011b30 con 0x7f85ec0b4e70 2023-10-06T05:07:33.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.659+0000 7f85eb59e700 1 -- 172.21.15.55:0/4130156311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85ec0b4e70 msgr2=0x7f85ec0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:33.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.659+0000 7f85eb59e700 1 --2- 172.21.15.55:0/4130156311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85ec0b4e70 0x7f85ec0ccff0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0098e0 tx=0x7f85dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.659+0000 7f85eb59e700 1 -- 172.21.15.55:0/4130156311 shutdown_connections 2023-10-06T05:07:33.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.659+0000 7f85eb59e700 1 --2- 172.21.15.55:0/4130156311 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85ec0b4e70 0x7f85ec0ccff0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.659+0000 7f85eb59e700 1 --2- 172.21.15.55:0/4130156311 >> v2:172.21.15.163:3300/0 conn(0x7f85ec0b16d0 0x7f85ec0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.659+0000 7f85eb59e700 1 -- 172.21.15.55:0/4130156311 >> 172.21.15.55:0/4130156311 conn(0x7f85ec0d10c0 msgr2=0x7f85ec0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:33.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.659+0000 7f85eb59e700 1 -- 172.21.15.55:0/4130156311 shutdown_connections 2023-10-06T05:07:33.663 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.659+0000 7f85eb59e700 1 -- 172.21.15.55:0/4130156311 wait complete. 2023-10-06T05:07:33.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.661+0000 7f85eb59e700 1 Processor -- start 2023-10-06T05:07:33.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.661+0000 7f85eb59e700 1 -- start start 2023-10-06T05:07:33.665 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85eb59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85ec0b16d0 0x7f85ec1527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:33.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85eb59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85ec0b4e70 0x7f85ec15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:33.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85eb59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85ec15d240 0x7f85ec15a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:33.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85eb59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec152dd0 con 0x7f85ec0b16d0 2023-10-06T05:07:33.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85eb59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1530b0 con 0x7f85ec0b4e70 2023-10-06T05:07:33.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85eb59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec153390 con 0x7f85ec15d240 2023-10-06T05:07:33.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85ea59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85ec0b16d0 0x7f85ec1527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:33.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85ea59c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85ec0b16d0 0x7f85ec1527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50840/0 (socket says 172.21.15.55:50840) 2023-10-06T05:07:33.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85ea59c700 1 -- 172.21.15.55:0/1621189145 learned_addr learned my addr 172.21.15.55:0/1621189145 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:33.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85e9d9b700 1 --2- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85ec0b4e70 0x7f85ec15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:33.666 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85ead9d700 1 --2- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85ec15d240 0x7f85ec15a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:33.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85ea59c700 1 -- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85ec15d240 msgr2=0x7f85ec15a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:33.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85ea59c700 1 --2- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85ec15d240 0x7f85ec15a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85ea59c700 1 -- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85ec0b4e70 msgr2=0x7f85ec15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:33.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85ea59c700 1 --2- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85ec0b4e70 0x7f85ec15b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.662+0000 7f85ea59c700 1 -- 172.21.15.55:0/1621189145 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e0009420 con 0x7f85ec0b16d0 2023-10-06T05:07:33.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.663+0000 7f85ea59c700 1 --2- 172.21.15.55:0/1621189145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85ec0b16d0 0x7f85ec1527e0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0b5d80 tx=0x7f85dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:33.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.663+0000 7f85da7fc700 1 -- 172.21.15.55:0/1621189145 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc01f070 con 0x7f85ec0b16d0 2023-10-06T05:07:33.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.663+0000 7f85da7fc700 1 -- 172.21.15.55:0/1621189145 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc000bc0 con 0x7f85ec0b16d0 2023-10-06T05:07:33.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.663+0000 7f85da7fc700 1 -- 172.21.15.55:0/1621189145 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc011b30 con 0x7f85ec0b16d0 2023-10-06T05:07:33.667 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.663+0000 7f85eb59e700 1 -- 172.21.15.55:0/1621189145 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85dc009420 con 0x7f85ec0b16d0 2023-10-06T05:07:33.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.663+0000 7f85eb59e700 1 -- 172.21.15.55:0/1621189145 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ec153950 con 0x7f85ec0b16d0 2023-10-06T05:07:33.668 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.664+0000 7f85eb59e700 1 -- 172.21.15.55:0/1621189145 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ec088990 con 0x7f85ec0b16d0 2023-10-06T05:07:33.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.665+0000 7f85da7fc700 1 -- 172.21.15.55:0/1621189145 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 101) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f85dc023070 con 0x7f85ec0b16d0 2023-10-06T05:07:33.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.666+0000 7f85da7fc700 1 --2- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f85cc066f80 0x7f85cc069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:33.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.666+0000 7f85da7fc700 1 -- 172.21.15.55:0/1621189145 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f85dc085be0 con 0x7f85ec0b16d0 2023-10-06T05:07:33.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.667+0000 7f85e9d9b700 1 -- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f85cc066f80 msgr2=0x7f85cc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:33.669 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.667+0000 7f85e9d9b700 1 --2- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f85cc066f80 0x7f85cc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:33.672 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.670+0000 7f85da7fc700 1 -- 172.21.15.55:0/1621189145 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85dc052a90 con 0x7f85ec0b16d0 2023-10-06T05:07:33.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.868+0000 7f85e9d9b700 1 -- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f85cc066f80 msgr2=0x7f85cc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:33.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.868+0000 7f85e9d9b700 1 --2- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f85cc066f80 0x7f85cc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:33.880 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.878+0000 7f85eb59e700 1 -- 172.21.15.55:0/1621189145 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f85ec04bb80 con 0x7f85ec0b16d0 2023-10-06T05:07:33.884 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.882+0000 7f85da7fc700 1 -- 172.21.15.55:0/1621189145 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v72)=0 v72) v1 ==== 133+0+0 (secure 0 0 0) 0x7f85dc02a020 con 0x7f85ec0b16d0 2023-10-06T05:07:33.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.885+0000 7f85eb59e700 1 -- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f85cc066f80 msgr2=0x7f85cc069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:33.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.885+0000 7f85eb59e700 1 --2- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f85cc066f80 0x7f85cc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.885+0000 7f85eb59e700 1 -- 172.21.15.55:0/1621189145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85ec0b16d0 msgr2=0x7f85ec1527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:33.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.885+0000 7f85eb59e700 1 --2- 172.21.15.55:0/1621189145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85ec0b16d0 0x7f85ec1527e0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f85ec0b5d80 tx=0x7f85dc017000 comp rx=0 tx=0).stop 2023-10-06T05:07:33.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.885+0000 7f85eb59e700 1 -- 172.21.15.55:0/1621189145 shutdown_connections 2023-10-06T05:07:33.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.885+0000 7f85eb59e700 1 --2- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f85cc066f80 0x7f85cc069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.885+0000 7f85eb59e700 1 --2- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f85ec15d240 0x7f85ec15a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.885+0000 7f85eb59e700 1 --2- 172.21.15.55:0/1621189145 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f85ec0b16d0 0x7f85ec1527e0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.885+0000 7f85eb59e700 1 --2- 172.21.15.55:0/1621189145 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f85ec0b4e70 0x7f85ec15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:33.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.885+0000 7f85eb59e700 1 -- 172.21.15.55:0/1621189145 >> 172.21.15.55:0/1621189145 conn(0x7f85ec0d10c0 msgr2=0x7f85ec0cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:33.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.885+0000 7f85eb59e700 1 -- 172.21.15.55:0/1621189145 shutdown_connections 2023-10-06T05:07:33.888 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:33.885+0000 7f85eb59e700 1 -- 172.21.15.55:0/1621189145 wait complete. 2023-10-06T05:07:33.905 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:07:33.906 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-06T05:07:33.952 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-10-06T05:07:34.186 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.184+0000 7f44bad9d700 1 Processor -- start 2023-10-06T05:07:34.187 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.184+0000 7f44bad9d700 1 -- start start 2023-10-06T05:07:34.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.185+0000 7f44bad9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40cca20 0x7f44b40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:34.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.186+0000 7f44bad9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b40cfa00 con 0x7f44b40b03c0 2023-10-06T05:07:34.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.186+0000 7f44bad9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b40cb580 con 0x7f44b40b0790 2023-10-06T05:07:34.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.186+0000 7f44bad9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40cf160 con 0x7f44b40cca20 2023-10-06T05:07:34.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.186+0000 7f44b9d9b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f44b40b03c0 0x7f44b40b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34496/0 (socket says 172.21.15.55:34496) 2023-10-06T05:07:34.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.186+0000 7f44b9d9b700 1 -- 172.21.15.55:0/3901137200 learned_addr learned my addr 172.21.15.55:0/3901137200 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:34.188 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.186+0000 7f44ba59c700 1 --2- 172.21.15.55:0/3901137200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40cca20 0x7f44b40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:34.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.186+0000 7f44b8d99700 1 -- 172.21.15.55:0/3901137200 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1676223600 0 0) 0x7f44b40cfa00 con 0x7f44b40b03c0 2023-10-06T05:07:34.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.186+0000 7f44b8d99700 1 -- 172.21.15.55:0/3901137200 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44a0003570 con 0x7f44b40b03c0 2023-10-06T05:07:34.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.186+0000 7f44b8d99700 1 -- 172.21.15.55:0/3901137200 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263479468 0 0) 0x7f44b40cb580 con 0x7f44b40b0790 2023-10-06T05:07:34.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.186+0000 7f44b8d99700 1 -- 172.21.15.55:0/3901137200 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44b40cfa00 con 0x7f44b40b0790 2023-10-06T05:07:34.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.187+0000 7f44ba59c700 1 -- 172.21.15.55:0/3901137200 >> v1:172.21.15.163:6789/0 conn(0x7f44b40b0790 legacy=0x7f44b41591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:34.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.187+0000 7f44ba59c700 1 -- 172.21.15.55:0/3901137200 >> v1:172.21.15.55:6789/0 conn(0x7f44b40b03c0 legacy=0x7f44b40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:34.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.187+0000 7f44ba59c700 1 -- 172.21.15.55:0/3901137200 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b40cf440 con 0x7f44b40cca20 2023-10-06T05:07:34.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.187+0000 7f44ba59c700 1 --2- 172.21.15.55:0/3901137200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40cca20 0x7f44b40cea70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009ec0 tx=0x7f44ac009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3320f5f8a59a8d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:34.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.187+0000 7f44b8d99700 1 -- 172.21.15.55:0/3901137200 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac00e070 con 0x7f44b40cca20 2023-10-06T05:07:34.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.187+0000 7f44b8d99700 1 -- 172.21.15.55:0/3901137200 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac00bb40 con 0x7f44b40cca20 2023-10-06T05:07:34.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.187+0000 7f44b8d99700 1 -- 172.21.15.55:0/3901137200 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac012430 con 0x7f44b40cca20 2023-10-06T05:07:34.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.188+0000 7f44bad9d700 1 -- 172.21.15.55:0/3901137200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40cca20 msgr2=0x7f44b40cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:34.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.188+0000 7f44bad9d700 1 --2- 172.21.15.55:0/3901137200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40cca20 0x7f44b40cea70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009ec0 tx=0x7f44ac009780 comp rx=0 tx=0).stop 2023-10-06T05:07:34.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.188+0000 7f44bad9d700 1 -- 172.21.15.55:0/3901137200 shutdown_connections 2023-10-06T05:07:34.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.188+0000 7f44bad9d700 1 --2- 172.21.15.55:0/3901137200 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40cca20 0x7f44b40cea70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:34.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.188+0000 7f44bad9d700 1 -- 172.21.15.55:0/3901137200 >> 172.21.15.55:0/3901137200 conn(0x7f44b4063230 msgr2=0x7f44b4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:34.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.188+0000 7f44bad9d700 1 -- 172.21.15.55:0/3901137200 shutdown_connections 2023-10-06T05:07:34.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.188+0000 7f44bad9d700 1 -- 172.21.15.55:0/3901137200 wait complete. 2023-10-06T05:07:34.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.189+0000 7f44bad9d700 1 Processor -- start 2023-10-06T05:07:34.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.189+0000 7f44bad9d700 1 -- start start 2023-10-06T05:07:34.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.190+0000 7f44bad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b40b03c0 0x7f44b40c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:34.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.190+0000 7f44bad9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40b0790 0x7f44b40c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:34.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.190+0000 7f44bad9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44b40cca20 0x7f44b40c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:34.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.190+0000 7f44bad9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40ca640 con 0x7f44b40b03c0 2023-10-06T05:07:34.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.190+0000 7f44bad9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b415bf60 con 0x7f44b40cca20 2023-10-06T05:07:34.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.190+0000 7f44bad9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b415b630 con 0x7f44b40b0790 2023-10-06T05:07:34.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.191+0000 7f44b9d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b40b03c0 0x7f44b40c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:34.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.191+0000 7f44b9d9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b40b03c0 0x7f44b40c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50850/0 (socket says 172.21.15.55:50850) 2023-10-06T05:07:34.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.191+0000 7f44b9d9b700 1 -- 172.21.15.55:0/3042207445 learned_addr learned my addr 172.21.15.55:0/3042207445 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:34.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.191+0000 7f44ba59c700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44b40cca20 0x7f44b40c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:34.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.191+0000 7f44b959a700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40b0790 0x7f44b40c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:34.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.191+0000 7f44b9d9b700 1 -- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40b0790 msgr2=0x7f44b40c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:34.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.191+0000 7f44b9d9b700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40b0790 0x7f44b40c41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:34.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.191+0000 7f44b9d9b700 1 -- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44b40cca20 msgr2=0x7f44b40c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:34.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.191+0000 7f44b9d9b700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44b40cca20 0x7f44b40c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:34.196 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.191+0000 7f44b9d9b700 1 -- 172.21.15.55:0/3042207445 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ac009a20 con 0x7f44b40b03c0 2023-10-06T05:07:34.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.191+0000 7f44b9d9b700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b40b03c0 0x7f44b40c75a0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f44b000c790 tx=0x7f44b0007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:34.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.192+0000 7f44a9ffb700 1 -- 172.21.15.55:0/3042207445 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0005b30 con 0x7f44b40b03c0 2023-10-06T05:07:34.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.192+0000 7f44a9ffb700 1 -- 172.21.15.55:0/3042207445 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b0005cd0 con 0x7f44b40b03c0 2023-10-06T05:07:34.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.192+0000 7f44a9ffb700 1 -- 172.21.15.55:0/3042207445 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0007320 con 0x7f44b40b03c0 2023-10-06T05:07:34.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.192+0000 7f44bad9d700 1 -- 172.21.15.55:0/3042207445 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b415b910 con 0x7f44b40b03c0 2023-10-06T05:07:34.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.192+0000 7f44bad9d700 1 -- 172.21.15.55:0/3042207445 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b415c3f0 con 0x7f44b40b03c0 2023-10-06T05:07:34.197 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.193+0000 7f44bad9d700 1 -- 172.21.15.55:0/3042207445 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b415c670 con 0x7f44b40b03c0 2023-10-06T05:07:34.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.194+0000 7f44a9ffb700 1 -- 172.21.15.55:0/3042207445 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 101) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f44b0012400 con 0x7f44b40b03c0 2023-10-06T05:07:34.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.195+0000 7f44a9ffb700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 0x7f4498069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:34.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.195+0000 7f44a9ffb700 1 -- 172.21.15.55:0/3042207445 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6506+0+0 (secure 0 0 0) 0x7f44b0083c50 con 0x7f44b40b03c0 2023-10-06T05:07:34.198 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.195+0000 7f44b959a700 1 -- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 msgr2=0x7f4498069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:34.199 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.195+0000 7f44b959a700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 0x7f4498069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:34.200 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.199+0000 7f44a9ffb700 1 -- 172.21.15.55:0/3042207445 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44b004b5a0 con 0x7f44b40b03c0 2023-10-06T05:07:34.289 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:34.288+0000 7f26ac736180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:07:34.370 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:34.368+0000 7f5eaa350180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:07:34.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.396+0000 7f44b959a700 1 -- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 msgr2=0x7f4498069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:34.398 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.396+0000 7f44b959a700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 0x7f4498069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:34.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.428+0000 7f44bad9d700 1 -- 172.21.15.55:0/3042207445 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f44b415c0a0 con 0x7f4498066f80 2023-10-06T05:07:34.452 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:34.451+0000 7f26ac736180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:07:34.540 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:34.539+0000 7f26ac736180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:07:34.541 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:34.539+0000 7f5eaa350180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:07:34.631 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:34.629+0000 7f5eaa350180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:07:34.767 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:34.766+0000 7f26ac736180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:07:34.799 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.797+0000 7f44b959a700 1 -- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 msgr2=0x7f4498069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:34.799 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:34.797+0000 7f44b959a700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 0x7f4498069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-06T05:07:34.863 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:34.861+0000 7f5eaa350180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:07:34.944 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:34.942+0000 7f26ac736180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:07:35.037 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:35.036+0000 7f26ac736180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:07:35.048 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:35.046+0000 7f5eaa350180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:07:35.145 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:35.143+0000 7f5eaa350180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:07:35.201 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:35.199+0000 7f26ac736180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:07:35.301 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:35.300+0000 7f26ac736180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:07:35.312 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:35.310+0000 7f5eaa350180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:07:35.417 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:35.415+0000 7f5eaa350180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:07:35.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:35.598+0000 7f44b959a700 1 -- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 msgr2=0x7f4498069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:35.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:35.598+0000 7f44b959a700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 0x7f4498069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-06T05:07:35.785 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:35.784+0000 7f26ac736180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:07:35.881 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:35.879+0000 7f26ac736180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:07:35.920 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:35.918+0000 7f5eaa350180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:07:35.971 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:35.970+0000 7f26ac736180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:07:36.018 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:36.017+0000 7f5eaa350180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:07:36.114 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:36.112+0000 7f5eaa350180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:07:36.598 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:36.596+0000 7f26ac736180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:07:36.765 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:36.763+0000 7f5eaa350180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:07:36.771 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:36.770+0000 7f26ac736180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:07:36.869 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:36.868+0000 7f26ac736180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:07:36.945 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:36.943+0000 7f5eaa350180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:07:37.046 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:37.044+0000 7f5eaa350180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:07:37.169 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:37.166+0000 7f26ac736180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:07:37.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:37.200+0000 7f44b959a700 1 -- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 msgr2=0x7f4498069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:37.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:37.200+0000 7f44b959a700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 0x7f4498069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-06T05:07:37.265 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:37.263+0000 7f26ac736180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:07:37.349 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:37.347+0000 7f5eaa350180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:07:37.449 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:37.447+0000 7f5eaa350180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:07:37.977 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:37.975+0000 7f26ac736180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:07:38.189 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:38.187+0000 7f5eaa350180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:07:38.266 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:38.265+0000 7f26ac736180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:07:38.360 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:38.358+0000 7f26ac736180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:07:38.483 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:38.481+0000 7f5eaa350180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:07:38.514 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:38.513+0000 7f26ac736180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:07:38.580 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:38.578+0000 7f5eaa350180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:07:38.619 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:38.618+0000 7f26ac736180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:07:38.744 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:38.742+0000 7f5eaa350180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:07:38.811 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:38.810+0000 7f26ac736180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:07:38.853 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:38.851+0000 7f5eaa350180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:07:38.905 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:38.904+0000 7f26ac736180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:07:38.990 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:07:38.997 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:07:39.004 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:38.999+0000 7f26ac736180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:07:39.065 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:39.063+0000 7f5eaa350180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:07:39.102 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:39.101+0000 7f26ac736180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:07:39.114 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:39.113+0000 7f44a9ffb700 1 -- 172.21.15.55:0/3042207445 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 102) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f44b0050f80 con 0x7f44b40b03c0 2023-10-06T05:07:39.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:39.113+0000 7f44a9ffb700 1 -- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 msgr2=0x7f4498069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:39.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:39.113+0000 7f44a9ffb700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 0x7f4498069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:39.171 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:39.170+0000 7f5eaa350180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:07:39.243 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:39.241+0000 7f261ea71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:39.243 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:39.241+0000 7f261ea71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:39.243 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:39.241+0000 7f261ea71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:39.243 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:39.241+0000 7f261ea71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:39.243 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:39.241+0000 7f261ea71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:39.245 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:39.243+0000 7f2621a77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:39.245 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:39.243+0000 7f2621a77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:39.245 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:39.243+0000 7f2621a77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:39.245 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:39.243+0000 7f2621a77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:39.245 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:39.243+0000 7f2621a77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:39.278 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:39.276+0000 7f5eaa350180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:07:39.376 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:39.374+0000 7f5eaa350180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:07:40.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.117+0000 7f44a9ffb700 1 -- 172.21.15.55:0/3042207445 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mgrmap(e 103) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f44b00126b0 con 0x7f44b40b03c0 2023-10-06T05:07:40.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.117+0000 7f44a9ffb700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 0x7f4498069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:40.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.117+0000 7f44a9ffb700 1 -- 172.21.15.55:0/3042207445 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f449800a3f0 con 0x7f4498066f80 2023-10-06T05:07:40.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.120+0000 7f44b959a700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 0x7f4498069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:40.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.125+0000 7f44b959a700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 0x7f4498069430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f44b40c3de0 tx=0x7f44a4009040 comp rx=0 tx=0).ready entity=mgr.8558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:40.191 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.189+0000 7f44a9ffb700 1 -- 172.21.15.55:0/3042207445 <== mgr.8558 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f449800a3f0 con 0x7f4498066f80 2023-10-06T05:07:40.191 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule set for path snap_test_dir1 2023-10-06T05:07:40.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.192+0000 7f44bad9d700 1 -- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 msgr2=0x7f4498069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:40.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.192+0000 7f44bad9d700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 0x7f4498069430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f44b40c3de0 tx=0x7f44a4009040 comp rx=0 tx=0).stop 2023-10-06T05:07:40.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.192+0000 7f44bad9d700 1 -- 172.21.15.55:0/3042207445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b40b03c0 msgr2=0x7f44b40c75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:40.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.192+0000 7f44bad9d700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b40b03c0 0x7f44b40c75a0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f44b000c790 tx=0x7f44b0007e80 comp rx=0 tx=0).stop 2023-10-06T05:07:40.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.192+0000 7f44bad9d700 1 -- 172.21.15.55:0/3042207445 shutdown_connections 2023-10-06T05:07:40.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.192+0000 7f44bad9d700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4498066f80 0x7f4498069430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.192+0000 7f44bad9d700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f44b40cca20 0x7f44b40c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.192+0000 7f44bad9d700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f44b40b03c0 0x7f44b40c75a0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.192+0000 7f44bad9d700 1 --2- 172.21.15.55:0/3042207445 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f44b40b0790 0x7f44b40c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.192+0000 7f44bad9d700 1 -- 172.21.15.55:0/3042207445 >> 172.21.15.55:0/3042207445 conn(0x7f44b4063230 msgr2=0x7f44b40cba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:40.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.192+0000 7f44bad9d700 1 -- 172.21.15.55:0/3042207445 shutdown_connections 2023-10-06T05:07:40.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.192+0000 7f44bad9d700 1 -- 172.21.15.55:0/3042207445 wait complete. 2023-10-06T05:07:40.213 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-06T05:07:40.213 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-10-06T05:07:40.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.407+0000 7fee26f9d700 1 Processor -- start 2023-10-06T05:07:40.409 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.407+0000 7fee26f9d700 1 -- start start 2023-10-06T05:07:40.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.408+0000 7fee26f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee2006e630 0x7fee200abf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:40.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.408+0000 7fee26f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fee20159840 0x7fee200ac490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:40.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.408+0000 7fee26f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee20063870 con 0x7fee2015df30 2023-10-06T05:07:40.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.408+0000 7fee26f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee200acda0 con 0x7fee20159840 2023-10-06T05:07:40.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.408+0000 7fee26f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee200acac0 con 0x7fee2006e630 2023-10-06T05:07:40.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.409+0000 7fee25f9b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee2006e630 0x7fee200abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:40.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.409+0000 7fee2679c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fee20159840 0x7fee200ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:40.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.409+0000 7fee2579a700 1 --1- >> v1:172.21.15.163:6789/0 conn(0x7fee2015df30 0x7fee20159130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:54676/0 (socket says 172.21.15.55:54676) 2023-10-06T05:07:40.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.409+0000 7fee2579a700 1 -- 172.21.15.55:0/3071109556 learned_addr learned my addr 172.21.15.55:0/3071109556 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:40.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.409+0000 7fee24f99700 1 -- 172.21.15.55:0/3071109556 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323359276 0 0) 0x7fee20063870 con 0x7fee2015df30 2023-10-06T05:07:40.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.409+0000 7fee24f99700 1 -- 172.21.15.55:0/3071109556 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee0c003570 con 0x7fee2015df30 2023-10-06T05:07:40.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.409+0000 7fee25f9b700 1 -- 172.21.15.55:0/3071109556 >> v2:172.21.15.163:3300/0 conn(0x7fee20159840 msgr2=0x7fee200ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:40.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.409+0000 7fee25f9b700 1 --2- 172.21.15.55:0/3071109556 >> v2:172.21.15.163:3300/0 conn(0x7fee20159840 0x7fee200ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.409+0000 7fee25f9b700 1 -- 172.21.15.55:0/3071109556 >> v1:172.21.15.163:6789/0 conn(0x7fee2015df30 legacy=0x7fee20159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:40.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.409+0000 7fee25f9b700 1 -- 172.21.15.55:0/3071109556 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee2015d450 con 0x7fee2006e630 2023-10-06T05:07:40.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.409+0000 7fee2679c700 1 --2- 172.21.15.55:0/3071109556 >> v2:172.21.15.163:3300/0 conn(0x7fee20159840 0x7fee200ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:40.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.410+0000 7fee25f9b700 1 --2- 172.21.15.55:0/3071109556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee2006e630 0x7fee200abf50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0098e0 tx=0x7fee1c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2da0617b423ed0a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:40.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.410+0000 7fee24f99700 1 -- 172.21.15.55:0/3071109556 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c011630 con 0x7fee2006e630 2023-10-06T05:07:40.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.410+0000 7fee24f99700 1 -- 172.21.15.55:0/3071109556 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c011790 con 0x7fee2006e630 2023-10-06T05:07:40.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.410+0000 7fee24f99700 1 -- 172.21.15.55:0/3071109556 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c011b30 con 0x7fee2006e630 2023-10-06T05:07:40.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.410+0000 7fee26f9d700 1 -- 172.21.15.55:0/3071109556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee2006e630 msgr2=0x7fee200abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:40.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.410+0000 7fee26f9d700 1 --2- 172.21.15.55:0/3071109556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee2006e630 0x7fee200abf50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0098e0 tx=0x7fee1c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.410+0000 7fee26f9d700 1 -- 172.21.15.55:0/3071109556 shutdown_connections 2023-10-06T05:07:40.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.410+0000 7fee26f9d700 1 --2- 172.21.15.55:0/3071109556 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee2006e630 0x7fee200abf50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.410+0000 7fee26f9d700 1 --2- 172.21.15.55:0/3071109556 >> v2:172.21.15.163:3300/0 conn(0x7fee20159840 0x7fee200ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.414 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.410+0000 7fee26f9d700 1 -- 172.21.15.55:0/3071109556 >> 172.21.15.55:0/3071109556 conn(0x7fee200c9f50 msgr2=0x7fee2006dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:40.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.410+0000 7fee26f9d700 1 -- 172.21.15.55:0/3071109556 shutdown_connections 2023-10-06T05:07:40.415 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.410+0000 7fee26f9d700 1 -- 172.21.15.55:0/3071109556 wait complete. 2023-10-06T05:07:40.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.412+0000 7fee26f9d700 1 Processor -- start 2023-10-06T05:07:40.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.412+0000 7fee26f9d700 1 -- start start 2023-10-06T05:07:40.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.413+0000 7fee26f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee2006e630 0x7fee2016ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:40.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.413+0000 7fee26f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee20159840 0x7fee2015c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:40.417 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.413+0000 7fee26f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee2015df30 0x7fee200c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:40.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.413+0000 7fee26f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2016d8d0 con 0x7fee2006e630 2023-10-06T05:07:40.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.413+0000 7fee26f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2015a030 con 0x7fee20159840 2023-10-06T05:07:40.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.413+0000 7fee26f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2015a310 con 0x7fee2015df30 2023-10-06T05:07:40.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.413+0000 7fee25f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee2006e630 0x7fee2016ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:40.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.413+0000 7fee25f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee2006e630 0x7fee2016ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50864/0 (socket says 172.21.15.55:50864) 2023-10-06T05:07:40.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.413+0000 7fee25f9b700 1 -- 172.21.15.55:0/624508590 learned_addr learned my addr 172.21.15.55:0/624508590 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:40.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.413+0000 7fee2679c700 1 --2- 172.21.15.55:0/624508590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee2015df30 0x7fee200c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:40.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.413+0000 7fee2579a700 1 --2- 172.21.15.55:0/624508590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee20159840 0x7fee2015c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:40.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.414+0000 7fee25f9b700 1 -- 172.21.15.55:0/624508590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee2015df30 msgr2=0x7fee200c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:40.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.414+0000 7fee25f9b700 1 --2- 172.21.15.55:0/624508590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee2015df30 0x7fee200c4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.414+0000 7fee25f9b700 1 -- 172.21.15.55:0/624508590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee20159840 msgr2=0x7fee2015c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:40.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.414+0000 7fee25f9b700 1 --2- 172.21.15.55:0/624508590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee20159840 0x7fee2015c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.414+0000 7fee25f9b700 1 -- 172.21.15.55:0/624508590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee18009a20 con 0x7fee2006e630 2023-10-06T05:07:40.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.414+0000 7fee25f9b700 1 --2- 172.21.15.55:0/624508590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee2006e630 0x7fee2016ccb0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fee200acc00 tx=0x7fee1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:40.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.414+0000 7fee15ffb700 1 -- 172.21.15.55:0/624508590 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c011b10 con 0x7fee2006e630 2023-10-06T05:07:40.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.414+0000 7fee15ffb700 1 -- 172.21.15.55:0/624508590 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c000cc0 con 0x7fee2006e630 2023-10-06T05:07:40.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.414+0000 7fee15ffb700 1 -- 172.21.15.55:0/624508590 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c025620 con 0x7fee2006e630 2023-10-06T05:07:40.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.414+0000 7fee26f9d700 1 -- 172.21.15.55:0/624508590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1c009420 con 0x7fee2006e630 2023-10-06T05:07:40.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.414+0000 7fee26f9d700 1 -- 172.21.15.55:0/624508590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2015a8d0 con 0x7fee2006e630 2023-10-06T05:07:40.419 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.416+0000 7fee26f9d700 1 -- 172.21.15.55:0/624508590 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2004c3c0 con 0x7fee2006e630 2023-10-06T05:07:40.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.416+0000 7fee15ffb700 1 -- 172.21.15.55:0/624508590 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 103) v1 ==== 84595+0+0 (secure 0 0 0) 0x7fee1c025a10 con 0x7fee2006e630 2023-10-06T05:07:40.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.418+0000 7fee15ffb700 1 --2- 172.21.15.55:0/624508590 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fee04066dd0 0x7fee04069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:40.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.418+0000 7fee15ffb700 1 -- 172.21.15.55:0/624508590 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fee1c085e90 con 0x7fee2006e630 2023-10-06T05:07:40.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.418+0000 7fee2579a700 1 --2- 172.21.15.55:0/624508590 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fee04066dd0 0x7fee04069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:40.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.419+0000 7fee2579a700 1 --2- 172.21.15.55:0/624508590 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fee04066dd0 0x7fee04069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee10007fd0 tx=0x7fee10009040 comp rx=0 tx=0).ready entity=mgr.8558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:40.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.423+0000 7fee15ffb700 1 -- 172.21.15.55:0/624508590 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee1c052f80 con 0x7fee2006e630 2023-10-06T05:07:40.653 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.651+0000 7fee26f9d700 1 -- 172.21.15.55:0/624508590 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fee200acf40 con 0x7fee04066dd0 2023-10-06T05:07:40.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.715+0000 7fee15ffb700 1 -- 172.21.15.55:0/624508590 <== mgr.8558 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7fee200acf40 con 0x7fee04066dd0 2023-10-06T05:07:40.717 INFO:teuthology.orchestra.run.smithi055.stdout:Retention added to path snap_test_dir1 2023-10-06T05:07:40.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.717+0000 7fee26f9d700 1 -- 172.21.15.55:0/624508590 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fee04066dd0 msgr2=0x7fee04069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:40.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.717+0000 7fee26f9d700 1 --2- 172.21.15.55:0/624508590 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fee04066dd0 0x7fee04069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee10007fd0 tx=0x7fee10009040 comp rx=0 tx=0).stop 2023-10-06T05:07:40.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.717+0000 7fee26f9d700 1 -- 172.21.15.55:0/624508590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee2006e630 msgr2=0x7fee2016ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:40.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.717+0000 7fee26f9d700 1 --2- 172.21.15.55:0/624508590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee2006e630 0x7fee2016ccb0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fee200acc00 tx=0x7fee1c017000 comp rx=0 tx=0).stop 2023-10-06T05:07:40.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.717+0000 7fee26f9d700 1 -- 172.21.15.55:0/624508590 shutdown_connections 2023-10-06T05:07:40.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.717+0000 7fee26f9d700 1 --2- 172.21.15.55:0/624508590 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fee04066dd0 0x7fee04069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.717+0000 7fee26f9d700 1 --2- 172.21.15.55:0/624508590 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fee2015df30 0x7fee200c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.717+0000 7fee26f9d700 1 --2- 172.21.15.55:0/624508590 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fee2006e630 0x7fee2016ccb0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.717+0000 7fee26f9d700 1 --2- 172.21.15.55:0/624508590 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fee20159840 0x7fee2015c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.717+0000 7fee26f9d700 1 -- 172.21.15.55:0/624508590 >> 172.21.15.55:0/624508590 conn(0x7fee200c9f50 msgr2=0x7fee200cbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:40.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.717+0000 7fee26f9d700 1 -- 172.21.15.55:0/624508590 shutdown_connections 2023-10-06T05:07:40.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.717+0000 7fee26f9d700 1 -- 172.21.15.55:0/624508590 wait complete. 2023-10-06T05:07:40.737 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-06T05:07:40.741 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-06-05_08_00 in ~20s... 2023-10-06T05:07:40.741 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-10-06T05:07:40.741 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-10-06T05:07:40.932 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.930+0000 7f3ffffff700 1 Processor -- start 2023-10-06T05:07:40.933 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.931+0000 7f3ffffff700 1 -- start start 2023-10-06T05:07:40.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.932+0000 7f3ffffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f40000cb560 0x7f40000cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:40.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.932+0000 7f3ffffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f40000cc970 0x7f40000697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:40.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.932+0000 7f3ffffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4000159db0 con 0x7f4000069cf0 2023-10-06T05:07:40.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.932+0000 7f3ffffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f400006aa30 con 0x7f40000cb560 2023-10-06T05:07:40.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.932+0000 7f3ffffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40000cce10 con 0x7f40000cc970 2023-10-06T05:07:40.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.932+0000 7f3ffeffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f40000cb560 0x7f40000cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:40.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.932+0000 7f3ffeffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f40000cb560 0x7f40000cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50876/0 (socket says 172.21.15.55:50876) 2023-10-06T05:07:40.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.932+0000 7f3fff7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4000069cf0 0x7f400006d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34512/0 (socket says 172.21.15.55:34512) 2023-10-06T05:07:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.932+0000 7f3ffeffd700 1 -- 172.21.15.55:0/4111456733 learned_addr learned my addr 172.21.15.55:0/4111456733 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.932+0000 7f3ffe7fc700 1 --2- 172.21.15.55:0/4111456733 >> v2:172.21.15.163:3300/0 conn(0x7f40000cc970 0x7f40000697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.933+0000 7f3ffeffd700 1 -- 172.21.15.55:0/4111456733 >> v2:172.21.15.163:3300/0 conn(0x7f40000cc970 msgr2=0x7f40000697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.933+0000 7f3ffeffd700 1 --2- 172.21.15.55:0/4111456733 >> v2:172.21.15.163:3300/0 conn(0x7f40000cc970 0x7f40000697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.933+0000 7f3ffeffd700 1 -- 172.21.15.55:0/4111456733 >> v1:172.21.15.55:6789/0 conn(0x7f4000069cf0 legacy=0x7f400006d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.933+0000 7f3ffeffd700 1 -- 172.21.15.55:0/4111456733 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400006a470 con 0x7f40000cb560 2023-10-06T05:07:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.933+0000 7f3ffdffb700 1 -- 172.21.15.55:0/4111456733 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991717773 0 0) 0x7f4000159db0 con 0x7f4000069cf0 2023-10-06T05:07:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.933+0000 7f3ffeffd700 1 --2- 172.21.15.55:0/4111456733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40000cb560 0x7f40000cc430 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00099b0 tx=0x7f3ff0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a286feda11908e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.933+0000 7f3ffdffb700 1 -- 172.21.15.55:0/4111456733 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000b820 con 0x7f40000cb560 2023-10-06T05:07:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.933+0000 7f3ffdffb700 1 -- 172.21.15.55:0/4111456733 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff000b980 con 0x7f40000cb560 2023-10-06T05:07:40.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.933+0000 7f3ffdffb700 1 -- 172.21.15.55:0/4111456733 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000bd20 con 0x7f40000cb560 2023-10-06T05:07:40.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.934+0000 7f3ffffff700 1 -- 172.21.15.55:0/4111456733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40000cb560 msgr2=0x7f40000cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:40.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.934+0000 7f3ffffff700 1 --2- 172.21.15.55:0/4111456733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40000cb560 0x7f40000cc430 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00099b0 tx=0x7f3ff0009250 comp rx=0 tx=0).stop 2023-10-06T05:07:40.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.934+0000 7f3ffffff700 1 -- 172.21.15.55:0/4111456733 shutdown_connections 2023-10-06T05:07:40.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.934+0000 7f3ffffff700 1 --2- 172.21.15.55:0/4111456733 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40000cb560 0x7f40000cc430 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.934+0000 7f3ffffff700 1 --2- 172.21.15.55:0/4111456733 >> v2:172.21.15.163:3300/0 conn(0x7f40000cc970 0x7f40000697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.934+0000 7f3ffffff700 1 -- 172.21.15.55:0/4111456733 >> 172.21.15.55:0/4111456733 conn(0x7f400006baf0 msgr2=0x7f400006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:40.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.934+0000 7f3ffffff700 1 -- 172.21.15.55:0/4111456733 shutdown_connections 2023-10-06T05:07:40.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.934+0000 7f3ffffff700 1 -- 172.21.15.55:0/4111456733 wait complete. 2023-10-06T05:07:40.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.935+0000 7f3ffffff700 1 Processor -- start 2023-10-06T05:07:40.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.936+0000 7f3ffffff700 1 -- start start 2023-10-06T05:07:40.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.936+0000 7f3ffffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4000069cf0 0x7f40000c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:40.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.936+0000 7f3ffffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40000cb560 0x7f4000169670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3ffffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40000cc970 0x7f4000169bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3ffffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40000c7120 con 0x7f40000cc970 2023-10-06T05:07:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3ffffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4000155300 con 0x7f40000cb560 2023-10-06T05:07:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3ffffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40000c6b60 con 0x7f4000069cf0 2023-10-06T05:07:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3fff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40000cc970 0x7f4000169bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3fff7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40000cc970 0x7f4000169bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50888/0 (socket says 172.21.15.55:50888) 2023-10-06T05:07:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3fff7fe700 1 -- 172.21.15.55:0/3974840183 learned_addr learned my addr 172.21.15.55:0/3974840183 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3ffeffd700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4000069cf0 0x7f40000c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3ffe7fc700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40000cb560 0x7f4000169670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3fff7fe700 1 -- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4000069cf0 msgr2=0x7f40000c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:40.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3fff7fe700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4000069cf0 0x7f40000c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3fff7fe700 1 -- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40000cb560 msgr2=0x7f4000169670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:40.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3fff7fe700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40000cb560 0x7f4000169670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:40.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3fff7fe700 1 -- 172.21.15.55:0/3974840183 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff00094f0 con 0x7f40000cc970 2023-10-06T05:07:40.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3ffeffd700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4000069cf0 0x7f40000c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:40.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.937+0000 7f3ffe7fc700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40000cb560 0x7f4000169670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:40.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.938+0000 7f3fff7fe700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40000cc970 0x7f4000169bb0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800cd50 tx=0x7f3ff8001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:40.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.938+0000 7f3fe6ffd700 1 -- 172.21.15.55:0/3974840183 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8005df0 con 0x7f40000cc970 2023-10-06T05:07:40.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.938+0000 7f3fe6ffd700 1 -- 172.21.15.55:0/3974840183 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff800a960 con 0x7f40000cc970 2023-10-06T05:07:40.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.938+0000 7f3fe6ffd700 1 -- 172.21.15.55:0/3974840183 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff800ad20 con 0x7f40000cc970 2023-10-06T05:07:40.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.938+0000 7f3ffffff700 1 -- 172.21.15.55:0/3974840183 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40000c6e40 con 0x7f40000cc970 2023-10-06T05:07:40.942 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.938+0000 7f3ffffff700 1 -- 172.21.15.55:0/3974840183 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40000c74c0 con 0x7f40000cc970 2023-10-06T05:07:40.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.939+0000 7f3ffffff700 1 -- 172.21.15.55:0/3974840183 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4000154df0 con 0x7f40000cc970 2023-10-06T05:07:40.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.940+0000 7f3fe6ffd700 1 -- 172.21.15.55:0/3974840183 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 103) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f3ff8007300 con 0x7f40000cc970 2023-10-06T05:07:40.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.941+0000 7f3fe6ffd700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3fec066dd0 0x7f3fec069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:40.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.941+0000 7f3fe6ffd700 1 -- 172.21.15.55:0/3974840183 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f3ff80848e0 con 0x7f40000cc970 2023-10-06T05:07:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.941+0000 7f3ffeffd700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3fec066dd0 0x7f3fec069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:40.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.942+0000 7f3ffeffd700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3fec066dd0 0x7f3fec069280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00099b0 tx=0x7f3ff0017000 comp rx=0 tx=0).ready entity=mgr.8558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:40.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:40.945+0000 7f3fe6ffd700 1 -- 172.21.15.55:0/3974840183 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4000154df0 con 0x7f40000cc970 2023-10-06T05:07:41.165 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.163+0000 7f3fe6ffd700 1 -- 172.21.15.55:0/3974840183 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 104) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f3ff8051d60 con 0x7f40000cc970 2023-10-06T05:07:41.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.178+0000 7f3ffffff700 1 -- 172.21.15.55:0/3974840183 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4000154df0 con 0x7f3fec066dd0 2023-10-06T05:07:41.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.187+0000 7f3fe6ffd700 1 -- 172.21.15.55:0/3974840183 <== mgr.8558 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f4000154df0 con 0x7f3fec066dd0 2023-10-06T05:07:41.190 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:41.190 INFO:teuthology.orchestra.run.smithi055.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-06T05:07:41.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.190+0000 7f3ffffff700 1 -- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3fec066dd0 msgr2=0x7f3fec069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:41.192 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.190+0000 7f3ffffff700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3fec066dd0 0x7f3fec069280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ff00099b0 tx=0x7f3ff0017000 comp rx=0 tx=0).stop 2023-10-06T05:07:41.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.190+0000 7f3ffffff700 1 -- 172.21.15.55:0/3974840183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40000cc970 msgr2=0x7f4000169bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:41.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.190+0000 7f3ffffff700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40000cc970 0x7f4000169bb0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800cd50 tx=0x7f3ff8001b10 comp rx=0 tx=0).stop 2023-10-06T05:07:41.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.190+0000 7f3ffffff700 1 -- 172.21.15.55:0/3974840183 shutdown_connections 2023-10-06T05:07:41.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.190+0000 7f3ffffff700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3fec066dd0 0x7f3fec069280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:41.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.190+0000 7f3ffffff700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f40000cc970 0x7f4000169bb0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:41.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.190+0000 7f3ffffff700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4000069cf0 0x7f40000c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:41.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.190+0000 7f3ffffff700 1 --2- 172.21.15.55:0/3974840183 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f40000cb560 0x7f4000169670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:41.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.190+0000 7f3ffffff700 1 -- 172.21.15.55:0/3974840183 >> 172.21.15.55:0/3974840183 conn(0x7f400006baf0 msgr2=0x7f40000c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:41.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.190+0000 7f3ffffff700 1 -- 172.21.15.55:0/3974840183 shutdown_connections 2023-10-06T05:07:41.193 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.190+0000 7f3ffffff700 1 -- 172.21.15.55:0/3974840183 wait complete. 2023-10-06T05:07:41.209 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-06T05:07:41.210 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-10-06T05:07:41.210 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-06T05:07:41.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.400+0000 7fa0be59d700 1 Processor -- start 2023-10-06T05:07:41.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.401+0000 7fa0be59d700 1 -- start start 2023-10-06T05:07:41.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.402+0000 7fa0be59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa0b80ae180 0x7fa0b80acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:41.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.402+0000 7fa0be59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b80aeb10 con 0x7fa0b80656e0 2023-10-06T05:07:41.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.402+0000 7fa0be59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b80af540 con 0x7fa0b806b290 2023-10-06T05:07:41.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.402+0000 7fa0be59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b80aedf0 con 0x7fa0b80ae180 2023-10-06T05:07:41.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.402+0000 7fa0bcd9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fa0b80656e0 0x7fa0b80ad090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34528/0 (socket says 172.21.15.55:34528) 2023-10-06T05:07:41.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.402+0000 7fa0bcd9a700 1 -- 172.21.15.55:0/3184312910 learned_addr learned my addr 172.21.15.55:0/3184312910 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:41.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.402+0000 7fa0bd59b700 1 --2- 172.21.15.55:0/3184312910 >> v2:172.21.15.163:3300/0 conn(0x7fa0b80ae180 0x7fa0b80acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:41.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.402+0000 7fa0affff700 1 -- 172.21.15.55:0/3184312910 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489918206 0 0) 0x7fa0b80aeb10 con 0x7fa0b80656e0 2023-10-06T05:07:41.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.402+0000 7fa0affff700 1 -- 172.21.15.55:0/3184312910 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0a4003570 con 0x7fa0b80656e0 2023-10-06T05:07:41.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.403+0000 7fa0affff700 1 -- 172.21.15.55:0/3184312910 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191155931 0 0) 0x7fa0b80af540 con 0x7fa0b806b290 2023-10-06T05:07:41.405 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.403+0000 7fa0affff700 1 -- 172.21.15.55:0/3184312910 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0b80aeb10 con 0x7fa0b806b290 2023-10-06T05:07:41.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.403+0000 7fa0bd59b700 1 -- 172.21.15.55:0/3184312910 >> v1:172.21.15.163:6789/0 conn(0x7fa0b806b290 legacy=0x7fa0b80b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:41.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.403+0000 7fa0bd59b700 1 -- 172.21.15.55:0/3184312910 >> v1:172.21.15.55:6789/0 conn(0x7fa0b80656e0 legacy=0x7fa0b80ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:41.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.403+0000 7fa0bd59b700 1 -- 172.21.15.55:0/3184312910 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b80af820 con 0x7fa0b80ae180 2023-10-06T05:07:41.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.403+0000 7fa0bd59b700 1 --2- 172.21.15.55:0/3184312910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b80ae180 0x7fa0b80acb50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40098d0 tx=0x7fa0b4009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4539fac7ae17e1c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:41.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.403+0000 7fa0affff700 1 -- 172.21.15.55:0/3184312910 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4005c10 con 0x7fa0b80ae180 2023-10-06T05:07:41.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.403+0000 7fa0affff700 1 -- 172.21.15.55:0/3184312910 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b4005d70 con 0x7fa0b80ae180 2023-10-06T05:07:41.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.403+0000 7fa0affff700 1 -- 172.21.15.55:0/3184312910 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4006280 con 0x7fa0b80ae180 2023-10-06T05:07:41.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.404+0000 7fa0be59d700 1 -- 172.21.15.55:0/3184312910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b80ae180 msgr2=0x7fa0b80acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:41.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.404+0000 7fa0be59d700 1 --2- 172.21.15.55:0/3184312910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b80ae180 0x7fa0b80acb50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40098d0 tx=0x7fa0b4009120 comp rx=0 tx=0).stop 2023-10-06T05:07:41.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.404+0000 7fa0be59d700 1 -- 172.21.15.55:0/3184312910 shutdown_connections 2023-10-06T05:07:41.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.404+0000 7fa0be59d700 1 --2- 172.21.15.55:0/3184312910 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b80ae180 0x7fa0b80acb50 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:41.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.404+0000 7fa0be59d700 1 -- 172.21.15.55:0/3184312910 >> 172.21.15.55:0/3184312910 conn(0x7fa0b806f6a0 msgr2=0x7fa0b806ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:41.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.404+0000 7fa0be59d700 1 -- 172.21.15.55:0/3184312910 shutdown_connections 2023-10-06T05:07:41.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.404+0000 7fa0be59d700 1 -- 172.21.15.55:0/3184312910 wait complete. 2023-10-06T05:07:41.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.405+0000 7fa0be59d700 1 Processor -- start 2023-10-06T05:07:41.408 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.405+0000 7fa0be59d700 1 -- start start 2023-10-06T05:07:41.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.406+0000 7fa0be59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b80656e0 0x7fa0b8144a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:41.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.406+0000 7fa0be59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b806b290 0x7fa0b815e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:41.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.406+0000 7fa0be59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b80ae180 0x7fa0b8098430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:41.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.406+0000 7fa0be59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b815e640 con 0x7fa0b80ae180 2023-10-06T05:07:41.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.406+0000 7fa0be59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b81445e0 con 0x7fa0b806b290 2023-10-06T05:07:41.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.406+0000 7fa0be59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b81439d0 con 0x7fa0b80656e0 2023-10-06T05:07:41.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.406+0000 7fa0bdd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b80ae180 0x7fa0b8098430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:41.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.407+0000 7fa0bdd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b80ae180 0x7fa0b8098430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50906/0 (socket says 172.21.15.55:50906) 2023-10-06T05:07:41.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.407+0000 7fa0bdd9c700 1 -- 172.21.15.55:0/3168646065 learned_addr learned my addr 172.21.15.55:0/3168646065 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:41.410 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.407+0000 7fa0bcd9a700 1 --2- 172.21.15.55:0/3168646065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b806b290 0x7fa0b815e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:41.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.407+0000 7fa0bd59b700 1 --2- 172.21.15.55:0/3168646065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b80656e0 0x7fa0b8144a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:41.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.407+0000 7fa0bdd9c700 1 -- 172.21.15.55:0/3168646065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b80656e0 msgr2=0x7fa0b8144a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:41.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.407+0000 7fa0bdd9c700 1 --2- 172.21.15.55:0/3168646065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b80656e0 0x7fa0b8144a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:41.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.407+0000 7fa0bdd9c700 1 -- 172.21.15.55:0/3168646065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b806b290 msgr2=0x7fa0b815e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:41.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.407+0000 7fa0bdd9c700 1 --2- 172.21.15.55:0/3168646065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b806b290 0x7fa0b815e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:41.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.407+0000 7fa0bdd9c700 1 -- 172.21.15.55:0/3168646065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b40093c0 con 0x7fa0b80ae180 2023-10-06T05:07:41.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.407+0000 7fa0bdd9c700 1 --2- 172.21.15.55:0/3168646065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b80ae180 0x7fa0b8098430 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c910 tx=0x7fa0b0002720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:41.411 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.407+0000 7fa0ad7fa700 1 -- 172.21.15.55:0/3168646065 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0012630 con 0x7fa0b80ae180 2023-10-06T05:07:41.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.408+0000 7fa0ad7fa700 1 -- 172.21.15.55:0/3168646065 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b00127b0 con 0x7fa0b80ae180 2023-10-06T05:07:41.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.408+0000 7fa0ad7fa700 1 -- 172.21.15.55:0/3168646065 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0012b70 con 0x7fa0b80ae180 2023-10-06T05:07:41.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.408+0000 7fa0be59d700 1 -- 172.21.15.55:0/3168646065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b8143cb0 con 0x7fa0b80ae180 2023-10-06T05:07:41.412 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.408+0000 7fa0be59d700 1 -- 172.21.15.55:0/3168646065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b8144270 con 0x7fa0b80ae180 2023-10-06T05:07:41.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.409+0000 7fa0be59d700 1 -- 172.21.15.55:0/3168646065 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b804c3c0 con 0x7fa0b80ae180 2023-10-06T05:07:41.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.410+0000 7fa0ad7fa700 1 -- 172.21.15.55:0/3168646065 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 104) v1 ==== 84916+0+0 (secure 0 0 0) 0x7fa0b001b400 con 0x7fa0b80ae180 2023-10-06T05:07:41.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.410+0000 7fa0ad7fa700 1 --2- 172.21.15.55:0/3168646065 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa09c067130 0x7fa09c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:41.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.410+0000 7fa0ad7fa700 1 -- 172.21.15.55:0/3168646065 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fa0b0084f20 con 0x7fa0b80ae180 2023-10-06T05:07:41.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.411+0000 7fa0bd59b700 1 --2- 172.21.15.55:0/3168646065 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa09c067130 0x7fa09c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:41.413 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.412+0000 7fa0bd59b700 1 --2- 172.21.15.55:0/3168646065 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa09c067130 0x7fa09c0695e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009390 tx=0x7fa0b4016000 comp rx=0 tx=0).ready entity=mgr.8558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:41.416 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.415+0000 7fa0ad7fa700 1 -- 172.21.15.55:0/3168646065 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0b0051af0 con 0x7fa0b80ae180 2023-10-06T05:07:41.649 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.647+0000 7fa0be59d700 1 -- 172.21.15.55:0/3168646065 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0b806caa0 con 0x7fa09c067130 2023-10-06T05:07:41.719 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.717+0000 7fa0ad7fa700 1 -- 172.21.15.55:0/3168646065 <== mgr.8558 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa0b806caa0 con 0x7fa09c067130 2023-10-06T05:07:41.719 INFO:teuthology.orchestra.run.smithi055.stdout:Schedule removed for path snap_test_dir1 2023-10-06T05:07:41.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.719+0000 7fa0be59d700 1 -- 172.21.15.55:0/3168646065 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa09c067130 msgr2=0x7fa09c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:41.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.719+0000 7fa0be59d700 1 --2- 172.21.15.55:0/3168646065 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa09c067130 0x7fa09c0695e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009390 tx=0x7fa0b4016000 comp rx=0 tx=0).stop 2023-10-06T05:07:41.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.719+0000 7fa0be59d700 1 -- 172.21.15.55:0/3168646065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b80ae180 msgr2=0x7fa0b8098430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:41.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.719+0000 7fa0be59d700 1 --2- 172.21.15.55:0/3168646065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b80ae180 0x7fa0b8098430 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c910 tx=0x7fa0b0002720 comp rx=0 tx=0).stop 2023-10-06T05:07:41.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.719+0000 7fa0be59d700 1 -- 172.21.15.55:0/3168646065 shutdown_connections 2023-10-06T05:07:41.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.719+0000 7fa0be59d700 1 --2- 172.21.15.55:0/3168646065 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa09c067130 0x7fa09c0695e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:41.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.719+0000 7fa0be59d700 1 --2- 172.21.15.55:0/3168646065 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa0b80ae180 0x7fa0b8098430 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:41.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.719+0000 7fa0be59d700 1 --2- 172.21.15.55:0/3168646065 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa0b80656e0 0x7fa0b8144a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:41.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.719+0000 7fa0be59d700 1 --2- 172.21.15.55:0/3168646065 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa0b806b290 0x7fa0b815e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:41.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.719+0000 7fa0be59d700 1 -- 172.21.15.55:0/3168646065 >> 172.21.15.55:0/3168646065 conn(0x7fa0b806f6a0 msgr2=0x7fa0b806a200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:41.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.719+0000 7fa0be59d700 1 -- 172.21.15.55:0/3168646065 shutdown_connections 2023-10-06T05:07:41.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:41.720+0000 7fa0be59d700 1 -- 172.21.15.55:0/3168646065 wait complete. 2023-10-06T05:07:41.739 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-06T05:07:41.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:07:41.739 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-06T05:07:41.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:07:41.778 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-06T05:07:41.856 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-06T05:07:42.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.085+0000 7f111b59e700 1 Processor -- start 2023-10-06T05:07:42.087 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.085+0000 7f111b59e700 1 -- start start 2023-10-06T05:07:42.088 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.086+0000 7f111b59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f111c0b16d0 0x7f111c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.086+0000 7f111b59e700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111c0b2080 con 0x7f111c0b33b0 2023-10-06T05:07:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.086+0000 7f111b59e700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111c0cfb00 con 0x7f111c0b4e70 2023-10-06T05:07:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.086+0000 7f111b59e700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111c0b5c40 con 0x7f111c0b16d0 2023-10-06T05:07:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.087+0000 7f1119d9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f111c0b16d0 0x7f111c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.087+0000 7f111ad9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f111c0b33b0 0x7f111c0b37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34542/0 (socket says 172.21.15.55:34542) 2023-10-06T05:07:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.087+0000 7f111ad9d700 1 -- 172.21.15.55:0/2905483312 learned_addr learned my addr 172.21.15.55:0/2905483312 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:42.089 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.087+0000 7f111a59c700 1 --1- 172.21.15.55:0/2905483312 >> v1:172.21.15.163:6789/0 conn(0x7f111c0b4e70 0x7f111c0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:54700/0 (socket says 172.21.15.55:54700) 2023-10-06T05:07:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.087+0000 7f1119d9b700 1 -- 172.21.15.55:0/2905483312 >> v1:172.21.15.163:6789/0 conn(0x7f111c0b4e70 legacy=0x7f111c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.087+0000 7f111959a700 1 -- 172.21.15.55:0/2905483312 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411910812 0 0) 0x7f111c0cfb00 con 0x7f111c0b4e70 2023-10-06T05:07:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.087+0000 7f1119d9b700 1 -- 172.21.15.55:0/2905483312 >> v1:172.21.15.55:6789/0 conn(0x7f111c0b33b0 legacy=0x7f111c0b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.087+0000 7f1119d9b700 1 -- 172.21.15.55:0/2905483312 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c0b5f20 con 0x7f111c0b16d0 2023-10-06T05:07:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.087+0000 7f1119d9b700 1 --2- 172.21.15.55:0/2905483312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c0b16d0 0x7f111c0b1b40 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f11100098e0 tx=0x7f11100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0c69cfc1be1da0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.088+0000 7f111959a700 1 -- 172.21.15.55:0/2905483312 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110009e80 con 0x7f111c0b16d0 2023-10-06T05:07:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.088+0000 7f111959a700 1 -- 172.21.15.55:0/2905483312 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111000b760 con 0x7f111c0b16d0 2023-10-06T05:07:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.088+0000 7f111959a700 1 -- 172.21.15.55:0/2905483312 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111000bae0 con 0x7f111c0b16d0 2023-10-06T05:07:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.088+0000 7f111b59e700 1 -- 172.21.15.55:0/2905483312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c0b16d0 msgr2=0x7f111c0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:42.091 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.088+0000 7f111b59e700 1 --2- 172.21.15.55:0/2905483312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c0b16d0 0x7f111c0b1b40 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f11100098e0 tx=0x7f11100091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.088+0000 7f111b59e700 1 -- 172.21.15.55:0/2905483312 shutdown_connections 2023-10-06T05:07:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.088+0000 7f111b59e700 1 --2- 172.21.15.55:0/2905483312 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c0b16d0 0x7f111c0b1b40 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.088+0000 7f111b59e700 1 -- 172.21.15.55:0/2905483312 >> 172.21.15.55:0/2905483312 conn(0x7f111c0d10c0 msgr2=0x7f111c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.088+0000 7f111b59e700 1 -- 172.21.15.55:0/2905483312 shutdown_connections 2023-10-06T05:07:42.092 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.088+0000 7f111b59e700 1 -- 172.21.15.55:0/2905483312 wait complete. 2023-10-06T05:07:42.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.090+0000 7f111b59e700 1 Processor -- start 2023-10-06T05:07:42.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.090+0000 7f111b59e700 1 -- start start 2023-10-06T05:07:42.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.091+0000 7f111b59e700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111c0b16d0 0x7f111c16ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:42.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.091+0000 7f111b59e700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c0b33b0 0x7f111c0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:42.094 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.091+0000 7f111b59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c0b4e70 0x7f111c0a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:42.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.091+0000 7f111b59e700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c16b0c0 con 0x7f111c0b4e70 2023-10-06T05:07:42.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.091+0000 7f111b59e700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0c8ec0 con 0x7f111c0b16d0 2023-10-06T05:07:42.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.091+0000 7f111b59e700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c16b3e0 con 0x7f111c0b33b0 2023-10-06T05:07:42.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.091+0000 7f111ad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c0b4e70 0x7f111c0a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:42.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.091+0000 7f111ad9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c0b4e70 0x7f111c0a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:50920/0 (socket says 172.21.15.55:50920) 2023-10-06T05:07:42.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.091+0000 7f111ad9d700 1 -- 172.21.15.55:0/1780771361 learned_addr learned my addr 172.21.15.55:0/1780771361 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:42.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.091+0000 7f111a59c700 1 --2- 172.21.15.55:0/1780771361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111c0b16d0 0x7f111c16ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:42.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.091+0000 7f1119d9b700 1 --2- 172.21.15.55:0/1780771361 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c0b33b0 0x7f111c0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:42.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.092+0000 7f111ad9d700 1 -- 172.21.15.55:0/1780771361 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c0b33b0 msgr2=0x7f111c0c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:42.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.092+0000 7f111ad9d700 1 --2- 172.21.15.55:0/1780771361 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c0b33b0 0x7f111c0c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:42.095 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.092+0000 7f111ad9d700 1 -- 172.21.15.55:0/1780771361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111c0b16d0 msgr2=0x7f111c16ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:42.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.092+0000 7f111ad9d700 1 --2- 172.21.15.55:0/1780771361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111c0b16d0 0x7f111c16ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:42.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.092+0000 7f111ad9d700 1 -- 172.21.15.55:0/1780771361 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1110009420 con 0x7f111c0b4e70 2023-10-06T05:07:42.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.092+0000 7f111ad9d700 1 --2- 172.21.15.55:0/1780771361 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c0b4e70 0x7f111c0a6f00 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f1114002230 tx=0x7f1114001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:42.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.092+0000 7f110a7fc700 1 -- 172.21.15.55:0/1780771361 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111400aa80 con 0x7f111c0b4e70 2023-10-06T05:07:42.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.092+0000 7f110a7fc700 1 -- 172.21.15.55:0/1780771361 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111400abe0 con 0x7f111c0b4e70 2023-10-06T05:07:42.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.092+0000 7f110a7fc700 1 -- 172.21.15.55:0/1780771361 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11140070f0 con 0x7f111c0b4e70 2023-10-06T05:07:42.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.092+0000 7f111b59e700 1 -- 172.21.15.55:0/1780771361 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c0c83d0 con 0x7f111c0b4e70 2023-10-06T05:07:42.096 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.092+0000 7f111b59e700 1 -- 172.21.15.55:0/1780771361 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c0c8900 con 0x7f111c0b4e70 2023-10-06T05:07:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.093+0000 7f111b59e700 1 -- 172.21.15.55:0/1780771361 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c088990 con 0x7f111c0b4e70 2023-10-06T05:07:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.094+0000 7f110a7fc700 1 -- 172.21.15.55:0/1780771361 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 104) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f11140074a0 con 0x7f111c0b4e70 2023-10-06T05:07:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.095+0000 7f110a7fc700 1 --2- 172.21.15.55:0/1780771361 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f10fc067130 0x7f10fc0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.095+0000 7f110a7fc700 1 -- 172.21.15.55:0/1780771361 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f1114085190 con 0x7f111c0b4e70 2023-10-06T05:07:42.097 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.096+0000 7f111a59c700 1 --2- 172.21.15.55:0/1780771361 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f10fc067130 0x7f10fc0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:42.098 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.096+0000 7f111a59c700 1 --2- 172.21.15.55:0/1780771361 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f10fc067130 0x7f10fc0695e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f111c158ec0 tx=0x7f110c007ed0 comp rx=0 tx=0).ready entity=mgr.8558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:42.101 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.100+0000 7f110a7fc700 1 -- 172.21.15.55:0/1780771361 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1114051d40 con 0x7f111c0b4e70 2023-10-06T05:07:42.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:42.357+0000 7f111b59e700 1 -- 172.21.15.55:0/1780771361 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f111c0cf510 con 0x7f111c0b4e70 2023-10-06T05:07:43.189 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.187+0000 7f110a7fc700 1 -- 172.21.15.55:0/1780771361 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 105) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f1114052110 con 0x7f111c0b4e70 2023-10-06T05:07:43.190 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.189+0000 7f110a7fc700 1 -- 172.21.15.55:0/1780771361 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v105) v1 ==== 93+0+0 (secure 0 0 0) 0x7f1114051d40 con 0x7f111c0b4e70 2023-10-06T05:07:43.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.192+0000 7f111b59e700 1 -- 172.21.15.55:0/1780771361 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f10fc067130 msgr2=0x7f10fc0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:43.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.192+0000 7f111b59e700 1 --2- 172.21.15.55:0/1780771361 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f10fc067130 0x7f10fc0695e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f111c158ec0 tx=0x7f110c007ed0 comp rx=0 tx=0).stop 2023-10-06T05:07:43.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.192+0000 7f111b59e700 1 -- 172.21.15.55:0/1780771361 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c0b4e70 msgr2=0x7f111c0a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:43.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.192+0000 7f111b59e700 1 --2- 172.21.15.55:0/1780771361 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c0b4e70 0x7f111c0a6f00 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f1114002230 tx=0x7f1114001b90 comp rx=0 tx=0).stop 2023-10-06T05:07:43.194 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.192+0000 7f111b59e700 1 -- 172.21.15.55:0/1780771361 shutdown_connections 2023-10-06T05:07:43.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.192+0000 7f111b59e700 1 --2- 172.21.15.55:0/1780771361 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f10fc067130 0x7f10fc0695e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:43.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.192+0000 7f111b59e700 1 --2- 172.21.15.55:0/1780771361 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f111c0b4e70 0x7f111c0a6f00 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:43.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.192+0000 7f111b59e700 1 --2- 172.21.15.55:0/1780771361 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f111c0b16d0 0x7f111c16ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:43.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.192+0000 7f111b59e700 1 --2- 172.21.15.55:0/1780771361 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f111c0b33b0 0x7f111c0c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:43.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.192+0000 7f111b59e700 1 -- 172.21.15.55:0/1780771361 >> 172.21.15.55:0/1780771361 conn(0x7f111c0d10c0 msgr2=0x7f111c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:43.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.192+0000 7f111b59e700 1 -- 172.21.15.55:0/1780771361 shutdown_connections 2023-10-06T05:07:43.195 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.192+0000 7f111b59e700 1 -- 172.21.15.55:0/1780771361 wait complete. 2023-10-06T05:07:43.215 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T05:07:43.215 DEBUG:teuthology.orchestra.run.smithi055:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:07:43.319 DEBUG:teuthology.orchestra.run.smithi163:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T05:07:43.398 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T05:07:43.398 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-06T05:07:43.398 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T05:07:43.399 DEBUG:teuthology.orchestra.run.smithi055:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-06T05:07:43.482 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T05:07:43.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:43.482 DEBUG:teuthology.orchestra.run.smithi055:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T05:07:43.521 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:07:43.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.750+0000 7f041ffff700 1 Processor -- start 2023-10-06T05:07:43.752 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.750+0000 7f041ffff700 1 -- start start 2023-10-06T05:07:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.751+0000 7f041ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04201593b0 0x7f0420159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.752+0000 7f041ffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04200cfe20 con 0x7f04200b1e50 2023-10-06T05:07:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.752+0000 7f041ffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04200cf210 con 0x7f04200ceaa0 2023-10-06T05:07:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.752+0000 7f041ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04200cf4f0 con 0x7f04201593b0 2023-10-06T05:07:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.752+0000 7f041f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f04200b1e50 0x7f04200b3420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34104/0 (socket says 172.21.15.55:34104) 2023-10-06T05:07:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.752+0000 7f041f7fe700 1 -- 172.21.15.55:0/1093859570 learned_addr learned my addr 172.21.15.55:0/1093859570 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:43.754 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.752+0000 7f041effd700 1 --2- 172.21.15.55:0/1093859570 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04201593b0 0x7f0420159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:43.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.752+0000 7f041e7fc700 1 -- 172.21.15.55:0/1093859570 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741170029 0 0) 0x7f04200cfe20 con 0x7f04200b1e50 2023-10-06T05:07:43.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.752+0000 7f041e7fc700 1 -- 172.21.15.55:0/1093859570 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f040c003570 con 0x7f04200b1e50 2023-10-06T05:07:43.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.752+0000 7f041e7fc700 1 -- 172.21.15.55:0/1093859570 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501227536 0 0) 0x7f04200cf210 con 0x7f04200ceaa0 2023-10-06T05:07:43.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.752+0000 7f041e7fc700 1 -- 172.21.15.55:0/1093859570 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04200cfe20 con 0x7f04200ceaa0 2023-10-06T05:07:43.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.753+0000 7f041effd700 1 -- 172.21.15.55:0/1093859570 >> v1:172.21.15.163:6789/0 conn(0x7f04200ceaa0 legacy=0x7f04200b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:43.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.753+0000 7f041effd700 1 -- 172.21.15.55:0/1093859570 >> v1:172.21.15.55:6789/0 conn(0x7f04200b1e50 legacy=0x7f04200b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:43.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.753+0000 7f041effd700 1 -- 172.21.15.55:0/1093859570 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04200cf7d0 con 0x7f04201593b0 2023-10-06T05:07:43.756 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.753+0000 7f041effd700 1 --2- 172.21.15.55:0/1093859570 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04201593b0 0x7f0420159780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0410009910 tx=0x7f04100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de7982e57d1ebb88 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:43.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.753+0000 7f041e7fc700 1 -- 172.21.15.55:0/1093859570 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410011630 con 0x7f04201593b0 2023-10-06T05:07:43.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.753+0000 7f041e7fc700 1 -- 172.21.15.55:0/1093859570 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0410011790 con 0x7f04201593b0 2023-10-06T05:07:43.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.753+0000 7f041e7fc700 1 -- 172.21.15.55:0/1093859570 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410011b30 con 0x7f04201593b0 2023-10-06T05:07:43.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.754+0000 7f041ffff700 1 -- 172.21.15.55:0/1093859570 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04201593b0 msgr2=0x7f0420159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:43.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.754+0000 7f041ffff700 1 --2- 172.21.15.55:0/1093859570 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04201593b0 0x7f0420159780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0410009910 tx=0x7f04100091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:43.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.754+0000 7f041ffff700 1 -- 172.21.15.55:0/1093859570 shutdown_connections 2023-10-06T05:07:43.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.754+0000 7f041ffff700 1 --2- 172.21.15.55:0/1093859570 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04201593b0 0x7f0420159780 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:43.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.754+0000 7f041ffff700 1 -- 172.21.15.55:0/1093859570 >> 172.21.15.55:0/1093859570 conn(0x7f04200d1130 msgr2=0x7f04200b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:43.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.754+0000 7f041ffff700 1 -- 172.21.15.55:0/1093859570 shutdown_connections 2023-10-06T05:07:43.757 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.754+0000 7f041ffff700 1 -- 172.21.15.55:0/1093859570 wait complete. 2023-10-06T05:07:43.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.755+0000 7f041ffff700 1 Processor -- start 2023-10-06T05:07:43.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.755+0000 7f041ffff700 1 -- start start 2023-10-06T05:07:43.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.756+0000 7f041ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04200b1e50 0x7f042015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:43.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.756+0000 7f041ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04200ceaa0 0x7f04200cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:43.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.756+0000 7f041ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04201593b0 0x7f042015e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:43.759 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.756+0000 7f041ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0420159780 con 0x7f04201593b0 2023-10-06T05:07:43.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.756+0000 7f041ffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04200c61e0 con 0x7f04200ceaa0 2023-10-06T05:07:43.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.756+0000 7f041ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04201599d0 con 0x7f04200b1e50 2023-10-06T05:07:43.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.757+0000 7f041f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04201593b0 0x7f042015e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:43.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.757+0000 7f041effd700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04200b1e50 0x7f042015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:43.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.757+0000 7f041f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04201593b0 0x7f042015e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52146/0 (socket says 172.21.15.55:52146) 2023-10-06T05:07:43.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.757+0000 7f041f7fe700 1 -- 172.21.15.55:0/2678977256 learned_addr learned my addr 172.21.15.55:0/2678977256 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:43.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.757+0000 7f0417fff700 1 --2- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04200ceaa0 0x7f04200cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:43.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.757+0000 7f041f7fe700 1 -- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04200b1e50 msgr2=0x7f042015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:43.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.757+0000 7f041f7fe700 1 --2- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04200b1e50 0x7f042015a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:43.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.757+0000 7f041f7fe700 1 -- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04200ceaa0 msgr2=0x7f04200cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:43.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.757+0000 7f041f7fe700 1 --2- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04200ceaa0 0x7f04200cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:43.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.757+0000 7f041f7fe700 1 -- 172.21.15.55:0/2678977256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0410009420 con 0x7f04201593b0 2023-10-06T05:07:43.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.757+0000 7f041f7fe700 1 --2- 172.21.15.55:0/2678977256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04201593b0 0x7f042015e680 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f041800cc40 tx=0x7f0418001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:43.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.758+0000 7f0416ffd700 1 -- 172.21.15.55:0/2678977256 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041800a870 con 0x7f04201593b0 2023-10-06T05:07:43.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.758+0000 7f0416ffd700 1 -- 172.21.15.55:0/2678977256 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041800a9d0 con 0x7f04201593b0 2023-10-06T05:07:43.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.758+0000 7f0416ffd700 1 -- 172.21.15.55:0/2678977256 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041800ad90 con 0x7f04201593b0 2023-10-06T05:07:43.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.758+0000 7f041ffff700 1 -- 172.21.15.55:0/2678977256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0420159cb0 con 0x7f04201593b0 2023-10-06T05:07:43.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.758+0000 7f041ffff700 1 -- 172.21.15.55:0/2678977256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04200cc220 con 0x7f04201593b0 2023-10-06T05:07:43.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.759+0000 7f041ffff700 1 -- 172.21.15.55:0/2678977256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042015a1d0 con 0x7f04201593b0 2023-10-06T05:07:43.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.760+0000 7f0416ffd700 1 -- 172.21.15.55:0/2678977256 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 105) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f0418007300 con 0x7f04201593b0 2023-10-06T05:07:43.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.761+0000 7f0416ffd700 1 --2- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f04000670e0 0x7f0400069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:43.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.761+0000 7f0416ffd700 1 -- 172.21.15.55:0/2678977256 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f0418084fd0 con 0x7f04201593b0 2023-10-06T05:07:43.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.761+0000 7f041effd700 1 -- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f04000670e0 msgr2=0x7f0400069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:43.763 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.761+0000 7f041effd700 1 --2- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f04000670e0 0x7f0400069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:43.766 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.765+0000 7f0416ffd700 1 -- 172.21.15.55:0/2678977256 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f042015a1d0 con 0x7f04201593b0 2023-10-06T05:07:43.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.962+0000 7f041effd700 1 -- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f04000670e0 msgr2=0x7f0400069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:43.964 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:43.962+0000 7f041effd700 1 --2- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f04000670e0 0x7f0400069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:44.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.029+0000 7f041ffff700 1 -- 172.21.15.55:0/2678977256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f042015a1d0 con 0x7f04201593b0 2023-10-06T05:07:44.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.030+0000 7f0416ffd700 1 -- 172.21.15.55:0/2678977256 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4616 (secure 0 0 0) 0x7f042015a1d0 con 0x7f04201593b0 2023-10-06T05:07:44.032 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:44.032 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3704044820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3704044820},{"type":"v1","addr":"172.21.15.163:6834","nonce":3704044820}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":9772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2254022733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2254022733},{"type":"v1","addr":"172.21.15.163:6835","nonce":2254022733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":10224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3844040929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3844040929},{"type":"v1","addr":"172.21.15.55:6834","nonce":3844040929}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:07:22.724188+0000","modified":"2023-10-06T05:07:26.255934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10227":{"gid":10227,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/3632968604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3632968604},{"type":"v1","addr":"172.21.15.55:6835","nonce":3632968604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:07:44.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.032+0000 7f041ffff700 1 -- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f04000670e0 msgr2=0x7f0400069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:44.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.032+0000 7f041ffff700 1 --2- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f04000670e0 0x7f0400069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.032+0000 7f041ffff700 1 -- 172.21.15.55:0/2678977256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04201593b0 msgr2=0x7f042015e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:44.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.032+0000 7f041ffff700 1 --2- 172.21.15.55:0/2678977256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04201593b0 0x7f042015e680 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f041800cc40 tx=0x7f0418001a60 comp rx=0 tx=0).stop 2023-10-06T05:07:44.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.032+0000 7f041ffff700 1 -- 172.21.15.55:0/2678977256 shutdown_connections 2023-10-06T05:07:44.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.032+0000 7f041ffff700 1 --2- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f04000670e0 0x7f0400069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.032+0000 7f041ffff700 1 --2- 172.21.15.55:0/2678977256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f04201593b0 0x7f042015e680 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.032+0000 7f041ffff700 1 --2- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f04200b1e50 0x7f042015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.035 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.032+0000 7f041ffff700 1 --2- 172.21.15.55:0/2678977256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f04200ceaa0 0x7f04200cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.032+0000 7f041ffff700 1 -- 172.21.15.55:0/2678977256 >> 172.21.15.55:0/2678977256 conn(0x7f04200d1130 msgr2=0x7f0420156f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:44.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.033+0000 7f041ffff700 1 -- 172.21.15.55:0/2678977256 shutdown_connections 2023-10-06T05:07:44.036 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.033+0000 7f041ffff700 1 -- 172.21.15.55:0/2678977256 wait complete. 2023-10-06T05:07:44.036 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 105 2023-10-06T05:07:44.053 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:07:44.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.243+0000 7fc87339d700 1 Processor -- start 2023-10-06T05:07:44.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.243+0000 7fc87339d700 1 -- start start 2023-10-06T05:07:44.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.244+0000 7fc87339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86c1593b0 0x7fc86c159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:44.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.244+0000 7fc87339d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fc86c0ceaa0 0x7fc86c0b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:44.249 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.244+0000 7fc87339d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc86c0b6570 con 0x7fc86c0b1740 2023-10-06T05:07:44.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.244+0000 7fc87339d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86c0cfe20 con 0x7fc86c0ceaa0 2023-10-06T05:07:44.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.244+0000 7fc87339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0cf210 con 0x7fc86c1593b0 2023-10-06T05:07:44.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.245+0000 7fc872b9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fc86c0b1740 0x7fc86c15d260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34110/0 (socket says 172.21.15.55:34110) 2023-10-06T05:07:44.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.245+0000 7fc872b9c700 1 -- 172.21.15.55:0/905019925 learned_addr learned my addr 172.21.15.55:0/905019925 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:44.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.245+0000 7fc87239b700 1 --2- 172.21.15.55:0/905019925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86c1593b0 0x7fc86c159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:44.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.245+0000 7fc871b9a700 1 --2- 172.21.15.55:0/905019925 >> v2:172.21.15.163:3300/0 conn(0x7fc86c0ceaa0 0x7fc86c0b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:44.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.245+0000 7fc871399700 1 -- 172.21.15.55:0/905019925 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677375955 0 0) 0x7fc864001490 con 0x7fc86c0b1740 2023-10-06T05:07:44.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.245+0000 7fc871399700 1 -- 172.21.15.55:0/905019925 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc858003570 con 0x7fc86c0b1740 2023-10-06T05:07:44.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.245+0000 7fc871b9a700 1 -- 172.21.15.55:0/905019925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86c1593b0 msgr2=0x7fc86c159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:44.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.245+0000 7fc871b9a700 1 --2- 172.21.15.55:0/905019925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86c1593b0 0x7fc86c159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.250 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.245+0000 7fc871b9a700 1 -- 172.21.15.55:0/905019925 >> v1:172.21.15.55:6789/0 conn(0x7fc86c0b1740 legacy=0x7fc86c15d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:44.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.245+0000 7fc871b9a700 1 -- 172.21.15.55:0/905019925 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c0cf4f0 con 0x7fc86c0ceaa0 2023-10-06T05:07:44.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.245+0000 7fc87239b700 1 --2- 172.21.15.55:0/905019925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86c1593b0 0x7fc86c159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:44.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.246+0000 7fc871b9a700 1 --2- 172.21.15.55:0/905019925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c0ceaa0 0x7fc86c0b6030 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0098e0 tx=0x7fc85c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a3a15c28dee9757 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:44.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.246+0000 7fc871399700 1 -- 172.21.15.55:0/905019925 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c011630 con 0x7fc86c0ceaa0 2023-10-06T05:07:44.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.246+0000 7fc871399700 1 -- 172.21.15.55:0/905019925 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c011790 con 0x7fc86c0ceaa0 2023-10-06T05:07:44.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.246+0000 7fc871399700 1 -- 172.21.15.55:0/905019925 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c011b30 con 0x7fc86c0ceaa0 2023-10-06T05:07:44.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.246+0000 7fc87339d700 1 -- 172.21.15.55:0/905019925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c0ceaa0 msgr2=0x7fc86c0b6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:44.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.246+0000 7fc87339d700 1 --2- 172.21.15.55:0/905019925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c0ceaa0 0x7fc86c0b6030 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0098e0 tx=0x7fc85c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.247+0000 7fc87339d700 1 -- 172.21.15.55:0/905019925 shutdown_connections 2023-10-06T05:07:44.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.247+0000 7fc87339d700 1 --2- 172.21.15.55:0/905019925 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86c1593b0 0x7fc86c159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.247+0000 7fc87339d700 1 --2- 172.21.15.55:0/905019925 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c0ceaa0 0x7fc86c0b6030 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.251 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.247+0000 7fc87339d700 1 -- 172.21.15.55:0/905019925 >> 172.21.15.55:0/905019925 conn(0x7fc86c0d1130 msgr2=0x7fc86c0b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:44.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.247+0000 7fc87339d700 1 -- 172.21.15.55:0/905019925 shutdown_connections 2023-10-06T05:07:44.252 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.247+0000 7fc87339d700 1 -- 172.21.15.55:0/905019925 wait complete. 2023-10-06T05:07:44.253 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.248+0000 7fc87339d700 1 Processor -- start 2023-10-06T05:07:44.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.248+0000 7fc87339d700 1 -- start start 2023-10-06T05:07:44.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.249+0000 7fc87339d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc86c0b1740 0x7fc86c0c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:44.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.249+0000 7fc87339d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86c0ceaa0 0x7fc86c068dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:44.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.249+0000 7fc87339d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c1593b0 0x7fc86c169e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:44.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.249+0000 7fc87339d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c16ae10 con 0x7fc86c0b1740 2023-10-06T05:07:44.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.249+0000 7fc87339d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c16a850 con 0x7fc86c1593b0 2023-10-06T05:07:44.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.249+0000 7fc87339d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c16ab30 con 0x7fc86c0ceaa0 2023-10-06T05:07:44.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.250+0000 7fc87239b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc86c0b1740 0x7fc86c0c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:44.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.250+0000 7fc87239b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc86c0b1740 0x7fc86c0c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52150/0 (socket says 172.21.15.55:52150) 2023-10-06T05:07:44.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.250+0000 7fc87239b700 1 -- 172.21.15.55:0/2624443862 learned_addr learned my addr 172.21.15.55:0/2624443862 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:44.254 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.250+0000 7fc871b9a700 1 --2- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86c0ceaa0 0x7fc86c068dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:44.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.250+0000 7fc872b9c700 1 --2- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c1593b0 0x7fc86c169e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:44.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.250+0000 7fc87239b700 1 -- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86c0ceaa0 msgr2=0x7fc86c068dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:44.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.250+0000 7fc87239b700 1 --2- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86c0ceaa0 0x7fc86c068dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.250+0000 7fc87239b700 1 -- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c1593b0 msgr2=0x7fc86c169e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:44.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.250+0000 7fc87239b700 1 --2- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c1593b0 0x7fc86c169e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.250+0000 7fc87239b700 1 -- 172.21.15.55:0/2624443862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc868009420 con 0x7fc86c0b1740 2023-10-06T05:07:44.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.250+0000 7fc87239b700 1 --2- 172.21.15.55:0/2624443862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc86c0b1740 0x7fc86c0c5da0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fc86800ec80 tx=0x7fc8680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:44.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.250+0000 7fc8627fc700 1 -- 172.21.15.55:0/2624443862 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86800c850 con 0x7fc86c0b1740 2023-10-06T05:07:44.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.251+0000 7fc8627fc700 1 -- 172.21.15.55:0/2624443862 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86800c9b0 con 0x7fc86c0b1740 2023-10-06T05:07:44.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.251+0000 7fc8627fc700 1 -- 172.21.15.55:0/2624443862 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86800cd30 con 0x7fc86c0b1740 2023-10-06T05:07:44.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.251+0000 7fc87339d700 1 -- 172.21.15.55:0/2624443862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c009420 con 0x7fc86c0b1740 2023-10-06T05:07:44.255 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.251+0000 7fc87339d700 1 -- 172.21.15.55:0/2624443862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c0c5880 con 0x7fc86c0b1740 2023-10-06T05:07:44.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.252+0000 7fc87339d700 1 -- 172.21.15.55:0/2624443862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c0c5bf0 con 0x7fc86c0b1740 2023-10-06T05:07:44.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.253+0000 7fc8627fc700 1 -- 172.21.15.55:0/2624443862 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 105) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fc868013680 con 0x7fc86c0b1740 2023-10-06T05:07:44.256 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.254+0000 7fc8627fc700 1 --2- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc8500670e0 0x7fc850069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:44.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.254+0000 7fc8627fc700 1 -- 172.21.15.55:0/2624443862 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fc868084eb0 con 0x7fc86c0b1740 2023-10-06T05:07:44.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.254+0000 7fc871b9a700 1 -- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc8500670e0 msgr2=0x7fc850069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:44.257 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.254+0000 7fc871b9a700 1 --2- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc8500670e0 0x7fc850069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:44.259 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.258+0000 7fc8627fc700 1 -- 172.21.15.55:0/2624443862 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc868055510 con 0x7fc86c0b1740 2023-10-06T05:07:44.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.455+0000 7fc871b9a700 1 -- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc8500670e0 msgr2=0x7fc850069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:44.457 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.455+0000 7fc871b9a700 1 --2- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc8500670e0 0x7fc850069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:44.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.464+0000 7fc87339d700 1 -- 172.21.15.55:0/2624443862 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc86c0c5400 con 0x7fc86c0b1740 2023-10-06T05:07:44.466 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.465+0000 7fc8627fc700 1 -- 172.21.15.55:0/2624443862 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v208) v1 ==== 74+0+17989 (secure 0 0 0) 0x7fc86c0c5400 con 0x7fc86c0b1740 2023-10-06T05:07:44.467 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:44.467 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":208,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:07:39.103933+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:07:20.916360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:07:21.706951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"207","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/2709454586":"2023-10-07T05:07:39.103885+0000","172.21.15.163:6837/101089":"2023-10-07T05:07:39.103885+0000","172.21.15.163:6836/101089":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/2719429847":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/3618319475":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/2222063413":"2023-10-07T05:07:39.103885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:07:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.467+0000 7fc87339d700 1 -- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc8500670e0 msgr2=0x7fc850069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.467+0000 7fc87339d700 1 --2- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc8500670e0 0x7fc850069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.467+0000 7fc87339d700 1 -- 172.21.15.55:0/2624443862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc86c0b1740 msgr2=0x7fc86c0c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.467+0000 7fc87339d700 1 --2- 172.21.15.55:0/2624443862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc86c0b1740 0x7fc86c0c5da0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fc86800ec80 tx=0x7fc8680091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.467+0000 7fc87339d700 1 -- 172.21.15.55:0/2624443862 shutdown_connections 2023-10-06T05:07:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.468+0000 7fc87339d700 1 --2- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc8500670e0 0x7fc850069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.468+0000 7fc87339d700 1 --2- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc86c1593b0 0x7fc86c169e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.468+0000 7fc87339d700 1 --2- 172.21.15.55:0/2624443862 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc86c0b1740 0x7fc86c0c5da0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.468+0000 7fc87339d700 1 --2- 172.21.15.55:0/2624443862 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc86c0ceaa0 0x7fc86c068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.470 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.468+0000 7fc87339d700 1 -- 172.21.15.55:0/2624443862 >> 172.21.15.55:0/2624443862 conn(0x7fc86c0d1130 msgr2=0x7fc86c15e960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:44.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.468+0000 7fc87339d700 1 -- 172.21.15.55:0/2624443862 shutdown_connections 2023-10-06T05:07:44.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.468+0000 7fc87339d700 1 -- 172.21.15.55:0/2624443862 wait complete. 2023-10-06T05:07:44.488 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-10-06T03:33:08.008847+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-10-06T03:33:08.549094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T05:07:20.916360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '206', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T05:07:21.706951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '207', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T05:07:44.489 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T05:07:44.489 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:44.488+0000 7f562514f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:07:44.568 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:44.567+0000 7f1696d46180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T05:07:44.653 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:44.651+0000 7f562514f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:07:44.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.682+0000 7fb00759d700 1 Processor -- start 2023-10-06T05:07:44.684 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.682+0000 7fb00759d700 1 -- start start 2023-10-06T05:07:44.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.683+0000 7fb00759d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0001407e0 0x7fb00006a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:44.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.683+0000 7fb00759d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb000140bb0 0x7fb00006a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:44.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.683+0000 7fb00759d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb00006d4f0 0x7fb00006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:44.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.683+0000 7fb00759d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0000c6820 con 0x7fb00006d4f0 2023-10-06T05:07:44.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.683+0000 7fb00759d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0000c6de0 con 0x7fb000140bb0 2023-10-06T05:07:44.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.683+0000 7fb00759d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00006de70 con 0x7fb0001407e0 2023-10-06T05:07:44.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.684+0000 7fb006d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb00006d4f0 0x7fb00006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:44.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.684+0000 7fb005d9a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb000140bb0 0x7fb00006a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:44.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.684+0000 7fb006d9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb00006d4f0 0x7fb00006d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52160/0 (socket says 172.21.15.55:52160) 2023-10-06T05:07:44.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.684+0000 7fb006d9c700 1 -- 172.21.15.55:0/2239293360 learned_addr learned my addr 172.21.15.55:0/2239293360 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:44.686 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.684+0000 7fb00659b700 1 --2- 172.21.15.55:0/2239293360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0001407e0 0x7fb00006a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.684+0000 7fb006d9c700 1 -- 172.21.15.55:0/2239293360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0001407e0 msgr2=0x7fb00006a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.684+0000 7fb006d9c700 1 --2- 172.21.15.55:0/2239293360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0001407e0 0x7fb00006a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.684+0000 7fb006d9c700 1 -- 172.21.15.55:0/2239293360 >> v2:172.21.15.163:3300/0 conn(0x7fb000140bb0 msgr2=0x7fb00006a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.684+0000 7fb006d9c700 1 --2- 172.21.15.55:0/2239293360 >> v2:172.21.15.163:3300/0 conn(0x7fb000140bb0 0x7fb00006a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.684+0000 7fb006d9c700 1 -- 172.21.15.55:0/2239293360 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb000141270 con 0x7fb00006d4f0 2023-10-06T05:07:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.685+0000 7fb006d9c700 1 --2- 172.21.15.55:0/2239293360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00006d4f0 0x7fb00006d930 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb00006d1e0 tx=0x7faff8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103811a8613ea67f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:44.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.685+0000 7fb005599700 1 -- 172.21.15.55:0/2239293360 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff800e070 con 0x7fb00006d4f0 2023-10-06T05:07:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.685+0000 7fb005599700 1 -- 172.21.15.55:0/2239293360 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff800b970 con 0x7fb00006d4f0 2023-10-06T05:07:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.685+0000 7fb005599700 1 -- 172.21.15.55:0/2239293360 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff80092e0 con 0x7fb00006d4f0 2023-10-06T05:07:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.685+0000 7fb00759d700 1 -- 172.21.15.55:0/2239293360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00006d4f0 msgr2=0x7fb00006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.685+0000 7fb00759d700 1 --2- 172.21.15.55:0/2239293360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00006d4f0 0x7fb00006d930 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fb00006d1e0 tx=0x7faff8009840 comp rx=0 tx=0).stop 2023-10-06T05:07:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.686+0000 7fb00759d700 1 -- 172.21.15.55:0/2239293360 shutdown_connections 2023-10-06T05:07:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.686+0000 7fb00759d700 1 --2- 172.21.15.55:0/2239293360 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00006d4f0 0x7fb00006d930 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.686+0000 7fb00759d700 1 --2- 172.21.15.55:0/2239293360 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb0001407e0 0x7fb00006a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.686+0000 7fb00759d700 1 --2- 172.21.15.55:0/2239293360 >> v2:172.21.15.163:3300/0 conn(0x7fb000140bb0 0x7fb00006a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.686+0000 7fb00759d700 1 -- 172.21.15.55:0/2239293360 >> 172.21.15.55:0/2239293360 conn(0x7fb0000cc1e0 msgr2=0x7fb0000cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.686+0000 7fb00759d700 1 -- 172.21.15.55:0/2239293360 shutdown_connections 2023-10-06T05:07:44.689 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.686+0000 7fb00759d700 1 -- 172.21.15.55:0/2239293360 wait complete. 2023-10-06T05:07:44.691 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.687+0000 7fb00759d700 1 Processor -- start 2023-10-06T05:07:44.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.687+0000 7fb00759d700 1 -- start start 2023-10-06T05:07:44.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.688+0000 7fb00759d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00006d4f0 0x7fb00015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:44.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.688+0000 7fb00759d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0001407e0 0x7fb00015e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:44.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.688+0000 7fb00759d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb000140bb0 0x7fb00013bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:44.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.688+0000 7fb00759d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb000160340 con 0x7fb00006d4f0 2023-10-06T05:07:44.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.688+0000 7fb00759d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0000aff10 con 0x7fb0001407e0 2023-10-06T05:07:44.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.688+0000 7fb00759d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00015e0b0 con 0x7fb000140bb0 2023-10-06T05:07:44.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.688+0000 7fb00659b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00006d4f0 0x7fb00015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:44.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.689+0000 7fb00659b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00006d4f0 0x7fb00015d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52176/0 (socket says 172.21.15.55:52176) 2023-10-06T05:07:44.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.689+0000 7fb00659b700 1 -- 172.21.15.55:0/463739021 learned_addr learned my addr 172.21.15.55:0/463739021 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:44.692 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.689+0000 7fb005d9a700 1 --2- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0001407e0 0x7fb00015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:44.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.689+0000 7fb006d9c700 1 --2- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb000140bb0 0x7fb00013bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:44.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.689+0000 7fb00659b700 1 -- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb000140bb0 msgr2=0x7fb00013bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:44.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.689+0000 7fb00659b700 1 --2- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb000140bb0 0x7fb00013bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.689+0000 7fb00659b700 1 -- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0001407e0 msgr2=0x7fb00015e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:44.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.689+0000 7fb00659b700 1 --2- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0001407e0 0x7fb00015e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.689+0000 7fb00659b700 1 -- 172.21.15.55:0/463739021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff8009ae0 con 0x7fb00006d4f0 2023-10-06T05:07:44.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.689+0000 7fb00659b700 1 --2- 172.21.15.55:0/463739021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00006d4f0 0x7fb00015d3d0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7faffc00ec80 tx=0x7faffc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:44.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.689+0000 7faff67fc700 1 -- 172.21.15.55:0/463739021 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc00c850 con 0x7fb00006d4f0 2023-10-06T05:07:44.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.689+0000 7faff67fc700 1 -- 172.21.15.55:0/463739021 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc00c9d0 con 0x7fb00006d4f0 2023-10-06T05:07:44.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.690+0000 7faff67fc700 1 -- 172.21.15.55:0/463739021 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc00cd90 con 0x7fb00006d4f0 2023-10-06T05:07:44.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.690+0000 7fb00759d700 1 -- 172.21.15.55:0/463739021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00013c4f0 con 0x7fb00006d4f0 2023-10-06T05:07:44.693 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.690+0000 7fb00759d700 1 -- 172.21.15.55:0/463739021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb000160800 con 0x7fb00006d4f0 2023-10-06T05:07:44.694 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.691+0000 7fb00759d700 1 -- 172.21.15.55:0/463739021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0000b0060 con 0x7fb00006d4f0 2023-10-06T05:07:44.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.692+0000 7faff67fc700 1 -- 172.21.15.55:0/463739021 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 105) v1 ==== 84899+0+0 (secure 0 0 0) 0x7faffc013680 con 0x7fb00006d4f0 2023-10-06T05:07:44.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.692+0000 7faff67fc700 1 --2- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fafe40670e0 0x7fafe4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:44.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.692+0000 7faff67fc700 1 -- 172.21.15.55:0/463739021 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7faffc084fa0 con 0x7fb00006d4f0 2023-10-06T05:07:44.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.693+0000 7fb005d9a700 1 -- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fafe40670e0 msgr2=0x7fafe4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:44.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.693+0000 7fb005d9a700 1 --2- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fafe40670e0 0x7fafe4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:44.698 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.697+0000 7faff67fc700 1 -- 172.21.15.55:0/463739021 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb0000b0060 con 0x7fb00006d4f0 2023-10-06T05:07:44.739 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:44.738+0000 7f562514f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:07:44.744 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:44.742+0000 7f1696d46180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T05:07:44.833 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:44.831+0000 7f1696d46180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T05:07:44.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.894+0000 7fb005d9a700 1 -- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fafe40670e0 msgr2=0x7fafe4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:44.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.894+0000 7fb005d9a700 1 --2- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fafe40670e0 0x7fafe4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:44.900 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.899+0000 7fb00759d700 1 -- 172.21.15.55:0/463739021 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb0000b0060 con 0x7fb00006d4f0 2023-10-06T05:07:44.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.899+0000 7faff67fc700 1 -- 172.21.15.55:0/463739021 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v208) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb0000b0060 con 0x7fb00006d4f0 2023-10-06T05:07:44.901 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 8 2023-10-06T05:07:44.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.902+0000 7fb00759d700 1 -- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fafe40670e0 msgr2=0x7fafe4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:44.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.902+0000 7fb00759d700 1 --2- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fafe40670e0 0x7fafe4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.904 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.902+0000 7fb00759d700 1 -- 172.21.15.55:0/463739021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00006d4f0 msgr2=0x7fb00015d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.902+0000 7fb00759d700 1 --2- 172.21.15.55:0/463739021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00006d4f0 0x7fb00015d3d0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7faffc00ec80 tx=0x7faffc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.902+0000 7fb00759d700 1 -- 172.21.15.55:0/463739021 shutdown_connections 2023-10-06T05:07:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.902+0000 7fb00759d700 1 --2- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fafe40670e0 0x7fafe4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.902+0000 7fb00759d700 1 --2- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb000140bb0 0x7fb00013bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.902+0000 7fb00759d700 1 --2- 172.21.15.55:0/463739021 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb00006d4f0 0x7fb00015d3d0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.902+0000 7fb00759d700 1 --2- 172.21.15.55:0/463739021 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb0001407e0 0x7fb00015e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:44.905 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.902+0000 7fb00759d700 1 -- 172.21.15.55:0/463739021 >> 172.21.15.55:0/463739021 conn(0x7fb0000cc1e0 msgr2=0x7fb0000c21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:44.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.902+0000 7fb00759d700 1 -- 172.21.15.55:0/463739021 shutdown_connections 2023-10-06T05:07:44.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:44.902+0000 7fb00759d700 1 -- 172.21.15.55:0/463739021 wait complete. 2023-10-06T05:07:44.923 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T05:07:44.966 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:44.964+0000 7f562514f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:07:45.070 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:45.069+0000 7f1696d46180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:07:45.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.113+0000 7f10d49fe700 1 Processor -- start 2023-10-06T05:07:45.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.113+0000 7f10d49fe700 1 -- start start 2023-10-06T05:07:45.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.116+0000 7f10d49fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10d00b16d0 0x7f10d00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:45.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.116+0000 7f10d49fe700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f10d00b33b0 0x7f10d00b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:45.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.116+0000 7f10d49fe700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d00b2080 con 0x7f10d00b4e70 2023-10-06T05:07:45.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.116+0000 7f10d49fe700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d00cfb00 con 0x7f10d00b33b0 2023-10-06T05:07:45.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.116+0000 7f10d49fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00b5c40 con 0x7f10d00b16d0 2023-10-06T05:07:45.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.116+0000 7f10ced9d700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f10d00b4e70 0x7f10d00ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34126/0 (socket says 172.21.15.55:34126) 2023-10-06T05:07:45.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.116+0000 7f10ced9d700 1 -- 172.21.15.55:0/1252102433 learned_addr learned my addr 172.21.15.55:0/1252102433 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:45.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.116+0000 7f10ce59c700 1 --2- 172.21.15.55:0/1252102433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10d00b16d0 0x7f10d00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:45.119 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.116+0000 7f10cf59e700 1 --2- 172.21.15.55:0/1252102433 >> v2:172.21.15.163:3300/0 conn(0x7f10d00b33b0 0x7f10d00b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:45.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.116+0000 7f10cdd9b700 1 -- 172.21.15.55:0/1252102433 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584587993 0 0) 0x7f10d00b2080 con 0x7f10d00b4e70 2023-10-06T05:07:45.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.117+0000 7f10cdd9b700 1 -- 172.21.15.55:0/1252102433 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10b8003570 con 0x7f10d00b4e70 2023-10-06T05:07:45.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.117+0000 7f10cf59e700 1 -- 172.21.15.55:0/1252102433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10d00b16d0 msgr2=0x7f10d00b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:45.120 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.117+0000 7f10cf59e700 1 --2- 172.21.15.55:0/1252102433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10d00b16d0 0x7f10d00b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.117+0000 7f10cf59e700 1 -- 172.21.15.55:0/1252102433 >> v1:172.21.15.55:6789/0 conn(0x7f10d00b4e70 legacy=0x7f10d00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:45.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.117+0000 7f10cf59e700 1 -- 172.21.15.55:0/1252102433 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d00b5f20 con 0x7f10d00b33b0 2023-10-06T05:07:45.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.117+0000 7f10ce59c700 1 --2- 172.21.15.55:0/1252102433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10d00b16d0 0x7f10d00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:45.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.117+0000 7f10cf59e700 1 --2- 172.21.15.55:0/1252102433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10d00b33b0 0x7f10d00b3830 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009e60 tx=0x7f10c8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2645495284c1a5e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:45.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.117+0000 7f10cdd9b700 1 -- 172.21.15.55:0/1252102433 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c800e070 con 0x7f10d00b33b0 2023-10-06T05:07:45.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.117+0000 7f10cdd9b700 1 -- 172.21.15.55:0/1252102433 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c800bab0 con 0x7f10d00b33b0 2023-10-06T05:07:45.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.117+0000 7f10cdd9b700 1 -- 172.21.15.55:0/1252102433 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c800be50 con 0x7f10d00b33b0 2023-10-06T05:07:45.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.118+0000 7f10d49fe700 1 -- 172.21.15.55:0/1252102433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10d00b33b0 msgr2=0x7f10d00b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:45.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.118+0000 7f10d49fe700 1 --2- 172.21.15.55:0/1252102433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10d00b33b0 0x7f10d00b3830 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009e60 tx=0x7f10c8009780 comp rx=0 tx=0).stop 2023-10-06T05:07:45.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.118+0000 7f10d49fe700 1 -- 172.21.15.55:0/1252102433 shutdown_connections 2023-10-06T05:07:45.121 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.118+0000 7f10d49fe700 1 --2- 172.21.15.55:0/1252102433 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10d00b33b0 0x7f10d00b3830 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.118+0000 7f10d49fe700 1 --2- 172.21.15.55:0/1252102433 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10d00b16d0 0x7f10d00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.118+0000 7f10d49fe700 1 -- 172.21.15.55:0/1252102433 >> 172.21.15.55:0/1252102433 conn(0x7f10d00d10c0 msgr2=0x7f10d00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:45.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.118+0000 7f10d49fe700 1 -- 172.21.15.55:0/1252102433 shutdown_connections 2023-10-06T05:07:45.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.118+0000 7f10d49fe700 1 -- 172.21.15.55:0/1252102433 wait complete. 2023-10-06T05:07:45.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.121+0000 7f10d49fe700 1 Processor -- start 2023-10-06T05:07:45.123 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.121+0000 7f10d49fe700 1 -- start start 2023-10-06T05:07:45.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.122+0000 7f10d49fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10d00b16d0 0x7f10d0159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:45.127 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.122+0000 7f10d49fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10d00b33b0 0x7f10d0169bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:45.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10d49fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10d00b4e70 0x7f10d00caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:45.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10d49fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00ca080 con 0x7f10d00b4e70 2023-10-06T05:07:45.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10d49fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00c83c0 con 0x7f10d00b16d0 2023-10-06T05:07:45.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10d49fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d01597c0 con 0x7f10d00b33b0 2023-10-06T05:07:45.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10cf59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10d00b4e70 0x7f10d00caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:45.128 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10cf59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10d00b4e70 0x7f10d00caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52184/0 (socket says 172.21.15.55:52184) 2023-10-06T05:07:45.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10cf59e700 1 -- 172.21.15.55:0/3858897893 learned_addr learned my addr 172.21.15.55:0/3858897893 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:45.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10ced9d700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10d00b16d0 0x7f10d0159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:45.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10ce59c700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10d00b33b0 0x7f10d0169bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:45.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10ced9d700 1 -- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10d00b33b0 msgr2=0x7f10d0169bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:45.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10ced9d700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10d00b33b0 0x7f10d0169bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10ced9d700 1 -- 172.21.15.55:0/3858897893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10d00b4e70 msgr2=0x7f10d00caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:45.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10ced9d700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10d00b4e70 0x7f10d00caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10ced9d700 1 -- 172.21.15.55:0/3858897893 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c4009420 con 0x7f10d00b16d0 2023-10-06T05:07:45.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.123+0000 7f10cf59e700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10d00b4e70 0x7f10d00caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:45.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.124+0000 7f10ced9d700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10d00b16d0 0x7f10d0159280 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f10c000c5b0 tx=0x7f10c0007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:45.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.124+0000 7f10b6ffd700 1 -- 172.21.15.55:0/3858897893 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0005b30 con 0x7f10d00b16d0 2023-10-06T05:07:45.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.124+0000 7f10b6ffd700 1 -- 172.21.15.55:0/3858897893 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c0005cb0 con 0x7f10d00b16d0 2023-10-06T05:07:45.130 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.124+0000 7f10b6ffd700 1 -- 172.21.15.55:0/3858897893 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0007320 con 0x7f10d00b16d0 2023-10-06T05:07:45.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.124+0000 7f10d49fe700 1 -- 172.21.15.55:0/3858897893 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c8009a20 con 0x7f10d00b16d0 2023-10-06T05:07:45.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.124+0000 7f10d49fe700 1 -- 172.21.15.55:0/3858897893 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d00c6270 con 0x7f10d00b16d0 2023-10-06T05:07:45.131 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.125+0000 7f10d49fe700 1 -- 172.21.15.55:0/3858897893 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d0088990 con 0x7f10d00b16d0 2023-10-06T05:07:45.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.126+0000 7f10b6ffd700 1 -- 172.21.15.55:0/3858897893 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 105) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f10c001b3b0 con 0x7f10d00b16d0 2023-10-06T05:07:45.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.127+0000 7f10b6ffd700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f10bc0670e0 0x7f10bc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:45.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.127+0000 7f10b6ffd700 1 -- 172.21.15.55:0/3858897893 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f10c0084020 con 0x7f10d00b16d0 2023-10-06T05:07:45.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.127+0000 7f10ce59c700 1 -- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f10bc0670e0 msgr2=0x7f10bc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:45.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.127+0000 7f10ce59c700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f10bc0670e0 0x7f10bc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:45.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.132+0000 7f10b6ffd700 1 -- 172.21.15.55:0/3858897893 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f10c0050bd0 con 0x7f10d00b16d0 2023-10-06T05:07:45.146 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:45.144+0000 7f562514f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:07:45.240 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:45.239+0000 7f562514f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:07:45.263 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:45.261+0000 7f1696d46180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T05:07:45.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.328+0000 7f10ce59c700 1 -- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f10bc0670e0 msgr2=0x7f10bc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:45.330 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.328+0000 7f10ce59c700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f10bc0670e0 0x7f10bc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:45.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.336+0000 7f10d49fe700 1 -- 172.21.15.55:0/3858897893 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10d004bb80 con 0x7f10d00b16d0 2023-10-06T05:07:45.338 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.337+0000 7f10b6ffd700 1 -- 172.21.15.55:0/3858897893 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v208) v1 ==== 93+0+10 (secure 0 0 0) 0x7f10c00170a0 con 0x7f10d00b16d0 2023-10-06T05:07:45.339 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2023-10-06T05:07:45.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.339+0000 7f10d49fe700 1 -- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f10bc0670e0 msgr2=0x7f10bc069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:45.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.339+0000 7f10d49fe700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f10bc0670e0 0x7f10bc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.341 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.339+0000 7f10d49fe700 1 -- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10d00b16d0 msgr2=0x7f10d0159280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.339+0000 7f10d49fe700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10d00b16d0 0x7f10d0159280 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f10c000c5b0 tx=0x7f10c0007d50 comp rx=0 tx=0).stop 2023-10-06T05:07:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.339+0000 7f10d49fe700 1 -- 172.21.15.55:0/3858897893 shutdown_connections 2023-10-06T05:07:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.339+0000 7f10d49fe700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f10bc0670e0 0x7f10bc069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.339+0000 7f10d49fe700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f10d00b4e70 0x7f10d00caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.339+0000 7f10d49fe700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f10d00b16d0 0x7f10d0159280 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.339+0000 7f10d49fe700 1 --2- 172.21.15.55:0/3858897893 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f10d00b33b0 0x7f10d0169bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.339+0000 7f10d49fe700 1 -- 172.21.15.55:0/3858897893 >> 172.21.15.55:0/3858897893 conn(0x7f10d00d10c0 msgr2=0x7f10d015f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.339+0000 7f10d49fe700 1 -- 172.21.15.55:0/3858897893 shutdown_connections 2023-10-06T05:07:45.342 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.339+0000 7f10d49fe700 1 -- 172.21.15.55:0/3858897893 wait complete. 2023-10-06T05:07:45.359 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T05:07:45.365 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:45.361+0000 7f1696d46180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T05:07:45.403 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:45.402+0000 7f562514f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:07:45.504 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:45.502+0000 7f562514f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:07:45.534 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:45.532+0000 7f1696d46180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T05:07:45.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.554+0000 7f356ef9d700 1 Processor -- start 2023-10-06T05:07:45.556 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.554+0000 7f356ef9d700 1 -- start start 2023-10-06T05:07:45.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.555+0000 7f356ef9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f35680b16d0 0x7f35680b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:45.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.555+0000 7f356ef9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f35680b33b0 0x7f35680b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:45.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.555+0000 7f356ef9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35680b2080 con 0x7f35680b4e70 2023-10-06T05:07:45.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.555+0000 7f356ef9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35680cfb00 con 0x7f35680b33b0 2023-10-06T05:07:45.557 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.555+0000 7f356ef9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35680b5c40 con 0x7f35680b16d0 2023-10-06T05:07:45.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.556+0000 7f356e79c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f35680b33b0 0x7f35680b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:45.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.556+0000 7f356e79c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f35680b33b0 0x7f35680b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52200/0 (socket says 172.21.15.55:52200) 2023-10-06T05:07:45.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.556+0000 7f356e79c700 1 -- 172.21.15.55:0/822616124 learned_addr learned my addr 172.21.15.55:0/822616124 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:45.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.556+0000 7f356d79a700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f35680b16d0 0x7f35680b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:45.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.556+0000 7f356e79c700 1 -- 172.21.15.55:0/822616124 >> v2:172.21.15.163:3300/0 conn(0x7f35680b16d0 msgr2=0x7f35680b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:45.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.556+0000 7f356e79c700 1 --2- 172.21.15.55:0/822616124 >> v2:172.21.15.163:3300/0 conn(0x7f35680b16d0 0x7f35680b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.556+0000 7f356e79c700 1 -- 172.21.15.55:0/822616124 >> v1:172.21.15.163:6789/0 conn(0x7f35680b4e70 legacy=0x7f35680ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:45.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.556+0000 7f356e79c700 1 -- 172.21.15.55:0/822616124 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35680b5f20 con 0x7f35680b33b0 2023-10-06T05:07:45.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.556+0000 7f356e79c700 1 --2- 172.21.15.55:0/822616124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35680b33b0 0x7f35680b37d0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f3568089830 tx=0x7f3560009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96afd6ff09db08d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:45.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.557+0000 7f356cf99700 1 -- 172.21.15.55:0/822616124 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356000e070 con 0x7f35680b33b0 2023-10-06T05:07:45.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.557+0000 7f356cf99700 1 -- 172.21.15.55:0/822616124 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356000ba20 con 0x7f35680b33b0 2023-10-06T05:07:45.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.557+0000 7f356cf99700 1 -- 172.21.15.55:0/822616124 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35600092e0 con 0x7f35680b33b0 2023-10-06T05:07:45.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.557+0000 7f356ef9d700 1 -- 172.21.15.55:0/822616124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35680b33b0 msgr2=0x7f35680b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:45.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.557+0000 7f356ef9d700 1 --2- 172.21.15.55:0/822616124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35680b33b0 0x7f35680b37d0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f3568089830 tx=0x7f3560009840 comp rx=0 tx=0).stop 2023-10-06T05:07:45.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.557+0000 7f356ef9d700 1 -- 172.21.15.55:0/822616124 shutdown_connections 2023-10-06T05:07:45.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.557+0000 7f356ef9d700 1 --2- 172.21.15.55:0/822616124 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35680b33b0 0x7f35680b37d0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.557+0000 7f356ef9d700 1 --2- 172.21.15.55:0/822616124 >> v2:172.21.15.163:3300/0 conn(0x7f35680b16d0 0x7f35680b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.558+0000 7f356ef9d700 1 -- 172.21.15.55:0/822616124 >> 172.21.15.55:0/822616124 conn(0x7f35680d10c0 msgr2=0x7f35680affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:45.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.558+0000 7f356ef9d700 1 -- 172.21.15.55:0/822616124 shutdown_connections 2023-10-06T05:07:45.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.558+0000 7f356ef9d700 1 -- 172.21.15.55:0/822616124 wait complete. 2023-10-06T05:07:45.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.559+0000 7f356ef9d700 1 Processor -- start 2023-10-06T05:07:45.562 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.559+0000 7f356ef9d700 1 -- start start 2023-10-06T05:07:45.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.560+0000 7f356ef9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35680b16d0 0x7f35680af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:45.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.560+0000 7f356ef9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35680b33b0 0x7f35680c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:45.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.560+0000 7f356ef9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35680b4e70 0x7f356816d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:45.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.560+0000 7f356ef9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35680ccc00 con 0x7f35680b4e70 2023-10-06T05:07:45.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.560+0000 7f356ef9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35680cc770 con 0x7f35680b33b0 2023-10-06T05:07:45.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.560+0000 7f356ef9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35680cc440 con 0x7f35680b16d0 2023-10-06T05:07:45.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.560+0000 7f356d79a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35680b33b0 0x7f35680c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:45.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.560+0000 7f356e79c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35680b4e70 0x7f356816d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:45.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.560+0000 7f356d79a700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35680b33b0 0x7f35680c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:48570/0 (socket says 172.21.15.55:48570) 2023-10-06T05:07:45.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.560+0000 7f356d79a700 1 -- 172.21.15.55:0/1802543863 learned_addr learned my addr 172.21.15.55:0/1802543863 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:45.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.560+0000 7f356df9b700 1 --2- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35680b16d0 0x7f35680af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:45.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.561+0000 7f356d79a700 1 -- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35680b16d0 msgr2=0x7f35680af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:45.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.561+0000 7f356d79a700 1 --2- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35680b16d0 0x7f35680af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.561+0000 7f356d79a700 1 -- 172.21.15.55:0/1802543863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35680b4e70 msgr2=0x7f356816d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:45.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.561+0000 7f356d79a700 1 --2- 172.21.15.55:0/1802543863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35680b4e70 0x7f356816d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.561+0000 7f356d79a700 1 -- 172.21.15.55:0/1802543863 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3560009ae0 con 0x7f35680b33b0 2023-10-06T05:07:45.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.561+0000 7f356d79a700 1 --2- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35680b33b0 0x7f35680c9e80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f355800e950 tx=0x7f35580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:45.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.561+0000 7f355dffb700 1 -- 172.21.15.55:0/1802543863 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558013680 con 0x7f35680b33b0 2023-10-06T05:07:45.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.561+0000 7f355dffb700 1 -- 172.21.15.55:0/1802543863 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35580137e0 con 0x7f35680b33b0 2023-10-06T05:07:45.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.561+0000 7f355dffb700 1 -- 172.21.15.55:0/1802543863 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558013ba0 con 0x7f35680b33b0 2023-10-06T05:07:45.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.562+0000 7f356ef9d700 1 -- 172.21.15.55:0/1802543863 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35680cb350 con 0x7f35680b33b0 2023-10-06T05:07:45.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.562+0000 7f356ef9d700 1 -- 172.21.15.55:0/1802543863 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35680cb910 con 0x7f35680b33b0 2023-10-06T05:07:45.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.563+0000 7f356ef9d700 1 -- 172.21.15.55:0/1802543863 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3568088990 con 0x7f35680b33b0 2023-10-06T05:07:45.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.564+0000 7f355dffb700 1 -- 172.21.15.55:0/1802543863 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 105) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f35580123d0 con 0x7f35680b33b0 2023-10-06T05:07:45.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.565+0000 7f355dffb700 1 --2- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f354c0670e0 0x7f354c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:45.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.565+0000 7f355dffb700 1 -- 172.21.15.55:0/1802543863 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f3558084e30 con 0x7f35680b33b0 2023-10-06T05:07:45.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.565+0000 7f356df9b700 1 -- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f354c0670e0 msgr2=0x7f354c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:45.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.565+0000 7f356df9b700 1 --2- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f354c0670e0 0x7f354c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:45.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.569+0000 7f355dffb700 1 -- 172.21.15.55:0/1802543863 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3558051960 con 0x7f35680b33b0 2023-10-06T05:07:45.642 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:45.640+0000 7f1696d46180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T05:07:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.766+0000 7f356df9b700 1 -- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f354c0670e0 msgr2=0x7f354c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:45.768 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.766+0000 7f356df9b700 1 --2- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f354c0670e0 0x7f354c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:45.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.771+0000 7f356ef9d700 1 -- 172.21.15.55:0/1802543863 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35680cc8b0 con 0x7f35680b33b0 2023-10-06T05:07:45.774 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.772+0000 7f355dffb700 1 -- 172.21.15.55:0/1802543863 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v208) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3558017020 con 0x7f35680b33b0 2023-10-06T05:07:45.774 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 32 2023-10-06T05:07:45.776 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.774+0000 7f356ef9d700 1 -- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f354c0670e0 msgr2=0x7f354c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:45.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.774+0000 7f356ef9d700 1 --2- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f354c0670e0 0x7f354c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.774+0000 7f356ef9d700 1 -- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35680b33b0 msgr2=0x7f35680c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:45.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.774+0000 7f356ef9d700 1 --2- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35680b33b0 0x7f35680c9e80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f355800e950 tx=0x7f35580091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.774+0000 7f356ef9d700 1 -- 172.21.15.55:0/1802543863 shutdown_connections 2023-10-06T05:07:45.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.774+0000 7f356ef9d700 1 --2- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f354c0670e0 0x7f354c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.774+0000 7f356ef9d700 1 --2- 172.21.15.55:0/1802543863 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f35680b4e70 0x7f356816d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.774+0000 7f356ef9d700 1 --2- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f35680b16d0 0x7f35680af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.774+0000 7f356ef9d700 1 --2- 172.21.15.55:0/1802543863 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f35680b33b0 0x7f35680c9e80 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:45.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.774+0000 7f356ef9d700 1 -- 172.21.15.55:0/1802543863 >> 172.21.15.55:0/1802543863 conn(0x7f35680d10c0 msgr2=0x7f35680b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:45.777 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.774+0000 7f356ef9d700 1 -- 172.21.15.55:0/1802543863 shutdown_connections 2023-10-06T05:07:45.778 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.775+0000 7f356ef9d700 1 -- 172.21.15.55:0/1802543863 wait complete. 2023-10-06T05:07:45.796 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T05:07:45.991 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:45.989+0000 7f562514f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:07:45.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.991+0000 7f28ecb9d700 1 Processor -- start 2023-10-06T05:07:45.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.992+0000 7f28ecb9d700 1 -- start start 2023-10-06T05:07:46.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.993+0000 7f28ecb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e80b16d0 0x7f28e80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.993+0000 7f28ecb9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e80b2080 con 0x7f28e80b33b0 2023-10-06T05:07:46.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.993+0000 7f28ecb9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e80cfb00 con 0x7f28e80b4e70 2023-10-06T05:07:46.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.993+0000 7f28ecb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80b5c40 con 0x7f28e80b16d0 2023-10-06T05:07:46.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.993+0000 7f28e7fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f28e80b33b0 0x7f28e80b3830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34136/0 (socket says 172.21.15.55:34136) 2023-10-06T05:07:46.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.993+0000 7f28e7fff700 1 -- 172.21.15.55:0/94292326 learned_addr learned my addr 172.21.15.55:0/94292326 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:46.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.993+0000 7f28e6ffd700 1 --2- 172.21.15.55:0/94292326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e80b16d0 0x7f28e80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:46.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.994+0000 7f28e67fc700 1 -- 172.21.15.55:0/94292326 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221018282 0 0) 0x7f28e80b2080 con 0x7f28e80b33b0 2023-10-06T05:07:46.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.994+0000 7f28e67fc700 1 -- 172.21.15.55:0/94292326 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28d0003570 con 0x7f28e80b33b0 2023-10-06T05:07:46.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.994+0000 7f28e67fc700 1 -- 172.21.15.55:0/94292326 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719422852 0 0) 0x7f28e80cfb00 con 0x7f28e80b4e70 2023-10-06T05:07:46.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.994+0000 7f28e67fc700 1 -- 172.21.15.55:0/94292326 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28e80b2080 con 0x7f28e80b4e70 2023-10-06T05:07:46.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.994+0000 7f28e6ffd700 1 -- 172.21.15.55:0/94292326 >> v1:172.21.15.163:6789/0 conn(0x7f28e80b4e70 legacy=0x7f28e80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.994+0000 7f28e6ffd700 1 -- 172.21.15.55:0/94292326 >> v1:172.21.15.55:6789/0 conn(0x7f28e80b33b0 legacy=0x7f28e80b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.002 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.994+0000 7f28e6ffd700 1 -- 172.21.15.55:0/94292326 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e80b5f20 con 0x7f28e80b16d0 2023-10-06T05:07:46.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.994+0000 7f28e6ffd700 1 --2- 172.21.15.55:0/94292326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e80b16d0 0x7f28e80b1b40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009910 tx=0x7f28dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8945a36c5100a04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:46.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.995+0000 7f28e67fc700 1 -- 172.21.15.55:0/94292326 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc011630 con 0x7f28e80b16d0 2023-10-06T05:07:46.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.995+0000 7f28e67fc700 1 -- 172.21.15.55:0/94292326 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc011790 con 0x7f28e80b16d0 2023-10-06T05:07:46.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.995+0000 7f28e67fc700 1 -- 172.21.15.55:0/94292326 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc011b30 con 0x7f28e80b16d0 2023-10-06T05:07:46.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.995+0000 7f28ecb9d700 1 -- 172.21.15.55:0/94292326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e80b16d0 msgr2=0x7f28e80b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.995+0000 7f28ecb9d700 1 --2- 172.21.15.55:0/94292326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e80b16d0 0x7f28e80b1b40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009910 tx=0x7f28dc0091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.995+0000 7f28ecb9d700 1 -- 172.21.15.55:0/94292326 shutdown_connections 2023-10-06T05:07:46.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.995+0000 7f28ecb9d700 1 --2- 172.21.15.55:0/94292326 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e80b16d0 0x7f28e80b1b40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.995+0000 7f28ecb9d700 1 -- 172.21.15.55:0/94292326 >> 172.21.15.55:0/94292326 conn(0x7f28e80d10c0 msgr2=0x7f28e80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:46.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.995+0000 7f28ecb9d700 1 -- 172.21.15.55:0/94292326 shutdown_connections 2023-10-06T05:07:46.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.995+0000 7f28ecb9d700 1 -- 172.21.15.55:0/94292326 wait complete. 2023-10-06T05:07:46.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.997+0000 7f28ecb9d700 1 Processor -- start 2023-10-06T05:07:46.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.997+0000 7f28ecb9d700 1 -- start start 2023-10-06T05:07:46.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28ecb9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e80b16d0 0x7f28e8156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28ecb9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28e80b33b0 0x7f28e815b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28ecb9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28e80b4e70 0x7f28e815ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28ecb9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e815cf90 con 0x7f28e80b33b0 2023-10-06T05:07:46.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28ecb9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e8169a00 con 0x7f28e80b4e70 2023-10-06T05:07:46.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28ecb9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e815ba10 con 0x7f28e80b16d0 2023-10-06T05:07:46.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28e6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28e80b33b0 0x7f28e815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:46.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28e6ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28e80b33b0 0x7f28e815b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52222/0 (socket says 172.21.15.55:52222) 2023-10-06T05:07:46.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28e6ffd700 1 -- 172.21.15.55:0/2657043994 learned_addr learned my addr 172.21.15.55:0/2657043994 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:46.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28e77fe700 1 --2- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e80b16d0 0x7f28e8156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:46.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28e7fff700 1 --2- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28e80b4e70 0x7f28e815ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:46.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28e6ffd700 1 -- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e80b16d0 msgr2=0x7f28e8156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28e6ffd700 1 --2- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e80b16d0 0x7f28e8156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28e6ffd700 1 -- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28e80b4e70 msgr2=0x7f28e815ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28e6ffd700 1 --2- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28e80b4e70 0x7f28e815ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.998+0000 7f28e6ffd700 1 -- 172.21.15.55:0/2657043994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc009420 con 0x7f28e80b33b0 2023-10-06T05:07:46.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.999+0000 7f28e6ffd700 1 --2- 172.21.15.55:0/2657043994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28e80b33b0 0x7f28e815b330 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009f00 tx=0x7f28dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:46.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.999+0000 7f28cf7fe700 1 -- 172.21.15.55:0/2657043994 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc01f070 con 0x7f28e80b33b0 2023-10-06T05:07:46.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.999+0000 7f28cf7fe700 1 -- 172.21.15.55:0/2657043994 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28dc002f70 con 0x7f28e80b33b0 2023-10-06T05:07:46.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.999+0000 7f28cf7fe700 1 -- 172.21.15.55:0/2657043994 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc006af0 con 0x7f28e80b33b0 2023-10-06T05:07:46.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.999+0000 7f28ecb9d700 1 -- 172.21.15.55:0/2657043994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e815bcf0 con 0x7f28e80b33b0 2023-10-06T05:07:46.006 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:45.999+0000 7f28ecb9d700 1 -- 172.21.15.55:0/2657043994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e815c2b0 con 0x7f28e80b33b0 2023-10-06T05:07:46.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.000+0000 7f28ecb9d700 1 -- 172.21.15.55:0/2657043994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e8088990 con 0x7f28e80b33b0 2023-10-06T05:07:46.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.001+0000 7f28cf7fe700 1 -- 172.21.15.55:0/2657043994 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 105) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f28dc023070 con 0x7f28e80b33b0 2023-10-06T05:07:46.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.002+0000 7f28cf7fe700 1 --2- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f28d40670e0 0x7f28d4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.002+0000 7f28cf7fe700 1 -- 172.21.15.55:0/2657043994 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f28dc085f00 con 0x7f28e80b33b0 2023-10-06T05:07:46.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.002+0000 7f28e77fe700 1 -- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f28d40670e0 msgr2=0x7f28d4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:46.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.002+0000 7f28e77fe700 1 --2- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f28d40670e0 0x7f28d4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:46.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.006+0000 7f28cf7fe700 1 -- 172.21.15.55:0/2657043994 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28dc052a70 con 0x7f28e80b33b0 2023-10-06T05:07:46.088 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:46.086+0000 7f562514f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:07:46.158 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:46.156+0000 7f1696d46180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T05:07:46.179 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:46.177+0000 7f562514f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:07:46.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.203+0000 7f28e77fe700 1 -- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f28d40670e0 msgr2=0x7f28d4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:46.205 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.203+0000 7f28e77fe700 1 --2- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f28d40670e0 0x7f28d4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:46.211 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.210+0000 7f28ecb9d700 1 -- 172.21.15.55:0/2657043994 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28e804bb80 con 0x7f28e80b33b0 2023-10-06T05:07:46.212 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.210+0000 7f28cf7fe700 1 -- 172.21.15.55:0/2657043994 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v208) v1 ==== 100+0+11 (secure 0 0 0) 0x7f28dc015070 con 0x7f28e80b33b0 2023-10-06T05:07:46.212 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 64 2023-10-06T05:07:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.212+0000 7f28ecb9d700 1 -- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f28d40670e0 msgr2=0x7f28d4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.212+0000 7f28ecb9d700 1 --2- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f28d40670e0 0x7f28d4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.212+0000 7f28ecb9d700 1 -- 172.21.15.55:0/2657043994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28e80b33b0 msgr2=0x7f28e815b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.212+0000 7f28ecb9d700 1 --2- 172.21.15.55:0/2657043994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28e80b33b0 0x7f28e815b330 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009f00 tx=0x7f28dc017000 comp rx=0 tx=0).stop 2023-10-06T05:07:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.213+0000 7f28ecb9d700 1 -- 172.21.15.55:0/2657043994 shutdown_connections 2023-10-06T05:07:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.213+0000 7f28ecb9d700 1 --2- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f28d40670e0 0x7f28d4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.215 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.213+0000 7f28ecb9d700 1 --2- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f28e80b4e70 0x7f28e815ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.213+0000 7f28ecb9d700 1 --2- 172.21.15.55:0/2657043994 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f28e80b16d0 0x7f28e8156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.213+0000 7f28ecb9d700 1 --2- 172.21.15.55:0/2657043994 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f28e80b33b0 0x7f28e815b330 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.213+0000 7f28ecb9d700 1 -- 172.21.15.55:0/2657043994 >> 172.21.15.55:0/2657043994 conn(0x7f28e80d10c0 msgr2=0x7f28e81590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:46.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.213+0000 7f28ecb9d700 1 -- 172.21.15.55:0/2657043994 shutdown_connections 2023-10-06T05:07:46.216 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.213+0000 7f28ecb9d700 1 -- 172.21.15.55:0/2657043994 wait complete. 2023-10-06T05:07:46.233 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:07:46.264 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:46.262+0000 7f1696d46180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T05:07:46.361 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:46.359+0000 7f1696d46180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T05:07:46.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.424+0000 7f9d1519d700 1 Processor -- start 2023-10-06T05:07:46.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.424+0000 7f9d1519d700 1 -- start start 2023-10-06T05:07:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.425+0000 7f9d1519d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9d101593b0 0x7f9d10159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.425+0000 7f9d1519d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d100b1e50 0x7f9d100b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.425+0000 7f9d1519d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d100b6570 con 0x7f9d100ceaa0 2023-10-06T05:07:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.425+0000 7f9d1519d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d100cfe20 con 0x7f9d101593b0 2023-10-06T05:07:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.425+0000 7f9d1519d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100cf210 con 0x7f9d100b1e50 2023-10-06T05:07:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.426+0000 7f9d0f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9d100ceaa0 0x7f9d100b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34140/0 (socket says 172.21.15.55:34140) 2023-10-06T05:07:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.426+0000 7f9d0f7fe700 1 -- 172.21.15.55:0/49751164 learned_addr learned my addr 172.21.15.55:0/49751164 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.426+0000 7f9d1499c700 1 --2- 172.21.15.55:0/49751164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d100b1e50 0x7f9d100b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:46.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.426+0000 7f9d0ffff700 1 --2- 172.21.15.55:0/49751164 >> v2:172.21.15.163:3300/0 conn(0x7f9d101593b0 0x7f9d10159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:46.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.426+0000 7f9d0effd700 1 -- 172.21.15.55:0/49751164 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101285148 0 0) 0x7f9d04001110 con 0x7f9d100ceaa0 2023-10-06T05:07:46.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.426+0000 7f9d0effd700 1 -- 172.21.15.55:0/49751164 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cf8003570 con 0x7f9d100ceaa0 2023-10-06T05:07:46.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.426+0000 7f9d0ffff700 1 -- 172.21.15.55:0/49751164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d100b1e50 msgr2=0x7f9d100b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:46.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.426+0000 7f9d0ffff700 1 --2- 172.21.15.55:0/49751164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d100b1e50 0x7f9d100b6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.426+0000 7f9d0ffff700 1 -- 172.21.15.55:0/49751164 >> v1:172.21.15.55:6789/0 conn(0x7f9d100ceaa0 legacy=0x7f9d100b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.426+0000 7f9d0ffff700 1 -- 172.21.15.55:0/49751164 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d100cf4f0 con 0x7f9d101593b0 2023-10-06T05:07:46.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.427+0000 7f9d0ffff700 1 --2- 172.21.15.55:0/49751164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d101593b0 0x7f9d10159780 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f9d000098e0 tx=0x7f9d000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff8cdf7bf518e61c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:46.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.427+0000 7f9d0effd700 1 -- 172.21.15.55:0/49751164 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00011630 con 0x7f9d101593b0 2023-10-06T05:07:46.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.427+0000 7f9d0effd700 1 -- 172.21.15.55:0/49751164 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d00011790 con 0x7f9d101593b0 2023-10-06T05:07:46.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.427+0000 7f9d0effd700 1 -- 172.21.15.55:0/49751164 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00011b30 con 0x7f9d101593b0 2023-10-06T05:07:46.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.427+0000 7f9d1519d700 1 -- 172.21.15.55:0/49751164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d101593b0 msgr2=0x7f9d10159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.427+0000 7f9d1519d700 1 --2- 172.21.15.55:0/49751164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d101593b0 0x7f9d10159780 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f9d000098e0 tx=0x7f9d000091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.427+0000 7f9d1519d700 1 -- 172.21.15.55:0/49751164 shutdown_connections 2023-10-06T05:07:46.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.427+0000 7f9d1519d700 1 --2- 172.21.15.55:0/49751164 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d100b1e50 0x7f9d100b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.427+0000 7f9d1519d700 1 --2- 172.21.15.55:0/49751164 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d101593b0 0x7f9d10159780 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.428+0000 7f9d1519d700 1 -- 172.21.15.55:0/49751164 >> 172.21.15.55:0/49751164 conn(0x7f9d100d1130 msgr2=0x7f9d100b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:46.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.428+0000 7f9d1519d700 1 -- 172.21.15.55:0/49751164 shutdown_connections 2023-10-06T05:07:46.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.428+0000 7f9d1519d700 1 -- 172.21.15.55:0/49751164 wait complete. 2023-10-06T05:07:46.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.429+0000 7f9d1519d700 1 Processor -- start 2023-10-06T05:07:46.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.429+0000 7f9d1519d700 1 -- start start 2023-10-06T05:07:46.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.430+0000 7f9d1519d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d100b1e50 0x7f9d101444f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.430+0000 7f9d1519d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d100ceaa0 0x7f9d1015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.430+0000 7f9d1519d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d101593b0 0x7f9d1015ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.430+0000 7f9d1519d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d10144a30 con 0x7f9d101593b0 2023-10-06T05:07:46.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.430+0000 7f9d1519d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d100cbef0 con 0x7f9d100ceaa0 2023-10-06T05:07:46.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.430+0000 7f9d1519d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1016ab20 con 0x7f9d100b1e50 2023-10-06T05:07:46.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.431+0000 7f9d0f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d100ceaa0 0x7f9d1015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:46.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.431+0000 7f9d1499c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d101593b0 0x7f9d1015ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:46.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.431+0000 7f9d0ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d100b1e50 0x7f9d101444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:46.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.431+0000 7f9d1499c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d101593b0 0x7f9d1015ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52228/0 (socket says 172.21.15.55:52228) 2023-10-06T05:07:46.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.431+0000 7f9d0f7fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d100ceaa0 0x7f9d1015a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:48586/0 (socket says 172.21.15.55:48586) 2023-10-06T05:07:46.433 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.431+0000 7f9d1499c700 1 -- 172.21.15.55:0/1653918911 learned_addr learned my addr 172.21.15.55:0/1653918911 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:46.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.431+0000 7f9d0f7fe700 1 -- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d100b1e50 msgr2=0x7f9d101444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.431+0000 7f9d0f7fe700 1 --2- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d100b1e50 0x7f9d101444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.431+0000 7f9d0f7fe700 1 -- 172.21.15.55:0/1653918911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d101593b0 msgr2=0x7f9d1015ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.431+0000 7f9d0f7fe700 1 --2- 172.21.15.55:0/1653918911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d101593b0 0x7f9d1015ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.431+0000 7f9d0f7fe700 1 -- 172.21.15.55:0/1653918911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d00009420 con 0x7f9d100ceaa0 2023-10-06T05:07:46.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.431+0000 7f9d1499c700 1 --2- 172.21.15.55:0/1653918911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d101593b0 0x7f9d1015ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:46.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.431+0000 7f9d0f7fe700 1 --2- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d100ceaa0 0x7f9d1015a750 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9d040058d0 tx=0x7f9d04007bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:46.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.432+0000 7f9cf7fff700 1 -- 172.21.15.55:0/1653918911 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04005b30 con 0x7f9d100ceaa0 2023-10-06T05:07:46.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.432+0000 7f9cf7fff700 1 -- 172.21.15.55:0/1653918911 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d04005c90 con 0x7f9d100ceaa0 2023-10-06T05:07:46.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.432+0000 7f9d1519d700 1 -- 172.21.15.55:0/1653918911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1016ae00 con 0x7f9d100ceaa0 2023-10-06T05:07:46.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.432+0000 7f9cf7fff700 1 -- 172.21.15.55:0/1653918911 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d04007320 con 0x7f9d100ceaa0 2023-10-06T05:07:46.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.432+0000 7f9d1519d700 1 -- 172.21.15.55:0/1653918911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d100cc1d0 con 0x7f9d100ceaa0 2023-10-06T05:07:46.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.433+0000 7f9d1519d700 1 -- 172.21.15.55:0/1653918911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1016b320 con 0x7f9d100ceaa0 2023-10-06T05:07:46.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.434+0000 7f9cf7fff700 1 -- 172.21.15.55:0/1653918911 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 105) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f9d04007690 con 0x7f9d100ceaa0 2023-10-06T05:07:46.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.434+0000 7f9cf7fff700 1 --2- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9cfc0670e0 0x7f9cfc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.435+0000 7f9cf7fff700 1 -- 172.21.15.55:0/1653918911 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f9d04083fa0 con 0x7f9d100ceaa0 2023-10-06T05:07:46.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.435+0000 7f9d0ffff700 1 -- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9cfc0670e0 msgr2=0x7f9cfc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:46.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.435+0000 7f9d0ffff700 1 --2- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9cfc0670e0 0x7f9cfc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:46.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.439+0000 7f9cf7fff700 1 -- 172.21.15.55:0/1653918911 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d1016b320 con 0x7f9d100ceaa0 2023-10-06T05:07:46.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.636+0000 7f9d0ffff700 1 -- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9cfc0670e0 msgr2=0x7f9cfc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:46.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.636+0000 7f9d0ffff700 1 --2- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9cfc0670e0 0x7f9cfc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:46.705 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.703+0000 7f9d1519d700 1 -- 172.21.15.55:0/1653918911 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d100b6570 con 0x7f9d100ceaa0 2023-10-06T05:07:46.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.704+0000 7f9cf7fff700 1 -- 172.21.15.55:0/1653918911 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4616 (secure 0 0 0) 0x7f9d100b6570 con 0x7f9d100ceaa0 2023-10-06T05:07:46.706 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:46.706 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3704044820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3704044820},{"type":"v1","addr":"172.21.15.163:6834","nonce":3704044820}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":9772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2254022733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2254022733},{"type":"v1","addr":"172.21.15.163:6835","nonce":2254022733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":10224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3844040929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3844040929},{"type":"v1","addr":"172.21.15.55:6834","nonce":3844040929}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:07:22.724188+0000","modified":"2023-10-06T05:07:26.255934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10227":{"gid":10227,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/3632968604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3632968604},{"type":"v1","addr":"172.21.15.55:6835","nonce":3632968604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:07:46.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.707+0000 7f9d1519d700 1 -- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9cfc0670e0 msgr2=0x7f9cfc069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:46.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.707+0000 7f9d1519d700 1 --2- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9cfc0670e0 0x7f9cfc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.707+0000 7f9d1519d700 1 -- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d100ceaa0 msgr2=0x7f9d1015a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.707+0000 7f9d1519d700 1 --2- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d100ceaa0 0x7f9d1015a750 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9d040058d0 tx=0x7f9d04007bf0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.707+0000 7f9d1519d700 1 -- 172.21.15.55:0/1653918911 shutdown_connections 2023-10-06T05:07:46.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.707+0000 7f9d1519d700 1 --2- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9cfc0670e0 0x7f9cfc069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.707+0000 7f9d1519d700 1 --2- 172.21.15.55:0/1653918911 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9d101593b0 0x7f9d1015ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.707+0000 7f9d1519d700 1 --2- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9d100b1e50 0x7f9d101444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.707+0000 7f9d1519d700 1 --2- 172.21.15.55:0/1653918911 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9d100ceaa0 0x7f9d1015a750 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.707+0000 7f9d1519d700 1 -- 172.21.15.55:0/1653918911 >> 172.21.15.55:0/1653918911 conn(0x7f9d100d1130 msgr2=0x7f9d1015be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:46.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.707+0000 7f9d1519d700 1 -- 172.21.15.55:0/1653918911 shutdown_connections 2023-10-06T05:07:46.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.707+0000 7f9d1519d700 1 -- 172.21.15.55:0/1653918911 wait complete. 2023-10-06T05:07:46.711 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 105 2023-10-06T05:07:46.727 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:07:46.807 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:46.806+0000 7f562514f180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:07:46.982 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:46.981+0000 7f562514f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:07:46.987 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.986+0000 7f6160d9d700 1 Processor -- start 2023-10-06T05:07:46.988 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.986+0000 7f6160d9d700 1 -- start start 2023-10-06T05:07:46.989 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.987+0000 7f6160d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f615c0c6c60 0x7f615c0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.988+0000 7f6160d9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f615c0b1410 con 0x7f615c06e8c0 2023-10-06T05:07:46.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.988+0000 7f6160d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f615c063230 con 0x7f615c1595a0 2023-10-06T05:07:46.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.988+0000 7f6160d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f615c0b2b90 con 0x7f615c0c6c60 2023-10-06T05:07:46.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.988+0000 7f615affd700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f615c0c6c60 0x7f615c0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:46.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.988+0000 7f615bfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f615c06e8c0 0x7f615c159970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34156/0 (socket says 172.21.15.55:34156) 2023-10-06T05:07:46.990 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.988+0000 7f615bfff700 1 -- 172.21.15.55:0/4087695905 learned_addr learned my addr 172.21.15.55:0/4087695905 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:46.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.988+0000 7f615a7fc700 1 -- 172.21.15.55:0/4087695905 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689756915 0 0) 0x7f615c063230 con 0x7f615c1595a0 2023-10-06T05:07:46.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.988+0000 7f615a7fc700 1 -- 172.21.15.55:0/4087695905 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6144003570 con 0x7f615c1595a0 2023-10-06T05:07:46.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.989+0000 7f615affd700 1 -- 172.21.15.55:0/4087695905 >> v1:172.21.15.163:6789/0 conn(0x7f615c1595a0 legacy=0x7f615c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.989+0000 7f615affd700 1 -- 172.21.15.55:0/4087695905 >> v1:172.21.15.55:6789/0 conn(0x7f615c06e8c0 legacy=0x7f615c159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.989+0000 7f615a7fc700 1 -- 172.21.15.55:0/4087695905 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988500100 0 0) 0x7f615c0b1410 con 0x7f615c06e8c0 2023-10-06T05:07:46.991 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.989+0000 7f615affd700 1 -- 172.21.15.55:0/4087695905 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c0634f0 con 0x7f615c0c6c60 2023-10-06T05:07:46.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.989+0000 7f615affd700 1 --2- 172.21.15.55:0/4087695905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f615c0c6c60 0x7f615c0c8120 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f61500098e0 tx=0x7f61500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=107ce16cab74016d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:46.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.989+0000 7f615a7fc700 1 -- 172.21.15.55:0/4087695905 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615000b640 con 0x7f615c0c6c60 2023-10-06T05:07:46.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.989+0000 7f615a7fc700 1 -- 172.21.15.55:0/4087695905 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615000b7a0 con 0x7f615c0c6c60 2023-10-06T05:07:46.992 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.989+0000 7f615a7fc700 1 -- 172.21.15.55:0/4087695905 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615000bb40 con 0x7f615c0c6c60 2023-10-06T05:07:46.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.990+0000 7f6160d9d700 1 -- 172.21.15.55:0/4087695905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f615c0c6c60 msgr2=0x7f615c0c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.990+0000 7f6160d9d700 1 --2- 172.21.15.55:0/4087695905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f615c0c6c60 0x7f615c0c8120 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f61500098e0 tx=0x7f61500091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.991+0000 7f6160d9d700 1 -- 172.21.15.55:0/4087695905 shutdown_connections 2023-10-06T05:07:46.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.991+0000 7f6160d9d700 1 --2- 172.21.15.55:0/4087695905 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f615c0c6c60 0x7f615c0c8120 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.991+0000 7f6160d9d700 1 -- 172.21.15.55:0/4087695905 >> 172.21.15.55:0/4087695905 conn(0x7f615c06ddb0 msgr2=0x7f615c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:46.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.991+0000 7f6160d9d700 1 -- 172.21.15.55:0/4087695905 shutdown_connections 2023-10-06T05:07:46.993 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.991+0000 7f6160d9d700 1 -- 172.21.15.55:0/4087695905 wait complete. 2023-10-06T05:07:46.994 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.993+0000 7f6160d9d700 1 Processor -- start 2023-10-06T05:07:46.995 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.993+0000 7f6160d9d700 1 -- start start 2023-10-06T05:07:46.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.994+0000 7f6160d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06e8c0 0x7f615c15cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.994+0000 7f6160d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f615c0c6c60 0x7f615c16a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.997 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.994+0000 7f6160d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f615c1595a0 0x7f615c1511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.994+0000 7f6160d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c15d220 con 0x7f615c06e8c0 2023-10-06T05:07:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.994+0000 7f615b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06e8c0 0x7f615c15cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.994+0000 7f6160d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c16aa20 con 0x7f615c1595a0 2023-10-06T05:07:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.994+0000 7f6160d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615c16d260 con 0x7f615c0c6c60 2023-10-06T05:07:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.994+0000 7f615b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06e8c0 0x7f615c15cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52236/0 (socket says 172.21.15.55:52236) 2023-10-06T05:07:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.994+0000 7f615b7fe700 1 -- 172.21.15.55:0/2289275483 learned_addr learned my addr 172.21.15.55:0/2289275483 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.994+0000 7f615bfff700 1 --2- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f615c1595a0 0x7f615c1511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.994+0000 7f615affd700 1 --2- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f615c0c6c60 0x7f615c16a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.995+0000 7f615b7fe700 1 -- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f615c0c6c60 msgr2=0x7f615c16a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.995+0000 7f615b7fe700 1 --2- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f615c0c6c60 0x7f615c16a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.998 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.995+0000 7f615b7fe700 1 -- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f615c1595a0 msgr2=0x7f615c1511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:46.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.995+0000 7f615b7fe700 1 --2- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f615c1595a0 0x7f615c1511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:46.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.995+0000 7f615b7fe700 1 -- 172.21.15.55:0/2289275483 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6150009420 con 0x7f615c06e8c0 2023-10-06T05:07:46.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.995+0000 7f615b7fe700 1 --2- 172.21.15.55:0/2289275483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06e8c0 0x7f615c15cbb0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f614c00c5b0 tx=0x7f614c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:46.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.995+0000 7f61437fe700 1 -- 172.21.15.55:0/2289275483 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c010510 con 0x7f615c06e8c0 2023-10-06T05:07:46.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.995+0000 7f61437fe700 1 -- 172.21.15.55:0/2289275483 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c010670 con 0x7f615c06e8c0 2023-10-06T05:07:46.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.995+0000 7f61437fe700 1 -- 172.21.15.55:0/2289275483 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c010a30 con 0x7f615c06e8c0 2023-10-06T05:07:46.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.995+0000 7f6160d9d700 1 -- 172.21.15.55:0/2289275483 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615c16d540 con 0x7f615c06e8c0 2023-10-06T05:07:46.999 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.996+0000 7f6160d9d700 1 -- 172.21.15.55:0/2289275483 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615c16db00 con 0x7f615c06e8c0 2023-10-06T05:07:47.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.997+0000 7f6160d9d700 1 -- 172.21.15.55:0/2289275483 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615c04c3c0 con 0x7f615c06e8c0 2023-10-06T05:07:47.000 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.998+0000 7f61437fe700 1 -- 172.21.15.55:0/2289275483 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 105) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f614c01b400 con 0x7f615c06e8c0 2023-10-06T05:07:47.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.999+0000 7f61437fe700 1 --2- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f61480671b0 0x7f6148069660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:47.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.999+0000 7f61437fe700 1 -- 172.21.15.55:0/2289275483 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f614c0847d0 con 0x7f615c06e8c0 2023-10-06T05:07:47.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.999+0000 7f615affd700 1 -- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f61480671b0 msgr2=0x7f6148069660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:47.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:46.999+0000 7f615affd700 1 --2- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f61480671b0 0x7f6148069660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:47.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.003+0000 7f61437fe700 1 -- 172.21.15.55:0/2289275483 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f614c051420 con 0x7f615c06e8c0 2023-10-06T05:07:47.040 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:47.038+0000 7f1696d46180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T05:07:47.084 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:47.082+0000 7f562514f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:07:47.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.200+0000 7f615affd700 1 -- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f61480671b0 msgr2=0x7f6148069660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:47.202 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.200+0000 7f615affd700 1 --2- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f61480671b0 0x7f6148069660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:47.217 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.215+0000 7f6160d9d700 1 -- 172.21.15.55:0/2289275483 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f615c0637d0 con 0x7f615c06e8c0 2023-10-06T05:07:47.218 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.216+0000 7f61437fe700 1 -- 172.21.15.55:0/2289275483 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v208) v1 ==== 74+0+17989 (secure 0 0 0) 0x7f614c054a40 con 0x7f615c06e8c0 2023-10-06T05:07:47.218 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:47.218 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":208,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:07:39.103933+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:07:20.916360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:07:21.706951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"207","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/2709454586":"2023-10-07T05:07:39.103885+0000","172.21.15.163:6837/101089":"2023-10-07T05:07:39.103885+0000","172.21.15.163:6836/101089":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/2719429847":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/3618319475":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/2222063413":"2023-10-07T05:07:39.103885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:07:47.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.218+0000 7f6160d9d700 1 -- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f61480671b0 msgr2=0x7f6148069660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:47.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.218+0000 7f6160d9d700 1 --2- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f61480671b0 0x7f6148069660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.218+0000 7f6160d9d700 1 -- 172.21.15.55:0/2289275483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06e8c0 msgr2=0x7f615c15cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:47.221 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.218+0000 7f6160d9d700 1 --2- 172.21.15.55:0/2289275483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06e8c0 0x7f615c15cbb0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f614c00c5b0 tx=0x7f614c007e10 comp rx=0 tx=0).stop 2023-10-06T05:07:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.218+0000 7f6160d9d700 1 -- 172.21.15.55:0/2289275483 shutdown_connections 2023-10-06T05:07:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.218+0000 7f6160d9d700 1 --2- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f61480671b0 0x7f6148069660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.218+0000 7f6160d9d700 1 --2- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f615c1595a0 0x7f615c1511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.218+0000 7f6160d9d700 1 --2- 172.21.15.55:0/2289275483 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f615c06e8c0 0x7f615c15cbb0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.218+0000 7f6160d9d700 1 --2- 172.21.15.55:0/2289275483 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f615c0c6c60 0x7f615c16a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.218+0000 7f6160d9d700 1 -- 172.21.15.55:0/2289275483 >> 172.21.15.55:0/2289275483 conn(0x7f615c06ddb0 msgr2=0x7f615c0cb300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.218+0000 7f6160d9d700 1 -- 172.21.15.55:0/2289275483 shutdown_connections 2023-10-06T05:07:47.222 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.219+0000 7f6160d9d700 1 -- 172.21.15.55:0/2289275483 wait complete. 2023-10-06T05:07:47.223 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:47.220+0000 7f1696d46180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T05:07:47.239 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:07:47.239 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:07:47.325 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:47.323+0000 7f1696d46180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T05:07:47.381 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:47.379+0000 7f562514f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:07:47.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.428+0000 7fe2fddff700 1 Processor -- start 2023-10-06T05:07:47.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.429+0000 7fe2fddff700 1 -- start start 2023-10-06T05:07:47.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.430+0000 7fe2fddff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe2f8159890 0x7fe2f815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:47.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.430+0000 7fe2fddff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe2f8159c60 0x7fe2f815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:47.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.430+0000 7fe2fddff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2f815db10 0x7fe2f8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:47.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.430+0000 7fe2fddff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f815df50 con 0x7fe2f8159c60 2023-10-06T05:07:47.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.430+0000 7fe2fddff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2f815a160 con 0x7fe2f8159890 2023-10-06T05:07:47.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.430+0000 7fe2fddff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f80d13a0 con 0x7fe2f815db10 2023-10-06T05:07:47.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.430+0000 7fe2f6ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe2f8159c60 0x7fe2f815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:47.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.430+0000 7fe2f6ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe2f8159c60 0x7fe2f815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52240/0 (socket says 172.21.15.55:52240) 2023-10-06T05:07:47.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.430+0000 7fe2f6ffd700 1 -- 172.21.15.55:0/1478629755 learned_addr learned my addr 172.21.15.55:0/1478629755 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.431+0000 7fe2f7fff700 1 --2- 172.21.15.55:0/1478629755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2f815db10 0x7fe2f8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.430+0000 7fe2f77fe700 1 --2- 172.21.15.55:0/1478629755 >> v2:172.21.15.163:3300/0 conn(0x7fe2f8159890 0x7fe2f815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.431+0000 7fe2f6ffd700 1 -- 172.21.15.55:0/1478629755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2f815db10 msgr2=0x7fe2f8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.431+0000 7fe2f6ffd700 1 --2- 172.21.15.55:0/1478629755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2f815db10 0x7fe2f8069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.431+0000 7fe2f6ffd700 1 -- 172.21.15.55:0/1478629755 >> v2:172.21.15.163:3300/0 conn(0x7fe2f8159890 msgr2=0x7fe2f815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.431+0000 7fe2f6ffd700 1 --2- 172.21.15.55:0/1478629755 >> v2:172.21.15.163:3300/0 conn(0x7fe2f8159890 0x7fe2f815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.431+0000 7fe2f6ffd700 1 -- 172.21.15.55:0/1478629755 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f815e140 con 0x7fe2f8159c60 2023-10-06T05:07:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.431+0000 7fe2f6ffd700 1 --2- 172.21.15.55:0/1478629755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2f8159c60 0x7fe2f815d5d0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80d1090 tx=0x7fe2ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33dca93ce89c45d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.431+0000 7fe2f67fc700 1 -- 172.21.15.55:0/1478629755 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec00b640 con 0x7fe2f8159c60 2023-10-06T05:07:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.431+0000 7fe2f67fc700 1 -- 172.21.15.55:0/1478629755 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec00b7a0 con 0x7fe2f8159c60 2023-10-06T05:07:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.431+0000 7fe2f67fc700 1 -- 172.21.15.55:0/1478629755 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec00bb60 con 0x7fe2f8159c60 2023-10-06T05:07:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.432+0000 7fe2fddff700 1 -- 172.21.15.55:0/1478629755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2f8159c60 msgr2=0x7fe2f815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.432+0000 7fe2fddff700 1 --2- 172.21.15.55:0/1478629755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2f8159c60 0x7fe2f815d5d0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80d1090 tx=0x7fe2ec009250 comp rx=0 tx=0).stop 2023-10-06T05:07:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.432+0000 7fe2fddff700 1 -- 172.21.15.55:0/1478629755 shutdown_connections 2023-10-06T05:07:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.432+0000 7fe2fddff700 1 --2- 172.21.15.55:0/1478629755 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2f815db10 0x7fe2f8069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.432+0000 7fe2fddff700 1 --2- 172.21.15.55:0/1478629755 >> v2:172.21.15.163:3300/0 conn(0x7fe2f8159890 0x7fe2f815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.432+0000 7fe2fddff700 1 --2- 172.21.15.55:0/1478629755 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2f8159c60 0x7fe2f815d5d0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.432+0000 7fe2fddff700 1 -- 172.21.15.55:0/1478629755 >> 172.21.15.55:0/1478629755 conn(0x7fe2f8068e20 msgr2=0x7fe2f80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:47.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.432+0000 7fe2fddff700 1 -- 172.21.15.55:0/1478629755 shutdown_connections 2023-10-06T05:07:47.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.432+0000 7fe2fddff700 1 -- 172.21.15.55:0/1478629755 wait complete. 2023-10-06T05:07:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.434+0000 7fe2fddff700 1 Processor -- start 2023-10-06T05:07:47.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.434+0000 7fe2fddff700 1 -- start start 2023-10-06T05:07:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.435+0000 7fe2fddff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2f8159890 0x7fe2f80b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.435+0000 7fe2fddff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2f8159c60 0x7fe2f80b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.435+0000 7fe2fddff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f815db10 0x7fe2f80b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.435+0000 7fe2fddff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f80b05d0 con 0x7fe2f8159890 2023-10-06T05:07:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.435+0000 7fe2fddff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f80b46a0 con 0x7fe2f815db10 2023-10-06T05:07:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.435+0000 7fe2fddff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f80b4980 con 0x7fe2f8159c60 2023-10-06T05:07:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.435+0000 7fe2f77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2f8159890 0x7fe2f80b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.436+0000 7fe2f7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f815db10 0x7fe2f80b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.436+0000 7fe2f7fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f815db10 0x7fe2f80b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:48626/0 (socket says 172.21.15.55:48626) 2023-10-06T05:07:47.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.436+0000 7fe2f7fff700 1 -- 172.21.15.55:0/1837093960 learned_addr learned my addr 172.21.15.55:0/1837093960 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:47.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.436+0000 7fe2f6ffd700 1 --2- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2f8159c60 0x7fe2f80b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:47.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.436+0000 7fe2f7fff700 1 -- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2f8159c60 msgr2=0x7fe2f80b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:47.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.436+0000 7fe2f7fff700 1 --2- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2f8159c60 0x7fe2f80b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.436+0000 7fe2f7fff700 1 -- 172.21.15.55:0/1837093960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2f8159890 msgr2=0x7fe2f80b3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:47.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.436+0000 7fe2f7fff700 1 --2- 172.21.15.55:0/1837093960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2f8159890 0x7fe2f80b3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.436+0000 7fe2f7fff700 1 -- 172.21.15.55:0/1837093960 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec0094f0 con 0x7fe2f815db10 2023-10-06T05:07:47.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.436+0000 7fe2f7fff700 1 --2- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f815db10 0x7fe2f80b5550 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009e90 tx=0x7fe2f000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:47.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.437+0000 7fe2df7fe700 1 -- 172.21.15.55:0/1837093960 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f000cb70 con 0x7fe2f815db10 2023-10-06T05:07:47.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.437+0000 7fe2df7fe700 1 -- 172.21.15.55:0/1837093960 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f000f040 con 0x7fe2f815db10 2023-10-06T05:07:47.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.437+0000 7fe2fddff700 1 -- 172.21.15.55:0/1837093960 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f80b07e0 con 0x7fe2f815db10 2023-10-06T05:07:47.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.437+0000 7fe2df7fe700 1 -- 172.21.15.55:0/1837093960 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0014480 con 0x7fe2f815db10 2023-10-06T05:07:47.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.437+0000 7fe2fddff700 1 -- 172.21.15.55:0/1837093960 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f80c8560 con 0x7fe2f815db10 2023-10-06T05:07:47.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.438+0000 7fe2fddff700 1 -- 172.21.15.55:0/1837093960 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f80b4ba0 con 0x7fe2f815db10 2023-10-06T05:07:47.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.439+0000 7fe2df7fe700 1 -- 172.21.15.55:0/1837093960 <== mon.1 v2:172.21.15.163:3300/0 4 ==== mgrmap(e 105) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fe2f0014830 con 0x7fe2f815db10 2023-10-06T05:07:47.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.440+0000 7fe2df7fe700 1 --2- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe2e40670e0 0x7fe2e4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:47.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.440+0000 7fe2df7fe700 1 -- 172.21.15.55:0/1837093960 <== mon.1 v2:172.21.15.163:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fe2f0085770 con 0x7fe2f815db10 2023-10-06T05:07:47.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.440+0000 7fe2f77fe700 1 -- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe2e40670e0 msgr2=0x7fe2e4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:47.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.440+0000 7fe2f77fe700 1 --2- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe2e40670e0 0x7fe2e4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:47.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.444+0000 7fe2df7fe700 1 -- 172.21.15.55:0/1837093960 <== mon.1 v2:172.21.15.163:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2f0052190 con 0x7fe2f815db10 2023-10-06T05:07:47.480 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:47.479+0000 7f562514f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:07:47.629 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:47.627+0000 7f1696d46180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T05:07:47.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.641+0000 7fe2f77fe700 1 -- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe2e40670e0 msgr2=0x7fe2e4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:47.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.641+0000 7fe2f77fe700 1 --2- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe2e40670e0 0x7fe2e4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:47.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.693+0000 7fe2fddff700 1 -- 172.21.15.55:0/1837093960 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2f80b0d00 con 0x7fe2f815db10 2023-10-06T05:07:47.695 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.694+0000 7fe2df7fe700 1 -- 172.21.15.55:0/1837093960 <== mon.1 v2:172.21.15.163:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4616 (secure 0 0 0) 0x7fe2f80b0d00 con 0x7fe2f815db10 2023-10-06T05:07:47.696 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:47.696 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3704044820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3704044820},{"type":"v1","addr":"172.21.15.163:6834","nonce":3704044820}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":9772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2254022733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2254022733},{"type":"v1","addr":"172.21.15.163:6835","nonce":2254022733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":10224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3844040929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3844040929},{"type":"v1","addr":"172.21.15.55:6834","nonce":3844040929}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:07:22.724188+0000","modified":"2023-10-06T05:07:26.255934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10227":{"gid":10227,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/3632968604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3632968604},{"type":"v1","addr":"172.21.15.55:6835","nonce":3632968604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:07:47.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.696+0000 7fe2fddff700 1 -- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe2e40670e0 msgr2=0x7fe2e4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:47.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.696+0000 7fe2fddff700 1 --2- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe2e40670e0 0x7fe2e4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.696+0000 7fe2fddff700 1 -- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f815db10 msgr2=0x7fe2f80b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:47.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.696+0000 7fe2fddff700 1 --2- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f815db10 0x7fe2f80b5550 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009e90 tx=0x7fe2f000b040 comp rx=0 tx=0).stop 2023-10-06T05:07:47.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.696+0000 7fe2fddff700 1 -- 172.21.15.55:0/1837093960 shutdown_connections 2023-10-06T05:07:47.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.696+0000 7fe2fddff700 1 --2- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe2e40670e0 0x7fe2e4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.699 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.696+0000 7fe2fddff700 1 --2- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2f815db10 0x7fe2f80b5550 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.696+0000 7fe2fddff700 1 --2- 172.21.15.55:0/1837093960 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe2f8159890 0x7fe2f80b3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.696+0000 7fe2fddff700 1 --2- 172.21.15.55:0/1837093960 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2f8159c60 0x7fe2f80b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.696+0000 7fe2fddff700 1 -- 172.21.15.55:0/1837093960 >> 172.21.15.55:0/1837093960 conn(0x7fe2f8068e20 msgr2=0x7fe2f80cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:47.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.696+0000 7fe2fddff700 1 -- 172.21.15.55:0/1837093960 shutdown_connections 2023-10-06T05:07:47.700 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.696+0000 7fe2fddff700 1 -- 172.21.15.55:0/1837093960 wait complete. 2023-10-06T05:07:47.700 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 105 2023-10-06T05:07:47.716 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:07:47.731 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:47.730+0000 7f1696d46180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T05:07:47.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.904+0000 7f555ffff700 1 Processor -- start 2023-10-06T05:07:47.906 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.904+0000 7f555ffff700 1 -- start start 2023-10-06T05:07:47.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.905+0000 7f555ffff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f55601595a0 0x7f55600b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:47.907 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.905+0000 7f555ffff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f55600c8830 0x7f5560159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:47.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.905+0000 7f555ffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55600b1410 con 0x7f55600c6c60 2023-10-06T05:07:47.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.905+0000 7f555ffff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55600b2b90 con 0x7f55600c8830 2023-10-06T05:07:47.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.905+0000 7f555ffff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5560063230 con 0x7f55601595a0 2023-10-06T05:07:47.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.906+0000 7f555e7fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f55600c6c60 0x7f55600c8120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34166/0 (socket says 172.21.15.55:34166) 2023-10-06T05:07:47.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.906+0000 7f555e7fc700 1 -- 172.21.15.55:0/3292617767 learned_addr learned my addr 172.21.15.55:0/3292617767 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:47.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.906+0000 7f555effd700 1 --2- 172.21.15.55:0/3292617767 >> v2:172.21.15.163:3300/0 conn(0x7f55601595a0 0x7f55600b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:47.908 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.906+0000 7f555f7fe700 1 --2- 172.21.15.55:0/3292617767 >> v2:172.21.15.55:3300/0 conn(0x7f55600c8830 0x7f5560159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:47.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.906+0000 7f555dffb700 1 -- 172.21.15.55:0/3292617767 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866982114 0 0) 0x7f55600b1410 con 0x7f55600c6c60 2023-10-06T05:07:47.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.906+0000 7f555dffb700 1 -- 172.21.15.55:0/3292617767 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5548003570 con 0x7f55600c6c60 2023-10-06T05:07:47.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.906+0000 7f555f7fe700 1 -- 172.21.15.55:0/3292617767 >> v2:172.21.15.163:3300/0 conn(0x7f55601595a0 msgr2=0x7f55600b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:47.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.906+0000 7f555f7fe700 1 --2- 172.21.15.55:0/3292617767 >> v2:172.21.15.163:3300/0 conn(0x7f55601595a0 0x7f55600b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.906+0000 7f555f7fe700 1 -- 172.21.15.55:0/3292617767 >> v1:172.21.15.55:6789/0 conn(0x7f55600c6c60 legacy=0x7f55600c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:47.910 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.906+0000 7f555f7fe700 1 -- 172.21.15.55:0/3292617767 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55600634f0 con 0x7f55600c8830 2023-10-06T05:07:47.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.907+0000 7f555f7fe700 1 --2- 172.21.15.55:0/3292617767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55600c8830 0x7f5560159970 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f55600c8d10 tx=0x7f5558009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15031faaac1bffa3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:47.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.907+0000 7f555dffb700 1 -- 172.21.15.55:0/3292617767 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555800e070 con 0x7f55600c8830 2023-10-06T05:07:47.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.907+0000 7f555dffb700 1 -- 172.21.15.55:0/3292617767 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555800bab0 con 0x7f55600c8830 2023-10-06T05:07:47.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.907+0000 7f555dffb700 1 -- 172.21.15.55:0/3292617767 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555800be70 con 0x7f55600c8830 2023-10-06T05:07:47.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.908+0000 7f555ffff700 1 -- 172.21.15.55:0/3292617767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55600c8830 msgr2=0x7f5560159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:47.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.908+0000 7f555ffff700 1 --2- 172.21.15.55:0/3292617767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55600c8830 0x7f5560159970 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f55600c8d10 tx=0x7f5558009780 comp rx=0 tx=0).stop 2023-10-06T05:07:47.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.908+0000 7f555ffff700 1 -- 172.21.15.55:0/3292617767 shutdown_connections 2023-10-06T05:07:47.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.908+0000 7f555ffff700 1 --2- 172.21.15.55:0/3292617767 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55600c8830 0x7f5560159970 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.908+0000 7f555ffff700 1 --2- 172.21.15.55:0/3292617767 >> v2:172.21.15.163:3300/0 conn(0x7f55601595a0 0x7f55600b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.908+0000 7f555ffff700 1 -- 172.21.15.55:0/3292617767 >> 172.21.15.55:0/3292617767 conn(0x7f556006ddb0 msgr2=0x7f556006e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:47.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.908+0000 7f555ffff700 1 -- 172.21.15.55:0/3292617767 shutdown_connections 2023-10-06T05:07:47.911 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.908+0000 7f555ffff700 1 -- 172.21.15.55:0/3292617767 wait complete. 2023-10-06T05:07:47.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.909+0000 7f555ffff700 1 Processor -- start 2023-10-06T05:07:47.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.909+0000 7f555ffff700 1 -- start start 2023-10-06T05:07:47.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.910+0000 7f555ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55600c6c60 0x7f55600c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:47.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.910+0000 7f555ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55600c8830 0x7f556016c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:47.913 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.910+0000 7f555ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55601595a0 0x7f5560169dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.910+0000 7f555ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5560159cb0 con 0x7f55600c6c60 2023-10-06T05:07:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.910+0000 7f555ffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5560140350 con 0x7f55601595a0 2023-10-06T05:07:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.910+0000 7f555ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55600c5ca0 con 0x7f55600c8830 2023-10-06T05:07:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.911+0000 7f555effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55600c6c60 0x7f55600c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.911+0000 7f555effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55600c6c60 0x7f55600c5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52266/0 (socket says 172.21.15.55:52266) 2023-10-06T05:07:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.911+0000 7f555effd700 1 -- 172.21.15.55:0/3075862039 learned_addr learned my addr 172.21.15.55:0/3075862039 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.911+0000 7f555e7fc700 1 --2- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55600c8830 0x7f556016c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.911+0000 7f555f7fe700 1 --2- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55601595a0 0x7f5560169dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.911+0000 7f555effd700 1 -- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55600c8830 msgr2=0x7f556016c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.911+0000 7f555effd700 1 --2- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55600c8830 0x7f556016c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.914 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.911+0000 7f555effd700 1 -- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55601595a0 msgr2=0x7f5560169dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:47.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.911+0000 7f555effd700 1 --2- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55601595a0 0x7f5560169dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:47.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.911+0000 7f555effd700 1 -- 172.21.15.55:0/3075862039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5558009a20 con 0x7f55600c6c60 2023-10-06T05:07:47.915 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.912+0000 7f555effd700 1 --2- 172.21.15.55:0/3075862039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55600c6c60 0x7f55600c5760 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f555000e930 tx=0x7f55500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:47.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.912+0000 7f5546ffd700 1 -- 172.21.15.55:0/3075862039 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550013680 con 0x7f55600c6c60 2023-10-06T05:07:47.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.912+0000 7f5546ffd700 1 -- 172.21.15.55:0/3075862039 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55500137e0 con 0x7f55600c6c60 2023-10-06T05:07:47.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.912+0000 7f5546ffd700 1 -- 172.21.15.55:0/3075862039 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550013b80 con 0x7f55600c6c60 2023-10-06T05:07:47.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.912+0000 7f555ffff700 1 -- 172.21.15.55:0/3075862039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5560140b90 con 0x7f55600c6c60 2023-10-06T05:07:47.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.912+0000 7f555ffff700 1 -- 172.21.15.55:0/3075862039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5560140540 con 0x7f55600c6c60 2023-10-06T05:07:47.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.913+0000 7f555ffff700 1 -- 172.21.15.55:0/3075862039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556004c3c0 con 0x7f55600c6c60 2023-10-06T05:07:47.916 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.914+0000 7f5546ffd700 1 -- 172.21.15.55:0/3075862039 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 105) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f555000c410 con 0x7f55600c6c60 2023-10-06T05:07:47.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.915+0000 7f5546ffd700 1 --2- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f554c0670e0 0x7f554c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:47.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.915+0000 7f5546ffd700 1 -- 172.21.15.55:0/3075862039 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f5550084dc0 con 0x7f55600c6c60 2023-10-06T05:07:47.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.915+0000 7f555e7fc700 1 -- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f554c0670e0 msgr2=0x7f554c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:47.917 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.915+0000 7f555e7fc700 1 --2- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f554c0670e0 0x7f554c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:47.921 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:47.919+0000 7f5546ffd700 1 -- 172.21.15.55:0/3075862039 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55500518f0 con 0x7f55600c6c60 2023-10-06T05:07:48.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.116+0000 7f555e7fc700 1 -- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f554c0670e0 msgr2=0x7f554c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:48.118 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.116+0000 7f555e7fc700 1 --2- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f554c0670e0 0x7f554c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:48.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.168+0000 7f555ffff700 1 -- 172.21.15.55:0/3075862039 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55600637d0 con 0x7f55600c6c60 2023-10-06T05:07:48.170 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.168+0000 7f5546ffd700 1 -- 172.21.15.55:0/3075862039 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) v1 ==== 95+0+4616 (secure 0 0 0) 0x7f5550054f10 con 0x7f55600c6c60 2023-10-06T05:07:48.171 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:48.171 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":105,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3704044820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3704044820},{"type":"v1","addr":"172.21.15.163:6834","nonce":3704044820}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":9772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2254022733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2254022733},{"type":"v1","addr":"172.21.15.163:6835","nonce":2254022733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":10224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3844040929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3844040929},{"type":"v1","addr":"172.21.15.55:6834","nonce":3844040929}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T05:07:22.724188+0000","modified":"2023-10-06T05:07:26.255934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10227":{"gid":10227,"name":"a","rank":0,"incarnation":101,"state":"up:active","state_seq":3,"addr":"172.21.15.55:6835/3632968604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":3632968604},{"type":"v1","addr":"172.21.15.55:6835","nonce":3632968604}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-06T05:07:48.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.171+0000 7f555ffff700 1 -- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f554c0670e0 msgr2=0x7f554c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:48.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.171+0000 7f555ffff700 1 --2- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f554c0670e0 0x7f554c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.171+0000 7f555ffff700 1 -- 172.21.15.55:0/3075862039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55600c6c60 msgr2=0x7f55600c5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:48.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.171+0000 7f555ffff700 1 --2- 172.21.15.55:0/3075862039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55600c6c60 0x7f55600c5760 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f555000e930 tx=0x7f55500091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.171+0000 7f555ffff700 1 -- 172.21.15.55:0/3075862039 shutdown_connections 2023-10-06T05:07:48.173 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.171+0000 7f555ffff700 1 --2- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f554c0670e0 0x7f554c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.171+0000 7f555ffff700 1 --2- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f55601595a0 0x7f5560169dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.171+0000 7f555ffff700 1 --2- 172.21.15.55:0/3075862039 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f55600c6c60 0x7f55600c5760 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.171+0000 7f555ffff700 1 --2- 172.21.15.55:0/3075862039 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f55600c8830 0x7f556016c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.171+0000 7f555ffff700 1 -- 172.21.15.55:0/3075862039 >> 172.21.15.55:0/3075862039 conn(0x7f556006ddb0 msgr2=0x7f5560157680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:48.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.171+0000 7f555ffff700 1 -- 172.21.15.55:0/3075862039 shutdown_connections 2023-10-06T05:07:48.174 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.171+0000 7f555ffff700 1 -- 172.21.15.55:0/3075862039 wait complete. 2023-10-06T05:07:48.174 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 105 2023-10-06T05:07:48.190 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:48.189+0000 7f562514f180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:07:48.191 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:07:48.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.419+0000 7f5fe159d700 1 Processor -- start 2023-10-06T05:07:48.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.419+0000 7f5fe159d700 1 -- start start 2023-10-06T05:07:48.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.420+0000 7f5fe159d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f5fdc159420 0x7f5fdc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:48.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.420+0000 7f5fe159d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fdc1597f0 0x7f5fdc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:48.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.420+0000 7f5fe159d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fdc0b11d0 con 0x7f5fdc15d230 2023-10-06T05:07:48.422 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.420+0000 7f5fe159d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fdc0b14b0 con 0x7f5fdc159420 2023-10-06T05:07:48.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.420+0000 7f5fe159d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc0b1790 con 0x7f5fdc1597f0 2023-10-06T05:07:48.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.420+0000 7f5fe0d9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f5fdc15d230 0x7f5fdc0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34180/0 (socket says 172.21.15.55:34180) 2023-10-06T05:07:48.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.420+0000 7f5fe0d9c700 1 -- 172.21.15.55:0/4210374420 learned_addr learned my addr 172.21.15.55:0/4210374420 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:48.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.421+0000 7f5fdb7fe700 1 --2- 172.21.15.55:0/4210374420 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fdc1597f0 0x7f5fdc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:48.423 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.421+0000 7f5fdbfff700 1 --2- 172.21.15.55:0/4210374420 >> v2:172.21.15.163:3300/0 conn(0x7f5fdc159420 0x7f5fdc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:48.424 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.421+0000 7f5fdaffd700 1 -- 172.21.15.55:0/4210374420 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254940752 0 0) 0x7f5fdc0b11d0 con 0x7f5fdc15d230 2023-10-06T05:07:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.421+0000 7f5fdaffd700 1 -- 172.21.15.55:0/4210374420 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fc4003570 con 0x7f5fdc15d230 2023-10-06T05:07:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.421+0000 7f5fdb7fe700 1 -- 172.21.15.55:0/4210374420 >> v2:172.21.15.163:3300/0 conn(0x7f5fdc159420 msgr2=0x7f5fdc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.421+0000 7f5fdb7fe700 1 --2- 172.21.15.55:0/4210374420 >> v2:172.21.15.163:3300/0 conn(0x7f5fdc159420 0x7f5fdc0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.421+0000 7f5fdb7fe700 1 -- 172.21.15.55:0/4210374420 >> v1:172.21.15.55:6789/0 conn(0x7f5fdc15d230 legacy=0x7f5fdc0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.421+0000 7f5fdb7fe700 1 -- 172.21.15.55:0/4210374420 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc0b1a70 con 0x7f5fdc1597f0 2023-10-06T05:07:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.421+0000 7f5fdbfff700 1 --2- 172.21.15.55:0/4210374420 >> v2:172.21.15.163:3300/0 conn(0x7f5fdc159420 0x7f5fdc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.421+0000 7f5fdb7fe700 1 --2- 172.21.15.55:0/4210374420 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fdc1597f0 0x7f5fdc0b2270 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00098e0 tx=0x7f5fd00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e37a2fe947fb67d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.422+0000 7f5fdaffd700 1 -- 172.21.15.55:0/4210374420 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0011630 con 0x7f5fdc1597f0 2023-10-06T05:07:48.425 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.422+0000 7f5fdaffd700 1 -- 172.21.15.55:0/4210374420 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd0011790 con 0x7f5fdc1597f0 2023-10-06T05:07:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.422+0000 7f5fdaffd700 1 -- 172.21.15.55:0/4210374420 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0011b30 con 0x7f5fdc1597f0 2023-10-06T05:07:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.422+0000 7f5fe159d700 1 -- 172.21.15.55:0/4210374420 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fdc1597f0 msgr2=0x7f5fdc0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.422+0000 7f5fe159d700 1 --2- 172.21.15.55:0/4210374420 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fdc1597f0 0x7f5fdc0b2270 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5fd00098e0 tx=0x7f5fd00091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.422+0000 7f5fe159d700 1 -- 172.21.15.55:0/4210374420 shutdown_connections 2023-10-06T05:07:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.422+0000 7f5fe159d700 1 --2- 172.21.15.55:0/4210374420 >> v2:172.21.15.163:3300/0 conn(0x7f5fdc159420 0x7f5fdc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.422+0000 7f5fe159d700 1 --2- 172.21.15.55:0/4210374420 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fdc1597f0 0x7f5fdc0b2270 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.422+0000 7f5fe159d700 1 -- 172.21.15.55:0/4210374420 >> 172.21.15.55:0/4210374420 conn(0x7f5fdc0d0660 msgr2=0x7f5fdc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.422+0000 7f5fe159d700 1 -- 172.21.15.55:0/4210374420 shutdown_connections 2023-10-06T05:07:48.426 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.423+0000 7f5fe159d700 1 -- 172.21.15.55:0/4210374420 wait complete. 2023-10-06T05:07:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.424+0000 7f5fe159d700 1 Processor -- start 2023-10-06T05:07:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.424+0000 7f5fe159d700 1 -- start start 2023-10-06T05:07:48.428 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.425+0000 7f5fe159d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fdc159420 0x7f5fdc0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.425+0000 7f5fe159d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fdc1597f0 0x7f5fdc16d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.425+0000 7f5fe159d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fdc15d230 0x7f5fdc143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.425+0000 7f5fe159d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc157220 con 0x7f5fdc15d230 2023-10-06T05:07:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.425+0000 7f5fe159d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc157470 con 0x7f5fdc159420 2023-10-06T05:07:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.425+0000 7f5fe159d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc157750 con 0x7f5fdc1597f0 2023-10-06T05:07:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.425+0000 7f5fe0d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fdc15d230 0x7f5fdc143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.425+0000 7f5fe0d9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fdc15d230 0x7f5fdc143d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52278/0 (socket says 172.21.15.55:52278) 2023-10-06T05:07:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.425+0000 7f5fe0d9c700 1 -- 172.21.15.55:0/2466878493 learned_addr learned my addr 172.21.15.55:0/2466878493 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.425+0000 7f5fdbfff700 1 --2- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fdc159420 0x7f5fdc0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.425+0000 7f5fdb7fe700 1 --2- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fdc1597f0 0x7f5fdc16d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:48.429 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.426+0000 7f5fe0d9c700 1 -- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fdc1597f0 msgr2=0x7f5fdc16d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:48.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.426+0000 7f5fe0d9c700 1 --2- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fdc1597f0 0x7f5fdc16d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.426+0000 7f5fe0d9c700 1 -- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fdc159420 msgr2=0x7f5fdc0c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:48.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.426+0000 7f5fe0d9c700 1 --2- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fdc159420 0x7f5fdc0c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.426+0000 7f5fe0d9c700 1 -- 172.21.15.55:0/2466878493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc009420 con 0x7f5fdc15d230 2023-10-06T05:07:48.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.426+0000 7f5fe0d9c700 1 --2- 172.21.15.55:0/2466878493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fdc15d230 0x7f5fdc143d40 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4001460 tx=0x7f5fd4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:48.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.426+0000 7f5fc3fff700 1 -- 172.21.15.55:0/2466878493 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd400a870 con 0x7f5fdc15d230 2023-10-06T05:07:48.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.426+0000 7f5fc3fff700 1 -- 172.21.15.55:0/2466878493 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd400a9d0 con 0x7f5fdc15d230 2023-10-06T05:07:48.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.426+0000 7f5fe159d700 1 -- 172.21.15.55:0/2466878493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd0009420 con 0x7f5fdc15d230 2023-10-06T05:07:48.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.426+0000 7f5fc3fff700 1 -- 172.21.15.55:0/2466878493 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd400ad50 con 0x7f5fdc15d230 2023-10-06T05:07:48.430 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.426+0000 7f5fe159d700 1 -- 172.21.15.55:0/2466878493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fdc157d10 con 0x7f5fdc15d230 2023-10-06T05:07:48.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.427+0000 7f5fe159d700 1 -- 172.21.15.55:0/2466878493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fdc04c3c0 con 0x7f5fdc15d230 2023-10-06T05:07:48.431 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.429+0000 7f5fc3fff700 1 -- 172.21.15.55:0/2466878493 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 105) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f5fd4007300 con 0x7f5fdc15d230 2023-10-06T05:07:48.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.430+0000 7f5fc3fff700 1 --2- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5fc80670e0 0x7f5fc8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:48.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.430+0000 7f5fc3fff700 1 -- 172.21.15.55:0/2466878493 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f5fd4084fb0 con 0x7f5fdc15d230 2023-10-06T05:07:48.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.430+0000 7f5fdbfff700 1 -- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5fc80670e0 msgr2=0x7f5fc8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:48.432 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.430+0000 7f5fdbfff700 1 --2- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5fc80670e0 0x7f5fc8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:48.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.434+0000 7f5fc3fff700 1 -- 172.21.15.55:0/2466878493 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fd4051b60 con 0x7f5fdc15d230 2023-10-06T05:07:48.482 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:48.481+0000 7f562514f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:07:48.488 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:48.486+0000 7f1696d46180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T05:07:48.577 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:48.574+0000 7f562514f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:07:48.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.631+0000 7f5fdbfff700 1 -- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5fc80670e0 msgr2=0x7f5fc8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:48.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.631+0000 7f5fdbfff700 1 --2- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5fc80670e0 0x7f5fc8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:48.642 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.640+0000 7f5fe159d700 1 -- 172.21.15.55:0/2466878493 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fdc0b1d50 con 0x7f5fdc15d230 2023-10-06T05:07:48.643 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.641+0000 7f5fc3fff700 1 -- 172.21.15.55:0/2466878493 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v208) v1 ==== 74+0+17989 (secure 0 0 0) 0x7f5fd4055180 con 0x7f5fdc15d230 2023-10-06T05:07:48.644 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:48.644 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":208,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:07:39.103933+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-10-06T05:07:20.916360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-10-06T05:07:21.706951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"207","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/2709454586":"2023-10-07T05:07:39.103885+0000","172.21.15.163:6837/101089":"2023-10-07T05:07:39.103885+0000","172.21.15.163:6836/101089":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/2719429847":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/3618319475":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/2222063413":"2023-10-07T05:07:39.103885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:07:48.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.643+0000 7f5fe159d700 1 -- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5fc80670e0 msgr2=0x7f5fc8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:48.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.643+0000 7f5fe159d700 1 --2- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5fc80670e0 0x7f5fc8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.646 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.644+0000 7f5fe159d700 1 -- 172.21.15.55:0/2466878493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fdc15d230 msgr2=0x7f5fdc143d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:48.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.644+0000 7f5fe159d700 1 --2- 172.21.15.55:0/2466878493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fdc15d230 0x7f5fdc143d40 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4001460 tx=0x7f5fd4001a40 comp rx=0 tx=0).stop 2023-10-06T05:07:48.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.644+0000 7f5fe159d700 1 -- 172.21.15.55:0/2466878493 shutdown_connections 2023-10-06T05:07:48.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.644+0000 7f5fe159d700 1 --2- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f5fc80670e0 0x7f5fc8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.644+0000 7f5fe159d700 1 --2- 172.21.15.55:0/2466878493 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f5fdc15d230 0x7f5fdc143d40 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.644+0000 7f5fe159d700 1 --2- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f5fdc159420 0x7f5fdc0c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.644+0000 7f5fe159d700 1 --2- 172.21.15.55:0/2466878493 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f5fdc1597f0 0x7f5fdc16d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.644+0000 7f5fe159d700 1 -- 172.21.15.55:0/2466878493 >> 172.21.15.55:0/2466878493 conn(0x7f5fdc0d0660 msgr2=0x7f5fdc15d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:48.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.644+0000 7f5fe159d700 1 -- 172.21.15.55:0/2466878493 shutdown_connections 2023-10-06T05:07:48.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.644+0000 7f5fe159d700 1 -- 172.21.15.55:0/2466878493 wait complete. 2023-10-06T05:07:48.665 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T05:07:48.728 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:48.727+0000 7f562514f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:07:48.785 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:48.783+0000 7f1696d46180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T05:07:48.833 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:48.832+0000 7f562514f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:07:48.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.853+0000 7f03eb9ff700 1 Processor -- start 2023-10-06T05:07:48.855 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.853+0000 7f03eb9ff700 1 -- start start 2023-10-06T05:07:48.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.854+0000 7f03eb9ff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f03e4159890 0x7f03e415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:48.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.854+0000 7f03eb9ff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e415a160 con 0x7f03e4069560 2023-10-06T05:07:48.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.854+0000 7f03eb9ff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e40d13a0 con 0x7f03e4159c60 2023-10-06T05:07:48.856 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.854+0000 7f03eb9ff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e415e140 con 0x7f03e4159890 2023-10-06T05:07:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.855+0000 7f03e979b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f03e4159890 0x7f03e415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.855+0000 7f03e9f9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f03e4069560 0x7f03e40ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34186/0 (socket says 172.21.15.55:34186) 2023-10-06T05:07:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.855+0000 7f03e9f9c700 1 -- 172.21.15.55:0/3691128079 learned_addr learned my addr 172.21.15.55:0/3691128079 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.855+0000 7f03e979b700 1 -- 172.21.15.55:0/3691128079 >> v1:172.21.15.163:6789/0 conn(0x7f03e4159c60 legacy=0x7f03e415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.855+0000 7f03e979b700 1 -- 172.21.15.55:0/3691128079 >> v1:172.21.15.55:6789/0 conn(0x7f03e4069560 legacy=0x7f03e40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.855+0000 7f03e979b700 1 -- 172.21.15.55:0/3691128079 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e415a3d0 con 0x7f03e4159890 2023-10-06T05:07:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.855+0000 7f03dbfff700 1 -- 172.21.15.55:0/3691128079 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033515987 0 0) 0x7f03e40d13a0 con 0x7f03e4159c60 2023-10-06T05:07:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.855+0000 7f03e979b700 1 --2- 172.21.15.55:0/3691128079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4159890 0x7f03e415abd0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f03e00098e0 tx=0x7f03e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96024df48a910b97 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.856+0000 7f03dbfff700 1 -- 172.21.15.55:0/3691128079 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0009e80 con 0x7f03e4159890 2023-10-06T05:07:48.858 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.856+0000 7f03dbfff700 1 -- 172.21.15.55:0/3691128079 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e000b760 con 0x7f03e4159890 2023-10-06T05:07:48.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.856+0000 7f03dbfff700 1 -- 172.21.15.55:0/3691128079 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e000bae0 con 0x7f03e4159890 2023-10-06T05:07:48.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.856+0000 7f03eb9ff700 1 -- 172.21.15.55:0/3691128079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4159890 msgr2=0x7f03e415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:48.859 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.856+0000 7f03eb9ff700 1 --2- 172.21.15.55:0/3691128079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4159890 0x7f03e415abd0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f03e00098e0 tx=0x7f03e00091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.856+0000 7f03eb9ff700 1 -- 172.21.15.55:0/3691128079 shutdown_connections 2023-10-06T05:07:48.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.856+0000 7f03eb9ff700 1 --2- 172.21.15.55:0/3691128079 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4159890 0x7f03e415abd0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.857+0000 7f03eb9ff700 1 -- 172.21.15.55:0/3691128079 >> 172.21.15.55:0/3691128079 conn(0x7f03e4068e20 msgr2=0x7f03e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:48.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.857+0000 7f03eb9ff700 1 -- 172.21.15.55:0/3691128079 shutdown_connections 2023-10-06T05:07:48.860 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.857+0000 7f03eb9ff700 1 -- 172.21.15.55:0/3691128079 wait complete. 2023-10-06T05:07:48.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.859+0000 7f03eb9ff700 1 Processor -- start 2023-10-06T05:07:48.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.859+0000 7f03eb9ff700 1 -- start start 2023-10-06T05:07:48.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.860+0000 7f03eb9ff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03e4069560 0x7f03e40c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:48.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.860+0000 7f03eb9ff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03e4159890 0x7f03e4144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:48.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.860+0000 7f03eb9ff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4159c60 0x7f03e416d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:48.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.860+0000 7f03eb9ff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40ce720 con 0x7f03e4159c60 2023-10-06T05:07:48.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.860+0000 7f03eb9ff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40cebb0 con 0x7f03e4159890 2023-10-06T05:07:48.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.860+0000 7f03eb9ff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e40cdb40 con 0x7f03e4069560 2023-10-06T05:07:48.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.861+0000 7f03e9f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4159c60 0x7f03e416d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:48.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.861+0000 7f03e9f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4159c60 0x7f03e416d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52290/0 (socket says 172.21.15.55:52290) 2023-10-06T05:07:48.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.861+0000 7f03e9f9c700 1 -- 172.21.15.55:0/229118906 learned_addr learned my addr 172.21.15.55:0/229118906 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.861+0000 7f03e8f9a700 1 --2- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03e4159890 0x7f03e4144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.861+0000 7f03e979b700 1 --2- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03e4069560 0x7f03e40c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.861+0000 7f03e9f9c700 1 -- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03e4069560 msgr2=0x7f03e40c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.861+0000 7f03e9f9c700 1 --2- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03e4069560 0x7f03e40c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.861+0000 7f03e9f9c700 1 -- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03e4159890 msgr2=0x7f03e4144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.861+0000 7f03e9f9c700 1 --2- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03e4159890 0x7f03e4144130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.861+0000 7f03e9f9c700 1 -- 172.21.15.55:0/229118906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e0009420 con 0x7f03e4159c60 2023-10-06T05:07:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.861+0000 7f03e9f9c700 1 --2- 172.21.15.55:0/229118906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4159c60 0x7f03e416d2b0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00cd50 tx=0x7f03dc001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.862+0000 7f03d97fa700 1 -- 172.21.15.55:0/229118906 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc00ce50 con 0x7f03e4159c60 2023-10-06T05:07:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.862+0000 7f03d97fa700 1 -- 172.21.15.55:0/229118906 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc00aac0 con 0x7f03e4159c60 2023-10-06T05:07:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.862+0000 7f03d97fa700 1 -- 172.21.15.55:0/229118906 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc00ae80 con 0x7f03e4159c60 2023-10-06T05:07:48.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.862+0000 7f03eb9ff700 1 -- 172.21.15.55:0/229118906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e40cddf0 con 0x7f03e4159c60 2023-10-06T05:07:48.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.862+0000 7f03eb9ff700 1 -- 172.21.15.55:0/229118906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e40ce3b0 con 0x7f03e4159c60 2023-10-06T05:07:48.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.863+0000 7f03eb9ff700 1 -- 172.21.15.55:0/229118906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e40ce8d0 con 0x7f03e4159c60 2023-10-06T05:07:48.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.864+0000 7f03d97fa700 1 -- 172.21.15.55:0/229118906 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 105) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f03dc007300 con 0x7f03e4159c60 2023-10-06T05:07:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.865+0000 7f03d97fa700 1 --2- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f03c80670e0 0x7f03c8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.865+0000 7f03d97fa700 1 -- 172.21.15.55:0/229118906 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f03dc084fa0 con 0x7f03e4159c60 2023-10-06T05:07:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.865+0000 7f03e979b700 1 -- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f03c80670e0 msgr2=0x7f03c8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:48.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.866+0000 7f03e979b700 1 --2- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f03c80670e0 0x7f03c8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T05:07:48.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:48.870+0000 7f03d97fa700 1 -- 172.21.15.55:0/229118906 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03e40ce8d0 con 0x7f03e4159c60 2023-10-06T05:07:48.886 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:48.885+0000 7f1696d46180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T05:07:49.026 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:49.025+0000 7f562514f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:07:49.050 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:49.048+0000 7f1696d46180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T05:07:49.068 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.067+0000 7f03e979b700 1 -- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f03c80670e0 msgr2=0x7f03c8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.163:6836/101089 2023-10-06T05:07:49.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.067+0000 7f03e979b700 1 --2- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f03c80670e0 0x7f03c8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-06T05:07:49.120 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:49.118+0000 7f562514f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:07:49.122 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.120+0000 7f03eb9ff700 1 -- 172.21.15.55:0/229118906 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f03e404c3c0 con 0x7f03e4159c60 2023-10-06T05:07:49.122 INFO:tasks.ceph.mon.a.smithi055.stderr:2023-10-06T05:07:49.121+0000 7fe524ff1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T05:07:49.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.127+0000 7f03d97fa700 1 -- 172.21.15.55:0/229118906 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) v1 ==== 161+0+0 (secure 0 0 0) 0x7f03dc055680 con 0x7f03e4159c60 2023-10-06T05:07:49.131 INFO:tasks.ceph.mds.a.smithi055.stderr: -3621> 2023-10-06T05:07:17.479+0000 7f37065fd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-06T05:07:49.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.130+0000 7f03eb9ff700 1 -- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f03c80670e0 msgr2=0x7f03c8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T05:07:49.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.130+0000 7f03eb9ff700 1 --2- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f03c80670e0 0x7f03c8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:49.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.130+0000 7f03eb9ff700 1 -- 172.21.15.55:0/229118906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4159c60 msgr2=0x7f03e416d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:49.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.130+0000 7f03eb9ff700 1 --2- 172.21.15.55:0/229118906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4159c60 0x7f03e416d2b0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00cd50 tx=0x7f03dc001ab0 comp rx=0 tx=0).stop 2023-10-06T05:07:49.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.130+0000 7f03eb9ff700 1 -- 172.21.15.55:0/229118906 shutdown_connections 2023-10-06T05:07:49.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.130+0000 7f03eb9ff700 1 --2- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f03c80670e0 0x7f03c8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:49.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.130+0000 7f03eb9ff700 1 --2- 172.21.15.55:0/229118906 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f03e4159c60 0x7f03e416d2b0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:49.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.130+0000 7f03eb9ff700 1 --2- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f03e4069560 0x7f03e40c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:49.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.130+0000 7f03eb9ff700 1 --2- 172.21.15.55:0/229118906 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f03e4159890 0x7f03e4144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:49.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.131+0000 7f03eb9ff700 1 -- 172.21.15.55:0/229118906 >> 172.21.15.55:0/229118906 conn(0x7f03e4068e20 msgr2=0x7f03e4152d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:49.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.131+0000 7f03eb9ff700 1 -- 172.21.15.55:0/229118906 shutdown_connections 2023-10-06T05:07:49.134 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.131+0000 7f03eb9ff700 1 -- 172.21.15.55:0/229118906 wait complete. 2023-10-06T05:07:49.134 INFO:teuthology.orchestra.run.smithi055.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T05:07:49.135 INFO:tasks.ceph.mds.a.smithi055.stderr: -2423> 2023-10-06T05:07:29.198+0000 7f37005f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T05:07:49.156 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T05:07:49.165 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:49.163+0000 7f1696d46180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T05:07:49.181 INFO:tasks.ceph.mds.a.smithi055.stdout:starting mds.a at 2023-10-06T05:07:49.216 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:49.215+0000 7f562514f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:07:49.312 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:49.310+0000 7f562514f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:07:49.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.354+0000 7fa5ebbff700 1 Processor -- start 2023-10-06T05:07:49.356 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.354+0000 7fa5ebbff700 1 -- start start 2023-10-06T05:07:49.358 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.356+0000 7fa5ebbff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa5e4159c60 0x7fa5e415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:49.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.356+0000 7fa5ebbff700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa5e415db10 0x7fa5e4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:49.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.356+0000 7fa5ebbff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e415a160 con 0x7fa5e4159890 2023-10-06T05:07:49.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.356+0000 7fa5ebbff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e415df50 con 0x7fa5e4159c60 2023-10-06T05:07:49.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.356+0000 7fa5ebbff700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5e40d13a0 con 0x7fa5e415db10 2023-10-06T05:07:49.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.356+0000 7fa5e919a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa5e4159c60 0x7fa5e415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:49.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.356+0000 7fa5e919a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa5e4159c60 0x7fa5e415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52312/0 (socket says 172.21.15.55:52312) 2023-10-06T05:07:49.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.356+0000 7fa5e919a700 1 -- 172.21.15.55:0/1398059465 learned_addr learned my addr 172.21.15.55:0/1398059465 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:49.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.356+0000 7fa5ea19c700 1 --2- 172.21.15.55:0/1398059465 >> v2:172.21.15.163:3300/0 conn(0x7fa5e415db10 0x7fa5e4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:49.359 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.356+0000 7fa5e999b700 1 --1- 172.21.15.55:0/1398059465 >> v1:172.21.15.55:6789/0 conn(0x7fa5e4159890 0x7fa5e415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34194/0 (socket says 172.21.15.55:34194) 2023-10-06T05:07:49.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.357+0000 7fa5e919a700 1 -- 172.21.15.55:0/1398059465 >> v2:172.21.15.163:3300/0 conn(0x7fa5e415db10 msgr2=0x7fa5e4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:49.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.357+0000 7fa5e919a700 1 --2- 172.21.15.55:0/1398059465 >> v2:172.21.15.163:3300/0 conn(0x7fa5e415db10 0x7fa5e4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:49.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.357+0000 7fa5e919a700 1 -- 172.21.15.55:0/1398059465 >> v1:172.21.15.55:6789/0 conn(0x7fa5e4159890 legacy=0x7fa5e415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:49.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.357+0000 7fa5e919a700 1 -- 172.21.15.55:0/1398059465 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e415e140 con 0x7fa5e4159c60 2023-10-06T05:07:49.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.357+0000 7fa5ea19c700 1 --2- 172.21.15.55:0/1398059465 >> v2:172.21.15.163:3300/0 conn(0x7fa5e415db10 0x7fa5e4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:49.360 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.357+0000 7fa5e919a700 1 --2- 172.21.15.55:0/1398059465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5e4159c60 0x7fa5e415d5d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fa5e406a890 tx=0x7fa5d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d933b842426df9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:49.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.357+0000 7fa5e8999700 1 -- 172.21.15.55:0/1398059465 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d4009e70 con 0x7fa5e4159c60 2023-10-06T05:07:49.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.357+0000 7fa5e8999700 1 -- 172.21.15.55:0/1398059465 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d400b860 con 0x7fa5e4159c60 2023-10-06T05:07:49.361 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.357+0000 7fa5e8999700 1 -- 172.21.15.55:0/1398059465 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d400bc00 con 0x7fa5e4159c60 2023-10-06T05:07:49.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.358+0000 7fa5ebbff700 1 -- 172.21.15.55:0/1398059465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5e4159c60 msgr2=0x7fa5e415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:49.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.358+0000 7fa5ebbff700 1 --2- 172.21.15.55:0/1398059465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5e4159c60 0x7fa5e415d5d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fa5e406a890 tx=0x7fa5d4009250 comp rx=0 tx=0).stop 2023-10-06T05:07:49.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.358+0000 7fa5ebbff700 1 -- 172.21.15.55:0/1398059465 shutdown_connections 2023-10-06T05:07:49.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.358+0000 7fa5ebbff700 1 --2- 172.21.15.55:0/1398059465 >> v2:172.21.15.163:3300/0 conn(0x7fa5e415db10 0x7fa5e4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:49.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.358+0000 7fa5ebbff700 1 --2- 172.21.15.55:0/1398059465 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5e4159c60 0x7fa5e415d5d0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:49.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.358+0000 7fa5ebbff700 1 -- 172.21.15.55:0/1398059465 >> 172.21.15.55:0/1398059465 conn(0x7fa5e4068e20 msgr2=0x7fa5e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:49.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.358+0000 7fa5ebbff700 1 -- 172.21.15.55:0/1398059465 shutdown_connections 2023-10-06T05:07:49.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.358+0000 7fa5ebbff700 1 -- 172.21.15.55:0/1398059465 wait complete. 2023-10-06T05:07:49.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.360+0000 7fa5ebbff700 1 Processor -- start 2023-10-06T05:07:49.363 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.361+0000 7fa5ebbff700 1 -- start start 2023-10-06T05:07:49.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.362+0000 7fa5ebbff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e4159890 0x7fa5e40cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:49.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.362+0000 7fa5ebbff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e4159c60 0x7fa5e40cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:49.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.362+0000 7fa5ebbff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5e415db10 0x7fa5e40b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:49.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.362+0000 7fa5ebbff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e40cdfc0 con 0x7fa5e415db10 2023-10-06T05:07:49.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.362+0000 7fa5ebbff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e40cff80 con 0x7fa5e4159c60 2023-10-06T05:07:49.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.362+0000 7fa5ebbff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e4150e80 con 0x7fa5e4159890 2023-10-06T05:07:49.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.363+0000 7fa5e999b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e4159890 0x7fa5e40cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:49.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.363+0000 7fa5ea19c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5e415db10 0x7fa5e40b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:49.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.363+0000 7fa5e999b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e4159890 0x7fa5e40cda80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:41252/0 (socket says 172.21.15.55:41252) 2023-10-06T05:07:49.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.363+0000 7fa5e999b700 1 -- 172.21.15.55:0/960449096 learned_addr learned my addr 172.21.15.55:0/960449096 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:49.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.363+0000 7fa5e919a700 1 --2- 172.21.15.55:0/960449096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e4159c60 0x7fa5e40cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:49.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.363+0000 7fa5ea19c700 1 -- 172.21.15.55:0/960449096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e4159890 msgr2=0x7fa5e40cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:49.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.363+0000 7fa5ea19c700 1 --2- 172.21.15.55:0/960449096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e4159890 0x7fa5e40cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:49.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.363+0000 7fa5ea19c700 1 -- 172.21.15.55:0/960449096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e4159c60 msgr2=0x7fa5e40cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:49.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.363+0000 7fa5ea19c700 1 --2- 172.21.15.55:0/960449096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e4159c60 0x7fa5e40cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:49.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.363+0000 7fa5ea19c700 1 -- 172.21.15.55:0/960449096 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc009a20 con 0x7fa5e415db10 2023-10-06T05:07:49.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.363+0000 7fa5ea19c700 1 --2- 172.21.15.55:0/960449096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5e415db10 0x7fa5e40b0db0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009ed0 tx=0x7fa5dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:49.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.364+0000 7fa5d9ffb700 1 -- 172.21.15.55:0/960449096 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc00cb70 con 0x7fa5e415db10 2023-10-06T05:07:49.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.364+0000 7fa5d9ffb700 1 -- 172.21.15.55:0/960449096 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5dc00f040 con 0x7fa5e415db10 2023-10-06T05:07:49.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.364+0000 7fa5d9ffb700 1 -- 172.21.15.55:0/960449096 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5dc014480 con 0x7fa5e415db10 2023-10-06T05:07:49.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.364+0000 7fa5ebbff700 1 -- 172.21.15.55:0/960449096 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d40094f0 con 0x7fa5e415db10 2023-10-06T05:07:49.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.364+0000 7fa5ebbff700 1 -- 172.21.15.55:0/960449096 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e40ce4b0 con 0x7fa5e415db10 2023-10-06T05:07:49.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.365+0000 7fa5ebbff700 1 -- 172.21.15.55:0/960449096 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e4151030 con 0x7fa5e415db10 2023-10-06T05:07:49.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.366+0000 7fa5d9ffb700 1 -- 172.21.15.55:0/960449096 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 106) v1 ==== 84451+0+0 (secure 0 0 0) 0x7fa5dc003020 con 0x7fa5e415db10 2023-10-06T05:07:49.369 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.367+0000 7fa5d9ffb700 1 -- 172.21.15.55:0/960449096 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 7151+0+0 (secure 0 0 0) 0x7fa5dc084ce0 con 0x7fa5e415db10 2023-10-06T05:07:49.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.371+0000 7fa5d9ffb700 1 -- 172.21.15.55:0/960449096 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa5e4151030 con 0x7fa5e415db10 2023-10-06T05:07:49.381 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:49.380+0000 7f1696d46180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T05:07:49.393 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:49.391+0000 7f55a3cca700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:49.393 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:49.391+0000 7f55a3cca700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:49.393 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:49.391+0000 7f55a3cca700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:49.393 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:49.391+0000 7f55a3cca700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:49.394 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:49.391+0000 7f55a3cca700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:49.394 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:49.393+0000 7f55a14c5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:49.394 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:49.393+0000 7f55a14c5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:49.394 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:49.393+0000 7f55a14c5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:49.395 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:49.393+0000 7f55a14c5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:49.395 INFO:tasks.ceph.mgr.x.smithi163.stderr:2023-10-06T05:07:49.393+0000 7f55a14c5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-06T05:07:49.485 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:49.482+0000 7f1696d46180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T05:07:49.581 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:49.580+0000 7f1696d46180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T05:07:49.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:49.622+0000 7fa5ebbff700 1 -- 172.21.15.55:0/960449096 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa5e40ce4b0 con 0x7fa5e415db10 2023-10-06T05:07:49.680 INFO:tasks.ceph.mgr.y.smithi055.stderr:2023-10-06T05:07:49.678+0000 7f1696d46180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T05:07:50.129 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.128+0000 7fa5d9ffb700 1 -- 172.21.15.55:0/960449096 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v107) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa5e40ce4b0 con 0x7fa5e415db10 2023-10-06T05:07:50.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.130+0000 7fa5ebbff700 1 -- 172.21.15.55:0/960449096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5e415db10 msgr2=0x7fa5e40b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:50.132 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.130+0000 7fa5ebbff700 1 --2- 172.21.15.55:0/960449096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5e415db10 0x7fa5e40b0db0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fa5dc009ed0 tx=0x7fa5dc00b040 comp rx=0 tx=0).stop 2023-10-06T05:07:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.130+0000 7fa5ebbff700 1 -- 172.21.15.55:0/960449096 shutdown_connections 2023-10-06T05:07:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.130+0000 7fa5ebbff700 1 --2- 172.21.15.55:0/960449096 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa5e415db10 0x7fa5e40b0db0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.130+0000 7fa5ebbff700 1 --2- 172.21.15.55:0/960449096 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa5e4159890 0x7fa5e40cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.130+0000 7fa5ebbff700 1 --2- 172.21.15.55:0/960449096 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa5e4159c60 0x7fa5e40cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.130+0000 7fa5ebbff700 1 -- 172.21.15.55:0/960449096 >> 172.21.15.55:0/960449096 conn(0x7fa5e4068e20 msgr2=0x7fa5e4153170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.130+0000 7fa5ebbff700 1 -- 172.21.15.55:0/960449096 shutdown_connections 2023-10-06T05:07:50.133 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.131+0000 7fa5ebbff700 1 -- 172.21.15.55:0/960449096 wait complete. 2023-10-06T05:07:50.151 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T05:07:50.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.388+0000 7fb031f9d700 1 Processor -- start 2023-10-06T05:07:50.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.388+0000 7fb031f9d700 1 -- start start 2023-10-06T05:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.389+0000 7fb031f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fb02c15b050 0x7fb02c0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:50.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.389+0000 7fb031f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb02c15bb30 0x7fb02c0ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.389+0000 7fb031f9d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb02c157a50 con 0x7fb02c0cc8d0 2023-10-06T05:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.389+0000 7fb031f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb02c158f10 con 0x7fb02c15bb30 2023-10-06T05:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.389+0000 7fb031f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb02c1591b0 con 0x7fb02c15b050 2023-10-06T05:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.390+0000 7fb03179c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fb02c15bb30 0x7fb02c0ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.390+0000 7fb02bfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fb02c0cc8d0 0x7fb02c15b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34198/0 (socket says 172.21.15.55:34198) 2023-10-06T05:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.390+0000 7fb02bfff700 1 -- 172.21.15.55:0/1027842869 learned_addr learned my addr 172.21.15.55:0/1027842869 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.390+0000 7fb030f9b700 1 --2- 172.21.15.55:0/1027842869 >> v2:172.21.15.163:3300/0 conn(0x7fb02c15b050 0x7fb02c0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.390+0000 7fb02b7fe700 1 -- 172.21.15.55:0/1027842869 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881407775 0 0) 0x7fb02c157a50 con 0x7fb02c0cc8d0 2023-10-06T05:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.390+0000 7fb02b7fe700 1 -- 172.21.15.55:0/1027842869 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb01c003570 con 0x7fb02c0cc8d0 2023-10-06T05:07:50.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.390+0000 7fb03179c700 1 -- 172.21.15.55:0/1027842869 >> v2:172.21.15.163:3300/0 conn(0x7fb02c15b050 msgr2=0x7fb02c0ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.390+0000 7fb03179c700 1 --2- 172.21.15.55:0/1027842869 >> v2:172.21.15.163:3300/0 conn(0x7fb02c15b050 0x7fb02c0ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.390+0000 7fb03179c700 1 -- 172.21.15.55:0/1027842869 >> v1:172.21.15.55:6789/0 conn(0x7fb02c0cc8d0 legacy=0x7fb02c15b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.390+0000 7fb03179c700 1 -- 172.21.15.55:0/1027842869 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c0ca9e0 con 0x7fb02c15bb30 2023-10-06T05:07:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.391+0000 7fb03179c700 1 --2- 172.21.15.55:0/1027842869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb02c15bb30 0x7fb02c0ada30 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fb02c064f60 tx=0x7fb024009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d3cff7e8a9707c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.391+0000 7fb02b7fe700 1 -- 172.21.15.55:0/1027842869 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02400e070 con 0x7fb02c15bb30 2023-10-06T05:07:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.391+0000 7fb02b7fe700 1 -- 172.21.15.55:0/1027842869 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02400bab0 con 0x7fb02c15bb30 2023-10-06T05:07:50.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.391+0000 7fb02b7fe700 1 -- 172.21.15.55:0/1027842869 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02400be70 con 0x7fb02c15bb30 2023-10-06T05:07:50.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.391+0000 7fb031f9d700 1 -- 172.21.15.55:0/1027842869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb02c15bb30 msgr2=0x7fb02c0ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:50.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.391+0000 7fb031f9d700 1 --2- 172.21.15.55:0/1027842869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb02c15bb30 0x7fb02c0ada30 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fb02c064f60 tx=0x7fb024009780 comp rx=0 tx=0).stop 2023-10-06T05:07:50.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.391+0000 7fb031f9d700 1 -- 172.21.15.55:0/1027842869 shutdown_connections 2023-10-06T05:07:50.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.391+0000 7fb031f9d700 1 --2- 172.21.15.55:0/1027842869 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb02c15bb30 0x7fb02c0ada30 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:50.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.391+0000 7fb031f9d700 1 --2- 172.21.15.55:0/1027842869 >> v2:172.21.15.163:3300/0 conn(0x7fb02c15b050 0x7fb02c0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:50.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.391+0000 7fb031f9d700 1 -- 172.21.15.55:0/1027842869 >> 172.21.15.55:0/1027842869 conn(0x7fb02c068e20 msgr2=0x7fb02c0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:50.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.392+0000 7fb031f9d700 1 -- 172.21.15.55:0/1027842869 shutdown_connections 2023-10-06T05:07:50.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.392+0000 7fb031f9d700 1 -- 172.21.15.55:0/1027842869 wait complete. 2023-10-06T05:07:50.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.393+0000 7fb031f9d700 1 Processor -- start 2023-10-06T05:07:50.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.393+0000 7fb031f9d700 1 -- start start 2023-10-06T05:07:50.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.394+0000 7fb031f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb02c0cc8d0 0x7fb02c150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:50.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.394+0000 7fb031f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb02c15b050 0x7fb02c0c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:50.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.394+0000 7fb031f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb02c15bb30 0x7fb02c0c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:50.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.394+0000 7fb031f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c0c1280 con 0x7fb02c0cc8d0 2023-10-06T05:07:50.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.394+0000 7fb031f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c0c7920 con 0x7fb02c15b050 2023-10-06T05:07:50.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.394+0000 7fb031f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c0c7c00 con 0x7fb02c15bb30 2023-10-06T05:07:50.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.394+0000 7fb030f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb02c0cc8d0 0x7fb02c150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:50.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.394+0000 7fb030f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb02c0cc8d0 0x7fb02c150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52356/0 (socket says 172.21.15.55:52356) 2023-10-06T05:07:50.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.394+0000 7fb030f9b700 1 -- 172.21.15.55:0/1375590219 learned_addr learned my addr 172.21.15.55:0/1375590219 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:50.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.394+0000 7fb02bfff700 1 --2- 172.21.15.55:0/1375590219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb02c15b050 0x7fb02c0c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:50.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.395+0000 7fb03179c700 1 --2- 172.21.15.55:0/1375590219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb02c15bb30 0x7fb02c0c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:50.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.395+0000 7fb030f9b700 1 -- 172.21.15.55:0/1375590219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb02c15bb30 msgr2=0x7fb02c0c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:50.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.395+0000 7fb030f9b700 1 --2- 172.21.15.55:0/1375590219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb02c15bb30 0x7fb02c0c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:50.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.395+0000 7fb030f9b700 1 -- 172.21.15.55:0/1375590219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb02c15b050 msgr2=0x7fb02c0c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:50.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.395+0000 7fb030f9b700 1 --2- 172.21.15.55:0/1375590219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb02c15b050 0x7fb02c0c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:50.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.395+0000 7fb030f9b700 1 -- 172.21.15.55:0/1375590219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb024009a20 con 0x7fb02c0cc8d0 2023-10-06T05:07:50.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.395+0000 7fb030f9b700 1 --2- 172.21.15.55:0/1375590219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb02c0cc8d0 0x7fb02c150c10 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fb02000ec80 tx=0x7fb0200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:50.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.395+0000 7fb028ff9700 1 -- 172.21.15.55:0/1375590219 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02000c850 con 0x7fb02c0cc8d0 2023-10-06T05:07:50.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.395+0000 7fb028ff9700 1 -- 172.21.15.55:0/1375590219 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02000c9b0 con 0x7fb02c0cc8d0 2023-10-06T05:07:50.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.395+0000 7fb028ff9700 1 -- 172.21.15.55:0/1375590219 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02000cd50 con 0x7fb02c0cc8d0 2023-10-06T05:07:50.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.395+0000 7fb031f9d700 1 -- 172.21.15.55:0/1375590219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02c0c7ee0 con 0x7fb02c0cc8d0 2023-10-06T05:07:50.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.396+0000 7fb031f9d700 1 -- 172.21.15.55:0/1375590219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02c151150 con 0x7fb02c0cc8d0 2023-10-06T05:07:50.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.396+0000 7fb031f9d700 1 -- 172.21.15.55:0/1375590219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02c04c3c0 con 0x7fb02c0cc8d0 2023-10-06T05:07:50.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.397+0000 7fb028ff9700 1 -- 172.21.15.55:0/1375590219 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 107) v1 ==== 84578+0+0 (secure 0 0 0) 0x7fb020013680 con 0x7fb02c0cc8d0 2023-10-06T05:07:50.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.398+0000 7fb028ff9700 1 --2- 172.21.15.55:0/1375590219 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb010066d80 0x7fb010069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:50.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.398+0000 7fb028ff9700 1 -- 172.21.15.55:0/1375590219 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 7151+0+0 (secure 0 0 0) 0x7fb020084a70 con 0x7fb02c0cc8d0 2023-10-06T05:07:50.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.399+0000 7fb02bfff700 1 --2- 172.21.15.55:0/1375590219 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb010066d80 0x7fb010069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:50.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.399+0000 7fb02bfff700 1 --2- 172.21.15.55:0/1375590219 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb010066d80 0x7fb010069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb02c155c10 tx=0x7fb018009040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:50.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.402+0000 7fb028ff9700 1 -- 172.21.15.55:0/1375590219 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb020051920 con 0x7fb02c0cc8d0 2023-10-06T05:07:50.607 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:50.605+0000 7fb031f9d700 1 -- 172.21.15.55:0/1375590219 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb02c0cacc0 con 0x7fb02c0cc8d0 2023-10-06T05:07:51.337 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.335+0000 7fb028ff9700 1 -- 172.21.15.55:0/1375590219 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb020051cf0 con 0x7fb02c0cc8d0 2023-10-06T05:07:51.339 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.337+0000 7fb028ff9700 1 -- 172.21.15.55:0/1375590219 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v211) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb0200553b0 con 0x7fb02c0cc8d0 2023-10-06T05:07:51.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.340+0000 7fb031f9d700 1 -- 172.21.15.55:0/1375590219 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb010066d80 msgr2=0x7fb010069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:51.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.340+0000 7fb031f9d700 1 --2- 172.21.15.55:0/1375590219 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb010066d80 0x7fb010069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb02c155c10 tx=0x7fb018009040 comp rx=0 tx=0).stop 2023-10-06T05:07:51.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.340+0000 7fb031f9d700 1 -- 172.21.15.55:0/1375590219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb02c0cc8d0 msgr2=0x7fb02c150c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:51.343 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.340+0000 7fb031f9d700 1 --2- 172.21.15.55:0/1375590219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb02c0cc8d0 0x7fb02c150c10 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fb02000ec80 tx=0x7fb0200091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.340+0000 7fb031f9d700 1 -- 172.21.15.55:0/1375590219 shutdown_connections 2023-10-06T05:07:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.340+0000 7fb031f9d700 1 --2- 172.21.15.55:0/1375590219 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fb010066d80 0x7fb010069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.340+0000 7fb031f9d700 1 --2- 172.21.15.55:0/1375590219 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fb02c15bb30 0x7fb02c0c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.340+0000 7fb031f9d700 1 --2- 172.21.15.55:0/1375590219 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fb02c0cc8d0 0x7fb02c150c10 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.340+0000 7fb031f9d700 1 --2- 172.21.15.55:0/1375590219 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fb02c15b050 0x7fb02c0c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.340+0000 7fb031f9d700 1 -- 172.21.15.55:0/1375590219 >> 172.21.15.55:0/1375590219 conn(0x7fb02c068e20 msgr2=0x7fb02c06e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.341+0000 7fb031f9d700 1 -- 172.21.15.55:0/1375590219 shutdown_connections 2023-10-06T05:07:51.344 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.341+0000 7fb031f9d700 1 -- 172.21.15.55:0/1375590219 wait complete. 2023-10-06T05:07:51.345 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_metadata' removed 2023-10-06T05:07:51.363 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T05:07:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.556+0000 7f46f3fff700 1 Processor -- start 2023-10-06T05:07:51.558 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.556+0000 7f46f3fff700 1 -- start start 2023-10-06T05:07:51.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.557+0000 7f46f3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f40b03c0 0x7f46f40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:51.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.557+0000 7f46f3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f46f41591e0 0x7f46f4159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:51.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.557+0000 7f46f3fff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f40cfa00 con 0x7f46f40b0790 2023-10-06T05:07:51.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.557+0000 7f46f3fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f40cb580 con 0x7f46f41591e0 2023-10-06T05:07:51.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.557+0000 7f46f3fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40cf160 con 0x7f46f40b03c0 2023-10-06T05:07:51.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.558+0000 7f46f27fc700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f46f40b0790 0x7f46f40b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34214/0 (socket says 172.21.15.55:34214) 2023-10-06T05:07:51.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.558+0000 7f46f27fc700 1 -- 172.21.15.55:0/2844361177 learned_addr learned my addr 172.21.15.55:0/2844361177 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:51.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.558+0000 7f46f37fe700 1 --2- 172.21.15.55:0/2844361177 >> v2:172.21.15.55:3300/0 conn(0x7f46f41591e0 0x7f46f4159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:51.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.558+0000 7f46f2ffd700 1 --2- 172.21.15.55:0/2844361177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f40b03c0 0x7f46f40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:51.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.558+0000 7f46f1ffb700 1 -- 172.21.15.55:0/2844361177 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473713670 0 0) 0x7f46f40cfa00 con 0x7f46f40b0790 2023-10-06T05:07:51.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.558+0000 7f46f1ffb700 1 -- 172.21.15.55:0/2844361177 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46dc003570 con 0x7f46f40b0790 2023-10-06T05:07:51.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.558+0000 7f46f37fe700 1 -- 172.21.15.55:0/2844361177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f40b03c0 msgr2=0x7f46f40cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:51.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.558+0000 7f46f37fe700 1 --2- 172.21.15.55:0/2844361177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f40b03c0 0x7f46f40cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:51.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.558+0000 7f46f37fe700 1 -- 172.21.15.55:0/2844361177 >> v1:172.21.15.55:6789/0 conn(0x7f46f40b0790 legacy=0x7f46f40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:51.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.558+0000 7f46f37fe700 1 -- 172.21.15.55:0/2844361177 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f40cf440 con 0x7f46f41591e0 2023-10-06T05:07:51.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.559+0000 7f46f37fe700 1 --2- 172.21.15.55:0/2844361177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f41591e0 0x7f46f4159620 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009e60 tx=0x7f46ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c93d8c2aaaa3966 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.559+0000 7f46f1ffb700 1 -- 172.21.15.55:0/2844361177 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec00e070 con 0x7f46f41591e0 2023-10-06T05:07:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.559+0000 7f46f1ffb700 1 -- 172.21.15.55:0/2844361177 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec00bb60 con 0x7f46f41591e0 2023-10-06T05:07:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.559+0000 7f46f1ffb700 1 -- 172.21.15.55:0/2844361177 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec012430 con 0x7f46f41591e0 2023-10-06T05:07:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.559+0000 7f46f3fff700 1 -- 172.21.15.55:0/2844361177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f41591e0 msgr2=0x7f46f4159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.559+0000 7f46f3fff700 1 --2- 172.21.15.55:0/2844361177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f41591e0 0x7f46f4159620 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009e60 tx=0x7f46ec009780 comp rx=0 tx=0).stop 2023-10-06T05:07:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.559+0000 7f46f3fff700 1 -- 172.21.15.55:0/2844361177 shutdown_connections 2023-10-06T05:07:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.559+0000 7f46f3fff700 1 --2- 172.21.15.55:0/2844361177 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f41591e0 0x7f46f4159620 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.559+0000 7f46f3fff700 1 --2- 172.21.15.55:0/2844361177 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f40b03c0 0x7f46f40cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.559+0000 7f46f3fff700 1 -- 172.21.15.55:0/2844361177 >> 172.21.15.55:0/2844361177 conn(0x7f46f4063230 msgr2=0x7f46f4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:51.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.559+0000 7f46f3fff700 1 -- 172.21.15.55:0/2844361177 shutdown_connections 2023-10-06T05:07:51.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.560+0000 7f46f3fff700 1 -- 172.21.15.55:0/2844361177 wait complete. 2023-10-06T05:07:51.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.561+0000 7f46f3fff700 1 Processor -- start 2023-10-06T05:07:51.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.561+0000 7f46f3fff700 1 -- start start 2023-10-06T05:07:51.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.562+0000 7f46f3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46f40b03c0 0x7f46f40c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:51.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.562+0000 7f46f3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f40b0790 0x7f46f414ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:51.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.562+0000 7f46f3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f41591e0 0x7f46f40c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:51.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.562+0000 7f46f3fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f415ac30 con 0x7f46f41591e0 2023-10-06T05:07:51.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.562+0000 7f46f3fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f415a900 con 0x7f46f40b03c0 2023-10-06T05:07:51.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.562+0000 7f46f3fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40c8cd0 con 0x7f46f40b0790 2023-10-06T05:07:51.566 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.562+0000 7f46f37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f41591e0 0x7f46f40c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:51.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.562+0000 7f46f37fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f41591e0 0x7f46f40c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52364/0 (socket says 172.21.15.55:52364) 2023-10-06T05:07:51.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.562+0000 7f46f37fe700 1 -- 172.21.15.55:0/2177494218 learned_addr learned my addr 172.21.15.55:0/2177494218 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:51.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.562+0000 7f46f2ffd700 1 --2- 172.21.15.55:0/2177494218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46f40b03c0 0x7f46f40c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:51.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.562+0000 7f46f27fc700 1 --2- 172.21.15.55:0/2177494218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f40b0790 0x7f46f414ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:51.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.563+0000 7f46f37fe700 1 -- 172.21.15.55:0/2177494218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f40b0790 msgr2=0x7f46f414ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:51.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.563+0000 7f46f37fe700 1 --2- 172.21.15.55:0/2177494218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f40b0790 0x7f46f414ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:51.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.563+0000 7f46f37fe700 1 -- 172.21.15.55:0/2177494218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46f40b03c0 msgr2=0x7f46f40c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:51.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.563+0000 7f46f37fe700 1 --2- 172.21.15.55:0/2177494218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46f40b03c0 0x7f46f40c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:51.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.563+0000 7f46f37fe700 1 -- 172.21.15.55:0/2177494218 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec009a20 con 0x7f46f41591e0 2023-10-06T05:07:51.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.563+0000 7f46f37fe700 1 --2- 172.21.15.55:0/2177494218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f41591e0 0x7f46f40c62c0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f46f40cfe40 tx=0x7f46ec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:51.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.563+0000 7f46daffd700 1 -- 172.21.15.55:0/2177494218 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec012e80 con 0x7f46f41591e0 2023-10-06T05:07:51.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.563+0000 7f46daffd700 1 -- 172.21.15.55:0/2177494218 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec01d630 con 0x7f46f41591e0 2023-10-06T05:07:51.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.563+0000 7f46f3fff700 1 -- 172.21.15.55:0/2177494218 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f40c8fb0 con 0x7f46f41591e0 2023-10-06T05:07:51.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.563+0000 7f46daffd700 1 -- 172.21.15.55:0/2177494218 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec01d9f0 con 0x7f46f41591e0 2023-10-06T05:07:51.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.563+0000 7f46f3fff700 1 -- 172.21.15.55:0/2177494218 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f40c8970 con 0x7f46f41591e0 2023-10-06T05:07:51.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.564+0000 7f46f3fff700 1 -- 172.21.15.55:0/2177494218 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f404c3c0 con 0x7f46f41591e0 2023-10-06T05:07:51.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.565+0000 7f46daffd700 1 -- 172.21.15.55:0/2177494218 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f46ec024070 con 0x7f46f41591e0 2023-10-06T05:07:51.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.566+0000 7f46daffd700 1 --2- 172.21.15.55:0/2177494218 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f46e0066f30 0x7f46e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:51.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.566+0000 7f46daffd700 1 -- 172.21.15.55:0/2177494218 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6713+0+0 (secure 0 0 0) 0x7f46ec0871e0 con 0x7f46f41591e0 2023-10-06T05:07:51.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.567+0000 7f46f2ffd700 1 --2- 172.21.15.55:0/2177494218 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f46e0066f30 0x7f46e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:51.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.567+0000 7f46f2ffd700 1 --2- 172.21.15.55:0/2177494218 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f46e0066f30 0x7f46e00693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f46e4006fd0 tx=0x7f46e4008040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:51.572 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.570+0000 7f46daffd700 1 -- 172.21.15.55:0/2177494218 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f46ec053fc0 con 0x7f46f41591e0 2023-10-06T05:07:51.773 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:51.771+0000 7f46f3fff700 1 -- 172.21.15.55:0/2177494218 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f46f40c9450 con 0x7f46f41591e0 2023-10-06T05:07:52.347 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.345+0000 7f46daffd700 1 -- 172.21.15.55:0/2177494218 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v212) v1 ==== 169+0+0 (secure 0 0 0) 0x7f46ec0575e0 con 0x7f46f41591e0 2023-10-06T05:07:52.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.348+0000 7f46f3fff700 1 -- 172.21.15.55:0/2177494218 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f46e0066f30 msgr2=0x7f46e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:52.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.348+0000 7f46f3fff700 1 --2- 172.21.15.55:0/2177494218 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f46e0066f30 0x7f46e00693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f46e4006fd0 tx=0x7f46e4008040 comp rx=0 tx=0).stop 2023-10-06T05:07:52.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.348+0000 7f46f3fff700 1 -- 172.21.15.55:0/2177494218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f41591e0 msgr2=0x7f46f40c62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:52.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.348+0000 7f46f3fff700 1 --2- 172.21.15.55:0/2177494218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f41591e0 0x7f46f40c62c0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f46f40cfe40 tx=0x7f46ec019000 comp rx=0 tx=0).stop 2023-10-06T05:07:52.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.348+0000 7f46f3fff700 1 -- 172.21.15.55:0/2177494218 shutdown_connections 2023-10-06T05:07:52.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.348+0000 7f46f3fff700 1 --2- 172.21.15.55:0/2177494218 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f46e0066f30 0x7f46e00693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:52.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.348+0000 7f46f3fff700 1 --2- 172.21.15.55:0/2177494218 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f46f41591e0 0x7f46f40c62c0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:52.351 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.348+0000 7f46f3fff700 1 --2- 172.21.15.55:0/2177494218 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f46f40b03c0 0x7f46f40c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:52.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.348+0000 7f46f3fff700 1 --2- 172.21.15.55:0/2177494218 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f46f40b0790 0x7f46f414ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:52.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.348+0000 7f46f3fff700 1 -- 172.21.15.55:0/2177494218 >> 172.21.15.55:0/2177494218 conn(0x7f46f4063230 msgr2=0x7f46f40cba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:52.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.348+0000 7f46f3fff700 1 -- 172.21.15.55:0/2177494218 shutdown_connections 2023-10-06T05:07:52.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.349+0000 7f46f3fff700 1 -- 172.21.15.55:0/2177494218 wait complete. 2023-10-06T05:07:52.352 INFO:teuthology.orchestra.run.smithi055.stderr:pool 'cephfs_data' removed 2023-10-06T05:07:52.369 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T05:07:52.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.557+0000 7f1c0ffff700 1 Processor -- start 2023-10-06T05:07:52.559 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.557+0000 7f1c0ffff700 1 -- start start 2023-10-06T05:07:52.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.558+0000 7f1c0ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c10159a70 0x7f1c100c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:52.560 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.558+0000 7f1c0ffff700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c100ca920 con 0x7f1c10068aa0 2023-10-06T05:07:52.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.559+0000 7f1c0ffff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c1006af00 con 0x7f1c1006bfa0 2023-10-06T05:07:52.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.559+0000 7f1c0ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1006b1e0 con 0x7f1c10159a70 2023-10-06T05:07:52.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.559+0000 7f1c0f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1c10068aa0 0x7f1c100caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:34222/0 (socket says 172.21.15.55:34222) 2023-10-06T05:07:52.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.559+0000 7f1c0f7fe700 1 -- 172.21.15.55:0/2377725694 learned_addr learned my addr 172.21.15.55:0/2377725694 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:52.561 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.559+0000 7f1c0effd700 1 --2- 172.21.15.55:0/2377725694 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c10159a70 0x7f1c100c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:52.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.559+0000 7f1c0dffb700 1 -- 172.21.15.55:0/2377725694 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556508999 0 0) 0x7f1c100ca920 con 0x7f1c10068aa0 2023-10-06T05:07:52.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.559+0000 7f1c0dffb700 1 -- 172.21.15.55:0/2377725694 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bf8003570 con 0x7f1c10068aa0 2023-10-06T05:07:52.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.559+0000 7f1c0dffb700 1 -- 172.21.15.55:0/2377725694 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836698040 0 0) 0x7f1c1006af00 con 0x7f1c1006bfa0 2023-10-06T05:07:52.563 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.559+0000 7f1c0dffb700 1 -- 172.21.15.55:0/2377725694 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c100ca920 con 0x7f1c1006bfa0 2023-10-06T05:07:52.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.559+0000 7f1c0effd700 1 -- 172.21.15.55:0/2377725694 >> v1:172.21.15.163:6789/0 conn(0x7f1c1006bfa0 legacy=0x7f1c10068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:52.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.559+0000 7f1c0effd700 1 -- 172.21.15.55:0/2377725694 >> v1:172.21.15.55:6789/0 conn(0x7f1c10068aa0 legacy=0x7f1c100caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:52.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.559+0000 7f1c0effd700 1 -- 172.21.15.55:0/2377725694 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1006b4c0 con 0x7f1c10159a70 2023-10-06T05:07:52.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.560+0000 7f1c0effd700 1 --2- 172.21.15.55:0/2377725694 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c10159a70 0x7f1c100c82f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009910 tx=0x7f1c000091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7703a4abc453600b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:52.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.560+0000 7f1c0dffb700 1 -- 172.21.15.55:0/2377725694 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00011630 con 0x7f1c10159a70 2023-10-06T05:07:52.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.560+0000 7f1c0dffb700 1 -- 172.21.15.55:0/2377725694 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c00011790 con 0x7f1c10159a70 2023-10-06T05:07:52.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.560+0000 7f1c0dffb700 1 -- 172.21.15.55:0/2377725694 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00011b10 con 0x7f1c10159a70 2023-10-06T05:07:52.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.561+0000 7f1c0ffff700 1 -- 172.21.15.55:0/2377725694 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c10159a70 msgr2=0x7f1c100c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:52.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.561+0000 7f1c0ffff700 1 --2- 172.21.15.55:0/2377725694 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c10159a70 0x7f1c100c82f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009910 tx=0x7f1c000091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:52.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.561+0000 7f1c0ffff700 1 -- 172.21.15.55:0/2377725694 shutdown_connections 2023-10-06T05:07:52.564 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.561+0000 7f1c0ffff700 1 --2- 172.21.15.55:0/2377725694 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c10159a70 0x7f1c100c82f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:52.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.561+0000 7f1c0ffff700 1 -- 172.21.15.55:0/2377725694 >> 172.21.15.55:0/2377725694 conn(0x7f1c100cddc0 msgr2=0x7f1c100c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:52.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.561+0000 7f1c0ffff700 1 -- 172.21.15.55:0/2377725694 shutdown_connections 2023-10-06T05:07:52.565 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.561+0000 7f1c0ffff700 1 -- 172.21.15.55:0/2377725694 wait complete. 2023-10-06T05:07:52.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.562+0000 7f1c0ffff700 1 Processor -- start 2023-10-06T05:07:52.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.562+0000 7f1c0ffff700 1 -- start start 2023-10-06T05:07:52.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.563+0000 7f1c0ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c10068aa0 0x7f1c1015a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:52.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.563+0000 7f1c0ffff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1006bfa0 0x7f1c10154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:52.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.563+0000 7f1c0ffff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c10159a70 0x7f1c100c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:52.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.563+0000 7f1c0ffff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1015ad40 con 0x7f1c10068aa0 2023-10-06T05:07:52.567 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.563+0000 7f1c0ffff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1015a1a0 con 0x7f1c10159a70 2023-10-06T05:07:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.563+0000 7f1c0ffff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c10154890 con 0x7f1c1006bfa0 2023-10-06T05:07:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.563+0000 7f1c0effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c10068aa0 0x7f1c1015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.564+0000 7f1c0effd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c10068aa0 0x7f1c1015a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52368/0 (socket says 172.21.15.55:52368) 2023-10-06T05:07:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.564+0000 7f1c0effd700 1 -- 172.21.15.55:0/1260448697 learned_addr learned my addr 172.21.15.55:0/1260448697 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.564+0000 7f1c0f7fe700 1 --2- 172.21.15.55:0/1260448697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c10159a70 0x7f1c100c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.564+0000 7f1c0e7fc700 1 --2- 172.21.15.55:0/1260448697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1006bfa0 0x7f1c10154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.564+0000 7f1c0effd700 1 -- 172.21.15.55:0/1260448697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1006bfa0 msgr2=0x7f1c10154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.564+0000 7f1c0effd700 1 --2- 172.21.15.55:0/1260448697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1006bfa0 0x7f1c10154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.564+0000 7f1c0effd700 1 -- 172.21.15.55:0/1260448697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c10159a70 msgr2=0x7f1c100c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.564+0000 7f1c0effd700 1 --2- 172.21.15.55:0/1260448697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c10159a70 0x7f1c100c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:52.568 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.564+0000 7f1c0effd700 1 -- 172.21.15.55:0/1260448697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c00009420 con 0x7f1c10068aa0 2023-10-06T05:07:52.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.564+0000 7f1c0effd700 1 --2- 172.21.15.55:0/1260448697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c10068aa0 0x7f1c1015a800 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f1c1006a6a0 tx=0x7f1c00017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:52.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.564+0000 7f1bf6ffd700 1 -- 172.21.15.55:0/1260448697 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00020070 con 0x7f1c10068aa0 2023-10-06T05:07:52.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.565+0000 7f1bf6ffd700 1 -- 172.21.15.55:0/1260448697 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c00011900 con 0x7f1c10068aa0 2023-10-06T05:07:52.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.565+0000 7f1c0ffff700 1 -- 172.21.15.55:0/1260448697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1015e3d0 con 0x7f1c10068aa0 2023-10-06T05:07:52.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.565+0000 7f1bf6ffd700 1 -- 172.21.15.55:0/1260448697 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c00024660 con 0x7f1c10068aa0 2023-10-06T05:07:52.569 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.565+0000 7f1c0ffff700 1 -- 172.21.15.55:0/1260448697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c10156d80 con 0x7f1c10068aa0 2023-10-06T05:07:52.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.566+0000 7f1c0ffff700 1 -- 172.21.15.55:0/1260448697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c10063d10 con 0x7f1c10068aa0 2023-10-06T05:07:52.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.566+0000 7f1bf6ffd700 1 -- 172.21.15.55:0/1260448697 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c000193b0 con 0x7f1c10068aa0 2023-10-06T05:07:52.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.567+0000 7f1bf6ffd700 1 --2- 172.21.15.55:0/1260448697 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1bfc066f30 0x7f1bfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:52.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.567+0000 7f1bf6ffd700 1 -- 172.21.15.55:0/1260448697 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f1c00085c20 con 0x7f1c10068aa0 2023-10-06T05:07:52.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.568+0000 7f1c0e7fc700 1 --2- 172.21.15.55:0/1260448697 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1bfc066f30 0x7f1bfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:52.570 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.568+0000 7f1c0e7fc700 1 --2- 172.21.15.55:0/1260448697 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1bfc066f30 0x7f1bfc0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1c04007fd0 tx=0x7f1c04009040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:52.573 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.572+0000 7f1bf6ffd700 1 -- 172.21.15.55:0/1260448697 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c00052ba0 con 0x7f1c10068aa0 2023-10-06T05:07:52.788 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.786+0000 7f1c0ffff700 1 -- 172.21.15.55:0/1260448697 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1c1004bb80 con 0x7f1c10068aa0 2023-10-06T05:07:52.792 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.790+0000 7f1bf6ffd700 1 -- 172.21.15.55:0/1260448697 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1c000290a0 con 0x7f1c10068aa0 2023-10-06T05:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.793+0000 7f1c0ffff700 1 -- 172.21.15.55:0/1260448697 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1bfc066f30 msgr2=0x7f1bfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.793+0000 7f1c0ffff700 1 --2- 172.21.15.55:0/1260448697 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1bfc066f30 0x7f1bfc0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1c04007fd0 tx=0x7f1c04009040 comp rx=0 tx=0).stop 2023-10-06T05:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.793+0000 7f1c0ffff700 1 -- 172.21.15.55:0/1260448697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c10068aa0 msgr2=0x7f1c1015a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.793+0000 7f1c0ffff700 1 --2- 172.21.15.55:0/1260448697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c10068aa0 0x7f1c1015a800 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f1c1006a6a0 tx=0x7f1c00017000 comp rx=0 tx=0).stop 2023-10-06T05:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.793+0000 7f1c0ffff700 1 -- 172.21.15.55:0/1260448697 shutdown_connections 2023-10-06T05:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.793+0000 7f1c0ffff700 1 --2- 172.21.15.55:0/1260448697 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1bfc066f30 0x7f1bfc0693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:52.795 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.793+0000 7f1c0ffff700 1 --2- 172.21.15.55:0/1260448697 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1c10159a70 0x7f1c100c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:52.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.793+0000 7f1c0ffff700 1 --2- 172.21.15.55:0/1260448697 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1c10068aa0 0x7f1c1015a800 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:52.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.793+0000 7f1c0ffff700 1 --2- 172.21.15.55:0/1260448697 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1c1006bfa0 0x7f1c10154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:52.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.793+0000 7f1c0ffff700 1 -- 172.21.15.55:0/1260448697 >> 172.21.15.55:0/1260448697 conn(0x7f1c100cddc0 msgr2=0x7f1c100c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:52.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.793+0000 7f1c0ffff700 1 -- 172.21.15.55:0/1260448697 shutdown_connections 2023-10-06T05:07:52.796 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.793+0000 7f1c0ffff700 1 -- 172.21.15.55:0/1260448697 wait complete. 2023-10-06T05:07:52.812 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-06T05:07:53.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.999+0000 7f68b0f9d700 1 Processor -- start 2023-10-06T05:07:53.001 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:52.999+0000 7f68b0f9d700 1 -- start start 2023-10-06T05:07:53.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.000+0000 7f68b0f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f68ac0c9db0 0x7f68ac06a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.000+0000 7f68b0f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f68ac157130 0x7f68ac06a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.003 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.000+0000 7f68b0f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ac06e150 con 0x7f68ac153900 2023-10-06T05:07:53.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.000+0000 7f68b0f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ac0c7610 con 0x7f68ac157130 2023-10-06T05:07:53.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.000+0000 7f68b0f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ac0c7800 con 0x7f68ac0c9db0 2023-10-06T05:07:53.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.001+0000 7f68abfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f68ac157130 0x7f68ac06a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.001+0000 7f68abfff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f68ac157130 0x7f68ac06a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52380/0 (socket says 172.21.15.55:52380) 2023-10-06T05:07:53.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.001+0000 7f68abfff700 1 -- 172.21.15.55:0/701638434 learned_addr learned my addr 172.21.15.55:0/701638434 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:53.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.001+0000 7f68aaffd700 1 --2- 172.21.15.55:0/701638434 >> v2:172.21.15.163:3300/0 conn(0x7f68ac0c9db0 0x7f68ac06a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.001+0000 7f68abfff700 1 -- 172.21.15.55:0/701638434 >> v2:172.21.15.163:3300/0 conn(0x7f68ac0c9db0 msgr2=0x7f68ac06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:53.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.001+0000 7f68abfff700 1 --2- 172.21.15.55:0/701638434 >> v2:172.21.15.163:3300/0 conn(0x7f68ac0c9db0 0x7f68ac06a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.004 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.001+0000 7f68abfff700 1 -- 172.21.15.55:0/701638434 >> v1:172.21.15.163:6789/0 conn(0x7f68ac153900 legacy=0x7f68ac0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:53.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.001+0000 7f68abfff700 1 -- 172.21.15.55:0/701638434 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac0c7ae0 con 0x7f68ac157130 2023-10-06T05:07:53.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.001+0000 7f68aaffd700 1 --2- 172.21.15.55:0/701638434 >> v2:172.21.15.163:3300/0 conn(0x7f68ac0c9db0 0x7f68ac06a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:07:53.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.001+0000 7f68abfff700 1 --2- 172.21.15.55:0/701638434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68ac157130 0x7f68ac06a540 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0aed40 tx=0x7f68a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11bb6e46899c47b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:53.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.002+0000 7f68aa7fc700 1 -- 172.21.15.55:0/701638434 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a400e070 con 0x7f68ac157130 2023-10-06T05:07:53.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.002+0000 7f68aa7fc700 1 -- 172.21.15.55:0/701638434 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a400ba10 con 0x7f68ac157130 2023-10-06T05:07:53.005 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.002+0000 7f68aa7fc700 1 -- 172.21.15.55:0/701638434 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a40092e0 con 0x7f68ac157130 2023-10-06T05:07:53.007 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.002+0000 7f68b0f9d700 1 -- 172.21.15.55:0/701638434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68ac157130 msgr2=0x7f68ac06a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.002+0000 7f68b0f9d700 1 --2- 172.21.15.55:0/701638434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68ac157130 0x7f68ac06a540 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0aed40 tx=0x7f68a4009840 comp rx=0 tx=0).stop 2023-10-06T05:07:53.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.002+0000 7f68b0f9d700 1 -- 172.21.15.55:0/701638434 shutdown_connections 2023-10-06T05:07:53.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.002+0000 7f68b0f9d700 1 --2- 172.21.15.55:0/701638434 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68ac157130 0x7f68ac06a540 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.002+0000 7f68b0f9d700 1 --2- 172.21.15.55:0/701638434 >> v2:172.21.15.163:3300/0 conn(0x7f68ac0c9db0 0x7f68ac06a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.002+0000 7f68b0f9d700 1 -- 172.21.15.55:0/701638434 >> 172.21.15.55:0/701638434 conn(0x7f68ac0cc1e0 msgr2=0x7f68ac0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:53.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.002+0000 7f68b0f9d700 1 -- 172.21.15.55:0/701638434 shutdown_connections 2023-10-06T05:07:53.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.002+0000 7f68b0f9d700 1 -- 172.21.15.55:0/701638434 wait complete. 2023-10-06T05:07:53.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.004+0000 7f68b0f9d700 1 Processor -- start 2023-10-06T05:07:53.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.004+0000 7f68b0f9d700 1 -- start start 2023-10-06T05:07:53.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.005+0000 7f68b0f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68ac0c9db0 0x7f68ac16df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.005+0000 7f68b0f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68ac153900 0x7f68ac142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.005+0000 7f68b0f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac157130 0x7f68ac155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.005+0000 7f68b0f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac1561b0 con 0x7f68ac0c9db0 2023-10-06T05:07:53.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.005+0000 7f68b0f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac152cc0 con 0x7f68ac157130 2023-10-06T05:07:53.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.005+0000 7f68b0f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac152fa0 con 0x7f68ac153900 2023-10-06T05:07:53.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.005+0000 7f68ab7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68ac0c9db0 0x7f68ac16df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.005+0000 7f68ab7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68ac0c9db0 0x7f68ac16df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:52384/0 (socket says 172.21.15.55:52384) 2023-10-06T05:07:53.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.005+0000 7f68ab7fe700 1 -- 172.21.15.55:0/3169208935 learned_addr learned my addr 172.21.15.55:0/3169208935 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:53.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.005+0000 7f68abfff700 1 --2- 172.21.15.55:0/3169208935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac157130 0x7f68ac155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.009 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.005+0000 7f68aaffd700 1 --2- 172.21.15.55:0/3169208935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68ac153900 0x7f68ac142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.006+0000 7f68ab7fe700 1 -- 172.21.15.55:0/3169208935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68ac153900 msgr2=0x7f68ac142f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.006+0000 7f68ab7fe700 1 --2- 172.21.15.55:0/3169208935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68ac153900 0x7f68ac142f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.006+0000 7f68ab7fe700 1 -- 172.21.15.55:0/3169208935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac157130 msgr2=0x7f68ac155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.006+0000 7f68ab7fe700 1 --2- 172.21.15.55:0/3169208935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac157130 0x7f68ac155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.006+0000 7f68ab7fe700 1 -- 172.21.15.55:0/3169208935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a4009ae0 con 0x7f68ac0c9db0 2023-10-06T05:07:53.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.006+0000 7f68ab7fe700 1 --2- 172.21.15.55:0/3169208935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68ac0c9db0 0x7f68ac16df10 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f689c00c5d0 tx=0x7f689c007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:53.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.006+0000 7f68937fe700 1 -- 172.21.15.55:0/3169208935 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c011590 con 0x7f68ac0c9db0 2023-10-06T05:07:53.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.006+0000 7f68937fe700 1 -- 172.21.15.55:0/3169208935 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689c011710 con 0x7f68ac0c9db0 2023-10-06T05:07:53.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.006+0000 7f68937fe700 1 -- 172.21.15.55:0/3169208935 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689c011ad0 con 0x7f68ac0c9db0 2023-10-06T05:07:53.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.006+0000 7f68b0f9d700 1 -- 172.21.15.55:0/3169208935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac153280 con 0x7f68ac0c9db0 2023-10-06T05:07:53.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.006+0000 7f68b0f9d700 1 -- 172.21.15.55:0/3169208935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac0ad4f0 con 0x7f68ac0c9db0 2023-10-06T05:07:53.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.007+0000 7f68b0f9d700 1 -- 172.21.15.55:0/3169208935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac06c450 con 0x7f68ac0c9db0 2023-10-06T05:07:53.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.008+0000 7f68937fe700 1 -- 172.21.15.55:0/3169208935 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f689c01c460 con 0x7f68ac0c9db0 2023-10-06T05:07:53.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.010+0000 7f68937fe700 1 --2- 172.21.15.55:0/3169208935 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6898066f30 0x7f68980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.010+0000 7f68937fe700 1 -- 172.21.15.55:0/3169208935 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f689c0845f0 con 0x7f68ac0c9db0 2023-10-06T05:07:53.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.010+0000 7f68aaffd700 1 --2- 172.21.15.55:0/3169208935 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6898066f30 0x7f68980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.011+0000 7f68aaffd700 1 --2- 172.21.15.55:0/3169208935 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6898066f30 0x7f68980693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f68ac149260 tx=0x7f68a0006f90 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:53.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.014+0000 7f68937fe700 1 -- 172.21.15.55:0/3169208935 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f689c051570 con 0x7f68ac0c9db0 2023-10-06T05:07:53.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.219+0000 7f68b0f9d700 1 -- 172.21.15.55:0/3169208935 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f68ac153cd0 con 0x7f68ac0c9db0 2023-10-06T05:07:53.228 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.226+0000 7f68937fe700 1 -- 172.21.15.55:0/3169208935 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v74) v1 ==== 115+0+0 (secure 0 0 0) 0x7f689c054b90 con 0x7f68ac0c9db0 2023-10-06T05:07:53.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.229+0000 7f68b0f9d700 1 -- 172.21.15.55:0/3169208935 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6898066f30 msgr2=0x7f68980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.229+0000 7f68b0f9d700 1 --2- 172.21.15.55:0/3169208935 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6898066f30 0x7f68980693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f68ac149260 tx=0x7f68a0006f90 comp rx=0 tx=0).stop 2023-10-06T05:07:53.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.229+0000 7f68b0f9d700 1 -- 172.21.15.55:0/3169208935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68ac0c9db0 msgr2=0x7f68ac16df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.229+0000 7f68b0f9d700 1 --2- 172.21.15.55:0/3169208935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68ac0c9db0 0x7f68ac16df10 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f689c00c5d0 tx=0x7f689c007f60 comp rx=0 tx=0).stop 2023-10-06T05:07:53.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.229+0000 7f68b0f9d700 1 -- 172.21.15.55:0/3169208935 shutdown_connections 2023-10-06T05:07:53.231 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.229+0000 7f68b0f9d700 1 --2- 172.21.15.55:0/3169208935 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6898066f30 0x7f68980693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.229+0000 7f68b0f9d700 1 --2- 172.21.15.55:0/3169208935 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f68ac157130 0x7f68ac155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.229+0000 7f68b0f9d700 1 --2- 172.21.15.55:0/3169208935 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f68ac0c9db0 0x7f68ac16df10 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.229+0000 7f68b0f9d700 1 --2- 172.21.15.55:0/3169208935 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f68ac153900 0x7f68ac142f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.229+0000 7f68b0f9d700 1 -- 172.21.15.55:0/3169208935 >> 172.21.15.55:0/3169208935 conn(0x7f68ac0cc1e0 msgr2=0x7f68ac14d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:53.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.229+0000 7f68b0f9d700 1 -- 172.21.15.55:0/3169208935 shutdown_connections 2023-10-06T05:07:53.232 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.229+0000 7f68b0f9d700 1 -- 172.21.15.55:0/3169208935 wait complete. 2023-10-06T05:07:53.248 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-06T05:07:53.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.435+0000 7f6f1f59d700 1 Processor -- start 2023-10-06T05:07:53.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.435+0000 7f6f1f59d700 1 -- start start 2023-10-06T05:07:53.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.436+0000 7f6f1f59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6f180b4e70 0x7f6f180ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.436+0000 7f6f1f59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6f180b1b40 0x7f6f1815d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.436+0000 7f6f1f59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f180cfb00 con 0x7f6f180b16d0 2023-10-06T05:07:53.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.436+0000 7f6f1f59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f180b5c40 con 0x7f6f180b4e70 2023-10-06T05:07:53.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.436+0000 7f6f1f59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f180b5f20 con 0x7f6f180b1b40 2023-10-06T05:07:53.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.437+0000 7f6f1e59b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6f180b4e70 0x7f6f180ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.437+0000 7f6f1e59b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f6f180b4e70 0x7f6f180ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54524/0 (socket says 172.21.15.55:54524) 2023-10-06T05:07:53.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.437+0000 7f6f1e59b700 1 -- 172.21.15.55:0/3417479453 learned_addr learned my addr 172.21.15.55:0/3417479453 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:53.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.437+0000 7f6f1ed9c700 1 --2- 172.21.15.55:0/3417479453 >> v2:172.21.15.163:3300/0 conn(0x7f6f180b1b40 0x7f6f1815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.437+0000 7f6f1e59b700 1 -- 172.21.15.55:0/3417479453 >> v2:172.21.15.163:3300/0 conn(0x7f6f180b1b40 msgr2=0x7f6f1815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:53.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.437+0000 7f6f1e59b700 1 --2- 172.21.15.55:0/3417479453 >> v2:172.21.15.163:3300/0 conn(0x7f6f180b1b40 0x7f6f1815d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.437+0000 7f6f1e59b700 1 -- 172.21.15.55:0/3417479453 >> v1:172.21.15.163:6789/0 conn(0x7f6f180b16d0 legacy=0x7f6f1815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.437+0000 7f6f1e59b700 1 -- 172.21.15.55:0/3417479453 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f180b6200 con 0x7f6f180b4e70 2023-10-06T05:07:53.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.437+0000 7f6f1ed9c700 1 --2- 172.21.15.55:0/3417479453 >> v2:172.21.15.163:3300/0 conn(0x7f6f180b1b40 0x7f6f1815d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:53.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.437+0000 7f6f1e59b700 1 --2- 172.21.15.55:0/3417479453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f180b4e70 0x7f6f180ccff0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f6f18071380 tx=0x7f6f14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e2d826aa2ed5b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:53.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.438+0000 7f6f1d599700 1 -- 172.21.15.55:0/3417479453 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1400b860 con 0x7f6f180b4e70 2023-10-06T05:07:53.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.438+0000 7f6f1d599700 1 -- 172.21.15.55:0/3417479453 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1400b9c0 con 0x7f6f180b4e70 2023-10-06T05:07:53.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.438+0000 7f6f1d599700 1 -- 172.21.15.55:0/3417479453 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1400bd60 con 0x7f6f180b4e70 2023-10-06T05:07:53.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.438+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3417479453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f180b4e70 msgr2=0x7f6f180ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.438+0000 7f6f1f59d700 1 --2- 172.21.15.55:0/3417479453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f180b4e70 0x7f6f180ccff0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f6f18071380 tx=0x7f6f14009250 comp rx=0 tx=0).stop 2023-10-06T05:07:53.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.438+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3417479453 shutdown_connections 2023-10-06T05:07:53.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.438+0000 7f6f1f59d700 1 --2- 172.21.15.55:0/3417479453 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f180b4e70 0x7f6f180ccff0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.438+0000 7f6f1f59d700 1 --2- 172.21.15.55:0/3417479453 >> v2:172.21.15.163:3300/0 conn(0x7f6f180b1b40 0x7f6f1815d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.438+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3417479453 >> 172.21.15.55:0/3417479453 conn(0x7f6f180d10c0 msgr2=0x7f6f180affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:53.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.438+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3417479453 shutdown_connections 2023-10-06T05:07:53.442 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.439+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3417479453 wait complete. 2023-10-06T05:07:53.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.440+0000 7f6f1f59d700 1 Processor -- start 2023-10-06T05:07:53.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.440+0000 7f6f1f59d700 1 -- start start 2023-10-06T05:07:53.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.441+0000 7f6f1f59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f180b16d0 0x7f6f180a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.441+0000 7f6f1f59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f180b1b40 0x7f6f180ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.441+0000 7f6f1f59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f180b4e70 0x7f6f1816a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.441+0000 7f6f1f59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f180ca7c0 con 0x7f6f180b16d0 2023-10-06T05:07:53.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.441+0000 7f6f1f59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f180cbce0 con 0x7f6f180b4e70 2023-10-06T05:07:53.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.441+0000 7f6f1f59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f180cc170 con 0x7f6f180b1b40 2023-10-06T05:07:53.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.441+0000 7f6f1e59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f180b16d0 0x7f6f180a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.441+0000 7f6f1e59b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f180b16d0 0x7f6f180a78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54532/0 (socket says 172.21.15.55:54532) 2023-10-06T05:07:53.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.441+0000 7f6f1e59b700 1 -- 172.21.15.55:0/3485189213 learned_addr learned my addr 172.21.15.55:0/3485189213 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:53.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f1ed9c700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f180b4e70 0x7f6f1816a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f1e59b700 1 -- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f180b1b40 msgr2=0x7f6f180ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f1dd9a700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f180b1b40 0x7f6f180ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f1e59b700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f180b1b40 0x7f6f180ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f1e59b700 1 -- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f180b4e70 msgr2=0x7f6f1816a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f1e59b700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f180b4e70 0x7f6f1816a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f1e59b700 1 -- 172.21.15.55:0/3485189213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f140094f0 con 0x7f6f180b16d0 2023-10-06T05:07:53.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f1ed9c700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f180b4e70 0x7f6f1816a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:07:53.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f1dd9a700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f180b1b40 0x7f6f180ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:07:53.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f1e59b700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f180b16d0 0x7f6f180a78e0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009290 tx=0x7f6f14017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:53.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f0e7fc700 1 -- 172.21.15.55:0/3485189213 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14011e60 con 0x7f6f180b16d0 2023-10-06T05:07:53.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f0e7fc700 1 -- 172.21.15.55:0/3485189213 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1401a660 con 0x7f6f180b16d0 2023-10-06T05:07:53.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3485189213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f180a7f70 con 0x7f6f180b16d0 2023-10-06T05:07:53.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f0e7fc700 1 -- 172.21.15.55:0/3485189213 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1401aa20 con 0x7f6f180b16d0 2023-10-06T05:07:53.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.442+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3485189213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f180cb970 con 0x7f6f180b16d0 2023-10-06T05:07:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.443+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3485189213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f180cbfc0 con 0x7f6f180b16d0 2023-10-06T05:07:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.444+0000 7f6f0e7fc700 1 -- 172.21.15.55:0/3485189213 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f1401b430 con 0x7f6f180b16d0 2023-10-06T05:07:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.445+0000 7f6f0e7fc700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6efc066f30 0x7f6efc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.445+0000 7f6f0e7fc700 1 -- 172.21.15.55:0/3485189213 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f6f14085f60 con 0x7f6f180b16d0 2023-10-06T05:07:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.446+0000 7f6f1dd9a700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6efc066f30 0x7f6efc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.448 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.447+0000 7f6f1dd9a700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6efc066f30 0x7f6efc0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6f180c90f0 tx=0x7f6f08009040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:53.452 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.450+0000 7f6f0e7fc700 1 -- 172.21.15.55:0/3485189213 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f14052ee0 con 0x7f6f180b16d0 2023-10-06T05:07:53.652 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.651+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3485189213 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f6f180ce1e0 con 0x7f6f180b16d0 2023-10-06T05:07:53.656 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.655+0000 7f6f0e7fc700 1 -- 172.21.15.55:0/3485189213 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v75) v1 ==== 120+0+0 (secure 0 0 0) 0x7f6f14056500 con 0x7f6f180b16d0 2023-10-06T05:07:53.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.657+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6efc066f30 msgr2=0x7f6efc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.657+0000 7f6f1f59d700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6efc066f30 0x7f6efc0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6f180c90f0 tx=0x7f6f08009040 comp rx=0 tx=0).stop 2023-10-06T05:07:53.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.657+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3485189213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f180b16d0 msgr2=0x7f6f180a78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.657+0000 7f6f1f59d700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f180b16d0 0x7f6f180a78e0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009290 tx=0x7f6f14017000 comp rx=0 tx=0).stop 2023-10-06T05:07:53.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.657+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3485189213 shutdown_connections 2023-10-06T05:07:53.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.657+0000 7f6f1f59d700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6efc066f30 0x7f6efc0693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.657+0000 7f6f1f59d700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6f180b4e70 0x7f6f1816a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.660 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.657+0000 7f6f1f59d700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6f180b16d0 0x7f6f180a78e0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.657+0000 7f6f1f59d700 1 --2- 172.21.15.55:0/3485189213 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6f180b1b40 0x7f6f180ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.657+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3485189213 >> 172.21.15.55:0/3485189213 conn(0x7f6f180d10c0 msgr2=0x7f6f180cd090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:53.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.658+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3485189213 shutdown_connections 2023-10-06T05:07:53.661 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.658+0000 7f6f1f59d700 1 -- 172.21.15.55:0/3485189213 wait complete. 2023-10-06T05:07:53.677 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-06T05:07:53.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.862+0000 7feefd39d700 1 Processor -- start 2023-10-06T05:07:53.864 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.862+0000 7feefd39d700 1 -- start start 2023-10-06T05:07:53.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.863+0000 7feefd39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feef8157270 0x7feef8068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.863+0000 7feefd39d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7feef80688d0 0x7feef80b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.863+0000 7feefd39d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef8068d40 con 0x7feef80b0a00 2023-10-06T05:07:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.863+0000 7feefd39d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef815e310 con 0x7feef80688d0 2023-10-06T05:07:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.863+0000 7feefd39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef815a980 con 0x7feef8157270 2023-10-06T05:07:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.864+0000 7feefcb9c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7feef80b0a00 0x7feef806a7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:40132/0 (socket says 172.21.15.55:40132) 2023-10-06T05:07:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.864+0000 7feefcb9c700 1 -- 172.21.15.55:0/3451250155 learned_addr learned my addr 172.21.15.55:0/3451250155 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.864+0000 7feef7fff700 1 --2- 172.21.15.55:0/3451250155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feef8157270 0x7feef8068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.864+0000 7feef77fe700 1 --2- 172.21.15.55:0/3451250155 >> v2:172.21.15.163:3300/0 conn(0x7feef80688d0 0x7feef80b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.864+0000 7feef6ffd700 1 -- 172.21.15.55:0/3451250155 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940259286 0 0) 0x7feef8068d40 con 0x7feef80b0a00 2023-10-06T05:07:53.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.864+0000 7feef6ffd700 1 -- 172.21.15.55:0/3451250155 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feee0003570 con 0x7feef80b0a00 2023-10-06T05:07:53.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.865+0000 7feef77fe700 1 -- 172.21.15.55:0/3451250155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feef8157270 msgr2=0x7feef8068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:53.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.865+0000 7feef77fe700 1 --2- 172.21.15.55:0/3451250155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feef8157270 0x7feef8068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.865+0000 7feef6ffd700 1 -- 172.21.15.55:0/3451250155 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 127534590 0 0) 0x7feee0003570 con 0x7feef80b0a00 2023-10-06T05:07:53.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.865+0000 7feef77fe700 1 -- 172.21.15.55:0/3451250155 >> v1:172.21.15.55:6789/0 conn(0x7feef80b0a00 legacy=0x7feef806a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.868 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.865+0000 7feef77fe700 1 -- 172.21.15.55:0/3451250155 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef815a050 con 0x7feef80688d0 2023-10-06T05:07:53.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.865+0000 7feef77fe700 1 --2- 172.21.15.55:0/3451250155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feef80688d0 0x7feef80b04c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7feeec0098e0 tx=0x7feeec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6e96cf6123ece0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:53.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.865+0000 7feef6ffd700 1 -- 172.21.15.55:0/3451250155 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef00017f0 con 0x7feef80688d0 2023-10-06T05:07:53.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.865+0000 7feef6ffd700 1 -- 172.21.15.55:0/3451250155 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec011420 con 0x7feef80688d0 2023-10-06T05:07:53.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.865+0000 7feef6ffd700 1 -- 172.21.15.55:0/3451250155 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0117c0 con 0x7feef80688d0 2023-10-06T05:07:53.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.866+0000 7feefd39d700 1 -- 172.21.15.55:0/3451250155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feef80688d0 msgr2=0x7feef80b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.866+0000 7feefd39d700 1 --2- 172.21.15.55:0/3451250155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feef80688d0 0x7feef80b04c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7feeec0098e0 tx=0x7feeec0091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.866+0000 7feefd39d700 1 -- 172.21.15.55:0/3451250155 shutdown_connections 2023-10-06T05:07:53.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.866+0000 7feefd39d700 1 --2- 172.21.15.55:0/3451250155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feef8157270 0x7feef8068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.866+0000 7feefd39d700 1 --2- 172.21.15.55:0/3451250155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feef80688d0 0x7feef80b04c0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.866+0000 7feefd39d700 1 -- 172.21.15.55:0/3451250155 >> 172.21.15.55:0/3451250155 conn(0x7feef80cd310 msgr2=0x7feef80cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:53.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.866+0000 7feefd39d700 1 -- 172.21.15.55:0/3451250155 shutdown_connections 2023-10-06T05:07:53.869 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.866+0000 7feefd39d700 1 -- 172.21.15.55:0/3451250155 wait complete. 2023-10-06T05:07:53.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.867+0000 7feefd39d700 1 Processor -- start 2023-10-06T05:07:53.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.868+0000 7feefd39d700 1 -- start start 2023-10-06T05:07:53.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.868+0000 7feefd39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feef80688d0 0x7feef80aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.868+0000 7feefd39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef80b0a00 0x7feef815d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.868+0000 7feefd39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feef8157270 0x7feef8147960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feefd39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef815c7e0 con 0x7feef80b0a00 2023-10-06T05:07:53.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feefd39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef815bbd0 con 0x7feef80688d0 2023-10-06T05:07:53.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feefd39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef815beb0 con 0x7feef8157270 2023-10-06T05:07:53.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feef77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef80b0a00 0x7feef815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feef77fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef80b0a00 0x7feef815d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54546/0 (socket says 172.21.15.55:54546) 2023-10-06T05:07:53.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feef77fe700 1 -- 172.21.15.55:0/2217867055 learned_addr learned my addr 172.21.15.55:0/2217867055 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:53.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feef7fff700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feef80688d0 0x7feef80aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feefcb9c700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feef8157270 0x7feef8147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feef77fe700 1 -- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feef8157270 msgr2=0x7feef8147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feef77fe700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feef8157270 0x7feef8147960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feef77fe700 1 -- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feef80688d0 msgr2=0x7feef80aeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:53.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feef77fe700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feef80688d0 0x7feef80aeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:53.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feef77fe700 1 -- 172.21.15.55:0/2217867055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec009420 con 0x7feef80b0a00 2023-10-06T05:07:53.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feef7fff700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feef80688d0 0x7feef80aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.869+0000 7feef77fe700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef80b0a00 0x7feef815d190 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7feeec0030c0 tx=0x7feeec016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.870+0000 7feedffff700 1 -- 172.21.15.55:0/2217867055 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec01cc90 con 0x7feef80b0a00 2023-10-06T05:07:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.870+0000 7feedffff700 1 -- 172.21.15.55:0/2217867055 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec01cdf0 con 0x7feef80b0a00 2023-10-06T05:07:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.870+0000 7feedffff700 1 -- 172.21.15.55:0/2217867055 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec01a6a0 con 0x7feef80b0a00 2023-10-06T05:07:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.870+0000 7feefd39d700 1 -- 172.21.15.55:0/2217867055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef815c190 con 0x7feef80b0a00 2023-10-06T05:07:53.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.870+0000 7feefd39d700 1 -- 172.21.15.55:0/2217867055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef815b880 con 0x7feef80b0a00 2023-10-06T05:07:53.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.871+0000 7feefd39d700 1 -- 172.21.15.55:0/2217867055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef804c3c0 con 0x7feef80b0a00 2023-10-06T05:07:53.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.872+0000 7feedffff700 1 -- 172.21.15.55:0/2217867055 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feeec01aad0 con 0x7feef80b0a00 2023-10-06T05:07:53.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.873+0000 7feedffff700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7feee4066f30 0x7feee40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:53.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.873+0000 7feedffff700 1 -- 172.21.15.55:0/2217867055 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7feeec086280 con 0x7feef80b0a00 2023-10-06T05:07:53.875 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.873+0000 7feef7fff700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7feee4066f30 0x7feee40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:53.876 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.874+0000 7feef7fff700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7feee4066f30 0x7feee40693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feee8009ce0 tx=0x7feee8008040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:53.878 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:53.877+0000 7feedffff700 1 -- 172.21.15.55:0/2217867055 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feeec053200 con 0x7feef80b0a00 2023-10-06T05:07:54.159 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:54.158+0000 7feefd39d700 1 -- 172.21.15.55:0/2217867055 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7feef815a330 con 0x7feef80b0a00 2023-10-06T05:07:54.349 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:54.347+0000 7feedffff700 1 -- 172.21.15.55:0/2217867055 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3070) v1 ==== 155+0+0 (secure 0 0 0) 0x7feeec056820 con 0x7feef80b0a00 2023-10-06T05:07:54.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:54.350+0000 7feefd39d700 1 -- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7feee4066f30 msgr2=0x7feee40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:54.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:54.350+0000 7feefd39d700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7feee4066f30 0x7feee40693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feee8009ce0 tx=0x7feee8008040 comp rx=0 tx=0).stop 2023-10-06T05:07:54.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:54.350+0000 7feefd39d700 1 -- 172.21.15.55:0/2217867055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef80b0a00 msgr2=0x7feef815d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:54.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:54.350+0000 7feefd39d700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef80b0a00 0x7feef815d190 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7feeec0030c0 tx=0x7feeec016000 comp rx=0 tx=0).stop 2023-10-06T05:07:54.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:54.350+0000 7feefd39d700 1 -- 172.21.15.55:0/2217867055 shutdown_connections 2023-10-06T05:07:54.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:54.350+0000 7feefd39d700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7feee4066f30 0x7feee40693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:54.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:54.350+0000 7feefd39d700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7feef8157270 0x7feef8147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:54.352 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:54.350+0000 7feefd39d700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7feef80688d0 0x7feef80aeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:54.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:54.350+0000 7feefd39d700 1 --2- 172.21.15.55:0/2217867055 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7feef80b0a00 0x7feef815d190 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:54.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:54.350+0000 7feefd39d700 1 -- 172.21.15.55:0/2217867055 >> 172.21.15.55:0/2217867055 conn(0x7feef80cd310 msgr2=0x7feef81563d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:54.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:54.350+0000 7feefd39d700 1 -- 172.21.15.55:0/2217867055 shutdown_connections 2023-10-06T05:07:54.353 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:54.350+0000 7feefd39d700 1 -- 172.21.15.55:0/2217867055 wait complete. 2023-10-06T05:07:54.369 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-10-06T05:07:54.369 INFO:tasks.cephfs_test_runner: 2023-10-06T05:07:54.370 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T05:07:54.370 INFO:tasks.cephfs_test_runner:Ran 11 tests in 5640.653s 2023-10-06T05:07:54.370 INFO:tasks.cephfs_test_runner: 2023-10-06T05:07:54.370 INFO:tasks.cephfs_test_runner:OK 2023-10-06T05:07:54.370 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-06T05:07:54.381 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-06T05:07:54.390 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-06T05:07:54.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:54.390 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T05:07:54.428 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T05:07:54.428 INFO:teuthology.orchestra.run.smithi055.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T05:07:54.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:54.429 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T05:07:54.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:54.532 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T05:07:54.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:54.603 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link delete ceph-brx 2023-10-06T05:07:54.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:54.725 DEBUG:teuthology.orchestra.run.smithi055:> ip netns list 2023-10-06T05:07:54.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:54.762 DEBUG:teuthology.orchestra.run.smithi055:> sudo ip link delete ceph-brx 2023-10-06T05:07:54.870 INFO:teuthology.orchestra.run.smithi055.stderr:Cannot find device "ceph-brx" 2023-10-06T05:07:54.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:07:54.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:54.874 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-06T05:07:54.915 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T05:07:54.915 INFO:teuthology.orchestra.run.smithi163.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T05:07:54.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:54.916 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T05:07:55.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:55.024 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T05:07:55.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:55.091 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2023-10-06T05:07:55.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:55.230 DEBUG:teuthology.orchestra.run.smithi163:> ip netns list 2023-10-06T05:07:55.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T05:07:55.269 DEBUG:teuthology.orchestra.run.smithi163:> sudo ip link delete ceph-brx 2023-10-06T05:07:55.377 INFO:teuthology.orchestra.run.smithi163.stderr:Cannot find device "ceph-brx" 2023-10-06T05:07:55.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:07:55.380 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T05:07:55.391 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T05:07:55.392 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:07:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.580+0000 7f678af9d700 1 Processor -- start 2023-10-06T05:07:55.582 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.580+0000 7f678af9d700 1 -- start start 2023-10-06T05:07:55.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.582+0000 7f678af9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6784150550 0x7f6784157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:55.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.582+0000 7f678af9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f67840c7410 0x7f6784069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:55.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.582+0000 7f678af9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678406e2a0 con 0x7f6784150180 2023-10-06T05:07:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.582+0000 7f678af9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678406b2e0 con 0x7f67840c7410 2023-10-06T05:07:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.582+0000 7f678af9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67840c78e0 con 0x7f6784150550 2023-10-06T05:07:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.582+0000 7f678a79c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f67840c7410 0x7f6784069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.582+0000 7f678a79c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f67840c7410 0x7f6784069f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54560/0 (socket says 172.21.15.55:54560) 2023-10-06T05:07:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.582+0000 7f678a79c700 1 -- 172.21.15.55:0/2842629451 learned_addr learned my addr 172.21.15.55:0/2842629451 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.582+0000 7f678979a700 1 --2- 172.21.15.55:0/2842629451 >> v2:172.21.15.163:3300/0 conn(0x7f6784150550 0x7f6784157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.582+0000 7f678a79c700 1 -- 172.21.15.55:0/2842629451 >> v2:172.21.15.163:3300/0 conn(0x7f6784150550 msgr2=0x7f6784157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.582+0000 7f678a79c700 1 --2- 172.21.15.55:0/2842629451 >> v2:172.21.15.163:3300/0 conn(0x7f6784150550 0x7f6784157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.583+0000 7f678a79c700 1 -- 172.21.15.55:0/2842629451 >> v1:172.21.15.163:6789/0 conn(0x7f6784150180 legacy=0x7f67840c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.583+0000 7f678a79c700 1 -- 172.21.15.55:0/2842629451 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67840c7bc0 con 0x7f67840c7410 2023-10-06T05:07:55.585 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.583+0000 7f678a79c700 1 --2- 172.21.15.55:0/2842629451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67840c7410 0x7f6784069f50 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f678406d1e0 tx=0x7f677c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46e1bf52028e1a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:55.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.583+0000 7f6788f99700 1 -- 172.21.15.55:0/2842629451 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c00e070 con 0x7f67840c7410 2023-10-06T05:07:55.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.583+0000 7f6788f99700 1 -- 172.21.15.55:0/2842629451 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c00ba50 con 0x7f67840c7410 2023-10-06T05:07:55.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.583+0000 7f6788f99700 1 -- 172.21.15.55:0/2842629451 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c0092e0 con 0x7f67840c7410 2023-10-06T05:07:55.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.584+0000 7f678af9d700 1 -- 172.21.15.55:0/2842629451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67840c7410 msgr2=0x7f6784069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:55.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.584+0000 7f678af9d700 1 --2- 172.21.15.55:0/2842629451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67840c7410 0x7f6784069f50 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f678406d1e0 tx=0x7f677c009840 comp rx=0 tx=0).stop 2023-10-06T05:07:55.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.584+0000 7f678af9d700 1 -- 172.21.15.55:0/2842629451 shutdown_connections 2023-10-06T05:07:55.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.584+0000 7f678af9d700 1 --2- 172.21.15.55:0/2842629451 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f67840c7410 0x7f6784069f50 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:55.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.584+0000 7f678af9d700 1 --2- 172.21.15.55:0/2842629451 >> v2:172.21.15.163:3300/0 conn(0x7f6784150550 0x7f6784157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:55.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.584+0000 7f678af9d700 1 -- 172.21.15.55:0/2842629451 >> 172.21.15.55:0/2842629451 conn(0x7f67840cc1f0 msgr2=0x7f67840cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:55.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.584+0000 7f678af9d700 1 -- 172.21.15.55:0/2842629451 shutdown_connections 2023-10-06T05:07:55.587 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.584+0000 7f678af9d700 1 -- 172.21.15.55:0/2842629451 wait complete. 2023-10-06T05:07:55.589 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.585+0000 7f678af9d700 1 Processor -- start 2023-10-06T05:07:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.585+0000 7f678af9d700 1 -- start start 2023-10-06T05:07:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.586+0000 7f678af9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67840c7410 0x7f678413ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.586+0000 7f678af9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6784150180 0x7f67840c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.586+0000 7f678af9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6784150550 0x7f67840c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.586+0000 7f678af9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67840afd40 con 0x7f6784150550 2023-10-06T05:07:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.586+0000 7f678af9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67840af130 con 0x7f6784150180 2023-10-06T05:07:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.586+0000 7f678af9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67840af410 con 0x7f67840c7410 2023-10-06T05:07:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.587+0000 7f678a79c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6784150550 0x7f67840c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.587+0000 7f678a79c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6784150550 0x7f67840c4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54562/0 (socket says 172.21.15.55:54562) 2023-10-06T05:07:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.587+0000 7f678a79c700 1 -- 172.21.15.55:0/4015505772 learned_addr learned my addr 172.21.15.55:0/4015505772 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:55.590 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.587+0000 7f678979a700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6784150180 0x7f67840c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:55.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.587+0000 7f6789f9b700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67840c7410 0x7f678413ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:55.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.587+0000 7f678a79c700 1 -- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67840c7410 msgr2=0x7f678413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:55.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.587+0000 7f678a79c700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67840c7410 0x7f678413ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:55.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.587+0000 7f678a79c700 1 -- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6784150180 msgr2=0x7f67840c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:55.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.587+0000 7f678a79c700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6784150180 0x7f67840c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:55.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.587+0000 7f678a79c700 1 -- 172.21.15.55:0/4015505772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c009ae0 con 0x7f6784150550 2023-10-06T05:07:55.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.587+0000 7f6789f9b700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67840c7410 0x7f678413ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:55.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.587+0000 7f678a79c700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6784150550 0x7f67840c4ea0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f677c009880 tx=0x7f677c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:55.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.588+0000 7f6779ffb700 1 -- 172.21.15.55:0/4015505772 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c008dd0 con 0x7f6784150550 2023-10-06T05:07:55.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.588+0000 7f6779ffb700 1 -- 172.21.15.55:0/4015505772 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677c026460 con 0x7f6784150550 2023-10-06T05:07:55.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.588+0000 7f6779ffb700 1 -- 172.21.15.55:0/4015505772 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677c026820 con 0x7f6784150550 2023-10-06T05:07:55.591 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.588+0000 7f678af9d700 1 -- 172.21.15.55:0/4015505772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67840af6f0 con 0x7f6784150550 2023-10-06T05:07:55.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.588+0000 7f678af9d700 1 -- 172.21.15.55:0/4015505772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678414b290 con 0x7f6784150550 2023-10-06T05:07:55.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.589+0000 7f678af9d700 1 -- 172.21.15.55:0/4015505772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67840afef0 con 0x7f6784150550 2023-10-06T05:07:55.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.590+0000 7f6779ffb700 1 -- 172.21.15.55:0/4015505772 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f677c003040 con 0x7f6784150550 2023-10-06T05:07:55.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.590+0000 7f6779ffb700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6768066f30 0x7f67680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:55.592 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.590+0000 7f6779ffb700 1 -- 172.21.15.55:0/4015505772 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f677c086200 con 0x7f6784150550 2023-10-06T05:07:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.591+0000 7f6789f9b700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6768066f30 0x7f67680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:55.593 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.592+0000 7f6789f9b700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6768066f30 0x7f67680693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6780007fd0 tx=0x7f6780009040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:55.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.595+0000 7f6779ffb700 1 -- 172.21.15.55:0/4015505772 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67840afef0 con 0x7f6784150550 2023-10-06T05:07:55.799 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.797+0000 7f678af9d700 1 -- 172.21.15.55:0/4015505772 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67840c9a40 con 0x7f6784150550 2023-10-06T05:07:55.800 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.798+0000 7f6779ffb700 1 -- 172.21.15.55:0/4015505772 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v212) v1 ==== 74+0+15337 (secure 0 0 0) 0x7f677c053180 con 0x7f6784150550 2023-10-06T05:07:55.800 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:55.801 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":212,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:07:52.329367+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/1081678834":"2023-10-07T05:07:49.312979+0000","172.21.15.163:0/717025412":"2023-10-07T05:07:49.312979+0000","172.21.15.163:0/843041363":"2023-10-07T05:07:49.312979+0000","172.21.15.163:0/1037233743":"2023-10-07T05:07:49.312979+0000","172.21.15.163:0/1370950731":"2023-10-07T05:07:49.312979+0000","172.21.15.163:0/2693142845":"2023-10-07T05:07:49.312979+0000","172.21.15.55:6833/3632968604":"2023-10-07T05:07:49.122009+0000","172.21.15.55:6835/3632968604":"2023-10-07T05:07:49.122009+0000","172.21.15.163:0/2709454586":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/838216209":"2023-10-07T05:07:49.312979+0000","172.21.15.163:6837/101089":"2023-10-07T05:07:39.103885+0000","172.21.15.163:6836/101089":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/2719429847":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/3618319475":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/2222063413":"2023-10-07T05:07:39.103885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:07:55.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.801+0000 7f678af9d700 1 -- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6768066f30 msgr2=0x7f67680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:55.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.801+0000 7f678af9d700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6768066f30 0x7f67680693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6780007fd0 tx=0x7f6780009040 comp rx=0 tx=0).stop 2023-10-06T05:07:55.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.801+0000 7f678af9d700 1 -- 172.21.15.55:0/4015505772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6784150550 msgr2=0x7f67840c4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:55.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.801+0000 7f678af9d700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6784150550 0x7f67840c4ea0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f677c009880 tx=0x7f677c018000 comp rx=0 tx=0).stop 2023-10-06T05:07:55.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.801+0000 7f678af9d700 1 -- 172.21.15.55:0/4015505772 shutdown_connections 2023-10-06T05:07:55.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.801+0000 7f678af9d700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6768066f30 0x7f67680693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:55.803 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.801+0000 7f678af9d700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6784150550 0x7f67840c4ea0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:55.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.801+0000 7f678af9d700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f67840c7410 0x7f678413ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:55.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.801+0000 7f678af9d700 1 --2- 172.21.15.55:0/4015505772 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6784150180 0x7f67840c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:55.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.801+0000 7f678af9d700 1 -- 172.21.15.55:0/4015505772 >> 172.21.15.55:0/4015505772 conn(0x7f67840cc1f0 msgr2=0x7f6784157800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:55.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.801+0000 7f678af9d700 1 -- 172.21.15.55:0/4015505772 shutdown_connections 2023-10-06T05:07:55.804 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:55.801+0000 7f678af9d700 1 -- 172.21.15.55:0/4015505772 wait complete. 2023-10-06T05:07:55.821 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T05:07:55.821 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:07:56.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.006+0000 7f8afe59d700 1 Processor -- start 2023-10-06T05:07:56.008 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.006+0000 7f8afe59d700 1 -- start start 2023-10-06T05:07:56.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.007+0000 7f8afe59d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f8af8159420 0x7f8af80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.010 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.007+0000 7f8afe59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8af81597f0 0x7f8af80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.008+0000 7f8afe59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af80b11d0 con 0x7f8af815d230 2023-10-06T05:07:56.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.008+0000 7f8afe59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af80b14b0 con 0x7f8af81597f0 2023-10-06T05:07:56.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.008+0000 7f8afe59d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8af80b1790 con 0x7f8af8159420 2023-10-06T05:07:56.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.008+0000 7f8afcd9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8af81597f0 0x7f8af80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.008+0000 7f8afcd9a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f8af81597f0 0x7f8af80b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54578/0 (socket says 172.21.15.55:54578) 2023-10-06T05:07:56.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.008+0000 7f8afcd9a700 1 -- 172.21.15.55:0/2525763166 learned_addr learned my addr 172.21.15.55:0/2525763166 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:56.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.008+0000 7f8afd59b700 1 --2- 172.21.15.55:0/2525763166 >> v2:172.21.15.163:3300/0 conn(0x7f8af8159420 0x7f8af80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.008+0000 7f8afcd9a700 1 -- 172.21.15.55:0/2525763166 >> v2:172.21.15.163:3300/0 conn(0x7f8af8159420 msgr2=0x7f8af80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:56.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.008+0000 7f8afcd9a700 1 --2- 172.21.15.55:0/2525763166 >> v2:172.21.15.163:3300/0 conn(0x7f8af8159420 0x7f8af80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.008+0000 7f8afcd9a700 1 -- 172.21.15.55:0/2525763166 >> v1:172.21.15.163:6789/0 conn(0x7f8af815d230 legacy=0x7f8af815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.011 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.008+0000 7f8afcd9a700 1 -- 172.21.15.55:0/2525763166 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af80b1a70 con 0x7f8af81597f0 2023-10-06T05:07:56.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.009+0000 7f8afcd9a700 1 --2- 172.21.15.55:0/2525763166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af81597f0 0x7f8af80b2270 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f8af80b2f20 tx=0x7f8ae8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7aa0a7b7a86f35f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:56.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.009+0000 7f8aeffff700 1 -- 172.21.15.55:0/2525763166 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae800b860 con 0x7f8af81597f0 2023-10-06T05:07:56.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.009+0000 7f8aeffff700 1 -- 172.21.15.55:0/2525763166 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae800b9c0 con 0x7f8af81597f0 2023-10-06T05:07:56.012 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.009+0000 7f8aeffff700 1 -- 172.21.15.55:0/2525763166 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae800bd60 con 0x7f8af81597f0 2023-10-06T05:07:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.009+0000 7f8afe59d700 1 -- 172.21.15.55:0/2525763166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af81597f0 msgr2=0x7f8af80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.009+0000 7f8afe59d700 1 --2- 172.21.15.55:0/2525763166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af81597f0 0x7f8af80b2270 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f8af80b2f20 tx=0x7f8ae8009250 comp rx=0 tx=0).stop 2023-10-06T05:07:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.009+0000 7f8afe59d700 1 -- 172.21.15.55:0/2525763166 shutdown_connections 2023-10-06T05:07:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.009+0000 7f8afe59d700 1 --2- 172.21.15.55:0/2525763166 >> v2:172.21.15.163:3300/0 conn(0x7f8af8159420 0x7f8af80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.009+0000 7f8afe59d700 1 --2- 172.21.15.55:0/2525763166 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af81597f0 0x7f8af80b2270 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.009+0000 7f8afe59d700 1 -- 172.21.15.55:0/2525763166 >> 172.21.15.55:0/2525763166 conn(0x7f8af80d0660 msgr2=0x7f8af80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.010+0000 7f8afe59d700 1 -- 172.21.15.55:0/2525763166 shutdown_connections 2023-10-06T05:07:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.010+0000 7f8afe59d700 1 -- 172.21.15.55:0/2525763166 wait complete. 2023-10-06T05:07:56.013 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.011+0000 7f8afe59d700 1 Processor -- start 2023-10-06T05:07:56.014 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.011+0000 7f8afe59d700 1 -- start start 2023-10-06T05:07:56.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.012+0000 7f8afe59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af8159420 0x7f8af816adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.012+0000 7f8afe59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af81597f0 0x7f8af8169bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.015 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.012+0000 7f8afe59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af815d230 0x7f8af8155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.012+0000 7f8afe59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af80c5770 con 0x7f8af815d230 2023-10-06T05:07:56.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.012+0000 7f8afe59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af816a4d0 con 0x7f8af8159420 2023-10-06T05:07:56.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.012+0000 7f8afe59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af816a7b0 con 0x7f8af81597f0 2023-10-06T05:07:56.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.012+0000 7f8afdd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af815d230 0x7f8af8155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.012+0000 7f8afdd9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af815d230 0x7f8af8155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54590/0 (socket says 172.21.15.55:54590) 2023-10-06T05:07:56.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.012+0000 7f8afdd9c700 1 -- 172.21.15.55:0/596407688 learned_addr learned my addr 172.21.15.55:0/596407688 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:56.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.012+0000 7f8afd59b700 1 --2- 172.21.15.55:0/596407688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af8159420 0x7f8af816adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.012+0000 7f8afcd9a700 1 --2- 172.21.15.55:0/596407688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af81597f0 0x7f8af8169bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.013+0000 7f8afdd9c700 1 -- 172.21.15.55:0/596407688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af81597f0 msgr2=0x7f8af8169bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.013+0000 7f8afdd9c700 1 --2- 172.21.15.55:0/596407688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af81597f0 0x7f8af8169bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.013+0000 7f8afdd9c700 1 -- 172.21.15.55:0/596407688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af8159420 msgr2=0x7f8af816adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.016 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.013+0000 7f8afdd9c700 1 --2- 172.21.15.55:0/596407688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af8159420 0x7f8af816adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.013+0000 7f8afdd9c700 1 -- 172.21.15.55:0/596407688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae80094f0 con 0x7f8af815d230 2023-10-06T05:07:56.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.013+0000 7f8afdd9c700 1 --2- 172.21.15.55:0/596407688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af815d230 0x7f8af8155d60 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f8af000c810 tx=0x7f8af0001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:56.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.013+0000 7f8aed7fa700 1 -- 172.21.15.55:0/596407688 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0012630 con 0x7f8af815d230 2023-10-06T05:07:56.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.013+0000 7f8aed7fa700 1 -- 172.21.15.55:0/596407688 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af00127b0 con 0x7f8af815d230 2023-10-06T05:07:56.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.013+0000 7f8aed7fa700 1 -- 172.21.15.55:0/596407688 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0012b70 con 0x7f8af815d230 2023-10-06T05:07:56.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.013+0000 7f8afe59d700 1 -- 172.21.15.55:0/596407688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af816aa90 con 0x7f8af815d230 2023-10-06T05:07:56.017 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.013+0000 7f8afe59d700 1 -- 172.21.15.55:0/596407688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af816b300 con 0x7f8af815d230 2023-10-06T05:07:56.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.014+0000 7f8afe59d700 1 -- 172.21.15.55:0/596407688 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af80b35d0 con 0x7f8af815d230 2023-10-06T05:07:56.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.015+0000 7f8aed7fa700 1 -- 172.21.15.55:0/596407688 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8af0003030 con 0x7f8af815d230 2023-10-06T05:07:56.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.016+0000 7f8aed7fa700 1 --2- 172.21.15.55:0/596407688 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8adc066f30 0x7f8adc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.016+0000 7f8aed7fa700 1 -- 172.21.15.55:0/596407688 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f8af00846d0 con 0x7f8af815d230 2023-10-06T05:07:56.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.016+0000 7f8afd59b700 1 --2- 172.21.15.55:0/596407688 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8adc066f30 0x7f8adc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.019 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.017+0000 7f8afd59b700 1 --2- 172.21.15.55:0/596407688 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8adc066f30 0x7f8adc0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8af4006fd0 tx=0x7f8af4008040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:56.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.020+0000 7f8aed7fa700 1 -- 172.21.15.55:0/596407688 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8af0051650 con 0x7f8af815d230 2023-10-06T05:07:56.220 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.218+0000 7f8afe59d700 1 -- 172.21.15.55:0/596407688 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8af816a920 con 0x7f8adc066f30 2023-10-06T05:07:56.223 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.222+0000 7f8aed7fa700 1 -- 172.21.15.55:0/596407688 <== mgr.10033 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72684 (secure 0 0 0) 0x7f8af816a920 con 0x7f8adc066f30 2023-10-06T05:07:56.224 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:56.226 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.224+0000 7f8afe59d700 1 -- 172.21.15.55:0/596407688 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8adc066f30 msgr2=0x7f8adc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.224+0000 7f8afe59d700 1 --2- 172.21.15.55:0/596407688 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8adc066f30 0x7f8adc0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8af4006fd0 tx=0x7f8af4008040 comp rx=0 tx=0).stop 2023-10-06T05:07:56.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.224+0000 7f8afe59d700 1 -- 172.21.15.55:0/596407688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af815d230 msgr2=0x7f8af8155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.224+0000 7f8afe59d700 1 --2- 172.21.15.55:0/596407688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af815d230 0x7f8af8155d60 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f8af000c810 tx=0x7f8af0001650 comp rx=0 tx=0).stop 2023-10-06T05:07:56.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.224+0000 7f8afe59d700 1 -- 172.21.15.55:0/596407688 shutdown_connections 2023-10-06T05:07:56.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.224+0000 7f8afe59d700 1 --2- 172.21.15.55:0/596407688 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f8adc066f30 0x7f8adc0693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.224+0000 7f8afe59d700 1 --2- 172.21.15.55:0/596407688 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f8af815d230 0x7f8af8155d60 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.224+0000 7f8afe59d700 1 --2- 172.21.15.55:0/596407688 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f8af8159420 0x7f8af816adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.224+0000 7f8afe59d700 1 --2- 172.21.15.55:0/596407688 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f8af81597f0 0x7f8af8169bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.224+0000 7f8afe59d700 1 -- 172.21.15.55:0/596407688 >> 172.21.15.55:0/596407688 conn(0x7f8af80d0660 msgr2=0x7f8af80b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:56.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.224+0000 7f8afe59d700 1 -- 172.21.15.55:0/596407688 shutdown_connections 2023-10-06T05:07:56.227 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.224+0000 7f8afe59d700 1 -- 172.21.15.55:0/596407688 wait complete. 2023-10-06T05:07:56.228 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T05:07:56.244 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2023-10-06T05:07:55.333895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445556,"kb_used_data":22196,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":752529164,"statfs":{"total":773094113280,"available":770589863936,"internally_reserved":0,"allocated":22728704,"data_stored":12737419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668755,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-281,"num_write_kb":-2052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.995111"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":191,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:07:49.128504+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T05:07:49.128504+0000","last_peered":"2023-10-06T05:07:49.128504+0000","last_clean":"2023-10-06T05:07:49.128504+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T05:07:49.128504+0000","last_undegraded":"2023-10-06T05:07:49.128504+0000","last_fullsized":"2023-10-06T05:07:49.128504+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":180,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-06T05:07:51.875137+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T05:07:51.875137+0000","last_peered":"2023-10-06T05:07:51.875137+0000","last_clean":"2023-10-06T05:07:51.875137+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T05:07:51.875137+0000","last_undegraded":"2023-10-06T05:07:51.875137+0000","last_fullsized":"2023-10-06T05:07:51.875137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":175,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-06T05:07:51.332982+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T05:07:51.332982+0000","last_peered":"2023-10-06T05:07:51.332982+0000","last_clean":"2023-10-06T05:07:51.332982+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T05:07:51.332982+0000","last_undegraded":"2023-10-06T05:07:51.332982+0000","last_fullsized":"2023-10-06T05:07:51.332982+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"210'370","reported_seq":1221,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-10-06T05:07:49.402673+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T05:07:49.402673+0000","last_peered":"2023-10-06T05:07:49.402673+0000","last_clean":"2023-10-06T05:07:49.402673+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T05:07:49.402673+0000","last_undegraded":"2023-10-06T05:07:49.402673+0000","last_fullsized":"2023-10-06T05:07:49.402673+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":175,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-10-06T05:07:39.110639+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T05:07:39.110639+0000","last_peered":"2023-10-06T05:07:39.110639+0000","last_clean":"2023-10-06T05:07:39.110639+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T05:07:39.110639+0000","last_undegraded":"2023-10-06T05:07:39.110639+0000","last_fullsized":"2023-10-06T05:07:39.110639+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":175,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-10-06T05:07:39.110733+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T05:07:39.110733+0000","last_peered":"2023-10-06T05:07:39.110733+0000","last_clean":"2023-10-06T05:07:39.110733+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T05:07:39.110733+0000","last_undegraded":"2023-10-06T05:07:39.110733+0000","last_fullsized":"2023-10-06T05:07:39.110733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":168,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-10-06T05:07:39.171063+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T05:07:39.171063+0000","last_peered":"2023-10-06T05:07:39.171063+0000","last_clean":"2023-10-06T05:07:39.171063+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T05:07:39.171063+0000","last_undegraded":"2023-10-06T05:07:39.171063+0000","last_fullsized":"2023-10-06T05:07:39.171063+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":203,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:07:49.128754+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T05:07:49.128754+0000","last_peered":"2023-10-06T05:07:49.128754+0000","last_clean":"2023-10-06T05:07:49.128754+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T05:07:49.128754+0000","last_undegraded":"2023-10-06T05:07:49.128754+0000","last_fullsized":"2023-10-06T05:07:49.128754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":203,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:07:49.128855+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T05:07:49.128855+0000","last_peered":"2023-10-06T05:07:49.128855+0000","last_clean":"2023-10-06T05:07:49.128855+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T05:07:49.128855+0000","last_undegraded":"2023-10-06T05:07:49.128855+0000","last_fullsized":"2023-10-06T05:07:49.128855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575968,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304528,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067312,"statfs":{"total":96636764160,"available":96324927488,"internally_reserved":0,"allocated":3219456,"data_stored":1983282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94999999999999996,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0529999999999999},"last":0.63300000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79300000000000004,"15min":0.80600000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.996,"5min":1.0640000000000001,"15min":1.073},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.80300000000000005,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.0860000000000001},"last":0.65800000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72199999999999998,"15min":0.74099999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.52500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0509999999999999,"5min":1.107,"15min":1.1160000000000001},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.101},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.753,"15min":0.76600000000000001},"min":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96099999999999997,"5min":1.0149999999999999,"15min":1.0369999999999999},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.355},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64000000000000001,"15min":0.64500000000000002},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.40999999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.208},"last":0.47999999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47699999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.754,"15min":0.749},"min":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.48799999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.51500000000000001,"5min":0.48999999999999999,"15min":0.46700000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.53000000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300188,"kb_used_data":2900,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071652,"statfs":{"total":96636764160,"available":96329371648,"internally_reserved":0,"allocated":2969600,"data_stored":1722295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":1.0009999999999999,"15min":1.2969999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94999999999999996,"5min":1.083,"15min":1.083},"last":0.64200000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0640000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0960000000000001,"5min":1.425,"15min":1.425},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.752,"15min":0.76100000000000001},"min":{"1min":0.48099999999999998,"5min":0.47299999999999998,"15min":0.44700000000000001},"max":{"1min":0.91800000000000004,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96499999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.505}]},{"osd":3,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78200000000000003,"15min":0.78600000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":0.998,"5min":1.016,"15min":1.1699999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.79800000000000004},"min":{"1min":0.61299999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94599999999999995,"5min":1.0409999999999999,"15min":1.1479999999999999},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.40200000000000002,"15min":0.38900000000000001},"max":{"1min":0.81399999999999995,"5min":0.95999999999999996,"15min":1.0269999999999999},"last":0.628},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.64400000000000002},"min":{"1min":0.51800000000000002,"5min":0.46600000000000003,"15min":0.39300000000000002},"max":{"1min":0.89400000000000002,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.51400000000000001,"5min":0.5,"15min":0.41699999999999998},"max":{"1min":0.94199999999999995,"5min":1.02,"15min":1.069},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.41199999999999998},"max":{"1min":0.94199999999999995,"5min":1.0349999999999999,"15min":1.1950000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93999999999999995,"5min":1.208,"15min":1.208},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.48599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97399999999999998,"5min":1.0449999999999999,"15min":1.1579999999999999},"last":0.72599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316852,"kb_used_data":3180,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94054988,"statfs":{"total":96636764160,"available":96312307712,"internally_reserved":0,"allocated":3256320,"data_stored":1998524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.32700000000000001},"max":{"1min":1.093,"5min":1.155,"15min":1.155},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.59299999999999997},"min":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.38900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.90100000000000002},"last":0.46600000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.72699999999999998,"5min":0.76400000000000001,"15min":1.153},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.42199999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1,"5min":1.018,"15min":1.018},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.435,"15min":0.41799999999999998},"max":{"1min":0.95499999999999996,"5min":1.113,"15min":1.131},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92200000000000004,"5min":0.95499999999999996,"15min":0.96999999999999997},"last":0.55300000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79600000000000004,"15min":0.80600000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.02,"5min":1.2210000000000001,"15min":1.2250000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.82199999999999995,"15min":0.84299999999999997},"min":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.147,"5min":1.1930000000000001,"15min":1.236},"last":0.70799999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.82699999999999996,"15min":0.83999999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.53600000000000003},"max":{"1min":1.0660000000000001,"5min":1.1240000000000001,"15min":1.228},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.51000000000000001},"max":{"1min":1.139,"5min":1.1799999999999999,"15min":1.2829999999999999},"last":0.873}]},{"osd":6,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85199999999999998,"15min":0.85699999999999998},"min":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.59199999999999997},"max":{"1min":1.1779999999999999,"5min":1.2490000000000001,"15min":1.2709999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.85999999999999999,"15min":0.86399999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.163,"5min":1.242,"15min":1.2470000000000001},"last":0.92500000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.63,"5min":0.623,"15min":0.623},"max":{"1min":1.198,"5min":1.222,"15min":1.2949999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.872,"15min":0.88},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.224,"5min":1.224,"15min":1.3080000000000001},"last":0.72099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575971,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303980,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067860,"statfs":{"total":96636764160,"available":96325488640,"internally_reserved":0,"allocated":2658304,"data_stored":1407178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.28599999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.379},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.35999999999999999,"15min":0.34399999999999997},"max":{"1min":0.67600000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55500000000000005,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.38700000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.91000000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.46500000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.86699999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.63700000000000001,"15min":0.64700000000000002},"min":{"1min":0.49299999999999999,"5min":0.439,"15min":0.42899999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.67100000000000004,"5min":0.57899999999999996,"15min":0.54300000000000004},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.3049999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.63600000000000001,"5min":0.55700000000000005,"15min":0.55300000000000005},"max":{"1min":1.085,"5min":1.127,"15min":1.3440000000000001},"last":0.96199999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.83499999999999996,"15min":0.84399999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.2729999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82899999999999996,"15min":0.85099999999999998},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.45600000000000002},"max":{"1min":1.1339999999999999,"5min":1.204,"15min":1.218},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81999999999999995,"15min":0.83799999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.53800000000000003},"max":{"1min":1.0529999999999999,"5min":1.1240000000000001,"15min":1.2869999999999999},"last":1.004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86199999999999999,"15min":0.88300000000000001},"min":{"1min":0.67700000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.246,"5min":1.246,"15min":1.4399999999999999},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.85399999999999998,"15min":0.876},"min":{"1min":0.626,"5min":0.624,"15min":0.59799999999999998},"max":{"1min":1.216,"5min":1.2490000000000001,"15min":1.367},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.68300000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.294},"last":0.751}]}]},{"osd":2,"up_from":13,"seq":55834575970,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299884,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071956,"statfs":{"total":96636764160,"available":96329682944,"internally_reserved":0,"allocated":2658304,"data_stored":1407178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47299999999999998,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.32000000000000001,"15min":0.312},"max":{"1min":0.73699999999999999,"5min":0.746,"15min":0.80300000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.91200000000000003},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73899999999999999,"5min":0.78300000000000003,"15min":0.92400000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.51300000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.32400000000000001},"max":{"1min":0.76100000000000001,"5min":0.89800000000000002,"15min":0.90600000000000003},"last":0.70499999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.84799999999999998,"15min":0.85399999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0580000000000001,"5min":1.286,"15min":1.286},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.59199999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.254},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.88400000000000001,"15min":0.88800000000000001},"min":{"1min":0.67100000000000004,"5min":0.59099999999999997,"15min":0.57599999999999996},"max":{"1min":1.165,"5min":1.26,"15min":1.345},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.89000000000000001,"15min":0.89700000000000002},"min":{"1min":0.65500000000000003,"5min":0.628,"15min":0.628},"max":{"1min":1.085,"5min":1.2090000000000001,"15min":1.339},"last":1.0049999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58199999999999996},"max":{"1min":0.997,"5min":1.1279999999999999,"15min":1.2989999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1439999999999999,"5min":1.226,"15min":1.226},"last":0.98799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85999999999999999,"15min":0.86599999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1000000000000001,"5min":1.141,"15min":1.276},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90700000000000003,"15min":0.90900000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.59399999999999997},"max":{"1min":1.1850000000000001,"5min":1.2170000000000001,"15min":1.3680000000000001},"last":0.95999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575970,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299888,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071952,"statfs":{"total":96636764160,"available":96329678848,"internally_reserved":0,"allocated":2662400,"data_stored":1408156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51100000000000001,"15min":0.49399999999999999},"min":{"1min":0.34399999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67400000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.378},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85799999999999998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.76400000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.58299999999999996},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.48899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.74199999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65000000000000002,"15min":0.63500000000000001},"min":{"1min":0.48899999999999999,"5min":0.45100000000000001,"15min":0.38500000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.83599999999999997,"15min":0.82699999999999996},"min":{"1min":0.625,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1599999999999999,"5min":1.232,"15min":1.2849999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.87},"min":{"1min":0.64800000000000002,"5min":0.53900000000000003,"15min":0.52500000000000002},"max":{"1min":1.0860000000000001,"5min":1.232,"15min":1.4019999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80600000000000005,"15min":0.81999999999999995},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.52200000000000002},"max":{"1min":1.01,"5min":1.22,"15min":1.5760000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0980000000000001,"5min":1.1419999999999999,"15min":1.3300000000000001},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.84899999999999998,"15min":0.85099999999999998},"min":{"1min":0.66100000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0900000000000001,"5min":1.212,"15min":1.2150000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.87,"5min":0.88700000000000001,"15min":0.89100000000000001},"min":{"1min":0.64900000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1399999999999999,"5min":1.2929999999999999,"15min":1.321},"last":0.59199999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.87,"15min":0.87},"min":{"1min":0.70299999999999996,"5min":0.59399999999999997,"15min":0.57599999999999996},"max":{"1min":1.1619999999999999,"5min":1.198,"15min":1.2769999999999999},"last":0.624},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.86599999999999999,"15min":0.85399999999999998},"min":{"1min":0.69099999999999995,"5min":0.59299999999999997,"15min":0.55800000000000005},"max":{"1min":1.1200000000000001,"5min":1.1890000000000001,"15min":1.389},"last":0.60599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312168,"kb_used_data":2592,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059672,"statfs":{"total":96636764160,"available":96317104128,"internally_reserved":0,"allocated":2654208,"data_stored":1405898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.747},"min":{"1min":0.53800000000000003,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.89500000000000002,"5min":1.0820000000000001,"15min":1.2},"last":0.503},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.753,"15min":0.755},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97599999999999998,"5min":1.048,"15min":1.048},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72299999999999998,"15min":0.73699999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.96199999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94299999999999995,"5min":0.99099999999999999,"15min":1.258},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.50800000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75900000000000001,"15min":0.78300000000000003},"min":{"1min":0.50600000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.75700000000000001},"min":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.49299999999999999},"max":{"1min":0.97499999999999998,"5min":0.97999999999999998,"15min":1.2330000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.54900000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.95599999999999996,"5min":0.96799999999999997,"15min":1.034},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61399999999999999,"15min":0.63400000000000001},"min":{"1min":0.46000000000000002,"5min":0.43099999999999999,"15min":0.41199999999999998},"max":{"1min":0.81599999999999995,"5min":1.04,"15min":1.323},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.65200000000000002},"min":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.39700000000000002},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.1910000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.67700000000000005},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":1.024,"15min":1.079},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68799999999999994,"15min":0.71199999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.41999999999999998},"max":{"1min":0.84699999999999998,"5min":0.95099999999999996,"15min":1.3540000000000001},"last":0.621}]},{"osd":7,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.70599999999999996},"min":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.82699999999999996,"5min":1.0640000000000001,"15min":1.155},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.47299999999999998},"max":{"1min":0.93000000000000005,"5min":1.079,"15min":1.149},"last":0.66200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308068,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063772,"statfs":{"total":96636764160,"available":96321302528,"internally_reserved":0,"allocated":2650112,"data_stored":1404908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.748},"min":{"1min":0.53200000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.93500000000000005,"5min":1.079,"15min":1.079},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.72999999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88800000000000001,"5min":0.999,"15min":1.02},"last":0.498}]},{"osd":1,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.745,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.53600000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98399999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79600000000000004,"15min":0.79400000000000004},"min":{"1min":0.65300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.99199999999999999,"5min":1.0700000000000001,"15min":1.0780000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80100000000000005,"15min":0.80700000000000005},"min":{"1min":0.57599999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0149999999999999,"5min":1.048,"15min":1.048},"last":0.90700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.48599999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.94399999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.63500000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0169999999999999,"5min":1.095,"15min":1.095},"last":0.97099999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.629},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.38200000000000001},"max":{"1min":0.84399999999999997,"5min":0.97699999999999998,"15min":1.0049999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.5,"5min":0.38,"15min":0.38},"max":{"1min":0.89000000000000001,"5min":1.01,"15min":1.042},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.48499999999999999,"5min":0.45900000000000002,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.0369999999999999},"last":1.006},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.42799999999999999},"max":{"1min":0.90700000000000003,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.55000000000000004,"5min":0.48499999999999999,"15min":0.44500000000000001},"max":{"1min":0.92600000000000005,"5min":1.0269999999999999,"15min":1.0669999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.54000000000000004,"5min":0.51300000000000001,"15min":0.47199999999999998},"max":{"1min":0.93000000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.875}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:07:56.246 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:07:56.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.433+0000 7fe20a19d700 1 Processor -- start 2023-10-06T05:07:56.435 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.433+0000 7fe20a19d700 1 -- start start 2023-10-06T05:07:56.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.435+0000 7fe20a19d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe204156310 0x7fe2040c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.436 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.435+0000 7fe20a19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2040ae710 0x7fe2040c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.435+0000 7fe20a19d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2040ca860 con 0x7fe2041566e0 2023-10-06T05:07:56.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.435+0000 7fe20a19d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2040cab40 con 0x7fe204156310 2023-10-06T05:07:56.437 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.435+0000 7fe20a19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2040cae20 con 0x7fe2040ae710 2023-10-06T05:07:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.435+0000 7fe20999c700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2040ae710 0x7fe2040c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.435+0000 7fe20919b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe204156310 0x7fe2040c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.435+0000 7fe20919b700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fe204156310 0x7fe2040c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:45936/0 (socket says 172.21.15.55:45936) 2023-10-06T05:07:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.435+0000 7fe20919b700 1 -- 172.21.15.55:0/2534620280 learned_addr learned my addr 172.21.15.55:0/2534620280 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.435+0000 7fe20899a700 1 --1- 172.21.15.55:0/2534620280 >> v1:172.21.15.163:6789/0 conn(0x7fe2041566e0 0x7fe2040ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.163:6789/0 says I am v1:172.21.15.55:40412/0 (socket says 172.21.15.55:40412) 2023-10-06T05:07:56.438 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.436+0000 7fe20919b700 1 -- 172.21.15.55:0/2534620280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2040ae710 msgr2=0x7fe2040c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:56.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.436+0000 7fe20919b700 1 --2- 172.21.15.55:0/2534620280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2040ae710 0x7fe2040c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.436+0000 7fe20919b700 1 -- 172.21.15.55:0/2534620280 >> v1:172.21.15.163:6789/0 conn(0x7fe2041566e0 legacy=0x7fe2040ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.436+0000 7fe20919b700 1 -- 172.21.15.55:0/2534620280 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2040cb100 con 0x7fe204156310 2023-10-06T05:07:56.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.436+0000 7fe20999c700 1 --2- 172.21.15.55:0/2534620280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2040ae710 0x7fe2040c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:07:56.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.436+0000 7fe1fbfff700 1 -- 172.21.15.55:0/2534620280 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678336658 0 0) 0x7fe2040ca860 con 0x7fe2041566e0 2023-10-06T05:07:56.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.436+0000 7fe20919b700 1 --2- 172.21.15.55:0/2534620280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe204156310 0x7fe2040c9dc0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe2000099b0 tx=0x7fe200009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b0cf82a74bfe65f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:56.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.436+0000 7fe1fbfff700 1 -- 172.21.15.55:0/2534620280 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe200009ea0 con 0x7fe204156310 2023-10-06T05:07:56.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.436+0000 7fe1fbfff700 1 -- 172.21.15.55:0/2534620280 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20000b860 con 0x7fe204156310 2023-10-06T05:07:56.439 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.436+0000 7fe1fbfff700 1 -- 172.21.15.55:0/2534620280 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20000bc00 con 0x7fe204156310 2023-10-06T05:07:56.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.437+0000 7fe20a19d700 1 -- 172.21.15.55:0/2534620280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe204156310 msgr2=0x7fe2040c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.437+0000 7fe20a19d700 1 --2- 172.21.15.55:0/2534620280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe204156310 0x7fe2040c9dc0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe2000099b0 tx=0x7fe200009250 comp rx=0 tx=0).stop 2023-10-06T05:07:56.440 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.437+0000 7fe20a19d700 1 -- 172.21.15.55:0/2534620280 shutdown_connections 2023-10-06T05:07:56.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.437+0000 7fe20a19d700 1 --2- 172.21.15.55:0/2534620280 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2040ae710 0x7fe2040c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.437+0000 7fe20a19d700 1 --2- 172.21.15.55:0/2534620280 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe204156310 0x7fe2040c9dc0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.437+0000 7fe20a19d700 1 -- 172.21.15.55:0/2534620280 >> 172.21.15.55:0/2534620280 conn(0x7fe2040cc1f0 msgr2=0x7fe2040cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:56.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.437+0000 7fe20a19d700 1 -- 172.21.15.55:0/2534620280 shutdown_connections 2023-10-06T05:07:56.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.437+0000 7fe20a19d700 1 -- 172.21.15.55:0/2534620280 wait complete. 2023-10-06T05:07:56.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.439+0000 7fe20a19d700 1 Processor -- start 2023-10-06T05:07:56.441 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.439+0000 7fe20a19d700 1 -- start start 2023-10-06T05:07:56.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.440+0000 7fe20a19d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2040ae710 0x7fe2041576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.440+0000 7fe20a19d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe204156310 0x7fe204159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.440+0000 7fe20a19d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2041566e0 0x7fe204146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.440+0000 7fe20a19d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe204167820 con 0x7fe204156310 2023-10-06T05:07:56.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.440+0000 7fe20a19d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe204167af0 con 0x7fe2040ae710 2023-10-06T05:07:56.443 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.440+0000 7fe20a19d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe204157cc0 con 0x7fe2041566e0 2023-10-06T05:07:56.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.440+0000 7fe20899a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe204156310 0x7fe204159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.440+0000 7fe20899a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe204156310 0x7fe204159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54604/0 (socket says 172.21.15.55:54604) 2023-10-06T05:07:56.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.440+0000 7fe20899a700 1 -- 172.21.15.55:0/2195218090 learned_addr learned my addr 172.21.15.55:0/2195218090 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:56.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.440+0000 7fe20919b700 1 --2- 172.21.15.55:0/2195218090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2040ae710 0x7fe2041576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.440+0000 7fe20999c700 1 --2- 172.21.15.55:0/2195218090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2041566e0 0x7fe204146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.440+0000 7fe20899a700 1 -- 172.21.15.55:0/2195218090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2041566e0 msgr2=0x7fe204146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.440+0000 7fe20899a700 1 --2- 172.21.15.55:0/2195218090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2041566e0 0x7fe204146ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.440+0000 7fe20899a700 1 -- 172.21.15.55:0/2195218090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2040ae710 msgr2=0x7fe2041576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.441+0000 7fe20899a700 1 --2- 172.21.15.55:0/2195218090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2040ae710 0x7fe2041576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.441+0000 7fe20899a700 1 -- 172.21.15.55:0/2195218090 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc009a20 con 0x7fe204156310 2023-10-06T05:07:56.444 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.441+0000 7fe20899a700 1 --2- 172.21.15.55:0/2195218090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe204156310 0x7fe204159680 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400c7e0 tx=0x7fe1f4007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:56.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.441+0000 7fe1f97fa700 1 -- 172.21.15.55:0/2195218090 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4005b40 con 0x7fe204156310 2023-10-06T05:07:56.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.441+0000 7fe1f97fa700 1 -- 172.21.15.55:0/2195218090 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f4005ca0 con 0x7fe204156310 2023-10-06T05:07:56.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.441+0000 7fe1f97fa700 1 -- 172.21.15.55:0/2195218090 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f4007320 con 0x7fe204156310 2023-10-06T05:07:56.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.441+0000 7fe20a19d700 1 -- 172.21.15.55:0/2195218090 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2000094f0 con 0x7fe204156310 2023-10-06T05:07:56.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.441+0000 7fe20a19d700 1 -- 172.21.15.55:0/2195218090 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe204158280 con 0x7fe204156310 2023-10-06T05:07:56.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.442+0000 7fe20a19d700 1 -- 172.21.15.55:0/2195218090 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20404c3c0 con 0x7fe204156310 2023-10-06T05:07:56.445 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.443+0000 7fe1f97fa700 1 -- 172.21.15.55:0/2195218090 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1f401b3b0 con 0x7fe204156310 2023-10-06T05:07:56.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.444+0000 7fe1f97fa700 1 --2- 172.21.15.55:0/2195218090 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe1e8066f30 0x7fe1e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.444+0000 7fe1f97fa700 1 -- 172.21.15.55:0/2195218090 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7fe1f4083c80 con 0x7fe204156310 2023-10-06T05:07:56.446 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.445+0000 7fe20919b700 1 --2- 172.21.15.55:0/2195218090 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe1e8066f30 0x7fe1e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.447 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.445+0000 7fe20919b700 1 --2- 172.21.15.55:0/2195218090 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe1e8066f30 0x7fe1e80693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe2000099b0 tx=0x7fe200017000 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:56.450 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.449+0000 7fe1f97fa700 1 -- 172.21.15.55:0/2195218090 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1f4050cc0 con 0x7fe204156310 2023-10-06T05:07:56.647 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.646+0000 7fe20a19d700 1 -- 172.21.15.55:0/2195218090 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe204167c30 con 0x7fe1e8066f30 2023-10-06T05:07:56.651 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.649+0000 7fe1f97fa700 1 -- 172.21.15.55:0/2195218090 <== mgr.10033 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72684 (secure 0 0 0) 0x7fe204167c30 con 0x7fe1e8066f30 2023-10-06T05:07:56.652 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:56.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.652+0000 7fe20a19d700 1 -- 172.21.15.55:0/2195218090 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe1e8066f30 msgr2=0x7fe1e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.654 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.652+0000 7fe20a19d700 1 --2- 172.21.15.55:0/2195218090 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe1e8066f30 0x7fe1e80693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe2000099b0 tx=0x7fe200017000 comp rx=0 tx=0).stop 2023-10-06T05:07:56.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.652+0000 7fe20a19d700 1 -- 172.21.15.55:0/2195218090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe204156310 msgr2=0x7fe204159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.652+0000 7fe20a19d700 1 --2- 172.21.15.55:0/2195218090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe204156310 0x7fe204159680 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fe1f400c7e0 tx=0x7fe1f4007de0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.652+0000 7fe20a19d700 1 -- 172.21.15.55:0/2195218090 shutdown_connections 2023-10-06T05:07:56.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.652+0000 7fe20a19d700 1 --2- 172.21.15.55:0/2195218090 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe1e8066f30 0x7fe1e80693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.652+0000 7fe20a19d700 1 --2- 172.21.15.55:0/2195218090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe2041566e0 0x7fe204146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.652+0000 7fe20a19d700 1 --2- 172.21.15.55:0/2195218090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe2040ae710 0x7fe2041576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.652+0000 7fe20a19d700 1 --2- 172.21.15.55:0/2195218090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe204156310 0x7fe204159680 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.652+0000 7fe20a19d700 1 -- 172.21.15.55:0/2195218090 >> 172.21.15.55:0/2195218090 conn(0x7fe2040cc1f0 msgr2=0x7fe204069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:56.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.652+0000 7fe20a19d700 1 -- 172.21.15.55:0/2195218090 shutdown_connections 2023-10-06T05:07:56.655 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.652+0000 7fe20a19d700 1 -- 172.21.15.55:0/2195218090 wait complete. 2023-10-06T05:07:56.656 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T05:07:56.672 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2023-10-06T05:07:55.333895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445556,"kb_used_data":22196,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":752529164,"statfs":{"total":773094113280,"available":770589863936,"internally_reserved":0,"allocated":22728704,"data_stored":12737419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668755,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-281,"num_write_kb":-2052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.995111"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":191,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:07:49.128504+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T05:07:49.128504+0000","last_peered":"2023-10-06T05:07:49.128504+0000","last_clean":"2023-10-06T05:07:49.128504+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T05:07:49.128504+0000","last_undegraded":"2023-10-06T05:07:49.128504+0000","last_fullsized":"2023-10-06T05:07:49.128504+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":180,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-06T05:07:51.875137+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T05:07:51.875137+0000","last_peered":"2023-10-06T05:07:51.875137+0000","last_clean":"2023-10-06T05:07:51.875137+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T05:07:51.875137+0000","last_undegraded":"2023-10-06T05:07:51.875137+0000","last_fullsized":"2023-10-06T05:07:51.875137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":175,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-06T05:07:51.332982+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T05:07:51.332982+0000","last_peered":"2023-10-06T05:07:51.332982+0000","last_clean":"2023-10-06T05:07:51.332982+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T05:07:51.332982+0000","last_undegraded":"2023-10-06T05:07:51.332982+0000","last_fullsized":"2023-10-06T05:07:51.332982+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"210'370","reported_seq":1221,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-10-06T05:07:49.402673+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T05:07:49.402673+0000","last_peered":"2023-10-06T05:07:49.402673+0000","last_clean":"2023-10-06T05:07:49.402673+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T05:07:49.402673+0000","last_undegraded":"2023-10-06T05:07:49.402673+0000","last_fullsized":"2023-10-06T05:07:49.402673+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":175,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-10-06T05:07:39.110639+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T05:07:39.110639+0000","last_peered":"2023-10-06T05:07:39.110639+0000","last_clean":"2023-10-06T05:07:39.110639+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T05:07:39.110639+0000","last_undegraded":"2023-10-06T05:07:39.110639+0000","last_fullsized":"2023-10-06T05:07:39.110639+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":175,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-10-06T05:07:39.110733+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T05:07:39.110733+0000","last_peered":"2023-10-06T05:07:39.110733+0000","last_clean":"2023-10-06T05:07:39.110733+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T05:07:39.110733+0000","last_undegraded":"2023-10-06T05:07:39.110733+0000","last_fullsized":"2023-10-06T05:07:39.110733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":168,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-10-06T05:07:39.171063+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T05:07:39.171063+0000","last_peered":"2023-10-06T05:07:39.171063+0000","last_clean":"2023-10-06T05:07:39.171063+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T05:07:39.171063+0000","last_undegraded":"2023-10-06T05:07:39.171063+0000","last_fullsized":"2023-10-06T05:07:39.171063+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":203,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:07:49.128754+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T05:07:49.128754+0000","last_peered":"2023-10-06T05:07:49.128754+0000","last_clean":"2023-10-06T05:07:49.128754+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T05:07:49.128754+0000","last_undegraded":"2023-10-06T05:07:49.128754+0000","last_fullsized":"2023-10-06T05:07:49.128754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":203,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:07:49.128855+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T05:07:49.128855+0000","last_peered":"2023-10-06T05:07:49.128855+0000","last_clean":"2023-10-06T05:07:49.128855+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T05:07:49.128855+0000","last_undegraded":"2023-10-06T05:07:49.128855+0000","last_fullsized":"2023-10-06T05:07:49.128855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575968,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304528,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067312,"statfs":{"total":96636764160,"available":96324927488,"internally_reserved":0,"allocated":3219456,"data_stored":1983282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94999999999999996,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0529999999999999},"last":0.63300000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79300000000000004,"15min":0.80600000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.996,"5min":1.0640000000000001,"15min":1.073},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.80300000000000005,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.0860000000000001},"last":0.65800000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72199999999999998,"15min":0.74099999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.52500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0509999999999999,"5min":1.107,"15min":1.1160000000000001},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.101},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.753,"15min":0.76600000000000001},"min":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96099999999999997,"5min":1.0149999999999999,"15min":1.0369999999999999},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.355},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64000000000000001,"15min":0.64500000000000002},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.40999999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.208},"last":0.47999999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47699999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.754,"15min":0.749},"min":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.48799999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.51500000000000001,"5min":0.48999999999999999,"15min":0.46700000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.53000000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300188,"kb_used_data":2900,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071652,"statfs":{"total":96636764160,"available":96329371648,"internally_reserved":0,"allocated":2969600,"data_stored":1722295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":1.0009999999999999,"15min":1.2969999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94999999999999996,"5min":1.083,"15min":1.083},"last":0.64200000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0640000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0960000000000001,"5min":1.425,"15min":1.425},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.752,"15min":0.76100000000000001},"min":{"1min":0.48099999999999998,"5min":0.47299999999999998,"15min":0.44700000000000001},"max":{"1min":0.91800000000000004,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96499999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.505}]},{"osd":3,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78200000000000003,"15min":0.78600000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":0.998,"5min":1.016,"15min":1.1699999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.79800000000000004},"min":{"1min":0.61299999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94599999999999995,"5min":1.0409999999999999,"15min":1.1479999999999999},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.40200000000000002,"15min":0.38900000000000001},"max":{"1min":0.81399999999999995,"5min":0.95999999999999996,"15min":1.0269999999999999},"last":0.628},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.64400000000000002},"min":{"1min":0.51800000000000002,"5min":0.46600000000000003,"15min":0.39300000000000002},"max":{"1min":0.89400000000000002,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.51400000000000001,"5min":0.5,"15min":0.41699999999999998},"max":{"1min":0.94199999999999995,"5min":1.02,"15min":1.069},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.41199999999999998},"max":{"1min":0.94199999999999995,"5min":1.0349999999999999,"15min":1.1950000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93999999999999995,"5min":1.208,"15min":1.208},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.48599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97399999999999998,"5min":1.0449999999999999,"15min":1.1579999999999999},"last":0.72599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316852,"kb_used_data":3180,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94054988,"statfs":{"total":96636764160,"available":96312307712,"internally_reserved":0,"allocated":3256320,"data_stored":1998524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.32700000000000001},"max":{"1min":1.093,"5min":1.155,"15min":1.155},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.59299999999999997},"min":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.38900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.90100000000000002},"last":0.46600000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.72699999999999998,"5min":0.76400000000000001,"15min":1.153},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.42199999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1,"5min":1.018,"15min":1.018},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.435,"15min":0.41799999999999998},"max":{"1min":0.95499999999999996,"5min":1.113,"15min":1.131},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92200000000000004,"5min":0.95499999999999996,"15min":0.96999999999999997},"last":0.55300000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79600000000000004,"15min":0.80600000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.02,"5min":1.2210000000000001,"15min":1.2250000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.82199999999999995,"15min":0.84299999999999997},"min":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.147,"5min":1.1930000000000001,"15min":1.236},"last":0.70799999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.82699999999999996,"15min":0.83999999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.53600000000000003},"max":{"1min":1.0660000000000001,"5min":1.1240000000000001,"15min":1.228},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.51000000000000001},"max":{"1min":1.139,"5min":1.1799999999999999,"15min":1.2829999999999999},"last":0.873}]},{"osd":6,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85199999999999998,"15min":0.85699999999999998},"min":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.59199999999999997},"max":{"1min":1.1779999999999999,"5min":1.2490000000000001,"15min":1.2709999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.85999999999999999,"15min":0.86399999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.163,"5min":1.242,"15min":1.2470000000000001},"last":0.92500000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.63,"5min":0.623,"15min":0.623},"max":{"1min":1.198,"5min":1.222,"15min":1.2949999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.872,"15min":0.88},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.224,"5min":1.224,"15min":1.3080000000000001},"last":0.72099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575971,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303980,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067860,"statfs":{"total":96636764160,"available":96325488640,"internally_reserved":0,"allocated":2658304,"data_stored":1407178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.28599999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.379},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.35999999999999999,"15min":0.34399999999999997},"max":{"1min":0.67600000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55500000000000005,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.38700000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.91000000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.46500000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.86699999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.63700000000000001,"15min":0.64700000000000002},"min":{"1min":0.49299999999999999,"5min":0.439,"15min":0.42899999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.67100000000000004,"5min":0.57899999999999996,"15min":0.54300000000000004},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.3049999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.63600000000000001,"5min":0.55700000000000005,"15min":0.55300000000000005},"max":{"1min":1.085,"5min":1.127,"15min":1.3440000000000001},"last":0.96199999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.83499999999999996,"15min":0.84399999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.2729999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82899999999999996,"15min":0.85099999999999998},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.45600000000000002},"max":{"1min":1.1339999999999999,"5min":1.204,"15min":1.218},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81999999999999995,"15min":0.83799999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.53800000000000003},"max":{"1min":1.0529999999999999,"5min":1.1240000000000001,"15min":1.2869999999999999},"last":1.004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86199999999999999,"15min":0.88300000000000001},"min":{"1min":0.67700000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.246,"5min":1.246,"15min":1.4399999999999999},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.85399999999999998,"15min":0.876},"min":{"1min":0.626,"5min":0.624,"15min":0.59799999999999998},"max":{"1min":1.216,"5min":1.2490000000000001,"15min":1.367},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.68300000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.294},"last":0.751}]}]},{"osd":2,"up_from":13,"seq":55834575970,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299884,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071956,"statfs":{"total":96636764160,"available":96329682944,"internally_reserved":0,"allocated":2658304,"data_stored":1407178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47299999999999998,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.32000000000000001,"15min":0.312},"max":{"1min":0.73699999999999999,"5min":0.746,"15min":0.80300000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.91200000000000003},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73899999999999999,"5min":0.78300000000000003,"15min":0.92400000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.51300000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.32400000000000001},"max":{"1min":0.76100000000000001,"5min":0.89800000000000002,"15min":0.90600000000000003},"last":0.70499999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.84799999999999998,"15min":0.85399999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0580000000000001,"5min":1.286,"15min":1.286},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.59199999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.254},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.88400000000000001,"15min":0.88800000000000001},"min":{"1min":0.67100000000000004,"5min":0.59099999999999997,"15min":0.57599999999999996},"max":{"1min":1.165,"5min":1.26,"15min":1.345},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.89000000000000001,"15min":0.89700000000000002},"min":{"1min":0.65500000000000003,"5min":0.628,"15min":0.628},"max":{"1min":1.085,"5min":1.2090000000000001,"15min":1.339},"last":1.0049999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58199999999999996},"max":{"1min":0.997,"5min":1.1279999999999999,"15min":1.2989999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1439999999999999,"5min":1.226,"15min":1.226},"last":0.98799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85999999999999999,"15min":0.86599999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1000000000000001,"5min":1.141,"15min":1.276},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90700000000000003,"15min":0.90900000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.59399999999999997},"max":{"1min":1.1850000000000001,"5min":1.2170000000000001,"15min":1.3680000000000001},"last":0.95999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575970,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299888,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071952,"statfs":{"total":96636764160,"available":96329678848,"internally_reserved":0,"allocated":2662400,"data_stored":1408156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51100000000000001,"15min":0.49399999999999999},"min":{"1min":0.34399999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67400000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.378},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85799999999999998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.76400000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.58299999999999996},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.48899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.74199999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65000000000000002,"15min":0.63500000000000001},"min":{"1min":0.48899999999999999,"5min":0.45100000000000001,"15min":0.38500000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.83599999999999997,"15min":0.82699999999999996},"min":{"1min":0.625,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1599999999999999,"5min":1.232,"15min":1.2849999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.87},"min":{"1min":0.64800000000000002,"5min":0.53900000000000003,"15min":0.52500000000000002},"max":{"1min":1.0860000000000001,"5min":1.232,"15min":1.4019999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80600000000000005,"15min":0.81999999999999995},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.52200000000000002},"max":{"1min":1.01,"5min":1.22,"15min":1.5760000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0980000000000001,"5min":1.1419999999999999,"15min":1.3300000000000001},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.84899999999999998,"15min":0.85099999999999998},"min":{"1min":0.66100000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0900000000000001,"5min":1.212,"15min":1.2150000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.87,"5min":0.88700000000000001,"15min":0.89100000000000001},"min":{"1min":0.64900000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1399999999999999,"5min":1.2929999999999999,"15min":1.321},"last":0.59199999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.87,"15min":0.87},"min":{"1min":0.70299999999999996,"5min":0.59399999999999997,"15min":0.57599999999999996},"max":{"1min":1.1619999999999999,"5min":1.198,"15min":1.2769999999999999},"last":0.624},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.86599999999999999,"15min":0.85399999999999998},"min":{"1min":0.69099999999999995,"5min":0.59299999999999997,"15min":0.55800000000000005},"max":{"1min":1.1200000000000001,"5min":1.1890000000000001,"15min":1.389},"last":0.60599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312168,"kb_used_data":2592,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059672,"statfs":{"total":96636764160,"available":96317104128,"internally_reserved":0,"allocated":2654208,"data_stored":1405898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.747},"min":{"1min":0.53800000000000003,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.89500000000000002,"5min":1.0820000000000001,"15min":1.2},"last":0.503},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.753,"15min":0.755},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97599999999999998,"5min":1.048,"15min":1.048},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72299999999999998,"15min":0.73699999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.96199999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94299999999999995,"5min":0.99099999999999999,"15min":1.258},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.50800000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75900000000000001,"15min":0.78300000000000003},"min":{"1min":0.50600000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.75700000000000001},"min":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.49299999999999999},"max":{"1min":0.97499999999999998,"5min":0.97999999999999998,"15min":1.2330000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.54900000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.95599999999999996,"5min":0.96799999999999997,"15min":1.034},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61399999999999999,"15min":0.63400000000000001},"min":{"1min":0.46000000000000002,"5min":0.43099999999999999,"15min":0.41199999999999998},"max":{"1min":0.81599999999999995,"5min":1.04,"15min":1.323},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.65200000000000002},"min":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.39700000000000002},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.1910000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.67700000000000005},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":1.024,"15min":1.079},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68799999999999994,"15min":0.71199999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.41999999999999998},"max":{"1min":0.84699999999999998,"5min":0.95099999999999996,"15min":1.3540000000000001},"last":0.621}]},{"osd":7,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.70599999999999996},"min":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.82699999999999996,"5min":1.0640000000000001,"15min":1.155},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.47299999999999998},"max":{"1min":0.93000000000000005,"5min":1.079,"15min":1.149},"last":0.66200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308068,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063772,"statfs":{"total":96636764160,"available":96321302528,"internally_reserved":0,"allocated":2650112,"data_stored":1404908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.748},"min":{"1min":0.53200000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.93500000000000005,"5min":1.079,"15min":1.079},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.72999999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88800000000000001,"5min":0.999,"15min":1.02},"last":0.498}]},{"osd":1,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.745,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.53600000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98399999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79600000000000004,"15min":0.79400000000000004},"min":{"1min":0.65300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.99199999999999999,"5min":1.0700000000000001,"15min":1.0780000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80100000000000005,"15min":0.80700000000000005},"min":{"1min":0.57599999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0149999999999999,"5min":1.048,"15min":1.048},"last":0.90700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.48599999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.94399999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.63500000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0169999999999999,"5min":1.095,"15min":1.095},"last":0.97099999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.629},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.38200000000000001},"max":{"1min":0.84399999999999997,"5min":0.97699999999999998,"15min":1.0049999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.5,"5min":0.38,"15min":0.38},"max":{"1min":0.89000000000000001,"5min":1.01,"15min":1.042},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.48499999999999999,"5min":0.45900000000000002,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.0369999999999999},"last":1.006},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.42799999999999999},"max":{"1min":0.90700000000000003,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.55000000000000004,"5min":0.48499999999999999,"15min":0.44500000000000001},"max":{"1min":0.92600000000000005,"5min":1.0269999999999999,"15min":1.0669999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.54000000000000004,"5min":0.51300000000000001,"15min":0.47199999999999998},"max":{"1min":0.93000000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.875}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:07:56.674 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T05:07:56.674 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:07:56.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.859+0000 7f609999d700 1 Processor -- start 2023-10-06T05:07:56.861 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.859+0000 7f609999d700 1 -- start start 2023-10-06T05:07:56.862 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.860+0000 7f609999d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f6094068aa0 0x7f60940b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.860+0000 7f609999d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609415a980 con 0x7f60940b0a00 2023-10-06T05:07:56.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.860+0000 7f609999d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609415a050 con 0x7f6094157270 2023-10-06T05:07:56.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.860+0000 7f609999d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f609415a330 con 0x7f6094068aa0 2023-10-06T05:07:56.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.861+0000 7f609919c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f60940b0a00 0x7f609415d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:40142/0 (socket says 172.21.15.55:40142) 2023-10-06T05:07:56.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.861+0000 7f609919c700 1 -- 172.21.15.55:0/3238177101 learned_addr learned my addr 172.21.15.55:0/3238177101 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:56.863 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.861+0000 7f6093fff700 1 --2- 172.21.15.55:0/3238177101 >> v2:172.21.15.163:3300/0 conn(0x7f6094068aa0 0x7f60940b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.861+0000 7f60937fe700 1 -- 172.21.15.55:0/3238177101 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561953452 0 0) 0x7f609415a980 con 0x7f60940b0a00 2023-10-06T05:07:56.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.861+0000 7f60937fe700 1 -- 172.21.15.55:0/3238177101 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6084003570 con 0x7f60940b0a00 2023-10-06T05:07:56.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.861+0000 7f60937fe700 1 -- 172.21.15.55:0/3238177101 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697260147 0 0) 0x7f609415a050 con 0x7f6094157270 2023-10-06T05:07:56.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.861+0000 7f60937fe700 1 -- 172.21.15.55:0/3238177101 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f609415a980 con 0x7f6094157270 2023-10-06T05:07:56.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.861+0000 7f60937fe700 1 -- 172.21.15.55:0/3238177101 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2339756255 0 0) 0x7f6084003570 con 0x7f60940b0a00 2023-10-06T05:07:56.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.861+0000 7f6093fff700 1 -- 172.21.15.55:0/3238177101 >> v1:172.21.15.163:6789/0 conn(0x7f6094157270 legacy=0x7f6094068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.865 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.861+0000 7f6093fff700 1 -- 172.21.15.55:0/3238177101 >> v1:172.21.15.55:6789/0 conn(0x7f60940b0a00 legacy=0x7f609415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.862+0000 7f6093fff700 1 -- 172.21.15.55:0/3238177101 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609415a610 con 0x7f6094068aa0 2023-10-06T05:07:56.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.862+0000 7f6093fff700 1 --2- 172.21.15.55:0/3238177101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6094068aa0 0x7f60940b04c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f60800098e0 tx=0x7f60800091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a85abfac9a53673e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:56.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.862+0000 7f60937fe700 1 -- 172.21.15.55:0/3238177101 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c0017f0 con 0x7f6094068aa0 2023-10-06T05:07:56.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.862+0000 7f60937fe700 1 -- 172.21.15.55:0/3238177101 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6080011420 con 0x7f6094068aa0 2023-10-06T05:07:56.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.862+0000 7f60937fe700 1 -- 172.21.15.55:0/3238177101 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60800117c0 con 0x7f6094068aa0 2023-10-06T05:07:56.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.863+0000 7f609999d700 1 -- 172.21.15.55:0/3238177101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6094068aa0 msgr2=0x7f60940b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.863+0000 7f609999d700 1 --2- 172.21.15.55:0/3238177101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6094068aa0 0x7f60940b04c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f60800098e0 tx=0x7f60800091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.863+0000 7f609999d700 1 -- 172.21.15.55:0/3238177101 shutdown_connections 2023-10-06T05:07:56.866 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.863+0000 7f609999d700 1 --2- 172.21.15.55:0/3238177101 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f6094068aa0 0x7f60940b04c0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.863+0000 7f609999d700 1 -- 172.21.15.55:0/3238177101 >> 172.21.15.55:0/3238177101 conn(0x7f60940cd310 msgr2=0x7f60940cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:56.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.863+0000 7f609999d700 1 -- 172.21.15.55:0/3238177101 shutdown_connections 2023-10-06T05:07:56.867 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.863+0000 7f609999d700 1 -- 172.21.15.55:0/3238177101 wait complete. 2023-10-06T05:07:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.864+0000 7f609999d700 1 Processor -- start 2023-10-06T05:07:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.864+0000 7f609999d700 1 -- start start 2023-10-06T05:07:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.865+0000 7f609999d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6094068aa0 0x7f609414d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.865+0000 7f609999d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60940b0a00 0x7f60940af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.870 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.865+0000 7f609999d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6094157270 0x7f6094169a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.865+0000 7f609999d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609414b3a0 con 0x7f6094068aa0 2023-10-06T05:07:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.865+0000 7f609999d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609414db20 con 0x7f60940b0a00 2023-10-06T05:07:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.865+0000 7f609999d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60940afb20 con 0x7f6094157270 2023-10-06T05:07:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.865+0000 7f609899b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6094068aa0 0x7f609414d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.866+0000 7f609899b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6094068aa0 0x7f609414d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54616/0 (socket says 172.21.15.55:54616) 2023-10-06T05:07:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.866+0000 7f609899b700 1 -- 172.21.15.55:0/3084745155 learned_addr learned my addr 172.21.15.55:0/3084745155 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.866+0000 7f6093fff700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60940b0a00 0x7f60940af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.866+0000 7f609919c700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6094157270 0x7f6094169a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.866+0000 7f609899b700 1 -- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6094157270 msgr2=0x7f6094169a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.871 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.866+0000 7f609899b700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6094157270 0x7f6094169a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.866+0000 7f609899b700 1 -- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60940b0a00 msgr2=0x7f60940af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.866+0000 7f609899b700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60940b0a00 0x7f60940af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.866+0000 7f609899b700 1 -- 172.21.15.55:0/3084745155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6080009420 con 0x7f6094068aa0 2023-10-06T05:07:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.866+0000 7f609919c700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6094157270 0x7f6094169a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:07:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.866+0000 7f609899b700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6094068aa0 0x7f609414d5e0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f608800c580 tx=0x7f6088007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.866+0000 7f6090ff9700 1 -- 172.21.15.55:0/3084745155 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088010510 con 0x7f6094068aa0 2023-10-06T05:07:56.872 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.866+0000 7f6090ff9700 1 -- 172.21.15.55:0/3084745155 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6088010670 con 0x7f6094068aa0 2023-10-06T05:07:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.866+0000 7f6090ff9700 1 -- 172.21.15.55:0/3084745155 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088010a30 con 0x7f6094068aa0 2023-10-06T05:07:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.867+0000 7f609999d700 1 -- 172.21.15.55:0/3084745155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609414e030 con 0x7f6094068aa0 2023-10-06T05:07:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.867+0000 7f609999d700 1 -- 172.21.15.55:0/3084745155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609414e5f0 con 0x7f6094068aa0 2023-10-06T05:07:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.868+0000 7f609999d700 1 -- 172.21.15.55:0/3084745155 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609414ddb0 con 0x7f6094068aa0 2023-10-06T05:07:56.873 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.868+0000 7f6090ff9700 1 -- 172.21.15.55:0/3084745155 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f608801b400 con 0x7f6094068aa0 2023-10-06T05:07:56.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.869+0000 7f6090ff9700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6078066f30 0x7f60780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:56.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.869+0000 7f6090ff9700 1 -- 172.21.15.55:0/3084745155 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f6088084440 con 0x7f6094068aa0 2023-10-06T05:07:56.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.870+0000 7f6093fff700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6078066f30 0x7f60780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:56.874 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.871+0000 7f6093fff700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6078066f30 0x7f60780693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6094068ea0 tx=0x7f6080016000 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:56.877 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:56.874+0000 7f6090ff9700 1 -- 172.21.15.55:0/3084745155 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60880513c0 con 0x7f6094068aa0 2023-10-06T05:07:57.073 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.071+0000 7f609999d700 1 -- 172.21.15.55:0/3084745155 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60940b0e80 con 0x7f6078066f30 2023-10-06T05:07:57.076 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.075+0000 7f6090ff9700 1 -- 172.21.15.55:0/3084745155 <== mgr.10033 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72684 (secure 0 0 0) 0x7f60940b0e80 con 0x7f6078066f30 2023-10-06T05:07:57.077 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:57.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.077+0000 7f609999d700 1 -- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6078066f30 msgr2=0x7f60780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:57.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.077+0000 7f609999d700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6078066f30 0x7f60780693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6094068ea0 tx=0x7f6080016000 comp rx=0 tx=0).stop 2023-10-06T05:07:57.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.077+0000 7f609999d700 1 -- 172.21.15.55:0/3084745155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6094068aa0 msgr2=0x7f609414d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:57.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.077+0000 7f609999d700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6094068aa0 0x7f609414d5e0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f608800c580 tx=0x7f6088007e10 comp rx=0 tx=0).stop 2023-10-06T05:07:57.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.077+0000 7f609999d700 1 -- 172.21.15.55:0/3084745155 shutdown_connections 2023-10-06T05:07:57.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.077+0000 7f609999d700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f6078066f30 0x7f60780693e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.077+0000 7f609999d700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f6094157270 0x7f6094169a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.077+0000 7f609999d700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f6094068aa0 0x7f609414d5e0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.077+0000 7f609999d700 1 --2- 172.21.15.55:0/3084745155 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f60940b0a00 0x7f60940af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.077+0000 7f609999d700 1 -- 172.21.15.55:0/3084745155 >> 172.21.15.55:0/3084745155 conn(0x7f60940cd310 msgr2=0x7f609415bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:57.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.077+0000 7f609999d700 1 -- 172.21.15.55:0/3084745155 shutdown_connections 2023-10-06T05:07:57.080 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.077+0000 7f609999d700 1 -- 172.21.15.55:0/3084745155 wait complete. 2023-10-06T05:07:57.080 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T05:07:57.096 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2023-10-06T05:07:55.333895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445556,"kb_used_data":22196,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":752529164,"statfs":{"total":773094113280,"available":770589863936,"internally_reserved":0,"allocated":22728704,"data_stored":12737419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668755,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-281,"num_write_kb":-2052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.995111"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":191,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:07:49.128504+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T05:07:49.128504+0000","last_peered":"2023-10-06T05:07:49.128504+0000","last_clean":"2023-10-06T05:07:49.128504+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T05:07:49.128504+0000","last_undegraded":"2023-10-06T05:07:49.128504+0000","last_fullsized":"2023-10-06T05:07:49.128504+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":180,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-06T05:07:51.875137+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T05:07:51.875137+0000","last_peered":"2023-10-06T05:07:51.875137+0000","last_clean":"2023-10-06T05:07:51.875137+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T05:07:51.875137+0000","last_undegraded":"2023-10-06T05:07:51.875137+0000","last_fullsized":"2023-10-06T05:07:51.875137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":175,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-06T05:07:51.332982+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T05:07:51.332982+0000","last_peered":"2023-10-06T05:07:51.332982+0000","last_clean":"2023-10-06T05:07:51.332982+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T05:07:51.332982+0000","last_undegraded":"2023-10-06T05:07:51.332982+0000","last_fullsized":"2023-10-06T05:07:51.332982+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"210'370","reported_seq":1221,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-10-06T05:07:49.402673+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T05:07:49.402673+0000","last_peered":"2023-10-06T05:07:49.402673+0000","last_clean":"2023-10-06T05:07:49.402673+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T05:07:49.402673+0000","last_undegraded":"2023-10-06T05:07:49.402673+0000","last_fullsized":"2023-10-06T05:07:49.402673+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":175,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-10-06T05:07:39.110639+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T05:07:39.110639+0000","last_peered":"2023-10-06T05:07:39.110639+0000","last_clean":"2023-10-06T05:07:39.110639+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T05:07:39.110639+0000","last_undegraded":"2023-10-06T05:07:39.110639+0000","last_fullsized":"2023-10-06T05:07:39.110639+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":175,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-10-06T05:07:39.110733+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T05:07:39.110733+0000","last_peered":"2023-10-06T05:07:39.110733+0000","last_clean":"2023-10-06T05:07:39.110733+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T05:07:39.110733+0000","last_undegraded":"2023-10-06T05:07:39.110733+0000","last_fullsized":"2023-10-06T05:07:39.110733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":168,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-10-06T05:07:39.171063+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T05:07:39.171063+0000","last_peered":"2023-10-06T05:07:39.171063+0000","last_clean":"2023-10-06T05:07:39.171063+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T05:07:39.171063+0000","last_undegraded":"2023-10-06T05:07:39.171063+0000","last_fullsized":"2023-10-06T05:07:39.171063+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":203,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:07:49.128754+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T05:07:49.128754+0000","last_peered":"2023-10-06T05:07:49.128754+0000","last_clean":"2023-10-06T05:07:49.128754+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T05:07:49.128754+0000","last_undegraded":"2023-10-06T05:07:49.128754+0000","last_fullsized":"2023-10-06T05:07:49.128754+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":203,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-06T05:07:49.128855+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T05:07:49.128855+0000","last_peered":"2023-10-06T05:07:49.128855+0000","last_clean":"2023-10-06T05:07:49.128855+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T05:07:49.128855+0000","last_undegraded":"2023-10-06T05:07:49.128855+0000","last_fullsized":"2023-10-06T05:07:49.128855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575968,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304528,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067312,"statfs":{"total":96636764160,"available":96324927488,"internally_reserved":0,"allocated":3219456,"data_stored":1983282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94999999999999996,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0529999999999999},"last":0.63300000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79300000000000004,"15min":0.80600000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.996,"5min":1.0640000000000001,"15min":1.073},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.80300000000000005,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.0860000000000001},"last":0.65800000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72199999999999998,"15min":0.74099999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.52500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0509999999999999,"5min":1.107,"15min":1.1160000000000001},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.101},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.753,"15min":0.76600000000000001},"min":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96099999999999997,"5min":1.0149999999999999,"15min":1.0369999999999999},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.355},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64000000000000001,"15min":0.64500000000000002},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.40999999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.208},"last":0.47999999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47699999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.754,"15min":0.749},"min":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.48799999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.51500000000000001,"5min":0.48999999999999999,"15min":0.46700000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.53000000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300188,"kb_used_data":2900,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071652,"statfs":{"total":96636764160,"available":96329371648,"internally_reserved":0,"allocated":2969600,"data_stored":1722295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":1.0009999999999999,"15min":1.2969999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94999999999999996,"5min":1.083,"15min":1.083},"last":0.64200000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0640000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0960000000000001,"5min":1.425,"15min":1.425},"last":0.68500000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.752,"15min":0.76100000000000001},"min":{"1min":0.48099999999999998,"5min":0.47299999999999998,"15min":0.44700000000000001},"max":{"1min":0.91800000000000004,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96499999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.505}]},{"osd":3,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78200000000000003,"15min":0.78600000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":0.998,"5min":1.016,"15min":1.1699999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.79800000000000004},"min":{"1min":0.61299999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94599999999999995,"5min":1.0409999999999999,"15min":1.1479999999999999},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.40200000000000002,"15min":0.38900000000000001},"max":{"1min":0.81399999999999995,"5min":0.95999999999999996,"15min":1.0269999999999999},"last":0.628},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.64400000000000002},"min":{"1min":0.51800000000000002,"5min":0.46600000000000003,"15min":0.39300000000000002},"max":{"1min":0.89400000000000002,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.51400000000000001,"5min":0.5,"15min":0.41699999999999998},"max":{"1min":0.94199999999999995,"5min":1.02,"15min":1.069},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.41199999999999998},"max":{"1min":0.94199999999999995,"5min":1.0349999999999999,"15min":1.1950000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93999999999999995,"5min":1.208,"15min":1.208},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.48599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97399999999999998,"5min":1.0449999999999999,"15min":1.1579999999999999},"last":0.72599999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575969,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316852,"kb_used_data":3180,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94054988,"statfs":{"total":96636764160,"available":96312307712,"internally_reserved":0,"allocated":3256320,"data_stored":1998524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.32700000000000001},"max":{"1min":1.093,"5min":1.155,"15min":1.155},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.59299999999999997},"min":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.38900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.90100000000000002},"last":0.46600000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.72699999999999998,"5min":0.76400000000000001,"15min":1.153},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.42199999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1,"5min":1.018,"15min":1.018},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.435,"15min":0.41799999999999998},"max":{"1min":0.95499999999999996,"5min":1.113,"15min":1.131},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92200000000000004,"5min":0.95499999999999996,"15min":0.96999999999999997},"last":0.55300000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79600000000000004,"15min":0.80600000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.02,"5min":1.2210000000000001,"15min":1.2250000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.82199999999999995,"15min":0.84299999999999997},"min":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.147,"5min":1.1930000000000001,"15min":1.236},"last":0.70799999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.82699999999999996,"15min":0.83999999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.53600000000000003},"max":{"1min":1.0660000000000001,"5min":1.1240000000000001,"15min":1.228},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.51000000000000001},"max":{"1min":1.139,"5min":1.1799999999999999,"15min":1.2829999999999999},"last":0.873}]},{"osd":6,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85199999999999998,"15min":0.85699999999999998},"min":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.59199999999999997},"max":{"1min":1.1779999999999999,"5min":1.2490000000000001,"15min":1.2709999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.85999999999999999,"15min":0.86399999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.163,"5min":1.242,"15min":1.2470000000000001},"last":0.92500000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.63,"5min":0.623,"15min":0.623},"max":{"1min":1.198,"5min":1.222,"15min":1.2949999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.872,"15min":0.88},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.224,"5min":1.224,"15min":1.3080000000000001},"last":0.72099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575971,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303980,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067860,"statfs":{"total":96636764160,"available":96325488640,"internally_reserved":0,"allocated":2658304,"data_stored":1407178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.28599999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.379},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.35999999999999999,"15min":0.34399999999999997},"max":{"1min":0.67600000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55500000000000005,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.38700000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.91000000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.46500000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.86699999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.63700000000000001,"15min":0.64700000000000002},"min":{"1min":0.49299999999999999,"5min":0.439,"15min":0.42899999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.67100000000000004,"5min":0.57899999999999996,"15min":0.54300000000000004},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.3049999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.63600000000000001,"5min":0.55700000000000005,"15min":0.55300000000000005},"max":{"1min":1.085,"5min":1.127,"15min":1.3440000000000001},"last":0.96199999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.83499999999999996,"15min":0.84399999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.2729999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82899999999999996,"15min":0.85099999999999998},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.45600000000000002},"max":{"1min":1.1339999999999999,"5min":1.204,"15min":1.218},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81999999999999995,"15min":0.83799999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.53800000000000003},"max":{"1min":1.0529999999999999,"5min":1.1240000000000001,"15min":1.2869999999999999},"last":1.004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86199999999999999,"15min":0.88300000000000001},"min":{"1min":0.67700000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.246,"5min":1.246,"15min":1.4399999999999999},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.85399999999999998,"15min":0.876},"min":{"1min":0.626,"5min":0.624,"15min":0.59799999999999998},"max":{"1min":1.216,"5min":1.2490000000000001,"15min":1.367},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.68300000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.294},"last":0.751}]}]},{"osd":2,"up_from":13,"seq":55834575970,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299884,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071956,"statfs":{"total":96636764160,"available":96329682944,"internally_reserved":0,"allocated":2658304,"data_stored":1407178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47299999999999998,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.32000000000000001,"15min":0.312},"max":{"1min":0.73699999999999999,"5min":0.746,"15min":0.80300000000000005},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.91200000000000003},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73899999999999999,"5min":0.78300000000000003,"15min":0.92400000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.51300000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.32400000000000001},"max":{"1min":0.76100000000000001,"5min":0.89800000000000002,"15min":0.90600000000000003},"last":0.70499999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.84799999999999998,"15min":0.85399999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0580000000000001,"5min":1.286,"15min":1.286},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.59199999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.254},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.88400000000000001,"15min":0.88800000000000001},"min":{"1min":0.67100000000000004,"5min":0.59099999999999997,"15min":0.57599999999999996},"max":{"1min":1.165,"5min":1.26,"15min":1.345},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.88,"5min":0.89000000000000001,"15min":0.89700000000000002},"min":{"1min":0.65500000000000003,"5min":0.628,"15min":0.628},"max":{"1min":1.085,"5min":1.2090000000000001,"15min":1.339},"last":1.0049999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58199999999999996},"max":{"1min":0.997,"5min":1.1279999999999999,"15min":1.2989999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1439999999999999,"5min":1.226,"15min":1.226},"last":0.98799999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85999999999999999,"15min":0.86599999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1000000000000001,"5min":1.141,"15min":1.276},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90700000000000003,"15min":0.90900000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.59399999999999997},"max":{"1min":1.1850000000000001,"5min":1.2170000000000001,"15min":1.3680000000000001},"last":0.95999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575970,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299888,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071952,"statfs":{"total":96636764160,"available":96329678848,"internally_reserved":0,"allocated":2662400,"data_stored":1408156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.51100000000000001,"15min":0.49399999999999999},"min":{"1min":0.34399999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67400000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.378},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.85799999999999998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52700000000000002,"15min":0.51600000000000001},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.76400000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.59599999999999997,"15min":0.58299999999999996},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81999999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.48899999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.74199999999999999,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65000000000000002,"15min":0.63500000000000001},"min":{"1min":0.48899999999999999,"5min":0.45100000000000001,"15min":0.38500000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.83599999999999997,"15min":0.82699999999999996},"min":{"1min":0.625,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1599999999999999,"5min":1.232,"15min":1.2849999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.87},"min":{"1min":0.64800000000000002,"5min":0.53900000000000003,"15min":0.52500000000000002},"max":{"1min":1.0860000000000001,"5min":1.232,"15min":1.4019999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.80600000000000005,"15min":0.81999999999999995},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.52200000000000002},"max":{"1min":1.01,"5min":1.22,"15min":1.5760000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0980000000000001,"5min":1.1419999999999999,"15min":1.3300000000000001},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.84899999999999998,"15min":0.85099999999999998},"min":{"1min":0.66100000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0900000000000001,"5min":1.212,"15min":1.2150000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.87,"5min":0.88700000000000001,"15min":0.89100000000000001},"min":{"1min":0.64900000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1399999999999999,"5min":1.2929999999999999,"15min":1.321},"last":0.59199999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:06:58 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.87,"15min":0.87},"min":{"1min":0.70299999999999996,"5min":0.59399999999999997,"15min":0.57599999999999996},"max":{"1min":1.1619999999999999,"5min":1.198,"15min":1.2769999999999999},"last":0.624},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.86599999999999999,"15min":0.85399999999999998},"min":{"1min":0.69099999999999995,"5min":0.59299999999999997,"15min":0.55800000000000005},"max":{"1min":1.1200000000000001,"5min":1.1890000000000001,"15min":1.389},"last":0.60599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312168,"kb_used_data":2592,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059672,"statfs":{"total":96636764160,"available":96317104128,"internally_reserved":0,"allocated":2654208,"data_stored":1405898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.747},"min":{"1min":0.53800000000000003,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.89500000000000002,"5min":1.0820000000000001,"15min":1.2},"last":0.503},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.753,"15min":0.755},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97599999999999998,"5min":1.048,"15min":1.048},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72299999999999998,"15min":0.73699999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.96199999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94299999999999995,"5min":0.99099999999999999,"15min":1.258},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.50800000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75900000000000001,"15min":0.78300000000000003},"min":{"1min":0.50600000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.75700000000000001},"min":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.49299999999999999},"max":{"1min":0.97499999999999998,"5min":0.97999999999999998,"15min":1.2330000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.54900000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.95599999999999996,"5min":0.96799999999999997,"15min":1.034},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61399999999999999,"15min":0.63400000000000001},"min":{"1min":0.46000000000000002,"5min":0.43099999999999999,"15min":0.41199999999999998},"max":{"1min":0.81599999999999995,"5min":1.04,"15min":1.323},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.65200000000000002},"min":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.39700000000000002},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.1910000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.67700000000000005},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":1.024,"15min":1.079},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68799999999999994,"15min":0.71199999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.41999999999999998},"max":{"1min":0.84699999999999998,"5min":0.95099999999999996,"15min":1.3540000000000001},"last":0.621}]},{"osd":7,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.70599999999999996},"min":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.82699999999999996,"5min":1.0640000000000001,"15min":1.155},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.47299999999999998},"max":{"1min":0.93000000000000005,"5min":1.079,"15min":1.149},"last":0.66200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575969,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308068,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063772,"statfs":{"total":96636764160,"available":96321302528,"internally_reserved":0,"allocated":2650112,"data_stored":1404908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.748},"min":{"1min":0.53200000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.93500000000000005,"5min":1.079,"15min":1.079},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.72999999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88800000000000001,"5min":0.999,"15min":1.02},"last":0.498}]},{"osd":1,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.745,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.53600000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98399999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79600000000000004,"15min":0.79400000000000004},"min":{"1min":0.65300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.99199999999999999,"5min":1.0700000000000001,"15min":1.0780000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80100000000000005,"15min":0.80700000000000005},"min":{"1min":0.57599999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0149999999999999,"5min":1.048,"15min":1.048},"last":0.90700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.48599999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.94399999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.63500000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0169999999999999,"5min":1.095,"15min":1.095},"last":0.97099999999999997}]},{"osd":4,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.629},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.38200000000000001},"max":{"1min":0.84399999999999997,"5min":0.97699999999999998,"15min":1.0049999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.5,"5min":0.38,"15min":0.38},"max":{"1min":0.89000000000000001,"5min":1.01,"15min":1.042},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.48499999999999999,"5min":0.45900000000000002,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.0369999999999999},"last":1.006},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.42799999999999999},"max":{"1min":0.90700000000000003,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.55000000000000004,"5min":0.48499999999999999,"15min":0.44500000000000001},"max":{"1min":0.92600000000000005,"5min":1.0269999999999999,"15min":1.0669999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.54000000000000004,"5min":0.51300000000000001,"15min":0.47199999999999998},"max":{"1min":0.93000000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.875}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:07:57.099 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T05:07:57.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.284+0000 7fd6a139d700 1 Processor -- start 2023-10-06T05:07:57.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.284+0000 7fd6a139d700 1 -- start start 2023-10-06T05:07:57.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.286+0000 7fd6a139d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fd69c157270 0x7fd69c068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:57.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.286+0000 7fd6a139d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd69c0688d0 0x7fd69c0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:57.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.286+0000 7fd6a139d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd69c0b0a00 0x7fd69c06a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:57.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.286+0000 7fd6a139d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69c068d50 con 0x7fd69c0688d0 2023-10-06T05:07:57.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.286+0000 7fd6a139d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69c15a980 con 0x7fd69c157270 2023-10-06T05:07:57.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.286+0000 7fd6a139d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c15a050 con 0x7fd69c0b0a00 2023-10-06T05:07:57.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.286+0000 7fd69b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd69c0688d0 0x7fd69c0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:57.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.286+0000 7fd69b7fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fd69c0688d0 0x7fd69c0b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54626/0 (socket says 172.21.15.55:54626) 2023-10-06T05:07:57.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.286+0000 7fd69b7fe700 1 -- 172.21.15.55:0/1097795541 learned_addr learned my addr 172.21.15.55:0/1097795541 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:57.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.286+0000 7fd6a0b9c700 1 --2- 172.21.15.55:0/1097795541 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd69c0b0a00 0x7fd69c06a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:57.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.286+0000 7fd69bfff700 1 --2- 172.21.15.55:0/1097795541 >> v2:172.21.15.163:3300/0 conn(0x7fd69c157270 0x7fd69c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:57.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.287+0000 7fd69b7fe700 1 -- 172.21.15.55:0/1097795541 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd69c0b0a00 msgr2=0x7fd69c06a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:57.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.287+0000 7fd69b7fe700 1 --2- 172.21.15.55:0/1097795541 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd69c0b0a00 0x7fd69c06a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.287+0000 7fd69b7fe700 1 -- 172.21.15.55:0/1097795541 >> v2:172.21.15.163:3300/0 conn(0x7fd69c157270 msgr2=0x7fd69c068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:57.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.287+0000 7fd69b7fe700 1 --2- 172.21.15.55:0/1097795541 >> v2:172.21.15.163:3300/0 conn(0x7fd69c157270 0x7fd69c068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.287+0000 7fd69b7fe700 1 -- 172.21.15.55:0/1097795541 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c15a330 con 0x7fd69c0688d0 2023-10-06T05:07:57.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.287+0000 7fd69b7fe700 1 --2- 172.21.15.55:0/1097795541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd69c0688d0 0x7fd69c0b04c0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0b3360 tx=0x7fd690009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab13683c6205fb9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:57.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.287+0000 7fd69affd700 1 -- 172.21.15.55:0/1097795541 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69000b640 con 0x7fd69c0688d0 2023-10-06T05:07:57.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.287+0000 7fd69affd700 1 -- 172.21.15.55:0/1097795541 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69000b7a0 con 0x7fd69c0688d0 2023-10-06T05:07:57.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.287+0000 7fd69affd700 1 -- 172.21.15.55:0/1097795541 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69000bb60 con 0x7fd69c0688d0 2023-10-06T05:07:57.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.288+0000 7fd6a139d700 1 -- 172.21.15.55:0/1097795541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd69c0688d0 msgr2=0x7fd69c0b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:57.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.288+0000 7fd6a139d700 1 --2- 172.21.15.55:0/1097795541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd69c0688d0 0x7fd69c0b04c0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fd69c0b3360 tx=0x7fd690009250 comp rx=0 tx=0).stop 2023-10-06T05:07:57.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.288+0000 7fd6a139d700 1 -- 172.21.15.55:0/1097795541 shutdown_connections 2023-10-06T05:07:57.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.288+0000 7fd6a139d700 1 --2- 172.21.15.55:0/1097795541 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd69c0b0a00 0x7fd69c06a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.288+0000 7fd6a139d700 1 --2- 172.21.15.55:0/1097795541 >> v2:172.21.15.163:3300/0 conn(0x7fd69c157270 0x7fd69c068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.288+0000 7fd6a139d700 1 --2- 172.21.15.55:0/1097795541 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd69c0688d0 0x7fd69c0b04c0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.288+0000 7fd6a139d700 1 -- 172.21.15.55:0/1097795541 >> 172.21.15.55:0/1097795541 conn(0x7fd69c0cd310 msgr2=0x7fd69c0cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:57.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.288+0000 7fd6a139d700 1 -- 172.21.15.55:0/1097795541 shutdown_connections 2023-10-06T05:07:57.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.288+0000 7fd6a139d700 1 -- 172.21.15.55:0/1097795541 wait complete. 2023-10-06T05:07:57.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.289+0000 7fd6a139d700 1 Processor -- start 2023-10-06T05:07:57.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.289+0000 7fd6a139d700 1 -- start start 2023-10-06T05:07:57.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.290+0000 7fd6a139d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd69c0688d0 0x7fd69c0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:57.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.290+0000 7fd6a139d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd69c0b0a00 0x7fd69c1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:57.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.290+0000 7fd6a139d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd69c157270 0x7fd69c151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:57.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.290+0000 7fd6a139d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0aeb20 con 0x7fd69c0b0a00 2023-10-06T05:07:57.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.290+0000 7fd6a139d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0ccbb0 con 0x7fd69c157270 2023-10-06T05:07:57.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.290+0000 7fd6a139d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c0aedc0 con 0x7fd69c0688d0 2023-10-06T05:07:57.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.291+0000 7fd69b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd69c0b0a00 0x7fd69c1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:57.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.291+0000 7fd69b7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd69c0b0a00 0x7fd69c1552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54638/0 (socket says 172.21.15.55:54638) 2023-10-06T05:07:57.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.291+0000 7fd69b7fe700 1 -- 172.21.15.55:0/1896445608 learned_addr learned my addr 172.21.15.55:0/1896445608 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:57.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.291+0000 7fd6a0b9c700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd69c157270 0x7fd69c151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:57.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.291+0000 7fd69b7fe700 1 -- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd69c0688d0 msgr2=0x7fd69c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:57.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.291+0000 7fd69bfff700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd69c0688d0 0x7fd69c0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:57.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.291+0000 7fd69b7fe700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd69c0688d0 0x7fd69c0cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.291+0000 7fd69b7fe700 1 -- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd69c157270 msgr2=0x7fd69c151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:57.296 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.291+0000 7fd69b7fe700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd69c157270 0x7fd69c151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.291+0000 7fd69b7fe700 1 -- 172.21.15.55:0/1896445608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6900094f0 con 0x7fd69c0b0a00 2023-10-06T05:07:57.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.291+0000 7fd6a0b9c700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd69c157270 0x7fd69c151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:07:57.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.291+0000 7fd69bfff700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd69c0688d0 0x7fd69c0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:07:57.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.291+0000 7fd69b7fe700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd69c0b0a00 0x7fd69c1552b0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fd69c068ed0 tx=0x7fd690017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:57.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.292+0000 7fd683fff700 1 -- 172.21.15.55:0/1896445608 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690000d10 con 0x7fd69c0b0a00 2023-10-06T05:07:57.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.292+0000 7fd683fff700 1 -- 172.21.15.55:0/1896445608 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69000bb20 con 0x7fd69c0b0a00 2023-10-06T05:07:57.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.292+0000 7fd6a139d700 1 -- 172.21.15.55:0/1896445608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69c0af0a0 con 0x7fd69c0b0a00 2023-10-06T05:07:57.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.292+0000 7fd683fff700 1 -- 172.21.15.55:0/1896445608 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd690011670 con 0x7fd69c0b0a00 2023-10-06T05:07:57.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.292+0000 7fd6a139d700 1 -- 172.21.15.55:0/1896445608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69c16d540 con 0x7fd69c0b0a00 2023-10-06T05:07:57.297 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.293+0000 7fd6a139d700 1 -- 172.21.15.55:0/1896445608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69c04c3c0 con 0x7fd69c0b0a00 2023-10-06T05:07:57.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.294+0000 7fd683fff700 1 -- 172.21.15.55:0/1896445608 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd6900119e0 con 0x7fd69c0b0a00 2023-10-06T05:07:57.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.295+0000 7fd683fff700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fd688066f30 0x7fd6880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:57.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.295+0000 7fd683fff700 1 -- 172.21.15.55:0/1896445608 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7fd690085ff0 con 0x7fd69c0b0a00 2023-10-06T05:07:57.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.296+0000 7fd69bfff700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fd688066f30 0x7fd6880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:57.299 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.296+0000 7fd69bfff700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fd688066f30 0x7fd6880693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd68c006f00 tx=0x7fd68c008040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:57.302 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.300+0000 7fd683fff700 1 -- 172.21.15.55:0/1896445608 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd690052fd0 con 0x7fd69c0b0a00 2023-10-06T05:07:57.508 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.506+0000 7fd6a139d700 1 -- 172.21.15.55:0/1896445608 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd69c15a610 con 0x7fd69c0b0a00 2023-10-06T05:07:57.509 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.507+0000 7fd683fff700 1 -- 172.21.15.55:0/1896445608 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v212) v1 ==== 74+0+15337 (secure 0 0 0) 0x7fd6900565f0 con 0x7fd69c0b0a00 2023-10-06T05:07:57.509 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:07:57.509 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":212,"fsid":"8132c8ba-ee1c-41a2-b781-913925034ab6","created":"2023-10-06T03:32:55.479127+0000","modified":"2023-10-06T05:07:52.329367+0000","last_up_change":"2023-10-06T03:33:07.443920+0000","last_in_change":"2023-10-06T03:32:58.864136+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-10-06T03:33:08.008847+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-10-06T03:33:08.549094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a60fada0-fedf-4541-8e31-aaa108953efe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":115443},{"type":"v1","addr":"172.21.15.55:6801","nonce":115443}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":115443},{"type":"v1","addr":"172.21.15.55:6803","nonce":115443}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":115443},{"type":"v1","addr":"172.21.15.55:6807","nonce":115443}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":115443},{"type":"v1","addr":"172.21.15.55:6805","nonce":115443}]},"public_addr":"172.21.15.55:6801/115443","cluster_addr":"172.21.15.55:6803/115443","heartbeat_back_addr":"172.21.15.55:6807/115443","heartbeat_front_addr":"172.21.15.55:6805/115443","state":["exists","up"]},{"osd":1,"uuid":"880c7d1c-2d15-48fb-9065-15bc117b6ebd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":115446},{"type":"v1","addr":"172.21.15.55:6809","nonce":115446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6810","nonce":115446},{"type":"v1","addr":"172.21.15.55:6811","nonce":115446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6814","nonce":115446},{"type":"v1","addr":"172.21.15.55:6815","nonce":115446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6812","nonce":115446},{"type":"v1","addr":"172.21.15.55:6813","nonce":115446}]},"public_addr":"172.21.15.55:6809/115446","cluster_addr":"172.21.15.55:6811/115446","heartbeat_back_addr":"172.21.15.55:6815/115446","heartbeat_front_addr":"172.21.15.55:6813/115446","state":["exists","up"]},{"osd":2,"uuid":"e36c0a33-7bf6-4b80-b9fc-e9512ae922cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6816","nonce":115462},{"type":"v1","addr":"172.21.15.55:6817","nonce":115462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6818","nonce":115462},{"type":"v1","addr":"172.21.15.55:6819","nonce":115462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6822","nonce":115462},{"type":"v1","addr":"172.21.15.55:6823","nonce":115462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6820","nonce":115462},{"type":"v1","addr":"172.21.15.55:6821","nonce":115462}]},"public_addr":"172.21.15.55:6817/115462","cluster_addr":"172.21.15.55:6819/115462","heartbeat_back_addr":"172.21.15.55:6823/115462","heartbeat_front_addr":"172.21.15.55:6821/115462","state":["exists","up"]},{"osd":3,"uuid":"3311eaac-bc3b-443c-8513-53016de7c85a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6824","nonce":115464},{"type":"v1","addr":"172.21.15.55:6825","nonce":115464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6826","nonce":115464},{"type":"v1","addr":"172.21.15.55:6827","nonce":115464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6830","nonce":115464},{"type":"v1","addr":"172.21.15.55:6831","nonce":115464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6828","nonce":115464},{"type":"v1","addr":"172.21.15.55:6829","nonce":115464}]},"public_addr":"172.21.15.55:6825/115464","cluster_addr":"172.21.15.55:6827/115464","heartbeat_back_addr":"172.21.15.55:6831/115464","heartbeat_front_addr":"172.21.15.55:6829/115464","state":["exists","up"]},{"osd":4,"uuid":"1c3239d0-0585-40df-bdb0-2ffc191022d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6824","nonce":85783},{"type":"v1","addr":"172.21.15.163:6825","nonce":85783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6826","nonce":85783},{"type":"v1","addr":"172.21.15.163:6827","nonce":85783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6830","nonce":85783},{"type":"v1","addr":"172.21.15.163:6831","nonce":85783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6828","nonce":85783},{"type":"v1","addr":"172.21.15.163:6829","nonce":85783}]},"public_addr":"172.21.15.163:6825/85783","cluster_addr":"172.21.15.163:6827/85783","heartbeat_back_addr":"172.21.15.163:6831/85783","heartbeat_front_addr":"172.21.15.163:6829/85783","state":["exists","up"]},{"osd":5,"uuid":"f49dfabb-4d9c-46bd-811b-984484ad87a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6816","nonce":85786},{"type":"v1","addr":"172.21.15.163:6817","nonce":85786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6818","nonce":85786},{"type":"v1","addr":"172.21.15.163:6819","nonce":85786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6822","nonce":85786},{"type":"v1","addr":"172.21.15.163:6823","nonce":85786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6820","nonce":85786},{"type":"v1","addr":"172.21.15.163:6821","nonce":85786}]},"public_addr":"172.21.15.163:6817/85786","cluster_addr":"172.21.15.163:6819/85786","heartbeat_back_addr":"172.21.15.163:6823/85786","heartbeat_front_addr":"172.21.15.163:6821/85786","state":["exists","up"]},{"osd":6,"uuid":"d78418d1-8ff5-4678-a71d-1d18d19ffbfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6800","nonce":85781},{"type":"v1","addr":"172.21.15.163:6801","nonce":85781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6802","nonce":85781},{"type":"v1","addr":"172.21.15.163:6803","nonce":85781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6806","nonce":85781},{"type":"v1","addr":"172.21.15.163:6807","nonce":85781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6804","nonce":85781},{"type":"v1","addr":"172.21.15.163:6805","nonce":85781}]},"public_addr":"172.21.15.163:6801/85781","cluster_addr":"172.21.15.163:6803/85781","heartbeat_back_addr":"172.21.15.163:6807/85781","heartbeat_front_addr":"172.21.15.163:6805/85781","state":["exists","up"]},{"osd":7,"uuid":"7b768a49-37d1-4cb3-a670-ab9027d1248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6808","nonce":85782},{"type":"v1","addr":"172.21.15.163:6809","nonce":85782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6810","nonce":85782},{"type":"v1","addr":"172.21.15.163:6811","nonce":85782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6814","nonce":85782},{"type":"v1","addr":"172.21.15.163:6815","nonce":85782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6812","nonce":85782},{"type":"v1","addr":"172.21.15.163:6813","nonce":85782}]},"public_addr":"172.21.15.163:6809/85782","cluster_addr":"172.21.15.163:6811/85782","heartbeat_back_addr":"172.21.15.163:6815/85782","heartbeat_front_addr":"172.21.15.163:6813/85782","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.096175+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.189936+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.387003+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.507681+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.269229+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:04.248447+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.839836+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T03:33:03.893338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.163:0/1081678834":"2023-10-07T05:07:49.312979+0000","172.21.15.163:0/717025412":"2023-10-07T05:07:49.312979+0000","172.21.15.163:0/843041363":"2023-10-07T05:07:49.312979+0000","172.21.15.163:0/1037233743":"2023-10-07T05:07:49.312979+0000","172.21.15.163:0/1370950731":"2023-10-07T05:07:49.312979+0000","172.21.15.163:0/2693142845":"2023-10-07T05:07:49.312979+0000","172.21.15.55:6833/3632968604":"2023-10-07T05:07:49.122009+0000","172.21.15.55:6835/3632968604":"2023-10-07T05:07:49.122009+0000","172.21.15.163:0/2709454586":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/838216209":"2023-10-07T05:07:49.312979+0000","172.21.15.163:6837/101089":"2023-10-07T05:07:39.103885+0000","172.21.15.163:6836/101089":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/2719429847":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/3618319475":"2023-10-07T05:07:39.103885+0000","172.21.15.163:0/2222063413":"2023-10-07T05:07:39.103885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":1},{"begin":6,"length":2},{"begin":14,"length":1},{"begin":18,"length":2},{"begin":26,"length":1},{"begin":30,"length":2},{"begin":36,"length":1},{"begin":40,"length":3},{"begin":47,"length":1},{"begin":51,"length":3},{"begin":58,"length":1},{"begin":63,"length":2},{"begin":69,"length":1},{"begin":74,"length":2},{"begin":80,"length":1},{"begin":84,"length":2},{"begin":90,"length":1},{"begin":95,"length":2},{"begin":101,"length":1},{"begin":105,"length":2},{"begin":110,"length":2},{"begin":188,"length":34}]},{"pool":8,"snaps":[{"begin":2,"length":4}]},{"pool":22,"snaps":[{"begin":2,"length":32}]},{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T05:07:57.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.509+0000 7fd6a139d700 1 -- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fd688066f30 msgr2=0x7fd6880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:57.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.509+0000 7fd6a139d700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fd688066f30 0x7fd6880693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd68c006f00 tx=0x7fd68c008040 comp rx=0 tx=0).stop 2023-10-06T05:07:57.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.509+0000 7fd6a139d700 1 -- 172.21.15.55:0/1896445608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd69c0b0a00 msgr2=0x7fd69c1552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:57.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.509+0000 7fd6a139d700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd69c0b0a00 0x7fd69c1552b0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fd69c068ed0 tx=0x7fd690017000 comp rx=0 tx=0).stop 2023-10-06T05:07:57.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.510+0000 7fd6a139d700 1 -- 172.21.15.55:0/1896445608 shutdown_connections 2023-10-06T05:07:57.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.510+0000 7fd6a139d700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fd688066f30 0x7fd6880693e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.510+0000 7fd6a139d700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fd69c157270 0x7fd69c151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.510+0000 7fd6a139d700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fd69c0688d0 0x7fd69c0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.510+0000 7fd6a139d700 1 --2- 172.21.15.55:0/1896445608 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fd69c0b0a00 0x7fd69c1552b0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:57.512 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.510+0000 7fd6a139d700 1 -- 172.21.15.55:0/1896445608 >> 172.21.15.55:0/1896445608 conn(0x7fd69c0cd310 msgr2=0x7fd69c159650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:57.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.510+0000 7fd6a139d700 1 -- 172.21.15.55:0/1896445608 shutdown_connections 2023-10-06T05:07:57.513 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:57.510+0000 7fd6a139d700 1 -- 172.21.15.55:0/1896445608 wait complete. 2023-10-06T05:07:58.530 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T05:07:58.531 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:07:58.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.732+0000 7f82f919d700 1 Processor -- start 2023-10-06T05:07:58.734 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.732+0000 7f82f919d700 1 -- start start 2023-10-06T05:07:58.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.733+0000 7f82f919d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f82f40cc8d0 0x7f82f415b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:58.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.733+0000 7f82f919d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82f415b960 0x7f82f40ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:58.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.733+0000 7f82f919d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f4157a50 con 0x7f82f415b050 2023-10-06T05:07:58.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.733+0000 7f82f919d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f4158f10 con 0x7f82f40cc8d0 2023-10-06T05:07:58.735 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.733+0000 7f82f919d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f41591b0 con 0x7f82f415b960 2023-10-06T05:07:58.736 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.734+0000 7f82f3fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f82f415b050 0x7f82f40ca100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:40158/0 (socket says 172.21.15.55:40158) 2023-10-06T05:07:58.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.734+0000 7f82f37fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f82f40cc8d0 0x7f82f415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:58.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.734+0000 7f82f3fff700 1 -- 172.21.15.55:0/2961447788 learned_addr learned my addr 172.21.15.55:0/2961447788 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:58.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.734+0000 7f82f899c700 1 --2- 172.21.15.55:0/2961447788 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82f415b960 0x7f82f40ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:58.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.734+0000 7f82f37fe700 1 -- 172.21.15.55:0/2961447788 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82f415b960 msgr2=0x7f82f40ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:58.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.734+0000 7f82f37fe700 1 --2- 172.21.15.55:0/2961447788 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82f415b960 0x7f82f40ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:58.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.734+0000 7f82f2ffd700 1 -- 172.21.15.55:0/2961447788 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644150091 0 0) 0x7f82f4157a50 con 0x7f82f415b050 2023-10-06T05:07:58.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.734+0000 7f82f37fe700 1 -- 172.21.15.55:0/2961447788 >> v1:172.21.15.55:6789/0 conn(0x7f82f415b050 legacy=0x7f82f40ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:58.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.734+0000 7f82f37fe700 1 -- 172.21.15.55:0/2961447788 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f40ca9e0 con 0x7f82f40cc8d0 2023-10-06T05:07:58.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.734+0000 7f82f37fe700 1 --2- 172.21.15.55:0/2961447788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f40cc8d0 0x7f82f415b420 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f82e80098e0 tx=0x7f82e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e80a6b4c68088ea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:58.737 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.735+0000 7f82f2ffd700 1 -- 172.21.15.55:0/2961447788 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e800b640 con 0x7f82f40cc8d0 2023-10-06T05:07:58.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.735+0000 7f82f2ffd700 1 -- 172.21.15.55:0/2961447788 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e800b7a0 con 0x7f82f40cc8d0 2023-10-06T05:07:58.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.735+0000 7f82f2ffd700 1 -- 172.21.15.55:0/2961447788 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e800bb40 con 0x7f82f40cc8d0 2023-10-06T05:07:58.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.735+0000 7f82f919d700 1 -- 172.21.15.55:0/2961447788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f40cc8d0 msgr2=0x7f82f415b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:58.738 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.735+0000 7f82f919d700 1 --2- 172.21.15.55:0/2961447788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f40cc8d0 0x7f82f415b420 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f82e80098e0 tx=0x7f82e80091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:58.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.735+0000 7f82f919d700 1 -- 172.21.15.55:0/2961447788 shutdown_connections 2023-10-06T05:07:58.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.735+0000 7f82f919d700 1 --2- 172.21.15.55:0/2961447788 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82f415b960 0x7f82f40ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:58.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.735+0000 7f82f919d700 1 --2- 172.21.15.55:0/2961447788 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f40cc8d0 0x7f82f415b420 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:58.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.735+0000 7f82f919d700 1 -- 172.21.15.55:0/2961447788 >> 172.21.15.55:0/2961447788 conn(0x7f82f4068e20 msgr2=0x7f82f40cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:58.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.735+0000 7f82f919d700 1 -- 172.21.15.55:0/2961447788 shutdown_connections 2023-10-06T05:07:58.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.735+0000 7f82f919d700 1 -- 172.21.15.55:0/2961447788 wait complete. 2023-10-06T05:07:58.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.737+0000 7f82f919d700 1 Processor -- start 2023-10-06T05:07:58.739 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.737+0000 7f82f919d700 1 -- start start 2023-10-06T05:07:58.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f919d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f40cc8d0 0x7f82f4145b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:58.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f919d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82f415b050 0x7f82f40a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:58.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f919d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f415b960 0x7f82f40a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:58.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f919d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40c6f70 con 0x7f82f415b960 2023-10-06T05:07:58.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f919d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40c6360 con 0x7f82f40cc8d0 2023-10-06T05:07:58.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f919d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40c6640 con 0x7f82f415b050 2023-10-06T05:07:58.741 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f899c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f415b960 0x7f82f40a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:58.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f899c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f415b960 0x7f82f40a5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54660/0 (socket says 172.21.15.55:54660) 2023-10-06T05:07:58.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f899c700 1 -- 172.21.15.55:0/3707139174 learned_addr learned my addr 172.21.15.55:0/3707139174 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:58.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f3fff700 1 --2- 172.21.15.55:0/3707139174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f40cc8d0 0x7f82f4145b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:58.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f37fe700 1 --2- 172.21.15.55:0/3707139174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82f415b050 0x7f82f40a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:58.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f899c700 1 -- 172.21.15.55:0/3707139174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82f415b050 msgr2=0x7f82f40a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:58.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f899c700 1 --2- 172.21.15.55:0/3707139174 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82f415b050 0x7f82f40a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:58.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f899c700 1 -- 172.21.15.55:0/3707139174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f40cc8d0 msgr2=0x7f82f4145b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:58.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.738+0000 7f82f899c700 1 --2- 172.21.15.55:0/3707139174 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82f40cc8d0 0x7f82f4145b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:58.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.739+0000 7f82f899c700 1 -- 172.21.15.55:0/3707139174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e8009420 con 0x7f82f415b960 2023-10-06T05:07:58.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.739+0000 7f82f899c700 1 --2- 172.21.15.55:0/3707139174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f415b960 0x7f82f40a5a00 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009e90 tx=0x7f82ec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:58.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.739+0000 7f82dbfff700 1 -- 172.21.15.55:0/3707139174 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec00cb70 con 0x7f82f415b960 2023-10-06T05:07:58.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.739+0000 7f82dbfff700 1 -- 172.21.15.55:0/3707139174 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec00f040 con 0x7f82f415b960 2023-10-06T05:07:58.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.739+0000 7f82dbfff700 1 -- 172.21.15.55:0/3707139174 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec014480 con 0x7f82f415b960 2023-10-06T05:07:58.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.739+0000 7f82f919d700 1 -- 172.21.15.55:0/3707139174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f40c6920 con 0x7f82f415b960 2023-10-06T05:07:58.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.739+0000 7f82f919d700 1 -- 172.21.15.55:0/3707139174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f4146090 con 0x7f82f415b960 2023-10-06T05:07:58.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.740+0000 7f82f919d700 1 -- 172.21.15.55:0/3707139174 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f82f404c3c0 con 0x7f82f415b960 2023-10-06T05:07:58.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.741+0000 7f82dbfff700 1 -- 172.21.15.55:0/3707139174 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f82ec014830 con 0x7f82f415b960 2023-10-06T05:07:58.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.742+0000 7f82dbfff700 1 --2- 172.21.15.55:0/3707139174 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f82e0066f30 0x7f82e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:58.744 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.742+0000 7f82dbfff700 1 -- 172.21.15.55:0/3707139174 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f82ec085250 con 0x7f82f415b960 2023-10-06T05:07:58.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.742+0000 7f82dbfff700 1 --2- 172.21.15.55:0/3707139174 >> [v2:172.21.15.55:6800/115443,v1:172.21.15.55:6801/115443] conn(0x7f82e006dc30 0x7f82e0070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:58.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.742+0000 7f82dbfff700 1 -- 172.21.15.55:0/3707139174 --> [v2:172.21.15.55:6800/115443,v1:172.21.15.55:6801/115443] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82e0070680 con 0x7f82e006dc30 2023-10-06T05:07:58.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.742+0000 7f82dbfff700 1 -- 172.21.15.55:0/3707139174 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=212) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82ec087750 con 0x7f82f415b960 2023-10-06T05:07:58.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.742+0000 7f82f3fff700 1 --2- 172.21.15.55:0/3707139174 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f82e0066f30 0x7f82e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:58.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.743+0000 7f82f37fe700 1 --2- 172.21.15.55:0/3707139174 >> [v2:172.21.15.55:6800/115443,v1:172.21.15.55:6801/115443] conn(0x7f82e006dc30 0x7f82e0070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:58.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.743+0000 7f82f3fff700 1 --2- 172.21.15.55:0/3707139174 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f82e0066f30 0x7f82e00693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f82e4007160 tx=0x7f82e4007f50 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:58.745 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.743+0000 7f82f37fe700 1 --2- 172.21.15.55:0/3707139174 >> [v2:172.21.15.55:6800/115443,v1:172.21.15.55:6801/115443] conn(0x7f82e006dc30 0x7f82e0070050 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:58.746 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.744+0000 7f82dbfff700 1 -- 172.21.15.55:0/3707139174 <== osd.0 v2:172.21.15.55:6800/115443 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f82e0070680 con 0x7f82e006dc30 2023-10-06T05:07:58.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.758+0000 7f82f919d700 1 -- 172.21.15.55:0/3707139174 --> [v2:172.21.15.55:6800/115443,v1:172.21.15.55:6801/115443] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f82f415ad10 con 0x7f82e006dc30 2023-10-06T05:07:58.760 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.759+0000 7f82dbfff700 1 -- 172.21.15.55:0/3707139174 <== osd.0 v2:172.21.15.55:6800/115443 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f82f415ad10 con 0x7f82e006dc30 2023-10-06T05:07:58.760 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T05:07:58.761 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:07:58.761 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T05:07:58.761 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.759+0000 7f82f919d700 1 -- 172.21.15.55:0/3707139174 >> [v2:172.21.15.55:6800/115443,v1:172.21.15.55:6801/115443] conn(0x7f82e006dc30 msgr2=0x7f82e0070050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:58.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.759+0000 7f82f919d700 1 --2- 172.21.15.55:0/3707139174 >> [v2:172.21.15.55:6800/115443,v1:172.21.15.55:6801/115443] conn(0x7f82e006dc30 0x7f82e0070050 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:58.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.759+0000 7f82f919d700 1 -- 172.21.15.55:0/3707139174 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f82e0066f30 msgr2=0x7f82e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:58.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.759+0000 7f82f919d700 1 --2- 172.21.15.55:0/3707139174 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f82e0066f30 0x7f82e00693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f82e4007160 tx=0x7f82e4007f50 comp rx=0 tx=0).stop 2023-10-06T05:07:58.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.759+0000 7f82f919d700 1 -- 172.21.15.55:0/3707139174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f415b960 msgr2=0x7f82f40a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:58.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.759+0000 7f82f919d700 1 --2- 172.21.15.55:0/3707139174 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82f415b960 0x7f82f40a5a00 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f82ec009e90 tx=0x7f82ec00b040 comp rx=0 tx=0).stop 2023-10-06T05:07:58.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.759+0000 7f82f899c700 1 -- 172.21.15.55:0/3707139174 reap_dead start 2023-10-06T05:07:58.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.759+0000 7f82f919d700 1 -- 172.21.15.55:0/3707139174 shutdown_connections 2023-10-06T05:07:58.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.759+0000 7f82f919d700 1 -- 172.21.15.55:0/3707139174 >> 172.21.15.55:0/3707139174 conn(0x7f82f4068e20 msgr2=0x7f82f406d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:58.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.759+0000 7f82f919d700 1 -- 172.21.15.55:0/3707139174 shutdown_connections 2023-10-06T05:07:58.762 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.759+0000 7f82f919d700 1 -- 172.21.15.55:0/3707139174 wait complete. 2023-10-06T05:07:58.777 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T05:07:58.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.963+0000 7f4bc279d700 1 Processor -- start 2023-10-06T05:07:58.965 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.963+0000 7f4bc279d700 1 -- start start 2023-10-06T05:07:58.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.965+0000 7f4bc279d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4bbc0ae710 0x7f4bbc0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:58.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.965+0000 7f4bc279d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bbc0ca860 con 0x7f4bbc156310 2023-10-06T05:07:58.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.965+0000 7f4bc279d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bbc0cab40 con 0x7f4bbc1566e0 2023-10-06T05:07:58.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.965+0000 7f4bc279d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bbc0cae20 con 0x7f4bbc0ae710 2023-10-06T05:07:58.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.965+0000 7f4bc1f9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f4bbc0ae710 0x7f4bbc0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:58.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.965+0000 7f4bc179b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f4bbc156310 0x7f4bbc0c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:40162/0 (socket says 172.21.15.55:40162) 2023-10-06T05:07:58.967 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.965+0000 7f4bc179b700 1 -- 172.21.15.55:0/1181973 learned_addr learned my addr 172.21.15.55:0/1181973 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:58.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.966+0000 7f4bc1f9c700 1 -- 172.21.15.55:0/1181973 >> v1:172.21.15.163:6789/0 conn(0x7f4bbc1566e0 legacy=0x7f4bbc0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:58.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.966+0000 7f4bc1f9c700 1 -- 172.21.15.55:0/1181973 >> v1:172.21.15.55:6789/0 conn(0x7f4bbc156310 legacy=0x7f4bbc0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:58.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.966+0000 7f4bc1f9c700 1 -- 172.21.15.55:0/1181973 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc0cb100 con 0x7f4bbc0ae710 2023-10-06T05:07:58.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.966+0000 7f4bb3fff700 1 -- 172.21.15.55:0/1181973 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136306960 0 0) 0x7f4bbc0ca860 con 0x7f4bbc156310 2023-10-06T05:07:58.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.966+0000 7f4bc1f9c700 1 --2- 172.21.15.55:0/1181973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc0ae710 0x7f4bbc0c6830 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009e60 tx=0x7f4bb4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f833ac1fb007f1fb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:58.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.966+0000 7f4bb3fff700 1 -- 172.21.15.55:0/1181973 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb400e070 con 0x7f4bbc0ae710 2023-10-06T05:07:58.969 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.966+0000 7f4bb3fff700 1 -- 172.21.15.55:0/1181973 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb400bc30 con 0x7f4bbc0ae710 2023-10-06T05:07:58.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.966+0000 7f4bb3fff700 1 -- 172.21.15.55:0/1181973 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4012430 con 0x7f4bbc0ae710 2023-10-06T05:07:58.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.967+0000 7f4bc279d700 1 -- 172.21.15.55:0/1181973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc0ae710 msgr2=0x7f4bbc0c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:58.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.967+0000 7f4bc279d700 1 --2- 172.21.15.55:0/1181973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc0ae710 0x7f4bbc0c6830 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009e60 tx=0x7f4bb4009780 comp rx=0 tx=0).stop 2023-10-06T05:07:58.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.967+0000 7f4bc279d700 1 -- 172.21.15.55:0/1181973 shutdown_connections 2023-10-06T05:07:58.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.967+0000 7f4bc279d700 1 --2- 172.21.15.55:0/1181973 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc0ae710 0x7f4bbc0c6830 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:58.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.967+0000 7f4bc279d700 1 -- 172.21.15.55:0/1181973 >> 172.21.15.55:0/1181973 conn(0x7f4bbc0cc1f0 msgr2=0x7f4bbc0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:58.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.967+0000 7f4bc279d700 1 -- 172.21.15.55:0/1181973 shutdown_connections 2023-10-06T05:07:58.970 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.967+0000 7f4bc279d700 1 -- 172.21.15.55:0/1181973 wait complete. 2023-10-06T05:07:58.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.968+0000 7f4bc279d700 1 Processor -- start 2023-10-06T05:07:58.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.969+0000 7f4bc279d700 1 -- start start 2023-10-06T05:07:58.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.969+0000 7f4bc279d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bbc0ae710 0x7f4bbc151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:58.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.969+0000 7f4bc279d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bbc156310 0x7f4bbc0aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:58.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.969+0000 7f4bc279d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc1566e0 0x7f4bbc0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:58.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.969+0000 7f4bc279d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc169700 con 0x7f4bbc1566e0 2023-10-06T05:07:58.972 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.969+0000 7f4bc279d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc1540d0 con 0x7f4bbc0ae710 2023-10-06T05:07:58.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.969+0000 7f4bc279d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc1543b0 con 0x7f4bbc156310 2023-10-06T05:07:58.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.970+0000 7f4bc1f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc1566e0 0x7f4bbc0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:58.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.970+0000 7f4bc1f9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc1566e0 0x7f4bbc0af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54688/0 (socket says 172.21.15.55:54688) 2023-10-06T05:07:58.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.970+0000 7f4bc1f9c700 1 -- 172.21.15.55:0/1145548543 learned_addr learned my addr 172.21.15.55:0/1145548543 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:58.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.970+0000 7f4bc179b700 1 --2- 172.21.15.55:0/1145548543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bbc0ae710 0x7f4bbc151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:58.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.970+0000 7f4bc0f9a700 1 --2- 172.21.15.55:0/1145548543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bbc156310 0x7f4bbc0aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:58.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.970+0000 7f4bc1f9c700 1 -- 172.21.15.55:0/1145548543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bbc156310 msgr2=0x7f4bbc0aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:58.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.970+0000 7f4bc1f9c700 1 --2- 172.21.15.55:0/1145548543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bbc156310 0x7f4bbc0aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:58.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.970+0000 7f4bc1f9c700 1 -- 172.21.15.55:0/1145548543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bbc0ae710 msgr2=0x7f4bbc151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:58.973 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.970+0000 7f4bc1f9c700 1 --2- 172.21.15.55:0/1145548543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bbc0ae710 0x7f4bbc151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:58.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.970+0000 7f4bc1f9c700 1 -- 172.21.15.55:0/1145548543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb4009a20 con 0x7f4bbc1566e0 2023-10-06T05:07:58.974 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.970+0000 7f4bc1f9c700 1 --2- 172.21.15.55:0/1145548543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc1566e0 0x7f4bbc0af4d0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc06fe40 tx=0x7f4bb4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:58.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.971+0000 7f4bb17fa700 1 -- 172.21.15.55:0/1145548543 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb401b630 con 0x7f4bbc1566e0 2023-10-06T05:07:58.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.971+0000 7f4bb17fa700 1 -- 172.21.15.55:0/1145548543 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb401b7d0 con 0x7f4bbc1566e0 2023-10-06T05:07:58.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.971+0000 7f4bb17fa700 1 -- 172.21.15.55:0/1145548543 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb401bb90 con 0x7f4bbc1566e0 2023-10-06T05:07:58.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.971+0000 7f4bc279d700 1 -- 172.21.15.55:0/1145548543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bbc154690 con 0x7f4bbc1566e0 2023-10-06T05:07:58.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.971+0000 7f4bc279d700 1 -- 172.21.15.55:0/1145548543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bbc0c6ae0 con 0x7f4bbc1566e0 2023-10-06T05:07:58.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.972+0000 7f4bc279d700 1 -- 172.21.15.55:0/1145548543 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bbc0c6940 con 0x7f4bbc1566e0 2023-10-06T05:07:58.975 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.973+0000 7f4bb17fa700 1 -- 172.21.15.55:0/1145548543 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4bb401c400 con 0x7f4bbc1566e0 2023-10-06T05:07:58.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.974+0000 7f4bb17fa700 1 --2- 172.21.15.55:0/1145548543 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4ba0066f30 0x7f4ba00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:58.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.974+0000 7f4bb17fa700 1 -- 172.21.15.55:0/1145548543 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f4bb4086720 con 0x7f4bbc1566e0 2023-10-06T05:07:58.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.974+0000 7f4bc179b700 1 --2- 172.21.15.55:0/1145548543 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4ba0066f30 0x7f4ba00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:58.976 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.975+0000 7f4bc179b700 1 --2- 172.21.15.55:0/1145548543 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4ba0066f30 0x7f4ba00693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8007fd0 tx=0x7f4bb8007d30 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:58.979 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:58.978+0000 7f4bb17fa700 1 -- 172.21.15.55:0/1145548543 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bbc0c6940 con 0x7f4bbc1566e0 2023-10-06T05:07:59.179 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.177+0000 7f4bc279d700 1 -- 172.21.15.55:0/1145548543 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bbc154520 con 0x7f4ba0066f30 2023-10-06T05:07:59.180 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.178+0000 7f4bb17fa700 1 -- 172.21.15.55:0/1145548543 <== mgr.10033 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4bbc154520 con 0x7f4ba0066f30 2023-10-06T05:07:59.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.180+0000 7f4bc279d700 1 -- 172.21.15.55:0/1145548543 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4ba0066f30 msgr2=0x7f4ba00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.182 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.180+0000 7f4bc279d700 1 --2- 172.21.15.55:0/1145548543 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4ba0066f30 0x7f4ba00693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8007fd0 tx=0x7f4bb8007d30 comp rx=0 tx=0).stop 2023-10-06T05:07:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.180+0000 7f4bc279d700 1 -- 172.21.15.55:0/1145548543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc1566e0 msgr2=0x7f4bbc0af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.180+0000 7f4bc279d700 1 --2- 172.21.15.55:0/1145548543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc1566e0 0x7f4bbc0af4d0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc06fe40 tx=0x7f4bb4019000 comp rx=0 tx=0).stop 2023-10-06T05:07:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.180+0000 7f4bc279d700 1 -- 172.21.15.55:0/1145548543 shutdown_connections 2023-10-06T05:07:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.180+0000 7f4bc279d700 1 --2- 172.21.15.55:0/1145548543 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4ba0066f30 0x7f4ba00693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.180+0000 7f4bc279d700 1 --2- 172.21.15.55:0/1145548543 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f4bbc1566e0 0x7f4bbc0af4d0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.180+0000 7f4bc279d700 1 --2- 172.21.15.55:0/1145548543 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f4bbc0ae710 0x7f4bbc151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.180+0000 7f4bc279d700 1 --2- 172.21.15.55:0/1145548543 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f4bbc156310 0x7f4bbc0aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.180+0000 7f4bc279d700 1 -- 172.21.15.55:0/1145548543 >> 172.21.15.55:0/1145548543 conn(0x7f4bbc0cc1f0 msgr2=0x7f4bbc0c9650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.180+0000 7f4bc279d700 1 -- 172.21.15.55:0/1145548543 shutdown_connections 2023-10-06T05:07:59.183 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.180+0000 7f4bc279d700 1 -- 172.21.15.55:0/1145548543 wait complete. 2023-10-06T05:07:59.184 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T05:07:59.200 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T05:07:59.200 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:07:59.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.390+0000 7f9b0a79d700 1 Processor -- start 2023-10-06T05:07:59.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.390+0000 7f9b0a79d700 1 -- start start 2023-10-06T05:07:59.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.391+0000 7f9b0a79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b040685e0 0x7f9b0415a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:59.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.391+0000 7f9b0a79d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9b04154000 0x7f9b0406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:59.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.391+0000 7f9b0a79d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b040c9480 con 0x7f9b0415d3f0 2023-10-06T05:07:59.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.391+0000 7f9b0a79d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b0415d880 con 0x7f9b04154000 2023-10-06T05:07:59.393 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.391+0000 7f9b0a79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0415db60 con 0x7f9b040685e0 2023-10-06T05:07:59.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.391+0000 7f9b0979b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b040685e0 0x7f9b0415a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:59.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.391+0000 7f9b0979b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b040685e0 0x7f9b0415a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:45556/0 (socket says 172.21.15.55:45556) 2023-10-06T05:07:59.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.391+0000 7f9b09f9c700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9b04154000 0x7f9b0406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:59.394 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.391+0000 7f9b0979b700 1 -- 172.21.15.55:0/885609980 learned_addr learned my addr 172.21.15.55:0/885609980 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:59.395 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.392+0000 7f9b0979b700 1 -- 172.21.15.55:0/885609980 >> v2:172.21.15.163:3300/0 conn(0x7f9b04154000 msgr2=0x7f9b0406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:07:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.392+0000 7f9b0979b700 1 --2- 172.21.15.55:0/885609980 >> v2:172.21.15.163:3300/0 conn(0x7f9b04154000 0x7f9b0406eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.392+0000 7f9b0979b700 1 -- 172.21.15.55:0/885609980 >> v1:172.21.15.163:6789/0 conn(0x7f9b0415d3f0 legacy=0x7f9b041538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.392+0000 7f9b0979b700 1 -- 172.21.15.55:0/885609980 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0415de40 con 0x7f9b040685e0 2023-10-06T05:07:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.392+0000 7f9b0979b700 1 --2- 172.21.15.55:0/885609980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b040685e0 0x7f9b0415a6f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9b000099b0 tx=0x7f9b00009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78dcbaa9635475b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.392+0000 7f9afbfff700 1 -- 172.21.15.55:0/885609980 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00009ea0 con 0x7f9b040685e0 2023-10-06T05:07:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.392+0000 7f9afbfff700 1 -- 172.21.15.55:0/885609980 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0000b8f0 con 0x7f9b040685e0 2023-10-06T05:07:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.392+0000 7f9afbfff700 1 -- 172.21.15.55:0/885609980 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000bc90 con 0x7f9b040685e0 2023-10-06T05:07:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.393+0000 7f9b0a79d700 1 -- 172.21.15.55:0/885609980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b040685e0 msgr2=0x7f9b0415a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.396 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.393+0000 7f9b0a79d700 1 --2- 172.21.15.55:0/885609980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b040685e0 0x7f9b0415a6f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9b000099b0 tx=0x7f9b00009250 comp rx=0 tx=0).stop 2023-10-06T05:07:59.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.393+0000 7f9b0a79d700 1 -- 172.21.15.55:0/885609980 shutdown_connections 2023-10-06T05:07:59.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.393+0000 7f9b0a79d700 1 --2- 172.21.15.55:0/885609980 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b040685e0 0x7f9b0415a6f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.393+0000 7f9b0a79d700 1 --2- 172.21.15.55:0/885609980 >> v2:172.21.15.163:3300/0 conn(0x7f9b04154000 0x7f9b0406eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.393+0000 7f9b0a79d700 1 -- 172.21.15.55:0/885609980 >> 172.21.15.55:0/885609980 conn(0x7f9b0406b420 msgr2=0x7f9b040cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:59.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.393+0000 7f9b0a79d700 1 -- 172.21.15.55:0/885609980 shutdown_connections 2023-10-06T05:07:59.397 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.393+0000 7f9b0a79d700 1 -- 172.21.15.55:0/885609980 wait complete. 2023-10-06T05:07:59.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.394+0000 7f9b0a79d700 1 Processor -- start 2023-10-06T05:07:59.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.395+0000 7f9b0a79d700 1 -- start start 2023-10-06T05:07:59.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.395+0000 7f9b0a79d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b040685e0 0x7f9b04159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:59.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.395+0000 7f9b0a79d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b04154000 0x7f9b04151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:59.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b0a79d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b0415d3f0 0x7f9b040c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:59.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b0a79d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b04169f40 con 0x7f9b040685e0 2023-10-06T05:07:59.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b0a79d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0416aa50 con 0x7f9b04154000 2023-10-06T05:07:59.399 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b0a79d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0416aee0 con 0x7f9b0415d3f0 2023-10-06T05:07:59.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b0979b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b040685e0 0x7f9b04159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:59.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b0979b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b040685e0 0x7f9b04159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54692/0 (socket says 172.21.15.55:54692) 2023-10-06T05:07:59.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b0979b700 1 -- 172.21.15.55:0/3792083089 learned_addr learned my addr 172.21.15.55:0/3792083089 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:59.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b08f9a700 1 --2- 172.21.15.55:0/3792083089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b04154000 0x7f9b04151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:59.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b09f9c700 1 --2- 172.21.15.55:0/3792083089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b0415d3f0 0x7f9b040c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:59.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b0979b700 1 -- 172.21.15.55:0/3792083089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b0415d3f0 msgr2=0x7f9b040c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b0979b700 1 --2- 172.21.15.55:0/3792083089 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9b0415d3f0 0x7f9b040c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b0979b700 1 -- 172.21.15.55:0/3792083089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b04154000 msgr2=0x7f9b04151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b0979b700 1 --2- 172.21.15.55:0/3792083089 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9b04154000 0x7f9b04151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.400 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b0979b700 1 -- 172.21.15.55:0/3792083089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b000094f0 con 0x7f9b040685e0 2023-10-06T05:07:59.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.396+0000 7f9b0979b700 1 --2- 172.21.15.55:0/3792083089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b040685e0 0x7f9b04159640 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f9b04063fd0 tx=0x7f9b00017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:59.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.397+0000 7f9af97fa700 1 -- 172.21.15.55:0/3792083089 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0000ba60 con 0x7f9b040685e0 2023-10-06T05:07:59.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.397+0000 7f9af97fa700 1 -- 172.21.15.55:0/3792083089 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0000bbe0 con 0x7f9b040685e0 2023-10-06T05:07:59.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.397+0000 7f9af97fa700 1 -- 172.21.15.55:0/3792083089 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b00026680 con 0x7f9b040685e0 2023-10-06T05:07:59.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.397+0000 7f9b0a79d700 1 -- 172.21.15.55:0/3792083089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b04159c20 con 0x7f9b040685e0 2023-10-06T05:07:59.401 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.397+0000 7f9b0a79d700 1 -- 172.21.15.55:0/3792083089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0416a400 con 0x7f9b040685e0 2023-10-06T05:07:59.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.398+0000 7f9b0a79d700 1 -- 172.21.15.55:0/3792083089 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9b0404c3c0 con 0x7f9b040685e0 2023-10-06T05:07:59.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.399+0000 7f9af97fa700 1 -- 172.21.15.55:0/3792083089 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9b00023030 con 0x7f9b040685e0 2023-10-06T05:07:59.402 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.400+0000 7f9af97fa700 1 --2- 172.21.15.55:0/3792083089 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9ae8066f30 0x7f9ae80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:59.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.400+0000 7f9af97fa700 1 -- 172.21.15.55:0/3792083089 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f9b00086630 con 0x7f9b040685e0 2023-10-06T05:07:59.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.400+0000 7f9af97fa700 1 --2- 172.21.15.55:0/3792083089 >> [v2:172.21.15.55:6808/115446,v1:172.21.15.55:6809/115446] conn(0x7f9ae806dc30 0x7f9ae8070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:59.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.400+0000 7f9af97fa700 1 -- 172.21.15.55:0/3792083089 --> [v2:172.21.15.55:6808/115446,v1:172.21.15.55:6809/115446] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ae8070680 con 0x7f9ae806dc30 2023-10-06T05:07:59.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.400+0000 7f9af97fa700 1 -- 172.21.15.55:0/3792083089 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=212) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b00086920 con 0x7f9b040685e0 2023-10-06T05:07:59.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.400+0000 7f9b08f9a700 1 --2- 172.21.15.55:0/3792083089 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9ae8066f30 0x7f9ae80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:59.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.400+0000 7f9b09f9c700 1 --2- 172.21.15.55:0/3792083089 >> [v2:172.21.15.55:6808/115446,v1:172.21.15.55:6809/115446] conn(0x7f9ae806dc30 0x7f9ae8070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:59.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.401+0000 7f9b08f9a700 1 --2- 172.21.15.55:0/3792083089 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9ae8066f30 0x7f9ae80693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9b040afa70 tx=0x7f9af4009040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:59.403 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.401+0000 7f9b09f9c700 1 --2- 172.21.15.55:0/3792083089 >> [v2:172.21.15.55:6808/115446,v1:172.21.15.55:6809/115446] conn(0x7f9ae806dc30 0x7f9ae8070050 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:59.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.401+0000 7f9af97fa700 1 -- 172.21.15.55:0/3792083089 <== osd.1 v2:172.21.15.55:6808/115446 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f9ae8070680 con 0x7f9ae806dc30 2023-10-06T05:07:59.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.416+0000 7f9b0a79d700 1 -- 172.21.15.55:0/3792083089 --> [v2:172.21.15.55:6808/115446,v1:172.21.15.55:6809/115446] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9b0415e120 con 0x7f9ae806dc30 2023-10-06T05:07:59.418 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.416+0000 7f9af97fa700 1 -- 172.21.15.55:0/3792083089 <== osd.1 v2:172.21.15.55:6808/115446 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9b0415e120 con 0x7f9ae806dc30 2023-10-06T05:07:59.418 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T05:07:59.419 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:07:59.419 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T05:07:59.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.417+0000 7f9b0a79d700 1 -- 172.21.15.55:0/3792083089 >> [v2:172.21.15.55:6808/115446,v1:172.21.15.55:6809/115446] conn(0x7f9ae806dc30 msgr2=0x7f9ae8070050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.417+0000 7f9b0a79d700 1 --2- 172.21.15.55:0/3792083089 >> [v2:172.21.15.55:6808/115446,v1:172.21.15.55:6809/115446] conn(0x7f9ae806dc30 0x7f9ae8070050 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.417+0000 7f9b0a79d700 1 -- 172.21.15.55:0/3792083089 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9ae8066f30 msgr2=0x7f9ae80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.417+0000 7f9b0a79d700 1 --2- 172.21.15.55:0/3792083089 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9ae8066f30 0x7f9ae80693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9b040afa70 tx=0x7f9af4009040 comp rx=0 tx=0).stop 2023-10-06T05:07:59.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.417+0000 7f9b0a79d700 1 -- 172.21.15.55:0/3792083089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b040685e0 msgr2=0x7f9b04159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.417+0000 7f9b0a79d700 1 --2- 172.21.15.55:0/3792083089 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f9b040685e0 0x7f9b04159640 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f9b04063fd0 tx=0x7f9b00017000 comp rx=0 tx=0).stop 2023-10-06T05:07:59.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.417+0000 7f9b09f9c700 1 -- 172.21.15.55:0/3792083089 reap_dead start 2023-10-06T05:07:59.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.417+0000 7f9b0a79d700 1 -- 172.21.15.55:0/3792083089 shutdown_connections 2023-10-06T05:07:59.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.417+0000 7f9b0a79d700 1 -- 172.21.15.55:0/3792083089 >> 172.21.15.55:0/3792083089 conn(0x7f9b0406b420 msgr2=0x7f9b04152700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:59.420 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.417+0000 7f9b0a79d700 1 -- 172.21.15.55:0/3792083089 shutdown_connections 2023-10-06T05:07:59.421 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.417+0000 7f9b0a79d700 1 -- 172.21.15.55:0/3792083089 wait complete. 2023-10-06T05:07:59.435 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T05:07:59.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.621+0000 7f993719d700 1 Processor -- start 2023-10-06T05:07:59.623 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.621+0000 7f993719d700 1 -- start start 2023-10-06T05:07:59.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.623+0000 7f993719d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f9930150180 0x7f9930157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:59.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.623+0000 7f993719d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993006e2a0 con 0x7f9930150550 2023-10-06T05:07:59.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.623+0000 7f993719d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993006b2e0 con 0x7f99300c7410 2023-10-06T05:07:59.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.623+0000 7f993719d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99300c78e0 con 0x7f9930150180 2023-10-06T05:07:59.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.623+0000 7f993599a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f9930150550 0x7f99300c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:40172/0 (socket says 172.21.15.55:40172) 2023-10-06T05:07:59.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.623+0000 7f993599a700 1 -- 172.21.15.55:0/4069055187 learned_addr learned my addr 172.21.15.55:0/4069055187 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:07:59.625 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.623+0000 7f993619b700 1 --2- 172.21.15.55:0/4069055187 >> v2:172.21.15.163:3300/0 conn(0x7f9930150180 0x7f9930157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:59.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.623+0000 7f9935199700 1 -- 172.21.15.55:0/4069055187 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246084309 0 0) 0x7f993006e2a0 con 0x7f9930150550 2023-10-06T05:07:59.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.623+0000 7f9935199700 1 -- 172.21.15.55:0/4069055187 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f991c004680 con 0x7f9930150550 2023-10-06T05:07:59.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.624+0000 7f993619b700 1 -- 172.21.15.55:0/4069055187 >> v1:172.21.15.163:6789/0 conn(0x7f99300c7410 legacy=0x7f9930069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.624+0000 7f9935199700 1 -- 172.21.15.55:0/4069055187 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2478784017 0 0) 0x7f993006b2e0 con 0x7f99300c7410 2023-10-06T05:07:59.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.624+0000 7f993619b700 1 -- 172.21.15.55:0/4069055187 >> v1:172.21.15.55:6789/0 conn(0x7f9930150550 legacy=0x7f99300c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.624+0000 7f993619b700 1 -- 172.21.15.55:0/4069055187 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99300c7bc0 con 0x7f9930150180 2023-10-06T05:07:59.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.624+0000 7f993619b700 1 --2- 172.21.15.55:0/4069055187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9930150180 0x7f9930157770 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f992c0098e0 tx=0x7f992c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cbddb4b561a5ba4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:59.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.624+0000 7f9935199700 1 -- 172.21.15.55:0/4069055187 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c00f6b0 con 0x7f9930150180 2023-10-06T05:07:59.627 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.624+0000 7f9935199700 1 -- 172.21.15.55:0/4069055187 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c00f810 con 0x7f9930150180 2023-10-06T05:07:59.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.624+0000 7f9935199700 1 -- 172.21.15.55:0/4069055187 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c00fb90 con 0x7f9930150180 2023-10-06T05:07:59.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.625+0000 7f993719d700 1 -- 172.21.15.55:0/4069055187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9930150180 msgr2=0x7f9930157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.625+0000 7f993719d700 1 --2- 172.21.15.55:0/4069055187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9930150180 0x7f9930157770 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f992c0098e0 tx=0x7f992c0091a0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.625+0000 7f993719d700 1 -- 172.21.15.55:0/4069055187 shutdown_connections 2023-10-06T05:07:59.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.625+0000 7f993719d700 1 --2- 172.21.15.55:0/4069055187 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9930150180 0x7f9930157770 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.625+0000 7f993719d700 1 -- 172.21.15.55:0/4069055187 >> 172.21.15.55:0/4069055187 conn(0x7f99300cc1f0 msgr2=0x7f99300cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:59.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.625+0000 7f993719d700 1 -- 172.21.15.55:0/4069055187 shutdown_connections 2023-10-06T05:07:59.628 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.625+0000 7f993719d700 1 -- 172.21.15.55:0/4069055187 wait complete. 2023-10-06T05:07:59.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.626+0000 7f993719d700 1 Processor -- start 2023-10-06T05:07:59.629 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.627+0000 7f993719d700 1 -- start start 2023-10-06T05:07:59.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.627+0000 7f993719d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99300c7410 0x7f99300c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:59.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.627+0000 7f993719d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9930150180 0x7f9930149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:59.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.627+0000 7f993719d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9930150550 0x7f99300c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:59.630 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.628+0000 7f993719d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9930166d20 con 0x7f99300c7410 2023-10-06T05:07:59.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.628+0000 7f993719d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99300acf70 con 0x7f9930150180 2023-10-06T05:07:59.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.628+0000 7f993719d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993014a2b0 con 0x7f9930150550 2023-10-06T05:07:59.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.628+0000 7f993619b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99300c7410 0x7f99300c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:59.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.628+0000 7f993619b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99300c7410 0x7f99300c2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54698/0 (socket says 172.21.15.55:54698) 2023-10-06T05:07:59.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.628+0000 7f993619b700 1 -- 172.21.15.55:0/720889246 learned_addr learned my addr 172.21.15.55:0/720889246 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:07:59.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.628+0000 7f993699c700 1 --2- 172.21.15.55:0/720889246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9930150550 0x7f99300c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:59.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.628+0000 7f993599a700 1 --2- 172.21.15.55:0/720889246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9930150180 0x7f9930149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:59.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.628+0000 7f993619b700 1 -- 172.21.15.55:0/720889246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9930150550 msgr2=0x7f99300c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.628+0000 7f993619b700 1 --2- 172.21.15.55:0/720889246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9930150550 0x7f99300c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.628+0000 7f993619b700 1 -- 172.21.15.55:0/720889246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9930150180 msgr2=0x7f9930149d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.631 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.628+0000 7f993619b700 1 --2- 172.21.15.55:0/720889246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9930150180 0x7f9930149d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.628+0000 7f993619b700 1 -- 172.21.15.55:0/720889246 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c009420 con 0x7f99300c7410 2023-10-06T05:07:59.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.629+0000 7f993619b700 1 --2- 172.21.15.55:0/720889246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99300c7410 0x7f99300c2d30 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f992c0093f0 tx=0x7f992c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:59.632 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.629+0000 7f99267fc700 1 -- 172.21.15.55:0/720889246 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c00fb90 con 0x7f99300c7410 2023-10-06T05:07:59.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.629+0000 7f99267fc700 1 -- 172.21.15.55:0/720889246 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c01e070 con 0x7f99300c7410 2023-10-06T05:07:59.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.629+0000 7f99267fc700 1 -- 172.21.15.55:0/720889246 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c0196b0 con 0x7f99300c7410 2023-10-06T05:07:59.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.629+0000 7f993719d700 1 -- 172.21.15.55:0/720889246 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99300c5920 con 0x7f99300c7410 2023-10-06T05:07:59.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.629+0000 7f993719d700 1 -- 172.21.15.55:0/720889246 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99300acc00 con 0x7f99300c7410 2023-10-06T05:07:59.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.630+0000 7f993719d700 1 -- 172.21.15.55:0/720889246 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99300c5e40 con 0x7f99300c7410 2023-10-06T05:07:59.633 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.631+0000 7f99267fc700 1 -- 172.21.15.55:0/720889246 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f992c019a60 con 0x7f99300c7410 2023-10-06T05:07:59.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.632+0000 7f99267fc700 1 --2- 172.21.15.55:0/720889246 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9914066f30 0x7f99140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:07:59.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.632+0000 7f99267fc700 1 -- 172.21.15.55:0/720889246 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f992c014070 con 0x7f99300c7410 2023-10-06T05:07:59.634 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.632+0000 7f993599a700 1 --2- 172.21.15.55:0/720889246 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9914066f30 0x7f99140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:07:59.635 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.633+0000 7f993599a700 1 --2- 172.21.15.55:0/720889246 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9914066f30 0x7f99140693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f993016a260 tx=0x7f9920009040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:07:59.638 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.636+0000 7f99267fc700 1 -- 172.21.15.55:0/720889246 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f992c055790 con 0x7f99300c7410 2023-10-06T05:07:59.839 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.837+0000 7f993719d700 1 -- 172.21.15.55:0/720889246 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f99300aca80 con 0x7f9914066f30 2023-10-06T05:07:59.840 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.838+0000 7f99267fc700 1 -- 172.21.15.55:0/720889246 <== mgr.10033 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f992000f620 con 0x7f9914066f30 2023-10-06T05:07:59.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.840+0000 7f993719d700 1 -- 172.21.15.55:0/720889246 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9914066f30 msgr2=0x7f99140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.840+0000 7f993719d700 1 --2- 172.21.15.55:0/720889246 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9914066f30 0x7f99140693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f993016a260 tx=0x7f9920009040 comp rx=0 tx=0).stop 2023-10-06T05:07:59.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.840+0000 7f993719d700 1 -- 172.21.15.55:0/720889246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99300c7410 msgr2=0x7f99300c2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:07:59.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.840+0000 7f993719d700 1 --2- 172.21.15.55:0/720889246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99300c7410 0x7f99300c2d30 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f992c0093f0 tx=0x7f992c019000 comp rx=0 tx=0).stop 2023-10-06T05:07:59.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.840+0000 7f993719d700 1 -- 172.21.15.55:0/720889246 shutdown_connections 2023-10-06T05:07:59.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.840+0000 7f993719d700 1 --2- 172.21.15.55:0/720889246 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f9914066f30 0x7f99140693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.840+0000 7f993719d700 1 --2- 172.21.15.55:0/720889246 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f9930150550 0x7f99300c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.840+0000 7f993719d700 1 --2- 172.21.15.55:0/720889246 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f99300c7410 0x7f99300c2d30 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.840+0000 7f993719d700 1 --2- 172.21.15.55:0/720889246 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f9930150180 0x7f9930149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:07:59.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.840+0000 7f993719d700 1 -- 172.21.15.55:0/720889246 >> 172.21.15.55:0/720889246 conn(0x7f99300cc1f0 msgr2=0x7f99300c8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:07:59.843 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.840+0000 7f993719d700 1 -- 172.21.15.55:0/720889246 shutdown_connections 2023-10-06T05:07:59.844 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:07:59.841+0000 7f993719d700 1 -- 172.21.15.55:0/720889246 wait complete. 2023-10-06T05:07:59.844 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T05:07:59.860 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T05:07:59.860 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:08:00.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.050+0000 7f469199d700 1 Processor -- start 2023-10-06T05:08:00.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.050+0000 7f469199d700 1 -- start start 2023-10-06T05:08:00.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.052+0000 7f469199d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f468c153480 0x7f468c0ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.052+0000 7f469199d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468c0aec90 con 0x7f468c153850 2023-10-06T05:08:00.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.052+0000 7f469199d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468c15a480 con 0x7f468c159dc0 2023-10-06T05:08:00.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.052+0000 7f469199d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f468c0afb40 con 0x7f468c153480 2023-10-06T05:08:00.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.052+0000 7f468bfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f468c153850 0x7f468c1596b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:40174/0 (socket says 172.21.15.55:40174) 2023-10-06T05:08:00.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.052+0000 7f468bfff700 1 -- 172.21.15.55:0/1021011868 learned_addr learned my addr 172.21.15.55:0/1021011868 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:08:00.054 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.052+0000 7f469099b700 1 --2- 172.21.15.55:0/1021011868 >> v2:172.21.15.163:3300/0 conn(0x7f468c153480 0x7f468c0ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.052+0000 7f468b7fe700 1 -- 172.21.15.55:0/1021011868 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669487907 0 0) 0x7f468c0aec90 con 0x7f468c153850 2023-10-06T05:08:00.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.052+0000 7f468b7fe700 1 -- 172.21.15.55:0/1021011868 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f467c003570 con 0x7f468c153850 2023-10-06T05:08:00.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.052+0000 7f468b7fe700 1 -- 172.21.15.55:0/1021011868 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883142630 0 0) 0x7f468c15a480 con 0x7f468c159dc0 2023-10-06T05:08:00.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.052+0000 7f468b7fe700 1 -- 172.21.15.55:0/1021011868 --> v1:172.21.15.163:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f468c0aec90 con 0x7f468c159dc0 2023-10-06T05:08:00.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.053+0000 7f469099b700 1 -- 172.21.15.55:0/1021011868 >> v1:172.21.15.163:6789/0 conn(0x7f468c159dc0 legacy=0x7f468c157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.056 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.053+0000 7f469099b700 1 -- 172.21.15.55:0/1021011868 >> v1:172.21.15.55:6789/0 conn(0x7f468c153850 legacy=0x7f468c1596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.053+0000 7f469099b700 1 -- 172.21.15.55:0/1021011868 --> v2:172.21.15.163:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c0afe20 con 0x7f468c153480 2023-10-06T05:08:00.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.053+0000 7f469099b700 1 --2- 172.21.15.55:0/1021011868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f468c153480 0x7f468c0ae750 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f468000aa90 tx=0x7f468000a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fcb5f238279b125 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:00.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.053+0000 7f468b7fe700 1 -- 172.21.15.55:0/1021011868 <== mon.1 v2:172.21.15.163:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000c970 con 0x7f468c153480 2023-10-06T05:08:00.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.053+0000 7f468b7fe700 1 -- 172.21.15.55:0/1021011868 <== mon.1 v2:172.21.15.163:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468000cad0 con 0x7f468c153480 2023-10-06T05:08:00.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.053+0000 7f468b7fe700 1 -- 172.21.15.55:0/1021011868 <== mon.1 v2:172.21.15.163:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000ce70 con 0x7f468c153480 2023-10-06T05:08:00.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.054+0000 7f469199d700 1 -- 172.21.15.55:0/1021011868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f468c153480 msgr2=0x7f468c0ae750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.054+0000 7f469199d700 1 --2- 172.21.15.55:0/1021011868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f468c153480 0x7f468c0ae750 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f468000aa90 tx=0x7f468000a330 comp rx=0 tx=0).stop 2023-10-06T05:08:00.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.054+0000 7f469199d700 1 -- 172.21.15.55:0/1021011868 shutdown_connections 2023-10-06T05:08:00.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.054+0000 7f469199d700 1 --2- 172.21.15.55:0/1021011868 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f468c153480 0x7f468c0ae750 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.054+0000 7f469199d700 1 -- 172.21.15.55:0/1021011868 >> 172.21.15.55:0/1021011868 conn(0x7f468c068e20 msgr2=0x7f468c0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:00.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.054+0000 7f469199d700 1 -- 172.21.15.55:0/1021011868 shutdown_connections 2023-10-06T05:08:00.057 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.054+0000 7f469199d700 1 -- 172.21.15.55:0/1021011868 wait complete. 2023-10-06T05:08:00.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.055+0000 7f469199d700 1 Processor -- start 2023-10-06T05:08:00.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.055+0000 7f469199d700 1 -- start start 2023-10-06T05:08:00.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.056+0000 7f469199d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f468c153480 0x7f468c0ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.056+0000 7f469199d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f468c153850 0x7f468c0a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.056+0000 7f469199d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f468c159dc0 0x7f468c0c1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.056+0000 7f469199d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c0a8e10 con 0x7f468c153850 2023-10-06T05:08:00.062 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.056+0000 7f469199d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c0a90f0 con 0x7f468c159dc0 2023-10-06T05:08:00.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.056+0000 7f469199d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468c0a93d0 con 0x7f468c153480 2023-10-06T05:08:00.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.057+0000 7f468bfff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f468c153850 0x7f468c0a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.057+0000 7f469099b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f468c153480 0x7f468c0ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.057+0000 7f469099b700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f468c153480 0x7f468c0ac8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:45582/0 (socket says 172.21.15.55:45582) 2023-10-06T05:08:00.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.057+0000 7f469099b700 1 -- 172.21.15.55:0/4142580822 learned_addr learned my addr 172.21.15.55:0/4142580822 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:00.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.057+0000 7f469119c700 1 --2- 172.21.15.55:0/4142580822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f468c159dc0 0x7f468c0c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.057+0000 7f468bfff700 1 -- 172.21.15.55:0/4142580822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f468c153480 msgr2=0x7f468c0ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.057+0000 7f468bfff700 1 --2- 172.21.15.55:0/4142580822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f468c153480 0x7f468c0ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.057+0000 7f468bfff700 1 -- 172.21.15.55:0/4142580822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f468c159dc0 msgr2=0x7f468c0c1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.057+0000 7f468bfff700 1 --2- 172.21.15.55:0/4142580822 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f468c159dc0 0x7f468c0c1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.057+0000 7f468bfff700 1 -- 172.21.15.55:0/4142580822 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468000a5d0 con 0x7f468c153850 2023-10-06T05:08:00.063 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.057+0000 7f469099b700 1 --2- 172.21.15.55:0/4142580822 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f468c153480 0x7f468c0ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:08:00.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.057+0000 7f468bfff700 1 --2- 172.21.15.55:0/4142580822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f468c153850 0x7f468c0a84a0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f467800c580 tx=0x7f4678007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:00.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.058+0000 7f4688ff9700 1 -- 172.21.15.55:0/4142580822 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678005b30 con 0x7f468c153850 2023-10-06T05:08:00.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.058+0000 7f4688ff9700 1 -- 172.21.15.55:0/4142580822 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4678005c90 con 0x7f468c153850 2023-10-06T05:08:00.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.058+0000 7f469199d700 1 -- 172.21.15.55:0/4142580822 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468c0a96b0 con 0x7f468c153850 2023-10-06T05:08:00.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.058+0000 7f4688ff9700 1 -- 172.21.15.55:0/4142580822 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678007320 con 0x7f468c153850 2023-10-06T05:08:00.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.058+0000 7f469199d700 1 -- 172.21.15.55:0/4142580822 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c0ace30 con 0x7f468c153850 2023-10-06T05:08:00.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.058+0000 7f469199d700 1 -- 172.21.15.55:0/4142580822 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f468c15a480 con 0x7f468c153850 2023-10-06T05:08:00.064 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.060+0000 7f4688ff9700 1 -- 172.21.15.55:0/4142580822 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f467801b3b0 con 0x7f468c153850 2023-10-06T05:08:00.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.060+0000 7f4688ff9700 1 --2- 172.21.15.55:0/4142580822 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4670066f30 0x7f46700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.065 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.061+0000 7f4688ff9700 1 -- 172.21.15.55:0/4142580822 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f46780843a0 con 0x7f468c153850 2023-10-06T05:08:00.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.061+0000 7f4688ff9700 1 --2- 172.21.15.55:0/4142580822 >> [v2:172.21.15.55:6816/115462,v1:172.21.15.55:6817/115462] conn(0x7f467006dc30 0x7f4670070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.061+0000 7f4688ff9700 1 -- 172.21.15.55:0/4142580822 --> [v2:172.21.15.55:6816/115462,v1:172.21.15.55:6817/115462] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4670070680 con 0x7f467006dc30 2023-10-06T05:08:00.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.061+0000 7f4688ff9700 1 -- 172.21.15.55:0/4142580822 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=212) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4678084690 con 0x7f468c153850 2023-10-06T05:08:00.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.061+0000 7f469099b700 1 --2- 172.21.15.55:0/4142580822 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4670066f30 0x7f46700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.061+0000 7f469119c700 1 --2- 172.21.15.55:0/4142580822 >> [v2:172.21.15.55:6816/115462,v1:172.21.15.55:6817/115462] conn(0x7f467006dc30 0x7f4670070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.062+0000 7f469119c700 1 --2- 172.21.15.55:0/4142580822 >> [v2:172.21.15.55:6816/115462,v1:172.21.15.55:6817/115462] conn(0x7f467006dc30 0x7f4670070050 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:00.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.062+0000 7f469099b700 1 --2- 172.21.15.55:0/4142580822 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4670066f30 0x7f46700693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f468000a370 tx=0x7f468001a000 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:00.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.062+0000 7f4688ff9700 1 -- 172.21.15.55:0/4142580822 <== osd.2 v2:172.21.15.55:6816/115462 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f4670070680 con 0x7f467006dc30 2023-10-06T05:08:00.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.077+0000 7f469199d700 1 -- 172.21.15.55:0/4142580822 --> [v2:172.21.15.55:6816/115462,v1:172.21.15.55:6817/115462] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f468c04c3c0 con 0x7f467006dc30 2023-10-06T05:08:00.079 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.077+0000 7f4688ff9700 1 -- 172.21.15.55:0/4142580822 <== osd.2 v2:172.21.15.55:6816/115462 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f468c04c3c0 con 0x7f467006dc30 2023-10-06T05:08:00.080 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T05:08:00.080 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:08:00.080 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T05:08:00.081 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.078+0000 7f469199d700 1 -- 172.21.15.55:0/4142580822 >> [v2:172.21.15.55:6816/115462,v1:172.21.15.55:6817/115462] conn(0x7f467006dc30 msgr2=0x7f4670070050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.078+0000 7f469199d700 1 --2- 172.21.15.55:0/4142580822 >> [v2:172.21.15.55:6816/115462,v1:172.21.15.55:6817/115462] conn(0x7f467006dc30 0x7f4670070050 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.082 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.078+0000 7f469199d700 1 -- 172.21.15.55:0/4142580822 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4670066f30 msgr2=0x7f46700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.078+0000 7f469199d700 1 --2- 172.21.15.55:0/4142580822 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f4670066f30 0x7f46700693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f468000a370 tx=0x7f468001a000 comp rx=0 tx=0).stop 2023-10-06T05:08:00.083 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.078+0000 7f469199d700 1 -- 172.21.15.55:0/4142580822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f468c153850 msgr2=0x7f468c0a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.078+0000 7f469199d700 1 --2- 172.21.15.55:0/4142580822 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f468c153850 0x7f468c0a84a0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f467800c580 tx=0x7f4678007d70 comp rx=0 tx=0).stop 2023-10-06T05:08:00.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.078+0000 7f469119c700 1 -- 172.21.15.55:0/4142580822 reap_dead start 2023-10-06T05:08:00.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.078+0000 7f469199d700 1 -- 172.21.15.55:0/4142580822 shutdown_connections 2023-10-06T05:08:00.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.078+0000 7f469199d700 1 -- 172.21.15.55:0/4142580822 >> 172.21.15.55:0/4142580822 conn(0x7f468c068e20 msgr2=0x7f468c069610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:00.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.078+0000 7f469199d700 1 -- 172.21.15.55:0/4142580822 shutdown_connections 2023-10-06T05:08:00.084 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.078+0000 7f469199d700 1 -- 172.21.15.55:0/4142580822 wait complete. 2023-10-06T05:08:00.096 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T05:08:00.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.281+0000 7f3dc139d700 1 Processor -- start 2023-10-06T05:08:00.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.282+0000 7f3dc139d700 1 -- start start 2023-10-06T05:08:00.285 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.283+0000 7f3dc139d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dbc150550 0x7f3dbc157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.283+0000 7f3dc139d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dbc06e2a0 con 0x7f3dbc150180 2023-10-06T05:08:00.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.283+0000 7f3dc139d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dbc06b2e0 con 0x7f3dbc0c7410 2023-10-06T05:08:00.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.283+0000 7f3dc139d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc0c78e0 con 0x7f3dbc150550 2023-10-06T05:08:00.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.283+0000 7f3dbbfff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f3dbc150180 0x7f3dbc0c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:40176/0 (socket says 172.21.15.55:40176) 2023-10-06T05:08:00.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.283+0000 7f3dbbfff700 1 -- 172.21.15.55:0/4265173887 learned_addr learned my addr 172.21.15.55:0/4265173887 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:08:00.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.283+0000 7f3dbb7fe700 1 --2- 172.21.15.55:0/4265173887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dbc150550 0x7f3dbc157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.284+0000 7f3dbaffd700 1 -- 172.21.15.55:0/4265173887 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623064973 0 0) 0x7f3dbc06e2a0 con 0x7f3dbc150180 2023-10-06T05:08:00.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.284+0000 7f3dbaffd700 1 -- 172.21.15.55:0/4265173887 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3da4003570 con 0x7f3dbc150180 2023-10-06T05:08:00.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.284+0000 7f3dbb7fe700 1 -- 172.21.15.55:0/4265173887 >> v1:172.21.15.163:6789/0 conn(0x7f3dbc0c7410 legacy=0x7f3dbc069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.284+0000 7f3dbaffd700 1 -- 172.21.15.55:0/4265173887 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377368242 0 0) 0x7f3dbc06b2e0 con 0x7f3dbc0c7410 2023-10-06T05:08:00.286 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.284+0000 7f3dbb7fe700 1 -- 172.21.15.55:0/4265173887 >> v1:172.21.15.55:6789/0 conn(0x7f3dbc150180 legacy=0x7f3dbc0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.284+0000 7f3dbb7fe700 1 -- 172.21.15.55:0/4265173887 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc0c7bc0 con 0x7f3dbc150550 2023-10-06T05:08:00.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.284+0000 7f3dbb7fe700 1 --2- 172.21.15.55:0/4265173887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dbc150550 0x7f3dbc157770 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3db000aaf0 tx=0x7f3db000a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c200f0fe8fdb18 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:00.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.284+0000 7f3dbaffd700 1 -- 172.21.15.55:0/4265173887 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db000c930 con 0x7f3dbc150550 2023-10-06T05:08:00.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.284+0000 7f3dbaffd700 1 -- 172.21.15.55:0/4265173887 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db000cab0 con 0x7f3dbc150550 2023-10-06T05:08:00.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.284+0000 7f3dbaffd700 1 -- 172.21.15.55:0/4265173887 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db000ce70 con 0x7f3dbc150550 2023-10-06T05:08:00.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.285+0000 7f3dc139d700 1 -- 172.21.15.55:0/4265173887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dbc150550 msgr2=0x7f3dbc157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.285+0000 7f3dc139d700 1 --2- 172.21.15.55:0/4265173887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dbc150550 0x7f3dbc157770 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3db000aaf0 tx=0x7f3db000a330 comp rx=0 tx=0).stop 2023-10-06T05:08:00.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.285+0000 7f3dc139d700 1 -- 172.21.15.55:0/4265173887 shutdown_connections 2023-10-06T05:08:00.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.285+0000 7f3dc139d700 1 --2- 172.21.15.55:0/4265173887 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dbc150550 0x7f3dbc157770 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.285+0000 7f3dc139d700 1 -- 172.21.15.55:0/4265173887 >> 172.21.15.55:0/4265173887 conn(0x7f3dbc0cc1f0 msgr2=0x7f3dbc0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:00.288 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.285+0000 7f3dc139d700 1 -- 172.21.15.55:0/4265173887 shutdown_connections 2023-10-06T05:08:00.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.285+0000 7f3dc139d700 1 -- 172.21.15.55:0/4265173887 wait complete. 2023-10-06T05:08:00.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.287+0000 7f3dc139d700 1 Processor -- start 2023-10-06T05:08:00.289 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.287+0000 7f3dc139d700 1 -- start start 2023-10-06T05:08:00.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.288+0000 7f3dc139d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dbc0c7410 0x7f3dbc148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.288+0000 7f3dc139d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dbc150180 0x7f3dbc14a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.288+0000 7f3dc139d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dbc150550 0x7f3dbc14aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.288+0000 7f3dc139d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc16d7f0 con 0x7f3dbc150550 2023-10-06T05:08:00.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.288+0000 7f3dc139d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc16e370 con 0x7f3dbc0c7410 2023-10-06T05:08:00.290 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.288+0000 7f3dc139d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc149290 con 0x7f3dbc150180 2023-10-06T05:08:00.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.288+0000 7f3dc0b9c700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dbc150550 0x7f3dbc14aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.291 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.288+0000 7f3dbbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dbc0c7410 0x7f3dbc148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.288+0000 7f3dbb7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dbc150180 0x7f3dbc14a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.288+0000 7f3dbb7fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dbc150180 0x7f3dbc14a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3301/0 says I am v2:172.21.15.55:45612/0 (socket says 172.21.15.55:45612) 2023-10-06T05:08:00.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.288+0000 7f3dbb7fe700 1 -- 172.21.15.55:0/2227858858 learned_addr learned my addr 172.21.15.55:0/2227858858 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:00.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.288+0000 7f3dbbfff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dbc0c7410 0x7f3dbc148d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.163:3300/0 says I am v2:172.21.15.55:46048/0 (socket says 172.21.15.55:46048) 2023-10-06T05:08:00.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.288+0000 7f3dbb7fe700 1 -- 172.21.15.55:0/2227858858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dbc0c7410 msgr2=0x7f3dbc148d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.288+0000 7f3dbb7fe700 1 --2- 172.21.15.55:0/2227858858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dbc0c7410 0x7f3dbc148d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.289+0000 7f3dbb7fe700 1 -- 172.21.15.55:0/2227858858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dbc150550 msgr2=0x7f3dbc14aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.289+0000 7f3dbb7fe700 1 --2- 172.21.15.55:0/2227858858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dbc150550 0x7f3dbc14aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.289+0000 7f3dbb7fe700 1 -- 172.21.15.55:0/2227858858 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db000a5d0 con 0x7f3dbc150180 2023-10-06T05:08:00.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.289+0000 7f3dc0b9c700 1 --2- 172.21.15.55:0/2227858858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dbc150550 0x7f3dbc14aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:08:00.292 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.289+0000 7f3dbb7fe700 1 --2- 172.21.15.55:0/2227858858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dbc150180 0x7f3dbc14a5b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3db000aac0 tx=0x7f3db001a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:00.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.289+0000 7f3da3fff700 1 -- 172.21.15.55:0/2227858858 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0023410 con 0x7f3dbc150180 2023-10-06T05:08:00.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.289+0000 7f3da3fff700 1 -- 172.21.15.55:0/2227858858 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db00235b0 con 0x7f3dbc150180 2023-10-06T05:08:00.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.289+0000 7f3da3fff700 1 -- 172.21.15.55:0/2227858858 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db0023970 con 0x7f3dbc150180 2023-10-06T05:08:00.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.289+0000 7f3dc139d700 1 -- 172.21.15.55:0/2227858858 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dbc0ab290 con 0x7f3dbc150180 2023-10-06T05:08:00.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.290+0000 7f3dc139d700 1 -- 172.21.15.55:0/2227858858 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dbc16dd20 con 0x7f3dbc150180 2023-10-06T05:08:00.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.291+0000 7f3dc139d700 1 -- 172.21.15.55:0/2227858858 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dbc16e4e0 con 0x7f3dbc150180 2023-10-06T05:08:00.293 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.292+0000 7f3da3fff700 1 -- 172.21.15.55:0/2227858858 <== mon.2 v2:172.21.15.163:3301/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3db0023ce0 con 0x7f3dbc150180 2023-10-06T05:08:00.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.292+0000 7f3da3fff700 1 --2- 172.21.15.55:0/2227858858 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3da8067000 0x7f3da80694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.293+0000 7f3da3fff700 1 -- 172.21.15.55:0/2227858858 <== mon.2 v2:172.21.15.163:3301/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f3db00862d0 con 0x7f3dbc150180 2023-10-06T05:08:00.294 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.293+0000 7f3dbbfff700 1 --2- 172.21.15.55:0/2227858858 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3da8067000 0x7f3da80694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.295 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.293+0000 7f3dbbfff700 1 --2- 172.21.15.55:0/2227858858 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3da8067000 0x7f3da80694b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3dac007fd0 tx=0x7f3dac009040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:00.298 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.296+0000 7f3da3fff700 1 -- 172.21.15.55:0/2227858858 <== mon.2 v2:172.21.15.163:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3db0053400 con 0x7f3dbc150180 2023-10-06T05:08:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.499+0000 7f3dc139d700 1 -- 172.21.15.55:0/2227858858 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3dbc04c3c0 con 0x7f3da8067000 2023-10-06T05:08:00.501 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.499+0000 7f3da3fff700 1 -- 172.21.15.55:0/2227858858 <== mgr.10033 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3dbc04c3c0 con 0x7f3da8067000 2023-10-06T05:08:00.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.501+0000 7f3dc139d700 1 -- 172.21.15.55:0/2227858858 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3da8067000 msgr2=0x7f3da80694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.501+0000 7f3dc139d700 1 --2- 172.21.15.55:0/2227858858 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3da8067000 0x7f3da80694b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3dac007fd0 tx=0x7f3dac009040 comp rx=0 tx=0).stop 2023-10-06T05:08:00.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.501+0000 7f3dc139d700 1 -- 172.21.15.55:0/2227858858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dbc150180 msgr2=0x7f3dbc14a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.501+0000 7f3dc139d700 1 --2- 172.21.15.55:0/2227858858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dbc150180 0x7f3dbc14a5b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3db000aac0 tx=0x7f3db001a000 comp rx=0 tx=0).stop 2023-10-06T05:08:00.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.502+0000 7f3dc139d700 1 -- 172.21.15.55:0/2227858858 shutdown_connections 2023-10-06T05:08:00.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.502+0000 7f3dc139d700 1 --2- 172.21.15.55:0/2227858858 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f3da8067000 0x7f3da80694b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.504 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.502+0000 7f3dc139d700 1 --2- 172.21.15.55:0/2227858858 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f3dbc150550 0x7f3dbc14aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.502+0000 7f3dc139d700 1 --2- 172.21.15.55:0/2227858858 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f3dbc0c7410 0x7f3dbc148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.502+0000 7f3dc139d700 1 --2- 172.21.15.55:0/2227858858 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f3dbc150180 0x7f3dbc14a5b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.502+0000 7f3dc139d700 1 -- 172.21.15.55:0/2227858858 >> 172.21.15.55:0/2227858858 conn(0x7f3dbc0cc1f0 msgr2=0x7f3dbc1597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.502+0000 7f3dc139d700 1 -- 172.21.15.55:0/2227858858 shutdown_connections 2023-10-06T05:08:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.502+0000 7f3dc139d700 1 -- 172.21.15.55:0/2227858858 wait complete. 2023-10-06T05:08:00.505 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T05:08:00.521 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T05:08:00.521 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:08:00.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.711+0000 7fe8f49fe700 1 Processor -- start 2023-10-06T05:08:00.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.711+0000 7fe8f49fe700 1 -- start start 2023-10-06T05:08:00.714 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.712+0000 7fe8f49fe700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8f00afa40 0x7fe8f01647e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.712+0000 7fe8f49fe700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f00afe60 con 0x7fe8f00ac0e0 2023-10-06T05:08:00.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.712+0000 7fe8f49fe700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f006e510 con 0x7fe8f01597d0 2023-10-06T05:08:00.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.712+0000 7fe8f49fe700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f015e450 con 0x7fe8f00afa40 2023-10-06T05:08:00.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.713+0000 7fe8ef59e700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fe8f00afa40 0x7fe8f01647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.713+0000 7fe8ee59c700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fe8f00ac0e0 0x7fe8f006f0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:40178/0 (socket says 172.21.15.55:40178) 2023-10-06T05:08:00.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.713+0000 7fe8ee59c700 1 -- 172.21.15.55:0/283892012 learned_addr learned my addr 172.21.15.55:0/283892012 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:08:00.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.713+0000 7fe8edd9b700 1 -- 172.21.15.55:0/283892012 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3707441361 0 0) 0x7fe8f00afe60 con 0x7fe8f00ac0e0 2023-10-06T05:08:00.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.713+0000 7fe8edd9b700 1 -- 172.21.15.55:0/283892012 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8d8003570 con 0x7fe8f00ac0e0 2023-10-06T05:08:00.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.713+0000 7fe8ef59e700 1 -- 172.21.15.55:0/283892012 >> v1:172.21.15.163:6789/0 conn(0x7fe8f01597d0 legacy=0x7fe8f006e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.713+0000 7fe8edd9b700 1 -- 172.21.15.55:0/283892012 <== mon.1 v1:172.21.15.163:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778811613 0 0) 0x7fe8f006e510 con 0x7fe8f01597d0 2023-10-06T05:08:00.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.713+0000 7fe8ef59e700 1 -- 172.21.15.55:0/283892012 >> v1:172.21.15.55:6789/0 conn(0x7fe8f00ac0e0 legacy=0x7fe8f006f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.713+0000 7fe8ef59e700 1 -- 172.21.15.55:0/283892012 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f0069180 con 0x7fe8f00afa40 2023-10-06T05:08:00.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.714+0000 7fe8ef59e700 1 --2- 172.21.15.55:0/283892012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8f00afa40 0x7fe8f01647e0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009e60 tx=0x7fe8e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8cc06dbd2bbdf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:00.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.714+0000 7fe8edd9b700 1 -- 172.21.15.55:0/283892012 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800e070 con 0x7fe8f00afa40 2023-10-06T05:08:00.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.714+0000 7fe8edd9b700 1 -- 172.21.15.55:0/283892012 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e800bb60 con 0x7fe8f00afa40 2023-10-06T05:08:00.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.714+0000 7fe8edd9b700 1 -- 172.21.15.55:0/283892012 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8012430 con 0x7fe8f00afa40 2023-10-06T05:08:00.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.714+0000 7fe8f49fe700 1 -- 172.21.15.55:0/283892012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8f00afa40 msgr2=0x7fe8f01647e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.714+0000 7fe8f49fe700 1 --2- 172.21.15.55:0/283892012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8f00afa40 0x7fe8f01647e0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009e60 tx=0x7fe8e8009780 comp rx=0 tx=0).stop 2023-10-06T05:08:00.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.714+0000 7fe8f49fe700 1 -- 172.21.15.55:0/283892012 shutdown_connections 2023-10-06T05:08:00.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.714+0000 7fe8f49fe700 1 --2- 172.21.15.55:0/283892012 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8f00afa40 0x7fe8f01647e0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.714+0000 7fe8f49fe700 1 -- 172.21.15.55:0/283892012 >> 172.21.15.55:0/283892012 conn(0x7fe8f006adf0 msgr2=0x7fe8f00ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:00.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.715+0000 7fe8f49fe700 1 -- 172.21.15.55:0/283892012 shutdown_connections 2023-10-06T05:08:00.718 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.715+0000 7fe8f49fe700 1 -- 172.21.15.55:0/283892012 wait complete. 2023-10-06T05:08:00.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.716+0000 7fe8f49fe700 1 Processor -- start 2023-10-06T05:08:00.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.716+0000 7fe8f49fe700 1 -- start start 2023-10-06T05:08:00.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.717+0000 7fe8f49fe700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f00afa40 0x7fe8f00c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.720 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.717+0000 7fe8f49fe700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8f01597d0 0x7fe8f01690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.717+0000 7fe8f49fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8f015d090 0x7fe8f016bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.717+0000 7fe8f49fe700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00c1f00 con 0x7fe8f015d090 2023-10-06T05:08:00.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.717+0000 7fe8f49fe700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00c2130 con 0x7fe8f01597d0 2023-10-06T05:08:00.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.717+0000 7fe8f49fe700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00c2410 con 0x7fe8f00afa40 2023-10-06T05:08:00.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.717+0000 7fe8ef59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8f015d090 0x7fe8f016bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.717+0000 7fe8ef59e700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8f015d090 0x7fe8f016bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54752/0 (socket says 172.21.15.55:54752) 2023-10-06T05:08:00.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.718+0000 7fe8ef59e700 1 -- 172.21.15.55:0/3569599529 learned_addr learned my addr 172.21.15.55:0/3569599529 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:00.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.717+0000 7fe8eed9d700 1 --2- 172.21.15.55:0/3569599529 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f00afa40 0x7fe8f00c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.718+0000 7fe8ee59c700 1 --2- 172.21.15.55:0/3569599529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8f01597d0 0x7fe8f01690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.718+0000 7fe8ef59e700 1 -- 172.21.15.55:0/3569599529 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f00afa40 msgr2=0x7fe8f00c19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.721 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.718+0000 7fe8ef59e700 1 --2- 172.21.15.55:0/3569599529 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fe8f00afa40 0x7fe8f00c19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.718+0000 7fe8ef59e700 1 -- 172.21.15.55:0/3569599529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8f01597d0 msgr2=0x7fe8f01690f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.718+0000 7fe8ef59e700 1 --2- 172.21.15.55:0/3569599529 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fe8f01597d0 0x7fe8f01690f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.718+0000 7fe8ef59e700 1 -- 172.21.15.55:0/3569599529 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e8009a20 con 0x7fe8f015d090 2023-10-06T05:08:00.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.718+0000 7fe8ef59e700 1 --2- 172.21.15.55:0/3569599529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8f015d090 0x7fe8f016bdf0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fe8f006dad0 tx=0x7fe8e8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:00.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.718+0000 7fe8d6ffd700 1 -- 172.21.15.55:0/3569599529 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8026480 con 0x7fe8f015d090 2023-10-06T05:08:00.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.718+0000 7fe8d6ffd700 1 -- 172.21.15.55:0/3569599529 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e8026640 con 0x7fe8f015d090 2023-10-06T05:08:00.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.718+0000 7fe8d6ffd700 1 -- 172.21.15.55:0/3569599529 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8026a00 con 0x7fe8f015d090 2023-10-06T05:08:00.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.718+0000 7fe8f49fe700 1 -- 172.21.15.55:0/3569599529 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f00c26f0 con 0x7fe8f015d090 2023-10-06T05:08:00.722 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.719+0000 7fe8f49fe700 1 -- 172.21.15.55:0/3569599529 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f00c2cb0 con 0x7fe8f015d090 2023-10-06T05:08:00.723 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.719+0000 7fe8f49fe700 1 -- 172.21.15.55:0/3569599529 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe8f004c3c0 con 0x7fe8f015d090 2023-10-06T05:08:00.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.720+0000 7fe8d6ffd700 1 -- 172.21.15.55:0/3569599529 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8e801c400 con 0x7fe8f015d090 2023-10-06T05:08:00.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.721+0000 7fe8d6ffd700 1 --2- 172.21.15.55:0/3569599529 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe8dc066f30 0x7fe8dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.721+0000 7fe8d6ffd700 1 -- 172.21.15.55:0/3569599529 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7fe8e8086d30 con 0x7fe8f015d090 2023-10-06T05:08:00.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.722+0000 7fe8d6ffd700 1 --2- 172.21.15.55:0/3569599529 >> [v2:172.21.15.55:6824/115464,v1:172.21.15.55:6825/115464] conn(0x7fe8dc06dc30 0x7fe8dc070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.722+0000 7fe8d6ffd700 1 -- 172.21.15.55:0/3569599529 --> [v2:172.21.15.55:6824/115464,v1:172.21.15.55:6825/115464] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8dc070680 con 0x7fe8dc06dc30 2023-10-06T05:08:00.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.722+0000 7fe8eed9d700 1 --2- 172.21.15.55:0/3569599529 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe8dc066f30 0x7fe8dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.722+0000 7fe8d6ffd700 1 -- 172.21.15.55:0/3569599529 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=212) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8e8087020 con 0x7fe8f015d090 2023-10-06T05:08:00.724 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.722+0000 7fe8ee59c700 1 --2- 172.21.15.55:0/3569599529 >> [v2:172.21.15.55:6824/115464,v1:172.21.15.55:6825/115464] conn(0x7fe8dc06dc30 0x7fe8dc070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.722+0000 7fe8eed9d700 1 --2- 172.21.15.55:0/3569599529 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe8dc066f30 0x7fe8dc0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0007fd0 tx=0x7fe8e0007d30 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:00.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.722+0000 7fe8ee59c700 1 --2- 172.21.15.55:0/3569599529 >> [v2:172.21.15.55:6824/115464,v1:172.21.15.55:6825/115464] conn(0x7fe8dc06dc30 0x7fe8dc070050 crc :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:00.725 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.723+0000 7fe8d6ffd700 1 -- 172.21.15.55:0/3569599529 <== osd.3 v2:172.21.15.55:6824/115464 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe8dc070680 con 0x7fe8dc06dc30 2023-10-06T05:08:00.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.738+0000 7fe8f49fe700 1 -- 172.21.15.55:0/3569599529 --> [v2:172.21.15.55:6824/115464,v1:172.21.15.55:6825/115464] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe8f00cd4a0 con 0x7fe8dc06dc30 2023-10-06T05:08:00.740 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.738+0000 7fe8d6ffd700 1 -- 172.21.15.55:0/3569599529 <== osd.3 v2:172.21.15.55:6824/115464 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe8f00cd4a0 con 0x7fe8dc06dc30 2023-10-06T05:08:00.740 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T05:08:00.740 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:08:00.741 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T05:08:00.742 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.738+0000 7fe8f49fe700 1 -- 172.21.15.55:0/3569599529 >> [v2:172.21.15.55:6824/115464,v1:172.21.15.55:6825/115464] conn(0x7fe8dc06dc30 msgr2=0x7fe8dc070050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.738+0000 7fe8f49fe700 1 --2- 172.21.15.55:0/3569599529 >> [v2:172.21.15.55:6824/115464,v1:172.21.15.55:6825/115464] conn(0x7fe8dc06dc30 0x7fe8dc070050 crc :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.739+0000 7fe8f49fe700 1 -- 172.21.15.55:0/3569599529 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe8dc066f30 msgr2=0x7fe8dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.739+0000 7fe8f49fe700 1 --2- 172.21.15.55:0/3569599529 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fe8dc066f30 0x7fe8dc0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0007fd0 tx=0x7fe8e0007d30 comp rx=0 tx=0).stop 2023-10-06T05:08:00.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.739+0000 7fe8f49fe700 1 -- 172.21.15.55:0/3569599529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8f015d090 msgr2=0x7fe8f016bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.739+0000 7fe8f49fe700 1 --2- 172.21.15.55:0/3569599529 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fe8f015d090 0x7fe8f016bdf0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fe8f006dad0 tx=0x7fe8e8019000 comp rx=0 tx=0).stop 2023-10-06T05:08:00.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.739+0000 7fe8ef59e700 1 -- 172.21.15.55:0/3569599529 reap_dead start 2023-10-06T05:08:00.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.739+0000 7fe8f49fe700 1 -- 172.21.15.55:0/3569599529 shutdown_connections 2023-10-06T05:08:00.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.739+0000 7fe8f49fe700 1 -- 172.21.15.55:0/3569599529 >> 172.21.15.55:0/3569599529 conn(0x7fe8f006adf0 msgr2=0x7fe8f006e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:00.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.739+0000 7fe8f49fe700 1 -- 172.21.15.55:0/3569599529 shutdown_connections 2023-10-06T05:08:00.743 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.739+0000 7fe8f49fe700 1 -- 172.21.15.55:0/3569599529 wait complete. 2023-10-06T05:08:00.757 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T05:08:00.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.945+0000 7ff67519d700 1 Processor -- start 2023-10-06T05:08:00.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.945+0000 7ff67519d700 1 -- start start 2023-10-06T05:08:00.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.946+0000 7ff67519d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff67006c0f0 0x7ff6701642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.946+0000 7ff67519d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6700b40f0 con 0x7ff670142ea0 2023-10-06T05:08:00.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.946+0000 7ff67519d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff670069640 con 0x7ff6700ad870 2023-10-06T05:08:00.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.946+0000 7ff67519d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff670069920 con 0x7ff67006c0f0 2023-10-06T05:08:00.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.946+0000 7ff66ffff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7ff670142ea0 0x7ff6700b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:40190/0 (socket says 172.21.15.55:40190) 2023-10-06T05:08:00.948 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.946+0000 7ff66ffff700 1 -- 172.21.15.55:0/2029369457 learned_addr learned my addr 172.21.15.55:0/2029369457 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:08:00.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 -- 172.21.15.55:0/2029369457 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812987523 0 0) 0x7ff6700b40f0 con 0x7ff670142ea0 2023-10-06T05:08:00.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff67499c700 1 --2- 172.21.15.55:0/2029369457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff67006c0f0 0x7ff6701642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.951 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 -- 172.21.15.55:0/2029369457 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff658003570 con 0x7ff670142ea0 2023-10-06T05:08:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 -- 172.21.15.55:0/2029369457 <== mon.0 v1:172.21.15.55:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 524487453 0 0) 0x7ff658003570 con 0x7ff670142ea0 2023-10-06T05:08:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 -- 172.21.15.55:0/2029369457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff67006c0f0 msgr2=0x7ff6701642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 --2- 172.21.15.55:0/2029369457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff67006c0f0 0x7ff6701642e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 -- 172.21.15.55:0/2029369457 >> v1:172.21.15.163:6789/0 conn(0x7ff6700ad870 legacy=0x7ff67006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 -- 172.21.15.55:0/2029369457 --> v1:172.21.15.55:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff670069c00 con 0x7ff670142ea0 2023-10-06T05:08:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 -- 172.21.15.55:0/2029369457 <== mon.0 v1:172.21.15.55:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 752228847 0 0) 0x7ff660001470 con 0x7ff670142ea0 2023-10-06T05:08:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 -- 172.21.15.55:0/2029369457 >> v1:172.21.15.55:6789/0 conn(0x7ff670142ea0 legacy=0x7ff6700b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 --2- 172.21.15.55:0/2029369457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff658003a90 0x7ff658003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 --2- 172.21.15.55:0/2029369457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff658004810 0x7ff658004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 --2- 172.21.15.55:0/2029369457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff67006c0f0 0x7ff6580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.952 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 -- 172.21.15.55:0/2029369457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff670069920 con 0x7ff658003a90 2023-10-06T05:08:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 -- 172.21.15.55:0/2029369457 --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65800bb70 con 0x7ff658004810 2023-10-06T05:08:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.947+0000 7ff66effd700 1 -- 172.21.15.55:0/2029369457 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65800bd30 con 0x7ff67006c0f0 2023-10-06T05:08:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.948+0000 7ff66ffff700 1 --2- 172.21.15.55:0/2029369457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff658003a90 0x7ff658003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.948+0000 7ff66f7fe700 1 --2- 172.21.15.55:0/2029369457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff658004810 0x7ff658004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.948+0000 7ff67499c700 1 --2- 172.21.15.55:0/2029369457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff67006c0f0 0x7ff6580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.948+0000 7ff66ffff700 1 -- 172.21.15.55:0/2029369457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff67006c0f0 msgr2=0x7ff6580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.948+0000 7ff66ffff700 1 --2- 172.21.15.55:0/2029369457 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff67006c0f0 0x7ff6580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.948+0000 7ff66ffff700 1 -- 172.21.15.55:0/2029369457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff658004810 msgr2=0x7ff658004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.948+0000 7ff66ffff700 1 --2- 172.21.15.55:0/2029369457 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff658004810 0x7ff658004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.948+0000 7ff66ffff700 1 -- 172.21.15.55:0/2029369457 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff658003570 con 0x7ff658003a90 2023-10-06T05:08:00.953 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.948+0000 7ff66ffff700 1 --2- 172.21.15.55:0/2029369457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff658003a90 0x7ff658003e60 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7ff66000c2d0 tx=0x7ff660001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b87735d856e3be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:00.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.948+0000 7ff66effd700 1 -- 172.21.15.55:0/2029369457 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66000cba0 con 0x7ff658003a90 2023-10-06T05:08:00.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.949+0000 7ff66effd700 1 -- 172.21.15.55:0/2029369457 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66000cd20 con 0x7ff658003a90 2023-10-06T05:08:00.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.949+0000 7ff66effd700 1 -- 172.21.15.55:0/2029369457 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660005b70 con 0x7ff658003a90 2023-10-06T05:08:00.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.949+0000 7ff67519d700 1 -- 172.21.15.55:0/2029369457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff658003a90 msgr2=0x7ff658003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.949+0000 7ff67519d700 1 --2- 172.21.15.55:0/2029369457 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff658003a90 0x7ff658003e60 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7ff66000c2d0 tx=0x7ff660001da0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.949+0000 7ff67499c700 1 -- 172.21.15.55:0/2029369457 reap_dead start 2023-10-06T05:08:00.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.949+0000 7ff67519d700 1 -- 172.21.15.55:0/2029369457 shutdown_connections 2023-10-06T05:08:00.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.949+0000 7ff67519d700 1 -- 172.21.15.55:0/2029369457 >> 172.21.15.55:0/2029369457 conn(0x7ff6700ce430 msgr2=0x7ff6700ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:00.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.949+0000 7ff67519d700 1 -- 172.21.15.55:0/2029369457 shutdown_connections 2023-10-06T05:08:00.954 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.949+0000 7ff67519d700 1 -- 172.21.15.55:0/2029369457 wait complete. 2023-10-06T05:08:00.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.951+0000 7ff67519d700 1 Processor -- start 2023-10-06T05:08:00.956 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.951+0000 7ff67519d700 1 -- start start 2023-10-06T05:08:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff67519d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff658003a90 0x7ff67016dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff67519d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff67015ca90 0x7ff6700c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff67519d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff67013eca0 0x7ff67015f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff67519d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700c86b0 con 0x7ff658003a90 2023-10-06T05:08:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff67519d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700c9220 con 0x7ff67013eca0 2023-10-06T05:08:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff67519d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67015cf00 con 0x7ff67015ca90 2023-10-06T05:08:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff66ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff658003a90 0x7ff67016dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff66ffff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff658003a90 0x7ff67016dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54774/0 (socket says 172.21.15.55:54774) 2023-10-06T05:08:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff66ffff700 1 -- 172.21.15.55:0/3979698256 learned_addr learned my addr 172.21.15.55:0/3979698256 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff66f7fe700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff67015ca90 0x7ff6700c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.957 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff67499c700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff67013eca0 0x7ff67015f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff66ffff700 1 -- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff67015ca90 msgr2=0x7ff6700c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff66ffff700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff67015ca90 0x7ff6700c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff66ffff700 1 -- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff67013eca0 msgr2=0x7ff67015f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff66ffff700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff67013eca0 0x7ff67015f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff66ffff700 1 -- 172.21.15.55:0/3979698256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6700c7e10 con 0x7ff658003a90 2023-10-06T05:08:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.952+0000 7ff67499c700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff67013eca0 0x7ff67015f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:08:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.953+0000 7ff66ffff700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff658003a90 0x7ff67016dd60 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7ff65800bb40 tx=0x7ff660015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.953+0000 7ff657fff700 1 -- 172.21.15.55:0/3979698256 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66000cba0 con 0x7ff658003a90 2023-10-06T05:08:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.953+0000 7ff657fff700 1 -- 172.21.15.55:0/3979698256 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66000ce90 con 0x7ff658003a90 2023-10-06T05:08:00.958 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.953+0000 7ff67519d700 1 -- 172.21.15.55:0/3979698256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6700c80f0 con 0x7ff658003a90 2023-10-06T05:08:00.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.953+0000 7ff657fff700 1 -- 172.21.15.55:0/3979698256 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660007510 con 0x7ff658003a90 2023-10-06T05:08:00.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.953+0000 7ff67519d700 1 -- 172.21.15.55:0/3979698256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6700c7b20 con 0x7ff658003a90 2023-10-06T05:08:00.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.954+0000 7ff67519d700 1 -- 172.21.15.55:0/3979698256 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6700c8ff0 con 0x7ff658003a90 2023-10-06T05:08:00.959 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.955+0000 7ff657fff700 1 -- 172.21.15.55:0/3979698256 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff660019030 con 0x7ff658003a90 2023-10-06T05:08:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.956+0000 7ff657fff700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff65c066f30 0x7ff65c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.956+0000 7ff657fff700 1 -- 172.21.15.55:0/3979698256 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7ff6600859b0 con 0x7ff658003a90 2023-10-06T05:08:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.956+0000 7ff66f7fe700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff65c066f30 0x7ff65c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:00.960 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.957+0000 7ff66f7fe700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff65c066f30 0x7ff65c0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff6640010e0 tx=0x7ff664007d10 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:00.962 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:00.960+0000 7ff657fff700 1 -- 172.21.15.55:0/3979698256 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff6700c8ff0 con 0x7ff658003a90 2023-10-06T05:08:01.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.161+0000 7ff67519d700 1 -- 172.21.15.55:0/3979698256 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6700c8ff0 con 0x7ff65c066f30 2023-10-06T05:08:01.163 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.162+0000 7ff657fff700 1 -- 172.21.15.55:0/3979698256 <== mgr.10033 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff6700c8ff0 con 0x7ff65c066f30 2023-10-06T05:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.164+0000 7ff67519d700 1 -- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff65c066f30 msgr2=0x7ff65c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.164+0000 7ff67519d700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff65c066f30 0x7ff65c0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff6640010e0 tx=0x7ff664007d10 comp rx=0 tx=0).stop 2023-10-06T05:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.164+0000 7ff67519d700 1 -- 172.21.15.55:0/3979698256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff658003a90 msgr2=0x7ff67016dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.164+0000 7ff67519d700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff658003a90 0x7ff67016dd60 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7ff65800bb40 tx=0x7ff660015000 comp rx=0 tx=0).stop 2023-10-06T05:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.164+0000 7ff67519d700 1 -- 172.21.15.55:0/3979698256 shutdown_connections 2023-10-06T05:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.164+0000 7ff67519d700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7ff65c066f30 0x7ff65c0693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.164+0000 7ff67519d700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7ff67013eca0 0x7ff67015f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.166 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.164+0000 7ff67519d700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7ff658003a90 0x7ff67016dd60 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.164+0000 7ff67519d700 1 --2- 172.21.15.55:0/3979698256 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7ff67015ca90 0x7ff6700c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.164+0000 7ff67519d700 1 -- 172.21.15.55:0/3979698256 >> 172.21.15.55:0/3979698256 conn(0x7ff6700ce430 msgr2=0x7ff6700cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.164+0000 7ff67519d700 1 -- 172.21.15.55:0/3979698256 shutdown_connections 2023-10-06T05:08:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.164+0000 7ff67519d700 1 -- 172.21.15.55:0/3979698256 wait complete. 2023-10-06T05:08:01.167 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T05:08:01.183 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T05:08:01.183 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:08:01.376 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.374+0000 7f1b3919d700 1 Processor -- start 2023-10-06T05:08:01.377 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.375+0000 7f1b3919d700 1 -- start start 2023-10-06T05:08:01.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.376+0000 7f1b3919d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f1b340b0c00 0x7f1b340ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:01.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.376+0000 7f1b3919d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b340b0570 con 0x7f1b34156350 2023-10-06T05:08:01.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.376+0000 7f1b3919d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b340af4d0 con 0x7f1b340b0760 2023-10-06T05:08:01.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.376+0000 7f1b3919d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b340af7b0 con 0x7f1b340b0c00 2023-10-06T05:08:01.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.376+0000 7f1b33fff700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f1b34156350 0x7f1b34157370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:40206/0 (socket says 172.21.15.55:40206) 2023-10-06T05:08:01.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.376+0000 7f1b33fff700 1 -- 172.21.15.55:0/2317119738 learned_addr learned my addr 172.21.15.55:0/2317119738 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:08:01.378 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.376+0000 7f1b3899c700 1 --2- 172.21.15.55:0/2317119738 >> v2:172.21.15.55:3300/0 conn(0x7f1b340b0c00 0x7f1b340ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:01.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.377+0000 7f1b32ffd700 1 -- 172.21.15.55:0/2317119738 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3068437146 0 0) 0x7f1b340b0570 con 0x7f1b34156350 2023-10-06T05:08:01.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.377+0000 7f1b32ffd700 1 -- 172.21.15.55:0/2317119738 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b1c003570 con 0x7f1b34156350 2023-10-06T05:08:01.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.377+0000 7f1b3899c700 1 -- 172.21.15.55:0/2317119738 >> v1:172.21.15.163:6789/0 conn(0x7f1b340b0760 legacy=0x7f1b340ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:01.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.377+0000 7f1b3899c700 1 -- 172.21.15.55:0/2317119738 >> v1:172.21.15.55:6789/0 conn(0x7f1b34156350 legacy=0x7f1b34157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.380 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.377+0000 7f1b3899c700 1 -- 172.21.15.55:0/2317119738 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b340afa90 con 0x7f1b340b0c00 2023-10-06T05:08:01.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.377+0000 7f1b3899c700 1 --2- 172.21.15.55:0/2317119738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b340b0c00 0x7f1b340ad570 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009e60 tx=0x7f1b2c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95ec4f1942f4da4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:01.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.377+0000 7f1b32ffd700 1 -- 172.21.15.55:0/2317119738 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00e070 con 0x7f1b340b0c00 2023-10-06T05:08:01.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.377+0000 7f1b32ffd700 1 -- 172.21.15.55:0/2317119738 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2c00bb20 con 0x7f1b340b0c00 2023-10-06T05:08:01.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.377+0000 7f1b32ffd700 1 -- 172.21.15.55:0/2317119738 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c012430 con 0x7f1b340b0c00 2023-10-06T05:08:01.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.378+0000 7f1b3919d700 1 -- 172.21.15.55:0/2317119738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b340b0c00 msgr2=0x7f1b340ad570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.378+0000 7f1b3919d700 1 --2- 172.21.15.55:0/2317119738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b340b0c00 0x7f1b340ad570 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009e60 tx=0x7f1b2c009780 comp rx=0 tx=0).stop 2023-10-06T05:08:01.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.378+0000 7f1b3919d700 1 -- 172.21.15.55:0/2317119738 shutdown_connections 2023-10-06T05:08:01.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.378+0000 7f1b3919d700 1 --2- 172.21.15.55:0/2317119738 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b340b0c00 0x7f1b340ad570 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.378+0000 7f1b3919d700 1 -- 172.21.15.55:0/2317119738 >> 172.21.15.55:0/2317119738 conn(0x7f1b34068e20 msgr2=0x7f1b340cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:01.381 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.378+0000 7f1b3919d700 1 -- 172.21.15.55:0/2317119738 shutdown_connections 2023-10-06T05:08:01.382 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.378+0000 7f1b3919d700 1 -- 172.21.15.55:0/2317119738 wait complete. 2023-10-06T05:08:01.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.379+0000 7f1b3919d700 1 Processor -- start 2023-10-06T05:08:01.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.380+0000 7f1b3919d700 1 -- start start 2023-10-06T05:08:01.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.380+0000 7f1b3919d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b340b0760 0x7f1b340aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:01.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.380+0000 7f1b3919d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b340b0c00 0x7f1b340a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:01.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b3919d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b34156350 0x7f1b340a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:01.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b3919d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b340a5f40 con 0x7f1b340b0760 2023-10-06T05:08:01.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b3919d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3406ee30 con 0x7f1b340b0c00 2023-10-06T05:08:01.384 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b3919d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b340c9f80 con 0x7f1b34156350 2023-10-06T05:08:01.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b33fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b340b0760 0x7f1b340aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:01.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b33fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b340b0760 0x7f1b340aa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54784/0 (socket says 172.21.15.55:54784) 2023-10-06T05:08:01.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b33fff700 1 -- 172.21.15.55:0/1326042090 learned_addr learned my addr 172.21.15.55:0/1326042090 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:01.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b3899c700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b34156350 0x7f1b340a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:01.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b33fff700 1 -- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b34156350 msgr2=0x7f1b340a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b337fe700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b340b0c00 0x7f1b340a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:01.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b33fff700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b34156350 0x7f1b340a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b33fff700 1 -- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b340b0c00 msgr2=0x7f1b340a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b33fff700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b340b0c00 0x7f1b340a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b33fff700 1 -- 172.21.15.55:0/1326042090 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2c009a20 con 0x7f1b340b0760 2023-10-06T05:08:01.385 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b3899c700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f1b34156350 0x7f1b340a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:08:01.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.381+0000 7f1b337fe700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f1b340b0c00 0x7f1b340a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:08:01.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.382+0000 7f1b33fff700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b340b0760 0x7f1b340aa8e0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400c7c0 tx=0x7f1b24007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:01.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.382+0000 7f1b1bfff700 1 -- 172.21.15.55:0/1326042090 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b24005b30 con 0x7f1b340b0760 2023-10-06T05:08:01.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.382+0000 7f1b1bfff700 1 -- 172.21.15.55:0/1326042090 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b24005c90 con 0x7f1b340b0760 2023-10-06T05:08:01.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.382+0000 7f1b1bfff700 1 -- 172.21.15.55:0/1326042090 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b24007340 con 0x7f1b340b0760 2023-10-06T05:08:01.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.382+0000 7f1b3919d700 1 -- 172.21.15.55:0/1326042090 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b340c9650 con 0x7f1b340b0760 2023-10-06T05:08:01.386 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.382+0000 7f1b3919d700 1 -- 172.21.15.55:0/1326042090 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b340c9c10 con 0x7f1b340b0760 2023-10-06T05:08:01.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.383+0000 7f1b3919d700 1 -- 172.21.15.55:0/1326042090 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1b3404c3c0 con 0x7f1b340b0760 2023-10-06T05:08:01.387 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.384+0000 7f1b1bfff700 1 -- 172.21.15.55:0/1326042090 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b2401b400 con 0x7f1b340b0760 2023-10-06T05:08:01.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.385+0000 7f1b1bfff700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1b20066f30 0x7f1b200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:01.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.385+0000 7f1b1bfff700 1 -- 172.21.15.55:0/1326042090 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f1b24084330 con 0x7f1b340b0760 2023-10-06T05:08:01.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.385+0000 7f1b1bfff700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:6824/85783,v1:172.21.15.163:6825/85783] conn(0x7f1b2006dc30 0x7f1b20070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:01.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.385+0000 7f1b1bfff700 1 -- 172.21.15.55:0/1326042090 --> [v2:172.21.15.163:6824/85783,v1:172.21.15.163:6825/85783] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b20070680 con 0x7f1b2006dc30 2023-10-06T05:08:01.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.385+0000 7f1b1bfff700 1 -- 172.21.15.55:0/1326042090 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=212) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b24086830 con 0x7f1b340b0760 2023-10-06T05:08:01.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.385+0000 7f1b337fe700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1b20066f30 0x7f1b200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:01.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.385+0000 7f1b3899c700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:6824/85783,v1:172.21.15.163:6825/85783] conn(0x7f1b2006dc30 0x7f1b20070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:01.388 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.386+0000 7f1b337fe700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1b20066f30 0x7f1b200693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b28006fd0 tx=0x7f1b28008040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:01.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.386+0000 7f1b3899c700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:6824/85783,v1:172.21.15.163:6825/85783] conn(0x7f1b2006dc30 0x7f1b20070050 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:01.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.387+0000 7f1b1bfff700 1 -- 172.21.15.55:0/1326042090 <== osd.4 v2:172.21.15.163:6824/85783 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f1b20070680 con 0x7f1b2006dc30 2023-10-06T05:08:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.402+0000 7f1b3919d700 1 -- 172.21.15.55:0/1326042090 --> [v2:172.21.15.163:6824/85783,v1:172.21.15.163:6825/85783] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1b340afd70 con 0x7f1b2006dc30 2023-10-06T05:08:01.404 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.402+0000 7f1b1bfff700 1 -- 172.21.15.55:0/1326042090 <== osd.4 v2:172.21.15.163:6824/85783 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1b340afd70 con 0x7f1b2006dc30 2023-10-06T05:08:01.405 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T05:08:01.405 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:08:01.405 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T05:08:01.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.403+0000 7f1b3919d700 1 -- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:6824/85783,v1:172.21.15.163:6825/85783] conn(0x7f1b2006dc30 msgr2=0x7f1b20070050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.403+0000 7f1b3919d700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:6824/85783,v1:172.21.15.163:6825/85783] conn(0x7f1b2006dc30 0x7f1b20070050 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.403+0000 7f1b3919d700 1 -- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1b20066f30 msgr2=0x7f1b200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.403+0000 7f1b3919d700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f1b20066f30 0x7f1b200693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b28006fd0 tx=0x7f1b28008040 comp rx=0 tx=0).stop 2023-10-06T05:08:01.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.403+0000 7f1b3919d700 1 -- 172.21.15.55:0/1326042090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b340b0760 msgr2=0x7f1b340aa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.403+0000 7f1b3919d700 1 --2- 172.21.15.55:0/1326042090 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f1b340b0760 0x7f1b340aa8e0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400c7c0 tx=0x7f1b24007e00 comp rx=0 tx=0).stop 2023-10-06T05:08:01.406 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.403+0000 7f1b3899c700 1 -- 172.21.15.55:0/1326042090 reap_dead start 2023-10-06T05:08:01.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.403+0000 7f1b3919d700 1 -- 172.21.15.55:0/1326042090 shutdown_connections 2023-10-06T05:08:01.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.403+0000 7f1b3919d700 1 -- 172.21.15.55:0/1326042090 >> 172.21.15.55:0/1326042090 conn(0x7f1b34068e20 msgr2=0x7f1b340cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:01.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.403+0000 7f1b3919d700 1 -- 172.21.15.55:0/1326042090 shutdown_connections 2023-10-06T05:08:01.407 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.403+0000 7f1b3919d700 1 -- 172.21.15.55:0/1326042090 wait complete. 2023-10-06T05:08:01.421 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T05:08:01.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.608+0000 7efc2f59d700 1 Processor -- start 2023-10-06T05:08:01.610 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.608+0000 7efc2f59d700 1 -- start start 2023-10-06T05:08:01.611 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.609+0000 7efc2f59d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efc28150180 0x7efc28157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:01.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.609+0000 7efc2f59d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2806e2a0 con 0x7efc28150550 2023-10-06T05:08:01.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.609+0000 7efc2f59d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2806b2e0 con 0x7efc280c7410 2023-10-06T05:08:01.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.609+0000 7efc2f59d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc280c78e0 con 0x7efc28150180 2023-10-06T05:08:01.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.610+0000 7efc2e59b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7efc28150180 0x7efc28157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:01.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.610+0000 7efc2dd9a700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7efc28150550 0x7efc280c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:40208/0 (socket says 172.21.15.55:40208) 2023-10-06T05:08:01.612 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.610+0000 7efc2dd9a700 1 -- 172.21.15.55:0/249623495 learned_addr learned my addr 172.21.15.55:0/249623495 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:08:01.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.610+0000 7efc2e59b700 1 -- 172.21.15.55:0/249623495 >> v1:172.21.15.163:6789/0 conn(0x7efc280c7410 legacy=0x7efc28069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.610+0000 7efc2d599700 1 -- 172.21.15.55:0/249623495 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2523277374 0 0) 0x7efc2806e2a0 con 0x7efc28150550 2023-10-06T05:08:01.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.610+0000 7efc2e59b700 1 -- 172.21.15.55:0/249623495 >> v1:172.21.15.55:6789/0 conn(0x7efc28150550 legacy=0x7efc280c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.610+0000 7efc2e59b700 1 -- 172.21.15.55:0/249623495 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc280c7bc0 con 0x7efc28150180 2023-10-06T05:08:01.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.610+0000 7efc2e59b700 1 --2- 172.21.15.55:0/249623495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc28150180 0x7efc28157770 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7efc240098e0 tx=0x7efc240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc16f3195d11aa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:01.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.611+0000 7efc2d599700 1 -- 172.21.15.55:0/249623495 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24009e80 con 0x7efc28150180 2023-10-06T05:08:01.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.611+0000 7efc2d599700 1 -- 172.21.15.55:0/249623495 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2400b860 con 0x7efc28150180 2023-10-06T05:08:01.614 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.611+0000 7efc2d599700 1 -- 172.21.15.55:0/249623495 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2400bbe0 con 0x7efc28150180 2023-10-06T05:08:01.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.611+0000 7efc2f59d700 1 -- 172.21.15.55:0/249623495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc28150180 msgr2=0x7efc28157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.611+0000 7efc2f59d700 1 --2- 172.21.15.55:0/249623495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc28150180 0x7efc28157770 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7efc240098e0 tx=0x7efc240091a0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.611+0000 7efc2f59d700 1 -- 172.21.15.55:0/249623495 shutdown_connections 2023-10-06T05:08:01.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.611+0000 7efc2f59d700 1 --2- 172.21.15.55:0/249623495 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc28150180 0x7efc28157770 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.611+0000 7efc2f59d700 1 -- 172.21.15.55:0/249623495 >> 172.21.15.55:0/249623495 conn(0x7efc280cc1f0 msgr2=0x7efc280cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:01.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.611+0000 7efc2f59d700 1 -- 172.21.15.55:0/249623495 shutdown_connections 2023-10-06T05:08:01.615 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.611+0000 7efc2f59d700 1 -- 172.21.15.55:0/249623495 wait complete. 2023-10-06T05:08:01.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.613+0000 7efc2f59d700 1 Processor -- start 2023-10-06T05:08:01.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.613+0000 7efc2f59d700 1 -- start start 2023-10-06T05:08:01.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.614+0000 7efc2f59d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc280c7410 0x7efc280b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:01.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.614+0000 7efc2f59d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc28150180 0x7efc280ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:01.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.614+0000 7efc2f59d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc28150550 0x7efc2814b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:01.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.614+0000 7efc2f59d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2814da80 con 0x7efc28150180 2023-10-06T05:08:01.618 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.614+0000 7efc2f59d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc280b0fe0 con 0x7efc280c7410 2023-10-06T05:08:01.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.614+0000 7efc2f59d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc280b12c0 con 0x7efc28150550 2023-10-06T05:08:01.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.614+0000 7efc2dd9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc28150180 0x7efc280ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:01.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.614+0000 7efc2dd9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc28150180 0x7efc280ae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54796/0 (socket says 172.21.15.55:54796) 2023-10-06T05:08:01.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.614+0000 7efc2dd9a700 1 -- 172.21.15.55:0/1789483810 learned_addr learned my addr 172.21.15.55:0/1789483810 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:01.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.614+0000 7efc2e59b700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc280c7410 0x7efc280b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:01.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.614+0000 7efc2ed9c700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc28150550 0x7efc2814b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:01.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.615+0000 7efc2dd9a700 1 -- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc28150550 msgr2=0x7efc2814b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.615+0000 7efc2dd9a700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc28150550 0x7efc2814b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.615+0000 7efc2dd9a700 1 -- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc280c7410 msgr2=0x7efc280b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.619 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.615+0000 7efc2dd9a700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc280c7410 0x7efc280b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.615+0000 7efc2dd9a700 1 -- 172.21.15.55:0/1789483810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc24009420 con 0x7efc28150180 2023-10-06T05:08:01.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.615+0000 7efc2ed9c700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc28150550 0x7efc2814b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:08:01.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.615+0000 7efc2dd9a700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc28150180 0x7efc280ae2f0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7efc1800c560 tx=0x7efc18000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:01.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.615+0000 7efc1e7fc700 1 -- 172.21.15.55:0/1789483810 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18005b30 con 0x7efc28150180 2023-10-06T05:08:01.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.615+0000 7efc1e7fc700 1 -- 172.21.15.55:0/1789483810 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc18005c90 con 0x7efc28150180 2023-10-06T05:08:01.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.615+0000 7efc2f59d700 1 -- 172.21.15.55:0/1789483810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2814d710 con 0x7efc28150180 2023-10-06T05:08:01.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.615+0000 7efc1e7fc700 1 -- 172.21.15.55:0/1789483810 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18007320 con 0x7efc28150180 2023-10-06T05:08:01.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.615+0000 7efc2f59d700 1 -- 172.21.15.55:0/1789483810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2814b5e0 con 0x7efc28150180 2023-10-06T05:08:01.620 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.616+0000 7efc2f59d700 1 -- 172.21.15.55:0/1789483810 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2814d570 con 0x7efc28150180 2023-10-06T05:08:01.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.617+0000 7efc1e7fc700 1 -- 172.21.15.55:0/1789483810 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc1801b3b0 con 0x7efc28150180 2023-10-06T05:08:01.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.618+0000 7efc1e7fc700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efc0c066f30 0x7efc0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:01.621 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.618+0000 7efc1e7fc700 1 -- 172.21.15.55:0/1789483810 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7efc18083c80 con 0x7efc28150180 2023-10-06T05:08:01.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.618+0000 7efc2e59b700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efc0c066f30 0x7efc0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:01.622 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.619+0000 7efc2e59b700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efc0c066f30 0x7efc0c0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc240093f0 tx=0x7efc24017000 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:01.624 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.622+0000 7efc1e7fc700 1 -- 172.21.15.55:0/1789483810 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc2814d570 con 0x7efc28150180 2023-10-06T05:08:01.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.825+0000 7efc2f59d700 1 -- 172.21.15.55:0/1789483810 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7efc2814d570 con 0x7efc0c066f30 2023-10-06T05:08:01.827 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.826+0000 7efc1e7fc700 1 -- 172.21.15.55:0/1789483810 <== mgr.10033 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efc2814d570 con 0x7efc0c066f30 2023-10-06T05:08:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.828+0000 7efc2f59d700 1 -- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efc0c066f30 msgr2=0x7efc0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.830 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.828+0000 7efc2f59d700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efc0c066f30 0x7efc0c0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc240093f0 tx=0x7efc24017000 comp rx=0 tx=0).stop 2023-10-06T05:08:01.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.828+0000 7efc2f59d700 1 -- 172.21.15.55:0/1789483810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc28150180 msgr2=0x7efc280ae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:01.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.828+0000 7efc2f59d700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc28150180 0x7efc280ae2f0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7efc1800c560 tx=0x7efc18000fe0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.828+0000 7efc2f59d700 1 -- 172.21.15.55:0/1789483810 shutdown_connections 2023-10-06T05:08:01.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.828+0000 7efc2f59d700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7efc0c066f30 0x7efc0c0693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.828+0000 7efc2f59d700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7efc28150550 0x7efc2814b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.828+0000 7efc2f59d700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7efc280c7410 0x7efc280b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.828+0000 7efc2f59d700 1 --2- 172.21.15.55:0/1789483810 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7efc28150180 0x7efc280ae2f0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:01.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.828+0000 7efc2f59d700 1 -- 172.21.15.55:0/1789483810 >> 172.21.15.55:0/1789483810 conn(0x7efc280cc1f0 msgr2=0x7efc281562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:01.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.828+0000 7efc2f59d700 1 -- 172.21.15.55:0/1789483810 shutdown_connections 2023-10-06T05:08:01.831 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:01.828+0000 7efc2f59d700 1 -- 172.21.15.55:0/1789483810 wait complete. 2023-10-06T05:08:01.832 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T05:08:01.847 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T05:08:01.848 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:08:02.039 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.037+0000 7f0936f9d700 1 Processor -- start 2023-10-06T05:08:02.040 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.038+0000 7f0936f9d700 1 -- start start 2023-10-06T05:08:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.039+0000 7f0936f9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0930156350 0x7f0930157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.039+0000 7f0936f9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f09300b0760 0x7f09300b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.039+0000 7f0936f9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09300b0570 con 0x7f0930159f20 2023-10-06T05:08:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.039+0000 7f0936f9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09300b1140 con 0x7f0930156350 2023-10-06T05:08:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.039+0000 7f0936f9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09301578b0 con 0x7f09300b0760 2023-10-06T05:08:02.043 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.039+0000 7f0935f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0930156350 0x7f0930157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.039+0000 7f0935f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f0930156350 0x7f0930157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54808/0 (socket says 172.21.15.55:54808) 2023-10-06T05:08:02.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.039+0000 7f0935f9b700 1 -- 172.21.15.55:0/4291064283 learned_addr learned my addr 172.21.15.55:0/4291064283 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:02.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.039+0000 7f093579a700 1 --2- 172.21.15.55:0/4291064283 >> v2:172.21.15.163:3300/0 conn(0x7f09300b0760 0x7f09300b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.040+0000 7f0935f9b700 1 -- 172.21.15.55:0/4291064283 >> v2:172.21.15.163:3300/0 conn(0x7f09300b0760 msgr2=0x7f09300b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:02.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.040+0000 7f0935f9b700 1 --2- 172.21.15.55:0/4291064283 >> v2:172.21.15.163:3300/0 conn(0x7f09300b0760 0x7f09300b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.040+0000 7f0935f9b700 1 -- 172.21.15.55:0/4291064283 >> v1:172.21.15.163:6789/0 conn(0x7f0930159f20 legacy=0x7f093015a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.040+0000 7f0935f9b700 1 -- 172.21.15.55:0/4291064283 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09300af4d0 con 0x7f0930156350 2023-10-06T05:08:02.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.040+0000 7f093579a700 1 --2- 172.21.15.55:0/4291064283 >> v2:172.21.15.163:3300/0 conn(0x7f09300b0760 0x7f09300b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:08:02.044 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.040+0000 7f0935f9b700 1 --2- 172.21.15.55:0/4291064283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0930156350 0x7f0930157370 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f09300642a0 tx=0x7f092c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9a9aa0d7d5aec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.040+0000 7f0934f99700 1 -- 172.21.15.55:0/4291064283 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c00dc50 con 0x7f0930156350 2023-10-06T05:08:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.040+0000 7f0934f99700 1 -- 172.21.15.55:0/4291064283 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c00ddb0 con 0x7f0930156350 2023-10-06T05:08:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.040+0000 7f0934f99700 1 -- 172.21.15.55:0/4291064283 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c0196d0 con 0x7f0930156350 2023-10-06T05:08:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.041+0000 7f0936f9d700 1 -- 172.21.15.55:0/4291064283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0930156350 msgr2=0x7f0930157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.041+0000 7f0936f9d700 1 --2- 172.21.15.55:0/4291064283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0930156350 0x7f0930157370 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f09300642a0 tx=0x7f092c00b4f0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.041+0000 7f0936f9d700 1 -- 172.21.15.55:0/4291064283 shutdown_connections 2023-10-06T05:08:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.041+0000 7f0936f9d700 1 --2- 172.21.15.55:0/4291064283 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0930156350 0x7f0930157370 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.045 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.041+0000 7f0936f9d700 1 --2- 172.21.15.55:0/4291064283 >> v2:172.21.15.163:3300/0 conn(0x7f09300b0760 0x7f09300b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.041+0000 7f0936f9d700 1 -- 172.21.15.55:0/4291064283 >> 172.21.15.55:0/4291064283 conn(0x7f0930068e20 msgr2=0x7f09300cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:02.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.041+0000 7f0936f9d700 1 -- 172.21.15.55:0/4291064283 shutdown_connections 2023-10-06T05:08:02.046 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.041+0000 7f0936f9d700 1 -- 172.21.15.55:0/4291064283 wait complete. 2023-10-06T05:08:02.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.042+0000 7f0936f9d700 1 Processor -- start 2023-10-06T05:08:02.048 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.043+0000 7f0936f9d700 1 -- start start 2023-10-06T05:08:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.043+0000 7f0936f9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09300b0760 0x7f093014e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f0936f9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0930156350 0x7f0930151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f0936f9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0930159f20 0x7f093006e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f0936f9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093014e570 con 0x7f0930156350 2023-10-06T05:08:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f0936f9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093016aa70 con 0x7f0930159f20 2023-10-06T05:08:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f0936f9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093016a140 con 0x7f09300b0760 2023-10-06T05:08:02.049 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f093579a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0930156350 0x7f0930151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f093579a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0930156350 0x7f0930151c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54816/0 (socket says 172.21.15.55:54816) 2023-10-06T05:08:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f093579a700 1 -- 172.21.15.55:0/2898679436 learned_addr learned my addr 172.21.15.55:0/2898679436 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f0935f9b700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09300b0760 0x7f093014e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f093679c700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0930159f20 0x7f093006e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f093579a700 1 -- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09300b0760 msgr2=0x7f093014e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f093579a700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09300b0760 0x7f093014e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f093579a700 1 -- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0930159f20 msgr2=0x7f093006e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f093579a700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f0930159f20 0x7f093006e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f093579a700 1 -- 172.21.15.55:0/2898679436 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c00b790 con 0x7f0930156350 2023-10-06T05:08:02.050 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f0935f9b700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f09300b0760 0x7f093014e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:08:02.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.044+0000 7f093579a700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0930156350 0x7f0930151c80 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f092000d860 tx=0x7f0920008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:02.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.045+0000 7f0925ffb700 1 -- 172.21.15.55:0/2898679436 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09200125f0 con 0x7f0930156350 2023-10-06T05:08:02.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.045+0000 7f0925ffb700 1 -- 172.21.15.55:0/2898679436 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0920012750 con 0x7f0930156350 2023-10-06T05:08:02.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.045+0000 7f0925ffb700 1 -- 172.21.15.55:0/2898679436 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920012ad0 con 0x7f0930156350 2023-10-06T05:08:02.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.045+0000 7f0936f9d700 1 -- 172.21.15.55:0/2898679436 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093016a420 con 0x7f0930156350 2023-10-06T05:08:02.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.045+0000 7f0936f9d700 1 -- 172.21.15.55:0/2898679436 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09301521c0 con 0x7f0930156350 2023-10-06T05:08:02.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.046+0000 7f0936f9d700 1 -- 172.21.15.55:0/2898679436 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f093004c3c0 con 0x7f0930156350 2023-10-06T05:08:02.051 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.047+0000 7f0925ffb700 1 -- 172.21.15.55:0/2898679436 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0920014430 con 0x7f0930156350 2023-10-06T05:08:02.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.048+0000 7f0925ffb700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0914066f30 0x7f09140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.048+0000 7f0925ffb700 1 -- 172.21.15.55:0/2898679436 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f0920084ca0 con 0x7f0930156350 2023-10-06T05:08:02.052 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.048+0000 7f0925ffb700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:6816/85786,v1:172.21.15.163:6817/85786] conn(0x7f091406dc30 0x7f0914070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.048+0000 7f0925ffb700 1 -- 172.21.15.55:0/2898679436 --> [v2:172.21.15.163:6816/85786,v1:172.21.15.163:6817/85786] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0914070680 con 0x7f091406dc30 2023-10-06T05:08:02.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.048+0000 7f0925ffb700 1 -- 172.21.15.55:0/2898679436 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=212) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09200891c0 con 0x7f0930156350 2023-10-06T05:08:02.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.048+0000 7f0935f9b700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0914066f30 0x7f09140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.048+0000 7f093679c700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:6816/85786,v1:172.21.15.163:6817/85786] conn(0x7f091406dc30 0x7f0914070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.049+0000 7f0935f9b700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0914066f30 0x7f09140693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f09300642a0 tx=0x7f092c01b000 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:02.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.049+0000 7f093679c700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:6816/85786,v1:172.21.15.163:6817/85786] conn(0x7f091406dc30 0x7f0914070050 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:02.053 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.050+0000 7f0925ffb700 1 -- 172.21.15.55:0/2898679436 <== osd.5 v2:172.21.15.163:6816/85786 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0914070680 con 0x7f091406dc30 2023-10-06T05:08:02.066 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.065+0000 7f0936f9d700 1 -- 172.21.15.55:0/2898679436 --> [v2:172.21.15.163:6816/85786,v1:172.21.15.163:6817/85786] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f09300af7b0 con 0x7f091406dc30 2023-10-06T05:08:02.067 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.065+0000 7f0925ffb700 1 -- 172.21.15.55:0/2898679436 <== osd.5 v2:172.21.15.163:6816/85786 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f09300af7b0 con 0x7f091406dc30 2023-10-06T05:08:02.067 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T05:08:02.067 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:08:02.067 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T05:08:02.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.066+0000 7f0936f9d700 1 -- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:6816/85786,v1:172.21.15.163:6817/85786] conn(0x7f091406dc30 msgr2=0x7f0914070050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.066+0000 7f0936f9d700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:6816/85786,v1:172.21.15.163:6817/85786] conn(0x7f091406dc30 0x7f0914070050 crc :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.066+0000 7f0936f9d700 1 -- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0914066f30 msgr2=0x7f09140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.066+0000 7f0936f9d700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f0914066f30 0x7f09140693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f09300642a0 tx=0x7f092c01b000 comp rx=0 tx=0).stop 2023-10-06T05:08:02.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.066+0000 7f0936f9d700 1 -- 172.21.15.55:0/2898679436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0930156350 msgr2=0x7f0930151c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.066+0000 7f0936f9d700 1 --2- 172.21.15.55:0/2898679436 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f0930156350 0x7f0930151c80 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f092000d860 tx=0x7f0920008040 comp rx=0 tx=0).stop 2023-10-06T05:08:02.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.066+0000 7f093679c700 1 -- 172.21.15.55:0/2898679436 reap_dead start 2023-10-06T05:08:02.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.066+0000 7f0936f9d700 1 -- 172.21.15.55:0/2898679436 shutdown_connections 2023-10-06T05:08:02.069 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.066+0000 7f0936f9d700 1 -- 172.21.15.55:0/2898679436 >> 172.21.15.55:0/2898679436 conn(0x7f0930068e20 msgr2=0x7f09300cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:02.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.066+0000 7f0936f9d700 1 -- 172.21.15.55:0/2898679436 shutdown_connections 2023-10-06T05:08:02.070 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.066+0000 7f0936f9d700 1 -- 172.21.15.55:0/2898679436 wait complete. 2023-10-06T05:08:02.084 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T05:08:02.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.270+0000 7f721af9d700 1 Processor -- start 2023-10-06T05:08:02.272 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.270+0000 7f721af9d700 1 -- start start 2023-10-06T05:08:02.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.272+0000 7f721af9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7214156310 0x7f72140c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.272+0000 7f721af9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f72141566e0 0x7f72140ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.272+0000 7f721af9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72140ca300 0x7f72140ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.272+0000 7f721af9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72140ca860 con 0x7f7214156310 2023-10-06T05:08:02.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.272+0000 7f721af9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72140cab40 con 0x7f72141566e0 2023-10-06T05:08:02.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.272+0000 7f721af9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72140cae20 con 0x7f72140ca300 2023-10-06T05:08:02.275 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.272+0000 7f7219f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7214156310 0x7f72140c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.272+0000 7f7219f9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7214156310 0x7f72140c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54822/0 (socket says 172.21.15.55:54822) 2023-10-06T05:08:02.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.272+0000 7f7219f9b700 1 -- 172.21.15.55:0/2149519724 learned_addr learned my addr 172.21.15.55:0/2149519724 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:02.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.272+0000 7f721979a700 1 --2- 172.21.15.55:0/2149519724 >> v2:172.21.15.163:3300/0 conn(0x7f72141566e0 0x7f72140ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.272+0000 7f721a79c700 1 --2- 172.21.15.55:0/2149519724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72140ca300 0x7f72140ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.273+0000 7f7219f9b700 1 -- 172.21.15.55:0/2149519724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72140ca300 msgr2=0x7f72140ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:02.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.273+0000 7f7219f9b700 1 --2- 172.21.15.55:0/2149519724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72140ca300 0x7f72140ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.273+0000 7f7219f9b700 1 -- 172.21.15.55:0/2149519724 >> v2:172.21.15.163:3300/0 conn(0x7f72141566e0 msgr2=0x7f72140ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:02.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.273+0000 7f7219f9b700 1 --2- 172.21.15.55:0/2149519724 >> v2:172.21.15.163:3300/0 conn(0x7f72141566e0 0x7f72140ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.276 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.273+0000 7f7219f9b700 1 -- 172.21.15.55:0/2149519724 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72140cb100 con 0x7f7214156310 2023-10-06T05:08:02.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.273+0000 7f721979a700 1 --2- 172.21.15.55:0/2149519724 >> v2:172.21.15.163:3300/0 conn(0x7f72141566e0 0x7f72140ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:08:02.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.273+0000 7f7219f9b700 1 --2- 172.21.15.55:0/2149519724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7214156310 0x7f72140c9dc0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f721406c000 tx=0x7f7210009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8051144a94d44c33 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:02.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.273+0000 7f7218f99700 1 -- 172.21.15.55:0/2149519724 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721000b860 con 0x7f7214156310 2023-10-06T05:08:02.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.273+0000 7f7218f99700 1 -- 172.21.15.55:0/2149519724 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721000b9c0 con 0x7f7214156310 2023-10-06T05:08:02.277 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.273+0000 7f7218f99700 1 -- 172.21.15.55:0/2149519724 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721000bd60 con 0x7f7214156310 2023-10-06T05:08:02.279 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.274+0000 7f721af9d700 1 -- 172.21.15.55:0/2149519724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7214156310 msgr2=0x7f72140c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.274+0000 7f721af9d700 1 --2- 172.21.15.55:0/2149519724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7214156310 0x7f72140c9dc0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f721406c000 tx=0x7f7210009250 comp rx=0 tx=0).stop 2023-10-06T05:08:02.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.274+0000 7f721af9d700 1 -- 172.21.15.55:0/2149519724 shutdown_connections 2023-10-06T05:08:02.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.274+0000 7f721af9d700 1 --2- 172.21.15.55:0/2149519724 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f72140ca300 0x7f72140ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.274+0000 7f721af9d700 1 --2- 172.21.15.55:0/2149519724 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7214156310 0x7f72140c9dc0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.274+0000 7f721af9d700 1 --2- 172.21.15.55:0/2149519724 >> v2:172.21.15.163:3300/0 conn(0x7f72141566e0 0x7f72140ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.274+0000 7f721af9d700 1 -- 172.21.15.55:0/2149519724 >> 172.21.15.55:0/2149519724 conn(0x7f72140cc1f0 msgr2=0x7f72140cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:02.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.274+0000 7f721af9d700 1 -- 172.21.15.55:0/2149519724 shutdown_connections 2023-10-06T05:08:02.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.274+0000 7f721af9d700 1 -- 172.21.15.55:0/2149519724 wait complete. 2023-10-06T05:08:02.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.275+0000 7f721af9d700 1 Processor -- start 2023-10-06T05:08:02.280 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.275+0000 7f721af9d700 1 -- start start 2023-10-06T05:08:02.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.276+0000 7f721af9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72140ca300 0x7f7214069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.276+0000 7f721af9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7214156310 0x7f72140b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.276+0000 7f721af9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72141566e0 0x7f7214145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.276+0000 7f721af9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72140aef90 con 0x7f72140ca300 2023-10-06T05:08:02.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.276+0000 7f721af9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72140af830 con 0x7f72141566e0 2023-10-06T05:08:02.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.276+0000 7f721af9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72140b0e30 con 0x7f7214156310 2023-10-06T05:08:02.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.277+0000 7f7219f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72140ca300 0x7f7214069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.277+0000 7f7219f9b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72140ca300 0x7f7214069ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54826/0 (socket says 172.21.15.55:54826) 2023-10-06T05:08:02.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.277+0000 7f7219f9b700 1 -- 172.21.15.55:0/282900547 learned_addr learned my addr 172.21.15.55:0/282900547 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:02.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.277+0000 7f7219f9b700 1 -- 172.21.15.55:0/282900547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7214156310 msgr2=0x7f72140b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.281 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.277+0000 7f7219f9b700 1 --2- 172.21.15.55:0/282900547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7214156310 0x7f72140b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.277+0000 7f7219f9b700 1 -- 172.21.15.55:0/282900547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72141566e0 msgr2=0x7f7214145830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.277+0000 7f7219f9b700 1 --2- 172.21.15.55:0/282900547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72141566e0 0x7f7214145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.277+0000 7f7219f9b700 1 -- 172.21.15.55:0/282900547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72100094f0 con 0x7f72140ca300 2023-10-06T05:08:02.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.277+0000 7f7219f9b700 1 --2- 172.21.15.55:0/282900547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72140ca300 0x7f7214069ac0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f72100090b0 tx=0x7f7210017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:02.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.277+0000 7f7209ffb700 1 -- 172.21.15.55:0/282900547 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210026480 con 0x7f72140ca300 2023-10-06T05:08:02.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.277+0000 7f7209ffb700 1 -- 172.21.15.55:0/282900547 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721001f070 con 0x7f72140ca300 2023-10-06T05:08:02.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.277+0000 7f7209ffb700 1 -- 172.21.15.55:0/282900547 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210026830 con 0x7f72140ca300 2023-10-06T05:08:02.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.278+0000 7f721af9d700 1 -- 172.21.15.55:0/282900547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72140af1e0 con 0x7f72140ca300 2023-10-06T05:08:02.282 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.278+0000 7f721af9d700 1 -- 172.21.15.55:0/282900547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72141500d0 con 0x7f72140ca300 2023-10-06T05:08:02.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.279+0000 7f721af9d700 1 -- 172.21.15.55:0/282900547 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721404c3c0 con 0x7f72140ca300 2023-10-06T05:08:02.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.280+0000 7f7209ffb700 1 -- 172.21.15.55:0/282900547 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7210026ba0 con 0x7f72140ca300 2023-10-06T05:08:02.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.281+0000 7f7209ffb700 1 --2- 172.21.15.55:0/282900547 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f71f8066f30 0x7f71f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.281+0000 7f7209ffb700 1 -- 172.21.15.55:0/282900547 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f7210021070 con 0x7f72140ca300 2023-10-06T05:08:02.283 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.281+0000 7f721979a700 1 --2- 172.21.15.55:0/282900547 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f71f8066f30 0x7f71f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.284 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.282+0000 7f721979a700 1 --2- 172.21.15.55:0/282900547 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f71f8066f30 0x7f71f80693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7204006f00 tx=0x7f7204008040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:02.287 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.286+0000 7f7209ffb700 1 -- 172.21.15.55:0/282900547 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7210053630 con 0x7f72140ca300 2023-10-06T05:08:02.490 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.488+0000 7f721af9d700 1 -- 172.21.15.55:0/282900547 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f72140cd0b0 con 0x7f71f8066f30 2023-10-06T05:08:02.491 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.489+0000 7f7209ffb700 1 -- 172.21.15.55:0/282900547 <== mgr.10033 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f72140cd0b0 con 0x7f71f8066f30 2023-10-06T05:08:02.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.492+0000 7f721af9d700 1 -- 172.21.15.55:0/282900547 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f71f8066f30 msgr2=0x7f71f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.494 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.492+0000 7f721af9d700 1 --2- 172.21.15.55:0/282900547 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f71f8066f30 0x7f71f80693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7204006f00 tx=0x7f7204008040 comp rx=0 tx=0).stop 2023-10-06T05:08:02.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.492+0000 7f721af9d700 1 -- 172.21.15.55:0/282900547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72140ca300 msgr2=0x7f7214069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.492+0000 7f721af9d700 1 --2- 172.21.15.55:0/282900547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72140ca300 0x7f7214069ac0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f72100090b0 tx=0x7f7210017000 comp rx=0 tx=0).stop 2023-10-06T05:08:02.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.492+0000 7f721af9d700 1 -- 172.21.15.55:0/282900547 shutdown_connections 2023-10-06T05:08:02.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.492+0000 7f721af9d700 1 --2- 172.21.15.55:0/282900547 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f71f8066f30 0x7f71f80693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.492+0000 7f721af9d700 1 --2- 172.21.15.55:0/282900547 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f72141566e0 0x7f7214145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.492+0000 7f721af9d700 1 --2- 172.21.15.55:0/282900547 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f72140ca300 0x7f7214069ac0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.495 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.492+0000 7f721af9d700 1 --2- 172.21.15.55:0/282900547 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7214156310 0x7f72140b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.492+0000 7f721af9d700 1 -- 172.21.15.55:0/282900547 >> 172.21.15.55:0/282900547 conn(0x7f72140cc1f0 msgr2=0x7f7214155210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:02.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.492+0000 7f721af9d700 1 -- 172.21.15.55:0/282900547 shutdown_connections 2023-10-06T05:08:02.496 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.492+0000 7f721af9d700 1 -- 172.21.15.55:0/282900547 wait complete. 2023-10-06T05:08:02.496 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T05:08:02.511 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T05:08:02.512 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:08:02.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.701+0000 7fa6f5b9d700 1 Processor -- start 2023-10-06T05:08:02.703 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.701+0000 7fa6f5b9d700 1 -- start start 2023-10-06T05:08:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f5b9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6f0156350 0x7fa6f0157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f5b9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fa6f00b0760 0x7fa6f00b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f5b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f0159f20 0x7fa6f015a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f5b9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f00b1140 con 0x7fa6f0156350 2023-10-06T05:08:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f5b9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f00b0570 con 0x7fa6f00b0760 2023-10-06T05:08:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f5b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f01578b0 con 0x7fa6f0159f20 2023-10-06T05:08:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f4b9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6f0156350 0x7fa6f0157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.706 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f4b9b700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fa6f0156350 0x7fa6f0157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54828/0 (socket says 172.21.15.55:54828) 2023-10-06T05:08:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f4b9b700 1 -- 172.21.15.55:0/1977961731 learned_addr learned my addr 172.21.15.55:0/1977961731 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6effff700 1 --2- 172.21.15.55:0/1977961731 >> v2:172.21.15.163:3300/0 conn(0x7fa6f00b0760 0x7fa6f00b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f539c700 1 --2- 172.21.15.55:0/1977961731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f0159f20 0x7fa6f015a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f4b9b700 1 -- 172.21.15.55:0/1977961731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f0159f20 msgr2=0x7fa6f015a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f4b9b700 1 --2- 172.21.15.55:0/1977961731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f0159f20 0x7fa6f015a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f4b9b700 1 -- 172.21.15.55:0/1977961731 >> v2:172.21.15.163:3300/0 conn(0x7fa6f00b0760 msgr2=0x7fa6f00b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f4b9b700 1 --2- 172.21.15.55:0/1977961731 >> v2:172.21.15.163:3300/0 conn(0x7fa6f00b0760 0x7fa6f00b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f4b9b700 1 -- 172.21.15.55:0/1977961731 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f00af4d0 con 0x7fa6f0156350 2023-10-06T05:08:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6effff700 1 --2- 172.21.15.55:0/1977961731 >> v2:172.21.15.163:3300/0 conn(0x7fa6f00b0760 0x7fa6f00b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:08:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.703+0000 7fa6f539c700 1 --2- 172.21.15.55:0/1977961731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f0159f20 0x7fa6f015a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:08:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.704+0000 7fa6f4b9b700 1 --2- 172.21.15.55:0/1977961731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f0156350 0x7fa6f0157370 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00642a0 tx=0x7fa6e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592d2c6de4cc83de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:02.707 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.704+0000 7fa6ef7fe700 1 -- 172.21.15.55:0/1977961731 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400b860 con 0x7fa6f0156350 2023-10-06T05:08:02.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.704+0000 7fa6ef7fe700 1 -- 172.21.15.55:0/1977961731 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e400b9c0 con 0x7fa6f0156350 2023-10-06T05:08:02.708 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.704+0000 7fa6ef7fe700 1 -- 172.21.15.55:0/1977961731 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400bd60 con 0x7fa6f0156350 2023-10-06T05:08:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.705+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/1977961731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f0156350 msgr2=0x7fa6f0157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.705+0000 7fa6f5b9d700 1 --2- 172.21.15.55:0/1977961731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f0156350 0x7fa6f0157370 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00642a0 tx=0x7fa6e4009250 comp rx=0 tx=0).stop 2023-10-06T05:08:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.705+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/1977961731 shutdown_connections 2023-10-06T05:08:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.705+0000 7fa6f5b9d700 1 --2- 172.21.15.55:0/1977961731 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f0159f20 0x7fa6f015a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.705+0000 7fa6f5b9d700 1 --2- 172.21.15.55:0/1977961731 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f0156350 0x7fa6f0157370 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.705+0000 7fa6f5b9d700 1 --2- 172.21.15.55:0/1977961731 >> v2:172.21.15.163:3300/0 conn(0x7fa6f00b0760 0x7fa6f00b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.705+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/1977961731 >> 172.21.15.55:0/1977961731 conn(0x7fa6f0068e20 msgr2=0x7fa6f00cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.705+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/1977961731 shutdown_connections 2023-10-06T05:08:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.705+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/1977961731 wait complete. 2023-10-06T05:08:02.709 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.706+0000 7fa6f5b9d700 1 Processor -- start 2023-10-06T05:08:02.710 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.707+0000 7fa6f5b9d700 1 -- start start 2023-10-06T05:08:02.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.707+0000 7fa6f5b9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f00b0760 0x7fa6f00c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.711 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.707+0000 7fa6f5b9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f0156350 0x7fa6f014cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.707+0000 7fa6f5b9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f0159f20 0x7fa6f00ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.707+0000 7fa6f5b9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f0150030 con 0x7fa6f0156350 2023-10-06T05:08:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.708+0000 7fa6f5b9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f0150310 con 0x7fa6f00b0760 2023-10-06T05:08:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.708+0000 7fa6f5b9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f01505f0 con 0x7fa6f0159f20 2023-10-06T05:08:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.708+0000 7fa6effff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f0156350 0x7fa6f014cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.708+0000 7fa6effff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f0156350 0x7fa6f014cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54840/0 (socket says 172.21.15.55:54840) 2023-10-06T05:08:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.708+0000 7fa6effff700 1 -- 172.21.15.55:0/582875566 learned_addr learned my addr 172.21.15.55:0/582875566 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.708+0000 7fa6f4b9b700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f00b0760 0x7fa6f00c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.708+0000 7fa6f539c700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f0159f20 0x7fa6f00ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.708+0000 7fa6effff700 1 -- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f0159f20 msgr2=0x7fa6f00ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.712 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.708+0000 7fa6effff700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f0159f20 0x7fa6f00ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.708+0000 7fa6effff700 1 -- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f00b0760 msgr2=0x7fa6f00c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.708+0000 7fa6effff700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fa6f00b0760 0x7fa6f00c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.708+0000 7fa6effff700 1 -- 172.21.15.55:0/582875566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e40094f0 con 0x7fa6f0156350 2023-10-06T05:08:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.708+0000 7fa6f539c700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fa6f0159f20 0x7fa6f00ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:08:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.709+0000 7fa6effff700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f0156350 0x7fa6f014cd60 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00c880 tx=0x7fa6dc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.709+0000 7fa6ecff9700 1 -- 172.21.15.55:0/582875566 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc011600 con 0x7fa6f0156350 2023-10-06T05:08:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.709+0000 7fa6ecff9700 1 -- 172.21.15.55:0/582875566 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6dc011760 con 0x7fa6f0156350 2023-10-06T05:08:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.709+0000 7fa6ecff9700 1 -- 172.21.15.55:0/582875566 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc011b20 con 0x7fa6f0156350 2023-10-06T05:08:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.709+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/582875566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f01508d0 con 0x7fa6f0156350 2023-10-06T05:08:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.709+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/582875566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f0150e90 con 0x7fa6f0156350 2023-10-06T05:08:02.713 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.710+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/582875566 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6f004c3c0 con 0x7fa6f0156350 2023-10-06T05:08:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.711+0000 7fa6ecff9700 1 -- 172.21.15.55:0/582875566 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa6dc0233e0 con 0x7fa6f0156350 2023-10-06T05:08:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.712+0000 7fa6ecff9700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa6d4066f30 0x7fa6d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.712+0000 7fa6ecff9700 1 -- 172.21.15.55:0/582875566 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7fa6dc084d80 con 0x7fa6f0156350 2023-10-06T05:08:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.712+0000 7fa6ecff9700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:6800/85781,v1:172.21.15.163:6801/85781] conn(0x7fa6d406dc30 0x7fa6d4070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.715 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.712+0000 7fa6ecff9700 1 -- 172.21.15.55:0/582875566 --> [v2:172.21.15.163:6800/85781,v1:172.21.15.163:6801/85781] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6d4070680 con 0x7fa6d406dc30 2023-10-06T05:08:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.712+0000 7fa6ecff9700 1 -- 172.21.15.55:0/582875566 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=212) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6dc085070 con 0x7fa6f0156350 2023-10-06T05:08:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.712+0000 7fa6f4b9b700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa6d4066f30 0x7fa6d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.712+0000 7fa6f539c700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:6800/85781,v1:172.21.15.163:6801/85781] conn(0x7fa6d406dc30 0x7fa6d4070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.713+0000 7fa6f4b9b700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa6d4066f30 0x7fa6d40693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00642a0 tx=0x7fa6e4017000 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:02.716 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.713+0000 7fa6f539c700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:6800/85781,v1:172.21.15.163:6801/85781] conn(0x7fa6d406dc30 0x7fa6d4070050 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:02.717 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.714+0000 7fa6ecff9700 1 -- 172.21.15.55:0/582875566 <== osd.6 v2:172.21.15.163:6800/85781 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa6d4070680 con 0x7fa6d406dc30 2023-10-06T05:08:02.730 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.728+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/582875566 --> [v2:172.21.15.163:6800/85781,v1:172.21.15.163:6801/85781] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa6f00af7b0 con 0x7fa6d406dc30 2023-10-06T05:08:02.731 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.729+0000 7fa6ecff9700 1 -- 172.21.15.55:0/582875566 <== osd.6 v2:172.21.15.163:6800/85781 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa6f00af7b0 con 0x7fa6d406dc30 2023-10-06T05:08:02.731 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T05:08:02.731 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:08:02.731 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T05:08:02.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.730+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:6800/85781,v1:172.21.15.163:6801/85781] conn(0x7fa6d406dc30 msgr2=0x7fa6d4070050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.730+0000 7fa6f5b9d700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:6800/85781,v1:172.21.15.163:6801/85781] conn(0x7fa6d406dc30 0x7fa6d4070050 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.730+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa6d4066f30 msgr2=0x7fa6d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.732 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.730+0000 7fa6f5b9d700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fa6d4066f30 0x7fa6d40693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00642a0 tx=0x7fa6e4017000 comp rx=0 tx=0).stop 2023-10-06T05:08:02.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.730+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/582875566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f0156350 msgr2=0x7fa6f014cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.730+0000 7fa6f5b9d700 1 --2- 172.21.15.55:0/582875566 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fa6f0156350 0x7fa6f014cd60 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00c880 tx=0x7fa6dc008040 comp rx=0 tx=0).stop 2023-10-06T05:08:02.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.730+0000 7fa6f539c700 1 -- 172.21.15.55:0/582875566 reap_dead start 2023-10-06T05:08:02.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.730+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/582875566 shutdown_connections 2023-10-06T05:08:02.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.730+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/582875566 >> 172.21.15.55:0/582875566 conn(0x7fa6f0068e20 msgr2=0x7fa6f0158100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:02.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.730+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/582875566 shutdown_connections 2023-10-06T05:08:02.733 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.730+0000 7fa6f5b9d700 1 -- 172.21.15.55:0/582875566 wait complete. 2023-10-06T05:08:02.748 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T05:08:02.935 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.934+0000 7faad3fff700 1 Processor -- start 2023-10-06T05:08:02.936 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.934+0000 7faad3fff700 1 -- start start 2023-10-06T05:08:02.937 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.935+0000 7faad3fff700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faad4156310 0x7faad40c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.935+0000 7faad3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faad40ae710 0x7faad40c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.935+0000 7faad3fff700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad40ca860 con 0x7faad41566e0 2023-10-06T05:08:02.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.935+0000 7faad3fff700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad40cab40 con 0x7faad4156310 2023-10-06T05:08:02.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.935+0000 7faad3fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad40cae20 con 0x7faad40ae710 2023-10-06T05:08:02.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.935+0000 7faad2ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faad4156310 0x7faad40c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.935+0000 7faad2ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7faad4156310 0x7faad40c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54848/0 (socket says 172.21.15.55:54848) 2023-10-06T05:08:02.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.935+0000 7faad2ffd700 1 -- 172.21.15.55:0/2503543815 learned_addr learned my addr 172.21.15.55:0/2503543815 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:02.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.936+0000 7faad37fe700 1 --2- 172.21.15.55:0/2503543815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faad40ae710 0x7faad40c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.938 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.936+0000 7faad2ffd700 1 -- 172.21.15.55:0/2503543815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faad40ae710 msgr2=0x7faad40c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:02.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.936+0000 7faad2ffd700 1 --2- 172.21.15.55:0/2503543815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faad40ae710 0x7faad40c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.936+0000 7faad2ffd700 1 -- 172.21.15.55:0/2503543815 >> v1:172.21.15.163:6789/0 conn(0x7faad41566e0 legacy=0x7faad40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.936+0000 7faad2ffd700 1 -- 172.21.15.55:0/2503543815 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad40cb100 con 0x7faad4156310 2023-10-06T05:08:02.939 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.936+0000 7faad2ffd700 1 --2- 172.21.15.55:0/2503543815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faad4156310 0x7faad40c9dc0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7faac40099b0 tx=0x7faac4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb98db8e12953a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:02.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.936+0000 7faad1ffb700 1 -- 172.21.15.55:0/2503543815 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac4009ea0 con 0x7faad4156310 2023-10-06T05:08:02.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.936+0000 7faad1ffb700 1 -- 172.21.15.55:0/2503543815 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac400b860 con 0x7faad4156310 2023-10-06T05:08:02.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.936+0000 7faad1ffb700 1 -- 172.21.15.55:0/2503543815 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac400bc00 con 0x7faad4156310 2023-10-06T05:08:02.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.937+0000 7faad3fff700 1 -- 172.21.15.55:0/2503543815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faad4156310 msgr2=0x7faad40c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.940 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.937+0000 7faad3fff700 1 --2- 172.21.15.55:0/2503543815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faad4156310 0x7faad40c9dc0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7faac40099b0 tx=0x7faac4009250 comp rx=0 tx=0).stop 2023-10-06T05:08:02.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.937+0000 7faad3fff700 1 -- 172.21.15.55:0/2503543815 shutdown_connections 2023-10-06T05:08:02.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.937+0000 7faad3fff700 1 --2- 172.21.15.55:0/2503543815 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faad40ae710 0x7faad40c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.937+0000 7faad3fff700 1 --2- 172.21.15.55:0/2503543815 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faad4156310 0x7faad40c9dc0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.937+0000 7faad3fff700 1 -- 172.21.15.55:0/2503543815 >> 172.21.15.55:0/2503543815 conn(0x7faad40cc1f0 msgr2=0x7faad40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:02.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.937+0000 7faad3fff700 1 -- 172.21.15.55:0/2503543815 shutdown_connections 2023-10-06T05:08:02.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.937+0000 7faad3fff700 1 -- 172.21.15.55:0/2503543815 wait complete. 2023-10-06T05:08:02.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.939+0000 7faad3fff700 1 Processor -- start 2023-10-06T05:08:02.941 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.939+0000 7faad3fff700 1 -- start start 2023-10-06T05:08:02.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad3fff700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faad40ae710 0x7faad40c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad3fff700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faad4156310 0x7faad4158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad3fff700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faad41566e0 0x7faad4145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad3fff700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad41535a0 con 0x7faad4156310 2023-10-06T05:08:02.943 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad3fff700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad40c1570 con 0x7faad40ae710 2023-10-06T05:08:02.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad3fff700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad40c1850 con 0x7faad41566e0 2023-10-06T05:08:02.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad27fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faad4156310 0x7faad4158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad27fc700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faad4156310 0x7faad4158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:54850/0 (socket says 172.21.15.55:54850) 2023-10-06T05:08:02.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad27fc700 1 -- 172.21.15.55:0/920896237 learned_addr learned my addr 172.21.15.55:0/920896237 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:02.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad37fe700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faad41566e0 0x7faad4145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad27fc700 1 -- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faad41566e0 msgr2=0x7faad4145c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad2ffd700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faad40ae710 0x7faad40c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad27fc700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faad41566e0 0x7faad4145c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad27fc700 1 -- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faad40ae710 msgr2=0x7faad40c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:02.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad27fc700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faad40ae710 0x7faad40c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:02.944 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad27fc700 1 -- 172.21.15.55:0/920896237 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac40094f0 con 0x7faad4156310 2023-10-06T05:08:02.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad37fe700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faad41566e0 0x7faad4145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:08:02.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.940+0000 7faad2ffd700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faad40ae710 0x7faad40c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:08:02.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.941+0000 7faad27fc700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faad4156310 0x7faad4158190 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7faac800c590 tx=0x7faac8007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:02.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.941+0000 7faabaffd700 1 -- 172.21.15.55:0/920896237 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8011590 con 0x7faad4156310 2023-10-06T05:08:02.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.941+0000 7faabaffd700 1 -- 172.21.15.55:0/920896237 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac8011710 con 0x7faad4156310 2023-10-06T05:08:02.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.941+0000 7faad3fff700 1 -- 172.21.15.55:0/920896237 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad40c1b30 con 0x7faad4156310 2023-10-06T05:08:02.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.941+0000 7faabaffd700 1 -- 172.21.15.55:0/920896237 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8011ad0 con 0x7faad4156310 2023-10-06T05:08:02.945 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.941+0000 7faad3fff700 1 -- 172.21.15.55:0/920896237 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad4146180 con 0x7faad4156310 2023-10-06T05:08:02.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.942+0000 7faad3fff700 1 -- 172.21.15.55:0/920896237 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad4153750 con 0x7faad4156310 2023-10-06T05:08:02.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.943+0000 7faabaffd700 1 -- 172.21.15.55:0/920896237 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faac801c440 con 0x7faad4156310 2023-10-06T05:08:02.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.944+0000 7faabaffd700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faac0066f30 0x7faac00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:02.946 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.944+0000 7faabaffd700 1 -- 172.21.15.55:0/920896237 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7faac8083c50 con 0x7faad4156310 2023-10-06T05:08:02.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.945+0000 7faad2ffd700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faac0066f30 0x7faac00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:02.947 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.945+0000 7faad2ffd700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faac0066f30 0x7faac00693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faac40099b0 tx=0x7faac4017000 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:02.950 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:02.948+0000 7faabaffd700 1 -- 172.21.15.55:0/920896237 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faad4153750 con 0x7faad4156310 2023-10-06T05:08:03.150 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.149+0000 7faad3fff700 1 -- 172.21.15.55:0/920896237 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7faad40c19c0 con 0x7faac0066f30 2023-10-06T05:08:03.151 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.149+0000 7faabaffd700 1 -- 172.21.15.55:0/920896237 <== mgr.10033 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7faad40c19c0 con 0x7faac0066f30 2023-10-06T05:08:03.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.151+0000 7faad3fff700 1 -- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faac0066f30 msgr2=0x7faac00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.151+0000 7faad3fff700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faac0066f30 0x7faac00693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faac40099b0 tx=0x7faac4017000 comp rx=0 tx=0).stop 2023-10-06T05:08:03.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.152+0000 7faad3fff700 1 -- 172.21.15.55:0/920896237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faad4156310 msgr2=0x7faad4158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.152+0000 7faad3fff700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faad4156310 0x7faad4158190 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7faac800c590 tx=0x7faac8007f50 comp rx=0 tx=0).stop 2023-10-06T05:08:03.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.152+0000 7faad3fff700 1 -- 172.21.15.55:0/920896237 shutdown_connections 2023-10-06T05:08:03.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.152+0000 7faad3fff700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7faac0066f30 0x7faac00693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.152+0000 7faad3fff700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7faad41566e0 0x7faad4145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.154 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.152+0000 7faad3fff700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7faad40ae710 0x7faad40c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.152+0000 7faad3fff700 1 --2- 172.21.15.55:0/920896237 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7faad4156310 0x7faad4158190 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.152+0000 7faad3fff700 1 -- 172.21.15.55:0/920896237 >> 172.21.15.55:0/920896237 conn(0x7faad40cc1f0 msgr2=0x7faad4151cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:03.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.152+0000 7faad3fff700 1 -- 172.21.15.55:0/920896237 shutdown_connections 2023-10-06T05:08:03.155 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.152+0000 7faad3fff700 1 -- 172.21.15.55:0/920896237 wait complete. 2023-10-06T05:08:03.155 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T05:08:03.171 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T05:08:03.171 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T05:08:03.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.360+0000 7fcd6659d700 1 Processor -- start 2023-10-06T05:08:03.362 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.360+0000 7fcd6659d700 1 -- start start 2023-10-06T05:08:03.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.362+0000 7fcd6659d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7fcd600ac0e0 0x7fcd600afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:03.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.362+0000 7fcd6659d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6006f0b0 0x7fcd6015d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:03.364 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.362+0000 7fcd6659d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6006e510 con 0x7fcd601597d0 2023-10-06T05:08:03.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.362+0000 7fcd6659d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd60069180 con 0x7fcd600ac0e0 2023-10-06T05:08:03.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.362+0000 7fcd6659d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6006dec0 con 0x7fcd6006f0b0 2023-10-06T05:08:03.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.362+0000 7fcd6559b700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7fcd601597d0 0x7fcd6006e9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:45064/0 (socket says 172.21.15.55:45064) 2023-10-06T05:08:03.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.362+0000 7fcd6559b700 1 -- 172.21.15.55:0/2658937374 learned_addr learned my addr 172.21.15.55:0/2658937374 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:08:03.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.362+0000 7fcd64d9a700 1 --2- 172.21.15.55:0/2658937374 >> v2:172.21.15.163:3300/0 conn(0x7fcd600ac0e0 0x7fcd600afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:03.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.362+0000 7fcd65d9c700 1 --2- 172.21.15.55:0/2658937374 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6006f0b0 0x7fcd6015d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:03.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.362+0000 7fcd57fff700 1 -- 172.21.15.55:0/2658937374 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2495188772 0 0) 0x7fcd6006e510 con 0x7fcd601597d0 2023-10-06T05:08:03.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.363+0000 7fcd57fff700 1 -- 172.21.15.55:0/2658937374 --> v1:172.21.15.55:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd4c003570 con 0x7fcd601597d0 2023-10-06T05:08:03.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.363+0000 7fcd65d9c700 1 -- 172.21.15.55:0/2658937374 >> v2:172.21.15.163:3300/0 conn(0x7fcd600ac0e0 msgr2=0x7fcd600afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:03.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.363+0000 7fcd65d9c700 1 --2- 172.21.15.55:0/2658937374 >> v2:172.21.15.163:3300/0 conn(0x7fcd600ac0e0 0x7fcd600afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.363+0000 7fcd65d9c700 1 -- 172.21.15.55:0/2658937374 >> v1:172.21.15.55:6789/0 conn(0x7fcd601597d0 legacy=0x7fcd6006e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.365 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.363+0000 7fcd65d9c700 1 -- 172.21.15.55:0/2658937374 --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6006e1a0 con 0x7fcd6006f0b0 2023-10-06T05:08:03.366 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.363+0000 7fcd64d9a700 1 --2- 172.21.15.55:0/2658937374 >> v2:172.21.15.163:3300/0 conn(0x7fcd600ac0e0 0x7fcd600afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T05:08:03.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.363+0000 7fcd65d9c700 1 --2- 172.21.15.55:0/2658937374 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6006f0b0 0x7fcd6015d440 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009e60 tx=0x7fcd58009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be549d99d11e2cf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:03.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.363+0000 7fcd57fff700 1 -- 172.21.15.55:0/2658937374 <== mon.2 v2:172.21.15.163:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5800e070 con 0x7fcd6006f0b0 2023-10-06T05:08:03.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.363+0000 7fcd57fff700 1 -- 172.21.15.55:0/2658937374 <== mon.2 v2:172.21.15.163:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5800bab0 con 0x7fcd6006f0b0 2023-10-06T05:08:03.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.363+0000 7fcd57fff700 1 -- 172.21.15.55:0/2658937374 <== mon.2 v2:172.21.15.163:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5800be70 con 0x7fcd6006f0b0 2023-10-06T05:08:03.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.364+0000 7fcd6659d700 1 -- 172.21.15.55:0/2658937374 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6006f0b0 msgr2=0x7fcd6015d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.364+0000 7fcd6659d700 1 --2- 172.21.15.55:0/2658937374 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6006f0b0 0x7fcd6015d440 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009e60 tx=0x7fcd58009780 comp rx=0 tx=0).stop 2023-10-06T05:08:03.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.364+0000 7fcd6659d700 1 -- 172.21.15.55:0/2658937374 shutdown_connections 2023-10-06T05:08:03.367 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.364+0000 7fcd6659d700 1 --2- 172.21.15.55:0/2658937374 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd6006f0b0 0x7fcd6015d440 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.364+0000 7fcd6659d700 1 --2- 172.21.15.55:0/2658937374 >> v2:172.21.15.163:3300/0 conn(0x7fcd600ac0e0 0x7fcd600afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.364+0000 7fcd6659d700 1 -- 172.21.15.55:0/2658937374 >> 172.21.15.55:0/2658937374 conn(0x7fcd6006adf0 msgr2=0x7fcd600ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:03.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.364+0000 7fcd6659d700 1 -- 172.21.15.55:0/2658937374 shutdown_connections 2023-10-06T05:08:03.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.364+0000 7fcd6659d700 1 -- 172.21.15.55:0/2658937374 wait complete. 2023-10-06T05:08:03.368 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.365+0000 7fcd6659d700 1 Processor -- start 2023-10-06T05:08:03.369 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.366+0000 7fcd6659d700 1 -- start start 2023-10-06T05:08:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.366+0000 7fcd6659d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd6006f0b0 0x7fcd601475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.366+0000 7fcd6659d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd601597d0 0x7fcd6016a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.366+0000 7fcd6659d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd60156af0 0x7fcd6016ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.366+0000 7fcd6659d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6016dd60 con 0x7fcd6006f0b0 2023-10-06T05:08:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.367+0000 7fcd6659d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6016d7a0 con 0x7fcd601597d0 2023-10-06T05:08:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.367+0000 7fcd6659d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6016da80 con 0x7fcd60156af0 2023-10-06T05:08:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.367+0000 7fcd6559b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd6006f0b0 0x7fcd601475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.367+0000 7fcd6559b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd6006f0b0 0x7fcd601475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60392/0 (socket says 172.21.15.55:60392) 2023-10-06T05:08:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.367+0000 7fcd6559b700 1 -- 172.21.15.55:0/188979443 learned_addr learned my addr 172.21.15.55:0/188979443 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.367+0000 7fcd65d9c700 1 --2- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd60156af0 0x7fcd6016ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:03.370 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.367+0000 7fcd64d9a700 1 --2- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd601597d0 0x7fcd6016a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:03.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.367+0000 7fcd6559b700 1 -- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd60156af0 msgr2=0x7fcd6016ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.367+0000 7fcd6559b700 1 --2- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fcd60156af0 0x7fcd6016ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.367+0000 7fcd6559b700 1 -- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd601597d0 msgr2=0x7fcd6016a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.367+0000 7fcd6559b700 1 --2- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fcd601597d0 0x7fcd6016a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.367+0000 7fcd6559b700 1 -- 172.21.15.55:0/188979443 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50009420 con 0x7fcd6006f0b0 2023-10-06T05:08:03.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.368+0000 7fcd6559b700 1 --2- 172.21.15.55:0/188979443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd6006f0b0 0x7fcd601475d0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00c590 tx=0x7fcd5c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:03.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.368+0000 7fcd557fa700 1 -- 172.21.15.55:0/188979443 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c005b30 con 0x7fcd6006f0b0 2023-10-06T05:08:03.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.368+0000 7fcd557fa700 1 -- 172.21.15.55:0/188979443 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c005cb0 con 0x7fcd6006f0b0 2023-10-06T05:08:03.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.368+0000 7fcd557fa700 1 -- 172.21.15.55:0/188979443 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c007320 con 0x7fcd6006f0b0 2023-10-06T05:08:03.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.368+0000 7fcd6659d700 1 -- 172.21.15.55:0/188979443 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd58009a20 con 0x7fcd6006f0b0 2023-10-06T05:08:03.371 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.368+0000 7fcd6659d700 1 -- 172.21.15.55:0/188979443 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd60169cb0 con 0x7fcd6006f0b0 2023-10-06T05:08:03.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.369+0000 7fcd6659d700 1 -- 172.21.15.55:0/188979443 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcd60169830 con 0x7fcd6006f0b0 2023-10-06T05:08:03.372 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.370+0000 7fcd557fa700 1 -- 172.21.15.55:0/188979443 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd5c01b3b0 con 0x7fcd6006f0b0 2023-10-06T05:08:03.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.371+0000 7fcd557fa700 1 --2- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcd44066f30 0x7fcd440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:03.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.371+0000 7fcd557fa700 1 -- 172.21.15.55:0/188979443 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7fcd5c083e40 con 0x7fcd6006f0b0 2023-10-06T05:08:03.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.371+0000 7fcd557fa700 1 --2- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:6808/85782,v1:172.21.15.163:6809/85782] conn(0x7fcd4406dc30 0x7fcd44070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:03.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.371+0000 7fcd557fa700 1 -- 172.21.15.55:0/188979443 --> [v2:172.21.15.163:6808/85782,v1:172.21.15.163:6809/85782] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd44070680 con 0x7fcd4406dc30 2023-10-06T05:08:03.373 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.371+0000 7fcd557fa700 1 -- 172.21.15.55:0/188979443 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_get_version_reply(handle=1 version=212) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd5c086340 con 0x7fcd6006f0b0 2023-10-06T05:08:03.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.371+0000 7fcd64d9a700 1 --2- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcd44066f30 0x7fcd440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:03.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.372+0000 7fcd65d9c700 1 --2- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:6808/85782,v1:172.21.15.163:6809/85782] conn(0x7fcd4406dc30 0x7fcd44070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:03.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.372+0000 7fcd64d9a700 1 --2- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcd44066f30 0x7fcd440693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcd500093f0 tx=0x7fcd50008040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:03.374 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.372+0000 7fcd65d9c700 1 --2- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:6808/85782,v1:172.21.15.163:6809/85782] conn(0x7fcd4406dc30 0x7fcd44070050 crc :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:03.375 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.373+0000 7fcd557fa700 1 -- 172.21.15.55:0/188979443 <== osd.7 v2:172.21.15.163:6808/85782 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fcd44070680 con 0x7fcd4406dc30 2023-10-06T05:08:03.389 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.387+0000 7fcd6659d700 1 -- 172.21.15.55:0/188979443 --> [v2:172.21.15.163:6808/85782,v1:172.21.15.163:6809/85782] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcd6004c3c0 con 0x7fcd4406dc30 2023-10-06T05:08:03.390 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.388+0000 7fcd557fa700 1 -- 172.21.15.55:0/188979443 <== osd.7 v2:172.21.15.163:6808/85782 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcd6004c3c0 con 0x7fcd4406dc30 2023-10-06T05:08:03.390 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2023-10-06T05:08:03.390 INFO:teuthology.orchestra.run.smithi055.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T05:08:03.390 INFO:teuthology.orchestra.run.smithi055.stdout:} 2023-10-06T05:08:03.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.389+0000 7fcd6659d700 1 -- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:6808/85782,v1:172.21.15.163:6809/85782] conn(0x7fcd4406dc30 msgr2=0x7fcd44070050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.389+0000 7fcd6659d700 1 --2- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:6808/85782,v1:172.21.15.163:6809/85782] conn(0x7fcd4406dc30 0x7fcd44070050 crc :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.389+0000 7fcd6659d700 1 -- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcd44066f30 msgr2=0x7fcd440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.391 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.389+0000 7fcd6659d700 1 --2- 172.21.15.55:0/188979443 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fcd44066f30 0x7fcd440693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcd500093f0 tx=0x7fcd50008040 comp rx=0 tx=0).stop 2023-10-06T05:08:03.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.389+0000 7fcd6659d700 1 -- 172.21.15.55:0/188979443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd6006f0b0 msgr2=0x7fcd601475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.389+0000 7fcd6659d700 1 --2- 172.21.15.55:0/188979443 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fcd6006f0b0 0x7fcd601475d0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00c590 tx=0x7fcd5c007e00 comp rx=0 tx=0).stop 2023-10-06T05:08:03.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.389+0000 7fcd65d9c700 1 -- 172.21.15.55:0/188979443 reap_dead start 2023-10-06T05:08:03.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.389+0000 7fcd6659d700 1 -- 172.21.15.55:0/188979443 shutdown_connections 2023-10-06T05:08:03.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.389+0000 7fcd6659d700 1 -- 172.21.15.55:0/188979443 >> 172.21.15.55:0/188979443 conn(0x7fcd6006adf0 msgr2=0x7fcd601588a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:03.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.389+0000 7fcd6659d700 1 -- 172.21.15.55:0/188979443 shutdown_connections 2023-10-06T05:08:03.392 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.389+0000 7fcd6659d700 1 -- 172.21.15.55:0/188979443 wait complete. 2023-10-06T05:08:03.407 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T05:08:03.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.593+0000 7f7dda39d700 1 Processor -- start 2023-10-06T05:08:03.595 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.593+0000 7f7dda39d700 1 -- start start 2023-10-06T05:08:03.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.594+0000 7f7dda39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7dd4150550 0x7f7dd4157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:03.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.594+0000 7f7dda39d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7dd40c7410 0x7f7dd4069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:03.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.594+0000 7f7dda39d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd406e2a0 con 0x7f7dd4150180 2023-10-06T05:08:03.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.594+0000 7f7dda39d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd406b2e0 con 0x7f7dd40c7410 2023-10-06T05:08:03.596 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.594+0000 7f7dda39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40c78e0 con 0x7f7dd4150550 2023-10-06T05:08:03.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.594+0000 7f7dd9b9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7dd40c7410 0x7f7dd4069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:03.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.594+0000 7f7dd9b9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f7dd40c7410 0x7f7dd4069f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60398/0 (socket says 172.21.15.55:60398) 2023-10-06T05:08:03.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.594+0000 7f7dd9b9c700 1 -- 172.21.15.55:0/2506276743 learned_addr learned my addr 172.21.15.55:0/2506276743 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:03.597 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.594+0000 7f7dd8b9a700 1 --2- 172.21.15.55:0/2506276743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7dd4150550 0x7f7dd4157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:03.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.595+0000 7f7dd9b9c700 1 -- 172.21.15.55:0/2506276743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7dd4150550 msgr2=0x7f7dd4157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:03.598 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.595+0000 7f7dd9b9c700 1 --2- 172.21.15.55:0/2506276743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7dd4150550 0x7f7dd4157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.595+0000 7f7dd9b9c700 1 -- 172.21.15.55:0/2506276743 >> v1:172.21.15.163:6789/0 conn(0x7f7dd4150180 legacy=0x7f7dd40c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.595+0000 7f7dd9b9c700 1 -- 172.21.15.55:0/2506276743 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd40c7bc0 con 0x7f7dd40c7410 2023-10-06T05:08:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.595+0000 7f7dd9b9c700 1 --2- 172.21.15.55:0/2506276743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dd40c7410 0x7f7dd4069f50 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00c100 tx=0x7f7dcc00a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d476f49ec9937d63 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.595+0000 7f7dcbfff700 1 -- 172.21.15.55:0/2506276743 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc00f070 con 0x7f7dd40c7410 2023-10-06T05:08:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.595+0000 7f7dcbfff700 1 -- 172.21.15.55:0/2506276743 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc00cde0 con 0x7f7dd40c7410 2023-10-06T05:08:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.595+0000 7f7dcbfff700 1 -- 172.21.15.55:0/2506276743 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc00a320 con 0x7f7dd40c7410 2023-10-06T05:08:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.596+0000 7f7dda39d700 1 -- 172.21.15.55:0/2506276743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dd40c7410 msgr2=0x7f7dd4069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.596+0000 7f7dda39d700 1 --2- 172.21.15.55:0/2506276743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dd40c7410 0x7f7dd4069f50 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00c100 tx=0x7f7dcc00a950 comp rx=0 tx=0).stop 2023-10-06T05:08:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.596+0000 7f7dda39d700 1 -- 172.21.15.55:0/2506276743 shutdown_connections 2023-10-06T05:08:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.596+0000 7f7dda39d700 1 --2- 172.21.15.55:0/2506276743 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dd40c7410 0x7f7dd4069f50 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.599 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.596+0000 7f7dda39d700 1 --2- 172.21.15.55:0/2506276743 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7dd4150550 0x7f7dd4157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.596+0000 7f7dda39d700 1 -- 172.21.15.55:0/2506276743 >> 172.21.15.55:0/2506276743 conn(0x7f7dd40cc1f0 msgr2=0x7f7dd40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:03.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.596+0000 7f7dda39d700 1 -- 172.21.15.55:0/2506276743 shutdown_connections 2023-10-06T05:08:03.600 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.596+0000 7f7dda39d700 1 -- 172.21.15.55:0/2506276743 wait complete. 2023-10-06T05:08:03.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.597+0000 7f7dda39d700 1 Processor -- start 2023-10-06T05:08:03.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.597+0000 7f7dda39d700 1 -- start start 2023-10-06T05:08:03.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.598+0000 7f7dda39d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd40c7410 0x7f7dd414ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:03.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.598+0000 7f7dda39d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dd4150180 0x7f7dd40abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:03.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.598+0000 7f7dda39d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7dd4150550 0x7f7dd4158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:03.602 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.598+0000 7f7dda39d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40af3a0 con 0x7f7dd4150180 2023-10-06T05:08:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.598+0000 7f7dda39d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40acf20 con 0x7f7dd40c7410 2023-10-06T05:08:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.598+0000 7f7dda39d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd40ac490 con 0x7f7dd4150550 2023-10-06T05:08:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.599+0000 7f7dd8b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dd4150180 0x7f7dd40abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.599+0000 7f7dd8b9a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dd4150180 0x7f7dd40abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60400/0 (socket says 172.21.15.55:60400) 2023-10-06T05:08:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.599+0000 7f7dd8b9a700 1 -- 172.21.15.55:0/1224825153 learned_addr learned my addr 172.21.15.55:0/1224825153 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.599+0000 7f7dd9b9c700 1 --2- 172.21.15.55:0/1224825153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7dd4150550 0x7f7dd4158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.599+0000 7f7dd939b700 1 --2- 172.21.15.55:0/1224825153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd40c7410 0x7f7dd414ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.599+0000 7f7dd8b9a700 1 -- 172.21.15.55:0/1224825153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7dd4150550 msgr2=0x7f7dd4158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.599+0000 7f7dd8b9a700 1 --2- 172.21.15.55:0/1224825153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7dd4150550 0x7f7dd4158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.599+0000 7f7dd8b9a700 1 -- 172.21.15.55:0/1224825153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd40c7410 msgr2=0x7f7dd414ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.599+0000 7f7dd8b9a700 1 --2- 172.21.15.55:0/1224825153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd40c7410 0x7f7dd414ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.603 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.599+0000 7f7dd8b9a700 1 -- 172.21.15.55:0/1224825153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc00abf0 con 0x7f7dd4150180 2023-10-06T05:08:03.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.599+0000 7f7dd8b9a700 1 --2- 172.21.15.55:0/1224825153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dd4150180 0x7f7dd40abab0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400d890 tx=0x7f7dc4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:03.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.600+0000 7f7dc97fa700 1 -- 172.21.15.55:0/1224825153 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40125f0 con 0x7f7dd4150180 2023-10-06T05:08:03.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.600+0000 7f7dc97fa700 1 -- 172.21.15.55:0/1224825153 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc4012750 con 0x7f7dd4150180 2023-10-06T05:08:03.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.600+0000 7f7dda39d700 1 -- 172.21.15.55:0/1224825153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd40ac770 con 0x7f7dd4150180 2023-10-06T05:08:03.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.600+0000 7f7dc97fa700 1 -- 172.21.15.55:0/1224825153 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4012af0 con 0x7f7dd4150180 2023-10-06T05:08:03.604 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.600+0000 7f7dda39d700 1 -- 172.21.15.55:0/1224825153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd40af030 con 0x7f7dd4150180 2023-10-06T05:08:03.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.601+0000 7f7dda39d700 1 -- 172.21.15.55:0/1224825153 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd4157770 con 0x7f7dd4150180 2023-10-06T05:08:03.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.602+0000 7f7dc97fa700 1 -- 172.21.15.55:0/1224825153 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7dc401e460 con 0x7f7dd4150180 2023-10-06T05:08:03.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.602+0000 7f7dc97fa700 1 --2- 172.21.15.55:0/1224825153 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7db8066f30 0x7f7db80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:03.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.603+0000 7f7dc97fa700 1 -- 172.21.15.55:0/1224825153 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f7dc4084d90 con 0x7f7dd4150180 2023-10-06T05:08:03.605 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.603+0000 7f7dd939b700 1 --2- 172.21.15.55:0/1224825153 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7db8066f30 0x7f7db80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:03.606 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.604+0000 7f7dd939b700 1 --2- 172.21.15.55:0/1224825153 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7db8066f30 0x7f7db80693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0003a80 tx=0x7f7dd0007ec0 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:03.608 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.607+0000 7f7dc97fa700 1 -- 172.21.15.55:0/1224825153 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7dc4051d10 con 0x7f7dd4150180 2023-10-06T05:08:03.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.806+0000 7f7dda39d700 1 -- 172.21.15.55:0/1224825153 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f7dd40af1c0 con 0x7f7db8066f30 2023-10-06T05:08:03.808 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.807+0000 7f7dc97fa700 1 -- 172.21.15.55:0/1224825153 <== mgr.10033 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7dd40af1c0 con 0x7f7db8066f30 2023-10-06T05:08:03.811 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.809+0000 7f7dda39d700 1 -- 172.21.15.55:0/1224825153 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7db8066f30 msgr2=0x7f7db80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.809+0000 7f7dda39d700 1 --2- 172.21.15.55:0/1224825153 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7db8066f30 0x7f7db80693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0003a80 tx=0x7f7dd0007ec0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.809+0000 7f7dda39d700 1 -- 172.21.15.55:0/1224825153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dd4150180 msgr2=0x7f7dd40abab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:03.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.809+0000 7f7dda39d700 1 --2- 172.21.15.55:0/1224825153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dd4150180 0x7f7dd40abab0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400d890 tx=0x7f7dc4008040 comp rx=0 tx=0).stop 2023-10-06T05:08:03.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.809+0000 7f7dda39d700 1 -- 172.21.15.55:0/1224825153 shutdown_connections 2023-10-06T05:08:03.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.809+0000 7f7dda39d700 1 --2- 172.21.15.55:0/1224825153 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f7db8066f30 0x7f7db80693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.812 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.809+0000 7f7dda39d700 1 --2- 172.21.15.55:0/1224825153 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f7dd4150550 0x7f7dd4158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.809+0000 7f7dda39d700 1 --2- 172.21.15.55:0/1224825153 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f7dd40c7410 0x7f7dd414ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.809+0000 7f7dda39d700 1 --2- 172.21.15.55:0/1224825153 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f7dd4150180 0x7f7dd40abab0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:03.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.809+0000 7f7dda39d700 1 -- 172.21.15.55:0/1224825153 >> 172.21.15.55:0/1224825153 conn(0x7f7dd40cc1f0 msgr2=0x7f7dd406a190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:03.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.809+0000 7f7dda39d700 1 -- 172.21.15.55:0/1224825153 shutdown_connections 2023-10-06T05:08:03.813 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:03.809+0000 7f7dda39d700 1 -- 172.21.15.55:0/1224825153 wait complete. 2023-10-06T05:08:03.814 INFO:teuthology.orchestra.run.smithi055.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T05:08:03.829 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:08:04.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.016+0000 7f82c8d9d700 1 Processor -- start 2023-10-06T05:08:04.018 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.016+0000 7f82c8d9d700 1 -- start start 2023-10-06T05:08:04.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.017+0000 7f82c8d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4157270 0x7f82c4068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:04.021 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.017+0000 7f82c8d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f82c40688d0 0x7f82c40b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:04.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.017+0000 7f82c8d9d700 1 --2- >> v2:172.21.15.163:3300/0 conn(0x7f82c40b0a00 0x7f82c406a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:04.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.017+0000 7f82c8d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c4068d40 con 0x7f82c40688d0 2023-10-06T05:08:04.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.017+0000 7f82c8d9d700 1 -- --> v2:172.21.15.163:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c415a980 con 0x7f82c40b0a00 2023-10-06T05:08:04.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.017+0000 7f82c8d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c415a050 con 0x7f82c4157270 2023-10-06T05:08:04.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.017+0000 7f82c2ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f82c40688d0 0x7f82c40b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:04.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.017+0000 7f82c2ffd700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f82c40688d0 0x7f82c40b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60410/0 (socket says 172.21.15.55:60410) 2023-10-06T05:08:04.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.017+0000 7f82c2ffd700 1 -- 172.21.15.55:0/2742748816 learned_addr learned my addr 172.21.15.55:0/2742748816 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:04.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.018+0000 7f82c37fe700 1 --2- 172.21.15.55:0/2742748816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4157270 0x7f82c4068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:04.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.018+0000 7f82c2ffd700 1 -- 172.21.15.55:0/2742748816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4157270 msgr2=0x7f82c4068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:04.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.018+0000 7f82c2ffd700 1 --2- 172.21.15.55:0/2742748816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4157270 0x7f82c4068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:04.022 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.018+0000 7f82c2ffd700 1 -- 172.21.15.55:0/2742748816 >> v2:172.21.15.163:3300/0 conn(0x7f82c40b0a00 msgr2=0x7f82c406a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:04.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.018+0000 7f82c3fff700 1 --2- 172.21.15.55:0/2742748816 >> v2:172.21.15.163:3300/0 conn(0x7f82c40b0a00 0x7f82c406a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:04.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.018+0000 7f82c2ffd700 1 --2- 172.21.15.55:0/2742748816 >> v2:172.21.15.163:3300/0 conn(0x7f82c40b0a00 0x7f82c406a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:04.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.018+0000 7f82c2ffd700 1 -- 172.21.15.55:0/2742748816 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c415a330 con 0x7f82c40688d0 2023-10-06T05:08:04.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.018+0000 7f82c37fe700 1 --2- 172.21.15.55:0/2742748816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4157270 0x7f82c4068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:08:04.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.018+0000 7f82c3fff700 1 --2- 172.21.15.55:0/2742748816 >> v2:172.21.15.163:3300/0 conn(0x7f82c40b0a00 0x7f82c406a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:08:04.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.018+0000 7f82c2ffd700 1 --2- 172.21.15.55:0/2742748816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c40688d0 0x7f82c40b04c0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f82c406ed10 tx=0x7f82b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58efdb2872b6ab4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:04.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.018+0000 7f82c27fc700 1 -- 172.21.15.55:0/2742748816 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b800b860 con 0x7f82c40688d0 2023-10-06T05:08:04.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.018+0000 7f82c27fc700 1 -- 172.21.15.55:0/2742748816 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b800b9c0 con 0x7f82c40688d0 2023-10-06T05:08:04.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.018+0000 7f82c27fc700 1 -- 172.21.15.55:0/2742748816 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b800bd60 con 0x7f82c40688d0 2023-10-06T05:08:04.023 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.019+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2742748816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c40688d0 msgr2=0x7f82c40b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:04.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.019+0000 7f82c8d9d700 1 --2- 172.21.15.55:0/2742748816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c40688d0 0x7f82c40b04c0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f82c406ed10 tx=0x7f82b8009250 comp rx=0 tx=0).stop 2023-10-06T05:08:04.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.019+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2742748816 shutdown_connections 2023-10-06T05:08:04.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.019+0000 7f82c8d9d700 1 --2- 172.21.15.55:0/2742748816 >> v2:172.21.15.163:3300/0 conn(0x7f82c40b0a00 0x7f82c406a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:04.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.019+0000 7f82c8d9d700 1 --2- 172.21.15.55:0/2742748816 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4157270 0x7f82c4068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:04.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.019+0000 7f82c8d9d700 1 --2- 172.21.15.55:0/2742748816 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c40688d0 0x7f82c40b04c0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:04.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.019+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2742748816 >> 172.21.15.55:0/2742748816 conn(0x7f82c40cd310 msgr2=0x7f82c40cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:04.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.019+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2742748816 shutdown_connections 2023-10-06T05:08:04.024 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.019+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2742748816 wait complete. 2023-10-06T05:08:04.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.021+0000 7f82c8d9d700 1 Processor -- start 2023-10-06T05:08:04.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.021+0000 7f82c8d9d700 1 -- start start 2023-10-06T05:08:04.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.022+0000 7f82c8d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c40688d0 0x7f82c4143620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:04.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.022+0000 7f82c8d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c40b0a00 0x7f82c40c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:04.027 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.022+0000 7f82c8d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4157270 0x7f82c4155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.022+0000 7f82c8d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c4152410 con 0x7f82c40b0a00 2023-10-06T05:08:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.022+0000 7f82c8d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c4153ad0 con 0x7f82c40688d0 2023-10-06T05:08:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.022+0000 7f82c8d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c4153db0 con 0x7f82c4157270 2023-10-06T05:08:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.022+0000 7f82c2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c40b0a00 0x7f82c40c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.022+0000 7f82c2ffd700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c40b0a00 0x7f82c40c20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:60426/0 (socket says 172.21.15.55:60426) 2023-10-06T05:08:04.028 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.022+0000 7f82c2ffd700 1 -- 172.21.15.55:0/2748980873 learned_addr learned my addr 172.21.15.55:0/2748980873 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:04.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.022+0000 7f82c37fe700 1 --2- 172.21.15.55:0/2748980873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c40688d0 0x7f82c4143620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:04.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.022+0000 7f82c3fff700 1 --2- 172.21.15.55:0/2748980873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4157270 0x7f82c4155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:04.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.023+0000 7f82c2ffd700 1 -- 172.21.15.55:0/2748980873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4157270 msgr2=0x7f82c4155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:04.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.023+0000 7f82c2ffd700 1 --2- 172.21.15.55:0/2748980873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4157270 0x7f82c4155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:04.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.023+0000 7f82c2ffd700 1 -- 172.21.15.55:0/2748980873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c40688d0 msgr2=0x7f82c4143620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:04.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.023+0000 7f82c2ffd700 1 --2- 172.21.15.55:0/2748980873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c40688d0 0x7f82c4143620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:04.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.023+0000 7f82c2ffd700 1 -- 172.21.15.55:0/2748980873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b80094f0 con 0x7f82c40b0a00 2023-10-06T05:08:04.029 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.023+0000 7f82c2ffd700 1 --2- 172.21.15.55:0/2748980873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c40b0a00 0x7f82c40c20b0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f82c4068ed0 tx=0x7f82b8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:04.030 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.026+0000 7f82ab7fe700 1 -- 172.21.15.55:0/2748980873 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8011dc0 con 0x7f82c40b0a00 2023-10-06T05:08:04.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.026+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2748980873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c4154a80 con 0x7f82c40b0a00 2023-10-06T05:08:04.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.026+0000 7f82ab7fe700 1 -- 172.21.15.55:0/2748980873 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b8024440 con 0x7f82c40b0a00 2023-10-06T05:08:04.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.026+0000 7f82ab7fe700 1 -- 172.21.15.55:0/2748980873 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b8024800 con 0x7f82c40b0a00 2023-10-06T05:08:04.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.026+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2748980873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c4155040 con 0x7f82c40b0a00 2023-10-06T05:08:04.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.027+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2748980873 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c4155230 con 0x7f82c40b0a00 2023-10-06T05:08:04.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.028+0000 7f82ab7fe700 1 -- 172.21.15.55:0/2748980873 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f82b801a420 con 0x7f82c40b0a00 2023-10-06T05:08:04.031 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.028+0000 7f82ab7fe700 1 --2- 172.21.15.55:0/2748980873 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f82b0066f30 0x7f82b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:04.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.029+0000 7f82ab7fe700 1 -- 172.21.15.55:0/2748980873 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f82b8086820 con 0x7f82c40b0a00 2023-10-06T05:08:04.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.029+0000 7f82c37fe700 1 --2- 172.21.15.55:0/2748980873 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f82b0066f30 0x7f82b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:04.032 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.030+0000 7f82c37fe700 1 --2- 172.21.15.55:0/2748980873 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f82b0066f30 0x7f82b00693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f82b4006fd0 tx=0x7f82b4008040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:04.042 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.038+0000 7f82ab7fe700 1 -- 172.21.15.55:0/2748980873 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82b80537a0 con 0x7f82c40b0a00 2023-10-06T05:08:04.238 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.236+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2748980873 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82c404c3c0 con 0x7f82b0066f30 2023-10-06T05:08:04.241 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.240+0000 7f82ab7fe700 1 -- 172.21.15.55:0/2748980873 <== mgr.10033 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72554 (secure 0 0 0) 0x7f82c404c3c0 con 0x7f82b0066f30 2023-10-06T05:08:04.242 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:08:04.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.242+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2748980873 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f82b0066f30 msgr2=0x7f82b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:04.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.242+0000 7f82c8d9d700 1 --2- 172.21.15.55:0/2748980873 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f82b0066f30 0x7f82b00693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f82b4006fd0 tx=0x7f82b4008040 comp rx=0 tx=0).stop 2023-10-06T05:08:04.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.242+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2748980873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c40b0a00 msgr2=0x7f82c40c20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:04.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.242+0000 7f82c8d9d700 1 --2- 172.21.15.55:0/2748980873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c40b0a00 0x7f82c40c20b0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f82c4068ed0 tx=0x7f82b8017000 comp rx=0 tx=0).stop 2023-10-06T05:08:04.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.242+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2748980873 shutdown_connections 2023-10-06T05:08:04.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.242+0000 7f82c8d9d700 1 --2- 172.21.15.55:0/2748980873 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f82b0066f30 0x7f82b00693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:04.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.242+0000 7f82c8d9d700 1 --2- 172.21.15.55:0/2748980873 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f82c4157270 0x7f82c4155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:04.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.242+0000 7f82c8d9d700 1 --2- 172.21.15.55:0/2748980873 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f82c40688d0 0x7f82c4143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:04.244 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.242+0000 7f82c8d9d700 1 --2- 172.21.15.55:0/2748980873 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f82c40b0a00 0x7f82c40c20b0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:04.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.242+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2748980873 >> 172.21.15.55:0/2748980873 conn(0x7f82c40cd310 msgr2=0x7f82c4158f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:04.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.242+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2748980873 shutdown_connections 2023-10-06T05:08:04.245 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:04.242+0000 7f82c8d9d700 1 -- 172.21.15.55:0/2748980873 wait complete. 2023-10-06T05:08:04.245 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T05:08:04.261 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2023-10-06T05:08:03.335417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445304,"kb_used_data":21944,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":752529416,"statfs":{"total":773094113280,"available":770590121984,"internally_reserved":0,"allocated":22470656,"data_stored":12442072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.990921"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":193,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-06T05:07:51.338562+0000","last_change":"2023-10-06T03:33:09.492212+0000","last_active":"2023-10-06T05:07:51.338562+0000","last_peered":"2023-10-06T05:07:51.338562+0000","last_clean":"2023-10-06T05:07:51.338562+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T05:07:51.338562+0000","last_undegraded":"2023-10-06T05:07:51.338562+0000","last_fullsized":"2023-10-06T05:07:51.338562+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":180,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-06T05:07:51.875137+0000","last_change":"2023-10-06T03:33:09.473709+0000","last_active":"2023-10-06T05:07:51.875137+0000","last_peered":"2023-10-06T05:07:51.875137+0000","last_clean":"2023-10-06T05:07:51.875137+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T05:07:51.875137+0000","last_undegraded":"2023-10-06T05:07:51.875137+0000","last_fullsized":"2023-10-06T05:07:51.875137+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:21:31.922009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":176,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-10-06T05:07:52.657189+0000","last_change":"2023-10-06T03:33:09.468489+0000","last_active":"2023-10-06T05:07:52.657189+0000","last_peered":"2023-10-06T05:07:52.657189+0000","last_clean":"2023-10-06T05:07:52.657189+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T05:07:52.657189+0000","last_undegraded":"2023-10-06T05:07:52.657189+0000","last_fullsized":"2023-10-06T05:07:52.657189+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"210'370","reported_seq":1223,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-06T05:07:51.874376+0000","last_change":"2023-10-06T03:33:10.471669+0000","last_active":"2023-10-06T05:07:51.874376+0000","last_peered":"2023-10-06T05:07:51.874376+0000","last_clean":"2023-10-06T05:07:51.874376+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T05:07:51.874376+0000","last_undegraded":"2023-10-06T05:07:51.874376+0000","last_fullsized":"2023-10-06T05:07:51.874376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:09.448931+0000","last_clean_scrub_stamp":"2023-10-06T03:33:09.448931+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:58:57.374506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":177,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-06T05:07:51.874916+0000","last_change":"2023-10-06T03:33:09.584785+0000","last_active":"2023-10-06T05:07:51.874916+0000","last_peered":"2023-10-06T05:07:51.874916+0000","last_clean":"2023-10-06T05:07:51.874916+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T05:07:51.874916+0000","last_undegraded":"2023-10-06T05:07:51.874916+0000","last_fullsized":"2023-10-06T05:07:51.874916+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:10:58.685119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":177,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-06T05:07:51.874328+0000","last_change":"2023-10-06T03:33:09.584777+0000","last_active":"2023-10-06T05:07:51.874328+0000","last_peered":"2023-10-06T05:07:51.874328+0000","last_clean":"2023-10-06T05:07:51.874328+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T05:07:51.874328+0000","last_undegraded":"2023-10-06T05:07:51.874328+0000","last_fullsized":"2023-10-06T05:07:51.874328+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":170,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-06T05:07:51.872345+0000","last_change":"2023-10-06T03:33:09.584762+0000","last_active":"2023-10-06T05:07:51.872345+0000","last_peered":"2023-10-06T05:07:51.872345+0000","last_clean":"2023-10-06T05:07:51.872345+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T05:07:51.872345+0000","last_undegraded":"2023-10-06T05:07:51.872345+0000","last_fullsized":"2023-10-06T05:07:51.872345+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:55:14.096370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":205,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-06T05:07:51.336687+0000","last_change":"2023-10-06T03:33:09.585012+0000","last_active":"2023-10-06T05:07:51.336687+0000","last_peered":"2023-10-06T05:07:51.336687+0000","last_clean":"2023-10-06T05:07:51.336687+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T05:07:51.336687+0000","last_undegraded":"2023-10-06T05:07:51.336687+0000","last_fullsized":"2023-10-06T05:07:51.336687+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:20:54.244187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":205,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-06T05:07:51.336912+0000","last_change":"2023-10-06T03:33:09.468293+0000","last_active":"2023-10-06T05:07:51.336912+0000","last_peered":"2023-10-06T05:07:51.336912+0000","last_clean":"2023-10-06T05:07:51.336912+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T05:07:51.336912+0000","last_undegraded":"2023-10-06T05:07:51.336912+0000","last_fullsized":"2023-10-06T05:07:51.336912+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T03:33:08.446317+0000","last_clean_scrub_stamp":"2023-10-06T03:33:08.446317+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:15:05.124967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575970,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304560,"kb_used_data":3176,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067280,"statfs":{"total":96636764160,"available":96324894720,"internally_reserved":0,"allocated":3252224,"data_stored":1997546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94999999999999996,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0529999999999999},"last":0.70499999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79300000000000004,"15min":0.80600000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.996,"5min":1.0640000000000001,"15min":1.073},"last":0.746},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.80300000000000005,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.0860000000000001},"last":0.73499999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72199999999999998,"15min":0.74099999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.52500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0509999999999999,"5min":1.107,"15min":1.1160000000000001},"last":0.83899999999999997}]},{"osd":3,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.101},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.753,"15min":0.76600000000000001},"min":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96099999999999997,"5min":1.0149999999999999,"15min":1.0369999999999999},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.355},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64000000000000001,"15min":0.64500000000000002},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.40999999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.208},"last":0.55900000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.497},{"interface":"front","average":{"1min":0.751,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47699999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.754,"15min":0.749},"min":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.48799999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.51500000000000001,"5min":0.48999999999999999,"15min":0.46700000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.60299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575971,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299888,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071952,"statfs":{"total":96636764160,"available":96329678848,"internally_reserved":0,"allocated":2662400,"data_stored":1408156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.54000000000000004,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92700000000000005,"5min":1.0009999999999999,"15min":1.2969999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.48399999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.94999999999999996,"5min":1.083,"15min":1.083},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0640000000000001,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.754,"15min":0.76000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.0960000000000001,"5min":1.425,"15min":1.425},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.752,"15min":0.76100000000000001},"min":{"1min":0.48099999999999998,"5min":0.47299999999999998,"15min":0.44700000000000001},"max":{"1min":0.91800000000000004,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.76000000000000001,"15min":0.77000000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96499999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78200000000000003,"15min":0.78600000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":0.998,"5min":1.016,"15min":1.1699999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.80100000000000005,"15min":0.79800000000000004},"min":{"1min":0.61299999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.94599999999999995,"5min":1.0409999999999999,"15min":1.1479999999999999},"last":0.75800000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.40200000000000002,"15min":0.38900000000000001},"max":{"1min":0.81399999999999995,"5min":0.95999999999999996,"15min":1.0269999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66100000000000003,"15min":0.64400000000000002},"min":{"1min":0.51800000000000002,"5min":0.46600000000000003,"15min":0.39300000000000002},"max":{"1min":0.89400000000000002,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70199999999999996,"15min":0.70799999999999996},"min":{"1min":0.51400000000000001,"5min":0.5,"15min":0.41699999999999998},"max":{"1min":0.94199999999999995,"5min":1.02,"15min":1.069},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.41199999999999998},"max":{"1min":0.94199999999999995,"5min":1.0349999999999999,"15min":1.1950000000000001},"last":0.77400000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71799999999999997,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93999999999999995,"5min":1.208,"15min":1.208},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.48599999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.97399999999999998,"5min":1.0449999999999999,"15min":1.1579999999999999},"last":0.79000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575970,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316852,"kb_used_data":3180,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94054988,"statfs":{"total":96636764160,"available":96312307712,"internally_reserved":0,"allocated":3256320,"data_stored":1998524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.32700000000000001},"max":{"1min":1.093,"5min":1.155,"15min":1.155},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.59299999999999997},"min":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.38900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.90100000000000002},"last":0.503}]},{"osd":2,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.72699999999999998,"5min":0.76400000000000001,"15min":1.153},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.42199999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1,"5min":1.018,"15min":1.018},"last":0.54500000000000004}]},{"osd":3,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.435,"15min":0.41799999999999998},"max":{"1min":0.95499999999999996,"5min":1.113,"15min":1.131},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92200000000000004,"5min":0.95499999999999996,"15min":0.96999999999999997},"last":0.56999999999999995}]},{"osd":4,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79600000000000004,"15min":0.80600000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.02,"5min":1.2210000000000001,"15min":1.2250000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.82199999999999995,"15min":0.84299999999999997},"min":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.147,"5min":1.1930000000000001,"15min":1.236},"last":0.77500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.82699999999999996,"15min":0.83999999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.53600000000000003},"max":{"1min":1.0660000000000001,"5min":1.1240000000000001,"15min":1.228},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.51000000000000001},"max":{"1min":1.139,"5min":1.1799999999999999,"15min":1.2829999999999999},"last":0.78400000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85199999999999998,"15min":0.85699999999999998},"min":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.59199999999999997},"max":{"1min":1.1779999999999999,"5min":1.2490000000000001,"15min":1.2709999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.85999999999999999,"15min":0.86399999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.163,"5min":1.242,"15min":1.2470000000000001},"last":0.748}]},{"osd":7,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.63,"5min":0.623,"15min":0.623},"max":{"1min":1.198,"5min":1.222,"15min":1.2949999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.872,"15min":0.88},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.224,"5min":1.224,"15min":1.3080000000000001},"last":0.80200000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575972,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303980,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067860,"statfs":{"total":96636764160,"available":96325488640,"internally_reserved":0,"allocated":2658304,"data_stored":1407178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.28599999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.35999999999999999,"15min":0.34399999999999997},"max":{"1min":0.67600000000000005,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.75600000000000001}]},{"osd":2,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55500000000000005,"15min":0.57299999999999995},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.38700000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.91000000000000003},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.57999999999999996},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.44900000000000001}]},{"osd":3,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.39700000000000002,"5min":0.39200000000000002,"15min":0.36299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.86699999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.63700000000000001,"15min":0.64700000000000002},"min":{"1min":0.49299999999999999,"5min":0.439,"15min":0.42899999999999999},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.60399999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.67100000000000004,"5min":0.57899999999999996,"15min":0.54300000000000004},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.3049999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.85399999999999998,"15min":0.86399999999999999},"min":{"1min":0.63600000000000001,"5min":0.55700000000000005,"15min":0.55300000000000005},"max":{"1min":1.085,"5min":1.127,"15min":1.3440000000000001},"last":0.94899999999999995}]},{"osd":5,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.83499999999999996,"15min":0.84399999999999997},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.2729999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82899999999999996,"15min":0.85099999999999998},"min":{"1min":0.623,"5min":0.57399999999999995,"15min":0.45600000000000002},"max":{"1min":1.1339999999999999,"5min":1.204,"15min":1.218},"last":0.66100000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81999999999999995,"15min":0.83799999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.53800000000000003},"max":{"1min":1.0529999999999999,"5min":1.1240000000000001,"15min":1.2869999999999999},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86199999999999999,"15min":0.88300000000000001},"min":{"1min":0.67700000000000005,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.246,"5min":1.246,"15min":1.4399999999999999},"last":0.77800000000000002}]},{"osd":7,"last update":"Fri Oct 6 05:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.85399999999999998,"15min":0.876},"min":{"1min":0.626,"5min":0.624,"15min":0.59799999999999998},"max":{"1min":1.216,"5min":1.2490000000000001,"15min":1.367},"last":0.62},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.68300000000000005,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.113,"5min":1.113,"15min":1.294},"last":1.0820000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575971,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299884,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071956,"statfs":{"total":96636764160,"available":96329682944,"internally_reserved":0,"allocated":2658304,"data_stored":1407178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47299999999999998,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.32000000000000001,"15min":0.312},"max":{"1min":0.73699999999999999,"5min":0.746,"15min":0.80300000000000005},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.54400000000000004,"15min":0.55500000000000005},"min":{"1min":0.35299999999999998,"5min":0.34000000000000002,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.91200000000000003},"last":0.42199999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.50800000000000001,"15min":0.503},"min":{"1min":0.42499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.73899999999999999,"5min":0.78300000000000003,"15min":0.92400000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.46000000000000002,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.60899999999999999,"15min":0.60699999999999998},"min":{"1min":0.51300000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57699999999999996,"15min":0.59299999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.32400000000000001},"max":{"1min":0.76100000000000001,"5min":0.89800000000000002,"15min":0.90600000000000003},"last":0.59699999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.84799999999999998,"15min":0.85399999999999998},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0580000000000001,"5min":1.286,"15min":1.286},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.873,"15min":0.88100000000000001},"min":{"1min":0.623,"5min":0.623,"15min":0.59199999999999997},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.254},"last":0.67000000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.88400000000000001,"15min":0.88800000000000001},"min":{"1min":0.67100000000000004,"5min":0.59099999999999997,"15min":0.57599999999999996},"max":{"1min":1.165,"5min":1.26,"15min":1.345},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.89000000000000001,"15min":0.89700000000000002},"min":{"1min":0.65500000000000003,"5min":0.628,"15min":0.628},"max":{"1min":1.085,"5min":1.2090000000000001,"15min":1.339},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.85199999999999998,"15min":0.86599999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58199999999999996},"max":{"1min":0.997,"5min":1.1279999999999999,"15min":1.2989999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84899999999999998,"15min":0.86099999999999999},"min":{"1min":0.63400000000000001,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.1439999999999999,"5min":1.226,"15min":1.226},"last":0.75600000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:07:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85999999999999999,"15min":0.86599999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.1000000000000001,"5min":1.141,"15min":1.276},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.90700000000000003,"15min":0.90900000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.59399999999999997},"max":{"1min":1.1850000000000001,"5min":1.2170000000000001,"15min":1.3680000000000001},"last":0.621}]}]},{"osd":3,"up_from":13,"seq":55834575971,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299888,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071952,"statfs":{"total":96636764160,"available":96329678848,"internally_reserved":0,"allocated":2662400,"data_stored":1408156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.505,"15min":0.49399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59999999999999998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.50900000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.31900000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.64900000000000002,"5min":0.80500000000000005,"15min":0.96099999999999997},"last":0.442},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.46000000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.58199999999999996,"15min":0.59499999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70699999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.38500000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.81200000000000006,"15min":0.82099999999999995},"min":{"1min":0.48699999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.01,"5min":1.232,"15min":1.2849999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.83799999999999997,"15min":0.86099999999999999},"min":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.52500000000000002},"max":{"1min":1.0509999999999999,"5min":1.232,"15min":1.4019999999999999},"last":0.64400000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.79200000000000004,"15min":0.81299999999999994},"min":{"1min":0.52900000000000003,"5min":0.52400000000000002,"15min":0.52200000000000002},"max":{"1min":0.92000000000000004,"5min":1.22,"15min":1.5760000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.121,"5min":1.1419999999999999,"15min":1.3300000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Oct 6 05:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.82499999999999996,"15min":0.84499999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.087,"5min":1.212,"15min":1.2150000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.86399999999999999,"15min":0.88400000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.111,"5min":1.2929999999999999,"15min":1.321},"last":0.68300000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.84299999999999997,"15min":0.86399999999999999},"min":{"1min":0.622,"5min":0.59399999999999997,"15min":0.57599999999999996},"max":{"1min":1.145,"5min":1.198,"15min":1.2769999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.83999999999999997,"15min":0.84899999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.155,"5min":1.1890000000000001,"15min":1.389},"last":0.71699999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575971,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312176,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059664,"statfs":{"total":96636764160,"available":96317095936,"internally_reserved":0,"allocated":2662400,"data_stored":1408156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.747},"min":{"1min":0.53800000000000003,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.89500000000000002,"5min":1.0820000000000001,"15min":1.2},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.753,"15min":0.755},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97599999999999998,"5min":1.048,"15min":1.048},"last":0.876}]},{"osd":1,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72299999999999998,"15min":0.73699999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.96199999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94299999999999995,"5min":0.99099999999999999,"15min":1.258},"last":0.77400000000000002}]},{"osd":2,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.50800000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75900000000000001,"15min":0.78300000000000003},"min":{"1min":0.50600000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.81499999999999995}]},{"osd":3,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.75700000000000001},"min":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.49299999999999999},"max":{"1min":0.97499999999999998,"5min":0.97999999999999998,"15min":1.2330000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.54900000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.95599999999999996,"5min":0.96799999999999997,"15min":1.034},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61399999999999999,"15min":0.63400000000000001},"min":{"1min":0.46000000000000002,"5min":0.43099999999999999,"15min":0.41199999999999998},"max":{"1min":0.81599999999999995,"5min":1.04,"15min":1.323},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.65200000000000002},"min":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.39700000000000002},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.1910000000000001},"last":0.66300000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.67700000000000005},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":1.024,"15min":1.079},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68799999999999994,"15min":0.71199999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.41999999999999998},"max":{"1min":0.84699999999999998,"5min":0.95099999999999996,"15min":1.3540000000000001},"last":0.86099999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.70599999999999996},"min":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.82699999999999996,"5min":1.0640000000000001,"15min":1.155},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.47299999999999998},"max":{"1min":0.93000000000000005,"5min":1.079,"15min":1.149},"last":0.89300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575971,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308076,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063764,"statfs":{"total":96636764160,"available":96321294336,"internally_reserved":0,"allocated":2658304,"data_stored":1407178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75700000000000001,"15min":0.748},"min":{"1min":0.53200000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.93500000000000005,"5min":1.079,"15min":1.079},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73499999999999999,"15min":0.72999999999999998},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.88800000000000001,"5min":0.999,"15min":1.02},"last":0.60899999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.745,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97799999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.747,"15min":0.76500000000000001},"min":{"1min":0.53600000000000003,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.98399999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.67800000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79600000000000004,"15min":0.79400000000000004},"min":{"1min":0.65300000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.99199999999999999,"5min":1.0700000000000001,"15min":1.0780000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80100000000000005,"15min":0.80700000000000005},"min":{"1min":0.57599999999999996,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0149999999999999,"5min":1.048,"15min":1.048},"last":0.70899999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.48599999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.94399999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.78200000000000003},"min":{"1min":0.63500000000000001,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0169999999999999,"5min":1.095,"15min":1.095},"last":0.54300000000000004}]},{"osd":4,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.629},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.38200000000000001},"max":{"1min":0.84399999999999997,"5min":0.97699999999999998,"15min":1.0049999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.64200000000000002},"min":{"1min":0.5,"5min":0.38,"15min":0.38},"max":{"1min":0.89000000000000001,"5min":1.01,"15min":1.042},"last":0.66300000000000003}]},{"osd":6,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.48499999999999999,"5min":0.45900000000000002,"15min":0.45100000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.0369999999999999},"last":0.75},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.42799999999999999},"max":{"1min":0.90700000000000003,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Fri Oct 6 05:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.68999999999999995},"min":{"1min":0.55000000000000004,"5min":0.48499999999999999,"15min":0.44500000000000001},"max":{"1min":0.92600000000000005,"5min":1.0269999999999999,"15min":1.0669999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.54000000000000004,"5min":0.51300000000000001,"15min":0.47199999999999998},"max":{"1min":0.93000000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.80500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:08:04.265 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-10-06T03:33:08.446317+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=33, tm_sec=8, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=7, tm_sec=57, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:08:04.266 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-10-06T03:33:08.446317+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=33, tm_sec=8, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=7, tm_sec=57, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:08:04.266 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-10-06T03:33:08.446317+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=33, tm_sec=8, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=7, tm_sec=57, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:08:04.266 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T03:33:09.448931+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=33, tm_sec=9, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=7, tm_sec=57, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:08:04.266 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-10-06T03:33:08.446317+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=33, tm_sec=8, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=7, tm_sec=57, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:08:04.266 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T03:33:08.446317+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=33, tm_sec=8, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=7, tm_sec=57, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:08:04.266 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-10-06T03:33:08.446317+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=33, tm_sec=8, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=7, tm_sec=57, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:08:04.266 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-10-06T03:33:08.446317+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=33, tm_sec=8, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=7, tm_sec=57, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:08:04.267 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-10-06T03:33:08.446317+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=3, tm_min=33, tm_sec=8, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=5, tm_min=7, tm_sec=57, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T05:08:04.267 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T05:08:09.138 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:08:09.143 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:08:24.268 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T05:08:24.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.467+0000 7f2a9579d700 1 Processor -- start 2023-10-06T05:08:24.469 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.467+0000 7f2a9579d700 1 -- start start 2023-10-06T05:08:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.469+0000 7f2a9579d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a90156310 0x7f2a900c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.469+0000 7f2a9579d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2a900ae710 0x7f2a900c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.469+0000 7f2a9579d700 1 -- --> v1:172.21.15.55:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a900ca860 con 0x7f2a901566e0 2023-10-06T05:08:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.469+0000 7f2a9579d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a900cab40 con 0x7f2a900ae710 2023-10-06T05:08:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.469+0000 7f2a9579d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a900cae20 con 0x7f2a90156310 2023-10-06T05:08:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.469+0000 7f2a94f9c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f2a900ae710 0x7f2a900c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.469+0000 7f2a8f7fe700 1 --1- >> v1:172.21.15.55:6789/0 conn(0x7f2a901566e0 0x7f2a900ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.55:6789/0 says I am v1:172.21.15.55:57904/0 (socket says 172.21.15.55:57904) 2023-10-06T05:08:24.471 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.469+0000 7f2a8f7fe700 1 -- 172.21.15.55:0/2395770201 learned_addr learned my addr 172.21.15.55:0/2395770201 (peer_addr_for_me v1:172.21.15.55:0/0) 2023-10-06T05:08:24.472 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.469+0000 7f2a8ffff700 1 --2- 172.21.15.55:0/2395770201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a90156310 0x7f2a900c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:24.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.470+0000 7f2a94f9c700 1 -- 172.21.15.55:0/2395770201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a90156310 msgr2=0x7f2a900c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:24.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.470+0000 7f2a94f9c700 1 --2- 172.21.15.55:0/2395770201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a90156310 0x7f2a900c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.470+0000 7f2a94f9c700 1 -- 172.21.15.55:0/2395770201 >> v1:172.21.15.55:6789/0 conn(0x7f2a901566e0 legacy=0x7f2a900ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:24.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.470+0000 7f2a94f9c700 1 -- 172.21.15.55:0/2395770201 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a900cb100 con 0x7f2a900ae710 2023-10-06T05:08:24.473 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.470+0000 7f2a8effd700 1 -- 172.21.15.55:0/2395770201 <== mon.0 v1:172.21.15.55:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116535650 0 0) 0x7f2a900ca860 con 0x7f2a901566e0 2023-10-06T05:08:24.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.470+0000 7f2a94f9c700 1 --2- 172.21.15.55:0/2395770201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a900ae710 0x7f2a900c6830 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009e60 tx=0x7f2a88009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62b04b02bd87492 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:24.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.470+0000 7f2a8effd700 1 -- 172.21.15.55:0/2395770201 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8800e070 con 0x7f2a900ae710 2023-10-06T05:08:24.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.470+0000 7f2a8effd700 1 -- 172.21.15.55:0/2395770201 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8800bb60 con 0x7f2a900ae710 2023-10-06T05:08:24.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.470+0000 7f2a8effd700 1 -- 172.21.15.55:0/2395770201 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a88012430 con 0x7f2a900ae710 2023-10-06T05:08:24.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.471+0000 7f2a9579d700 1 -- 172.21.15.55:0/2395770201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a900ae710 msgr2=0x7f2a900c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:24.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.471+0000 7f2a9579d700 1 --2- 172.21.15.55:0/2395770201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a900ae710 0x7f2a900c6830 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009e60 tx=0x7f2a88009780 comp rx=0 tx=0).stop 2023-10-06T05:08:24.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.471+0000 7f2a9579d700 1 -- 172.21.15.55:0/2395770201 shutdown_connections 2023-10-06T05:08:24.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.471+0000 7f2a9579d700 1 --2- 172.21.15.55:0/2395770201 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a900ae710 0x7f2a900c6830 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.471+0000 7f2a9579d700 1 --2- 172.21.15.55:0/2395770201 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a90156310 0x7f2a900c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.471+0000 7f2a9579d700 1 -- 172.21.15.55:0/2395770201 >> 172.21.15.55:0/2395770201 conn(0x7f2a900cc1f0 msgr2=0x7f2a900cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:24.474 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.471+0000 7f2a9579d700 1 -- 172.21.15.55:0/2395770201 shutdown_connections 2023-10-06T05:08:24.475 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.471+0000 7f2a9579d700 1 -- 172.21.15.55:0/2395770201 wait complete. 2023-10-06T05:08:24.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.472+0000 7f2a9579d700 1 Processor -- start 2023-10-06T05:08:24.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.472+0000 7f2a9579d700 1 -- start start 2023-10-06T05:08:24.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.473+0000 7f2a9579d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a900ae710 0x7f2a90153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:24.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.473+0000 7f2a9579d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a90156310 0x7f2a90148900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:24.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.473+0000 7f2a9579d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a901566e0 0x7f2a90148e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:24.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.473+0000 7f2a9579d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9014d5a0 con 0x7f2a90156310 2023-10-06T05:08:24.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.473+0000 7f2a9579d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9014ccb0 con 0x7f2a900ae710 2023-10-06T05:08:24.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.473+0000 7f2a9579d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9014cf50 con 0x7f2a901566e0 2023-10-06T05:08:24.477 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.474+0000 7f2a8f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a90156310 0x7f2a90148900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:24.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.474+0000 7f2a8f7fe700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a90156310 0x7f2a90148900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40828/0 (socket says 172.21.15.55:40828) 2023-10-06T05:08:24.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.474+0000 7f2a8f7fe700 1 -- 172.21.15.55:0/2786971394 learned_addr learned my addr 172.21.15.55:0/2786971394 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:24.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.474+0000 7f2a8ffff700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a900ae710 0x7f2a90153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:24.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.474+0000 7f2a94f9c700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a901566e0 0x7f2a90148e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:24.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.474+0000 7f2a8f7fe700 1 -- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a901566e0 msgr2=0x7f2a90148e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:24.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.474+0000 7f2a8f7fe700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a901566e0 0x7f2a90148e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.474+0000 7f2a8f7fe700 1 -- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a900ae710 msgr2=0x7f2a90153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:24.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.474+0000 7f2a8f7fe700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a900ae710 0x7f2a90153b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.474+0000 7f2a8f7fe700 1 -- 172.21.15.55:0/2786971394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a88009a20 con 0x7f2a90156310 2023-10-06T05:08:24.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.474+0000 7f2a94f9c700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a901566e0 0x7f2a90148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:08:24.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.474+0000 7f2a8ffff700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a900ae710 0x7f2a90153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:08:24.478 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.474+0000 7f2a8f7fe700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a90156310 0x7f2a90148900 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400c810 tx=0x7f2a84007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:24.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.475+0000 7f2a77fff700 1 -- 172.21.15.55:0/2786971394 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84005c00 con 0x7f2a90156310 2023-10-06T05:08:24.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.475+0000 7f2a77fff700 1 -- 172.21.15.55:0/2786971394 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a84005d60 con 0x7f2a90156310 2023-10-06T05:08:24.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.475+0000 7f2a77fff700 1 -- 172.21.15.55:0/2786971394 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84007320 con 0x7f2a90156310 2023-10-06T05:08:24.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.475+0000 7f2a9579d700 1 -- 172.21.15.55:0/2786971394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9014d230 con 0x7f2a90156310 2023-10-06T05:08:24.479 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.475+0000 7f2a9579d700 1 -- 172.21.15.55:0/2786971394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a901540d0 con 0x7f2a90156310 2023-10-06T05:08:24.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.476+0000 7f2a9579d700 1 -- 172.21.15.55:0/2786971394 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9004c3c0 con 0x7f2a90156310 2023-10-06T05:08:24.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.477+0000 7f2a77fff700 1 -- 172.21.15.55:0/2786971394 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a8401b400 con 0x7f2a90156310 2023-10-06T05:08:24.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.478+0000 7f2a77fff700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2a7c066f30 0x7f2a7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:24.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.478+0000 7f2a77fff700 1 -- 172.21.15.55:0/2786971394 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f2a840843d0 con 0x7f2a90156310 2023-10-06T05:08:24.480 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.478+0000 7f2a8ffff700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2a7c066f30 0x7f2a7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:24.481 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.479+0000 7f2a8ffff700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2a7c066f30 0x7f2a7c0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009d80 tx=0x7f2a80008040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:24.484 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.482+0000 7f2a77fff700 1 -- 172.21.15.55:0/2786971394 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a840513d0 con 0x7f2a90156310 2023-10-06T05:08:24.681 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.679+0000 7f2a9579d700 1 -- 172.21.15.55:0/2786971394 --> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a9014d6e0 con 0x7f2a7c066f30 2023-10-06T05:08:24.685 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.683+0000 7f2a77fff700 1 -- 172.21.15.55:0/2786971394 <== mgr.10033 v2:172.21.15.163:6836/101089 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72755 (secure 0 0 0) 0x7f2a9014d6e0 con 0x7f2a7c066f30 2023-10-06T05:08:24.686 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:08:24.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.685+0000 7f2a9579d700 1 -- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2a7c066f30 msgr2=0x7f2a7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:24.687 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.685+0000 7f2a9579d700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2a7c066f30 0x7f2a7c0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009d80 tx=0x7f2a80008040 comp rx=0 tx=0).stop 2023-10-06T05:08:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.685+0000 7f2a9579d700 1 -- 172.21.15.55:0/2786971394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a90156310 msgr2=0x7f2a90148900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.685+0000 7f2a9579d700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a90156310 0x7f2a90148900 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400c810 tx=0x7f2a84007db0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.685+0000 7f2a9579d700 1 -- 172.21.15.55:0/2786971394 shutdown_connections 2023-10-06T05:08:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.685+0000 7f2a9579d700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f2a7c066f30 0x7f2a7c0693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.685+0000 7f2a9579d700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f2a901566e0 0x7f2a90148e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.685+0000 7f2a9579d700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f2a900ae710 0x7f2a90153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.685+0000 7f2a9579d700 1 --2- 172.21.15.55:0/2786971394 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f2a90156310 0x7f2a90148900 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.685+0000 7f2a9579d700 1 -- 172.21.15.55:0/2786971394 >> 172.21.15.55:0/2786971394 conn(0x7f2a900cc1f0 msgr2=0x7f2a900addf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.685+0000 7f2a9579d700 1 -- 172.21.15.55:0/2786971394 shutdown_connections 2023-10-06T05:08:24.688 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.685+0000 7f2a9579d700 1 -- 172.21.15.55:0/2786971394 wait complete. 2023-10-06T05:08:24.689 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2023-10-06T05:08:24.704 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-06T05:08:23.339644+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":371,"ondisk_log_size":371,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2445304,"kb_used_data":21944,"kb_used_omap":0,"kb_used_meta":2423296,"kb_avail":752529416,"statfs":{"total":773094113280,"available":770590121984,"internally_reserved":0,"allocated":22470656,"data_stored":12442072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2481455104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002595"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":204,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-10-06T05:08:01.923718+0000","last_change":"2023-10-06T05:08:01.923658+0000","last_active":"2023-10-06T05:08:01.923718+0000","last_peered":"2023-10-06T05:08:01.923718+0000","last_clean":"2023-10-06T05:08:01.923718+0000","last_became_active":"2023-10-06T03:33:09.491928+0000","last_became_peered":"2023-10-06T03:33:09.491928+0000","last_unstale":"2023-10-06T05:08:01.923718+0000","last_undegraded":"2023-10-06T05:08:01.923718+0000","last_fullsized":"2023-10-06T05:08:01.923718+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:08:01.923558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:08:01.923558+0000","last_clean_scrub_stamp":"2023-10-06T05:08:01.923558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:46.527496+0000","scrub_duration":0.002055622,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":191,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-10-06T05:08:01.888056+0000","last_change":"2023-10-06T05:08:01.888029+0000","last_active":"2023-10-06T05:08:01.888056+0000","last_peered":"2023-10-06T05:08:01.888056+0000","last_clean":"2023-10-06T05:08:01.888056+0000","last_became_active":"2023-10-06T03:33:09.473295+0000","last_became_peered":"2023-10-06T03:33:09.473295+0000","last_unstale":"2023-10-06T05:08:01.888056+0000","last_undegraded":"2023-10-06T05:08:01.888056+0000","last_fullsized":"2023-10-06T05:08:01.888056+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:08:01.887949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:08:01.887949+0000","last_clean_scrub_stamp":"2023-10-06T05:08:01.887949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:05:46.491887+0000","scrub_duration":0.0024337769999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":186,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-10-06T05:08:04.879655+0000","last_change":"2023-10-06T05:08:04.879621+0000","last_active":"2023-10-06T05:08:04.879655+0000","last_peered":"2023-10-06T05:08:04.879655+0000","last_clean":"2023-10-06T05:08:04.879655+0000","last_became_active":"2023-10-06T03:33:09.468123+0000","last_became_peered":"2023-10-06T03:33:09.468123+0000","last_unstale":"2023-10-06T05:08:04.879655+0000","last_undegraded":"2023-10-06T05:08:04.879655+0000","last_fullsized":"2023-10-06T05:08:04.879655+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:08:04.879513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:08:04.879513+0000","last_clean_scrub_stamp":"2023-10-06T05:08:04.879513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:08:58.039260+0000","scrub_duration":0.002428319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"212'371","reported_seq":1235,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-10-06T05:08:09.175419+0000","last_change":"2023-10-06T05:08:09.175371+0000","last_active":"2023-10-06T05:08:09.175419+0000","last_peered":"2023-10-06T05:08:09.175419+0000","last_clean":"2023-10-06T05:08:09.175419+0000","last_became_active":"2023-10-06T03:33:10.471367+0000","last_became_peered":"2023-10-06T03:33:10.471367+0000","last_unstale":"2023-10-06T05:08:09.175419+0000","last_undegraded":"2023-10-06T05:08:09.175419+0000","last_fullsized":"2023-10-06T05:08:09.175419+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"212'371","last_scrub_stamp":"2023-10-06T05:08:09.175271+0000","last_deep_scrub":"212'371","last_deep_scrub_stamp":"2023-10-06T05:08:09.175271+0000","last_clean_scrub_stamp":"2023-10-06T05:08:09.175271+0000","objects_scrubbed":2,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:42:40.076633+0000","scrub_duration":0.0070001630000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":188,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-10-06T05:08:06.116977+0000","last_change":"2023-10-06T05:08:06.116923+0000","last_active":"2023-10-06T05:08:06.116977+0000","last_peered":"2023-10-06T05:08:06.116977+0000","last_clean":"2023-10-06T05:08:06.116977+0000","last_became_active":"2023-10-06T03:33:09.584484+0000","last_became_peered":"2023-10-06T03:33:09.584484+0000","last_unstale":"2023-10-06T05:08:06.116977+0000","last_undegraded":"2023-10-06T05:08:06.116977+0000","last_fullsized":"2023-10-06T05:08:06.116977+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:08:06.116816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:08:06.116816+0000","last_clean_scrub_stamp":"2023-10-06T05:08:06.116816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:39:59.288796+0000","scrub_duration":0.0025920180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":188,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-10-06T05:08:05.119603+0000","last_change":"2023-10-06T05:08:05.119572+0000","last_active":"2023-10-06T05:08:05.119603+0000","last_peered":"2023-10-06T05:08:05.119603+0000","last_clean":"2023-10-06T05:08:05.119603+0000","last_became_active":"2023-10-06T03:33:09.584436+0000","last_became_peered":"2023-10-06T03:33:09.584436+0000","last_unstale":"2023-10-06T05:08:05.119603+0000","last_undegraded":"2023-10-06T05:08:05.119603+0000","last_fullsized":"2023-10-06T05:08:05.119603+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:08:05.119448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:08:05.119448+0000","last_clean_scrub_stamp":"2023-10-06T05:08:05.119448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:17:55.790571+0000","scrub_duration":0.0023823999999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":181,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-10-06T05:08:07.237133+0000","last_change":"2023-10-06T05:08:07.237062+0000","last_active":"2023-10-06T05:08:07.237133+0000","last_peered":"2023-10-06T05:08:07.237133+0000","last_clean":"2023-10-06T05:08:07.237133+0000","last_became_active":"2023-10-06T03:33:09.584448+0000","last_became_peered":"2023-10-06T03:33:09.584448+0000","last_unstale":"2023-10-06T05:08:07.237133+0000","last_undegraded":"2023-10-06T05:08:07.237133+0000","last_fullsized":"2023-10-06T05:08:07.237133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:08:07.236954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:08:07.236954+0000","last_clean_scrub_stamp":"2023-10-06T05:08:07.236954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:50.500173+0000","scrub_duration":0.0028745950000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":216,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-10-06T05:08:03.857106+0000","last_change":"2023-10-06T05:08:03.857061+0000","last_active":"2023-10-06T05:08:03.857106+0000","last_peered":"2023-10-06T05:08:03.857106+0000","last_clean":"2023-10-06T05:08:03.857106+0000","last_became_active":"2023-10-06T03:33:09.584564+0000","last_became_peered":"2023-10-06T03:33:09.584564+0000","last_unstale":"2023-10-06T05:08:03.857106+0000","last_undegraded":"2023-10-06T05:08:03.857106+0000","last_fullsized":"2023-10-06T05:08:03.857106+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:08:03.856959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:08:03.856959+0000","last_clean_scrub_stamp":"2023-10-06T05:08:03.856959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:16:17.755089+0000","scrub_duration":0.001950465,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":216,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-10-06T05:08:05.803066+0000","last_change":"2023-10-06T05:08:05.803014+0000","last_active":"2023-10-06T05:08:05.803066+0000","last_peered":"2023-10-06T05:08:05.803066+0000","last_clean":"2023-10-06T05:08:05.803066+0000","last_became_active":"2023-10-06T03:33:09.467954+0000","last_became_peered":"2023-10-06T03:33:09.467954+0000","last_unstale":"2023-10-06T05:08:05.803066+0000","last_undegraded":"2023-10-06T05:08:05.803066+0000","last_fullsized":"2023-10-06T05:08:05.803066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T05:08:05.802903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T05:08:05.802903+0000","last_clean_scrub_stamp":"2023-10-06T05:08:05.802903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:44:34.146717+0000","scrub_duration":0.0029377259999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":371,"ondisk_log_size":371,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575974,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304560,"kb_used_data":3176,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067280,"statfs":{"total":96636764160,"available":96324894720,"internally_reserved":0,"allocated":3252224,"data_stored":1997546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71499999999999997,"15min":0.73199999999999998},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94999999999999996,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0529999999999999},"last":0.77800000000000002}]},{"osd":1,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.79300000000000004,"15min":0.80600000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.996,"5min":1.0640000000000001,"15min":1.073},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.80300000000000005,"15min":0.80900000000000005},"min":{"1min":0.58699999999999997,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.042,"5min":1.042,"15min":1.0860000000000001},"last":0.86399999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72199999999999998,"15min":0.74099999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.76500000000000001,"15min":0.77800000000000002},"min":{"1min":0.52500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0509999999999999,"5min":1.107,"15min":1.1160000000000001},"last":0.82899999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.101},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.753,"15min":0.76600000000000001},"min":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.96099999999999997,"5min":1.0149999999999999,"15min":1.0369999999999999},"last":0.749}]},{"osd":4,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.40899999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.355},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64000000000000001,"15min":0.64500000000000002},"min":{"1min":0.44600000000000001,"5min":0.42899999999999999,"15min":0.40999999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.208},"last":0.58899999999999997}]},{"osd":5,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.68799999999999994,"15min":0.69699999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47699999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.754,"15min":0.749},"min":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.48799999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.51500000000000001,"5min":0.48999999999999999,"15min":0.46700000000000003},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.69599999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575975,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299888,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071952,"statfs":{"total":96636764160,"available":96329678848,"internally_reserved":0,"allocated":2662400,"data_stored":1408156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.68999999999999995,"15min":0.71199999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74199999999999999,"5min":1.0009999999999999,"15min":1.2969999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.72199999999999998,"15min":0.747},"min":{"1min":0.47099999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84799999999999998,"5min":1.083,"15min":1.083},"last":1.0569999999999999}]},{"osd":1,"last update":"Fri Oct 6 05:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.73099999999999998,"15min":0.749},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86899999999999999,"5min":1.0640000000000001,"15min":1.3069999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.72199999999999998,"15min":0.752},"min":{"1min":0.50600000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.78200000000000003,"5min":1.0960000000000001,"15min":1.425},"last":0.79700000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.47499999999999998,"5min":0.47299999999999998,"15min":0.44700000000000001},"max":{"1min":0.85299999999999998,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.73399999999999999,"15min":0.76300000000000001},"min":{"1min":0.46800000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.95899999999999996,"5min":1.03,"15min":1.0640000000000001},"last":1.0309999999999999}]},{"osd":3,"last update":"Fri Oct 6 05:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.76700000000000002,"15min":0.78200000000000003},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.86299999999999999,"5min":1.016,"15min":1.1699999999999999},"last":0.995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.78300000000000003,"15min":0.79300000000000004},"min":{"1min":0.53900000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.996,"5min":1.0409999999999999,"15min":1.1479999999999999},"last":1.097}]},{"osd":4,"last update":"Fri Oct 6 05:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.47099999999999997,"5min":0.40200000000000002,"15min":0.38900000000000001},"max":{"1min":0.77000000000000002,"5min":0.95999999999999996,"15min":1.0269999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.51300000000000001,"5min":0.46600000000000003,"15min":0.39300000000000002},"max":{"1min":0.93999999999999995,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Oct 6 05:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69099999999999995,"15min":0.71099999999999997},"min":{"1min":0.55300000000000005,"5min":0.5,"15min":0.41699999999999998},"max":{"1min":0.876,"5min":0.98899999999999999,"15min":1.069},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.72299999999999998,"15min":0.73399999999999999},"min":{"1min":0.60799999999999998,"5min":0.432,"15min":0.41199999999999998},"max":{"1min":0.83299999999999996,"5min":1.0349999999999999,"15min":1.1950000000000001},"last":0.84599999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:08:07 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72599999999999998,"15min":0.71599999999999997},"min":{"1min":0.56799999999999995,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0469999999999999,"5min":1.208,"15min":1.208},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73499999999999999,"15min":0.745},"min":{"1min":0.63100000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.91500000000000004,"5min":1.0449999999999999,"15min":1.1579999999999999},"last":0.76000000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316852,"kb_used_data":3180,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94054988,"statfs":{"total":96636764160,"available":96312307712,"internally_reserved":0,"allocated":3256320,"data_stored":1998524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.33700000000000002,"5min":0.33000000000000002,"15min":0.32700000000000001},"max":{"1min":1.093,"5min":1.155,"15min":1.155},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58699999999999997,"15min":0.59299999999999997},"min":{"1min":0.41899999999999998,"5min":0.39000000000000001,"15min":0.38900000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.90100000000000002},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.51100000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.72699999999999998,"5min":0.76400000000000001,"15min":1.153},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58999999999999997},"min":{"1min":0.42199999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1,"5min":1.018,"15min":1.018},"last":0.53700000000000003}]},{"osd":3,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.435,"15min":0.41799999999999998},"max":{"1min":0.95499999999999996,"5min":1.113,"15min":1.131},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92200000000000004,"5min":0.95499999999999996,"15min":0.96999999999999997},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.79600000000000004,"15min":0.80600000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.02,"5min":1.2210000000000001,"15min":1.2250000000000001},"last":0.995},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.82199999999999995,"15min":0.84299999999999997},"min":{"1min":0.58699999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.147,"5min":1.1930000000000001,"15min":1.236},"last":0.84899999999999998}]},{"osd":5,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.82699999999999996,"15min":0.83999999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.53600000000000003},"max":{"1min":1.0660000000000001,"5min":1.1240000000000001,"15min":1.228},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.83199999999999996,"15min":0.84499999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.51000000000000001},"max":{"1min":1.139,"5min":1.1799999999999999,"15min":1.2829999999999999},"last":1.0409999999999999}]},{"osd":6,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85199999999999998,"15min":0.85699999999999998},"min":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.59199999999999997},"max":{"1min":1.1779999999999999,"5min":1.2490000000000001,"15min":1.2709999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.85999999999999999,"15min":0.86399999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.163,"5min":1.242,"15min":1.2470000000000001},"last":0.92900000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.873,"15min":0.88400000000000001},"min":{"1min":0.63,"5min":0.623,"15min":0.623},"max":{"1min":1.198,"5min":1.222,"15min":1.2949999999999999},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.872,"15min":0.88},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.224,"5min":1.224,"15min":1.3080000000000001},"last":0.96499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575976,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303980,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94067860,"statfs":{"total":96636764160,"available":96325488640,"internally_reserved":0,"allocated":2658304,"data_stored":1407178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 05:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.46700000000000003,"15min":0.496},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.84299999999999997,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53500000000000003,"15min":0.55000000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.75600000000000001,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Oct 6 05:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54800000000000004,"15min":0.56999999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.90200000000000002,"15min":0.91000000000000003},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.56299999999999994,"15min":0.57199999999999995},"min":{"1min":0.41299999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.70999999999999996,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70999999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53600000000000003,"15min":0.54500000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.441},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.61699999999999999,"15min":0.64000000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.74399999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.74399999999999999}]},{"osd":4,"last update":"Fri Oct 6 05:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.84799999999999998,"15min":0.86299999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.54300000000000004},"max":{"1min":1.1000000000000001,"5min":1.2669999999999999,"15min":1.3049999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85199999999999998,"15min":0.86399999999999999},"min":{"1min":0.60099999999999998,"5min":0.55700000000000005,"15min":0.55300000000000005},"max":{"1min":1.1020000000000001,"5min":1.127,"15min":1.3440000000000001},"last":0.89200000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.82099999999999995,"15min":0.83899999999999997},"min":{"1min":0.59599999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.024,"5min":1.1919999999999999,"15min":1.2729999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82899999999999996,"15min":0.84799999999999998},"min":{"1min":0.59499999999999997,"5min":0.57399999999999995,"15min":0.45600000000000002},"max":{"1min":1.179,"5min":1.204,"15min":1.218},"last":0.872}]},{"osd":6,"last update":"Fri Oct 6 05:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.82099999999999995,"15min":0.83699999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.0249999999999999,"5min":1.1240000000000001,"15min":1.2869999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.84599999999999997,"15min":0.876},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.077,"5min":1.246,"15min":1.4399999999999999},"last":0.96499999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:08:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.83799999999999997,"15min":0.86899999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.141,"5min":1.2490000000000001,"15min":1.367},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83799999999999997,"15min":0.86799999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.294},"last":0.80700000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299884,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071956,"statfs":{"total":96636764160,"available":96329682944,"internally_reserved":0,"allocated":2658304,"data_stored":1407178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.32900000000000001,"5min":0.32000000000000001,"15min":0.312},"max":{"1min":0.54100000000000004,"5min":0.746,"15min":0.80300000000000005},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.53200000000000003,"15min":0.55200000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.31},"max":{"1min":0.70399999999999996,"5min":0.83099999999999996,"15min":0.91200000000000003},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Oct 6 05:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.496,"15min":0.498},"min":{"1min":0.34599999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72999999999999998,"5min":0.78300000000000003,"15min":0.92400000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.5,"5min":0.56100000000000005,"15min":0.57499999999999996},"min":{"1min":0.38600000000000001,"5min":0.38200000000000001,"15min":0.35999999999999999},"max":{"1min":0.63900000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.438}]},{"osd":3,"last update":"Fri Oct 6 05:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.58699999999999997,"15min":0.59999999999999998},"min":{"1min":0.38800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.66300000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.503},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56599999999999995,"15min":0.58699999999999997},"min":{"1min":0.377,"5min":0.36699999999999999,"15min":0.32400000000000001},"max":{"1min":0.70499999999999996,"5min":0.76800000000000002,"15min":0.90600000000000003},"last":0.63600000000000001}]},{"osd":4,"last update":"Fri Oct 6 05:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.82399999999999995,"15min":0.84599999999999997},"min":{"1min":0.56299999999999994,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98299999999999998,"5min":1.286,"15min":1.286},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.84899999999999998,"15min":0.86899999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.59199999999999997},"max":{"1min":0.95299999999999996,"5min":1.2110000000000001,"15min":1.254},"last":0.67000000000000004}]},{"osd":5,"last update":"Fri Oct 6 05:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.85799999999999998,"15min":0.879},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.028,"5min":1.26,"15min":1.345},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.86299999999999999,"15min":0.88800000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.006,"5min":1.2090000000000001,"15min":1.339},"last":0.89700000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.82799999999999996,"15min":0.85899999999999999},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.93400000000000005,"5min":1.1140000000000001,"15min":1.2989999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.82999999999999996,"15min":0.85299999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.044,"5min":1.226,"15min":1.226},"last":0.79800000000000004}]},{"osd":7,"last update":"Fri Oct 6 05:08:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.82599999999999996,"15min":0.85699999999999998},"min":{"1min":0.59099999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.95699999999999996,"5min":1.123,"15min":1.276},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.876,"15min":0.89900000000000002},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.98799999999999999,"5min":1.1970000000000001,"15min":1.3680000000000001},"last":0.93799999999999994}]}]},{"osd":3,"up_from":13,"seq":55834575975,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299888,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071952,"statfs":{"total":96636764160,"available":96329678848,"internally_reserved":0,"allocated":2662400,"data_stored":1408156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.505,"15min":0.49399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59999999999999998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.57999999999999996}]},{"osd":1,"last update":"Fri Oct 6 05:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.50800000000000001,"15min":0.51300000000000001},"min":{"1min":0.31900000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.64900000000000002,"5min":0.80500000000000005,"15min":0.96099999999999997},"last":0.628},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri Oct 6 05:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.58199999999999996,"15min":0.59499999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70699999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.38500000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Oct 6 05:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.81200000000000006,"15min":0.82099999999999995},"min":{"1min":0.48699999999999999,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.01,"5min":1.232,"15min":1.2849999999999999},"last":1.0860000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.83799999999999997,"15min":0.86099999999999999},"min":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.52500000000000002},"max":{"1min":1.0509999999999999,"5min":1.232,"15min":1.4019999999999999},"last":1.014}]},{"osd":5,"last update":"Fri Oct 6 05:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.79200000000000004,"15min":0.81299999999999994},"min":{"1min":0.52900000000000003,"5min":0.52400000000000002,"15min":0.52200000000000002},"max":{"1min":0.92000000000000004,"5min":1.22,"15min":1.5760000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.80500000000000005,"15min":0.82899999999999996},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.121,"5min":1.1419999999999999,"15min":1.3300000000000001},"last":0.96299999999999997}]},{"osd":6,"last update":"Fri Oct 6 05:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.82499999999999996,"15min":0.84499999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.087,"5min":1.212,"15min":1.2150000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.86399999999999999,"15min":0.88400000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.111,"5min":1.2929999999999999,"15min":1.321},"last":1.0660000000000001}]},{"osd":7,"last update":"Fri Oct 6 05:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.84299999999999997,"15min":0.86399999999999999},"min":{"1min":0.622,"5min":0.59399999999999997,"15min":0.57599999999999996},"max":{"1min":1.145,"5min":1.198,"15min":1.2769999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.83999999999999997,"15min":0.84899999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.155,"5min":1.1890000000000001,"15min":1.389},"last":1.044}]}]},{"osd":4,"up_from":13,"seq":55834575975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312176,"kb_used_data":2600,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059664,"statfs":{"total":96636764160,"available":96317095936,"internally_reserved":0,"allocated":2662400,"data_stored":1408156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.747},"min":{"1min":0.53800000000000003,"5min":0.42399999999999999,"15min":0.39400000000000002},"max":{"1min":0.89500000000000002,"5min":1.0820000000000001,"15min":1.2},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.753,"15min":0.755},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.97599999999999998,"5min":1.048,"15min":1.048},"last":0.80100000000000005}]},{"osd":1,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72299999999999998,"15min":0.73699999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.96199999999999997,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94299999999999995,"5min":0.99099999999999999,"15min":1.258},"last":0.78900000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.753,"15min":0.76900000000000002},"min":{"1min":0.50800000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75900000000000001,"15min":0.78300000000000003},"min":{"1min":0.50600000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.75700000000000001},"min":{"1min":0.53700000000000003,"5min":0.52200000000000002,"15min":0.49299999999999999},"max":{"1min":0.97499999999999998,"5min":0.97999999999999998,"15min":1.2330000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.54900000000000004,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.95599999999999996,"5min":0.96799999999999997,"15min":1.034},"last":0.89500000000000002}]},{"osd":5,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.61399999999999999,"15min":0.63400000000000001},"min":{"1min":0.46000000000000002,"5min":0.43099999999999999,"15min":0.41199999999999998},"max":{"1min":0.81599999999999995,"5min":1.04,"15min":1.323},"last":0.497},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64400000000000002,"15min":0.65200000000000002},"min":{"1min":0.46000000000000002,"5min":0.44700000000000001,"15min":0.39700000000000002},"max":{"1min":0.85599999999999998,"5min":1.01,"15min":1.1910000000000001},"last":0.68100000000000005}]},{"osd":6,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.67700000000000005},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":1.024,"15min":1.079},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68799999999999994,"15min":0.71199999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.41999999999999998},"max":{"1min":0.84699999999999998,"5min":0.95099999999999996,"15min":1.3540000000000001},"last":0.55800000000000005}]},{"osd":7,"last update":"Fri Oct 6 05:07:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67900000000000005,"15min":0.70599999999999996},"min":{"1min":0.49199999999999999,"5min":0.46800000000000003,"15min":0.45200000000000001},"max":{"1min":0.82699999999999996,"5min":1.0640000000000001,"15min":1.155},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.747},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.47299999999999998},"max":{"1min":0.93000000000000005,"5min":1.079,"15min":1.149},"last":0.75700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575975,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308076,"kb_used_data":2596,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063764,"statfs":{"total":96636764160,"available":96321294336,"internally_reserved":0,"allocated":2658304,"data_stored":1407178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 05:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.73299999999999998,"15min":0.74299999999999999},"min":{"1min":0.52000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.94299999999999995,"5min":1.079,"15min":1.079},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.70699999999999996,"15min":0.72599999999999998},"min":{"1min":0.44600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.91700000000000004,"5min":0.999,"15min":1.02},"last":0.68899999999999995}]},{"osd":1,"last update":"Fri Oct 6 05:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.72199999999999998,"15min":0.74199999999999999},"min":{"1min":0.443,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.92100000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.72099999999999997,"15min":0.75600000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91200000000000003,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.53300000000000003}]},{"osd":2,"last update":"Fri Oct 6 05:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.52800000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97499999999999998,"5min":1.0700000000000001,"15min":1.0780000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.77700000000000002,"15min":0.79900000000000004},"min":{"1min":0.52700000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.98799999999999999,"5min":1.048,"15min":1.048},"last":0.70699999999999996}]},{"osd":3,"last update":"Fri Oct 6 05:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.73199999999999998,"15min":0.74299999999999999},"min":{"1min":0.504,"5min":0.433,"15min":0.433},"max":{"1min":1.0029999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.77600000000000002,"15min":0.78100000000000003},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97099999999999997,"5min":1.095,"15min":1.095},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Oct 6 05:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.622,"15min":0.63100000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.38200000000000001},"max":{"1min":0.85099999999999998,"5min":0.93799999999999994,"15min":1.0049999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66100000000000003,"15min":0.64600000000000002},"min":{"1min":0.48599999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.84299999999999997,"5min":1.01,"15min":1.042},"last":0.64300000000000002}]},{"osd":6,"last update":"Fri Oct 6 05:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.45100000000000001},"max":{"1min":1.006,"5min":1.006,"15min":1.0369999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69799999999999995,"15min":0.70699999999999996},"min":{"1min":0.48599999999999999,"5min":0.47099999999999997,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.58999999999999997}]},{"osd":7,"last update":"Fri Oct 6 05:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67200000000000004,"15min":0.69099999999999995},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.44500000000000001},"max":{"1min":0.95999999999999996,"5min":0.99099999999999999,"15min":1.0669999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.71999999999999997,"15min":0.73799999999999999},"min":{"1min":0.53700000000000003,"5min":0.51300000000000001,"15min":0.47199999999999998},"max":{"1min":0.875,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.79900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T05:08:24.707 DEBUG:teuthology.orchestra.run.smithi055:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T05:08:24.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.885+0000 7f97dd99d700 1 Processor -- start 2023-10-06T05:08:24.887 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.885+0000 7f97dd99d700 1 -- start start 2023-10-06T05:08:24.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.887+0000 7f97dd99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97d81407e0 0x7f97d806a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:24.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.887+0000 7f97dd99d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f97d806dc00 0x7f97d806a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:24.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.887+0000 7f97dd99d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d80c6820 con 0x7f97d8140bb0 2023-10-06T05:08:24.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.887+0000 7f97dd99d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d80c6de0 con 0x7f97d806dc00 2023-10-06T05:08:24.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.887+0000 7f97dd99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d8141270 con 0x7f97d81407e0 2023-10-06T05:08:24.889 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.887+0000 7f97dd19c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f97d806dc00 0x7f97d806a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:24.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.887+0000 7f97dd19c700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7f97d806dc00 0x7f97d806a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40844/0 (socket says 172.21.15.55:40844) 2023-10-06T05:08:24.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.887+0000 7f97dd19c700 1 -- 172.21.15.55:0/730678462 learned_addr learned my addr 172.21.15.55:0/730678462 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:24.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.887+0000 7f97dc99b700 1 --2- 172.21.15.55:0/730678462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97d81407e0 0x7f97d806a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:24.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.887+0000 7f97dd19c700 1 -- 172.21.15.55:0/730678462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97d81407e0 msgr2=0x7f97d806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:24.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.887+0000 7f97dd19c700 1 --2- 172.21.15.55:0/730678462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97d81407e0 0x7f97d806a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.887+0000 7f97dd19c700 1 -- 172.21.15.55:0/730678462 >> v1:172.21.15.163:6789/0 conn(0x7f97d8140bb0 legacy=0x7f97d806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:24.890 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.888+0000 7f97dd19c700 1 -- 172.21.15.55:0/730678462 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d806acf0 con 0x7f97d806dc00 2023-10-06T05:08:24.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.888+0000 7f97dd19c700 1 --2- 172.21.15.55:0/730678462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97d806dc00 0x7f97d806a6c0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009f20 tx=0x7f97d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c8b7730be7eaa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:24.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.888+0000 7f97d77fe700 1 -- 172.21.15.55:0/730678462 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d000e070 con 0x7f97d806dc00 2023-10-06T05:08:24.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.888+0000 7f97d77fe700 1 -- 172.21.15.55:0/730678462 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d000ba80 con 0x7f97d806dc00 2023-10-06T05:08:24.891 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.888+0000 7f97d77fe700 1 -- 172.21.15.55:0/730678462 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d00092e0 con 0x7f97d806dc00 2023-10-06T05:08:24.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.888+0000 7f97dd99d700 1 -- 172.21.15.55:0/730678462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97d806dc00 msgr2=0x7f97d806a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:24.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.888+0000 7f97dd99d700 1 --2- 172.21.15.55:0/730678462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97d806dc00 0x7f97d806a6c0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009f20 tx=0x7f97d0009840 comp rx=0 tx=0).stop 2023-10-06T05:08:24.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.889+0000 7f97dd99d700 1 -- 172.21.15.55:0/730678462 shutdown_connections 2023-10-06T05:08:24.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.889+0000 7f97dd99d700 1 --2- 172.21.15.55:0/730678462 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97d806dc00 0x7f97d806a6c0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.889+0000 7f97dd99d700 1 --2- 172.21.15.55:0/730678462 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97d81407e0 0x7f97d806a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.889+0000 7f97dd99d700 1 -- 172.21.15.55:0/730678462 >> 172.21.15.55:0/730678462 conn(0x7f97d80cc1e0 msgr2=0x7f97d80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:24.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.889+0000 7f97dd99d700 1 -- 172.21.15.55:0/730678462 shutdown_connections 2023-10-06T05:08:24.892 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.889+0000 7f97dd99d700 1 -- 172.21.15.55:0/730678462 wait complete. 2023-10-06T05:08:24.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.890+0000 7f97dd99d700 1 Processor -- start 2023-10-06T05:08:24.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.890+0000 7f97dd99d700 1 -- start start 2023-10-06T05:08:24.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.891+0000 7f97dd99d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97d806dc00 0x7f97d813f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:24.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.891+0000 7f97dd99d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97d81407e0 0x7f97d8156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:24.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.891+0000 7f97dd99d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97d8140bb0 0x7f97d80c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:24.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.891+0000 7f97dd99d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d813fe70 con 0x7f97d806dc00 2023-10-06T05:08:24.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.891+0000 7f97dd99d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d813ebd0 con 0x7f97d8140bb0 2023-10-06T05:08:24.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.891+0000 7f97dd99d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d813dfc0 con 0x7f97d81407e0 2023-10-06T05:08:24.894 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.891+0000 7f97dc99b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97d806dc00 0x7f97d813f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:24.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.891+0000 7f97dc99b700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97d806dc00 0x7f97d813f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40852/0 (socket says 172.21.15.55:40852) 2023-10-06T05:08:24.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.891+0000 7f97dc99b700 1 -- 172.21.15.55:0/4176589461 learned_addr learned my addr 172.21.15.55:0/4176589461 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:24.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.891+0000 7f97d7fff700 1 --2- 172.21.15.55:0/4176589461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97d81407e0 0x7f97d8156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:24.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.892+0000 7f97dd19c700 1 --2- 172.21.15.55:0/4176589461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97d8140bb0 0x7f97d80c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:24.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.892+0000 7f97dc99b700 1 -- 172.21.15.55:0/4176589461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97d81407e0 msgr2=0x7f97d8156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:24.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.892+0000 7f97dc99b700 1 --2- 172.21.15.55:0/4176589461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97d81407e0 0x7f97d8156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.892+0000 7f97dc99b700 1 -- 172.21.15.55:0/4176589461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97d8140bb0 msgr2=0x7f97d80c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:24.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.892+0000 7f97dc99b700 1 --2- 172.21.15.55:0/4176589461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97d8140bb0 0x7f97d80c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:24.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.892+0000 7f97dc99b700 1 -- 172.21.15.55:0/4176589461 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d0009ae0 con 0x7f97d806dc00 2023-10-06T05:08:24.895 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.892+0000 7f97dc99b700 1 --2- 172.21.15.55:0/4176589461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97d806dc00 0x7f97d813f930 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00d890 tx=0x7f97cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:24.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.892+0000 7f97d4ff9700 1 -- 172.21.15.55:0/4176589461 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0125f0 con 0x7f97d806dc00 2023-10-06T05:08:24.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.892+0000 7f97d4ff9700 1 -- 172.21.15.55:0/4176589461 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc012750 con 0x7f97d806dc00 2023-10-06T05:08:24.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.892+0000 7f97d4ff9700 1 -- 172.21.15.55:0/4176589461 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc012af0 con 0x7f97d806dc00 2023-10-06T05:08:24.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.892+0000 7f97dd99d700 1 -- 172.21.15.55:0/4176589461 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d813e2a0 con 0x7f97d806dc00 2023-10-06T05:08:24.896 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.893+0000 7f97dd99d700 1 -- 172.21.15.55:0/4176589461 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d813e860 con 0x7f97d806dc00 2023-10-06T05:08:24.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.893+0000 7f97dd99d700 1 -- 172.21.15.55:0/4176589461 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d813eeb0 con 0x7f97d806dc00 2023-10-06T05:08:24.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.894+0000 7f97d4ff9700 1 -- 172.21.15.55:0/4176589461 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97cc01e460 con 0x7f97d806dc00 2023-10-06T05:08:24.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.895+0000 7f97d4ff9700 1 --2- 172.21.15.55:0/4176589461 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f97bc066f30 0x7f97bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:24.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.895+0000 7f97d4ff9700 1 -- 172.21.15.55:0/4176589461 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7f97cc084d90 con 0x7f97d806dc00 2023-10-06T05:08:24.897 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.896+0000 7f97d7fff700 1 --2- 172.21.15.55:0/4176589461 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f97bc066f30 0x7f97bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:24.898 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.896+0000 7f97d7fff700 1 --2- 172.21.15.55:0/4176589461 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f97bc066f30 0x7f97bc0693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f97d80c2e10 tx=0x7f97c4007ec0 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:24.901 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:24.899+0000 7f97d4ff9700 1 -- 172.21.15.55:0/4176589461 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97cc051d10 con 0x7f97d806dc00 2023-10-06T05:08:25.107 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.105+0000 7f97dd99d700 1 -- 172.21.15.55:0/4176589461 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f97d806afd0 con 0x7f97d806dc00 2023-10-06T05:08:25.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.110+0000 7f97d4ff9700 1 -- 172.21.15.55:0/4176589461 <== mon.0 v2:172.21.15.55:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f97cc051d10 con 0x7f97d806dc00 2023-10-06T05:08:25.111 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.110+0000 7f97d4ff9700 1 -- 172.21.15.55:0/4176589461 <== mon.0 v2:172.21.15.55:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v76)=0 v76) v1 ==== 123+0+0 (secure 0 0 0) 0x7f97cc055330 con 0x7f97d806dc00 2023-10-06T05:08:25.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.112+0000 7f97dd99d700 1 -- 172.21.15.55:0/4176589461 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f97bc066f30 msgr2=0x7f97bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:25.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.112+0000 7f97dd99d700 1 --2- 172.21.15.55:0/4176589461 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f97bc066f30 0x7f97bc0693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f97d80c2e10 tx=0x7f97c4007ec0 comp rx=0 tx=0).stop 2023-10-06T05:08:25.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.112+0000 7f97dd99d700 1 -- 172.21.15.55:0/4176589461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97d806dc00 msgr2=0x7f97d813f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:25.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.112+0000 7f97dd99d700 1 --2- 172.21.15.55:0/4176589461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97d806dc00 0x7f97d813f930 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00d890 tx=0x7f97cc008040 comp rx=0 tx=0).stop 2023-10-06T05:08:25.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.112+0000 7f97dd99d700 1 -- 172.21.15.55:0/4176589461 shutdown_connections 2023-10-06T05:08:25.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.112+0000 7f97dd99d700 1 --2- 172.21.15.55:0/4176589461 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7f97bc066f30 0x7f97bc0693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:25.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.112+0000 7f97dd99d700 1 --2- 172.21.15.55:0/4176589461 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7f97d8140bb0 0x7f97d80c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:25.115 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.112+0000 7f97dd99d700 1 --2- 172.21.15.55:0/4176589461 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7f97d806dc00 0x7f97d813f930 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:25.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.112+0000 7f97dd99d700 1 --2- 172.21.15.55:0/4176589461 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7f97d81407e0 0x7f97d8156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:25.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.112+0000 7f97dd99d700 1 -- 172.21.15.55:0/4176589461 >> 172.21.15.55:0/4176589461 conn(0x7f97d80cc1e0 msgr2=0x7f97d8069770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:25.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.113+0000 7f97dd99d700 1 -- 172.21.15.55:0/4176589461 shutdown_connections 2023-10-06T05:08:25.116 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.113+0000 7f97dd99d700 1 -- 172.21.15.55:0/4176589461 wait complete. 2023-10-06T05:08:25.132 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T05:08:25.132 DEBUG:teuthology.orchestra.run.smithi055:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T05:08:25.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.317+0000 7fc9e6d9d700 1 Processor -- start 2023-10-06T05:08:25.319 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.317+0000 7fc9e6d9d700 1 -- start start 2023-10-06T05:08:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.318+0000 7fc9e6d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e0142ea0 0x7fc9e00b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.318+0000 7fc9e6d9d700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc9e00ad870 0x7fc9e006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.318+0000 7fc9e6d9d700 1 -- --> v1:172.21.15.163:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e00b40f0 con 0x7fc9e00b3b70 2023-10-06T05:08:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.318+0000 7fc9e6d9d700 1 -- --> v2:172.21.15.55:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e0069640 con 0x7fc9e00ad870 2023-10-06T05:08:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.318+0000 7fc9e6d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e0069920 con 0x7fc9e0142ea0 2023-10-06T05:08:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.318+0000 7fc9e559a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc9e00ad870 0x7fc9e006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.318+0000 7fc9e559a700 1 --2- >> v2:172.21.15.55:3300/0 conn(0x7fc9e00ad870 0x7fc9e006b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40866/0 (socket says 172.21.15.55:40866) 2023-10-06T05:08:25.322 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.318+0000 7fc9e559a700 1 -- 172.21.15.55:0/2734420432 learned_addr learned my addr 172.21.15.55:0/2734420432 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.319+0000 7fc9e5d9b700 1 --2- 172.21.15.55:0/2734420432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e0142ea0 0x7fc9e00b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.319+0000 7fc9e559a700 1 -- 172.21.15.55:0/2734420432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e0142ea0 msgr2=0x7fc9e00b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T05:08:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.319+0000 7fc9e559a700 1 --2- 172.21.15.55:0/2734420432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e0142ea0 0x7fc9e00b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.319+0000 7fc9e559a700 1 -- 172.21.15.55:0/2734420432 >> v1:172.21.15.163:6789/0 conn(0x7fc9e00b3b70 legacy=0x7fc9e006cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.319+0000 7fc9e559a700 1 -- 172.21.15.55:0/2734420432 --> v2:172.21.15.55:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e0069c00 con 0x7fc9e00ad870 2023-10-06T05:08:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.319+0000 7fc9e5d9b700 1 --2- 172.21.15.55:0/2734420432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e0142ea0 0x7fc9e00b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T05:08:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.319+0000 7fc9e559a700 1 --2- 172.21.15.55:0/2734420432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e00ad870 0x7fc9e006b9e0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00099b0 tx=0x7fc9d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca978f561e5f9e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.319+0000 7fc9e4d99700 1 -- 172.21.15.55:0/2734420432 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d000b860 con 0x7fc9e00ad870 2023-10-06T05:08:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.319+0000 7fc9e4d99700 1 -- 172.21.15.55:0/2734420432 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc9d000b9c0 con 0x7fc9e00ad870 2023-10-06T05:08:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.319+0000 7fc9e4d99700 1 -- 172.21.15.55:0/2734420432 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d000bd80 con 0x7fc9e00ad870 2023-10-06T05:08:25.323 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.320+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/2734420432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e00ad870 msgr2=0x7fc9e006b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:25.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.320+0000 7fc9e6d9d700 1 --2- 172.21.15.55:0/2734420432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e00ad870 0x7fc9e006b9e0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00099b0 tx=0x7fc9d0009250 comp rx=0 tx=0).stop 2023-10-06T05:08:25.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.320+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/2734420432 shutdown_connections 2023-10-06T05:08:25.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.320+0000 7fc9e6d9d700 1 --2- 172.21.15.55:0/2734420432 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e0142ea0 0x7fc9e00b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:25.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.320+0000 7fc9e6d9d700 1 --2- 172.21.15.55:0/2734420432 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e00ad870 0x7fc9e006b9e0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:25.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.320+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/2734420432 >> 172.21.15.55:0/2734420432 conn(0x7fc9e00ce430 msgr2=0x7fc9e00ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:25.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.320+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/2734420432 shutdown_connections 2023-10-06T05:08:25.324 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.320+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/2734420432 wait complete. 2023-10-06T05:08:25.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.322+0000 7fc9e6d9d700 1 Processor -- start 2023-10-06T05:08:25.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.322+0000 7fc9e6d9d700 1 -- start start 2023-10-06T05:08:25.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e6d9d700 1 --2- >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e00b3b70 0x7fc9e00ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:25.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e6d9d700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e0142ea0 0x7fc9e0159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:25.327 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e6d9d700 1 --2- >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e015d040 0x7fc9e015e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e6d9d700 1 -- --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e015e1a0 con 0x7fc9e0142ea0 2023-10-06T05:08:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e6d9d700 1 -- --> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e00ca5d0 con 0x7fc9e00b3b70 2023-10-06T05:08:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e6d9d700 1 -- --> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e00c7870 con 0x7fc9e015d040 2023-10-06T05:08:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e559a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e0142ea0 0x7fc9e0159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e559a700 1 --2- >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e0142ea0 0x7fc9e0159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.55:3300/0 says I am v2:172.21.15.55:40882/0 (socket says 172.21.15.55:40882) 2023-10-06T05:08:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e559a700 1 -- 172.21.15.55:0/438005306 learned_addr learned my addr 172.21.15.55:0/438005306 (peer_addr_for_me v2:172.21.15.55:0/0) 2023-10-06T05:08:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e5d9b700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e00b3b70 0x7fc9e00ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e559a700 1 -- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e015d040 msgr2=0x7fc9e015e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e659c700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e015d040 0x7fc9e015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e559a700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e015d040 0x7fc9e015e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:25.328 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e559a700 1 -- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e00b3b70 msgr2=0x7fc9e00ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e559a700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e00b3b70 0x7fc9e00ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e559a700 1 -- 172.21.15.55:0/438005306 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d00094f0 con 0x7fc9e0142ea0 2023-10-06T05:08:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e5d9b700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e00b3b70 0x7fc9e00ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:08:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.323+0000 7fc9e659c700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e015d040 0x7fc9e015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T05:08:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.324+0000 7fc9e559a700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e0142ea0 0x7fc9e0159000 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fc9e006f0d0 tx=0x7fc9d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.324+0000 7fc9d5ffb700 1 -- 172.21.15.55:0/438005306 <== mon.0 v2:172.21.15.55:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0011e70 con 0x7fc9e0142ea0 2023-10-06T05:08:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.324+0000 7fc9d5ffb700 1 -- 172.21.15.55:0/438005306 <== mon.0 v2:172.21.15.55:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc9d001f070 con 0x7fc9e0142ea0 2023-10-06T05:08:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.324+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/438005306 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e015d590 con 0x7fc9e0142ea0 2023-10-06T05:08:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.324+0000 7fc9d5ffb700 1 -- 172.21.15.55:0/438005306 <== mon.0 v2:172.21.15.55:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d00266b0 con 0x7fc9e0142ea0 2023-10-06T05:08:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.324+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/438005306 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e015db50 con 0x7fc9e0142ea0 2023-10-06T05:08:25.329 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.325+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/438005306 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e015e350 con 0x7fc9e0142ea0 2023-10-06T05:08:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.326+0000 7fc9d5ffb700 1 -- 172.21.15.55:0/438005306 <== mon.0 v2:172.21.15.55:3300/0 4 ==== mgrmap(e 108) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc9d0026a60 con 0x7fc9e0142ea0 2023-10-06T05:08:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.327+0000 7fc9d5ffb700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc9c4066f30 0x7fc9c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T05:08:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.327+0000 7fc9d5ffb700 1 -- 172.21.15.55:0/438005306 <== mon.0 v2:172.21.15.55:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6299+0+0 (secure 0 0 0) 0x7fc9d0021070 con 0x7fc9e0142ea0 2023-10-06T05:08:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.328+0000 7fc9e5d9b700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc9c4066f30 0x7fc9c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T05:08:25.331 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.328+0000 7fc9e5d9b700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc9c4066f30 0x7fc9c40693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc006f00 tx=0x7fc9dc008040 comp rx=0 tx=0).ready entity=mgr.10033 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T05:08:25.333 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.331+0000 7fc9d5ffb700 1 -- 172.21.15.55:0/438005306 <== mon.0 v2:172.21.15.55:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9e015e350 con 0x7fc9e0142ea0 2023-10-06T05:08:25.580 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.578+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/438005306 --> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9e015e350 con 0x7fc9e0142ea0 2023-10-06T05:08:25.581 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.579+0000 7fc9d5ffb700 1 -- 172.21.15.55:0/438005306 <== mon.0 v2:172.21.15.55:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+3541 (secure 0 0 0) 0x7fc9e015e350 con 0x7fc9e0142ea0 2023-10-06T05:08:25.581 INFO:teuthology.orchestra.run.smithi055.stdout: 2023-10-06T05:08:25.581 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":107,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8438,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6834/3704044820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6832","nonce":3704044820},{"type":"v1","addr":"172.21.15.163:6834","nonce":3704044820}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":9772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.163:6835/2254022733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.163:6833","nonce":2254022733},{"type":"v1","addr":"172.21.15.163:6835","nonce":2254022733}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100},{"gid":10224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6834/3844040929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6832","nonce":3844040929},{"type":"v1","addr":"172.21.15.55:6834","nonce":3844040929}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":99},{"gid":10584,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.55:6835/2507813904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6833","nonce":2507813904},{"type":"v1","addr":"172.21.15.55:6835","nonce":2507813904}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[]} 2023-10-06T05:08:25.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.581+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc9c4066f30 msgr2=0x7fc9c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:25.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.581+0000 7fc9e6d9d700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc9c4066f30 0x7fc9c40693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc006f00 tx=0x7fc9dc008040 comp rx=0 tx=0).stop 2023-10-06T05:08:25.583 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.581+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/438005306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e0142ea0 msgr2=0x7fc9e0159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T05:08:25.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.581+0000 7fc9e6d9d700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e0142ea0 0x7fc9e0159000 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fc9e006f0d0 tx=0x7fc9d0009250 comp rx=0 tx=0).stop 2023-10-06T05:08:25.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.581+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/438005306 shutdown_connections 2023-10-06T05:08:25.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.581+0000 7fc9e6d9d700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:6836/101089,v1:172.21.15.163:6837/101089] conn(0x7fc9c4066f30 0x7fc9c40693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:25.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.581+0000 7fc9e6d9d700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:3301/0,v1:172.21.15.163:6790/0] conn(0x7fc9e015d040 0x7fc9e015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:25.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.581+0000 7fc9e6d9d700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.163:3300/0,v1:172.21.15.163:6789/0] conn(0x7fc9e00b3b70 0x7fc9e00ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:25.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.581+0000 7fc9e6d9d700 1 --2- 172.21.15.55:0/438005306 >> [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] conn(0x7fc9e0142ea0 0x7fc9e0159000 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T05:08:25.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.581+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/438005306 >> 172.21.15.55:0/438005306 conn(0x7fc9e00ce430 msgr2=0x7fc9e0141ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T05:08:25.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.581+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/438005306 shutdown_connections 2023-10-06T05:08:25.584 INFO:teuthology.orchestra.run.smithi055.stderr:2023-10-06T05:08:25.581+0000 7fc9e6d9d700 1 -- 172.21.15.55:0/438005306 wait complete. 2023-10-06T05:08:25.585 INFO:teuthology.orchestra.run.smithi055.stderr:dumped fsmap epoch 107 2023-10-06T05:08:25.601 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-06T05:08:25.601 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T05:08:25.602 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T05:08:25.602 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:25.614 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T05:08:25.615 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T05:08:25.615 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:25.668 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T05:08:25.669 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T05:08:25.669 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:25.684 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T05:08:25.684 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T05:08:25.684 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:25.740 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T05:08:25.741 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T05:08:25.741 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T05:08:25.741 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:25.787 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T05:08:25.787 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T05:08:25.788 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:25.837 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T05:08:25.838 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T05:08:25.838 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:25.884 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T05:08:25.885 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T05:08:25.885 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:25.933 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T05:08:25.934 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T05:08:25.934 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:25.984 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T05:08:25.984 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T05:08:25.984 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:26.036 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T05:08:26.037 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T05:08:26.037 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:26.087 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T05:08:26.087 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T05:08:26.087 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:26.150 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T05:08:26.150 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T05:08:26.150 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T05:08:26.150 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:26.213 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T05:08:26.213 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T05:08:26.213 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:26.279 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T05:08:26.280 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T05:08:26.280 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T05:08:26.280 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:26.311 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T05:08:26.312 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T05:08:26.312 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:26.336 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T05:08:26.337 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T05:08:26.337 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T05:08:26.400 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T05:08:26.401 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T05:08:26.401 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-06T05:08:26.472 INFO:teuthology.orchestra.run.smithi055.stdout:1696563189.4484687 mon.a (mon.0) 107 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T05:08:26.474 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T05:08:26.474 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-06T05:08:26.546 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-06T05:08:26.618 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-06T05:08:26.689 INFO:teuthology.orchestra.run.smithi055.stdout:1696563189.4484687 mon.a (mon.0) 107 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T05:08:26.691 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T05:08:26.691 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T05:08:27.013 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T05:08:27.013 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T05:08:27.207 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T05:08:27.208 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T05:08:27.418 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T05:08:27.419 DEBUG:teuthology.orchestra.run.smithi055:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T05:08:27.595 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-06T05:08:27.596 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T05:08:27.981 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-06T05:08:27.981 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T05:08:28.169 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-06T05:08:28.170 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T05:08:28.391 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi163.front.sepia.ceph.com 2023-10-06T05:08:28.392 DEBUG:teuthology.orchestra.run.smithi163:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T05:08:28.577 INFO:tasks.ceph:Archiving mon data... 2023-10-06T05:08:28.578 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199/data/mon.a.tgz 2023-10-06T05:08:28.578 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2023-10-06T05:08:28.620 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.1ZVWna40RF 2023-10-06T05:08:28.621 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.1ZVWna40RF 2023-10-06T05:08:29.138 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.1ZVWna40RF 2023-10-06T05:08:29.240 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.1ZVWna40RF is 2MB 2023-10-06T05:08:29.362 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.1ZVWna40RF 2023-10-06T05:08:29.399 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199/data/mon.b.tgz 2023-10-06T05:08:29.399 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2023-10-06T05:08:29.438 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.1ROF5DzLpj 2023-10-06T05:08:29.439 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.1ROF5DzLpj 2023-10-06T05:08:29.990 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.1ROF5DzLpj 2023-10-06T05:08:30.089 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.1ROF5DzLpj is 3MB 2023-10-06T05:08:30.185 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.1ROF5DzLpj 2023-10-06T05:08:30.222 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199/data/mon.c.tgz 2023-10-06T05:08:30.222 DEBUG:teuthology.orchestra.run.smithi163:> mktemp 2023-10-06T05:08:30.297 INFO:teuthology.orchestra.run.smithi163.stdout:/tmp/tmp.OVOYez5JHh 2023-10-06T05:08:30.298 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.OVOYez5JHh 2023-10-06T05:08:30.829 DEBUG:teuthology.orchestra.run.smithi163:> sudo chmod 0666 /tmp/tmp.OVOYez5JHh 2023-10-06T05:08:30.928 DEBUG:teuthology.orchestra.remote:smithi163:/tmp/tmp.OVOYez5JHh is 3MB 2023-10-06T05:08:31.065 DEBUG:teuthology.orchestra.run.smithi163:> rm -fr /tmp/tmp.OVOYez5JHh 2023-10-06T05:08:31.102 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T05:08:31.103 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T05:08:31.107 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T05:08:31.203 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T05:08:31.203 DEBUG:teuthology.orchestra.run.smithi055:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T05:08:31.208 DEBUG:teuthology.orchestra.run.smithi163:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T05:08:31.280 INFO:teuthology.orchestra.run.smithi163.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T05:08:31.288 INFO:teuthology.orchestra.run.smithi055.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T05:08:31.304 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T05:08:31.306 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199/remote/smithi055/crash 2023-10-06T05:08:31.307 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T05:08:31.369 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199/remote/smithi163/crash 2023-10-06T05:08:31.370 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T05:08:31.430 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T05:08:31.430 DEBUG:teuthology.orchestra.run.smithi055:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:08:31.434 DEBUG:teuthology.orchestra.run.smithi163:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T05:08:31.543 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T05:08:31.605 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T05:08:31.666 INFO:tasks.ceph:Compressing logs... 2023-10-06T05:08:31.666 DEBUG:teuthology.orchestra.run.smithi055:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:08:31.671 DEBUG:teuthology.orchestra.run.smithi163:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:08:49.995 INFO:tasks.ceph:Archiving logs... 2023-10-06T05:08:49.995 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199/remote/smithi055/log 2023-10-06T05:08:49.996 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T05:08:52.524 DEBUG:teuthology.misc:Transferring archived files from smithi163:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199/remote/smithi163/log 2023-10-06T05:08:52.525 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T05:08:54.642 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T05:08:54.651 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T05:08:54.652 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T05:08:54.658 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T05:08:54.782 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T05:08:54.794 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T05:08:54.794 DEBUG:teuthology.orchestra.run.smithi055:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:08:54.800 DEBUG:teuthology.orchestra.run.smithi163:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T05:08:54.832 INFO:teuthology.orchestra.run.smithi055.stderr:bash: ntpq: command not found 2023-10-06T05:08:54.835 INFO:teuthology.orchestra.run.smithi163.stderr:bash: ntpq: command not found 2023-10-06T05:08:54.838 INFO:teuthology.orchestra.run.smithi055.stdout:210 Number of sources = 4 2023-10-06T05:08:54.838 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:08:54.838 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2023-10-06T05:08:54.838 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 58 +276us[ +276us] +/- 86ms 2023-10-06T05:08:54.838 INFO:teuthology.orchestra.run.smithi055.stdout:^* hv02.front.sepia.ceph.com 3 8 377 184 -297us[ -336us] +/- 57ms 2023-10-06T05:08:54.838 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 123 +223us[ +223us] +/- 64ms 2023-10-06T05:08:54.838 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:08:54.840 INFO:teuthology.orchestra.run.smithi163.stdout:210 Number of sources = 4 2023-10-06T05:08:54.841 INFO:teuthology.orchestra.run.smithi163.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T05:08:54.841 INFO:teuthology.orchestra.run.smithi163.stdout:=============================================================================== 2023-10-06T05:08:54.841 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 116 +334us[ +319us] +/- 86ms 2023-10-06T05:08:54.841 INFO:teuthology.orchestra.run.smithi163.stdout:^* hv02.front.sepia.ceph.com 3 8 377 67 -311us[ -327us] +/- 59ms 2023-10-06T05:08:54.841 INFO:teuthology.orchestra.run.smithi163.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 250 +191us[ +181us] +/- 62ms 2023-10-06T05:08:54.841 INFO:teuthology.orchestra.run.smithi163.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T05:08:54.842 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T05:08:54.851 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T05:08:54.852 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T05:08:54.861 DEBUG:teuthology.orchestra.run.smithi055:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T05:08:54.987 DEBUG:teuthology.orchestra.run.smithi163:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T05:08:55.149 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T05:08:55.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:08:55.215 DEBUG:teuthology.orchestra.run.smithi163:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T05:08:55.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:08:55.284 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T05:08:55.294 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T05:08:55.304 INFO:teuthology.task.internal:Duration was 6630.058425 seconds 2023-10-06T05:08:55.305 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T05:08:55.314 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T05:08:55.314 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T05:08:55.319 DEBUG:teuthology.orchestra.run.smithi163:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T05:08:55.426 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T05:08:55.429 INFO:teuthology.orchestra.run.smithi163.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T05:08:55.748 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T05:08:55.748 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi055.front.sepia.ceph.com 2023-10-06T05:08:55.749 DEBUG:teuthology.orchestra.run.smithi055:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T05:08:55.800 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi163.front.sepia.ceph.com 2023-10-06T05:08:55.801 DEBUG:teuthology.orchestra.run.smithi163:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T05:08:55.852 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T05:08:55.852 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:08:55.857 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T05:08:55.941 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T05:08:55.941 DEBUG:teuthology.orchestra.run.smithi055:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T05:08:55.946 DEBUG:teuthology.orchestra.run.smithi163:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T05:08:56.089 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T05:08:56.099 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T05:08:56.099 DEBUG:teuthology.orchestra.run.smithi055:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T05:08:56.163 DEBUG:teuthology.orchestra.run.smithi163:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T05:08:56.225 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T05:08:56.235 DEBUG:teuthology.orchestra.run.smithi055:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:08:56.240 DEBUG:teuthology.orchestra.run.smithi163:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:08:56.298 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = core 2023-10-06T05:08:56.301 INFO:teuthology.orchestra.run.smithi163.stdout:kernel.core_pattern = core 2023-10-06T05:08:56.336 DEBUG:teuthology.orchestra.run.smithi055:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:08:56.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:08:56.371 DEBUG:teuthology.orchestra.run.smithi163:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T05:08:56.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T05:08:56.406 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T05:08:56.417 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T05:08:56.418 DEBUG:teuthology.misc:Transferring archived files from smithi055:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199/remote/smithi055 2023-10-06T05:08:56.418 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T05:08:56.490 DEBUG:teuthology.misc:Transferring archived files from smithi163:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199/remote/smithi163 2023-10-06T05:08:56.490 DEBUG:teuthology.orchestra.run.smithi163:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T05:08:56.564 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T05:08:56.564 DEBUG:teuthology.orchestra.run.smithi055:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T05:08:56.569 DEBUG:teuthology.orchestra.run.smithi163:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T05:08:56.606 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T05:08:56.618 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T05:08:56.619 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T05:08:56.629 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T05:08:56.630 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T05:08:56.647 DEBUG:teuthology.orchestra.run.smithi163:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T05:08:56.679 INFO:teuthology.orchestra.run.smithi055.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 05:08 /home/ubuntu/cephtest 2023-10-06T05:08:56.683 INFO:teuthology.orchestra.run.smithi163.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 05:08 /home/ubuntu/cephtest 2023-10-06T05:08:56.687 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T05:08:56.697 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T05:08:56.720 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T05:08:56.746 DEBUG:teuthology.nuke:shortname: smithi055 2023-10-06T05:08:56.746 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T05:08:56.767 DEBUG:teuthology.nuke:shortname: smithi163 2023-10-06T05:08:56.767 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T05:08:56.772 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 03:08:09.668313', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGqd+edrJM33HqoVOXuByo5UCZMCd9lwtqSdRynCQEwqmfYkHm6qMqUcBj6r1uCSm4Q/dDGu1v0wzAQCwCu2NcM='} 2023-10-06T05:08:56.786 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi163.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412199', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 03:08:09.666339', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbdyRKYMgXgX8qIaeWtbeZtloToSZebR7zRlsyDZRCmJ9QOXny++FCvXp1BImVGFJHlIh2orVhQWKxCENrLDWI='} 2023-10-06T05:08:56.821 INFO:teuthology.orchestra.console.smithi055:Power off 2023-10-06T05:08:56.821 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T05:08:56.834 INFO:teuthology.orchestra.console.smithi163:Power off 2023-10-06T05:08:56.835 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T05:08:56.844 DEBUG:teuthology.orchestra.console.smithi055:power off output: Chassis Power Control: Down/Off 2023-10-06T05:08:56.844 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:08:56.855 DEBUG:teuthology.orchestra.console.smithi163:power off output: Chassis Power Control: Down/Off 2023-10-06T05:08:56.856 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:08:56.867 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2023-10-06T05:08:56.878 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2023-10-06T05:09:00.869 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:09:00.879 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:09:00.980 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2023-10-06T05:09:01.084 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is on 2023-10-06T05:09:05.086 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:09:05.095 DEBUG:teuthology.orchestra.console.smithi163:pexpect command: ipmitool -H smithi163.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T05:09:05.196 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2023-10-06T05:09:05.203 DEBUG:teuthology.orchestra.console.smithi163:check power output: Chassis Power is off 2023-10-06T05:09:05.297 INFO:teuthology.orchestra.console.smithi055:Power off completed 2023-10-06T05:09:05.305 INFO:teuthology.orchestra.console.smithi163:Power off completed 2023-10-06T05:09:05.409 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c client.1-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6630.058424949646 failure_reason: '"1696563189.4484687 mon.a (mon.0) 107 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-06T05:09:05.410 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T05:09:05.485 INFO:teuthology.run:FAIL